2025-10-09T21:42:44.604287Z :BasicWriteSession INFO: Random seed for debugging is 1760046164604277
2025-10-09T21:42:44.709778Z node 1 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7559340714354308590:2088];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:44.709805Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-10-09T21:42:44.722602Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:123: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, got undelivered to scheme cache: ActorUnknown
2025-10-09T21:42:44.724945Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:44.742921Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/owmb/000355/r3tmp/tmptLVWyc/pdisk_1.dat
2025-10-09T21:42:44.805575Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:44.805853Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:44.805913Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
2025-10-09T21:42:44.830355Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:44.830379Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:44.843388Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:42:44.850715Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-10-09T21:42:44.851130Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:44.857613Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:44.861073Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:44.870607Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
TServer::EnableGrpc on GrpcPort 11444, node 1
2025-10-09T21:42:44.904264Z INFO: TTestServer started on Port 19671 GrpcPort 11444
2025-10-09T21:42:44.902249Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/owmb/000355/r3tmp/yandexXMmUNP.tmp
2025-10-09T21:42:44.902261Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/owmb/000355/r3tmp/yandexXMmUNP.tmp
2025-10-09T21:42:44.918818Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/owmb/000355/r3tmp/yandexXMmUNP.tmp
2025-10-09T21:42:44.918929Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
TClient is connected to server localhost:19671
PQClient connected to localhost:11444
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-09T21:42:44.961439Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
waiting...
2025-10-09T21:42:44.984005Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976720658, at schemeshard: 72057594046644480
2025-10-09T21:42:44.984461Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
waiting...
waiting...
2025-10-09T21:42:45.006804Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:45.312821Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340718649276813:2323], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.312850Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.313022Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340718649276850:2328], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.313031Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340718649276851:2329], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.313052Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.313907Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976720661:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-10-09T21:42:45.315880Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340718649276886:2332], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.315897Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.316026Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340718649276888:2333], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.316034Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.343955Z node 1 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [1:7559340718649276854:2330], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720661 completed, doublechecking }
2025-10-09T21:42:45.367316Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:45.368322Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [2:7559340720089275721:2301], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:45.369026Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=2&id=NDZlZjczZDctNWFlYzNkYmYtZjk4NTFlZGEtYzczMGY2Yg==, ActorId: [2:7559340720089275679:2294], ActorState: ExecuteState, TraceId: 01k75etbaw7tqpk8qm7r0c1vbh, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:45.369484Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:45.428884Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:45.432215Z node 1 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [1:7559340718649277122:2804] txid# 281474976720664, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:45.438225Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [1:7559340718649277141:2349], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:5:17: Error: At function: KiReadTable!
        <main>:5:17: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Versions]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:45.439343Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=1&id=OTM4Yzg3YmUtOTkzMTIxNjktM2VlOWRmNC0xZTAwM2VhOA==, ActorId: [1:7559340718649276809:2320], ActorState: ExecuteState, TraceId: 01k75etb9ybne981p49315nrd8, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:45.439470Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:45.481415Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720665:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:11444", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, false, 1000);
2025-10-09T21:42:45.519045Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720666. Ctx: { TraceId: 01k75etbfs9rv3tcq1sj7eg9n7, Database: , SessionId: ydb://session/3?node_id=1&id=MjZhMDUwMjItYWExM2M4MjUtOTg1ZTQ1ZjItY2YyZjhmMg==, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [1:7559340718649277395:3001] 
2025-10-09T21:42:45.710624Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:45.730011Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:49.714178Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7559340714354308590:2088];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:49.714207Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
2025-10-09T21:42:50.799057Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720672:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
waiting...
PQ Client: create topic: rt3.dc1--test-topic with 1 partitions
CallPersQueueGRPC request to localhost:11444
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:50.845252Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:11444
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720673
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-09T21:42:50.865167Z node 2 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][] pipe [1:7559340740124114402:3321] connected; active server actors: 1
2025-10-09T21:42:50.865654Z node 2 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1517: [72075186224037893][rt3.dc1--test-topic] updating configuration. Deleted partitions []. Added partitions [0]
2025-10-09T21:42:50.865902Z node 2 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:940: [72075186224037893][rt3.dc1--test-topic] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-09T21:42:50.870290Z node 2 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:138: [72075186224037893][rt3.dc1--test-topic] BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) 
2025-10-09T21:42:50.871786Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:50.872499Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:50.872551Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:784: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-09T21:42:50.872559Z node 1 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:50.872568Z node 1 :PERSQUEUE INFO: pq_impl.cpp:609: [PQ: 72075186224037892] doesn't have tx writes info
2025-10-09T21:42:50.873029Z node 2 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 1, Generation 1
2025-10-09T21:42:50.873117Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340740124114401:3320], now have 1 active actors on pipe
2025-10-09T21:42:50.873124Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [2:7559340741564112518:2360], now have 1 active actors on pipe
2025-10-09T21:42:50.873222Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.873607Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720674. Ctx: { TraceId: 01k75etgq752fcj4j03z85gy1n, Database: , SessionId: ydb://session/3?node_id=1&id=YjhlMTU1YWMtZWNiYjA3ZDEtNWVkYjQyNi05MThiMWI3, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-10-09T21:42:50.880051Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.887010Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340740124114478:3375], now have 1 active actors on pipe
2025-10-09T21:42:50.887060Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
GetTopicVersionFromPath:  record Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
 name rt3.dc1--test-topic version0
2025-10-09T21:42:50.898517Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.898710Z node 1 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:50.898729Z node 1 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:50.898779Z node 1 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:50.898849Z node 1 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [1:7559340740124114502:2464]
2025-10-09T21:42:50.899351Z node 1 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:50.899354Z node 1 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:50.899357Z node 1 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [1:7559340740124114502:2464]
2025-10-09T21:42:50.899361Z node 1 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:50.899598Z node 1 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-10-09T21:42:50.899602Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.899621Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.899623Z node 1 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:50.899625Z node 1 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:50.899627Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.899628Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.899629Z node 1 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:50.899635Z node 1 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:50.899688Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.899703Z node 1 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.900898Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.900937Z node 1 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046170932, TxId 281474976720673
2025-10-09T21:42:50.900941Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.900943Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.900964Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.900996Z node 1 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:50.901490Z node 1 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:50.901532Z node 1 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:50.901538Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.901540Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.901541Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.901546Z node 1 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.901626Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:395: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-10-09T21:42:50.901638Z node 1 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:50.901694Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.903365Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.903386Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.903812Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:51.000791Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.000810Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.000813Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.106234Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.106248Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.106251Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.210116Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.210136Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.210140Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.308526Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.308543Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.308546Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046170932
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046170932
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:11444
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:51.403874Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
2025-10-09T21:42:51.408773Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-09T21:42:51.409004Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-09T21:42:51.409011Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:11444
2025-10-09T21:42:51.411471Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:51.412301Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.412315Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.412318Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.412395Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:51.412403Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1
2025-10-09T21:42:51.412832Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:51.412862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:56230
2025-10-09T21:42:51.412868Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:56230 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:51.412872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:51.413369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-09T21:42:51.413427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:51.413429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:51.413431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:51.413436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340744419081881:2483] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:51.414077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340744419081881:2483] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:51.450760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [1:7559340744419081881:2483] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:51.451999Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [1:7559340744419081881:2483] (SourceId=src, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src
2025-10-09T21:42:51.452011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340744419081881:2483] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:51.451347Z node 2 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340744419081915:2483] connected; active server actors: 1
2025-10-09T21:42:51.455723Z node 2 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340744419081915:2483] disconnected.
2025-10-09T21:42:51.455737Z node 2 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340744419081915:2483] disconnected; active server actors: 1
2025-10-09T21:42:51.455740Z node 2 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340744419081915:2483] disconnected no session
2025-10-09T21:42:51.481928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340744419081881:2483] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:51.481946Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340744419081881:2483] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:42:51.481949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340744419081881:2483] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:51.481966Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:51.482530Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 1, Generation: 1
2025-10-09T21:42:51.482548Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340744419081936:2483], now have 1 active actors on pipe
2025-10-09T21:42:51.482626Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.482635Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.482665Z node 1 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src|4350852a-59e36d59-93e779af-8ee1ab2b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src
2025-10-09T21:42:51.482677Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.482683Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.482686Z node 1 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:51.482699Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.482716Z node 1 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:51.482733Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.483316Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046171483 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:51.483348Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|4350852a-59e36d59-93e779af-8ee1ab2b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:51.482838Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.482843Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.482857Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.482884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1 partition: 0 MaxSeqNo: 0 sessionId: src|4350852a-59e36d59-93e779af-8ee1ab2b_0
2025-10-09T21:42:51.483485Z :INFO: [] MessageGroupId [src] SessionId [src|4350852a-59e36d59-93e779af-8ee1ab2b_0] Write session: close. Timeout = 0 ms
2025-10-09T21:42:51.483496Z :INFO: [] MessageGroupId [src] SessionId [src|4350852a-59e36d59-93e779af-8ee1ab2b_0] Write session will now close
2025-10-09T21:42:51.483503Z :DEBUG: [] MessageGroupId [src] SessionId [src|4350852a-59e36d59-93e779af-8ee1ab2b_0] Write session: aborting
2025-10-09T21:42:51.483628Z :INFO: [] MessageGroupId [src] SessionId [src|4350852a-59e36d59-93e779af-8ee1ab2b_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:42:51.483634Z :DEBUG: [] MessageGroupId [src] SessionId [src|4350852a-59e36d59-93e779af-8ee1ab2b_0] Write session: destroy
2025-10-09T21:42:51.483771Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId: src|4350852a-59e36d59-93e779af-8ee1ab2b_0 grpc read done: success: 0 data: 
2025-10-09T21:42:51.483778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 1 sessionId: src|4350852a-59e36d59-93e779af-8ee1ab2b_0 grpc read failed
2025-10-09T21:42:51.483783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 1 sessionId: src|4350852a-59e36d59-93e779af-8ee1ab2b_0 grpc closed
2025-10-09T21:42:51.483788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1 sessionId: src|4350852a-59e36d59-93e779af-8ee1ab2b_0 is DEAD
2025-10-09T21:42:51.483927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:42:51.484124Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340744419081936:2483] destroyed
2025-10-09T21:42:51.484134Z node 1 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:42:51.484139Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.484140Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.484142Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Session was created
Got new write session event: ReadyToAcceptEvent
Got new write session event: ReadyToAcceptEvent
2025-10-09T21:42:51.513663Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.513677Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.513679Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: ok
2025-10-09T21:42:51.546000Z :INFO: [/Root] [] [] Start federated write session to database 'dc1' (previous was <empty>) FederationState: { Status: SUCCESS SelfLocation: "fancy_datacenter" DbInfos: [ { name: "dc1" path: "/Root" id: "account-dc1" endpoint: "localhost:11444" location: "dc1" status: AVAILABLE weight: 1000 } { name: "dc2" path: "/Root" id: "account-dc2" endpoint: "localhost:11444" location: "dc2" status: AVAILABLE weight: 500 } { name: "dc3" path: "/Root" id: "account-dc3" endpoint: "localhost:11444" location: "dc3" status: AVAILABLE weight: 500 } ] ControlPlaneEndpoint: cp.logbroker-federation:2135 }
2025-10-09T21:42:51.562285Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: try to update token
2025-10-09T21:42:51.562530Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Start write session. Will connect to nodeId: 0
2025-10-09T21:42:51.613914Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.613930Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.613933Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.638521Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: write to message_group: src_id
2025-10-09T21:42:51.638800Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:51.638836Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2
2025-10-09T21:42:51.640310Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: send init request: init_request { path: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:51.640460Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: OnWriteDone gRpcStatusCode: 0
2025-10-09T21:42:51.640618Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId:  grpc read done: success: 1 data: init_request { path: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:51.640679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2 path: "test-topic" message_group_id: "src_id" from ipv6:[::1]:45066
2025-10-09T21:42:51.640683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2 sessionId= userAgent="topic server" ip=ipv6:[::1]:45066 proto=topic  topic=test-topic durationSec=0
2025-10-09T21:42:51.640688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:51.641288Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2 sessionId:  describe result for acl check
2025-10-09T21:42:51.641336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:51.641342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:51.641344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:51.641350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340744419081947:2496] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:51.641983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340744419081947:2496] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:51.646879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [1:7559340744419081947:2496] (SourceId=src_id, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:51.647067Z node 2 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340744419081966:2496] connected; active server actors: 1
2025-10-09T21:42:51.647253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [1:7559340744419081947:2496] (SourceId=src_id, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src_id
2025-10-09T21:42:51.647264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340744419081947:2496] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:51.647362Z node 2 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340744419081966:2496] disconnected.
2025-10-09T21:42:51.647371Z node 2 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340744419081966:2496] disconnected; active server actors: 1
2025-10-09T21:42:51.647374Z node 2 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340744419081966:2496] disconnected no session
2025-10-09T21:42:51.651932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340744419081947:2496] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:51.651949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340744419081947:2496] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:42:51.651953Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340744419081947:2496] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:51.651963Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:51.652363Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 1, Generation: 1
2025-10-09T21:42:51.652385Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340744419081979:2496], now have 1 active actors on pipe
2025-10-09T21:42:51.652391Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.652398Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.652428Z node 1 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:51.652439Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.652444Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.652448Z node 1 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:51.652472Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.652488Z node 1 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:51.652508Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.652692Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.652699Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.652735Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.652770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2 partition: 0 MaxSeqNo: 0 sessionId: src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0
2025-10-09T21:42:51.654100Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: OnReadDone gRpcStatusCode: 0
2025-10-09T21:42:51.654123Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046171654 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:51.654160Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session established. Init response: session_id: "src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0" supported_codecs { codecs: 1 codecs: 2 codecs: 3 }
2025-10-09T21:42:51.654248Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0]  MessageGroupId [src_id] Write 1 messages with Id from 1 to 1
2025-10-09T21:42:51.654600Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0]  MessageGroupId [src_id] Write session: try to update token
2025-10-09T21:42:51.654613Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0]  MessageGroupId [src_id] Send 1 message(s) (0 left), first sequence number is 1
2025-10-09T21:42:51.654859Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.654985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.655067Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.655076Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.655109Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:42:51.655144Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:42:51.655270Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.655277Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.655389Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 0 messageNo: 1 size 96 offset: -1
2025-10-09T21:42:51.655401Z node 1 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.655419Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.655427Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.655430Z node 1 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:51.655463Z node 1 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 0
2025-10-09T21:42:51.655510Z node 1 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 1 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 170 count 1 nextOffset 1 batches 1
2025-10-09T21:42:51.655517Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.655571Z node 1 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 0,1 HeadOffset 0 endOffset 0 curOffset 1 d0000000000_00000000000000000000_00000_0000000001_00000? size 158 WTime 1760046171655
2025-10-09T21:42:51.655595Z node 1 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:51.655606Z node 1 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 1 size 158
2025-10-09T21:42:51.656883Z node 1 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 1 size 158 actorID [1:7559340740124114431:2464]
2025-10-09T21:42:51.656901Z node 1 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:51.656921Z node 1 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 103 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:51.656929Z node 1 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.656938Z node 1 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 1, partNo: 0, Offset: 0 is stored on disk
2025-10-09T21:42:51.656968Z node 1 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:51.656974Z node 1 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:51.656979Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.656982Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.656984Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.656990Z node 1 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:51.656999Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1  requestId: <none> cookie: 1
2025-10-09T21:42:51.657045Z node 1 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 1 max time lag 0ms effective offset 0
2025-10-09T21:42:51.657062Z node 1 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 158 count 1 last offset 0, current partition end offset: 1
2025-10-09T21:42:51.657064Z node 1 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:51.657074Z node 1 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 0 source 1 size 158 accessed 0 times before, last time 2025-10-09T21:42:51.000000Z
2025-10-09T21:42:51.657078Z node 1 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 3. All 1 blobs are from cache.
2025-10-09T21:42:51.657092Z node 1 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:51.657105Z node 1 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 1 count 1 size 138 from pos 0 cbcount 1
2025-10-09T21:42:51.657122Z node 1 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171655 queuesize 0 startOffset 0
2025-10-09T21:42:51.657137Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:42:51.657422Z node 1 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63' size 158
2025-10-09T21:42:51.657433Z node 1 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63'
2025-10-09T21:42:51.658090Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0]  MessageGroupId [src_id] Write session: OnReadDone gRpcStatusCode: 0
2025-10-09T21:42:51.658243Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0]  MessageGroupId [src_id] Write session got write response: acks { seq_no: 1 written { } } write_statistics { persisting_time { nanos: 1000000 } min_queue_wait_time { } max_queue_wait_time { } partition_quota_wait_time { } topic_quota_wait_time { } }
2025-10-09T21:42:51.658251Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0]  MessageGroupId [src_id] OnAck: seqNo=1, txId=?
2025-10-09T21:42:51.658256Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0]  MessageGroupId [src_id] Write session: acknoledged message 1
Got new write session event: AcksEvent: { seqNo : 1, State : EES_WRITTEN, offset : 0, partitionId : 0 } write stat: Write time 0.001000s minimal time in partition queue 0.000000s maximal time in partition queue 0.000000s partition quoted time 0.000000s topic quoted time 0.000000s
2025-10-09T21:42:51.662171Z :INFO: [/Root]  TraceId []  SessionId [src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0]  MessageGroupId [src_id] Write session: close. Timeout 0.000000s
2025-10-09T21:42:51.662177Z :INFO: [/Root]  TraceId []  SessionId [src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0]  MessageGroupId [src_id] Write session will now close
2025-10-09T21:42:51.662188Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0]  MessageGroupId [src_id] Write session: aborting
2025-10-09T21:42:51.662314Z :INFO: [/Root]  TraceId []  SessionId [src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0]  MessageGroupId [src_id] Write session: gracefully shut down, all writes complete
2025-10-09T21:42:51.662321Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0]  MessageGroupId [src_id] Write session: destroy
2025-10-09T21:42:51.662855Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0 grpc read done: success: 0 data: 
2025-10-09T21:42:51.662862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 2 sessionId: src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0 grpc read failed
2025-10-09T21:42:51.662871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 2 sessionId: src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0 grpc closed
2025-10-09T21:42:51.662879Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2 sessionId: src_id|98afed7c-3eeb28eb-40deeab6-d360087e_0 is DEAD
2025-10-09T21:42:51.663108Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:42:51.663335Z node 1 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340744419081979:2496] destroyed
2025-10-09T21:42:51.663348Z node 1 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:42:51.663357Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.663360Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.663362Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.714272Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.714284Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.714286Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.815631Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.815644Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.815646Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.915964Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.915978Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.915987Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.016446Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.016462Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.016465Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.118304Z node 1 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.118321Z node 1 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.118324Z node 1 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.315081Z :CloseWriteSessionImmediately INFO: Random seed for debugging is 1760046172315073
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/owmb/000355/r3tmp/tmppBdPet/pdisk_1.dat
2025-10-09T21:42:52.464887Z node 4 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:123: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, got undelivered to scheme cache: ActorUnknown
2025-10-09T21:42:52.464940Z node 4 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[4:7559340751380483898:2178];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:52.465064Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-10-09T21:42:52.465067Z node 4 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:52.464063Z node 3 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:52.464795Z node 3 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:52.464819Z node 3 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
2025-10-09T21:42:52.496322Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:52.496349Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:52.496672Z node 3 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:42:52.501578Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:52.501596Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:52.512458Z node 3 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 4 Cookie 4
2025-10-09T21:42:52.512514Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:52.514271Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Connecting -> Connected
TServer::EnableGrpc on GrpcPort 30984, node 3
2025-10-09T21:42:52.538927Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/owmb/000355/r3tmp/yandexvEdGtA.tmp
2025-10-09T21:42:52.538943Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/owmb/000355/r3tmp/yandexvEdGtA.tmp
2025-10-09T21:42:52.546138Z INFO: TTestServer started on Port 29288 GrpcPort 30984
2025-10-09T21:42:52.553885Z node 4 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
TClient is connected to server localhost:29288
PQClient connected to localhost:30984
WaitRootIsUp 'Root'...
TClient::Ls request: Root
2025-10-09T21:42:52.596266Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/owmb/000355/r3tmp/yandexvEdGtA.tmp
2025-10-09T21:42:52.596551Z node 3 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-09T21:42:52.606015Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976710657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
2025-10-09T21:42:52.611015Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976710657, at schemeshard: 72057594046644480
waiting...
waiting...
waiting...
2025-10-09T21:42:52.651704Z node 3 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:52.952269Z node 4 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [4:7559340751380484102:2299], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.952298Z node 4 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [4:7559340751380484070:2296], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.952378Z node 4 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.953255Z node 4 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [4:7559340751380484108:2301], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.953280Z node 4 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.955329Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976720657:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-10-09T21:42:52.971986Z node 4 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [4:7559340751380484107:2300], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720657 completed, doublechecking }
2025-10-09T21:42:53.055117Z node 4 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [4:7559340755675451433:2134] txid# 281474976720658, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:53.062099Z node 4 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [4:7559340755675451448:2305], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:53.063131Z node 4 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=4&id=NDFkZmRkNTYtZTljNzQ0ODktYjFmYWNiN2MtMWNmMTA2YzM=, ActorId: [4:7559340751380484068:2295], ActorState: ExecuteState, TraceId: 01k75etjrp48p1mtxs1v9yvk0y, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:53.063332Z node 4 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:53.075220Z node 3 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [3:7559340753257076251:2330], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:53.076064Z node 3 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=3&id=ZjlmMjc5MWYtOWUxN2IwYTItNTk0MTZiMzktNWMyMWZiYzM=, ActorId: [3:7559340753257076208:2321], ActorState: ExecuteState, TraceId: 01k75etjwa9k0gjnzx9kh9yp3p, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:53.076222Z node 3 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:53.082351Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976710661:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:53.137511Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976710662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:53.186383Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976710663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:30984", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, false, 1000);
2025-10-09T21:42:53.249502Z node 3 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976710664. Ctx: { TraceId: 01k75etk14ahbq3vmhrhaj13a0, Database: , SessionId: ydb://session/3?node_id=3&id=MTc5ZGMzY2QtNGJjN2I5M2MtODdmODc2NzktN2Q2NTcxNTU=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [3:7559340753257076643:2938] 
2025-10-09T21:42:53.462346Z node 3 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:53.468443Z node 4 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:57.465113Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[4:7559340751380483898:2178];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:57.465153Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
2025-10-09T21:42:58.407438Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976710670:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
waiting...
PQ Client: create topic: rt3.dc1--test-topic with 1 partitions
CallPersQueueGRPC request to localhost:30984
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:58.432079Z node 3 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:30984
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976710671
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-09T21:42:58.447591Z node 3 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][] pipe [3:7559340774731913523:3195] connected; active server actors: 1
2025-10-09T21:42:58.447757Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1517: [72075186224037893][rt3.dc1--test-topic] updating configuration. Deleted partitions []. Added partitions [0]
2025-10-09T21:42:58.448039Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:940: [72075186224037893][rt3.dc1--test-topic] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-09T21:42:58.448488Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:138: [72075186224037893][rt3.dc1--test-topic] BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) 
2025-10-09T21:42:58.454092Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:58.457470Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:58.457569Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:784: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-09T21:42:58.457580Z node 4 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:58.457605Z node 4 :PERSQUEUE INFO: pq_impl.cpp:609: [PQ: 72075186224037892] doesn't have tx writes info
2025-10-09T21:42:58.457755Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340774731913522:3194], now have 1 active actors on pipe
2025-10-09T21:42:58.457761Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340774731913550:2423], now have 1 active actors on pipe
2025-10-09T21:42:58.462069Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:58.463749Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 4, Generation 1
2025-10-09T21:42:58.464545Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:58.467999Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340774731913592:3244], now have 1 active actors on pipe
2025-10-09T21:42:58.468151Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:58.474660Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:58.474864Z node 4 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:58.474878Z node 4 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:58.474968Z node 4 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:58.475005Z node 4 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [4:7559340777150288395:2388]
2025-10-09T21:42:58.475722Z node 4 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:58.475727Z node 4 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:58.475735Z node 4 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [4:7559340777150288395:2388]
2025-10-09T21:42:58.475742Z node 4 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:58.476045Z node 4 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-10-09T21:42:58.476053Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.476071Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.476074Z node 4 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:58.476079Z node 4 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:58.476082Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.476084Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.476088Z node 4 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:58.476095Z node 4 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:58.476177Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:58.476215Z node 4 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:58.479020Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:58.479068Z node 4 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046178513, TxId 281474976710671
2025-10-09T21:42:58.479085Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.479087Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.479117Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.479169Z node 4 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:58.480007Z node 4 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:58.481402Z node 3 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976710672. Ctx: { TraceId: 01k75etr4w2590cdzrrazj5qct, Database: , SessionId: ydb://session/3?node_id=3&id=NTcwODM1YzgtNzM2MmNkMGEtZDMwYmMxMDAtZjNhZjNlZjc=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-10-09T21:42:58.482390Z node 4 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-10-09T21:42:58.482481Z node 4 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:58.482505Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.482507Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.482509Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.482520Z node 4 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:58.482662Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:395: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-10-09T21:42:58.482684Z node 4 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:58.482741Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:58.485325Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:58.485353Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:58.486308Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976710671
    CreateStep: 1760046178513
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976710671
    CreateStep: 1760046178513
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:30984
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:58.506453Z node 3 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
2025-10-09T21:42:58.508121Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-09T21:42:58.508371Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-09T21:42:58.508377Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:30984
2025-10-09T21:42:58.508772Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:58.509199Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:58.509218Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1
2025-10-09T21:42:58.509335Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:58.509362Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:59578
2025-10-09T21:42:58.509367Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:59578 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:58.509371Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:58.510101Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-09T21:42:58.510133Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:58.510135Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:58.510137Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:58.510143Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [3:7559340774731913664:2443] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:58.510729Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [3:7559340774731913664:2443] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:58.541787Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [3:7559340774731913664:2443] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:58.541864Z node 3 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340774731913698:2443] connected; active server actors: 1
2025-10-09T21:42:58.541884Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [3:7559340774731913664:2443] (SourceId=src, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src
2025-10-09T21:42:58.541888Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [3:7559340774731913664:2443] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:58.541969Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340774731913698:2443] disconnected.
2025-10-09T21:42:58.541981Z node 3 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340774731913698:2443] disconnected; active server actors: 1
2025-10-09T21:42:58.541985Z node 3 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340774731913698:2443] disconnected no session
2025-10-09T21:42:58.563318Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [3:7559340774731913664:2443] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:58.563333Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [3:7559340774731913664:2443] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:42:58.563337Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [3:7559340774731913664:2443] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:58.563348Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:58.563685Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340774731913721:2443], now have 1 active actors on pipe
2025-10-09T21:42:58.563726Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 4, Generation: 1
2025-10-09T21:42:58.563817Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.563831Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.563859Z node 4 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src|1031eed1-5b87ca71-51a84186-c419e190_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src
2025-10-09T21:42:58.563874Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.563884Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.563887Z node 4 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:58.563899Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.563919Z node 4 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:58.563942Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:58.564329Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.564343Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.564362Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:58.564497Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1 partition: 0 MaxSeqNo: 0 sessionId: src|1031eed1-5b87ca71-51a84186-c419e190_0
2025-10-09T21:42:58.564816Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046178564 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:58.564858Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|1031eed1-5b87ca71-51a84186-c419e190_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:58.564963Z :INFO: [] MessageGroupId [src] SessionId [src|1031eed1-5b87ca71-51a84186-c419e190_0] Write session: close. Timeout = 0 ms
2025-10-09T21:42:58.564975Z :INFO: [] MessageGroupId [src] SessionId [src|1031eed1-5b87ca71-51a84186-c419e190_0] Write session will now close
2025-10-09T21:42:58.564983Z :DEBUG: [] MessageGroupId [src] SessionId [src|1031eed1-5b87ca71-51a84186-c419e190_0] Write session: aborting
2025-10-09T21:42:58.565143Z :INFO: [] MessageGroupId [src] SessionId [src|1031eed1-5b87ca71-51a84186-c419e190_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:42:58.565150Z :DEBUG: [] MessageGroupId [src] SessionId [src|1031eed1-5b87ca71-51a84186-c419e190_0] Write session: destroy
2025-10-09T21:42:58.565341Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId: src|1031eed1-5b87ca71-51a84186-c419e190_0 grpc read done: success: 0 data: 
2025-10-09T21:42:58.565360Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 1 sessionId: src|1031eed1-5b87ca71-51a84186-c419e190_0 grpc read failed
2025-10-09T21:42:58.565367Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 1 sessionId: src|1031eed1-5b87ca71-51a84186-c419e190_0 grpc closed
2025-10-09T21:42:58.565372Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1 sessionId: src|1031eed1-5b87ca71-51a84186-c419e190_0 is DEAD
2025-10-09T21:42:58.565608Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:42:58.565726Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [3:7559340774731913721:2443] destroyed
2025-10-09T21:42:58.565744Z node 4 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:42:58.565751Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.565754Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.565756Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.594109Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.594129Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.594132Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Session was created
2025-10-09T21:42:58.692870Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.692883Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.692886Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.794160Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.794176Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.794179Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:44.616249Z :FallbackToSingleDb INFO: Random seed for debugging is 1760046164616246
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/owmb/000352/r3tmp/tmpuZegFK/pdisk_1.dat
2025-10-09T21:42:44.778805Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:44.780541Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:44.860906Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:44.860946Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
2025-10-09T21:42:44.862529Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:44.862576Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
2025-10-09T21:42:44.866678Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:44.866702Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:44.867033Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:44.867043Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:44.870789Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-10-09T21:42:44.870890Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:44.871040Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:44.875130Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
TServer::EnableGrpc on GrpcPort 18074, node 1
2025-10-09T21:42:44.900260Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/owmb/000352/r3tmp/yandexpqqhWx.tmp
2025-10-09T21:42:44.900271Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/owmb/000352/r3tmp/yandexpqqhWx.tmp
2025-10-09T21:42:44.904245Z INFO: TTestServer started on Port 22421 GrpcPort 18074
2025-10-09T21:42:44.918611Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/owmb/000352/r3tmp/yandexpqqhWx.tmp
2025-10-09T21:42:44.918681Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
TClient is connected to server localhost:22421
PQClient connected to localhost:18074
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-09T21:42:44.957246Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976715657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
waiting...
waiting...
waiting...
2025-10-09T21:42:45.038311Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:45.082606Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:45.275645Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340721126508620:2325], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.275687Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.276261Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340721126508647:2328], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.276275Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340721126508648:2329], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.276302Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.277192Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976715661:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-10-09T21:42:45.279641Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7559340719763483103:2299], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.279682Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7559340719763483071:2296], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.279702Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.280153Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7559340719763483109:2301], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.280171Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.282956Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340721126508683:2332], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.283062Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.289418Z node 2 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [2:7559340719763483110:2131] txid# 281474976720657, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exists but creating right now (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateCreate), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:45.298935Z node 1 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [1:7559340721126508651:2330], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976715661 completed, doublechecking }
2025-10-09T21:42:45.299082Z node 2 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [2:7559340719763483108:2300], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976715661 completed, doublechecking }
2025-10-09T21:42:45.321458Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976715662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:45.357602Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976715663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:45.383722Z node 1 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [1:7559340721126508977:2852] txid# 281474976715664, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:45.397855Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976715665:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:45.405822Z node 2 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [2:7559340719763483202:2177] txid# 281474976720658, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:45.415351Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [1:7559340721126508988:2349], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:5:17: Error: At function: KiReadTable!
        <main>:5:17: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Versions]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:45.416174Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=1&id=ZDM1Mzc1YWMtZTJmODFkNjgtNWVlZjBmNS02ZDc5ODVkZg==, ActorId: [1:7559340721126508617:2323], ActorState: ExecuteState, TraceId: 01k75etb8s09rrwmq3qn5edy2k, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:45.416661Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:45.423122Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [2:7559340719763483216:2309], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:5:17: Error: At function: KiReadTable!
        <main>:5:17: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Versions]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:45.423823Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=2&id=MzgwMjQyMDAtYjFlNWJiOWUtZWFkMjBhLThmNmFlNGM=, ActorId: [2:7559340719763483069:2295], ActorState: ExecuteState, TraceId: 01k75etb8yb6rzbw6r2e21v7w8, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:45.423939Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:18074", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, false, 1000);
2025-10-09T21:42:45.476135Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976715666. Ctx: { TraceId: 01k75etbe21rmbggnvncjy909r, Database: , SessionId: ydb://session/3?node_id=1&id=Y2FlZTFlOC0xYzlmYzM1ZS01M2QyYzBiZi04NjJhYzhlYQ==, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [1:7559340721126509143:2956] 
2025-10-09T21:42:45.722243Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:45.758112Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
=== CheckClustersList. Ok
2025-10-09T21:42:50.834395Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976715672:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
waiting...
PQ Client: create topic: rt3.dc1--test-topic with 1 partitions
CallPersQueueGRPC request to localhost:18074
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:50.933194Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:18074
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976715673
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-09T21:42:50.956510Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][] pipe [1:7559340742601346049:3225] connected; active server actors: 1
2025-10-09T21:42:50.962134Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1517: [72075186224037893][rt3.dc1--test-topic] updating configuration. Deleted partitions []. Added partitions [0]
2025-10-09T21:42:50.962389Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:940: [72075186224037893][rt3.dc1--test-topic] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-09T21:42:50.962835Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:138: [72075186224037893][rt3.dc1--test-topic] BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) 
2025-10-09T21:42:50.975914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:50.976663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:50.976713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:784: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-09T21:42:50.976721Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:50.976731Z node 2 :PERSQUEUE INFO: pq_impl.cpp:609: [PQ: 72075186224037892] doesn't have tx writes info
2025-10-09T21:42:50.977110Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-10-09T21:42:50.977252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340742601346067:2434], now have 1 active actors on pipe
2025-10-09T21:42:50.977259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340742601346048:3224], now have 1 active actors on pipe
2025-10-09T21:42:50.977373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.980846Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976715674. Ctx: { TraceId: 01k75etgtef76ybgb1pvb8z30v, Database: , SessionId: ydb://session/3?node_id=1&id=NTg0NmVhMGItZDExNzNmNmMtOTVlYTVmMTEtYjJkNWYxNWI=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-10-09T21:42:50.981076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.983703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340742601346119:3274], now have 1 active actors on pipe
2025-10-09T21:42:50.983942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.985278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.985527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:50.985545Z node 2 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:50.985591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:50.985669Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340741238320118:2390]
2025-10-09T21:42:50.986381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:50.986385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:50.986390Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [2:7559340741238320118:2390]
2025-10-09T21:42:50.986397Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:50.986698Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-10-09T21:42:50.986705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.986738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.986741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:50.986744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:50.986746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.986747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.986748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:50.986755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:50.986823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.986847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.989815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.989847Z node 2 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046171030, TxId 281474976715673
2025-10-09T21:42:50.989851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.989853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.989873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.989904Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
2025-10-09T21:42:50.992823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:50.992884Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:50.992893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.992895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.992897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.992903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.993010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:395: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-10-09T21:42:50.993028Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:50.993070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976715673
    CreateStep: 1760046171030
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976715673
    CreateStep: 1760046171030
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:18074
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:50.997894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.997924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.999208Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-10-09T21:42:50.999295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
2025-10-09T21:42:51.002624Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-09T21:42:51.002847Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-09T21:42:51.002855Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:18074
2025-10-09T21:42:51.005169Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:51.005769Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:51.005787Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1
2025-10-09T21:42:51.006050Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:51.006083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:37490
2025-10-09T21:42:51.006089Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:37490 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:51.006093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:51.006635Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-09T21:42:51.006676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:51.006678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:51.006679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:51.006685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340746896313469:2450] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:51.007272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340746896313469:2450] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:51.037795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [1:7559340746896313469:2450] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:51.038347Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340746896313500:2450] connected; active server actors: 1
2025-10-09T21:42:51.042211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [1:7559340746896313469:2450] (SourceId=src, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src
2025-10-09T21:42:51.042221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340746896313469:2450] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:51.045797Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340746896313500:2450] disconnected.
2025-10-09T21:42:51.045807Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340746896313500:2450] disconnected; active server actors: 1
2025-10-09T21:42:51.045809Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340746896313500:2450] disconnected no session
2025-10-09T21:42:51.071603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340746896313469:2450] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:51.071620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340746896313469:2450] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:42:51.071624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340746896313469:2450] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:51.071636Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:51.072639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340746896313523:2450], now have 1 active actors on pipe
2025-10-09T21:42:51.072801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1
2025-10-09T21:42:51.072971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.072983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.073029Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src|9aed0964-7e0c60f1-97a21e2e-c5f118b1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src
2025-10-09T21:42:51.073043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.073050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.073054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:51.073068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.073087Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:51.073111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.073501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.073509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.073535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.073812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1 partition: 0 MaxSeqNo: 0 sessionId: src|9aed0964-7e0c60f1-97a21e2e-c5f118b1_0
2025-10-09T21:42:51.074795Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046171074 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:51.074840Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|9aed0964-7e0c60f1-97a21e2e-c5f118b1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:51.074946Z :INFO: [] MessageGroupId [src] SessionId [src|9aed0964-7e0c60f1-97a21e2e-c5f118b1_0] Write session: close. Timeout = 0 ms
2025-10-09T21:42:51.074952Z :INFO: [] MessageGroupId [src] SessionId [src|9aed0964-7e0c60f1-97a21e2e-c5f118b1_0] Write session will now close
2025-10-09T21:42:51.074958Z :DEBUG: [] MessageGroupId [src] SessionId [src|9aed0964-7e0c60f1-97a21e2e-c5f118b1_0] Write session: aborting
2025-10-09T21:42:51.075058Z :INFO: [] MessageGroupId [src] SessionId [src|9aed0964-7e0c60f1-97a21e2e-c5f118b1_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:42:51.075062Z :DEBUG: [] MessageGroupId [src] SessionId [src|9aed0964-7e0c60f1-97a21e2e-c5f118b1_0] Write session: destroy
2025-10-09T21:42:51.075319Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId: src|9aed0964-7e0c60f1-97a21e2e-c5f118b1_0 grpc read done: success: 0 data: 
2025-10-09T21:42:51.075331Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 1 sessionId: src|9aed0964-7e0c60f1-97a21e2e-c5f118b1_0 grpc read failed
2025-10-09T21:42:51.075339Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 1 sessionId: src|9aed0964-7e0c60f1-97a21e2e-c5f118b1_0 grpc closed
2025-10-09T21:42:51.075344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1 sessionId: src|9aed0964-7e0c60f1-97a21e2e-c5f118b1_0 is DEAD
2025-10-09T21:42:51.075535Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:42:51.075818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340746896313523:2450] destroyed
2025-10-09T21:42:51.075833Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:42:51.075842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.075845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.075849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.086934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.086948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.086951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Session was created
2025-10-09T21:42:51.096795Z :INFO: [/Root] OnFederationDiscovery fall back to single mode, database=/Root
2025-10-09T21:42:51.096845Z :INFO: [/Root] [] [60c755ae-ae8c041-fb579c67-b4a7dbfe] Open read subsessions to databases: { name: , endpoint: localhost:18074, path: /Root }
2025-10-09T21:42:51.096927Z :INFO: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] Starting read session
2025-10-09T21:42:51.096932Z :DEBUG: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] Starting single session
2025-10-09T21:42:51.097176Z :DEBUG: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:51.097181Z :DEBUG: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:51.097185Z :DEBUG: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] [] Reconnecting session to cluster  in 0.000000s
2025-10-09T21:42:51.102969Z :DEBUG: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] [] Successfully connected. Initializing session
2025-10-09T21:42:51.103464Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:42:51.103478Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 1
2025-10-09T21:42:51.105374Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:42:51.105473Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 read init: from# ipv6:[::1]:41430, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:42:51.105574Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 auth for : user
2025-10-09T21:42:51.105827Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 Handle describe topics response
2025-10-09T21:42:51.105858Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 auth is DEAD
2025-10-09T21:42:51.105879Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:42:51.106184Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:42:51.106445Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340746896313532:2464] connected; active server actors: 1
2025-10-09T21:42:51.106663Z :INFO: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] [] Got InitResponse. ReadSessionId: shared/user_1_1_3714194436971602743_v1
2025-10-09T21:42:51.106677Z :DEBUG: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] [] In ContinueReadingDataImpl, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:51.106735Z :DEBUG: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:42:51.106858Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 grpc read done: success# 1, data# { read_request { bytes_size: 524288 } }
2025-10-09T21:42:51.106880Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 got read request: guid# 2075d8fd-c5e0cd83-97aa1df6-1782a4c1
2025-10-09T21:42:51.106893Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559340746896313532:2464] session shared/user_1_1_3714194436971602743_v1
2025-10-09T21:42:51.106912Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:42:51.106923Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:42:51.106937Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_1_3714194436971602743_v1" (Sender=[1:7559340746896313529:2464], Pipe=[1:7559340746896313532:2464], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:42:51.106941Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:42:51.106953Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:42:51.106960Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_1_3714194436971602743_v1" (Sender=[1:7559340746896313529:2464], Pipe=[1:7559340746896313532:2464], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:42:51.106971Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_1_3714194436971602743_v1" sender [1:7559340746896313529:2464] lock partition 0 for ReadingSession "shared/user_1_1_3714194436971602743_v1" (Sender=[1:7559340746896313529:2464], Pipe=[1:7559340746896313532:2464], Partitions=[], ActiveFamilyCount=1) generation 1 step 1
2025-10-09T21:42:51.106985Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:42:51.106990Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000034s
2025-10-09T21:42:51.107205Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 1 Session: "shared/user_1_1_3714194436971602743_v1" ClientId: "user" PipeClient { RawX1: 7559340746896313532 RawX2: 4503603922340256 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:42:51.107226Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:42:51.109470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340746896313535:2467], now have 1 active actors on pipe
2025-10-09T21:42:51.109630Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1, pipe: [1:7559340746896313535:2467]
2025-10-09T21:42:51.109791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.109802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.109816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_1_3714194436971602743_v1 on pipe: [1:7559340746896313535:2467]
2025-10-09T21:42:51.109837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.109843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.109846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:51.109864Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_1_3714194436971602743_v1
2025-10-09T21:42:51.109868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.109898Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:51.110023Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_1_3714194436971602743_v1:1 with generation 1
2025-10-09T21:42:51.111375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:51.111461Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-10-09T21:42:51.111483Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:51.111491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.111493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.111495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.111502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:51.111513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:42:51.111804Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 0 SizeLag: 0 WriteTimestampEstimateMS: 1760046171086 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:42:51.111820Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 0 readOffset 0 committedOffset 0
2025-10-09T21:42:51.111845Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 sending to client partition status
Got new read session event: StartPartitionSession { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name:  Database path: /Root Database id:  CommittedOffset: 0 EndOffset: 0 }
2025-10-09T21:42:51.112815Z :INFO: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] [] Confirm partition stream create. Partition stream id: 1. Cluster: "-". Topic: "test-topic". Partition: 0. Read offset: (NULL)
2025-10-09T21:42:51.112884Z :INFO: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] Closing read session. Close timeout: 0.010000s
2025-10-09T21:42:51.112895Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:0:1:0:0
2025-10-09T21:42:51.112903Z :INFO: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] Counters: { Errors: 0 CurrentSessionLifetimeMs: 15 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:51.113026Z :INFO: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] Closing read session. Close timeout: 0.000000s
2025-10-09T21:42:51.113031Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:0:1:0:0
2025-10-09T21:42:51.113035Z :INFO: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] Counters: { Errors: 0 CurrentSessionLifetimeMs: 16 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:51.113043Z :INFO: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] Closing read session. Close timeout: 0.000000s
2025-10-09T21:42:51.113046Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:0:1:0:0
2025-10-09T21:42:51.113049Z :INFO: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] Counters: { Errors: 0 CurrentSessionLifetimeMs: 16 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:51.113062Z :NOTICE: [/Root] [/Root] [489b1129-b9577ab0-8dfdfc69-e055dffb] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-09T21:42:51.113553Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 grpc read done: success# 1, data# { start_partition_session_response { partition_session_id: 1 } }
2025-10-09T21:42:51.113576Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:92: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 grpc closed
2025-10-09T21:42:51.113584Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 1 consumer shared/user session shared/user_1_1_3714194436971602743_v1 is DEAD
2025-10-09T21:42:51.114050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_1_3714194436971602743_v1
2025-10-09T21:42:51.114075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340746896313535:2467] destroyed
2025-10-09T21:42:51.114099Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_1_3714194436971602743_v1
2025-10-09T21:42:51.114216Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340746896313532:2464] disconnected.
2025-10-09T21:42:51.114227Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340746896313532:2464] disconnected; active server actors: 1
2025-10-09T21:42:51.114230Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340746896313532:2464] client user disconnected session shared/user_1_1_3714194436971602743_v1
2025-10-09T21:42:51.665877Z :FallbackToSingleDbAfterBadRequest INFO: Random seed for debugging is 1760046171665871
2025-10-09T21:42:51.775208Z node 3 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/owmb/000352/r3tmp/tmpLpUEO7/pdisk_1.dat
2025-10-09T21:42:51.782615Z node 3 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:51.782661Z node 3 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
2025-10-09T21:42:51.777037Z node 4 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:51.782683Z node 4 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:51.782711Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
2025-10-09T21:42:51.859697Z node 4 :CONFIGS_DISPATCHER ERROR: configs_dispatcher.cpp:1229: Notification cookie mismatch for subscription [4:7559340745734672063:2073] 1760046171769453 != 1760046171769456
2025-10-09T21:42:51.857546Z node 3 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:42:51.863082Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:51.863106Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:51.873590Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:51.873620Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:51.874880Z node 3 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 4 Cookie 4
2025-10-09T21:42:51.874911Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:51.878476Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Connecting -> Connected
TServer::EnableGrpc on GrpcPort 63308, node 3
2025-10-09T21:42:51.901375Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/owmb/000352/r3tmp/yandexK0M50E.tmp
2025-10-09T21:42:51.901388Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/owmb/000352/r3tmp/yandexK0M50E.tmp
2025-10-09T21:42:51.901442Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/owmb/000352/r3tmp/yandexK0M50E.tmp
2025-10-09T21:42:51.901492Z node 3 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-10-09T21:42:51.904806Z INFO: TTestServer started on Port 14626 GrpcPort 63308
TClient is connected to server localhost:14626
PQClient connected to localhost:63308
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-09T21:42:51.921494Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976710657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
waiting...
waiting...
waiting...
2025-10-09T21:42:52.014592Z node 3 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:52.068830Z node 4 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:52.257859Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340749045651150:2325], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.257884Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.258087Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340749045651186:2329], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.258097Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.258169Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340749045651185:2328], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.261162Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976710661:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-10-09T21:42:52.261716Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340749045651193:2332], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.261733Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.261862Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340749045651224:2333], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.261870Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.268899Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976710662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:52.276177Z node 3 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [3:7559340749045651192:2331], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976710661 completed, doublechecking }
2025-10-09T21:42:52.281854Z node 4 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [4:7559340750029639742:2301], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:52.283170Z node 4 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=4&id=Y2Q1NGI0Zi03MGI3NjM5My04OGNhNGUxMS01N2U1ZGZi, ActorId: [4:7559340750029639702:2295], ActorState: ExecuteState, TraceId: 01k75etj3ebeghtrp6hgdhm9ks, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:52.283314Z node 4 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:52.328458Z node 3 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [3:7559340749045651330:2706] txid# 281474976710663, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:52.333659Z node 3 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [3:7559340749045651349:2342], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:52.334349Z node 3 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=3&id=OTJmNDk1NmItNTcxODQwNTctMzVkODljMzQtZDM4ZDhlNDU=, ActorId: [3:7559340749045651145:2321], ActorState: ExecuteState, TraceId: 01k75etj30f54n91e64b0jehny, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:52.334495Z node 3 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:52.346555Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976710664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:52.427988Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976710665:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:63308", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, false, 1000);
2025-10-09T21:42:52.514846Z node 3 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976710666. Ctx: { TraceId: 01k75etj9zdq74he3e4vavqyz8, Database: , SessionId: ydb://session/3?node_id=3&id=YzJhNGYwN2EtYjFhN2ZiY2ItZjllNzU0NWItOWUwYjIyOGU=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [3:7559340749045651662:2948] 
2025-10-09T21:42:52.757515Z node 3 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:52.765471Z node 4 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
=== CheckClustersList. Ok
2025-10-09T21:42:57.639314Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976710672:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
waiting...
PQ Client: create topic: rt3.dc1--test-topic with 1 partitions
CallPersQueueGRPC request to localhost:63308
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:57.675599Z node 3 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:63308
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976710673
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-09T21:42:57.698565Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:57.700492Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:57.700561Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:784: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-09T21:42:57.700569Z node 4 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:57.700585Z node 4 :PERSQUEUE INFO: pq_impl.cpp:609: [PQ: 72075186224037892] doesn't have tx writes info
2025-10-09T21:42:57.700861Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340770520488567:3220], now have 1 active actors on pipe
2025-10-09T21:42:57.701305Z node 3 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][] pipe [3:7559340770520488568:3221] connected; active server actors: 1
2025-10-09T21:42:57.701370Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1517: [72075186224037893][rt3.dc1--test-topic] updating configuration. Deleted partitions []. Added partitions [0]
2025-10-09T21:42:57.701751Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.703685Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.702789Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:940: [72075186224037893][rt3.dc1--test-topic] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-09T21:42:57.702826Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:138: [72075186224037893][rt3.dc1--test-topic] BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) 
2025-10-09T21:42:57.702901Z node 3 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976710674. Ctx: { TraceId: 01k75etqcmcp32bhxtw54ag8t1, Database: , SessionId: ydb://session/3?node_id=3&id=ZjIzNzAyY2UtOGZlNmFjYTktMWIzMDMwNWQtMTQ3ZGU5ODc=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-10-09T21:42:57.704344Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 4, Generation 1
2025-10-09T21:42:57.704432Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340770520488598:2440], now have 1 active actors on pipe
2025-10-09T21:42:57.708709Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340770520488630:3265], now have 1 active actors on pipe
2025-10-09T21:42:57.709319Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
GetTopicVersionFromPath:  record Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
 name rt3.dc1--test-topic version0
2025-10-09T21:42:57.710991Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.711179Z node 4 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:57.711194Z node 4 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:57.711260Z node 4 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:57.711308Z node 4 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [4:7559340771504476705:2381]
2025-10-09T21:42:57.712020Z node 4 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:57.712024Z node 4 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:57.712030Z node 4 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [4:7559340771504476705:2381]
2025-10-09T21:42:57.712037Z node 4 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:57.712339Z node 4 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-10-09T21:42:57.712346Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.712364Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.712367Z node 4 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:57.712371Z node 4 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:57.712373Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.712376Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.712378Z node 4 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:57.712384Z node 4 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:57.712465Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.712490Z node 4 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:57.714693Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.714738Z node 4 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046177757, TxId 281474976710673
2025-10-09T21:42:57.714745Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.714747Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.714777Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.714833Z node 4 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:57.715827Z node 4 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:57.715897Z node 4 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:57.715906Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.715911Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.715912Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.715919Z node 4 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:57.716034Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:395: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-10-09T21:42:57.716054Z node 4 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:57.716115Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.720013Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.720045Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.720949Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.812612Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.812642Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.812646Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.912931Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.912948Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.912952Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.013299Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.013319Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.013322Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.113616Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.113632Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.113636Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976710673
    CreateStep: 1760046177757
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976710673
    CreateStep: 1760046177757
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:63308
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:58.213986Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.214010Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.214013Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.215332Z node 3 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
2025-10-09T21:42:58.216817Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-09T21:42:58.217065Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-09T21:42:58.217070Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:63308
2025-10-09T21:42:58.217421Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:58.217734Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:58.217753Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1
2025-10-09T21:42:58.217921Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:58.217961Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:33208
2025-10-09T21:42:58.217974Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:33208 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:58.217979Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:58.218543Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-09T21:42:58.218582Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:58.218584Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:58.218586Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:58.218592Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [3:7559340774815455999:2450] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:58.219196Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [3:7559340774815455999:2450] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:58.248242Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [3:7559340774815455999:2450] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:58.248455Z node 3 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340774815456033:2450] connected; active server actors: 1
2025-10-09T21:42:58.248479Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [3:7559340774815455999:2450] (SourceId=src, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src
2025-10-09T21:42:58.248483Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [3:7559340774815455999:2450] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:58.248573Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340774815456033:2450] disconnected.
2025-10-09T21:42:58.248578Z node 3 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340774815456033:2450] disconnected; active server actors: 1
2025-10-09T21:42:58.248581Z node 3 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340774815456033:2450] disconnected no session
2025-10-09T21:42:58.269834Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [3:7559340774815455999:2450] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:58.269850Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [3:7559340774815455999:2450] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:42:58.269853Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [3:7559340774815455999:2450] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:58.269863Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:58.270393Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340774815456057:2450], now have 1 active actors on pipe
2025-10-09T21:42:58.271934Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.271949Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.271993Z node 4 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src|59bb96df-cc379f15-e20a326c-e4006823_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src
2025-10-09T21:42:58.272011Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.272018Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.272022Z node 4 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:58.272036Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.272065Z node 4 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:58.272092Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:58.272532Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.272554Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.272584Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:58.273260Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046178273 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:58.273299Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|59bb96df-cc379f15-e20a326c-e4006823_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:58.273368Z :INFO: [] MessageGroupId [src] SessionId [src|59bb96df-cc379f15-e20a326c-e4006823_0] Write session: close. Timeout = 0 ms
2025-10-09T21:42:58.273374Z :INFO: [] MessageGroupId [src] SessionId [src|59bb96df-cc379f15-e20a326c-e4006823_0] Write session will now close
2025-10-09T21:42:58.273380Z :DEBUG: [] MessageGroupId [src] SessionId [src|59bb96df-cc379f15-e20a326c-e4006823_0] Write session: aborting
2025-10-09T21:42:58.273664Z :INFO: [] MessageGroupId [src] SessionId [src|59bb96df-cc379f15-e20a326c-e4006823_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:42:58.271706Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 4, Generation: 1
2025-10-09T21:42:58.272941Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1 partition: 0 MaxSeqNo: 0 sessionId: src|59bb96df-cc379f15-e20a326c-e4006823_0
2025-10-09T21:42:58.273752Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId: src|59bb96df-cc379f15-e20a326c-e4006823_0 grpc read done: success: 0 data: 
2025-10-09T21:42:58.273756Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 1 sessionId: src|59bb96df-cc379f15-e20a326c-e4006823_0 grpc read failed
2025-10-09T21:42:58.273762Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 1 sessionId: src|59bb96df-cc379f15-e20a326c-e4006823_0 grpc closed
2025-10-09T21:42:58.273768Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1 sessionId: src|59bb96df-cc379f15-e20a326c-e4006823_0 is DEAD
2025-10-09T21:42:58.274059Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:42:58.274147Z :DEBUG: [] MessageGroupId [src] SessionId [src|59bb96df-cc379f15-e20a326c-e4006823_0] Write session is aborting and will not restart
2025-10-09T21:42:58.274278Z :DEBUG: [] MessageGroupId [src] SessionId [src|59bb96df-cc379f15-e20a326c-e4006823_0] Write session: destroy
2025-10-09T21:42:58.274917Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [3:7559340774815456057:2450] destroyed
2025-10-09T21:42:58.274937Z node 4 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:42:58.274945Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.274948Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.274953Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
PORTS 63308 15918
Session was created
2025-10-09T21:42:58.314569Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.314586Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.314589Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.418266Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.418282Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.418285Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.522123Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.522151Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.522154Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.622931Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.622947Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.622950Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.722929Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.722945Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.722948Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.824389Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.824405Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.824408Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.924619Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.924636Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.924640Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.024982Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.025001Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.025004Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.125348Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.125382Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.125385Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.225703Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.225731Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.225734Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: ok
2025-10-09T21:42:59.282924Z :INFO: [/Root] OnFederationDiscovery fall back to single mode, database=/Root
2025-10-09T21:42:59.282953Z :INFO: [/Root] [] [c0e3645d-afc30a37-7d827ce3-538eb01] Open read subsessions to databases: { name: , endpoint: localhost:15918, path: /Root }
2025-10-09T21:42:59.283002Z :INFO: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] Starting read session
2025-10-09T21:42:59.283013Z :DEBUG: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] Starting single session
2025-10-09T21:42:59.283245Z :DEBUG: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:59.283250Z :DEBUG: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:59.283255Z :DEBUG: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] [] Reconnecting session to cluster  in 0.000000s
2025-10-09T21:42:59.283313Z :ERROR: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] [] Got error. Status: CLIENT_CALL_UNIMPLEMENTED. Description: <main>: Error: GRpc error: (12):  <main>: Error: Grpc error response on endpoint localhost:15918 <main>: Error: Endpoint list is empty for database /Root, cluster endpoint localhost:15918. 
2025-10-09T21:42:59.283321Z :DEBUG: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:59.283323Z :DEBUG: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:59.283360Z :INFO: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] [] Closing session to cluster: SessionClosed { Status: CLIENT_CALL_UNIMPLEMENTED Issues: "<main>: Error: Failed to establish connection to server "localhost:15918" ( cluster ). Attempts done: 1     <main>: Error: GRpc error: (12):      <main>: Error: Grpc error response on endpoint localhost:15918     <main>: Error: Endpoint list is empty for database /Root, cluster endpoint localhost:15918. " }
2025-10-09T21:42:59.283456Z :NOTICE: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-09T21:42:59.283470Z :DEBUG: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] [] Abort session to cluster
Got new read session event: SessionClosed { Status: CLIENT_CALL_UNIMPLEMENTED Issues: "<main>: Error: Failed to establish connection to server "localhost:15918" ( cluster ). Attempts done: 1     <main>: Error: GRpc error: (12):      <main>: Error: Grpc error response on endpoint localhost:15918     <main>: Error: Endpoint list is empty for database /Root, cluster endpoint localhost:15918. " }
2025-10-09T21:42:59.283492Z :INFO: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] Closing read session. Close timeout: 0.010000s
2025-10-09T21:42:59.283501Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset):
2025-10-09T21:42:59.283508Z :INFO: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] Counters: { Errors: 1 CurrentSessionLifetimeMs: 0 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:59.283514Z :INFO: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] Closing read session. Close timeout: 0.000000s
2025-10-09T21:42:59.283517Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset):
2025-10-09T21:42:59.283519Z :INFO: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] Counters: { Errors: 1 CurrentSessionLifetimeMs: 0 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:59.283524Z :INFO: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] Closing read session. Close timeout: 0.000000s
2025-10-09T21:42:59.283529Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset):
2025-10-09T21:42:59.283531Z :INFO: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] Counters: { Errors: 1 CurrentSessionLifetimeMs: 0 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:59.283535Z :NOTICE: [/Root] [/Root] [b0684e3c-3fc644e1-4e34a569-56103b84] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-09T21:42:59.330281Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.330299Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.330303Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:44.658396Z :GetAllStartPartitionSessions INFO: Random seed for debugging is 1760046164658391
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/owmb/00035a/r3tmp/tmpNjlLfo/pdisk_1.dat
2025-10-09T21:42:44.779897Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:44.780752Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:44.846068Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:44.846132Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
2025-10-09T21:42:44.852166Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:44.852203Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
2025-10-09T21:42:44.877958Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:44.877984Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:44.887392Z node 2 :CONFIGS_DISPATCHER ERROR: configs_dispatcher.cpp:1229: Notification cookie mismatch for subscription [2:7559340717327569059:2073] 1760046164759834 != 1760046164759837
2025-10-09T21:42:44.887676Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:44.887699Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:44.888075Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:42:44.895524Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:44.895675Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-10-09T21:42:44.898316Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
TServer::EnableGrpc on GrpcPort 31892, node 1
2025-10-09T21:42:44.922212Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/owmb/00035a/r3tmp/yandexEuzDsL.tmp
2025-10-09T21:42:44.922231Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/owmb/00035a/r3tmp/yandexEuzDsL.tmp
2025-10-09T21:42:44.922288Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/owmb/00035a/r3tmp/yandexEuzDsL.tmp
2025-10-09T21:42:44.922328Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-10-09T21:42:44.925690Z INFO: TTestServer started on Port 24772 GrpcPort 31892
TClient is connected to server localhost:24772
PQClient connected to localhost:31892
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-09T21:42:44.953555Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
waiting...
waiting...
waiting...
2025-10-09T21:42:45.146084Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:45.153041Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:45.204108Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7559340721622536700:2296], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.204149Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.204393Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7559340721622536735:2299], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.204825Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7559340721622536737:2300], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.204837Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.205656Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7559340721622536762:2302], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.205666Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.206194Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976710657:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-10-09T21:42:45.226835Z node 2 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [2:7559340721622536738:2301], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976710657 completed, doublechecking }
2025-10-09T21:42:45.282769Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [1:7559340718901224858:2330], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:45.284856Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=1&id=MWExNmM5YWItYTI2YTY1YWQtNWRjNWFiZS00ODdmZmFjZQ==, ActorId: [1:7559340718901224793:2320], ActorState: ExecuteState, TraceId: 01k75etb6t8m7h2vfgtzspdysd, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:45.285394Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:45.286714Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720661:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:45.311375Z node 2 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [2:7559340721622536788:2151] txid# 281474976710658, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:45.321518Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [2:7559340721622536804:2308], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:45.322131Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=2&id=MzM3YzI2NWEtMzAyNzAzMGItYTdhZWM4ZjktYTE4Y2FkZmU=, ActorId: [2:7559340721622536698:2295], ActorState: ExecuteState, TraceId: 01k75etb6ka9xv4zgvx1tx90bc, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:45.322254Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:45.359250Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:45.408394Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:31892", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, false, 1000);
2025-10-09T21:42:45.502927Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720664. Ctx: { TraceId: 01k75etbf4273nkyt4eysxdrgv, Database: , SessionId: ydb://session/3?node_id=1&id=MThhZTQ5ZDUtZjdjMGIwNy1kYTQ0NjdjOC05ZjFhNGMzZQ==, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [1:7559340718901225278:2926] 
2025-10-09T21:42:45.764708Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:45.766097Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
=== CheckClustersList. Ok
2025-10-09T21:42:50.674951Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720670:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
waiting...
PQ Client: create topic: rt3.dc1--test-topic with 5 partitions
CallPersQueueGRPC request to localhost:31892
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:50.728942Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:31892
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 5
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720671
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-09T21:42:50.771705Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720672. Ctx: { TraceId: 01k75etgkse51kce8r7n3a8kbf, Database: , SessionId: ydb://session/3?node_id=1&id=YTM1Y2NiODItNjQ3Y2Q5ZmYtZGI0YjU0ZTQtMTYwMDhjMjc=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-10-09T21:42:50.772249Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][] pipe [1:7559340740376062191:3205] connected; active server actors: 1
2025-10-09T21:42:50.772313Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1517: [72075186224037893][rt3.dc1--test-topic] updating configuration. Deleted partitions []. Added partitions [1, 2, 3, 4, 0]
2025-10-09T21:42:50.779439Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:940: [72075186224037893][rt3.dc1--test-topic] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-09T21:42:50.779514Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:138: [72075186224037893][rt3.dc1--test-topic] BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) (1, 72075186224037892) (2, 72075186224037892) (3, 72075186224037892) (4, 72075186224037892) 
2025-10-09T21:42:50.789637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
2025-10-09T21:42:50.796685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:50.796782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:784: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-09T21:42:50.796792Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:50.796805Z node 2 :PERSQUEUE INFO: pq_impl.cpp:609: [PQ: 72075186224037892] doesn't have tx writes info
2025-10-09T21:42:50.797040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340740376062190:3204], now have 1 active actors on pipe
2025-10-09T21:42:50.797046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340740376062224:2432], now have 1 active actors on pipe
2025-10-09T21:42:50.798221Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-10-09T21:42:50.799116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.801463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
TClient::Ls response: Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
GetTopicVersionFromPath:  record Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
 name rt3.dc1--test-topic version0
2025-10-09T21:42:50.803719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340740376062265:3254], now have 1 active actors on pipe
2025-10-09T21:42:50.804046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.804965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.805848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:1:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:50.805858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:2:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:50.805863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:3:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:50.805866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:4:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:50.805870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:50.805891Z node 2 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:50.805894Z node 2 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][2][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:50.805896Z node 2 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][3][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:50.805899Z node 2 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][1][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:50.805902Z node 2 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][4][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:50.806005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:1:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:50.806108Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][1][StateInit] bootstrapping 1 [2:7559340743097373733:2385]
2025-10-09T21:42:50.806802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:1:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:50.806806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:1:Initializer] Initializing completed.
2025-10-09T21:42:50.806811Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][1][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 1 generation 1 [2:7559340743097373733:2385]
2025-10-09T21:42:50.806817Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][1][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 1 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:50.807108Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][1][StateIdle] Process pending events. Count 1
2025-10-09T21:42:50.807114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.807144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.807147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][1][StateIdle] Batch completed (1)
2025-10-09T21:42:50.807150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][1][StateIdle] Still pending for come commits
2025-10-09T21:42:50.807152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.807153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.807154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][1][StateIdle] Still pending for come commits
2025-10-09T21:42:50.807159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][1][StateIdle] Topic 'rt3.dc1--test-topic' partition 1 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:50.807193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:2:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:50.807227Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][2][StateInit] bootstrapping 2 [2:7559340743097373734:2385]
2025-10-09T21:42:50.807628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:2:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:50.807630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:2:Initializer] Initializing completed.
2025-10-09T21:42:50.807633Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][2][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 2 generation 1 [2:7559340743097373734:2385]
2025-10-09T21:42:50.807636Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][2][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 2 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:50.807837Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][2][StateIdle] Process pending events. Count 1
2025-10-09T21:42:50.807840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.807848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.807849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][2][StateIdle] Batch completed (1)
2025-10-09T21:42:50.807851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][2][StateIdle] Still pending for come commits
2025-10-09T21:42:50.807854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.807855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.807856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][2][StateIdle] Still pending for come commits
2025-10-09T21:42:50.807859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][2][StateIdle] Topic 'rt3.dc1--test-topic' partition 2 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:50.807876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:3:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:50.807890Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][3][StateInit] bootstrapping 3 [2:7559340743097373735:2385]
2025-10-09T21:42:50.808299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:3:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:50.808301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:3:Initializer] Initializing completed.
2025-10-09T21:42:50.808303Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][3][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 3 generation 1 [2:7559340743097373735:2385]
2025-10-09T21:42:50.808306Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][3][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 3 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:50.808539Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][3][StateIdle] Process pending events. Count 1
2025-10-09T21:42:50.808541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.808549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.808550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][3][StateIdle] Batch completed (1)
2025-10-09T21:42:50.808553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][3][StateIdle] Still pending for come commits
2025-10-09T21:42:50.808555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.808557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.808558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][3][StateIdle] Still pending for come commits
2025-10-09T21:42:50.808560Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][3][StateIdle] Topic 'rt3.dc1--test-topic' partition 3 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:50.808579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:4:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:50.808606Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][4][StateInit] bootstrapping 4 [2:7559340743097373736:2385]
2025-10-09T21:42:50.808993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:4:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:50.808994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:4:Initializer] Initializing completed.
2025-10-09T21:42:50.808996Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][4][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 4 generation 1 [2:7559340743097373736:2385]
2025-10-09T21:42:50.808999Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][4][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 4 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:50.809205Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][4][StateIdle] Process pending events. Count 1
2025-10-09T21:42:50.809207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.809215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.809216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][4][StateIdle] Batch completed (1)
2025-10-09T21:42:50.809218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][4][StateIdle] Still pending for come commits
2025-10-09T21:42:50.809220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.809221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.809223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][4][StateIdle] Still pending for come commits
2025-10-09T21:42:50.809226Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][4][StateIdle] Topic 'rt3.dc1--test-topic' partition 4 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:50.809244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:50.809277Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340743097373737:2385]
2025-10-09T21:42:50.809670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:50.809673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:50.809676Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [2:7559340743097373737:2385]
2025-10-09T21:42:50.809680Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:50.809895Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-10-09T21:42:50.809898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.809908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.809910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:50.809912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:50.809913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.809915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.809916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:50.809919Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:50.809966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][1][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.810007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][2][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.811158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][3][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.811196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][4][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.811244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.811319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.812926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.812960Z node 2 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046170848, TxId 281474976720671
2025-10-09T21:42:50.812964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.812965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.812988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.813022Z node 2 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][1][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046170848, TxId 281474976720671
2025-10-09T21:42:50.813024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.813025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.813036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:50.813049Z node 2 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][2][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046170848, TxId 281474976720671
2025-10-09T21:42:50.813052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.813053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.813063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:50.813075Z node 2 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][3][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046170848, TxId 281474976720671
2025-10-09T21:42:50.813077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.813078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.813088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:50.813100Z node 2 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][4][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046170848, TxId 281474976720671
2025-10-09T21:42:50.813103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.813104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.813114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:50.813129Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:50.813137Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:50.813140Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:50.813142Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:50.813145Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:50.814608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:50.814655Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:50.814662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.814663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.814665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.814669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.814673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][1][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:50.814692Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][1][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:50.814694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.814695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.814696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:50.814697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][1][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.814700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][2][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:50.814722Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][2][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:50.814723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.814724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.814725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:50.814727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][2][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.814730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][3][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:50.814749Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][3][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:50.814750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.814751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.814752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:50.814753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][3][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.814756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][4][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:50.818090Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][4][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-10-09T21:42:50.818138Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][4][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:50.818144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.818146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.818148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:50.818152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][4][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.818257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:395: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 5 SourceIdMaxCounts: 6000000 } PartitionIds: 1 PartitionIds: 2 PartitionIds: 3 PartitionIds: 4 PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 1 Status: Active CreateVersion: 1 TabletId: 0 } Partitions { PartitionId: 2 Status: Active CreateVersion: 1 TabletId: 0 } Partitions { PartitionId: 3 Status: Active CreateVersion: 1 TabletId: 0 } Partitions { PartitionId: 4 Status: Active CreateVersion: 1 TabletId: 0 } Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 1 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 2 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 3 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 4 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-10-09T21:42:50.818284Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:50.818335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.822016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.822050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.826408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.910638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.910651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.910654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:50.910672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.910674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.910676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:50.910683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.910684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.910687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:50.910705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.910707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.910710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:50.910717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.910718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.910721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.011490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.011508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.011512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:51.011537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.011539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.011542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:51.011549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.011552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.011554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:51.011560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.011562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.011563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:51.011570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.011573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.011575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.111792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.111813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.111817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:51.111836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.111839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.111840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:51.111847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.111849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.111851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:51.111858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.111860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.111862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:51.111868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.111870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.111879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.215569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.215586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.215589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:51.215616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.215619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.215624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:51.215633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.215635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.215636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:51.215644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.215647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.215649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:51.215656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.215659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.215660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720671
    CreateStep: 1760046170848
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 5
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720671
    CreateStep: 1760046170848
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 5
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    Partitions {
      PartitionId: 1
      TabletId: 72075186224037892
      Status: Active
    }
    Partitions {
      PartitionId: 2
      TabletId: 72075186224037892
      Status: Active
    }
    Partitions {
      PartitionId: 3
      TabletId: 72075186224037892
      Status: Active
    }
    Partitions {
      PartitionId: 4
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 5
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 5
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:31892
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:51.314351Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 5
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
2025-10-09T21:42:51.314924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.314942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.314945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:51.314972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.314974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.314976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:51.314983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.314985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.314988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:51.314995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.314999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.315001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:51.315007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.315008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.315012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.319199Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-09T21:42:51.319444Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-09T21:42:51.319451Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:31892
2025-10-09T21:42:51.321832Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:51.322452Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:51.322468Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1
2025-10-09T21:42:51.322841Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:51.322872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:39056
2025-10-09T21:42:51.322878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:39056 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:51.322882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:51.323417Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-09T21:42:51.323465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:51.323467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:51.323469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:51.323476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340744671029623:2445] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:51.324066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340744671029623:2445] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:51.401058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [1:7559340744671029623:2445] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:51.401390Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340744671029658:2445] connected; active server actors: 1
2025-10-09T21:42:51.401453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [1:7559340744671029623:2445] (SourceId=src, PreferedPartition=(NULL)) Received partition 1 from PQRB for SourceId=src
2025-10-09T21:42:51.401459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340744671029623:2445] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:51.404953Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340744671029658:2445] disconnected.
2025-10-09T21:42:51.404968Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340744671029658:2445] disconnected; active server actors: 1
2025-10-09T21:42:51.404972Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340744671029658:2445] disconnected no session
2025-10-09T21:42:51.416790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.416806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.416809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:51.416834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.416837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.416839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:51.416847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.416850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.416852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:51.416858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.416860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.416861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:51.416868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.416870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.416872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.450689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340744671029623:2445] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:51.450705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340744671029623:2445] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=1, SeqNo=(NULL)
2025-10-09T21:42:51.450709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340744671029623:2445] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:51.450719Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1 sessionId:  partition: 1 expectedGeneration: (NULL)
2025-10-09T21:42:51.464516Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=1) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1
2025-10-09T21:42:51.464433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340744671029680:2445], now have 1 active actors on pipe
2025-10-09T21:42:51.464727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.464747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 1
2025-10-09T21:42:51.464786Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src|17753798-371b80dc-d32a752-120632b7_0 generated for partition 1 topic 'rt3.dc1--test-topic' owner src
2025-10-09T21:42:51.464801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.464807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.464811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][1][StateIdle] Batch completed (1)
2025-10-09T21:42:51.464830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:51.464850Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][1][StateIdle] TPartition::ReplyOwnerOk. Partition: 1
2025-10-09T21:42:51.464874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 1 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.470292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.470315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 1
2025-10-09T21:42:51.470367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 1 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.474248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1 partition: 1 MaxSeqNo: 0 sessionId: src|17753798-371b80dc-d32a752-120632b7_0
2025-10-09T21:42:51.474664Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046171474 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:51.474704Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|17753798-371b80dc-d32a752-120632b7_0" topic: "test-topic" cluster: "dc1" partition_id: 1 supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:51.474830Z :INFO: [] MessageGroupId [src] SessionId [src|17753798-371b80dc-d32a752-120632b7_0] Write session: close. Timeout = 0 ms
2025-10-09T21:42:51.474835Z :INFO: [] MessageGroupId [src] SessionId [src|17753798-371b80dc-d32a752-120632b7_0] Write session will now close
2025-10-09T21:42:51.474841Z :DEBUG: [] MessageGroupId [src] SessionId [src|17753798-371b80dc-d32a752-120632b7_0] Write session: aborting
2025-10-09T21:42:51.474941Z :INFO: [] MessageGroupId [src] SessionId [src|17753798-371b80dc-d32a752-120632b7_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:42:51.474947Z :DEBUG: [] MessageGroupId [src] SessionId [src|17753798-371b80dc-d32a752-120632b7_0] Write session: destroy
2025-10-09T21:42:51.475264Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId: src|17753798-371b80dc-d32a752-120632b7_0 grpc read done: success: 0 data: 
2025-10-09T21:42:51.475271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 1 sessionId: src|17753798-371b80dc-d32a752-120632b7_0 grpc read failed
2025-10-09T21:42:51.475278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 1 sessionId: src|17753798-371b80dc-d32a752-120632b7_0 grpc closed
2025-10-09T21:42:51.475281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1 sessionId: src|17753798-371b80dc-d32a752-120632b7_0 is DEAD
2025-10-09T21:42:51.475467Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=1) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:42:51.475760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340744671029680:2445] destroyed
2025-10-09T21:42:51.475785Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][1][StateIdle] TPartition::DropOwner.
2025-10-09T21:42:51.475795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.475798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.475800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
Session was created
2025-10-09T21:42:51.516753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.516765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.516769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:51.517177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.517185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.517187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:51.517205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.517207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.517210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:51.517220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.517223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.517225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:51.517232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.517234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.517240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.617020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.617035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.617048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:51.617494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.617497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.617500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:51.617509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.617512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.617514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:51.617521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.617526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.617528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:51.617537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.617538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.617540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.717517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.717534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.717537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:51.718050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.718061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.718063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:51.718071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.718072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.718073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:51.718078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.718080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.718084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:51.718090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.718091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.718092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.817774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.817789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.817792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:51.818255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.818266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.818268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:51.818289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.818292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.818294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:51.818301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.818303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.818305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:51.818311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.818313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.818314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.918161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.918176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.918179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:51.918599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.918603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.918606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:51.918615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.918616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.918625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:51.918634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.918635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.918637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:51.918643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.918645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.918646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.019403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.019422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.019425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:52.019447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.019449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.019451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:52.019458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.019460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.019463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:52.019470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.019472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.019475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:52.019481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.019494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.019496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.121703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.121722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.121724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:52.121746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.121749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.121750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:52.121757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.121758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.121761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:52.121768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.121770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.121772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:52.121778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.121781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.121783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.223065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.223088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.223092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:52.223122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.223123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.223125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:52.223132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.223135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.223136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:52.223145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.223147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.223149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:52.223155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.223156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.223158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.325975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.325988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.325990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:52.326010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.326012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.326014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:52.326018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.326020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.326022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:52.326041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.326045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.326047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:52.326055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.326058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.326060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.430096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.430116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.430119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:52.430145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.430149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.430151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:52.430159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.430162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.430167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:52.430175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.430177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.430179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:52.430186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.430189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.430190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: ok
2025-10-09T21:42:52.493450Z :INFO: [/Root] [] [fadbcdb4-eee7f22d-4ca69ecf-2483ed52] Open read subsessions to databases: { name: dc1, endpoint: localhost:31892, path: /Root }, { name: dc2, endpoint: localhost:31892, path: /Root }
2025-10-09T21:42:52.499820Z :INFO: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] Starting read session
2025-10-09T21:42:52.499834Z :DEBUG: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] Starting single session
2025-10-09T21:42:52.500078Z :DEBUG: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:52.500084Z :DEBUG: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:52.500089Z :DEBUG: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] [] Reconnecting session to cluster  in 0.000000s
2025-10-09T21:42:52.500842Z :INFO: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] Starting read session
2025-10-09T21:42:52.500849Z :DEBUG: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] Starting single session
2025-10-09T21:42:52.500868Z :DEBUG: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:52.500872Z :DEBUG: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:52.500875Z :DEBUG: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] [] Reconnecting session to cluster  in 0.000000s
2025-10-09T21:42:52.530469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.530485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.530488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:52.530510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.530512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.530515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:52.530520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.530523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.530525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:52.530533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.530535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.530537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:52.530555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.530557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.530560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.634442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.634457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.634462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:52.634486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.634488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.634490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:52.634496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.634497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.634499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:52.634505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.634507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.634508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:52.634513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.634514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.634525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.730853Z :DEBUG: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] [] Successfully connected. Initializing session
2025-10-09T21:42:52.730950Z :DEBUG: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] [] Successfully connected. Initializing session
2025-10-09T21:42:52.731315Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:42:52.731332Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 1
2025-10-09T21:42:52.731361Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:42:52.731363Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 2
2025-10-09T21:42:52.733219Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:42:52.733297Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 read init: from# ipv6:[::1]:42768, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:42:52.733367Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:42:52.733382Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 read init: from# ipv6:[::1]:42768, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:42:52.733407Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 auth for : user
2025-10-09T21:42:52.733410Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 auth for : user
2025-10-09T21:42:52.734262Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 Handle describe topics response
2025-10-09T21:42:52.734293Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 auth is DEAD
2025-10-09T21:42:52.734311Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:42:52.734819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.734836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.734838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:52.734863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.734864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.734866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:52.734872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.734873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.734874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:52.734880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.734881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.734882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:52.734888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.734890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.734892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.734632Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:42:52.734695Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 Handle describe topics response
2025-10-09T21:42:52.734709Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 auth is DEAD
2025-10-09T21:42:52.734721Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:42:52.734889Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:42:52.736113Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340748965997054:2473] connected; active server actors: 1
2025-10-09T21:42:52.736153Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559340748965997054:2473] session shared/user_1_1_4612699217477481448_v1
2025-10-09T21:42:52.736169Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:42:52.736181Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:42:52.736188Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 1
2025-10-09T21:42:52.736202Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=2 (Status=Free, Partitions=[1])
2025-10-09T21:42:52.736205Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 2
2025-10-09T21:42:52.736210Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=3 (Status=Free, Partitions=[2])
2025-10-09T21:42:52.736212Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 3
2025-10-09T21:42:52.736215Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=4 (Status=Free, Partitions=[3])
2025-10-09T21:42:52.736217Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 4
2025-10-09T21:42:52.736220Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=5 (Status=Free, Partitions=[4])
2025-10-09T21:42:52.736226Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_1_4612699217477481448_v1" (Sender=[1:7559340748965997049:2473], Pipe=[1:7559340748965997054:2473], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:42:52.736231Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:42:52.736237Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340748965997056:2474] connected; active server actors: 1
2025-10-09T21:42:52.736246Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=5, UnradableFamilies=5 [5 (4), 4 (3), 3 (2), 2 (1), 1 (0), ], RequireBalancing=0 []
2025-10-09T21:42:52.736255Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=5 (Status=Free, Partitions=[4]) for ReadingSession "shared/user_1_1_4612699217477481448_v1" (Sender=[1:7559340748965997049:2473], Pipe=[1:7559340748965997054:2473], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:42:52.736265Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 5 status Active partitions [4] session "shared/user_1_1_4612699217477481448_v1" sender [1:7559340748965997049:2473] lock partition 4 for ReadingSession "shared/user_1_1_4612699217477481448_v1" (Sender=[1:7559340748965997049:2473], Pipe=[1:7559340748965997054:2473], Partitions=[], ActiveFamilyCount=1) generation 1 step 1
2025-10-09T21:42:52.736274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=4 (Status=Free, Partitions=[3]) for ReadingSession "shared/user_1_1_4612699217477481448_v1" (Sender=[1:7559340748965997049:2473], Pipe=[1:7559340748965997054:2473], Partitions=[], ActiveFamilyCount=1)
2025-10-09T21:42:52.736279Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 4 status Active partitions [3] session "shared/user_1_1_4612699217477481448_v1" sender [1:7559340748965997049:2473] lock partition 3 for ReadingSession "shared/user_1_1_4612699217477481448_v1" (Sender=[1:7559340748965997049:2473], Pipe=[1:7559340748965997054:2473], Partitions=[], ActiveFamilyCount=2) generation 1 step 2
2025-10-09T21:42:52.736285Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=3 (Status=Free, Partitions=[2]) for ReadingSession "shared/user_1_1_4612699217477481448_v1" (Sender=[1:7559340748965997049:2473], Pipe=[1:7559340748965997054:2473], Partitions=[], ActiveFamilyCount=2)
2025-10-09T21:42:52.736290Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 3 status Active partitions [2] session "shared/user_1_1_4612699217477481448_v1" sender [1:7559340748965997049:2473] lock partition 2 for ReadingSession "shared/user_1_1_4612699217477481448_v1" (Sender=[1:7559340748965997049:2473], Pipe=[1:7559340748965997054:2473], Partitions=[], ActiveFamilyCount=3) generation 1 step 3
2025-10-09T21:42:52.736295Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=2 (Status=Free, Partitions=[1]) for ReadingSession "shared/user_1_1_4612699217477481448_v1" (Sender=[1:7559340748965997049:2473], Pipe=[1:7559340748965997054:2473], Partitions=[], ActiveFamilyCount=3)
2025-10-09T21:42:52.736299Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 2 status Active partitions [1] session "shared/user_1_1_4612699217477481448_v1" sender [1:7559340748965997049:2473] lock partition 1 for ReadingSession "shared/user_1_1_4612699217477481448_v1" (Sender=[1:7559340748965997049:2473], Pipe=[1:7559340748965997054:2473], Partitions=[], ActiveFamilyCount=4) generation 1 step 4
2025-10-09T21:42:52.736305Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_1_4612699217477481448_v1" (Sender=[1:7559340748965997049:2473], Pipe=[1:7559340748965997054:2473], Partitions=[], ActiveFamilyCount=4)
2025-10-09T21:42:52.736309Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_1_4612699217477481448_v1" sender [1:7559340748965997049:2473] lock partition 0 for ReadingSession "shared/user_1_1_4612699217477481448_v1" (Sender=[1:7559340748965997049:2473], Pipe=[1:7559340748965997054:2473], Partitions=[], ActiveFamilyCount=5) generation 1 step 5
2025-10-09T21:42:52.736314Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=5, sessionCount=1, desiredFamilyCount=5, allowPlusOne=0
2025-10-09T21:42:52.736319Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000070s
2025-10-09T21:42:52.736611Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 assign: record# { Partition: 4 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 1 Session: "shared/user_1_1_4612699217477481448_v1" ClientId: "user" PipeClient { RawX1: 7559340748965997054 RawX2: 4503603922340265 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:42:52.736637Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 assign: record# { Partition: 3 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 2 Session: "shared/user_1_1_4612699217477481448_v1" ClientId: "user" PipeClient { RawX1: 7559340748965997054 RawX2: 4503603922340265 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:42:52.736655Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 assign: record# { Partition: 2 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 3 Session: "shared/user_1_1_4612699217477481448_v1" ClientId: "user" PipeClient { RawX1: 7559340748965997054 RawX2: 4503603922340265 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:42:52.736678Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 assign: record# { Partition: 1 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 4 Session: "shared/user_1_1_4612699217477481448_v1" ClientId: "user" PipeClient { RawX1: 7559340748965997054 RawX2: 4503603922340265 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:42:52.736700Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 5 Session: "shared/user_1_1_4612699217477481448_v1" ClientId: "user" PipeClient { RawX1: 7559340748965997054 RawX2: 4503603922340265 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:42:52.736721Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 1(assignId:4)
2025-10-09T21:42:52.736742Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:5)
2025-10-09T21:42:52.736754Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 4(assignId:1)
2025-10-09T21:42:52.736762Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 3(assignId:2)
2025-10-09T21:42:52.736770Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 2(assignId:3)
2025-10-09T21:42:52.737591Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559340748965997056:2474] session shared/user_1_2_12090568940580064917_v1
2025-10-09T21:42:52.737606Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_2_12090568940580064917_v1" (Sender=[1:7559340748965997050:2474], Pipe=[1:7559340748965997056:2474], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:42:52.737620Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:42:52.737629Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=2, Families=5, UnradableFamilies=0 [], RequireBalancing=0 []
2025-10-09T21:42:52.737634Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=5, sessionCount=2, desiredFamilyCount=2, allowPlusOne=1
2025-10-09T21:42:52.737643Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:175: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_1_4612699217477481448_v1" sender [1:7559340748965997049:2473]  release partitions [0]. Target status Free
2025-10-09T21:42:52.737655Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:175: [72075186224037893][rt3.dc1--test-topic] consumer user family 2 status Active partitions [1] session "shared/user_1_1_4612699217477481448_v1" sender [1:7559340748965997049:2473]  release partitions [1]. Target status Free
2025-10-09T21:42:52.737659Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000028s
2025-10-09T21:42:52.738118Z :INFO: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] [] Got InitResponse. ReadSessionId: shared/user_1_1_4612699217477481448_v1
2025-10-09T21:42:52.738130Z :DEBUG: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] [] In ContinueReadingDataImpl, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:52.738219Z :DEBUG: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:42:52.738244Z :INFO: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] [] Got InitResponse. ReadSessionId: shared/user_1_2_12090568940580064917_v1
2025-10-09T21:42:52.738247Z :DEBUG: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] [] In ContinueReadingDataImpl, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:52.738277Z :DEBUG: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:42:52.739217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340748965997066:2479], now have 1 active actors on pipe
2025-10-09T21:42:52.739233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340748965997067:2480], now have 1 active actors on pipe
2025-10-09T21:42:52.739236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340748965997068:2481], now have 1 active actors on pipe
2025-10-09T21:42:52.738115Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1576: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 gone release: partition# 0
2025-10-09T21:42:52.738128Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1561: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 releasing: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:5)
2025-10-09T21:42:52.738134Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1721: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 got all from client, actual releasing: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:5)
2025-10-09T21:42:52.738138Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1639: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 released: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:5)
2025-10-09T21:42:52.738153Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1576: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 gone release: partition# 1
2025-10-09T21:42:52.738166Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1561: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 releasing: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 1(assignId:4)
2025-10-09T21:42:52.738169Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1721: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 got all from client, actual releasing: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 1(assignId:4)
2025-10-09T21:42:52.738172Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1639: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 released: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 1(assignId:4)
2025-10-09T21:42:52.738476Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 grpc read done: success# 1, data# { read_request { bytes_size: 524288 } }
2025-10-09T21:42:52.738516Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 got read request: guid# 5a671688-e2bf47f3-a4bc91a1-1af4ca12
2025-10-09T21:42:52.738530Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 grpc read done: success# 1, data# { read_request { bytes_size: 524288 } }
2025-10-09T21:42:52.738538Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 got read request: guid# c448a434-22e96eb9-914558de-aeb80c71
2025-10-09T21:42:52.739396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.739413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 4
2025-10-09T21:42:52.739426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_1_4612699217477481448_v1 on pipe: [1:7559340748965997066:2479]
2025-10-09T21:42:52.739443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.739447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 3
2025-10-09T21:42:52.739451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_1_4612699217477481448_v1 on pipe: [1:7559340748965997067:2480]
2025-10-09T21:42:52.739463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.739465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 2
2025-10-09T21:42:52.739469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_1_4612699217477481448_v1 on pipe: [1:7559340748965997068:2481]
2025-10-09T21:42:52.739483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.739490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.739492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][4][StateIdle] Batch completed (1)
2025-10-09T21:42:52.739511Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][4][StateIdle] Topic 'rt3.dc1--test-topic' partition 4 user user session is set to 0 (startOffset 0) session shared/user_1_1_4612699217477481448_v1
2025-10-09T21:42:52.739515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:52.739546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.739549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.739551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][3][StateIdle] Batch completed (1)
2025-10-09T21:42:52.739559Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][3][StateIdle] Topic 'rt3.dc1--test-topic' partition 3 user user session is set to 0 (startOffset 0) session shared/user_1_1_4612699217477481448_v1
2025-10-09T21:42:52.739561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:52.739574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.739576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.739578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][2][StateIdle] Batch completed (1)
2025-10-09T21:42:52.739586Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][2][StateIdle] Topic 'rt3.dc1--test-topic' partition 2 user user session is set to 0 (startOffset 0) session shared/user_1_1_4612699217477481448_v1
2025-10-09T21:42:52.739588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:52.739599Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:52.739608Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:52.739612Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:52.739857Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_1_4612699217477481448_v1:1 with generation 1
2025-10-09T21:42:52.739864Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_1_4612699217477481448_v1:2 with generation 1
2025-10-09T21:42:52.739868Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_1_4612699217477481448_v1:3 with generation 1
2025-10-09T21:42:52.738693Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1606: [72075186224037893][rt3.dc1--test-topic] client user released partition from pipe [1:7559340748965997054:2473] session shared/user_1_1_4612699217477481448_v1 partition 0
2025-10-09T21:42:52.738706Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:42:52.738717Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1606: [72075186224037893][rt3.dc1--test-topic] client user released partition from pipe [1:7559340748965997054:2473] session shared/user_1_1_4612699217477481448_v1 partition 1
2025-10-09T21:42:52.738728Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=2, Families=5, UnradableFamilies=2 [2 (1), 1 (0), ], RequireBalancing=0 []
2025-10-09T21:42:52.738741Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=2 (Status=Free, Partitions=[1]) for ReadingSession "shared/user_1_2_12090568940580064917_v1" (Sender=[1:7559340748965997050:2474], Pipe=[1:7559340748965997056:2474], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:42:52.738752Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 2 status Active partitions [1] session "shared/user_1_2_12090568940580064917_v1" sender [1:7559340748965997050:2474] lock partition 1 for ReadingSession "shared/user_1_2_12090568940580064917_v1" (Sender=[1:7559340748965997050:2474], Pipe=[1:7559340748965997056:2474], Partitions=[], ActiveFamilyCount=1) generation 1 step 6
2025-10-09T21:42:52.738763Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_2_12090568940580064917_v1" (Sender=[1:7559340748965997050:2474], Pipe=[1:7559340748965997056:2474], Partitions=[], ActiveFamilyCount=1)
2025-10-09T21:42:52.738768Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_2_12090568940580064917_v1" sender [1:7559340748965997050:2474] lock partition 0 for ReadingSession "shared/user_1_2_12090568940580064917_v1" (Sender=[1:7559340748965997050:2474], Pipe=[1:7559340748965997056:2474], Partitions=[], ActiveFamilyCount=2) generation 1 step 7
2025-10-09T21:42:52.738773Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=5, sessionCount=2, desiredFamilyCount=2, allowPlusOne=1
2025-10-09T21:42:52.738778Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000047s
2025-10-09T21:42:52.738996Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 assign: record# { Partition: 1 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 6 Session: "shared/user_1_2_12090568940580064917_v1" ClientId: "user" PipeClient { RawX1: 7559340748965997056 RawX2: 4503603922340266 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:42:52.739018Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 7 Session: "shared/user_1_2_12090568940580064917_v1" ClientId: "user" PipeClient { RawX1: 7559340748965997056 RawX2: 4503603922340266 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:42:52.739032Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 1(assignId:1)
2025-10-09T21:42:52.739054Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:2)
2025-10-09T21:42:52.739188Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 4(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1, pipe: [1:7559340748965997066:2479]
2025-10-09T21:42:52.739200Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 3(assignId:2) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1, pipe: [1:7559340748965997067:2480]
2025-10-09T21:42:52.739212Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 2(assignId:3) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1, pipe: [1:7559340748965997068:2481]
2025-10-09T21:42:52.740996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340748965997071:2484], now have 1 active actors on pipe
2025-10-09T21:42:52.741006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340748965997072:2485], now have 1 active actors on pipe
2025-10-09T21:42:52.741087Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 1(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1, pipe: [1:7559340748965997071:2484]
2025-10-09T21:42:52.741102Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:2) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1, pipe: [1:7559340748965997072:2485]
2025-10-09T21:42:52.741268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.741277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 1
2025-10-09T21:42:52.741285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_2_12090568940580064917_v1 on pipe: [1:7559340748965997071:2484]
2025-10-09T21:42:52.741296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.741298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.741302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_2_12090568940580064917_v1 on pipe: [1:7559340748965997072:2485]
2025-10-09T21:42:52.741312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.741318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.741320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][1][StateIdle] Batch completed (1)
2025-10-09T21:42:52.741333Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][1][StateIdle] Topic 'rt3.dc1--test-topic' partition 1 user user session is set to 0 (startOffset 0) session shared/user_1_2_12090568940580064917_v1
2025-10-09T21:42:52.741335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:52.741357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.741360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.741361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:52.741367Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_2_12090568940580064917_v1
2025-10-09T21:42:52.741369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.741378Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:52.741383Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:52.741524Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_2_12090568940580064917_v1:1 with generation 1
2025-10-09T21:42:52.741530Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_2_12090568940580064917_v1:2 with generation 1
2025-10-09T21:42:52.744497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][4][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:52.744529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][4][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:52.744555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.744558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.744560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:52.744567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][4][StateIdle] No data for blobs compaction
2025-10-09T21:42:52.744572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][3][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:52.744603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][3][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:52.744606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.744608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.744609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:52.744612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][3][StateIdle] No data for blobs compaction
2025-10-09T21:42:52.744615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][2][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:52.744632Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][2][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:52.744635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.744637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.744638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:52.744641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][2][StateIdle] No data for blobs compaction
2025-10-09T21:42:52.744645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][1][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:52.744663Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][1][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:52.744665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.744668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.744669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:52.744671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][1][StateIdle] No data for blobs compaction
2025-10-09T21:42:52.744674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:52.744691Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:52.744695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.744697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.744699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.744702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:52.744713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 4 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:42:52.744726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 3 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:42:52.744732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 2 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:42:52.744737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 1 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:42:52.744742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:42:52.745865Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 4(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 0 SizeLag: 0 WriteTimestampEstimateMS: 1760046172734 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:42:52.745883Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 4(assignId:1) EndOffset 0 readOffset 0 committedOffset 0
2025-10-09T21:42:52.745899Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 3(assignId:2) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 0 SizeLag: 0 WriteTimestampEstimateMS: 1760046172734 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:42:52.745904Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 3(assignId:2) EndOffset 0 readOffset 0 committedOffset 0
2025-10-09T21:42:52.745916Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 2(assignId:3) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 0 SizeLag: 0 WriteTimestampEstimateMS: 1760046172734 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:42:52.745920Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 2(assignId:3) EndOffset 0 readOffset 0 committedOffset 0
2025-10-09T21:42:52.745932Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 1(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 0 SizeLag: 0 WriteTimestampEstimateMS: 1760046172734 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:42:52.745936Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 1(assignId:1) EndOffset 0 readOffset 0 committedOffset 0
2025-10-09T21:42:52.745948Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:2) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 0 SizeLag: 0 WriteTimestampEstimateMS: 1760046172734 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:42:52.745952Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:2) EndOffset 0 readOffset 0 committedOffset 0
2025-10-09T21:42:52.745976Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 sending to client partition status
2025-10-09T21:42:52.746113Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 sending to client partition status
2025-10-09T21:42:52.746120Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 sending to client partition status
2025-10-09T21:42:52.746143Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 sending to client partition status
2025-10-09T21:42:52.746183Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 sending to client partition status
Got new read session event: StartPartitionSession { Partition session id: 1 Topic: "test-topic" Partition: 4 Database name: dc1 Database path: /Root Database id: account-dc1 CommittedOffset: 0 EndOffset: 0 }
2025-10-09T21:42:52.746682Z :INFO: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] [] Confirm partition stream create. Partition stream id: 1. Cluster: "-". Topic: "test-topic". Partition: 4. Read offset: (NULL)
Got new read session event: StartPartitionSession { Partition session id: 1 Topic: "test-topic" Partition: 1 Database name: dc2 Database path: /Root Database id: account-dc2 CommittedOffset: 0 EndOffset: 0 }
2025-10-09T21:42:52.746850Z :INFO: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] [] Confirm partition stream create. Partition stream id: 1. Cluster: "-". Topic: "test-topic". Partition: 1. Read offset: (NULL)
Got new read session event: StartPartitionSession { Partition session id: 2 Topic: "test-topic" Partition: 3 Database name: dc1 Database path: /Root Database id: account-dc1 CommittedOffset: 0 EndOffset: 0 }
2025-10-09T21:42:52.746899Z :INFO: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] [] Confirm partition stream create. Partition stream id: 2. Cluster: "-". Topic: "test-topic". Partition: 3. Read offset: (NULL)
Got new read session event: StartPartitionSession { Partition session id: 2 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 CommittedOffset: 0 EndOffset: 0 }
2025-10-09T21:42:52.746913Z :INFO: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] [] Confirm partition stream create. Partition stream id: 2. Cluster: "-". Topic: "test-topic". Partition: 0. Read offset: (NULL)
Got new read session event: StartPartitionSession { Partition session id: 3 Topic: "test-topic" Partition: 2 Database name: dc1 Database path: /Root Database id: account-dc1 CommittedOffset: 0 EndOffset: 0 }
2025-10-09T21:42:52.746927Z :INFO: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] [] Confirm partition stream create. Partition stream id: 3. Cluster: "-". Topic: "test-topic". Partition: 2. Read offset: (NULL)
2025-10-09T21:42:52.746933Z :INFO: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] Closing read session. Close timeout: 0.010000s
2025-10-09T21:42:52.746945Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:2:3:0:0 -:test-topic:3:2:0:0 -:test-topic:4:1:0:0
2025-10-09T21:42:52.746955Z :INFO: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] Counters: { Errors: 0 CurrentSessionLifetimeMs: 247 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:52.748668Z :INFO: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] Closing read session. Close timeout: 0.010000s
2025-10-09T21:42:52.748689Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:0:2:0:0 -:test-topic:1:1:0:0
2025-10-09T21:42:52.748698Z :INFO: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] Counters: { Errors: 0 CurrentSessionLifetimeMs: 247 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:52.748843Z :INFO: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] Closing read session. Close timeout: 0.000000s
2025-10-09T21:42:52.748852Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:2:3:0:0 -:test-topic:3:2:0:0 -:test-topic:4:1:0:0
2025-10-09T21:42:52.748858Z :INFO: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] Counters: { Errors: 0 CurrentSessionLifetimeMs: 249 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:52.748864Z :INFO: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] Closing read session. Close timeout: 0.000000s
2025-10-09T21:42:52.748868Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:0:2:0:0 -:test-topic:1:1:0:0
2025-10-09T21:42:52.748871Z :INFO: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] Counters: { Errors: 0 CurrentSessionLifetimeMs: 248 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:52.748876Z :INFO: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] Closing read session. Close timeout: 0.000000s
2025-10-09T21:42:52.748881Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:0:2:0:0 -:test-topic:1:1:0:0
2025-10-09T21:42:52.748885Z :INFO: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] Counters: { Errors: 0 CurrentSessionLifetimeMs: 248 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:52.748897Z :NOTICE: [/Root] [/Root] [9614f5ff-61c9b648-f2c2ff36-6af87cda] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-09T21:42:52.748924Z :INFO: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] Closing read session. Close timeout: 0.000000s
2025-10-09T21:42:52.748928Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:2:3:0:0 -:test-topic:3:2:0:0 -:test-topic:4:1:0:0
2025-10-09T21:42:52.748931Z :INFO: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] Counters: { Errors: 0 CurrentSessionLifetimeMs: 249 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:52.748936Z :NOTICE: [/Root] [/Root] [38b2a656-a2a49ffb-17f0814-d143728e] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-09T21:42:52.749207Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 grpc read done: success# 1, data# { start_partition_session_response { partition_session_id: 1 } }
2025-10-09T21:42:52.749244Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:92: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 grpc closed
2025-10-09T21:42:52.749256Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 1 consumer shared/user session shared/user_1_1_4612699217477481448_v1 is DEAD
2025-10-09T21:42:52.749465Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 grpc read done: success# 1, data# { start_partition_session_response { partition_session_id: 1 } }
2025-10-09T21:42:52.749478Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:92: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 grpc closed
2025-10-09T21:42:52.749485Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 2 consumer shared/user session shared/user_1_2_12090568940580064917_v1 is DEAD
2025-10-09T21:42:52.750373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_1_4612699217477481448_v1
2025-10-09T21:42:52.750391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340748965997068:2481] destroyed
2025-10-09T21:42:52.750396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_1_4612699217477481448_v1
2025-10-09T21:42:52.750398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340748965997067:2480] destroyed
2025-10-09T21:42:52.750400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_1_4612699217477481448_v1
2025-10-09T21:42:52.750401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340748965997066:2479] destroyed
2025-10-09T21:42:52.750403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_2_12090568940580064917_v1
2025-10-09T21:42:52.750405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340748965997072:2485] destroyed
2025-10-09T21:42:52.750407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_2_12090568940580064917_v1
2025-10-09T21:42:52.750409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340748965997071:2484] destroyed
2025-10-09T21:42:52.750432Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_1_4612699217477481448_v1
2025-10-09T21:42:52.750435Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_1_4612699217477481448_v1
2025-10-09T21:42:52.750437Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_1_4612699217477481448_v1
2025-10-09T21:42:52.750438Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_2_12090568940580064917_v1
2025-10-09T21:42:52.750439Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_2_12090568940580064917_v1
2025-10-09T21:42:52.750544Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340748965997054:2473] disconnected.
2025-10-09T21:42:52.750552Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340748965997054:2473] disconnected; active server actors: 1
2025-10-09T21:42:52.750555Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340748965997054:2473] client user disconnected session shared/user_1_1_4612699217477481448_v1
2025-10-09T21:42:52.750570Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:42:52.750574Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340748965997056:2474] disconnected.
2025-10-09T21:42:52.750575Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340748965997056:2474] disconnected; active server actors: 1
2025-10-09T21:42:52.750577Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340748965997056:2474] client user disconnected session shared/user_1_2_12090568940580064917_v1
2025-10-09T21:42:52.835225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.835238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.835241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:52.835262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.835264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.835266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:52.835274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.835276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.835278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:52.835285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.835287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.835289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:52.835296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.835297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.835299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.938229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][1][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.938252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][1][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.938255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][1][StateIdle] Try persist
2025-10-09T21:42:52.938281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][2][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.938284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][2][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.938285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][2][StateIdle] Try persist
2025-10-09T21:42:52.938293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][3][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.938296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][3][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.938299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][3][StateIdle] Try persist
2025-10-09T21:42:52.938309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][4][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.938311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][4][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.938314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][4][StateIdle] Try persist
2025-10-09T21:42:52.938321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.938324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.938326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:53.464264Z :PreferredDatabaseNoFallback INFO: Random seed for debugging is 1760046173464258
2025-10-09T21:42:53.557485Z node 4 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[4:7559340755623115683:2143];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:53.557505Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-10-09T21:42:53.559894Z node 3 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/owmb/00035a/r3tmp/tmpFnAyHJ/pdisk_1.dat
2025-10-09T21:42:53.563345Z node 3 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:53.563371Z node 3 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
2025-10-09T21:42:53.561674Z node 4 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:53.562011Z node 4 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:53.594172Z node 3 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
TServer::EnableGrpc on GrpcPort 24185, node 3
2025-10-09T21:42:53.611019Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/owmb/00035a/r3tmp/yandexbGV7fQ.tmp
2025-10-09T21:42:53.611034Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/owmb/00035a/r3tmp/yandexbGV7fQ.tmp
2025-10-09T21:42:53.611082Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/owmb/00035a/r3tmp/yandexbGV7fQ.tmp
2025-10-09T21:42:53.611109Z node 3 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-10-09T21:42:53.612711Z INFO: TTestServer started on Port 8865 GrpcPort 24185
TClient is connected to server localhost:8865
PQClient connected to localhost:24185
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-09T21:42:53.656062Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:53.656095Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:53.661310Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:53.661342Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:53.662019Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:53.662551Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
2025-10-09T21:42:53.663200Z node 3 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 4 Cookie 4
2025-10-09T21:42:53.666231Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Connecting -> Connected
waiting...
waiting...
waiting...
waiting...
2025-10-09T21:42:53.803101Z node 4 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:53.843906Z node 3 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:53.976130Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340756501205821:2325], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:53.976158Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:53.976352Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340756501205856:2328], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:53.976364Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340756501205857:2329], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:53.976370Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:53.977223Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976720661:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-10-09T21:42:53.983805Z node 3 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [3:7559340756501205860:2330], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720661 completed, doublechecking }
2025-10-09T21:42:53.986065Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:54.018521Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:54.058612Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:54.063447Z node 4 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [4:7559340759918083302:2305], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:5:17: Error: At function: KiReadTable!
        <main>:5:17: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Versions]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:54.063749Z node 4 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=4&id=NTE1OGEyZi1kNzk2NGZhOC0zMDBhNmU4MC0zNDUzMGMwZA==, ActorId: [4:7559340759918083262:2298], ActorState: ExecuteState, TraceId: 01k75etkv791vdcc810z3fh9qb, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:54.063888Z node 4 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:54.070928Z node 3 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [3:7559340760796173526:2873] txid# 281474976720665, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:54.077122Z node 3 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [3:7559340760796173541:2359], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:5:17: Error: At function: KiReadTable!
        <main>:5:17: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Versions]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:54.077855Z node 3 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=3&id=YzA1ZDM3NjktODFlMzMzMi0yNjEyOTk5ZC01ZjllODJkMA==, ActorId: [3:7559340756501205817:2322], ActorState: ExecuteState, TraceId: 01k75etkrp9zm6crstjvn5kxa9, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:54.077990Z node 3 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:24185", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, false, 1000);
2025-10-09T21:42:54.127836Z node 3 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720666. Ctx: { TraceId: 01k75etkwq9xv2accr9rqmpk00, Database: , SessionId: ydb://session/3?node_id=3&id=YmJiZTNhMTgtNDMxNDgyNC1jNzg4YmUwYy04YWRkYTVkYg==, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [3:7559340760796173643:2954] 
2025-10-09T21:42:54.557534Z node 3 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:54.561047Z node 4 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:58.557751Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[4:7559340755623115683:2143];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:58.557800Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
2025-10-09T21:42:59.422596Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720672:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
waiting...
PQ Client: create topic: rt3.dc1--test-topic with 1 partitions
CallPersQueueGRPC request to localhost:24185
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:59.468893Z node 3 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:24185
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720673
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-09T21:42:59.486175Z node 3 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][] pipe [3:7559340782271010538:3220] connected; active server actors: 1
2025-10-09T21:42:59.486235Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1517: [72075186224037893][rt3.dc1--test-topic] updating configuration. Deleted partitions []. Added partitions [0]
2025-10-09T21:42:59.486650Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:940: [72075186224037893][rt3.dc1--test-topic] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-09T21:42:59.486914Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:138: [72075186224037893][rt3.dc1--test-topic] BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) 
2025-10-09T21:42:59.489109Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:59.490059Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:59.490124Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:784: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-09T21:42:59.490131Z node 4 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:59.490143Z node 4 :PERSQUEUE INFO: pq_impl.cpp:609: [PQ: 72075186224037892] doesn't have tx writes info
2025-10-09T21:42:59.490311Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340782271010571:2434], now have 1 active actors on pipe
2025-10-09T21:42:59.490318Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340782271010537:3219], now have 1 active actors on pipe
2025-10-09T21:42:59.490607Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 4, Generation 1
2025-10-09T21:42:59.490821Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:59.498150Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:59.500864Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340782271010594:3265], now have 1 active actors on pipe
2025-10-09T21:42:59.501350Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:59.502691Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:59.502915Z node 4 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:59.502927Z node 4 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:59.502993Z node 4 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:59.503027Z node 4 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [4:7559340781392920198:2382]
2025-10-09T21:42:59.503732Z node 4 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:59.503737Z node 4 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:59.503743Z node 4 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [4:7559340781392920198:2382]
2025-10-09T21:42:59.503750Z node 4 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:59.504082Z node 4 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-10-09T21:42:59.504089Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.504109Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.504111Z node 4 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:59.504116Z node 4 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:59.504118Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.504120Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.504121Z node 4 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:59.504129Z node 4 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:59.504214Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:59.504242Z node 4 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:59.506216Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:59.506256Z node 4 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046179549, TxId 281474976720673
2025-10-09T21:42:59.506261Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.506263Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.506289Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.506328Z node 4 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:59.507396Z node 4 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:59.510787Z node 3 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720674. Ctx: { TraceId: 01k75ets4m9dj1jsrs59xa7px6, Database: , SessionId: ydb://session/3?node_id=3&id=NTBiMzhjMC0yNzY3NThiLTQ4ZjUzOTIwLWYyZGQ1MjEw, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-10-09T21:42:59.513587Z node 4 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-10-09T21:42:59.513666Z node 4 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:59.513673Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.513676Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.513678Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.513687Z node 4 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:59.513820Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:395: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-10-09T21:42:59.513842Z node 4 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:59.513900Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:59.517610Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:59.517636Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:59.518479Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046179549
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046179549
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:24185
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:59.526862Z node 3 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
2025-10-09T21:42:59.528113Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-09T21:42:59.528357Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-09T21:42:59.528364Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:24185
2025-10-09T21:42:59.528800Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:59.529052Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:59.529063Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1
2025-10-09T21:42:59.529476Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:59.529499Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:55436
2025-10-09T21:42:59.529503Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:55436 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:59.529506Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:59.529853Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-09T21:42:59.529879Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:59.529880Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:59.529881Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:59.529885Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [3:7559340782271010662:2448] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:59.530402Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [3:7559340782271010662:2448] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:59.560855Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [3:7559340782271010662:2448] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:59.561183Z node 3 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340782271010696:2448] connected; active server actors: 1
2025-10-09T21:42:59.561244Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [3:7559340782271010662:2448] (SourceId=src, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src
2025-10-09T21:42:59.561248Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [3:7559340782271010662:2448] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:59.562831Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340782271010696:2448] disconnected.
2025-10-09T21:42:59.562842Z node 3 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340782271010696:2448] disconnected; active server actors: 1
2025-10-09T21:42:59.562846Z node 3 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340782271010696:2448] disconnected no session
2025-10-09T21:42:59.582673Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [3:7559340782271010662:2448] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:59.582687Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [3:7559340782271010662:2448] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:42:59.582690Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [3:7559340782271010662:2448] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:59.582698Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:59.583182Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340782271010721:2448], now have 1 active actors on pipe
2025-10-09T21:42:59.583460Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 4, Generation: 1
2025-10-09T21:42:59.583676Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:59.583695Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:59.583755Z node 4 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src|9786fa0f-c209814b-f5fac7a4-7827c1fc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src
2025-10-09T21:42:59.583768Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.583782Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.583789Z node 4 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:59.583805Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.583828Z node 4 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:59.583853Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:59.585144Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:59.585161Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:59.585190Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:59.585343Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1 partition: 0 MaxSeqNo: 0 sessionId: src|9786fa0f-c209814b-f5fac7a4-7827c1fc_0
2025-10-09T21:42:59.585664Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046179585 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:59.585700Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|9786fa0f-c209814b-f5fac7a4-7827c1fc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:59.585793Z :INFO: [] MessageGroupId [src] SessionId [src|9786fa0f-c209814b-f5fac7a4-7827c1fc_0] Write session: close. Timeout = 0 ms
2025-10-09T21:42:59.585799Z :INFO: [] MessageGroupId [src] SessionId [src|9786fa0f-c209814b-f5fac7a4-7827c1fc_0] Write session will now close
2025-10-09T21:42:59.585804Z :DEBUG: [] MessageGroupId [src] SessionId [src|9786fa0f-c209814b-f5fac7a4-7827c1fc_0] Write session: aborting
2025-10-09T21:42:59.586105Z :INFO: [] MessageGroupId [src] SessionId [src|9786fa0f-c209814b-f5fac7a4-7827c1fc_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:42:59.586124Z :DEBUG: [] MessageGroupId [src] SessionId [src|9786fa0f-c209814b-f5fac7a4-7827c1fc_0] Write session is aborting and will not restart
2025-10-09T21:42:59.586139Z :DEBUG: [] MessageGroupId [src] SessionId [src|9786fa0f-c209814b-f5fac7a4-7827c1fc_0] Write session: destroy
2025-10-09T21:42:59.586097Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId: src|9786fa0f-c209814b-f5fac7a4-7827c1fc_0 grpc read done: success: 0 data: 
2025-10-09T21:42:59.586105Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 1 sessionId: src|9786fa0f-c209814b-f5fac7a4-7827c1fc_0 grpc read failed
2025-10-09T21:42:59.586112Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 1 sessionId: src|9786fa0f-c209814b-f5fac7a4-7827c1fc_0 grpc closed
2025-10-09T21:42:59.586116Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1 sessionId: src|9786fa0f-c209814b-f5fac7a4-7827c1fc_0 is DEAD
2025-10-09T21:42:59.586292Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:42:59.586525Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [3:7559340782271010721:2448] destroyed
2025-10-09T21:42:59.586542Z node 4 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:42:59.586551Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.586553Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.586555Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.604297Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.604312Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.604315Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
====TYdbPqTestRetryPolicy()
====ExpectBreakDown
=== Session was created, waiting for retries
>>> Ready to answer: ok
====CreateRetryState
====CreateRetryState Initialized
Test retry state: get retry delay
2025-10-09T21:42:59.658587Z :NOTICE: [/Root] [] [] Retry to update federation state in 2.000000s
2025-10-09T21:42:59.704686Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.704704Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.704707Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.805416Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.805436Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.805440Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.905846Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.905863Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.905875Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.012451Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.012469Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.012473Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.111121Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.111152Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.111156Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.211513Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.211531Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.211534Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.311875Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.311892Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.311895Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.412228Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.412261Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.412263Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.512551Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.512566Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.512569Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.612900Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.612924Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.612927Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.713279Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.713294Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.713298Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.813636Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.813655Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.813658Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.913956Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.913974Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.913978Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.018093Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.018113Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.018117Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.126067Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.126089Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.126092Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.226308Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.226337Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.226341Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.328002Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.328021Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.328024Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.427436Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.427455Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.427458Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.530176Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.530194Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.530198Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.630869Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.630884Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.630888Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Test retry state: get retry delay
2025-10-09T21:43:01.666181Z :NOTICE: [/Root] [] [] Retry to update federation state in 2.000000s
2025-10-09T21:43:01.734105Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.734126Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.734129Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.834053Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.834072Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.834076Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.934394Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.934424Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.934427Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.034728Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.034745Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.034748Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.135116Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.135140Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.135143Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.235518Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.235551Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.235554Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.338109Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.338130Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.338133Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.438835Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.438853Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.438856Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.539042Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.539055Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.539057Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.639459Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.639477Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.639481Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.740045Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.740060Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.740063Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.840620Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.840634Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.840637Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.940979Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.940997Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.941000Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.042087Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.042104Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.042108Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.142176Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.142190Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.142195Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.242632Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.242649Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.242653Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.343370Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.343386Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.343391Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.444188Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.444202Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.444205Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
E1009 21:43:03.457843798 3631915 chttp2_transport.cc:1143]             ipv4:127.0.0.1:19109: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 1250ms
2025-10-09T21:43:03.544720Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.544734Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.544738Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.645195Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.645208Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.645210Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Test retry state: get retry delay
2025-10-09T21:43:03.668719Z :NOTICE: [/Root] [] [] Retry to update federation state in 2.000000s
=== In the next federation discovery response dc2 will be available
2025-10-09T21:43:03.749390Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.749411Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.749414Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.849830Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.849846Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.849848Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.950730Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.950741Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.950744Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.052563Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.052579Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.052582Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.153268Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.153283Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.153286Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.253194Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.253209Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.253212Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.358469Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.358484Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.358487Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.456464Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.456478Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.456482Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.506156Z node 4 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:04.556833Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.556848Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.556852Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
E1009 21:43:04.602974935 3628006 chttp2_transport.cc:1143]             ipv6:%5B::1%5D:19109: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 1250ms
2025-10-09T21:43:04.657157Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.657184Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.657186Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.757518Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.757545Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.757548Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.857895Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.857913Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.857916Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.959560Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.959578Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.959581Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.059657Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.059675Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.059691Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.159980Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.160005Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.160012Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.260328Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.260347Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.260350Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.360666Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.360682Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.360685Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.461045Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.461062Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.461065Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.561350Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.561366Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.561368Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.661690Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.661716Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.661719Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Test retry state: get retry delay
2025-10-09T21:43:05.669235Z :NOTICE: [/Root] [] [] Retry to update federation state in 2.000000s
2025-10-09T21:43:05.762020Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.762052Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.762055Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.865563Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.865580Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.865583Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.965951Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.965970Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.965974Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.066390Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.066405Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.066409Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.166566Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.166593Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.166596Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.266938Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.266957Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.266961Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.370102Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.370124Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.370127Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.470174Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.470192Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.470196Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.570549Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.570569Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.570572Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.670892Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.670910Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.670913Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.771253Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.771270Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.771274Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.871577Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.871593Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.871596Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.971925Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.971952Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.971956Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.072290Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.072308Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.072311Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.172751Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.172771Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.172781Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.272976Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.273000Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.273004Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.373315Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.373334Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.373338Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.473723Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.473745Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.473749Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.574046Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.574060Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.574063Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Test retry state: get retry delay
2025-10-09T21:43:07.670228Z :NOTICE: [/Root] [] [] Retry to update federation state in 2.000000s
2025-10-09T21:43:07.674814Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.674830Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.674833Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.775316Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.775331Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.775335Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.878098Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.878118Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.878121Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.978571Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.978588Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.978592Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.078844Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.078859Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.078863Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.179221Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.179237Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.179241Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.283500Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.283517Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.283520Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.383550Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.383566Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.383569Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.486217Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.486234Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.486237Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.577987Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard_impl.cpp:7721: Cannot get console configs
2025-10-09T21:43:08.578001Z node 3 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:43:08.587800Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.587813Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.587823Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.690246Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.690263Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.690266Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.793541Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.793556Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.793559Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.893472Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.893491Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.893494Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.993848Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.993865Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.993868Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.094232Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.094247Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.094250Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.196150Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.196167Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.196170Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.302119Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.302136Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.302140Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.403488Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.403504Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.403507Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.503854Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.503870Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.503873Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.505860Z node 4 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:09.604607Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.604627Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.604631Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Test retry state: get retry delay
2025-10-09T21:43:09.677760Z :NOTICE: [/Root] [] [] Retry to update federation state in 2.000000s
2025-10-09T21:43:09.704990Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.705004Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.705008Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.805393Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.805409Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.805412Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.905659Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.905674Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.905676Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.006198Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.006217Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.006220Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.108430Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.108446Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.108448Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.208020Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.208122Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.208127Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.308434Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.308454Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.308457Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.414137Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.414156Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.414159Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.514173Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.514191Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.514195Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.614346Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.614364Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.614367Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.718104Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.718123Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.718126Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.818175Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.818193Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.818196Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.918874Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.918899Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.918903Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.022549Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.022564Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.022568Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.122741Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.122760Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.122774Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.224251Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.224274Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.224277Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.326129Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.326147Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.326150Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.426358Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.426373Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.426376Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.526728Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.526744Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.526748Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.627056Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.627072Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.627075Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Test retry state: get retry delay
2025-10-09T21:43:11.678245Z :NOTICE: [/Root] [] [] Retry to update federation state in 2.000000s
2025-10-09T21:43:11.728040Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.728058Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.728060Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.828414Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.828428Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.828431Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.928627Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.928643Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.928647Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.028973Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.028989Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.028992Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.129298Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.129309Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.129311Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.229856Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.229871Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.229874Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.330123Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.330140Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.330143Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.430529Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.430545Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.430548Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.531003Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.531019Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.531021Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.631065Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.631081Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.631084Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.731460Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.731477Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.731480Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.831768Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.831790Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.831793Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.932126Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.932143Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.932147Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.032490Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.032506Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.032509Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.132806Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.132824Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.132827Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.233193Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.233209Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.233212Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.334258Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.334273Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.334276Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.436784Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.436799Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.436802Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.541147Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.541162Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.541165Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.641244Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.641260Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.641270Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Test retry state: get retry delay
2025-10-09T21:43:13.682154Z :NOTICE: [/Root] [] [] Retry to update federation state in 2.000000s
2025-10-09T21:43:13.741808Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.741824Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.741827Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.842303Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.842319Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.842322Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.942832Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.942851Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.942854Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.043152Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.043168Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.043170Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.143544Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.143561Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.143573Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.243897Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.243916Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.243919Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.344249Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.344266Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.344269Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.444561Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.444572Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.444574Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.506809Z node 4 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:14.544873Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.544886Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.544890Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.645540Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.645562Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.645565Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.745645Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.745660Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.745663Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.846002Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.846017Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.846020Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.946294Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.946311Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.946313Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.046703Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.046719Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.046722Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.147052Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.147069Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.147073Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.247871Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.247889Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.247892Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.348265Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.348281Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.348284Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.450079Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.450098Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.450101Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.552705Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.552720Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.552722Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.652883Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.652912Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.652915Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Test retry state: get retry delay
2025-10-09T21:43:15.686118Z :NOTICE: [/Root] [] [] Retry to update federation state in 2.000000s
2025-10-09T21:43:15.753468Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.753503Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.753507Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.853619Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.853647Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.853650Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.953924Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.953941Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.953944Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.054514Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.054535Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.054539Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.158076Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.158099Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.158103Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.258114Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.258130Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.258134Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.358541Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.358557Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.358561Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.458769Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.458788Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.458806Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.559363Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.559379Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.559383Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.659524Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.659548Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.659551Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.760710Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.760728Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.760731Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.861277Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.861291Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.861294Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.961589Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.961606Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.961610Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.062042Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.062063Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.062067Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
E1009 21:43:17.107212119 3631916 chttp2_transport.cc:1143]             ipv4:127.0.0.1:19109: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 2500ms
2025-10-09T21:43:17.162317Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.162336Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.162339Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.262653Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.262682Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.262686Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.363077Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.363093Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.363096Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.463374Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.463390Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.463394Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.563699Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.563716Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.563719Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.664033Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.664055Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.664058Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Test retry state: get retry delay
2025-10-09T21:43:17.687179Z :NOTICE: [/Root] [] [] Retry to update federation state in 2.000000s
2025-10-09T21:43:17.764410Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.764426Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.764429Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.864946Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.864961Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.864964Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.965086Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.965111Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.965115Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.065419Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.065439Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.065442Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.165872Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.165890Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.165892Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.266190Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.266211Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.266214Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.366500Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.366519Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.366523Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.466821Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.466844Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.466848Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.567202Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.567220Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.567224Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.667532Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.667556Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.667558Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.767911Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.767926Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.767929Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.868356Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.868373Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.868377Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.968623Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.968639Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.968643Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.068989Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.069014Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.069017Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.169429Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.169444Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.169447Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.269687Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.269715Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.269718Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.369992Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.370010Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.370014Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.471496Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.471512Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.471515Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.506268Z node 4 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:19.571771Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.571785Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.571789Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.672147Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.672173Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.672176Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Test retry state: get retry delay
2025-10-09T21:43:19.688219Z :NOTICE: [/Root] [] [] Retry to update federation state in 2.000000s
2025-10-09T21:43:19.772476Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.772493Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.772496Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.872886Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.872904Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.872908Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.973216Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.973235Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.973253Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.073678Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.073694Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.073697Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.174280Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.174298Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.174302Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.274561Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.274580Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.274584Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.374651Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.374668Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.374672Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.475720Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.475744Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.475748Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.576349Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.576364Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.576368Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.676652Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.676671Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.676674Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.777263Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.777275Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.777278Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.877340Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.877357Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.877360Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.977729Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.977746Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.977750Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.078194Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.078209Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.078212Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.178442Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.178470Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.178473Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.282094Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.282116Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.282119Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.382571Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.382588Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.382591Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.483033Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.483053Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.483057Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.583389Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.583404Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.583407Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.683677Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.683694Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.683698Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Test retry state: get retry delay
2025-10-09T21:43:21.690112Z :NOTICE: [/Root] [] [] Retry to update federation state in 2.000000s
2025-10-09T21:43:21.786106Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.786126Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.786129Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.886503Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.886522Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.886525Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.988918Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.988935Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.988938Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.089224Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.089250Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.089254Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.189585Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.189601Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.189604Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.289970Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.289986Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.289989Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.390297Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.390315Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.390319Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.490615Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.490639Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.490642Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.593988Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.594005Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.594008Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.698203Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.698219Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.698222Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.798322Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.798337Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.798339Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.901459Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.901485Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.901489Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.001632Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.001650Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.001653Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.101985Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.102005Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.102008Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.205901Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.205930Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.205934Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.306336Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.306352Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.306355Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.410133Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.410149Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.410153Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.509818Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.509833Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.509836Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.610466Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.610484Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.610487Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Test retry state: get retry delay
2025-10-09T21:43:23.691242Z :NOTICE: [/Root] [] [] Retry to update federation state in 2.000000s
2025-10-09T21:43:23.710763Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.710778Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.710781Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.811130Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.811143Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.811147Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.911438Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.911462Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.911464Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.011759Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.011778Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.011782Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.112193Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.112221Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.112224Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.212495Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.212511Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.212515Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.314229Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.314248Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.314251Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.414187Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.414205Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.414208Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.506447Z node 4 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:24.514598Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.514617Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.514621Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.614980Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.614998Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.615001Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.715309Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.715327Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.715330Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.818368Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.818386Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.818389Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.918506Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.918523Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.918526Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.020561Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.020581Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.020584Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.122214Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.122236Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.122240Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.222703Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.222720Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.222723Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.326840Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.326859Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.326863Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.431011Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.431029Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.431032Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.530899Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.530915Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.530918Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.631244Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.631273Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.631276Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Test retry state: get retry delay
2025-10-09T21:43:25.694185Z :NOTICE: [/Root] [] [] Retry to update federation state in 2.000000s
2025-10-09T21:43:25.731607Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.731623Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.731625Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.834159Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.834178Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.834182Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.934386Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.934403Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.934407Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.035043Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.035065Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.035069Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.135876Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.135893Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.135896Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.238482Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.238502Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.238506Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.341845Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.341862Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.341865Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.441741Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.441764Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.441767Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.542151Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.542167Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.542171Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.646437Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.646456Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.646459Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.746432Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.746450Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.746453Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.849116Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.849133Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.849135Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.949238Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.949253Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.949257Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.049991Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.050010Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.050014Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.157107Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.157123Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.157126Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.257380Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.257395Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.257397Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.357723Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.357738Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.357741Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.458080Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.458099Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.458102Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.561498Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.561513Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.561516Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.661488Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.661504Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.661507Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Test retry state: get retry delay
2025-10-09T21:43:27.698214Z :NOTICE: [/Root] [] [] Retry to update federation state in 2.000000s
2025-10-09T21:43:27.761681Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.761698Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.761702Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.862604Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.862622Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.862625Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.966319Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.966338Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.966341Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.070187Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.070204Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.070208Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.170301Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.170320Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.170323Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.270626Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.270641Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.270643Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.371041Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.371057Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.371060Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.474095Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.474114Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.474117Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.574234Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.574252Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.574255Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.674569Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.674585Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.674588Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.775688Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.775705Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.775709Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.875838Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.875856Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.875860Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.976667Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.976682Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.976685Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.076997Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.077024Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.077027Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.177412Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.177428Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.177431Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.281245Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.281263Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.281266Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.384806Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.384820Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.384822Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.488387Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.488408Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.488411Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.489752Z node 4 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 0 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:43:29.486607Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:43:29.486648Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 1
2025-10-09T21:43:29.490760Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 1 DataSize: 0 UsedReserveSize: 0
2025-10-09T21:43:29.491010Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:43:29.491147Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72057594046644480, NodeId 3, Generation 2
2025-10-09T21:43:29.510244Z node 4 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:29.590552Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.590565Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.590567Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
=== Waiting for repair
>>> Ready to answer: ok
2025-10-09T21:43:29.690622Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.690639Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.690642Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.702821Z :INFO: [/Root] [] [] Start federated write session to database 'dc2' (previous was <empty>) FederationState: { Status: SUCCESS SelfLocation: "fancy_datacenter" DbInfos: [ { name: "dc1" path: "/Root" id: "account-dc1" endpoint: "localhost:24185" location: "dc1" status: AVAILABLE weight: 1000 } { name: "dc2" path: "/Root" id: "account-dc2" endpoint: "localhost:24185" location: "dc2" status: AVAILABLE weight: 500 } { name: "dc3" path: "/Root" id: "account-dc3" endpoint: "localhost:24185" location: "dc3" status: AVAILABLE weight: 500 } ] ControlPlaneEndpoint: cp.logbroker-federation:2135 }
2025-10-09T21:43:29.706239Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: try to update token
=== Closing the session
2025-10-09T21:43:29.706622Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Start write session. Will connect to nodeId: 0
2025-10-09T21:43:29.709488Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: write to message_group: src_id
2025-10-09T21:43:29.709590Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: send init request: init_request { path: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:29.709722Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: OnWriteDone gRpcStatusCode: 0
2025-10-09T21:43:29.710258Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: close. Timeout 0.000000s
2025-10-09T21:43:29.710271Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session will now close
2025-10-09T21:43:29.710292Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: aborting
2025-10-09T21:43:29.710207Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:29.710227Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2
2025-10-09T21:43:29.710470Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: gracefully shut down, all writes complete
2025-10-09T21:43:29.710492Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: destroy
2025-10-09T21:43:29.710819Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId:  grpc read done: success: 1 data: init_request { path: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:29.710881Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2 path: "test-topic" message_group_id: "src_id" from ipv6:[::1]:50768
2025-10-09T21:43:29.710895Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2 sessionId= userAgent="topic server" ip=ipv6:[::1]:50768 proto=topic  topic=test-topic durationSec=0
2025-10-09T21:43:29.710900Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:29.716150Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2 sessionId:  describe result for acl check
2025-10-09T21:43:29.716217Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 2 sessionId:  grpc closed
2025-10-09T21:43:29.716219Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2 sessionId:  is DEAD
2025-10-09T21:43:29.716489Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:29.716501Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:29.716502Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:29.716509Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [3:7559340911120030933:2721] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:44.611597Z :PropagateSessionClosed INFO: Random seed for debugging is 1760046164611592
2025-10-09T21:42:44.698908Z node 1 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7559340715079663876:2232];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:44.698941Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-10-09T21:42:44.702925Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:123: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, got undelivered to scheme cache: ActorUnknown
2025-10-09T21:42:44.703064Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/owmb/000361/r3tmp/tmpaDboYU/pdisk_1.dat
2025-10-09T21:42:44.714872Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:44.778399Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:44.778827Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:42:44.778513Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:44.778602Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
TServer::EnableGrpc on GrpcPort 11460, node 1
2025-10-09T21:42:44.813227Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:44.813272Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:44.815442Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:44.815471Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:44.820627Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/owmb/000361/r3tmp/yandexvBHgke.tmp
2025-10-09T21:42:44.820640Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/owmb/000361/r3tmp/yandexvBHgke.tmp
2025-10-09T21:42:44.820710Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/owmb/000361/r3tmp/yandexvBHgke.tmp
2025-10-09T21:42:44.820766Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-10-09T21:42:44.825308Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-10-09T21:42:44.825345Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:44.825711Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:44.835574Z INFO: TTestServer started on Port 27629 GrpcPort 11460
TClient is connected to server localhost:27629
PQClient connected to localhost:11460
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-09T21:42:44.901018Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
waiting...
waiting...
waiting...
2025-10-09T21:42:44.972262Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:45.003090Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:45.302791Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340719374631923:2325], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.302825Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.302972Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340719374631943:2328], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.302988Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340719374631944:2329], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.303068Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.303961Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976720661:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-10-09T21:42:45.314131Z node 1 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [1:7559340719374631947:2330], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720661 completed, doublechecking }
2025-10-09T21:42:45.367584Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:45.381378Z node 1 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [1:7559340719374632093:2698] txid# 281474976720663, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:45.401158Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [1:7559340719374632103:2341], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:45.401828Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=1&id=NGJjYjE4YTItMTNmZDljMDMtZTY1NmEyZC05NDc3N2U5Yw==, ActorId: [1:7559340719374631918:2322], ActorState: ExecuteState, TraceId: 01k75etb9m3ed0ykmty1wh17m0, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:45.402410Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:45.447232Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:45.483573Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720665:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:45.522900Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [2:7559340719012824620:2306], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:5:17: Error: At function: KiReadTable!
        <main>:5:17: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Versions]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:45.523031Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=2&id=ZWZhZGIyZWMtZmI4NTc0ZWQtODYxMmQ4YjktZmE1OTI2MmQ=, ActorId: [2:7559340719012824568:2298], ActorState: ExecuteState, TraceId: 01k75etbfpcmk0t88nqx1fv58b, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:45.523242Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:11460", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, false, 1000);
2025-10-09T21:42:45.605951Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720666. Ctx: { TraceId: 01k75etbjg2y0v5z92aw29d3be, Database: , SessionId: ydb://session/3?node_id=1&id=NjA3OWFmMy00ZDc1YjMyOC1hZjcyN2E1Yy02ZTc5YTI5MA==, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [1:7559340719374632424:2946] 
2025-10-09T21:42:45.699620Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:45.706156Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:49.702072Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7559340715079663876:2232];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:49.702109Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
2025-10-09T21:42:50.771082Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720672:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
waiting...
PQ Client: create topic: rt3.dc1--test-topic with 1 partitions
CallPersQueueGRPC request to localhost:11460
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:50.842824Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:11460
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720673
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-09T21:42:50.860536Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][] pipe [1:7559340740849469323:3209] connected; active server actors: 1
2025-10-09T21:42:50.860586Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1517: [72075186224037893][rt3.dc1--test-topic] updating configuration. Deleted partitions []. Added partitions [0]
2025-10-09T21:42:50.861265Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:940: [72075186224037893][rt3.dc1--test-topic] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-09T21:42:50.870761Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:138: [72075186224037893][rt3.dc1--test-topic] BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) 
2025-10-09T21:42:50.878734Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720674. Ctx: { TraceId: 01k75etgq5crhg50texf675gkm, Database: , SessionId: ydb://session/3?node_id=1&id=OWU2MWU1ODgtZWNlMTM2NDQtOGI0MDllZTAtOTc3ZDk4OWU=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-10-09T21:42:50.879482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:50.880529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:50.880585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:784: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-09T21:42:50.880597Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:50.880611Z node 2 :PERSQUEUE INFO: pq_impl.cpp:609: [PQ: 72075186224037892] doesn't have tx writes info
2025-10-09T21:42:50.880838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340740849469322:3208], now have 1 active actors on pipe
2025-10-09T21:42:50.880846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340740849469355:2432], now have 1 active actors on pipe
2025-10-09T21:42:50.880910Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-10-09T21:42:50.881126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.882822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
2025-10-09T21:42:50.889739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340740849469401:3262], now have 1 active actors on pipe
2025-10-09T21:42:50.889853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.891269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.891515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:50.891537Z node 2 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:50.891593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:50.891666Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340740487661509:2380]
2025-10-09T21:42:50.892353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:50.892362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:50.892368Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [2:7559340740487661509:2380]
2025-10-09T21:42:50.892375Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:50.892626Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-10-09T21:42:50.892639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.892668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.892674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:50.892678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:50.892680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.892681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.892683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:50.892689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:50.892792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.892830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
TClient::Ls response: Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
GetTopicVersionFromPath:  record Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
 name rt3.dc1--test-topic version0
2025-10-09T21:42:50.894293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.894326Z node 2 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046170939, TxId 281474976720673
2025-10-09T21:42:50.894330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.894331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.894353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.894392Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:50.895033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:50.895093Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:50.895101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.895103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.895105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.895109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.895192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:395: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-10-09T21:42:50.895209Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:50.895245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.897150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.897171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.897819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.990135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.990149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.990152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.098101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.098121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.098125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.198107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.198119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.198123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.302186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.302205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.302208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046170939
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046170939
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:11460
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:51.402233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.402249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.402253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.403410Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
2025-10-09T21:42:51.410928Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-09T21:42:51.411153Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-09T21:42:51.411161Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:51.413492Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:51.413918Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:51.413933Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1
2025-10-09T21:42:51.414351Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:51.414380Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:44980
2025-10-09T21:42:51.414384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:44980 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:51.414387Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:51.414902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-09T21:42:51.414946Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:51.414948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:51.414949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:51.414954Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340745144436759:2450] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:51.415489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340745144436759:2450] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:51.450589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [1:7559340745144436759:2450] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:51.450963Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340745144436793:2450] connected; active server actors: 1
2025-10-09T21:42:51.451037Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [1:7559340745144436759:2450] (SourceId=src, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src
2025-10-09T21:42:51.451043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340745144436759:2450] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:51.454159Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340745144436793:2450] disconnected.
2025-10-09T21:42:51.454173Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340745144436793:2450] disconnected; active server actors: 1
2025-10-09T21:42:51.454177Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340745144436793:2450] disconnected no session
2025-10-09T21:42:51.475627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340745144436815:2450], now have 1 active actors on pipe
2025-10-09T21:42:51.476204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.476227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.476260Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src|7b3e804e-82b8518d-d9061b7d-1436e10e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src
2025-10-09T21:42:51.476271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.476277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.476280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:51.476292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.476306Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:51.476324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.476678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.476686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.476706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.477103Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046171477 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:51.477142Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|7b3e804e-82b8518d-d9061b7d-1436e10e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:51.477275Z :INFO: [] MessageGroupId [src] SessionId [src|7b3e804e-82b8518d-d9061b7d-1436e10e_0] Write session: close. Timeout = 0 ms
2025-10-09T21:42:51.477280Z :INFO: [] MessageGroupId [src] SessionId [src|7b3e804e-82b8518d-d9061b7d-1436e10e_0] Write session will now close
2025-10-09T21:42:51.475042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340745144436759:2450] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:51.475059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340745144436759:2450] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:42:51.475063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340745144436759:2450] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:51.475072Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:51.476073Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1
2025-10-09T21:42:51.476821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1 partition: 0 MaxSeqNo: 0 sessionId: src|7b3e804e-82b8518d-d9061b7d-1436e10e_0
2025-10-09T21:42:51.477286Z :DEBUG: [] MessageGroupId [src] SessionId [src|7b3e804e-82b8518d-d9061b7d-1436e10e_0] Write session: aborting
2025-10-09T21:42:51.477533Z :INFO: [] MessageGroupId [src] SessionId [src|7b3e804e-82b8518d-d9061b7d-1436e10e_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:42:51.477540Z :DEBUG: [] MessageGroupId [src] SessionId [src|7b3e804e-82b8518d-d9061b7d-1436e10e_0] Write session: destroy
2025-10-09T21:42:51.478154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340745144436815:2450] destroyed
2025-10-09T21:42:51.478169Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:42:51.478177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.478180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.478182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.477768Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId: src|7b3e804e-82b8518d-d9061b7d-1436e10e_0 grpc read done: success: 0 data: 
2025-10-09T21:42:51.477774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 1 sessionId: src|7b3e804e-82b8518d-d9061b7d-1436e10e_0 grpc read failed
2025-10-09T21:42:51.477780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 1 sessionId: src|7b3e804e-82b8518d-d9061b7d-1436e10e_0 grpc closed
2025-10-09T21:42:51.477785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1 sessionId: src|7b3e804e-82b8518d-d9061b7d-1436e10e_0 is DEAD
2025-10-09T21:42:51.477988Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
Session was created
2025-10-09T21:42:51.502710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.502728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.502732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: ok
2025-10-09T21:42:51.541520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [] Write session: try to update token
2025-10-09T21:42:51.541670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [] Write session: Do CDS request
2025-10-09T21:42:51.541700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:51.542186Z :INFO: [/Root] [] [23bb647a-7ef6e656-d10f7a44-aaa88419] Open read subsessions to databases: { name: dc1, endpoint: localhost:11460, path: /Root }, { name: dc2, endpoint: localhost:11460, path: /Invalid }
2025-10-09T21:42:51.548500Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] Starting read session
2025-10-09T21:42:51.548514Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] Starting single session
2025-10-09T21:42:51.548762Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:51.548770Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:51.548775Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.000000s
2025-10-09T21:42:51.593703Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:42:51.593713Z :INFO: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Starting read session
2025-10-09T21:42:51.593728Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:51.593734Z :DEBUG: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Starting single session
2025-10-09T21:42:51.593775Z :DEBUG: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:51.593779Z :DEBUG: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:51.593785Z :DEBUG: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] [] Reconnecting session to cluster  in 0.000000s
2025-10-09T21:42:51.593813Z :ERROR: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] [] Got error. Status: NOT_FOUND. Description: <main>: Error: Database Invalid does not exist, code: 200202 <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. 
2025-10-09T21:42:51.593818Z :DEBUG: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:51.593821Z :DEBUG: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:51.593865Z :INFO: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] [] Closing session to cluster: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:51.594411Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:51.594446Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2
2025-10-09T21:42:51.594451Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:42:51.594456Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 1
2025-10-09T21:42:51.594697Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:51.594739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:51.594749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:51.594753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:51.595188Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2 sessionId:  describe result for acl check
2025-10-09T21:42:51.595235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:51.595236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:51.595238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:51.595243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340745144436835:2469] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:51.595820Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340745144436835:2469] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:51.596524Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:42:51.596616Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:42:51.596690Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 auth for : user
2025-10-09T21:42:51.596842Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 Handle describe topics response
2025-10-09T21:42:51.596879Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 auth is DEAD
2025-10-09T21:42:51.596893Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:42:51.597191Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:42:51.597337Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340745144436843:2470] connected; active server actors: 1
2025-10-09T21:42:51.597354Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559340745144436843:2470] session shared/user_1_1_7577822143567276789_v1
2025-10-09T21:42:51.597365Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:42:51.597376Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:42:51.597382Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_1_7577822143567276789_v1" (Sender=[1:7559340745144436833:2470], Pipe=[1:7559340745144436843:2470], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:42:51.597387Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:42:51.597398Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:42:51.597412Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_1_7577822143567276789_v1" (Sender=[1:7559340745144436833:2470], Pipe=[1:7559340745144436843:2470], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:42:51.597423Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_1_7577822143567276789_v1" sender [1:7559340745144436833:2470] lock partition 0 for ReadingSession "shared/user_1_1_7577822143567276789_v1" (Sender=[1:7559340745144436833:2470], Pipe=[1:7559340745144436843:2470], Partitions=[], ActiveFamilyCount=1) generation 1 step 1
2025-10-09T21:42:51.597431Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:42:51.597441Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000034s
2025-10-09T21:42:51.597551Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_1_7577822143567276789_v1
2025-10-09T21:42:51.597572Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In ContinueReadingDataImpl, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:42:51.597685Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:42:51.597685Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 1 Session: "shared/user_1_1_7577822143567276789_v1" ClientId: "user" PipeClient { RawX1: 7559340745144436843 RawX2: 4503603922340262 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:42:51.597709Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:42:51.597830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340745144436854:2476], now have 1 active actors on pipe
2025-10-09T21:42:51.597881Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1, pipe: [1:7559340745144436854:2476]
2025-10-09T21:42:51.598001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.598015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.598049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_1_7577822143567276789_v1 on pipe: [1:7559340745144436854:2476]
2025-10-09T21:42:51.598074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.598077Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_1_7577822143567276789_v1:1 with generation 1
2025-10-09T21:42:51.598091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.598100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:51.598211Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 grpc read done: success# 1, data# { read_request { bytes_size: 524288 } }
2025-10-09T21:42:51.598240Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 got read request: guid# 354d4e6f-cd20a7bb-a8d58309-2e8f983b
2025-10-09T21:42:51.598121Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_1_7577822143567276789_v1
2025-10-09T21:42:51.598128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.598161Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:51.598744Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [1:7559340745144436835:2469] (SourceId=src_id, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:51.598796Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340745144436861:2469] connected; active server actors: 1
2025-10-09T21:42:51.598811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [1:7559340745144436835:2469] (SourceId=src_id, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src_id
2025-10-09T21:42:51.598816Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340745144436835:2469] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:51.598867Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340745144436861:2469] disconnected.
2025-10-09T21:42:51.598874Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340745144436861:2469] disconnected; active server actors: 1
2025-10-09T21:42:51.598877Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340745144436861:2469] disconnected no session
2025-10-09T21:42:51.598967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:51.598998Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:51.599010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.599012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.599013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.599019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:51.599033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:42:51.599304Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 0 SizeLag: 0 WriteTimestampEstimateMS: 1760046171502 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:42:51.599319Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 0 readOffset 0 committedOffset 0
2025-10-09T21:42:51.599334Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 sending to client partition status
2025-10-09T21:42:51.601570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340745144436835:2469] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:51.601583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340745144436835:2469] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:42:51.601587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340745144436835:2469] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:51.601595Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:51.601820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340745144436878:2469], now have 1 active actors on pipe
2025-10-09T21:42:51.601882Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1
2025-10-09T21:42:51.602014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.602045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.602074Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:51.602088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.602097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.602112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:51.602125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.602139Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:51.602157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.602449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.602457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.602475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.602578Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2 partition: 0 MaxSeqNo: 0 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0
2025-10-09T21:42:51.602863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046171602 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:51.602898Z :INFO: [/Root] MessageGroupId [src_id] SessionId [] Write session established. Init response: session_id: "src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:51.602892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.602898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.602900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.605554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 1 to 1
2025-10-09T21:42:51.607578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.607597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 1
>>> Got event: StartPartitionSession { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 CommittedOffset: 0 EndOffset: 0 }
2025-10-09T21:42:51.613043Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Confirm partition stream create. Partition stream id: 1. Cluster: "-". Topic: "test-topic". Partition: 0. Read offset: (NULL)
2025-10-09T21:42:51.613045Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.613443Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 grpc read done: success# 1, data# { start_partition_session_response { partition_session_id: 1 } }
2025-10-09T21:42:51.613521Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:539: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 got StartRead from client: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 0, commitOffset# (empty maybe)
2025-10-09T21:42:51.613543Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1023: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 Start reading TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 0 readOffset 0 committedOffset 0 clientCommitOffset (empty maybe) clientReadOffset 0
2025-10-09T21:42:51.613553Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 1 from offset 0
2025-10-09T21:42:51.615223Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.615405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.615424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.615473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:42:51.615659Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.619195Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 2 to 2
2025-10-09T21:42:51.621176Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.621198Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 2
2025-10-09T21:42:51.624042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.624069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.624298Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-09T21:42:51.624314Z :DEBUG: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] [] Abort session to cluster
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:51.624628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:51.624689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:51.624832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:51.624892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:51.624962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 4 messageNo: 1 size: 205039
2025-10-09T21:42:51.624971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 4 messageNo: 1 size 205039 offset: -1
2025-10-09T21:42:51.625002Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.625035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.625051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.625059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (5)
2025-10-09T21:42:51.625096Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 0
2025-10-09T21:42:51.625113Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 1
2025-10-09T21:42:51.625120Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 2
2025-10-09T21:42:51.625123Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 3
2025-10-09T21:42:51.625127Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 4
2025-10-09T21:42:51.625770Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 1 partNo 4 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2253229 count 1 nextOffset 1 batches 5
2025-10-09T21:42:51.625792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.625896Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 0,1 HeadOffset 0 endOffset 0 curOffset 1 d0000000000_00000000000000000000_00000_0000000001_00004? size 2253219 WTime 1760046171625
2025-10-09T21:42:51.627691Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:51.627713Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 1 size 2253219
2025-10-09T21:42:51.636922Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 3 to 3
2025-10-09T21:42:51.638808Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.638827Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 3
2025-10-09T21:42:51.641587Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.643642Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.646259Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:51.647183Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.646596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.646619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.646851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 2  requestId: <none> cookie: 2
2025-10-09T21:42:51.649715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.649729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.650228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 2 partNo : 0 messageNo: 3 size: 511959
2025-10-09T21:42:51.651562Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 4 to 4
2025-10-09T21:42:51.650680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 2 partNo : 1 messageNo: 3 size: 511959
2025-10-09T21:42:51.651110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 2 partNo : 2 messageNo: 3 size: 511959
2025-10-09T21:42:51.651522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 2 partNo : 3 messageNo: 3 size: 511959
2025-10-09T21:42:51.664761Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.664784Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 4
2025-10-09T21:42:51.668130Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.670563Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.671507Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:51.670815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 2 partNo : 4 messageNo: 3 size: 205039
2025-10-09T21:42:51.670832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 2 partNo : 4 messageNo: 3 size 205039 offset: -1
2025-10-09T21:42:51.670887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 1 size 2253219 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:51.670933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.670941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.670957Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.670993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.670997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:51.671017Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 2252910 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:51.671024Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.671035Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 1, partNo: 0, Offset: 0 is stored on disk
2025-10-09T21:42:51.671038Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.671043Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 1, partNo: 1, Offset: 0 is stored on disk
2025-10-09T21:42:51.671045Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.671050Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 1, partNo: 2, Offset: 0 is stored on disk
2025-10-09T21:42:51.671052Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.671057Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 1, partNo: 3, Offset: 0 is stored on disk
2025-10-09T21:42:51.671065Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.671070Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 1, partNo: 4, Offset: 0 is stored on disk
2025-10-09T21:42:51.671865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:51.671873Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:51.671878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.671891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.671893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (5)
2025-10-09T21:42:51.671921Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 0
2025-10-09T21:42:51.671933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 1
2025-10-09T21:42:51.671938Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 2
2025-10-09T21:42:51.671943Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 3
2025-10-09T21:42:51.671946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 4
2025-10-09T21:42:51.672529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 2 partNo 4 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 2253229 count 1 nextOffset 2 batches 5
2025-10-09T21:42:51.672542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.672660Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 1,1 HeadOffset 1 endOffset 1 curOffset 2 d0000000000_00000000000000000001_00000_0000000001_00004? size 2253219 WTime 1760046171672
2025-10-09T21:42:51.672926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:51.672968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1  requestId: <none> cookie: 1
2025-10-09T21:42:51.672987Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:51.673000Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 1 partNo 0 count 1 size 2253219
2025-10-09T21:42:51.673009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 4  requestId: <none> cookie: 3
2025-10-09T21:42:51.673082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 1 max time lag 0ms effective offset 0
2025-10-09T21:42:51.673098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 2253219 count 1 last offset 0, current partition end offset: 1
2025-10-09T21:42:51.673102Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:51.673127Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 4 source 1 size 2253219 accessed 0 times before, last time 2025-10-09T21:42:51.000000Z
2025-10-09T21:42:51.673131Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 3. All 1 blobs are from cache.
2025-10-09T21:42:51.673148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:51.673317Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:42:51.673444Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.673570Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.673782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.674020Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 1 count 1 size 205085 from pos 0 cbcount 1
2025-10-09T21:42:51.674125Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 4 suffix '63' size 2253219
2025-10-09T21:42:51.674136Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 4 suffix '63'
2025-10-09T21:42:51.674266Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 0 EndOffset 0 newEndOffset 1 commitOffset 0 clientCommitOffset 0 cookie 1 readingFinished 0 firstRead 1
2025-10-09T21:42:51.674279Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 0 endOffset 1
2025-10-09T21:42:51.674302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.674324Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.674611Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 0, endOffset# 1, WTime# 0, sizeLag# 2253219
2025-10-09T21:42:51.674615Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:42:51.674633Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 performing read request: guid# d7046cee-d358985e-8b3b666e-15b71937, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 1, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:42:51.674657Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 1 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 0 EndOffset 1 ClientCommitOffset 0 committedOffset 0 Guid d7046cee-d358985e-8b3b666e-15b71937
2025-10-09T21:42:51.675915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.675930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.675953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:42:51.675988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 4 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 524288 endOffset 1 max time lag 0ms effective offset 0
2025-10-09T21:42:51.676007Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 4 added 1 blobs, size 2253219 count 1 last offset 0, current partition end offset: 1
2025-10-09T21:42:51.676010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 4. Send blob request.
2025-10-09T21:42:51.676021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 4 source 1 size 2253219 accessed 1 times before, last time 2025-10-09T21:42:51.000000Z
2025-10-09T21:42:51.676042Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 4. All 1 blobs are from cache.
2025-10-09T21:42:51.676055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:51.676224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:42:51.676352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.676485Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.676630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.676675Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 4 suffix '63'
2025-10-09T21:42:51.676696Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 1 count 1 size 205085 from pos 0 cbcount 1
2025-10-09T21:42:51.676733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.679439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.679456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.679528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 3 partNo : 0 messageNo: 5 size: 511959
2025-10-09T21:42:51.679578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 3 partNo : 1 messageNo: 5 size: 511959
2025-10-09T21:42:51.679627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 3 partNo : 2 messageNo: 5 size: 511959
2025-10-09T21:42:51.679677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 3 partNo : 3 messageNo: 5 size: 511959
2025-10-09T21:42:51.679700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 3 partNo : 4 messageNo: 5 size: 205039
2025-10-09T21:42:51.679705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 3 partNo : 4 messageNo: 5 size 205039 offset: -1
2025-10-09T21:42:51.679723Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.679744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.682797Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 1 offsets: 0 already_written: false write_statistics { persist_duration_ms: 46 }
2025-10-09T21:42:51.682814Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 1
2025-10-09T21:42:51.683471Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 1 Result { Offset: 0 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 1 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1760046171624 CreateTimestampMS: 1760046171603 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 110 RealReadOffset: 0 WaitQuotaTimeMs: 0 EndOffset: 1 StartOffset: 0 } Cookie: 0 }
2025-10-09T21:42:51.683730Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 2 from offset 1
2025-10-09T21:42:51.683742Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 1 ReadOffset 1 ReadGuid d7046cee-d358985e-8b3b666e-15b71937 has messages 1
2025-10-09T21:42:51.683782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.684145Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.684196Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 read done: guid# d7046cee-d358985e-8b3b666e-15b71937, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 2252976
2025-10-09T21:42:51.684205Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 response to read: guid# d7046cee-d358985e-8b3b666e-15b71937
2025-10-09T21:42:51.685270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.685282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.685311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 6  requestId: <none> cookie: 4
2025-10-09T21:42:51.688931Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got ReadResponse, serverBytesSize = 2252976, now ReadSizeBudget = 0, ReadSizeServerDelta = -1728688
2025-10-09T21:42:51.689035Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (0-0)
2025-10-09T21:42:51.689062Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 2252976 to budget
2025-10-09T21:42:51.688330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.688348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.688465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 4 partNo : 0 messageNo: 7 size: 511959
2025-10-09T21:42:51.688661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 4 partNo : 1 messageNo: 7 size: 511959
2025-10-09T21:42:51.688846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 4 partNo : 2 messageNo: 7 size: 511959
2025-10-09T21:42:51.688908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 4 partNo : 3 messageNo: 7 size: 511959
2025-10-09T21:42:51.688957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 4 partNo : 4 messageNo: 7 size: 205039
2025-10-09T21:42:51.688962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 4 partNo : 4 messageNo: 7 size 205039 offset: -1
2025-10-09T21:42:51.688986Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.689013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.691465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 5 to 5
2025-10-09T21:42:51.685981Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.685989Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 Process answer. Aval parts: 0
2025-10-09T21:42:51.691824Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.691837Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 5
2025-10-09T21:42:51.693500Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (0-0)
2025-10-09T21:42:51.693523Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
2025-10-09T21:42:51.693536Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In ContinueReadingDataImpl, ReadSizeBudget = 2252976, ReadSizeServerDelta = -1728688
2025-10-09T21:42:51.693618Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:42:51.693633Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 0 to budget
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 0 ProducerId: "src_id" SeqNo: 1 CreateTime: 2025-10-09T21:42:51.603000Z WriteTime: 2025-10-09T21:42:51.624000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:57708", "logtype": "unknown", "server": "ipv6:[::1]:57708" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-09T21:42:51.691407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 1 count 1 size 2253219 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:51.691438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:51.691458Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 2252910 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:51.691465Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.691476Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 2, partNo: 0, Offset: 1 is stored on disk
2025-10-09T21:42:51.691480Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.691485Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 2, partNo: 1, Offset: 1 is stored on disk
2025-10-09T21:42:51.691487Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.691492Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 2, partNo: 2, Offset: 1 is stored on disk
2025-10-09T21:42:51.691494Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.691498Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 2, partNo: 3, Offset: 1 is stored on disk
2025-10-09T21:42:51.691513Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.691517Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 2, partNo: 4, Offset: 1 is stored on disk
2025-10-09T21:42:51.691572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.691589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.691591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (10)
2025-10-09T21:42:51.691626Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 0
2025-10-09T21:42:51.691641Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 1
2025-10-09T21:42:51.691647Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 2
2025-10-09T21:42:51.691652Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 3
2025-10-09T21:42:51.691656Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 4
2025-10-09T21:42:51.692637Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 3 partNo 4 FormedBlobsCount 0 NewHead: Offset 2 PartNo 0 PackedSize 2253229 count 1 nextOffset 3 batches 5
2025-10-09T21:42:51.692658Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 0
2025-10-09T21:42:51.692666Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 1
2025-10-09T21:42:51.692670Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 2
2025-10-09T21:42:51.692675Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 3
2025-10-09T21:42:51.692678Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 4
2025-10-09T21:42:51.693766Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 4 partNo 4 FormedBlobsCount 0 NewHead: Offset 2 PartNo 0 PackedSize 4506426 count 2 nextOffset 4 batches 9
2025-10-09T21:42:51.693776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.693886Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 2,2 HeadOffset 2 endOffset 2 curOffset 4 d0000000000_00000000000000000002_00000_0000000002_00008? size 4506416 WTime 1760046171692
2025-10-09T21:42:51.696834Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 1 EndOffset 1 newEndOffset 2 commitOffset 0 clientCommitOffset 0 cookie 2 readingFinished 0 firstRead 0
2025-10-09T21:42:51.696849Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 1 endOffset 2
2025-10-09T21:42:51.696871Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 1, endOffset# 2, WTime# 1760046171624, sizeLag# 2253219
2025-10-09T21:42:51.696874Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:42:51.696700Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 4 suffix '63' size 2253219
2025-10-09T21:42:51.697559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 6 to 6
2025-10-09T21:42:51.699161Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 grpc read done: success# 1, data# { read_request { bytes_size: 2252976 } }
2025-10-09T21:42:51.699202Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 got read request: guid# 3abef7e2-65e24188-c6ab30c-b30d6c5b
2025-10-09T21:42:51.699220Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 performing read request: guid# 190a0625-d134d0d0-d82ef9ea-3f141d24, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 1, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:42:51.699248Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 1 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 1 EndOffset 2 ClientCommitOffset 0 committedOffset 0 Guid 190a0625-d134d0d0-d82ef9ea-3f141d24
2025-10-09T21:42:51.698859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:51.698914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 3  requestId: <none> cookie: 2
2025-10-09T21:42:51.698934Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:51.698947Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 2 partNo 0 count 2 size 4506416
2025-10-09T21:42:51.699892Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.701412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.701427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.701474Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 5 Topic 'rt3.dc1--test-topic' partition 0 user user offset 1 count 1 size 524288 endOffset 2 max time lag 0ms effective offset 1
2025-10-09T21:42:51.701506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 5 added 1 blobs, size 2253219 count 1 last offset 1, current partition end offset: 2
2025-10-09T21:42:51.701512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 5. Send blob request.
2025-10-09T21:42:51.701525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 1 partno 0 count 1 parts_count 4 source 1 size 2253219 accessed 0 times before, last time 2025-10-09T21:42:51.000000Z
2025-10-09T21:42:51.701536Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 5. All 1 blobs are from cache.
2025-10-09T21:42:51.701553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:51.701655Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 4 suffix '63'
2025-10-09T21:42:51.708262Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.711360Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.711634Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 1 totakecount 1 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:42:51.711807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 1 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.711946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 1 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.712087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 1 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.712163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 1 totakecount 1 count 1 size 205085 from pos 0 cbcount 1
2025-10-09T21:42:51.712232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.712254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.712265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.712294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:42:51.715700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 8  requestId: <none> cookie: 5
2025-10-09T21:42:51.717693Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 2 Result { Offset: 1 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 2 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1760046171671 CreateTimestampMS: 1760046171618 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 110 RealReadOffset: 1 WaitQuotaTimeMs: 0 EndOffset: 2 StartOffset: 0 } Cookie: 1 }
2025-10-09T21:42:51.721082Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.726183Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.726208Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 6
2025-10-09T21:42:51.726623Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:51.724341Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 3 from offset 2
2025-10-09T21:42:51.724366Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 2 ReadOffset 2 ReadGuid 190a0625-d134d0d0-d82ef9ea-3f141d24 has messages 1
2025-10-09T21:42:51.724458Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 read done: guid# 190a0625-d134d0d0-d82ef9ea-3f141d24, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 2252978
2025-10-09T21:42:51.724469Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 response to read: guid# 190a0625-d134d0d0-d82ef9ea-3f141d24
2025-10-09T21:42:51.725746Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 Process answer. Aval parts: 0
2025-10-09T21:42:51.727696Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got ReadResponse, serverBytesSize = 2252978, now ReadSizeBudget = 0, ReadSizeServerDelta = -1728690
2025-10-09T21:42:51.727789Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (1-1)
2025-10-09T21:42:51.727795Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 2252978 to budget
2025-10-09T21:42:51.729903Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 7 to 7
2025-10-09T21:42:51.731765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.731789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.731868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 5 partNo : 0 messageNo: 9 size: 511959
2025-10-09T21:42:51.731925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 5 partNo : 1 messageNo: 9 size: 511959
2025-10-09T21:42:51.731976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 5 partNo : 2 messageNo: 9 size: 511959
2025-10-09T21:42:51.732026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 5 partNo : 3 messageNo: 9 size: 511959
2025-10-09T21:42:51.732060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 5 partNo : 4 messageNo: 9 size: 205039
2025-10-09T21:42:51.732065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 5 partNo : 4 messageNo: 9 size 205039 offset: -1
2025-10-09T21:42:51.732499Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.732528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.733163Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.733176Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 7
2025-10-09T21:42:51.734205Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.734639Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.736029Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (1-1)
2025-10-09T21:42:51.736050Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
2025-10-09T21:42:51.736062Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In ContinueReadingDataImpl, ReadSizeBudget = 2252978, ReadSizeServerDelta = -1728690
2025-10-09T21:42:51.738500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.738518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.738647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 2 count 2 size 4506416 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:51.738665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 10  requestId: <none> cookie: 6
2025-10-09T21:42:51.738678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:51.738699Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 4505820 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:51.738706Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.738716Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 3, partNo: 0, Offset: 2 is stored on disk
2025-10-09T21:42:51.738720Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.738725Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 3, partNo: 1, Offset: 2 is stored on disk
2025-10-09T21:42:51.738727Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.738732Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 3, partNo: 2, Offset: 2 is stored on disk
2025-10-09T21:42:51.738735Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.738739Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 3, partNo: 3, Offset: 2 is stored on disk
2025-10-09T21:42:51.738745Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.738753Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 3, partNo: 4, Offset: 2 is stored on disk
2025-10-09T21:42:51.738755Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.738759Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 4, partNo: 0, Offset: 3 is stored on disk
2025-10-09T21:42:51.738761Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.738765Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 4, partNo: 1, Offset: 3 is stored on disk
2025-10-09T21:42:51.738767Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.738771Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 4, partNo: 2, Offset: 3 is stored on disk
2025-10-09T21:42:51.738773Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.738778Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 4, partNo: 3, Offset: 3 is stored on disk
2025-10-09T21:42:51.738780Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.738783Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 4, partNo: 4, Offset: 3 is stored on disk
2025-10-09T21:42:51.738835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.738846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.738849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (5)
2025-10-09T21:42:51.738876Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 0
2025-10-09T21:42:51.738887Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 1
2025-10-09T21:42:51.738892Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 2
2025-10-09T21:42:51.738896Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 3
2025-10-09T21:42:51.738899Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 4
2025-10-09T21:42:51.739297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 5 partNo 4 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 2253229 count 1 nextOffset 5 batches 5
2025-10-09T21:42:51.739313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.739419Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 4,1 HeadOffset 4 endOffset 4 curOffset 5 d0000000000_00000000000000000004_00000_0000000001_00004? size 2253219 WTime 1760046171739
2025-10-09T21:42:51.742007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000000_00000_0000000001_00004?
2025-10-09T21:42:51.742010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000001_00000_0000000001_00004?
2025-10-09T21:42:51.742013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000002_00000_0000000002_00008?
2025-10-09T21:42:51.742016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 9012854 bytes
2025-10-09T21:42:51.742811Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 2 parts 8 suffix '63' size 4506416
2025-10-09T21:42:51.746227Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 2 offsets: 1 already_written: false write_statistics { persist_duration_ms: 18 }
2025-10-09T21:42:51.746244Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 2
2025-10-09T21:42:51.746157Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 grpc read done: success# 1, data# { read_request { bytes_size: 2252978 } }
2025-10-09T21:42:51.746275Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 got read request: guid# 7e1fa03f-b919f375-83eeee0b-682e150c
2025-10-09T21:42:51.746541Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.747734Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 2 EndOffset 2 newEndOffset 4 commitOffset 0 clientCommitOffset 0 cookie 3 readingFinished 0 firstRead 0
2025-10-09T21:42:51.747740Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 2 endOffset 4
2025-10-09T21:42:51.747758Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 2, endOffset# 4, WTime# 1760046171671, sizeLag# 4506416
2025-10-09T21:42:51.747761Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:42:51.747788Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 performing read request: guid# ac24d051-dd3af3c1-cfcab5ee-3901469a, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 2, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:42:51.747807Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 2 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 2 EndOffset 4 ClientCommitOffset 0 committedOffset 0 Guid ac24d051-dd3af3c1-cfcab5ee-3901469a
2025-10-09T21:42:51.743209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 5  requestId: <none> cookie: 3
2025-10-09T21:42:51.743235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 7  requestId: <none> cookie: 4
2025-10-09T21:42:51.743244Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:51.743259Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 4 partNo 0 count 1 size 2253219
2025-10-09T21:42:51.743270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:51.743273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000000_00000_0000000001_00004?
2025-10-09T21:42:51.743281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000001_00000_0000000001_00004?
2025-10-09T21:42:51.743292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000002_00000_0000000002_00008?
2025-10-09T21:42:51.743308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:51.743364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 4 source 1 size 2253219 accessed 2 times before, last time 2025-10-09T21:42:51.000000Z
2025-10-09T21:42:51.743366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 1 partno 0 count 1 parts_count 4 source 1 size 2253219 accessed 1 times before, last time 2025-10-09T21:42:51.000000Z
2025-10-09T21:42:51.743369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 2 partno 0 count 2 parts_count 8 source 1 size 4506416 accessed 0 times before, last time 2025-10-09T21:42:51.000000Z
2025-10-09T21:42:51.743378Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 3 blobs are from cache.
2025-10-09T21:42:51.743415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:51.743420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000000_00000_0000000001_00004?
2025-10-09T21:42:51.743423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:51.743557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 0
2025-10-09T21:42:51.743690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 1
2025-10-09T21:42:51.743822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 2
2025-10-09T21:42:51.744391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 3
2025-10-09T21:42:51.744470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 4
2025-10-09T21:42:51.744861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 1 partNo 4 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2253229 count 1 nextOffset 1 batches 5
2025-10-09T21:42:51.744869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000001_00000_0000000001_00004?
2025-10-09T21:42:51.744870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:51.744993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 0
2025-10-09T21:42:51.745118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 1
2025-10-09T21:42:51.745243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 2
2025-10-09T21:42:51.745365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 3
2025-10-09T21:42:51.745413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 4
2025-10-09T21:42:51.747586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 2 partNo 4 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4506426 count 2 nextOffset 2 batches 9
2025-10-09T21:42:51.748227Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.748523Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.748692Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.748713Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.748654Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 4 suffix '63'
2025-10-09T21:42:51.748665Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 4 suffix '63'
2025-10-09T21:42:51.748669Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 2 parts 8 suffix '63'
2025-10-09T21:42:51.749410Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:42:51.749434Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:42:51.752406Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 3 offsets: 2 already_written: false write_statistics { persist_duration_ms: 48 queued_in_partition_duration_ms: 10 }
2025-10-09T21:42:51.752424Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 3
2025-10-09T21:42:51.752567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 4 offsets: 3 already_written: false write_statistics { persist_duration_ms: 45 queued_in_partition_duration_ms: 4 }
2025-10-09T21:42:51.752576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 4
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 1 ProducerId: "src_id" SeqNo: 2 CreateTime: 2025-10-09T21:42:51.618000Z WriteTime: 2025-10-09T21:42:51.671000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:57708", "server": "ipv6:[::1]:57708" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-09T21:42:51.760082Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 8 to 8
2025-10-09T21:42:51.760359Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.760373Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 8
2025-10-09T21:42:51.761956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000002_00000_0000000002_00008?
2025-10-09T21:42:51.761971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:51.762131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 0
2025-10-09T21:42:51.762277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 1
2025-10-09T21:42:51.762814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 2
2025-10-09T21:42:51.763339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 3
2025-10-09T21:42:51.764222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 4
2025-10-09T21:42:51.766870Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:51.770904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 3 partNo 4 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6759623 count 3 nextOffset 3 batches 13
2025-10-09T21:42:51.770937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 0
2025-10-09T21:42:51.771063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 1
2025-10-09T21:42:51.771300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 2
2025-10-09T21:42:51.771499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 3
2025-10-09T21:42:51.774344Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 9 to 9
2025-10-09T21:42:51.774635Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.774649Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 9
2025-10-09T21:42:51.782756Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:51.789546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 10 to 10
2025-10-09T21:42:51.790132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:91: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '' seqNo 4 partNo 3 result is x0000000000_00000000000000000000_00000_0000000003_00015 size 8295635
2025-10-09T21:42:51.790220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 4
2025-10-09T21:42:51.790233Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1098: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000000_00000_0000000003_00015 new key d0000000000_00000000000000000000_00000_0000000003_00015 size 8295635 WTime 1760046171789
2025-10-09T21:42:51.790386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 4 partNo 4 FormedBlobsCount 1 NewHead: Offset 3 PartNo 3 PackedSize 717144 count 1 nextOffset 4 batches 2
2025-10-09T21:42:51.790524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:537: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 3,1 HeadOffset 0 endOffset 0 curOffset 4 d0000000000_00000000000000000003_00003_0000000001_00001| size 717134 WTime 1760046171790
2025-10-09T21:42:51.792120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.792139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.792819Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.792835Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 10
2025-10-09T21:42:51.792223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 6 partNo : 0 messageNo: 11 size: 511959
2025-10-09T21:42:51.792285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 6 partNo : 1 messageNo: 11 size: 511959
2025-10-09T21:42:51.792340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 6 partNo : 2 messageNo: 11 size: 511959
2025-10-09T21:42:51.792418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 6 partNo : 3 messageNo: 11 size: 511959
2025-10-09T21:42:51.792447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 6 partNo : 4 messageNo: 11 size: 205039
2025-10-09T21:42:51.792452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 6 partNo : 4 messageNo: 11 size 205039 offset: -1
2025-10-09T21:42:51.792506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.792509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.792524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.792526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.792555Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:51.792572Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 3 size 8295635
2025-10-09T21:42:51.792580Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 3 partNo 3 count 1 size 717134
2025-10-09T21:42:51.792596Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.792621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.792830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 12  requestId: <none> cookie: 7
2025-10-09T21:42:51.792866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 6 Topic 'rt3.dc1--test-topic' partition 0 user user offset 2 count 2 size 524288 endOffset 4 max time lag 0ms effective offset 2
2025-10-09T21:42:51.792884Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 6 added 1 blobs, size 4506416 count 2 last offset 2, current partition end offset: 4
2025-10-09T21:42:51.792888Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 6. Send blob request.
2025-10-09T21:42:51.792901Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 2 partno 0 count 2 parts_count 8 source 1 size 4506416 accessed 1 times before, last time 2025-10-09T21:42:51.000000Z
2025-10-09T21:42:51.792907Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 6. All 1 blobs are from cache.
2025-10-09T21:42:51.792929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:51.793097Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 2 totakecount 2 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:42:51.793229Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 2 totakecount 2 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.794771Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:51.794999Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.793542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 2 totakecount 2 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.793689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 2 totakecount 2 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.794201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 2 totakecount 2 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:42:51.794319Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 2 parts 8 suffix '63'
2025-10-09T21:42:51.794339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 3 totakecount 2 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.794515Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 3 totakecount 2 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.794656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 3 totakecount 2 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.794739Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 3 totakecount 2 count 1 size 205085 from pos 0 cbcount 1
2025-10-09T21:42:51.794789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 2
2025-10-09T21:42:51.800682Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 11 to 11
2025-10-09T21:42:51.807587Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.807610Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 11
2025-10-09T21:42:51.808779Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:51.812105Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 4 Result { Offset: 2 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 3 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1760046171689 CreateTimestampMS: 1760046171636 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } Result { Offset: 3 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 4 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1760046171692 CreateTimestampMS: 1760046171651 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 198 RealReadOffset: 3 WaitQuotaTimeMs: 0 EndOffset: 4 StartOffset: 0 } Cookie: 2 }
2025-10-09T21:42:51.816995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.817019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.817110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 7 partNo : 0 messageNo: 13 size: 511959
2025-10-09T21:42:51.817167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 7 partNo : 1 messageNo: 13 size: 511959
2025-10-09T21:42:51.817225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 7 partNo : 2 messageNo: 13 size: 511959
2025-10-09T21:42:51.817283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 7 partNo : 3 messageNo: 13 size: 511959
2025-10-09T21:42:51.817325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 7 partNo : 4 messageNo: 13 size: 205039
2025-10-09T21:42:51.817331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 7 partNo : 4 messageNo: 13 size 205039 offset: -1
2025-10-09T21:42:51.817462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.817467Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.817490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.817506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 4 count 1 size 2253219 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:51.817527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:51.817546Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 2252910 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:51.817552Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.817562Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 5, partNo: 0, Offset: 4 is stored on disk
2025-10-09T21:42:51.817565Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.817570Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 5, partNo: 1, Offset: 4 is stored on disk
2025-10-09T21:42:51.817572Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.817576Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 5, partNo: 2, Offset: 4 is stored on disk
2025-10-09T21:42:51.817578Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.817581Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 5, partNo: 3, Offset: 4 is stored on disk
2025-10-09T21:42:51.817587Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.817591Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 5, partNo: 4, Offset: 4 is stored on disk
2025-10-09T21:42:51.817632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.817644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.817646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (10)
2025-10-09T21:42:51.817673Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 0
2025-10-09T21:42:51.817685Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 1
2025-10-09T21:42:51.817689Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 2
2025-10-09T21:42:51.817692Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 3
2025-10-09T21:42:51.817696Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 4
2025-10-09T21:42:51.817844Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 4 suffix '63' size 2253219
2025-10-09T21:42:51.819549Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 4 from offset 4
2025-10-09T21:42:51.819570Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 4 ReadOffset 4 ReadGuid ac24d051-dd3af3c1-cfcab5ee-3901469a has messages 1
2025-10-09T21:42:51.820048Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 6 partNo 4 FormedBlobsCount 0 NewHead: Offset 5 PartNo 0 PackedSize 2253229 count 1 nextOffset 6 batches 5
2025-10-09T21:42:51.820088Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 0
2025-10-09T21:42:51.820100Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 1
2025-10-09T21:42:51.820104Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 2
2025-10-09T21:42:51.820107Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 3
2025-10-09T21:42:51.820125Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 4
2025-10-09T21:42:51.820826Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 7 partNo 4 FormedBlobsCount 0 NewHead: Offset 5 PartNo 0 PackedSize 4506426 count 2 nextOffset 7 batches 9
2025-10-09T21:42:51.820840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.820947Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 5,2 HeadOffset 5 endOffset 5 curOffset 7 d0000000000_00000000000000000005_00000_0000000002_00008? size 4506416 WTime 1760046171818
2025-10-09T21:42:51.825400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:42:51.825462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 9  requestId: <none> cookie: 5
2025-10-09T21:42:51.826900Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 12 to 12
2025-10-09T21:42:51.827181Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.827197Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 12
2025-10-09T21:42:51.825754Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 read done: guid# ac24d051-dd3af3c1-cfcab5ee-3901469a, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 4505940
2025-10-09T21:42:51.825777Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 response to read: guid# ac24d051-dd3af3c1-cfcab5ee-3901469a
2025-10-09T21:42:51.831909Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:51.834486Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 Process answer. Aval parts: 0
2025-10-09T21:42:51.834555Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.834575Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.840696Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:51.840732Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 5 partNo 0 count 2 size 4506416
2025-10-09T21:42:51.845481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:51.845500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 3 count 1 size 717134 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:51.845521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:51.845525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:458: [72075186224037892][Partition][0][StateIdle] Blobs compaction is completed
2025-10-09T21:42:51.845633Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:42:51.845643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 3 partno 3 count 1 parts 1 suffix '124' size 717134
2025-10-09T21:42:51.847571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.847585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:51.849818Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got ReadResponse, serverBytesSize = 4505940, now ReadSizeBudget = 0, ReadSizeServerDelta = -3981652
2025-10-09T21:42:51.849954Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (2-2)
2025-10-09T21:42:51.849976Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 4505940 to budget
2025-10-09T21:42:51.853433Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.854177Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 13 to 13
2025-10-09T21:42:51.854524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.854538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 13
2025-10-09T21:42:51.854839Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (2-2)
2025-10-09T21:42:51.854857Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
2025-10-09T21:42:51.854489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.854508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.854567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 14  requestId: <none> cookie: 8
2025-10-09T21:42:51.856870Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 4 EndOffset 4 newEndOffset 5 commitOffset 0 clientCommitOffset 0 cookie 4 readingFinished 0 firstRead 0
2025-10-09T21:42:51.856894Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 4 endOffset 5
2025-10-09T21:42:51.856914Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.857307Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 5 offsets: 4 already_written: false write_statistics { persist_duration_ms: 78 queued_in_partition_duration_ms: 6 }
2025-10-09T21:42:51.857322Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 5
2025-10-09T21:42:51.858147Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (3-3)
2025-10-09T21:42:51.858169Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:42:51.857331Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 4, endOffset# 5, WTime# 1760046171692, sizeLag# 2253219
2025-10-09T21:42:51.857344Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:42:51.865472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.865496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.865587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 8 partNo : 0 messageNo: 15 size: 511959
2025-10-09T21:42:51.865773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 8 partNo : 1 messageNo: 15 size: 511959
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 2 ProducerId: "src_id" SeqNo: 3 CreateTime: 2025-10-09T21:42:51.636000Z WriteTime: 2025-10-09T21:42:51.689000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:57708", "server": "ipv6:[::1]:57708" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-09T21:42:51.869465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 14 to 14
2025-10-09T21:42:51.869526Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.874325Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.874345Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 14
2025-10-09T21:42:51.874407Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:51.873265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 8 partNo : 2 messageNo: 15 size: 511959
2025-10-09T21:42:51.873349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 8 partNo : 3 messageNo: 15 size: 511959
2025-10-09T21:42:51.873395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 8 partNo : 4 messageNo: 15 size: 205039
2025-10-09T21:42:51.873401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 8 partNo : 4 messageNo: 15 size 205039 offset: -1
2025-10-09T21:42:51.873467Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.873492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.875316Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.878534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 15 to 15
2025-10-09T21:42:51.881649Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.881669Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 15
2025-10-09T21:42:51.881677Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage 22528015[B] reached maximum (20971520[B])
2025-10-09T21:42:51.878739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.878756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.878872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 5 count 2 size 4506416 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:51.878884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 16  requestId: <none> cookie: 9
2025-10-09T21:42:51.878894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:51.878912Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 4505820 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:51.878921Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.878930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 6, partNo: 0, Offset: 5 is stored on disk
2025-10-09T21:42:51.878933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.878937Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 6, partNo: 1, Offset: 5 is stored on disk
2025-10-09T21:42:51.878939Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.878945Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 6, partNo: 2, Offset: 5 is stored on disk
2025-10-09T21:42:51.878946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.878949Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 6, partNo: 3, Offset: 5 is stored on disk
2025-10-09T21:42:51.878957Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.878961Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 6, partNo: 4, Offset: 5 is stored on disk
2025-10-09T21:42:51.878963Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.878966Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 7, partNo: 0, Offset: 6 is stored on disk
2025-10-09T21:42:51.878967Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.878971Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 7, partNo: 1, Offset: 6 is stored on disk
2025-10-09T21:42:51.878973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.878976Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 7, partNo: 2, Offset: 6 is stored on disk
2025-10-09T21:42:51.878977Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.878980Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 7, partNo: 3, Offset: 6 is stored on disk
2025-10-09T21:42:51.878982Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.878985Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 7, partNo: 4, Offset: 6 is stored on disk
2025-10-09T21:42:51.879184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.879197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.879200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (5)
2025-10-09T21:42:51.879227Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 0
2025-10-09T21:42:51.879240Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 1
2025-10-09T21:42:51.879243Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 2
2025-10-09T21:42:51.879247Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 3
2025-10-09T21:42:51.879250Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 4
2025-10-09T21:42:51.879734Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 8 partNo 4 FormedBlobsCount 0 NewHead: Offset 7 PartNo 0 PackedSize 2253229 count 1 nextOffset 8 batches 5
2025-10-09T21:42:51.879750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.879853Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 7,1 HeadOffset 7 endOffset 7 curOffset 8 d0000000000_00000000000000000007_00000_0000000001_00004? size 2253219 WTime 1760046171877
2025-10-09T21:42:51.881955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:51.882002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 11  requestId: <none> cookie: 6
2025-10-09T21:42:51.882025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 13  requestId: <none> cookie: 7
2025-10-09T21:42:51.882051Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:51.882064Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 0 count 1 size 2253219
2025-10-09T21:42:51.882070Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000000_00000_0000000001_00004?(+) to d0000000000_00000000000000000000_00000_0000000001_00004?(+)
2025-10-09T21:42:51.882072Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000001_00000_0000000001_00004?(+) to d0000000000_00000000000000000001_00000_0000000001_00004?(+)
2025-10-09T21:42:51.882073Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000002_00000_0000000002_00008?(+) to d0000000000_00000000000000000002_00000_0000000002_00008?(+)
2025-10-09T21:42:51.882232Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 2 parts 8 suffix '63' size 4506416
2025-10-09T21:42:51.884593Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 0} (3-3)
2025-10-09T21:42:51.884624Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
2025-10-09T21:42:51.884636Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In ContinueReadingDataImpl, ReadSizeBudget = 4505940, ReadSizeServerDelta = -3981652
2025-10-09T21:42:51.884775Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:42:51.884787Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:42:51.883957Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.883990Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.883998Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 3 ProducerId: "src_id" SeqNo: 4 CreateTime: 2025-10-09T21:42:51.651000Z WriteTime: 2025-10-09T21:42:51.692000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:57708", "logtype": "unknown", "_ip": "ipv6:[::1]:57708", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-09T21:42:51.888339Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 6 offsets: 5 already_written: false write_statistics { persist_duration_ms: 60 queued_in_partition_duration_ms: 25 }
2025-10-09T21:42:51.888356Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 6
2025-10-09T21:42:51.888361Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage got back to normal 20275214[B]
2025-10-09T21:42:51.889770Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 16 to 16
2025-10-09T21:42:51.887523Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.887863Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.890153Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.890175Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 16
2025-10-09T21:42:51.887957Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 grpc read done: success# 1, data# { read_request { bytes_size: 4505940 } }
2025-10-09T21:42:51.888035Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 got read request: guid# 11d524b5-3ee76ded-7c0ddfc3-f6cb6646
2025-10-09T21:42:51.888068Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 performing read request: guid# 68120ede-e680b54c-62d946cf-a6d60b96, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 1, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:42:51.888085Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 1 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 4 EndOffset 5 ClientCommitOffset 0 committedOffset 0 Guid 68120ede-e680b54c-62d946cf-a6d60b96
2025-10-09T21:42:51.890523Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage 22528016[B] reached maximum (20971520[B])
2025-10-09T21:42:51.890577Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:51.892462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.892484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.892563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 9 partNo : 0 messageNo: 17 size: 511959
2025-10-09T21:42:51.892620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 9 partNo : 1 messageNo: 17 size: 511959
2025-10-09T21:42:51.892695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 9 partNo : 2 messageNo: 17 size: 511959
2025-10-09T21:42:51.892753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 9 partNo : 3 messageNo: 17 size: 511959
2025-10-09T21:42:51.892790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 9 partNo : 4 messageNo: 17 size: 205039
2025-10-09T21:42:51.892795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 9 partNo : 4 messageNo: 17 size 205039 offset: -1
2025-10-09T21:42:51.892823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.892827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.892850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.892851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.892868Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.892893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.892919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 18  requestId: <none> cookie: 10
2025-10-09T21:42:51.892957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 7 Topic 'rt3.dc1--test-topic' partition 0 user user offset 4 count 1 size 524288 endOffset 7 max time lag 0ms effective offset 4
2025-10-09T21:42:51.892984Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 7 added 1 blobs, size 2253219 count 1 last offset 4, current partition end offset: 7
2025-10-09T21:42:51.892989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 7. Send blob request.
2025-10-09T21:42:51.893003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 4 partno 0 count 1 parts_count 4 source 1 size 2253219 accessed 0 times before, last time 2025-10-09T21:42:51.000000Z
2025-10-09T21:42:51.893015Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 7. All 1 blobs are from cache.
2025-10-09T21:42:51.893030Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 4 suffix '63'
2025-10-09T21:42:51.893032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:51.893232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 4 totakecount 1 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:42:51.893380Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 4 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.893517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 4 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.893885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 4 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.893956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 4 totakecount 1 count 1 size 205085 from pos 0 cbcount 1
2025-10-09T21:42:51.894004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 4
2025-10-09T21:42:51.893373Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.895219Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.895496Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.896395Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 7 offsets: 6 already_written: false write_statistics { persist_duration_ms: 58 queued_in_partition_duration_ms: 1 }
2025-10-09T21:42:51.896407Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 7
2025-10-09T21:42:51.896413Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage got back to normal 20275215[B]
2025-10-09T21:42:51.896825Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 17 to 17
2025-10-09T21:42:51.897140Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.897152Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 17
2025-10-09T21:42:51.897158Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage 22528017[B] reached maximum (20971520[B])
2025-10-09T21:42:51.897205Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:51.897246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.897259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.897339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 10 partNo : 0 messageNo: 19 size: 511959
2025-10-09T21:42:51.897397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 10 partNo : 1 messageNo: 19 size: 511959
2025-10-09T21:42:51.897454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 10 partNo : 2 messageNo: 19 size: 511959
2025-10-09T21:42:51.897509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 10 partNo : 3 messageNo: 19 size: 511959
2025-10-09T21:42:51.897537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 10 partNo : 4 messageNo: 19 size: 205039
2025-10-09T21:42:51.897543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 10 partNo : 4 messageNo: 19 size 205039 offset: -1
2025-10-09T21:42:51.897568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.897573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.897599Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.897632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.897784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 20  requestId: <none> cookie: 11
2025-10-09T21:42:51.899733Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 7 Result { Offset: 4 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 5 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1760046171738 CreateTimestampMS: 1760046171684 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 2253329 RealReadOffset: 4 WaitQuotaTimeMs: 0 EndOffset: 7 StartOffset: 0 } Cookie: 4 }
2025-10-09T21:42:51.900230Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 5 endOffset 7
2025-10-09T21:42:51.900241Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 7 ReadOffset 5 ReadGuid 68120ede-e680b54c-62d946cf-a6d60b96 has messages 1
2025-10-09T21:42:51.900270Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.901166Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 5, endOffset# 7, WTime# 1760046171738, sizeLag# 2253329
2025-10-09T21:42:51.901170Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1TEvPartitionReady. Aval parts: 0
2025-10-09T21:42:51.901184Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 read done: guid# 68120ede-e680b54c-62d946cf-a6d60b96, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 2252978
2025-10-09T21:42:51.901193Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 response to read: guid# 68120ede-e680b54c-62d946cf-a6d60b96
2025-10-09T21:42:51.902384Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 Process answer. Aval parts: 1
2025-10-09T21:42:51.903654Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got ReadResponse, serverBytesSize = 2252978, now ReadSizeBudget = 0, ReadSizeServerDelta = -1728690
2025-10-09T21:42:51.903757Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (4-4)
2025-10-09T21:42:51.903781Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 2252978 to budget
2025-10-09T21:42:51.906042Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.906387Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.906194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.906222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.906838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 11 partNo : 0 messageNo: 21 size: 511959
2025-10-09T21:42:51.906896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 11 partNo : 1 messageNo: 21 size: 511959
2025-10-09T21:42:51.906960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 11 partNo : 2 messageNo: 21 size: 511959
2025-10-09T21:42:51.907577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 11 partNo : 3 messageNo: 21 size: 511959
2025-10-09T21:42:51.907619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 11 partNo : 4 messageNo: 21 size: 205040
2025-10-09T21:42:51.907626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 11 partNo : 4 messageNo: 21 size 205040 offset: -1
2025-10-09T21:42:51.907740Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.907767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.907789Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 0 count 1 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:51.907795Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 1 count 1 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:51.907801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 2 count 2 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:51.907807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 2253219 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:51.907833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:51.907856Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 2252910 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:51.907863Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.907876Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 8, partNo: 0, Offset: 7 is stored on disk
2025-10-09T21:42:51.907879Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.907883Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 8, partNo: 1, Offset: 7 is stored on disk
2025-10-09T21:42:51.907887Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.907891Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 8, partNo: 2, Offset: 7 is stored on disk
2025-10-09T21:42:51.907893Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.907897Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 8, partNo: 3, Offset: 7 is stored on disk
2025-10-09T21:42:51.907903Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.907906Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 8, partNo: 4, Offset: 7 is stored on disk
2025-10-09T21:42:51.907941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.907956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.907958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (15)
2025-10-09T21:42:51.907984Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 0
2025-10-09T21:42:51.907996Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 1
2025-10-09T21:42:51.907999Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 2
2025-10-09T21:42:51.908002Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 3
2025-10-09T21:42:51.908006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 4
2025-10-09T21:42:51.908546Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 4 suffix '63' size 2253219
2025-10-09T21:42:51.908560Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 4 suffix '63' size 2253219
2025-10-09T21:42:51.908568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 2 parts 8 suffix '63' size 4506416
2025-10-09T21:42:51.908578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 4 suffix '63' size 2253219
2025-10-09T21:42:51.910793Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 9 partNo 4 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 2253229 count 1 nextOffset 9 batches 5
2025-10-09T21:42:51.910835Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 0
2025-10-09T21:42:51.910845Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 1
2025-10-09T21:42:51.910848Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 2
2025-10-09T21:42:51.910851Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 3
2025-10-09T21:42:51.910857Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 4
2025-10-09T21:42:51.912054Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 10 partNo 4 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 4506426 count 2 nextOffset 10 batches 9
2025-10-09T21:42:51.912072Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 0
2025-10-09T21:42:51.912078Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 1
2025-10-09T21:42:51.912081Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 2
2025-10-09T21:42:51.912084Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 3
2025-10-09T21:42:51.912091Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 4
2025-10-09T21:42:51.913181Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 11 partNo 4 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 6759624 count 3 nextOffset 11 batches 13
2025-10-09T21:42:51.913194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.913296Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 8,3 HeadOffset 8 endOffset 8 curOffset 11 d0000000000_00000000000000000008_00000_0000000003_00012? size 6759614 WTime 1760046171912
2025-10-09T21:42:51.919185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000004_00000_0000000001_00004?
2025-10-09T21:42:51.919200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000005_00000_0000000002_00008?
2025-10-09T21:42:51.919202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000007_00000_0000000001_00004?
2025-10-09T21:42:51.919205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 9012854 bytes
2025-10-09T21:42:51.919291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 15  requestId: <none> cookie: 8
2025-10-09T21:42:51.920262Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.920686Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 8 offsets: 7 already_written: false write_statistics { persist_duration_ms: 29 queued_in_partition_duration_ms: 4 }
2025-10-09T21:42:51.920696Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 8
2025-10-09T21:42:51.920701Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage got back to normal 20275216[B]
2025-10-09T21:42:51.922478Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 18 to 18
2025-10-09T21:42:51.922756Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.922767Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 18
2025-10-09T21:42:51.922775Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage 22528018[B] reached maximum (20971520[B])
2025-10-09T21:42:51.923080Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (4-4)
2025-10-09T21:42:51.923097Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
2025-10-09T21:42:51.923107Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In ContinueReadingDataImpl, ReadSizeBudget = 2252978, ReadSizeServerDelta = -1728690
2025-10-09T21:42:51.923203Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:42:51.923213Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:42:51.920527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.920540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.920561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.920567Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:51.920585Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 8 partNo 0 count 3 size 6759614
2025-10-09T21:42:51.920595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:51.920599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000004_00000_0000000001_00004?
2025-10-09T21:42:51.920610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000005_00000_0000000002_00008?
2025-10-09T21:42:51.920621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000007_00000_0000000001_00004?
2025-10-09T21:42:51.920635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:51.920700Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 4 partno 0 count 1 parts_count 4 source 1 size 2253219 accessed 1 times before, last time 2025-10-09T21:42:51.000000Z
2025-10-09T21:42:51.920703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 5 partno 0 count 2 parts_count 8 source 1 size 4506416 accessed 0 times before, last time 2025-10-09T21:42:51.000000Z
2025-10-09T21:42:51.920707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 7 partno 0 count 1 parts_count 4 source 1 size 2253219 accessed 0 times before, last time 2025-10-09T21:42:51.000000Z
2025-10-09T21:42:51.920719Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 3 blobs are from cache.
2025-10-09T21:42:51.920741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 22  requestId: <none> cookie: 12
2025-10-09T21:42:51.920764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:51.920769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000004_00000_0000000001_00004?
2025-10-09T21:42:51.920772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:51.920913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 0
2025-10-09T21:42:51.921059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 1
2025-10-09T21:42:51.921192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 2
2025-10-09T21:42:51.921326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 3
2025-10-09T21:42:51.921396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 4
2025-10-09T21:42:51.922794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 5 partNo 4 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 2253229 count 1 nextOffset 5 batches 5
2025-10-09T21:42:51.922807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000005_00000_0000000002_00008?
2025-10-09T21:42:51.922809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:51.923526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 0
2025-10-09T21:42:51.923679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 1
2025-10-09T21:42:51.923825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 2
2025-10-09T21:42:51.923972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 3
2025-10-09T21:42:51.924196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 4
2025-10-09T21:42:51.926080Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 4 suffix '63'
2025-10-09T21:42:51.926085Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 2 parts 8 suffix '63'
2025-10-09T21:42:51.926088Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 4 suffix '63'
2025-10-09T21:42:51.926327Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 grpc read done: success# 1, data# { read_request { bytes_size: 2252978 } }
2025-10-09T21:42:51.926400Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.926435Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 got read request: guid# 72990961-fd06ae75-c23d0c9c-558c42d3
2025-10-09T21:42:51.926471Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 performing read request: guid# 7f28a37c-2ef2b1a3-73af5457-53cecf58, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 2, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:42:51.926494Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 2 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 5 EndOffset 7 ClientCommitOffset 0 committedOffset 0 Guid 7f28a37c-2ef2b1a3-73af5457-53cecf58
2025-10-09T21:42:51.928918Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.929243Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.927073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 6 partNo 4 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 4506426 count 2 nextOffset 6 batches 9
2025-10-09T21:42:51.927095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 0
2025-10-09T21:42:51.927234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 1
2025-10-09T21:42:51.927379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 2
2025-10-09T21:42:51.927520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 3
2025-10-09T21:42:51.927581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 4
2025-10-09T21:42:51.928926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 7 partNo 4 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 6759623 count 3 nextOffset 7 batches 13
2025-10-09T21:42:51.928938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000007_00000_0000000001_00004?
2025-10-09T21:42:51.928943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:51.929081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 0
2025-10-09T21:42:51.929280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 1
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 4 ProducerId: "src_id" SeqNo: 5 CreateTime: 2025-10-09T21:42:51.684000Z WriteTime: 2025-10-09T21:42:51.738000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "server": "ipv6:[::1]:57708", "_ip": "ipv6:[::1]:57708", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-09T21:42:51.945241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:91: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '' seqNo 8 partNo 1 result is x0000000000_00000000000000000003_00003_0000000004_00014 size 7988778
2025-10-09T21:42:51.945404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 2
2025-10-09T21:42:51.945562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 3
2025-10-09T21:42:51.945740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 4
2025-10-09T21:42:51.945766Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1098: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000003_00003_0000000004_00014 new key d0000000000_00000000000000000003_00003_0000000004_00014 size 7988778 WTime 1760046171945
2025-10-09T21:42:51.946131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 8 partNo 4 FormedBlobsCount 1 NewHead: Offset 7 PartNo 1 PackedSize 1741198 count 1 nextOffset 8 batches 4
2025-10-09T21:42:51.946238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:537: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 7,1 HeadOffset 3 endOffset 4 curOffset 8 d0000000000_00000000000000000007_00001_0000000001_00003| size 1741188 WTime 1760046171945
2025-10-09T21:42:51.947992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.948001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.951071Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.953230Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.953581Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.950141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.950154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.950235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 12 partNo : 0 messageNo: 23 size: 511959
2025-10-09T21:42:51.950290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 12 partNo : 1 messageNo: 23 size: 511959
2025-10-09T21:42:51.950346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 12 partNo : 2 messageNo: 23 size: 511959
2025-10-09T21:42:51.950406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 12 partNo : 3 messageNo: 23 size: 511959
2025-10-09T21:42:51.950450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 12 partNo : 4 messageNo: 23 size: 205040
2025-10-09T21:42:51.950457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 12 partNo : 4 messageNo: 23 size 205040 offset: -1
2025-10-09T21:42:51.950486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.950490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.950502Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:51.950519Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 3 partNo 3 count 4 size 7988778
2025-10-09T21:42:51.950527Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 1 count 1 size 1741188
2025-10-09T21:42:51.950665Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.950690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.950822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 24  requestId: <none> cookie: 13
2025-10-09T21:42:51.950862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 8 Topic 'rt3.dc1--test-topic' partition 0 user user offset 5 count 2 size 524288 endOffset 8 max time lag 0ms effective offset 5
2025-10-09T21:42:51.950880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 8 added 1 blobs, size 4506416 count 2 last offset 5, current partition end offset: 8
2025-10-09T21:42:51.950884Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 8. Send blob request.
2025-10-09T21:42:51.950898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 5 partno 0 count 2 parts_count 8 source 1 size 4506416 accessed 1 times before, last time 2025-10-09T21:42:51.000000Z
2025-10-09T21:42:51.950903Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 8. All 1 blobs are from cache.
2025-10-09T21:42:51.950920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:51.951043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 2 parts 8 suffix '63'
2025-10-09T21:42:51.951517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 5 totakecount 2 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:42:51.952072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 5 totakecount 2 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.952760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 5 totakecount 2 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.953454Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 5 totakecount 2 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.954974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 5 totakecount 2 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:42:51.955119Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 6 totakecount 2 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.955259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 6 totakecount 2 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.955698Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 6 totakecount 2 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:51.955789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 6 totakecount 2 count 1 size 205085 from pos 0 cbcount 1
2025-10-09T21:42:51.958384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.958402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.958495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 13 partNo : 0 messageNo: 25 size: 511959
2025-10-09T21:42:51.959106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 13 partNo : 1 messageNo: 25 size: 511959
2025-10-09T21:42:51.959633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 13 partNo : 2 messageNo: 25 size: 511959
2025-10-09T21:42:51.960199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 13 partNo : 3 messageNo: 25 size: 511959
2025-10-09T21:42:51.960251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 13 partNo : 4 messageNo: 25 size: 205040
2025-10-09T21:42:51.960256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 13 partNo : 4 messageNo: 25 size 205040 offset: -1
2025-10-09T21:42:51.960384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.960389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.960417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 5
2025-10-09T21:42:51.960445Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.960468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.960484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 8 count 3 size 6759614 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:51.962004Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 3 parts 12 suffix '63' size 6759614
2025-10-09T21:42:51.963754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:51.963785Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 6758731 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:51.963793Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.963803Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 9, partNo: 0, Offset: 8 is stored on disk
2025-10-09T21:42:51.963806Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.963810Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 9, partNo: 1, Offset: 8 is stored on disk
2025-10-09T21:42:51.963814Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.963819Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 9, partNo: 2, Offset: 8 is stored on disk
2025-10-09T21:42:51.963821Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.963825Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 9, partNo: 3, Offset: 8 is stored on disk
2025-10-09T21:42:51.963832Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.963836Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 9, partNo: 4, Offset: 8 is stored on disk
2025-10-09T21:42:51.963838Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.963842Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 10, partNo: 0, Offset: 9 is stored on disk
2025-10-09T21:42:51.963844Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.963849Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 10, partNo: 1, Offset: 9 is stored on disk
2025-10-09T21:42:51.963851Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.963856Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 10, partNo: 2, Offset: 9 is stored on disk
2025-10-09T21:42:51.963857Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.963861Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 10, partNo: 3, Offset: 9 is stored on disk
2025-10-09T21:42:51.963864Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.963870Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 10, partNo: 4, Offset: 9 is stored on disk
2025-10-09T21:42:51.963874Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.963877Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 11, partNo: 0, Offset: 10 is stored on disk
2025-10-09T21:42:51.963879Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.963884Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 11, partNo: 1, Offset: 10 is stored on disk
2025-10-09T21:42:51.963886Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.963889Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 11, partNo: 2, Offset: 10 is stored on disk
2025-10-09T21:42:51.963891Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.963894Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 11, partNo: 3, Offset: 10 is stored on disk
2025-10-09T21:42:51.963897Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.963901Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 11, partNo: 4, Offset: 10 is stored on disk
2025-10-09T21:42:51.963948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.963962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.963964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (10)
2025-10-09T21:42:51.963990Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 0
2025-10-09T21:42:51.964002Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 1
2025-10-09T21:42:51.964006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 2
2025-10-09T21:42:51.964010Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 3
2025-10-09T21:42:51.964014Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 4
2025-10-09T21:42:51.964464Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 12 partNo 4 FormedBlobsCount 0 NewHead: Offset 11 PartNo 0 PackedSize 2253230 count 1 nextOffset 12 batches 5
2025-10-09T21:42:51.964485Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 0
2025-10-09T21:42:51.964492Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 1
2025-10-09T21:42:51.964498Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 2
2025-10-09T21:42:51.964501Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 3
2025-10-09T21:42:51.964504Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 4
2025-10-09T21:42:51.965146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 13 partNo 4 FormedBlobsCount 0 NewHead: Offset 11 PartNo 0 PackedSize 4506428 count 2 nextOffset 13 batches 9
2025-10-09T21:42:51.965158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.965648Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 11,2 HeadOffset 11 endOffset 11 curOffset 13 d0000000000_00000000000000000011_00000_0000000002_00008? size 4506418 WTime 1760046171964
2025-10-09T21:42:51.966725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:42:51.966762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 26  requestId: <none> cookie: 14
2025-10-09T21:42:51.966794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 17  requestId: <none> cookie: 9
2025-10-09T21:42:51.966811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 19  requestId: <none> cookie: 10
2025-10-09T21:42:51.966823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 21  requestId: <none> cookie: 11
2025-10-09T21:42:51.966831Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:51.966844Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 11 partNo 0 count 2 size 4506418
2025-10-09T21:42:51.967207Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 8 Result { Offset: 5 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 6 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1760046171817 CreateTimestampMS: 1760046171697 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } Result { Offset: 6 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 7 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1760046171818 CreateTimestampMS: 1760046171729 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 4506614 RealReadOffset: 6 WaitQuotaTimeMs: 0 EndOffset: 8 StartOffset: 0 } Cookie: 5 }
2025-10-09T21:42:51.967688Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 7 endOffset 8
2025-10-09T21:42:51.967699Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 8 ReadOffset 7 ReadGuid 7f28a37c-2ef2b1a3-73af5457-53cecf58 has messages 1
2025-10-09T21:42:51.967735Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.967768Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.967776Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.967780Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.969347Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.970542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 9 offsets: 8 already_written: false write_statistics { persist_duration_ms: 55 queued_in_partition_duration_ms: 14 }
2025-10-09T21:42:51.970561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 9
2025-10-09T21:42:51.970568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage got back to normal 20275217[B]
2025-10-09T21:42:51.969801Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 7, endOffset# 8, WTime# 1760046171818, sizeLag# 4506614
2025-10-09T21:42:51.969808Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1TEvPartitionReady. Aval parts: 0
2025-10-09T21:42:51.969826Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 read done: guid# 7f28a37c-2ef2b1a3-73af5457-53cecf58, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 4505940
2025-10-09T21:42:51.969836Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 response to read: guid# 7f28a37c-2ef2b1a3-73af5457-53cecf58
2025-10-09T21:42:51.971007Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 19 to 19
2025-10-09T21:42:51.971331Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.971343Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 19
2025-10-09T21:42:51.971369Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 Process answer. Aval parts: 1
2025-10-09T21:42:51.972799Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.973611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage 22528019[B] reached maximum (20971520[B])
2025-10-09T21:42:51.973663Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:51.974681Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.972911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.972931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.973260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 14 partNo : 0 messageNo: 27 size: 511959
2025-10-09T21:42:51.973590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 14 partNo : 1 messageNo: 27 size: 511959
2025-10-09T21:42:51.973990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 14 partNo : 2 messageNo: 27 size: 511959
2025-10-09T21:42:51.974063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 14 partNo : 3 messageNo: 27 size: 511959
2025-10-09T21:42:51.974270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 14 partNo : 4 messageNo: 27 size: 205040
2025-10-09T21:42:51.974277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 14 partNo : 4 messageNo: 27 size 205040 offset: -1
2025-10-09T21:42:51.974334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.974341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.974358Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.974387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.974406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 28  requestId: <none> cookie: 15
2025-10-09T21:42:51.977040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.977049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.977116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 15 partNo : 0 messageNo: 29 size: 511959
2025-10-09T21:42:51.977171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 15 partNo : 1 messageNo: 29 size: 511959
2025-10-09T21:42:51.977221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 15 partNo : 2 messageNo: 29 size: 511959
2025-10-09T21:42:51.977270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 15 partNo : 3 messageNo: 29 size: 511959
2025-10-09T21:42:51.977311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 15 partNo : 4 messageNo: 29 size: 205040
2025-10-09T21:42:51.977315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 15 partNo : 4 messageNo: 29 size 205040 offset: -1
2025-10-09T21:42:51.977339Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.977365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.977660Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.977966Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.978630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.978644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.978670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 30  requestId: <none> cookie: 16
2025-10-09T21:42:51.981587Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got ReadResponse, serverBytesSize = 4505940, now ReadSizeBudget = 0, ReadSizeServerDelta = -3981652
2025-10-09T21:42:51.981700Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 10 offsets: 9 already_written: false write_statistics { persist_duration_ms: 52 queued_in_partition_duration_ms: 13 }
2025-10-09T21:42:51.981720Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 10
2025-10-09T21:42:51.981726Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage got back to normal 20275218[B]
2025-10-09T21:42:51.981888Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 11 offsets: 10 already_written: false write_statistics { persist_duration_ms: 51 queued_in_partition_duration_ms: 4 }
2025-10-09T21:42:51.981900Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 11
2025-10-09T21:42:51.982073Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (5-5)
2025-10-09T21:42:51.982093Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 4505940 to budget
2025-10-09T21:42:51.980003Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.982992Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 20 to 20
2025-10-09T21:42:51.983275Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.983287Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 20
2025-10-09T21:42:51.982475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.982495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.982572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 16 partNo : 0 messageNo: 31 size: 511959
2025-10-09T21:42:51.982658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 16 partNo : 1 messageNo: 31 size: 511959
2025-10-09T21:42:51.982713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 16 partNo : 2 messageNo: 31 size: 511959
2025-10-09T21:42:51.982770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 16 partNo : 3 messageNo: 31 size: 511959
2025-10-09T21:42:51.982815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 16 partNo : 4 messageNo: 31 size: 205040
2025-10-09T21:42:51.982820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 16 partNo : 4 messageNo: 31 size 205040 offset: -1
2025-10-09T21:42:51.982860Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.982886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.983643Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.983945Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.984068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.984081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.984111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 32  requestId: <none> cookie: 17
2025-10-09T21:42:51.984549Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (5-5)
2025-10-09T21:42:51.984568Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
2025-10-09T21:42:51.984617Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (6-6)
2025-10-09T21:42:51.984632Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 0 to budget
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 5 ProducerId: "src_id" SeqNo: 6 CreateTime: 2025-10-09T21:42:51.697000Z WriteTime: 2025-10-09T21:42:51.817000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:57708", "_ip": "ipv6:[::1]:57708" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-09T21:42:51.986132Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.988312Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.989003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.989021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.989098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 17 partNo : 0 messageNo: 33 size: 511959
2025-10-09T21:42:51.989157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 17 partNo : 1 messageNo: 33 size: 511959
2025-10-09T21:42:51.989219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 17 partNo : 2 messageNo: 33 size: 511959
2025-10-09T21:42:51.989282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 17 partNo : 3 messageNo: 33 size: 511959
2025-10-09T21:42:51.989334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 17 partNo : 4 messageNo: 33 size: 205040
2025-10-09T21:42:51.989344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 17 partNo : 4 messageNo: 33 size 205040 offset: -1
2025-10-09T21:42:51.989368Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.989399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.990917Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.991199Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 21 to 21
2025-10-09T21:42:51.991262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 3 count 4 size 7988778 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:51.991274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 1741188 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:51.991294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:51.991298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:458: [72075186224037892][Partition][0][StateIdle] Blobs compaction is completed
2025-10-09T21:42:51.991649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.991654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:51.992534Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 3 partno 3 count 4 parts 14 suffix '0' size 7988778
2025-10-09T21:42:51.992551Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:42:51.992555Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 1 count 1 parts 3 suffix '124' size 1741188
2025-10-09T21:42:51.992593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 0 size 8295635 cause it's been evicted from L2. Actual L1 size: 7
2025-10-09T21:42:51.992600Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-09T21:42:51.993666Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:51.993680Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 21
2025-10-09T21:42:51.993743Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.995226Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-09T21:42:51.995275Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:51.996319Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.996634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.993595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.993613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.993642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 34  requestId: <none> cookie: 18
2025-10-09T21:42:51.997503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.997511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.997584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 18 partNo : 0 messageNo: 35 size: 511959
2025-10-09T21:42:51.997754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 18 partNo : 1 messageNo: 35 size: 511959
2025-10-09T21:42:51.997811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 18 partNo : 2 messageNo: 35 size: 511959
2025-10-09T21:42:51.997864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 18 partNo : 3 messageNo: 35 size: 511959
2025-10-09T21:42:51.997901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 18 partNo : 4 messageNo: 35 size: 205040
2025-10-09T21:42:51.997905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 18 partNo : 4 messageNo: 35 size 205040 offset: -1
2025-10-09T21:42:51.997926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.997930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.997941Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.997966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.997979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 36  requestId: <none> cookie: 19
2025-10-09T21:42:51.998088Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.000833Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.001211Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:52.002440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.002464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.002540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 19 partNo : 0 messageNo: 37 size: 511959
2025-10-09T21:42:52.003311Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.004847Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.005161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:52.002601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 19 partNo : 1 messageNo: 37 size: 511959
2025-10-09T21:42:52.002663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 19 partNo : 2 messageNo: 37 size: 511959
2025-10-09T21:42:52.002717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 19 partNo : 3 messageNo: 37 size: 511959
2025-10-09T21:42:52.002753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 19 partNo : 4 messageNo: 37 size: 205040
2025-10-09T21:42:52.002759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 19 partNo : 4 messageNo: 37 size 205040 offset: -1
2025-10-09T21:42:52.002782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.002786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.002797Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:52.002822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.002838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 38  requestId: <none> cookie: 20
2025-10-09T21:42:52.008437Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.007605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.007627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.007705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 20 partNo : 0 messageNo: 39 size: 511959
2025-10-09T21:42:52.007937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 20 partNo : 1 messageNo: 39 size: 511959
2025-10-09T21:42:52.008105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 20 partNo : 2 messageNo: 39 size: 511959
2025-10-09T21:42:52.008163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 20 partNo : 3 messageNo: 39 size: 511959
2025-10-09T21:42:52.008198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 20 partNo : 4 messageNo: 39 size: 205040
2025-10-09T21:42:52.008206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 20 partNo : 4 messageNo: 39 size 205040 offset: -1
2025-10-09T21:42:52.008235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.008239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.008251Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:52.008276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.008294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 40  requestId: <none> cookie: 21
2025-10-09T21:42:52.010974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.010995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.011070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 21 partNo : 0 messageNo: 41 size: 511959
2025-10-09T21:42:52.011127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 21 partNo : 1 messageNo: 41 size: 511959
2025-10-09T21:42:52.011194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 21 partNo : 2 messageNo: 41 size: 511959
2025-10-09T21:42:52.011256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 21 partNo : 3 messageNo: 41 size: 511959
2025-10-09T21:42:52.011299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 21 partNo : 4 messageNo: 41 size: 205040
2025-10-09T21:42:52.011305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 21 partNo : 4 messageNo: 41 size 205040 offset: -1
2025-10-09T21:42:52.011332Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:52.011359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.020778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.021084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 11 count 2 size 4506418 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.021114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:52.021138Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 4505822 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:52.021140Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 2 parts 8 suffix '63' size 4506418
2025-10-09T21:42:52.021166Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.021185Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 12, partNo: 0, Offset: 11 is stored on disk
2025-10-09T21:42:52.021196Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.021201Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 12, partNo: 1, Offset: 11 is stored on disk
2025-10-09T21:42:52.021203Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.021209Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 12, partNo: 2, Offset: 11 is stored on disk
2025-10-09T21:42:52.021214Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.021220Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 12, partNo: 3, Offset: 11 is stored on disk
2025-10-09T21:42:52.021227Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.021232Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 12, partNo: 4, Offset: 11 is stored on disk
2025-10-09T21:42:52.021234Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.021255Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 13, partNo: 0, Offset: 12 is stored on disk
2025-10-09T21:42:52.021257Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.021262Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 13, partNo: 1, Offset: 12 is stored on disk
2025-10-09T21:42:52.021264Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.021268Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 13, partNo: 2, Offset: 12 is stored on disk
2025-10-09T21:42:52.021270Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.021275Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 13, partNo: 3, Offset: 12 is stored on disk
2025-10-09T21:42:52.021277Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.021281Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 13, partNo: 4, Offset: 12 is stored on disk
2025-10-09T21:42:52.021356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.021408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.021415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (40)
2025-10-09T21:42:52.021446Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 0
2025-10-09T21:42:52.021464Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 1
2025-10-09T21:42:52.021473Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 2
2025-10-09T21:42:52.021478Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 3
2025-10-09T21:42:52.021482Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 4
2025-10-09T21:42:52.021961Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 14 partNo 4 FormedBlobsCount 0 NewHead: Offset 13 PartNo 0 PackedSize 2253230 count 1 nextOffset 14 batches 5
2025-10-09T21:42:52.021988Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 0
2025-10-09T21:42:52.021996Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 1
2025-10-09T21:42:52.022001Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 2
2025-10-09T21:42:52.022004Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 3
2025-10-09T21:42:52.022009Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 4
2025-10-09T21:42:52.022587Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 15 partNo 4 FormedBlobsCount 0 NewHead: Offset 13 PartNo 0 PackedSize 4506428 count 2 nextOffset 15 batches 9
2025-10-09T21:42:52.022603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 0
2025-10-09T21:42:52.022611Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 1
2025-10-09T21:42:52.022614Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 2
2025-10-09T21:42:52.022618Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 3
2025-10-09T21:42:52.022622Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 4
2025-10-09T21:42:52.023137Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 16 partNo 4 FormedBlobsCount 0 NewHead: Offset 13 PartNo 0 PackedSize 6759626 count 3 nextOffset 16 batches 13
2025-10-09T21:42:52.023163Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 0
2025-10-09T21:42:52.023174Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 1
2025-10-09T21:42:52.023179Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 2
2025-10-09T21:42:52.023183Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 3
2025-10-09T21:42:52.028975Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1380: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 17 partNo 3 result is x0000000000_00000000000000000013_00000_0000000003_00015? size 8295638
2025-10-09T21:42:52.029003Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 4
2025-10-09T21:42:52.029021Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1098: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000013_00000_0000000003_00015? new key d0000000000_00000000000000000013_00000_0000000003_00015? size 8295638 WTime 1760046172028
2025-10-09T21:42:52.029206Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 17 partNo 4 FormedBlobsCount 1 NewHead: Offset 16 PartNo 3 PackedSize 717145 count 1 nextOffset 17 batches 2
2025-10-09T21:42:52.029220Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 0
2025-10-09T21:42:52.029229Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 1
2025-10-09T21:42:52.029233Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 2
2025-10-09T21:42:52.029240Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 3
2025-10-09T21:42:52.029244Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 4
2025-10-09T21:42:52.030253Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 18 partNo 4 FormedBlobsCount 0 NewHead: Offset 16 PartNo 3 PackedSize 2970343 count 2 nextOffset 18 batches 6
2025-10-09T21:42:52.030283Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 0
2025-10-09T21:42:52.030289Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 1
2025-10-09T21:42:52.030293Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 2
2025-10-09T21:42:52.030297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 3
2025-10-09T21:42:52.030304Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 4
2025-10-09T21:42:52.030899Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 19 partNo 4 FormedBlobsCount 0 NewHead: Offset 16 PartNo 3 PackedSize 5223541 count 3 nextOffset 19 batches 10
2025-10-09T21:42:52.030923Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 0
2025-10-09T21:42:52.030932Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 1
2025-10-09T21:42:52.030936Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 2
2025-10-09T21:42:52.030939Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 3
2025-10-09T21:42:52.030945Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 4
2025-10-09T21:42:52.031500Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 20 partNo 4 FormedBlobsCount 0 NewHead: Offset 16 PartNo 3 PackedSize 7476739 count 4 nextOffset 20 batches 14
2025-10-09T21:42:52.031520Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 0
2025-10-09T21:42:52.031526Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 1
2025-10-09T21:42:52.041137Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1380: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 21 partNo 1 result is x0000000000_00000000000000000016_00003_0000000004_00014? size 7988760
2025-10-09T21:42:52.041174Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 2
2025-10-09T21:42:52.041184Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 3
2025-10-09T21:42:52.041189Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 4
2025-10-09T21:42:52.041202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1098: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000016_00003_0000000004_00014? new key d0000000000_00000000000000000016_00003_0000000004_00014? size 7988760 WTime 1760046172041
2025-10-09T21:42:52.041794Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 21 partNo 4 FormedBlobsCount 1 NewHead: Offset 20 PartNo 1 PackedSize 1741199 count 1 nextOffset 21 batches 4
2025-10-09T21:42:52.041818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.041944Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 20,1 HeadOffset 13 endOffset 13 curOffset 21 d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189 WTime 1760046172041
2025-10-09T21:42:52.042351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000008_00000_0000000003_00012?
2025-10-09T21:42:52.042354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000011_00000_0000000002_00008?
2025-10-09T21:42:52.042357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 2 keys were taken away. Let's read 11266032 bytes
2025-10-09T21:42:52.042452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 23  requestId: <none> cookie: 12
2025-10-09T21:42:52.042490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 25  requestId: <none> cookie: 13
2025-10-09T21:42:52.042500Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:52.042516Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 13 partNo 0 count 3 size 8295638
2025-10-09T21:42:52.042523Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 16 partNo 3 count 4 size 7988760
2025-10-09T21:42:52.042530Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 20 partNo 1 count 1 size 1741189
2025-10-09T21:42:52.042536Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000004_00000_0000000001_00004?(+) to d0000000000_00000000000000000004_00000_0000000001_00004?(+)
2025-10-09T21:42:52.042539Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000005_00000_0000000002_00008?(+) to d0000000000_00000000000000000005_00000_0000000002_00008?(+)
2025-10-09T21:42:52.042541Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00000_0000000001_00004?(+) to d0000000000_00000000000000000007_00000_0000000001_00004?(+)
2025-10-09T21:42:52.042544Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000003_00003_0000000001_00001|(+) to d0000000000_00000000000000000003_00003_0000000001_00001|(+)
2025-10-09T21:42:52.042554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 2 blobs
2025-10-09T21:42:52.042558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000008_00000_0000000003_00012?
2025-10-09T21:42:52.042569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000011_00000_0000000002_00008?
2025-10-09T21:42:52.042582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 2 blobs for compaction
2025-10-09T21:42:52.042658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 8 partno 0 count 3 parts_count 12 source 1 size 6759614 accessed 0 times before, last time 2025-10-09T21:42:51.000000Z
2025-10-09T21:42:52.042663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 11 partno 0 count 2 parts_count 8 source 1 size 4506418 accessed 0 times before, last time 2025-10-09T21:42:52.000000Z
2025-10-09T21:42:52.042676Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 2 blobs are from cache.
2025-10-09T21:42:52.042726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:52.042734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000008_00000_0000000003_00012?
2025-10-09T21:42:52.042736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:52.042761Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 3 parts 12 suffix '63'
2025-10-09T21:42:52.042767Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 2 parts 8 suffix '63'
2025-10-09T21:42:52.042891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 0
2025-10-09T21:42:52.043051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 1
2025-10-09T21:42:52.043308Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.043348Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.043735Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 12 offsets: 11 already_written: false write_statistics { persist_duration_ms: 56 queued_in_partition_duration_ms: 13 }
2025-10-09T21:42:52.043752Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 12
2025-10-09T21:42:52.043758Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage got back to normal 20275218[B]
2025-10-09T21:42:52.044184Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 22 to 22
2025-10-09T21:42:52.044478Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:52.043914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 2
2025-10-09T21:42:52.044079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 3
2025-10-09T21:42:52.044491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 22
2025-10-09T21:42:52.044305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 4
2025-10-09T21:42:52.044749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 9 partNo 4 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 2253229 count 1 nextOffset 9 batches 5
2025-10-09T21:42:52.044774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 0
2025-10-09T21:42:52.044921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 1
2025-10-09T21:42:52.045074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 2
2025-10-09T21:42:52.045232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 3
2025-10-09T21:42:52.045480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 4
2025-10-09T21:42:52.045744Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-09T21:42:52.046100Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 0} (6-6)
2025-10-09T21:42:52.046118Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
2025-10-09T21:42:52.046130Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In ContinueReadingDataImpl, ReadSizeBudget = 4505940, ReadSizeServerDelta = -3981652
2025-10-09T21:42:52.046206Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:42:52.046216Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:42:52.046044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 10 partNo 4 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 4506426 count 2 nextOffset 10 batches 9
2025-10-09T21:42:52.046060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 0
2025-10-09T21:42:52.046226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 1
2025-10-09T21:42:52.046375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 2
2025-10-09T21:42:52.046544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 3
2025-10-09T21:42:52.046640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 4
2025-10-09T21:42:52.047181Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 13 offsets: 12 already_written: false write_statistics { persist_duration_ms: 56 queued_in_partition_duration_ms: 5 }
2025-10-09T21:42:52.047191Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 13
2025-10-09T21:42:52.047196Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage got back to normal 20275218[B]
2025-10-09T21:42:52.049146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:91: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '' seqNo 11 partNo 4 result is x0000000000_00000000000000000007_00001_0000000003_00015 size 8295601
2025-10-09T21:42:52.049166Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1098: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000007_00001_0000000003_00015 new key d0000000000_00000000000000000007_00001_0000000003_00015 size 8295601 WTime 1760046172048
2025-10-09T21:42:52.049190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 11 partNo 4 FormedBlobsCount 1 NewHead: Offset 10 PartNo 4 PackedSize 205118 count 1 nextOffset 11 batches 1
2025-10-09T21:42:52.049198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000011_00000_0000000002_00008?
2025-10-09T21:42:52.049201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:52.049327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 0
2025-10-09T21:42:52.049473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 1
2025-10-09T21:42:52.049609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 2
2025-10-09T21:42:52.049750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 3
2025-10-09T21:42:52.049948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 4
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 6 ProducerId: "src_id" SeqNo: 7 CreateTime: 2025-10-09T21:42:51.729000Z WriteTime: 2025-10-09T21:42:51.818000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "server": "ipv6:[::1]:57708", "_ip": "ipv6:[::1]:57708", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-09T21:42:52.053503Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.053829Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:52.053914Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 grpc read done: success# 1, data# { read_request { bytes_size: 4505940 } }
2025-10-09T21:42:52.053957Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 got read request: guid# f7a2de75-7f791908-f075a9bd-22a91a99
2025-10-09T21:42:52.053981Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 performing read request: guid# 7fb7deb3-111f59af-bdd3b829-204df907, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 1, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:42:52.053998Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 1 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 7 EndOffset 8 ClientCommitOffset 0 committedOffset 0 Guid 7fb7deb3-111f59af-bdd3b829-204df907
2025-10-09T21:42:52.054665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 12 partNo 4 FormedBlobsCount 0 NewHead: Offset 10 PartNo 4 PackedSize 2458316 count 2 nextOffset 12 batches 5
2025-10-09T21:42:52.054687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 0
2025-10-09T21:42:52.054823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 1
2025-10-09T21:42:52.054965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 2
2025-10-09T21:42:52.055155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 3
2025-10-09T21:42:52.055233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 4
2025-10-09T21:42:52.055809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 13 partNo 4 FormedBlobsCount 0 NewHead: Offset 10 PartNo 4 PackedSize 4711514 count 3 nextOffset 13 batches 9
2025-10-09T21:42:52.055914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:537: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 10,3 HeadOffset 7 endOffset 8 curOffset 13 d0000000000_00000000000000000010_00004_0000000003_00008| size 4711504 WTime 1760046172055
2025-10-09T21:42:52.059307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.059325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.059349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.059353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.059361Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:52.059375Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 1 count 3 size 8295601
2025-10-09T21:42:52.059383Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 10 partNo 4 count 3 size 4711504
2025-10-09T21:42:52.059477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 42  requestId: <none> cookie: 22
2025-10-09T21:42:52.062205Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 23 to 23
2025-10-09T21:42:52.062164Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.064133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 9 Topic 'rt3.dc1--test-topic' partition 0 user user offset 7 count 1 size 524288 endOffset 13 max time lag 0ms effective offset 7
2025-10-09T21:42:52.064169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 9 added 1 blobs, size 0 count 0 last offset 3, current partition end offset: 13
2025-10-09T21:42:52.064429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 9. Send blob request.
2025-10-09T21:42:52.065211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.065229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.065313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 22 partNo : 0 messageNo: 43 size: 511959
2025-10-09T21:42:52.065381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 22 partNo : 1 messageNo: 43 size: 511959
2025-10-09T21:42:52.065441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 22 partNo : 2 messageNo: 43 size: 511959
2025-10-09T21:42:52.065501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 22 partNo : 3 messageNo: 43 size: 511959
2025-10-09T21:42:52.065535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 22 partNo : 4 messageNo: 43 size: 205040
2025-10-09T21:42:52.065539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 22 partNo : 4 messageNo: 43 size 205040 offset: -1
2025-10-09T21:42:52.065570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 3 partno 3 count 4 parts_count 14 source 1 size 7988778 accessed 0 times before, last time 2025-10-09T21:42:51.000000Z
2025-10-09T21:42:52.065585Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 9. All 1 blobs are from cache.
2025-10-09T21:42:52.065610Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:52.065636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.065648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:52.067932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 7 totakecount 4 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:42:52.068006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 7
2025-10-09T21:42:52.068519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: __TMP__REQUEST__MARKER__
2025-10-09T21:42:52.068528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.068567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 10 Topic 'rt3.dc1--test-topic' partition 0 user user offset 7 count 1 size 26214400 endOffset 13 max time lag 0ms effective offset 7
2025-10-09T21:42:52.068573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 10 added 0 blobs, size 0 count 0 last offset 7, current partition end offset: 13
2025-10-09T21:42:52.068617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1092: [72075186224037892][Partition][0][StateIdle] Reading cookie 10. All data is from uncompacted head.
2025-10-09T21:42:52.068621Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 0 blobs
2025-10-09T21:42:52.068639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: __TMP__REQUEST__MARKER__ cookie: 7
2025-10-09T21:42:52.069094Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 3 partno 3 count 4 parts 14 suffix '0'
2025-10-09T21:42:52.071324Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:52.071345Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 23
2025-10-09T21:42:52.072766Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-09T21:42:52.072822Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:52.075057Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 13 Result { Offset: 7 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 8 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1760046171877 CreateTimestampMS: 1760046171759 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 13314143 RealReadOffset: 7 WaitQuotaTimeMs: 0 EndOffset: 13 StartOffset: 0 } Cookie: 7 }
2025-10-09T21:42:52.075648Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 8 endOffset 13
2025-10-09T21:42:52.075677Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 13 ReadOffset 8 ReadGuid 7fb7deb3-111f59af-bdd3b829-204df907 has messages 1
2025-10-09T21:42:52.075718Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 8, endOffset# 13, WTime# 1760046171877, sizeLag# 13314143
2025-10-09T21:42:52.075725Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1TEvPartitionReady. Aval parts: 0
2025-10-09T21:42:52.075732Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.075739Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 read done: guid# 7fb7deb3-111f59af-bdd3b829-204df907, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 2252978
2025-10-09T21:42:52.075750Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 response to read: guid# 7fb7deb3-111f59af-bdd3b829-204df907
2025-10-09T21:42:52.077772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.077794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.077824Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:194: [72075186224037892][Partition][0][StateIdle] Reserve processing: maxWriteInflightSize riched. Partition: 0
2025-10-09T21:42:52.076167Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:52.077163Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 Process answer. Aval parts: 1
2025-10-09T21:42:52.078059Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got ReadResponse, serverBytesSize = 2252978, now ReadSizeBudget = 0, ReadSizeServerDelta = -1728690
2025-10-09T21:42:52.082108Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (7-7)
2025-10-09T21:42:52.082137Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 2252978 to budget
2025-10-09T21:42:52.106761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 4 count 1 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.106782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 5 count 2 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.106787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 7 count 1 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.106792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 3 count 1 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.106800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 13 count 3 size 8295638 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.106804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 16 count 4 size 7988760 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.106812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.106836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:52.106855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 4 suffix '63' size 2253219
2025-10-09T21:42:52.106857Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 18023288 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:52.106865Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.106878Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 2 parts 8 suffix '63' size 4506416
2025-10-09T21:42:52.106886Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 4 suffix '63' size 2253219
2025-10-09T21:42:52.106889Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 14, partNo: 0, Offset: 13 is stored on disk
2025-10-09T21:42:52.106893Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 3 partno 3 count 1 parts 1 suffix '124' size 717134
2025-10-09T21:42:52.106893Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.106900Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 14, partNo: 1, Offset: 13 is stored on disk
2025-10-09T21:42:52.106900Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 3 parts 15 suffix '63' size 8295638
2025-10-09T21:42:52.106902Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.106906Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 14, partNo: 2, Offset: 13 is stored on disk
2025-10-09T21:42:52.106908Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.106912Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 14, partNo: 3, Offset: 13 is stored on disk
2025-10-09T21:42:52.106916Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 7 partno 1 count 1 parts 3 suffix '124' size 1741188
2025-10-09T21:42:52.106918Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.106920Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 3 parts 12 suffix '63' size 6759614
2025-10-09T21:42:52.106923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 16 partno 3 count 4 parts 14 suffix '63' size 7988760
2025-10-09T21:42:52.106923Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 14, partNo: 4, Offset: 13 is stored on disk
2025-10-09T21:42:52.106926Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.106925Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:52.106929Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 15, partNo: 0, Offset: 14 is stored on disk
2025-10-09T21:42:52.106931Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.106935Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 15, partNo: 1, Offset: 14 is stored on disk
2025-10-09T21:42:52.106937Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.106941Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 15, partNo: 2, Offset: 14 is stored on disk
2025-10-09T21:42:52.106942Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.106946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 15, partNo: 3, Offset: 14 is stored on disk
2025-10-09T21:42:52.106949Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.106954Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 15, partNo: 4, Offset: 14 is stored on disk
2025-10-09T21:42:52.106956Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.106960Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 16, partNo: 0, Offset: 15 is stored on disk
2025-10-09T21:42:52.106961Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.106965Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 16, partNo: 1, Offset: 15 is stored on disk
2025-10-09T21:42:52.106966Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.106983Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 16, partNo: 2, Offset: 15 is stored on disk
2025-10-09T21:42:52.106985Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.106989Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 16, partNo: 3, Offset: 15 is stored on disk
2025-10-09T21:42:52.106992Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.106998Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 16, partNo: 4, Offset: 15 is stored on disk
2025-10-09T21:42:52.107001Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107004Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 17, partNo: 0, Offset: 16 is stored on disk
2025-10-09T21:42:52.107006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107011Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 17, partNo: 1, Offset: 16 is stored on disk
2025-10-09T21:42:52.107013Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107016Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 17, partNo: 2, Offset: 16 is stored on disk
2025-10-09T21:42:52.107018Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107022Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 17, partNo: 3, Offset: 16 is stored on disk
2025-10-09T21:42:52.107025Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107029Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 17, partNo: 4, Offset: 16 is stored on disk
2025-10-09T21:42:52.107030Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107034Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 18, partNo: 0, Offset: 17 is stored on disk
2025-10-09T21:42:52.107036Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107039Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 18, partNo: 1, Offset: 17 is stored on disk
2025-10-09T21:42:52.107041Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107046Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 18, partNo: 2, Offset: 17 is stored on disk
2025-10-09T21:42:52.107048Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107051Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 18, partNo: 3, Offset: 17 is stored on disk
2025-10-09T21:42:52.107053Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107056Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 18, partNo: 4, Offset: 17 is stored on disk
2025-10-09T21:42:52.107058Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107063Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 19, partNo: 0, Offset: 18 is stored on disk
2025-10-09T21:42:52.107065Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107069Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 19, partNo: 1, Offset: 18 is stored on disk
2025-10-09T21:42:52.107070Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107074Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 19, partNo: 2, Offset: 18 is stored on disk
2025-10-09T21:42:52.107075Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107079Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 19, partNo: 3, Offset: 18 is stored on disk
2025-10-09T21:42:52.107081Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107084Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 19, partNo: 4, Offset: 18 is stored on disk
2025-10-09T21:42:52.107086Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107089Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 20, partNo: 0, Offset: 19 is stored on disk
2025-10-09T21:42:52.107091Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107094Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 20, partNo: 1, Offset: 19 is stored on disk
2025-10-09T21:42:52.107105Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107108Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 20, partNo: 2, Offset: 19 is stored on disk
2025-10-09T21:42:52.107110Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107114Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 20, partNo: 3, Offset: 19 is stored on disk
2025-10-09T21:42:52.107116Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107119Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 20, partNo: 4, Offset: 19 is stored on disk
2025-10-09T21:42:52.107121Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107126Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 21, partNo: 0, Offset: 20 is stored on disk
2025-10-09T21:42:52.107128Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107132Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 21, partNo: 1, Offset: 20 is stored on disk
2025-10-09T21:42:52.107134Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107137Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 21, partNo: 2, Offset: 20 is stored on disk
2025-10-09T21:42:52.107139Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107142Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 21, partNo: 3, Offset: 20 is stored on disk
2025-10-09T21:42:52.107145Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.107148Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 21, partNo: 4, Offset: 20 is stored on disk
2025-10-09T21:42:52.107199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.107219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.107221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (5)
2025-10-09T21:42:52.107250Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 22 partNo 0
2025-10-09T21:42:52.107277Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 22 partNo 1
2025-10-09T21:42:52.107285Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 22 partNo 2
2025-10-09T21:42:52.107288Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 22 partNo 3
2025-10-09T21:42:52.107291Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 22 partNo 4
2025-10-09T21:42:52.107760Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 22 partNo 4 FormedBlobsCount 0 NewHead: Offset 21 PartNo 0 PackedSize 2253230 count 1 nextOffset 22 batches 5
2025-10-09T21:42:52.107784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.107905Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 21,1 HeadOffset 21 endOffset 21 curOffset 22 d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220 WTime 1760046172107
2025-10-09T21:42:52.110823Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.110860Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.110867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.110874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.110880Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.110885Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.110892Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.110898Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.110903Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.112683Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 14 offsets: 13 already_written: false write_statistics { persist_duration_ms: 85 queued_in_partition_duration_ms: 47 }
2025-10-09T21:42:52.112702Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 14
2025-10-09T21:42:52.112708Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage got back to normal 20275218[B]
2025-10-09T21:42:52.112776Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 15 offsets: 14 already_written: false write_statistics { persist_duration_ms: 84 queued_in_partition_duration_ms: 44 }
2025-10-09T21:42:52.112780Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 15
2025-10-09T21:42:52.112798Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 16 offsets: 15 already_written: false write_statistics { persist_duration_ms: 84 queued_in_partition_duration_ms: 39 }
2025-10-09T21:42:52.112800Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 16
2025-10-09T21:42:52.112817Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 17 offsets: 16 already_written: false write_statistics { persist_duration_ms: 83 queued_in_partition_duration_ms: 33 }
2025-10-09T21:42:52.112819Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 17
2025-10-09T21:42:52.112838Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 18 offsets: 17 already_written: false write_statistics { persist_duration_ms: 77 queued_in_partition_duration_ms: 31 }
2025-10-09T21:42:52.112840Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 18
2025-10-09T21:42:52.112857Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 19 offsets: 18 already_written: false write_statistics { persist_duration_ms: 76 queued_in_partition_duration_ms: 29 }
2025-10-09T21:42:52.112860Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 19
2025-10-09T21:42:52.112879Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 20 offsets: 19 already_written: false write_statistics { persist_duration_ms: 75 queued_in_partition_duration_ms: 23 }
2025-10-09T21:42:52.112882Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 20
2025-10-09T21:42:52.112900Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 21 offsets: 20 already_written: false write_statistics { persist_duration_ms: 75 queued_in_partition_duration_ms: 21 }
2025-10-09T21:42:52.112903Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 21
2025-10-09T21:42:52.113403Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 24 to 24
2025-10-09T21:42:52.113839Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:52.113861Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 24
2025-10-09T21:42:52.110518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:42:52.110590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 27  requestId: <none> cookie: 14
2025-10-09T21:42:52.110619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 8 size 6759614 cause it's been evicted from L2. Actual L1 size: 6
2025-10-09T21:42:52.110623Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 7 size 1741188 cause it's been evicted from L2. Actual L1 size: 5
2025-10-09T21:42:52.110626Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-09T21:42:52.110650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 29  requestId: <none> cookie: 15
2025-10-09T21:42:52.110672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 31  requestId: <none> cookie: 16
2025-10-09T21:42:52.110687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 33  requestId: <none> cookie: 17
2025-10-09T21:42:52.110702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 35  requestId: <none> cookie: 18
2025-10-09T21:42:52.110718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 37  requestId: <none> cookie: 19
2025-10-09T21:42:52.110731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 39  requestId: <none> cookie: 20
2025-10-09T21:42:52.110745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 41  requestId: <none> cookie: 21
2025-10-09T21:42:52.110753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 44  requestId: <none> cookie: 23
2025-10-09T21:42:52.110758Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:52.110769Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 21 partNo 0 count 1 size 2253220
2025-10-09T21:42:52.114804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.114818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.114928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 23 partNo : 0 messageNo: 45 size: 511959
2025-10-09T21:42:52.114990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 23 partNo : 1 messageNo: 45 size: 511959
2025-10-09T21:42:52.115048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 23 partNo : 2 messageNo: 45 size: 511959
2025-10-09T21:42:52.115105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 23 partNo : 3 messageNo: 45 size: 511959
2025-10-09T21:42:52.115287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 23 partNo : 4 messageNo: 45 size: 205040
2025-10-09T21:42:52.115291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 23 partNo : 4 messageNo: 45 size 205040 offset: -1
2025-10-09T21:42:52.115318Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:52.115345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.115770Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (7-7)
2025-10-09T21:42:52.115794Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
2025-10-09T21:42:52.115806Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In ContinueReadingDataImpl, ReadSizeBudget = 2252978, ReadSizeServerDelta = -1728690
2025-10-09T21:42:52.115892Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:42:52.115905Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:42:52.120014Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.120520Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:52.120633Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 grpc read done: success# 1, data# { read_request { bytes_size: 2252978 } }
2025-10-09T21:42:52.120677Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 got read request: guid# 2acf110b-1cce490c-183b8630-91536f78
2025-10-09T21:42:52.120702Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 performing read request: guid# 8addbbd0-f2bae237-bb90c452-2407df67, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 6, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:42:52.120725Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 6 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 8 EndOffset 13 ClientCommitOffset 0 committedOffset 0 Guid 8addbbd0-f2bae237-bb90c452-2407df67
2025-10-09T21:42:52.121356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.121400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.121409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.121440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.121443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.121485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 46  requestId: <none> cookie: 24
2025-10-09T21:42:52.121525Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 11 Topic 'rt3.dc1--test-topic' partition 0 user user offset 8 count 6 size 524288 endOffset 21 max time lag 0ms effective offset 8
2025-10-09T21:42:52.121551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 11 added 1 blobs, size 6759614 count 3 last offset 8, current partition end offset: 21
2025-10-09T21:42:52.121556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 11. Send blob request.
2025-10-09T21:42:52.121569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 8 partno 0 count 3 parts_count 12 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.121583Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 11. Have to read 1 of 1 from KV
2025-10-09T21:42:52.121689Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 8 partno 0 count 3 parts_count 12
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 7 ProducerId: "src_id" SeqNo: 8 CreateTime: 2025-10-09T21:42:51.759000Z WriteTime: 2025-10-09T21:42:51.877000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "server": "ipv6:[::1]:57708", "_ip": "ipv6:[::1]:57708", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-09T21:42:52.125385Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.128190Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 25 to 25
2025-10-09T21:42:52.128588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:52.128601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 25
2025-10-09T21:42:52.128016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.128039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.128114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 24 partNo : 0 messageNo: 47 size: 511959
2025-10-09T21:42:52.128165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 24 partNo : 1 messageNo: 47 size: 511959
2025-10-09T21:42:52.128239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 24 partNo : 2 messageNo: 47 size: 511959
2025-10-09T21:42:52.128330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 24 partNo : 3 messageNo: 47 size: 511959
2025-10-09T21:42:52.128370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 24 partNo : 4 messageNo: 47 size: 205040
2025-10-09T21:42:52.128377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 24 partNo : 4 messageNo: 47 size 205040 offset: -1
2025-10-09T21:42:52.128404Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:52.128431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.130749Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:52.134099Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.134449Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:52.134834Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 26 to 26
2025-10-09T21:42:52.135115Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:52.135128Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:52.136374Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:52.137621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.137642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.137684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 48  requestId: <none> cookie: 25
2025-10-09T21:42:52.139089Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.141940Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 27 to 27
2025-10-09T21:42:52.139204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 3 size 8295601 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.139214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 10 count 3 size 4711504 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.139233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:52.139236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:458: [72075186224037892][Partition][0][StateIdle] Blobs compaction is completed
2025-10-09T21:42:52.139980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.139988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000013_00000_0000000003_00015?
2025-10-09T21:42:52.139991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000016_00003_0000000004_00014?
2025-10-09T21:42:52.139995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 2 keys were taken away. Let's read 16284398 bytes
2025-10-09T21:42:52.140006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 2 blobs
2025-10-09T21:42:52.140009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000013_00000_0000000003_00015?
2025-10-09T21:42:52.140020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000016_00003_0000000004_00014?
2025-10-09T21:42:52.140032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 2 blobs for compaction
2025-10-09T21:42:52.140044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 13 partno 0 count 3 parts_count 15 source 1 size 8295638 accessed 0 times before, last time 2025-10-09T21:42:52.000000Z
2025-10-09T21:42:52.140048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 16 partno 3 count 4 parts_count 14 source 1 size 7988760 accessed 0 times before, last time 2025-10-09T21:42:52.000000Z
2025-10-09T21:42:52.140058Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 2 blobs are from cache.
2025-10-09T21:42:52.140099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:52.140103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000013_00000_0000000003_00015?
2025-10-09T21:42:52.140108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:52.140245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 0
2025-10-09T21:42:52.140385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 1
2025-10-09T21:42:52.140522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 2
2025-10-09T21:42:52.140599Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 2 parts 8 suffix '63' size 4506418
2025-10-09T21:42:52.140605Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 3 partno 3 count 4 parts 14 suffix '0' size 7988778
2025-10-09T21:42:52.140612Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 1 count 3 parts 15 suffix '0' size 8295601
2025-10-09T21:42:52.140617Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 10 partno 4 count 3 parts 8 suffix '124' size 4711504
2025-10-09T21:42:52.140629Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 3 parts 15 suffix '63'
2025-10-09T21:42:52.140632Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 16 partno 3 count 4 parts 14 suffix '63'
2025-10-09T21:42:52.140657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 3
2025-10-09T21:42:52.140967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 4
2025-10-09T21:42:52.142375Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:52.142391Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 27
2025-10-09T21:42:52.141971Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.142493Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:52.143822Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:52.145886Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 28 to 28
2025-10-09T21:42:52.150200Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:52.150219Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 28
2025-10-09T21:42:52.151535Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:52.147794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 14 partNo 4 FormedBlobsCount 0 NewHead: Offset 13 PartNo 0 PackedSize 2253230 count 1 nextOffset 14 batches 5
2025-10-09T21:42:52.147823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 0
2025-10-09T21:42:52.148555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 1
2025-10-09T21:42:52.148691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 2
2025-10-09T21:42:52.150131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:91: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '' seqNo 15 partNo 2 result is x0000000000_00000000000000000010_00004_0000000004_00014 size 7988760
2025-10-09T21:42:52.150269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 3
2025-10-09T21:42:52.150475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 4
2025-10-09T21:42:52.150492Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1098: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000010_00004_0000000004_00014 new key d0000000000_00000000000000000010_00004_0000000004_00014 size 7988760 WTime 1760046172150
2025-10-09T21:42:52.150699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 15 partNo 4 FormedBlobsCount 1 NewHead: Offset 14 PartNo 2 PackedSize 1229172 count 1 nextOffset 15 batches 3
2025-10-09T21:42:52.150711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 0
2025-10-09T21:42:52.150854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 1
2025-10-09T21:42:52.150992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 2
2025-10-09T21:42:52.151151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 3
2025-10-09T21:42:52.151220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 4
2025-10-09T21:42:52.151691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 16 partNo 4 FormedBlobsCount 0 NewHead: Offset 14 PartNo 2 PackedSize 3482370 count 2 nextOffset 16 batches 7
2025-10-09T21:42:52.152173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 0
2025-10-09T21:42:52.152179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 1
2025-10-09T21:42:52.152182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 2
2025-10-09T21:42:52.155862Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 29 to 29
2025-10-09T21:42:52.156157Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:52.156171Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 29
2025-10-09T21:42:52.152193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000016_00003_0000000004_00014?
2025-10-09T21:42:52.153945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:52.154109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 3
2025-10-09T21:42:52.154327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 4
2025-10-09T21:42:52.154874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 17 partNo 4 FormedBlobsCount 0 NewHead: Offset 14 PartNo 2 PackedSize 5735568 count 3 nextOffset 17 batches 11
2025-10-09T21:42:52.154886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 0
2025-10-09T21:42:52.155015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 1
2025-10-09T21:42:52.155156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 2
2025-10-09T21:42:52.155287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 3
2025-10-09T21:42:52.155463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 4
2025-10-09T21:42:52.158978Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:52.157860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 18 partNo 4 FormedBlobsCount 0 NewHead: Offset 14 PartNo 2 PackedSize 7988766 count 4 nextOffset 18 batches 15
2025-10-09T21:42:52.157879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 0
2025-10-09T21:42:52.160435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:91: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '' seqNo 19 partNo 0 result is x0000000000_00000000000000000014_00002_0000000004_00014 size 7988756
2025-10-09T21:42:52.163132Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 30 to 30
2025-10-09T21:42:52.163449Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:52.163463Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 30
2025-10-09T21:42:52.161127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 1
2025-10-09T21:42:52.161271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 2
2025-10-09T21:42:52.161409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 3
2025-10-09T21:42:52.161596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 4
2025-10-09T21:42:52.161613Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1098: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000014_00002_0000000004_00014 new key d0000000000_00000000000000000014_00002_0000000004_00014 size 7988756 WTime 1760046172161
2025-10-09T21:42:52.162363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 19 partNo 4 FormedBlobsCount 1 NewHead: Offset 18 PartNo 0 PackedSize 2253230 count 1 nextOffset 19 batches 5
2025-10-09T21:42:52.162377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 0
2025-10-09T21:42:52.162526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 1
2025-10-09T21:42:52.162663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 2
2025-10-09T21:42:52.162793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 3
2025-10-09T21:42:52.162932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 4
2025-10-09T21:42:52.163418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 20 partNo 4 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 4506428 count 2 nextOffset 20 batches 9
2025-10-09T21:42:52.163561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 0
2025-10-09T21:42:52.163653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:537: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 18,2 HeadOffset 10 endOffset 13 curOffset 20 d0000000000_00000000000000000018_00000_0000000002_00008| size 4506418 WTime 1760046172163
2025-10-09T21:42:52.165143Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:52.167150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 3 size 7988778 cause it's been evicted from L2. Actual L1 size: 6
2025-10-09T21:42:52.167165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 11 size 4506418 cause it's been evicted from L2. Actual L1 size: 5
2025-10-09T21:42:52.167172Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-09T21:42:52.168041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.168053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.168139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 25 partNo : 0 messageNo: 49 size: 511959
2025-10-09T21:42:52.168195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 25 partNo : 1 messageNo: 49 size: 511959
2025-10-09T21:42:52.168532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 25 partNo : 2 messageNo: 49 size: 511959
2025-10-09T21:42:52.168589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 25 partNo : 3 messageNo: 49 size: 511959
2025-10-09T21:42:52.170105Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 31 to 31
2025-10-09T21:42:52.170437Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:52.170453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 31
2025-10-09T21:42:52.168625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 25 partNo : 4 messageNo: 49 size: 205040
2025-10-09T21:42:52.168630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 25 partNo : 4 messageNo: 49 size 205040 offset: -1
2025-10-09T21:42:52.168676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.168681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.168727Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:52.168754Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 10 partNo 4 count 4 size 7988760
2025-10-09T21:42:52.168763Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 14 partNo 2 count 4 size 7988756
2025-10-09T21:42:52.168772Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 18 partNo 0 count 2 size 4506418
2025-10-09T21:42:52.168786Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:52.168813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.171047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 50  requestId: <none> cookie: 26
2025-10-09T21:42:52.171071Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:42:52.171074Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:52.171084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 8 count 3 size 6759614 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.171141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:52.171333Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 8 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:42:52.171484Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 8 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:52.171628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 8 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:52.171782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 8 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:52.172831Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 8 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:42:52.173008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 9 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:52.173491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 9 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:52.174489Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-09T21:42:52.174547Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:52.174365Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:52.174379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 7 partno 1 count 3 parts 15 suffix '0' size 8295601
2025-10-09T21:42:52.174384Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 3 parts 12 suffix '63' size 6759614
2025-10-09T21:42:52.174882Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.175053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 9 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:52.176742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.177095Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:52.175290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 9 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:42:52.175443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 10 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:52.175580Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 10 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:52.175722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 10 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:52.175796Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 10 totakecount 3 count 1 size 205086 from pos 0 cbcount 1
2025-10-09T21:42:52.175842Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 7 size 8295601 cause it's been evicted from L2. Actual L1 size: 5
2025-10-09T21:42:52.175845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 20 size 1741189 cause it's been evicted from L2. Actual L1 size: 4
2025-10-09T21:42:52.175850Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-09T21:42:52.175968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 8
2025-10-09T21:42:52.175995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.176062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:52.181528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:52.181560Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 2252911 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:52.181568Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.181580Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 22, partNo: 0, Offset: 21 is stored on disk
2025-10-09T21:42:52.181586Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.181591Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 22, partNo: 1, Offset: 21 is stored on disk
2025-10-09T21:42:52.181595Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.181599Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 22, partNo: 2, Offset: 21 is stored on disk
2025-10-09T21:42:52.181601Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.181604Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 22, partNo: 3, Offset: 21 is stored on disk
2025-10-09T21:42:52.181611Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.181615Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 22, partNo: 4, Offset: 21 is stored on disk
2025-10-09T21:42:52.181664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.181681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.181683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (15)
2025-10-09T21:42:52.181710Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 23 partNo 0
2025-10-09T21:42:52.181725Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 23 partNo 1
2025-10-09T21:42:52.181729Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 23 partNo 2
2025-10-09T21:42:52.181732Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 23 partNo 3
2025-10-09T21:42:52.181736Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 23 partNo 4
2025-10-09T21:42:52.182209Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 23 partNo 4 FormedBlobsCount 0 NewHead: Offset 22 PartNo 0 PackedSize 2253230 count 1 nextOffset 23 batches 5
2025-10-09T21:42:52.182231Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 24 partNo 0
2025-10-09T21:42:52.182238Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 24 partNo 1
2025-10-09T21:42:52.182241Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 24 partNo 2
2025-10-09T21:42:52.182245Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 24 partNo 3
2025-10-09T21:42:52.182248Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 24 partNo 4
2025-10-09T21:42:52.182747Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 24 partNo 4 FormedBlobsCount 0 NewHead: Offset 22 PartNo 0 PackedSize 4506428 count 2 nextOffset 24 batches 9
2025-10-09T21:42:52.182760Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 25 partNo 0
2025-10-09T21:42:52.182765Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 25 partNo 1
2025-10-09T21:42:52.182768Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 25 partNo 2
2025-10-09T21:42:52.182772Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 25 partNo 3
2025-10-09T21:42:52.182779Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 25 partNo 4
2025-10-09T21:42:52.183587Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 25 partNo 4 FormedBlobsCount 0 NewHead: Offset 22 PartNo 0 PackedSize 6759626 count 3 nextOffset 25 batches 13
2025-10-09T21:42:52.183599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.183718Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 22,3 HeadOffset 22 endOffset 22 curOffset 25 d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616 WTime 1760046172183
2025-10-09T21:42:52.187743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:42:52.189317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.189340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.189426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 51 size: 511959
2025-10-09T21:42:52.189862Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 21 Result { Offset: 8 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 9 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1760046171907 CreateTimestampMS: 1760046171769 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } Result { Offset: 9 Data: "... 2252875 bytes ..." SourceId: "\000src_id" SeqNo: 10 PartNo: 4 TotalParts: 5 TotalSize: 2252875 WriteTimestampMS: 1760046171910 CreateTimestampMS: 1760046171785 UncompressedSize: 2252801 PartitionKey: "" ExplicitHash: "" } Result { Offset: 10 Data: "... 2252876 bytes ..." SourceId: "\000src_id" SeqNo: 11 PartNo: 4 TotalParts: 5 TotalSize: 2252876 WriteTimestampMS: 1760046171911 CreateTimestampMS: 1760046171797 UncompressedSize: 2252802 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 1 BlobsFromCache: 0 SizeLag: 17820708 RealReadOffset: 10 WaitQuotaTimeMs: 0 EndOffset: 21 StartOffset: 0 } Cookie: 8 }
2025-10-09T21:42:52.189487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 51 size: 511959
2025-10-09T21:42:52.189553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 51 size: 511959
2025-10-09T21:42:52.189610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 51 size: 511959
2025-10-09T21:42:52.189656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 51 size: 205040
2025-10-09T21:42:52.189666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 51 size 205040 offset: -1
2025-10-09T21:42:52.189699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.189702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.189736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 43  requestId: <none> cookie: 22
2025-10-09T21:42:52.189763Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:52.189780Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 22 partNo 0 count 3 size 6759616
2025-10-09T21:42:52.189786Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000008_00000_0000000003_00012?(+) to d0000000000_00000000000000000008_00000_0000000003_00012?(+)
2025-10-09T21:42:52.189788Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000011_00000_0000000002_00008?(+) to d0000000000_00000000000000000011_00000_0000000002_00008?(+)
2025-10-09T21:42:52.189790Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00001_0000000001_00003|(+) to d0000000000_00000000000000000007_00001_0000000001_00003|(+)
2025-10-09T21:42:52.189799Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:52.189822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.189888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 52  requestId: <none> cookie: 27
2025-10-09T21:42:52.190410Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.190454Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.190764Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 11 endOffset 21
2025-10-09T21:42:52.190778Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 21 ReadOffset 11 ReadGuid 8addbbd0-f2bae237-bb90c452-2407df67 has messages 1
2025-10-09T21:42:52.190817Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 11, endOffset# 21, WTime# 1760046171911, sizeLag# 17820708
2025-10-09T21:42:52.190821Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1TEvPartitionReady. Aval parts: 0
2025-10-09T21:42:52.190835Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 read done: guid# 8addbbd0-f2bae237-bb90c452-2407df67, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 6758903
2025-10-09T21:42:52.190845Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 response to read: guid# 8addbbd0-f2bae237-bb90c452-2407df67
2025-10-09T21:42:52.192477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 22 offsets: 21 already_written: false write_statistics { persist_duration_ms: 74 queued_in_partition_duration_ms: 42 }
2025-10-09T21:42:52.192492Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 22
2025-10-09T21:42:52.192498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage got back to normal 20275218[B]
2025-10-09T21:42:52.194137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.194155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.194634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 53 size: 511959
2025-10-09T21:42:52.194710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 53 size: 511959
2025-10-09T21:42:52.194768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 53 size: 511959
2025-10-09T21:42:52.194827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 53 size: 511959
2025-10-09T21:42:52.194867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 53 size: 205040
2025-10-09T21:42:52.194873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 53 size 205040 offset: -1
2025-10-09T21:42:52.194916Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:52.194939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.196049Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 Process answer. Aval parts: 1
2025-10-09T21:42:52.196118Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.196431Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:52.197517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.197532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.197567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 54  requestId: <none> cookie: 28
2025-10-09T21:42:52.199252Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.201911Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got ReadResponse, serverBytesSize = 6758903, now ReadSizeBudget = 0, ReadSizeServerDelta = -6234615
2025-10-09T21:42:52.202014Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (8-8)
2025-10-09T21:42:52.202051Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 6758903 to budget
2025-10-09T21:42:52.204976Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.205325Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:52.208705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.208730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.208817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 55 size: 511959
2025-10-09T21:42:52.208992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 55 size: 511959
2025-10-09T21:42:52.209131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 55 size: 511959
2025-10-09T21:42:52.209299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 55 size: 511959
2025-10-09T21:42:52.209343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 55 size: 205040
2025-10-09T21:42:52.209349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 55 size 205040 offset: -1
2025-10-09T21:42:52.209402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.209407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.209421Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:52.209447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.209467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 56  requestId: <none> cookie: 29
2025-10-09T21:42:52.210060Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 32 to 32
2025-10-09T21:42:52.212082Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:52.212105Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 32
2025-10-09T21:42:52.213155Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-09T21:42:52.213473Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (8-8)
2025-10-09T21:42:52.213494Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
2025-10-09T21:42:52.213554Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (9-9)
2025-10-09T21:42:52.213574Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:42:52.214817Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.218644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.218672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.218757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 57 size: 511959
2025-10-09T21:42:52.218815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 57 size: 511959
2025-10-09T21:42:52.218873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 57 size: 511959
2025-10-09T21:42:52.218938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 57 size: 511959
2025-10-09T21:42:52.218977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 57 size: 205040
2025-10-09T21:42:52.218982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 57 size 205040 offset: -1
2025-10-09T21:42:52.219010Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:52.219043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.217674Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.219611Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:52.220986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.221002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.221039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 58  requestId: <none> cookie: 30
2025-10-09T21:42:52.221412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.221269Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.224379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.224398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.224477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 59 size: 511959
2025-10-09T21:42:52.224531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 59 size: 511959
2025-10-09T21:42:52.225220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 59 size: 511959
2025-10-09T21:42:52.226971Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.229633Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:52.225919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 59 size: 511959
2025-10-09T21:42:52.225975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 59 size: 205040
2025-10-09T21:42:52.225980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 59 size 205040 offset: -1
2025-10-09T21:42:52.226022Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:52.226060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.230482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.230498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.230533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 60  requestId: <none> cookie: 31
2025-10-09T21:42:52.230675Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 8 ProducerId: "src_id" SeqNo: 9 CreateTime: 2025-10-09T21:42:51.769000Z WriteTime: 2025-10-09T21:42:51.907000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:57708", "server": "ipv6:[::1]:57708" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-09T21:42:52.233656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.233671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.233751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 61 size: 511959
2025-10-09T21:42:52.234330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 61 size: 511959
2025-10-09T21:42:52.234393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 61 size: 511959
2025-10-09T21:42:52.234465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 61 size: 511959
2025-10-09T21:42:52.234508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 61 size: 205040
2025-10-09T21:42:52.234513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 61 size 205040 offset: -1
2025-10-09T21:42:52.234540Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:52.234568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.235650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 10 count 4 size 7988760 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.235660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 14 count 4 size 7988756 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.235665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 18 count 2 size 4506418 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.235685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:52.235690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:458: [72075186224037892][Partition][0][StateIdle] Blobs compaction is completed
2025-10-09T21:42:52.236360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.236365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:52.236506Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 10 partno 4 count 3 parts 8 suffix '124' size 4711504
2025-10-09T21:42:52.236516Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 10 partno 4 count 4 parts 14 suffix '0' size 7988760
2025-10-09T21:42:52.236523Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 3 parts 15 suffix '63' size 8295638
2025-10-09T21:42:52.236527Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 14 partno 2 count 4 parts 14 suffix '0' size 7988756
2025-10-09T21:42:52.236533Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 16 partno 3 count 4 parts 14 suffix '63' size 7988760
2025-10-09T21:42:52.236537Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 2 parts 8 suffix '124' size 4506418
2025-10-09T21:42:52.236975Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.237424Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:52.237544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 16 size 7988760 cause it's been evicted from L2. Actual L1 size: 7
2025-10-09T21:42:52.237556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 10 size 4711504 cause it's been evicted from L2. Actual L1 size: 6
2025-10-09T21:42:52.237559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 13 size 8295638 cause it's been evicted from L2. Actual L1 size: 5
2025-10-09T21:42:52.237563Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--test-topic, tablet id72075186224037892, cookie 0
2025-10-09T21:42:52.237908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.237921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.237954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 62  requestId: <none> cookie: 32
2025-10-09T21:42:52.238761Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.243859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.243884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.243967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 63 size: 511959
2025-10-09T21:42:52.244025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 63 size: 511959
2025-10-09T21:42:52.244083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 63 size: 511959
2025-10-09T21:42:52.244138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 63 size: 511959
2025-10-09T21:42:52.244175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 63 size: 205040
2025-10-09T21:42:52.244180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 63 size 205040 offset: -1
2025-10-09T21:42:52.244230Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:52.244257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.246179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 8 count 3 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.246198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.246227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:52.246244Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 6758733 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:52.246249Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.246260Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 23, partNo: 0, Offset: 22 is stored on disk
2025-10-09T21:42:52.246264Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.246267Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 23, partNo: 1, Offset: 22 is stored on disk
2025-10-09T21:42:52.246268Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.246271Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 23, partNo: 2, Offset: 22 is stored on disk
2025-10-09T21:42:52.246272Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.246275Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 23, partNo: 3, Offset: 22 is stored on disk
2025-10-09T21:42:52.246283Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.246285Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 23, partNo: 4, Offset: 22 is stored on disk
2025-10-09T21:42:52.246287Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.246290Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 24, partNo: 0, Offset: 23 is stored on disk
2025-10-09T21:42:52.246292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.246294Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 24, partNo: 1, Offset: 23 is stored on disk
2025-10-09T21:42:52.246295Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.246299Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 24, partNo: 2, Offset: 23 is stored on disk
2025-10-09T21:42:52.246300Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.246302Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 24, partNo: 3, Offset: 23 is stored on disk
2025-10-09T21:42:52.246304Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.246306Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 24, partNo: 4, Offset: 23 is stored on disk
2025-10-09T21:42:52.246308Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.246312Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 25, partNo: 0, Offset: 24 is stored on disk
2025-10-09T21:42:52.246314Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.246318Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 25, partNo: 1, Offset: 24 is stored on disk
2025-10-09T21:42:52.246320Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.246322Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 3 parts 12 suffix '63' size 6759614
2025-10-09T21:42:52.246323Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 25, partNo: 2, Offset: 24 is stored on disk
2025-10-09T21:42:52.246324Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.246327Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 25, partNo: 3, Offset: 24 is stored on disk
2025-10-09T21:42:52.246329Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:52.246333Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 25, partNo: 4, Offset: 24 is stored on disk
2025-10-09T21:42:52.246342Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:52.246372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.246393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.246395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (35)
2025-10-09T21:42:52.246430Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 0
2025-10-09T21:42:52.246440Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 1
2025-10-09T21:42:52.246442Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 2
2025-10-09T21:42:52.246445Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 3
2025-10-09T21:42:52.246447Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 4
2025-10-09T21:42:52.246873Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 26 partNo 4 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 2253230 count 1 nextOffset 26 batches 5
2025-10-09T21:42:52.246888Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 0
2025-10-09T21:42:52.246893Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 1
2025-10-09T21:42:52.246896Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 2
2025-10-09T21:42:52.246900Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 3
2025-10-09T21:42:52.246903Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 4
2025-10-09T21:42:52.247369Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 27 partNo 4 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 4506428 count 2 nextOffset 27 batches 9
2025-10-09T21:42:52.247380Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 0
2025-10-09T21:42:52.247383Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 1
2025-10-09T21:42:52.247386Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 2
2025-10-09T21:42:52.247388Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 3
2025-10-09T21:42:52.247392Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 4
2025-10-09T21:42:52.247912Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 28 partNo 4 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 6759626 count 3 nextOffset 28 batches 13
2025-10-09T21:42:52.247927Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 0
2025-10-09T21:42:52.247933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 1
2025-10-09T21:42:52.247936Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 2
2025-10-09T21:42:52.247940Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 3
2025-10-09T21:42:52.249291Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1380: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 29 partNo 3 result is x0000000000_00000000000000000025_00000_0000000003_00015? size 8295638
2025-10-09T21:42:52.249301Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 4
2025-10-09T21:42:52.249315Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1098: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000025_00000_0000000003_00015? new key d0000000000_00000000000000000025_00000_0000000003_00015? size 8295638 WTime 1760046172248
2025-10-09T21:42:52.249450Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 29 partNo 4 FormedBlobsCount 1 NewHead: Offset 28 PartNo 3 PackedSize 717145 count 1 nextOffset 29 batches 2
2025-10-09T21:42:52.249460Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 0
2025-10-09T21:42:52.249466Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 1
2025-10-09T21:42:52.249470Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 2
2025-10-09T21:42:52.249473Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 3
2025-10-09T21:42:52.249477Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 4
2025-10-09T21:42:52.249985Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 30 partNo 4 FormedBlobsCount 0 NewHead: Offset 28 PartNo 3 PackedSize 2970343 count 2 nextOffset 30 batches 6
2025-10-09T21:42:52.249998Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 0
2025-10-09T21:42:52.250003Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 1
2025-10-09T21:42:52.250007Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 2
2025-10-09T21:42:52.250011Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 3
2025-10-09T21:42:52.250017Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 4
2025-10-09T21:42:52.250606Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 31 partNo 4 FormedBlobsCount 0 NewHead: Offset 28 PartNo 3 PackedSize 5223541 count 3 nextOffset 31 batches 10
2025-10-09T21:42:52.250630Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 0
2025-10-09T21:42:52.252973Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.252997Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.253001Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.253340Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 23 offsets: 22 already_written: false write_statistics { persist_duration_ms: 64 queued_in_partition_duration_ms: 66 }
2025-10-09T21:42:52.253357Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 23
2025-10-09T21:42:52.253362Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage got back to normal 20275218[B]
2025-10-09T21:42:52.253741Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 33 to 33
2025-10-09T21:42:52.254008Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:52.254020Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 33
2025-10-09T21:42:52.250637Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 1
2025-10-09T21:42:52.250645Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 2
2025-10-09T21:42:52.250649Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 3
2025-10-09T21:42:52.250653Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 4
2025-10-09T21:42:52.251317Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 32 partNo 4 FormedBlobsCount 0 NewHead: Offset 28 PartNo 3 PackedSize 7476739 count 4 nextOffset 32 batches 14
2025-10-09T21:42:52.251332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.251441Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 28,4 HeadOffset 25 endOffset 25 curOffset 32 d0000000000_00000000000000000028_00003_0000000004_00013? size 7476729 WTime 1760046172251
2025-10-09T21:42:52.252307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.252309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:52.252312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:52.252315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:52.252356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 45  requestId: <none> cookie: 23
2025-10-09T21:42:52.252381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 47  requestId: <none> cookie: 24
2025-10-09T21:42:52.252395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 49  requestId: <none> cookie: 25
2025-10-09T21:42:52.252402Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:52.252411Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 25 partNo 0 count 3 size 8295638
2025-10-09T21:42:52.252414Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 28 partNo 3 count 4 size 7476729
2025-10-09T21:42:52.252419Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000013_00000_0000000003_00015?(+) to d0000000000_00000000000000000013_00000_0000000003_00015?(+)
2025-10-09T21:42:52.252420Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000016_00003_0000000004_00014?(+) to d0000000000_00000000000000000016_00003_0000000004_00014?(+)
2025-10-09T21:42:52.252421Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000010_00004_0000000003_00008|(+) to d0000000000_00000000000000000010_00004_0000000003_00008|(+)
2025-10-09T21:42:52.252427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:52.252429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.252435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:52.252440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:52.252448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:52.252509Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.252513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 21 partno 0 count 1 parts_count 4 source 1 size 2253220 accessed 0 times before, last time 2025-10-09T21:42:52.000000Z
2025-10-09T21:42:52.252515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 22 partno 0 count 3 parts_count 12 source 1 size 6759616 accessed 0 times before, last time 2025-10-09T21:42:52.000000Z
2025-10-09T21:42:52.252531Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 1 of 3 from KV
2025-10-09T21:42:52.252628Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63'
2025-10-09T21:42:52.252630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63'
2025-10-09T21:42:52.252634Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:52.255271Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-09T21:42:52.255331Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:52.256364Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.256627Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:52.256782Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 24 offsets: 23 already_written: false write_statistics { persist_duration_ms: 64 queued_in_partition_duration_ms: 53 }
2025-10-09T21:42:52.256801Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 24
2025-10-09T21:42:52.256805Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage got back to normal 20275218[B]
2025-10-09T21:42:52.256871Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session got write response: sequence_numbers: 25 offsets: 24 already_written: false write_statistics { persist_duration_ms: 63 queued_in_partition_duration_ms: 13 }
2025-10-09T21:42:52.256874Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: acknoledged message 25
2025-10-09T21:42:52.256829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.256847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.256882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 64  requestId: <none> cookie: 33
2025-10-09T21:42:52.258769Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 34 to 34
2025-10-09T21:42:52.259053Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:52.259063Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 34
2025-10-09T21:42:52.260653Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 0} (9-9)
2025-10-09T21:42:52.260678Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] The application data is transferred to the client. Number of messages 1, size 2252801 bytes
2025-10-09T21:42:52.260817Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (10-10)
2025-10-09T21:42:52.260833Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Returning serverBytesSize = 0 to budget
>>> Got event: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 Message { Data: ..2252801 bytes.. Information: { Offset: 9 ProducerId: "src_id" SeqNo: 10 CreateTime: 2025-10-09T21:42:51.785000Z WriteTime: 2025-10-09T21:42:51.910000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "server": "ipv6:[::1]:57708", "_ip": "ipv6:[::1]:57708", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 } }
2025-10-09T21:42:52.260345Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.263090Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.263386Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:52.264868Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write 1 messages with Id from 35 to 35
2025-10-09T21:42:52.265142Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: try to update token
2025-10-09T21:42:52.265152Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Send 1 message(s) (0 left), first sequence number is 35
2025-10-09T21:42:52.260806Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 3 from KV. Status 1
2025-10-09T21:42:52.260821Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:52.260832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340740487661485:2380]
2025-10-09T21:42:52.260879Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:52.260885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:52.260893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.260898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:52.261037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 1
2025-10-09T21:42:52.261182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 2
2025-10-09T21:42:52.261339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 3
2025-10-09T21:42:52.261416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:57: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 4
2025-10-09T21:42:52.261836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:130: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '' seqNo 21 partNo 4 FormedBlobsCount 0 NewHead: Offset 20 PartNo 1 PackedSize 2253230 count 1 nextOffset 21 batches 5
2025-10-09T21:42:52.261853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:52.261854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:52.261992Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:52.267168Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Estimated memory usage 22528020[B] reached maximum (20971520[B])
2025-10-09T21:42:52.267215Z :NOTICE: [/Invalid] [/Invalid] [adac221e-bd80b861-11dbce39-e4399803] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to establish connection to server "localhost:11460" ( cluster ). Attempts done: 1     <main>: Error: Database Invalid does not exist, code: 200202     <main>: Error: Endpoint list is empty for database /Invalid, cluster endpoint localhost:11460. " }
2025-10-09T21:42:52.491426Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::ExecRequestForCompaction(NKikimr::NPQ::TWriteMsg&, NKikimr::NPQ::TPartition::TProcessParametersBase&, NKikimr::TEvKeyValue::TEvRequest*, TInstant)+1759 (0x2452124F)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+981 (0x245266B5)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:52.494938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.494967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.495054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 65 size: 511959
2025-10-09T21:42:52.495111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 65 size: 511959
2025-10-09T21:42:52.495170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 65 size: 511959
2025-10-09T21:42:52.495227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 65 size: 511959
2025-10-09T21:42:52.495270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 65 size: 205040
2025-10-09T21:42:52.495276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 65 size 205040 offset: -1
2025-10-09T21:42:52.495317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.495322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.496661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_1_7577822143567276789_v1
2025-10-09T21:42:52.496684Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 55 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:52.496691Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 57 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:52.496695Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 59 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:52.496699Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 61 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:52.496703Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 63 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:52.496706Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 65 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:52.496710Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 66 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:52.496714Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 51 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:52.496719Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 53 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:52.497053Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_1_7577822143567276789_v1
2025-10-09T21:42:52.501048Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340745144436854:2476]
2025-10-09T21:42:52.501089Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.501171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0
2025-10-09T21:42:52.501349Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2 sessionId: src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0 is DEAD
2025-10-09T21:42:52.501545Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:52.501921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:42:52.501930Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session will restart in 0.008921s
2025-10-09T21:42:52.501960Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: Do CDS request
2025-10-09T21:42:52.501966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:52.512610Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:52.512984Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:52.513000Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 3
2025-10-09T21:42:52.513218Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 3 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:52.513248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 3 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:52.513254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=3 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:52.513258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:52.513802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 3 sessionId:  describe result for acl check
2025-10-09T21:42:52.513838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:52.513842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:52.513845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:52.513852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340749439404290:2487] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:52.514455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340749439404290:2487] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:52.519769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340749439404290:2487] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:52.523394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:52.524062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:52.524259Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:52.524262Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:52.524273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:52.524329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:52.524366Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340749077596337:2406]
2025-10-09T21:42:52.524900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:52.525295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340749439404290:2487] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:52.525311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340749439404290:2487] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:52.525315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340749439404290:2487] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:52.525325Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 3 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:52.526110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:52.526165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:52.526499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:52.526572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:52.526573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:52.526575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:52.526576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000003_00008|
2025-10-09T21:42:52.526578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:52.526579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:52.526580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:52.526581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000013_00000_0000000003_00015?
2025-10-09T21:42:52.526583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000016_00003_0000000004_00014?
2025-10-09T21:42:52.526585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[9]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.526586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[10]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:52.526588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[11]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:52.526593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:52.526603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:52.526607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:52.526610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000003_00008|
2025-10-09T21:42:52.526615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:574: replace key d0000000000_00000000000000000010_00004_0000000003_00008| to d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:52.526619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:52.526622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:52.526627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:603: ignore key d0000000000_00000000000000000013_00000_0000000003_00015?
2025-10-09T21:42:52.526631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:603: ignore key d0000000000_00000000000000000016_00003_0000000004_00014?
2025-10-09T21:42:52.526634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.526638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:52.526641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:52.526649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:52.526656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:52.526663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:52.526667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:52.526674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:52.526678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:52.526683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000003_00008|
2025-10-09T21:42:52.526688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:665: [rt3.dc1--test-topic:0:TInitDataRangeStep] unknown key d0000000000_00000000000000000010_00004_0000000003_00008| will be deleted
2025-10-09T21:42:52.526692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:52.526697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:52.526702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000013_00000_0000000003_00015?
2025-10-09T21:42:52.526706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:665: [rt3.dc1--test-topic:0:TInitDataRangeStep] unknown key d0000000000_00000000000000000013_00000_0000000003_00015? will be deleted
2025-10-09T21:42:52.526708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:52.526712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:52.526716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000016_00003_0000000004_00014?
2025-10-09T21:42:52.526721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:665: [rt3.dc1--test-topic:0:TInitDataRangeStep] unknown key d0000000000_00000000000000000016_00003_0000000004_00014? will be deleted
2025-10-09T21:42:52.526724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:52.526730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:52.526735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.526740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.526743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:52.526749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:52.526753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:52.526758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:52.526784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:52.537977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:52.539255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:52.539274Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:52.539278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:52.539283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:52.539292Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2 [2:7559340749077596337:2406]
2025-10-09T21:42:52.539308Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:52.539333Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:52.539336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.539339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.539343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.539385Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:52.539392Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:52.539400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:52.539445Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:52.539456Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000016_00003_0000000004_00014?(+) to d0000000000_00000000000000000016_00003_0000000004_00014?(+)
2025-10-09T21:42:52.539459Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000013_00000_0000000003_00015?(+) to d0000000000_00000000000000000013_00000_0000000003_00015?(+)
2025-10-09T21:42:52.539462Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000010_00004_0000000003_00008|(+) to d0000000000_00000000000000000010_00004_0000000003_00008|(+)
2025-10-09T21:42:52.539502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.539504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:52.539506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:52.539510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:52.539643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:52.539647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.539659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:52.539671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:52.539690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:52.539722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340749077596331:2406]
2025-10-09T21:42:52.539725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340749077596331:2406]
2025-10-09T21:42:52.539727Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340749077596331:2406]
2025-10-09T21:42:52.539743Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:52.539758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:52.539773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:52.539776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:52.539927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340749077596331:2406]
2025-10-09T21:42:52.539930Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:52.540193Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:52.540196Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:52.540198Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:52.540204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:52.541391Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2
2025-10-09T21:42:52.541424Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2
2025-10-09T21:42:52.545201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340749439404331:2487], now have 1 active actors on pipe
2025-10-09T21:42:52.545219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340749439404278:2432], now have 1 active actors on pipe
2025-10-09T21:42:52.545249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.545262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.545380Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3aea68ec-17110afa-1ced021-52c20a0d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:52.545393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:52.545446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:52.545463Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:52.545476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.545482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.545486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:52.545499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.545515Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:52.545524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:42:52.545536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:52.546130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.546138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.546159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:52.546286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 3 partition: 0 MaxSeqNo: 25 sessionId: src_id|3aea68ec-17110afa-1ced021-52c20a0d_0
2025-10-09T21:42:52.548398Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ae7534c-b0cae242-99a6e948-e5520e20_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3aea68ec-17110afa-1ced021-52c20a0d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:52.550199Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:0:1:10:0
2025-10-09T21:42:52.550223Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1001 BytesRead: 24780812 MessagesRead: 11 BytesReadCompressed: 24780812 BytesInflightUncompressed: 2252802 BytesInflightCompressed: 0 BytesInflightTotal: 2252802 MessagesInflight: 1 }
2025-10-09T21:42:52.551667Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3aea68ec-17110afa-1ced021-52c20a0d_0] Write session: try to update token
2025-10-09T21:42:52.551688Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3aea68ec-17110afa-1ced021-52c20a0d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:52.601505Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340749439404345:2476]
2025-10-09T21:42:52.602128Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 2, pipe: [1:7559340749439404345:2476]
2025-10-09T21:42:52.616304Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 3 sessionId: src_id|3aea68ec-17110afa-1ced021-52c20a0d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.622436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340749439404345:2476], now have 1 active actors on pipe
2025-10-09T21:42:52.622462Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:52.622465Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:52.622467Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:52.622469Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:52.622489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340749077596331:2406]
2025-10-09T21:42:52.622494Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340749077596331:2406]
2025-10-09T21:42:52.622501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340749077596331:2406]
2025-10-09T21:42:52.622553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:52.622562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.622564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:52.622736Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:52.623231Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:52.623239Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:52.623244Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:52.625507Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:52.818764Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:52.819036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.819046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.819793Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:52.822439Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340749439404345:2476]
2025-10-09T21:42:52.822470Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:52.822555Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 3 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3aea68ec-17110afa-1ced021-52c20a0d_0
2025-10-09T21:42:52.822743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 3 sessionId: src_id|3aea68ec-17110afa-1ced021-52c20a0d_0 is DEAD
2025-10-09T21:42:52.822790Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:52.826151Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3aea68ec-17110afa-1ced021-52c20a0d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:42:52.826168Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3aea68ec-17110afa-1ced021-52c20a0d_0] Write session will restart in 0.007898s
2025-10-09T21:42:52.826194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3aea68ec-17110afa-1ced021-52c20a0d_0] Write session: Do CDS request
2025-10-09T21:42:52.826200Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3aea68ec-17110afa-1ced021-52c20a0d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:52.838736Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3aea68ec-17110afa-1ced021-52c20a0d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:52.841703Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:52.841720Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 4
2025-10-09T21:42:52.842682Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 4 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:52.842718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 4 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:52.842724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=4 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:52.842728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:52.844082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 4 sessionId:  describe result for acl check
2025-10-09T21:42:52.844117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:52.844118Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:52.844120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:52.844125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340749439404373:2493] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:52.844661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340749439404373:2493] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:52.848833Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340749439404373:2493] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:52.856057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:52.861866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:52.862141Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:52.862146Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:52.862159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:52.862213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:52.862261Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340749077596405:2411]
2025-10-09T21:42:52.862826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:52.863382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340749439404373:2493] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:52.863396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340749439404373:2493] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:52.863399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340749439404373:2493] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:52.863409Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 4 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:52.864453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:52.864501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:52.864841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:52.864876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:52.864877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:52.864879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:52.864880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:52.864881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:52.864882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:52.864883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.864885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:52.864886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:52.864893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:52.864902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:52.864905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:52.864909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:52.864912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:52.864915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:52.864919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.864922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:52.864925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:52.864931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:52.864939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:52.864944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:52.864948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:52.864952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:52.864956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:52.864959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:52.864963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:52.864966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:52.864970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:52.864974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:52.864978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:52.864982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.864985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.864989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:52.864993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:52.864997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:52.865000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:52.865024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:52.878063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:52.879391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:52.879397Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:52.879399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:52.879409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:52.879417Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 3 [2:7559340749077596405:2411]
2025-10-09T21:42:52.879432Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:52.879452Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:52.879456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.879459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.879462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.879482Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:52.879490Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:52.879497Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:52.879566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.879567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:52.879570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:52.879572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:52.879653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:52.879656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.879668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:52.879680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:52.879693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:52.879714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340749077596399:2411]
2025-10-09T21:42:52.879716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340749077596399:2411]
2025-10-09T21:42:52.879718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340749077596399:2411]
2025-10-09T21:42:52.879733Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:52.879747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:52.879761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:52.879763Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:52.879805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340749077596399:2411]
2025-10-09T21:42:52.879808Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:52.880097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:52.880102Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:52.880104Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:52.880110Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:52.880855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 3
2025-10-09T21:42:52.880885Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 3
2025-10-09T21:42:52.894150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340749439404411:2493], now have 1 active actors on pipe
2025-10-09T21:42:52.894169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340749439404358:2432], now have 1 active actors on pipe
2025-10-09T21:42:52.894200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.894207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.894258Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ed57ed09-718a20ea-6429ba29-9a9cc3c1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:52.894275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.894289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.894292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:52.894309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.894327Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:52.894350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:52.897740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:52.897762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:52.897805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:52.902333Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 4 partition: 0 MaxSeqNo: 25 sessionId: src_id|ed57ed09-718a20ea-6429ba29-9a9cc3c1_0
2025-10-09T21:42:52.902830Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3aea68ec-17110afa-1ced021-52c20a0d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ed57ed09-718a20ea-6429ba29-9a9cc3c1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:52.906529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ed57ed09-718a20ea-6429ba29-9a9cc3c1_0] Write session: try to update token
2025-10-09T21:42:52.906553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ed57ed09-718a20ea-6429ba29-9a9cc3c1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:52.924022Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340749439404434:2476]
2025-10-09T21:42:52.926508Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:52.926526Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:52.926532Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:52.926534Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:52.926543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340749077596399:2411]
2025-10-09T21:42:52.926547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340749077596399:2411]
2025-10-09T21:42:52.926550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340749077596399:2411]
2025-10-09T21:42:52.926595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:52.926601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:52.926603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:52.926787Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:52.930793Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:52.930801Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:52.930806Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:52.931051Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 3, pipe: [1:7559340749439404434:2476]
2025-10-09T21:42:52.942223Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 4 sessionId: src_id|ed57ed09-718a20ea-6429ba29-9a9cc3c1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:52.945054Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:53.178930Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:53.179223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340749439404434:2476], now have 1 active actors on pipe
2025-10-09T21:42:53.179254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:53.179263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:53.179757Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:53.184691Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340749439404434:2476]
2025-10-09T21:42:53.184725Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:53.184764Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:53.184872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 4 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ed57ed09-718a20ea-6429ba29-9a9cc3c1_0
2025-10-09T21:42:53.185057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 4 sessionId: src_id|ed57ed09-718a20ea-6429ba29-9a9cc3c1_0 is DEAD
2025-10-09T21:42:53.186130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed57ed09-718a20ea-6429ba29-9a9cc3c1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:42:53.186141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed57ed09-718a20ea-6429ba29-9a9cc3c1_0] Write session will restart in 0.009328s
2025-10-09T21:42:53.186172Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed57ed09-718a20ea-6429ba29-9a9cc3c1_0] Write session: Do CDS request
2025-10-09T21:42:53.186179Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed57ed09-718a20ea-6429ba29-9a9cc3c1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:53.198625Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ed57ed09-718a20ea-6429ba29-9a9cc3c1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:53.199035Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:53.199053Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 5
2025-10-09T21:42:53.199259Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 5 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:53.199294Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 5 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:53.199301Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=5 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:53.199305Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:53.199650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:53.199878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:53.200055Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:53.200059Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:53.200071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:53.200121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:53.200161Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340753372563798:2425]
2025-10-09T21:42:53.200764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:53.199812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 5 sessionId:  describe result for acl check
2025-10-09T21:42:53.199852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:53.199854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:53.199855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:53.199862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340753734371767:2504] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:53.200526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340753734371767:2504] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:53.201484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:53.201539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:53.201890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:53.201965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:53.201966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:53.201967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:53.201968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:53.201969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:53.201970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:53.201971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.201972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.201973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.201980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:53.201990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:53.201994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:53.201997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:53.202001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:53.202005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:53.202009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.202012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.202018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.202041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:53.202052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:53.202058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:53.202062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:53.202065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:53.202069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:53.202073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:53.202077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:53.202081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:53.202085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:53.202090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:53.202095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:53.202100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.202105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.202109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.202113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.202116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.202121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.202150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:53.209728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340753734371767:2504] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:53.217892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:53.219116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:53.219123Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:53.219125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:53.219134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:53.219143Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 4 [2:7559340753372563798:2425]
2025-10-09T21:42:53.219161Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:53.219183Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:53.219187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:53.219190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:53.219193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:53.219215Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:53.219224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:53.219230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:53.219303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.219306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.219308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.219310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:53.219388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:53.219391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.219404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.219416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.219431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:53.219453Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340753372563792:2425]
2025-10-09T21:42:53.219455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340753372563792:2425]
2025-10-09T21:42:53.219457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340753372563792:2425]
2025-10-09T21:42:53.219473Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:53.219489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:53.219503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:53.219508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:53.219551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340753372563792:2425]
2025-10-09T21:42:53.219554Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:53.220740Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 4
2025-10-09T21:42:53.219829Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:53.219834Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:53.219836Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:53.219852Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:53.221233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340753734371749:2432], now have 1 active actors on pipe
2025-10-09T21:42:53.225436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340753734371767:2504] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:53.225453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340753734371767:2504] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:53.225457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340753734371767:2504] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:53.225468Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 5 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:53.226567Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 4
2025-10-09T21:42:53.226717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340753734371810:2504], now have 1 active actors on pipe
2025-10-09T21:42:53.226744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:53.226752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:53.226786Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|33266adc-bb516e1c-d17d9826-1d3d25f2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:53.226802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:53.226807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:53.226811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:53.226825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:53.226847Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:53.226867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:53.227172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:53.227177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:53.227193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:53.227351Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 5 partition: 0 MaxSeqNo: 25 sessionId: src_id|33266adc-bb516e1c-d17d9826-1d3d25f2_0
2025-10-09T21:42:53.231369Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed57ed09-718a20ea-6429ba29-9a9cc3c1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|33266adc-bb516e1c-d17d9826-1d3d25f2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:53.234374Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33266adc-bb516e1c-d17d9826-1d3d25f2_0] Write session: try to update token
2025-10-09T21:42:53.234396Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33266adc-bb516e1c-d17d9826-1d3d25f2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:53.272649Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:53.272674Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:53.272682Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:53.272685Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:53.272697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340753372563792:2425]
2025-10-09T21:42:53.272701Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340753372563792:2425]
2025-10-09T21:42:53.272705Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340753372563792:2425]
2025-10-09T21:42:53.272752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:53.272761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.272764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:53.272961Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:53.275590Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:53.275605Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:53.275611Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:53.279816Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 5 sessionId: src_id|33266adc-bb516e1c-d17d9826-1d3d25f2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:53.282466Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:53.288100Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340753734371814:2476]
2025-10-09T21:42:53.290403Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 4, pipe: [1:7559340753734371814:2476]
2025-10-09T21:42:53.494775Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:53.495186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:53.495197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:53.495220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340753734371814:2476], now have 1 active actors on pipe
2025-10-09T21:42:53.497731Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:53.498521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340753734371814:2476]
2025-10-09T21:42:53.498553Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:53.498591Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:53.498706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 5 reason: status is not ok: tablet will be restarted right now sessionId: src_id|33266adc-bb516e1c-d17d9826-1d3d25f2_0
2025-10-09T21:42:53.498921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 5 sessionId: src_id|33266adc-bb516e1c-d17d9826-1d3d25f2_0 is DEAD
2025-10-09T21:42:53.502220Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33266adc-bb516e1c-d17d9826-1d3d25f2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:42:53.502234Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33266adc-bb516e1c-d17d9826-1d3d25f2_0] Write session will restart in 0.008859s
2025-10-09T21:42:53.502265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33266adc-bb516e1c-d17d9826-1d3d25f2_0] Write session: Do CDS request
2025-10-09T21:42:53.502280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33266adc-bb516e1c-d17d9826-1d3d25f2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:53.512640Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33266adc-bb516e1c-d17d9826-1d3d25f2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:53.513039Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:53.513054Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 6
2025-10-09T21:42:53.514355Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 6 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:53.514410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 6 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:53.514461Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=6 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:53.514468Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:53.515126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 6 sessionId:  describe result for acl check
2025-10-09T21:42:53.515164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:53.515166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:53.515168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:53.515173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340753734371834:2509] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:53.516578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340753734371834:2509] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:53.520528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340753734371834:2509] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:53.529213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340753734371834:2509] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:53.529229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340753734371834:2509] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:53.529233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340753734371834:2509] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:53.529241Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 6 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:53.524403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:53.528935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:53.529239Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:53.529244Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:53.529260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:53.529335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:53.529399Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340753372563863:2427]
2025-10-09T21:42:53.529977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:53.538361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:53.538479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:53.538913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:53.538986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:53.538987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:53.538989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:53.538992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:53.538994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:53.538995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:53.538996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.538997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.538999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.539006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:53.539016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:53.539020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:53.539023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:53.539027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:53.539030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:53.539034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.539037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.539041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.539050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:53.539059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:53.539065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:53.539070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:53.539074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:53.539080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:53.539084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:53.539088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:53.539093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:53.539098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:53.539102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:53.539106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:53.539114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.539119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.539123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.539129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.539138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.539142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.539170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:53.564188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:53.570085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:53.570108Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:53.570112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:53.570118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:53.570128Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 5 [2:7559340753372563863:2427]
2025-10-09T21:42:53.570147Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:53.570171Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:53.570174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:53.570178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:53.570181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:53.570202Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:53.570209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:53.570217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:53.570305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.570308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.570311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.570313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:53.570402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:53.570407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.570438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.570454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.570472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:53.570493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340753372563858:2427]
2025-10-09T21:42:53.570496Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340753372563858:2427]
2025-10-09T21:42:53.570498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340753372563858:2427]
2025-10-09T21:42:53.570513Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:53.570530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:53.570546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:53.570549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:53.570596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340753372563858:2427]
2025-10-09T21:42:53.570599Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:53.570881Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:53.570887Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:53.570889Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:53.570904Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:53.571281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340753734371874:2509], now have 1 active actors on pipe
2025-10-09T21:42:53.571288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340753734371819:2432], now have 1 active actors on pipe
2025-10-09T21:42:53.571850Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 5
2025-10-09T21:42:53.571893Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 5
2025-10-09T21:42:53.572156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:53.572168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:53.572204Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8c41e0e9-471d871d-e983a931-6f805c6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:53.572214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:53.572219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:53.572222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:53.572235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:53.572249Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:53.572271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:53.572601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:53.572607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:53.572623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:53.572762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 6 partition: 0 MaxSeqNo: 25 sessionId: src_id|8c41e0e9-471d871d-e983a931-6f805c6_0
2025-10-09T21:42:53.574382Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33266adc-bb516e1c-d17d9826-1d3d25f2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8c41e0e9-471d871d-e983a931-6f805c6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:53.597007Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:53.597023Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:53.597026Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:53.597028Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:53.597038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340753372563858:2427]
2025-10-09T21:42:53.597045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340753372563858:2427]
2025-10-09T21:42:53.597049Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340753372563858:2427]
2025-10-09T21:42:53.597096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:53.597104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.597106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:53.597242Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:53.598134Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340753734371887:2476]
2025-10-09T21:42:53.598238Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:53.598245Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:53.598249Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:53.600270Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 5, pipe: [1:7559340753734371887:2476]
2025-10-09T21:42:53.606470Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c41e0e9-471d871d-e983a931-6f805c6_0] Write session: try to update token
2025-10-09T21:42:53.606500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c41e0e9-471d871d-e983a931-6f805c6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:53.643140Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 6 sessionId: src_id|8c41e0e9-471d871d-e983a931-6f805c6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:53.654605Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:53.784026Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:53.784326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340753734371887:2476], now have 1 active actors on pipe
2025-10-09T21:42:53.784392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:53.784403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:53.785633Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:53.785690Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340753734371887:2476]
2025-10-09T21:42:53.785712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:53.785715Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:53.785783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 6 reason: error at write: Disconnected sessionId: src_id|8c41e0e9-471d871d-e983a931-6f805c6_0
2025-10-09T21:42:53.785979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 6 sessionId: src_id|8c41e0e9-471d871d-e983a931-6f805c6_0 is DEAD
2025-10-09T21:42:53.786520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c41e0e9-471d871d-e983a931-6f805c6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:53.786531Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c41e0e9-471d871d-e983a931-6f805c6_0] Write session will restart in 0.008164s
2025-10-09T21:42:53.786560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c41e0e9-471d871d-e983a931-6f805c6_0] Write session: Do CDS request
2025-10-09T21:42:53.786569Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c41e0e9-471d871d-e983a931-6f805c6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:53.787241Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:53.795631Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c41e0e9-471d871d-e983a931-6f805c6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:53.795978Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:53.795995Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 7
2025-10-09T21:42:53.796318Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 7 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:53.796354Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 7 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:53.796360Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=7 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:53.796363Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:53.797010Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 7 sessionId:  describe result for acl check
2025-10-09T21:42:53.797044Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:53.797047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:53.797049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:53.797054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340753734371908:2515] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:53.797674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340753734371908:2515] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:53.805733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340753734371908:2515] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:53.813464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:53.816052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340753734371908:2515] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:53.816071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340753734371908:2515] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:53.816075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340753734371908:2515] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:53.816085Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 7 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:53.814015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:53.814259Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:53.814263Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:53.814275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:53.814325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:53.814369Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340753372563927:2430]
2025-10-09T21:42:53.814978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:53.815696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:53.815763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:53.816131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:53.816184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:53.816185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:53.816187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:53.816188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:53.816190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:53.816191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:53.816192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.816193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.816194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.816200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:53.816210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:53.816214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:53.816218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:53.816221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:53.816225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:53.816228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.816232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.816236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.816244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:53.816253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:53.816259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:53.816264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:53.816268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:53.816273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:53.816288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:53.816292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:53.816295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:53.816300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:53.816304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:53.816309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:53.816314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.816319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.816323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.816328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.816333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.816337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.816365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:53.833161Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 6
2025-10-09T21:42:53.833204Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 6
2025-10-09T21:42:53.830674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:53.831990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:53.831996Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:53.831999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:53.832005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:53.832014Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 6 [2:7559340753372563927:2430]
2025-10-09T21:42:53.832032Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:53.832055Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:53.832059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:53.832063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:53.832065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:53.832087Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:53.832093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:53.832101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:53.832177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.832180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.832183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.832186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:53.832271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:53.832275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.832287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:53.832296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:53.832310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:53.832332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340753372563921:2430]
2025-10-09T21:42:53.832335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340753372563921:2430]
2025-10-09T21:42:53.832337Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340753372563921:2430]
2025-10-09T21:42:53.832346Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:53.832368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:53.832370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:53.832385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:53.832385Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:53.832387Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:53.832388Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:53.832431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340753372563921:2430]
2025-10-09T21:42:53.832439Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:53.832450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340753734371897:2432], now have 1 active actors on pipe
2025-10-09T21:42:53.832454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340753734371955:2515], now have 1 active actors on pipe
2025-10-09T21:42:53.832494Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:53.837626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:53.837642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:53.837696Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4063b56f-7604cab9-3d2d6224-bd4d6a4f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:53.837726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:53.837732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:53.837736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:53.837750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:53.837769Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:53.837803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:53.838140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:53.838152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:53.838172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:53.838314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 7 partition: 0 MaxSeqNo: 25 sessionId: src_id|4063b56f-7604cab9-3d2d6224-bd4d6a4f_0
2025-10-09T21:42:53.839865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c41e0e9-471d871d-e983a931-6f805c6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4063b56f-7604cab9-3d2d6224-bd4d6a4f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:53.842665Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4063b56f-7604cab9-3d2d6224-bd4d6a4f_0] Write session: try to update token
2025-10-09T21:42:53.842689Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4063b56f-7604cab9-3d2d6224-bd4d6a4f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:53.860234Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:53.860250Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:53.860253Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:53.860256Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:53.860266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340753372563921:2430]
2025-10-09T21:42:53.860270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340753372563921:2430]
2025-10-09T21:42:53.860277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340753372563921:2430]
2025-10-09T21:42:53.860328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:53.860336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:53.860339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:53.860388Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:53.860401Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:53.860406Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:53.860528Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:53.882983Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 7 sessionId: src_id|4063b56f-7604cab9-3d2d6224-bd4d6a4f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:53.886568Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340753734371969:2476]
2025-10-09T21:42:53.887688Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 6, pipe: [1:7559340753734371969:2476]
2025-10-09T21:42:53.895405Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:54.048057Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:54.048384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340753734371969:2476], now have 1 active actors on pipe
2025-10-09T21:42:54.048415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:54.048424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:54.048771Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:54.048827Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340753734371969:2476]
2025-10-09T21:42:54.048828Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:54.048833Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:54.048862Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:54.048920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 7 reason: error at write: Disconnected sessionId: src_id|4063b56f-7604cab9-3d2d6224-bd4d6a4f_0
2025-10-09T21:42:54.049119Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 7 sessionId: src_id|4063b56f-7604cab9-3d2d6224-bd4d6a4f_0 is DEAD
2025-10-09T21:42:54.050411Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4063b56f-7604cab9-3d2d6224-bd4d6a4f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:54.050443Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4063b56f-7604cab9-3d2d6224-bd4d6a4f_0] Write session will restart in 0.009433s
2025-10-09T21:42:54.050474Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4063b56f-7604cab9-3d2d6224-bd4d6a4f_0] Write session: Do CDS request
2025-10-09T21:42:54.050482Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4063b56f-7604cab9-3d2d6224-bd4d6a4f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:54.062628Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4063b56f-7604cab9-3d2d6224-bd4d6a4f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:54.063313Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:54.063330Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 8
2025-10-09T21:42:54.063712Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 8 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:54.063754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 8 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:54.063766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=8 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:54.063772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:54.064312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 8 sessionId:  describe result for acl check
2025-10-09T21:42:54.064349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:54.064353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:54.064355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:54.064361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340758029339292:2519] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:54.064935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340758029339292:2519] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:54.066939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:54.067735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:54.067958Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:54.067972Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:54.067987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:54.068054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:54.068119Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340757667531286:2433]
2025-10-09T21:42:54.068316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340758029339292:2519] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:54.068728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:54.069206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:54.069265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:54.069664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:54.069718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.069724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.069726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.069727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.069728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.069730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.069732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.069733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.069734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.069740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.069753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.069758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.069762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.069766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.069771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.069775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.069779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.069784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.069791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.069799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.069806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.069812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.069817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.069823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.069827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.069834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.069844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.069849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.069854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.069861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.069870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.069877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.069895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.069905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.069910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.069920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.069966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:54.079234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340758029339292:2519] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:54.079252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340758029339292:2519] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:54.079257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340758029339292:2519] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:54.079270Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 8 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:54.083278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:54.084484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:54.084502Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:54.084505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:54.084510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:54.084519Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 7 [2:7559340757667531286:2433]
2025-10-09T21:42:54.084535Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:54.084554Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:54.084557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:54.084560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:54.084562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:54.084582Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:54.084587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:54.084594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:54.084663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.084670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.084672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.084677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:54.084760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:54.084768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.084782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.084798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.084818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:54.084840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340757667531281:2433]
2025-10-09T21:42:54.084848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340757667531281:2433]
2025-10-09T21:42:54.084851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340757667531281:2433]
2025-10-09T21:42:54.084860Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:54.084871Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:54.084874Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:54.084877Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:54.084877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:54.084893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:54.084896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:54.084909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340758029339270:2432], now have 1 active actors on pipe
2025-10-09T21:42:54.084913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340758029339330:2519], now have 1 active actors on pipe
2025-10-09T21:42:54.084954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340757667531281:2433]
2025-10-09T21:42:54.084961Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:54.085206Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:54.085800Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 7
2025-10-09T21:42:54.085843Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 7
2025-10-09T21:42:54.086978Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 8 partition: 0 MaxSeqNo: 25 sessionId: src_id|a9ebce36-76751745-a67b7f65-260ec8e9_0
2025-10-09T21:42:54.087678Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4063b56f-7604cab9-3d2d6224-bd4d6a4f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a9ebce36-76751745-a67b7f65-260ec8e9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:54.086192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:54.086206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:54.086241Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a9ebce36-76751745-a67b7f65-260ec8e9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:54.086254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:54.086259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:54.086261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:54.086276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:54.086294Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:54.086313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:54.086673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:54.086678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:54.086704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:54.108869Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9ebce36-76751745-a67b7f65-260ec8e9_0] Write session: try to update token
2025-10-09T21:42:54.108897Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9ebce36-76751745-a67b7f65-260ec8e9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:54.113509Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:54.113529Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:54.113532Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:54.113533Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:54.113543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340757667531281:2433]
2025-10-09T21:42:54.113547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340757667531281:2433]
2025-10-09T21:42:54.113550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340757667531281:2433]
2025-10-09T21:42:54.113605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:54.113613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.113617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:54.113623Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:54.113630Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:54.113635Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:54.113921Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:54.139132Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 8 sessionId: src_id|a9ebce36-76751745-a67b7f65-260ec8e9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:54.149124Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340758029339362:2476]
2025-10-09T21:42:54.149531Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 7, pipe: [1:7559340758029339362:2476]
2025-10-09T21:42:54.152129Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:54.284688Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:54.285060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340758029339362:2476], now have 1 active actors on pipe
2025-10-09T21:42:54.285104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:54.285117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:54.285840Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:54.286687Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9ebce36-76751745-a67b7f65-260ec8e9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:54.286702Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9ebce36-76751745-a67b7f65-260ec8e9_0] Write session will restart in 0.009607s
2025-10-09T21:42:54.286728Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9ebce36-76751745-a67b7f65-260ec8e9_0] Write session: Do CDS request
2025-10-09T21:42:54.286735Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9ebce36-76751745-a67b7f65-260ec8e9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:54.286159Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340758029339362:2476]
2025-10-09T21:42:54.286193Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:54.286228Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:54.286230Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:54.286309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 8 reason: error at write: Disconnected sessionId: src_id|a9ebce36-76751745-a67b7f65-260ec8e9_0
2025-10-09T21:42:54.286527Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 8 sessionId: src_id|a9ebce36-76751745-a67b7f65-260ec8e9_0 is DEAD
2025-10-09T21:42:54.296267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:54.296787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:54.297660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9ebce36-76751745-a67b7f65-260ec8e9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:54.296954Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:54.296966Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:54.296976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:54.297020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:54.297057Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340757667531387:2447]
2025-10-09T21:42:54.297695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:54.298015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:54.298132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:54.298209Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:54.298226Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 9
2025-10-09T21:42:54.298368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:54.298411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.298425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.298426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.298427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.298428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.298429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.298431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.298432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.298433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.298439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.298448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.298450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.298452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.298454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.298457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.298459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.298461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.298463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.298468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.298473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.298476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.298480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.298483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.298488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.298491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.298494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.298497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.298500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.298502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.298505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.298508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.298511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.298514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.298517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.298519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.298522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.298540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:54.302248Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 9 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:54.302315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 9 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:54.302325Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=9 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:54.302331Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:54.302822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 9 sessionId:  describe result for acl check
2025-10-09T21:42:54.302862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:54.302868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:54.302870Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:54.302876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340758029339394:2530] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:54.303430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340758029339394:2530] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:54.305890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:54.307376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:54.307391Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:54.307394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:54.307398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:54.307405Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 8 [2:7559340757667531387:2447]
2025-10-09T21:42:54.307422Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:54.307439Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:54.307442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:54.307445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:54.307449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:54.307465Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:54.307482Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:54.307497Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:54.307557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.307564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.307566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.307568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:54.307571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340758029339394:2530] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:54.307759Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 8
2025-10-09T21:42:54.307638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:54.307649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.307672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.307685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.307704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:54.307723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340758029339364:2432], now have 1 active actors on pipe
2025-10-09T21:42:54.307731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340757667531382:2447]
2025-10-09T21:42:54.307733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340757667531382:2447]
2025-10-09T21:42:54.307735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340757667531382:2447]
2025-10-09T21:42:54.307749Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:54.307761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:54.307762Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:54.307766Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:54.307768Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:54.307776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:54.307779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:54.307809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340757667531382:2447]
2025-10-09T21:42:54.307812Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:54.307814Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:54.315516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340758029339394:2530] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:54.315532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340758029339394:2530] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:54.315536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340758029339394:2530] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:54.315545Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 9 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:54.315805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340758029339432:2530], now have 1 active actors on pipe
2025-10-09T21:42:54.315951Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 8
2025-10-09T21:42:54.316015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:54.316028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:54.316055Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e5060e94-761a1921-ede41577-d2002970_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:54.316070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:54.316081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:54.316084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:54.316096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:54.316120Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:54.316142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:54.316347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:54.316355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:54.316369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:54.316467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 9 partition: 0 MaxSeqNo: 25 sessionId: src_id|e5060e94-761a1921-ede41577-d2002970_0
2025-10-09T21:42:54.318094Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9ebce36-76751745-a67b7f65-260ec8e9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e5060e94-761a1921-ede41577-d2002970_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:54.321290Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5060e94-761a1921-ede41577-d2002970_0] Write session: try to update token
2025-10-09T21:42:54.321313Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5060e94-761a1921-ede41577-d2002970_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:54.326251Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:54.326269Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:54.326273Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:54.326275Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:54.326285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340757667531382:2447]
2025-10-09T21:42:54.326289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340757667531382:2447]
2025-10-09T21:42:54.326294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340757667531382:2447]
2025-10-09T21:42:54.326344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:54.326358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.326365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:54.326364Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:54.326373Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:54.326379Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:54.327076Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:54.342005Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 9 sessionId: src_id|e5060e94-761a1921-ede41577-d2002970_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:54.345379Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:54.385631Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340758029339438:2476]
2025-10-09T21:42:54.385937Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 8, pipe: [1:7559340758029339438:2476]
2025-10-09T21:42:54.509707Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:54.511766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:54.511804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:54.511840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340758029339438:2476], now have 1 active actors on pipe
2025-10-09T21:42:54.513375Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340758029339438:2476]
2025-10-09T21:42:54.513443Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:54.513525Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:54.513527Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:54.513917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 9 reason: error at write: Disconnected sessionId: src_id|e5060e94-761a1921-ede41577-d2002970_0
2025-10-09T21:42:54.514218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 9 sessionId: src_id|e5060e94-761a1921-ede41577-d2002970_0 is DEAD
2025-10-09T21:42:54.513891Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:54.516524Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5060e94-761a1921-ede41577-d2002970_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:54.516555Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5060e94-761a1921-ede41577-d2002970_0] Write session will restart in 0.008279s
2025-10-09T21:42:54.516640Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5060e94-761a1921-ede41577-d2002970_0] Write session: Do CDS request
2025-10-09T21:42:54.516647Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5060e94-761a1921-ede41577-d2002970_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:54.526496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:54.526748Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5060e94-761a1921-ede41577-d2002970_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:54.527211Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:54.527222Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 10
2025-10-09T21:42:54.527406Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 10 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:54.527438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:54.527504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 10 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:54.527520Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=10 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:54.527523Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:54.527725Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:54.527748Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:54.527758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:54.527806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:54.527871Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340757667531455:2450]
2025-10-09T21:42:54.528263Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 10 sessionId:  describe result for acl check
2025-10-09T21:42:54.528719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:54.529346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:54.528312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:54.529391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:54.528320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:54.528321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:54.528327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340758029339466:2535] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:54.529214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340758029339466:2535] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:54.529834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:54.529895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.529901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.529902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.529903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.529905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.529918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.529919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.529920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.529921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.529925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.529934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.529938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.529942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.529945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.529949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.529952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.529956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.529960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.529966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.529973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.529979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.529983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.529987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.529992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.529995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.530000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.530005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.530009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.530013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.530018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.530022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.530062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.530073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.530078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.530091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.530096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.530162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:54.534815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340758029339466:2535] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:54.540629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340758029339466:2535] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:54.540645Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340758029339466:2535] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:54.540649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340758029339466:2535] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:54.540659Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 10 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:54.541436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:54.557798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:54.557820Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:54.557823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:54.557828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:54.557837Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 9 [2:7559340757667531455:2450]
2025-10-09T21:42:54.557866Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:54.557893Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:54.557897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:54.557901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:54.557903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:54.557920Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:54.557927Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:54.557934Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:54.558191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.558202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.558205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.558208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:54.558285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:54.558288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.558305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.558319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.558339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:54.558357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340757667531449:2450]
2025-10-09T21:42:54.558360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340757667531449:2450]
2025-10-09T21:42:54.558362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340757667531449:2450]
2025-10-09T21:42:54.558379Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:54.558437Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:54.558453Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:54.558457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:54.558499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340757667531449:2450]
2025-10-09T21:42:54.558503Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:54.558658Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:54.558665Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:54.558667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:54.558674Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:54.558870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340758029339440:2432], now have 1 active actors on pipe
2025-10-09T21:42:54.558879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340758029339503:2535], now have 1 active actors on pipe
2025-10-09T21:42:54.558978Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 9
2025-10-09T21:42:54.559810Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 9
2025-10-09T21:42:54.559950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:54.559972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:54.559999Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3507796e-d60e05ec-36a93cf5-c78690c0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:54.560019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:54.560030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:54.560033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:54.560044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:54.560057Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:54.560083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:54.561460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:54.561474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:54.561497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:54.561611Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 10 partition: 0 MaxSeqNo: 25 sessionId: src_id|3507796e-d60e05ec-36a93cf5-c78690c0_0
2025-10-09T21:42:54.562075Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5060e94-761a1921-ede41577-d2002970_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3507796e-d60e05ec-36a93cf5-c78690c0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:54.576361Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:42:54.576378Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:54.576387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340757667531449:2450]
2025-10-09T21:42:54.576432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:54.576436Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 10 partno 4 count 4 parts 14 suffix '0' size 7988760
2025-10-09T21:42:54.576447Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:42:54.576614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:42:54.576757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:54.578506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:54.580386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:54.580611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:42:54.580656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:42:54.582539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3507796e-d60e05ec-36a93cf5-c78690c0_0] Write session: try to update token
2025-10-09T21:42:54.582559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3507796e-d60e05ec-36a93cf5-c78690c0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:54.591976Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:54.591990Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:54.591992Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:54.591995Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:54.592003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340757667531449:2450]
2025-10-09T21:42:54.592007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340757667531449:2450]
2025-10-09T21:42:54.592011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340757667531449:2450]
2025-10-09T21:42:54.592057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:54.592063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.592067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:54.592131Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:54.592163Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:54.592168Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:54.592200Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:54.612971Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340758029339512:2476]
2025-10-09T21:42:54.613294Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 9, pipe: [1:7559340758029339512:2476]
2025-10-09T21:42:54.647993Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 10 sessionId: src_id|3507796e-d60e05ec-36a93cf5-c78690c0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:54.650764Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:54.758487Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:54.758764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340758029339512:2476], now have 1 active actors on pipe
2025-10-09T21:42:54.758797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:54.758807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:54.759040Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340758029339512:2476]
2025-10-09T21:42:54.759070Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:54.759079Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:54.759079Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:54.759571Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3507796e-d60e05ec-36a93cf5-c78690c0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:54.759581Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3507796e-d60e05ec-36a93cf5-c78690c0_0] Write session will restart in 0.008269s
2025-10-09T21:42:54.759603Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3507796e-d60e05ec-36a93cf5-c78690c0_0] Write session: Do CDS request
2025-10-09T21:42:54.759110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:54.759186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 10 reason: error at write: Disconnected sessionId: src_id|3507796e-d60e05ec-36a93cf5-c78690c0_0
2025-10-09T21:42:54.759609Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3507796e-d60e05ec-36a93cf5-c78690c0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:54.759386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 10 sessionId: src_id|3507796e-d60e05ec-36a93cf5-c78690c0_0 is DEAD
2025-10-09T21:42:54.769594Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3507796e-d60e05ec-36a93cf5-c78690c0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:54.769718Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:54.769734Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 11
2025-10-09T21:42:54.769402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:54.769877Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 11 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:54.769913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 11 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:54.769923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=11 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:54.769938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:54.769987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:54.770179Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:54.770188Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:54.770197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:54.770251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:54.770293Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340757667531521:2452]
2025-10-09T21:42:54.770327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 11 sessionId:  describe result for acl check
2025-10-09T21:42:54.770358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:54.770365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:54.770385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:54.770395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340758029339556:2541] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:54.770864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:54.771050Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340758029339556:2541] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:54.771185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:54.771250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:54.771660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:54.771731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.771737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.771738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.771740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.771741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.771744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.771746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.771747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.771748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.771754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.771763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.771766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.771770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.771773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.771776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.771780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.771783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.771786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.771793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.771809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.771814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.771819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.771845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.771851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.771855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.771861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.771866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.771870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.771882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.771888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.771893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.771898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.771902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.771919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.771924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.771931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.771958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:54.772871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340758029339556:2541] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:54.779900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340758029339556:2541] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:54.779927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340758029339556:2541] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:54.779931Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340758029339556:2541] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:54.779941Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 11 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:54.780590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:54.782821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:54.782839Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:54.782841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:54.782846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:54.782853Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 10 [2:7559340757667531521:2452]
2025-10-09T21:42:54.782878Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:54.782897Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:54.782905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:54.782909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:54.782911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:54.782927Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:54.782932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:54.782942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:54.783026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.783036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.783038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.783041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:54.783115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:54.783122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.783135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.783151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.783181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:54.783206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340758029339530:2432], now have 1 active actors on pipe
2025-10-09T21:42:54.783210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340758029339595:2541], now have 1 active actors on pipe
2025-10-09T21:42:54.783216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340757667531516:2452]
2025-10-09T21:42:54.783220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340757667531516:2452]
2025-10-09T21:42:54.783222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340757667531516:2452]
2025-10-09T21:42:54.783237Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:54.783249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:54.783268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:54.783272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:54.783275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:54.783277Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:54.783280Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:54.783307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340757667531516:2452]
2025-10-09T21:42:54.783314Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:54.783316Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:54.783342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:54.783293Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 10
2025-10-09T21:42:54.783350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:54.783379Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e8359274-edbd2df2-86f72452-bb805085_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:54.783386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:54.783390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:54.783393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:54.783309Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 10
2025-10-09T21:42:54.783400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:54.783413Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:54.783429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:54.783854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:54.783876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:54.783913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:54.784039Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 11 partition: 0 MaxSeqNo: 25 sessionId: src_id|e8359274-edbd2df2-86f72452-bb805085_0
2025-10-09T21:42:54.784400Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3507796e-d60e05ec-36a93cf5-c78690c0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e8359274-edbd2df2-86f72452-bb805085_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:54.787537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8359274-edbd2df2-86f72452-bb805085_0] Write session: try to update token
2025-10-09T21:42:54.787553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8359274-edbd2df2-86f72452-bb805085_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:54.807058Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:54.807078Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:54.807081Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:54.807084Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:54.807107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340757667531516:2452]
2025-10-09T21:42:54.807116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340757667531516:2452]
2025-10-09T21:42:54.807122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340757667531516:2452]
2025-10-09T21:42:54.807173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:54.807181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.807184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:54.807386Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:54.809206Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:54.809226Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:54.809231Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:54.816565Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 11 sessionId: src_id|e8359274-edbd2df2-86f72452-bb805085_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:54.820114Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:54.859311Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340758029339606:2476]
2025-10-09T21:42:54.859786Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 10, pipe: [1:7559340758029339606:2476]
2025-10-09T21:42:54.965001Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:54.965342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:54.965354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:54.965402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340758029339606:2476], now have 1 active actors on pipe
2025-10-09T21:42:54.965789Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:54.968708Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8359274-edbd2df2-86f72452-bb805085_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:42:54.968725Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8359274-edbd2df2-86f72452-bb805085_0] Write session will restart in 0.006457s
2025-10-09T21:42:54.968751Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8359274-edbd2df2-86f72452-bb805085_0] Write session: Do CDS request
2025-10-09T21:42:54.968757Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8359274-edbd2df2-86f72452-bb805085_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:54.965817Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:54.965877Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:54.965981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 11 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e8359274-edbd2df2-86f72452-bb805085_0
2025-10-09T21:42:54.966136Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340758029339606:2476]
2025-10-09T21:42:54.966166Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 11 sessionId: src_id|e8359274-edbd2df2-86f72452-bb805085_0 is DEAD
2025-10-09T21:42:54.975438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:54.975917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:54.976121Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:54.976132Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:54.976144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:54.976193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:54.976262Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340757667531589:2455]
2025-10-09T21:42:54.976543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8359274-edbd2df2-86f72452-bb805085_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:54.976687Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:54.976704Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 12
2025-10-09T21:42:54.976885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:54.976931Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 12 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:54.976978Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 12 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:54.976985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=12 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:54.976990Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:54.977240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:54.977300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:54.977686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:54.977734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.977739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.977741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.977742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.977743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.977744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.977745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.977746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.977748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.977764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.977775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.977779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.977783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.977787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.977790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.977796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.977800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.977803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.977810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.977818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:54.977824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.977829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:54.977833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.977837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:54.977843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.977847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:54.977852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.977856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:54.977861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.977866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:54.977871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.977876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.977881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.977888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.977897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.977902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.977941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:54.977422Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 12 sessionId:  describe result for acl check
2025-10-09T21:42:54.977456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:54.977458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:54.977460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:54.977467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340758029339641:2545] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:54.978100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340758029339641:2545] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:54.980050Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340758029339641:2545] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:54.986007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340758029339641:2545] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:54.986047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340758029339641:2545] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:54.986053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340758029339641:2545] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:54.986066Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 12 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:54.986337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:54.987632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:54.987647Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:54.987649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:54.987654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:54.987661Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 11 [2:7559340757667531589:2455]
2025-10-09T21:42:54.987679Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:54.987699Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:54.987707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:54.987714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:54.987717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:54.987738Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:54.987748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:54.987755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:54.987822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.987829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.987832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.987834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:54.987907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:54.987914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:54.987927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:54.987944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:54.987965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:54.987990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340758029339675:2545], now have 1 active actors on pipe
2025-10-09T21:42:54.987997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340758029339612:2432], now have 1 active actors on pipe
2025-10-09T21:42:54.988005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340757667531584:2455]
2025-10-09T21:42:54.988012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340757667531584:2455]
2025-10-09T21:42:54.988017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340757667531584:2455]
2025-10-09T21:42:54.988030Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:54.988059Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:54.988063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:54.988065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:54.988088Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:54.988093Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 11
2025-10-09T21:42:54.988104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:54.988112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:54.988155Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340757667531584:2455]
2025-10-09T21:42:54.988159Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:54.988094Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 11
2025-10-09T21:42:54.988177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:54.988196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:54.988205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:54.988252Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c7468639-67044557-55e2a216-4bba3ec5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:54.988265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:54.988274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:54.988277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:54.988287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:54.988302Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:54.988318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:54.988511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:54.988526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:54.988556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:54.988688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 12 partition: 0 MaxSeqNo: 25 sessionId: src_id|c7468639-67044557-55e2a216-4bba3ec5_0
2025-10-09T21:42:54.989157Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8359274-edbd2df2-86f72452-bb805085_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c7468639-67044557-55e2a216-4bba3ec5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:54.992192Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c7468639-67044557-55e2a216-4bba3ec5_0] Write session: try to update token
2025-10-09T21:42:54.992213Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c7468639-67044557-55e2a216-4bba3ec5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:55.009750Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:55.009773Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:55.009776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:55.009779Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:55.009790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340757667531584:2455]
2025-10-09T21:42:55.009794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340757667531584:2455]
2025-10-09T21:42:55.009798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340757667531584:2455]
2025-10-09T21:42:55.009851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:55.009864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.009867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:55.009878Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:55.009887Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:55.009892Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:55.010055Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:55.013245Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 12 sessionId: src_id|c7468639-67044557-55e2a216-4bba3ec5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:55.015998Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:55.066125Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340762324306979:2476]
2025-10-09T21:42:55.066489Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 11, pipe: [1:7559340762324306979:2476]
2025-10-09T21:42:55.181588Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:55.182237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:55.182253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:55.182310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340762324306979:2476], now have 1 active actors on pipe
2025-10-09T21:42:55.182760Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:55.183796Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:55.183979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 12 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c7468639-67044557-55e2a216-4bba3ec5_0
2025-10-09T21:42:55.184228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 12 sessionId: src_id|c7468639-67044557-55e2a216-4bba3ec5_0 is DEAD
2025-10-09T21:42:55.184447Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:55.184522Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340762324306979:2476]
2025-10-09T21:42:55.184933Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7468639-67044557-55e2a216-4bba3ec5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:42:55.184953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7468639-67044557-55e2a216-4bba3ec5_0] Write session will restart in 0.007970s
2025-10-09T21:42:55.184998Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7468639-67044557-55e2a216-4bba3ec5_0] Write session: Do CDS request
2025-10-09T21:42:55.185004Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7468639-67044557-55e2a216-4bba3ec5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:55.194985Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c7468639-67044557-55e2a216-4bba3ec5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:55.195263Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:55.195280Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 13
2025-10-09T21:42:55.195446Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 13 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:55.195517Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 13 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:55.195538Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=13 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:55.195542Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:55.196136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 13 sessionId:  describe result for acl check
2025-10-09T21:42:55.196184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:55.196203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:55.196209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:55.196215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340762324307017:2556] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:55.196852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340762324307017:2556] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:55.199347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340762324307017:2556] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:55.201094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:55.201620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:55.201898Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:55.201910Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:55.201924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:55.201982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:55.202065Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340761962498991:2465]
2025-10-09T21:42:55.202709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340762324307017:2556] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:55.202727Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340762324307017:2556] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:55.202731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340762324307017:2556] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:55.202740Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 13 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:55.202731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:55.203096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:55.203167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:55.203607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:55.203669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:55.203677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:55.203678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:55.203679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:55.203681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:55.203682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:55.203683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.203685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.203686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.203694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:55.203704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:55.203708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:55.203712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:55.203717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:55.203721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:55.203724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.203727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.203731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.203738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:55.203747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:55.203754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:55.203759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:55.203764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:55.203770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:55.203774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:55.203780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:55.203785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:55.203791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:55.203795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:55.203800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:55.203805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.203810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.203815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.203821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.203826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.203832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.203866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:55.214895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:55.216197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:55.216212Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:55.216220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:55.216230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:55.216239Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 12 [2:7559340761962498991:2465]
2025-10-09T21:42:55.216258Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:55.216288Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:55.216295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:55.216299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:55.216301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:55.216325Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:55.216338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:55.216347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:55.216460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.216469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.216471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.216474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:55.216573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:55.216584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.216600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.216613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.216631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:55.216662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340762324307066:2556], now have 1 active actors on pipe
2025-10-09T21:42:55.216675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340761962498985:2465]
2025-10-09T21:42:55.216681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340761962498985:2465]
2025-10-09T21:42:55.216684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340761962498985:2465]
2025-10-09T21:42:55.216722Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:55.216747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340762324306994:2432], now have 1 active actors on pipe
2025-10-09T21:42:55.216756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:55.216771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:55.216780Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:55.216788Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 12
2025-10-09T21:42:55.216822Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 12
2025-10-09T21:42:55.216830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340761962498985:2465]
2025-10-09T21:42:55.216835Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:55.216839Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:55.216839Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:55.216843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:55.216850Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:55.216906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:55.216915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:55.216944Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dacd24a6-168569dd-3f3c92c8-81a1e92e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:55.216960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:55.216965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:55.216967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:55.216979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:55.216997Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:55.217023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:55.217235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:55.217251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:55.217276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:55.217363Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 13 partition: 0 MaxSeqNo: 25 sessionId: src_id|dacd24a6-168569dd-3f3c92c8-81a1e92e_0
2025-10-09T21:42:55.217813Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7468639-67044557-55e2a216-4bba3ec5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dacd24a6-168569dd-3f3c92c8-81a1e92e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:55.221002Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dacd24a6-168569dd-3f3c92c8-81a1e92e_0] Write session: try to update token
2025-10-09T21:42:55.221025Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dacd24a6-168569dd-3f3c92c8-81a1e92e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:55.240703Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 13 sessionId: src_id|dacd24a6-168569dd-3f3c92c8-81a1e92e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:55.241388Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:55.241407Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:55.241409Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:55.241411Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:55.241422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340761962498985:2465]
2025-10-09T21:42:55.241428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340761962498985:2465]
2025-10-09T21:42:55.241432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340761962498985:2465]
2025-10-09T21:42:55.241482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:55.241498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.241498Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:55.241504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:55.241507Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:55.241512Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:55.241669Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:55.243711Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:55.284223Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340762324307078:2476]
2025-10-09T21:42:55.284580Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 12, pipe: [1:7559340762324307078:2476]
E1009 21:42:55.294821182 3624801 chttp2_transport.cc:1143]             ipv4:127.0.0.1:14704: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 1250ms
2025-10-09T21:42:55.406647Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:55.406965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:55.406976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:55.407021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340762324307078:2476], now have 1 active actors on pipe
2025-10-09T21:42:55.407335Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:55.407953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dacd24a6-168569dd-3f3c92c8-81a1e92e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:55.407967Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dacd24a6-168569dd-3f3c92c8-81a1e92e_0] Write session will restart in 0.008769s
2025-10-09T21:42:55.407993Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dacd24a6-168569dd-3f3c92c8-81a1e92e_0] Write session: Do CDS request
2025-10-09T21:42:55.408000Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dacd24a6-168569dd-3f3c92c8-81a1e92e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:55.407350Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:55.407380Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:55.407459Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 13 reason: error at write: Disconnected sessionId: src_id|dacd24a6-168569dd-3f3c92c8-81a1e92e_0
2025-10-09T21:42:55.407675Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 13 sessionId: src_id|dacd24a6-168569dd-3f3c92c8-81a1e92e_0 is DEAD
2025-10-09T21:42:55.407856Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340762324307078:2476]
2025-10-09T21:42:55.407871Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:55.415473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:55.416148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:55.416364Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:55.416373Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:55.416385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:55.416433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:55.416475Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340761962499058:2471]
2025-10-09T21:42:55.417103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:55.417399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:55.417437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:55.417704Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dacd24a6-168569dd-3f3c92c8-81a1e92e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:55.417788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:55.417837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:55.417842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:55.417843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:55.417844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:55.417845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:55.417846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:55.417847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.417848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.417850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.417854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:55.417864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:55.417868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:55.417871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:55.417874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:55.417878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:55.417882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.417887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.417890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.417897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:55.417904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:55.417910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:55.417915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:55.417919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:55.417923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:55.417927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:55.417933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:55.417937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:55.417942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:55.417946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:55.417951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:55.417955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.417959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.417964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.417968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.417972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.417977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.418005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:55.418238Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:55.418263Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 14
2025-10-09T21:42:55.418449Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 14 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:55.418494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 14 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:55.418500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=14 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:55.418504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:55.418983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 14 sessionId:  describe result for acl check
2025-10-09T21:42:55.419014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:55.419016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:55.419018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:55.419024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340762324307117:2561] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:55.419644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340762324307117:2561] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:55.421960Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340762324307117:2561] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:55.428629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340762324307117:2561] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:55.428647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340762324307117:2561] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:55.428651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340762324307117:2561] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:55.428662Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 14 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:55.429024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:55.430379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:55.430396Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:55.430399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:55.430404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:55.430411Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 13 [2:7559340761962499058:2471]
2025-10-09T21:42:55.430443Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:55.430462Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:55.430466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:55.430469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:55.430474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:55.430493Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:55.430502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:55.430509Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:55.430568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.430575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.430576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.430579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:55.430655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:55.430666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.430678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.430688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.430704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:55.430796Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 13
2025-10-09T21:42:55.430801Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 13
2025-10-09T21:42:55.430726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340762324307152:2561], now have 1 active actors on pipe
2025-10-09T21:42:55.430734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340762324307083:2432], now have 1 active actors on pipe
2025-10-09T21:42:55.430742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340761962499053:2471]
2025-10-09T21:42:55.430744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340761962499053:2471]
2025-10-09T21:42:55.430746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340761962499053:2471]
2025-10-09T21:42:55.430764Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:55.430778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:55.430792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:55.430795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:55.430807Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:55.430811Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:55.430813Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:55.430837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340761962499053:2471]
2025-10-09T21:42:55.430840Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:55.430852Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:55.430879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:55.430885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:55.430920Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ba2373ef-538c81f4-9bc668d5-f70020d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:55.430934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:55.430939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:55.430941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:55.430954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:55.430965Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:55.430982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:55.431161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:55.431171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:55.431188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:55.431321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 14 partition: 0 MaxSeqNo: 25 sessionId: src_id|ba2373ef-538c81f4-9bc668d5-f70020d_0
2025-10-09T21:42:55.431820Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dacd24a6-168569dd-3f3c92c8-81a1e92e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ba2373ef-538c81f4-9bc668d5-f70020d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:55.434767Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ba2373ef-538c81f4-9bc668d5-f70020d_0] Write session: try to update token
2025-10-09T21:42:55.434786Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ba2373ef-538c81f4-9bc668d5-f70020d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:55.451592Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:55.451611Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:55.451614Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:55.451616Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:55.451630Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340761962499053:2471]
2025-10-09T21:42:55.451635Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340761962499053:2471]
2025-10-09T21:42:55.451639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340761962499053:2471]
2025-10-09T21:42:55.451691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:55.451698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.451702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:55.451872Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:55.454119Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:55.454132Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:55.454137Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:55.455638Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 14 sessionId: src_id|ba2373ef-538c81f4-9bc668d5-f70020d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:55.458397Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:55.507950Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340762324307161:2476]
2025-10-09T21:42:55.508274Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 13, pipe: [1:7559340762324307161:2476]
2025-10-09T21:42:55.613806Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:55.614150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:55.614168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:55.614221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340762324307161:2476], now have 1 active actors on pipe
2025-10-09T21:42:55.614497Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:55.614877Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:55.614890Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:55.614952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 14 reason: error at write: Disconnected sessionId: src_id|ba2373ef-538c81f4-9bc668d5-f70020d_0
2025-10-09T21:42:55.615138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 14 sessionId: src_id|ba2373ef-538c81f4-9bc668d5-f70020d_0 is DEAD
2025-10-09T21:42:55.615346Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba2373ef-538c81f4-9bc668d5-f70020d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:55.615355Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba2373ef-538c81f4-9bc668d5-f70020d_0] Write session will restart in 0.009741s
2025-10-09T21:42:55.615346Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340762324307161:2476]
2025-10-09T21:42:55.615365Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:55.615393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba2373ef-538c81f4-9bc668d5-f70020d_0] Write session: Do CDS request
2025-10-09T21:42:55.615401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba2373ef-538c81f4-9bc668d5-f70020d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:55.623923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:55.624460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:55.624652Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:55.624663Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:55.624675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:55.624723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:55.624771Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340761962499131:2476]
2025-10-09T21:42:55.625372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:55.625711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:55.625775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:55.626198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:55.626270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:55.626277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:55.626278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:55.626280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:55.626282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:55.626330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:55.626332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.626333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.626334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.626341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:55.626350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:55.626358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:55.626361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:55.626364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:55.626368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:55.626372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.626375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.626379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.626400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:55.626424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:55.626432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:55.626441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:55.626446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:55.626453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:55.626458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:55.626463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:55.626468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:55.626473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:55.626477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:55.626487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:55.626492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.626501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.626507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.626516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.626521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.626527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.626556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:55.626688Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ba2373ef-538c81f4-9bc668d5-f70020d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:55.626800Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:55.626813Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 15
2025-10-09T21:42:55.626941Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 15 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:55.626981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 15 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:55.626991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=15 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:55.626996Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:55.627446Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 15 sessionId:  describe result for acl check
2025-10-09T21:42:55.627492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:55.627498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:55.627501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:55.627507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340762324307199:2565] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:55.628092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340762324307199:2565] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:55.631087Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340762324307199:2565] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:55.634279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340762324307199:2565] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:55.634293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340762324307199:2565] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:55.634296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340762324307199:2565] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:55.634306Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 15 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:55.635018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:55.636088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:55.636103Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:55.636106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:55.636112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:55.636120Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 14 [2:7559340761962499131:2476]
2025-10-09T21:42:55.636134Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:55.636154Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:55.636162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:55.636165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:55.636170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:55.636190Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:55.636196Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:55.636203Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:55.636270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.636282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.636283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.636286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:55.636359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:55.636381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.636392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.636406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.636422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:55.636453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340762324307233:2565], now have 1 active actors on pipe
2025-10-09T21:42:55.636456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340762324307165:2432], now have 1 active actors on pipe
2025-10-09T21:42:55.636464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340761962499126:2476]
2025-10-09T21:42:55.636477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340761962499126:2476]
2025-10-09T21:42:55.636483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340761962499126:2476]
2025-10-09T21:42:55.636496Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:55.636512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:55.636513Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:55.636517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:55.636519Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:55.636525Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:55.636528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:55.636505Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 14
2025-10-09T21:42:55.636525Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 14
2025-10-09T21:42:55.636570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340761962499126:2476]
2025-10-09T21:42:55.636584Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:55.636601Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:55.636604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:55.636609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:55.636664Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9e785fdb-e034deaa-8ec8e04c-56d9142e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:55.636676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:55.636681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:55.636684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:55.636695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:55.636707Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:55.636738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:55.636927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:55.636934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:55.636949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:55.637035Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 15 partition: 0 MaxSeqNo: 25 sessionId: src_id|9e785fdb-e034deaa-8ec8e04c-56d9142e_0
2025-10-09T21:42:55.637455Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba2373ef-538c81f4-9bc668d5-f70020d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9e785fdb-e034deaa-8ec8e04c-56d9142e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:55.671484Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e785fdb-e034deaa-8ec8e04c-56d9142e_0] Write session: try to update token
2025-10-09T21:42:55.671509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e785fdb-e034deaa-8ec8e04c-56d9142e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:55.674795Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:55.674809Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:55.674813Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:55.674816Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:55.674826Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340761962499126:2476]
2025-10-09T21:42:55.674831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340761962499126:2476]
2025-10-09T21:42:55.674835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340761962499126:2476]
2025-10-09T21:42:55.674911Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:55.674926Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:55.674931Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:55.678989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:55.679009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.679012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:55.679211Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:55.691675Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 15 sessionId: src_id|9e785fdb-e034deaa-8ec8e04c-56d9142e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:55.701056Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:55.715295Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340762324307245:2476]
2025-10-09T21:42:55.715569Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 14, pipe: [1:7559340762324307245:2476]
2025-10-09T21:42:55.845364Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:55.845658Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:42:55.845663Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:55.845673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340761962499126:2476]
2025-10-09T21:42:55.845730Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:42:55.845739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:55.845749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:55.845771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340762324307245:2476], now have 1 active actors on pipe
2025-10-09T21:42:55.846056Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:55.846060Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:55.846070Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:55.846064Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:55.846118Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340762324307245:2476]
2025-10-09T21:42:55.846141Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 15 reason: error at write: Disconnected sessionId: src_id|9e785fdb-e034deaa-8ec8e04c-56d9142e_0
2025-10-09T21:42:55.846347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 15 sessionId: src_id|9e785fdb-e034deaa-8ec8e04c-56d9142e_0 is DEAD
2025-10-09T21:42:55.850137Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e785fdb-e034deaa-8ec8e04c-56d9142e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:55.850152Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e785fdb-e034deaa-8ec8e04c-56d9142e_0] Write session will restart in 0.009437s
2025-10-09T21:42:55.850178Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e785fdb-e034deaa-8ec8e04c-56d9142e_0] Write session: Do CDS request
2025-10-09T21:42:55.850186Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e785fdb-e034deaa-8ec8e04c-56d9142e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:55.854915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:55.855479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:55.855716Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:55.855726Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:55.855740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:55.855816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:55.855893Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340761962499201:2479]
2025-10-09T21:42:55.856548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:55.856913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:55.856975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:55.857339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:55.857417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:55.857423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:55.857424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:55.857425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:55.857427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:55.857428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:55.857429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.857430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.857431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.857437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:55.857448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:55.857452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:55.857456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:55.857459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:55.857463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:55.857466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.857470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.857473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.857482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:55.857489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:55.857500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:55.857507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:55.857517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:55.857522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:55.857526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:55.857538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:55.857546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:55.857551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:55.857556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:55.857561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:55.857566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.857572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.857597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.857607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.857612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.857617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.857650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:55.860612Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e785fdb-e034deaa-8ec8e04c-56d9142e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:55.860721Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:55.860736Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 16
2025-10-09T21:42:55.860881Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 16 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:55.860919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 16 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:55.860926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=16 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:55.860930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:55.861442Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 16 sessionId:  describe result for acl check
2025-10-09T21:42:55.861483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:55.861484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:55.861486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:55.861491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340762324307292:2571] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:55.862022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340762324307292:2571] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:55.864746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340762324307292:2571] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:55.864804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:55.866071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:55.866086Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:55.866090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:55.866096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:55.866103Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 15 [2:7559340761962499201:2479]
2025-10-09T21:42:55.866118Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:55.866137Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:55.866139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:55.866142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:55.866144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:55.866162Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:55.866176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:55.866185Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:55.866248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.866257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.866259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.866262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:55.866332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:55.866338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.866349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:55.866359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:55.866373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:55.866403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340762324307256:2432], now have 1 active actors on pipe
2025-10-09T21:42:55.866412Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340761962499196:2479]
2025-10-09T21:42:55.866426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340761962499196:2479]
2025-10-09T21:42:55.866428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340761962499196:2479]
2025-10-09T21:42:55.866440Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:55.866459Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:55.866472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:55.866478Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:55.866481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:55.866482Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:55.866485Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:55.866501Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 15
2025-10-09T21:42:55.866516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340761962499196:2479]
2025-10-09T21:42:55.866524Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:55.866534Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:55.873245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340762324307292:2571] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:55.873263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340762324307292:2571] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:55.873268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340762324307292:2571] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:55.873281Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 16 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:55.873885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340762324307329:2571], now have 1 active actors on pipe
2025-10-09T21:42:55.873952Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 15
2025-10-09T21:42:55.874139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:55.874159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:55.874215Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|46125326-7c69e9e8-545b100d-f59f9b19_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:55.874233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:55.874238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:55.874247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:55.874264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:55.874287Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:55.874313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:55.874597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:55.874615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:55.874637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:55.874750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 16 partition: 0 MaxSeqNo: 25 sessionId: src_id|46125326-7c69e9e8-545b100d-f59f9b19_0
2025-10-09T21:42:55.875288Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e785fdb-e034deaa-8ec8e04c-56d9142e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|46125326-7c69e9e8-545b100d-f59f9b19_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:55.878454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46125326-7c69e9e8-545b100d-f59f9b19_0] Write session: try to update token
2025-10-09T21:42:55.878477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46125326-7c69e9e8-545b100d-f59f9b19_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:55.880223Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:42:55.880235Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:55.880244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340761962499196:2479]
2025-10-09T21:42:55.880289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:55.880331Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:42:55.880479Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:42:55.880634Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:55.880795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:55.880991Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:55.881182Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:42:55.881227Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:42:55.898266Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 16 sessionId: src_id|46125326-7c69e9e8-545b100d-f59f9b19_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:55.899020Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:55.899037Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:55.899040Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:55.899042Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:55.899053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340761962499196:2479]
2025-10-09T21:42:55.899056Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340761962499196:2479]
2025-10-09T21:42:55.899061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340761962499196:2479]
2025-10-09T21:42:55.899106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:55.899106Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:55.899112Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:55.899117Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:55.899120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:55.899123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:55.899295Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:55.906782Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:55.946234Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340762324307334:2476]
2025-10-09T21:42:55.946607Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 15, pipe: [1:7559340762324307334:2476]
2025-10-09T21:42:56.064233Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:56.064551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:56.064565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:56.064590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340762324307334:2476], now have 1 active actors on pipe
2025-10-09T21:42:56.064826Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:56.064888Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340762324307334:2476]
2025-10-09T21:42:56.064897Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:56.064902Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:56.064917Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:56.064989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 16 reason: error at write: Disconnected sessionId: src_id|46125326-7c69e9e8-545b100d-f59f9b19_0
2025-10-09T21:42:56.065244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 16 sessionId: src_id|46125326-7c69e9e8-545b100d-f59f9b19_0 is DEAD
2025-10-09T21:42:56.065621Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46125326-7c69e9e8-545b100d-f59f9b19_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:56.065636Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46125326-7c69e9e8-545b100d-f59f9b19_0] Write session will restart in 0.006463s
2025-10-09T21:42:56.065667Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46125326-7c69e9e8-545b100d-f59f9b19_0] Write session: Do CDS request
2025-10-09T21:42:56.065673Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46125326-7c69e9e8-545b100d-f59f9b19_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:56.073593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:56.073650Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46125326-7c69e9e8-545b100d-f59f9b19_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:56.073771Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:56.073795Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 17
2025-10-09T21:42:56.073928Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 17 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:56.073968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 17 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:56.073980Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=17 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:56.073984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:56.074218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:56.074449Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:56.074458Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:56.074470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:56.074524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:56.074553Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 17 sessionId:  describe result for acl check
2025-10-09T21:42:56.074600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:56.074606Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:56.074608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:56.074615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340766619274665:2575] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:56.074633Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340766257466567:2481]
2025-10-09T21:42:56.075234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340766619274665:2575] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:56.075258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:56.075620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:56.075686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:56.076076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:56.076146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:56.076153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:56.076154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:56.076156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:56.076157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:56.076159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:56.076161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.076162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.076163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.076169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:56.076180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:56.076185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:56.076189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:56.076193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:56.076197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:56.076201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.076205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.076209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.076217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:56.076225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:56.076233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:56.076239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:56.076244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:56.076249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:56.076254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:56.076259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:56.076264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:56.076269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:56.076274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:56.076279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:56.076284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.076290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.076294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.076299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.076304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.076310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.076342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:56.077276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340766619274665:2575] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:56.082493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340766619274665:2575] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:56.082508Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340766619274665:2575] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:56.082511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340766619274665:2575] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:56.082521Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 17 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:56.082882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:56.084094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:56.084112Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:56.084116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:56.084121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:56.084128Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 16 [2:7559340766257466567:2481]
2025-10-09T21:42:56.084144Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:56.084165Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:56.084168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:56.084172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:56.084175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:56.084194Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:56.084203Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:56.084219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:56.084285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.084293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.084295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.084298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:56.084378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:56.084387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.084401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.084425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.084441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:56.084468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340766619274633:2432], now have 1 active actors on pipe
2025-10-09T21:42:56.084472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340766619274704:2575], now have 1 active actors on pipe
2025-10-09T21:42:56.084466Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 16
2025-10-09T21:42:56.084480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340766257466562:2481]
2025-10-09T21:42:56.084494Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 16
2025-10-09T21:42:56.084483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340766257466562:2481]
2025-10-09T21:42:56.084485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340766257466562:2481]
2025-10-09T21:42:56.084494Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:56.084510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:56.084511Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:56.084514Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:56.084516Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:56.084526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:56.084530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:56.084583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:56.084595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:56.084610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340766257466562:2481]
2025-10-09T21:42:56.084625Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:56.084636Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:56.084660Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|360e6501-3925eb56-681785a1-83f6cc61_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:56.084673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:56.084682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:56.084685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:56.084698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:56.084711Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:56.084745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:56.084930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:56.084939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:56.084955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:56.085075Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 17 partition: 0 MaxSeqNo: 25 sessionId: src_id|360e6501-3925eb56-681785a1-83f6cc61_0
2025-10-09T21:42:56.085496Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46125326-7c69e9e8-545b100d-f59f9b19_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|360e6501-3925eb56-681785a1-83f6cc61_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:56.088433Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|360e6501-3925eb56-681785a1-83f6cc61_0] Write session: try to update token
2025-10-09T21:42:56.088451Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|360e6501-3925eb56-681785a1-83f6cc61_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:56.102410Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:56.102440Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:56.102443Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:56.102445Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:56.102455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340766257466562:2481]
2025-10-09T21:42:56.102460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340766257466562:2481]
2025-10-09T21:42:56.102464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340766257466562:2481]
2025-10-09T21:42:56.102513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:56.102518Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:56.102522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.102525Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:56.102525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:56.102530Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:56.102858Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:56.104465Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 17 sessionId: src_id|360e6501-3925eb56-681785a1-83f6cc61_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:56.106902Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:56.166103Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340766619274711:2476]
2025-10-09T21:42:56.166641Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 16, pipe: [1:7559340766619274711:2476]
2025-10-09T21:42:56.268251Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:56.268495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:56.268512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:56.268554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340766619274711:2476], now have 1 active actors on pipe
2025-10-09T21:42:56.268777Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:56.268769Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340766619274711:2476]
2025-10-09T21:42:56.268795Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:56.268834Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:56.268842Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:56.268908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 17 reason: error at write: Disconnected sessionId: src_id|360e6501-3925eb56-681785a1-83f6cc61_0
2025-10-09T21:42:56.269083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 17 sessionId: src_id|360e6501-3925eb56-681785a1-83f6cc61_0 is DEAD
2025-10-09T21:42:56.269289Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|360e6501-3925eb56-681785a1-83f6cc61_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:56.269300Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|360e6501-3925eb56-681785a1-83f6cc61_0] Write session will restart in 0.005725s
2025-10-09T21:42:56.269339Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|360e6501-3925eb56-681785a1-83f6cc61_0] Write session: Do CDS request
2025-10-09T21:42:56.269346Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|360e6501-3925eb56-681785a1-83f6cc61_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:56.276705Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|360e6501-3925eb56-681785a1-83f6cc61_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:56.276757Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:56.276770Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 18
2025-10-09T21:42:56.276941Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 18 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:56.276997Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 18 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:56.277006Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=18 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:56.277010Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:56.277493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 18 sessionId:  describe result for acl check
2025-10-09T21:42:56.277532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:56.277538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:56.277540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:56.277546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340766619274771:2586] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:56.277593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:56.278171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340766619274771:2586] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:56.278328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:56.278616Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:56.278625Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:56.278638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:56.278702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:56.278758Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340766257466677:2495]
2025-10-09T21:42:56.279639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:56.279942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:56.279992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:56.280366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:56.280424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:56.280430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:56.280431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:56.280432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:56.280433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:56.280435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:56.280436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.280437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.280438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.280444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:56.280454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:56.280458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:56.280462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:56.280469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:56.280474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:56.280478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.280481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.280485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.280492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:56.280500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:56.280506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:56.280511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:56.280516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:56.280521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:56.280525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:56.280533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:56.280538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:56.280543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:56.280547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:56.280554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:56.280561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.280566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.280570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.280575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.280584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.280589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.280620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:56.280638Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340766619274771:2586] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:56.286322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340766619274771:2586] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:56.286338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340766619274771:2586] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:56.286343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340766619274771:2586] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:56.286353Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 18 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:56.289753Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 17
2025-10-09T21:42:56.289790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 17
2025-10-09T21:42:56.290651Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 18 partition: 0 MaxSeqNo: 25 sessionId: src_id|d7bc55f-fc4a9ed-cba3cf2a-d1d23a92_0
2025-10-09T21:42:56.287411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:56.288585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:56.288592Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:56.288598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:56.288602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:56.288610Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 17 [2:7559340766257466677:2495]
2025-10-09T21:42:56.288626Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:56.288648Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:56.288651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:56.288655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:56.288657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:56.288678Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:56.288684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:56.288692Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:56.288780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.288783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.288785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.288788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:56.288868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:56.288871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.288884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.288897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.288915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:56.288937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340766257466672:2495]
2025-10-09T21:42:56.288940Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340766257466672:2495]
2025-10-09T21:42:56.288942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340766257466672:2495]
2025-10-09T21:42:56.288951Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:56.288964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:56.288967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:56.288969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:56.288970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340766619274739:2432], now have 1 active actors on pipe
2025-10-09T21:42:56.288977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:56.288993Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:56.288996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:56.289005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340766619274808:2586], now have 1 active actors on pipe
2025-10-09T21:42:56.289043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340766257466672:2495]
2025-10-09T21:42:56.291050Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|360e6501-3925eb56-681785a1-83f6cc61_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d7bc55f-fc4a9ed-cba3cf2a-d1d23a92_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:56.293815Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7bc55f-fc4a9ed-cba3cf2a-d1d23a92_0] Write session: try to update token
2025-10-09T21:42:56.293830Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7bc55f-fc4a9ed-cba3cf2a-d1d23a92_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:56.289052Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:56.289086Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:56.290055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:56.290064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:56.290108Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d7bc55f-fc4a9ed-cba3cf2a-d1d23a92_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:56.290119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:56.290125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:56.290128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:56.290142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:56.290156Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:56.290175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:56.290486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:56.290494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:56.290513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:56.309271Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:56.309284Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:56.309287Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:56.309289Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:56.309299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340766257466672:2495]
2025-10-09T21:42:56.309303Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340766257466672:2495]
2025-10-09T21:42:56.309306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340766257466672:2495]
2025-10-09T21:42:56.309357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:56.309368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.309367Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:56.309371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:56.309375Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:56.309379Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:56.309532Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:56.324472Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 18 sessionId: src_id|d7bc55f-fc4a9ed-cba3cf2a-d1d23a92_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:56.327254Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:56.369152Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340766619274817:2476]
2025-10-09T21:42:56.369450Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 17, pipe: [1:7559340766619274817:2476]
2025-10-09T21:42:56.469500Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:56.469804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:56.469822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:56.469868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340766619274817:2476], now have 1 active actors on pipe
2025-10-09T21:42:56.470132Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:56.470193Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:56.470194Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:56.470212Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:56.470245Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340766619274817:2476]
2025-10-09T21:42:56.470287Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 18 reason: error at write: Disconnected sessionId: src_id|d7bc55f-fc4a9ed-cba3cf2a-d1d23a92_0
2025-10-09T21:42:56.470502Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 18 sessionId: src_id|d7bc55f-fc4a9ed-cba3cf2a-d1d23a92_0 is DEAD
2025-10-09T21:42:56.470682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7bc55f-fc4a9ed-cba3cf2a-d1d23a92_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:56.470691Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7bc55f-fc4a9ed-cba3cf2a-d1d23a92_0] Write session will restart in 0.005790s
2025-10-09T21:42:56.470717Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7bc55f-fc4a9ed-cba3cf2a-d1d23a92_0] Write session: Do CDS request
2025-10-09T21:42:56.470723Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7bc55f-fc4a9ed-cba3cf2a-d1d23a92_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:56.477584Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7bc55f-fc4a9ed-cba3cf2a-d1d23a92_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:56.477904Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:56.477932Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 19
2025-10-09T21:42:56.478106Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 19 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:56.478142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 19 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:56.478185Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=19 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:56.478193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:56.478673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 19 sessionId:  describe result for acl check
2025-10-09T21:42:56.478712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:56.478720Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:56.478722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:56.478728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340766619274853:2591] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:56.479001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:56.479597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:56.479817Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:56.479827Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:56.479841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:56.479303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340766619274853:2591] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:56.479897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:56.479940Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340766257466749:2497]
2025-10-09T21:42:56.480495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:56.480805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:56.480852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:56.481552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340766619274853:2591] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:56.481208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:56.481259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:56.481260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:56.481261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:56.481262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:56.481264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:56.481265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:56.481266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.481267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.481268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.481274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:56.481284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:56.481288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:56.481292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:56.481296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:56.481299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:56.481303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.481306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.481309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.481315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:56.481324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:56.481331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:56.481336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:56.481340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:56.481346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:56.481350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:56.481355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:56.481360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:56.481366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:56.481370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:56.481375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:56.481381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.481386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.481391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.481396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.481401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.481406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.481434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:56.489802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:56.491064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:56.491080Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:56.491083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:56.491088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:56.491097Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 18 [2:7559340766257466749:2497]
2025-10-09T21:42:56.491112Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:56.491134Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:56.491141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:56.491145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:56.491148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:56.491166Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:56.491134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340766619274853:2591] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:56.491173Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:56.491180Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:56.491153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340766619274853:2591] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:56.491157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340766619274853:2591] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:56.491169Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 19 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:56.491245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.491256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.491259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.491262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:56.491521Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 18
2025-10-09T21:42:56.491627Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 18
2025-10-09T21:42:56.491343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:56.491349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.491364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.491379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.491400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:56.491421Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340766257466744:2497]
2025-10-09T21:42:56.491424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340766257466744:2497]
2025-10-09T21:42:56.491426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340766257466744:2497]
2025-10-09T21:42:56.491435Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:56.491451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:56.491451Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:56.491454Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:56.491457Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:56.491466Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:56.491469Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:56.491482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340766619274820:2432], now have 1 active actors on pipe
2025-10-09T21:42:56.491526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340766257466744:2497]
2025-10-09T21:42:56.491529Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:56.491541Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:56.491563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340766619274893:2591], now have 1 active actors on pipe
2025-10-09T21:42:56.491698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:56.491704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:56.491733Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bee2ef98-93619134-acc0b059-3de176c9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:56.491742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:56.491747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:56.491750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:56.491763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:56.491776Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:56.491793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:56.491982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:56.491992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:56.492008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:56.492442Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 19 partition: 0 MaxSeqNo: 25 sessionId: src_id|bee2ef98-93619134-acc0b059-3de176c9_0
2025-10-09T21:42:56.492801Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7bc55f-fc4a9ed-cba3cf2a-d1d23a92_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bee2ef98-93619134-acc0b059-3de176c9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:56.495866Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bee2ef98-93619134-acc0b059-3de176c9_0] Write session: try to update token
2025-10-09T21:42:56.495886Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bee2ef98-93619134-acc0b059-3de176c9_0] Send 10 message(s) (0 left), first sequence number is 26
E1009 21:42:56.497280508 3615540 chttp2_transport.cc:1143]             ipv6:%5B::1%5D:14704: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 1250ms
2025-10-09T21:42:56.507382Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:42:56.507402Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:56.507416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340766257466744:2497]
2025-10-09T21:42:56.507480Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:42:56.507487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:56.507653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:42:56.507794Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:56.507951Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:56.508094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:56.508296Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:42:56.508347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:42:56.513206Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 19 sessionId: src_id|bee2ef98-93619134-acc0b059-3de176c9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:56.515898Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:56.516158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:56.516180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:56.516222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:42:56.516309Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:56.527276Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:56.527294Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:56.527297Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:56.527300Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:56.527311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340766257466744:2497]
2025-10-09T21:42:56.527317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340766257466744:2497]
2025-10-09T21:42:56.527322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340766257466744:2497]
2025-10-09T21:42:56.527377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:56.527386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.527386Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:56.527389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:56.527393Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:56.527398Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:56.527539Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:56.570377Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340766619274902:2476]
2025-10-09T21:42:56.571006Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 18, pipe: [1:7559340766619274902:2476]
2025-10-09T21:42:56.702117Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:56.726295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:56.726325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:56.726535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:56.726599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:56.726661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:56.726723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:56.726762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:56.726768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:56.726836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:56.726899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:56.726963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:56.727023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:56.727062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:56.727065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:56.727131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:56.727189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:56.727253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:56.727350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:56.727478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:56.727482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:56.727549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:56.727605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:56.727667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:56.727728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:56.727780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:56.727784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:56.728045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:56.728303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:56.728370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:56.728435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:56.728568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:56.728572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:56.728644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:56.728706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:56.728767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:56.728825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:56.728951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:56.728954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:56.729020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:56.729093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:56.729155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:56.729251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:56.729300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:56.729303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:56.729394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:56.729452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:56.729519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:56.729581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:56.729645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:56.729649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:56.729715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:56.729773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:56.729832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:56.729892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:56.729956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:56.729959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:56.742837Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:56.742848Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:56.742855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:56.742900Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 19 reason: error at write: Disconnected sessionId: src_id|bee2ef98-93619134-acc0b059-3de176c9_0
2025-10-09T21:42:56.742908Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340766619274902:2476]
2025-10-09T21:42:56.743163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 19 sessionId: src_id|bee2ef98-93619134-acc0b059-3de176c9_0 is DEAD
2025-10-09T21:42:56.743540Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bee2ef98-93619134-acc0b059-3de176c9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:56.743554Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bee2ef98-93619134-acc0b059-3de176c9_0] Write session will restart in 0.005761s
2025-10-09T21:42:56.743579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bee2ef98-93619134-acc0b059-3de176c9_0] Write session: Do CDS request
2025-10-09T21:42:56.743585Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bee2ef98-93619134-acc0b059-3de176c9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:56.742089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:56.742184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:56.742245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:56.742309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:56.742381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:56.742387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:56.742500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340766619274902:2476], now have 1 active actors on pipe
2025-10-09T21:42:56.743538Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:56.754483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bee2ef98-93619134-acc0b059-3de176c9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:56.754840Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:56.754855Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 20
2025-10-09T21:42:56.758279Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 20 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:56.758330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 20 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:56.758336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=20 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:56.758341Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:56.759047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 20 sessionId:  describe result for acl check
2025-10-09T21:42:56.759085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:56.759092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:56.759095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:56.759101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340766619274949:2599] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:56.759703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340766619274949:2599] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:56.761224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:56.761427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:56.761613Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:56.761617Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:56.761628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:56.761676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:56.761715Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340766257466825:2500]
2025-10-09T21:42:56.762359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:56.765038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340766619274949:2599] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:56.766548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:56.766620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:56.767015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:56.767062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:56.767063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:56.767064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:56.767066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:56.767068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:56.767069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:56.767071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.767072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.767074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.767080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:56.767091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:56.767096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:56.767101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:56.767107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:56.767112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:56.767117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.767121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.767126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.767133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:56.767142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:56.767149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:56.767155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:56.767163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:56.767169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:56.767175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:56.767181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:56.767186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:56.767192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:56.767197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:56.767203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:56.767209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.767215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.767219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.767225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.767231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.767236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.767270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:56.777471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340766619274949:2599] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:56.777492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340766619274949:2599] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:56.777495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340766619274949:2599] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:56.777506Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 20 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:56.782115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:56.783302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:56.783326Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:56.783331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:56.783336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:56.783345Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 19 [2:7559340766257466825:2500]
2025-10-09T21:42:56.783363Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:56.783389Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:56.783398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:56.783402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:56.783405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:56.783427Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:56.783437Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:56.783444Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:56.783522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.783531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.783534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.783538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:56.783621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:56.783628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.783643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:56.783657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:56.783681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:56.783707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340766257466820:2500]
2025-10-09T21:42:56.783710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340766257466820:2500]
2025-10-09T21:42:56.783712Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340766257466820:2500]
2025-10-09T21:42:56.783720Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:56.783736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:56.783739Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:56.783740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340766619274987:2599], now have 1 active actors on pipe
2025-10-09T21:42:56.783742Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:56.783745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340766619274915:2432], now have 1 active actors on pipe
2025-10-09T21:42:56.783751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:56.783769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:56.783773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:56.783818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340766257466820:2500]
2025-10-09T21:42:56.783822Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:56.783899Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:56.784052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:56.783812Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 19
2025-10-09T21:42:56.783816Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 19
2025-10-09T21:42:56.784058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:56.784104Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8cb50a7e-4e039a43-59f36cc7-ac91b790_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:56.784114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:56.784120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:56.784122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:56.784133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:56.784145Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:56.784162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:56.784310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:56.784319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:56.784334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:56.784623Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 20 partition: 0 MaxSeqNo: 25 sessionId: src_id|8cb50a7e-4e039a43-59f36cc7-ac91b790_0
2025-10-09T21:42:56.786603Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bee2ef98-93619134-acc0b059-3de176c9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8cb50a7e-4e039a43-59f36cc7-ac91b790_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:56.789616Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8cb50a7e-4e039a43-59f36cc7-ac91b790_0] Write session: try to update token
2025-10-09T21:42:56.789636Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8cb50a7e-4e039a43-59f36cc7-ac91b790_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:56.807755Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:56.807776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:56.807779Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:56.807781Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:56.807791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340766257466820:2500]
2025-10-09T21:42:56.807796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340766257466820:2500]
2025-10-09T21:42:56.807801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340766257466820:2500]
2025-10-09T21:42:56.807841Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:56.807846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:56.807854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:56.807857Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:56.807857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:56.807861Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:56.807989Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:56.823181Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 20 sessionId: src_id|8cb50a7e-4e039a43-59f36cc7-ac91b790_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:56.825554Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:56.843316Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340766619274998:2476]
2025-10-09T21:42:56.846398Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 19, pipe: [1:7559340766619274998:2476]
2025-10-09T21:42:56.984098Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:56.984471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:56.984483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:56.984508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340766619274998:2476], now have 1 active actors on pipe
2025-10-09T21:42:56.985178Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340766619274998:2476]
2025-10-09T21:42:56.985228Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:56.985232Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:56.985315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 20 reason: error at write: Disconnected sessionId: src_id|8cb50a7e-4e039a43-59f36cc7-ac91b790_0
2025-10-09T21:42:56.985564Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:56.985916Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8cb50a7e-4e039a43-59f36cc7-ac91b790_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:56.985928Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8cb50a7e-4e039a43-59f36cc7-ac91b790_0] Write session will restart in 0.008399s
2025-10-09T21:42:56.985954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8cb50a7e-4e039a43-59f36cc7-ac91b790_0] Write session: Do CDS request
2025-10-09T21:42:56.985960Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8cb50a7e-4e039a43-59f36cc7-ac91b790_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:56.986105Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:56.987142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 20 sessionId: src_id|8cb50a7e-4e039a43-59f36cc7-ac91b790_0 is DEAD
2025-10-09T21:42:56.998221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:56.998408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:56.998586Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:56.998590Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:56.998602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:56.998644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:56.998683Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340766257466901:2503]
2025-10-09T21:42:56.999295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:56.999804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:56.999844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:57.000187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:57.000231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:57.000232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:57.000234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:57.000235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:57.000236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:57.000237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:57.000239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.000240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.000241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.000246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:57.000257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:57.000262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:57.000266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:57.000270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:57.000273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:57.000279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.000283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.000287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.000295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:57.000303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:57.000309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:57.000316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:57.000320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:57.000325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:57.000329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:57.000334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:57.000340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:57.000345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:57.000349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:57.000354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:57.000359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.000364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.000368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.000373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.000377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.000382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.000411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:57.000620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8cb50a7e-4e039a43-59f36cc7-ac91b790_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:57.001847Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:57.001873Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 21
2025-10-09T21:42:57.002002Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 21 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:57.002044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 21 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:57.002050Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=21 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:57.002056Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:57.002986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 21 sessionId:  describe result for acl check
2025-10-09T21:42:57.003028Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:57.003030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:57.003032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:57.003038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340770914242340:2603] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:57.003626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340770914242340:2603] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:57.011367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:57.012594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:57.012612Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:57.012617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:57.012622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:57.012631Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 20 [2:7559340766257466901:2503]
2025-10-09T21:42:57.012650Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:57.012672Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:57.012675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.012683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.012687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.012707Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:57.012719Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:57.012727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:57.012806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.012814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.012816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.012819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:57.012899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:57.012910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.012923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.012934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.012950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:57.013179Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 20
2025-10-09T21:42:57.012974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340766619275004:2432], now have 1 active actors on pipe
2025-10-09T21:42:57.012990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340766257466894:2503]
2025-10-09T21:42:57.012993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340766257466894:2503]
2025-10-09T21:42:57.012995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340766257466894:2503]
2025-10-09T21:42:57.013008Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:57.013025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:57.013037Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:57.013040Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:57.013041Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:57.013044Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:57.013044Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:57.013090Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340766257466894:2503]
2025-10-09T21:42:57.013097Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:57.013106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:57.014165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340770914242340:2603] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:57.028506Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340770914242340:2603] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:57.028526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340770914242340:2603] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:57.028530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340770914242340:2603] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:57.028542Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 21 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:57.029539Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 20
2025-10-09T21:42:57.029466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340770914242374:2603], now have 1 active actors on pipe
2025-10-09T21:42:57.029700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.029713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.029752Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7db06a26-12bd219d-6f4a78f8-f840a94d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:57.029767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.029774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.029778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:57.029793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.029816Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:57.029840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:57.030529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.030543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.030565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:57.030826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 21 partition: 0 MaxSeqNo: 25 sessionId: src_id|7db06a26-12bd219d-6f4a78f8-f840a94d_0
2025-10-09T21:42:57.031312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8cb50a7e-4e039a43-59f36cc7-ac91b790_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7db06a26-12bd219d-6f4a78f8-f840a94d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:57.034776Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7db06a26-12bd219d-6f4a78f8-f840a94d_0] Write session: try to update token
2025-10-09T21:42:57.034804Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7db06a26-12bd219d-6f4a78f8-f840a94d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:57.035173Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:57.035188Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:57.035190Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:57.035192Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:57.035202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340766257466894:2503]
2025-10-09T21:42:57.035210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340766257466894:2503]
2025-10-09T21:42:57.035215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340766257466894:2503]
2025-10-09T21:42:57.035260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:57.035267Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:57.035269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.035272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:57.035274Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:57.035279Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:57.035522Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:57.073980Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 21 sessionId: src_id|7db06a26-12bd219d-6f4a78f8-f840a94d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:57.076836Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:57.085484Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340770914242379:2476]
2025-10-09T21:42:57.086180Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 20, pipe: [1:7559340770914242379:2476]
2025-10-09T21:42:57.206346Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:57.206705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.206723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.206747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340770914242379:2476], now have 1 active actors on pipe
2025-10-09T21:42:57.207027Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:57.207466Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:57.207480Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:57.207549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 21 reason: error at write: Disconnected sessionId: src_id|7db06a26-12bd219d-6f4a78f8-f840a94d_0
2025-10-09T21:42:57.207715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 21 sessionId: src_id|7db06a26-12bd219d-6f4a78f8-f840a94d_0 is DEAD
2025-10-09T21:42:57.207878Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:57.207913Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340770914242379:2476]
2025-10-09T21:42:57.208461Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7db06a26-12bd219d-6f4a78f8-f840a94d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:57.208473Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7db06a26-12bd219d-6f4a78f8-f840a94d_0] Write session will restart in 0.005687s
2025-10-09T21:42:57.208501Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7db06a26-12bd219d-6f4a78f8-f840a94d_0] Write session: Do CDS request
2025-10-09T21:42:57.208508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7db06a26-12bd219d-6f4a78f8-f840a94d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:57.215713Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7db06a26-12bd219d-6f4a78f8-f840a94d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:57.218211Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:57.218227Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 22
2025-10-09T21:42:57.222194Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 22 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:57.222249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 22 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:57.222256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=22 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:57.222262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:57.223394Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 22 sessionId:  describe result for acl check
2025-10-09T21:42:57.223442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:57.223450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:57.223452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:57.223458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340770914242416:2607] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:57.224092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340770914242416:2607] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:57.228062Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340770914242416:2607] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:57.234941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340770914242416:2607] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:57.234957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340770914242416:2607] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:57.234961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340770914242416:2607] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:57.234969Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 22 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:57.242212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:57.243410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:57.243654Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:57.243659Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:57.243671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:57.243751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:57.243794Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340770552434273:2505]
2025-10-09T21:42:57.244452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:57.244835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:57.244881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:57.245200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:57.245259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:57.245260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:57.245261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:57.245263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:57.245264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:57.245266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:57.245267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.245268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.245269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.245274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:57.245284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:57.245289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:57.245293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:57.245296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:57.245300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:57.245304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.245311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.245315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.245323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:57.245330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:57.245335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:57.245341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:57.245345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:57.245351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:57.245354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:57.245359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:57.245364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:57.245369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:57.245374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:57.245379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:57.245386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.245391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.245396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.245403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.245408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.245414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.245444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:57.254140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:57.255516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:57.255537Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:57.255542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:57.255547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:57.255559Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 21 [2:7559340770552434273:2505]
2025-10-09T21:42:57.255577Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:57.255600Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:57.255603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.255607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.255614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.255635Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:57.255642Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:57.255653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:57.255737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.255749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.255752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.255755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:57.255839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:57.255848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.255863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.255914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.255937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:57.255967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340770914242453:2607], now have 1 active actors on pipe
2025-10-09T21:42:57.255977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340770914242381:2432], now have 1 active actors on pipe
2025-10-09T21:42:57.255990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340770552434267:2505]
2025-10-09T21:42:57.255997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340770552434267:2505]
2025-10-09T21:42:57.256000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340770552434267:2505]
2025-10-09T21:42:57.256015Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:57.256033Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:57.256036Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:57.256043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:57.256045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:57.256052Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:57.256023Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 21
2025-10-09T21:42:57.256056Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:57.256027Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 21
2025-10-09T21:42:57.256100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340770552434267:2505]
2025-10-09T21:42:57.256116Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:57.256135Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:57.256311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.256325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.256361Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cd2c7fe0-59f8bb81-265a9e93-2c22c52e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:57.256376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.256385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.256389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:57.256402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.256424Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:57.256455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:57.256789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.256805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.256837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:57.256970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 22 partition: 0 MaxSeqNo: 25 sessionId: src_id|cd2c7fe0-59f8bb81-265a9e93-2c22c52e_0
2025-10-09T21:42:57.257414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7db06a26-12bd219d-6f4a78f8-f840a94d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cd2c7fe0-59f8bb81-265a9e93-2c22c52e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:57.260817Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd2c7fe0-59f8bb81-265a9e93-2c22c52e_0] Write session: try to update token
2025-10-09T21:42:57.260838Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd2c7fe0-59f8bb81-265a9e93-2c22c52e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:57.277883Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:42:57.277902Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:57.277911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340770552434267:2505]
2025-10-09T21:42:57.277963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:57.278159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:42:57.278340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:57.278490Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:57.278630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:57.278810Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:42:57.278847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:42:57.279034Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:42:57.296317Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 22 sessionId: src_id|cd2c7fe0-59f8bb81-265a9e93-2c22c52e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:57.299373Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:57.299989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.300013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.302810Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:57.302830Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:57.302832Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:57.302834Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:57.302843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340770552434267:2505]
2025-10-09T21:42:57.302847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340770552434267:2505]
2025-10-09T21:42:57.302851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340770552434267:2505]
2025-10-09T21:42:57.302897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:42:57.302913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:57.302920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.302922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:57.303064Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:57.304343Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:57.304376Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:57.304381Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:57.305106Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:57.324873Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340770914242478:2476]
2025-10-09T21:42:57.328160Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 21, pipe: [1:7559340770914242478:2476]
2025-10-09T21:42:57.480370Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:57.480718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340770914242478:2476], now have 1 active actors on pipe
2025-10-09T21:42:57.491192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.491226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.491399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:57.491462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:57.491527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:57.491587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:57.492533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:57.492541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:57.492610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:57.492663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:57.492725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:57.492784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:57.492820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:57.492824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:57.492885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:57.492937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:57.492992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:57.493049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:57.493094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:57.493098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:57.493171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:57.493234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:57.493299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:57.493364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:57.493409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:57.493412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:57.493483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:57.493548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:57.493605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:57.493660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:57.493703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:57.493706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:57.493766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:57.493818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:57.493872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:57.493935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:57.493998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:57.494002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:57.494848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:57.494924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:57.494998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:57.495069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:57.495113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:57.495118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:57.495192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:57.495256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:57.495321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:57.495378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:57.495425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:57.495429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:57.495493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:57.495558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:57.495613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:57.495668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:57.495707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:57.495711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:57.495775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:57.495841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:57.495895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:57.495954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:57.495992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:57.495995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:57.498934Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:57.499238Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:57.499251Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:57.499286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 22 reason: error at write: Disconnected sessionId: src_id|cd2c7fe0-59f8bb81-265a9e93-2c22c52e_0
2025-10-09T21:42:57.499484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 22 sessionId: src_id|cd2c7fe0-59f8bb81-265a9e93-2c22c52e_0 is DEAD
2025-10-09T21:42:57.499660Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340770914242478:2476]
2025-10-09T21:42:57.499680Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:57.499836Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd2c7fe0-59f8bb81-265a9e93-2c22c52e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:57.499848Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd2c7fe0-59f8bb81-265a9e93-2c22c52e_0] Write session will restart in 0.007117s
2025-10-09T21:42:57.499879Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd2c7fe0-59f8bb81-265a9e93-2c22c52e_0] Write session: Do CDS request
2025-10-09T21:42:57.499886Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd2c7fe0-59f8bb81-265a9e93-2c22c52e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:57.513270Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd2c7fe0-59f8bb81-265a9e93-2c22c52e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:57.513627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:57.513644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 23
2025-10-09T21:42:57.513876Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 23 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:57.513910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 23 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:57.513916Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=23 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:57.513920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:57.514840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 23 sessionId:  describe result for acl check
2025-10-09T21:42:57.514890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:57.514893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:57.514895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:57.514901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340770914242507:2619] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:57.515490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340770914242507:2619] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:57.529198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340770914242507:2619] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:57.530641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:57.535295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:57.535555Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:57.535565Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:57.535577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:57.535636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:57.535675Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340770552434391:2519]
2025-10-09T21:42:57.536349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:57.538119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:57.538201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:57.538671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:57.538746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:57.538747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:57.538749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:57.538750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:57.538752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:57.538753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:57.538755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.538756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.538757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.538765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:57.538777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:57.538781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:57.538785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:57.538790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:57.538795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:57.538799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.538803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.538807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.538817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:57.538826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:57.538834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:57.538839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:57.538844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:57.538850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:57.538856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:57.538861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:57.538867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:57.538872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:57.538878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:57.538883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:57.538890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.538895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.538900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.538906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.538911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.538917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.538952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:57.549347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340770914242507:2619] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:57.549364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340770914242507:2619] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:57.549369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340770914242507:2619] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:57.549379Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 23 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:57.558839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:57.559925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:57.559932Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:57.559936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:57.559941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:57.559950Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 22 [2:7559340770552434391:2519]
2025-10-09T21:42:57.559968Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:57.559989Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:57.559992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.559997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.560000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.560019Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:57.560025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:57.560033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:57.560106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.560109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.560112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.560115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:57.560214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:57.560217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.560233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.560246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.560264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:57.560287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340770914242492:2432], now have 1 active actors on pipe
2025-10-09T21:42:57.560295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340770552434386:2519]
2025-10-09T21:42:57.560298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340770552434386:2519]
2025-10-09T21:42:57.560303Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340770552434386:2519]
2025-10-09T21:42:57.560314Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:57.560324Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:57.560328Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:57.560330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:57.560332Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:57.560346Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:57.560350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:57.560356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340770914242570:2619], now have 1 active actors on pipe
2025-10-09T21:42:57.560399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340770552434386:2519]
2025-10-09T21:42:57.560413Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:57.560578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:57.560814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.560824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.560854Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9f6ecfc0-feb1e1fa-350c54a8-7841d8dc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:57.560862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.560867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.560870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:57.560882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.560895Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:57.560913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:57.560606Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 22
2025-10-09T21:42:57.560657Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 22
2025-10-09T21:42:57.565903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.565921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.565958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:57.566178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 23 partition: 0 MaxSeqNo: 25 sessionId: src_id|9f6ecfc0-feb1e1fa-350c54a8-7841d8dc_0
2025-10-09T21:42:57.566662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd2c7fe0-59f8bb81-265a9e93-2c22c52e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9f6ecfc0-feb1e1fa-350c54a8-7841d8dc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:57.569722Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f6ecfc0-feb1e1fa-350c54a8-7841d8dc_0] Write session: try to update token
2025-10-09T21:42:57.569745Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f6ecfc0-feb1e1fa-350c54a8-7841d8dc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:57.585882Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:57.585899Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:57.585901Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:57.585904Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:57.585914Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340770552434386:2519]
2025-10-09T21:42:57.585919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340770552434386:2519]
2025-10-09T21:42:57.585924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340770552434386:2519]
2025-10-09T21:42:57.585975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:57.585983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.585986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:57.585985Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:57.585993Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:57.585997Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:57.586159Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:57.598471Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340770914242577:2476]
2025-10-09T21:42:57.598967Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 22, pipe: [1:7559340770914242577:2476]
2025-10-09T21:42:57.614236Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 23 sessionId: src_id|9f6ecfc0-feb1e1fa-350c54a8-7841d8dc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:57.616636Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:57.769249Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:57.769941Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:57.769954Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:57.770053Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 23 reason: error at write: Disconnected sessionId: src_id|9f6ecfc0-feb1e1fa-350c54a8-7841d8dc_0
2025-10-09T21:42:57.770231Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:57.770262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 23 sessionId: src_id|9f6ecfc0-feb1e1fa-350c54a8-7841d8dc_0 is DEAD
2025-10-09T21:42:57.770290Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340770914242577:2476]
2025-10-09T21:42:57.769615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340770914242577:2476], now have 1 active actors on pipe
2025-10-09T21:42:57.769643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.769652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.770875Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f6ecfc0-feb1e1fa-350c54a8-7841d8dc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:57.770891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f6ecfc0-feb1e1fa-350c54a8-7841d8dc_0] Write session will restart in 0.005553s
2025-10-09T21:42:57.770915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f6ecfc0-feb1e1fa-350c54a8-7841d8dc_0] Write session: Do CDS request
2025-10-09T21:42:57.770925Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f6ecfc0-feb1e1fa-350c54a8-7841d8dc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:57.772167Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:57.777650Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f6ecfc0-feb1e1fa-350c54a8-7841d8dc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:57.777705Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:57.777721Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 24
2025-10-09T21:42:57.778366Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 24 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:57.778406Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 24 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:57.778412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=24 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:57.778429Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:57.778978Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 24 sessionId:  describe result for acl check
2025-10-09T21:42:57.779024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:57.779032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:57.779034Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:57.779042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340770914242600:2625] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:57.779632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340770914242600:2625] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:57.783573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340770914242600:2625] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:57.784326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:57.785305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:57.785512Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:57.785538Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:57.785555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:57.785613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:57.785660Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340770552434470:2522]
2025-10-09T21:42:57.786245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:57.786613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:57.786680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:57.787023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:57.787077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:57.787078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:57.787079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:57.787080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:57.787081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:57.787082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:57.787084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.787085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.787086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.787091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:57.787102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:57.787106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:57.787110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:57.787114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:57.787118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:57.787122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.787126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.787130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.787137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:57.787145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:57.787151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:57.787157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:57.787161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:57.787166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:57.787172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:57.787176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:57.787181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:57.787185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:57.787190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:57.787195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:57.787200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.787205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.787209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.787214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.787219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.787223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.787250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:57.790400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340770914242600:2625] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:57.790429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340770914242600:2625] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:57.790434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340770914242600:2625] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:57.790468Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 24 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:57.795248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:57.796335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:57.796347Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:57.796351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:57.796356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:57.796364Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 23 [2:7559340770552434470:2522]
2025-10-09T21:42:57.796380Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:57.796401Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:57.796404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.796408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.796410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.796429Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:57.796440Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:57.796448Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:57.796514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.796521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.796523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.796526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:57.796603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:57.796611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.796624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:57.796640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:57.796660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:57.796684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340770552434464:2522]
2025-10-09T21:42:57.796692Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340770552434464:2522]
2025-10-09T21:42:57.796694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340770552434464:2522]
2025-10-09T21:42:57.796707Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:57.796726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:57.796740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:57.796744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:57.796785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340770552434464:2522]
2025-10-09T21:42:57.796792Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:57.796978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:57.796986Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:57.796988Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:57.796994Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:57.797848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340770914242664:2625], now have 1 active actors on pipe
2025-10-09T21:42:57.797859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340770914242587:2432], now have 1 active actors on pipe
2025-10-09T21:42:57.797879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.797888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.797475Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 23
2025-10-09T21:42:57.797488Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 23
2025-10-09T21:42:57.797930Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f2b8328e-3e7c5fdc-6db23c08-e0150bd8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:57.797943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.797948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.797956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:57.797976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.797996Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:57.798019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:57.798531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.798538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.798555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:57.798700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 24 partition: 0 MaxSeqNo: 25 sessionId: src_id|f2b8328e-3e7c5fdc-6db23c08-e0150bd8_0
2025-10-09T21:42:57.799090Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f6ecfc0-feb1e1fa-350c54a8-7841d8dc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f2b8328e-3e7c5fdc-6db23c08-e0150bd8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:57.801958Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2b8328e-3e7c5fdc-6db23c08-e0150bd8_0] Write session: try to update token
2025-10-09T21:42:57.801979Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2b8328e-3e7c5fdc-6db23c08-e0150bd8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:57.813876Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:57.813894Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:57.813897Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:57.813899Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:57.813910Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340770552434464:2522]
2025-10-09T21:42:57.813915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340770552434464:2522]
2025-10-09T21:42:57.813923Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340770552434464:2522]
2025-10-09T21:42:57.813971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:57.813978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:57.813980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:57.814153Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:57.816283Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:57.816292Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:57.816297Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:57.826549Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 24 sessionId: src_id|f2b8328e-3e7c5fdc-6db23c08-e0150bd8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:57.829158Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:57.870340Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340770914242675:2476]
2025-10-09T21:42:57.870796Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 23, pipe: [1:7559340770914242675:2476]
2025-10-09T21:42:57.976982Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:57.977479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.977492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.977517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340770914242675:2476], now have 1 active actors on pipe
2025-10-09T21:42:57.978049Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:57.978522Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:57.978601Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340770914242675:2476]
2025-10-09T21:42:57.978633Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:57.978731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 24 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f2b8328e-3e7c5fdc-6db23c08-e0150bd8_0
2025-10-09T21:42:57.978924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 24 sessionId: src_id|f2b8328e-3e7c5fdc-6db23c08-e0150bd8_0 is DEAD
2025-10-09T21:42:57.981445Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2b8328e-3e7c5fdc-6db23c08-e0150bd8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:42:57.981461Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2b8328e-3e7c5fdc-6db23c08-e0150bd8_0] Write session will restart in 0.009233s
2025-10-09T21:42:57.981489Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2b8328e-3e7c5fdc-6db23c08-e0150bd8_0] Write session: Do CDS request
2025-10-09T21:42:57.981496Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2b8328e-3e7c5fdc-6db23c08-e0150bd8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:57.994590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2b8328e-3e7c5fdc-6db23c08-e0150bd8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:57.995002Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:57.995021Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 25
2025-10-09T21:42:57.995210Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 25 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:57.995248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 25 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:57.995255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=25 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:57.995265Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:57.995924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 25 sessionId:  describe result for acl check
2025-10-09T21:42:57.995977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:57.995979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:57.995981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:57.995987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340770914242692:2629] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:57.996588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340770914242692:2629] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:57.998920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340770914242692:2629] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:58.004068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:58.005696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:58.005931Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:58.005935Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:58.005946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:58.006011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:58.006060Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340774847401848:2525]
2025-10-09T21:42:58.006698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:58.008198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:58.008267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:58.008662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:58.008724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:58.008732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:58.008734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:58.008735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:58.008736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:58.008738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:58.008739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.008740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.008742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.008748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:58.008759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:58.008762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:58.008767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:58.008770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:58.008774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:58.008778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.008783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.008787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.008795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:58.008804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:58.008810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:58.008815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:58.008822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:58.008827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:58.008831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:58.008836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:58.008841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:58.008845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:58.008849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:58.008854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:58.008859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.008875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.008878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.008884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.008890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.008895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.008927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:58.010929Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340770914242692:2629] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:58.010948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340770914242692:2629] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:58.010953Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340770914242692:2629] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:58.010963Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 25 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:58.023325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:58.024633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:58.024653Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:58.024656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:58.024659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:58.024667Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 24 [2:7559340774847401848:2525]
2025-10-09T21:42:58.024680Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:58.024696Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:58.024699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.024701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.024703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.024717Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:58.024722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:58.024727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:58.024779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.024781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.024783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.024785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:58.024994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:58.025018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.025043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.025063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.025100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:58.025143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340770914242681:2432], now have 1 active actors on pipe
2025-10-09T21:42:58.025154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340775209210052:2629], now have 1 active actors on pipe
2025-10-09T21:42:58.025171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340774847401843:2525]
2025-10-09T21:42:58.025173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340774847401843:2525]
2025-10-09T21:42:58.025175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340774847401843:2525]
2025-10-09T21:42:58.025160Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 24
2025-10-09T21:42:58.025196Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:58.025219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:58.025222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 24
2025-10-09T21:42:58.025237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:58.025246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:58.025257Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:58.025267Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:58.025269Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:58.025298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340774847401843:2525]
2025-10-09T21:42:58.025303Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:58.025305Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:58.025360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.025373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.025413Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|70447579-dd462bef-70ae6d7f-37f684e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:58.025429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.025434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.025437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:58.025449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.025466Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:58.025497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:58.026104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.026116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.026148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:58.034229Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2b8328e-3e7c5fdc-6db23c08-e0150bd8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|70447579-dd462bef-70ae6d7f-37f684e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:58.033871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 25 partition: 0 MaxSeqNo: 25 sessionId: src_id|70447579-dd462bef-70ae6d7f-37f684e_0
2025-10-09T21:42:58.037284Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70447579-dd462bef-70ae6d7f-37f684e_0] Write session: try to update token
2025-10-09T21:42:58.037307Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70447579-dd462bef-70ae6d7f-37f684e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:58.044333Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:58.044353Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:58.044356Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:58.044358Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:58.044367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340774847401843:2525]
2025-10-09T21:42:58.044372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340774847401843:2525]
2025-10-09T21:42:58.044376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340774847401843:2525]
2025-10-09T21:42:58.044422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:58.044429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.044432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:58.044581Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:58.046239Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:58.046250Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:58.046254Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:58.071754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 25 sessionId: src_id|70447579-dd462bef-70ae6d7f-37f684e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:58.074805Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:58.078753Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340775209210063:2476]
2025-10-09T21:42:58.079521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 24, pipe: [1:7559340775209210063:2476]
2025-10-09T21:42:58.272060Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:58.272078Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:58.272153Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 25 reason: error at write: Disconnected sessionId: src_id|70447579-dd462bef-70ae6d7f-37f684e_0
2025-10-09T21:42:58.272152Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:58.272203Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340775209210063:2476]
2025-10-09T21:42:58.272345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 25 sessionId: src_id|70447579-dd462bef-70ae6d7f-37f684e_0 is DEAD
2025-10-09T21:42:58.273061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70447579-dd462bef-70ae6d7f-37f684e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:58.273072Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70447579-dd462bef-70ae6d7f-37f684e_0] Write session will restart in 0.006221s
2025-10-09T21:42:58.273096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70447579-dd462bef-70ae6d7f-37f684e_0] Write session: Do CDS request
2025-10-09T21:42:58.273102Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70447579-dd462bef-70ae6d7f-37f684e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:58.271286Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:58.271615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.271625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.271647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340775209210063:2476], now have 1 active actors on pipe
2025-10-09T21:42:58.272706Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:58.280619Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70447579-dd462bef-70ae6d7f-37f684e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:58.281487Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:58.281500Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 26
2025-10-09T21:42:58.281631Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 26 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:58.281682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 26 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:58.281692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=26 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:58.281696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:58.282905Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 26 sessionId:  describe result for acl check
2025-10-09T21:42:58.282954Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:58.282956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:58.282958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:58.282964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340775209210106:2633] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:58.283557Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340775209210106:2633] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:58.283973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:58.285443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:58.285684Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:58.285688Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:58.285701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:58.285759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:58.285798Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340774847401929:2527]
2025-10-09T21:42:58.286812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340775209210106:2633] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:58.286482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:58.287678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:58.287727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:58.288098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:58.288151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:58.288153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:58.288154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:58.288156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:58.288157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:58.288158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:58.288160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.288161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.288162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.288168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:58.288179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:58.288183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:58.288187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:58.288192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:58.288196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:58.288200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.288204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.288209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.288216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:58.288225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:58.288232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:58.288238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:58.288243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:58.288248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:58.288253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:58.288258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:58.288263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:58.288268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:58.288272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:58.288278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:58.288283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.288288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.288293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.288298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.288304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.288309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.288341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:58.301135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340775209210106:2633] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:58.301156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340775209210106:2633] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:58.301161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340775209210106:2633] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:58.301171Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 26 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:58.303397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:58.304576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:58.304583Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:58.304586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:58.304591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:58.304599Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 25 [2:7559340774847401929:2527]
2025-10-09T21:42:58.304616Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:58.304638Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:58.304641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.304645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.304648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.304668Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:58.304675Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:58.304686Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:58.304754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.304757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.304760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.304763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:58.304847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:58.304853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.304866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.304879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.304897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:58.304919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340774847401922:2527]
2025-10-09T21:42:58.304921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340774847401922:2527]
2025-10-09T21:42:58.304924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340774847401922:2527]
2025-10-09T21:42:58.304939Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:58.304955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:58.304973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:58.304976Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:58.305021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340774847401922:2527]
2025-10-09T21:42:58.305024Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:58.305130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:58.305133Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:58.305136Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:58.305141Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:58.305929Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 25
2025-10-09T21:42:58.307413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340775209210066:2432], now have 1 active actors on pipe
2025-10-09T21:42:58.307442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340775209210146:2633], now have 1 active actors on pipe
2025-10-09T21:42:58.307461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.307468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.307514Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c3f915d9-956d33ca-a24939e-b91dcf5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:58.307528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.307534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.307540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:58.307554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.307582Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:58.307603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:58.307983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.307989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.308010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:58.306743Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 25
2025-10-09T21:42:58.308183Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 26 partition: 0 MaxSeqNo: 25 sessionId: src_id|c3f915d9-956d33ca-a24939e-b91dcf5_0
2025-10-09T21:42:58.310655Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70447579-dd462bef-70ae6d7f-37f684e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c3f915d9-956d33ca-a24939e-b91dcf5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:58.314514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f915d9-956d33ca-a24939e-b91dcf5_0] Write session: try to update token
2025-10-09T21:42:58.314547Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f915d9-956d33ca-a24939e-b91dcf5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:58.337526Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:42:58.337551Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:58.337561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340774847401922:2527]
2025-10-09T21:42:58.337613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:58.337782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:42:58.337924Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:58.338104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:58.338238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:58.338270Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:42:58.338459Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:42:58.338510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:42:58.344047Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 26 sessionId: src_id|c3f915d9-956d33ca-a24939e-b91dcf5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:58.346722Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:58.347067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.347090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.347141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:42:58.347394Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:58.367514Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:58.367531Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:58.367533Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:58.367535Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:58.367545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340774847401922:2527]
2025-10-09T21:42:58.367550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340774847401922:2527]
2025-10-09T21:42:58.367553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340774847401922:2527]
2025-10-09T21:42:58.367597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:58.367604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.367607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:58.367609Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:58.367613Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:58.367617Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:58.367767Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:58.372077Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340775209210169:2476]
2025-10-09T21:42:58.372534Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 25, pipe: [1:7559340775209210169:2476]
2025-10-09T21:42:58.550502Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:58.559167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.559189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.559362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:58.559422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:58.559486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:58.559557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:58.559665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:58.559670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:58.559738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:58.559816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:58.559878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:58.559943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:58.559981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:58.559984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:58.560058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:58.560113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:58.560173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:58.560237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:58.560276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:58.560279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:58.560342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:58.560404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:58.560458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:58.560520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:58.560561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:58.560564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:58.560633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:58.560688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:58.560750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:58.560808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:58.560850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:58.560853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:58.560961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:58.561017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:58.561078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:58.561139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:58.561189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:58.561194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:58.561265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:58.561321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:58.561457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:58.561523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:58.561572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:58.561576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:58.561640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:58.561697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:58.561763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:58.561829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:58.561869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:58.561871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:58.561939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:58.562000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:58.563709Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:58.563760Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340775209210169:2476]
2025-10-09T21:42:58.563790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:58.563793Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:58.563821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 26 reason: error at write: Disconnected sessionId: src_id|c3f915d9-956d33ca-a24939e-b91dcf5_0
2025-10-09T21:42:58.564003Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 26 sessionId: src_id|c3f915d9-956d33ca-a24939e-b91dcf5_0 is DEAD
2025-10-09T21:42:58.562479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:58.562544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:58.562587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:58.562589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:58.562655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:42:58.562713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:42:58.562770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:42:58.562830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:42:58.562869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:42:58.562872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:42:58.562937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340775209210169:2476], now have 1 active actors on pipe
2025-10-09T21:42:58.563784Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:58.565964Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f915d9-956d33ca-a24939e-b91dcf5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:58.565977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f915d9-956d33ca-a24939e-b91dcf5_0] Write session will restart in 0.008968s
2025-10-09T21:42:58.565999Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f915d9-956d33ca-a24939e-b91dcf5_0] Write session: Do CDS request
2025-10-09T21:42:58.566006Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f915d9-956d33ca-a24939e-b91dcf5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:58.574682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:58.575432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:58.575658Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:58.575667Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:58.575679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:58.575735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:58.575797Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340774847402050:2542]
2025-10-09T21:42:58.576414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:58.576622Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f915d9-956d33ca-a24939e-b91dcf5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:58.576789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:58.576836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:58.576997Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:58.577012Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 27
2025-10-09T21:42:58.577149Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 27 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:58.577190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 27 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:58.577200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=27 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:58.577204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:58.577217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:58.577279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:58.577285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:58.577286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:58.577287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:58.577289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:58.577290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:58.577291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.577292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.577294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.577298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:58.577310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:58.577314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:58.577318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:58.577322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:58.577326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:58.577330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.577334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.577338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.577345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:58.577353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:58.577359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:58.577365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:58.577370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:58.577375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:58.577387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:58.577392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:58.577396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:58.577401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:58.577407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:58.577412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:58.577417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.577423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.577427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.577433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.577438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.577443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.577472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:58.577734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 27 sessionId:  describe result for acl check
2025-10-09T21:42:58.577770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:58.577771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:58.577773Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:58.577779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340775209210228:2645] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:58.578384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340775209210228:2645] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:58.580485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340775209210228:2645] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:58.584249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340775209210228:2645] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:58.584265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340775209210228:2645] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:58.584270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340775209210228:2645] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:58.584279Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 27 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:58.584860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:58.586959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:58.586966Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:58.586970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:58.586974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:58.586982Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 26 [2:7559340774847402050:2542]
2025-10-09T21:42:58.586997Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:58.587017Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:58.587021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.587025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.587028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.587047Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:58.587053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:58.587464Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 26
2025-10-09T21:42:58.587517Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 26
2025-10-09T21:42:58.587061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:58.587132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.587134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.587136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.587140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:58.587227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:58.587230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.587244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.587259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.587279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:58.587301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340774847402045:2542]
2025-10-09T21:42:58.587303Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340774847402045:2542]
2025-10-09T21:42:58.587306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340774847402045:2542]
2025-10-09T21:42:58.587315Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:58.587326Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:58.587330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:58.587330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:58.587333Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:58.587346Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:58.587350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:58.587363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340775209210263:2645], now have 1 active actors on pipe
2025-10-09T21:42:58.587368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340775209210182:2432], now have 1 active actors on pipe
2025-10-09T21:42:58.587406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340774847402045:2542]
2025-10-09T21:42:58.587412Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:58.587430Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:58.587766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.587775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.587804Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|71c20501-c10329e-e376a6bb-b33a98e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:58.587813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.587817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.587819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:58.587831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.587841Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:58.587857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:58.588589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.588598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.588617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:58.588825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 27 partition: 0 MaxSeqNo: 25 sessionId: src_id|71c20501-c10329e-e376a6bb-b33a98e_0
2025-10-09T21:42:58.589561Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f915d9-956d33ca-a24939e-b91dcf5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|71c20501-c10329e-e376a6bb-b33a98e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:58.592944Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|71c20501-c10329e-e376a6bb-b33a98e_0] Write session: try to update token
2025-10-09T21:42:58.592971Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|71c20501-c10329e-e376a6bb-b33a98e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:58.605287Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:58.605305Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:58.605308Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:58.605309Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:58.605320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340774847402045:2542]
2025-10-09T21:42:58.605326Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340774847402045:2542]
2025-10-09T21:42:58.605330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340774847402045:2542]
2025-10-09T21:42:58.605378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:58.605385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.605389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:58.605410Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:58.605415Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:58.605419Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:58.605548Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:58.626746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 27 sessionId: src_id|71c20501-c10329e-e376a6bb-b33a98e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:58.629671Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:58.665577Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340775209210270:2476]
2025-10-09T21:42:58.666103Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 26, pipe: [1:7559340775209210270:2476]
2025-10-09T21:42:58.819265Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:58.820168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.820181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.820205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340775209210270:2476], now have 1 active actors on pipe
2025-10-09T21:42:58.820755Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:58.820810Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340775209210270:2476]
2025-10-09T21:42:58.820829Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:58.820832Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:58.820890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 27 reason: error at write: Disconnected sessionId: src_id|71c20501-c10329e-e376a6bb-b33a98e_0
2025-10-09T21:42:58.821079Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 27 sessionId: src_id|71c20501-c10329e-e376a6bb-b33a98e_0 is DEAD
2025-10-09T21:42:58.821317Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71c20501-c10329e-e376a6bb-b33a98e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:58.821331Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71c20501-c10329e-e376a6bb-b33a98e_0] Write session will restart in 0.005448s
2025-10-09T21:42:58.821356Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71c20501-c10329e-e376a6bb-b33a98e_0] Write session: Do CDS request
2025-10-09T21:42:58.821365Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71c20501-c10329e-e376a6bb-b33a98e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:58.826477Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:58.827764Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|71c20501-c10329e-e376a6bb-b33a98e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:58.830179Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:58.830195Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 28
2025-10-09T21:42:58.830444Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 28 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:58.830476Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 28 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:58.830482Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=28 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:58.830486Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:58.831015Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 28 sessionId:  describe result for acl check
2025-10-09T21:42:58.831053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:58.831055Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:58.831057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:58.831062Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340775209210290:2651] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:58.831656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340775209210290:2651] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:58.838912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340775209210290:2651] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:58.847369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:58.848756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:58.848974Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:58.848988Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:58.849001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:58.849070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:58.849135Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340774847402137:2545]
2025-10-09T21:42:58.849751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:58.850168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:58.850218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:58.850591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:58.850643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:58.850648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:58.850650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:58.850651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:58.850652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:58.850654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:58.850655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.850657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.850658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.850664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:58.850676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:58.850680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:58.850684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:58.850689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:58.850693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:58.850697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.850702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.850706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.850714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:58.850722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:58.850732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:58.850738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:58.850743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:58.850753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:58.850758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:58.850764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:58.850773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:58.850779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:58.850786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:58.850791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:58.850796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.850802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.850808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.850818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.850823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.850829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.850864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:58.848947Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340775209210290:2651] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:58.848966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340775209210290:2651] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:58.848971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340775209210290:2651] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:58.848981Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 28 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:58.857986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:58.859202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:58.859220Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:58.859224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:58.859230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:58.859238Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 27 [2:7559340774847402137:2545]
2025-10-09T21:42:58.859251Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:58.859274Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:58.859278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.859282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.859284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.859303Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:58.859314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:58.859322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:58.859388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.859397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.859399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.859402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:58.859479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:58.859487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.859499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:58.859510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:58.859526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:58.859551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340774847402131:2545]
2025-10-09T21:42:58.859554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340774847402131:2545]
2025-10-09T21:42:58.859556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340774847402131:2545]
2025-10-09T21:42:58.859571Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:58.859587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:58.859603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:58.859607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:58.859651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340774847402131:2545]
2025-10-09T21:42:58.859658Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:58.859675Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:58.859685Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:58.859687Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:58.859694Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:58.859928Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 27
2025-10-09T21:42:58.859968Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 27
2025-10-09T21:42:58.859964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340775209210283:2432], now have 1 active actors on pipe
2025-10-09T21:42:58.859974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340775209210360:2651], now have 1 active actors on pipe
2025-10-09T21:42:58.860056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.860067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.860111Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e9dcfe07-267b6c50-eae65991-43cae7de_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:58.860119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.860124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.860127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:58.860140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.860152Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:58.860170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:58.860559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.860586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.860611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:58.860760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 28 partition: 0 MaxSeqNo: 25 sessionId: src_id|e9dcfe07-267b6c50-eae65991-43cae7de_0
2025-10-09T21:42:58.861655Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71c20501-c10329e-e376a6bb-b33a98e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e9dcfe07-267b6c50-eae65991-43cae7de_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:58.864676Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9dcfe07-267b6c50-eae65991-43cae7de_0] Write session: try to update token
2025-10-09T21:42:58.864696Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9dcfe07-267b6c50-eae65991-43cae7de_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:58.877618Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:58.877630Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:58.877633Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:58.877635Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:58.877644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340774847402131:2545]
2025-10-09T21:42:58.877656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340774847402131:2545]
2025-10-09T21:42:58.877660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340774847402131:2545]
2025-10-09T21:42:58.877708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:58.877720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:58.877723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:58.877724Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:58.877732Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:58.877737Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:58.877870Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:58.885308Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 28 sessionId: src_id|e9dcfe07-267b6c50-eae65991-43cae7de_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:58.887854Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:58.919961Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340775209210373:2476]
2025-10-09T21:42:58.920464Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 27, pipe: [1:7559340775209210373:2476]
2025-10-09T21:42:59.054571Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:59.055121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:59.055132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:59.055183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340775209210373:2476], now have 1 active actors on pipe
2025-10-09T21:42:59.055458Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:59.055945Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:59.056042Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 28 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e9dcfe07-267b6c50-eae65991-43cae7de_0
2025-10-09T21:42:59.056245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 28 sessionId: src_id|e9dcfe07-267b6c50-eae65991-43cae7de_0 is DEAD
2025-10-09T21:42:59.056430Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9dcfe07-267b6c50-eae65991-43cae7de_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:42:59.056445Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9dcfe07-267b6c50-eae65991-43cae7de_0] Write session will restart in 0.006646s
2025-10-09T21:42:59.056475Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9dcfe07-267b6c50-eae65991-43cae7de_0] Write session: Do CDS request
2025-10-09T21:42:59.056482Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9dcfe07-267b6c50-eae65991-43cae7de_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:59.056409Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340775209210373:2476]
2025-10-09T21:42:59.056446Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:59.064675Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9dcfe07-267b6c50-eae65991-43cae7de_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:59.066382Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:59.066404Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 29
2025-10-09T21:42:59.066937Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 29 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:59.066973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 29 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:59.066980Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=29 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:59.066984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:59.067497Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 29 sessionId:  describe result for acl check
2025-10-09T21:42:59.067531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:59.067533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:59.067534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:59.067541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340779504177684:2655] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:59.068158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340779504177684:2655] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:59.073929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:59.071865Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340779504177684:2655] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:59.081800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:59.082055Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:59.082070Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:59.082084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:59.082159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:59.082202Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340779142369516:2547]
2025-10-09T21:42:59.082659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340779504177684:2655] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:59.082678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340779504177684:2655] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:59.082682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340779504177684:2655] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:59.082692Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 29 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:59.082841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:59.084041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:59.084101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:59.084493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:59.084565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:59.084573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:59.084574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:59.084575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:59.084577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:59.084578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:59.084580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.084582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.084583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.084589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:59.084600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:59.084604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:59.084608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:59.084612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:59.084617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:59.084621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.084625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.084629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.084637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:59.084645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:59.084652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:59.084658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:59.084675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:59.084681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:59.084686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:59.084691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:59.084696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:59.084702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:59.084707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:59.084717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:59.084722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.084728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.084731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.084737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.084745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.084759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.084791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:59.093216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:59.094328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:59.094335Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:59.094339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:59.094345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:59.094353Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 28 [2:7559340779142369516:2547]
2025-10-09T21:42:59.094370Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:59.094389Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:59.094394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.094398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.094401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.094434Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:59.094442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:59.094450Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:59.094523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.094529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.094532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.094535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:59.094642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:59.094648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.094662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.094677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.094697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:59.094721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340779142369510:2547]
2025-10-09T21:42:59.094723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340779142369510:2547]
2025-10-09T21:42:59.094726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340779142369510:2547]
2025-10-09T21:42:59.094736Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:59.094748Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:59.094751Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:59.094753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:59.094760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340779504177671:2432], now have 1 active actors on pipe
2025-10-09T21:42:59.094764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340779504177750:2655], now have 1 active actors on pipe
2025-10-09T21:42:59.094770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:59.094787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:59.094792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:59.094831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340779142369510:2547]
2025-10-09T21:42:59.094842Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:59.094847Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:59.095648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:59.095660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:59.095706Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|faff3f79-f90f5152-2c288bfb-2f5106eb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:59.095717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.095723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.095726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:59.095738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.095752Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:59.095781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:59.096170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:59.096180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:59.096209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:59.095246Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 28
2025-10-09T21:42:59.095288Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 28
2025-10-09T21:42:59.096416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 29 partition: 0 MaxSeqNo: 25 sessionId: src_id|faff3f79-f90f5152-2c288bfb-2f5106eb_0
2025-10-09T21:42:59.100093Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9dcfe07-267b6c50-eae65991-43cae7de_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|faff3f79-f90f5152-2c288bfb-2f5106eb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:59.102950Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|faff3f79-f90f5152-2c288bfb-2f5106eb_0] Write session: try to update token
2025-10-09T21:42:59.102972Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|faff3f79-f90f5152-2c288bfb-2f5106eb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:59.114520Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:59.114533Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:59.114540Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:59.114542Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:59.114552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340779142369510:2547]
2025-10-09T21:42:59.114557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340779142369510:2547]
2025-10-09T21:42:59.114560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340779142369510:2547]
2025-10-09T21:42:59.114608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:59.114615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.114618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:59.114718Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:59.114746Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:59.114753Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:59.114758Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:59.126747Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 29 sessionId: src_id|faff3f79-f90f5152-2c288bfb-2f5106eb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:59.129343Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:59.156468Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340779504177762:2476]
2025-10-09T21:42:59.156984Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 28, pipe: [1:7559340779504177762:2476]
2025-10-09T21:42:59.317609Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:59.317952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:59.326546Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:59.326605Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:59.326703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 29 reason: status is not ok: tablet will be restarted right now sessionId: src_id|faff3f79-f90f5152-2c288bfb-2f5106eb_0
2025-10-09T21:42:59.326077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:59.326129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340779504177762:2476], now have 1 active actors on pipe
2025-10-09T21:42:59.326373Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:59.326898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 29 sessionId: src_id|faff3f79-f90f5152-2c288bfb-2f5106eb_0 is DEAD
2025-10-09T21:42:59.327011Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340779504177762:2476]
2025-10-09T21:42:59.330272Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|faff3f79-f90f5152-2c288bfb-2f5106eb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:42:59.330287Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|faff3f79-f90f5152-2c288bfb-2f5106eb_0] Write session will restart in 0.006370s
2025-10-09T21:42:59.330313Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|faff3f79-f90f5152-2c288bfb-2f5106eb_0] Write session: Do CDS request
2025-10-09T21:42:59.330319Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|faff3f79-f90f5152-2c288bfb-2f5106eb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:59.338562Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|faff3f79-f90f5152-2c288bfb-2f5106eb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:59.338648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:59.338668Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 30
2025-10-09T21:42:59.342310Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 30 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:59.342373Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 30 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:59.342385Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=30 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:59.342390Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:59.342927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 30 sessionId:  describe result for acl check
2025-10-09T21:42:59.342967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:59.342969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:59.342972Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:59.342978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340779504177781:2660] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:59.343602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340779504177781:2660] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:59.347024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340779504177781:2660] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:59.351444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:59.352753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:59.352989Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:59.352994Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:59.353007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:59.353076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:59.351575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340779504177781:2660] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:59.351595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340779504177781:2660] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:59.351600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340779504177781:2660] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:59.351632Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 30 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:59.353121Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340779142369603:2549]
2025-10-09T21:42:59.353734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:59.354143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:59.354204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:59.354562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:59.354630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:59.354632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:59.354633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:59.354634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:59.354636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:59.354637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:59.354639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.354640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.354642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.354647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:59.354660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:59.354665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:59.354669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:59.354674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:59.354678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:59.354681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.354685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.354688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.354696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:59.354704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:59.354710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:59.354716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:59.354724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:59.354730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:59.354735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:59.354740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:59.354745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:59.354750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:59.354754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:59.354759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:59.354765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.354770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.354775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.354780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.354785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.354790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.354821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:59.363107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:59.364336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:59.364342Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:59.364346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:59.364350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:59.364358Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 29 [2:7559340779142369603:2549]
2025-10-09T21:42:59.364376Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:59.364822Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 29
2025-10-09T21:42:59.364861Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 29
2025-10-09T21:42:59.364398Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:59.364401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.364405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.364408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.364429Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:59.364435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:59.364446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:59.364519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.364521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.364524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.364527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:59.364617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:59.364619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.364631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.364644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.364661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:59.364684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340779504177847:2660], now have 1 active actors on pipe
2025-10-09T21:42:59.364688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340779504177765:2432], now have 1 active actors on pipe
2025-10-09T21:42:59.364695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340779142369597:2549]
2025-10-09T21:42:59.364698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340779142369597:2549]
2025-10-09T21:42:59.364700Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340779142369597:2549]
2025-10-09T21:42:59.364717Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:59.364733Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:59.364747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:59.364751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:59.364795Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340779142369597:2549]
2025-10-09T21:42:59.364798Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:59.365042Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:59.365046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:59.365049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:59.365057Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:59.365160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:59.365194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:59.365230Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|685469b6-fa4d20b3-7242a1af-7a62d322_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:59.365241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.365247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.365250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:59.365263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.365279Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:59.365297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:59.367132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:59.367150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:59.367197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:59.367390Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 30 partition: 0 MaxSeqNo: 25 sessionId: src_id|685469b6-fa4d20b3-7242a1af-7a62d322_0
2025-10-09T21:42:59.367832Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|faff3f79-f90f5152-2c288bfb-2f5106eb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|685469b6-fa4d20b3-7242a1af-7a62d322_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:59.371194Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|685469b6-fa4d20b3-7242a1af-7a62d322_0] Write session: try to update token
2025-10-09T21:42:59.371221Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|685469b6-fa4d20b3-7242a1af-7a62d322_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:59.391467Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:59.391485Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:59.391488Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:59.391490Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:59.391500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340779142369597:2549]
2025-10-09T21:42:59.391505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340779142369597:2549]
2025-10-09T21:42:59.391508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340779142369597:2549]
2025-10-09T21:42:59.391555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:59.391563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.391565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:59.391564Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:59.391572Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:59.391577Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:59.391744Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:59.404182Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 30 sessionId: src_id|685469b6-fa4d20b3-7242a1af-7a62d322_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:59.407274Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:59.426860Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340779504177873:2476]
2025-10-09T21:42:59.427363Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 29, pipe: [1:7559340779504177873:2476]
2025-10-09T21:42:59.561006Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:59.561021Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:59.561078Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340779504177873:2476]
2025-10-09T21:42:59.561100Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:59.561144Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 30 reason: error at write: Disconnected sessionId: src_id|685469b6-fa4d20b3-7242a1af-7a62d322_0
2025-10-09T21:42:59.561365Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 30 sessionId: src_id|685469b6-fa4d20b3-7242a1af-7a62d322_0 is DEAD
2025-10-09T21:42:59.561563Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|685469b6-fa4d20b3-7242a1af-7a62d322_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:59.561574Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|685469b6-fa4d20b3-7242a1af-7a62d322_0] Write session will restart in 0.006333s
2025-10-09T21:42:59.561600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|685469b6-fa4d20b3-7242a1af-7a62d322_0] Write session: Do CDS request
2025-10-09T21:42:59.561606Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|685469b6-fa4d20b3-7242a1af-7a62d322_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:59.560354Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:59.560719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:59.560732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:59.560758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340779504177873:2476], now have 1 active actors on pipe
2025-10-09T21:42:59.561471Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:59.573044Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|685469b6-fa4d20b3-7242a1af-7a62d322_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:59.573711Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:59.573730Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 31
2025-10-09T21:42:59.573865Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 31 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:59.573899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 31 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:59.573911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=31 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:59.573917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:59.574764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 31 sessionId:  describe result for acl check
2025-10-09T21:42:59.574826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:59.574832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:59.574835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:59.574841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340779504177910:2671] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:59.575519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340779504177910:2671] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:59.579027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:59.579537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:59.579730Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:59.579740Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:59.579752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:59.579806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:59.579856Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340779142369727:2564]
2025-10-09T21:42:59.580518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:59.580908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:59.580983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:59.578796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340779504177910:2671] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:59.581358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:59.581443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:59.581444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:59.581445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:59.581447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:59.581448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:59.581449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:59.581450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.581451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.581453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.581459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:59.581469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:59.581473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:59.581477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:59.581481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:59.581484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:59.581487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.581491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.581495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.581502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:59.581510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:59.581516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:59.581521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:59.581525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:59.581531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:59.581535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:59.581540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:59.581544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:59.581549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:59.581553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:59.581558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:59.581563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.581568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.581572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.581576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.581581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.581585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.581638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:59.587469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340779504177910:2671] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:59.587489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340779504177910:2671] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:59.587494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340779504177910:2671] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:59.587505Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 31 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:59.588881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:59.590282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:59.590296Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:59.590300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:59.590309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:59.590328Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 30 [2:7559340779142369727:2564]
2025-10-09T21:42:59.590348Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:59.590383Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:59.590392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.590396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.590398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.590428Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:59.590435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:59.590442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:59.590876Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 30
2025-10-09T21:42:59.590511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.590881Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 30
2025-10-09T21:42:59.590513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.590515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.590518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:59.590596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:59.590599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.590611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.590622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.590640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:59.590662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340779142369721:2564]
2025-10-09T21:42:59.590665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340779142369721:2564]
2025-10-09T21:42:59.590668Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340779142369721:2564]
2025-10-09T21:42:59.590686Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:59.590704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:59.590718Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:59.590719Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:59.590721Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:59.590723Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:59.590725Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:59.590766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340779142369721:2564]
2025-10-09T21:42:59.590769Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:59.590778Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:59.590795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340779504177970:2671], now have 1 active actors on pipe
2025-10-09T21:42:59.590799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340779504177884:2432], now have 1 active actors on pipe
2025-10-09T21:42:59.591220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:59.591232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:59.591264Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4ada80fc-40dbcc5-ecefc817-489f0cb8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:59.591273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.591284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.591286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:59.591299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.591313Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:59.591330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:59.591545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:59.591555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:59.591575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:59.591681Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 31 partition: 0 MaxSeqNo: 25 sessionId: src_id|4ada80fc-40dbcc5-ecefc817-489f0cb8_0
2025-10-09T21:42:59.592428Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|685469b6-fa4d20b3-7242a1af-7a62d322_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4ada80fc-40dbcc5-ecefc817-489f0cb8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:59.595567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ada80fc-40dbcc5-ecefc817-489f0cb8_0] Write session: try to update token
2025-10-09T21:42:59.595590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ada80fc-40dbcc5-ecefc817-489f0cb8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:59.606866Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:42:59.606884Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:59.606894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340779142369721:2564]
2025-10-09T21:42:59.606948Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:59.606951Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:42:59.607178Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:42:59.607321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:59.607485Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:59.607643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:42:59.607906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:42:59.607947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:42:59.619881Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 31 sessionId: src_id|4ada80fc-40dbcc5-ecefc817-489f0cb8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:59.622197Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:59.622214Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:59.622220Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:59.622223Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:59.622233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340779142369721:2564]
2025-10-09T21:42:59.622237Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340779142369721:2564]
2025-10-09T21:42:59.622241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340779142369721:2564]
2025-10-09T21:42:59.622299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:59.622314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.622317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:59.622514Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:59.622874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:59.626205Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:59.626222Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:59.626227Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:59.661336Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340779504177977:2476]
2025-10-09T21:42:59.661670Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 30, pipe: [1:7559340779504177977:2476]
2025-10-09T21:42:59.768380Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard_impl.cpp:7721: Cannot get console configs
2025-10-09T21:42:59.768404Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:42:59.799557Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:42:59.799872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:59.799891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:59.799917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340779504177977:2476], now have 1 active actors on pipe
2025-10-09T21:42:59.800902Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ada80fc-40dbcc5-ecefc817-489f0cb8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:42:59.800917Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ada80fc-40dbcc5-ecefc817-489f0cb8_0] Write session will restart in 0.006431s
2025-10-09T21:42:59.800953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ada80fc-40dbcc5-ecefc817-489f0cb8_0] Write session: Do CDS request
2025-10-09T21:42:59.800961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ada80fc-40dbcc5-ecefc817-489f0cb8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:42:59.800217Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:42:59.800263Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:42:59.800291Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:42:59.800383Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 31 reason: error at write: Disconnected sessionId: src_id|4ada80fc-40dbcc5-ecefc817-489f0cb8_0
2025-10-09T21:42:59.800599Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 31 sessionId: src_id|4ada80fc-40dbcc5-ecefc817-489f0cb8_0 is DEAD
2025-10-09T21:42:59.800622Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:42:59.800677Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340779504177977:2476]
2025-10-09T21:42:59.810548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ada80fc-40dbcc5-ecefc817-489f0cb8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:59.810978Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:59.810993Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 32
2025-10-09T21:42:59.811296Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 32 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:59.811349Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 32 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:42:59.811355Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=32 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:59.811360Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:59.811817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 32 sessionId:  describe result for acl check
2025-10-09T21:42:59.811862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:59.811865Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:59.811867Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:59.811873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340779504178007:2677] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:59.812487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340779504178007:2677] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:59.817191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:59.819234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:59.819505Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:59.819526Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:42:59.819540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:59.819611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:59.819656Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340779142369816:2566]
2025-10-09T21:42:59.818736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340779504178007:2677] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:59.820299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:42:59.821137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:42:59.821189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:42:59.821572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:42:59.821637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:59.821644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:59.821646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:59.821647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:59.821649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:59.821651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:59.821652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.821654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.821655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.821661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:59.821671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:59.821675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:59.821680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:59.821684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:59.821688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:59.821692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.821695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.821700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.821707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:59.821716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:42:59.821724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:59.821730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:42:59.821735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:59.821740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:42:59.821744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:59.821748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:42:59.821753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:59.821757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:42:59.821762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:59.821767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:42:59.821771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.821775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.821780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.821786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.821791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.821797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.821827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:42:59.823230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340779504178007:2677] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:59.823246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340779504178007:2677] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:42:59.823250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340779504178007:2677] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:59.823261Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 32 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:59.831728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:42:59.832890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:42:59.832906Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:42:59.832910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:59.832915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:59.833347Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 31
2025-10-09T21:42:59.833398Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 31
2025-10-09T21:42:59.832923Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 31 [2:7559340779142369816:2566]
2025-10-09T21:42:59.832940Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:42:59.832960Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:42:59.832964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.832967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.832970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.832989Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:42:59.832996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:59.833005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:59.833072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.833074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.833076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.833080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:42:59.833173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:42:59.833177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.833191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:42:59.833202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:42:59.833219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:42:59.833240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340779504178081:2677], now have 1 active actors on pipe
2025-10-09T21:42:59.833250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340779142369811:2566]
2025-10-09T21:42:59.833252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340779142369811:2566]
2025-10-09T21:42:59.833255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340779142369811:2566]
2025-10-09T21:42:59.833268Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:42:59.833284Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:42:59.833285Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:42:59.833289Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:42:59.833292Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:42:59.833299Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:42:59.833303Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:59.833310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340779504177989:2432], now have 1 active actors on pipe
2025-10-09T21:42:59.833350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340779142369811:2566]
2025-10-09T21:42:59.833357Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:42:59.833372Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:42:59.833786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:59.833796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:59.833825Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2790f325-6284279a-e53bf00b-313753fa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:59.833833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.833837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.833840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:59.833852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.833863Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:59.833880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:59.834158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:59.834166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:59.834183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:59.834312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 32 partition: 0 MaxSeqNo: 25 sessionId: src_id|2790f325-6284279a-e53bf00b-313753fa_0
2025-10-09T21:42:59.837569Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ada80fc-40dbcc5-ecefc817-489f0cb8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2790f325-6284279a-e53bf00b-313753fa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:59.842268Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2790f325-6284279a-e53bf00b-313753fa_0] Write session: try to update token
2025-10-09T21:42:59.842292Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2790f325-6284279a-e53bf00b-313753fa_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:42:59.848355Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:42:59.848374Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:42:59.848377Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:42:59.848379Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:42:59.848389Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340779142369811:2566]
2025-10-09T21:42:59.848394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340779142369811:2566]
2025-10-09T21:42:59.848398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340779142369811:2566]
2025-10-09T21:42:59.848444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:42:59.848451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:42:59.848454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:42:59.848458Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:42:59.848466Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:42:59.848470Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:42:59.848614Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:42:59.868454Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 32 sessionId: src_id|2790f325-6284279a-e53bf00b-313753fa_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:59.871171Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:59.900416Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340779504178089:2476]
2025-10-09T21:42:59.900788Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 31, pipe: [1:7559340779504178089:2476]
2025-10-09T21:43:00.009810Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:00.010172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:00.010189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:00.010215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340779504178089:2476], now have 1 active actors on pipe
2025-10-09T21:43:00.011450Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:00.015135Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:00.015227Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340779504178089:2476]
2025-10-09T21:43:00.015263Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:00.015362Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 32 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2790f325-6284279a-e53bf00b-313753fa_0
2025-10-09T21:43:00.015574Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 32 sessionId: src_id|2790f325-6284279a-e53bf00b-313753fa_0 is DEAD
2025-10-09T21:43:00.021908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2790f325-6284279a-e53bf00b-313753fa_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:00.021927Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2790f325-6284279a-e53bf00b-313753fa_0] Write session will restart in 0.005494s
2025-10-09T21:43:00.021961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2790f325-6284279a-e53bf00b-313753fa_0] Write session: Do CDS request
2025-10-09T21:43:00.021968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2790f325-6284279a-e53bf00b-313753fa_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:00.028757Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2790f325-6284279a-e53bf00b-313753fa_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:00.030396Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:00.030429Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 33
2025-10-09T21:43:00.030663Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 33 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:00.030698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 33 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:00.030704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=33 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:00.030709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:00.031135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 33 sessionId:  describe result for acl check
2025-10-09T21:43:00.031166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:00.031168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:00.031170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:00.031177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340783799145399:2681] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:00.031810Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340783799145399:2681] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:00.036847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340783799145399:2681] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:00.048392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340783799145399:2681] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:00.048424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340783799145399:2681] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:00.048429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340783799145399:2681] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:00.048440Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 33 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:00.053353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:00.053878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:00.054085Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:00.054095Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:00.054108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:00.054165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:00.054210Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340783437337201:2569]
2025-10-09T21:43:00.054845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:00.055631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:00.055679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:00.056027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:00.056087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:00.056088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:00.056090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:00.056091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:00.056092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:00.056093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:00.056095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.056096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.056098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.056104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:00.056115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:00.056118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:00.056122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:00.056125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:00.056128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:00.056132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.056135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.056138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.056146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:00.056154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:00.056161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:00.056165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:00.056169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:00.056174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:00.056177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:00.056181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:00.056186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:00.056190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:00.056195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:00.056204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:00.056209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.056214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.056220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.056224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.056230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.056234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.056263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:00.064920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:00.066101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:00.066108Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:00.066111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:00.066116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:00.066125Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 32 [2:7559340783437337201:2569]
2025-10-09T21:43:00.066145Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:00.066167Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:00.066171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.066175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.066178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.066197Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:00.066204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:00.066212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:00.066287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.066293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.066296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.066299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:00.066384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:00.066388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.066401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.066412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.066444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:00.066466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340783437337195:2569]
2025-10-09T21:43:00.066468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340783437337195:2569]
2025-10-09T21:43:00.066470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340783437337195:2569]
2025-10-09T21:43:00.066486Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:00.066506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:00.066525Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:00.066529Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:00.066574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340783437337195:2569]
2025-10-09T21:43:00.066577Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:00.066982Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:00.066989Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:00.066992Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:00.066999Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:00.067454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340783799145390:2432], now have 1 active actors on pipe
2025-10-09T21:43:00.067463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340783799145466:2681], now have 1 active actors on pipe
2025-10-09T21:43:00.067140Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 32
2025-10-09T21:43:00.067744Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 32
2025-10-09T21:43:00.070904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:00.070922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:00.070964Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|af70267b-dafe68d2-24967b1d-d99df112_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:00.070977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.070985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.070989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:00.071006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.071026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:00.071047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:00.071392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:00.071408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:00.071446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:00.071748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 33 partition: 0 MaxSeqNo: 25 sessionId: src_id|af70267b-dafe68d2-24967b1d-d99df112_0
2025-10-09T21:43:00.072309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2790f325-6284279a-e53bf00b-313753fa_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|af70267b-dafe68d2-24967b1d-d99df112_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:00.075574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af70267b-dafe68d2-24967b1d-d99df112_0] Write session: try to update token
2025-10-09T21:43:00.075600Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af70267b-dafe68d2-24967b1d-d99df112_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:00.089189Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:00.089206Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:00.089209Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:00.089212Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:00.089222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340783437337195:2569]
2025-10-09T21:43:00.089227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340783437337195:2569]
2025-10-09T21:43:00.089231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340783437337195:2569]
2025-10-09T21:43:00.089273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:00.089285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.089291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:00.089447Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:00.091746Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:00.091756Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:00.091760Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:00.109176Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 33 sessionId: src_id|af70267b-dafe68d2-24967b1d-d99df112_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:00.111745Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:00.116641Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340783799145482:2476]
2025-10-09T21:43:00.117007Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 32, pipe: [1:7559340783799145482:2476]
2025-10-09T21:43:00.296141Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:00.297164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:00.297177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:00.297201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340783799145482:2476], now have 1 active actors on pipe
2025-10-09T21:43:00.298752Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:00.298370Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:00.298388Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:00.298392Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:00.298468Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 33 reason: error at write: Disconnected sessionId: src_id|af70267b-dafe68d2-24967b1d-d99df112_0
2025-10-09T21:43:00.298476Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340783799145482:2476]
2025-10-09T21:43:00.298657Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 33 sessionId: src_id|af70267b-dafe68d2-24967b1d-d99df112_0 is DEAD
2025-10-09T21:43:00.299008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af70267b-dafe68d2-24967b1d-d99df112_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:00.299019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af70267b-dafe68d2-24967b1d-d99df112_0] Write session will restart in 0.009933s
2025-10-09T21:43:00.299054Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af70267b-dafe68d2-24967b1d-d99df112_0] Write session: Do CDS request
2025-10-09T21:43:00.299060Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af70267b-dafe68d2-24967b1d-d99df112_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:00.311114Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af70267b-dafe68d2-24967b1d-d99df112_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:00.312068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:00.312811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:00.313045Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:00.313054Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:00.313066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:00.313135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:00.313174Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340783437337289:2571]
2025-10-09T21:43:00.313811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:00.314262Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:00.314278Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 34
2025-10-09T21:43:00.315011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:00.315079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:00.315472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:00.315536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:00.315537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:00.315538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:00.315540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:00.315541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:00.315543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:00.315544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.315545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.315547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.315553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:00.315563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:00.315566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:00.315569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:00.315573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:00.315576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:00.315580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.315583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.315587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.315593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:00.315602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:00.315608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:00.315614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:00.315619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:00.315624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:00.315629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:00.315634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:00.315638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:00.315644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:00.315648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:00.315654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:00.315659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.315669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.315673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.315678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.315683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.315687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.315719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:00.314540Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 34 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:00.314584Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 34 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:00.314590Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=34 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:00.314594Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:00.315105Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 34 sessionId:  describe result for acl check
2025-10-09T21:43:00.315140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:00.315142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:00.315144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:00.315149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340783799145538:2685] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:00.315703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340783799145538:2685] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:00.319932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340783799145538:2685] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:00.325518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:00.326760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:00.326768Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:00.326770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:00.326775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:00.326785Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 33 [2:7559340783437337289:2571]
2025-10-09T21:43:00.326802Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:00.326824Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:00.326828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.326831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.326834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.326853Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:00.326860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:00.326868Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:00.326941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.326943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.326946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.326954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:00.327047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:00.327050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.327062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.327072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.327088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:00.327113Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340783437337283:2571]
2025-10-09T21:43:00.327115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340783437337283:2571]
2025-10-09T21:43:00.327117Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340783437337283:2571]
2025-10-09T21:43:00.327135Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:00.327152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:00.327167Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:00.327170Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:00.327216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340783437337283:2571]
2025-10-09T21:43:00.327218Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:00.327623Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:00.327629Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:00.327631Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:00.327637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:00.327947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340783799145486:2432], now have 1 active actors on pipe
2025-10-09T21:43:00.328623Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 33
2025-10-09T21:43:00.328945Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340783799145538:2685] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:00.328968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340783799145538:2685] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:00.328972Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340783799145538:2685] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:00.328981Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 34 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:00.329779Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 33
2025-10-09T21:43:00.330556Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 34 partition: 0 MaxSeqNo: 25 sessionId: src_id|3019646e-7b8378d9-ca860abf-835d068d_0
2025-10-09T21:43:00.331017Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af70267b-dafe68d2-24967b1d-d99df112_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3019646e-7b8378d9-ca860abf-835d068d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:00.329535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340783799145577:2685], now have 1 active actors on pipe
2025-10-09T21:43:00.329866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:00.329874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:00.329914Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3019646e-7b8378d9-ca860abf-835d068d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:00.329928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.329934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.329937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:00.329950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.329969Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:00.329989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:00.330313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:00.330319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:00.330338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:00.334249Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3019646e-7b8378d9-ca860abf-835d068d_0] Write session: try to update token
2025-10-09T21:43:00.334267Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3019646e-7b8378d9-ca860abf-835d068d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:00.347333Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:00.347352Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:00.347355Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:00.347357Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:00.347370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340783437337283:2571]
2025-10-09T21:43:00.347374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340783437337283:2571]
2025-10-09T21:43:00.347378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340783437337283:2571]
2025-10-09T21:43:00.347425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:00.347432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.347436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:00.347570Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:00.350090Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:00.350106Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:00.350110Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:00.365119Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 34 sessionId: src_id|3019646e-7b8378d9-ca860abf-835d068d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:00.367598Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:00.399286Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340783799145583:2476]
2025-10-09T21:43:00.399638Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 33, pipe: [1:7559340783799145583:2476]
2025-10-09T21:43:00.527336Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:00.527688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:00.527699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:00.527725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340783799145583:2476], now have 1 active actors on pipe
2025-10-09T21:43:00.528162Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:00.532179Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3019646e-7b8378d9-ca860abf-835d068d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:00.532193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3019646e-7b8378d9-ca860abf-835d068d_0] Write session will restart in 0.006872s
2025-10-09T21:43:00.532218Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3019646e-7b8378d9-ca860abf-835d068d_0] Write session: Do CDS request
2025-10-09T21:43:00.532225Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3019646e-7b8378d9-ca860abf-835d068d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:00.530978Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340783799145583:2476]
2025-10-09T21:43:00.531017Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:00.531116Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 34 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3019646e-7b8378d9-ca860abf-835d068d_0
2025-10-09T21:43:00.531330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 34 sessionId: src_id|3019646e-7b8378d9-ca860abf-835d068d_0 is DEAD
2025-10-09T21:43:00.531429Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:00.546009Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3019646e-7b8378d9-ca860abf-835d068d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:00.546172Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:00.546193Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 35
2025-10-09T21:43:00.546508Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 35 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:00.546561Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 35 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:00.546567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=35 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:00.546581Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:00.547379Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 35 sessionId:  describe result for acl check
2025-10-09T21:43:00.547425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:00.547428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:00.547432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:00.547440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340783799145632:2698] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:00.548091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340783799145632:2698] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:00.556066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340783799145632:2698] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:00.563564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340783799145632:2698] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:00.563582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340783799145632:2698] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:00.563587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340783799145632:2698] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:00.563615Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 35 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:00.563867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:00.567732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:00.567995Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:00.568000Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:00.568013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:00.568082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:00.568119Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340783437337422:2586]
2025-10-09T21:43:00.568752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:00.569575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:00.569633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:00.569994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:00.570081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:00.570085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:00.570087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:00.570088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:00.570089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:00.570091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:00.570092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.570093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.570095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.570101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:00.570112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:00.570120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:00.570125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:00.570129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:00.570133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:00.570137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.570142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.570146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.570154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:00.570162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:00.570178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:00.570188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:00.570194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:00.570200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:00.570206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:00.570213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:00.570219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:00.570229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:00.570234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:00.570240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:00.570245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.570251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.570255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.570261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.570267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.570276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.570309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:00.582275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:00.583589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:00.583607Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:00.583610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:00.583615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:00.583622Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 34 [2:7559340783437337422:2586]
2025-10-09T21:43:00.583643Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:00.583662Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:00.583666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.583671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.583673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.583689Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:00.583700Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:00.583709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:00.583786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.583796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.583798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.583802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:00.583884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:00.583887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.583900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.583910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.583925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:00.583958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340783437337415:2586]
2025-10-09T21:43:00.583960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340783437337415:2586]
2025-10-09T21:43:00.583963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340783437337415:2586]
2025-10-09T21:43:00.583977Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:00.583993Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:00.584007Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:00.584010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:00.584050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340783437337415:2586]
2025-10-09T21:43:00.584053Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:00.584230Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:00.584233Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:00.584235Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:00.584242Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:00.584984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340783799145617:2432], now have 1 active actors on pipe
2025-10-09T21:43:00.584995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340783799145698:2698], now have 1 active actors on pipe
2025-10-09T21:43:00.585009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:00.585015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:00.585044Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|54e83dd9-c4e5120f-875ffb69-86a1ace2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:00.585054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.585058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.585061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:00.585072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.585084Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:00.585102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:00.584812Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 34
2025-10-09T21:43:00.584852Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 34
2025-10-09T21:43:00.585856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:00.585869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:00.585889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:00.586526Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3019646e-7b8378d9-ca860abf-835d068d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|54e83dd9-c4e5120f-875ffb69-86a1ace2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:00.586080Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 35 partition: 0 MaxSeqNo: 25 sessionId: src_id|54e83dd9-c4e5120f-875ffb69-86a1ace2_0
2025-10-09T21:43:00.589400Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54e83dd9-c4e5120f-875ffb69-86a1ace2_0] Write session: try to update token
2025-10-09T21:43:00.589422Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54e83dd9-c4e5120f-875ffb69-86a1ace2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:00.612076Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 35 sessionId: src_id|54e83dd9-c4e5120f-875ffb69-86a1ace2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:00.615045Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:00.615065Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:00.615068Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:00.615070Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:00.615083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340783437337415:2586]
2025-10-09T21:43:00.615087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340783437337415:2586]
2025-10-09T21:43:00.615091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340783437337415:2586]
2025-10-09T21:43:00.615140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:00.615147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.615150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:00.615151Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:00.615158Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:00.615162Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:00.615132Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:00.615683Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:00.629478Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340783799145712:2476]
2025-10-09T21:43:00.629968Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 34, pipe: [1:7559340783799145712:2476]
2025-10-09T21:43:00.803129Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:00.803470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:00.803487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:00.803535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340783799145712:2476], now have 1 active actors on pipe
2025-10-09T21:43:00.803897Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340783799145712:2476]
2025-10-09T21:43:00.803899Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:00.803937Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:00.803939Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:00.804031Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 35 reason: error at write: Disconnected sessionId: src_id|54e83dd9-c4e5120f-875ffb69-86a1ace2_0
2025-10-09T21:43:00.804204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 35 sessionId: src_id|54e83dd9-c4e5120f-875ffb69-86a1ace2_0 is DEAD
2025-10-09T21:43:00.803883Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:00.804352Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54e83dd9-c4e5120f-875ffb69-86a1ace2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:00.804363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54e83dd9-c4e5120f-875ffb69-86a1ace2_0] Write session will restart in 0.008650s
2025-10-09T21:43:00.804388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54e83dd9-c4e5120f-875ffb69-86a1ace2_0] Write session: Do CDS request
2025-10-09T21:43:00.804393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54e83dd9-c4e5120f-875ffb69-86a1ace2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:00.814572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54e83dd9-c4e5120f-875ffb69-86a1ace2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:00.815894Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:00.815918Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 36
2025-10-09T21:43:00.816283Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 36 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:00.816332Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 36 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:00.816339Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=36 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:00.816344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:00.817258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 36 sessionId:  describe result for acl check
2025-10-09T21:43:00.817314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:00.817324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:00.817326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:00.817333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340783799145777:2705] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:00.817926Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340783799145777:2705] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:00.821167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:00.821859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:00.822124Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:00.822136Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:00.822147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:00.822212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:00.822259Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340783437337514:2589]
2025-10-09T21:43:00.822311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340783799145777:2705] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:00.822973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:00.823349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:00.823428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:00.823779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:00.823839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:00.823840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:00.823841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:00.823843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:00.823844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:00.823845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:00.823847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.823848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.823849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.823855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:00.823865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:00.823870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:00.823874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:00.823877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:00.823881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:00.823885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.823888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.823892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.823899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:00.823908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:00.823914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:00.823919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:00.823924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:00.823930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:00.823935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:00.823940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:00.823944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:00.823949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:00.823955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:00.823961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:00.823965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.823972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.823977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.823981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.823986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.823991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.824022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:00.831189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340783799145777:2705] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:00.831209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340783799145777:2705] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:00.831213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340783799145777:2705] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:00.831222Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 36 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:00.831802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:00.833004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:00.833020Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:00.833023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:00.833027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:00.833035Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 35 [2:7559340783437337514:2589]
2025-10-09T21:43:00.833049Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:00.833067Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:00.833070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.833074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.833076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.833092Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:00.833101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:00.833107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:00.833166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.833173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.833176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.833178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:00.833378Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 35
2025-10-09T21:43:00.833246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:00.833414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 35
2025-10-09T21:43:00.833258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.833272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:00.833282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:00.833299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:00.833317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340783799145726:2432], now have 1 active actors on pipe
2025-10-09T21:43:00.833321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340783799145820:2705], now have 1 active actors on pipe
2025-10-09T21:43:00.833329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340783437337509:2589]
2025-10-09T21:43:00.833332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340783437337509:2589]
2025-10-09T21:43:00.833334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340783437337509:2589]
2025-10-09T21:43:00.833347Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:00.833360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:00.833374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:00.833376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:00.833409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340783437337509:2589]
2025-10-09T21:43:00.833412Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:00.833421Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:00.833424Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:00.833426Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:00.833432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:00.833459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:00.833463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:00.833485Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|399efa95-f138843c-44e122ea-83539137_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:00.833492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.833499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.833502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:00.833510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.833518Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:00.833531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:00.833691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:00.833699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:00.833712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:00.833795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 36 partition: 0 MaxSeqNo: 25 sessionId: src_id|399efa95-f138843c-44e122ea-83539137_0
2025-10-09T21:43:00.838329Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54e83dd9-c4e5120f-875ffb69-86a1ace2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|399efa95-f138843c-44e122ea-83539137_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:00.841323Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|399efa95-f138843c-44e122ea-83539137_0] Write session: try to update token
2025-10-09T21:43:00.841351Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|399efa95-f138843c-44e122ea-83539137_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:00.849176Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:00.849198Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:00.849209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340783437337509:2589]
2025-10-09T21:43:00.849263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:00.849264Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:00.849431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:00.849576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:00.849740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:00.849889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:00.850119Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:00.850172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:00.865584Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:00.865606Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:00.865609Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:00.865613Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:00.865624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340783437337509:2589]
2025-10-09T21:43:00.865629Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340783437337509:2589]
2025-10-09T21:43:00.865634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340783437337509:2589]
2025-10-09T21:43:00.865682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:00.865691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:00.865698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:00.865824Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:00.868103Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:00.868120Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:00.868125Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:00.877460Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 36 sessionId: src_id|399efa95-f138843c-44e122ea-83539137_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:00.880693Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:00.903976Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340783799145828:2476]
2025-10-09T21:43:00.904427Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 35, pipe: [1:7559340783799145828:2476]
2025-10-09T21:43:01.047480Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:01.047820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:01.047841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:01.047865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340783799145828:2476], now have 1 active actors on pipe
2025-10-09T21:43:01.048161Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:01.048163Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:01.048179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:01.048205Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:01.048248Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340783799145828:2476]
2025-10-09T21:43:01.048264Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 36 reason: error at write: Disconnected sessionId: src_id|399efa95-f138843c-44e122ea-83539137_0
2025-10-09T21:43:01.048463Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 36 sessionId: src_id|399efa95-f138843c-44e122ea-83539137_0 is DEAD
2025-10-09T21:43:01.048922Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|399efa95-f138843c-44e122ea-83539137_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:01.048934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|399efa95-f138843c-44e122ea-83539137_0] Write session will restart in 0.007158s
2025-10-09T21:43:01.048959Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|399efa95-f138843c-44e122ea-83539137_0] Write session: Do CDS request
2025-10-09T21:43:01.048966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|399efa95-f138843c-44e122ea-83539137_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:01.062559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|399efa95-f138843c-44e122ea-83539137_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:01.065572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:01.066300Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:01.066316Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 37
2025-10-09T21:43:01.066494Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 37 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:01.066527Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 37 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:01.066538Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=37 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:01.066543Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:01.066326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:01.066644Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:01.066659Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:01.066692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:01.066853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:01.066912Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340787732304902:2591]
2025-10-09T21:43:01.067573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:01.066939Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 37 sessionId:  describe result for acl check
2025-10-09T21:43:01.066975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:01.066976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:01.066979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:01.066985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340788094113182:2709] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:01.067594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340788094113182:2709] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:01.068125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:01.068198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:01.068592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:01.068652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:01.068659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:01.068660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:01.068662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:01.068663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:01.068664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:01.068665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.068667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.068668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.068675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:01.068685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:01.068688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:01.068692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:01.068697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:01.068701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:01.068705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.068709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.068713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.068722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:01.068729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:01.068735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:01.068740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:01.068744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:01.068749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:01.068754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:01.068759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:01.068767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:01.068772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:01.068777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:01.068791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:01.068797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.068803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.068807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.068812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.068818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.068823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.068852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:01.070017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340788094113182:2709] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:01.075977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340788094113182:2709] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:01.075993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340788094113182:2709] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:01.075997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340788094113182:2709] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:01.076007Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 37 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:01.076601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:01.077732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:01.077756Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:01.077761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:01.077766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:01.077775Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 36 [2:7559340787732304902:2591]
2025-10-09T21:43:01.077793Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:01.077817Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:01.077820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.077828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.077831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.077849Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:01.077855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:01.077863Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:01.077934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.077944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.077947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.077951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:01.078058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:01.078068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.078085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.078096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.078115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:01.078143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340787732304896:2591]
2025-10-09T21:43:01.078145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340787732304896:2591]
2025-10-09T21:43:01.078147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340787732304896:2591]
2025-10-09T21:43:01.078156Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:01.078166Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:01.078170Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:01.078171Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:01.078173Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:01.078186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:01.078189Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:01.078203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340788094113216:2709], now have 1 active actors on pipe
2025-10-09T21:43:01.078207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340788094113129:2432], now have 1 active actors on pipe
2025-10-09T21:43:01.078249Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340787732304896:2591]
2025-10-09T21:43:01.078263Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:01.078303Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:01.078481Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 36
2025-10-09T21:43:01.078513Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 36
2025-10-09T21:43:01.078622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:01.078637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:01.078676Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8b324de1-4ca6812e-db28055b-5a72cd31_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:01.078691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.078707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.078710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:01.078723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.078741Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:01.078765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:01.079092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:01.079104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:01.079126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:01.079248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 37 partition: 0 MaxSeqNo: 25 sessionId: src_id|8b324de1-4ca6812e-db28055b-5a72cd31_0
2025-10-09T21:43:01.080188Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|399efa95-f138843c-44e122ea-83539137_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8b324de1-4ca6812e-db28055b-5a72cd31_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:01.083626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b324de1-4ca6812e-db28055b-5a72cd31_0] Write session: try to update token
2025-10-09T21:43:01.083665Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b324de1-4ca6812e-db28055b-5a72cd31_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:01.105080Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:01.105103Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:01.105115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340787732304896:2591]
2025-10-09T21:43:01.105172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:01.105179Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:01.105366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:01.105532Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:01.105680Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:01.105845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:01.107442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:01.107521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:01.108937Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 37 sessionId: src_id|8b324de1-4ca6812e-db28055b-5a72cd31_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:01.111521Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:01.112646Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:01.112405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:01.112471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:01.112530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:01.134653Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:01.134669Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:01.134673Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:01.134675Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:01.134691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340787732304896:2591]
2025-10-09T21:43:01.134695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340787732304896:2591]
2025-10-09T21:43:01.134699Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340787732304896:2591]
2025-10-09T21:43:01.134748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:01.134750Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:01.134756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.134757Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:01.134759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:01.134761Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:01.134899Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:01.154606Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340788094113226:2476]
2025-10-09T21:43:01.155255Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 36, pipe: [1:7559340788094113226:2476]
2025-10-09T21:43:01.304574Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:01.313775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:01.313810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:01.314007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:01.314157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:01.314233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:01.314302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:01.314353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:01.318065Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340788094113226:2476]
2025-10-09T21:43:01.318105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:01.318108Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:01.318141Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 37 reason: error at write: Disconnected sessionId: src_id|8b324de1-4ca6812e-db28055b-5a72cd31_0
2025-10-09T21:43:01.314359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:01.314453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:01.314512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:01.314579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:01.314645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:01.314697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:01.314701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:01.314776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:01.314839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:01.314905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:01.314972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:01.315016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:01.315020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:01.315092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:01.315163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:01.315234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:01.315307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:01.315355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:01.315363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:01.315444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:01.315516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:01.315581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:01.315646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:01.315688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:01.315691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:01.315760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:01.315821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:01.315883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:01.315955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:01.315999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:01.316003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:01.316075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:01.316133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:01.316196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:01.316265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:01.316312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:01.316316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:01.316391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:01.316450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:01.316513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:01.316574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:01.316619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:01.316623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:01.316693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:01.316759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:01.316835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:01.316900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:01.316967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:01.316970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:01.317042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:01.317104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:01.317167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:01.317232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:01.317284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:01.317287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:01.317377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340788094113226:2476], now have 1 active actors on pipe
2025-10-09T21:43:01.318528Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b324de1-4ca6812e-db28055b-5a72cd31_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:01.318544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b324de1-4ca6812e-db28055b-5a72cd31_0] Write session will restart in 0.005235s
2025-10-09T21:43:01.318568Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b324de1-4ca6812e-db28055b-5a72cd31_0] Write session: Do CDS request
2025-10-09T21:43:01.318576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b324de1-4ca6812e-db28055b-5a72cd31_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:01.318725Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:01.318333Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 37 sessionId: src_id|8b324de1-4ca6812e-db28055b-5a72cd31_0 is DEAD
2025-10-09T21:43:01.318520Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:01.324628Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b324de1-4ca6812e-db28055b-5a72cd31_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:01.324771Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:01.324785Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 38
2025-10-09T21:43:01.324923Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 38 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:01.324957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 38 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:01.324968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=38 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:01.324981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:01.325582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 38 sessionId:  describe result for acl check
2025-10-09T21:43:01.325624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:01.325632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:01.325634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:01.325640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340788094113244:2713] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:01.326272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340788094113244:2713] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:01.328934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340788094113244:2713] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:01.332951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:01.334645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:01.334897Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:01.334908Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:01.334922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:01.335001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:01.335054Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340787732304996:2593]
2025-10-09T21:43:01.335729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:01.338459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340788094113244:2713] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:01.338475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340788094113244:2713] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:01.338479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340788094113244:2713] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:01.338491Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 38 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:01.342485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:01.342668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:01.343145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:01.343256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:01.343258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:01.343260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:01.343261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:01.343263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:01.343265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:01.343267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.343268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.343270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.343279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:01.343306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:01.343311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:01.343316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:01.343320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:01.343325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:01.343329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.343336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.343341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.343352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:01.343363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:01.343370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:01.343376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:01.343386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:01.343392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:01.343397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:01.343403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:01.343411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:01.343417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:01.343423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:01.343430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:01.343436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.343443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.343447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.343453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.343459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.343464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.343498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:01.357896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:01.359072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:01.359080Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:01.359083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:01.359089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:01.359098Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 37 [2:7559340787732304996:2593]
2025-10-09T21:43:01.359118Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:01.359140Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:01.359144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.359148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.359151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.359172Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:01.359178Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:01.359186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:01.359263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.359266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.359268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.359271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:01.359366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:01.359370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.359388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.359401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.359420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:01.359442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340787732304991:2593]
2025-10-09T21:43:01.359445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340787732304991:2593]
2025-10-09T21:43:01.359448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340787732304991:2593]
2025-10-09T21:43:01.359465Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:01.359481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:01.359498Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:01.359502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:01.359533Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:01.359537Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:01.359540Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:01.359554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340787732304991:2593]
2025-10-09T21:43:01.359558Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:01.359653Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:01.360019Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 37
2025-10-09T21:43:01.360079Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 37
2025-10-09T21:43:01.361266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340788094113229:2432], now have 1 active actors on pipe
2025-10-09T21:43:01.361280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340788094113317:2713], now have 1 active actors on pipe
2025-10-09T21:43:01.361302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:01.361309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:01.361361Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d9e7934b-c345e38b-c5dab110-8d7b4e69_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:01.361373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.361380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.361383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:01.361399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.361417Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:01.361439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:01.362149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:01.362168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:01.362204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:01.362426Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 38 partition: 0 MaxSeqNo: 25 sessionId: src_id|d9e7934b-c345e38b-c5dab110-8d7b4e69_0
2025-10-09T21:43:01.362901Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b324de1-4ca6812e-db28055b-5a72cd31_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d9e7934b-c345e38b-c5dab110-8d7b4e69_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:01.365996Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d9e7934b-c345e38b-c5dab110-8d7b4e69_0] Write session: try to update token
2025-10-09T21:43:01.366021Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d9e7934b-c345e38b-c5dab110-8d7b4e69_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:01.382764Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:01.382792Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:01.382794Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:01.382796Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:01.382809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340787732304991:2593]
2025-10-09T21:43:01.382814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340787732304991:2593]
2025-10-09T21:43:01.382817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340787732304991:2593]
2025-10-09T21:43:01.382865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:01.382873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.382885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:01.383037Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:01.386463Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:01.386484Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:01.386490Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:01.402818Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 38 sessionId: src_id|d9e7934b-c345e38b-c5dab110-8d7b4e69_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:01.405696Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:01.417989Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340788094113330:2476]
2025-10-09T21:43:01.418661Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 37, pipe: [1:7559340788094113330:2476]
2025-10-09T21:43:01.621180Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:01.621524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:01.621539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:01.621585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340788094113330:2476], now have 1 active actors on pipe
2025-10-09T21:43:01.621844Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:01.622124Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:01.622174Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:01.622259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 38 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d9e7934b-c345e38b-c5dab110-8d7b4e69_0
2025-10-09T21:43:01.622464Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 38 sessionId: src_id|d9e7934b-c345e38b-c5dab110-8d7b4e69_0 is DEAD
2025-10-09T21:43:01.623047Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9e7934b-c345e38b-c5dab110-8d7b4e69_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:01.623096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9e7934b-c345e38b-c5dab110-8d7b4e69_0] Write session will restart in 0.005803s
2025-10-09T21:43:01.623122Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9e7934b-c345e38b-c5dab110-8d7b4e69_0] Write session: Do CDS request
2025-10-09T21:43:01.623128Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9e7934b-c345e38b-c5dab110-8d7b4e69_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:01.623712Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340788094113330:2476]
2025-10-09T21:43:01.630603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d9e7934b-c345e38b-c5dab110-8d7b4e69_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:01.631829Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:01.631849Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 39
2025-10-09T21:43:01.632276Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 39 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:01.632324Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 39 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:01.632330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=39 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:01.632335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:01.638907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 39 sessionId:  describe result for acl check
2025-10-09T21:43:01.638963Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:01.638965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:01.638966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:01.638973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340788094113407:2726] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:01.639593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340788094113407:2726] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:01.642852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:01.644446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:01.644729Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:01.644740Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:01.644754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:01.644839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:01.644892Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340787732305121:2606]
2025-10-09T21:43:01.645562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:01.646490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:01.646572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:01.646956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:01.647011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:01.647012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:01.647014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:01.647016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:01.647018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:01.647019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:01.647021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.647022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.647024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.647030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:01.647041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:01.647047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:01.647052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:01.647057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:01.647062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:01.647067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.647072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.647077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.647085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:01.647094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:01.647101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:01.647108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:01.647114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:01.647121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:01.647126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:01.647132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:01.647138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:01.647145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:01.647150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:01.647756Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340788094113407:2726] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:01.647156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:01.647163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.647169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.647175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.647182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.647188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.647195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.647229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:01.659001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:01.659426Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340788094113407:2726] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:01.659444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340788094113407:2726] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:01.659449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340788094113407:2726] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:01.659460Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 39 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:01.660136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:01.660153Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:01.660156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:01.660161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:01.660169Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 38 [2:7559340787732305121:2606]
2025-10-09T21:43:01.660185Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:01.660203Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:01.660213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.660216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.660217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.660233Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:01.660241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:01.660247Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:01.660299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.660305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.660307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.660310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:01.660383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:01.660395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.660409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.660420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.660437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:01.660485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340788094113352:2432], now have 1 active actors on pipe
2025-10-09T21:43:01.660495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340788094113448:2726], now have 1 active actors on pipe
2025-10-09T21:43:01.660504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340787732305116:2606]
2025-10-09T21:43:01.660507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340787732305116:2606]
2025-10-09T21:43:01.660510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340787732305116:2606]
2025-10-09T21:43:01.660507Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 38
2025-10-09T21:43:01.660544Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 38
2025-10-09T21:43:01.661174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 39 partition: 0 MaxSeqNo: 25 sessionId: src_id|baebfc14-d255ac3-b6a1cbb4-a80a9d88_0
2025-10-09T21:43:01.660526Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:01.660543Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:01.660548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:01.660548Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:01.660551Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:01.660564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:01.660567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:01.660614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340787732305116:2606]
2025-10-09T21:43:01.660618Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:01.660624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:01.660639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:01.660644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:01.660701Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|baebfc14-d255ac3-b6a1cbb4-a80a9d88_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:01.660709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.660713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.660716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:01.660726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.660737Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:01.660764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:01.661071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:01.661077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:01.661097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:01.666162Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9e7934b-c345e38b-c5dab110-8d7b4e69_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|baebfc14-d255ac3-b6a1cbb4-a80a9d88_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:01.669295Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|baebfc14-d255ac3-b6a1cbb4-a80a9d88_0] Write session: try to update token
2025-10-09T21:43:01.669318Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|baebfc14-d255ac3-b6a1cbb4-a80a9d88_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:01.677846Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:01.677869Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:01.677871Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:01.677874Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:01.677885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340787732305116:2606]
2025-10-09T21:43:01.677889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340787732305116:2606]
2025-10-09T21:43:01.677893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340787732305116:2606]
2025-10-09T21:43:01.677949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:01.677960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.677963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:01.678125Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:01.682140Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:01.682156Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:01.682160Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:01.696426Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 39 sessionId: src_id|baebfc14-d255ac3-b6a1cbb4-a80a9d88_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:01.698856Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:01.723646Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340788094113460:2476]
2025-10-09T21:43:01.723955Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 38, pipe: [1:7559340788094113460:2476]
2025-10-09T21:43:01.836254Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:01.836725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:01.836743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:01.836767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340788094113460:2476], now have 1 active actors on pipe
2025-10-09T21:43:01.837017Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:01.837064Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:01.837056Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340788094113460:2476]
2025-10-09T21:43:01.837064Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:01.837067Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:01.837128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 39 reason: error at write: Disconnected sessionId: src_id|baebfc14-d255ac3-b6a1cbb4-a80a9d88_0
2025-10-09T21:43:01.837322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 39 sessionId: src_id|baebfc14-d255ac3-b6a1cbb4-a80a9d88_0 is DEAD
2025-10-09T21:43:01.837571Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baebfc14-d255ac3-b6a1cbb4-a80a9d88_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:01.837586Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baebfc14-d255ac3-b6a1cbb4-a80a9d88_0] Write session will restart in 0.006670s
2025-10-09T21:43:01.837614Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baebfc14-d255ac3-b6a1cbb4-a80a9d88_0] Write session: Do CDS request
2025-10-09T21:43:01.837620Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baebfc14-d255ac3-b6a1cbb4-a80a9d88_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:01.845659Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|baebfc14-d255ac3-b6a1cbb4-a80a9d88_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:01.845756Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:01.845772Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 40
2025-10-09T21:43:01.848364Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 40 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:01.848416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 40 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:01.848422Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=40 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:01.848429Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:01.848874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 40 sessionId:  describe result for acl check
2025-10-09T21:43:01.848906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:01.848908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:01.848910Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:01.848916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340788094113525:2732] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:01.849531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340788094113525:2732] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:01.851089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:01.853216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340788094113525:2732] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:01.856670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:01.856929Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:01.856938Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:01.856951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:01.857012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:01.857063Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340787732305225:2611]
2025-10-09T21:43:01.857666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:01.857996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:01.858053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:01.858366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:01.858440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:01.858446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:01.858447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:01.858448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:01.858450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:01.858451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:01.858452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.858453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.858454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.858459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:01.858469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:01.858472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:01.858475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:01.858478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:01.858481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:01.858485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.858488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.858491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.858498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:01.858504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:01.858509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:01.858514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:01.858517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:01.858522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:01.858525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:01.858529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:01.858533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:01.858537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:01.858540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:01.858545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:01.858549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.858553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.858559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.858563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.858567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.858573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.858600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:01.863087Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340788094113525:2732] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:01.863104Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340788094113525:2732] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:01.863108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340788094113525:2732] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:01.863119Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 40 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:01.868617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:01.869739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:01.869756Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:01.869760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:01.869765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:01.869777Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 39 [2:7559340787732305225:2611]
2025-10-09T21:43:01.870251Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 39
2025-10-09T21:43:01.870259Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 39
2025-10-09T21:43:01.869804Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:01.869828Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:01.869831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.869835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.871579Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 40 partition: 0 MaxSeqNo: 25 sessionId: src_id|7faf34f8-4b894c9d-42e1861-b35f2bf9_0
2025-10-09T21:43:01.869837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.869854Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:01.869860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:01.869868Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:01.869936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.869938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.869940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.869943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:01.870022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:01.870025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.870055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:01.870066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:01.870083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:01.870103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340787732305220:2611]
2025-10-09T21:43:01.870106Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340787732305220:2611]
2025-10-09T21:43:01.870108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340787732305220:2611]
2025-10-09T21:43:01.870117Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:01.870127Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:01.870130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:01.870132Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:01.870138Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:01.870153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:01.870157Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:01.870170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340788094113564:2732], now have 1 active actors on pipe
2025-10-09T21:43:01.870174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340788094113471:2432], now have 1 active actors on pipe
2025-10-09T21:43:01.870216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340787732305220:2611]
2025-10-09T21:43:01.870224Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:01.870230Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:01.870472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:01.870481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:01.870515Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7faf34f8-4b894c9d-42e1861-b35f2bf9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:01.870526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.870530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.870534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:01.870545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.870557Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:01.870598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:01.870749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:01.870756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:01.870776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:01.872038Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baebfc14-d255ac3-b6a1cbb4-a80a9d88_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7faf34f8-4b894c9d-42e1861-b35f2bf9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:01.874939Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7faf34f8-4b894c9d-42e1861-b35f2bf9_0] Write session: try to update token
2025-10-09T21:43:01.874960Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7faf34f8-4b894c9d-42e1861-b35f2bf9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:01.889473Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:01.889493Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:01.889495Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:01.889498Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:01.889507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340787732305220:2611]
2025-10-09T21:43:01.889513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340787732305220:2611]
2025-10-09T21:43:01.889517Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340787732305220:2611]
2025-10-09T21:43:01.889566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:01.889580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:01.889586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:01.889576Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:01.889595Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:01.889600Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:01.889748Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:01.905435Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 40 sessionId: src_id|7faf34f8-4b894c9d-42e1861-b35f2bf9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:01.908564Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:01.937169Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340788094113573:2476]
2025-10-09T21:43:01.938394Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 39, pipe: [1:7559340788094113573:2476]
2025-10-09T21:43:02.049440Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:02.049794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:02.049808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:02.049831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340788094113573:2476], now have 1 active actors on pipe
2025-10-09T21:43:02.050105Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:02.050085Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:02.050113Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340788094113573:2476]
2025-10-09T21:43:02.050139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:02.050142Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:02.050232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 40 reason: error at write: Disconnected sessionId: src_id|7faf34f8-4b894c9d-42e1861-b35f2bf9_0
2025-10-09T21:43:02.050458Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 40 sessionId: src_id|7faf34f8-4b894c9d-42e1861-b35f2bf9_0 is DEAD
2025-10-09T21:43:02.050608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7faf34f8-4b894c9d-42e1861-b35f2bf9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:02.050618Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7faf34f8-4b894c9d-42e1861-b35f2bf9_0] Write session will restart in 0.006721s
2025-10-09T21:43:02.050644Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7faf34f8-4b894c9d-42e1861-b35f2bf9_0] Write session: Do CDS request
2025-10-09T21:43:02.050651Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7faf34f8-4b894c9d-42e1861-b35f2bf9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:02.058743Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7faf34f8-4b894c9d-42e1861-b35f2bf9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:02.058898Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:02.058926Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 41
2025-10-09T21:43:02.061866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:02.062406Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 41 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:02.062475Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 41 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:02.062488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=41 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:02.062494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:02.063188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:02.063426Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:02.063437Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:02.063450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:02.063437Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 41 sessionId:  describe result for acl check
2025-10-09T21:43:02.063487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:02.063489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:02.063491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:02.063497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340792389080929:2736] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:02.063508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:02.063568Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340792027272614:2613]
2025-10-09T21:43:02.064187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:02.064099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340792389080929:2736] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:02.065964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:02.066053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:02.066477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:02.066545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:02.066546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:02.066547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:02.066548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:02.066550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:02.066551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:02.066552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.066553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.066555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.066561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:02.066571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:02.066574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:02.066578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:02.066581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:02.066585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:02.066588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.066591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.066594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.066603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:02.066611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:02.066618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:02.066626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:02.066634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:02.066643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:02.066650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:02.066657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:02.066662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:02.066666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:02.066671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:02.066675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:02.066680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.066688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.066692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.066697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.066702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.066707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.066735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:02.070858Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340792389080929:2736] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:02.074614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:02.074918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340792389080929:2736] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:02.074936Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340792389080929:2736] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:02.074939Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340792389080929:2736] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:02.074949Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 41 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:02.075772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:02.075792Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:02.075795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:02.075800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:02.075808Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 40 [2:7559340792027272614:2613]
2025-10-09T21:43:02.075824Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:02.075845Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:02.075852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.075855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.075858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.075877Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:02.075883Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:02.075891Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:02.075967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.075977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.075979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.075982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:02.076062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:02.076069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.076084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.076096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.076114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:02.076197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 40
2025-10-09T21:43:02.076232Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 40
2025-10-09T21:43:02.076139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340792389080966:2736], now have 1 active actors on pipe
2025-10-09T21:43:02.076147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340792389080872:2432], now have 1 active actors on pipe
2025-10-09T21:43:02.076155Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340792027272609:2613]
2025-10-09T21:43:02.076157Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340792027272609:2613]
2025-10-09T21:43:02.076160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340792027272609:2613]
2025-10-09T21:43:02.076176Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:02.076191Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:02.076207Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:02.076207Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:02.076210Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:02.076211Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:02.076213Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:02.076254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340792027272609:2613]
2025-10-09T21:43:02.076257Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:02.076262Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:02.076273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:02.076278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:02.076335Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|38c00ac-8a1d9265-5da2caa6-5b99f311_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:02.076343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.076348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.076350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:02.076364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.076378Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:02.076726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 41 partition: 0 MaxSeqNo: 25 sessionId: src_id|38c00ac-8a1d9265-5da2caa6-5b99f311_0
2025-10-09T21:43:02.076399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:02.076614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:02.076618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:02.076635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:02.077156Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7faf34f8-4b894c9d-42e1861-b35f2bf9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|38c00ac-8a1d9265-5da2caa6-5b99f311_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:02.080211Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38c00ac-8a1d9265-5da2caa6-5b99f311_0] Write session: try to update token
2025-10-09T21:43:02.080232Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38c00ac-8a1d9265-5da2caa6-5b99f311_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:02.091352Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:02.091373Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:02.091386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340792027272609:2613]
2025-10-09T21:43:02.091442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:02.091441Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:02.091638Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:02.091815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:02.091968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:02.092126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:02.092335Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:02.092387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:02.103561Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:02.103581Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:02.103583Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:02.103586Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:02.103597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340792027272609:2613]
2025-10-09T21:43:02.103602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340792027272609:2613]
2025-10-09T21:43:02.103608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340792027272609:2613]
2025-10-09T21:43:02.103661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:02.103660Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:02.103666Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:02.103670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.103670Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:02.103673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:02.103826Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:02.112630Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 41 sessionId: src_id|38c00ac-8a1d9265-5da2caa6-5b99f311_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:02.115768Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:02.150466Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340792389080975:2476]
2025-10-09T21:43:02.150993Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 40, pipe: [1:7559340792389080975:2476]
2025-10-09T21:43:02.268944Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:02.269289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:02.269308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:02.269334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340792389080975:2476], now have 1 active actors on pipe
2025-10-09T21:43:02.269765Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340792389080975:2476]
2025-10-09T21:43:02.269800Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:02.269803Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:02.269804Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:02.269871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 41 reason: error at write: Disconnected sessionId: src_id|38c00ac-8a1d9265-5da2caa6-5b99f311_0
2025-10-09T21:43:02.270088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 41 sessionId: src_id|38c00ac-8a1d9265-5da2caa6-5b99f311_0 is DEAD
2025-10-09T21:43:02.271191Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:02.274155Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38c00ac-8a1d9265-5da2caa6-5b99f311_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:02.274173Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38c00ac-8a1d9265-5da2caa6-5b99f311_0] Write session will restart in 0.009596s
2025-10-09T21:43:02.274204Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38c00ac-8a1d9265-5da2caa6-5b99f311_0] Write session: Do CDS request
2025-10-09T21:43:02.274211Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38c00ac-8a1d9265-5da2caa6-5b99f311_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:02.286593Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38c00ac-8a1d9265-5da2caa6-5b99f311_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:02.288320Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:02.288336Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 42
2025-10-09T21:43:02.288563Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 42 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:02.288599Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 42 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:02.288610Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=42 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:02.288615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:02.289574Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 42 sessionId:  describe result for acl check
2025-10-09T21:43:02.289618Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:02.289620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:02.289622Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:02.289629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340792389081035:2740] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:02.290223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340792389081035:2740] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:02.294450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:02.294992Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340792389081035:2740] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:02.302244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:02.302545Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:02.302551Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:02.302565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:02.302635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:02.302682Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340792027272711:2615]
2025-10-09T21:43:02.303344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:02.303783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:02.303857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:02.304234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:02.304302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:02.304304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:02.304305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:02.304307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:02.304308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:02.304310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:02.304311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.304312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.304314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.304320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:02.304332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:02.304336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:02.304340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:02.304345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:02.304348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:02.304352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.304356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.304360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.304369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:02.304379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:02.304386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:02.304392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:02.304400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:02.304406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:02.304411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:02.304416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:02.304422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:02.304430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:02.304444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:02.304450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:02.304456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.304461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.304467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.304473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.304478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.304484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.304515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:02.306611Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340792389081035:2740] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:02.306642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340792389081035:2740] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:02.306646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340792389081035:2740] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:02.306673Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 42 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:02.315467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:02.317505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:02.317512Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:02.317515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:02.317520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:02.317528Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 41 [2:7559340792027272711:2615]
2025-10-09T21:43:02.317545Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:02.317568Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:02.317572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.317575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.317579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.317599Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:02.317605Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:02.317614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:02.317687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.317690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.317692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.317695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:02.317780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:02.317783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.317798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.317809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.317825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:02.317846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340792027272706:2615]
2025-10-09T21:43:02.317848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340792027272706:2615]
2025-10-09T21:43:02.317851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340792027272706:2615]
2025-10-09T21:43:02.317860Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:02.317871Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:02.317873Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:02.317875Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:02.317877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340792389081073:2740], now have 1 active actors on pipe
2025-10-09T21:43:02.317880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340792389080978:2432], now have 1 active actors on pipe
2025-10-09T21:43:02.317890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:02.317906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:02.317909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:02.317957Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340792027272706:2615]
2025-10-09T21:43:02.317963Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:02.317970Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:02.319303Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 41
2025-10-09T21:43:02.319340Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 41
2025-10-09T21:43:02.319596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:02.319614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:02.319663Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e8530b24-8445f4fa-813ca1da-643a8a06_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:02.319678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.319684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.319688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:02.319712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.319731Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:02.319754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:02.320792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:02.320803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:02.320830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:02.321016Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 42 partition: 0 MaxSeqNo: 25 sessionId: src_id|e8530b24-8445f4fa-813ca1da-643a8a06_0
2025-10-09T21:43:02.321545Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38c00ac-8a1d9265-5da2caa6-5b99f311_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e8530b24-8445f4fa-813ca1da-643a8a06_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:02.324771Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8530b24-8445f4fa-813ca1da-643a8a06_0] Write session: try to update token
2025-10-09T21:43:02.324798Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8530b24-8445f4fa-813ca1da-643a8a06_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:02.340638Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:02.340657Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:02.340660Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:02.340662Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:02.340673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340792027272706:2615]
2025-10-09T21:43:02.340678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340792027272706:2615]
2025-10-09T21:43:02.340688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340792027272706:2615]
2025-10-09T21:43:02.340734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:02.340742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.340745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:02.340888Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:02.342882Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:02.342893Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:02.342898Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:02.364678Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 42 sessionId: src_id|e8530b24-8445f4fa-813ca1da-643a8a06_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:02.367280Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:02.369948Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340792389081083:2476]
2025-10-09T21:43:02.370300Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 41, pipe: [1:7559340792389081083:2476]
2025-10-09T21:43:02.582846Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:02.583203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:02.583219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:02.583243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340792389081083:2476], now have 1 active actors on pipe
2025-10-09T21:43:02.583552Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:02.583539Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:02.583553Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:02.583585Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:02.583623Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 42 reason: error at write: Disconnected sessionId: src_id|e8530b24-8445f4fa-813ca1da-643a8a06_0
2025-10-09T21:43:02.583628Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340792389081083:2476]
2025-10-09T21:43:02.583827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 42 sessionId: src_id|e8530b24-8445f4fa-813ca1da-643a8a06_0 is DEAD
2025-10-09T21:43:02.583982Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8530b24-8445f4fa-813ca1da-643a8a06_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:02.583992Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8530b24-8445f4fa-813ca1da-643a8a06_0] Write session will restart in 0.009863s
2025-10-09T21:43:02.584021Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8530b24-8445f4fa-813ca1da-643a8a06_0] Write session: Do CDS request
2025-10-09T21:43:02.584028Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8530b24-8445f4fa-813ca1da-643a8a06_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:02.596222Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8530b24-8445f4fa-813ca1da-643a8a06_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:02.596590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:02.596617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 43
2025-10-09T21:43:02.596844Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 43 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:02.596893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 43 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:02.596904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=43 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:02.596908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:02.597454Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 43 sessionId:  describe result for acl check
2025-10-09T21:43:02.597494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:02.597501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:02.597503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:02.597510Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340792389081108:2745] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:02.598135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340792389081108:2745] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:02.600725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340792389081108:2745] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:02.604414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340792389081108:2745] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:02.604430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340792389081108:2745] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:02.604435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340792389081108:2745] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:02.604445Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 43 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:02.604497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:02.606197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:02.606530Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:02.606540Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:02.606557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:02.606645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:02.606696Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340792027272810:2617]
2025-10-09T21:43:02.607364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:02.607817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:02.607904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:02.608312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:02.608397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:02.608398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:02.608400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:02.608401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:02.608403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:02.608404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:02.608406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.608407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.608408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.608415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:02.608426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:02.608430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:02.608435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:02.608439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:02.608443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:02.608448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.608452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.608456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.608468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:02.608477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:02.608484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:02.608489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:02.608495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:02.608501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:02.608506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:02.608511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:02.608517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:02.608522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:02.608528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:02.608534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:02.608540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.608545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.608551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.608557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.608563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.608568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.608601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:02.615773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:02.616932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:02.616951Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:02.616955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:02.616959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:02.616966Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 42 [2:7559340792027272810:2617]
2025-10-09T21:43:02.616981Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:02.616999Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:02.617010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.617013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.617016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.617034Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:02.617047Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:02.617054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:02.617122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.617131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.617136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.617139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:02.617213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:02.617223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.617235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.617246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.617270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:02.617296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340792027272801:2617]
2025-10-09T21:43:02.617303Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340792027272801:2617]
2025-10-09T21:43:02.617305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340792027272801:2617]
2025-10-09T21:43:02.617315Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:02.617332Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:02.617336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:02.617344Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:02.617347Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:02.617351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:02.617354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:02.617366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340792389081178:2745], now have 1 active actors on pipe
2025-10-09T21:43:02.617376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340792389081086:2432], now have 1 active actors on pipe
2025-10-09T21:43:02.617417Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340792027272801:2617]
2025-10-09T21:43:02.617429Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:02.617435Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:02.617447Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 42
2025-10-09T21:43:02.617450Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 42
2025-10-09T21:43:02.617490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:02.617504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:02.617534Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:02.617542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.617546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.617549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:02.617560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.617577Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:02.617598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:02.617756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:02.617767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:02.617799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:02.617916Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 43 partition: 0 MaxSeqNo: 25 sessionId: src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0
2025-10-09T21:43:02.618317Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8530b24-8445f4fa-813ca1da-643a8a06_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:02.621444Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0] Write session: try to update token
2025-10-09T21:43:02.621463Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:02.629711Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:02.629727Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:02.629737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340792027272801:2617]
2025-10-09T21:43:02.629784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:02.629784Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:02.629956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:02.630121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:02.630266Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:02.630410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:02.630632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:02.630671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:02.636534Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 43 sessionId: src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:02.639069Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:02.639305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:02.639324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:02.639375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:02.639559Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:02.642729Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:02.642746Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:02.642751Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:02.642753Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:02.642762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340792027272801:2617]
2025-10-09T21:43:02.642766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340792027272801:2617]
2025-10-09T21:43:02.642769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340792027272801:2617]
2025-10-09T21:43:02.642814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:02.642820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.642823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:02.642830Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:02.642836Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:02.642840Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:02.642971Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:02.683646Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340792389081204:2476]
2025-10-09T21:43:02.684028Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 42, pipe: [1:7559340792389081204:2476]
2025-10-09T21:43:02.820361Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:02.829391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:02.829424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:02.829597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:02.829660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:02.829719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:02.829774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:02.829822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:02.829834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:02.829907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:02.829962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:02.830020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:02.830188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:02.830245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:02.830254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:02.832920Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:02.832934Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:02.832970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 43 reason: error at write: Disconnected sessionId: src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0
2025-10-09T21:43:02.832972Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:02.833030Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340792389081204:2476]
2025-10-09T21:43:02.833177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 43 sessionId: src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0 is DEAD
2025-10-09T21:43:02.830320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:02.830384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:02.830455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:02.830510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:02.830548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:02.830552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:02.830620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:02.830682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:02.830742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:02.830802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:02.830847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:02.830850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:02.830918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:02.830975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:02.831033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:02.831094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:02.831131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:02.831134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:02.831202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:02.831263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:02.831320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:02.831383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:02.831435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:02.831438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:02.831505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:02.831566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:02.831635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:02.831700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:02.831740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:02.831746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:02.831811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:02.831872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:02.831930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:02.831987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:02.832029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:02.832032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:02.832101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:02.832161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:02.832222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:02.832289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:02.832330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:02.832333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:02.832403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:02.832459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:02.832520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:02.832585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:02.832623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:02.832626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:02.832693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340792389081204:2476], now have 1 active actors on pipe
2025-10-09T21:43:02.833873Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:02.841311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:02.841329Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0] Write session will restart in 0.008546s
2025-10-09T21:43:02.841355Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0] Write session: Do CDS request
2025-10-09T21:43:02.841362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:02.851957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:02.853179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:02.853396Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:02.853407Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:02.853420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:02.853480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:02.853522Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340792027272949:2633]
2025-10-09T21:43:02.854264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:02.854776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:02.854826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:02.855221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:02.855271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:02.855272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:02.855274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:02.855275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:02.855276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:02.855278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:02.855279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.855281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.855282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.855288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:02.855300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:02.855304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:02.855308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:02.855312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:02.855317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:02.855321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.855325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.855329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.855336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:02.855344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:02.855351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:02.855356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:02.855361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:02.855366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:02.855373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:02.855378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:02.855383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:02.855388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:02.855394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:02.855399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:02.855405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.855410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.855414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.855420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.855425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.855430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.855462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:02.856538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:02.862700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:02.863940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:02.863960Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:02.863964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:02.863968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:02.863977Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 43 [2:7559340792027272949:2633]
2025-10-09T21:43:02.863992Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:02.864011Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:02.864015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.864019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.864021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.864038Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:02.864043Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:02.864055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:02.864125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.864135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.864137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.864140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:02.864220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:02.864231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.864245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:02.864258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:02.864276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:02.864292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340792027272943:2633]
2025-10-09T21:43:02.864295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340792027272943:2633]
2025-10-09T21:43:02.864298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340792027272943:2633]
2025-10-09T21:43:02.864314Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:02.864328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:02.864344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:02.864347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:02.864385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340792027272943:2633]
2025-10-09T21:43:02.864388Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:02.864476Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:02.864486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:02.864491Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:02.864497Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:02.864828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340792389081226:2432], now have 1 active actors on pipe
2025-10-09T21:43:02.862539Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:02.862559Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 44
2025-10-09T21:43:02.865271Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 43
2025-10-09T21:43:02.866268Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 44 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:02.866329Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 44 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:02.866336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=44 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:02.866342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:02.870562Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 44 sessionId:  describe result for acl check
2025-10-09T21:43:02.870616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:02.870618Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:02.870620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:02.870628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340792389081294:2758] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:02.871280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340792389081294:2758] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:02.885093Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:02.885108Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:02.885111Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:02.885113Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:02.885145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340792027272943:2633]
2025-10-09T21:43:02.885149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340792027272943:2633]
2025-10-09T21:43:02.885153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340792027272943:2633]
2025-10-09T21:43:02.885194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:02.885203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:02.885213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:02.885365Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:02.888634Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:02.888654Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:02.888658Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:02.894960Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340792389081294:2758] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:02.907193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340792389081294:2758] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:02.907211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340792389081294:2758] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:02.907216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340792389081294:2758] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:02.907225Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 44 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:02.910482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 43
2025-10-09T21:43:02.933016Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340792389081332:2476]
2025-10-09T21:43:02.933390Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 43, pipe: [1:7559340792389081332:2476]
2025-10-09T21:43:03.056999Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340792389081332:2476]
2025-10-09T21:43:03.057034Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:03.057087Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:03.057121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 44 reason: pipe to partition's 0 tablet is dead #2 sessionId: 
2025-10-09T21:43:03.057345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 44 sessionId:  is DEAD
2025-10-09T21:43:03.056310Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:03.056662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340792389081329:2758], now have 1 active actors on pipe
2025-10-09T21:43:03.056691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:03.056699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:03.056717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340792389081332:2476], now have 1 active actors on pipe
2025-10-09T21:43:03.057785Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:03.060145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: pipe to partition's 0 tablet is dead #2, code: 500027 
2025-10-09T21:43:03.060160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0] Write session will restart in 0.012969s
2025-10-09T21:43:03.060194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0] Write session: Do CDS request
2025-10-09T21:43:03.060201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:03.069912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:03.073246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:03.073496Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:03.073501Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:03.073515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:03.073578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:03.073623Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340796322240344:2635]
2025-10-09T21:43:03.074266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:03.075059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:03.075124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:03.075568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:03.075658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:03.075666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:03.075668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:03.075669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:03.075671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:03.075673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:03.075674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.075676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.075678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.075685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:03.075696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:03.075701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:03.075706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:03.075711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:03.075716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:03.075719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.075723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.075728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.075736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:03.075745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:03.075755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:03.075760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:03.075764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:03.075769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:03.075774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:03.075780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:03.075785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:03.075790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:03.075794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:03.075799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:03.075806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.075811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.075816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.075822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.075826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.075831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.075871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:03.077449Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:03.077707Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:03.077725Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 45
2025-10-09T21:43:03.078098Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 45 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:03.078158Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 45 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:03.078165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=45 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:03.078171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:03.078973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 45 sessionId:  describe result for acl check
2025-10-09T21:43:03.079009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:03.079012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:03.079014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:03.079020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340796684048695:2762] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:03.079643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340796684048695:2762] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:03.086737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:03.088043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:03.088050Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:03.088053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:03.088058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:03.088066Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 44 [2:7559340796322240344:2635]
2025-10-09T21:43:03.088083Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:03.088104Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:03.088107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.088110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.088113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.088135Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:03.088141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:03.088148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:03.088216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.088218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.088220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.088223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:03.088474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:03.088482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.088504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.088518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.088537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:03.088587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340796322240338:2635]
2025-10-09T21:43:03.088596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340796322240338:2635]
2025-10-09T21:43:03.088598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340796322240338:2635]
2025-10-09T21:43:03.088611Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:03.088657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:03.088684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:03.088689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:03.088744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340796322240338:2635]
2025-10-09T21:43:03.088747Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:03.088765Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:03.088769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:03.088772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:03.088779Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:03.089164Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 44
2025-10-09T21:43:03.090968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340796684048695:2762] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:03.093023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340796684048630:2432], now have 1 active actors on pipe
2025-10-09T21:43:03.106083Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340796684048695:2762] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:03.106105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340796684048695:2762] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:03.106110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340796684048695:2762] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:03.106122Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 45 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:03.107174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340796684048733:2762], now have 1 active actors on pipe
2025-10-09T21:43:03.107301Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 44
2025-10-09T21:43:03.107448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:03.107458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:03.107488Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4721dca7-1be77a2e-b4ce2100-c9483e01_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:03.107504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.107508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.107511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:03.107524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.107542Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:03.107563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:03.107876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:03.107882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:03.107894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:03.108032Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 45 partition: 0 MaxSeqNo: 25 sessionId: src_id|4721dca7-1be77a2e-b4ce2100-c9483e01_0
2025-10-09T21:43:03.108448Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7608aaf3-72a9016e-2b2e881f-22b6a61a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4721dca7-1be77a2e-b4ce2100-c9483e01_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:03.111877Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4721dca7-1be77a2e-b4ce2100-c9483e01_0] Write session: try to update token
2025-10-09T21:43:03.111900Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4721dca7-1be77a2e-b4ce2100-c9483e01_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:03.128362Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:03.128382Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:03.128384Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:03.128387Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:03.128397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340796322240338:2635]
2025-10-09T21:43:03.128403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340796322240338:2635]
2025-10-09T21:43:03.128408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340796322240338:2635]
2025-10-09T21:43:03.128458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:03.128466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.128469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:03.128640Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:03.131528Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:03.131544Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:03.131549Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:03.142409Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 45 sessionId: src_id|4721dca7-1be77a2e-b4ce2100-c9483e01_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:03.145589Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:03.157519Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340796684048737:2476]
2025-10-09T21:43:03.158470Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 44, pipe: [1:7559340796684048737:2476]
2025-10-09T21:43:03.342754Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:03.343107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:03.343119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:03.343141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340796684048737:2476], now have 1 active actors on pipe
2025-10-09T21:43:03.343582Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:03.343599Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:03.343697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 45 reason: error at write: Disconnected sessionId: src_id|4721dca7-1be77a2e-b4ce2100-c9483e01_0
2025-10-09T21:43:03.344091Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 45 sessionId: src_id|4721dca7-1be77a2e-b4ce2100-c9483e01_0 is DEAD
2025-10-09T21:43:03.344130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4721dca7-1be77a2e-b4ce2100-c9483e01_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:03.344141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4721dca7-1be77a2e-b4ce2100-c9483e01_0] Write session will restart in 0.007096s
2025-10-09T21:43:03.344135Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340796684048737:2476]
2025-10-09T21:43:03.344166Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4721dca7-1be77a2e-b4ce2100-c9483e01_0] Write session: Do CDS request
2025-10-09T21:43:03.344173Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4721dca7-1be77a2e-b4ce2100-c9483e01_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:03.344167Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:03.343486Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:03.352634Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4721dca7-1be77a2e-b4ce2100-c9483e01_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:03.353103Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:03.353119Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 46
2025-10-09T21:43:03.353306Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 46 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:03.353337Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 46 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:03.353344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=46 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:03.353347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:03.353840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 46 sessionId:  describe result for acl check
2025-10-09T21:43:03.353873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:03.353876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:03.353877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:03.353884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340796684048767:2767] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:03.354539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340796684048767:2767] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:03.360326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:03.361256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:03.361467Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:03.361471Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:03.361484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:03.361541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:03.361577Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340796322240446:2637]
2025-10-09T21:43:03.362207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:03.362699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340796684048767:2767] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:03.362637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:03.362680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:03.363008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:03.363059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:03.363060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:03.363062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:03.363063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:03.363065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:03.363067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:03.363068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.363070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.363071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.363076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:03.363087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:03.363092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:03.363096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:03.363100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:03.363105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:03.363109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.363113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.363117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.363124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:03.363132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:03.363138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:03.363143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:03.363148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:03.363153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:03.363159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:03.363164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:03.363169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:03.363174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:03.363178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:03.363183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:03.363188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.363193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.363199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.363204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.363208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.363213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.363242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:03.371408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340796684048767:2767] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:03.371423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340796684048767:2767] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:03.371431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340796684048767:2767] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:03.371440Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 46 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:03.372205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:03.373530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:03.373549Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:03.373552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:03.373556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:03.373564Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 45 [2:7559340796322240446:2637]
2025-10-09T21:43:03.373580Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:03.373599Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:03.373602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.373605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.373607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.373626Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:03.373640Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:03.373647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:03.373714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.373721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.373724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.373726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:03.373800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:03.373810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.373822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.373833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.373850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:03.373868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340796684048841:2767], now have 1 active actors on pipe
2025-10-09T21:43:03.373871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340796684048742:2432], now have 1 active actors on pipe
2025-10-09T21:43:03.373879Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340796322240440:2637]
2025-10-09T21:43:03.373882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340796322240440:2637]
2025-10-09T21:43:03.373884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340796322240440:2637]
2025-10-09T21:43:03.373896Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:03.373910Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:03.373925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:03.373928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:03.373963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340796322240440:2637]
2025-10-09T21:43:03.373966Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:03.374094Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 45
2025-10-09T21:43:03.374150Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 45
2025-10-09T21:43:03.374160Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:03.374169Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:03.374171Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:03.374177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:03.374667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:03.374680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:03.374712Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7b582a4f-176439f-95d44d83-5cb8b31b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:03.374725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.374730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.374734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:03.374745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.374762Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:03.374784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:03.376689Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4721dca7-1be77a2e-b4ce2100-c9483e01_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7b582a4f-176439f-95d44d83-5cb8b31b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:03.375317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:03.375327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:03.375345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:03.375732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 46 partition: 0 MaxSeqNo: 25 sessionId: src_id|7b582a4f-176439f-95d44d83-5cb8b31b_0
2025-10-09T21:43:03.379960Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b582a4f-176439f-95d44d83-5cb8b31b_0] Write session: try to update token
2025-10-09T21:43:03.379981Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b582a4f-176439f-95d44d83-5cb8b31b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:03.393462Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:03.393479Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:03.393489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340796322240440:2637]
2025-10-09T21:43:03.393543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:03.393559Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:03.393746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:03.393907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:03.394097Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:03.394263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:03.394541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:03.394594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:03.402212Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 46 sessionId: src_id|7b582a4f-176439f-95d44d83-5cb8b31b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:03.405351Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:03.411157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:03.411181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:03.411228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:03.413705Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:03.418853Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:03.418872Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:03.418876Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:03.418878Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:03.418896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340796322240440:2637]
2025-10-09T21:43:03.418902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340796322240440:2637]
2025-10-09T21:43:03.418906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340796322240440:2637]
2025-10-09T21:43:03.418970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:03.418978Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:03.418986Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:03.418990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.418993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:03.418997Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:03.419152Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:03.445048Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340796684048850:2476]
2025-10-09T21:43:03.445486Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 45, pipe: [1:7559340796684048850:2476]
2025-10-09T21:43:03.645751Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:03.646235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340796684048850:2476], now have 1 active actors on pipe
2025-10-09T21:43:03.656557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:03.656584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:03.656765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:03.656821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:03.656883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:03.656940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:03.657021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:03.657026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:03.657094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:03.657155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:03.657222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:03.657287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:03.657328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:03.657335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:03.657401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:03.657457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:03.657516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:03.657573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:03.657623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:03.657626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:03.657692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:03.657750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:03.657806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:03.657867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:03.657909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:03.657912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:03.657980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:03.658106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:03.658166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:03.658230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:03.658269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:03.658272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:03.658340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:03.658627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:03.658705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:03.658767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:03.658813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:03.658817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:03.658882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:03.658954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:03.659017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:03.659079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:03.659148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:03.659151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:03.659222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:03.659278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:03.659336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:03.659392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:03.659452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:03.659455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:03.659519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:03.659591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:03.659655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:03.659716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:03.659777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:03.659781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:03.659847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:03.659900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:03.659960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:03.660027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:03.660144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:03.660149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:03.660721Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:03.660793Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340796684048850:2476]
2025-10-09T21:43:03.660824Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:03.660827Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:03.660857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 46 reason: error at write: Disconnected sessionId: src_id|7b582a4f-176439f-95d44d83-5cb8b31b_0
2025-10-09T21:43:03.661041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 46 sessionId: src_id|7b582a4f-176439f-95d44d83-5cb8b31b_0 is DEAD
2025-10-09T21:43:03.661531Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b582a4f-176439f-95d44d83-5cb8b31b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:03.661541Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b582a4f-176439f-95d44d83-5cb8b31b_0] Write session will restart in 0.006662s
2025-10-09T21:43:03.661566Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b582a4f-176439f-95d44d83-5cb8b31b_0] Write session: Do CDS request
2025-10-09T21:43:03.661573Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b582a4f-176439f-95d44d83-5cb8b31b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:03.662524Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:03.669859Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b582a4f-176439f-95d44d83-5cb8b31b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:03.669985Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:03.670001Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 47
2025-10-09T21:43:03.670263Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 47 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:03.670328Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 47 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:03.670339Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=47 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:03.670344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:03.672253Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 47 sessionId:  describe result for acl check
2025-10-09T21:43:03.672296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:03.672298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:03.672301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:03.672308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340796684048867:2771] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:03.672904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340796684048867:2771] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:03.693408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:03.694488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:03.694721Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:03.694725Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:03.694737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:03.694794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:03.694834Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340796322240564:2640]
2025-10-09T21:43:03.695470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:03.696746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340796684048867:2771] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:03.696100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:03.696162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:03.696519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:03.696583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:03.696584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:03.696585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:03.696586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:03.696587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:03.696589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:03.696589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.696591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.696592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.696599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:03.696610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:03.696613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:03.696617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:03.696620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:03.696623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:03.696626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.696630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.696633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.696640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:03.696648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:03.696654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:03.696658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:03.696662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:03.696667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:03.696670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:03.696675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:03.696679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:03.696683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:03.696687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:03.696692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:03.696697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.696702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.696705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.696710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.696715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.696719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.696753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:03.717386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340796684048867:2771] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:03.717409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340796684048867:2771] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:03.717413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340796684048867:2771] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:03.717423Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 47 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:03.720986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:03.723121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:03.723139Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:03.723141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:03.723146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:03.723164Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 46 [2:7559340796322240564:2640]
2025-10-09T21:43:03.723178Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:03.723198Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:03.723202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.723205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.723207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.723229Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:03.723236Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:03.723243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:03.723312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.723314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.723316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.723319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:03.723397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:03.723401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.723415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.723427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.723443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:03.723464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340796322240558:2640]
2025-10-09T21:43:03.723466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340796322240558:2640]
2025-10-09T21:43:03.723468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340796322240558:2640]
2025-10-09T21:43:03.723483Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:03.723497Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:03.723512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:03.723515Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:03.723559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340796322240558:2640]
2025-10-09T21:43:03.723562Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:03.723777Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:03.723782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:03.723784Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:03.723790Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:03.724341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340796684048957:2771], now have 1 active actors on pipe
2025-10-09T21:43:03.724349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340796684048853:2432], now have 1 active actors on pipe
2025-10-09T21:43:03.724642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:03.724654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:03.724687Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2c37f16a-27719d45-255c5924-3706fa6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:03.724697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.724702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.724704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:03.724716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.724730Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:03.724748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:03.725446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:03.725453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:03.725469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:03.724560Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 46
2025-10-09T21:43:03.724591Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 46
2025-10-09T21:43:03.733290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b582a4f-176439f-95d44d83-5cb8b31b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2c37f16a-27719d45-255c5924-3706fa6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:03.731432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 47 partition: 0 MaxSeqNo: 25 sessionId: src_id|2c37f16a-27719d45-255c5924-3706fa6_0
2025-10-09T21:43:03.736522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c37f16a-27719d45-255c5924-3706fa6_0] Write session: try to update token
2025-10-09T21:43:03.736549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c37f16a-27719d45-255c5924-3706fa6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:03.760167Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:03.760187Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:03.760190Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:03.760192Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:03.760203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340796322240558:2640]
2025-10-09T21:43:03.760209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340796322240558:2640]
2025-10-09T21:43:03.760214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340796322240558:2640]
2025-10-09T21:43:03.760263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:03.760271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.760274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:03.760418Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:03.760504Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 47 sessionId: src_id|2c37f16a-27719d45-255c5924-3706fa6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:03.763027Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:03.763104Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340796684048982:2476]
2025-10-09T21:43:03.763427Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:03.763441Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:03.763445Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:03.764024Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 46, pipe: [1:7559340796684048982:2476]
2025-10-09T21:43:03.927233Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:03.927559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:03.927569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:03.927594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340796684048982:2476], now have 1 active actors on pipe
2025-10-09T21:43:03.928272Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:03.928285Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:03.928349Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 47 reason: error at write: Disconnected sessionId: src_id|2c37f16a-27719d45-255c5924-3706fa6_0
2025-10-09T21:43:03.928532Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 47 sessionId: src_id|2c37f16a-27719d45-255c5924-3706fa6_0 is DEAD
2025-10-09T21:43:03.928706Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:03.928932Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c37f16a-27719d45-255c5924-3706fa6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:03.928943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c37f16a-27719d45-255c5924-3706fa6_0] Write session will restart in 0.006082s
2025-10-09T21:43:03.928968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c37f16a-27719d45-255c5924-3706fa6_0] Write session: Do CDS request
2025-10-09T21:43:03.928975Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c37f16a-27719d45-255c5924-3706fa6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:03.927998Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:03.928742Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340796684048982:2476]
2025-10-09T21:43:03.936628Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c37f16a-27719d45-255c5924-3706fa6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:03.936711Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:03.936724Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 48
2025-10-09T21:43:03.936896Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 48 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:03.936930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 48 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:03.936936Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=48 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:03.936940Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:03.937488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 48 sessionId:  describe result for acl check
2025-10-09T21:43:03.937527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:03.937534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:03.937536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:03.937542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340796684049009:2784] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:03.938147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340796684049009:2784] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:03.940986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340796684049009:2784] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:03.944748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:03.944718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340796684049009:2784] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:03.944731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340796684049009:2784] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:03.944735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340796684049009:2784] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:03.944744Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 48 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:03.945482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:03.945689Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:03.945697Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:03.945709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:03.945775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:03.945816Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340796322240695:2655]
2025-10-09T21:43:03.946411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:03.946933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:03.946990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:03.947359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:03.947421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:03.947422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:03.947423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:03.947425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:03.947426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:03.947427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:03.947428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.947430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.947431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.947437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:03.947456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:03.947460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:03.947464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:03.947468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:03.947473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:03.947476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.947481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.947485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.947493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:03.947502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:03.947508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:03.947514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:03.947519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:03.947525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:03.947530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:03.947538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:03.947543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:03.947549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:03.947553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:03.947559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:03.947566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.947572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.947577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.947582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.947589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.947594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.947626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:03.955064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:03.956326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:03.956334Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:03.956339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:03.956344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:03.956354Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 47 [2:7559340796322240695:2655]
2025-10-09T21:43:03.956372Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:03.956395Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:03.956399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.956403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.956405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.956426Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:03.956432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:03.956441Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:03.956534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.956537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.956539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.956542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:03.956638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:03.956641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.956655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:03.956668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:03.956688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:03.956715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340796684048997:2432], now have 1 active actors on pipe
2025-10-09T21:43:03.956719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340796684049092:2784], now have 1 active actors on pipe
2025-10-09T21:43:03.956728Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340796322240688:2655]
2025-10-09T21:43:03.956731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340796322240688:2655]
2025-10-09T21:43:03.956733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340796322240688:2655]
2025-10-09T21:43:03.956750Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:03.956764Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:03.956767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:03.956768Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:03.956771Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:03.956785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:03.956789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:03.956833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340796322240688:2655]
2025-10-09T21:43:03.956843Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:03.956849Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:03.957322Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 47
2025-10-09T21:43:03.957362Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 47
2025-10-09T21:43:03.957970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:03.958001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:03.958058Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e4a401a3-acd7fe43-e214c840-6bcb6a62_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:03.958070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.958075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.958079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:03.958093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.958108Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:03.958129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:03.958683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:03.958695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:03.958716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:03.958975Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 48 partition: 0 MaxSeqNo: 25 sessionId: src_id|e4a401a3-acd7fe43-e214c840-6bcb6a62_0
2025-10-09T21:43:03.959448Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c37f16a-27719d45-255c5924-3706fa6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e4a401a3-acd7fe43-e214c840-6bcb6a62_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:03.962680Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4a401a3-acd7fe43-e214c840-6bcb6a62_0] Write session: try to update token
2025-10-09T21:43:03.962705Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4a401a3-acd7fe43-e214c840-6bcb6a62_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:03.978050Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:03.978067Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:03.978071Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:03.978072Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:03.978084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340796322240688:2655]
2025-10-09T21:43:03.978089Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340796322240688:2655]
2025-10-09T21:43:03.978094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340796322240688:2655]
2025-10-09T21:43:03.978215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:03.978229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:03.978231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:03.978367Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:03.981948Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:03.981965Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:03.981969Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:03.998582Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 48 sessionId: src_id|e4a401a3-acd7fe43-e214c840-6bcb6a62_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:04.001381Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:04.029153Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340800979016401:2476]
2025-10-09T21:43:04.029938Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 47, pipe: [1:7559340800979016401:2476]
2025-10-09T21:43:04.151904Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:04.152738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:04.152754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:04.152778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340800979016401:2476], now have 1 active actors on pipe
2025-10-09T21:43:04.153025Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:04.153073Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:04.153142Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340800979016401:2476]
2025-10-09T21:43:04.153162Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:04.153165Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:04.153220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 48 reason: error at write: Disconnected sessionId: src_id|e4a401a3-acd7fe43-e214c840-6bcb6a62_0
2025-10-09T21:43:04.153638Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4a401a3-acd7fe43-e214c840-6bcb6a62_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:04.153651Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4a401a3-acd7fe43-e214c840-6bcb6a62_0] Write session will restart in 0.007061s
2025-10-09T21:43:04.153678Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4a401a3-acd7fe43-e214c840-6bcb6a62_0] Write session: Do CDS request
2025-10-09T21:43:04.153687Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4a401a3-acd7fe43-e214c840-6bcb6a62_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:04.153677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 48 sessionId: src_id|e4a401a3-acd7fe43-e214c840-6bcb6a62_0 is DEAD
2025-10-09T21:43:04.161628Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4a401a3-acd7fe43-e214c840-6bcb6a62_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:04.162136Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:04.162152Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 49
2025-10-09T21:43:04.162398Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 49 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:04.162448Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 49 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:04.162455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=49 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:04.162462Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:04.163026Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 49 sessionId:  describe result for acl check
2025-10-09T21:43:04.163065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:04.163067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:04.163069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:04.163077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340800979016411:2788] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:04.163737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340800979016411:2788] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:04.169165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340800979016411:2788] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:04.175623Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340800979016411:2788] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:04.175643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340800979016411:2788] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:04.175647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340800979016411:2788] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:04.175657Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 49 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:04.182541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:04.183865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:04.184096Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:04.184100Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:04.184112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:04.184177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:04.184214Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340800617208096:2657]
2025-10-09T21:43:04.184849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:04.185624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:04.185682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:04.186051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:04.186117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:04.186118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:04.186119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:04.186121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:04.186122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:04.186123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:04.186125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.186127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.186128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.186135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:04.186145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:04.186149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:04.186154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:04.186158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:04.186164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:04.186168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.186172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.186176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.186185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:04.186194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:04.186200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:04.186206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:04.186210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:04.186219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:04.186224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:04.186230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:04.186234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:04.186238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:04.186242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:04.186247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:04.186252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.186257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.186261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.186266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.186271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.186276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.186318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:04.196353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:04.197639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:04.197646Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:04.197649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:04.197655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:04.197665Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 48 [2:7559340800617208096:2657]
2025-10-09T21:43:04.197683Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:04.197707Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:04.197712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.197715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.197718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.197740Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:04.197747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:04.197755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:04.197829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.197831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.197834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.197837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:04.197922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:04.197926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.197941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.197954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.197970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:04.197992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340800617208090:2657]
2025-10-09T21:43:04.197995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340800617208090:2657]
2025-10-09T21:43:04.197998Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340800617208090:2657]
2025-10-09T21:43:04.198007Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:04.198025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:04.198061Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:04.198065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:04.198067Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:04.198070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:04.198074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:04.198087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340800979016497:2788], now have 1 active actors on pipe
2025-10-09T21:43:04.198094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340800979016404:2432], now have 1 active actors on pipe
2025-10-09T21:43:04.198140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340800617208090:2657]
2025-10-09T21:43:04.198150Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:04.198228Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:04.201287Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 48
2025-10-09T21:43:04.201338Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 48
2025-10-09T21:43:04.202245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:04.202266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:04.202316Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d7a21d16-4d09e0b6-df26d659-8661e822_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:04.202330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.202336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.202340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:04.202355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.202374Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:04.202399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:04.205838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:04.205868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:04.205916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:04.206327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 49 partition: 0 MaxSeqNo: 25 sessionId: src_id|d7a21d16-4d09e0b6-df26d659-8661e822_0
2025-10-09T21:43:04.206812Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4a401a3-acd7fe43-e214c840-6bcb6a62_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d7a21d16-4d09e0b6-df26d659-8661e822_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:04.210745Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7a21d16-4d09e0b6-df26d659-8661e822_0] Write session: try to update token
2025-10-09T21:43:04.210773Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7a21d16-4d09e0b6-df26d659-8661e822_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:04.231278Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:04.231297Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:04.231300Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:04.231302Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:04.231313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340800617208090:2657]
2025-10-09T21:43:04.231319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340800617208090:2657]
2025-10-09T21:43:04.231324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340800617208090:2657]
2025-10-09T21:43:04.231373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:04.231380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.231383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:04.231534Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:04.235062Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:04.235075Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:04.235090Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:04.243438Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 49 sessionId: src_id|d7a21d16-4d09e0b6-df26d659-8661e822_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:04.246567Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:04.254682Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340800979016514:2476]
2025-10-09T21:43:04.255033Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 48, pipe: [1:7559340800979016514:2476]
2025-10-09T21:43:04.397830Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:04.398886Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:04.398901Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:04.398971Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 49 reason: error at write: Disconnected sessionId: src_id|d7a21d16-4d09e0b6-df26d659-8661e822_0
2025-10-09T21:43:04.399155Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 49 sessionId: src_id|d7a21d16-4d09e0b6-df26d659-8661e822_0 is DEAD
2025-10-09T21:43:04.399212Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340800979016514:2476]
2025-10-09T21:43:04.399267Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:04.399359Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7a21d16-4d09e0b6-df26d659-8661e822_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:04.399375Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7a21d16-4d09e0b6-df26d659-8661e822_0] Write session will restart in 0.008306s
2025-10-09T21:43:04.399398Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7a21d16-4d09e0b6-df26d659-8661e822_0] Write session: Do CDS request
2025-10-09T21:43:04.399405Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7a21d16-4d09e0b6-df26d659-8661e822_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:04.398461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:04.398478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:04.398527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340800979016514:2476], now have 1 active actors on pipe
2025-10-09T21:43:04.398793Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:04.410506Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7a21d16-4d09e0b6-df26d659-8661e822_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:04.411037Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:04.411066Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 50
2025-10-09T21:43:04.411257Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 50 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:04.411292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 50 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:04.411303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=50 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:04.411308Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:04.411979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 50 sessionId:  describe result for acl check
2025-10-09T21:43:04.412016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:04.412018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:04.412021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:04.412028Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340800979016568:2793] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:04.412621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340800979016568:2793] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:04.415819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340800979016568:2793] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:04.415821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:04.416802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:04.417012Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:04.417022Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:04.417033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:04.417089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:04.417134Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340800617208199:2659]
2025-10-09T21:43:04.417796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:04.418246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:04.418303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:04.418694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:04.418764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:04.418768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:04.418769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:04.418771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:04.418772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:04.418774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:04.418775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.418776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.418778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.418783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:04.418793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:04.418797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:04.418801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:04.418805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:04.418809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:04.418813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.418816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.418819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.418827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:04.418836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:04.418842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:04.418848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:04.418853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:04.418858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:04.418863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:04.418868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:04.418873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:04.418878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:04.418883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:04.418889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:04.418896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.418901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.418907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.418911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.418923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.418928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.418958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:04.420835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340800979016568:2793] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:04.420851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340800979016568:2793] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:04.420856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340800979016568:2793] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:04.420866Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 50 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:04.426101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:04.427232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:04.427245Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:04.427249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:04.427254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:04.427263Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 49 [2:7559340800617208199:2659]
2025-10-09T21:43:04.427280Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:04.427303Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:04.427317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.427321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.427326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.427345Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:04.427351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:04.427359Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:04.427417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.427424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.427427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.427430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:04.427501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:04.427509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.427524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.427541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.427560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:04.427586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340800617208194:2659]
2025-10-09T21:43:04.427593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340800617208194:2659]
2025-10-09T21:43:04.427595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340800617208194:2659]
2025-10-09T21:43:04.427610Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:04.427633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340800979016622:2793], now have 1 active actors on pipe
2025-10-09T21:43:04.427632Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:04.427636Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:04.427639Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:04.427646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340800979016519:2432], now have 1 active actors on pipe
2025-10-09T21:43:04.427651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:04.427666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:04.427672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:04.427715Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340800617208194:2659]
2025-10-09T21:43:04.427729Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:04.427715Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 49
2025-10-09T21:43:04.427751Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:04.427716Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 49
2025-10-09T21:43:04.427982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:04.427997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:04.428026Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cdb2d541-3bfaaf7a-525e45fa-9524f030_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:04.428044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.428048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.428051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:04.428062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.428077Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:04.428094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:04.429123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:04.429138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:04.429158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:04.429315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 50 partition: 0 MaxSeqNo: 25 sessionId: src_id|cdb2d541-3bfaaf7a-525e45fa-9524f030_0
2025-10-09T21:43:04.431105Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7a21d16-4d09e0b6-df26d659-8661e822_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cdb2d541-3bfaaf7a-525e45fa-9524f030_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:04.434498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cdb2d541-3bfaaf7a-525e45fa-9524f030_0] Write session: try to update token
2025-10-09T21:43:04.434522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cdb2d541-3bfaaf7a-525e45fa-9524f030_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:04.447875Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:04.447898Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:04.447900Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:04.447902Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:04.447917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340800617208194:2659]
2025-10-09T21:43:04.447922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340800617208194:2659]
2025-10-09T21:43:04.447926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340800617208194:2659]
2025-10-09T21:43:04.447972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:04.447981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.447983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:04.447998Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:04.448006Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:04.448010Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:04.448148Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:04.467131Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 50 sessionId: src_id|cdb2d541-3bfaaf7a-525e45fa-9524f030_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:04.469682Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:04.501380Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340800979016631:2476]
2025-10-09T21:43:04.501897Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 49, pipe: [1:7559340800979016631:2476]
2025-10-09T21:43:04.610944Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:04.611402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:04.611420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:04.611444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340800979016631:2476], now have 1 active actors on pipe
2025-10-09T21:43:04.611753Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:04.612842Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cdb2d541-3bfaaf7a-525e45fa-9524f030_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:04.612857Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cdb2d541-3bfaaf7a-525e45fa-9524f030_0] Write session will restart in 0.009588s
2025-10-09T21:43:04.612320Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:04.612363Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:04.612449Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 50 reason: status is not ok: tablet will be restarted right now sessionId: src_id|cdb2d541-3bfaaf7a-525e45fa-9524f030_0
2025-10-09T21:43:04.612636Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 50 sessionId: src_id|cdb2d541-3bfaaf7a-525e45fa-9524f030_0 is DEAD
2025-10-09T21:43:04.612786Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340800979016631:2476]
2025-10-09T21:43:04.612895Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cdb2d541-3bfaaf7a-525e45fa-9524f030_0] Write session: Do CDS request
2025-10-09T21:43:04.612903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cdb2d541-3bfaaf7a-525e45fa-9524f030_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:04.623606Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cdb2d541-3bfaaf7a-525e45fa-9524f030_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:04.623743Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:04.623760Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 51
2025-10-09T21:43:04.624013Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 51 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:04.624067Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 51 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:04.624079Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=51 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:04.624084Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:04.624532Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 51 sessionId:  describe result for acl check
2025-10-09T21:43:04.624571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:04.624578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:04.624580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:04.624586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340800979016698:2797] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:04.625151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340800979016698:2797] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:04.631477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:04.632505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340800979016698:2797] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:04.635924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:04.636195Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:04.636206Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:04.636223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:04.636297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:04.636342Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340800617208307:2661]
2025-10-09T21:43:04.636906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340800979016698:2797] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:04.636925Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340800979016698:2797] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:04.636929Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340800979016698:2797] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:04.636939Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 51 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:04.636960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:04.637603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:04.637687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:04.638096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:04.638183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:04.638194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:04.638195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:04.638196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:04.638198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:04.638200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:04.638213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.638214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.638217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.638224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:04.638234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:04.638238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:04.638242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:04.638246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:04.638249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:04.638254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.638257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.638261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.638269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:04.638277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:04.638286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:04.638291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:04.638296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:04.638301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:04.638305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:04.638310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:04.638315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:04.638320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:04.638326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:04.638332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:04.638338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.638343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.638349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.638355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.638360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.638375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.638431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:04.649126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:04.651602Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 50
2025-10-09T21:43:04.651657Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 50
2025-10-09T21:43:04.650484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:04.650509Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:04.650513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:04.650517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:04.650527Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 50 [2:7559340800617208307:2661]
2025-10-09T21:43:04.650545Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:04.650565Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:04.650568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.650572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.650574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.650592Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:04.650598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:04.650606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:04.650679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.650681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.650683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.650686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:04.650770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:04.650773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.650787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.650796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.650811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:04.650831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340800617208301:2661]
2025-10-09T21:43:04.650837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340800617208301:2661]
2025-10-09T21:43:04.650839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340800617208301:2661]
2025-10-09T21:43:04.650854Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:04.650869Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:04.650895Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:04.650899Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:04.650938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340800617208301:2661]
2025-10-09T21:43:04.650941Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:04.651045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:04.651047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:04.651050Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:04.651055Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:04.656416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340800979016634:2432], now have 1 active actors on pipe
2025-10-09T21:43:04.656429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340800979016734:2797], now have 1 active actors on pipe
2025-10-09T21:43:04.656449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:04.656457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:04.656491Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|682a9ff1-d85dcec5-2fe50610-cb527586_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:04.656502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.656508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.656511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:04.656528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.656546Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:04.656566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:04.661083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:04.661105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:04.661139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:04.661410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 51 partition: 0 MaxSeqNo: 25 sessionId: src_id|682a9ff1-d85dcec5-2fe50610-cb527586_0
2025-10-09T21:43:04.661890Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cdb2d541-3bfaaf7a-525e45fa-9524f030_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|682a9ff1-d85dcec5-2fe50610-cb527586_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:04.664893Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|682a9ff1-d85dcec5-2fe50610-cb527586_0] Write session: try to update token
2025-10-09T21:43:04.664914Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|682a9ff1-d85dcec5-2fe50610-cb527586_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:04.667406Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:04.667427Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:04.667430Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:04.667432Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:04.667441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340800617208301:2661]
2025-10-09T21:43:04.667445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340800617208301:2661]
2025-10-09T21:43:04.667453Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340800617208301:2661]
2025-10-09T21:43:04.667497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:04.667505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.667508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:04.667654Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:04.670130Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:04.670141Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:04.670145Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:04.692133Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 51 sessionId: src_id|682a9ff1-d85dcec5-2fe50610-cb527586_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:04.695282Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:04.713192Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340800979016747:2476]
2025-10-09T21:43:04.715893Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 50, pipe: [1:7559340800979016747:2476]
2025-10-09T21:43:04.835963Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:04.836294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:04.836315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:04.836339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340800979016747:2476], now have 1 active actors on pipe
2025-10-09T21:43:04.836583Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:04.836702Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:04.836720Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:04.836798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 51 reason: error at write: Disconnected sessionId: src_id|682a9ff1-d85dcec5-2fe50610-cb527586_0
2025-10-09T21:43:04.837014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 51 sessionId: src_id|682a9ff1-d85dcec5-2fe50610-cb527586_0 is DEAD
2025-10-09T21:43:04.837233Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|682a9ff1-d85dcec5-2fe50610-cb527586_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:04.837241Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|682a9ff1-d85dcec5-2fe50610-cb527586_0] Write session will restart in 0.007752s
2025-10-09T21:43:04.837232Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:04.837265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|682a9ff1-d85dcec5-2fe50610-cb527586_0] Write session: Do CDS request
2025-10-09T21:43:04.837270Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|682a9ff1-d85dcec5-2fe50610-cb527586_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:04.837273Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340800979016747:2476]
2025-10-09T21:43:04.846573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|682a9ff1-d85dcec5-2fe50610-cb527586_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:04.846666Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:04.846685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 52
2025-10-09T21:43:04.846912Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 52 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:04.846968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 52 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:04.846974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=52 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:04.846980Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:04.847446Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 52 sessionId:  describe result for acl check
2025-10-09T21:43:04.847483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:04.847485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:04.847487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:04.847493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340800979016832:2809] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:04.848082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340800979016832:2809] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:04.851180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340800979016832:2809] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:04.856189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:04.856208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340800979016832:2809] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:04.856227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340800979016832:2809] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:04.856231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340800979016832:2809] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:04.856241Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 52 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:04.860936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:04.861143Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:04.861153Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:04.861164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:04.861222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:04.861292Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340800617208456:2675]
2025-10-09T21:43:04.861901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:04.862222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:04.862272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:04.862599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:04.862657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:04.862662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:04.862663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:04.862664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:04.862666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:04.862668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:04.862669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.862670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.862671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.862677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:04.862689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:04.862693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:04.862698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:04.862701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:04.862705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:04.862708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.862711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.862715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.862722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:04.862730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:04.862735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:04.862741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:04.862746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:04.862752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:04.862760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:04.862766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:04.862771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:04.862778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:04.862787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:04.862794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:04.862802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.862808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.862819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.862829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.862834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.862841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.862876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:04.870559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:04.872774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:04.872794Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:04.872798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:04.872804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:04.872815Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 51 [2:7559340800617208456:2675]
2025-10-09T21:43:04.872833Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:04.872857Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:04.872870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.872875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.872878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.872899Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:04.872906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:04.872914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:04.872996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.873008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.873011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.873014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:04.873513Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 51
2025-10-09T21:43:04.873558Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 51
2025-10-09T21:43:04.873103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:04.873107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.873122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:04.873136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:04.873158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:04.873179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340800617208442:2675]
2025-10-09T21:43:04.873182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340800617208442:2675]
2025-10-09T21:43:04.873184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340800617208442:2675]
2025-10-09T21:43:04.873194Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:04.873210Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:04.873227Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:04.873231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:04.873237Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:04.873241Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:04.873261Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:04.873286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340800979016781:2432], now have 1 active actors on pipe
2025-10-09T21:43:04.873290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340800617208442:2675]
2025-10-09T21:43:04.873304Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:04.873311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340800979016878:2809], now have 1 active actors on pipe
2025-10-09T21:43:04.873330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:04.873614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:04.873623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:04.873657Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|85427707-3cfa6a4-a297d13f-c2e0c6cd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:04.873667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.873672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.873674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:04.873688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.873712Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:04.873732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:04.874019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:04.874052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:04.874086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:04.874220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 52 partition: 0 MaxSeqNo: 25 sessionId: src_id|85427707-3cfa6a4-a297d13f-c2e0c6cd_0
2025-10-09T21:43:04.874888Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|682a9ff1-d85dcec5-2fe50610-cb527586_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|85427707-3cfa6a4-a297d13f-c2e0c6cd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:04.890802Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:04.890819Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:04.890830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340800617208442:2675]
2025-10-09T21:43:04.890885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:04.891280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:04.891440Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:04.891595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:04.891942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:04.892156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:04.892202Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:04.892396Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:04.904449Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85427707-3cfa6a4-a297d13f-c2e0c6cd_0] Write session: try to update token
2025-10-09T21:43:04.904479Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85427707-3cfa6a4-a297d13f-c2e0c6cd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:04.912276Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:04.912298Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:04.912301Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:04.912303Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:04.912319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340800617208442:2675]
2025-10-09T21:43:04.912324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340800617208442:2675]
2025-10-09T21:43:04.912332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340800617208442:2675]
2025-10-09T21:43:04.912384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:04.912391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:04.912393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:04.912560Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:04.913362Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:04.913373Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:04.913378Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:04.937456Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340800979016901:2476]
2025-10-09T21:43:04.937945Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 51, pipe: [1:7559340800979016901:2476]
2025-10-09T21:43:04.962782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 52 sessionId: src_id|85427707-3cfa6a4-a297d13f-c2e0c6cd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:04.965227Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:05.081791Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:05.082136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340800979016901:2476], now have 1 active actors on pipe
2025-10-09T21:43:05.082171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:05.082180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:05.082640Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:05.083478Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340800979016901:2476]
2025-10-09T21:43:05.083505Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:05.083540Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:05.083635Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 52 reason: status is not ok: tablet will be restarted right now sessionId: src_id|85427707-3cfa6a4-a297d13f-c2e0c6cd_0
2025-10-09T21:43:05.083820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 52 sessionId: src_id|85427707-3cfa6a4-a297d13f-c2e0c6cd_0 is DEAD
2025-10-09T21:43:05.084097Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85427707-3cfa6a4-a297d13f-c2e0c6cd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:05.084119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85427707-3cfa6a4-a297d13f-c2e0c6cd_0] Write session will restart in 0.007562s
2025-10-09T21:43:05.084151Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85427707-3cfa6a4-a297d13f-c2e0c6cd_0] Write session: Do CDS request
2025-10-09T21:43:05.084174Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85427707-3cfa6a4-a297d13f-c2e0c6cd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:05.092599Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85427707-3cfa6a4-a297d13f-c2e0c6cd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:05.093032Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:05.093051Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 53
2025-10-09T21:43:05.093437Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 53 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:05.093476Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 53 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:05.093483Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=53 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:05.093488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:05.094063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 53 sessionId:  describe result for acl check
2025-10-09T21:43:05.094105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:05.094107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:05.094109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:05.094115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340805273984264:2814] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:05.094938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340805273984264:2814] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:05.096490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:05.097934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:05.098159Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:05.098169Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:05.098182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:05.098234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:05.098280Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340804912175858:2679]
2025-10-09T21:43:05.098288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340805273984264:2814] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:05.099117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:05.102131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340805273984264:2814] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:05.102151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340805273984264:2814] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:05.102156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340805273984264:2814] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:05.102166Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 53 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:05.103140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:05.103244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:05.103676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:05.103750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:05.103751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:05.103753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:05.103754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:05.103756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:05.103758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:05.103759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.103760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.103762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.103768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:05.103780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:05.103784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:05.103789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:05.103793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:05.103797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:05.103801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.103805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.103810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.103817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:05.103826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:05.103833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:05.103839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:05.103845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:05.103851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:05.103855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:05.103862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:05.103867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:05.103873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:05.103879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:05.103886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:05.103893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.103899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.103904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.103909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.103914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.103920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.103957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:05.113727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:05.115604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:05.115632Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:05.115636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:05.115643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:05.115653Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 52 [2:7559340804912175858:2679]
2025-10-09T21:43:05.115671Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:05.115696Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:05.115700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.115704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.115706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.116241Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 52
2025-10-09T21:43:05.116244Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 52
2025-10-09T21:43:05.115729Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:05.115748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:05.115757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:05.115849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.115852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.115855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.115858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:05.115953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:05.115957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.115975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.115989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.116009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:05.116034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340804912175852:2679]
2025-10-09T21:43:05.116037Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340804912175852:2679]
2025-10-09T21:43:05.116040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340804912175852:2679]
2025-10-09T21:43:05.116052Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:05.116065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:05.116069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:05.116072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:05.116072Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:05.116089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:05.116094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:05.116109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340805273984303:2814], now have 1 active actors on pipe
2025-10-09T21:43:05.116113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340805273984203:2432], now have 1 active actors on pipe
2025-10-09T21:43:05.116161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340804912175852:2679]
2025-10-09T21:43:05.116175Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:05.116204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:05.116738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:05.116749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:05.116786Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bf3b0a82-f9fd8cef-2687e279-ff12651e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:05.116797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.116803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.116807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:05.116818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.116852Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:05.116879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:05.117630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:05.117648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:05.117670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:05.118532Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85427707-3cfa6a4-a297d13f-c2e0c6cd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bf3b0a82-f9fd8cef-2687e279-ff12651e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:05.121788Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf3b0a82-f9fd8cef-2687e279-ff12651e_0] Write session: try to update token
2025-10-09T21:43:05.121808Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf3b0a82-f9fd8cef-2687e279-ff12651e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:05.118166Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 53 partition: 0 MaxSeqNo: 25 sessionId: src_id|bf3b0a82-f9fd8cef-2687e279-ff12651e_0
2025-10-09T21:43:05.136029Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:05.136047Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:05.136060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340804912175852:2679]
2025-10-09T21:43:05.136118Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:05.136153Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:05.136475Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:05.136758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:05.137010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:05.137323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:05.138571Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:05.139438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:05.161422Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:05.161447Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:05.161452Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:05.161468Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:05.161569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340804912175852:2679]
2025-10-09T21:43:05.161589Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340804912175852:2679]
2025-10-09T21:43:05.161608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340804912175852:2679]
2025-10-09T21:43:05.161685Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:05.161704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:05.161709Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:05.161735Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:05.161737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.161759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:05.161998Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:05.162113Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 53 sessionId: src_id|bf3b0a82-f9fd8cef-2687e279-ff12651e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:05.165068Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:05.184624Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340805273984315:2476]
2025-10-09T21:43:05.190497Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 52, pipe: [1:7559340805273984315:2476]
2025-10-09T21:43:05.376918Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:05.378313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:05.378332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:05.378364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340805273984315:2476], now have 1 active actors on pipe
2025-10-09T21:43:05.379718Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:05.379732Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:05.379827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 53 reason: error at write: Disconnected sessionId: src_id|bf3b0a82-f9fd8cef-2687e279-ff12651e_0
2025-10-09T21:43:05.380032Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 53 sessionId: src_id|bf3b0a82-f9fd8cef-2687e279-ff12651e_0 is DEAD
2025-10-09T21:43:05.380047Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:05.380152Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340805273984315:2476]
2025-10-09T21:43:05.380736Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf3b0a82-f9fd8cef-2687e279-ff12651e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:05.380753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf3b0a82-f9fd8cef-2687e279-ff12651e_0] Write session will restart in 0.009841s
2025-10-09T21:43:05.380803Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf3b0a82-f9fd8cef-2687e279-ff12651e_0] Write session: Do CDS request
2025-10-09T21:43:05.380812Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf3b0a82-f9fd8cef-2687e279-ff12651e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:05.385693Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:05.391719Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf3b0a82-f9fd8cef-2687e279-ff12651e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:05.394253Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:05.394270Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 54
2025-10-09T21:43:05.394502Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 54 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:05.394536Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 54 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:05.394544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=54 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:05.394549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:05.398996Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 54 sessionId:  describe result for acl check
2025-10-09T21:43:05.399054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:05.399056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:05.399063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:05.399070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340805273984328:2819] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:05.399841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340805273984328:2819] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:05.408639Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340805273984328:2819] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:05.417069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340805273984328:2819] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:05.417087Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340805273984328:2819] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:05.417094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340805273984328:2819] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:05.417104Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 54 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:05.427471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:05.430221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:05.430606Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:05.430611Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:05.430626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:05.430699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:05.430755Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340804912175968:2681]
2025-10-09T21:43:05.431451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:05.436260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:05.436341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:05.436793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:05.436856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:05.436861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:05.436862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:05.436864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:05.436865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:05.436867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:05.436868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.436869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.436871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.436877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:05.436889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:05.436892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:05.436896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:05.436901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:05.436905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:05.436908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.436911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.436915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.436923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:05.436932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:05.436939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:05.436944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:05.436948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:05.436953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:05.436957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:05.436962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:05.436967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:05.436972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:05.436977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:05.436983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:05.436998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.437002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.437006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.437010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.437014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.437018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.437044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:05.445414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:05.448091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:05.448098Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:05.448101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:05.448107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:05.448116Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 53 [2:7559340804912175968:2681]
2025-10-09T21:43:05.448136Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:05.448157Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:05.448162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.448166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.448169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.448191Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:05.448198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:05.448207Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:05.448281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.448283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.448286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.448290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:05.448386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:05.448390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.448405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.448418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.448438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:05.448460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340804912175962:2681]
2025-10-09T21:43:05.448464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340804912175962:2681]
2025-10-09T21:43:05.448466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340804912175962:2681]
2025-10-09T21:43:05.448477Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:05.448494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:05.448498Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:05.448501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:05.448504Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:05.448510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:05.448514Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:05.448526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340805273984391:2819], now have 1 active actors on pipe
2025-10-09T21:43:05.448531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340805273984320:2432], now have 1 active actors on pipe
2025-10-09T21:43:05.448570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340804912175962:2681]
2025-10-09T21:43:05.448579Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:05.448584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:05.449120Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 53
2025-10-09T21:43:05.449162Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 53
2025-10-09T21:43:05.449467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:05.449480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:05.449524Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a2d4f157-b6380c0-b80e73c7-e826180a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:05.449534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.449540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.449543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:05.449557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.449569Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:05.449587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:05.455029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:05.455048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:05.455085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:05.455340Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 54 partition: 0 MaxSeqNo: 25 sessionId: src_id|a2d4f157-b6380c0-b80e73c7-e826180a_0
2025-10-09T21:43:05.455958Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf3b0a82-f9fd8cef-2687e279-ff12651e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a2d4f157-b6380c0-b80e73c7-e826180a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:05.465066Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a2d4f157-b6380c0-b80e73c7-e826180a_0] Write session: try to update token
2025-10-09T21:43:05.465090Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a2d4f157-b6380c0-b80e73c7-e826180a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:05.478979Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:05.478994Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:05.479007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340804912175962:2681]
2025-10-09T21:43:05.479057Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:05.479258Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:05.479591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:05.479860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:05.480062Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:05.480246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:05.480478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:05.480544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:05.484978Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340805273984436:2476]
2025-10-09T21:43:05.486298Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 53, pipe: [1:7559340805273984436:2476]
2025-10-09T21:43:05.487951Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 54 sessionId: src_id|a2d4f157-b6380c0-b80e73c7-e826180a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:05.489216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340805273984436:2476], now have 1 active actors on pipe
2025-10-09T21:43:05.490532Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:05.490807Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:05.490826Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:05.490829Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:05.490831Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:05.490842Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340804912175962:2681]
2025-10-09T21:43:05.490849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340804912175962:2681]
2025-10-09T21:43:05.490854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340804912175962:2681]
2025-10-09T21:43:05.490902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:05.490909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.490914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:05.491031Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:05.491045Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:05.491049Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:05.491075Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:05.777929Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:05.783127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:05.783146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:05.783659Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:05.783930Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:05.783960Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:05.783998Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:05.784023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 54 reason: error at write: Disconnected sessionId: src_id|a2d4f157-b6380c0-b80e73c7-e826180a_0
2025-10-09T21:43:05.784069Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340805273984436:2476]
2025-10-09T21:43:05.784220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 54 sessionId: src_id|a2d4f157-b6380c0-b80e73c7-e826180a_0 is DEAD
2025-10-09T21:43:05.784951Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2d4f157-b6380c0-b80e73c7-e826180a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:05.784966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2d4f157-b6380c0-b80e73c7-e826180a_0] Write session will restart in 0.006938s
2025-10-09T21:43:05.784992Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2d4f157-b6380c0-b80e73c7-e826180a_0] Write session: Do CDS request
2025-10-09T21:43:05.785001Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2d4f157-b6380c0-b80e73c7-e826180a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:05.793532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a2d4f157-b6380c0-b80e73c7-e826180a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:05.798547Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:05.798562Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 55
2025-10-09T21:43:05.798753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 55 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:05.798788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 55 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:05.798794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=55 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:05.798799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:05.803309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 55 sessionId:  describe result for acl check
2025-10-09T21:43:05.803366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:05.803368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:05.803369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:05.803375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340805273984457:2825] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:05.803924Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340805273984457:2825] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:05.812943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340805273984457:2825] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:05.830678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:05.835670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340805273984457:2825] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:05.835687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340805273984457:2825] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:05.835691Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340805273984457:2825] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:05.835701Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 55 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:05.835254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:05.835563Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:05.835567Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:05.835581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:05.835639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:05.835681Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340804912176096:2684]
2025-10-09T21:43:05.836307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:05.837168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:05.837216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:05.837562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:05.837609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:05.837610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:05.837612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:05.837613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:05.837614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:05.837615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:05.837617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.837618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.837619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.837624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:05.837634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:05.837638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:05.837643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:05.837647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:05.837651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:05.837668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.837672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.837676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.837683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:05.837690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:05.837697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:05.837704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:05.837709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:05.837713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:05.837717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:05.837722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:05.837727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:05.837733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:05.837737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:05.837742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:05.837747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.837752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.837756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.837761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.837766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.837771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.837799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:05.853371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:05.854685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:05.854692Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:05.854696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:05.854701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:05.854709Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 54 [2:7559340804912176096:2684]
2025-10-09T21:43:05.854725Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:05.854745Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:05.854749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.854754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.854757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.854779Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:05.854785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:05.854793Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:05.854857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.854859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.854862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.854865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:05.854947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:05.854950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.854964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:05.854977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:05.854995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:05.855016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340804912176090:2684]
2025-10-09T21:43:05.855019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340804912176090:2684]
2025-10-09T21:43:05.855021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340804912176090:2684]
2025-10-09T21:43:05.855034Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:05.855050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:05.855068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:05.855072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:05.855116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340804912176090:2684]
2025-10-09T21:43:05.855120Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:05.855459Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:05.855467Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:05.855470Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:05.855477Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:05.855876Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 54
2025-10-09T21:43:05.855910Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 54
2025-10-09T21:43:05.864993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340805273984558:2825], now have 1 active actors on pipe
2025-10-09T21:43:05.865016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340805273984448:2432], now have 1 active actors on pipe
2025-10-09T21:43:05.865042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:05.865049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:05.865091Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e8e0f210-642751c4-7f1a6e04-5808dc4c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:05.865124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.865134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.865138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:05.865150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.865169Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:05.865193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:05.865496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:05.865510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:05.865534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:05.870219Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 55 partition: 0 MaxSeqNo: 25 sessionId: src_id|e8e0f210-642751c4-7f1a6e04-5808dc4c_0
2025-10-09T21:43:05.875313Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2d4f157-b6380c0-b80e73c7-e826180a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e8e0f210-642751c4-7f1a6e04-5808dc4c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:05.883796Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340805273984581:2476]
2025-10-09T21:43:05.884101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340805273984581:2476], now have 1 active actors on pipe
2025-10-09T21:43:05.884180Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 54, pipe: [1:7559340805273984581:2476]
2025-10-09T21:43:05.884934Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8e0f210-642751c4-7f1a6e04-5808dc4c_0] Write session: try to update token
2025-10-09T21:43:05.884952Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8e0f210-642751c4-7f1a6e04-5808dc4c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:05.915813Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:05.915834Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:05.915836Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:05.915838Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:05.915847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340804912176090:2684]
2025-10-09T21:43:05.915850Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340804912176090:2684]
2025-10-09T21:43:05.915855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340804912176090:2684]
2025-10-09T21:43:05.915902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:05.915908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:05.915911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:05.915912Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:05.915919Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:05.915923Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:05.916057Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:05.941517Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 55 sessionId: src_id|e8e0f210-642751c4-7f1a6e04-5808dc4c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:05.947851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:06.113343Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:06.113783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:06.113796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:06.114182Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:06.115016Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340805273984581:2476]
2025-10-09T21:43:06.115034Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:06.115047Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:06.115143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 55 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e8e0f210-642751c4-7f1a6e04-5808dc4c_0
2025-10-09T21:43:06.115560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8e0f210-642751c4-7f1a6e04-5808dc4c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:06.115576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8e0f210-642751c4-7f1a6e04-5808dc4c_0] Write session will restart in 0.008829s
2025-10-09T21:43:06.115503Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 55 sessionId: src_id|e8e0f210-642751c4-7f1a6e04-5808dc4c_0 is DEAD
2025-10-09T21:43:06.115600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8e0f210-642751c4-7f1a6e04-5808dc4c_0] Write session: Do CDS request
2025-10-09T21:43:06.115607Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8e0f210-642751c4-7f1a6e04-5808dc4c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:06.125613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8e0f210-642751c4-7f1a6e04-5808dc4c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:06.126178Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:06.126208Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 56
2025-10-09T21:43:06.126422Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 56 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:06.126483Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 56 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:06.126491Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=56 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:06.126495Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:06.127719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 56 sessionId:  describe result for acl check
2025-10-09T21:43:06.127772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:06.127774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:06.127776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:06.127783Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340809568951907:2836] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:06.128469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340809568951907:2836] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:06.135367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340809568951907:2836] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:06.141709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340809568951907:2836] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:06.141734Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340809568951907:2836] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:06.141740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340809568951907:2836] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:06.141751Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 56 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:06.145686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:06.146346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:06.146570Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:06.146582Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:06.146594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:06.146646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:06.146691Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340809207143530:2699]
2025-10-09T21:43:06.147454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:06.148249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:06.148325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:06.148691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:06.148751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:06.148752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:06.148754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:06.148755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:06.148757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:06.148758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:06.148759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.148760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.148762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.148768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:06.148779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:06.148783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:06.148786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:06.148791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:06.148795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:06.148799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.148803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.148808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.148815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:06.148824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:06.148830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:06.148836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:06.148841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:06.148846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:06.148851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:06.148856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:06.148861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:06.148867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:06.148873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:06.148878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:06.148883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.148889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.148893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.148899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.148903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.148908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.148939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:06.163454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:06.164736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:06.164757Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:06.164760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:06.164765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:06.164773Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 55 [2:7559340809207143530:2699]
2025-10-09T21:43:06.164792Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:06.164816Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:06.164820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.164824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.164827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.164849Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:06.164866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:06.164874Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:06.164944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.164954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.164956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.164959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:06.165039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:06.165049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.165065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.165079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.165105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:06.165131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340809207143525:2699]
2025-10-09T21:43:06.165139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340809207143525:2699]
2025-10-09T21:43:06.165142Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340809207143525:2699]
2025-10-09T21:43:06.165158Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:06.165175Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:06.165193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:06.165201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:06.165222Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:06.165233Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:06.165238Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:06.165248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340809207143525:2699]
2025-10-09T21:43:06.165251Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:06.165348Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:06.165527Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 55
2025-10-09T21:43:06.165565Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 55
2025-10-09T21:43:06.166250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340809568951979:2836], now have 1 active actors on pipe
2025-10-09T21:43:06.166264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340809568951893:2432], now have 1 active actors on pipe
2025-10-09T21:43:06.166283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:06.166289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:06.166332Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|42ce9872-880ce37d-d313401a-f2f8a761_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:06.166351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.166358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.166361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:06.166375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.166390Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:06.166412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:06.166975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:06.167002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:06.167029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:06.167194Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 56 partition: 0 MaxSeqNo: 25 sessionId: src_id|42ce9872-880ce37d-d313401a-f2f8a761_0
2025-10-09T21:43:06.167621Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8e0f210-642751c4-7f1a6e04-5808dc4c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|42ce9872-880ce37d-d313401a-f2f8a761_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:06.171186Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42ce9872-880ce37d-d313401a-f2f8a761_0] Write session: try to update token
2025-10-09T21:43:06.171206Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42ce9872-880ce37d-d313401a-f2f8a761_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:06.191973Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 56 sessionId: src_id|42ce9872-880ce37d-d313401a-f2f8a761_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:06.194951Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:06.194940Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:06.194959Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:06.194969Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340809207143525:2699]
2025-10-09T21:43:06.195026Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:06.195039Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:06.195200Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:06.195718Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:06.195856Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:06.196050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:06.196328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:06.196373Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:06.196862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:06.196879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:06.196916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:06.197043Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:06.215661Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340809568952012:2476]
2025-10-09T21:43:06.218236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340809568952012:2476], now have 1 active actors on pipe
2025-10-09T21:43:06.221092Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 55, pipe: [1:7559340809568952012:2476]
2025-10-09T21:43:06.222825Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:06.222841Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:06.222844Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:06.222846Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:06.222856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340809207143525:2699]
2025-10-09T21:43:06.222860Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340809207143525:2699]
2025-10-09T21:43:06.222864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340809207143525:2699]
2025-10-09T21:43:06.222909Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:06.222916Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:06.222920Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:06.222922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:06.222930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.222933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:06.223093Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:06.396933Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:06.406159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:06.406186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:06.406395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:06.406487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:06.406566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:06.406633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:06.406679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:06.406684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:06.406754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:06.406810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:06.406865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:06.406931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:06.406970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:06.406972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:06.407043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:06.407101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:06.407162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:06.407275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:06.407327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:06.407330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:06.407391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:06.407604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:06.407803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:06.407895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:06.407982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:06.407986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:06.408092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:06.408239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:06.408352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:06.408432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:06.408488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:06.408491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:06.408559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:06.408620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:06.408678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:06.408787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:06.408844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:06.408847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:06.408912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:06.409007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:06.409099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:06.409245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:06.409330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:06.409333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:06.409446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:06.409547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:06.409635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:06.409686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:06.409749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:06.409752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:06.409840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:06.409895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:06.409952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:06.410011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:06.410352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:06.410361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:06.410437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:06.410500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:06.410569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:06.410639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:06.410705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:06.410713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:06.411634Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42ce9872-880ce37d-d313401a-f2f8a761_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:06.411648Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42ce9872-880ce37d-d313401a-f2f8a761_0] Write session will restart in 0.006057s
2025-10-09T21:43:06.411676Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42ce9872-880ce37d-d313401a-f2f8a761_0] Write session: Do CDS request
2025-10-09T21:43:06.411683Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42ce9872-880ce37d-d313401a-f2f8a761_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:06.411887Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:06.411219Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:06.411235Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:06.411270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 56 reason: error at write: Disconnected sessionId: src_id|42ce9872-880ce37d-d313401a-f2f8a761_0
2025-10-09T21:43:06.411489Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 56 sessionId: src_id|42ce9872-880ce37d-d313401a-f2f8a761_0 is DEAD
2025-10-09T21:43:06.411668Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:06.411700Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340809568952012:2476]
2025-10-09T21:43:06.418637Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42ce9872-880ce37d-d313401a-f2f8a761_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:06.421915Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:06.421935Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 57
2025-10-09T21:43:06.422320Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 57 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:06.422365Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 57 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:06.422372Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=57 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:06.422376Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:06.423543Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 57 sessionId:  describe result for acl check
2025-10-09T21:43:06.423586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:06.423593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:06.423595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:06.423603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340809568952040:2841] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:06.424235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340809568952040:2841] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:06.427888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340809568952040:2841] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:06.433626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340809568952040:2841] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:06.433648Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340809568952040:2841] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:06.433654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340809568952040:2841] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:06.433666Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 57 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:06.434094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:06.435342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:06.435629Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:06.435639Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:06.435656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:06.435736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:06.435781Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340809207143643:2701]
2025-10-09T21:43:06.436368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:06.436795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:06.436883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:06.437307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:06.437476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:06.437484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:06.437485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:06.437487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:06.437488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:06.437489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:06.437491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.437492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.437493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.437498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:06.437522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:06.437526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:06.437530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:06.437534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:06.437538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:06.437542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.437545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.437548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.437559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:06.437567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:06.437574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:06.437578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:06.437582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:06.437587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:06.437591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:06.437596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:06.437600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:06.437605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:06.437609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:06.437614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:06.437619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.437624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.437628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.437633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.437639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.437644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.437680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:06.448826Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 56
2025-10-09T21:43:06.448826Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 56
2025-10-09T21:43:06.447150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:06.448375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:06.448381Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:06.448384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:06.448389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:06.448399Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 56 [2:7559340809207143643:2701]
2025-10-09T21:43:06.448416Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:06.448439Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:06.448442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.448446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.448449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.448468Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:06.448475Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:06.448483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:06.448560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.448563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.448567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.448571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:06.448670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:06.448673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.448687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.448697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.448713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:06.448737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340809568952122:2841], now have 1 active actors on pipe
2025-10-09T21:43:06.448741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340809568952017:2432], now have 1 active actors on pipe
2025-10-09T21:43:06.448749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340809207143637:2701]
2025-10-09T21:43:06.448752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340809207143637:2701]
2025-10-09T21:43:06.448755Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340809207143637:2701]
2025-10-09T21:43:06.448768Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:06.448785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:06.448784Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:06.448788Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:06.448799Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:06.448800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:06.448803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:06.448847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340809207143637:2701]
2025-10-09T21:43:06.448851Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:06.448853Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:06.448885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:06.448891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:06.448927Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|addac5a9-ae85d442-39d92e1b-d8bea7ca_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:06.448935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.448939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.448942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:06.448953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.448963Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:06.448983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:06.449227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:06.449233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:06.449263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:06.449386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 57 partition: 0 MaxSeqNo: 25 sessionId: src_id|addac5a9-ae85d442-39d92e1b-d8bea7ca_0
2025-10-09T21:43:06.450020Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42ce9872-880ce37d-d313401a-f2f8a761_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|addac5a9-ae85d442-39d92e1b-d8bea7ca_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:06.453161Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|addac5a9-ae85d442-39d92e1b-d8bea7ca_0] Write session: try to update token
2025-10-09T21:43:06.453185Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|addac5a9-ae85d442-39d92e1b-d8bea7ca_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:06.467068Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:06.467084Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:06.467087Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:06.467089Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:06.467105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340809207143637:2701]
2025-10-09T21:43:06.467110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340809207143637:2701]
2025-10-09T21:43:06.467113Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340809207143637:2701]
2025-10-09T21:43:06.467158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:06.467165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.467168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:06.467167Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:06.467174Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:06.467178Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:06.467305Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:06.473551Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 57 sessionId: src_id|addac5a9-ae85d442-39d92e1b-d8bea7ca_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:06.476311Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:06.511881Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340809568952136:2476]
2025-10-09T21:43:06.512170Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 56, pipe: [1:7559340809568952136:2476]
2025-10-09T21:43:06.629697Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:06.630138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:06.630154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:06.630202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340809568952136:2476], now have 1 active actors on pipe
2025-10-09T21:43:06.630457Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340809568952136:2476]
2025-10-09T21:43:06.630471Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:06.630475Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:06.630490Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:06.630546Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 57 reason: error at write: Disconnected sessionId: src_id|addac5a9-ae85d442-39d92e1b-d8bea7ca_0
2025-10-09T21:43:06.630770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 57 sessionId: src_id|addac5a9-ae85d442-39d92e1b-d8bea7ca_0 is DEAD
2025-10-09T21:43:06.632784Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:06.633790Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|addac5a9-ae85d442-39d92e1b-d8bea7ca_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:06.633804Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|addac5a9-ae85d442-39d92e1b-d8bea7ca_0] Write session will restart in 0.007480s
2025-10-09T21:43:06.633831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|addac5a9-ae85d442-39d92e1b-d8bea7ca_0] Write session: Do CDS request
2025-10-09T21:43:06.633838Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|addac5a9-ae85d442-39d92e1b-d8bea7ca_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:06.642584Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|addac5a9-ae85d442-39d92e1b-d8bea7ca_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:06.643033Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:06.643051Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 58
2025-10-09T21:43:06.643198Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 58 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:06.643251Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 58 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:06.643257Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=58 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:06.643261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:06.643694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 58 sessionId:  describe result for acl check
2025-10-09T21:43:06.643737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:06.643746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:06.643748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:06.643755Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340809568952215:2847] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:06.644308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340809568952215:2847] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:06.646426Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340809568952215:2847] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:06.646649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:06.647139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:06.647352Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:06.647363Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:06.647413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:06.647470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:06.647521Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340809207143757:2704]
2025-10-09T21:43:06.647986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:06.648286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:06.648354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:06.648708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:06.648701Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340809568952215:2847] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:06.648714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340809568952215:2847] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:06.648718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340809568952215:2847] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:06.648727Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 58 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:06.648757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:06.648762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:06.648763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:06.648764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:06.648766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:06.648767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:06.648768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.648769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.648770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.648787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:06.648797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:06.648801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:06.648804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:06.648808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:06.648811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:06.648814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.648817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.648822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.648829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:06.648838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:06.648844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:06.648849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:06.648854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:06.648858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:06.648868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:06.648874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:06.648878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:06.648884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:06.648888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:06.648898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:06.648903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.648912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.648916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.648921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.648930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.648935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.648961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:06.657737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:06.659059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:06.659078Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:06.659082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:06.659086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:06.659095Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 57 [2:7559340809207143757:2704]
2025-10-09T21:43:06.659124Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:06.659147Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:06.659154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.659158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.659161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.659190Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:06.659196Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:06.659203Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:06.659278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.659284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.659287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.659291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:06.659536Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 57
2025-10-09T21:43:06.659566Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 57
2025-10-09T21:43:06.659369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:06.659380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.659397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.659411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.659428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:06.659451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340809568952252:2847], now have 1 active actors on pipe
2025-10-09T21:43:06.659454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340809568952141:2432], now have 1 active actors on pipe
2025-10-09T21:43:06.659462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340809207143752:2704]
2025-10-09T21:43:06.659464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340809207143752:2704]
2025-10-09T21:43:06.659466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340809207143752:2704]
2025-10-09T21:43:06.659481Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:06.659497Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:06.659505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:06.659510Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:06.659511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:06.659513Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:06.659514Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:06.659558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340809207143752:2704]
2025-10-09T21:43:06.659561Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:06.659565Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:06.659589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:06.659594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:06.659634Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8af64b84-4d8e60a5-124eac71-bfccfafd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:06.659642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.659647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.659650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:06.659661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.659671Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:06.659691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:06.659992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:06.660003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:06.660020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:06.660103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 58 partition: 0 MaxSeqNo: 25 sessionId: src_id|8af64b84-4d8e60a5-124eac71-bfccfafd_0
2025-10-09T21:43:06.660503Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|addac5a9-ae85d442-39d92e1b-d8bea7ca_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8af64b84-4d8e60a5-124eac71-bfccfafd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:06.663844Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8af64b84-4d8e60a5-124eac71-bfccfafd_0] Write session: try to update token
2025-10-09T21:43:06.663865Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8af64b84-4d8e60a5-124eac71-bfccfafd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:06.678935Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:06.678953Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:06.678956Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:06.678958Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:06.678968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340809207143752:2704]
2025-10-09T21:43:06.678973Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340809207143752:2704]
2025-10-09T21:43:06.678978Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340809207143752:2704]
2025-10-09T21:43:06.679023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:06.679041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.679044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:06.679179Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:06.679711Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:06.679726Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:06.679730Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:06.683458Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 58 sessionId: src_id|8af64b84-4d8e60a5-124eac71-bfccfafd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:06.685783Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:06.730478Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340809568952265:2476]
2025-10-09T21:43:06.731292Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 57, pipe: [1:7559340809568952265:2476]
2025-10-09T21:43:06.838989Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:06.839315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:06.839334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:06.839382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340809568952265:2476], now have 1 active actors on pipe
2025-10-09T21:43:06.839686Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:06.839699Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340809568952265:2476]
2025-10-09T21:43:06.839698Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:06.839700Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:06.839719Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:06.839767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 58 reason: error at write: Disconnected sessionId: src_id|8af64b84-4d8e60a5-124eac71-bfccfafd_0
2025-10-09T21:43:06.839971Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 58 sessionId: src_id|8af64b84-4d8e60a5-124eac71-bfccfafd_0 is DEAD
2025-10-09T21:43:06.840585Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8af64b84-4d8e60a5-124eac71-bfccfafd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:06.840600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8af64b84-4d8e60a5-124eac71-bfccfafd_0] Write session will restart in 0.009365s
2025-10-09T21:43:06.840628Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8af64b84-4d8e60a5-124eac71-bfccfafd_0] Write session: Do CDS request
2025-10-09T21:43:06.840635Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8af64b84-4d8e60a5-124eac71-bfccfafd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:06.851527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8af64b84-4d8e60a5-124eac71-bfccfafd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:06.851732Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:06.851747Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 59
2025-10-09T21:43:06.851998Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 59 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:06.852038Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 59 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:06.852043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=59 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:06.852047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:06.852125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:06.852535Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 59 sessionId:  describe result for acl check
2025-10-09T21:43:06.852573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:06.852576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:06.852577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:06.852584Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340809568952354:2853] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:06.853042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:06.853138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340809568952354:2853] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:06.853239Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:06.853249Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:06.853261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:06.853321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:06.853369Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340809207143872:2707]
2025-10-09T21:43:06.853936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:06.854368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:06.854444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:06.854767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:06.854816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:06.854821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:06.854822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:06.854824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:06.854825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:06.854826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:06.854827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.854829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.854830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.854836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:06.854846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:06.854850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:06.854853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:06.854857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:06.854861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:06.854865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.854869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.854873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.854879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:06.854887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:06.854895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:06.854900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:06.854904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:06.854909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:06.854914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:06.854920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:06.854925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:06.854931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:06.854936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:06.854941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:06.854951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.854956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.854964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.854970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.854980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.854986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.855018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:06.855753Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340809568952354:2853] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:06.863584Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340809568952354:2853] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:06.863604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340809568952354:2853] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:06.863608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340809568952354:2853] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:06.863618Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 59 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:06.864404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:06.865529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:06.865536Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:06.865539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:06.865543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:06.865551Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 58 [2:7559340809207143872:2707]
2025-10-09T21:43:06.865566Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:06.865586Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:06.865590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.865593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.865596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.865615Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:06.865622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:06.865631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:06.865689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.865692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.865693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.865697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:06.865767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:06.865770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.865783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:06.865794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:06.865811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:06.865830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340809207143867:2707]
2025-10-09T21:43:06.865833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340809207143867:2707]
2025-10-09T21:43:06.865835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340809207143867:2707]
2025-10-09T21:43:06.865842Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:06.865858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:06.865872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:06.865875Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:06.865903Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:06.865906Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:06.865908Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:06.865913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340809207143867:2707]
2025-10-09T21:43:06.865916Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:06.865925Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:06.865937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340809568952279:2432], now have 1 active actors on pipe
2025-10-09T21:43:06.865948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340809568952391:2853], now have 1 active actors on pipe
2025-10-09T21:43:06.866407Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 58
2025-10-09T21:43:06.866464Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 58
2025-10-09T21:43:06.866673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:06.866697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:06.866727Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ef7919ba-540ec859-9371cdbb-49a2a643_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:06.866737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.866742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.866751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:06.866763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.866775Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:06.866793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:06.867080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:06.867092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:06.867114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:06.867214Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 59 partition: 0 MaxSeqNo: 25 sessionId: src_id|ef7919ba-540ec859-9371cdbb-49a2a643_0
2025-10-09T21:43:06.867807Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8af64b84-4d8e60a5-124eac71-bfccfafd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ef7919ba-540ec859-9371cdbb-49a2a643_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:06.870716Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef7919ba-540ec859-9371cdbb-49a2a643_0] Write session: try to update token
2025-10-09T21:43:06.870736Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef7919ba-540ec859-9371cdbb-49a2a643_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:06.883681Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:06.883701Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:06.883703Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:06.883705Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:06.883714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340809207143867:2707]
2025-10-09T21:43:06.883719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340809207143867:2707]
2025-10-09T21:43:06.883723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340809207143867:2707]
2025-10-09T21:43:06.883782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:06.883790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:06.883792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:06.883802Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:06.883808Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:06.883812Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:06.883950Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:06.890070Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 59 sessionId: src_id|ef7919ba-540ec859-9371cdbb-49a2a643_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:06.892661Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:06.939889Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340809568952399:2476]
2025-10-09T21:43:06.940225Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 58, pipe: [1:7559340809568952399:2476]
2025-10-09T21:43:07.040652Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:07.041149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:07.041163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:07.041216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340809568952399:2476], now have 1 active actors on pipe
2025-10-09T21:43:07.041566Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:07.041563Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:07.041577Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:07.041582Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340809568952399:2476]
2025-10-09T21:43:07.041606Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:07.041642Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 59 reason: error at write: Disconnected sessionId: src_id|ef7919ba-540ec859-9371cdbb-49a2a643_0
2025-10-09T21:43:07.041843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 59 sessionId: src_id|ef7919ba-540ec859-9371cdbb-49a2a643_0 is DEAD
2025-10-09T21:43:07.046169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef7919ba-540ec859-9371cdbb-49a2a643_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:07.046189Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef7919ba-540ec859-9371cdbb-49a2a643_0] Write session will restart in 0.007610s
2025-10-09T21:43:07.046218Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef7919ba-540ec859-9371cdbb-49a2a643_0] Write session: Do CDS request
2025-10-09T21:43:07.046226Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef7919ba-540ec859-9371cdbb-49a2a643_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:07.058664Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef7919ba-540ec859-9371cdbb-49a2a643_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:07.058838Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:07.058854Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 60
2025-10-09T21:43:07.059054Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 60 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:07.059101Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 60 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:07.059112Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=60 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:07.059117Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:07.060048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 60 sessionId:  describe result for acl check
2025-10-09T21:43:07.060093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:07.060095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:07.060097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:07.060104Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340813863919761:2864] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:07.060723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340813863919761:2864] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:07.064446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340813863919761:2864] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:07.066623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:07.067269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:07.067492Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:07.067502Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:07.067515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:07.067505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340813863919761:2864] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:07.067529Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340813863919761:2864] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:07.067533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340813863919761:2864] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:07.067543Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 60 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:07.067571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:07.067612Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340813502111321:2721]
2025-10-09T21:43:07.068210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:07.068519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:07.068567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:07.068938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:07.068992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.068998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.069000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.069001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.069002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.069003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.069005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.069006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.069007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.069022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.069033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.069037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.069041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.069045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.069049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.069053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.069057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.069061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.069077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.069087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.069093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.069099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.069104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.069109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.069119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.069124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.069129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.069133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.069138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.069144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.069155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.069161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.069165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.069172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.069178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.069183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.069216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:07.076337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:07.077546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:07.077559Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:07.077563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:07.077568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:07.077577Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 59 [2:7559340813502111321:2721]
2025-10-09T21:43:07.077592Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:07.077624Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:07.077628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.077631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.077634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.077653Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:07.077664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:07.077672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:07.077736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.077744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.077746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.077749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:07.077828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:07.077836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.077850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.077867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.077887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:07.077914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340813863919830:2864], now have 1 active actors on pipe
2025-10-09T21:43:07.077920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340813863919723:2432], now have 1 active actors on pipe
2025-10-09T21:43:07.077928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340813502111316:2721]
2025-10-09T21:43:07.077931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340813502111316:2721]
2025-10-09T21:43:07.077933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340813502111316:2721]
2025-10-09T21:43:07.077948Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:07.077966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:07.078001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:07.078010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:07.078065Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340813502111316:2721]
2025-10-09T21:43:07.078074Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:07.078078Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:07.078082Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:07.078084Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:07.078091Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:07.078009Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 59
2025-10-09T21:43:07.078074Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 59
2025-10-09T21:43:07.078279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:07.078295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:07.078345Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|76f228b4-c0458ba9-e4600478-5449f990_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:07.078360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.078364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.078367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:07.078379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.078395Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:07.078436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:07.078844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:07.078858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:07.078880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:07.079078Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 60 partition: 0 MaxSeqNo: 25 sessionId: src_id|76f228b4-c0458ba9-e4600478-5449f990_0
2025-10-09T21:43:07.079595Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef7919ba-540ec859-9371cdbb-49a2a643_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|76f228b4-c0458ba9-e4600478-5449f990_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:07.082554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76f228b4-c0458ba9-e4600478-5449f990_0] Write session: try to update token
2025-10-09T21:43:07.082577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76f228b4-c0458ba9-e4600478-5449f990_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:07.095064Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:07.095082Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:07.095092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340813502111316:2721]
2025-10-09T21:43:07.095140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:07.095324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:07.095467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:07.095601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:07.095742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:07.095925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:07.095967Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:07.096056Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:07.110049Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 60 sessionId: src_id|76f228b4-c0458ba9-e4600478-5449f990_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:07.110736Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:07.110758Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:07.110761Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:07.110763Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:07.110774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340813502111316:2721]
2025-10-09T21:43:07.110777Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340813502111316:2721]
2025-10-09T21:43:07.110781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340813502111316:2721]
2025-10-09T21:43:07.110827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:07.110833Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:07.110840Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:07.110841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.110844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:07.110844Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:07.111044Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:07.112560Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:07.141665Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340813863919841:2476]
2025-10-09T21:43:07.142003Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 59, pipe: [1:7559340813863919841:2476]
2025-10-09T21:43:07.288409Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:07.288723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:07.288743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:07.288766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340813863919841:2476], now have 1 active actors on pipe
2025-10-09T21:43:07.289032Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340813863919841:2476]
2025-10-09T21:43:07.289075Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:07.289078Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:07.289081Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:07.289140Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:07.289155Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 60 reason: error at write: Disconnected sessionId: src_id|76f228b4-c0458ba9-e4600478-5449f990_0
2025-10-09T21:43:07.289600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76f228b4-c0458ba9-e4600478-5449f990_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:07.289616Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76f228b4-c0458ba9-e4600478-5449f990_0] Write session will restart in 0.008853s
2025-10-09T21:43:07.289642Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76f228b4-c0458ba9-e4600478-5449f990_0] Write session: Do CDS request
2025-10-09T21:43:07.289651Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76f228b4-c0458ba9-e4600478-5449f990_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:07.289741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 60 sessionId: src_id|76f228b4-c0458ba9-e4600478-5449f990_0 is DEAD
2025-10-09T21:43:07.299489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:07.299779Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76f228b4-c0458ba9-e4600478-5449f990_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:07.300883Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:07.300899Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 61
2025-10-09T21:43:07.301099Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 61 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:07.301148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 61 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:07.301159Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=61 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:07.301164Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:07.300233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:07.300420Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:07.300424Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:07.300436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:07.300484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:07.300520Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340813502111435:2723]
2025-10-09T21:43:07.301159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:07.301429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:07.301471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:07.301573Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 61 sessionId:  describe result for acl check
2025-10-09T21:43:07.301604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:07.301612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:07.301614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:07.301620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340813863919923:2868] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:07.301790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:07.301838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.301844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.301846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.301847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.301848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.301849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.301850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.301852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.301853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.301858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.301869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.301874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.301878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.301883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.301887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.301891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.301896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.301900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.301907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.301915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.301922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.301928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.301933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.301939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.301952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.301958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.301965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.301971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.301976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.301982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.301987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.301993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.301998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.302004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.302009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.302014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.302062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:07.302241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340813863919923:2868] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:07.304378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340813863919923:2868] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:07.310241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340813863919923:2868] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:07.310261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340813863919923:2868] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:07.310265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340813863919923:2868] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:07.310279Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 61 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:07.310807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:07.312170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:07.312188Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:07.312192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:07.312196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:07.312204Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 60 [2:7559340813502111435:2723]
2025-10-09T21:43:07.312222Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:07.312240Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:07.312243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.312246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.312249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.312269Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:07.312282Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:07.312290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:07.312350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.312359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.312361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.312363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:07.312437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:07.312446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.312460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.312471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.312486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:07.312515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340813863919959:2868], now have 1 active actors on pipe
2025-10-09T21:43:07.312518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340813863919844:2432], now have 1 active actors on pipe
2025-10-09T21:43:07.312526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340813502111430:2723]
2025-10-09T21:43:07.312529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340813502111430:2723]
2025-10-09T21:43:07.312531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340813502111430:2723]
2025-10-09T21:43:07.312548Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:07.312560Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:07.312563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:07.312569Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:07.312572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:07.312578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:07.312582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:07.312587Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 60
2025-10-09T21:43:07.312623Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340813502111430:2723]
2025-10-09T21:43:07.312637Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 60
2025-10-09T21:43:07.312637Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:07.312717Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:07.312810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:07.312821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:07.312848Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3d17c1e2-df826750-95e7bb5f-c32b4b97_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:07.312863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.312867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.312870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:07.312882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.312894Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:07.312916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:07.313171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:07.313183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:07.313202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:07.313376Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 61 partition: 0 MaxSeqNo: 25 sessionId: src_id|3d17c1e2-df826750-95e7bb5f-c32b4b97_0
2025-10-09T21:43:07.313798Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76f228b4-c0458ba9-e4600478-5449f990_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3d17c1e2-df826750-95e7bb5f-c32b4b97_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:07.317245Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d17c1e2-df826750-95e7bb5f-c32b4b97_0] Write session: try to update token
2025-10-09T21:43:07.317271Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d17c1e2-df826750-95e7bb5f-c32b4b97_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:07.336533Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:07.336550Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:07.336552Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:07.336555Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:07.336566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340813502111430:2723]
2025-10-09T21:43:07.336571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340813502111430:2723]
2025-10-09T21:43:07.336576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340813502111430:2723]
2025-10-09T21:43:07.336624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:07.336624Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:07.336631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.336632Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:07.336634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:07.336636Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:07.336796Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:07.348826Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 61 sessionId: src_id|3d17c1e2-df826750-95e7bb5f-c32b4b97_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:07.351535Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:07.389775Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340813863919969:2476]
2025-10-09T21:43:07.390397Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 60, pipe: [1:7559340813863919969:2476]
2025-10-09T21:43:07.495875Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:07.496205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:07.496220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:07.496243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340813863919969:2476], now have 1 active actors on pipe
2025-10-09T21:43:07.496464Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:07.496771Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:07.496822Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340813863919969:2476]
2025-10-09T21:43:07.496834Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:07.496924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 61 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3d17c1e2-df826750-95e7bb5f-c32b4b97_0
2025-10-09T21:43:07.497280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d17c1e2-df826750-95e7bb5f-c32b4b97_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:07.497295Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d17c1e2-df826750-95e7bb5f-c32b4b97_0] Write session will restart in 0.007321s
2025-10-09T21:43:07.497319Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d17c1e2-df826750-95e7bb5f-c32b4b97_0] Write session: Do CDS request
2025-10-09T21:43:07.497326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d17c1e2-df826750-95e7bb5f-c32b4b97_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:07.497108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 61 sessionId: src_id|3d17c1e2-df826750-95e7bb5f-c32b4b97_0 is DEAD
2025-10-09T21:43:07.505631Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d17c1e2-df826750-95e7bb5f-c32b4b97_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:07.506204Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:07.506220Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 62
2025-10-09T21:43:07.506437Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 62 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:07.506488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 62 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:07.506500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=62 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:07.506505Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:07.506955Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 62 sessionId:  describe result for acl check
2025-10-09T21:43:07.506995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:07.506999Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:07.507001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:07.507006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340813863920042:2873] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:07.507603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340813863920042:2873] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:07.509266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:07.509959Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340813863920042:2873] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:07.509969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:07.510186Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:07.510199Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:07.510212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:07.510275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:07.510313Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340813502111553:2725]
2025-10-09T21:43:07.510895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:07.511210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:07.511263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:07.511606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:07.511660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.511667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.511668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.511669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.511670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.511671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.511672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.511673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.511674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.511680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.511689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.511693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.511697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.511700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.511705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.511708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.511710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.511713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.511721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.511729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.511735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.511739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.511744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.511748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.511753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.511758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.511763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.511768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.511777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.511782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.511792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.511798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.511802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.511814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.511819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.511824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.511852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:07.517081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340813863920042:2873] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:07.517097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340813863920042:2873] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:07.517101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340813863920042:2873] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:07.517111Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 62 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:07.519824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:07.521002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:07.521018Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:07.521021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:07.521025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:07.521033Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 61 [2:7559340813502111553:2725]
2025-10-09T21:43:07.521048Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:07.521066Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:07.521069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.521072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.521074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.521090Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:07.521097Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:07.521105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:07.521166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.521175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.521177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.521179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:07.521254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:07.521263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.521274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.521284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.521299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:07.521325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340813863919972:2432], now have 1 active actors on pipe
2025-10-09T21:43:07.521331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340813863920087:2873], now have 1 active actors on pipe
2025-10-09T21:43:07.521339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340813502111548:2725]
2025-10-09T21:43:07.521341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340813502111548:2725]
2025-10-09T21:43:07.521343Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340813502111548:2725]
2025-10-09T21:43:07.521358Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:07.521371Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:07.521375Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:07.521377Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:07.521379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:07.521385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:07.521388Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:07.521429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340813502111548:2725]
2025-10-09T21:43:07.521441Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:07.521455Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 61
2025-10-09T21:43:07.521482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 61
2025-10-09T21:43:07.521470Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:07.521617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:07.521629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:07.521661Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a9858eb9-b3a5ca52-7b5ee6a8-2e2f7d2f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:07.521673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.521682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.521685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:07.521695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.521711Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:07.521732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:07.521880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:07.521888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:07.521904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:07.521992Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 62 partition: 0 MaxSeqNo: 25 sessionId: src_id|a9858eb9-b3a5ca52-7b5ee6a8-2e2f7d2f_0
2025-10-09T21:43:07.522520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d17c1e2-df826750-95e7bb5f-c32b4b97_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a9858eb9-b3a5ca52-7b5ee6a8-2e2f7d2f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:07.525631Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9858eb9-b3a5ca52-7b5ee6a8-2e2f7d2f_0] Write session: try to update token
2025-10-09T21:43:07.525651Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9858eb9-b3a5ca52-7b5ee6a8-2e2f7d2f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:07.537643Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:07.537661Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:07.537664Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:07.537666Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:07.537675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340813502111548:2725]
2025-10-09T21:43:07.537680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340813502111548:2725]
2025-10-09T21:43:07.537684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340813502111548:2725]
2025-10-09T21:43:07.537732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:07.537731Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:07.537737Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:07.537739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.537741Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:07.537741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:07.537893Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:07.556261Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 62 sessionId: src_id|a9858eb9-b3a5ca52-7b5ee6a8-2e2f7d2f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:07.558724Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:07.596824Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340813863920095:2476]
2025-10-09T21:43:07.597368Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 61, pipe: [1:7559340813863920095:2476]
2025-10-09T21:43:07.699601Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340813863920095:2476]
2025-10-09T21:43:07.699612Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:07.699639Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:07.699642Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:07.699704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 62 reason: error at write: Disconnected sessionId: src_id|a9858eb9-b3a5ca52-7b5ee6a8-2e2f7d2f_0
2025-10-09T21:43:07.699906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 62 sessionId: src_id|a9858eb9-b3a5ca52-7b5ee6a8-2e2f7d2f_0 is DEAD
2025-10-09T21:43:07.700412Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9858eb9-b3a5ca52-7b5ee6a8-2e2f7d2f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:07.700429Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9858eb9-b3a5ca52-7b5ee6a8-2e2f7d2f_0] Write session will restart in 0.008861s
2025-10-09T21:43:07.700454Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9858eb9-b3a5ca52-7b5ee6a8-2e2f7d2f_0] Write session: Do CDS request
2025-10-09T21:43:07.700460Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9858eb9-b3a5ca52-7b5ee6a8-2e2f7d2f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:07.698790Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:07.699250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:07.699261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:07.699286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340813863920095:2476], now have 1 active actors on pipe
2025-10-09T21:43:07.699512Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:07.713874Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9858eb9-b3a5ca52-7b5ee6a8-2e2f7d2f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:07.714682Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:07.714705Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 63
2025-10-09T21:43:07.715101Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 63 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:07.715137Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 63 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:07.715143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=63 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:07.715149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:07.715732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 63 sessionId:  describe result for acl check
2025-10-09T21:43:07.715768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:07.715771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:07.715773Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:07.715779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340813863920165:2877] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:07.716318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340813863920165:2877] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:07.720404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340813863920165:2877] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:07.718178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:07.719016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:07.719216Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:07.719220Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:07.719233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:07.719288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:07.719321Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340813502111673:2728]
2025-10-09T21:43:07.719873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:07.720137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:07.720172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:07.720475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:07.720514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.720515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.720516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.720517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.720519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.720520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.720521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.720522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.720523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.720528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.720537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.720540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.720544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.720547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.720551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.720555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.720559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.720562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.720569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.720577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.720583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.720588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.720592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.720597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.720601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.720606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.720610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.720615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.720619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.720624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.720629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.720634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.720638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.720642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.720646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.720651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.720678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:07.727231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:07.726965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340813863920165:2877] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:07.726982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340813863920165:2877] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:07.726986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340813863920165:2877] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:07.727001Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 63 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:07.728402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:07.728420Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:07.728423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:07.728427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:07.728434Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 62 [2:7559340813502111673:2728]
2025-10-09T21:43:07.728448Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:07.728468Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:07.728471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.728488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.728496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.728516Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:07.728528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:07.728535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:07.728605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.728614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.728616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.728619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:07.728695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:07.728704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.728717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.728740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.728762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:07.728814Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 62
2025-10-09T21:43:07.728786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340813863920216:2877], now have 1 active actors on pipe
2025-10-09T21:43:07.728845Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 62
2025-10-09T21:43:07.728796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340813863920098:2432], now have 1 active actors on pipe
2025-10-09T21:43:07.728805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340813502111668:2728]
2025-10-09T21:43:07.728808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340813502111668:2728]
2025-10-09T21:43:07.728810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340813502111668:2728]
2025-10-09T21:43:07.728825Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:07.728842Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:07.728843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:07.728845Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:07.728860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:07.728861Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:07.728864Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:07.728913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:07.728924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:07.728942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340813502111668:2728]
2025-10-09T21:43:07.728955Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:07.728966Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:07.728992Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7b0e06fe-3aa0e681-87a144fb-3c9397d7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:07.729004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.729009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.729012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:07.729023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.729038Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:07.729069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:07.729230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:07.729239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:07.729255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:07.729347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 63 partition: 0 MaxSeqNo: 25 sessionId: src_id|7b0e06fe-3aa0e681-87a144fb-3c9397d7_0
2025-10-09T21:43:07.729813Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9858eb9-b3a5ca52-7b5ee6a8-2e2f7d2f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7b0e06fe-3aa0e681-87a144fb-3c9397d7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:07.733211Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b0e06fe-3aa0e681-87a144fb-3c9397d7_0] Write session: try to update token
2025-10-09T21:43:07.733229Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b0e06fe-3aa0e681-87a144fb-3c9397d7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:07.750970Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:07.750989Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:07.750992Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:07.750993Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:07.751003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340813502111668:2728]
2025-10-09T21:43:07.751007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340813502111668:2728]
2025-10-09T21:43:07.751011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340813502111668:2728]
2025-10-09T21:43:07.751061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:07.751063Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:07.751069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.751070Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:07.751072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:07.751074Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:07.751453Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:07.762738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 63 sessionId: src_id|7b0e06fe-3aa0e681-87a144fb-3c9397d7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:07.765168Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:07.799552Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340813863920231:2476]
2025-10-09T21:43:07.799879Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 62, pipe: [1:7559340813863920231:2476]
2025-10-09T21:43:07.918796Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:07.919270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:07.919281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:07.919304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340813863920231:2476], now have 1 active actors on pipe
2025-10-09T21:43:07.919569Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:07.921079Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b0e06fe-3aa0e681-87a144fb-3c9397d7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:07.921096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b0e06fe-3aa0e681-87a144fb-3c9397d7_0] Write session will restart in 0.007074s
2025-10-09T21:43:07.921122Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b0e06fe-3aa0e681-87a144fb-3c9397d7_0] Write session: Do CDS request
2025-10-09T21:43:07.921128Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b0e06fe-3aa0e681-87a144fb-3c9397d7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:07.920329Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:07.920411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 63 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7b0e06fe-3aa0e681-87a144fb-3c9397d7_0
2025-10-09T21:43:07.920594Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 63 sessionId: src_id|7b0e06fe-3aa0e681-87a144fb-3c9397d7_0 is DEAD
2025-10-09T21:43:07.920730Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:07.920760Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340813863920231:2476]
2025-10-09T21:43:07.929714Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b0e06fe-3aa0e681-87a144fb-3c9397d7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:07.930004Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:07.930019Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 64
2025-10-09T21:43:07.933660Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 64 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:07.933717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 64 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:07.933727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=64 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:07.933736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:07.934306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 64 sessionId:  describe result for acl check
2025-10-09T21:43:07.934343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:07.934345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:07.934346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:07.934353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340813863920252:2883] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:07.935039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340813863920252:2883] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:07.947498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:07.948783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:07.948986Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:07.948990Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:07.949002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:07.949057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:07.949097Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340813502111793:2731]
2025-10-09T21:43:07.949667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:07.949703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340813863920252:2883] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:07.950598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:07.950653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:07.950995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:07.951072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.951073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.951074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.951076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.951077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.951078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.951079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.951080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.951081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.951087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.951097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.951100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.951104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.951108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.951111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.951114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.951117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.951121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.951127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.951134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:07.951140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.951145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:07.951149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.951153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:07.951157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.951162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:07.951166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.951171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:07.951175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.951179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:07.951184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.951189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.951194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.951199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.951203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.951209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.951235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:07.958232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340813863920252:2883] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:07.958256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340813863920252:2883] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:07.958263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340813863920252:2883] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:07.958278Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 64 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:07.959357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:07.961053Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 63
2025-10-09T21:43:07.961064Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 63
2025-10-09T21:43:07.960626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:07.960644Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:07.960647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:07.960652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:07.960659Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 63 [2:7559340813502111793:2731]
2025-10-09T21:43:07.960676Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:07.960695Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:07.960699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.960702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.960705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.960721Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:07.960728Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:07.960734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:07.960795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.960797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.960798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.960801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:07.960875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:07.960877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:07.960888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:07.960897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:07.960910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:07.960931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340813502111787:2731]
2025-10-09T21:43:07.960933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340813502111787:2731]
2025-10-09T21:43:07.960935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340813502111787:2731]
2025-10-09T21:43:07.960943Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:07.960955Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:07.960957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:07.960958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:07.960961Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:07.960970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:07.960973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:07.960984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340813863920354:2883], now have 1 active actors on pipe
2025-10-09T21:43:07.960987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340813863920237:2432], now have 1 active actors on pipe
2025-10-09T21:43:07.961023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340813502111787:2731]
2025-10-09T21:43:07.961032Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:07.961045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:07.961094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:07.961098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:07.961127Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8d8d158a-f01d8eef-c608dc81-e34589f5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:07.961135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.961139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.961141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:07.961151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.961161Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:07.961176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:07.961395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:07.961401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:07.961412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:07.961517Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 64 partition: 0 MaxSeqNo: 25 sessionId: src_id|8d8d158a-f01d8eef-c608dc81-e34589f5_0
2025-10-09T21:43:07.961909Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b0e06fe-3aa0e681-87a144fb-3c9397d7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8d8d158a-f01d8eef-c608dc81-e34589f5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:07.965426Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8d8d158a-f01d8eef-c608dc81-e34589f5_0] Write session: try to update token
2025-10-09T21:43:07.965452Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8d8d158a-f01d8eef-c608dc81-e34589f5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:07.995365Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 64 sessionId: src_id|8d8d158a-f01d8eef-c608dc81-e34589f5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:07.997961Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:07.995501Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:07.995517Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:07.995526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340813502111787:2731]
2025-10-09T21:43:07.995575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:07.995886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:07.996071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:07.996214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:07.996350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:07.996784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:07.996821Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:07.996898Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:08.002092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:08.002114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:08.002158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:08.006131Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:08.020722Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340818158887659:2476]
2025-10-09T21:43:08.018329Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:08.018348Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:08.018352Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:08.018354Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:08.018364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340813502111787:2731]
2025-10-09T21:43:08.018367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340813502111787:2731]
2025-10-09T21:43:08.018370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340813502111787:2731]
2025-10-09T21:43:08.018429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:08.018436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.018441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:08.018606Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:08.022152Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:08.022158Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:08.022165Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:08.022655Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 63, pipe: [1:7559340818158887659:2476]
2025-10-09T21:43:08.196524Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:08.196959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340818158887659:2476], now have 1 active actors on pipe
2025-10-09T21:43:08.208475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:08.208498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:08.208669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:08.208722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:08.208783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:08.208845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:08.208894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:08.208899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:08.208966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:08.209024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:08.209084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:08.209147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:08.209186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:08.209190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:08.209254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:08.209313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:08.209373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:08.209432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:08.209471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:08.209474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:08.209551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:08.209627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:08.209695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:08.209760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:08.209804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:08.209807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:08.209878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:08.209967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:08.210053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:08.210119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:08.210162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:08.210165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:08.210237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:08.210297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:08.210364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:08.210436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:08.210496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:08.210501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:08.210573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:08.210633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:08.210706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:08.210772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:08.210811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:08.210814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:08.211181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:08.211253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:08.211316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:08.211382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:08.211434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:08.211437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:08.211505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:08.211564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:08.211664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:08.211723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:08.211769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:08.211771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:08.211835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:08.211894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:08.211956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:08.212054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:08.212225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:08.212247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:08.214604Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d8d158a-f01d8eef-c608dc81-e34589f5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:08.214615Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d8d158a-f01d8eef-c608dc81-e34589f5_0] Write session will restart in 0.006929s
2025-10-09T21:43:08.214639Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d8d158a-f01d8eef-c608dc81-e34589f5_0] Write session: Do CDS request
2025-10-09T21:43:08.214646Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d8d158a-f01d8eef-c608dc81-e34589f5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:08.214822Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:08.214059Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340818158887659:2476]
2025-10-09T21:43:08.214106Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:08.214109Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:08.214138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 64 reason: error at write: Disconnected sessionId: src_id|8d8d158a-f01d8eef-c608dc81-e34589f5_0
2025-10-09T21:43:08.214371Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:08.218491Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 64 sessionId: src_id|8d8d158a-f01d8eef-c608dc81-e34589f5_0 is DEAD
2025-10-09T21:43:08.229820Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8d8d158a-f01d8eef-c608dc81-e34589f5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:08.232801Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:08.232818Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 65
2025-10-09T21:43:08.233053Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 65 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:08.233087Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 65 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:08.233093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=65 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:08.233097Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:08.233564Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 65 sessionId:  describe result for acl check
2025-10-09T21:43:08.233607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:08.233608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:08.233610Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:08.233617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340818158887701:2894] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:08.234234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340818158887701:2894] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:08.247102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340818158887701:2894] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:08.262380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:08.263728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340818158887701:2894] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:08.263749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340818158887701:2894] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:08.263754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340818158887701:2894] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:08.263767Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 65 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:08.267917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:08.268210Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:08.268220Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:08.268232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:08.268295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:08.268339Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340817797079249:2745]
2025-10-09T21:43:08.268963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:08.273742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:08.273820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:08.274196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:08.274270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:08.274272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:08.274273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:08.274275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:08.274276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:08.274277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:08.274278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.274280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.274281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.274288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:08.274297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:08.274301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:08.274305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:08.274308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:08.274312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:08.274315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.274319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.274322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.274329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:08.274338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:08.274345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:08.274351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:08.274356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:08.274361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:08.274365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:08.274370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:08.274374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:08.274379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:08.274383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:08.274389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:08.274394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.274398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.274402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.274406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.274411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.274428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.274459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:08.296483Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 64
2025-10-09T21:43:08.296515Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 64
2025-10-09T21:43:08.294235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:08.295512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:08.295520Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:08.295523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:08.295528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:08.295537Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 64 [2:7559340817797079249:2745]
2025-10-09T21:43:08.295555Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:08.295580Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:08.295584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.295587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.295590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.295611Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:08.295617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:08.295626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:08.295698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.295701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.295704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.295706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:08.295790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:08.295793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.295808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.295819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.295836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:08.295856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340817797079242:2745]
2025-10-09T21:43:08.295859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340817797079242:2745]
2025-10-09T21:43:08.295862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340817797079242:2745]
2025-10-09T21:43:08.295877Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:08.295894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:08.295910Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:08.295913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:08.295960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340817797079242:2745]
2025-10-09T21:43:08.295964Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:08.296103Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:08.296118Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:08.296121Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:08.296128Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:08.296968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340818158887800:2894], now have 1 active actors on pipe
2025-10-09T21:43:08.296973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340818158887686:2432], now have 1 active actors on pipe
2025-10-09T21:43:08.296988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:08.296995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:08.297028Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ca2ca061-cbe476d0-29061845-909f5caf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:08.297038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.297042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.297046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:08.297057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.297070Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:08.297090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:08.298997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:08.299012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:08.299040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:08.306144Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 65 partition: 0 MaxSeqNo: 25 sessionId: src_id|ca2ca061-cbe476d0-29061845-909f5caf_0
2025-10-09T21:43:08.309999Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d8d158a-f01d8eef-c608dc81-e34589f5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ca2ca061-cbe476d0-29061845-909f5caf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:08.313406Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca2ca061-cbe476d0-29061845-909f5caf_0] Write session: try to update token
2025-10-09T21:43:08.313428Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca2ca061-cbe476d0-29061845-909f5caf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:08.314548Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340818158887812:2476]
2025-10-09T21:43:08.315199Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 64, pipe: [1:7559340818158887812:2476]
2025-10-09T21:43:08.314345Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:08.314362Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:08.314365Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:08.314367Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:08.314376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340817797079242:2745]
2025-10-09T21:43:08.314381Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340817797079242:2745]
2025-10-09T21:43:08.314385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340817797079242:2745]
2025-10-09T21:43:08.314438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:08.314446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.314448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:08.314599Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:08.315068Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:08.315075Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:08.315079Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:08.343834Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 65 sessionId: src_id|ca2ca061-cbe476d0-29061845-909f5caf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:08.346363Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:08.487448Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:08.487845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340818158887812:2476], now have 1 active actors on pipe
2025-10-09T21:43:08.487896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:08.487903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:08.488116Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:08.488129Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:08.488203Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 65 reason: error at write: Disconnected sessionId: src_id|ca2ca061-cbe476d0-29061845-909f5caf_0
2025-10-09T21:43:08.488107Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:08.488393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 65 sessionId: src_id|ca2ca061-cbe476d0-29061845-909f5caf_0 is DEAD
2025-10-09T21:43:08.488572Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:08.488810Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca2ca061-cbe476d0-29061845-909f5caf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:08.488821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca2ca061-cbe476d0-29061845-909f5caf_0] Write session will restart in 0.009114s
2025-10-09T21:43:08.488845Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca2ca061-cbe476d0-29061845-909f5caf_0] Write session: Do CDS request
2025-10-09T21:43:08.488853Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca2ca061-cbe476d0-29061845-909f5caf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:08.489064Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340818158887812:2476]
2025-10-09T21:43:08.499540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca2ca061-cbe476d0-29061845-909f5caf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:08.500433Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:08.500452Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 66
2025-10-09T21:43:08.500609Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 66 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:08.500651Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 66 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:08.500663Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=66 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:08.500668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:08.501259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 66 sessionId:  describe result for acl check
2025-10-09T21:43:08.501294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:08.501296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:08.501297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:08.501304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340818158887897:2899] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:08.501789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340818158887897:2899] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:08.505651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340818158887897:2899] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:08.506975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:08.510712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:08.511050Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:08.511062Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:08.511078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:08.511133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340818158887897:2899] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:08.511148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340818158887897:2899] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:08.511152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340818158887897:2899] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:08.511162Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 66 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:08.511164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:08.511225Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340817797079369:2747]
2025-10-09T21:43:08.511794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:08.512129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:08.512178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:08.512485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:08.512539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:08.512540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:08.512541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:08.512542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:08.512543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:08.512544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:08.512546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.512547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.512548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.512553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:08.512563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:08.512566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:08.512569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:08.512573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:08.512577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:08.512581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.512585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.512590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.512596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:08.512604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:08.512612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:08.512618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:08.512624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:08.512630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:08.512634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:08.512640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:08.512645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:08.512650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:08.512654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:08.512659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:08.512664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.512669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.512673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.512677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.512681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.512686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.512718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:08.519434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:08.520703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:08.520718Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:08.520720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:08.520725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:08.520731Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 65 [2:7559340817797079369:2747]
2025-10-09T21:43:08.520746Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:08.520769Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:08.520776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.520779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.520781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.520794Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:08.520802Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:08.520814Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:08.520881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.520888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.520890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.520893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:08.520958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:08.520964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.520976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.520992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.521015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:08.521037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340818158887818:2432], now have 1 active actors on pipe
2025-10-09T21:43:08.521040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340818158887933:2899], now have 1 active actors on pipe
2025-10-09T21:43:08.521048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340817797079364:2747]
2025-10-09T21:43:08.521054Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340817797079364:2747]
2025-10-09T21:43:08.521057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340817797079364:2747]
2025-10-09T21:43:08.521069Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:08.521082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:08.521082Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:08.521085Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:08.521087Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:08.521098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:08.521107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:08.521138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340817797079364:2747]
2025-10-09T21:43:08.521146Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:08.521150Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:08.521133Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 65
2025-10-09T21:43:08.521146Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 65
2025-10-09T21:43:08.521269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:08.521278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:08.521309Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fdf68535-af33728b-2c9ed60f-436305b7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:08.521316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.521320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.521332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:08.521343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.521354Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:08.521367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:08.521646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:08.521656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:08.521672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:08.521786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 66 partition: 0 MaxSeqNo: 25 sessionId: src_id|fdf68535-af33728b-2c9ed60f-436305b7_0
2025-10-09T21:43:08.522283Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca2ca061-cbe476d0-29061845-909f5caf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fdf68535-af33728b-2c9ed60f-436305b7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:08.526663Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf68535-af33728b-2c9ed60f-436305b7_0] Write session: try to update token
2025-10-09T21:43:08.526684Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf68535-af33728b-2c9ed60f-436305b7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:08.533246Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:08.533266Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:08.533276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340817797079364:2747]
2025-10-09T21:43:08.533323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:08.533524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:08.533672Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:08.533687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:08.533832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:08.533978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:08.534212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:08.534262Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:08.556165Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:08.556180Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:08.556183Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:08.556185Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:08.556195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340817797079364:2747]
2025-10-09T21:43:08.556199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340817797079364:2747]
2025-10-09T21:43:08.556202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340817797079364:2747]
2025-10-09T21:43:08.556245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:08.556252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.556254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:08.556259Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:08.556265Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:08.556268Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:08.556630Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:08.563323Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 66 sessionId: src_id|fdf68535-af33728b-2c9ed60f-436305b7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:08.566165Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:08.589058Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340818158887945:2476]
2025-10-09T21:43:08.589542Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 65, pipe: [1:7559340818158887945:2476]
2025-10-09T21:43:08.745360Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:08.745857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:08.745873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:08.745898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340818158887945:2476], now have 1 active actors on pipe
2025-10-09T21:43:08.746705Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340818158887945:2476]
2025-10-09T21:43:08.746728Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:08.746756Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:08.746759Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:08.746813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 66 reason: error at write: Disconnected sessionId: src_id|fdf68535-af33728b-2c9ed60f-436305b7_0
2025-10-09T21:43:08.747005Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 66 sessionId: src_id|fdf68535-af33728b-2c9ed60f-436305b7_0 is DEAD
2025-10-09T21:43:08.748389Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf68535-af33728b-2c9ed60f-436305b7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:08.748399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf68535-af33728b-2c9ed60f-436305b7_0] Write session will restart in 0.009856s
2025-10-09T21:43:08.748423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf68535-af33728b-2c9ed60f-436305b7_0] Write session: Do CDS request
2025-10-09T21:43:08.748433Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf68535-af33728b-2c9ed60f-436305b7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:08.748657Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:08.762868Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf68535-af33728b-2c9ed60f-436305b7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:08.765682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:08.766546Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:08.766561Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 67
2025-10-09T21:43:08.766739Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 67 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:08.766777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 67 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:08.766788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=67 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:08.766792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:08.766970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:08.767196Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:08.767206Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:08.767218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:08.767269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:08.767299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 67 sessionId:  describe result for acl check
2025-10-09T21:43:08.767341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:08.767347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:08.767348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:08.767349Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340817797079493:2750]
2025-10-09T21:43:08.767354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340818158888038:2905] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:08.767943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340818158888038:2905] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:08.767968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:08.768296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:08.768339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:08.768696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:08.768745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:08.768751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:08.768752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:08.768754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:08.768755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:08.768756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:08.768757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.768758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.768760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.768765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:08.768776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:08.768781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:08.768785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:08.768789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:08.768794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:08.768798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.768802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.768807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.768814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:08.768822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:08.768829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:08.768835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:08.768841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:08.768847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:08.768852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:08.768857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:08.768863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:08.768868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:08.768872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:08.768878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:08.768884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.768890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.768894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.768899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.768905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.768910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.768941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:08.775366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340818158888038:2905] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:08.776395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:08.780933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340818158888038:2905] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:08.780949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340818158888038:2905] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:08.780953Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340818158888038:2905] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:08.780962Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 67 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:08.781526Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 66
2025-10-09T21:43:08.781916Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 66
2025-10-09T21:43:08.780493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:08.780510Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:08.780513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:08.780518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:08.780525Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 66 [2:7559340817797079493:2750]
2025-10-09T21:43:08.780545Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:08.780564Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:08.780567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.780570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.780573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.780594Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:08.780601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:08.780609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:08.780674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.780676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.780678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.780680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:08.780902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:08.780906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.780920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:08.780931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:08.780948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:08.780971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340817797079488:2750]
2025-10-09T21:43:08.780973Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340817797079488:2750]
2025-10-09T21:43:08.780975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340817797079488:2750]
2025-10-09T21:43:08.780990Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:08.781007Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:08.781022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:08.781026Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:08.781076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340817797079488:2750]
2025-10-09T21:43:08.781080Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:08.781177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:08.781180Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:08.781182Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:08.781188Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:08.781513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340818158887952:2432], now have 1 active actors on pipe
2025-10-09T21:43:08.781860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340818158888075:2905], now have 1 active actors on pipe
2025-10-09T21:43:08.781974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:08.781980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:08.782009Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|53d80a11-cc5aed83-337692e0-af606315_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:08.782020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.782024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.782039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:08.782051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.782064Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:08.782082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:08.783075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:08.783100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:08.783122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:08.786205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 67 partition: 0 MaxSeqNo: 25 sessionId: src_id|53d80a11-cc5aed83-337692e0-af606315_0
2025-10-09T21:43:08.790161Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf68535-af33728b-2c9ed60f-436305b7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|53d80a11-cc5aed83-337692e0-af606315_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:08.803196Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53d80a11-cc5aed83-337692e0-af606315_0] Write session: try to update token
2025-10-09T21:43:08.803223Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53d80a11-cc5aed83-337692e0-af606315_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:08.805692Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:08.805707Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:08.805717Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340817797079488:2750]
2025-10-09T21:43:08.805765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:08.805766Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:08.805938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:08.806248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:08.806404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:08.806550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:08.806733Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:08.806771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:08.820555Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:08.820572Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:08.820574Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:08.820576Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:08.820587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340817797079488:2750]
2025-10-09T21:43:08.820591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340817797079488:2750]
2025-10-09T21:43:08.820594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340817797079488:2750]
2025-10-09T21:43:08.820635Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:08.820638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:08.820641Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:08.820645Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:08.820647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:08.820650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:08.820852Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:08.846104Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340818158888087:2476]
2025-10-09T21:43:08.846826Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 66, pipe: [1:7559340818158888087:2476]
2025-10-09T21:43:08.883754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 67 sessionId: src_id|53d80a11-cc5aed83-337692e0-af606315_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:08.886434Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:09.009743Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:09.010361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340818158888087:2476], now have 1 active actors on pipe
2025-10-09T21:43:09.010404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:09.010412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:09.010771Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:09.010783Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:09.011381Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53d80a11-cc5aed83-337692e0-af606315_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:09.011392Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53d80a11-cc5aed83-337692e0-af606315_0] Write session will restart in 0.006912s
2025-10-09T21:43:09.011419Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53d80a11-cc5aed83-337692e0-af606315_0] Write session: Do CDS request
2025-10-09T21:43:09.011426Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53d80a11-cc5aed83-337692e0-af606315_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:09.010845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 67 reason: error at write: Disconnected sessionId: src_id|53d80a11-cc5aed83-337692e0-af606315_0
2025-10-09T21:43:09.011025Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 67 sessionId: src_id|53d80a11-cc5aed83-337692e0-af606315_0 is DEAD
2025-10-09T21:43:09.011116Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:09.011154Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340818158888087:2476]
2025-10-09T21:43:09.011931Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:09.019551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53d80a11-cc5aed83-337692e0-af606315_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:09.019929Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:09.019947Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 68
2025-10-09T21:43:09.020084Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 68 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:09.020119Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 68 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:09.020125Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=68 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:09.020128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:09.020680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 68 sessionId:  describe result for acl check
2025-10-09T21:43:09.020715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:09.020717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:09.020719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:09.020725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340822453855415:2909] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:09.021300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340822453855415:2909] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:09.025043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340822453855415:2909] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:09.030154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:09.030387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:09.030575Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:09.030578Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:09.030590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:09.030631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:09.030667Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340822092046913:2753]
2025-10-09T21:43:09.030912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340822453855415:2909] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:09.030933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340822453855415:2909] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:09.030937Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340822453855415:2909] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:09.030948Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 68 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:09.031236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:09.031837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:09.031871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:09.032170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:09.032207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:09.032209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:09.032210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:09.032211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:09.032212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:09.032214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:09.032215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.032216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.032218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.032222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:09.032231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:09.032235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:09.032238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:09.032241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:09.032245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:09.032247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.032250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.032253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.032261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:09.032268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:09.032273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:09.032278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:09.032282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:09.032287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:09.032292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:09.032296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:09.032301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:09.032305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:09.032309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:09.032313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:09.032319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.032324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.032328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.032332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.032337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.032340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.032365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:09.040630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:09.042312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:09.042319Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:09.042321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:09.042327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:09.042347Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 67 [2:7559340822092046913:2753]
2025-10-09T21:43:09.042363Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:09.042382Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:09.042385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.042388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.042390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.042404Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:09.042410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:09.042434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:09.042499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.042501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.042504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.042507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:09.042586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:09.042589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.042604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.042615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.042634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:09.042665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340822092046907:2753]
2025-10-09T21:43:09.042667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340822092046907:2753]
2025-10-09T21:43:09.042670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340822092046907:2753]
2025-10-09T21:43:09.042679Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:09.042689Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:09.042692Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:09.042694Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:09.042702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340822453855501:2909], now have 1 active actors on pipe
2025-10-09T21:43:09.042706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340822453855389:2432], now have 1 active actors on pipe
2025-10-09T21:43:09.042712Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:09.042727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:09.042730Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:09.042775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340822092046907:2753]
2025-10-09T21:43:09.042788Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:09.043024Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 67
2025-10-09T21:43:09.043055Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 67
2025-10-09T21:43:09.043146Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:09.043389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:09.043401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:09.043432Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a9c849e0-a327932c-673cb5cb-cc256950_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:09.043440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.043447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.043450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:09.043461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.043474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:09.043491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:09.043670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:09.043673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:09.043684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:09.043800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 68 partition: 0 MaxSeqNo: 25 sessionId: src_id|a9c849e0-a327932c-673cb5cb-cc256950_0
2025-10-09T21:43:09.048940Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53d80a11-cc5aed83-337692e0-af606315_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a9c849e0-a327932c-673cb5cb-cc256950_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:09.052587Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9c849e0-a327932c-673cb5cb-cc256950_0] Write session: try to update token
2025-10-09T21:43:09.052609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9c849e0-a327932c-673cb5cb-cc256950_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:09.069621Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:09.069638Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:09.069641Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:09.069649Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:09.069661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340822092046907:2753]
2025-10-09T21:43:09.069665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340822092046907:2753]
2025-10-09T21:43:09.069668Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340822092046907:2753]
2025-10-09T21:43:09.069763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:09.069770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.069772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:09.069983Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:09.072705Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:09.072719Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:09.072767Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:09.115013Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340822453855513:2476]
2025-10-09T21:43:09.115862Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 67, pipe: [1:7559340822453855513:2476]
2025-10-09T21:43:09.136052Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 68 sessionId: src_id|a9c849e0-a327932c-673cb5cb-cc256950_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:09.142585Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:09.321199Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:09.321739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340822453855513:2476], now have 1 active actors on pipe
2025-10-09T21:43:09.321771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:09.321778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:09.323262Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:09.323274Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:09.323329Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 68 reason: error at write: Disconnected sessionId: src_id|a9c849e0-a327932c-673cb5cb-cc256950_0
2025-10-09T21:43:09.323522Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 68 sessionId: src_id|a9c849e0-a327932c-673cb5cb-cc256950_0 is DEAD
2025-10-09T21:43:09.323680Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:09.323712Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340822453855513:2476]
2025-10-09T21:43:09.322651Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:09.326156Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9c849e0-a327932c-673cb5cb-cc256950_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:09.326169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9c849e0-a327932c-673cb5cb-cc256950_0] Write session will restart in 0.006521s
2025-10-09T21:43:09.326233Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9c849e0-a327932c-673cb5cb-cc256950_0] Write session: Do CDS request
2025-10-09T21:43:09.326240Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9c849e0-a327932c-673cb5cb-cc256950_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:09.333991Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9c849e0-a327932c-673cb5cb-cc256950_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:09.334100Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:09.334114Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 69
2025-10-09T21:43:09.334633Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 69 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:09.334667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 69 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:09.334673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=69 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:09.334676Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:09.335444Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 69 sessionId:  describe result for acl check
2025-10-09T21:43:09.335483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:09.335490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:09.335492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:09.335497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340822453855556:2920] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:09.336102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340822453855556:2920] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:09.341089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340822453855556:2920] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:09.348909Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340822453855556:2920] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:09.348924Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340822453855556:2920] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:09.348928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340822453855556:2920] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:09.348939Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 69 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:09.363525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:09.365704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:09.366108Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:09.366113Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:09.366130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:09.366206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:09.366257Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340822092047073:2767]
2025-10-09T21:43:09.366935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:09.367550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:09.367602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:09.367929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:09.367975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:09.367977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:09.367978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:09.367979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:09.367980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:09.367981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:09.367982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.367984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.367985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.367991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:09.368002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:09.368006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:09.368010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:09.368014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:09.368018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:09.368021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.368025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.368029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.368038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:09.368046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:09.368053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:09.368058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:09.368061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:09.368066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:09.368070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:09.368076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:09.368080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:09.368085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:09.368089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:09.368094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:09.368100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.368105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.368110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.368115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.368120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.368126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.368156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:09.378590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:09.384324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:09.384341Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:09.384344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:09.384348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:09.384355Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 68 [2:7559340822092047073:2767]
2025-10-09T21:43:09.384366Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:09.384384Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:09.384386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.384389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.384390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.384406Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:09.384410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:09.384417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:09.384478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.384480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.384482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.384484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:09.384548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:09.384550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.384559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.384566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.384575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:09.384591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340822092047067:2767]
2025-10-09T21:43:09.384593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340822092047067:2767]
2025-10-09T21:43:09.384595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340822092047067:2767]
2025-10-09T21:43:09.384605Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:09.384618Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:09.384627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:09.384629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:09.384665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340822092047067:2767]
2025-10-09T21:43:09.384667Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:09.385659Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 68
2025-10-09T21:43:09.385689Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 68
2025-10-09T21:43:09.384978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:09.384982Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:09.384985Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:09.384990Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:09.394294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340822453855542:2432], now have 1 active actors on pipe
2025-10-09T21:43:09.394308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340822453855626:2920], now have 1 active actors on pipe
2025-10-09T21:43:09.394340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:09.394348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:09.394390Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2f762271-dc8a3574-e240af0-16d3dda5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:09.394405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.394411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.394427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:09.394443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.394465Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:09.394488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:09.395585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:09.395593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:09.395609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:09.395745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 69 partition: 0 MaxSeqNo: 25 sessionId: src_id|2f762271-dc8a3574-e240af0-16d3dda5_0
2025-10-09T21:43:09.396158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9c849e0-a327932c-673cb5cb-cc256950_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2f762271-dc8a3574-e240af0-16d3dda5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:09.423599Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340822453855671:2476]
2025-10-09T21:43:09.423968Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 68, pipe: [1:7559340822453855671:2476]
2025-10-09T21:43:09.423887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340822453855671:2476], now have 1 active actors on pipe
2025-10-09T21:43:09.430950Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2f762271-dc8a3574-e240af0-16d3dda5_0] Write session: try to update token
2025-10-09T21:43:09.430969Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2f762271-dc8a3574-e240af0-16d3dda5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:09.431632Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:09.431645Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:09.431647Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:09.431650Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:09.431660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340822092047067:2767]
2025-10-09T21:43:09.431666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340822092047067:2767]
2025-10-09T21:43:09.431670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340822092047067:2767]
2025-10-09T21:43:09.431715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:09.431723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.431726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:09.432058Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:09.434715Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:09.434726Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:09.434732Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:09.466095Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 69 sessionId: src_id|2f762271-dc8a3574-e240af0-16d3dda5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:09.468951Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:09.632016Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:09.632492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:09.632503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:09.633494Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340822453855671:2476]
2025-10-09T21:43:09.633524Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:09.633556Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:09.633565Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:09.633625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 69 reason: error at write: Disconnected sessionId: src_id|2f762271-dc8a3574-e240af0-16d3dda5_0
2025-10-09T21:43:09.633981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f762271-dc8a3574-e240af0-16d3dda5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:09.633989Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f762271-dc8a3574-e240af0-16d3dda5_0] Write session will restart in 0.006832s
2025-10-09T21:43:09.634013Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f762271-dc8a3574-e240af0-16d3dda5_0] Write session: Do CDS request
2025-10-09T21:43:09.634021Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f762271-dc8a3574-e240af0-16d3dda5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:09.633816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 69 sessionId: src_id|2f762271-dc8a3574-e240af0-16d3dda5_0 is DEAD
2025-10-09T21:43:09.638215Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:09.646673Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2f762271-dc8a3574-e240af0-16d3dda5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:09.648205Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:09.648224Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 70
2025-10-09T21:43:09.648438Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 70 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:09.648471Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 70 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:09.648478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=70 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:09.648482Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:09.649233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 70 sessionId:  describe result for acl check
2025-10-09T21:43:09.649276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:09.649278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:09.649280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:09.649286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340822453855684:2925] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:09.649884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340822453855684:2925] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:09.659378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340822453855684:2925] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:09.667213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340822453855684:2925] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:09.667232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340822453855684:2925] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:09.667237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340822453855684:2925] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:09.667262Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 70 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:09.675676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:09.681046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:09.681333Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:09.681338Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:09.681354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:09.681427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:09.681471Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340822092047197:2770]
2025-10-09T21:43:09.682104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:09.682767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:09.682825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:09.683234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:09.683299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:09.683300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:09.683301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:09.683302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:09.683303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:09.683305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:09.683307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.683308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.683309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.683315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:09.683327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:09.683331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:09.683336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:09.683357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:09.683362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:09.683366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.683369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.683374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.683381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:09.683389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:09.683396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:09.683401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:09.683416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:09.683422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:09.683426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:09.683431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:09.683435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:09.683440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:09.683445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:09.683450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:09.683458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.683463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.683468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.683475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.683480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.683486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.683527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:09.697991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:09.699469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:09.699476Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:09.699480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:09.699485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:09.699493Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 69 [2:7559340822092047197:2770]
2025-10-09T21:43:09.699508Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:09.699526Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:09.699530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.699534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.699537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.699557Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:09.699563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:09.699571Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:09.699635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.699638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.699640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.699643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:09.699718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:09.699722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.699736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.699745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.699761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:09.699781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340822092047191:2770]
2025-10-09T21:43:09.699784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340822092047191:2770]
2025-10-09T21:43:09.699787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340822092047191:2770]
2025-10-09T21:43:09.699796Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:09.699814Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:09.699827Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:09.699830Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:09.699831Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:09.699833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:09.699834Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:09.699876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340822092047191:2770]
2025-10-09T21:43:09.699884Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:09.699894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340822453855676:2432], now have 1 active actors on pipe
2025-10-09T21:43:09.699898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340822453855786:2925], now have 1 active actors on pipe
2025-10-09T21:43:09.700180Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 69
2025-10-09T21:43:09.700216Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 69
2025-10-09T21:43:09.700314Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:09.701159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:09.701170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:09.701203Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|39f09283-fe1724b4-fe5b821e-fbc0d777_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:09.701214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.701218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.701221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:09.701232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.701246Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:09.701271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:09.701614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:09.701621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:09.701635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:09.701783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 70 partition: 0 MaxSeqNo: 25 sessionId: src_id|39f09283-fe1724b4-fe5b821e-fbc0d777_0
2025-10-09T21:43:09.705954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f762271-dc8a3574-e240af0-16d3dda5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|39f09283-fe1724b4-fe5b821e-fbc0d777_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:09.709245Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|39f09283-fe1724b4-fe5b821e-fbc0d777_0] Write session: try to update token
2025-10-09T21:43:09.709266Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|39f09283-fe1724b4-fe5b821e-fbc0d777_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:09.733093Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340822453855808:2476]
2025-10-09T21:43:09.733752Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 69, pipe: [1:7559340822453855808:2476]
2025-10-09T21:43:09.733273Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:09.733289Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:09.733291Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:09.733292Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:09.733300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340822092047191:2770]
2025-10-09T21:43:09.733303Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340822092047191:2770]
2025-10-09T21:43:09.733307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340822092047191:2770]
2025-10-09T21:43:09.733343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:09.733349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.733351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:09.733483Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:09.733594Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:09.733599Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:09.733603Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:09.765958Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 70 sessionId: src_id|39f09283-fe1724b4-fe5b821e-fbc0d777_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:09.768798Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:09.897410Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:09.897947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340822453855808:2476], now have 1 active actors on pipe
2025-10-09T21:43:09.898044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:09.898055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:09.898584Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:09.899297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39f09283-fe1724b4-fe5b821e-fbc0d777_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:09.899309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39f09283-fe1724b4-fe5b821e-fbc0d777_0] Write session will restart in 0.007678s
2025-10-09T21:43:09.899334Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39f09283-fe1724b4-fe5b821e-fbc0d777_0] Write session: Do CDS request
2025-10-09T21:43:09.899346Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39f09283-fe1724b4-fe5b821e-fbc0d777_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:09.898571Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:09.898645Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340822453855808:2476]
2025-10-09T21:43:09.898663Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:09.898665Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:09.898734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 70 reason: error at write: Disconnected sessionId: src_id|39f09283-fe1724b4-fe5b821e-fbc0d777_0
2025-10-09T21:43:09.898926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 70 sessionId: src_id|39f09283-fe1724b4-fe5b821e-fbc0d777_0 is DEAD
2025-10-09T21:43:09.918860Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|39f09283-fe1724b4-fe5b821e-fbc0d777_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:09.919733Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:09.919748Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 71
2025-10-09T21:43:09.919990Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 71 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:09.920022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 71 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:09.920028Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=71 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:09.920032Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:09.921301Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 71 sessionId:  describe result for acl check
2025-10-09T21:43:09.921352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:09.921354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:09.921355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:09.921362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340822453855836:2931] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:09.922022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340822453855836:2931] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:09.925154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340822453855836:2931] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:09.930290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340822453855836:2931] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:09.930302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340822453855836:2931] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:09.930306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340822453855836:2931] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:09.930315Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 71 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:09.937208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:09.938326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:09.938574Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:09.938583Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:09.938594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:09.938646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:09.938684Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340822092047322:2773]
2025-10-09T21:43:09.939276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:09.939757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:09.939792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:09.940097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:09.940157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:09.940158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:09.940160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:09.940161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:09.940162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:09.940163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:09.940164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.940166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.940166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.940171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:09.940181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:09.940184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:09.940187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:09.940190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:09.940193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:09.940197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.940200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.940203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.940210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:09.940218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:09.940224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:09.940229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:09.940233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:09.940238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:09.940242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:09.940246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:09.940251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:09.940256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:09.940260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:09.940264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:09.940269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.940273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.940278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.940282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.940286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.940290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.940319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:09.952866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:09.954162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:09.954169Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:09.954173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:09.954179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:09.954217Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 70 [2:7559340822092047322:2773]
2025-10-09T21:43:09.954232Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:09.954252Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:09.954262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.954265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.954267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.954285Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:09.954293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:09.954300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:09.954369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.954371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.954373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.954377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:09.954505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:09.954509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.954520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:09.954530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:09.954549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:09.954570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340822092047316:2773]
2025-10-09T21:43:09.954573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340822092047316:2773]
2025-10-09T21:43:09.954577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340822092047316:2773]
2025-10-09T21:43:09.954592Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:09.954602Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:09.954605Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:09.954607Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:09.954608Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:09.954622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:09.954625Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:09.954637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340822453855823:2432], now have 1 active actors on pipe
2025-10-09T21:43:09.954641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340822453855901:2931], now have 1 active actors on pipe
2025-10-09T21:43:09.954676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340822092047316:2773]
2025-10-09T21:43:09.954685Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:09.954984Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 70
2025-10-09T21:43:09.955027Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 70
2025-10-09T21:43:09.955162Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:09.955261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:09.955272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:09.955301Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|92545914-f414f6be-81887574-95a7404e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:09.955316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.955320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.955323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:09.955335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.955347Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:09.955364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:09.955690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:09.955699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:09.955714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:09.955802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 71 partition: 0 MaxSeqNo: 25 sessionId: src_id|92545914-f414f6be-81887574-95a7404e_0
2025-10-09T21:43:09.957577Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39f09283-fe1724b4-fe5b821e-fbc0d777_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|92545914-f414f6be-81887574-95a7404e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:09.960963Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92545914-f414f6be-81887574-95a7404e_0] Write session: try to update token
2025-10-09T21:43:09.960983Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92545914-f414f6be-81887574-95a7404e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:09.975623Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:09.975641Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:09.975643Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:09.975645Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:09.975655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340822092047316:2773]
2025-10-09T21:43:09.975660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340822092047316:2773]
2025-10-09T21:43:09.975664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340822092047316:2773]
2025-10-09T21:43:09.975709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:09.975717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:09.975721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:09.975791Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:09.975798Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:09.975801Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:09.976280Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:09.998547Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340822453855956:2476]
2025-10-09T21:43:09.998772Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 70, pipe: [1:7559340822453855956:2476]
2025-10-09T21:43:10.012204Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 71 sessionId: src_id|92545914-f414f6be-81887574-95a7404e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:10.015214Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:10.159593Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:10.159903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340822453855956:2476], now have 1 active actors on pipe
2025-10-09T21:43:10.159930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:10.159937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:10.160320Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:10.160340Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:10.160412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 71 reason: error at write: Disconnected sessionId: src_id|92545914-f414f6be-81887574-95a7404e_0
2025-10-09T21:43:10.160608Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:10.160608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 71 sessionId: src_id|92545914-f414f6be-81887574-95a7404e_0 is DEAD
2025-10-09T21:43:10.160651Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340822453855956:2476]
2025-10-09T21:43:10.161015Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:10.162127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92545914-f414f6be-81887574-95a7404e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:10.162141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92545914-f414f6be-81887574-95a7404e_0] Write session will restart in 0.005496s
2025-10-09T21:43:10.162169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92545914-f414f6be-81887574-95a7404e_0] Write session: Do CDS request
2025-10-09T21:43:10.162177Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92545914-f414f6be-81887574-95a7404e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:10.169074Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92545914-f414f6be-81887574-95a7404e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:10.169099Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:10.169116Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 72
2025-10-09T21:43:10.169459Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 72 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:10.169494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 72 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:10.169504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=72 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:10.169508Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:10.170563Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 72 sessionId:  describe result for acl check
2025-10-09T21:43:10.170596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:10.170598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:10.170600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:10.170605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340826748823276:2935] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:10.171091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340826748823276:2935] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:10.174842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340826748823276:2935] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:10.179244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340826748823276:2935] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:10.179256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340826748823276:2935] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:10.179259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340826748823276:2935] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:10.179267Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 72 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:10.181708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:10.182755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:10.182939Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:10.182942Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:10.182953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:10.182998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:10.183030Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340826387014742:2775]
2025-10-09T21:43:10.183656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:10.184128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:10.184169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:10.184493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:10.184530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:10.184531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:10.184532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:10.184534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:10.184535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:10.184536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:10.184537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.184539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.184540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.184544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:10.184556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:10.184560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:10.184564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:10.184568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:10.184572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:10.184575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.184579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.184583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.184590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:10.184598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:10.184604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:10.184610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:10.184615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:10.184621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:10.184626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:10.184632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:10.184637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:10.184642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:10.184647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:10.184653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:10.184659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.184664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.184669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.184674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.184680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.184686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.184717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:10.194827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:10.196160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:10.196166Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:10.196169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:10.196173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:10.196180Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 71 [2:7559340826387014742:2775]
2025-10-09T21:43:10.196197Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:10.196216Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:10.196220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.196223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.196225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.196242Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:10.196248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:10.196255Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:10.196310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.196312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.196315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.196318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:10.196385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:10.196387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.196398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.196408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.196424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:10.196444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340826387014736:2775]
2025-10-09T21:43:10.196446Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340826387014736:2775]
2025-10-09T21:43:10.196448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340826387014736:2775]
2025-10-09T21:43:10.196461Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:10.196476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:10.196489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:10.196492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:10.196525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340826387014736:2775]
2025-10-09T21:43:10.196527Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:10.198399Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:10.198409Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:10.198412Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:10.198429Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:10.199877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340826748823254:2432], now have 1 active actors on pipe
2025-10-09T21:43:10.199894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340826748823368:2935], now have 1 active actors on pipe
2025-10-09T21:43:10.201915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:10.201923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:10.201957Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|597b2433-6ed9e72b-a236374b-b4cf5ea5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:10.201968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.201973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.201977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:10.201990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.202006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:10.202053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:10.201313Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 71
2025-10-09T21:43:10.201369Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 71
2025-10-09T21:43:10.202445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:10.202452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:10.202468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:10.202604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 72 partition: 0 MaxSeqNo: 25 sessionId: src_id|597b2433-6ed9e72b-a236374b-b4cf5ea5_0
2025-10-09T21:43:10.203051Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92545914-f414f6be-81887574-95a7404e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|597b2433-6ed9e72b-a236374b-b4cf5ea5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:10.206406Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|597b2433-6ed9e72b-a236374b-b4cf5ea5_0] Write session: try to update token
2025-10-09T21:43:10.206448Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|597b2433-6ed9e72b-a236374b-b4cf5ea5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:10.260871Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:10.260896Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:10.260897Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:10.260899Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:10.260907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340826387014736:2775]
2025-10-09T21:43:10.260913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340826387014736:2775]
2025-10-09T21:43:10.260915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340826387014736:2775]
2025-10-09T21:43:10.260956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:10.260962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.260965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:10.261121Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:10.262287Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:10.262294Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:10.262297Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:10.262176Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 72 sessionId: src_id|597b2433-6ed9e72b-a236374b-b4cf5ea5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:10.273852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340826748823401:2476]
2025-10-09T21:43:10.288041Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:10.288578Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 71, pipe: [1:7559340826748823401:2476]
2025-10-09T21:43:10.447916Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:10.448342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:10.448353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:10.448376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340826748823401:2476], now have 1 active actors on pipe
2025-10-09T21:43:10.449894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|597b2433-6ed9e72b-a236374b-b4cf5ea5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:10.449909Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|597b2433-6ed9e72b-a236374b-b4cf5ea5_0] Write session will restart in 0.005189s
2025-10-09T21:43:10.449325Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:10.449932Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|597b2433-6ed9e72b-a236374b-b4cf5ea5_0] Write session: Do CDS request
2025-10-09T21:43:10.449371Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:10.449456Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 72 reason: status is not ok: tablet will be restarted right now sessionId: src_id|597b2433-6ed9e72b-a236374b-b4cf5ea5_0
2025-10-09T21:43:10.449939Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|597b2433-6ed9e72b-a236374b-b4cf5ea5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:10.449708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 72 sessionId: src_id|597b2433-6ed9e72b-a236374b-b4cf5ea5_0 is DEAD
2025-10-09T21:43:10.449796Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340826748823401:2476]
2025-10-09T21:43:10.448732Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:10.456748Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|597b2433-6ed9e72b-a236374b-b4cf5ea5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:10.456686Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:10.456698Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 73
2025-10-09T21:43:10.457016Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 73 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:10.457058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 73 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:10.457064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=73 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:10.457067Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:10.457535Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 73 sessionId:  describe result for acl check
2025-10-09T21:43:10.457563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:10.457565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:10.457566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:10.457573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340826748823431:2947] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:10.458599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340826748823431:2947] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:10.464960Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340826748823431:2947] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:10.470203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:10.471181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340826748823431:2947] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:10.471194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340826748823431:2947] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:10.471198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340826748823431:2947] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:10.471207Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 73 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:10.471252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:10.471482Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:10.471486Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:10.471498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:10.471547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:10.471580Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340826387014906:2789]
2025-10-09T21:43:10.472074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:10.474621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:10.474675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:10.474980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:10.475018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:10.475019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:10.475020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:10.475020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:10.475021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:10.475022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:10.475022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.475023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.475023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.475028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:10.475035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:10.475038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:10.475040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:10.475042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:10.475044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:10.475046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.475048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.475050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.475056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:10.475061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:10.475065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:10.475068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:10.475071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:10.475074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:10.475078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:10.475081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:10.475084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:10.475087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:10.475089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:10.475095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:10.475100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.475105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.475110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.475115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.475119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.475122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.475138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:10.488043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:10.489270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:10.489276Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:10.489278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:10.489283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:10.489291Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 72 [2:7559340826387014906:2789]
2025-10-09T21:43:10.489308Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:10.489326Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:10.489329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.489332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.489334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.489353Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:10.489358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:10.489367Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:10.489427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.489429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.489431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.489433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:10.489505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:10.489508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.489519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.489529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.489545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:10.489564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340826387014901:2789]
2025-10-09T21:43:10.489567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340826387014901:2789]
2025-10-09T21:43:10.489568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340826387014901:2789]
2025-10-09T21:43:10.489581Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:10.489596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:10.489609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:10.489612Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:10.489650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340826387014901:2789]
2025-10-09T21:43:10.489652Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:10.494202Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:10.494213Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:10.494215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:10.494223Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:10.494380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340826748823540:2947], now have 1 active actors on pipe
2025-10-09T21:43:10.494387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340826748823417:2432], now have 1 active actors on pipe
2025-10-09T21:43:10.495100Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 72
2025-10-09T21:43:10.495130Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 72
2025-10-09T21:43:10.498365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:10.498379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:10.498439Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c2514c90-cdd435c0-7afc13eb-50565f0b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:10.498452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.498457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.498461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:10.498475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.498493Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:10.498531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:10.499076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:10.499087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:10.499106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:10.499264Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 73 partition: 0 MaxSeqNo: 25 sessionId: src_id|c2514c90-cdd435c0-7afc13eb-50565f0b_0
2025-10-09T21:43:10.499727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|597b2433-6ed9e72b-a236374b-b4cf5ea5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c2514c90-cdd435c0-7afc13eb-50565f0b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:10.503462Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c2514c90-cdd435c0-7afc13eb-50565f0b_0] Write session: try to update token
2025-10-09T21:43:10.503482Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c2514c90-cdd435c0-7afc13eb-50565f0b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:10.546328Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:10.546348Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:10.546351Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:10.546353Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:10.546363Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340826387014901:2789]
2025-10-09T21:43:10.546367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340826387014901:2789]
2025-10-09T21:43:10.546371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340826387014901:2789]
2025-10-09T21:43:10.546424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:10.546433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.546436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:10.546591Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:10.549411Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 73 sessionId: src_id|c2514c90-cdd435c0-7afc13eb-50565f0b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:10.552111Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:10.552182Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340826748823550:2476]
2025-10-09T21:43:10.552316Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:10.552328Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:10.552332Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:10.552553Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 72, pipe: [1:7559340826748823550:2476]
2025-10-09T21:43:10.742785Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:10.743187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:10.743198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:10.743221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340826748823550:2476], now have 1 active actors on pipe
2025-10-09T21:43:10.743838Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:10.743889Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340826748823550:2476]
2025-10-09T21:43:10.743910Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:10.743917Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:10.743610Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:10.743982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 73 reason: error at write: Disconnected sessionId: src_id|c2514c90-cdd435c0-7afc13eb-50565f0b_0
2025-10-09T21:43:10.744155Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 73 sessionId: src_id|c2514c90-cdd435c0-7afc13eb-50565f0b_0 is DEAD
2025-10-09T21:43:10.746550Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2514c90-cdd435c0-7afc13eb-50565f0b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:10.746566Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2514c90-cdd435c0-7afc13eb-50565f0b_0] Write session will restart in 0.006593s
2025-10-09T21:43:10.746593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2514c90-cdd435c0-7afc13eb-50565f0b_0] Write session: Do CDS request
2025-10-09T21:43:10.746600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2514c90-cdd435c0-7afc13eb-50565f0b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:10.754732Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c2514c90-cdd435c0-7afc13eb-50565f0b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:10.756137Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:10.756154Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 74
2025-10-09T21:43:10.756282Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 74 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:10.756321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 74 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:10.756327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=74 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:10.756330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:10.756813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 74 sessionId:  describe result for acl check
2025-10-09T21:43:10.756856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:10.756862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:10.756864Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:10.756871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340826748823649:2953] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:10.757472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340826748823649:2953] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:10.758669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:10.760032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:10.760259Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:10.760269Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:10.760281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:10.760355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:10.760404Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340826387015038:2792]
2025-10-09T21:43:10.761055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:10.761390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:10.761450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:10.761600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340826748823649:2953] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:10.761812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:10.761866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:10.761872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:10.761874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:10.761875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:10.761876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:10.761878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:10.761879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.761880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.761882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.761887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:10.761899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:10.761904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:10.761908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:10.761912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:10.761917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:10.761921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.761926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.761930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.761937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:10.761967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:10.761980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:10.761986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:10.761991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:10.761997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:10.762002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:10.762008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:10.762014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:10.762020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:10.762024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:10.762044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:10.762049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.762060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.762065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.762070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.762074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.762079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.762108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:10.769580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340826748823649:2953] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:10.769598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340826748823649:2953] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:10.769603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340826748823649:2953] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:10.769617Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 74 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:10.774222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 73
2025-10-09T21:43:10.774261Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 73
2025-10-09T21:43:10.778403Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 74 partition: 0 MaxSeqNo: 25 sessionId: src_id|6773f543-ab61ce05-a1adbfb2-ea80cee9_0
2025-10-09T21:43:10.778940Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2514c90-cdd435c0-7afc13eb-50565f0b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6773f543-ab61ce05-a1adbfb2-ea80cee9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:10.770235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:10.771979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:10.771986Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:10.771988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:10.771993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:10.772000Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 73 [2:7559340826387015038:2792]
2025-10-09T21:43:10.772016Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:10.772037Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:10.772040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.772044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.772046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.772065Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:10.772071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:10.772080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:10.772143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.772145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.772146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.772149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:10.772231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:10.772234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.772246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:10.772280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:10.772297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:10.772319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340826748823689:2953], now have 1 active actors on pipe
2025-10-09T21:43:10.772323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340826748823557:2432], now have 1 active actors on pipe
2025-10-09T21:43:10.772332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340826387015033:2792]
2025-10-09T21:43:10.772334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340826387015033:2792]
2025-10-09T21:43:10.772336Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340826387015033:2792]
2025-10-09T21:43:10.772351Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:10.772368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:10.772377Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:10.772380Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:10.772382Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:10.772385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:10.772388Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:10.772426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340826387015033:2792]
2025-10-09T21:43:10.772432Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:10.772435Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:10.774488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:10.774498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:10.774534Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6773f543-ab61ce05-a1adbfb2-ea80cee9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:10.774546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.774550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.774553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:10.774577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.774590Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:10.774610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:10.775392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:10.775399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:10.775421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:10.789857Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6773f543-ab61ce05-a1adbfb2-ea80cee9_0] Write session: try to update token
2025-10-09T21:43:10.789888Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6773f543-ab61ce05-a1adbfb2-ea80cee9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:10.810239Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:10.810257Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:10.810259Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:10.810261Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:10.810271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340826387015033:2792]
2025-10-09T21:43:10.810275Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340826387015033:2792]
2025-10-09T21:43:10.810280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340826387015033:2792]
2025-10-09T21:43:10.810325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:10.810337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:10.810340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:10.810540Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:10.813124Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:10.813137Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:10.813141Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:10.840498Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 74 sessionId: src_id|6773f543-ab61ce05-a1adbfb2-ea80cee9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:10.843375Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:10.843894Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340826748823701:2476]
2025-10-09T21:43:10.844259Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 73, pipe: [1:7559340826748823701:2476]
2025-10-09T21:43:10.994328Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:10.994731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:10.994741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:10.994766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340826748823701:2476], now have 1 active actors on pipe
2025-10-09T21:43:10.995193Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:10.995395Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340826748823701:2476]
2025-10-09T21:43:10.995422Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:10.995505Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 74 reason: status is not ok: tablet will be restarted right now sessionId: src_id|6773f543-ab61ce05-a1adbfb2-ea80cee9_0
2025-10-09T21:43:10.995682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 74 sessionId: src_id|6773f543-ab61ce05-a1adbfb2-ea80cee9_0 is DEAD
2025-10-09T21:43:10.995743Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:10.997545Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6773f543-ab61ce05-a1adbfb2-ea80cee9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:10.997557Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6773f543-ab61ce05-a1adbfb2-ea80cee9_0] Write session will restart in 0.007776s
2025-10-09T21:43:10.997585Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6773f543-ab61ce05-a1adbfb2-ea80cee9_0] Write session: Do CDS request
2025-10-09T21:43:10.997591Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6773f543-ab61ce05-a1adbfb2-ea80cee9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:11.006700Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6773f543-ab61ce05-a1adbfb2-ea80cee9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:11.007336Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:11.007362Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 75
2025-10-09T21:43:11.007498Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 75 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:11.007526Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 75 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:11.007534Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=75 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:11.007539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:11.007918Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 75 sessionId:  describe result for acl check
2025-10-09T21:43:11.007957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:11.007959Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:11.007960Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:11.007967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340831043791026:2957] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:11.008410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340831043791026:2957] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:11.011874Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340831043791026:2957] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:11.017738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340831043791026:2957] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:11.017753Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340831043791026:2957] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:11.017757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340831043791026:2957] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:11.017767Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 75 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:11.025032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:11.026280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:11.026489Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:11.026493Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:11.026504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:11.026551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:11.026589Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340830681982468:2795]
2025-10-09T21:43:11.027005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:11.027523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:11.027575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:11.027901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:11.027962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:11.027964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:11.027965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:11.027966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:11.027967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:11.027968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:11.027970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.027971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.027973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.027977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:11.027984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:11.027987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:11.027989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:11.027991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:11.027993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:11.027995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.027997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.027999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.028004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:11.028011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:11.028017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:11.028023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:11.028027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:11.028031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:11.028036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:11.028042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:11.028046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:11.028051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:11.028054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:11.028056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:11.028060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.028063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.028065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.028068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.028071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.028074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.028090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:11.038609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:11.039796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:11.039814Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:11.039817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:11.039822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:11.039833Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 74 [2:7559340830681982468:2795]
2025-10-09T21:43:11.039850Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:11.039873Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:11.039881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.039885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.039888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.039909Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:11.039919Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:11.039928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:11.040001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.040008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.040010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.040013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:11.040394Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 74
2025-10-09T21:43:11.040448Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 74
2025-10-09T21:43:11.040090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:11.040096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.040108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.040118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.040144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:11.040166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340830681982462:2795]
2025-10-09T21:43:11.040168Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340830681982462:2795]
2025-10-09T21:43:11.040170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340830681982462:2795]
2025-10-09T21:43:11.040178Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:11.040187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:11.040191Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:11.040193Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:11.040195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:11.040209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:11.040213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:11.040225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340826748823706:2432], now have 1 active actors on pipe
2025-10-09T21:43:11.040228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340831043791076:2957], now have 1 active actors on pipe
2025-10-09T21:43:11.040268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340830681982462:2795]
2025-10-09T21:43:11.040274Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:11.040624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:11.045666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:11.045685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:11.045723Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|23a6f6ca-efa1f8a2-e39b8076-f9b77e7a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:11.045737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.045741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.045744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:11.045759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.045778Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:11.045797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:11.046442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:11.046455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:11.046478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:11.047100Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6773f543-ab61ce05-a1adbfb2-ea80cee9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|23a6f6ca-efa1f8a2-e39b8076-f9b77e7a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:11.046637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 75 partition: 0 MaxSeqNo: 25 sessionId: src_id|23a6f6ca-efa1f8a2-e39b8076-f9b77e7a_0
2025-10-09T21:43:11.050768Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23a6f6ca-efa1f8a2-e39b8076-f9b77e7a_0] Write session: try to update token
2025-10-09T21:43:11.050788Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23a6f6ca-efa1f8a2-e39b8076-f9b77e7a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:11.065637Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:11.065653Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:11.065656Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:11.065658Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:11.065669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340830681982462:2795]
2025-10-09T21:43:11.065674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340830681982462:2795]
2025-10-09T21:43:11.065678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340830681982462:2795]
2025-10-09T21:43:11.065722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:11.065730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.065732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:11.065890Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:11.070303Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:11.070314Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:11.070317Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:11.084806Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 75 sessionId: src_id|23a6f6ca-efa1f8a2-e39b8076-f9b77e7a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:11.087848Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:11.095285Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340831043791136:2476]
2025-10-09T21:43:11.095894Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 74, pipe: [1:7559340831043791136:2476]
2025-10-09T21:43:11.303429Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:11.303487Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340831043791136:2476]
2025-10-09T21:43:11.303507Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:11.303510Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:11.303567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 75 reason: error at write: Disconnected sessionId: src_id|23a6f6ca-efa1f8a2-e39b8076-f9b77e7a_0
2025-10-09T21:43:11.303757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 75 sessionId: src_id|23a6f6ca-efa1f8a2-e39b8076-f9b77e7a_0 is DEAD
2025-10-09T21:43:11.302307Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:11.302816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:11.302826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:11.302849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340831043791136:2476], now have 1 active actors on pipe
2025-10-09T21:43:11.303110Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:11.306377Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23a6f6ca-efa1f8a2-e39b8076-f9b77e7a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:11.306389Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23a6f6ca-efa1f8a2-e39b8076-f9b77e7a_0] Write session will restart in 0.006196s
2025-10-09T21:43:11.306429Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23a6f6ca-efa1f8a2-e39b8076-f9b77e7a_0] Write session: Do CDS request
2025-10-09T21:43:11.306449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23a6f6ca-efa1f8a2-e39b8076-f9b77e7a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:11.316793Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23a6f6ca-efa1f8a2-e39b8076-f9b77e7a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:11.316958Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:11.316975Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 76
2025-10-09T21:43:11.322154Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 76 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:11.322209Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 76 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:11.322216Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=76 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:11.322223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:11.323086Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 76 sessionId:  describe result for acl check
2025-10-09T21:43:11.323125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:11.323127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:11.323129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:11.323135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340831043791218:2961] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:11.323725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340831043791218:2961] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:11.326477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:11.327366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:11.327611Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:11.327615Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:11.327626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:11.327676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:11.327710Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340830681982598:2797]
2025-10-09T21:43:11.328243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:11.334256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:11.334337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:11.334636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340831043791218:2961] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:11.334830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:11.334890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:11.334896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:11.334897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:11.334898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:11.334899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:11.334901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:11.334902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.334903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.334904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.334909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:11.334918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:11.334922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:11.334925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:11.334929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:11.334933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:11.334936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.334939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.334942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.334948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:11.334956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:11.334961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:11.334967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:11.334971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:11.334975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:11.334978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:11.334982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:11.334996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:11.335002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:11.335006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:11.335010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:11.335015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.335019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.335028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.335032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.335037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.335041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.335067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:11.341940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340831043791218:2961] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:11.341957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340831043791218:2961] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:11.341961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340831043791218:2961] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:11.341970Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 76 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:11.351027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:11.352344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:11.352350Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:11.352353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:11.352357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:11.352366Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 75 [2:7559340830681982598:2797]
2025-10-09T21:43:11.352380Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:11.352398Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:11.352402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.352405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.352408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.352426Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:11.352433Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:11.352441Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:11.352502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.352504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.352506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.352509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:11.352587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:11.352589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.352601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.352613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.352629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:11.352651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340830681982592:2797]
2025-10-09T21:43:11.352653Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340830681982592:2797]
2025-10-09T21:43:11.352656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340830681982592:2797]
2025-10-09T21:43:11.352670Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:11.352685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:11.352701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:11.352704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:11.352747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340830681982592:2797]
2025-10-09T21:43:11.352750Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:11.354498Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:11.354508Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:11.354510Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:11.354518Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:11.354539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340831043791282:2961], now have 1 active actors on pipe
2025-10-09T21:43:11.354556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340831043791140:2432], now have 1 active actors on pipe
2025-10-09T21:43:11.355444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:11.355458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:11.355497Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5810b921-cd6d4570-7a46698e-6acfe41b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:11.355509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.355514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.355517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:11.355529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.355545Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:11.355564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:11.355841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:11.355848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:11.355862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:11.354977Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 75
2025-10-09T21:43:11.355009Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 75
2025-10-09T21:43:11.358168Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 76 partition: 0 MaxSeqNo: 25 sessionId: src_id|5810b921-cd6d4570-7a46698e-6acfe41b_0
2025-10-09T21:43:11.358638Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23a6f6ca-efa1f8a2-e39b8076-f9b77e7a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5810b921-cd6d4570-7a46698e-6acfe41b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:11.361816Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5810b921-cd6d4570-7a46698e-6acfe41b_0] Write session: try to update token
2025-10-09T21:43:11.361840Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5810b921-cd6d4570-7a46698e-6acfe41b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:11.395377Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:11.395395Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:11.395397Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:11.395399Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:11.395408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340830681982592:2797]
2025-10-09T21:43:11.395413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340830681982592:2797]
2025-10-09T21:43:11.395416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340830681982592:2797]
2025-10-09T21:43:11.395463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:11.395466Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:11.395472Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:11.395475Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:11.395475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.395478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:11.395645Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:11.395478Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 76 sessionId: src_id|5810b921-cd6d4570-7a46698e-6acfe41b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:11.398522Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:11.403495Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340831043791302:2476]
2025-10-09T21:43:11.403867Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 75, pipe: [1:7559340831043791302:2476]
2025-10-09T21:43:11.601598Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:11.601881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:11.601894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:11.601918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340831043791302:2476], now have 1 active actors on pipe
2025-10-09T21:43:11.602368Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340831043791302:2476]
2025-10-09T21:43:11.602406Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:11.602409Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:11.602483Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 76 reason: error at write: Disconnected sessionId: src_id|5810b921-cd6d4570-7a46698e-6acfe41b_0
2025-10-09T21:43:11.602610Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 76 sessionId: src_id|5810b921-cd6d4570-7a46698e-6acfe41b_0 is DEAD
2025-10-09T21:43:11.602912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5810b921-cd6d4570-7a46698e-6acfe41b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:11.602929Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5810b921-cd6d4570-7a46698e-6acfe41b_0] Write session will restart in 0.006510s
2025-10-09T21:43:11.602955Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5810b921-cd6d4570-7a46698e-6acfe41b_0] Write session: Do CDS request
2025-10-09T21:43:11.602974Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5810b921-cd6d4570-7a46698e-6acfe41b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:11.603269Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:11.603823Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:11.610869Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5810b921-cd6d4570-7a46698e-6acfe41b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:11.614284Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:11.614297Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 77
2025-10-09T21:43:11.614676Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 77 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:11.614708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 77 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:11.614715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=77 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:11.614719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:11.615866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 77 sessionId:  describe result for acl check
2025-10-09T21:43:11.615905Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:11.615908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:11.615909Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:11.615916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340831043791329:2975] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:11.616485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340831043791329:2975] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:11.627924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:11.629856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:11.630087Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:11.630092Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:11.630104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:11.630160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:11.630197Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340830681982770:2811]
2025-10-09T21:43:11.630821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:11.632643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340831043791329:2975] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:11.633358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:11.633412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:11.633758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:11.633801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:11.633802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:11.633804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:11.633805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:11.633806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:11.633807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:11.633809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.633810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.633811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.633816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:11.633828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:11.633833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:11.633837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:11.633842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:11.633846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:11.633850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.633854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.633859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.633866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:11.633875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:11.633882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:11.633888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:11.633893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:11.633899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:11.633903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:11.633909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:11.633913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:11.633918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:11.633922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:11.633927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:11.633933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.633938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.633942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.633947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.633951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.633956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.633982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:11.647196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340831043791329:2975] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:11.647212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340831043791329:2975] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:11.647217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340831043791329:2975] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:11.647228Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 77 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:11.650746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:11.651884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:11.651899Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:11.651901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:11.651906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:11.651913Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 76 [2:7559340830681982770:2811]
2025-10-09T21:43:11.651937Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:11.651967Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:11.651970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.651974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.651976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.651995Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:11.652001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:11.652008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:11.652069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.652071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.652073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.652077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:11.652151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:11.652154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.652167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.652185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.652203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:11.652224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340830681982764:2811]
2025-10-09T21:43:11.652227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340830681982764:2811]
2025-10-09T21:43:11.652230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340830681982764:2811]
2025-10-09T21:43:11.652238Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:11.652249Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:11.652252Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:11.652254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340831043791438:2975], now have 1 active actors on pipe
2025-10-09T21:43:11.652256Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:11.652258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340831043791305:2432], now have 1 active actors on pipe
2025-10-09T21:43:11.652267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:11.652282Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:11.652285Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:11.652324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340830681982764:2811]
2025-10-09T21:43:11.652332Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:11.652348Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:11.654326Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 76
2025-10-09T21:43:11.654362Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 76
2025-10-09T21:43:11.654618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:11.654631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:11.654683Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d6d9014e-836a0f3a-c42806fd-ea1e2982_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:11.654694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.654699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.654702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:11.654714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.654727Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:11.654756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:11.655363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:11.655368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:11.655395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:11.657815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 77 partition: 0 MaxSeqNo: 25 sessionId: src_id|d6d9014e-836a0f3a-c42806fd-ea1e2982_0
2025-10-09T21:43:11.658393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5810b921-cd6d4570-7a46698e-6acfe41b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d6d9014e-836a0f3a-c42806fd-ea1e2982_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:11.661453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d6d9014e-836a0f3a-c42806fd-ea1e2982_0] Write session: try to update token
2025-10-09T21:43:11.661477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d6d9014e-836a0f3a-c42806fd-ea1e2982_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:11.683975Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 77 sessionId: src_id|d6d9014e-836a0f3a-c42806fd-ea1e2982_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:11.686404Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:11.686696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:11.686731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:11.686777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:11.686917Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:11.698502Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:11.698518Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:11.698521Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:11.698524Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:11.698534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340830681982764:2811]
2025-10-09T21:43:11.698539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340830681982764:2811]
2025-10-09T21:43:11.698542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340830681982764:2811]
2025-10-09T21:43:11.698589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:11.698602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.698604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:11.698789Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:11.700368Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:11.700381Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:11.700386Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:11.702498Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340831043791447:2476]
2025-10-09T21:43:11.702789Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 76, pipe: [1:7559340831043791447:2476]
2025-10-09T21:43:11.873626Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:11.873961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340831043791447:2476], now have 1 active actors on pipe
2025-10-09T21:43:11.881812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:11.881838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:11.881995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:11.882087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:11.882152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:11.882210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:11.882622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:11.882634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:11.882696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:11.882755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:11.882821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:11.882929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:11.882971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:11.882980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:11.883081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:11.883172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:11.883231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:11.883288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:11.883331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:11.883339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:11.883406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:11.883562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:11.883667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:11.883785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:11.884093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:11.884102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:11.884168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:11.884262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:11.884329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:11.884417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:11.884465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:11.884472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:11.884537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:11.884595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:11.884652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:11.884740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:11.884788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:11.884794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:11.885676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:11.885783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:11.885886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:11.885990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:11.886065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:11.886074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:11.886176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:11.886267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:11.886338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:11.886396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:11.886477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:11.886480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:11.886551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:11.886613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:11.886677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:11.886743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:11.886804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:11.886807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:11.888194Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:11.886870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:11.888210Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:11.886923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:11.886983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:11.887041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:11.887080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:11.887083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:11.888242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 77 reason: error at write: Disconnected sessionId: src_id|d6d9014e-836a0f3a-c42806fd-ea1e2982_0
2025-10-09T21:43:11.888428Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 77 sessionId: src_id|d6d9014e-836a0f3a-c42806fd-ea1e2982_0 is DEAD
2025-10-09T21:43:11.888445Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340831043791447:2476]
2025-10-09T21:43:11.888466Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:11.888797Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6d9014e-836a0f3a-c42806fd-ea1e2982_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:11.888806Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6d9014e-836a0f3a-c42806fd-ea1e2982_0] Write session will restart in 0.008380s
2025-10-09T21:43:11.888830Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6d9014e-836a0f3a-c42806fd-ea1e2982_0] Write session: Do CDS request
2025-10-09T21:43:11.888836Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6d9014e-836a0f3a-c42806fd-ea1e2982_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:11.888980Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:11.899854Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d6d9014e-836a0f3a-c42806fd-ea1e2982_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:11.900398Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:11.900412Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 78
2025-10-09T21:43:11.901082Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 78 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:11.901148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 78 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:11.901155Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=78 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:11.901159Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:11.901615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 78 sessionId:  describe result for acl check
2025-10-09T21:43:11.901655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:11.901663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:11.901664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:11.901670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340831043791472:2981] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:11.902206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340831043791472:2981] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:11.911805Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340831043791472:2981] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:11.916393Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340831043791472:2981] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:11.916411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340831043791472:2981] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:11.916416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340831043791472:2981] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:11.916429Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 78 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:11.929061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:11.929450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:11.929658Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:11.929663Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:11.929675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:11.929728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:11.929774Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340830681982906:2815]
2025-10-09T21:43:11.930392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:11.931361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:11.931412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:11.931760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:11.931820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:11.931826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:11.931827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:11.931828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:11.931830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:11.931831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:11.931832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.931833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.931834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.931840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:11.931851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:11.931855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:11.931860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:11.931863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:11.931868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:11.931871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.931875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.931879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.931886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:11.931898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:11.931909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:11.931919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:11.931924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:11.931934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:11.931940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:11.931951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:11.931957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:11.931962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:11.931971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:11.931977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:11.931986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.931992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.932000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.932006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.932015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.932021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.932056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:11.940874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:11.942232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:11.942238Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:11.942241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:11.942248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:11.942256Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 77 [2:7559340830681982906:2815]
2025-10-09T21:43:11.942271Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:11.942291Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:11.942295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.942298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.942301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.942318Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:11.942325Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:11.942332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:11.942392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.942396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.942398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.942401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:11.942491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:11.942493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.942507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:11.942520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:11.942537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:11.942557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340830681982900:2815]
2025-10-09T21:43:11.942559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340830681982900:2815]
2025-10-09T21:43:11.942560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340830681982900:2815]
2025-10-09T21:43:11.942577Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:11.942592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:11.942608Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:11.942612Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:11.942638Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:11.942644Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:11.942646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:11.942652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340830681982900:2815]
2025-10-09T21:43:11.942655Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:11.942683Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:11.942701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340831043791517:2981], now have 1 active actors on pipe
2025-10-09T21:43:11.942705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340831043791462:2432], now have 1 active actors on pipe
2025-10-09T21:43:11.943213Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 77
2025-10-09T21:43:11.943244Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 77
2025-10-09T21:43:11.945685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:11.945701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:11.945749Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|15364b7a-5ef9ce92-8ff892fd-1667ceb0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:11.945763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.945769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.945772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:11.945786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.945801Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:11.945820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:11.946487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:11.946497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:11.946518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:11.946699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 78 partition: 0 MaxSeqNo: 25 sessionId: src_id|15364b7a-5ef9ce92-8ff892fd-1667ceb0_0
2025-10-09T21:43:11.947137Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6d9014e-836a0f3a-c42806fd-ea1e2982_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|15364b7a-5ef9ce92-8ff892fd-1667ceb0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:11.950289Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15364b7a-5ef9ce92-8ff892fd-1667ceb0_0] Write session: try to update token
2025-10-09T21:43:11.950309Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15364b7a-5ef9ce92-8ff892fd-1667ceb0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:11.966302Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:11.966320Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:11.966323Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:11.966325Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:11.966335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340830681982900:2815]
2025-10-09T21:43:11.966340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340830681982900:2815]
2025-10-09T21:43:11.966344Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340830681982900:2815]
2025-10-09T21:43:11.966398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:11.966406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:11.966409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:11.966594Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:11.969287Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:11.969298Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:11.969304Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:11.971682Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 78 sessionId: src_id|15364b7a-5ef9ce92-8ff892fd-1667ceb0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:11.974201Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:11.988738Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340831043791600:2476]
2025-10-09T21:43:11.992399Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 77, pipe: [1:7559340831043791600:2476]
2025-10-09T21:43:12.171416Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:12.171851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:12.171863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:12.171910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340831043791600:2476], now have 1 active actors on pipe
2025-10-09T21:43:12.172169Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:12.174111Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:12.174841Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15364b7a-5ef9ce92-8ff892fd-1667ceb0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:12.174851Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15364b7a-5ef9ce92-8ff892fd-1667ceb0_0] Write session will restart in 0.009066s
2025-10-09T21:43:12.174869Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15364b7a-5ef9ce92-8ff892fd-1667ceb0_0] Write session: Do CDS request
2025-10-09T21:43:12.174875Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15364b7a-5ef9ce92-8ff892fd-1667ceb0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:12.174201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 78 reason: status is not ok: tablet will be restarted right now sessionId: src_id|15364b7a-5ef9ce92-8ff892fd-1667ceb0_0
2025-10-09T21:43:12.174370Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 78 sessionId: src_id|15364b7a-5ef9ce92-8ff892fd-1667ceb0_0 is DEAD
2025-10-09T21:43:12.174556Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340831043791600:2476]
2025-10-09T21:43:12.174569Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:12.185553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15364b7a-5ef9ce92-8ff892fd-1667ceb0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:12.185666Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:12.185681Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 79
2025-10-09T21:43:12.185943Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 79 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:12.186008Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 79 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:12.186021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=79 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:12.186045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:12.186573Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 79 sessionId:  describe result for acl check
2025-10-09T21:43:12.186613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:12.186620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:12.186624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:12.186630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340835338758951:2985] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:12.187228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340835338758951:2985] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:12.189927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340835338758951:2985] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:12.189844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:12.190667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:12.190872Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:12.190876Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:12.190887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:12.190937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:12.190972Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340834976950336:2817]
2025-10-09T21:43:12.191546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:12.191916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:12.191985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:12.192320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:12.192393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:12.192395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:12.192396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:12.192397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:12.192398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:12.192399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:12.192401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.192402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.192403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.192409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:12.192419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:12.192423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:12.192428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:12.192432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:12.192436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:12.192440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.192445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.192449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.192457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:12.192465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:12.192471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:12.192478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:12.192484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:12.192490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:12.192494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:12.192499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:12.192504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:12.192509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:12.192514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:12.192519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:12.192524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.192530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.192534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.192539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.192556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.192562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.192593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:12.194614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340835338758951:2985] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:12.194628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340835338758951:2985] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:12.194635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340835338758951:2985] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:12.194645Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 79 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:12.204514Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 78
2025-10-09T21:43:12.204520Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 78
2025-10-09T21:43:12.205035Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 79 partition: 0 MaxSeqNo: 25 sessionId: src_id|e02e8390-b8d51596-ed269713-23f7ffe0_0
2025-10-09T21:43:12.205475Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15364b7a-5ef9ce92-8ff892fd-1667ceb0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e02e8390-b8d51596-ed269713-23f7ffe0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:12.202739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:12.204058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:12.204065Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:12.204068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:12.204072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:12.204079Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 78 [2:7559340834976950336:2817]
2025-10-09T21:43:12.204094Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:12.204115Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:12.204117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.204120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.204123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.204139Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:12.204146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:12.204154Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:12.204217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.204219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.204221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.204223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:12.204302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:12.204305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.204319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.204331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.204349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:12.204372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340835338759033:2985], now have 1 active actors on pipe
2025-10-09T21:43:12.204375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340835338758899:2432], now have 1 active actors on pipe
2025-10-09T21:43:12.204383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340834976950331:2817]
2025-10-09T21:43:12.204386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340834976950331:2817]
2025-10-09T21:43:12.204388Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340834976950331:2817]
2025-10-09T21:43:12.204403Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:12.204418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:12.204419Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:12.204422Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:12.204424Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:12.204434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:12.204438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:12.204476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340834976950331:2817]
2025-10-09T21:43:12.204484Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:12.204507Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:12.204631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:12.204640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:12.204679Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e02e8390-b8d51596-ed269713-23f7ffe0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:12.204689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.204694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.204697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:12.204718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.204731Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:12.204748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:12.204924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:12.204929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:12.204946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:12.208705Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e02e8390-b8d51596-ed269713-23f7ffe0_0] Write session: try to update token
2025-10-09T21:43:12.208729Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e02e8390-b8d51596-ed269713-23f7ffe0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:12.220220Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:12.220239Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:12.220242Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:12.220244Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:12.220254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340834976950331:2817]
2025-10-09T21:43:12.220258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340834976950331:2817]
2025-10-09T21:43:12.220265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340834976950331:2817]
2025-10-09T21:43:12.220311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:12.220318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.220321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:12.220328Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:12.220335Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:12.220340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:12.220456Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:12.233986Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 79 sessionId: src_id|e02e8390-b8d51596-ed269713-23f7ffe0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:12.236409Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:12.274408Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340835338759040:2476]
2025-10-09T21:43:12.274710Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 78, pipe: [1:7559340835338759040:2476]
2025-10-09T21:43:12.396684Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:12.396988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:12.397005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:12.397030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340835338759040:2476], now have 1 active actors on pipe
2025-10-09T21:43:12.397254Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:12.398726Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340835338759040:2476]
2025-10-09T21:43:12.398758Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:12.398846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 79 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e02e8390-b8d51596-ed269713-23f7ffe0_0
2025-10-09T21:43:12.399065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 79 sessionId: src_id|e02e8390-b8d51596-ed269713-23f7ffe0_0 is DEAD
2025-10-09T21:43:12.399211Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:12.399311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e02e8390-b8d51596-ed269713-23f7ffe0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:12.399327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e02e8390-b8d51596-ed269713-23f7ffe0_0] Write session will restart in 0.005609s
2025-10-09T21:43:12.399358Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e02e8390-b8d51596-ed269713-23f7ffe0_0] Write session: Do CDS request
2025-10-09T21:43:12.399365Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e02e8390-b8d51596-ed269713-23f7ffe0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:12.406562Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e02e8390-b8d51596-ed269713-23f7ffe0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:12.410903Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:12.410922Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 80
2025-10-09T21:43:12.411085Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 80 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:12.411128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 80 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:12.411138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=80 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:12.411142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:12.411839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 80 sessionId:  describe result for acl check
2025-10-09T21:43:12.411881Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:12.411883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:12.411884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:12.411890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340835338759068:2996] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:12.412470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340835338759068:2996] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:12.427363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340835338759068:2996] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:12.439222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:12.439169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340835338759068:2996] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:12.439186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340835338759068:2996] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:12.439191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340835338759068:2996] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:12.439202Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 80 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:12.440537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:12.440771Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:12.440786Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:12.440801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:12.440863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:12.440906Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340834976950483:2819]
2025-10-09T21:43:12.441457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:12.442115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:12.442169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:12.442533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:12.442596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:12.442598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:12.442599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:12.442601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:12.442602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:12.442603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:12.442604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.442605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.442606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.442612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:12.442635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:12.442642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:12.442646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:12.442650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:12.442653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:12.442657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.442660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.442664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.442671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:12.442678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:12.442684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:12.442689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:12.442693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:12.442697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:12.442701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:12.442705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:12.442712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:12.442717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:12.442720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:12.442725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:12.442730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.442734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.442737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.442742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.442746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.442751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.442777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:12.458770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:12.459871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:12.459877Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:12.459880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:12.459885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:12.459893Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 79 [2:7559340834976950483:2819]
2025-10-09T21:43:12.459908Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:12.459927Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:12.459931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.459934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.459937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.459955Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:12.459961Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:12.459968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:12.460027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.460029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.460031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.460034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:12.460107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:12.460110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.460122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.460133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.460150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:12.460170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340834976950476:2819]
2025-10-09T21:43:12.460173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340834976950476:2819]
2025-10-09T21:43:12.460175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340834976950476:2819]
2025-10-09T21:43:12.460183Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:12.460195Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:12.460199Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:12.460199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:12.460204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:12.460214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:12.460217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:12.460228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340835338759060:2432], now have 1 active actors on pipe
2025-10-09T21:43:12.460232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340835338759186:2996], now have 1 active actors on pipe
2025-10-09T21:43:12.460267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340834976950476:2819]
2025-10-09T21:43:12.460277Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:12.460282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:12.460712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 79
2025-10-09T21:43:12.460743Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 79
2025-10-09T21:43:12.461326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 80 partition: 0 MaxSeqNo: 25 sessionId: src_id|929453de-c709d373-3bfcf7d8-3935a377_0
2025-10-09T21:43:12.461777Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e02e8390-b8d51596-ed269713-23f7ffe0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|929453de-c709d373-3bfcf7d8-3935a377_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:12.464801Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|929453de-c709d373-3bfcf7d8-3935a377_0] Write session: try to update token
2025-10-09T21:43:12.464824Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|929453de-c709d373-3bfcf7d8-3935a377_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:12.460820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:12.460833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:12.460873Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|929453de-c709d373-3bfcf7d8-3935a377_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:12.460883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.460887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.460890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:12.460901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.460928Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:12.460946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:12.461156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:12.461162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:12.461183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:12.488847Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 80 sessionId: src_id|929453de-c709d373-3bfcf7d8-3935a377_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:12.491724Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:12.491960Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:12.491979Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:12.491982Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:12.491984Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:12.491993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340834976950476:2819]
2025-10-09T21:43:12.491997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340834976950476:2819]
2025-10-09T21:43:12.492001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340834976950476:2819]
2025-10-09T21:43:12.492056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:12.492060Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:12.492065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:12.492075Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:12.492079Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:12.492088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:12.492099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.492106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:12.492261Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:12.498711Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340835338759207:2476]
2025-10-09T21:43:12.499228Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 79, pipe: [1:7559340835338759207:2476]
2025-10-09T21:43:12.657243Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:12.657574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340835338759207:2476], now have 1 active actors on pipe
2025-10-09T21:43:12.657890Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340835338759207:2476]
2025-10-09T21:43:12.657934Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:12.657976Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:12.657985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:12.658062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 80 reason: error at write: Disconnected sessionId: src_id|929453de-c709d373-3bfcf7d8-3935a377_0
2025-10-09T21:43:12.657907Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:12.658259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 80 sessionId: src_id|929453de-c709d373-3bfcf7d8-3935a377_0 is DEAD
2025-10-09T21:43:12.658625Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|929453de-c709d373-3bfcf7d8-3935a377_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:12.658643Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|929453de-c709d373-3bfcf7d8-3935a377_0] Write session will restart in 0.007179s
2025-10-09T21:43:12.658698Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|929453de-c709d373-3bfcf7d8-3935a377_0] Write session: Do CDS request
2025-10-09T21:43:12.658707Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|929453de-c709d373-3bfcf7d8-3935a377_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:12.667619Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|929453de-c709d373-3bfcf7d8-3935a377_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:12.670814Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:12.670896Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 81
2025-10-09T21:43:12.671138Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 81 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:12.671179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 81 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:12.671191Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=81 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:12.671195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:12.671965Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 81 sessionId:  describe result for acl check
2025-10-09T21:43:12.672005Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:12.672007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:12.672009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:12.672015Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340835338759243:3001] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:12.672608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340835338759243:3001] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:12.677757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340835338759243:3001] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:12.679707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:12.680430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:12.680635Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:12.680644Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:12.680656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:12.680704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:12.680743Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340834976950646:2834]
2025-10-09T21:43:12.680862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340835338759243:3001] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:12.680877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340835338759243:3001] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:12.680881Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340835338759243:3001] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:12.680891Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 81 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:12.681328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:12.681606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:12.681641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:12.681930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:12.681973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:12.681979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:12.681980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:12.681982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:12.681983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:12.681984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:12.681985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.681986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.681987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.681992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:12.682002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:12.682006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:12.682010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:12.682013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:12.682017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:12.682021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.682025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.682046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.682053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:12.682061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:12.682067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:12.682072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:12.682076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:12.682080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:12.682085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:12.682090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:12.682095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:12.682099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:12.682103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:12.682108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:12.682112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.682118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.682121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.682126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.682130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.682136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.682163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:12.690958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:12.692224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:12.692241Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:12.692243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:12.692248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:12.692257Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 80 [2:7559340834976950646:2834]
2025-10-09T21:43:12.692274Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:12.692295Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:12.692299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.692303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.692305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.692326Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:12.692335Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:12.692343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:12.692720Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 80
2025-10-09T21:43:12.692723Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 80
2025-10-09T21:43:12.692410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.692418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.692420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.692423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:12.692506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:12.692510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.692524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.692536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.692554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:12.692576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340834976950641:2834]
2025-10-09T21:43:12.692579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340834976950641:2834]
2025-10-09T21:43:12.692582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340834976950641:2834]
2025-10-09T21:43:12.692591Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:12.692606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:12.692607Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:12.692610Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:12.692613Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:12.692621Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:12.692625Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:12.692637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340835338759340:3001], now have 1 active actors on pipe
2025-10-09T21:43:12.692642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340835338759210:2432], now have 1 active actors on pipe
2025-10-09T21:43:12.692680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340834976950641:2834]
2025-10-09T21:43:12.692690Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:12.692704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:12.692999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:12.693016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:12.693051Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|99847c22-4f18aed9-888fcf66-3ea4907c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:12.693071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.693076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.693078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:12.693091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.693104Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:12.693123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:12.693399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:12.693411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:12.693431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:12.693627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 81 partition: 0 MaxSeqNo: 25 sessionId: src_id|99847c22-4f18aed9-888fcf66-3ea4907c_0
2025-10-09T21:43:12.694061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|929453de-c709d373-3bfcf7d8-3935a377_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|99847c22-4f18aed9-888fcf66-3ea4907c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:12.697166Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99847c22-4f18aed9-888fcf66-3ea4907c_0] Write session: try to update token
2025-10-09T21:43:12.697191Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99847c22-4f18aed9-888fcf66-3ea4907c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:12.708621Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:12.708642Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:12.708644Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:12.708646Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:12.708657Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340834976950641:2834]
2025-10-09T21:43:12.708661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340834976950641:2834]
2025-10-09T21:43:12.708666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340834976950641:2834]
2025-10-09T21:43:12.708711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:12.708724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.708727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:12.708727Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:12.708732Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:12.708736Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:12.708904Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:12.723439Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 81 sessionId: src_id|99847c22-4f18aed9-888fcf66-3ea4907c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:12.725939Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:12.758347Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340835338759362:2476]
2025-10-09T21:43:12.759600Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 80, pipe: [1:7559340835338759362:2476]
2025-10-09T21:43:12.879630Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:12.880015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:12.880030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:12.880053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340835338759362:2476], now have 1 active actors on pipe
2025-10-09T21:43:12.880339Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:12.880368Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340835338759362:2476]
2025-10-09T21:43:12.880411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:12.880414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:12.880482Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 81 reason: error at write: Disconnected sessionId: src_id|99847c22-4f18aed9-888fcf66-3ea4907c_0
2025-10-09T21:43:12.880686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 81 sessionId: src_id|99847c22-4f18aed9-888fcf66-3ea4907c_0 is DEAD
2025-10-09T21:43:12.880873Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:12.880912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99847c22-4f18aed9-888fcf66-3ea4907c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:12.880924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99847c22-4f18aed9-888fcf66-3ea4907c_0] Write session will restart in 0.005110s
2025-10-09T21:43:12.880952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99847c22-4f18aed9-888fcf66-3ea4907c_0] Write session: Do CDS request
2025-10-09T21:43:12.880960Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99847c22-4f18aed9-888fcf66-3ea4907c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:12.887699Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99847c22-4f18aed9-888fcf66-3ea4907c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:12.887864Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:12.887891Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 82
2025-10-09T21:43:12.888102Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 82 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:12.888136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 82 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:12.888147Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=82 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:12.888150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:12.888615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 82 sessionId:  describe result for acl check
2025-10-09T21:43:12.888650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:12.888653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:12.888655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:12.888661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340835338759381:3007] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:12.889336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340835338759381:3007] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:12.891268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340835338759381:3007] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:12.895239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340835338759381:3007] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:12.895252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340835338759381:3007] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:12.895256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340835338759381:3007] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:12.895265Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 82 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:12.896140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:12.896927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:12.897154Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:12.897164Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:12.897178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:12.897236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:12.897293Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340834976950785:2837]
2025-10-09T21:43:12.897893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:12.898318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:12.898359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:12.898705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:12.898755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:12.898761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:12.898763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:12.898764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:12.898766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:12.898767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:12.898769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.898770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.898771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.898776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:12.898789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:12.898794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:12.898799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:12.898804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:12.898809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:12.898814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.898818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.898823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.898830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:12.898838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:12.898844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:12.898851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:12.898856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:12.898863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:12.898872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:12.898878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:12.898886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:12.898893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:12.898903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:12.898909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:12.898914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.898925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.898930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.898936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.898941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.898947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.898980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:12.905667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:12.906972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:12.906989Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:12.906992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:12.906997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:12.907003Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 81 [2:7559340834976950785:2837]
2025-10-09T21:43:12.907017Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:12.907037Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:12.907040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.907043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.907045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.907060Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:12.907066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:12.907074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:12.907147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.907159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.907162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.907164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:12.907240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:12.907248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.907262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:12.907273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:12.907297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:12.907322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340835338759368:2432], now have 1 active actors on pipe
2025-10-09T21:43:12.907333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340835338759497:3007], now have 1 active actors on pipe
2025-10-09T21:43:12.907341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340834976950780:2837]
2025-10-09T21:43:12.907344Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340834976950780:2837]
2025-10-09T21:43:12.907346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340834976950780:2837]
2025-10-09T21:43:12.907361Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:12.907377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:12.907395Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 81
2025-10-09T21:43:12.907391Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:12.907414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 81
2025-10-09T21:43:12.907403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:12.907406Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:12.907407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:12.907409Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:12.907445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340834976950780:2837]
2025-10-09T21:43:12.907449Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:12.907452Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:12.907466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:12.907471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:12.907598Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|10078573-7e37cc44-d27cf690-af5d1c17_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:12.907611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.907615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.907618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:12.907629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.907639Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:12.907679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:12.907825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:12.907834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:12.907849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:12.907953Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 82 partition: 0 MaxSeqNo: 25 sessionId: src_id|10078573-7e37cc44-d27cf690-af5d1c17_0
2025-10-09T21:43:12.908362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99847c22-4f18aed9-888fcf66-3ea4907c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|10078573-7e37cc44-d27cf690-af5d1c17_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:12.911556Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10078573-7e37cc44-d27cf690-af5d1c17_0] Write session: try to update token
2025-10-09T21:43:12.911573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10078573-7e37cc44-d27cf690-af5d1c17_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:12.922142Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:12.922164Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:12.922166Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:12.922168Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:12.922177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340834976950780:2837]
2025-10-09T21:43:12.922181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340834976950780:2837]
2025-10-09T21:43:12.922185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340834976950780:2837]
2025-10-09T21:43:12.922229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:12.922235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:12.922238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:12.922246Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:12.922253Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:12.922257Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:12.922375Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:12.929374Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 82 sessionId: src_id|10078573-7e37cc44-d27cf690-af5d1c17_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:12.932439Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:12.980493Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340835338759511:2476]
2025-10-09T21:43:12.980823Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 81, pipe: [1:7559340835338759511:2476]
2025-10-09T21:43:13.078750Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:13.079114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:13.079129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:13.079153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340835338759511:2476], now have 1 active actors on pipe
2025-10-09T21:43:13.079373Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:13.079428Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:13.079438Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:13.079446Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:13.079497Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340835338759511:2476]
2025-10-09T21:43:13.079523Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 82 reason: error at write: Disconnected sessionId: src_id|10078573-7e37cc44-d27cf690-af5d1c17_0
2025-10-09T21:43:13.079707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 82 sessionId: src_id|10078573-7e37cc44-d27cf690-af5d1c17_0 is DEAD
2025-10-09T21:43:13.079867Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10078573-7e37cc44-d27cf690-af5d1c17_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:13.079876Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10078573-7e37cc44-d27cf690-af5d1c17_0] Write session will restart in 0.008943s
2025-10-09T21:43:13.079904Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10078573-7e37cc44-d27cf690-af5d1c17_0] Write session: Do CDS request
2025-10-09T21:43:13.079911Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10078573-7e37cc44-d27cf690-af5d1c17_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:13.089888Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10078573-7e37cc44-d27cf690-af5d1c17_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:13.090241Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:13.090257Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 83
2025-10-09T21:43:13.090483Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 83 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:13.090561Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 83 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:13.090572Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=83 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:13.090577Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:13.091198Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 83 sessionId:  describe result for acl check
2025-10-09T21:43:13.091242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:13.091249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:13.091252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:13.091258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340839633726874:3011] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:13.091916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340839633726874:3011] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:13.093847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:13.094860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:13.094852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340839633726874:3011] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:13.095228Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:13.095243Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:13.095261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:13.095333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:13.095373Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340839271918217:2839]
2025-10-09T21:43:13.096007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:13.096411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:13.096462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:13.096830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:13.096878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:13.096880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:13.096881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:13.096883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:13.096885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:13.096886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:13.096888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.096889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.096891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.096898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:13.096910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:13.096915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:13.096920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:13.096924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:13.096929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:13.096934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.096938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.096943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.096951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:13.096960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:13.096967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:13.096974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:13.096980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:13.096987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:13.096993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:13.096999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:13.097017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:13.097025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:13.097030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:13.097037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:13.097043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.097049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.097054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.097060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.097066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.097072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.097103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:13.098109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340839633726874:3011] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:13.098126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340839633726874:3011] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:13.098130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340839633726874:3011] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:13.098151Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 83 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:13.104135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:13.105375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:13.105394Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:13.105398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:13.105403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:13.105410Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 82 [2:7559340839271918217:2839]
2025-10-09T21:43:13.105428Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:13.105449Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:13.105460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.105463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.105466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.105480Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:13.105491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:13.105500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:13.105573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.105580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.105583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.105587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:13.105677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:13.105685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.105700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.105719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.105739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:13.105762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340839271918212:2839]
2025-10-09T21:43:13.105770Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340839271918212:2839]
2025-10-09T21:43:13.105773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340839271918212:2839]
2025-10-09T21:43:13.105782Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:13.105793Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:13.105798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:13.105803Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:13.105806Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:13.105814Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:13.105818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:13.105833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340839633726947:3011], now have 1 active actors on pipe
2025-10-09T21:43:13.105844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340839633726810:2432], now have 1 active actors on pipe
2025-10-09T21:43:13.105885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340839271918212:2839]
2025-10-09T21:43:13.105893Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:13.105891Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 82
2025-10-09T21:43:13.105892Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 82
2025-10-09T21:43:13.105942Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:13.106011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:13.106025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:13.106077Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8c3e21d0-79aacb6b-4dc8c9f7-b0d063d2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:13.106094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.106103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.106106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:13.106118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.106136Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:13.106160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:13.106458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:13.106473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:13.106507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:13.110345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 83 partition: 0 MaxSeqNo: 25 sessionId: src_id|8c3e21d0-79aacb6b-4dc8c9f7-b0d063d2_0
2025-10-09T21:43:13.110903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10078573-7e37cc44-d27cf690-af5d1c17_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8c3e21d0-79aacb6b-4dc8c9f7-b0d063d2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:13.114687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c3e21d0-79aacb6b-4dc8c9f7-b0d063d2_0] Write session: try to update token
2025-10-09T21:43:13.114712Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c3e21d0-79aacb6b-4dc8c9f7-b0d063d2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:13.128125Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:13.128147Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:13.128150Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:13.128153Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:13.128170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340839271918212:2839]
2025-10-09T21:43:13.128176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340839271918212:2839]
2025-10-09T21:43:13.128179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340839271918212:2839]
2025-10-09T21:43:13.128233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:13.128232Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:13.128240Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:13.128241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.128244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:13.128244Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:13.128437Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:13.137961Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 83 sessionId: src_id|8c3e21d0-79aacb6b-4dc8c9f7-b0d063d2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:13.141246Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:13.179641Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340839633726954:2476]
2025-10-09T21:43:13.179910Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 82, pipe: [1:7559340839633726954:2476]
2025-10-09T21:43:13.286265Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:13.286553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:13.286564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:13.286608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340839633726954:2476], now have 1 active actors on pipe
2025-10-09T21:43:13.286890Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340839633726954:2476]
2025-10-09T21:43:13.286924Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:13.286939Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:13.287000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 83 reason: error at write: Disconnected sessionId: src_id|8c3e21d0-79aacb6b-4dc8c9f7-b0d063d2_0
2025-10-09T21:43:13.287195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 83 sessionId: src_id|8c3e21d0-79aacb6b-4dc8c9f7-b0d063d2_0 is DEAD
2025-10-09T21:43:13.287208Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:13.287464Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c3e21d0-79aacb6b-4dc8c9f7-b0d063d2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:13.287476Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c3e21d0-79aacb6b-4dc8c9f7-b0d063d2_0] Write session will restart in 0.008089s
2025-10-09T21:43:13.287564Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c3e21d0-79aacb6b-4dc8c9f7-b0d063d2_0] Write session: Do CDS request
2025-10-09T21:43:13.287583Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c3e21d0-79aacb6b-4dc8c9f7-b0d063d2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:13.288635Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:13.296933Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c3e21d0-79aacb6b-4dc8c9f7-b0d063d2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:13.297031Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:13.297054Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 84
2025-10-09T21:43:13.297395Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 84 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:13.297436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 84 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:13.297442Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=84 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:13.297446Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:13.298046Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 84 sessionId:  describe result for acl check
2025-10-09T21:43:13.298084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:13.298089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:13.298091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:13.298097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340839633727004:3015] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:13.298694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340839633727004:3015] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:13.302365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340839633727004:3015] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:13.303306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:13.304032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:13.304249Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:13.304258Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:13.304272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:13.304332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:13.304383Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340839271918355:2841]
2025-10-09T21:43:13.304972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:13.305317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:13.305364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:13.305680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:13.305736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:13.305737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:13.305738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:13.305739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:13.305740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:13.305741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:13.305743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.305744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.305745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.305749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:13.305759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:13.305762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:13.305765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:13.305769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:13.305772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:13.305775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.305778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.305782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.305789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:13.305796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:13.305801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:13.305808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:13.305812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:13.305816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:13.305820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:13.305824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:13.305828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:13.305833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:13.305836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:13.305841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:13.305845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.305850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.305854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.305859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.305864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.305869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.305898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:13.306532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340839633727004:3015] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:13.306548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340839633727004:3015] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:13.306551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340839633727004:3015] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:13.306561Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 84 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:13.313677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:13.314960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:13.314978Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:13.314981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:13.314987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:13.314994Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 83 [2:7559340839271918355:2841]
2025-10-09T21:43:13.315008Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:13.315028Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:13.315033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.315036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.315039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.315058Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:13.315064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:13.315072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:13.315137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.315139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.315140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.315143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:13.315224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:13.315231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.315243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.315269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.315283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:13.315304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340839271918350:2841]
2025-10-09T21:43:13.315307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340839271918350:2841]
2025-10-09T21:43:13.315308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340839271918350:2841]
2025-10-09T21:43:13.315321Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:13.315330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:13.315333Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:13.315337Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:13.315338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340839633726957:2432], now have 1 active actors on pipe
2025-10-09T21:43:13.315342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340839633727094:3015], now have 1 active actors on pipe
2025-10-09T21:43:13.315351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:13.315368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:13.315377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:13.315418Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340839271918350:2841]
2025-10-09T21:43:13.315434Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:13.315521Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:13.315995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:13.316008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:13.316040Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8591fa88-a94c5272-a3007ab3-9681ff5f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:13.316049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.316054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.316056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:13.316068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.316086Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:13.316102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:13.316461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:13.316474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:13.316495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:13.315635Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 83
2025-10-09T21:43:13.315672Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 83
2025-10-09T21:43:13.316619Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 84 partition: 0 MaxSeqNo: 25 sessionId: src_id|8591fa88-a94c5272-a3007ab3-9681ff5f_0
2025-10-09T21:43:13.319183Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c3e21d0-79aacb6b-4dc8c9f7-b0d063d2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8591fa88-a94c5272-a3007ab3-9681ff5f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:13.322714Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8591fa88-a94c5272-a3007ab3-9681ff5f_0] Write session: try to update token
2025-10-09T21:43:13.322735Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8591fa88-a94c5272-a3007ab3-9681ff5f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:13.334583Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:13.334604Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:13.334605Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:13.334608Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:13.334618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340839271918350:2841]
2025-10-09T21:43:13.334622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340839271918350:2841]
2025-10-09T21:43:13.334626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340839271918350:2841]
2025-10-09T21:43:13.334670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:13.334677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.334679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:13.334876Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:13.335265Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:13.335272Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:13.335276Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:13.357080Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 84 sessionId: src_id|8591fa88-a94c5272-a3007ab3-9681ff5f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:13.359569Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:13.390451Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340839633727105:2476]
2025-10-09T21:43:13.391663Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 83, pipe: [1:7559340839633727105:2476]
2025-10-09T21:43:13.553741Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:13.558390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:13.558410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:13.558445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340839633727105:2476], now have 1 active actors on pipe
2025-10-09T21:43:13.558614Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:13.559238Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:13.559282Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340839633727105:2476]
2025-10-09T21:43:13.559317Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:13.559383Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 84 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8591fa88-a94c5272-a3007ab3-9681ff5f_0
2025-10-09T21:43:13.559540Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 84 sessionId: src_id|8591fa88-a94c5272-a3007ab3-9681ff5f_0 is DEAD
2025-10-09T21:43:13.560323Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8591fa88-a94c5272-a3007ab3-9681ff5f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:13.560333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8591fa88-a94c5272-a3007ab3-9681ff5f_0] Write session will restart in 0.007330s
2025-10-09T21:43:13.560360Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8591fa88-a94c5272-a3007ab3-9681ff5f_0] Write session: Do CDS request
2025-10-09T21:43:13.560366Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8591fa88-a94c5272-a3007ab3-9681ff5f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:13.568565Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8591fa88-a94c5272-a3007ab3-9681ff5f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:13.569508Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:13.569531Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 85
2025-10-09T21:43:13.570090Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 85 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:13.570131Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 85 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:13.570149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=85 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:13.570154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:13.570641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 85 sessionId:  describe result for acl check
2025-10-09T21:43:13.570679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:13.570680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:13.570682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:13.570719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340839633727146:3027] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:13.571271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340839633727146:3027] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:13.574583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340839633727146:3027] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:13.582577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340839633727146:3027] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:13.582599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340839633727146:3027] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:13.582603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340839633727146:3027] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:13.582617Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 85 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:13.584950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:13.586080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:13.586285Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:13.586298Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:13.586310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:13.586367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:13.586405Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340839271918533:2855]
2025-10-09T21:43:13.586983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:13.587349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:13.587387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:13.587697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:13.587746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:13.587747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:13.587748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:13.587749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:13.587750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:13.587752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:13.587753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.587754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.587755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.587760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:13.587772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:13.587776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:13.587780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:13.587784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:13.587788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:13.587791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.587794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.587797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.587805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:13.587813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:13.587819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:13.587824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:13.587828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:13.587833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:13.587837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:13.587841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:13.587846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:13.587851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:13.587854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:13.587860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:13.587864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.587869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.587872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.587877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.587882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.587888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.587914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:13.597647Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 84
2025-10-09T21:43:13.597685Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 84
2025-10-09T21:43:13.595775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:13.597009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:13.597015Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:13.597018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:13.597021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:13.597029Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 84 [2:7559340839271918533:2855]
2025-10-09T21:43:13.597045Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:13.597063Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:13.597066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.597069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.597071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.597090Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:13.597096Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:13.597103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:13.597159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.597161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.597163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.597165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:13.597236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:13.597238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.597251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.597263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.597279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:13.597299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340839271918528:2855]
2025-10-09T21:43:13.597302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340839271918528:2855]
2025-10-09T21:43:13.597304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340839271918528:2855]
2025-10-09T21:43:13.597313Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:13.597323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:13.597326Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:13.597328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:13.597329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:13.597343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:13.597348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:13.597359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340839633727259:3027], now have 1 active actors on pipe
2025-10-09T21:43:13.597363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340839633727133:2432], now have 1 active actors on pipe
2025-10-09T21:43:13.597397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340839271918528:2855]
2025-10-09T21:43:13.597406Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:13.597696Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:13.598023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:13.598045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:13.598077Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4702e625-5bb31e6e-b2325ae6-38a441d3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:13.598086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.598090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.598093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:13.598105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.598117Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:13.598133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:13.606178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 85 partition: 0 MaxSeqNo: 25 sessionId: src_id|4702e625-5bb31e6e-b2325ae6-38a441d3_0
2025-10-09T21:43:13.606594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8591fa88-a94c5272-a3007ab3-9681ff5f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4702e625-5bb31e6e-b2325ae6-38a441d3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:13.602246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:13.602261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:13.602293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:13.609667Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4702e625-5bb31e6e-b2325ae6-38a441d3_0] Write session: try to update token
2025-10-09T21:43:13.609694Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4702e625-5bb31e6e-b2325ae6-38a441d3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:13.627915Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:13.627931Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:13.627934Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:13.627936Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:13.627947Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340839271918528:2855]
2025-10-09T21:43:13.627951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340839271918528:2855]
2025-10-09T21:43:13.627955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340839271918528:2855]
2025-10-09T21:43:13.627998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:13.628005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.628008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:13.628146Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:13.630225Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:13.630239Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:13.630243Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:13.632671Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 85 sessionId: src_id|4702e625-5bb31e6e-b2325ae6-38a441d3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:13.636249Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:13.661351Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340839633727276:2476]
2025-10-09T21:43:13.661961Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 84, pipe: [1:7559340839633727276:2476]
2025-10-09T21:43:13.838825Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:13.839416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:13.839427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:13.839476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340839633727276:2476], now have 1 active actors on pipe
2025-10-09T21:43:13.840630Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4702e625-5bb31e6e-b2325ae6-38a441d3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:13.840642Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4702e625-5bb31e6e-b2325ae6-38a441d3_0] Write session will restart in 0.007558s
2025-10-09T21:43:13.840675Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4702e625-5bb31e6e-b2325ae6-38a441d3_0] Write session: Do CDS request
2025-10-09T21:43:13.840682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4702e625-5bb31e6e-b2325ae6-38a441d3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:13.840162Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340839633727276:2476]
2025-10-09T21:43:13.840201Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:13.840203Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:13.840270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 85 reason: error at write: Disconnected sessionId: src_id|4702e625-5bb31e6e-b2325ae6-38a441d3_0
2025-10-09T21:43:13.840460Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 85 sessionId: src_id|4702e625-5bb31e6e-b2325ae6-38a441d3_0 is DEAD
2025-10-09T21:43:13.840611Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:13.841162Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:13.849746Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4702e625-5bb31e6e-b2325ae6-38a441d3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:13.850015Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:13.850050Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 86
2025-10-09T21:43:13.850206Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 86 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:13.850239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 86 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:13.850244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=86 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:13.850248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:13.850826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 86 sessionId:  describe result for acl check
2025-10-09T21:43:13.850865Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:13.850871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:13.850873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:13.850880Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340839633727304:3033] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:13.851445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340839633727304:3033] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:13.854168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340839633727304:3033] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:13.858689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340839633727304:3033] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:13.858706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340839633727304:3033] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:13.858711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340839633727304:3033] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:13.858720Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 86 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:13.863001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:13.864080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:13.864396Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:13.864408Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:13.864429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:13.864491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:13.864542Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340839271918675:2859]
2025-10-09T21:43:13.865144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:13.865434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:13.865480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:13.865783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:13.865839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:13.865846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:13.865847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:13.865848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:13.865849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:13.865850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:13.865851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.865852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.865853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.865858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:13.865869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:13.865872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:13.865876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:13.865879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:13.865883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:13.865887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.865891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.865895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.865902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:13.865911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:13.865917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:13.865922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:13.865926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:13.865931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:13.865935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:13.865940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:13.865944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:13.865949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:13.865954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:13.865959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:13.865964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.865968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.865973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.865978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.865982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.865988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.866017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:13.874890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:13.879595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:13.879615Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:13.879618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:13.879623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:13.879631Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 85 [2:7559340839271918675:2859]
2025-10-09T21:43:13.879649Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:13.879670Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:13.879673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.879677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.879679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.879696Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:13.879703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:13.879711Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:13.879782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.879792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.879794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.879797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:13.879926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:13.879935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.879948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:13.879958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:13.879973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:13.880000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340839633727367:3033], now have 1 active actors on pipe
2025-10-09T21:43:13.880007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340839633727291:2432], now have 1 active actors on pipe
2025-10-09T21:43:13.880015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340839271918669:2859]
2025-10-09T21:43:13.880020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340839271918669:2859]
2025-10-09T21:43:13.880022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340839271918669:2859]
2025-10-09T21:43:13.880034Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:13.880057Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:13.880073Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:13.880080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:13.880123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340839271918669:2859]
2025-10-09T21:43:13.880130Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:13.880105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 85
2025-10-09T21:43:13.880148Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 85
2025-10-09T21:43:13.880177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:13.880196Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:13.880198Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:13.880205Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:13.880307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:13.880317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:13.880352Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|480fb93a-aaf28d1b-6cd313bb-a9dbbfd1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:13.880364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.880374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.880376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:13.880388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.880403Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:13.880425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:13.880789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:13.880800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:13.880818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:13.881100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 86 partition: 0 MaxSeqNo: 25 sessionId: src_id|480fb93a-aaf28d1b-6cd313bb-a9dbbfd1_0
2025-10-09T21:43:13.881554Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4702e625-5bb31e6e-b2325ae6-38a441d3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|480fb93a-aaf28d1b-6cd313bb-a9dbbfd1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:13.898176Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:13.898193Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:13.898198Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:13.898199Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:13.898211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340839271918669:2859]
2025-10-09T21:43:13.898219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340839271918669:2859]
2025-10-09T21:43:13.898222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340839271918669:2859]
2025-10-09T21:43:13.898268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:13.898280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:13.898283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:13.898288Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:13.898296Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:13.898300Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:13.899972Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:13.908554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|480fb93a-aaf28d1b-6cd313bb-a9dbbfd1_0] Write session: try to update token
2025-10-09T21:43:13.908583Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|480fb93a-aaf28d1b-6cd313bb-a9dbbfd1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:13.942026Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340839633727439:2476]
2025-10-09T21:43:13.942657Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 85, pipe: [1:7559340839633727439:2476]
2025-10-09T21:43:13.951238Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 86 sessionId: src_id|480fb93a-aaf28d1b-6cd313bb-a9dbbfd1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:13.953916Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:14.068365Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:14.068757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340839633727439:2476], now have 1 active actors on pipe
2025-10-09T21:43:14.068790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:14.068798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:14.069168Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:14.069733Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340839633727439:2476]
2025-10-09T21:43:14.069749Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:14.069763Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:14.069840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 86 reason: status is not ok: tablet will be restarted right now sessionId: src_id|480fb93a-aaf28d1b-6cd313bb-a9dbbfd1_0
2025-10-09T21:43:14.070210Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|480fb93a-aaf28d1b-6cd313bb-a9dbbfd1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:14.070222Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|480fb93a-aaf28d1b-6cd313bb-a9dbbfd1_0] Write session will restart in 0.009683s
2025-10-09T21:43:14.070246Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|480fb93a-aaf28d1b-6cd313bb-a9dbbfd1_0] Write session: Do CDS request
2025-10-09T21:43:14.070253Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|480fb93a-aaf28d1b-6cd313bb-a9dbbfd1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:14.070068Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 86 sessionId: src_id|480fb93a-aaf28d1b-6cd313bb-a9dbbfd1_0 is DEAD
2025-10-09T21:43:14.081613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|480fb93a-aaf28d1b-6cd313bb-a9dbbfd1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:14.081971Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:14.081987Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 87
2025-10-09T21:43:14.082287Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 87 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:14.082320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 87 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:14.082326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=87 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:14.082337Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:14.082750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 87 sessionId:  describe result for acl check
2025-10-09T21:43:14.082777Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:14.082780Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:14.082782Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:14.082787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340843928694763:3037] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:14.083335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340843928694763:3037] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:14.087947Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340843928694763:3037] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:14.091437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340843928694763:3037] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:14.091450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340843928694763:3037] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:14.091454Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340843928694763:3037] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:14.091463Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 87 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:14.092514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:14.093326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:14.093543Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:14.093557Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:14.093568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:14.093626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:14.093667Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340843566886110:2861]
2025-10-09T21:43:14.094208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:14.094739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:14.094779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:14.095073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:14.095117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:14.095122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:14.095123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:14.095124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:14.095125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:14.095126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:14.095127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.095128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.095129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.095134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:14.095145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:14.095148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:14.095152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:14.095155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:14.095158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:14.095162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.095165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.095168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.095175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:14.095182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:14.095188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:14.095192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:14.095196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:14.095200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:14.095214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:14.095219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:14.095223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:14.095229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:14.095232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:14.095237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:14.095241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.095245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.095249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.095252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.095256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.095260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.095285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:14.102319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:14.103729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:14.103743Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:14.103745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:14.103749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:14.103756Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 86 [2:7559340843566886110:2861]
2025-10-09T21:43:14.103770Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:14.103789Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:14.103798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.103800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.103803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.103821Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:14.103826Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:14.103833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:14.103895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.103902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.103903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.103906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:14.103980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:14.103988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.104001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.104011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.104025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:14.104050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340843928694738:2432], now have 1 active actors on pipe
2025-10-09T21:43:14.104063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340843566886104:2861]
2025-10-09T21:43:14.104069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340843566886104:2861]
2025-10-09T21:43:14.104071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340843566886104:2861]
2025-10-09T21:43:14.104086Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:14.104100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340843928694872:3037], now have 1 active actors on pipe
2025-10-09T21:43:14.104102Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:14.104106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:14.104108Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:14.104113Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:14.104127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:14.104134Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:14.104197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 86
2025-10-09T21:43:14.104243Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 86
2025-10-09T21:43:14.104172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340843566886104:2861]
2025-10-09T21:43:14.104187Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:14.104192Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:14.104232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:14.104237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:14.104266Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a5c1576c-2a6125d-31c0341-16241dc9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:14.104273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.104276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.104278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:14.104288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.104297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:14.104310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:14.104538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:14.104552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:14.104587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:14.104752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 87 partition: 0 MaxSeqNo: 25 sessionId: src_id|a5c1576c-2a6125d-31c0341-16241dc9_0
2025-10-09T21:43:14.105203Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|480fb93a-aaf28d1b-6cd313bb-a9dbbfd1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a5c1576c-2a6125d-31c0341-16241dc9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:14.109122Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a5c1576c-2a6125d-31c0341-16241dc9_0] Write session: try to update token
2025-10-09T21:43:14.109145Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a5c1576c-2a6125d-31c0341-16241dc9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:14.125090Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:14.125109Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:14.125119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340843566886104:2861]
2025-10-09T21:43:14.125172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:14.125174Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:14.125427Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:14.125574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:14.125746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:14.125911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:14.126290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:14.126329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:14.134384Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:14.134401Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:14.134404Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:14.134409Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:14.134432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340843566886104:2861]
2025-10-09T21:43:14.134439Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340843566886104:2861]
2025-10-09T21:43:14.134443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340843566886104:2861]
2025-10-09T21:43:14.134493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:14.134499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.134502Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:14.134509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:14.134510Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:14.134514Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:14.134672Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:14.149765Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 87 sessionId: src_id|a5c1576c-2a6125d-31c0341-16241dc9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:14.152525Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:14.169841Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340843928694886:2476]
2025-10-09T21:43:14.170294Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 86, pipe: [1:7559340843928694886:2476]
2025-10-09T21:43:14.444903Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:14.445234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:14.445249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:14.445272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340843928694886:2476], now have 1 active actors on pipe
2025-10-09T21:43:14.445644Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:14.445699Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340843928694886:2476]
2025-10-09T21:43:14.445722Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:14.445729Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:14.445798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 87 reason: error at write: Disconnected sessionId: src_id|a5c1576c-2a6125d-31c0341-16241dc9_0
2025-10-09T21:43:14.445982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 87 sessionId: src_id|a5c1576c-2a6125d-31c0341-16241dc9_0 is DEAD
2025-10-09T21:43:14.446147Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a5c1576c-2a6125d-31c0341-16241dc9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:14.446161Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a5c1576c-2a6125d-31c0341-16241dc9_0] Write session will restart in 0.005864s
2025-10-09T21:43:14.446186Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a5c1576c-2a6125d-31c0341-16241dc9_0] Write session: Do CDS request
2025-10-09T21:43:14.446196Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a5c1576c-2a6125d-31c0341-16241dc9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:14.447772Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:14.453540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a5c1576c-2a6125d-31c0341-16241dc9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:14.453678Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:14.453694Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 88
2025-10-09T21:43:14.453887Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 88 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:14.453930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 88 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:14.453935Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=88 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:14.453939Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:14.454433Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 88 sessionId:  describe result for acl check
2025-10-09T21:43:14.454472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:14.454474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:14.454476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:14.454483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340843928694907:3042] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:14.455092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340843928694907:3042] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:14.462272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340843928694907:3042] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:14.466147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340843928694907:3042] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:14.466163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340843928694907:3042] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:14.466167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340843928694907:3042] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:14.466177Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 88 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:14.466592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:14.467759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:14.467979Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:14.467983Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:14.467995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:14.468046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:14.468109Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340843566886250:2863]
2025-10-09T21:43:14.468739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:14.469094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:14.469195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:14.469560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:14.469606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:14.469607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:14.469609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:14.469610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:14.469611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:14.469613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:14.469614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.469616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.469617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.469623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:14.469635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:14.469640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:14.469645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:14.469649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:14.469653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:14.469658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.469662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.469667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.469674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:14.469682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:14.469688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:14.469693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:14.469697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:14.469703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:14.469708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:14.469714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:14.469719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:14.469724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:14.469730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:14.469736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:14.469741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.469747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.469752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.469758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.469763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.469768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.469802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:14.477343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:14.478865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:14.478871Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:14.478874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:14.478879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:14.478887Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 87 [2:7559340843566886250:2863]
2025-10-09T21:43:14.478902Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:14.478922Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:14.478926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.478930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.478932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.478951Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:14.478958Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:14.478966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:14.479027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.479030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.479032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.479035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:14.479109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:14.479112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.479126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.479138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.479158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:14.479179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340843566886243:2863]
2025-10-09T21:43:14.479182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340843566886243:2863]
2025-10-09T21:43:14.479184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340843566886243:2863]
2025-10-09T21:43:14.479198Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:14.479212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:14.479228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:14.479231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:14.479268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340843566886243:2863]
2025-10-09T21:43:14.479270Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:14.479734Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:14.479741Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:14.479743Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:14.479750Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:14.480179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340843928694893:2432], now have 1 active actors on pipe
2025-10-09T21:43:14.480188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340843928695029:3042], now have 1 active actors on pipe
2025-10-09T21:43:14.480492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:14.480501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:14.480531Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5d0f0e3a-d2225fa5-dd58a69a-dbe5f108_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:14.480541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.480545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.480562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:14.480574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.480587Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:14.480603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:14.481172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:14.481180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:14.481198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:14.481788Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a5c1576c-2a6125d-31c0341-16241dc9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5d0f0e3a-d2225fa5-dd58a69a-dbe5f108_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:14.480175Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 87
2025-10-09T21:43:14.480221Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 87
2025-10-09T21:43:14.481353Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 88 partition: 0 MaxSeqNo: 25 sessionId: src_id|5d0f0e3a-d2225fa5-dd58a69a-dbe5f108_0
2025-10-09T21:43:14.485197Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5d0f0e3a-d2225fa5-dd58a69a-dbe5f108_0] Write session: try to update token
2025-10-09T21:43:14.485213Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5d0f0e3a-d2225fa5-dd58a69a-dbe5f108_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:14.497682Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:14.497697Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:14.497700Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:14.497701Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:14.497711Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340843566886243:2863]
2025-10-09T21:43:14.497715Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340843566886243:2863]
2025-10-09T21:43:14.497719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340843566886243:2863]
2025-10-09T21:43:14.497760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:14.497765Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:14.497767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.497770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:14.497771Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:14.497776Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:14.497922Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:14.505233Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 88 sessionId: src_id|5d0f0e3a-d2225fa5-dd58a69a-dbe5f108_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:14.508027Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:14.545706Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340843928695043:2476]
2025-10-09T21:43:14.546254Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 87, pipe: [1:7559340843928695043:2476]
2025-10-09T21:43:14.700364Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:14.700823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:14.700843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:14.700865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340843928695043:2476], now have 1 active actors on pipe
2025-10-09T21:43:14.701094Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:14.701994Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340843928695043:2476]
2025-10-09T21:43:14.702020Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:14.702058Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:14.702135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 88 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5d0f0e3a-d2225fa5-dd58a69a-dbe5f108_0
2025-10-09T21:43:14.702377Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 88 sessionId: src_id|5d0f0e3a-d2225fa5-dd58a69a-dbe5f108_0 is DEAD
2025-10-09T21:43:14.702584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5d0f0e3a-d2225fa5-dd58a69a-dbe5f108_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:14.702595Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5d0f0e3a-d2225fa5-dd58a69a-dbe5f108_0] Write session will restart in 0.006902s
2025-10-09T21:43:14.702628Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5d0f0e3a-d2225fa5-dd58a69a-dbe5f108_0] Write session: Do CDS request
2025-10-09T21:43:14.702634Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5d0f0e3a-d2225fa5-dd58a69a-dbe5f108_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:14.711812Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5d0f0e3a-d2225fa5-dd58a69a-dbe5f108_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:14.711965Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:14.711983Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 89
2025-10-09T21:43:14.712587Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 89 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:14.712630Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 89 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:14.712641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=89 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:14.712645Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:14.713363Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 89 sessionId:  describe result for acl check
2025-10-09T21:43:14.713402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:14.713403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:14.713405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:14.713412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340843928695112:3053] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:14.713971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340843928695112:3053] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:14.716931Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340843928695112:3053] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:14.722823Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340843928695112:3053] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:14.722839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340843928695112:3053] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:14.722843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340843928695112:3053] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:14.722852Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 89 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:14.723029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:14.725679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:14.725971Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:14.725982Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:14.725996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:14.726077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:14.726126Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340843566886434:2878]
2025-10-09T21:43:14.726672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:14.726970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:14.727022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:14.727353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:14.727409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:14.727414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:14.727415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:14.727416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:14.727417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:14.727419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:14.727420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.727421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.727423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.727427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:14.727438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:14.727442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:14.727446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:14.727449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:14.727453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:14.727457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.727461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.727465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.727471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:14.727479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:14.727484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:14.727489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:14.727494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:14.727499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:14.727503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:14.727507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:14.727516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:14.727521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:14.727529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:14.727533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:14.727537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.727547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.727551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.727561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.727565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.727574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.727602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:14.735116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:14.737418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:14.737439Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:14.737442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:14.737446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:14.737453Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 88 [2:7559340843566886434:2878]
2025-10-09T21:43:14.737472Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:14.737490Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:14.737498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.737501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.737503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.737520Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:14.737530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:14.737538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:14.737598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.737608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.737610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.737613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:14.737754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:14.737762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.737774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.737784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.737806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:14.737825Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 88
2025-10-09T21:43:14.737833Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 88
2025-10-09T21:43:14.737833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340843928695206:3053], now have 1 active actors on pipe
2025-10-09T21:43:14.737844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340843928695071:2432], now have 1 active actors on pipe
2025-10-09T21:43:14.737852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340843566886428:2878]
2025-10-09T21:43:14.737854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340843566886428:2878]
2025-10-09T21:43:14.737857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340843566886428:2878]
2025-10-09T21:43:14.737872Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:14.737888Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:14.737903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:14.737906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:14.737942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340843566886428:2878]
2025-10-09T21:43:14.737945Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:14.737950Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:14.737953Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:14.737955Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:14.737962Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:14.738050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:14.738058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:14.738091Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7ebc04fb-d49be81-154d40a6-c5884e1d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:14.738109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.738114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.738116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:14.738129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.738142Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:14.738158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:14.738579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:14.738600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:14.738622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:14.738952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 89 partition: 0 MaxSeqNo: 25 sessionId: src_id|7ebc04fb-d49be81-154d40a6-c5884e1d_0
2025-10-09T21:43:14.739399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5d0f0e3a-d2225fa5-dd58a69a-dbe5f108_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7ebc04fb-d49be81-154d40a6-c5884e1d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:14.744333Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ebc04fb-d49be81-154d40a6-c5884e1d_0] Write session: try to update token
2025-10-09T21:43:14.744354Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ebc04fb-d49be81-154d40a6-c5884e1d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:14.756908Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:14.756924Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:14.756926Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:14.756928Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:14.756937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340843566886428:2878]
2025-10-09T21:43:14.756940Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340843566886428:2878]
2025-10-09T21:43:14.756944Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340843566886428:2878]
2025-10-09T21:43:14.756987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:14.756994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.756993Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:14.756997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:14.757000Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:14.757004Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:14.757138Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:14.777041Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 89 sessionId: src_id|7ebc04fb-d49be81-154d40a6-c5884e1d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:14.779774Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:14.802119Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340843928695229:2476]
2025-10-09T21:43:14.802387Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 88, pipe: [1:7559340843928695229:2476]
2025-10-09T21:43:14.942320Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:14.942738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:14.942768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:14.942795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340843928695229:2476], now have 1 active actors on pipe
2025-10-09T21:43:14.942991Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:14.943030Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340843928695229:2476]
2025-10-09T21:43:14.943035Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:14.943039Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:14.943071Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:14.943096Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 89 reason: error at write: Disconnected sessionId: src_id|7ebc04fb-d49be81-154d40a6-c5884e1d_0
2025-10-09T21:43:14.943278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 89 sessionId: src_id|7ebc04fb-d49be81-154d40a6-c5884e1d_0 is DEAD
2025-10-09T21:43:14.943399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ebc04fb-d49be81-154d40a6-c5884e1d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:14.943408Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ebc04fb-d49be81-154d40a6-c5884e1d_0] Write session will restart in 0.008924s
2025-10-09T21:43:14.943434Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ebc04fb-d49be81-154d40a6-c5884e1d_0] Write session: Do CDS request
2025-10-09T21:43:14.943440Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ebc04fb-d49be81-154d40a6-c5884e1d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:14.953624Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ebc04fb-d49be81-154d40a6-c5884e1d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:14.954557Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:14.954582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 90
2025-10-09T21:43:14.954756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 90 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:14.954798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 90 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:14.954805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=90 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:14.954808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:14.955284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 90 sessionId:  describe result for acl check
2025-10-09T21:43:14.955329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:14.955348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:14.955351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:14.955357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340843928695274:3059] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:14.956153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340843928695274:3059] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:14.959886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340843928695274:3059] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:14.960028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:14.960985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:14.961203Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:14.961211Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:14.961222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:14.961283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:14.961344Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340843566886581:2881]
2025-10-09T21:43:14.961920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:14.962160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340843928695274:3059] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:14.962173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340843928695274:3059] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:14.962177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340843928695274:3059] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:14.962183Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 90 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:14.962263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:14.962308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:14.962657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:14.962705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:14.962711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:14.962712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:14.962713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:14.962714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:14.962715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:14.962716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.962717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.962718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.962723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:14.962731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:14.962735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:14.962738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:14.962741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:14.962745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:14.962748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.962752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.962755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.962762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:14.962779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:14.962785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:14.962791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:14.962795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:14.962804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:14.962807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:14.962812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:14.962817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:14.962822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:14.962826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:14.962831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:14.962840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.962845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.962848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.962856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.962860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.962865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.962890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:14.972181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:14.973557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:14.973569Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:14.973572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:14.973576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:14.973583Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 89 [2:7559340843566886581:2881]
2025-10-09T21:43:14.973596Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:14.973619Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:14.973622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.973626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.973629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.973658Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:14.973664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:14.973678Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:14.973746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.973766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.973769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.973773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:14.973842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:14.973850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.973863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:14.973877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:14.973898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:14.973922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340843566886576:2881]
2025-10-09T21:43:14.973927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340843566886576:2881]
2025-10-09T21:43:14.973929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340843566886576:2881]
2025-10-09T21:43:14.973945Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:14.973963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:14.973966Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:14.973970Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:14.973972Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:14.973977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:14.973980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:14.974012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340843566886576:2881]
2025-10-09T21:43:14.974021Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:14.974106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:14.974206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340843928695237:2432], now have 1 active actors on pipe
2025-10-09T21:43:14.974217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340843928695380:3059], now have 1 active actors on pipe
2025-10-09T21:43:14.974390Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 89
2025-10-09T21:43:14.974446Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 89
2025-10-09T21:43:14.974729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:14.974741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:14.974774Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|74941ebd-b3b083f4-1149b9e0-cad6f902_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:14.974790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.974794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.974797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:14.974809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.974826Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:14.974846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:14.975100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:14.975107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:14.975122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:14.975242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 90 partition: 0 MaxSeqNo: 25 sessionId: src_id|74941ebd-b3b083f4-1149b9e0-cad6f902_0
2025-10-09T21:43:14.975736Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ebc04fb-d49be81-154d40a6-c5884e1d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|74941ebd-b3b083f4-1149b9e0-cad6f902_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:14.979436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74941ebd-b3b083f4-1149b9e0-cad6f902_0] Write session: try to update token
2025-10-09T21:43:14.979458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74941ebd-b3b083f4-1149b9e0-cad6f902_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:14.994677Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:14.994695Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:14.994698Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:14.994699Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:14.994709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340843566886576:2881]
2025-10-09T21:43:14.994713Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340843566886576:2881]
2025-10-09T21:43:14.994721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340843566886576:2881]
2025-10-09T21:43:14.994767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:14.994775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:14.994785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:14.995326Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:14.997625Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:14.997638Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:14.997643Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:15.043588Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340848223662686:2476]
2025-10-09T21:43:15.043894Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 89, pipe: [1:7559340848223662686:2476]
2025-10-09T21:43:15.045590Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 90 sessionId: src_id|74941ebd-b3b083f4-1149b9e0-cad6f902_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:15.048471Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:15.166413Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:15.166757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340848223662686:2476], now have 1 active actors on pipe
2025-10-09T21:43:15.166787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:15.166795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:15.167136Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340848223662686:2476]
2025-10-09T21:43:15.167177Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:15.167199Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:15.167204Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:15.167275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 90 reason: error at write: Disconnected sessionId: src_id|74941ebd-b3b083f4-1149b9e0-cad6f902_0
2025-10-09T21:43:15.167092Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:15.167451Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 90 sessionId: src_id|74941ebd-b3b083f4-1149b9e0-cad6f902_0 is DEAD
2025-10-09T21:43:15.167727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74941ebd-b3b083f4-1149b9e0-cad6f902_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:15.167738Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74941ebd-b3b083f4-1149b9e0-cad6f902_0] Write session will restart in 0.009877s
2025-10-09T21:43:15.167765Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74941ebd-b3b083f4-1149b9e0-cad6f902_0] Write session: Do CDS request
2025-10-09T21:43:15.167771Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74941ebd-b3b083f4-1149b9e0-cad6f902_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:15.178718Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74941ebd-b3b083f4-1149b9e0-cad6f902_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:15.179393Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:15.179409Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 91
2025-10-09T21:43:15.179768Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 91 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:15.179809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 91 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:15.179816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=91 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:15.179820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:15.180405Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 91 sessionId:  describe result for acl check
2025-10-09T21:43:15.180447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:15.180449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:15.180451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:15.180458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340848223662703:3063] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:15.181039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340848223662703:3063] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:15.185659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340848223662703:3063] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:15.195260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340848223662703:3063] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:15.195282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340848223662703:3063] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:15.195286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340848223662703:3063] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:15.195297Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 91 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:15.195958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:15.199015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:15.199278Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:15.199294Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:15.199309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:15.199379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:15.199426Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340847861854023:2883]
2025-10-09T21:43:15.200045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:15.202476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:15.202551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:15.202900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:15.202947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:15.202952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:15.202954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:15.202955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:15.202956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:15.202957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:15.202959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.202960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.202961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.202967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:15.202977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:15.202981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:15.202984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:15.202988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:15.202991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:15.202995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.202998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.203001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.203008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:15.203018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:15.203024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:15.203030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:15.203034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:15.203039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:15.203043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:15.203048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:15.203052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:15.203056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:15.203060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:15.203066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:15.203070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.203074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.203078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.203082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.203087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.203092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.203130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:15.214725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:15.215999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:15.216015Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:15.216018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:15.216022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:15.216030Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 90 [2:7559340847861854023:2883]
2025-10-09T21:43:15.216045Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:15.216064Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:15.216075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.216078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.216080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.216099Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:15.216105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:15.216112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:15.216174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.216181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.216182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.216185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:15.216257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:15.216265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.216276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.216285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.216300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:15.216321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340847861854016:2883]
2025-10-09T21:43:15.216328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340847861854016:2883]
2025-10-09T21:43:15.216330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340847861854016:2883]
2025-10-09T21:43:15.216339Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:15.216349Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:15.216355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:15.216358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:15.216361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:15.216368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:15.216371Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:15.216383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340848223662827:3063], now have 1 active actors on pipe
2025-10-09T21:43:15.216388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340848223662688:2432], now have 1 active actors on pipe
2025-10-09T21:43:15.216425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340847861854016:2883]
2025-10-09T21:43:15.216439Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:15.216458Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:15.216462Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 90
2025-10-09T21:43:15.216464Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 90
2025-10-09T21:43:15.218446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:15.218457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:15.218484Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f2a19ecc-73b20a1e-9fc07788-8d730f15_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:15.218494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.218504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.218507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:15.218518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.218534Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:15.218549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:15.222057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:15.222071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:15.222092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:15.222282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 91 partition: 0 MaxSeqNo: 25 sessionId: src_id|f2a19ecc-73b20a1e-9fc07788-8d730f15_0
2025-10-09T21:43:15.222742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74941ebd-b3b083f4-1149b9e0-cad6f902_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f2a19ecc-73b20a1e-9fc07788-8d730f15_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:15.226016Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a19ecc-73b20a1e-9fc07788-8d730f15_0] Write session: try to update token
2025-10-09T21:43:15.226364Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a19ecc-73b20a1e-9fc07788-8d730f15_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:15.239018Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:15.239039Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:15.239051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340847861854016:2883]
2025-10-09T21:43:15.239108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:15.239123Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:15.239326Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:15.239519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:15.240384Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:15.240592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:15.241259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:15.241309Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:15.252060Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 91 sessionId: src_id|f2a19ecc-73b20a1e-9fc07788-8d730f15_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:15.255001Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:15.257804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:15.257823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:15.257863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:15.258624Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:15.260047Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:15.260064Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:15.260066Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:15.260068Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:15.260079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340847861854016:2883]
2025-10-09T21:43:15.260084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340847861854016:2883]
2025-10-09T21:43:15.260088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340847861854016:2883]
2025-10-09T21:43:15.260127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:15.260135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.260138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:15.260284Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:15.262227Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:15.262241Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:15.262245Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:15.266681Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340848223662841:2476]
2025-10-09T21:43:15.270253Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 90, pipe: [1:7559340848223662841:2476]
2025-10-09T21:43:15.422160Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:15.422408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340848223662841:2476], now have 1 active actors on pipe
2025-10-09T21:43:15.432533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:15.432568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:15.432873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.433064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.433256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.433456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.433573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.433578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.433784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.433980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.434300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.434574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.434705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.434713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.435383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.435608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.436037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.436455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.436587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.436591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.436963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.437212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.437481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.437705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.437855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.437858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.438245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.438494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.438770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.439012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.439161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.439164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.439505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.439738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.439958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.440204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.440346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.440349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.440577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.440796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.441015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.441235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.441633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.441639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.441862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.442184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.442424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.442654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.442933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.442940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.443181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.443415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.443655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.443887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.444032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.444039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.444280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.444514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.445647Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:15.445665Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:15.445710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 91 reason: error at write: Disconnected sessionId: src_id|f2a19ecc-73b20a1e-9fc07788-8d730f15_0
2025-10-09T21:43:15.445949Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340848223662841:2476]
2025-10-09T21:43:15.445966Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:15.444757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.444997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.445267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.445270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.446402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a19ecc-73b20a1e-9fc07788-8d730f15_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:15.446440Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a19ecc-73b20a1e-9fc07788-8d730f15_0] Write session will restart in 0.006729s
2025-10-09T21:43:15.446484Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a19ecc-73b20a1e-9fc07788-8d730f15_0] Write session: Do CDS request
2025-10-09T21:43:15.446491Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a19ecc-73b20a1e-9fc07788-8d730f15_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:15.446789Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:15.447320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 91 sessionId: src_id|f2a19ecc-73b20a1e-9fc07788-8d730f15_0 is DEAD
2025-10-09T21:43:15.454680Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a19ecc-73b20a1e-9fc07788-8d730f15_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:15.454689Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:15.454703Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 92
2025-10-09T21:43:15.455156Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 92 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:15.455203Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 92 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:15.455209Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=92 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:15.455213Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:15.455845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 92 sessionId:  describe result for acl check
2025-10-09T21:43:15.455878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:15.455880Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:15.455883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:15.455888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340848223662862:3068] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:15.456392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340848223662862:3068] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:15.461087Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340848223662862:3068] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:15.471714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:15.480167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340848223662862:3068] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:15.480189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340848223662862:3068] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:15.480194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340848223662862:3068] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:15.480208Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 92 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:15.480167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:15.480442Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:15.480448Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:15.480463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:15.480538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:15.480603Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340847861854171:2885]
2025-10-09T21:43:15.481242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:15.482605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:15.482696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:15.483108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:15.483169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:15.483171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:15.483173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:15.483174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:15.483175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:15.483177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:15.483178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.483180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.483181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.483188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:15.483197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:15.483201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:15.483205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:15.483209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:15.483213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:15.483217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.483220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.483224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.483231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:15.483240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:15.483246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:15.483251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:15.483256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:15.483261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:15.483266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:15.483271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:15.483276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:15.483281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:15.483286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:15.483292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:15.483296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.483301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.483306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.483312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.483319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.483324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.483355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:15.495919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:15.497190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:15.497198Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:15.497201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:15.497206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:15.497214Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 91 [2:7559340847861854171:2885]
2025-10-09T21:43:15.497231Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:15.497251Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:15.497254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.497257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.497261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.497281Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:15.497287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:15.497295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:15.497359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.497362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.497365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.497368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:15.497453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:15.497456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.497470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.497482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.497499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:15.497520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340847861854165:2885]
2025-10-09T21:43:15.497526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340847861854165:2885]
2025-10-09T21:43:15.497528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340847861854165:2885]
2025-10-09T21:43:15.497544Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:15.497560Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:15.497574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:15.497577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:15.497621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340847861854165:2885]
2025-10-09T21:43:15.497624Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:15.497889Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:15.497892Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:15.497898Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:15.497903Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:15.500895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340848223662989:3068], now have 1 active actors on pipe
2025-10-09T21:43:15.500911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340848223662847:2432], now have 1 active actors on pipe
2025-10-09T21:43:15.500928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:15.500935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:15.500979Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f0e42e74-d013a91d-739b9640-ad5a32a2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:15.500996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.501001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.501005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:15.501020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.501037Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:15.501056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:15.498363Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 91
2025-10-09T21:43:15.498405Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 91
2025-10-09T21:43:15.501291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:15.501301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:15.501319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:15.502111Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 92 partition: 0 MaxSeqNo: 25 sessionId: src_id|f0e42e74-d013a91d-739b9640-ad5a32a2_0
2025-10-09T21:43:15.502690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a19ecc-73b20a1e-9fc07788-8d730f15_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f0e42e74-d013a91d-739b9640-ad5a32a2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:15.513605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0e42e74-d013a91d-739b9640-ad5a32a2_0] Write session: try to update token
2025-10-09T21:43:15.513645Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0e42e74-d013a91d-739b9640-ad5a32a2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:15.521440Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:15.521461Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:15.521463Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:15.521465Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:15.521475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340847861854165:2885]
2025-10-09T21:43:15.521480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340847861854165:2885]
2025-10-09T21:43:15.521484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340847861854165:2885]
2025-10-09T21:43:15.521535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:15.521543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.521542Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:15.521545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:15.521552Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:15.521556Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:15.521696Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:15.540473Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 92 sessionId: src_id|f0e42e74-d013a91d-739b9640-ad5a32a2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:15.545594Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340848223663002:2476]
2025-10-09T21:43:15.546516Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 91, pipe: [1:7559340848223663002:2476]
2025-10-09T21:43:15.552732Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:15.692715Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:15.692988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340848223663002:2476], now have 1 active actors on pipe
2025-10-09T21:43:15.693013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:15.693019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:15.693204Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:15.695560Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340848223663002:2476]
2025-10-09T21:43:15.695599Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:15.695627Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:15.695707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 92 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f0e42e74-d013a91d-739b9640-ad5a32a2_0
2025-10-09T21:43:15.695892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 92 sessionId: src_id|f0e42e74-d013a91d-739b9640-ad5a32a2_0 is DEAD
2025-10-09T21:43:15.697250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0e42e74-d013a91d-739b9640-ad5a32a2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:15.697261Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0e42e74-d013a91d-739b9640-ad5a32a2_0] Write session will restart in 0.005201s
2025-10-09T21:43:15.697288Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0e42e74-d013a91d-739b9640-ad5a32a2_0] Write session: Do CDS request
2025-10-09T21:43:15.697295Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0e42e74-d013a91d-739b9640-ad5a32a2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:15.706554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0e42e74-d013a91d-739b9640-ad5a32a2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:15.710823Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:15.710838Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 93
2025-10-09T21:43:15.710967Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 93 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:15.710998Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 93 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:15.711007Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=93 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:15.711010Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:15.711438Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 93 sessionId:  describe result for acl check
2025-10-09T21:43:15.711486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:15.711492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:15.711493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:15.711498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340848223663044:3079] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:15.712001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340848223663044:3079] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:15.717298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340848223663044:3079] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:15.720465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340848223663044:3079] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:15.720482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340848223663044:3079] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:15.720486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340848223663044:3079] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:15.720495Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 93 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:15.726169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:15.727364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:15.727571Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:15.727574Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:15.727587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:15.727639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:15.727673Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340847861854361:2894]
2025-10-09T21:43:15.728248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:15.728666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:15.728706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:15.729040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:15.729089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:15.729090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:15.729091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:15.729092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:15.729093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:15.729094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:15.729095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.729096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.729097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.729102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:15.729114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:15.729118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:15.729134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:15.729138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:15.729142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:15.729146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.729150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.729154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.729161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:15.729169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:15.729176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:15.729181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:15.729186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:15.729192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:15.729197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:15.729202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:15.729206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:15.729211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:15.729215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:15.729220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:15.729225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.729230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.729234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.729240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.729246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.729251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.729280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:15.737982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:15.739272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:15.739288Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:15.739292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:15.739296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:15.739303Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 92 [2:7559340847861854361:2894]
2025-10-09T21:43:15.739317Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:15.739334Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:15.739344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.739347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.739350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.739370Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:15.739384Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:15.739392Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:15.739454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.739463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.739465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.739468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:15.739548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:15.739557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.739569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.739582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.739606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:15.739629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340848223663166:3079], now have 1 active actors on pipe
2025-10-09T21:43:15.739639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340848223663020:2432], now have 1 active actors on pipe
2025-10-09T21:43:15.739647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340847861854355:2894]
2025-10-09T21:43:15.739654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340847861854355:2894]
2025-10-09T21:43:15.739656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340847861854355:2894]
2025-10-09T21:43:15.739669Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:15.739685Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:15.739688Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:15.739689Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:15.739691Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:15.739705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:15.739715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:15.739693Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 92
2025-10-09T21:43:15.739729Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 92
2025-10-09T21:43:15.739755Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340847861854355:2894]
2025-10-09T21:43:15.739767Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:15.739773Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:15.740018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:15.740030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:15.740064Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2d9534e-25750486-e6a14f63-df52b763_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:15.740075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.740080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.740082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:15.740095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.740107Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:15.740124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:15.740477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:15.740488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:15.740508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:15.740608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 93 partition: 0 MaxSeqNo: 25 sessionId: src_id|2d9534e-25750486-e6a14f63-df52b763_0
2025-10-09T21:43:15.740985Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0e42e74-d013a91d-739b9640-ad5a32a2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2d9534e-25750486-e6a14f63-df52b763_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:15.745071Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d9534e-25750486-e6a14f63-df52b763_0] Write session: try to update token
2025-10-09T21:43:15.745111Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d9534e-25750486-e6a14f63-df52b763_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:15.770581Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:15.770597Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:15.770618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340847861854355:2894]
2025-10-09T21:43:15.770670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:15.770926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:15.771083Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:15.771217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:15.771349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:15.771656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:15.771696Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:15.768982Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 93 sessionId: src_id|2d9534e-25750486-e6a14f63-df52b763_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:15.771737Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:15.771841Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:15.771959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:15.771974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:15.772006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:15.772383Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:15.780833Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:15.780851Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:15.780853Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:15.780856Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:15.780864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340847861854355:2894]
2025-10-09T21:43:15.780869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340847861854355:2894]
2025-10-09T21:43:15.780872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340847861854355:2894]
2025-10-09T21:43:15.780916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:15.780920Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:15.780924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.780927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:15.780929Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:15.780933Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:15.781063Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:15.795922Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340848223663190:2476]
2025-10-09T21:43:15.796421Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 92, pipe: [1:7559340848223663190:2476]
2025-10-09T21:43:15.940696Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:15.948567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:15.948593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:15.948757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.948816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.948886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.948951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.949003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.949008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.949071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.949125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.949185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.949249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.949291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.949294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.949359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.949408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.949464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.949527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.949571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.949574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.949645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.949708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.949770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.949829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.949868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.949871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.949934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.950187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.950269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.950341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.950384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.950386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.950473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.950532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.950590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.950646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.950708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.950711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.950782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.950835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.950894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.950954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.951100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.951103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.951174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.951227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.951282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.951342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.951470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.951473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.951544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.951603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.951660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.951719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.951865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.951869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.951945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:15.952003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:15.952064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:15.952125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:15.952279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:15.952283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:15.952348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340848223663190:2476], now have 1 active actors on pipe
2025-10-09T21:43:15.953254Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:15.954166Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:15.954224Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 93 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2d9534e-25750486-e6a14f63-df52b763_0
2025-10-09T21:43:15.954424Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 93 sessionId: src_id|2d9534e-25750486-e6a14f63-df52b763_0 is DEAD
2025-10-09T21:43:15.954567Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:15.954598Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340848223663190:2476]
2025-10-09T21:43:15.954688Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d9534e-25750486-e6a14f63-df52b763_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:15.954704Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d9534e-25750486-e6a14f63-df52b763_0] Write session will restart in 0.006796s
2025-10-09T21:43:15.954730Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d9534e-25750486-e6a14f63-df52b763_0] Write session: Do CDS request
2025-10-09T21:43:15.954737Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d9534e-25750486-e6a14f63-df52b763_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:15.965130Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d9534e-25750486-e6a14f63-df52b763_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:15.965263Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:15.965277Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 94
2025-10-09T21:43:15.965451Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 94 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:15.965487Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 94 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:15.965494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=94 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:15.965497Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:15.966154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 94 sessionId:  describe result for acl check
2025-10-09T21:43:15.966189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:15.966195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:15.966196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:15.966203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340848223663212:3085] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:15.966833Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340848223663212:3085] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:15.970106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340848223663212:3085] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:15.973107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340848223663212:3085] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:15.973119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340848223663212:3085] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:15.973123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340848223663212:3085] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:15.973132Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 94 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:15.976527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:15.977489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:15.977732Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:15.977742Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:15.977755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:15.977817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:15.977878Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340847861854512:2903]
2025-10-09T21:43:15.978473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:15.978914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:15.978974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:15.979331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:15.979384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:15.979390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:15.979392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:15.979393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:15.979394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:15.979395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:15.979396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.979397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.979399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.979403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:15.979414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:15.979418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:15.979422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:15.979427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:15.979431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:15.979435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.979439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.979443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.979450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:15.979458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:15.979464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:15.979470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:15.979474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:15.979479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:15.979485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:15.979490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:15.979495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:15.979500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:15.979504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:15.979510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:15.979515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.979519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.979522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.979526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.979530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.979535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.979564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:15.987907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:15.989171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:15.989184Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:15.989188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:15.989193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:15.989201Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 93 [2:7559340847861854512:2903]
2025-10-09T21:43:15.989215Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:15.989233Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:15.989236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.989240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.989243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.989264Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:15.989274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:15.989282Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:15.989351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.989355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.989357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.989360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:15.989429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:15.989437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:15.989451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:15.989463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:15.989478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:15.989506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340848223663201:2432], now have 1 active actors on pipe
2025-10-09T21:43:15.989514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340848223663331:3085], now have 1 active actors on pipe
2025-10-09T21:43:15.989522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340847861854507:2903]
2025-10-09T21:43:15.989524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340847861854507:2903]
2025-10-09T21:43:15.989526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340847861854507:2903]
2025-10-09T21:43:15.989537Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:15.989556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:15.989566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:15.989570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:15.989574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:15.989578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:15.989578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:15.989614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340847861854507:2903]
2025-10-09T21:43:15.989625Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:15.989647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:15.989622Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 93
2025-10-09T21:43:15.989629Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 93
2025-10-09T21:43:15.989671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:15.989683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:15.989712Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ab91fc2a-6a09a95f-a91984d6-c6404c9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:15.989719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.989723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.989726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:15.989736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.989747Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:15.989762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:15.989979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:15.989991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:15.990013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:15.990137Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 94 partition: 0 MaxSeqNo: 25 sessionId: src_id|ab91fc2a-6a09a95f-a91984d6-c6404c9_0
2025-10-09T21:43:15.990566Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d9534e-25750486-e6a14f63-df52b763_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ab91fc2a-6a09a95f-a91984d6-c6404c9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:15.993219Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ab91fc2a-6a09a95f-a91984d6-c6404c9_0] Write session: try to update token
2025-10-09T21:43:15.993241Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ab91fc2a-6a09a95f-a91984d6-c6404c9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:16.006862Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:16.006881Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:16.006883Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:16.006885Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:16.006895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340847861854507:2903]
2025-10-09T21:43:16.006899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340847861854507:2903]
2025-10-09T21:43:16.006903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340847861854507:2903]
2025-10-09T21:43:16.006952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:16.006958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.006956Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:16.006962Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:16.006962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:16.006965Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:16.007108Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:16.022604Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 94 sessionId: src_id|ab91fc2a-6a09a95f-a91984d6-c6404c9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:16.025810Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:16.054563Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340852518630646:2476]
2025-10-09T21:43:16.055059Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 93, pipe: [1:7559340852518630646:2476]
2025-10-09T21:43:16.183561Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:16.183946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:16.183961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:16.183985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340852518630646:2476], now have 1 active actors on pipe
2025-10-09T21:43:16.184218Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:16.184299Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:16.184351Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340852518630646:2476]
2025-10-09T21:43:16.184369Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:16.184440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 94 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ab91fc2a-6a09a95f-a91984d6-c6404c9_0
2025-10-09T21:43:16.184637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 94 sessionId: src_id|ab91fc2a-6a09a95f-a91984d6-c6404c9_0 is DEAD
2025-10-09T21:43:16.184822Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab91fc2a-6a09a95f-a91984d6-c6404c9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:16.184836Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab91fc2a-6a09a95f-a91984d6-c6404c9_0] Write session will restart in 0.009107s
2025-10-09T21:43:16.184865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab91fc2a-6a09a95f-a91984d6-c6404c9_0] Write session: Do CDS request
2025-10-09T21:43:16.184872Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab91fc2a-6a09a95f-a91984d6-c6404c9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:16.195627Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ab91fc2a-6a09a95f-a91984d6-c6404c9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:16.196354Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:16.196385Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 95
2025-10-09T21:43:16.196577Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 95 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:16.196605Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 95 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:16.196612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=95 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:16.196616Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:16.197002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 95 sessionId:  describe result for acl check
2025-10-09T21:43:16.197038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:16.197044Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:16.197046Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:16.197052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340852518630663:3089] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:16.197650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340852518630663:3089] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:16.201701Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340852518630663:3089] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:16.205194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340852518630663:3089] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:16.205207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340852518630663:3089] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:16.205211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340852518630663:3089] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:16.205201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:16.205220Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 95 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:16.206075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:16.206280Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:16.206292Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:16.206304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:16.206356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:16.206404Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340852156821957:2905]
2025-10-09T21:43:16.207071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:16.207335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:16.207376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:16.207693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:16.207744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:16.207749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:16.207751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:16.207752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:16.207753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:16.207754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:16.207755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.207756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.207758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.207763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:16.207774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:16.207778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:16.207782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:16.207786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:16.207790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:16.207794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.207809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.207818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.207827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:16.207836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:16.207842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:16.207847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:16.207851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:16.207855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:16.207860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:16.207866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:16.207875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:16.207880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:16.207884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:16.207890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:16.207899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.207905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.207909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.207918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.207923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.207928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.207956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:16.215030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:16.216199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:16.216213Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:16.216217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:16.216222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:16.216230Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 94 [2:7559340852156821957:2905]
2025-10-09T21:43:16.216245Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:16.216265Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:16.216269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.216273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.216276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.216295Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:16.216309Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:16.216318Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:16.216378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.216384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.216387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.216390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:16.216467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:16.216474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.216488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.216502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.216523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:16.216546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340852518630648:2432], now have 1 active actors on pipe
2025-10-09T21:43:16.216555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340852518630792:3089], now have 1 active actors on pipe
2025-10-09T21:43:16.216564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340852156821951:2905]
2025-10-09T21:43:16.216567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340852156821951:2905]
2025-10-09T21:43:16.216569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340852156821951:2905]
2025-10-09T21:43:16.216587Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:16.216607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:16.216624Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:16.216628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:16.216652Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:16.216663Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:16.216666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340852156821951:2905]
2025-10-09T21:43:16.216667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:16.216668Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:16.216675Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:16.216939Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 94
2025-10-09T21:43:16.216965Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 94
2025-10-09T21:43:16.217175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:16.217189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:16.217230Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ee21af2b-fe97315b-e16f162e-871d8332_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:16.217244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.217253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.217256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:16.217272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.217302Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:16.217319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:16.217524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:16.217535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:16.217556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:16.217647Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 95 partition: 0 MaxSeqNo: 25 sessionId: src_id|ee21af2b-fe97315b-e16f162e-871d8332_0
2025-10-09T21:43:16.218113Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab91fc2a-6a09a95f-a91984d6-c6404c9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ee21af2b-fe97315b-e16f162e-871d8332_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:16.221380Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee21af2b-fe97315b-e16f162e-871d8332_0] Write session: try to update token
2025-10-09T21:43:16.221404Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee21af2b-fe97315b-e16f162e-871d8332_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:16.233634Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:16.233652Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:16.233655Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:16.233656Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:16.233665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340852156821951:2905]
2025-10-09T21:43:16.233669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340852156821951:2905]
2025-10-09T21:43:16.233674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340852156821951:2905]
2025-10-09T21:43:16.233719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:16.233726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.233729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:16.233738Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:16.233745Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:16.233749Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:16.233872Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:16.241091Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 95 sessionId: src_id|ee21af2b-fe97315b-e16f162e-871d8332_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:16.244025Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:16.284443Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340852518630805:2476]
2025-10-09T21:43:16.284878Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 94, pipe: [1:7559340852518630805:2476]
2025-10-09T21:43:16.388088Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:16.388547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:16.388564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:16.388605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340852518630805:2476], now have 1 active actors on pipe
2025-10-09T21:43:16.388794Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:16.388824Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340852518630805:2476]
2025-10-09T21:43:16.388851Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:16.388859Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:16.388862Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:16.388937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 95 reason: error at write: Disconnected sessionId: src_id|ee21af2b-fe97315b-e16f162e-871d8332_0
2025-10-09T21:43:16.389128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 95 sessionId: src_id|ee21af2b-fe97315b-e16f162e-871d8332_0 is DEAD
2025-10-09T21:43:16.389298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee21af2b-fe97315b-e16f162e-871d8332_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:16.389311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee21af2b-fe97315b-e16f162e-871d8332_0] Write session will restart in 0.008668s
2025-10-09T21:43:16.389353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee21af2b-fe97315b-e16f162e-871d8332_0] Write session: Do CDS request
2025-10-09T21:43:16.389359Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee21af2b-fe97315b-e16f162e-871d8332_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:16.399631Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee21af2b-fe97315b-e16f162e-871d8332_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:16.399780Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:16.399820Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 96
2025-10-09T21:43:16.399970Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 96 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:16.399978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:16.400015Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 96 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:16.400027Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=96 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:16.400031Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:16.400468Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 96 sessionId:  describe result for acl check
2025-10-09T21:43:16.400501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:16.400503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:16.400504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:16.400509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340852518630921:3094] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:16.400853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:16.400993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340852518630921:3094] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:16.401022Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:16.401030Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:16.401039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:16.401090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:16.401126Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340852156822107:2907]
2025-10-09T21:43:16.401641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:16.401932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:16.401970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:16.402262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:16.402299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:16.402300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:16.402301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:16.402301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:16.402302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:16.402303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:16.402304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.402305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.402306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.402310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:16.402318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:16.402321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:16.402325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:16.402328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:16.402331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:16.402334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.402338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.402340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.402345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:16.402351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:16.402356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:16.402360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:16.402363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:16.402367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:16.402371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:16.402375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:16.402378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:16.402382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:16.402386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:16.402392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:16.402396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.402399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.402403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.402407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.402410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.402424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.402450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:16.403026Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340852518630921:3094] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:16.408416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340852518630921:3094] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:16.408430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340852518630921:3094] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:16.408434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340852518630921:3094] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:16.408451Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 96 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:16.408805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:16.410097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:16.410108Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:16.410111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:16.410114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:16.410120Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 95 [2:7559340852156822107:2907]
2025-10-09T21:43:16.410132Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:16.410149Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:16.410152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.410154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.410156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.410171Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:16.410176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:16.410182Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:16.410440Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 95
2025-10-09T21:43:16.410449Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 95
2025-10-09T21:43:16.410231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.410239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.410240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.410243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:16.410308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:16.410311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.410320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.410329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.410342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:16.410363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340852518630960:3094], now have 1 active actors on pipe
2025-10-09T21:43:16.410366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340852518630810:2432], now have 1 active actors on pipe
2025-10-09T21:43:16.410373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340852156822101:2907]
2025-10-09T21:43:16.410375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340852156822101:2907]
2025-10-09T21:43:16.410376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340852156822101:2907]
2025-10-09T21:43:16.410389Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:16.410402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:16.410409Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:16.410412Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:16.410412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:16.410421Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:16.410423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:16.410452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340852156822101:2907]
2025-10-09T21:43:16.410455Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:16.410476Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:16.410484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:16.410488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:16.410516Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4136d542-dd92a914-109f566b-9060a612_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:16.410523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.410526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.410528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:16.410538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.410546Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:16.410559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:16.410939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:16.410945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:16.410959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:16.411088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 96 partition: 0 MaxSeqNo: 25 sessionId: src_id|4136d542-dd92a914-109f566b-9060a612_0
2025-10-09T21:43:16.411670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee21af2b-fe97315b-e16f162e-871d8332_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4136d542-dd92a914-109f566b-9060a612_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:16.414806Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4136d542-dd92a914-109f566b-9060a612_0] Write session: try to update token
2025-10-09T21:43:16.414829Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4136d542-dd92a914-109f566b-9060a612_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:16.423754Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:16.423770Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:16.423778Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340852156822101:2907]
2025-10-09T21:43:16.423820Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:16.423822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:16.424338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:16.424498Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:16.424656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:16.424798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:16.424982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:16.425013Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:16.433747Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 96 sessionId: src_id|4136d542-dd92a914-109f566b-9060a612_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:16.436704Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:16.438121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:16.438140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:16.438181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:16.438358Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:16.442468Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:16.442482Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:16.442484Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:16.442486Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:16.442498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340852156822101:2907]
2025-10-09T21:43:16.442501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340852156822101:2907]
2025-10-09T21:43:16.442507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340852156822101:2907]
2025-10-09T21:43:16.442550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:16.442555Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:16.442559Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:16.442562Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:16.442562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.442565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:16.442714Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:16.489213Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340852518630967:2476]
2025-10-09T21:43:16.489478Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 95, pipe: [1:7559340852518630967:2476]
2025-10-09T21:43:16.603553Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:16.615797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:16.615828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:16.616010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:16.616066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:16.616127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:16.616191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:16.616244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:16.616249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:16.616320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:16.616379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:16.616443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:16.616505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:16.616545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:16.616549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:16.616618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:16.616675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:16.616736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:16.616796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:16.616838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:16.616841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:16.616916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:16.616983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:16.617047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:16.617110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:16.617155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:16.617159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:16.617232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:16.617293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:16.617355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:16.617418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:16.617461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:16.617464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:16.617544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:16.617609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:16.617669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:16.617727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:16.617770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:16.617773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:16.617848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:16.617912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:16.617980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:16.618333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:16.618404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:16.618408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:16.618490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:16.618552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:16.618621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:16.618688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:16.618737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:16.618740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:16.618814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:16.618878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:16.618944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:16.619011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:16.619056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:16.619062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:16.619133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:16.619189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:16.619283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:16.619340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:16.619432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:16.619435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:16.622255Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:16.622312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 96 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4136d542-dd92a914-109f566b-9060a612_0
2025-10-09T21:43:16.622323Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340852518630967:2476]
2025-10-09T21:43:16.622341Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:16.619509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340852518630967:2476], now have 1 active actors on pipe
2025-10-09T21:43:16.621071Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:16.622661Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 96 sessionId: src_id|4136d542-dd92a914-109f566b-9060a612_0 is DEAD
2025-10-09T21:43:16.624905Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4136d542-dd92a914-109f566b-9060a612_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:16.624920Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4136d542-dd92a914-109f566b-9060a612_0] Write session will restart in 0.008905s
2025-10-09T21:43:16.624946Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4136d542-dd92a914-109f566b-9060a612_0] Write session: Do CDS request
2025-10-09T21:43:16.624953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4136d542-dd92a914-109f566b-9060a612_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:16.638387Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4136d542-dd92a914-109f566b-9060a612_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:16.638594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:16.638624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 97
2025-10-09T21:43:16.638802Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 97 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:16.638851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 97 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:16.638861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=97 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:16.638865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:16.640482Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 97 sessionId:  describe result for acl check
2025-10-09T21:43:16.640526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:16.640528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:16.640530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:16.640536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340852518631066:3100] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:16.641129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340852518631066:3100] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:16.648738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340852518631066:3100] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:16.645978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:16.648435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:16.648648Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:16.648653Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:16.648667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:16.648729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:16.648767Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340852156822259:2909]
2025-10-09T21:43:16.649398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:16.649739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:16.649776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:16.650130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:16.650179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:16.650191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:16.650192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:16.650193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:16.650195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:16.650196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:16.650197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.650199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.650200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.650206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:16.650217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:16.650222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:16.650226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:16.650231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:16.650235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:16.650240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.650244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.650248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.650256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:16.650266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:16.650273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:16.650279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:16.650284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:16.650290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:16.650294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:16.650299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:16.650304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:16.650310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:16.650314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:16.650320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:16.650325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.650331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.650335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.650341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.650346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.650352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.650383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:16.655836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340852518631066:3100] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:16.655854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340852518631066:3100] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:16.655859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340852518631066:3100] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:16.655868Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 97 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:16.658483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:16.660115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:16.660130Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:16.660133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:16.660137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:16.660145Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 96 [2:7559340852156822259:2909]
2025-10-09T21:43:16.660159Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:16.660180Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:16.660183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.660187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.660189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.660206Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:16.660218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:16.660226Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:16.660286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.660297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.660298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.660302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:16.660380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:16.660387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.660401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.660413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.660431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:16.660458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340852156822254:2909]
2025-10-09T21:43:16.660464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340852156822254:2909]
2025-10-09T21:43:16.660466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340852156822254:2909]
2025-10-09T21:43:16.660475Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:16.660494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:16.660498Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:16.660501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:16.660504Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:16.660515Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:16.660518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:16.660560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340852156822254:2909]
2025-10-09T21:43:16.660571Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:16.660581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340852518631125:3100], now have 1 active actors on pipe
2025-10-09T21:43:16.660585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340852518630974:2432], now have 1 active actors on pipe
2025-10-09T21:43:16.660606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:16.660794Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 96
2025-10-09T21:43:16.660802Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 96
2025-10-09T21:43:16.660957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:16.660969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:16.660998Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ae485b7d-5f4011be-b813f28d-5fa847cc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:16.661012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.661017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.661028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:16.661041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.661054Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:16.661071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:16.661335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:16.661343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:16.661356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:16.661511Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 97 partition: 0 MaxSeqNo: 25 sessionId: src_id|ae485b7d-5f4011be-b813f28d-5fa847cc_0
2025-10-09T21:43:16.661979Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4136d542-dd92a914-109f566b-9060a612_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ae485b7d-5f4011be-b813f28d-5fa847cc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:16.664991Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae485b7d-5f4011be-b813f28d-5fa847cc_0] Write session: try to update token
2025-10-09T21:43:16.665009Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae485b7d-5f4011be-b813f28d-5fa847cc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:16.680298Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:16.680316Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:16.680318Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:16.680320Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:16.680329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340852156822254:2909]
2025-10-09T21:43:16.680334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340852156822254:2909]
2025-10-09T21:43:16.680338Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340852156822254:2909]
2025-10-09T21:43:16.680383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:16.680390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.680393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:16.680396Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:16.680404Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:16.680409Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:16.680529Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:16.685513Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 97 sessionId: src_id|ae485b7d-5f4011be-b813f28d-5fa847cc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:16.688400Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:16.722706Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340852518631139:2476]
2025-10-09T21:43:16.723280Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 96, pipe: [1:7559340852518631139:2476]
2025-10-09T21:43:16.844388Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:16.844687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:16.844698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:16.844746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340852518631139:2476], now have 1 active actors on pipe
2025-10-09T21:43:16.845182Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:16.847362Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340852518631139:2476]
2025-10-09T21:43:16.847398Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:16.847483Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 97 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ae485b7d-5f4011be-b813f28d-5fa847cc_0
2025-10-09T21:43:16.847683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 97 sessionId: src_id|ae485b7d-5f4011be-b813f28d-5fa847cc_0 is DEAD
2025-10-09T21:43:16.847863Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:16.850185Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae485b7d-5f4011be-b813f28d-5fa847cc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:16.850210Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae485b7d-5f4011be-b813f28d-5fa847cc_0] Write session will restart in 0.008807s
2025-10-09T21:43:16.850239Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae485b7d-5f4011be-b813f28d-5fa847cc_0] Write session: Do CDS request
2025-10-09T21:43:16.850247Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae485b7d-5f4011be-b813f28d-5fa847cc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:16.860515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae485b7d-5f4011be-b813f28d-5fa847cc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:16.860676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:16.860692Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 98
2025-10-09T21:43:16.860802Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 98 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:16.860833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 98 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:16.860839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=98 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:16.860842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:16.861349Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 98 sessionId:  describe result for acl check
2025-10-09T21:43:16.861390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:16.861396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:16.861398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:16.861404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340852518631184:3113] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:16.861937Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340852518631184:3113] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:16.866687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340852518631184:3113] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:16.871592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:16.872721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:16.872926Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:16.872930Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:16.872942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:16.872998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:16.873034Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340852156822453:2925]
2025-10-09T21:43:16.873611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:16.873997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:16.874064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:16.874403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:16.874461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:16.874462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:16.874463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:16.874465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:16.874466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:16.874467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:16.874468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.874469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.874470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.874475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:16.874484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:16.874487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:16.874491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:16.874495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:16.874499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:16.874503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.874506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.874510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.874517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:16.874524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:16.874530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:16.874536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:16.874540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:16.874544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:16.874550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:16.874555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:16.874559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:16.874564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:16.874569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:16.874574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:16.874579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.874583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.874588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.874592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.874597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.874602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.874627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:16.879335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340852518631184:3113] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:16.879354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340852518631184:3113] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:16.879359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340852518631184:3113] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:16.879371Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 98 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:16.883424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:16.884583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:16.884600Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:16.884603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:16.884609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:16.884617Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 97 [2:7559340852156822453:2925]
2025-10-09T21:43:16.884632Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:16.884654Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:16.884657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.884661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.884664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.884685Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:16.884690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:16.884697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:16.884766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.884776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.884778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.884781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:16.884903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:16.884912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.884925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:16.884936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:16.884952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:16.884983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340852156822448:2925]
2025-10-09T21:43:16.884985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340852156822448:2925]
2025-10-09T21:43:16.884987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340852156822448:2925]
2025-10-09T21:43:16.885005Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:16.885029Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:16.885043Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:16.885046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:16.885071Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:16.885075Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:16.885077Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:16.885089Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340852156822448:2925]
2025-10-09T21:43:16.885093Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:16.885171Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:16.885335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340852518631317:3113], now have 1 active actors on pipe
2025-10-09T21:43:16.885349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340852518631171:2432], now have 1 active actors on pipe
2025-10-09T21:43:16.887366Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 97
2025-10-09T21:43:16.887404Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 97
2025-10-09T21:43:16.887510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:16.887526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:16.887575Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7a0bacc9-3610bfa5-bffc86a9-92b3fab0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:16.887588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.887604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.887608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:16.887620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.887637Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:16.887659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:16.887994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:16.888003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:16.888023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:16.888168Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 98 partition: 0 MaxSeqNo: 25 sessionId: src_id|7a0bacc9-3610bfa5-bffc86a9-92b3fab0_0
2025-10-09T21:43:16.888615Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae485b7d-5f4011be-b813f28d-5fa847cc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7a0bacc9-3610bfa5-bffc86a9-92b3fab0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:16.891486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a0bacc9-3610bfa5-bffc86a9-92b3fab0_0] Write session: try to update token
2025-10-09T21:43:16.891506Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a0bacc9-3610bfa5-bffc86a9-92b3fab0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:16.903739Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:16.903757Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:16.903760Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:16.903762Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:16.903772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340852156822448:2925]
2025-10-09T21:43:16.903776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340852156822448:2925]
2025-10-09T21:43:16.903780Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340852156822448:2925]
2025-10-09T21:43:16.903830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:16.903837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:16.903840Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:16.903846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:16.903847Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:16.903851Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:16.904010Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:16.922217Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 98 sessionId: src_id|7a0bacc9-3610bfa5-bffc86a9-92b3fab0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:16.924977Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:16.947444Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340852518631327:2476]
2025-10-09T21:43:16.947856Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 97, pipe: [1:7559340852518631327:2476]
2025-10-09T21:43:17.122825Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:17.123160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:17.123170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:17.123196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340852518631327:2476], now have 1 active actors on pipe
2025-10-09T21:43:17.123461Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:17.123476Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:17.123541Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 98 reason: error at write: Disconnected sessionId: src_id|7a0bacc9-3610bfa5-bffc86a9-92b3fab0_0
2025-10-09T21:43:17.123466Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:17.123736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 98 sessionId: src_id|7a0bacc9-3610bfa5-bffc86a9-92b3fab0_0 is DEAD
2025-10-09T21:43:17.123850Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340852518631327:2476]
2025-10-09T21:43:17.123867Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:17.124048Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a0bacc9-3610bfa5-bffc86a9-92b3fab0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:17.124061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a0bacc9-3610bfa5-bffc86a9-92b3fab0_0] Write session will restart in 0.005684s
2025-10-09T21:43:17.124091Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a0bacc9-3610bfa5-bffc86a9-92b3fab0_0] Write session: Do CDS request
2025-10-09T21:43:17.124097Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a0bacc9-3610bfa5-bffc86a9-92b3fab0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:17.130608Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a0bacc9-3610bfa5-bffc86a9-92b3fab0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:17.134190Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:17.134206Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 99
2025-10-09T21:43:17.138167Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 99 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:17.138235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 99 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:17.138246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=99 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:17.138251Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:17.139187Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 99 sessionId:  describe result for acl check
2025-10-09T21:43:17.139234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:17.139240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:17.139242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:17.139249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340856813598725:3117] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:17.139852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340856813598725:3117] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:17.142311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340856813598725:3117] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:17.142480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:17.144041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:17.144263Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:17.144274Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:17.144288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:17.144346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:17.144389Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340856451789902:2927]
2025-10-09T21:43:17.144993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:17.145353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:17.145414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:17.145782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:17.145835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:17.145841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:17.145842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:17.145844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:17.145846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:17.145847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:17.145848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.145850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.145851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.145857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:17.145868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:17.145872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:17.145876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:17.145881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:17.145886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:17.145891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.145901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.145905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.145919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:17.145931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:17.145943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:17.145954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:17.145959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:17.145970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:17.145976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:17.145986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:17.145992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:17.146001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:17.146009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:17.146015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:17.146025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.146067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.146077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.146083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.146094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.146102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.146139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:17.147074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340856813598725:3117] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:17.147092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340856813598725:3117] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:17.147098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340856813598725:3117] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:17.147109Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 99 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:17.153965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:17.155264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:17.155281Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:17.155285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:17.155290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:17.155299Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 98 [2:7559340856451789902:2927]
2025-10-09T21:43:17.155316Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:17.155734Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 98
2025-10-09T21:43:17.155735Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 98
2025-10-09T21:43:17.155338Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:17.155352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.155355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.155358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.155379Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:17.155386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:17.155395Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:17.155463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.155466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.155469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.155472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:17.155579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:17.155583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.155597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.155612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.155632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:17.155657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340856813598782:3117], now have 1 active actors on pipe
2025-10-09T21:43:17.155662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340856813598629:2432], now have 1 active actors on pipe
2025-10-09T21:43:17.155671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340856451789896:2927]
2025-10-09T21:43:17.155673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340856451789896:2927]
2025-10-09T21:43:17.155676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340856451789896:2927]
2025-10-09T21:43:17.155694Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:17.155709Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:17.155711Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:17.155713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:17.155718Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:17.155728Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:17.155732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:17.155773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340856451789896:2927]
2025-10-09T21:43:17.155777Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:17.155779Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:17.155798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:17.155804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:17.155864Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2dec1314-a411a178-48a4355f-786b0d31_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:17.155872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.155877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.155880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:17.155891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.155902Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:17.155929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:17.156198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:17.156208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:17.156224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:17.156394Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 99 partition: 0 MaxSeqNo: 25 sessionId: src_id|2dec1314-a411a178-48a4355f-786b0d31_0
2025-10-09T21:43:17.156839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a0bacc9-3610bfa5-bffc86a9-92b3fab0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2dec1314-a411a178-48a4355f-786b0d31_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:17.159840Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2dec1314-a411a178-48a4355f-786b0d31_0] Write session: try to update token
2025-10-09T21:43:17.159861Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2dec1314-a411a178-48a4355f-786b0d31_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:17.185159Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 99 sessionId: src_id|2dec1314-a411a178-48a4355f-786b0d31_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:17.186321Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:17.186339Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:17.186342Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:17.186344Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:17.186354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340856451789896:2927]
2025-10-09T21:43:17.186358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340856451789896:2927]
2025-10-09T21:43:17.186362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340856451789896:2927]
2025-10-09T21:43:17.186403Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:17.186408Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:17.186408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:17.186411Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:17.186429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.186432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:17.186626Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:17.188021Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:17.223435Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340856813598790:2476]
2025-10-09T21:43:17.223741Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 98, pipe: [1:7559340856813598790:2476]
2025-10-09T21:43:17.373782Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:17.374171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:17.374188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:17.374239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340856813598790:2476], now have 1 active actors on pipe
2025-10-09T21:43:17.374466Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:17.376092Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2dec1314-a411a178-48a4355f-786b0d31_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:17.376104Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2dec1314-a411a178-48a4355f-786b0d31_0] Write session will restart in 0.008092s
2025-10-09T21:43:17.376129Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2dec1314-a411a178-48a4355f-786b0d31_0] Write session: Do CDS request
2025-10-09T21:43:17.376135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2dec1314-a411a178-48a4355f-786b0d31_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:17.375414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:17.375498Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 99 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2dec1314-a411a178-48a4355f-786b0d31_0
2025-10-09T21:43:17.375690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 99 sessionId: src_id|2dec1314-a411a178-48a4355f-786b0d31_0 is DEAD
2025-10-09T21:43:17.375853Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:17.375884Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340856813598790:2476]
2025-10-09T21:43:17.385600Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2dec1314-a411a178-48a4355f-786b0d31_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:17.387373Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:17.387388Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 100
2025-10-09T21:43:17.388573Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 100 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:17.388612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 100 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:17.388617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=100 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:17.388621Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:17.389936Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 100 sessionId:  describe result for acl check
2025-10-09T21:43:17.389969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:17.389971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:17.389973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:17.389979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340856813598824:3122] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:17.390575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340856813598824:3122] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:17.404408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340856813598824:3122] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:17.407039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:17.407903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340856813598824:3122] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:17.407917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340856813598824:3122] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:17.407921Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340856813598824:3122] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:17.407931Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 100 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:17.408164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:17.408405Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:17.408409Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:17.408422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:17.408491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:17.408526Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340856451790056:2929]
2025-10-09T21:43:17.409149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:17.409504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:17.409552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:17.409935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:17.409977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:17.409978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:17.409980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:17.409981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:17.409983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:17.409984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:17.409985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.409987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.409988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.409994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:17.410006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:17.410010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:17.410015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:17.410020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:17.410024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:17.410045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.410048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.410052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.410060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:17.410068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:17.410074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:17.410081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:17.410085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:17.410090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:17.410094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:17.410098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:17.410102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:17.410106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:17.410111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:17.410116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:17.410120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.410124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.410128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.410133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.410139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.410144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.410170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:17.418996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:17.420572Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 99
2025-10-09T21:43:17.420616Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 99
2025-10-09T21:43:17.420134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:17.420152Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:17.420155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:17.420160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:17.420167Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 99 [2:7559340856451790056:2929]
2025-10-09T21:43:17.420183Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:17.420204Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:17.420206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.420210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.420212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.420230Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:17.420236Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:17.420245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:17.420306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.420308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.420310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.420313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:17.420386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:17.420389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.420401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.420412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.420427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:17.420448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340856451790049:2929]
2025-10-09T21:43:17.420450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340856451790049:2929]
2025-10-09T21:43:17.420453Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340856451790049:2929]
2025-10-09T21:43:17.420462Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:17.420472Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:17.420475Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:17.420480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340856813598794:2432], now have 1 active actors on pipe
2025-10-09T21:43:17.420481Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:17.420489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:17.420503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:17.420507Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:17.420512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340856813598944:3122], now have 1 active actors on pipe
2025-10-09T21:43:17.420550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340856451790049:2929]
2025-10-09T21:43:17.420561Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:17.420676Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:17.422904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:17.422921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:17.422956Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|341c2c09-ffa42277-ce30011c-1070bdbf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:17.422970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.422976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.422979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:17.422993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.423011Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:17.423029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:17.426181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:17.426200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:17.426235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:17.426825Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2dec1314-a411a178-48a4355f-786b0d31_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|341c2c09-ffa42277-ce30011c-1070bdbf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:17.426386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 100 partition: 0 MaxSeqNo: 25 sessionId: src_id|341c2c09-ffa42277-ce30011c-1070bdbf_0
2025-10-09T21:43:17.429349Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|341c2c09-ffa42277-ce30011c-1070bdbf_0] Write session: try to update token
2025-10-09T21:43:17.429369Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|341c2c09-ffa42277-ce30011c-1070bdbf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:17.443162Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:17.443181Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:17.443183Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:17.443186Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:17.443196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340856451790049:2929]
2025-10-09T21:43:17.443200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340856451790049:2929]
2025-10-09T21:43:17.443204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340856451790049:2929]
2025-10-09T21:43:17.443266Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:17.443269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:17.443273Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:17.443276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.443276Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:17.443279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:17.443439Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:17.448227Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 100 sessionId: src_id|341c2c09-ffa42277-ce30011c-1070bdbf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:17.450890Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:17.475344Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340856813598955:2476]
2025-10-09T21:43:17.478410Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 99, pipe: [1:7559340856813598955:2476]
2025-10-09T21:43:17.608674Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:17.610371Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340856813598955:2476]
2025-10-09T21:43:17.610424Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:17.610459Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:17.610530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 100 reason: status is not ok: tablet will be restarted right now sessionId: src_id|341c2c09-ffa42277-ce30011c-1070bdbf_0
2025-10-09T21:43:17.611040Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|341c2c09-ffa42277-ce30011c-1070bdbf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:17.611057Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|341c2c09-ffa42277-ce30011c-1070bdbf_0] Write session will restart in 0.005085s
2025-10-09T21:43:17.611082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|341c2c09-ffa42277-ce30011c-1070bdbf_0] Write session: Do CDS request
2025-10-09T21:43:17.611089Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|341c2c09-ffa42277-ce30011c-1070bdbf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:17.610715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 100 sessionId: src_id|341c2c09-ffa42277-ce30011c-1070bdbf_0 is DEAD
2025-10-09T21:43:17.609199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:17.609216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:17.609264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340856813598955:2476], now have 1 active actors on pipe
2025-10-09T21:43:17.609972Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:17.617617Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|341c2c09-ffa42277-ce30011c-1070bdbf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:17.617821Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:17.617841Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 101
2025-10-09T21:43:17.618128Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 101 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:17.618206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 101 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:17.618220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=101 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:17.618226Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:17.618743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 101 sessionId:  describe result for acl check
2025-10-09T21:43:17.618788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:17.618791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:17.618793Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:17.618800Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340856813598971:3126] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:17.619434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340856813598971:3126] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:17.628195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340856813598971:3126] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:17.630983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340856813598971:3126] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:17.631013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340856813598971:3126] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:17.631019Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340856813598971:3126] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:17.631028Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 101 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:17.631856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:17.633198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:17.633447Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:17.633458Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:17.633471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:17.633524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:17.633566Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340856451790210:2931]
2025-10-09T21:43:17.634126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:17.635413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:17.635465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:17.635806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:17.635851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:17.635857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:17.635858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:17.635859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:17.635860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:17.635861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:17.635862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.635864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.635864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.635870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:17.635881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:17.635885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:17.635889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:17.635893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:17.635897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:17.635901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.635905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.635909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.635917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:17.635925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:17.635931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:17.635940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:17.635944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:17.635950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:17.635955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:17.635960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:17.635965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:17.635970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:17.635973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:17.635978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:17.635982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.635986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.635990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.635995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.635999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.636005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.636034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:17.645250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:17.646582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:17.646595Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:17.646599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:17.646604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:17.646612Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 100 [2:7559340856451790210:2931]
2025-10-09T21:43:17.646627Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:17.646647Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:17.646651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.646654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.646657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.646676Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:17.646681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:17.646688Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:17.646751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.646752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.646754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.646757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:17.646831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:17.646839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.646852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.646864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.646880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:17.646901Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340856451790204:2931]
2025-10-09T21:43:17.646904Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340856451790204:2931]
2025-10-09T21:43:17.646906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340856451790204:2931]
2025-10-09T21:43:17.646914Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:17.646925Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:17.646928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:17.646928Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:17.646931Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:17.646943Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:17.646945Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:17.646956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340856813599108:3126], now have 1 active actors on pipe
2025-10-09T21:43:17.646960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340856813598958:2432], now have 1 active actors on pipe
2025-10-09T21:43:17.646997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340856451790204:2931]
2025-10-09T21:43:17.647008Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:17.647082Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 100
2025-10-09T21:43:17.647115Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 100
2025-10-09T21:43:17.647209Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:17.647395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:17.647410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:17.647443Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a6ed8c1c-d1b46600-89e5b04a-723d4041_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:17.647459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.647463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.647471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:17.647485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.647495Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:17.647514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:17.648212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:17.648222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:17.648247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:17.648365Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 101 partition: 0 MaxSeqNo: 25 sessionId: src_id|a6ed8c1c-d1b46600-89e5b04a-723d4041_0
2025-10-09T21:43:17.648868Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|341c2c09-ffa42277-ce30011c-1070bdbf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a6ed8c1c-d1b46600-89e5b04a-723d4041_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:17.651594Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6ed8c1c-d1b46600-89e5b04a-723d4041_0] Write session: try to update token
2025-10-09T21:43:17.651613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6ed8c1c-d1b46600-89e5b04a-723d4041_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:17.665889Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:17.665912Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:17.665915Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:17.665918Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:17.665929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340856451790204:2931]
2025-10-09T21:43:17.665934Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340856451790204:2931]
2025-10-09T21:43:17.665938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340856451790204:2931]
2025-10-09T21:43:17.665985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:17.665987Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:17.665994Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:17.665998Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:17.666005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.666008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:17.666202Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:17.678531Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 101 sessionId: src_id|a6ed8c1c-d1b46600-89e5b04a-723d4041_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:17.681358Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:17.710324Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 11 EndOffset 21 InitDone 1 WaitForData 0, pipe: [1:7559340856813599120:2476]
2025-10-09T21:43:17.710733Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 100, pipe: [1:7559340856813599120:2476]
2025-10-09T21:43:17.840911Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:17.841224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:17.841238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:17.841260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340856813599120:2476], now have 1 active actors on pipe
2025-10-09T21:43:17.841521Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:17.841695Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340856813599120:2476]
2025-10-09T21:43:17.841729Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:17.841774Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:17.841805Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:43:17.841807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 101 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a6ed8c1c-d1b46600-89e5b04a-723d4041_0
2025-10-09T21:43:17.841902Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 1 consumer shared/user session shared/user_1_1_7577822143567276789_v1 is DEAD
2025-10-09T21:43:17.841921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 101 sessionId: src_id|a6ed8c1c-d1b46600-89e5b04a-723d4041_0 is DEAD
2025-10-09T21:43:17.842044Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340745144436843:2470] disconnected.
2025-10-09T21:43:17.842050Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340745144436843:2470] disconnected; active server actors: 1
2025-10-09T21:43:17.842053Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340745144436843:2470] client user disconnected session shared/user_1_1_7577822143567276789_v1
2025-10-09T21:43:17.842595Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6ed8c1c-d1b46600-89e5b04a-723d4041_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:17.842612Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6ed8c1c-d1b46600-89e5b04a-723d4041_0] Write session will restart in 0.009264s
2025-10-09T21:43:17.842596Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:43:17.842635Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6ed8c1c-d1b46600-89e5b04a-723d4041_0] Write session: Do CDS request
2025-10-09T21:43:17.842641Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6ed8c1c-d1b46600-89e5b04a-723d4041_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:17.842760Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 6758903, ReadSizeServerDelta = -6234615
2025-10-09T21:43:17.842769Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:43:17.842819Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.005081s
2025-10-09T21:43:17.848527Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:43:17.848792Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:43:17.848805Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 2
2025-10-09T21:43:17.849189Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:43:17.849249Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:43:17.849346Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 auth for : user
2025-10-09T21:43:17.849649Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 Handle describe topics response
2025-10-09T21:43:17.849676Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 auth is DEAD
2025-10-09T21:43:17.849680Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:43:17.849924Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:43:17.850013Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_2_11159088021448487788_v1
2025-10-09T21:43:17.851885Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340856813599176:3139] connected; active server actors: 1
2025-10-09T21:43:17.852512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6ed8c1c-d1b46600-89e5b04a-723d4041_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:17.852046Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559340856813599176:3139] session shared/user_1_2_11159088021448487788_v1
2025-10-09T21:43:17.852066Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:43:17.852078Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:43:17.852085Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_2_11159088021448487788_v1" (Sender=[1:7559340856813599172:3139], Pipe=[1:7559340856813599176:3139], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:43:17.852089Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:43:17.852104Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:43:17.852115Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_2_11159088021448487788_v1" (Sender=[1:7559340856813599172:3139], Pipe=[1:7559340856813599176:3139], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:43:17.852125Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_2_11159088021448487788_v1" sender [1:7559340856813599172:3139] lock partition 0 for ReadingSession "shared/user_1_2_11159088021448487788_v1" (Sender=[1:7559340856813599172:3139], Pipe=[1:7559340856813599176:3139], Partitions=[], ActiveFamilyCount=1) generation 1 step 2
2025-10-09T21:43:17.852134Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:43:17.852139Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000031s
2025-10-09T21:43:17.852382Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 2 Session: "shared/user_1_2_11159088021448487788_v1" ClientId: "user" PipeClient { RawX1: 7559340856813599176 RawX2: 4503603922340931 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:43:17.852400Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:43:17.855055Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:17.855068Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 102
2025-10-09T21:43:17.855216Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 102 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:17.855266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 102 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:17.855280Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=102 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:17.855284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:17.855952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 102 sessionId:  describe result for acl check
2025-10-09T21:43:17.855989Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:17.855995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:17.855996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:17.856002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340856813599248:3143] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:17.856616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340856813599248:3143] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:17.863540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340856813599248:3143] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:17.865005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:17.866169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:17.866367Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:17.866378Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:17.866393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:17.866471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:17.866519Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340856451790407:2947]
2025-10-09T21:43:17.866775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340856813599248:3143] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:17.866792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340856813599248:3143] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:17.866795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340856813599248:3143] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:17.866805Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 102 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:17.867081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:17.867465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:17.867516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:17.867925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:17.867988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:17.867994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:17.867995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:17.867996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:17.867997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:17.867998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:17.867999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.868000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.868001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.868007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:17.868017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:17.868021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:17.868025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:17.868028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:17.868033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:17.868036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.868040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.868044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.868051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:17.868059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:17.868064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:17.868069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:17.868074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:17.868078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:17.868088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:17.868093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:17.868102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:17.868107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:17.868110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:17.868114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:17.868119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.868130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.868133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.868142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.868146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.868150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.868179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:17.874813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:17.876095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:17.876110Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:17.876112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:17.876116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:17.876123Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 101 [2:7559340856451790407:2947]
2025-10-09T21:43:17.876139Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:17.876160Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:17.876163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.876166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.876168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.876187Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:17.876194Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:17.876202Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:17.876264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.876272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.876273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.876276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:17.876346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:17.876351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.876362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:17.876375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:17.876392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:17.876411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340856451790401:2947]
2025-10-09T21:43:17.876414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340856451790401:2947]
2025-10-09T21:43:17.877307Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 101
2025-10-09T21:43:17.877344Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 101, pipe: [1:7559340856813599214:3142]
2025-10-09T21:43:17.877362Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 101
2025-10-09T21:43:17.876416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340856451790401:2947]
2025-10-09T21:43:17.876430Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:17.876443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:17.876459Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:17.876462Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:17.876498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340856451790401:2947]
2025-10-09T21:43:17.876501Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:17.876820Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:17.876822Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:17.876824Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:17.876831Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:17.877581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340856813599314:3143], now have 1 active actors on pipe
2025-10-09T21:43:17.877594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340856813599214:3142], now have 1 active actors on pipe
2025-10-09T21:43:17.877597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340856813599160:2432], now have 1 active actors on pipe
2025-10-09T21:43:17.877612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:17.877618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:17.877637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:17.877640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:17.877648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_2_11159088021448487788_v1 on pipe: [1:7559340856813599214:3142]
2025-10-09T21:43:17.877704Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f862eb9b-ef95d253-f55faa3c-d3bd3f64_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:17.877716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.877720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.877723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:17.877735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.877749Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:17.877761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.877766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.877768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:17.877788Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_2_11159088021448487788_v1
2025-10-09T21:43:17.877791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.877834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:17.877853Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:17.877984Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_2_11159088021448487788_v1:1 with generation 101
2025-10-09T21:43:17.881061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:17.881077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:17.881107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:17.882330Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6ed8c1c-d1b46600-89e5b04a-723d4041_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f862eb9b-ef95d253-f55faa3c-d3bd3f64_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:17.881874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 102 partition: 0 MaxSeqNo: 25 sessionId: src_id|f862eb9b-ef95d253-f55faa3c-d3bd3f64_0
2025-10-09T21:43:17.885019Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f862eb9b-ef95d253-f55faa3c-d3bd3f64_0] Write session: try to update token
2025-10-09T21:43:17.885037Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f862eb9b-ef95d253-f55faa3c-d3bd3f64_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:17.886285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:17.886333Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:17.886344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.886347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.886349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.886362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:43:17.886378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:43:17.886690Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046197877 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:43:17.886712Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:43:17.886748Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 sending to client partition status
2025-10-09T21:43:17.895303Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:17.895323Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:17.895326Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:17.895328Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:17.895341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340856451790401:2947]
2025-10-09T21:43:17.895346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340856451790401:2947]
2025-10-09T21:43:17.895350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340856451790401:2947]
2025-10-09T21:43:17.895396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:17.895402Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:17.895408Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:17.895409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:17.895411Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:17.895412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:17.895582Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:17.907924Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 102 sessionId: src_id|f862eb9b-ef95d253-f55faa3c-d3bd3f64_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:17.910403Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:18.060825Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:18.061213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:18.061231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:18.062091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_2_11159088021448487788_v1
2025-10-09T21:43:18.062118Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:18.061620Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:18.062155Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f862eb9b-ef95d253-f55faa3c-d3bd3f64_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:18.062186Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f862eb9b-ef95d253-f55faa3c-d3bd3f64_0] Write session will restart in 0.005188s
2025-10-09T21:43:18.061636Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:18.061700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 102 reason: error at write: Disconnected sessionId: src_id|f862eb9b-ef95d253-f55faa3c-d3bd3f64_0
2025-10-09T21:43:18.061903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 102 sessionId: src_id|f862eb9b-ef95d253-f55faa3c-d3bd3f64_0 is DEAD
2025-10-09T21:43:18.061912Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340856813599214:3142]
2025-10-09T21:43:18.061931Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:18.062213Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f862eb9b-ef95d253-f55faa3c-d3bd3f64_0] Write session: Do CDS request
2025-10-09T21:43:18.062232Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f862eb9b-ef95d253-f55faa3c-d3bd3f64_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:18.062478Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_2_11159088021448487788_v1
2025-10-09T21:43:18.068561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f862eb9b-ef95d253-f55faa3c-d3bd3f64_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:18.068817Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:18.068834Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 103
2025-10-09T21:43:18.069108Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 103 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:18.069145Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 103 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:18.069157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=103 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:18.069161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:18.069669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 103 sessionId:  describe result for acl check
2025-10-09T21:43:18.069709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:18.069717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:18.069719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:18.069725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340861108566642:3147] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:18.070328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340861108566642:3147] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:18.072802Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340861108566642:3147] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:18.077126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340861108566642:3147] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:18.077144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340861108566642:3147] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:18.077148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340861108566642:3147] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:18.077158Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 103 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:18.081086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:18.084535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:18.084850Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:18.084865Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:18.084879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:18.084956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:18.085005Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340860746757864:2949]
2025-10-09T21:43:18.085627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:18.086064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:18.086117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:18.086479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:18.086551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.086560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.086561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.086562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.086563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.086565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.086566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.086567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.086568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.086574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.086586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.086589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.086594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.086598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.086603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.086607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.086611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.086615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.086622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.086629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.086635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.086641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.086648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.086654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.086658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.086664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.086669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.086676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.086680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.086686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.086691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.086696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.086708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.086713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.086718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.086722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.086752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:18.094943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:18.096508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:18.096527Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:18.096530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:18.096536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:18.096544Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 102 [2:7559340860746757864:2949]
2025-10-09T21:43:18.096562Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:18.096584Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:18.096588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.096592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.096595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.096617Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:18.096630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:18.096638Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:18.096714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.096724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.096726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.096730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:18.096808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:18.096812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.096825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.096840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.096859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:18.096881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340860746757859:2949]
2025-10-09T21:43:18.096886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340860746757859:2949]
2025-10-09T21:43:18.096889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340860746757859:2949]
2025-10-09T21:43:18.096900Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:18.096915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:18.096918Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:18.096931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:18.096933Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:18.096934Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:18.096935Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:18.096947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340861108566755:3147], now have 1 active actors on pipe
2025-10-09T21:43:18.096951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340861108566628:2432], now have 1 active actors on pipe
2025-10-09T21:43:18.096994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340860746757859:2949]
2025-10-09T21:43:18.097010Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:18.097023Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:18.097032Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 102
2025-10-09T21:43:18.097074Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 102
2025-10-09T21:43:18.097264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:18.097279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:18.097314Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ba776679-61d9e642-b7131faf-3c42e400_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:18.097329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.097334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.097341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:18.097355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.097370Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:18.097403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:18.097754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:18.097768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:18.097791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:18.097888Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 103 partition: 0 MaxSeqNo: 25 sessionId: src_id|ba776679-61d9e642-b7131faf-3c42e400_0
2025-10-09T21:43:18.098320Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f862eb9b-ef95d253-f55faa3c-d3bd3f64_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ba776679-61d9e642-b7131faf-3c42e400_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:18.101152Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ba776679-61d9e642-b7131faf-3c42e400_0] Write session: try to update token
2025-10-09T21:43:18.101174Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ba776679-61d9e642-b7131faf-3c42e400_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:18.126219Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:18.126238Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:18.126241Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:18.126244Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:18.126256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340860746757859:2949]
2025-10-09T21:43:18.126261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340860746757859:2949]
2025-10-09T21:43:18.126265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340860746757859:2949]
2025-10-09T21:43:18.126315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:18.126324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.126327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:18.126502Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:18.127955Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 103 sessionId: src_id|ba776679-61d9e642-b7131faf-3c42e400_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:18.130146Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:18.130167Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:18.130171Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:18.131367Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:18.166296Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340861108566790:3142]
2025-10-09T21:43:18.170574Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 102, pipe: [1:7559340861108566790:3142]
2025-10-09T21:43:18.291173Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:18.291501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:18.291519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:18.291564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340861108566790:3142], now have 1 active actors on pipe
2025-10-09T21:43:18.291790Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:18.292367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba776679-61d9e642-b7131faf-3c42e400_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:18.292385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba776679-61d9e642-b7131faf-3c42e400_0] Write session will restart in 0.009990s
2025-10-09T21:43:18.292419Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba776679-61d9e642-b7131faf-3c42e400_0] Write session: Do CDS request
2025-10-09T21:43:18.292426Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba776679-61d9e642-b7131faf-3c42e400_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:18.291890Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:18.291936Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:18.291940Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:18.292000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 103 reason: error at write: Disconnected sessionId: src_id|ba776679-61d9e642-b7131faf-3c42e400_0
2025-10-09T21:43:18.292184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 103 sessionId: src_id|ba776679-61d9e642-b7131faf-3c42e400_0 is DEAD
2025-10-09T21:43:18.292308Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340861108566790:3142]
2025-10-09T21:43:18.303828Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ba776679-61d9e642-b7131faf-3c42e400_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:18.303962Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:18.303989Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 104
2025-10-09T21:43:18.304254Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 104 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:18.304314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 104 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:18.304325Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=104 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:18.304330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:18.304861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 104 sessionId:  describe result for acl check
2025-10-09T21:43:18.304924Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:18.304933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:18.304935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:18.304941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340861108566831:3151] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:18.305499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340861108566831:3151] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:18.312899Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340861108566831:3151] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:18.317858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:18.318790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:18.319070Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:18.319074Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:18.319088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:18.319154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:18.319190Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340860746758021:2951]
2025-10-09T21:43:18.319782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:18.320100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:18.320157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:18.320496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:18.320541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.320543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.320544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.320545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.320546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.320547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.320549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.320550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.320551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.320557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.320568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.320573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.320577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.320580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.320585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.320589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.320593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.320597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.320604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.320613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.320621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.320627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.320632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.320638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.320642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.320647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.320651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.320657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.320661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.320666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.320672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.320678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.320682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.320687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.320691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.320697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.320729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:18.330682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340861108566831:3151] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:18.330702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340861108566831:3151] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:18.330706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340861108566831:3151] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:18.330717Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 104 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:18.335450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:18.336781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:18.336801Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:18.336804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:18.336808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:18.336815Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 103 [2:7559340860746758021:2951]
2025-10-09T21:43:18.336829Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:18.336847Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:18.336854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.336857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.336859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.336876Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:18.336885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:18.336899Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:18.336961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.336969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.336971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.336974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:18.337047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:18.337054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.337066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.337082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.337099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:18.337122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340860746758016:2951]
2025-10-09T21:43:18.337129Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340860746758016:2951]
2025-10-09T21:43:18.337131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340860746758016:2951]
2025-10-09T21:43:18.337146Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:18.337160Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:18.337174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:18.337181Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:18.337454Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 103
2025-10-09T21:43:18.337497Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 103
2025-10-09T21:43:18.337204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:18.337215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:18.337217Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:18.337220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340860746758016:2951]
2025-10-09T21:43:18.337222Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:18.337224Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:18.337570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340861108566793:2432], now have 1 active actors on pipe
2025-10-09T21:43:18.337574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340861108566950:3151], now have 1 active actors on pipe
2025-10-09T21:43:18.337727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:18.337739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:18.337769Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f1e81a4-a813f635-620a21b6-4c9b9637_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:18.337796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.337805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.337807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:18.337819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.337832Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:18.337848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:18.338589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:18.338601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:18.338621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:18.338938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 104 partition: 0 MaxSeqNo: 25 sessionId: src_id|f1e81a4-a813f635-620a21b6-4c9b9637_0
2025-10-09T21:43:18.339462Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba776679-61d9e642-b7131faf-3c42e400_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f1e81a4-a813f635-620a21b6-4c9b9637_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:18.341958Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f1e81a4-a813f635-620a21b6-4c9b9637_0] Write session: try to update token
2025-10-09T21:43:18.341978Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f1e81a4-a813f635-620a21b6-4c9b9637_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:18.354395Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:18.354437Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:18.354439Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:18.354442Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:18.354451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340860746758016:2951]
2025-10-09T21:43:18.354456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340860746758016:2951]
2025-10-09T21:43:18.354460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340860746758016:2951]
2025-10-09T21:43:18.354506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:18.354517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.354520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:18.354656Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:18.358099Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:18.358114Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:18.358119Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:18.359567Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 104 sessionId: src_id|f1e81a4-a813f635-620a21b6-4c9b9637_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:18.361891Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:18.392308Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340861108566961:3142]
2025-10-09T21:43:18.392576Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 103, pipe: [1:7559340861108566961:3142]
2025-10-09T21:43:18.516135Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:18.516841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:18.516860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:18.516909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340861108566961:3142], now have 1 active actors on pipe
2025-10-09T21:43:18.517238Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:18.517302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:18.517305Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:18.517411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 104 reason: error at write: Disconnected sessionId: src_id|f1e81a4-a813f635-620a21b6-4c9b9637_0
2025-10-09T21:43:18.517781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f1e81a4-a813f635-620a21b6-4c9b9637_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:18.517800Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f1e81a4-a813f635-620a21b6-4c9b9637_0] Write session will restart in 0.006115s
2025-10-09T21:43:18.517829Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f1e81a4-a813f635-620a21b6-4c9b9637_0] Write session: Do CDS request
2025-10-09T21:43:18.517837Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f1e81a4-a813f635-620a21b6-4c9b9637_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:18.517605Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:18.517602Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 104 sessionId: src_id|f1e81a4-a813f635-620a21b6-4c9b9637_0 is DEAD
2025-10-09T21:43:18.517756Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340861108566961:3142]
2025-10-09T21:43:18.525711Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f1e81a4-a813f635-620a21b6-4c9b9637_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:18.525802Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:18.525818Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 105
2025-10-09T21:43:18.526003Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 105 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:18.526049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 105 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:18.526055Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=105 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:18.526058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:18.526513Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 105 sessionId:  describe result for acl check
2025-10-09T21:43:18.526554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:18.526556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:18.526558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:18.526564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340861108566977:3156] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:18.527147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340861108566977:3156] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:18.529586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340861108566977:3156] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:18.534156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340861108566977:3156] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:18.534175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340861108566977:3156] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:18.534180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340861108566977:3156] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:18.534189Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 105 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:18.539742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:18.541029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:18.541405Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:18.541419Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:18.541436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:18.541510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:18.541555Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340860746758182:2953]
2025-10-09T21:43:18.542128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:18.542548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:18.542592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:18.542893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:18.542937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.542938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.542939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.542942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.542943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.542944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.542945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.542946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.542947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.542952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.542964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.542968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.542972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.542976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.542981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.542984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.542988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.542992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.543000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.543009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.543015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.543021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.543026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.543031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.543035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.543040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.543045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.543050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.543054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.543059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.543064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.543071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.543075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.543080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.543085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.543090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.543120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:18.550554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:18.552242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:18.552251Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:18.552253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:18.552257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:18.552263Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 104 [2:7559340860746758182:2953]
2025-10-09T21:43:18.552273Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:18.552290Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:18.552292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.552294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.552295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.552309Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:18.552313Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:18.552319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:18.552365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.552366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.552368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.552370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:18.552499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:18.552502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.552511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.552518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.552529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:18.552545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340860746758176:2953]
2025-10-09T21:43:18.552547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340860746758176:2953]
2025-10-09T21:43:18.552548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340860746758176:2953]
2025-10-09T21:43:18.552558Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:18.552570Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:18.552579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:18.552581Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:18.552614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340860746758176:2953]
2025-10-09T21:43:18.552615Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:18.552988Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:18.552996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:18.552998Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:18.553004Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:18.553482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 104
2025-10-09T21:43:18.553518Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 104
2025-10-09T21:43:18.554150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340861108567076:3156], now have 1 active actors on pipe
2025-10-09T21:43:18.554177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340861108566963:2432], now have 1 active actors on pipe
2025-10-09T21:43:18.554199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:18.554206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:18.554241Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|96182e2d-7587d516-85ffa0dc-b259b46_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:18.554252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.554257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.554259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:18.554271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.554284Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:18.554303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:18.554941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:18.554951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:18.554981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:18.555140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 105 partition: 0 MaxSeqNo: 25 sessionId: src_id|96182e2d-7587d516-85ffa0dc-b259b46_0
2025-10-09T21:43:18.555534Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f1e81a4-a813f635-620a21b6-4c9b9637_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|96182e2d-7587d516-85ffa0dc-b259b46_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:18.563643Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96182e2d-7587d516-85ffa0dc-b259b46_0] Write session: try to update token
2025-10-09T21:43:18.563668Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96182e2d-7587d516-85ffa0dc-b259b46_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:18.571457Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:18.571478Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:18.571481Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:18.571483Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:18.571493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340860746758176:2953]
2025-10-09T21:43:18.571499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340860746758176:2953]
2025-10-09T21:43:18.571503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340860746758176:2953]
2025-10-09T21:43:18.571553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:18.571561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.571569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:18.571949Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:18.574188Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:18.574199Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:18.574205Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:18.605087Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 105 sessionId: src_id|96182e2d-7587d516-85ffa0dc-b259b46_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:18.607545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:18.618127Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340861108567130:3142]
2025-10-09T21:43:18.618432Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 104, pipe: [1:7559340861108567130:3142]
2025-10-09T21:43:18.737362Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:18.737687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:18.737697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:18.737719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340861108567130:3142], now have 1 active actors on pipe
2025-10-09T21:43:18.738075Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:18.738090Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:18.738160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 105 reason: error at write: Disconnected sessionId: src_id|96182e2d-7587d516-85ffa0dc-b259b46_0
2025-10-09T21:43:18.738341Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 105 sessionId: src_id|96182e2d-7587d516-85ffa0dc-b259b46_0 is DEAD
2025-10-09T21:43:18.738537Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96182e2d-7587d516-85ffa0dc-b259b46_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:18.738550Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96182e2d-7587d516-85ffa0dc-b259b46_0] Write session will restart in 0.006522s
2025-10-09T21:43:18.738581Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96182e2d-7587d516-85ffa0dc-b259b46_0] Write session: Do CDS request
2025-10-09T21:43:18.738588Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96182e2d-7587d516-85ffa0dc-b259b46_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:18.738511Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340861108567130:3142]
2025-10-09T21:43:18.738532Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:18.739538Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:18.746541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96182e2d-7587d516-85ffa0dc-b259b46_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:18.747056Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:18.747075Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 106
2025-10-09T21:43:18.747270Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 106 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:18.747315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 106 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:18.747326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=106 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:18.747330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:18.747794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 106 sessionId:  describe result for acl check
2025-10-09T21:43:18.747832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:18.747835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:18.747837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:18.747843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340861108567149:3160] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:18.748413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340861108567149:3160] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:18.752875Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340861108567149:3160] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:18.754744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:18.755756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:18.755989Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:18.756000Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:18.756012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:18.756068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:18.756076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340861108567149:3160] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:18.756090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340861108567149:3160] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:18.756094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340861108567149:3160] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:18.756102Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 106 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:18.756110Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340860746758344:2956]
2025-10-09T21:43:18.756732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:18.757086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:18.757148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:18.757499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:18.757566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.757572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.757574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.757575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.757576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.757577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.757578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.757579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.757580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.757586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.757596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.757600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.757604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.757607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.757612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.757615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.757619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.757622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.757630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.757638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.757650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.757656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.757660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.757665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.757669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.757674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.757679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.757683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.757687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.757692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.757697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.757707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.757711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.757732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.757741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.757746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.757775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:18.768239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:18.769502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:18.769517Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:18.769520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:18.769524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:18.769531Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 105 [2:7559340860746758344:2956]
2025-10-09T21:43:18.769547Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:18.769566Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:18.769569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.769572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.769575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.769590Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:18.769601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:18.769609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:18.769677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.769685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.769687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.769690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:18.769764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:18.769771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.769784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.769797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.769813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:18.769838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340860746758339:2956]
2025-10-09T21:43:18.769844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340860746758339:2956]
2025-10-09T21:43:18.769846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340860746758339:2956]
2025-10-09T21:43:18.769859Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:18.769878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:18.769897Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:18.769900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:18.769916Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:18.769928Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:18.769932Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:18.769940Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340860746758339:2956]
2025-10-09T21:43:18.769948Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:18.770020Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:18.770171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340861108567135:2432], now have 1 active actors on pipe
2025-10-09T21:43:18.770186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340861108567293:3160], now have 1 active actors on pipe
2025-10-09T21:43:18.770358Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 105
2025-10-09T21:43:18.770424Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 105
2025-10-09T21:43:18.770479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:18.770493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:18.770526Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|469f5364-eb3ea768-80f1723d-b41af316_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:18.770539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.770550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.770553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:18.770565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.770577Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:18.770613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:18.770848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:18.770858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:18.770874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:18.771021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 106 partition: 0 MaxSeqNo: 25 sessionId: src_id|469f5364-eb3ea768-80f1723d-b41af316_0
2025-10-09T21:43:18.771449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96182e2d-7587d516-85ffa0dc-b259b46_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|469f5364-eb3ea768-80f1723d-b41af316_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:18.774078Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|469f5364-eb3ea768-80f1723d-b41af316_0] Write session: try to update token
2025-10-09T21:43:18.774099Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|469f5364-eb3ea768-80f1723d-b41af316_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:18.788385Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:18.788405Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:18.788407Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:18.788410Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:18.788420Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340860746758339:2956]
2025-10-09T21:43:18.788424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340860746758339:2956]
2025-10-09T21:43:18.788429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340860746758339:2956]
2025-10-09T21:43:18.788481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:18.788488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.788490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:18.788493Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:18.788501Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:18.788505Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:18.788737Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:18.799817Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 106 sessionId: src_id|469f5364-eb3ea768-80f1723d-b41af316_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:18.802463Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:18.838211Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340861108567323:3142]
2025-10-09T21:43:18.842107Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 105, pipe: [1:7559340861108567323:3142]
2025-10-09T21:43:18.953659Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:18.953707Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:18.953710Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:18.952933Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:18.953396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:18.953408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:18.953432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340861108567323:3142], now have 1 active actors on pipe
2025-10-09T21:43:18.954453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|469f5364-eb3ea768-80f1723d-b41af316_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:18.954465Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|469f5364-eb3ea768-80f1723d-b41af316_0] Write session will restart in 0.009752s
2025-10-09T21:43:18.954494Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|469f5364-eb3ea768-80f1723d-b41af316_0] Write session: Do CDS request
2025-10-09T21:43:18.954501Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|469f5364-eb3ea768-80f1723d-b41af316_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:18.954392Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:18.953773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 106 reason: error at write: Disconnected sessionId: src_id|469f5364-eb3ea768-80f1723d-b41af316_0
2025-10-09T21:43:18.953968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 106 sessionId: src_id|469f5364-eb3ea768-80f1723d-b41af316_0 is DEAD
2025-10-09T21:43:18.954085Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340861108567323:3142]
2025-10-09T21:43:18.965586Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|469f5364-eb3ea768-80f1723d-b41af316_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:18.965972Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:18.965994Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 107
2025-10-09T21:43:18.966227Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 107 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:18.966265Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 107 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:18.966271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=107 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:18.966275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:18.966752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 107 sessionId:  describe result for acl check
2025-10-09T21:43:18.966795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:18.966798Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:18.966799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:18.966805Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340861108567362:3172] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:18.967405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340861108567362:3172] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:18.970634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340861108567362:3172] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:18.975236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340861108567362:3172] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:18.975251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340861108567362:3172] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:18.975255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340861108567362:3172] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:18.975265Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 107 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:18.976388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:18.977461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:18.977696Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:18.977706Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:18.977719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:18.977777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:18.977820Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340860746758546:2971]
2025-10-09T21:43:18.978410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:18.978968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:18.979038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:18.979400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:18.979452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.979458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.979460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.979461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.979462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.979464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.979465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.979466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.979467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.979473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.979485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.979489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.979494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.979498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.979502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.979506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.979510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.979515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.979522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.979532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:18.979540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.979547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:18.979551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.979557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:18.979562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.979568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:18.979573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.979579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:18.979583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.979589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:18.979602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.979608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.979612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.979618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.979623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.979628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.979659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:18.995947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:18.997171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:18.997178Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:18.997181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:18.997186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:18.997196Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 106 [2:7559340860746758546:2971]
2025-10-09T21:43:18.997211Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:18.997232Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:18.997236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.997239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.997241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.997261Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:18.997267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:18.997277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:18.997348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.997351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.997355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.997358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:18.997433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:18.997436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:18.997450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:18.997464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:18.997483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:18.997504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340860746758541:2971]
2025-10-09T21:43:18.997506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340860746758541:2971]
2025-10-09T21:43:18.997509Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340860746758541:2971]
2025-10-09T21:43:18.997521Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:18.997536Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:18.997552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:18.997555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:18.997596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340860746758541:2971]
2025-10-09T21:43:18.997598Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:18.998019Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:18.998022Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:18.998024Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:18.998048Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:18.998336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340861108567337:2432], now have 1 active actors on pipe
2025-10-09T21:43:18.998349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340861108567487:3172], now have 1 active actors on pipe
2025-10-09T21:43:18.998806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:18.998822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:18.998192Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 106
2025-10-09T21:43:18.998244Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 106
2025-10-09T21:43:18.998858Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|14d64556-c2227aef-f9dde55-f466687f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:18.998883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.998888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.998891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:18.998904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.998920Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:18.998951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:19.002576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:19.002599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:19.002635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:19.005933Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 107 partition: 0 MaxSeqNo: 25 sessionId: src_id|14d64556-c2227aef-f9dde55-f466687f_0
2025-10-09T21:43:19.006463Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|469f5364-eb3ea768-80f1723d-b41af316_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|14d64556-c2227aef-f9dde55-f466687f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:19.009029Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14d64556-c2227aef-f9dde55-f466687f_0] Write session: try to update token
2025-10-09T21:43:19.009054Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14d64556-c2227aef-f9dde55-f466687f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:19.019827Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:19.019846Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:19.019849Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:19.019854Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:19.019864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340860746758541:2971]
2025-10-09T21:43:19.019869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340860746758541:2971]
2025-10-09T21:43:19.019872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340860746758541:2971]
2025-10-09T21:43:19.019921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:19.019935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.019938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:19.020075Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:19.020140Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:19.020156Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:19.020160Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:19.039925Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 107 sessionId: src_id|14d64556-c2227aef-f9dde55-f466687f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:19.043270Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:19.056501Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340865403534799:3142]
2025-10-09T21:43:19.057393Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 106, pipe: [1:7559340865403534799:3142]
2025-10-09T21:43:19.189011Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:19.189504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:19.189521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:19.189545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340865403534799:3142], now have 1 active actors on pipe
2025-10-09T21:43:19.189898Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:19.190292Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:19.190340Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:19.190431Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 107 reason: status is not ok: tablet will be restarted right now sessionId: src_id|14d64556-c2227aef-f9dde55-f466687f_0
2025-10-09T21:43:19.190616Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 107 sessionId: src_id|14d64556-c2227aef-f9dde55-f466687f_0 is DEAD
2025-10-09T21:43:19.190791Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340865403534799:3142]
2025-10-09T21:43:19.194149Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14d64556-c2227aef-f9dde55-f466687f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:19.194165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14d64556-c2227aef-f9dde55-f466687f_0] Write session will restart in 0.009681s
2025-10-09T21:43:19.194198Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14d64556-c2227aef-f9dde55-f466687f_0] Write session: Do CDS request
2025-10-09T21:43:19.194206Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14d64556-c2227aef-f9dde55-f466687f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:19.209106Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14d64556-c2227aef-f9dde55-f466687f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:19.209652Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:19.209667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 108
2025-10-09T21:43:19.209827Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 108 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:19.209854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 108 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:19.209860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=108 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:19.209864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:19.210440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 108 sessionId:  describe result for acl check
2025-10-09T21:43:19.210477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:19.210479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:19.210481Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:19.210486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340865403534813:3176] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:19.211063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340865403534813:3176] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:19.213198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340865403534813:3176] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:19.218371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340865403534813:3176] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:19.218390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340865403534813:3176] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:19.218395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340865403534813:3176] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:19.218404Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 108 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:19.232078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:19.233269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:19.233762Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:19.233767Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:19.233782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:19.233852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:19.233935Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340865041726006:2973]
2025-10-09T21:43:19.234654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:19.235912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:19.235964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:19.236396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:19.236459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:19.236463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:19.236464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:19.236467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:19.236469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:19.236470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:19.236471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.236472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.236475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.236481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:19.236493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:19.236497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:19.236501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:19.236505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:19.236512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:19.236518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.236522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.236526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.236533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:19.236542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:19.236550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:19.236555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:19.236570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:19.236575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:19.236583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:19.236588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:19.236593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:19.236599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:19.236608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:19.236614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:19.236619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.236624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.236630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.236635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.236639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.236644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.236675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:19.248911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:19.251891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:19.251909Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:19.251912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:19.251917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:19.251925Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 107 [2:7559340865041726006:2973]
2025-10-09T21:43:19.251941Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:19.251964Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:19.251970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.251973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.251975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.251994Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:19.252000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:19.252010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:19.252081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.252084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.252086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.252089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:19.252182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:19.252187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.252202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.252216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.252235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:19.252256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340865041726000:2973]
2025-10-09T21:43:19.252259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340865041726000:2973]
2025-10-09T21:43:19.252261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340865041726000:2973]
2025-10-09T21:43:19.252272Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:19.252286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:19.252300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:19.252304Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:19.252309Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:19.252312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:19.252314Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:19.252346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340865041726000:2973]
2025-10-09T21:43:19.252351Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:19.252354Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:19.252363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340865403534895:3176], now have 1 active actors on pipe
2025-10-09T21:43:19.252367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340865403534801:2432], now have 1 active actors on pipe
2025-10-09T21:43:19.253184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:19.253204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:19.253238Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2372ff48-acb13b0f-e6f86f0b-8d5afb2f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:19.253253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.253257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.253259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:19.253271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.253295Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:19.253330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:19.253733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:19.253747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:19.253768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:19.252431Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 107
2025-10-09T21:43:19.252547Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 107
2025-10-09T21:43:19.253925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 108 partition: 0 MaxSeqNo: 25 sessionId: src_id|2372ff48-acb13b0f-e6f86f0b-8d5afb2f_0
2025-10-09T21:43:19.254714Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14d64556-c2227aef-f9dde55-f466687f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2372ff48-acb13b0f-e6f86f0b-8d5afb2f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:19.259722Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2372ff48-acb13b0f-e6f86f0b-8d5afb2f_0] Write session: try to update token
2025-10-09T21:43:19.259744Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2372ff48-acb13b0f-e6f86f0b-8d5afb2f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:19.272598Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:19.272615Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:19.272618Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:19.272620Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:19.272630Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340865041726000:2973]
2025-10-09T21:43:19.272635Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340865041726000:2973]
2025-10-09T21:43:19.272639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340865041726000:2973]
2025-10-09T21:43:19.272681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:19.272688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.272691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:19.272693Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:19.272700Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:19.272704Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:19.273805Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:19.290134Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340865403534971:3142]
2025-10-09T21:43:19.290795Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 107, pipe: [1:7559340865403534971:3142]
2025-10-09T21:43:19.318953Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 108 sessionId: src_id|2372ff48-acb13b0f-e6f86f0b-8d5afb2f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:19.321499Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:19.591626Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:19.592006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340865403534971:3142], now have 1 active actors on pipe
2025-10-09T21:43:19.592035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:19.592042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:19.598191Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:19.598205Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:19.598286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 108 reason: error at write: Disconnected sessionId: src_id|2372ff48-acb13b0f-e6f86f0b-8d5afb2f_0
2025-10-09T21:43:19.598475Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 108 sessionId: src_id|2372ff48-acb13b0f-e6f86f0b-8d5afb2f_0 is DEAD
2025-10-09T21:43:19.598585Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340865403534971:3142]
2025-10-09T21:43:19.598612Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:19.599154Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2372ff48-acb13b0f-e6f86f0b-8d5afb2f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:19.599166Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2372ff48-acb13b0f-e6f86f0b-8d5afb2f_0] Write session will restart in 0.009315s
2025-10-09T21:43:19.599194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2372ff48-acb13b0f-e6f86f0b-8d5afb2f_0] Write session: Do CDS request
2025-10-09T21:43:19.599200Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2372ff48-acb13b0f-e6f86f0b-8d5afb2f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:19.606429Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:19.609830Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2372ff48-acb13b0f-e6f86f0b-8d5afb2f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:19.610533Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:19.610565Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 109
2025-10-09T21:43:19.610733Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 109 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:19.610770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 109 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:19.610778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=109 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:19.610789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:19.611519Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 109 sessionId:  describe result for acl check
2025-10-09T21:43:19.611564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:19.611572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:19.611574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:19.611591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340865403534994:3181] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:19.612277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340865403534994:3181] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:19.617402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340865403534994:3181] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:19.621170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340865403534994:3181] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:19.621172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:19.621189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340865403534994:3181] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:19.621193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340865403534994:3181] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:19.621202Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 109 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:19.622742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:19.623030Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:19.623057Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:19.623077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:19.623164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:19.623362Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340865041726171:2975]
2025-10-09T21:43:19.623917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:19.624483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:19.624539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:19.624915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:19.624972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:19.624973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:19.624974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:19.624975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:19.624977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:19.624978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:19.624979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.624980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.624982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.624987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:19.625006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:19.625010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:19.625014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:19.625018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:19.625022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:19.625026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.625030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.625034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.625042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:19.625050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:19.625055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:19.625061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:19.625065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:19.625071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:19.625075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:19.625080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:19.625085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:19.625090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:19.625094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:19.625099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:19.625104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.625109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.625113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.625118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.625124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.625128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.625158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:19.632115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:19.633418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:19.633431Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:19.633433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:19.633438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:19.633446Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 108 [2:7559340865041726171:2975]
2025-10-09T21:43:19.633462Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:19.633480Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:19.633483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.633486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.633489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.633506Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:19.633513Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:19.633521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:19.633581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.633583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.633585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.633588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:19.633722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:19.633734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.633748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.633761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.633780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:19.633800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340865041726165:2975]
2025-10-09T21:43:19.633805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340865041726165:2975]
2025-10-09T21:43:19.633807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340865041726165:2975]
2025-10-09T21:43:19.633823Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:19.633844Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:19.633857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:19.633860Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:19.633861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:19.633862Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:19.633864Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:19.633902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340865041726165:2975]
2025-10-09T21:43:19.633905Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:19.633942Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:19.634223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340865403535136:3181], now have 1 active actors on pipe
2025-10-09T21:43:19.634231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340865403534978:2432], now have 1 active actors on pipe
2025-10-09T21:43:19.634263Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 108
2025-10-09T21:43:19.634330Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 108
2025-10-09T21:43:19.634452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:19.634462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:19.634495Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|379afeb5-35f3982b-51d58fee-b5411fb7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:19.634505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.634509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.634511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:19.634521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.634533Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:19.634549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:19.634863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:19.634871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:19.634887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:19.635011Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 109 partition: 0 MaxSeqNo: 25 sessionId: src_id|379afeb5-35f3982b-51d58fee-b5411fb7_0
2025-10-09T21:43:19.635444Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2372ff48-acb13b0f-e6f86f0b-8d5afb2f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|379afeb5-35f3982b-51d58fee-b5411fb7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:19.638527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|379afeb5-35f3982b-51d58fee-b5411fb7_0] Write session: try to update token
2025-10-09T21:43:19.638551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|379afeb5-35f3982b-51d58fee-b5411fb7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:19.651262Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:19.651280Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:19.651283Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:19.651285Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:19.651294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340865041726165:2975]
2025-10-09T21:43:19.651300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340865041726165:2975]
2025-10-09T21:43:19.651304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340865041726165:2975]
2025-10-09T21:43:19.651348Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:19.651352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:19.651353Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:19.651357Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:19.651358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.651361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:19.651502Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:19.655343Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 109 sessionId: src_id|379afeb5-35f3982b-51d58fee-b5411fb7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:19.658341Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:19.702223Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340865403535148:3142]
2025-10-09T21:43:19.702739Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 108, pipe: [1:7559340865403535148:3142]
2025-10-09T21:43:19.811005Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:19.811389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:19.811416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:19.811466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340865403535148:3142], now have 1 active actors on pipe
2025-10-09T21:43:19.812843Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:19.812820Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:19.812840Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:19.812886Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340865403535148:3142]
2025-10-09T21:43:19.812907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 109 reason: error at write: Disconnected sessionId: src_id|379afeb5-35f3982b-51d58fee-b5411fb7_0
2025-10-09T21:43:19.812915Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:19.813090Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 109 sessionId: src_id|379afeb5-35f3982b-51d58fee-b5411fb7_0 is DEAD
2025-10-09T21:43:19.813533Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|379afeb5-35f3982b-51d58fee-b5411fb7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:19.813540Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|379afeb5-35f3982b-51d58fee-b5411fb7_0] Write session will restart in 0.006183s
2025-10-09T21:43:19.813565Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|379afeb5-35f3982b-51d58fee-b5411fb7_0] Write session: Do CDS request
2025-10-09T21:43:19.813573Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|379afeb5-35f3982b-51d58fee-b5411fb7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:19.820629Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|379afeb5-35f3982b-51d58fee-b5411fb7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:19.823534Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:19.823550Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 110
2025-10-09T21:43:19.823717Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 110 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:19.823749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 110 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:19.823755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=110 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:19.823759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:19.824186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 110 sessionId:  describe result for acl check
2025-10-09T21:43:19.824224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:19.824229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:19.824231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:19.824237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340865403535172:3186] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:19.824712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340865403535172:3186] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:19.827113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340865403535172:3186] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:19.831371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340865403535172:3186] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:19.831387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340865403535172:3186] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:19.831391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340865403535172:3186] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:19.831402Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 110 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:19.844150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:19.845156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:19.845362Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:19.845373Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:19.845386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:19.845444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:19.845494Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340865041726338:2978]
2025-10-09T21:43:19.846063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:19.846585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:19.846643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:19.846949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:19.846993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:19.846994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:19.846995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:19.846996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:19.846997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:19.846999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:19.847000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.847001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.847003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.847008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:19.847019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:19.847022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:19.847026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:19.847030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:19.847034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:19.847038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.847041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.847045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.847051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:19.847059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:19.847065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:19.847071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:19.847075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:19.847078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:19.847082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:19.847087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:19.847091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:19.847095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:19.847099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:19.847105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:19.847109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.847114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.847118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.847124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.847128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.847133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.847162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:19.856949Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 109
2025-10-09T21:43:19.856994Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 109
2025-10-09T21:43:19.857613Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 110 partition: 0 MaxSeqNo: 25 sessionId: src_id|111bf144-247aafa2-6953623-8ea56362_0
2025-10-09T21:43:19.855066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:19.856354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:19.856361Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:19.856364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:19.856368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:19.856375Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 109 [2:7559340865041726338:2978]
2025-10-09T21:43:19.856395Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:19.856414Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:19.856418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.856421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.856423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.856441Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:19.856447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:19.856454Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:19.856527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.856529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.856531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.856534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:19.856610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:19.856612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.856623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:19.856634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:19.856652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:19.856672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340865041726333:2978]
2025-10-09T21:43:19.856675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340865041726333:2978]
2025-10-09T21:43:19.856677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340865041726333:2978]
2025-10-09T21:43:19.856686Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:19.856701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:19.856717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:19.856721Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:19.856758Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340865041726333:2978]
2025-10-09T21:43:19.856761Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:19.856769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:19.856783Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:19.856786Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:19.856792Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:19.856870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340865403535294:3186], now have 1 active actors on pipe
2025-10-09T21:43:19.856873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340865403535158:2432], now have 1 active actors on pipe
2025-10-09T21:43:19.857006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:19.857013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:19.857054Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|111bf144-247aafa2-6953623-8ea56362_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:19.858024Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|379afeb5-35f3982b-51d58fee-b5411fb7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|111bf144-247aafa2-6953623-8ea56362_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:19.857065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.857070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.857073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:19.857085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.857095Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:19.857111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:19.857357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:19.857363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:19.857384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:19.860760Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|111bf144-247aafa2-6953623-8ea56362_0] Write session: try to update token
2025-10-09T21:43:19.860780Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|111bf144-247aafa2-6953623-8ea56362_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:19.886851Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:19.886869Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:19.886872Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:19.886874Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:19.886884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340865041726333:2978]
2025-10-09T21:43:19.886889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340865041726333:2978]
2025-10-09T21:43:19.886892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340865041726333:2978]
2025-10-09T21:43:19.886935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:19.886942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:19.886945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:19.887199Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:19.894381Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:19.894397Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:19.894400Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:19.910375Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340865403535346:3142]
2025-10-09T21:43:19.910689Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 109, pipe: [1:7559340865403535346:3142]
2025-10-09T21:43:19.918097Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 110 sessionId: src_id|111bf144-247aafa2-6953623-8ea56362_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:19.920487Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:20.066634Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:20.067200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340865403535346:3142], now have 1 active actors on pipe
2025-10-09T21:43:20.067233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:20.067240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:20.070436Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:20.070371Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:20.070444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:20.070447Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:20.070525Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 110 reason: error at write: Disconnected sessionId: src_id|111bf144-247aafa2-6953623-8ea56362_0
2025-10-09T21:43:20.070724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 110 sessionId: src_id|111bf144-247aafa2-6953623-8ea56362_0 is DEAD
2025-10-09T21:43:20.070857Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340865403535346:3142]
2025-10-09T21:43:20.074290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|111bf144-247aafa2-6953623-8ea56362_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:20.074306Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|111bf144-247aafa2-6953623-8ea56362_0] Write session will restart in 0.005300s
2025-10-09T21:43:20.074346Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|111bf144-247aafa2-6953623-8ea56362_0] Write session: Do CDS request
2025-10-09T21:43:20.074354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|111bf144-247aafa2-6953623-8ea56362_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:20.082520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|111bf144-247aafa2-6953623-8ea56362_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:20.082796Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:20.082812Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 111
2025-10-09T21:43:20.082992Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 111 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:20.083046Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 111 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:20.083057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=111 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:20.083061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:20.084178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 111 sessionId:  describe result for acl check
2025-10-09T21:43:20.084216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:20.084222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:20.084224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:20.084230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340869698502679:3197] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:20.084803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340869698502679:3197] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:20.087515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340869698502679:3197] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:20.092670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340869698502679:3197] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:20.092684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340869698502679:3197] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:20.092688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340869698502679:3197] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:20.092697Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 111 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:20.091499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:20.092391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:20.092576Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:20.092579Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:20.092591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:20.092644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:20.092679Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340869336693837:2993]
2025-10-09T21:43:20.093296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:20.093742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:20.093799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:20.094142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:20.094197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:20.094198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:20.094200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:20.094201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:20.094202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:20.094203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:20.094204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.094204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.094205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.094210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:20.094220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:20.094224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:20.094228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:20.094232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:20.094236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:20.094240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.094244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.094247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.094254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:20.094262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:20.094268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:20.094273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:20.094278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:20.094283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:20.094288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:20.094293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:20.094297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:20.094302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:20.094307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:20.094312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:20.094317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.094322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.094326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.094331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.094335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.094341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.094366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:20.104545Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 110
2025-10-09T21:43:20.104596Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 110
2025-10-09T21:43:20.105255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 111 partition: 0 MaxSeqNo: 25 sessionId: src_id|429c7fe6-bfd1ab52-9e2bfcb5-73654029_0
2025-10-09T21:43:20.105713Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|111bf144-247aafa2-6953623-8ea56362_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|429c7fe6-bfd1ab52-9e2bfcb5-73654029_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:20.102681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:20.104098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:20.104104Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:20.104107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:20.104136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:20.104144Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 110 [2:7559340869336693837:2993]
2025-10-09T21:43:20.104159Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:20.104178Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:20.104182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.104185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.104187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.104208Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:20.104215Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:20.104223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:20.104290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.104292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.104294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.104297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:20.104393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:20.104396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.104409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.104420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.104435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:20.104459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340869698502817:3197], now have 1 active actors on pipe
2025-10-09T21:43:20.104462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340869698502656:2432], now have 1 active actors on pipe
2025-10-09T21:43:20.104470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340869336693832:2993]
2025-10-09T21:43:20.104472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340869336693832:2993]
2025-10-09T21:43:20.104475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340869336693832:2993]
2025-10-09T21:43:20.104490Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:20.104506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:20.104517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:20.104520Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:20.104521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:20.104522Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:20.104524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:20.104566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340869336693832:2993]
2025-10-09T21:43:20.104575Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:20.104581Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:20.104836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:20.104843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:20.104872Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|429c7fe6-bfd1ab52-9e2bfcb5-73654029_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:20.104881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.104886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.104888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:20.104900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.104913Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:20.104930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:20.105155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:20.105159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:20.105177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:20.108368Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|429c7fe6-bfd1ab52-9e2bfcb5-73654029_0] Write session: try to update token
2025-10-09T21:43:20.108410Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|429c7fe6-bfd1ab52-9e2bfcb5-73654029_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:20.123529Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:20.123552Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:20.123555Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:20.123557Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:20.123567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340869336693832:2993]
2025-10-09T21:43:20.123572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340869336693832:2993]
2025-10-09T21:43:20.123576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340869336693832:2993]
2025-10-09T21:43:20.123623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:20.123622Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:20.123628Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:20.123633Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:20.123635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.123638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:20.123788Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:20.134476Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 111 sessionId: src_id|429c7fe6-bfd1ab52-9e2bfcb5-73654029_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:20.137197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:20.171053Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340869698502828:3142]
2025-10-09T21:43:20.171365Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 110, pipe: [1:7559340869698502828:3142]
2025-10-09T21:43:20.288785Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:20.289229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:20.289240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:20.289285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340869698502828:3142], now have 1 active actors on pipe
2025-10-09T21:43:20.289667Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:20.289716Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340869698502828:3142]
2025-10-09T21:43:20.290130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|429c7fe6-bfd1ab52-9e2bfcb5-73654029_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:20.290160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|429c7fe6-bfd1ab52-9e2bfcb5-73654029_0] Write session will restart in 0.005237s
2025-10-09T21:43:20.289737Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:20.289806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 111 reason: status is not ok: tablet will be restarted right now sessionId: src_id|429c7fe6-bfd1ab52-9e2bfcb5-73654029_0
2025-10-09T21:43:20.290189Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|429c7fe6-bfd1ab52-9e2bfcb5-73654029_0] Write session: Do CDS request
2025-10-09T21:43:20.290202Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|429c7fe6-bfd1ab52-9e2bfcb5-73654029_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:20.289975Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 111 sessionId: src_id|429c7fe6-bfd1ab52-9e2bfcb5-73654029_0 is DEAD
2025-10-09T21:43:20.289551Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:20.296569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|429c7fe6-bfd1ab52-9e2bfcb5-73654029_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:20.296709Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:20.296724Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 112
2025-10-09T21:43:20.296943Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 112 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:20.297000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 112 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:20.297022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=112 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:20.297025Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:20.297520Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 112 sessionId:  describe result for acl check
2025-10-09T21:43:20.297556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:20.297565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:20.297567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:20.297573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340869698502846:3201] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:20.298138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340869698502846:3201] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:20.300984Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340869698502846:3201] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:20.303332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:20.304461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:20.304583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340869698502846:3201] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:20.304597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340869698502846:3201] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:20.304602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340869698502846:3201] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:20.304610Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 112 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:20.304664Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:20.304674Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:20.304686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:20.304744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:20.304789Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340869336694002:2995]
2025-10-09T21:43:20.305343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:20.305652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:20.305698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:20.306075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:20.306129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:20.306134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:20.306136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:20.306137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:20.306138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:20.306148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:20.306149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.306150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.306152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.306157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:20.306167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:20.306170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:20.306174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:20.306177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:20.306181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:20.306185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.306189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.306192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.306199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:20.306207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:20.306213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:20.306218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:20.306223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:20.306228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:20.306232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:20.306237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:20.306242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:20.306247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:20.306251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:20.306255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:20.306260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.306265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.306274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.306279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.306284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.306289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.306317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:20.313427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:20.314842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:20.314861Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:20.314863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:20.314868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:20.314876Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 111 [2:7559340869336694002:2995]
2025-10-09T21:43:20.314894Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:20.314915Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:20.314923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.314927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.314929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.314949Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:20.314956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:20.314963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:20.315026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.315036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.315038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.315041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:20.315114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:20.315123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.315135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.315152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.315171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:20.315195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340869698502831:2432], now have 1 active actors on pipe
2025-10-09T21:43:20.315203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340869698502993:3201], now have 1 active actors on pipe
2025-10-09T21:43:20.315211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340869336693997:2995]
2025-10-09T21:43:20.315213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340869336693997:2995]
2025-10-09T21:43:20.315215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340869336693997:2995]
2025-10-09T21:43:20.315228Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:20.315243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:20.315262Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:20.315270Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:20.315285Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:20.315296Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:20.315298Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:20.315311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340869336693997:2995]
2025-10-09T21:43:20.315321Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:20.315330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:20.315259Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 111
2025-10-09T21:43:20.315279Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 111
2025-10-09T21:43:20.315373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:20.315387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:20.315425Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a968c2d-7dd4da05-18d95b1b-7ddcef2a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:20.315439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.315443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.315446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:20.315458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.315470Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:20.315486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:20.315673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:20.315683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:20.315701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:20.315882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 112 partition: 0 MaxSeqNo: 25 sessionId: src_id|a968c2d-7dd4da05-18d95b1b-7ddcef2a_0
2025-10-09T21:43:20.316234Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|429c7fe6-bfd1ab52-9e2bfcb5-73654029_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a968c2d-7dd4da05-18d95b1b-7ddcef2a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:20.319484Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a968c2d-7dd4da05-18d95b1b-7ddcef2a_0] Write session: try to update token
2025-10-09T21:43:20.319507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a968c2d-7dd4da05-18d95b1b-7ddcef2a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:20.330445Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:20.330462Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:20.330465Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:20.330467Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:20.330477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340869336693997:2995]
2025-10-09T21:43:20.330482Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340869336693997:2995]
2025-10-09T21:43:20.330486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340869336693997:2995]
2025-10-09T21:43:20.330531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:20.330539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.330542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:20.330579Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:20.330586Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:20.330589Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:20.330785Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:20.342435Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 112 sessionId: src_id|a968c2d-7dd4da05-18d95b1b-7ddcef2a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:20.344921Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:20.390152Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340869698503006:3142]
2025-10-09T21:43:20.390431Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 111, pipe: [1:7559340869698503006:3142]
2025-10-09T21:43:20.486696Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:20.487058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:20.487068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:20.487091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340869698503006:3142], now have 1 active actors on pipe
2025-10-09T21:43:20.487907Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a968c2d-7dd4da05-18d95b1b-7ddcef2a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:20.487931Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a968c2d-7dd4da05-18d95b1b-7ddcef2a_0] Write session will restart in 0.009081s
2025-10-09T21:43:20.487954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a968c2d-7dd4da05-18d95b1b-7ddcef2a_0] Write session: Do CDS request
2025-10-09T21:43:20.487960Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a968c2d-7dd4da05-18d95b1b-7ddcef2a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:20.487465Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:20.487512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:20.487516Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340869698503006:3142]
2025-10-09T21:43:20.487584Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 112 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a968c2d-7dd4da05-18d95b1b-7ddcef2a_0
2025-10-09T21:43:20.487768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 112 sessionId: src_id|a968c2d-7dd4da05-18d95b1b-7ddcef2a_0 is DEAD
2025-10-09T21:43:20.487298Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:20.500084Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a968c2d-7dd4da05-18d95b1b-7ddcef2a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:20.500285Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:20.500300Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 113
2025-10-09T21:43:20.500488Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 113 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:20.500534Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 113 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:20.500545Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=113 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:20.500549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:20.501022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 113 sessionId:  describe result for acl check
2025-10-09T21:43:20.501055Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:20.501061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:20.501062Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:20.501066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340869698503088:3206] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:20.501617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340869698503088:3206] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:20.504283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340869698503088:3206] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:20.504051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:20.504823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:20.505022Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:20.505026Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:20.505036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:20.505084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:20.505120Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340869336694169:2997]
2025-10-09T21:43:20.505707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:20.506010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:20.506083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:20.506397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:20.506461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:20.506462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:20.506463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:20.506465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:20.506466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:20.506467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:20.506468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.506469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.506470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.506476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:20.506486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:20.506489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:20.506493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:20.506497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:20.506501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:20.506505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.506508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.506512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.506519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:20.506527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:20.506533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:20.506538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:20.506543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:20.506548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:20.506554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:20.506558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:20.506563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:20.506568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:20.506572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:20.506577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:20.506581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.507145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340869698503088:3206] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:20.507159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340869698503088:3206] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:20.507164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340869698503088:3206] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:20.507171Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 113 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:20.506586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.506590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.506594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.506598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.506603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.506630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:20.517581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:20.519004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:20.519019Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:20.519022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:20.519026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:20.519037Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 112 [2:7559340869336694169:2997]
2025-10-09T21:43:20.519054Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:20.519073Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:20.519077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.519081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.519083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.519103Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:20.519111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:20.519121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:20.519224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.519236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.519238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.519242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:20.519314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:20.519326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.519347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.519359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.519381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:20.519407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340869336694164:2997]
2025-10-09T21:43:20.519415Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340869336694164:2997]
2025-10-09T21:43:20.519417Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340869336694164:2997]
2025-10-09T21:43:20.519428Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:20.519446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:20.519448Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:20.519454Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:20.519456Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:20.519462Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:20.519465Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:20.519479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340869698503175:3206], now have 1 active actors on pipe
2025-10-09T21:43:20.519492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340869698503009:2432], now have 1 active actors on pipe
2025-10-09T21:43:20.519537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340869336694164:2997]
2025-10-09T21:43:20.519548Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:20.519606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:20.519841Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 112
2025-10-09T21:43:20.519900Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 112
2025-10-09T21:43:20.519934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:20.519946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:20.519973Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|985bc83c-e8fb5e01-b4ab0046-8fc722f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:20.520007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.520018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.520021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:20.520033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.520045Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:20.520064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:20.520456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:20.520466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:20.520485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:20.520654Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 113 partition: 0 MaxSeqNo: 25 sessionId: src_id|985bc83c-e8fb5e01-b4ab0046-8fc722f_0
2025-10-09T21:43:20.521249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a968c2d-7dd4da05-18d95b1b-7ddcef2a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|985bc83c-e8fb5e01-b4ab0046-8fc722f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:20.524568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|985bc83c-e8fb5e01-b4ab0046-8fc722f_0] Write session: try to update token
2025-10-09T21:43:20.524589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|985bc83c-e8fb5e01-b4ab0046-8fc722f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:20.545046Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:20.545064Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:20.545067Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:20.545070Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:20.545080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340869336694164:2997]
2025-10-09T21:43:20.545085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340869336694164:2997]
2025-10-09T21:43:20.545089Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340869336694164:2997]
2025-10-09T21:43:20.545135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:20.545139Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:20.545142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.545145Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:20.545150Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:20.545153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:20.545313Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:20.558929Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 113 sessionId: src_id|985bc83c-e8fb5e01-b4ab0046-8fc722f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:20.561417Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:20.587794Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340869698503184:3142]
2025-10-09T21:43:20.588399Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 112, pipe: [1:7559340869698503184:3142]
2025-10-09T21:43:20.716889Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:20.717253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:20.717263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:20.717287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340869698503184:3142], now have 1 active actors on pipe
2025-10-09T21:43:20.717553Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:20.718759Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|985bc83c-e8fb5e01-b4ab0046-8fc722f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:20.718775Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|985bc83c-e8fb5e01-b4ab0046-8fc722f_0] Write session will restart in 0.006843s
2025-10-09T21:43:20.718803Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|985bc83c-e8fb5e01-b4ab0046-8fc722f_0] Write session: Do CDS request
2025-10-09T21:43:20.718810Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|985bc83c-e8fb5e01-b4ab0046-8fc722f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:20.718271Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:20.718359Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 113 reason: status is not ok: tablet will be restarted right now sessionId: src_id|985bc83c-e8fb5e01-b4ab0046-8fc722f_0
2025-10-09T21:43:20.718472Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340869698503184:3142]
2025-10-09T21:43:20.718498Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:20.718553Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 113 sessionId: src_id|985bc83c-e8fb5e01-b4ab0046-8fc722f_0 is DEAD
2025-10-09T21:43:20.726696Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|985bc83c-e8fb5e01-b4ab0046-8fc722f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:20.727012Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:20.727030Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 114
2025-10-09T21:43:20.727208Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 114 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:20.727246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 114 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:20.727259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=114 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:20.727263Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:20.727767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 114 sessionId:  describe result for acl check
2025-10-09T21:43:20.727804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:20.727811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:20.727814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:20.727820Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340869698503193:3210] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:20.728535Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340869698503193:3210] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:20.730630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340869698503193:3210] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:20.739008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340869698503193:3210] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:20.739025Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340869698503193:3210] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:20.739029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340869698503193:3210] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:20.739042Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 114 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:20.775681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:20.785757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:20.786023Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:20.786050Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:20.786062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:20.786120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:20.786160Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340869336694338:3000]
2025-10-09T21:43:20.786779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:20.787243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:20.787295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:20.787619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:20.787685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:20.787692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:20.787694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:20.787695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:20.787696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:20.787698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:20.787699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.787701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.787702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.787708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:20.787720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:20.787725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:20.787729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:20.787734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:20.787739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:20.787743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.787748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.787752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.787759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:20.787770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:20.787781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:20.787788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:20.787795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:20.787807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:20.787817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:20.787824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:20.787833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:20.787839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:20.787846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:20.787853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:20.787859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.787864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.787876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.787881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.787886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.787892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.787938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:20.801551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:20.803709Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 113
2025-10-09T21:43:20.803891Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 113
2025-10-09T21:43:20.804534Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 114 partition: 0 MaxSeqNo: 25 sessionId: src_id|7eb7f3ed-6a093ce-ef322718-7dbc2929_0
2025-10-09T21:43:20.802954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:20.802974Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:20.802977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:20.802981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:20.802987Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 113 [2:7559340869336694338:3000]
2025-10-09T21:43:20.803005Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:20.803024Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:20.803027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.803030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.803032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.803052Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:20.803057Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:20.803065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:20.803132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.803134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.803135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.803138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:20.803234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:20.803237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.803252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:20.803262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:20.803275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:20.803295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340869336694332:3000]
2025-10-09T21:43:20.803297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340869336694332:3000]
2025-10-09T21:43:20.803298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340869336694332:3000]
2025-10-09T21:43:20.803314Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:20.803329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:20.803341Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:20.803344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:20.803391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340869336694332:3000]
2025-10-09T21:43:20.803394Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:20.803588Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:20.803590Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:20.803591Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:20.803597Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:20.803611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340869698503186:2432], now have 1 active actors on pipe
2025-10-09T21:43:20.803890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340869698503250:3210], now have 1 active actors on pipe
2025-10-09T21:43:20.804145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:20.804153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:20.804181Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7eb7f3ed-6a093ce-ef322718-7dbc2929_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:20.804190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.804194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.804197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:20.804211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.804223Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:20.804240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:20.804397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:20.804400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:20.804415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:20.806247Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|985bc83c-e8fb5e01-b4ab0046-8fc722f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7eb7f3ed-6a093ce-ef322718-7dbc2929_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:20.809395Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7eb7f3ed-6a093ce-ef322718-7dbc2929_0] Write session: try to update token
2025-10-09T21:43:20.809421Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7eb7f3ed-6a093ce-ef322718-7dbc2929_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:20.818790Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340869698503367:3142]
2025-10-09T21:43:20.822467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340869698503367:3142], now have 1 active actors on pipe
2025-10-09T21:43:20.822530Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 113, pipe: [1:7559340869698503367:3142]
2025-10-09T21:43:20.843404Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 114 sessionId: src_id|7eb7f3ed-6a093ce-ef322718-7dbc2929_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:20.845951Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:20.847781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:20.847804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:20.847847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:20.850440Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:20.850456Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:20.850458Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:20.850460Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:20.850469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340869336694332:3000]
2025-10-09T21:43:20.850472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340869336694332:3000]
2025-10-09T21:43:20.850475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340869336694332:3000]
2025-10-09T21:43:20.848167Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:20.850519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:20.850532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:20.850534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:20.850674Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:20.851690Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:20.851696Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:20.851700Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:20.871614Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:43:20.871639Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 1
2025-10-09T21:43:21.011478Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:21.020271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:21.020302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:21.020473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.020541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.020598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.020658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.021129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.021136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.021213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.021270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.021327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.021413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.021535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.021539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.021611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.021670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.021750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.021845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.021956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.021959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.025833Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:21.025852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340869698503367:3142]
2025-10-09T21:43:21.025857Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 1
2025-10-09T21:43:21.025889Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:21.025893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:21.025925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 114 reason: error at write: Disconnected sessionId: src_id|7eb7f3ed-6a093ce-ef322718-7dbc2929_0
2025-10-09T21:43:21.022231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.022472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.022529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.022589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.022630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.022633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.022706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.022796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.023036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.023128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.023384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.023387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.023454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.023544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.023646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.023726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.023771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.023773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.023865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.023952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.024048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.024156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.024207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.024212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.024314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.024412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.024509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.024578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.024640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.024643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.024710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.024783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.024854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.024915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.024970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.024973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.025125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.025176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.025264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.025363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.025411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.025413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.025866Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:21.026311Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 114 sessionId: src_id|7eb7f3ed-6a093ce-ef322718-7dbc2929_0 is DEAD
2025-10-09T21:43:21.030322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7eb7f3ed-6a093ce-ef322718-7dbc2929_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:21.030337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7eb7f3ed-6a093ce-ef322718-7dbc2929_0] Write session will restart in 0.007747s
2025-10-09T21:43:21.030362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7eb7f3ed-6a093ce-ef322718-7dbc2929_0] Write session: Do CDS request
2025-10-09T21:43:21.030369Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7eb7f3ed-6a093ce-ef322718-7dbc2929_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:21.039676Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7eb7f3ed-6a093ce-ef322718-7dbc2929_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:21.040070Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:21.040086Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 115
2025-10-09T21:43:21.040268Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 115 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:21.040310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 115 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:21.040316Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=115 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:21.040320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:21.040733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 115 sessionId:  describe result for acl check
2025-10-09T21:43:21.040770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:21.040772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:21.040774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:21.040779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340873993470711:3222] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:21.041255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340873993470711:3222] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:21.050727Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340873993470711:3222] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:21.054140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340873993470711:3222] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:21.054163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340873993470711:3222] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:21.054166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340873993470711:3222] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:21.054176Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 115 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:21.054191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:21.055793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:21.056050Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:21.056054Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:21.056066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:21.056114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:21.056149Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340873631661844:3016]
2025-10-09T21:43:21.056722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:21.057207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:21.057290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:21.057681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:21.057749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:21.057751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:21.057752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:21.057753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:21.057754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:21.057756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:21.057757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.057758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.057760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.057768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:21.057778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:21.057783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:21.057787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:21.057791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:21.057795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:21.057799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.057803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.057807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.057818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:21.057829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:21.057835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:21.057842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:21.057847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:21.057853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:21.057857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:21.057862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:21.057868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:21.057873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:21.057878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:21.057882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:21.057887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.057893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.057897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.057902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.057907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.057912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.057941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:21.066528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:21.068145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:21.068151Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:21.068155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:21.068159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:21.068167Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 114 [2:7559340873631661844:3016]
2025-10-09T21:43:21.068184Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:21.068203Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:21.068207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.068211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.068214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.068232Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:21.068239Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:21.068249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:21.068322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.068325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.068327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.068330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:21.068411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:21.068414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.068428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.068441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.068459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:21.068476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340873631661837:3016]
2025-10-09T21:43:21.068478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340873631661837:3016]
2025-10-09T21:43:21.068481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340873631661837:3016]
2025-10-09T21:43:21.068496Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:21.068511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:21.068526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:21.068529Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:21.068573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340873631661837:3016]
2025-10-09T21:43:21.068577Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:21.068839Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:21.068848Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:21.068850Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:21.068857Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:21.069967Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 114
2025-10-09T21:43:21.069974Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 114
2025-10-09T21:43:21.070329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340873993470857:3222], now have 1 active actors on pipe
2025-10-09T21:43:21.070352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340873993470696:2432], now have 1 active actors on pipe
2025-10-09T21:43:21.070390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:21.070395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:21.070556Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:43:21.070652Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9d8208e6-55ceeaa8-cfec49f3-2d237437_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:21.070673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.070679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.070683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:21.070697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.070733Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:21.070765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:21.070976Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 1 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:43:21.071164Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:43:21.071383Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72057594046644480, NodeId 1, Generation 2
2025-10-09T21:43:21.071425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:21.071441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:21.071467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:21.071635Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 115 partition: 0 MaxSeqNo: 25 sessionId: src_id|9d8208e6-55ceeaa8-cfec49f3-2d237437_0
2025-10-09T21:43:21.072052Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7eb7f3ed-6a093ce-ef322718-7dbc2929_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9d8208e6-55ceeaa8-cfec49f3-2d237437_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:21.075167Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d8208e6-55ceeaa8-cfec49f3-2d237437_0] Write session: try to update token
2025-10-09T21:43:21.075191Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d8208e6-55ceeaa8-cfec49f3-2d237437_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:21.092357Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:21.092381Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:21.092384Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:21.092387Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:21.092399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340873631661837:3016]
2025-10-09T21:43:21.092404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340873631661837:3016]
2025-10-09T21:43:21.092409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340873631661837:3016]
2025-10-09T21:43:21.092455Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:21.092460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:21.092460Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:21.092464Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:21.092468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.092471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:21.092638Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:21.096361Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 115 sessionId: src_id|9d8208e6-55ceeaa8-cfec49f3-2d237437_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:21.099056Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:21.124881Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340873993470875:3142]
2025-10-09T21:43:21.125221Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 114, pipe: [1:7559340873993470875:3142]
2025-10-09T21:43:21.274336Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:21.274773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:21.274790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:21.274836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340873993470875:3142], now have 1 active actors on pipe
2025-10-09T21:43:21.277792Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:21.278284Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:21.278806Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d8208e6-55ceeaa8-cfec49f3-2d237437_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:21.278824Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d8208e6-55ceeaa8-cfec49f3-2d237437_0] Write session will restart in 0.008373s
2025-10-09T21:43:21.278850Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d8208e6-55ceeaa8-cfec49f3-2d237437_0] Write session: Do CDS request
2025-10-09T21:43:21.278867Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d8208e6-55ceeaa8-cfec49f3-2d237437_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:21.278385Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 115 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9d8208e6-55ceeaa8-cfec49f3-2d237437_0
2025-10-09T21:43:21.278593Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 115 sessionId: src_id|9d8208e6-55ceeaa8-cfec49f3-2d237437_0 is DEAD
2025-10-09T21:43:21.278755Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:21.278790Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340873993470875:3142]
2025-10-09T21:43:21.288589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d8208e6-55ceeaa8-cfec49f3-2d237437_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:21.289137Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:21.289151Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 116
2025-10-09T21:43:21.289385Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 116 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:21.289419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 116 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:21.289424Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=116 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:21.289428Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:21.290836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 116 sessionId:  describe result for acl check
2025-10-09T21:43:21.290873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:21.290875Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:21.290879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:21.290886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340873993470903:3226] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:21.291452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340873993470903:3226] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:21.299431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340873993470903:3226] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:21.312023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:21.313726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:21.313974Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:21.313986Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:21.313999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:21.314103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:21.314148Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340873631662013:3019]
2025-10-09T21:43:21.314141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340873993470903:3226] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:21.314160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340873993470903:3226] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:21.314164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340873993470903:3226] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:21.314175Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 116 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:21.314709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:21.315457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:21.315514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:21.315826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:21.315915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:21.315922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:21.315923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:21.315925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:21.315926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:21.315927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:21.315928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.315929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.315930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.315937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:21.315948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:21.315952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:21.315956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:21.315959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:21.315964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:21.315967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.315970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.315974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.315983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:21.315992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:21.315999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:21.316004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:21.316009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:21.316014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:21.316018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:21.316022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:21.316027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:21.316032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:21.316037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:21.316042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:21.316048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.316053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.316057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.316061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.316067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.316072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.316101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:21.326046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:21.327463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:21.327483Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:21.327486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:21.327491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:21.327498Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 115 [2:7559340873631662013:3019]
2025-10-09T21:43:21.327514Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:21.327537Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:21.327540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.327544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.327546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.327564Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:21.327571Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:21.327578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:21.327644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.327655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.327657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.327660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:21.327731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:21.327738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.327754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.327769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.327784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:21.327808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340873631662007:3019]
2025-10-09T21:43:21.327814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340873631662007:3019]
2025-10-09T21:43:21.327816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340873631662007:3019]
2025-10-09T21:43:21.327828Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:21.327848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340873993471045:3226], now have 1 active actors on pipe
2025-10-09T21:43:21.327855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340873993470880:2432], now have 1 active actors on pipe
2025-10-09T21:43:21.327858Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:21.327861Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:21.327864Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:21.327865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:21.327879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:21.327881Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:21.327927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340873631662007:3019]
2025-10-09T21:43:21.327940Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:21.327920Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 115
2025-10-09T21:43:21.327936Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 115
2025-10-09T21:43:21.327972Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:21.328065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:21.328082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:21.328119Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8a066790-f48a14d9-c8a9df94-6edeef28_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:21.328134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.328145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.328148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:21.328161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.328189Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:21.328214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:21.328829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:21.328843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:21.328878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:21.329056Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 116 partition: 0 MaxSeqNo: 25 sessionId: src_id|8a066790-f48a14d9-c8a9df94-6edeef28_0
2025-10-09T21:43:21.329709Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d8208e6-55ceeaa8-cfec49f3-2d237437_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8a066790-f48a14d9-c8a9df94-6edeef28_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:21.333060Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a066790-f48a14d9-c8a9df94-6edeef28_0] Write session: try to update token
2025-10-09T21:43:21.333085Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a066790-f48a14d9-c8a9df94-6edeef28_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:21.352724Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:21.352741Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:21.352756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340873631662007:3019]
2025-10-09T21:43:21.352808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:21.352827Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:21.352994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:21.353153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:21.353320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:21.353489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:21.353987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:21.354053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:21.355449Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 116 sessionId: src_id|8a066790-f48a14d9-c8a9df94-6edeef28_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:21.358062Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:21.358289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:21.358310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:21.358364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:21.358684Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:21.368946Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:21.368964Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:21.368967Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:21.368969Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:21.368980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340873631662007:3019]
2025-10-09T21:43:21.368984Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340873631662007:3019]
2025-10-09T21:43:21.368991Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340873631662007:3019]
2025-10-09T21:43:21.369036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:21.369037Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:21.369041Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:21.369044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.369044Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:21.369047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:21.369191Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:21.379334Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340873993471059:3142]
2025-10-09T21:43:21.379749Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 115, pipe: [1:7559340873993471059:3142]
2025-10-09T21:43:21.533492Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:21.533795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340873993471059:3142], now have 1 active actors on pipe
2025-10-09T21:43:21.541520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:21.541550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:21.541727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.541786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.541860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.541930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.545672Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340873993471059:3142]
2025-10-09T21:43:21.545690Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:21.545694Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:21.545711Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:21.545722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 116 reason: error at write: Disconnected sessionId: src_id|8a066790-f48a14d9-c8a9df94-6edeef28_0
2025-10-09T21:43:21.545872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 116 sessionId: src_id|8a066790-f48a14d9-c8a9df94-6edeef28_0 is DEAD
2025-10-09T21:43:21.546070Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a066790-f48a14d9-c8a9df94-6edeef28_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:21.546085Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a066790-f48a14d9-c8a9df94-6edeef28_0] Write session will restart in 0.008744s
2025-10-09T21:43:21.546114Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a066790-f48a14d9-c8a9df94-6edeef28_0] Write session: Do CDS request
2025-10-09T21:43:21.546120Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a066790-f48a14d9-c8a9df94-6edeef28_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:21.542135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.542152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.542218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.542265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.542339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.542396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.542450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.542453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.542561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.542611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.542664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.542720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.542766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.542769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.542864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.542965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.543060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.543120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.543161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.543164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.543243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.543320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.543388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.543470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.543534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.543537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.543622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.543684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.543752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.543817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.543860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.543864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.543929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.543988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.544090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.544183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.544240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.544243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.544343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.544399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.544481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.544538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.544595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.544598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.544662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.544716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.544779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.544838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.544885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.544889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.544954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.545010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.545068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.545130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.545187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.545190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.545757Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:21.556530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a066790-f48a14d9-c8a9df94-6edeef28_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:21.556672Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:21.556686Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 117
2025-10-09T21:43:21.556835Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 117 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:21.556867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 117 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:21.556873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=117 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:21.556886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:21.557485Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 117 sessionId:  describe result for acl check
2025-10-09T21:43:21.557538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:21.557543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:21.557545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:21.557551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340873993471074:3231] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:21.558142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340873993471074:3231] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:21.561296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340873993471074:3231] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:21.566361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340873993471074:3231] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:21.566380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340873993471074:3231] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:21.566386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340873993471074:3231] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:21.566395Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 117 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:21.570282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:21.571328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:21.571527Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:21.571538Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:21.571551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:21.571609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:21.571665Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340873631662185:3021]
2025-10-09T21:43:21.572224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:21.572687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:21.572747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:21.573072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:21.573138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:21.573143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:21.573144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:21.573145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:21.573146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:21.573147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:21.573148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.573149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.573151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.573156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:21.573166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:21.573170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:21.573175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:21.573179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:21.573182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:21.573185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.573189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.573193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.573201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:21.573209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:21.573216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:21.573222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:21.573225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:21.573230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:21.573234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:21.573240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:21.573245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:21.573251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:21.573255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:21.573264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:21.573269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.573275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.573281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.573296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.573304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.573310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.573339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:21.583422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:21.584968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:21.584988Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:21.584990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:21.584995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:21.585003Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 116 [2:7559340873631662185:3021]
2025-10-09T21:43:21.585020Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:21.585041Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:21.585045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.585048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.585051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.585068Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:21.585085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:21.585093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:21.585164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.585175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.585177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.585180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:21.585256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:21.585263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.585278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.585297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.585317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:21.585342Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340873631662179:3021]
2025-10-09T21:43:21.585345Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340873631662179:3021]
2025-10-09T21:43:21.585347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340873631662179:3021]
2025-10-09T21:43:21.585367Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:21.585381Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:21.585396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:21.585399Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:21.585443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340873631662179:3021]
2025-10-09T21:43:21.585446Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:21.585472Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:21.585474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340873993471203:3231], now have 1 active actors on pipe
2025-10-09T21:43:21.585475Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:21.585477Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:21.585484Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:21.585758Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 116
2025-10-09T21:43:21.585793Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 116
2025-10-09T21:43:21.585791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340873993471062:2432], now have 1 active actors on pipe
2025-10-09T21:43:21.586025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:21.586055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:21.586092Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|388f0d10-4edb37ac-eef42507-14d37567_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:21.586101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.586106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.586109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:21.586122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.586139Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:21.586159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:21.590991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 117 partition: 0 MaxSeqNo: 25 sessionId: src_id|388f0d10-4edb37ac-eef42507-14d37567_0
2025-10-09T21:43:21.591476Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a066790-f48a14d9-c8a9df94-6edeef28_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|388f0d10-4edb37ac-eef42507-14d37567_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:21.590821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:21.590840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:21.590874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:21.594675Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|388f0d10-4edb37ac-eef42507-14d37567_0] Write session: try to update token
2025-10-09T21:43:21.594698Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|388f0d10-4edb37ac-eef42507-14d37567_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:21.608562Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:21.608576Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:21.608584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340873631662179:3021]
2025-10-09T21:43:21.608625Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:21.608807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:21.608932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:21.609049Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:21.609168Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:21.609369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:21.609406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:21.609696Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:21.622958Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 117 sessionId: src_id|388f0d10-4edb37ac-eef42507-14d37567_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:21.625540Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:21.625777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:21.625799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:21.625842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:21.625963Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:21.630379Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:21.630391Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:21.630394Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:21.630396Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:21.630406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340873631662179:3021]
2025-10-09T21:43:21.630411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340873631662179:3021]
2025-10-09T21:43:21.630425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340873631662179:3021]
2025-10-09T21:43:21.630469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:21.630476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.630479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:21.630637Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:21.637862Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:21.637893Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:21.637897Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:21.645610Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340873993471244:3142]
2025-10-09T21:43:21.646397Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 116, pipe: [1:7559340873993471244:3142]
2025-10-09T21:43:21.795055Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:21.803174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:21.803204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:21.803378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.803445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.803520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.803612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.803853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.803857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.803942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.803994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.804051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.804109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.804147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.804150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.804210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.804259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.804316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.804374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.804416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.804419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.804483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.804541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.804601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.804661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.804700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.804706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.804769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.804821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.804877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.804936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.804981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.804983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.805048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.805102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.805164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.805220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.805265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.805270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.805337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.805390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.805451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.805512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.805578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.805581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.805644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.805692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.805747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.805806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.805843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.805846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.805906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.805962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.806019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.807021Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:21.807037Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:21.807083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 117 reason: error at write: Disconnected sessionId: src_id|388f0d10-4edb37ac-eef42507-14d37567_0
2025-10-09T21:43:21.807167Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:21.807216Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340873993471244:3142]
2025-10-09T21:43:21.807303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 117 sessionId: src_id|388f0d10-4edb37ac-eef42507-14d37567_0 is DEAD
2025-10-09T21:43:21.807639Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|388f0d10-4edb37ac-eef42507-14d37567_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:21.807649Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|388f0d10-4edb37ac-eef42507-14d37567_0] Write session will restart in 0.005295s
2025-10-09T21:43:21.807672Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|388f0d10-4edb37ac-eef42507-14d37567_0] Write session: Do CDS request
2025-10-09T21:43:21.807679Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|388f0d10-4edb37ac-eef42507-14d37567_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:21.806322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.806375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.806378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.806450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:21.806504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:21.806561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:21.806618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:21.806656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:21.806660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:21.806725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340873993471244:3142], now have 1 active actors on pipe
2025-10-09T21:43:21.807540Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:21.814696Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|388f0d10-4edb37ac-eef42507-14d37567_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:21.814921Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:21.814935Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 118
2025-10-09T21:43:21.815118Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 118 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:21.815167Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 118 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:21.815177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=118 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:21.815181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:21.815802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 118 sessionId:  describe result for acl check
2025-10-09T21:43:21.815854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:21.815856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:21.815859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:21.815864Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340873993471280:3238] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:21.816462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340873993471280:3238] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:21.821445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340873993471280:3238] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:21.823031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:21.824102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:21.824300Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:21.824308Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:21.824320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:21.824368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:21.824408Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340873631662357:3024]
2025-10-09T21:43:21.825016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:21.825277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:21.825316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:21.825645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:21.825698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:21.825704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:21.825705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:21.825706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:21.825707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:21.825708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:21.825709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.825710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.825711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.825725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:21.825734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:21.825738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:21.825741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:21.825745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:21.825748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:21.825751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.825755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.825758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.825765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:21.825772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:21.825778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:21.825782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:21.825786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:21.825791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:21.825795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:21.825799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:21.825804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:21.825808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:21.825812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:21.825817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:21.825823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.825827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.825831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.825836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.825840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.825845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.825873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:21.832449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340873993471280:3238] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:21.832469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340873993471280:3238] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:21.832474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340873993471280:3238] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:21.832485Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 118 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:21.833859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:21.835531Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 117
2025-10-09T21:43:21.835516Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 117
2025-10-09T21:43:21.835130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:21.835145Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:21.835148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:21.835152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:21.835160Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 117 [2:7559340873631662357:3024]
2025-10-09T21:43:21.835175Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:21.835195Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:21.835198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.835202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.835205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.835224Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:21.835230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:21.835238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:21.835300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.835303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.835305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.835308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:21.835380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:21.835383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.835395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:21.835407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:21.835423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:21.835445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340873993471424:3238], now have 1 active actors on pipe
2025-10-09T21:43:21.835449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340873993471253:2432], now have 1 active actors on pipe
2025-10-09T21:43:21.835456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340873631662352:3024]
2025-10-09T21:43:21.835459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340873631662352:3024]
2025-10-09T21:43:21.835461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340873631662352:3024]
2025-10-09T21:43:21.835474Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:21.835490Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:21.835498Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:21.835502Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:21.835505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:21.835506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:21.835509Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:21.835547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340873631662352:3024]
2025-10-09T21:43:21.835550Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:21.835562Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:21.835579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:21.835585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:21.835621Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a82f6625-fd540d2b-8b313ece-c70a8c43_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:21.835628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.835632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.835636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:21.835648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.835658Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:21.835677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:21.835975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:21.835978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:21.835991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:21.838690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 118 partition: 0 MaxSeqNo: 25 sessionId: src_id|a82f6625-fd540d2b-8b313ece-c70a8c43_0
2025-10-09T21:43:21.839123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|388f0d10-4edb37ac-eef42507-14d37567_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a82f6625-fd540d2b-8b313ece-c70a8c43_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:21.841859Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a82f6625-fd540d2b-8b313ece-c70a8c43_0] Write session: try to update token
2025-10-09T21:43:21.841878Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a82f6625-fd540d2b-8b313ece-c70a8c43_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:21.865530Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:21.865550Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:21.865552Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:21.865554Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:21.865564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340873631662352:3024]
2025-10-09T21:43:21.865569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340873631662352:3024]
2025-10-09T21:43:21.865574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340873631662352:3024]
2025-10-09T21:43:21.865619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:21.865626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:21.865628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:21.865727Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:21.865733Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:21.865741Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:21.865778Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:21.872420Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 118 sessionId: src_id|a82f6625-fd540d2b-8b313ece-c70a8c43_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:21.875302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:21.907166Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340873993471436:3142]
2025-10-09T21:43:21.907708Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 117, pipe: [1:7559340873993471436:3142]
2025-10-09T21:43:22.058992Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:22.059379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:22.059390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:22.059436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340873993471436:3142], now have 1 active actors on pipe
2025-10-09T21:43:22.060561Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:22.060718Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:22.060767Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340873993471436:3142]
2025-10-09T21:43:22.060786Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:22.060791Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:22.060841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 118 reason: error at write: Disconnected sessionId: src_id|a82f6625-fd540d2b-8b313ece-c70a8c43_0
2025-10-09T21:43:22.061019Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 118 sessionId: src_id|a82f6625-fd540d2b-8b313ece-c70a8c43_0 is DEAD
2025-10-09T21:43:22.066133Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a82f6625-fd540d2b-8b313ece-c70a8c43_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:22.066150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a82f6625-fd540d2b-8b313ece-c70a8c43_0] Write session will restart in 0.006316s
2025-10-09T21:43:22.066176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a82f6625-fd540d2b-8b313ece-c70a8c43_0] Write session: Do CDS request
2025-10-09T21:43:22.066185Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a82f6625-fd540d2b-8b313ece-c70a8c43_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:22.074508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a82f6625-fd540d2b-8b313ece-c70a8c43_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:22.074895Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:22.074910Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 119
2025-10-09T21:43:22.075094Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 119 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:22.075127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 119 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:22.075133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=119 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:22.075136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:22.075583Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 119 sessionId:  describe result for acl check
2025-10-09T21:43:22.075622Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:22.075624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:22.075626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:22.075632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340878288438831:3249] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:22.076179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340878288438831:3249] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:22.081023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340878288438831:3249] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:22.084196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340878288438831:3249] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:22.084209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340878288438831:3249] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:22.084213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340878288438831:3249] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:22.084223Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 119 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:22.084507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:22.086217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:22.086426Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:22.086431Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:22.086443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:22.086510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:22.086549Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340877926629868:3039]
2025-10-09T21:43:22.087122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:22.087665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:22.087713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:22.088031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:22.088073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:22.088074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:22.088076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:22.088077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:22.088078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:22.088079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:22.088081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.088082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.088083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.088088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:22.088100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:22.088105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:22.088110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:22.088114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:22.088119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:22.088123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.088127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.088131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.088139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:22.088147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:22.088154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:22.088160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:22.088165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:22.088170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:22.088174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:22.088180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:22.088185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:22.088191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:22.088196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:22.088202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:22.088207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.088212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.088216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.088221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.088227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.088232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.088263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:22.095169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:22.096683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:22.096700Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:22.096703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:22.096707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:22.096715Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 118 [2:7559340877926629868:3039]
2025-10-09T21:43:22.096730Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:22.096749Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:22.096753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.096756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.096758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.096776Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:22.096782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:22.096790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:22.096854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.096856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.096858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.096861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:22.096938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:22.096941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.096956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.096970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.096989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:22.097009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340877926629861:3039]
2025-10-09T21:43:22.097012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340877926629861:3039]
2025-10-09T21:43:22.097014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340877926629861:3039]
2025-10-09T21:43:22.097023Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:22.097038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:22.097055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:22.097058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:22.097060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:22.097063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:22.097065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:22.097093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340877926629861:3039]
2025-10-09T21:43:22.097097Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:22.097098Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:22.097108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340878288438922:3249], now have 1 active actors on pipe
2025-10-09T21:43:22.097111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340878288438759:2432], now have 1 active actors on pipe
2025-10-09T21:43:22.097279Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 118
2025-10-09T21:43:22.097311Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 118
2025-10-09T21:43:22.097380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:22.097390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:22.097421Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|55065b79-e533e8c4-15677330-3711de91_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:22.097430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.097434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.097437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:22.097447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.097457Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:22.097475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:22.097694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:22.097705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:22.097719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:22.097899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 119 partition: 0 MaxSeqNo: 25 sessionId: src_id|55065b79-e533e8c4-15677330-3711de91_0
2025-10-09T21:43:22.098331Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a82f6625-fd540d2b-8b313ece-c70a8c43_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|55065b79-e533e8c4-15677330-3711de91_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:22.100847Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55065b79-e533e8c4-15677330-3711de91_0] Write session: try to update token
2025-10-09T21:43:22.100870Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55065b79-e533e8c4-15677330-3711de91_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:22.120033Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:22.120055Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:22.120057Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:22.120060Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:22.120069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340877926629861:3039]
2025-10-09T21:43:22.120078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340877926629861:3039]
2025-10-09T21:43:22.120082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340877926629861:3039]
2025-10-09T21:43:22.120133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:22.120141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.120144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:22.120297Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:22.122399Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:22.122435Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:22.122443Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:22.126384Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 119 sessionId: src_id|55065b79-e533e8c4-15677330-3711de91_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:22.129321Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:22.161199Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340878288438937:3142]
2025-10-09T21:43:22.166390Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 118, pipe: [1:7559340878288438937:3142]
2025-10-09T21:43:22.302225Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:22.302754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:22.302773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:22.302824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340878288438937:3142], now have 1 active actors on pipe
2025-10-09T21:43:22.303188Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:22.303488Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:22.303509Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:22.303577Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 119 reason: error at write: Disconnected sessionId: src_id|55065b79-e533e8c4-15677330-3711de91_0
2025-10-09T21:43:22.303770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 119 sessionId: src_id|55065b79-e533e8c4-15677330-3711de91_0 is DEAD
2025-10-09T21:43:22.303918Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340878288438937:3142]
2025-10-09T21:43:22.303934Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:22.304047Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55065b79-e533e8c4-15677330-3711de91_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:22.304059Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55065b79-e533e8c4-15677330-3711de91_0] Write session will restart in 0.008437s
2025-10-09T21:43:22.304089Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55065b79-e533e8c4-15677330-3711de91_0] Write session: Do CDS request
2025-10-09T21:43:22.304096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55065b79-e533e8c4-15677330-3711de91_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:22.313890Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55065b79-e533e8c4-15677330-3711de91_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:22.314135Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:22.314154Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 120
2025-10-09T21:43:22.314466Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 120 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:22.314506Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 120 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:22.314512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=120 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:22.314516Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:22.314958Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 120 sessionId:  describe result for acl check
2025-10-09T21:43:22.314993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:22.314995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:22.314997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:22.315002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340878288438948:3253] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:22.315631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340878288438948:3253] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:22.318473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340878288438948:3253] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:22.324626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340878288438948:3253] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:22.324644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340878288438948:3253] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:22.324648Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340878288438948:3253] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:22.324658Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 120 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:22.335686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:22.336888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:22.337124Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:22.337128Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:22.337141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:22.337201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:22.337261Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340877926630042:3041]
2025-10-09T21:43:22.337835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:22.338292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:22.338356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:22.338746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:22.338808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:22.338813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:22.338814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:22.338816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:22.338817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:22.338818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:22.338819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.338820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.338821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.338826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:22.338836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:22.338840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:22.338844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:22.338848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:22.338852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:22.338856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.338861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.338864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.338873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:22.338882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:22.338889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:22.338894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:22.338899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:22.338905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:22.338909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:22.338916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:22.338921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:22.338926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:22.338931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:22.338937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:22.338943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.338949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.338952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.338958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.338964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.338974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.339004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:22.348095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:22.349450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:22.349470Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:22.349474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:22.349479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:22.349488Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 119 [2:7559340877926630042:3041]
2025-10-09T21:43:22.349507Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:22.349538Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:22.349542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.349546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.349549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.349571Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:22.349579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:22.349592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:22.349673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.349684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.349686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.349689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:22.349768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:22.349780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.349794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.349811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.349834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:22.349866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340878288438995:3253], now have 1 active actors on pipe
2025-10-09T21:43:22.349875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340878288438939:2432], now have 1 active actors on pipe
2025-10-09T21:43:22.349883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340877926630035:3041]
2025-10-09T21:43:22.349886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340877926630035:3041]
2025-10-09T21:43:22.349888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340877926630035:3041]
2025-10-09T21:43:22.349906Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:22.349925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:22.349942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:22.349951Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:22.349995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340877926630035:3041]
2025-10-09T21:43:22.350000Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:22.350014Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:22.349973Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 119
2025-10-09T21:43:22.350024Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:22.350047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:22.349977Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 119
2025-10-09T21:43:22.350054Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:22.350176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:22.350188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:22.350224Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fd2d5015-817ed3e4-8e54ad7d-3a056bcd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:22.350233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.350238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.350241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:22.350256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.350269Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:22.350289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:22.350578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:22.350591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:22.350614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:22.351459Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55065b79-e533e8c4-15677330-3711de91_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fd2d5015-817ed3e4-8e54ad7d-3a056bcd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:22.350972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 120 partition: 0 MaxSeqNo: 25 sessionId: src_id|fd2d5015-817ed3e4-8e54ad7d-3a056bcd_0
2025-10-09T21:43:22.373181Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:22.373196Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:22.373199Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:22.373201Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:22.373211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340877926630035:3041]
2025-10-09T21:43:22.373216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340877926630035:3041]
2025-10-09T21:43:22.373220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340877926630035:3041]
2025-10-09T21:43:22.373272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:22.373280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.373281Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:22.373283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:22.373289Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:22.373293Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:22.373462Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:22.378972Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fd2d5015-817ed3e4-8e54ad7d-3a056bcd_0] Write session: try to update token
2025-10-09T21:43:22.378997Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fd2d5015-817ed3e4-8e54ad7d-3a056bcd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:22.403865Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340878288439122:3142]
2025-10-09T21:43:22.404228Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 119, pipe: [1:7559340878288439122:3142]
2025-10-09T21:43:22.405629Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 120 sessionId: src_id|fd2d5015-817ed3e4-8e54ad7d-3a056bcd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:22.413717Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:22.537154Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:22.537689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340878288439122:3142], now have 1 active actors on pipe
2025-10-09T21:43:22.537718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:22.537724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:22.538475Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340878288439122:3142]
2025-10-09T21:43:22.538508Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:22.538511Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:22.538566Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 120 reason: error at write: Disconnected sessionId: src_id|fd2d5015-817ed3e4-8e54ad7d-3a056bcd_0
2025-10-09T21:43:22.538781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 120 sessionId: src_id|fd2d5015-817ed3e4-8e54ad7d-3a056bcd_0 is DEAD
2025-10-09T21:43:22.538910Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:22.540295Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd2d5015-817ed3e4-8e54ad7d-3a056bcd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:22.540309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd2d5015-817ed3e4-8e54ad7d-3a056bcd_0] Write session will restart in 0.008154s
2025-10-09T21:43:22.540332Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd2d5015-817ed3e4-8e54ad7d-3a056bcd_0] Write session: Do CDS request
2025-10-09T21:43:22.540343Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd2d5015-817ed3e4-8e54ad7d-3a056bcd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:22.538172Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:22.550521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fd2d5015-817ed3e4-8e54ad7d-3a056bcd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:22.551452Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:22.551467Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 121
2025-10-09T21:43:22.551809Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 121 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:22.551842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 121 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:22.551848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=121 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:22.551851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:22.552355Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 121 sessionId:  describe result for acl check
2025-10-09T21:43:22.552392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:22.552395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:22.552396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:22.552403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340878288439158:3258] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:22.552961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340878288439158:3258] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:22.562450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340878288439158:3258] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:22.571904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:22.572627Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340878288439158:3258] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:22.572644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340878288439158:3258] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:22.572648Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340878288439158:3258] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:22.572658Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 121 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:22.573146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:22.573354Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:22.573364Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:22.573377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:22.573434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:22.573481Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340877926630217:3043]
2025-10-09T21:43:22.574085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:22.574474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:22.574517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:22.574843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:22.574912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:22.574922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:22.574923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:22.574924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:22.574926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:22.574927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:22.574928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.574929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.574931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.574937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:22.574949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:22.574953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:22.574958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:22.574962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:22.574965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:22.574969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.574973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.574976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.574984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:22.574992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:22.574998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:22.575003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:22.575007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:22.575012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:22.575016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:22.575021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:22.575025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:22.575030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:22.575034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:22.575038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:22.575043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.575047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.575060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.575065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.575069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.575079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.575104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:22.581847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:22.583054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:22.583077Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:22.583080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:22.583085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:22.583092Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 120 [2:7559340877926630217:3043]
2025-10-09T21:43:22.583109Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:22.583130Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:22.583133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.583137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.583140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.583162Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:22.583169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:22.583177Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:22.583279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.583288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.583290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.583292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:22.583371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:22.583378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.583391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.583402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.583416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:22.583439Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340877926630211:3043]
2025-10-09T21:43:22.583442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340877926630211:3043]
2025-10-09T21:43:22.583444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340877926630211:3043]
2025-10-09T21:43:22.583459Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:22.583475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340878288439127:2432], now have 1 active actors on pipe
2025-10-09T21:43:22.583479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340878288439295:3258], now have 1 active actors on pipe
2025-10-09T21:43:22.583487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:22.583501Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:22.583504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:22.583544Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:22.583545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340877926630211:3043]
2025-10-09T21:43:22.583548Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:22.583550Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:22.583552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:22.583558Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:22.584010Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 120
2025-10-09T21:43:22.584043Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 120
2025-10-09T21:43:22.584593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:22.584608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:22.584643Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|edb9130a-5dd0362f-5bf948e0-fa60c2b8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:22.584674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.584683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.584686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:22.584699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.584725Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:22.584755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:22.585027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:22.585036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:22.585058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:22.585738Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd2d5015-817ed3e4-8e54ad7d-3a056bcd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|edb9130a-5dd0362f-5bf948e0-fa60c2b8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:22.585188Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 121 partition: 0 MaxSeqNo: 25 sessionId: src_id|edb9130a-5dd0362f-5bf948e0-fa60c2b8_0
2025-10-09T21:43:22.588859Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|edb9130a-5dd0362f-5bf948e0-fa60c2b8_0] Write session: try to update token
2025-10-09T21:43:22.588884Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|edb9130a-5dd0362f-5bf948e0-fa60c2b8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:22.606218Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:22.606235Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:22.606238Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:22.606240Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:22.606250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340877926630211:3043]
2025-10-09T21:43:22.606254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340877926630211:3043]
2025-10-09T21:43:22.606261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340877926630211:3043]
2025-10-09T21:43:22.606306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:22.606315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.606317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:22.606327Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:22.606334Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:22.606355Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:22.606479Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:22.611305Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 121 sessionId: src_id|edb9130a-5dd0362f-5bf948e0-fa60c2b8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:22.614719Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:22.638821Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340878288439307:3142]
2025-10-09T21:43:22.646245Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 120, pipe: [1:7559340878288439307:3142]
2025-10-09T21:43:22.764509Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:22.764821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:22.764836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:22.764888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340878288439307:3142], now have 1 active actors on pipe
2025-10-09T21:43:22.765089Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:22.765144Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340878288439307:3142]
2025-10-09T21:43:22.765161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:22.765168Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:22.765246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 121 reason: error at write: Disconnected sessionId: src_id|edb9130a-5dd0362f-5bf948e0-fa60c2b8_0
2025-10-09T21:43:22.765480Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 121 sessionId: src_id|edb9130a-5dd0362f-5bf948e0-fa60c2b8_0 is DEAD
2025-10-09T21:43:22.765209Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:22.765670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edb9130a-5dd0362f-5bf948e0-fa60c2b8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:22.765685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edb9130a-5dd0362f-5bf948e0-fa60c2b8_0] Write session will restart in 0.007636s
2025-10-09T21:43:22.765716Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edb9130a-5dd0362f-5bf948e0-fa60c2b8_0] Write session: Do CDS request
2025-10-09T21:43:22.765724Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edb9130a-5dd0362f-5bf948e0-fa60c2b8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:22.774739Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|edb9130a-5dd0362f-5bf948e0-fa60c2b8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:22.775130Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:22.775145Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 122
2025-10-09T21:43:22.775421Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 122 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:22.775455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 122 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:22.775460Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=122 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:22.775464Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:22.775891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 122 sessionId:  describe result for acl check
2025-10-09T21:43:22.775921Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:22.775924Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:22.775925Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:22.775930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340878288439329:3263] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:22.776474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340878288439329:3263] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:22.778904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340878288439329:3263] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:22.782649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340878288439329:3263] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:22.782665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340878288439329:3263] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:22.782669Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340878288439329:3263] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:22.782679Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 122 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:22.791810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:22.793068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:22.793273Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:22.793284Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:22.793296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:22.793350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:22.793396Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340877926630396:3046]
2025-10-09T21:43:22.793960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:22.794251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:22.794295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:22.794614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:22.794690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:22.794692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:22.794693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:22.794694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:22.794696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:22.794697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:22.794698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.794699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.794700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.794705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:22.794716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:22.794720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:22.794723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:22.794726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:22.794730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:22.794733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.794736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.794740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.794748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:22.794756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:22.794762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:22.794768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:22.794772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:22.794777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:22.794781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:22.794786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:22.794790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:22.794795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:22.794799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:22.794804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:22.794808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.794813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.794817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.794822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.794826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.794831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.794860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:22.803316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:22.804735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:22.804742Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:22.804745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:22.804750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:22.804758Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 121 [2:7559340877926630396:3046]
2025-10-09T21:43:22.804777Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:22.804799Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:22.804803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.804807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.804809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.804830Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:22.804839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:22.804846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:22.804918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.804920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.804921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.804924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:22.805021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:22.805024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.805039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:22.805050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:22.805066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:22.805088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340877926630390:3046]
2025-10-09T21:43:22.805091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340877926630390:3046]
2025-10-09T21:43:22.805093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340877926630390:3046]
2025-10-09T21:43:22.805111Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:22.805128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:22.805143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:22.805147Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:22.805488Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 121
2025-10-09T21:43:22.805170Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:22.805174Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:22.805176Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:22.805194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340877926630390:3046]
2025-10-09T21:43:22.805198Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:22.805284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:22.805469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340878288439418:3263], now have 1 active actors on pipe
2025-10-09T21:43:22.805474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340878288439315:2432], now have 1 active actors on pipe
2025-10-09T21:43:22.805545Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 121
2025-10-09T21:43:22.805783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:22.805792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:22.805824Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a0b6a3f5-4ce8365-a906f262-ff6b73f8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:22.805836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.805846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.805849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:22.805863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.805889Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:22.805910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:22.806794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 122 partition: 0 MaxSeqNo: 25 sessionId: src_id|a0b6a3f5-4ce8365-a906f262-ff6b73f8_0
2025-10-09T21:43:22.807247Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edb9130a-5dd0362f-5bf948e0-fa60c2b8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a0b6a3f5-4ce8365-a906f262-ff6b73f8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:22.806253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:22.806264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:22.806285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:22.810109Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a0b6a3f5-4ce8365-a906f262-ff6b73f8_0] Write session: try to update token
2025-10-09T21:43:22.810130Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a0b6a3f5-4ce8365-a906f262-ff6b73f8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:22.832194Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:22.832219Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:22.832221Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:22.832224Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:22.832234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340877926630390:3046]
2025-10-09T21:43:22.832240Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340877926630390:3046]
2025-10-09T21:43:22.832244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340877926630390:3046]
2025-10-09T21:43:22.832286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:22.832292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:22.832296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:22.832445Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:22.832470Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:22.832479Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:22.832482Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:22.834758Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 122 sessionId: src_id|a0b6a3f5-4ce8365-a906f262-ff6b73f8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:22.837734Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:22.866307Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340878288439502:3142]
2025-10-09T21:43:22.866960Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 121, pipe: [1:7559340878288439502:3142]
2025-10-09T21:43:23.017632Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:23.018442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:23.018465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:23.018524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340878288439502:3142], now have 1 active actors on pipe
2025-10-09T21:43:23.019171Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:23.019187Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:23.019519Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:23.019577Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340878288439502:3142]
2025-10-09T21:43:23.019689Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 122 reason: error at write: Disconnected sessionId: src_id|a0b6a3f5-4ce8365-a906f262-ff6b73f8_0
2025-10-09T21:43:23.019893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 122 sessionId: src_id|a0b6a3f5-4ce8365-a906f262-ff6b73f8_0 is DEAD
2025-10-09T21:43:23.020260Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0b6a3f5-4ce8365-a906f262-ff6b73f8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:23.020273Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0b6a3f5-4ce8365-a906f262-ff6b73f8_0] Write session will restart in 0.005436s
2025-10-09T21:43:23.020308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0b6a3f5-4ce8365-a906f262-ff6b73f8_0] Write session: Do CDS request
2025-10-09T21:43:23.020318Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0b6a3f5-4ce8365-a906f262-ff6b73f8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:23.020385Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:23.026672Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a0b6a3f5-4ce8365-a906f262-ff6b73f8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:23.030221Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:23.030240Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 123
2025-10-09T21:43:23.030460Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 123 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:23.030499Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 123 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:23.030510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=123 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:23.030514Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:23.031029Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 123 sessionId:  describe result for acl check
2025-10-09T21:43:23.031069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:23.031071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:23.031073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:23.031079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340882583406816:3267] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:23.031653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340882583406816:3267] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:23.041054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340882583406816:3267] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:23.073174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340882583406816:3267] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:23.073192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340882583406816:3267] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:23.073196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340882583406816:3267] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:23.073208Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 123 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:23.084213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:23.087590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:23.087847Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:23.087859Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:23.087873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:23.087939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:23.087988Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340882221597885:3049]
2025-10-09T21:43:23.088624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:23.090764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:23.090852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:23.091257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:23.091316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:23.091317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:23.091319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:23.091320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:23.091321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:23.091322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:23.091324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.091326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.091327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.091333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:23.091343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:23.091347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:23.091351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:23.091356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:23.091360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:23.091364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.091368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.091371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.091379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:23.091388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:23.091394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:23.091400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:23.091405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:23.091410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:23.091415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:23.091420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:23.091425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:23.091430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:23.091434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:23.091440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:23.091445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.091450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.091455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.091460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.091465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.091470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.091499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:23.119580Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340882583406992:3142]
2025-10-09T21:43:23.119816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:23.121422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:23.121437Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:23.121439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:23.121444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:23.121452Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 122 [2:7559340882221597885:3049]
2025-10-09T21:43:23.121468Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:23.121488Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:23.121492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.121495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.121497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.121518Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:23.121531Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:23.121541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:23.121608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.121610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.121612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.121615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:23.121693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:23.121696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.121713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.121726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.121745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:23.121768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340882221597879:3049]
2025-10-09T21:43:23.121771Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340882221597879:3049]
2025-10-09T21:43:23.121773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340882221597879:3049]
2025-10-09T21:43:23.121787Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:23.121802Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:23.121819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:23.121822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:23.121866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340882221597879:3049]
2025-10-09T21:43:23.121869Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:23.122134Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:23.122143Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:23.122144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:23.122150Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:23.123651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340882583406969:3267], now have 1 active actors on pipe
2025-10-09T21:43:23.123666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340882583406804:2432], now have 1 active actors on pipe
2025-10-09T21:43:23.125887Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 122
2025-10-09T21:43:23.125930Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 122
2025-10-09T21:43:23.127251Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 122, pipe: [1:7559340882583406992:3142]
2025-10-09T21:43:23.127347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:23.127365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:23.127392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340882583406992:3142], now have 1 active actors on pipe
2025-10-09T21:43:23.127416Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|179e4aeb-30b770d7-1d12c419-1c9370c6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:23.127432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.127437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.127440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:23.127455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.127475Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:23.127498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:23.131853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:23.131873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:23.131911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:23.132088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 123 partition: 0 MaxSeqNo: 25 sessionId: src_id|179e4aeb-30b770d7-1d12c419-1c9370c6_0
2025-10-09T21:43:23.134631Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0b6a3f5-4ce8365-a906f262-ff6b73f8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|179e4aeb-30b770d7-1d12c419-1c9370c6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:23.137198Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|179e4aeb-30b770d7-1d12c419-1c9370c6_0] Write session: try to update token
2025-10-09T21:43:23.137219Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|179e4aeb-30b770d7-1d12c419-1c9370c6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:23.154388Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:23.154408Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:23.154411Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:23.154432Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:23.154444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340882221597879:3049]
2025-10-09T21:43:23.154450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340882221597879:3049]
2025-10-09T21:43:23.154454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340882221597879:3049]
2025-10-09T21:43:23.154508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:23.154517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.154520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:23.154674Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:23.156234Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:23.156251Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:23.156255Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:23.166556Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 123 sessionId: src_id|179e4aeb-30b770d7-1d12c419-1c9370c6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:23.169249Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:23.339710Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:23.340372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:23.340387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:23.341042Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:23.341126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 123 reason: status is not ok: tablet will be restarted right now sessionId: src_id|179e4aeb-30b770d7-1d12c419-1c9370c6_0
2025-10-09T21:43:23.341317Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 123 sessionId: src_id|179e4aeb-30b770d7-1d12c419-1c9370c6_0 is DEAD
2025-10-09T21:43:23.341449Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:23.341482Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340882583406992:3142]
2025-10-09T21:43:23.341800Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|179e4aeb-30b770d7-1d12c419-1c9370c6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:23.341810Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|179e4aeb-30b770d7-1d12c419-1c9370c6_0] Write session will restart in 0.006811s
2025-10-09T21:43:23.341834Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|179e4aeb-30b770d7-1d12c419-1c9370c6_0] Write session: Do CDS request
2025-10-09T21:43:23.341841Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|179e4aeb-30b770d7-1d12c419-1c9370c6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:23.340657Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:23.349773Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|179e4aeb-30b770d7-1d12c419-1c9370c6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:23.349863Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:23.349879Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 124
2025-10-09T21:43:23.350590Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 124 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:23.350623Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 124 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:23.350629Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=124 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:23.350633Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:23.352644Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 124 sessionId:  describe result for acl check
2025-10-09T21:43:23.352683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:23.352685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:23.352687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:23.352693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340882583407027:3278] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:23.353228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340882583407027:3278] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:23.357422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340882583407027:3278] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:23.367254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340882583407027:3278] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:23.367269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340882583407027:3278] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:23.367273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340882583407027:3278] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:23.367283Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 124 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:23.374515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:23.376213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:23.376422Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:23.376426Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:23.376435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:23.376487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:23.376526Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340882221598089:3063]
2025-10-09T21:43:23.376997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:23.377456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:23.377487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:23.377717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:23.377751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:23.377752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:23.377752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:23.377753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:23.377754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:23.377754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:23.377755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.377756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.377756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.377760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:23.377766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:23.377768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:23.377771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:23.377773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:23.377775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:23.377777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.377779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.377781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.377786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:23.377798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:23.377805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:23.377810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:23.377814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:23.377819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:23.377823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:23.377829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:23.377831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:23.377834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:23.377836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:23.377839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:23.377842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.377845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.377847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.377850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.377852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.377855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.377878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:23.387583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:23.388732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:23.388738Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:23.388740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:23.388745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:23.388751Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 123 [2:7559340882221598089:3063]
2025-10-09T21:43:23.388762Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:23.388781Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:23.388783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.388786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.388788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.388804Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:23.388811Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:23.388818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:23.388885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.388886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.388888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.388890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:23.388950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:23.388953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.388965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.388976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.388991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:23.389013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340882221598083:3063]
2025-10-09T21:43:23.389016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340882221598083:3063]
2025-10-09T21:43:23.389019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340882221598083:3063]
2025-10-09T21:43:23.389036Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:23.389053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:23.389069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:23.389072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:23.389113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:23.389115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340882221598083:3063]
2025-10-09T21:43:23.389116Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:23.389118Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:23.389119Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:23.389126Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:23.389635Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 123
2025-10-09T21:43:23.389672Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 123
2025-10-09T21:43:23.390320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340882583407151:3278], now have 1 active actors on pipe
2025-10-09T21:43:23.390330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340882583407014:2432], now have 1 active actors on pipe
2025-10-09T21:43:23.390351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:23.390358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:23.390394Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|19cbe0f9-4db9e292-7e916795-e1384cd9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:23.390403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.390408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.390410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:23.390436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.390450Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:23.390479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:23.391413Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 124 partition: 0 MaxSeqNo: 25 sessionId: src_id|19cbe0f9-4db9e292-7e916795-e1384cd9_0
2025-10-09T21:43:23.391868Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|179e4aeb-30b770d7-1d12c419-1c9370c6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|19cbe0f9-4db9e292-7e916795-e1384cd9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:23.391229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:23.391238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:23.391261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:23.395087Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19cbe0f9-4db9e292-7e916795-e1384cd9_0] Write session: try to update token
2025-10-09T21:43:23.395111Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19cbe0f9-4db9e292-7e916795-e1384cd9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:23.418385Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:23.418400Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:23.418402Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:23.418404Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:23.418423Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340882221598083:3063]
2025-10-09T21:43:23.418428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340882221598083:3063]
2025-10-09T21:43:23.418432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340882221598083:3063]
2025-10-09T21:43:23.418477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:23.418485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.418488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:23.418701Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:23.418716Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:23.418723Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:23.418727Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:23.431787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 124 sessionId: src_id|19cbe0f9-4db9e292-7e916795-e1384cd9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:23.434484Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:23.440644Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340882583407201:3142]
2025-10-09T21:43:23.441725Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 123, pipe: [1:7559340882583407201:3142]
2025-10-09T21:43:23.661526Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:23.661931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:23.661941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:23.661964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340882583407201:3142], now have 1 active actors on pipe
2025-10-09T21:43:23.662411Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:23.662493Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:23.663589Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19cbe0f9-4db9e292-7e916795-e1384cd9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:23.663620Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19cbe0f9-4db9e292-7e916795-e1384cd9_0] Write session will restart in 0.008180s
2025-10-09T21:43:23.662799Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:23.662880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 124 reason: status is not ok: tablet will be restarted right now sessionId: src_id|19cbe0f9-4db9e292-7e916795-e1384cd9_0
2025-10-09T21:43:23.663647Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19cbe0f9-4db9e292-7e916795-e1384cd9_0] Write session: Do CDS request
2025-10-09T21:43:23.663662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19cbe0f9-4db9e292-7e916795-e1384cd9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:23.663174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 124 sessionId: src_id|19cbe0f9-4db9e292-7e916795-e1384cd9_0 is DEAD
2025-10-09T21:43:23.663299Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340882583407201:3142]
2025-10-09T21:43:23.676517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19cbe0f9-4db9e292-7e916795-e1384cd9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:23.677037Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:23.677054Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 125
2025-10-09T21:43:23.679062Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 125 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:23.679106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 125 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:23.679111Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=125 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:23.679115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:23.680390Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 125 sessionId:  describe result for acl check
2025-10-09T21:43:23.680436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:23.680438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:23.680440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:23.680446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340882583407246:3283] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:23.680923Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340882583407246:3283] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:23.688992Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340882583407246:3283] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:23.695216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:23.696816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:23.697048Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:23.697052Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:23.697063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:23.697129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:23.697179Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340882221598269:3065]
2025-10-09T21:43:23.697758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:23.698080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:23.698139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:23.698515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:23.698568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:23.698569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:23.698570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:23.698572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:23.698573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:23.698574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:23.698576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.698577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.698578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.698584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:23.698596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:23.698600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:23.698605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:23.698609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:23.698613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:23.698617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.698621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.698626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.698633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:23.698642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:23.698649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:23.698655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:23.698660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:23.698666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:23.698670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:23.698676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:23.698681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:23.698687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:23.698691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:23.698697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:23.698702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.698708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.698713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.698718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.698723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.698728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.698758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:23.702880Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340882583407246:3283] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:23.702900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340882583407246:3283] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:23.702904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340882583407246:3283] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:23.702915Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 125 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:23.708643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:23.710315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:23.710326Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:23.710328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:23.710333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:23.710340Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 124 [2:7559340882221598269:3065]
2025-10-09T21:43:23.710355Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:23.710378Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:23.710380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.710383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.710384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.710401Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:23.710405Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:23.710411Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:23.710488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.710491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.710492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.710494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:23.710559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:23.710560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.710570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:23.710575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:23.710585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:23.710604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340882583407205:2432], now have 1 active actors on pipe
2025-10-09T21:43:23.710607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340882583407383:3283], now have 1 active actors on pipe
2025-10-09T21:43:23.710615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340882221598263:3065]
2025-10-09T21:43:23.710617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340882221598263:3065]
2025-10-09T21:43:23.710619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340882221598263:3065]
2025-10-09T21:43:23.710628Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:23.710641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:23.710650Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:23.710652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:23.710653Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:23.710656Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:23.710658Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:23.710688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340882221598263:3065]
2025-10-09T21:43:23.710690Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:23.710693Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:23.710964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:23.710980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:23.711012Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f525ec0e-8051c343-b6bbca9d-e5af08a2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:23.711022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.711027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.711030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:23.711044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.711060Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:23.711077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:23.711792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:23.711802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:23.711823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:23.712852Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19cbe0f9-4db9e292-7e916795-e1384cd9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f525ec0e-8051c343-b6bbca9d-e5af08a2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:23.710699Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 124
2025-10-09T21:43:23.710743Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 124
2025-10-09T21:43:23.712453Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 125 partition: 0 MaxSeqNo: 25 sessionId: src_id|f525ec0e-8051c343-b6bbca9d-e5af08a2_0
2025-10-09T21:43:23.715781Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f525ec0e-8051c343-b6bbca9d-e5af08a2_0] Write session: try to update token
2025-10-09T21:43:23.715799Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f525ec0e-8051c343-b6bbca9d-e5af08a2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:23.737566Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 125 sessionId: src_id|f525ec0e-8051c343-b6bbca9d-e5af08a2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:23.740360Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:23.740918Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:23.740643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:23.740667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:23.740713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:23.746333Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:23.746352Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:23.746355Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:23.746357Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:23.746368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340882221598263:3065]
2025-10-09T21:43:23.746373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340882221598263:3065]
2025-10-09T21:43:23.746378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340882221598263:3065]
2025-10-09T21:43:23.746444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:23.746454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:23.746457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:23.746599Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:23.750729Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:23.750746Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:23.750750Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:23.766509Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340882583407396:3142]
2025-10-09T21:43:23.766974Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 124, pipe: [1:7559340882583407396:3142]
2025-10-09T21:43:23.960609Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:23.970834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:23.970855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:23.971039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:23.971103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:23.971169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:23.971237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:23.971575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:23.971581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:23.971677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:23.971736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:23.971796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:23.971860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:23.971903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:23.971907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:23.971974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:23.972033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:23.972094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:23.972157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:23.972201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:23.972204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:23.972270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:23.972334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:23.972400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:23.972459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:23.972505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:23.972508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:23.972574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:23.972635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:23.972701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:23.972765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:23.972809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:23.972813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:23.972913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:23.972988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:23.973065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:23.973143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:23.973193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:23.973196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:23.973275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:23.973339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:23.973400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:23.973460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:23.973511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:23.973514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:23.973580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:23.973638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:23.973702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:23.973760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:23.973801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:23.973804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:23.973871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:23.973935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:23.973998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:23.974926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:23.974983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:23.974987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:23.975072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:23.975130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:23.975195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:23.975263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:23.975351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:23.975354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:23.975442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340882583407396:3142], now have 1 active actors on pipe
2025-10-09T21:43:23.977132Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:23.977183Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:23.977187Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:23.977210Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 125 reason: error at write: Disconnected sessionId: src_id|f525ec0e-8051c343-b6bbca9d-e5af08a2_0
2025-10-09T21:43:23.977360Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 125 sessionId: src_id|f525ec0e-8051c343-b6bbca9d-e5af08a2_0 is DEAD
2025-10-09T21:43:23.977496Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340882583407396:3142]
2025-10-09T21:43:23.977637Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f525ec0e-8051c343-b6bbca9d-e5af08a2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:23.977645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f525ec0e-8051c343-b6bbca9d-e5af08a2_0] Write session will restart in 0.008060s
2025-10-09T21:43:23.977685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f525ec0e-8051c343-b6bbca9d-e5af08a2_0] Write session: Do CDS request
2025-10-09T21:43:23.977691Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f525ec0e-8051c343-b6bbca9d-e5af08a2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:23.977551Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:23.986571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f525ec0e-8051c343-b6bbca9d-e5af08a2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:23.989230Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:23.989247Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 126
2025-10-09T21:43:23.989463Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 126 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:23.989510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 126 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:23.989516Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=126 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:23.989532Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:23.990153Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 126 sessionId:  describe result for acl check
2025-10-09T21:43:23.990187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:23.990190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:23.990192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:23.990199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340882583407414:3288] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:23.990854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340882583407414:3288] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:24.002463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340882583407414:3288] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:24.021439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340882583407414:3288] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:24.021458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340882583407414:3288] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:24.021463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340882583407414:3288] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:24.021474Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 126 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:24.035963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:24.040693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:24.040998Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:24.041003Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:24.041019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:24.041090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:24.041133Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340886516565752:3069]
2025-10-09T21:43:24.041556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:24.042223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:24.042253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:24.042529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:24.042581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:24.042582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:24.042584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:24.042585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:24.042587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:24.042588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:24.042589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.042590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.042591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.042597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:24.042607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:24.042611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:24.042616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:24.042619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:24.042622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:24.042625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.042629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.042632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.042640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:24.042649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:24.042654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:24.042659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:24.042664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:24.042668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:24.042689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:24.042692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:24.042695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:24.042700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:24.042703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:24.042707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:24.042710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.042713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.042715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.042718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.042722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.042725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.042753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:24.053376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:24.055228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:24.055246Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:24.055249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:24.055253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:24.055262Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 125 [2:7559340886516565752:3069]
2025-10-09T21:43:24.055278Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:24.055299Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:24.055302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.055306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.055308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.055328Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:24.055344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:24.055358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:24.055427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.055429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.055431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.055434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:24.055511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:24.055514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.055529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.056412Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 125
2025-10-09T21:43:24.056452Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 125
2025-10-09T21:43:24.055544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.055570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:24.055591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340886516565747:3069]
2025-10-09T21:43:24.055595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340886516565747:3069]
2025-10-09T21:43:24.055597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340886516565747:3069]
2025-10-09T21:43:24.055612Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:24.055628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:24.055644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:24.055647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:24.055691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340886516565747:3069]
2025-10-09T21:43:24.055694Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:24.055798Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:24.055800Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:24.055802Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:24.055809Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:24.056087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340882583407407:2432], now have 1 active actors on pipe
2025-10-09T21:43:24.056095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340886878374805:3288], now have 1 active actors on pipe
2025-10-09T21:43:24.056814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:24.056822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:24.056855Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7830b340-72b029f4-b8298ec-64cf4a60_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:24.056865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.056870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.056873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:24.056886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.056900Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:24.056918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:24.057995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 126 partition: 0 MaxSeqNo: 25 sessionId: src_id|7830b340-72b029f4-b8298ec-64cf4a60_0
2025-10-09T21:43:24.058710Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f525ec0e-8051c343-b6bbca9d-e5af08a2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7830b340-72b029f4-b8298ec-64cf4a60_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:24.057827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:24.057839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:24.057862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:24.061487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7830b340-72b029f4-b8298ec-64cf4a60_0] Write session: try to update token
2025-10-09T21:43:24.061505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7830b340-72b029f4-b8298ec-64cf4a60_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:24.077083Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:24.077105Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:24.077109Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:24.077112Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:24.077124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340886516565747:3069]
2025-10-09T21:43:24.077130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340886516565747:3069]
2025-10-09T21:43:24.077135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340886516565747:3069]
2025-10-09T21:43:24.077191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:24.077200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.077203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:24.077399Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:24.077717Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:24.077722Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:24.077726Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:24.079286Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340886878374888:3142]
2025-10-09T21:43:24.080486Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 125, pipe: [1:7559340886878374888:3142]
2025-10-09T21:43:24.089199Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 126 sessionId: src_id|7830b340-72b029f4-b8298ec-64cf4a60_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:24.092466Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:24.277022Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:24.277462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340886878374888:3142], now have 1 active actors on pipe
2025-10-09T21:43:24.277499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:24.277506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:24.277951Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:24.278100Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340886878374888:3142]
2025-10-09T21:43:24.278131Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:24.278164Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:24.278166Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:24.278223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 126 reason: error at write: Disconnected sessionId: src_id|7830b340-72b029f4-b8298ec-64cf4a60_0
2025-10-09T21:43:24.278407Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 126 sessionId: src_id|7830b340-72b029f4-b8298ec-64cf4a60_0 is DEAD
2025-10-09T21:43:24.278668Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7830b340-72b029f4-b8298ec-64cf4a60_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:24.278676Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7830b340-72b029f4-b8298ec-64cf4a60_0] Write session will restart in 0.008667s
2025-10-09T21:43:24.278704Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7830b340-72b029f4-b8298ec-64cf4a60_0] Write session: Do CDS request
2025-10-09T21:43:24.278712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7830b340-72b029f4-b8298ec-64cf4a60_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:24.290898Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7830b340-72b029f4-b8298ec-64cf4a60_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:24.291851Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:24.291867Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 127
2025-10-09T21:43:24.292067Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 127 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:24.292098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 127 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:24.292104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=127 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:24.292108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:24.298456Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 127 sessionId:  describe result for acl check
2025-10-09T21:43:24.298503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:24.298505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:24.298507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:24.298513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340886878374996:3299] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:24.299088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340886878374996:3299] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:24.304740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340886878374996:3299] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:24.311452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:24.313221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:24.313431Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:24.313435Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:24.313447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:24.313495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:24.313555Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340886516565970:3083]
2025-10-09T21:43:24.314206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:24.314612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:24.314651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:24.314985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:24.315027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:24.315028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:24.315030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:24.315031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:24.314249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340886878374996:3299] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:24.314265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340886878374996:3299] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:24.314270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340886878374996:3299] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:24.314281Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 127 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:24.315032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:24.315034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:24.315035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.315037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.315038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.315044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:24.315056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:24.315060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:24.315065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:24.315069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:24.315073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:24.315077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.315081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.315086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.315093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:24.315101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:24.315107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:24.315112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:24.315117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:24.315122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:24.315127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:24.315133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:24.315138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:24.315144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:24.315148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:24.315154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:24.315159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.315164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.315169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.315175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.315180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.315185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.315217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:24.323642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:24.324736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:24.324742Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:24.324745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:24.324750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:24.324759Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 126 [2:7559340886516565970:3083]
2025-10-09T21:43:24.324774Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:24.324795Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:24.324798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.324802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.324804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.324824Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:24.324830Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:24.324838Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:24.324898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.324900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.324902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.324905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:24.324985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:24.324988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.325003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.325017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.325035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:24.325055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340886516565964:3083]
2025-10-09T21:43:24.325057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340886516565964:3083]
2025-10-09T21:43:24.325059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340886516565964:3083]
2025-10-09T21:43:24.325068Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:24.325082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:24.325094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:24.325097Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:24.325137Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340886516565964:3083]
2025-10-09T21:43:24.325139Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:24.325383Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:24.325386Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:24.325388Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:24.325393Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:24.325693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340886878375089:3299], now have 1 active actors on pipe
2025-10-09T21:43:24.325697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340886878374916:2432], now have 1 active actors on pipe
2025-10-09T21:43:24.325976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:24.325981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:24.326013Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|46b1121e-bc9527c9-f784cfea-d29d631d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:24.326022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.326025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.326042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:24.326054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.326068Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:24.326087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:24.325740Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 126
2025-10-09T21:43:24.325776Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 126
2025-10-09T21:43:24.329017Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 127 partition: 0 MaxSeqNo: 25 sessionId: src_id|46b1121e-bc9527c9-f784cfea-d29d631d_0
2025-10-09T21:43:24.329423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7830b340-72b029f4-b8298ec-64cf4a60_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|46b1121e-bc9527c9-f784cfea-d29d631d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:24.328761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:24.328773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:24.328798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:24.332266Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46b1121e-bc9527c9-f784cfea-d29d631d_0] Write session: try to update token
2025-10-09T21:43:24.332286Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46b1121e-bc9527c9-f784cfea-d29d631d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:24.347716Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:24.347733Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:24.347735Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:24.347737Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:24.347747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340886516565964:3083]
2025-10-09T21:43:24.347752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340886516565964:3083]
2025-10-09T21:43:24.347756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340886516565964:3083]
2025-10-09T21:43:24.347803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:24.347811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.347813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:24.347968Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:24.350340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:24.350349Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:24.350353Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:24.365798Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 127 sessionId: src_id|46b1121e-bc9527c9-f784cfea-d29d631d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:24.368712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:24.378120Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340886878375105:3142]
2025-10-09T21:43:24.378595Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 126, pipe: [1:7559340886878375105:3142]
2025-10-09T21:43:24.525474Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:24.526829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:24.526845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:24.526892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340886878375105:3142], now have 1 active actors on pipe
2025-10-09T21:43:24.527545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:24.527572Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:24.527689Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 127 reason: error at write: Disconnected sessionId: src_id|46b1121e-bc9527c9-f784cfea-d29d631d_0
2025-10-09T21:43:24.527609Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:24.527878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 127 sessionId: src_id|46b1121e-bc9527c9-f784cfea-d29d631d_0 is DEAD
2025-10-09T21:43:24.528102Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:24.528150Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340886878375105:3142]
2025-10-09T21:43:24.530690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46b1121e-bc9527c9-f784cfea-d29d631d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:24.530703Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46b1121e-bc9527c9-f784cfea-d29d631d_0] Write session will restart in 0.007311s
2025-10-09T21:43:24.530731Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46b1121e-bc9527c9-f784cfea-d29d631d_0] Write session: Do CDS request
2025-10-09T21:43:24.530738Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46b1121e-bc9527c9-f784cfea-d29d631d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:24.542512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46b1121e-bc9527c9-f784cfea-d29d631d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:24.546210Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:24.546228Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 128
2025-10-09T21:43:24.546428Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 128 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:24.546475Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 128 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:24.546496Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=128 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:24.546501Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:24.547025Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 128 sessionId:  describe result for acl check
2025-10-09T21:43:24.547064Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:24.547066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:24.547068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:24.547074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340886878375161:3304] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:24.547679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340886878375161:3304] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:24.551579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340886878375161:3304] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:24.558992Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340886878375161:3304] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:24.559009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340886878375161:3304] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:24.559013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340886878375161:3304] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:24.559023Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 128 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:24.559763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:24.561102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:24.561315Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:24.561337Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:24.561350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:24.561406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:24.561446Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340886516566152:3085]
2025-10-09T21:43:24.562070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:24.562412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:24.562473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:24.562768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:24.562813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:24.562818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:24.562819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:24.562820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:24.562821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:24.562822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:24.562823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.562824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.562825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.562830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:24.562839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:24.562842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:24.562845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:24.562849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:24.562852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:24.562855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.562858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.562861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.562868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:24.562876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:24.562881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:24.562886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:24.562895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:24.562899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:24.562903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:24.562907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:24.562911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:24.562915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:24.562919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:24.562924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:24.562929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.562933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.562937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.562941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.562945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.562949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.562973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:24.572947Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 127
2025-10-09T21:43:24.572980Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 127
2025-10-09T21:43:24.570818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:24.572022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:24.572027Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:24.572030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:24.572035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:24.572046Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 127 [2:7559340886516566152:3085]
2025-10-09T21:43:24.572161Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:24.572181Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:24.572185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.572188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.572191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.572209Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:24.572217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:24.572223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:24.572290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.572292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.572294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.572296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:24.572382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:24.572385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.572395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.572426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.572440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:24.572462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340886516566146:3085]
2025-10-09T21:43:24.572464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340886516566146:3085]
2025-10-09T21:43:24.572467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340886516566146:3085]
2025-10-09T21:43:24.572475Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:24.572484Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:24.572487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:24.572489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:24.572491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340886878375284:3304], now have 1 active actors on pipe
2025-10-09T21:43:24.572499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340886878375108:2432], now have 1 active actors on pipe
2025-10-09T21:43:24.572503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:24.572517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:24.572520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:24.572561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340886516566146:3085]
2025-10-09T21:43:24.572571Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:24.572575Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:24.575643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:24.575658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:24.575698Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|aef170b3-43f495eb-3da41dae-6c8af0c6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:24.575710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.575715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.575718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:24.575733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.575749Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:24.575772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:24.577474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:24.577486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:24.577526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:24.577889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 128 partition: 0 MaxSeqNo: 25 sessionId: src_id|aef170b3-43f495eb-3da41dae-6c8af0c6_0
2025-10-09T21:43:24.581434Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46b1121e-bc9527c9-f784cfea-d29d631d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|aef170b3-43f495eb-3da41dae-6c8af0c6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:24.584625Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aef170b3-43f495eb-3da41dae-6c8af0c6_0] Write session: try to update token
2025-10-09T21:43:24.584644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aef170b3-43f495eb-3da41dae-6c8af0c6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:24.611239Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:24.611257Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:24.611259Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:24.611261Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:24.611272Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340886516566146:3085]
2025-10-09T21:43:24.611276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340886516566146:3085]
2025-10-09T21:43:24.611281Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340886516566146:3085]
2025-10-09T21:43:24.611329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:24.611336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.611339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:24.611340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:24.611346Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:24.611350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:24.611483Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:24.622875Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 128 sessionId: src_id|aef170b3-43f495eb-3da41dae-6c8af0c6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:24.625666Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:24.628487Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340886878375297:3142]
2025-10-09T21:43:24.628765Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 127, pipe: [1:7559340886878375297:3142]
2025-10-09T21:43:24.778061Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:24.778513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:24.778532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:24.778555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340886878375297:3142], now have 1 active actors on pipe
2025-10-09T21:43:24.778765Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:24.779413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aef170b3-43f495eb-3da41dae-6c8af0c6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:24.779424Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aef170b3-43f495eb-3da41dae-6c8af0c6_0] Write session will restart in 0.007563s
2025-10-09T21:43:24.779448Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aef170b3-43f495eb-3da41dae-6c8af0c6_0] Write session: Do CDS request
2025-10-09T21:43:24.779455Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aef170b3-43f495eb-3da41dae-6c8af0c6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:24.778889Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:24.778950Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:24.779036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 128 reason: status is not ok: tablet will be restarted right now sessionId: src_id|aef170b3-43f495eb-3da41dae-6c8af0c6_0
2025-10-09T21:43:24.779117Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340886878375297:3142]
2025-10-09T21:43:24.779253Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 128 sessionId: src_id|aef170b3-43f495eb-3da41dae-6c8af0c6_0 is DEAD
2025-10-09T21:43:24.790604Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aef170b3-43f495eb-3da41dae-6c8af0c6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:24.790972Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:24.790998Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 129
2025-10-09T21:43:24.791201Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 129 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:24.791247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 129 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:24.791257Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=129 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:24.791261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:24.791801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 129 sessionId:  describe result for acl check
2025-10-09T21:43:24.791840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:24.791845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:24.791846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:24.791852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340886878375335:3309] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:24.792411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340886878375335:3309] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:24.799762Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340886878375335:3309] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:24.806778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:24.814328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:24.814586Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:24.814597Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:24.814608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:24.814669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:24.814706Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340886516566336:3088]
2025-10-09T21:43:24.815197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:24.815617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:24.815660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:24.815953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:24.815990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:24.815991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:24.815992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:24.815993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:24.815994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:24.815995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:24.815996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.815997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.815998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.816002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:24.816012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:24.816015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:24.816018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:24.816021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:24.816025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:24.816027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.816031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.816034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.816040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:24.816046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:24.816051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:24.816055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:24.816059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:24.816064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:24.816067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:24.816071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:24.816075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:24.816078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:24.816083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:24.816087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:24.816091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.816096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.816100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.816104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.816109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.816113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.816135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:24.822662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340886878375335:3309] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:24.822682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340886878375335:3309] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:24.822686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340886878375335:3309] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:24.822697Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 129 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:24.830950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:24.832381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:24.832399Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:24.832401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:24.832406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:24.832413Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 128 [2:7559340886516566336:3088]
2025-10-09T21:43:24.832431Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:24.832451Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:24.832459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.832462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.832465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.832483Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:24.832492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:24.832500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:24.832567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.832574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.832576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.832579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:24.832934Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 128
2025-10-09T21:43:24.832974Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 128
2025-10-09T21:43:24.832654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:24.832665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.832678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:24.832689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:24.832707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:24.832728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340886878375311:2432], now have 1 active actors on pipe
2025-10-09T21:43:24.832732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340886878375493:3309], now have 1 active actors on pipe
2025-10-09T21:43:24.832741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340886516566331:3088]
2025-10-09T21:43:24.832743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340886516566331:3088]
2025-10-09T21:43:24.832745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340886516566331:3088]
2025-10-09T21:43:24.832758Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:24.832774Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:24.832789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:24.832793Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:24.832832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340886516566331:3088]
2025-10-09T21:43:24.832835Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:24.832875Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:24.832878Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:24.832880Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:24.832886Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:24.833245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:24.833253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:24.833282Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2de6754d-a0b663eb-d620c6-174af57a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:24.833292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.833296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.833298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:24.833309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.833320Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:24.833338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:24.837411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:24.837426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:24.837464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:24.837618Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 129 partition: 0 MaxSeqNo: 25 sessionId: src_id|2de6754d-a0b663eb-d620c6-174af57a_0
2025-10-09T21:43:24.838114Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aef170b3-43f495eb-3da41dae-6c8af0c6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2de6754d-a0b663eb-d620c6-174af57a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:24.840773Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2de6754d-a0b663eb-d620c6-174af57a_0] Write session: try to update token
2025-10-09T21:43:24.840791Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2de6754d-a0b663eb-d620c6-174af57a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:24.852086Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:24.852104Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:24.852107Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:24.852109Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:24.852118Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340886516566331:3088]
2025-10-09T21:43:24.852126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340886516566331:3088]
2025-10-09T21:43:24.852130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340886516566331:3088]
2025-10-09T21:43:24.852181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:24.852181Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:24.852188Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:24.852188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:24.852191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:24.852191Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:24.852331Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:24.879329Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340886878375505:3142]
2025-10-09T21:43:24.879673Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 128, pipe: [1:7559340886878375505:3142]
2025-10-09T21:43:24.882184Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 129 sessionId: src_id|2de6754d-a0b663eb-d620c6-174af57a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:24.884796Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:25.023447Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:25.023839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340886878375505:3142], now have 1 active actors on pipe
2025-10-09T21:43:25.023869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:25.023876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:25.024366Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340886878375505:3142]
2025-10-09T21:43:25.024399Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:25.024433Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:25.024437Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:25.024499Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 129 reason: error at write: Disconnected sessionId: src_id|2de6754d-a0b663eb-d620c6-174af57a_0
2025-10-09T21:43:25.024353Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:25.025091Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2de6754d-a0b663eb-d620c6-174af57a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:25.025102Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2de6754d-a0b663eb-d620c6-174af57a_0] Write session will restart in 0.006916s
2025-10-09T21:43:25.025127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2de6754d-a0b663eb-d620c6-174af57a_0] Write session: Do CDS request
2025-10-09T21:43:25.025137Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2de6754d-a0b663eb-d620c6-174af57a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:25.024701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 129 sessionId: src_id|2de6754d-a0b663eb-d620c6-174af57a_0 is DEAD
2025-10-09T21:43:25.038496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2de6754d-a0b663eb-d620c6-174af57a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:25.038983Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:25.039000Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 130
2025-10-09T21:43:25.039182Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 130 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:25.039216Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 130 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:25.039223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=130 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:25.039226Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:25.039658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 130 sessionId:  describe result for acl check
2025-10-09T21:43:25.039701Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:25.039703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:25.039705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:25.039712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340891173342817:3313] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:25.040308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340891173342817:3313] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:25.048255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340891173342817:3313] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:25.064702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340891173342817:3313] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:25.064724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340891173342817:3313] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:25.064729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340891173342817:3313] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:25.064740Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 130 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:25.095558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:25.097365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:25.097581Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:25.097593Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:25.097606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:25.097669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:25.097717Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340890811533818:3091]
2025-10-09T21:43:25.098412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:25.100050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:25.100119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:25.100491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:25.100548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:25.100554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:25.100556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:25.100557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:25.100559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:25.100560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:25.100562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.100563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.100564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.100571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:25.100584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:25.100589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:25.100594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:25.100603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:25.100610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:25.100619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.100624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.100628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.100643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:25.100652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:25.100659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:25.100666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:25.100671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:25.100678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:25.100683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:25.100690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:25.100695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:25.100702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:25.100707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:25.100713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:25.100722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.100732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.100738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.100744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.100750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.100755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.100793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:25.117936Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 129
2025-10-09T21:43:25.117970Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 129
2025-10-09T21:43:25.119546Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 130 partition: 0 MaxSeqNo: 25 sessionId: src_id|3bd16cbc-d1c5626c-a054785b-3d87cab4_0
2025-10-09T21:43:25.119933Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2de6754d-a0b663eb-d620c6-174af57a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3bd16cbc-d1c5626c-a054785b-3d87cab4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:25.115759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:25.117200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:25.117208Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:25.117211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:25.117216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:25.122926Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3bd16cbc-d1c5626c-a054785b-3d87cab4_0] Write session: try to update token
2025-10-09T21:43:25.117224Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 129 [2:7559340890811533818:3091]
2025-10-09T21:43:25.117241Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:25.117261Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:25.117265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.122950Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3bd16cbc-d1c5626c-a054785b-3d87cab4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:25.117268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.117271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.117292Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:25.117298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:25.117306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:25.117372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.117375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.117378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.117381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:25.117470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:25.117473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.117488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.117501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.117521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:25.117547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340890811533812:3091]
2025-10-09T21:43:25.117550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340890811533812:3091]
2025-10-09T21:43:25.117552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340890811533812:3091]
2025-10-09T21:43:25.117562Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:25.117578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:25.117578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:25.117584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:25.117587Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:25.117595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:25.117598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:25.117611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340891173342873:3313], now have 1 active actors on pipe
2025-10-09T21:43:25.117615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340891173342806:2432], now have 1 active actors on pipe
2025-10-09T21:43:25.117658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340890811533812:3091]
2025-10-09T21:43:25.117673Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:25.118064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:25.118811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:25.118821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:25.118861Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3bd16cbc-d1c5626c-a054785b-3d87cab4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:25.118886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.118891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.118894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:25.118908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.118923Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:25.118944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:25.119370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:25.119377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:25.119400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:25.125546Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340891173342992:3142]
2025-10-09T21:43:25.127931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340891173342992:3142], now have 1 active actors on pipe
2025-10-09T21:43:25.128065Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 129, pipe: [1:7559340891173342992:3142]
2025-10-09T21:43:25.157218Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:25.157237Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:25.157241Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:25.157243Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:25.157255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340890811533812:3091]
2025-10-09T21:43:25.157261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340890811533812:3091]
2025-10-09T21:43:25.157266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340890811533812:3091]
2025-10-09T21:43:25.157316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:25.157324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.157327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:25.157467Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:25.163015Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:25.163030Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:25.163035Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:25.167582Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 130 sessionId: src_id|3bd16cbc-d1c5626c-a054785b-3d87cab4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:25.170626Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:25.401787Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:25.402331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:25.402350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:25.402713Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:25.403100Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340891173342992:3142]
2025-10-09T21:43:25.403137Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:25.403192Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:25.403211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 130 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3bd16cbc-d1c5626c-a054785b-3d87cab4_0
2025-10-09T21:43:25.403565Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3bd16cbc-d1c5626c-a054785b-3d87cab4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:25.403574Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3bd16cbc-d1c5626c-a054785b-3d87cab4_0] Write session will restart in 0.007786s
2025-10-09T21:43:25.403598Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3bd16cbc-d1c5626c-a054785b-3d87cab4_0] Write session: Do CDS request
2025-10-09T21:43:25.403605Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3bd16cbc-d1c5626c-a054785b-3d87cab4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:25.403399Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 130 sessionId: src_id|3bd16cbc-d1c5626c-a054785b-3d87cab4_0 is DEAD
2025-10-09T21:43:25.413001Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3bd16cbc-d1c5626c-a054785b-3d87cab4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:25.414894Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:25.414913Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 131
2025-10-09T21:43:25.415899Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 131 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:25.415957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 131 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:25.415969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=131 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:25.415973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:25.416394Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 131 sessionId:  describe result for acl check
2025-10-09T21:43:25.416437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:25.416439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:25.416440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:25.416447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340891173343062:3325] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:25.416996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340891173343062:3325] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:25.423125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340891173343062:3325] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:25.425326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:25.426508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:25.426740Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:25.426763Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:25.426777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:25.426840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:25.426888Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340890811534039:3105]
2025-10-09T21:43:25.427409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:25.427754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:25.427816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:25.428154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:25.428211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:25.428218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:25.428219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:25.428229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:25.428236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:25.428237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:25.428238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.428239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.428241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.428246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:25.428256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:25.428260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:25.428264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:25.428267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:25.428271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:25.428274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.428277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.428281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.428288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:25.428295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:25.428301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:25.428306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:25.428311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:25.428315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:25.428319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:25.428324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:25.428330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:25.428335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:25.428339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:25.428344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:25.428348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.428353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.428357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.428361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.428366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.428371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.428398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:25.429350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340891173343062:3325] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:25.429366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340891173343062:3325] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:25.429371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340891173343062:3325] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:25.429381Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 131 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:25.435265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:25.436588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:25.436604Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:25.436606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:25.436610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:25.436617Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 130 [2:7559340890811534039:3105]
2025-10-09T21:43:25.436632Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:25.436653Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:25.436657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.436660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.436662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.436681Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:25.436695Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:25.436702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:25.436766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.436775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.436777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.436779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:25.436863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:25.436870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.436882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.436893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.436915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:25.436936Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340890811534033:3105]
2025-10-09T21:43:25.436938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340890811534033:3105]
2025-10-09T21:43:25.436940Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340890811534033:3105]
2025-10-09T21:43:25.436949Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:25.437167Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 130
2025-10-09T21:43:25.437197Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 130
2025-10-09T21:43:25.436968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340891173343209:3325], now have 1 active actors on pipe
2025-10-09T21:43:25.436971Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:25.436974Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:25.436976Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:25.436978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340891173343024:2432], now have 1 active actors on pipe
2025-10-09T21:43:25.436984Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:25.437001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:25.437004Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:25.437040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340890811534033:3105]
2025-10-09T21:43:25.437050Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:25.437068Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:25.437259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:25.437268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:25.437305Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|93f62568-536bef97-17a0a0c0-f1816ab4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:25.437315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.437319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.437332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:25.437345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.437363Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:25.437395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:25.437854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:25.437866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:25.437886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:25.438047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 131 partition: 0 MaxSeqNo: 25 sessionId: src_id|93f62568-536bef97-17a0a0c0-f1816ab4_0
2025-10-09T21:43:25.438537Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3bd16cbc-d1c5626c-a054785b-3d87cab4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|93f62568-536bef97-17a0a0c0-f1816ab4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:25.441096Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|93f62568-536bef97-17a0a0c0-f1816ab4_0] Write session: try to update token
2025-10-09T21:43:25.441115Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|93f62568-536bef97-17a0a0c0-f1816ab4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:25.455289Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:25.455305Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:25.455308Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:25.455310Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:25.455323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340890811534033:3105]
2025-10-09T21:43:25.455327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340890811534033:3105]
2025-10-09T21:43:25.455331Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340890811534033:3105]
2025-10-09T21:43:25.455380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:25.455384Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:25.455386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.455389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:25.455399Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:25.455403Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:25.455543Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:25.490368Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 131 sessionId: src_id|93f62568-536bef97-17a0a0c0-f1816ab4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:25.492926Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:25.506199Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340891173343220:3142]
2025-10-09T21:43:25.506911Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 130, pipe: [1:7559340891173343220:3142]
2025-10-09T21:43:25.639969Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:25.640331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:25.640341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:25.640363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340891173343220:3142], now have 1 active actors on pipe
2025-10-09T21:43:25.640583Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:25.641671Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340891173343220:3142]
2025-10-09T21:43:25.641714Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:25.641794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 131 reason: status is not ok: tablet will be restarted right now sessionId: src_id|93f62568-536bef97-17a0a0c0-f1816ab4_0
2025-10-09T21:43:25.641988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 131 sessionId: src_id|93f62568-536bef97-17a0a0c0-f1816ab4_0 is DEAD
2025-10-09T21:43:25.642097Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:25.642313Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93f62568-536bef97-17a0a0c0-f1816ab4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:25.642325Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93f62568-536bef97-17a0a0c0-f1816ab4_0] Write session will restart in 0.007584s
2025-10-09T21:43:25.642353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93f62568-536bef97-17a0a0c0-f1816ab4_0] Write session: Do CDS request
2025-10-09T21:43:25.642360Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93f62568-536bef97-17a0a0c0-f1816ab4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:25.653538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|93f62568-536bef97-17a0a0c0-f1816ab4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:25.653648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:25.653666Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 132
2025-10-09T21:43:25.653799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 132 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:25.653836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 132 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:25.653851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=132 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:25.653859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:25.654403Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 132 sessionId:  describe result for acl check
2025-10-09T21:43:25.654451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:25.654453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:25.654455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:25.654460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340891173343237:3329] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:25.655010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340891173343237:3329] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:25.659616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340891173343237:3329] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:25.667382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340891173343237:3329] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:25.667400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340891173343237:3329] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:25.667403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340891173343237:3329] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:25.667412Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 132 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:25.678455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:25.682223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:25.682563Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:25.682568Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:25.682582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:25.682656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:25.682697Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340890811534226:3107]
2025-10-09T21:43:25.683323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:25.694360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:25.694481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:25.694880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:25.694962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:25.694964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:25.694965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:25.694967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:25.694968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:25.694969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:25.694971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.694972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.694973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.694981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:25.694992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:25.694996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:25.695000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:25.695004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:25.695008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:25.695011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.695015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.695018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.695027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:25.695035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:25.695042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:25.695047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:25.695052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:25.695057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:25.695061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:25.695066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:25.695091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:25.695097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:25.695101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:25.695107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:25.695112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.695117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.695121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.695126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.695130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.695135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.695162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:25.708902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:25.710203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:25.710227Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:25.710229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:25.710234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:25.710242Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 131 [2:7559340890811534226:3107]
2025-10-09T21:43:25.710260Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:25.710280Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:25.710283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.710287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.710289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.710320Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:25.710326Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:25.710333Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:25.710408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.710410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.710411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.710426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:25.710503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:25.710505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.710518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:25.710530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:25.710547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:25.710567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340890811534221:3107]
2025-10-09T21:43:25.710569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340890811534221:3107]
2025-10-09T21:43:25.710571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340890811534221:3107]
2025-10-09T21:43:25.710584Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:25.710598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:25.710612Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:25.710615Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:25.710654Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:25.710658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340890811534221:3107]
2025-10-09T21:43:25.710659Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:25.710660Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:25.710661Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:25.710668Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:25.710913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340891173343223:2432], now have 1 active actors on pipe
2025-10-09T21:43:25.710919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340891173343348:3329], now have 1 active actors on pipe
2025-10-09T21:43:25.710979Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 131
2025-10-09T21:43:25.711023Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 131
2025-10-09T21:43:25.711654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:25.711667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:25.711699Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|25428c43-591910f9-11503d9b-495f0ba9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:25.711709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.711714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.711716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:25.711728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.711746Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:25.711766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:25.712613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:25.712636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:25.712658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:25.713311Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 132 partition: 0 MaxSeqNo: 25 sessionId: src_id|25428c43-591910f9-11503d9b-495f0ba9_0
2025-10-09T21:43:25.713816Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93f62568-536bef97-17a0a0c0-f1816ab4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|25428c43-591910f9-11503d9b-495f0ba9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:25.716361Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25428c43-591910f9-11503d9b-495f0ba9_0] Write session: try to update token
2025-10-09T21:43:25.716382Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25428c43-591910f9-11503d9b-495f0ba9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:25.731674Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:25.731692Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:25.731696Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:25.731698Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:25.731710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340890811534221:3107]
2025-10-09T21:43:25.731714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340890811534221:3107]
2025-10-09T21:43:25.731721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340890811534221:3107]
2025-10-09T21:43:25.731768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:25.731774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:25.731777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:25.731966Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:25.732485Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:25.732491Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:25.732495Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:25.742101Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340891173343420:3142]
2025-10-09T21:43:25.742548Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 131, pipe: [1:7559340891173343420:3142]
2025-10-09T21:43:25.748631Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 132 sessionId: src_id|25428c43-591910f9-11503d9b-495f0ba9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:25.751469Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:25.944920Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:25.945244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340891173343420:3142], now have 1 active actors on pipe
2025-10-09T21:43:25.945281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:25.945293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:25.945568Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:25.947065Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25428c43-591910f9-11503d9b-495f0ba9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:25.947078Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25428c43-591910f9-11503d9b-495f0ba9_0] Write session will restart in 0.005962s
2025-10-09T21:43:25.947103Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25428c43-591910f9-11503d9b-495f0ba9_0] Write session: Do CDS request
2025-10-09T21:43:25.947111Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25428c43-591910f9-11503d9b-495f0ba9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:25.946441Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:25.946573Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 132 reason: status is not ok: tablet will be restarted right now sessionId: src_id|25428c43-591910f9-11503d9b-495f0ba9_0
2025-10-09T21:43:25.946706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 132 sessionId: src_id|25428c43-591910f9-11503d9b-495f0ba9_0 is DEAD
2025-10-09T21:43:25.946819Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:25.946844Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340891173343420:3142]
2025-10-09T21:43:25.954894Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25428c43-591910f9-11503d9b-495f0ba9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:25.955550Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:25.955567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 133
2025-10-09T21:43:25.955844Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 133 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:25.955891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 133 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:25.955896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=133 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:25.955901Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:25.958603Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 133 sessionId:  describe result for acl check
2025-10-09T21:43:25.958649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:25.958651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:25.958652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:25.958659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340891173343443:3334] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:25.959249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340891173343443:3334] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:25.968729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340891173343443:3334] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:25.975336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340891173343443:3334] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:25.975352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340891173343443:3334] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:25.975356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340891173343443:3334] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:25.975366Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 133 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:26.032722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:26.034362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:26.034728Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:26.034733Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:26.034748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:26.034824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:26.034866Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340895106501712:3111]
2025-10-09T21:43:26.035505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:26.035994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:26.036043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:26.036369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:26.036419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:26.036420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:26.036422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:26.036423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:26.036424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:26.036426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:26.036427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.036428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.036429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.036435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:26.036448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:26.036452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:26.036456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:26.036461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:26.036465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:26.036470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.036474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.036478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.036486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:26.036495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:26.036501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:26.036506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:26.036510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:26.036515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:26.036520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:26.036526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:26.036531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:26.036537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:26.036542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:26.036547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:26.036552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.036558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.036562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.036568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.036573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.036578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.036612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:26.046493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:26.047863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:26.047871Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:26.047873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:26.047879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:26.047888Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 132 [2:7559340895106501712:3111]
2025-10-09T21:43:26.047905Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:26.047928Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:26.047932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.047935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.047938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.047957Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:26.047964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:26.047972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:26.048040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.048042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.048044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.048048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:26.048134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:26.048137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.048153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.048167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.048185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:26.048208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340895106501706:3111]
2025-10-09T21:43:26.048210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340895106501706:3111]
2025-10-09T21:43:26.048215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340895106501706:3111]
2025-10-09T21:43:26.048230Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:26.048245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:26.048523Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340895468310915:3142]
2025-10-09T21:43:26.049464Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 132
2025-10-09T21:43:26.049504Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 132
2025-10-09T21:43:26.048261Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:26.048264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:26.048308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340895106501706:3111]
2025-10-09T21:43:26.048311Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:26.048690Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:26.048692Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:26.048694Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:26.048701Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:26.053104Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 132, pipe: [1:7559340895468310915:3142]
2025-10-09T21:43:26.058503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340891173343431:2432], now have 1 active actors on pipe
2025-10-09T21:43:26.058525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340891173343481:3334], now have 1 active actors on pipe
2025-10-09T21:43:26.058570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:26.058580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:26.058600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340895468310915:3142], now have 1 active actors on pipe
2025-10-09T21:43:26.058630Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c855c163-b15458ed-d9661f7c-9d6f056d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:26.058645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.058651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.058655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:26.058675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.058695Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:26.058718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:26.059610Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 133 partition: 0 MaxSeqNo: 25 sessionId: src_id|c855c163-b15458ed-d9661f7c-9d6f056d_0
2025-10-09T21:43:26.060115Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25428c43-591910f9-11503d9b-495f0ba9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c855c163-b15458ed-d9661f7c-9d6f056d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:26.062820Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c855c163-b15458ed-d9661f7c-9d6f056d_0] Write session: try to update token
2025-10-09T21:43:26.062844Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c855c163-b15458ed-d9661f7c-9d6f056d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:26.059438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:26.059451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:26.059475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:26.112759Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 133 sessionId: src_id|c855c163-b15458ed-d9661f7c-9d6f056d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:26.115544Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:26.123904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:26.123925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:26.123952Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:26.123956Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:26.123966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340895106501706:3111]
2025-10-09T21:43:26.124033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:26.124231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:26.124377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:26.124520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:26.124666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:26.124907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:26.124949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:26.124974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:26.125486Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:26.125326Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:26.137494Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:26.137514Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:26.137517Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:26.137519Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:26.137532Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340895106501706:3111]
2025-10-09T21:43:26.137537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340895106501706:3111]
2025-10-09T21:43:26.137541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340895106501706:3111]
2025-10-09T21:43:26.137590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:26.137598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.137601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:26.137741Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:26.140201Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:26.140211Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:26.140216Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:26.561925Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:26.575103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:26.575135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:26.575308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:26.575370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:26.575434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:26.575500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:26.576063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:26.576079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:26.576157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:26.576221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:26.576316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:26.576392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:26.576441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:26.576450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:26.576525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:26.576593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:26.576662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:26.576730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:26.576772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:26.576782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:26.576856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:26.576925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:26.576996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:26.577060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:26.577104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:26.577113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:26.577200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:26.577266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:26.577337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:26.577404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:26.577453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:26.577461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:26.577539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:26.577610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:26.577691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:26.577781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:26.577832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:26.577835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:26.577942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:26.578002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:26.582124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:26.582203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:26.582245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:26.582257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:26.582331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:26.582396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:26.582480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:26.582548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:26.582596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:26.582600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:26.582678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:26.582740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:26.582808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:26.582876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:26.582947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:26.582950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:26.583019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:26.583079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:26.583147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:26.583211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:26.583254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:26.583257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:26.585226Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c855c163-b15458ed-d9661f7c-9d6f056d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:26.584506Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340895468310915:3142]
2025-10-09T21:43:26.585238Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c855c163-b15458ed-d9661f7c-9d6f056d_0] Write session will restart in 0.008724s
2025-10-09T21:43:26.584537Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:26.584802Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:26.584807Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:26.584837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 133 reason: error at write: Disconnected sessionId: src_id|c855c163-b15458ed-d9661f7c-9d6f056d_0
2025-10-09T21:43:26.585041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 133 sessionId: src_id|c855c163-b15458ed-d9661f7c-9d6f056d_0 is DEAD
2025-10-09T21:43:26.585282Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c855c163-b15458ed-d9661f7c-9d6f056d_0] Write session: Do CDS request
2025-10-09T21:43:26.585289Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c855c163-b15458ed-d9661f7c-9d6f056d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:26.585751Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:26.595765Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c855c163-b15458ed-d9661f7c-9d6f056d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:26.596103Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:26.596118Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 134
2025-10-09T21:43:26.596314Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 134 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:26.596359Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 134 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:26.596365Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=134 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:26.596369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:26.596899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 134 sessionId:  describe result for acl check
2025-10-09T21:43:26.596941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:26.596943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:26.596945Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:26.596951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340895468310980:3346] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:26.597518Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340895468310980:3346] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:26.604594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340895468310980:3346] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:26.614247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340895468310980:3346] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:26.614265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340895468310980:3346] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:26.614270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340895468310980:3346] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:26.614279Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 134 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:26.622680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:26.624187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:26.624416Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:26.624429Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:26.624443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:26.624515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:26.624559Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340895106501938:3125]
2025-10-09T21:43:26.625179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:26.625540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:26.625597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:26.625937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:26.626011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:26.626017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:26.626019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:26.626020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:26.626022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:26.626023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:26.626024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.626026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.626041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.626049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:26.626060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:26.626065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:26.626068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:26.626072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:26.626076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:26.626080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.626084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.626087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.626096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:26.626103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:26.626110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:26.626116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:26.626120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:26.626131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:26.626135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:26.626141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:26.626152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:26.626157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:26.626162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:26.626167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:26.626172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.626178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.626182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.626188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.626202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.626208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.626240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:26.636368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:26.637879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:26.637896Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:26.637899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:26.637903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:26.637911Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 133 [2:7559340895106501938:3125]
2025-10-09T21:43:26.637925Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:26.637946Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:26.637950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.637954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.637956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.637976Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:26.637982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:26.637997Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:26.638101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.638103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.638106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.638109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:26.638191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:26.638195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.638208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.638220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.638239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:26.638262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340895106501932:3125]
2025-10-09T21:43:26.638265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340895106501932:3125]
2025-10-09T21:43:26.638267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340895106501932:3125]
2025-10-09T21:43:26.638276Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:26.638285Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:26.638288Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:26.638291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:26.638293Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:26.638307Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:26.638311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:26.638323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340895468311098:3346], now have 1 active actors on pipe
2025-10-09T21:43:26.638327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340895468310955:2432], now have 1 active actors on pipe
2025-10-09T21:43:26.638367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340895106501932:3125]
2025-10-09T21:43:26.638375Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:26.638480Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:26.638634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 133
2025-10-09T21:43:26.638667Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 133
2025-10-09T21:43:26.638899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:26.638910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:26.638947Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4952beb3-a2ceafad-5d9c8125-b90d1e6e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:26.638957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.638962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.638965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:26.638976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.638988Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:26.639006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:26.640249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:26.640259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:26.640282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:26.641107Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 134 partition: 0 MaxSeqNo: 25 sessionId: src_id|4952beb3-a2ceafad-5d9c8125-b90d1e6e_0
2025-10-09T21:43:26.644508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c855c163-b15458ed-d9661f7c-9d6f056d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4952beb3-a2ceafad-5d9c8125-b90d1e6e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:26.647930Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4952beb3-a2ceafad-5d9c8125-b90d1e6e_0] Write session: try to update token
2025-10-09T21:43:26.647966Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4952beb3-a2ceafad-5d9c8125-b90d1e6e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:26.661916Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:26.661933Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:26.661936Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:26.661938Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:26.661948Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340895106501932:3125]
2025-10-09T21:43:26.661952Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340895106501932:3125]
2025-10-09T21:43:26.661956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340895106501932:3125]
2025-10-09T21:43:26.662004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:26.662012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.662014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:26.662240Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:26.666661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:26.666676Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:26.666680Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:26.685062Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340895468311152:3142]
2025-10-09T21:43:26.690182Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 133, pipe: [1:7559340895468311152:3142]
2025-10-09T21:43:26.691062Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 134 sessionId: src_id|4952beb3-a2ceafad-5d9c8125-b90d1e6e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:26.693724Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:26.865204Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:26.865227Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:26.864163Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:26.864431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340895468311152:3142], now have 1 active actors on pipe
2025-10-09T21:43:26.864484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:26.864491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:26.866244Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340895468311152:3142]
2025-10-09T21:43:26.866274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:26.866392Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 134 reason: error at write: Disconnected sessionId: src_id|4952beb3-a2ceafad-5d9c8125-b90d1e6e_0
2025-10-09T21:43:26.866995Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4952beb3-a2ceafad-5d9c8125-b90d1e6e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:26.867017Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4952beb3-a2ceafad-5d9c8125-b90d1e6e_0] Write session will restart in 0.008043s
2025-10-09T21:43:26.867039Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4952beb3-a2ceafad-5d9c8125-b90d1e6e_0] Write session: Do CDS request
2025-10-09T21:43:26.867046Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4952beb3-a2ceafad-5d9c8125-b90d1e6e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:26.866602Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 134 sessionId: src_id|4952beb3-a2ceafad-5d9c8125-b90d1e6e_0 is DEAD
2025-10-09T21:43:26.869503Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:26.877270Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4952beb3-a2ceafad-5d9c8125-b90d1e6e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:26.877420Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:26.877435Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 135
2025-10-09T21:43:26.879527Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 135 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:26.879574Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 135 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:26.879598Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=135 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:26.879606Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:26.880233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 135 sessionId:  describe result for acl check
2025-10-09T21:43:26.880280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:26.880282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:26.880284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:26.880290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340895468311186:3353] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:26.880884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340895468311186:3353] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:26.886550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340895468311186:3353] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:26.903042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340895468311186:3353] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:26.903063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340895468311186:3353] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:26.903067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340895468311186:3353] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:26.903078Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 135 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:26.903567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:26.905899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:26.906139Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:26.906144Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:26.906157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:26.906214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:26.906253Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340895106502132:3128]
2025-10-09T21:43:26.906913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:26.907555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:26.907600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:26.908286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:26.908368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:26.908369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:26.908370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:26.908371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:26.908373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:26.908374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:26.908375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.908376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.908378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.908384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:26.908395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:26.908399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:26.908403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:26.908407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:26.908411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:26.908415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.908419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.908423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.908431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:26.908439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:26.908445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:26.908451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:26.908457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:26.908462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:26.908466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:26.908472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:26.908477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:26.908482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:26.908486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:26.908491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:26.908496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.908501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.908506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.908511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.908516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.908522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.908551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:26.920279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:26.921488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:26.921495Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:26.921498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:26.921503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:26.921512Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 134 [2:7559340895106502132:3128]
2025-10-09T21:43:26.921528Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:26.921549Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:26.921552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.921555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.921558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.921577Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:26.921584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:26.921591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:26.921661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.921663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.921665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.921668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:26.921745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:26.921748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.921762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:26.921775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:26.921790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:26.921812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340895106502125:3128]
2025-10-09T21:43:26.921814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340895106502125:3128]
2025-10-09T21:43:26.921816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340895106502125:3128]
2025-10-09T21:43:26.921831Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:26.921847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:26.921863Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:26.921867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:26.921908Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340895106502125:3128]
2025-10-09T21:43:26.921910Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:26.922248Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:26.922254Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:26.922256Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:26.922262Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:26.924978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340895468311349:3353], now have 1 active actors on pipe
2025-10-09T21:43:26.924990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340895468311166:2432], now have 1 active actors on pipe
2025-10-09T21:43:26.925010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:26.925017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:26.925069Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d6925b56-f8db7581-b59f94d5-dfffbe8f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:26.925080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.925086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.925089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:26.925102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.925127Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:26.925150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:26.922627Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 134
2025-10-09T21:43:26.922672Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 134
2025-10-09T21:43:26.928042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:26.928060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:26.928110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:26.928446Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 135 partition: 0 MaxSeqNo: 25 sessionId: src_id|d6925b56-f8db7581-b59f94d5-dfffbe8f_0
2025-10-09T21:43:26.930001Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4952beb3-a2ceafad-5d9c8125-b90d1e6e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d6925b56-f8db7581-b59f94d5-dfffbe8f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:26.933331Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d6925b56-f8db7581-b59f94d5-dfffbe8f_0] Write session: try to update token
2025-10-09T21:43:26.933355Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d6925b56-f8db7581-b59f94d5-dfffbe8f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:26.943237Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:26.943254Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:26.943265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340895106502125:3128]
2025-10-09T21:43:26.943319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:26.943509Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:26.943662Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:26.943807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:26.943978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:26.944312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:26.944369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:26.945271Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:26.964017Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:26.964037Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:26.964040Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:26.964042Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:26.964054Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340895106502125:3128]
2025-10-09T21:43:26.964059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340895106502125:3128]
2025-10-09T21:43:26.964063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340895106502125:3128]
2025-10-09T21:43:26.964116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:26.964124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:26.964127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:26.964201Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:26.964211Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:26.964214Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:26.964279Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:26.970722Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340895468311361:3142]
2025-10-09T21:43:26.971130Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 134, pipe: [1:7559340895468311361:3142]
2025-10-09T21:43:26.981059Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 135 sessionId: src_id|d6925b56-f8db7581-b59f94d5-dfffbe8f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:26.984024Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:27.234706Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:27.235622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340895468311361:3142], now have 1 active actors on pipe
2025-10-09T21:43:27.235665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:27.235672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:27.238890Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:27.239036Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:27.239049Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:27.239118Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 135 reason: error at write: Disconnected sessionId: src_id|d6925b56-f8db7581-b59f94d5-dfffbe8f_0
2025-10-09T21:43:27.239307Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 135 sessionId: src_id|d6925b56-f8db7581-b59f94d5-dfffbe8f_0 is DEAD
2025-10-09T21:43:27.239446Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340895468311361:3142]
2025-10-09T21:43:27.239460Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:27.240671Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6925b56-f8db7581-b59f94d5-dfffbe8f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:27.240700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6925b56-f8db7581-b59f94d5-dfffbe8f_0] Write session will restart in 0.006729s
2025-10-09T21:43:27.240738Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6925b56-f8db7581-b59f94d5-dfffbe8f_0] Write session: Do CDS request
2025-10-09T21:43:27.240746Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6925b56-f8db7581-b59f94d5-dfffbe8f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:27.248649Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d6925b56-f8db7581-b59f94d5-dfffbe8f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:27.249060Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:27.249084Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 136
2025-10-09T21:43:27.249340Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 136 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:27.249398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 136 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:27.249409Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=136 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:27.249414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:27.250374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 136 sessionId:  describe result for acl check
2025-10-09T21:43:27.250438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:27.250445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:27.250448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:27.250455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340899763278669:3357] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:27.251169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340899763278669:3357] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:27.255868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340899763278669:3357] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:27.265220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340899763278669:3357] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:27.265239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340899763278669:3357] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:27.265243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340899763278669:3357] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:27.265254Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 136 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:27.277249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:27.279111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:27.279438Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:27.279452Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:27.279468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:27.279542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:27.279585Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340899401469618:3131]
2025-10-09T21:43:27.280167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:27.280927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:27.280988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:27.281336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:27.281405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:27.281406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:27.281407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:27.281408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:27.281409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:27.281410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:27.281412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.281413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.281414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.281420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:27.281432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:27.281437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:27.281441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:27.281445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:27.281449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:27.281452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.281455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.281459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.281467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:27.281477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:27.281483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:27.281488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:27.281494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:27.281500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:27.281504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:27.281510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:27.281514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:27.281519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:27.281523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:27.281528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:27.281534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.281539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.281543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.281549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.281554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.281559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.281589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:27.289516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:27.290747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:27.290761Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:27.290764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:27.290770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:27.290780Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 135 [2:7559340899401469618:3131]
2025-10-09T21:43:27.290799Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:27.290820Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:27.290823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.290826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.290829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.290849Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:27.290861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:27.290868Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:27.290937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.290945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.290947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.290950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:27.291027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:27.291034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.291049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.291063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.291084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:27.291110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340899763278765:3357], now have 1 active actors on pipe
2025-10-09T21:43:27.291114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340899763278662:2432], now have 1 active actors on pipe
2025-10-09T21:43:27.291123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340899401469613:3131]
2025-10-09T21:43:27.291126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340899401469613:3131]
2025-10-09T21:43:27.291128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340899401469613:3131]
2025-10-09T21:43:27.291145Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:27.291211Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 135
2025-10-09T21:43:27.291221Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 135
2025-10-09T21:43:27.291160Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:27.291182Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:27.291186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:27.291227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340899401469613:3131]
2025-10-09T21:43:27.291230Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:27.291315Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:27.291318Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:27.291320Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:27.291327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:27.291898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:27.291911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:27.291950Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|25b51461-d0d35772-5bfd775b-fafb54c0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:27.291962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.291967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.291970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:27.291992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.292005Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:27.292024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:27.292934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:27.292946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:27.292966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:27.293122Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 136 partition: 0 MaxSeqNo: 25 sessionId: src_id|25b51461-d0d35772-5bfd775b-fafb54c0_0
2025-10-09T21:43:27.293549Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6925b56-f8db7581-b59f94d5-dfffbe8f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|25b51461-d0d35772-5bfd775b-fafb54c0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:27.296721Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25b51461-d0d35772-5bfd775b-fafb54c0_0] Write session: try to update token
2025-10-09T21:43:27.296750Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25b51461-d0d35772-5bfd775b-fafb54c0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:27.310290Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:27.310307Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:27.310310Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:27.310312Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:27.310323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340899401469613:3131]
2025-10-09T21:43:27.310328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340899401469613:3131]
2025-10-09T21:43:27.310332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340899401469613:3131]
2025-10-09T21:43:27.310382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:27.310383Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:27.310388Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:27.310389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.310392Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:27.310392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:27.310541Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:27.327861Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 136 sessionId: src_id|25b51461-d0d35772-5bfd775b-fafb54c0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:27.331141Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:27.338547Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340899763278859:3142]
2025-10-09T21:43:27.339164Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 135, pipe: [1:7559340899763278859:3142]
2025-10-09T21:43:27.498162Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:27.498554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:27.498575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:27.498600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340899763278859:3142], now have 1 active actors on pipe
2025-10-09T21:43:27.498839Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:27.500311Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:27.500380Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:27.500891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25b51461-d0d35772-5bfd775b-fafb54c0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:27.500902Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25b51461-d0d35772-5bfd775b-fafb54c0_0] Write session will restart in 0.007944s
2025-10-09T21:43:27.500934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25b51461-d0d35772-5bfd775b-fafb54c0_0] Write session: Do CDS request
2025-10-09T21:43:27.500941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25b51461-d0d35772-5bfd775b-fafb54c0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:27.500451Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 136 reason: status is not ok: tablet will be restarted right now sessionId: src_id|25b51461-d0d35772-5bfd775b-fafb54c0_0
2025-10-09T21:43:27.500651Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 136 sessionId: src_id|25b51461-d0d35772-5bfd775b-fafb54c0_0 is DEAD
2025-10-09T21:43:27.500825Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340899763278859:3142]
2025-10-09T21:43:27.512475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25b51461-d0d35772-5bfd775b-fafb54c0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:27.513258Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:27.513276Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 137
2025-10-09T21:43:27.513471Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 137 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:27.513510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 137 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:27.513515Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=137 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:27.513530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:27.514024Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 137 sessionId:  describe result for acl check
2025-10-09T21:43:27.514073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:27.514077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:27.514078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:27.514085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340899763278946:3368] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:27.514647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340899763278946:3368] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:27.525659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340899763278946:3368] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:27.534179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340899763278946:3368] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:27.534197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340899763278946:3368] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:27.534201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340899763278946:3368] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:27.534210Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 137 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:27.534026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:27.536716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:27.536911Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:27.536915Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:27.536926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:27.536972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:27.537008Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340899401469830:3139]
2025-10-09T21:43:27.537556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:27.538680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:27.538750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:27.539086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:27.539135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:27.539136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:27.539138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:27.539139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:27.539140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:27.539141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:27.539142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.539143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.539144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.539149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:27.539158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:27.539162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:27.539165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:27.539169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:27.539173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:27.539176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.539179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.539182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.539188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:27.539196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:27.539203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:27.539207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:27.539211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:27.539216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:27.539220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:27.539224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:27.539228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:27.539233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:27.539237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:27.539243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:27.539248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.539253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.539256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.539261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.539264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.539268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.539296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:27.548912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:27.550222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:27.550237Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:27.550240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:27.550244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:27.550251Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 136 [2:7559340899401469830:3139]
2025-10-09T21:43:27.550267Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:27.550285Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:27.550288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.550293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.550295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.550314Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:27.550320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:27.550328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:27.550398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.550405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.550407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.550410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:27.550495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:27.550503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.550517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.550528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.550546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:27.550571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340899401469815:3139]
2025-10-09T21:43:27.550574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340899401469815:3139]
2025-10-09T21:43:27.550576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340899401469815:3139]
2025-10-09T21:43:27.550589Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:27.550605Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:27.550624Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:27.550635Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:27.550675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340899401469815:3139]
2025-10-09T21:43:27.550682Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:27.550939Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:27.550948Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:27.550950Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:27.550956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:27.551213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340899763278879:2432], now have 1 active actors on pipe
2025-10-09T21:43:27.551224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340899763279062:3368], now have 1 active actors on pipe
2025-10-09T21:43:27.551392Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 136
2025-10-09T21:43:27.551441Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 136
2025-10-09T21:43:27.551975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:27.551987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:27.552017Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b8adfda5-a0480f28-58d5febc-352f4c1d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:27.552036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.552040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.552043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:27.552054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.552067Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:27.552090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:27.552403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:27.552412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:27.552439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:27.552598Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 137 partition: 0 MaxSeqNo: 25 sessionId: src_id|b8adfda5-a0480f28-58d5febc-352f4c1d_0
2025-10-09T21:43:27.553258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25b51461-d0d35772-5bfd775b-fafb54c0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b8adfda5-a0480f28-58d5febc-352f4c1d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:27.556287Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b8adfda5-a0480f28-58d5febc-352f4c1d_0] Write session: try to update token
2025-10-09T21:43:27.556309Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b8adfda5-a0480f28-58d5febc-352f4c1d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:27.573687Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:27.573704Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:27.573707Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:27.573712Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:27.573722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340899401469815:3139]
2025-10-09T21:43:27.573726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340899401469815:3139]
2025-10-09T21:43:27.573731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340899401469815:3139]
2025-10-09T21:43:27.573781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:27.573795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.573797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:27.573979Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:27.574219Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:27.574226Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:27.574230Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:27.585661Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 137 sessionId: src_id|b8adfda5-a0480f28-58d5febc-352f4c1d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:27.588542Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:27.601156Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340899763279083:3142]
2025-10-09T21:43:27.601636Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 136, pipe: [1:7559340899763279083:3142]
2025-10-09T21:43:27.740953Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:27.741272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:27.741283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:27.741307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340899763279083:3142], now have 1 active actors on pipe
2025-10-09T21:43:27.741773Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:27.741777Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:27.741788Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:27.741824Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340899763279083:3142]
2025-10-09T21:43:27.741845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 137 reason: error at write: Disconnected sessionId: src_id|b8adfda5-a0480f28-58d5febc-352f4c1d_0
2025-10-09T21:43:27.742231Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8adfda5-a0480f28-58d5febc-352f4c1d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:27.742058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 137 sessionId: src_id|b8adfda5-a0480f28-58d5febc-352f4c1d_0 is DEAD
2025-10-09T21:43:27.742255Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8adfda5-a0480f28-58d5febc-352f4c1d_0] Write session will restart in 0.005603s
2025-10-09T21:43:27.742380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8adfda5-a0480f28-58d5febc-352f4c1d_0] Write session: Do CDS request
2025-10-09T21:43:27.742393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8adfda5-a0480f28-58d5febc-352f4c1d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:27.742681Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:27.749677Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b8adfda5-a0480f28-58d5febc-352f4c1d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:27.749892Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:27.749907Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 138
2025-10-09T21:43:27.750060Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 138 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:27.750096Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 138 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:27.750102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=138 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:27.750105Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:27.750640Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 138 sessionId:  describe result for acl check
2025-10-09T21:43:27.750677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:27.750679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:27.750681Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:27.750688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340899763279100:3373] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:27.751273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340899763279100:3373] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:27.756409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340899763279100:3373] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:27.773465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340899763279100:3373] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:27.773483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340899763279100:3373] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:27.773487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340899763279100:3373] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:27.773498Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 138 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:27.781059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:27.782582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:27.782818Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:27.782823Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:27.782835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:27.782891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:27.782929Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340899401470041:3148]
2025-10-09T21:43:27.783555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:27.784119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:27.784163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:27.784486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:27.784529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:27.784530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:27.784530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:27.784531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:27.784532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:27.784532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:27.784533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.784533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.784534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.784538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:27.784545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:27.784547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:27.784550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:27.784552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:27.784554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:27.784556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.784558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.784560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.784566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:27.784572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:27.784576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:27.784580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:27.784582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:27.784585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:27.784588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:27.784591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:27.784593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:27.784596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:27.784599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:27.784602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:27.784605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.784608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.784610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.784613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.784617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.784620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.784636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:27.793223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:27.794756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:27.794770Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:27.794773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:27.794777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:27.794784Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 137 [2:7559340899401470041:3148]
2025-10-09T21:43:27.794799Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:27.794819Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:27.794822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.794825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.794828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.794847Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:27.794854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:27.794862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:27.794927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.794929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.794931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.794934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:27.795003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:27.795006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.795020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:27.795032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:27.795050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:27.795070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340899401470035:3148]
2025-10-09T21:43:27.795073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340899401470035:3148]
2025-10-09T21:43:27.795075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340899401470035:3148]
2025-10-09T21:43:27.795091Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:27.795107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:27.795120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:27.795123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:27.795130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:27.795132Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:27.795134Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:27.795171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340899401470035:3148]
2025-10-09T21:43:27.795174Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:27.795216Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:27.795463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340899763279219:3373], now have 1 active actors on pipe
2025-10-09T21:43:27.795468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340899763279086:2432], now have 1 active actors on pipe
2025-10-09T21:43:27.795766Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 137
2025-10-09T21:43:27.795796Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 137
2025-10-09T21:43:27.796276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:27.796287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:27.796320Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d11a461d-9b03b6d6-40dcd592-dfb04328_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:27.796331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.796335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.796338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:27.796349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.796361Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:27.796380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:27.796692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:27.796698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:27.796712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:27.796821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 138 partition: 0 MaxSeqNo: 25 sessionId: src_id|d11a461d-9b03b6d6-40dcd592-dfb04328_0
2025-10-09T21:43:27.797222Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8adfda5-a0480f28-58d5febc-352f4c1d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d11a461d-9b03b6d6-40dcd592-dfb04328_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:27.800176Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d11a461d-9b03b6d6-40dcd592-dfb04328_0] Write session: try to update token
2025-10-09T21:43:27.800191Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d11a461d-9b03b6d6-40dcd592-dfb04328_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:27.814159Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:27.814174Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:27.814177Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:27.814180Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:27.814191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340899401470035:3148]
2025-10-09T21:43:27.814196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340899401470035:3148]
2025-10-09T21:43:27.814201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340899401470035:3148]
2025-10-09T21:43:27.814248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:27.814257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:27.814259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:27.814395Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:27.818779Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:27.818794Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:27.818807Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:27.834232Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 138 sessionId: src_id|d11a461d-9b03b6d6-40dcd592-dfb04328_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:27.836884Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:27.844243Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340899763279292:3142]
2025-10-09T21:43:27.844814Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 137, pipe: [1:7559340899763279292:3142]
2025-10-09T21:43:28.016534Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:28.016872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:28.016890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:28.016914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340899763279292:3142], now have 1 active actors on pipe
2025-10-09T21:43:28.017112Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:28.017416Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340899763279292:3142]
2025-10-09T21:43:28.017440Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:28.017468Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:28.017519Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 138 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d11a461d-9b03b6d6-40dcd592-dfb04328_0
2025-10-09T21:43:28.017689Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 138 sessionId: src_id|d11a461d-9b03b6d6-40dcd592-dfb04328_0 is DEAD
2025-10-09T21:43:28.018076Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d11a461d-9b03b6d6-40dcd592-dfb04328_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:28.018091Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d11a461d-9b03b6d6-40dcd592-dfb04328_0] Write session will restart in 0.008938s
2025-10-09T21:43:28.018118Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d11a461d-9b03b6d6-40dcd592-dfb04328_0] Write session: Do CDS request
2025-10-09T21:43:28.018125Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d11a461d-9b03b6d6-40dcd592-dfb04328_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:28.028585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d11a461d-9b03b6d6-40dcd592-dfb04328_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:28.028726Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:28.028744Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 139
2025-10-09T21:43:28.029100Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 139 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:28.029140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 139 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:28.029149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=139 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:28.029154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:28.029543Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 139 sessionId:  describe result for acl check
2025-10-09T21:43:28.029577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:28.029583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:28.029587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:28.029592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340904058246611:3378] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:28.030175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340904058246611:3378] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:28.033287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340904058246611:3378] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:28.038888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340904058246611:3378] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:28.038910Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340904058246611:3378] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:28.038914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340904058246611:3378] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:28.038924Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 139 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:28.047800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:28.051669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:28.051930Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:28.051941Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:28.051952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:28.052011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:28.052058Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340903696437533:3151]
2025-10-09T21:43:28.052715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:28.053345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:28.053407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:28.053751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:28.053799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:28.053805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:28.053807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:28.053808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:28.053810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:28.053811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:28.053812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.053814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.053815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.053821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:28.053831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:28.053836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:28.053841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:28.053845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:28.053850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:28.053854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.053858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.053863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.053870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:28.053878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:28.053885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:28.053891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:28.053896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:28.053901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:28.053906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:28.053910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:28.053916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:28.053921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:28.053927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:28.053934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:28.053950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.053956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.053960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.053967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.053972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.053978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.054011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:28.064727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:28.066009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:28.066018Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:28.066020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:28.066025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:28.066058Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 138 [2:7559340903696437533:3151]
2025-10-09T21:43:28.066077Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:28.066100Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:28.066103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.066106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.066109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.066128Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:28.066136Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:28.066143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:28.066222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.066224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.066226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.066229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:28.066308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:28.066311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.066326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.066337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.066352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:28.066373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340903696437527:3151]
2025-10-09T21:43:28.066376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340903696437527:3151]
2025-10-09T21:43:28.066378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340903696437527:3151]
2025-10-09T21:43:28.066387Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:28.066402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:28.066429Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:28.066432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:28.066433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:28.066435Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:28.066436Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:28.066480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340903696437527:3151]
2025-10-09T21:43:28.066491Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:28.066504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340904058246760:3378], now have 1 active actors on pipe
2025-10-09T21:43:28.066508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340904058246597:2432], now have 1 active actors on pipe
2025-10-09T21:43:28.066988Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:28.067102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:28.067113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:28.067159Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6226dea0-a47fdd5a-cc5c8b9a-82210490_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:28.067169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.067174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.067177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:28.067189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.067203Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:28.067237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:28.067837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:28.067848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:28.067869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:28.068478Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d11a461d-9b03b6d6-40dcd592-dfb04328_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6226dea0-a47fdd5a-cc5c8b9a-82210490_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:28.066592Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 138
2025-10-09T21:43:28.066630Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 138
2025-10-09T21:43:28.068034Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 139 partition: 0 MaxSeqNo: 25 sessionId: src_id|6226dea0-a47fdd5a-cc5c8b9a-82210490_0
2025-10-09T21:43:28.071058Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6226dea0-a47fdd5a-cc5c8b9a-82210490_0] Write session: try to update token
2025-10-09T21:43:28.071077Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6226dea0-a47fdd5a-cc5c8b9a-82210490_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:28.097020Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 139 sessionId: src_id|6226dea0-a47fdd5a-cc5c8b9a-82210490_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:28.099712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:28.101713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:28.101733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:28.101788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:28.119967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340904058246792:3142], now have 1 active actors on pipe
2025-10-09T21:43:28.116863Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:28.118411Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340904058246792:3142]
2025-10-09T21:43:28.120644Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 138, pipe: [1:7559340904058246792:3142]
2025-10-09T21:43:28.144355Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:28.144371Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:28.144374Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:28.144376Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:28.144387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340903696437527:3151]
2025-10-09T21:43:28.144391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340903696437527:3151]
2025-10-09T21:43:28.144394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340903696437527:3151]
2025-10-09T21:43:28.144444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:28.144451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.144454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:28.144594Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:28.146183Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:28.146195Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:28.146199Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:28.372951Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:28.381968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:28.381997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:28.383442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:28.383515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:28.383573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:28.383638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:28.383678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:28.383684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:28.383751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:28.383811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:28.383868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:28.383937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:28.383977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:28.383980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:28.384048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:28.384105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:28.384168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:28.384232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:28.384270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:28.384273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:28.384338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:28.384399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:28.384458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:28.384534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:28.384582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:28.384585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:28.384650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:28.384704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:28.384764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:28.384827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:28.384868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:28.384871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:28.384956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:28.385030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:28.385102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:28.385177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:28.385221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:28.385225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:28.385305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:28.385372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:28.385444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:28.385512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:28.385559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:28.385562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:28.385629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:28.385686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:28.385742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:28.385805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:28.385842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:28.385845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:28.385917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:28.385980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:28.386065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:28.386127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:28.386174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:28.386177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:28.386247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:28.386305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:28.386365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:28.386826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:28.386873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:28.386878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:28.387321Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:28.387323Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340904058246792:3142]
2025-10-09T21:43:28.387335Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:28.387344Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:28.387384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 139 reason: error at write: Disconnected sessionId: src_id|6226dea0-a47fdd5a-cc5c8b9a-82210490_0
2025-10-09T21:43:28.387631Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 139 sessionId: src_id|6226dea0-a47fdd5a-cc5c8b9a-82210490_0 is DEAD
2025-10-09T21:43:28.387731Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6226dea0-a47fdd5a-cc5c8b9a-82210490_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:28.387742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6226dea0-a47fdd5a-cc5c8b9a-82210490_0] Write session will restart in 0.009016s
2025-10-09T21:43:28.387765Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6226dea0-a47fdd5a-cc5c8b9a-82210490_0] Write session: Do CDS request
2025-10-09T21:43:28.387772Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6226dea0-a47fdd5a-cc5c8b9a-82210490_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:28.388051Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:28.397576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6226dea0-a47fdd5a-cc5c8b9a-82210490_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:28.397747Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:28.397763Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 140
2025-10-09T21:43:28.398247Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 140 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:28.398298Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 140 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:28.398308Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=140 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:28.398313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:28.398915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 140 sessionId:  describe result for acl check
2025-10-09T21:43:28.398949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:28.398951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:28.398952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:28.398958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340904058246827:3383] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:28.399518Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340904058246827:3383] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:28.405806Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340904058246827:3383] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:28.409394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340904058246827:3383] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:28.409410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340904058246827:3383] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:28.409414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340904058246827:3383] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:28.409424Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 140 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:28.409703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:28.411030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:28.411217Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:28.411220Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:28.411230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:28.411275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:28.411310Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340903696437729:3153]
2025-10-09T21:43:28.411812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:28.412128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:28.412171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:28.412444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:28.412479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:28.412480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:28.412481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:28.412482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:28.412483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:28.412484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:28.412484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.412485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.412486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.412491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:28.412500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:28.412503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:28.412506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:28.412510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:28.412513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:28.412515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.412518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.412520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.412527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:28.412534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:28.412539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:28.412543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:28.412547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:28.412551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:28.412554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:28.412558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:28.412562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:28.412566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:28.412570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:28.412574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:28.412578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.412582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.412585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.412589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.412592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.412596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.412619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:28.426189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:28.427376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:28.427390Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:28.427393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:28.427398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:28.427405Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 139 [2:7559340903696437729:3153]
2025-10-09T21:43:28.427423Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:28.427443Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:28.427446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.427449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.427452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.427473Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:28.427479Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:28.427488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:28.427567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.427572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.427574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.427577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:28.427655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:28.427658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.427671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.427681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.427696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:28.427718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340903696437722:3153]
2025-10-09T21:43:28.427720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340903696437722:3153]
2025-10-09T21:43:28.427723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340903696437722:3153]
2025-10-09T21:43:28.427732Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:28.427748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:28.427763Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:28.427767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:28.427814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340903696437722:3153]
2025-10-09T21:43:28.427817Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:28.428293Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:28.428296Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:28.428298Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:28.428304Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:28.428904Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 139
2025-10-09T21:43:28.428948Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 139
2025-10-09T21:43:28.430103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 140 partition: 0 MaxSeqNo: 25 sessionId: src_id|12eac734-b78ef820-c54d45b5-ba144d1b_0
2025-10-09T21:43:28.428472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340904058246801:2432], now have 1 active actors on pipe
2025-10-09T21:43:28.428484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340904058246989:3383], now have 1 active actors on pipe
2025-10-09T21:43:28.429427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:28.429435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:28.429476Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|12eac734-b78ef820-c54d45b5-ba144d1b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:28.429487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.429493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.429496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:28.429509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.429525Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:28.429545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:28.429831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:28.429835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:28.429854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:28.434374Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6226dea0-a47fdd5a-cc5c8b9a-82210490_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|12eac734-b78ef820-c54d45b5-ba144d1b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:28.437236Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12eac734-b78ef820-c54d45b5-ba144d1b_0] Write session: try to update token
2025-10-09T21:43:28.437253Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12eac734-b78ef820-c54d45b5-ba144d1b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:28.454111Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:28.454131Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:28.454133Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:28.454136Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:28.454147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340903696437722:3153]
2025-10-09T21:43:28.454152Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340903696437722:3153]
2025-10-09T21:43:28.454156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340903696437722:3153]
2025-10-09T21:43:28.454204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:28.454212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.454215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:28.454367Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:28.459349Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:28.459367Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:28.459371Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:28.464001Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 140 sessionId: src_id|12eac734-b78ef820-c54d45b5-ba144d1b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:28.466905Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:28.490318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340904058247002:3142]
2025-10-09T21:43:28.494122Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 139, pipe: [1:7559340904058247002:3142]
2025-10-09T21:43:28.637430Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:28.637858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:28.637875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:28.637923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340904058247002:3142], now have 1 active actors on pipe
2025-10-09T21:43:28.638176Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:28.640574Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:28.640631Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340904058247002:3142]
2025-10-09T21:43:28.641105Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12eac734-b78ef820-c54d45b5-ba144d1b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:28.641117Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12eac734-b78ef820-c54d45b5-ba144d1b_0] Write session will restart in 0.006384s
2025-10-09T21:43:28.641143Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12eac734-b78ef820-c54d45b5-ba144d1b_0] Write session: Do CDS request
2025-10-09T21:43:28.641150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12eac734-b78ef820-c54d45b5-ba144d1b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:28.640655Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:28.640735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 140 reason: status is not ok: tablet will be restarted right now sessionId: src_id|12eac734-b78ef820-c54d45b5-ba144d1b_0
2025-10-09T21:43:28.640917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 140 sessionId: src_id|12eac734-b78ef820-c54d45b5-ba144d1b_0 is DEAD
2025-10-09T21:43:28.648544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12eac734-b78ef820-c54d45b5-ba144d1b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:28.650159Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:28.650177Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 141
2025-10-09T21:43:28.650324Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 141 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:28.650357Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 141 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:28.650362Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=141 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:28.650365Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:28.652051Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 141 sessionId:  describe result for acl check
2025-10-09T21:43:28.652086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:28.652087Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:28.652089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:28.652093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340904058247043:3393] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:28.652505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340904058247043:3393] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:28.662814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340904058247043:3393] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:28.668978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340904058247043:3393] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:28.668999Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340904058247043:3393] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:28.669004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340904058247043:3393] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:28.669014Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 141 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:28.671614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:28.673120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:28.673347Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:28.673357Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:28.673371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:28.673445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:28.673488Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340903696437963:3161]
2025-10-09T21:43:28.673947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:28.674532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:28.674592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:28.674940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:28.674996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:28.674998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:28.674999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:28.675001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:28.675002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:28.675003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:28.675005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.675006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.675007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.675013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:28.675025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:28.675029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:28.675033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:28.675037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:28.675041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:28.675044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.675047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.675050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.675058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:28.675066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:28.675072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:28.675077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:28.675082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:28.675086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:28.675090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:28.675095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:28.675099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:28.675103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:28.675106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:28.675112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:28.675116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.675121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.675125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.675130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.675135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.675139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.675166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:28.683459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:28.684824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:28.684839Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:28.684841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:28.684847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:28.684853Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 140 [2:7559340903696437963:3161]
2025-10-09T21:43:28.684867Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:28.684888Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:28.684891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.684894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.684896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.684916Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:28.684922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:28.684929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:28.685003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.685006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.685008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.685011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:28.685088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:28.685100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.685113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.685126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.685145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:28.685170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340903696437957:3161]
2025-10-09T21:43:28.685173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340903696437957:3161]
2025-10-09T21:43:28.685175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340903696437957:3161]
2025-10-09T21:43:28.685189Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:28.685209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:28.685224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:28.685227Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:28.685268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340903696437957:3161]
2025-10-09T21:43:28.685271Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:28.685374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340904058247020:2432], now have 1 active actors on pipe
2025-10-09T21:43:28.685381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340904058247203:3393], now have 1 active actors on pipe
2025-10-09T21:43:28.685459Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:28.685462Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:28.685464Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:28.685470Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:28.685621Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 140
2025-10-09T21:43:28.685662Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 140
2025-10-09T21:43:28.685722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:28.685735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:28.685764Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3e682088-85ce8df6-171c2a49-5a8ca8a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:28.685779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.685783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.685786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:28.685809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.685822Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:28.685843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:28.686071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:28.686080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:28.686096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:28.686179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 141 partition: 0 MaxSeqNo: 25 sessionId: src_id|3e682088-85ce8df6-171c2a49-5a8ca8a_0
2025-10-09T21:43:28.690160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12eac734-b78ef820-c54d45b5-ba144d1b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3e682088-85ce8df6-171c2a49-5a8ca8a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:28.692842Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3e682088-85ce8df6-171c2a49-5a8ca8a_0] Write session: try to update token
2025-10-09T21:43:28.692865Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3e682088-85ce8df6-171c2a49-5a8ca8a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:28.717359Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:28.717378Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:28.717389Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340903696437957:3161]
2025-10-09T21:43:28.717445Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:28.717652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:28.717782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:28.717922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:28.718089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:28.718276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:28.718330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:28.718485Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:28.731597Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:28.731616Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:28.731619Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:28.731621Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:28.731634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340903696437957:3161]
2025-10-09T21:43:28.731639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340903696437957:3161]
2025-10-09T21:43:28.731643Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340903696437957:3161]
2025-10-09T21:43:28.731692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:28.731700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.731707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:28.731871Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:28.736214Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:28.736229Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:28.736233Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:28.742128Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340904058247227:3142]
2025-10-09T21:43:28.742538Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 140, pipe: [1:7559340904058247227:3142]
2025-10-09T21:43:28.747282Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 141 sessionId: src_id|3e682088-85ce8df6-171c2a49-5a8ca8a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:28.749701Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:28.908849Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:28.909472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340904058247227:3142], now have 1 active actors on pipe
2025-10-09T21:43:28.909519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:28.909527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:28.910078Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:28.910132Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340904058247227:3142]
2025-10-09T21:43:28.910150Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:28.910153Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:28.910211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 141 reason: error at write: Disconnected sessionId: src_id|3e682088-85ce8df6-171c2a49-5a8ca8a_0
2025-10-09T21:43:28.910412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 141 sessionId: src_id|3e682088-85ce8df6-171c2a49-5a8ca8a_0 is DEAD
2025-10-09T21:43:28.910051Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:28.913466Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e682088-85ce8df6-171c2a49-5a8ca8a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:28.913482Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e682088-85ce8df6-171c2a49-5a8ca8a_0] Write session will restart in 0.006940s
2025-10-09T21:43:28.913523Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e682088-85ce8df6-171c2a49-5a8ca8a_0] Write session: Do CDS request
2025-10-09T21:43:28.913531Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e682088-85ce8df6-171c2a49-5a8ca8a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:28.922743Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3e682088-85ce8df6-171c2a49-5a8ca8a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:28.926111Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:28.926129Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 142
2025-10-09T21:43:28.926808Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 142 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:28.926845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 142 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:28.926851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=142 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:28.926855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:28.927505Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 142 sessionId:  describe result for acl check
2025-10-09T21:43:28.927543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:28.927544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:28.927546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:28.927552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340904058247253:3398] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:28.928203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340904058247253:3398] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:28.942908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340904058247253:3398] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:28.957323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:28.963354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340904058247253:3398] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:28.963376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340904058247253:3398] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:28.963379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340904058247253:3398] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:28.963389Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 142 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:28.963187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:28.963458Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:28.963462Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:28.963476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:28.963543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:28.963594Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340903696438162:3171]
2025-10-09T21:43:28.964127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:28.964512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:28.964576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:28.964934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:28.964980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:28.964982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:28.964983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:28.964984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:28.964986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:28.964987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:28.964989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.964990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.964991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.964997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:28.965010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:28.965015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:28.965020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:28.965024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:28.965029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:28.965034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.965039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.965043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.965051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:28.965060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:28.965067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:28.965074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:28.965080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:28.965086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:28.965092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:28.965098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:28.965103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:28.965109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:28.965113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:28.965119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:28.965125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.965131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.965136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.965142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.965147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.965153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.965184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:28.974828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:28.975944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:28.975950Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:28.975953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:28.975959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:28.975967Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 141 [2:7559340903696438162:3171]
2025-10-09T21:43:28.975984Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:28.976005Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:28.976008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.976012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.976015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.976033Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:28.976039Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:28.976046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:28.976115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.976117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.976119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.976122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:28.976203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:28.976206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:28.976218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:28.976228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:28.976244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:28.976264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340903696438156:3171]
2025-10-09T21:43:28.976267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340903696438156:3171]
2025-10-09T21:43:28.976270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340903696438156:3171]
2025-10-09T21:43:28.976284Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:28.976299Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:28.976313Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:28.976317Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:28.976359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340903696438156:3171]
2025-10-09T21:43:28.976362Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:28.977090Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:28.977096Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:28.977105Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:28.977112Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:28.978078Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 141
2025-10-09T21:43:28.978122Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 141
2025-10-09T21:43:28.978613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340904058247428:3398], now have 1 active actors on pipe
2025-10-09T21:43:28.978624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340904058247238:2432], now have 1 active actors on pipe
2025-10-09T21:43:28.978649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:28.978655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:28.978693Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8f88dba1-bba8b410-fffbb020-92c9efe5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:28.978705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.978710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.978714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:28.978727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.978742Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:28.978765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:28.982610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:28.982625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:28.982660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:28.986555Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 142 partition: 0 MaxSeqNo: 25 sessionId: src_id|8f88dba1-bba8b410-fffbb020-92c9efe5_0
2025-10-09T21:43:28.990605Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e682088-85ce8df6-171c2a49-5a8ca8a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8f88dba1-bba8b410-fffbb020-92c9efe5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:28.994020Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8f88dba1-bba8b410-fffbb020-92c9efe5_0] Write session: try to update token
2025-10-09T21:43:28.994066Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8f88dba1-bba8b410-fffbb020-92c9efe5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:28.999949Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:28.999964Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:28.999967Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:28.999969Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:28.999979Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340903696438156:3171]
2025-10-09T21:43:28.999984Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340903696438156:3171]
2025-10-09T21:43:28.999989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340903696438156:3171]
2025-10-09T21:43:29.000033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:29.000041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.000044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:29.000178Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:29.003436Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:29.003450Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:29.003454Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:29.013048Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340908353214736:3142]
2025-10-09T21:43:29.014065Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 141, pipe: [1:7559340908353214736:3142]
2025-10-09T21:43:29.020047Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 142 sessionId: src_id|8f88dba1-bba8b410-fffbb020-92c9efe5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:29.022919Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:29.219538Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:29.219958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340908353214736:3142], now have 1 active actors on pipe
2025-10-09T21:43:29.220005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:29.220012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:29.220562Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:29.223172Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8f88dba1-bba8b410-fffbb020-92c9efe5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:29.223186Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8f88dba1-bba8b410-fffbb020-92c9efe5_0] Write session will restart in 0.007827s
2025-10-09T21:43:29.223215Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8f88dba1-bba8b410-fffbb020-92c9efe5_0] Write session: Do CDS request
2025-10-09T21:43:29.223223Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8f88dba1-bba8b410-fffbb020-92c9efe5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:29.222673Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:29.222771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 142 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8f88dba1-bba8b410-fffbb020-92c9efe5_0
2025-10-09T21:43:29.223128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 142 sessionId: src_id|8f88dba1-bba8b410-fffbb020-92c9efe5_0 is DEAD
2025-10-09T21:43:29.223281Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340908353214736:3142]
2025-10-09T21:43:29.223296Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:29.232786Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8f88dba1-bba8b410-fffbb020-92c9efe5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:29.233075Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:29.233096Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 143
2025-10-09T21:43:29.233439Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 143 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:29.233484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 143 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:29.233490Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=143 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:29.233496Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:29.234210Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 143 sessionId:  describe result for acl check
2025-10-09T21:43:29.234261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:29.234263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:29.234265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:29.234271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340908353214747:3402] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:29.234913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340908353214747:3402] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:29.249191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340908353214747:3402] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:29.257504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340908353214747:3402] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:29.257523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340908353214747:3402] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:29.257528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340908353214747:3402] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:29.257541Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 143 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:29.288463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:29.290195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:29.290497Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:29.290501Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:29.290517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:29.290595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:29.290645Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340907991405657:3173]
2025-10-09T21:43:29.291280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:29.293323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:29.293405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:29.293821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:29.293897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:29.293898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:29.293899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:29.293901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:29.293902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:29.293904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:29.293905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.293906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.293907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.293914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:29.293925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:29.293929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:29.293933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:29.293936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:29.293941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:29.293944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.293948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.293952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.293962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:29.293971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:29.293978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:29.293983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:29.293988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:29.293993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:29.293998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:29.294004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:29.294009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:29.294014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:29.294018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:29.294023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:29.294048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.294054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.294058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.294062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.294067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.294073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.294106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:29.311540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:29.312920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:29.312929Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:29.312932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:29.312937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:29.312947Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 142 [2:7559340907991405657:3173]
2025-10-09T21:43:29.312966Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:29.312989Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:29.312993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.312998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.313000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.313023Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:29.313029Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:29.313037Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:29.313111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.313114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.313116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.313120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:29.313213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:29.313216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.313232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.313243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.313258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:29.313281Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340907991405651:3173]
2025-10-09T21:43:29.313283Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340907991405651:3173]
2025-10-09T21:43:29.313286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340907991405651:3173]
2025-10-09T21:43:29.313302Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:29.313323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:29.313339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:29.313343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:29.313392Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340907991405651:3173]
2025-10-09T21:43:29.313394Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:29.319384Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 142
2025-10-09T21:43:29.319429Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 142
2025-10-09T21:43:29.318478Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:29.318492Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:29.318494Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:29.318503Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:29.320088Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340908353214939:3142]
2025-10-09T21:43:29.320960Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 142, pipe: [1:7559340908353214939:3142]
2025-10-09T21:43:29.320998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340908353214740:2432], now have 1 active actors on pipe
2025-10-09T21:43:29.321010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340908353214791:3402], now have 1 active actors on pipe
2025-10-09T21:43:29.321035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:29.321043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:29.321059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340908353214939:3142], now have 1 active actors on pipe
2025-10-09T21:43:29.321085Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a6aeb572-e24f3a73-7a75bbed-c0758114_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:29.321099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.321105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.321109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:29.321124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.321144Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:29.321168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:29.321594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:29.321604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:29.321625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:29.322861Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8f88dba1-bba8b410-fffbb020-92c9efe5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a6aeb572-e24f3a73-7a75bbed-c0758114_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:29.321790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 143 partition: 0 MaxSeqNo: 25 sessionId: src_id|a6aeb572-e24f3a73-7a75bbed-c0758114_0
2025-10-09T21:43:29.325941Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6aeb572-e24f3a73-7a75bbed-c0758114_0] Write session: try to update token
2025-10-09T21:43:29.325968Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6aeb572-e24f3a73-7a75bbed-c0758114_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:29.359285Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 143 sessionId: src_id|a6aeb572-e24f3a73-7a75bbed-c0758114_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:29.360069Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:29.360087Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:29.360089Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:29.360091Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:29.360102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340907991405651:3173]
2025-10-09T21:43:29.360106Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340907991405651:3173]
2025-10-09T21:43:29.360110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340907991405651:3173]
2025-10-09T21:43:29.360158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:29.360165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.360168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:29.360331Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:29.361496Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:29.361506Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:29.361509Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:29.363654Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:29.566917Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:29.567231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:29.567243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:29.568701Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6aeb572-e24f3a73-7a75bbed-c0758114_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:29.568714Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6aeb572-e24f3a73-7a75bbed-c0758114_0] Write session will restart in 0.009363s
2025-10-09T21:43:29.568741Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6aeb572-e24f3a73-7a75bbed-c0758114_0] Write session: Do CDS request
2025-10-09T21:43:29.568204Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:29.568749Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6aeb572-e24f3a73-7a75bbed-c0758114_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:29.568252Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:29.568254Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:29.568315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 143 reason: error at write: Disconnected sessionId: src_id|a6aeb572-e24f3a73-7a75bbed-c0758114_0
2025-10-09T21:43:29.568510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 143 sessionId: src_id|a6aeb572-e24f3a73-7a75bbed-c0758114_0 is DEAD
2025-10-09T21:43:29.568664Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340908353214939:3142]
2025-10-09T21:43:29.569110Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:29.579661Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6aeb572-e24f3a73-7a75bbed-c0758114_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:29.582765Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:29.582781Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 144
2025-10-09T21:43:29.583074Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 144 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:29.583111Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 144 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:29.583118Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=144 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:29.583121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:29.583696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 144 sessionId:  describe result for acl check
2025-10-09T21:43:29.583729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:29.583731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:29.583733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:29.583740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340908353214965:3407] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:29.584302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340908353214965:3407] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:29.595413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340908353214965:3407] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:29.612329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340908353214965:3407] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:29.612351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340908353214965:3407] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:29.612355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340908353214965:3407] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:29.612366Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 144 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:29.630576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:29.630854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:29.631033Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:29.631037Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:29.631050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:29.631100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:29.631140Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340907991405857:3175]
2025-10-09T21:43:29.631751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:29.632322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:29.632372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:29.632713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:29.632766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:29.632768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:29.632769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:29.632770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:29.632772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:29.632773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:29.632775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.632776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.632777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.632784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:29.632796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:29.632801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:29.632805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:29.632810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:29.632815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:29.632819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.632824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.632828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.632837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:29.632846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:29.632853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:29.632859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:29.632865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:29.632870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:29.632875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:29.632880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:29.632886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:29.632891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:29.632896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:29.632902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:29.632907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.632914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.632919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.632925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.632930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.632956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.632988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:29.641661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:29.643367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:29.643375Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:29.643378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:29.643384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:29.643392Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 143 [2:7559340907991405857:3175]
2025-10-09T21:43:29.643409Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:29.643430Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:29.643434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.643437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.643440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.643459Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:29.643465Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:29.643473Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:29.643548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.643551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.643553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.643556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:29.643645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:29.643648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.643664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.643677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.643696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:29.643719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340907991405851:3175]
2025-10-09T21:43:29.643722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340907991405851:3175]
2025-10-09T21:43:29.643725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340907991405851:3175]
2025-10-09T21:43:29.643742Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:29.643762Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:29.643778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:29.643782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:29.643829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340907991405851:3175]
2025-10-09T21:43:29.643832Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:29.644096Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:29.644104Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:29.644106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:29.644113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:29.644978Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 143
2025-10-09T21:43:29.645013Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 143
2025-10-09T21:43:29.645454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340908353215061:3407], now have 1 active actors on pipe
2025-10-09T21:43:29.645466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340908353214949:2432], now have 1 active actors on pipe
2025-10-09T21:43:29.645482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:29.645487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:29.645522Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a16b1f50-f9c58166-76e07f9f-a620ee0b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:29.645532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.645538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.645542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:29.645556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.645571Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:29.645590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:29.645900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:29.645907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:29.645922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:29.646079Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 144 partition: 0 MaxSeqNo: 25 sessionId: src_id|a16b1f50-f9c58166-76e07f9f-a620ee0b_0
2025-10-09T21:43:29.646521Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6aeb572-e24f3a73-7a75bbed-c0758114_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a16b1f50-f9c58166-76e07f9f-a620ee0b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:29.649202Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a16b1f50-f9c58166-76e07f9f-a620ee0b_0] Write session: try to update token
2025-10-09T21:43:29.649224Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a16b1f50-f9c58166-76e07f9f-a620ee0b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:29.670272Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340908353215157:3142]
2025-10-09T21:43:29.670547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340908353215157:3142], now have 1 active actors on pipe
2025-10-09T21:43:29.670782Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 143, pipe: [1:7559340908353215157:3142]
2025-10-09T21:43:29.676247Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 144 sessionId: src_id|a16b1f50-f9c58166-76e07f9f-a620ee0b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:29.679037Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:29.679821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:29.679841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:29.679883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:29.680341Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:29.699679Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:29.699699Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:29.699701Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:29.699703Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:29.699712Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340907991405851:3175]
2025-10-09T21:43:29.699716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340907991405851:3175]
2025-10-09T21:43:29.699720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340907991405851:3175]
2025-10-09T21:43:29.699777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:29.699785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.699794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:29.699949Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:29.700505Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:29.700518Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:29.700524Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:29.872504Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:29.881543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:29.881572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:29.881734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:29.881789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:29.881853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:29.881911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:29.882291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:29.882301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:29.882372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:29.882440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:29.882497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:29.882556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:29.882596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:29.882599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:29.882668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:29.882728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:29.882793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:29.882856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:29.882898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:29.882901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:29.882970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:29.883029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:29.883097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:29.883161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:29.883232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:29.883235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:29.883305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:29.883366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:29.883426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:29.883495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:29.883536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:29.883539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:29.883610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:29.883671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:29.883736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:29.883815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:29.883862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:29.883864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:29.883935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:29.883994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:29.884058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:29.884125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:29.884173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:29.884176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:29.884250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:29.884310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:29.884376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:29.884441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:29.884480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:29.884483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:29.884554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:29.884615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:29.884677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:29.884744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:29.884785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:29.884787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:29.884860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:29.884924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:29.884996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:29.885059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:29.885101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:29.885106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:29.886547Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:29.886561Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:29.886591Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 144 reason: error at write: Disconnected sessionId: src_id|a16b1f50-f9c58166-76e07f9f-a620ee0b_0
2025-10-09T21:43:29.886786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 144 sessionId: src_id|a16b1f50-f9c58166-76e07f9f-a620ee0b_0 is DEAD
2025-10-09T21:43:29.886939Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340908353215157:3142]
2025-10-09T21:43:29.886954Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:29.887233Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a16b1f50-f9c58166-76e07f9f-a620ee0b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:29.887246Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a16b1f50-f9c58166-76e07f9f-a620ee0b_0] Write session will restart in 0.005809s
2025-10-09T21:43:29.887283Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a16b1f50-f9c58166-76e07f9f-a620ee0b_0] Write session: Do CDS request
2025-10-09T21:43:29.887290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a16b1f50-f9c58166-76e07f9f-a620ee0b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:29.887409Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:29.894605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a16b1f50-f9c58166-76e07f9f-a620ee0b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:29.895006Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:29.895022Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 145
2025-10-09T21:43:29.895277Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 145 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:29.895313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 145 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:29.895319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=145 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:29.895323Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:29.896302Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 145 sessionId:  describe result for acl check
2025-10-09T21:43:29.896340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:29.896342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:29.896344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:29.896350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340908353215205:3419] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:29.896908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340908353215205:3419] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:29.914312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340908353215205:3419] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:29.925194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340908353215205:3419] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:29.925214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340908353215205:3419] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:29.925219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340908353215205:3419] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:29.925230Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 145 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:29.928853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:29.930587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:29.930804Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:29.930808Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:29.930822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:29.930884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:29.931146Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340907991406098:3190]
2025-10-09T21:43:29.931715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:29.932184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:29.932235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:29.932552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:29.932613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:29.932614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:29.932616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:29.932617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:29.932618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:29.932619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:29.932621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.932622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.932623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.932629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:29.932640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:29.932644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:29.932648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:29.932651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:29.932656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:29.932659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.932663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.932666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.932674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:29.932682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:29.932688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:29.932693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:29.932698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:29.932704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:29.932708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:29.932715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:29.932720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:29.932725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:29.932730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:29.932736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:29.932741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.932746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.932751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.932756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.932760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.932766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.932797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:29.940827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:29.942339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:29.942345Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:29.942348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:29.942353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:29.942362Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 144 [2:7559340907991406098:3190]
2025-10-09T21:43:29.942379Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:29.942408Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:29.942412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.942427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.942429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.942450Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:29.942456Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:29.942464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:29.942536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.942538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.942540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.942544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:29.942622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:29.942626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.942638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:29.942649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:29.942666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:29.942689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340907991406092:3190]
2025-10-09T21:43:29.942691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340907991406092:3190]
2025-10-09T21:43:29.942693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340907991406092:3190]
2025-10-09T21:43:29.942708Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:29.942726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:29.942742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:29.942746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:29.942790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340907991406092:3190]
2025-10-09T21:43:29.942793Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:29.942885Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:29.942892Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:29.942895Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:29.942902Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:29.943438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340908353215195:2432], now have 1 active actors on pipe
2025-10-09T21:43:29.943447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340908353215386:3419], now have 1 active actors on pipe
2025-10-09T21:43:29.943553Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 144
2025-10-09T21:43:29.943566Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 144
2025-10-09T21:43:29.944202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:29.944214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:29.944263Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|69e0385a-fdef8b38-422fee45-b02ef38d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:29.944274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.944279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.944282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:29.944295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.944318Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:29.944336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:29.945409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:29.945418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:29.945440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:29.945832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 145 partition: 0 MaxSeqNo: 25 sessionId: src_id|69e0385a-fdef8b38-422fee45-b02ef38d_0
2025-10-09T21:43:29.946812Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a16b1f50-f9c58166-76e07f9f-a620ee0b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|69e0385a-fdef8b38-422fee45-b02ef38d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:29.949673Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|69e0385a-fdef8b38-422fee45-b02ef38d_0] Write session: try to update token
2025-10-09T21:43:29.949693Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|69e0385a-fdef8b38-422fee45-b02ef38d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:29.969192Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:29.969214Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:29.969217Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:29.969219Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:29.969231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340907991406092:3190]
2025-10-09T21:43:29.969237Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340907991406092:3190]
2025-10-09T21:43:29.969242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340907991406092:3190]
2025-10-09T21:43:29.969293Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:29.969299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:29.969307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:29.969309Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:29.969310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:29.969313Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:29.969488Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:29.985705Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 145 sessionId: src_id|69e0385a-fdef8b38-422fee45-b02ef38d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:29.986665Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340908353215401:3142]
2025-10-09T21:43:29.988677Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:29.992153Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 144, pipe: [1:7559340908353215401:3142]
2025-10-09T21:43:30.207042Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:30.207943Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340908353215401:3142]
2025-10-09T21:43:30.208001Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:30.208000Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:30.208008Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:30.208069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 145 reason: error at write: Disconnected sessionId: src_id|69e0385a-fdef8b38-422fee45-b02ef38d_0
2025-10-09T21:43:30.207591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:30.207611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:30.207637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340908353215401:3142], now have 1 active actors on pipe
2025-10-09T21:43:30.208066Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:30.208265Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 145 sessionId: src_id|69e0385a-fdef8b38-422fee45-b02ef38d_0 is DEAD
2025-10-09T21:43:30.209611Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69e0385a-fdef8b38-422fee45-b02ef38d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:30.209622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69e0385a-fdef8b38-422fee45-b02ef38d_0] Write session will restart in 0.008431s
2025-10-09T21:43:30.209648Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69e0385a-fdef8b38-422fee45-b02ef38d_0] Write session: Do CDS request
2025-10-09T21:43:30.209655Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69e0385a-fdef8b38-422fee45-b02ef38d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:30.219626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|69e0385a-fdef8b38-422fee45-b02ef38d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:30.219998Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:30.220017Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 146
2025-10-09T21:43:30.220221Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 146 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:30.220256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 146 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:30.220263Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=146 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:30.220267Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:30.220774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 146 sessionId:  describe result for acl check
2025-10-09T21:43:30.220810Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:30.220812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:30.220813Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:30.220821Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340912648182716:3423] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:30.221378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340912648182716:3423] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:30.228288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340912648182716:3423] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:30.239877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340912648182716:3423] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:30.239897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340912648182716:3423] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:30.239902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340912648182716:3423] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:30.239913Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 146 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:30.244081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:30.245064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:30.245284Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:30.245298Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:30.245311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:30.245369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:30.245416Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340912286373593:3193]
2025-10-09T21:43:30.246184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:30.246636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:30.246685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:30.246991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:30.247040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:30.247041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:30.247042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:30.247043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:30.247044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:30.247045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:30.247047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.247048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.247049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.247054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:30.247063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:30.247066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:30.247069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:30.247073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:30.247076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:30.247079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.247083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.247086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.247093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:30.247100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:30.247106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:30.247111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:30.247115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:30.247119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:30.247123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:30.247127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:30.247131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:30.247136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:30.247141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:30.247146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:30.247150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.247155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.247158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.247162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.247166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.247170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.247194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:30.256836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:30.258342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:30.258362Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:30.258365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:30.258370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:30.258377Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 145 [2:7559340912286373593:3193]
2025-10-09T21:43:30.258395Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:30.258426Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:30.258430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.258433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.258436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.258456Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:30.258463Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:30.258477Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:30.258561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.258570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.258572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.258575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:30.258652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:30.258660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.258675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.258688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.258710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:30.258736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340912286373587:3193]
2025-10-09T21:43:30.258738Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340912286373587:3193]
2025-10-09T21:43:30.258742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340912286373587:3193]
2025-10-09T21:43:30.258751Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:30.258768Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:30.258784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:30.258787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:30.258788Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:30.258791Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:30.258795Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:30.258838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340912648182896:3423], now have 1 active actors on pipe
2025-10-09T21:43:30.258847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340912648182700:2432], now have 1 active actors on pipe
2025-10-09T21:43:30.258849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340912286373587:3193]
2025-10-09T21:43:30.258857Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:30.258870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:30.259234Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 145
2025-10-09T21:43:30.259292Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 145
2025-10-09T21:43:30.259569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:30.259586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:30.259627Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7799a88a-774b607d-4e8d8c20-692aac7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:30.259637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.259642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.259644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:30.259656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.259686Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:30.259710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:30.260078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:30.260088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:30.260116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:30.260258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 146 partition: 0 MaxSeqNo: 25 sessionId: src_id|7799a88a-774b607d-4e8d8c20-692aac7_0
2025-10-09T21:43:30.260715Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69e0385a-fdef8b38-422fee45-b02ef38d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7799a88a-774b607d-4e8d8c20-692aac7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:30.263885Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7799a88a-774b607d-4e8d8c20-692aac7_0] Write session: try to update token
2025-10-09T21:43:30.263929Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7799a88a-774b607d-4e8d8c20-692aac7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:30.275325Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:30.275344Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:30.275347Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:30.275349Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:30.275358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340912286373587:3193]
2025-10-09T21:43:30.275362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340912286373587:3193]
2025-10-09T21:43:30.275366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340912286373587:3193]
2025-10-09T21:43:30.275411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:30.275417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.275420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:30.275552Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:30.278504Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:30.278531Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:30.278535Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:30.294123Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 146 sessionId: src_id|7799a88a-774b607d-4e8d8c20-692aac7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:30.297208Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:30.305582Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340912648182908:3142]
2025-10-09T21:43:30.309594Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 145, pipe: [1:7559340912648182908:3142]
2025-10-09T21:43:30.500753Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:30.501290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:30.501308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:30.501333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340912648182908:3142], now have 1 active actors on pipe
2025-10-09T21:43:30.501571Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:30.501660Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:30.501661Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340912648182908:3142]
2025-10-09T21:43:30.501684Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:30.501686Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:30.501766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 146 reason: error at write: Disconnected sessionId: src_id|7799a88a-774b607d-4e8d8c20-692aac7_0
2025-10-09T21:43:30.501974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 146 sessionId: src_id|7799a88a-774b607d-4e8d8c20-692aac7_0 is DEAD
2025-10-09T21:43:30.506144Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7799a88a-774b607d-4e8d8c20-692aac7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:30.506160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7799a88a-774b607d-4e8d8c20-692aac7_0] Write session will restart in 0.006596s
2025-10-09T21:43:30.506191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7799a88a-774b607d-4e8d8c20-692aac7_0] Write session: Do CDS request
2025-10-09T21:43:30.506199Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7799a88a-774b607d-4e8d8c20-692aac7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:30.513673Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7799a88a-774b607d-4e8d8c20-692aac7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:30.513746Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:30.513784Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 147
2025-10-09T21:43:30.514007Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 147 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:30.514064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 147 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:30.514071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=147 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:30.514077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:30.514545Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 147 sessionId:  describe result for acl check
2025-10-09T21:43:30.514596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:30.514602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:30.514606Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:30.514613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340912648182986:3428] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:30.515203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340912648182986:3428] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:30.520614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340912648182986:3428] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:30.524303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:30.525002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340912648182986:3428] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:30.525021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340912648182986:3428] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:30.525026Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340912648182986:3428] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:30.525036Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 147 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:30.525908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:30.526209Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:30.526220Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:30.526235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:30.526307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:30.526356Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340912286373794:3195]
2025-10-09T21:43:30.528034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:30.530788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:30.530877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:30.531305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:30.531385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:30.531394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:30.531396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:30.531397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:30.531398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:30.531400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:30.531401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.531403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.531404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.531411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:30.531423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:30.531428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:30.531431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:30.531435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:30.531439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:30.531443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.531447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.531451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.531459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:30.531468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:30.531474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:30.531479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:30.531485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:30.531490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:30.531494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:30.531500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:30.531505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:30.531510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:30.531515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:30.531520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:30.531526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.531531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.531538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.531544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.531550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.531555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.531590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:30.539677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:30.541068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:30.541082Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:30.541085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:30.541089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:30.541098Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 146 [2:7559340912286373794:3195]
2025-10-09T21:43:30.541114Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:30.541136Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:30.541139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.541143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.541145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.541163Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:30.541172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:30.541188Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:30.541256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.541266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.541268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.541270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:30.541354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:30.541360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.541374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.541384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.541401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:30.541429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340912286373788:3195]
2025-10-09T21:43:30.541432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340912286373788:3195]
2025-10-09T21:43:30.541434Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340912286373788:3195]
2025-10-09T21:43:30.541443Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:30.541460Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:30.541461Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:30.541464Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:30.541466Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:30.541476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:30.541480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:30.541493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340912648182917:2432], now have 1 active actors on pipe
2025-10-09T21:43:30.541497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340912648183113:3428], now have 1 active actors on pipe
2025-10-09T21:43:30.541539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340912286373788:3195]
2025-10-09T21:43:30.541554Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:30.541566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:30.541615Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 146
2025-10-09T21:43:30.541663Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 146
2025-10-09T21:43:30.542052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:30.542067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:30.542101Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|94bbd679-bb8f3db1-308f14e4-6150de96_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:30.542117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.542126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.542129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:30.542150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.542164Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:30.542185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:30.545279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:30.545299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:30.545333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:30.545476Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 147 partition: 0 MaxSeqNo: 25 sessionId: src_id|94bbd679-bb8f3db1-308f14e4-6150de96_0
2025-10-09T21:43:30.546010Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7799a88a-774b607d-4e8d8c20-692aac7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|94bbd679-bb8f3db1-308f14e4-6150de96_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:30.548830Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94bbd679-bb8f3db1-308f14e4-6150de96_0] Write session: try to update token
2025-10-09T21:43:30.548853Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94bbd679-bb8f3db1-308f14e4-6150de96_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:30.555916Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:30.555935Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:30.555949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340912286373788:3195]
2025-10-09T21:43:30.556002Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:30.556056Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:30.556169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:30.556312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:30.556451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:30.556585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:30.556763Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:30.556798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:30.571123Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 147 sessionId: src_id|94bbd679-bb8f3db1-308f14e4-6150de96_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:30.573613Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:30.575083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:30.575109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:30.575134Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:30.575138Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:30.575140Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:30.575143Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:30.575154Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340912286373788:3195]
2025-10-09T21:43:30.575159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340912286373788:3195]
2025-10-09T21:43:30.575165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340912286373788:3195]
2025-10-09T21:43:30.575222Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:30.575229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:30.575238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.575241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:30.575244Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:30.575248Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:30.575379Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:30.602020Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340912648183126:3142]
2025-10-09T21:43:30.603095Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 146, pipe: [1:7559340912648183126:3142]
2025-10-09T21:43:30.740288Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:30.740641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:30.740685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340912648183126:3142], now have 1 active actors on pipe
2025-10-09T21:43:30.741021Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:30.741149Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:30.741152Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:30.741252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 147 reason: error at write: Disconnected sessionId: src_id|94bbd679-bb8f3db1-308f14e4-6150de96_0
2025-10-09T21:43:30.741320Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:30.741369Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340912648183126:3142]
2025-10-09T21:43:30.741440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 147 sessionId: src_id|94bbd679-bb8f3db1-308f14e4-6150de96_0 is DEAD
2025-10-09T21:43:30.741662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94bbd679-bb8f3db1-308f14e4-6150de96_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:30.741673Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94bbd679-bb8f3db1-308f14e4-6150de96_0] Write session will restart in 0.009957s
2025-10-09T21:43:30.741698Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94bbd679-bb8f3db1-308f14e4-6150de96_0] Write session: Do CDS request
2025-10-09T21:43:30.741708Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94bbd679-bb8f3db1-308f14e4-6150de96_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:30.752698Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94bbd679-bb8f3db1-308f14e4-6150de96_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:30.752817Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:30.752831Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 148
2025-10-09T21:43:30.753049Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 148 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:30.753090Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 148 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:30.753097Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=148 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:30.753100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:30.754008Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 148 sessionId:  describe result for acl check
2025-10-09T21:43:30.754064Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:30.754066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:30.754067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:30.754073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340912648183145:3432] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:30.754689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340912648183145:3432] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:30.758174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340912648183145:3432] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:30.771794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340912648183145:3432] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:30.771814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340912648183145:3432] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:30.771819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340912648183145:3432] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:30.771828Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 148 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:30.779181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:30.781836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:30.782107Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:30.782112Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:30.782128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:30.782203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:30.782244Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340912286374014:3197]
2025-10-09T21:43:30.782833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:30.783588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:30.783645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:30.783980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:30.784046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:30.784047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:30.784048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:30.784049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:30.784051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:30.784052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:30.784053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.784054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.784055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.784062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:30.784073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:30.784076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:30.784080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:30.784083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:30.784087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:30.784090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.784094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.784097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.784105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:30.784114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:30.784120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:30.784126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:30.784130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:30.784135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:30.784139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:30.784144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:30.784149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:30.784153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:30.784158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:30.784164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:30.784169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.784174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.784179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.784184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.784190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.784195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.784223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:30.799220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:30.800476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:30.800494Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:30.800497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:30.800502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:30.800509Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 147 [2:7559340912286374014:3197]
2025-10-09T21:43:30.800525Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:30.800545Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:30.800549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.800553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.800555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.800575Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:30.800588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:30.800596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:30.800668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.800675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.800678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.800681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:30.800988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:30.800999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.801014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:30.801030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:30.801055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:30.801083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340912648183314:3432], now have 1 active actors on pipe
2025-10-09T21:43:30.801093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340912648183131:2432], now have 1 active actors on pipe
2025-10-09T21:43:30.801102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340912286374008:3197]
2025-10-09T21:43:30.801105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340912286374008:3197]
2025-10-09T21:43:30.801108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340912286374008:3197]
2025-10-09T21:43:30.801122Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:30.801138Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:30.801143Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:30.801148Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:30.801151Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:30.801160Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:30.801167Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:30.801212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340912286374008:3197]
2025-10-09T21:43:30.801226Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:30.801249Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:30.801370Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 147
2025-10-09T21:43:30.801407Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 147
2025-10-09T21:43:30.801448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:30.801464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:30.801499Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f3dad57a-826e82c4-5f4c3a40-a92a2793_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:30.801514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.801525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.801528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:30.801539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.801573Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:30.801599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:30.805827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:30.805844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:30.805892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:30.809474Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 148 partition: 0 MaxSeqNo: 25 sessionId: src_id|f3dad57a-826e82c4-5f4c3a40-a92a2793_0
2025-10-09T21:43:30.809947Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94bbd679-bb8f3db1-308f14e4-6150de96_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f3dad57a-826e82c4-5f4c3a40-a92a2793_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:30.812665Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3dad57a-826e82c4-5f4c3a40-a92a2793_0] Write session: try to update token
2025-10-09T21:43:30.812687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3dad57a-826e82c4-5f4c3a40-a92a2793_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:30.821924Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:30.821943Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:30.821946Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:30.821948Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:30.821958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340912286374008:3197]
2025-10-09T21:43:30.821964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340912286374008:3197]
2025-10-09T21:43:30.821968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340912286374008:3197]
2025-10-09T21:43:30.822015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:30.822022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:30.822026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:30.822059Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:30.822067Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:30.822071Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:30.822204Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:30.832726Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 148 sessionId: src_id|f3dad57a-826e82c4-5f4c3a40-a92a2793_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:30.835481Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:30.842440Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340912648183358:3142]
2025-10-09T21:43:30.843577Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 147, pipe: [1:7559340912648183358:3142]
2025-10-09T21:43:31.025878Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:31.027351Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:31.027352Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:31.027365Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:31.027413Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340912648183358:3142]
2025-10-09T21:43:31.027432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 148 reason: error at write: Disconnected sessionId: src_id|f3dad57a-826e82c4-5f4c3a40-a92a2793_0
2025-10-09T21:43:31.026548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:31.027642Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 148 sessionId: src_id|f3dad57a-826e82c4-5f4c3a40-a92a2793_0 is DEAD
2025-10-09T21:43:31.026564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:31.026611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340912648183358:3142], now have 1 active actors on pipe
2025-10-09T21:43:31.027816Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3dad57a-826e82c4-5f4c3a40-a92a2793_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:31.027823Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3dad57a-826e82c4-5f4c3a40-a92a2793_0] Write session will restart in 0.005338s
2025-10-09T21:43:31.027849Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3dad57a-826e82c4-5f4c3a40-a92a2793_0] Write session: Do CDS request
2025-10-09T21:43:31.027859Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3dad57a-826e82c4-5f4c3a40-a92a2793_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:31.027863Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:31.034626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3dad57a-826e82c4-5f4c3a40-a92a2793_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:31.035071Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:31.035090Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 149
2025-10-09T21:43:31.035242Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 149 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:31.035274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 149 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:31.035284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=149 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:31.035288Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:31.035802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 149 sessionId:  describe result for acl check
2025-10-09T21:43:31.035852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:31.035858Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:31.035860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:31.035867Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340916943150686:3444] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:31.036444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340916943150686:3444] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:31.039498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340916943150686:3444] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:31.049545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340916943150686:3444] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:31.049562Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340916943150686:3444] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:31.049567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340916943150686:3444] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:31.049577Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 149 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:31.051889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:31.053360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:31.053589Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:31.053599Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:31.053611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:31.053680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:31.053729Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340916581341541:3213]
2025-10-09T21:43:31.054390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:31.054809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:31.054859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:31.055228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:31.055283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:31.055291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:31.055292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:31.055293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:31.055295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:31.055296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:31.055297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.055299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.055300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.055306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:31.055317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:31.055321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:31.055325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:31.055330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:31.055333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:31.055337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.055340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.055344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.055352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:31.055361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:31.055369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:31.055375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:31.055383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:31.055389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:31.055393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:31.055400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:31.055405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:31.055410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:31.055415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:31.055420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:31.055425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.055431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.055436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.055441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.055446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.055451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.055483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:31.063426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:31.064939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:31.064955Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:31.064958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:31.064963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:31.064970Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 148 [2:7559340916581341541:3213]
2025-10-09T21:43:31.064987Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:31.065006Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:31.065009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.065013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.065015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.065033Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:31.065045Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:31.065053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:31.065420Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 148
2025-10-09T21:43:31.065118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.065128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.065130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.065435Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 148
2025-10-09T21:43:31.065133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:31.065207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:31.065210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.065224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.065235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.065251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:31.065273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340916581341536:3213]
2025-10-09T21:43:31.065275Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340916581341536:3213]
2025-10-09T21:43:31.065277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340916581341536:3213]
2025-10-09T21:43:31.065287Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:31.065305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340916943150672:2432], now have 1 active actors on pipe
2025-10-09T21:43:31.065308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:31.065309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340916943150860:3444], now have 1 active actors on pipe
2025-10-09T21:43:31.065311Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:31.065313Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:31.065314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:31.065329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:31.065332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:31.065374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340916581341536:3213]
2025-10-09T21:43:31.065383Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:31.065391Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:31.065549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:31.065569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:31.065606Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e8b70ece-c9e9f19-3cb33203-d159aeeb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:31.065616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.065621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.065624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:31.065636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.065662Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:31.065684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:31.066276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:31.066287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:31.066317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:31.066544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 149 partition: 0 MaxSeqNo: 25 sessionId: src_id|e8b70ece-c9e9f19-3cb33203-d159aeeb_0
2025-10-09T21:43:31.066980Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3dad57a-826e82c4-5f4c3a40-a92a2793_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e8b70ece-c9e9f19-3cb33203-d159aeeb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:31.069682Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8b70ece-c9e9f19-3cb33203-d159aeeb_0] Write session: try to update token
2025-10-09T21:43:31.069703Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8b70ece-c9e9f19-3cb33203-d159aeeb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:31.083714Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:31.083733Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:31.083736Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:31.083739Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:31.083749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340916581341536:3213]
2025-10-09T21:43:31.083753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340916581341536:3213]
2025-10-09T21:43:31.083758Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340916581341536:3213]
2025-10-09T21:43:31.083807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:31.083815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.083818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:31.083910Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:31.083921Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:31.083925Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:31.083959Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:31.097570Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 149 sessionId: src_id|e8b70ece-c9e9f19-3cb33203-d159aeeb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:31.100260Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:31.128330Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340916943150881:3142]
2025-10-09T21:43:31.128874Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 148, pipe: [1:7559340916943150881:3142]
2025-10-09T21:43:31.291887Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:31.292296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:31.292314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:31.292337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340916943150881:3142], now have 1 active actors on pipe
2025-10-09T21:43:31.292564Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:31.292709Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:31.292725Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:31.292726Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:31.292770Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340916943150881:3142]
2025-10-09T21:43:31.292804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 149 reason: error at write: Disconnected sessionId: src_id|e8b70ece-c9e9f19-3cb33203-d159aeeb_0
2025-10-09T21:43:31.292991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 149 sessionId: src_id|e8b70ece-c9e9f19-3cb33203-d159aeeb_0 is DEAD
2025-10-09T21:43:31.293168Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8b70ece-c9e9f19-3cb33203-d159aeeb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:31.293180Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8b70ece-c9e9f19-3cb33203-d159aeeb_0] Write session will restart in 0.008843s
2025-10-09T21:43:31.293209Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8b70ece-c9e9f19-3cb33203-d159aeeb_0] Write session: Do CDS request
2025-10-09T21:43:31.293215Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8b70ece-c9e9f19-3cb33203-d159aeeb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:31.303621Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8b70ece-c9e9f19-3cb33203-d159aeeb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:31.303696Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:31.303730Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 150
2025-10-09T21:43:31.304003Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 150 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:31.304048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 150 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:31.304058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=150 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:31.304063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:31.304594Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 150 sessionId:  describe result for acl check
2025-10-09T21:43:31.304637Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:31.304645Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:31.304647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:31.304654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340916943150916:3448] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:31.305260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340916943150916:3448] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:31.310229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340916943150916:3448] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:31.316130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340916943150916:3448] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:31.316148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340916943150916:3448] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:31.316152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340916943150916:3448] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:31.316161Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 150 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:31.318540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:31.319971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:31.320219Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:31.320230Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:31.320256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:31.320318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:31.320376Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340916581341746:3215]
2025-10-09T21:43:31.321014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:31.321458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:31.321538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:31.321903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:31.321970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:31.321972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:31.321973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:31.321974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:31.321975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:31.321977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:31.321978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.321979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.321980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.321986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:31.321997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:31.322001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:31.322005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:31.322009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:31.322012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:31.322015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.322018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.322021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.322050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:31.322059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:31.322065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:31.322070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:31.322074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:31.322080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:31.322084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:31.322088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:31.322093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:31.322098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:31.322103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:31.322107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:31.322112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.322117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.322122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.322128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.322133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.322138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.322169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:31.333543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:31.335169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:31.335175Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:31.335178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:31.335183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:31.335190Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 149 [2:7559340916581341746:3215]
2025-10-09T21:43:31.335207Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:31.335228Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:31.335232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.335236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.335238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.335259Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:31.335269Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:31.335278Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:31.335359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.335361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.335363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.335366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:31.335481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:31.335485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.335506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.335519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.335536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:31.335568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340916581341740:3215]
2025-10-09T21:43:31.335572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340916581341740:3215]
2025-10-09T21:43:31.335575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340916581341740:3215]
2025-10-09T21:43:31.335588Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:31.335613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:31.335631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:31.335637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:31.335689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340916581341740:3215]
2025-10-09T21:43:31.335692Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:31.336020Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:31.336024Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:31.336027Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:31.336034Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:31.336393Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 149
2025-10-09T21:43:31.336433Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 149
2025-10-09T21:43:31.336624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340916943151083:3448], now have 1 active actors on pipe
2025-10-09T21:43:31.336635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340916943150884:2432], now have 1 active actors on pipe
2025-10-09T21:43:31.336651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:31.336657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:31.336704Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|844a7d93-8f38c097-f778f4bc-c49b0076_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:31.336716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.336721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.336724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:31.336738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.336766Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:31.336787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:31.337148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:31.337156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:31.337191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:31.337464Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 150 partition: 0 MaxSeqNo: 25 sessionId: src_id|844a7d93-8f38c097-f778f4bc-c49b0076_0
2025-10-09T21:43:31.337902Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8b70ece-c9e9f19-3cb33203-d159aeeb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|844a7d93-8f38c097-f778f4bc-c49b0076_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:31.340478Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|844a7d93-8f38c097-f778f4bc-c49b0076_0] Write session: try to update token
2025-10-09T21:43:31.340497Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|844a7d93-8f38c097-f778f4bc-c49b0076_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:31.358874Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:31.358893Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:31.358896Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:31.358898Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:31.358909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340916581341740:3215]
2025-10-09T21:43:31.358915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340916581341740:3215]
2025-10-09T21:43:31.358919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340916581341740:3215]
2025-10-09T21:43:31.358968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:31.358976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.358979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:31.359121Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:31.364977Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:31.364994Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:31.364999Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:31.372945Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 150 sessionId: src_id|844a7d93-8f38c097-f778f4bc-c49b0076_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:31.375515Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:31.393438Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340916943151098:3142]
2025-10-09T21:43:31.393972Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 149, pipe: [1:7559340916943151098:3142]
2025-10-09T21:43:31.545762Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:31.546302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:31.546316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:31.546341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340916943151098:3142], now have 1 active actors on pipe
2025-10-09T21:43:31.546780Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:31.546794Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:31.546866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 150 reason: error at write: Disconnected sessionId: src_id|844a7d93-8f38c097-f778f4bc-c49b0076_0
2025-10-09T21:43:31.547063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 150 sessionId: src_id|844a7d93-8f38c097-f778f4bc-c49b0076_0 is DEAD
2025-10-09T21:43:31.547231Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340916943151098:3142]
2025-10-09T21:43:31.547249Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:31.547498Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|844a7d93-8f38c097-f778f4bc-c49b0076_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:31.547511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|844a7d93-8f38c097-f778f4bc-c49b0076_0] Write session will restart in 0.008904s
2025-10-09T21:43:31.547536Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|844a7d93-8f38c097-f778f4bc-c49b0076_0] Write session: Do CDS request
2025-10-09T21:43:31.547547Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|844a7d93-8f38c097-f778f4bc-c49b0076_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:31.546930Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:31.561280Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|844a7d93-8f38c097-f778f4bc-c49b0076_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:31.567573Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:31.567595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 151
2025-10-09T21:43:31.567931Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 151 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:31.567966Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 151 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:31.567972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=151 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:31.567976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:31.568433Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 151 sessionId:  describe result for acl check
2025-10-09T21:43:31.568471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:31.568474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:31.568475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:31.568483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340916943151147:3453] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:31.569025Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340916943151147:3453] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:31.575156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340916943151147:3453] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:31.585771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340916943151147:3453] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:31.585791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340916943151147:3453] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:31.585796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340916943151147:3453] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:31.585806Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 151 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:31.587468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:31.588749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:31.588981Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:31.588985Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:31.588999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:31.589079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:31.589122Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340916581341953:3217]
2025-10-09T21:43:31.589753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:31.590498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:31.590557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:31.590931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:31.590995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:31.590996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:31.590997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:31.590998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:31.590999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:31.591001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:31.591019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.591020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.591021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.591029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:31.591040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:31.591043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:31.591047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:31.591051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:31.591055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:31.591058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.591062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.591065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.591074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:31.591081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:31.591089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:31.591095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:31.591099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:31.591104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:31.591108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:31.591114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:31.591119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:31.591123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:31.591127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:31.591132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:31.591137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.591143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.591147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.591153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.591158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.591163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.591190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:31.601875Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 150
2025-10-09T21:43:31.601916Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 150
2025-10-09T21:43:31.599562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:31.600968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:31.600976Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:31.600978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:31.600983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:31.600992Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 150 [2:7559340916581341953:3217]
2025-10-09T21:43:31.601009Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:31.601030Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:31.601034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.601038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.601041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.601061Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:31.601069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:31.601077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:31.601154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.601156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.601159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.601162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:31.601245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:31.601248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.601264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.601278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.601298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:31.601319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340916581341948:3217]
2025-10-09T21:43:31.601322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340916581341948:3217]
2025-10-09T21:43:31.601325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340916581341948:3217]
2025-10-09T21:43:31.601340Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:31.601358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:31.601377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:31.601381Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:31.601428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340916581341948:3217]
2025-10-09T21:43:31.601431Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:31.601568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:31.601570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:31.601573Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:31.601579Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:31.603749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 151 partition: 0 MaxSeqNo: 25 sessionId: src_id|6fcff224-f7ea0a5-d17bc05-503f7c67_0
2025-10-09T21:43:31.604214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|844a7d93-8f38c097-f778f4bc-c49b0076_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6fcff224-f7ea0a5-d17bc05-503f7c67_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:31.602989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340916943151299:3453], now have 1 active actors on pipe
2025-10-09T21:43:31.603004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340916943151100:2432], now have 1 active actors on pipe
2025-10-09T21:43:31.603025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:31.603031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:31.603067Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6fcff224-f7ea0a5-d17bc05-503f7c67_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:31.603079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.603084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.603087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:31.603101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.603119Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:31.603141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:31.603388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:31.603392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:31.603410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:31.606960Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6fcff224-f7ea0a5-d17bc05-503f7c67_0] Write session: try to update token
2025-10-09T21:43:31.606980Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6fcff224-f7ea0a5-d17bc05-503f7c67_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:31.619830Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:31.619850Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:31.619852Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:31.619854Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:31.619865Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340916581341948:3217]
2025-10-09T21:43:31.619870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340916581341948:3217]
2025-10-09T21:43:31.619873Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340916581341948:3217]
2025-10-09T21:43:31.619943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:31.619951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.619954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:31.619958Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:31.619981Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:31.619985Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:31.620103Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:31.626174Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 151 sessionId: src_id|6fcff224-f7ea0a5-d17bc05-503f7c67_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:31.628897Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:31.647475Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340916943151317:3142]
2025-10-09T21:43:31.647774Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 150, pipe: [1:7559340916943151317:3142]
2025-10-09T21:43:31.793154Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:31.793732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:31.793751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:31.793803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340916943151317:3142], now have 1 active actors on pipe
2025-10-09T21:43:31.794199Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:31.794166Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:31.794179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:31.794254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 151 reason: error at write: Disconnected sessionId: src_id|6fcff224-f7ea0a5-d17bc05-503f7c67_0
2025-10-09T21:43:31.794460Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 151 sessionId: src_id|6fcff224-f7ea0a5-d17bc05-503f7c67_0 is DEAD
2025-10-09T21:43:31.794539Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340916943151317:3142]
2025-10-09T21:43:31.794556Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:31.794874Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6fcff224-f7ea0a5-d17bc05-503f7c67_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:31.794898Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6fcff224-f7ea0a5-d17bc05-503f7c67_0] Write session will restart in 0.006575s
2025-10-09T21:43:31.794931Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6fcff224-f7ea0a5-d17bc05-503f7c67_0] Write session: Do CDS request
2025-10-09T21:43:31.794938Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6fcff224-f7ea0a5-d17bc05-503f7c67_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:31.803821Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6fcff224-f7ea0a5-d17bc05-503f7c67_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:31.804007Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:31.804031Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 152
2025-10-09T21:43:31.804577Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 152 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:31.804623Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 152 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:31.804638Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=152 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:31.804642Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:31.805222Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 152 sessionId:  describe result for acl check
2025-10-09T21:43:31.805266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:31.805273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:31.805275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:31.805282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340916943151374:3460] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:31.805882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340916943151374:3460] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:31.812196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340916943151374:3460] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:31.814990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:31.816279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:31.816514Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:31.816526Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:31.816540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:31.816594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:31.816638Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340916581342165:3220]
2025-10-09T21:43:31.816694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340916943151374:3460] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:31.816713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340916943151374:3460] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:31.816717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340916943151374:3460] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:31.816728Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 152 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:31.817360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:31.817711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:31.817764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:31.818651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:31.818743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:31.818750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:31.818751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:31.818753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:31.818755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:31.818756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:31.818757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.818759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.818760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.818766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:31.818779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:31.818784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:31.818788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:31.818793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:31.818797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:31.818801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.818805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.818810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.818818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:31.818827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:31.818834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:31.818848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:31.818853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:31.818859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:31.818864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:31.818870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:31.818875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:31.818881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:31.818886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:31.818893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:31.818898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.818903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.818908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.818914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.818919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.818925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.818961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:31.828181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:31.831544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:31.831566Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:31.831569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:31.831576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:31.831585Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 151 [2:7559340916581342165:3220]
2025-10-09T21:43:31.831602Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:31.831624Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:31.831628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.831631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.831634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.831653Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:31.831659Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:31.831668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:31.831743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.831751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.831753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.831757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:31.831837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:31.831845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.831861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:31.831878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:31.831897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:31.831920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340916943151327:2432], now have 1 active actors on pipe
2025-10-09T21:43:31.831933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340916581342159:3220]
2025-10-09T21:43:31.831934Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340916581342159:3220]
2025-10-09T21:43:31.831937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340916581342159:3220]
2025-10-09T21:43:31.831945Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:31.831958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:31.831960Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:31.831961Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:31.831962Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:31.831974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:31.831978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:31.832026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340916943151529:3460], now have 1 active actors on pipe
2025-10-09T21:43:31.832035Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340916581342159:3220]
2025-10-09T21:43:31.832045Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:31.832049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:31.832439Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 151
2025-10-09T21:43:31.832494Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 151
2025-10-09T21:43:31.833267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:31.833281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:31.833325Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|761f3785-24f48f73-8533f64a-a2f49372_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:31.833340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.833345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.833348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:31.833360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.833375Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:31.833393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:31.833629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:31.833637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:31.833658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:31.833768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 152 partition: 0 MaxSeqNo: 25 sessionId: src_id|761f3785-24f48f73-8533f64a-a2f49372_0
2025-10-09T21:43:31.834265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6fcff224-f7ea0a5-d17bc05-503f7c67_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|761f3785-24f48f73-8533f64a-a2f49372_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:31.843062Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|761f3785-24f48f73-8533f64a-a2f49372_0] Write session: try to update token
2025-10-09T21:43:31.843089Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|761f3785-24f48f73-8533f64a-a2f49372_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:31.853357Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:31.853379Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:31.853385Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:31.853387Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:31.853398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340916581342159:3220]
2025-10-09T21:43:31.853405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340916581342159:3220]
2025-10-09T21:43:31.853408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340916581342159:3220]
2025-10-09T21:43:31.853460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:31.853473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:31.853476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:31.853500Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:31.853506Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:31.853511Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:31.853616Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:31.873936Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 152 sessionId: src_id|761f3785-24f48f73-8533f64a-a2f49372_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:31.876689Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:31.896445Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340916943151557:3142]
2025-10-09T21:43:31.896952Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 151, pipe: [1:7559340916943151557:3142]
2025-10-09T21:43:32.029421Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:32.029782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:32.029801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:32.029826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340916943151557:3142], now have 1 active actors on pipe
2025-10-09T21:43:32.030073Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:32.031700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|761f3785-24f48f73-8533f64a-a2f49372_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:32.031714Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|761f3785-24f48f73-8533f64a-a2f49372_0] Write session will restart in 0.006539s
2025-10-09T21:43:32.031747Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|761f3785-24f48f73-8533f64a-a2f49372_0] Write session: Do CDS request
2025-10-09T21:43:32.031755Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|761f3785-24f48f73-8533f64a-a2f49372_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:32.031236Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:32.031326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 152 reason: status is not ok: tablet will be restarted right now sessionId: src_id|761f3785-24f48f73-8533f64a-a2f49372_0
2025-10-09T21:43:32.031504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 152 sessionId: src_id|761f3785-24f48f73-8533f64a-a2f49372_0 is DEAD
2025-10-09T21:43:32.031671Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:32.031705Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340916943151557:3142]
2025-10-09T21:43:32.042261Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|761f3785-24f48f73-8533f64a-a2f49372_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:32.042335Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:32.042352Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 153
2025-10-09T21:43:32.042972Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 153 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:32.043007Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 153 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:32.043013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=153 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:32.043016Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:32.043798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 153 sessionId:  describe result for acl check
2025-10-09T21:43:32.043832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:32.043834Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:32.043836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:32.043843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340921238118874:3471] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:32.044891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340921238118874:3471] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:32.047747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340921238118874:3471] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:32.056532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340921238118874:3471] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:32.056552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340921238118874:3471] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:32.056556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340921238118874:3471] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:32.056567Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 153 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:32.085627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:32.087208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:32.087449Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:32.087461Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:32.087476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:32.087534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:32.087584Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340920876309707:3235]
2025-10-09T21:43:32.088244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:32.088671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:32.088718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:32.089048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:32.089093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:32.089094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:32.089095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:32.089097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:32.089098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:32.089099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:32.089100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.089101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.089103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.089122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:32.089134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:32.089138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:32.089143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:32.089147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:32.089151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:32.089155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.089159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.089163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.089171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:32.089179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:32.089186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:32.089191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:32.089196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:32.089202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:32.089206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:32.089210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:32.089215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:32.089219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:32.089222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:32.089227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:32.089231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.089235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.089239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.089244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.089252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.089257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.089282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:32.105958Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 152
2025-10-09T21:43:32.106015Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 152
2025-10-09T21:43:32.103469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:32.104933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:32.104942Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:32.104946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:32.104952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:32.104963Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 152 [2:7559340920876309707:3235]
2025-10-09T21:43:32.104979Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:32.105004Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:32.105009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.105013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.105015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.105038Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:32.105045Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:32.105054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:32.105131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.105134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.105136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.105140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:32.105235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:32.105239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.105256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.105269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.105288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:32.105311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340920876309702:3235]
2025-10-09T21:43:32.105317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340920876309702:3235]
2025-10-09T21:43:32.105319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340920876309702:3235]
2025-10-09T21:43:32.105330Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:32.105348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:32.105361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:32.105365Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:32.105365Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:32.105368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:32.105370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:32.105382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340921238118866:2432], now have 1 active actors on pipe
2025-10-09T21:43:32.105386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340921238118922:3471], now have 1 active actors on pipe
2025-10-09T21:43:32.105432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340920876309702:3235]
2025-10-09T21:43:32.105435Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:32.105442Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:32.108020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:32.108042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:32.108097Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1b26208a-ba734428-b402735c-7fe0480a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:32.108111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.108118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.108121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:32.108149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.108170Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:32.108195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:32.108717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:32.108724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:32.108764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:32.111558Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|761f3785-24f48f73-8533f64a-a2f49372_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1b26208a-ba734428-b402735c-7fe0480a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:32.111097Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 153 partition: 0 MaxSeqNo: 25 sessionId: src_id|1b26208a-ba734428-b402735c-7fe0480a_0
2025-10-09T21:43:32.114325Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b26208a-ba734428-b402735c-7fe0480a_0] Write session: try to update token
2025-10-09T21:43:32.114344Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b26208a-ba734428-b402735c-7fe0480a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:32.130355Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340921238119077:3142]
2025-10-09T21:43:32.132236Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 152, pipe: [1:7559340921238119077:3142]
2025-10-09T21:43:32.132201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340921238119077:3142], now have 1 active actors on pipe
2025-10-09T21:43:32.143610Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:32.143629Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:32.143631Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:32.143633Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:32.143644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340920876309702:3235]
2025-10-09T21:43:32.143649Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340920876309702:3235]
2025-10-09T21:43:32.143653Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340920876309702:3235]
2025-10-09T21:43:32.143700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:32.143708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.143710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:32.144019Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:32.145906Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:32.145912Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:32.145920Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:32.164324Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 153 sessionId: src_id|1b26208a-ba734428-b402735c-7fe0480a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:32.166891Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:32.323282Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:32.323837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:32.323853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:32.324106Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:32.324189Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340921238119077:3142]
2025-10-09T21:43:32.324244Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:32.324286Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:32.324295Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:32.324365Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 153 reason: error at write: Disconnected sessionId: src_id|1b26208a-ba734428-b402735c-7fe0480a_0
2025-10-09T21:43:32.324556Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 153 sessionId: src_id|1b26208a-ba734428-b402735c-7fe0480a_0 is DEAD
2025-10-09T21:43:32.325356Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b26208a-ba734428-b402735c-7fe0480a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:32.325366Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b26208a-ba734428-b402735c-7fe0480a_0] Write session will restart in 0.009560s
2025-10-09T21:43:32.325391Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b26208a-ba734428-b402735c-7fe0480a_0] Write session: Do CDS request
2025-10-09T21:43:32.325414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b26208a-ba734428-b402735c-7fe0480a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:32.337188Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b26208a-ba734428-b402735c-7fe0480a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:32.338551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:32.338569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 154
2025-10-09T21:43:32.338726Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 154 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:32.338759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 154 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:32.338766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=154 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:32.338769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:32.339159Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 154 sessionId:  describe result for acl check
2025-10-09T21:43:32.339190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:32.339192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:32.339194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:32.339202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340921238119097:3475] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:32.339739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340921238119097:3475] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:32.343421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340921238119097:3475] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:32.349448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340921238119097:3475] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:32.349466Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340921238119097:3475] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:32.349471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340921238119097:3475] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:32.349481Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 154 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:32.379329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:32.382969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:32.383206Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:32.383217Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:32.383229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:32.383283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:32.383324Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340920876309914:3237]
2025-10-09T21:43:32.383947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:32.384410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:32.384453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:32.384783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:32.384829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:32.384830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:32.384832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:32.384833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:32.384834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:32.384836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:32.384837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.384838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.384840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.384845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:32.384857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:32.384861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:32.384866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:32.384870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:32.384874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:32.384878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.384882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.384887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.384895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:32.384903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:32.384909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:32.384916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:32.384920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:32.384926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:32.384931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:32.384937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:32.384941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:32.384946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:32.384951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:32.384957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:32.384962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.384968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.384973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.384979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.384984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.384989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.385023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:32.392726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:32.393949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:32.393968Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:32.393971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:32.393975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:32.393984Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 153 [2:7559340920876309914:3237]
2025-10-09T21:43:32.393998Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:32.394020Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:32.394023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.394045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.394053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.394075Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:32.394087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:32.394096Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:32.394162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.394170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.394172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.394175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:32.394704Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 153
2025-10-09T21:43:32.394735Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 153
2025-10-09T21:43:32.394254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:32.394262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.394278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.394288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.394304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:32.394327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340920876309908:3237]
2025-10-09T21:43:32.394329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340920876309908:3237]
2025-10-09T21:43:32.394332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340920876309908:3237]
2025-10-09T21:43:32.394345Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:32.394353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:32.394356Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:32.394359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:32.394362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340921238119168:3475], now have 1 active actors on pipe
2025-10-09T21:43:32.394366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340921238119082:2432], now have 1 active actors on pipe
2025-10-09T21:43:32.394375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:32.394389Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:32.394392Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:32.394444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340920876309908:3237]
2025-10-09T21:43:32.394454Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:32.394910Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:32.398396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:32.398428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:32.398484Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dabde370-841c6927-536e5edb-75edb205_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:32.398498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.398505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.398508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:32.398524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.398553Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:32.398578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:32.401195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:32.401210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:32.401243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:32.401419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 154 partition: 0 MaxSeqNo: 25 sessionId: src_id|dabde370-841c6927-536e5edb-75edb205_0
2025-10-09T21:43:32.401917Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b26208a-ba734428-b402735c-7fe0480a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dabde370-841c6927-536e5edb-75edb205_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:32.404666Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dabde370-841c6927-536e5edb-75edb205_0] Write session: try to update token
2025-10-09T21:43:32.404689Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dabde370-841c6927-536e5edb-75edb205_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:32.424149Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340921238119297:3142]
2025-10-09T21:43:32.424730Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 153, pipe: [1:7559340921238119297:3142]
2025-10-09T21:43:32.424795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340921238119297:3142], now have 1 active actors on pipe
2025-10-09T21:43:32.439995Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:32.440009Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:32.440023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340920876309908:3237]
2025-10-09T21:43:32.440085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:32.440089Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:32.440248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:32.440478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:32.440613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:32.440784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:32.441388Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:32.441444Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:32.441917Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 154 sessionId: src_id|dabde370-841c6927-536e5edb-75edb205_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:32.444595Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:32.444928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:32.444944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:32.444997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:32.447642Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:32.467313Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:32.467331Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:32.467333Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:32.467335Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:32.467347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340920876309908:3237]
2025-10-09T21:43:32.467352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340920876309908:3237]
2025-10-09T21:43:32.467357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340920876309908:3237]
2025-10-09T21:43:32.467399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:32.467407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.467409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:32.467527Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:32.467532Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:32.467536Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:32.467579Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:32.632438Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:32.641235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:32.641260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:32.641436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:32.641489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:32.641549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:32.641602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:32.641641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:32.641649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:32.641747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:32.641803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:32.641856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:32.641909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:32.642363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:32.642374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:32.642448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:32.642502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:32.642556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:32.642619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:32.642655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:32.642658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:32.642720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:32.642774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:32.642866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:32.642923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:32.643102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:32.643105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:32.643175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:32.643235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:32.643332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:32.643395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:32.643457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:32.643460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:32.643536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:32.643601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:32.643676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:32.643746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:32.643792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:32.643795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:32.643859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:32.643913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:32.644016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:32.644106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:32.644143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:32.644149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:32.644279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:32.644368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:32.644429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:32.644523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:32.644574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:32.644577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:32.644724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:32.644828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:32.644914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:32.645005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:32.645059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:32.645062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:32.645179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:32.645229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:32.645336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:32.645411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:32.645478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:32.645481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:32.646873Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:32.645955Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340921238119297:3142]
2025-10-09T21:43:32.646014Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:32.646017Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:32.646088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 154 reason: error at write: Disconnected sessionId: src_id|dabde370-841c6927-536e5edb-75edb205_0
2025-10-09T21:43:32.646284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 154 sessionId: src_id|dabde370-841c6927-536e5edb-75edb205_0 is DEAD
2025-10-09T21:43:32.646446Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:32.650051Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dabde370-841c6927-536e5edb-75edb205_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:32.650064Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dabde370-841c6927-536e5edb-75edb205_0] Write session will restart in 0.007388s
2025-10-09T21:43:32.650090Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dabde370-841c6927-536e5edb-75edb205_0] Write session: Do CDS request
2025-10-09T21:43:32.650096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dabde370-841c6927-536e5edb-75edb205_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:32.658552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dabde370-841c6927-536e5edb-75edb205_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:32.658798Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:32.658814Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 155
2025-10-09T21:43:32.659034Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 155 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:32.659073Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 155 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:32.659084Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=155 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:32.659088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:32.659654Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 155 sessionId:  describe result for acl check
2025-10-09T21:43:32.659703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:32.659706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:32.659708Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:32.659714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340921238119317:3480] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:32.660310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340921238119317:3480] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:32.670457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340921238119317:3480] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:32.682747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340921238119317:3480] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:32.682768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340921238119317:3480] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:32.682772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340921238119317:3480] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:32.682782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 155 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:32.686729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:32.691523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:32.691772Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:32.691785Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:32.691798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:32.691857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:32.691914Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340920876310125:3239]
2025-10-09T21:43:32.692638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:32.693091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:32.693151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:32.693532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:32.693591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:32.693596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:32.693597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:32.693599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:32.693600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:32.693601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:32.693603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.693604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.693606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.693611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:32.693624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:32.693629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:32.693634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:32.693639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:32.693644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:32.693649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.693653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.693659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.693667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:32.693676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:32.693690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:32.693696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:32.693702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:32.693708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:32.693713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:32.693719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:32.693725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:32.693731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:32.693736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:32.693742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:32.693749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.693755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.693760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.693784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.693795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.693802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.693837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:32.700888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:32.702272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:32.702288Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:32.702291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:32.702296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:32.702306Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 154 [2:7559340920876310125:3239]
2025-10-09T21:43:32.702324Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:32.702345Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:32.702348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.702351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.702353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.702373Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:32.702379Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:32.702387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:32.702472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.702474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.702476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.702479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:32.702559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:32.702562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.702575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.702587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.702603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:32.702623Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340920876310120:3239]
2025-10-09T21:43:32.702625Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340920876310120:3239]
2025-10-09T21:43:32.702627Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340920876310120:3239]
2025-10-09T21:43:32.702644Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:32.702662Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:32.702677Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:32.702680Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:32.702720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340920876310120:3239]
2025-10-09T21:43:32.702723Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:32.702727Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:32.702731Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:32.702733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:32.702740Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:32.702892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340921238119302:2432], now have 1 active actors on pipe
2025-10-09T21:43:32.702897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340921238119470:3480], now have 1 active actors on pipe
2025-10-09T21:43:32.703189Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 154
2025-10-09T21:43:32.703244Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 154
2025-10-09T21:43:32.703330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:32.703342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:32.703372Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|96f3aff8-3273c6e0-c79109cf-57df7946_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:32.703386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.703395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.703398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:32.703421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.703439Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:32.703460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:32.703965Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 155 partition: 0 MaxSeqNo: 25 sessionId: src_id|96f3aff8-3273c6e0-c79109cf-57df7946_0
2025-10-09T21:43:32.704498Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dabde370-841c6927-536e5edb-75edb205_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|96f3aff8-3273c6e0-c79109cf-57df7946_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:32.703797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:32.703809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:32.703830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:32.707345Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96f3aff8-3273c6e0-c79109cf-57df7946_0] Write session: try to update token
2025-10-09T21:43:32.707369Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96f3aff8-3273c6e0-c79109cf-57df7946_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:32.720566Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:32.720584Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:32.720586Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:32.720588Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:32.720598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340920876310120:3239]
2025-10-09T21:43:32.720602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340920876310120:3239]
2025-10-09T21:43:32.720606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340920876310120:3239]
2025-10-09T21:43:32.720655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:32.720662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.720665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:32.720810Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:32.722909Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:32.722923Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:32.722926Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:32.730090Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 155 sessionId: src_id|96f3aff8-3273c6e0-c79109cf-57df7946_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:32.732671Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:32.746311Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340921238119519:3142]
2025-10-09T21:43:32.747261Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 154, pipe: [1:7559340921238119519:3142]
2025-10-09T21:43:32.893060Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:32.893513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:32.893523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:32.893546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340921238119519:3142], now have 1 active actors on pipe
2025-10-09T21:43:32.894304Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340921238119519:3142]
2025-10-09T21:43:32.894335Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:32.894367Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:32.894370Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:32.894819Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96f3aff8-3273c6e0-c79109cf-57df7946_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:32.894831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96f3aff8-3273c6e0-c79109cf-57df7946_0] Write session will restart in 0.005534s
2025-10-09T21:43:32.894865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96f3aff8-3273c6e0-c79109cf-57df7946_0] Write session: Do CDS request
2025-10-09T21:43:32.894872Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96f3aff8-3273c6e0-c79109cf-57df7946_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:32.893943Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:32.894434Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 155 reason: error at write: Disconnected sessionId: src_id|96f3aff8-3273c6e0-c79109cf-57df7946_0
2025-10-09T21:43:32.894625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 155 sessionId: src_id|96f3aff8-3273c6e0-c79109cf-57df7946_0 is DEAD
2025-10-09T21:43:32.901667Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96f3aff8-3273c6e0-c79109cf-57df7946_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:32.901757Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:32.901773Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 156
2025-10-09T21:43:32.901978Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 156 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:32.902011Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 156 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:32.902016Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=156 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:32.902022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:32.902484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 156 sessionId:  describe result for acl check
2025-10-09T21:43:32.902517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:32.902519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:32.902521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:32.902526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340921238119540:3485] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:32.903072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340921238119540:3485] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:32.905816Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340921238119540:3485] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:32.908438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340921238119540:3485] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:32.908455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340921238119540:3485] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:32.908458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340921238119540:3485] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:32.908473Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 156 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:32.951803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:32.953293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:32.953543Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:32.953555Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:32.953569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:32.953647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:32.953692Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340920876310370:3249]
2025-10-09T21:43:32.954323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:32.955129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:32.955195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:32.955539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:32.955604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:32.955605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:32.955607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:32.955608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:32.955609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:32.955610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:32.955611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.955612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.955613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.955619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:32.955629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:32.955633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:32.955637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:32.955641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:32.955644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:32.955648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.955651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.955655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.955662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:32.955671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:32.955677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:32.955681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:32.955686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:32.955690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:32.955696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:32.955701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:32.955707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:32.955712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:32.955716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:32.955721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:32.955726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.955731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.955734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.955740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.955744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.955749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.955788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:32.964558Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 155
2025-10-09T21:43:32.964605Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 155
2025-10-09T21:43:32.962727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:32.964122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:32.964128Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:32.964130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:32.964134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:32.964141Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 155 [2:7559340920876310370:3249]
2025-10-09T21:43:32.964155Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:32.964173Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:32.964177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.964180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.964183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.964201Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:32.964207Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:32.964214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:32.964272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.964274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.964278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.964281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:32.964365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:32.964367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.964383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:32.964397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:32.964417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:32.964442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340921238119574:3485], now have 1 active actors on pipe
2025-10-09T21:43:32.964446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340921238119529:2432], now have 1 active actors on pipe
2025-10-09T21:43:32.964454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340920876310364:3249]
2025-10-09T21:43:32.964457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340920876310364:3249]
2025-10-09T21:43:32.964459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340920876310364:3249]
2025-10-09T21:43:32.964474Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:32.964489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:32.964495Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:32.964498Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:32.964500Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:32.964505Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:32.964508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:32.964561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340920876310364:3249]
2025-10-09T21:43:32.964568Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:32.964622Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:32.965566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:32.965578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:32.965612Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d107866-5b393419-781a5a9e-64e457ea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:32.965626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.965630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.965633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:32.965643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.965661Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:32.965682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:32.965980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:32.965991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:32.966013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:32.966249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 156 partition: 0 MaxSeqNo: 25 sessionId: src_id|d107866-5b393419-781a5a9e-64e457ea_0
2025-10-09T21:43:32.970362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96f3aff8-3273c6e0-c79109cf-57df7946_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d107866-5b393419-781a5a9e-64e457ea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:32.973025Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d107866-5b393419-781a5a9e-64e457ea_0] Write session: try to update token
2025-10-09T21:43:32.973044Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d107866-5b393419-781a5a9e-64e457ea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:32.978126Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:32.978144Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:32.978153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340920876310364:3249]
2025-10-09T21:43:32.978198Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:32.978201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:32.978434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:32.978584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:32.978767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:32.980488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:32.980677Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:32.980728Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:32.993937Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:32.993956Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:32.993959Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:32.993961Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:32.993971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340920876310364:3249]
2025-10-09T21:43:32.993975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340920876310364:3249]
2025-10-09T21:43:32.993979Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340920876310364:3249]
2025-10-09T21:43:32.994025Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:32.994068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:32.994068Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:32.994072Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:32.994083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:32.994085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:32.994228Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:32.992111Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 156 sessionId: src_id|d107866-5b393419-781a5a9e-64e457ea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:32.994556Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340921238119769:3142]
2025-10-09T21:43:32.994850Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:32.995301Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 155, pipe: [1:7559340921238119769:3142]
2025-10-09T21:43:33.160241Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:33.160603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340921238119769:3142], now have 1 active actors on pipe
2025-10-09T21:43:33.160634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:33.160641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:33.161534Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340921238119769:3142]
2025-10-09T21:43:33.161569Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:33.161571Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:33.161628Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 156 reason: error at write: Disconnected sessionId: src_id|d107866-5b393419-781a5a9e-64e457ea_0
2025-10-09T21:43:33.161813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 156 sessionId: src_id|d107866-5b393419-781a5a9e-64e457ea_0 is DEAD
2025-10-09T21:43:33.161946Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:33.162224Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d107866-5b393419-781a5a9e-64e457ea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:33.162236Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d107866-5b393419-781a5a9e-64e457ea_0] Write session will restart in 0.007292s
2025-10-09T21:43:33.162262Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d107866-5b393419-781a5a9e-64e457ea_0] Write session: Do CDS request
2025-10-09T21:43:33.162271Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d107866-5b393419-781a5a9e-64e457ea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:33.162905Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:33.170629Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d107866-5b393419-781a5a9e-64e457ea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:33.171168Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:33.171186Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 157
2025-10-09T21:43:33.171599Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 157 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:33.171638Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 157 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:33.171644Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=157 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:33.171648Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:33.172433Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 157 sessionId:  describe result for acl check
2025-10-09T21:43:33.172474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:33.172477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:33.172478Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:33.172485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340925533087076:3496] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:33.173054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340925533087076:3496] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:33.176881Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340925533087076:3496] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:33.196724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340925533087076:3496] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:33.196744Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340925533087076:3496] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:33.196748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340925533087076:3496] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:33.196759Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 157 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:33.211391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:33.213488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:33.213763Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:33.213768Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:33.213784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:33.213850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:33.213897Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340925171277884:3257]
2025-10-09T21:43:33.214552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:33.215203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:33.215263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:33.215610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:33.215782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:33.215783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:33.215785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:33.215786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:33.215787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:33.215788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:33.215790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.215791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.215792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.215800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:33.215812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:33.215816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:33.215820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:33.215824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:33.215829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:33.215833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.215836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.215840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.215848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:33.215856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:33.215862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:33.215867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:33.215874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:33.215879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:33.215884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:33.215890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:33.215894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:33.215900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:33.215904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:33.215910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:33.215915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.215920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.215924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.215930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.215935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.215941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.215973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:33.224534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:33.225997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:33.226018Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:33.226021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:33.226025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:33.226055Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 156 [2:7559340925171277884:3257]
2025-10-09T21:43:33.226076Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:33.226102Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:33.226106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.226110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.226113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.226134Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:33.226150Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:33.226158Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:33.226233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.226235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.226239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.226243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:33.226323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:33.226326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.226339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.226350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.226368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:33.226396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340925533087068:2432], now have 1 active actors on pipe
2025-10-09T21:43:33.226405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340925171277878:3257]
2025-10-09T21:43:33.226407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340925171277878:3257]
2025-10-09T21:43:33.226409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340925171277878:3257]
2025-10-09T21:43:33.226441Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:33.226452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340925533087191:3496], now have 1 active actors on pipe
2025-10-09T21:43:33.226462Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:33.226473Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:33.226478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:33.226478Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:33.226481Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:33.226482Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:33.226529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340925171277878:3257]
2025-10-09T21:43:33.226539Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:33.226568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:33.226724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:33.226737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:33.226782Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fb6a6d38-9ca0dc4e-d813e1be-a1e90db_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:33.226792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.226796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.226799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:33.226813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.226825Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:33.226847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:33.226530Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 156
2025-10-09T21:43:33.226590Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 156
2025-10-09T21:43:33.227116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:33.227128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:33.227146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:33.227242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 157 partition: 0 MaxSeqNo: 25 sessionId: src_id|fb6a6d38-9ca0dc4e-d813e1be-a1e90db_0
2025-10-09T21:43:33.227669Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d107866-5b393419-781a5a9e-64e457ea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fb6a6d38-9ca0dc4e-d813e1be-a1e90db_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:33.231103Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fb6a6d38-9ca0dc4e-d813e1be-a1e90db_0] Write session: try to update token
2025-10-09T21:43:33.231130Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fb6a6d38-9ca0dc4e-d813e1be-a1e90db_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:33.241772Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:33.241791Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:33.241803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340925171277878:3257]
2025-10-09T21:43:33.241853Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:33.241864Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:33.242068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:33.242222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:33.242374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:33.242567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:33.242760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:33.242807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:33.257676Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:33.257697Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:33.257700Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:33.257703Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:33.257715Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340925171277878:3257]
2025-10-09T21:43:33.257718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340925171277878:3257]
2025-10-09T21:43:33.257723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340925171277878:3257]
2025-10-09T21:43:33.257776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:33.257789Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:33.257792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.257795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:33.257796Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:33.257800Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:33.258011Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:33.258336Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 157 sessionId: src_id|fb6a6d38-9ca0dc4e-d813e1be-a1e90db_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:33.261154Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:33.261827Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340925533087287:3142]
2025-10-09T21:43:33.262224Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 156, pipe: [1:7559340925533087287:3142]
2025-10-09T21:43:33.440602Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:33.441104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:33.441116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:33.441140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340925533087287:3142], now have 1 active actors on pipe
2025-10-09T21:43:33.441634Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:33.443541Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340925533087287:3142]
2025-10-09T21:43:33.443579Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:33.443654Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 157 reason: status is not ok: tablet will be restarted right now sessionId: src_id|fb6a6d38-9ca0dc4e-d813e1be-a1e90db_0
2025-10-09T21:43:33.443801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 157 sessionId: src_id|fb6a6d38-9ca0dc4e-d813e1be-a1e90db_0 is DEAD
2025-10-09T21:43:33.444061Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:33.444534Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb6a6d38-9ca0dc4e-d813e1be-a1e90db_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:33.444544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb6a6d38-9ca0dc4e-d813e1be-a1e90db_0] Write session will restart in 0.007388s
2025-10-09T21:43:33.444583Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb6a6d38-9ca0dc4e-d813e1be-a1e90db_0] Write session: Do CDS request
2025-10-09T21:43:33.444590Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb6a6d38-9ca0dc4e-d813e1be-a1e90db_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:33.453672Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fb6a6d38-9ca0dc4e-d813e1be-a1e90db_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:33.454017Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:33.454052Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 158
2025-10-09T21:43:33.454318Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 158 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:33.454358Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 158 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:33.454380Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=158 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:33.454384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:33.455064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 158 sessionId:  describe result for acl check
2025-10-09T21:43:33.455100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:33.455102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:33.455107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:33.455112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340925533087301:3501] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:33.455696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340925533087301:3501] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:33.473397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340925533087301:3501] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:33.484874Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340925533087301:3501] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:33.484903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340925533087301:3501] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:33.484907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340925533087301:3501] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:33.484935Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 158 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:33.509947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:33.510332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:33.510662Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:33.510667Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:33.510683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:33.510757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:33.510800Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340925171278096:3259]
2025-10-09T21:43:33.511393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:33.512266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:33.512321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:33.512658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:33.512741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:33.512746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:33.512747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:33.512748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:33.512749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:33.512750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:33.512751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.512753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.512754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.512760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:33.512774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:33.512779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:33.512783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:33.512787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:33.512791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:33.512794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.512798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.512803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.512812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:33.512821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:33.512830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:33.512835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:33.512839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:33.512844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:33.512849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:33.512853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:33.512858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:33.512863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:33.512866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:33.512871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:33.512876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.512882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.512887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.512892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.512898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.512904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.512934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:33.524890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:33.526315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:33.526331Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:33.526334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:33.526338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:33.526346Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 157 [2:7559340925171278096:3259]
2025-10-09T21:43:33.526366Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:33.526390Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:33.526394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.526397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.526399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.526431Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:33.526444Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:33.526453Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:33.526525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.526533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.526536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.526539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:33.526625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:33.526632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.526647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.526664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.526686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:33.526712Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340925171278090:3259]
2025-10-09T21:43:33.526718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340925171278090:3259]
2025-10-09T21:43:33.526722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340925171278090:3259]
2025-10-09T21:43:33.527484Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 157
2025-10-09T21:43:33.527526Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 157
2025-10-09T21:43:33.526737Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:33.526757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:33.526772Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:33.526775Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:33.526780Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:33.526781Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:33.526783Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:33.526821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340925171278090:3259]
2025-10-09T21:43:33.526824Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:33.526876Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:33.529056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340925533087292:2432], now have 1 active actors on pipe
2025-10-09T21:43:33.529070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340925533087400:3501], now have 1 active actors on pipe
2025-10-09T21:43:33.529093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:33.529102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:33.529146Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|26c24842-283e0212-103072ec-6ceef28b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:33.529159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.529165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.529168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:33.529184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.529219Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:33.529248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:33.530287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:33.530298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:33.530324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:33.530957Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb6a6d38-9ca0dc4e-d813e1be-a1e90db_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|26c24842-283e0212-103072ec-6ceef28b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:33.533734Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26c24842-283e0212-103072ec-6ceef28b_0] Write session: try to update token
2025-10-09T21:43:33.533760Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26c24842-283e0212-103072ec-6ceef28b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:33.530519Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 158 partition: 0 MaxSeqNo: 25 sessionId: src_id|26c24842-283e0212-103072ec-6ceef28b_0
2025-10-09T21:43:33.542534Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340925533087509:3142]
2025-10-09T21:43:33.546688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340925533087509:3142], now have 1 active actors on pipe
2025-10-09T21:43:33.548097Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 157, pipe: [1:7559340925533087509:3142]
2025-10-09T21:43:33.586149Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 158 sessionId: src_id|26c24842-283e0212-103072ec-6ceef28b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:33.588866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:33.586773Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:33.586788Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:33.586792Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:33.586794Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:33.586806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340925171278090:3259]
2025-10-09T21:43:33.586812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340925171278090:3259]
2025-10-09T21:43:33.586819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340925171278090:3259]
2025-10-09T21:43:33.586870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:33.586884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.586886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:33.587049Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:33.589105Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:33.589112Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:33.589120Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:33.759307Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:33.759894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:33.759910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:33.761405Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:33.761479Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340925533087509:3142]
2025-10-09T21:43:33.761508Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:33.761512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:33.761573Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 158 reason: error at write: Disconnected sessionId: src_id|26c24842-283e0212-103072ec-6ceef28b_0
2025-10-09T21:43:33.761762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 158 sessionId: src_id|26c24842-283e0212-103072ec-6ceef28b_0 is DEAD
2025-10-09T21:43:33.762169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26c24842-283e0212-103072ec-6ceef28b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:33.762182Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26c24842-283e0212-103072ec-6ceef28b_0] Write session will restart in 0.005434s
2025-10-09T21:43:33.762230Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26c24842-283e0212-103072ec-6ceef28b_0] Write session: Do CDS request
2025-10-09T21:43:33.762239Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26c24842-283e0212-103072ec-6ceef28b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:33.762469Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:33.770645Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26c24842-283e0212-103072ec-6ceef28b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:33.772821Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:33.772842Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 159
2025-10-09T21:43:33.775081Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 159 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:33.775143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 159 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:33.775150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=159 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:33.775154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:33.775745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 159 sessionId:  describe result for acl check
2025-10-09T21:43:33.775788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:33.775793Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:33.775795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:33.775801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340925533087533:3505] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:33.776925Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340925533087533:3505] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:33.779819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340925533087533:3505] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:33.787157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340925533087533:3505] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:33.787177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340925533087533:3505] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:33.787181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340925533087533:3505] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:33.787192Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 159 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:33.812732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:33.822573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:33.822915Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:33.822931Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:33.822946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:33.823014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:33.823056Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340925171278311:3262]
2025-10-09T21:43:33.823682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:33.827221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:33.827312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:33.827716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:33.827782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:33.827783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:33.827784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:33.827786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:33.827787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:33.827788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:33.827789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.827790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.827792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.827799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:33.827810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:33.827814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:33.827818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:33.827822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:33.827826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:33.827830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.827834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.827837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.827846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:33.827855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:33.827861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:33.827867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:33.827871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:33.827876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:33.827881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:33.827885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:33.827889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:33.827894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:33.827898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:33.827903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:33.827908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.827914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.827918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.827923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.827929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.827937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.827969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:33.847453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:33.849505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:33.849512Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:33.849516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:33.849521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:33.849530Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 158 [2:7559340925171278311:3262]
2025-10-09T21:43:33.849546Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:33.849567Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:33.849570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.849573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.849575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.849596Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:33.849602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:33.849610Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:33.849682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.849684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.849687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.849690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:33.849772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:33.849775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.849789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:33.849798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:33.849813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:33.849834Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340925171278306:3262]
2025-10-09T21:43:33.849836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340925171278306:3262]
2025-10-09T21:43:33.849838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340925171278306:3262]
2025-10-09T21:43:33.849852Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:33.849869Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:33.849883Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:33.849886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:33.849929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340925171278306:3262]
2025-10-09T21:43:33.849931Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:33.850199Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:33.850211Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:33.850212Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:33.850219Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:33.850713Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 158
2025-10-09T21:43:33.850730Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 158
2025-10-09T21:43:33.855494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340925533087520:2432], now have 1 active actors on pipe
2025-10-09T21:43:33.855510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340925533087578:3505], now have 1 active actors on pipe
2025-10-09T21:43:33.855538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:33.855544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:33.855586Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a7c65e23-864f4bc9-2de8c621-ff89b23f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:33.855599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.855607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.855610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:33.855625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.855645Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:33.855665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:33.855940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:33.855946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:33.855971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:33.856116Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 159 partition: 0 MaxSeqNo: 25 sessionId: src_id|a7c65e23-864f4bc9-2de8c621-ff89b23f_0
2025-10-09T21:43:33.856515Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26c24842-283e0212-103072ec-6ceef28b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a7c65e23-864f4bc9-2de8c621-ff89b23f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:33.858632Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340925533087752:3142]
2025-10-09T21:43:33.859105Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 158, pipe: [1:7559340925533087752:3142]
2025-10-09T21:43:33.859158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340925533087752:3142], now have 1 active actors on pipe
2025-10-09T21:43:33.861601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c65e23-864f4bc9-2de8c621-ff89b23f_0] Write session: try to update token
2025-10-09T21:43:33.861623Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c65e23-864f4bc9-2de8c621-ff89b23f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:33.876514Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:33.876533Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:33.876536Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:33.876539Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:33.876550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340925171278306:3262]
2025-10-09T21:43:33.876556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340925171278306:3262]
2025-10-09T21:43:33.876560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340925171278306:3262]
2025-10-09T21:43:33.876607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:33.876615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:33.876618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:33.876758Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:33.881373Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:33.881385Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:33.881388Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:33.880298Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 159 sessionId: src_id|a7c65e23-864f4bc9-2de8c621-ff89b23f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:33.882885Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:34.110764Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:34.111289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:34.111303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:34.111856Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:34.111929Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:34.111932Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:34.112007Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 159 reason: error at write: Disconnected sessionId: src_id|a7c65e23-864f4bc9-2de8c621-ff89b23f_0
2025-10-09T21:43:34.112246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 159 sessionId: src_id|a7c65e23-864f4bc9-2de8c621-ff89b23f_0 is DEAD
2025-10-09T21:43:34.112397Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340925533087752:3142]
2025-10-09T21:43:34.112613Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:34.113618Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c65e23-864f4bc9-2de8c621-ff89b23f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:34.113646Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c65e23-864f4bc9-2de8c621-ff89b23f_0] Write session will restart in 0.008178s
2025-10-09T21:43:34.113671Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c65e23-864f4bc9-2de8c621-ff89b23f_0] Write session: Do CDS request
2025-10-09T21:43:34.113681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c65e23-864f4bc9-2de8c621-ff89b23f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:34.123106Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c65e23-864f4bc9-2de8c621-ff89b23f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:34.126225Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:34.126250Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 160
2025-10-09T21:43:34.130204Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 160 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:34.130262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 160 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:34.130268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=160 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:34.130275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:34.134470Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 160 sessionId:  describe result for acl check
2025-10-09T21:43:34.134521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:34.134523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:34.134525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:34.134533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340929828055104:3517] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:34.135063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340929828055104:3517] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:34.153931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:34.155877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340929828055104:3517] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:34.156564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:34.156840Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:34.156845Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:34.156858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:34.157098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:34.157143Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340929466245861:3277]
2025-10-09T21:43:34.157730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:34.158112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:34.158156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:34.158521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:34.158576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:34.158577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:34.158578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:34.158580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:34.158581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:34.158582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:34.158583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.158585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.158586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.158591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:34.158600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:34.158604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:34.158608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:34.158611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:34.158615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:34.158620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.158624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.158628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.158635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:34.158643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:34.158650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:34.158655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:34.158660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:34.158666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:34.158671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:34.158676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:34.158680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:34.158684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:34.158690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:34.158696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:34.158701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.158706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.158711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.158716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.158721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.158727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.158757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:34.165320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340929828055104:3517] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:34.165339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340929828055104:3517] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:34.165343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340929828055104:3517] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:34.165355Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 160 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:34.170641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:34.171830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:34.171837Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:34.171842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:34.171847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:34.171856Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 159 [2:7559340929466245861:3277]
2025-10-09T21:43:34.171873Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:34.171895Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:34.171900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.171903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.171906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.171930Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:34.171936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:34.171945Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:34.172019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.172022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.172024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.172027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:34.172111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:34.172114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.172131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.172144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.172162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:34.172186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340929466245856:3277]
2025-10-09T21:43:34.172189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340929466245856:3277]
2025-10-09T21:43:34.172191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340929466245856:3277]
2025-10-09T21:43:34.172211Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:34.172230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:34.172246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:34.172250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:34.172301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340929466245856:3277]
2025-10-09T21:43:34.172303Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:34.172843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:34.172853Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:34.172856Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:34.172863Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:34.173039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340929828055079:2432], now have 1 active actors on pipe
2025-10-09T21:43:34.173051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340929828055293:3517], now have 1 active actors on pipe
2025-10-09T21:43:34.173408Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 159
2025-10-09T21:43:34.173418Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 159
2025-10-09T21:43:34.173787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:34.173803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:34.173855Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4a7d2828-99193b8-e973ca6d-16ff384d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:34.173873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.173883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.173886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:34.173901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.173922Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:34.173949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:34.174618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:34.174629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:34.174650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:34.175345Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c65e23-864f4bc9-2de8c621-ff89b23f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4a7d2828-99193b8-e973ca6d-16ff384d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:34.177948Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a7d2828-99193b8-e973ca6d-16ff384d_0] Write session: try to update token
2025-10-09T21:43:34.177966Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a7d2828-99193b8-e973ca6d-16ff384d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:34.174907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 160 partition: 0 MaxSeqNo: 25 sessionId: src_id|4a7d2828-99193b8-e973ca6d-16ff384d_0
2025-10-09T21:43:34.194428Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:34.194448Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:34.194451Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:34.194453Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:34.194464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340929466245856:3277]
2025-10-09T21:43:34.194469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340929466245856:3277]
2025-10-09T21:43:34.194472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340929466245856:3277]
2025-10-09T21:43:34.194522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:34.194535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.194537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:34.194744Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:34.196471Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:34.196485Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:34.196490Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:34.197446Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 160 sessionId: src_id|4a7d2828-99193b8-e973ca6d-16ff384d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:34.200112Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:34.214306Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340929828055301:3142]
2025-10-09T21:43:34.214671Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 159, pipe: [1:7559340929828055301:3142]
2025-10-09T21:43:34.464340Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:34.464756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:34.464772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:34.464823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340929828055301:3142], now have 1 active actors on pipe
2025-10-09T21:43:34.466539Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:34.467360Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:34.467412Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340929828055301:3142]
2025-10-09T21:43:34.467434Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:34.467439Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:34.467497Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 160 reason: error at write: Disconnected sessionId: src_id|4a7d2828-99193b8-e973ca6d-16ff384d_0
2025-10-09T21:43:34.467700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 160 sessionId: src_id|4a7d2828-99193b8-e973ca6d-16ff384d_0 is DEAD
2025-10-09T21:43:34.470738Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a7d2828-99193b8-e973ca6d-16ff384d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:34.470753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a7d2828-99193b8-e973ca6d-16ff384d_0] Write session will restart in 0.008352s
2025-10-09T21:43:34.470780Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a7d2828-99193b8-e973ca6d-16ff384d_0] Write session: Do CDS request
2025-10-09T21:43:34.470787Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a7d2828-99193b8-e973ca6d-16ff384d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:34.480565Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a7d2828-99193b8-e973ca6d-16ff384d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:34.481630Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:34.481647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 161
2025-10-09T21:43:34.481863Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 161 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:34.481899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 161 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:34.481906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=161 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:34.481910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:34.483151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 161 sessionId:  describe result for acl check
2025-10-09T21:43:34.483185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:34.483186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:34.483188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:34.483194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340929828055328:3522] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:34.483752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340929828055328:3522] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:34.501976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340929828055328:3522] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:34.512966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340929828055328:3522] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:34.512983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340929828055328:3522] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:34.512988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340929828055328:3522] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:34.512999Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 161 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:34.514925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:34.518901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:34.519259Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:34.519264Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:34.519281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:34.519357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:34.519397Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340929466246079:3279]
2025-10-09T21:43:34.520008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:34.520472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:34.520515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:34.520832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:34.520876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:34.520877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:34.520878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:34.520879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:34.520881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:34.520882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:34.520883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.520885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.520886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.520891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:34.520902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:34.520906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:34.520910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:34.520914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:34.520918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:34.520922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.520925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.520929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.520936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:34.520944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:34.520951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:34.520957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:34.520962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:34.520967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:34.520971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:34.520976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:34.520981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:34.520986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:34.520991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:34.520996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:34.521002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.521007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.521012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.521017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.521021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.521026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.521058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:34.531927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:34.533329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:34.533336Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:34.533338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:34.533343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:34.533352Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 160 [2:7559340929466246079:3279]
2025-10-09T21:43:34.533367Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:34.533387Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:34.533390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.533394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.533396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.533416Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:34.533422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:34.533430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:34.533501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.533503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.533505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.533508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:34.533588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:34.533591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.533605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.533618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.533636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:34.533656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340929466246073:3279]
2025-10-09T21:43:34.533659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340929466246073:3279]
2025-10-09T21:43:34.533661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340929466246073:3279]
2025-10-09T21:43:34.533670Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:34.533686Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:34.533699Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:34.533702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:34.533744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340929466246073:3279]
2025-10-09T21:43:34.533747Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:34.533765Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:34.533769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:34.533771Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:34.533778Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:34.534435Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 160
2025-10-09T21:43:34.534472Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 160
2025-10-09T21:43:34.534728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340929828055515:3522], now have 1 active actors on pipe
2025-10-09T21:43:34.534740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340929828055307:2432], now have 1 active actors on pipe
2025-10-09T21:43:34.534757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:34.534763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:34.534800Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a9952817-4f27547e-43e969a0-2c3554cd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:34.534810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.534816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.534818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:34.534831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.534845Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:34.534865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:34.536394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:34.536408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:34.536438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:34.536652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 161 partition: 0 MaxSeqNo: 25 sessionId: src_id|a9952817-4f27547e-43e969a0-2c3554cd_0
2025-10-09T21:43:34.537084Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a7d2828-99193b8-e973ca6d-16ff384d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a9952817-4f27547e-43e969a0-2c3554cd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:34.539726Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9952817-4f27547e-43e969a0-2c3554cd_0] Write session: try to update token
2025-10-09T21:43:34.539752Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9952817-4f27547e-43e969a0-2c3554cd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:34.568423Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340929828055528:3142]
2025-10-09T21:43:34.568778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340929828055528:3142], now have 1 active actors on pipe
2025-10-09T21:43:34.571817Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 160, pipe: [1:7559340929828055528:3142]
2025-10-09T21:43:34.572855Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 161 sessionId: src_id|a9952817-4f27547e-43e969a0-2c3554cd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:34.575975Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:34.573609Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:34.573631Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:34.573639Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:34.573641Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:34.573652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340929466246073:3279]
2025-10-09T21:43:34.573656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340929466246073:3279]
2025-10-09T21:43:34.573660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340929466246073:3279]
2025-10-09T21:43:34.573715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:34.573724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.573727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:34.574093Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:34.577154Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:34.577176Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:34.577180Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:34.754886Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:34.755258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:34.755297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:34.755554Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:34.756036Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:34.756095Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:34.756182Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 161 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a9952817-4f27547e-43e969a0-2c3554cd_0
2025-10-09T21:43:34.756378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 161 sessionId: src_id|a9952817-4f27547e-43e969a0-2c3554cd_0 is DEAD
2025-10-09T21:43:34.756528Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340929828055528:3142]
2025-10-09T21:43:34.756617Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9952817-4f27547e-43e969a0-2c3554cd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:34.756631Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9952817-4f27547e-43e969a0-2c3554cd_0] Write session will restart in 0.008352s
2025-10-09T21:43:34.756674Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9952817-4f27547e-43e969a0-2c3554cd_0] Write session: Do CDS request
2025-10-09T21:43:34.756682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9952817-4f27547e-43e969a0-2c3554cd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:34.766810Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9952817-4f27547e-43e969a0-2c3554cd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:34.767912Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:34.767930Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 162
2025-10-09T21:43:34.768118Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 162 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:34.768169Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 162 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:34.768176Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=162 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:34.768181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:34.768612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 162 sessionId:  describe result for acl check
2025-10-09T21:43:34.768646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:34.768648Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:34.768650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:34.768657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340929828055568:3526] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:34.769280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340929828055568:3526] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:34.773366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340929828055568:3526] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:34.780642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340929828055568:3526] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:34.780663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340929828055568:3526] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:34.780667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340929828055568:3526] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:34.780678Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 162 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:34.787309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:34.789406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:34.789798Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:34.789805Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:34.789822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:34.789896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:34.789941Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340929466246300:3282]
2025-10-09T21:43:34.790669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:34.791370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:34.791471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:34.791848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:34.791923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:34.791930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:34.791931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:34.791933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:34.791934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:34.791935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:34.791936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.791938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.791939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.791946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:34.791957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:34.791961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:34.791965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:34.791969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:34.791973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:34.791977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.791981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.791985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.791993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:34.792002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:34.792008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:34.792044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:34.792048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:34.792053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:34.792056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:34.792061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:34.792065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:34.792069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:34.792074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:34.792079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:34.792084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.792088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.792094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.792099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.792105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.792110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.792143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:34.802606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:34.804113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:34.804138Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:34.804141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:34.804146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:34.804153Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 161 [2:7559340929466246300:3282]
2025-10-09T21:43:34.804170Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:34.804191Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:34.804200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.804205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.804207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.804226Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:34.804233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:34.804242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:34.804321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.804323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.804336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.804339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:34.804427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:34.804439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.804454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:34.804469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:34.804486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:34.804507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340929466246294:3282]
2025-10-09T21:43:34.804515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340929466246294:3282]
2025-10-09T21:43:34.804517Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340929466246294:3282]
2025-10-09T21:43:34.804533Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:34.804556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:34.804573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:34.804577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:34.804622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340929466246294:3282]
2025-10-09T21:43:34.804625Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:34.804636Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:34.804643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:34.804645Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:34.804651Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:34.805206Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 161
2025-10-09T21:43:34.805245Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 161
2025-10-09T21:43:34.805093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340929828055714:3526], now have 1 active actors on pipe
2025-10-09T21:43:34.805106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340929828055535:2432], now have 1 active actors on pipe
2025-10-09T21:43:34.805789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:34.805805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:34.805856Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|76be170f-41424dfe-6e54119a-8814566b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:34.805872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.805878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.805881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:34.805894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.805911Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:34.805936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:34.806600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:34.806610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:34.806633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:34.806797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 162 partition: 0 MaxSeqNo: 25 sessionId: src_id|76be170f-41424dfe-6e54119a-8814566b_0
2025-10-09T21:43:34.807340Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9952817-4f27547e-43e969a0-2c3554cd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|76be170f-41424dfe-6e54119a-8814566b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:34.810057Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76be170f-41424dfe-6e54119a-8814566b_0] Write session: try to update token
2025-10-09T21:43:34.810082Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76be170f-41424dfe-6e54119a-8814566b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:34.828881Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 162 sessionId: src_id|76be170f-41424dfe-6e54119a-8814566b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:34.831781Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:34.832028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:34.832053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:34.832108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:34.832265Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:34.839255Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:34.839275Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:34.839277Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:34.839280Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:34.839290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340929466246294:3282]
2025-10-09T21:43:34.839296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340929466246294:3282]
2025-10-09T21:43:34.839300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340929466246294:3282]
2025-10-09T21:43:34.839354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:34.839358Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:34.839363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:34.839366Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:34.839366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:34.839370Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:34.839573Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:34.855171Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340929828055767:3142]
2025-10-09T21:43:34.855414Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 161, pipe: [1:7559340929828055767:3142]
2025-10-09T21:43:35.043110Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:35.043425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340929828055767:3142], now have 1 active actors on pipe
2025-10-09T21:43:35.051589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:35.051616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:35.051772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:35.051834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:35.051899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:35.051959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:35.052006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:35.052019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:35.052080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:35.052139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:35.052196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:35.052257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:35.052293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:35.052299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:35.052392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:35.052444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:35.052499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:35.052555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:35.052601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:35.052606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:35.052673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:35.053015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:35.053076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:35.053152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:35.053290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:35.053294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:35.053361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:35.053446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:35.053550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:35.053654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:35.053714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:35.053717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:35.053800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:35.053865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:35.053935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:35.054009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:35.054244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:35.054258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:35.054326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:35.054409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:35.054486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:35.054547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:35.054605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:35.054608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:35.054679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:35.054745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:35.054835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:35.054927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:35.054982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:35.054985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:35.055089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:35.055181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:35.055314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:35.055446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:35.055506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:35.055509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:35.055612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:35.055669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:35.055727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:35.055819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:35.055874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:35.055877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:35.057100Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:35.057389Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:35.057403Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:35.057430Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 162 reason: error at write: Disconnected sessionId: src_id|76be170f-41424dfe-6e54119a-8814566b_0
2025-10-09T21:43:35.057624Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 162 sessionId: src_id|76be170f-41424dfe-6e54119a-8814566b_0 is DEAD
2025-10-09T21:43:35.057776Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76be170f-41424dfe-6e54119a-8814566b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:35.057788Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76be170f-41424dfe-6e54119a-8814566b_0] Write session will restart in 0.007100s
2025-10-09T21:43:35.057786Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:35.057827Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76be170f-41424dfe-6e54119a-8814566b_0] Write session: Do CDS request
2025-10-09T21:43:35.057823Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340929828055767:3142]
2025-10-09T21:43:35.057834Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76be170f-41424dfe-6e54119a-8814566b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:35.066614Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76be170f-41424dfe-6e54119a-8814566b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:35.066750Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:35.066766Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 163
2025-10-09T21:43:35.066948Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 163 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:35.066979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 163 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:35.066985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=163 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:35.066995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:35.067594Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 163 sessionId:  describe result for acl check
2025-10-09T21:43:35.067628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:35.067634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:35.067636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:35.067640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340934123023093:3537] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:35.068130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340934123023093:3537] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:35.070671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340934123023093:3537] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:35.073676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340934123023093:3537] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:35.073692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340934123023093:3537] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:35.073696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340934123023093:3537] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:35.073704Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 163 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:35.097707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:35.099437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:35.099658Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:35.099668Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:35.099681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:35.099738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:35.099802Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340933761213846:3291]
2025-10-09T21:43:35.100439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:35.100806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:35.100855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:35.101218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:35.101275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:35.101281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:35.101282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:35.101283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:35.101285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:35.101286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:35.101287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.101288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.101290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.101295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:35.101305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:35.101308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:35.101312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:35.101316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:35.101320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:35.101324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.101328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.101332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.101340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:35.101349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:35.101356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:35.101361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:35.101365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:35.101371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:35.101375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:35.101381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:35.101386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:35.101391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:35.101396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:35.101401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:35.101406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.101410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.101414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.101419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.101423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.101428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.101459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:35.110583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:35.111889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:35.111906Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:35.111908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:35.111912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:35.111919Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 162 [2:7559340933761213846:3291]
2025-10-09T21:43:35.111933Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:35.111953Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:35.111957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.111960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.111962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.111981Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:35.111988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:35.112006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:35.112073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.112093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.112095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.112098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:35.112182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:35.112190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.112203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.112213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.112228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:35.112250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340933761213841:3291]
2025-10-09T21:43:35.112253Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340933761213841:3291]
2025-10-09T21:43:35.112255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340933761213841:3291]
2025-10-09T21:43:35.112264Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:35.112283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:35.112303Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:35.112307Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:35.112318Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:35.112321Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:35.112323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:35.112353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340933761213841:3291]
2025-10-09T21:43:35.112357Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:35.112358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:35.112368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340934123023085:2432], now have 1 active actors on pipe
2025-10-09T21:43:35.112372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340934123023136:3537], now have 1 active actors on pipe
2025-10-09T21:43:35.112548Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 162
2025-10-09T21:43:35.112613Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 162
2025-10-09T21:43:35.112768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:35.112782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:35.112811Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e8d02849-13bd6ce1-4d3be2b0-c34b474a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:35.112821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.112825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.112834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:35.112846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.112858Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:35.112876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:35.113126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:35.113139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:35.113164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:35.113283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 163 partition: 0 MaxSeqNo: 25 sessionId: src_id|e8d02849-13bd6ce1-4d3be2b0-c34b474a_0
2025-10-09T21:43:35.113733Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76be170f-41424dfe-6e54119a-8814566b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e8d02849-13bd6ce1-4d3be2b0-c34b474a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:35.116621Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8d02849-13bd6ce1-4d3be2b0-c34b474a_0] Write session: try to update token
2025-10-09T21:43:35.116642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8d02849-13bd6ce1-4d3be2b0-c34b474a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:35.155572Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:35.155593Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:35.155597Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:35.155599Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:35.155611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340933761213841:3291]
2025-10-09T21:43:35.155616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340933761213841:3291]
2025-10-09T21:43:35.155621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340933761213841:3291]
2025-10-09T21:43:35.155674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:35.155682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.155685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:35.155827Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:35.158567Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340934123023319:3142]
2025-10-09T21:43:35.159773Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:35.159791Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:35.159795Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:35.162832Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 162, pipe: [1:7559340934123023319:3142]
2025-10-09T21:43:35.166352Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 163 sessionId: src_id|e8d02849-13bd6ce1-4d3be2b0-c34b474a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:35.188761Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:35.396333Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:35.396648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340934123023319:3142], now have 1 active actors on pipe
2025-10-09T21:43:35.396710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:35.396717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:35.400234Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:35.403089Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8d02849-13bd6ce1-4d3be2b0-c34b474a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:35.403116Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8d02849-13bd6ce1-4d3be2b0-c34b474a_0] Write session will restart in 0.006226s
2025-10-09T21:43:35.403147Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8d02849-13bd6ce1-4d3be2b0-c34b474a_0] Write session: Do CDS request
2025-10-09T21:43:35.403154Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8d02849-13bd6ce1-4d3be2b0-c34b474a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:35.402538Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340934123023319:3142]
2025-10-09T21:43:35.402575Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:35.402614Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:35.402693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 163 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e8d02849-13bd6ce1-4d3be2b0-c34b474a_0
2025-10-09T21:43:35.402893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 163 sessionId: src_id|e8d02849-13bd6ce1-4d3be2b0-c34b474a_0 is DEAD
2025-10-09T21:43:35.414429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8d02849-13bd6ce1-4d3be2b0-c34b474a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:35.416767Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:35.416787Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 164
2025-10-09T21:43:35.418111Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 164 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:35.418148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 164 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:35.418154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=164 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:35.418159Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:35.418651Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 164 sessionId:  describe result for acl check
2025-10-09T21:43:35.418684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:35.418686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:35.418687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:35.418694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340934123023339:3543] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:35.419270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340934123023339:3543] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:35.424606Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340934123023339:3543] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:35.429722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340934123023339:3543] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:35.429742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340934123023339:3543] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:35.429746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340934123023339:3543] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:35.429756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 164 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:35.440482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:35.442629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:35.442987Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:35.442999Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:35.443017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:35.443105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:35.443156Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340933761214072:3299]
2025-10-09T21:43:35.443786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:35.444655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:35.444729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:35.445105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:35.445171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:35.445172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:35.445174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:35.445175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:35.445177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:35.445178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:35.445180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.445181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.445182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.445189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:35.445201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:35.445205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:35.445209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:35.445213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:35.445218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:35.445223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.445227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.445231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.445239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:35.445248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:35.445255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:35.445261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:35.445266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:35.445271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:35.445276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:35.445281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:35.445287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:35.445292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:35.445297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:35.445303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:35.445309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.445315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.445319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.445325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.445331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.445338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.445369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:35.455347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:35.457181Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 163
2025-10-09T21:43:35.457226Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 163
2025-10-09T21:43:35.456590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:35.456607Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:35.456611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:35.456617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:35.456624Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 163 [2:7559340933761214072:3299]
2025-10-09T21:43:35.456640Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:35.456662Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:35.456666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.456669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.456672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.456695Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:35.456703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:35.456711Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:35.456783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.456785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.456788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.456791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:35.457151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:35.457154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.457168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.457179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.457196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:35.457220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340934123023448:3543], now have 1 active actors on pipe
2025-10-09T21:43:35.457223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340934123023326:2432], now have 1 active actors on pipe
2025-10-09T21:43:35.457231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340933761214066:3299]
2025-10-09T21:43:35.457233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340933761214066:3299]
2025-10-09T21:43:35.457235Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340933761214066:3299]
2025-10-09T21:43:35.457261Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:35.457264Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:35.457266Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:35.458234Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:35.458272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:35.458293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:35.458297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:35.458319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:35.458326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:35.458385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340933761214066:3299]
2025-10-09T21:43:35.458396Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:35.458431Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|45a36150-bbb348cc-10ef99c1-af403657_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:35.458449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.458453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.458456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:35.458469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.458484Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:35.458530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:35.458535Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:35.458716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:35.458726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:35.458760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:35.458957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 164 partition: 0 MaxSeqNo: 25 sessionId: src_id|45a36150-bbb348cc-10ef99c1-af403657_0
2025-10-09T21:43:35.459430Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8d02849-13bd6ce1-4d3be2b0-c34b474a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|45a36150-bbb348cc-10ef99c1-af403657_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:35.467959Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45a36150-bbb348cc-10ef99c1-af403657_0] Write session: try to update token
2025-10-09T21:43:35.467989Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45a36150-bbb348cc-10ef99c1-af403657_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:35.480151Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:35.480174Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:35.480176Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:35.480178Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:35.480188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340933761214066:3299]
2025-10-09T21:43:35.480192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340933761214066:3299]
2025-10-09T21:43:35.480197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340933761214066:3299]
2025-10-09T21:43:35.480257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:35.480265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.480268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:35.480432Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:35.482324Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:35.482340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:35.482345Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:35.488998Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 164 sessionId: src_id|45a36150-bbb348cc-10ef99c1-af403657_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:35.492501Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:35.505520Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340934123023550:3142]
2025-10-09T21:43:35.514381Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 163, pipe: [1:7559340934123023550:3142]
2025-10-09T21:43:35.816996Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:35.817369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:35.817381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:35.817406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340934123023550:3142], now have 1 active actors on pipe
2025-10-09T21:43:35.817918Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:35.817785Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:35.818315Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45a36150-bbb348cc-10ef99c1-af403657_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:35.817795Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:35.817801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:35.818328Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45a36150-bbb348cc-10ef99c1-af403657_0] Write session will restart in 0.007394s
2025-10-09T21:43:35.817850Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340934123023550:3142]
2025-10-09T21:43:35.817901Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 164 reason: error at write: Disconnected sessionId: src_id|45a36150-bbb348cc-10ef99c1-af403657_0
2025-10-09T21:43:35.818124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 164 sessionId: src_id|45a36150-bbb348cc-10ef99c1-af403657_0 is DEAD
2025-10-09T21:43:35.818362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45a36150-bbb348cc-10ef99c1-af403657_0] Write session: Do CDS request
2025-10-09T21:43:35.818371Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45a36150-bbb348cc-10ef99c1-af403657_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:35.826860Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45a36150-bbb348cc-10ef99c1-af403657_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:35.830246Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:35.830265Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 165
2025-10-09T21:43:35.830971Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 165 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:35.831030Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 165 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:35.831036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=165 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:35.831041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:35.831888Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 165 sessionId:  describe result for acl check
2025-10-09T21:43:35.831933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:35.831935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:35.831938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:35.831944Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340934123023591:3548] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:35.832557Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340934123023591:3548] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:35.835474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340934123023591:3548] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:35.840111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340934123023591:3548] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:35.840129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340934123023591:3548] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:35.840133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340934123023591:3548] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:35.840145Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 165 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:35.851111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:35.853938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:35.854360Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:35.854374Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:35.854391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:35.854490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:35.854535Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340933761214293:3302]
2025-10-09T21:43:35.855155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:35.855502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:35.855551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:35.855924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:35.855989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:35.855990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:35.855991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:35.855992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:35.855993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:35.855995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:35.855996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.855997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.855999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.856004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:35.856015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:35.856019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:35.856023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:35.856027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:35.856031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:35.856036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.856039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.856043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.856051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:35.856059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:35.856068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:35.856074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:35.856078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:35.856083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:35.856087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:35.856093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:35.856097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:35.856103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:35.856107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:35.856111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:35.856116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.856120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.856124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.856129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.856136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.856141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.856169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:35.865640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:35.867064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:35.867080Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:35.867083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:35.867088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:35.867095Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 164 [2:7559340933761214293:3302]
2025-10-09T21:43:35.867109Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:35.867131Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:35.867134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.867137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.867140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.867159Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:35.867166Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:35.867176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:35.867249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.867257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.867261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.867265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:35.867341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:35.867348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.867361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:35.867372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:35.867387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:35.867409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340933761214288:3302]
2025-10-09T21:43:35.867411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340933761214288:3302]
2025-10-09T21:43:35.867413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340933761214288:3302]
2025-10-09T21:43:35.867424Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:35.867456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340934123023696:3548], now have 1 active actors on pipe
2025-10-09T21:43:35.867460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340934123023567:2432], now have 1 active actors on pipe
2025-10-09T21:43:35.867469Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:35.867485Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:35.867489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:35.867534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340933761214288:3302]
2025-10-09T21:43:35.867543Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:35.867558Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:35.867561Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:35.867563Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:35.867570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:35.867897Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 164
2025-10-09T21:43:35.867938Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 164
2025-10-09T21:43:35.868043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:35.868058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:35.868093Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a3711f68-fba9173a-c6ef81e3-e4e6bc02_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:35.868110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.868120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.868124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:35.868146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.868160Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:35.868183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:35.868641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:35.868650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:35.868669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:35.868783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 165 partition: 0 MaxSeqNo: 25 sessionId: src_id|a3711f68-fba9173a-c6ef81e3-e4e6bc02_0
2025-10-09T21:43:35.869247Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45a36150-bbb348cc-10ef99c1-af403657_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a3711f68-fba9173a-c6ef81e3-e4e6bc02_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:35.871931Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a3711f68-fba9173a-c6ef81e3-e4e6bc02_0] Write session: try to update token
2025-10-09T21:43:35.871951Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a3711f68-fba9173a-c6ef81e3-e4e6bc02_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:35.893560Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:35.893579Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:35.893591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340933761214288:3302]
2025-10-09T21:43:35.893648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:35.893828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:35.893967Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:35.894276Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:35.894403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:35.894560Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:35.894855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:35.894901Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:35.900434Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 165 sessionId: src_id|a3711f68-fba9173a-c6ef81e3-e4e6bc02_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:35.903295Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:35.904120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:35.904141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:35.904185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:35.904422Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:35.918575Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340934123023796:3142]
2025-10-09T21:43:35.918994Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 164, pipe: [1:7559340934123023796:3142]
2025-10-09T21:43:35.916817Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:35.916832Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:35.916836Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:35.916838Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:35.916849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340933761214288:3302]
2025-10-09T21:43:35.916853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340933761214288:3302]
2025-10-09T21:43:35.916856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340933761214288:3302]
2025-10-09T21:43:35.916909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:35.916916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:35.916919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:35.916922Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:35.916930Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:35.916934Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:35.917060Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:36.131389Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:36.131865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340934123023796:3142], now have 1 active actors on pipe
2025-10-09T21:43:36.140395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:36.140427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:36.140608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:36.140671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:36.140735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:36.140797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:36.140842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:36.140847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:36.140915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:36.140979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:36.141043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:36.141111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:36.141152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:36.141155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:36.141224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:36.141283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:36.141394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:36.141455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:36.141496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:36.141498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:36.141571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:36.141640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:36.141706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:36.141807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:36.141847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:36.141850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:36.141925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:36.141994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:36.142292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:36.142445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:36.142515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:36.142518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:36.142607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:36.142679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:36.142761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:36.142835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:36.142972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:36.142976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:36.143047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:36.143110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:36.143175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:36.143244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:36.143381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:36.143385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:36.143453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:36.143518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:36.143576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:36.143637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:36.143680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:36.143683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:36.143752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:36.143808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:36.143878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:36.143964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:36.144346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:36.144350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:36.144415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:36.144475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:36.144576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:36.144641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:36.144689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:36.144692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:36.146117Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:36.146854Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3711f68-fba9173a-c6ef81e3-e4e6bc02_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:36.146867Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3711f68-fba9173a-c6ef81e3-e4e6bc02_0] Write session will restart in 0.005114s
2025-10-09T21:43:36.146899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3711f68-fba9173a-c6ef81e3-e4e6bc02_0] Write session: Do CDS request
2025-10-09T21:43:36.146906Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3711f68-fba9173a-c6ef81e3-e4e6bc02_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:36.146322Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:36.146339Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:36.146369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 165 reason: error at write: Disconnected sessionId: src_id|a3711f68-fba9173a-c6ef81e3-e4e6bc02_0
2025-10-09T21:43:36.146646Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 165 sessionId: src_id|a3711f68-fba9173a-c6ef81e3-e4e6bc02_0 is DEAD
2025-10-09T21:43:36.146796Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340934123023796:3142]
2025-10-09T21:43:36.146826Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:36.158572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a3711f68-fba9173a-c6ef81e3-e4e6bc02_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:36.158925Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:36.158942Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 166
2025-10-09T21:43:36.159122Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 166 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:36.159152Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 166 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:36.159158Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=166 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:36.159161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:36.159739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 166 sessionId:  describe result for acl check
2025-10-09T21:43:36.159776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:36.159778Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:36.159780Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:36.159785Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340938417991123:3558] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:36.160338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340938417991123:3558] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:36.169437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340938417991123:3558] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:36.181544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340938417991123:3558] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:36.181565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340938417991123:3558] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:36.181570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340938417991123:3558] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:36.181580Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 166 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:36.192067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:36.194135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:36.194493Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:36.194498Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:36.194515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:36.194583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:36.194623Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340938056181841:3311]
2025-10-09T21:43:36.195675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:36.198325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:36.198448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:36.198807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:36.198853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:36.198855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:36.198856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:36.198857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:36.198859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:36.198860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:36.198861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.198863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.198864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.198872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:36.198883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:36.198887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:36.198891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:36.198895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:36.198900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:36.198904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.198908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.198912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.198920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:36.198928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:36.198934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:36.198939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:36.198943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:36.198949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:36.198952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:36.198958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:36.198962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:36.198969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:36.198974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:36.198979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:36.198984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.198989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.198993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.198999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.199003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.199008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.199036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:36.213618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:36.214950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:36.214973Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:36.214976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:36.214981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:36.214989Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 165 [2:7559340938056181841:3311]
2025-10-09T21:43:36.215005Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:36.215027Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:36.215031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.215034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.215036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.215058Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:36.215064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:36.215072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:36.215152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.215154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.215156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.215159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:36.215251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:36.215254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.215267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.215277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.215292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:36.215313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340938056181822:3311]
2025-10-09T21:43:36.215316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340938056181822:3311]
2025-10-09T21:43:36.215318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340938056181822:3311]
2025-10-09T21:43:36.215333Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:36.215348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:36.215363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:36.215366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:36.215410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340938056181822:3311]
2025-10-09T21:43:36.215412Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:36.215569Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:36.215571Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:36.215573Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:36.215578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:36.216384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340938417991109:2432], now have 1 active actors on pipe
2025-10-09T21:43:36.216392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340938417991320:3558], now have 1 active actors on pipe
2025-10-09T21:43:36.216411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:36.216417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:36.216447Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|71bdd704-25c9612a-8588d554-de7ff441_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:36.216456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.216460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.216462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:36.216474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.216488Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:36.216507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:36.216735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:36.216741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:36.216757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:36.216138Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 165
2025-10-09T21:43:36.216197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 165
2025-10-09T21:43:36.216897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 166 partition: 0 MaxSeqNo: 25 sessionId: src_id|71bdd704-25c9612a-8588d554-de7ff441_0
2025-10-09T21:43:36.218230Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3711f68-fba9173a-c6ef81e3-e4e6bc02_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|71bdd704-25c9612a-8588d554-de7ff441_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:36.220992Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|71bdd704-25c9612a-8588d554-de7ff441_0] Write session: try to update token
2025-10-09T21:43:36.221017Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|71bdd704-25c9612a-8588d554-de7ff441_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:36.246750Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340938417991343:3142]
2025-10-09T21:43:36.244647Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:36.244663Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:36.244665Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:36.244667Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:36.244675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340938056181822:3311]
2025-10-09T21:43:36.244679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340938056181822:3311]
2025-10-09T21:43:36.244683Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340938056181822:3311]
2025-10-09T21:43:36.244721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:36.244728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.244731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:36.244863Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:36.248374Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:36.248383Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:36.248386Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:36.248632Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 165, pipe: [1:7559340938417991343:3142]
2025-10-09T21:43:36.256817Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 166 sessionId: src_id|71bdd704-25c9612a-8588d554-de7ff441_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:36.259573Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:36.522262Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:36.522716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340938417991343:3142], now have 1 active actors on pipe
2025-10-09T21:43:36.522753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:36.522765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:36.523428Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:36.523479Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:36.523482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:36.523541Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 166 reason: error at write: Disconnected sessionId: src_id|71bdd704-25c9612a-8588d554-de7ff441_0
2025-10-09T21:43:36.523744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 166 sessionId: src_id|71bdd704-25c9612a-8588d554-de7ff441_0 is DEAD
2025-10-09T21:43:36.523857Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340938417991343:3142]
2025-10-09T21:43:36.523981Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:36.526365Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71bdd704-25c9612a-8588d554-de7ff441_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:36.526380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71bdd704-25c9612a-8588d554-de7ff441_0] Write session will restart in 0.006866s
2025-10-09T21:43:36.526407Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71bdd704-25c9612a-8588d554-de7ff441_0] Write session: Do CDS request
2025-10-09T21:43:36.526413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71bdd704-25c9612a-8588d554-de7ff441_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:36.538549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|71bdd704-25c9612a-8588d554-de7ff441_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:36.538720Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:36.538746Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 167
2025-10-09T21:43:36.542195Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 167 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:36.542251Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 167 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:36.542266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=167 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:36.542271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:36.543727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 167 sessionId:  describe result for acl check
2025-10-09T21:43:36.543781Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:36.543783Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:36.543787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:36.543795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340938417991411:3564] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:36.544393Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340938417991411:3564] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:36.550715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340938417991411:3564] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:36.557323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:36.566760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340938417991411:3564] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:36.566779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340938417991411:3564] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:36.566786Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340938417991411:3564] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:36.566797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 167 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:36.566667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:36.566920Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:36.566924Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:36.566939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:36.566999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:36.567040Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340938056182074:3319]
2025-10-09T21:43:36.567635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:36.568615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:36.568673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:36.569015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:36.569070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:36.569071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:36.569072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:36.569074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:36.569076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:36.569077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:36.569078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.569080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.569081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.569087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:36.569098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:36.569102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:36.569107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:36.569110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:36.569114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:36.569117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.569120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.569124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.569131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:36.569140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:36.569155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:36.569160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:36.569165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:36.569169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:36.569173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:36.569178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:36.569183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:36.569189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:36.569193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:36.569197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:36.569203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.569209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.569213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.569218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.569222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.569228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.569259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:36.580052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:36.581120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:36.581127Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:36.581130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:36.581135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:36.581142Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 166 [2:7559340938056182074:3319]
2025-10-09T21:43:36.581159Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:36.581181Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:36.581186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.581189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.581192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.581211Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:36.581220Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:36.581227Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:36.581287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.581289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.581292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.581295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:36.581373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:36.581375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.581388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.581400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.581416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:36.581436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340938056182068:3319]
2025-10-09T21:43:36.581439Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340938056182068:3319]
2025-10-09T21:43:36.581442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340938056182068:3319]
2025-10-09T21:43:36.581451Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:36.581465Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:36.581469Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:36.581473Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:36.581475Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:36.581480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:36.581483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:36.581495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340938417991351:2432], now have 1 active actors on pipe
2025-10-09T21:43:36.581499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340938417991566:3564], now have 1 active actors on pipe
2025-10-09T21:43:36.581663Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 166
2025-10-09T21:43:36.581700Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 166
2025-10-09T21:43:36.581539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340938056182068:3319]
2025-10-09T21:43:36.581548Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:36.581854Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:36.582229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:36.582240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:36.582272Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3f07e901-4c58f112-3879e52b-340c7b9e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:36.582285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.582290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.582292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:36.582305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.582318Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:36.582335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:36.582879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:36.582888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:36.582917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:36.583051Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 167 partition: 0 MaxSeqNo: 25 sessionId: src_id|3f07e901-4c58f112-3879e52b-340c7b9e_0
2025-10-09T21:43:36.583508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71bdd704-25c9612a-8588d554-de7ff441_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3f07e901-4c58f112-3879e52b-340c7b9e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:36.586218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3f07e901-4c58f112-3879e52b-340c7b9e_0] Write session: try to update token
2025-10-09T21:43:36.586238Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3f07e901-4c58f112-3879e52b-340c7b9e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:36.600983Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:36.601000Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:36.601010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340938056182068:3319]
2025-10-09T21:43:36.601058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:36.601229Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:36.601406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:36.601530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:36.601664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:36.601848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:36.601885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:36.602219Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:36.627401Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340938417991581:3142]
2025-10-09T21:43:36.628012Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 166, pipe: [1:7559340938417991581:3142]
2025-10-09T21:43:36.624608Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:36.624625Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:36.624628Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:36.624630Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:36.624640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340938056182068:3319]
2025-10-09T21:43:36.624644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340938056182068:3319]
2025-10-09T21:43:36.624649Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340938056182068:3319]
2025-10-09T21:43:36.624694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:36.624701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.624704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:36.624723Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:36.624727Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:36.624731Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:36.624999Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:36.641379Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 167 sessionId: src_id|3f07e901-4c58f112-3879e52b-340c7b9e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:36.644189Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:36.827974Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:36.828610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340938417991581:3142], now have 1 active actors on pipe
2025-10-09T21:43:36.828648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:36.828656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:36.829416Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340938417991581:3142]
2025-10-09T21:43:36.829444Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:36.829477Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:36.829479Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:36.829537Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 167 reason: error at write: Disconnected sessionId: src_id|3f07e901-4c58f112-3879e52b-340c7b9e_0
2025-10-09T21:43:36.829716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 167 sessionId: src_id|3f07e901-4c58f112-3879e52b-340c7b9e_0 is DEAD
2025-10-09T21:43:36.830529Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3f07e901-4c58f112-3879e52b-340c7b9e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:36.830542Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3f07e901-4c58f112-3879e52b-340c7b9e_0] Write session will restart in 0.009317s
2025-10-09T21:43:36.830567Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3f07e901-4c58f112-3879e52b-340c7b9e_0] Write session: Do CDS request
2025-10-09T21:43:36.830577Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3f07e901-4c58f112-3879e52b-340c7b9e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:36.831733Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:36.841588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3f07e901-4c58f112-3879e52b-340c7b9e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:36.841676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:36.841693Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 168
2025-10-09T21:43:36.842077Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 168 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:36.842115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 168 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:36.842120Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=168 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:36.842124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:36.842527Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 168 sessionId:  describe result for acl check
2025-10-09T21:43:36.842559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:36.842560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:36.842562Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:36.842567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340938417991614:3571] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:36.843092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340938417991614:3571] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:36.848292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340938417991614:3571] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:36.855172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340938417991614:3571] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:36.855195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340938417991614:3571] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:36.855198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340938417991614:3571] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:36.855209Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 168 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:36.876124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:36.879520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:36.879785Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:36.879795Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:36.879810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:36.879876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:36.879919Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340938056182301:3322]
2025-10-09T21:43:36.880320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:36.880725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:36.880772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:36.881005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:36.881055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:36.881062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:36.881063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:36.881065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:36.881066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:36.881067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:36.881069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.881070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.881071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.881077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:36.881088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:36.881092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:36.881096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:36.881101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:36.881105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:36.881110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.881114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.881118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.881127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:36.881140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:36.881146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:36.881151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:36.881155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:36.881160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:36.881164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:36.881168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:36.881173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:36.881177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:36.881182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:36.881190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:36.881194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.881199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.881204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.881209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.881214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.881219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.881249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:36.892188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:36.893300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:36.893318Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:36.893321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:36.893326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:36.893334Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 167 [2:7559340938056182301:3322]
2025-10-09T21:43:36.893344Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:36.893363Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:36.893365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.893368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.893371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.893392Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:36.893398Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:36.893406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:36.893468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.893475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.893476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.893478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:36.893559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:36.893566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.893579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:36.893591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:36.893613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:36.893638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340938056182294:3322]
2025-10-09T21:43:36.893644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340938056182294:3322]
2025-10-09T21:43:36.893647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340938056182294:3322]
2025-10-09T21:43:36.893661Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:36.893678Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:36.893689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:36.893691Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:36.893725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340938056182294:3322]
2025-10-09T21:43:36.893727Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:36.893981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:36.893983Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:36.893985Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:36.893991Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:36.894980Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 167
2025-10-09T21:43:36.895014Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 167
2025-10-09T21:43:36.895713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340938417991711:3571], now have 1 active actors on pipe
2025-10-09T21:43:36.895728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340938417991590:2432], now have 1 active actors on pipe
2025-10-09T21:43:36.895745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:36.895751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:36.895788Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2eab632a-865d2730-c5d68c12-f2ba8961_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:36.895799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.895805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.895808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:36.895821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.895839Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:36.895863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:36.899614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:36.899633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:36.899668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:36.899819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 168 partition: 0 MaxSeqNo: 25 sessionId: src_id|2eab632a-865d2730-c5d68c12-f2ba8961_0
2025-10-09T21:43:36.900264Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3f07e901-4c58f112-3879e52b-340c7b9e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2eab632a-865d2730-c5d68c12-f2ba8961_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:36.902790Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2eab632a-865d2730-c5d68c12-f2ba8961_0] Write session: try to update token
2025-10-09T21:43:36.902812Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2eab632a-865d2730-c5d68c12-f2ba8961_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:36.917517Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:36.917536Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:36.917539Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:36.917540Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:36.917550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340938056182294:3322]
2025-10-09T21:43:36.917555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340938056182294:3322]
2025-10-09T21:43:36.917558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340938056182294:3322]
2025-10-09T21:43:36.917615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:36.917627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:36.917627Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:36.917630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:36.917634Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:36.917638Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:36.917764Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:36.930465Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340938417991823:3142]
2025-10-09T21:43:36.930778Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 167, pipe: [1:7559340938417991823:3142]
2025-10-09T21:43:36.941661Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 168 sessionId: src_id|2eab632a-865d2730-c5d68c12-f2ba8961_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:36.944346Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:37.111547Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:37.112098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340938417991823:3142], now have 1 active actors on pipe
2025-10-09T21:43:37.112136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:37.112144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:37.112581Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:37.112473Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340938417991823:3142]
2025-10-09T21:43:37.112511Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:37.112513Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:37.112574Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 168 reason: error at write: Disconnected sessionId: src_id|2eab632a-865d2730-c5d68c12-f2ba8961_0
2025-10-09T21:43:37.112775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 168 sessionId: src_id|2eab632a-865d2730-c5d68c12-f2ba8961_0 is DEAD
2025-10-09T21:43:37.112822Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:37.113438Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2eab632a-865d2730-c5d68c12-f2ba8961_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:37.113448Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2eab632a-865d2730-c5d68c12-f2ba8961_0] Write session will restart in 0.007336s
2025-10-09T21:43:37.113484Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2eab632a-865d2730-c5d68c12-f2ba8961_0] Write session: Do CDS request
2025-10-09T21:43:37.113491Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2eab632a-865d2730-c5d68c12-f2ba8961_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:37.121596Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2eab632a-865d2730-c5d68c12-f2ba8961_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:37.122255Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:37.122273Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 169
2025-10-09T21:43:37.122488Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 169 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:37.122529Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 169 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:37.122536Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=169 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:37.122539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:37.123445Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 169 sessionId:  describe result for acl check
2025-10-09T21:43:37.123492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:37.123494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:37.123496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:37.123505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340942712959134:3575] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:37.124109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340942712959134:3575] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:37.133498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340942712959134:3575] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:37.145846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340942712959134:3575] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:37.145867Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340942712959134:3575] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:37.145871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340942712959134:3575] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:37.145882Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 169 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:37.158059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:37.160036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:37.160308Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:37.160313Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:37.160328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:37.160398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:37.160443Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340942351149822:3325]
2025-10-09T21:43:37.161029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:37.163232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:37.163309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:37.163685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:37.163784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:37.163786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:37.163787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:37.163788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:37.163790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:37.163791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:37.163792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.163793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.163794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.163801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:37.163813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:37.163817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:37.163821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:37.163825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:37.163828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:37.163833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.163836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.163840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.163850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:37.163859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:37.163866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:37.163875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:37.163883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:37.163887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:37.163891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:37.163896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:37.163904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:37.163909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:37.163913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:37.163918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:37.163923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.163929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.163933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.163938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.163942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.163947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.163976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:37.174727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:37.175950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:37.175970Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:37.175973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:37.175979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:37.175987Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 168 [2:7559340942351149822:3325]
2025-10-09T21:43:37.176011Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:37.176035Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:37.176039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.176043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.176045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.176066Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:37.176078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:37.176087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:37.176169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.176180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.176184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.176188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:37.176568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:37.176578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.176592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.176607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.176624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:37.176650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340942712959263:3575], now have 1 active actors on pipe
2025-10-09T21:43:37.176653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340942712959122:2432], now have 1 active actors on pipe
2025-10-09T21:43:37.176662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340942351149816:3325]
2025-10-09T21:43:37.176665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340942351149816:3325]
2025-10-09T21:43:37.176667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340942351149816:3325]
2025-10-09T21:43:37.176681Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:37.176700Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:37.176716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:37.176719Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:37.176764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340942351149816:3325]
2025-10-09T21:43:37.176766Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:37.176943Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:37.176944Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:37.176946Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:37.176953Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:37.177230Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 168
2025-10-09T21:43:37.177265Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 168
2025-10-09T21:43:37.177593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:37.177605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:37.177641Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|85049bf6-77c1b780-344dbbca-c3877dd5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:37.177665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.177670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.177673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:37.177687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.177702Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:37.177724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:37.178367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:37.178377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:37.178395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:37.178558Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 169 partition: 0 MaxSeqNo: 25 sessionId: src_id|85049bf6-77c1b780-344dbbca-c3877dd5_0
2025-10-09T21:43:37.179068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2eab632a-865d2730-c5d68c12-f2ba8961_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|85049bf6-77c1b780-344dbbca-c3877dd5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:37.181882Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85049bf6-77c1b780-344dbbca-c3877dd5_0] Write session: try to update token
2025-10-09T21:43:37.181902Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85049bf6-77c1b780-344dbbca-c3877dd5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:37.208948Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 169 sessionId: src_id|85049bf6-77c1b780-344dbbca-c3877dd5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:37.211697Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:37.213384Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:37.213401Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:37.213403Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:37.213405Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:37.213416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340942351149816:3325]
2025-10-09T21:43:37.213420Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340942351149816:3325]
2025-10-09T21:43:37.213424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340942351149816:3325]
2025-10-09T21:43:37.213468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:37.213474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.213477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:37.213611Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:37.218076Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:37.218091Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:37.218094Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:37.221935Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340942712959353:3142]
2025-10-09T21:43:37.222376Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 168, pipe: [1:7559340942712959353:3142]
2025-10-09T21:43:37.425101Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:37.425476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:37.425486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:37.425512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340942712959353:3142], now have 1 active actors on pipe
2025-10-09T21:43:37.425851Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:37.426299Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:37.426383Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 169 reason: status is not ok: tablet will be restarted right now sessionId: src_id|85049bf6-77c1b780-344dbbca-c3877dd5_0
2025-10-09T21:43:37.427896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85049bf6-77c1b780-344dbbca-c3877dd5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:37.427908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85049bf6-77c1b780-344dbbca-c3877dd5_0] Write session will restart in 0.006325s
2025-10-09T21:43:37.427931Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85049bf6-77c1b780-344dbbca-c3877dd5_0] Write session: Do CDS request
2025-10-09T21:43:37.427938Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85049bf6-77c1b780-344dbbca-c3877dd5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:37.426573Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 169 sessionId: src_id|85049bf6-77c1b780-344dbbca-c3877dd5_0 is DEAD
2025-10-09T21:43:37.426716Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:37.426754Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340942712959353:3142]
2025-10-09T21:43:37.441116Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85049bf6-77c1b780-344dbbca-c3877dd5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:37.441151Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:37.441171Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 170
2025-10-09T21:43:37.441362Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 170 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:37.441391Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 170 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:37.441397Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=170 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:37.441400Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:37.442075Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 170 sessionId:  describe result for acl check
2025-10-09T21:43:37.442114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:37.442120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:37.442122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:37.442130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340942712959390:3587] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:37.442732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340942712959390:3587] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:37.450786Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340942712959390:3587] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:37.463484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340942712959390:3587] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:37.463502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340942712959390:3587] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:37.463507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340942712959390:3587] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:37.463517Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 170 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:37.501008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:37.501323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:37.501499Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:37.501503Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:37.501514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:37.501558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:37.501601Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340942351150085:3339]
2025-10-09T21:43:37.502158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:37.502970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:37.503029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:37.503383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:37.503449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:37.503450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:37.503451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:37.503452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:37.503454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:37.503455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:37.503456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.503457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.503459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.503465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:37.503477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:37.503482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:37.503486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:37.503491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:37.503495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:37.503499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.503503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.503507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.503515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:37.503524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:37.503532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:37.503537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:37.503542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:37.503547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:37.503552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:37.503557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:37.503562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:37.503568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:37.503573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:37.503579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:37.503584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.503589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.503594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.503599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.503604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.503609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.503640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:37.512518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:37.513993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:37.514002Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:37.514005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:37.514010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:37.514017Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 169 [2:7559340942351150085:3339]
2025-10-09T21:43:37.514047Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:37.514069Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:37.514072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.514076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.514078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.514097Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:37.514103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:37.514111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:37.514183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.514185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.514187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.514190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:37.514270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:37.514273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.514287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.514298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.514312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:37.514334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340942351150079:3339]
2025-10-09T21:43:37.514338Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340942351150079:3339]
2025-10-09T21:43:37.514340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340942351150079:3339]
2025-10-09T21:43:37.514355Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:37.514372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:37.514386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:37.514390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:37.514449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340942351150079:3339]
2025-10-09T21:43:37.514453Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:37.516262Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 169
2025-10-09T21:43:37.516300Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 169
2025-10-09T21:43:37.515113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:37.515125Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:37.515127Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:37.515134Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:37.516531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340942712959442:3587], now have 1 active actors on pipe
2025-10-09T21:43:37.516539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340942712959382:2432], now have 1 active actors on pipe
2025-10-09T21:43:37.516795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:37.516809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:37.516847Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c9963b75-aa706e8-b08fc7b5-67510643_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:37.516861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.516866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.516871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:37.516883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.516903Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:37.516926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:37.517618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:37.517627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:37.517646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:37.518246Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85049bf6-77c1b780-344dbbca-c3877dd5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c9963b75-aa706e8-b08fc7b5-67510643_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:37.520917Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c9963b75-aa706e8-b08fc7b5-67510643_0] Write session: try to update token
2025-10-09T21:43:37.520935Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c9963b75-aa706e8-b08fc7b5-67510643_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:37.517827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 170 partition: 0 MaxSeqNo: 25 sessionId: src_id|c9963b75-aa706e8-b08fc7b5-67510643_0
2025-10-09T21:43:37.526305Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340942712959613:3142]
2025-10-09T21:43:37.526785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340942712959613:3142], now have 1 active actors on pipe
2025-10-09T21:43:37.529033Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 169, pipe: [1:7559340942712959613:3142]
2025-10-09T21:43:37.549359Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:37.549378Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:37.549381Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:37.549383Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:37.549393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340942351150079:3339]
2025-10-09T21:43:37.549398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340942351150079:3339]
2025-10-09T21:43:37.549403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340942351150079:3339]
2025-10-09T21:43:37.549452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:37.549459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.549462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:37.549615Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:37.550394Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:37.550407Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:37.550411Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:37.567733Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 170 sessionId: src_id|c9963b75-aa706e8-b08fc7b5-67510643_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:37.570525Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:37.747229Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:37.747710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:37.747720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:37.752763Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:37.751901Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340942712959613:3142]
2025-10-09T21:43:37.751932Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:37.751967Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:37.751969Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:37.752031Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 170 reason: error at write: Disconnected sessionId: src_id|c9963b75-aa706e8-b08fc7b5-67510643_0
2025-10-09T21:43:37.752206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 170 sessionId: src_id|c9963b75-aa706e8-b08fc7b5-67510643_0 is DEAD
2025-10-09T21:43:37.757648Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9963b75-aa706e8-b08fc7b5-67510643_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:37.757662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9963b75-aa706e8-b08fc7b5-67510643_0] Write session will restart in 0.007187s
2025-10-09T21:43:37.757687Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9963b75-aa706e8-b08fc7b5-67510643_0] Write session: Do CDS request
2025-10-09T21:43:37.757697Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9963b75-aa706e8-b08fc7b5-67510643_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:37.766638Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c9963b75-aa706e8-b08fc7b5-67510643_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:37.770775Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:37.770790Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 171
2025-10-09T21:43:37.770974Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 171 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:37.771009Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 171 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:37.771016Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=171 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:37.771021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:37.771595Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 171 sessionId:  describe result for acl check
2025-10-09T21:43:37.771637Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:37.771639Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:37.771641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:37.771647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340942712959633:3591] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:37.772217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340942712959633:3591] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:37.775677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340942712959633:3591] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:37.781072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340942712959633:3591] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:37.781091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340942712959633:3591] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:37.781096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340942712959633:3591] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:37.781105Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 171 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:37.798337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:37.799907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:37.800139Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:37.800144Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:37.800157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:37.800227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:37.800268Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340942351150314:3341]
2025-10-09T21:43:37.800874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:37.802909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:37.802979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:37.803383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:37.803442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:37.803444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:37.803445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:37.803446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:37.803448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:37.803449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:37.803450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.803451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.803452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.803459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:37.803470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:37.803475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:37.803479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:37.803482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:37.803486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:37.803490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.803493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.803497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.803505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:37.803513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:37.803520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:37.803540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:37.803544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:37.803550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:37.803556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:37.803560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:37.803565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:37.803570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:37.803574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:37.803579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:37.803584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.803590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.803595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.803600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.803605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.803611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.803641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:37.812285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:37.813535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:37.813549Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:37.813552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:37.813557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:37.813565Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 170 [2:7559340942351150314:3341]
2025-10-09T21:43:37.813839Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:37.813870Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:37.813879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.813882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.813884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.813904Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:37.813911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:37.813922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:37.813996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.813999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.814000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.814004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:37.814107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:37.814117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.814132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:37.814146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:37.814507Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 170
2025-10-09T21:43:37.814553Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 170
2025-10-09T21:43:37.814164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:37.814187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340942351150308:3341]
2025-10-09T21:43:37.814189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340942351150308:3341]
2025-10-09T21:43:37.814191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340942351150308:3341]
2025-10-09T21:43:37.814200Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:37.814215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:37.814216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340942712959618:2432], now have 1 active actors on pipe
2025-10-09T21:43:37.814218Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:37.814219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340942712959687:3591], now have 1 active actors on pipe
2025-10-09T21:43:37.814220Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:37.814228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:37.814243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:37.814246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:37.814291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340942351150308:3341]
2025-10-09T21:43:37.814298Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:37.814809Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:37.815069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:37.815077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:37.815118Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a39c9689-d652844f-5347b45e-b4fe0900_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:37.815128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.815133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.815137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:37.815150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.815164Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:37.815182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:37.816047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:37.816061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:37.816082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:37.816249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 171 partition: 0 MaxSeqNo: 25 sessionId: src_id|a39c9689-d652844f-5347b45e-b4fe0900_0
2025-10-09T21:43:37.816801Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9963b75-aa706e8-b08fc7b5-67510643_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a39c9689-d652844f-5347b45e-b4fe0900_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:37.819322Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a39c9689-d652844f-5347b45e-b4fe0900_0] Write session: try to update token
2025-10-09T21:43:37.819343Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a39c9689-d652844f-5347b45e-b4fe0900_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:37.836319Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:37.836343Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:37.836345Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:37.836348Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:37.836357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340942351150308:3341]
2025-10-09T21:43:37.836362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340942351150308:3341]
2025-10-09T21:43:37.836366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340942351150308:3341]
2025-10-09T21:43:37.836410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:37.836417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:37.836420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:37.836429Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:37.836436Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:37.836439Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:37.836591Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:37.847461Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 171 sessionId: src_id|a39c9689-d652844f-5347b45e-b4fe0900_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:37.854212Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340942712959856:3142]
2025-10-09T21:43:37.854556Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:37.857361Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 170, pipe: [1:7559340942712959856:3142]
2025-10-09T21:43:38.006893Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:38.007307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:38.007318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:38.007343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340942712959856:3142], now have 1 active actors on pipe
2025-10-09T21:43:38.007759Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:38.007812Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:38.007816Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:38.007890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 171 reason: error at write: Disconnected sessionId: src_id|a39c9689-d652844f-5347b45e-b4fe0900_0
2025-10-09T21:43:38.008070Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 171 sessionId: src_id|a39c9689-d652844f-5347b45e-b4fe0900_0 is DEAD
2025-10-09T21:43:38.008227Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340942712959856:3142]
2025-10-09T21:43:38.008135Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:38.010952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a39c9689-d652844f-5347b45e-b4fe0900_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:38.010965Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a39c9689-d652844f-5347b45e-b4fe0900_0] Write session will restart in 0.006530s
2025-10-09T21:43:38.010993Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a39c9689-d652844f-5347b45e-b4fe0900_0] Write session: Do CDS request
2025-10-09T21:43:38.011000Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a39c9689-d652844f-5347b45e-b4fe0900_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:38.021292Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a39c9689-d652844f-5347b45e-b4fe0900_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:38.022352Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:38.022369Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 172
2025-10-09T21:43:38.028289Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 172 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:38.028354Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 172 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:38.028361Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=172 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:38.028366Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:38.028949Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 172 sessionId:  describe result for acl check
2025-10-09T21:43:38.028993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:38.028995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:38.028997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:38.029004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340947007927176:3596] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:38.029614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340947007927176:3596] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:38.037084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340947007927176:3596] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:38.050583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340947007927176:3596] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:38.050602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340947007927176:3596] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:38.050607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340947007927176:3596] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:38.050618Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 172 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:38.060487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:38.062839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:38.063094Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:38.063106Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:38.063120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:38.063186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:38.063236Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340946646117839:3345]
2025-10-09T21:43:38.063848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:38.064173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:38.064222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:38.064619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:38.064704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:38.064713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:38.064714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:38.064716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:38.064717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:38.064719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:38.064720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.064721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.064723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.064729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:38.064740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:38.064756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:38.064760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:38.064764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:38.064768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:38.064772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.064776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.064780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.064789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:38.064800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:38.064808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:38.064814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:38.064819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:38.064825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:38.064830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:38.064836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:38.064843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:38.064849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:38.064854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:38.064859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:38.064865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.064871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.064876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.064881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.064887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.064893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.064924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:38.072403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:38.073997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:38.074013Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:38.074016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:38.074020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:38.074050Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 171 [2:7559340946646117839:3345]
2025-10-09T21:43:38.074067Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:38.074090Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:38.074093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.074096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.074099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.074119Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:38.074125Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:38.074132Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:38.074198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.074209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.074211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.074213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:38.074296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:38.074301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.074316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.074328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.074345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:38.074366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340946646117832:3345]
2025-10-09T21:43:38.074369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340946646117832:3345]
2025-10-09T21:43:38.074371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340946646117832:3345]
2025-10-09T21:43:38.074387Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:38.074401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:38.074425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:38.074429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:38.074473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340946646117832:3345]
2025-10-09T21:43:38.074475Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:38.074663Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:38.074665Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:38.074667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:38.074673Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:38.074944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340947007927340:3596], now have 1 active actors on pipe
2025-10-09T21:43:38.074950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340947007927161:2432], now have 1 active actors on pipe
2025-10-09T21:43:38.075203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:38.075213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:38.075248Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c6f33e52-8fba1c33-797dee55-873c9514_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:38.075260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.075264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.075267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:38.075279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.075292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:38.075311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:38.076347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:38.076360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:38.076392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:38.078308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a39c9689-d652844f-5347b45e-b4fe0900_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c6f33e52-8fba1c33-797dee55-873c9514_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:38.075035Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 171
2025-10-09T21:43:38.075070Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 171
2025-10-09T21:43:38.077425Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 172 partition: 0 MaxSeqNo: 25 sessionId: src_id|c6f33e52-8fba1c33-797dee55-873c9514_0
2025-10-09T21:43:38.080907Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c6f33e52-8fba1c33-797dee55-873c9514_0] Write session: try to update token
2025-10-09T21:43:38.080927Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c6f33e52-8fba1c33-797dee55-873c9514_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:38.110118Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340947007927388:3142]
2025-10-09T21:43:38.110619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340947007927388:3142], now have 1 active actors on pipe
2025-10-09T21:43:38.113694Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:38.113710Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:38.113713Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:38.113715Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:38.113725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340946646117832:3345]
2025-10-09T21:43:38.113729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340946646117832:3345]
2025-10-09T21:43:38.113733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340946646117832:3345]
2025-10-09T21:43:38.113781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:38.113790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.113792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:38.113950Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:38.114147Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 171, pipe: [1:7559340947007927388:3142]
2025-10-09T21:43:38.117915Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:38.117932Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:38.117937Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:38.134108Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 172 sessionId: src_id|c6f33e52-8fba1c33-797dee55-873c9514_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:38.136967Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:38.400275Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:38.400578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:38.400598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:38.401202Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:38.401219Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:38.401287Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 172 reason: error at write: Disconnected sessionId: src_id|c6f33e52-8fba1c33-797dee55-873c9514_0
2025-10-09T21:43:38.401477Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 172 sessionId: src_id|c6f33e52-8fba1c33-797dee55-873c9514_0 is DEAD
2025-10-09T21:43:38.401533Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:38.401624Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340947007927388:3142]
2025-10-09T21:43:38.401644Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:38.402334Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6f33e52-8fba1c33-797dee55-873c9514_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:38.402347Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6f33e52-8fba1c33-797dee55-873c9514_0] Write session will restart in 0.005924s
2025-10-09T21:43:38.402374Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6f33e52-8fba1c33-797dee55-873c9514_0] Write session: Do CDS request
2025-10-09T21:43:38.402383Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6f33e52-8fba1c33-797dee55-873c9514_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:38.410576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c6f33e52-8fba1c33-797dee55-873c9514_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:38.410833Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:38.410851Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 173
2025-10-09T21:43:38.411392Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 173 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:38.411435Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 173 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:38.411441Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=173 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:38.411445Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:38.412045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 173 sessionId:  describe result for acl check
2025-10-09T21:43:38.412084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:38.412086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:38.412088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:38.412094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340947007927429:3608] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:38.412683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340947007927429:3608] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:38.420170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340947007927429:3608] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:38.427058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340947007927429:3608] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:38.427078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340947007927429:3608] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:38.427082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340947007927429:3608] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:38.427093Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 173 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:38.447453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:38.458677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:38.459042Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:38.459055Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:38.459070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:38.459138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:38.459188Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340946646118106:3359]
2025-10-09T21:43:38.459765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:38.460208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:38.460246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:38.460546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:38.460585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:38.460586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:38.460587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:38.460588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:38.460590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:38.460591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:38.460592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.460593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.460594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.460599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:38.460610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:38.460613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:38.460616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:38.460620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:38.460623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:38.460627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.460630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.460633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.460641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:38.460649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:38.460654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:38.460662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:38.460668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:38.460673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:38.460677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:38.460682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:38.460687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:38.460692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:38.460696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:38.460701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:38.460706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.460711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.460715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.460720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.460725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.460730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.460760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:38.471817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:38.473210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:38.473216Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:38.473219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:38.473224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:38.473232Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 172 [2:7559340946646118106:3359]
2025-10-09T21:43:38.473252Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:38.473272Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:38.473276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.473279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.473281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.473300Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:38.473306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:38.473314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:38.473386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.473389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.473391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.473394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:38.473483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:38.473486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.473501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.473513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.473531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:38.473555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340946646118100:3359]
2025-10-09T21:43:38.473559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340946646118100:3359]
2025-10-09T21:43:38.473561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340946646118100:3359]
2025-10-09T21:43:38.473576Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:38.473593Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:38.473611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:38.473615Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:38.473656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340946646118100:3359]
2025-10-09T21:43:38.473659Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:38.474010Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:38.474024Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:38.474042Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:38.474049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:38.474394Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 172
2025-10-09T21:43:38.474448Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 172
2025-10-09T21:43:38.474881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340947007927469:3608], now have 1 active actors on pipe
2025-10-09T21:43:38.474893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340947007927422:2432], now have 1 active actors on pipe
2025-10-09T21:43:38.474908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:38.474914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:38.474960Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e762a651-1da92ae1-d94dbd3f-579df32e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:38.474969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.474974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.474982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:38.474995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.475014Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:38.475037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:38.475298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:38.475307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:38.475321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:38.475438Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 173 partition: 0 MaxSeqNo: 25 sessionId: src_id|e762a651-1da92ae1-d94dbd3f-579df32e_0
2025-10-09T21:43:38.476038Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6f33e52-8fba1c33-797dee55-873c9514_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e762a651-1da92ae1-d94dbd3f-579df32e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:38.478688Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e762a651-1da92ae1-d94dbd3f-579df32e_0] Write session: try to update token
2025-10-09T21:43:38.478706Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e762a651-1da92ae1-d94dbd3f-579df32e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:38.501660Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340947007927652:3142]
2025-10-09T21:43:38.501924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340947007927652:3142], now have 1 active actors on pipe
2025-10-09T21:43:38.501998Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 172, pipe: [1:7559340947007927652:3142]
2025-10-09T21:43:38.504012Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 173 sessionId: src_id|e762a651-1da92ae1-d94dbd3f-579df32e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:38.504365Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:38.504386Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:38.504388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:38.504390Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:38.504399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340946646118100:3359]
2025-10-09T21:43:38.504405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340946646118100:3359]
2025-10-09T21:43:38.504408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340946646118100:3359]
2025-10-09T21:43:38.504459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:38.504470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.504473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:38.504617Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:38.506302Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:38.506313Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:38.506317Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:38.506678Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:38.696408Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:38.696690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:38.696700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:38.698445Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:38.698701Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340947007927652:3142]
2025-10-09T21:43:38.698738Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:38.698741Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:38.698814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 173 reason: error at write: Disconnected sessionId: src_id|e762a651-1da92ae1-d94dbd3f-579df32e_0
2025-10-09T21:43:38.698993Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 173 sessionId: src_id|e762a651-1da92ae1-d94dbd3f-579df32e_0 is DEAD
2025-10-09T21:43:38.699166Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:38.702159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e762a651-1da92ae1-d94dbd3f-579df32e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:38.702175Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e762a651-1da92ae1-d94dbd3f-579df32e_0] Write session will restart in 0.006341s
2025-10-09T21:43:38.702201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e762a651-1da92ae1-d94dbd3f-579df32e_0] Write session: Do CDS request
2025-10-09T21:43:38.702210Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e762a651-1da92ae1-d94dbd3f-579df32e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:38.710530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e762a651-1da92ae1-d94dbd3f-579df32e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:38.710664Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:38.710680Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 174
2025-10-09T21:43:38.710824Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 174 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:38.710866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 174 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:38.710873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=174 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:38.710877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:38.711525Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 174 sessionId:  describe result for acl check
2025-10-09T21:43:38.711567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:38.711570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:38.711572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:38.711578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340947007927673:3612] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:38.712148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340947007927673:3612] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:38.732030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340947007927673:3612] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:38.741464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340947007927673:3612] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:38.741482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340947007927673:3612] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:38.741486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340947007927673:3612] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:38.741497Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 174 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:38.766835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:38.772611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:38.772885Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:38.772898Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:38.772912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:38.772983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:38.773035Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340946646118338:3361]
2025-10-09T21:43:38.773619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:38.777431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:38.777530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:38.778226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:38.778305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:38.778306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:38.778308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:38.778309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:38.778310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:38.778312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:38.778313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.778314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.778315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.778322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:38.778333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:38.778337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:38.778341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:38.778345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:38.778349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:38.778352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.778356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.778360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.778368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:38.778378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:38.778384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:38.778396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:38.778401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:38.778406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:38.778411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:38.778432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:38.778437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:38.778443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:38.778451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:38.778456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:38.778464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.778471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.778475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.778480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.778485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.778490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.778518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:38.799001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:38.800514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:38.800523Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:38.800526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:38.800531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:38.800540Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 173 [2:7559340946646118338:3361]
2025-10-09T21:43:38.800558Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:38.800582Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:38.800587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.800590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.800593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.800614Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:38.800620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:38.800628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:38.800705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.800708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.800710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.800713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:38.800798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:38.800802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.800817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:38.800829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:38.800847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:38.800869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340946646118330:3361]
2025-10-09T21:43:38.800871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340946646118330:3361]
2025-10-09T21:43:38.800874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340946646118330:3361]
2025-10-09T21:43:38.800890Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:38.800906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:38.800922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:38.800926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:38.800974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340946646118330:3361]
2025-10-09T21:43:38.800977Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:38.802432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:38.802449Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:38.802452Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:38.802459Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:38.802772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340947007927657:2432], now have 1 active actors on pipe
2025-10-09T21:43:38.802779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340947007927814:3612], now have 1 active actors on pipe
2025-10-09T21:43:38.806111Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340947007927894:3142]
2025-10-09T21:43:38.806210Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 173
2025-10-09T21:43:38.806246Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 173
2025-10-09T21:43:38.808657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:38.808679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:38.808722Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e9edeeac-f6c82930-86d1edbb-57e36a0b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:38.808736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.808742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.808746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:38.808763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.808784Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:38.808810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:38.811660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340947007927894:3142], now have 1 active actors on pipe
2025-10-09T21:43:38.811812Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 173, pipe: [1:7559340947007927894:3142]
2025-10-09T21:43:38.811991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:38.812003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:38.812043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:38.814928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 174 partition: 0 MaxSeqNo: 25 sessionId: src_id|e9edeeac-f6c82930-86d1edbb-57e36a0b_0
2025-10-09T21:43:38.815523Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e762a651-1da92ae1-d94dbd3f-579df32e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e9edeeac-f6c82930-86d1edbb-57e36a0b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:38.818471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9edeeac-f6c82930-86d1edbb-57e36a0b_0] Write session: try to update token
2025-10-09T21:43:38.818500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9edeeac-f6c82930-86d1edbb-57e36a0b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:38.850241Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 174 sessionId: src_id|e9edeeac-f6c82930-86d1edbb-57e36a0b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:38.852931Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:38.858355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:38.858378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:38.858432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:38.862443Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:38.864423Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:38.864438Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:38.864441Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:38.864443Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:38.864452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340946646118330:3361]
2025-10-09T21:43:38.864456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340946646118330:3361]
2025-10-09T21:43:38.864460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340946646118330:3361]
2025-10-09T21:43:38.864505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:38.864515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:38.864518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:38.864526Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:38.864543Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:38.864547Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:38.864663Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:39.075362Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:39.088064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.088095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.088265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:39.088331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:39.088393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:39.088450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:39.088845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:39.088851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:39.088914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:39.088965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:39.089021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:39.089080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:39.089116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:39.089121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:39.089185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:39.089243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:39.089298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:39.089355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:39.089395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:39.089399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:39.089469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:39.089530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:39.089588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:39.089645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:39.089690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:39.089693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:39.089756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:39.089811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:39.089875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:39.089937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:39.089981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:39.089984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:39.090095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:39.090163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:39.090238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:39.090330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:39.090386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:39.090389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:39.090477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:39.090530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:39.090591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:39.090650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:39.090691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:39.090695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:39.090762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:39.090826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:39.090886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:39.090946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:39.090985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:39.090988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:39.091055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:39.091109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:39.091166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:39.091227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:39.091273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:39.091276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:39.091344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:39.091397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:39.091457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:39.091516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:39.091556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:39.091560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:39.092697Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:39.092754Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:39.092757Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:39.092787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 174 reason: error at write: Disconnected sessionId: src_id|e9edeeac-f6c82930-86d1edbb-57e36a0b_0
2025-10-09T21:43:39.092956Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 174 sessionId: src_id|e9edeeac-f6c82930-86d1edbb-57e36a0b_0 is DEAD
2025-10-09T21:43:39.093116Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340947007927894:3142]
2025-10-09T21:43:39.093461Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9edeeac-f6c82930-86d1edbb-57e36a0b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:39.093474Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9edeeac-f6c82930-86d1edbb-57e36a0b_0] Write session will restart in 0.009139s
2025-10-09T21:43:39.093500Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9edeeac-f6c82930-86d1edbb-57e36a0b_0] Write session: Do CDS request
2025-10-09T21:43:39.092875Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:39.093515Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9edeeac-f6c82930-86d1edbb-57e36a0b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:39.103561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9edeeac-f6c82930-86d1edbb-57e36a0b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:39.103999Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:39.104016Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 175
2025-10-09T21:43:39.104122Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 175 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:39.104155Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 175 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:39.104165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=175 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:39.104169Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:39.104607Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 175 sessionId:  describe result for acl check
2025-10-09T21:43:39.104641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:39.104644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:39.104646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:39.104652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340951302895218:3617] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:39.105258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340951302895218:3617] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:39.112486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340951302895218:3617] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:39.122965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340951302895218:3617] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:39.122984Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340951302895218:3617] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:39.122989Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340951302895218:3617] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:39.123000Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 175 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:39.144579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:39.145905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:39.146283Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:39.146297Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:39.146314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:39.146390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:39.146450Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340950941085866:3365]
2025-10-09T21:43:39.147054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:39.147616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:39.147671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:39.148004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:39.148070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:39.148072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:39.148073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:39.148074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:39.148076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:39.148077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:39.148079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.148080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.148081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.148088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:39.148099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:39.148104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:39.148108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:39.148113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:39.148118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:39.148121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.148125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.148128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.148136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:39.148144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:39.148151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:39.148157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:39.148161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:39.148166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:39.148170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:39.148176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:39.148180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:39.148185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:39.148190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:39.148195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:39.148200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.148206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.148210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.148215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.148220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.148225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.148257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:39.157138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:39.158429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:39.158437Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:39.158440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:39.158446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:39.158454Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 174 [2:7559340950941085866:3365]
2025-10-09T21:43:39.158472Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:39.158495Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:39.158499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.158503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.158506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.158528Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:39.158535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:39.158543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:39.158902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.158907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.158909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.158912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:39.159001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:39.159003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.159017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.159027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.159045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:39.159066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340950941085861:3365]
2025-10-09T21:43:39.159069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340950941085861:3365]
2025-10-09T21:43:39.159071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340950941085861:3365]
2025-10-09T21:43:39.159090Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:39.159110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:39.159125Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:39.159128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:39.159158Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:39.159162Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:39.159165Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:39.159171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340950941085861:3365]
2025-10-09T21:43:39.159174Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:39.159253Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:39.159628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340951302895343:3617], now have 1 active actors on pipe
2025-10-09T21:43:39.159642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340951302895204:2432], now have 1 active actors on pipe
2025-10-09T21:43:39.159990Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 174
2025-10-09T21:43:39.160028Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 174
2025-10-09T21:43:39.160830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.160843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.160879Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|28b8429d-70679ee2-f5ef0909-8764cf00_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:39.160892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.160896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.160899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:39.160911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.160925Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:39.160945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:39.161236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.161242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.161258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:39.161390Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 175 partition: 0 MaxSeqNo: 25 sessionId: src_id|28b8429d-70679ee2-f5ef0909-8764cf00_0
2025-10-09T21:43:39.161766Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9edeeac-f6c82930-86d1edbb-57e36a0b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|28b8429d-70679ee2-f5ef0909-8764cf00_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:39.164497Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28b8429d-70679ee2-f5ef0909-8764cf00_0] Write session: try to update token
2025-10-09T21:43:39.164513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28b8429d-70679ee2-f5ef0909-8764cf00_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:39.181036Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:39.181057Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:39.181060Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:39.181062Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:39.181071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340950941085861:3365]
2025-10-09T21:43:39.181077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340950941085861:3365]
2025-10-09T21:43:39.181081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340950941085861:3365]
2025-10-09T21:43:39.181129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:39.181137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.181140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:39.181280Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:39.185623Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:39.185649Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:39.185657Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:39.193538Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340951302895440:3142]
2025-10-09T21:43:39.193646Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 175 sessionId: src_id|28b8429d-70679ee2-f5ef0909-8764cf00_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.194389Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 174, pipe: [1:7559340951302895440:3142]
2025-10-09T21:43:39.196509Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.377856Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:39.378374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340951302895440:3142], now have 1 active actors on pipe
2025-10-09T21:43:39.378401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.378407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.379239Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:39.381322Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:39.381406Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 175 reason: status is not ok: tablet will be restarted right now sessionId: src_id|28b8429d-70679ee2-f5ef0909-8764cf00_0
2025-10-09T21:43:39.381566Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 175 sessionId: src_id|28b8429d-70679ee2-f5ef0909-8764cf00_0 is DEAD
2025-10-09T21:43:39.381641Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:39.381679Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340951302895440:3142]
2025-10-09T21:43:39.384588Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28b8429d-70679ee2-f5ef0909-8764cf00_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:39.384600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28b8429d-70679ee2-f5ef0909-8764cf00_0] Write session will restart in 0.008159s
2025-10-09T21:43:39.384626Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28b8429d-70679ee2-f5ef0909-8764cf00_0] Write session: Do CDS request
2025-10-09T21:43:39.384633Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28b8429d-70679ee2-f5ef0909-8764cf00_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:39.394638Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28b8429d-70679ee2-f5ef0909-8764cf00_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:39.398411Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:39.398440Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 176
2025-10-09T21:43:39.399235Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 176 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:39.399272Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 176 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:39.399277Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=176 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:39.399282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:39.401043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 176 sessionId:  describe result for acl check
2025-10-09T21:43:39.401079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:39.401081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:39.401082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:39.401088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340951302895537:3628] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:39.401617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340951302895537:3628] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:39.411560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340951302895537:3628] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:39.424071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340951302895537:3628] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:39.424102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340951302895537:3628] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:39.424106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340951302895537:3628] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:39.424115Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 176 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:39.425625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:39.427296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:39.427500Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:39.427504Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:39.427516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:39.427579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:39.427613Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340950941086111:3373]
2025-10-09T21:43:39.428224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:39.428864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:39.428920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:39.429249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:39.429297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:39.429298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:39.429299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:39.429300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:39.429302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:39.429303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:39.429304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.429305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.429307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.429312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:39.429322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:39.429327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:39.429331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:39.429335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:39.429339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:39.429344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.429347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.429351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.429359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:39.429367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:39.429373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:39.429379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:39.429384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:39.429389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:39.429394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:39.429400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:39.429405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:39.429428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:39.429433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:39.429441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:39.429447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.429453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.429457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.429462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.429467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.429472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.429503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:39.438876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:39.439973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:39.439980Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:39.439982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:39.439987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:39.439994Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 175 [2:7559340950941086111:3373]
2025-10-09T21:43:39.440008Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:39.440027Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:39.440031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.440034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.440036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.440053Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:39.440059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:39.440068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:39.440130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.440132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.440134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.440137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:39.440204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:39.440207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.440221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.440233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.440252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:39.440277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340951302895679:3628], now have 1 active actors on pipe
2025-10-09T21:43:39.440280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340951302895453:2432], now have 1 active actors on pipe
2025-10-09T21:43:39.440289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340950941086105:3373]
2025-10-09T21:43:39.440291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340950941086105:3373]
2025-10-09T21:43:39.440294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340950941086105:3373]
2025-10-09T21:43:39.440315Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:39.440331Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:39.440347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:39.440351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:39.440392Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340950941086105:3373]
2025-10-09T21:43:39.440395Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:39.440584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:39.440587Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:39.440589Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:39.440597Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:39.440927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 175
2025-10-09T21:43:39.440963Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 175
2025-10-09T21:43:39.441334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.441349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.441383Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|70a420dd-5de19c19-b812c4de-87942ae2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:39.441393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.441407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.441409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:39.441422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.441437Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:39.441454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:39.441808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.441819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.441842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:39.441984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 176 partition: 0 MaxSeqNo: 25 sessionId: src_id|70a420dd-5de19c19-b812c4de-87942ae2_0
2025-10-09T21:43:39.446340Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28b8429d-70679ee2-f5ef0909-8764cf00_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|70a420dd-5de19c19-b812c4de-87942ae2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:39.448973Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70a420dd-5de19c19-b812c4de-87942ae2_0] Write session: try to update token
2025-10-09T21:43:39.448999Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70a420dd-5de19c19-b812c4de-87942ae2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:39.465353Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:39.465369Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:39.465371Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:39.465372Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:39.465382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340950941086105:3373]
2025-10-09T21:43:39.465384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340950941086105:3373]
2025-10-09T21:43:39.465388Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340950941086105:3373]
2025-10-09T21:43:39.465427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:39.465433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.465434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:39.465438Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:39.465445Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:39.465449Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:39.465550Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:39.481681Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340951302895701:3142]
2025-10-09T21:43:39.482252Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 175, pipe: [1:7559340951302895701:3142]
2025-10-09T21:43:39.491951Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 176 sessionId: src_id|70a420dd-5de19c19-b812c4de-87942ae2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.494760Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.654611Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:39.655310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340951302895701:3142], now have 1 active actors on pipe
2025-10-09T21:43:39.655370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.655382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.655600Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:39.656857Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:39.656945Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 176 reason: status is not ok: tablet will be restarted right now sessionId: src_id|70a420dd-5de19c19-b812c4de-87942ae2_0
2025-10-09T21:43:39.657136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 176 sessionId: src_id|70a420dd-5de19c19-b812c4de-87942ae2_0 is DEAD
2025-10-09T21:43:39.657766Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70a420dd-5de19c19-b812c4de-87942ae2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:39.657777Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70a420dd-5de19c19-b812c4de-87942ae2_0] Write session will restart in 0.009367s
2025-10-09T21:43:39.657449Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340951302895701:3142]
2025-10-09T21:43:39.657472Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:39.657802Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70a420dd-5de19c19-b812c4de-87942ae2_0] Write session: Do CDS request
2025-10-09T21:43:39.657839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70a420dd-5de19c19-b812c4de-87942ae2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:39.670608Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70a420dd-5de19c19-b812c4de-87942ae2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:39.670942Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:39.670957Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 177
2025-10-09T21:43:39.671127Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 177 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:39.671157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 177 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:39.671163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=177 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:39.671167Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:39.678574Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 177 sessionId:  describe result for acl check
2025-10-09T21:43:39.678618Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:39.678620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:39.678621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:39.678627Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340951302895744:3633] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:39.679162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340951302895744:3633] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:39.688833Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340951302895744:3633] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:39.699480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340951302895744:3633] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:39.699497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340951302895744:3633] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:39.699501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340951302895744:3633] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:39.699512Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 177 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:39.711503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:39.715495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:39.715765Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:39.715770Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:39.715784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:39.715853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:39.715911Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340950941086362:3381]
2025-10-09T21:43:39.716538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:39.722566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:39.722672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:39.723077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:39.723140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:39.723141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:39.723143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:39.723144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:39.723146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:39.723147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:39.723149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.723150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.723151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.723158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:39.723168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:39.723172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:39.723175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:39.723178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:39.723182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:39.723185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.723189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.723192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.723201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:39.723209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:39.723216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:39.723223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:39.723228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:39.723244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:39.723249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:39.723255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:39.723260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:39.723265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:39.723269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:39.723274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:39.723280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.723286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.723290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.723296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.723300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.723306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.723338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:39.735854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:39.737203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:39.737209Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:39.737211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:39.737217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:39.737225Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 176 [2:7559340950941086362:3381]
2025-10-09T21:43:39.737243Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:39.737265Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:39.737268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.737271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.737274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.737293Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:39.737299Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:39.737306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:39.737372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.737374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.737376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.737379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:39.737462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:39.737465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.737477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:39.737489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:39.737505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:39.737525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340950941086357:3381]
2025-10-09T21:43:39.737528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340950941086357:3381]
2025-10-09T21:43:39.737530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340950941086357:3381]
2025-10-09T21:43:39.737549Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:39.737563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:39.737578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:39.737580Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:39.737621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340950941086357:3381]
2025-10-09T21:43:39.737623Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:39.737810Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:39.737813Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:39.737816Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:39.737823Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:39.737885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340951302895706:2432], now have 1 active actors on pipe
2025-10-09T21:43:39.737893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340951302895866:3633], now have 1 active actors on pipe
2025-10-09T21:43:39.738235Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 176
2025-10-09T21:43:39.738284Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 176
2025-10-09T21:43:39.738487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.738500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.738530Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|384e9f30-4b709bf0-b3117678-a1118814_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:39.738541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.738546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.738549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:39.738561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.738575Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:39.738592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:39.742839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.742855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.742894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:39.743065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 177 partition: 0 MaxSeqNo: 25 sessionId: src_id|384e9f30-4b709bf0-b3117678-a1118814_0
2025-10-09T21:43:39.744664Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70a420dd-5de19c19-b812c4de-87942ae2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|384e9f30-4b709bf0-b3117678-a1118814_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:39.747801Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|384e9f30-4b709bf0-b3117678-a1118814_0] Write session: try to update token
2025-10-09T21:43:39.747823Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|384e9f30-4b709bf0-b3117678-a1118814_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:39.757565Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340951302895940:3142]
2025-10-09T21:43:39.758430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340951302895940:3142], now have 1 active actors on pipe
2025-10-09T21:43:39.762127Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 176, pipe: [1:7559340951302895940:3142]
2025-10-09T21:43:39.764468Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:39.764486Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:39.764488Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:39.764491Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:39.764505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340950941086357:3381]
2025-10-09T21:43:39.764510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340950941086357:3381]
2025-10-09T21:43:39.764514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340950941086357:3381]
2025-10-09T21:43:39.764566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:39.764579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:39.764582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:39.764728Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:39.766006Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:39.766012Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:39.766017Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:39.773493Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 177 sessionId: src_id|384e9f30-4b709bf0-b3117678-a1118814_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.776161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.998707Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:40.021076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:40.021102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:40.026692Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:40.026750Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340951302895940:3142]
2025-10-09T21:43:40.026864Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:40.026867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:40.026931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 177 reason: error at write: Disconnected sessionId: src_id|384e9f30-4b709bf0-b3117678-a1118814_0
2025-10-09T21:43:40.027140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 177 sessionId: src_id|384e9f30-4b709bf0-b3117678-a1118814_0 is DEAD
2025-10-09T21:43:40.027643Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|384e9f30-4b709bf0-b3117678-a1118814_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:40.027650Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|384e9f30-4b709bf0-b3117678-a1118814_0] Write session will restart in 0.005086s
2025-10-09T21:43:40.027670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|384e9f30-4b709bf0-b3117678-a1118814_0] Write session: Do CDS request
2025-10-09T21:43:40.027676Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|384e9f30-4b709bf0-b3117678-a1118814_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:40.030252Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:40.037743Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|384e9f30-4b709bf0-b3117678-a1118814_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:40.042125Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:40.042147Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 178
2025-10-09T21:43:40.042627Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 178 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:40.042672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 178 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:40.042679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=178 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:40.042683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:40.043196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 178 sessionId:  describe result for acl check
2025-10-09T21:43:40.043240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:40.043243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:40.043245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:40.043251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340955597863267:3638] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:40.043854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340955597863267:3638] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:40.046179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340955597863267:3638] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:40.057963Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340955597863267:3638] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:40.057981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340955597863267:3638] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:40.057985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340955597863267:3638] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:40.057995Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 178 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:40.092555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:40.094314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:40.094594Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:40.094598Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:40.094612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:40.094680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:40.094723Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340955236053891:3385]
2025-10-09T21:43:40.095363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:40.096026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:40.096079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:40.096395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:40.096456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:40.096457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:40.096458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:40.096459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:40.096461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:40.096462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:40.096463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.096464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.096466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.096472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:40.096484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:40.096487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:40.096491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:40.096495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:40.096499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:40.096502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.096506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.096508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.096516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:40.096524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:40.096530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:40.096535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:40.096538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:40.096543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:40.096547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:40.096552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:40.096557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:40.096561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:40.096566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:40.096570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:40.096575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.096580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.096585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.096605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.096610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.096615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.096643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:40.105058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:40.106534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:40.106550Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:40.106553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:40.106558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:40.106566Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 177 [2:7559340955236053891:3385]
2025-10-09T21:43:40.106584Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:40.106605Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:40.106613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.106616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.106619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.106640Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:40.106647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:40.106655Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:40.106995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.107004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.107006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.107009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:40.107092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:40.107098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.107114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.107124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.107141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:40.107163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340955236053885:3385]
2025-10-09T21:43:40.107165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340955236053885:3385]
2025-10-09T21:43:40.107167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340955236053885:3385]
2025-10-09T21:43:40.107176Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:40.107182Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:40.107187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:40.107189Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:40.107196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340955597863320:3638], now have 1 active actors on pipe
2025-10-09T21:43:40.107199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340955597863255:2432], now have 1 active actors on pipe
2025-10-09T21:43:40.107206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:40.107219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:40.107222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:40.107263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340955236053885:3385]
2025-10-09T21:43:40.107270Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:40.107326Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:40.108024Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 177
2025-10-09T21:43:40.108061Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 177
2025-10-09T21:43:40.108401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:40.108414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:40.108452Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6402f27c-e039b20-644ccccc-30752ff5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:40.108470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.108478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.108481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:40.108500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.108517Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:40.108540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:40.109114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:40.109124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:40.109145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:40.109310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 178 partition: 0 MaxSeqNo: 25 sessionId: src_id|6402f27c-e039b20-644ccccc-30752ff5_0
2025-10-09T21:43:40.109758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|384e9f30-4b709bf0-b3117678-a1118814_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6402f27c-e039b20-644ccccc-30752ff5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:40.112510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6402f27c-e039b20-644ccccc-30752ff5_0] Write session: try to update token
2025-10-09T21:43:40.112531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6402f27c-e039b20-644ccccc-30752ff5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:40.125261Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340955597863489:3142]
2025-10-09T21:43:40.125650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340955597863489:3142], now have 1 active actors on pipe
2025-10-09T21:43:40.125771Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 177, pipe: [1:7559340955597863489:3142]
2025-10-09T21:43:40.132577Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:40.132594Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:40.132597Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:40.132599Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:40.132609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340955236053885:3385]
2025-10-09T21:43:40.132615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340955236053885:3385]
2025-10-09T21:43:40.132619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340955236053885:3385]
2025-10-09T21:43:40.132665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:40.132673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.132676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:40.132818Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:40.137349Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:40.137362Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:40.137367Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:40.144569Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 178 sessionId: src_id|6402f27c-e039b20-644ccccc-30752ff5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:40.147567Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:40.342764Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:40.343085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:40.343096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:40.344120Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:40.344135Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:40.344198Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 178 reason: error at write: Disconnected sessionId: src_id|6402f27c-e039b20-644ccccc-30752ff5_0
2025-10-09T21:43:40.344385Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 178 sessionId: src_id|6402f27c-e039b20-644ccccc-30752ff5_0 is DEAD
2025-10-09T21:43:40.344526Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:40.344561Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340955597863489:3142]
2025-10-09T21:43:40.344927Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6402f27c-e039b20-644ccccc-30752ff5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:40.344937Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6402f27c-e039b20-644ccccc-30752ff5_0] Write session will restart in 0.008874s
2025-10-09T21:43:40.344964Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6402f27c-e039b20-644ccccc-30752ff5_0] Write session: Do CDS request
2025-10-09T21:43:40.344971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6402f27c-e039b20-644ccccc-30752ff5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:40.345432Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:40.357997Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6402f27c-e039b20-644ccccc-30752ff5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:40.358667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:40.358685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 179
2025-10-09T21:43:40.358848Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 179 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:40.358881Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 179 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:40.358887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=179 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:40.358891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:40.359818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 179 sessionId:  describe result for acl check
2025-10-09T21:43:40.359857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:40.359859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:40.359861Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:40.359869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340955597863508:3642] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:40.360470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340955597863508:3642] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:40.365752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340955597863508:3642] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:40.385000Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340955597863508:3642] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:40.385018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340955597863508:3642] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:40.385022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340955597863508:3642] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:40.385033Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 179 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:40.415197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:40.417591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:40.417832Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:40.417836Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:40.417849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:40.417908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:40.417947Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340955236054121:3387]
2025-10-09T21:43:40.418791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:40.419549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:40.419610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:40.419967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:40.420016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:40.420017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:40.420019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:40.420020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:40.420022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:40.420023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:40.420025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.420026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.420028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.420034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:40.420047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:40.420052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:40.420056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:40.420060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:40.420066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:40.420070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.420075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.420079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.420087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:40.420097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:40.420103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:40.420109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:40.420114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:40.420119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:40.420125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:40.420131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:40.420136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:40.420143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:40.420148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:40.420155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:40.420160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.420166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.420171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.420178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.420183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.420189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.420223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:40.434744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:40.436096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:40.436103Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:40.436106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:40.436111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:40.436120Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 178 [2:7559340955236054121:3387]
2025-10-09T21:43:40.436138Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:40.436160Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:40.436164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.436168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.436170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.436189Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:40.436196Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:40.436204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:40.436268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.436270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.436272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.436275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:40.436613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:40.436616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.436630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.436644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.436663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:40.436684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340955236054115:3387]
2025-10-09T21:43:40.436687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340955236054115:3387]
2025-10-09T21:43:40.436689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340955236054115:3387]
2025-10-09T21:43:40.436706Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:40.436722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:40.436738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:40.436742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:40.436761Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:40.436766Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:40.436769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:40.436788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340955236054115:3387]
2025-10-09T21:43:40.436791Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:40.436878Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:40.437049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340955597863494:2432], now have 1 active actors on pipe
2025-10-09T21:43:40.437059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340955597863597:3642], now have 1 active actors on pipe
2025-10-09T21:43:40.437362Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 178
2025-10-09T21:43:40.437404Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 178
2025-10-09T21:43:40.438190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:40.438204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:40.438238Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9dff1c01-b59b85bc-7667d6b2-77a2447_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:40.438248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.438253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.438256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:40.438270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.438285Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:40.438303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:40.442276Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340955597863732:3142]
2025-10-09T21:43:40.447691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:40.447710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:40.447733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340955597863732:3142], now have 1 active actors on pipe
2025-10-09T21:43:40.447766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:40.448781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6402f27c-e039b20-644ccccc-30752ff5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9dff1c01-b59b85bc-7667d6b2-77a2447_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:40.451512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9dff1c01-b59b85bc-7667d6b2-77a2447_0] Write session: try to update token
2025-10-09T21:43:40.451534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9dff1c01-b59b85bc-7667d6b2-77a2447_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:40.448303Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 178, pipe: [1:7559340955597863732:3142]
2025-10-09T21:43:40.448353Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 179 partition: 0 MaxSeqNo: 25 sessionId: src_id|9dff1c01-b59b85bc-7667d6b2-77a2447_0
2025-10-09T21:43:40.467156Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:40.467176Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:40.467178Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:40.467180Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:40.467193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340955236054115:3387]
2025-10-09T21:43:40.467196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340955236054115:3387]
2025-10-09T21:43:40.467199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340955236054115:3387]
2025-10-09T21:43:40.467243Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:40.467247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:40.467248Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:40.467251Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:40.467254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.467257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:40.467388Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:40.475278Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 179 sessionId: src_id|9dff1c01-b59b85bc-7667d6b2-77a2447_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:40.478122Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:40.652098Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:40.652634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:40.652652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:40.653149Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340955597863732:3142]
2025-10-09T21:43:40.653177Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:40.653210Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:40.653213Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:40.653270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 179 reason: error at write: Disconnected sessionId: src_id|9dff1c01-b59b85bc-7667d6b2-77a2447_0
2025-10-09T21:43:40.653454Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 179 sessionId: src_id|9dff1c01-b59b85bc-7667d6b2-77a2447_0 is DEAD
2025-10-09T21:43:40.654123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9dff1c01-b59b85bc-7667d6b2-77a2447_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:40.654134Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9dff1c01-b59b85bc-7667d6b2-77a2447_0] Write session will restart in 0.008601s
2025-10-09T21:43:40.654159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9dff1c01-b59b85bc-7667d6b2-77a2447_0] Write session: Do CDS request
2025-10-09T21:43:40.654164Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9dff1c01-b59b85bc-7667d6b2-77a2447_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:40.653662Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:40.666637Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9dff1c01-b59b85bc-7667d6b2-77a2447_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:40.667032Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:40.667048Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 180
2025-10-09T21:43:40.667245Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 180 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:40.667277Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 180 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:40.667283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=180 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:40.667287Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:40.667775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 180 sessionId:  describe result for acl check
2025-10-09T21:43:40.667809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:40.667813Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:40.667815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:40.667821Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340955597863778:3654] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:40.668346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340955597863778:3654] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:40.672907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340955597863778:3654] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:40.682577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340955597863778:3654] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:40.682598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340955597863778:3654] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:40.682602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340955597863778:3654] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:40.682612Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 180 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:40.695679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:40.697293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:40.697495Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:40.697498Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:40.697509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:40.697557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:40.697592Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340955236054393:3401]
2025-10-09T21:43:40.698106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:40.698392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:40.698441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:40.698713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:40.698748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:40.698749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:40.698750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:40.698751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:40.698752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:40.698753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:40.698754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.698755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.698756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.698760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:40.698769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:40.698773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:40.698776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:40.698779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:40.698782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:40.698785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.698788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.698792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.698798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:40.698805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:40.698811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:40.698815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:40.698820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:40.698824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:40.698827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:40.698832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:40.698835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:40.698840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:40.698843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:40.698848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:40.698852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.698857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.698860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.698864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.698868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.698872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.698897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:40.707804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:40.709162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:40.709178Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:40.709181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:40.709186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:40.709196Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 179 [2:7559340955236054393:3401]
2025-10-09T21:43:40.709212Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:40.709234Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:40.709240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.709244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.709247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.709266Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:40.709273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:40.709281Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:40.709348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.709359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.709361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.709365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:40.709446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:40.709454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.709470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:40.709483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:40.709503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:40.709528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340955597863879:3654], now have 1 active actors on pipe
2025-10-09T21:43:40.709532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340955597863763:2432], now have 1 active actors on pipe
2025-10-09T21:43:40.709541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340955236054387:3401]
2025-10-09T21:43:40.709544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340955236054387:3401]
2025-10-09T21:43:40.709546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340955236054387:3401]
2025-10-09T21:43:40.709560Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:40.709835Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 179
2025-10-09T21:43:40.709876Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 179
2025-10-09T21:43:40.709577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:40.709598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:40.709602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:40.709660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340955236054387:3401]
2025-10-09T21:43:40.709663Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:40.709976Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:40.709979Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:40.709981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:40.709989Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:40.713502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:40.713517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:40.713559Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e04cf26-6ad102ca-c3da9cbc-74e9c431_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:40.713570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.713578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.713581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:40.713594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.713610Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:40.713632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:40.713937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:40.713942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:40.713954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:40.714082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 180 partition: 0 MaxSeqNo: 25 sessionId: src_id|e04cf26-6ad102ca-c3da9cbc-74e9c431_0
2025-10-09T21:43:40.714589Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9dff1c01-b59b85bc-7667d6b2-77a2447_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e04cf26-6ad102ca-c3da9cbc-74e9c431_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:40.717290Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e04cf26-6ad102ca-c3da9cbc-74e9c431_0] Write session: try to update token
2025-10-09T21:43:40.717311Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e04cf26-6ad102ca-c3da9cbc-74e9c431_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:40.740726Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:40.740747Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:40.740750Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:40.740753Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:40.740763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340955236054387:3401]
2025-10-09T21:43:40.740768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340955236054387:3401]
2025-10-09T21:43:40.740772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340955236054387:3401]
2025-10-09T21:43:40.740820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:40.740828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:40.740831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:40.740989Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:40.745587Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:40.745597Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:40.745603Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:40.753398Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340955597864005:3142]
2025-10-09T21:43:40.754501Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 179, pipe: [1:7559340955597864005:3142]
2025-10-09T21:43:40.757750Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 180 sessionId: src_id|e04cf26-6ad102ca-c3da9cbc-74e9c431_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:40.760304Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:40.958609Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:40.958895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340955597864005:3142], now have 1 active actors on pipe
2025-10-09T21:43:40.958925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:40.958933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:40.959447Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340955597864005:3142]
2025-10-09T21:43:40.959485Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:40.959487Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:40.959558Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 180 reason: error at write: Disconnected sessionId: src_id|e04cf26-6ad102ca-c3da9cbc-74e9c431_0
2025-10-09T21:43:40.959764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 180 sessionId: src_id|e04cf26-6ad102ca-c3da9cbc-74e9c431_0 is DEAD
2025-10-09T21:43:40.959924Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:40.960021Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e04cf26-6ad102ca-c3da9cbc-74e9c431_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:40.960033Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e04cf26-6ad102ca-c3da9cbc-74e9c431_0] Write session will restart in 0.006439s
2025-10-09T21:43:40.960058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e04cf26-6ad102ca-c3da9cbc-74e9c431_0] Write session: Do CDS request
2025-10-09T21:43:40.960068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e04cf26-6ad102ca-c3da9cbc-74e9c431_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:40.959544Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:40.967783Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e04cf26-6ad102ca-c3da9cbc-74e9c431_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:40.968026Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:40.968056Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 181
2025-10-09T21:43:40.971355Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 181 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:40.971417Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 181 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:40.971423Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=181 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:40.971428Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:40.972373Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 181 sessionId:  describe result for acl check
2025-10-09T21:43:40.972409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:40.972411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:40.972415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:40.972421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340955597864043:3659] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:40.973010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340955597864043:3659] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:40.976088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340955597864043:3659] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:40.984278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340955597864043:3659] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:40.984300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340955597864043:3659] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:40.984304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340955597864043:3659] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:40.984315Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 181 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:40.999812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:41.003018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:41.003263Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:41.003267Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:41.003280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:41.003351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:41.003406Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340959531021930:3405]
2025-10-09T21:43:41.004067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:41.005496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:41.005556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:41.005935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:41.005981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:41.005982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:41.005984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:41.005985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:41.005987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:41.005989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:41.005990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.005991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.005993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.005999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:41.006011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:41.006016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:41.006021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:41.006025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:41.006041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:41.006045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.006049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.006052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.006061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:41.006069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:41.006075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:41.006080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:41.006085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:41.006089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:41.006093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:41.006098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:41.006103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:41.006108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:41.006112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:41.006117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:41.006122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.006127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.006132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.006137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.006142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.006147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.006176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:41.025971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:41.027248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:41.027258Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:41.027261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:41.027267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:41.027276Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 180 [2:7559340959531021930:3405]
2025-10-09T21:43:41.027299Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:41.027325Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:41.027329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.027332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.027335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.027356Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:41.027363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:41.027372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:41.027605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.027607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.027610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.027613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:41.027703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:41.027706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.027721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.027735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.027754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:41.027776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340955236054629:3405]
2025-10-09T21:43:41.027779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340955236054629:3405]
2025-10-09T21:43:41.027781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340955236054629:3405]
2025-10-09T21:43:41.027792Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:41.027802Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:41.027806Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:41.027807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:41.027812Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:41.027825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:41.027829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:41.027843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340955597864019:2432], now have 1 active actors on pipe
2025-10-09T21:43:41.027848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340955597864149:3659], now have 1 active actors on pipe
2025-10-09T21:43:41.027890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340955236054629:3405]
2025-10-09T21:43:41.027902Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:41.028167Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 180
2025-10-09T21:43:41.028212Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 180
2025-10-09T21:43:41.028526Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:41.028737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:41.028754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:41.028790Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c680f1a9-7b2a4345-e22323a-5176c485_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:41.028807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.028813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.028817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:41.028831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.028849Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:41.028869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:41.029306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:41.029320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:41.029344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:41.029460Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 181 partition: 0 MaxSeqNo: 25 sessionId: src_id|c680f1a9-7b2a4345-e22323a-5176c485_0
2025-10-09T21:43:41.029978Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e04cf26-6ad102ca-c3da9cbc-74e9c431_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c680f1a9-7b2a4345-e22323a-5176c485_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:41.032655Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c680f1a9-7b2a4345-e22323a-5176c485_0] Write session: try to update token
2025-10-09T21:43:41.032676Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c680f1a9-7b2a4345-e22323a-5176c485_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:41.045465Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:41.045488Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:41.045491Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:41.045494Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:41.045504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340955236054629:3405]
2025-10-09T21:43:41.045510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340955236054629:3405]
2025-10-09T21:43:41.045514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340955236054629:3405]
2025-10-09T21:43:41.045568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:41.045575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.045579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:41.045718Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:41.047656Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:41.047671Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:41.047675Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:41.061068Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340959892831556:3142]
2025-10-09T21:43:41.063769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 181 sessionId: src_id|c680f1a9-7b2a4345-e22323a-5176c485_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:41.066677Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:41.067281Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 180, pipe: [1:7559340959892831556:3142]
2025-10-09T21:43:41.223072Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:41.223410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:41.223426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:41.223453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340959892831556:3142], now have 1 active actors on pipe
2025-10-09T21:43:41.224312Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:41.225859Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:41.225874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:41.225949Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 181 reason: error at write: Disconnected sessionId: src_id|c680f1a9-7b2a4345-e22323a-5176c485_0
2025-10-09T21:43:41.226129Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:41.226180Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 181 sessionId: src_id|c680f1a9-7b2a4345-e22323a-5176c485_0 is DEAD
2025-10-09T21:43:41.226182Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340959892831556:3142]
2025-10-09T21:43:41.226513Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c680f1a9-7b2a4345-e22323a-5176c485_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:41.226526Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c680f1a9-7b2a4345-e22323a-5176c485_0] Write session will restart in 0.009051s
2025-10-09T21:43:41.226553Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c680f1a9-7b2a4345-e22323a-5176c485_0] Write session: Do CDS request
2025-10-09T21:43:41.226561Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c680f1a9-7b2a4345-e22323a-5176c485_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:41.236624Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c680f1a9-7b2a4345-e22323a-5176c485_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:41.236912Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:41.236929Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 182
2025-10-09T21:43:41.238222Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 182 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:41.238284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 182 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:41.238290Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=182 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:41.238302Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:41.239038Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 182 sessionId:  describe result for acl check
2025-10-09T21:43:41.239070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:41.239072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:41.239074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:41.239083Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340959892831574:3663] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:41.239652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340959892831574:3663] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:41.257937Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340959892831574:3663] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:41.267910Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340959892831574:3663] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:41.267929Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340959892831574:3663] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:41.267935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340959892831574:3663] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:41.267946Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 182 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:41.270591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:41.272272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:41.272512Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:41.272525Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:41.272538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:41.272604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:41.272773Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340959531022168:3407]
2025-10-09T21:43:41.273349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:41.274075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:41.274153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:41.274527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:41.274591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:41.274597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:41.274598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:41.274600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:41.274601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:41.274602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:41.274603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.274605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.274620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.274627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:41.274639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:41.274643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:41.274647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:41.274651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:41.274655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:41.274659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.274663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.274667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.274676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:41.274685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:41.274691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:41.274697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:41.274702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:41.274707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:41.274711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:41.274716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:41.274722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:41.274727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:41.274732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:41.274737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:41.274742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.274748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.274752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.274757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.274761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.274767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.274798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:41.290895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:41.292048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:41.292066Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:41.292069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:41.292074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:41.292083Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 181 [2:7559340959531022168:3407]
2025-10-09T21:43:41.292100Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:41.292121Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:41.292124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.292127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.292130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.292152Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:41.292165Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:41.292173Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:41.292244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.292252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.292254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.292257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:41.292334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:41.292343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.292356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.292369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.292391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:41.292416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340959531022162:3407]
2025-10-09T21:43:41.292422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340959531022162:3407]
2025-10-09T21:43:41.292425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340959531022162:3407]
2025-10-09T21:43:41.292434Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:41.292447Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:41.292450Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:41.292453Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:41.292453Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:41.292468Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:41.292472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:41.292489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340959892831787:3663], now have 1 active actors on pipe
2025-10-09T21:43:41.292494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340959892831559:2432], now have 1 active actors on pipe
2025-10-09T21:43:41.292533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340959531022162:3407]
2025-10-09T21:43:41.292546Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:41.292723Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:41.293116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:41.293127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:41.293171Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|138ed18e-768b2838-7f5d1028-e7743ce3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:41.293180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.293184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.293187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:41.293199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.293213Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:41.293233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:41.293761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:41.293770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:41.293789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:41.292615Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 181
2025-10-09T21:43:41.292650Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 181
2025-10-09T21:43:41.294019Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 182 partition: 0 MaxSeqNo: 25 sessionId: src_id|138ed18e-768b2838-7f5d1028-e7743ce3_0
2025-10-09T21:43:41.298163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c680f1a9-7b2a4345-e22323a-5176c485_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|138ed18e-768b2838-7f5d1028-e7743ce3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:41.300867Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|138ed18e-768b2838-7f5d1028-e7743ce3_0] Write session: try to update token
2025-10-09T21:43:41.300889Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|138ed18e-768b2838-7f5d1028-e7743ce3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:41.314524Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:41.314547Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:41.314551Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:41.314552Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:41.314562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340959531022162:3407]
2025-10-09T21:43:41.314568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340959531022162:3407]
2025-10-09T21:43:41.314572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340959531022162:3407]
2025-10-09T21:43:41.314622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:41.314626Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:41.314629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.314631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:41.314633Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:41.314638Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:41.314858Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:41.326980Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340959892831802:3142]
2025-10-09T21:43:41.328272Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 181, pipe: [1:7559340959892831802:3142]
2025-10-09T21:43:41.336778Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 182 sessionId: src_id|138ed18e-768b2838-7f5d1028-e7743ce3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:41.339464Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:41.477603Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:41.478227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340959892831802:3142], now have 1 active actors on pipe
2025-10-09T21:43:41.478260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:41.478271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:41.478921Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:41.479013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 182 reason: status is not ok: tablet will be restarted right now sessionId: src_id|138ed18e-768b2838-7f5d1028-e7743ce3_0
2025-10-09T21:43:41.479218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 182 sessionId: src_id|138ed18e-768b2838-7f5d1028-e7743ce3_0 is DEAD
2025-10-09T21:43:41.478486Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:41.479375Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|138ed18e-768b2838-7f5d1028-e7743ce3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:41.479369Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:41.479387Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|138ed18e-768b2838-7f5d1028-e7743ce3_0] Write session will restart in 0.009303s
2025-10-09T21:43:41.479422Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|138ed18e-768b2838-7f5d1028-e7743ce3_0] Write session: Do CDS request
2025-10-09T21:43:41.479428Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|138ed18e-768b2838-7f5d1028-e7743ce3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:41.479413Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340959892831802:3142]
2025-10-09T21:43:41.489597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|138ed18e-768b2838-7f5d1028-e7743ce3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:41.489756Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:41.489772Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 183
2025-10-09T21:43:41.489932Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 183 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:41.489985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 183 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:41.489991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=183 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:41.489994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:41.490618Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 183 sessionId:  describe result for acl check
2025-10-09T21:43:41.490655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:41.490656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:41.490658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:41.490666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340959892831821:3668] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:41.491221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340959892831821:3668] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:41.505417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340959892831821:3668] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:41.513764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340959892831821:3668] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:41.513783Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340959892831821:3668] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:41.513788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340959892831821:3668] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:41.513798Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 183 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:41.528524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:41.530568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:41.530791Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:41.530803Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:41.530814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:41.530876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:41.530921Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340959531022405:3409]
2025-10-09T21:43:41.531476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:41.531796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:41.531842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:41.532153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:41.532213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:41.532220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:41.532221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:41.532222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:41.532223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:41.532225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:41.532226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.532227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.532228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.532233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:41.532243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:41.532247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:41.532251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:41.532254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:41.532257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:41.532260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.532264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.532267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.532275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:41.532283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:41.532289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:41.532295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:41.532299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:41.532304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:41.532311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:41.532316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:41.532322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:41.532331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:41.532335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:41.532341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:41.532346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.532351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.532357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.532367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.532372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.532377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.532404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:41.549251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:41.550490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:41.550506Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:41.550509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:41.550514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:41.550522Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 182 [2:7559340959531022405:3409]
2025-10-09T21:43:41.550543Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:41.550563Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:41.550566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.550570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.550572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.550596Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:41.550602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:41.550609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:41.550683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.550685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.550686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.550689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:41.550767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:41.550770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.550781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.550792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.550807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:41.550829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340959531022400:3409]
2025-10-09T21:43:41.550831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340959531022400:3409]
2025-10-09T21:43:41.550832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340959531022400:3409]
2025-10-09T21:43:41.550848Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:41.550862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:41.550875Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:41.550880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:41.550929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340959531022400:3409]
2025-10-09T21:43:41.550934Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:41.551291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:41.551294Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:41.551296Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:41.551302Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:41.553331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340959892831807:2432], now have 1 active actors on pipe
2025-10-09T21:43:41.553349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340959892831959:3668], now have 1 active actors on pipe
2025-10-09T21:43:41.554353Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 182
2025-10-09T21:43:41.554396Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 182
2025-10-09T21:43:41.554917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:41.554932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:41.554971Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4576f041-28e5e33b-a1fc0863-5eb77519_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:41.554985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.554990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.554994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:41.555007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.555025Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:41.555044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:41.559149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:41.559166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:41.559205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:41.559565Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 183 partition: 0 MaxSeqNo: 25 sessionId: src_id|4576f041-28e5e33b-a1fc0863-5eb77519_0
2025-10-09T21:43:41.559939Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|138ed18e-768b2838-7f5d1028-e7743ce3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4576f041-28e5e33b-a1fc0863-5eb77519_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:41.562451Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4576f041-28e5e33b-a1fc0863-5eb77519_0] Write session: try to update token
2025-10-09T21:43:41.562472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4576f041-28e5e33b-a1fc0863-5eb77519_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:41.581009Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340959892832063:3142]
2025-10-09T21:43:41.581577Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 182, pipe: [1:7559340959892832063:3142]
2025-10-09T21:43:41.586126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340959892832063:3142], now have 1 active actors on pipe
2025-10-09T21:43:41.586457Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 183 sessionId: src_id|4576f041-28e5e33b-a1fc0863-5eb77519_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:41.588978Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:41.589402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:41.589435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:41.589481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:41.589690Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:41.596342Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:41.596368Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:41.596371Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:41.596373Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:41.596382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340959531022400:3409]
2025-10-09T21:43:41.596388Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340959531022400:3409]
2025-10-09T21:43:41.596391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340959531022400:3409]
2025-10-09T21:43:41.596439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:41.596459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.596461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:41.596602Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:41.598163Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:41.598174Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:41.598178Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:41.787122Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:41.800068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:41.800102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:41.800294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:41.800362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:41.800432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:41.800504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:41.800940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:41.800952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:41.801021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:41.801084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:41.801151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:41.801218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:41.801266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:41.801777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:41.801866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:41.801932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:41.801997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:41.802075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:41.802119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:41.802121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:41.802213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:41.802551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:41.802611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:41.802674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:41.802719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:41.802722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:41.802789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:41.802847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:41.802907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:41.802976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:41.803016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:41.803019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:41.803092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:41.803160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:41.803232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:41.803304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:41.803359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:41.803362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:41.803430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:41.803483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:41.803585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:41.803646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:41.803689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:41.803693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:41.803761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:41.803838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:41.803901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:41.803959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:41.804011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:41.804014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:41.804078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:41.804134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:41.804192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:41.804249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:41.804288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:41.804291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:41.804359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:41.804412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:41.804471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:41.804531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:41.804573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:41.804580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:41.805122Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:41.805880Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:41.805889Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340959892832063:3142]
2025-10-09T21:43:41.805917Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:41.805962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 183 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4576f041-28e5e33b-a1fc0863-5eb77519_0
2025-10-09T21:43:41.806183Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 183 sessionId: src_id|4576f041-28e5e33b-a1fc0863-5eb77519_0 is DEAD
2025-10-09T21:43:41.806867Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4576f041-28e5e33b-a1fc0863-5eb77519_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:41.806878Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4576f041-28e5e33b-a1fc0863-5eb77519_0] Write session will restart in 0.007190s
2025-10-09T21:43:41.806913Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4576f041-28e5e33b-a1fc0863-5eb77519_0] Write session: Do CDS request
2025-10-09T21:43:41.806919Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4576f041-28e5e33b-a1fc0863-5eb77519_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:41.818654Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4576f041-28e5e33b-a1fc0863-5eb77519_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:41.818778Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:41.818799Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 184
2025-10-09T21:43:41.819045Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 184 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:41.819099Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 184 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:41.819106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=184 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:41.819110Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:41.819756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 184 sessionId:  describe result for acl check
2025-10-09T21:43:41.819801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:41.819807Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:41.819809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:41.819816Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340959892832101:3682] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:41.820420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340959892832101:3682] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:41.824573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340959892832101:3682] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:41.835724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340959892832101:3682] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:41.835752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340959892832101:3682] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:41.835758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340959892832101:3682] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:41.835769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 184 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:41.856992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:41.859114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:41.859343Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:41.859348Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:41.859359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:41.859415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:41.859451Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340959531022686:3424]
2025-10-09T21:43:41.860048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:41.860461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:41.860509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:41.860841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:41.860885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:41.860887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:41.860888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:41.860889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:41.860890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:41.860891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:41.860892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.860893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.860895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.860899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:41.860910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:41.860914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:41.860917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:41.860921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:41.860925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:41.860928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.860931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.860934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.860940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:41.860948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:41.860956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:41.860961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:41.860965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:41.860968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:41.860972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:41.860977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:41.860982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:41.860986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:41.860989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:41.860993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:41.860998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.861003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.861007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.861011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.861015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.861019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.861060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:41.870142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:41.871302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:41.871308Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:41.871311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:41.871316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:41.871324Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 183 [2:7559340959531022686:3424]
2025-10-09T21:43:41.871340Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:41.871361Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:41.871364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.871368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.871370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.871390Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:41.871397Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:41.871406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:41.871480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.871482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.871497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.871501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:41.871593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:41.871596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.871609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:41.871622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:41.871639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:41.871663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340959892832155:3682], now have 1 active actors on pipe
2025-10-09T21:43:41.871672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340959531022680:3424]
2025-10-09T21:43:41.871674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340959531022680:3424]
2025-10-09T21:43:41.871677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340959531022680:3424]
2025-10-09T21:43:41.871698Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:41.871709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340959892832091:2432], now have 1 active actors on pipe
2025-10-09T21:43:41.871712Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:41.871720Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:41.871733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:41.871735Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:41.871738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:41.871738Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:41.871779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340959531022680:3424]
2025-10-09T21:43:41.871789Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:41.871828Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:41.871904Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 183
2025-10-09T21:43:41.871939Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 183
2025-10-09T21:43:41.872262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:41.872277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:41.872313Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b5cff119-30e8c38e-26c5ed1d-e84a6f3c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:41.872323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.872327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.872330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:41.872346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.872360Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:41.872378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:41.872962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:41.872971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:41.872989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:41.873363Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 184 partition: 0 MaxSeqNo: 25 sessionId: src_id|b5cff119-30e8c38e-26c5ed1d-e84a6f3c_0
2025-10-09T21:43:41.874141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4576f041-28e5e33b-a1fc0863-5eb77519_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b5cff119-30e8c38e-26c5ed1d-e84a6f3c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:41.876848Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5cff119-30e8c38e-26c5ed1d-e84a6f3c_0] Write session: try to update token
2025-10-09T21:43:41.876870Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5cff119-30e8c38e-26c5ed1d-e84a6f3c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:41.889890Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:43:41.889906Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:41.889915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559340959531022680:3424]
2025-10-09T21:43:41.889959Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:41.890178Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:43:41.890316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:41.890465Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:41.890604Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:43:41.890818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:43:41.890858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:43:41.891922Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:43:41.905921Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340959892832341:3142]
2025-10-09T21:43:41.906411Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 183, pipe: [1:7559340959892832341:3142]
2025-10-09T21:43:41.906511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340959892832341:3142], now have 1 active actors on pipe
2025-10-09T21:43:41.909194Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 184 sessionId: src_id|b5cff119-30e8c38e-26c5ed1d-e84a6f3c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:41.912213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:41.912236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:41.912280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:41.914121Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:41.914126Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:41.914128Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:41.914130Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:41.914142Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340959531022680:3424]
2025-10-09T21:43:41.914146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340959531022680:3424]
2025-10-09T21:43:41.914150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340959531022680:3424]
2025-10-09T21:43:41.914196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:41.914203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:41.914206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:41.914339Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:41.911807Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:41.916932Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:41.916946Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:41.916950Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:41.917915Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:42.123199Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:42.133171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:42.133202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:42.133404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:42.133612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:42.133710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:42.133791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:42.133860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:42.133866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:42.133960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:42.134084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:42.134158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:42.134240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:42.134802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:42.134808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:42.134885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:42.134947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:42.135053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:42.135117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:42.135159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:42.135162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:42.135243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:42.135309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:42.135375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:42.135441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:42.135482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:42.135485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:42.135587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:42.135656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:42.135724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:42.135789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:42.135835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:42.135841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:42.135915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:42.135976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:42.136038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:42.136111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:42.136165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:42.136168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:42.136253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:42.136322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:42.136405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:42.136505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:42.136557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:42.136561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:42.136636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:42.136701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:42.136798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:42.136873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:42.136921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:42.136924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:42.137003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:42.137070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:42.137141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:42.137207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:42.137268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:42.137272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:42.137345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:42.137409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:42.137474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:42.137539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:42.137583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:42.137586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:42.138895Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5cff119-30e8c38e-26c5ed1d-e84a6f3c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:42.138907Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5cff119-30e8c38e-26c5ed1d-e84a6f3c_0] Write session will restart in 0.007120s
2025-10-09T21:43:42.138946Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5cff119-30e8c38e-26c5ed1d-e84a6f3c_0] Write session: Do CDS request
2025-10-09T21:43:42.138954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5cff119-30e8c38e-26c5ed1d-e84a6f3c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:42.138450Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:42.138466Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:42.138518Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 184 reason: error at write: Disconnected sessionId: src_id|b5cff119-30e8c38e-26c5ed1d-e84a6f3c_0
2025-10-09T21:43:42.138736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 184 sessionId: src_id|b5cff119-30e8c38e-26c5ed1d-e84a6f3c_0 is DEAD
2025-10-09T21:43:42.138773Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:42.138951Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340959892832341:3142]
2025-10-09T21:43:42.138454Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:42.147664Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5cff119-30e8c38e-26c5ed1d-e84a6f3c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:42.147902Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:42.147918Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 185
2025-10-09T21:43:42.150743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 185 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:42.150817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 185 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:42.150824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=185 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:42.150829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:42.151931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 185 sessionId:  describe result for acl check
2025-10-09T21:43:42.151981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:42.151988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:42.151993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:42.152000Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340964187799658:3686] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:42.152612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340964187799658:3686] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:42.160536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340964187799658:3686] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:42.170934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340964187799658:3686] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:42.170955Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340964187799658:3686] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:42.170962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340964187799658:3686] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:42.170978Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 185 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:42.183651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:42.185870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:42.186163Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:42.186169Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:42.186187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:42.186261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:42.186304Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340963825990225:3427]
2025-10-09T21:43:42.186949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:42.188000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:42.188087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:42.188480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:42.188550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:42.188551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:42.188553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:42.188554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:42.188556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:42.188558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:42.188559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.188560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.188562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.188569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:42.188581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:42.188585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:42.188590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:42.188594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:42.188598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:42.188603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.188606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.188610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.188619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:42.188629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:42.188636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:42.188642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:42.188647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:42.188654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:42.188659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:42.188664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:42.188669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:42.188674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:42.188679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:42.188685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:42.188693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.188697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.188702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.188708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.188714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.188720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.188752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:42.198220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:42.199686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:42.199693Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:42.199695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:42.199701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:42.199708Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 184 [2:7559340963825990225:3427]
2025-10-09T21:43:42.199727Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:42.199750Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:42.199754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.199758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.199761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.199782Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:42.199789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:42.199798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:42.199872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.199875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.199877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.199881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:42.199972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:42.199976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.199991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.200005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.200024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:42.200046Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340963825990220:3427]
2025-10-09T21:43:42.200049Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340963825990220:3427]
2025-10-09T21:43:42.200051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340963825990220:3427]
2025-10-09T21:43:42.200060Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:42.200075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:42.200093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:42.200096Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:42.200100Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:42.200104Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:42.200106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:42.200139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340963825990220:3427]
2025-10-09T21:43:42.200154Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:42.200166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340964187799803:3686], now have 1 active actors on pipe
2025-10-09T21:43:42.200170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340964187799643:2432], now have 1 active actors on pipe
2025-10-09T21:43:42.200455Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:42.200864Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 184
2025-10-09T21:43:42.200917Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 184
2025-10-09T21:43:42.201586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:42.201601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:42.201639Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d6cd6ba5-a8b234d9-a0b8ab33-383fedda_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:42.201666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.201671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.201675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:42.201688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.201704Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:42.201724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:42.205804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:42.205830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:42.205883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:42.206765Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5cff119-30e8c38e-26c5ed1d-e84a6f3c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d6cd6ba5-a8b234d9-a0b8ab33-383fedda_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:42.209685Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d6cd6ba5-a8b234d9-a0b8ab33-383fedda_0] Write session: try to update token
2025-10-09T21:43:42.209711Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d6cd6ba5-a8b234d9-a0b8ab33-383fedda_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:42.206259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 185 partition: 0 MaxSeqNo: 25 sessionId: src_id|d6cd6ba5-a8b234d9-a0b8ab33-383fedda_0
2025-10-09T21:43:42.219040Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:42.219061Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:42.219064Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:42.219067Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:42.219078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340963825990220:3427]
2025-10-09T21:43:42.219083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340963825990220:3427]
2025-10-09T21:43:42.219086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340963825990220:3427]
2025-10-09T21:43:42.219146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:42.219152Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:42.219155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.219158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:42.219160Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:42.219165Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:42.219335Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:42.234009Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 185 sessionId: src_id|d6cd6ba5-a8b234d9-a0b8ab33-383fedda_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:42.237049Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:42.240202Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340964187799890:3142]
2025-10-09T21:43:42.244175Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 184, pipe: [1:7559340964187799890:3142]
2025-10-09T21:43:42.423893Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:42.424414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:42.424433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:42.424760Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340964187799890:3142]
2025-10-09T21:43:42.424762Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:42.424767Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:42.424796Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:42.424831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 185 reason: error at write: Disconnected sessionId: src_id|d6cd6ba5-a8b234d9-a0b8ab33-383fedda_0
2025-10-09T21:43:42.424458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340964187799890:3142], now have 1 active actors on pipe
2025-10-09T21:43:42.425031Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 185 sessionId: src_id|d6cd6ba5-a8b234d9-a0b8ab33-383fedda_0 is DEAD
2025-10-09T21:43:42.425276Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:42.426156Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6cd6ba5-a8b234d9-a0b8ab33-383fedda_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:42.426169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6cd6ba5-a8b234d9-a0b8ab33-383fedda_0] Write session will restart in 0.006534s
2025-10-09T21:43:42.426197Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6cd6ba5-a8b234d9-a0b8ab33-383fedda_0] Write session: Do CDS request
2025-10-09T21:43:42.426205Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6cd6ba5-a8b234d9-a0b8ab33-383fedda_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:42.434565Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d6cd6ba5-a8b234d9-a0b8ab33-383fedda_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:42.434964Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:42.434982Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 186
2025-10-09T21:43:42.438182Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 186 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:42.438246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 186 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:42.438258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=186 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:42.438263Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:42.438783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 186 sessionId:  describe result for acl check
2025-10-09T21:43:42.438821Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:42.438824Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:42.438825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:42.438833Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340964187799910:3691] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:42.439416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340964187799910:3691] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:42.441613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340964187799910:3691] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:42.450667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340964187799910:3691] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:42.450688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340964187799910:3691] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:42.450693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340964187799910:3691] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:42.450704Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 186 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:42.471256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:42.475642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:42.475908Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:42.475912Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:42.475925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:42.475984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:42.476024Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340963825990467:3429]
2025-10-09T21:43:42.476661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:42.482598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:42.482687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:42.483072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:42.483139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:42.483140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:42.483141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:42.483142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:42.483143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:42.483145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:42.483146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.483147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.483148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.483156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:42.483167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:42.483171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:42.483175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:42.483179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:42.483182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:42.483186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.483189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.483193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.483201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:42.483210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:42.483236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:42.483241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:42.483245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:42.483250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:42.483255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:42.483261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:42.483266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:42.483270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:42.483275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:42.483280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:42.483285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.483290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.483293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.483298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.483302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.483307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.483336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:42.508380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:42.509497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:42.509512Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:42.509515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:42.509521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:42.509529Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 185 [2:7559340963825990467:3429]
2025-10-09T21:43:42.509545Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:42.509567Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:42.509570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.509574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.509577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.509595Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:42.509602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:42.509610Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:42.509682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.509693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.509695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.509698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:42.509776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:42.509784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.509798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.509811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.509831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:42.509857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340964187799981:3691], now have 1 active actors on pipe
2025-10-09T21:43:42.509861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340964187799895:2432], now have 1 active actors on pipe
2025-10-09T21:43:42.509871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340963825990462:3429]
2025-10-09T21:43:42.509874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340963825990462:3429]
2025-10-09T21:43:42.509876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340963825990462:3429]
2025-10-09T21:43:42.509891Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:42.509906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:42.509925Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:42.509928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:42.509931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:42.509935Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:42.509938Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:42.509914Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 185
2025-10-09T21:43:42.509971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340963825990462:3429]
2025-10-09T21:43:42.509984Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:42.510004Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:42.509980Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 185
2025-10-09T21:43:42.510321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:42.510337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:42.510387Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c04e7197-73fb4e2b-ad4ffe1d-4bc51765_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:42.510403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.510412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.510424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:42.510438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.510466Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:42.510487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:42.510764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:42.510774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:42.510791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:42.511357Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 186 partition: 0 MaxSeqNo: 25 sessionId: src_id|c04e7197-73fb4e2b-ad4ffe1d-4bc51765_0
2025-10-09T21:43:42.513099Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6cd6ba5-a8b234d9-a0b8ab33-383fedda_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c04e7197-73fb4e2b-ad4ffe1d-4bc51765_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:42.515712Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c04e7197-73fb4e2b-ad4ffe1d-4bc51765_0] Write session: try to update token
2025-10-09T21:43:42.515733Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c04e7197-73fb4e2b-ad4ffe1d-4bc51765_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:42.525290Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340964187800140:3142]
2025-10-09T21:43:42.525613Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 185, pipe: [1:7559340964187800140:3142]
2025-10-09T21:43:42.526726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340964187800140:3142], now have 1 active actors on pipe
2025-10-09T21:43:42.526751Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:42.526755Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:42.526757Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:42.526759Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:42.526766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340963825990462:3429]
2025-10-09T21:43:42.526772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340963825990462:3429]
2025-10-09T21:43:42.526775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340963825990462:3429]
2025-10-09T21:43:42.526820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:42.526835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.526837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:42.526986Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:42.526992Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:42.526997Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:42.527000Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:42.541549Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 186 sessionId: src_id|c04e7197-73fb4e2b-ad4ffe1d-4bc51765_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:42.544207Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:42.700575Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:42.701202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:42.701213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:42.701511Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:42.702270Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c04e7197-73fb4e2b-ad4ffe1d-4bc51765_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:42.702283Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c04e7197-73fb4e2b-ad4ffe1d-4bc51765_0] Write session will restart in 0.009594s
2025-10-09T21:43:42.702308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c04e7197-73fb4e2b-ad4ffe1d-4bc51765_0] Write session: Do CDS request
2025-10-09T21:43:42.702315Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c04e7197-73fb4e2b-ad4ffe1d-4bc51765_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:42.701586Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:42.701599Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:42.701660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 186 reason: error at write: Disconnected sessionId: src_id|c04e7197-73fb4e2b-ad4ffe1d-4bc51765_0
2025-10-09T21:43:42.701851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 186 sessionId: src_id|c04e7197-73fb4e2b-ad4ffe1d-4bc51765_0 is DEAD
2025-10-09T21:43:42.701990Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:42.702014Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340964187800140:3142]
2025-10-09T21:43:42.713777Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c04e7197-73fb4e2b-ad4ffe1d-4bc51765_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:42.714161Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:42.714181Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 187
2025-10-09T21:43:42.714432Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 187 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:42.714486Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 187 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:42.714498Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=187 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:42.714502Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:42.714960Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 187 sessionId:  describe result for acl check
2025-10-09T21:43:42.715004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:42.715006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:42.715008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:42.715014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340964187800186:3702] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:42.715595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340964187800186:3702] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:42.723253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340964187800186:3702] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:42.728071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340964187800186:3702] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:42.728090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340964187800186:3702] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:42.728094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340964187800186:3702] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:42.728107Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 187 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:42.736530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:42.738104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:42.738328Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:42.738339Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:42.738354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:42.738432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:42.738489Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340963825990748:3443]
2025-10-09T21:43:42.739107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:42.739566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:42.739627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:42.739998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:42.740072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:42.740078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:42.740080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:42.740081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:42.740082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:42.740083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:42.740084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.740086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.740087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.740094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:42.740106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:42.740110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:42.740114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:42.740118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:42.740122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:42.740126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.740130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.740133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.740156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:42.740166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:42.740177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:42.740183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:42.740188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:42.740193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:42.740197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:42.740202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:42.740207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:42.740213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:42.740217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:42.740223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:42.740228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.740240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.740245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.740270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.740280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.740286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.740317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:42.748752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:42.750024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:42.750063Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:42.750066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:42.750071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:42.750078Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 186 [2:7559340963825990748:3443]
2025-10-09T21:43:42.750093Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:42.750113Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:42.750116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.750120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.750123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.750142Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:42.750149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:42.750156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:42.750224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.750234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.750235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.750238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:42.750315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:42.750324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.750338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:42.750349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:42.750365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:42.750393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340964187800311:3702], now have 1 active actors on pipe
2025-10-09T21:43:42.750401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340964187800170:2432], now have 1 active actors on pipe
2025-10-09T21:43:42.750409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340963825990743:3443]
2025-10-09T21:43:42.750427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340963825990743:3443]
2025-10-09T21:43:42.750430Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340963825990743:3443]
2025-10-09T21:43:42.750444Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:42.750463Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:42.750473Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:42.750479Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:42.750481Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:42.750482Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:42.750485Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:42.750526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340963825990743:3443]
2025-10-09T21:43:42.750538Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:42.750592Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:42.750623Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 186
2025-10-09T21:43:42.750676Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 186
2025-10-09T21:43:42.750833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:42.750845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:42.750875Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|eab1b5a6-dc8efaf0-65d8db0a-f0aa7169_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:42.750885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.750889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.750892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:42.750903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.750915Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:42.750931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:42.751742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:42.751764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:42.751785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:42.752071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 187 partition: 0 MaxSeqNo: 25 sessionId: src_id|eab1b5a6-dc8efaf0-65d8db0a-f0aa7169_0
2025-10-09T21:43:42.752457Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c04e7197-73fb4e2b-ad4ffe1d-4bc51765_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|eab1b5a6-dc8efaf0-65d8db0a-f0aa7169_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:42.755149Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eab1b5a6-dc8efaf0-65d8db0a-f0aa7169_0] Write session: try to update token
2025-10-09T21:43:42.755167Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eab1b5a6-dc8efaf0-65d8db0a-f0aa7169_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:42.771054Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:42.771075Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:42.771078Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:42.771080Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:42.771092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340963825990743:3443]
2025-10-09T21:43:42.771097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340963825990743:3443]
2025-10-09T21:43:42.771100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340963825990743:3443]
2025-10-09T21:43:42.771148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:42.771157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:42.771159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:42.771306Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:42.772073Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:42.772087Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:42.772091Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:42.794244Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 187 sessionId: src_id|eab1b5a6-dc8efaf0-65d8db0a-f0aa7169_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:42.796847Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:42.801967Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340964187800421:3142]
2025-10-09T21:43:42.802333Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 186, pipe: [1:7559340964187800421:3142]
2025-10-09T21:43:42.941786Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:42.944830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:42.944850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:42.944876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340964187800421:3142], now have 1 active actors on pipe
2025-10-09T21:43:42.945234Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340964187800421:3142]
2025-10-09T21:43:42.945276Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:42.945279Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:42.945338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 187 reason: error at write: Disconnected sessionId: src_id|eab1b5a6-dc8efaf0-65d8db0a-f0aa7169_0
2025-10-09T21:43:42.945547Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 187 sessionId: src_id|eab1b5a6-dc8efaf0-65d8db0a-f0aa7169_0 is DEAD
2025-10-09T21:43:42.945556Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:42.946277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eab1b5a6-dc8efaf0-65d8db0a-f0aa7169_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:42.946288Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eab1b5a6-dc8efaf0-65d8db0a-f0aa7169_0] Write session will restart in 0.009598s
2025-10-09T21:43:42.946327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eab1b5a6-dc8efaf0-65d8db0a-f0aa7169_0] Write session: Do CDS request
2025-10-09T21:43:42.946337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eab1b5a6-dc8efaf0-65d8db0a-f0aa7169_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:42.947571Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:42.963629Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eab1b5a6-dc8efaf0-65d8db0a-f0aa7169_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:42.963755Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:42.963772Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 188
2025-10-09T21:43:42.963934Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 188 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:42.963962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 188 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:42.963968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=188 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:42.963972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:42.967501Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 188 sessionId:  describe result for acl check
2025-10-09T21:43:42.967546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:42.967552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:42.967554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:42.967562Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340964187800456:3707] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:42.968181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340964187800456:3707] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:42.976819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340964187800456:3707] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:42.990543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340964187800456:3707] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:42.990565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340964187800456:3707] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:42.990568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340964187800456:3707] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:42.990578Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 188 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:42.993943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:43.001585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:43.001872Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:43.001883Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:43.001896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:43.001966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:43.002013Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340968120958290:3447]
2025-10-09T21:43:43.002623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:43.005821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:43.005904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:43.006302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:43.006599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:43.006613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:43.006614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:43.006616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:43.006617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:43.006619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:43.006620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.006622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.006623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.006632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:43.006644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:43.006648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:43.006652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:43.006656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:43.006661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:43.006665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.006669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.006673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.006682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:43.006691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:43.006697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:43.006703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:43.006708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:43.006714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:43.006719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:43.006724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:43.006729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:43.006734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:43.006739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:43.006745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:43.006749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.006755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.006759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.006765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.006774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.006780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.006812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:43.021640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:43.025504Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 187
2025-10-09T21:43:43.025547Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 187
2025-10-09T21:43:43.023217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:43.023233Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:43.023236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:43.023241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:43.023249Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 187 [2:7559340968120958290:3447]
2025-10-09T21:43:43.023267Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:43.023391Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:43.023396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.023399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.023401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.023419Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:43.023426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:43.023433Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:43.023787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.023790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.023791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.023794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:43.023870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:43.023873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.023889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.023902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.023921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:43.023945Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340963825990988:3447]
2025-10-09T21:43:43.023947Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340963825990988:3447]
2025-10-09T21:43:43.023949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340963825990988:3447]
2025-10-09T21:43:43.023963Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:43.023979Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:43.024010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:43.024013Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:43.024187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340963825990988:3447]
2025-10-09T21:43:43.024190Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:43.025200Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:43.025205Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:43.025207Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:43.025219Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:43.025863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340964187800427:2432], now have 1 active actors on pipe
2025-10-09T21:43:43.025869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340964187800661:3707], now have 1 active actors on pipe
2025-10-09T21:43:43.025890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:43.025896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:43.025930Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d7ec7727-6175ade4-6df7b44d-6deb09cd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:43.025941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.025946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.025949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:43.025962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.025978Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:43.025997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:43.026446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:43.026459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:43.026481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:43.033734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 188 partition: 0 MaxSeqNo: 25 sessionId: src_id|d7ec7727-6175ade4-6df7b44d-6deb09cd_0
2025-10-09T21:43:43.038201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eab1b5a6-dc8efaf0-65d8db0a-f0aa7169_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d7ec7727-6175ade4-6df7b44d-6deb09cd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:43.040779Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ec7727-6175ade4-6df7b44d-6deb09cd_0] Write session: try to update token
2025-10-09T21:43:43.040805Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ec7727-6175ade4-6df7b44d-6deb09cd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:43.045707Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340968482767971:3142]
2025-10-09T21:43:43.047044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340968482767971:3142], now have 1 active actors on pipe
2025-10-09T21:43:43.047348Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 187, pipe: [1:7559340968482767971:3142]
2025-10-09T21:43:43.049617Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:43.049637Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:43.049639Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:43.049641Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:43.049650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340963825990988:3447]
2025-10-09T21:43:43.049654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340963825990988:3447]
2025-10-09T21:43:43.049658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340963825990988:3447]
2025-10-09T21:43:43.049703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:43.049709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.049712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:43.049858Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:43.053444Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:43.053460Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:43.053466Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:43.068217Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 188 sessionId: src_id|d7ec7727-6175ade4-6df7b44d-6deb09cd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:43.070920Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:43.243307Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:43.243602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:43.243614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:43.244507Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:43.244560Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340968482767971:3142]
2025-10-09T21:43:43.244581Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:43.244584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:43.244643Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 188 reason: error at write: Disconnected sessionId: src_id|d7ec7727-6175ade4-6df7b44d-6deb09cd_0
2025-10-09T21:43:43.244851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 188 sessionId: src_id|d7ec7727-6175ade4-6df7b44d-6deb09cd_0 is DEAD
2025-10-09T21:43:43.246263Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ec7727-6175ade4-6df7b44d-6deb09cd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:43.246277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ec7727-6175ade4-6df7b44d-6deb09cd_0] Write session will restart in 0.005598s
2025-10-09T21:43:43.246303Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ec7727-6175ade4-6df7b44d-6deb09cd_0] Write session: Do CDS request
2025-10-09T21:43:43.246310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ec7727-6175ade4-6df7b44d-6deb09cd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:43.246814Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:43.253658Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ec7727-6175ade4-6df7b44d-6deb09cd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:43.254276Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:43.254292Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 189
2025-10-09T21:43:43.254817Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 189 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:43.254876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 189 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:43.254882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=189 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:43.254886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:43.255336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 189 sessionId:  describe result for acl check
2025-10-09T21:43:43.255366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:43.255368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:43.255370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:43.255377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340968482767990:3711] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:43.255950Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340968482767990:3711] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:43.270356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340968482767990:3711] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:43.283699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340968482767990:3711] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:43.283719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340968482767990:3711] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:43.283724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340968482767990:3711] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:43.283735Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 189 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:43.294964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:43.296956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:43.297171Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:43.297175Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:43.297187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:43.297246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:43.297287Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340968120958535:3449]
2025-10-09T21:43:43.297863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:43.301893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:43.301974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:43.302388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:43.302476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:43.302477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:43.302478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:43.302480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:43.302481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:43.302482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:43.302483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.302484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.302485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.302491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:43.302502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:43.302506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:43.302509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:43.302513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:43.302517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:43.302521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.302524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.302528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.302535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:43.302544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:43.302551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:43.302556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:43.302560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:43.302564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:43.302568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:43.302573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:43.302578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:43.302582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:43.302586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:43.302590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:43.302596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.302601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.302605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.302610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.302614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.302619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.302648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:43.323133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:43.324434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:43.324440Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:43.324444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:43.324449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:43.324458Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 188 [2:7559340968120958535:3449]
2025-10-09T21:43:43.324498Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:43.324520Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:43.324523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.324527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.324529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.324549Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:43.324555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:43.324564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:43.324636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.324639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.324641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.324644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:43.324729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:43.324732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.324745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.324756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.324772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:43.324795Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340968120958529:3449]
2025-10-09T21:43:43.324798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340968120958529:3449]
2025-10-09T21:43:43.324801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340968120958529:3449]
2025-10-09T21:43:43.324814Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:43.324832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:43.324850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:43.324854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:43.324895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340968120958529:3449]
2025-10-09T21:43:43.324898Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:43.326387Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:43.326396Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:43.326398Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:43.326405Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:43.326835Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 188
2025-10-09T21:43:43.326886Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 188
2025-10-09T21:43:43.327521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340968482768131:3711], now have 1 active actors on pipe
2025-10-09T21:43:43.327534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340968482767976:2432], now have 1 active actors on pipe
2025-10-09T21:43:43.327552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:43.327558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:43.327596Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cb178aa4-1a0d0a5c-6a0f061c-54559574_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:43.327608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.327613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.327616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:43.327630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.327645Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:43.327667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:43.332281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:43.332301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:43.332341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:43.332513Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 189 partition: 0 MaxSeqNo: 25 sessionId: src_id|cb178aa4-1a0d0a5c-6a0f061c-54559574_0
2025-10-09T21:43:43.332961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ec7727-6175ade4-6df7b44d-6deb09cd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cb178aa4-1a0d0a5c-6a0f061c-54559574_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:43.335713Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cb178aa4-1a0d0a5c-6a0f061c-54559574_0] Write session: try to update token
2025-10-09T21:43:43.335733Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cb178aa4-1a0d0a5c-6a0f061c-54559574_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:43.346269Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340968482768226:3142]
2025-10-09T21:43:43.348081Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 188, pipe: [1:7559340968482768226:3142]
2025-10-09T21:43:43.347569Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:43.347592Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:43.347595Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:43.347597Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:43.347608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340968120958529:3449]
2025-10-09T21:43:43.347613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340968120958529:3449]
2025-10-09T21:43:43.347618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340968120958529:3449]
2025-10-09T21:43:43.347669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:43.347675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.347678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:43.347752Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:43.347758Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:43.347763Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:43.347823Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:43.358284Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 189 sessionId: src_id|cb178aa4-1a0d0a5c-6a0f061c-54559574_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:43.360921Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:43.657755Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:43.658846Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340968482768226:3142]
2025-10-09T21:43:43.658879Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:43.658882Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:43.658947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 189 reason: error at write: Disconnected sessionId: src_id|cb178aa4-1a0d0a5c-6a0f061c-54559574_0
2025-10-09T21:43:43.658975Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:43.659147Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 189 sessionId: src_id|cb178aa4-1a0d0a5c-6a0f061c-54559574_0 is DEAD
2025-10-09T21:43:43.658387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340968482768226:3142], now have 1 active actors on pipe
2025-10-09T21:43:43.658440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:43.658448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:43.662131Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb178aa4-1a0d0a5c-6a0f061c-54559574_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:43.662143Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb178aa4-1a0d0a5c-6a0f061c-54559574_0] Write session will restart in 0.008881s
2025-10-09T21:43:43.662169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb178aa4-1a0d0a5c-6a0f061c-54559574_0] Write session: Do CDS request
2025-10-09T21:43:43.662177Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb178aa4-1a0d0a5c-6a0f061c-54559574_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:43.662195Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:43.674678Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cb178aa4-1a0d0a5c-6a0f061c-54559574_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:43.678249Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:43.678273Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 190
2025-10-09T21:43:43.678506Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 190 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:43.678550Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 190 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:43.678556Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=190 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:43.678561Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:43.679043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 190 sessionId:  describe result for acl check
2025-10-09T21:43:43.679076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:43.679079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:43.679080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:43.679089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340968482768248:3716] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:43.679666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340968482768248:3716] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:43.688917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340968482768248:3716] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:43.693530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340968482768248:3716] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:43.693543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340968482768248:3716] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:43.693546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340968482768248:3716] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:43.693553Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 190 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:43.726958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:43.734227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:43.734508Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:43.734512Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:43.734527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:43.734597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:43.734635Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340968120958794:3451]
2025-10-09T21:43:43.735245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:43.742223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:43.742325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:43.742740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:43.742814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:43.742818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:43.742819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:43.742821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:43.742822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:43.742823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:43.742824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.742826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.742827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.742835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:43.742847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:43.742851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:43.742856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:43.742860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:43.742864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:43.742869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.742873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.742877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.742886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:43.742895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:43.742901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:43.742907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:43.742913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:43.742919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:43.742924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:43.742930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:43.742935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:43.742941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:43.742945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:43.742951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:43.742956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.742962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.742968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.742972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.742977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.742982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.743012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:43.759149Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340968482768492:3142]
2025-10-09T21:43:43.763020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:43.764436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:43.764443Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:43.764445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:43.764450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:43.764456Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 189 [2:7559340968120958794:3451]
2025-10-09T21:43:43.764474Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:43.764492Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:43.764495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.764499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.764501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.764520Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:43.764527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:43.764535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:43.764602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.764604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.764606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.764609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:43.764691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:43.764694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.764709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:43.764722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:43.764741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:43.764763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340968120958788:3451]
2025-10-09T21:43:43.764766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340968120958788:3451]
2025-10-09T21:43:43.764768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340968120958788:3451]
2025-10-09T21:43:43.764783Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:43.764799Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:43.764815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:43.764819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:43.764861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340968120958788:3451]
2025-10-09T21:43:43.764864Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:43.764992Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:43.764995Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:43.764998Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:43.765004Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:43.765797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340968482768233:2432], now have 1 active actors on pipe
2025-10-09T21:43:43.765803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340968482768492:3142], now have 1 active actors on pipe
2025-10-09T21:43:43.765806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340968482768297:3716], now have 1 active actors on pipe
2025-10-09T21:43:43.766239Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 189
2025-10-09T21:43:43.766274Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 189, pipe: [1:7559340968482768492:3142]
2025-10-09T21:43:43.766299Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 189
2025-10-09T21:43:43.774881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:43.774906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:43.774945Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|24639920-a4cdf5f4-da5eeb0f-5ce5f895_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:43.774958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.774963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.774966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:43.774982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.775001Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:43.775022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:43.775753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:43.775763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:43.775787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:43.775997Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 190 partition: 0 MaxSeqNo: 25 sessionId: src_id|24639920-a4cdf5f4-da5eeb0f-5ce5f895_0
2025-10-09T21:43:43.778367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb178aa4-1a0d0a5c-6a0f061c-54559574_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|24639920-a4cdf5f4-da5eeb0f-5ce5f895_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:43.781048Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|24639920-a4cdf5f4-da5eeb0f-5ce5f895_0] Write session: try to update token
2025-10-09T21:43:43.781072Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|24639920-a4cdf5f4-da5eeb0f-5ce5f895_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:43.799753Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:43.799773Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:43.799778Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:43.799780Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:43.799792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340968120958788:3451]
2025-10-09T21:43:43.799796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340968120958788:3451]
2025-10-09T21:43:43.799799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340968120958788:3451]
2025-10-09T21:43:43.799849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:43.799857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:43.799859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:43.799861Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:43.799868Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:43.799872Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:43.799996Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:43.813270Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 190 sessionId: src_id|24639920-a4cdf5f4-da5eeb0f-5ce5f895_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:43.815946Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:44.021152Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:44.021775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:44.021792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:44.022674Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:44.022726Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340968482768492:3142]
2025-10-09T21:43:44.022748Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:44.022755Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:44.022818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 190 reason: error at write: Disconnected sessionId: src_id|24639920-a4cdf5f4-da5eeb0f-5ce5f895_0
2025-10-09T21:43:44.023010Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 190 sessionId: src_id|24639920-a4cdf5f4-da5eeb0f-5ce5f895_0 is DEAD
2025-10-09T21:43:44.023403Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:44.028511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24639920-a4cdf5f4-da5eeb0f-5ce5f895_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:44.028526Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24639920-a4cdf5f4-da5eeb0f-5ce5f895_0] Write session will restart in 0.009150s
2025-10-09T21:43:44.028551Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24639920-a4cdf5f4-da5eeb0f-5ce5f895_0] Write session: Do CDS request
2025-10-09T21:43:44.028561Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24639920-a4cdf5f4-da5eeb0f-5ce5f895_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:44.038594Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|24639920-a4cdf5f4-da5eeb0f-5ce5f895_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:44.038784Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:44.038808Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 191
2025-10-09T21:43:44.038988Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 191 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:44.039030Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 191 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:44.039036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=191 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:44.039041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:44.039964Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 191 sessionId:  describe result for acl check
2025-10-09T21:43:44.040029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:44.040031Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:44.040033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:44.040040Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340972777735840:3728] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:44.040644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340972777735840:3728] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:44.048237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340972777735840:3728] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:44.054937Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340972777735840:3728] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:44.054956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340972777735840:3728] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:44.054960Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340972777735840:3728] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:44.054974Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 191 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:44.078725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:44.079447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:44.079677Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:44.079681Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:44.079695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:44.079758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:44.079820Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340972415926365:3467]
2025-10-09T21:43:44.080471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:44.081005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:44.081057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:44.081382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:44.081442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:44.081443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:44.081445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:44.081446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:44.081453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:44.081455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:44.081456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.081457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.081458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.081467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:44.081481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:44.081486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:44.081492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:44.081497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:44.081502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:44.081506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.081511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.081515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.081523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:44.081530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:44.081537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:44.081543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:44.081549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:44.081556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:44.081560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:44.081566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:44.081572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:44.081578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:44.081582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:44.081587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:44.081592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.081598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.081602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.081608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.081613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.081619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.081670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:44.090411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:44.092746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:44.092753Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:44.092756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:44.092761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:44.092769Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 190 [2:7559340972415926365:3467]
2025-10-09T21:43:44.092783Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:44.092805Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:44.092809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.092815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.092817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.092835Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:44.092842Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:44.092850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:44.092926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.092929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.092931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.092934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:44.093025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:44.093029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.093043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.093054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.093071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:44.093092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340972415926359:3467]
2025-10-09T21:43:44.093095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340972415926359:3467]
2025-10-09T21:43:44.093098Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340972415926359:3467]
2025-10-09T21:43:44.093106Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:44.093120Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:44.093123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:44.093124Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:44.093127Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:44.093139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:44.093142Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:44.093155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340972777735821:2432], now have 1 active actors on pipe
2025-10-09T21:43:44.093159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340972777735913:3728], now have 1 active actors on pipe
2025-10-09T21:43:44.093200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340972415926359:3467]
2025-10-09T21:43:44.093212Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:44.093230Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:44.093590Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 190
2025-10-09T21:43:44.093662Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 190
2025-10-09T21:43:44.094020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:44.094053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:44.094089Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6e409dab-8087413b-b461c95a-ed4d936b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:44.094099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.094103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.094106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:44.094120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.094134Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:44.094154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:44.099527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:44.099547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:44.099583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:44.099776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 191 partition: 0 MaxSeqNo: 25 sessionId: src_id|6e409dab-8087413b-b461c95a-ed4d936b_0
2025-10-09T21:43:44.102178Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24639920-a4cdf5f4-da5eeb0f-5ce5f895_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6e409dab-8087413b-b461c95a-ed4d936b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:44.111108Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e409dab-8087413b-b461c95a-ed4d936b_0] Write session: try to update token
2025-10-09T21:43:44.111137Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e409dab-8087413b-b461c95a-ed4d936b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:44.119658Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:44.119682Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:44.119684Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:44.119686Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:44.119696Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340972415926359:3467]
2025-10-09T21:43:44.119700Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340972415926359:3467]
2025-10-09T21:43:44.119704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340972415926359:3467]
2025-10-09T21:43:44.119749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:44.119756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.119759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:44.120287Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:44.122350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:44.122361Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:44.122365Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:44.123629Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340972777736071:3142]
2025-10-09T21:43:44.127083Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 190, pipe: [1:7559340972777736071:3142]
2025-10-09T21:43:44.152190Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 191 sessionId: src_id|6e409dab-8087413b-b461c95a-ed4d936b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:44.155108Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:44.455389Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:44.456016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340972777736071:3142], now have 1 active actors on pipe
2025-10-09T21:43:44.456071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:44.456078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:44.456585Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:44.457576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e409dab-8087413b-b461c95a-ed4d936b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:44.457588Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e409dab-8087413b-b461c95a-ed4d936b_0] Write session will restart in 0.006445s
2025-10-09T21:43:44.457614Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e409dab-8087413b-b461c95a-ed4d936b_0] Write session: Do CDS request
2025-10-09T21:43:44.457620Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e409dab-8087413b-b461c95a-ed4d936b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:44.457038Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340972777736071:3142]
2025-10-09T21:43:44.457072Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:44.457155Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 191 reason: status is not ok: tablet will be restarted right now sessionId: src_id|6e409dab-8087413b-b461c95a-ed4d936b_0
2025-10-09T21:43:44.457363Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 191 sessionId: src_id|6e409dab-8087413b-b461c95a-ed4d936b_0 is DEAD
2025-10-09T21:43:44.457597Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:44.465578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e409dab-8087413b-b461c95a-ed4d936b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:44.465980Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:44.466003Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 192
2025-10-09T21:43:44.467878Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 192 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:44.467940Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 192 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:44.467949Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=192 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:44.467954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:44.468466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 192 sessionId:  describe result for acl check
2025-10-09T21:43:44.468505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:44.468507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:44.468509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:44.468515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340972777736089:3733] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:44.469119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340972777736089:3733] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:44.476251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340972777736089:3733] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:44.488667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340972777736089:3733] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:44.488685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340972777736089:3733] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:44.488689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340972777736089:3733] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:44.488700Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 192 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:44.522614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:44.526458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:44.526727Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:44.526732Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:44.526746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:44.526810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:44.526983Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340972415926615:3469]
2025-10-09T21:43:44.527608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:44.538313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:44.538432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:44.538846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:44.538926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:44.538927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:44.538928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:44.538929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:44.538931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:44.538932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:44.538934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.538935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.538937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.538946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:44.538958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:44.538962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:44.538966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:44.538976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:44.538980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:44.538984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.538988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.538992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.539002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:44.539012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:44.539018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:44.539022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:44.539026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:44.539031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:44.539035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:44.539039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:44.539046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:44.539051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:44.539055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:44.539060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:44.539065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.539069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.539074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.539079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.539083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.539088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.539121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:44.551019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:44.552357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:44.552364Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:44.552367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:44.552372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:44.552397Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 191 [2:7559340972415926615:3469]
2025-10-09T21:43:44.552418Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:44.552441Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:44.552445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.552448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.552450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.552471Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:44.552477Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:44.552486Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:44.552557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.552559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.552565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.552568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:44.552655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:44.552658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.552671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.552681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.552696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:44.552721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340972777736167:3733], now have 1 active actors on pipe
2025-10-09T21:43:44.552725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340972777736078:2432], now have 1 active actors on pipe
2025-10-09T21:43:44.552733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340972415926609:3469]
2025-10-09T21:43:44.552735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340972415926609:3469]
2025-10-09T21:43:44.552737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340972415926609:3469]
2025-10-09T21:43:44.552746Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:44.552757Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:44.552761Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:44.552763Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:44.552763Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:44.552777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:44.552781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:44.552824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340972415926609:3469]
2025-10-09T21:43:44.552836Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:44.552875Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:44.554356Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 191
2025-10-09T21:43:44.554393Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 191
2025-10-09T21:43:44.556972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 192 partition: 0 MaxSeqNo: 25 sessionId: src_id|7bffe9e1-f6c13f80-7ea2c26-1bba98f4_0
2025-10-09T21:43:44.557285Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340972777736330:3142]
2025-10-09T21:43:44.557721Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 191, pipe: [1:7559340972777736330:3142]
2025-10-09T21:43:44.555057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:44.555073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:44.555110Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7bffe9e1-f6c13f80-7ea2c26-1bba98f4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:44.555135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.555140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.555143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:44.555157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.555174Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:44.555196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:44.556807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:44.556814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:44.556839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:44.557586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340972777736330:3142], now have 1 active actors on pipe
2025-10-09T21:43:44.558307Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e409dab-8087413b-b461c95a-ed4d936b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7bffe9e1-f6c13f80-7ea2c26-1bba98f4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:44.560886Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7bffe9e1-f6c13f80-7ea2c26-1bba98f4_0] Write session: try to update token
2025-10-09T21:43:44.560908Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7bffe9e1-f6c13f80-7ea2c26-1bba98f4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:44.584742Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:44.584765Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:44.584768Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:44.584770Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:44.584779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340972415926609:3469]
2025-10-09T21:43:44.584783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340972415926609:3469]
2025-10-09T21:43:44.584788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340972415926609:3469]
2025-10-09T21:43:44.584834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:44.584840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.584843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:44.584974Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:44.589256Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:44.589281Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:44.589285Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:44.602086Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 192 sessionId: src_id|7bffe9e1-f6c13f80-7ea2c26-1bba98f4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:44.604684Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:44.757257Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:44.757823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:44.757840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:44.762512Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:44.766172Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bffe9e1-f6c13f80-7ea2c26-1bba98f4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:44.766186Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bffe9e1-f6c13f80-7ea2c26-1bba98f4_0] Write session will restart in 0.008551s
2025-10-09T21:43:44.766211Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bffe9e1-f6c13f80-7ea2c26-1bba98f4_0] Write session: Do CDS request
2025-10-09T21:43:44.766218Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bffe9e1-f6c13f80-7ea2c26-1bba98f4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:44.764782Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:44.764875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 192 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7bffe9e1-f6c13f80-7ea2c26-1bba98f4_0
2025-10-09T21:43:44.765063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 192 sessionId: src_id|7bffe9e1-f6c13f80-7ea2c26-1bba98f4_0 is DEAD
2025-10-09T21:43:44.765070Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:44.765112Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340972777736330:3142]
2025-10-09T21:43:44.775586Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7bffe9e1-f6c13f80-7ea2c26-1bba98f4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:44.775794Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:44.775812Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 193
2025-10-09T21:43:44.776022Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 193 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:44.776053Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 193 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:44.776059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=193 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:44.776063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:44.776990Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 193 sessionId:  describe result for acl check
2025-10-09T21:43:44.777022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:44.777027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:44.777028Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:44.777036Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340972777736350:3737] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:44.777886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340972777736350:3737] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:44.782900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340972777736350:3737] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:44.817907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340972777736350:3737] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:44.817929Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340972777736350:3737] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:44.817933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340972777736350:3737] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:44.817945Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 193 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:44.866966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:44.866506Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340972777736604:3142]
2025-10-09T21:43:44.867335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:44.867522Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:44.867535Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:44.867549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:44.867600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:44.867660Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340972415926882:3471]
2025-10-09T21:43:44.868298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:44.868880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:44.868933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:44.869262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:44.869320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:44.869327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:44.869328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:44.869330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:44.869332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:44.869333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:44.869335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.869336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.869338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.869343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:44.869354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:44.869358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:44.869363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:44.869367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:44.869371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:44.869375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.869379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.869382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.869389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:44.869402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:44.869412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:44.869418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:44.869426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:44.869431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:44.869436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:44.869462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:44.869467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:44.869473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:44.869478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:44.869487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:44.869492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.869501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.869505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.869510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.869514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.869519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.869553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:44.887237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:44.888605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:44.888612Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:44.888614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:44.888619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:44.888626Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 192 [2:7559340972415926882:3471]
2025-10-09T21:43:44.888640Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:44.888660Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:44.888664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.888667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.888670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.888689Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:44.888695Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:44.888702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:44.888765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.888767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.888769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.888773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:44.888858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:44.888861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.888877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:44.888889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:44.888906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:44.888931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340972415926875:3471]
2025-10-09T21:43:44.888934Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340972415926875:3471]
2025-10-09T21:43:44.888936Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340972415926875:3471]
2025-10-09T21:43:44.888945Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:44.888961Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:44.888977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:44.888980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:44.889027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340972415926875:3471]
2025-10-09T21:43:44.889029Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:44.889124Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:44.889126Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:44.889128Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:44.889134Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:44.890047Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 192, pipe: [1:7559340972777736604:3142]
2025-10-09T21:43:44.890089Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 192
2025-10-09T21:43:44.890116Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 192
2025-10-09T21:43:44.895749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340972777736604:3142], now have 1 active actors on pipe
2025-10-09T21:43:44.895769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340972777736337:2432], now have 1 active actors on pipe
2025-10-09T21:43:44.895773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340972777736450:3737], now have 1 active actors on pipe
2025-10-09T21:43:44.895806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:44.895813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:44.895851Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b5c358ce-9f6604f0-6487b1a2-5b2da77c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:44.895864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.895869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.895872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:44.895886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.895905Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:44.895926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:44.896316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:44.896322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:44.896335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:44.896471Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 193 partition: 0 MaxSeqNo: 25 sessionId: src_id|b5c358ce-9f6604f0-6487b1a2-5b2da77c_0
2025-10-09T21:43:44.898191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bffe9e1-f6c13f80-7ea2c26-1bba98f4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b5c358ce-9f6604f0-6487b1a2-5b2da77c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:44.900876Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5c358ce-9f6604f0-6487b1a2-5b2da77c_0] Write session: try to update token
2025-10-09T21:43:44.900899Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5c358ce-9f6604f0-6487b1a2-5b2da77c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:44.935153Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:44.935183Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:44.935186Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:44.935189Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:44.935200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340972415926875:3471]
2025-10-09T21:43:44.935208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340972415926875:3471]
2025-10-09T21:43:44.935214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340972415926875:3471]
2025-10-09T21:43:44.935273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:44.935289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:44.935295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:44.935492Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:44.939602Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:44.939626Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:44.939637Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:44.938076Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 193 sessionId: src_id|b5c358ce-9f6604f0-6487b1a2-5b2da77c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:44.940736Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:45.140019Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:45.140722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:45.140734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:45.142627Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340972777736604:3142]
2025-10-09T21:43:45.142684Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:45.142743Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:45.142755Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:45.142836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 193 reason: error at write: Disconnected sessionId: src_id|b5c358ce-9f6604f0-6487b1a2-5b2da77c_0
2025-10-09T21:43:45.143302Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5c358ce-9f6604f0-6487b1a2-5b2da77c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:45.143315Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5c358ce-9f6604f0-6487b1a2-5b2da77c_0] Write session will restart in 0.007893s
2025-10-09T21:43:45.143359Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5c358ce-9f6604f0-6487b1a2-5b2da77c_0] Write session: Do CDS request
2025-10-09T21:43:45.143366Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5c358ce-9f6604f0-6487b1a2-5b2da77c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:45.143076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 193 sessionId: src_id|b5c358ce-9f6604f0-6487b1a2-5b2da77c_0 is DEAD
2025-10-09T21:43:45.143214Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:45.152699Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5c358ce-9f6604f0-6487b1a2-5b2da77c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:45.156588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:45.156606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 194
2025-10-09T21:43:45.156816Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 194 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:45.156857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 194 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:45.156862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=194 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:45.156867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:45.157452Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 194 sessionId:  describe result for acl check
2025-10-09T21:43:45.157493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:45.157495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:45.157497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:45.157503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340977072703940:3749] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:45.158141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340977072703940:3749] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:45.161906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340977072703940:3749] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:45.170956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340977072703940:3749] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:45.170982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340977072703940:3749] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:45.170986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340977072703940:3749] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:45.170997Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 194 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:45.225311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:45.232008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:45.232356Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:45.232362Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:45.232377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:45.232452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:45.232501Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340976710894450:3487]
2025-10-09T21:43:45.233127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:45.238457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:45.238558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:45.238932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:45.238977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:45.238978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:45.238979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:45.238980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:45.238981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:45.238983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:45.238984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.238985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.238986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.238993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:45.239003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:45.239006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:45.239010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:45.239013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:45.239017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:45.239020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.239024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.239027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.239035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:45.239043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:45.239049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:45.239054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:45.239058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:45.239063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:45.239067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:45.239072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:45.239077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:45.239097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:45.239100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:45.239105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:45.239110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.239114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.239119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.239124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.239128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.239133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.239162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:45.242771Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340977072704180:3142]
2025-10-09T21:43:45.248372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:45.250161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:45.250168Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:45.250171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:45.250176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:45.250186Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 193 [2:7559340976710894450:3487]
2025-10-09T21:43:45.250201Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:45.250223Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:45.250227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.250230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.250232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.250252Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:45.250259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:45.250267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:45.250335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.250336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.250338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.250340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:45.250435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:45.250438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.250452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.250461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.250475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:45.250498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340976710894444:3487]
2025-10-09T21:43:45.250501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340976710894444:3487]
2025-10-09T21:43:45.250503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340976710894444:3487]
2025-10-09T21:43:45.250513Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:45.250529Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:45.250543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:45.250547Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:45.250592Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340976710894444:3487]
2025-10-09T21:43:45.250595Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:45.251133Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:45.251141Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:45.251143Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:45.251149Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:45.251944Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 193
2025-10-09T21:43:45.251979Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 193, pipe: [1:7559340977072704180:3142]
2025-10-09T21:43:45.251997Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 193
2025-10-09T21:43:45.262123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340977072703976:3749], now have 1 active actors on pipe
2025-10-09T21:43:45.262148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340977072704180:3142], now have 1 active actors on pipe
2025-10-09T21:43:45.262152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340977072703930:2432], now have 1 active actors on pipe
2025-10-09T21:43:45.262181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:45.262191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:45.262235Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e49a1cf4-9a6a7231-cd4adbb8-9deb637b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:45.262247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.262253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.262256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:45.262272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.262288Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:45.262322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:45.264269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:45.264286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:45.264319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:45.264944Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 194 partition: 0 MaxSeqNo: 25 sessionId: src_id|e49a1cf4-9a6a7231-cd4adbb8-9deb637b_0
2025-10-09T21:43:45.265367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5c358ce-9f6604f0-6487b1a2-5b2da77c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e49a1cf4-9a6a7231-cd4adbb8-9deb637b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:45.272345Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e49a1cf4-9a6a7231-cd4adbb8-9deb637b_0] Write session: try to update token
2025-10-09T21:43:45.272378Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e49a1cf4-9a6a7231-cd4adbb8-9deb637b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:45.282362Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:45.282381Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:45.282387Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:45.282389Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:45.282399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340976710894444:3487]
2025-10-09T21:43:45.282407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340976710894444:3487]
2025-10-09T21:43:45.282411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340976710894444:3487]
2025-10-09T21:43:45.282477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:45.282484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.282487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:45.282518Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:45.282524Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:45.282532Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:45.282702Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:45.297202Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 194 sessionId: src_id|e49a1cf4-9a6a7231-cd4adbb8-9deb637b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:45.299929Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:45.455080Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:45.455441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:45.455455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:45.456053Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:45.456519Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:45.456615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 194 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e49a1cf4-9a6a7231-cd4adbb8-9deb637b_0
2025-10-09T21:43:45.456805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 194 sessionId: src_id|e49a1cf4-9a6a7231-cd4adbb8-9deb637b_0 is DEAD
2025-10-09T21:43:45.456910Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340977072704180:3142]
2025-10-09T21:43:45.456933Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:45.457620Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e49a1cf4-9a6a7231-cd4adbb8-9deb637b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:45.457631Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e49a1cf4-9a6a7231-cd4adbb8-9deb637b_0] Write session will restart in 0.007556s
2025-10-09T21:43:45.457656Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e49a1cf4-9a6a7231-cd4adbb8-9deb637b_0] Write session: Do CDS request
2025-10-09T21:43:45.457664Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e49a1cf4-9a6a7231-cd4adbb8-9deb637b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:45.466927Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e49a1cf4-9a6a7231-cd4adbb8-9deb637b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:45.467176Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:45.467194Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 195
2025-10-09T21:43:45.470270Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 195 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:45.470325Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 195 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:45.470332Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=195 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:45.470337Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:45.470896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 195 sessionId:  describe result for acl check
2025-10-09T21:43:45.470942Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:45.470944Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:45.471012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:45.471030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340977072704207:3754] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:45.474146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340977072704207:3754] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:45.482471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340977072704207:3754] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:45.491261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340977072704207:3754] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:45.491279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340977072704207:3754] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:45.491283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340977072704207:3754] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:45.491293Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 195 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:45.518770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:45.530827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:45.531119Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:45.531124Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:45.531138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:45.531202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:45.531245Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340976710894699:3489]
2025-10-09T21:43:45.531921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:45.538445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:45.538547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:45.538972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:45.539029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:45.539031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:45.539033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:45.539034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:45.539036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:45.539038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:45.539040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.539041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.539043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.539052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:45.539064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:45.539069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:45.539075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:45.539079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:45.539084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:45.539089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.539094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.539099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.539109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:45.539120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:45.539127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:45.539134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:45.539141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:45.539148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:45.539153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:45.539159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:45.539165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:45.539171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:45.539177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:45.539183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:45.539189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.539196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.539204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.539209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.539215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.539221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.539256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:45.557469Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340977072704446:3142]
2025-10-09T21:43:45.559165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:45.560678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:45.560685Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:45.560688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:45.560693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:45.560701Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 194 [2:7559340976710894699:3489]
2025-10-09T21:43:45.560717Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:45.560738Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:45.560742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.560746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.560748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.560768Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:45.560778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:45.560788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:45.560857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.560860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.560862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.560865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:45.560962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:45.560966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.560981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.560996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.561016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:45.561038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340976710894694:3489]
2025-10-09T21:43:45.561041Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340976710894694:3489]
2025-10-09T21:43:45.561043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340976710894694:3489]
2025-10-09T21:43:45.561058Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:45.561073Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:45.561090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:45.561094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:45.561137Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340976710894694:3489]
2025-10-09T21:43:45.561140Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:45.561639Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:45.561647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:45.561650Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:45.561657Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:45.561795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340977072704333:3754], now have 1 active actors on pipe
2025-10-09T21:43:45.561800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340977072704446:3142], now have 1 active actors on pipe
2025-10-09T21:43:45.561804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340977072704192:2432], now have 1 active actors on pipe
2025-10-09T21:43:45.562366Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 194
2025-10-09T21:43:45.562396Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 194, pipe: [1:7559340977072704446:3142]
2025-10-09T21:43:45.562428Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 194
2025-10-09T21:43:45.567901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:45.567924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:45.567971Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|57db1c15-255f92e6-a9255674-e2386556_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:45.567985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.567992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.567995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:45.568012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.568033Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:45.568057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:45.573531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:45.573553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:45.573594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:45.574186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 195 partition: 0 MaxSeqNo: 25 sessionId: src_id|57db1c15-255f92e6-a9255674-e2386556_0
2025-10-09T21:43:45.574600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e49a1cf4-9a6a7231-cd4adbb8-9deb637b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|57db1c15-255f92e6-a9255674-e2386556_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:45.577178Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57db1c15-255f92e6-a9255674-e2386556_0] Write session: try to update token
2025-10-09T21:43:45.577198Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57db1c15-255f92e6-a9255674-e2386556_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:45.601322Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:45.601342Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:45.601345Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:45.601347Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:45.601358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340976710894694:3489]
2025-10-09T21:43:45.601362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340976710894694:3489]
2025-10-09T21:43:45.601366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340976710894694:3489]
2025-10-09T21:43:45.601418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:45.601426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.601429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:45.601491Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:45.601498Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:45.601502Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:45.601607Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:45.608040Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 195 sessionId: src_id|57db1c15-255f92e6-a9255674-e2386556_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:45.610696Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:45.792450Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:45.792940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:45.792959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:45.793626Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:45.793635Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:45.793644Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:45.793687Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340977072704446:3142]
2025-10-09T21:43:45.793718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 195 reason: error at write: Disconnected sessionId: src_id|57db1c15-255f92e6-a9255674-e2386556_0
2025-10-09T21:43:45.793902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 195 sessionId: src_id|57db1c15-255f92e6-a9255674-e2386556_0 is DEAD
2025-10-09T21:43:45.794437Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57db1c15-255f92e6-a9255674-e2386556_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:45.794447Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57db1c15-255f92e6-a9255674-e2386556_0] Write session will restart in 0.006616s
2025-10-09T21:43:45.794472Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57db1c15-255f92e6-a9255674-e2386556_0] Write session: Do CDS request
2025-10-09T21:43:45.794479Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57db1c15-255f92e6-a9255674-e2386556_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:45.793726Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:45.806908Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57db1c15-255f92e6-a9255674-e2386556_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:45.808608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:45.808627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 196
2025-10-09T21:43:45.808847Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 196 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:45.808889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 196 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:45.808895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=196 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:45.808900Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:45.812160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 196 sessionId:  describe result for acl check
2025-10-09T21:43:45.812202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:45.812205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:45.812210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:45.812218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340977072704473:3758] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:45.813008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340977072704473:3758] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:45.823473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340977072704473:3758] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:45.841555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340977072704473:3758] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:45.841583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340977072704473:3758] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:45.841589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340977072704473:3758] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:45.841599Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 196 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:45.879823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:45.883770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:45.884076Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:45.884081Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:45.884095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:45.884162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:45.884207Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340976710894955:3492]
2025-10-09T21:43:45.884912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:45.885329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:45.885391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:45.885762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:45.885812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:45.885813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:45.885814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:45.885815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:45.885817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:45.885818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:45.885819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.885820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.885821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.885827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:45.885837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:45.885841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:45.885844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:45.885848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:45.885851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:45.885854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.885857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.885861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.885868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:45.885876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:45.885881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:45.885886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:45.885890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:45.885895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:45.885899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:45.885905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:45.885910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:45.885914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:45.885918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:45.885923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:45.885927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.885932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.885936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.885940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.885945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.885951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.885980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:45.893897Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340977072704719:3142]
2025-10-09T21:43:45.900984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:45.904851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 195
2025-10-09T21:43:45.904893Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 195, pipe: [1:7559340977072704719:3142]
2025-10-09T21:43:45.904916Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 195
2025-10-09T21:43:45.902481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:45.902500Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:45.902503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:45.902507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:45.902515Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 195 [2:7559340976710894955:3492]
2025-10-09T21:43:45.902531Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:45.902552Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:45.902555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.902559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.902561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.902583Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:45.902589Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:45.902596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:45.902667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.902669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.902671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.902674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:45.902754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:45.902757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.902773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:45.902786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:45.902802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:45.902824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340976710894949:3492]
2025-10-09T21:43:45.902827Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340976710894949:3492]
2025-10-09T21:43:45.902829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340976710894949:3492]
2025-10-09T21:43:45.902848Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:45.902865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:45.902881Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:45.902884Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:45.902928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340976710894949:3492]
2025-10-09T21:43:45.902932Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:45.903315Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:45.903321Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:45.903323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:45.903330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:45.904113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340977072704540:3758], now have 1 active actors on pipe
2025-10-09T21:43:45.904119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340977072704719:3142], now have 1 active actors on pipe
2025-10-09T21:43:45.904124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340977072704456:2432], now have 1 active actors on pipe
2025-10-09T21:43:45.905249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:45.905269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:45.905306Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cfb6b0ed-91a2ad1f-cb2b8a1a-55f496a1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:45.905328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.905334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.905336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:45.905350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.905368Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:45.905407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:45.906672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 196 partition: 0 MaxSeqNo: 25 sessionId: src_id|cfb6b0ed-91a2ad1f-cb2b8a1a-55f496a1_0
2025-10-09T21:43:45.907299Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57db1c15-255f92e6-a9255674-e2386556_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cfb6b0ed-91a2ad1f-cb2b8a1a-55f496a1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:45.906412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:45.906438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:45.906463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:45.910225Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cfb6b0ed-91a2ad1f-cb2b8a1a-55f496a1_0] Write session: try to update token
2025-10-09T21:43:45.910246Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cfb6b0ed-91a2ad1f-cb2b8a1a-55f496a1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:45.930266Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:45.930286Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:45.930290Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:45.930293Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:45.930307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340976710894949:3492]
2025-10-09T21:43:45.930313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340976710894949:3492]
2025-10-09T21:43:45.930318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340976710894949:3492]
2025-10-09T21:43:45.930375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:45.930384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:45.930387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:45.930693Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:45.934101Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:45.934108Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:45.934112Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:45.943140Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 196 sessionId: src_id|cfb6b0ed-91a2ad1f-cb2b8a1a-55f496a1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:45.945856Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:46.123413Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:46.123812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:46.123822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:46.125386Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cfb6b0ed-91a2ad1f-cb2b8a1a-55f496a1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:46.125401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cfb6b0ed-91a2ad1f-cb2b8a1a-55f496a1_0] Write session will restart in 0.005892s
2025-10-09T21:43:46.125433Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cfb6b0ed-91a2ad1f-cb2b8a1a-55f496a1_0] Write session: Do CDS request
2025-10-09T21:43:46.125440Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cfb6b0ed-91a2ad1f-cb2b8a1a-55f496a1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:46.124980Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:46.125057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 196 reason: status is not ok: tablet will be restarted right now sessionId: src_id|cfb6b0ed-91a2ad1f-cb2b8a1a-55f496a1_0
2025-10-09T21:43:46.125232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 196 sessionId: src_id|cfb6b0ed-91a2ad1f-cb2b8a1a-55f496a1_0 is DEAD
2025-10-09T21:43:46.125429Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340977072704719:3142]
2025-10-09T21:43:46.125442Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:46.124421Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:46.132570Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cfb6b0ed-91a2ad1f-cb2b8a1a-55f496a1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:46.132699Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:46.132716Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 197
2025-10-09T21:43:46.132862Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 197 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:46.132897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 197 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:46.132904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=197 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:46.132907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:46.133402Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 197 sessionId:  describe result for acl check
2025-10-09T21:43:46.133440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:46.133441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:46.133443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:46.133448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340981367672047:3769] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:46.134014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340981367672047:3769] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:46.140921Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340981367672047:3769] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:46.155710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340981367672047:3769] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:46.155731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340981367672047:3769] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:46.155735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340981367672047:3769] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:46.155745Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 197 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:46.180762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:46.182904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:46.183143Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:46.183148Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:46.183161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:46.183223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:46.183263Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340981005862543:3505]
2025-10-09T21:43:46.183981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:46.184415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:46.184469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:46.184813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:46.184874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:46.184875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:46.184876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:46.184878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:46.184879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:46.184880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:46.184881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.184882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.184883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.184889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:46.184900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:46.184905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:46.184908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:46.184912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:46.184915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:46.184918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.184921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.184925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.184933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:46.184941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:46.184947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:46.184952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:46.184956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:46.184961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:46.184966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:46.184970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:46.184975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:46.184979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:46.184983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:46.184987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:46.184994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.184999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.185003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.185007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.185011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.185016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.185043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:46.195898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:46.197183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:46.197191Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:46.197193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:46.197199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:46.197210Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 196 [2:7559340981005862543:3505]
2025-10-09T21:43:46.197227Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:46.197250Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:46.197255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.197259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.197261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.197281Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:46.197287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:46.197294Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:46.197361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.197363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.197365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.197368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:46.197452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:46.197455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.197470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.197480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.197497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:46.197520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340981005862537:3505]
2025-10-09T21:43:46.197523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340981005862537:3505]
2025-10-09T21:43:46.197525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340981005862537:3505]
2025-10-09T21:43:46.197541Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:46.197556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:46.197572Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:46.197575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:46.197623Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340981005862537:3505]
2025-10-09T21:43:46.197625Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:46.197911Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:46.197922Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:46.197924Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:46.197931Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:46.198695Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 196
2025-10-09T21:43:46.198743Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 196
2025-10-09T21:43:46.199723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340981367672111:3769], now have 1 active actors on pipe
2025-10-09T21:43:46.199736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340981367672040:2432], now have 1 active actors on pipe
2025-10-09T21:43:46.199755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:46.199762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:46.199801Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5c453215-de0c58d8-1c484e0c-8a82d82e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:46.199816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.199825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.199828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:46.199843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.199861Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:46.199883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:46.200241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:46.200254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:46.200272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:46.200436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 197 partition: 0 MaxSeqNo: 25 sessionId: src_id|5c453215-de0c58d8-1c484e0c-8a82d82e_0
2025-10-09T21:43:46.200916Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cfb6b0ed-91a2ad1f-cb2b8a1a-55f496a1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5c453215-de0c58d8-1c484e0c-8a82d82e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:46.203521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c453215-de0c58d8-1c484e0c-8a82d82e_0] Write session: try to update token
2025-10-09T21:43:46.203540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c453215-de0c58d8-1c484e0c-8a82d82e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:46.223597Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:46.223619Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:46.223622Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:46.223624Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:46.223635Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340981005862537:3505]
2025-10-09T21:43:46.223640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340981005862537:3505]
2025-10-09T21:43:46.223648Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340981005862537:3505]
2025-10-09T21:43:46.223701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:46.223716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.223719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:46.223916Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:46.226436Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:46.226454Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:46.226458Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:46.224155Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340981367672307:3142]
2025-10-09T21:43:46.227558Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 196, pipe: [1:7559340981367672307:3142]
2025-10-09T21:43:46.241165Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 197 sessionId: src_id|5c453215-de0c58d8-1c484e0c-8a82d82e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:46.244131Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:46.440001Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:46.440293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340981367672307:3142], now have 1 active actors on pipe
2025-10-09T21:43:46.440353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:46.440365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:46.442294Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:46.442899Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:46.442975Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 197 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5c453215-de0c58d8-1c484e0c-8a82d82e_0
2025-10-09T21:43:46.443150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 197 sessionId: src_id|5c453215-de0c58d8-1c484e0c-8a82d82e_0 is DEAD
2025-10-09T21:43:46.443275Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340981367672307:3142]
2025-10-09T21:43:46.443288Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:46.443637Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c453215-de0c58d8-1c484e0c-8a82d82e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:46.443645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c453215-de0c58d8-1c484e0c-8a82d82e_0] Write session will restart in 0.007744s
2025-10-09T21:43:46.443673Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c453215-de0c58d8-1c484e0c-8a82d82e_0] Write session: Do CDS request
2025-10-09T21:43:46.443681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c453215-de0c58d8-1c484e0c-8a82d82e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:46.455299Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c453215-de0c58d8-1c484e0c-8a82d82e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:46.455525Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:46.455542Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 198
2025-10-09T21:43:46.455742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 198 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:46.455792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 198 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:46.455798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=198 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:46.455802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:46.456241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 198 sessionId:  describe result for acl check
2025-10-09T21:43:46.456288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:46.456293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:46.456295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:46.456300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340981367672322:3775] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:46.456868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340981367672322:3775] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:46.462322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340981367672322:3775] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:46.474256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340981367672322:3775] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:46.474273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340981367672322:3775] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:46.474277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340981367672322:3775] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:46.474288Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 198 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:46.509732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:46.512597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:46.512892Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:46.512898Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:46.512912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:46.512974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:46.513039Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340981005862796:3509]
2025-10-09T21:43:46.513617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:46.514194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:46.514250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:46.514600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:46.514673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:46.514674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:46.514676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:46.514677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:46.514679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:46.514680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:46.514681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.514682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.514683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.514688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:46.514699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:46.514702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:46.514706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:46.514710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:46.514714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:46.514717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.514722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.514725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.514733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:46.514742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:46.514748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:46.514752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:46.514759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:46.514764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:46.514768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:46.514773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:46.514777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:46.514782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:46.514786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:46.514791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:46.514795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.514800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.514804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.514828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.514834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.514840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.514866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:46.534528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:46.536071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:46.536078Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:46.536081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:46.536086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:46.536094Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 197 [2:7559340981005862796:3509]
2025-10-09T21:43:46.536112Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:46.536132Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:46.536136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.536140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.536142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.536161Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:46.536166Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:46.536174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:46.536242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.536244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.536246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.536249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:46.536326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:46.536329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.536345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.536358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.536375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:46.536395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340981005862790:3509]
2025-10-09T21:43:46.536398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340981005862790:3509]
2025-10-09T21:43:46.536400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340981005862790:3509]
2025-10-09T21:43:46.536415Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:46.536432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:46.536447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:46.536450Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:46.536492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340981005862790:3509]
2025-10-09T21:43:46.536495Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:46.537003Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:46.537013Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:46.537015Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:46.537023Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:46.537211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340981367672313:2432], now have 1 active actors on pipe
2025-10-09T21:43:46.537262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340981367672404:3775], now have 1 active actors on pipe
2025-10-09T21:43:46.537601Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 197
2025-10-09T21:43:46.537650Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 197
2025-10-09T21:43:46.538462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:46.538479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:46.538515Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5c021ba4-7fe5eba0-94b216cb-7ec2afd8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:46.538528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.538534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.538537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:46.538552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.538571Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:46.538590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:46.543280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:46.543300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:46.543342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:46.543367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340981367672569:3142], now have 1 active actors on pipe
2025-10-09T21:43:46.543905Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c453215-de0c58d8-1c484e0c-8a82d82e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5c021ba4-7fe5eba0-94b216cb-7ec2afd8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:46.543089Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340981367672569:3142]
2025-10-09T21:43:46.543525Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 198 partition: 0 MaxSeqNo: 25 sessionId: src_id|5c021ba4-7fe5eba0-94b216cb-7ec2afd8_0
2025-10-09T21:43:46.543698Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 197, pipe: [1:7559340981367672569:3142]
2025-10-09T21:43:46.546671Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c021ba4-7fe5eba0-94b216cb-7ec2afd8_0] Write session: try to update token
2025-10-09T21:43:46.546693Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c021ba4-7fe5eba0-94b216cb-7ec2afd8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:46.585211Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 198 sessionId: src_id|5c021ba4-7fe5eba0-94b216cb-7ec2afd8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:46.587904Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:46.589250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:46.589273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:46.590789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:46.590822Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:46.590825Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:46.590827Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:46.590829Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:46.590839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340981005862790:3509]
2025-10-09T21:43:46.590844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340981005862790:3509]
2025-10-09T21:43:46.590848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340981005862790:3509]
2025-10-09T21:43:46.590890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:46.590899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.590909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:46.591041Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:46.594363Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:46.594376Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:46.594381Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:46.594836Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:46.827992Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:46.855699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:46.855725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:46.855901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:46.855964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:46.856022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:46.856080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:46.856121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:46.856136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:46.856202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:46.856267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:46.856392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:46.856482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:46.856522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:46.856526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:46.856623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:46.856680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:46.856772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:46.856854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:46.856989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:46.856993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:46.857107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:46.857177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:46.857240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:46.857305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:46.857344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:46.857347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:46.857420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:46.857483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:46.857593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:46.857693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:46.857752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:46.857760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:46.857829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:46.857935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:46.857993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:46.858096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:46.858161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:46.858164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:46.858230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:46.858319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:46.858390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:46.858526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:46.858586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:46.858589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:46.858711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:46.858764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:46.858828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:46.858891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:46.858933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:46.858936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:46.859002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:46.859066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:46.859126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:46.859194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:46.859257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:46.859265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:46.859333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:46.859397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:46.859457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:46.859525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:46.859570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:46.859578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:46.860821Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:46.861631Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:46.861764Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340981367672569:3142]
2025-10-09T21:43:46.861794Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:46.861833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 198 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5c021ba4-7fe5eba0-94b216cb-7ec2afd8_0
2025-10-09T21:43:46.862022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 198 sessionId: src_id|5c021ba4-7fe5eba0-94b216cb-7ec2afd8_0 is DEAD
2025-10-09T21:43:46.862469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c021ba4-7fe5eba0-94b216cb-7ec2afd8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:46.862481Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c021ba4-7fe5eba0-94b216cb-7ec2afd8_0] Write session will restart in 0.007339s
2025-10-09T21:43:46.862511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c021ba4-7fe5eba0-94b216cb-7ec2afd8_0] Write session: Do CDS request
2025-10-09T21:43:46.862522Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c021ba4-7fe5eba0-94b216cb-7ec2afd8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:46.871543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c021ba4-7fe5eba0-94b216cb-7ec2afd8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:46.871850Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:46.871873Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 199
2025-10-09T21:43:46.872086Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 199 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:46.872122Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 199 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:46.872129Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=199 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:46.872132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:46.872607Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 199 sessionId:  describe result for acl check
2025-10-09T21:43:46.872643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:46.872645Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:46.872651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:46.872658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340981367672601:3782] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:46.873258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340981367672601:3782] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:46.886900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340981367672601:3782] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:46.899789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340981367672601:3782] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:46.899809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340981367672601:3782] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:46.899813Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340981367672601:3782] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:46.899824Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 199 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:46.919285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:46.920007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:46.920217Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:46.920221Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:46.920235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:46.920314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:46.920352Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340981005863057:3512]
2025-10-09T21:43:46.920930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:46.921569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:46.921621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:46.921956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:46.922018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:46.922019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:46.922020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:46.922021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:46.922023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:46.922024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:46.922025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.922026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.922054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.922061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:46.922072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:46.922075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:46.922079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:46.922082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:46.922086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:46.922089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.922092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.922095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.922114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:46.922122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:46.922129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:46.922134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:46.922138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:46.922143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:46.922148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:46.922152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:46.922157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:46.922161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:46.922166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:46.922171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:46.922176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.922181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.922185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.922190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.922195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.922201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.922229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:46.933308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:46.934636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:46.934641Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:46.934644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:46.934648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:46.934657Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 198 [2:7559340981005863057:3512]
2025-10-09T21:43:46.934673Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:46.934694Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:46.934699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.934702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.934704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.934722Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:46.934728Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:46.934735Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:46.934813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.934815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.934820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.934834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:46.934936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:46.934938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.934951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:46.934963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:46.934980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:46.935000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340981005863049:3512]
2025-10-09T21:43:46.935003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340981005863049:3512]
2025-10-09T21:43:46.935005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340981005863049:3512]
2025-10-09T21:43:46.935018Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:46.935032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:46.935046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:46.935050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:46.935091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340981005863049:3512]
2025-10-09T21:43:46.935093Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:46.935481Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:46.935489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:46.935491Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:46.935497Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:46.936175Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 198
2025-10-09T21:43:46.936215Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 198
2025-10-09T21:43:46.940247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340981367672587:2432], now have 1 active actors on pipe
2025-10-09T21:43:46.940261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340981367672727:3782], now have 1 active actors on pipe
2025-10-09T21:43:46.940290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:46.940298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:46.940353Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5a677ea6-dd02a169-38a29b89-5c1fa20a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:46.940363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.940370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.940374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:46.940388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.940408Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:46.940430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:46.940795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:46.940802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:46.940816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:46.940959Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 199 partition: 0 MaxSeqNo: 25 sessionId: src_id|5a677ea6-dd02a169-38a29b89-5c1fa20a_0
2025-10-09T21:43:46.941569Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c021ba4-7fe5eba0-94b216cb-7ec2afd8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5a677ea6-dd02a169-38a29b89-5c1fa20a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:46.964430Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340981367672848:3142]
2025-10-09T21:43:46.965613Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 198, pipe: [1:7559340981367672848:3142]
2025-10-09T21:43:46.971482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340981367672848:3142], now have 1 active actors on pipe
2025-10-09T21:43:46.971505Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:46.971508Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:46.971511Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:46.971513Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:46.971521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340981005863049:3512]
2025-10-09T21:43:46.971526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340981005863049:3512]
2025-10-09T21:43:46.971530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340981005863049:3512]
2025-10-09T21:43:46.971576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:46.971584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:46.971586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:46.971687Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:46.971693Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:46.971697Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:46.971721Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:46.982466Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a677ea6-dd02a169-38a29b89-5c1fa20a_0] Write session: try to update token
2025-10-09T21:43:46.982495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a677ea6-dd02a169-38a29b89-5c1fa20a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:47.031217Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 199 sessionId: src_id|5a677ea6-dd02a169-38a29b89-5c1fa20a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:47.037397Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:47.203394Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:47.204497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:47.204514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:47.204715Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:47.205313Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340981367672848:3142]
2025-10-09T21:43:47.205348Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:47.205414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 199 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5a677ea6-dd02a169-38a29b89-5c1fa20a_0
2025-10-09T21:43:47.205589Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 199 sessionId: src_id|5a677ea6-dd02a169-38a29b89-5c1fa20a_0 is DEAD
2025-10-09T21:43:47.205702Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:47.206277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a677ea6-dd02a169-38a29b89-5c1fa20a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:47.206290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a677ea6-dd02a169-38a29b89-5c1fa20a_0] Write session will restart in 0.009427s
2025-10-09T21:43:47.206312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a677ea6-dd02a169-38a29b89-5c1fa20a_0] Write session: Do CDS request
2025-10-09T21:43:47.206319Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a677ea6-dd02a169-38a29b89-5c1fa20a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:47.216561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a677ea6-dd02a169-38a29b89-5c1fa20a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:47.216884Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:47.216910Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 200
2025-10-09T21:43:47.217208Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 200 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:47.217281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 200 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:47.217314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=200 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:47.217318Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:47.218106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 200 sessionId:  describe result for acl check
2025-10-09T21:43:47.218147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:47.218151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:47.218153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:47.218158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340985662640180:3792] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:47.218741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340985662640180:3792] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:47.233998Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340985662640180:3792] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:47.242693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340985662640180:3792] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:47.242713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340985662640180:3792] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:47.242717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340985662640180:3792] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:47.242727Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 200 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:47.248688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:47.250788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:47.251022Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:47.251034Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:47.251048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:47.251110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:47.251157Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340985300830647:3522]
2025-10-09T21:43:47.251751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:47.252312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:47.252401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:47.252766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:47.252840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:47.252848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:47.252849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:47.252850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:47.252852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:47.252853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:47.252854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:47.252855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:47.252857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:47.252864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:47.252875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:47.252879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:47.252882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:47.252886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:47.252890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:47.252893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:47.252897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:47.252900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:47.252908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:47.252926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:47.252932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:47.252938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:47.252942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:47.252947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:47.252952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:47.252957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:47.252961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:47.252966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:47.252970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:47.252974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:47.252979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:47.252983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:47.252988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:47.252993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:47.252997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:47.253002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:47.253033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:47.264825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:47.266172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:47.266186Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:47.266189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:47.266194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:47.266202Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 199 [2:7559340985300830647:3522]
2025-10-09T21:43:47.266221Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:47.266243Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:47.266258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.266270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.266273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.266292Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:47.266298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:47.266305Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:47.266382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:47.266390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:47.266393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:47.266395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:47.266482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:47.266490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:47.266504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:47.266519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:47.266537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:47.266575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340985662640356:3792], now have 1 active actors on pipe
2025-10-09T21:43:47.266585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340985662640162:2432], now have 1 active actors on pipe
2025-10-09T21:43:47.266594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340985300830642:3522]
2025-10-09T21:43:47.266600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340985300830642:3522]
2025-10-09T21:43:47.266603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340985300830642:3522]
2025-10-09T21:43:47.266617Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:47.266637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:47.266656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:47.266664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:47.266719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340985300830642:3522]
2025-10-09T21:43:47.266726Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:47.267017Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:47.267026Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:47.267028Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:47.267035Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:47.267327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:47.267341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:47.267387Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|86167cc5-a7c6dd4e-2ef9161d-3ae38161_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:47.267401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.267410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.267413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:47.267425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.267443Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:47.267468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:47.267717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:47.267725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:47.267742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:47.266808Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 199
2025-10-09T21:43:47.266885Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 199
2025-10-09T21:43:47.267951Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 200 partition: 0 MaxSeqNo: 25 sessionId: src_id|86167cc5-a7c6dd4e-2ef9161d-3ae38161_0
2025-10-09T21:43:47.270299Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a677ea6-dd02a169-38a29b89-5c1fa20a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|86167cc5-a7c6dd4e-2ef9161d-3ae38161_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:47.272798Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86167cc5-a7c6dd4e-2ef9161d-3ae38161_0] Write session: try to update token
2025-10-09T21:43:47.272816Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86167cc5-a7c6dd4e-2ef9161d-3ae38161_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:47.303768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:47.303794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:47.303851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:47.300677Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 200 sessionId: src_id|86167cc5-a7c6dd4e-2ef9161d-3ae38161_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:47.303523Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:47.303961Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:47.305852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340985662640431:3142], now have 1 active actors on pipe
2025-10-09T21:43:47.305432Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340985662640431:3142]
2025-10-09T21:43:47.305955Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 199, pipe: [1:7559340985662640431:3142]
2025-10-09T21:43:47.315323Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:47.315339Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:47.315341Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:47.315343Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:47.315352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340985300830642:3522]
2025-10-09T21:43:47.315357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340985300830642:3522]
2025-10-09T21:43:47.315362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340985300830642:3522]
2025-10-09T21:43:47.315407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:47.315415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:47.315425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:47.315648Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:47.317530Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:47.317536Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:47.317541Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:47.485985Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:47.494581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:47.494611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:47.494786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:47.494839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:47.494893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:47.494947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:47.494988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:47.494994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:47.495060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:47.495114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:47.495169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:47.495231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:47.495275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:47.495279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:47.495484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:47.495536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:47.495627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:47.495707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:47.495948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:47.495959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:47.496029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:47.496092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:47.496171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:47.496229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:47.496268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:47.496270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:47.496360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:47.496439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:47.496512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:47.496572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:47.496670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:47.496673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:47.496735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:47.496816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:47.496870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:47.496928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:47.496970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:47.496973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:47.497033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:47.497082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:47.497141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:47.497194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:47.497232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:47.497234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:47.497295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:47.497345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:47.497398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:47.497457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:47.497499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:47.497502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:47.497566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:47.497620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:47.497680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:47.497739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:47.497870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:47.497874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:47.497939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:47.497992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:47.498073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:47.498134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:47.498174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:47.498177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:47.500977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86167cc5-a7c6dd4e-2ef9161d-3ae38161_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:47.500991Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86167cc5-a7c6dd4e-2ef9161d-3ae38161_0] Write session will restart in 0.009562s
2025-10-09T21:43:47.501033Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86167cc5-a7c6dd4e-2ef9161d-3ae38161_0] Write session: Do CDS request
2025-10-09T21:43:47.501040Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86167cc5-a7c6dd4e-2ef9161d-3ae38161_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:47.500296Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:47.500312Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:47.500342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 200 reason: error at write: Disconnected sessionId: src_id|86167cc5-a7c6dd4e-2ef9161d-3ae38161_0
2025-10-09T21:43:47.500531Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 200 sessionId: src_id|86167cc5-a7c6dd4e-2ef9161d-3ae38161_0 is DEAD
2025-10-09T21:43:47.500676Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340985662640431:3142]
2025-10-09T21:43:47.500690Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:47.501467Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:47.514669Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86167cc5-a7c6dd4e-2ef9161d-3ae38161_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:47.515083Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:47.515110Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 201
2025-10-09T21:43:47.519843Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 201 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:47.519911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 201 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:47.519917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=201 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:47.519922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:47.520800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 201 sessionId:  describe result for acl check
2025-10-09T21:43:47.520857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:47.520859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:47.520861Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:47.520866Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340985662640448:3798] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:47.521427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340985662640448:3798] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:47.531240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340985662640448:3798] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:47.540023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340985662640448:3798] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:47.540045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340985662640448:3798] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:47.540049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340985662640448:3798] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:47.540059Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 201 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:47.567425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:47.572212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:47.572458Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:47.572469Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:47.572481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:47.572543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:47.572589Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340985300830904:3529]
2025-10-09T21:43:47.573219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:47.578177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:47.578263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:47.578668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:47.578713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:47.578714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:47.578715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:47.578717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:47.578719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:47.578720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:47.578721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:47.578723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:47.578724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:47.578731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:47.578744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:47.578749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:47.578753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:47.578757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:47.578762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:47.578766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:47.578771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:47.578774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:47.578782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:47.578790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:47.578797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:47.578803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:47.578809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:47.578814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:47.578819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:47.578825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:47.578829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:47.578835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:47.578839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:47.578843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:47.578848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:47.578854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:47.578858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:47.578863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:47.578868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:47.578873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:47.578904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:47.599719Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 200
2025-10-09T21:43:47.599765Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 200
2025-10-09T21:43:47.599788Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340985662640697:3142]
2025-10-09T21:43:47.596980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:47.598286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:47.598294Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:47.598296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:47.598302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:47.598310Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 200 [2:7559340985300830904:3529]
2025-10-09T21:43:47.598326Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:47.598348Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:47.598351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.598355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.598357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.598377Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:47.598384Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:47.598393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:47.598478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:47.598482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:47.598484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:47.598487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:47.598569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:47.598571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:47.598585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:47.598597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:47.598614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:47.598636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340985300830898:3529]
2025-10-09T21:43:47.598638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340985300830898:3529]
2025-10-09T21:43:47.598640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340985300830898:3529]
2025-10-09T21:43:47.598658Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:47.598674Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:47.598689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:47.598692Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:47.598730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:47.598734Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:47.598736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:47.598737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340985300830898:3529]
2025-10-09T21:43:47.598740Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:47.598743Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:47.598938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340985662640520:3798], now have 1 active actors on pipe
2025-10-09T21:43:47.598946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340985662640438:2432], now have 1 active actors on pipe
2025-10-09T21:43:47.602123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:47.602144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:47.602164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340985662640697:3142], now have 1 active actors on pipe
2025-10-09T21:43:47.602190Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|27c073c8-917c1335-9092368-c9bf6d67_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:47.602202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.602215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.602219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:47.602237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.602254Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:47.602276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:47.601232Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 200, pipe: [1:7559340985662640697:3142]
2025-10-09T21:43:47.609974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:47.610000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:47.610062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:47.610597Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 201 partition: 0 MaxSeqNo: 25 sessionId: src_id|27c073c8-917c1335-9092368-c9bf6d67_0
2025-10-09T21:43:47.614750Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86167cc5-a7c6dd4e-2ef9161d-3ae38161_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|27c073c8-917c1335-9092368-c9bf6d67_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:47.617524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|27c073c8-917c1335-9092368-c9bf6d67_0] Write session: try to update token
2025-10-09T21:43:47.617546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|27c073c8-917c1335-9092368-c9bf6d67_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:47.617388Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:47.617410Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:47.617413Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:47.617415Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:47.617426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340985300830898:3529]
2025-10-09T21:43:47.617431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340985300830898:3529]
2025-10-09T21:43:47.617434Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340985300830898:3529]
2025-10-09T21:43:47.617481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:47.617489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:47.617491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:47.617533Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:47.617540Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:47.617544Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:47.617633Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:47.648660Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 201 sessionId: src_id|27c073c8-917c1335-9092368-c9bf6d67_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:47.652254Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:48.113266Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:48.113280Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:48.113357Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 201 reason: error at write: Disconnected sessionId: src_id|27c073c8-917c1335-9092368-c9bf6d67_0
2025-10-09T21:43:48.113370Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340985662640697:3142]
2025-10-09T21:43:48.113392Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:48.113428Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:43:48.113538Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 2 consumer shared/user session shared/user_1_2_11159088021448487788_v1 is DEAD
2025-10-09T21:43:48.113560Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 201 sessionId: src_id|27c073c8-917c1335-9092368-c9bf6d67_0 is DEAD
2025-10-09T21:43:48.113674Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340856813599176:3139] disconnected.
2025-10-09T21:43:48.113680Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340856813599176:3139] disconnected; active server actors: 1
2025-10-09T21:43:48.113684Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340856813599176:3139] client user disconnected session shared/user_1_2_11159088021448487788_v1
2025-10-09T21:43:48.112214Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:48.112801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:48.112813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:48.116354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27c073c8-917c1335-9092368-c9bf6d67_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:48.116367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27c073c8-917c1335-9092368-c9bf6d67_0] Write session will restart in 0.005092s
2025-10-09T21:43:48.116413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27c073c8-917c1335-9092368-c9bf6d67_0] Write session: Do CDS request
2025-10-09T21:43:48.116420Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27c073c8-917c1335-9092368-c9bf6d67_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:48.116720Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:43:48.116806Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:43:48.116810Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:43:48.116816Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.008953s
2025-10-09T21:43:48.119491Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:48.126081Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|27c073c8-917c1335-9092368-c9bf6d67_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:48.126481Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:43:48.126563Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:48.126592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 202
2025-10-09T21:43:48.128424Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 202 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:48.128476Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 202 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:48.128483Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=202 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:48.128488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:48.128539Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:43:48.128542Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 3
2025-10-09T21:43:48.129409Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 3 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:43:48.129467Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:43:48.129613Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 auth for : user
2025-10-09T21:43:48.129908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 202 sessionId:  describe result for acl check
2025-10-09T21:43:48.129934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:48.129936Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:48.129937Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:48.129943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340989957608032:3803] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:48.130536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340989957608032:3803] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:48.130691Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 Handle describe topics response
2025-10-09T21:43:48.130721Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 auth is DEAD
2025-10-09T21:43:48.130740Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:43:48.131006Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:43:48.131181Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_3_11984607147350421102_v1
2025-10-09T21:43:48.131709Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340989957608036:3804] connected; active server actors: 1
2025-10-09T21:43:48.131817Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559340989957608036:3804] session shared/user_1_3_11984607147350421102_v1
2025-10-09T21:43:48.131829Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:43:48.131841Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:43:48.131860Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_3_11984607147350421102_v1" (Sender=[1:7559340989957608029:3804], Pipe=[1:7559340989957608036:3804], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:43:48.131865Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:43:48.131876Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:43:48.131885Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_3_11984607147350421102_v1" (Sender=[1:7559340989957608029:3804], Pipe=[1:7559340989957608036:3804], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:43:48.131896Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_3_11984607147350421102_v1" sender [1:7559340989957608029:3804] lock partition 0 for ReadingSession "shared/user_1_3_11984607147350421102_v1" (Sender=[1:7559340989957608029:3804], Pipe=[1:7559340989957608036:3804], Partitions=[], ActiveFamilyCount=1) generation 1 step 3
2025-10-09T21:43:48.131904Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:43:48.131909Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000029s
2025-10-09T21:43:48.132154Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 3 Session: "shared/user_1_3_11984607147350421102_v1" ClientId: "user" PipeClient { RawX1: 7559340989957608036 RawX2: 4503603922341596 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:43:48.132169Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:43:48.137503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340989957608032:3803] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:48.142581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340989957608032:3803] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:48.142596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340989957608032:3803] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:48.142600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340989957608032:3803] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:48.142610Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 202 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:48.204748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:48.210365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:48.210661Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:48.210665Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:48.210678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:48.210741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:48.210782Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340989595798466:3533]
2025-10-09T21:43:48.211429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:48.214975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:48.215067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:48.215622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:48.215695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:48.215697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:48.215698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:48.215700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:48.215701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:48.215702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:48.215704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.215705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.215709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.215716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:48.215727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:48.215731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:48.215735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:48.215739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:48.215743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:48.215747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.215750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.215755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.215762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:48.215771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:48.215778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:48.215783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:48.215790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:48.215795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:48.215800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:48.215816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:48.215821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:48.215826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:48.215829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:48.215833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:48.215838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.215842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.215847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.215853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.215858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.215864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.215895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:48.235444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:48.236852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:48.236859Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:48.236861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:48.236868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:48.236876Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 201 [2:7559340989595798466:3533]
2025-10-09T21:43:48.236896Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:48.236917Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:48.236920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.236923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.236926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.236946Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:48.236952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:48.236960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:48.237028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.237031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.237033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.237036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:48.237117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:48.237120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.237135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.237146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.237160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:48.237182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340989595798460:3533]
2025-10-09T21:43:48.237185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340989595798460:3533]
2025-10-09T21:43:48.237187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340989595798460:3533]
2025-10-09T21:43:48.237197Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:48.237212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:48.237212Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:48.237214Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:48.237217Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:48.237228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:48.237231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:48.237281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340989957608080:3803], now have 1 active actors on pipe
2025-10-09T21:43:48.237284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340989595798460:3533]
2025-10-09T21:43:48.237293Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:48.237299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340989957608016:2432], now have 1 active actors on pipe
2025-10-09T21:43:48.237302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340989957608043:3810], now have 1 active actors on pipe
2025-10-09T21:43:48.237336Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:48.237853Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 201, pipe: [1:7559340989957608043:3810]
2025-10-09T21:43:48.237904Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 201
2025-10-09T21:43:48.237924Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 201
2025-10-09T21:43:48.242114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:48.242132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:48.242149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_3_11984607147350421102_v1 on pipe: [1:7559340989957608043:3810]
2025-10-09T21:43:48.242171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:48.242172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:48.242196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.242203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.242206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:48.242571Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_3_11984607147350421102_v1
2025-10-09T21:43:48.242581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.242628Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ef3d33cd-30626001-281356e-c7eac4ce_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:48.242636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:48.242645Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:48.242840Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_3_11984607147350421102_v1:1 with generation 201
2025-10-09T21:43:48.258024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:48.258079Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:48.258092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.258100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.258103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:48.258126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.258147Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:48.258160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:43:48.258176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:43:48.258192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:48.258543Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046228236 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:43:48.258576Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:43:48.258655Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 sending to client partition status
2025-10-09T21:43:48.259271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:48.259288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:48.259318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:48.259563Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 202 partition: 0 MaxSeqNo: 25 sessionId: src_id|ef3d33cd-30626001-281356e-c7eac4ce_0
2025-10-09T21:43:48.261992Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27c073c8-917c1335-9092368-c9bf6d67_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ef3d33cd-30626001-281356e-c7eac4ce_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:48.264661Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef3d33cd-30626001-281356e-c7eac4ce_0] Write session: try to update token
2025-10-09T21:43:48.264687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef3d33cd-30626001-281356e-c7eac4ce_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:48.302454Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:48.302474Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:48.302476Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:48.302478Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:48.302491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340989595798460:3533]
2025-10-09T21:43:48.302495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340989595798460:3533]
2025-10-09T21:43:48.302499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340989595798460:3533]
2025-10-09T21:43:48.302549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:48.302556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.302559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:48.302725Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:48.303778Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:48.303785Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:48.303789Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:48.326810Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 202 sessionId: src_id|ef3d33cd-30626001-281356e-c7eac4ce_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:48.329250Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:48.514833Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:48.515175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:48.515186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:48.515714Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:48.515730Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:48.515789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_3_11984607147350421102_v1
2025-10-09T21:43:48.515810Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:48.516288Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef3d33cd-30626001-281356e-c7eac4ce_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:48.516298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef3d33cd-30626001-281356e-c7eac4ce_0] Write session will restart in 0.009579s
2025-10-09T21:43:48.516322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef3d33cd-30626001-281356e-c7eac4ce_0] Write session: Do CDS request
2025-10-09T21:43:48.516328Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef3d33cd-30626001-281356e-c7eac4ce_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:48.515805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 202 reason: error at write: Disconnected sessionId: src_id|ef3d33cd-30626001-281356e-c7eac4ce_0
2025-10-09T21:43:48.516007Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 202 sessionId: src_id|ef3d33cd-30626001-281356e-c7eac4ce_0 is DEAD
2025-10-09T21:43:48.516155Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:48.516592Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340989957608043:3810]
2025-10-09T21:43:48.516355Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_3_11984607147350421102_v1
2025-10-09T21:43:48.528481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef3d33cd-30626001-281356e-c7eac4ce_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:48.529462Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:48.529478Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 203
2025-10-09T21:43:48.529630Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 203 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:48.529673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 203 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:48.529679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=203 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:48.529683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:48.535006Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 203 sessionId:  describe result for acl check
2025-10-09T21:43:48.535055Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:48.535057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:48.535059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:48.535066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340989957608330:3819] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:48.535645Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340989957608330:3819] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:48.548936Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340989957608330:3819] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:48.555448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340989957608330:3819] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:48.555474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340989957608330:3819] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:48.555478Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340989957608330:3819] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:48.555488Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 203 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:48.586243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:48.587901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:48.588122Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:48.588125Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:48.588137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:48.588187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:48.588227Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340989595798764:3547]
2025-10-09T21:43:48.588828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:48.589387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:48.589435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:48.589780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:48.589858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:48.589859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:48.589861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:48.589862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:48.589863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:48.589865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:48.589866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.589867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.589869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.589875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:48.589886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:48.589891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:48.589895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:48.589898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:48.589902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:48.589907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.589911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.589915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.589923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:48.589931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:48.589938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:48.589944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:48.589949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:48.589955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:48.589960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:48.589965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:48.589970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:48.589975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:48.589980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:48.589986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:48.589991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.589997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.590003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.590008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.590014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.590019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.590068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:48.599511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:48.601282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:48.601299Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:48.601302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:48.601308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:48.601316Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 202 [2:7559340989595798764:3547]
2025-10-09T21:43:48.601331Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:48.601355Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:48.601359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.601363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.601365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.601387Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:48.601396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:48.601404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:48.601479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.601482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.601485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.601488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:48.601578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:48.601582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.601598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.601611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.601630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:48.601651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340989595798758:3547]
2025-10-09T21:43:48.601654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340989595798758:3547]
2025-10-09T21:43:48.601656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340989595798758:3547]
2025-10-09T21:43:48.601670Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:48.601686Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:48.601704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:48.601707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:48.601753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340989595798758:3547]
2025-10-09T21:43:48.601756Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:48.602997Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 202
2025-10-09T21:43:48.603048Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 202
2025-10-09T21:43:48.601990Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:48.601999Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:48.602002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:48.602008Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:48.607665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340989957608395:3819], now have 1 active actors on pipe
2025-10-09T21:43:48.607683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340989957608320:2432], now have 1 active actors on pipe
2025-10-09T21:43:48.607711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:48.607719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:48.607763Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f629f29b-b0fc6c0f-3af6932d-69d9b269_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:48.607777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.607783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.607787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:48.607802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.607823Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:48.607847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:48.608264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:48.608272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:48.608287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:48.608439Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 203 partition: 0 MaxSeqNo: 25 sessionId: src_id|f629f29b-b0fc6c0f-3af6932d-69d9b269_0
2025-10-09T21:43:48.609148Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef3d33cd-30626001-281356e-c7eac4ce_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f629f29b-b0fc6c0f-3af6932d-69d9b269_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:48.611923Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f629f29b-b0fc6c0f-3af6932d-69d9b269_0] Write session: try to update token
2025-10-09T21:43:48.611951Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f629f29b-b0fc6c0f-3af6932d-69d9b269_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:48.617969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340989957608583:3810], now have 1 active actors on pipe
2025-10-09T21:43:48.617555Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340989957608583:3810]
2025-10-09T21:43:48.618162Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 202, pipe: [1:7559340989957608583:3810]
2025-10-09T21:43:48.630824Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:48.630848Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:48.630852Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:48.630854Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:48.630864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340989595798758:3547]
2025-10-09T21:43:48.630869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340989595798758:3547]
2025-10-09T21:43:48.630873Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340989595798758:3547]
2025-10-09T21:43:48.630923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:48.630931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.630933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:48.631122Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:48.634827Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:48.634841Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:48.634845Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:48.657793Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 203 sessionId: src_id|f629f29b-b0fc6c0f-3af6932d-69d9b269_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:48.661305Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:48.903156Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:48.903808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:48.903820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:48.904317Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:48.904794Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:48.904882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 203 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f629f29b-b0fc6c0f-3af6932d-69d9b269_0
2025-10-09T21:43:48.905079Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 203 sessionId: src_id|f629f29b-b0fc6c0f-3af6932d-69d9b269_0 is DEAD
2025-10-09T21:43:48.905212Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:48.905247Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340989957608583:3810]
2025-10-09T21:43:48.905680Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f629f29b-b0fc6c0f-3af6932d-69d9b269_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:48.905693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f629f29b-b0fc6c0f-3af6932d-69d9b269_0] Write session will restart in 0.006618s
2025-10-09T21:43:48.905718Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f629f29b-b0fc6c0f-3af6932d-69d9b269_0] Write session: Do CDS request
2025-10-09T21:43:48.905724Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f629f29b-b0fc6c0f-3af6932d-69d9b269_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:48.915119Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f629f29b-b0fc6c0f-3af6932d-69d9b269_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:48.915520Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:48.915536Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 204
2025-10-09T21:43:48.920312Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 204 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:48.920373Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 204 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:48.920380Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=204 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:48.920386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:48.921530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 204 sessionId:  describe result for acl check
2025-10-09T21:43:48.921574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:48.921581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:48.921582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:48.921590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340989957608640:3824] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:48.922236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340989957608640:3824] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:48.933264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340989957608640:3824] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:48.951778Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340989957608640:3824] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:48.951798Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340989957608640:3824] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:48.951802Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340989957608640:3824] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:48.951813Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 204 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:48.964641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:48.966714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:48.966936Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:48.966941Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:48.966953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:48.967015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:48.967056Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340989595799025:3550]
2025-10-09T21:43:48.967664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:48.968225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:48.968275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:48.968622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:48.968691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:48.968693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:48.968694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:48.968695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:48.968697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:48.968698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:48.968699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.968701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.968702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.968708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:48.968718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:48.968722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:48.968726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:48.968729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:48.968733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:48.968736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.968739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.968743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.968752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:48.968760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:48.968765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:48.968770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:48.968774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:48.968778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:48.968782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:48.968788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:48.968792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:48.968797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:48.968801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:48.968806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:48.968812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.968817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.968821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.968826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.968831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.968859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.968888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:48.977924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:48.979157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:48.979164Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:48.979167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:48.979171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:48.979180Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 203 [2:7559340989595799025:3550]
2025-10-09T21:43:48.979199Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:48.979221Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:48.979225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.979228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.979231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.979249Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:48.979256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:48.979264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:48.979336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.979338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.979341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.979343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:48.979421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:48.979424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:48.979439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:48.979450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:48.979465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:48.979487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340989595799019:3550]
2025-10-09T21:43:48.979491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340989595799019:3550]
2025-10-09T21:43:48.979493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340989595799019:3550]
2025-10-09T21:43:48.979510Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:48.979526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:48.979539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:48.979542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:48.979583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340989595799019:3550]
2025-10-09T21:43:48.979585Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:48.980560Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:48.980571Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:48.980573Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:48.980580Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:48.981445Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 203
2025-10-09T21:43:48.981477Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 203
2025-10-09T21:43:48.982230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340989957608789:3824], now have 1 active actors on pipe
2025-10-09T21:43:48.982243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340989957608597:2432], now have 1 active actors on pipe
2025-10-09T21:43:48.982260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:48.982266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:48.982304Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|146541dd-47edf042-607d6be0-bad9b10e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:48.982317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.982323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.982326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:48.982340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.982357Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:48.982382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:48.982818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:48.982829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:48.982858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:48.983047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 204 partition: 0 MaxSeqNo: 25 sessionId: src_id|146541dd-47edf042-607d6be0-bad9b10e_0
2025-10-09T21:43:48.986305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f629f29b-b0fc6c0f-3af6932d-69d9b269_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|146541dd-47edf042-607d6be0-bad9b10e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:48.989010Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|146541dd-47edf042-607d6be0-bad9b10e_0] Write session: try to update token
2025-10-09T21:43:48.989033Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|146541dd-47edf042-607d6be0-bad9b10e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:49.006894Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:49.006914Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:49.006917Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:49.006919Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:49.006931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340989595799019:3550]
2025-10-09T21:43:49.006935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340989595799019:3550]
2025-10-09T21:43:49.006939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340989595799019:3550]
2025-10-09T21:43:49.006987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:49.006996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:49.006999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:49.007143Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:49.010104Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340994252576156:3810]
2025-10-09T21:43:49.011075Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:49.011098Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:49.011102Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:49.012102Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 203, pipe: [1:7559340994252576156:3810]
2025-10-09T21:43:49.058181Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 204 sessionId: src_id|146541dd-47edf042-607d6be0-bad9b10e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:49.060704Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:49.261396Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:49.262344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340994252576156:3810], now have 1 active actors on pipe
2025-10-09T21:43:49.262412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:49.262435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:49.263302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:49.263322Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:49.263511Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:49.263386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 204 reason: error at write: Disconnected sessionId: src_id|146541dd-47edf042-607d6be0-bad9b10e_0
2025-10-09T21:43:49.263594Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 204 sessionId: src_id|146541dd-47edf042-607d6be0-bad9b10e_0 is DEAD
2025-10-09T21:43:49.263754Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:49.263793Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340994252576156:3810]
2025-10-09T21:43:49.264920Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|146541dd-47edf042-607d6be0-bad9b10e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:49.264934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|146541dd-47edf042-607d6be0-bad9b10e_0] Write session will restart in 0.006381s
2025-10-09T21:43:49.264966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|146541dd-47edf042-607d6be0-bad9b10e_0] Write session: Do CDS request
2025-10-09T21:43:49.264973Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|146541dd-47edf042-607d6be0-bad9b10e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:49.274635Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|146541dd-47edf042-607d6be0-bad9b10e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:49.274885Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:49.274905Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 205
2025-10-09T21:43:49.275115Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 205 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:49.275152Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 205 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:49.275158Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=205 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:49.275162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:49.275708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 205 sessionId:  describe result for acl check
2025-10-09T21:43:49.275750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:49.275754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:49.275756Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:49.275763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340994252576177:3828] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:49.276357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340994252576177:3828] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:49.284907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340994252576177:3828] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:49.289856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340994252576177:3828] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:49.289877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340994252576177:3828] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:49.289882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340994252576177:3828] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:49.289899Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 205 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:49.320375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:49.320681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:49.320861Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:49.320865Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:49.320879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:49.320934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:49.320977Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340993890766580:3553]
2025-10-09T21:43:49.321562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:49.322236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:49.322290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:49.322642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:49.322707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:49.322708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:49.322710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:49.322711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:49.322712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:49.322714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:49.322715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:49.322716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:49.322717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:49.322723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:49.322734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:49.322738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:49.322741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:49.322759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:49.322763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:49.322767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:49.322771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:49.322774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:49.322781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:49.322789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:49.322795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:49.322801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:49.322805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:49.322810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:49.322815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:49.322820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:49.322825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:49.322830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:49.322833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:49.322839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:49.322844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:49.322848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:49.322854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:49.322859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:49.322864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:49.322869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:49.322898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:49.334621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:49.335896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:49.335905Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:49.335908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:49.335913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:49.335922Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 204 [2:7559340993890766580:3553]
2025-10-09T21:43:49.335946Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:49.335971Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:49.335976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.335979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.335982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.336003Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:49.336010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:49.336018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:49.336091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:49.336093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:49.336095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:49.336098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:49.336177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:49.336180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:49.336195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:49.336208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:49.336226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:49.336247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340993890766574:3553]
2025-10-09T21:43:49.336250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340993890766574:3553]
2025-10-09T21:43:49.336252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340993890766574:3553]
2025-10-09T21:43:49.336266Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:49.336281Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:49.336298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:49.336301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:49.336356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340993890766574:3553]
2025-10-09T21:43:49.336359Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:49.336693Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:49.336702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:49.336704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:49.336711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:49.337707Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 204
2025-10-09T21:43:49.337759Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 204
2025-10-09T21:43:49.347003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340994252576163:2432], now have 1 active actors on pipe
2025-10-09T21:43:49.347023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340994252576235:3828], now have 1 active actors on pipe
2025-10-09T21:43:49.347062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:49.347070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:49.347112Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9a8316dc-ea85184c-1ed2b229-e0a9860c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:49.347124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.347130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.347133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:49.347149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.347170Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:49.347191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:49.349767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:49.349789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:49.349824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:49.350453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|146541dd-47edf042-607d6be0-bad9b10e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9a8316dc-ea85184c-1ed2b229-e0a9860c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:49.353056Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9a8316dc-ea85184c-1ed2b229-e0a9860c_0] Write session: try to update token
2025-10-09T21:43:49.353076Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9a8316dc-ea85184c-1ed2b229-e0a9860c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:49.349984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 205 partition: 0 MaxSeqNo: 25 sessionId: src_id|9a8316dc-ea85184c-1ed2b229-e0a9860c_0
2025-10-09T21:43:49.366717Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340994252576440:3810]
2025-10-09T21:43:49.367240Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 204, pipe: [1:7559340994252576440:3810]
2025-10-09T21:43:49.370134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340994252576440:3810], now have 1 active actors on pipe
2025-10-09T21:43:49.388271Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:49.388290Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:49.388293Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:49.388296Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:49.388306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340993890766574:3553]
2025-10-09T21:43:49.388310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340993890766574:3553]
2025-10-09T21:43:49.388314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340993890766574:3553]
2025-10-09T21:43:49.388359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:49.388366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:49.388369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:49.388505Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:49.389194Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:49.389200Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:49.389204Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:49.414975Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 205 sessionId: src_id|9a8316dc-ea85184c-1ed2b229-e0a9860c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:49.417541Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:49.641914Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:49.642575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:49.642595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:49.643393Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:49.643232Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:49.643280Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:49.643283Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:49.643342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 205 reason: error at write: Disconnected sessionId: src_id|9a8316dc-ea85184c-1ed2b229-e0a9860c_0
2025-10-09T21:43:49.643533Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 205 sessionId: src_id|9a8316dc-ea85184c-1ed2b229-e0a9860c_0 is DEAD
2025-10-09T21:43:49.643700Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340994252576440:3810]
2025-10-09T21:43:49.644514Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a8316dc-ea85184c-1ed2b229-e0a9860c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:49.644526Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a8316dc-ea85184c-1ed2b229-e0a9860c_0] Write session will restart in 0.009405s
2025-10-09T21:43:49.644558Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a8316dc-ea85184c-1ed2b229-e0a9860c_0] Write session: Do CDS request
2025-10-09T21:43:49.644568Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a8316dc-ea85184c-1ed2b229-e0a9860c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:49.655689Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9a8316dc-ea85184c-1ed2b229-e0a9860c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:49.655817Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:49.655837Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 206
2025-10-09T21:43:49.655990Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 206 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:49.656030Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 206 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:49.656041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=206 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:49.656045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:49.656665Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 206 sessionId:  describe result for acl check
2025-10-09T21:43:49.656706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:49.656708Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:49.656710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:49.656716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340994252576474:3840] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:49.657286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340994252576474:3840] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:49.667254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340994252576474:3840] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:49.691598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340994252576474:3840] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:49.691618Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340994252576474:3840] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:49.691622Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340994252576474:3840] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:49.691632Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 206 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:49.724003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:49.731909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:49.732187Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:49.732192Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:49.732204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:49.732266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:49.732306Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340993890766879:3567]
2025-10-09T21:43:49.732943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:49.733525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:49.733574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:49.733912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:49.733960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:49.733961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:49.733963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:49.733964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:49.733965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:49.733967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:49.733968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:49.733970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:49.733971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:49.733977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:49.733989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:49.733993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:49.733997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:49.734001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:49.734042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:49.734046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:49.734049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:49.734053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:49.734061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:49.734069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:49.734075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:49.734080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:49.734084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:49.734089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:49.734094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:49.734099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:49.734103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:49.734107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:49.734111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:49.734116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:49.734121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:49.734126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:49.734130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:49.734135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:49.734139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:49.734143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:49.734171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:49.744123Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340994252576721:3810]
2025-10-09T21:43:49.753646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:49.754878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:49.754886Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:49.754889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:49.754894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:49.754901Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 205 [2:7559340993890766879:3567]
2025-10-09T21:43:49.754924Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:49.754944Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:49.754947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.754951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.754953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.754972Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:49.754977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:49.754986Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:49.755054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:49.755056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:49.755059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:49.755062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:49.755142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:49.755146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:49.755159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:49.755170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:49.755186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:49.755209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340993890766873:3567]
2025-10-09T21:43:49.755211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340993890766873:3567]
2025-10-09T21:43:49.755214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340993890766873:3567]
2025-10-09T21:43:49.755223Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:49.755238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:49.755254Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:49.755258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:49.755302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340993890766873:3567]
2025-10-09T21:43:49.755305Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:49.755570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:49.755579Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:49.755580Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:49.755587Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:49.757203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340994252576595:3840], now have 1 active actors on pipe
2025-10-09T21:43:49.757212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340994252576460:2432], now have 1 active actors on pipe
2025-10-09T21:43:49.757216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340994252576721:3810], now have 1 active actors on pipe
2025-10-09T21:43:49.757792Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 205
2025-10-09T21:43:49.757830Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 205, pipe: [1:7559340994252576721:3810]
2025-10-09T21:43:49.757856Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 205
2025-10-09T21:43:49.759640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:49.759658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:49.759693Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|35e1ed43-33d45e32-8e1ff04a-2f174404_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:49.759707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.759712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.759715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:49.759728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.759745Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:49.759764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:49.760686Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a8316dc-ea85184c-1ed2b229-e0a9860c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|35e1ed43-33d45e32-8e1ff04a-2f174404_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:49.763241Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|35e1ed43-33d45e32-8e1ff04a-2f174404_0] Write session: try to update token
2025-10-09T21:43:49.763260Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|35e1ed43-33d45e32-8e1ff04a-2f174404_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:49.760100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:49.760109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:49.760128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:49.760256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 206 partition: 0 MaxSeqNo: 25 sessionId: src_id|35e1ed43-33d45e32-8e1ff04a-2f174404_0
2025-10-09T21:43:49.787953Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 206 sessionId: src_id|35e1ed43-33d45e32-8e1ff04a-2f174404_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:49.790526Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:49.792333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:49.792355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:49.792396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:49.792549Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:49.797710Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:49.797729Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:49.797731Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:49.797733Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:49.797743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340993890766873:3567]
2025-10-09T21:43:49.797747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340993890766873:3567]
2025-10-09T21:43:49.797752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340993890766873:3567]
2025-10-09T21:43:49.797793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:49.797800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:49.797802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:49.797951Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:49.802139Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:49.802157Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:49.802161Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:50.246324Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:50.263893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:50.263923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:50.264099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:50.264157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:50.264253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:50.264312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:50.264351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:50.264356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:50.264423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:50.264480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:50.264544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:50.264605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:50.264644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:50.264648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:50.264714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:50.264770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:50.264829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:50.264886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:50.264928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:50.264931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:50.265000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:50.265062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:50.265162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:50.265222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:50.265260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:50.265263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:50.265331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:50.265413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:50.265480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:50.265542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:50.265580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:50.265584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:50.265644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:50.265695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:50.265750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:50.265804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:50.265936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:50.265940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:50.266000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:50.270926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:50.271039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:50.271123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:50.271203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:50.271209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:50.271283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:50.271352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:50.271409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:50.271483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:50.271611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:50.271616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:50.271688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:50.271770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:50.271833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:50.271896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:50.271939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:50.271942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:50.272007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:50.272085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:50.272145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:50.272204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:50.272245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:50.272248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:50.275027Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:50.275085Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340994252576721:3810]
2025-10-09T21:43:50.275799Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35e1ed43-33d45e32-8e1ff04a-2f174404_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:50.275811Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35e1ed43-33d45e32-8e1ff04a-2f174404_0] Write session will restart in 0.008413s
2025-10-09T21:43:50.275836Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35e1ed43-33d45e32-8e1ff04a-2f174404_0] Write session: Do CDS request
2025-10-09T21:43:50.275844Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35e1ed43-33d45e32-8e1ff04a-2f174404_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:50.275112Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:50.275120Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:50.275149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 206 reason: error at write: Disconnected sessionId: src_id|35e1ed43-33d45e32-8e1ff04a-2f174404_0
2025-10-09T21:43:50.275342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 206 sessionId: src_id|35e1ed43-33d45e32-8e1ff04a-2f174404_0 is DEAD
2025-10-09T21:43:50.280719Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:50.288869Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:50.288887Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 207
2025-10-09T21:43:50.290268Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|35e1ed43-33d45e32-8e1ff04a-2f174404_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:50.290652Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 207 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:50.290698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 207 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:50.290704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=207 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:50.290708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:50.291316Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 207 sessionId:  describe result for acl check
2025-10-09T21:43:50.291355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:50.291357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:50.291359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:50.291367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340998547544081:3845] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:50.291970Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340998547544081:3845] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:50.298346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340998547544081:3845] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:50.303500Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340998547544081:3845] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:50.303518Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340998547544081:3845] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:50.303522Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340998547544081:3845] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:50.303533Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 207 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:50.313734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:50.315698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:50.315938Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:50.315951Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:50.315964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:50.316022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:50.316064Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340998185734441:3571]
2025-10-09T21:43:50.316650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:50.317270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:50.317322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:50.317648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:50.317701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:50.317703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:50.317704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:50.317705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:50.317707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:50.317708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:50.317709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.317711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.317712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.317717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:50.317729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:50.317732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:50.317736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:50.317739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:50.317743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:50.317747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.317751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.317755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.317765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:50.317774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:50.317780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:50.317785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:50.317791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:50.317796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:50.317802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:50.317808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:50.317812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:50.317817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:50.317822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:50.317827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:50.317833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.317838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.317843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.317861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.317867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.317873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.317901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:50.327045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:50.328485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:50.328501Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:50.328504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:50.328509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:50.328517Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 206 [2:7559340998185734441:3571]
2025-10-09T21:43:50.328534Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:50.328556Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:50.328559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.328563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.328565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.328584Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:50.328591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:50.328598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:50.328665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.328667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.328669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.328672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:50.328751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:50.328754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.328769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.328780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.328798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:50.328819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340998185734435:3571]
2025-10-09T21:43:50.328823Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340998185734435:3571]
2025-10-09T21:43:50.328825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340998185734435:3571]
2025-10-09T21:43:50.328840Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:50.328856Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:50.328871Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:50.328875Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:50.328920Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340998185734435:3571]
2025-10-09T21:43:50.328922Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:50.329925Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 206
2025-10-09T21:43:50.329975Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 206
2025-10-09T21:43:50.329130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:50.329136Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:50.329139Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:50.329144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:50.330475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340998547544212:3845], now have 1 active actors on pipe
2025-10-09T21:43:50.330495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340998547544043:2432], now have 1 active actors on pipe
2025-10-09T21:43:50.330512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:50.330520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:50.330557Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5c5085ab-da78f577-7c9e75f1-a9ae4df9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:50.330567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.330572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.330575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:50.330588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.330605Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:50.330626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:50.330946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:50.330955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:50.330973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:50.331118Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 207 partition: 0 MaxSeqNo: 25 sessionId: src_id|5c5085ab-da78f577-7c9e75f1-a9ae4df9_0
2025-10-09T21:43:50.331502Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35e1ed43-33d45e32-8e1ff04a-2f174404_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5c5085ab-da78f577-7c9e75f1-a9ae4df9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:50.334219Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c5085ab-da78f577-7c9e75f1-a9ae4df9_0] Write session: try to update token
2025-10-09T21:43:50.334241Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c5085ab-da78f577-7c9e75f1-a9ae4df9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:50.361886Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:50.361907Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:50.361910Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:50.361913Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:50.361925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340998185734435:3571]
2025-10-09T21:43:50.361930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340998185734435:3571]
2025-10-09T21:43:50.361935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340998185734435:3571]
2025-10-09T21:43:50.361985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:50.361993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.361996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:50.362163Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:50.372207Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:50.372223Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:50.372227Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:50.375800Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340998547544311:3810]
2025-10-09T21:43:50.377383Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 206, pipe: [1:7559340998547544311:3810]
2025-10-09T21:43:50.383388Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 207 sessionId: src_id|5c5085ab-da78f577-7c9e75f1-a9ae4df9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:50.385932Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:50.573039Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:50.573953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340998547544311:3810], now have 1 active actors on pipe
2025-10-09T21:43:50.573974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:50.573982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:50.575197Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340998547544311:3810]
2025-10-09T21:43:50.575236Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:50.575238Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:50.575298Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 207 reason: error at write: Disconnected sessionId: src_id|5c5085ab-da78f577-7c9e75f1-a9ae4df9_0
2025-10-09T21:43:50.575489Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 207 sessionId: src_id|5c5085ab-da78f577-7c9e75f1-a9ae4df9_0 is DEAD
2025-10-09T21:43:50.575927Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c5085ab-da78f577-7c9e75f1-a9ae4df9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:50.575940Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c5085ab-da78f577-7c9e75f1-a9ae4df9_0] Write session will restart in 0.009987s
2025-10-09T21:43:50.575964Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c5085ab-da78f577-7c9e75f1-a9ae4df9_0] Write session: Do CDS request
2025-10-09T21:43:50.575971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c5085ab-da78f577-7c9e75f1-a9ae4df9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:50.575671Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:50.578405Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:50.587795Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c5085ab-da78f577-7c9e75f1-a9ae4df9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:50.594874Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:50.594890Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 208
2025-10-09T21:43:50.596185Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 208 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:50.596234Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 208 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:50.596244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=208 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:50.596248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:50.600165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 208 sessionId:  describe result for acl check
2025-10-09T21:43:50.600212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:50.600217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:50.600219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:50.600226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340998547544368:3857] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:50.600816Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340998547544368:3857] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:50.620347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340998547544368:3857] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:50.639907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340998547544368:3857] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:50.639931Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340998547544368:3857] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:50.639935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340998547544368:3857] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:50.639946Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 208 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:50.652448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:50.653717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:50.653960Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:50.653965Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:50.653980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:50.654086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:50.654132Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340998185734748:3581]
2025-10-09T21:43:50.654774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:50.655308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:50.655362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:50.655698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:50.655750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:50.655751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:50.655753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:50.655754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:50.655755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:50.655756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:50.655758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.655758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.655760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.655766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:50.655778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:50.655782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:50.655786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:50.655790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:50.655794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:50.655798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.655802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.655806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.655814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:50.655823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:50.655829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:50.655835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:50.655839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:50.655844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:50.655850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:50.655855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:50.655860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:50.655865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:50.655870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:50.655877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:50.655882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.655887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.655893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.655897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.655921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.655927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.655959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:50.665063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:50.666830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:50.666843Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:50.666846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:50.666851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:50.666859Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 207 [2:7559340998185734748:3581]
2025-10-09T21:43:50.666877Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:50.666900Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:50.666908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.666912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.666915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.666933Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:50.666940Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:50.666949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:50.667007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.667015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.667017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.667020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:50.667097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:50.667105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.667119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.667142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.667158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:50.667178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340998185734742:3581]
2025-10-09T21:43:50.667181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340998185734742:3581]
2025-10-09T21:43:50.667183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340998185734742:3581]
2025-10-09T21:43:50.667192Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:50.667201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:50.667203Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:50.667206Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:50.667219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340998547544540:3857], now have 1 active actors on pipe
2025-10-09T21:43:50.667221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340998547544343:2432], now have 1 active actors on pipe
2025-10-09T21:43:50.667230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:50.667245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:50.667249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:50.667292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340998185734742:3581]
2025-10-09T21:43:50.667299Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:50.668085Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 207
2025-10-09T21:43:50.668116Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 207
2025-10-09T21:43:50.667343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:50.668713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:50.668728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:50.668763Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a3cda7c1-b7f9b2de-2e900711-7cd9987f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:50.668774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.668779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.668782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:50.668794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.668809Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:50.668832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:50.669116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:50.669122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:50.669137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:50.669271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 208 partition: 0 MaxSeqNo: 25 sessionId: src_id|a3cda7c1-b7f9b2de-2e900711-7cd9987f_0
2025-10-09T21:43:50.669706Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c5085ab-da78f577-7c9e75f1-a9ae4df9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a3cda7c1-b7f9b2de-2e900711-7cd9987f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:50.672320Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a3cda7c1-b7f9b2de-2e900711-7cd9987f_0] Write session: try to update token
2025-10-09T21:43:50.672346Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a3cda7c1-b7f9b2de-2e900711-7cd9987f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:50.683272Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559340998547544608:3810]
2025-10-09T21:43:50.684826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340998547544608:3810], now have 1 active actors on pipe
2025-10-09T21:43:50.684959Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 207, pipe: [1:7559340998547544608:3810]
2025-10-09T21:43:50.692093Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:50.692111Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:50.692114Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:50.692116Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:50.692126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340998185734742:3581]
2025-10-09T21:43:50.692132Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340998185734742:3581]
2025-10-09T21:43:50.692136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340998185734742:3581]
2025-10-09T21:43:50.692184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:50.692191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.692194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:50.692335Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:50.697813Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:50.697829Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:50.697833Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:50.709566Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 208 sessionId: src_id|a3cda7c1-b7f9b2de-2e900711-7cd9987f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:50.712148Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:50.871540Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:43:50.871570Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 2
2025-10-09T21:43:50.918618Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:50.920385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:50.920405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:50.921080Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:50.921484Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559340998547544608:3810]
2025-10-09T21:43:50.921519Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:50.921522Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:50.921576Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 208 reason: error at write: Disconnected sessionId: src_id|a3cda7c1-b7f9b2de-2e900711-7cd9987f_0
2025-10-09T21:43:50.921760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 208 sessionId: src_id|a3cda7c1-b7f9b2de-2e900711-7cd9987f_0 is DEAD
2025-10-09T21:43:50.921922Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:50.921933Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 2
2025-10-09T21:43:50.922354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3cda7c1-b7f9b2de-2e900711-7cd9987f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:50.922367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3cda7c1-b7f9b2de-2e900711-7cd9987f_0] Write session will restart in 0.005145s
2025-10-09T21:43:50.922407Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3cda7c1-b7f9b2de-2e900711-7cd9987f_0] Write session: Do CDS request
2025-10-09T21:43:50.922414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3cda7c1-b7f9b2de-2e900711-7cd9987f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:50.928641Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a3cda7c1-b7f9b2de-2e900711-7cd9987f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:50.929327Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:50.929345Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 209
2025-10-09T21:43:50.929661Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 209 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:50.929698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 209 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:50.929709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=209 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:50.929713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:50.930208Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 209 sessionId:  describe result for acl check
2025-10-09T21:43:50.930253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:50.930259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:50.930264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:50.930270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340998547544639:3862] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:50.930868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340998547544639:3862] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:50.942605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340998547544639:3862] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:50.959131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340998547544639:3862] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:50.959151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340998547544639:3862] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:50.959156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340998547544639:3862] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:50.959166Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 209 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:50.971280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:50.973862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:50.974145Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:50.974158Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:50.974173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:50.974263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:50.974311Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340998185735018:3589]
2025-10-09T21:43:50.974903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:50.985585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:50.985696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:50.986119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:50.986197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:50.986198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:50.986200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:50.986201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:50.986203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:50.986204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:50.986205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.986206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.986208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.986217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:50.986229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:50.986232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:50.986236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:50.986240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:50.986244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:50.986247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.986251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.986254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.986264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:50.986273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:50.986280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:50.986285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:50.986290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:50.986296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:50.986300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:50.986306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:50.986319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:50.986324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:50.986329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:50.986335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:50.986340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.986345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.986349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.986354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.986359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.986364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.986396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:50.996579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:50.997740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:50.997747Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:50.997750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:50.997756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:50.997764Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 208 [2:7559340998185735018:3589]
2025-10-09T21:43:50.998167Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:50.998189Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:50.998193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.998196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.998198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.998219Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:50.998225Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:50.998233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:50.998297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.998300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.998302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.998305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:50.998383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:50.998388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:50.998402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:50.998428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:50.998447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:50.998468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559340998185735012:3589]
2025-10-09T21:43:50.998471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559340998185735012:3589]
2025-10-09T21:43:50.998474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559340998185735012:3589]
2025-10-09T21:43:50.998489Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:50.998505Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:50.998521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:50.998524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:50.998566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559340998185735012:3589]
2025-10-09T21:43:50.998568Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:50.999491Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 208
2025-10-09T21:43:50.999541Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 208
2025-10-09T21:43:50.999290Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:50.999300Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:50.999302Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:50.999309Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:51.000657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340998547544807:3862], now have 1 active actors on pipe
2025-10-09T21:43:51.000670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340998547544627:2432], now have 1 active actors on pipe
2025-10-09T21:43:51.000690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:51.000696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:51.000755Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6177bb25-8bcc69f5-3dc5d6cb-5fb1b384_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:51.000766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.000772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.000775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:51.000790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.000805Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:51.000866Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:43:51.000904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:51.005581Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 2 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:43:51.005633Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:43:51.005928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:51.005947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:51.005983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:51.009379Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3cda7c1-b7f9b2de-2e900711-7cd9987f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6177bb25-8bcc69f5-3dc5d6cb-5fb1b384_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:51.008924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 209 partition: 0 MaxSeqNo: 25 sessionId: src_id|6177bb25-8bcc69f5-3dc5d6cb-5fb1b384_0
2025-10-09T21:43:51.012683Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6177bb25-8bcc69f5-3dc5d6cb-5fb1b384_0] Write session: try to update token
2025-10-09T21:43:51.012705Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6177bb25-8bcc69f5-3dc5d6cb-5fb1b384_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:51.022468Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341002842512192:3810]
2025-10-09T21:43:51.023620Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:51.023641Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:51.023643Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:51.023645Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:51.023655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559340998185735012:3589]
2025-10-09T21:43:51.023659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559340998185735012:3589]
2025-10-09T21:43:51.023664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559340998185735012:3589]
2025-10-09T21:43:51.023711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:51.023718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:51.023721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:51.023873Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:51.027796Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:51.027815Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:51.027819Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:51.028223Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 208, pipe: [1:7559341002842512192:3810]
2025-10-09T21:43:51.060531Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 209 sessionId: src_id|6177bb25-8bcc69f5-3dc5d6cb-5fb1b384_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:51.063185Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:51.239825Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:51.240126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341002842512192:3810], now have 1 active actors on pipe
2025-10-09T21:43:51.240183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:51.240190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:51.241635Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:51.242867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:51.242957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 209 reason: status is not ok: tablet will be restarted right now sessionId: src_id|6177bb25-8bcc69f5-3dc5d6cb-5fb1b384_0
2025-10-09T21:43:51.243154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 209 sessionId: src_id|6177bb25-8bcc69f5-3dc5d6cb-5fb1b384_0 is DEAD
2025-10-09T21:43:51.243339Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341002842512192:3810]
2025-10-09T21:43:51.243355Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:51.243698Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6177bb25-8bcc69f5-3dc5d6cb-5fb1b384_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:51.243711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6177bb25-8bcc69f5-3dc5d6cb-5fb1b384_0] Write session will restart in 0.007459s
2025-10-09T21:43:51.243747Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6177bb25-8bcc69f5-3dc5d6cb-5fb1b384_0] Write session: Do CDS request
2025-10-09T21:43:51.243754Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6177bb25-8bcc69f5-3dc5d6cb-5fb1b384_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:51.253157Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6177bb25-8bcc69f5-3dc5d6cb-5fb1b384_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:51.253382Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:51.253401Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 210
2025-10-09T21:43:51.253838Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 210 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:51.253885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 210 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:51.253891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=210 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:51.253895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:51.255560Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 210 sessionId:  describe result for acl check
2025-10-09T21:43:51.255608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:51.255610Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:51.255613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:51.255619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341002842512215:3866] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:51.256249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341002842512215:3866] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:51.265624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341002842512215:3866] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:51.287370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341002842512215:3866] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:51.287390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341002842512215:3866] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:51.287395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341002842512215:3866] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:51.287405Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 210 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:51.328123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:51.330100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:51.330323Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:51.330327Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:51.330340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:51.330393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:51.330452Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341002480702581:3593]
2025-10-09T21:43:51.331098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:51.331638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:51.331689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:51.332035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:51.332087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:51.332088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:51.332090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:51.332091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:51.332093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:51.332094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:51.332095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:51.332096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:51.332098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:51.332104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:51.332116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:51.332121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:51.332125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:51.332129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:51.332134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:51.332138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:51.332142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:51.332146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:51.332154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:51.332162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:51.332169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:51.332175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:51.332180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:51.332185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:51.332190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:51.332195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:51.332200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:51.332205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:51.332210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:51.332214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:51.332220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:51.332226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:51.332229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:51.332233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:51.332238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:51.332243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:51.332278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:51.340723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:51.342087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:51.342094Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:51.342097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:51.342103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:51.342111Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 209 [2:7559341002480702581:3593]
2025-10-09T21:43:51.342142Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:51.342166Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:51.342170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.342174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.342176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.342195Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:51.342202Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:51.342213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:51.342281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:51.342283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:51.342286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:51.342290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:51.342378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:51.342381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:51.342396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:51.342408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:51.342442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:51.342464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341002480702575:3593]
2025-10-09T21:43:51.342467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341002480702575:3593]
2025-10-09T21:43:51.342469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341002480702575:3593]
2025-10-09T21:43:51.342485Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:51.342500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:51.342517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:51.342520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:51.342563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341002480702575:3593]
2025-10-09T21:43:51.342566Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:51.342901Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:51.342909Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:51.342912Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:51.342918Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:51.343721Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 209
2025-10-09T21:43:51.343755Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 209
2025-10-09T21:43:51.345105Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341002842512470:3810]
2025-10-09T21:43:51.349152Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 209, pipe: [1:7559341002842512470:3810]
2025-10-09T21:43:51.350280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341002842512328:3866], now have 1 active actors on pipe
2025-10-09T21:43:51.350298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341002842512201:2432], now have 1 active actors on pipe
2025-10-09T21:43:51.350325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:51.350333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:51.350631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341002842512470:3810], now have 1 active actors on pipe
2025-10-09T21:43:51.350655Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|61ee874b-17c914f1-749295c6-b5f0ef83_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:51.350670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.350676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.350679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:51.350693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.350723Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:51.350748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:51.352696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:51.352715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:51.352743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:51.352916Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 210 partition: 0 MaxSeqNo: 25 sessionId: src_id|61ee874b-17c914f1-749295c6-b5f0ef83_0
2025-10-09T21:43:51.353405Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6177bb25-8bcc69f5-3dc5d6cb-5fb1b384_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|61ee874b-17c914f1-749295c6-b5f0ef83_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:51.356076Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|61ee874b-17c914f1-749295c6-b5f0ef83_0] Write session: try to update token
2025-10-09T21:43:51.356100Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|61ee874b-17c914f1-749295c6-b5f0ef83_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:51.384720Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:51.384739Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:51.384742Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:51.384744Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:51.384754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341002480702575:3593]
2025-10-09T21:43:51.384758Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341002480702575:3593]
2025-10-09T21:43:51.384763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341002480702575:3593]
2025-10-09T21:43:51.384811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:51.384818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:51.384821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:51.384954Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:51.389102Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:51.389116Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:51.389121Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:51.389418Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 210 sessionId: src_id|61ee874b-17c914f1-749295c6-b5f0ef83_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:51.391927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:51.548303Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: CLIENT_DEADLINE_EXCEEDED. Description: [ { <main>: Error: GRpc error: (4): Deadline Exceeded } { <main>: Error: Grpc error response on endpoint localhost:14704 } ]
2025-10-09T21:43:51.655702Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:51.656195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:51.656206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:51.657205Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:51.660095Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61ee874b-17c914f1-749295c6-b5f0ef83_0] Counters: { Errors: 0 CurrentSessionLifetimeMs: 306 BytesWritten: 56320040 MessagesWritten: 25 BytesWrittenCompressed: 0 BytesInflightUncompressed: 22528020 BytesInflightCompressed: 0 BytesInflightTotal: 18022416 MessagesInflight: 10 }
2025-10-09T21:43:51.660131Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61ee874b-17c914f1-749295c6-b5f0ef83_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:51.660136Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61ee874b-17c914f1-749295c6-b5f0ef83_0] Write session will restart in 0.008013s
2025-10-09T21:43:51.660160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61ee874b-17c914f1-749295c6-b5f0ef83_0] Write session: Do CDS request
2025-10-09T21:43:51.660167Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61ee874b-17c914f1-749295c6-b5f0ef83_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:51.659451Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:51.659549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 210 reason: status is not ok: tablet will be restarted right now sessionId: src_id|61ee874b-17c914f1-749295c6-b5f0ef83_0
2025-10-09T21:43:51.659737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 210 sessionId: src_id|61ee874b-17c914f1-749295c6-b5f0ef83_0 is DEAD
2025-10-09T21:43:51.659868Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341002842512470:3810]
2025-10-09T21:43:51.659882Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:51.674578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|61ee874b-17c914f1-749295c6-b5f0ef83_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:51.674910Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:51.674927Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 211
2025-10-09T21:43:51.675066Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 211 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:51.675096Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 211 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:51.675101Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=211 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:51.675105Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:51.675601Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 211 sessionId:  describe result for acl check
2025-10-09T21:43:51.675641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:51.675643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:51.675645Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:51.675651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341002842512519:3885] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:51.676330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341002842512519:3885] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:51.697293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341002842512519:3885] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:51.730606Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341002842512519:3885] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:51.730631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341002842512519:3885] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:51.730638Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341002842512519:3885] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:51.730648Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 211 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:51.740875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:51.745526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:51.745780Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:51.745784Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:51.745795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:51.745849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:51.745891Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341002480702878:3601]
2025-10-09T21:43:51.746593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:51.747129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:51.747175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:51.747523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:51.747576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:51.747577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:51.747579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:51.747580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:51.747581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:51.747582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:51.747584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:51.747585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:51.747587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:51.747593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:51.747605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:51.747610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:51.747615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:51.747619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:51.747624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:51.747628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:51.747633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:51.747638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:51.747646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:51.747654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:51.747661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:51.747667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:51.747672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:51.747677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:51.747681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:51.747687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:51.747692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:51.747697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:51.747701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:51.747707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:51.747715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:51.747721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:51.747726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:51.747732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:51.747737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:51.747743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:51.747774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:51.756170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:51.757240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:51.757247Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:51.757250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:51.757255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:51.757263Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 210 [2:7559341002480702878:3601]
2025-10-09T21:43:51.757278Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:51.757298Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:51.757302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.757305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.757307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.757327Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:51.757332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:51.757341Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:51.757402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:51.757404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:51.757406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:51.757409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:51.757487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:51.757490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:51.757504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:51.757518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:51.757537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:51.757559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341002480702872:3601]
2025-10-09T21:43:51.757561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341002480702872:3601]
2025-10-09T21:43:51.757564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341002480702872:3601]
2025-10-09T21:43:51.757578Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:51.757594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:51.757611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:51.757615Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:51.757657Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341002480702872:3601]
2025-10-09T21:43:51.757660Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:51.757969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:51.757976Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:51.757979Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:51.757985Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:51.758742Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 210
2025-10-09T21:43:51.758781Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 210
2025-10-09T21:43:51.759310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341002842512504:2432], now have 1 active actors on pipe
2025-10-09T21:43:51.759322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341002842512743:3885], now have 1 active actors on pipe
2025-10-09T21:43:51.759343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:51.759349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:51.759379Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|badbb3d-3a175319-990dff1f-b961c89f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:51.759390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.759394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.759397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:51.759410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.759426Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:51.759443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:51.759753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:51.759759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:51.759773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:51.759902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 211 partition: 0 MaxSeqNo: 25 sessionId: src_id|badbb3d-3a175319-990dff1f-b961c89f_0
2025-10-09T21:43:51.760336Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61ee874b-17c914f1-749295c6-b5f0ef83_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|badbb3d-3a175319-990dff1f-b961c89f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:51.763104Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|badbb3d-3a175319-990dff1f-b961c89f_0] Write session: try to update token
2025-10-09T21:43:51.763127Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|badbb3d-3a175319-990dff1f-b961c89f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:51.762786Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341002842512771:3810]
2025-10-09T21:43:51.764511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341002842512771:3810], now have 1 active actors on pipe
2025-10-09T21:43:51.764620Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 210, pipe: [1:7559341002842512771:3810]
2025-10-09T21:43:51.789983Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:51.790001Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:51.790003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:51.790005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:51.790016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341002480702872:3601]
2025-10-09T21:43:51.790019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341002480702872:3601]
2025-10-09T21:43:51.790023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341002480702872:3601]
2025-10-09T21:43:51.790078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:51.790086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:51.790088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:51.790231Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:51.794459Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:51.794474Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:51.794478Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:51.795528Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 211 sessionId: src_id|badbb3d-3a175319-990dff1f-b961c89f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:51.798094Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:52.051916Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:52.053155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:52.053175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:52.053466Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:52.053861Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:52.053914Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:52.053987Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 211 reason: status is not ok: tablet will be restarted right now sessionId: src_id|badbb3d-3a175319-990dff1f-b961c89f_0
2025-10-09T21:43:52.054206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 211 sessionId: src_id|badbb3d-3a175319-990dff1f-b961c89f_0 is DEAD
2025-10-09T21:43:52.054432Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341002842512771:3810]
2025-10-09T21:43:52.058405Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|badbb3d-3a175319-990dff1f-b961c89f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:52.058426Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|badbb3d-3a175319-990dff1f-b961c89f_0] Write session will restart in 0.009840s
2025-10-09T21:43:52.058450Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|badbb3d-3a175319-990dff1f-b961c89f_0] Write session: Do CDS request
2025-10-09T21:43:52.058457Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|badbb3d-3a175319-990dff1f-b961c89f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:52.070473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|badbb3d-3a175319-990dff1f-b961c89f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:52.073916Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:52.073933Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 212
2025-10-09T21:43:52.074165Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 212 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:52.074201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 212 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:52.074208Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=212 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:52.074213Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:52.074687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 212 sessionId:  describe result for acl check
2025-10-09T21:43:52.074727Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:52.074729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:52.074731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:52.074737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341007137480157:3891] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:52.075330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341007137480157:3891] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:52.089872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341007137480157:3891] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:52.100742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341007137480157:3891] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:52.100766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341007137480157:3891] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:52.100770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341007137480157:3891] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:52.100781Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 212 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:52.104493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:52.104812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:52.105095Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:52.105100Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:52.105117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:52.105182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:52.105223Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341006775670453:3611]
2025-10-09T21:43:52.105812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:52.106563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:52.106618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:52.106951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:52.107028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:52.107030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:52.107032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:52.107033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:52.107035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:52.107036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:52.107037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.107038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:52.107040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:52.107046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:52.107058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:52.107061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:52.107066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:52.107070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:52.107074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:52.107078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.107082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:52.107086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:52.107095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:52.107104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:52.107110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:52.107115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:52.107120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:52.107126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:52.107131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:52.107136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:52.107141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:52.107147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:52.107151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:52.107157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:52.107162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.107166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.107170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:52.107175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:52.107178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:52.107183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:52.107213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:52.116160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:52.117451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:52.117458Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:52.117460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:52.117465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:52.117474Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 211 [2:7559341006775670453:3611]
2025-10-09T21:43:52.117490Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:52.117511Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:52.117515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.117519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.117522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.117542Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:52.117548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:52.117556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:52.117625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.117629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:52.117632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:52.117635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:52.117958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:52.117961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.117976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:52.117991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:52.118013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:52.118055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341006775670446:3611]
2025-10-09T21:43:52.118057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341006775670446:3611]
2025-10-09T21:43:52.118059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341006775670446:3611]
2025-10-09T21:43:52.118081Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:52.118098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:52.118112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:52.118115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:52.118160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341006775670446:3611]
2025-10-09T21:43:52.118163Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:52.118219Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:52.118223Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:52.118226Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:52.118233Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:52.118640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341007137480097:2432], now have 1 active actors on pipe
2025-10-09T21:43:52.118652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341007137480355:3891], now have 1 active actors on pipe
2025-10-09T21:43:52.118987Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 211
2025-10-09T21:43:52.119027Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 211
2025-10-09T21:43:52.121764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:52.121785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:52.121827Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3987fdd9-6ea7cad-31d3374e-6d014301_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:52.121844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.121850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.121854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:52.121869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.121890Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:52.121912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:52.122294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:52.122309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:52.122329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:52.122476Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 212 partition: 0 MaxSeqNo: 25 sessionId: src_id|3987fdd9-6ea7cad-31d3374e-6d014301_0
2025-10-09T21:43:52.122926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|badbb3d-3a175319-990dff1f-b961c89f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3987fdd9-6ea7cad-31d3374e-6d014301_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:52.125877Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3987fdd9-6ea7cad-31d3374e-6d014301_0] Write session: try to update token
2025-10-09T21:43:52.125897Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3987fdd9-6ea7cad-31d3374e-6d014301_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:52.137267Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:52.137290Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:52.137293Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:52.137294Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:52.137305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341006775670446:3611]
2025-10-09T21:43:52.137310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341006775670446:3611]
2025-10-09T21:43:52.137313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341006775670446:3611]
2025-10-09T21:43:52.137363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:52.137373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.137377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:52.137608Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:52.140812Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:52.140834Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:52.140843Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:52.161301Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341007137480368:3810]
2025-10-09T21:43:52.163532Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 211, pipe: [1:7559341007137480368:3810]
2025-10-09T21:43:52.186797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 212 sessionId: src_id|3987fdd9-6ea7cad-31d3374e-6d014301_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:52.189268Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:52.423992Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:52.424282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341007137480368:3810], now have 1 active actors on pipe
2025-10-09T21:43:52.424337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:52.424343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:52.424756Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341007137480368:3810]
2025-10-09T21:43:52.424787Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:52.424822Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:52.424824Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:52.424877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 212 reason: error at write: Disconnected sessionId: src_id|3987fdd9-6ea7cad-31d3374e-6d014301_0
2025-10-09T21:43:52.425054Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 212 sessionId: src_id|3987fdd9-6ea7cad-31d3374e-6d014301_0 is DEAD
2025-10-09T21:43:52.424716Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:52.426267Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3987fdd9-6ea7cad-31d3374e-6d014301_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:52.426278Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3987fdd9-6ea7cad-31d3374e-6d014301_0] Write session will restart in 0.008034s
2025-10-09T21:43:52.426304Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3987fdd9-6ea7cad-31d3374e-6d014301_0] Write session: Do CDS request
2025-10-09T21:43:52.426310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3987fdd9-6ea7cad-31d3374e-6d014301_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:52.438953Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3987fdd9-6ea7cad-31d3374e-6d014301_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:52.439823Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:52.439839Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 213
2025-10-09T21:43:52.440659Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 213 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:52.440702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 213 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:52.440709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=213 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:52.440712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:52.441655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 213 sessionId:  describe result for acl check
2025-10-09T21:43:52.441692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:52.441694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:52.441696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:52.441704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341007137480386:3896] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:52.442298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341007137480386:3896] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:52.449015Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341007137480386:3896] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:52.452357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341007137480386:3896] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:52.452371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341007137480386:3896] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:52.452375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341007137480386:3896] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:52.452386Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 213 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:52.495371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:52.497432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:52.497656Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:52.497660Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:52.497674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:52.497738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:52.497780Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341006775670724:3613]
2025-10-09T21:43:52.498447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:52.499066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:52.499123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:52.499448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:52.499508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:52.499509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:52.499510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:52.499511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:52.499513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:52.499514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:52.499515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.499516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:52.499517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:52.499523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:52.499534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:52.499537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:52.499541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:52.499544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:52.499548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:52.499551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.499555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:52.499559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:52.499568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:52.499575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:52.499582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:52.499587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:52.499591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:52.499596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:52.499601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:52.499605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:52.499610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:52.499614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:52.499620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:52.499625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:52.499630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.499635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.499640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:52.499645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:52.499651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:52.499656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:52.499683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:52.512632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:52.515399Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 212
2025-10-09T21:43:52.515440Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 212
2025-10-09T21:43:52.514011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:52.514049Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:52.514053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:52.514058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:52.514067Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 212 [2:7559341006775670724:3613]
2025-10-09T21:43:52.514086Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:52.514110Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:52.514113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.514117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.514119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.514139Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:52.514145Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:52.514153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:52.514220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.514223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:52.514225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:52.514227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:52.514307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:52.514310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.514323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:52.514334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:52.514350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:52.514372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341006775670718:3613]
2025-10-09T21:43:52.514374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341006775670718:3613]
2025-10-09T21:43:52.514376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341006775670718:3613]
2025-10-09T21:43:52.514390Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:52.514405Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:52.514439Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:52.514442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:52.514487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341006775670718:3613]
2025-10-09T21:43:52.514490Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:52.514596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:52.514598Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:52.514601Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:52.514607Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:52.518771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341007137480375:2432], now have 1 active actors on pipe
2025-10-09T21:43:52.518786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341007137480423:3896], now have 1 active actors on pipe
2025-10-09T21:43:52.518813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:52.518820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:52.518858Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c0c47a90-e8517b51-673e9df8-6f5b59de_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:52.518870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.518875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.518878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:52.518891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.518909Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:52.518929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:52.519288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:52.519296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:52.519311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:52.519452Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 213 partition: 0 MaxSeqNo: 25 sessionId: src_id|c0c47a90-e8517b51-673e9df8-6f5b59de_0
2025-10-09T21:43:52.519873Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3987fdd9-6ea7cad-31d3374e-6d014301_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c0c47a90-e8517b51-673e9df8-6f5b59de_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:52.522386Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c0c47a90-e8517b51-673e9df8-6f5b59de_0] Write session: try to update token
2025-10-09T21:43:52.522404Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c0c47a90-e8517b51-673e9df8-6f5b59de_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:52.531985Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341007137480648:3810]
2025-10-09T21:43:52.533190Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 212, pipe: [1:7559341007137480648:3810]
2025-10-09T21:43:52.533259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341007137480648:3810], now have 1 active actors on pipe
2025-10-09T21:43:52.560147Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 213 sessionId: src_id|c0c47a90-e8517b51-673e9df8-6f5b59de_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:52.562674Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:52.563117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:52.563139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:52.563183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:52.567222Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:52.567240Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:52.567242Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:52.567244Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:52.567254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341006775670718:3613]
2025-10-09T21:43:52.567262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341006775670718:3613]
2025-10-09T21:43:52.567266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341006775670718:3613]
2025-10-09T21:43:52.567311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:52.567319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.567322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:52.567463Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:52.570764Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:52.570778Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:52.570781Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:52.570875Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:52.670267Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] Counters: { Errors: 2 CurrentSessionLifetimeMs: 61121 BytesRead: 24780812 MessagesRead: 11 BytesReadCompressed: 24780812 BytesInflightUncompressed: 2252802 BytesInflightCompressed: 0 BytesInflightTotal: 2252802 MessagesInflight: 1 }
2025-10-09T21:43:52.873776Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:52.895293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:52.895318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:52.895488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:52.895555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:52.895618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:52.895677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:52.895723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:52.895729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:52.895797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:52.895855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:52.895917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:52.896023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:52.896085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:52.896090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:52.896210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:52.896271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:52.896337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:52.896414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:52.896457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:52.896461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:52.896549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:52.896624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:52.896702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:52.896784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:52.896827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:52.896831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:52.896915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:52.896978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:52.897043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:52.897112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:52.897153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:52.897157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:52.897224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:52.897283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:52.897342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:52.897407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:52.897450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:52.897453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:52.897515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:52.897569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:52.897630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:52.897688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:52.897730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:52.897735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:52.897796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:52.897852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:52.897909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:52.897963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:52.898011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:52.898014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:52.902129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:52.902212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:52.902277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:52.902348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:52.902407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:52.902412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:52.902511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:52.902567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:52.902626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:52.902687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:52.902736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:52.902743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:52.904100Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:52.906636Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341007137480648:3810]
2025-10-09T21:43:52.906664Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:52.906698Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:52.906742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 213 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c0c47a90-e8517b51-673e9df8-6f5b59de_0
2025-10-09T21:43:52.906931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 213 sessionId: src_id|c0c47a90-e8517b51-673e9df8-6f5b59de_0 is DEAD
2025-10-09T21:43:52.910836Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0c47a90-e8517b51-673e9df8-6f5b59de_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:52.910849Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0c47a90-e8517b51-673e9df8-6f5b59de_0] Write session will restart in 0.006461s
2025-10-09T21:43:52.910874Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0c47a90-e8517b51-673e9df8-6f5b59de_0] Write session: Do CDS request
2025-10-09T21:43:52.910881Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0c47a90-e8517b51-673e9df8-6f5b59de_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:52.922846Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c0c47a90-e8517b51-673e9df8-6f5b59de_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:52.922718Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:52.922733Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 214
2025-10-09T21:43:52.923086Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 214 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:52.923121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 214 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:52.923127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=214 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:52.923132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:52.924203Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 214 sessionId:  describe result for acl check
2025-10-09T21:43:52.924238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:52.924240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:52.924242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:52.924249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341007137480700:3908] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:52.924803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341007137480700:3908] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:52.933096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341007137480700:3908] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:52.939422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341007137480700:3908] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:52.939441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341007137480700:3908] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:52.939445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341007137480700:3908] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:52.939455Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 214 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:52.989655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:52.991918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:52.992140Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:52.992144Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:52.992157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:52.992214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:52.992257Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341006775671037:3628]
2025-10-09T21:43:52.992844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:52.996817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:52.996905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:52.997292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:52.997372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:52.997373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:52.997374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:52.997376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:52.997377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:52.997379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:52.997380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.997381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:52.997382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:52.997389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:52.997401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:52.997405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:52.997410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:52.997414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:52.997418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:52.997422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.997426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:52.997430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:52.997440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:52.997464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:52.997469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:52.997475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:52.997480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:52.997486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:52.997491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:52.997495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:52.997500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:52.997505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:52.997509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:52.997515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:52.997521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.997526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:52.997532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:52.997537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:52.997543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:52.997548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:52.997582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:53.006573Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341011432448255:3810]
2025-10-09T21:43:53.010536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:53.011814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:53.011822Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:53.011826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:53.011831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:53.011840Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 213 [2:7559341006775671037:3628]
2025-10-09T21:43:53.011856Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:53.011877Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:53.011881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.011884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.011887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.011906Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:53.011913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:53.011920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:53.011993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.011995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:53.011997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:53.012000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:53.012082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:53.012086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.012101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:53.012111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:53.012131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:53.012153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341006775671031:3628]
2025-10-09T21:43:53.012155Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341006775671031:3628]
2025-10-09T21:43:53.012158Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341006775671031:3628]
2025-10-09T21:43:53.012174Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:53.012190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:53.012209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:53.012212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:53.012256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341006775671031:3628]
2025-10-09T21:43:53.012258Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:53.012509Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:53.012517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:53.012520Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:53.012526Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:53.013350Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 213, pipe: [1:7559341011432448255:3810]
2025-10-09T21:43:53.013387Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 213
2025-10-09T21:43:53.013413Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 213
2025-10-09T21:43:53.014100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341011432448255:3810], now have 1 active actors on pipe
2025-10-09T21:43:53.014113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341007137480691:2432], now have 1 active actors on pipe
2025-10-09T21:43:53.014117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341007137480739:3908], now have 1 active actors on pipe
2025-10-09T21:43:53.014135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:53.014142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:53.014184Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|30451874-f8d403-1ebcc4d6-7dded2b4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:53.014197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.014203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.014207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:53.014222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.014240Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:53.014263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:53.018258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:53.018279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:53.018322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:53.018567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 214 partition: 0 MaxSeqNo: 25 sessionId: src_id|30451874-f8d403-1ebcc4d6-7dded2b4_0
2025-10-09T21:43:53.019014Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0c47a90-e8517b51-673e9df8-6f5b59de_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|30451874-f8d403-1ebcc4d6-7dded2b4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:53.021549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|30451874-f8d403-1ebcc4d6-7dded2b4_0] Write session: try to update token
2025-10-09T21:43:53.021569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|30451874-f8d403-1ebcc4d6-7dded2b4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:53.070645Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:53.070669Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:53.070672Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:53.070675Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:53.070686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341006775671031:3628]
2025-10-09T21:43:53.070707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341006775671031:3628]
2025-10-09T21:43:53.070712Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341006775671031:3628]
2025-10-09T21:43:53.070765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:53.070777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.070780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:53.070930Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:53.074801Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:53.074815Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:53.074820Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:53.081221Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 214 sessionId: src_id|30451874-f8d403-1ebcc4d6-7dded2b4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:53.083940Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:53.422742Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:53.423340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:53.423353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:53.423867Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341011432448255:3810]
2025-10-09T21:43:53.423949Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:53.424000Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:53.424003Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:53.424062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 214 reason: error at write: Disconnected sessionId: src_id|30451874-f8d403-1ebcc4d6-7dded2b4_0
2025-10-09T21:43:53.424242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 214 sessionId: src_id|30451874-f8d403-1ebcc4d6-7dded2b4_0 is DEAD
2025-10-09T21:43:53.424317Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:53.430200Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30451874-f8d403-1ebcc4d6-7dded2b4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:53.430217Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30451874-f8d403-1ebcc4d6-7dded2b4_0] Write session will restart in 0.009224s
2025-10-09T21:43:53.430246Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30451874-f8d403-1ebcc4d6-7dded2b4_0] Write session: Do CDS request
2025-10-09T21:43:53.430256Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30451874-f8d403-1ebcc4d6-7dded2b4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:53.440576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|30451874-f8d403-1ebcc4d6-7dded2b4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:53.441836Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:53.441851Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 215
2025-10-09T21:43:53.442049Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 215 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:53.442083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 215 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:53.442090Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=215 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:53.442094Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:53.443315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 215 sessionId:  describe result for acl check
2025-10-09T21:43:53.443355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:53.443358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:53.443359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:53.443365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341011432448279:3913] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:53.443905Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341011432448279:3913] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:53.454751Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341011432448279:3913] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:53.466005Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341011432448279:3913] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:53.466023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341011432448279:3913] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:53.466048Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341011432448279:3913] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:53.466059Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 215 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:53.504119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:53.506848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:53.507091Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:53.507095Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:53.507109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:53.507169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:53.507209Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341011070638606:3631]
2025-10-09T21:43:53.507782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:53.508433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:53.508484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:53.508802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:53.508859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:53.508860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:53.508861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:53.508863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:53.508864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:53.508865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:53.508866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.508867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:53.508868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:53.508873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:53.508885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:53.508889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:53.508893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:53.508896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:53.508900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:53.508904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.508908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:53.508912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:53.508921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:53.508930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:53.508937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:53.508943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:53.508947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:53.508952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:53.508957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:53.508962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:53.508966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:53.508971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:53.508974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:53.508979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:53.508984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.508988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.508992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:53.508996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:53.509000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:53.509007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:53.509053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:53.518686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:53.519747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:53.519754Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:53.519758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:53.519763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:53.519771Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 214 [2:7559341011070638606:3631]
2025-10-09T21:43:53.519786Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:53.519806Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:53.519810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.519813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.519815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.519834Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:53.519840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:53.519848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:53.519912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.519914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:53.519916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:53.519919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:53.520342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:53.520347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.520363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:53.520377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:53.520397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:53.520421Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341011070638600:3631]
2025-10-09T21:43:53.520423Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341011070638600:3631]
2025-10-09T21:43:53.520426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341011070638600:3631]
2025-10-09T21:43:53.520436Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:53.520453Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:53.520468Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:53.520471Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:53.520512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341011070638600:3631]
2025-10-09T21:43:53.520514Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:53.520638Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:53.520640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:53.520642Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:53.520647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:53.520984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341011432448365:3913], now have 1 active actors on pipe
2025-10-09T21:43:53.520988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341011432448269:2432], now have 1 active actors on pipe
2025-10-09T21:43:53.522355Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 214
2025-10-09T21:43:53.522385Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 214
2025-10-09T21:43:53.523204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:53.523218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:53.523256Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2ea9d199-71cae093-c255db9c-43cb92e8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:53.523270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.523275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.523279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:53.523291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.523309Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:53.523329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:53.523782Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341011432448545:3810]
2025-10-09T21:43:53.524330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:53.524340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:53.524362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:53.524550Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 215 partition: 0 MaxSeqNo: 25 sessionId: src_id|2ea9d199-71cae093-c255db9c-43cb92e8_0
2025-10-09T21:43:53.525000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341011432448545:3810], now have 1 active actors on pipe
2025-10-09T21:43:53.525122Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 214, pipe: [1:7559341011432448545:3810]
2025-10-09T21:43:53.526451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30451874-f8d403-1ebcc4d6-7dded2b4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2ea9d199-71cae093-c255db9c-43cb92e8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:53.528938Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ea9d199-71cae093-c255db9c-43cb92e8_0] Write session: try to update token
2025-10-09T21:43:53.528958Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ea9d199-71cae093-c255db9c-43cb92e8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:53.565718Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 215 sessionId: src_id|2ea9d199-71cae093-c255db9c-43cb92e8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:53.567964Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:53.567989Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:53.567991Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:53.567993Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:53.568003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341011070638600:3631]
2025-10-09T21:43:53.568012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341011070638600:3631]
2025-10-09T21:43:53.568015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341011070638600:3631]
2025-10-09T21:43:53.568064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:53.568070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.568073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:53.568425Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:53.568212Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:53.569151Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:53.569157Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:53.569161Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:53.785374Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:53.785645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:53.785661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:53.786324Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:53.786338Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:53.786396Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 215 reason: error at write: Disconnected sessionId: src_id|2ea9d199-71cae093-c255db9c-43cb92e8_0
2025-10-09T21:43:53.786603Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 215 sessionId: src_id|2ea9d199-71cae093-c255db9c-43cb92e8_0 is DEAD
2025-10-09T21:43:53.786736Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341011432448545:3810]
2025-10-09T21:43:53.786750Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:53.786904Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ea9d199-71cae093-c255db9c-43cb92e8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:53.786932Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ea9d199-71cae093-c255db9c-43cb92e8_0] Write session will restart in 0.007882s
2025-10-09T21:43:53.786958Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ea9d199-71cae093-c255db9c-43cb92e8_0] Write session: Do CDS request
2025-10-09T21:43:53.786965Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ea9d199-71cae093-c255db9c-43cb92e8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:53.786143Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:53.798722Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ea9d199-71cae093-c255db9c-43cb92e8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:53.799637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:53.799654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 216
2025-10-09T21:43:53.799871Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 216 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:53.799907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 216 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:53.799913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=216 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:53.799917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:53.800517Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 216 sessionId:  describe result for acl check
2025-10-09T21:43:53.800555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:53.800557Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:53.800559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:53.800565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341011432448567:3917] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:53.801219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341011432448567:3917] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:53.809776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341011432448567:3917] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:53.822498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341011432448567:3917] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:53.822517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341011432448567:3917] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:53.822521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341011432448567:3917] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:53.822531Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 216 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:53.843734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:53.845998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:53.846278Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:53.846283Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:53.846295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:53.846376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:53.846429Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341011070638909:3637]
2025-10-09T21:43:53.847052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:53.854310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:53.854409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:53.854798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:53.854877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:53.854879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:53.854880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:53.854881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:53.854882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:53.854884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:53.854885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.854886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:53.854887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:53.854894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:53.854906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:53.854910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:53.854915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:53.854919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:53.854922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:53.854926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.854930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:53.854933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:53.854943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:53.854952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:53.854958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:53.854964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:53.854969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:53.854974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:53.854978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:53.854983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:53.854988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:53.854993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:53.854998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:53.855002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:53.855008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.855012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.855017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:53.855021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:53.855026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:53.855031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:53.855061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:53.866832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:53.868381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:53.868388Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:53.868391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:53.868396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:53.868405Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 215 [2:7559341011070638909:3637]
2025-10-09T21:43:53.868420Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:53.868440Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:53.868444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.868448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.868450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.868470Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:53.868476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:53.868485Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:53.868554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.868557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:53.868559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:53.868562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:53.868648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:53.868651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.868665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:53.868676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:53.868693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:53.868716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341011070638897:3637]
2025-10-09T21:43:53.868719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341011070638897:3637]
2025-10-09T21:43:53.868721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341011070638897:3637]
2025-10-09T21:43:53.868736Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:53.868751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:53.868767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:53.868771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:53.868814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341011070638897:3637]
2025-10-09T21:43:53.868817Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:53.868836Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:53.868840Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:53.868842Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:53.868850Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:53.869292Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 215
2025-10-09T21:43:53.869312Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 215
2025-10-09T21:43:53.869384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341011432448551:2432], now have 1 active actors on pipe
2025-10-09T21:43:53.869395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341011432448696:3917], now have 1 active actors on pipe
2025-10-09T21:43:53.869511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:53.869520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:53.869554Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b66230ab-c185d0c9-e0580fab-694b2f71_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:53.869565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.869572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.869575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:53.869586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.869597Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:53.869615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:53.869890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:53.869896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:53.869911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:53.873638Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 216 partition: 0 MaxSeqNo: 25 sessionId: src_id|b66230ab-c185d0c9-e0580fab-694b2f71_0
2025-10-09T21:43:53.878639Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ea9d199-71cae093-c255db9c-43cb92e8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b66230ab-c185d0c9-e0580fab-694b2f71_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:53.881115Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b66230ab-c185d0c9-e0580fab-694b2f71_0] Write session: try to update token
2025-10-09T21:43:53.881137Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b66230ab-c185d0c9-e0580fab-694b2f71_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:53.890407Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341011432448833:3810]
2025-10-09T21:43:53.892250Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:53.892270Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:53.892274Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:53.892277Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:53.892287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341011070638897:3637]
2025-10-09T21:43:53.892292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341011070638897:3637]
2025-10-09T21:43:53.892296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341011070638897:3637]
2025-10-09T21:43:53.892347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:53.892356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:53.892358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:53.892524Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:53.894460Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:53.894472Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:53.894476Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:53.895002Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 215, pipe: [1:7559341011432448833:3810]
2025-10-09T21:43:53.930670Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 216 sessionId: src_id|b66230ab-c185d0c9-e0580fab-694b2f71_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:53.933076Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:54.433633Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:54.431135Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:54.431714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341011432448833:3810], now have 1 active actors on pipe
2025-10-09T21:43:54.431770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:54.431777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:54.432834Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:54.433722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 216 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b66230ab-c185d0c9-e0580fab-694b2f71_0
2025-10-09T21:43:54.434752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b66230ab-c185d0c9-e0580fab-694b2f71_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:54.434765Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b66230ab-c185d0c9-e0580fab-694b2f71_0] Write session will restart in 0.007417s
2025-10-09T21:43:54.434788Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b66230ab-c185d0c9-e0580fab-694b2f71_0] Write session: Do CDS request
2025-10-09T21:43:54.434795Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b66230ab-c185d0c9-e0580fab-694b2f71_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:54.433901Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 216 sessionId: src_id|b66230ab-c185d0c9-e0580fab-694b2f71_0 is DEAD
2025-10-09T21:43:54.434065Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341011432448833:3810]
2025-10-09T21:43:54.434086Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:54.443715Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b66230ab-c185d0c9-e0580fab-694b2f71_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:54.450242Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:54.450260Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 217
2025-10-09T21:43:54.450507Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 217 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:54.450539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 217 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:54.450544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=217 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:54.450548Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:54.451777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 217 sessionId:  describe result for acl check
2025-10-09T21:43:54.451814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:54.451816Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:54.451817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:54.451824Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341015727416255:3930] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:54.452365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341015727416255:3930] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:54.466116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341015727416255:3930] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:54.475619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341015727416255:3930] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:54.475635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341015727416255:3930] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:54.475639Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341015727416255:3930] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:54.475766Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 217 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:54.478205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:54.479778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:54.480138Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:54.480142Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:54.480155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:54.480220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:54.480268Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341015365606491:3649]
2025-10-09T21:43:54.480871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:54.481259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:54.481305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:54.481609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:54.481657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:54.481658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:54.481659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:54.481660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:54.481661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:54.481662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:54.481663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:54.481664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:54.481665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:54.481671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:54.481683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:54.481687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:54.481692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:54.481695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:54.481699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:54.481703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:54.481707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:54.481711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:54.481721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:54.481729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:54.481736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:54.481742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:54.481746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:54.481751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:54.481756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:54.481761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:54.481767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:54.481772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:54.481776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:54.481781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:54.481786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:54.481792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:54.481796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:54.481801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:54.481806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:54.481811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:54.481841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:54.489600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:54.490876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:54.490882Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:54.490885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:54.490889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:54.490897Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 216 [2:7559341015365606491:3649]
2025-10-09T21:43:54.490916Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:54.490936Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:54.490940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.490943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.490946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.490964Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:54.490970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:54.490978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:54.491039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:54.491042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:54.491044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:54.491047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:54.491121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:54.491123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:54.491136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:54.491147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:54.491161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:54.491182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341015365606485:3649]
2025-10-09T21:43:54.491185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341015365606485:3649]
2025-10-09T21:43:54.491187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341015365606485:3649]
2025-10-09T21:43:54.491201Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:54.491215Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:54.491228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:54.491231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:54.491270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341015365606485:3649]
2025-10-09T21:43:54.491273Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:54.492010Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 216
2025-10-09T21:43:54.492058Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 216
2025-10-09T21:43:54.491677Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:54.491684Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:54.491686Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:54.491692Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:54.492368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341015727416172:2432], now have 1 active actors on pipe
2025-10-09T21:43:54.492373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341015727416433:3930], now have 1 active actors on pipe
2025-10-09T21:43:54.492582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:54.492592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:54.492623Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|739bfc48-cdfb5282-d01e4a76-ea763e07_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:54.492632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.492637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.492640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:54.492653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.492666Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:54.492694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:54.492952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:54.492959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:54.492973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:54.493095Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 217 partition: 0 MaxSeqNo: 25 sessionId: src_id|739bfc48-cdfb5282-d01e4a76-ea763e07_0
2025-10-09T21:43:54.494527Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b66230ab-c185d0c9-e0580fab-694b2f71_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|739bfc48-cdfb5282-d01e4a76-ea763e07_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:54.497009Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|739bfc48-cdfb5282-d01e4a76-ea763e07_0] Write session: try to update token
2025-10-09T21:43:54.497028Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|739bfc48-cdfb5282-d01e4a76-ea763e07_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:54.511522Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:54.511540Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:54.511543Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:54.511545Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:54.511555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341015365606485:3649]
2025-10-09T21:43:54.511560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341015365606485:3649]
2025-10-09T21:43:54.511563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341015365606485:3649]
2025-10-09T21:43:54.511608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:54.511616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:54.511619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:54.511768Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:54.514462Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:54.514475Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:54.514498Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:54.519122Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 217 sessionId: src_id|739bfc48-cdfb5282-d01e4a76-ea763e07_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:54.521511Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:54.534372Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341015727416448:3810]
2025-10-09T21:43:54.535320Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 216, pipe: [1:7559341015727416448:3810]
2025-10-09T21:43:54.758407Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:54.759021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:54.759034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:54.759084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341015727416448:3810], now have 1 active actors on pipe
2025-10-09T21:43:54.760717Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|739bfc48-cdfb5282-d01e4a76-ea763e07_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:54.760730Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|739bfc48-cdfb5282-d01e4a76-ea763e07_0] Write session will restart in 0.008135s
2025-10-09T21:43:54.760766Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|739bfc48-cdfb5282-d01e4a76-ea763e07_0] Write session: Do CDS request
2025-10-09T21:43:54.760773Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|739bfc48-cdfb5282-d01e4a76-ea763e07_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:54.759965Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:54.759979Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:54.760039Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 217 reason: error at write: Disconnected sessionId: src_id|739bfc48-cdfb5282-d01e4a76-ea763e07_0
2025-10-09T21:43:54.760235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 217 sessionId: src_id|739bfc48-cdfb5282-d01e4a76-ea763e07_0 is DEAD
2025-10-09T21:43:54.760440Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341015727416448:3810]
2025-10-09T21:43:54.760455Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:54.762895Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:54.770598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|739bfc48-cdfb5282-d01e4a76-ea763e07_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:54.774013Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:54.774054Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 218
2025-10-09T21:43:54.776593Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 218 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:54.776641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 218 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:54.776647Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=218 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:54.776652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:54.777149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 218 sessionId:  describe result for acl check
2025-10-09T21:43:54.777189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:54.777191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:54.777193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:54.777199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341015727416473:3934] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:54.777693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341015727416473:3934] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:54.788664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341015727416473:3934] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:54.800109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341015727416473:3934] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:54.800127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341015727416473:3934] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:54.800132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341015727416473:3934] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:54.800141Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 218 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:54.816118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:54.816413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:54.816574Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:54.816578Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:54.816590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:54.816639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:54.816679Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341015365606767:3651]
2025-10-09T21:43:54.817276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:54.817903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:54.817953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:54.818282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:54.818330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:54.818331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:54.818333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:54.818334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:54.818335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:54.818336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:54.818337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:54.818338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:54.818340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:54.818345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:54.818356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:54.818360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:54.818363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:54.818367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:54.818371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:54.818375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:54.818379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:54.818383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:54.818393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:54.818402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:54.818408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:54.818413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:54.818430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:54.818434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:54.818437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:54.818443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:54.818447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:54.818452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:54.818456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:54.818461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:54.818465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:54.818470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:54.818474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:54.818479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:54.818483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:54.818488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:54.818530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:54.826922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:54.828216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:54.828233Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:54.828236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:54.828240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:54.828247Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 217 [2:7559341015365606767:3651]
2025-10-09T21:43:54.828263Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:54.828282Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:54.828286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.828289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.828291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.828309Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:54.828318Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:54.828325Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:54.828389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:54.828391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:54.828393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:54.828396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:54.828456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:54.828458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:54.828472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:54.828484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:54.828493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:54.828508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341015365606761:3651]
2025-10-09T21:43:54.828510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341015365606761:3651]
2025-10-09T21:43:54.828511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341015365606761:3651]
2025-10-09T21:43:54.828521Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:54.828534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:54.828542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:54.828544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:54.828576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341015365606761:3651]
2025-10-09T21:43:54.828578Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:54.831044Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:54.831056Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:54.831059Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:54.831067Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:54.831471Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 217
2025-10-09T21:43:54.831530Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 217
2025-10-09T21:43:54.835923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341015727416453:2432], now have 1 active actors on pipe
2025-10-09T21:43:54.835941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341015727416631:3934], now have 1 active actors on pipe
2025-10-09T21:43:54.835964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:54.835972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:54.836016Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fc0dcd1a-aaaa69fd-d4780f07-b3627ed4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:54.836029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.836035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.836038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:54.836053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.836074Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:54.836098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:54.836672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 218 partition: 0 MaxSeqNo: 25 sessionId: src_id|fc0dcd1a-aaaa69fd-d4780f07-b3627ed4_0
2025-10-09T21:43:54.837145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|739bfc48-cdfb5282-d01e4a76-ea763e07_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fc0dcd1a-aaaa69fd-d4780f07-b3627ed4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:54.836444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:54.836468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:54.836491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:54.839624Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc0dcd1a-aaaa69fd-d4780f07-b3627ed4_0] Write session: try to update token
2025-10-09T21:43:54.839644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc0dcd1a-aaaa69fd-d4780f07-b3627ed4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:54.852200Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:54.852218Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:54.852221Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:54.852223Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:54.852232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341015365606761:3651]
2025-10-09T21:43:54.852236Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341015365606761:3651]
2025-10-09T21:43:54.852241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341015365606761:3651]
2025-10-09T21:43:54.852285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:54.852292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:54.852298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:54.853122Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:54.854171Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:54.854182Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:54.854186Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:54.860082Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 218 sessionId: src_id|fc0dcd1a-aaaa69fd-d4780f07-b3627ed4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:54.860475Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341015727416739:3810]
2025-10-09T21:43:54.861327Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 217, pipe: [1:7559341015727416739:3810]
2025-10-09T21:43:54.862680Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:55.059531Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:55.060088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341015727416739:3810], now have 1 active actors on pipe
2025-10-09T21:43:55.060124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:55.060132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:55.060669Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:55.061318Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc0dcd1a-aaaa69fd-d4780f07-b3627ed4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:55.061331Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc0dcd1a-aaaa69fd-d4780f07-b3627ed4_0] Write session will restart in 0.005995s
2025-10-09T21:43:55.061355Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc0dcd1a-aaaa69fd-d4780f07-b3627ed4_0] Write session: Do CDS request
2025-10-09T21:43:55.061362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc0dcd1a-aaaa69fd-d4780f07-b3627ed4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:55.060707Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:55.060722Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:55.060785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 218 reason: error at write: Disconnected sessionId: src_id|fc0dcd1a-aaaa69fd-d4780f07-b3627ed4_0
2025-10-09T21:43:55.060954Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:55.060982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 218 sessionId: src_id|fc0dcd1a-aaaa69fd-d4780f07-b3627ed4_0 is DEAD
2025-10-09T21:43:55.060993Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341015727416739:3810]
2025-10-09T21:43:55.072369Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc0dcd1a-aaaa69fd-d4780f07-b3627ed4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:55.072863Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:55.072883Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 219
2025-10-09T21:43:55.073052Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 219 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:55.073088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 219 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:55.073098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=219 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:55.073102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:55.073853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 219 sessionId:  describe result for acl check
2025-10-09T21:43:55.073895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:55.073901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:55.073903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:55.073908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341020022384054:3939] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:55.074584Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341020022384054:3939] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:55.080431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341020022384054:3939] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:55.095130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341020022384054:3939] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:55.095149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341020022384054:3939] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:55.095153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341020022384054:3939] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:55.095167Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 219 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:55.123379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:55.128345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:55.128597Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:55.128602Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:55.128615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:55.128677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:55.128716Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341019660574368:3665]
2025-10-09T21:43:55.129320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:55.130025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:55.130097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:55.130453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:55.130502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:55.130504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:55.130505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:55.130506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:55.130508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:55.130509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:55.130510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.130512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.130513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.130519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:55.130531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:55.130535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:55.130539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:55.130542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:55.130545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:55.130549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.130553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.130556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.130563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:55.130570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:55.130576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:55.130582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:55.130586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:55.130591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:55.130595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:55.130600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:55.130604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:55.130609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:55.130613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:55.130618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:55.130625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.130629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.130633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.130638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.130645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.130650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.130676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:55.146906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:55.148157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:55.148164Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:55.148166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:55.148171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:55.148179Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 218 [2:7559341019660574368:3665]
2025-10-09T21:43:55.148192Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:55.148214Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:55.148218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.148221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.148224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.148244Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:55.148250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:55.148257Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:55.148324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.148326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.148329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.148332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:55.148408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:55.148411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.148426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.148439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.148458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:55.148479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341019660574362:3665]
2025-10-09T21:43:55.148481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341019660574362:3665]
2025-10-09T21:43:55.148484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341019660574362:3665]
2025-10-09T21:43:55.148497Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:55.148513Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:55.148529Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:55.148532Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:55.148573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341019660574362:3665]
2025-10-09T21:43:55.148576Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:55.148837Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:55.148845Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:55.148847Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:55.148854Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:55.151421Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 218
2025-10-09T21:43:55.151455Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 218
2025-10-09T21:43:55.154091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341020022384145:3939], now have 1 active actors on pipe
2025-10-09T21:43:55.154107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341020022384044:2432], now have 1 active actors on pipe
2025-10-09T21:43:55.154128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:55.154135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:55.154179Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e68fee70-cf73999a-351e5518-5c1338e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:55.154194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.154200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.154204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:55.154218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.154236Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:55.154259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:55.156355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:55.156371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:55.156403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:55.160411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 219 partition: 0 MaxSeqNo: 25 sessionId: src_id|e68fee70-cf73999a-351e5518-5c1338e_0
2025-10-09T21:43:55.161113Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341020022384334:3810]
2025-10-09T21:43:55.161467Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc0dcd1a-aaaa69fd-d4780f07-b3627ed4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e68fee70-cf73999a-351e5518-5c1338e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:55.162441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341020022384334:3810], now have 1 active actors on pipe
2025-10-09T21:43:55.162711Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 218, pipe: [1:7559341020022384334:3810]
2025-10-09T21:43:55.163950Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e68fee70-cf73999a-351e5518-5c1338e_0] Write session: try to update token
2025-10-09T21:43:55.163969Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e68fee70-cf73999a-351e5518-5c1338e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:55.173128Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:55.173149Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:55.173151Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:55.173153Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:55.173162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341019660574362:3665]
2025-10-09T21:43:55.173165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341019660574362:3665]
2025-10-09T21:43:55.173168Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341019660574362:3665]
2025-10-09T21:43:55.173210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:55.173218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.173221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:55.173359Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:55.176245Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:55.176260Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:55.176264Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:55.213923Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 219 sessionId: src_id|e68fee70-cf73999a-351e5518-5c1338e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:55.216446Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:55.361417Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:55.361717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:55.361728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:55.362507Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:55.362521Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:55.362574Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:55.362583Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 219 reason: error at write: Disconnected sessionId: src_id|e68fee70-cf73999a-351e5518-5c1338e_0
2025-10-09T21:43:55.362621Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341020022384334:3810]
2025-10-09T21:43:55.362785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 219 sessionId: src_id|e68fee70-cf73999a-351e5518-5c1338e_0 is DEAD
2025-10-09T21:43:55.363080Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:55.366398Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e68fee70-cf73999a-351e5518-5c1338e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:55.366410Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e68fee70-cf73999a-351e5518-5c1338e_0] Write session will restart in 0.005301s
2025-10-09T21:43:55.366450Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e68fee70-cf73999a-351e5518-5c1338e_0] Write session: Do CDS request
2025-10-09T21:43:55.366461Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e68fee70-cf73999a-351e5518-5c1338e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:55.374806Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e68fee70-cf73999a-351e5518-5c1338e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:55.378229Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:55.378246Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 220
2025-10-09T21:43:55.378500Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 220 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:55.378532Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 220 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:55.378539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=220 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:55.378544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:55.379393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 220 sessionId:  describe result for acl check
2025-10-09T21:43:55.379437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:55.379439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:55.379441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:55.379446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341020022384372:3950] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:55.379994Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341020022384372:3950] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:55.383914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341020022384372:3950] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:55.392361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341020022384372:3950] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:55.392379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341020022384372:3950] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:55.392384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341020022384372:3950] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:55.392395Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 220 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:55.418548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:55.420908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:55.421131Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:55.421135Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:55.421150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:55.421214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:55.421250Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341019660574651:3669]
2025-10-09T21:43:55.421813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:55.422757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:55.422815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:55.423164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:55.423210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:55.423211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:55.423212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:55.423213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:55.423214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:55.423216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:55.423217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.423218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.423219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.423225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:55.423237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:55.423240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:55.423245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:55.423248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:55.423253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:55.423257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.423260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.423264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.423271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:55.423280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:55.423287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:55.423292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:55.423296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:55.423302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:55.423306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:55.423311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:55.423316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:55.423321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:55.423325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:55.423330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:55.423335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.423340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.423344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.423349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.423354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.423359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.423391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:55.432833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:55.434125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:55.434131Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:55.434134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:55.434139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:55.434148Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 219 [2:7559341019660574651:3669]
2025-10-09T21:43:55.434164Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:55.434187Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:55.434190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.434194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.434197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.434215Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:55.434221Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:55.434229Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:55.434301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.434306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.434308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.434310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:55.434391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:55.434394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.434407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.434432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.434450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:55.434473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341019660574645:3669]
2025-10-09T21:43:55.434475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341019660574645:3669]
2025-10-09T21:43:55.434478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341019660574645:3669]
2025-10-09T21:43:55.434492Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:55.434506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:55.434520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:55.434524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:55.434564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341019660574645:3669]
2025-10-09T21:43:55.434567Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:55.434834Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:55.434842Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:55.434844Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:55.434850Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:55.436371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341020022384441:3950], now have 1 active actors on pipe
2025-10-09T21:43:55.436385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341020022384351:2432], now have 1 active actors on pipe
2025-10-09T21:43:55.436413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:55.436419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:55.436453Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cf843526-15a369ff-1a5cc207-1a9f9b56_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:55.436464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.436469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.436472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:55.436485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.436501Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:55.436521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:55.437287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:55.437298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:55.437318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:55.435676Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 219
2025-10-09T21:43:55.435698Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 219
2025-10-09T21:43:55.437535Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 220 partition: 0 MaxSeqNo: 25 sessionId: src_id|cf843526-15a369ff-1a5cc207-1a9f9b56_0
2025-10-09T21:43:55.438446Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e68fee70-cf73999a-351e5518-5c1338e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cf843526-15a369ff-1a5cc207-1a9f9b56_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:55.440915Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf843526-15a369ff-1a5cc207-1a9f9b56_0] Write session: try to update token
2025-10-09T21:43:55.440935Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf843526-15a369ff-1a5cc207-1a9f9b56_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:55.455275Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:55.455295Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:55.455298Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:55.455300Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:55.455310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341019660574645:3669]
2025-10-09T21:43:55.455315Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341019660574645:3669]
2025-10-09T21:43:55.455318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341019660574645:3669]
2025-10-09T21:43:55.455358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:55.455365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.455368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:55.455649Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:55.459414Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:55.459430Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:55.459443Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:55.466099Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341020022384633:3810]
2025-10-09T21:43:55.466562Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 219, pipe: [1:7559341020022384633:3810]
2025-10-09T21:43:55.469617Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 220 sessionId: src_id|cf843526-15a369ff-1a5cc207-1a9f9b56_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:55.472130Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:55.634141Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:55.634748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341020022384633:3810], now have 1 active actors on pipe
2025-10-09T21:43:55.634784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:55.634804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:55.635544Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:55.635594Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341020022384633:3810]
2025-10-09T21:43:55.635612Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:55.635614Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:55.635660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 220 reason: error at write: Disconnected sessionId: src_id|cf843526-15a369ff-1a5cc207-1a9f9b56_0
2025-10-09T21:43:55.635841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 220 sessionId: src_id|cf843526-15a369ff-1a5cc207-1a9f9b56_0 is DEAD
2025-10-09T21:43:55.636263Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf843526-15a369ff-1a5cc207-1a9f9b56_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:55.636277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf843526-15a369ff-1a5cc207-1a9f9b56_0] Write session will restart in 0.009843s
2025-10-09T21:43:55.636302Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf843526-15a369ff-1a5cc207-1a9f9b56_0] Write session: Do CDS request
2025-10-09T21:43:55.636312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf843526-15a369ff-1a5cc207-1a9f9b56_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:55.635321Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:55.648726Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf843526-15a369ff-1a5cc207-1a9f9b56_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:55.649004Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:55.649023Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 221
2025-10-09T21:43:55.649205Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 221 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:55.649240Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 221 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:55.649246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=221 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:55.649255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:55.649720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 221 sessionId:  describe result for acl check
2025-10-09T21:43:55.649753Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:55.649757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:55.649758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:55.649764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341020022384666:3955] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:55.650322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341020022384666:3955] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:55.661294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341020022384666:3955] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:55.672124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341020022384666:3955] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:55.672145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341020022384666:3955] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:55.672149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341020022384666:3955] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:55.672160Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 221 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:55.683754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:55.685945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:55.686168Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:55.686172Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:55.686184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:55.686238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:55.686277Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341019660574927:3671]
2025-10-09T21:43:55.686843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:55.687399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:55.687444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:55.687740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:55.687796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:55.687797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:55.687798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:55.687799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:55.687800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:55.687802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:55.687803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.687804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.687805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.687810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:55.687821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:55.687825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:55.687828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:55.687832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:55.687836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:55.687839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.687842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.687846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.687853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:55.687860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:55.687866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:55.687871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:55.687874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:55.687879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:55.687883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:55.687888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:55.687893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:55.687898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:55.687902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:55.687907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:55.687912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.687916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.687921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.687926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.687930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.687935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.687964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:55.697721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:55.699532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:55.699548Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:55.699550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:55.699555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:55.699563Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 220 [2:7559341019660574927:3671]
2025-10-09T21:43:55.699579Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:55.699600Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:55.699604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.699607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.699610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.699630Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:55.699636Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:55.699643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:55.699714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.699716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.699718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.699720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:55.699799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:55.699802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.699816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:55.699826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:55.699840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:55.699862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341019660574921:3671]
2025-10-09T21:43:55.699864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341019660574921:3671]
2025-10-09T21:43:55.699866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341019660574921:3671]
2025-10-09T21:43:55.699880Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:55.699896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:55.699911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:55.699915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:55.699958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341019660574921:3671]
2025-10-09T21:43:55.699960Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:55.701089Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 220
2025-10-09T21:43:55.701128Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 220
2025-10-09T21:43:55.700219Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:55.700225Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:55.700226Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:55.700232Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:55.700511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341020022384636:2432], now have 1 active actors on pipe
2025-10-09T21:43:55.700527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341020022384826:3955], now have 1 active actors on pipe
2025-10-09T21:43:55.705556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:55.705571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:55.705606Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2908a322-4672e055-dfaf94be-ea160b6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:55.705617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.705622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.705624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:55.705636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.705659Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:55.705676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:55.706053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:55.706071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:55.706107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:55.706256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 221 partition: 0 MaxSeqNo: 25 sessionId: src_id|2908a322-4672e055-dfaf94be-ea160b6_0
2025-10-09T21:43:55.710112Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf843526-15a369ff-1a5cc207-1a9f9b56_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2908a322-4672e055-dfaf94be-ea160b6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:55.716397Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2908a322-4672e055-dfaf94be-ea160b6_0] Write session: try to update token
2025-10-09T21:43:55.716427Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2908a322-4672e055-dfaf94be-ea160b6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:55.729204Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:55.729225Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:55.729229Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:55.729231Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:55.729242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341019660574921:3671]
2025-10-09T21:43:55.729247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341019660574921:3671]
2025-10-09T21:43:55.729251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341019660574921:3671]
2025-10-09T21:43:55.729299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:55.729306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:55.729309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:55.729568Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:55.734125Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:55.734151Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:55.734156Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:55.736272Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341020022384918:3810]
2025-10-09T21:43:55.740193Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 220, pipe: [1:7559341020022384918:3810]
2025-10-09T21:43:55.745160Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 221 sessionId: src_id|2908a322-4672e055-dfaf94be-ea160b6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:55.747776Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:56.111360Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:56.111904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341020022384918:3810], now have 1 active actors on pipe
2025-10-09T21:43:56.111973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:56.111980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:56.118085Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:56.119021Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:56.119071Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:56.119149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 221 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2908a322-4672e055-dfaf94be-ea160b6_0
2025-10-09T21:43:56.119344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 221 sessionId: src_id|2908a322-4672e055-dfaf94be-ea160b6_0 is DEAD
2025-10-09T21:43:56.119489Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341020022384918:3810]
2025-10-09T21:43:56.122689Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2908a322-4672e055-dfaf94be-ea160b6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:56.122706Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2908a322-4672e055-dfaf94be-ea160b6_0] Write session will restart in 0.009075s
2025-10-09T21:43:56.122736Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2908a322-4672e055-dfaf94be-ea160b6_0] Write session: Do CDS request
2025-10-09T21:43:56.122743Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2908a322-4672e055-dfaf94be-ea160b6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:56.139045Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2908a322-4672e055-dfaf94be-ea160b6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:56.139646Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:56.139669Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 222
2025-10-09T21:43:56.139927Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 222 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:56.139990Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 222 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:56.139996Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=222 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:56.140001Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:56.141264Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 222 sessionId:  describe result for acl check
2025-10-09T21:43:56.141319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:56.141321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:56.141323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:56.141329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341024317352246:3960] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:56.141951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341024317352246:3960] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:56.152999Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341024317352246:3960] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:56.167662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341024317352246:3960] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:56.167688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341024317352246:3960] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:56.167692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341024317352246:3960] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:56.167701Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 222 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:56.198503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:56.204957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:56.205212Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:56.205216Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:56.205229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:56.205285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:56.205330Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341023955542536:3685]
2025-10-09T21:43:56.205909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:56.206978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:56.207039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:56.207418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:56.207466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:56.207467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:56.207469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:56.207470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:56.207472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:56.207473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:56.207475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.207476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.207477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.207484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:56.207496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:56.207500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:56.207505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:56.207509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:56.207514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:56.207518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.207522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.207527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.207534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:56.207543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:56.207549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:56.207555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:56.207562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:56.207568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:56.207572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:56.207578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:56.207583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:56.207589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:56.207593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:56.207599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:56.207604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.207610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.207615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.207620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.207625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.207630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.207662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:56.212220Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341024317352521:3810]
2025-10-09T21:43:56.219096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:56.220571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:56.220589Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:56.220592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:56.220596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:56.220604Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 221 [2:7559341023955542536:3685]
2025-10-09T21:43:56.220621Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:56.220642Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:56.220645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.220649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.220651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.220670Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:56.220677Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:56.220684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:56.220754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.220757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.220759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.220762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:56.220855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:56.220858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.220873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.220887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.220906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:56.220926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341023955542531:3685]
2025-10-09T21:43:56.220929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341023955542531:3685]
2025-10-09T21:43:56.220931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341023955542531:3685]
2025-10-09T21:43:56.220945Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:56.220963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:56.220980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:56.220983Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:56.221029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341023955542531:3685]
2025-10-09T21:43:56.221031Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:56.222082Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 221, pipe: [1:7559341024317352521:3810]
2025-10-09T21:43:56.222126Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 221
2025-10-09T21:43:56.222148Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 221
2025-10-09T21:43:56.221253Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:56.221259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:56.221262Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:56.221267Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:56.227362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341024317352521:3810], now have 1 active actors on pipe
2025-10-09T21:43:56.227382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341024317352367:3960], now have 1 active actors on pipe
2025-10-09T21:43:56.227386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341024317352231:2432], now have 1 active actors on pipe
2025-10-09T21:43:56.227418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:56.227426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:56.227483Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ca7626fb-5ec23ec6-afb03cc1-7e17be9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:56.227498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.227512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.227516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:56.227532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.227553Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:56.227578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:56.229771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:56.229789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:56.229822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:56.230023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 222 partition: 0 MaxSeqNo: 25 sessionId: src_id|ca7626fb-5ec23ec6-afb03cc1-7e17be9_0
2025-10-09T21:43:56.230786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2908a322-4672e055-dfaf94be-ea160b6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ca7626fb-5ec23ec6-afb03cc1-7e17be9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:56.234002Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca7626fb-5ec23ec6-afb03cc1-7e17be9_0] Write session: try to update token
2025-10-09T21:43:56.235893Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca7626fb-5ec23ec6-afb03cc1-7e17be9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:56.250182Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:56.250198Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:56.250201Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:56.250203Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:56.250213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341023955542531:3685]
2025-10-09T21:43:56.250219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341023955542531:3685]
2025-10-09T21:43:56.250222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341023955542531:3685]
2025-10-09T21:43:56.250270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:56.250278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.250281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:56.250432Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:56.259120Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:56.259134Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:56.259139Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:56.298459Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 222 sessionId: src_id|ca7626fb-5ec23ec6-afb03cc1-7e17be9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:56.301572Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:56.488343Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:56.488676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:56.488689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:56.489794Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:56.490524Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341024317352521:3810]
2025-10-09T21:43:56.490564Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:56.490567Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:56.490627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 222 reason: error at write: Disconnected sessionId: src_id|ca7626fb-5ec23ec6-afb03cc1-7e17be9_0
2025-10-09T21:43:56.490818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 222 sessionId: src_id|ca7626fb-5ec23ec6-afb03cc1-7e17be9_0 is DEAD
2025-10-09T21:43:56.490955Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:56.493313Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca7626fb-5ec23ec6-afb03cc1-7e17be9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:56.493327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca7626fb-5ec23ec6-afb03cc1-7e17be9_0] Write session will restart in 0.009529s
2025-10-09T21:43:56.493353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca7626fb-5ec23ec6-afb03cc1-7e17be9_0] Write session: Do CDS request
2025-10-09T21:43:56.493362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca7626fb-5ec23ec6-afb03cc1-7e17be9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:56.504752Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca7626fb-5ec23ec6-afb03cc1-7e17be9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:56.505382Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:56.505398Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 223
2025-10-09T21:43:56.506864Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 223 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:56.506915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 223 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:56.506921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=223 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:56.506925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:56.507636Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 223 sessionId:  describe result for acl check
2025-10-09T21:43:56.507684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:56.507686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:56.507688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:56.507694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341024317352559:3972] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:56.508218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341024317352559:3972] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:56.521826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341024317352559:3972] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:56.533712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341024317352559:3972] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:56.533733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341024317352559:3972] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:56.533737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341024317352559:3972] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:56.533753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 223 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:56.556437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:56.556813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:56.557018Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:56.557022Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:56.557036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:56.557095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:56.557141Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341023955542822:3689]
2025-10-09T21:43:56.557728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:56.562516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:56.562626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:56.562997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:56.563055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:56.563056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:56.563057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:56.563059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:56.563060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:56.563061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:56.563062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.563063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.563064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.563070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:56.563080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:56.563084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:56.563088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:56.563091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:56.563095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:56.563098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.563101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.563104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.563115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:56.563123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:56.563129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:56.563134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:56.563138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:56.563156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:56.563160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:56.563165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:56.563169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:56.563173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:56.563177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:56.563181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:56.563185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.563190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.563194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.563198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.563202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.563207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.563233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:56.572961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:56.574264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:56.574272Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:56.574279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:56.574284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:56.574293Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 222 [2:7559341023955542822:3689]
2025-10-09T21:43:56.574310Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:56.574334Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:56.574337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.574340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.574343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.574363Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:56.574370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:56.574378Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:56.574467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.574469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.574471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.574475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:56.574564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:56.574567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.574580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.574590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.574607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:56.574630Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341023955542816:3689]
2025-10-09T21:43:56.574633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341023955542816:3689]
2025-10-09T21:43:56.574638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341023955542816:3689]
2025-10-09T21:43:56.574654Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:56.574669Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:56.574685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:56.574688Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:56.574733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341023955542816:3689]
2025-10-09T21:43:56.574736Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:56.577030Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 222
2025-10-09T21:43:56.577074Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 222
2025-10-09T21:43:56.575243Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:56.575253Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:56.575256Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:56.575263Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:56.582023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341024317352545:2432], now have 1 active actors on pipe
2025-10-09T21:43:56.582086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341024317352716:3972], now have 1 active actors on pipe
2025-10-09T21:43:56.582132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:56.582141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:56.582181Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1c618582-9d7cb5b3-f686e26c-7b6dd020_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:56.582195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.582200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.582203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:56.582218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.582240Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:56.582273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:56.583273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:56.583292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:56.583311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:56.583500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 223 partition: 0 MaxSeqNo: 25 sessionId: src_id|1c618582-9d7cb5b3-f686e26c-7b6dd020_0
2025-10-09T21:43:56.588386Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca7626fb-5ec23ec6-afb03cc1-7e17be9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1c618582-9d7cb5b3-f686e26c-7b6dd020_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:56.591234Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c618582-9d7cb5b3-f686e26c-7b6dd020_0] Write session: try to update token
2025-10-09T21:43:56.591258Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c618582-9d7cb5b3-f686e26c-7b6dd020_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:56.594670Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341024317352826:3810]
2025-10-09T21:43:56.602682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341024317352826:3810], now have 1 active actors on pipe
2025-10-09T21:43:56.603457Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 222, pipe: [1:7559341024317352826:3810]
2025-10-09T21:43:56.607229Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:56.607247Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:56.607250Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:56.607252Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:56.607261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341023955542816:3689]
2025-10-09T21:43:56.607265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341023955542816:3689]
2025-10-09T21:43:56.607268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341023955542816:3689]
2025-10-09T21:43:56.607311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:56.607317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.607320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:56.607461Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:56.614402Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:56.614431Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:56.614435Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:56.631940Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 223 sessionId: src_id|1c618582-9d7cb5b3-f686e26c-7b6dd020_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:56.634376Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:56.866348Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:56.867241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:56.867261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:56.869082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c618582-9d7cb5b3-f686e26c-7b6dd020_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:56.869096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c618582-9d7cb5b3-f686e26c-7b6dd020_0] Write session will restart in 0.006137s
2025-10-09T21:43:56.869127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c618582-9d7cb5b3-f686e26c-7b6dd020_0] Write session: Do CDS request
2025-10-09T21:43:56.869134Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c618582-9d7cb5b3-f686e26c-7b6dd020_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:56.868433Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:56.868481Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341024317352826:3810]
2025-10-09T21:43:56.868501Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:56.868504Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:56.868556Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 223 reason: error at write: Disconnected sessionId: src_id|1c618582-9d7cb5b3-f686e26c-7b6dd020_0
2025-10-09T21:43:56.868737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 223 sessionId: src_id|1c618582-9d7cb5b3-f686e26c-7b6dd020_0 is DEAD
2025-10-09T21:43:56.869737Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:56.877053Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c618582-9d7cb5b3-f686e26c-7b6dd020_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:56.877578Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:56.877593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 224
2025-10-09T21:43:56.877805Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 224 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:56.877837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 224 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:56.877844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=224 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:56.877848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:56.878262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 224 sessionId:  describe result for acl check
2025-10-09T21:43:56.878296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:56.878298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:56.878300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:56.878305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341024317352855:3979] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:56.878926Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341024317352855:3979] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:56.895332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341024317352855:3979] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:56.910654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341024317352855:3979] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:56.910677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341024317352855:3979] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:56.910682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341024317352855:3979] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:56.910694Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 224 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:56.940194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:56.940596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:56.940794Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:56.940798Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:56.940811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:56.940856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:56.940895Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341023955543104:3692]
2025-10-09T21:43:56.941561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:56.944689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:56.944780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:56.945173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:56.945239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:56.945240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:56.945242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:56.945243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:56.945244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:56.945245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:56.945246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.945248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.945253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.945278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:56.945288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:56.945292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:56.945297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:56.945300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:56.945304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:56.945309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.945313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.945316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.945325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:56.945335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:56.945341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:56.945347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:56.945352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:56.945358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:56.945362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:56.945368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:56.945373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:56.945378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:56.945381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:56.945386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:56.945391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.945396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.945401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.945406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.945413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.945419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.945452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:56.965680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:56.968565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:56.968586Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:56.968589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:56.968593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:56.968604Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 223 [2:7559341023955543104:3692]
2025-10-09T21:43:56.968620Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:56.968645Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:56.968648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.968652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.968654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.968677Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:56.968685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:56.968693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:56.968766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.968768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.968770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.968773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:56.968858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:56.968861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:56.968876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:56.968886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:56.968903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:56.968926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341023955543098:3692]
2025-10-09T21:43:56.968928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341023955543098:3692]
2025-10-09T21:43:56.968930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341023955543098:3692]
2025-10-09T21:43:56.968945Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:56.968959Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:56.968975Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:56.968978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:56.969025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341023955543098:3692]
2025-10-09T21:43:56.969028Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:56.970322Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 223
2025-10-09T21:43:56.970362Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 223
2025-10-09T21:43:56.970386Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341024317353123:3810]
2025-10-09T21:43:56.969487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:56.969493Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:56.969496Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:56.969504Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:56.971773Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 223, pipe: [1:7559341024317353123:3810]
2025-10-09T21:43:56.976202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341024317352977:3979], now have 1 active actors on pipe
2025-10-09T21:43:56.976219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341024317352841:2432], now have 1 active actors on pipe
2025-10-09T21:43:56.976244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:56.976254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:56.976289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341024317353123:3810], now have 1 active actors on pipe
2025-10-09T21:43:56.976317Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b470145b-62fcccb3-500a19e3-38f593fa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:56.976345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.976352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.976355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:56.976371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.976399Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:56.976422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:56.977085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:56.977095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:56.977130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:56.977344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 224 partition: 0 MaxSeqNo: 25 sessionId: src_id|b470145b-62fcccb3-500a19e3-38f593fa_0
2025-10-09T21:43:56.977910Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c618582-9d7cb5b3-f686e26c-7b6dd020_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b470145b-62fcccb3-500a19e3-38f593fa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:56.980532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b470145b-62fcccb3-500a19e3-38f593fa_0] Write session: try to update token
2025-10-09T21:43:56.980561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b470145b-62fcccb3-500a19e3-38f593fa_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:57.033596Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 224 sessionId: src_id|b470145b-62fcccb3-500a19e3-38f593fa_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:57.037671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:57.037694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:57.037738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:57.036296Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:57.038319Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:57.045783Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:57.045803Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:57.045805Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:57.045807Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:57.045816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341023955543098:3692]
2025-10-09T21:43:57.045821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341023955543098:3692]
2025-10-09T21:43:57.045825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341023955543098:3692]
2025-10-09T21:43:57.045874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:57.045881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:57.045883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:57.046018Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:57.050840Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:57.050856Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:57.050859Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:57.311593Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:57.321196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:57.321778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:57.321970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:57.322047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:57.322111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:57.322176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:57.322461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:57.322467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:57.322540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:57.322601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:57.322668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:57.322735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:57.322775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:57.322778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:57.322848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:57.322903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:57.322959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:57.323023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:57.323063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:57.323066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:57.323142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:57.323521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:57.323594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:57.323664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:57.323707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:57.323711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:57.323782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:57.323847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:57.323915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:57.323984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:57.324038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:57.324041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:57.324116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:57.324179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:57.324237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:57.324301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:57.324369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:57.324372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:57.324440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:57.324502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:57.324574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:57.324632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:57.324672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:57.324675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:57.324745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:57.324804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:57.324870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:57.325279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:57.325338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:57.325341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:57.325425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:57.325490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:57.325555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:57.325622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:57.325664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:57.325667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:57.325738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:57.325802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:57.325867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:57.325934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:57.325976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:57.325981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:57.328004Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341024317353123:3810]
2025-10-09T21:43:57.328040Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:57.328081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 224 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b470145b-62fcccb3-500a19e3-38f593fa_0
2025-10-09T21:43:57.328274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 224 sessionId: src_id|b470145b-62fcccb3-500a19e3-38f593fa_0 is DEAD
2025-10-09T21:43:57.328439Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:57.327060Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:57.334186Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b470145b-62fcccb3-500a19e3-38f593fa_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:57.334200Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b470145b-62fcccb3-500a19e3-38f593fa_0] Write session will restart in 0.008779s
2025-10-09T21:43:57.334225Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b470145b-62fcccb3-500a19e3-38f593fa_0] Write session: Do CDS request
2025-10-09T21:43:57.334232Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b470145b-62fcccb3-500a19e3-38f593fa_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:57.347177Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b470145b-62fcccb3-500a19e3-38f593fa_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:57.349180Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:57.349199Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 225
2025-10-09T21:43:57.352548Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 225 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:57.352604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 225 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:57.352610Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=225 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:57.352614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:57.358647Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 225 sessionId:  describe result for acl check
2025-10-09T21:43:57.358692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:57.358694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:57.358699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:57.358707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341028612320518:3987] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:57.359299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341028612320518:3987] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:57.389022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341028612320518:3987] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:57.406148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341028612320518:3987] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:57.406170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341028612320518:3987] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:57.406174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341028612320518:3987] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:57.406184Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 225 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:57.415948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:57.419377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:57.419718Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:57.419723Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:57.419737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:57.419803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:57.419842Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341028250510723:3705]
2025-10-09T21:43:57.420443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:57.420986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:57.421038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:57.421377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:57.421427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:57.421428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:57.421429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:57.421430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:57.421432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:57.421433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:57.421435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:57.421436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:57.421437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:57.421443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:57.421456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:57.421460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:57.421464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:57.421468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:57.421472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:57.421475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:57.421480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:57.421484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:57.421492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:57.421500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:57.421506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:57.421512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:57.421516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:57.421523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:57.421527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:57.421533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:57.421538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:57.421543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:57.421548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:57.421554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:57.421559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:57.421564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:57.421569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:57.421574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:57.421581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:57.421586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:57.421616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:57.428226Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341028612320742:3810]
2025-10-09T21:43:57.432207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:57.433707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:57.433716Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:57.433718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:57.433722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:57.433730Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 224 [2:7559341028250510723:3705]
2025-10-09T21:43:57.433747Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:57.433770Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:57.433774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.433777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.433780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.433800Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:57.433810Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:57.433819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:57.433889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:57.433891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:57.433893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:57.433896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:57.433981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:57.433983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:57.433996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:57.434008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:57.434024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:57.434060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341028250510709:3705]
2025-10-09T21:43:57.434063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341028250510709:3705]
2025-10-09T21:43:57.434064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341028250510709:3705]
2025-10-09T21:43:57.434079Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:57.434098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:57.434113Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:57.434116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:57.434163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341028250510709:3705]
2025-10-09T21:43:57.434165Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:57.435513Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 224
2025-10-09T21:43:57.435571Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 224, pipe: [1:7559341028612320742:3810]
2025-10-09T21:43:57.435605Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 224
2025-10-09T21:43:57.434668Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:57.434678Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:57.434680Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:57.434687Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:57.436188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341028612320433:2432], now have 1 active actors on pipe
2025-10-09T21:43:57.436198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341028612320742:3810], now have 1 active actors on pipe
2025-10-09T21:43:57.436201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341028612320722:3987], now have 1 active actors on pipe
2025-10-09T21:43:57.436216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:57.436223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:57.436269Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ef492c38-f1d5909c-91bd768f-9da811dc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:57.436281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.436286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.436289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:57.436303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.436322Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:57.436351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:57.438205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:57.438221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:57.438252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:57.440628Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 225 partition: 0 MaxSeqNo: 25 sessionId: src_id|ef492c38-f1d5909c-91bd768f-9da811dc_0
2025-10-09T21:43:57.442712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b470145b-62fcccb3-500a19e3-38f593fa_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ef492c38-f1d5909c-91bd768f-9da811dc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:57.445412Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef492c38-f1d5909c-91bd768f-9da811dc_0] Write session: try to update token
2025-10-09T21:43:57.445433Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef492c38-f1d5909c-91bd768f-9da811dc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:57.457355Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:57.457374Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:57.457377Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:57.457379Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:57.457390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341028250510709:3705]
2025-10-09T21:43:57.457394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341028250510709:3705]
2025-10-09T21:43:57.457398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341028250510709:3705]
2025-10-09T21:43:57.457445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:57.457453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:57.457456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:57.457613Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:57.459973Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:57.459981Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:57.459985Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:57.502338Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 225 sessionId: src_id|ef492c38-f1d5909c-91bd768f-9da811dc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:57.505036Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:57.667315Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:57.667805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:57.667816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:57.668759Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:57.668809Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341028612320742:3810]
2025-10-09T21:43:57.668827Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:57.668830Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:57.668888Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 225 reason: error at write: Disconnected sessionId: src_id|ef492c38-f1d5909c-91bd768f-9da811dc_0
2025-10-09T21:43:57.669074Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 225 sessionId: src_id|ef492c38-f1d5909c-91bd768f-9da811dc_0 is DEAD
2025-10-09T21:43:57.671944Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef492c38-f1d5909c-91bd768f-9da811dc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:57.671956Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef492c38-f1d5909c-91bd768f-9da811dc_0] Write session will restart in 0.006443s
2025-10-09T21:43:57.671982Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef492c38-f1d5909c-91bd768f-9da811dc_0] Write session: Do CDS request
2025-10-09T21:43:57.671989Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef492c38-f1d5909c-91bd768f-9da811dc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:57.671639Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:57.679758Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef492c38-f1d5909c-91bd768f-9da811dc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:57.681040Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:57.681058Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 226
2025-10-09T21:43:57.682647Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 226 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:57.682701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 226 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:57.682707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=226 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:57.682719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:57.683255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 226 sessionId:  describe result for acl check
2025-10-09T21:43:57.683297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:57.683299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:57.683301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:57.683307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341028612320772:3995] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:57.683808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341028612320772:3995] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:57.694565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341028612320772:3995] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:57.707384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341028612320772:3995] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:57.707404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341028612320772:3995] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:57.707408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341028612320772:3995] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:57.707418Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 226 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:57.735213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:57.735540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:57.735708Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:57.735711Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:57.735723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:57.735761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:57.735796Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341028250511004:3709]
2025-10-09T21:43:57.736339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:57.736931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:57.736970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:57.737253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:57.737292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:57.737293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:57.737294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:57.737295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:57.737296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:57.737298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:57.737299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:57.737300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:57.737301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:57.737306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:57.737315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:57.737318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:57.737322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:57.737325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:57.737329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:57.737332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:57.737335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:57.737339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:57.737346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:57.737353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:57.737358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:57.737362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:57.737366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:57.737371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:57.737375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:57.737380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:57.737385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:57.737389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:57.737392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:57.737396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:57.737400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:57.737404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:57.737408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:57.737412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:57.737416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:57.737420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:57.737447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:57.746917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:57.748074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:57.748081Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:57.748084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:57.748090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:57.748101Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 225 [2:7559341028250511004:3709]
2025-10-09T21:43:57.748117Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:57.748138Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:57.748142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.748145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.748148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.748166Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:57.748173Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:57.748181Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:57.748245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:57.748248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:57.748251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:57.748254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:57.748338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:57.748341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:57.748353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:57.748363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:57.748384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:57.748404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341028250510998:3709]
2025-10-09T21:43:57.748407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341028250510998:3709]
2025-10-09T21:43:57.748409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341028250510998:3709]
2025-10-09T21:43:57.748425Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:57.748440Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:57.748456Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:57.748459Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:57.748502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341028250510998:3709]
2025-10-09T21:43:57.748504Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:57.748956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:57.748965Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:57.748967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:57.748973Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:57.752128Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 225
2025-10-09T21:43:57.752162Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 225
2025-10-09T21:43:57.752619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341028612320887:3995], now have 1 active actors on pipe
2025-10-09T21:43:57.752632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341028612320751:2432], now have 1 active actors on pipe
2025-10-09T21:43:57.752653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:57.752661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:57.752698Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5004f015-f592bb2a-88cf6a8b-7f91b33d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:57.752712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.752717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.752721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:57.752735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.752754Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:57.752776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:57.753110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:57.753118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:57.753134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:57.753287Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 226 partition: 0 MaxSeqNo: 25 sessionId: src_id|5004f015-f592bb2a-88cf6a8b-7f91b33d_0
2025-10-09T21:43:57.753741Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef492c38-f1d5909c-91bd768f-9da811dc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5004f015-f592bb2a-88cf6a8b-7f91b33d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:57.756486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5004f015-f592bb2a-88cf6a8b-7f91b33d_0] Write session: try to update token
2025-10-09T21:43:57.756509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5004f015-f592bb2a-88cf6a8b-7f91b33d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:57.770285Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341028612321036:3810]
2025-10-09T21:43:57.770797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341028612321036:3810], now have 1 active actors on pipe
2025-10-09T21:43:57.770943Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 225, pipe: [1:7559341028612321036:3810]
2025-10-09T21:43:57.813461Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 226 sessionId: src_id|5004f015-f592bb2a-88cf6a8b-7f91b33d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:57.816101Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:57.820891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:57.820915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:57.820962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:57.822336Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:57.826318Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:57.826336Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:57.826338Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:57.826340Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:57.826350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341028250510998:3709]
2025-10-09T21:43:57.826356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341028250510998:3709]
2025-10-09T21:43:57.826359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341028250510998:3709]
2025-10-09T21:43:57.826406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:57.826412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:57.826431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:57.826573Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:57.831031Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:57.831042Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:57.831046Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:58.053727Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:58.063656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:58.063687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:58.063860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:58.063921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:58.063989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:58.064055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:58.064102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:58.064108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:58.064177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:58.064238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:58.064305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:58.064637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:58.064680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:58.064683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:58.064753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:58.064816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:58.064878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:58.064938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:58.064980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:58.064983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:58.065081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:58.065149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:58.065218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:58.065287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:58.065329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:58.065332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:58.065405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:58.065469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:58.065535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:58.065599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:58.065646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:58.065649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:58.065716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:58.065779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:58.065836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:58.065904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:58.065950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:58.065952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:58.066375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:58.066468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:58.066535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:58.066597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:58.066641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:58.066645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:58.066713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:58.066779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:58.066844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:58.066910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:58.066978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:58.066981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:58.067051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:58.067139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:58.067215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:58.067280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:58.067327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:58.067330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:58.067397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:43:58.067456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:43:58.067521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:43:58.067586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:43:58.067630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:43:58.067634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:43:58.069309Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:58.070043Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341028612321036:3810]
2025-10-09T21:43:58.070083Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:58.070086Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:58.070114Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 226 reason: error at write: Disconnected sessionId: src_id|5004f015-f592bb2a-88cf6a8b-7f91b33d_0
2025-10-09T21:43:58.070308Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 226 sessionId: src_id|5004f015-f592bb2a-88cf6a8b-7f91b33d_0 is DEAD
2025-10-09T21:43:58.070464Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:58.071548Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5004f015-f592bb2a-88cf6a8b-7f91b33d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:58.071561Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5004f015-f592bb2a-88cf6a8b-7f91b33d_0] Write session will restart in 0.007950s
2025-10-09T21:43:58.071586Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5004f015-f592bb2a-88cf6a8b-7f91b33d_0] Write session: Do CDS request
2025-10-09T21:43:58.071594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5004f015-f592bb2a-88cf6a8b-7f91b33d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:58.081151Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5004f015-f592bb2a-88cf6a8b-7f91b33d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:58.083514Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:58.083545Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 227
2025-10-09T21:43:58.083737Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 227 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:58.083779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 227 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:58.083784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=227 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:58.083797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:58.085855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 227 sessionId:  describe result for acl check
2025-10-09T21:43:58.085894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:58.085896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:58.085898Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:58.085904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341032907288358:4000] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:58.086523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341032907288358:4000] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:58.102376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341032907288358:4000] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:58.122889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341032907288358:4000] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:58.122913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341032907288358:4000] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:58.122918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341032907288358:4000] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:58.122927Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 227 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:58.159984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:58.161541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:58.161768Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:58.161772Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:58.161785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:58.161844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:58.161883Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341032545478600:3713]
2025-10-09T21:43:58.162490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:58.162836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:58.162879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:58.163188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:58.163242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:58.163243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:58.163244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:58.163246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:58.163247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:58.163248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:58.163249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.163250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.163251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.163257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:58.163266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:58.163270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:58.163274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:58.163278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:58.163282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:58.163285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.163289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.163293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.163300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:58.163308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:58.163314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:58.163320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:58.163325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:58.163330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:58.163334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:58.163339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:58.163344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:58.163349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:58.163352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:58.163357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:58.163361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.163366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.163371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.163375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.163379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.163383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.163412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:58.164596Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341032907288632:3810]
2025-10-09T21:43:58.174195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:58.175972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:58.175979Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:58.175981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:58.175986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:58.175994Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 226 [2:7559341032545478600:3713]
2025-10-09T21:43:58.176011Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:58.176033Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:58.176037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.176040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.176041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.176060Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:58.176067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:58.176075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:58.176141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.176143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.176147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.176150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:58.176225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:58.176228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.176241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.176252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.176270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:58.176292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341032545478594:3713]
2025-10-09T21:43:58.176294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341032545478594:3713]
2025-10-09T21:43:58.176296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341032545478594:3713]
2025-10-09T21:43:58.176313Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:58.176330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:58.176345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:58.176349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:58.176389Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341032545478594:3713]
2025-10-09T21:43:58.176392Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:58.177167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:58.177178Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:58.177180Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:58.177187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:58.178140Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 226, pipe: [1:7559341032907288632:3810]
2025-10-09T21:43:58.178179Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 226
2025-10-09T21:43:58.178208Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 226
2025-10-09T21:43:58.179332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341032907288501:4000], now have 1 active actors on pipe
2025-10-09T21:43:58.179344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341032907288632:3810], now have 1 active actors on pipe
2025-10-09T21:43:58.179347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341032907288352:2432], now have 1 active actors on pipe
2025-10-09T21:43:58.179380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:58.179387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:58.179420Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d89388c4-643696a6-abfdacc-90d33159_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:58.179431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.179436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.179439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:58.179450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.179467Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:58.179495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:58.179866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:58.179874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:58.179900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:58.180188Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 227 partition: 0 MaxSeqNo: 25 sessionId: src_id|d89388c4-643696a6-abfdacc-90d33159_0
2025-10-09T21:43:58.182248Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5004f015-f592bb2a-88cf6a8b-7f91b33d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d89388c4-643696a6-abfdacc-90d33159_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:58.184956Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d89388c4-643696a6-abfdacc-90d33159_0] Write session: try to update token
2025-10-09T21:43:58.184973Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d89388c4-643696a6-abfdacc-90d33159_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:58.221605Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:58.221629Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:58.221632Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:58.221635Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:58.221645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341032545478594:3713]
2025-10-09T21:43:58.221650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341032545478594:3713]
2025-10-09T21:43:58.221654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341032545478594:3713]
2025-10-09T21:43:58.221702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:58.221709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.221712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:58.221873Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:58.227717Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:58.227745Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:58.227749Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:58.226199Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 227 sessionId: src_id|d89388c4-643696a6-abfdacc-90d33159_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:58.229240Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:58.492450Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:58.492700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:58.492709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:58.494476Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:58.494907Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:58.494983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 227 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d89388c4-643696a6-abfdacc-90d33159_0
2025-10-09T21:43:58.495151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 227 sessionId: src_id|d89388c4-643696a6-abfdacc-90d33159_0 is DEAD
2025-10-09T21:43:58.495273Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341032907288632:3810]
2025-10-09T21:43:58.495285Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:58.501151Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d89388c4-643696a6-abfdacc-90d33159_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:58.501165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d89388c4-643696a6-abfdacc-90d33159_0] Write session will restart in 0.009103s
2025-10-09T21:43:58.501191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d89388c4-643696a6-abfdacc-90d33159_0] Write session: Do CDS request
2025-10-09T21:43:58.501198Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d89388c4-643696a6-abfdacc-90d33159_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:58.515802Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d89388c4-643696a6-abfdacc-90d33159_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:58.518574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:58.518590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 228
2025-10-09T21:43:58.522267Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 228 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:58.522315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 228 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:58.522321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=228 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:58.522326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:58.523924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 228 sessionId:  describe result for acl check
2025-10-09T21:43:58.523962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:58.523964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:58.523965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:58.523971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341032907288717:4012] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:58.524513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341032907288717:4012] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:58.532935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341032907288717:4012] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:58.550807Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341032907288717:4012] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:58.550826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341032907288717:4012] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:58.550830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341032907288717:4012] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:58.550840Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 228 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:58.566097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:58.572300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:58.572528Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:58.572532Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:58.572542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:58.572587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:58.572623Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341032545478910:3725]
2025-10-09T21:43:58.573173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:58.573588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:58.573623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:58.573896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:58.573930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:58.573931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:58.573932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:58.573933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:58.573934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:58.573935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:58.573936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.573937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.573938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.573943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:58.573953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:58.573957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:58.573960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:58.573963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:58.573967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:58.573970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.573973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.573976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.573983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:58.573990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:58.573995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:58.574000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:58.574003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:58.574007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:58.574011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:58.574015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:58.574020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:58.574023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:58.574043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:58.574048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:58.574052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.574056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.574060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.574064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.574068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.574072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.574097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:58.582509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:58.583635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:58.583641Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:58.583643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:58.583648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:58.583655Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 227 [2:7559341032545478910:3725]
2025-10-09T21:43:58.583669Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:58.583689Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:58.583692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.583695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.583697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.583716Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:58.583722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:58.583729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:58.583786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.583789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.583790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.583792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:58.583856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:58.583858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.583871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.583880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.583894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:58.583913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341032545478904:3725]
2025-10-09T21:43:58.583915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341032545478904:3725]
2025-10-09T21:43:58.583917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341032545478904:3725]
2025-10-09T21:43:58.583926Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:58.583939Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:58.583951Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:58.583954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:58.583990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341032545478904:3725]
2025-10-09T21:43:58.583992Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:58.584257Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:58.584264Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:58.584266Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:58.584271Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:58.584881Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 227
2025-10-09T21:43:58.584916Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 227
2025-10-09T21:43:58.586329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341032907288661:2432], now have 1 active actors on pipe
2025-10-09T21:43:58.586342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341032907288907:4012], now have 1 active actors on pipe
2025-10-09T21:43:58.590120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:58.590137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:58.590177Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ab62ef3a-1af9d9f7-cb930b44-d0f34603_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:58.590191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.590195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.590198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:58.590213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.590232Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:58.590251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:58.590771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:58.590782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:58.590801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:58.590927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 228 partition: 0 MaxSeqNo: 25 sessionId: src_id|ab62ef3a-1af9d9f7-cb930b44-d0f34603_0
2025-10-09T21:43:58.593159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d89388c4-643696a6-abfdacc-90d33159_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ab62ef3a-1af9d9f7-cb930b44-d0f34603_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:58.594849Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341032907288957:3810]
2025-10-09T21:43:58.595809Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ab62ef3a-1af9d9f7-cb930b44-d0f34603_0] Write session: try to update token
2025-10-09T21:43:58.595825Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ab62ef3a-1af9d9f7-cb930b44-d0f34603_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:58.595945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341032907288957:3810], now have 1 active actors on pipe
2025-10-09T21:43:58.596095Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 227, pipe: [1:7559341032907288957:3810]
2025-10-09T21:43:58.610277Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:58.610294Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:58.610296Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:58.610298Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:58.610309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341032545478904:3725]
2025-10-09T21:43:58.610314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341032545478904:3725]
2025-10-09T21:43:58.610318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341032545478904:3725]
2025-10-09T21:43:58.610365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:58.610374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.610376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:58.610529Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:58.616215Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:58.616225Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:58.616229Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:58.637850Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 228 sessionId: src_id|ab62ef3a-1af9d9f7-cb930b44-d0f34603_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:58.640687Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:58.872027Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:58.872670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:58.872681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:58.874780Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341032907288957:3810]
2025-10-09T21:43:58.874803Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:58.874833Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:58.874835Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:58.874887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 228 reason: error at write: Disconnected sessionId: src_id|ab62ef3a-1af9d9f7-cb930b44-d0f34603_0
2025-10-09T21:43:58.875061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 228 sessionId: src_id|ab62ef3a-1af9d9f7-cb930b44-d0f34603_0 is DEAD
2025-10-09T21:43:58.875556Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:58.878249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab62ef3a-1af9d9f7-cb930b44-d0f34603_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:58.878261Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab62ef3a-1af9d9f7-cb930b44-d0f34603_0] Write session will restart in 0.008194s
2025-10-09T21:43:58.878287Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab62ef3a-1af9d9f7-cb930b44-d0f34603_0] Write session: Do CDS request
2025-10-09T21:43:58.878297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab62ef3a-1af9d9f7-cb930b44-d0f34603_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:58.893658Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ab62ef3a-1af9d9f7-cb930b44-d0f34603_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:58.895311Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:58.895327Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 229
2025-10-09T21:43:58.895563Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 229 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:58.895598Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 229 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:58.895604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=229 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:58.895608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:58.896031Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 229 sessionId:  describe result for acl check
2025-10-09T21:43:58.896061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:58.896063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:58.896065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:58.896070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341032907288985:4017] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:58.896677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341032907288985:4017] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:58.913389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341032907288985:4017] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:58.927783Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341032907288985:4017] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:58.927799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341032907288985:4017] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:58.927803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341032907288985:4017] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:58.927818Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 229 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:58.946572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:58.950946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:58.951188Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:58.951193Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:58.951206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:58.951263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:58.951307Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341032545479197:3730]
2025-10-09T21:43:58.952237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:58.952660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:58.952714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:58.953042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:58.953098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:58.953099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:58.953101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:58.953102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:58.953104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:58.953105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:58.953107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.953108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.953109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.953115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:58.953126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:58.953130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:58.953134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:58.953138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:58.953142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:58.953146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.953149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.953154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.953162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:58.953170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:58.953177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:58.953183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:58.953188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:58.953192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:58.953196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:58.953201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:58.953206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:58.953212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:58.953217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:58.953222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:58.953227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.953232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.953239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.953245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.953249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.953255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.953283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:58.963980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:58.965580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:58.965588Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:58.965591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:58.965595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:58.965603Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 228 [2:7559341032545479197:3730]
2025-10-09T21:43:58.965618Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:58.965639Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:58.965643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.965646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.965648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.965666Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:58.965673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:58.965683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:58.965752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.965754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.965756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.965759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:58.965839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:58.965842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:58.965855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:58.965866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:58.965881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:58.965901Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341032545479191:3730]
2025-10-09T21:43:58.965904Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341032545479191:3730]
2025-10-09T21:43:58.965906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341032545479191:3730]
2025-10-09T21:43:58.965924Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:58.965939Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:58.965954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:58.965957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:58.965999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341032545479191:3730]
2025-10-09T21:43:58.966001Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:58.966531Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:58.966540Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:58.966542Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:58.966548Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:58.967537Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 228
2025-10-09T21:43:58.967577Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 228
2025-10-09T21:43:58.972109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341032907288970:2432], now have 1 active actors on pipe
2025-10-09T21:43:58.972123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341032907289156:4017], now have 1 active actors on pipe
2025-10-09T21:43:58.972148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:58.972155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:58.972189Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3ee4789a-727275f4-8e498523-8031143e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:58.972202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.972207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.972210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:58.972223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.972240Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:58.972258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:58.972585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:58.972590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:58.972608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:58.972754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 229 partition: 0 MaxSeqNo: 25 sessionId: src_id|3ee4789a-727275f4-8e498523-8031143e_0
2025-10-09T21:43:58.977078Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab62ef3a-1af9d9f7-cb930b44-d0f34603_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3ee4789a-727275f4-8e498523-8031143e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:58.979867Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3ee4789a-727275f4-8e498523-8031143e_0] Write session: try to update token
2025-10-09T21:43:58.979887Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3ee4789a-727275f4-8e498523-8031143e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:58.979119Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341032907289262:3810]
2025-10-09T21:43:58.989329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341032907289262:3810], now have 1 active actors on pipe
2025-10-09T21:43:58.989568Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 228, pipe: [1:7559341032907289262:3810]
2025-10-09T21:43:59.005752Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:59.005771Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:59.005774Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:59.005776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:59.005787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341032545479191:3730]
2025-10-09T21:43:59.005792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341032545479191:3730]
2025-10-09T21:43:59.005797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341032545479191:3730]
2025-10-09T21:43:59.005845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:59.005852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:59.005855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:59.006024Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:59.010546Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:59.010561Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:59.010578Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:59.010070Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 229 sessionId: src_id|3ee4789a-727275f4-8e498523-8031143e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:59.012933Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:59.206368Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:59.206948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:59.206961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:59.209956Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:59.211127Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:59.211179Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341032907289262:3810]
2025-10-09T21:43:59.211202Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:43:59.211274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 229 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3ee4789a-727275f4-8e498523-8031143e_0
2025-10-09T21:43:59.211458Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 229 sessionId: src_id|3ee4789a-727275f4-8e498523-8031143e_0 is DEAD
2025-10-09T21:43:59.211907Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ee4789a-727275f4-8e498523-8031143e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:43:59.211917Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ee4789a-727275f4-8e498523-8031143e_0] Write session will restart in 0.008049s
2025-10-09T21:43:59.211942Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ee4789a-727275f4-8e498523-8031143e_0] Write session: Do CDS request
2025-10-09T21:43:59.211949Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ee4789a-727275f4-8e498523-8031143e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:59.223222Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3ee4789a-727275f4-8e498523-8031143e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:59.223622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:59.223644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 230
2025-10-09T21:43:59.224012Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 230 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:59.224050Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 230 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:59.224056Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=230 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:59.224059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:59.225130Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 230 sessionId:  describe result for acl check
2025-10-09T21:43:59.225174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:59.225182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:59.225183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:59.225190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341037202256576:4021] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:59.225750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341037202256576:4021] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:59.229551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341037202256576:4021] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:59.240156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341037202256576:4021] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:59.240174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341037202256576:4021] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:59.240178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341037202256576:4021] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:59.240188Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 230 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:59.280083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:59.294812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:59.295140Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:59.295145Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:59.295158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:59.295221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:59.295258Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341036840446793:3733]
2025-10-09T21:43:59.295889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:59.302333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:59.302440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:59.302860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:59.302917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:59.302919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:59.302920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:59.302921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:59.302923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:59.302924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:59.302925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:59.302926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:59.302928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:59.302936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:59.302947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:59.302952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:59.302957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:59.302962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:59.302966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:59.302971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:59.302978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:59.302983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:59.302992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:59.303002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:59.303009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:59.303015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:59.303020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:59.303026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:59.303031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:59.303037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:59.303042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:59.303048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:59.303053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:59.303060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:59.303067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:59.303073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:59.303078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:59.303084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:59.303089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:59.303095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:59.303127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:59.313994Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341037202256848:3810]
2025-10-09T21:43:59.315181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:59.316390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:59.316396Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:59.316399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:59.316405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:59.316412Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 229 [2:7559341036840446793:3733]
2025-10-09T21:43:59.316428Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:59.316448Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:59.316451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.316454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.316459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.316478Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:59.316487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:59.316494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:59.316563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:59.316565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:59.316567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:59.316570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:59.316870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:59.316873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:59.316888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:59.316901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:59.316921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:59.316944Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341036840446782:3733]
2025-10-09T21:43:59.316947Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341036840446782:3733]
2025-10-09T21:43:59.316949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341036840446782:3733]
2025-10-09T21:43:59.316965Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:59.316982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:59.316996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:59.316999Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:59.317036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341036840446782:3733]
2025-10-09T21:43:59.317038Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:59.318256Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 229
2025-10-09T21:43:59.318296Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 229
2025-10-09T21:43:59.317291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:59.317299Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:59.317300Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:59.317306Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:59.319031Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 229, pipe: [1:7559341037202256848:3810]
2025-10-09T21:43:59.319125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341037202256563:2432], now have 1 active actors on pipe
2025-10-09T21:43:59.319136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341037202256638:4021], now have 1 active actors on pipe
2025-10-09T21:43:59.319157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:59.319163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:59.319178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341037202256848:3810], now have 1 active actors on pipe
2025-10-09T21:43:59.319200Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9afac851-44f19a1e-d6442b44-d4401bc9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:59.319211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.319216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.319219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:59.319232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.319249Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:59.319268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:59.319566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:59.319573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:59.319588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:59.319722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 230 partition: 0 MaxSeqNo: 25 sessionId: src_id|9afac851-44f19a1e-d6442b44-d4401bc9_0
2025-10-09T21:43:59.320140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ee4789a-727275f4-8e498523-8031143e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9afac851-44f19a1e-d6442b44-d4401bc9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:59.323025Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9afac851-44f19a1e-d6442b44-d4401bc9_0] Write session: try to update token
2025-10-09T21:43:59.323043Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9afac851-44f19a1e-d6442b44-d4401bc9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:59.362142Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:59.362159Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:59.362162Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:59.362164Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:59.362174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341036840446782:3733]
2025-10-09T21:43:59.362178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341036840446782:3733]
2025-10-09T21:43:59.362183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341036840446782:3733]
2025-10-09T21:43:59.362227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:59.362234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:59.362237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:59.362610Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:59.366261Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:59.366274Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:59.366279Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:43:59.376476Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 230 sessionId: src_id|9afac851-44f19a1e-d6442b44-d4401bc9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:59.379257Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:59.739609Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:43:59.740600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:59.740620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:59.743126Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9afac851-44f19a1e-d6442b44-d4401bc9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:43:59.743139Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9afac851-44f19a1e-d6442b44-d4401bc9_0] Write session will restart in 0.006965s
2025-10-09T21:43:59.742630Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341037202256848:3810]
2025-10-09T21:43:59.743163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9afac851-44f19a1e-d6442b44-d4401bc9_0] Write session: Do CDS request
2025-10-09T21:43:59.743170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9afac851-44f19a1e-d6442b44-d4401bc9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:43:59.742657Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:43:59.742693Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:43:59.742696Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:43:59.742756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 230 reason: error at write: Disconnected sessionId: src_id|9afac851-44f19a1e-d6442b44-d4401bc9_0
2025-10-09T21:43:59.742925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 230 sessionId: src_id|9afac851-44f19a1e-d6442b44-d4401bc9_0 is DEAD
2025-10-09T21:43:59.746211Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:43:59.751561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9afac851-44f19a1e-d6442b44-d4401bc9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:59.751901Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:59.751917Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 231
2025-10-09T21:43:59.752054Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 231 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:59.752086Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 231 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:43:59.752096Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=231 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:59.752100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:59.753093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 231 sessionId:  describe result for acl check
2025-10-09T21:43:59.753126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:59.753127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:59.753129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:59.753135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341037202256899:4033] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:59.753680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341037202256899:4033] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:59.763099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341037202256899:4033] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:59.772377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341037202256899:4033] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:59.772395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341037202256899:4033] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:59.772398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341037202256899:4033] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:59.772409Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 231 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:59.827031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:59.829766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:59.830009Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:59.830014Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:43:59.830046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:59.830117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:59.830160Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341036840447108:3747]
2025-10-09T21:43:59.830769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:43:59.831843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:43:59.831902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:43:59.832291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:43:59.832357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:59.832358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:59.832360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:59.832361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:59.832362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:59.832364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:59.832365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:59.832366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:59.832368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:59.832374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:59.832387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:59.832392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:59.832397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:59.832401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:59.832406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:59.832411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:59.832415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:59.832420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:59.832428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:59.832436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:43:59.832443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:59.832450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:43:59.832455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:59.832461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:43:59.832466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:59.832472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:43:59.832477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:59.832483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:43:59.832487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:59.832493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:43:59.832499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:59.832505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:59.832510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:59.832516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:59.832521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:59.832527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:59.832560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:43:59.842947Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341037202257175:3810]
2025-10-09T21:43:59.844331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:43:59.845874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:43:59.845882Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:43:59.845885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:59.845889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:59.845897Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 230 [2:7559341036840447108:3747]
2025-10-09T21:43:59.845914Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:43:59.845934Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:43:59.845937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.845940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.845942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.845960Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:43:59.845969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:59.845978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:59.846072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:59.846073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:59.846075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:59.846078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:43:59.846154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:43:59.846156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:59.846170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:43:59.846181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:43:59.846199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:43:59.846222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341036840447102:3747]
2025-10-09T21:43:59.846225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341036840447102:3747]
2025-10-09T21:43:59.846227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341036840447102:3747]
2025-10-09T21:43:59.846240Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:43:59.846257Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:43:59.846272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:43:59.846275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:59.846316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341036840447102:3747]
2025-10-09T21:43:59.846319Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:43:59.846581Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:43:59.846590Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:43:59.846592Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:43:59.846599Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:43:59.847630Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 230
2025-10-09T21:43:59.847661Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 230
2025-10-09T21:43:59.847877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341037202256935:4033], now have 1 active actors on pipe
2025-10-09T21:43:59.847891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341037202256886:2432], now have 1 active actors on pipe
2025-10-09T21:43:59.848129Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 230, pipe: [1:7559341037202257175:3810]
2025-10-09T21:43:59.848050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:59.848060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:59.848079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341037202257175:3810], now have 1 active actors on pipe
2025-10-09T21:43:59.848099Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c15c258c-d4331f44-778f8e50-88bfb43c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:59.848109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.848122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.848124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:59.848136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.848151Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:59.848172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:59.848379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:59.848384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:59.848401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:59.853724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 231 partition: 0 MaxSeqNo: 25 sessionId: src_id|c15c258c-d4331f44-778f8e50-88bfb43c_0
2025-10-09T21:43:59.854214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9afac851-44f19a1e-d6442b44-d4401bc9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c15c258c-d4331f44-778f8e50-88bfb43c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:59.856774Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c15c258c-d4331f44-778f8e50-88bfb43c_0] Write session: try to update token
2025-10-09T21:43:59.856792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c15c258c-d4331f44-778f8e50-88bfb43c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:59.887577Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 231 sessionId: src_id|c15c258c-d4331f44-778f8e50-88bfb43c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:59.890169Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:59.888944Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:43:59.888962Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:43:59.888965Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:43:59.888967Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:43:59.888977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341036840447102:3747]
2025-10-09T21:43:59.888981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341036840447102:3747]
2025-10-09T21:43:59.888986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341036840447102:3747]
2025-10-09T21:43:59.889032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:43:59.889039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:43:59.889042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:43:59.889213Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:43:59.890790Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:43:59.890803Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:43:59.890808Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:00.273996Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:00.274727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:00.274752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:00.276113Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:00.277223Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:00.277305Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 231 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c15c258c-d4331f44-778f8e50-88bfb43c_0
2025-10-09T21:44:00.277478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 231 sessionId: src_id|c15c258c-d4331f44-778f8e50-88bfb43c_0 is DEAD
2025-10-09T21:44:00.277592Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:00.277624Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341037202257175:3810]
2025-10-09T21:44:00.277922Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c15c258c-d4331f44-778f8e50-88bfb43c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:00.277930Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c15c258c-d4331f44-778f8e50-88bfb43c_0] Write session will restart in 0.009765s
2025-10-09T21:44:00.277954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c15c258c-d4331f44-778f8e50-88bfb43c_0] Write session: Do CDS request
2025-10-09T21:44:00.277960Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c15c258c-d4331f44-778f8e50-88bfb43c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:00.290908Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c15c258c-d4331f44-778f8e50-88bfb43c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:00.291277Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:00.291294Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 232
2025-10-09T21:44:00.291619Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 232 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:00.291658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 232 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:00.291664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=232 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:00.291668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:00.292358Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 232 sessionId:  describe result for acl check
2025-10-09T21:44:00.292405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:00.292410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:00.292412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:00.292418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341041497224502:4038] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:00.293171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341041497224502:4038] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:00.299271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341041497224502:4038] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:00.316857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341041497224502:4038] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:00.316878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341041497224502:4038] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:00.316882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341041497224502:4038] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:00.316893Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 232 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:00.369384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:00.373164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:00.373576Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:00.373582Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:00.373597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:00.373687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:00.373723Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341041135414711:3751]
2025-10-09T21:44:00.374341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:00.374856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:00.374921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:00.375267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:00.375330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:00.375332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:00.375334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:00.375335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:00.375337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:00.375338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:00.375339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:00.375341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:00.375342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:00.375348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:00.375359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:00.375363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:00.375367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:00.375370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:00.375374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:00.375377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:00.375380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:00.375383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:00.375390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:00.375397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:00.375403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:00.375408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:00.375412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:00.375416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:00.375420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:00.375425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:00.375429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:00.375434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:00.375439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:00.375443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:00.375448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:00.375453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:00.375457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:00.375461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:00.375465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:00.375470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:00.375497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:00.382971Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341041497224779:3810]
2025-10-09T21:44:00.387357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:00.388860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:00.388868Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:00.388870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:00.388875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:00.388883Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 231 [2:7559341041135414711:3751]
2025-10-09T21:44:00.388899Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:00.388920Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:00.388924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.388927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.388930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.388948Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:00.388955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:00.388962Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:00.389033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:00.389035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:00.389036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:00.389039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:00.389115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:00.389118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:00.389132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:00.389143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:00.389159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:00.389181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341041135414705:3751]
2025-10-09T21:44:00.389183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341041135414705:3751]
2025-10-09T21:44:00.389186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341041135414705:3751]
2025-10-09T21:44:00.389196Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:00.389211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:00.389229Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:00.389233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:00.389274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341041135414705:3751]
2025-10-09T21:44:00.389277Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:00.389867Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:00.389878Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:00.389882Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:00.389889Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:00.390615Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 231, pipe: [1:7559341041497224779:3810]
2025-10-09T21:44:00.390651Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 231
2025-10-09T21:44:00.390678Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 231
2025-10-09T21:44:00.391097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341041497224779:3810], now have 1 active actors on pipe
2025-10-09T21:44:00.391118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341041497224491:2432], now have 1 active actors on pipe
2025-10-09T21:44:00.391121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341041497224543:4038], now have 1 active actors on pipe
2025-10-09T21:44:00.391146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:00.391151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:00.391183Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b81f1c1c-114526bf-4d32caea-3dbb734d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:00.391195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.391200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.391202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:00.391213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.391228Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:00.391245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:00.397079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:00.397099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:00.397138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:00.397292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 232 partition: 0 MaxSeqNo: 25 sessionId: src_id|b81f1c1c-114526bf-4d32caea-3dbb734d_0
2025-10-09T21:44:00.397800Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c15c258c-d4331f44-778f8e50-88bfb43c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b81f1c1c-114526bf-4d32caea-3dbb734d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:00.400429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b81f1c1c-114526bf-4d32caea-3dbb734d_0] Write session: try to update token
2025-10-09T21:44:00.400453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b81f1c1c-114526bf-4d32caea-3dbb734d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:00.438084Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:00.438102Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:00.438104Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:00.438106Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:00.438117Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341041135414705:3751]
2025-10-09T21:44:00.438122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341041135414705:3751]
2025-10-09T21:44:00.438126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341041135414705:3751]
2025-10-09T21:44:00.438175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:00.438183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:00.438186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:00.438333Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:00.441925Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 232 sessionId: src_id|b81f1c1c-114526bf-4d32caea-3dbb734d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:00.441905Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:00.441920Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:00.441924Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:00.444728Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:00.677280Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:00.677755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:00.677766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:00.679762Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:00.682972Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:00.683024Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341041497224779:3810]
2025-10-09T21:44:00.687910Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:00.688003Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 232 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b81f1c1c-114526bf-4d32caea-3dbb734d_0
2025-10-09T21:44:00.688189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 232 sessionId: src_id|b81f1c1c-114526bf-4d32caea-3dbb734d_0 is DEAD
2025-10-09T21:44:00.690455Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b81f1c1c-114526bf-4d32caea-3dbb734d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:00.690467Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b81f1c1c-114526bf-4d32caea-3dbb734d_0] Write session will restart in 0.006902s
2025-10-09T21:44:00.690491Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b81f1c1c-114526bf-4d32caea-3dbb734d_0] Write session: Do CDS request
2025-10-09T21:44:00.690498Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b81f1c1c-114526bf-4d32caea-3dbb734d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:00.702552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b81f1c1c-114526bf-4d32caea-3dbb734d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:00.706511Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:00.706533Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 233
2025-10-09T21:44:00.710188Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 233 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:00.710254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 233 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:00.710260Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=233 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:00.710266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:00.714472Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 233 sessionId:  describe result for acl check
2025-10-09T21:44:00.714523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:00.714526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:00.714528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:00.714536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341041497224834:4049] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:00.715112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341041497224834:4049] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:00.733851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341041497224834:4049] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:00.752556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341041497224834:4049] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:00.752573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341041497224834:4049] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:00.752577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341041497224834:4049] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:00.752593Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 233 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:00.779383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:00.784153Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341041497225104:3810]
2025-10-09T21:44:00.786944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:00.787196Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:00.787200Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:00.787212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:00.787279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:00.787318Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341041135415029:3763]
2025-10-09T21:44:00.787964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:00.788968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:00.789046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:00.789354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:00.789402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:00.789403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:00.789404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:00.789420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:00.789421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:00.789422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:00.789423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:00.789424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:00.789425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:00.789431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:00.789441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:00.789444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:00.789448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:00.789451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:00.789454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:00.789457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:00.789460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:00.789463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:00.789469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:00.789476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:00.789482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:00.789486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:00.789490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:00.789494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:00.789498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:00.789503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:00.789507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:00.789513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:00.789517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:00.789522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:00.789527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:00.789531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:00.789534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:00.789539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:00.789544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:00.789548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:00.789573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:00.803230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:00.804699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:00.804705Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:00.804708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:00.804713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:00.804720Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 232 [2:7559341041135415029:3763]
2025-10-09T21:44:00.804737Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:00.804757Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:00.804761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.804765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.804767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.804787Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:00.804792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:00.804799Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:00.804864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:00.804866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:00.804868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:00.804870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:00.804950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:00.804953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:00.804967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:00.804977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:00.804991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:00.805010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341041135415016:3763]
2025-10-09T21:44:00.805012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341041135415016:3763]
2025-10-09T21:44:00.805014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341041135415016:3763]
2025-10-09T21:44:00.805029Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:00.805043Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:00.805056Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:00.805059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:00.805100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341041135415016:3763]
2025-10-09T21:44:00.805102Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:00.805379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:00.805385Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:00.805387Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:00.805392Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:00.806586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341041497225104:3810], now have 1 active actors on pipe
2025-10-09T21:44:00.806596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341041497225030:4049], now have 1 active actors on pipe
2025-10-09T21:44:00.806599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341041497224808:2432], now have 1 active actors on pipe
2025-10-09T21:44:00.806614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:00.806620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:00.806653Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|48be946b-4d614aae-9f5253aa-1dec49cf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:00.806662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.806667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.806669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:00.806682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.806697Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:00.806716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:00.807284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:00.807292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:00.807312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:00.807864Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b81f1c1c-114526bf-4d32caea-3dbb734d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|48be946b-4d614aae-9f5253aa-1dec49cf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:00.806134Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 232, pipe: [1:7559341041497225104:3810]
2025-10-09T21:44:00.806176Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 232
2025-10-09T21:44:00.806195Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 232
2025-10-09T21:44:00.807459Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 233 partition: 0 MaxSeqNo: 25 sessionId: src_id|48be946b-4d614aae-9f5253aa-1dec49cf_0
2025-10-09T21:44:00.810458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48be946b-4d614aae-9f5253aa-1dec49cf_0] Write session: try to update token
2025-10-09T21:44:00.810481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48be946b-4d614aae-9f5253aa-1dec49cf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:00.838192Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:00.838224Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:00.838226Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:00.838228Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:00.838240Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341041135415016:3763]
2025-10-09T21:44:00.838245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341041135415016:3763]
2025-10-09T21:44:00.838249Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341041135415016:3763]
2025-10-09T21:44:00.838296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:00.838304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:00.838306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:00.838466Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:00.842279Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:00.842296Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:00.842301Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:00.843569Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 233 sessionId: src_id|48be946b-4d614aae-9f5253aa-1dec49cf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:00.845935Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:01.181908Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:01.182440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:01.182453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:01.183166Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:01.183757Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341041497225104:3810]
2025-10-09T21:44:01.183795Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:01.183800Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:01.183857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 233 reason: error at write: Disconnected sessionId: src_id|48be946b-4d614aae-9f5253aa-1dec49cf_0
2025-10-09T21:44:01.184046Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 233 sessionId: src_id|48be946b-4d614aae-9f5253aa-1dec49cf_0 is DEAD
2025-10-09T21:44:01.184174Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:01.186445Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48be946b-4d614aae-9f5253aa-1dec49cf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:01.186454Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48be946b-4d614aae-9f5253aa-1dec49cf_0] Write session will restart in 0.006321s
2025-10-09T21:44:01.186478Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48be946b-4d614aae-9f5253aa-1dec49cf_0] Write session: Do CDS request
2025-10-09T21:44:01.186489Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48be946b-4d614aae-9f5253aa-1dec49cf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:01.194555Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48be946b-4d614aae-9f5253aa-1dec49cf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:01.194919Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:01.194936Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 234
2025-10-09T21:44:01.195132Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 234 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:01.195163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 234 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:01.195169Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=234 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:01.195173Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:01.195650Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 234 sessionId:  describe result for acl check
2025-10-09T21:44:01.195684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:01.195687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:01.195689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:01.195695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341045792192448:4055] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:01.196269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341045792192448:4055] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:01.218629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341045792192448:4055] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:01.224589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341045792192448:4055] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:01.224608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341045792192448:4055] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:01.224613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341045792192448:4055] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:01.224624Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 234 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:01.269008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:01.272522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:01.272776Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:01.272781Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:01.272800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:01.272851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:01.272901Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341045430382621:3769]
2025-10-09T21:44:01.273473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:01.279865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:01.279942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:01.280358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:01.280406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:01.280407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:01.280408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:01.280410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:01.280411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:01.280413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:01.280414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.280415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.280416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.280423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:01.280434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:01.280439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:01.280443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:01.280446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:01.280451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:01.280455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.280458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.280462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.280470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:01.280478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:01.280485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:01.280490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:01.280497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:01.280502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:01.280507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:01.280512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:01.280516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:01.280521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:01.280526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:01.280531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:01.280537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.280543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.280547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.280552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.280557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.280562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.280592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:01.280898Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341045792192722:3810]
2025-10-09T21:44:01.295325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:01.296877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:01.296884Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:01.296887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:01.296891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:01.296900Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 233 [2:7559341045430382621:3769]
2025-10-09T21:44:01.296915Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:01.296936Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:01.296940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.296943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.296945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.296963Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:01.296969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:01.296977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:01.297043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.297045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.297048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.297050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:01.297129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:01.297131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.297143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.297154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.297168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:01.297188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341045430382615:3769]
2025-10-09T21:44:01.297191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341045430382615:3769]
2025-10-09T21:44:01.297192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341045430382615:3769]
2025-10-09T21:44:01.297201Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:01.297215Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:01.297229Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:01.297232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:01.297273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341045430382615:3769]
2025-10-09T21:44:01.297275Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:01.298074Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:01.298090Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:01.298092Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:01.298101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:01.298277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341045792192435:2432], now have 1 active actors on pipe
2025-10-09T21:44:01.298281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341045792192493:4055], now have 1 active actors on pipe
2025-10-09T21:44:01.298284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341045792192722:3810], now have 1 active actors on pipe
2025-10-09T21:44:01.299078Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 233
2025-10-09T21:44:01.299117Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 233
2025-10-09T21:44:01.299135Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 233, pipe: [1:7559341045792192722:3810]
2025-10-09T21:44:01.303158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:01.303179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:01.303221Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|468f4ca3-3ea3313f-af4e90a4-5927c003_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:01.303235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.303240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.303243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:01.303258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.303277Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:01.303305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:01.311300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:01.311327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:01.311366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:01.311688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 234 partition: 0 MaxSeqNo: 25 sessionId: src_id|468f4ca3-3ea3313f-af4e90a4-5927c003_0
2025-10-09T21:44:01.314518Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48be946b-4d614aae-9f5253aa-1dec49cf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|468f4ca3-3ea3313f-af4e90a4-5927c003_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:01.317175Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|468f4ca3-3ea3313f-af4e90a4-5927c003_0] Write session: try to update token
2025-10-09T21:44:01.317194Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|468f4ca3-3ea3313f-af4e90a4-5927c003_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:01.332185Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:01.332205Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:01.332208Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:01.332210Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:01.332220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341045430382615:3769]
2025-10-09T21:44:01.332225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341045430382615:3769]
2025-10-09T21:44:01.332229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341045430382615:3769]
2025-10-09T21:44:01.332274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:01.332281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.332284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:01.332491Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:01.333208Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:01.333215Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:01.333220Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:01.347697Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 234 sessionId: src_id|468f4ca3-3ea3313f-af4e90a4-5927c003_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:01.350148Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:01.552398Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:01.552887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:01.552898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:01.553222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:01.553238Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:01.553295Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 234 reason: error at write: Disconnected sessionId: src_id|468f4ca3-3ea3313f-af4e90a4-5927c003_0
2025-10-09T21:44:01.553530Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341045792192722:3810]
2025-10-09T21:44:01.553779Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|468f4ca3-3ea3313f-af4e90a4-5927c003_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:01.553786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|468f4ca3-3ea3313f-af4e90a4-5927c003_0] Write session will restart in 0.005844s
2025-10-09T21:44:01.553810Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|468f4ca3-3ea3313f-af4e90a4-5927c003_0] Write session: Do CDS request
2025-10-09T21:44:01.553834Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|468f4ca3-3ea3313f-af4e90a4-5927c003_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:01.553622Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:01.555920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 234 sessionId: src_id|468f4ca3-3ea3313f-af4e90a4-5927c003_0 is DEAD
2025-10-09T21:44:01.558262Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:01.563737Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|468f4ca3-3ea3313f-af4e90a4-5927c003_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:01.563919Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:01.563937Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 235
2025-10-09T21:44:01.564473Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 235 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:01.564508Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 235 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:01.564513Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=235 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:01.564517Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:01.565289Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 235 sessionId:  describe result for acl check
2025-10-09T21:44:01.565320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:01.565321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:01.565323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:01.565328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341045792192748:4060] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:01.565829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341045792192748:4060] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:01.581603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341045792192748:4060] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:01.594491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341045792192748:4060] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:01.594510Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341045792192748:4060] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:01.594514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341045792192748:4060] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:01.594524Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 235 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:01.628266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:01.630790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:01.631010Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:01.631014Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:01.631026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:01.631077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:01.631117Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341045430382951:3777]
2025-10-09T21:44:01.631753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:01.634250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:01.634334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:01.634724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:01.634780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:01.634781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:01.634783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:01.634785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:01.634786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:01.634787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:01.634789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.634790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.634792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.634799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:01.634811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:01.634815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:01.634820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:01.634824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:01.634829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:01.634833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.634837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.634842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.634850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:01.634860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:01.634866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:01.634872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:01.634877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:01.634882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:01.634887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:01.634892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:01.634897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:01.634903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:01.634908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:01.634913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:01.634919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.634925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.634929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.634935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.634940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.634946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.634980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:01.653246Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341045792193032:3810]
2025-10-09T21:44:01.670802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:01.672603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:01.672610Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:01.672612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:01.672617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:01.672625Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 234 [2:7559341045430382951:3777]
2025-10-09T21:44:01.672643Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:01.672664Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:01.672667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.672671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.672673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.672691Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:01.672699Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:01.672707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:01.672789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.672791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.672793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.672796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:01.672883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:01.672886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.672900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.672912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.672929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:01.672950Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341045430382940:3777]
2025-10-09T21:44:01.672953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341045430382940:3777]
2025-10-09T21:44:01.672955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341045430382940:3777]
2025-10-09T21:44:01.672969Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:01.672985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:01.673000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:01.673003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:01.673047Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341045430382940:3777]
2025-10-09T21:44:01.673050Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:01.678658Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 234
2025-10-09T21:44:01.678695Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 234, pipe: [1:7559341045792193032:3810]
2025-10-09T21:44:01.678719Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 234
2025-10-09T21:44:01.674328Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:01.674338Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:01.674340Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:01.674348Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:01.679016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341045792192736:2432], now have 1 active actors on pipe
2025-10-09T21:44:01.679027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341045792193032:3810], now have 1 active actors on pipe
2025-10-09T21:44:01.679031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341045792192851:4060], now have 1 active actors on pipe
2025-10-09T21:44:01.679052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:01.679059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:01.679093Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bc976164-7a8cf19-55c1b401-f08badd7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:01.679105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.679114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.679117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:01.679132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.679148Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:01.679169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:01.679790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:01.679798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:01.679815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:01.679953Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 235 partition: 0 MaxSeqNo: 25 sessionId: src_id|bc976164-7a8cf19-55c1b401-f08badd7_0
2025-10-09T21:44:01.680417Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|468f4ca3-3ea3313f-af4e90a4-5927c003_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bc976164-7a8cf19-55c1b401-f08badd7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:01.683086Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc976164-7a8cf19-55c1b401-f08badd7_0] Write session: try to update token
2025-10-09T21:44:01.683109Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc976164-7a8cf19-55c1b401-f08badd7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:01.735654Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:01.735676Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:01.735678Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:01.735680Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:01.735689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341045430382940:3777]
2025-10-09T21:44:01.735694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341045430382940:3777]
2025-10-09T21:44:01.735698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341045430382940:3777]
2025-10-09T21:44:01.735745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:01.735754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.735757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:01.736040Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:01.738650Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:01.738661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:01.738665Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:01.747100Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 235 sessionId: src_id|bc976164-7a8cf19-55c1b401-f08badd7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:01.749793Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:01.941806Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:01.942104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:01.942115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:01.944946Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:01.946142Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:01.946197Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341045792193032:3810]
2025-10-09T21:44:01.946221Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:01.946297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 235 reason: status is not ok: tablet will be restarted right now sessionId: src_id|bc976164-7a8cf19-55c1b401-f08badd7_0
2025-10-09T21:44:01.946502Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 235 sessionId: src_id|bc976164-7a8cf19-55c1b401-f08badd7_0 is DEAD
2025-10-09T21:44:01.946958Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc976164-7a8cf19-55c1b401-f08badd7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:01.946967Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc976164-7a8cf19-55c1b401-f08badd7_0] Write session will restart in 0.009942s
2025-10-09T21:44:01.946991Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc976164-7a8cf19-55c1b401-f08badd7_0] Write session: Do CDS request
2025-10-09T21:44:01.946997Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc976164-7a8cf19-55c1b401-f08badd7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:01.961488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc976164-7a8cf19-55c1b401-f08badd7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:01.961983Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:01.961998Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 236
2025-10-09T21:44:01.962177Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 236 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:01.962223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 236 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:01.962229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=236 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:01.962232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:01.962647Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 236 sessionId:  describe result for acl check
2025-10-09T21:44:01.962678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:01.962680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:01.962682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:01.962689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341045792193088:4073] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:01.963363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341045792193088:4073] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:01.967845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:01.968774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:01.969000Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:01.969004Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:01.969015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:01.969086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:01.969127Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341045430383012:3784]
2025-10-09T21:44:01.969774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:01.970824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:01.970875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:01.971211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:01.971269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:01.971270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:01.971271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:01.971272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:01.971274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:01.971275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:01.971276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.971278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.971281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.971288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:01.971297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:01.971300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:01.971304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:01.971308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:01.971312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:01.971315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.971318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.971321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.971328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:01.971337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:01.971343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:01.971348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:01.971352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:01.971357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:01.971361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:01.971365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:01.971370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:01.971374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:01.971378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:01.971383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:01.971387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.971392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.971397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.971402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.971406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.971411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.971439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:01.978187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341045792193088:4073] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:01.985704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:01.986943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:01.986968Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:01.986971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:01.986977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:01.986983Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 235 [2:7559341045430383012:3784]
2025-10-09T21:44:01.987000Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:01.987021Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:01.987025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.987028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.987031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.987050Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:01.987056Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:01.987063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:01.987130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.987132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.987134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.987137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:01.987267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:01.987270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:01.987284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:01.987297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:01.987317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:01.987339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341045430383007:3784]
2025-10-09T21:44:01.987341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341045430383007:3784]
2025-10-09T21:44:01.987344Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341045430383007:3784]
2025-10-09T21:44:01.987356Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:01.987372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:01.987388Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:01.987391Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:01.987433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341045430383007:3784]
2025-10-09T21:44:01.987436Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:01.988075Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:01.988085Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:01.988088Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:01.988096Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:01.989389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341045792193067:2432], now have 1 active actors on pipe
2025-10-09T21:44:01.989654Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 235
2025-10-09T21:44:01.995641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341045792193088:4073] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:01.995661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341045792193088:4073] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:01.995666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341045792193088:4073] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:01.995677Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 236 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:01.996622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341045792193137:4073], now have 1 active actors on pipe
2025-10-09T21:44:01.996826Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 235
2025-10-09T21:44:01.996975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:01.996989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:01.997031Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b8d33ef3-fffd9c9b-332a6ef1-6e9121_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:01.997045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.997051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.997055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:01.997069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.997087Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:01.997109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:01.997501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:01.997511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:01.997541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:01.998258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc976164-7a8cf19-55c1b401-f08badd7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b8d33ef3-fffd9c9b-332a6ef1-6e9121_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:02.000856Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b8d33ef3-fffd9c9b-332a6ef1-6e9121_0] Write session: try to update token
2025-10-09T21:44:02.000878Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b8d33ef3-fffd9c9b-332a6ef1-6e9121_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:01.997795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 236 partition: 0 MaxSeqNo: 25 sessionId: src_id|b8d33ef3-fffd9c9b-332a6ef1-6e9121_0
2025-10-09T21:44:02.023284Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 236 sessionId: src_id|b8d33ef3-fffd9c9b-332a6ef1-6e9121_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:02.026349Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:02.026674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:02.026695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:02.026739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:02.026885Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:02.031756Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:02.031778Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:02.031781Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:02.031782Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:02.031792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341045430383007:3784]
2025-10-09T21:44:02.031797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341045430383007:3784]
2025-10-09T21:44:02.031800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341045430383007:3784]
2025-10-09T21:44:02.031849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:02.031862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.031865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:02.032001Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:02.034107Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:02.034115Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:02.034120Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:02.043368Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341050087160437:3810]
2025-10-09T21:44:02.046546Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 235, pipe: [1:7559341050087160437:3810]
2025-10-09T21:44:02.213905Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:02.214586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341050087160437:3810], now have 1 active actors on pipe
2025-10-09T21:44:02.224932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:02.224961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:02.225143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.225218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.225285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.225356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.225409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.225417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.225495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.225559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.225624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.225731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.225776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.225779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.225869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.225940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.226002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.226430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.226909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.226917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.227003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.227074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.227143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.227231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.227273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.227277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.227352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.227418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.227489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.227563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.227612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.227615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.227693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.227759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.227821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.227885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.227938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.227941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.228011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.228081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.228165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.228232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.228275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.228281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.228351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.228419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.228483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.228548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.228594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.228597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.228788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.228851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.228918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.228985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.229034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.229037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.229111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.229177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.229246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.229316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.229370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.229374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.229764Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:02.231007Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:02.231062Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:02.231108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 236 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b8d33ef3-fffd9c9b-332a6ef1-6e9121_0
2025-10-09T21:44:02.231302Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 236 sessionId: src_id|b8d33ef3-fffd9c9b-332a6ef1-6e9121_0 is DEAD
2025-10-09T21:44:02.231468Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341050087160437:3810]
2025-10-09T21:44:02.233183Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8d33ef3-fffd9c9b-332a6ef1-6e9121_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:02.233197Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8d33ef3-fffd9c9b-332a6ef1-6e9121_0] Write session will restart in 0.005582s
2025-10-09T21:44:02.233227Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8d33ef3-fffd9c9b-332a6ef1-6e9121_0] Write session: Do CDS request
2025-10-09T21:44:02.233234Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8d33ef3-fffd9c9b-332a6ef1-6e9121_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:02.241515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b8d33ef3-fffd9c9b-332a6ef1-6e9121_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:02.241718Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:02.241734Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 237
2025-10-09T21:44:02.242194Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 237 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:02.242249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 237 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:02.242256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=237 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:02.242265Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:02.243040Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 237 sessionId:  describe result for acl check
2025-10-09T21:44:02.243078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:02.243080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:02.243082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:02.243088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341050087160456:4078] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:02.243683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341050087160456:4078] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:02.246632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:02.251382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341050087160456:4078] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:02.254201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:02.254503Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:02.254507Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:02.254521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:02.254585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:02.254661Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341049725350378:3789]
2025-10-09T21:44:02.255401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:02.256077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:02.256130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:02.256515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:02.256572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:02.256576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:02.256578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:02.256579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:02.256580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:02.256581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:02.256583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.256584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.256585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.256604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:02.256615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:02.256619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:02.256622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:02.256625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:02.256628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:02.256631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.256634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.256637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.256648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:02.256656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:02.256662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:02.256668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:02.256673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:02.256679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:02.256684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:02.256689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:02.256694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:02.256700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:02.256705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:02.256711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:02.256716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.256721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.256729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.256734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.256742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.256747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.256778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:02.259590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341050087160456:4078] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:02.259609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341050087160456:4078] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:02.259613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341050087160456:4078] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:02.259624Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 237 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:02.270515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:02.271747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:02.271769Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:02.271773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:02.271778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:02.271787Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 236 [2:7559341049725350378:3789]
2025-10-09T21:44:02.271804Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:02.271827Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:02.271835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.271839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.271847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.271869Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:02.271876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:02.271891Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:02.271962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.271970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.271972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.271976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:02.272276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:02.272285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.272299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.272310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.272328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:02.272354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341050087160496:4078], now have 1 active actors on pipe
2025-10-09T21:44:02.272358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341050087160440:2432], now have 1 active actors on pipe
2025-10-09T21:44:02.272367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341049725350373:3789]
2025-10-09T21:44:02.272368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341049725350373:3789]
2025-10-09T21:44:02.272370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341049725350373:3789]
2025-10-09T21:44:02.272769Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 236
2025-10-09T21:44:02.272803Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 236
2025-10-09T21:44:02.272384Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:02.272401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:02.272415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:02.272418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:02.272462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341049725350373:3789]
2025-10-09T21:44:02.272465Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:02.272550Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:02.272553Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:02.272555Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:02.272561Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:02.273393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:02.273408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:02.273444Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9c0821eb-95e64c85-85676551-e5945d0b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:02.273454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.273459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.273462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:02.273475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.273490Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:02.273510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:02.274542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:02.274553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:02.274574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:02.274735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 237 partition: 0 MaxSeqNo: 25 sessionId: src_id|9c0821eb-95e64c85-85676551-e5945d0b_0
2025-10-09T21:44:02.275234Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8d33ef3-fffd9c9b-332a6ef1-6e9121_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9c0821eb-95e64c85-85676551-e5945d0b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:02.278128Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c0821eb-95e64c85-85676551-e5945d0b_0] Write session: try to update token
2025-10-09T21:44:02.278145Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c0821eb-95e64c85-85676551-e5945d0b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:02.296353Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:02.296372Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:02.296375Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:02.296377Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:02.296390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341049725350373:3789]
2025-10-09T21:44:02.296395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341049725350373:3789]
2025-10-09T21:44:02.296398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341049725350373:3789]
2025-10-09T21:44:02.296443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:02.296449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.296452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:02.296593Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:02.301402Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:02.301418Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:02.301422Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:02.305777Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 237 sessionId: src_id|9c0821eb-95e64c85-85676551-e5945d0b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:02.308632Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:02.336725Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341050087160505:3810]
2025-10-09T21:44:02.338912Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 236, pipe: [1:7559341050087160505:3810]
2025-10-09T21:44:02.486195Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:02.488134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:02.488153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:02.488201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341050087160505:3810], now have 1 active actors on pipe
2025-10-09T21:44:02.489746Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:02.489797Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:02.489799Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:02.489860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 237 reason: error at write: Disconnected sessionId: src_id|9c0821eb-95e64c85-85676551-e5945d0b_0
2025-10-09T21:44:02.490060Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 237 sessionId: src_id|9c0821eb-95e64c85-85676551-e5945d0b_0 is DEAD
2025-10-09T21:44:02.490222Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341050087160505:3810]
2025-10-09T21:44:02.490392Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c0821eb-95e64c85-85676551-e5945d0b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:02.490405Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c0821eb-95e64c85-85676551-e5945d0b_0] Write session will restart in 0.007826s
2025-10-09T21:44:02.490443Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c0821eb-95e64c85-85676551-e5945d0b_0] Write session: Do CDS request
2025-10-09T21:44:02.490453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c0821eb-95e64c85-85676551-e5945d0b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:02.490972Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:02.503260Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c0821eb-95e64c85-85676551-e5945d0b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:02.504115Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:02.504131Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 238
2025-10-09T21:44:02.504275Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 238 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:02.504309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 238 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:02.504315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=238 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:02.504319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:02.506991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 238 sessionId:  describe result for acl check
2025-10-09T21:44:02.507041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:02.507044Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:02.507046Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:02.507053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341050087160517:4083] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:02.507662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341050087160517:4083] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:02.523501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341050087160517:4083] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:02.529877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341050087160517:4083] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:02.529896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341050087160517:4083] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:02.529901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341050087160517:4083] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:02.529910Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 238 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:02.529371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:02.529731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:02.529930Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:02.529934Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:02.529947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:02.530010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:02.530060Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341049725350437:3791]
2025-10-09T21:44:02.530605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:02.531483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:02.531537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:02.531862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:02.531911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:02.531912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:02.531913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:02.531931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:02.531932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:02.531934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:02.531935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.531936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.531937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.531942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:02.531953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:02.531957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:02.531961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:02.531965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:02.531969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:02.531972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.531976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.531980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.531989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:02.531997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:02.532003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:02.532008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:02.532014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:02.532019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:02.532023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:02.532028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:02.532032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:02.532037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:02.532041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:02.532046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:02.532051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.532056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.532060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.532064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.532068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.532073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.532101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:02.542179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:02.543662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:02.543670Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:02.543674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:02.543680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:02.543689Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 237 [2:7559341049725350437:3791]
2025-10-09T21:44:02.543704Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:02.543730Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:02.543734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.543737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.543740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.543759Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:02.543765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:02.543773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:02.543847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.543849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.543851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.543854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:02.543933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:02.543936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.543952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.543963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.543979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:02.544003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341049725350431:3791]
2025-10-09T21:44:02.544005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341049725350431:3791]
2025-10-09T21:44:02.544007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341049725350431:3791]
2025-10-09T21:44:02.544022Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:02.544041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:02.544057Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:02.544060Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:02.544104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341049725350431:3791]
2025-10-09T21:44:02.544111Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:02.544788Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 237
2025-10-09T21:44:02.544833Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 237
2025-10-09T21:44:02.544446Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:02.544453Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:02.544455Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:02.544461Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:02.549389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341050087160510:2432], now have 1 active actors on pipe
2025-10-09T21:44:02.549405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341050087160563:4083], now have 1 active actors on pipe
2025-10-09T21:44:02.549438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:02.549445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:02.549487Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e2d703ce-ebc62c65-9777d2da-f740d1b4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:02.549501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.549510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.549513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:02.549528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.549549Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:02.549570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:02.550241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:02.550253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:02.550275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:02.550472Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 238 partition: 0 MaxSeqNo: 25 sessionId: src_id|e2d703ce-ebc62c65-9777d2da-f740d1b4_0
2025-10-09T21:44:02.550934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c0821eb-95e64c85-85676551-e5945d0b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e2d703ce-ebc62c65-9777d2da-f740d1b4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:02.553696Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2d703ce-ebc62c65-9777d2da-f740d1b4_0] Write session: try to update token
2025-10-09T21:44:02.553714Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2d703ce-ebc62c65-9777d2da-f740d1b4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:02.590200Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341050087160573:3810]
2025-10-09T21:44:02.590852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 237, pipe: [1:7559341050087160573:3810]
2025-10-09T21:44:02.593778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341050087160573:3810], now have 1 active actors on pipe
2025-10-09T21:44:02.606154Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 238 sessionId: src_id|e2d703ce-ebc62c65-9777d2da-f740d1b4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:02.609017Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:02.610213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:02.610237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:02.611686Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:02.611691Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:02.611693Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:02.611695Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:02.611706Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341049725350431:3791]
2025-10-09T21:44:02.611711Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341049725350431:3791]
2025-10-09T21:44:02.611714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341049725350431:3791]
2025-10-09T21:44:02.611771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:02.611785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:02.611792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.611795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:02.611942Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:02.615661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:02.615675Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:02.615680Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:02.616833Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:02.828109Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:02.837792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:02.837823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:02.837995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.838076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.838146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.838870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.839160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.839167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.839247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.839311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.839375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.839451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.839494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.839498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.839580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.839641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.839699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.839778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.839818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.839821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.839899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.839965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.840042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.840110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.840156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.840162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.840234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.840293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.840358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.840422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.840465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.840468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.840534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.840592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.840664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.840723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.846645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2d703ce-ebc62c65-9777d2da-f740d1b4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:02.846661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2d703ce-ebc62c65-9777d2da-f740d1b4_0] Write session will restart in 0.007682s
2025-10-09T21:44:02.846061Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:02.846688Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2d703ce-ebc62c65-9777d2da-f740d1b4_0] Write session: Do CDS request
2025-10-09T21:44:02.846119Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341050087160573:3810]
2025-10-09T21:44:02.846695Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2d703ce-ebc62c65-9777d2da-f740d1b4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:02.846214Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:02.846217Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:02.846250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 238 reason: error at write: Disconnected sessionId: src_id|e2d703ce-ebc62c65-9777d2da-f740d1b4_0
2025-10-09T21:44:02.846480Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 238 sessionId: src_id|e2d703ce-ebc62c65-9777d2da-f740d1b4_0 is DEAD
2025-10-09T21:44:02.840776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.840783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.840847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.840907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.840973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.841033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.841075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.841080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.841144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.841198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.841262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.841326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.841369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.841375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.841443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.841504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.841568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.841630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.841673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.841676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.841746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:02.841806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:02.841866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:02.842538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:02.842606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:02.842609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:02.847119Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:02.858679Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2d703ce-ebc62c65-9777d2da-f740d1b4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:02.859106Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:02.859124Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 239
2025-10-09T21:44:02.859459Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 239 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:02.859493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 239 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:02.859503Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=239 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:02.859507Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:02.860031Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 239 sessionId:  describe result for acl check
2025-10-09T21:44:02.860063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:02.860065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:02.860067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:02.860073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341050087160589:4087] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:02.860643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341050087160589:4087] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:02.868261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341050087160589:4087] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:02.872640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341050087160589:4087] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:02.872656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341050087160589:4087] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:02.872661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341050087160589:4087] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:02.872672Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 239 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:02.882965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:02.885240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:02.885470Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:02.885480Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:02.885493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:02.885549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:02.885591Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341049725350532:3804]
2025-10-09T21:44:02.886228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:02.889328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:02.889407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:02.889798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:02.889849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:02.889850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:02.889852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:02.889853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:02.889855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:02.889856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:02.889857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.889858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.889860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.889865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:02.889876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:02.889880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:02.889884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:02.889887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:02.889891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:02.889894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.889898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.889901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.889909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:02.889917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:02.889924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:02.889929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:02.889933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:02.889938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:02.889942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:02.889946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:02.889952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:02.889957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:02.889961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:02.889966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:02.889972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.889977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.889981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.889985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.889990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.889995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.890022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:02.901222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:02.902433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:02.902450Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:02.902453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:02.902463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:02.902471Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 238 [2:7559341049725350532:3804]
2025-10-09T21:44:02.902488Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:02.902511Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:02.902515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.902518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.902522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.902546Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:02.902563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:02.902575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:02.902642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.902649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.902651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.902654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:02.902736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:02.902744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.902758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:02.902772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:02.902788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:02.902813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341049725350527:3804]
2025-10-09T21:44:02.902820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341049725350527:3804]
2025-10-09T21:44:02.902822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341049725350527:3804]
2025-10-09T21:44:02.902838Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:02.902854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:02.902876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:02.902884Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:02.902927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341049725350527:3804]
2025-10-09T21:44:02.902934Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:02.904082Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:02.904095Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:02.904097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:02.904106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:02.904286Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 238
2025-10-09T21:44:02.904332Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 238
2025-10-09T21:44:02.911694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341050087160585:2432], now have 1 active actors on pipe
2025-10-09T21:44:02.911715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341050087160632:4087], now have 1 active actors on pipe
2025-10-09T21:44:02.911758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:02.911766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:02.911810Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|15a8c053-365fa0e4-3968c147-6e36e512_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:02.911826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.911832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.911835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:02.911851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.911873Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:02.911895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:02.916260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:02.916287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:02.916330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:02.916617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 239 partition: 0 MaxSeqNo: 25 sessionId: src_id|15a8c053-365fa0e4-3968c147-6e36e512_0
2025-10-09T21:44:02.917145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2d703ce-ebc62c65-9777d2da-f740d1b4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|15a8c053-365fa0e4-3968c147-6e36e512_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:02.919884Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15a8c053-365fa0e4-3968c147-6e36e512_0] Write session: try to update token
2025-10-09T21:44:02.919905Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15a8c053-365fa0e4-3968c147-6e36e512_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:02.944351Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 239 sessionId: src_id|15a8c053-365fa0e4-3968c147-6e36e512_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:02.947380Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:02.947472Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341050087160655:3810]
2025-10-09T21:44:02.948206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:02.948229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:02.948275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:02.948531Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:02.952549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341050087160655:3810], now have 1 active actors on pipe
2025-10-09T21:44:02.953869Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 238, pipe: [1:7559341050087160655:3810]
2025-10-09T21:44:02.959865Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:02.959891Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:02.959893Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:02.959898Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:02.959909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341049725350527:3804]
2025-10-09T21:44:02.959915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341049725350527:3804]
2025-10-09T21:44:02.959920Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341049725350527:3804]
2025-10-09T21:44:02.959966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:02.959975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:02.959977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:02.960111Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:02.966190Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:02.966206Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:02.966211Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:03.307582Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:03.335450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:03.335481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:03.335654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:03.335717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:03.335775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:03.335831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:03.335870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:03.335875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:03.335939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:03.335996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:03.336057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:03.336118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:03.336156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:03.336159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:03.336220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:03.336272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:03.336330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:03.336396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:03.336434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:03.336437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:03.336504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:03.336847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:03.336909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:03.336970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:03.337012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:03.337015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:03.337082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:03.337138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:03.337197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:03.337255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:03.337400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:03.337404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:03.337469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:03.337522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:03.337582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:03.337641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:03.337680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:03.337683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:03.337748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:03.337801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:03.337861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:03.337923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:03.337967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:03.337972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:03.350382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:03.350487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:03.350571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:03.350628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:03.350673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:03.350679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:03.350759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:03.350814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:03.350871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:03.350933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:03.351004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:03.351007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:03.351078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:03.351155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:03.351220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:03.351279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:03.351347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:03.351351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:03.353920Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:03.353177Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:03.353190Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:03.353220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 239 reason: error at write: Disconnected sessionId: src_id|15a8c053-365fa0e4-3968c147-6e36e512_0
2025-10-09T21:44:03.353399Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 239 sessionId: src_id|15a8c053-365fa0e4-3968c147-6e36e512_0 is DEAD
2025-10-09T21:44:03.353540Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:03.353572Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341050087160655:3810]
2025-10-09T21:44:03.359535Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15a8c053-365fa0e4-3968c147-6e36e512_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:03.359549Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15a8c053-365fa0e4-3968c147-6e36e512_0] Write session will restart in 0.008423s
2025-10-09T21:44:03.359575Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15a8c053-365fa0e4-3968c147-6e36e512_0] Write session: Do CDS request
2025-10-09T21:44:03.359583Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15a8c053-365fa0e4-3968c147-6e36e512_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:03.369963Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15a8c053-365fa0e4-3968c147-6e36e512_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:03.370335Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:03.370351Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 240
2025-10-09T21:44:03.370918Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 240 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:03.370955Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 240 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:03.370962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=240 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:03.370965Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:03.371586Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 240 sessionId:  describe result for acl check
2025-10-09T21:44:03.371623Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:03.371625Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:03.371627Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:03.371633Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341054382128003:4099] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:03.372198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341054382128003:4099] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:03.378019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:03.381855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:03.382181Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:03.382186Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:03.382212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:03.382291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:03.382335Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341054020317902:3809]
2025-10-09T21:44:03.382994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:03.384822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341054382128003:4099] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:03.387239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:03.387316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:03.387694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:03.387751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:03.387752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:03.387753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:03.387755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:03.387756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:03.387757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:03.387759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:03.387760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:03.387761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:03.387767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:03.387777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:03.387780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:03.387784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:03.387788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:03.387792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:03.387796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:03.387800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:03.387804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:03.387813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:03.387821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:03.387828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:03.387833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:03.387839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:03.387844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:03.387848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:03.387853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:03.387858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:03.387863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:03.387870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:03.387875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:03.387880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:03.387885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:03.387890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:03.387895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:03.387901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:03.387905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:03.387936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:03.400379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341054382128003:4099] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:03.400398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341054382128003:4099] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:03.400402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341054382128003:4099] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:03.400414Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 240 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:03.402083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:03.403240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:03.403247Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:03.403250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:03.403255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:03.403263Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 239 [2:7559341054020317902:3809]
2025-10-09T21:44:03.403277Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:03.403296Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:03.403298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.403301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.403304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.403317Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:03.403323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:03.403331Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:03.403408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:03.403410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:03.403412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:03.403415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:03.403768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:03.403771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:03.403786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:03.403800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:03.403820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:03.403853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341054020317897:3809]
2025-10-09T21:44:03.403856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341054020317897:3809]
2025-10-09T21:44:03.403858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341054020317897:3809]
2025-10-09T21:44:03.403873Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:03.403891Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:03.403907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:03.403910Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:03.403951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341054020317897:3809]
2025-10-09T21:44:03.403953Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:03.405588Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 239
2025-10-09T21:44:03.404733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:03.404743Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:03.404746Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:03.404753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:03.406187Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 239
2025-10-09T21:44:03.406469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341054382127984:2432], now have 1 active actors on pipe
2025-10-09T21:44:03.406486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341054382128043:4099], now have 1 active actors on pipe
2025-10-09T21:44:03.406501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:03.406507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:03.406546Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|51595d1e-507b39d8-bdebee31-879faa32_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:03.406565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.406571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.406574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:03.406587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.406603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:03.406629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:03.412678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:03.412711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:03.412754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:03.413051Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 240 partition: 0 MaxSeqNo: 25 sessionId: src_id|51595d1e-507b39d8-bdebee31-879faa32_0
2025-10-09T21:44:03.413510Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15a8c053-365fa0e4-3968c147-6e36e512_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|51595d1e-507b39d8-bdebee31-879faa32_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:03.416057Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51595d1e-507b39d8-bdebee31-879faa32_0] Write session: try to update token
2025-10-09T21:44:03.416079Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51595d1e-507b39d8-bdebee31-879faa32_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:03.452269Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341054382128051:3810]
2025-10-09T21:44:03.452879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341054382128051:3810], now have 1 active actors on pipe
2025-10-09T21:44:03.453029Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 239, pipe: [1:7559341054382128051:3810]
2025-10-09T21:44:03.468857Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:03.468881Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:03.468884Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:03.468885Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:03.468896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341054020317897:3809]
2025-10-09T21:44:03.468900Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341054020317897:3809]
2025-10-09T21:44:03.468903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341054020317897:3809]
2025-10-09T21:44:03.468949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:03.468958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:03.468961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:03.469097Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:03.473208Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:03.473224Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:03.473229Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:03.474396Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 240 sessionId: src_id|51595d1e-507b39d8-bdebee31-879faa32_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:03.476808Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:03.852658Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:03.854138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:03.854156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:03.855854Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:03.855867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:03.855923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 240 reason: error at write: Disconnected sessionId: src_id|51595d1e-507b39d8-bdebee31-879faa32_0
2025-10-09T21:44:03.856081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 240 sessionId: src_id|51595d1e-507b39d8-bdebee31-879faa32_0 is DEAD
2025-10-09T21:44:03.856203Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341054382128051:3810]
2025-10-09T21:44:03.856216Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:03.856559Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51595d1e-507b39d8-bdebee31-879faa32_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:03.856567Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51595d1e-507b39d8-bdebee31-879faa32_0] Write session will restart in 0.006994s
2025-10-09T21:44:03.856590Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51595d1e-507b39d8-bdebee31-879faa32_0] Write session: Do CDS request
2025-10-09T21:44:03.856596Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51595d1e-507b39d8-bdebee31-879faa32_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:03.858205Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:03.868032Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51595d1e-507b39d8-bdebee31-879faa32_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:03.868812Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:03.868826Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 241
2025-10-09T21:44:03.869022Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 241 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:03.869051Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 241 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:03.869057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=241 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:03.869061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:03.871489Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 241 sessionId:  describe result for acl check
2025-10-09T21:44:03.871533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:03.871539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:03.871541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:03.871548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341054382128077:4105] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:03.872158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341054382128077:4105] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:03.876070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341054382128077:4105] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:03.880855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341054382128077:4105] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:03.880871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341054382128077:4105] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:03.880875Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341054382128077:4105] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:03.880885Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 241 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:03.890727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:03.894487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:03.894741Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:03.894746Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:03.894761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:03.894829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:03.894906Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341054020317984:3818]
2025-10-09T21:44:03.895494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:03.898156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:03.898222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:03.898632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:03.898700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:03.898701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:03.898702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:03.898704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:03.898705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:03.898707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:03.898708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:03.898709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:03.898711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:03.898717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:03.898729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:03.898735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:03.898739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:03.898743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:03.898748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:03.898752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:03.898756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:03.898760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:03.898768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:03.898777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:03.898784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:03.898789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:03.898794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:03.898799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:03.898804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:03.898809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:03.898814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:03.898819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:03.898824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:03.898829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:03.898834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:03.898840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:03.898844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:03.898849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:03.898853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:03.898858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:03.898889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:03.908476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:03.909633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:03.909640Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:03.909643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:03.909648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:03.909672Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 240 [2:7559341054020317984:3818]
2025-10-09T21:44:03.909693Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:03.909716Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:03.909721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.909725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.909726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.909747Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:03.909754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:03.909762Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:03.909834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:03.909837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:03.909839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:03.909842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:03.909922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:03.909925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:03.909941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:03.909955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:03.909973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:03.909995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341054020317978:3818]
2025-10-09T21:44:03.909998Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341054020317978:3818]
2025-10-09T21:44:03.910000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341054020317978:3818]
2025-10-09T21:44:03.910015Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:03.910045Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:03.910058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:03.910065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:03.910108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341054020317978:3818]
2025-10-09T21:44:03.910110Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:03.910658Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:03.910667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:03.910670Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:03.910677Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:03.911408Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 240
2025-10-09T21:44:03.911448Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 240
2025-10-09T21:44:03.916487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341054382128064:2432], now have 1 active actors on pipe
2025-10-09T21:44:03.916503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341054382128120:4105], now have 1 active actors on pipe
2025-10-09T21:44:03.916536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:03.916544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:03.916581Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|65f7bdb2-4bf75dd-1a462769-e10faeca_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:03.916594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.916599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.916603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:03.916618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.916637Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:03.916657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:03.917082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:03.917089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:03.917103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:03.917235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 241 partition: 0 MaxSeqNo: 25 sessionId: src_id|65f7bdb2-4bf75dd-1a462769-e10faeca_0
2025-10-09T21:44:03.917688Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51595d1e-507b39d8-bdebee31-879faa32_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|65f7bdb2-4bf75dd-1a462769-e10faeca_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:03.920244Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|65f7bdb2-4bf75dd-1a462769-e10faeca_0] Write session: try to update token
2025-10-09T21:44:03.920268Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|65f7bdb2-4bf75dd-1a462769-e10faeca_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:03.946270Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:03.946289Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:03.946291Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:03.946293Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:03.946303Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341054020317978:3818]
2025-10-09T21:44:03.946308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341054020317978:3818]
2025-10-09T21:44:03.946311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341054020317978:3818]
2025-10-09T21:44:03.946356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:03.946363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:03.946367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:03.946518Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:03.950372Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:03.950386Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:03.950389Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:03.958336Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341054382128138:3810]
2025-10-09T21:44:03.959187Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 240, pipe: [1:7559341054382128138:3810]
2025-10-09T21:44:03.960280Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 241 sessionId: src_id|65f7bdb2-4bf75dd-1a462769-e10faeca_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:03.962792Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:04.161721Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:04.162925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341054382128138:3810], now have 1 active actors on pipe
2025-10-09T21:44:04.162966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:04.162972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:04.167253Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:04.167304Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341054382128138:3810]
2025-10-09T21:44:04.167327Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:04.167329Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:04.167399Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 241 reason: error at write: Disconnected sessionId: src_id|65f7bdb2-4bf75dd-1a462769-e10faeca_0
2025-10-09T21:44:04.167582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 241 sessionId: src_id|65f7bdb2-4bf75dd-1a462769-e10faeca_0 is DEAD
2025-10-09T21:44:04.170494Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|65f7bdb2-4bf75dd-1a462769-e10faeca_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:04.170508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|65f7bdb2-4bf75dd-1a462769-e10faeca_0] Write session will restart in 0.008130s
2025-10-09T21:44:04.170539Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|65f7bdb2-4bf75dd-1a462769-e10faeca_0] Write session: Do CDS request
2025-10-09T21:44:04.170548Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|65f7bdb2-4bf75dd-1a462769-e10faeca_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:04.172794Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:04.180732Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|65f7bdb2-4bf75dd-1a462769-e10faeca_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:04.181677Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:04.181694Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 242
2025-10-09T21:44:04.182570Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 242 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:04.182610Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 242 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:04.182615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=242 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:04.182619Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:04.185840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 242 sessionId:  describe result for acl check
2025-10-09T21:44:04.185877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:04.185878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:04.185880Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:04.185885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341058677095465:4115] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:04.186471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341058677095465:4115] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:04.194474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:04.198083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:04.198344Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:04.198359Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:04.198373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:04.200743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341058677095465:4115] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:04.198467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:04.198522Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341058315285363:3825]
2025-10-09T21:44:04.199127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:04.205025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:04.205133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:04.205539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:04.205608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:04.205610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:04.205611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:04.205612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:04.205617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:04.205619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:04.205620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:04.205622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:04.205624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:04.205632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:04.205643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:04.205647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:04.205651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:04.205654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:04.205659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:04.205662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:04.205665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:04.205669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:04.205677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:04.205685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:04.205692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:04.205696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:04.205702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:04.205706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:04.205710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:04.205715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:04.205721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:04.205739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:04.205746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:04.205752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:04.205756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:04.205760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:04.205767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:04.205772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:04.205775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:04.205780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:04.205808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:04.224607Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 241
2025-10-09T21:44:04.218810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:04.219968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:04.219975Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:04.219978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:04.219983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:04.219992Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 241 [2:7559341058315285363:3825]
2025-10-09T21:44:04.220011Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:04.220033Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:04.220038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.220041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.220044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.220063Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:04.220070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:04.220077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:04.220423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:04.220426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:04.220430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:04.220432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:04.220510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:04.220513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:04.220529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:04.220543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:04.220562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:04.220585Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341058315285357:3825]
2025-10-09T21:44:04.220587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341058315285357:3825]
2025-10-09T21:44:04.220590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341058315285357:3825]
2025-10-09T21:44:04.220604Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:04.220620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:04.220636Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:04.220639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:04.220680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341058315285357:3825]
2025-10-09T21:44:04.220682Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:04.220954Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:04.220956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:04.220959Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:04.220965Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:04.224419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341058677095450:2432], now have 1 active actors on pipe
2025-10-09T21:44:04.236105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341058677095465:4115] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:04.236123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341058677095465:4115] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:04.236127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341058677095465:4115] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:04.236136Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 242 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:04.238275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341058677095517:4115], now have 1 active actors on pipe
2025-10-09T21:44:04.238796Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 241
2025-10-09T21:44:04.239016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:04.239031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:04.239078Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|594f4810-450c0115-e40a0e82-86a11dc5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:04.239094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.239100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.239102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:04.239123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.239142Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:04.239166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:04.239537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:04.239543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:04.239558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:04.239714Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 242 partition: 0 MaxSeqNo: 25 sessionId: src_id|594f4810-450c0115-e40a0e82-86a11dc5_0
2025-10-09T21:44:04.240196Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|65f7bdb2-4bf75dd-1a462769-e10faeca_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|594f4810-450c0115-e40a0e82-86a11dc5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:04.243305Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|594f4810-450c0115-e40a0e82-86a11dc5_0] Write session: try to update token
2025-10-09T21:44:04.243330Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|594f4810-450c0115-e40a0e82-86a11dc5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:04.261801Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341058677095519:3810]
2025-10-09T21:44:04.259619Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:04.259640Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:04.259646Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:04.259648Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:04.259658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341058315285357:3825]
2025-10-09T21:44:04.259662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341058315285357:3825]
2025-10-09T21:44:04.259665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341058315285357:3825]
2025-10-09T21:44:04.259712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:04.259719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:04.259721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:04.259874Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:04.263267Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:04.263282Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:04.263286Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:04.263509Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 241, pipe: [1:7559341058677095519:3810]
2025-10-09T21:44:04.294746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 242 sessionId: src_id|594f4810-450c0115-e40a0e82-86a11dc5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:04.297678Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:04.562842Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:04.563147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341058677095519:3810], now have 1 active actors on pipe
2025-10-09T21:44:04.563211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:04.563220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:04.564020Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341058677095519:3810]
2025-10-09T21:44:04.564059Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:04.564062Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:04.564119Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 242 reason: error at write: Disconnected sessionId: src_id|594f4810-450c0115-e40a0e82-86a11dc5_0
2025-10-09T21:44:04.564306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 242 sessionId: src_id|594f4810-450c0115-e40a0e82-86a11dc5_0 is DEAD
2025-10-09T21:44:04.564461Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:04.564681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|594f4810-450c0115-e40a0e82-86a11dc5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:04.564693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|594f4810-450c0115-e40a0e82-86a11dc5_0] Write session will restart in 0.007014s
2025-10-09T21:44:04.564716Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|594f4810-450c0115-e40a0e82-86a11dc5_0] Write session: Do CDS request
2025-10-09T21:44:04.564725Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|594f4810-450c0115-e40a0e82-86a11dc5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:04.565189Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:04.572942Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|594f4810-450c0115-e40a0e82-86a11dc5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:04.574408Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:04.574442Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 243
2025-10-09T21:44:04.574786Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 243 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:04.574829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 243 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:04.574835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=243 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:04.574839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:04.582775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 243 sessionId:  describe result for acl check
2025-10-09T21:44:04.582836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:04.582838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:04.582839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:04.582847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341058677095544:4121] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:04.583894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341058677095544:4121] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:04.595991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341058677095544:4121] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:04.600849Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341058677095544:4121] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:04.600869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341058677095544:4121] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:04.600874Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341058677095544:4121] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:04.600895Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 243 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:04.624283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:04.627255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:04.627512Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:04.627515Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:04.627527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:04.627579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:04.627620Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341058315285438:3829]
2025-10-09T21:44:04.628161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:04.629678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:04.629736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:04.630106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:04.630174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:04.630176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:04.630177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:04.630178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:04.630180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:04.630181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:04.630183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:04.630184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:04.630185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:04.630192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:04.630203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:04.630207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:04.630211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:04.630215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:04.630220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:04.630223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:04.630227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:04.630231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:04.630240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:04.630249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:04.630256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:04.630262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:04.630268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:04.630273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:04.630278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:04.630283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:04.630287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:04.630292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:04.630300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:04.630306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:04.630311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:04.630316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:04.630321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:04.630327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:04.630342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:04.630348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:04.630384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:04.657019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:04.658192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:04.658200Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:04.658203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:04.658208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:04.658216Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 242 [2:7559341058315285438:3829]
2025-10-09T21:44:04.658233Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:04.658252Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:04.658255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.658258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.658261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.658281Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:04.658287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:04.658295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:04.658362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:04.658364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:04.658366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:04.658369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:04.658466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:04.658469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:04.658481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:04.658490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:04.658503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:04.658522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341058315285433:3829]
2025-10-09T21:44:04.658524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341058315285433:3829]
2025-10-09T21:44:04.658526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341058315285433:3829]
2025-10-09T21:44:04.658540Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:04.658554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:04.658568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:04.658571Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:04.658613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341058315285433:3829]
2025-10-09T21:44:04.658615Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:04.658786Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:04.658788Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:04.658791Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:04.658796Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:04.659827Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 242
2025-10-09T21:44:04.659860Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 242
2025-10-09T21:44:04.665152Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341058677095608:3810]
2025-10-09T21:44:04.666501Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 242, pipe: [1:7559341058677095608:3810]
2025-10-09T21:44:04.677075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341058677095583:4121], now have 1 active actors on pipe
2025-10-09T21:44:04.677111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341058677095537:2432], now have 1 active actors on pipe
2025-10-09T21:44:04.677143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:04.677151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:04.677178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341058677095608:3810], now have 1 active actors on pipe
2025-10-09T21:44:04.677205Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|42111b22-9ea33d8-c1e0d83a-4de33418_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:04.677219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.677224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.677227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:04.677241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.677261Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:04.677287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:04.680788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:04.680810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:04.680847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:04.681048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 243 partition: 0 MaxSeqNo: 25 sessionId: src_id|42111b22-9ea33d8-c1e0d83a-4de33418_0
2025-10-09T21:44:04.681441Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|594f4810-450c0115-e40a0e82-86a11dc5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|42111b22-9ea33d8-c1e0d83a-4de33418_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:04.684391Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42111b22-9ea33d8-c1e0d83a-4de33418_0] Write session: try to update token
2025-10-09T21:44:04.684416Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42111b22-9ea33d8-c1e0d83a-4de33418_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:04.751217Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:04.751231Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:04.751243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341058315285433:3829]
2025-10-09T21:44:04.751300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:04.751487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:04.751673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:04.751818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:04.751956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:04.752144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:04.752186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:04.752403Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:04.759194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.759208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.759212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.764386Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 243 sessionId: src_id|42111b22-9ea33d8-c1e0d83a-4de33418_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:04.767069Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:04.767800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:04.767819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:04.767861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:04.782411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:04.784577Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:04.784596Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:04.784599Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:04.784601Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:04.784613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341058315285433:3829]
2025-10-09T21:44:04.784617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341058315285433:3829]
2025-10-09T21:44:04.784621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341058315285433:3829]
2025-10-09T21:44:04.784664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:04.784672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:04.784675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:04.784831Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:04.786444Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:04.786450Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:04.786454Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:05.041511Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:05.061112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:05.061142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:05.061316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:05.061376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:05.061436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:05.061496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:05.061540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:05.061549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:05.061621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:05.061679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:05.061739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:05.061799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:05.078222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:05.078246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:05.078359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:05.078448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:05.078514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:05.078588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:05.078629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:05.078642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:05.078711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:05.078775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:05.078840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:05.078905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:05.078952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:05.078955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:05.079027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:05.079090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:05.079165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:05.079234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:05.079277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:05.079280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:05.079355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:05.079418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:05.079480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:05.079545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:05.079589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:05.079592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:05.079664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:05.079726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:05.079792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:05.079860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:05.079900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:05.079903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:05.079973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:05.080030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:05.080104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:05.080165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:05.080214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:05.080217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:05.080308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:05.080379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:05.080441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:05.080503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:05.080556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:05.080559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:05.080631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:05.080722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:05.080792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:05.080862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:05.080906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:05.080909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:05.082607Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42111b22-9ea33d8-c1e0d83a-4de33418_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:05.082619Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42111b22-9ea33d8-c1e0d83a-4de33418_0] Write session will restart in 0.007490s
2025-10-09T21:44:05.082643Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42111b22-9ea33d8-c1e0d83a-4de33418_0] Write session: Do CDS request
2025-10-09T21:44:05.081824Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341058677095608:3810]
2025-10-09T21:44:05.081860Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:05.081863Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:05.081892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 243 reason: error at write: Disconnected sessionId: src_id|42111b22-9ea33d8-c1e0d83a-4de33418_0
2025-10-09T21:44:05.082105Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 243 sessionId: src_id|42111b22-9ea33d8-c1e0d83a-4de33418_0 is DEAD
2025-10-09T21:44:05.082557Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:05.082650Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42111b22-9ea33d8-c1e0d83a-4de33418_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:05.085018Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:05.093954Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42111b22-9ea33d8-c1e0d83a-4de33418_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:05.095063Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:05.095086Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 244
2025-10-09T21:44:05.095260Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 244 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:05.095306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 244 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:05.095312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=244 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:05.095316Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:05.096393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 244 sessionId:  describe result for acl check
2025-10-09T21:44:05.096432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:05.096433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:05.096435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:05.096441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341062972062946:4126] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:05.096995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341062972062946:4126] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:05.105715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341062972062946:4126] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:05.107117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:05.109490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:05.109712Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:05.109716Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:05.109728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:05.109777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:05.109817Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341062610252837:3843]
2025-10-09T21:44:05.110564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:05.111454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:05.111513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:05.111875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:05.111924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:05.111926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:05.111927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:05.111929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:05.111930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:05.111931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:05.111932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.111936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.111937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.111942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:05.111952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:05.111957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:05.111961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:05.111964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:05.111967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:05.111970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.111973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.111976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.111986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:05.111995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:05.112001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:05.112005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:05.112009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:05.112015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:05.112020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:05.112025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:05.112029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:05.112033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:05.112038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:05.112043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:05.112051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.112055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.112059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.112063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.112068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.112074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.112099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:05.123061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341062972062946:4126] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:05.123085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341062972062946:4126] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:05.123090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341062972062946:4126] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:05.123101Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 244 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:05.125185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:05.126718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:05.126725Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:05.126731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:05.126737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:05.126745Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 243 [2:7559341062610252837:3843]
2025-10-09T21:44:05.126760Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:05.126780Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:05.126784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.126787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.126790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.126811Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:05.126817Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:05.126825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:05.126897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.126899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.126902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.126906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:05.126986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:05.126989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.127002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.127011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.127027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:05.127051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341062610252832:3843]
2025-10-09T21:44:05.127053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341062610252832:3843]
2025-10-09T21:44:05.127055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341062610252832:3843]
2025-10-09T21:44:05.127075Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:05.127090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:05.127104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:05.127107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:05.127154Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341062610252832:3843]
2025-10-09T21:44:05.127156Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:05.127432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:05.127441Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:05.127443Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:05.127450Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:05.128390Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 243
2025-10-09T21:44:05.128931Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 243
2025-10-09T21:44:05.133478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341062972062929:2432], now have 1 active actors on pipe
2025-10-09T21:44:05.133512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341062972062993:4126], now have 1 active actors on pipe
2025-10-09T21:44:05.133532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:05.133539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:05.133584Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|595bd9db-8ad1c573-5277812a-a87fc5e3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:05.133600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.133606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.133609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:05.133648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.133669Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:05.133691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:05.134197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:05.134205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:05.134221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:05.134372Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 244 partition: 0 MaxSeqNo: 25 sessionId: src_id|595bd9db-8ad1c573-5277812a-a87fc5e3_0
2025-10-09T21:44:05.134796Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42111b22-9ea33d8-c1e0d83a-4de33418_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|595bd9db-8ad1c573-5277812a-a87fc5e3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:05.137461Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|595bd9db-8ad1c573-5277812a-a87fc5e3_0] Write session: try to update token
2025-10-09T21:44:05.137486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|595bd9db-8ad1c573-5277812a-a87fc5e3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:05.168583Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:05.168602Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:05.168605Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:05.168607Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:05.168617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341062610252832:3843]
2025-10-09T21:44:05.168621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341062610252832:3843]
2025-10-09T21:44:05.168625Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341062610252832:3843]
2025-10-09T21:44:05.168671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:05.168678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.168681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:05.168814Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:05.177240Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 244 sessionId: src_id|595bd9db-8ad1c573-5277812a-a87fc5e3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:05.180037Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:05.180463Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:05.180485Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:05.180489Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:05.184575Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341062972063002:3810]
2025-10-09T21:44:05.185123Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 243, pipe: [1:7559341062972063002:3810]
2025-10-09T21:44:05.471753Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:05.472055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:05.472066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:05.472091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341062972063002:3810], now have 1 active actors on pipe
2025-10-09T21:44:05.472871Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:05.472880Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:05.472894Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:05.472952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 244 reason: error at write: Disconnected sessionId: src_id|595bd9db-8ad1c573-5277812a-a87fc5e3_0
2025-10-09T21:44:05.473127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 244 sessionId: src_id|595bd9db-8ad1c573-5277812a-a87fc5e3_0 is DEAD
2025-10-09T21:44:05.473381Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341062972063002:3810]
2025-10-09T21:44:05.473397Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:05.474128Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|595bd9db-8ad1c573-5277812a-a87fc5e3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:05.474141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|595bd9db-8ad1c573-5277812a-a87fc5e3_0] Write session will restart in 0.005796s
2025-10-09T21:44:05.474167Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|595bd9db-8ad1c573-5277812a-a87fc5e3_0] Write session: Do CDS request
2025-10-09T21:44:05.474175Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|595bd9db-8ad1c573-5277812a-a87fc5e3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:05.481482Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|595bd9db-8ad1c573-5277812a-a87fc5e3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:05.490566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:05.491218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:05.491421Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:05.491433Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:05.491446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:05.491505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:05.491554Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341062610252913:3847]
2025-10-09T21:44:05.491881Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:05.491895Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 245
2025-10-09T21:44:05.492090Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 245 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:05.492126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 245 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:05.492133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=245 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:05.492136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:05.492117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:05.492596Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 245 sessionId:  describe result for acl check
2025-10-09T21:44:05.492631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:05.492633Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:05.492634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:05.492640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341062972063060:4138] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:05.493192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341062972063060:4138] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:05.494532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:05.494592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:05.494908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:05.494983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:05.494986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:05.494987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:05.494988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:05.494989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:05.494990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:05.494991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.494992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.494993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.494999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:05.495008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:05.495012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:05.495015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:05.495018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:05.495022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:05.495025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.495028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.495032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.495040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:05.495048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:05.495053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:05.495058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:05.495062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:05.495066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:05.495070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:05.495074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:05.495078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:05.495082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:05.495086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:05.495090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:05.495094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.495098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.495103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.495108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.495112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.495116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.495142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:05.501899Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341062972063060:4138] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:05.507881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:05.509168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:05.509175Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:05.509178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:05.509183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:05.509190Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 244 [2:7559341062610252913:3847]
2025-10-09T21:44:05.509205Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:05.509224Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:05.509227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.509230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.509232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.509249Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:05.509255Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:05.509262Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:05.509331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.509333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.509335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.509337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:05.509420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:05.509423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.509435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.509445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.509460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:05.509483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341062610252908:3847]
2025-10-09T21:44:05.509485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341062610252908:3847]
2025-10-09T21:44:05.509487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341062610252908:3847]
2025-10-09T21:44:05.509501Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:05.509518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:05.509532Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:05.509535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:05.509577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341062610252908:3847]
2025-10-09T21:44:05.509579Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:05.509952Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:05.509962Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:05.509964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:05.509970Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:05.511106Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 244
2025-10-09T21:44:05.511733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341062972063033:2432], now have 1 active actors on pipe
2025-10-09T21:44:05.515081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341062972063060:4138] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:05.515102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341062972063060:4138] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:05.515106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341062972063060:4138] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:05.515116Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 245 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:05.516249Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 244
2025-10-09T21:44:05.516416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341062972063099:4138], now have 1 active actors on pipe
2025-10-09T21:44:05.516446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:05.516455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:05.516775Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4aa50720-a9887b32-fc723324-5e7ccc76_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:05.516793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.516799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.516802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:05.516816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.516847Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:05.516869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:05.517224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:05.517232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:05.517248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:05.517494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 245 partition: 0 MaxSeqNo: 25 sessionId: src_id|4aa50720-a9887b32-fc723324-5e7ccc76_0
2025-10-09T21:44:05.521059Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|595bd9db-8ad1c573-5277812a-a87fc5e3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4aa50720-a9887b32-fc723324-5e7ccc76_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:05.523741Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4aa50720-a9887b32-fc723324-5e7ccc76_0] Write session: try to update token
2025-10-09T21:44:05.523762Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4aa50720-a9887b32-fc723324-5e7ccc76_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:05.536898Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:05.536918Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:05.536920Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:05.536923Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:05.536934Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341062610252908:3847]
2025-10-09T21:44:05.536939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341062610252908:3847]
2025-10-09T21:44:05.536942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341062610252908:3847]
2025-10-09T21:44:05.536986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:05.536993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.536996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:05.537156Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:05.543026Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:05.543038Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:05.543042Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:05.555287Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 245 sessionId: src_id|4aa50720-a9887b32-fc723324-5e7ccc76_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:05.557715Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:05.573184Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341062972063103:3810]
2025-10-09T21:44:05.578239Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 244, pipe: [1:7559341062972063103:3810]
2025-10-09T21:44:05.745082Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:05.745984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:05.746002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:05.746056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341062972063103:3810], now have 1 active actors on pipe
2025-10-09T21:44:05.746926Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341062972063103:3810]
2025-10-09T21:44:05.746960Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:05.746963Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:05.747025Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 245 reason: error at write: Disconnected sessionId: src_id|4aa50720-a9887b32-fc723324-5e7ccc76_0
2025-10-09T21:44:05.747193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 245 sessionId: src_id|4aa50720-a9887b32-fc723324-5e7ccc76_0 is DEAD
2025-10-09T21:44:05.747335Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:05.747551Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:05.749470Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4aa50720-a9887b32-fc723324-5e7ccc76_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:05.749481Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4aa50720-a9887b32-fc723324-5e7ccc76_0] Write session will restart in 0.009163s
2025-10-09T21:44:05.749506Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4aa50720-a9887b32-fc723324-5e7ccc76_0] Write session: Do CDS request
2025-10-09T21:44:05.749513Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4aa50720-a9887b32-fc723324-5e7ccc76_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:05.762783Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4aa50720-a9887b32-fc723324-5e7ccc76_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:05.764354Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:05.764369Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 246
2025-10-09T21:44:05.764565Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 246 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:05.764591Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 246 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:05.764597Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=246 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:05.764608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:05.765444Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 246 sessionId:  describe result for acl check
2025-10-09T21:44:05.765474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:05.765476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:05.765477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:05.765485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341062972063136:4142] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:05.765981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341062972063136:4142] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:05.773725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341062972063136:4142] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:05.779100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:05.779295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:05.779459Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:05.779463Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:05.779474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:05.779513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:05.779555Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341062610252989:3849]
2025-10-09T21:44:05.780125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:05.781774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341062972063136:4142] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:05.781789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341062972063136:4142] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:05.781793Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341062972063136:4142] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:05.781802Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 246 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:05.786635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:05.786717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:05.787089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:05.787134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:05.787136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:05.787137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:05.787138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:05.787140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:05.787141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:05.787142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.787143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.787144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.787151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:05.787175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:05.787179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:05.787182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:05.787185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:05.787189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:05.787193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.787196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.787199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.787206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:05.787214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:05.787219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:05.787224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:05.787228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:05.787233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:05.787236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:05.787240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:05.787245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:05.787251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:05.787255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:05.787260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:05.787265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.787270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.787274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.787279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.787284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.787288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.787315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:05.803753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:05.804847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:05.804852Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:05.804856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:05.804861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:05.804870Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 245 [2:7559341062610252989:3849]
2025-10-09T21:44:05.804888Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:05.804910Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:05.804914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.804918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.804920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.804941Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:05.804948Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:05.804956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:05.805028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.805030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.805033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.805036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:05.805119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:05.805122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.805136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:05.805147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:05.805161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:05.805181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341062610252984:3849]
2025-10-09T21:44:05.805183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341062610252984:3849]
2025-10-09T21:44:05.805187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341062610252984:3849]
2025-10-09T21:44:05.805202Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:05.805218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:05.805232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:05.805235Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:05.805278Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341062610252984:3849]
2025-10-09T21:44:05.805281Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:05.806929Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 245
2025-10-09T21:44:05.806963Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 245
2025-10-09T21:44:05.805862Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:05.805871Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:05.805873Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:05.805879Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:05.809006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341062972063173:4142], now have 1 active actors on pipe
2025-10-09T21:44:05.809022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341062972063108:2432], now have 1 active actors on pipe
2025-10-09T21:44:05.809042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:05.809049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:05.809091Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fe060232-a8d12a00-211840f5-74734285_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:05.809106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.809111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.809114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:05.809128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.809158Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:05.809182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:05.811288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:05.811304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:05.811348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:05.811603Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 246 partition: 0 MaxSeqNo: 25 sessionId: src_id|fe060232-a8d12a00-211840f5-74734285_0
2025-10-09T21:44:05.815044Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4aa50720-a9887b32-fc723324-5e7ccc76_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fe060232-a8d12a00-211840f5-74734285_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:05.817506Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe060232-a8d12a00-211840f5-74734285_0] Write session: try to update token
2025-10-09T21:44:05.817525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe060232-a8d12a00-211840f5-74734285_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:05.841961Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:05.841980Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:05.841991Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341062610252984:3849]
2025-10-09T21:44:05.842079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:05.842259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:05.842397Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:05.842549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:05.842681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:05.842889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:05.842926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:05.843625Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:05.845490Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341062972063184:3810]
2025-10-09T21:44:05.848003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341062972063184:3810], now have 1 active actors on pipe
2025-10-09T21:44:05.848309Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 245, pipe: [1:7559341062972063184:3810]
2025-10-09T21:44:05.877868Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 246 sessionId: src_id|fe060232-a8d12a00-211840f5-74734285_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:05.880759Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:05.881479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:05.881504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:05.881547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:05.886332Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:05.898745Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:05.898767Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:05.898770Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:05.898772Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:05.898784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341062610252984:3849]
2025-10-09T21:44:05.898789Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341062610252984:3849]
2025-10-09T21:44:05.898793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341062610252984:3849]
2025-10-09T21:44:05.898843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:05.898852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:05.898855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:05.899007Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:05.906120Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:05.906150Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:05.906154Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:06.200811Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:06.209679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:06.210500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:06.210698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:06.210770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:06.210833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:06.210894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:06.210944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:06.210949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:06.211040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:06.211106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:06.211166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:06.211225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:06.211264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:06.211270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:06.211335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:06.211394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:06.211450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:06.211512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:06.212124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:06.212127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:06.212197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:06.212253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:06.212310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:06.212375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:06.212415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:06.212418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:06.212492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:06.212555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:06.212616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:06.212677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:06.212819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:06.212823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:06.212894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:06.212955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:06.213020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:06.213081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:06.213121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:06.213124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:06.213186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:06.213238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:06.213295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:06.213355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:06.213393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:06.213396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:06.213456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:06.213511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:06.213566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:06.213622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:06.213662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:06.213664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:06.213728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:06.213781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:06.213836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:06.213894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:06.213937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:06.213940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:06.214003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:06.215346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:06.215430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:06.215486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:06.215526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:06.215529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:06.218961Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341062972063184:3810]
2025-10-09T21:44:06.218989Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:06.219031Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:06.219033Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:06.219065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 246 reason: error at write: Disconnected sessionId: src_id|fe060232-a8d12a00-211840f5-74734285_0
2025-10-09T21:44:06.219259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 246 sessionId: src_id|fe060232-a8d12a00-211840f5-74734285_0 is DEAD
2025-10-09T21:44:06.219924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe060232-a8d12a00-211840f5-74734285_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:06.219934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe060232-a8d12a00-211840f5-74734285_0] Write session will restart in 0.008198s
2025-10-09T21:44:06.219958Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe060232-a8d12a00-211840f5-74734285_0] Write session: Do CDS request
2025-10-09T21:44:06.219964Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe060232-a8d12a00-211840f5-74734285_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:06.224837Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:06.230468Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:06.230485Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 247
2025-10-09T21:44:06.230943Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe060232-a8d12a00-211840f5-74734285_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:06.231157Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 247 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:06.231196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 247 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:06.231203Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=247 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:06.231207Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:06.231762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 247 sessionId:  describe result for acl check
2025-10-09T21:44:06.231791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:06.231806Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:06.231808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:06.231814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341067267030502:4147] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:06.232346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341067267030502:4147] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:06.266430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341067267030502:4147] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:06.298145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341067267030502:4147] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:06.298164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341067267030502:4147] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:06.298169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341067267030502:4147] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:06.298179Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 247 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:06.311775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:06.312611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:06.312821Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:06.312825Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:06.312838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:06.312889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:06.312929Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341066905220376:3859]
2025-10-09T21:44:06.313505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:06.322750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:06.322847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:06.323275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:06.323322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:06.323324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:06.323325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:06.323327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:06.323328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:06.323330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:06.323332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:06.323333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:06.323335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:06.323343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:06.323357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:06.323362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:06.323366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:06.323372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:06.323377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:06.323382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:06.323386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:06.323391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:06.323400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:06.323413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:06.323421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:06.323427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:06.323433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:06.323439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:06.323444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:06.323449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:06.323455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:06.323461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:06.323466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:06.323472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:06.323478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:06.323484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:06.323489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:06.323495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:06.323501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:06.323507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:06.323544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:06.335486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:06.336651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:06.336657Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:06.336661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:06.336665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:06.336674Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 246 [2:7559341066905220376:3859]
2025-10-09T21:44:06.336690Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:06.336708Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:06.336712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.336716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.336719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.336739Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:06.336748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:06.336758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:06.336822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:06.336824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:06.336826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:06.336829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:06.336908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:06.336911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:06.336924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:06.336938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:06.336956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:06.336977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341066905220371:3859]
2025-10-09T21:44:06.336980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341066905220371:3859]
2025-10-09T21:44:06.336982Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341066905220371:3859]
2025-10-09T21:44:06.336995Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:06.337009Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:06.337025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:06.337028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:06.337068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341066905220371:3859]
2025-10-09T21:44:06.337070Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:06.337360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:06.337367Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:06.337369Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:06.337375Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:06.338634Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 246
2025-10-09T21:44:06.338672Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 246
2025-10-09T21:44:06.338692Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341067267030571:3810]
2025-10-09T21:44:06.339400Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 246, pipe: [1:7559341067267030571:3810]
2025-10-09T21:44:06.345635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341067267030498:2432], now have 1 active actors on pipe
2025-10-09T21:44:06.345660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341067267030558:4147], now have 1 active actors on pipe
2025-10-09T21:44:06.345704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:06.345716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:06.345735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341067267030571:3810], now have 1 active actors on pipe
2025-10-09T21:44:06.345762Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f0ab96e-b05a4123-40ba7264-ed0d2fab_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:06.345775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.345780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.345783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:06.345798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.345816Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:06.345840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:06.349537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:06.349564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:06.349606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:06.352310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 247 partition: 0 MaxSeqNo: 25 sessionId: src_id|f0ab96e-b05a4123-40ba7264-ed0d2fab_0
2025-10-09T21:44:06.352876Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe060232-a8d12a00-211840f5-74734285_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f0ab96e-b05a4123-40ba7264-ed0d2fab_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:06.355462Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0ab96e-b05a4123-40ba7264-ed0d2fab_0] Write session: try to update token
2025-10-09T21:44:06.355481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0ab96e-b05a4123-40ba7264-ed0d2fab_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:06.410533Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:06.410560Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:06.410563Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:06.410565Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:06.410574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341066905220371:3859]
2025-10-09T21:44:06.410578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341066905220371:3859]
2025-10-09T21:44:06.410582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341066905220371:3859]
2025-10-09T21:44:06.410629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:06.410637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:06.410639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:06.410969Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:06.414291Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:06.414304Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:06.414314Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:06.434899Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 247 sessionId: src_id|f0ab96e-b05a4123-40ba7264-ed0d2fab_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:06.437471Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:06.807842Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:06.808563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:06.808577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:06.809198Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:06.809248Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:06.809251Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:06.809308Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 247 reason: error at write: Disconnected sessionId: src_id|f0ab96e-b05a4123-40ba7264-ed0d2fab_0
2025-10-09T21:44:06.809502Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 247 sessionId: src_id|f0ab96e-b05a4123-40ba7264-ed0d2fab_0 is DEAD
2025-10-09T21:44:06.809636Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341067267030571:3810]
2025-10-09T21:44:06.809971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0ab96e-b05a4123-40ba7264-ed0d2fab_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:06.809979Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0ab96e-b05a4123-40ba7264-ed0d2fab_0] Write session will restart in 0.005336s
2025-10-09T21:44:06.810002Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0ab96e-b05a4123-40ba7264-ed0d2fab_0] Write session: Do CDS request
2025-10-09T21:44:06.810008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0ab96e-b05a4123-40ba7264-ed0d2fab_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:06.809308Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:06.819754Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0ab96e-b05a4123-40ba7264-ed0d2fab_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:06.821612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:06.821629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 248
2025-10-09T21:44:06.821876Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 248 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:06.821907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 248 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:06.821913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=248 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:06.821917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:06.822620Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 248 sessionId:  describe result for acl check
2025-10-09T21:44:06.822655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:06.822657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:06.822658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:06.822665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341067267030622:4161] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:06.823194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341067267030622:4161] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:06.832817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341067267030622:4161] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:06.841670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341067267030622:4161] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:06.841691Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341067267030622:4161] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:06.841695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341067267030622:4161] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:06.841706Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 248 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:06.847296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:06.847491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:06.847774Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:06.847778Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:06.847793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:06.847856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:06.847904Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341066905220474:3867]
2025-10-09T21:44:06.848481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:06.852361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:06.852467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:06.852851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:06.852907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:06.852909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:06.852910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:06.852911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:06.852913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:06.852914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:06.852915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:06.852916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:06.852918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:06.852924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:06.852935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:06.852940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:06.852944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:06.852948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:06.852952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:06.852956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:06.852960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:06.852963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:06.852971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:06.852981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:06.852987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:06.852993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:06.852998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:06.853003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:06.853008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:06.853013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:06.853018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:06.853023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:06.853027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:06.853032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:06.853036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:06.853041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:06.853044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:06.853049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:06.853055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:06.853060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:06.853091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:06.864968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:06.867001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:06.867007Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:06.867010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:06.867015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:06.867023Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 247 [2:7559341066905220474:3867]
2025-10-09T21:44:06.867039Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:06.867062Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:06.867066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.867069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.867071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.867090Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:06.867097Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:06.867105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:06.867171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:06.867173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:06.867175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:06.867178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:06.867260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:06.867262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:06.867277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:06.867288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:06.867306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:06.867328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341066905220469:3867]
2025-10-09T21:44:06.867331Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341066905220469:3867]
2025-10-09T21:44:06.867333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341066905220469:3867]
2025-10-09T21:44:06.867343Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:06.867358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:06.867374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:06.867377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:06.867422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341066905220469:3867]
2025-10-09T21:44:06.867427Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:06.868976Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 247
2025-10-09T21:44:06.869011Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 247
2025-10-09T21:44:06.868016Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:06.868027Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:06.868029Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:06.868035Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:06.876302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341067267030673:4161], now have 1 active actors on pipe
2025-10-09T21:44:06.876331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341067267030611:2432], now have 1 active actors on pipe
2025-10-09T21:44:06.876378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:06.876388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:06.876437Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c1d2322b-91267022-dcc497f9-c93b6f70_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:06.876450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.876457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.876460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:06.876481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.876501Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:06.876527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:06.878245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:06.878277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:06.878311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:06.878760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 248 partition: 0 MaxSeqNo: 25 sessionId: src_id|c1d2322b-91267022-dcc497f9-c93b6f70_0
2025-10-09T21:44:06.880930Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0ab96e-b05a4123-40ba7264-ed0d2fab_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c1d2322b-91267022-dcc497f9-c93b6f70_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:06.901199Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1d2322b-91267022-dcc497f9-c93b6f70_0] Write session: try to update token
2025-10-09T21:44:06.901227Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1d2322b-91267022-dcc497f9-c93b6f70_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:06.909144Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341067267030691:3810]
2025-10-09T21:44:06.912165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341067267030691:3810], now have 1 active actors on pipe
2025-10-09T21:44:06.912379Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 247, pipe: [1:7559341067267030691:3810]
2025-10-09T21:44:06.927388Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:06.927407Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:06.927411Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:06.927413Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:06.927424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341066905220469:3867]
2025-10-09T21:44:06.927428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341066905220469:3867]
2025-10-09T21:44:06.927433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341066905220469:3867]
2025-10-09T21:44:06.927480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:06.927488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:06.927491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:06.927871Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:06.931030Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:06.931049Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:06.931053Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:06.982209Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 248 sessionId: src_id|c1d2322b-91267022-dcc497f9-c93b6f70_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:06.984912Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:07.334679Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:07.335062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:07.335073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:07.335973Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:07.335985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:07.336036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 248 reason: error at write: Disconnected sessionId: src_id|c1d2322b-91267022-dcc497f9-c93b6f70_0
2025-10-09T21:44:07.336229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 248 sessionId: src_id|c1d2322b-91267022-dcc497f9-c93b6f70_0 is DEAD
2025-10-09T21:44:07.336361Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:07.336392Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341067267030691:3810]
2025-10-09T21:44:07.337430Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:07.338361Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1d2322b-91267022-dcc497f9-c93b6f70_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:07.338371Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1d2322b-91267022-dcc497f9-c93b6f70_0] Write session will restart in 0.005394s
2025-10-09T21:44:07.338394Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1d2322b-91267022-dcc497f9-c93b6f70_0] Write session: Do CDS request
2025-10-09T21:44:07.338404Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1d2322b-91267022-dcc497f9-c93b6f70_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:07.344770Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1d2322b-91267022-dcc497f9-c93b6f70_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:07.345177Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:07.345194Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 249
2025-10-09T21:44:07.345607Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 249 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:07.345646Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 249 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:07.345651Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=249 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:07.345657Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:07.347968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 249 sessionId:  describe result for acl check
2025-10-09T21:44:07.348008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:07.348010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:07.348013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:07.348019Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341071561998013:4166] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:07.348609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341071561998013:4166] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:07.377016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341071561998013:4166] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:07.378734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:07.384081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:07.384342Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:07.384347Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:07.384359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:07.384420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:07.384465Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341071200187860:3871]
2025-10-09T21:44:07.385122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:07.387683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:07.387745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:07.388141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:07.388185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:07.388187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:07.388188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:07.388189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:07.388191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:07.388192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:07.388193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:07.388195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:07.388196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:07.388202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:07.388213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:07.388217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:07.388221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:07.388225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:07.388231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:07.388235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:07.388239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:07.388242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:07.388251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:07.388260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:07.388266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:07.388271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:07.388276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:07.388281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:07.388285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:07.388290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:07.388295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:07.388300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:07.388304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:07.388309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:07.388313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:07.388319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:07.388323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:07.388328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:07.388332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:07.388337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:07.388367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:07.409956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341071561998013:4166] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:07.409974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341071561998013:4166] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:07.409979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341071561998013:4166] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:07.410000Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 249 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:07.414352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:07.415574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:07.415582Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:07.415584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:07.415589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:07.415597Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 248 [2:7559341071200187860:3871]
2025-10-09T21:44:07.415611Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:07.415631Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:07.415642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.415646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.415649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.415670Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:07.415676Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:07.415684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:07.415755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:07.415757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:07.415759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:07.415762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:07.415948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:07.415951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:07.415965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:07.415977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:07.415996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:07.416019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341071200187855:3871]
2025-10-09T21:44:07.416022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341071200187855:3871]
2025-10-09T21:44:07.416024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341071200187855:3871]
2025-10-09T21:44:07.416042Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:07.416058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:07.416074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:07.416077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:07.416123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341071200187855:3871]
2025-10-09T21:44:07.416130Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:07.416364Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:07.416374Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:07.416378Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:07.416385Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:07.419706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341071561998070:4166], now have 1 active actors on pipe
2025-10-09T21:44:07.419722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341071561998000:2432], now have 1 active actors on pipe
2025-10-09T21:44:07.421484Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 248
2025-10-09T21:44:07.421516Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 248
2025-10-09T21:44:07.424045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:07.424073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:07.424123Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|441a966e-d2065306-49db74ae-746f088e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:07.424143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.424152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.424155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:07.424169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.424197Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:07.424220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:07.426849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 249 partition: 0 MaxSeqNo: 25 sessionId: src_id|441a966e-d2065306-49db74ae-746f088e_0
2025-10-09T21:44:07.427311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1d2322b-91267022-dcc497f9-c93b6f70_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|441a966e-d2065306-49db74ae-746f088e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:07.426629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:07.426659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:07.426689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:07.430406Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|441a966e-d2065306-49db74ae-746f088e_0] Write session: try to update token
2025-10-09T21:44:07.430445Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|441a966e-d2065306-49db74ae-746f088e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:07.436414Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341071561998076:3810]
2025-10-09T21:44:07.437173Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 248, pipe: [1:7559341071561998076:3810]
2025-10-09T21:44:07.437275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341071561998076:3810], now have 1 active actors on pipe
2025-10-09T21:44:07.489253Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 249 sessionId: src_id|441a966e-d2065306-49db74ae-746f088e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:07.491981Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:07.492636Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:07.492659Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:07.492661Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:07.492663Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:07.492672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341071200187855:3871]
2025-10-09T21:44:07.492677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341071200187855:3871]
2025-10-09T21:44:07.492680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341071200187855:3871]
2025-10-09T21:44:07.492731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:07.492739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:07.492741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:07.492893Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:07.494055Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:07.494072Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:07.494076Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:07.731117Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:07.733152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:07.733175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:07.735983Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:07.736995Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:07.737084Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 249 reason: status is not ok: tablet will be restarted right now sessionId: src_id|441a966e-d2065306-49db74ae-746f088e_0
2025-10-09T21:44:07.737262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 249 sessionId: src_id|441a966e-d2065306-49db74ae-746f088e_0 is DEAD
2025-10-09T21:44:07.737391Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341071561998076:3810]
2025-10-09T21:44:07.737422Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:07.737912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|441a966e-d2065306-49db74ae-746f088e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:07.737922Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|441a966e-d2065306-49db74ae-746f088e_0] Write session will restart in 0.005618s
2025-10-09T21:44:07.737946Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|441a966e-d2065306-49db74ae-746f088e_0] Write session: Do CDS request
2025-10-09T21:44:07.737953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|441a966e-d2065306-49db74ae-746f088e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:07.744667Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|441a966e-d2065306-49db74ae-746f088e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:07.750159Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:07.750176Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 250
2025-10-09T21:44:07.751342Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 250 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:07.751398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 250 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:07.751406Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=250 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:07.751409Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:07.752056Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 250 sessionId:  describe result for acl check
2025-10-09T21:44:07.752094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:07.752096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:07.752098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:07.752104Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341071561998141:4178] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:07.752643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341071561998141:4178] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:07.756715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341071561998141:4178] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:07.762890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:07.763132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:07.763588Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:07.763598Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:07.763613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:07.763670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:07.763716Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341071200187958:3885]
2025-10-09T21:44:07.764374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:07.765994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:07.766090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:07.766478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:07.766559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:07.766560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:07.766562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:07.766563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:07.766565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:07.766566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:07.766567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:07.766568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:07.766570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:07.766577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:07.766591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:07.766594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:07.766598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:07.766601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:07.766605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:07.766628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:07.766632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:07.766637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:07.766646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:07.766655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:07.766661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:07.766667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:07.766671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:07.766676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:07.766681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:07.766687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:07.766692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:07.766698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:07.766702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:07.766709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:07.766714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:07.766719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:07.766724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:07.766730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:07.766736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:07.766741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:07.766778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:07.780475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341071561998141:4178] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:07.780493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341071561998141:4178] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:07.780498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341071561998141:4178] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:07.780508Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 250 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:07.781944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:07.783607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:07.783627Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:07.783629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:07.783634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:07.783642Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 249 [2:7559341071200187958:3885]
2025-10-09T21:44:07.783656Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:07.783678Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:07.783686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.783689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.783692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.783712Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:07.783718Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:07.783732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:07.783802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:07.783811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:07.783812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:07.783815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:07.783893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:07.783896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:07.783910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:07.783925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:07.783940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:07.783963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341071200187953:3885]
2025-10-09T21:44:07.783966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341071200187953:3885]
2025-10-09T21:44:07.783968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341071200187953:3885]
2025-10-09T21:44:07.783977Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:07.783992Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:07.784007Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:07.784009Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:07.784043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:07.784046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:07.784049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:07.784057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341071200187953:3885]
2025-10-09T21:44:07.784061Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:07.784801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 249
2025-10-09T21:44:07.784832Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 249
2025-10-09T21:44:07.784089Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:07.784112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341071561998178:4178], now have 1 active actors on pipe
2025-10-09T21:44:07.784116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341071561998111:2432], now have 1 active actors on pipe
2025-10-09T21:44:07.786622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:07.786650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:07.786694Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f12eedbc-cf977d68-c26ccd16-2bc5c264_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:07.786707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.786723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.786727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:07.786740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.786757Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:07.786787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:07.789361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:07.789377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:07.789408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:07.789573Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 250 partition: 0 MaxSeqNo: 25 sessionId: src_id|f12eedbc-cf977d68-c26ccd16-2bc5c264_0
2025-10-09T21:44:07.790020Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|441a966e-d2065306-49db74ae-746f088e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f12eedbc-cf977d68-c26ccd16-2bc5c264_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:07.792632Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f12eedbc-cf977d68-c26ccd16-2bc5c264_0] Write session: try to update token
2025-10-09T21:44:07.792654Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f12eedbc-cf977d68-c26ccd16-2bc5c264_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:07.838209Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341071561998186:3810]
2025-10-09T21:44:07.837923Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:07.837943Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:07.837946Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:07.837948Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:07.837965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341071200187953:3885]
2025-10-09T21:44:07.837970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341071200187953:3885]
2025-10-09T21:44:07.837973Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341071200187953:3885]
2025-10-09T21:44:07.838019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:07.838026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:07.838046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:07.838186Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:07.848611Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:07.848628Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:07.848632Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:07.849330Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 249, pipe: [1:7559341071561998186:3810]
2025-10-09T21:44:07.857086Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 250 sessionId: src_id|f12eedbc-cf977d68-c26ccd16-2bc5c264_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:07.859734Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:08.125595Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:08.126087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341071561998186:3810], now have 1 active actors on pipe
2025-10-09T21:44:08.126124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:08.126132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:08.127528Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:08.127542Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:08.127603Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 250 reason: error at write: Disconnected sessionId: src_id|f12eedbc-cf977d68-c26ccd16-2bc5c264_0
2025-10-09T21:44:08.127784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 250 sessionId: src_id|f12eedbc-cf977d68-c26ccd16-2bc5c264_0 is DEAD
2025-10-09T21:44:08.127919Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:08.127959Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341071561998186:3810]
2025-10-09T21:44:08.128516Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f12eedbc-cf977d68-c26ccd16-2bc5c264_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:08.128525Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f12eedbc-cf977d68-c26ccd16-2bc5c264_0] Write session will restart in 0.009593s
2025-10-09T21:44:08.128550Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f12eedbc-cf977d68-c26ccd16-2bc5c264_0] Write session: Do CDS request
2025-10-09T21:44:08.128559Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f12eedbc-cf977d68-c26ccd16-2bc5c264_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:08.128194Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:08.140089Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f12eedbc-cf977d68-c26ccd16-2bc5c264_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:08.140441Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:08.140460Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 251
2025-10-09T21:44:08.140991Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 251 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:08.141037Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 251 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:08.141044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=251 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:08.141048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:08.141792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 251 sessionId:  describe result for acl check
2025-10-09T21:44:08.141837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:08.141839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:08.141840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:08.141847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341075856965508:4183] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:08.142538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341075856965508:4183] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:08.150129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341075856965508:4183] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:08.166412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341075856965508:4183] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:08.166442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341075856965508:4183] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:08.166448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341075856965508:4183] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:08.166459Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 251 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:08.186684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:08.188818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:08.189112Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:08.189116Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:08.189129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:08.189196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:08.189241Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341075495155335:3889]
2025-10-09T21:44:08.189851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:08.190456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:08.190515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:08.190845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:08.190908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:08.190910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:08.190911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:08.190912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:08.190915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:08.190916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:08.190918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.190919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.190920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.190927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:08.190938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:08.190942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:08.190946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:08.190950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:08.190953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:08.190957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.190961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.190964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.190972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:08.190980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:08.190987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:08.190994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:08.190999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:08.191003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:08.191007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:08.191011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:08.191015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:08.191019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:08.191023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:08.191028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:08.191033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.191037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.191042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.191046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.191051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.191058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.191107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:08.202091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:08.203329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:08.203335Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:08.203338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:08.203342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:08.203351Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 250 [2:7559341075495155335:3889]
2025-10-09T21:44:08.203368Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:08.203391Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:08.203395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.203399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.203401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.203420Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:08.203426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:08.203434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:08.203505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.203507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.203509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.203512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:08.203591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:08.203594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.203605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.203616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.203631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:08.203656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341075495155330:3889]
2025-10-09T21:44:08.203659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341075495155330:3889]
2025-10-09T21:44:08.203661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341075495155330:3889]
2025-10-09T21:44:08.203676Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:08.203691Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:08.203708Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:08.203711Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:08.203754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341075495155330:3889]
2025-10-09T21:44:08.203756Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:08.205250Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:08.205260Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:08.205263Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:08.205270Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:08.205934Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 250
2025-10-09T21:44:08.205979Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 250
2025-10-09T21:44:08.213523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341075856965548:4183], now have 1 active actors on pipe
2025-10-09T21:44:08.213543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341075856965502:2432], now have 1 active actors on pipe
2025-10-09T21:44:08.213569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:08.213577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:08.213628Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4e0a9f18-52a8de9a-d2c1fe6c-d9b45316_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:08.213642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.213660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.213664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:08.213680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.213700Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:08.213726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:08.217560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:08.217584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:08.217629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:08.218284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 251 partition: 0 MaxSeqNo: 25 sessionId: src_id|4e0a9f18-52a8de9a-d2c1fe6c-d9b45316_0
2025-10-09T21:44:08.218781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f12eedbc-cf977d68-c26ccd16-2bc5c264_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4e0a9f18-52a8de9a-d2c1fe6c-d9b45316_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:08.221433Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e0a9f18-52a8de9a-d2c1fe6c-d9b45316_0] Write session: try to update token
2025-10-09T21:44:08.221457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e0a9f18-52a8de9a-d2c1fe6c-d9b45316_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:08.227614Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341075856965578:3810]
2025-10-09T21:44:08.228378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341075856965578:3810], now have 1 active actors on pipe
2025-10-09T21:44:08.228691Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 250, pipe: [1:7559341075856965578:3810]
2025-10-09T21:44:08.259517Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:08.259537Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:08.259540Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:08.259542Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:08.259552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341075495155330:3889]
2025-10-09T21:44:08.259557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341075495155330:3889]
2025-10-09T21:44:08.259564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341075495155330:3889]
2025-10-09T21:44:08.259614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:08.259621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.259624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:08.259765Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:08.262204Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:08.262233Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:08.262237Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:08.275844Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 251 sessionId: src_id|4e0a9f18-52a8de9a-d2c1fe6c-d9b45316_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:08.278450Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:08.497548Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:08.502360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:08.502385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:08.503405Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:08.503701Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:08.503715Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:08.503772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 251 reason: error at write: Disconnected sessionId: src_id|4e0a9f18-52a8de9a-d2c1fe6c-d9b45316_0
2025-10-09T21:44:08.503850Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:08.503891Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341075856965578:3810]
2025-10-09T21:44:08.503945Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 251 sessionId: src_id|4e0a9f18-52a8de9a-d2c1fe6c-d9b45316_0 is DEAD
2025-10-09T21:44:08.504353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e0a9f18-52a8de9a-d2c1fe6c-d9b45316_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:08.504361Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e0a9f18-52a8de9a-d2c1fe6c-d9b45316_0] Write session will restart in 0.007673s
2025-10-09T21:44:08.504383Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e0a9f18-52a8de9a-d2c1fe6c-d9b45316_0] Write session: Do CDS request
2025-10-09T21:44:08.504394Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e0a9f18-52a8de9a-d2c1fe6c-d9b45316_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:08.513511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e0a9f18-52a8de9a-d2c1fe6c-d9b45316_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:08.513860Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:08.513874Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 252
2025-10-09T21:44:08.514083Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 252 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:08.514126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 252 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:08.514131Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=252 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:08.514135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:08.516306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 252 sessionId:  describe result for acl check
2025-10-09T21:44:08.516345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:08.516347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:08.516348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:08.516353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341075856965595:4188] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:08.516882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341075856965595:4188] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:08.547914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341075856965595:4188] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:08.554041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:08.556918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:08.557186Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:08.557193Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:08.557209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:08.557274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:08.557321Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341075495155422:3897]
2025-10-09T21:44:08.557850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:08.561708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:08.561796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:08.562186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:08.562254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:08.562255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:08.562257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:08.562258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:08.562260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:08.562261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:08.562263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.562265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.562266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.562273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:08.562284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:08.562287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:08.562292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:08.562296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:08.562300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:08.562304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.562308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.562312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.562323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:08.562332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:08.562338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:08.562344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:08.562348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:08.562353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:08.562357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:08.562363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:08.562367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:08.562372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:08.562377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:08.562382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:08.562387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.562392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.562397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.562402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.562406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.562411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.562463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:08.569072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341075856965595:4188] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:08.569089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341075856965595:4188] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:08.569092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341075856965595:4188] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:08.569102Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 252 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:08.575682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:08.576763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:08.576783Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:08.576785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:08.576790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:08.576797Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 251 [2:7559341075495155422:3897]
2025-10-09T21:44:08.576810Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:08.576829Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:08.576832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.576834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.576836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.576850Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:08.576855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:08.576862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:08.576920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.576922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.576925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.576928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:08.577195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:08.577199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.577213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.577225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.577245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:08.577267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341075495155416:3897]
2025-10-09T21:44:08.577270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341075495155416:3897]
2025-10-09T21:44:08.577273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341075495155416:3897]
2025-10-09T21:44:08.577283Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:08.577297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:08.577312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:08.577315Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:08.577357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341075495155416:3897]
2025-10-09T21:44:08.577359Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:08.577476Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:08.577480Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:08.577483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:08.577490Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:08.577765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341075856965659:4188], now have 1 active actors on pipe
2025-10-09T21:44:08.577769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341075856965586:2432], now have 1 active actors on pipe
2025-10-09T21:44:08.577928Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 251
2025-10-09T21:44:08.577952Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 251
2025-10-09T21:44:08.580359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:08.580378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:08.580415Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|24164c4c-d7122b21-71e57d4e-51c02c33_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:08.580425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.580429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.580431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:08.580441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.580454Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:08.580471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:08.582328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:08.582341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:08.582365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:08.584869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 252 partition: 0 MaxSeqNo: 25 sessionId: src_id|24164c4c-d7122b21-71e57d4e-51c02c33_0
2025-10-09T21:44:08.585218Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e0a9f18-52a8de9a-d2c1fe6c-d9b45316_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|24164c4c-d7122b21-71e57d4e-51c02c33_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:08.587986Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|24164c4c-d7122b21-71e57d4e-51c02c33_0] Write session: try to update token
2025-10-09T21:44:08.588004Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|24164c4c-d7122b21-71e57d4e-51c02c33_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:08.601565Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:08.601586Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:08.601589Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:08.601592Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:08.601602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341075495155416:3897]
2025-10-09T21:44:08.601607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341075495155416:3897]
2025-10-09T21:44:08.601611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341075495155416:3897]
2025-10-09T21:44:08.601663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:08.601677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.601679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:08.601820Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:08.605080Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341075856965667:3810]
2025-10-09T21:44:08.605726Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:08.605741Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:08.605765Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:08.606072Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 251, pipe: [1:7559341075856965667:3810]
2025-10-09T21:44:08.630640Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 252 sessionId: src_id|24164c4c-d7122b21-71e57d4e-51c02c33_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:08.633271Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:08.911557Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:08.912152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341075856965667:3810], now have 1 active actors on pipe
2025-10-09T21:44:08.912214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:08.912223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:08.913788Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:08.913807Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:08.913875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 252 reason: error at write: Disconnected sessionId: src_id|24164c4c-d7122b21-71e57d4e-51c02c33_0
2025-10-09T21:44:08.914113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 252 sessionId: src_id|24164c4c-d7122b21-71e57d4e-51c02c33_0 is DEAD
2025-10-09T21:44:08.914387Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24164c4c-d7122b21-71e57d4e-51c02c33_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:08.914403Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24164c4c-d7122b21-71e57d4e-51c02c33_0] Write session will restart in 0.005515s
2025-10-09T21:44:08.914446Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24164c4c-d7122b21-71e57d4e-51c02c33_0] Write session: Do CDS request
2025-10-09T21:44:08.914454Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24164c4c-d7122b21-71e57d4e-51c02c33_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:08.914308Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:08.914359Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341075856965667:3810]
2025-10-09T21:44:08.918178Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:08.921505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|24164c4c-d7122b21-71e57d4e-51c02c33_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:08.922306Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:08.922321Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 253
2025-10-09T21:44:08.922457Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 253 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:08.922493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 253 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:08.922499Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=253 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:08.922503Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:08.922993Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 253 sessionId:  describe result for acl check
2025-10-09T21:44:08.923030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:08.923033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:08.923034Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:08.923040Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341075856965715:4200] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:08.923612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341075856965715:4200] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:08.932451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341075856965715:4200] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:08.936140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:08.938440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341075856965715:4200] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:08.938461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341075856965715:4200] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:08.938464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341075856965715:4200] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:08.938476Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 253 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:08.939375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:08.939663Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:08.939668Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:08.939680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:08.939740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:08.939781Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341075495155526:3906]
2025-10-09T21:44:08.940349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:08.948943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:08.949060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:08.949502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:08.949571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:08.949572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:08.949574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:08.949575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:08.949577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:08.949578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:08.949579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.949581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.949582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.949589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:08.949600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:08.949604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:08.949608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:08.949624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:08.949629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:08.949634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.949638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.949641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.949652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:08.949667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:08.949674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:08.949680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:08.949685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:08.949690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:08.949695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:08.949699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:08.949704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:08.949709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:08.949714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:08.949719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:08.949723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.949728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.949732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.949737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.949742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.949747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.949780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:08.963651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:08.967183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:08.967199Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:08.967203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:08.967208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:08.967217Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 252 [2:7559341075495155526:3906]
2025-10-09T21:44:08.967234Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:08.967259Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:08.967262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.967265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.967272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.967293Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:08.967300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:08.967308Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:08.967377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.967379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.967381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.967384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:08.967468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:08.967471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:08.967487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:08.967498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:08.967513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:08.967534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341075495155520:3906]
2025-10-09T21:44:08.967537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341075495155520:3906]
2025-10-09T21:44:08.967539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341075495155520:3906]
2025-10-09T21:44:08.967554Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:08.967570Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:08.967584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:08.967587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:08.967633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341075495155520:3906]
2025-10-09T21:44:08.967635Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:08.968866Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 252
2025-10-09T21:44:08.968912Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 252
2025-10-09T21:44:08.967888Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:08.967895Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:08.967897Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:08.967903Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:08.973948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341075856965701:2432], now have 1 active actors on pipe
2025-10-09T21:44:08.973966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341075856965769:4200], now have 1 active actors on pipe
2025-10-09T21:44:08.973999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:08.974007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:08.974071Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5b5bc669-810d9d41-68589241-140dfe19_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:08.974086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.974093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.974097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:08.974113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.974134Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:08.974161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:08.974793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:08.974805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:08.974826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:08.975507Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 253 partition: 0 MaxSeqNo: 25 sessionId: src_id|5b5bc669-810d9d41-68589241-140dfe19_0
2025-10-09T21:44:08.976161Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24164c4c-d7122b21-71e57d4e-51c02c33_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5b5bc669-810d9d41-68589241-140dfe19_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:08.980978Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b5bc669-810d9d41-68589241-140dfe19_0] Write session: try to update token
2025-10-09T21:44:08.981001Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b5bc669-810d9d41-68589241-140dfe19_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:09.015234Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:09.015255Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:09.015257Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:09.015259Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:09.015270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341075495155520:3906]
2025-10-09T21:44:09.015274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341075495155520:3906]
2025-10-09T21:44:09.015278Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341075495155520:3906]
2025-10-09T21:44:09.015324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:09.015331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:09.015334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:09.015525Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:09.018173Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341080151933077:3810]
2025-10-09T21:44:09.019082Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:09.019096Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:09.019100Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:09.022313Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 252, pipe: [1:7559341080151933077:3810]
2025-10-09T21:44:09.051390Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 253 sessionId: src_id|5b5bc669-810d9d41-68589241-140dfe19_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:09.054281Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:09.483939Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:09.485026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341080151933077:3810], now have 1 active actors on pipe
2025-10-09T21:44:09.485086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:09.485094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:09.487406Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341080151933077:3810]
2025-10-09T21:44:09.487436Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:09.487470Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:09.487473Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:09.487528Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 253 reason: error at write: Disconnected sessionId: src_id|5b5bc669-810d9d41-68589241-140dfe19_0
2025-10-09T21:44:09.487707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 253 sessionId: src_id|5b5bc669-810d9d41-68589241-140dfe19_0 is DEAD
2025-10-09T21:44:09.488150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b5bc669-810d9d41-68589241-140dfe19_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:09.488163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b5bc669-810d9d41-68589241-140dfe19_0] Write session will restart in 0.006864s
2025-10-09T21:44:09.488190Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b5bc669-810d9d41-68589241-140dfe19_0] Write session: Do CDS request
2025-10-09T21:44:09.488201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b5bc669-810d9d41-68589241-140dfe19_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:09.497411Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:09.498923Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b5bc669-810d9d41-68589241-140dfe19_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:09.499544Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:09.499560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 254
2025-10-09T21:44:09.499743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 254 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:09.499775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 254 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:09.499780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=254 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:09.499783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:09.500361Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 254 sessionId:  describe result for acl check
2025-10-09T21:44:09.500398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:09.500401Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:09.500403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:09.500409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341080151933096:4205] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:09.500976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341080151933096:4205] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:09.512194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341080151933096:4205] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:09.517102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341080151933096:4205] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:09.517120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341080151933096:4205] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:09.517124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341080151933096:4205] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:09.517134Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 254 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:09.525531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:09.525771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:09.525971Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:09.525975Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:09.525987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:09.526065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:09.526104Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341079790122905:3909]
2025-10-09T21:44:09.526728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:09.529156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:09.529227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:09.529642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:09.529721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:09.529723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:09.529727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:09.529728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:09.529729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:09.529730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:09.529732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:09.529733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:09.529735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:09.529741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:09.529754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:09.529758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:09.529762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:09.529765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:09.529769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:09.529772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:09.529775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:09.529778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:09.529786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:09.529794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:09.529800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:09.529805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:09.529812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:09.529819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:09.529823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:09.529827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:09.529832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:09.529836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:09.529840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:09.529845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:09.529849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:09.529853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:09.529857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:09.529864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:09.529868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:09.529872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:09.529897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:09.547618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:09.549077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:09.549083Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:09.549087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:09.549092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:09.549101Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 253 [2:7559341079790122905:3909]
2025-10-09T21:44:09.549116Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:09.549137Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:09.549140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.549143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.549145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.549164Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:09.549170Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:09.549178Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:09.549249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:09.549251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:09.549253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:09.549256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:09.549344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:09.549347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:09.549361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:09.549372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:09.549387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:09.549407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341079790122900:3909]
2025-10-09T21:44:09.549412Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341079790122900:3909]
2025-10-09T21:44:09.549453Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341079790122900:3909]
2025-10-09T21:44:09.549469Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:09.549489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:09.549503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:09.549507Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:09.549549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341079790122900:3909]
2025-10-09T21:44:09.549551Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:09.550584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 253
2025-10-09T21:44:09.550615Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 253
2025-10-09T21:44:09.550052Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:09.550065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:09.550067Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:09.550074Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:09.556141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341080151933153:4205], now have 1 active actors on pipe
2025-10-09T21:44:09.556158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341080151933086:2432], now have 1 active actors on pipe
2025-10-09T21:44:09.556183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:09.556190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:09.556236Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9e108f3b-e0d0e24f-968433b6-e3f8bedd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:09.556250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.556255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.556259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:09.556274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.556295Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:09.556317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:09.558714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:09.558728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:09.558758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:09.559397Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b5bc669-810d9d41-68589241-140dfe19_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9e108f3b-e0d0e24f-968433b6-e3f8bedd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:09.558941Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 254 partition: 0 MaxSeqNo: 25 sessionId: src_id|9e108f3b-e0d0e24f-968433b6-e3f8bedd_0
2025-10-09T21:44:09.562076Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e108f3b-e0d0e24f-968433b6-e3f8bedd_0] Write session: try to update token
2025-10-09T21:44:09.562097Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e108f3b-e0d0e24f-968433b6-e3f8bedd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:09.594128Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341080151933166:3810]
2025-10-09T21:44:09.594723Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 253, pipe: [1:7559341080151933166:3810]
2025-10-09T21:44:09.598152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341080151933166:3810], now have 1 active actors on pipe
2025-10-09T21:44:09.633687Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 254 sessionId: src_id|9e108f3b-e0d0e24f-968433b6-e3f8bedd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:09.636315Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:09.637043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:09.637065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:09.637108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:09.641450Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:09.641468Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:09.641470Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:09.641472Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:09.641481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341079790122900:3909]
2025-10-09T21:44:09.641488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341079790122900:3909]
2025-10-09T21:44:09.641491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341079790122900:3909]
2025-10-09T21:44:09.641538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:09.641545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:09.641547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:09.641738Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:09.646092Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:09.646250Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:09.646265Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:09.646269Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:10.297176Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:10.312346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:10.312377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:10.312551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:10.312617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:10.312685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:10.312744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:10.312787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:10.312803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:10.312875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:10.312934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:10.313002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:10.313072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:10.313179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:10.313190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:10.313254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:10.313311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:10.313372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:10.313436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:10.313482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:10.313490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:10.313559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:10.313667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:10.313778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:10.313843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:10.313890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:10.313896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:10.314008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:10.316989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:10.317119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:10.317220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:10.317285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:10.317291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:10.317402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:10.317463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:10.317560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:10.317642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:10.317707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:10.317710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:10.317784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:10.317844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:10.317915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:10.317975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:10.318124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:10.318135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:10.318202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:10.318259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:10.318320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:10.318383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:10.318445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:10.318448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:10.318527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:10.318595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:10.318698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:10.318761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:10.318819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:10.318822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:10.318890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:10.318989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:10.319051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:10.319112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:10.319177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:10.319188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:10.321120Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341080151933166:3810]
2025-10-09T21:44:10.321166Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:10.321169Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:10.321198Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 254 reason: error at write: Disconnected sessionId: src_id|9e108f3b-e0d0e24f-968433b6-e3f8bedd_0
2025-10-09T21:44:10.321389Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 254 sessionId: src_id|9e108f3b-e0d0e24f-968433b6-e3f8bedd_0 is DEAD
2025-10-09T21:44:10.321531Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:10.321937Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e108f3b-e0d0e24f-968433b6-e3f8bedd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:10.321946Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e108f3b-e0d0e24f-968433b6-e3f8bedd_0] Write session will restart in 0.006625s
2025-10-09T21:44:10.321971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e108f3b-e0d0e24f-968433b6-e3f8bedd_0] Write session: Do CDS request
2025-10-09T21:44:10.321977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e108f3b-e0d0e24f-968433b6-e3f8bedd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:10.321431Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:10.329758Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e108f3b-e0d0e24f-968433b6-e3f8bedd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:10.330213Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:10.330230Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 255
2025-10-09T21:44:10.330408Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 255 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:10.330455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 255 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:10.330461Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=255 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:10.330465Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:10.330992Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 255 sessionId:  describe result for acl check
2025-10-09T21:44:10.331037Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:10.331040Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:10.331042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:10.331049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341084446900519:4217] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:10.331739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341084446900519:4217] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:10.337500Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341084446900519:4217] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:10.350522Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341084446900519:4217] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:10.350557Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341084446900519:4217] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:10.350561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341084446900519:4217] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:10.350572Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 255 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:10.351608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:10.355135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:10.355396Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:10.355401Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:10.355415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:10.355475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:10.355516Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341084085090325:3925]
2025-10-09T21:44:10.356106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:10.356518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:10.356574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:10.356947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:10.357000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:10.357007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:10.357008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:10.357010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:10.357011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:10.357012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:10.357013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:10.357015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:10.357016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:10.357023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:10.357035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:10.357040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:10.357045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:10.357049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:10.357054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:10.357059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:10.357063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:10.357066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:10.357080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:10.357090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:10.357101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:10.357107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:10.357111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:10.357117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:10.357122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:10.357127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:10.357133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:10.357138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:10.357144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:10.357150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:10.357156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:10.357166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:10.357171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:10.357181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:10.357186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:10.357197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:10.357229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:10.369203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:10.370391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:10.370398Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:10.370401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:10.370407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:10.370430Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 254 [2:7559341084085090325:3925]
2025-10-09T21:44:10.370447Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:10.370470Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:10.370473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.370477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.370480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.370500Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:10.370506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:10.370515Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:10.370590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:10.370592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:10.370594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:10.370597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:10.370682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:10.370685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:10.370701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:10.370713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:10.370729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:10.370752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341084085090320:3925]
2025-10-09T21:44:10.370754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341084085090320:3925]
2025-10-09T21:44:10.370757Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341084085090320:3925]
2025-10-09T21:44:10.370775Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:10.370791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:10.370807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:10.370811Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:10.370856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341084085090320:3925]
2025-10-09T21:44:10.370859Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:10.371436Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:10.371444Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:10.371447Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:10.371454Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:10.372518Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 254
2025-10-09T21:44:10.372562Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 254
2025-10-09T21:44:10.373667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341084446900581:4217], now have 1 active actors on pipe
2025-10-09T21:44:10.373684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341084446900511:2432], now have 1 active actors on pipe
2025-10-09T21:44:10.373709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:10.373717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:10.373773Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e56307c8-34f403c4-a2aeb61f-59841aba_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:10.373789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.373795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.373799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:10.373815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.373835Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:10.373858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:10.379849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:10.379874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:10.379931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:10.380210Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 255 partition: 0 MaxSeqNo: 25 sessionId: src_id|e56307c8-34f403c4-a2aeb61f-59841aba_0
2025-10-09T21:44:10.380728Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e108f3b-e0d0e24f-968433b6-e3f8bedd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e56307c8-34f403c4-a2aeb61f-59841aba_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:10.383754Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e56307c8-34f403c4-a2aeb61f-59841aba_0] Write session: try to update token
2025-10-09T21:44:10.383780Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e56307c8-34f403c4-a2aeb61f-59841aba_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:10.403628Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:10.403652Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:10.403655Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:10.403657Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:10.403669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341084085090320:3925]
2025-10-09T21:44:10.403674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341084085090320:3925]
2025-10-09T21:44:10.403678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341084085090320:3925]
2025-10-09T21:44:10.403729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:10.403735Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:10.403738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:10.403741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:10.403741Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:10.403745Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:10.403882Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:10.421406Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341084446900595:3810]
2025-10-09T21:44:10.422064Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 255 sessionId: src_id|e56307c8-34f403c4-a2aeb61f-59841aba_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:10.422568Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 254, pipe: [1:7559341084446900595:3810]
2025-10-09T21:44:10.424428Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:10.683518Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:10.683964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341084446900595:3810], now have 1 active actors on pipe
2025-10-09T21:44:10.684006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:10.684013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:10.685088Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341084446900595:3810]
2025-10-09T21:44:10.685122Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:10.685125Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:10.685182Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 255 reason: error at write: Disconnected sessionId: src_id|e56307c8-34f403c4-a2aeb61f-59841aba_0
2025-10-09T21:44:10.685368Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 255 sessionId: src_id|e56307c8-34f403c4-a2aeb61f-59841aba_0 is DEAD
2025-10-09T21:44:10.685496Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:10.685864Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e56307c8-34f403c4-a2aeb61f-59841aba_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:10.685874Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e56307c8-34f403c4-a2aeb61f-59841aba_0] Write session will restart in 0.007428s
2025-10-09T21:44:10.685899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e56307c8-34f403c4-a2aeb61f-59841aba_0] Write session: Do CDS request
2025-10-09T21:44:10.685909Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e56307c8-34f403c4-a2aeb61f-59841aba_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:10.687015Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:10.694659Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e56307c8-34f403c4-a2aeb61f-59841aba_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:10.695025Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:10.695042Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 256
2025-10-09T21:44:10.695510Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 256 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:10.695549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 256 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:10.695555Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=256 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:10.695559Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:10.699154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 256 sessionId:  describe result for acl check
2025-10-09T21:44:10.699202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:10.699204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:10.699206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:10.699212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341084446900611:4222] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:10.699857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341084446900611:4222] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:10.709409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341084446900611:4222] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:10.719751Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341084446900611:4222] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:10.719767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341084446900611:4222] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:10.719770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341084446900611:4222] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:10.719780Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 256 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:10.733239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:10.734776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:10.735023Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:10.735027Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:10.735041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:10.735110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:10.735150Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341084085090403:3927]
2025-10-09T21:44:10.735731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:10.736616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:10.736680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:10.737066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:10.737149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:10.737151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:10.737152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:10.737154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:10.737155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:10.737156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:10.737157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:10.737159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:10.737160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:10.737167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:10.737179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:10.737183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:10.737188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:10.737192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:10.737196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:10.737199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:10.737203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:10.737206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:10.737214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:10.737224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:10.737230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:10.737237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:10.737242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:10.737248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:10.737253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:10.737258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:10.737264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:10.737271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:10.737276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:10.737280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:10.737286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:10.737291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:10.737295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:10.737300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:10.737305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:10.737310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:10.737337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:10.749786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:10.750999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:10.751006Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:10.751010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:10.751015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:10.751024Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 255 [2:7559341084085090403:3927]
2025-10-09T21:44:10.751041Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:10.751066Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:10.751070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.751074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.751077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.751100Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:10.751106Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:10.751114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:10.751189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:10.751192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:10.751195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:10.751198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:10.751279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:10.751283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:10.751296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:10.751308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:10.751325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:10.751348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341084085090398:3927]
2025-10-09T21:44:10.751351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341084085090398:3927]
2025-10-09T21:44:10.751353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341084085090398:3927]
2025-10-09T21:44:10.751369Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:10.751385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:10.751401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:10.751404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:10.751449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341084085090398:3927]
2025-10-09T21:44:10.751451Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:10.752690Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 255
2025-10-09T21:44:10.752733Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 255
2025-10-09T21:44:10.751855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:10.751863Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:10.751866Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:10.751872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:10.753455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341084446900602:2432], now have 1 active actors on pipe
2025-10-09T21:44:10.753462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341084446900650:4222], now have 1 active actors on pipe
2025-10-09T21:44:10.753899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:10.753927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:10.753963Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b5aa0394-c99c9629-ddaf4b2f-b90ab9f6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:10.753984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.753989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.753992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:10.754006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.754021Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:10.754077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:10.754343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:10.754351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:10.754369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:10.754537Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 256 partition: 0 MaxSeqNo: 25 sessionId: src_id|b5aa0394-c99c9629-ddaf4b2f-b90ab9f6_0
2025-10-09T21:44:10.755384Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e56307c8-34f403c4-a2aeb61f-59841aba_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b5aa0394-c99c9629-ddaf4b2f-b90ab9f6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:10.758016Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5aa0394-c99c9629-ddaf4b2f-b90ab9f6_0] Write session: try to update token
2025-10-09T21:44:10.758058Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5aa0394-c99c9629-ddaf4b2f-b90ab9f6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:10.782394Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:10.782412Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:10.782415Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:10.782430Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:10.782440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341084085090398:3927]
2025-10-09T21:44:10.782445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341084085090398:3927]
2025-10-09T21:44:10.782448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341084085090398:3927]
2025-10-09T21:44:10.782501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:10.782512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:10.782516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:10.785442Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341084446900687:3810]
2025-10-09T21:44:10.786464Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 255, pipe: [1:7559341084446900687:3810]
2025-10-09T21:44:10.782691Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:10.786142Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:10.786150Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:10.786154Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:10.798285Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 256 sessionId: src_id|b5aa0394-c99c9629-ddaf4b2f-b90ab9f6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:10.800941Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:11.228434Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:11.229313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341084446900687:3810], now have 1 active actors on pipe
2025-10-09T21:44:11.229356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:11.229363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:11.230226Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:11.231070Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:11.231124Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:11.231139Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341084446900687:3810]
2025-10-09T21:44:11.231204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 256 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b5aa0394-c99c9629-ddaf4b2f-b90ab9f6_0
2025-10-09T21:44:11.231727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5aa0394-c99c9629-ddaf4b2f-b90ab9f6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:11.231741Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5aa0394-c99c9629-ddaf4b2f-b90ab9f6_0] Write session will restart in 0.007392s
2025-10-09T21:44:11.231767Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5aa0394-c99c9629-ddaf4b2f-b90ab9f6_0] Write session: Do CDS request
2025-10-09T21:44:11.231774Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5aa0394-c99c9629-ddaf4b2f-b90ab9f6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:11.231412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 256 sessionId: src_id|b5aa0394-c99c9629-ddaf4b2f-b90ab9f6_0 is DEAD
2025-10-09T21:44:11.244429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5aa0394-c99c9629-ddaf4b2f-b90ab9f6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:11.244843Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:11.244861Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 257
2025-10-09T21:44:11.245349Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 257 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:11.245392Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 257 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:11.245399Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=257 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:11.245404Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:11.246021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 257 sessionId:  describe result for acl check
2025-10-09T21:44:11.246078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:11.246080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:11.246082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:11.246089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341088741868041:4234] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:11.246721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341088741868041:4234] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:11.257061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341088741868041:4234] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:11.272007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:11.275861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:11.276163Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:11.276174Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:11.276189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:11.276259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:11.276304Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341088380057821:3943]
2025-10-09T21:44:11.276913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:11.278738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341088741868041:4234] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:11.278759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341088741868041:4234] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:11.278764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341088741868041:4234] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:11.278775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 257 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:11.279880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:11.279974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:11.280405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:11.280479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:11.280481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:11.280482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:11.280483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:11.280484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:11.280486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:11.280487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.280488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.280489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.280496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:11.280522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:11.280527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:11.280530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:11.280534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:11.280537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:11.280540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.280544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.280547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.280556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:11.280565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:11.280571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:11.280577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:11.280581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:11.280585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:11.280590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:11.280595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:11.280600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:11.280604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:11.280608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:11.280613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:11.280618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.280624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.280628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.280635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.280640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.280645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.280673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:11.291831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:11.293107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:11.293114Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:11.293118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:11.293123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:11.293132Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 256 [2:7559341088380057821:3943]
2025-10-09T21:44:11.293152Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:11.293173Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:11.293178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.293182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.293184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.293205Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:11.293212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:11.293220Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:11.293300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.293302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.293305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.293307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:11.293399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:11.293402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.293415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.293426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.293441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:11.293466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341088380057816:3943]
2025-10-09T21:44:11.293469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341088380057816:3943]
2025-10-09T21:44:11.293470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341088380057816:3943]
2025-10-09T21:44:11.293488Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:11.293504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:11.293519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:11.293526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:11.293571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341088380057816:3943]
2025-10-09T21:44:11.293575Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:11.295813Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 256
2025-10-09T21:44:11.295856Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 256
2025-10-09T21:44:11.294297Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:11.294310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:11.294312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:11.294320Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:11.297855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341088741868027:2432], now have 1 active actors on pipe
2025-10-09T21:44:11.297872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341088741868099:4234], now have 1 active actors on pipe
2025-10-09T21:44:11.297912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:11.297919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:11.297958Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5bd3fa20-ee6f6dab-ade60218-cd03d953_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:11.297974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.297982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.297986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:11.298000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.298020Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:11.298060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:11.299262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:11.299276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:11.299303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:11.299504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 257 partition: 0 MaxSeqNo: 25 sessionId: src_id|5bd3fa20-ee6f6dab-ade60218-cd03d953_0
2025-10-09T21:44:11.304896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5aa0394-c99c9629-ddaf4b2f-b90ab9f6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5bd3fa20-ee6f6dab-ade60218-cd03d953_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:11.307751Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5bd3fa20-ee6f6dab-ade60218-cd03d953_0] Write session: try to update token
2025-10-09T21:44:11.307774Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5bd3fa20-ee6f6dab-ade60218-cd03d953_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:11.332210Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341088741868109:3810]
2025-10-09T21:44:11.333680Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 256, pipe: [1:7559341088741868109:3810]
2025-10-09T21:44:11.337031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341088741868109:3810], now have 1 active actors on pipe
2025-10-09T21:44:11.337058Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:11.337063Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:11.337073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341088380057816:3943]
2025-10-09T21:44:11.337123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:11.337325Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:11.337472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:11.337644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:11.337815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:11.338503Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:11.343078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:11.343167Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:11.355244Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:11.355261Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:11.355264Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:11.355266Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:11.355283Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341088380057816:3943]
2025-10-09T21:44:11.355286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341088380057816:3943]
2025-10-09T21:44:11.355291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341088380057816:3943]
2025-10-09T21:44:11.355338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:11.355347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.355349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:11.355511Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:11.359446Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:11.359464Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:11.359468Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:11.357445Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 257 sessionId: src_id|5bd3fa20-ee6f6dab-ade60218-cd03d953_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:11.360103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:11.545344Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:11.546650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:11.546668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:11.546897Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:11.547260Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:11.547339Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 257 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5bd3fa20-ee6f6dab-ade60218-cd03d953_0
2025-10-09T21:44:11.547471Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:11.547513Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 257 sessionId: src_id|5bd3fa20-ee6f6dab-ade60218-cd03d953_0 is DEAD
2025-10-09T21:44:11.547525Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341088741868109:3810]
2025-10-09T21:44:11.548001Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5bd3fa20-ee6f6dab-ade60218-cd03d953_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:11.548017Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5bd3fa20-ee6f6dab-ade60218-cd03d953_0] Write session will restart in 0.005084s
2025-10-09T21:44:11.548043Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5bd3fa20-ee6f6dab-ade60218-cd03d953_0] Write session: Do CDS request
2025-10-09T21:44:11.548050Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5bd3fa20-ee6f6dab-ade60218-cd03d953_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:11.554556Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5bd3fa20-ee6f6dab-ade60218-cd03d953_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:11.555305Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:11.555322Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 258
2025-10-09T21:44:11.555518Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 258 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:11.555557Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 258 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:11.555563Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=258 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:11.555566Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:11.556321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 258 sessionId:  describe result for acl check
2025-10-09T21:44:11.556357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:11.556361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:11.556362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:11.556369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341088741868126:4239] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:11.556907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341088741868126:4239] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:11.579280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341088741868126:4239] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:11.587470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341088741868126:4239] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:11.587490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341088741868126:4239] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:11.587494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341088741868126:4239] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:11.587503Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 258 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:11.598751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:11.600268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:11.600497Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:11.600501Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:11.600512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:11.600572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:11.600619Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341088380057903:3945]
2025-10-09T21:44:11.601394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:11.603194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:11.603256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:11.603618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:11.603690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:11.603691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:11.603692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:11.603694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:11.603695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:11.603696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:11.603698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.603699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.603700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.603707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:11.603717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:11.603722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:11.603726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:11.603729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:11.603733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:11.603736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.603739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.603742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.603751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:11.603760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:11.603772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:11.603777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:11.603782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:11.603787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:11.603792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:11.603797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:11.603801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:11.603806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:11.603810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:11.603815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:11.603820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.603825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.603832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.603837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.603841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.603845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.603875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:11.615661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:11.617270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:11.617277Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:11.617281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:11.617284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:11.617292Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 257 [2:7559341088380057903:3945]
2025-10-09T21:44:11.617307Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:11.617328Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:11.617332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.617336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.617338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.617359Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:11.617364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:11.617376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:11.617446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.617448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.617450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.617452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:11.617534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:11.617537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.617551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.617561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.617576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:11.617595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341088380057898:3945]
2025-10-09T21:44:11.617598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341088380057898:3945]
2025-10-09T21:44:11.617600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341088380057898:3945]
2025-10-09T21:44:11.617615Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:11.617630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:11.617644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:11.617647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:11.617688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341088380057898:3945]
2025-10-09T21:44:11.617690Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:11.619720Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 257
2025-10-09T21:44:11.619765Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 257
2025-10-09T21:44:11.618466Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:11.618479Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:11.618481Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:11.618489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:11.622428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341088741868116:2432], now have 1 active actors on pipe
2025-10-09T21:44:11.622446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341088741868173:4239], now have 1 active actors on pipe
2025-10-09T21:44:11.622474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:11.622480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:11.622517Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2d832b25-35b90d5a-138a856e-d359b8c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:11.622530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.622535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.622538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:11.622554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.622573Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:11.622594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:11.623507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:11.623518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:11.623549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:11.623760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 258 partition: 0 MaxSeqNo: 25 sessionId: src_id|2d832b25-35b90d5a-138a856e-d359b8c_0
2025-10-09T21:44:11.624165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5bd3fa20-ee6f6dab-ade60218-cd03d953_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2d832b25-35b90d5a-138a856e-d359b8c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:11.627436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d832b25-35b90d5a-138a856e-d359b8c_0] Write session: try to update token
2025-10-09T21:44:11.627457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d832b25-35b90d5a-138a856e-d359b8c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:11.647403Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341088741868204:3810]
2025-10-09T21:44:11.650014Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 257, pipe: [1:7559341088741868204:3810]
2025-10-09T21:44:11.646158Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:11.646177Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:11.646180Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:11.646181Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:11.646195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341088380057898:3945]
2025-10-09T21:44:11.646200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341088380057898:3945]
2025-10-09T21:44:11.646203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341088380057898:3945]
2025-10-09T21:44:11.646253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:11.646261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.646264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:11.646432Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:11.649558Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:11.649565Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:11.649582Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:11.657589Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 258 sessionId: src_id|2d832b25-35b90d5a-138a856e-d359b8c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:11.660199Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:11.901716Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:11.902244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341088741868204:3810], now have 1 active actors on pipe
2025-10-09T21:44:11.902279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:11.902286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:11.903370Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:11.903440Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341088741868204:3810]
2025-10-09T21:44:11.903463Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:11.903465Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:11.903535Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 258 reason: error at write: Disconnected sessionId: src_id|2d832b25-35b90d5a-138a856e-d359b8c_0
2025-10-09T21:44:11.903730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 258 sessionId: src_id|2d832b25-35b90d5a-138a856e-d359b8c_0 is DEAD
2025-10-09T21:44:11.906160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d832b25-35b90d5a-138a856e-d359b8c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:11.906174Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d832b25-35b90d5a-138a856e-d359b8c_0] Write session will restart in 0.009305s
2025-10-09T21:44:11.906200Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d832b25-35b90d5a-138a856e-d359b8c_0] Write session: Do CDS request
2025-10-09T21:44:11.906210Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d832b25-35b90d5a-138a856e-d359b8c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:11.910356Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:11.917229Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d832b25-35b90d5a-138a856e-d359b8c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:11.918470Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:11.918487Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 259
2025-10-09T21:44:11.919576Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 259 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:11.919634Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 259 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:11.919641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=259 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:11.919645Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:11.921686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 259 sessionId:  describe result for acl check
2025-10-09T21:44:11.921739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:11.921741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:11.921742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:11.921749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341088741868255:4246] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:11.922559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341088741868255:4246] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:11.937129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341088741868255:4246] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:11.938540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:11.940793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:11.941044Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:11.941050Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:11.941064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:11.941127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:11.941169Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341088380057983:3947]
2025-10-09T21:44:11.941749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:11.944729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:11.944827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:11.945252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:11.945327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:11.945329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:11.945330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:11.945332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:11.945333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:11.945334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:11.945336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.945337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.945339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.945346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:11.945369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:11.945373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:11.945377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:11.945380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:11.945383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:11.945387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.945392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.945395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.945407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:11.945417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:11.945423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:11.945428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:11.945433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:11.945438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:11.945443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:11.945448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:11.945453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:11.945457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:11.945462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:11.945466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:11.945471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.945476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.945480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.945484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.945488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.945493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.945523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:11.961540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:11.963165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:11.963173Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:11.963175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:11.963181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:11.963190Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 258 [2:7559341088380057983:3947]
2025-10-09T21:44:11.963205Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:11.963229Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:11.963233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.963236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.963238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.963257Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:11.963263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:11.963271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:11.963355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.963357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.963360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.963363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:11.963451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:11.963454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:11.963469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:11.963480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:11.963497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:11.963520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341088380057978:3947]
2025-10-09T21:44:11.963523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341088380057978:3947]
2025-10-09T21:44:11.963525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341088380057978:3947]
2025-10-09T21:44:11.963543Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:11.963563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:11.963579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:11.963582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:11.963634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341088380057978:3947]
2025-10-09T21:44:11.963637Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:11.964848Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 258
2025-10-09T21:44:11.963949Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:11.963960Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:11.963962Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:11.963970Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:11.965659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341088741868216:2432], now have 1 active actors on pipe
2025-10-09T21:44:11.969923Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341088741868255:4246] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:11.969943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341088741868255:4246] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:11.969948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341088741868255:4246] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:11.969958Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 259 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:11.971013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341088741868303:4246], now have 1 active actors on pipe
2025-10-09T21:44:11.971545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 258
2025-10-09T21:44:11.972020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:11.972036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:11.972071Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2bd9444b-7c948c43-935a5d46-67eb33d0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:11.972084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.972090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.972093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:11.972106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.972126Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:11.972159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:11.973668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:11.973682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:11.973710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:11.973869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 259 partition: 0 MaxSeqNo: 25 sessionId: src_id|2bd9444b-7c948c43-935a5d46-67eb33d0_0
2025-10-09T21:44:11.974333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d832b25-35b90d5a-138a856e-d359b8c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2bd9444b-7c948c43-935a5d46-67eb33d0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:11.977052Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2bd9444b-7c948c43-935a5d46-67eb33d0_0] Write session: try to update token
2025-10-09T21:44:11.977071Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2bd9444b-7c948c43-935a5d46-67eb33d0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:12.003841Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341093036835601:3810]
2025-10-09T21:44:12.006363Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:12.006385Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:12.006388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:12.006390Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:12.006401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341088380057978:3947]
2025-10-09T21:44:12.006406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341088380057978:3947]
2025-10-09T21:44:12.006411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341088380057978:3947]
2025-10-09T21:44:12.006475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:12.006483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:12.006487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:12.006637Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:12.007469Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:12.007475Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:12.007479Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:12.010136Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 259 sessionId: src_id|2bd9444b-7c948c43-935a5d46-67eb33d0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:12.012172Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 258, pipe: [1:7559341093036835601:3810]
2025-10-09T21:44:12.012886Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:12.271301Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:12.271551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341093036835601:3810], now have 1 active actors on pipe
2025-10-09T21:44:12.271579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:12.271587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:12.273368Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:12.274670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2bd9444b-7c948c43-935a5d46-67eb33d0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:12.274682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2bd9444b-7c948c43-935a5d46-67eb33d0_0] Write session will restart in 0.009866s
2025-10-09T21:44:12.274707Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2bd9444b-7c948c43-935a5d46-67eb33d0_0] Write session: Do CDS request
2025-10-09T21:44:12.274714Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2bd9444b-7c948c43-935a5d46-67eb33d0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:12.273468Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:12.273493Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:12.273567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 259 reason: error at write: Disconnected sessionId: src_id|2bd9444b-7c948c43-935a5d46-67eb33d0_0
2025-10-09T21:44:12.273770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 259 sessionId: src_id|2bd9444b-7c948c43-935a5d46-67eb33d0_0 is DEAD
2025-10-09T21:44:12.273916Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341093036835601:3810]
2025-10-09T21:44:12.273931Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:12.285639Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2bd9444b-7c948c43-935a5d46-67eb33d0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:12.286259Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:12.286275Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 260
2025-10-09T21:44:12.286493Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 260 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:12.286533Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 260 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:12.286538Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=260 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:12.286542Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:12.287037Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 260 sessionId:  describe result for acl check
2025-10-09T21:44:12.287076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:12.287078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:12.287079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:12.287085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341093036835665:4257] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:12.287641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341093036835665:4257] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:12.296652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341093036835665:4257] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:12.297490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:12.297930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:12.298149Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:12.298152Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:12.298168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:12.298226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:12.298263Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341092675025403:3961]
2025-10-09T21:44:12.298862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:12.299578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:12.299653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:12.300277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:12.300362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:12.300363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:12.300365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:12.300366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:12.300367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:12.300369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:12.300370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:12.300371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:12.300372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:12.300378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:12.300388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:12.300391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:12.300395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:12.300399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:12.300403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:12.300406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:12.300410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:12.300413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:12.300421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:12.300432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:12.300438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:12.300443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:12.300448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:12.300452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:12.300456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:12.300461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:12.300465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:12.300470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:12.300474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:12.300479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:12.300484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:12.300488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:12.300491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:12.300495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:12.300499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:12.300503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:12.300529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:12.313179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:12.314485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:12.314492Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:12.314494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:12.314499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:12.314506Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 259 [2:7559341092675025403:3961]
2025-10-09T21:44:12.314522Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:12.314543Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:12.314546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.314549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.314551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.314571Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:12.314578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:12.314586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:12.314643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:12.314645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:12.314647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:12.314650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:12.314726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:12.314730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:12.314744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:12.314756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:12.314774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:12.314793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341092675025398:3961]
2025-10-09T21:44:12.314796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341092675025398:3961]
2025-10-09T21:44:12.314798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341092675025398:3961]
2025-10-09T21:44:12.314812Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:12.314828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:12.314843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:12.314847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:12.314888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341092675025398:3961]
2025-10-09T21:44:12.314892Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:12.315702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:12.315713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:12.315715Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:12.315721Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:12.317100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341093036835628:2432], now have 1 active actors on pipe
2025-10-09T21:44:12.319112Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 259
2025-10-09T21:44:12.320509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341093036835665:4257] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:12.320524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341093036835665:4257] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:12.320528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341093036835665:4257] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:12.320539Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 260 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:12.321506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341093036835716:4257], now have 1 active actors on pipe
2025-10-09T21:44:12.321852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:12.321865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:12.321911Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9cc62b32-a8cf7ed7-61d067d1-56587fda_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:12.321925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.321931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.321934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:12.321949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.321982Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:12.322006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:12.322517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:12.322529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:12.322549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:12.323194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2bd9444b-7c948c43-935a5d46-67eb33d0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9cc62b32-a8cf7ed7-61d067d1-56587fda_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:12.325987Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9cc62b32-a8cf7ed7-61d067d1-56587fda_0] Write session: try to update token
2025-10-09T21:44:12.326008Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9cc62b32-a8cf7ed7-61d067d1-56587fda_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:12.321681Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 259
2025-10-09T21:44:12.322745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 260 partition: 0 MaxSeqNo: 25 sessionId: src_id|9cc62b32-a8cf7ed7-61d067d1-56587fda_0
2025-10-09T21:44:12.347924Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:12.347941Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:12.347943Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:12.347944Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:12.347952Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341092675025398:3961]
2025-10-09T21:44:12.347956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341092675025398:3961]
2025-10-09T21:44:12.347958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341092675025398:3961]
2025-10-09T21:44:12.347999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:12.348005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:12.348007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:12.348142Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:12.350235Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:12.350248Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:12.350252Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:12.369103Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 260 sessionId: src_id|9cc62b32-a8cf7ed7-61d067d1-56587fda_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:12.371820Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:12.376102Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341093036835720:3810]
2025-10-09T21:44:12.377359Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 259, pipe: [1:7559341093036835720:3810]
2025-10-09T21:44:12.559062Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:12.559492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:12.559504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:12.559527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341093036835720:3810], now have 1 active actors on pipe
2025-10-09T21:44:12.560413Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:12.564037Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:12.564098Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:12.564178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 260 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9cc62b32-a8cf7ed7-61d067d1-56587fda_0
2025-10-09T21:44:12.564373Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 260 sessionId: src_id|9cc62b32-a8cf7ed7-61d067d1-56587fda_0 is DEAD
2025-10-09T21:44:12.564614Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9cc62b32-a8cf7ed7-61d067d1-56587fda_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:12.564628Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9cc62b32-a8cf7ed7-61d067d1-56587fda_0] Write session will restart in 0.009968s
2025-10-09T21:44:12.564657Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9cc62b32-a8cf7ed7-61d067d1-56587fda_0] Write session: Do CDS request
2025-10-09T21:44:12.564665Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9cc62b32-a8cf7ed7-61d067d1-56587fda_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:12.564792Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341093036835720:3810]
2025-10-09T21:44:12.575680Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9cc62b32-a8cf7ed7-61d067d1-56587fda_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:12.578949Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:12.578968Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 261
2025-10-09T21:44:12.579370Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 261 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:12.579412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 261 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:12.579418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=261 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:12.579422Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:12.606529Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 261 sessionId:  describe result for acl check
2025-10-09T21:44:12.606580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:12.606582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:12.606584Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:12.606591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341093036835733:4262] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:12.607171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341093036835733:4262] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:12.626537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341093036835733:4262] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:12.628336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:12.631928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:12.632162Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:12.632167Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:12.632179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:12.632235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:12.632277Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341092675025486:3965]
2025-10-09T21:44:12.632897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:12.633843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:12.633899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:12.634289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:12.634357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:12.634359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:12.634360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:12.634361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:12.634363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:12.634364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:12.634365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:12.634369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:12.634370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:12.634376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:12.634387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:12.634390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:12.634394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:12.634398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:12.634403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:12.634406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:12.634410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:12.634414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:12.634437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:12.634447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:12.634453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:12.634458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:12.634462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:12.634467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:12.634470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:12.634476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:12.634480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:12.634484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:12.634489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:12.634494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:12.634498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:12.634503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:12.634507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:12.634513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:12.634517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:12.634522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:12.634551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:12.639292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341093036835733:4262] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:12.639311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341093036835733:4262] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:12.639315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341093036835733:4262] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:12.639326Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 261 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:12.645866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:12.647259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:12.647269Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:12.647272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:12.647277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:12.647285Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 260 [2:7559341092675025486:3965]
2025-10-09T21:44:12.647303Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:12.647325Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:12.647328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.647331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.647334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.647355Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:12.647362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:12.647370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:12.647436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:12.647439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:12.647441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:12.647444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:12.647531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:12.647534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:12.647549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:12.647563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:12.647582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:12.647605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341092675025480:3965]
2025-10-09T21:44:12.647608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341092675025480:3965]
2025-10-09T21:44:12.647610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341092675025480:3965]
2025-10-09T21:44:12.647631Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:12.647646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:12.647663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:12.647666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:12.647710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341092675025480:3965]
2025-10-09T21:44:12.647713Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:12.648000Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:12.648009Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:12.648011Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:12.648017Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:12.648810Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 260
2025-10-09T21:44:12.648842Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 260
2025-10-09T21:44:12.652966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341093036835807:4262], now have 1 active actors on pipe
2025-10-09T21:44:12.652981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341093036835725:2432], now have 1 active actors on pipe
2025-10-09T21:44:12.653005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:12.653014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:12.653058Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fb77f191-420f771-92a85e11-b8e198db_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:12.653071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.653081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.653084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:12.653098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.653117Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:12.653144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:12.653603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:12.653614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:12.653659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:12.654056Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 261 partition: 0 MaxSeqNo: 25 sessionId: src_id|fb77f191-420f771-92a85e11-b8e198db_0
2025-10-09T21:44:12.654530Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9cc62b32-a8cf7ed7-61d067d1-56587fda_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fb77f191-420f771-92a85e11-b8e198db_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:12.657261Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fb77f191-420f771-92a85e11-b8e198db_0] Write session: try to update token
2025-10-09T21:44:12.657310Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fb77f191-420f771-92a85e11-b8e198db_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:12.664479Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341093036835813:3810]
2025-10-09T21:44:12.665055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341093036835813:3810], now have 1 active actors on pipe
2025-10-09T21:44:12.665171Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 260, pipe: [1:7559341093036835813:3810]
2025-10-09T21:44:12.689843Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 261 sessionId: src_id|fb77f191-420f771-92a85e11-b8e198db_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:12.692375Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:12.690550Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:12.690571Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:12.690574Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:12.690576Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:12.690587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341092675025480:3965]
2025-10-09T21:44:12.690592Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341092675025480:3965]
2025-10-09T21:44:12.690596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341092675025480:3965]
2025-10-09T21:44:12.690650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:12.690660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:12.690662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:12.690845Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:12.692974Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:12.692987Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:12.692990Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:13.029079Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:13.030684Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:13.030699Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:13.030763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 261 reason: error at write: Disconnected sessionId: src_id|fb77f191-420f771-92a85e11-b8e198db_0
2025-10-09T21:44:13.030950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 261 sessionId: src_id|fb77f191-420f771-92a85e11-b8e198db_0 is DEAD
2025-10-09T21:44:13.031078Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:13.031114Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341093036835813:3810]
2025-10-09T21:44:13.029403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:13.029423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:13.034275Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb77f191-420f771-92a85e11-b8e198db_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:13.034300Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb77f191-420f771-92a85e11-b8e198db_0] Write session will restart in 0.005415s
2025-10-09T21:44:13.034327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb77f191-420f771-92a85e11-b8e198db_0] Write session: Do CDS request
2025-10-09T21:44:13.034334Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb77f191-420f771-92a85e11-b8e198db_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:13.040605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fb77f191-420f771-92a85e11-b8e198db_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:13.040705Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:13.040730Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 262
2025-10-09T21:44:13.040533Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:13.041298Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 262 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:13.041337Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 262 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:13.041345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=262 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:13.041349Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:13.041987Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 262 sessionId:  describe result for acl check
2025-10-09T21:44:13.042050Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:13.042052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:13.042054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:13.042060Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341097331803139:4267] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:13.042671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341097331803139:4267] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:13.047878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341097331803139:4267] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:13.051922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:13.054478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:13.054789Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:13.054812Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:13.054830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:13.054916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:13.054969Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341096969992869:3968]
2025-10-09T21:44:13.055689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:13.056951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341097331803139:4267] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:13.056971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341097331803139:4267] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:13.056975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341097331803139:4267] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:13.056986Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 262 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:13.058107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:13.058188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:13.058606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:13.058678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:13.058679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:13.058681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:13.058682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:13.058684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:13.058685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:13.058686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.058688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.058689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.058696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:13.058707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:13.058712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:13.058716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:13.058720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:13.058725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:13.058730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.058747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.058754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.058765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:13.058775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:13.058781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:13.058787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:13.058793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:13.058799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:13.058803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:13.058808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:13.058814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:13.058819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:13.058824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:13.058829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:13.058834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.058840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.058845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.058851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.058857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.058863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.058896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:13.071158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:13.072747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:13.072757Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:13.072760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:13.072765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:13.072774Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 261 [2:7559341096969992869:3968]
2025-10-09T21:44:13.072794Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:13.072819Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:13.072823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.072826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.072829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.072851Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:13.072858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:13.072866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:13.072944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.072946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.072950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.072953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:13.073041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:13.073044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.073057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.073067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.073083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:13.073106Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341096969992864:3968]
2025-10-09T21:44:13.073109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341096969992864:3968]
2025-10-09T21:44:13.073110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341096969992864:3968]
2025-10-09T21:44:13.073126Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:13.073141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:13.073154Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:13.073157Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:13.073203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341096969992864:3968]
2025-10-09T21:44:13.073205Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:13.073727Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:13.073736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:13.073739Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:13.073744Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:13.074627Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 261
2025-10-09T21:44:13.074660Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 261
2025-10-09T21:44:13.078962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341097331803202:4267], now have 1 active actors on pipe
2025-10-09T21:44:13.078982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341097331803126:2432], now have 1 active actors on pipe
2025-10-09T21:44:13.079005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:13.079012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:13.079060Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|36903945-108000df-9b4f7fb-a425788f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:13.079073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.079078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.079081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:13.079097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.079117Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:13.079139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:13.079578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:13.079586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:13.079605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:13.079754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 262 partition: 0 MaxSeqNo: 25 sessionId: src_id|36903945-108000df-9b4f7fb-a425788f_0
2025-10-09T21:44:13.080335Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb77f191-420f771-92a85e11-b8e198db_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|36903945-108000df-9b4f7fb-a425788f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:13.083228Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36903945-108000df-9b4f7fb-a425788f_0] Write session: try to update token
2025-10-09T21:44:13.083252Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36903945-108000df-9b4f7fb-a425788f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:13.129952Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 262 sessionId: src_id|36903945-108000df-9b4f7fb-a425788f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:13.132884Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:13.132974Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341097331803212:3810]
2025-10-09T21:44:13.133481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:13.133501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:13.133547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:13.133745Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:13.140149Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:13.140174Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:13.140177Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:13.140179Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:13.140190Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341096969992864:3968]
2025-10-09T21:44:13.140196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341096969992864:3968]
2025-10-09T21:44:13.140199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341096969992864:3968]
2025-10-09T21:44:13.140249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:13.140259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.140262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:13.140407Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:13.142126Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:13.142134Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:13.142138Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:13.150366Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 261, pipe: [1:7559341097331803212:3810]
2025-10-09T21:44:13.416793Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:13.417127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341097331803212:3810], now have 1 active actors on pipe
2025-10-09T21:44:13.427723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:13.427757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:13.427945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:13.428003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:13.428070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:13.428130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:13.428617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:13.428623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:13.428700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:13.428759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:13.428819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:13.428882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:13.429087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:13.429090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:13.429160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:13.429216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:13.429276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:13.429339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:13.429380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:13.429383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:13.429458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:13.429525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:13.429592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:13.429654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:13.429697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:13.429701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:13.429771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:13.429830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:13.429902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:13.429965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:13.430007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:13.430013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:13.430282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:13.430357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:13.430432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:13.430487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:13.430526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:13.430529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:13.430621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:13.430674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:13.430736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:13.430796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:13.430835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:13.430837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:13.430905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:13.430961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:13.431021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:13.431080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:13.431125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:13.431130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:13.431195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:13.431248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:13.431308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:13.431370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:13.431409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:13.431412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:13.431475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:13.431531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:13.431591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:13.431648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:13.431779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:13.431783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:13.433596Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36903945-108000df-9b4f7fb-a425788f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:13.433607Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36903945-108000df-9b4f7fb-a425788f_0] Write session will restart in 0.009478s
2025-10-09T21:44:13.433635Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36903945-108000df-9b4f7fb-a425788f_0] Write session: Do CDS request
2025-10-09T21:44:13.433642Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36903945-108000df-9b4f7fb-a425788f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:13.432717Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:13.432976Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:13.433007Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:13.433010Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:13.433028Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341097331803212:3810]
2025-10-09T21:44:13.433040Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 262 reason: error at write: Disconnected sessionId: src_id|36903945-108000df-9b4f7fb-a425788f_0
2025-10-09T21:44:13.433220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 262 sessionId: src_id|36903945-108000df-9b4f7fb-a425788f_0 is DEAD
2025-10-09T21:44:13.447601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36903945-108000df-9b4f7fb-a425788f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:13.448011Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:13.448030Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 263
2025-10-09T21:44:13.450342Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 263 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:13.450408Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 263 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:13.450414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=263 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:13.450427Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:13.451148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 263 sessionId:  describe result for acl check
2025-10-09T21:44:13.451183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:13.451185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:13.451186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:13.451192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341097331803285:4278] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:13.451756Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341097331803285:4278] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:13.464557Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341097331803285:4278] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:13.467774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341097331803285:4278] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:13.467792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341097331803285:4278] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:13.467796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341097331803285:4278] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:13.467835Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 263 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:13.476921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:13.477616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:13.478010Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:13.478015Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:13.478047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:13.478122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:13.478164Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341096969992995:3983]
2025-10-09T21:44:13.478711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:13.480049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:13.480100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:13.480407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:13.480447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:13.480448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:13.480450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:13.480451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:13.480452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:13.480454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:13.480455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.480456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.480458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.480463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:13.480473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:13.480476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:13.480480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:13.480483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:13.480487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:13.480490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.480492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.480495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.480502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:13.480510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:13.480515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:13.480520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:13.480524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:13.480528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:13.480531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:13.480535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:13.480539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:13.480543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:13.480546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:13.480550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:13.480555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.480559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.480563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.480567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.480572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.480576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.480602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:13.502733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:13.503826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:13.503832Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:13.503835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:13.503839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:13.503847Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 262 [2:7559341096969992995:3983]
2025-10-09T21:44:13.503861Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:13.503882Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:13.503886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.503889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.503892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.503910Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:13.503916Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:13.503925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:13.503994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.503996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.503997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.504000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:13.504099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:13.504102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.504114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.504123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.504136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:13.504158Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341096969992990:3983]
2025-10-09T21:44:13.504161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341096969992990:3983]
2025-10-09T21:44:13.504162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341096969992990:3983]
2025-10-09T21:44:13.504171Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:13.504186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:13.504200Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:13.504202Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:13.504247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341096969992990:3983]
2025-10-09T21:44:13.504250Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:13.504262Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:13.504268Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:13.504270Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:13.504277Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:13.505182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341097331803242:2432], now have 1 active actors on pipe
2025-10-09T21:44:13.505188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341097331803323:4278], now have 1 active actors on pipe
2025-10-09T21:44:13.506261Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 262
2025-10-09T21:44:13.506637Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 262
2025-10-09T21:44:13.507641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:13.507669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:13.507709Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|87de5085-6bc43492-670f4d24-7da911a8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:13.507722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.507728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.507731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:13.507746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.507762Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:13.507786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:13.508128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:13.508137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:13.508153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:13.508328Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 263 partition: 0 MaxSeqNo: 25 sessionId: src_id|87de5085-6bc43492-670f4d24-7da911a8_0
2025-10-09T21:44:13.510912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36903945-108000df-9b4f7fb-a425788f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|87de5085-6bc43492-670f4d24-7da911a8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:13.513525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87de5085-6bc43492-670f4d24-7da911a8_0] Write session: try to update token
2025-10-09T21:44:13.513545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87de5085-6bc43492-670f4d24-7da911a8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:13.534888Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341097331803334:3810]
2025-10-09T21:44:13.537805Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:13.537828Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:13.537831Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:13.537833Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:13.537845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341096969992990:3983]
2025-10-09T21:44:13.537851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341096969992990:3983]
2025-10-09T21:44:13.537857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341096969992990:3983]
2025-10-09T21:44:13.537907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:13.537915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.537918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:13.538117Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:13.541776Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:13.541790Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:13.541793Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:13.542533Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 262, pipe: [1:7559341097331803334:3810]
2025-10-09T21:44:13.565962Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 263 sessionId: src_id|87de5085-6bc43492-670f4d24-7da911a8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:13.568588Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:13.790881Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:13.791526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341097331803334:3810], now have 1 active actors on pipe
2025-10-09T21:44:13.791586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:13.791594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:13.792608Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:13.794395Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87de5085-6bc43492-670f4d24-7da911a8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:13.794409Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87de5085-6bc43492-670f4d24-7da911a8_0] Write session will restart in 0.008748s
2025-10-09T21:44:13.794448Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87de5085-6bc43492-670f4d24-7da911a8_0] Write session: Do CDS request
2025-10-09T21:44:13.794456Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87de5085-6bc43492-670f4d24-7da911a8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:13.793465Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341097331803334:3810]
2025-10-09T21:44:13.793496Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:13.793541Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:13.793616Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 263 reason: status is not ok: tablet will be restarted right now sessionId: src_id|87de5085-6bc43492-670f4d24-7da911a8_0
2025-10-09T21:44:13.796567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 263 sessionId: src_id|87de5085-6bc43492-670f4d24-7da911a8_0 is DEAD
2025-10-09T21:44:13.804615Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87de5085-6bc43492-670f4d24-7da911a8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:13.805996Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:13.806022Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 264
2025-10-09T21:44:13.806309Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 264 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:13.806354Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 264 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:13.806360Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=264 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:13.806364Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:13.806925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 264 sessionId:  describe result for acl check
2025-10-09T21:44:13.806964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:13.806974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:13.806976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:13.806982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341097331803384:4283] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:13.807572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341097331803384:4283] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:13.812387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341097331803384:4283] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:13.820263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:13.820458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:13.820645Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:13.820649Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:13.820662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:13.820709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:13.820751Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341096969993080:3985]
2025-10-09T21:44:13.821283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:13.822883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:13.822950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:13.823310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:13.823382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:13.823384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:13.823385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:13.823387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:13.823388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:13.823389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:13.823391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.823392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.823393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.823399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:13.823410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:13.823415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:13.823419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:13.823422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:13.823426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:13.823430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.823434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.823437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.823445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:13.823454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:13.823459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:13.823466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:13.823470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:13.823475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:13.823481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:13.823485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:13.823490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:13.823494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:13.823498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:13.823502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:13.823506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.823510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.823513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.823532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.823539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.823544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.823572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:13.830348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341097331803384:4283] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:13.830369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341097331803384:4283] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:13.830374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341097331803384:4283] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:13.830386Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 264 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:13.834978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:13.836181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:13.836188Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:13.836191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:13.836196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:13.836205Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 263 [2:7559341096969993080:3985]
2025-10-09T21:44:13.836225Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:13.836249Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:13.836252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.836256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.836259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.836282Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:13.836289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:13.836297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:13.836377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.836379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.836381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.836384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:13.836784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:13.836788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.836803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:13.836816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:13.836833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:13.836858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341096969993075:3985]
2025-10-09T21:44:13.836861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341096969993075:3985]
2025-10-09T21:44:13.836863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341096969993075:3985]
2025-10-09T21:44:13.836883Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:13.836901Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:13.836917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:13.836921Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:13.836968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341096969993075:3985]
2025-10-09T21:44:13.836971Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:13.837253Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:13.837264Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:13.837266Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:13.837272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:13.838294Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 263
2025-10-09T21:44:13.838330Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 263
2025-10-09T21:44:13.842067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341097331803426:4283], now have 1 active actors on pipe
2025-10-09T21:44:13.842084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341097331803340:2432], now have 1 active actors on pipe
2025-10-09T21:44:13.842104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:13.842111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:13.842166Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6ac38406-dff68916-fb1e2385-d3df3ef0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:13.842183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.842190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.842193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:13.842208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.842235Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:13.842260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:13.842702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:13.842712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:13.842733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:13.842887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 264 partition: 0 MaxSeqNo: 25 sessionId: src_id|6ac38406-dff68916-fb1e2385-d3df3ef0_0
2025-10-09T21:44:13.843418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87de5085-6bc43492-670f4d24-7da911a8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6ac38406-dff68916-fb1e2385-d3df3ef0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:13.846084Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ac38406-dff68916-fb1e2385-d3df3ef0_0] Write session: try to update token
2025-10-09T21:44:13.846104Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ac38406-dff68916-fb1e2385-d3df3ef0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:13.893640Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341097331803437:3810]
2025-10-09T21:44:13.895374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341097331803437:3810], now have 1 active actors on pipe
2025-10-09T21:44:13.895515Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 263, pipe: [1:7559341097331803437:3810]
2025-10-09T21:44:13.900186Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:13.900206Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:13.900208Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:13.900210Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:13.900220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341096969993075:3985]
2025-10-09T21:44:13.900229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341096969993075:3985]
2025-10-09T21:44:13.900233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341096969993075:3985]
2025-10-09T21:44:13.900274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:13.900281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:13.900284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:13.900422Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:13.903345Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:13.903359Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:13.903364Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:13.910770Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 264 sessionId: src_id|6ac38406-dff68916-fb1e2385-d3df3ef0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:13.913489Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:14.101296Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:14.101737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:14.101751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:14.105162Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:14.105183Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:14.105252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 264 reason: error at write: Disconnected sessionId: src_id|6ac38406-dff68916-fb1e2385-d3df3ef0_0
2025-10-09T21:44:14.105471Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 264 sessionId: src_id|6ac38406-dff68916-fb1e2385-d3df3ef0_0 is DEAD
2025-10-09T21:44:14.105636Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341097331803437:3810]
2025-10-09T21:44:14.105655Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:14.106087Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ac38406-dff68916-fb1e2385-d3df3ef0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:14.106101Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ac38406-dff68916-fb1e2385-d3df3ef0_0] Write session will restart in 0.005364s
2025-10-09T21:44:14.106133Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ac38406-dff68916-fb1e2385-d3df3ef0_0] Write session: Do CDS request
2025-10-09T21:44:14.106140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ac38406-dff68916-fb1e2385-d3df3ef0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:14.111620Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:14.112590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ac38406-dff68916-fb1e2385-d3df3ef0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:14.113088Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:14.113103Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 265
2025-10-09T21:44:14.113287Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 265 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:14.113330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 265 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:14.113336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=265 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:14.113340Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:14.114454Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 265 sessionId:  describe result for acl check
2025-10-09T21:44:14.114497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:14.114499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:14.114500Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:14.114506Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341101626770762:4288] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:14.115084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341101626770762:4288] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:14.119656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341101626770762:4288] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:14.128419Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341101626770762:4288] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:14.128439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341101626770762:4288] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:14.128444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341101626770762:4288] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:14.128453Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 265 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:14.132455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:14.142831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:14.143111Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:14.143115Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:14.143128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:14.143203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:14.143241Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341101264960466:3988]
2025-10-09T21:44:14.143873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:14.147631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:14.147733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:14.148114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:14.148177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:14.148178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:14.148180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:14.148181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:14.148183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:14.148184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:14.148187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.148188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.148190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.148197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:14.148207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:14.148209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:14.148211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:14.148214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:14.148216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:14.148218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.148220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.148223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.148230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:14.148239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:14.148246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:14.148251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:14.148256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:14.148261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:14.148266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:14.148271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:14.148274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:14.148279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:14.148284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:14.148289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:14.148296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.148301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.148305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.148310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.148315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.148320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.148350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:14.166756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:14.168239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:14.168250Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:14.168253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:14.168259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:14.168267Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 264 [2:7559341101264960466:3988]
2025-10-09T21:44:14.168288Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:14.168312Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:14.168316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.168319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.168321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.168342Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:14.168348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:14.168356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:14.168433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.168435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.168437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.168439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:14.168673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:14.168678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.168694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.168705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.168720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:14.168743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341101264960460:3988]
2025-10-09T21:44:14.168745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341101264960460:3988]
2025-10-09T21:44:14.168747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341101264960460:3988]
2025-10-09T21:44:14.168768Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:14.168784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:14.168801Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:14.168804Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:14.168853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341101264960460:3988]
2025-10-09T21:44:14.168856Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:14.169684Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 264
2025-10-09T21:44:14.169737Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 264
2025-10-09T21:44:14.169311Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:14.169320Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:14.169323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:14.169330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:14.174739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341101626770826:4288], now have 1 active actors on pipe
2025-10-09T21:44:14.174759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341101626770748:2432], now have 1 active actors on pipe
2025-10-09T21:44:14.174795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:14.174803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:14.174846Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ff4a679d-8c492d5b-29677619-827dcb2c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:14.174861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.174867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.174870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:14.174887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.174908Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:14.174930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:14.175382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:14.175395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:14.175414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:14.175635Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 265 partition: 0 MaxSeqNo: 25 sessionId: src_id|ff4a679d-8c492d5b-29677619-827dcb2c_0
2025-10-09T21:44:14.176159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ac38406-dff68916-fb1e2385-d3df3ef0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ff4a679d-8c492d5b-29677619-827dcb2c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:14.188404Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff4a679d-8c492d5b-29677619-827dcb2c_0] Write session: try to update token
2025-10-09T21:44:14.188431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff4a679d-8c492d5b-29677619-827dcb2c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:14.202310Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:14.202327Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:14.202330Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:14.202332Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:14.202342Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341101264960460:3988]
2025-10-09T21:44:14.202346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341101264960460:3988]
2025-10-09T21:44:14.202351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341101264960460:3988]
2025-10-09T21:44:14.202411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:14.202435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.202441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:14.202599Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:14.204949Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341101626770843:3810]
2025-10-09T21:44:14.205110Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:14.205124Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:14.205128Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:14.205457Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 264, pipe: [1:7559341101626770843:3810]
2025-10-09T21:44:14.236812Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 265 sessionId: src_id|ff4a679d-8c492d5b-29677619-827dcb2c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:14.243933Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:14.393857Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:14.394164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341101626770843:3810], now have 1 active actors on pipe
2025-10-09T21:44:14.394240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:14.394268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:14.395105Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341101626770843:3810]
2025-10-09T21:44:14.395140Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:14.395143Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:14.395201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 265 reason: error at write: Disconnected sessionId: src_id|ff4a679d-8c492d5b-29677619-827dcb2c_0
2025-10-09T21:44:14.395406Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 265 sessionId: src_id|ff4a679d-8c492d5b-29677619-827dcb2c_0 is DEAD
2025-10-09T21:44:14.395592Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:14.396407Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:14.398530Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff4a679d-8c492d5b-29677619-827dcb2c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:14.398556Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff4a679d-8c492d5b-29677619-827dcb2c_0] Write session will restart in 0.006287s
2025-10-09T21:44:14.398584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff4a679d-8c492d5b-29677619-827dcb2c_0] Write session: Do CDS request
2025-10-09T21:44:14.398590Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff4a679d-8c492d5b-29677619-827dcb2c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:14.406624Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff4a679d-8c492d5b-29677619-827dcb2c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:14.407105Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:14.407122Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 266
2025-10-09T21:44:14.410280Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 266 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:14.410330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 266 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:14.410335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=266 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:14.410338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:14.410799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 266 sessionId:  describe result for acl check
2025-10-09T21:44:14.410839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:14.410841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:14.410843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:14.410847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341101626770883:4298] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:14.411369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341101626770883:4298] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:14.413976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341101626770883:4298] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:14.418101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341101626770883:4298] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:14.418117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341101626770883:4298] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:14.418123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341101626770883:4298] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:14.418131Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 266 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:14.423172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:14.427017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:14.427424Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:14.427438Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:14.427455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:14.427535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:14.427575Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341101264960594:4001]
2025-10-09T21:44:14.428164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:14.429028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:14.429101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:14.429448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:14.429513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:14.429519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:14.429521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:14.429522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:14.429523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:14.429525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:14.429526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.429527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.429529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.429535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:14.429545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:14.429549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:14.429553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:14.429557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:14.429563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:14.429567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.429570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.429573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.429580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:14.429588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:14.429593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:14.429598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:14.429602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:14.429607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:14.429610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:14.429615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:14.429621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:14.429627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:14.429631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:14.429636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:14.429641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.429646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.429650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.429655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.429660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.429665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.429694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:14.437281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:14.438725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:14.438743Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:14.438745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:14.438750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:14.438760Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 265 [2:7559341101264960594:4001]
2025-10-09T21:44:14.438774Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:14.438796Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:14.438807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.438811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.438813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.438831Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:14.438843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:14.438850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:14.438919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.438930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.438932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.438934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:14.439006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:14.439017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.439029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.439038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.439054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:14.439075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341101264960589:4001]
2025-10-09T21:44:14.439083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341101264960589:4001]
2025-10-09T21:44:14.439086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341101264960589:4001]
2025-10-09T21:44:14.439094Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:14.439101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:14.439103Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:14.439105Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:14.439111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:14.439124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:14.439127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:14.439140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341101626770949:4298], now have 1 active actors on pipe
2025-10-09T21:44:14.439150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341101626770862:2432], now have 1 active actors on pipe
2025-10-09T21:44:14.439187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341101264960589:4001]
2025-10-09T21:44:14.439202Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:14.439467Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:14.439468Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 265
2025-10-09T21:44:14.439507Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 265
2025-10-09T21:44:14.439718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:14.439731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:14.439781Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1baa4691-6c8bb4b1-3b91c99d-2570d1c1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:14.439796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.439816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.439819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:14.439830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.439843Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:14.439863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:14.440535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:14.440547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:14.440567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:14.442328Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 266 partition: 0 MaxSeqNo: 25 sessionId: src_id|1baa4691-6c8bb4b1-3b91c99d-2570d1c1_0
2025-10-09T21:44:14.446319Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff4a679d-8c492d5b-29677619-827dcb2c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1baa4691-6c8bb4b1-3b91c99d-2570d1c1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:14.448727Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1baa4691-6c8bb4b1-3b91c99d-2570d1c1_0] Write session: try to update token
2025-10-09T21:44:14.448748Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1baa4691-6c8bb4b1-3b91c99d-2570d1c1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:14.459882Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:14.459904Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:14.459906Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:14.459908Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:14.459919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341101264960589:4001]
2025-10-09T21:44:14.459923Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341101264960589:4001]
2025-10-09T21:44:14.459927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341101264960589:4001]
2025-10-09T21:44:14.459969Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:14.459974Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:14.459975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:14.459977Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:14.459982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.459985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:14.460126Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:14.485225Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 266 sessionId: src_id|1baa4691-6c8bb4b1-3b91c99d-2570d1c1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:14.488607Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:14.494396Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341101626770966:3810]
2025-10-09T21:44:14.494917Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 265, pipe: [1:7559341101626770966:3810]
2025-10-09T21:44:14.619084Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:14.619537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:14.619549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:14.619574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341101626770966:3810], now have 1 active actors on pipe
2025-10-09T21:44:14.620334Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:14.620350Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:14.620380Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:14.620408Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 266 reason: error at write: Disconnected sessionId: src_id|1baa4691-6c8bb4b1-3b91c99d-2570d1c1_0
2025-10-09T21:44:14.620425Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341101626770966:3810]
2025-10-09T21:44:14.620615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 266 sessionId: src_id|1baa4691-6c8bb4b1-3b91c99d-2570d1c1_0 is DEAD
2025-10-09T21:44:14.620865Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:14.622467Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1baa4691-6c8bb4b1-3b91c99d-2570d1c1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:14.622479Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1baa4691-6c8bb4b1-3b91c99d-2570d1c1_0] Write session will restart in 0.008268s
2025-10-09T21:44:14.622507Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1baa4691-6c8bb4b1-3b91c99d-2570d1c1_0] Write session: Do CDS request
2025-10-09T21:44:14.622513Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1baa4691-6c8bb4b1-3b91c99d-2570d1c1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:14.634516Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1baa4691-6c8bb4b1-3b91c99d-2570d1c1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:14.634877Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:14.634894Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 267
2025-10-09T21:44:14.635099Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 267 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:14.635132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 267 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:14.635138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=267 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:14.635142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:14.635631Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 267 sessionId:  describe result for acl check
2025-10-09T21:44:14.635668Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:14.635670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:14.635672Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:14.635678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341101626771018:4304] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:14.636249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341101626771018:4304] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:14.639867Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341101626771018:4304] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:14.644065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:14.646484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:14.646728Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:14.646733Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:14.646745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:14.646802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:14.646844Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341101264960681:4005]
2025-10-09T21:44:14.647436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:14.648486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:14.648548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:14.648904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:14.648962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:14.648963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:14.648967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:14.648968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:14.648970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:14.648971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:14.648972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.648973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.648975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.648982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:14.648992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:14.648995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:14.648999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:14.649002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:14.649005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:14.649009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.649013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.649016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.649024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:14.649044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:14.649050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:14.649056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:14.649060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:14.649065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:14.649070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:14.649075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:14.649079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:14.649085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:14.649089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:14.649095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:14.649100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.649105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.649109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.649116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.649121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.649126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.649158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:14.659115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341101626771018:4304] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:14.659134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341101626771018:4304] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:14.659138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341101626771018:4304] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:14.659149Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 267 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:14.660248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:14.661360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:14.661367Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:14.661369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:14.661374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:14.661382Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 266 [2:7559341101264960681:4005]
2025-10-09T21:44:14.661398Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:14.661418Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:14.661421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.661425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.661427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.661447Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:14.661453Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:14.661461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:14.661528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.661531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.661533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.661536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:14.661621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:14.661624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.661637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.661649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.661666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:14.661687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341101264960675:4005]
2025-10-09T21:44:14.661690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341101264960675:4005]
2025-10-09T21:44:14.661691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341101264960675:4005]
2025-10-09T21:44:14.661708Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:14.661727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:14.661743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:14.661747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:14.661788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341101264960675:4005]
2025-10-09T21:44:14.661791Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:14.662010Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:14.662013Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:14.662019Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:14.662026Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:14.663001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341101626770967:2432], now have 1 active actors on pipe
2025-10-09T21:44:14.663015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341101626771056:4304], now have 1 active actors on pipe
2025-10-09T21:44:14.663500Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 266
2025-10-09T21:44:14.663540Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 266
2025-10-09T21:44:14.668007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:14.668031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:14.668073Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ccc27730-86163002-16e2b734-223f1137_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:14.668087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.668094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.668098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:14.668115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.668135Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:14.668157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:14.670730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 267 partition: 0 MaxSeqNo: 25 sessionId: src_id|ccc27730-86163002-16e2b734-223f1137_0
2025-10-09T21:44:14.672039Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1baa4691-6c8bb4b1-3b91c99d-2570d1c1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ccc27730-86163002-16e2b734-223f1137_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:14.670279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:14.670298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:14.670330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:14.674556Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ccc27730-86163002-16e2b734-223f1137_0] Write session: try to update token
2025-10-09T21:44:14.674577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ccc27730-86163002-16e2b734-223f1137_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:14.685775Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:14.685796Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:14.685799Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:14.685801Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:14.685814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341101264960675:4005]
2025-10-09T21:44:14.685818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341101264960675:4005]
2025-10-09T21:44:14.685821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341101264960675:4005]
2025-10-09T21:44:14.685866Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:14.685870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:14.685876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.685877Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:14.685879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:14.685881Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:14.686021Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:14.714878Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 267 sessionId: src_id|ccc27730-86163002-16e2b734-223f1137_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:14.717465Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:14.727226Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341101626771065:3810]
2025-10-09T21:44:14.727853Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 266, pipe: [1:7559341101626771065:3810]
2025-10-09T21:44:14.951332Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:14.951641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:14.951656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:14.951680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341101626771065:3810], now have 1 active actors on pipe
2025-10-09T21:44:14.954931Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341101626771065:3810]
2025-10-09T21:44:14.954968Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:14.955041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 267 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ccc27730-86163002-16e2b734-223f1137_0
2025-10-09T21:44:14.955222Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 267 sessionId: src_id|ccc27730-86163002-16e2b734-223f1137_0 is DEAD
2025-10-09T21:44:14.955347Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:14.953689Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:14.958141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ccc27730-86163002-16e2b734-223f1137_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:14.958154Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ccc27730-86163002-16e2b734-223f1137_0] Write session will restart in 0.006108s
2025-10-09T21:44:14.958180Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ccc27730-86163002-16e2b734-223f1137_0] Write session: Do CDS request
2025-10-09T21:44:14.958187Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ccc27730-86163002-16e2b734-223f1137_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:14.966502Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ccc27730-86163002-16e2b734-223f1137_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:14.966663Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:14.966678Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 268
2025-10-09T21:44:14.966877Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 268 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:14.966911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 268 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:14.966924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=268 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:14.966927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:14.969695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 268 sessionId:  describe result for acl check
2025-10-09T21:44:14.969737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:14.969739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:14.969740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:14.969745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341101626771100:4309] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:14.970303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341101626771100:4309] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:14.982051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:14.981041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341101626771100:4309] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:14.990719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341101626771100:4309] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:14.990740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341101626771100:4309] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:14.990744Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341101626771100:4309] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:14.990756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 268 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:14.994184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:14.994470Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:14.994476Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:14.994510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:14.994580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:14.994621Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341101264960773:4007]
2025-10-09T21:44:14.995172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:14.998470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:14.998557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:14.998911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:14.998974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:14.998976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:14.998977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:14.998979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:14.998980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:14.998981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:14.998982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.998984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.998988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.998994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:14.999005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:14.999008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:14.999013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:14.999016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:14.999019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:14.999023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.999026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.999029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.999037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:14.999045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:14.999051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:14.999056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:14.999061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:14.999066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:14.999070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:14.999074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:14.999080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:14.999085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:14.999089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:14.999094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:14.999100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.999105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:14.999110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.999115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:14.999119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.999124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:14.999157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:15.008980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:15.010121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:15.010140Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:15.010143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:15.010151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:15.010189Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 267 [2:7559341101264960773:4007]
2025-10-09T21:44:15.010206Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:15.010228Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:15.010231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.010235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.010237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.010256Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:15.010262Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:15.010270Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:15.010344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.010346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.010348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.010351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:15.010449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:15.010452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.010481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.010491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.010506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:15.010527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341101264960766:4007]
2025-10-09T21:44:15.010530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341101264960766:4007]
2025-10-09T21:44:15.010532Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341101264960766:4007]
2025-10-09T21:44:15.010553Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:15.010569Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:15.010583Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:15.010587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:15.010618Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:15.010622Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:15.010625Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:15.010633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341101264960766:4007]
2025-10-09T21:44:15.010636Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:15.010724Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:15.010900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341101626771167:4309], now have 1 active actors on pipe
2025-10-09T21:44:15.010908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341101626771085:2432], now have 1 active actors on pipe
2025-10-09T21:44:15.014150Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 267
2025-10-09T21:44:15.014206Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 267
2025-10-09T21:44:15.022652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:15.022685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:15.022730Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e3f561a3-63cd0031-25330e2-758073e2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:15.022745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.022761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.022770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:15.022787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.022813Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:15.022843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:15.023299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:15.023317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:15.023340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:15.023499Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 268 partition: 0 MaxSeqNo: 25 sessionId: src_id|e3f561a3-63cd0031-25330e2-758073e2_0
2025-10-09T21:44:15.030285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ccc27730-86163002-16e2b734-223f1137_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e3f561a3-63cd0031-25330e2-758073e2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:15.032957Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3f561a3-63cd0031-25330e2-758073e2_0] Write session: try to update token
2025-10-09T21:44:15.032988Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3f561a3-63cd0031-25330e2-758073e2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:15.037999Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:15.038020Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:15.038023Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:15.038025Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:15.038050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341101264960766:4007]
2025-10-09T21:44:15.038056Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341101264960766:4007]
2025-10-09T21:44:15.038061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341101264960766:4007]
2025-10-09T21:44:15.038116Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:15.038120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:15.038122Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:15.038126Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:15.038131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.038134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:15.038278Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:15.054372Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341105921738475:3810]
2025-10-09T21:44:15.054919Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 267, pipe: [1:7559341105921738475:3810]
2025-10-09T21:44:15.089046Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 268 sessionId: src_id|e3f561a3-63cd0031-25330e2-758073e2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:15.091669Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:15.212063Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:15.212651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341105921738475:3810], now have 1 active actors on pipe
2025-10-09T21:44:15.212714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:15.212722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:15.213123Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:15.213140Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:15.213884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3f561a3-63cd0031-25330e2-758073e2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:15.213896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3f561a3-63cd0031-25330e2-758073e2_0] Write session will restart in 0.008056s
2025-10-09T21:44:15.213921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3f561a3-63cd0031-25330e2-758073e2_0] Write session: Do CDS request
2025-10-09T21:44:15.213928Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3f561a3-63cd0031-25330e2-758073e2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:15.213214Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 268 reason: error at write: Disconnected sessionId: src_id|e3f561a3-63cd0031-25330e2-758073e2_0
2025-10-09T21:44:15.213422Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 268 sessionId: src_id|e3f561a3-63cd0031-25330e2-758073e2_0 is DEAD
2025-10-09T21:44:15.213551Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:15.213559Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341105921738475:3810]
2025-10-09T21:44:15.213298Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:15.223580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3f561a3-63cd0031-25330e2-758073e2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:15.223765Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:15.223785Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 269
2025-10-09T21:44:15.224058Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 269 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:15.224111Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 269 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:15.224122Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=269 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:15.224126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:15.225485Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 269 sessionId:  describe result for acl check
2025-10-09T21:44:15.225544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:15.225552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:15.225554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:15.225561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341105921738486:4313] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:15.226193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341105921738486:4313] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:15.229454Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341105921738486:4313] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:15.235319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341105921738486:4313] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:15.235337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341105921738486:4313] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:15.235342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341105921738486:4313] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:15.235354Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 269 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:15.262925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:15.271108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:15.271380Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:15.271395Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:15.271408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:15.271470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:15.271521Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341105559928164:4011]
2025-10-09T21:44:15.272133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:15.272632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:15.272705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:15.273046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:15.273107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:15.273108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:15.273109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:15.273110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:15.273112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:15.273113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:15.273114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.273116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.273118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.273124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:15.273136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:15.273140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:15.273144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:15.273148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:15.273152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:15.273157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.273161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.273165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.273176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:15.273185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:15.273192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:15.273198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:15.273204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:15.273210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:15.273215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:15.273220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:15.273225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:15.273229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:15.273233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:15.273238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:15.273243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.273247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.273252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.273257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.273261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.273266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.273297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:15.283302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:15.284412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:15.284433Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:15.284437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:15.284441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:15.284450Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 268 [2:7559341105559928164:4011]
2025-10-09T21:44:15.284468Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:15.284492Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:15.284505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.284508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.284511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.284534Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:15.284546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:15.284554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:15.284630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.284638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.284641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.284643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:15.284721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:15.284730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.284744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.284760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.284780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:15.284807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341105559928158:4011]
2025-10-09T21:44:15.284814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341105559928158:4011]
2025-10-09T21:44:15.284816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341105559928158:4011]
2025-10-09T21:44:15.284827Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:15.284846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:15.284865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:15.284873Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:15.284919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341105559928158:4011]
2025-10-09T21:44:15.284926Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:15.285254Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:15.285263Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:15.285265Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:15.285272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:15.289331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341105921738478:2432], now have 1 active actors on pipe
2025-10-09T21:44:15.289355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341105921738522:4313], now have 1 active actors on pipe
2025-10-09T21:44:15.289385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:15.289392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:15.289437Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4163254f-6ae81644-a031cc8d-4ad918fb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:15.289453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.289465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.289475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:15.289499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.289523Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:15.289548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:15.290138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:15.290161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:15.290181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:15.286568Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 268
2025-10-09T21:44:15.286626Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 268
2025-10-09T21:44:15.290371Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 269 partition: 0 MaxSeqNo: 25 sessionId: src_id|4163254f-6ae81644-a031cc8d-4ad918fb_0
2025-10-09T21:44:15.291263Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3f561a3-63cd0031-25330e2-758073e2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4163254f-6ae81644-a031cc8d-4ad918fb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:15.293792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4163254f-6ae81644-a031cc8d-4ad918fb_0] Write session: try to update token
2025-10-09T21:44:15.293811Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4163254f-6ae81644-a031cc8d-4ad918fb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:15.313406Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341105921738576:3810]
2025-10-09T21:44:15.314695Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 268, pipe: [1:7559341105921738576:3810]
2025-10-09T21:44:15.315653Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 269 sessionId: src_id|4163254f-6ae81644-a031cc8d-4ad918fb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:15.318117Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:15.319215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341105921738576:3810], now have 1 active actors on pipe
2025-10-09T21:44:15.322223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:15.322246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:15.322269Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:15.322273Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:15.322276Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:15.322278Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:15.322286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341105559928158:4011]
2025-10-09T21:44:15.322290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341105559928158:4011]
2025-10-09T21:44:15.322294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341105559928158:4011]
2025-10-09T21:44:15.322356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:15.322367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.322370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:15.322548Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:15.322771Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:15.322777Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:15.322780Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:15.548479Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:15.549044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:15.549931Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:15.555562Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:15.555632Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341105921738576:3810]
2025-10-09T21:44:15.563139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:15.563154Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:15.563194Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 269 reason: error at write: Disconnected sessionId: src_id|4163254f-6ae81644-a031cc8d-4ad918fb_0
2025-10-09T21:44:15.563353Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 269 sessionId: src_id|4163254f-6ae81644-a031cc8d-4ad918fb_0 is DEAD
2025-10-09T21:44:15.567832Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4163254f-6ae81644-a031cc8d-4ad918fb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:15.567848Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4163254f-6ae81644-a031cc8d-4ad918fb_0] Write session will restart in 0.008217s
2025-10-09T21:44:15.567875Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4163254f-6ae81644-a031cc8d-4ad918fb_0] Write session: Do CDS request
2025-10-09T21:44:15.567883Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4163254f-6ae81644-a031cc8d-4ad918fb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:15.577583Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4163254f-6ae81644-a031cc8d-4ad918fb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:15.578200Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:15.578219Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 270
2025-10-09T21:44:15.578632Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 270 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:15.578678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 270 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:15.578685Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=270 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:15.578689Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:15.579895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 270 sessionId:  describe result for acl check
2025-10-09T21:44:15.579936Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:15.579938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:15.579940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:15.579947Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341105921738664:4327] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:15.580535Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341105921738664:4327] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:15.585028Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341105921738664:4327] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:15.586268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:15.586758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:15.586934Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:15.586945Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:15.586958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:15.587011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:15.587048Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341105559928299:4027]
2025-10-09T21:44:15.587629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:15.588139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:15.588190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:15.588530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:15.588728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:15.588730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:15.588731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:15.588732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:15.588733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:15.588734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:15.588736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.588737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.588738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.588744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:15.588754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:15.588758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:15.588784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:15.588788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:15.588791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:15.588795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.588798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.588801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.588810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:15.588817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:15.588824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:15.588831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:15.588836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:15.588841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:15.588845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:15.588849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:15.588853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:15.588857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:15.588861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:15.588866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:15.588872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.588876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.588881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.588885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.588889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.588894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.588920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:15.597765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341105921738664:4327] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:15.597785Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341105921738664:4327] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:15.597789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341105921738664:4327] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:15.597799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 270 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:15.598131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:15.599478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:15.599495Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:15.599498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:15.599503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:15.599510Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 269 [2:7559341105559928299:4027]
2025-10-09T21:44:15.599524Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:15.599544Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:15.599547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.599551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.599554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.599573Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:15.599580Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:15.599587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:15.599653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.599656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.599658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.599660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:15.599741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:15.599745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.599758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.600688Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 269
2025-10-09T21:44:15.600735Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 269
2025-10-09T21:44:15.601903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 270 partition: 0 MaxSeqNo: 25 sessionId: src_id|650c4338-6b9077ed-5716f2d2-ac4ba152_0
2025-10-09T21:44:15.599770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.599791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:15.599812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341105559928293:4027]
2025-10-09T21:44:15.599814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341105559928293:4027]
2025-10-09T21:44:15.599816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341105559928293:4027]
2025-10-09T21:44:15.599831Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:15.599847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:15.599861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:15.599865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:15.599906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341105559928293:4027]
2025-10-09T21:44:15.599909Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:15.599926Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:15.599930Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:15.599933Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:15.599939Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:15.601176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341105921738705:4327], now have 1 active actors on pipe
2025-10-09T21:44:15.601183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341105921738617:2432], now have 1 active actors on pipe
2025-10-09T21:44:15.601206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:15.601212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:15.601249Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|650c4338-6b9077ed-5716f2d2-ac4ba152_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:15.601261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.601266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.601269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:15.601283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.601300Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:15.601317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:15.601691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:15.601697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:15.601716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:15.602425Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4163254f-6ae81644-a031cc8d-4ad918fb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|650c4338-6b9077ed-5716f2d2-ac4ba152_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:15.605110Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|650c4338-6b9077ed-5716f2d2-ac4ba152_0] Write session: try to update token
2025-10-09T21:44:15.605133Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|650c4338-6b9077ed-5716f2d2-ac4ba152_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:15.633702Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:15.633734Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:15.633737Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:15.633739Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:15.633750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341105559928293:4027]
2025-10-09T21:44:15.633755Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341105559928293:4027]
2025-10-09T21:44:15.633759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341105559928293:4027]
2025-10-09T21:44:15.633809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:15.633817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.633820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:15.633976Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:15.635485Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 270 sessionId: src_id|650c4338-6b9077ed-5716f2d2-ac4ba152_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:15.638300Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:15.638646Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:15.638661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:15.638665Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:15.655964Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341105921738714:3810]
2025-10-09T21:44:15.657385Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 269, pipe: [1:7559341105921738714:3810]
2025-10-09T21:44:15.911590Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:15.911992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:15.912011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:15.912061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341105921738714:3810], now have 1 active actors on pipe
2025-10-09T21:44:15.916571Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:15.918009Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:15.919200Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|650c4338-6b9077ed-5716f2d2-ac4ba152_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:15.919213Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|650c4338-6b9077ed-5716f2d2-ac4ba152_0] Write session will restart in 0.008024s
2025-10-09T21:44:15.919249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|650c4338-6b9077ed-5716f2d2-ac4ba152_0] Write session: Do CDS request
2025-10-09T21:44:15.919256Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|650c4338-6b9077ed-5716f2d2-ac4ba152_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:15.918199Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 270 reason: status is not ok: tablet will be restarted right now sessionId: src_id|650c4338-6b9077ed-5716f2d2-ac4ba152_0
2025-10-09T21:44:15.918400Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 270 sessionId: src_id|650c4338-6b9077ed-5716f2d2-ac4ba152_0 is DEAD
2025-10-09T21:44:15.918604Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:15.918649Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341105921738714:3810]
2025-10-09T21:44:15.928689Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|650c4338-6b9077ed-5716f2d2-ac4ba152_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:15.929874Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:15.929894Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 271
2025-10-09T21:44:15.930611Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 271 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:15.930676Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 271 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:15.930686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=271 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:15.930690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:15.931212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 271 sessionId:  describe result for acl check
2025-10-09T21:44:15.931252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:15.931254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:15.931255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:15.931262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341105921738738:4332] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:15.931903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341105921738738:4332] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:15.936932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341105921738738:4332] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:15.943609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:15.945578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:15.945843Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:15.945847Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:15.945862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:15.945932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:15.945985Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341105559928392:4029]
2025-10-09T21:44:15.946620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:15.947192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:15.947257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:15.946759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341105921738738:4332] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:15.946780Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341105921738738:4332] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:15.946787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341105921738738:4332] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:15.946798Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 271 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:15.947708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:15.947791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:15.947804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:15.947805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:15.947806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:15.947808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:15.947809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:15.947810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.947812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.947813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.947819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:15.947847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:15.947851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:15.947855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:15.947858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:15.947861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:15.947864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.947867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.947871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.947879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:15.947888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:15.947894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:15.947900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:15.947906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:15.947911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:15.947915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:15.947923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:15.947928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:15.947934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:15.947938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:15.947944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:15.947949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.947954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.947969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.947974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.947979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.947983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.948017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:15.960061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:15.961567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:15.961574Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:15.961577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:15.961582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:15.961591Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 270 [2:7559341105559928392:4029]
2025-10-09T21:44:15.961610Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:15.961632Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:15.961636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.961639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.961641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.961659Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:15.961666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:15.961673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:15.961743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.961745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.961747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.961750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:15.961855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:15.961858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.961873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:15.961884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:15.961903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:15.961926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341105559928386:4029]
2025-10-09T21:44:15.961928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341105559928386:4029]
2025-10-09T21:44:15.961930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341105559928386:4029]
2025-10-09T21:44:15.961939Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:15.961955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:15.961971Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:15.961973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:15.961973Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:15.961977Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:15.961980Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:15.962018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341105559928386:4029]
2025-10-09T21:44:15.962021Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:15.962025Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:15.962080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341105921738814:4332], now have 1 active actors on pipe
2025-10-09T21:44:15.962085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341105921738727:2432], now have 1 active actors on pipe
2025-10-09T21:44:15.962690Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 270
2025-10-09T21:44:15.962732Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 270
2025-10-09T21:44:15.963049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:15.963066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:15.963098Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2c69e47b-5cfa072b-134d52c8-27c0fd0f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:15.963109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.963114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.963117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:15.963131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.963146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:15.963164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:15.963573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:15.963584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:15.963605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:15.964337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|650c4338-6b9077ed-5716f2d2-ac4ba152_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2c69e47b-5cfa072b-134d52c8-27c0fd0f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:15.967009Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c69e47b-5cfa072b-134d52c8-27c0fd0f_0] Write session: try to update token
2025-10-09T21:44:15.967033Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c69e47b-5cfa072b-134d52c8-27c0fd0f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:15.963818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 271 partition: 0 MaxSeqNo: 25 sessionId: src_id|2c69e47b-5cfa072b-134d52c8-27c0fd0f_0
2025-10-09T21:44:15.991998Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:15.992018Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:15.992020Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:15.992022Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:15.992034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341105559928386:4029]
2025-10-09T21:44:15.992039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341105559928386:4029]
2025-10-09T21:44:15.992043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341105559928386:4029]
2025-10-09T21:44:15.992084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:15.992091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:15.992094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:15.992236Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:15.994439Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:15.994457Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:15.994461Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:16.003681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 271 sessionId: src_id|2c69e47b-5cfa072b-134d52c8-27c0fd0f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:16.006289Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:16.019923Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341110216706123:3810]
2025-10-09T21:44:16.022618Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 270, pipe: [1:7559341110216706123:3810]
2025-10-09T21:44:16.183158Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:16.183605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:16.183624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:16.183649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341110216706123:3810], now have 1 active actors on pipe
2025-10-09T21:44:16.184270Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:16.184285Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:16.184362Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 271 reason: error at write: Disconnected sessionId: src_id|2c69e47b-5cfa072b-134d52c8-27c0fd0f_0
2025-10-09T21:44:16.184104Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:16.184448Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341110216706123:3810]
2025-10-09T21:44:16.184467Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:16.184573Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 271 sessionId: src_id|2c69e47b-5cfa072b-134d52c8-27c0fd0f_0 is DEAD
2025-10-09T21:44:16.185123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c69e47b-5cfa072b-134d52c8-27c0fd0f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:16.185136Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c69e47b-5cfa072b-134d52c8-27c0fd0f_0] Write session will restart in 0.008220s
2025-10-09T21:44:16.185163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c69e47b-5cfa072b-134d52c8-27c0fd0f_0] Write session: Do CDS request
2025-10-09T21:44:16.185184Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c69e47b-5cfa072b-134d52c8-27c0fd0f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:16.195497Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c69e47b-5cfa072b-134d52c8-27c0fd0f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:16.195537Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:16.195554Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 272
2025-10-09T21:44:16.195793Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 272 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:16.195823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 272 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:16.195829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=272 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:16.195833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:16.196323Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 272 sessionId:  describe result for acl check
2025-10-09T21:44:16.196363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:16.196365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:16.196366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:16.196372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341110216706149:4336] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:16.197007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341110216706149:4336] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:16.211534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341110216706149:4336] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:16.214617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:16.214978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:16.215158Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:16.215161Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:16.215174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:16.215226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:16.215267Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341109854895785:4033]
2025-10-09T21:44:16.215898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:16.217087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:16.217147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:16.217473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:16.217531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:16.217532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:16.217533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:16.217537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:16.217538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:16.217539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:16.217540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.217541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.217542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.217549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:16.217560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:16.217563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:16.217568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:16.217572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:16.217575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:16.217579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.217582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.217585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.217594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:16.217602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:16.217607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:16.217612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:16.217616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:16.217621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:16.217625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:16.217630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:16.217634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:16.217638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:16.217642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:16.217647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:16.217655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.217659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.217663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.217667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.217673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.217679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.217707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:16.220549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341110216706149:4336] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:16.220569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341110216706149:4336] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:16.220573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341110216706149:4336] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:16.220583Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 272 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:16.229258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:16.230624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:16.230645Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:16.230647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:16.230652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:16.230660Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 271 [2:7559341109854895785:4033]
2025-10-09T21:44:16.230679Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:16.230700Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:16.230704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.230707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.230709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.230730Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:16.230741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:16.230749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:16.230831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.230839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.230842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.230844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:16.230985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:16.230996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.231011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.231025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.231043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:16.231068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341110216706126:2432], now have 1 active actors on pipe
2025-10-09T21:44:16.231072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341110216706218:4336], now have 1 active actors on pipe
2025-10-09T21:44:16.231081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341109854895780:4033]
2025-10-09T21:44:16.231083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341109854895780:4033]
2025-10-09T21:44:16.231084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341109854895780:4033]
2025-10-09T21:44:16.231100Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:16.231116Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:16.231118Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:16.231107Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 271
2025-10-09T21:44:16.231168Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 271
2025-10-09T21:44:16.231119Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:16.231122Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:16.231131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:16.231134Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:16.231180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341109854895780:4033]
2025-10-09T21:44:16.231189Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:16.231229Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:16.231429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:16.231450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:16.231486Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|90bafa3d-fa4df62b-fef6cace-5aca9d05_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:16.231497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.231501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.231504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:16.231517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.231529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:16.231569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:16.232285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:16.232298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:16.232330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:16.232553Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 272 partition: 0 MaxSeqNo: 25 sessionId: src_id|90bafa3d-fa4df62b-fef6cace-5aca9d05_0
2025-10-09T21:44:16.233363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c69e47b-5cfa072b-134d52c8-27c0fd0f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|90bafa3d-fa4df62b-fef6cace-5aca9d05_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:16.236023Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90bafa3d-fa4df62b-fef6cace-5aca9d05_0] Write session: try to update token
2025-10-09T21:44:16.236045Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90bafa3d-fa4df62b-fef6cace-5aca9d05_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:16.263942Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:16.263958Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:16.263969Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341109854895780:4033]
2025-10-09T21:44:16.264023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:16.264088Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:16.264214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:16.264351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:16.264489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:16.264804Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:16.265734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:16.265785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:16.280996Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 272 sessionId: src_id|90bafa3d-fa4df62b-fef6cace-5aca9d05_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:16.283851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:16.281562Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:16.281584Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:16.281586Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:16.281588Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:16.281602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341109854895780:4033]
2025-10-09T21:44:16.281606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341109854895780:4033]
2025-10-09T21:44:16.281611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341109854895780:4033]
2025-10-09T21:44:16.281659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:16.281666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.281669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:16.281817Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:16.284507Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:16.284513Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:16.284517Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:16.289283Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341110216706226:3810]
2025-10-09T21:44:16.289848Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 271, pipe: [1:7559341110216706226:3810]
2025-10-09T21:44:16.465277Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:16.465606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:16.465624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:16.465648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341110216706226:3810], now have 1 active actors on pipe
2025-10-09T21:44:16.466106Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:16.466451Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:16.466508Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:16.466530Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341110216706226:3810]
2025-10-09T21:44:16.466605Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 272 reason: status is not ok: tablet will be restarted right now sessionId: src_id|90bafa3d-fa4df62b-fef6cace-5aca9d05_0
2025-10-09T21:44:16.466820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 272 sessionId: src_id|90bafa3d-fa4df62b-fef6cace-5aca9d05_0 is DEAD
2025-10-09T21:44:16.467022Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90bafa3d-fa4df62b-fef6cace-5aca9d05_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:16.467030Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90bafa3d-fa4df62b-fef6cace-5aca9d05_0] Write session will restart in 0.009095s
2025-10-09T21:44:16.467057Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90bafa3d-fa4df62b-fef6cace-5aca9d05_0] Write session: Do CDS request
2025-10-09T21:44:16.467064Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90bafa3d-fa4df62b-fef6cace-5aca9d05_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:16.478563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90bafa3d-fa4df62b-fef6cace-5aca9d05_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:16.478911Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:16.478929Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 273
2025-10-09T21:44:16.479127Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 273 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:16.479168Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 273 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:16.479187Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=273 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:16.479191Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:16.479598Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 273 sessionId:  describe result for acl check
2025-10-09T21:44:16.479634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:16.479640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:16.479643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:16.479649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341110216706277:4345] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:16.480201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341110216706277:4345] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:16.488242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341110216706277:4345] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:16.489734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:16.491494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:16.491803Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:16.491815Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:16.491830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:16.491896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:16.491943Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341109854895897:4041]
2025-10-09T21:44:16.492510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:16.492986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:16.493044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:16.493374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:16.493423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:16.493430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:16.493431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:16.493432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:16.493433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:16.493434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:16.493435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.493437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.493438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.493443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:16.493453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:16.493457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:16.493461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:16.493464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:16.493468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:16.493471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.493474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.493478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.493484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:16.493492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:16.493499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:16.493505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:16.493511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:16.493515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:16.493519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:16.493524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:16.493528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:16.493532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:16.493536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:16.493541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:16.493545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.493550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.493554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.493559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.493563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.493568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.493596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:16.501041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341110216706277:4345] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:16.501060Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341110216706277:4345] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:16.501065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341110216706277:4345] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:16.501075Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 273 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:16.504617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:16.506744Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 272
2025-10-09T21:44:16.506780Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 272
2025-10-09T21:44:16.505729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:16.505747Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:16.505750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:16.505755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:16.505763Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 272 [2:7559341109854895897:4041]
2025-10-09T21:44:16.505777Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:16.505797Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:16.505801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.505804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.505807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.505826Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:16.505832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:16.505840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:16.505912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.505914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.505916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.505918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:16.505994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:16.505997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.506010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.506022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.506055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:16.506078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341109854895887:4041]
2025-10-09T21:44:16.506081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341109854895887:4041]
2025-10-09T21:44:16.506083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341109854895887:4041]
2025-10-09T21:44:16.506093Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:16.506108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:16.506112Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:16.506115Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:16.506118Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:16.506125Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:16.506128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:16.506140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341110216706338:4345], now have 1 active actors on pipe
2025-10-09T21:44:16.506144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341110216706231:2432], now have 1 active actors on pipe
2025-10-09T21:44:16.506184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341109854895887:4041]
2025-10-09T21:44:16.506193Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:16.506260Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:16.507937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:16.507953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:16.507994Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|eb7ea211-2093cb2-6173e99-14ffc50b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:16.508005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.508010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.508012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:16.508024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.508039Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:16.508059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:16.508813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:16.508826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:16.508849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:16.509139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 273 partition: 0 MaxSeqNo: 25 sessionId: src_id|eb7ea211-2093cb2-6173e99-14ffc50b_0
2025-10-09T21:44:16.510581Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90bafa3d-fa4df62b-fef6cace-5aca9d05_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|eb7ea211-2093cb2-6173e99-14ffc50b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:16.513175Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eb7ea211-2093cb2-6173e99-14ffc50b_0] Write session: try to update token
2025-10-09T21:44:16.513196Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eb7ea211-2093cb2-6173e99-14ffc50b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:16.548784Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 273 sessionId: src_id|eb7ea211-2093cb2-6173e99-14ffc50b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:16.551438Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:16.552411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:16.552436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:16.552480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:16.557998Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:16.558018Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:16.558021Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:16.554022Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:16.558023Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:16.558066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341109854895887:4041]
2025-10-09T21:44:16.558074Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341109854895887:4041]
2025-10-09T21:44:16.558078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341109854895887:4041]
2025-10-09T21:44:16.558128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:16.558135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.558138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:16.558137Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:16.558144Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:16.558148Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:16.558283Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:16.567319Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341110216706355:3810]
2025-10-09T21:44:16.567983Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 272, pipe: [1:7559341110216706355:3810]
2025-10-09T21:44:16.756221Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:16.756544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341110216706355:3810], now have 1 active actors on pipe
2025-10-09T21:44:16.765510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:16.765541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:16.766436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:16.766514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:16.766584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:16.766644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:16.766818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:16.766824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:16.766896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:16.766953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:16.767013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:16.767073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:16.767115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:16.767119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:16.767191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:16.767247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:16.767308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:16.767410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:16.767451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:16.767454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:16.767528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:16.767595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:16.767659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:16.767721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:16.767761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:16.767764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:16.767864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:16.767927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:16.768036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:16.768105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:16.768173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:16.768177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:16.768241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:16.768296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:16.768357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:16.768417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:16.768457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:16.768461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:16.768529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:16.768583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:16.768646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:16.768703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:16.768743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:16.768747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:16.768812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:16.768868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:16.768926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:16.768981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:16.769021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:16.769023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:16.771139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:16.771156Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:16.771192Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 273 reason: error at write: Disconnected sessionId: src_id|eb7ea211-2093cb2-6173e99-14ffc50b_0
2025-10-09T21:44:16.769560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:16.769642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:16.769705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:16.769769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:16.769809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:16.769812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:16.769879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:16.769937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:16.770000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:16.770075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:16.770117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:16.770121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:16.771300Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:16.771997Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb7ea211-2093cb2-6173e99-14ffc50b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:16.772008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb7ea211-2093cb2-6173e99-14ffc50b_0] Write session will restart in 0.008726s
2025-10-09T21:44:16.772034Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb7ea211-2093cb2-6173e99-14ffc50b_0] Write session: Do CDS request
2025-10-09T21:44:16.772040Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb7ea211-2093cb2-6173e99-14ffc50b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:16.771390Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 273 sessionId: src_id|eb7ea211-2093cb2-6173e99-14ffc50b_0 is DEAD
2025-10-09T21:44:16.771540Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:16.771573Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341110216706355:3810]
2025-10-09T21:44:16.782162Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eb7ea211-2093cb2-6173e99-14ffc50b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:16.782616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:16.782640Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 274
2025-10-09T21:44:16.783312Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 274 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:16.783374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 274 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:16.783381Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=274 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:16.783385Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:16.784933Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 274 sessionId:  describe result for acl check
2025-10-09T21:44:16.784969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:16.784971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:16.784973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:16.784980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341110216706375:4354] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:16.785583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341110216706375:4354] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:16.801201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341110216706375:4354] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:16.803588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:16.811292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:16.811565Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:16.811569Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:16.811582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:16.811651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:16.811698Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341109854896014:4049]
2025-10-09T21:44:16.812339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:16.812693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341110216706375:4354] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:16.812715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341110216706375:4354] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:16.812720Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341110216706375:4354] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:16.812731Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 274 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:16.813800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:16.813875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:16.814250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:16.814314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:16.814315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:16.814316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:16.814317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:16.814318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:16.814319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:16.814320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.814322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.814323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.814330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:16.814341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:16.814345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:16.814348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:16.814352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:16.814355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:16.814358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.814362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.814365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.814374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:16.814382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:16.814389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:16.814397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:16.814401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:16.814406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:16.814426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:16.814431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:16.814436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:16.814441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:16.814447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:16.814451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:16.814456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.814461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.814466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.814471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.814476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.814481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.814511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:16.823942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:16.825457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:16.825466Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:16.825469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:16.825473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:16.825485Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 273 [2:7559341109854896014:4049]
2025-10-09T21:44:16.825501Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:16.825522Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:16.825528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.825531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.825533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.825552Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:16.825559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:16.825567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:16.825629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.825631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.825634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.825637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:16.825717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:16.825720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.825735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:16.825748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:16.825775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:16.825795Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341109854896008:4049]
2025-10-09T21:44:16.825798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341109854896008:4049]
2025-10-09T21:44:16.825800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341109854896008:4049]
2025-10-09T21:44:16.825813Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:16.825828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:16.825843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:16.825846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:16.825924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341109854896008:4049]
2025-10-09T21:44:16.825930Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:16.826492Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:16.826503Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:16.826505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:16.826512Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:16.827483Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 273
2025-10-09T21:44:16.827520Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 273
2025-10-09T21:44:16.828166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341110216706452:4354], now have 1 active actors on pipe
2025-10-09T21:44:16.828182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341110216706361:2432], now have 1 active actors on pipe
2025-10-09T21:44:16.828205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:16.828212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:16.828261Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4140ac5b-c83d2413-a908f054-ec9dbaac_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:16.828278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.828291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.828294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:16.828308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.828332Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:16.828358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:16.833238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:16.833260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:16.833316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:16.833594Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 274 partition: 0 MaxSeqNo: 25 sessionId: src_id|4140ac5b-c83d2413-a908f054-ec9dbaac_0
2025-10-09T21:44:16.834070Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb7ea211-2093cb2-6173e99-14ffc50b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4140ac5b-c83d2413-a908f054-ec9dbaac_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:16.836648Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4140ac5b-c83d2413-a908f054-ec9dbaac_0] Write session: try to update token
2025-10-09T21:44:16.836671Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4140ac5b-c83d2413-a908f054-ec9dbaac_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:16.861561Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 274 sessionId: src_id|4140ac5b-c83d2413-a908f054-ec9dbaac_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:16.864237Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:16.863103Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:16.863124Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:16.863127Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:16.863129Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:16.863138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341109854896008:4049]
2025-10-09T21:44:16.863143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341109854896008:4049]
2025-10-09T21:44:16.863146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341109854896008:4049]
2025-10-09T21:44:16.863189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:16.863197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:16.863199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:16.863333Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:16.867319Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:16.867335Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:16.867338Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:16.872436Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341110216706467:3810]
2025-10-09T21:44:16.875558Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 273, pipe: [1:7559341110216706467:3810]
2025-10-09T21:44:17.064263Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:17.064591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:17.064606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:17.064634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341110216706467:3810], now have 1 active actors on pipe
2025-10-09T21:44:17.065414Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:17.065521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341110216706467:3810]
2025-10-09T21:44:17.065557Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:17.065560Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:17.065624Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 274 reason: error at write: Disconnected sessionId: src_id|4140ac5b-c83d2413-a908f054-ec9dbaac_0
2025-10-09T21:44:17.065825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 274 sessionId: src_id|4140ac5b-c83d2413-a908f054-ec9dbaac_0 is DEAD
2025-10-09T21:44:17.065968Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:17.066208Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4140ac5b-c83d2413-a908f054-ec9dbaac_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:17.066221Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4140ac5b-c83d2413-a908f054-ec9dbaac_0] Write session will restart in 0.006424s
2025-10-09T21:44:17.066247Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4140ac5b-c83d2413-a908f054-ec9dbaac_0] Write session: Do CDS request
2025-10-09T21:44:17.066255Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4140ac5b-c83d2413-a908f054-ec9dbaac_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:17.075202Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4140ac5b-c83d2413-a908f054-ec9dbaac_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:17.076673Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:17.076691Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 275
2025-10-09T21:44:17.078058Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 275 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:17.078111Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 275 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:17.078117Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=275 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:17.078121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:17.078938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 275 sessionId:  describe result for acl check
2025-10-09T21:44:17.078977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:17.078979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:17.078980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:17.078987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341114511673785:4359] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:17.079597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341114511673785:4359] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:17.083991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341114511673785:4359] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:17.092774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341114511673785:4359] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:17.092796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341114511673785:4359] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:17.092801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341114511673785:4359] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:17.092812Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 275 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:17.113651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:17.116865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:17.117127Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:17.117138Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:17.117152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:17.117219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:17.117264Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341114149863411:4052]
2025-10-09T21:44:17.117909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:17.122258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:17.122356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:17.122756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:17.122839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:17.122841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:17.122842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:17.122844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:17.122845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:17.122847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:17.122849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.122850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.122851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.122858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:17.122868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:17.122872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:17.122875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:17.122879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:17.122882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:17.122885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.122888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.122892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.122901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:17.122910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:17.122916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:17.122922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:17.122926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:17.122931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:17.122935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:17.122940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:17.122945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:17.122950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:17.122955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:17.122960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:17.122966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.122971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.122976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.122982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.122986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.122992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.123024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:17.133603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:17.134730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:17.134750Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:17.134754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:17.134759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:17.134771Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 274 [2:7559341114149863411:4052]
2025-10-09T21:44:17.134787Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:17.134807Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:17.134810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.134814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.134816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.134834Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:17.134840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:17.134848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:17.134915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.134918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.134921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.134923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:17.135003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:17.135007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.135023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.135034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.135050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:17.135073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341114149863406:4052]
2025-10-09T21:44:17.135075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341114149863406:4052]
2025-10-09T21:44:17.135078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341114149863406:4052]
2025-10-09T21:44:17.135092Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:17.135110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:17.135125Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:17.135128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:17.135174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341114149863406:4052]
2025-10-09T21:44:17.135176Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:17.135504Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:17.135517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:17.135519Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:17.135525Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:17.136305Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 274
2025-10-09T21:44:17.136337Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 274
2025-10-09T21:44:17.136797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341114511673824:4359], now have 1 active actors on pipe
2025-10-09T21:44:17.136809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341114511673776:2432], now have 1 active actors on pipe
2025-10-09T21:44:17.136825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:17.136831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:17.136864Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1d28924e-ee0fd706-c1b2888f-5a7875df_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:17.136874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.136880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.136883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:17.136904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.136920Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:17.136941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:17.137785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 275 partition: 0 MaxSeqNo: 25 sessionId: src_id|1d28924e-ee0fd706-c1b2888f-5a7875df_0
2025-10-09T21:44:17.138258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4140ac5b-c83d2413-a908f054-ec9dbaac_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1d28924e-ee0fd706-c1b2888f-5a7875df_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:17.140814Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1d28924e-ee0fd706-c1b2888f-5a7875df_0] Write session: try to update token
2025-10-09T21:44:17.140839Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1d28924e-ee0fd706-c1b2888f-5a7875df_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:17.137596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:17.137605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:17.137626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:17.165810Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341114511673878:3810]
2025-10-09T21:44:17.166597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341114511673878:3810], now have 1 active actors on pipe
2025-10-09T21:44:17.166616Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 274, pipe: [1:7559341114511673878:3810]
2025-10-09T21:44:17.173779Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:17.173800Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:17.173803Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:17.173806Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:17.173816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341114149863406:4052]
2025-10-09T21:44:17.173820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341114149863406:4052]
2025-10-09T21:44:17.173824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341114149863406:4052]
2025-10-09T21:44:17.173874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:17.173881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.173887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:17.174086Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:17.178198Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:17.178227Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:17.178232Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:17.200193Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 275 sessionId: src_id|1d28924e-ee0fd706-c1b2888f-5a7875df_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:17.202923Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:17.380475Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:17.381448Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:17.381501Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:17.381504Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:17.381564Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 275 reason: error at write: Disconnected sessionId: src_id|1d28924e-ee0fd706-c1b2888f-5a7875df_0
2025-10-09T21:44:17.381743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 275 sessionId: src_id|1d28924e-ee0fd706-c1b2888f-5a7875df_0 is DEAD
2025-10-09T21:44:17.381873Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341114511673878:3810]
2025-10-09T21:44:17.382570Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d28924e-ee0fd706-c1b2888f-5a7875df_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:17.382582Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d28924e-ee0fd706-c1b2888f-5a7875df_0] Write session will restart in 0.008840s
2025-10-09T21:44:17.382608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d28924e-ee0fd706-c1b2888f-5a7875df_0] Write session: Do CDS request
2025-10-09T21:44:17.382615Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d28924e-ee0fd706-c1b2888f-5a7875df_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:17.381052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:17.381072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:17.381456Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:17.392642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1d28924e-ee0fd706-c1b2888f-5a7875df_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:17.396689Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:17.396706Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 276
2025-10-09T21:44:17.397619Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 276 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:17.397666Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 276 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:17.397672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=276 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:17.397678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:17.398511Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 276 sessionId:  describe result for acl check
2025-10-09T21:44:17.398553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:17.398555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:17.398557Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:17.398564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341114511673897:4363] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:17.399173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341114511673897:4363] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:17.406581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341114511673897:4363] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:17.409842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:17.412837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:17.413171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341114511673897:4363] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:17.413189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341114511673897:4363] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:17.413193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341114511673897:4363] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:17.413204Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 276 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:17.413090Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:17.413100Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:17.413113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:17.413188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:17.413227Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341114149863509:4055]
2025-10-09T21:44:17.413840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:17.414437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:17.414489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:17.414828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:17.414881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:17.414882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:17.414883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:17.414884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:17.414885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:17.414886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:17.414887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.414888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.414890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.414895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:17.414905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:17.414910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:17.414914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:17.414918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:17.414923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:17.414927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.414931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.414935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.414942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:17.414950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:17.414956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:17.414963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:17.414967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:17.414988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:17.414992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:17.414998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:17.415002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:17.415007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:17.415012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:17.415017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:17.415023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.415028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.415032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.415038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.415042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.415047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.415077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:17.429692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:17.431292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:17.431308Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:17.431310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:17.431314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:17.431322Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 275 [2:7559341114149863509:4055]
2025-10-09T21:44:17.431341Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:17.431361Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:17.431367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.431371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.431373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.431392Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:17.431403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:17.431411Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:17.431481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.431483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.431484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.431487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:17.431563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:17.431566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.431577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.431587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.431604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:17.431633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341114149863504:4055]
2025-10-09T21:44:17.431635Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341114149863504:4055]
2025-10-09T21:44:17.431637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341114149863504:4055]
2025-10-09T21:44:17.431646Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:17.431661Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:17.431676Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:17.431679Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:17.431682Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:17.431686Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:17.431688Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:17.431726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341114149863504:4055]
2025-10-09T21:44:17.431743Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:17.431755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341114511673975:4363], now have 1 active actors on pipe
2025-10-09T21:44:17.431759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341114511673883:2432], now have 1 active actors on pipe
2025-10-09T21:44:17.432231Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 275
2025-10-09T21:44:17.432270Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 275
2025-10-09T21:44:17.431898Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:17.433253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:17.433277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:17.433313Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|990a74d2-66962188-7aa8f47b-f817964a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:17.433333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.433338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.433341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:17.433353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.433368Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:17.433387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:17.437015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:17.437030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:17.437066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:17.437224Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 276 partition: 0 MaxSeqNo: 25 sessionId: src_id|990a74d2-66962188-7aa8f47b-f817964a_0
2025-10-09T21:44:17.437617Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d28924e-ee0fd706-c1b2888f-5a7875df_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|990a74d2-66962188-7aa8f47b-f817964a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:17.440302Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|990a74d2-66962188-7aa8f47b-f817964a_0] Write session: try to update token
2025-10-09T21:44:17.440323Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|990a74d2-66962188-7aa8f47b-f817964a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:17.456018Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:17.456042Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:17.456046Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:17.456048Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:17.456059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341114149863504:4055]
2025-10-09T21:44:17.456066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341114149863504:4055]
2025-10-09T21:44:17.456071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341114149863504:4055]
2025-10-09T21:44:17.456128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:17.456136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.456139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:17.456256Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:17.456262Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:17.456267Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:17.456315Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:17.471042Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 276 sessionId: src_id|990a74d2-66962188-7aa8f47b-f817964a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:17.473669Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:17.480656Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341114511673989:3810]
2025-10-09T21:44:17.481064Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 275, pipe: [1:7559341114511673989:3810]
2025-10-09T21:44:17.656576Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:17.657052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:17.657062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:17.657088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341114511673989:3810], now have 1 active actors on pipe
2025-10-09T21:44:17.658111Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:17.658125Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:17.658176Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 276 reason: error at write: Disconnected sessionId: src_id|990a74d2-66962188-7aa8f47b-f817964a_0
2025-10-09T21:44:17.658224Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341114511673989:3810]
2025-10-09T21:44:17.658242Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:17.658374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 276 sessionId: src_id|990a74d2-66962188-7aa8f47b-f817964a_0 is DEAD
2025-10-09T21:44:17.658776Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|990a74d2-66962188-7aa8f47b-f817964a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:17.658790Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|990a74d2-66962188-7aa8f47b-f817964a_0] Write session will restart in 0.009067s
2025-10-09T21:44:17.658814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|990a74d2-66962188-7aa8f47b-f817964a_0] Write session: Do CDS request
2025-10-09T21:44:17.658823Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|990a74d2-66962188-7aa8f47b-f817964a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:17.659568Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:17.672896Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|990a74d2-66962188-7aa8f47b-f817964a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:17.673364Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:17.673382Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 277
2025-10-09T21:44:17.673565Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 277 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:17.673607Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 277 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:17.673613Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=277 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:17.673617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:17.674206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 277 sessionId:  describe result for acl check
2025-10-09T21:44:17.674248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:17.674255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:17.674257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:17.674263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341114511674077:4375] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:17.674845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341114511674077:4375] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:17.679171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:17.681787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:17.682050Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:17.682062Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:17.682076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:17.682142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:17.682189Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341114149863645:4065]
2025-10-09T21:44:17.682934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:17.686814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:17.686902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:17.687282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:17.687335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:17.687336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:17.687337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:17.687338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:17.687340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:17.687341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:17.687342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.687343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.687344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.687351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:17.687363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:17.687367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:17.687371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:17.687375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:17.687379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:17.687383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.687387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.687391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.687400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:17.687409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:17.687415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:17.687421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:17.687426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:17.687431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:17.687436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:17.687441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:17.687446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:17.687451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:17.687456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:17.687461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:17.687467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.687473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.687477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.687482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.687486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.687490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.687517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:17.689631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341114511674077:4375] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:17.701300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:17.702435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:17.702451Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:17.702454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:17.702459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:17.702468Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 276 [2:7559341114149863645:4065]
2025-10-09T21:44:17.702483Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:17.702503Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:17.702506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.702510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.702512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.702532Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:17.702538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:17.702545Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:17.702616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.702624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.702626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.702629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:17.702702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:17.702709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.702722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:17.702738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:17.702755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:17.702780Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341114149863640:4065]
2025-10-09T21:44:17.702783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341114149863640:4065]
2025-10-09T21:44:17.702786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341114149863640:4065]
2025-10-09T21:44:17.702800Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:17.702814Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:17.702828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:17.702832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:17.702871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341114149863640:4065]
2025-10-09T21:44:17.702874Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:17.706648Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341114511674077:4375] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:17.706669Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341114511674077:4375] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:17.706673Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341114511674077:4375] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:17.706684Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 277 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:17.707260Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:17.707274Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:17.707276Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:17.707284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:17.707451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341114511674017:2432], now have 1 active actors on pipe
2025-10-09T21:44:17.707564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341114511674112:4375], now have 1 active actors on pipe
2025-10-09T21:44:17.708244Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 276
2025-10-09T21:44:17.708281Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 276
2025-10-09T21:44:17.708738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:17.708755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:17.708801Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|71b247bd-7af46600-7d045213-cd764280_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:17.708814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.708820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.708824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:17.708846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.708866Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:17.708888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:17.714496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:17.714525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:17.714565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:17.714756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 277 partition: 0 MaxSeqNo: 25 sessionId: src_id|71b247bd-7af46600-7d045213-cd764280_0
2025-10-09T21:44:17.716668Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|990a74d2-66962188-7aa8f47b-f817964a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|71b247bd-7af46600-7d045213-cd764280_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:17.719464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|71b247bd-7af46600-7d045213-cd764280_0] Write session: try to update token
2025-10-09T21:44:17.719489Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|71b247bd-7af46600-7d045213-cd764280_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:17.727825Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:17.727851Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:17.727854Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:17.727857Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:17.727868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341114149863640:4065]
2025-10-09T21:44:17.727873Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341114149863640:4065]
2025-10-09T21:44:17.727878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341114149863640:4065]
2025-10-09T21:44:17.727937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:17.727945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:17.727948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:17.727949Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:17.727956Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:17.727961Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:17.728105Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:17.741643Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 277 sessionId: src_id|71b247bd-7af46600-7d045213-cd764280_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:17.744451Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:17.758687Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341114511674120:3810]
2025-10-09T21:44:17.760615Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 276, pipe: [1:7559341114511674120:3810]
2025-10-09T21:44:18.159358Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:18.160000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:18.160012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:18.160037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341114511674120:3810], now have 1 active actors on pipe
2025-10-09T21:44:18.161944Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71b247bd-7af46600-7d045213-cd764280_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:18.161959Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71b247bd-7af46600-7d045213-cd764280_0] Write session will restart in 0.007299s
2025-10-09T21:44:18.161984Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71b247bd-7af46600-7d045213-cd764280_0] Write session: Do CDS request
2025-10-09T21:44:18.161991Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71b247bd-7af46600-7d045213-cd764280_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:18.161356Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:18.161411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:18.161414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:18.161478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 277 reason: error at write: Disconnected sessionId: src_id|71b247bd-7af46600-7d045213-cd764280_0
2025-10-09T21:44:18.161735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 277 sessionId: src_id|71b247bd-7af46600-7d045213-cd764280_0 is DEAD
2025-10-09T21:44:18.161949Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341114511674120:3810]
2025-10-09T21:44:18.165911Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:18.174795Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:18.174813Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 278
2025-10-09T21:44:18.175757Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|71b247bd-7af46600-7d045213-cd764280_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:18.176236Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 278 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:18.176301Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 278 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:18.176308Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=278 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:18.176313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:18.177057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 278 sessionId:  describe result for acl check
2025-10-09T21:44:18.177124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:18.177126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:18.177129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:18.177135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341118806641445:4380] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:18.177751Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341118806641445:4380] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:18.182722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341118806641445:4380] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:18.189460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341118806641445:4380] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:18.189480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341118806641445:4380] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:18.189485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341118806641445:4380] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:18.189498Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 278 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:18.221318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:18.225182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:18.225474Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:18.225479Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:18.225493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:18.225561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:18.225607Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341118444831048:4073]
2025-10-09T21:44:18.226233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:18.226881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:18.226952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:18.227312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:18.227378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:18.227379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:18.227381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:18.227383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:18.227384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:18.227386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:18.227388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:18.227389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:18.227390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:18.227398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:18.227409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:18.227413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:18.227418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:18.227422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:18.227427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:18.227431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:18.227435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:18.227440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:18.227450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:18.227460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:18.227466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:18.227472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:18.227477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:18.227483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:18.227488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:18.227493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:18.227500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:18.227505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:18.227509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:18.227514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:18.227523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:18.227528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:18.227532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:18.227538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:18.227543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:18.227548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:18.227578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:18.249250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:18.250908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:18.250917Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:18.250920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:18.250925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:18.250934Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 277 [2:7559341118444831048:4073]
2025-10-09T21:44:18.250956Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:18.250980Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:18.250984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.250987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.250989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.251013Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:18.251020Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:18.251031Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:18.251104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:18.251106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:18.251108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:18.251110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:18.251196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:18.251199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:18.251214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:18.251224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:18.251238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:18.251260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341118444831042:4073]
2025-10-09T21:44:18.251262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341118444831042:4073]
2025-10-09T21:44:18.251264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341118444831042:4073]
2025-10-09T21:44:18.251280Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:18.251297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:18.251313Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:18.251316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:18.251367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341118444831042:4073]
2025-10-09T21:44:18.251370Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:18.251800Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:18.251810Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:18.251813Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:18.251820Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:18.252610Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 277
2025-10-09T21:44:18.252656Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 277
2025-10-09T21:44:18.259986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341118806641436:2432], now have 1 active actors on pipe
2025-10-09T21:44:18.260008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341118806641482:4380], now have 1 active actors on pipe
2025-10-09T21:44:18.260045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:18.260053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:18.260096Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f66bb426-a415eb6d-a4c1d742-18cc0bfd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:18.260112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.260118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.260125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:18.260144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.260164Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:18.260203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:18.263825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:18.263848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:18.263875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341118806641540:3810], now have 1 active actors on pipe
2025-10-09T21:44:18.263903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:18.264759Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71b247bd-7af46600-7d045213-cd764280_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f66bb426-a415eb6d-a4c1d742-18cc0bfd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:18.262840Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341118806641540:3810]
2025-10-09T21:44:18.264188Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 277, pipe: [1:7559341118806641540:3810]
2025-10-09T21:44:18.264341Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 278 partition: 0 MaxSeqNo: 25 sessionId: src_id|f66bb426-a415eb6d-a4c1d742-18cc0bfd_0
2025-10-09T21:44:18.267575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f66bb426-a415eb6d-a4c1d742-18cc0bfd_0] Write session: try to update token
2025-10-09T21:44:18.267596Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f66bb426-a415eb6d-a4c1d742-18cc0bfd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:18.299403Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:18.299424Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:18.299427Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:18.299429Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:18.299439Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341118444831042:4073]
2025-10-09T21:44:18.299443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341118444831042:4073]
2025-10-09T21:44:18.299447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341118444831042:4073]
2025-10-09T21:44:18.299497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:18.299505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:18.299508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:18.299679Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:18.302707Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:18.302721Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:18.302725Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:18.331485Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 278 sessionId: src_id|f66bb426-a415eb6d-a4c1d742-18cc0bfd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:18.334071Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:18.646650Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:18.646949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:18.646959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:18.648304Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f66bb426-a415eb6d-a4c1d742-18cc0bfd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:18.648325Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f66bb426-a415eb6d-a4c1d742-18cc0bfd_0] Write session will restart in 0.005778s
2025-10-09T21:44:18.648349Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f66bb426-a415eb6d-a4c1d742-18cc0bfd_0] Write session: Do CDS request
2025-10-09T21:44:18.648355Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f66bb426-a415eb6d-a4c1d742-18cc0bfd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:18.648452Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:18.647808Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341118806641540:3810]
2025-10-09T21:44:18.647841Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:18.647886Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:18.647889Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:18.647969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 278 reason: error at write: Disconnected sessionId: src_id|f66bb426-a415eb6d-a4c1d742-18cc0bfd_0
2025-10-09T21:44:18.648154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 278 sessionId: src_id|f66bb426-a415eb6d-a4c1d742-18cc0bfd_0 is DEAD
2025-10-09T21:44:18.657414Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f66bb426-a415eb6d-a4c1d742-18cc0bfd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:18.659666Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:18.659681Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 279
2025-10-09T21:44:18.659894Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 279 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:18.659921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 279 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:18.659928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=279 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:18.659931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:18.660841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 279 sessionId:  describe result for acl check
2025-10-09T21:44:18.660876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:18.660878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:18.660880Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:18.660885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341118806641558:4385] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:18.661427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341118806641558:4385] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:18.693377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341118806641558:4385] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:18.710639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:18.711353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:18.711560Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:18.711564Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:18.711578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:18.711637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:18.711676Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341118444831162:4075]
2025-10-09T21:44:18.712289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:18.717480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341118806641558:4385] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:18.717502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341118806641558:4385] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:18.717506Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341118806641558:4385] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:18.717518Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 279 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:18.721110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:18.721221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:18.721622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:18.721680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:18.721681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:18.721683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:18.721684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:18.721685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:18.721687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:18.721688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:18.721689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:18.721690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:18.721701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:18.721711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:18.721715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:18.721719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:18.721723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:18.721727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:18.721731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:18.721735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:18.721739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:18.721747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:18.721755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:18.721763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:18.721770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:18.721775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:18.721780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:18.721783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:18.721789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:18.721793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:18.721798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:18.721802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:18.721808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:18.721812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:18.721817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:18.721821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:18.721826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:18.721831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:18.721836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:18.721867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:18.747893Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341118806641664:3810]
2025-10-09T21:44:18.748828Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 278, pipe: [1:7559341118806641664:3810]
2025-10-09T21:44:18.748863Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 278
2025-10-09T21:44:18.748880Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 278
2025-10-09T21:44:18.746828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:18.748366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:18.748373Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:18.748375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:18.748382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:18.748390Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 278 [2:7559341118444831162:4075]
2025-10-09T21:44:18.748403Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:18.748423Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:18.748426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.748429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.748432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.748453Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:18.748458Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:18.748465Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:18.748525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:18.748527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:18.748529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:18.748532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:18.748611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:18.748614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:18.748626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:18.748636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:18.748652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:18.748673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341118444831156:4075]
2025-10-09T21:44:18.748675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341118444831156:4075]
2025-10-09T21:44:18.748677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341118444831156:4075]
2025-10-09T21:44:18.748686Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:18.748696Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:18.748700Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:18.748704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341118806641664:3810], now have 1 active actors on pipe
2025-10-09T21:44:18.748705Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:18.748709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341118806641660:4385], now have 1 active actors on pipe
2025-10-09T21:44:18.748717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:18.748730Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:18.748733Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:18.748739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341118806641549:2432], now have 1 active actors on pipe
2025-10-09T21:44:18.748793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341118444831156:4075]
2025-10-09T21:44:18.748800Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:18.749365Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:18.752569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:18.752588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:18.752632Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bc9d8e41-897d209c-cd4a7e88-6397a40f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:18.752645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.752650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.752652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:18.752666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.752683Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:18.752705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:18.756964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:18.756987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:18.757030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:18.757254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 279 partition: 0 MaxSeqNo: 25 sessionId: src_id|bc9d8e41-897d209c-cd4a7e88-6397a40f_0
2025-10-09T21:44:18.757643Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f66bb426-a415eb6d-a4c1d742-18cc0bfd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bc9d8e41-897d209c-cd4a7e88-6397a40f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:18.760315Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc9d8e41-897d209c-cd4a7e88-6397a40f_0] Write session: try to update token
2025-10-09T21:44:18.760339Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc9d8e41-897d209c-cd4a7e88-6397a40f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:18.784462Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 279 sessionId: src_id|bc9d8e41-897d209c-cd4a7e88-6397a40f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:18.786891Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:18.787263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:18.787287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:18.787327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:18.787687Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:18.790181Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:18.790201Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:18.790204Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:18.790206Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:18.790216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341118444831156:4075]
2025-10-09T21:44:18.790222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341118444831156:4075]
2025-10-09T21:44:18.790226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341118444831156:4075]
2025-10-09T21:44:18.790275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:18.790282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:18.790285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:18.790527Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:18.793554Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:18.793569Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:18.793572Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:19.070779Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:19.094328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:19.094363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:19.094562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:19.094629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:19.094690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:19.094752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:19.094804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:19.094810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:19.094876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:19.094930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:19.094989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:19.095051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:19.095091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:19.095094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:19.095162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:19.095217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:19.095302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:19.095361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:19.095406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:19.095413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:19.095485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:19.095542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:19.095600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:19.095683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:19.095726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:19.095729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:19.095797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:19.095855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:19.095958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:19.096053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:19.096094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:19.096097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:19.096185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:19.096236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:19.096294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:19.096397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:19.096440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:19.096443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:19.111426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:19.111552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:19.111610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:19.111667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:19.111730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:19.111737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:19.111827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:19.111881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:19.111944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:19.112003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:19.112061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:19.112065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:19.112140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:19.112199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:19.112265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:19.112326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:19.112375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:19.112378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:19.112443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:19.112497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:19.112555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:19.112617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:19.112656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:19.112658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:19.115079Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:19.115718Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc9d8e41-897d209c-cd4a7e88-6397a40f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:19.115733Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc9d8e41-897d209c-cd4a7e88-6397a40f_0] Write session will restart in 0.009125s
2025-10-09T21:44:19.115760Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc9d8e41-897d209c-cd4a7e88-6397a40f_0] Write session: Do CDS request
2025-10-09T21:44:19.115766Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc9d8e41-897d209c-cd4a7e88-6397a40f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:19.115158Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341118806641664:3810]
2025-10-09T21:44:19.115208Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:19.115253Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:19.115255Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:19.115311Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 279 reason: error at write: Disconnected sessionId: src_id|bc9d8e41-897d209c-cd4a7e88-6397a40f_0
2025-10-09T21:44:19.115531Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 279 sessionId: src_id|bc9d8e41-897d209c-cd4a7e88-6397a40f_0 is DEAD
2025-10-09T21:44:19.126643Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc9d8e41-897d209c-cd4a7e88-6397a40f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:19.130350Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:19.130383Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 280
2025-10-09T21:44:19.130570Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 280 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:19.130599Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 280 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:19.130605Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=280 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:19.130610Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:19.131051Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 280 sessionId:  describe result for acl check
2025-10-09T21:44:19.131085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:19.131087Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:19.131089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:19.131096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341123101609006:4397] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:19.132019Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341123101609006:4397] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:19.141427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341123101609006:4397] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:19.147524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:19.150468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341123101609006:4397] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:19.150489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341123101609006:4397] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:19.150493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341123101609006:4397] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:19.150505Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 280 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:19.150357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:19.150674Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:19.150679Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:19.150691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:19.150750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:19.150789Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341122739798586:4090]
2025-10-09T21:44:19.151417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:19.151848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:19.151891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:19.152255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:19.152301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:19.152312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:19.152314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:19.152315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:19.152317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:19.152318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:19.152320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.152321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.152323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.152328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:19.152340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:19.152345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:19.152350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:19.152355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:19.152360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:19.152364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.152369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.152373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.152380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:19.152390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:19.152397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:19.152403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:19.152408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:19.152413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:19.152418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:19.152423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:19.152429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:19.152435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:19.152440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:19.152446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:19.152451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.152457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.152462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.152468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.152472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.152478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.152510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:19.169341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:19.170566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:19.170574Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:19.170577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:19.170583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:19.170593Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 279 [2:7559341122739798586:4090]
2025-10-09T21:44:19.170611Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:19.170635Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:19.170639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.170643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.170646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.170665Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:19.170676Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:19.170684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:19.170759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.170761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.170763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.170766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:19.170850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:19.170854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.170865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.170876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.170892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:19.170915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341122739798581:4090]
2025-10-09T21:44:19.170918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341122739798581:4090]
2025-10-09T21:44:19.170920Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341122739798581:4090]
2025-10-09T21:44:19.170937Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:19.170957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:19.170972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:19.170976Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:19.171021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341122739798581:4090]
2025-10-09T21:44:19.171024Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:19.171532Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:19.171544Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:19.171546Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:19.171554Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:19.172157Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 279
2025-10-09T21:44:19.172197Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 279
2025-10-09T21:44:19.172629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341123101609088:4397], now have 1 active actors on pipe
2025-10-09T21:44:19.172643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341123101608991:2432], now have 1 active actors on pipe
2025-10-09T21:44:19.172662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:19.172681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:19.172719Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9bb504b0-141e78e1-44f61b9d-573ed896_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:19.172732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.172737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.172741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:19.172753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.172769Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:19.172791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:19.174023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:19.174050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:19.174072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:19.174277Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 280 partition: 0 MaxSeqNo: 25 sessionId: src_id|9bb504b0-141e78e1-44f61b9d-573ed896_0
2025-10-09T21:44:19.174721Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc9d8e41-897d209c-cd4a7e88-6397a40f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9bb504b0-141e78e1-44f61b9d-573ed896_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:19.177366Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9bb504b0-141e78e1-44f61b9d-573ed896_0] Write session: try to update token
2025-10-09T21:44:19.177383Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9bb504b0-141e78e1-44f61b9d-573ed896_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:19.194644Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:19.194669Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:19.194672Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:19.194675Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:19.194685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341122739798581:4090]
2025-10-09T21:44:19.194693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341122739798581:4090]
2025-10-09T21:44:19.194697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341122739798581:4090]
2025-10-09T21:44:19.194747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:19.194761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.194765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:19.194907Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:19.195439Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:19.195454Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:19.195458Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:19.203173Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 280 sessionId: src_id|9bb504b0-141e78e1-44f61b9d-573ed896_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:19.205758Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:19.215390Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341123101609100:3810]
2025-10-09T21:44:19.216085Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 279, pipe: [1:7559341123101609100:3810]
2025-10-09T21:44:19.359034Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:19.359793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:19.359816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:19.359867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341123101609100:3810], now have 1 active actors on pipe
2025-10-09T21:44:19.360377Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:19.360271Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:19.360288Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:19.360368Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 280 reason: error at write: Disconnected sessionId: src_id|9bb504b0-141e78e1-44f61b9d-573ed896_0
2025-10-09T21:44:19.360408Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:19.360781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9bb504b0-141e78e1-44f61b9d-573ed896_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:19.360790Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9bb504b0-141e78e1-44f61b9d-573ed896_0] Write session will restart in 0.007072s
2025-10-09T21:44:19.360456Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341123101609100:3810]
2025-10-09T21:44:19.360577Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 280 sessionId: src_id|9bb504b0-141e78e1-44f61b9d-573ed896_0 is DEAD
2025-10-09T21:44:19.360813Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9bb504b0-141e78e1-44f61b9d-573ed896_0] Write session: Do CDS request
2025-10-09T21:44:19.360821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9bb504b0-141e78e1-44f61b9d-573ed896_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:19.369735Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9bb504b0-141e78e1-44f61b9d-573ed896_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:19.369818Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:19.369837Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 281
2025-10-09T21:44:19.370061Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 281 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:19.370120Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 281 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:19.370127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=281 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:19.370131Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:19.370614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 281 sessionId:  describe result for acl check
2025-10-09T21:44:19.370654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:19.370663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:19.370665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:19.370671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341123101609111:4401] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:19.371247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341123101609111:4401] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:19.377697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341123101609111:4401] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:19.387229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341123101609111:4401] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:19.387249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341123101609111:4401] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:19.387254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341123101609111:4401] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:19.387265Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 281 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:19.391249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:19.394022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:19.394295Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:19.394306Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:19.394319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:19.394384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:19.394453Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341122739798689:4093]
2025-10-09T21:44:19.395200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:19.395663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:19.395744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:19.396077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:19.396147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:19.396154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:19.396156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:19.396157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:19.396158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:19.396160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:19.396161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.396163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.396164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.396168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:19.396186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:19.396190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:19.396194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:19.396198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:19.396202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:19.396205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.396208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.396211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.396219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:19.396228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:19.396235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:19.396241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:19.396245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:19.396250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:19.396255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:19.396260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:19.396265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:19.396271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:19.396275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:19.396292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:19.396297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.396303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.396307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.396319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.396324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.396330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.396358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:19.405463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:19.406837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:19.406853Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:19.406856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:19.406861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:19.406868Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 280 [2:7559341122739798689:4093]
2025-10-09T21:44:19.406883Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:19.406917Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:19.406921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.406925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.406929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.406948Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:19.406955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:19.406967Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:19.407037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.407044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.407046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.407049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:19.407126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:19.407134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.407146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.407161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.407179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:19.407202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341122739798683:4093]
2025-10-09T21:44:19.407209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341122739798683:4093]
2025-10-09T21:44:19.407211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341122739798683:4093]
2025-10-09T21:44:19.407220Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:19.407237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:19.407251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:19.407253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:19.407277Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:19.407279Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:19.407281Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:19.407296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341122739798683:4093]
2025-10-09T21:44:19.407298Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:19.407301Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:19.407309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341123101609199:4401], now have 1 active actors on pipe
2025-10-09T21:44:19.407312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341123101609103:2432], now have 1 active actors on pipe
2025-10-09T21:44:19.408108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:19.408122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:19.408157Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4af990d6-3ff784c5-3b09c9f7-1e0f592c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:19.408172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.408180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.408183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:19.408194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.408213Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:19.407430Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 280
2025-10-09T21:44:19.407430Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 280
2025-10-09T21:44:19.408237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:19.408732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:19.408743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:19.408763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:19.408886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 281 partition: 0 MaxSeqNo: 25 sessionId: src_id|4af990d6-3ff784c5-3b09c9f7-1e0f592c_0
2025-10-09T21:44:19.409401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9bb504b0-141e78e1-44f61b9d-573ed896_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4af990d6-3ff784c5-3b09c9f7-1e0f592c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:19.412032Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4af990d6-3ff784c5-3b09c9f7-1e0f592c_0] Write session: try to update token
2025-10-09T21:44:19.412054Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4af990d6-3ff784c5-3b09c9f7-1e0f592c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:19.426854Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:19.426874Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:19.426877Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:19.426879Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:19.426890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341122739798683:4093]
2025-10-09T21:44:19.426894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341122739798683:4093]
2025-10-09T21:44:19.426898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341122739798683:4093]
2025-10-09T21:44:19.426945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:19.426951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.426954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:19.426960Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:19.426965Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:19.426969Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:19.427105Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:19.433308Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 281 sessionId: src_id|4af990d6-3ff784c5-3b09c9f7-1e0f592c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:19.436040Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:19.458582Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341123101609212:3810]
2025-10-09T21:44:19.459368Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 280, pipe: [1:7559341123101609212:3810]
2025-10-09T21:44:19.591486Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:19.591790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:19.591801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:19.591851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341123101609212:3810], now have 1 active actors on pipe
2025-10-09T21:44:19.592110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:19.592129Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:19.592147Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:19.592848Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4af990d6-3ff784c5-3b09c9f7-1e0f592c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:19.592861Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4af990d6-3ff784c5-3b09c9f7-1e0f592c_0] Write session will restart in 0.008922s
2025-10-09T21:44:19.592887Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4af990d6-3ff784c5-3b09c9f7-1e0f592c_0] Write session: Do CDS request
2025-10-09T21:44:19.592195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 281 reason: error at write: Disconnected sessionId: src_id|4af990d6-3ff784c5-3b09c9f7-1e0f592c_0
2025-10-09T21:44:19.592201Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341123101609212:3810]
2025-10-09T21:44:19.592221Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:19.592421Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 281 sessionId: src_id|4af990d6-3ff784c5-3b09c9f7-1e0f592c_0 is DEAD
2025-10-09T21:44:19.592894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4af990d6-3ff784c5-3b09c9f7-1e0f592c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:19.610687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4af990d6-3ff784c5-3b09c9f7-1e0f592c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:19.610655Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:19.610673Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 282
2025-10-09T21:44:19.610913Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 282 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:19.610953Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 282 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:19.610959Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=282 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:19.610963Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:19.611331Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 282 sessionId:  describe result for acl check
2025-10-09T21:44:19.611365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:19.611367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:19.611369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:19.611375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341123101609280:4406] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:19.611928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341123101609280:4406] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:19.612691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:19.614686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:19.614889Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:19.614904Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:19.614917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:19.614975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:19.615019Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341122739798790:4095]
2025-10-09T21:44:19.615609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:19.616703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:19.616763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:19.617092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:19.617167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:19.617168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:19.617169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:19.617170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:19.617171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:19.617172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:19.617174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.617175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.617176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.617182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:19.617192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:19.617196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:19.617201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:19.617204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:19.617209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:19.617213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.617217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.617221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.617229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:19.617237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:19.617243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:19.617248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:19.617252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:19.617256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:19.617261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:19.617264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:19.617268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:19.617273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:19.617278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:19.617283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:19.617288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.617293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.617296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.617300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.617305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.617309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.617337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:19.615662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341123101609280:4406] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:19.625702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341123101609280:4406] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:19.625725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341123101609280:4406] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:19.625730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341123101609280:4406] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:19.625741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 282 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:19.632318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:19.633778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:19.633807Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:19.633810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:19.633814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:19.633823Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 281 [2:7559341122739798790:4095]
2025-10-09T21:44:19.633838Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:19.633859Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:19.633862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.633866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.633868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.633895Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:19.633912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:19.633920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:19.634004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.634012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.634014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.634017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:19.634123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:19.634134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.634148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.634160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.634175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:19.634200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341123101609319:4406], now have 1 active actors on pipe
2025-10-09T21:44:19.634203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341123101609217:2432], now have 1 active actors on pipe
2025-10-09T21:44:19.634213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341122739798785:4095]
2025-10-09T21:44:19.634214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341122739798785:4095]
2025-10-09T21:44:19.634216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341122739798785:4095]
2025-10-09T21:44:19.634230Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:19.634245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:19.634260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:19.634263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:19.634307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341122739798785:4095]
2025-10-09T21:44:19.634309Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:19.634335Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:19.634339Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:19.634341Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:19.634348Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:19.635119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:19.635131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:19.635163Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6ee98a90-3f276ae4-71f32ca1-8fa1c111_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:19.635176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.635180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.635194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:19.635207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.635221Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:19.635239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:19.635810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:19.635821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:19.635840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:19.634604Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 281
2025-10-09T21:44:19.634604Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 281
2025-10-09T21:44:19.636455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 282 partition: 0 MaxSeqNo: 25 sessionId: src_id|6ee98a90-3f276ae4-71f32ca1-8fa1c111_0
2025-10-09T21:44:19.638315Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4af990d6-3ff784c5-3b09c9f7-1e0f592c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6ee98a90-3f276ae4-71f32ca1-8fa1c111_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:19.640843Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ee98a90-3f276ae4-71f32ca1-8fa1c111_0] Write session: try to update token
2025-10-09T21:44:19.640864Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ee98a90-3f276ae4-71f32ca1-8fa1c111_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:19.651708Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:19.651731Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:19.651733Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:19.651735Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:19.651744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341122739798785:4095]
2025-10-09T21:44:19.651748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341122739798785:4095]
2025-10-09T21:44:19.651752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341122739798785:4095]
2025-10-09T21:44:19.651797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:19.651805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.651807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:19.651816Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:19.651822Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:19.651825Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:19.651954Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:19.659606Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 282 sessionId: src_id|6ee98a90-3f276ae4-71f32ca1-8fa1c111_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:19.662388Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:19.691639Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341123101609328:3810]
2025-10-09T21:44:19.692984Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 281, pipe: [1:7559341123101609328:3810]
2025-10-09T21:44:19.869105Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:19.869675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:19.869685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:19.869727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341123101609328:3810], now have 1 active actors on pipe
2025-10-09T21:44:19.871088Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341123101609328:3810]
2025-10-09T21:44:19.871126Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:19.871129Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:19.871189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 282 reason: error at write: Disconnected sessionId: src_id|6ee98a90-3f276ae4-71f32ca1-8fa1c111_0
2025-10-09T21:44:19.871379Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 282 sessionId: src_id|6ee98a90-3f276ae4-71f32ca1-8fa1c111_0 is DEAD
2025-10-09T21:44:19.871517Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:19.871949Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ee98a90-3f276ae4-71f32ca1-8fa1c111_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:19.871961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ee98a90-3f276ae4-71f32ca1-8fa1c111_0] Write session will restart in 0.009011s
2025-10-09T21:44:19.871985Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ee98a90-3f276ae4-71f32ca1-8fa1c111_0] Write session: Do CDS request
2025-10-09T21:44:19.871995Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ee98a90-3f276ae4-71f32ca1-8fa1c111_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:19.879370Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:19.882709Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ee98a90-3f276ae4-71f32ca1-8fa1c111_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:19.883101Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:19.883119Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 283
2025-10-09T21:44:19.883383Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 283 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:19.883417Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 283 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:19.883422Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=283 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:19.883436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:19.884227Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 283 sessionId:  describe result for acl check
2025-10-09T21:44:19.884270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:19.884277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:19.884279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:19.884285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341123101609363:4416] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:19.884875Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341123101609363:4416] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:19.895363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341123101609363:4416] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:19.900304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:19.904134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341123101609363:4416] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:19.904154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341123101609363:4416] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:19.904158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341123101609363:4416] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:19.904171Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 283 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:19.903952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:19.904188Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:19.904193Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:19.904207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:19.904267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:19.904304Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341122739798923:4103]
2025-10-09T21:44:19.904927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:19.906162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:19.906222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:19.906575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:19.906625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:19.906626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:19.906627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:19.906628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:19.906630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:19.906631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:19.906632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.906633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.906634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.906641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:19.906652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:19.906656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:19.906660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:19.906664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:19.906668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:19.906672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.906676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.906680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.906688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:19.906697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:19.906703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:19.906708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:19.906713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:19.906718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:19.906723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:19.906728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:19.906733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:19.906738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:19.906741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:19.906745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:19.906750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.906754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.906758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.906762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.906766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.906771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.906800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:19.931341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:19.932722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:19.932744Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:19.932747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:19.932752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:19.932759Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 282 [2:7559341122739798923:4103]
2025-10-09T21:44:19.932774Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:19.933499Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 282
2025-10-09T21:44:19.932795Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:19.932799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.933535Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 282
2025-10-09T21:44:19.932802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.932804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.932826Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:19.932833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:19.932841Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:19.932910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.932912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.932914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.932917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:19.933001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:19.933004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.933018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:19.933031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:19.933049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:19.933070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341122739798907:4103]
2025-10-09T21:44:19.933072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341122739798907:4103]
2025-10-09T21:44:19.933074Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341122739798907:4103]
2025-10-09T21:44:19.933083Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:19.933114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:19.933121Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:19.933127Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:19.933129Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:19.933130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:19.933133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:19.933187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341122739798907:4103]
2025-10-09T21:44:19.933195Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:19.933207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341123101609449:4416], now have 1 active actors on pipe
2025-10-09T21:44:19.933222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341123101609348:2432], now have 1 active actors on pipe
2025-10-09T21:44:19.933637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:19.935832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:19.935862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:19.935909Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|88dfa375-2f2199ec-53513c78-8dd0b794_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:19.935927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.935932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.935936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:19.935950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.935989Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:19.936013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:19.936437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:19.936449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:19.936474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:19.936645Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 283 partition: 0 MaxSeqNo: 25 sessionId: src_id|88dfa375-2f2199ec-53513c78-8dd0b794_0
2025-10-09T21:44:19.937109Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ee98a90-3f276ae4-71f32ca1-8fa1c111_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|88dfa375-2f2199ec-53513c78-8dd0b794_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:19.939787Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88dfa375-2f2199ec-53513c78-8dd0b794_0] Write session: try to update token
2025-10-09T21:44:19.939813Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88dfa375-2f2199ec-53513c78-8dd0b794_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:19.966145Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 283 sessionId: src_id|88dfa375-2f2199ec-53513c78-8dd0b794_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:19.968870Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:19.969182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:19.969206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:19.969248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:19.969745Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:19.973117Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341123101609471:3810]
2025-10-09T21:44:19.973559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341123101609471:3810], now have 1 active actors on pipe
2025-10-09T21:44:19.973687Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 282, pipe: [1:7559341123101609471:3810]
2025-10-09T21:44:19.976872Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:19.976892Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:19.976895Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:19.976897Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:19.976907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341122739798907:4103]
2025-10-09T21:44:19.976912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341122739798907:4103]
2025-10-09T21:44:19.976916Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341122739798907:4103]
2025-10-09T21:44:19.976964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:19.976971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:19.976974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:19.977114Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:19.982072Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:19.982095Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:19.982100Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:20.222876Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:20.230961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:20.230990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:20.231159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:20.231212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:20.231274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:20.231329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:20.231368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:20.231372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:20.231438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:20.231490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:20.231550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:20.231616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:20.231656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:20.231659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:20.231724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:20.231783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:20.231844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:20.231905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:20.231945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:20.231948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:20.232020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:20.232083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:20.232147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:20.232205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:20.232245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:20.232247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:20.232313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:20.232369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:20.234716Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:20.234733Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:20.234770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 283 reason: error at write: Disconnected sessionId: src_id|88dfa375-2f2199ec-53513c78-8dd0b794_0
2025-10-09T21:44:20.234966Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 283 sessionId: src_id|88dfa375-2f2199ec-53513c78-8dd0b794_0 is DEAD
2025-10-09T21:44:20.235124Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341123101609471:3810]
2025-10-09T21:44:20.235140Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:20.232430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:20.232492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:20.232532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:20.232535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:20.232603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:20.232656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:20.232713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:20.232771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:20.232815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:20.232819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:20.232897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:20.232951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:20.233012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:20.233070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:20.233108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:20.233114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:20.233178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:20.233231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:20.233286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:20.233341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:20.233381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:20.233384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:20.233448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:20.233502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:20.233560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:20.233618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:20.233655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:20.233658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:20.233721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:20.233778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:20.233834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:20.233891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:20.233931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:20.233934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:20.234696Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:20.235425Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88dfa375-2f2199ec-53513c78-8dd0b794_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:20.235437Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88dfa375-2f2199ec-53513c78-8dd0b794_0] Write session will restart in 0.008399s
2025-10-09T21:44:20.235463Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88dfa375-2f2199ec-53513c78-8dd0b794_0] Write session: Do CDS request
2025-10-09T21:44:20.235486Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88dfa375-2f2199ec-53513c78-8dd0b794_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:20.245578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88dfa375-2f2199ec-53513c78-8dd0b794_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:20.245704Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:20.245725Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 284
2025-10-09T21:44:20.246020Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 284 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:20.246078Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 284 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:20.246091Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=284 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:20.246105Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:20.246770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 284 sessionId:  describe result for acl check
2025-10-09T21:44:20.246808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:20.246811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:20.246813Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:20.246822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341127396576837:4422] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:20.247437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341127396576837:4422] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:20.249485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:20.251636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:20.251887Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:20.251892Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:20.251904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:20.251958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:20.251999Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341127034766338:4113]
2025-10-09T21:44:20.252653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:20.253233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:20.253294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:20.253638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:20.253938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:20.253949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:20.253950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:20.253951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:20.253952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:20.253953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:20.253954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.253955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.253957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.253963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:20.253973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:20.253977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:20.253980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:20.253984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:20.253987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:20.253991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.253994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.253998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.254006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:20.254025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:20.254048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:20.254053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:20.254057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:20.254062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:20.254067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:20.254072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:20.254076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:20.254089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:20.254093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:20.254098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:20.254102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.254106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.254110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.254116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.254120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.254125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.254157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:20.255300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341127396576837:4422] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:20.262107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341127396576837:4422] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:20.262133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341127396576837:4422] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:20.262137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341127396576837:4422] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:20.262151Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 284 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:20.263964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:20.265169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:20.265189Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:20.265192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:20.265197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:20.265207Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 283 [2:7559341127034766338:4113]
2025-10-09T21:44:20.265224Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:20.265246Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:20.265250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.265253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.265257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.265279Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:20.265292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:20.265300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:20.265375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.265386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.265388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.265392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:20.265472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:20.265480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.265494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.265508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.265529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:20.265551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341127034766333:4113]
2025-10-09T21:44:20.265556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341127034766333:4113]
2025-10-09T21:44:20.265559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341127034766333:4113]
2025-10-09T21:44:20.265568Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:20.265591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341127396576879:4422], now have 1 active actors on pipe
2025-10-09T21:44:20.265590Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:20.265594Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:20.265599Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:20.265602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:20.265618Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:20.265627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:20.265634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341127396576774:2432], now have 1 active actors on pipe
2025-10-09T21:44:20.265681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341127034766333:4113]
2025-10-09T21:44:20.265695Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:20.265717Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 283
2025-10-09T21:44:20.265720Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 283
2025-10-09T21:44:20.265729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:20.266057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:20.266070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:20.266103Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5992bd8c-70e87622-a9212cb9-e499f7a8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:20.266116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.266121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.266124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:20.266136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.266152Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:20.266175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:20.266699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:20.266710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:20.266729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:20.266878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 284 partition: 0 MaxSeqNo: 25 sessionId: src_id|5992bd8c-70e87622-a9212cb9-e499f7a8_0
2025-10-09T21:44:20.267308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88dfa375-2f2199ec-53513c78-8dd0b794_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5992bd8c-70e87622-a9212cb9-e499f7a8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:20.270187Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5992bd8c-70e87622-a9212cb9-e499f7a8_0] Write session: try to update token
2025-10-09T21:44:20.270207Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5992bd8c-70e87622-a9212cb9-e499f7a8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:20.288797Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:20.288815Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:20.288818Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:20.288820Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:20.288830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341127034766333:4113]
2025-10-09T21:44:20.288835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341127034766333:4113]
2025-10-09T21:44:20.288839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341127034766333:4113]
2025-10-09T21:44:20.288892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:20.288892Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:20.288898Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:20.288902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.288903Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:20.288904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:20.289061Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:20.287445Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 284 sessionId: src_id|5992bd8c-70e87622-a9212cb9-e499f7a8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:20.290192Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:20.335179Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341127396576886:3810]
2025-10-09T21:44:20.335645Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 283, pipe: [1:7559341127396576886:3810]
2025-10-09T21:44:20.459249Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:20.459753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:20.459767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:20.459816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341127396576886:3810], now have 1 active actors on pipe
2025-10-09T21:44:20.460382Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:20.462229Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:20.462312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 284 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5992bd8c-70e87622-a9212cb9-e499f7a8_0
2025-10-09T21:44:20.462512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 284 sessionId: src_id|5992bd8c-70e87622-a9212cb9-e499f7a8_0 is DEAD
2025-10-09T21:44:20.462761Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:20.462794Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341127396576886:3810]
2025-10-09T21:44:20.466191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5992bd8c-70e87622-a9212cb9-e499f7a8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:20.466207Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5992bd8c-70e87622-a9212cb9-e499f7a8_0] Write session will restart in 0.008909s
2025-10-09T21:44:20.466234Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5992bd8c-70e87622-a9212cb9-e499f7a8_0] Write session: Do CDS request
2025-10-09T21:44:20.466242Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5992bd8c-70e87622-a9212cb9-e499f7a8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:20.476629Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5992bd8c-70e87622-a9212cb9-e499f7a8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:20.476876Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:20.476911Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 285
2025-10-09T21:44:20.477090Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 285 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:20.477124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 285 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:20.477130Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=285 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:20.477134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:20.477632Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 285 sessionId:  describe result for acl check
2025-10-09T21:44:20.477664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:20.477666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:20.477668Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:20.477673Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341127396576905:4427] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:20.478329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341127396576905:4427] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:20.484479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341127396576905:4427] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:20.489920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341127396576905:4427] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:20.489937Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341127396576905:4427] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:20.489941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341127396576905:4427] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:20.489956Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 285 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:20.492039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:20.492828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:20.493013Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:20.493017Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:20.493028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:20.493081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:20.493121Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341127034766445:4115]
2025-10-09T21:44:20.493726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:20.494241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:20.494286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:20.494656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:20.494712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:20.494714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:20.494715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:20.494717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:20.494718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:20.494720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:20.494721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.494722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.494723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.494730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:20.494741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:20.494746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:20.494750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:20.494754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:20.494759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:20.494763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.494766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.494770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.494779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:20.494787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:20.494794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:20.494799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:20.494804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:20.494810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:20.494815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:20.494820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:20.494825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:20.494831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:20.494852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:20.494859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:20.494865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.494870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.494875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.494881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.494886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.494891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.494922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:20.502779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:20.504184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:20.504190Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:20.504192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:20.504197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:20.504205Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 284 [2:7559341127034766445:4115]
2025-10-09T21:44:20.504221Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:20.504241Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:20.504244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.504248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.504250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.504270Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:20.504277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:20.504284Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:20.504344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.504348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.504352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.504355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:20.504430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:20.504432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.504444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.504453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.504467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:20.504488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341127034766440:4115]
2025-10-09T21:44:20.504491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341127034766440:4115]
2025-10-09T21:44:20.504492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341127034766440:4115]
2025-10-09T21:44:20.504501Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:20.504517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:20.504530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:20.504532Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:20.504571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341127034766440:4115]
2025-10-09T21:44:20.504573Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:20.505488Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 284
2025-10-09T21:44:20.505517Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 284
2025-10-09T21:44:20.504827Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:20.504835Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:20.504837Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:20.504843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:20.509305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341127396576991:4427], now have 1 active actors on pipe
2025-10-09T21:44:20.509322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341127396576889:2432], now have 1 active actors on pipe
2025-10-09T21:44:20.509345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:20.509353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:20.509398Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a816fd16-c87aa61-47adaec6-30140183_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:20.509410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.509417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.509420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:20.509435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.509452Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:20.509477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:20.509858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:20.509869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:20.509891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:20.510068Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 285 partition: 0 MaxSeqNo: 25 sessionId: src_id|a816fd16-c87aa61-47adaec6-30140183_0
2025-10-09T21:44:20.514151Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5992bd8c-70e87622-a9212cb9-e499f7a8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a816fd16-c87aa61-47adaec6-30140183_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:20.516833Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a816fd16-c87aa61-47adaec6-30140183_0] Write session: try to update token
2025-10-09T21:44:20.516855Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a816fd16-c87aa61-47adaec6-30140183_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:20.524143Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:20.524163Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:20.524166Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:20.524171Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:20.524180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341127034766440:4115]
2025-10-09T21:44:20.524183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341127034766440:4115]
2025-10-09T21:44:20.524186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341127034766440:4115]
2025-10-09T21:44:20.524228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:20.524236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.524239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:20.524367Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:20.526255Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:20.526270Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:20.526276Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:20.554691Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 285 sessionId: src_id|a816fd16-c87aa61-47adaec6-30140183_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:20.557204Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:20.566150Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341127396577005:3810]
2025-10-09T21:44:20.566488Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 284, pipe: [1:7559341127396577005:3810]
2025-10-09T21:44:20.872096Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:44:20.872121Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 3
2025-10-09T21:44:20.905358Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:20.905814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:20.905823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:20.905847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341127396577005:3810], now have 1 active actors on pipe
2025-10-09T21:44:20.907352Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:20.907687Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341127396577005:3810]
2025-10-09T21:44:20.907728Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:20.907728Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:20.907731Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:20.907742Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 3
2025-10-09T21:44:20.907786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 285 reason: error at write: Disconnected sessionId: src_id|a816fd16-c87aa61-47adaec6-30140183_0
2025-10-09T21:44:20.907977Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 285 sessionId: src_id|a816fd16-c87aa61-47adaec6-30140183_0 is DEAD
2025-10-09T21:44:20.916516Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a816fd16-c87aa61-47adaec6-30140183_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:20.916530Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a816fd16-c87aa61-47adaec6-30140183_0] Write session will restart in 0.008831s
2025-10-09T21:44:20.916556Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a816fd16-c87aa61-47adaec6-30140183_0] Write session: Do CDS request
2025-10-09T21:44:20.916563Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a816fd16-c87aa61-47adaec6-30140183_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:20.926567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a816fd16-c87aa61-47adaec6-30140183_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:20.926780Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:20.926799Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 286
2025-10-09T21:44:20.926952Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 286 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:20.926979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 286 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:20.926984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=286 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:20.926988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:20.927368Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 286 sessionId:  describe result for acl check
2025-10-09T21:44:20.927396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:20.927401Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:20.927402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:20.927408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341127396577054:4438] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:20.927932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341127396577054:4438] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:20.933184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341127396577054:4438] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:20.937958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:20.937685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341127396577054:4438] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:20.937706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341127396577054:4438] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:20.937710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341127396577054:4438] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:20.937719Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 286 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:20.942559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:20.942939Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:20.942950Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:20.942962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:20.943021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:20.943060Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341127034766568:4124]
2025-10-09T21:44:20.943670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:20.946633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:20.946712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:20.947081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:20.947132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:20.947133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:20.947134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:20.947135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:20.947136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:20.947137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:20.947138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.947139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.947140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.947146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:20.947156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:20.947159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:20.947163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:20.947166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:20.947170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:20.947174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.947177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.947181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.947188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:20.947196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:20.947202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:20.947206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:20.947210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:20.947217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:20.947221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:20.947226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:20.947230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:20.947234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:20.947238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:20.947243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:20.947248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.947252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.947258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.947262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.947266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.947271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.947299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:20.967022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:20.968451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:20.968459Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:20.968461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:20.968465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:20.968472Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 285 [2:7559341127034766568:4124]
2025-10-09T21:44:20.968491Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:20.968522Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:20.968526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.968529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.968531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.968550Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:20.968557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:20.968564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:20.968638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.968640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.968642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.968644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:20.968713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:20.968715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.968729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:20.968740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:20.968755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:20.968776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341127034766562:4124]
2025-10-09T21:44:20.968779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341127034766562:4124]
2025-10-09T21:44:20.968781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341127034766562:4124]
2025-10-09T21:44:20.968789Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:20.968805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:20.968820Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:20.968823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:20.968864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341127034766562:4124]
2025-10-09T21:44:20.968867Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:20.969275Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:20.969278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:20.969281Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:20.969286Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:20.969723Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 285
2025-10-09T21:44:20.969770Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 285
2025-10-09T21:44:20.970492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341127396577131:4438], now have 1 active actors on pipe
2025-10-09T21:44:20.970506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341127396577017:2432], now have 1 active actors on pipe
2025-10-09T21:44:20.970523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:20.970530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:20.970577Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d40bc104-b095f9bd-103116a8-1439a99f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:20.970588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.970593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.970596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:20.970609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.970625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:20.970697Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:44:20.970725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:20.973270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:20.973284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:20.973314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:20.973236Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 3 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:44:20.973280Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:44:20.973403Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 286 partition: 0 MaxSeqNo: 25 sessionId: src_id|d40bc104-b095f9bd-103116a8-1439a99f_0
2025-10-09T21:44:20.973821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a816fd16-c87aa61-47adaec6-30140183_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d40bc104-b095f9bd-103116a8-1439a99f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:20.976403Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d40bc104-b095f9bd-103116a8-1439a99f_0] Write session: try to update token
2025-10-09T21:44:20.976427Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d40bc104-b095f9bd-103116a8-1439a99f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:20.988307Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:20.988338Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:20.988341Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:20.988343Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:20.988352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341127034766562:4124]
2025-10-09T21:44:20.988357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341127034766562:4124]
2025-10-09T21:44:20.988361Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341127034766562:4124]
2025-10-09T21:44:20.988403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:20.988412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:20.988414Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:20.988416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:20.988421Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:20.988424Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:20.988556Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:20.999627Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 286 sessionId: src_id|d40bc104-b095f9bd-103116a8-1439a99f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:21.002304Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:21.007878Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341131691544456:3810]
2025-10-09T21:44:21.008503Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 285, pipe: [1:7559341131691544456:3810]
2025-10-09T21:44:21.184156Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:21.184943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:21.184955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:21.184982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341131691544456:3810], now have 1 active actors on pipe
2025-10-09T21:44:21.185655Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:21.185629Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341131691544456:3810]
2025-10-09T21:44:21.185648Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:21.185669Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:21.185672Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:21.185731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 286 reason: error at write: Disconnected sessionId: src_id|d40bc104-b095f9bd-103116a8-1439a99f_0
2025-10-09T21:44:21.185938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 286 sessionId: src_id|d40bc104-b095f9bd-103116a8-1439a99f_0 is DEAD
2025-10-09T21:44:21.186410Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d40bc104-b095f9bd-103116a8-1439a99f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:21.186424Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d40bc104-b095f9bd-103116a8-1439a99f_0] Write session will restart in 0.006276s
2025-10-09T21:44:21.186451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d40bc104-b095f9bd-103116a8-1439a99f_0] Write session: Do CDS request
2025-10-09T21:44:21.186457Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d40bc104-b095f9bd-103116a8-1439a99f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:21.194687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d40bc104-b095f9bd-103116a8-1439a99f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:21.194728Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:21.194763Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 287
2025-10-09T21:44:21.198424Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 287 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:21.198488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 287 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:21.198494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=287 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:21.198499Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:21.198978Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 287 sessionId:  describe result for acl check
2025-10-09T21:44:21.199017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:21.199019Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:21.199021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:21.199033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341131691544477:4443] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:21.199630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341131691544477:4443] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:21.212041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341131691544477:4443] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:21.212895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:21.215365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:21.215603Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:21.215619Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:21.215633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:21.215697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:21.215756Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341131329734001:4135]
2025-10-09T21:44:21.216365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:21.216764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:21.216816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:21.217156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:21.217213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:21.217215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:21.217216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:21.217217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:21.217219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:21.217220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:21.217221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.217222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.217224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.217230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:21.217241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:21.217246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:21.217250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:21.217254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:21.217259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:21.217264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.217268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.217272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.217281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:21.217289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:21.217294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:21.217299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:21.217303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:21.217308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:21.217312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:21.217316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:21.217321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:21.217326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:21.217330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:21.217335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:21.217340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.217347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.217351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.217356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.217360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.217365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.217395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:21.225049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341131691544477:4443] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:21.225072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341131691544477:4443] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:21.225079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341131691544477:4443] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:21.225090Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 287 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:21.226747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:21.227962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:21.227969Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:21.227971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:21.227977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:21.227984Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 286 [2:7559341131329734001:4135]
2025-10-09T21:44:21.228000Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:21.228021Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:21.228024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.228028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.228030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.228049Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:21.228056Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:21.228063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:21.228129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.228131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.228133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.228136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:21.228213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:21.228216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.228229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.228241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.228259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:21.228280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341131329733995:4135]
2025-10-09T21:44:21.228282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341131329733995:4135]
2025-10-09T21:44:21.228285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341131329733995:4135]
2025-10-09T21:44:21.228304Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:21.228322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:21.228336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:21.228341Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:21.228358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:21.228361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:21.228363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:21.228383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341131329733995:4135]
2025-10-09T21:44:21.228386Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:21.228442Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:21.228614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341131691544570:4443], now have 1 active actors on pipe
2025-10-09T21:44:21.228623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341131691544462:2432], now have 1 active actors on pipe
2025-10-09T21:44:21.230197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 286
2025-10-09T21:44:21.230241Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 286
2025-10-09T21:44:21.232926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 287 partition: 0 MaxSeqNo: 25 sessionId: src_id|309553c2-ea604e2f-5e3e0036-c4e6fb75_0
2025-10-09T21:44:21.231391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:21.231411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:21.231453Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|309553c2-ea604e2f-5e3e0036-c4e6fb75_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:21.231466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.231472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.231476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:21.231489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.231507Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:21.233361Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d40bc104-b095f9bd-103116a8-1439a99f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|309553c2-ea604e2f-5e3e0036-c4e6fb75_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:21.231529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:21.232576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:21.232591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:21.232616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:21.235910Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|309553c2-ea604e2f-5e3e0036-c4e6fb75_0] Write session: try to update token
2025-10-09T21:44:21.235932Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|309553c2-ea604e2f-5e3e0036-c4e6fb75_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:21.253124Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:21.253152Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:21.253155Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:21.253157Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:21.253168Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341131329733995:4135]
2025-10-09T21:44:21.253174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341131329733995:4135]
2025-10-09T21:44:21.253180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341131329733995:4135]
2025-10-09T21:44:21.253232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:21.253243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.253250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:21.253356Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:21.253365Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:21.253370Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:21.253444Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:21.262105Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 287 sessionId: src_id|309553c2-ea604e2f-5e3e0036-c4e6fb75_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:21.264629Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:21.290085Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341131691544577:3810]
2025-10-09T21:44:21.290717Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 286, pipe: [1:7559341131691544577:3810]
2025-10-09T21:44:21.447798Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:21.448123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:21.448133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:21.448156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341131691544577:3810], now have 1 active actors on pipe
2025-10-09T21:44:21.448649Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:21.455069Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|309553c2-ea604e2f-5e3e0036-c4e6fb75_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:21.455086Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|309553c2-ea604e2f-5e3e0036-c4e6fb75_0] Write session will restart in 0.008043s
2025-10-09T21:44:21.455114Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|309553c2-ea604e2f-5e3e0036-c4e6fb75_0] Write session: Do CDS request
2025-10-09T21:44:21.455120Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|309553c2-ea604e2f-5e3e0036-c4e6fb75_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:21.454276Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341131691544577:3810]
2025-10-09T21:44:21.454316Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:21.454382Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:21.454484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 287 reason: status is not ok: tablet will be restarted right now sessionId: src_id|309553c2-ea604e2f-5e3e0036-c4e6fb75_0
2025-10-09T21:44:21.454668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 287 sessionId: src_id|309553c2-ea604e2f-5e3e0036-c4e6fb75_0 is DEAD
2025-10-09T21:44:21.464610Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|309553c2-ea604e2f-5e3e0036-c4e6fb75_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:21.465567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:21.465586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 288
2025-10-09T21:44:21.469804Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 288 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:21.469863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 288 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:21.469871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=288 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:21.469875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:21.472763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 288 sessionId:  describe result for acl check
2025-10-09T21:44:21.472808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:21.472811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:21.472812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:21.472820Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341131691544652:4447] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:21.473925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:21.473626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341131691544652:4447] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:21.477120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:21.477376Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:21.477389Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:21.477404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:21.477470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:21.477513Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341131329734111:4137]
2025-10-09T21:44:21.478181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:21.478623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:21.478685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:21.479002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:21.479067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:21.479068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:21.479069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:21.479070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:21.479071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:21.479072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:21.479073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.479074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.479076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.479080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:21.479090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:21.479094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:21.479097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:21.479100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:21.479103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:21.479107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.479110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.479113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.479119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:21.479126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:21.479133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:21.479138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:21.479143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:21.479148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:21.479151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:21.479156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:21.479160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:21.479166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:21.479170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:21.479175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:21.479179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.479183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.479187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.479191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.479195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.479199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.479224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:21.479474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341131691544652:4447] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:21.486885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341131691544652:4447] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:21.486913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341131691544652:4447] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:21.486918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341131691544652:4447] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:21.486931Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 288 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:21.487041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:21.488291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:21.488298Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:21.488301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:21.488305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:21.488313Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 287 [2:7559341131329734111:4137]
2025-10-09T21:44:21.488342Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:21.488361Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:21.488364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.488368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.488370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.488389Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:21.488394Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:21.488401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:21.488465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.488468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.488470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.488473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:21.488550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:21.488553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.488567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.488579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.488597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:21.488617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341131329734105:4137]
2025-10-09T21:44:21.488619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341131329734105:4137]
2025-10-09T21:44:21.488621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341131329734105:4137]
2025-10-09T21:44:21.488636Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:21.488654Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:21.488669Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:21.488672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:21.488714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341131329734105:4137]
2025-10-09T21:44:21.488717Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:21.488740Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:21.488744Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:21.488746Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:21.488755Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:21.489134Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 287
2025-10-09T21:44:21.489181Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 287
2025-10-09T21:44:21.490105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341131691544580:2432], now have 1 active actors on pipe
2025-10-09T21:44:21.490122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341131691544695:4447], now have 1 active actors on pipe
2025-10-09T21:44:21.490146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:21.490152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:21.490184Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|44362ec4-9993c8f9-ea940cf9-8d8beafb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:21.490204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.490210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.490212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:21.490225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.490239Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:21.490258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:21.492438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:21.492454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:21.492485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:21.492702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 288 partition: 0 MaxSeqNo: 25 sessionId: src_id|44362ec4-9993c8f9-ea940cf9-8d8beafb_0
2025-10-09T21:44:21.493176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|309553c2-ea604e2f-5e3e0036-c4e6fb75_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|44362ec4-9993c8f9-ea940cf9-8d8beafb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:21.495869Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|44362ec4-9993c8f9-ea940cf9-8d8beafb_0] Write session: try to update token
2025-10-09T21:44:21.495891Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|44362ec4-9993c8f9-ea940cf9-8d8beafb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:21.513247Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:21.513269Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:21.513271Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:21.513273Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:21.513283Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341131329734105:4137]
2025-10-09T21:44:21.513291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341131329734105:4137]
2025-10-09T21:44:21.513295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341131329734105:4137]
2025-10-09T21:44:21.513341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:21.513349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.513351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:21.513496Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:21.517051Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:21.517067Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:21.517072Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:21.534155Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 288 sessionId: src_id|44362ec4-9993c8f9-ea940cf9-8d8beafb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:21.536798Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:21.553381Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341131691544703:3810]
2025-10-09T21:44:21.554008Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 287, pipe: [1:7559341131691544703:3810]
2025-10-09T21:44:21.693565Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:21.694379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:21.694410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:21.694435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341131691544703:3810], now have 1 active actors on pipe
2025-10-09T21:44:21.694683Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341131691544703:3810]
2025-10-09T21:44:21.694724Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:21.694763Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:21.694771Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:21.694840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 288 reason: error at write: Disconnected sessionId: src_id|44362ec4-9993c8f9-ea940cf9-8d8beafb_0
2025-10-09T21:44:21.695014Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:21.695170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44362ec4-9993c8f9-ea940cf9-8d8beafb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:21.695178Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44362ec4-9993c8f9-ea940cf9-8d8beafb_0] Write session will restart in 0.005090s
2025-10-09T21:44:21.695201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44362ec4-9993c8f9-ea940cf9-8d8beafb_0] Write session: Do CDS request
2025-10-09T21:44:21.695224Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44362ec4-9993c8f9-ea940cf9-8d8beafb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:21.695034Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 288 sessionId: src_id|44362ec4-9993c8f9-ea940cf9-8d8beafb_0 is DEAD
2025-10-09T21:44:21.702643Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|44362ec4-9993c8f9-ea940cf9-8d8beafb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:21.703486Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:21.703504Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 289
2025-10-09T21:44:21.706581Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 289 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:21.706635Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 289 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:21.706641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=289 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:21.706645Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:21.707121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 289 sessionId:  describe result for acl check
2025-10-09T21:44:21.707169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:21.707171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:21.707173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:21.707179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341131691544718:4454] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:21.707752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341131691544718:4454] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:21.712364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341131691544718:4454] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:21.724447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:21.724177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341131691544718:4454] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:21.724200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341131691544718:4454] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:21.724205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341131691544718:4454] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:21.724227Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 289 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:21.727377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:21.727636Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:21.727640Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:21.727654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:21.727717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:21.727759Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341131329734222:4139]
2025-10-09T21:44:21.728325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:21.728899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:21.728952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:21.729294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:21.729353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:21.729355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:21.729356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:21.729357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:21.729359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:21.729360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:21.729361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.729363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.729364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.729370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:21.729382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:21.729387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:21.729391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:21.729395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:21.729399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:21.729403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.729408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.729413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.729420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:21.729429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:21.729436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:21.729441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:21.729445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:21.729452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:21.729456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:21.729461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:21.729465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:21.729469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:21.729474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:21.729479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:21.729484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.729489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.729493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.729497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.729502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.729506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.729537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:21.738788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:21.740012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:21.740033Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:21.740035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:21.740041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:21.740049Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 288 [2:7559341131329734222:4139]
2025-10-09T21:44:21.740068Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:21.740091Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:21.740094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.740098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.740100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.740123Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:21.740130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:21.740141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:21.740207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.740219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.740221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.740225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:21.740328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:21.740339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.740353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:21.740367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:21.740386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:21.740414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341131329734215:4139]
2025-10-09T21:44:21.740422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341131329734215:4139]
2025-10-09T21:44:21.740424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341131329734215:4139]
2025-10-09T21:44:21.740441Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:21.740456Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:21.740472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:21.740481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:21.740525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341131329734215:4139]
2025-10-09T21:44:21.740533Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:21.741077Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:21.741088Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:21.741091Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:21.741098Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:21.741255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341131691544815:4454], now have 1 active actors on pipe
2025-10-09T21:44:21.741284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341131691544710:2432], now have 1 active actors on pipe
2025-10-09T21:44:21.741551Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 288
2025-10-09T21:44:21.741608Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 288
2025-10-09T21:44:21.741702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:21.741714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:21.741746Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5dabcc06-e9103e4e-7374878d-4106a014_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:21.741762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.741766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.741769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:21.741794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.741817Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:21.741841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:21.742135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:21.742147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:21.742171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:21.742486Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 289 partition: 0 MaxSeqNo: 25 sessionId: src_id|5dabcc06-e9103e4e-7374878d-4106a014_0
2025-10-09T21:44:21.742921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44362ec4-9993c8f9-ea940cf9-8d8beafb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5dabcc06-e9103e4e-7374878d-4106a014_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:21.745569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5dabcc06-e9103e4e-7374878d-4106a014_0] Write session: try to update token
2025-10-09T21:44:21.745587Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5dabcc06-e9103e4e-7374878d-4106a014_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:21.765272Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 289 sessionId: src_id|5dabcc06-e9103e4e-7374878d-4106a014_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:21.767789Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:21.765780Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:21.765799Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:21.765801Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:21.765803Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:21.765813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341131329734215:4139]
2025-10-09T21:44:21.765818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341131329734215:4139]
2025-10-09T21:44:21.765822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341131329734215:4139]
2025-10-09T21:44:21.765867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:21.765875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:21.765877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:21.765976Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:21.765983Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:21.765987Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:21.766016Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:21.794860Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341131691544828:3810]
2025-10-09T21:44:21.795534Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 288, pipe: [1:7559341131691544828:3810]
2025-10-09T21:44:21.999454Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:21.999701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:21.999712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:21.999757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341131691544828:3810], now have 1 active actors on pipe
2025-10-09T21:44:22.000499Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:22.000513Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:22.000567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 289 reason: error at write: Disconnected sessionId: src_id|5dabcc06-e9103e4e-7374878d-4106a014_0
2025-10-09T21:44:22.000890Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5dabcc06-e9103e4e-7374878d-4106a014_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:22.000899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5dabcc06-e9103e4e-7374878d-4106a014_0] Write session will restart in 0.006573s
2025-10-09T21:44:22.000922Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5dabcc06-e9103e4e-7374878d-4106a014_0] Write session: Do CDS request
2025-10-09T21:44:22.000928Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5dabcc06-e9103e4e-7374878d-4106a014_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:22.000735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 289 sessionId: src_id|5dabcc06-e9103e4e-7374878d-4106a014_0 is DEAD
2025-10-09T21:44:22.001344Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341131691544828:3810]
2025-10-09T21:44:22.001365Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:22.001764Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:22.008605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5dabcc06-e9103e4e-7374878d-4106a014_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:22.008687Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:22.008720Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 290
2025-10-09T21:44:22.008853Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 290 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:22.008884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 290 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:22.008892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=290 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:22.008896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:22.009388Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 290 sessionId:  describe result for acl check
2025-10-09T21:44:22.009430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:22.009437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:22.009438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:22.009445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341135986512163:4465] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:22.010020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341135986512163:4465] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:22.012330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341135986512163:4465] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:22.015727Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341135986512163:4465] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:22.015744Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341135986512163:4465] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:22.015749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341135986512163:4465] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:22.015759Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 290 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:22.019250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:22.024394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:22.024665Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:22.024679Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:22.024692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:22.024757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:22.024806Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341135624701645:4147]
2025-10-09T21:44:22.025403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:22.025907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:22.025996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:22.026371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:22.026455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:22.026457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:22.026459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:22.026460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:22.026461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:22.026463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:22.026464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.026465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.026467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.026473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:22.026484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:22.026488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:22.026492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:22.026496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:22.026500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:22.026504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.026507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.026510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.026522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:22.026530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:22.026537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:22.026542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:22.026546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:22.026550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:22.026554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:22.026559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:22.026568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:22.026574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:22.026578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:22.026583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:22.026588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.026594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.026599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.026604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.026608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.026616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.026645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:22.046127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:22.047510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:22.047518Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:22.047520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:22.047525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:22.047533Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 289 [2:7559341135624701645:4147]
2025-10-09T21:44:22.047550Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:22.047572Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:22.047576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.047579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.047581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.047601Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:22.047607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:22.047614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:22.047678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.047682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.047684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.047687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:22.047772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:22.047775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.047790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.047803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.047818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:22.047840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341135624701638:4147]
2025-10-09T21:44:22.047843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341135624701638:4147]
2025-10-09T21:44:22.047847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341135624701638:4147]
2025-10-09T21:44:22.047863Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:22.047877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:22.047892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:22.047895Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:22.047939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341135624701638:4147]
2025-10-09T21:44:22.047945Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:22.048301Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:22.048312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:22.048314Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:22.048320Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:22.048690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341135986512240:4465], now have 1 active actors on pipe
2025-10-09T21:44:22.048701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341135986512156:2432], now have 1 active actors on pipe
2025-10-09T21:44:22.048847Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 289
2025-10-09T21:44:22.048886Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 289
2025-10-09T21:44:22.049125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:22.049140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:22.049177Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e2aca86d-6b4dab70-2fe8f08f-5f12aeda_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:22.049188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.049193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.049197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:22.049210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.049225Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:22.049245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:22.049841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:22.049852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:22.049879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:22.051285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5dabcc06-e9103e4e-7374878d-4106a014_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e2aca86d-6b4dab70-2fe8f08f-5f12aeda_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:22.050861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 290 partition: 0 MaxSeqNo: 25 sessionId: src_id|e2aca86d-6b4dab70-2fe8f08f-5f12aeda_0
2025-10-09T21:44:22.053978Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2aca86d-6b4dab70-2fe8f08f-5f12aeda_0] Write session: try to update token
2025-10-09T21:44:22.053999Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2aca86d-6b4dab70-2fe8f08f-5f12aeda_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:22.083434Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:22.083454Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:22.083457Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:22.083460Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:22.083470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341135624701638:4147]
2025-10-09T21:44:22.083474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341135624701638:4147]
2025-10-09T21:44:22.083477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341135624701638:4147]
2025-10-09T21:44:22.083525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:22.083532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.083535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:22.083679Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:22.083777Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:22.083784Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:22.083788Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:22.088599Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 290 sessionId: src_id|e2aca86d-6b4dab70-2fe8f08f-5f12aeda_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:22.091275Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:22.102368Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341135986512278:3810]
2025-10-09T21:44:22.103716Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 289, pipe: [1:7559341135986512278:3810]
2025-10-09T21:44:22.320356Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:22.320650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:22.320659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:22.320707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341135986512278:3810], now have 1 active actors on pipe
2025-10-09T21:44:22.320970Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:22.321816Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341135986512278:3810]
2025-10-09T21:44:22.322351Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2aca86d-6b4dab70-2fe8f08f-5f12aeda_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:22.321850Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:22.321919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 290 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e2aca86d-6b4dab70-2fe8f08f-5f12aeda_0
2025-10-09T21:44:22.322363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2aca86d-6b4dab70-2fe8f08f-5f12aeda_0] Write session will restart in 0.006212s
2025-10-09T21:44:22.322118Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 290 sessionId: src_id|e2aca86d-6b4dab70-2fe8f08f-5f12aeda_0 is DEAD
2025-10-09T21:44:22.322251Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:22.322401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2aca86d-6b4dab70-2fe8f08f-5f12aeda_0] Write session: Do CDS request
2025-10-09T21:44:22.322409Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2aca86d-6b4dab70-2fe8f08f-5f12aeda_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:22.330167Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2aca86d-6b4dab70-2fe8f08f-5f12aeda_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:22.330619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:22.330638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 291
2025-10-09T21:44:22.330888Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 291 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:22.330924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 291 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:22.330930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=291 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:22.330934Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:22.331466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 291 sessionId:  describe result for acl check
2025-10-09T21:44:22.331503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:22.331505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:22.331507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:22.331513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341135986512297:4470] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:22.332116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341135986512297:4470] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:22.337262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341135986512297:4470] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:22.343120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341135986512297:4470] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:22.343140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341135986512297:4470] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:22.343144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341135986512297:4470] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:22.343153Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 291 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:22.357993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:22.362467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:22.362740Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:22.362744Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:22.362758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:22.362823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:22.362869Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341135624701785:4157]
2025-10-09T21:44:22.363486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:22.364075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:22.364133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:22.364933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:22.365021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:22.365023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:22.365024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:22.365025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:22.365027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:22.365028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:22.365029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.365030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.365031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.365037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:22.365048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:22.365052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:22.365056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:22.365060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:22.365063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:22.365066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.365070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.365074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.365083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:22.365091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:22.365097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:22.365101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:22.365105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:22.365110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:22.365114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:22.365118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:22.365123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:22.365127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:22.365131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:22.365137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:22.365142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.365147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.365151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.365155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.365160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.365165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.365194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:22.374046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:22.377878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:22.377897Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:22.377899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:22.377904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:22.377912Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 290 [2:7559341135624701785:4157]
2025-10-09T21:44:22.377930Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:22.377953Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:22.377956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.377960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.377962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.377982Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:22.377988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:22.377995Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:22.378200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.378209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.378210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.378213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:22.378292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:22.378294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.378310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.378322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.378339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:22.378360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341135624701780:4157]
2025-10-09T21:44:22.378365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341135624701780:4157]
2025-10-09T21:44:22.378367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341135624701780:4157]
2025-10-09T21:44:22.378384Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:22.378422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:22.378438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:22.378441Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:22.378489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341135624701780:4157]
2025-10-09T21:44:22.378494Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:22.379146Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 290
2025-10-09T21:44:22.379196Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 290
2025-10-09T21:44:22.378674Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:22.378683Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:22.378685Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:22.378691Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:22.383652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341135986512282:2432], now have 1 active actors on pipe
2025-10-09T21:44:22.383670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341135986512384:4470], now have 1 active actors on pipe
2025-10-09T21:44:22.383711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:22.383718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:22.383756Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b7fecbbd-e41d7665-57146cb2-fa0e01b6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:22.383770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.383779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.383783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:22.383798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.383819Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:22.383839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:22.384859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:22.384875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:22.384900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:22.385581Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2aca86d-6b4dab70-2fe8f08f-5f12aeda_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b7fecbbd-e41d7665-57146cb2-fa0e01b6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:22.385109Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 291 partition: 0 MaxSeqNo: 25 sessionId: src_id|b7fecbbd-e41d7665-57146cb2-fa0e01b6_0
2025-10-09T21:44:22.388265Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7fecbbd-e41d7665-57146cb2-fa0e01b6_0] Write session: try to update token
2025-10-09T21:44:22.388288Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7fecbbd-e41d7665-57146cb2-fa0e01b6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:22.410715Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:22.410748Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:22.410751Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:22.410753Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:22.410763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341135624701780:4157]
2025-10-09T21:44:22.410768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341135624701780:4157]
2025-10-09T21:44:22.410773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341135624701780:4157]
2025-10-09T21:44:22.410822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:22.410831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.410834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:22.411012Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:22.415325Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:22.415341Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:22.415345Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:22.422580Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341135986512401:3810]
2025-10-09T21:44:22.422645Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 291 sessionId: src_id|b7fecbbd-e41d7665-57146cb2-fa0e01b6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:22.425038Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 290, pipe: [1:7559341135986512401:3810]
2025-10-09T21:44:22.425099Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:22.618688Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:22.618980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341135986512401:3810], now have 1 active actors on pipe
2025-10-09T21:44:22.619018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:22.619025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:22.622259Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:22.622318Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:22.622321Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:22.622398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 291 reason: error at write: Disconnected sessionId: src_id|b7fecbbd-e41d7665-57146cb2-fa0e01b6_0
2025-10-09T21:44:22.622829Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7fecbbd-e41d7665-57146cb2-fa0e01b6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:22.622863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7fecbbd-e41d7665-57146cb2-fa0e01b6_0] Write session will restart in 0.006843s
2025-10-09T21:44:22.622880Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7fecbbd-e41d7665-57146cb2-fa0e01b6_0] Write session: Do CDS request
2025-10-09T21:44:22.622885Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7fecbbd-e41d7665-57146cb2-fa0e01b6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:22.622547Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:22.622557Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 291 sessionId: src_id|b7fecbbd-e41d7665-57146cb2-fa0e01b6_0 is DEAD
2025-10-09T21:44:22.622704Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341135986512401:3810]
2025-10-09T21:44:22.630813Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7fecbbd-e41d7665-57146cb2-fa0e01b6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:22.630894Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:22.630913Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 292
2025-10-09T21:44:22.631072Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 292 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:22.631109Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 292 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:22.631115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=292 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:22.631119Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:22.633846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 292 sessionId:  describe result for acl check
2025-10-09T21:44:22.633894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:22.633899Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:22.633901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:22.633908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341135986512413:4475] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:22.634544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341135986512413:4475] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:22.641566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341135986512413:4475] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:22.651366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341135986512413:4475] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:22.651388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341135986512413:4475] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:22.651393Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341135986512413:4475] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:22.651404Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 292 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:22.654665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:22.657616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:22.657852Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:22.657866Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:22.657878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:22.657950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:22.657996Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341135624701899:4159]
2025-10-09T21:44:22.658579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:22.659123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:22.659175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:22.659467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:22.659516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:22.659517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:22.659518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:22.659519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:22.659521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:22.659521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:22.659522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.659523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.659524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.659530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:22.659540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:22.659543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:22.659546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:22.659549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:22.659553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:22.659556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.659558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.659561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.659568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:22.659574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:22.659579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:22.659584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:22.659588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:22.659592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:22.659598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:22.659602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:22.659606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:22.659609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:22.659613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:22.659618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:22.659623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.659627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.659631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.659635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.659638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.659642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.659665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:22.667938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:22.669171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:22.669177Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:22.669181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:22.669186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:22.669194Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 291 [2:7559341135624701899:4159]
2025-10-09T21:44:22.669208Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:22.669227Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:22.669231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.669234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.669237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.669258Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:22.669264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:22.669272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:22.669333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.669338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.669340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.669343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:22.669420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:22.669423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.669435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.669445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.669458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:22.669478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341135624701894:4159]
2025-10-09T21:44:22.669481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341135624701894:4159]
2025-10-09T21:44:22.669483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341135624701894:4159]
2025-10-09T21:44:22.669498Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:22.669513Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:22.669527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:22.669530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:22.669570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341135624701894:4159]
2025-10-09T21:44:22.669572Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:22.669911Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:22.669915Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:22.669917Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:22.669925Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:22.670331Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 291
2025-10-09T21:44:22.670365Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 291
2025-10-09T21:44:22.671078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341135986512501:4475], now have 1 active actors on pipe
2025-10-09T21:44:22.671092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341135986512406:2432], now have 1 active actors on pipe
2025-10-09T21:44:22.671109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:22.671116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:22.671151Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1ad49b45-bcbbb57e-dfc134f-dc8fd360_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:22.671162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.671167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.671170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:22.671182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.671198Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:22.671218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:22.671539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:22.671547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:22.671565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:22.672001Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 292 partition: 0 MaxSeqNo: 25 sessionId: src_id|1ad49b45-bcbbb57e-dfc134f-dc8fd360_0
2025-10-09T21:44:22.673410Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7fecbbd-e41d7665-57146cb2-fa0e01b6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1ad49b45-bcbbb57e-dfc134f-dc8fd360_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:22.676155Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad49b45-bcbbb57e-dfc134f-dc8fd360_0] Write session: try to update token
2025-10-09T21:44:22.676178Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad49b45-bcbbb57e-dfc134f-dc8fd360_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:22.704850Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:22.704866Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:22.704868Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:22.704870Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:22.704884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341135624701894:4159]
2025-10-09T21:44:22.704887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341135624701894:4159]
2025-10-09T21:44:22.704890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341135624701894:4159]
2025-10-09T21:44:22.704932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:22.704939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.704941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:22.705076Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:22.707349Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:22.707371Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:22.707374Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:22.721359Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 292 sessionId: src_id|1ad49b45-bcbbb57e-dfc134f-dc8fd360_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:22.723831Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341135986512527:3810]
2025-10-09T21:44:22.723833Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:22.724305Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 291, pipe: [1:7559341135986512527:3810]
2025-10-09T21:44:22.881872Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:22.882219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:22.882233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:22.882256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341135986512527:3810], now have 1 active actors on pipe
2025-10-09T21:44:22.882774Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:22.883332Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad49b45-bcbbb57e-dfc134f-dc8fd360_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:22.883349Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad49b45-bcbbb57e-dfc134f-dc8fd360_0] Write session will restart in 0.005212s
2025-10-09T21:44:22.883395Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad49b45-bcbbb57e-dfc134f-dc8fd360_0] Write session: Do CDS request
2025-10-09T21:44:22.883402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad49b45-bcbbb57e-dfc134f-dc8fd360_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:22.882884Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341135986512527:3810]
2025-10-09T21:44:22.882918Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:22.882920Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:22.882974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 292 reason: error at write: Disconnected sessionId: src_id|1ad49b45-bcbbb57e-dfc134f-dc8fd360_0
2025-10-09T21:44:22.883149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 292 sessionId: src_id|1ad49b45-bcbbb57e-dfc134f-dc8fd360_0 is DEAD
2025-10-09T21:44:22.883279Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:22.889892Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad49b45-bcbbb57e-dfc134f-dc8fd360_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:22.890492Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:22.890509Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 293
2025-10-09T21:44:22.890677Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 293 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:22.890712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 293 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:22.890719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=293 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:22.890723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:22.891246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 293 sessionId:  describe result for acl check
2025-10-09T21:44:22.891289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:22.891297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:22.891299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:22.891304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341135986512544:4479] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:22.891856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341135986512544:4479] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:22.902014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341135986512544:4479] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:22.911536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341135986512544:4479] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:22.911556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341135986512544:4479] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:22.911561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341135986512544:4479] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:22.911578Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 293 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:22.915939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:22.920664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:22.920943Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:22.920948Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:22.920961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:22.921023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:22.921066Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341135624702013:4161]
2025-10-09T21:44:22.921622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:22.922505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:22.922578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:22.922948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:22.923018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:22.923019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:22.923020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:22.923021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:22.923023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:22.923024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:22.923026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.923027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.923028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.923034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:22.923045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:22.923048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:22.923051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:22.923054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:22.923077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:22.923081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.923084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.923087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.923096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:22.923104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:22.923111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:22.923117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:22.923124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:22.923129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:22.923134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:22.923138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:22.923143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:22.923148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:22.923153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:22.923158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:22.923164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.923170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.923174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.923179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.923183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.923189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.923215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:22.933470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:22.934716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:22.934724Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:22.934727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:22.934732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:22.934740Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 292 [2:7559341135624702013:4161]
2025-10-09T21:44:22.934756Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:22.934780Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:22.934784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.934787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.934789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.934808Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:22.934816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:22.934825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:22.934895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.934898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.934902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.934905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:22.935303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:22.935307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.935321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:22.935334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:22.935351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:22.935372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341135624702007:4161]
2025-10-09T21:44:22.935374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341135624702007:4161]
2025-10-09T21:44:22.935376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341135624702007:4161]
2025-10-09T21:44:22.935394Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:22.935409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:22.935423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:22.935426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:22.935468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341135624702007:4161]
2025-10-09T21:44:22.935470Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:22.937504Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 292
2025-10-09T21:44:22.937545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 292
2025-10-09T21:44:22.936634Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:22.936648Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:22.936654Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:22.936660Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:22.938081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341135986512532:2432], now have 1 active actors on pipe
2025-10-09T21:44:22.938090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341135986512646:4479], now have 1 active actors on pipe
2025-10-09T21:44:22.938116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:22.938121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:22.938163Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|97ff9d1-cb796874-979f302-ea873c68_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:22.938176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.938181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.938184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:22.938197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.938215Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:22.938237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:22.940765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:22.940787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:22.940822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:22.942228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 293 partition: 0 MaxSeqNo: 25 sessionId: src_id|97ff9d1-cb796874-979f302-ea873c68_0
2025-10-09T21:44:22.942747Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad49b45-bcbbb57e-dfc134f-dc8fd360_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|97ff9d1-cb796874-979f302-ea873c68_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:22.945218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97ff9d1-cb796874-979f302-ea873c68_0] Write session: try to update token
2025-10-09T21:44:22.945238Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97ff9d1-cb796874-979f302-ea873c68_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:22.982217Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341135986512659:3810]
2025-10-09T21:44:22.983425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341135986512659:3810], now have 1 active actors on pipe
2025-10-09T21:44:22.983592Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 292, pipe: [1:7559341135986512659:3810]
2025-10-09T21:44:22.988475Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 293 sessionId: src_id|97ff9d1-cb796874-979f302-ea873c68_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:22.991968Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:22.989752Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:22.989777Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:22.989779Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:22.989781Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:22.989794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341135624702007:4161]
2025-10-09T21:44:22.989800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341135624702007:4161]
2025-10-09T21:44:22.989804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341135624702007:4161]
2025-10-09T21:44:22.989851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:22.989858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:22.989861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:22.990061Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:22.990756Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:22.990762Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:22.990766Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:23.193291Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:23.193770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:23.193783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:23.198676Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:23.199354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97ff9d1-cb796874-979f302-ea873c68_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:23.199364Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97ff9d1-cb796874-979f302-ea873c68_0] Write session will restart in 0.005934s
2025-10-09T21:44:23.199388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97ff9d1-cb796874-979f302-ea873c68_0] Write session: Do CDS request
2025-10-09T21:44:23.199395Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97ff9d1-cb796874-979f302-ea873c68_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:23.198829Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:23.198852Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:23.198922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 293 reason: error at write: Disconnected sessionId: src_id|97ff9d1-cb796874-979f302-ea873c68_0
2025-10-09T21:44:23.199119Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 293 sessionId: src_id|97ff9d1-cb796874-979f302-ea873c68_0 is DEAD
2025-10-09T21:44:23.199282Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341135986512659:3810]
2025-10-09T21:44:23.199297Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:23.206593Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97ff9d1-cb796874-979f302-ea873c68_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:23.207147Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:23.207165Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 294
2025-10-09T21:44:23.207684Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 294 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:23.207737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 294 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:23.207744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=294 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:23.207749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:23.208272Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 294 sessionId:  describe result for acl check
2025-10-09T21:44:23.208317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:23.208320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:23.208322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:23.208328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341140281479997:4491] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:23.208912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341140281479997:4491] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:23.213950Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341140281479997:4491] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:23.217432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341140281479997:4491] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:23.217451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341140281479997:4491] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:23.217455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341140281479997:4491] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:23.217464Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 294 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:23.231621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:23.238688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:23.238976Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:23.238993Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:23.239009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:23.239088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:23.239140Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341139919669466:4177]
2025-10-09T21:44:23.239797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:23.246276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:23.246407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:23.246828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:23.246889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:23.246891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:23.246892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:23.246893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:23.246894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:23.246896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:23.246897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.246899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.246900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.246908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:23.246919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:23.246923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:23.246927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:23.246931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:23.246935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:23.246939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.246942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.246946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.246955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:23.246964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:23.246971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:23.246978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:23.246983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:23.246990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:23.246994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:23.247002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:23.247008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:23.247014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:23.247021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:23.247027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:23.247034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.247039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.247044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.247049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.247053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.247059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.247094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:23.256696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:23.258440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:23.258457Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:23.258460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:23.258465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:23.258473Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 293 [2:7559341139919669466:4177]
2025-10-09T21:44:23.258492Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:23.258512Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:23.258515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.258519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.258521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.258542Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:23.258551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:23.258560Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:23.258627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.258629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.258632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.258635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:23.258712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:23.258715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.258732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.258744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.258762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:23.258785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341139919669460:4177]
2025-10-09T21:44:23.258788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341139919669460:4177]
2025-10-09T21:44:23.258790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341139919669460:4177]
2025-10-09T21:44:23.258800Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:23.258815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:23.258831Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:23.258834Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:23.258879Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341139919669460:4177]
2025-10-09T21:44:23.258882Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:23.258888Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:23.258892Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:23.258894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:23.258901Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:23.258920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341140281479986:2432], now have 1 active actors on pipe
2025-10-09T21:44:23.258924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341140281480067:4491], now have 1 active actors on pipe
2025-10-09T21:44:23.259726Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 293
2025-10-09T21:44:23.259771Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 293
2025-10-09T21:44:23.260068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:23.260083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:23.260140Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8a904089-95608d11-a91d141d-a22b745c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:23.260156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.260168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.260170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:23.260181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.260197Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:23.260218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:23.260553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:23.260566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:23.260584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:23.260989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 294 partition: 0 MaxSeqNo: 25 sessionId: src_id|8a904089-95608d11-a91d141d-a22b745c_0
2025-10-09T21:44:23.261414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97ff9d1-cb796874-979f302-ea873c68_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8a904089-95608d11-a91d141d-a22b745c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:23.263994Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a904089-95608d11-a91d141d-a22b745c_0] Write session: try to update token
2025-10-09T21:44:23.264016Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a904089-95608d11-a91d141d-a22b745c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:23.277948Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:23.277967Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:23.277969Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:23.277971Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:23.277981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341139919669460:4177]
2025-10-09T21:44:23.277985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341139919669460:4177]
2025-10-09T21:44:23.277989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341139919669460:4177]
2025-10-09T21:44:23.278050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:23.278051Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:23.278056Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:23.278057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.278059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:23.278061Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:23.278203Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:23.283636Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 294 sessionId: src_id|8a904089-95608d11-a91d141d-a22b745c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:23.286300Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:23.298142Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341140281480106:3810]
2025-10-09T21:44:23.298527Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 293, pipe: [1:7559341140281480106:3810]
2025-10-09T21:44:23.453300Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:23.453613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:23.453625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:23.453674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341140281480106:3810], now have 1 active actors on pipe
2025-10-09T21:44:23.453967Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:23.454075Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:23.454146Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341140281480106:3810]
2025-10-09T21:44:23.454177Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:23.454179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:23.454232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 294 reason: error at write: Disconnected sessionId: src_id|8a904089-95608d11-a91d141d-a22b745c_0
2025-10-09T21:44:23.454437Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 294 sessionId: src_id|8a904089-95608d11-a91d141d-a22b745c_0 is DEAD
2025-10-09T21:44:23.454694Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a904089-95608d11-a91d141d-a22b745c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:23.454706Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a904089-95608d11-a91d141d-a22b745c_0] Write session will restart in 0.009596s
2025-10-09T21:44:23.454735Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a904089-95608d11-a91d141d-a22b745c_0] Write session: Do CDS request
2025-10-09T21:44:23.454741Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a904089-95608d11-a91d141d-a22b745c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:23.465557Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a904089-95608d11-a91d141d-a22b745c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:23.465800Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:23.465818Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 295
2025-10-09T21:44:23.466281Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 295 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:23.466327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 295 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:23.466332Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=295 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:23.466336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:23.467841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 295 sessionId:  describe result for acl check
2025-10-09T21:44:23.467886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:23.467888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:23.467889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:23.467896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341140281480161:4495] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:23.468480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341140281480161:4495] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:23.474447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341140281480161:4495] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:23.476690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:23.478167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:23.478421Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:23.478425Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:23.478438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:23.478491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:23.478528Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341139919669580:4179]
2025-10-09T21:44:23.479238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:23.480508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:23.480569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:23.480905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:23.480968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:23.480969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:23.480970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:23.480971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:23.480973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:23.480974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:23.480975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.480976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.480977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.480984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:23.480994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:23.480998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:23.481002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:23.481006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:23.481010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:23.481014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.481017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.481020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.481028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:23.481036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:23.481042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:23.481047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:23.481051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:23.481055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:23.481060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:23.481065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:23.481069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:23.481074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:23.481078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:23.481084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:23.481088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.481093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.481096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.481101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.481105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.481110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.481137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:23.480285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341140281480161:4495] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:23.480304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341140281480161:4495] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:23.480309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341140281480161:4495] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:23.480319Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 295 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:23.491255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:23.492678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:23.492697Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:23.492700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:23.492704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:23.492711Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 294 [2:7559341139919669580:4179]
2025-10-09T21:44:23.492727Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:23.492748Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:23.492751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.492755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.492758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.492777Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:23.492789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:23.492797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:23.492867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.492876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.492878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.492881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:23.492957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:23.492965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.492979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.492990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.493005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:23.493026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341139919669575:4179]
2025-10-09T21:44:23.493028Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341139919669575:4179]
2025-10-09T21:44:23.493031Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341139919669575:4179]
2025-10-09T21:44:23.493039Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:23.493055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:23.493065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:23.493068Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:23.493070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:23.493070Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:23.493072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:23.493124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341140281480224:4495], now have 1 active actors on pipe
2025-10-09T21:44:23.493128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341140281480109:2432], now have 1 active actors on pipe
2025-10-09T21:44:23.493130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341139919669575:4179]
2025-10-09T21:44:23.493139Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:23.493150Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:23.493446Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 294
2025-10-09T21:44:23.493492Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 294
2025-10-09T21:44:23.493748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:23.493760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:23.493803Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|610b591a-ce1d308a-645b0ffc-5262611e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:23.493821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.493831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.493834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:23.493858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.493876Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:23.493894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:23.496770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:23.496787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:23.496827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:23.497646Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 295 partition: 0 MaxSeqNo: 25 sessionId: src_id|610b591a-ce1d308a-645b0ffc-5262611e_0
2025-10-09T21:44:23.498188Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a904089-95608d11-a91d141d-a22b745c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|610b591a-ce1d308a-645b0ffc-5262611e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:23.500907Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|610b591a-ce1d308a-645b0ffc-5262611e_0] Write session: try to update token
2025-10-09T21:44:23.500929Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|610b591a-ce1d308a-645b0ffc-5262611e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:23.511616Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:23.511635Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:23.511638Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:23.511640Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:23.511650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341139919669575:4179]
2025-10-09T21:44:23.511655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341139919669575:4179]
2025-10-09T21:44:23.511658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341139919669575:4179]
2025-10-09T21:44:23.511706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:23.511708Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:23.511712Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:23.511715Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:23.511715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.511718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:23.511863Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:23.539207Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 295 sessionId: src_id|610b591a-ce1d308a-645b0ffc-5262611e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:23.541761Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:23.554072Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341140281480235:3810]
2025-10-09T21:44:23.554528Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 294, pipe: [1:7559341140281480235:3810]
2025-10-09T21:44:23.704421Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:23.704948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:23.704962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:23.704987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341140281480235:3810], now have 1 active actors on pipe
2025-10-09T21:44:23.705720Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341140281480235:3810]
2025-10-09T21:44:23.705753Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:23.705756Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:23.705817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 295 reason: error at write: Disconnected sessionId: src_id|610b591a-ce1d308a-645b0ffc-5262611e_0
2025-10-09T21:44:23.706006Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 295 sessionId: src_id|610b591a-ce1d308a-645b0ffc-5262611e_0 is DEAD
2025-10-09T21:44:23.706144Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:23.706323Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:23.706397Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|610b591a-ce1d308a-645b0ffc-5262611e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:23.706405Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|610b591a-ce1d308a-645b0ffc-5262611e_0] Write session will restart in 0.007728s
2025-10-09T21:44:23.706431Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|610b591a-ce1d308a-645b0ffc-5262611e_0] Write session: Do CDS request
2025-10-09T21:44:23.706437Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|610b591a-ce1d308a-645b0ffc-5262611e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:23.717126Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|610b591a-ce1d308a-645b0ffc-5262611e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:23.717281Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:23.717300Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 296
2025-10-09T21:44:23.719865Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 296 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:23.719916Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 296 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:23.719922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=296 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:23.719926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:23.720834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 296 sessionId:  describe result for acl check
2025-10-09T21:44:23.720887Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:23.720893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:23.720895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:23.720901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341140281480251:4500] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:23.721469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341140281480251:4500] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:23.723853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341140281480251:4500] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:23.732675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341140281480251:4500] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:23.732697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341140281480251:4500] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:23.732701Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341140281480251:4500] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:23.732712Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 296 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:23.732866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:23.733969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:23.734237Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:23.734241Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:23.734256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:23.734338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:23.734381Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341139919669696:4181]
2025-10-09T21:44:23.735033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:23.739084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:23.739182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:23.739589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:23.739671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:23.739673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:23.739674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:23.739675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:23.739677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:23.739678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:23.739679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.739680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.739682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.739688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:23.739699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:23.739703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:23.739706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:23.739710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:23.739714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:23.739718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.739722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.739726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.739735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:23.739744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:23.739760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:23.739765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:23.739770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:23.739775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:23.739778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:23.739783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:23.739788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:23.739793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:23.739798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:23.739804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:23.739810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.739815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.739821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.739826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.739831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.739839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.739874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:23.749306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:23.751068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:23.751090Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:23.751092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:23.751096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:23.751104Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 295 [2:7559341139919669696:4181]
2025-10-09T21:44:23.751120Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:23.751140Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:23.751148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.751151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.751153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.751172Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:23.751179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:23.751186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:23.751252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.751260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.751262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.751265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:23.752594Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 295
2025-10-09T21:44:23.752626Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 295
2025-10-09T21:44:23.751341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:23.751350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.751366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:23.751376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:23.751392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:23.751416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341139919669690:4181]
2025-10-09T21:44:23.751417Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341139919669690:4181]
2025-10-09T21:44:23.751420Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341139919669690:4181]
2025-10-09T21:44:23.751434Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:23.751452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:23.751466Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:23.751469Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:23.751512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341139919669690:4181]
2025-10-09T21:44:23.751514Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:23.751683Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:23.751687Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:23.751689Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:23.751696Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:23.751935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341140281480349:4500], now have 1 active actors on pipe
2025-10-09T21:44:23.751943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341140281480238:2432], now have 1 active actors on pipe
2025-10-09T21:44:23.752999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:23.753009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:23.753049Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e3450d98-a634b22c-86031f4b-96e6545c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:23.753069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.753074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.753077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:23.753090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.753107Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:23.753128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:23.754481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:23.754508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:23.754534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:23.754882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 296 partition: 0 MaxSeqNo: 25 sessionId: src_id|e3450d98-a634b22c-86031f4b-96e6545c_0
2025-10-09T21:44:23.755330Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|610b591a-ce1d308a-645b0ffc-5262611e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e3450d98-a634b22c-86031f4b-96e6545c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:23.758066Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3450d98-a634b22c-86031f4b-96e6545c_0] Write session: try to update token
2025-10-09T21:44:23.758084Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3450d98-a634b22c-86031f4b-96e6545c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:23.778731Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:23.778752Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:23.778754Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:23.778756Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:23.778767Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341139919669690:4181]
2025-10-09T21:44:23.778772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341139919669690:4181]
2025-10-09T21:44:23.778776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341139919669690:4181]
2025-10-09T21:44:23.778825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:23.778834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:23.778837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:23.778973Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:23.782178Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:23.782198Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:23.782201Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:23.796205Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 296 sessionId: src_id|e3450d98-a634b22c-86031f4b-96e6545c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:23.798726Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:23.805386Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341140281480362:3810]
2025-10-09T21:44:23.806246Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 295, pipe: [1:7559341140281480362:3810]
2025-10-09T21:44:24.030849Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:24.031185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:24.031202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:24.031227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341140281480362:3810], now have 1 active actors on pipe
2025-10-09T21:44:24.031571Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:24.031808Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:24.031824Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:24.031893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 296 reason: error at write: Disconnected sessionId: src_id|e3450d98-a634b22c-86031f4b-96e6545c_0
2025-10-09T21:44:24.032085Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 296 sessionId: src_id|e3450d98-a634b22c-86031f4b-96e6545c_0 is DEAD
2025-10-09T21:44:24.032231Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3450d98-a634b22c-86031f4b-96e6545c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:24.032244Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3450d98-a634b22c-86031f4b-96e6545c_0] Write session will restart in 0.006444s
2025-10-09T21:44:24.032269Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3450d98-a634b22c-86031f4b-96e6545c_0] Write session: Do CDS request
2025-10-09T21:44:24.032275Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3450d98-a634b22c-86031f4b-96e6545c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:24.032290Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:24.032329Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341140281480362:3810]
2025-10-09T21:44:24.039660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3450d98-a634b22c-86031f4b-96e6545c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:24.039730Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:24.039749Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 297
2025-10-09T21:44:24.039983Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 297 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:24.040023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 297 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:24.040029Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=297 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:24.040036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:24.040774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 297 sessionId:  describe result for acl check
2025-10-09T21:44:24.040815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:24.040817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:24.040819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:24.040825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341144576447689:4505] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:24.041419Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341144576447689:4505] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:24.047936Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341144576447689:4505] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:24.052724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341144576447689:4505] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:24.052747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341144576447689:4505] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:24.052752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341144576447689:4505] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:24.052764Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 297 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:24.056844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:24.063445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:24.063756Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:24.063784Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:24.063803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:24.063887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:24.063942Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341144214637111:4184]
2025-10-09T21:44:24.064588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:24.065131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:24.065199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:24.065699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:24.065782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:24.065791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:24.065793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:24.065794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:24.065796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:24.065797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:24.065798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.065800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.065801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.065809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:24.065819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:24.065824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:24.065828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:24.065831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:24.065835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:24.065839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.065842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.065846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.065854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:24.065863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:24.065870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:24.065876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:24.065890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:24.065895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:24.065900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:24.065905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:24.065910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:24.065915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:24.065919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:24.065924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:24.065929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.065934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.065938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.065944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.065953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.065959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.065990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:24.080873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:24.081985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:24.082004Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:24.082008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:24.082013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:24.082021Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 296 [2:7559341144214637111:4184]
2025-10-09T21:44:24.082075Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:24.082096Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:24.082106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.082110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.082113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.082133Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:24.082140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:24.082147Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:24.082218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.082220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.082224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.082227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:24.082305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:24.082308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.082321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.082331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.082346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:24.082369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341144214637106:4184]
2025-10-09T21:44:24.082372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341144214637106:4184]
2025-10-09T21:44:24.082374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341144214637106:4184]
2025-10-09T21:44:24.082397Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:24.082414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:24.082428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:24.082432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:24.082478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341144214637106:4184]
2025-10-09T21:44:24.082483Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:24.082615Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:24.082617Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:24.082620Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:24.082626Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:24.083479Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 296
2025-10-09T21:44:24.083523Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 296
2025-10-09T21:44:24.083192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341144576447783:4505], now have 1 active actors on pipe
2025-10-09T21:44:24.083205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341144576447675:2432], now have 1 active actors on pipe
2025-10-09T21:44:24.083930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:24.083943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:24.083979Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8ec4c480-6d31eb31-5807b25c-7a834c6c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:24.083988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.083993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.083996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:24.084009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.084023Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:24.084043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:24.084799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:24.084803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:24.084823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:24.085415Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3450d98-a634b22c-86031f4b-96e6545c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8ec4c480-6d31eb31-5807b25c-7a834c6c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:24.088012Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8ec4c480-6d31eb31-5807b25c-7a834c6c_0] Write session: try to update token
2025-10-09T21:44:24.088054Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8ec4c480-6d31eb31-5807b25c-7a834c6c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:24.085006Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 297 partition: 0 MaxSeqNo: 25 sessionId: src_id|8ec4c480-6d31eb31-5807b25c-7a834c6c_0
2025-10-09T21:44:24.099695Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:24.099718Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:24.099721Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:24.099723Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:24.099733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341144214637106:4184]
2025-10-09T21:44:24.099737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341144214637106:4184]
2025-10-09T21:44:24.099741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341144214637106:4184]
2025-10-09T21:44:24.099789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:24.099793Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:24.099796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.099799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:24.099799Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:24.099803Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:24.099936Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:24.127499Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 297 sessionId: src_id|8ec4c480-6d31eb31-5807b25c-7a834c6c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:24.130078Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:24.132617Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341144576447802:3810]
2025-10-09T21:44:24.133012Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 296, pipe: [1:7559341144576447802:3810]
2025-10-09T21:44:24.273409Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:24.274192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:24.274210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:24.274236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341144576447802:3810], now have 1 active actors on pipe
2025-10-09T21:44:24.274603Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:24.274619Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:24.274693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 297 reason: error at write: Disconnected sessionId: src_id|8ec4c480-6d31eb31-5807b25c-7a834c6c_0
2025-10-09T21:44:24.274887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 297 sessionId: src_id|8ec4c480-6d31eb31-5807b25c-7a834c6c_0 is DEAD
2025-10-09T21:44:24.275045Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:24.275080Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341144576447802:3810]
2025-10-09T21:44:24.274620Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:24.278410Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ec4c480-6d31eb31-5807b25c-7a834c6c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:24.278426Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ec4c480-6d31eb31-5807b25c-7a834c6c_0] Write session will restart in 0.006959s
2025-10-09T21:44:24.278457Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ec4c480-6d31eb31-5807b25c-7a834c6c_0] Write session: Do CDS request
2025-10-09T21:44:24.278464Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ec4c480-6d31eb31-5807b25c-7a834c6c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:24.289810Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8ec4c480-6d31eb31-5807b25c-7a834c6c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:24.290203Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:24.290221Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 298
2025-10-09T21:44:24.290411Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 298 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:24.290447Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 298 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:24.290453Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=298 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:24.290457Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:24.294503Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 298 sessionId:  describe result for acl check
2025-10-09T21:44:24.294560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:24.294562Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:24.294564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:24.294571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341144576447903:4516] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:24.295147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341144576447903:4516] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:24.300334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:24.300526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:24.300700Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:24.300704Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:24.300717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:24.300768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:24.300816Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341144214637269:4199]
2025-10-09T21:44:24.301435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:24.301960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:24.302008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:24.302357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:24.302440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:24.302441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:24.302442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:24.302443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:24.302445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:24.302446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:24.302447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.302448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.302449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.302454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:24.302465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:24.302469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:24.302473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:24.302477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:24.302481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:24.302485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.302488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.302491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.302500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:24.302509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:24.302515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:24.302520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:24.302525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:24.302529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:24.302533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:24.302538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:24.302544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:24.302548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:24.302553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:24.302558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:24.302564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.302568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.302575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.302582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.302586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.302591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.302635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:24.306952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341144576447903:4516] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:24.315021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341144576447903:4516] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:24.315043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341144576447903:4516] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:24.315048Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341144576447903:4516] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:24.315058Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 298 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:24.316748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:24.318380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:24.318407Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:24.318410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:24.318414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:24.318421Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 297 [2:7559341144214637269:4199]
2025-10-09T21:44:24.318436Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:24.318456Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:24.318459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.318462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.318464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.318487Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:24.318493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:24.318500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:24.318562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.318564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.318566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.318569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:24.318640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:24.318642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.318656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.318667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.318681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:24.318704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341144214637264:4199]
2025-10-09T21:44:24.318706Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341144214637264:4199]
2025-10-09T21:44:24.318708Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341144214637264:4199]
2025-10-09T21:44:24.318722Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:24.318743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:24.318758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:24.318762Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:24.318770Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:24.318773Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:24.318775Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:24.318808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341144214637264:4199]
2025-10-09T21:44:24.318811Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:24.318850Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:24.318877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341144576447825:2432], now have 1 active actors on pipe
2025-10-09T21:44:24.318881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341144576447940:4516], now have 1 active actors on pipe
2025-10-09T21:44:24.319200Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 297
2025-10-09T21:44:24.319245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 297
2025-10-09T21:44:24.320861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:24.320876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:24.320912Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bf83f8ab-8eb1c9d-cfcbc6d4-3228203d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:24.320925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.320929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.320932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:24.320947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.320962Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:24.320981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:24.323239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:24.323255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:24.323286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:24.323463Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 298 partition: 0 MaxSeqNo: 25 sessionId: src_id|bf83f8ab-8eb1c9d-cfcbc6d4-3228203d_0
2025-10-09T21:44:24.326550Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ec4c480-6d31eb31-5807b25c-7a834c6c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bf83f8ab-8eb1c9d-cfcbc6d4-3228203d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:24.329178Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf83f8ab-8eb1c9d-cfcbc6d4-3228203d_0] Write session: try to update token
2025-10-09T21:44:24.329200Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf83f8ab-8eb1c9d-cfcbc6d4-3228203d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:24.341358Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:24.341381Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:24.341404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341144214637264:4199]
2025-10-09T21:44:24.341457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:24.341629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:24.341806Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:24.341964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:24.342112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:24.342310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:24.342355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:24.342742Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:24.363726Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 298 sessionId: src_id|bf83f8ab-8eb1c9d-cfcbc6d4-3228203d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:24.364029Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:24.364048Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:24.364050Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:24.364053Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:24.364064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341144214637264:4199]
2025-10-09T21:44:24.364079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341144214637264:4199]
2025-10-09T21:44:24.364084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341144214637264:4199]
2025-10-09T21:44:24.364131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:24.364139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.364141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:24.364296Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:24.366243Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:24.366265Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:24.366268Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:24.366296Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:24.378147Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341144576447950:3810]
2025-10-09T21:44:24.378946Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 297, pipe: [1:7559341144576447950:3810]
2025-10-09T21:44:24.623091Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:24.623397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:24.623408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:24.623434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341144576447950:3810], now have 1 active actors on pipe
2025-10-09T21:44:24.623768Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:24.623864Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341144576447950:3810]
2025-10-09T21:44:24.623910Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:24.623947Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:24.624044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 298 reason: status is not ok: tablet will be restarted right now sessionId: src_id|bf83f8ab-8eb1c9d-cfcbc6d4-3228203d_0
2025-10-09T21:44:24.624298Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 298 sessionId: src_id|bf83f8ab-8eb1c9d-cfcbc6d4-3228203d_0 is DEAD
2025-10-09T21:44:24.624387Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf83f8ab-8eb1c9d-cfcbc6d4-3228203d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:24.624398Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf83f8ab-8eb1c9d-cfcbc6d4-3228203d_0] Write session will restart in 0.007171s
2025-10-09T21:44:24.624423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf83f8ab-8eb1c9d-cfcbc6d4-3228203d_0] Write session: Do CDS request
2025-10-09T21:44:24.624430Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf83f8ab-8eb1c9d-cfcbc6d4-3228203d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:24.632580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf83f8ab-8eb1c9d-cfcbc6d4-3228203d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:24.633077Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:24.633094Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 299
2025-10-09T21:44:24.633270Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 299 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:24.633306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 299 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:24.633313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=299 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:24.633318Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:24.633794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 299 sessionId:  describe result for acl check
2025-10-09T21:44:24.633840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:24.633847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:24.633849Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:24.633855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341144576447969:4521] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:24.634534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341144576447969:4521] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:24.640835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341144576447969:4521] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:24.643284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:24.643841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:24.644059Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:24.644063Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:24.644075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:24.644134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:24.644175Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341144214637390:4201]
2025-10-09T21:44:24.644777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:24.645397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:24.645457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:24.645764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:24.645830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:24.645831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:24.645832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:24.645833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:24.645834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:24.645836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:24.645837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.645838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.645839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.645844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:24.645854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:24.645857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:24.645862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:24.645865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:24.645869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:24.645872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.645875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.645879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.645889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:24.645897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:24.645902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:24.645907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:24.645911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:24.645915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:24.645919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:24.645923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:24.645928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:24.645933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:24.645936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:24.645941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:24.645945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.645950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.645954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.645957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.645961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.645984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.646013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:24.646803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341144576447969:4521] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:24.646822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341144576447969:4521] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:24.646826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341144576447969:4521] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:24.646835Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 299 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:24.652867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:24.654449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:24.654464Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:24.654467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:24.654471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:24.654478Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 298 [2:7559341144214637390:4201]
2025-10-09T21:44:24.654492Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:24.654512Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:24.654515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.654519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.654521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.654543Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:24.654555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:24.654563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:24.654628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.654637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.654639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.654642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:24.654713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:24.654720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.654734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.654751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.654773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:24.654797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341144214637384:4201]
2025-10-09T21:44:24.654804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341144214637384:4201]
2025-10-09T21:44:24.654806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341144214637384:4201]
2025-10-09T21:44:24.654814Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:24.654833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:24.654851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:24.654859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:24.654864Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:24.654868Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:24.654871Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:24.654898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341144214637384:4201]
2025-10-09T21:44:24.654901Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:24.654906Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:24.654912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341144576448074:4521], now have 1 active actors on pipe
2025-10-09T21:44:24.654916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341144576447955:2432], now have 1 active actors on pipe
2025-10-09T21:44:24.657639Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 298
2025-10-09T21:44:24.657678Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 298
2025-10-09T21:44:24.658649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:24.658672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:24.658725Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c7010bb9-68afaf5b-28846f6b-773ab5b6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:24.658751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.658761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.658765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:24.658780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.658803Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:24.658829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:24.666789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:24.666824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:24.666882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:24.667054Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 299 partition: 0 MaxSeqNo: 25 sessionId: src_id|c7010bb9-68afaf5b-28846f6b-773ab5b6_0
2025-10-09T21:44:24.667551Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf83f8ab-8eb1c9d-cfcbc6d4-3228203d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c7010bb9-68afaf5b-28846f6b-773ab5b6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:24.670000Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c7010bb9-68afaf5b-28846f6b-773ab5b6_0] Write session: try to update token
2025-10-09T21:44:24.670022Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c7010bb9-68afaf5b-28846f6b-773ab5b6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:24.683213Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:24.683234Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:24.683237Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:24.683239Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:24.683250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341144214637384:4201]
2025-10-09T21:44:24.683254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341144214637384:4201]
2025-10-09T21:44:24.683258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341144214637384:4201]
2025-10-09T21:44:24.683312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:24.683327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.683330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:24.683501Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:24.686525Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:24.686539Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:24.686543Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:24.696570Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 299 sessionId: src_id|c7010bb9-68afaf5b-28846f6b-773ab5b6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:24.699119Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:24.723941Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341144576448084:3810]
2025-10-09T21:44:24.724269Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 298, pipe: [1:7559341144576448084:3810]
2025-10-09T21:44:24.884449Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:24.884920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:24.884932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:24.884982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341144576448084:3810], now have 1 active actors on pipe
2025-10-09T21:44:24.885312Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:24.885389Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:24.885406Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:24.885466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 299 reason: error at write: Disconnected sessionId: src_id|c7010bb9-68afaf5b-28846f6b-773ab5b6_0
2025-10-09T21:44:24.885647Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 299 sessionId: src_id|c7010bb9-68afaf5b-28846f6b-773ab5b6_0 is DEAD
2025-10-09T21:44:24.885650Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:24.885692Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341144576448084:3810]
2025-10-09T21:44:24.886175Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7010bb9-68afaf5b-28846f6b-773ab5b6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:24.886189Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7010bb9-68afaf5b-28846f6b-773ab5b6_0] Write session will restart in 0.005130s
2025-10-09T21:44:24.886223Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7010bb9-68afaf5b-28846f6b-773ab5b6_0] Write session: Do CDS request
2025-10-09T21:44:24.886231Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7010bb9-68afaf5b-28846f6b-773ab5b6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:24.892616Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c7010bb9-68afaf5b-28846f6b-773ab5b6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:24.894460Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:24.894479Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 300
2025-10-09T21:44:24.894809Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 300 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:24.894848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 300 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:24.894853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=300 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:24.894857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:24.895582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 300 sessionId:  describe result for acl check
2025-10-09T21:44:24.895624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:24.895629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:24.895631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:24.895637Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341144576448111:4525] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:24.896216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341144576448111:4525] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:24.903400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341144576448111:4525] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:24.909222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341144576448111:4525] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:24.909243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341144576448111:4525] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:24.909248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341144576448111:4525] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:24.909258Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 300 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:24.920539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:24.925036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:24.925313Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:24.925325Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:24.925339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:24.925407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:24.925449Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341144214637514:4203]
2025-10-09T21:44:24.925984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:24.930226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:24.930315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:24.930917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:24.931006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:24.931008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:24.931009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:24.931011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:24.931012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:24.931014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:24.931015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.931016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.931018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.931026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:24.931037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:24.931041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:24.931045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:24.931048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:24.931052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:24.931055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.931058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.931062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.931071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:24.931079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:24.931085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:24.931091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:24.931096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:24.931101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:24.931105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:24.931110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:24.931115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:24.931120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:24.931124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:24.931130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:24.931135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.931141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.931146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.931152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.931160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.931166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.931198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:24.944814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:24.946202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:24.946207Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:24.946209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:24.946214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:24.946222Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 299 [2:7559341144214637514:4203]
2025-10-09T21:44:24.946237Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:24.946263Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:24.946266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.946269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.946271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.946290Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:24.946297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:24.946305Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:24.946364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.946366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.946368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.946370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:24.946469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:24.946471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.946483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:24.946494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:24.946508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:24.946532Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341144214637509:4203]
2025-10-09T21:44:24.946534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341144214637509:4203]
2025-10-09T21:44:24.946536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341144214637509:4203]
2025-10-09T21:44:24.946544Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:24.946557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:24.946571Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:24.946574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:24.946593Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:24.946597Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:24.946602Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:24.946617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341144214637509:4203]
2025-10-09T21:44:24.946623Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:24.946625Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:24.946635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341144576448199:4525], now have 1 active actors on pipe
2025-10-09T21:44:24.946653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341144576448091:2432], now have 1 active actors on pipe
2025-10-09T21:44:24.946829Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 299
2025-10-09T21:44:24.946879Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 299
2025-10-09T21:44:24.947015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:24.947030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:24.947062Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d52c2f2b-bc2bd020-9677f747-51588e20_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:24.947071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.947075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.947077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:24.947091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.947103Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:24.947120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:24.948977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7010bb9-68afaf5b-28846f6b-773ab5b6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d52c2f2b-bc2bd020-9677f747-51588e20_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:24.948305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:24.948322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:24.948350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:24.948608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 300 partition: 0 MaxSeqNo: 25 sessionId: src_id|d52c2f2b-bc2bd020-9677f747-51588e20_0
2025-10-09T21:44:24.951910Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d52c2f2b-bc2bd020-9677f747-51588e20_0] Write session: try to update token
2025-10-09T21:44:24.951931Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d52c2f2b-bc2bd020-9677f747-51588e20_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:24.982229Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:24.982249Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:24.982251Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:24.982253Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:24.982263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341144214637509:4203]
2025-10-09T21:44:24.982267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341144214637509:4203]
2025-10-09T21:44:24.982271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341144214637509:4203]
2025-10-09T21:44:24.982317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:24.982326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:24.982329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:24.982479Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:24.986125Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:24.986141Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:24.986145Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:24.985354Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 300 sessionId: src_id|d52c2f2b-bc2bd020-9677f747-51588e20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:24.986083Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341144576448227:3810]
2025-10-09T21:44:24.987142Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 299, pipe: [1:7559341144576448227:3810]
2025-10-09T21:44:24.987989Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:25.217144Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:25.217479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341144576448227:3810], now have 1 active actors on pipe
2025-10-09T21:44:25.217511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:25.217517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:25.218259Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:25.220413Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:25.220467Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341144576448227:3810]
2025-10-09T21:44:25.220569Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:25.220643Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 300 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d52c2f2b-bc2bd020-9677f747-51588e20_0
2025-10-09T21:44:25.220832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 300 sessionId: src_id|d52c2f2b-bc2bd020-9677f747-51588e20_0 is DEAD
2025-10-09T21:44:25.221679Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d52c2f2b-bc2bd020-9677f747-51588e20_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:25.221691Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d52c2f2b-bc2bd020-9677f747-51588e20_0] Write session will restart in 0.009054s
2025-10-09T21:44:25.221717Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d52c2f2b-bc2bd020-9677f747-51588e20_0] Write session: Do CDS request
2025-10-09T21:44:25.221725Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d52c2f2b-bc2bd020-9677f747-51588e20_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:25.231768Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d52c2f2b-bc2bd020-9677f747-51588e20_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:25.232097Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:25.232116Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 301
2025-10-09T21:44:25.232372Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 301 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:25.232414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 301 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:25.232418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=301 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:25.232422Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:25.233411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 301 sessionId:  describe result for acl check
2025-10-09T21:44:25.233455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:25.233457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:25.233459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:25.233465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341148871415555:4536] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:25.234005Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341148871415555:4536] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:25.251192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:25.251421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341148871415555:4536] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:25.252732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:25.252932Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:25.252935Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:25.252947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:25.252995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:25.253028Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341148509604949:4213]
2025-10-09T21:44:25.253595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:25.254105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:25.254146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:25.254752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:25.254845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:25.254847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:25.254848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:25.254849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:25.254850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:25.254852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:25.254853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.254853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.254854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.254861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:25.254870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:25.254873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:25.254877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:25.254880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:25.254883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:25.254886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.254889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.254892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.254899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:25.254907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:25.254912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:25.254917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:25.254922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:25.254927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:25.254931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:25.254935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:25.254939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:25.254944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:25.254948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:25.254952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:25.254956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.254961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.254964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.254969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.254974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.254978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.255015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:25.271682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341148871415555:4536] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:25.271702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341148871415555:4536] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:25.271706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341148871415555:4536] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:25.271716Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 301 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:25.273905Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 300
2025-10-09T21:44:25.273933Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 300
2025-10-09T21:44:25.272133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:25.273296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:25.273302Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:25.273304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:25.273309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:25.273316Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 300 [2:7559341148509604949:4213]
2025-10-09T21:44:25.273330Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:25.273350Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:25.273353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.273356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.273358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.273376Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:25.273383Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:25.273390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:25.273446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.273448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.273449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.273452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:25.273530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:25.273533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.273547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.273558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.273574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:25.273598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341148871415667:4536], now have 1 active actors on pipe
2025-10-09T21:44:25.273602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341148871415541:2432], now have 1 active actors on pipe
2025-10-09T21:44:25.273610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341148509604944:4213]
2025-10-09T21:44:25.273612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341148509604944:4213]
2025-10-09T21:44:25.273614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341148509604944:4213]
2025-10-09T21:44:25.273629Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:25.273644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:25.273657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:25.273660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:25.273701Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341148509604944:4213]
2025-10-09T21:44:25.273703Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:25.275621Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:25.275632Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:25.275635Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:25.275641Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:25.275887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:25.275897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:25.275944Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ec2e1e46-ad0dee80-aa9fa76d-14c5716b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:25.275956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.275961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.275963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:25.275974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.275997Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:25.276019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:25.276982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:25.276992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:25.277011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:25.277853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 301 partition: 0 MaxSeqNo: 25 sessionId: src_id|ec2e1e46-ad0dee80-aa9fa76d-14c5716b_0
2025-10-09T21:44:25.278462Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d52c2f2b-bc2bd020-9677f747-51588e20_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ec2e1e46-ad0dee80-aa9fa76d-14c5716b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:25.281040Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ec2e1e46-ad0dee80-aa9fa76d-14c5716b_0] Write session: try to update token
2025-10-09T21:44:25.281061Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ec2e1e46-ad0dee80-aa9fa76d-14c5716b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:25.300812Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:25.300834Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:25.300837Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:25.300839Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:25.300848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341148509604944:4213]
2025-10-09T21:44:25.300852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341148509604944:4213]
2025-10-09T21:44:25.300857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341148509604944:4213]
2025-10-09T21:44:25.300902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:25.300910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.300913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:25.301048Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:25.304829Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:25.304847Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:25.304852Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:25.319941Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341148871415679:3810]
2025-10-09T21:44:25.321420Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 300, pipe: [1:7559341148871415679:3810]
2025-10-09T21:44:25.328417Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 301 sessionId: src_id|ec2e1e46-ad0dee80-aa9fa76d-14c5716b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:25.331303Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:25.527483Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:25.527817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341148871415679:3810], now have 1 active actors on pipe
2025-10-09T21:44:25.527873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:25.527879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:25.528375Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341148871415679:3810]
2025-10-09T21:44:25.528406Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:25.528408Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:25.528470Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 301 reason: error at write: Disconnected sessionId: src_id|ec2e1e46-ad0dee80-aa9fa76d-14c5716b_0
2025-10-09T21:44:25.528331Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:25.529199Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec2e1e46-ad0dee80-aa9fa76d-14c5716b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:25.529210Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec2e1e46-ad0dee80-aa9fa76d-14c5716b_0] Write session will restart in 0.005181s
2025-10-09T21:44:25.529236Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec2e1e46-ad0dee80-aa9fa76d-14c5716b_0] Write session: Do CDS request
2025-10-09T21:44:25.529243Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec2e1e46-ad0dee80-aa9fa76d-14c5716b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:25.529495Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:44:25.529573Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:44:25.529577Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:44:25.529583Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.006820s
2025-10-09T21:44:25.528650Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 301 sessionId: src_id|ec2e1e46-ad0dee80-aa9fa76d-14c5716b_0 is DEAD
2025-10-09T21:44:25.528790Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:25.528813Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:44:25.528893Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 3 consumer shared/user session shared/user_1_3_11984607147350421102_v1 is DEAD
2025-10-09T21:44:25.529288Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340989957608036:3804] disconnected.
2025-10-09T21:44:25.529294Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340989957608036:3804] disconnected; active server actors: 1
2025-10-09T21:44:25.529298Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340989957608036:3804] client user disconnected session shared/user_1_3_11984607147350421102_v1
2025-10-09T21:44:25.535588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ec2e1e46-ad0dee80-aa9fa76d-14c5716b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:25.536856Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:25.536877Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 302
2025-10-09T21:44:25.538409Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 302 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:25.538449Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 302 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:25.538455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=302 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:25.538459Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:25.539115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 302 sessionId:  describe result for acl check
2025-10-09T21:44:25.539169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:25.539171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:25.539173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:25.539179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341148871415692:4542] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:25.539879Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:44:25.542138Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_4_257779208761094781_v1
2025-10-09T21:44:25.539760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341148871415692:4542] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:25.540872Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:44:25.540875Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 4
2025-10-09T21:44:25.541206Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 4 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:44:25.541268Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:44:25.541385Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 auth for : user
2025-10-09T21:44:25.541630Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 Handle describe topics response
2025-10-09T21:44:25.541648Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 auth is DEAD
2025-10-09T21:44:25.541662Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:44:25.541905Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:44:25.542631Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341148871415703:4545] connected; active server actors: 1
2025-10-09T21:44:25.542752Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559341148871415703:4545] session shared/user_1_4_257779208761094781_v1
2025-10-09T21:44:25.542763Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:44:25.542774Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:44:25.542781Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_4_257779208761094781_v1" (Sender=[1:7559341148871415698:4545], Pipe=[1:7559341148871415703:4545], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:44:25.542786Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:44:25.542797Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:44:25.542806Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_4_257779208761094781_v1" (Sender=[1:7559341148871415698:4545], Pipe=[1:7559341148871415703:4545], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:44:25.542816Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_4_257779208761094781_v1" sender [1:7559341148871415698:4545] lock partition 0 for ReadingSession "shared/user_1_4_257779208761094781_v1" (Sender=[1:7559341148871415698:4545], Pipe=[1:7559341148871415703:4545], Partitions=[], ActiveFamilyCount=1) generation 1 step 4
2025-10-09T21:44:25.542825Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:44:25.542830Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000028s
2025-10-09T21:44:25.543071Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 4 Session: "shared/user_1_4_257779208761094781_v1" ClientId: "user" PipeClient { RawX1: 7559341148871415703 RawX2: 4503603922342337 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:44:25.543222Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:44:25.550250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341148871415692:4542] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:25.575746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341148871415692:4542] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:25.575765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341148871415692:4542] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:25.575769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341148871415692:4542] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:25.575778Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 302 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:25.577354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:25.581434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:25.581718Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:25.581731Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:25.581747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:25.581812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:25.581853Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341148509605100:4221]
2025-10-09T21:44:25.582457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:25.583148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:25.583203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:25.583543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:25.583602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:25.583612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:25.583613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:25.583615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:25.583616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:25.583617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:25.583618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.583619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.583621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.583626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:25.583637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:25.583640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:25.583644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:25.583648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:25.583651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:25.583655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.583658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.583661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.583669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:25.583677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:25.583685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:25.583691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:25.583696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:25.583702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:25.583707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:25.583712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:25.583716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:25.583722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:25.583727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:25.583743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:25.583754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.583760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.583764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.583769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.583773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.583778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.583805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:25.599793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:25.601294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:25.601301Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:25.601303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:25.601308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:25.601316Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 301 [2:7559341148509605100:4221]
2025-10-09T21:44:25.601331Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:25.601352Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:25.601355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.601358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.601361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.601381Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:25.601387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:25.601394Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:25.601463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.601465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.601468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.601471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:25.601547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:25.601549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.601566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.601578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.601595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:25.601616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341148509605093:4221]
2025-10-09T21:44:25.601619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341148509605093:4221]
2025-10-09T21:44:25.601621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341148509605093:4221]
2025-10-09T21:44:25.601636Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:25.601653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:25.601667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:25.601670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:25.601711Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341148509605093:4221]
2025-10-09T21:44:25.601714Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:25.601960Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:25.601965Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:25.601968Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:25.601973Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:25.602317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341148871415810:4542], now have 1 active actors on pipe
2025-10-09T21:44:25.602322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341148871415685:2432], now have 1 active actors on pipe
2025-10-09T21:44:25.602325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341148871415713:4549], now have 1 active actors on pipe
2025-10-09T21:44:25.605161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 301
2025-10-09T21:44:25.605212Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 301
2025-10-09T21:44:25.605253Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 301, pipe: [1:7559341148871415713:4549]
2025-10-09T21:44:25.606369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:25.606403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:25.606420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_4_257779208761094781_v1 on pipe: [1:7559341148871415713:4549]
2025-10-09T21:44:25.606456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:25.606458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:25.606492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.606501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.606505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:25.606534Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_4_257779208761094781_v1
2025-10-09T21:44:25.606538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.606589Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e34c862d-8206ade1-31182439-973d200c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:25.606596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:44:25.606605Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:44:25.606783Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_4_257779208761094781_v1:1 with generation 301
2025-10-09T21:44:25.617701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:44:25.617746Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:44:25.617760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.617768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.617771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:25.617786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.617805Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:25.617821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:44:25.617843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:44:25.617865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:25.618130Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046265600 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:44:25.618155Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:44:25.618228Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 sending to client partition status
2025-10-09T21:44:25.619136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:25.619157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:25.619202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:25.620025Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec2e1e46-ad0dee80-aa9fa76d-14c5716b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e34c862d-8206ade1-31182439-973d200c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:25.619699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 302 partition: 0 MaxSeqNo: 25 sessionId: src_id|e34c862d-8206ade1-31182439-973d200c_0
2025-10-09T21:44:25.622685Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e34c862d-8206ade1-31182439-973d200c_0] Write session: try to update token
2025-10-09T21:44:25.622704Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e34c862d-8206ade1-31182439-973d200c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:25.630401Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:25.630421Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:25.630423Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:25.630426Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:25.630440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341148509605093:4221]
2025-10-09T21:44:25.630446Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341148509605093:4221]
2025-10-09T21:44:25.630449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341148509605093:4221]
2025-10-09T21:44:25.630497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:25.630506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.630509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:25.630537Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:25.630543Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:25.630550Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:25.630725Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:25.645649Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 302 sessionId: src_id|e34c862d-8206ade1-31182439-973d200c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:25.648549Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:25.841654Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:25.842134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:25.842145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:25.842643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_4_257779208761094781_v1
2025-10-09T21:44:25.842663Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:25.843012Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_4_257779208761094781_v1
2025-10-09T21:44:25.843154Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:25.843242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 302 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e34c862d-8206ade1-31182439-973d200c_0
2025-10-09T21:44:25.843443Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 302 sessionId: src_id|e34c862d-8206ade1-31182439-973d200c_0 is DEAD
2025-10-09T21:44:25.843579Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:25.843614Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341148871415713:4549]
2025-10-09T21:44:25.846404Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e34c862d-8206ade1-31182439-973d200c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:25.846416Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e34c862d-8206ade1-31182439-973d200c_0] Write session will restart in 0.008319s
2025-10-09T21:44:25.846445Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e34c862d-8206ade1-31182439-973d200c_0] Write session: Do CDS request
2025-10-09T21:44:25.846451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e34c862d-8206ade1-31182439-973d200c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:25.856352Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e34c862d-8206ade1-31182439-973d200c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:25.859577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:25.859594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 303
2025-10-09T21:44:25.859793Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 303 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:25.859817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 303 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:25.859820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=303 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:25.859825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:25.860195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 303 sessionId:  describe result for acl check
2025-10-09T21:44:25.860219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:25.860221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:25.860223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:25.860227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341148871415842:4550] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:25.860703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341148871415842:4550] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:25.866410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341148871415842:4550] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:25.873133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341148871415842:4550] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:25.873153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341148871415842:4550] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:25.873157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341148871415842:4550] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:25.873166Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 303 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:25.881696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:25.882247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:25.882626Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:25.882630Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:25.882646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:25.882712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:25.882757Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341148509605227:4223]
2025-10-09T21:44:25.883337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:25.885380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:25.885448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:25.885840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:25.885896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:25.885897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:25.885898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:25.885900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:25.885901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:25.885902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:25.885903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.885903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.885904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.885910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:25.885920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:25.885924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:25.885928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:25.885931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:25.885934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:25.885938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.885941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.885945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.885954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:25.885962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:25.885969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:25.885974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:25.885982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:25.885987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:25.885991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:25.885996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:25.886002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:25.886007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:25.886011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:25.886016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:25.886021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.886046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.886051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.886056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.886061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.886066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.886098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:25.895817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:25.897241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:25.897281Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:25.897283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:25.897288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:25.897296Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 302 [2:7559341148509605227:4223]
2025-10-09T21:44:25.897312Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:25.897335Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:25.897339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.897343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.897345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.897367Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:25.897374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:25.897383Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:25.897453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.897463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.897466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.897469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:25.897553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:25.897556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.897571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:25.897583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:25.897599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:25.897624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341148509605220:4223]
2025-10-09T21:44:25.897626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341148509605220:4223]
2025-10-09T21:44:25.897629Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341148509605220:4223]
2025-10-09T21:44:25.897646Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:25.897664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:25.897679Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:25.897683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:25.897728Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341148509605220:4223]
2025-10-09T21:44:25.897731Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:25.898523Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:25.898533Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:25.898535Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:25.898542Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:25.899323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341148871415943:4550], now have 1 active actors on pipe
2025-10-09T21:44:25.899329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341148871415827:2432], now have 1 active actors on pipe
2025-10-09T21:44:25.899453Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 302
2025-10-09T21:44:25.899489Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 302
2025-10-09T21:44:25.902365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:25.902399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:25.902448Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8edaff9e-3d16c26e-8394787-57e54aaf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:25.902462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.902477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.902481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:25.902496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.902525Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:25.902558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:25.905620Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 303 partition: 0 MaxSeqNo: 25 sessionId: src_id|8edaff9e-3d16c26e-8394787-57e54aaf_0
2025-10-09T21:44:25.906699Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e34c862d-8206ade1-31182439-973d200c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8edaff9e-3d16c26e-8394787-57e54aaf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:25.905393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:25.905414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:25.905453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:25.914183Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8edaff9e-3d16c26e-8394787-57e54aaf_0] Write session: try to update token
2025-10-09T21:44:25.914211Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8edaff9e-3d16c26e-8394787-57e54aaf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:25.931464Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:25.931486Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:25.931489Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:25.931491Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:25.931521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341148509605220:4223]
2025-10-09T21:44:25.931527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341148509605220:4223]
2025-10-09T21:44:25.931531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341148509605220:4223]
2025-10-09T21:44:25.931585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:25.931601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:25.931608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:25.931757Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:25.934659Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:25.934679Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:25.934686Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:25.944675Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341148871415963:4549]
2025-10-09T21:44:25.945261Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 302, pipe: [1:7559341148871415963:4549]
2025-10-09T21:44:25.960930Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 303 sessionId: src_id|8edaff9e-3d16c26e-8394787-57e54aaf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:25.963767Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:26.121223Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:26.121722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341148871415963:4549], now have 1 active actors on pipe
2025-10-09T21:44:26.121784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:26.121792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:26.122652Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:26.122669Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:26.122734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 303 reason: error at write: Disconnected sessionId: src_id|8edaff9e-3d16c26e-8394787-57e54aaf_0
2025-10-09T21:44:26.122927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 303 sessionId: src_id|8edaff9e-3d16c26e-8394787-57e54aaf_0 is DEAD
2025-10-09T21:44:26.123062Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:26.123095Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341148871415963:4549]
2025-10-09T21:44:26.123468Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8edaff9e-3d16c26e-8394787-57e54aaf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:26.123478Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8edaff9e-3d16c26e-8394787-57e54aaf_0] Write session will restart in 0.005449s
2025-10-09T21:44:26.123503Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8edaff9e-3d16c26e-8394787-57e54aaf_0] Write session: Do CDS request
2025-10-09T21:44:26.123514Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8edaff9e-3d16c26e-8394787-57e54aaf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:26.123869Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:26.130745Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8edaff9e-3d16c26e-8394787-57e54aaf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:26.131218Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:26.131235Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 304
2025-10-09T21:44:26.131516Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 304 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:26.131554Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 304 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:26.131560Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=304 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:26.131565Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:26.132446Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 304 sessionId:  describe result for acl check
2025-10-09T21:44:26.132479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:26.132481Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:26.132483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:26.132488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341153166383278:4555] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:26.133090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341153166383278:4555] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:26.141165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341153166383278:4555] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:26.155688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341153166383278:4555] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:26.155710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341153166383278:4555] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:26.155714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341153166383278:4555] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:26.155729Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 304 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:26.165899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:26.166450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:26.166670Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:26.166673Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:26.166687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:26.166737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:26.166777Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341152804572650:4226]
2025-10-09T21:44:26.167394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:26.168024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:26.168080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:26.168425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:26.168496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:26.168497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:26.168499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:26.168500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:26.168502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:26.168503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:26.168504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.168505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.168507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.168513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:26.168523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:26.168526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:26.168530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:26.168533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:26.168537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:26.168540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.168544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.168547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.168556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:26.168564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:26.168571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:26.168576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:26.168581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:26.168586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:26.168590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:26.168594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:26.168605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:26.168610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:26.168614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:26.168621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:26.168625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.168630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.168633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.168637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.168644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.168648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.168677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:26.181413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:26.182672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:26.182701Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:26.182703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:26.182708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:26.182716Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 303 [2:7559341152804572650:4226]
2025-10-09T21:44:26.182733Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:26.182755Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:26.182759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:26.182762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:26.182765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:26.182785Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:26.182791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:26.182799Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:26.182874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.182877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.182879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.182882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:26.182960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:26.182964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.182978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.182989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.183005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:26.183026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341152804572644:4226]
2025-10-09T21:44:26.183029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341152804572644:4226]
2025-10-09T21:44:26.183031Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341152804572644:4226]
2025-10-09T21:44:26.183047Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:26.183064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:26.183080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:26.183084Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:26.183133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341152804572644:4226]
2025-10-09T21:44:26.183135Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:26.183606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:26.183608Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:26.183610Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:26.183618Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:26.184095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341153166383367:4555], now have 1 active actors on pipe
2025-10-09T21:44:26.184112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341153166383265:2432], now have 1 active actors on pipe
2025-10-09T21:44:26.184451Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 303
2025-10-09T21:44:26.184486Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 303
2025-10-09T21:44:26.184925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:26.184942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:26.184977Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9700e0c2-9d16ea7f-cb170fed-1965a509_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:26.184994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:26.184999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:26.185003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:26.185017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:26.185901Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8edaff9e-3d16c26e-8394787-57e54aaf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9700e0c2-9d16ea7f-cb170fed-1965a509_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:26.188607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9700e0c2-9d16ea7f-cb170fed-1965a509_0] Write session: try to update token
2025-10-09T21:44:26.188628Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9700e0c2-9d16ea7f-cb170fed-1965a509_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:26.185454Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 304 partition: 0 MaxSeqNo: 25 sessionId: src_id|9700e0c2-9d16ea7f-cb170fed-1965a509_0
2025-10-09T21:44:26.185034Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:26.185059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:26.185303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:26.185308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:26.185325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:26.223076Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341153166383396:4549]
2025-10-09T21:44:26.223631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341153166383396:4549], now have 1 active actors on pipe
2025-10-09T21:44:26.223913Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 303, pipe: [1:7559341153166383396:4549]
2025-10-09T21:44:26.237269Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 304 sessionId: src_id|9700e0c2-9d16ea7f-cb170fed-1965a509_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:26.241069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:26.241114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:26.241138Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:26.241141Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:26.241144Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:26.241147Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:26.241156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341152804572644:4226]
2025-10-09T21:44:26.241160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341152804572644:4226]
2025-10-09T21:44:26.241167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341152804572644:4226]
2025-10-09T21:44:26.241231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:26.241238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.241240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:26.241382Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:26.244624Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:26.244631Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:26.244643Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:26.239874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:26.452784Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:26.453402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:26.454524Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:26.454603Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:26.454605Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:26.454670Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 304 reason: error at write: Disconnected sessionId: src_id|9700e0c2-9d16ea7f-cb170fed-1965a509_0
2025-10-09T21:44:26.454823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 304 sessionId: src_id|9700e0c2-9d16ea7f-cb170fed-1965a509_0 is DEAD
2025-10-09T21:44:26.454970Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341153166383396:4549]
2025-10-09T21:44:26.454984Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:26.459971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9700e0c2-9d16ea7f-cb170fed-1965a509_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:26.459980Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9700e0c2-9d16ea7f-cb170fed-1965a509_0] Write session will restart in 0.007932s
2025-10-09T21:44:26.460007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9700e0c2-9d16ea7f-cb170fed-1965a509_0] Write session: Do CDS request
2025-10-09T21:44:26.460014Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9700e0c2-9d16ea7f-cb170fed-1965a509_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:26.474506Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9700e0c2-9d16ea7f-cb170fed-1965a509_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:26.475116Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:26.475134Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 305
2025-10-09T21:44:26.475326Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 305 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:26.475359Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 305 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:26.475365Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=305 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:26.475369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:26.476346Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 305 sessionId:  describe result for acl check
2025-10-09T21:44:26.476386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:26.476388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:26.476390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:26.476395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341153166383461:4567] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:26.476958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341153166383461:4567] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:26.489123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:26.492757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:26.492978Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:26.492982Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:26.492993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:26.493044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:26.493082Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341152804572817:4241]
2025-10-09T21:44:26.493666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:26.494393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:26.494474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:26.494833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:26.495051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:26.495060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:26.495061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:26.495062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:26.495064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:26.495065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:26.495066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.495067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.495068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.495075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:26.495086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:26.495090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:26.495093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:26.495097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:26.495101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:26.495114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.495118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.495122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.495131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:26.495140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:26.495148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:26.495153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:26.495158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:26.495163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:26.495168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:26.495173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:26.495178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:26.495183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:26.495187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:26.495192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:26.495200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.495205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.495210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.495215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.495219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.495225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.495254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:26.500056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341153166383461:4567] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:26.508670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341153166383461:4567] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:26.508694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341153166383461:4567] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:26.508698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341153166383461:4567] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:26.508709Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 305 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:26.516713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:26.518078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:26.518104Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:26.518108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:26.518115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:26.518126Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 304 [2:7559341152804572817:4241]
2025-10-09T21:44:26.518144Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:26.518170Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:26.518187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:26.518198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:26.518201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:26.518224Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:26.518238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:26.518254Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:26.518349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.518365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.518367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.518370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:26.518481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:26.518495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.518513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.518534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.518559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:26.518589Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341152804572811:4241]
2025-10-09T21:44:26.518600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341152804572811:4241]
2025-10-09T21:44:26.518603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341152804572811:4241]
2025-10-09T21:44:26.518617Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:26.518642Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:26.518646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:26.518649Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:26.518652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:26.518672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:26.518683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:26.518699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341153166383553:4567], now have 1 active actors on pipe
2025-10-09T21:44:26.518761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341153166383426:2432], now have 1 active actors on pipe
2025-10-09T21:44:26.518772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341152804572811:4241]
2025-10-09T21:44:26.518792Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:26.519025Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 304
2025-10-09T21:44:26.519058Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 304
2025-10-09T21:44:26.519352Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:26.521306Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9700e0c2-9d16ea7f-cb170fed-1965a509_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cc56e4b9-b6e58adf-745a2902-daefcd84_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:26.520872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 305 partition: 0 MaxSeqNo: 25 sessionId: src_id|cc56e4b9-b6e58adf-745a2902-daefcd84_0
2025-10-09T21:44:26.524239Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc56e4b9-b6e58adf-745a2902-daefcd84_0] Write session: try to update token
2025-10-09T21:44:26.524259Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc56e4b9-b6e58adf-745a2902-daefcd84_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:26.520061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:26.520076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:26.520127Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cc56e4b9-b6e58adf-745a2902-daefcd84_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:26.520140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:26.520145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:26.520148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:26.520161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:26.520188Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:26.520209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:26.520706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:26.520711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:26.520731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:26.554847Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341153166383560:4549]
2025-10-09T21:44:26.557395Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 305 sessionId: src_id|cc56e4b9-b6e58adf-745a2902-daefcd84_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:26.560143Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:26.561230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:26.561260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:26.561285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341153166383560:4549], now have 1 active actors on pipe
2025-10-09T21:44:26.561331Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 304, pipe: [1:7559341153166383560:4549]
2025-10-09T21:44:26.562694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:26.562730Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:26.562733Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:26.562735Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:26.562737Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:26.562746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341152804572811:4241]
2025-10-09T21:44:26.562752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341152804572811:4241]
2025-10-09T21:44:26.562755Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341152804572811:4241]
2025-10-09T21:44:26.562797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:26.562810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.562813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:26.562946Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:26.566847Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:26.566854Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:26.566859Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:26.568165Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:26.889441Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:26.898682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:26.898712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:26.898893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:26.898963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:26.899031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:26.899104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:26.899313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:26.899326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:26.899398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:26.899467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:26.899527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:26.899587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:26.899635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:26.899638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:26.899702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:26.899759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:26.899820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:26.899881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:26.899921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:26.899924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:26.899999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:26.900060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:26.900129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:26.900196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:26.900235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:26.900238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:26.900304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:26.900360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:26.900424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:26.900483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:26.900530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:26.900533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:26.900614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:26.900673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:26.900732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:26.900796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:26.900835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:26.900838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:26.900905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:26.900963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:26.901022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:26.901085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:26.901131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:26.901134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:26.901200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:26.901262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:26.901328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:26.901390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:26.901438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:26.901440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:26.901509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:26.901572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:26.901640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:26.901721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:26.901776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:26.901781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:26.901849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:26.901905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:26.901966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:26.902336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:26.902417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:26.902427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:26.902733Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341153166383560:4549]
2025-10-09T21:44:26.902773Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:26.902790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:26.902826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 305 reason: error at write: Disconnected sessionId: src_id|cc56e4b9-b6e58adf-745a2902-daefcd84_0
2025-10-09T21:44:26.903036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 305 sessionId: src_id|cc56e4b9-b6e58adf-745a2902-daefcd84_0 is DEAD
2025-10-09T21:44:26.903218Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:26.903273Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:26.906232Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc56e4b9-b6e58adf-745a2902-daefcd84_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:26.906245Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc56e4b9-b6e58adf-745a2902-daefcd84_0] Write session will restart in 0.007284s
2025-10-09T21:44:26.906272Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc56e4b9-b6e58adf-745a2902-daefcd84_0] Write session: Do CDS request
2025-10-09T21:44:26.906279Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc56e4b9-b6e58adf-745a2902-daefcd84_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:26.914755Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc56e4b9-b6e58adf-745a2902-daefcd84_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:26.915095Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:26.915115Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 306
2025-10-09T21:44:26.915250Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 306 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:26.915285Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 306 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:26.915292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=306 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:26.915295Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:26.916384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 306 sessionId:  describe result for acl check
2025-10-09T21:44:26.916434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:26.916436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:26.916438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:26.916444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341153166383651:4574] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:26.917174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341153166383651:4574] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:26.920583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341153166383651:4574] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:26.920059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:26.922046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:26.922299Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:26.922309Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:26.922323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:26.922408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:26.922464Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341152804572946:4243]
2025-10-09T21:44:26.923078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:26.923421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:26.923471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:26.923782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:26.923847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:26.923848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:26.923849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:26.923851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:26.923852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:26.923853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:26.923855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.923856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.923857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.923862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:26.923873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:26.923878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:26.923882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:26.923885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:26.923889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:26.923893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.923896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.923899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.923907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:26.923914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:26.923921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:26.923926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:26.923931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:26.923945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:26.923949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:26.923954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:26.923959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:26.923963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:26.923968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:26.923972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:26.923978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.923983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.923988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.923994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.924002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.924007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.924037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:26.925012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341153166383651:4574] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:26.925030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341153166383651:4574] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:26.925035Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341153166383651:4574] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:26.925045Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 306 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:26.931486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:26.932962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:26.932982Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:26.932985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:26.932990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:26.932998Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 305 [2:7559341152804572946:4243]
2025-10-09T21:44:26.933017Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:26.933040Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:26.933043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:26.933047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:26.933049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:26.933071Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:26.933082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:26.933090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:26.933160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.933166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.933168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.933171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:26.933252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:26.933261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.933276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:26.933287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:26.933304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:26.933333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341153166383575:2432], now have 1 active actors on pipe
2025-10-09T21:44:26.933341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341153166383703:4574], now have 1 active actors on pipe
2025-10-09T21:44:26.933350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341152804572941:4243]
2025-10-09T21:44:26.933353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341152804572941:4243]
2025-10-09T21:44:26.933355Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341152804572941:4243]
2025-10-09T21:44:26.933370Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:26.933390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:26.933407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:26.933411Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:26.933419Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 305
2025-10-09T21:44:26.933451Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:26.933454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341152804572941:4243]
2025-10-09T21:44:26.933458Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:26.933465Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:26.933434Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 305
2025-10-09T21:44:26.933468Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:26.933475Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:26.933582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:26.933599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:26.933633Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2f37715f-f38a1571-ee2a14fc-d51fb57c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:26.933648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:26.933653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:26.933659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:26.933671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:26.933688Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:26.933713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:26.933945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:26.933958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:26.933979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:26.934740Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc56e4b9-b6e58adf-745a2902-daefcd84_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2f37715f-f38a1571-ee2a14fc-d51fb57c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:26.934123Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 306 partition: 0 MaxSeqNo: 25 sessionId: src_id|2f37715f-f38a1571-ee2a14fc-d51fb57c_0
2025-10-09T21:44:26.937318Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2f37715f-f38a1571-ee2a14fc-d51fb57c_0] Write session: try to update token
2025-10-09T21:44:26.937336Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2f37715f-f38a1571-ee2a14fc-d51fb57c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:26.950309Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:26.950333Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:26.950336Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:26.950338Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:26.950347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341152804572941:4243]
2025-10-09T21:44:26.950353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341152804572941:4243]
2025-10-09T21:44:26.950357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341152804572941:4243]
2025-10-09T21:44:26.950417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:26.950425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:26.950428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:26.950575Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:26.950997Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:26.951003Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:26.951007Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:26.980517Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 306 sessionId: src_id|2f37715f-f38a1571-ee2a14fc-d51fb57c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:26.983038Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:27.002822Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341157461351009:4549]
2025-10-09T21:44:27.003499Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 305, pipe: [1:7559341157461351009:4549]
2025-10-09T21:44:27.128057Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:27.128573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:27.128595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:27.129434Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:27.128620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341157461351009:4549], now have 1 active actors on pipe
2025-10-09T21:44:27.128794Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:27.129516Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 306 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2f37715f-f38a1571-ee2a14fc-d51fb57c_0
2025-10-09T21:44:27.129712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 306 sessionId: src_id|2f37715f-f38a1571-ee2a14fc-d51fb57c_0 is DEAD
2025-10-09T21:44:27.129893Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f37715f-f38a1571-ee2a14fc-d51fb57c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:27.129906Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f37715f-f38a1571-ee2a14fc-d51fb57c_0] Write session will restart in 0.008930s
2025-10-09T21:44:27.129932Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f37715f-f38a1571-ee2a14fc-d51fb57c_0] Write session: Do CDS request
2025-10-09T21:44:27.129939Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f37715f-f38a1571-ee2a14fc-d51fb57c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:27.129913Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:27.129953Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341157461351009:4549]
2025-10-09T21:44:27.141332Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2f37715f-f38a1571-ee2a14fc-d51fb57c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:27.141441Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:27.141460Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 307
2025-10-09T21:44:27.141649Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 307 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:27.141679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 307 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:27.141687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=307 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:27.141700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:27.142207Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 307 sessionId:  describe result for acl check
2025-10-09T21:44:27.142247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:27.142249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:27.142251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:27.142256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341157461351022:4578] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:27.142860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341157461351022:4578] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:27.145565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341157461351022:4578] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:27.150704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341157461351022:4578] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:27.150733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341157461351022:4578] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:27.150737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341157461351022:4578] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:27.150746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 307 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:27.160510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:27.163047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:27.163345Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:27.163361Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:27.163376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:27.163456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:27.163503Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341157099540373:4246]
2025-10-09T21:44:27.164129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:27.164973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:27.165040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:27.165441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:27.165522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:27.165529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:27.165530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:27.165532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:27.165533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:27.165534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:27.165535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.165537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.165538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.165544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:27.165556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:27.165559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:27.165563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:27.165568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:27.165573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:27.165577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.165582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.165586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.165595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:27.165604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:27.165611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:27.165616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:27.165623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:27.165628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:27.165632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:27.165649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:27.165655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:27.165661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:27.165666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:27.165672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:27.165677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.165683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.165687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.165693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.165698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.165704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.165732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:27.174732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:27.176762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:27.176775Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:27.176778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:27.176783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:27.176792Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 306 [2:7559341157099540373:4246]
2025-10-09T21:44:27.176810Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:27.176830Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:27.176837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:27.176841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:27.176843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:27.176862Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:27.176873Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:27.176881Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:27.176960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.176970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.176972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.176974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:27.177065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:27.177250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.177272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.177285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.177301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:27.177344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341157461351093:4578], now have 1 active actors on pipe
2025-10-09T21:44:27.177353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341157461351012:2432], now have 1 active actors on pipe
2025-10-09T21:44:27.177361Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341157099540368:4246]
2025-10-09T21:44:27.177369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341157099540368:4246]
2025-10-09T21:44:27.177371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341157099540368:4246]
2025-10-09T21:44:27.177388Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:27.177404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:27.177425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:27.177432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:27.177476Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:27.177481Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 306
2025-10-09T21:44:27.177487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:27.177520Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 306
2025-10-09T21:44:27.177489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:27.177489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341157099540368:4246]
2025-10-09T21:44:27.177493Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:27.177498Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:27.177638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:27.177651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:27.177682Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|73f50ebe-2706fd6f-70fa8166-74d9037c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:27.177695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:27.177700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:27.177702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:27.177734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:27.177751Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:27.177782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:27.178053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:27.178069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:27.178091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:27.178240Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 307 partition: 0 MaxSeqNo: 25 sessionId: src_id|73f50ebe-2706fd6f-70fa8166-74d9037c_0
2025-10-09T21:44:27.178652Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f37715f-f38a1571-ee2a14fc-d51fb57c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|73f50ebe-2706fd6f-70fa8166-74d9037c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:27.181244Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73f50ebe-2706fd6f-70fa8166-74d9037c_0] Write session: try to update token
2025-10-09T21:44:27.181265Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73f50ebe-2706fd6f-70fa8166-74d9037c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:27.198084Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:27.198105Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:27.198107Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:27.198109Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:27.198119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341157099540368:4246]
2025-10-09T21:44:27.198123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341157099540368:4246]
2025-10-09T21:44:27.198127Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341157099540368:4246]
2025-10-09T21:44:27.198189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:27.198196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.198199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:27.198357Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:27.202804Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:27.202822Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:27.202827Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:27.210585Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 307 sessionId: src_id|73f50ebe-2706fd6f-70fa8166-74d9037c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:27.213148Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:27.230106Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341157461351148:4549]
2025-10-09T21:44:27.230734Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 306, pipe: [1:7559341157461351148:4549]
2025-10-09T21:44:27.386219Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:27.386577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:27.386587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:27.386632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341157461351148:4549], now have 1 active actors on pipe
2025-10-09T21:44:27.387488Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:27.388183Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341157461351148:4549]
2025-10-09T21:44:27.388222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:27.388224Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:27.388275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 307 reason: error at write: Disconnected sessionId: src_id|73f50ebe-2706fd6f-70fa8166-74d9037c_0
2025-10-09T21:44:27.388467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 307 sessionId: src_id|73f50ebe-2706fd6f-70fa8166-74d9037c_0 is DEAD
2025-10-09T21:44:27.388630Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:27.389118Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73f50ebe-2706fd6f-70fa8166-74d9037c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:27.389145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73f50ebe-2706fd6f-70fa8166-74d9037c_0] Write session will restart in 0.007656s
2025-10-09T21:44:27.389173Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73f50ebe-2706fd6f-70fa8166-74d9037c_0] Write session: Do CDS request
2025-10-09T21:44:27.389179Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73f50ebe-2706fd6f-70fa8166-74d9037c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:27.398280Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73f50ebe-2706fd6f-70fa8166-74d9037c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:27.398883Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:27.398901Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 308
2025-10-09T21:44:27.399116Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 308 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:27.399160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 308 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:27.399165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=308 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:27.399170Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:27.401166Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 308 sessionId:  describe result for acl check
2025-10-09T21:44:27.401216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:27.401218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:27.401220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:27.401226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341157461351174:4588] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:27.401973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341157461351174:4588] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:27.411243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341157461351174:4588] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:27.419097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341157461351174:4588] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:27.419119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341157461351174:4588] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:27.419125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341157461351174:4588] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:27.419166Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 308 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:27.428514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:27.428695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:27.428945Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:27.428950Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:27.428966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:27.429032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:27.429439Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341157099540516:4255]
2025-10-09T21:44:27.429978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:27.430470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:27.430524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:27.430823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:27.430882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:27.430884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:27.430885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:27.430886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:27.430887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:27.430889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:27.430890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.430891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.430892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.430898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:27.430909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:27.430913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:27.430917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:27.430920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:27.430924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:27.430928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.430932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.430935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.430941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:27.430945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:27.430950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:27.430953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:27.430956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:27.430959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:27.430972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:27.430976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:27.430980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:27.430983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:27.430985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:27.430988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:27.430991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.430994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.430996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.430999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.431002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.431005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.431024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:27.453394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:27.454565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:27.454571Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:27.454573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:27.454577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:27.454584Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 307 [2:7559341157099540516:4255]
2025-10-09T21:44:27.454599Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:27.454618Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:27.454621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:27.454623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:27.454624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:27.454638Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:27.454642Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:27.454647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:27.454691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.454693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.454694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.454696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:27.454754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:27.454756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.454764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.454770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.454779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:27.454801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341157099540511:4255]
2025-10-09T21:44:27.454804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341157099540511:4255]
2025-10-09T21:44:27.454805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341157099540511:4255]
2025-10-09T21:44:27.454819Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:27.454833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:27.454849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:27.454852Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:27.454891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341157099540511:4255]
2025-10-09T21:44:27.454893Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:27.455188Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:27.455194Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:27.455196Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:27.455201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:27.455877Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 307
2025-10-09T21:44:27.455918Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 307
2025-10-09T21:44:27.457752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341157461351166:2432], now have 1 active actors on pipe
2025-10-09T21:44:27.457765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341157461351285:4588], now have 1 active actors on pipe
2025-10-09T21:44:27.459110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:27.459125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:27.459172Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ab01c898-1721fca1-a2785e8d-36567cb8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:27.459185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:27.459190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:27.459193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:27.459208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:27.459227Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:27.459248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:27.461120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:27.461131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:27.461156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:27.461335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 308 partition: 0 MaxSeqNo: 25 sessionId: src_id|ab01c898-1721fca1-a2785e8d-36567cb8_0
2025-10-09T21:44:27.461735Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73f50ebe-2706fd6f-70fa8166-74d9037c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ab01c898-1721fca1-a2785e8d-36567cb8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:27.464315Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ab01c898-1721fca1-a2785e8d-36567cb8_0] Write session: try to update token
2025-10-09T21:44:27.464339Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ab01c898-1721fca1-a2785e8d-36567cb8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:27.481997Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:27.482016Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:27.482018Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:27.482021Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:27.482045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341157099540511:4255]
2025-10-09T21:44:27.482051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341157099540511:4255]
2025-10-09T21:44:27.482055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341157099540511:4255]
2025-10-09T21:44:27.482101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:27.482109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.482111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:27.482245Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:27.484568Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:27.484577Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:27.484582Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:27.489727Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 308 sessionId: src_id|ab01c898-1721fca1-a2785e8d-36567cb8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:27.492192Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:27.492278Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341157461351311:4549]
2025-10-09T21:44:27.493631Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 307, pipe: [1:7559341157461351311:4549]
2025-10-09T21:44:27.673796Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:27.674511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:27.674528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:27.674566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341157461351311:4549], now have 1 active actors on pipe
2025-10-09T21:44:27.675852Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:27.676242Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341157461351311:4549]
2025-10-09T21:44:27.676272Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:27.676305Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:27.676374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 308 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ab01c898-1721fca1-a2785e8d-36567cb8_0
2025-10-09T21:44:27.676570Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 308 sessionId: src_id|ab01c898-1721fca1-a2785e8d-36567cb8_0 is DEAD
2025-10-09T21:44:27.676961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab01c898-1721fca1-a2785e8d-36567cb8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:27.676973Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab01c898-1721fca1-a2785e8d-36567cb8_0] Write session will restart in 0.008749s
2025-10-09T21:44:27.677009Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab01c898-1721fca1-a2785e8d-36567cb8_0] Write session: Do CDS request
2025-10-09T21:44:27.677017Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab01c898-1721fca1-a2785e8d-36567cb8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:27.686575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ab01c898-1721fca1-a2785e8d-36567cb8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:27.686965Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:27.686984Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 309
2025-10-09T21:44:27.687184Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 309 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:27.687218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 309 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:27.687224Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=309 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:27.687227Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:27.687614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 309 sessionId:  describe result for acl check
2025-10-09T21:44:27.687647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:27.687649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:27.687650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:27.687656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341157461351327:4594] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:27.688252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341157461351327:4594] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:27.693078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341157461351327:4594] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:27.708247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341157461351327:4594] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:27.708267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341157461351327:4594] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:27.708271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341157461351327:4594] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:27.708283Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 309 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:27.711157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:27.714321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:27.714645Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:27.714651Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:27.714667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:27.714747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:27.714789Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341157099540670:4263]
2025-10-09T21:44:27.715374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:27.718432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:27.718508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:27.719324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:27.719423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:27.719430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:27.719432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:27.719433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:27.719435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:27.719436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:27.719437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.719438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.719440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.719447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:27.719458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:27.719462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:27.719466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:27.719469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:27.719473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:27.719477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.719480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.719484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.719494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:27.719512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:27.719519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:27.719524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:27.719528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:27.719532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:27.719537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:27.719542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:27.719547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:27.719551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:27.719555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:27.719561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:27.719566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.719577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.719581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.719586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.719591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.719596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.719626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:27.730168Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 308
2025-10-09T21:44:27.730201Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 308
2025-10-09T21:44:27.728093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:27.729347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:27.729354Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:27.729357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:27.729362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:27.729372Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 308 [2:7559341157099540670:4263]
2025-10-09T21:44:27.729390Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:27.729413Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:27.729417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:27.729420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:27.729422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:27.729444Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:27.729451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:27.729459Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:27.729536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.729539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.729541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.729544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:27.729627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:27.729630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.729646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:27.729657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:27.729673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:27.729696Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341157099540665:4263]
2025-10-09T21:44:27.729701Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341157099540665:4263]
2025-10-09T21:44:27.729703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341157099540665:4263]
2025-10-09T21:44:27.729713Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:27.729730Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:27.729744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:27.729744Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:27.729748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:27.729748Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:27.729750Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:27.729929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341157461351429:4594], now have 1 active actors on pipe
2025-10-09T21:44:27.729935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341157099540665:4263]
2025-10-09T21:44:27.729946Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:27.729952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341157461351315:2432], now have 1 active actors on pipe
2025-10-09T21:44:27.730483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:27.732013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:27.732023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:27.732066Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e8983d7f-c7d62f83-b0fb53-a0b4386d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:27.732094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:27.732100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:27.732103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:27.732118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:27.732136Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:27.732158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:27.738101Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 309 partition: 0 MaxSeqNo: 25 sessionId: src_id|e8983d7f-c7d62f83-b0fb53-a0b4386d_0
2025-10-09T21:44:27.738618Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab01c898-1721fca1-a2785e8d-36567cb8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e8983d7f-c7d62f83-b0fb53-a0b4386d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:27.737585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:27.737608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:27.737662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:27.741235Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8983d7f-c7d62f83-b0fb53-a0b4386d_0] Write session: try to update token
2025-10-09T21:44:27.741256Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8983d7f-c7d62f83-b0fb53-a0b4386d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:27.771191Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:27.771209Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:27.771212Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:27.771214Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:27.771223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341157099540665:4263]
2025-10-09T21:44:27.771228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341157099540665:4263]
2025-10-09T21:44:27.771233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341157099540665:4263]
2025-10-09T21:44:27.771280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:27.771287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:27.771290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:27.771425Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:27.774409Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:27.774431Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:27.774435Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:27.776491Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341157461351452:4549]
2025-10-09T21:44:27.777164Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 308, pipe: [1:7559341157461351452:4549]
2025-10-09T21:44:27.800419Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 309 sessionId: src_id|e8983d7f-c7d62f83-b0fb53-a0b4386d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:27.803127Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:28.030533Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:28.031278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341157461351452:4549], now have 1 active actors on pipe
2025-10-09T21:44:28.031313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:28.031322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:28.033207Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341157461351452:4549]
2025-10-09T21:44:28.033247Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:28.033256Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:28.033319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 309 reason: error at write: Disconnected sessionId: src_id|e8983d7f-c7d62f83-b0fb53-a0b4386d_0
2025-10-09T21:44:28.033504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 309 sessionId: src_id|e8983d7f-c7d62f83-b0fb53-a0b4386d_0 is DEAD
2025-10-09T21:44:28.033638Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:28.033999Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8983d7f-c7d62f83-b0fb53-a0b4386d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:28.034011Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8983d7f-c7d62f83-b0fb53-a0b4386d_0] Write session will restart in 0.005220s
2025-10-09T21:44:28.034692Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8983d7f-c7d62f83-b0fb53-a0b4386d_0] Write session: Do CDS request
2025-10-09T21:44:28.034710Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8983d7f-c7d62f83-b0fb53-a0b4386d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:28.034538Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:28.041770Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8983d7f-c7d62f83-b0fb53-a0b4386d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:28.041866Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:28.041886Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 310
2025-10-09T21:44:28.042822Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 310 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:28.042867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 310 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:28.042873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=310 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:28.042878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:28.043789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 310 sessionId:  describe result for acl check
2025-10-09T21:44:28.043826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:28.043829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:28.043831Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:28.043836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341161756318771:4599] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:28.044404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341161756318771:4599] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:28.068019Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341161756318771:4599] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:28.078571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341161756318771:4599] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:28.078593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341161756318771:4599] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:28.078597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341161756318771:4599] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:28.078608Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 310 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:28.091541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:28.092897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:28.093121Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:28.093125Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:28.093140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:28.093223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:28.093264Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341161394508098:4266]
2025-10-09T21:44:28.093919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:28.094589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:28.094638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:28.094948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:28.095010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:28.095011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:28.095013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:28.095014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:28.095015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:28.095016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:28.095018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.095019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.095020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.095025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:28.095037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:28.095040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:28.095044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:28.095049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:28.095052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:28.095056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.095060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.095063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.095072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:28.095080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:28.095087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:28.095093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:28.095098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:28.095103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:28.095108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:28.095113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:28.095118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:28.095123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:28.095127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:28.095133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:28.095138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.095142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.095147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.095152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.095156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.095161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.095207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:28.103985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:28.105902Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 309
2025-10-09T21:44:28.106137Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 309
2025-10-09T21:44:28.107668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 310 partition: 0 MaxSeqNo: 25 sessionId: src_id|76e087bc-2ab32001-c06e20cf-468b32c8_0
2025-10-09T21:44:28.108163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8983d7f-c7d62f83-b0fb53-a0b4386d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|76e087bc-2ab32001-c06e20cf-468b32c8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:28.105070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:28.105085Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:28.105088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:28.105093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:28.105101Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 309 [2:7559341161394508098:4266]
2025-10-09T21:44:28.105117Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:28.105139Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:28.105143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:28.105146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:28.105149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:28.105169Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:28.105175Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:28.105186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:28.105248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.105251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.105253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.105256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:28.105331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:28.105334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.105346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.105358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.105376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:28.105397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341161394508092:4266]
2025-10-09T21:44:28.105400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341161394508092:4266]
2025-10-09T21:44:28.105402Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341161394508092:4266]
2025-10-09T21:44:28.105416Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:28.105430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:28.105446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:28.105449Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:28.105489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341161394508092:4266]
2025-10-09T21:44:28.105492Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:28.105764Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:28.105767Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:28.105769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:28.105775Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:28.106151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341161756318864:4599], now have 1 active actors on pipe
2025-10-09T21:44:28.106156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341161756318764:2432], now have 1 active actors on pipe
2025-10-09T21:44:28.106171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:28.106177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:28.106207Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|76e087bc-2ab32001-c06e20cf-468b32c8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:28.106216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:28.106220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:28.106222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:28.106235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:28.106247Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:28.106265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:28.107197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:28.107202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:28.107223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:28.112687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76e087bc-2ab32001-c06e20cf-468b32c8_0] Write session: try to update token
2025-10-09T21:44:28.112710Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76e087bc-2ab32001-c06e20cf-468b32c8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:28.133337Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341161756318899:4549]
2025-10-09T21:44:28.134021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341161756318899:4549], now have 1 active actors on pipe
2025-10-09T21:44:28.135176Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 309, pipe: [1:7559341161756318899:4549]
2025-10-09T21:44:28.138418Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:28.138433Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:28.138436Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:28.138438Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:28.138448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341161394508092:4266]
2025-10-09T21:44:28.138453Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341161394508092:4266]
2025-10-09T21:44:28.138457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341161394508092:4266]
2025-10-09T21:44:28.138506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:28.138513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.138516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:28.138673Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:28.143409Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:28.143423Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:28.143428Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:28.156921Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 310 sessionId: src_id|76e087bc-2ab32001-c06e20cf-468b32c8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:28.159534Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:28.394896Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:28.395582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:28.395598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:28.395926Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:28.395994Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:28.395998Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:28.396071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 310 reason: error at write: Disconnected sessionId: src_id|76e087bc-2ab32001-c06e20cf-468b32c8_0
2025-10-09T21:44:28.396262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 310 sessionId: src_id|76e087bc-2ab32001-c06e20cf-468b32c8_0 is DEAD
2025-10-09T21:44:28.396353Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341161756318899:4549]
2025-10-09T21:44:28.396756Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76e087bc-2ab32001-c06e20cf-468b32c8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:28.396767Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76e087bc-2ab32001-c06e20cf-468b32c8_0] Write session will restart in 0.005422s
2025-10-09T21:44:28.396796Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76e087bc-2ab32001-c06e20cf-468b32c8_0] Write session: Do CDS request
2025-10-09T21:44:28.396802Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76e087bc-2ab32001-c06e20cf-468b32c8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:28.396637Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:28.403596Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76e087bc-2ab32001-c06e20cf-468b32c8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:28.404220Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:28.404240Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 311
2025-10-09T21:44:28.404606Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 311 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:28.404643Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 311 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:28.404649Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=311 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:28.404653Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:28.405563Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 311 sessionId:  describe result for acl check
2025-10-09T21:44:28.405607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:28.405609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:28.405611Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:28.405617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341161756318919:4603] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:28.406278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341161756318919:4603] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:28.414148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341161756318919:4603] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:28.421528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341161756318919:4603] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:28.421552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341161756318919:4603] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:28.421556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341161756318919:4603] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:28.421567Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 311 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:28.431748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:28.436642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:28.436931Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:28.436943Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:28.436956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:28.437016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:28.437056Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341161394508231:4269]
2025-10-09T21:44:28.437679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:28.438320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:28.438386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:28.438772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:28.438837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:28.438838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:28.438839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:28.438840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:28.438842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:28.438844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:28.438845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.438846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.438847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.438853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:28.438863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:28.438867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:28.438871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:28.438875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:28.438879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:28.438882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.438885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.438889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.438908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:28.438916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:28.438922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:28.438927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:28.438931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:28.438935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:28.438940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:28.438945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:28.438951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:28.438956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:28.438960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:28.438965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:28.438970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.438975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.438979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.438984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.438989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.438994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.439024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:28.448566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:28.449983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:28.450000Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:28.450003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:28.450007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:28.450014Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 310 [2:7559341161394508231:4269]
2025-10-09T21:44:28.450047Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:28.450067Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:28.450071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:28.450074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:28.450077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:28.450096Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:28.450102Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:28.450110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:28.450182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.450184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.450186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.450188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:28.450264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:28.450267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.450279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.450291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.450306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:28.450327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341161394508226:4269]
2025-10-09T21:44:28.450331Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341161394508226:4269]
2025-10-09T21:44:28.450333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341161394508226:4269]
2025-10-09T21:44:28.450342Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:28.450358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:28.450361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:28.450365Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:28.450378Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:28.450385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:28.450388Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:28.450401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341161756319031:4603], now have 1 active actors on pipe
2025-10-09T21:44:28.450405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341161756318905:2432], now have 1 active actors on pipe
2025-10-09T21:44:28.450445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341161394508226:4269]
2025-10-09T21:44:28.450456Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:28.450813Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:28.454805Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 310
2025-10-09T21:44:28.455042Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 310
2025-10-09T21:44:28.457033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:28.457049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:28.457088Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b32a898b-f3ab3972-26e05648-5b1697c7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:28.457101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:28.457107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:28.457110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:28.457124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:28.457143Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:28.457163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:28.457980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:28.457991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:28.458014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:28.458206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 311 partition: 0 MaxSeqNo: 25 sessionId: src_id|b32a898b-f3ab3972-26e05648-5b1697c7_0
2025-10-09T21:44:28.461896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76e087bc-2ab32001-c06e20cf-468b32c8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b32a898b-f3ab3972-26e05648-5b1697c7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:28.464544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b32a898b-f3ab3972-26e05648-5b1697c7_0] Write session: try to update token
2025-10-09T21:44:28.464567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b32a898b-f3ab3972-26e05648-5b1697c7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:28.469512Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:28.469527Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:28.469529Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:28.469531Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:28.469543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341161394508226:4269]
2025-10-09T21:44:28.469546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341161394508226:4269]
2025-10-09T21:44:28.469549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341161394508226:4269]
2025-10-09T21:44:28.469602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:28.469609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.469611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:28.469745Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:28.474261Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:28.474276Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:28.474282Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:28.495340Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 311 sessionId: src_id|b32a898b-f3ab3972-26e05648-5b1697c7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:28.497922Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:28.498002Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341161756319059:4549]
2025-10-09T21:44:28.498952Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 310, pipe: [1:7559341161756319059:4549]
2025-10-09T21:44:28.689746Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:28.692970Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:28.692987Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:28.693055Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 311 reason: error at write: Disconnected sessionId: src_id|b32a898b-f3ab3972-26e05648-5b1697c7_0
2025-10-09T21:44:28.693250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 311 sessionId: src_id|b32a898b-f3ab3972-26e05648-5b1697c7_0 is DEAD
2025-10-09T21:44:28.693412Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341161756319059:4549]
2025-10-09T21:44:28.693426Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:28.693783Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b32a898b-f3ab3972-26e05648-5b1697c7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:28.693793Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b32a898b-f3ab3972-26e05648-5b1697c7_0] Write session will restart in 0.008279s
2025-10-09T21:44:28.693818Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b32a898b-f3ab3972-26e05648-5b1697c7_0] Write session: Do CDS request
2025-10-09T21:44:28.693825Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b32a898b-f3ab3972-26e05648-5b1697c7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:28.690287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:28.690308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:28.690362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341161756319059:4549], now have 1 active actors on pipe
2025-10-09T21:44:28.695336Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:28.706696Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b32a898b-f3ab3972-26e05648-5b1697c7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:28.706932Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:28.706948Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 312
2025-10-09T21:44:28.707046Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 312 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:28.707072Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 312 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:28.707078Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=312 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:28.707082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:28.709667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 312 sessionId:  describe result for acl check
2025-10-09T21:44:28.709712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:28.709714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:28.709715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:28.709721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341161756319087:4615] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:28.710471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341161756319087:4615] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:28.718941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341161756319087:4615] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:28.723463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341161756319087:4615] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:28.723486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341161756319087:4615] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:28.723490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341161756319087:4615] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:28.723500Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 312 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:28.739801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:28.742797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:28.743072Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:28.743078Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:28.743093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:28.743187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:28.743238Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341161394508403:4283]
2025-10-09T21:44:28.743856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:28.745884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:28.745959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:28.746352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:28.746456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:28.746462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:28.746463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:28.746465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:28.746467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:28.746468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:28.746469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.746470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.746472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.746479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:28.746491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:28.746496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:28.746500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:28.746504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:28.746508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:28.746512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.746516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.746521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.746529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:28.746550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:28.746557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:28.746563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:28.746569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:28.746575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:28.746580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:28.746585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:28.746590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:28.746603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:28.746608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:28.746613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:28.746619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.746626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.746631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.746636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.746641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.746647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.746678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:28.755565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:28.756900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:28.756906Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:28.756909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:28.756913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:28.756921Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 311 [2:7559341161394508403:4283]
2025-10-09T21:44:28.756935Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:28.756954Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:28.756958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:28.756961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:28.756964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:28.756982Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:28.756988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:28.756995Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:28.757058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.757060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.757061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.757064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:28.757145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:28.757148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.757162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:28.757175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:28.757194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:28.757214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341161394508397:4283]
2025-10-09T21:44:28.757217Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341161394508397:4283]
2025-10-09T21:44:28.757219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341161394508397:4283]
2025-10-09T21:44:28.757235Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:28.757249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:28.757264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:28.757267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:28.757306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341161394508397:4283]
2025-10-09T21:44:28.757308Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:28.757556Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:28.757560Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:28.757562Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:28.757570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:28.757809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341161756319141:4615], now have 1 active actors on pipe
2025-10-09T21:44:28.757824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341161756319073:2432], now have 1 active actors on pipe
2025-10-09T21:44:28.759089Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 311
2025-10-09T21:44:28.759124Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 311
2025-10-09T21:44:28.759868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:28.759883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:28.759931Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d4ff769f-3918aae0-950487c9-3b5ab359_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:28.759949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:28.759957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:28.759959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:28.759974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:28.759991Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:28.760017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:28.761500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:28.761513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:28.761543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:28.762916Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 312 partition: 0 MaxSeqNo: 25 sessionId: src_id|d4ff769f-3918aae0-950487c9-3b5ab359_0
2025-10-09T21:44:28.763313Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b32a898b-f3ab3972-26e05648-5b1697c7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d4ff769f-3918aae0-950487c9-3b5ab359_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:28.765892Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d4ff769f-3918aae0-950487c9-3b5ab359_0] Write session: try to update token
2025-10-09T21:44:28.765914Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d4ff769f-3918aae0-950487c9-3b5ab359_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:28.777483Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:28.777502Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:28.777504Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:28.777506Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:28.777516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341161394508397:4283]
2025-10-09T21:44:28.777520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341161394508397:4283]
2025-10-09T21:44:28.777524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341161394508397:4283]
2025-10-09T21:44:28.777570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:28.777577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:28.777580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:28.777716Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:28.782689Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:28.782701Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:28.782705Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:28.787908Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 312 sessionId: src_id|d4ff769f-3918aae0-950487c9-3b5ab359_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:28.790627Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:28.793383Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341161756319215:4549]
2025-10-09T21:44:28.794345Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 311, pipe: [1:7559341161756319215:4549]
2025-10-09T21:44:29.020125Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:29.020806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:29.020816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:29.020838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341161756319215:4549], now have 1 active actors on pipe
2025-10-09T21:44:29.021561Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:29.021662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 312 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d4ff769f-3918aae0-950487c9-3b5ab359_0
2025-10-09T21:44:29.021165Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:29.021988Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4ff769f-3918aae0-950487c9-3b5ab359_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:29.022000Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4ff769f-3918aae0-950487c9-3b5ab359_0] Write session will restart in 0.005204s
2025-10-09T21:44:29.021828Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341161756319215:4549]
2025-10-09T21:44:29.021841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 312 sessionId: src_id|d4ff769f-3918aae0-950487c9-3b5ab359_0 is DEAD
2025-10-09T21:44:29.021851Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:29.022048Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4ff769f-3918aae0-950487c9-3b5ab359_0] Write session: Do CDS request
2025-10-09T21:44:29.022056Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4ff769f-3918aae0-950487c9-3b5ab359_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:29.028648Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d4ff769f-3918aae0-950487c9-3b5ab359_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:29.028775Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:29.028792Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 313
2025-10-09T21:44:29.028948Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 313 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:29.028986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 313 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:29.029008Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=313 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:29.029012Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:29.029546Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 313 sessionId:  describe result for acl check
2025-10-09T21:44:29.029590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:29.029597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:29.029615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:29.029626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341166051286544:4620] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:29.030239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341166051286544:4620] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:29.035522Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341166051286544:4620] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:29.038850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:29.041081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:29.041311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341166051286544:4620] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:29.041339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341166051286544:4620] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:29.041344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341166051286544:4620] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:29.041353Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 313 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:29.041403Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:29.041415Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:29.041430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:29.041514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:29.041568Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341165689475834:4285]
2025-10-09T21:44:29.042168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:29.042814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:29.042888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:29.043274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:29.043338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:29.043360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:29.043361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:29.043363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:29.043364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:29.043366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:29.043367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.043368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.043370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.043376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:29.043388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:29.043392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:29.043397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:29.043413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:29.043417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:29.043421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.043425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.043428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.043436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:29.043445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:29.043457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:29.043463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:29.043468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:29.043473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:29.043483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:29.043489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:29.043495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:29.043506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:29.043512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:29.043521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:29.043526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.043531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.043543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.043549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.043556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.043562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.043597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:29.059144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:29.060544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:29.060551Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:29.060554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:29.060559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:29.060566Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 312 [2:7559341165689475834:4285]
2025-10-09T21:44:29.060586Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:29.060607Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:29.060611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.060614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.060617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.060637Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:29.060643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:29.060652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:29.060741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.060745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.061223Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 312
2025-10-09T21:44:29.061258Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 312
2025-10-09T21:44:29.060747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.060750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:29.060834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:29.060837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.060849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.060859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.060876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:29.060898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341165689475829:4285]
2025-10-09T21:44:29.060904Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341165689475829:4285]
2025-10-09T21:44:29.060906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341165689475829:4285]
2025-10-09T21:44:29.060922Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:29.060941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341166051286656:4620], now have 1 active actors on pipe
2025-10-09T21:44:29.060949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341166051286529:2432], now have 1 active actors on pipe
2025-10-09T21:44:29.060954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:29.060969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:29.060972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:29.061015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341165689475829:4285]
2025-10-09T21:44:29.061018Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:29.061301Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:29.061303Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:29.061305Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:29.061312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:29.065290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:29.065312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:29.065354Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a1c3c8f4-fd4e5c4e-c3bbc26-b0ff1bda_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:29.065372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.065377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.065381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:29.065395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.065419Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:29.065634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:29.066294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:29.066306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:29.066327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:29.066499Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 313 partition: 0 MaxSeqNo: 25 sessionId: src_id|a1c3c8f4-fd4e5c4e-c3bbc26-b0ff1bda_0
2025-10-09T21:44:29.066976Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4ff769f-3918aae0-950487c9-3b5ab359_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a1c3c8f4-fd4e5c4e-c3bbc26-b0ff1bda_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:29.069534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a1c3c8f4-fd4e5c4e-c3bbc26-b0ff1bda_0] Write session: try to update token
2025-10-09T21:44:29.069556Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a1c3c8f4-fd4e5c4e-c3bbc26-b0ff1bda_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:29.085933Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:29.085948Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:29.085951Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:29.085952Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:29.085963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341165689475829:4285]
2025-10-09T21:44:29.085967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341165689475829:4285]
2025-10-09T21:44:29.085974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341165689475829:4285]
2025-10-09T21:44:29.086049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:29.086058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.086061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:29.086191Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:29.089868Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:29.089886Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:29.089890Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:29.105707Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 313 sessionId: src_id|a1c3c8f4-fd4e5c4e-c3bbc26-b0ff1bda_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:29.108310Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:29.121504Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341166051286668:4549]
2025-10-09T21:44:29.122246Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 312, pipe: [1:7559341166051286668:4549]
2025-10-09T21:44:29.260043Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:29.260323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:29.260331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:29.260351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341166051286668:4549], now have 1 active actors on pipe
2025-10-09T21:44:29.261771Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1c3c8f4-fd4e5c4e-c3bbc26-b0ff1bda_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:29.261785Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1c3c8f4-fd4e5c4e-c3bbc26-b0ff1bda_0] Write session will restart in 0.009031s
2025-10-09T21:44:29.261811Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1c3c8f4-fd4e5c4e-c3bbc26-b0ff1bda_0] Write session: Do CDS request
2025-10-09T21:44:29.261817Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1c3c8f4-fd4e5c4e-c3bbc26-b0ff1bda_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:29.261093Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:29.261133Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341166051286668:4549]
2025-10-09T21:44:29.261152Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:29.261154Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:29.261243Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 313 reason: error at write: Disconnected sessionId: src_id|a1c3c8f4-fd4e5c4e-c3bbc26-b0ff1bda_0
2025-10-09T21:44:29.261430Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 313 sessionId: src_id|a1c3c8f4-fd4e5c4e-c3bbc26-b0ff1bda_0 is DEAD
2025-10-09T21:44:29.263002Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:29.271576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a1c3c8f4-fd4e5c4e-c3bbc26-b0ff1bda_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:29.271938Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:29.271956Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 314
2025-10-09T21:44:29.272064Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 314 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:29.272094Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 314 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:29.272103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=314 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:29.272107Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:29.272513Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 314 sessionId:  describe result for acl check
2025-10-09T21:44:29.272541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:29.272543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:29.272547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:29.272553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341166051286685:4624] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:29.273127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341166051286685:4624] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:29.277483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341166051286685:4624] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:29.280956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341166051286685:4624] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:29.280966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341166051286685:4624] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:29.280970Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341166051286685:4624] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:29.280979Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 314 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:29.286069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:29.287012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:29.287234Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:29.287239Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:29.287252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:29.287322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:29.287363Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341165689475973:4289]
2025-10-09T21:44:29.287980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:29.288504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:29.288560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:29.288897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:29.288955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:29.288956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:29.288958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:29.288959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:29.288961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:29.288962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:29.288963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.288964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.288965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.288971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:29.288982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:29.288986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:29.288989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:29.288992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:29.288996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:29.288999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.289003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.289006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.289014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:29.289022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:29.289029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:29.289034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:29.289038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:29.289042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:29.289047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:29.289052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:29.289056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:29.289061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:29.289066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:29.289070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:29.289074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.289078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.289084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.289087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.289091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.289096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.289123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:29.297132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:29.298829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:29.298850Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:29.298853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:29.298858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:29.298866Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 313 [2:7559341165689475973:4289]
2025-10-09T21:44:29.298884Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:29.298905Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:29.298914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.298918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.298920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.298942Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:29.298949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:29.298956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:29.299022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.299030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.299031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.299034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:29.299113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:29.299125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.299139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.299150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.299164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:29.299189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341166051286798:4624], now have 1 active actors on pipe
2025-10-09T21:44:29.299193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341166051286671:2432], now have 1 active actors on pipe
2025-10-09T21:44:29.299201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341165689475968:4289]
2025-10-09T21:44:29.299204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341165689475968:4289]
2025-10-09T21:44:29.299208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341165689475968:4289]
2025-10-09T21:44:29.299224Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:29.299239Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:29.299242Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:29.299246Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:29.299248Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:29.299253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:29.299256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:29.299300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341165689475968:4289]
2025-10-09T21:44:29.299310Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:29.299321Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:29.300240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:29.300253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:29.300295Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6b1b9de5-5929d7b-3f6b4029-2f3360d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:29.300307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.300312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.300316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:29.300328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.300343Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:29.300383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:29.300737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:29.300746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:29.300764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:29.301391Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1c3c8f4-fd4e5c4e-c3bbc26-b0ff1bda_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6b1b9de5-5929d7b-3f6b4029-2f3360d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:29.299339Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 313
2025-10-09T21:44:29.299374Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 313
2025-10-09T21:44:29.300921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 314 partition: 0 MaxSeqNo: 25 sessionId: src_id|6b1b9de5-5929d7b-3f6b4029-2f3360d_0
2025-10-09T21:44:29.304346Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b1b9de5-5929d7b-3f6b4029-2f3360d_0] Write session: try to update token
2025-10-09T21:44:29.304365Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b1b9de5-5929d7b-3f6b4029-2f3360d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:29.312673Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: CLIENT_DEADLINE_EXCEEDED. Description: [ { <main>: Error: GRpc error: (4): Deadline Exceeded } { <main>: Error: Grpc error response on endpoint localhost:14704 } ]
2025-10-09T21:44:29.333316Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:29.333335Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:29.333347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341165689475968:4289]
2025-10-09T21:44:29.333407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:29.333612Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:29.333784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:29.333943Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:29.334085Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:29.335395Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 314 sessionId: src_id|6b1b9de5-5929d7b-3f6b4029-2f3360d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:29.338295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:29.338545Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:29.338620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:29.338828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:29.338838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:29.338876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:29.338222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:29.343344Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:29.366426Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341166051286813:4549]
2025-10-09T21:44:29.367082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341166051286813:4549], now have 1 active actors on pipe
2025-10-09T21:44:29.368389Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:29.368394Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:29.368396Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:29.368398Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:29.368411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341165689475968:4289]
2025-10-09T21:44:29.368414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341165689475968:4289]
2025-10-09T21:44:29.368417Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341165689475968:4289]
2025-10-09T21:44:29.368463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:29.368471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.368473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:29.368631Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:29.370394Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 313, pipe: [1:7559341166051286813:4549]
2025-10-09T21:44:29.370487Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:29.370496Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:29.370499Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:29.547878Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:29.556022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:29.556049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:29.556212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:29.556265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:29.556326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:29.556382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:29.556862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:29.556868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:29.556936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:29.556990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:29.557053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:29.557115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:29.557155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:29.557158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:29.557226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:29.557285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:29.557346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:29.557410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:29.557449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:29.557452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:29.557541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:29.557602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:29.557659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:29.557718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:29.557758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:29.557760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:29.557829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:29.557889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:29.557951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:29.558014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:29.558231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:29.558240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:29.558311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:29.558383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:29.558445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:29.558504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:29.558545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:29.558548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:29.558621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:29.558675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:29.558731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:29.558791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:29.558836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:29.558838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:29.558908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:29.558965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:29.559024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:29.559085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:29.559125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:29.559128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:29.559196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:29.559255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:29.559312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:29.559372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:29.559419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:29.559423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:29.559490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:29.559547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:29.559604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:29.559665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:29.559709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:29.559712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:29.561343Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:29.567188Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b1b9de5-5929d7b-3f6b4029-2f3360d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:29.567214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b1b9de5-5929d7b-3f6b4029-2f3360d_0] Write session will restart in 0.006277s
2025-10-09T21:44:29.567241Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b1b9de5-5929d7b-3f6b4029-2f3360d_0] Write session: Do CDS request
2025-10-09T21:44:29.567249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b1b9de5-5929d7b-3f6b4029-2f3360d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:29.566747Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:29.566813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 314 reason: status is not ok: tablet will be restarted right now sessionId: src_id|6b1b9de5-5929d7b-3f6b4029-2f3360d_0
2025-10-09T21:44:29.567027Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 314 sessionId: src_id|6b1b9de5-5929d7b-3f6b4029-2f3360d_0 is DEAD
2025-10-09T21:44:29.567194Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:29.567242Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341166051286813:4549]
2025-10-09T21:44:29.574529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b1b9de5-5929d7b-3f6b4029-2f3360d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:29.576978Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:29.576995Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 315
2025-10-09T21:44:29.577251Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 315 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:29.577296Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 315 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:29.577301Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=315 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:29.577305Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:29.578467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 315 sessionId:  describe result for acl check
2025-10-09T21:44:29.578519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:29.578521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:29.578522Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:29.578528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341166051286855:4635] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:29.579116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341166051286855:4635] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:29.590388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341166051286855:4635] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:29.595217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:29.596165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:29.596390Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:29.596400Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:29.596412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:29.596464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:29.596507Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341165689476122:4297]
2025-10-09T21:44:29.597099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:29.597540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:29.597587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:29.597850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:29.597896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:29.597902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:29.597903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:29.597903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:29.597904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:29.597906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:29.597906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.597907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.597908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.597914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:29.597924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:29.597928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:29.597933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:29.597937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:29.597940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:29.597944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.597947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.597949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.597957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:29.597965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:29.597970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:29.597976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:29.597980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:29.597986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:29.597991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:29.597996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:29.598001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:29.598005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:29.598010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:29.598015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:29.598020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.598024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.598043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.598048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.598052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.598057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.598099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:29.613054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:29.612733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341166051286855:4635] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:29.612752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341166051286855:4635] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:29.612757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341166051286855:4635] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:29.612767Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 315 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:29.614608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:29.614622Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:29.614625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:29.614629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:29.614637Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 314 [2:7559341165689476122:4297]
2025-10-09T21:44:29.614651Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:29.614671Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:29.614674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.614677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.614679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.614698Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:29.614704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:29.614711Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:29.614772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.614774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.614776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.614780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:29.614853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:29.614856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.614870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.614883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.614900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:29.614922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341165689476116:4297]
2025-10-09T21:44:29.614924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341165689476116:4297]
2025-10-09T21:44:29.614926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341165689476116:4297]
2025-10-09T21:44:29.614936Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:29.614952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:29.614966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:29.614969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:29.615010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341165689476116:4297]
2025-10-09T21:44:29.615012Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:29.615078Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:29.615081Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:29.615083Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:29.615100Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:29.615438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341166051286973:4635], now have 1 active actors on pipe
2025-10-09T21:44:29.615448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341166051286833:2432], now have 1 active actors on pipe
2025-10-09T21:44:29.616101Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 314
2025-10-09T21:44:29.616129Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 314
2025-10-09T21:44:29.616592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:29.616606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:29.616647Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cdcb090e-81782413-72145549-c5731ba0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:29.616659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.616663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.616666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:29.616679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.616693Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:29.616724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:29.618284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:29.618306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:29.618331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:29.618476Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 315 partition: 0 MaxSeqNo: 25 sessionId: src_id|cdcb090e-81782413-72145549-c5731ba0_0
2025-10-09T21:44:29.618885Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b1b9de5-5929d7b-3f6b4029-2f3360d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cdcb090e-81782413-72145549-c5731ba0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:29.621452Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cdcb090e-81782413-72145549-c5731ba0_0] Write session: try to update token
2025-10-09T21:44:29.621473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cdcb090e-81782413-72145549-c5731ba0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:29.635667Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:29.635684Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:29.635688Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:29.635689Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:29.635700Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341165689476116:4297]
2025-10-09T21:44:29.635705Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341165689476116:4297]
2025-10-09T21:44:29.635710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341165689476116:4297]
2025-10-09T21:44:29.635761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:29.635769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.635776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:29.635932Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:29.638136Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:29.638155Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:29.638159Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:29.649497Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 315 sessionId: src_id|cdcb090e-81782413-72145549-c5731ba0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:29.652173Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:29.666805Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341166051286984:4549]
2025-10-09T21:44:29.667314Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 314, pipe: [1:7559341166051286984:4549]
2025-10-09T21:44:29.812090Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:29.813860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:29.813877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:29.813902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341166051286984:4549], now have 1 active actors on pipe
2025-10-09T21:44:29.814194Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:29.814355Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341166051286984:4549]
2025-10-09T21:44:29.814408Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:29.814411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:29.814474Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 315 reason: error at write: Disconnected sessionId: src_id|cdcb090e-81782413-72145549-c5731ba0_0
2025-10-09T21:44:29.814684Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 315 sessionId: src_id|cdcb090e-81782413-72145549-c5731ba0_0 is DEAD
2025-10-09T21:44:29.814716Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:29.815138Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cdcb090e-81782413-72145549-c5731ba0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:29.815154Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cdcb090e-81782413-72145549-c5731ba0_0] Write session will restart in 0.009560s
2025-10-09T21:44:29.815184Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cdcb090e-81782413-72145549-c5731ba0_0] Write session: Do CDS request
2025-10-09T21:44:29.815192Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cdcb090e-81782413-72145549-c5731ba0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:29.825716Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cdcb090e-81782413-72145549-c5731ba0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:29.826138Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:29.826157Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 316
2025-10-09T21:44:29.826360Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 316 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:29.826408Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 316 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:29.826415Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=316 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:29.826419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:29.826929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 316 sessionId:  describe result for acl check
2025-10-09T21:44:29.826968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:29.826969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:29.826971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:29.826976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341166051286997:4640] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:29.827531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341166051286997:4640] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:29.834985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341166051286997:4640] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:29.844768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341166051286997:4640] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:29.844801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341166051286997:4640] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:29.844808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341166051286997:4640] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:29.844820Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 316 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:29.851193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:29.854013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:29.854459Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:29.854467Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:29.854480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:29.854537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:29.854579Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341165689476285:4305]
2025-10-09T21:44:29.855205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:29.855632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:29.855672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:29.856229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:29.856301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:29.856302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:29.856303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:29.856304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:29.856306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:29.856307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:29.856308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.856309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.856310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.856316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:29.856328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:29.856332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:29.856336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:29.856353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:29.856357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:29.856361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.856365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.856368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.856377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:29.856385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:29.856391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:29.856396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:29.856400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:29.856406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:29.856411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:29.856416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:29.856420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:29.856425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:29.856429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:29.856435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:29.856440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.856444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.856449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.856454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.856458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.856464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.856494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:29.868807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:29.870310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:29.870318Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:29.870321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:29.870326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:29.870334Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 315 [2:7559341165689476285:4305]
2025-10-09T21:44:29.870352Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:29.870386Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:29.870389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.870392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.870394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.870414Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:29.870422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:29.870430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:29.870496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.870498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.870499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.870502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:29.870587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:29.870589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.870604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:29.870614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:29.870632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:29.870652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341165689476280:4305]
2025-10-09T21:44:29.870654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341165689476280:4305]
2025-10-09T21:44:29.870656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341165689476280:4305]
2025-10-09T21:44:29.870671Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:29.870684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:29.870698Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:29.870701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:29.870743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341165689476280:4305]
2025-10-09T21:44:29.870746Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:29.871053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:29.871058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:29.871060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:29.871066Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:29.871889Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 315
2025-10-09T21:44:29.871922Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 315
2025-10-09T21:44:29.872364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341166051287092:4640], now have 1 active actors on pipe
2025-10-09T21:44:29.872375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341166051286987:2432], now have 1 active actors on pipe
2025-10-09T21:44:29.872391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:29.872397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:29.872434Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f032879c-4f1b8380-12ad5b13-e596f760_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:29.872446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.872451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.872454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:29.872467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.872482Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:29.872504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:29.873292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:29.873300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:29.873321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:29.873723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 316 partition: 0 MaxSeqNo: 25 sessionId: src_id|f032879c-4f1b8380-12ad5b13-e596f760_0
2025-10-09T21:44:29.874276Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cdcb090e-81782413-72145549-c5731ba0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f032879c-4f1b8380-12ad5b13-e596f760_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:29.877723Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f032879c-4f1b8380-12ad5b13-e596f760_0] Write session: try to update token
2025-10-09T21:44:29.877746Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f032879c-4f1b8380-12ad5b13-e596f760_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:29.898112Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 316 sessionId: src_id|f032879c-4f1b8380-12ad5b13-e596f760_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:29.900830Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:29.902527Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:29.902544Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:29.902547Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:29.902549Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:29.902559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341165689476280:4305]
2025-10-09T21:44:29.902563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341165689476280:4305]
2025-10-09T21:44:29.902567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341165689476280:4305]
2025-10-09T21:44:29.902614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:29.902620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:29.902623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:29.902754Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:29.904204Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:29.904211Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:29.904215Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:29.914824Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341166051287139:4549]
2025-10-09T21:44:29.916320Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 315, pipe: [1:7559341166051287139:4549]
2025-10-09T21:44:30.097492Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:30.097888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:30.097901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:30.097951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341166051287139:4549], now have 1 active actors on pipe
2025-10-09T21:44:30.098305Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:30.098356Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341166051287139:4549]
2025-10-09T21:44:30.098392Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:30.098397Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:30.098549Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:30.099087Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 316 reason: error at write: Disconnected sessionId: src_id|f032879c-4f1b8380-12ad5b13-e596f760_0
2025-10-09T21:44:30.099296Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 316 sessionId: src_id|f032879c-4f1b8380-12ad5b13-e596f760_0 is DEAD
2025-10-09T21:44:30.099744Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f032879c-4f1b8380-12ad5b13-e596f760_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:30.099759Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f032879c-4f1b8380-12ad5b13-e596f760_0] Write session will restart in 0.009724s
2025-10-09T21:44:30.099788Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f032879c-4f1b8380-12ad5b13-e596f760_0] Write session: Do CDS request
2025-10-09T21:44:30.099796Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f032879c-4f1b8380-12ad5b13-e596f760_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:30.110855Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f032879c-4f1b8380-12ad5b13-e596f760_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:30.111054Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:30.111076Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 317
2025-10-09T21:44:30.111274Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 317 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:30.111312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 317 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:30.111322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=317 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:30.111328Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:30.112645Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 317 sessionId:  describe result for acl check
2025-10-09T21:44:30.112689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:30.112698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:30.112700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:30.112707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341170346254458:4645] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:30.113312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341170346254458:4645] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:30.119716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341170346254458:4645] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:30.130567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:30.129873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341170346254458:4645] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:30.129893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341170346254458:4645] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:30.129897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341170346254458:4645] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:30.129908Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 317 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:30.134417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:30.134711Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:30.134715Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:30.134730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:30.134810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:30.134852Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341169984443722:4308]
2025-10-09T21:44:30.135471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:30.138901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:30.138988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:30.139409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:30.139493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:30.139495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:30.139496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:30.139497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:30.139499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:30.139500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:30.139501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.139503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.139504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.139513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:30.139525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:30.139530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:30.139534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:30.139537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:30.139542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:30.139545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.139549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.139552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.139564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:30.139573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:30.139580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:30.139585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:30.139590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:30.139595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:30.139599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:30.139604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:30.139609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:30.139614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:30.139619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:30.139625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:30.139631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.139636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.139642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.139648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.139653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.139658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.139687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:30.152273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:30.153583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:30.153591Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:30.153595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:30.153599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:30.153608Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 316 [2:7559341169984443722:4308]
2025-10-09T21:44:30.153624Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:30.153644Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:30.153648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.153652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.153654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.153676Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:30.153682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:30.153691Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:30.153763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.153765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.153767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.153770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:30.153867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:30.153870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.153884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.153895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.153910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:30.153933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341169984443715:4308]
2025-10-09T21:44:30.153936Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341169984443715:4308]
2025-10-09T21:44:30.153938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341169984443715:4308]
2025-10-09T21:44:30.153951Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:30.153970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:30.153984Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:30.153987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:30.154053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341169984443715:4308]
2025-10-09T21:44:30.154056Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:30.155591Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 316
2025-10-09T21:44:30.155630Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 316
2025-10-09T21:44:30.154843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:30.154853Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:30.154855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:30.154863Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:30.161887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341170346254443:2432], now have 1 active actors on pipe
2025-10-09T21:44:30.161906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341170346254575:4645], now have 1 active actors on pipe
2025-10-09T21:44:30.161957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:30.161966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:30.162010Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|40deb803-253dc285-c6e18cbe-fc2e9482_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:30.162024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.162045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.162048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:30.162067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.162089Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:30.162127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:30.165029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:30.165050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:30.165095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:30.166861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 317 partition: 0 MaxSeqNo: 25 sessionId: src_id|40deb803-253dc285-c6e18cbe-fc2e9482_0
2025-10-09T21:44:30.167824Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f032879c-4f1b8380-12ad5b13-e596f760_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|40deb803-253dc285-c6e18cbe-fc2e9482_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:30.170702Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|40deb803-253dc285-c6e18cbe-fc2e9482_0] Write session: try to update token
2025-10-09T21:44:30.170727Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|40deb803-253dc285-c6e18cbe-fc2e9482_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:30.179552Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:30.179572Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:30.179575Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:30.179579Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:30.179589Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341169984443715:4308]
2025-10-09T21:44:30.179595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341169984443715:4308]
2025-10-09T21:44:30.179600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341169984443715:4308]
2025-10-09T21:44:30.179656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:30.179673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.179676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:30.179814Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:30.183755Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:30.183769Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:30.183774Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:30.200206Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341170346254588:4549]
2025-10-09T21:44:30.201697Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 316, pipe: [1:7559341170346254588:4549]
2025-10-09T21:44:30.207723Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 317 sessionId: src_id|40deb803-253dc285-c6e18cbe-fc2e9482_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:30.210410Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:30.373160Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:30.373698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341170346254588:4549], now have 1 active actors on pipe
2025-10-09T21:44:30.373727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:30.373735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:30.374164Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:30.374495Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:30.374536Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341170346254588:4549]
2025-10-09T21:44:30.374550Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:30.374553Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:30.374613Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 317 reason: error at write: Disconnected sessionId: src_id|40deb803-253dc285-c6e18cbe-fc2e9482_0
2025-10-09T21:44:30.374799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 317 sessionId: src_id|40deb803-253dc285-c6e18cbe-fc2e9482_0 is DEAD
2025-10-09T21:44:30.376247Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|40deb803-253dc285-c6e18cbe-fc2e9482_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:30.376266Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|40deb803-253dc285-c6e18cbe-fc2e9482_0] Write session will restart in 0.006748s
2025-10-09T21:44:30.376311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|40deb803-253dc285-c6e18cbe-fc2e9482_0] Write session: Do CDS request
2025-10-09T21:44:30.376335Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|40deb803-253dc285-c6e18cbe-fc2e9482_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:30.386855Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|40deb803-253dc285-c6e18cbe-fc2e9482_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:30.387353Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:30.387373Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 318
2025-10-09T21:44:30.388103Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 318 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:30.388147Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 318 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:30.388154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=318 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:30.388158Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:30.395296Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 318 sessionId:  describe result for acl check
2025-10-09T21:44:30.395351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:30.395354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:30.395355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:30.395363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341170346254625:4649] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:30.396023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341170346254625:4649] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:30.407479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341170346254625:4649] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:30.415209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341170346254625:4649] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:30.415231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341170346254625:4649] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:30.415235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341170346254625:4649] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:30.415246Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 318 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:30.418134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:30.420962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:30.421233Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:30.421237Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:30.421253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:30.421327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:30.421368Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341169984443865:4311]
2025-10-09T21:44:30.421974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:30.423979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:30.424060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:30.424441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:30.424521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:30.424528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:30.424530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:30.424531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:30.424533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:30.424534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:30.424536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.424537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.424538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.424546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:30.424558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:30.424562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:30.424566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:30.424569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:30.424574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:30.424577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.424581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.424585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.424595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:30.424605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:30.424613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:30.424618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:30.424623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:30.424629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:30.424634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:30.424640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:30.424646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:30.424652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:30.424657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:30.424665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:30.424672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.424677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.424683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.424689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.424694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.424701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.424742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:30.440451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:30.441551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:30.441558Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:30.441561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:30.441568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:30.441576Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 317 [2:7559341169984443865:4311]
2025-10-09T21:44:30.441594Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:30.443525Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 317
2025-10-09T21:44:30.443578Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 317
2025-10-09T21:44:30.446182Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 318 partition: 0 MaxSeqNo: 25 sessionId: src_id|183dceeb-6d03b31b-ef8946c0-829f90cc_0
2025-10-09T21:44:30.442588Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:30.442601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.442605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.442608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.442630Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:30.442637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:30.442645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:30.442720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.442722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.442725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.442728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:30.442809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:30.442813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.442830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.442845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.442864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:30.442885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341169984443859:4311]
2025-10-09T21:44:30.442888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341169984443859:4311]
2025-10-09T21:44:30.442891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341169984443859:4311]
2025-10-09T21:44:30.442909Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:30.442925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:30.442942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:30.442945Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:30.442991Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341169984443859:4311]
2025-10-09T21:44:30.442995Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:30.443132Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:30.443135Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:30.443138Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:30.443144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:30.444769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341170346254591:2432], now have 1 active actors on pipe
2025-10-09T21:44:30.444775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341170346254723:4649], now have 1 active actors on pipe
2025-10-09T21:44:30.444797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:30.444803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:30.444836Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|183dceeb-6d03b31b-ef8946c0-829f90cc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:30.444847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.444852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.444855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:30.444867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.444883Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:30.444906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:30.445947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:30.445954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:30.445978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:30.450176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|40deb803-253dc285-c6e18cbe-fc2e9482_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|183dceeb-6d03b31b-ef8946c0-829f90cc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:30.452788Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|183dceeb-6d03b31b-ef8946c0-829f90cc_0] Write session: try to update token
2025-10-09T21:44:30.452813Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|183dceeb-6d03b31b-ef8946c0-829f90cc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:30.475681Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341170346254737:4549]
2025-10-09T21:44:30.477696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341170346254737:4549], now have 1 active actors on pipe
2025-10-09T21:44:30.477828Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 317, pipe: [1:7559341170346254737:4549]
2025-10-09T21:44:30.485331Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:30.485349Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:30.485352Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:30.485354Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:30.485365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341169984443859:4311]
2025-10-09T21:44:30.485369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341169984443859:4311]
2025-10-09T21:44:30.485376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341169984443859:4311]
2025-10-09T21:44:30.485424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:30.485432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.485435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:30.485577Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:30.489263Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:30.489278Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:30.489282Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:30.499770Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 318 sessionId: src_id|183dceeb-6d03b31b-ef8946c0-829f90cc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:30.502326Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:30.713776Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:30.716903Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341170346254737:4549]
2025-10-09T21:44:30.716943Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:30.716946Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:30.717003Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 318 reason: error at write: Disconnected sessionId: src_id|183dceeb-6d03b31b-ef8946c0-829f90cc_0
2025-10-09T21:44:30.717175Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 318 sessionId: src_id|183dceeb-6d03b31b-ef8946c0-829f90cc_0 is DEAD
2025-10-09T21:44:30.717299Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:30.717531Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|183dceeb-6d03b31b-ef8946c0-829f90cc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:30.717550Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|183dceeb-6d03b31b-ef8946c0-829f90cc_0] Write session will restart in 0.007739s
2025-10-09T21:44:30.717576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|183dceeb-6d03b31b-ef8946c0-829f90cc_0] Write session: Do CDS request
2025-10-09T21:44:30.717584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|183dceeb-6d03b31b-ef8946c0-829f90cc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:30.714793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:30.714810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:30.716840Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:30.726590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|183dceeb-6d03b31b-ef8946c0-829f90cc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:30.726825Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:30.726850Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 319
2025-10-09T21:44:30.730264Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 319 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:30.730322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 319 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:30.730329Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=319 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:30.730334Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:30.730928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 319 sessionId:  describe result for acl check
2025-10-09T21:44:30.730967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:30.730969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:30.730971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:30.730978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341170346254787:4661] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:30.731550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341170346254787:4661] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:30.734534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341170346254787:4661] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:30.749709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341170346254787:4661] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:30.749730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341170346254787:4661] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:30.749734Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341170346254787:4661] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:30.749743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 319 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:30.772484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:30.772955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:30.773320Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:30.773323Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:30.773337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:30.773394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:30.773432Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341169984444045:4325]
2025-10-09T21:44:30.774011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:30.775087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:30.775155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:30.775499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:30.775546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:30.775548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:30.775549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:30.775550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:30.775551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:30.775552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:30.775553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.775555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.775556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.775562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:30.775573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:30.775577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:30.775580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:30.775584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:30.775588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:30.775592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.775596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.775601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.775608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:30.775617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:30.775624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:30.775630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:30.775635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:30.775641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:30.775646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:30.775651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:30.775656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:30.775662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:30.775666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:30.775671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:30.775675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.775681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.775685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.775691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.775696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.775701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.775733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:30.788150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:30.789604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:30.789612Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:30.789614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:30.789619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:30.789627Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 318 [2:7559341169984444045:4325]
2025-10-09T21:44:30.789642Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:30.789666Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:30.789669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.789673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.789678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.789699Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:30.789704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:30.789712Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:30.789776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.789778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.789780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.789783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:30.789863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:30.789866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.789883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:30.789894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:30.789912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:30.789933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341169984444039:4325]
2025-10-09T21:44:30.789936Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341169984444039:4325]
2025-10-09T21:44:30.789938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341169984444039:4325]
2025-10-09T21:44:30.789952Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:30.789967Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:30.789983Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:30.789987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:30.790048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341169984444039:4325]
2025-10-09T21:44:30.790050Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:30.790544Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:30.790551Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:30.790553Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:30.790559Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:30.791364Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 318
2025-10-09T21:44:30.791405Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 318
2025-10-09T21:44:30.792306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341170346254773:2432], now have 1 active actors on pipe
2025-10-09T21:44:30.792320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341170346254854:4661], now have 1 active actors on pipe
2025-10-09T21:44:30.792346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:30.792354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:30.792392Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c550871c-498b9ff7-4ed8c4e0-a6ed1837_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:30.792404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.792409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.792413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:30.792427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.792446Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:30.792466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:30.792815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:30.792821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:30.792836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:30.792971Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 319 partition: 0 MaxSeqNo: 25 sessionId: src_id|c550871c-498b9ff7-4ed8c4e0-a6ed1837_0
2025-10-09T21:44:30.793474Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|183dceeb-6d03b31b-ef8946c0-829f90cc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c550871c-498b9ff7-4ed8c4e0-a6ed1837_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:30.796059Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c550871c-498b9ff7-4ed8c4e0-a6ed1837_0] Write session: try to update token
2025-10-09T21:44:30.796080Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c550871c-498b9ff7-4ed8c4e0-a6ed1837_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:30.815915Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:30.815933Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:30.815936Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:30.815938Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:30.815947Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341169984444039:4325]
2025-10-09T21:44:30.815952Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341169984444039:4325]
2025-10-09T21:44:30.815956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341169984444039:4325]
2025-10-09T21:44:30.816002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:30.816010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:30.816013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:30.816157Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:30.818869Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341170346254918:4549]
2025-10-09T21:44:30.819310Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:30.819322Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:30.819327Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:30.819511Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 318, pipe: [1:7559341170346254918:4549]
2025-10-09T21:44:30.820370Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 319 sessionId: src_id|c550871c-498b9ff7-4ed8c4e0-a6ed1837_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:30.822887Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:30.999601Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:30.999890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341170346254918:4549], now have 1 active actors on pipe
2025-10-09T21:44:30.999927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:30.999940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:31.000301Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:31.000940Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341170346254918:4549]
2025-10-09T21:44:31.000977Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:31.001047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 319 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c550871c-498b9ff7-4ed8c4e0-a6ed1837_0
2025-10-09T21:44:31.001247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 319 sessionId: src_id|c550871c-498b9ff7-4ed8c4e0-a6ed1837_0 is DEAD
2025-10-09T21:44:31.001449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c550871c-498b9ff7-4ed8c4e0-a6ed1837_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:31.001466Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c550871c-498b9ff7-4ed8c4e0-a6ed1837_0] Write session will restart in 0.006680s
2025-10-09T21:44:31.001494Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c550871c-498b9ff7-4ed8c4e0-a6ed1837_0] Write session: Do CDS request
2025-10-09T21:44:31.001501Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c550871c-498b9ff7-4ed8c4e0-a6ed1837_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:31.001386Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:31.012609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c550871c-498b9ff7-4ed8c4e0-a6ed1837_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:31.012824Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:31.012844Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 320
2025-10-09T21:44:31.012970Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 320 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:31.013001Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 320 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:31.013010Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=320 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:31.013015Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:31.013685Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 320 sessionId:  describe result for acl check
2025-10-09T21:44:31.013735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:31.013738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:31.013740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:31.013747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341174641222236:4666] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:31.014442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341174641222236:4666] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:31.037264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341174641222236:4666] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:31.044381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341174641222236:4666] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:31.044400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341174641222236:4666] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:31.044404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341174641222236:4666] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:31.044414Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 320 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:31.050171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:31.053479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:31.053738Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:31.053743Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:31.053755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:31.053809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:31.053850Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341174279411483:4327]
2025-10-09T21:44:31.054527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:31.057175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:31.057263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:31.057647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:31.057716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:31.057717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:31.057719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:31.057720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:31.057721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:31.057723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:31.057725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.057730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.057731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.057738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:31.057749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:31.057753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:31.057757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:31.057761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:31.057764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:31.057768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.057771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.057775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.057784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:31.057795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:31.057801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:31.057805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:31.057810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:31.057815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:31.057819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:31.057824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:31.057828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:31.057833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:31.057837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:31.057843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:31.057847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.057851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.057856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.057860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.057865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.057870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.057903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:31.081389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:31.083083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:31.083100Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:31.083103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:31.083108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:31.083115Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 319 [2:7559341174279411483:4327]
2025-10-09T21:44:31.083131Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:31.083152Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:31.083156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.083160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.083162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.083183Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:31.083189Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:31.083199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:31.083277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.083287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.083289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.083291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:31.083366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:31.083373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.083389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.083402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.083428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:31.083450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341174279411477:4327]
2025-10-09T21:44:31.083457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341174279411477:4327]
2025-10-09T21:44:31.083459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341174279411477:4327]
2025-10-09T21:44:31.083468Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:31.083482Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:31.083485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341174641222352:4666], now have 1 active actors on pipe
2025-10-09T21:44:31.083486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:31.083489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341174641222228:2432], now have 1 active actors on pipe
2025-10-09T21:44:31.083491Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:31.083499Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:31.083515Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:31.083527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:31.083572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341174279411477:4327]
2025-10-09T21:44:31.083586Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:31.084126Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:31.085824Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 319
2025-10-09T21:44:31.085879Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 319
2025-10-09T21:44:31.086539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:31.086558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:31.086600Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3191d64f-a4d80806-897fbe77-e290d9d1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:31.086616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.086622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.086625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:31.086639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.086658Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:31.086681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:31.091557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:31.091580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:31.091624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:31.091969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 320 partition: 0 MaxSeqNo: 25 sessionId: src_id|3191d64f-a4d80806-897fbe77-e290d9d1_0
2025-10-09T21:44:31.092417Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c550871c-498b9ff7-4ed8c4e0-a6ed1837_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3191d64f-a4d80806-897fbe77-e290d9d1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:31.095188Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3191d64f-a4d80806-897fbe77-e290d9d1_0] Write session: try to update token
2025-10-09T21:44:31.095213Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3191d64f-a4d80806-897fbe77-e290d9d1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:31.100898Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341174641222371:4549]
2025-10-09T21:44:31.102529Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 319, pipe: [1:7559341174641222371:4549]
2025-10-09T21:44:31.102384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341174641222371:4549], now have 1 active actors on pipe
2025-10-09T21:44:31.123744Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 320 sessionId: src_id|3191d64f-a4d80806-897fbe77-e290d9d1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:31.126230Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:31.126557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:31.126576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:31.126621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:31.126905Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:31.137948Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:31.137969Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:31.137971Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:31.137973Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:31.137983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341174279411477:4327]
2025-10-09T21:44:31.137988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341174279411477:4327]
2025-10-09T21:44:31.137992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341174279411477:4327]
2025-10-09T21:44:31.138061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:31.138062Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:31.138068Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:31.138069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.138072Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:31.138072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:31.138217Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:31.486116Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:31.494617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:31.494649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:31.494834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:31.494898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:31.494961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:31.495024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:31.495179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:31.495185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:31.495258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:31.495317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:31.495385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:31.495791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:31.495842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:31.495845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:31.495920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:31.495980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:31.496047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:31.496111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:31.496156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:31.496159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:31.496229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:31.496292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:31.496361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:31.496430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:31.496470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:31.496473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:31.496543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:31.496602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:31.496668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:31.496737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:31.496781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:31.496783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:31.496853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:31.496911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:31.496964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:31.497026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:31.497066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:31.497069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:31.497137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:31.497193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:31.497257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:31.497315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:31.497385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:31.497388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:31.497453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:31.497506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:31.497563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:31.497627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:31.497668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:31.497671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:31.497738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:31.497797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:31.497864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:31.497929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:31.497973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:31.497976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:31.498773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:31.498843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:31.498901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:31.498963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:31.499003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:31.499007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:31.501720Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3191d64f-a4d80806-897fbe77-e290d9d1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:31.501737Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3191d64f-a4d80806-897fbe77-e290d9d1_0] Write session will restart in 0.008534s
2025-10-09T21:44:31.501765Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3191d64f-a4d80806-897fbe77-e290d9d1_0] Write session: Do CDS request
2025-10-09T21:44:31.501773Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3191d64f-a4d80806-897fbe77-e290d9d1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:31.500980Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:31.500995Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:31.501025Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 320 reason: error at write: Disconnected sessionId: src_id|3191d64f-a4d80806-897fbe77-e290d9d1_0
2025-10-09T21:44:31.501217Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 320 sessionId: src_id|3191d64f-a4d80806-897fbe77-e290d9d1_0 is DEAD
2025-10-09T21:44:31.501379Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341174641222371:4549]
2025-10-09T21:44:31.501394Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:31.510302Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:31.511757Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3191d64f-a4d80806-897fbe77-e290d9d1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:31.514195Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:31.514210Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 321
2025-10-09T21:44:31.518291Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 321 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:31.518344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 321 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:31.518353Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=321 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:31.518371Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:31.519221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 321 sessionId:  describe result for acl check
2025-10-09T21:44:31.519256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:31.519258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:31.519260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:31.519265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341174641222396:4671] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:31.519842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341174641222396:4671] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:31.524656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341174641222396:4671] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:31.535437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341174641222396:4671] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:31.535465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341174641222396:4671] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:31.535470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341174641222396:4671] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:31.535482Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 321 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:31.553649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:31.556169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:31.556404Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:31.556409Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:31.556421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:31.556486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:31.556527Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341174279411630:4331]
2025-10-09T21:44:31.557081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:31.561724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:31.561800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:31.562217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:31.562280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:31.562281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:31.562282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:31.562284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:31.562285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:31.562287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:31.562288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.562289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.562290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.562296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:31.562307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:31.562310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:31.562314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:31.562318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:31.562322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:31.562326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.562330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.562333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.562342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:31.562351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:31.562371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:31.562377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:31.562382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:31.562388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:31.562392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:31.562398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:31.562407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:31.562412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:31.562417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:31.562425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:31.562430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.562435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.562440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.562444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.562448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.562453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.562482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:31.571653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:31.572969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:31.572976Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:31.572979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:31.572984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:31.572992Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 320 [2:7559341174279411630:4331]
2025-10-09T21:44:31.573009Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:31.573033Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:31.573037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.573040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.573042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.573061Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:31.573068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:31.573075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:31.573142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.573144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.573146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.573149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:31.573225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:31.573229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.573242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.573253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.573269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:31.573291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341174279411624:4331]
2025-10-09T21:44:31.573294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341174279411624:4331]
2025-10-09T21:44:31.573296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341174279411624:4331]
2025-10-09T21:44:31.573313Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:31.573329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:31.573346Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:31.573349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:31.573410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341174279411624:4331]
2025-10-09T21:44:31.573414Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:31.573693Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:31.573700Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:31.573702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:31.573709Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:31.574626Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 320
2025-10-09T21:44:31.574664Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 320
2025-10-09T21:44:31.578598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341174641222439:4671], now have 1 active actors on pipe
2025-10-09T21:44:31.578611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341174641222382:2432], now have 1 active actors on pipe
2025-10-09T21:44:31.578640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:31.578647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:31.578685Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e4544413-6adbbd8c-2d3e5cc9-e09f2c20_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:31.578700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.578707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.578710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:31.578723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.578742Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:31.578765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:31.582333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:31.582356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:31.582411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:31.583644Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3191d64f-a4d80806-897fbe77-e290d9d1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e4544413-6adbbd8c-2d3e5cc9-e09f2c20_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:31.582976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 321 partition: 0 MaxSeqNo: 25 sessionId: src_id|e4544413-6adbbd8c-2d3e5cc9-e09f2c20_0
2025-10-09T21:44:31.586463Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4544413-6adbbd8c-2d3e5cc9-e09f2c20_0] Write session: try to update token
2025-10-09T21:44:31.586485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4544413-6adbbd8c-2d3e5cc9-e09f2c20_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:31.605580Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341174641222530:4549]
2025-10-09T21:44:31.606300Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 320, pipe: [1:7559341174641222530:4549]
2025-10-09T21:44:31.606165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341174641222530:4549], now have 1 active actors on pipe
2025-10-09T21:44:31.611310Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:31.611325Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:31.611327Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:31.611329Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:31.611339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341174279411624:4331]
2025-10-09T21:44:31.611343Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341174279411624:4331]
2025-10-09T21:44:31.611347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341174279411624:4331]
2025-10-09T21:44:31.611400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:31.611408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.611407Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:31.611411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:31.611412Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:31.611419Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:31.611556Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:31.671651Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 321 sessionId: src_id|e4544413-6adbbd8c-2d3e5cc9-e09f2c20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:31.674333Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:31.805281Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:31.805295Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:31.805351Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 321 reason: error at write: Disconnected sessionId: src_id|e4544413-6adbbd8c-2d3e5cc9-e09f2c20_0
2025-10-09T21:44:31.805542Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 321 sessionId: src_id|e4544413-6adbbd8c-2d3e5cc9-e09f2c20_0 is DEAD
2025-10-09T21:44:31.805665Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341174641222530:4549]
2025-10-09T21:44:31.805678Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:31.805881Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4544413-6adbbd8c-2d3e5cc9-e09f2c20_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:31.805900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4544413-6adbbd8c-2d3e5cc9-e09f2c20_0] Write session will restart in 0.006099s
2025-10-09T21:44:31.805937Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4544413-6adbbd8c-2d3e5cc9-e09f2c20_0] Write session: Do CDS request
2025-10-09T21:44:31.805945Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4544413-6adbbd8c-2d3e5cc9-e09f2c20_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:31.803063Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:31.803360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:31.803370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:31.804764Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:31.814958Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4544413-6adbbd8c-2d3e5cc9-e09f2c20_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:31.818246Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:31.818266Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 322
2025-10-09T21:44:31.826261Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 322 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:31.826342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 322 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:31.826349Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=322 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:31.826354Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:31.826935Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 322 sessionId:  describe result for acl check
2025-10-09T21:44:31.826982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:31.826984Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:31.826986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:31.826993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341174641222577:4684] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:31.827548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341174641222577:4684] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:31.831069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341174641222577:4684] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:31.843277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341174641222577:4684] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:31.843297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341174641222577:4684] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:31.843301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341174641222577:4684] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:31.843311Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 322 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:31.855244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:31.859041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:31.859322Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:31.859326Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:31.859340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:31.859405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:31.859448Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341174279411812:4345]
2025-10-09T21:44:31.860030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:31.861292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:31.861365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:31.861782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:31.861859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:31.861860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:31.861861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:31.861863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:31.861864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:31.861865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:31.861867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.861868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.861870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.861876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:31.861887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:31.861892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:31.861896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:31.861899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:31.861904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:31.861908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.861912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.861916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.861923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:31.861933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:31.861940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:31.861948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:31.861952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:31.861957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:31.861961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:31.861965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:31.861969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:31.861974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:31.861978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:31.861982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:31.861987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.861991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.861995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.861999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.862004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.862008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.862053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:31.871559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:31.872710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:31.872718Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:31.872722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:31.872726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:31.872736Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 321 [2:7559341174279411812:4345]
2025-10-09T21:44:31.872753Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:31.872775Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:31.872778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.872782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.872785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.872822Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:31.872829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:31.872837Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:31.872905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.872907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.872909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.872912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:31.872991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:31.872994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.873007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:31.873017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:31.873034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:31.873056Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341174279411806:4345]
2025-10-09T21:44:31.873059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341174279411806:4345]
2025-10-09T21:44:31.873061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341174279411806:4345]
2025-10-09T21:44:31.873077Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:31.873093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:31.873111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:31.873115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:31.873161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341174279411806:4345]
2025-10-09T21:44:31.873164Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:31.873657Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:31.873665Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:31.873667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:31.873674Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:31.874499Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 321
2025-10-09T21:44:31.874534Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 321
2025-10-09T21:44:31.874967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341174641222664:4684], now have 1 active actors on pipe
2025-10-09T21:44:31.874976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341174641222562:2432], now have 1 active actors on pipe
2025-10-09T21:44:31.875004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:31.875010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:31.875045Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3b71d7d2-c8ce9ccd-b0101cc-97475715_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:31.875057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.875062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.875065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:31.875078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.875096Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:31.875116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:31.875752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:31.875760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:31.875791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:31.875955Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 322 partition: 0 MaxSeqNo: 25 sessionId: src_id|3b71d7d2-c8ce9ccd-b0101cc-97475715_0
2025-10-09T21:44:31.878428Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4544413-6adbbd8c-2d3e5cc9-e09f2c20_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3b71d7d2-c8ce9ccd-b0101cc-97475715_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:31.881307Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3b71d7d2-c8ce9ccd-b0101cc-97475715_0] Write session: try to update token
2025-10-09T21:44:31.881335Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3b71d7d2-c8ce9ccd-b0101cc-97475715_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:31.905425Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:31.905437Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:31.905449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341174279411806:4345]
2025-10-09T21:44:31.905501Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:31.905709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:31.905867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:31.906013Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:31.906307Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341174641222715:4549]
2025-10-09T21:44:31.906518Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:31.910212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:31.910758Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 321, pipe: [1:7559341174641222715:4549]
2025-10-09T21:44:31.916340Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 322 sessionId: src_id|3b71d7d2-c8ce9ccd-b0101cc-97475715_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:31.918984Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:31.919598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:31.919685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341174641222715:4549], now have 1 active actors on pipe
2025-10-09T21:44:31.919709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:31.919722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:31.919744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:31.919918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:31.920225Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:31.935661Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:31.935676Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:31.935678Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:31.935680Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:31.935690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341174279411806:4345]
2025-10-09T21:44:31.935694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341174279411806:4345]
2025-10-09T21:44:31.935698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341174279411806:4345]
2025-10-09T21:44:31.935743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:31.935750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:31.935753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:31.935899Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:31.938520Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:31.938532Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:31.938536Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:32.138782Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:32.147913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:32.147947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:32.148132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.148203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.148267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.148330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.148388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.148397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.148463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.148533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.148603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.148670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.148721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.148729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.148798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.148860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.148926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.148993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.149045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.149048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.149117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.149180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.149242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.149315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.149357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.149360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.149437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.149520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.149585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.149648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.149709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.149712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.149785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.149850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.149917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.149986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.150260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.150269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.150343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.150424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.150489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.150557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.150601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.150604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.150674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.150732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.150792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.150858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.150899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.150902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.150973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.151029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.151094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.151163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.151202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.151205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.151271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.151339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.151407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.151471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.151512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.151515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.154098Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b71d7d2-c8ce9ccd-b0101cc-97475715_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:32.154113Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b71d7d2-c8ce9ccd-b0101cc-97475715_0] Write session will restart in 0.006248s
2025-10-09T21:44:32.154150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b71d7d2-c8ce9ccd-b0101cc-97475715_0] Write session: Do CDS request
2025-10-09T21:44:32.154158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b71d7d2-c8ce9ccd-b0101cc-97475715_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:32.153700Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:32.153717Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:32.153749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 322 reason: error at write: Disconnected sessionId: src_id|3b71d7d2-c8ce9ccd-b0101cc-97475715_0
2025-10-09T21:44:32.153958Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 322 sessionId: src_id|3b71d7d2-c8ce9ccd-b0101cc-97475715_0 is DEAD
2025-10-09T21:44:32.154140Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341174641222715:4549]
2025-10-09T21:44:32.154154Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:32.155846Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:32.162156Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3b71d7d2-c8ce9ccd-b0101cc-97475715_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:32.162935Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:32.162954Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 323
2025-10-09T21:44:32.163259Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 323 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:32.163294Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 323 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:32.163300Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=323 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:32.163304Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:32.164107Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 323 sessionId:  describe result for acl check
2025-10-09T21:44:32.164142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:32.164145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:32.164146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:32.164152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341178936190036:4689] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:32.164725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341178936190036:4689] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:32.172470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341178936190036:4689] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:32.179538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341178936190036:4689] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:32.179560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341178936190036:4689] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:32.179564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341178936190036:4689] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:32.179575Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 323 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:32.184417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:32.186931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:32.187183Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:32.187187Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:32.187198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:32.187241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:32.187273Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341178574379253:4348]
2025-10-09T21:44:32.187680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:32.187987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:32.188027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:32.188252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:32.188284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:32.188285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:32.188285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:32.188286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:32.188287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:32.188288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:32.188288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.188289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.188290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.188293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:32.188300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:32.188302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:32.188305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:32.188307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:32.188309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:32.188311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.188313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.188315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.188320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:32.188325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:32.188330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:32.188334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:32.188337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:32.188340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:32.188343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:32.188347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:32.188350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:32.188353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:32.188355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:32.188358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:32.188361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.188364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.188367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.188370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.188374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.188377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.188395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:32.196899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:32.198109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:32.198117Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:32.198120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:32.198125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:32.198134Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 322 [2:7559341178574379253:4348]
2025-10-09T21:44:32.198151Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:32.198174Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:32.198178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.198182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.198184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.198203Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:32.198211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:32.198219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:32.198294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.198296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.198298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.198301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:32.198403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:32.198407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.198423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.198434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.198450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:32.198474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341178574379248:4348]
2025-10-09T21:44:32.198477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341178574379248:4348]
2025-10-09T21:44:32.198479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341178574379248:4348]
2025-10-09T21:44:32.198489Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:32.198506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:32.198523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:32.198526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:32.198573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341178574379248:4348]
2025-10-09T21:44:32.198576Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:32.199326Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 322
2025-10-09T21:44:32.199378Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 322
2025-10-09T21:44:32.198952Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:32.198958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:32.198959Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:32.198964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:32.203827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341178936190023:2432], now have 1 active actors on pipe
2025-10-09T21:44:32.203845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341178936190134:4689], now have 1 active actors on pipe
2025-10-09T21:44:32.203879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:32.203886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:32.203932Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b0a01cdd-cdfce82c-c670891-9608f29d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:32.203948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.203956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.203959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:32.203975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.203998Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:32.204019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:32.207307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:32.207328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:32.207392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:32.207666Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 323 partition: 0 MaxSeqNo: 25 sessionId: src_id|b0a01cdd-cdfce82c-c670891-9608f29d_0
2025-10-09T21:44:32.208110Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b71d7d2-c8ce9ccd-b0101cc-97475715_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b0a01cdd-cdfce82c-c670891-9608f29d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:32.210955Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0a01cdd-cdfce82c-c670891-9608f29d_0] Write session: try to update token
2025-10-09T21:44:32.210985Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0a01cdd-cdfce82c-c670891-9608f29d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:32.222438Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:32.222455Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:32.222458Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:32.222461Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:32.222472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341178574379248:4348]
2025-10-09T21:44:32.222478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341178574379248:4348]
2025-10-09T21:44:32.222482Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341178574379248:4348]
2025-10-09T21:44:32.222533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:32.222542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.222545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:32.222840Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:32.227114Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:32.227128Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:32.227132Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:32.245303Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 323 sessionId: src_id|b0a01cdd-cdfce82c-c670891-9608f29d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:32.248154Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:32.252435Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341178936190174:4549]
2025-10-09T21:44:32.253224Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 322, pipe: [1:7559341178936190174:4549]
2025-10-09T21:44:32.431256Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:32.432069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:32.432080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:32.432104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341178936190174:4549], now have 1 active actors on pipe
2025-10-09T21:44:32.433085Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:32.439179Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0a01cdd-cdfce82c-c670891-9608f29d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:32.439194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0a01cdd-cdfce82c-c670891-9608f29d_0] Write session will restart in 0.006812s
2025-10-09T21:44:32.439225Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0a01cdd-cdfce82c-c670891-9608f29d_0] Write session: Do CDS request
2025-10-09T21:44:32.439232Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0a01cdd-cdfce82c-c670891-9608f29d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:32.438208Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341178936190174:4549]
2025-10-09T21:44:32.438252Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:32.438293Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:32.438403Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 323 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b0a01cdd-cdfce82c-c670891-9608f29d_0
2025-10-09T21:44:32.438597Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 323 sessionId: src_id|b0a01cdd-cdfce82c-c670891-9608f29d_0 is DEAD
2025-10-09T21:44:32.447781Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0a01cdd-cdfce82c-c670891-9608f29d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:32.447880Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:32.447899Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 324
2025-10-09T21:44:32.448082Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 324 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:32.448121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 324 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:32.448126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=324 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:32.448131Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:32.448647Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 324 sessionId:  describe result for acl check
2025-10-09T21:44:32.448689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:32.448692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:32.448694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:32.448700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341178936190185:4693] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:32.449258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341178936190185:4693] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:32.456573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341178936190185:4693] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:32.465837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341178936190185:4693] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:32.465862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341178936190185:4693] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:32.465866Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341178936190185:4693] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:32.465877Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 324 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:32.488889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:32.493532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:32.493774Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:32.493782Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:32.493795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:32.493853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:32.493889Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341178574379399:4351]
2025-10-09T21:44:32.494533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:32.495391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:32.495441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:32.495772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:32.495830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:32.495831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:32.495832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:32.495833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:32.495834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:32.495835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:32.495836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.495837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.495838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.495844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:32.495854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:32.495858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:32.495862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:32.495865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:32.495869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:32.495873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.495876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.495880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.495888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:32.495896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:32.495902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:32.495907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:32.495914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:32.495919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:32.495923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:32.495928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:32.495933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:32.495938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:32.495942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:32.495947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:32.495952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.495957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.495963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.495969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.495973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.495978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.496004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:32.506499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:32.507567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:32.507573Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:32.507577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:32.507582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:32.507591Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 323 [2:7559341178574379399:4351]
2025-10-09T21:44:32.507607Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:32.507630Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:32.507634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.507638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.507641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.507661Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:32.507667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:32.507675Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:32.507743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.507746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.507748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.507752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:32.507831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:32.507835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.507851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.507864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.507884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:32.507905Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341178574379394:4351]
2025-10-09T21:44:32.507909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341178574379394:4351]
2025-10-09T21:44:32.507912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341178574379394:4351]
2025-10-09T21:44:32.507923Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:32.507940Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:32.507955Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:32.507956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:32.507959Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:32.507960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:32.507964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:32.508007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341178574379394:4351]
2025-10-09T21:44:32.508155Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 323
2025-10-09T21:44:32.508198Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 323
2025-10-09T21:44:32.508014Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:32.508027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341178936190235:4693], now have 1 active actors on pipe
2025-10-09T21:44:32.508032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341178936190178:2432], now have 1 active actors on pipe
2025-10-09T21:44:32.508356Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:32.508784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:32.508802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:32.508839Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|93140745-191c3ff9-b67b304f-c1044ba5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:32.508853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.508859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.508868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:32.508881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.508898Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:32.508923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:32.515261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 324 partition: 0 MaxSeqNo: 25 sessionId: src_id|93140745-191c3ff9-b67b304f-c1044ba5_0
2025-10-09T21:44:32.514172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:32.514195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:32.514232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:32.518768Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0a01cdd-cdfce82c-c670891-9608f29d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|93140745-191c3ff9-b67b304f-c1044ba5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:32.521227Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|93140745-191c3ff9-b67b304f-c1044ba5_0] Write session: try to update token
2025-10-09T21:44:32.521248Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|93140745-191c3ff9-b67b304f-c1044ba5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:32.526520Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:32.526537Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:32.526548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341178574379394:4351]
2025-10-09T21:44:32.526598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:32.526773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:32.526912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:32.527066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:32.527204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:32.527385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:32.527424Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:32.528821Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:32.537265Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341178936190332:4549]
2025-10-09T21:44:32.537961Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 323, pipe: [1:7559341178936190332:4549]
2025-10-09T21:44:32.537908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341178936190332:4549], now have 1 active actors on pipe
2025-10-09T21:44:32.551604Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 324 sessionId: src_id|93140745-191c3ff9-b67b304f-c1044ba5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:32.554170Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:32.554463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:32.554499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:32.554545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:32.555780Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:32.564041Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:32.564059Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:32.564061Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:32.564063Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:32.564075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341178574379394:4351]
2025-10-09T21:44:32.564079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341178574379394:4351]
2025-10-09T21:44:32.564083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341178574379394:4351]
2025-10-09T21:44:32.564131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:32.564139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.564142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:32.564303Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:32.569372Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:32.569384Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:32.569388Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:32.769222Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:32.778321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:32.778348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:32.778535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.778592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.778655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.778720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.778761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.778766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.778837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.778895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.778964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.779028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.779069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.779073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.779141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.779201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.779256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.779318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.779358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.779362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.779433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.779494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.779560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.779624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.779667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.779670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.779738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.779798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.779860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.779919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.779963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.779967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.780045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.780104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.780164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.780221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.780266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.780270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.780331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.780383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.780454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.780513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.780555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.780558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.780619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.780673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.780734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.780796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.780838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.780841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.780906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.780961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.781017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.781081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.781127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.781131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.781198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:32.781256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:32.781319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:32.781379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:32.781419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:32.781422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:32.783416Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93140745-191c3ff9-b67b304f-c1044ba5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:32.783445Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93140745-191c3ff9-b67b304f-c1044ba5_0] Write session will restart in 0.008780s
2025-10-09T21:44:32.783469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93140745-191c3ff9-b67b304f-c1044ba5_0] Write session: Do CDS request
2025-10-09T21:44:32.783476Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93140745-191c3ff9-b67b304f-c1044ba5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:32.783053Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:32.783068Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:32.783098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 324 reason: error at write: Disconnected sessionId: src_id|93140745-191c3ff9-b67b304f-c1044ba5_0
2025-10-09T21:44:32.783276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 324 sessionId: src_id|93140745-191c3ff9-b67b304f-c1044ba5_0 is DEAD
2025-10-09T21:44:32.783419Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:32.783451Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341178936190332:4549]
2025-10-09T21:44:32.791667Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:32.793482Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|93140745-191c3ff9-b67b304f-c1044ba5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:32.793793Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:32.793810Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 325
2025-10-09T21:44:32.794003Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 325 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:32.794051Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 325 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:32.794057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=325 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:32.794060Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:32.795810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 325 sessionId:  describe result for acl check
2025-10-09T21:44:32.795853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:32.795855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:32.795857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:32.795863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341178936190369:4698] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:32.796440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341178936190369:4698] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:32.802012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341178936190369:4698] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:32.811077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341178936190369:4698] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:32.811097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341178936190369:4698] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:32.811103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341178936190369:4698] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:32.811115Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 325 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:32.825789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:32.827376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:32.827737Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:32.827742Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:32.827757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:32.827827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:32.827866Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341178574379550:4353]
2025-10-09T21:44:32.828492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:32.837605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:32.837707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:32.838123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:32.838195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:32.838196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:32.838197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:32.838199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:32.838200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:32.838201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:32.838202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.838203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.838204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.838211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:32.838222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:32.838225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:32.838229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:32.838232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:32.838236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:32.838239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.838243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.838246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.838254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:32.838263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:32.838269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:32.838274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:32.838280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:32.838284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:32.838289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:32.838293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:32.838297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:32.838303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:32.838307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:32.838312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:32.838319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.838324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.838328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.838333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.838337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.838343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.838385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:32.853821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:32.855016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:32.855024Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:32.855026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:32.855031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:32.855039Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 324 [2:7559341178574379550:4353]
2025-10-09T21:44:32.855056Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:32.855077Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:32.855081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.855085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.855087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.855108Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:32.855115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:32.855122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:32.855187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.855189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.855192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.855196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:32.855278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:32.855282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.855295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:32.855308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:32.855327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:32.855349Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341178574379543:4353]
2025-10-09T21:44:32.855353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341178574379543:4353]
2025-10-09T21:44:32.855355Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341178574379543:4353]
2025-10-09T21:44:32.855368Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:32.855385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:32.855402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:32.855405Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:32.855449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341178574379543:4353]
2025-10-09T21:44:32.855451Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:32.856197Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:32.856204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:32.856207Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:32.856214Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:32.858157Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 324
2025-10-09T21:44:32.858197Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 324
2025-10-09T21:44:32.860880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341178936190488:4698], now have 1 active actors on pipe
2025-10-09T21:44:32.860895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341178936190344:2432], now have 1 active actors on pipe
2025-10-09T21:44:32.860918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:32.860926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:32.860971Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3396c932-45ece6a0-7d6ec162-5fbd9083_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:32.860985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.860994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.860998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:32.861012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.861033Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:32.861057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:32.865983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:32.866003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:32.866058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:32.866732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 325 partition: 0 MaxSeqNo: 25 sessionId: src_id|3396c932-45ece6a0-7d6ec162-5fbd9083_0
2025-10-09T21:44:32.867180Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93140745-191c3ff9-b67b304f-c1044ba5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3396c932-45ece6a0-7d6ec162-5fbd9083_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:32.869835Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3396c932-45ece6a0-7d6ec162-5fbd9083_0] Write session: try to update token
2025-10-09T21:44:32.869857Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3396c932-45ece6a0-7d6ec162-5fbd9083_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:32.883685Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341178936190518:4549]
2025-10-09T21:44:32.885269Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 324, pipe: [1:7559341178936190518:4549]
2025-10-09T21:44:32.886735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341178936190518:4549], now have 1 active actors on pipe
2025-10-09T21:44:32.904375Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:32.904389Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:32.904392Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:32.904394Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:32.904404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341178574379543:4353]
2025-10-09T21:44:32.904409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341178574379543:4353]
2025-10-09T21:44:32.904413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341178574379543:4353]
2025-10-09T21:44:32.904461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:32.904468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:32.904472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:32.904617Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:32.906827Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:32.906839Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:32.906844Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:32.914919Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 325 sessionId: src_id|3396c932-45ece6a0-7d6ec162-5fbd9083_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:32.917605Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:33.124168Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:33.124518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:33.124538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:33.125501Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341178936190518:4549]
2025-10-09T21:44:33.125509Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:33.125538Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:33.125541Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:33.125612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 325 reason: error at write: Disconnected sessionId: src_id|3396c932-45ece6a0-7d6ec162-5fbd9083_0
2025-10-09T21:44:33.125803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 325 sessionId: src_id|3396c932-45ece6a0-7d6ec162-5fbd9083_0 is DEAD
2025-10-09T21:44:33.126266Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3396c932-45ece6a0-7d6ec162-5fbd9083_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:33.126279Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3396c932-45ece6a0-7d6ec162-5fbd9083_0] Write session will restart in 0.005603s
2025-10-09T21:44:33.126304Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3396c932-45ece6a0-7d6ec162-5fbd9083_0] Write session: Do CDS request
2025-10-09T21:44:33.126310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3396c932-45ece6a0-7d6ec162-5fbd9083_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:33.126189Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:33.134495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3396c932-45ece6a0-7d6ec162-5fbd9083_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:33.134857Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:33.134876Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 326
2025-10-09T21:44:33.137623Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 326 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:33.137720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 326 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:33.137727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=326 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:33.137731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:33.138206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 326 sessionId:  describe result for acl check
2025-10-09T21:44:33.138254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:33.138256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:33.138257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:33.138264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341183231157847:4710] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:33.139204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341183231157847:4710] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:33.144096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341183231157847:4710] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:33.153273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:33.155745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:33.156005Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:33.156017Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:33.156031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:33.156095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:33.156144Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341182869347027:4368]
2025-10-09T21:44:33.156776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:33.157399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:33.157457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:33.157797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:33.157868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:33.157869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:33.157870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:33.157871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:33.157872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:33.157873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:33.157874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.157875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.157876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.157882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:33.157893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:33.157896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:33.157900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:33.157904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:33.157908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:33.157912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.157915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.157918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.157928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:33.157936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:33.157942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:33.157947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:33.157951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:33.157957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:33.157964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:33.157970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:33.157989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:33.157995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:33.158000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:33.158010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:33.158018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.158027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.158044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.158050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.158060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.158065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.158093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:33.173746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:33.170223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341183231157847:4710] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:33.170246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341183231157847:4710] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:33.170250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341183231157847:4710] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:33.170262Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 326 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:33.174950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:33.174968Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:33.174971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:33.174976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:33.174986Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 325 [2:7559341182869347027:4368]
2025-10-09T21:44:33.175006Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:33.175030Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:33.175033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:33.175037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:33.175039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:33.175062Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:33.175068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:33.175078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:33.175156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.175158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.175160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.175163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:33.175249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:33.175252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.175268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.175279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.175296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:33.175318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341182869347021:4368]
2025-10-09T21:44:33.175320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341182869347021:4368]
2025-10-09T21:44:33.175323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341182869347021:4368]
2025-10-09T21:44:33.175341Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:33.175360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:33.175375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:33.175379Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:33.175426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341182869347021:4368]
2025-10-09T21:44:33.175436Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:33.175773Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:33.175778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341183231157978:4710], now have 1 active actors on pipe
2025-10-09T21:44:33.175783Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:33.175784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341183231157832:2432], now have 1 active actors on pipe
2025-10-09T21:44:33.175786Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:33.175793Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:33.177804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:33.177834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:33.176932Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 325
2025-10-09T21:44:33.176969Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 325
2025-10-09T21:44:33.177877Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|94551eb8-5f6e7043-9211a6ea-8f7deaa2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:33.177895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:33.177901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:33.177904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:33.177927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:33.177946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:33.177969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:33.179437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:33.179449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:33.179474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:33.179609Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 326 partition: 0 MaxSeqNo: 25 sessionId: src_id|94551eb8-5f6e7043-9211a6ea-8f7deaa2_0
2025-10-09T21:44:33.182967Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3396c932-45ece6a0-7d6ec162-5fbd9083_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|94551eb8-5f6e7043-9211a6ea-8f7deaa2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:33.185536Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94551eb8-5f6e7043-9211a6ea-8f7deaa2_0] Write session: try to update token
2025-10-09T21:44:33.185558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94551eb8-5f6e7043-9211a6ea-8f7deaa2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:33.193842Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:33.193860Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:33.193863Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:33.193865Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:33.193874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341182869347021:4368]
2025-10-09T21:44:33.193879Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341182869347021:4368]
2025-10-09T21:44:33.193883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341182869347021:4368]
2025-10-09T21:44:33.193928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:33.193936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.193939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:33.194091Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:33.196665Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:33.196677Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:33.196681Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:33.225774Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341183231157986:4549]
2025-10-09T21:44:33.226598Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 325, pipe: [1:7559341183231157986:4549]
2025-10-09T21:44:33.236829Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 326 sessionId: src_id|94551eb8-5f6e7043-9211a6ea-8f7deaa2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:33.239422Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:33.430873Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:33.431286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341183231157986:4549], now have 1 active actors on pipe
2025-10-09T21:44:33.431322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:33.431330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:33.434381Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:33.434675Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:33.434691Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:33.434758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 326 reason: error at write: Disconnected sessionId: src_id|94551eb8-5f6e7043-9211a6ea-8f7deaa2_0
2025-10-09T21:44:33.434937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 326 sessionId: src_id|94551eb8-5f6e7043-9211a6ea-8f7deaa2_0 is DEAD
2025-10-09T21:44:33.435074Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341183231157986:4549]
2025-10-09T21:44:33.435088Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:33.435903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94551eb8-5f6e7043-9211a6ea-8f7deaa2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:33.435910Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94551eb8-5f6e7043-9211a6ea-8f7deaa2_0] Write session will restart in 0.007304s
2025-10-09T21:44:33.435937Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94551eb8-5f6e7043-9211a6ea-8f7deaa2_0] Write session: Do CDS request
2025-10-09T21:44:33.435944Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94551eb8-5f6e7043-9211a6ea-8f7deaa2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:33.446904Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94551eb8-5f6e7043-9211a6ea-8f7deaa2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:33.447232Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:33.447250Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 327
2025-10-09T21:44:33.447802Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 327 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:33.447839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 327 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:33.447845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=327 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:33.447849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:33.448407Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 327 sessionId:  describe result for acl check
2025-10-09T21:44:33.448466Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:33.448468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:33.448469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:33.448475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341183231158000:4714] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:33.449052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341183231158000:4714] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:33.455136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341183231158000:4714] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:33.459092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341183231158000:4714] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:33.459109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341183231158000:4714] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:33.459113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341183231158000:4714] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:33.459122Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 327 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:33.470100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:33.472268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:33.472476Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:33.472481Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:33.472493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:33.472553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:33.472588Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341182869347178:4371]
2025-10-09T21:44:33.473242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:33.473744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:33.473784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:33.474144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:33.474193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:33.474194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:33.474195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:33.474197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:33.474198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:33.474199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:33.474200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.474201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.474203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.474209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:33.474219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:33.474224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:33.474228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:33.474233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:33.474237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:33.474240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.474243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.474247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.474254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:33.474264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:33.474270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:33.474276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:33.474286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:33.474293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:33.474297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:33.474302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:33.474306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:33.474310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:33.474314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:33.474319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:33.474323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.474328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.474332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.474337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.474343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.474349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.474390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:33.483575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:33.485022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:33.485028Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:33.485031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:33.485035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:33.485044Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 326 [2:7559341182869347178:4371]
2025-10-09T21:44:33.485059Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:33.485078Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:33.485082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:33.485085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:33.485087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:33.485103Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:33.485109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:33.485119Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:33.485179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.485181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.485183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.485186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:33.485256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:33.485258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.485270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.485280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.485296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:33.485316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341182869347172:4371]
2025-10-09T21:44:33.485319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341182869347172:4371]
2025-10-09T21:44:33.485321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341182869347172:4371]
2025-10-09T21:44:33.485334Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:33.485349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:33.485363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:33.485366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:33.485406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341182869347172:4371]
2025-10-09T21:44:33.485409Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:33.485882Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:33.485888Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:33.485890Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:33.485895Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:33.486343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341183231158064:4714], now have 1 active actors on pipe
2025-10-09T21:44:33.486350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341183231157989:2432], now have 1 active actors on pipe
2025-10-09T21:44:33.486864Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 326
2025-10-09T21:44:33.486894Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 326
2025-10-09T21:44:33.487360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:33.487371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:33.487405Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1ed29e35-1198b998-ac6174f9-7dd195c9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:33.487415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:33.487420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:33.487422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:33.487433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:33.487446Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:33.487475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:33.488208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:33.488215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:33.488231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:33.488892Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94551eb8-5f6e7043-9211a6ea-8f7deaa2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1ed29e35-1198b998-ac6174f9-7dd195c9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:33.491330Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ed29e35-1198b998-ac6174f9-7dd195c9_0] Write session: try to update token
2025-10-09T21:44:33.491347Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ed29e35-1198b998-ac6174f9-7dd195c9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:33.488419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 327 partition: 0 MaxSeqNo: 25 sessionId: src_id|1ed29e35-1198b998-ac6174f9-7dd195c9_0
2025-10-09T21:44:33.505085Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:33.505103Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:33.505106Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:33.505108Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:33.505119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341182869347172:4371]
2025-10-09T21:44:33.505125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341182869347172:4371]
2025-10-09T21:44:33.505129Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341182869347172:4371]
2025-10-09T21:44:33.505180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:33.505188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.505191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:33.505339Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:33.510982Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:33.510994Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:33.510998Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:33.521686Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 327 sessionId: src_id|1ed29e35-1198b998-ac6174f9-7dd195c9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:33.524303Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:33.542105Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341183231158146:4549]
2025-10-09T21:44:33.545714Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 326, pipe: [1:7559341183231158146:4549]
2025-10-09T21:44:33.693465Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:33.694775Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:33.694841Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:33.694844Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:33.694915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 327 reason: error at write: Disconnected sessionId: src_id|1ed29e35-1198b998-ac6174f9-7dd195c9_0
2025-10-09T21:44:33.695105Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341183231158146:4549]
2025-10-09T21:44:33.695133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 327 sessionId: src_id|1ed29e35-1198b998-ac6174f9-7dd195c9_0 is DEAD
2025-10-09T21:44:33.695299Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ed29e35-1198b998-ac6174f9-7dd195c9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:33.695502Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ed29e35-1198b998-ac6174f9-7dd195c9_0] Write session will restart in 0.006797s
2025-10-09T21:44:33.695523Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ed29e35-1198b998-ac6174f9-7dd195c9_0] Write session: Do CDS request
2025-10-09T21:44:33.695528Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ed29e35-1198b998-ac6174f9-7dd195c9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:33.694219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:33.694236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:33.694288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341183231158146:4549], now have 1 active actors on pipe
2025-10-09T21:44:33.695169Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:33.703812Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ed29e35-1198b998-ac6174f9-7dd195c9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:33.705537Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:33.705555Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 328
2025-10-09T21:44:33.705806Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 328 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:33.705846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 328 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:33.705852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=328 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:33.705856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:33.706740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 328 sessionId:  describe result for acl check
2025-10-09T21:44:33.706806Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:33.706809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:33.706811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:33.706817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341183231158162:4719] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:33.707595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341183231158162:4719] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:33.713794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341183231158162:4719] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:33.720883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341183231158162:4719] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:33.720903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341183231158162:4719] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:33.720907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341183231158162:4719] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:33.720918Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 328 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:33.725532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:33.735963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:33.736239Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:33.736244Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:33.736257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:33.736324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:33.736371Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341182869347328:4373]
2025-10-09T21:44:33.737039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:33.738330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:33.738416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:33.738780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:33.738850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:33.738851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:33.738853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:33.738854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:33.738856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:33.738858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:33.738859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.738861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.738862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.738869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:33.738881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:33.738885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:33.738889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:33.738893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:33.738896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:33.738899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.738904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.738908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.738916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:33.738925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:33.738932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:33.738939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:33.738944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:33.738950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:33.738954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:33.738960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:33.738965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:33.738971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:33.738975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:33.738981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:33.738987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.738992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.738998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.739003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.739007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.739011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.739058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:33.750181Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 327
2025-10-09T21:44:33.750224Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 327
2025-10-09T21:44:33.748072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:33.749146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:33.749152Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:33.749156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:33.749161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:33.749169Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 327 [2:7559341182869347328:4373]
2025-10-09T21:44:33.749183Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:33.749204Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:33.749207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:33.749210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:33.749213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:33.749232Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:33.749238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:33.749248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:33.749309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.749311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.749313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.749316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:33.749395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:33.749397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.749411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:33.749424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:33.749442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:33.749462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341182869347322:4373]
2025-10-09T21:44:33.749465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341182869347322:4373]
2025-10-09T21:44:33.749467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341182869347322:4373]
2025-10-09T21:44:33.749476Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:33.749492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:33.749503Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:33.749506Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:33.749507Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:33.749509Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:33.749510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:33.749522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341183231158289:4719], now have 1 active actors on pipe
2025-10-09T21:44:33.749526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341183231158149:2432], now have 1 active actors on pipe
2025-10-09T21:44:33.749567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341182869347322:4373]
2025-10-09T21:44:33.749570Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:33.749576Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:33.750643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:33.750662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:33.750706Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bba7e71f-70cb21be-d11518de-3ea63e78_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:33.750718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:33.750724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:33.750727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:33.750740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:33.750766Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:33.750786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:33.751338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:33.751350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:33.751371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:33.751491Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 328 partition: 0 MaxSeqNo: 25 sessionId: src_id|bba7e71f-70cb21be-d11518de-3ea63e78_0
2025-10-09T21:44:33.751912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ed29e35-1198b998-ac6174f9-7dd195c9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bba7e71f-70cb21be-d11518de-3ea63e78_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:33.754554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bba7e71f-70cb21be-d11518de-3ea63e78_0] Write session: try to update token
2025-10-09T21:44:33.754573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bba7e71f-70cb21be-d11518de-3ea63e78_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:33.765835Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:33.765850Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:33.765860Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341182869347322:4373]
2025-10-09T21:44:33.765909Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:33.765911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:33.766132Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:33.766306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:33.766488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:33.766630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:33.766820Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:33.766863Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:33.779882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:33.779906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:33.779950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:33.777166Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 328 sessionId: src_id|bba7e71f-70cb21be-d11518de-3ea63e78_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:33.779669Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:33.783541Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:33.786236Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:33.786255Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:33.786257Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:33.786259Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:33.786272Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341182869347322:4373]
2025-10-09T21:44:33.786276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341182869347322:4373]
2025-10-09T21:44:33.786280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341182869347322:4373]
2025-10-09T21:44:33.786860Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:33.786873Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:33.786879Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:33.786977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:33.786985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:33.786993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:33.787141Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:33.793586Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341183231158306:4549]
2025-10-09T21:44:33.794346Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 327, pipe: [1:7559341183231158306:4549]
2025-10-09T21:44:33.961397Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:33.963005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341183231158306:4549], now have 1 active actors on pipe
2025-10-09T21:44:33.970650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:33.970674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:33.970840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:33.970895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:33.970949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:33.971006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:33.971048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:33.971053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:33.971150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:33.971201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:33.971255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:33.971316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:33.971353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:33.971356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:33.971419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:33.971469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:33.971523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:33.971578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:33.971614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:33.971616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:33.971680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:33.971734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:33.971788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:33.971841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:33.971880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:33.971882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:33.971951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:33.972006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:33.972066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:33.972127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:33.972181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:33.972183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:33.972296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:33.972349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:33.972403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:33.972456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:33.972497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:33.972500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:33.972564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:33.972615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:33.972679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:33.972734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:33.972772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:33.972775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:33.972841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:33.972895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:33.972951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:33.973005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:33.973044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:33.973047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:33.973111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:33.973164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:33.973224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:33.973283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:33.975453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:33.975465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:33.975541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:33.975598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:33.975662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:33.975725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:33.975766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:33.975768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:33.976330Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341183231158306:4549]
2025-10-09T21:44:33.976447Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:33.976456Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:33.976490Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 328 reason: error at write: Disconnected sessionId: src_id|bba7e71f-70cb21be-d11518de-3ea63e78_0
2025-10-09T21:44:33.976718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 328 sessionId: src_id|bba7e71f-70cb21be-d11518de-3ea63e78_0 is DEAD
2025-10-09T21:44:33.976932Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:33.977157Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bba7e71f-70cb21be-d11518de-3ea63e78_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:33.977169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bba7e71f-70cb21be-d11518de-3ea63e78_0] Write session will restart in 0.006394s
2025-10-09T21:44:33.977194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bba7e71f-70cb21be-d11518de-3ea63e78_0] Write session: Do CDS request
2025-10-09T21:44:33.977211Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bba7e71f-70cb21be-d11518de-3ea63e78_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:33.977569Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:33.986693Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bba7e71f-70cb21be-d11518de-3ea63e78_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:33.990838Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:33.990860Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 329
2025-10-09T21:44:33.991041Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 329 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:33.991079Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 329 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:33.991091Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=329 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:33.991095Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:33.992644Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 329 sessionId:  describe result for acl check
2025-10-09T21:44:33.992689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:33.992692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:33.992694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:33.992700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341183231158351:4724] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:33.994070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341183231158351:4724] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:34.001166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341183231158351:4724] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:34.025092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341183231158351:4724] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:34.025114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341183231158351:4724] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:34.025118Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341183231158351:4724] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:34.025137Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 329 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:34.029177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:34.033904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:34.034184Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:34.034198Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:34.034213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:34.034278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:34.034316Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341187164314788:4375]
2025-10-09T21:44:34.034966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:34.035463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:34.035545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:34.035909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:34.035961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:34.035962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:34.035964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:34.035965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:34.035966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:34.035967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:34.035969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.035970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.035971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.035977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:34.035999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:34.036002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:34.036005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:34.036009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:34.036013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:34.036016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.036020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.036023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.036031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:34.036049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:34.036055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:34.036062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:34.036066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:34.036070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:34.036074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:34.036078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:34.036083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:34.036089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:34.036093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:34.036099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:34.036106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.036111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.036116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.036120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.036127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.036132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.036160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:34.054652Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 328
2025-10-09T21:44:34.054693Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 328
2025-10-09T21:44:34.052604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:34.053989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:34.053998Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:34.054001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:34.054006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:34.054014Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 328 [2:7559341187164314788:4375]
2025-10-09T21:44:34.054051Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:34.054070Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:34.054074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.054078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.054080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.054100Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:34.054110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:34.054129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:34.054202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.054204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.054209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.054212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:34.054291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:34.054294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.054308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.054319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.054336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:34.054371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341187164314782:4375]
2025-10-09T21:44:34.054374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341187164314782:4375]
2025-10-09T21:44:34.054377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341187164314782:4375]
2025-10-09T21:44:34.054385Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:34.054401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:34.054416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:34.054419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:34.054450Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:34.054452Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:34.054455Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:34.054464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341187164314782:4375]
2025-10-09T21:44:34.054467Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:34.054470Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:34.054492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341187526125772:4724], now have 1 active actors on pipe
2025-10-09T21:44:34.054503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341183231158322:2432], now have 1 active actors on pipe
2025-10-09T21:44:34.056363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:34.056811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:34.056890Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ef361e48-68dbff4-f6064b5b-75575cb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:34.056920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.056930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.056933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:34.056947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.056971Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:34.056994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:34.060412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 329 partition: 0 MaxSeqNo: 25 sessionId: src_id|ef361e48-68dbff4-f6064b5b-75575cb_0
2025-10-09T21:44:34.059651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:34.059665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:34.059710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:34.062314Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bba7e71f-70cb21be-d11518de-3ea63e78_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ef361e48-68dbff4-f6064b5b-75575cb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:34.064881Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef361e48-68dbff4-f6064b5b-75575cb_0] Write session: try to update token
2025-10-09T21:44:34.064901Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef361e48-68dbff4-f6064b5b-75575cb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:34.075825Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:34.075843Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:34.075845Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:34.075847Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:34.075857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341187164314782:4375]
2025-10-09T21:44:34.075862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341187164314782:4375]
2025-10-09T21:44:34.075865Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341187164314782:4375]
2025-10-09T21:44:34.075914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:34.075923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.075925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:34.076068Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:34.078082Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:34.078096Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:34.078100Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:34.076674Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341187526125794:4549]
2025-10-09T21:44:34.082104Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 328, pipe: [1:7559341187526125794:4549]
2025-10-09T21:44:34.128667Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 329 sessionId: src_id|ef361e48-68dbff4-f6064b5b-75575cb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:34.131339Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:34.389200Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:34.389654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341187526125794:4549], now have 1 active actors on pipe
2025-10-09T21:44:34.389709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:34.389717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:34.390953Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:34.390969Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:34.391030Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 329 reason: error at write: Disconnected sessionId: src_id|ef361e48-68dbff4-f6064b5b-75575cb_0
2025-10-09T21:44:34.391387Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef361e48-68dbff4-f6064b5b-75575cb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:34.391399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef361e48-68dbff4-f6064b5b-75575cb_0] Write session will restart in 0.007460s
2025-10-09T21:44:34.391426Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef361e48-68dbff4-f6064b5b-75575cb_0] Write session: Do CDS request
2025-10-09T21:44:34.391433Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef361e48-68dbff4-f6064b5b-75575cb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:34.391933Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:34.393712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 329 sessionId: src_id|ef361e48-68dbff4-f6064b5b-75575cb_0 is DEAD
2025-10-09T21:44:34.393895Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:34.393928Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341187526125794:4549]
2025-10-09T21:44:34.400598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef361e48-68dbff4-f6064b5b-75575cb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:34.400736Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:34.400751Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 330
2025-10-09T21:44:34.400967Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 330 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:34.401001Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 330 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:34.401007Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=330 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:34.401011Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:34.401502Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 330 sessionId:  describe result for acl check
2025-10-09T21:44:34.401531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:34.401533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:34.401535Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:34.401541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341187526125810:4735] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:34.402165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341187526125810:4735] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:34.418451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341187526125810:4735] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:34.423762Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341187526125810:4735] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:34.423782Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341187526125810:4735] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:34.423786Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341187526125810:4735] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:34.423797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 330 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:34.425655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:34.429045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:34.429313Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:34.429324Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:34.429338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:34.429401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:34.429446Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341187164314970:4391]
2025-10-09T21:44:34.430182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:34.431755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:34.431820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:34.432202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:34.432244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:34.432246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:34.432247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:34.432249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:34.432250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:34.432251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:34.432253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.432254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.432255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.432261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:34.432274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:34.432279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:34.432284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:34.432289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:34.432294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:34.432299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.432303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.432308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.432315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:34.432325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:34.432332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:34.432338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:34.432343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:34.432349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:34.432354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:34.432360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:34.432365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:34.432371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:34.432377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:34.432383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:34.432389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.432395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.432400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.432406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.432411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.432417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.432463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:34.444238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:34.445390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:34.445397Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:34.445400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:34.445406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:34.445414Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 329 [2:7559341187164314970:4391]
2025-10-09T21:44:34.445431Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:34.445451Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:34.445454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.445458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.445460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.445481Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:34.445488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:34.445498Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:34.445559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.445561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.445564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.445567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:34.445654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:34.445658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.445672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.445685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.445701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:34.445725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341187526125799:2432], now have 1 active actors on pipe
2025-10-09T21:44:34.445729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341187526125942:4735], now have 1 active actors on pipe
2025-10-09T21:44:34.445737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341187164314965:4391]
2025-10-09T21:44:34.445739Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341187164314965:4391]
2025-10-09T21:44:34.445741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341187164314965:4391]
2025-10-09T21:44:34.445760Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:34.445776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:34.445786Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:34.445791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:34.445791Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:34.445793Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:34.445794Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:34.445873Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341187164314965:4391]
2025-10-09T21:44:34.445881Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:34.446130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:34.446319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:34.446327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:34.446373Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|51fcb659-9ad869f5-5646bdd6-f0cf89bf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:34.446382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.446387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.446390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:34.446403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.446416Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:34.446434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:34.445819Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 329
2025-10-09T21:44:34.445878Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 329
2025-10-09T21:44:34.448453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:34.448469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:34.448499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:34.448672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 330 partition: 0 MaxSeqNo: 25 sessionId: src_id|51fcb659-9ad869f5-5646bdd6-f0cf89bf_0
2025-10-09T21:44:34.449092Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef361e48-68dbff4-f6064b5b-75575cb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|51fcb659-9ad869f5-5646bdd6-f0cf89bf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:34.451651Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51fcb659-9ad869f5-5646bdd6-f0cf89bf_0] Write session: try to update token
2025-10-09T21:44:34.451672Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51fcb659-9ad869f5-5646bdd6-f0cf89bf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:34.466562Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:34.466579Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:34.466582Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:34.466584Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:34.466595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341187164314965:4391]
2025-10-09T21:44:34.466601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341187164314965:4391]
2025-10-09T21:44:34.466605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341187164314965:4391]
2025-10-09T21:44:34.466652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:34.466659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.466662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:34.466796Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:34.470762Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:34.470777Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:34.470782Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:34.473660Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 330 sessionId: src_id|51fcb659-9ad869f5-5646bdd6-f0cf89bf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:34.476180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:34.491403Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341187526125959:4549]
2025-10-09T21:44:34.491775Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 329, pipe: [1:7559341187526125959:4549]
2025-10-09T21:44:34.641174Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:34.641801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:34.641821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:34.641869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341187526125959:4549], now have 1 active actors on pipe
2025-10-09T21:44:34.642126Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:34.642626Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341187526125959:4549]
2025-10-09T21:44:34.642670Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:34.642739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 330 reason: status is not ok: tablet will be restarted right now sessionId: src_id|51fcb659-9ad869f5-5646bdd6-f0cf89bf_0
2025-10-09T21:44:34.642929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 330 sessionId: src_id|51fcb659-9ad869f5-5646bdd6-f0cf89bf_0 is DEAD
2025-10-09T21:44:34.643123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51fcb659-9ad869f5-5646bdd6-f0cf89bf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:34.643135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51fcb659-9ad869f5-5646bdd6-f0cf89bf_0] Write session will restart in 0.009791s
2025-10-09T21:44:34.643160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51fcb659-9ad869f5-5646bdd6-f0cf89bf_0] Write session: Do CDS request
2025-10-09T21:44:34.643168Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51fcb659-9ad869f5-5646bdd6-f0cf89bf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:34.643122Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:34.654567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51fcb659-9ad869f5-5646bdd6-f0cf89bf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:34.654731Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:34.654748Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 331
2025-10-09T21:44:34.654899Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 331 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:34.654953Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 331 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:34.654965Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=331 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:34.654969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:34.655477Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 331 sessionId:  describe result for acl check
2025-10-09T21:44:34.655518Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:34.655525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:34.655527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:34.655533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341187526126073:4740] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:34.656316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341187526126073:4740] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:34.657513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:34.659216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341187526126073:4740] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:34.659848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:34.660068Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:34.660081Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:34.660093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:34.660150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:34.660193Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341187164315121:4393]
2025-10-09T21:44:34.660873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:34.661228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:34.661290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:34.661655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:34.661722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:34.661729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:34.661730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:34.661732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:34.661733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:34.661734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:34.661735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.661737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.661738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.661743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:34.661755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:34.661759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:34.661763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:34.661767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:34.661772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:34.661775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.661779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.661783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.661792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:34.661800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:34.661807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:34.661812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:34.661818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:34.661823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:34.661828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:34.661833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:34.661837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:34.661842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:34.661847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:34.661852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:34.661858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.661863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.661866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.661870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.661876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.661880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.661910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:34.666230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341187526126073:4740] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:34.666249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341187526126073:4740] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:34.666253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341187526126073:4740] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:34.666263Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 331 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:34.669502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:34.670614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:34.670632Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:34.670635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:34.670639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:34.670646Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 330 [2:7559341187164315121:4393]
2025-10-09T21:44:34.670661Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:34.671018Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 330
2025-10-09T21:44:34.671087Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 330
2025-10-09T21:44:34.670680Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:34.670694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.670697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.670700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.670721Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:34.670728Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:34.670735Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:34.670800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.670802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.670804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.670807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:34.670884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:34.670887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.670899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.670911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.670928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:34.670951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341187526125962:2432], now have 1 active actors on pipe
2025-10-09T21:44:34.670961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341187164315116:4393]
2025-10-09T21:44:34.670963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341187164315116:4393]
2025-10-09T21:44:34.670965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341187164315116:4393]
2025-10-09T21:44:34.670974Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:34.670988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:34.670990Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:34.670993Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:34.670995Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:34.671003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:34.671006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:34.671043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341187526126113:4740], now have 1 active actors on pipe
2025-10-09T21:44:34.671045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341187164315116:4393]
2025-10-09T21:44:34.671049Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:34.671051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:34.671502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:34.671509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:34.671541Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b4eb083a-e6f0e152-1f531e82-15533d2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:34.671551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.671555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.671558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:34.671570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.671582Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:34.671598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:34.672084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:34.672089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:34.672108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:34.672393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 331 partition: 0 MaxSeqNo: 25 sessionId: src_id|b4eb083a-e6f0e152-1f531e82-15533d2_0
2025-10-09T21:44:34.672801Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51fcb659-9ad869f5-5646bdd6-f0cf89bf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b4eb083a-e6f0e152-1f531e82-15533d2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:34.675482Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4eb083a-e6f0e152-1f531e82-15533d2_0] Write session: try to update token
2025-10-09T21:44:34.675503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4eb083a-e6f0e152-1f531e82-15533d2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:34.688210Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:34.688225Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:34.688229Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:34.688230Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:34.688241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341187164315116:4393]
2025-10-09T21:44:34.688246Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341187164315116:4393]
2025-10-09T21:44:34.688249Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341187164315116:4393]
2025-10-09T21:44:34.688300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:34.688308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.688311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:34.688329Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:34.688350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:34.688355Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:34.688463Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:34.695336Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 331 sessionId: src_id|b4eb083a-e6f0e152-1f531e82-15533d2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:34.698089Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:34.742795Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341187526126122:4549]
2025-10-09T21:44:34.743176Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 330, pipe: [1:7559341187526126122:4549]
2025-10-09T21:44:34.891732Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:34.892195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:34.892203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:34.892244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341187526126122:4549], now have 1 active actors on pipe
2025-10-09T21:44:34.892480Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341187526126122:4549]
2025-10-09T21:44:34.892500Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:34.892504Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:34.892509Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:34.892486Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:34.892574Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 331 reason: error at write: Disconnected sessionId: src_id|b4eb083a-e6f0e152-1f531e82-15533d2_0
2025-10-09T21:44:34.892758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 331 sessionId: src_id|b4eb083a-e6f0e152-1f531e82-15533d2_0 is DEAD
2025-10-09T21:44:34.892995Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4eb083a-e6f0e152-1f531e82-15533d2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:34.893007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4eb083a-e6f0e152-1f531e82-15533d2_0] Write session will restart in 0.007475s
2025-10-09T21:44:34.893034Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4eb083a-e6f0e152-1f531e82-15533d2_0] Write session: Do CDS request
2025-10-09T21:44:34.893040Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4eb083a-e6f0e152-1f531e82-15533d2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:34.901644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4eb083a-e6f0e152-1f531e82-15533d2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:34.901770Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:34.901791Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 332
2025-10-09T21:44:34.902076Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 332 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:34.902136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 332 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:34.902142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=332 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:34.902146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:34.902627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 332 sessionId:  describe result for acl check
2025-10-09T21:44:34.902707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:34.902709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:34.902711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:34.902718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341187526126150:4744] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:34.903355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341187526126150:4744] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:34.907340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341187526126150:4744] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:34.912639Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341187526126150:4744] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:34.912658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341187526126150:4744] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:34.912663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341187526126150:4744] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:34.912673Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 332 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:34.917936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:34.920556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:34.920791Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:34.920801Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:34.920813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:34.920878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:34.920930Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341187164315277:4395]
2025-10-09T21:44:34.921494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:34.921980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:34.922045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:34.922369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:34.922438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:34.922439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:34.922440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:34.922441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:34.922442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:34.922443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:34.922444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.922445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.922446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.922451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:34.922463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:34.922467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:34.922471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:34.922476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:34.922480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:34.922484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.922488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.922492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.922500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:34.922509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:34.922517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:34.922523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:34.922528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:34.922533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:34.922537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:34.922541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:34.922545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:34.922550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:34.922554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:34.922560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:34.922564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.922570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.922574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.922580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.922586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.922591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.922621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:34.932715Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 331
2025-10-09T21:44:34.932756Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 331
2025-10-09T21:44:34.930663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:34.932216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:34.932224Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:34.932227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:34.932231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:34.932240Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 331 [2:7559341187164315277:4395]
2025-10-09T21:44:34.932256Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:34.932278Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:34.932283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.932286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.932289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.932308Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:34.932314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:34.932321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:34.932391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.932394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.932396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.932399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:34.932479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:34.932483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.932499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:34.932511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:34.932530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:34.932553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341187164315271:4395]
2025-10-09T21:44:34.932556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341187164315271:4395]
2025-10-09T21:44:34.932574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341187164315271:4395]
2025-10-09T21:44:34.932584Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:34.932599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:34.932600Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:34.932603Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:34.932605Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:34.932618Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:34.932622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:34.932642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341187526126265:4744], now have 1 active actors on pipe
2025-10-09T21:44:34.932646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341187526126136:2432], now have 1 active actors on pipe
2025-10-09T21:44:34.932688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341187164315271:4395]
2025-10-09T21:44:34.932698Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:34.932704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:34.934311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:34.934327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:34.934383Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|81b1e49f-408912db-491bbb0-fea5874e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:34.934394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.934399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.934402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:34.934416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.934432Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:34.934452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:34.934823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:34.934835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:34.934858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:34.935022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 332 partition: 0 MaxSeqNo: 25 sessionId: src_id|81b1e49f-408912db-491bbb0-fea5874e_0
2025-10-09T21:44:34.935435Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4eb083a-e6f0e152-1f531e82-15533d2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|81b1e49f-408912db-491bbb0-fea5874e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:34.938463Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|81b1e49f-408912db-491bbb0-fea5874e_0] Write session: try to update token
2025-10-09T21:44:34.938486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|81b1e49f-408912db-491bbb0-fea5874e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:34.956391Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:34.956409Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:34.956412Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:34.956414Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:34.956424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341187164315271:4395]
2025-10-09T21:44:34.956429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341187164315271:4395]
2025-10-09T21:44:34.956433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341187164315271:4395]
2025-10-09T21:44:34.956479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:34.956487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:34.956489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:34.956628Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:34.958179Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:34.958184Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:34.958188Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:34.963853Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 332 sessionId: src_id|81b1e49f-408912db-491bbb0-fea5874e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:34.966513Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:34.993852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341187526126302:4549]
2025-10-09T21:44:34.996592Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 331, pipe: [1:7559341187526126302:4549]
2025-10-09T21:44:35.149136Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:35.149955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:35.149965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:35.150009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341187526126302:4549], now have 1 active actors on pipe
2025-10-09T21:44:35.150755Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:35.150771Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:35.150841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 332 reason: error at write: Disconnected sessionId: src_id|81b1e49f-408912db-491bbb0-fea5874e_0
2025-10-09T21:44:35.151046Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 332 sessionId: src_id|81b1e49f-408912db-491bbb0-fea5874e_0 is DEAD
2025-10-09T21:44:35.151192Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341187526126302:4549]
2025-10-09T21:44:35.151212Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:35.151077Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:35.154189Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|81b1e49f-408912db-491bbb0-fea5874e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:35.154202Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|81b1e49f-408912db-491bbb0-fea5874e_0] Write session will restart in 0.005050s
2025-10-09T21:44:35.154229Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|81b1e49f-408912db-491bbb0-fea5874e_0] Write session: Do CDS request
2025-10-09T21:44:35.154237Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|81b1e49f-408912db-491bbb0-fea5874e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:35.162523Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|81b1e49f-408912db-491bbb0-fea5874e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:35.162958Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:35.162977Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 333
2025-10-09T21:44:35.163775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 333 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:35.163832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 333 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:35.163837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=333 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:35.163842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:35.164502Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 333 sessionId:  describe result for acl check
2025-10-09T21:44:35.164540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:35.164542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:35.164544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:35.164551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341191821093640:4756] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:35.165112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341191821093640:4756] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:35.173475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341191821093640:4756] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:35.178934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:35.181219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:35.181571Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:35.181584Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:35.181598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:35.181669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:35.181722Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341191459282767:4406]
2025-10-09T21:44:35.182343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:35.178871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341191821093640:4756] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:35.178893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341191821093640:4756] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:35.178897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341191821093640:4756] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:35.178909Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 333 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:35.182787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:35.182842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:35.183182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:35.183250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:35.183257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:35.183259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:35.183260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:35.183261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:35.183262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:35.183263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.183264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.183266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.183272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:35.183283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:35.183287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:35.183291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:35.183295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:35.183299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:35.183302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.183306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.183309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.183317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:35.183326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:35.183332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:35.183338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:35.183343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:35.183348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:35.183353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:35.183359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:35.183364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:35.183370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:35.183375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:35.183380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:35.183385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.183401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.183406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.183411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.183417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.183422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.183454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:35.191028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:35.192158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:35.192174Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:35.192177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:35.192182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:35.192190Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 332 [2:7559341191459282767:4406]
2025-10-09T21:44:35.192204Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:35.192225Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:35.192229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:35.192232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:35.192235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:35.192253Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:35.192259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:35.192267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:35.192331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.192340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.192342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.192345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:35.192420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:35.192431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.192446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.192457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.192474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:35.192499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341191821093770:4756], now have 1 active actors on pipe
2025-10-09T21:44:35.192515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341191459282761:4406]
2025-10-09T21:44:35.192518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341191459282761:4406]
2025-10-09T21:44:35.192520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341191459282761:4406]
2025-10-09T21:44:35.192536Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:35.192549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341191821093627:2432], now have 1 active actors on pipe
2025-10-09T21:44:35.192568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:35.192583Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:35.192585Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:35.192586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:35.192588Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:35.192590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:35.192612Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 332
2025-10-09T21:44:35.192631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341191459282761:4406]
2025-10-09T21:44:35.192640Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:35.192651Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 332
2025-10-09T21:44:35.192641Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:35.192669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:35.192674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:35.193105Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ff2bef47-ca630061-1990bc8a-31a6634c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:35.193119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:35.193123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:35.193125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:35.193139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:35.193150Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:35.193194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:35.193680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:35.193702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:35.193725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:35.193845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 333 partition: 0 MaxSeqNo: 25 sessionId: src_id|ff2bef47-ca630061-1990bc8a-31a6634c_0
2025-10-09T21:44:35.194285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|81b1e49f-408912db-491bbb0-fea5874e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ff2bef47-ca630061-1990bc8a-31a6634c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:35.196934Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff2bef47-ca630061-1990bc8a-31a6634c_0] Write session: try to update token
2025-10-09T21:44:35.196956Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff2bef47-ca630061-1990bc8a-31a6634c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:35.212863Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:35.212883Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:35.212885Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:35.212887Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:35.212897Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341191459282761:4406]
2025-10-09T21:44:35.212903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341191459282761:4406]
2025-10-09T21:44:35.212907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341191459282761:4406]
2025-10-09T21:44:35.212955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:35.212963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.212966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:35.212966Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:35.212973Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:35.212977Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:35.213101Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:35.224701Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 333 sessionId: src_id|ff2bef47-ca630061-1990bc8a-31a6634c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:35.227636Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:35.251572Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341191821093783:4549]
2025-10-09T21:44:35.256965Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 332, pipe: [1:7559341191821093783:4549]
2025-10-09T21:44:35.381185Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:35.381738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:35.381765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:35.381794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341191821093783:4549], now have 1 active actors on pipe
2025-10-09T21:44:35.382081Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:35.382970Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff2bef47-ca630061-1990bc8a-31a6634c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:35.382983Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff2bef47-ca630061-1990bc8a-31a6634c_0] Write session will restart in 0.008366s
2025-10-09T21:44:35.383007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff2bef47-ca630061-1990bc8a-31a6634c_0] Write session: Do CDS request
2025-10-09T21:44:35.383015Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff2bef47-ca630061-1990bc8a-31a6634c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:35.382500Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341191821093783:4549]
2025-10-09T21:44:35.382537Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:35.382608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 333 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ff2bef47-ca630061-1990bc8a-31a6634c_0
2025-10-09T21:44:35.382807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 333 sessionId: src_id|ff2bef47-ca630061-1990bc8a-31a6634c_0 is DEAD
2025-10-09T21:44:35.382935Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:35.392564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff2bef47-ca630061-1990bc8a-31a6634c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:35.392782Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:35.392801Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 334
2025-10-09T21:44:35.393428Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 334 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:35.393469Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 334 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:35.393476Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=334 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:35.393479Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:35.394115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 334 sessionId:  describe result for acl check
2025-10-09T21:44:35.394158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:35.394161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:35.394162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:35.394169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341191821093794:4760] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:35.394761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341191821093794:4760] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:35.398760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341191821093794:4760] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:35.403007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341191821093794:4760] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:35.403028Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341191821093794:4760] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:35.403033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341191821093794:4760] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:35.403044Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 334 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:35.413904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:35.416153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:35.416375Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:35.416386Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:35.416399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:35.416439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:35.416482Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341191459282924:4413]
2025-10-09T21:44:35.416993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:35.417438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:35.417482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:35.417749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:35.417792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:35.417793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:35.417794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:35.417795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:35.417796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:35.417797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:35.417797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.417798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.417799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.417802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:35.417810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:35.417812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:35.417815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:35.417817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:35.417819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:35.417821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.417824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.417826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.417831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:35.417836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:35.417843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:35.417846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:35.417851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:35.417854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:35.417857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:35.417860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:35.417863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:35.417866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:35.417868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:35.417871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:35.417874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.417877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.417880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.417883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.417886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.417889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.417906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:35.428342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:35.429999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:35.430019Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:35.430022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:35.430027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:35.430093Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 333 [2:7559341191459282924:4413]
2025-10-09T21:44:35.430109Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:35.430132Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:35.430136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:35.430139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:35.430142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:35.430162Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:35.430185Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:35.430197Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:35.430268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.430276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.430278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.430281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:35.430374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:35.430377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.430395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.430404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.430422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:35.430446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341191821093786:2432], now have 1 active actors on pipe
2025-10-09T21:44:35.430454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341191459282919:4413]
2025-10-09T21:44:35.430456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341191459282919:4413]
2025-10-09T21:44:35.430458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341191459282919:4413]
2025-10-09T21:44:35.430471Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:35.430481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341191821093834:4760], now have 1 active actors on pipe
2025-10-09T21:44:35.430483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:35.430486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:35.430488Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:35.430490Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:35.430504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:35.430508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:35.430553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341191459282919:4413]
2025-10-09T21:44:35.430562Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:35.430572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:35.431454Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 333
2025-10-09T21:44:35.431496Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 333
2025-10-09T21:44:35.432211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:35.432231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:35.432272Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|137890d-2dcdf5f3-61afde8f-66a10b96_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:35.432288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:35.432299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:35.432302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:35.432316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:35.432333Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:35.432358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:35.432806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:35.432822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:35.432856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:35.432973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 334 partition: 0 MaxSeqNo: 25 sessionId: src_id|137890d-2dcdf5f3-61afde8f-66a10b96_0
2025-10-09T21:44:35.433402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff2bef47-ca630061-1990bc8a-31a6634c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|137890d-2dcdf5f3-61afde8f-66a10b96_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:35.435936Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|137890d-2dcdf5f3-61afde8f-66a10b96_0] Write session: try to update token
2025-10-09T21:44:35.435958Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|137890d-2dcdf5f3-61afde8f-66a10b96_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:35.461388Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:35.461407Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:35.461410Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:35.461412Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:35.461421Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341191459282919:4413]
2025-10-09T21:44:35.461425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341191459282919:4413]
2025-10-09T21:44:35.461429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341191459282919:4413]
2025-10-09T21:44:35.461476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:35.461486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.461488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:35.461656Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:35.461117Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 334 sessionId: src_id|137890d-2dcdf5f3-61afde8f-66a10b96_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:35.463656Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:35.464246Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:35.464259Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:35.464262Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:35.481605Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341191821093949:4549]
2025-10-09T21:44:35.482216Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 333, pipe: [1:7559341191821093949:4549]
2025-10-09T21:44:35.642920Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:35.643218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:35.643233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:35.643281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341191821093949:4549], now have 1 active actors on pipe
2025-10-09T21:44:35.643820Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:35.645524Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|137890d-2dcdf5f3-61afde8f-66a10b96_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:35.645537Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|137890d-2dcdf5f3-61afde8f-66a10b96_0] Write session will restart in 0.009198s
2025-10-09T21:44:35.645562Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|137890d-2dcdf5f3-61afde8f-66a10b96_0] Write session: Do CDS request
2025-10-09T21:44:35.645568Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|137890d-2dcdf5f3-61afde8f-66a10b96_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:35.643651Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:35.643677Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:35.643755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 334 reason: error at write: Disconnected sessionId: src_id|137890d-2dcdf5f3-61afde8f-66a10b96_0
2025-10-09T21:44:35.643958Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 334 sessionId: src_id|137890d-2dcdf5f3-61afde8f-66a10b96_0 is DEAD
2025-10-09T21:44:35.643992Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341191821093949:4549]
2025-10-09T21:44:35.644010Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:35.655545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|137890d-2dcdf5f3-61afde8f-66a10b96_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:35.656221Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:35.656240Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 335
2025-10-09T21:44:35.657009Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 335 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:35.657047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 335 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:35.657053Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=335 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:35.657057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:35.657614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 335 sessionId:  describe result for acl check
2025-10-09T21:44:35.657644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:35.657646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:35.657649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:35.657655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341191821093964:4765] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:35.658277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341191821093964:4765] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:35.662530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341191821093964:4765] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:35.671608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341191821093964:4765] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:35.671629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341191821093964:4765] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:35.671633Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341191821093964:4765] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:35.671643Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 335 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:35.688862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:35.691542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:35.691772Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:35.691775Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:35.691788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:35.691844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:35.691879Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341191459283080:4415]
2025-10-09T21:44:35.692393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:35.692805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:35.692861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:35.693199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:35.693269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:35.693270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:35.693271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:35.693272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:35.693273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:35.693275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:35.693276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.693277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.693278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.693284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:35.693296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:35.693300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:35.693304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:35.693308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:35.693312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:35.693316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.693320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.693324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.693332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:35.693340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:35.693346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:35.693352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:35.693357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:35.693362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:35.693366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:35.693371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:35.693375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:35.693382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:35.693386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:35.693392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:35.693396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.693402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.693406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.693412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.693417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.693422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.693453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:35.701007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:35.702847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:35.702865Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:35.702867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:35.702872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:35.702880Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 334 [2:7559341191459283080:4415]
2025-10-09T21:44:35.702894Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:35.702920Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:35.702924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:35.702927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:35.702929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:35.702949Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:35.702956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:35.702970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:35.703043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.703052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.703054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.703057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:35.703134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:35.703141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.703154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.703164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.703181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:35.703208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341191459283074:4415]
2025-10-09T21:44:35.703212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341191459283074:4415]
2025-10-09T21:44:35.703215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341191459283074:4415]
2025-10-09T21:44:35.703231Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:35.703241Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:35.703243Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:35.703245Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:35.703252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:35.703269Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:35.703275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:35.703288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341191821094011:4765], now have 1 active actors on pipe
2025-10-09T21:44:35.703293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341191821093954:2432], now have 1 active actors on pipe
2025-10-09T21:44:35.703340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341191459283074:4415]
2025-10-09T21:44:35.703353Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:35.703974Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 334
2025-10-09T21:44:35.704014Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 334
2025-10-09T21:44:35.703384Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:35.704585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:35.704599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:35.704651Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9262b445-5eec9810-709800d0-b9297ad3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:35.704677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:35.704682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:35.704685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:35.704700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:35.704719Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:35.704740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:35.710696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:35.710728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:35.710769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:35.711049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 335 partition: 0 MaxSeqNo: 25 sessionId: src_id|9262b445-5eec9810-709800d0-b9297ad3_0
2025-10-09T21:44:35.711519Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|137890d-2dcdf5f3-61afde8f-66a10b96_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9262b445-5eec9810-709800d0-b9297ad3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:35.714011Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9262b445-5eec9810-709800d0-b9297ad3_0] Write session: try to update token
2025-10-09T21:44:35.714051Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9262b445-5eec9810-709800d0-b9297ad3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:35.743749Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341191821094115:4549]
2025-10-09T21:44:35.744279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341191821094115:4549], now have 1 active actors on pipe
2025-10-09T21:44:35.744713Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 334, pipe: [1:7559341191821094115:4549]
2025-10-09T21:44:35.744931Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 335 sessionId: src_id|9262b445-5eec9810-709800d0-b9297ad3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:35.745486Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:35.745502Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:35.745504Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:35.745507Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:35.745516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341191459283074:4415]
2025-10-09T21:44:35.745521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341191459283074:4415]
2025-10-09T21:44:35.745526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341191459283074:4415]
2025-10-09T21:44:35.745572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:35.745584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.745587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:35.745600Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:35.745606Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:35.745609Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:35.745722Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:35.747505Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:35.949735Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:35.948601Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:35.949815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 335 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9262b445-5eec9810-709800d0-b9297ad3_0
2025-10-09T21:44:35.950008Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 335 sessionId: src_id|9262b445-5eec9810-709800d0-b9297ad3_0 is DEAD
2025-10-09T21:44:35.950093Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341191821094115:4549]
2025-10-09T21:44:35.948933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:35.948945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:35.950108Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:35.949228Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:35.950656Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9262b445-5eec9810-709800d0-b9297ad3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:35.950669Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9262b445-5eec9810-709800d0-b9297ad3_0] Write session will restart in 0.005361s
2025-10-09T21:44:35.950695Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9262b445-5eec9810-709800d0-b9297ad3_0] Write session: Do CDS request
2025-10-09T21:44:35.950701Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9262b445-5eec9810-709800d0-b9297ad3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:35.957723Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9262b445-5eec9810-709800d0-b9297ad3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:35.957737Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:35.957754Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 336
2025-10-09T21:44:35.958127Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 336 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:35.958172Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 336 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:35.958177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=336 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:35.958184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:35.960111Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 336 sessionId:  describe result for acl check
2025-10-09T21:44:35.960154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:35.960156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:35.960157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:35.960163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341191821094141:4770] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:35.960792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341191821094141:4770] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:35.964498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341191821094141:4770] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:35.973556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341191821094141:4770] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:35.973577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341191821094141:4770] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:35.973581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341191821094141:4770] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:35.973591Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 336 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:35.993885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:35.994130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:35.994477Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:35.994482Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:35.994502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:35.994578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:35.994622Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341191459283238:4417]
2025-10-09T21:44:35.995229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:35.995983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:35.996052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:35.996432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:35.996483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:35.996484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:35.996486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:35.996487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:35.996489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:35.996491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:35.996492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.996493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.996495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.996501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:35.996514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:35.996518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:35.996523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:35.996527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:35.996532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:35.996537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.996541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.996544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.996552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:35.996561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:35.996569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:35.996576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:35.996580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:35.996586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:35.996590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:35.996597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:35.996602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:35.996608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:35.996613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:35.996619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:35.996624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.996630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:35.996635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.996640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:35.996646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.996653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:35.996689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:36.007849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:36.009285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:36.009291Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:36.009292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:36.009296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:36.009303Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 335 [2:7559341191459283238:4417]
2025-10-09T21:44:36.009315Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:36.009332Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:36.009337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.009339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.009341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.009356Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:36.009361Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:36.009367Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:36.009434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.009436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.009437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.009439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:36.009514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:36.009517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.009531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.009540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.009555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:36.009576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341191459283233:4417]
2025-10-09T21:44:36.009578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341191459283233:4417]
2025-10-09T21:44:36.009580Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341191459283233:4417]
2025-10-09T21:44:36.009588Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:36.009596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:36.009601Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:36.009603Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:36.009606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:36.009620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:36.009623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:36.009636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341191821094128:2432], now have 1 active actors on pipe
2025-10-09T21:44:36.009640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341191821094225:4770], now have 1 active actors on pipe
2025-10-09T21:44:36.009680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341191459283233:4417]
2025-10-09T21:44:36.009691Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:36.009694Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:36.009814Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 335
2025-10-09T21:44:36.009855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 335
2025-10-09T21:44:36.010163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:36.010176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:36.010209Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f5d395a4-657c73c-b0d4c28a-f5f2982_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:36.010218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.010223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.010225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:36.010237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.010252Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:36.010272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:36.013693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:36.013712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:36.013752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:36.014562Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9262b445-5eec9810-709800d0-b9297ad3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f5d395a4-657c73c-b0d4c28a-f5f2982_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:36.014114Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 336 partition: 0 MaxSeqNo: 25 sessionId: src_id|f5d395a4-657c73c-b0d4c28a-f5f2982_0
2025-10-09T21:44:36.017194Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5d395a4-657c73c-b0d4c28a-f5f2982_0] Write session: try to update token
2025-10-09T21:44:36.017214Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5d395a4-657c73c-b0d4c28a-f5f2982_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:36.033658Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:36.033677Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:36.033680Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:36.033682Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:36.033691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341191459283233:4417]
2025-10-09T21:44:36.033698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341191459283233:4417]
2025-10-09T21:44:36.033702Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341191459283233:4417]
2025-10-09T21:44:36.033748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:36.033756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.033758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:36.033907Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:36.039327Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:36.039341Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:36.039345Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:36.049136Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 336 sessionId: src_id|f5d395a4-657c73c-b0d4c28a-f5f2982_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:36.050213Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341196116061592:4549]
2025-10-09T21:44:36.051780Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:36.051880Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 335, pipe: [1:7559341196116061592:4549]
2025-10-09T21:44:36.262216Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:36.262263Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:36.262266Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:36.262321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 336 reason: error at write: Disconnected sessionId: src_id|f5d395a4-657c73c-b0d4c28a-f5f2982_0
2025-10-09T21:44:36.262473Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341196116061592:4549]
2025-10-09T21:44:36.262520Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 336 sessionId: src_id|f5d395a4-657c73c-b0d4c28a-f5f2982_0 is DEAD
2025-10-09T21:44:36.262858Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5d395a4-657c73c-b0d4c28a-f5f2982_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:36.262865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5d395a4-657c73c-b0d4c28a-f5f2982_0] Write session will restart in 0.007897s
2025-10-09T21:44:36.262887Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5d395a4-657c73c-b0d4c28a-f5f2982_0] Write session: Do CDS request
2025-10-09T21:44:36.262893Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5d395a4-657c73c-b0d4c28a-f5f2982_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:36.261213Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:36.261620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341196116061592:4549], now have 1 active actors on pipe
2025-10-09T21:44:36.261645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:36.261653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:36.262594Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:36.271490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5d395a4-657c73c-b0d4c28a-f5f2982_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:36.276245Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:36.276266Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 337
2025-10-09T21:44:36.277096Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 337 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:36.277133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 337 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:36.277139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=337 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:36.277143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:36.278244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 337 sessionId:  describe result for acl check
2025-10-09T21:44:36.278277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:36.278279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:36.278280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:36.278287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341196116061662:4781] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:36.278896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341196116061662:4781] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:36.287341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341196116061662:4781] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:36.291968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:36.295062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:36.295298Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:36.295302Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:36.295316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:36.295378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:36.295547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341196116061662:4781] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:36.295568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341196116061662:4781] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:36.295572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341196116061662:4781] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:36.295582Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 337 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:36.295715Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341195754250733:4430]
2025-10-09T21:44:36.296325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:36.297390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:36.297459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:36.297797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:36.297848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:36.297850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:36.297851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:36.297852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:36.297853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:36.297855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:36.297856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.297857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.297858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.297863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:36.297875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:36.297879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:36.297883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:36.297886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:36.297890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:36.297893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.297897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.297902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.297910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:36.297919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:36.297925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:36.297930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:36.297935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:36.297939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:36.297943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:36.297949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:36.297954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:36.297958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:36.297963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:36.297967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:36.297972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.297976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.297981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.297985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.297989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.297994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.298021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:36.306519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:36.307927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:36.307934Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:36.307937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:36.307942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:36.307951Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 336 [2:7559341195754250733:4430]
2025-10-09T21:44:36.307968Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:36.307990Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:36.307994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.307998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.308000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.308023Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:36.308032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:36.308041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:36.308116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.308118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.308120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.308122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:36.308208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:36.308211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.308224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.308236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.308249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:36.308270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341195754250728:4430]
2025-10-09T21:44:36.308272Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341195754250728:4430]
2025-10-09T21:44:36.308273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341195754250728:4430]
2025-10-09T21:44:36.308293Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:36.308308Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:36.308323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:36.308327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:36.308353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:36.308357Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:36.308359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:36.308370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341195754250728:4430]
2025-10-09T21:44:36.308373Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:36.308377Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:36.309304Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 336
2025-10-09T21:44:36.309347Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 336
2025-10-09T21:44:36.313167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341196116061619:2432], now have 1 active actors on pipe
2025-10-09T21:44:36.313185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341196116061770:4781], now have 1 active actors on pipe
2025-10-09T21:44:36.313219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:36.313227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:36.313283Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4010c3a7-44d8a13a-6e6da468-d3d6816_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:36.313296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.313302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.313305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:36.313320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.313349Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:36.313372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:36.315913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:36.315937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:36.315981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:36.318341Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5d395a4-657c73c-b0d4c28a-f5f2982_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4010c3a7-44d8a13a-6e6da468-d3d6816_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:36.316193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 337 partition: 0 MaxSeqNo: 25 sessionId: src_id|4010c3a7-44d8a13a-6e6da468-d3d6816_0
2025-10-09T21:44:36.320884Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4010c3a7-44d8a13a-6e6da468-d3d6816_0] Write session: try to update token
2025-10-09T21:44:36.320908Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4010c3a7-44d8a13a-6e6da468-d3d6816_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:36.329886Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:36.329904Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:36.329907Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:36.329909Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:36.329919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341195754250728:4430]
2025-10-09T21:44:36.329924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341195754250728:4430]
2025-10-09T21:44:36.329928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341195754250728:4430]
2025-10-09T21:44:36.329977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:36.329986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.329989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:36.330140Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:36.333830Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:36.333845Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:36.333850Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:36.356591Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 337 sessionId: src_id|4010c3a7-44d8a13a-6e6da468-d3d6816_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:36.359745Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:36.360839Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341196116061784:4549]
2025-10-09T21:44:36.361164Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 336, pipe: [1:7559341196116061784:4549]
2025-10-09T21:44:36.516133Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:36.516680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:36.516691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:36.516717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341196116061784:4549], now have 1 active actors on pipe
2025-10-09T21:44:36.518488Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:36.519253Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:36.519291Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341196116061784:4549]
2025-10-09T21:44:36.519302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:36.519305Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:36.519356Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 337 reason: error at write: Disconnected sessionId: src_id|4010c3a7-44d8a13a-6e6da468-d3d6816_0
2025-10-09T21:44:36.519461Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 337 sessionId: src_id|4010c3a7-44d8a13a-6e6da468-d3d6816_0 is DEAD
2025-10-09T21:44:36.519834Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4010c3a7-44d8a13a-6e6da468-d3d6816_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:36.519843Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4010c3a7-44d8a13a-6e6da468-d3d6816_0] Write session will restart in 0.009568s
2025-10-09T21:44:36.519871Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4010c3a7-44d8a13a-6e6da468-d3d6816_0] Write session: Do CDS request
2025-10-09T21:44:36.519877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4010c3a7-44d8a13a-6e6da468-d3d6816_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:36.530772Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4010c3a7-44d8a13a-6e6da468-d3d6816_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:36.532187Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:36.532204Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 338
2025-10-09T21:44:36.532429Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 338 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:36.532455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 338 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:36.532459Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=338 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:36.532462Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:36.533420Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 338 sessionId:  describe result for acl check
2025-10-09T21:44:36.533452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:36.533453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:36.533454Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:36.533458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341196116061850:4786] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:36.533875Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341196116061850:4786] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:36.545732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341196116061850:4786] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:36.556538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341196116061850:4786] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:36.556556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341196116061850:4786] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:36.556561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341196116061850:4786] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:36.556569Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 338 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:36.560624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:36.566000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:36.566266Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:36.566278Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:36.566291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:36.566368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:36.566411Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341195754250892:4435]
2025-10-09T21:44:36.566963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:36.568425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:36.568499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:36.568838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:36.568889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:36.568890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:36.568891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:36.568893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:36.568894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:36.568895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:36.568896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.568898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.568899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.568904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:36.568914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:36.568918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:36.568921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:36.568924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:36.568927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:36.568931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.568934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.568937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.568947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:36.568954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:36.568960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:36.568964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:36.568967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:36.568971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:36.568974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:36.568978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:36.568982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:36.568986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:36.568990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:36.568994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:36.568998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.569003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.569007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.569012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.569019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.569023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.569048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:36.579221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:36.580649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:36.580656Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:36.580658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:36.580663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:36.580672Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 337 [2:7559341195754250892:4435]
2025-10-09T21:44:36.580689Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:36.580712Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:36.580716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.580720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.580722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.580745Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:36.580752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:36.580760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:36.580829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.580830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.580832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.580835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:36.580918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:36.580921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.580935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.580946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.580961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:36.580983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341195754250886:4435]
2025-10-09T21:44:36.580986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341195754250886:4435]
2025-10-09T21:44:36.580989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341195754250886:4435]
2025-10-09T21:44:36.581004Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:36.581020Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:36.581034Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:36.581037Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:36.581084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341195754250886:4435]
2025-10-09T21:44:36.581087Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:36.582574Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 337
2025-10-09T21:44:36.582628Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 337
2025-10-09T21:44:36.581368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:36.581375Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:36.581377Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:36.581383Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:36.592537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341196116061789:2432], now have 1 active actors on pipe
2025-10-09T21:44:36.592555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341196116061940:4786], now have 1 active actors on pipe
2025-10-09T21:44:36.592605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:36.592614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:36.592658Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5388235e-9383cb4-e786d698-8ad44ed9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:36.592671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.592677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.592680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:36.592697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.592718Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:36.592741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:36.593963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:36.593975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:36.593998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:36.594788Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4010c3a7-44d8a13a-6e6da468-d3d6816_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5388235e-9383cb4-e786d698-8ad44ed9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:36.594354Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 338 partition: 0 MaxSeqNo: 25 sessionId: src_id|5388235e-9383cb4-e786d698-8ad44ed9_0
2025-10-09T21:44:36.597524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5388235e-9383cb4-e786d698-8ad44ed9_0] Write session: try to update token
2025-10-09T21:44:36.597546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5388235e-9383cb4-e786d698-8ad44ed9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:36.603794Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:36.603811Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:36.603814Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:36.603816Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:36.603825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341195754250886:4435]
2025-10-09T21:44:36.603830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341195754250886:4435]
2025-10-09T21:44:36.603834Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341195754250886:4435]
2025-10-09T21:44:36.603882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:36.603889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.603892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:36.604036Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:36.605720Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:36.605727Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:36.605732Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:36.620260Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341196116061955:4549]
2025-10-09T21:44:36.624599Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 337, pipe: [1:7559341196116061955:4549]
2025-10-09T21:44:36.635632Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 338 sessionId: src_id|5388235e-9383cb4-e786d698-8ad44ed9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:36.638386Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:36.816489Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:36.816888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341196116061955:4549], now have 1 active actors on pipe
2025-10-09T21:44:36.816921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:36.816929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:36.817229Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:36.817245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:36.817314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 338 reason: error at write: Disconnected sessionId: src_id|5388235e-9383cb4-e786d698-8ad44ed9_0
2025-10-09T21:44:36.817510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 338 sessionId: src_id|5388235e-9383cb4-e786d698-8ad44ed9_0 is DEAD
2025-10-09T21:44:36.817272Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:36.817695Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341196116061955:4549]
2025-10-09T21:44:36.817715Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:36.817916Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5388235e-9383cb4-e786d698-8ad44ed9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:36.817927Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5388235e-9383cb4-e786d698-8ad44ed9_0] Write session will restart in 0.005074s
2025-10-09T21:44:36.817954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5388235e-9383cb4-e786d698-8ad44ed9_0] Write session: Do CDS request
2025-10-09T21:44:36.817960Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5388235e-9383cb4-e786d698-8ad44ed9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:36.824601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5388235e-9383cb4-e786d698-8ad44ed9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:36.826422Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:36.826441Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 339
2025-10-09T21:44:36.826628Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 339 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:36.826663Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 339 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:36.826669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=339 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:36.826673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:36.827222Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 339 sessionId:  describe result for acl check
2025-10-09T21:44:36.827261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:36.827263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:36.827264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:36.827270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341196116061976:4792] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:36.827838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341196116061976:4792] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:36.832161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341196116061976:4792] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:36.839237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341196116061976:4792] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:36.839255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341196116061976:4792] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:36.839260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341196116061976:4792] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:36.839270Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 339 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:36.844150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:36.847146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:36.847397Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:36.847402Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:36.847416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:36.847482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:36.847525Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341195754251051:4437]
2025-10-09T21:44:36.848102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:36.848641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:36.848693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:36.849004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:36.849084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:36.849092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:36.849093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:36.849095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:36.849096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:36.849097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:36.849098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.849100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.849101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.849107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:36.849117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:36.849121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:36.849125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:36.849128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:36.849132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:36.849135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.849139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.849143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.849150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:36.849158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:36.849164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:36.849170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:36.849174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:36.849179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:36.849183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:36.849187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:36.849191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:36.849195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:36.849201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:36.849205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:36.849210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.849221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.849225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.849229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.849234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.849237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.849262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:36.861455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:36.862992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:36.863010Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:36.863013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:36.863018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:36.863025Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 338 [2:7559341195754251051:4437]
2025-10-09T21:44:36.863040Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:36.863060Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:36.863064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.863067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.863069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.863088Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:36.863094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:36.863101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:36.863163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.863165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.863167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.863169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:36.863244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:36.863247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.863262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:36.863273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:36.863291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:36.863311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341195754251045:4437]
2025-10-09T21:44:36.863313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341195754251045:4437]
2025-10-09T21:44:36.863315Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341195754251045:4437]
2025-10-09T21:44:36.863329Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:36.863343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:36.863359Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:36.863362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:36.863404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341195754251045:4437]
2025-10-09T21:44:36.863406Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:36.863695Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:36.863697Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:36.863699Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:36.863704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:36.864546Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 338
2025-10-09T21:44:36.864581Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 338
2025-10-09T21:44:36.864752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341196116062094:4792], now have 1 active actors on pipe
2025-10-09T21:44:36.864761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341196116061961:2432], now have 1 active actors on pipe
2025-10-09T21:44:36.864775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:36.864784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:36.864818Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3013d08e-295afdf0-fc57acb0-cebcd5a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:36.864830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.864835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.864837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:36.864850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.864864Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:36.864883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:36.865192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:36.865198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:36.865213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:36.865627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 339 partition: 0 MaxSeqNo: 25 sessionId: src_id|3013d08e-295afdf0-fc57acb0-cebcd5a_0
2025-10-09T21:44:36.866075Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5388235e-9383cb4-e786d698-8ad44ed9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3013d08e-295afdf0-fc57acb0-cebcd5a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:36.868792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3013d08e-295afdf0-fc57acb0-cebcd5a_0] Write session: try to update token
2025-10-09T21:44:36.868816Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3013d08e-295afdf0-fc57acb0-cebcd5a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:36.894754Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:36.894774Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:36.894776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:36.894779Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:36.894789Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341195754251045:4437]
2025-10-09T21:44:36.894793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341195754251045:4437]
2025-10-09T21:44:36.894797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341195754251045:4437]
2025-10-09T21:44:36.894845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:36.894852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:36.894855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:36.894992Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:36.898315Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:36.898334Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:36.898358Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:36.897492Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 339 sessionId: src_id|3013d08e-295afdf0-fc57acb0-cebcd5a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:36.900054Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:36.922149Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341196116062131:4549]
2025-10-09T21:44:36.922788Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 338, pipe: [1:7559341196116062131:4549]
2025-10-09T21:44:37.148375Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:37.148691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:37.148707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:37.148757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341196116062131:4549], now have 1 active actors on pipe
2025-10-09T21:44:37.150929Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341196116062131:4549]
2025-10-09T21:44:37.150965Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:37.150968Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:37.151025Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 339 reason: error at write: Disconnected sessionId: src_id|3013d08e-295afdf0-fc57acb0-cebcd5a_0
2025-10-09T21:44:37.151170Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:37.151226Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 339 sessionId: src_id|3013d08e-295afdf0-fc57acb0-cebcd5a_0 is DEAD
2025-10-09T21:44:37.151910Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3013d08e-295afdf0-fc57acb0-cebcd5a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:37.151923Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3013d08e-295afdf0-fc57acb0-cebcd5a_0] Write session will restart in 0.009639s
2025-10-09T21:44:37.151962Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3013d08e-295afdf0-fc57acb0-cebcd5a_0] Write session: Do CDS request
2025-10-09T21:44:37.151971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3013d08e-295afdf0-fc57acb0-cebcd5a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:37.150608Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:37.162711Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3013d08e-295afdf0-fc57acb0-cebcd5a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:37.163094Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:37.163112Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 340
2025-10-09T21:44:37.163305Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 340 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:37.163338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 340 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:37.163344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=340 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:37.163347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:37.165232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 340 sessionId:  describe result for acl check
2025-10-09T21:44:37.165271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:37.165273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:37.165275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:37.165281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341200411029445:4797] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:37.165881Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341200411029445:4797] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:37.177848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341200411029445:4797] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:37.184575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341200411029445:4797] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:37.184597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341200411029445:4797] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:37.184602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341200411029445:4797] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:37.184613Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 340 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:37.213838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:37.225892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:37.226397Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:37.226403Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:37.226420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:37.226520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:37.226565Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341200049218510:4440]
2025-10-09T21:44:37.227200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:37.231644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:37.231742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:37.232156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:37.232245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:37.232255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:37.232256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:37.232257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:37.232260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:37.232261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:37.232263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.232264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.232266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.232273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:37.232285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:37.232290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:37.232294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:37.232298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:37.232303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:37.232306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.232311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.232315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.232324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:37.232333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:37.232339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:37.232345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:37.232350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:37.232355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:37.232360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:37.232366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:37.232370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:37.232376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:37.232381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:37.232387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:37.232395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.232401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.232406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.232412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.232417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.232423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.232456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:37.243145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:37.244481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:37.244488Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:37.244491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:37.244495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:37.244504Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 339 [2:7559341200049218510:4440]
2025-10-09T21:44:37.244519Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:37.244540Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:37.244545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:37.244548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:37.244550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:37.244569Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:37.244575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:37.244583Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:37.244649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.244653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.244655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.244658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:37.244744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:37.244749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.244764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.244777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.244795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:37.244817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341200049218504:4440]
2025-10-09T21:44:37.244820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341200049218504:4440]
2025-10-09T21:44:37.244822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341200049218504:4440]
2025-10-09T21:44:37.244841Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:37.244856Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:37.244873Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:37.244876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:37.244919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341200049218504:4440]
2025-10-09T21:44:37.244922Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:37.244933Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:37.244937Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:37.244940Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:37.244947Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:37.245237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341200411029435:2432], now have 1 active actors on pipe
2025-10-09T21:44:37.245245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341200411029512:4797], now have 1 active actors on pipe
2025-10-09T21:44:37.248544Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 339
2025-10-09T21:44:37.248572Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 339
2025-10-09T21:44:37.250271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:37.250287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:37.250325Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d11c4591-898464b0-d5780dd7-2d85f04e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:37.250353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:37.250359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:37.250363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:37.250379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:37.250398Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:37.250419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:37.250514Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341200411029603:4549]
2025-10-09T21:44:37.251175Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 339, pipe: [1:7559341200411029603:4549]
2025-10-09T21:44:37.254103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:37.254124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:37.254154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341200411029603:4549], now have 1 active actors on pipe
2025-10-09T21:44:37.254182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:37.254518Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 340 partition: 0 MaxSeqNo: 25 sessionId: src_id|d11c4591-898464b0-d5780dd7-2d85f04e_0
2025-10-09T21:44:37.255016Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3013d08e-295afdf0-fc57acb0-cebcd5a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d11c4591-898464b0-d5780dd7-2d85f04e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:37.257613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d11c4591-898464b0-d5780dd7-2d85f04e_0] Write session: try to update token
2025-10-09T21:44:37.257637Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d11c4591-898464b0-d5780dd7-2d85f04e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:37.275936Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:37.275956Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:37.275959Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:37.275960Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:37.275971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341200049218504:4440]
2025-10-09T21:44:37.275976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341200049218504:4440]
2025-10-09T21:44:37.275980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341200049218504:4440]
2025-10-09T21:44:37.276028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:37.276036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.276039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:37.276169Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:37.277261Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:37.277268Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:37.277273Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:37.286975Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 340 sessionId: src_id|d11c4591-898464b0-d5780dd7-2d85f04e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:37.289365Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:37.539331Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:37.540027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:37.540039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:37.540416Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:37.540909Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341200411029603:4549]
2025-10-09T21:44:37.540957Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:37.541023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 340 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d11c4591-898464b0-d5780dd7-2d85f04e_0
2025-10-09T21:44:37.541217Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 340 sessionId: src_id|d11c4591-898464b0-d5780dd7-2d85f04e_0 is DEAD
2025-10-09T21:44:37.541382Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:37.541481Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d11c4591-898464b0-d5780dd7-2d85f04e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:37.541498Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d11c4591-898464b0-d5780dd7-2d85f04e_0] Write session will restart in 0.008708s
2025-10-09T21:44:37.541530Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d11c4591-898464b0-d5780dd7-2d85f04e_0] Write session: Do CDS request
2025-10-09T21:44:37.541537Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d11c4591-898464b0-d5780dd7-2d85f04e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:37.551667Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d11c4591-898464b0-d5780dd7-2d85f04e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:37.551923Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:37.551955Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 341
2025-10-09T21:44:37.552084Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 341 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:37.552117Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 341 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:37.552123Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=341 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:37.552127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:37.552691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 341 sessionId:  describe result for acl check
2025-10-09T21:44:37.552735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:37.552738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:37.552739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:37.552746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341200411029649:4809] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:37.553356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341200411029649:4809] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:37.559962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341200411029649:4809] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:37.571242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341200411029649:4809] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:37.571264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341200411029649:4809] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:37.571268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341200411029649:4809] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:37.571278Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 341 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:37.576025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:37.579588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:37.579852Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:37.579862Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:37.579875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:37.579940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:37.580167Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341200049218713:4455]
2025-10-09T21:44:37.580774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:37.581391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:37.581458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:37.581841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:37.581905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:37.581912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:37.581914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:37.581915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:37.581917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:37.581918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:37.581919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.581920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.581921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.581927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:37.581938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:37.581942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:37.581945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:37.581949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:37.581953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:37.581956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.581961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.581964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.581972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:37.581980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:37.581987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:37.581992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:37.581997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:37.582002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:37.582006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:37.582011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:37.582016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:37.582020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:37.582025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:37.582047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:37.582051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.582056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.582066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.582071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.582076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.582080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.582109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:37.590242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:37.592037Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 340
2025-10-09T21:44:37.592083Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 340
2025-10-09T21:44:37.591583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:37.591601Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:37.591604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:37.591608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:37.591616Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 340 [2:7559341200049218713:4455]
2025-10-09T21:44:37.591633Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:37.591654Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:37.591658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:37.591661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:37.591663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:37.591683Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:37.591689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:37.591698Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:37.591763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.591764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.591766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.591769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:37.591844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:37.591847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.591862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.591875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.591893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:37.591915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341200411029637:2432], now have 1 active actors on pipe
2025-10-09T21:44:37.591919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341200411029791:4809], now have 1 active actors on pipe
2025-10-09T21:44:37.591926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341200049218708:4455]
2025-10-09T21:44:37.591928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341200049218708:4455]
2025-10-09T21:44:37.591930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341200049218708:4455]
2025-10-09T21:44:37.591943Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:37.591958Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:37.591958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:37.591962Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:37.591964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:37.591972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:37.591975Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:37.592015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341200049218708:4455]
2025-10-09T21:44:37.592024Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:37.592115Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:37.592338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:37.592346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:37.592388Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bbb05cf4-545d83a7-619e67e6-5f159461_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:37.592397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:37.592401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:37.592404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:37.592415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:37.592428Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:37.592446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:37.597398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:37.597422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:37.597462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:37.597917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 341 partition: 0 MaxSeqNo: 25 sessionId: src_id|bbb05cf4-545d83a7-619e67e6-5f159461_0
2025-10-09T21:44:37.598649Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d11c4591-898464b0-d5780dd7-2d85f04e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bbb05cf4-545d83a7-619e67e6-5f159461_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:37.601328Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bbb05cf4-545d83a7-619e67e6-5f159461_0] Write session: try to update token
2025-10-09T21:44:37.601353Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bbb05cf4-545d83a7-619e67e6-5f159461_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:37.616937Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:37.616956Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:37.616959Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:37.616961Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:37.616972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341200049218708:4455]
2025-10-09T21:44:37.616977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341200049218708:4455]
2025-10-09T21:44:37.616981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341200049218708:4455]
2025-10-09T21:44:37.617033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:37.617034Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:37.617040Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:37.617047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.617047Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:37.617050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:37.617190Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:37.624081Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 341 sessionId: src_id|bbb05cf4-545d83a7-619e67e6-5f159461_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:37.626636Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:37.641233Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341200411029806:4549]
2025-10-09T21:44:37.641674Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 340, pipe: [1:7559341200411029806:4549]
2025-10-09T21:44:37.815120Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:37.815631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:37.815650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:37.815717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341200411029806:4549], now have 1 active actors on pipe
2025-10-09T21:44:37.816055Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:37.816074Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:37.816144Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:37.816136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 341 reason: error at write: Disconnected sessionId: src_id|bbb05cf4-545d83a7-619e67e6-5f159461_0
2025-10-09T21:44:37.816339Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 341 sessionId: src_id|bbb05cf4-545d83a7-619e67e6-5f159461_0 is DEAD
2025-10-09T21:44:37.816350Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341200411029806:4549]
2025-10-09T21:44:37.816377Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:37.816496Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbb05cf4-545d83a7-619e67e6-5f159461_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:37.816503Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbb05cf4-545d83a7-619e67e6-5f159461_0] Write session will restart in 0.005714s
2025-10-09T21:44:37.816529Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbb05cf4-545d83a7-619e67e6-5f159461_0] Write session: Do CDS request
2025-10-09T21:44:37.816535Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbb05cf4-545d83a7-619e67e6-5f159461_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:37.823633Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bbb05cf4-545d83a7-619e67e6-5f159461_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:37.825177Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:37.825196Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 342
2025-10-09T21:44:37.825362Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 342 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:37.825398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 342 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:37.825408Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=342 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:37.825412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:37.825799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 342 sessionId:  describe result for acl check
2025-10-09T21:44:37.825839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:37.825840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:37.825842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:37.825848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341200411029832:4813] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:37.826421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341200411029832:4813] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:37.833709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341200411029832:4813] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:37.841891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:37.842053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341200411029832:4813] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:37.842075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341200411029832:4813] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:37.842079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341200411029832:4813] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:37.842090Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 342 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:37.842937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:37.843162Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:37.843172Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:37.843186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:37.843252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:37.843295Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341200049218876:4457]
2025-10-09T21:44:37.843832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:37.844381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:37.844437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:37.844797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:37.844865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:37.844870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:37.844872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:37.844873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:37.844874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:37.844875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:37.844876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.844877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.844879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.844884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:37.844895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:37.844899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:37.844903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:37.844907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:37.844911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:37.844914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.844918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.844921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.844930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:37.844938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:37.844944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:37.844951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:37.844955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:37.844960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:37.844963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:37.844969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:37.844974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:37.844978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:37.844982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:37.844987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:37.844992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.844997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.845001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.845004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.845008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.845012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.845041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:37.853887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:37.855018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:37.855034Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:37.855037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:37.855041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:37.855049Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 341 [2:7559341200049218876:4457]
2025-10-09T21:44:37.855066Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:37.855085Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:37.855093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:37.855096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:37.855099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:37.855118Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:37.855129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:37.855136Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:37.855207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.855216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.855219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.855222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:37.855295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:37.855298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.855312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:37.855324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:37.855342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:37.855361Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341200049218870:4457]
2025-10-09T21:44:37.855364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341200049218870:4457]
2025-10-09T21:44:37.855366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341200049218870:4457]
2025-10-09T21:44:37.855378Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:37.855394Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:37.855411Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:37.855414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:37.855454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341200049218870:4457]
2025-10-09T21:44:37.855457Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:37.855886Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:37.855896Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:37.855899Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:37.855907Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:37.856069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341200411029809:2432], now have 1 active actors on pipe
2025-10-09T21:44:37.856075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341200411029966:4813], now have 1 active actors on pipe
2025-10-09T21:44:37.856723Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 341
2025-10-09T21:44:37.856767Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 341
2025-10-09T21:44:37.857206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:37.857218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:37.857265Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f71cc89f-fb634e68-c3c87580-5033f728_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:37.857280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:37.857289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:37.857292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:37.857304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:37.857317Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:37.857335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:37.860094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:37.860110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:37.860149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:37.860982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 342 partition: 0 MaxSeqNo: 25 sessionId: src_id|f71cc89f-fb634e68-c3c87580-5033f728_0
2025-10-09T21:44:37.861467Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbb05cf4-545d83a7-619e67e6-5f159461_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f71cc89f-fb634e68-c3c87580-5033f728_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:37.864080Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f71cc89f-fb634e68-c3c87580-5033f728_0] Write session: try to update token
2025-10-09T21:44:37.864102Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f71cc89f-fb634e68-c3c87580-5033f728_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:37.882253Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:37.882272Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:37.882274Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:37.882277Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:37.882285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341200049218870:4457]
2025-10-09T21:44:37.882290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341200049218870:4457]
2025-10-09T21:44:37.882294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341200049218870:4457]
2025-10-09T21:44:37.882349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:37.882358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:37.882360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:37.882495Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:37.885781Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:37.885791Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:37.885796Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:37.886697Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 342 sessionId: src_id|f71cc89f-fb634e68-c3c87580-5033f728_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:37.889287Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:37.916695Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341200411029984:4549]
2025-10-09T21:44:37.918690Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 341, pipe: [1:7559341200411029984:4549]
2025-10-09T21:44:38.061385Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:38.061980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:38.061991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:38.062125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341200411029984:4549], now have 1 active actors on pipe
2025-10-09T21:44:38.062513Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:38.062573Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341200411029984:4549]
2025-10-09T21:44:38.062592Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:38.062664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 342 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f71cc89f-fb634e68-c3c87580-5033f728_0
2025-10-09T21:44:38.062867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 342 sessionId: src_id|f71cc89f-fb634e68-c3c87580-5033f728_0 is DEAD
2025-10-09T21:44:38.062493Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:38.066145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f71cc89f-fb634e68-c3c87580-5033f728_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:38.066160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f71cc89f-fb634e68-c3c87580-5033f728_0] Write session will restart in 0.007447s
2025-10-09T21:44:38.066192Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f71cc89f-fb634e68-c3c87580-5033f728_0] Write session: Do CDS request
2025-10-09T21:44:38.066200Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f71cc89f-fb634e68-c3c87580-5033f728_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:38.077739Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f71cc89f-fb634e68-c3c87580-5033f728_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:38.078179Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:38.078201Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 343
2025-10-09T21:44:38.078399Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 343 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:38.078438Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 343 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:38.078444Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=343 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:38.078449Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:38.078848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 343 sessionId:  describe result for acl check
2025-10-09T21:44:38.078882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:38.078883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:38.078885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:38.078891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341204705997302:4818] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:38.079467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341204705997302:4818] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:38.095652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:38.097604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:38.097854Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:38.097864Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:38.097878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:38.098021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:38.091906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341204705997302:4818] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:38.094605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341204705997302:4818] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:38.094613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341204705997302:4818] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:38.094617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341204705997302:4818] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:38.098083Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341204344186338:4460]
2025-10-09T21:44:38.094626Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 343 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:38.098709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:38.098995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:38.099042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:38.099374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:38.099451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:38.099458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:38.099459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:38.099460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:38.099461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:38.099463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:38.099464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.099465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.099466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.099471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:38.099482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:38.099485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:38.099489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:38.099492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:38.099495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:38.099498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.099501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.099504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.099513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:38.099520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:38.099526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:38.099558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:38.099563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:38.099568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:38.099572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:38.099577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:38.099582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:38.099588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:38.099592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:38.099597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:38.099602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.099607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.099611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.099618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.099622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.099627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.099657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:38.106781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:38.108491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:38.108508Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:38.108511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:38.108515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:38.108524Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 342 [2:7559341204344186338:4460]
2025-10-09T21:44:38.108541Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:38.108564Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:38.108568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.108572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.108574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.108594Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:38.108600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:38.108608Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:38.108682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.108690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.108692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.108696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:38.108779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:38.108788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.108802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.108814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.108828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:38.108852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341204344186333:4460]
2025-10-09T21:44:38.108859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341204344186333:4460]
2025-10-09T21:44:38.108862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341204344186333:4460]
2025-10-09T21:44:38.108871Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:38.108880Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:38.108887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341204705997288:2432], now have 1 active actors on pipe
2025-10-09T21:44:38.108888Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:38.108891Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:38.108892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341204705997447:4818], now have 1 active actors on pipe
2025-10-09T21:44:38.108901Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:38.108916Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:38.108926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:38.108970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341204344186333:4460]
2025-10-09T21:44:38.108978Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:38.108996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:38.109194Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 342
2025-10-09T21:44:38.109239Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 342
2025-10-09T21:44:38.109478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:38.109490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:38.109522Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|277cd54e-1025e8c7-d55e6773-9f61df63_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:38.109533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.109542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.109545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:38.109557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.109571Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:38.109640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:38.111261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:38.111276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:38.111301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:38.111521Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 343 partition: 0 MaxSeqNo: 25 sessionId: src_id|277cd54e-1025e8c7-d55e6773-9f61df63_0
2025-10-09T21:44:38.114203Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f71cc89f-fb634e68-c3c87580-5033f728_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|277cd54e-1025e8c7-d55e6773-9f61df63_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:38.116931Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|277cd54e-1025e8c7-d55e6773-9f61df63_0] Write session: try to update token
2025-10-09T21:44:38.116957Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|277cd54e-1025e8c7-d55e6773-9f61df63_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:38.128278Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:38.128300Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:38.128305Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:38.128308Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:38.128336Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341204344186333:4460]
2025-10-09T21:44:38.128344Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341204344186333:4460]
2025-10-09T21:44:38.128348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341204344186333:4460]
2025-10-09T21:44:38.128405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:38.128413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.128416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:38.128416Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:38.128423Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:38.128426Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:38.128636Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:38.143064Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 343 sessionId: src_id|277cd54e-1025e8c7-d55e6773-9f61df63_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:38.145672Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:38.163401Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341204705997459:4549]
2025-10-09T21:44:38.163829Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 342, pipe: [1:7559341204705997459:4549]
2025-10-09T21:44:38.296947Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:38.297494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:38.297505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:38.297527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341204705997459:4549], now have 1 active actors on pipe
2025-10-09T21:44:38.297795Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:38.297954Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:38.298015Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:38.298521Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|277cd54e-1025e8c7-d55e6773-9f61df63_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:38.298129Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 343 reason: status is not ok: tablet will be restarted right now sessionId: src_id|277cd54e-1025e8c7-d55e6773-9f61df63_0
2025-10-09T21:44:38.298533Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|277cd54e-1025e8c7-d55e6773-9f61df63_0] Write session will restart in 0.008043s
2025-10-09T21:44:38.298267Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341204705997459:4549]
2025-10-09T21:44:38.298351Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 343 sessionId: src_id|277cd54e-1025e8c7-d55e6773-9f61df63_0 is DEAD
2025-10-09T21:44:38.298560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|277cd54e-1025e8c7-d55e6773-9f61df63_0] Write session: Do CDS request
2025-10-09T21:44:38.298568Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|277cd54e-1025e8c7-d55e6773-9f61df63_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:38.307581Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|277cd54e-1025e8c7-d55e6773-9f61df63_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:38.307775Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:38.307795Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 344
2025-10-09T21:44:38.308056Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 344 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:38.308101Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 344 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:38.308107Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=344 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:38.308116Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:38.308601Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 344 sessionId:  describe result for acl check
2025-10-09T21:44:38.308653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:38.308655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:38.308658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:38.308664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341204705997478:4822] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:38.309298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341204705997478:4822] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:38.313268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341204705997478:4822] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:38.318317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341204705997478:4822] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:38.318348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341204705997478:4822] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:38.318352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341204705997478:4822] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:38.318379Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 344 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:38.321845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:38.324492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:38.324728Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:38.324737Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:38.324750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:38.324821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:38.324864Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341204344186505:4463]
2025-10-09T21:44:38.325500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:38.325876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:38.325947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:38.326320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:38.326393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:38.326399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:38.326400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:38.326402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:38.326403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:38.326404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:38.326405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.326407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.326408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.326413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:38.326424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:38.326429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:38.326433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:38.326437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:38.326442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:38.326446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.326450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.326454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.326462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:38.326470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:38.326476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:38.326482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:38.326495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:38.326501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:38.326506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:38.326511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:38.326517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:38.326527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:38.326531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:38.326537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:38.326544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.326549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.326558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.326564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.326571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.326576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.326607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:38.335085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:38.336663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:38.336681Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:38.336684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:38.336689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:38.336695Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 343 [2:7559341204344186505:4463]
2025-10-09T21:44:38.336723Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:38.336745Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:38.336753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.336757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.336759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.336780Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:38.336787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:38.336794Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:38.336862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.336870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.336872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.336875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:38.336949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:38.336957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.336972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.336988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.337005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:38.337035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341204705997610:4822], now have 1 active actors on pipe
2025-10-09T21:44:38.337039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341204705997462:2432], now have 1 active actors on pipe
2025-10-09T21:44:38.337046Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341204344186500:4463]
2025-10-09T21:44:38.337049Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341204344186500:4463]
2025-10-09T21:44:38.337051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341204344186500:4463]
2025-10-09T21:44:38.337064Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:38.337082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:38.337093Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:38.337097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:38.337097Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:38.337099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:38.337101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:38.337143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341204344186500:4463]
2025-10-09T21:44:38.337149Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:38.337160Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:38.337142Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 343
2025-10-09T21:44:38.337187Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 343
2025-10-09T21:44:38.337206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:38.337220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:38.337249Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b94e03-3b1bd2d0-a0e52a90-60a8fe58_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:38.337258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.337262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.337264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:38.337275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.337288Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:38.337310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:38.337739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:38.337751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:38.337784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:38.342438Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 344 partition: 0 MaxSeqNo: 25 sessionId: src_id|b94e03-3b1bd2d0-a0e52a90-60a8fe58_0
2025-10-09T21:44:38.346357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|277cd54e-1025e8c7-d55e6773-9f61df63_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b94e03-3b1bd2d0-a0e52a90-60a8fe58_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:38.349104Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b94e03-3b1bd2d0-a0e52a90-60a8fe58_0] Write session: try to update token
2025-10-09T21:44:38.349145Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b94e03-3b1bd2d0-a0e52a90-60a8fe58_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:38.357035Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:38.357051Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:38.357054Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:38.357056Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:38.357066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341204344186500:4463]
2025-10-09T21:44:38.357070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341204344186500:4463]
2025-10-09T21:44:38.357074Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341204344186500:4463]
2025-10-09T21:44:38.357127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:38.357139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.357142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:38.357229Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:38.357236Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:38.357240Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:38.357285Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:38.385456Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 344 sessionId: src_id|b94e03-3b1bd2d0-a0e52a90-60a8fe58_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:38.388046Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:38.398460Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341204705997637:4549]
2025-10-09T21:44:38.398810Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 343, pipe: [1:7559341204705997637:4549]
2025-10-09T21:44:38.548416Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:38.548937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:38.548957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:38.548980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341204705997637:4549], now have 1 active actors on pipe
2025-10-09T21:44:38.549261Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341204705997637:4549]
2025-10-09T21:44:38.549283Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:38.549288Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:38.549296Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:38.549348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 344 reason: error at write: Disconnected sessionId: src_id|b94e03-3b1bd2d0-a0e52a90-60a8fe58_0
2025-10-09T21:44:38.549563Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 344 sessionId: src_id|b94e03-3b1bd2d0-a0e52a90-60a8fe58_0 is DEAD
2025-10-09T21:44:38.549467Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:38.550259Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b94e03-3b1bd2d0-a0e52a90-60a8fe58_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:38.550272Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b94e03-3b1bd2d0-a0e52a90-60a8fe58_0] Write session will restart in 0.008111s
2025-10-09T21:44:38.550311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b94e03-3b1bd2d0-a0e52a90-60a8fe58_0] Write session: Do CDS request
2025-10-09T21:44:38.550319Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b94e03-3b1bd2d0-a0e52a90-60a8fe58_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:38.559810Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b94e03-3b1bd2d0-a0e52a90-60a8fe58_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:38.559963Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:38.559984Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 345
2025-10-09T21:44:38.562320Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 345 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:38.562398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 345 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:38.562410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=345 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:38.562415Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:38.563120Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 345 sessionId:  describe result for acl check
2025-10-09T21:44:38.563162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:38.563164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:38.563166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:38.563172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341204705997691:4834] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:38.563763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341204705997691:4834] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:38.565785Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341204705997691:4834] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:38.570713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341204705997691:4834] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:38.570728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341204705997691:4834] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:38.570732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341204705997691:4834] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:38.570740Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 345 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:38.579126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:38.582074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:38.582298Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:38.582313Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:38.582327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:38.582403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:38.582454Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341204344186710:4477]
2025-10-09T21:44:38.583045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:38.583729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:38.583774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:38.584096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:38.584141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:38.584142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:38.584143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:38.584144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:38.584145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:38.584146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:38.584148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.584149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.584150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.584154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:38.584164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:38.584167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:38.584171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:38.584174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:38.584178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:38.584181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.584185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.584188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.584194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:38.584201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:38.584207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:38.584212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:38.584215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:38.584219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:38.584223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:38.584227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:38.584230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:38.584235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:38.584239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:38.584243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:38.584247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.584252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.584255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.584259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.584263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.584267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.584292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:38.597309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:38.598867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:38.598884Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:38.598886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:38.598892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:38.598902Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 344 [2:7559341204344186710:4477]
2025-10-09T21:44:38.598916Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:38.598938Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:38.598944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.598947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.598949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.598972Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:38.598978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:38.598994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:38.599060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.599069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.599071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.599074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:38.599151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:38.599159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.599173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.599185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.599202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:38.599224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341204344186704:4477]
2025-10-09T21:44:38.599233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341204344186704:4477]
2025-10-09T21:44:38.599235Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341204344186704:4477]
2025-10-09T21:44:38.599244Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:38.599259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:38.599280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:38.599283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:38.599290Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:38.599684Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 344
2025-10-09T21:44:38.599689Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 344
2025-10-09T21:44:38.599305Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:38.599313Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:38.599326Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341204344186704:4477]
2025-10-09T21:44:38.599331Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:38.599332Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:38.599355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341204705997823:4834], now have 1 active actors on pipe
2025-10-09T21:44:38.599358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341204705997667:2432], now have 1 active actors on pipe
2025-10-09T21:44:38.600308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:38.600331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:38.600366Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dc849bdf-7bdfa2a2-3e5c18e9-198b0d54_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:38.600376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.600383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.600386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:38.600399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.600414Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:38.600437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:38.601822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:38.601838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:38.601861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:38.601992Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 345 partition: 0 MaxSeqNo: 25 sessionId: src_id|dc849bdf-7bdfa2a2-3e5c18e9-198b0d54_0
2025-10-09T21:44:38.602540Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b94e03-3b1bd2d0-a0e52a90-60a8fe58_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dc849bdf-7bdfa2a2-3e5c18e9-198b0d54_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:38.604939Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dc849bdf-7bdfa2a2-3e5c18e9-198b0d54_0] Write session: try to update token
2025-10-09T21:44:38.604959Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dc849bdf-7bdfa2a2-3e5c18e9-198b0d54_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:38.614992Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:38.615015Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:38.615018Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:38.615020Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:38.615032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341204344186704:4477]
2025-10-09T21:44:38.615038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341204344186704:4477]
2025-10-09T21:44:38.615041Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341204344186704:4477]
2025-10-09T21:44:38.615086Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:38.615092Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:38.615092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:38.615095Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:38.615100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.615103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:38.615290Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:38.624490Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 345 sessionId: src_id|dc849bdf-7bdfa2a2-3e5c18e9-198b0d54_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:38.627141Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:38.649367Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341204705997838:4549]
2025-10-09T21:44:38.649711Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 344, pipe: [1:7559341204705997838:4549]
2025-10-09T21:44:38.822085Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:38.822600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:38.822610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:38.822635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341204705997838:4549], now have 1 active actors on pipe
2025-10-09T21:44:38.823085Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:38.824874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:38.825923Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc849bdf-7bdfa2a2-3e5c18e9-198b0d54_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:38.825943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc849bdf-7bdfa2a2-3e5c18e9-198b0d54_0] Write session will restart in 0.005975s
2025-10-09T21:44:38.825967Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc849bdf-7bdfa2a2-3e5c18e9-198b0d54_0] Write session: Do CDS request
2025-10-09T21:44:38.825974Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc849bdf-7bdfa2a2-3e5c18e9-198b0d54_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:38.824976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 345 reason: status is not ok: tablet will be restarted right now sessionId: src_id|dc849bdf-7bdfa2a2-3e5c18e9-198b0d54_0
2025-10-09T21:44:38.825158Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 345 sessionId: src_id|dc849bdf-7bdfa2a2-3e5c18e9-198b0d54_0 is DEAD
2025-10-09T21:44:38.825412Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:38.825447Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341204705997838:4549]
2025-10-09T21:44:38.834507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dc849bdf-7bdfa2a2-3e5c18e9-198b0d54_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:38.834613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:38.834633Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 346
2025-10-09T21:44:38.834846Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 346 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:38.834887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 346 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:38.834897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=346 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:38.834901Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:38.835338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 346 sessionId:  describe result for acl check
2025-10-09T21:44:38.835385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:38.835387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:38.835389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:38.835395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341204705997850:4838] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:38.835940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341204705997850:4838] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:38.847869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341204705997850:4838] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:38.854856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341204705997850:4838] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:38.854878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341204705997850:4838] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:38.854883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341204705997850:4838] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:38.854892Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 346 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:38.860330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:38.863025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:38.863291Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:38.863296Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:38.863310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:38.863379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:38.863420Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341204344186877:4479]
2025-10-09T21:44:38.863970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:38.865715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:38.865778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:38.866134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:38.866213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:38.866215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:38.866216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:38.866236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:38.866237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:38.866238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:38.866239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.866241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.866242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.866248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:38.866258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:38.866262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:38.866266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:38.866269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:38.866272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:38.866275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.866278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.866281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.866289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:38.866297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:38.866303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:38.866308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:38.866311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:38.866317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:38.866322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:38.866326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:38.866330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:38.866348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:38.866352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:38.866357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:38.866361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.866366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.866369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.866374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.866382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.866386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.866413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:38.873474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:38.874977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:38.874997Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:38.874999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:38.875003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:38.875010Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 345 [2:7559341204344186877:4479]
2025-10-09T21:44:38.875026Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:38.875045Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:38.875048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.875052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.875054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.875074Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:38.875086Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:38.875093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:38.875164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.875170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.875172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.875176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:38.875482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 345
2025-10-09T21:44:38.875534Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 345
2025-10-09T21:44:38.875251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:38.875260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.875272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:38.875281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:38.875294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:38.875314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341204344186872:4479]
2025-10-09T21:44:38.875316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341204344186872:4479]
2025-10-09T21:44:38.875318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341204344186872:4479]
2025-10-09T21:44:38.875328Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:38.875337Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:38.875340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:38.875341Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:38.875343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:38.875355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:38.875358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:38.875370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341204705997979:4838], now have 1 active actors on pipe
2025-10-09T21:44:38.875376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341204705997843:2432], now have 1 active actors on pipe
2025-10-09T21:44:38.875413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341204344186872:4479]
2025-10-09T21:44:38.875423Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:38.875484Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:38.875858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:38.875870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:38.875905Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d7b815bb-e233105e-f28cde79-20e2fc45_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:38.875914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.875922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.875925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:38.875937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.875948Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:38.875965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:38.876296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:38.876305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:38.876323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:38.876457Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 346 partition: 0 MaxSeqNo: 25 sessionId: src_id|d7b815bb-e233105e-f28cde79-20e2fc45_0
2025-10-09T21:44:38.876887Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc849bdf-7bdfa2a2-3e5c18e9-198b0d54_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d7b815bb-e233105e-f28cde79-20e2fc45_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:38.879509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7b815bb-e233105e-f28cde79-20e2fc45_0] Write session: try to update token
2025-10-09T21:44:38.879530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7b815bb-e233105e-f28cde79-20e2fc45_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:38.893492Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:38.893514Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:38.893517Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:38.893519Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:38.893529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341204344186872:4479]
2025-10-09T21:44:38.893534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341204344186872:4479]
2025-10-09T21:44:38.893537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341204344186872:4479]
2025-10-09T21:44:38.893588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:38.893597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:38.893602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:38.893665Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:38.893671Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:38.893674Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:38.893737Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:38.899977Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 346 sessionId: src_id|d7b815bb-e233105e-f28cde79-20e2fc45_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:38.902367Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:38.925920Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341204705998020:4549]
2025-10-09T21:44:38.926294Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 345, pipe: [1:7559341204705998020:4549]
2025-10-09T21:44:39.061449Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:39.061857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:39.061868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:39.061917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341204705998020:4549], now have 1 active actors on pipe
2025-10-09T21:44:39.063266Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7b815bb-e233105e-f28cde79-20e2fc45_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:39.063278Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7b815bb-e233105e-f28cde79-20e2fc45_0] Write session will restart in 0.007387s
2025-10-09T21:44:39.063301Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7b815bb-e233105e-f28cde79-20e2fc45_0] Write session: Do CDS request
2025-10-09T21:44:39.063307Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7b815bb-e233105e-f28cde79-20e2fc45_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:39.062929Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:39.062484Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:39.062501Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:39.062584Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 346 reason: error at write: Disconnected sessionId: src_id|d7b815bb-e233105e-f28cde79-20e2fc45_0
2025-10-09T21:44:39.062763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 346 sessionId: src_id|d7b815bb-e233105e-f28cde79-20e2fc45_0 is DEAD
2025-10-09T21:44:39.062843Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:39.062884Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341204705998020:4549]
2025-10-09T21:44:39.071588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7b815bb-e233105e-f28cde79-20e2fc45_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:39.071610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:39.071628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 347
2025-10-09T21:44:39.071900Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 347 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:39.071930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 347 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:39.071937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=347 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:39.071941Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:39.072343Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 347 sessionId:  describe result for acl check
2025-10-09T21:44:39.072374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:39.072375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:39.072378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:39.072383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341209000965336:4843] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:39.072926Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341209000965336:4843] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:39.077727Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341209000965336:4843] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:39.088325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341209000965336:4843] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:39.088345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341209000965336:4843] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:39.088350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341209000965336:4843] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:39.088360Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 347 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:39.091458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:39.095272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:39.095541Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:39.095553Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:39.095566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:39.095647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:39.095925Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341208639154343:4482]
2025-10-09T21:44:39.096718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:39.101517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:39.101610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:39.102070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:39.102153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:39.102162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:39.102164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:39.102166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:39.102168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:39.102170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:39.102171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.102173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.102175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.102182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:39.102195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:39.102200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:39.102205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:39.102210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:39.102215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:39.102219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.102224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.102228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.102238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:39.102250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:39.102258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:39.102264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:39.102268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:39.102275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:39.102279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:39.102284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:39.102289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:39.102293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:39.102297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:39.102302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:39.102307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.102312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.102317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.102322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.102327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.102331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.102376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:39.118658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:39.119789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:39.119810Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:39.119813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:39.119819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:39.119827Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 346 [2:7559341208639154343:4482]
2025-10-09T21:44:39.119845Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:39.119866Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:39.119870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.119874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.119876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.119897Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:39.119914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:39.119922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:39.119998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.120001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.120003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.120006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:39.120082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:39.120085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.120102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.120115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.120135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:39.120157Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341208639154337:4482]
2025-10-09T21:44:39.120160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341208639154337:4482]
2025-10-09T21:44:39.120162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341208639154337:4482]
2025-10-09T21:44:39.120179Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:39.120195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:39.120211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:39.120215Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:39.120259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341208639154337:4482]
2025-10-09T21:44:39.120262Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:39.120672Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:39.120676Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:39.120678Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:39.120686Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:39.121457Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 346
2025-10-09T21:44:39.121505Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 346
2025-10-09T21:44:39.122292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341209000965482:4843], now have 1 active actors on pipe
2025-10-09T21:44:39.122305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341209000965322:2432], now have 1 active actors on pipe
2025-10-09T21:44:39.122345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:39.122351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:39.122389Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4001c22f-e2a68f89-1b5acf38-c93886f6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:39.122402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.122408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.122411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:39.122428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.122445Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:39.122465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:39.123352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:39.123364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:39.123389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:39.126275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 347 partition: 0 MaxSeqNo: 25 sessionId: src_id|4001c22f-e2a68f89-1b5acf38-c93886f6_0
2025-10-09T21:44:39.126767Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7b815bb-e233105e-f28cde79-20e2fc45_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4001c22f-e2a68f89-1b5acf38-c93886f6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:39.129528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4001c22f-e2a68f89-1b5acf38-c93886f6_0] Write session: try to update token
2025-10-09T21:44:39.129553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4001c22f-e2a68f89-1b5acf38-c93886f6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:39.142243Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:39.142261Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:39.142263Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:39.142265Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:39.142276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341208639154337:4482]
2025-10-09T21:44:39.142281Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341208639154337:4482]
2025-10-09T21:44:39.142285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341208639154337:4482]
2025-10-09T21:44:39.142347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:39.142356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.142359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:39.142504Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:39.146631Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:39.146643Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:39.146647Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:39.153032Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 347 sessionId: src_id|4001c22f-e2a68f89-1b5acf38-c93886f6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:39.155747Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:39.162417Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341209000965497:4549]
2025-10-09T21:44:39.162978Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 346, pipe: [1:7559341209000965497:4549]
2025-10-09T21:44:39.332687Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:39.333214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:39.333234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:39.333276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341209000965497:4549], now have 1 active actors on pipe
2025-10-09T21:44:39.333569Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:39.333656Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:39.333661Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341209000965497:4549]
2025-10-09T21:44:39.333673Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:39.333706Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:39.333735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 347 reason: error at write: Disconnected sessionId: src_id|4001c22f-e2a68f89-1b5acf38-c93886f6_0
2025-10-09T21:44:39.333941Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 347 sessionId: src_id|4001c22f-e2a68f89-1b5acf38-c93886f6_0 is DEAD
2025-10-09T21:44:39.334385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4001c22f-e2a68f89-1b5acf38-c93886f6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:39.334394Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4001c22f-e2a68f89-1b5acf38-c93886f6_0] Write session will restart in 0.007979s
2025-10-09T21:44:39.334421Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4001c22f-e2a68f89-1b5acf38-c93886f6_0] Write session: Do CDS request
2025-10-09T21:44:39.334427Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4001c22f-e2a68f89-1b5acf38-c93886f6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:39.344372Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4001c22f-e2a68f89-1b5acf38-c93886f6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:39.346367Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:39.346384Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 348
2025-10-09T21:44:39.353718Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 348 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:39.353775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 348 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:39.353781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=348 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:39.353786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:39.358508Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 348 sessionId:  describe result for acl check
2025-10-09T21:44:39.358564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:39.358568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:39.358571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:39.358577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341209000965618:4847] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:39.359060Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341209000965618:4847] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:39.361752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341209000965618:4847] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:39.364832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:39.367566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:39.367789Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:39.367793Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:39.367809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:39.367874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:39.367912Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341208639154513:4485]
2025-10-09T21:44:39.368481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:39.368958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341209000965618:4847] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:39.368975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341209000965618:4847] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:39.368979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341209000965618:4847] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:39.368989Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 348 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:39.369962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:39.370012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:39.370365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:39.370426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:39.370427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:39.370428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:39.370429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:39.370431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:39.370432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:39.370433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.370434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.370435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.370441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:39.370452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:39.370456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:39.370459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:39.370463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:39.370467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:39.370471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.370474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.370477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.370485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:39.370493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:39.370498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:39.370503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:39.370507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:39.370512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:39.370517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:39.370522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:39.370527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:39.370532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:39.370536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:39.370541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:39.370546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.370551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.370555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.370560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.370564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.370569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.370596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:39.378153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:39.379274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:39.379290Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:39.379292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:39.379297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:39.379305Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 347 [2:7559341208639154513:4485]
2025-10-09T21:44:39.379320Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:39.379343Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:39.379351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.379354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.379357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.379376Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:39.379382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:39.379390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:39.379453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.379464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.379466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.379469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:39.379542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:39.379549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.379563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.379576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.379593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:39.379614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341208639154507:4485]
2025-10-09T21:44:39.379616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341208639154507:4485]
2025-10-09T21:44:39.379619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341208639154507:4485]
2025-10-09T21:44:39.379628Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:39.379636Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:39.379641Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:39.379643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:39.379644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:39.379660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:39.379663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:39.379675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341209000965665:4847], now have 1 active actors on pipe
2025-10-09T21:44:39.379679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341209000965500:2432], now have 1 active actors on pipe
2025-10-09T21:44:39.379716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341208639154507:4485]
2025-10-09T21:44:39.379727Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:39.379755Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:39.379909Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 347
2025-10-09T21:44:39.379943Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 347
2025-10-09T21:44:39.380161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:39.380173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:39.380212Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|78579a83-5bec7656-346cc630-1c52845f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:39.380222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.380226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.380229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:39.380241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.380252Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:39.380279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:39.380756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:39.380766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:39.380783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:39.383283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 348 partition: 0 MaxSeqNo: 25 sessionId: src_id|78579a83-5bec7656-346cc630-1c52845f_0
2025-10-09T21:44:39.383679Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4001c22f-e2a68f89-1b5acf38-c93886f6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|78579a83-5bec7656-346cc630-1c52845f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:39.386317Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78579a83-5bec7656-346cc630-1c52845f_0] Write session: try to update token
2025-10-09T21:44:39.386351Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78579a83-5bec7656-346cc630-1c52845f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:39.403796Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:39.403814Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:39.403826Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341208639154507:4485]
2025-10-09T21:44:39.403883Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:39.403894Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:39.404080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:39.404224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:39.404384Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:39.404519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:39.404746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:39.404789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:39.410281Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 348 sessionId: src_id|78579a83-5bec7656-346cc630-1c52845f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:39.413089Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:39.414515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:39.414542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:39.414567Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:39.414570Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:39.414573Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:39.414575Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:39.414584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341208639154507:4485]
2025-10-09T21:44:39.414588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341208639154507:4485]
2025-10-09T21:44:39.414592Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341208639154507:4485]
2025-10-09T21:44:39.414648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:39.414661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.414663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:39.414680Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:39.414689Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:39.414693Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:39.414803Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:39.433791Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341209000965677:4549]
2025-10-09T21:44:39.434126Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 347, pipe: [1:7559341209000965677:4549]
2025-10-09T21:44:39.653679Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:39.654359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:39.654398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341209000965677:4549], now have 1 active actors on pipe
2025-10-09T21:44:39.658597Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:39.658955Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:39.658958Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:39.659036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 348 reason: error at write: Disconnected sessionId: src_id|78579a83-5bec7656-346cc630-1c52845f_0
2025-10-09T21:44:39.659176Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 348 sessionId: src_id|78579a83-5bec7656-346cc630-1c52845f_0 is DEAD
2025-10-09T21:44:39.659305Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341209000965677:4549]
2025-10-09T21:44:39.659330Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:39.662189Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78579a83-5bec7656-346cc630-1c52845f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:39.662204Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78579a83-5bec7656-346cc630-1c52845f_0] Write session will restart in 0.005058s
2025-10-09T21:44:39.662229Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78579a83-5bec7656-346cc630-1c52845f_0] Write session: Do CDS request
2025-10-09T21:44:39.662238Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78579a83-5bec7656-346cc630-1c52845f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:39.670215Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78579a83-5bec7656-346cc630-1c52845f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:39.670969Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:39.670987Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 349
2025-10-09T21:44:39.671109Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 349 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:39.671137Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 349 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:39.671144Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=349 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:39.671148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:39.671613Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 349 sessionId:  describe result for acl check
2025-10-09T21:44:39.671650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:39.671659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:39.671661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:39.671667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341209000965721:4859] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:39.672534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341209000965721:4859] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:39.681378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341209000965721:4859] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:39.689536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341209000965721:4859] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:39.689558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341209000965721:4859] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:39.689564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341209000965721:4859] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:39.689575Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 349 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:39.693477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:39.696620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:39.696866Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:39.696877Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:39.696890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:39.696957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:39.697045Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341208639154722:4499]
2025-10-09T21:44:39.697613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:39.698053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:39.698117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:39.698485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:39.698538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:39.698539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:39.698541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:39.698543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:39.698544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:39.698545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:39.698547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.698548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.698550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.698556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:39.698568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:39.698573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:39.698577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:39.698582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:39.698587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:39.698591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.698595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.698600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.698609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:39.698619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:39.698625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:39.698631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:39.698636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:39.698642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:39.698648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:39.698654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:39.698659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:39.698665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:39.698669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:39.698675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:39.698681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.698686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.698691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.698697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.698702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.698708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.698740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:39.708948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:39.710128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:39.710144Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:39.710147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:39.710154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:39.710162Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 348 [2:7559341208639154722:4499]
2025-10-09T21:44:39.710180Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:39.710202Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:39.710205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.710209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.710213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.710236Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:39.710242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:39.710250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:39.710321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.710323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.710325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.710328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:39.710421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:39.710424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.710440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.710454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.710474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:39.710495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341208639154717:4499]
2025-10-09T21:44:39.710498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341208639154717:4499]
2025-10-09T21:44:39.710501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341208639154717:4499]
2025-10-09T21:44:39.710516Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:39.710533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:39.711594Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 348
2025-10-09T21:44:39.711650Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 348
2025-10-09T21:44:39.710552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:39.710560Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:39.710603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341208639154717:4499]
2025-10-09T21:44:39.710606Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:39.710615Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:39.710618Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:39.710620Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:39.710627Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:39.710803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341209000965709:2432], now have 1 active actors on pipe
2025-10-09T21:44:39.710810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341209000965865:4859], now have 1 active actors on pipe
2025-10-09T21:44:39.711948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:39.711957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:39.712005Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ecf0e7d4-37631074-8b044298-6ee9e591_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:39.712016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.712021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.712024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:39.712042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.712067Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:39.712086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:39.718052Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78579a83-5bec7656-346cc630-1c52845f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ecf0e7d4-37631074-8b044298-6ee9e591_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:39.717588Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 349 partition: 0 MaxSeqNo: 25 sessionId: src_id|ecf0e7d4-37631074-8b044298-6ee9e591_0
2025-10-09T21:44:39.717373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:39.717392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:39.717427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:39.720833Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf0e7d4-37631074-8b044298-6ee9e591_0] Write session: try to update token
2025-10-09T21:44:39.720852Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf0e7d4-37631074-8b044298-6ee9e591_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:39.736157Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:39.736176Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:39.736180Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:39.736182Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:39.736193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341208639154717:4499]
2025-10-09T21:44:39.736198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341208639154717:4499]
2025-10-09T21:44:39.736202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341208639154717:4499]
2025-10-09T21:44:39.736252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:39.736256Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:39.736260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.736263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:39.736269Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:39.736273Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:39.736427Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:39.742111Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 349 sessionId: src_id|ecf0e7d4-37631074-8b044298-6ee9e591_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:39.744826Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:39.758872Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341209000965882:4549]
2025-10-09T21:44:39.759548Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 348, pipe: [1:7559341209000965882:4549]
2025-10-09T21:44:39.908934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf0e7d4-37631074-8b044298-6ee9e591_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:39.908968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf0e7d4-37631074-8b044298-6ee9e591_0] Write session will restart in 0.007023s
2025-10-09T21:44:39.908993Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf0e7d4-37631074-8b044298-6ee9e591_0] Write session: Do CDS request
2025-10-09T21:44:39.909000Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf0e7d4-37631074-8b044298-6ee9e591_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:39.907118Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:39.907383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:39.907395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:39.907445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341209000965882:4549], now have 1 active actors on pipe
2025-10-09T21:44:39.908882Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:39.908399Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:39.908508Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:39.908520Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:39.908593Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 349 reason: error at write: Disconnected sessionId: src_id|ecf0e7d4-37631074-8b044298-6ee9e591_0
2025-10-09T21:44:39.908782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 349 sessionId: src_id|ecf0e7d4-37631074-8b044298-6ee9e591_0 is DEAD
2025-10-09T21:44:39.908921Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341209000965882:4549]
2025-10-09T21:44:39.919703Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf0e7d4-37631074-8b044298-6ee9e591_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:39.922379Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:39.922398Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 350
2025-10-09T21:44:39.922578Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 350 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:39.922613Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 350 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:39.922619Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=350 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:39.922623Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:39.923120Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 350 sessionId:  describe result for acl check
2025-10-09T21:44:39.923152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:39.923156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:39.923157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:39.923163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341209000965939:4863] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:39.923992Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341209000965939:4863] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:39.927031Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341209000965939:4863] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:39.932056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:39.931686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341209000965939:4863] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:39.931705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341209000965939:4863] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:39.931709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341209000965939:4863] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:39.931718Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 350 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:39.935419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:39.935663Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:39.935674Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:39.935688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:39.935773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:39.935823Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341208639154892:4501]
2025-10-09T21:44:39.936390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:39.936889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:39.936949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:39.937308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:39.937363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:39.937369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:39.937370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:39.937371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:39.937373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:39.937374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:39.937375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.937376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.937377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.937382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:39.937392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:39.937396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:39.937400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:39.937403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:39.937408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:39.937411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.937414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.937418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.937426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:39.937435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:39.937440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:39.937445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:39.937449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:39.937454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:39.937458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:39.937470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:39.937475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:39.937480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:39.937489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:39.937494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:39.937499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.937508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.937511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.937516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.937521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.937525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.937552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:39.945316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:39.946923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:39.946943Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:39.946946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:39.946950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:39.946957Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 349 [2:7559341208639154892:4501]
2025-10-09T21:44:39.946972Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:39.946993Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:39.946996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.947000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.947003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.947023Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:39.947029Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:39.947037Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:39.947105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.947114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.947116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.947119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:39.947196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:39.947206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.947220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:39.947229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:39.947242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:39.947264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341208639154885:4501]
2025-10-09T21:44:39.947271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341208639154885:4501]
2025-10-09T21:44:39.947273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341208639154885:4501]
2025-10-09T21:44:39.947289Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:39.947309Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:39.947322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:39.947325Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:39.947340Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:39.947343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:39.947357Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:39.947365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341208639154885:4501]
2025-10-09T21:44:39.947368Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:39.947448Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:39.947558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341209000966057:4863], now have 1 active actors on pipe
2025-10-09T21:44:39.947565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341209000965890:2432], now have 1 active actors on pipe
2025-10-09T21:44:39.947719Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 349
2025-10-09T21:44:39.947785Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 349
2025-10-09T21:44:39.947892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:39.947906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:39.947939Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3ec4d558-17c96b5-6640f99c-e6945a17_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:39.947964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.947968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.947971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:39.947984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.947998Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:39.948030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:39.948220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:39.948230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:39.948247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:39.948395Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 350 partition: 0 MaxSeqNo: 25 sessionId: src_id|3ec4d558-17c96b5-6640f99c-e6945a17_0
2025-10-09T21:44:39.948799Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf0e7d4-37631074-8b044298-6ee9e591_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3ec4d558-17c96b5-6640f99c-e6945a17_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:39.951830Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3ec4d558-17c96b5-6640f99c-e6945a17_0] Write session: try to update token
2025-10-09T21:44:39.951849Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3ec4d558-17c96b5-6640f99c-e6945a17_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:39.963165Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:39.963184Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:39.963195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341208639154885:4501]
2025-10-09T21:44:39.963245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:39.963574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:39.963832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:39.964029Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:39.964165Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:39.964349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:39.964389Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:39.964711Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:39.970809Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 350 sessionId: src_id|3ec4d558-17c96b5-6640f99c-e6945a17_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:39.973467Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:39.973716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:39.973736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:39.973779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:39.973894Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:39.984043Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:39.984058Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:39.984060Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:39.984062Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:39.984073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341208639154885:4501]
2025-10-09T21:44:39.984077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341208639154885:4501]
2025-10-09T21:44:39.984080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341208639154885:4501]
2025-10-09T21:44:39.984120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:39.984127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:39.984129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:39.984260Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:39.986200Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:39.986206Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:39.986210Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:40.011545Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341213295933369:4549]
2025-10-09T21:44:40.013769Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 349, pipe: [1:7559341213295933369:4549]
2025-10-09T21:44:40.213738Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:40.222440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:40.222468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:40.222642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:40.222702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:40.222768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:40.222828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:40.222870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:40.222876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:40.222944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:40.223003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:40.223067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:40.223131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:40.223171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:40.223174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:40.223240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:40.223293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:40.223357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:40.223416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:40.223456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:40.223459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:40.223528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:40.223589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:40.223647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:40.223702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:40.223744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:40.223747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:40.223812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:40.223872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:40.223932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:40.223992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:40.224034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:40.224036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:40.224104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:40.224161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:40.224217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:40.224279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:40.224320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:40.224323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:40.224392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:40.224464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:40.224522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:40.224579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:40.224623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:40.224627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:40.224692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:40.224744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:40.224801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:40.224861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:40.224924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:40.224927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:40.224993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:40.225050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:40.225109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:40.225170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:40.225208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:40.225212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:40.225277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:40.225330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:40.225395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:40.225457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:40.225501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:40.225504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:40.225593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341213295933369:4549], now have 1 active actors on pipe
2025-10-09T21:44:40.226119Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:40.226139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:40.226173Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 350 reason: error at write: Disconnected sessionId: src_id|3ec4d558-17c96b5-6640f99c-e6945a17_0
2025-10-09T21:44:40.226316Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341213295933369:4549]
2025-10-09T21:44:40.226348Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:40.226369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 350 sessionId: src_id|3ec4d558-17c96b5-6640f99c-e6945a17_0 is DEAD
2025-10-09T21:44:40.226844Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ec4d558-17c96b5-6640f99c-e6945a17_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:40.226855Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ec4d558-17c96b5-6640f99c-e6945a17_0] Write session will restart in 0.006181s
2025-10-09T21:44:40.226877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ec4d558-17c96b5-6640f99c-e6945a17_0] Write session: Do CDS request
2025-10-09T21:44:40.226884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ec4d558-17c96b5-6640f99c-e6945a17_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:40.226995Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:40.238505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3ec4d558-17c96b5-6640f99c-e6945a17_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:40.242386Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:40.242407Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 351
2025-10-09T21:44:40.242588Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 351 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:40.242621Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 351 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:40.242628Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=351 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:40.242632Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:40.250641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 351 sessionId:  describe result for acl check
2025-10-09T21:44:40.250699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:40.250701Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:40.250707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:40.250715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341213295933388:4868] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:40.251335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341213295933388:4868] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:40.268695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341213295933388:4868] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:40.274806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:40.275791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341213295933388:4868] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:40.275811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341213295933388:4868] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:40.275815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341213295933388:4868] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:40.275825Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 351 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:40.277769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:40.278134Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:40.278146Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:40.278161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:40.278233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:40.278281Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341212934122363:4504]
2025-10-09T21:44:40.278900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:40.279305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:40.279358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:40.279691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:40.279762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:40.279763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:40.279764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:40.279765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:40.279767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:40.279768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:40.279769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.279770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.279771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.279776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:40.279788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:40.279794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:40.279797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:40.279801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:40.279805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:40.279808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.279811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.279814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.279823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:40.279832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:40.279838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:40.279843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:40.279848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:40.279853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:40.279857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:40.279862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:40.279867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:40.279872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:40.279878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:40.279883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:40.279889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.279893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.279897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.279902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.279906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.279911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.279942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:40.287097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:40.288221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:40.288236Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:40.288240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:40.288245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:40.288252Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 350 [2:7559341212934122363:4504]
2025-10-09T21:44:40.288267Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:40.288289Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:40.288293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:40.288296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:40.288299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:40.288319Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:40.288327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:40.288335Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:40.288735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.288746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.288749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.288752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:40.288830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:40.288838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.288851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.288862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.288879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:40.288903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341213295933542:4868], now have 1 active actors on pipe
2025-10-09T21:44:40.288906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341213295933373:2432], now have 1 active actors on pipe
2025-10-09T21:44:40.288914Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341212934122356:4504]
2025-10-09T21:44:40.288917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341212934122356:4504]
2025-10-09T21:44:40.288919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341212934122356:4504]
2025-10-09T21:44:40.288935Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:40.288950Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:40.288956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:40.288966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:40.288969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:40.288970Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:40.288972Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:40.289009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341212934122356:4504]
2025-10-09T21:44:40.289018Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:40.289038Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:40.289086Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 350
2025-10-09T21:44:40.289088Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 350
2025-10-09T21:44:40.289223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:40.289236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:40.289270Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5ace50d7-df64eb6-c80e8b68-905613f3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:40.289282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:40.289286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:40.289289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:40.289301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:40.289313Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:40.289332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:40.289576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:40.289594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:40.289616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:40.289730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 351 partition: 0 MaxSeqNo: 25 sessionId: src_id|5ace50d7-df64eb6-c80e8b68-905613f3_0
2025-10-09T21:44:40.290138Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ec4d558-17c96b5-6640f99c-e6945a17_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5ace50d7-df64eb6-c80e8b68-905613f3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:40.292693Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ace50d7-df64eb6-c80e8b68-905613f3_0] Write session: try to update token
2025-10-09T21:44:40.292713Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ace50d7-df64eb6-c80e8b68-905613f3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:40.307914Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:40.307933Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:40.307949Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:40.307951Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:40.307962Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341212934122356:4504]
2025-10-09T21:44:40.307966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341212934122356:4504]
2025-10-09T21:44:40.307971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341212934122356:4504]
2025-10-09T21:44:40.308021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:40.308034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.308037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:40.308169Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:40.308731Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 351 sessionId: src_id|5ace50d7-df64eb6-c80e8b68-905613f3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:40.311117Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:40.311322Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:40.311337Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:40.311343Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:40.323583Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341213295933553:4549]
2025-10-09T21:44:40.323869Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 350, pipe: [1:7559341213295933553:4549]
2025-10-09T21:44:40.486757Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:40.487051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:40.487068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:40.487114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341213295933553:4549], now have 1 active actors on pipe
2025-10-09T21:44:40.487380Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:40.487992Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ace50d7-df64eb6-c80e8b68-905613f3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:40.488006Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ace50d7-df64eb6-c80e8b68-905613f3_0] Write session will restart in 0.009808s
2025-10-09T21:44:40.488030Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ace50d7-df64eb6-c80e8b68-905613f3_0] Write session: Do CDS request
2025-10-09T21:44:40.488035Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ace50d7-df64eb6-c80e8b68-905613f3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:40.487440Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:40.487497Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341213295933553:4549]
2025-10-09T21:44:40.487527Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:40.487529Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:40.487593Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 351 reason: error at write: Disconnected sessionId: src_id|5ace50d7-df64eb6-c80e8b68-905613f3_0
2025-10-09T21:44:40.487776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 351 sessionId: src_id|5ace50d7-df64eb6-c80e8b68-905613f3_0 is DEAD
2025-10-09T21:44:40.498807Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ace50d7-df64eb6-c80e8b68-905613f3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:40.499035Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:40.499066Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 352
2025-10-09T21:44:40.499255Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 352 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:40.499318Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 352 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:40.499332Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=352 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:40.499335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:40.499845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 352 sessionId:  describe result for acl check
2025-10-09T21:44:40.499900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:40.499902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:40.499905Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:40.499911Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341213295933593:4873] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:40.500495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341213295933593:4873] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:40.505165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341213295933593:4873] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:40.511195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341213295933593:4873] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:40.511216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341213295933593:4873] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:40.511220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341213295933593:4873] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:40.511229Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 352 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:40.511649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:40.514455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:40.514679Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:40.514692Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:40.514705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:40.514772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:40.514823Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341212934122534:4507]
2025-10-09T21:44:40.515422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:40.515962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:40.516014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:40.516350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:40.516411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:40.516412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:40.516413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:40.516414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:40.516415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:40.516416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:40.516417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.516418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.516419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.516425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:40.516434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:40.516438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:40.516441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:40.516445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:40.516448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:40.516452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.516456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.516459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.516467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:40.516475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:40.516481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:40.516486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:40.516491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:40.516496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:40.516500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:40.516505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:40.516510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:40.516516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:40.516520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:40.516525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:40.516530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.516536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.516540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.516544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.516549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.516554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.516583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:40.526058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:40.527238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:40.527254Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:40.527257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:40.527263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:40.527271Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 351 [2:7559341212934122534:4507]
2025-10-09T21:44:40.527288Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:40.527310Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:40.527313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:40.527316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:40.527319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:40.527341Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:40.527348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:40.527357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:40.527438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.527446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.527449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.527452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:40.527531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:40.527539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.527554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.527569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.527587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:40.527611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341213295933556:2432], now have 1 active actors on pipe
2025-10-09T21:44:40.527627Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341212934122529:4507]
2025-10-09T21:44:40.527634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341212934122529:4507]
2025-10-09T21:44:40.527638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341212934122529:4507]
2025-10-09T21:44:40.527654Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:40.527664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341213295933725:4873], now have 1 active actors on pipe
2025-10-09T21:44:40.527667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:40.527670Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:40.527673Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:40.527678Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:40.527693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:40.527700Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:40.527765Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 351
2025-10-09T21:44:40.527744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341212934122529:4507]
2025-10-09T21:44:40.527756Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:40.527828Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 351
2025-10-09T21:44:40.527809Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:40.528098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:40.528113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:40.528144Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|39c7d606-ee3a83c1-2608935-e1016fd4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:40.528153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:40.528158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:40.528162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:40.528173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:40.528187Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:40.528204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:40.528642Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 352 partition: 0 MaxSeqNo: 25 sessionId: src_id|39c7d606-ee3a83c1-2608935-e1016fd4_0
2025-10-09T21:44:40.528474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:40.528485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:40.528504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:40.529093Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ace50d7-df64eb6-c80e8b68-905613f3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|39c7d606-ee3a83c1-2608935-e1016fd4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:40.531674Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|39c7d606-ee3a83c1-2608935-e1016fd4_0] Write session: try to update token
2025-10-09T21:44:40.531695Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|39c7d606-ee3a83c1-2608935-e1016fd4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:40.548682Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:40.548702Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:40.548704Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:40.548706Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:40.548716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341212934122529:4507]
2025-10-09T21:44:40.548720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341212934122529:4507]
2025-10-09T21:44:40.548725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341212934122529:4507]
2025-10-09T21:44:40.548775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:40.548783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.548786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:40.548927Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:40.551313Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:40.551329Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:40.551332Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:40.553445Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 352 sessionId: src_id|39c7d606-ee3a83c1-2608935-e1016fd4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:40.556371Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:40.587739Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341213295933739:4549]
2025-10-09T21:44:40.588102Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 351, pipe: [1:7559341213295933739:4549]
2025-10-09T21:44:40.778381Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:40.778985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:40.778997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:40.779043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341213295933739:4549], now have 1 active actors on pipe
2025-10-09T21:44:40.779205Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:40.780125Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:40.780229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 352 reason: status is not ok: tablet will be restarted right now sessionId: src_id|39c7d606-ee3a83c1-2608935-e1016fd4_0
2025-10-09T21:44:40.780595Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39c7d606-ee3a83c1-2608935-e1016fd4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:40.780608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39c7d606-ee3a83c1-2608935-e1016fd4_0] Write session will restart in 0.006928s
2025-10-09T21:44:40.780633Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39c7d606-ee3a83c1-2608935-e1016fd4_0] Write session: Do CDS request
2025-10-09T21:44:40.780651Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39c7d606-ee3a83c1-2608935-e1016fd4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:40.780421Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 352 sessionId: src_id|39c7d606-ee3a83c1-2608935-e1016fd4_0 is DEAD
2025-10-09T21:44:40.780546Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:40.780578Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341213295933739:4549]
2025-10-09T21:44:40.788576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|39c7d606-ee3a83c1-2608935-e1016fd4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:40.788669Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:40.788689Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 353
2025-10-09T21:44:40.788819Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 353 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:40.788847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 353 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:40.788853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=353 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:40.788857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:40.789260Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 353 sessionId:  describe result for acl check
2025-10-09T21:44:40.789290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:40.789292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:40.789294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:40.789300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341213295933779:4884] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:40.790062Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341213295933779:4884] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:40.792690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341213295933779:4884] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:40.796700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341213295933779:4884] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:40.796719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341213295933779:4884] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:40.796723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341213295933779:4884] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:40.796733Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 353 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:40.801617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:40.804246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:40.804486Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:40.804497Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:40.804512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:40.804576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:40.804640Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341212934122746:4521]
2025-10-09T21:44:40.805231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:40.805654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:40.805712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:40.806102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:40.806156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:40.806163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:40.806164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:40.806166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:40.806168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:40.806169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:40.806170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.806171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.806173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.806178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:40.806191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:40.806210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:40.806219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:40.806224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:40.806228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:40.806232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.806240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.806244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.806252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:40.806265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:40.806271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:40.806277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:40.806286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:40.806292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:40.806297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:40.806302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:40.806307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:40.806312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:40.806317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:40.806322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:40.806345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.806351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.806360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.806366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.806379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.806385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.806419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:40.813195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:40.814919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:40.814934Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:40.814936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:40.814941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:40.814951Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 352 [2:7559341212934122746:4521]
2025-10-09T21:44:40.814965Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:40.814984Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:40.814987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:40.814990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:40.814992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:40.815008Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:40.815014Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:40.815021Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:40.815085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.815095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.815097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.815100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:40.815174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:40.815177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.815192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:40.815204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:40.815220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:40.815241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341212934122740:4521]
2025-10-09T21:44:40.815243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341212934122740:4521]
2025-10-09T21:44:40.815245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341212934122740:4521]
2025-10-09T21:44:40.815263Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:40.815277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:40.815292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:40.815295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:40.815346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341212934122740:4521]
2025-10-09T21:44:40.815349Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:40.815406Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:40.815409Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:40.815411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:40.815428Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:40.815563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341213295933892:4884], now have 1 active actors on pipe
2025-10-09T21:44:40.815571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341213295933766:2432], now have 1 active actors on pipe
2025-10-09T21:44:40.815789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:40.815816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:40.815852Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3ee20eac-ea7648c7-b23e1103-a0e3f7f5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:40.815864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:40.815869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:40.815871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:40.815883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:40.815897Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:40.815916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:40.816280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:40.816291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:40.815634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 352
2025-10-09T21:44:40.816325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:40.815681Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 352
2025-10-09T21:44:40.816416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 353 partition: 0 MaxSeqNo: 25 sessionId: src_id|3ee20eac-ea7648c7-b23e1103-a0e3f7f5_0
2025-10-09T21:44:40.816838Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39c7d606-ee3a83c1-2608935-e1016fd4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3ee20eac-ea7648c7-b23e1103-a0e3f7f5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:40.819388Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3ee20eac-ea7648c7-b23e1103-a0e3f7f5_0] Write session: try to update token
2025-10-09T21:44:40.819408Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3ee20eac-ea7648c7-b23e1103-a0e3f7f5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:40.832389Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:40.832406Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:40.832410Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:40.832411Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:40.832422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341212934122740:4521]
2025-10-09T21:44:40.832431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341212934122740:4521]
2025-10-09T21:44:40.832435Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341212934122740:4521]
2025-10-09T21:44:40.832481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:40.832499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:40.832502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:40.832637Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:40.834632Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:40.834645Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:40.834651Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:40.834870Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 353 sessionId: src_id|3ee20eac-ea7648c7-b23e1103-a0e3f7f5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:40.837309Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:40.881102Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341213295933951:4549]
2025-10-09T21:44:40.882128Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 352, pipe: [1:7559341213295933951:4549]
2025-10-09T21:44:40.990572Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:40.990881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:40.990900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:40.990952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341213295933951:4549], now have 1 active actors on pipe
2025-10-09T21:44:40.991608Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:40.992294Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ee20eac-ea7648c7-b23e1103-a0e3f7f5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:40.992305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ee20eac-ea7648c7-b23e1103-a0e3f7f5_0] Write session will restart in 0.008107s
2025-10-09T21:44:40.992342Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ee20eac-ea7648c7-b23e1103-a0e3f7f5_0] Write session: Do CDS request
2025-10-09T21:44:40.992350Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ee20eac-ea7648c7-b23e1103-a0e3f7f5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:40.991835Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:40.991929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 353 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3ee20eac-ea7648c7-b23e1103-a0e3f7f5_0
2025-10-09T21:44:40.992116Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 353 sessionId: src_id|3ee20eac-ea7648c7-b23e1103-a0e3f7f5_0 is DEAD
2025-10-09T21:44:40.992329Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:40.992363Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341213295933951:4549]
2025-10-09T21:44:41.001580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3ee20eac-ea7648c7-b23e1103-a0e3f7f5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:41.001671Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:41.001693Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 354
2025-10-09T21:44:41.001876Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 354 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:41.001917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 354 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:41.001923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=354 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:41.001928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:41.003279Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 354 sessionId:  describe result for acl check
2025-10-09T21:44:41.003340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:41.003341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:41.003343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:41.003350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341217590901281:4889] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:41.003988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341217590901281:4889] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:41.013921Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341217590901281:4889] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:41.018521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341217590901281:4889] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:41.018545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341217590901281:4889] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:41.018550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341217590901281:4889] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:41.018561Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 354 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:41.021087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:41.023786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:41.024055Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:41.024071Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:41.024087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:41.024164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:41.024217Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341217229090218:4523]
2025-10-09T21:44:41.024885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:41.025357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:41.025442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:41.025835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:41.025912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:41.025913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:41.025915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:41.025916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:41.025918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:41.025919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:41.025920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.025922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.025923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.025930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:41.025940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:41.025944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:41.025948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:41.025951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:41.025955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:41.025959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.025964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.025968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.025976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:41.025985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:41.025993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:41.025998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:41.026003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:41.026009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:41.026014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:41.026019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:41.026064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:41.026071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:41.026077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:41.026083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:41.026088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.026093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.026098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.026103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.026108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.026114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.026146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:41.035582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:41.037021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:41.037037Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:41.037039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:41.037045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:41.037053Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 353 [2:7559341217229090218:4523]
2025-10-09T21:44:41.037071Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:41.037093Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:41.037096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.037099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.037102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.037122Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:41.037131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:41.037147Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:41.037212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.037220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.037223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.037226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:41.037306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:41.037314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.037329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.037341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.037365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:41.037392Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341217229090213:4523]
2025-10-09T21:44:41.037400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341217229090213:4523]
2025-10-09T21:44:41.037403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341217229090213:4523]
2025-10-09T21:44:41.037413Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:41.037428Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:41.037432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:41.037432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:41.037434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:41.037448Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:41.037452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:41.037464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341217590901422:4889], now have 1 active actors on pipe
2025-10-09T21:44:41.037469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341213295933965:2432], now have 1 active actors on pipe
2025-10-09T21:44:41.037509Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341217229090213:4523]
2025-10-09T21:44:41.037525Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:41.037574Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:41.038149Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 353
2025-10-09T21:44:41.038187Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 353
2025-10-09T21:44:41.038616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:41.038630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:41.038668Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|755c5ce9-afb27cf0-342e2d06-8a1d68d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:41.038679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.038684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.038688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:41.038701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.038717Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:41.038740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:41.039305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:41.039314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:41.039344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:41.039470Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 354 partition: 0 MaxSeqNo: 25 sessionId: src_id|755c5ce9-afb27cf0-342e2d06-8a1d68d_0
2025-10-09T21:44:41.039983Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ee20eac-ea7648c7-b23e1103-a0e3f7f5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|755c5ce9-afb27cf0-342e2d06-8a1d68d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:41.042808Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|755c5ce9-afb27cf0-342e2d06-8a1d68d_0] Write session: try to update token
2025-10-09T21:44:41.042832Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|755c5ce9-afb27cf0-342e2d06-8a1d68d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:41.056586Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:41.056606Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:41.056609Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:41.056612Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:41.056623Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341217229090213:4523]
2025-10-09T21:44:41.056628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341217229090213:4523]
2025-10-09T21:44:41.056632Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341217229090213:4523]
2025-10-09T21:44:41.056681Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:41.056684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:41.056687Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:41.056693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.056693Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:41.056696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:41.056833Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:41.066135Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 354 sessionId: src_id|755c5ce9-afb27cf0-342e2d06-8a1d68d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:41.071515Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:41.094150Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341217590901438:4549]
2025-10-09T21:44:41.094571Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 353, pipe: [1:7559341217590901438:4549]
2025-10-09T21:44:41.253832Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|755c5ce9-afb27cf0-342e2d06-8a1d68d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:41.253851Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|755c5ce9-afb27cf0-342e2d06-8a1d68d_0] Write session will restart in 0.006644s
2025-10-09T21:44:41.253888Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|755c5ce9-afb27cf0-342e2d06-8a1d68d_0] Write session: Do CDS request
2025-10-09T21:44:41.253894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|755c5ce9-afb27cf0-342e2d06-8a1d68d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:41.252650Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:41.252985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:41.252995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:41.253019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341217590901438:4549], now have 1 active actors on pipe
2025-10-09T21:44:41.253329Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:41.253335Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:41.253364Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:41.253441Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 354 reason: error at write: Disconnected sessionId: src_id|755c5ce9-afb27cf0-342e2d06-8a1d68d_0
2025-10-09T21:44:41.253669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 354 sessionId: src_id|755c5ce9-afb27cf0-342e2d06-8a1d68d_0 is DEAD
2025-10-09T21:44:41.253826Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341217590901438:4549]
2025-10-09T21:44:41.253841Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:41.261645Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|755c5ce9-afb27cf0-342e2d06-8a1d68d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:41.261714Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:41.261731Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 355
2025-10-09T21:44:41.261921Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 355 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:41.261974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 355 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:41.261985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=355 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:41.261989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:41.262474Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 355 sessionId:  describe result for acl check
2025-10-09T21:44:41.262511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:41.262518Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:41.262520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:41.262526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341217590901454:4893] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:41.263132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341217590901454:4893] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:41.266397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341217590901454:4893] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:41.271986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341217590901454:4893] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:41.272009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341217590901454:4893] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:41.272023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341217590901454:4893] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:41.272033Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 355 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:41.278232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:41.281944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:41.282235Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:41.282247Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:41.282260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:41.282318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:41.282627Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341217229090398:4526]
2025-10-09T21:44:41.283618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:41.284095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:41.284168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:41.284513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:41.284593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:41.284600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:41.284602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:41.284603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:41.284605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:41.284606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:41.284607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.284609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.284610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.284617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:41.284628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:41.284632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:41.284637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:41.284641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:41.284645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:41.284649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.284654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.284658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.284667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:41.284676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:41.284683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:41.284689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:41.284693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:41.284700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:41.284704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:41.284716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:41.284722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:41.284733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:41.284743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:41.284749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:41.284757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.284768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.284774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.284785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.284790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.284795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.284827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:41.293187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:41.294278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:41.294297Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:41.294300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:41.294305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:41.294313Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 354 [2:7559341217229090398:4526]
2025-10-09T21:44:41.294345Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:41.294368Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:41.294372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.294376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.294379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.294399Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:41.294410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:41.294417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:41.294486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.294495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.294497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.294500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:41.294579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:41.294590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.294606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.294617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.294633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:41.294656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341217229090393:4526]
2025-10-09T21:44:41.294659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341217229090393:4526]
2025-10-09T21:44:41.294662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341217229090393:4526]
2025-10-09T21:44:41.294672Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:41.294681Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:41.294684Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:41.294686Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:41.294692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341217590901441:2432], now have 1 active actors on pipe
2025-10-09T21:44:41.294699Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:41.294714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:41.294718Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:41.294725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341217590901561:4893], now have 1 active actors on pipe
2025-10-09T21:44:41.294764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341217229090393:4526]
2025-10-09T21:44:41.294774Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:41.294780Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 354
2025-10-09T21:44:41.294836Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 354
2025-10-09T21:44:41.294813Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:41.295156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:41.295168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:41.295214Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|73557dd7-aacf9456-cafca571-bf64fac2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:41.295224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.295229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.295231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:41.295243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.295255Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:41.295273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:41.295709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:41.295721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:41.295741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:41.298484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 355 partition: 0 MaxSeqNo: 25 sessionId: src_id|73557dd7-aacf9456-cafca571-bf64fac2_0
2025-10-09T21:44:41.298902Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|755c5ce9-afb27cf0-342e2d06-8a1d68d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|73557dd7-aacf9456-cafca571-bf64fac2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:41.301584Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73557dd7-aacf9456-cafca571-bf64fac2_0] Write session: try to update token
2025-10-09T21:44:41.301620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73557dd7-aacf9456-cafca571-bf64fac2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:41.313500Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:41.313521Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:41.313523Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:41.313526Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:41.313535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341217229090393:4526]
2025-10-09T21:44:41.313540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341217229090393:4526]
2025-10-09T21:44:41.313544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341217229090393:4526]
2025-10-09T21:44:41.313596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:41.313611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.313613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:41.313758Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:41.317341Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:41.317360Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:41.317365Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:41.320157Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 355 sessionId: src_id|73557dd7-aacf9456-cafca571-bf64fac2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:41.322866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:41.354253Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341217590901625:4549]
2025-10-09T21:44:41.354760Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 354, pipe: [1:7559341217590901625:4549]
2025-10-09T21:44:41.474561Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:41.475030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:41.475047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:41.475094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341217590901625:4549], now have 1 active actors on pipe
2025-10-09T21:44:41.475314Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:41.475386Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341217590901625:4549]
2025-10-09T21:44:41.475388Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:41.475855Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73557dd7-aacf9456-cafca571-bf64fac2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:41.475866Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73557dd7-aacf9456-cafca571-bf64fac2_0] Write session will restart in 0.005606s
2025-10-09T21:44:41.475889Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73557dd7-aacf9456-cafca571-bf64fac2_0] Write session: Do CDS request
2025-10-09T21:44:41.475896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73557dd7-aacf9456-cafca571-bf64fac2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:41.475422Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:41.475519Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 355 reason: status is not ok: tablet will be restarted right now sessionId: src_id|73557dd7-aacf9456-cafca571-bf64fac2_0
2025-10-09T21:44:41.475692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 355 sessionId: src_id|73557dd7-aacf9456-cafca571-bf64fac2_0 is DEAD
2025-10-09T21:44:41.482723Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73557dd7-aacf9456-cafca571-bf64fac2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:41.482810Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:41.482827Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 356
2025-10-09T21:44:41.482970Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 356 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:41.483012Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 356 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:41.483023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=356 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:41.483026Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:41.483558Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 356 sessionId:  describe result for acl check
2025-10-09T21:44:41.483611Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:41.483619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:41.483620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:41.483629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341217590901642:4897] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:41.484147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341217590901642:4897] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:41.488745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341217590901642:4897] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:41.494628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341217590901642:4897] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:41.494646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341217590901642:4897] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:41.494650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341217590901642:4897] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:41.494660Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 356 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:41.498185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:41.500278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:41.500648Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:41.500661Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:41.500678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:41.500765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:41.500817Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341217229090576:4529]
2025-10-09T21:44:41.501424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:41.501811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:41.501869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:41.502228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:41.502288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:41.502296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:41.502297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:41.502298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:41.502300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:41.502301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:41.502302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.502303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.502305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.502310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:41.502322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:41.502337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:41.502342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:41.502345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:41.502350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:41.502354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.502357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.502361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.502368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:41.502377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:41.502383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:41.502389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:41.502394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:41.502400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:41.502405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:41.502410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:41.502416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:41.502421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:41.502425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:41.502430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:41.502435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.502440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.502447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.502452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.502458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.502463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.502494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:41.508970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:41.510433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:41.510449Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:41.510451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:41.510455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:41.510462Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 355 [2:7559341217229090576:4529]
2025-10-09T21:44:41.510478Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:41.510497Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:41.510504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.510507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.510510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.510528Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:41.510539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:41.510546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:41.510608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.510617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.510619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.510621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:41.510691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:41.510698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.510709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.510720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.510768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:41.510881Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 355
2025-10-09T21:44:41.510913Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 355
2025-10-09T21:44:41.510790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341217590901790:4897], now have 1 active actors on pipe
2025-10-09T21:44:41.510798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341217590901628:2432], now have 1 active actors on pipe
2025-10-09T21:44:41.510806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341217229090571:4529]
2025-10-09T21:44:41.510808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341217229090571:4529]
2025-10-09T21:44:41.510810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341217229090571:4529]
2025-10-09T21:44:41.510824Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:41.510838Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:41.510838Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:41.510841Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:41.510843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:41.510851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:41.510854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:41.510892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341217229090571:4529]
2025-10-09T21:44:41.510901Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:41.510909Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:41.510941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:41.510946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:41.510977Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|93ecdd1f-f56f9e8a-a48c3e6-a486412e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:41.510985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.510989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.510991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:41.511000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.511010Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:41.511029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:41.511332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:41.511356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:41.511383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:41.511545Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 356 partition: 0 MaxSeqNo: 25 sessionId: src_id|93ecdd1f-f56f9e8a-a48c3e6-a486412e_0
2025-10-09T21:44:41.511979Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73557dd7-aacf9456-cafca571-bf64fac2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|93ecdd1f-f56f9e8a-a48c3e6-a486412e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:41.514524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|93ecdd1f-f56f9e8a-a48c3e6-a486412e_0] Write session: try to update token
2025-10-09T21:44:41.514546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|93ecdd1f-f56f9e8a-a48c3e6-a486412e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:41.532709Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 356 sessionId: src_id|93ecdd1f-f56f9e8a-a48c3e6-a486412e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:41.531597Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:41.531616Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:41.531618Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:41.531621Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:41.531632Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341217229090571:4529]
2025-10-09T21:44:41.531637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341217229090571:4529]
2025-10-09T21:44:41.531641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341217229090571:4529]
2025-10-09T21:44:41.531689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:41.531696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.531699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:41.531697Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:41.531705Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:41.531710Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:41.531842Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:41.535148Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:41.576898Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341217590901815:4549]
2025-10-09T21:44:41.577198Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 355, pipe: [1:7559341217590901815:4549]
2025-10-09T21:44:41.701939Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:41.702528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:41.702546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:41.702593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341217590901815:4549], now have 1 active actors on pipe
2025-10-09T21:44:41.706604Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:41.708994Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:41.709052Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:41.709125Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 356 reason: status is not ok: tablet will be restarted right now sessionId: src_id|93ecdd1f-f56f9e8a-a48c3e6-a486412e_0
2025-10-09T21:44:41.709303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 356 sessionId: src_id|93ecdd1f-f56f9e8a-a48c3e6-a486412e_0 is DEAD
2025-10-09T21:44:41.709444Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93ecdd1f-f56f9e8a-a48c3e6-a486412e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:41.709455Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93ecdd1f-f56f9e8a-a48c3e6-a486412e_0] Write session will restart in 0.008775s
2025-10-09T21:44:41.709479Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93ecdd1f-f56f9e8a-a48c3e6-a486412e_0] Write session: Do CDS request
2025-10-09T21:44:41.709485Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93ecdd1f-f56f9e8a-a48c3e6-a486412e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:41.709472Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341217590901815:4549]
2025-10-09T21:44:41.722472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|93ecdd1f-f56f9e8a-a48c3e6-a486412e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:41.722959Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:41.722997Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 357
2025-10-09T21:44:41.726153Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 357 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:41.726207Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 357 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:41.726211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=357 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:41.726226Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:41.727069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 357 sessionId:  describe result for acl check
2025-10-09T21:44:41.727121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:41.727124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:41.727125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:41.727131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341217590901922:4910] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:41.728015Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341217590901922:4910] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:41.734864Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341217590901922:4910] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:41.736694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:41.742259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341217590901922:4910] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:41.742283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341217590901922:4910] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:41.742287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341217590901922:4910] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:41.742298Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 357 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:41.742705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:41.742967Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:41.742981Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:41.742996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:41.743068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:41.743116Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341217229090792:4537]
2025-10-09T21:44:41.743720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:41.745229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:41.745307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:41.746019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:41.746133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:41.746135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:41.746136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:41.746138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:41.746139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:41.746140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:41.746141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.746143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.746144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.746151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:41.746163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:41.746167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:41.746171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:41.746175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:41.746179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:41.746182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.746187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.746191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.746198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:41.746206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:41.746213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:41.746218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:41.746225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:41.746230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:41.746234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:41.746239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:41.746244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:41.746250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:41.746253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:41.746258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:41.746262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.746268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.746271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.746275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.746279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.746284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.746311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:41.755190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:41.756715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:41.756729Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:41.756731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:41.756735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:41.756743Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 356 [2:7559341217229090792:4537]
2025-10-09T21:44:41.756758Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:41.756780Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:41.756784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.756787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.756791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.756811Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:41.756823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:41.756831Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:41.756899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.756906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.756908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.756911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:41.756987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:41.756995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.757008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:41.757018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:41.757046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:41.757072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341217590901834:2432], now have 1 active actors on pipe
2025-10-09T21:44:41.757080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341217590902015:4910], now have 1 active actors on pipe
2025-10-09T21:44:41.757087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341217229090777:4537]
2025-10-09T21:44:41.757091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341217229090777:4537]
2025-10-09T21:44:41.757093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341217229090777:4537]
2025-10-09T21:44:41.757107Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:41.757128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:41.757129Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:41.757132Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:41.757134Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:41.757142Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:41.757146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:41.757192Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 356
2025-10-09T21:44:41.757186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341217229090777:4537]
2025-10-09T21:44:41.757201Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:41.757208Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:41.757256Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 356
2025-10-09T21:44:41.757430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:41.757446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:41.757478Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ba32e4c9-f8f626dd-6fba0bf0-8ee9dc72_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:41.757506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.757516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.757519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:41.757531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.757547Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:41.757570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:41.757798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:41.757824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:41.757849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:41.757979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 357 partition: 0 MaxSeqNo: 25 sessionId: src_id|ba32e4c9-f8f626dd-6fba0bf0-8ee9dc72_0
2025-10-09T21:44:41.758465Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93ecdd1f-f56f9e8a-a48c3e6-a486412e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ba32e4c9-f8f626dd-6fba0bf0-8ee9dc72_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:41.761097Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ba32e4c9-f8f626dd-6fba0bf0-8ee9dc72_0] Write session: try to update token
2025-10-09T21:44:41.761119Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ba32e4c9-f8f626dd-6fba0bf0-8ee9dc72_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:41.775674Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:41.775690Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:41.775699Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341217229090777:4537]
2025-10-09T21:44:41.775750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:41.775756Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:41.775928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:41.776084Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:41.776427Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:41.776578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:41.776752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:41.776796Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:41.777055Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 357 sessionId: src_id|ba32e4c9-f8f626dd-6fba0bf0-8ee9dc72_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:41.779424Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:41.779637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:41.779655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:41.779710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:41.779831Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:41.798769Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:41.798788Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:41.798791Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:41.798793Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:41.798805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341217229090777:4537]
2025-10-09T21:44:41.798809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341217229090777:4537]
2025-10-09T21:44:41.798812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341217229090777:4537]
2025-10-09T21:44:41.798859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:41.798866Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:41.798871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:41.798873Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:41.798874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:41.798877Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:41.799003Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:41.809425Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341217590902027:4549]
2025-10-09T21:44:41.809949Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 356, pipe: [1:7559341217590902027:4549]
2025-10-09T21:44:41.966600Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:41.975109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:41.975142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:41.975330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:41.975390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:41.975452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:41.975513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:41.975554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:41.975559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:41.975629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:41.975696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:41.975755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:41.975821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:41.975860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:41.975863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:41.975939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:41.975996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:41.976058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:41.976122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:41.976162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:41.976165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:41.976232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:41.976287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:41.976345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:41.976404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:41.976444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:41.976446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:41.976515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:41.976572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:41.976633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:41.976690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:41.976733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:41.976735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:41.976802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:41.976857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:41.976919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:41.976985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:41.977025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:41.977029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:41.977091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:41.977148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:41.977207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:41.977266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:41.977305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:41.977309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:41.977373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:41.977427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:41.977491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:41.977550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:41.977586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:41.977590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:41.977653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:41.977728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:41.977787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:41.977845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:41.977894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:41.977897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:41.977967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:41.978021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:41.978877Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:41.978977Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341217590902027:4549]
2025-10-09T21:44:41.979022Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:41.979031Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:41.979063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 357 reason: error at write: Disconnected sessionId: src_id|ba32e4c9-f8f626dd-6fba0bf0-8ee9dc72_0
2025-10-09T21:44:41.979278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 357 sessionId: src_id|ba32e4c9-f8f626dd-6fba0bf0-8ee9dc72_0 is DEAD
2025-10-09T21:44:41.979429Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba32e4c9-f8f626dd-6fba0bf0-8ee9dc72_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:41.979440Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba32e4c9-f8f626dd-6fba0bf0-8ee9dc72_0] Write session will restart in 0.006395s
2025-10-09T21:44:41.979464Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba32e4c9-f8f626dd-6fba0bf0-8ee9dc72_0] Write session: Do CDS request
2025-10-09T21:44:41.979470Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba32e4c9-f8f626dd-6fba0bf0-8ee9dc72_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:41.978364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:41.978437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:41.978479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:41.978482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:41.978553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341217590902027:4549], now have 1 active actors on pipe
2025-10-09T21:44:41.979697Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:41.987591Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ba32e4c9-f8f626dd-6fba0bf0-8ee9dc72_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:41.987896Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:41.987935Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 358
2025-10-09T21:44:41.988098Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 358 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:41.988139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 358 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:41.988150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=358 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:41.988153Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:41.988644Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 358 sessionId:  describe result for acl check
2025-10-09T21:44:41.988688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:41.988695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:41.988699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:41.988705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341217590902055:4916] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:41.989412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341217590902055:4916] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:41.992918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341217590902055:4916] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:41.996335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341217590902055:4916] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:41.996353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341217590902055:4916] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:41.996357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341217590902055:4916] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:41.996366Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 358 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:41.998206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:42.000823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:42.001089Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:42.001106Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:42.001142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:42.001218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:42.001267Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341221524058268:4545]
2025-10-09T21:44:42.001885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:42.002347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:42.002419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:42.002846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:42.002924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:42.002933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:42.002934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:42.002935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:42.002937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:42.002938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:42.002940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.002941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.002942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.002948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:42.002959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:42.002963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:42.002967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:42.002971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:42.002975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:42.002979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.002983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.002987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.002996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:42.003005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:42.003012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:42.003018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:42.003023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:42.003029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:42.003035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:42.003041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:42.003046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:42.003051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:42.003055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:42.003061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:42.003065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.003070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.003074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.003078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.003082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.003090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.003138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:42.010471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:42.011792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:42.011807Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:42.011810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:42.011815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:42.011824Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 357 [2:7559341221524058268:4545]
2025-10-09T21:44:42.011841Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:42.011864Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:42.011872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.011875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.011877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.011897Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:42.011912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:42.011919Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:42.011988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.011995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.011997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.012000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:42.012076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:42.012084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.012098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.012114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.012130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:42.012153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341217229090966:4545]
2025-10-09T21:44:42.012161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341217229090966:4545]
2025-10-09T21:44:42.012164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341217229090966:4545]
2025-10-09T21:44:42.012173Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:42.012184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:42.012190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341217590902041:2432], now have 1 active actors on pipe
2025-10-09T21:44:42.012193Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:42.012194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341217590902210:4916], now have 1 active actors on pipe
2025-10-09T21:44:42.012196Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:42.012203Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:42.012219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:42.012223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:42.012264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341217229090966:4545]
2025-10-09T21:44:42.012279Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:42.012351Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:42.012377Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 357
2025-10-09T21:44:42.012429Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 357
2025-10-09T21:44:42.012638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:42.012651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:42.012685Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d3282108-aa972ed2-4a7f2f11-234098d8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:42.012695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.012700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.012702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:42.012715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.012728Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:42.012747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:42.012956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:42.012966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:42.013003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:42.013131Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 358 partition: 0 MaxSeqNo: 25 sessionId: src_id|d3282108-aa972ed2-4a7f2f11-234098d8_0
2025-10-09T21:44:42.013570Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba32e4c9-f8f626dd-6fba0bf0-8ee9dc72_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d3282108-aa972ed2-4a7f2f11-234098d8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:42.016091Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d3282108-aa972ed2-4a7f2f11-234098d8_0] Write session: try to update token
2025-10-09T21:44:42.016114Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d3282108-aa972ed2-4a7f2f11-234098d8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:42.031924Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 358 sessionId: src_id|d3282108-aa972ed2-4a7f2f11-234098d8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:42.031932Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:42.031948Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:42.031951Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:42.031954Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:42.031964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341217229090966:4545]
2025-10-09T21:44:42.031968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341217229090966:4545]
2025-10-09T21:44:42.031972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341217229090966:4545]
2025-10-09T21:44:42.032033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:42.032041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.032043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:42.032130Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:42.032137Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:42.032141Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:42.032191Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:42.034899Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:42.079292Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341221885869521:4549]
2025-10-09T21:44:42.079787Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 357, pipe: [1:7559341221885869521:4549]
2025-10-09T21:44:42.208813Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:42.209442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:42.209453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:42.209507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341221885869521:4549], now have 1 active actors on pipe
2025-10-09T21:44:42.209869Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341221885869521:4549]
2025-10-09T21:44:42.209903Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:42.209833Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:42.210025Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:42.210130Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 358 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d3282108-aa972ed2-4a7f2f11-234098d8_0
2025-10-09T21:44:42.210525Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3282108-aa972ed2-4a7f2f11-234098d8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:42.210536Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3282108-aa972ed2-4a7f2f11-234098d8_0] Write session will restart in 0.006975s
2025-10-09T21:44:42.210367Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 358 sessionId: src_id|d3282108-aa972ed2-4a7f2f11-234098d8_0 is DEAD
2025-10-09T21:44:42.210578Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3282108-aa972ed2-4a7f2f11-234098d8_0] Write session: Do CDS request
2025-10-09T21:44:42.210585Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3282108-aa972ed2-4a7f2f11-234098d8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:42.218594Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d3282108-aa972ed2-4a7f2f11-234098d8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:42.218798Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:42.218817Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 359
2025-10-09T21:44:42.218997Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 359 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:42.219062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 359 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:42.219069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=359 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:42.219073Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:42.219516Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 359 sessionId:  describe result for acl check
2025-10-09T21:44:42.219571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:42.219573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:42.219578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:42.219583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341221885869536:4920] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:42.220161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341221885869536:4920] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:42.228141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341221885869536:4920] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:42.231817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341221885869536:4920] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:42.231837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341221885869536:4920] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:42.231841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341221885869536:4920] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:42.231852Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 359 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:42.232916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:42.235779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:42.236041Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:42.236054Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:42.236067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:42.236127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:42.236174Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341221524058449:4548]
2025-10-09T21:44:42.236784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:42.237188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:42.237252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:42.237599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:42.237665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:42.237672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:42.237673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:42.237674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:42.237675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:42.237676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:42.237678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.237680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.237681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.237686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:42.237698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:42.237702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:42.237706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:42.237710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:42.237714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:42.237718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.237722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.237726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.237735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:42.237744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:42.237750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:42.237755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:42.237761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:42.237766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:42.237771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:42.237776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:42.237781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:42.237787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:42.237790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:42.237798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:42.237802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.237807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.237813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.237819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.237824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.237830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.237861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:42.246942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:42.248118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:42.248132Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:42.248136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:42.248141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:42.248150Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 358 [2:7559341221524058449:4548]
2025-10-09T21:44:42.248172Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:42.248195Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:42.248198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.248202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.248205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.248225Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:42.248231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:42.248238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:42.248315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.248323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.248326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.248330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:42.248420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:42.248429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.248442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.248456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.248478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:42.248506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341221885869698:4920], now have 1 active actors on pipe
2025-10-09T21:44:42.248514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341221885869524:2432], now have 1 active actors on pipe
2025-10-09T21:44:42.248522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341221524058444:4548]
2025-10-09T21:44:42.248530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341221524058444:4548]
2025-10-09T21:44:42.248533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341221524058444:4548]
2025-10-09T21:44:42.248547Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:42.248563Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:42.248565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:42.248574Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:42.248576Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:42.248580Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:42.248584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:42.248626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341221524058444:4548]
2025-10-09T21:44:42.248623Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 358
2025-10-09T21:44:42.248634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 358
2025-10-09T21:44:42.248636Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:42.248687Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:42.248838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:42.248866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:42.248926Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6966ed09-5b785749-f6f8ce0b-3be86ede_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:42.248941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.248947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.248950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:42.248961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.248974Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:42.249013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:42.249191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:42.249201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:42.249231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:42.249387Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 359 partition: 0 MaxSeqNo: 25 sessionId: src_id|6966ed09-5b785749-f6f8ce0b-3be86ede_0
2025-10-09T21:44:42.249800Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3282108-aa972ed2-4a7f2f11-234098d8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6966ed09-5b785749-f6f8ce0b-3be86ede_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:42.252380Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6966ed09-5b785749-f6f8ce0b-3be86ede_0] Write session: try to update token
2025-10-09T21:44:42.252399Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6966ed09-5b785749-f6f8ce0b-3be86ede_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:42.268006Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 359 sessionId: src_id|6966ed09-5b785749-f6f8ce0b-3be86ede_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:42.270779Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:42.270797Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:42.270799Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:42.270802Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:42.270811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341221524058444:4548]
2025-10-09T21:44:42.270817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341221524058444:4548]
2025-10-09T21:44:42.270820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341221524058444:4548]
2025-10-09T21:44:42.270868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:42.270891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.270892Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:42.270894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:42.270898Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:42.270902Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:42.271031Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:42.272602Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:42.310118Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341221885869711:4549]
2025-10-09T21:44:42.310731Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 358, pipe: [1:7559341221885869711:4549]
2025-10-09T21:44:42.438674Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:42.439403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:42.439415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:42.439463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341221885869711:4549], now have 1 active actors on pipe
2025-10-09T21:44:42.439740Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:42.439755Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:42.439834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 359 reason: error at write: Disconnected sessionId: src_id|6966ed09-5b785749-f6f8ce0b-3be86ede_0
2025-10-09T21:44:42.439719Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:42.440053Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 359 sessionId: src_id|6966ed09-5b785749-f6f8ce0b-3be86ede_0 is DEAD
2025-10-09T21:44:42.440063Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:42.440232Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6966ed09-5b785749-f6f8ce0b-3be86ede_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:42.440243Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6966ed09-5b785749-f6f8ce0b-3be86ede_0] Write session will restart in 0.008323s
2025-10-09T21:44:42.440107Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341221885869711:4549]
2025-10-09T21:44:42.440277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6966ed09-5b785749-f6f8ce0b-3be86ede_0] Write session: Do CDS request
2025-10-09T21:44:42.440283Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6966ed09-5b785749-f6f8ce0b-3be86ede_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:42.449583Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6966ed09-5b785749-f6f8ce0b-3be86ede_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:42.449685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:42.449717Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 360
2025-10-09T21:44:42.449832Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 360 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:42.449860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 360 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:42.449866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=360 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:42.449870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:42.450419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 360 sessionId:  describe result for acl check
2025-10-09T21:44:42.450459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:42.450461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:42.450463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:42.450469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341221885869728:4924] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:42.451036Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341221885869728:4924] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:42.456952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341221885869728:4924] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:42.461587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341221885869728:4924] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:42.461607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341221885869728:4924] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:42.461612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341221885869728:4924] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:42.461633Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 360 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:42.466565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:42.469039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:42.469274Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:42.469285Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:42.469298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:42.469358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:42.469404Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341221524058632:4551]
2025-10-09T21:44:42.470051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:42.470444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:42.470504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:42.470915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:42.470979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:42.470980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:42.470982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:42.470984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:42.470985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:42.470987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:42.470988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.470989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.470991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.470996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:42.471008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:42.471012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:42.471017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:42.471020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:42.471024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:42.471028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.471032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.471035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.471044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:42.471052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:42.471058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:42.471065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:42.471070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:42.471074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:42.471077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:42.471082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:42.471088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:42.471093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:42.471097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:42.471103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:42.471107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.471112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.471116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.471120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.471124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.471129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.471159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:42.479168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:42.480629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:42.480647Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:42.480649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:42.480654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:42.480661Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 359 [2:7559341221524058632:4551]
2025-10-09T21:44:42.480677Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:42.480697Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:42.480701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.480704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.480707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.480726Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:42.480740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:42.480748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:42.480814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.480821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.480823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.480826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:42.480904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:42.480913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.480927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.480937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.480957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:42.480984Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341221524058627:4551]
2025-10-09T21:44:42.480986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341221524058627:4551]
2025-10-09T21:44:42.480988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341221524058627:4551]
2025-10-09T21:44:42.480997Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:42.481014Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:42.481034Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:42.481037Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:42.481081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341221524058627:4551]
2025-10-09T21:44:42.481081Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:42.481084Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:42.481086Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:42.481088Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:42.481092Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:42.481133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341221885869714:2432], now have 1 active actors on pipe
2025-10-09T21:44:42.481244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341221885869882:4924], now have 1 active actors on pipe
2025-10-09T21:44:42.481370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:42.481380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:42.481276Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 359
2025-10-09T21:44:42.481299Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 359
2025-10-09T21:44:42.481420Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c26aa670-4612fe72-2c043607-f6cae3f8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:42.481435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.481444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.481447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:42.481458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.481479Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:42.481499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:42.482096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:42.482116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:42.482137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:42.482249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 360 partition: 0 MaxSeqNo: 25 sessionId: src_id|c26aa670-4612fe72-2c043607-f6cae3f8_0
2025-10-09T21:44:42.482667Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6966ed09-5b785749-f6f8ce0b-3be86ede_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c26aa670-4612fe72-2c043607-f6cae3f8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:42.485354Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c26aa670-4612fe72-2c043607-f6cae3f8_0] Write session: try to update token
2025-10-09T21:44:42.485376Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c26aa670-4612fe72-2c043607-f6cae3f8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:42.503369Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:42.503391Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:42.503394Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:42.503396Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:42.503406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341221524058627:4551]
2025-10-09T21:44:42.503410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341221524058627:4551]
2025-10-09T21:44:42.503413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341221524058627:4551]
2025-10-09T21:44:42.503462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:42.503470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.503474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:42.503616Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:42.505158Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 360 sessionId: src_id|c26aa670-4612fe72-2c043607-f6cae3f8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:42.506248Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:42.506263Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:42.506271Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:42.507631Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:42.540036Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341221885869905:4549]
2025-10-09T21:44:42.542870Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 359, pipe: [1:7559341221885869905:4549]
2025-10-09T21:44:42.724767Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:42.725226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:42.725247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:42.725295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341221885869905:4549], now have 1 active actors on pipe
2025-10-09T21:44:42.725559Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:42.725821Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341221885869905:4549]
2025-10-09T21:44:42.725866Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:42.725902Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:42.725905Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:42.725978Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 360 reason: error at write: Disconnected sessionId: src_id|c26aa670-4612fe72-2c043607-f6cae3f8_0
2025-10-09T21:44:42.726429Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c26aa670-4612fe72-2c043607-f6cae3f8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:42.726441Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c26aa670-4612fe72-2c043607-f6cae3f8_0] Write session will restart in 0.006450s
2025-10-09T21:44:42.726466Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c26aa670-4612fe72-2c043607-f6cae3f8_0] Write session: Do CDS request
2025-10-09T21:44:42.726477Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c26aa670-4612fe72-2c043607-f6cae3f8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:42.726344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 360 sessionId: src_id|c26aa670-4612fe72-2c043607-f6cae3f8_0 is DEAD
2025-10-09T21:44:42.734746Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c26aa670-4612fe72-2c043607-f6cae3f8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:42.735115Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:42.735144Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 361
2025-10-09T21:44:42.735338Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 361 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:42.735387Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 361 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:42.735393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=361 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:42.735398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:42.735914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 361 sessionId:  describe result for acl check
2025-10-09T21:44:42.735955Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:42.735958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:42.735959Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:42.735965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341221885869921:4929] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:42.736525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341221885869921:4929] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:42.740444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341221885869921:4929] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:42.748965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341221885869921:4929] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:42.748986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341221885869921:4929] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:42.748990Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341221885869921:4929] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:42.749000Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 361 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:42.750478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:42.753617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:42.753870Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:42.753874Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:42.753887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:42.753975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:42.754024Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341221524058828:4559]
2025-10-09T21:44:42.754682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:42.755141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:42.755208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:42.756406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:42.756500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:42.756510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:42.756511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:42.756512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:42.756513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:42.756515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:42.756516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.756518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.756519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.756525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:42.756536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:42.756540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:42.756544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:42.756551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:42.756554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:42.756558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.756561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.756565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.756572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:42.756580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:42.756586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:42.756591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:42.756595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:42.756600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:42.756604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:42.756609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:42.756615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:42.756620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:42.756624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:42.756629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:42.756634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.756639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.756644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.756649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.756655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.756659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.756689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:42.764265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:42.767202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:42.767224Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:42.767227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:42.767231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:42.767238Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 360 [2:7559341221524058828:4559]
2025-10-09T21:44:42.767255Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:42.767277Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:42.767447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.767457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.767460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.767478Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:42.767494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:42.767501Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:42.767563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.767571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.767574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.767577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:42.767662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:42.767673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.767689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:42.767702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:42.767719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:42.767750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341221885870095:4929], now have 1 active actors on pipe
2025-10-09T21:44:42.767763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341221885869908:2432], now have 1 active actors on pipe
2025-10-09T21:44:42.767771Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341221524058823:4559]
2025-10-09T21:44:42.767773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341221524058823:4559]
2025-10-09T21:44:42.767776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341221524058823:4559]
2025-10-09T21:44:42.767789Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:42.767806Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:42.767814Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:42.767823Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:42.767823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:42.767825Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:42.767827Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:42.767829Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 360
2025-10-09T21:44:42.767882Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 360
2025-10-09T21:44:42.767869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341221524058823:4559]
2025-10-09T21:44:42.767885Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:42.768025Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:42.768317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:42.768330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:42.768361Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|adba0da9-1f7c9ac-b7ba2a51-a707a0c5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:42.768371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.768375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.768378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:42.768389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.768400Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:42.768420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:42.768685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:42.768696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:42.768715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:42.768831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 361 partition: 0 MaxSeqNo: 25 sessionId: src_id|adba0da9-1f7c9ac-b7ba2a51-a707a0c5_0
2025-10-09T21:44:42.769237Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c26aa670-4612fe72-2c043607-f6cae3f8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|adba0da9-1f7c9ac-b7ba2a51-a707a0c5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:42.777780Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|adba0da9-1f7c9ac-b7ba2a51-a707a0c5_0] Write session: try to update token
2025-10-09T21:44:42.777806Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|adba0da9-1f7c9ac-b7ba2a51-a707a0c5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:42.785959Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:42.785981Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:42.785983Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:42.785985Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:42.785994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341221524058823:4559]
2025-10-09T21:44:42.785999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341221524058823:4559]
2025-10-09T21:44:42.786002Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341221524058823:4559]
2025-10-09T21:44:42.786077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:42.786085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:42.786085Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:42.786088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:42.786092Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:42.786096Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:42.786218Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:42.805020Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 361 sessionId: src_id|adba0da9-1f7c9ac-b7ba2a51-a707a0c5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:42.807601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:42.825712Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341221885870119:4549]
2025-10-09T21:44:42.826109Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 360, pipe: [1:7559341221885870119:4549]
2025-10-09T21:44:42.963643Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:42.963954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:42.963972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:42.963996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341221885870119:4549], now have 1 active actors on pipe
2025-10-09T21:44:42.964754Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:42.964770Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:42.964770Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341221885870119:4549]
2025-10-09T21:44:42.964791Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:42.964875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 361 reason: error at write: Disconnected sessionId: src_id|adba0da9-1f7c9ac-b7ba2a51-a707a0c5_0
2025-10-09T21:44:42.965084Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 361 sessionId: src_id|adba0da9-1f7c9ac-b7ba2a51-a707a0c5_0 is DEAD
2025-10-09T21:44:42.965378Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adba0da9-1f7c9ac-b7ba2a51-a707a0c5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:42.965390Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adba0da9-1f7c9ac-b7ba2a51-a707a0c5_0] Write session will restart in 0.006034s
2025-10-09T21:44:42.965429Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adba0da9-1f7c9ac-b7ba2a51-a707a0c5_0] Write session: Do CDS request
2025-10-09T21:44:42.965439Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adba0da9-1f7c9ac-b7ba2a51-a707a0c5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:42.964828Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:42.972615Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|adba0da9-1f7c9ac-b7ba2a51-a707a0c5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:42.972728Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:42.972749Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 362
2025-10-09T21:44:42.972921Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 362 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:42.972975Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 362 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:42.972992Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=362 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:42.972998Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:42.973509Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 362 sessionId:  describe result for acl check
2025-10-09T21:44:42.973551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:42.973555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:42.973559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:42.973565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341221885870138:4941] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:42.974404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341221885870138:4941] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:42.977897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341221885870138:4941] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:42.982524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341221885870138:4941] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:42.982545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341221885870138:4941] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:42.982549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341221885870138:4941] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:42.982559Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 362 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:43.003561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:43.008903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:43.009228Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:43.009255Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:43.009275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:43.009353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:43.009417Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341225819026333:4567]
2025-10-09T21:44:43.010153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:43.010622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:43.010689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:43.011018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:43.011070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:43.011071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:43.011073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:43.011074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:43.011076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:43.011077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:43.011078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.011079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.011080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.011086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:43.011097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:43.011101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:43.011105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:43.011108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:43.011113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:43.011116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.011120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.011124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.011131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:43.011139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:43.011146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:43.011151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:43.011156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:43.011161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:43.011165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:43.011186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:43.011193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:43.011197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:43.011202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:43.011208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:43.011213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.011218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.011225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.011230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.011235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.011240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.011270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:43.019010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:43.020195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:43.020215Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:43.020219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:43.020225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:43.020237Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 361 [2:7559341225819026333:4567]
2025-10-09T21:44:43.020252Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:43.020274Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:43.020278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.020281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.020284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.020673Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 361
2025-10-09T21:44:43.020695Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 361
2025-10-09T21:44:43.020304Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:43.020319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:43.020327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:43.020391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.020393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.020395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.020398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:43.020484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:43.020487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.020499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.020512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.020529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:43.020552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341225819026326:4567]
2025-10-09T21:44:43.020555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341225819026326:4567]
2025-10-09T21:44:43.020557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341225819026326:4567]
2025-10-09T21:44:43.020566Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:43.020579Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:43.020583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341221885870130:2432], now have 1 active actors on pipe
2025-10-09T21:44:43.020583Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:43.020586Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:43.020587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341221885870196:4941], now have 1 active actors on pipe
2025-10-09T21:44:43.020596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:43.020610Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:43.020613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:43.020659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341225819026326:4567]
2025-10-09T21:44:43.020669Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:43.020754Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:43.021013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:43.021028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:43.021072Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|863aee57-8dcb1026-a686b866-dab0fc29_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:43.021087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.021092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.021095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:43.021117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.021131Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:43.021151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:43.021925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:43.021939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:43.021960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:43.022942Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adba0da9-1f7c9ac-b7ba2a51-a707a0c5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|863aee57-8dcb1026-a686b866-dab0fc29_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:43.025576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|863aee57-8dcb1026-a686b866-dab0fc29_0] Write session: try to update token
2025-10-09T21:44:43.025599Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|863aee57-8dcb1026-a686b866-dab0fc29_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:43.022152Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 362 partition: 0 MaxSeqNo: 25 sessionId: src_id|863aee57-8dcb1026-a686b866-dab0fc29_0
2025-10-09T21:44:43.039214Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:43.039232Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:43.039235Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:43.039237Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:43.039248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341225819026326:4567]
2025-10-09T21:44:43.039253Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341225819026326:4567]
2025-10-09T21:44:43.039258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341225819026326:4567]
2025-10-09T21:44:43.039310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:43.039317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.039321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:43.039519Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:43.041313Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:43.041340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:43.041345Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:43.061775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 362 sessionId: src_id|863aee57-8dcb1026-a686b866-dab0fc29_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:43.064519Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:43.065488Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341226180837617:4549]
2025-10-09T21:44:43.065758Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 361, pipe: [1:7559341226180837617:4549]
2025-10-09T21:44:43.211489Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:43.212270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:43.212286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:43.212311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341226180837617:4549], now have 1 active actors on pipe
2025-10-09T21:44:43.212801Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:43.212816Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:43.212900Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341226180837617:4549]
2025-10-09T21:44:43.212921Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:43.212924Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:43.212997Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 362 reason: error at write: Disconnected sessionId: src_id|863aee57-8dcb1026-a686b866-dab0fc29_0
2025-10-09T21:44:43.213185Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 362 sessionId: src_id|863aee57-8dcb1026-a686b866-dab0fc29_0 is DEAD
2025-10-09T21:44:43.213505Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|863aee57-8dcb1026-a686b866-dab0fc29_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:43.213516Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|863aee57-8dcb1026-a686b866-dab0fc29_0] Write session will restart in 0.007653s
2025-10-09T21:44:43.213544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|863aee57-8dcb1026-a686b866-dab0fc29_0] Write session: Do CDS request
2025-10-09T21:44:43.213551Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|863aee57-8dcb1026-a686b866-dab0fc29_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:43.224555Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|863aee57-8dcb1026-a686b866-dab0fc29_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:43.224720Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:43.224737Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 363
2025-10-09T21:44:43.224892Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 363 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:43.224941Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 363 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:43.224952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=363 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:43.224956Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:43.225633Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 363 sessionId:  describe result for acl check
2025-10-09T21:44:43.225675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:43.225682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:43.225684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:43.225690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341226180837628:4945] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:43.226297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341226180837628:4945] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:43.233352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341226180837628:4945] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:43.238809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341226180837628:4945] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:43.238832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341226180837628:4945] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:43.238836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341226180837628:4945] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:43.238847Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 363 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:43.254153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:43.256614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:43.256910Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:43.256921Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:43.256935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:43.257008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:43.257053Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341225819026518:4570]
2025-10-09T21:44:43.257611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:43.258537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:43.258588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:43.258942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:43.259010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:43.259016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:43.259017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:43.259018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:43.259019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:43.259020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:43.259021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.259022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.259023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.259029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:43.259041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:43.259044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:43.259048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:43.259051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:43.259055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:43.259059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.259062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.259077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.259085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:43.259094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:43.259099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:43.259106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:43.259111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:43.259116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:43.259122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:43.259126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:43.259130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:43.259135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:43.259139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:43.259143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:43.259148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.259153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.259157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.259162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.259166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.259171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.259199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:43.268172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:43.269539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:43.269558Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:43.269561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:43.269565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:43.269573Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 362 [2:7559341225819026518:4570]
2025-10-09T21:44:43.269588Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:43.269610Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:43.269614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.269617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.269620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.269638Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:43.269645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:43.269653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:43.269717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.269719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.269721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.269724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:43.269801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:43.269804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.269818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.269828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.269842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:43.269864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341225819026513:4570]
2025-10-09T21:44:43.269866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341225819026513:4570]
2025-10-09T21:44:43.269870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341225819026513:4570]
2025-10-09T21:44:43.269879Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:43.269895Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:43.269898Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:43.269902Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:43.269904Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:43.269909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:43.269912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:43.269924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341226180837620:2432], now have 1 active actors on pipe
2025-10-09T21:44:43.269928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341226180837667:4945], now have 1 active actors on pipe
2025-10-09T21:44:43.269967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341225819026513:4570]
2025-10-09T21:44:43.269974Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:43.269993Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:43.270269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:43.270277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:43.270309Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6c721212-c20e31e4-6d48b1d0-2e44dec2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:43.270350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.270360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.270363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:43.270375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.270399Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:43.270417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:43.272326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|863aee57-8dcb1026-a686b866-dab0fc29_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6c721212-c20e31e4-6d48b1d0-2e44dec2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:43.270007Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 362
2025-10-09T21:44:43.270071Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 362
2025-10-09T21:44:43.271057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 363 partition: 0 MaxSeqNo: 25 sessionId: src_id|6c721212-c20e31e4-6d48b1d0-2e44dec2_0
2025-10-09T21:44:43.274882Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6c721212-c20e31e4-6d48b1d0-2e44dec2_0] Write session: try to update token
2025-10-09T21:44:43.274898Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6c721212-c20e31e4-6d48b1d0-2e44dec2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:43.270916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:43.270928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:43.270949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:43.296773Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 363 sessionId: src_id|6c721212-c20e31e4-6d48b1d0-2e44dec2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:43.299691Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:43.303107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:43.303134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:43.303157Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:43.303161Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:43.303171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341225819026513:4570]
2025-10-09T21:44:43.303234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:43.303456Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:43.303609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:43.303805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:43.303958Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:43.304617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:43.304677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:43.304709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:43.305200Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:43.305364Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:43.312800Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341226180837811:4549]
2025-10-09T21:44:43.313636Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 362, pipe: [1:7559341226180837811:4549]
2025-10-09T21:44:43.313691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341226180837811:4549], now have 1 active actors on pipe
2025-10-09T21:44:43.315989Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:43.316003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:43.316004Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:43.316008Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:43.316017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341225819026513:4570]
2025-10-09T21:44:43.316019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341225819026513:4570]
2025-10-09T21:44:43.316021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341225819026513:4570]
2025-10-09T21:44:43.316062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:43.316072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.316074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:43.316221Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:43.318771Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:43.318794Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:43.318799Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:43.502080Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:43.511166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:43.511196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:43.511370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:43.511440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:43.511508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:43.511578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:43.511686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:43.511694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:43.511768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:43.511826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:43.511889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:43.511947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:43.511987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:43.511990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:43.512063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:43.512122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:43.512189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:43.512263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:43.512307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:43.512310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:43.512434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:43.512499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:43.512604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:43.512669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:43.512715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:43.512718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:43.512788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:43.512849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:43.512929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:43.512990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:43.513034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:43.513037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:43.513105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:43.513164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:43.513222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:43.513284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:43.513326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:43.513329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:43.513395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:43.513455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:43.513516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:43.513599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:43.514339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:43.514350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:43.514463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:43.514566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:43.514626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:43.514682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:43.514738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:43.514741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:43.514812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:43.514871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:43.514930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:43.514988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:43.515058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:43.515061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:43.515132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:43.515188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:43.515246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:43.515311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:43.515618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:43.515621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:43.517071Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:43.517127Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341226180837811:4549]
2025-10-09T21:44:43.517151Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:43.517153Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:43.517185Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 363 reason: error at write: Disconnected sessionId: src_id|6c721212-c20e31e4-6d48b1d0-2e44dec2_0
2025-10-09T21:44:43.517402Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 363 sessionId: src_id|6c721212-c20e31e4-6d48b1d0-2e44dec2_0 is DEAD
2025-10-09T21:44:43.517809Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c721212-c20e31e4-6d48b1d0-2e44dec2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:43.517823Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c721212-c20e31e4-6d48b1d0-2e44dec2_0] Write session will restart in 0.006673s
2025-10-09T21:44:43.517861Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c721212-c20e31e4-6d48b1d0-2e44dec2_0] Write session: Do CDS request
2025-10-09T21:44:43.517868Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c721212-c20e31e4-6d48b1d0-2e44dec2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:43.518041Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:43.525582Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6c721212-c20e31e4-6d48b1d0-2e44dec2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:43.525727Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:43.525744Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 364
2025-10-09T21:44:43.526746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 364 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:43.526790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 364 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:43.526799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=364 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:43.526802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:43.527316Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 364 sessionId:  describe result for acl check
2025-10-09T21:44:43.527358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:43.527360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:43.527361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:43.527367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341226180837828:4950] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:43.527917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341226180837828:4950] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:43.539774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341226180837828:4950] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:43.547651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341226180837828:4950] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:43.547769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341226180837828:4950] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:43.547775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341226180837828:4950] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:43.547786Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 364 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:43.558892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:43.560425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:43.560651Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:43.560654Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:43.560666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:43.560718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:43.561110Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341225819026706:4573]
2025-10-09T21:44:43.561699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:43.562607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:43.562660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:43.562972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:43.563013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:43.563014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:43.563015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:43.563017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:43.563018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:43.563019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:43.563020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.563021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.563022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.563028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:43.563039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:43.563043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:43.563047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:43.563050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:43.563054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:43.563057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.563060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.563063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.563070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:43.563077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:43.563083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:43.563089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:43.563096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:43.563101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:43.563105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:43.563110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:43.563117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:43.563122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:43.563127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:43.563131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:43.563136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.563142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.563148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.563153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.563158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.563163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.563190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:43.572622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:43.574283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:43.574292Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:43.574295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:43.574300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:43.574310Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 363 [2:7559341225819026706:4573]
2025-10-09T21:44:43.574345Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:43.574372Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:43.574375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.574379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.574382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.574404Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:43.574411Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:43.574420Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:43.574503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.574506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.574508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.574511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:43.574613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:43.574618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.574632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.574643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.574659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:43.574684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341225819026701:4573]
2025-10-09T21:44:43.574687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341225819026701:4573]
2025-10-09T21:44:43.574690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341225819026701:4573]
2025-10-09T21:44:43.574711Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:43.574732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:43.574748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:43.574752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:43.574804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341225819026701:4573]
2025-10-09T21:44:43.574808Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:43.576348Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 363
2025-10-09T21:44:43.576401Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 363
2025-10-09T21:44:43.575496Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:43.575508Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:43.575510Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:43.575519Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:43.579535Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 364 partition: 0 MaxSeqNo: 25 sessionId: src_id|c1fe36a-5ba01033-386476f6-de6ca987_0
2025-10-09T21:44:43.579954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c721212-c20e31e4-6d48b1d0-2e44dec2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c1fe36a-5ba01033-386476f6-de6ca987_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:43.578817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341226180837985:4950], now have 1 active actors on pipe
2025-10-09T21:44:43.578835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341226180837816:2432], now have 1 active actors on pipe
2025-10-09T21:44:43.578863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:43.578886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:43.578933Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c1fe36a-5ba01033-386476f6-de6ca987_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:43.578947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.578952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.578956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:43.578974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.578994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:43.579018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:43.579390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:43.579396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:43.579417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:43.582666Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1fe36a-5ba01033-386476f6-de6ca987_0] Write session: try to update token
2025-10-09T21:44:43.582685Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1fe36a-5ba01033-386476f6-de6ca987_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:43.605355Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 364 sessionId: src_id|c1fe36a-5ba01033-386476f6-de6ca987_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:43.604286Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:43.604304Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:43.604307Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:43.604309Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:43.604319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341225819026701:4573]
2025-10-09T21:44:43.604323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341225819026701:4573]
2025-10-09T21:44:43.604327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341225819026701:4573]
2025-10-09T21:44:43.604371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:43.604379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.604381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:43.604519Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:43.606261Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:43.606273Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:43.606277Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:43.607818Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:43.617522Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341226180838009:4549]
2025-10-09T21:44:43.619319Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 363, pipe: [1:7559341226180838009:4549]
2025-10-09T21:44:43.915990Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:43.916620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:43.916631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:43.916655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341226180838009:4549], now have 1 active actors on pipe
2025-10-09T21:44:43.917075Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:43.917091Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:43.917092Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:43.917143Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341226180838009:4549]
2025-10-09T21:44:43.917153Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 364 reason: error at write: Disconnected sessionId: src_id|c1fe36a-5ba01033-386476f6-de6ca987_0
2025-10-09T21:44:43.917348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 364 sessionId: src_id|c1fe36a-5ba01033-386476f6-de6ca987_0 is DEAD
2025-10-09T21:44:43.917071Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:43.917637Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1fe36a-5ba01033-386476f6-de6ca987_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:43.917654Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1fe36a-5ba01033-386476f6-de6ca987_0] Write session will restart in 0.006236s
2025-10-09T21:44:43.917696Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1fe36a-5ba01033-386476f6-de6ca987_0] Write session: Do CDS request
2025-10-09T21:44:43.917704Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1fe36a-5ba01033-386476f6-de6ca987_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:43.925568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1fe36a-5ba01033-386476f6-de6ca987_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:43.925755Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:43.925772Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 365
2025-10-09T21:44:43.925950Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 365 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:43.925996Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 365 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:43.926013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=365 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:43.926017Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:43.926530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 365 sessionId:  describe result for acl check
2025-10-09T21:44:43.926574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:43.926581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:43.926583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:43.926590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341226180838056:4961] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:43.927143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341226180838056:4961] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:43.929368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341226180838056:4961] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:43.932406Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341226180838056:4961] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:43.932422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341226180838056:4961] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:43.932428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341226180838056:4961] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:43.932437Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 365 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:43.947824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:43.948908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:43.949444Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:43.949458Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:43.949473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:43.949550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:43.949597Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341225819026933:4587]
2025-10-09T21:44:43.950424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:43.952644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:43.952705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:43.953169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:43.953249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:43.953250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:43.953253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:43.953255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:43.953256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:43.953257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:43.953258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.953259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.953260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.953267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:43.953279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:43.953283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:43.953287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:43.953291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:43.953295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:43.953299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.953302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.953306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.953313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:43.953323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:43.953331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:43.953337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:43.953342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:43.953346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:43.953350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:43.953356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:43.953364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:43.953368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:43.953372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:43.953377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:43.953382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.953388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.953393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.953404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.953408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.953413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.953446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:43.964841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:43.966994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:43.967008Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:43.967012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:43.967017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:43.967026Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 364 [2:7559341225819026933:4587]
2025-10-09T21:44:43.967042Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:43.967065Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:43.967069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.967073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.967075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.967094Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:43.967099Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:43.967112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:43.967187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.967194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.967196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.967199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:43.967628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:43.967639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.967655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:43.967668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:43.967688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:43.967713Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341225819026928:4587]
2025-10-09T21:44:43.967721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341225819026928:4587]
2025-10-09T21:44:43.967723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341225819026928:4587]
2025-10-09T21:44:43.967735Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:43.967750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:43.967759Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:43.967763Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:43.967765Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:43.967767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:43.967770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:43.967818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341225819026928:4587]
2025-10-09T21:44:43.967826Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:43.967829Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:43.967837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341226180838043:2432], now have 1 active actors on pipe
2025-10-09T21:44:43.967841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341226180838136:4961], now have 1 active actors on pipe
2025-10-09T21:44:43.968038Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 364
2025-10-09T21:44:43.968093Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 364
2025-10-09T21:44:43.968261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:43.968280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:43.968314Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e8f0ee9f-aeccefd4-a8452653-a3435e48_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:43.968329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.968333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.968335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:43.968359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.968375Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:43.968396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:43.969264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:43.969275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:43.969305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:43.969495Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 365 partition: 0 MaxSeqNo: 25 sessionId: src_id|e8f0ee9f-aeccefd4-a8452653-a3435e48_0
2025-10-09T21:44:43.970165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1fe36a-5ba01033-386476f6-de6ca987_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e8f0ee9f-aeccefd4-a8452653-a3435e48_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:43.976714Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8f0ee9f-aeccefd4-a8452653-a3435e48_0] Write session: try to update token
2025-10-09T21:44:43.976741Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8f0ee9f-aeccefd4-a8452653-a3435e48_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:43.990558Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:43.990572Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:43.990575Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:43.990576Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:43.990589Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341225819026928:4587]
2025-10-09T21:44:43.990599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341225819026928:4587]
2025-10-09T21:44:43.990603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341225819026928:4587]
2025-10-09T21:44:43.990648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:43.990656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:43.990654Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:43.990658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:43.990666Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:43.990671Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:43.990813Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:44.014120Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 365 sessionId: src_id|e8f0ee9f-aeccefd4-a8452653-a3435e48_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:44.016638Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:44.026110Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341230475805534:4549]
2025-10-09T21:44:44.026630Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 364, pipe: [1:7559341230475805534:4549]
2025-10-09T21:44:44.170852Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:44.171360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:44.171381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:44.171408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341230475805534:4549], now have 1 active actors on pipe
2025-10-09T21:44:44.173397Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:44.173604Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341230475805534:4549]
2025-10-09T21:44:44.173647Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:44.173687Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:44.173690Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:44.173859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 365 reason: error at write: Disconnected sessionId: src_id|e8f0ee9f-aeccefd4-a8452653-a3435e48_0
2025-10-09T21:44:44.174079Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 365 sessionId: src_id|e8f0ee9f-aeccefd4-a8452653-a3435e48_0 is DEAD
2025-10-09T21:44:44.174594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8f0ee9f-aeccefd4-a8452653-a3435e48_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:44.174605Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8f0ee9f-aeccefd4-a8452653-a3435e48_0] Write session will restart in 0.005012s
2025-10-09T21:44:44.174639Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8f0ee9f-aeccefd4-a8452653-a3435e48_0] Write session: Do CDS request
2025-10-09T21:44:44.174646Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8f0ee9f-aeccefd4-a8452653-a3435e48_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:44.180635Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8f0ee9f-aeccefd4-a8452653-a3435e48_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:44.181029Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:44.181052Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 366
2025-10-09T21:44:44.181314Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 366 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:44.181381Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 366 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:44.181396Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=366 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:44.181408Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:44.182083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 366 sessionId:  describe result for acl check
2025-10-09T21:44:44.182129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:44.182135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:44.182138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:44.182144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341230475805544:4966] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:44.182805Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341230475805544:4966] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:44.188859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341230475805544:4966] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:44.199895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341230475805544:4966] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:44.199933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341230475805544:4966] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:44.199937Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341230475805544:4966] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:44.199948Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 366 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:44.212383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:44.218065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:44.218383Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:44.218388Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:44.218402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:44.218466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:44.218509Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341230113994418:4590]
2025-10-09T21:44:44.219117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:44.219544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:44.219599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:44.219929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:44.219979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:44.219981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:44.219982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:44.219983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:44.219985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:44.219986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:44.219987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.219989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.219990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.219995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:44.220007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:44.220011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:44.220015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:44.220019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:44.220024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:44.220028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.220031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.220035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.220043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:44.220051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:44.220058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:44.220064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:44.220070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:44.220075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:44.220079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:44.220084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:44.220089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:44.220094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:44.220099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:44.220104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:44.220110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.220115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.220120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.220125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.220130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.220136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.220167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:44.228607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:44.231491Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 365
2025-10-09T21:44:44.230189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:44.230195Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:44.230198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:44.231498Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 365
2025-10-09T21:44:44.230203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:44.230213Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 365 [2:7559341230113994418:4590]
2025-10-09T21:44:44.230230Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:44.230251Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:44.230254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.230258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.230260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.230280Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:44.230287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:44.230295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:44.230379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.230381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.230383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.230386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:44.230468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:44.230471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.230486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.230499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.230516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:44.230538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341230113994413:4590]
2025-10-09T21:44:44.230541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341230113994413:4590]
2025-10-09T21:44:44.230543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341230113994413:4590]
2025-10-09T21:44:44.230552Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:44.230566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:44.230582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:44.230582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:44.230585Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:44.230585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:44.230587Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:44.230629Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341230113994413:4590]
2025-10-09T21:44:44.230632Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:44.230635Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:44.231375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341230475805537:2432], now have 1 active actors on pipe
2025-10-09T21:44:44.231381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341230475805622:4966], now have 1 active actors on pipe
2025-10-09T21:44:44.231594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:44.231601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:44.231635Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2db7de73-2844514f-b875c5dd-3aa6a458_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:44.231646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.231650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.231653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:44.231664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.231676Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:44.231695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:44.232077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 366 partition: 0 MaxSeqNo: 25 sessionId: src_id|2db7de73-2844514f-b875c5dd-3aa6a458_0
2025-10-09T21:44:44.231903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:44.231916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:44.231937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:44.235332Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8f0ee9f-aeccefd4-a8452653-a3435e48_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2db7de73-2844514f-b875c5dd-3aa6a458_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:44.237836Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2db7de73-2844514f-b875c5dd-3aa6a458_0] Write session: try to update token
2025-10-09T21:44:44.237856Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2db7de73-2844514f-b875c5dd-3aa6a458_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:44.248138Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:44.248156Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:44.248159Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:44.248162Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:44.248172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341230113994413:4590]
2025-10-09T21:44:44.248176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341230113994413:4590]
2025-10-09T21:44:44.248181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341230113994413:4590]
2025-10-09T21:44:44.248234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:44.248242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.248244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:44.248262Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:44.248268Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:44.248272Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:44.248405Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:44.257293Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 366 sessionId: src_id|2db7de73-2844514f-b875c5dd-3aa6a458_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:44.259800Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:44.273670Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341230475805732:4549]
2025-10-09T21:44:44.274311Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 365, pipe: [1:7559341230475805732:4549]
2025-10-09T21:44:44.413838Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:44.414444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:44.414461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:44.414486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341230475805732:4549], now have 1 active actors on pipe
2025-10-09T21:44:44.414852Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:44.415295Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:44.415313Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:44.415386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 366 reason: error at write: Disconnected sessionId: src_id|2db7de73-2844514f-b875c5dd-3aa6a458_0
2025-10-09T21:44:44.415429Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341230475805732:4549]
2025-10-09T21:44:44.415453Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:44.415902Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2db7de73-2844514f-b875c5dd-3aa6a458_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:44.415914Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2db7de73-2844514f-b875c5dd-3aa6a458_0] Write session will restart in 0.007052s
2025-10-09T21:44:44.415950Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2db7de73-2844514f-b875c5dd-3aa6a458_0] Write session: Do CDS request
2025-10-09T21:44:44.415961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2db7de73-2844514f-b875c5dd-3aa6a458_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:44.415580Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 366 sessionId: src_id|2db7de73-2844514f-b875c5dd-3aa6a458_0 is DEAD
2025-10-09T21:44:44.426730Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2db7de73-2844514f-b875c5dd-3aa6a458_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:44.427280Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:44.427300Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 367
2025-10-09T21:44:44.427774Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 367 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:44.427818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 367 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:44.427824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=367 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:44.427828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:44.428683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 367 sessionId:  describe result for acl check
2025-10-09T21:44:44.428720Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:44.428722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:44.428724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:44.428730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341230475805750:4970] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:44.429345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341230475805750:4970] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:44.435355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341230475805750:4970] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:44.440592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341230475805750:4970] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:44.440613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341230475805750:4970] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:44.440617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341230475805750:4970] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:44.440627Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 367 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:44.444149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:44.446188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:44.446858Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:44.446870Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:44.446886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:44.446959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:44.447005Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341230113994608:4593]
2025-10-09T21:44:44.447781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:44.448168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:44.448222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:44.448568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:44.448633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:44.448640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:44.448641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:44.448642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:44.448643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:44.448644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:44.448645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.448646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.448647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.448654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:44.448677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:44.448681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:44.448684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:44.448688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:44.448693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:44.448698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.448707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.448711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.448728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:44.448741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:44.448752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:44.448758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:44.448764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:44.448775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:44.448780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:44.448785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:44.448790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:44.448795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:44.448800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:44.448811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:44.448821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.448830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.448835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.448845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.448850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.448860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.448889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:44.459512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:44.461115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:44.461130Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:44.461132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:44.461137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:44.461146Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 366 [2:7559341230113994608:4593]
2025-10-09T21:44:44.461170Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:44.461190Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:44.461194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.461197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.461200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.461215Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:44.461221Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:44.461232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:44.461301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.461311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.461312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.461315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:44.461407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:44.461414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.461427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.461437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.461452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:44.461473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341230113994603:4593]
2025-10-09T21:44:44.461475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341230113994603:4593]
2025-10-09T21:44:44.461477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341230113994603:4593]
2025-10-09T21:44:44.461487Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:44.461502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:44.461512Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:44.461515Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:44.461516Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:44.461517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:44.461519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:44.461531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341230475805735:2432], now have 1 active actors on pipe
2025-10-09T21:44:44.461534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341230475805913:4970], now have 1 active actors on pipe
2025-10-09T21:44:44.461572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341230113994603:4593]
2025-10-09T21:44:44.461581Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:44.461671Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 366
2025-10-09T21:44:44.461647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:44.461728Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 366
2025-10-09T21:44:44.469769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:44.469793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:44.469837Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|721ff78b-3c85349d-66855981-9b441833_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:44.469861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.469867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.469871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:44.469886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.469905Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:44.469931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:44.470803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 367 partition: 0 MaxSeqNo: 25 sessionId: src_id|721ff78b-3c85349d-66855981-9b441833_0
2025-10-09T21:44:44.470385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:44.470398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:44.470419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:44.474284Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2db7de73-2844514f-b875c5dd-3aa6a458_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|721ff78b-3c85349d-66855981-9b441833_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:44.476927Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|721ff78b-3c85349d-66855981-9b441833_0] Write session: try to update token
2025-10-09T21:44:44.476951Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|721ff78b-3c85349d-66855981-9b441833_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:44.480366Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:44.480386Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:44.480388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:44.480390Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:44.480400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341230113994603:4593]
2025-10-09T21:44:44.480406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341230113994603:4593]
2025-10-09T21:44:44.480409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341230113994603:4593]
2025-10-09T21:44:44.480457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:44.480464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.480468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:44.485448Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:44.485471Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:44.485476Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:44.486700Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:44.511647Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 367 sessionId: src_id|721ff78b-3c85349d-66855981-9b441833_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:44.514071Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:44.516960Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341230475805932:4549]
2025-10-09T21:44:44.518645Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 366, pipe: [1:7559341230475805932:4549]
2025-10-09T21:44:44.693539Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:44.698703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:44.698727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:44.698749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341230475805932:4549], now have 1 active actors on pipe
2025-10-09T21:44:44.699560Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:44.700761Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|721ff78b-3c85349d-66855981-9b441833_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:44.700781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|721ff78b-3c85349d-66855981-9b441833_0] Write session will restart in 0.006447s
2025-10-09T21:44:44.700806Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|721ff78b-3c85349d-66855981-9b441833_0] Write session: Do CDS request
2025-10-09T21:44:44.700813Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|721ff78b-3c85349d-66855981-9b441833_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:44.700296Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:44.700296Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341230475805932:4549]
2025-10-09T21:44:44.700319Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:44.700393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 367 reason: status is not ok: tablet will be restarted right now sessionId: src_id|721ff78b-3c85349d-66855981-9b441833_0
2025-10-09T21:44:44.700593Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 367 sessionId: src_id|721ff78b-3c85349d-66855981-9b441833_0 is DEAD
2025-10-09T21:44:44.708782Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|721ff78b-3c85349d-66855981-9b441833_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:44.714894Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:44.714915Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 368
2025-10-09T21:44:44.715088Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 368 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:44.715141Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 368 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:44.715155Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=368 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:44.715160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:44.715733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 368 sessionId:  describe result for acl check
2025-10-09T21:44:44.715774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:44.715776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:44.715778Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:44.715787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341230475805973:4975] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:44.716378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341230475805973:4975] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:44.720035Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341230475805973:4975] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:44.723975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341230475805973:4975] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:44.723993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341230475805973:4975] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:44.723997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341230475805973:4975] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:44.724008Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 368 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:44.734125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:44.737069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:44.737317Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:44.737329Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:44.737342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:44.737404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:44.737442Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341230113994799:4595]
2025-10-09T21:44:44.738098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:44.738502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:44.738559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:44.738913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:44.738974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:44.738975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:44.738977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:44.738978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:44.738979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:44.738980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:44.738981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.738983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.738984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.738989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:44.739000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:44.739004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:44.739007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:44.739011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:44.739014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:44.739018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.739021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.739024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.739032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:44.739041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:44.739048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:44.739053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:44.739059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:44.739064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:44.739068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:44.739074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:44.739079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:44.739084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:44.739089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:44.739094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:44.739099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.739105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.739110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.739116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.739121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.739127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.739159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:44.748676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:44.749885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:44.749904Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:44.749907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:44.749912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:44.749920Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 367 [2:7559341230113994799:4595]
2025-10-09T21:44:44.749939Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:44.749958Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:44.749962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.749965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.749969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.749988Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:44.749995Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:44.750003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:44.750090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.750094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.750097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.750111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:44.750828Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 367
2025-10-09T21:44:44.750904Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 367
2025-10-09T21:44:44.750184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:44.750187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.750202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.750218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.750235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:44.750257Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341230113994794:4595]
2025-10-09T21:44:44.750260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341230113994794:4595]
2025-10-09T21:44:44.750263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341230113994794:4595]
2025-10-09T21:44:44.750279Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:44.750295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:44.750311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:44.750327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:44.750372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341230113994794:4595]
2025-10-09T21:44:44.750375Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:44.750444Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:44.750447Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:44.750450Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:44.750457Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:44.750854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341230475805936:2432], now have 1 active actors on pipe
2025-10-09T21:44:44.750858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341230475806033:4975], now have 1 active actors on pipe
2025-10-09T21:44:44.751453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:44.751468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:44.751501Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ce8be5e9-811922fe-ec35a683-cc5dea2b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:44.751511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.751516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.751519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:44.751532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.751545Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:44.751564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:44.751936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:44.751951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:44.751973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:44.752324Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 368 partition: 0 MaxSeqNo: 25 sessionId: src_id|ce8be5e9-811922fe-ec35a683-cc5dea2b_0
2025-10-09T21:44:44.752758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|721ff78b-3c85349d-66855981-9b441833_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ce8be5e9-811922fe-ec35a683-cc5dea2b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:44.755386Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ce8be5e9-811922fe-ec35a683-cc5dea2b_0] Write session: try to update token
2025-10-09T21:44:44.755407Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ce8be5e9-811922fe-ec35a683-cc5dea2b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:44.772319Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:44.772335Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:44.772338Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:44.772340Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:44.772350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341230113994794:4595]
2025-10-09T21:44:44.772354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341230113994794:4595]
2025-10-09T21:44:44.772358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341230113994794:4595]
2025-10-09T21:44:44.772403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:44.772413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.772415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:44.772546Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:44.773435Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:44.773442Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:44.773447Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:44.785270Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 368 sessionId: src_id|ce8be5e9-811922fe-ec35a683-cc5dea2b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:44.788405Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:44.800526Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341230475806136:4549]
2025-10-09T21:44:44.800942Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 367, pipe: [1:7559341230475806136:4549]
2025-10-09T21:44:44.938486Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:44.938825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:44.938848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:44.938873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341230475806136:4549], now have 1 active actors on pipe
2025-10-09T21:44:44.939626Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:44.939621Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341230475806136:4549]
2025-10-09T21:44:44.939661Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:44.939697Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:44.939703Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:44.939763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 368 reason: error at write: Disconnected sessionId: src_id|ce8be5e9-811922fe-ec35a683-cc5dea2b_0
2025-10-09T21:44:44.939954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 368 sessionId: src_id|ce8be5e9-811922fe-ec35a683-cc5dea2b_0 is DEAD
2025-10-09T21:44:44.942328Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce8be5e9-811922fe-ec35a683-cc5dea2b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:44.942341Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce8be5e9-811922fe-ec35a683-cc5dea2b_0] Write session will restart in 0.008613s
2025-10-09T21:44:44.942382Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce8be5e9-811922fe-ec35a683-cc5dea2b_0] Write session: Do CDS request
2025-10-09T21:44:44.942388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce8be5e9-811922fe-ec35a683-cc5dea2b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:44.958927Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ce8be5e9-811922fe-ec35a683-cc5dea2b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:44.962397Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:44.962417Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 369
2025-10-09T21:44:44.962584Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 369 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:44.962624Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 369 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:44.962637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=369 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:44.962641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:44.963109Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 369 sessionId:  describe result for acl check
2025-10-09T21:44:44.963152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:44.963154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:44.963156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:44.963162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341230475806321:4987] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:44.963702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341230475806321:4987] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:44.965162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:44.967029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341230475806321:4987] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:44.968118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:44.968345Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:44.968372Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:44.968386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:44.968458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:44.968519Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341230113995026:4609]
2025-10-09T21:44:44.969155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:44.969488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:44.969541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:44.969875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:44.969949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:44.969956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:44.969957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:44.969958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:44.969960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:44.969961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:44.969963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.969964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.969965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.969971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:44.969982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:44.969987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:44.969990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:44.969994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:44.969998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:44.970002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.970005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.970009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.970017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:44.970025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:44.970051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:44.970056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:44.970062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:44.970066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:44.970072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:44.970087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:44.970092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:44.970097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:44.970101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:44.970106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:44.970112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.970122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.970126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.970136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.970142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.970147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.970172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:44.974985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341230475806321:4987] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:44.975007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341230475806321:4987] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:44.975011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341230475806321:4987] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:44.975021Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 369 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:44.983345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:44.984555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:44.984577Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:44.984581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:44.984586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:44.984594Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 368 [2:7559341230113995026:4609]
2025-10-09T21:44:44.984610Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:44.984631Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:44.984634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.984638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.984640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.984660Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:44.984667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:44.984678Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:44.984756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.984765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.984768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.984772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:44.984848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:44.984856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:44.984869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:44.984880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:44.984895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:44.984922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341230113995021:4609]
2025-10-09T21:44:44.984931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341230113995021:4609]
2025-10-09T21:44:44.984933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341230113995021:4609]
2025-10-09T21:44:44.984947Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:44.984964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:44.984969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:44.984984Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:44.984987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:44.984990Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:44.984993Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:44.985002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341230475806173:2432], now have 1 active actors on pipe
2025-10-09T21:44:44.985010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341230475806359:4987], now have 1 active actors on pipe
2025-10-09T21:44:44.985051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341230113995021:4609]
2025-10-09T21:44:44.985066Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:44.985062Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 368
2025-10-09T21:44:44.985085Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 368
2025-10-09T21:44:44.985107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:44.985142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:44.985147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:44.985175Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|825bc9f5-8334131-3bccf2de-70336e04_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:44.985189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.985194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.985197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:44.985208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.985470Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:44.985500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:44.985730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:44.985745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:44.985763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:44.985862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 369 partition: 0 MaxSeqNo: 25 sessionId: src_id|825bc9f5-8334131-3bccf2de-70336e04_0
2025-10-09T21:44:44.986264Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce8be5e9-811922fe-ec35a683-cc5dea2b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|825bc9f5-8334131-3bccf2de-70336e04_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:44.988792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|825bc9f5-8334131-3bccf2de-70336e04_0] Write session: try to update token
2025-10-09T21:44:44.988809Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|825bc9f5-8334131-3bccf2de-70336e04_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:45.001394Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:45.001411Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:45.001421Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341230113995021:4609]
2025-10-09T21:44:45.001473Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:45.001477Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:45.001664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:45.001828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:45.001980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:45.002169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:45.002401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:45.002450Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:45.014666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.014691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.014736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:45.011310Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 369 sessionId: src_id|825bc9f5-8334131-3bccf2de-70336e04_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:45.014362Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:45.015019Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:45.021192Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:45.021213Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:45.021218Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:45.021220Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:45.021233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341230113995021:4609]
2025-10-09T21:44:45.021238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341230113995021:4609]
2025-10-09T21:44:45.021242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341230113995021:4609]
2025-10-09T21:44:45.021294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:45.021308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.021311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:45.021459Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:45.025506Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:45.025520Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:45.025524Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:45.039521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341234770773665:4549]
2025-10-09T21:44:45.039905Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 368, pipe: [1:7559341234770773665:4549]
2025-10-09T21:44:45.187276Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:45.188189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341234770773665:4549], now have 1 active actors on pipe
2025-10-09T21:44:45.195984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.196015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.196195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.196267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.196334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.196400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.196450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.196456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.196523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.196587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.196643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.196707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.196746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.196749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.196819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.196878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.196976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.197035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.197081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.197085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.197153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.197212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.197275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.197341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.197380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.197383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.197454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.197511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.197574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.197637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.197680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.197682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.197778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.197842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.197900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.197984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.198196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.198207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.198271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.198347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.198413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.199944Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:45.200003Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:45.200007Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:45.200019Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341234770773665:4549]
2025-10-09T21:44:45.200051Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 369 reason: error at write: Disconnected sessionId: src_id|825bc9f5-8334131-3bccf2de-70336e04_0
2025-10-09T21:44:45.200263Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 369 sessionId: src_id|825bc9f5-8334131-3bccf2de-70336e04_0 is DEAD
2025-10-09T21:44:45.200434Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|825bc9f5-8334131-3bccf2de-70336e04_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:45.200444Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|825bc9f5-8334131-3bccf2de-70336e04_0] Write session will restart in 0.005747s
2025-10-09T21:44:45.200471Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|825bc9f5-8334131-3bccf2de-70336e04_0] Write session: Do CDS request
2025-10-09T21:44:45.200479Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|825bc9f5-8334131-3bccf2de-70336e04_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:45.198470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.198605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.198609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.198675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.198729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.198788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.198845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.198883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.198886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.198950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.199004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.199064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.199126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.199265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.199269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.199338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.199396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.199456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.199513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.199552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.199554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.200935Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:45.207661Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|825bc9f5-8334131-3bccf2de-70336e04_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:45.207736Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:45.207756Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 370
2025-10-09T21:44:45.208176Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 370 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:45.208232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 370 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:45.208243Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=370 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:45.208247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:45.208744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 370 sessionId:  describe result for acl check
2025-10-09T21:44:45.208783Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:45.208787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:45.208789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:45.208794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341234770773683:4991] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:45.209353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341234770773683:4991] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:45.213232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341234770773683:4991] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:45.218262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341234770773683:4991] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:45.218285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341234770773683:4991] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:45.218289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341234770773683:4991] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:45.218299Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 370 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:45.222737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:45.225807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:45.226113Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:45.226124Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:45.226140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:45.226211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:45.226290Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341234408962516:4612]
2025-10-09T21:44:45.226904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:45.227296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:45.227356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:45.227746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:45.227829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:45.227835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:45.227837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:45.227839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:45.227841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:45.227842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:45.227843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.227859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.227861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.227868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:45.227885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:45.227893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:45.227897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:45.227912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:45.227917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:45.227921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.227926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.227931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.227938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:45.227946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:45.227953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:45.227958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:45.227964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:45.227970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:45.227974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:45.227979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:45.227983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:45.227988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:45.227992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:45.227997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:45.228004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.228015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.228019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.228028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.228034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.228040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.228067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:45.235557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:45.237368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:45.237390Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:45.237392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:45.237397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:45.237405Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 369 [2:7559341234408962516:4612]
2025-10-09T21:44:45.237425Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:45.237446Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:45.237451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.237459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.237462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.237482Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:45.237488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:45.237497Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:45.237581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.237590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.237592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.237596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:45.237674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:45.237683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.237699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.237711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.237726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:45.237748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341234770773668:2432], now have 1 active actors on pipe
2025-10-09T21:44:45.237752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341234770773831:4991], now have 1 active actors on pipe
2025-10-09T21:44:45.237761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341234408962511:4612]
2025-10-09T21:44:45.237763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341234408962511:4612]
2025-10-09T21:44:45.237765Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341234408962511:4612]
2025-10-09T21:44:45.237774Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:45.237791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:45.237794Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:45.237799Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:45.237802Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:45.237807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:45.237809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:45.237834Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 369
2025-10-09T21:44:45.237856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341234408962511:4612]
2025-10-09T21:44:45.237857Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 369
2025-10-09T21:44:45.237868Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:45.237870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:45.237903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.237908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.237946Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8afa64d1-5d3b8720-bb1d3504-3840c5d7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:45.237964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.237968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.237971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:45.237982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.237994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:45.238018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:45.238635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.238649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.238683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:45.238961Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 370 partition: 0 MaxSeqNo: 25 sessionId: src_id|8afa64d1-5d3b8720-bb1d3504-3840c5d7_0
2025-10-09T21:44:45.239446Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|825bc9f5-8334131-3bccf2de-70336e04_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8afa64d1-5d3b8720-bb1d3504-3840c5d7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:45.242171Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8afa64d1-5d3b8720-bb1d3504-3840c5d7_0] Write session: try to update token
2025-10-09T21:44:45.242193Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8afa64d1-5d3b8720-bb1d3504-3840c5d7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:45.259359Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:45.259386Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:45.259388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:45.259390Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:45.259400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341234408962511:4612]
2025-10-09T21:44:45.259405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341234408962511:4612]
2025-10-09T21:44:45.259408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341234408962511:4612]
2025-10-09T21:44:45.259453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:45.259462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.259465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:45.259470Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:45.259477Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:45.259481Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:45.259614Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:45.258573Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 370 sessionId: src_id|8afa64d1-5d3b8720-bb1d3504-3840c5d7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:45.261228Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:45.300251Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341234770773867:4549]
2025-10-09T21:44:45.300641Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 369, pipe: [1:7559341234770773867:4549]
2025-10-09T21:44:45.434418Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:45.434731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.434746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.434793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341234770773867:4549], now have 1 active actors on pipe
2025-10-09T21:44:45.435050Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:45.435109Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341234770773867:4549]
2025-10-09T21:44:45.435141Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:45.435143Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:45.435146Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:45.435239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 370 reason: error at write: Disconnected sessionId: src_id|8afa64d1-5d3b8720-bb1d3504-3840c5d7_0
2025-10-09T21:44:45.435448Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 370 sessionId: src_id|8afa64d1-5d3b8720-bb1d3504-3840c5d7_0 is DEAD
2025-10-09T21:44:45.435617Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8afa64d1-5d3b8720-bb1d3504-3840c5d7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:45.435629Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8afa64d1-5d3b8720-bb1d3504-3840c5d7_0] Write session will restart in 0.005826s
2025-10-09T21:44:45.435660Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8afa64d1-5d3b8720-bb1d3504-3840c5d7_0] Write session: Do CDS request
2025-10-09T21:44:45.435671Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8afa64d1-5d3b8720-bb1d3504-3840c5d7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:45.446628Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8afa64d1-5d3b8720-bb1d3504-3840c5d7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:45.446730Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:45.446750Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 371
2025-10-09T21:44:45.446895Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 371 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:45.446928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 371 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:45.446938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=371 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:45.446942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:45.448162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 371 sessionId:  describe result for acl check
2025-10-09T21:44:45.448205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:45.448207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:45.448209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:45.448216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341234770773930:4997] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:45.449023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341234770773930:4997] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:45.452525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341234770773930:4997] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:45.454593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:45.458671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341234770773930:4997] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:45.458678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:45.458687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341234770773930:4997] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:45.458692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341234770773930:4997] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:45.458702Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 371 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:45.458906Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:45.458916Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:45.458929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:45.458991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:45.459034Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341234408962711:4615]
2025-10-09T21:44:45.459633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:45.459968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:45.460012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:45.460367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:45.460419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:45.460425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:45.460427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:45.460428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:45.460429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:45.460430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:45.460432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.460433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.460434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.460439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:45.460450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:45.460454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:45.460458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:45.460461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:45.460466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:45.460469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.460473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.460476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.460483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:45.460490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:45.460497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:45.460503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:45.460508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:45.460513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:45.460517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:45.460522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:45.460527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:45.460532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:45.460536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:45.460540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:45.460546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.460556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.460561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.460567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.460571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.460577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.460605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:45.468924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:45.469987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:45.470006Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:45.470010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:45.470015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:45.470023Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 370 [2:7559341234408962711:4615]
2025-10-09T21:44:45.470086Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:45.470114Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:45.470118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.470121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.470124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.470142Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:45.470150Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:45.470165Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:45.470228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.470240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.470242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.470245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:45.470333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:45.470341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.470352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.470364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.470377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:45.470403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341234770773873:2432], now have 1 active actors on pipe
2025-10-09T21:44:45.470413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341234408962706:4615]
2025-10-09T21:44:45.470416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341234408962706:4615]
2025-10-09T21:44:45.470417Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341234408962706:4615]
2025-10-09T21:44:45.470432Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:45.470448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341234770774060:4997], now have 1 active actors on pipe
2025-10-09T21:44:45.470458Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:45.470472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:45.470475Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:45.470500Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:45.470504Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:45.470506Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:45.470516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341234408962706:4615]
2025-10-09T21:44:45.470519Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:45.470521Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:45.470602Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 370
2025-10-09T21:44:45.470653Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 370
2025-10-09T21:44:45.470712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.470727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.470772Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e0b79327-ebdb1fb4-337f7623-8cfc0679_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:45.470786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.470791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.470794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:45.470821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.470834Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:45.470858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:45.471150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.471162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.471182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:45.471300Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 371 partition: 0 MaxSeqNo: 25 sessionId: src_id|e0b79327-ebdb1fb4-337f7623-8cfc0679_0
2025-10-09T21:44:45.471789Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8afa64d1-5d3b8720-bb1d3504-3840c5d7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e0b79327-ebdb1fb4-337f7623-8cfc0679_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:45.474415Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e0b79327-ebdb1fb4-337f7623-8cfc0679_0] Write session: try to update token
2025-10-09T21:44:45.474438Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e0b79327-ebdb1fb4-337f7623-8cfc0679_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:45.489230Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 371 sessionId: src_id|e0b79327-ebdb1fb4-337f7623-8cfc0679_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:45.491851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:45.494667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.494694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.494719Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:45.494722Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:45.494733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341234408962706:4615]
2025-10-09T21:44:45.494800Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:45.494803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:45.494994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:45.495159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:45.495320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:45.495469Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:45.495692Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:45.495743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:45.495772Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:45.495871Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:45.507015Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:45.507032Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:45.507035Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:45.507036Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:45.507048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341234408962706:4615]
2025-10-09T21:44:45.507052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341234408962706:4615]
2025-10-09T21:44:45.507055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341234408962706:4615]
2025-10-09T21:44:45.507102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:45.507118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.507119Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:45.507121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:45.507126Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:45.507129Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:45.507255Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:45.535234Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341234770774077:4549]
2025-10-09T21:44:45.535626Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 370, pipe: [1:7559341234770774077:4549]
2025-10-09T21:44:45.667091Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:45.684390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.684417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.684602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.684666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.684727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.684792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.684834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.684840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.684912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.684976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.685033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.685108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.685152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.685160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.685226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.685293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.685357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.685423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.685553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.685556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.685624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.685684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.685741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.685801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.685848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.685851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.685946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.686000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.686148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.688400Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:45.688433Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:45.688469Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 371 reason: error at write: Disconnected sessionId: src_id|e0b79327-ebdb1fb4-337f7623-8cfc0679_0
2025-10-09T21:44:45.688737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 371 sessionId: src_id|e0b79327-ebdb1fb4-337f7623-8cfc0679_0 is DEAD
2025-10-09T21:44:45.688896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0b79327-ebdb1fb4-337f7623-8cfc0679_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:45.688908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0b79327-ebdb1fb4-337f7623-8cfc0679_0] Write session will restart in 0.005731s
2025-10-09T21:44:45.688943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0b79327-ebdb1fb4-337f7623-8cfc0679_0] Write session: Do CDS request
2025-10-09T21:44:45.688949Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0b79327-ebdb1fb4-337f7623-8cfc0679_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:45.689093Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:45.689141Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341234770774077:4549]
2025-10-09T21:44:45.686212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.686255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.686258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.686344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.686398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.686451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.686507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.686661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.686664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.686727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.686781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.686841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.686893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.687013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.687017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.687074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.687128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.687178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.687230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.687356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.687360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.687428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.687487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.687546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.687603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.687644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.687647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.687717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.687769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.687826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.687926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.688050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.688054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.688129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341234770774077:4549], now have 1 active actors on pipe
2025-10-09T21:44:45.689612Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:45.695614Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e0b79327-ebdb1fb4-337f7623-8cfc0679_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:45.696030Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:45.696046Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 372
2025-10-09T21:44:45.696188Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 372 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:45.696240Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 372 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:45.696253Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=372 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:45.696257Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:45.696762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 372 sessionId:  describe result for acl check
2025-10-09T21:44:45.696802Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:45.696804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:45.696806Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:45.696812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341234770774095:5002] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:45.697395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341234770774095:5002] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:45.700478Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341234770774095:5002] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:45.704884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341234770774095:5002] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:45.704904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341234770774095:5002] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:45.704908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341234770774095:5002] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:45.704919Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 372 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:45.710507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:45.714837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:45.715103Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:45.715116Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:45.715129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:45.715192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:45.715241Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341234408962907:4619]
2025-10-09T21:44:45.715874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:45.716252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:45.716315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:45.716660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:45.716733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:45.716743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:45.716744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:45.716745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:45.716747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:45.716748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:45.716749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.716750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.716751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.716757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:45.716768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:45.716771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:45.716774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:45.716778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:45.716782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:45.716785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.716789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.716793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.716800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:45.716808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:45.716814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:45.716819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:45.716823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:45.716827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:45.716831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:45.716836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:45.716841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:45.716845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:45.716850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:45.716855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:45.716860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.716865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.716868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.716873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.716877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.716882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.716909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:45.726923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:45.728266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:45.728283Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:45.728286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:45.728291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:45.728300Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 371 [2:7559341234408962907:4619]
2025-10-09T21:44:45.728317Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:45.728339Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:45.728342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.728346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.728349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.728367Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:45.728374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:45.728382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:45.728445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.728452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.728455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.728458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:45.728529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:45.728537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.728550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.728566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.728586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:45.728612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341234408962902:4619]
2025-10-09T21:44:45.728620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341234408962902:4619]
2025-10-09T21:44:45.728622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341234408962902:4619]
2025-10-09T21:44:45.728631Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:45.728643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:45.728646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:45.728649Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:45.728650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341234770774235:5002], now have 1 active actors on pipe
2025-10-09T21:44:45.728654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341234770774081:2432], now have 1 active actors on pipe
2025-10-09T21:44:45.728662Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:45.728676Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:45.728685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:45.728731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341234408962902:4619]
2025-10-09T21:44:45.728745Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:45.728766Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:45.728799Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 371
2025-10-09T21:44:45.728792Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 371
2025-10-09T21:44:45.728918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.728931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.728975Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8e3b33bd-396e1935-c782cef7-c79a5243_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:45.728986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.728990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.728993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:45.729019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.729030Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:45.729047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:45.729296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.729309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.729333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:45.729467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 372 partition: 0 MaxSeqNo: 25 sessionId: src_id|8e3b33bd-396e1935-c782cef7-c79a5243_0
2025-10-09T21:44:45.729889Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0b79327-ebdb1fb4-337f7623-8cfc0679_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8e3b33bd-396e1935-c782cef7-c79a5243_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:45.734348Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3b33bd-396e1935-c782cef7-c79a5243_0] Write session: try to update token
2025-10-09T21:44:45.734368Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3b33bd-396e1935-c782cef7-c79a5243_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:45.743630Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:45.743650Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:45.743661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341234408962902:4619]
2025-10-09T21:44:45.743710Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:45.743723Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:45.743879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:45.744051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:45.744198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:45.744342Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:45.744548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:45.744602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:45.749686Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 372 sessionId: src_id|8e3b33bd-396e1935-c782cef7-c79a5243_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:45.756101Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:45.756352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.756376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.756434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:45.756561Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:45.764810Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:45.764827Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:45.764829Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:45.764832Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:45.764843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341234408962902:4619]
2025-10-09T21:44:45.764847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341234408962902:4619]
2025-10-09T21:44:45.764854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341234408962902:4619]
2025-10-09T21:44:45.764902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:45.764909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.764912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:45.764911Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:45.764920Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:45.764926Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:45.765070Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:45.789457Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341234770774282:4549]
2025-10-09T21:44:45.789791Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 371, pipe: [1:7559341234770774282:4549]
2025-10-09T21:44:45.927715Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:45.936481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.936505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.936669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.936729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.936791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.936853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.936902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.936910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.936980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.937040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.937100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.937167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.937206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.937209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.937275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.937331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.937390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.937452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.937493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.937495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.937565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.937628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.937693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.937755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.937796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.937798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.937868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.937926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.937988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.938343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.938388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.938391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.938477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.938544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.938603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.938662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.938813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.938816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.938914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.938990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.939071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.939133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.939172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.939175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.939239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.939296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.939357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.939415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.939454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.939459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.939520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.939575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.939634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.939694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.939823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.939827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.939894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:45.939953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:45.940013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:45.940074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:45.940114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:45.940118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:45.940184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341234770774282:4549], now have 1 active actors on pipe
2025-10-09T21:44:45.942237Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341234770774282:4549]
2025-10-09T21:44:45.942281Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:45.942329Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:45.942341Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:45.942369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 372 reason: error at write: Disconnected sessionId: src_id|8e3b33bd-396e1935-c782cef7-c79a5243_0
2025-10-09T21:44:45.942563Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 372 sessionId: src_id|8e3b33bd-396e1935-c782cef7-c79a5243_0 is DEAD
2025-10-09T21:44:45.941848Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:45.946140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3b33bd-396e1935-c782cef7-c79a5243_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:45.946152Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3b33bd-396e1935-c782cef7-c79a5243_0] Write session will restart in 0.008620s
2025-10-09T21:44:45.946184Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3b33bd-396e1935-c782cef7-c79a5243_0] Write session: Do CDS request
2025-10-09T21:44:45.946191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3b33bd-396e1935-c782cef7-c79a5243_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:45.955684Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3b33bd-396e1935-c782cef7-c79a5243_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:45.955829Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:45.955850Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 373
2025-10-09T21:44:45.956303Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 373 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:45.956362Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 373 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:45.956369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=373 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:45.956372Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:45.959250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 373 sessionId:  describe result for acl check
2025-10-09T21:44:45.959307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:45.959314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:45.959316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:45.959322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341234770774332:5013] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:45.959895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341234770774332:5013] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:45.967726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341234770774332:5013] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:45.973129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341234770774332:5013] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:45.973149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341234770774332:5013] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:45.973152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341234770774332:5013] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:45.973162Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 373 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:45.975327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:45.978612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:45.978906Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:45.978918Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:45.978931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:45.978989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:45.979036Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341234408963132:4627]
2025-10-09T21:44:45.979666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:45.980051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:45.980105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:45.980456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:45.980513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:45.980519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:45.980521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:45.980522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:45.980524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:45.980525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:45.980526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.980527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.980528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.980533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:45.980544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:45.980548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:45.980552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:45.980555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:45.980559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:45.980562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.980566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.980570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.980576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:45.980584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:45.980589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:45.980595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:45.980599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:45.980603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:45.980606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:45.980612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:45.980617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:45.980621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:45.980625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:45.980630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:45.980636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.980640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.980644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.980649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.980653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.980658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.980685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:45.988787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:45.990001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:45.990019Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:45.990021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:45.990025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:45.990051Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 372 [2:7559341234408963132:4627]
2025-10-09T21:44:45.990065Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:45.990086Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:45.990095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.990098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.990100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.990119Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:45.990129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:45.990136Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:45.990200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.990207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.990208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.990212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:45.990283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:45.990290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:45.990304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:45.990330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:45.990351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:45.990372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341234408963127:4627]
2025-10-09T21:44:45.990379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341234408963127:4627]
2025-10-09T21:44:45.990381Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341234408963127:4627]
2025-10-09T21:44:45.990395Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:45.990415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:45.990435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:45.990438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:45.990478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341234408963127:4627]
2025-10-09T21:44:45.990487Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:45.990494Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:45.990497Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:45.990500Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:45.990507Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:45.990673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341234770774491:5013], now have 1 active actors on pipe
2025-10-09T21:44:45.990686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341234770774305:2432], now have 1 active actors on pipe
2025-10-09T21:44:45.990879Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 372
2025-10-09T21:44:45.990888Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 372
2025-10-09T21:44:45.991029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.991044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.991091Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|18a2f35f-df89150-b0ecba7-1a7433e4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:45.991102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.991107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.991110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:45.991123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.991137Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:45.991156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:45.991505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.991515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.991532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:45.992037Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3b33bd-396e1935-c782cef7-c79a5243_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|18a2f35f-df89150-b0ecba7-1a7433e4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:45.991651Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 373 partition: 0 MaxSeqNo: 25 sessionId: src_id|18a2f35f-df89150-b0ecba7-1a7433e4_0
2025-10-09T21:44:45.995014Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|18a2f35f-df89150-b0ecba7-1a7433e4_0] Write session: try to update token
2025-10-09T21:44:45.995035Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|18a2f35f-df89150-b0ecba7-1a7433e4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:46.010025Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:46.010058Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:46.010061Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:46.010063Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:46.010073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341234408963127:4627]
2025-10-09T21:44:46.010077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341234408963127:4627]
2025-10-09T21:44:46.010082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341234408963127:4627]
2025-10-09T21:44:46.010133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:46.010141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.010143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:46.010186Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:46.010204Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:46.010208Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:46.010287Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:46.021254Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 373 sessionId: src_id|18a2f35f-df89150-b0ecba7-1a7433e4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:46.024372Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:46.045106Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341239065741814:4549]
2025-10-09T21:44:46.045645Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 372, pipe: [1:7559341239065741814:4549]
2025-10-09T21:44:46.189504Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:46.190410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:46.190433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:46.190484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341239065741814:4549], now have 1 active actors on pipe
2025-10-09T21:44:46.190951Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:46.191029Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:46.191045Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:46.191130Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 373 reason: error at write: Disconnected sessionId: src_id|18a2f35f-df89150-b0ecba7-1a7433e4_0
2025-10-09T21:44:46.191327Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341239065741814:4549]
2025-10-09T21:44:46.191521Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:46.191528Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 373 sessionId: src_id|18a2f35f-df89150-b0ecba7-1a7433e4_0 is DEAD
2025-10-09T21:44:46.191866Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18a2f35f-df89150-b0ecba7-1a7433e4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:46.191879Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18a2f35f-df89150-b0ecba7-1a7433e4_0] Write session will restart in 0.005317s
2025-10-09T21:44:46.191915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18a2f35f-df89150-b0ecba7-1a7433e4_0] Write session: Do CDS request
2025-10-09T21:44:46.191922Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18a2f35f-df89150-b0ecba7-1a7433e4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:46.199848Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|18a2f35f-df89150-b0ecba7-1a7433e4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:46.202435Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:46.202458Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 374
2025-10-09T21:44:46.202675Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 374 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:46.202719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 374 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:46.202732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=374 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:46.202736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:46.203306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 374 sessionId:  describe result for acl check
2025-10-09T21:44:46.203356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:46.203363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:46.203364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:46.203374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341239065741892:5018] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:46.203964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341239065741892:5018] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:46.206657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341239065741892:5018] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:46.210825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341239065741892:5018] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:46.210852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341239065741892:5018] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:46.210856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341239065741892:5018] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:46.210866Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 374 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:46.213310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:46.217234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:46.217507Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:46.217520Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:46.217532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:46.217601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:46.217651Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341238703930633:4636]
2025-10-09T21:44:46.218329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:46.218809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:46.218889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:46.219360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:46.219430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:46.219437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:46.219438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:46.219440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:46.219442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:46.219443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:46.219444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.219445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.219446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.219452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:46.219463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:46.219467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:46.219471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:46.219475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:46.219480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:46.219484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.219487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.219491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.219499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:46.219507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:46.219514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:46.219520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:46.219525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:46.219531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:46.219536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:46.219552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:46.219558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:46.219570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:46.219576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:46.219581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:46.219591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.219596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.219600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.219607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.219615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.219620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.219650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:46.228744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:46.229980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:46.229995Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:46.229999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:46.230004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:46.230012Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 373 [2:7559341238703930633:4636]
2025-10-09T21:44:46.230042Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:46.230067Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:46.230070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.230074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.230076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.230097Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:46.230112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:46.230120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:46.230190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.230198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.230201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.230204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:46.230283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:46.230293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.230308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.230333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.230350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:46.230378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341238703930628:4636]
2025-10-09T21:44:46.230388Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341238703930628:4636]
2025-10-09T21:44:46.230391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341238703930628:4636]
2025-10-09T21:44:46.230407Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:46.230430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341239065742007:5018], now have 1 active actors on pipe
2025-10-09T21:44:46.230433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341239065741819:2432], now have 1 active actors on pipe
2025-10-09T21:44:46.230443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:46.230458Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:46.230462Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:46.230523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341238703930628:4636]
2025-10-09T21:44:46.230534Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:46.230522Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 373
2025-10-09T21:44:46.230523Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 373
2025-10-09T21:44:46.230570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:46.230580Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:46.230582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:46.230589Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:46.230796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:46.230811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:46.230854Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|94232da7-ddf8461b-7f5a5505-1a419398_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:46.230870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.230875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.230878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:46.230907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.230925Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:46.230955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:46.231259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:46.231271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:46.231290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:46.231416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 374 partition: 0 MaxSeqNo: 25 sessionId: src_id|94232da7-ddf8461b-7f5a5505-1a419398_0
2025-10-09T21:44:46.231882Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18a2f35f-df89150-b0ecba7-1a7433e4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|94232da7-ddf8461b-7f5a5505-1a419398_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:46.234700Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94232da7-ddf8461b-7f5a5505-1a419398_0] Write session: try to update token
2025-10-09T21:44:46.234723Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94232da7-ddf8461b-7f5a5505-1a419398_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:46.251572Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:46.251588Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:46.251591Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:46.251594Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:46.251606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341238703930628:4636]
2025-10-09T21:44:46.251611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341238703930628:4636]
2025-10-09T21:44:46.251615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341238703930628:4636]
2025-10-09T21:44:46.251661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:46.251665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:46.251670Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:46.251673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.251674Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:46.251676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:46.251815Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:46.254634Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 374 sessionId: src_id|94232da7-ddf8461b-7f5a5505-1a419398_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:46.257277Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:46.291461Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341239065742022:4549]
2025-10-09T21:44:46.291776Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 373, pipe: [1:7559341239065742022:4549]
2025-10-09T21:44:46.416267Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:46.416540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:46.416558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:46.416602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341239065742022:4549], now have 1 active actors on pipe
2025-10-09T21:44:46.416807Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:46.416879Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341239065742022:4549]
2025-10-09T21:44:46.416904Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:46.416907Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:46.417515Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94232da7-ddf8461b-7f5a5505-1a419398_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:46.417556Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94232da7-ddf8461b-7f5a5505-1a419398_0] Write session will restart in 0.005214s
2025-10-09T21:44:46.417580Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94232da7-ddf8461b-7f5a5505-1a419398_0] Write session: Do CDS request
2025-10-09T21:44:46.417586Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94232da7-ddf8461b-7f5a5505-1a419398_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:46.417552Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:46.417075Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 374 reason: error at write: Disconnected sessionId: src_id|94232da7-ddf8461b-7f5a5505-1a419398_0
2025-10-09T21:44:46.417341Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 374 sessionId: src_id|94232da7-ddf8461b-7f5a5505-1a419398_0 is DEAD
2025-10-09T21:44:46.423647Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94232da7-ddf8461b-7f5a5505-1a419398_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:46.423804Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:46.423821Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 375
2025-10-09T21:44:46.424000Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 375 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:46.424059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 375 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:46.424071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=375 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:46.424076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:46.424530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 375 sessionId:  describe result for acl check
2025-10-09T21:44:46.424568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:46.424576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:46.424581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:46.424586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341239065742038:5022] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:46.425167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341239065742038:5022] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:46.428067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341239065742038:5022] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:46.431888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341239065742038:5022] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:46.431904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341239065742038:5022] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:46.431908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341239065742038:5022] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:46.431917Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 375 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:46.435803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:46.438651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:46.438897Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:46.438909Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:46.438922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:46.438972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:46.439024Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341238703930832:4639]
2025-10-09T21:44:46.439555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:46.439883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:46.439921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:46.440216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:46.440261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:46.440262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:46.440263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:46.440264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:46.440265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:46.440266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:46.440266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.440267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.440268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.440273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:46.440283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:46.440286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:46.440289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:46.440292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:46.440295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:46.440299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.440302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.440305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.440312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:46.440319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:46.440325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:46.440329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:46.440333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:46.440338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:46.440342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:46.440346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:46.440350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:46.440354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:46.440358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:46.440362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:46.440367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.440371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.440377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.440382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.440386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.440391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.440415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:46.447502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:46.448611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:46.448630Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:46.448633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:46.448637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:46.448645Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 374 [2:7559341238703930832:4639]
2025-10-09T21:44:46.448660Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:46.448682Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:46.448706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.448715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.448718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.448737Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:46.448743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:46.448751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:46.448814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.448822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.448824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.448828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:46.448901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:46.448909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.448923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.448935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.448952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:46.448973Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341238703930827:4639]
2025-10-09T21:44:46.448980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341238703930827:4639]
2025-10-09T21:44:46.448983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341238703930827:4639]
2025-10-09T21:44:46.448994Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:46.449005Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:46.449010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:46.449014Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:46.449016Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:46.449025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:46.449029Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:46.449041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341239065742025:2432], now have 1 active actors on pipe
2025-10-09T21:44:46.449052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341239065742174:5022], now have 1 active actors on pipe
2025-10-09T21:44:46.449087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341238703930827:4639]
2025-10-09T21:44:46.449100Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:46.449124Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:46.449119Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 374
2025-10-09T21:44:46.449180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 374
2025-10-09T21:44:46.449356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:46.449370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:46.449401Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|728ae472-f96154ee-a8f6bbd8-dadf1876_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:46.449415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.449419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.449423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:46.449433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.449453Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:46.449479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:46.449708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:46.449731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:46.449755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:46.449874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 375 partition: 0 MaxSeqNo: 25 sessionId: src_id|728ae472-f96154ee-a8f6bbd8-dadf1876_0
2025-10-09T21:44:46.450304Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94232da7-ddf8461b-7f5a5505-1a419398_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|728ae472-f96154ee-a8f6bbd8-dadf1876_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:46.452881Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|728ae472-f96154ee-a8f6bbd8-dadf1876_0] Write session: try to update token
2025-10-09T21:44:46.452901Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|728ae472-f96154ee-a8f6bbd8-dadf1876_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:46.465866Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:46.465886Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:46.465888Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:46.465890Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:46.465901Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341238703930827:4639]
2025-10-09T21:44:46.465906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341238703930827:4639]
2025-10-09T21:44:46.465910Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341238703930827:4639]
2025-10-09T21:44:46.465954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:46.465960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.465963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:46.465965Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:46.465981Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:46.465984Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:46.466121Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:46.467411Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 375 sessionId: src_id|728ae472-f96154ee-a8f6bbd8-dadf1876_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:46.470043Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:46.517269Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341239065742230:4549]
2025-10-09T21:44:46.517649Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 374, pipe: [1:7559341239065742230:4549]
2025-10-09T21:44:46.624937Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:46.625358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:46.625379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:46.625431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341239065742230:4549], now have 1 active actors on pipe
2025-10-09T21:44:46.625664Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:46.626233Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|728ae472-f96154ee-a8f6bbd8-dadf1876_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:46.626245Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|728ae472-f96154ee-a8f6bbd8-dadf1876_0] Write session will restart in 0.008637s
2025-10-09T21:44:46.626268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|728ae472-f96154ee-a8f6bbd8-dadf1876_0] Write session: Do CDS request
2025-10-09T21:44:46.626283Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|728ae472-f96154ee-a8f6bbd8-dadf1876_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:46.625765Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341239065742230:4549]
2025-10-09T21:44:46.625770Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:46.625819Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:46.625822Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:46.625904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 375 reason: error at write: Disconnected sessionId: src_id|728ae472-f96154ee-a8f6bbd8-dadf1876_0
2025-10-09T21:44:46.626112Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 375 sessionId: src_id|728ae472-f96154ee-a8f6bbd8-dadf1876_0 is DEAD
2025-10-09T21:44:46.636845Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|728ae472-f96154ee-a8f6bbd8-dadf1876_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:46.636856Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:46.636887Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 376
2025-10-09T21:44:46.637223Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 376 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:46.637270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 376 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:46.637281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=376 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:46.637285Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:46.637869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 376 sessionId:  describe result for acl check
2025-10-09T21:44:46.637909Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:46.637912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:46.637914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:46.637920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341239065742246:5027] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:46.638542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341239065742246:5027] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:46.640851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341239065742246:5027] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:46.647296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341239065742246:5027] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:46.647316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341239065742246:5027] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:46.647321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341239065742246:5027] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:46.647331Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 376 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:46.658954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:46.660187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:46.660452Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:46.660461Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:46.660475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:46.660536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:46.660592Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341238703931029:4641]
2025-10-09T21:44:46.661214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:46.662293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:46.662368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:46.662699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:46.662766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:46.662773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:46.662774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:46.662775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:46.662777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:46.662778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:46.662779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.662780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.662782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.662788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:46.662799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:46.662804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:46.662808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:46.662812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:46.662817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:46.662821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.662825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.662829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.662838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:46.662847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:46.662869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:46.662879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:46.662884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:46.662888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:46.662892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:46.662895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:46.662900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:46.662904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:46.662908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:46.662912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:46.662918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.662923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.662927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.662931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.662935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.662940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.662967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:46.669718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:46.671495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:46.671513Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:46.671515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:46.671520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:46.671527Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 375 [2:7559341238703931029:4641]
2025-10-09T21:44:46.671544Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:46.671564Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:46.671567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.671569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.671571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.671587Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:46.671594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:46.671606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:46.671687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.671689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.671691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.671694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:46.671780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:46.671783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.671795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.671804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.671820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:46.671841Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341238703931023:4641]
2025-10-09T21:44:46.671843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341238703931023:4641]
2025-10-09T21:44:46.671845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341238703931023:4641]
2025-10-09T21:44:46.671855Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:46.671864Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:46.671869Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:46.671871Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:46.671878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341239065742233:2432], now have 1 active actors on pipe
2025-10-09T21:44:46.671882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341239065742300:5027], now have 1 active actors on pipe
2025-10-09T21:44:46.671892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:46.671907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:46.671910Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:46.671948Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341238703931023:4641]
2025-10-09T21:44:46.671965Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:46.671978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:46.671955Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 375
2025-10-09T21:44:46.672019Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 375
2025-10-09T21:44:46.672203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:46.672229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:46.672260Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4daf391-a03a0a08-2364fb9e-dd672d0f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:46.672274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.672279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.672282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:46.672294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.672307Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:46.672341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:46.672546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:46.672578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:46.672604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:46.672719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 376 partition: 0 MaxSeqNo: 25 sessionId: src_id|4daf391-a03a0a08-2364fb9e-dd672d0f_0
2025-10-09T21:44:46.673124Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|728ae472-f96154ee-a8f6bbd8-dadf1876_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4daf391-a03a0a08-2364fb9e-dd672d0f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:46.675876Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4daf391-a03a0a08-2364fb9e-dd672d0f_0] Write session: try to update token
2025-10-09T21:44:46.675906Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4daf391-a03a0a08-2364fb9e-dd672d0f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:46.693247Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 376 sessionId: src_id|4daf391-a03a0a08-2364fb9e-dd672d0f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:46.692407Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:46.692433Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:46.692437Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:46.692441Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:46.692455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341238703931023:4641]
2025-10-09T21:44:46.692463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341238703931023:4641]
2025-10-09T21:44:46.692471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341238703931023:4641]
2025-10-09T21:44:46.692531Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:46.692537Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:46.692541Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:46.692542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:46.692551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.692554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:46.692773Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:46.695737Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:46.726128Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341239065742441:4549]
2025-10-09T21:44:46.726475Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 375, pipe: [1:7559341239065742441:4549]
2025-10-09T21:44:46.857615Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:46.857908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:46.857924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:46.857972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341239065742441:4549], now have 1 active actors on pipe
2025-10-09T21:44:46.858214Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:46.858281Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:46.858348Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341239065742441:4549]
2025-10-09T21:44:46.858367Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:46.858871Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4daf391-a03a0a08-2364fb9e-dd672d0f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:46.858888Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4daf391-a03a0a08-2364fb9e-dd672d0f_0] Write session will restart in 0.009562s
2025-10-09T21:44:46.858449Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 376 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4daf391-a03a0a08-2364fb9e-dd672d0f_0
2025-10-09T21:44:46.858912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4daf391-a03a0a08-2364fb9e-dd672d0f_0] Write session: Do CDS request
2025-10-09T21:44:46.858920Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4daf391-a03a0a08-2364fb9e-dd672d0f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:46.858648Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 376 sessionId: src_id|4daf391-a03a0a08-2364fb9e-dd672d0f_0 is DEAD
2025-10-09T21:44:46.869694Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4daf391-a03a0a08-2364fb9e-dd672d0f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:46.874882Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:46.874901Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 377
2025-10-09T21:44:46.875194Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 377 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:46.875232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 377 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:46.875239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=377 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:46.875243Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:46.875951Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 377 sessionId:  describe result for acl check
2025-10-09T21:44:46.875998Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:46.876001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:46.876004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:46.876011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341239065742490:5033] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:46.876659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341239065742490:5033] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:46.882607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341239065742490:5033] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:46.890064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:46.889005Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341239065742490:5033] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:46.889028Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341239065742490:5033] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:46.889032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341239065742490:5033] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:46.889042Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 377 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:46.893214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:46.893438Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:46.893442Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:46.893454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:46.893507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:46.893547Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341238703931228:4643]
2025-10-09T21:44:46.894168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:46.895220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:46.895288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:46.895630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:46.895675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:46.895676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:46.895678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:46.895679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:46.895680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:46.895681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:46.895683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.895684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.895685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.895690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:46.895702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:46.895706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:46.895710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:46.895714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:46.895719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:46.895722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.895726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.895730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.895738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:46.895746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:46.895753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:46.895758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:46.895762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:46.895767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:46.895772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:46.895777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:46.895781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:46.895786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:46.895791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:46.895796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:46.895802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.895808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.895813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.895818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.895823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.895828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.895857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:46.904165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:46.908891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4daf391-a03a0a08-2364fb9e-dd672d0f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|96df3b45-8fc19d95-e0f0f6cd-74ad1efd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:46.906078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:46.906093Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:46.906095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:46.906100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:46.906109Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 376 [2:7559341238703931228:4643]
2025-10-09T21:44:46.906123Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:46.907438Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 376
2025-10-09T21:44:46.907439Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 376
2025-10-09T21:44:46.908340Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 377 partition: 0 MaxSeqNo: 25 sessionId: src_id|96df3b45-8fc19d95-e0f0f6cd-74ad1efd_0
2025-10-09T21:44:46.906145Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:46.906149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.906153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.906155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.906175Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:46.906181Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:46.906191Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:46.906275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.906276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.906278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.906281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:46.906372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:46.906375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.906390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:46.906401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:46.906416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:46.906438Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341238703931222:4643]
2025-10-09T21:44:46.906441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341238703931222:4643]
2025-10-09T21:44:46.906443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341238703931222:4643]
2025-10-09T21:44:46.906461Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:46.906477Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:46.906823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:46.906833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:46.906892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341238703931222:4643]
2025-10-09T21:44:46.906910Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:46.907070Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:46.907073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:46.907076Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:46.907083Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:46.907354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341239065742445:2432], now have 1 active actors on pipe
2025-10-09T21:44:46.907360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341239065742637:5033], now have 1 active actors on pipe
2025-10-09T21:44:46.907694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:46.907711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:46.907744Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|96df3b45-8fc19d95-e0f0f6cd-74ad1efd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:46.907756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.907770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.907773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:46.907787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.907803Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:46.907823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:46.908107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:46.908112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:46.908139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:46.911544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96df3b45-8fc19d95-e0f0f6cd-74ad1efd_0] Write session: try to update token
2025-10-09T21:44:46.911567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96df3b45-8fc19d95-e0f0f6cd-74ad1efd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:46.923437Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:46.923453Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:46.923463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341238703931222:4643]
2025-10-09T21:44:46.923511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:46.923699Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:46.923835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:46.923982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:46.924118Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:46.924299Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:46.924343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:46.924733Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:46.933258Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 377 sessionId: src_id|96df3b45-8fc19d95-e0f0f6cd-74ad1efd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:46.936072Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:46.936803Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:46.936580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:46.936600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:46.936644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:46.943492Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:46.943513Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:46.943520Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:46.943522Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:46.943533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341238703931222:4643]
2025-10-09T21:44:46.943538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341238703931222:4643]
2025-10-09T21:44:46.943541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341238703931222:4643]
2025-10-09T21:44:46.943586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:46.943588Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:46.943593Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:46.943593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:46.943596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:46.943596Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:46.943737Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:46.958582Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341239065742656:4549]
2025-10-09T21:44:46.959597Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 376, pipe: [1:7559341239065742656:4549]
2025-10-09T21:44:47.165008Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:47.173615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:47.173651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:47.173825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:47.173881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:47.173941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:47.173997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:47.177929Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341239065742656:4549]
2025-10-09T21:44:47.177962Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:47.174143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:47.174154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:47.174228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:47.174289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:47.174368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:47.174442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:47.174483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:47.174490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:47.174563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:47.174620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:47.174682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:47.174743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:47.174785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:47.174789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:47.174865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:47.174932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:47.174995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:47.175058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:47.175099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:47.175102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:47.175202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:47.175259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:47.175321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:47.175434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:47.175479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:47.175483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:47.175551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:47.175613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:47.175670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:47.175733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:47.175776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:47.175779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:47.175850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:47.175907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:47.175977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:47.176040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:47.176101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:47.176105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:47.176170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:47.176228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:47.176289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:47.176349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:47.176410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:47.176413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:47.176485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:47.176542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:47.176602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:47.176664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:47.176708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:47.176710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:47.176776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:47.176837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:47.176897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:47.176961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:47.177089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:47.177092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:47.177175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341239065742656:4549], now have 1 active actors on pipe
2025-10-09T21:44:47.177796Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:47.178455Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96df3b45-8fc19d95-e0f0f6cd-74ad1efd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:47.178468Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96df3b45-8fc19d95-e0f0f6cd-74ad1efd_0] Write session will restart in 0.005534s
2025-10-09T21:44:47.178492Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96df3b45-8fc19d95-e0f0f6cd-74ad1efd_0] Write session: Do CDS request
2025-10-09T21:44:47.178498Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96df3b45-8fc19d95-e0f0f6cd-74ad1efd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:47.178002Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:47.178011Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:47.178053Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 377 reason: error at write: Disconnected sessionId: src_id|96df3b45-8fc19d95-e0f0f6cd-74ad1efd_0
2025-10-09T21:44:47.178258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 377 sessionId: src_id|96df3b45-8fc19d95-e0f0f6cd-74ad1efd_0 is DEAD
2025-10-09T21:44:47.185600Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96df3b45-8fc19d95-e0f0f6cd-74ad1efd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:47.186205Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:47.186224Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 378
2025-10-09T21:44:47.190169Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 378 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:47.190221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 378 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:47.190227Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=378 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:47.190231Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:47.191057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 378 sessionId:  describe result for acl check
2025-10-09T21:44:47.191094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:47.191096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:47.191098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:47.191104Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341243360710038:5045] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:47.191635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341243360710038:5045] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:47.197520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341243360710038:5045] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:47.201982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341243360710038:5045] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:47.202001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341243360710038:5045] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:47.202005Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341243360710038:5045] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:47.202015Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 378 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:47.204970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:47.208803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:47.209064Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:47.209075Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:47.209090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:47.209158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:47.209206Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341242998898763:4658]
2025-10-09T21:44:47.209805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:47.210208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:47.210279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:47.210656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:47.210722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:47.210724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:47.210725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:47.210726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:47.210727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:47.210729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:47.210730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.210731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.210732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.210738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:47.210749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:47.210754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:47.210758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:47.210761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:47.210766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:47.210769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.210773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.210777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.210787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:47.210795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:47.210802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:47.210807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:47.210812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:47.210816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:47.210820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:47.210827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:47.210834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:47.210839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:47.210844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:47.210850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:47.210856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.210860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.210864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.210869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.210874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.210879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.210926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:47.219646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:47.221373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:47.221392Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:47.221395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:47.221400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:47.221408Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 377 [2:7559341242998898763:4658]
2025-10-09T21:44:47.221426Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:47.221447Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:47.221450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.221454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.221456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.221476Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:47.221485Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:47.221493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:47.221565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.221575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.221577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.221579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:47.221659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:47.221668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.221682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.221698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.221719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:47.221745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341242998898758:4658]
2025-10-09T21:44:47.221747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341242998898758:4658]
2025-10-09T21:44:47.221750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341242998898758:4658]
2025-10-09T21:44:47.221765Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:47.221781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:47.221797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:47.221806Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:47.221838Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:47.221858Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:47.221863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341242998898758:4658]
2025-10-09T21:44:47.221866Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:47.222167Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 377
2025-10-09T21:44:47.221866Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:47.221874Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:47.222208Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 377
2025-10-09T21:44:47.222521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341243360710175:5045], now have 1 active actors on pipe
2025-10-09T21:44:47.222535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341243360709982:2432], now have 1 active actors on pipe
2025-10-09T21:44:47.222569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:47.222580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:47.222615Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7ff55e11-be733c54-a044a5c9-61dbeb71_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:47.222630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.222636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.222638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:47.222650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.222663Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:47.222687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:47.223040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:47.223051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:47.223072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:47.223178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 378 partition: 0 MaxSeqNo: 25 sessionId: src_id|7ff55e11-be733c54-a044a5c9-61dbeb71_0
2025-10-09T21:44:47.223604Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96df3b45-8fc19d95-e0f0f6cd-74ad1efd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7ff55e11-be733c54-a044a5c9-61dbeb71_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:47.226196Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ff55e11-be733c54-a044a5c9-61dbeb71_0] Write session: try to update token
2025-10-09T21:44:47.226218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ff55e11-be733c54-a044a5c9-61dbeb71_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:47.241057Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 378 sessionId: src_id|7ff55e11-be733c54-a044a5c9-61dbeb71_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:47.243738Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:47.244580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:47.244608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:47.244631Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:47.244636Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:47.244639Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:47.244641Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:47.244651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341242998898758:4658]
2025-10-09T21:44:47.244660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341242998898758:4658]
2025-10-09T21:44:47.244664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341242998898758:4658]
2025-10-09T21:44:47.244719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:47.244720Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:47.244726Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:47.244730Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:47.244732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.244736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:47.244883Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:47.278074Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341243360710190:4549]
2025-10-09T21:44:47.278552Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 377, pipe: [1:7559341243360710190:4549]
2025-10-09T21:44:47.458832Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:47.459436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:47.459512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341243360710190:4549], now have 1 active actors on pipe
2025-10-09T21:44:47.459708Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:47.460219Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:47.460267Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341243360710190:4549]
2025-10-09T21:44:47.473360Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:47.473378Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:47.473404Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 378 reason: error at write: Disconnected sessionId: src_id|7ff55e11-be733c54-a044a5c9-61dbeb71_0
2025-10-09T21:44:47.473562Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 378 sessionId: src_id|7ff55e11-be733c54-a044a5c9-61dbeb71_0 is DEAD
2025-10-09T21:44:47.473840Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ff55e11-be733c54-a044a5c9-61dbeb71_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:47.473865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ff55e11-be733c54-a044a5c9-61dbeb71_0] Write session will restart in 0.009441s
2025-10-09T21:44:47.473891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ff55e11-be733c54-a044a5c9-61dbeb71_0] Write session: Do CDS request
2025-10-09T21:44:47.473901Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ff55e11-be733c54-a044a5c9-61dbeb71_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:47.484601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ff55e11-be733c54-a044a5c9-61dbeb71_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:47.485014Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:47.485033Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 379
2025-10-09T21:44:47.486415Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 379 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:47.486479Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 379 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:47.486486Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=379 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:47.486510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:47.486998Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 379 sessionId:  describe result for acl check
2025-10-09T21:44:47.487043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:47.487049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:47.487051Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:47.487057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341243360710349:5049] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:47.487629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341243360710349:5049] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:47.489914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:47.491113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341243360710349:5049] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:47.492450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:47.492653Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:47.492657Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:47.492669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:47.492729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:47.492784Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341242998898965:4661]
2025-10-09T21:44:47.493344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:47.493975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:47.494069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:47.494432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:47.494495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:47.494501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:47.494502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:47.494503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:47.494504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:47.494505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:47.494506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.494508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.494509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.494515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:47.494525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:47.494528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:47.494533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:47.494536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:47.494540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:47.494544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.494547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.494551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.494559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:47.494567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:47.494572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:47.494577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:47.494581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:47.494586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:47.494589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:47.494594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:47.494598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:47.494602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:47.494606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:47.494611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:47.494617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.494622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.494626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.494630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.494635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.494640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.494667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:47.495720Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341243360710349:5049] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:47.495740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341243360710349:5049] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:47.495744Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341243360710349:5049] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:47.495754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 379 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:47.504345Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 378
2025-10-09T21:44:47.504406Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 378
2025-10-09T21:44:47.502290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:47.503899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:47.503905Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:47.503908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:47.503912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:47.503921Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 378 [2:7559341242998898965:4661]
2025-10-09T21:44:47.503940Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:47.503961Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:47.503965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.503969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.503972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.503992Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:47.503998Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:47.504006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:47.504069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.504072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.504073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.504076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:47.504151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:47.504155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.504171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.504182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.504199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:47.504222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341243360710193:2432], now have 1 active actors on pipe
2025-10-09T21:44:47.504226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341243360710392:5049], now have 1 active actors on pipe
2025-10-09T21:44:47.504235Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341242998898960:4661]
2025-10-09T21:44:47.504237Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341242998898960:4661]
2025-10-09T21:44:47.504239Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341242998898960:4661]
2025-10-09T21:44:47.504258Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:47.504273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:47.504289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:47.504292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:47.504333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341242998898960:4661]
2025-10-09T21:44:47.504335Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:47.504363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:47.504367Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:47.504370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:47.504376Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:47.505029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:47.505040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:47.505088Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bc72da0d-f3e6fa79-c680ed58-3047d7c5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:47.505098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.505103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.505106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:47.505137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.505151Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:47.505168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:47.505498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:47.505510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:47.505528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:47.505623Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 379 partition: 0 MaxSeqNo: 25 sessionId: src_id|bc72da0d-f3e6fa79-c680ed58-3047d7c5_0
2025-10-09T21:44:47.506167Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ff55e11-be733c54-a044a5c9-61dbeb71_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bc72da0d-f3e6fa79-c680ed58-3047d7c5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:47.508758Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc72da0d-f3e6fa79-c680ed58-3047d7c5_0] Write session: try to update token
2025-10-09T21:44:47.508779Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc72da0d-f3e6fa79-c680ed58-3047d7c5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:47.533772Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 379 sessionId: src_id|bc72da0d-f3e6fa79-c680ed58-3047d7c5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:47.536396Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:47.537960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:47.537985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:47.538008Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:47.538011Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:47.538013Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:47.538015Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:47.538024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341242998898960:4661]
2025-10-09T21:44:47.538047Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341242998898960:4661]
2025-10-09T21:44:47.538051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341242998898960:4661]
2025-10-09T21:44:47.538109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:47.538120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.538124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:47.538325Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:47.541396Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:47.541410Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:47.541414Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:47.559167Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341243360710403:4549]
2025-10-09T21:44:47.559519Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 378, pipe: [1:7559341243360710403:4549]
2025-10-09T21:44:47.698122Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:47.698723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:47.698789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341243360710403:4549], now have 1 active actors on pipe
2025-10-09T21:44:47.698990Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:47.700686Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341243360710403:4549]
2025-10-09T21:44:47.700736Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:47.713694Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc72da0d-f3e6fa79-c680ed58-3047d7c5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:47.713709Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc72da0d-f3e6fa79-c680ed58-3047d7c5_0] Write session will restart in 0.005151s
2025-10-09T21:44:47.713746Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc72da0d-f3e6fa79-c680ed58-3047d7c5_0] Write session: Do CDS request
2025-10-09T21:44:47.713753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc72da0d-f3e6fa79-c680ed58-3047d7c5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:47.712917Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:47.712936Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:47.712962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 379 reason: error at write: Disconnected sessionId: src_id|bc72da0d-f3e6fa79-c680ed58-3047d7c5_0
2025-10-09T21:44:47.713115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 379 sessionId: src_id|bc72da0d-f3e6fa79-c680ed58-3047d7c5_0 is DEAD
2025-10-09T21:44:47.720569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc72da0d-f3e6fa79-c680ed58-3047d7c5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:47.726241Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:47.726261Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 380
2025-10-09T21:44:47.726586Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 380 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:47.726675Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 380 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:47.726682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=380 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:47.726687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:47.727292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 380 sessionId:  describe result for acl check
2025-10-09T21:44:47.727331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:47.727337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:47.727339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:47.727347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341243360710465:5054] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:47.727988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341243360710465:5054] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:47.732578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341243360710465:5054] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:47.739529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:47.740079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341243360710465:5054] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:47.740112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341243360710465:5054] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:47.740116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341243360710465:5054] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:47.740126Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 380 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:47.744250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:47.744508Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:47.744518Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:47.744533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:47.744610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:47.744658Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341242998899167:4663]
2025-10-09T21:44:47.745270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:47.745784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:47.745844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:47.746284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:47.746378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:47.746389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:47.746391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:47.746392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:47.746393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:47.746394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:47.746395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.746396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.746397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.746404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:47.746426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:47.746430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:47.746434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:47.746438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:47.746442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:47.746445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.746449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.746453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.746461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:47.746470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:47.746476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:47.746482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:47.746487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:47.746492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:47.746496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:47.746501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:47.746507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:47.746538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:47.746543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:47.746549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:47.746555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.746560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.746573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.746579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.746584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.746589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.746620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:47.756129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:47.757547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:47.757561Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:47.757563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:47.757568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:47.757576Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 379 [2:7559341242998899167:4663]
2025-10-09T21:44:47.757591Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:47.757612Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:47.757615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.757618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.757621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.757642Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:47.757656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:47.757663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:47.757733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.757739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.757741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.757745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:47.757821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:47.757831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.757843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.757853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.757874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:47.757899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341243360710405:2432], now have 1 active actors on pipe
2025-10-09T21:44:47.757907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341243360710601:5054], now have 1 active actors on pipe
2025-10-09T21:44:47.757915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341242998899160:4663]
2025-10-09T21:44:47.757917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341242998899160:4663]
2025-10-09T21:44:47.757919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341242998899160:4663]
2025-10-09T21:44:47.757930Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:47.757948Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:47.757968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:47.757971Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:47.758013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341242998899160:4663]
2025-10-09T21:44:47.758020Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:47.758012Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 379
2025-10-09T21:44:47.758076Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 379
2025-10-09T21:44:47.758189Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:47.758199Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:47.758224Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:47.758237Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:47.758316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:47.758349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:47.758380Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3eab8052-a0bd35b9-41c65a2f-9cdec2cc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:47.758416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.758426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.758429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:47.758440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.758459Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:47.758482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:47.760034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:47.760047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:47.760073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:47.760243Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 380 partition: 0 MaxSeqNo: 25 sessionId: src_id|3eab8052-a0bd35b9-41c65a2f-9cdec2cc_0
2025-10-09T21:44:47.760651Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc72da0d-f3e6fa79-c680ed58-3047d7c5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3eab8052-a0bd35b9-41c65a2f-9cdec2cc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:47.763280Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3eab8052-a0bd35b9-41c65a2f-9cdec2cc_0] Write session: try to update token
2025-10-09T21:44:47.763302Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3eab8052-a0bd35b9-41c65a2f-9cdec2cc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:47.776950Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:47.776969Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:47.776971Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:47.776974Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:47.776984Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341242998899160:4663]
2025-10-09T21:44:47.776990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341242998899160:4663]
2025-10-09T21:44:47.776994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341242998899160:4663]
2025-10-09T21:44:47.777042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:47.777050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.777051Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:47.777053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:47.777058Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:47.777062Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:47.777192Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:47.792096Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 380 sessionId: src_id|3eab8052-a0bd35b9-41c65a2f-9cdec2cc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:47.794773Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:47.798240Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341243360710614:4549]
2025-10-09T21:44:47.799015Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 379, pipe: [1:7559341243360710614:4549]
2025-10-09T21:44:47.933794Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:47.934387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:47.934402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:47.934424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341243360710614:4549], now have 1 active actors on pipe
2025-10-09T21:44:47.934709Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:47.934747Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:47.934828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 380 reason: error at write: Disconnected sessionId: src_id|3eab8052-a0bd35b9-41c65a2f-9cdec2cc_0
2025-10-09T21:44:47.935012Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341243360710614:4549]
2025-10-09T21:44:47.935038Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:47.935043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 380 sessionId: src_id|3eab8052-a0bd35b9-41c65a2f-9cdec2cc_0 is DEAD
2025-10-09T21:44:47.935403Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:47.938144Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3eab8052-a0bd35b9-41c65a2f-9cdec2cc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:47.938157Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3eab8052-a0bd35b9-41c65a2f-9cdec2cc_0] Write session will restart in 0.008944s
2025-10-09T21:44:47.938190Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3eab8052-a0bd35b9-41c65a2f-9cdec2cc_0] Write session: Do CDS request
2025-10-09T21:44:47.938197Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3eab8052-a0bd35b9-41c65a2f-9cdec2cc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:47.948576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3eab8052-a0bd35b9-41c65a2f-9cdec2cc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:47.948707Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:47.948725Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 381
2025-10-09T21:44:47.948874Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 381 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:47.948906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 381 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:47.948912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=381 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:47.948915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:47.949580Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 381 sessionId:  describe result for acl check
2025-10-09T21:44:47.949619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:47.949626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:47.949628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:47.949634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341243360710703:5059] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:47.950177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341243360710703:5059] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:47.957760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341243360710703:5059] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:47.963504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341243360710703:5059] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:47.963526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341243360710703:5059] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:47.963530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341243360710703:5059] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:47.963540Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 381 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:47.968274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:47.972351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:47.972593Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:47.972603Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:47.972616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:47.972682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:47.972993Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341242998899369:4665]
2025-10-09T21:44:47.973511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:47.973937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:47.974000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:47.974361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:47.974464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:47.974466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:47.974467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:47.974468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:47.974469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:47.974470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:47.974471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.974472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.974474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.974479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:47.974490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:47.974494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:47.974497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:47.974501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:47.974505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:47.974508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.974511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.974514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.974525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:47.974533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:47.974540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:47.974545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:47.974553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:47.974558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:47.974562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:47.974566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:47.974571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:47.974576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:47.974579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:47.974584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:47.974589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.974593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.974597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.974602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.974607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.974612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.974641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:47.983719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:47.984879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:47.984896Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:47.984900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:47.984905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:47.984912Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 380 [2:7559341242998899369:4665]
2025-10-09T21:44:47.984927Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:47.984948Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:47.984951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.984954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.984956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.984973Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:47.984980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:47.984988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:47.985056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.985064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.985066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.985069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:47.985142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:47.985150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:47.985162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:47.985179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:47.985196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:47.985224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341243360710620:2432], now have 1 active actors on pipe
2025-10-09T21:44:47.985238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341242998899364:4665]
2025-10-09T21:44:47.985243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341242998899364:4665]
2025-10-09T21:44:47.985245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341242998899364:4665]
2025-10-09T21:44:47.985260Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:47.985276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341243360710816:5059], now have 1 active actors on pipe
2025-10-09T21:44:47.985276Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:47.985279Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:47.985281Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:47.985289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:47.985302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:47.985310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:47.985347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341242998899364:4665]
2025-10-09T21:44:47.985358Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:47.985356Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 380
2025-10-09T21:44:47.985396Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:47.985402Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 380
2025-10-09T21:44:47.985695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:47.985709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:47.985737Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|febe704-2e33001b-d04bd2ae-cb504c76_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:47.985745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.985749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.985751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:47.985764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.985775Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:47.985793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:47.986044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:47.986056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:47.986074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:47.986764Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3eab8052-a0bd35b9-41c65a2f-9cdec2cc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|febe704-2e33001b-d04bd2ae-cb504c76_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:47.986357Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 381 partition: 0 MaxSeqNo: 25 sessionId: src_id|febe704-2e33001b-d04bd2ae-cb504c76_0
2025-10-09T21:44:47.989163Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|febe704-2e33001b-d04bd2ae-cb504c76_0] Write session: try to update token
2025-10-09T21:44:47.989178Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|febe704-2e33001b-d04bd2ae-cb504c76_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:48.002671Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:48.002690Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:48.002692Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:48.002694Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:48.002705Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341242998899364:4665]
2025-10-09T21:44:48.002709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341242998899364:4665]
2025-10-09T21:44:48.002714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341242998899364:4665]
2025-10-09T21:44:48.002764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:48.002781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.002784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:48.002936Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:48.005910Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 381 sessionId: src_id|febe704-2e33001b-d04bd2ae-cb504c76_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:48.006433Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:48.006450Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:48.006454Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:48.009101Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:48.034264Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341247655678130:4549]
2025-10-09T21:44:48.034712Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 380, pipe: [1:7559341247655678130:4549]
2025-10-09T21:44:48.177470Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:48.177801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:48.177821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:48.177901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341247655678130:4549], now have 1 active actors on pipe
2025-10-09T21:44:48.178197Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:48.178438Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341247655678130:4549]
2025-10-09T21:44:48.178472Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:48.179055Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:48.179060Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:48.179133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 381 reason: error at write: Disconnected sessionId: src_id|febe704-2e33001b-d04bd2ae-cb504c76_0
2025-10-09T21:44:48.180157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 381 sessionId: src_id|febe704-2e33001b-d04bd2ae-cb504c76_0 is DEAD
2025-10-09T21:44:48.182154Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|febe704-2e33001b-d04bd2ae-cb504c76_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:48.182170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|febe704-2e33001b-d04bd2ae-cb504c76_0] Write session will restart in 0.007360s
2025-10-09T21:44:48.182203Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|febe704-2e33001b-d04bd2ae-cb504c76_0] Write session: Do CDS request
2025-10-09T21:44:48.182210Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|febe704-2e33001b-d04bd2ae-cb504c76_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:48.190626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|febe704-2e33001b-d04bd2ae-cb504c76_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:48.190772Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:48.190796Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 382
2025-10-09T21:44:48.190934Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 382 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:48.190970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 382 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:48.190977Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=382 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:48.190981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:48.191537Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 382 sessionId:  describe result for acl check
2025-10-09T21:44:48.191580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:48.191582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:48.191584Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:48.191591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341247655678173:5070] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:48.192166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341247655678173:5070] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:48.194967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341247655678173:5070] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:48.200129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341247655678173:5070] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:48.200150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341247655678173:5070] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:48.200154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341247655678173:5070] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:48.200165Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 382 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:48.206577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:48.209501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:48.209760Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:48.209765Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:48.209778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:48.209842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:48.209883Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341247293866907:4680]
2025-10-09T21:44:48.210548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:48.210976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:48.211052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:48.211443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:48.211530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:48.211532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:48.211533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:48.211534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:48.211536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:48.211537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:48.211538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.211539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.211544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.211550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:48.211561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:48.211564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:48.211568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:48.211571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:48.211575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:48.211579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.211582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.211585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.211594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:48.211602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:48.211609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:48.211614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:48.211618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:48.211624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:48.211628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:48.211633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:48.211637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:48.211643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:48.211647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:48.211652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:48.211657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.211662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.211666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.211672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.211677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.211682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.211711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:48.220636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:48.222253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:48.222270Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:48.222277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:48.222282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:48.222291Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 381 [2:7559341247293866907:4680]
2025-10-09T21:44:48.222327Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:48.222351Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:48.222355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:48.222359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:48.222361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:48.222381Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:48.222401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:48.222408Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:48.222487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.222489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.222492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.222495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:48.222580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:48.222583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.222598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.222608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.222624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:48.222645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341247293866902:4680]
2025-10-09T21:44:48.222647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341247293866902:4680]
2025-10-09T21:44:48.222649Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341247293866902:4680]
2025-10-09T21:44:48.222671Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:48.222686Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:48.222700Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:48.222703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:48.222729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:48.222733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:48.222735Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:48.222749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341247293866902:4680]
2025-10-09T21:44:48.222753Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:48.222863Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:48.223079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341247655678158:2432], now have 1 active actors on pipe
2025-10-09T21:44:48.223086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341247655678296:5070], now have 1 active actors on pipe
2025-10-09T21:44:48.223477Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 381
2025-10-09T21:44:48.223498Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 381
2025-10-09T21:44:48.223655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:48.223668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:48.223702Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fc38fe64-e3a1e94a-13f0161b-4874f4ff_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:48.223712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:48.223717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:48.223787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:48.223803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:48.223817Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:48.223837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:48.224718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:48.224730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:48.224754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:48.225279Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 382 partition: 0 MaxSeqNo: 25 sessionId: src_id|fc38fe64-e3a1e94a-13f0161b-4874f4ff_0
2025-10-09T21:44:48.225736Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|febe704-2e33001b-d04bd2ae-cb504c76_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fc38fe64-e3a1e94a-13f0161b-4874f4ff_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:48.228433Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc38fe64-e3a1e94a-13f0161b-4874f4ff_0] Write session: try to update token
2025-10-09T21:44:48.228454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc38fe64-e3a1e94a-13f0161b-4874f4ff_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:48.243359Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:48.243380Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:48.243382Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:48.243385Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:48.243396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341247293866902:4680]
2025-10-09T21:44:48.243401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341247293866902:4680]
2025-10-09T21:44:48.243405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341247293866902:4680]
2025-10-09T21:44:48.243459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:48.243467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.243470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:48.243470Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:48.243478Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:48.243482Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:48.243661Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:48.249972Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 382 sessionId: src_id|fc38fe64-e3a1e94a-13f0161b-4874f4ff_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:48.252624Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:48.280805Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341247655678367:4549]
2025-10-09T21:44:48.283728Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 381, pipe: [1:7559341247655678367:4549]
2025-10-09T21:44:48.409856Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:48.410212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:48.410228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:48.410277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341247655678367:4549], now have 1 active actors on pipe
2025-10-09T21:44:48.410581Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:48.411510Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341247655678367:4549]
2025-10-09T21:44:48.411572Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:48.411610Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:48.411689Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 382 reason: status is not ok: tablet will be restarted right now sessionId: src_id|fc38fe64-e3a1e94a-13f0161b-4874f4ff_0
2025-10-09T21:44:48.412148Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc38fe64-e3a1e94a-13f0161b-4874f4ff_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:48.412171Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc38fe64-e3a1e94a-13f0161b-4874f4ff_0] Write session will restart in 0.005518s
2025-10-09T21:44:48.412196Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc38fe64-e3a1e94a-13f0161b-4874f4ff_0] Write session: Do CDS request
2025-10-09T21:44:48.412204Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc38fe64-e3a1e94a-13f0161b-4874f4ff_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:48.412287Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 382 sessionId: src_id|fc38fe64-e3a1e94a-13f0161b-4874f4ff_0 is DEAD
2025-10-09T21:44:48.419379Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc38fe64-e3a1e94a-13f0161b-4874f4ff_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:48.419469Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:48.419494Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 383
2025-10-09T21:44:48.419757Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 383 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:48.419808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 383 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:48.419819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=383 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:48.419823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:48.420234Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 383 sessionId:  describe result for acl check
2025-10-09T21:44:48.420275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:48.420289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:48.420293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:48.420299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341247655678382:5074] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:48.420889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341247655678382:5074] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:48.424978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341247655678382:5074] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:48.431227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341247655678382:5074] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:48.431249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341247655678382:5074] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:48.431253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341247655678382:5074] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:48.431264Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 383 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:48.440294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:48.442686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:48.443177Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:48.443189Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:48.443206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:48.443284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:48.443337Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341247293867112:4683]
2025-10-09T21:44:48.443923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:48.444311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:48.444384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:48.444726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:48.444815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:48.444821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:48.444823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:48.444824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:48.444825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:48.444826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:48.444827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.444828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.444829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.444835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:48.444846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:48.444850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:48.444853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:48.444856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:48.444861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:48.444864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.444867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.444870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.444877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:48.444885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:48.444893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:48.444898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:48.444903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:48.444908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:48.444912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:48.444917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:48.444922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:48.444927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:48.444931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:48.444936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:48.444942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.444946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.444951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.444955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.444960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.444965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.444993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:48.452182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:48.453596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:48.453611Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:48.453613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:48.453617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:48.453624Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 382 [2:7559341247293867112:4683]
2025-10-09T21:44:48.453639Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:48.453658Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:48.453661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:48.453664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:48.453667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:48.453686Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:48.453700Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:48.453712Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:48.453783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.453791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.453793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.453796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:48.453871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:48.453881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.453894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.453905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.453922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:48.454016Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 382
2025-10-09T21:44:48.454156Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 382
2025-10-09T21:44:48.453947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341247655678369:2432], now have 1 active actors on pipe
2025-10-09T21:44:48.453958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341247293867107:4683]
2025-10-09T21:44:48.453961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341247293867107:4683]
2025-10-09T21:44:48.453963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341247293867107:4683]
2025-10-09T21:44:48.453975Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:48.453986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341247655678493:5074], now have 1 active actors on pipe
2025-10-09T21:44:48.453987Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:48.453991Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:48.453992Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:48.453994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:48.454006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:48.454010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:48.454066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341247293867107:4683]
2025-10-09T21:44:48.454074Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:48.454192Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:48.454300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:48.454318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:48.454364Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d75b7439-9fa066b9-99b57eac-b617717f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:48.454380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:48.454385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:48.454388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:48.454410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:48.454428Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:48.454448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:48.454650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:48.454661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:48.454682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:48.455440Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc38fe64-e3a1e94a-13f0161b-4874f4ff_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d75b7439-9fa066b9-99b57eac-b617717f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:48.454905Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 383 partition: 0 MaxSeqNo: 25 sessionId: src_id|d75b7439-9fa066b9-99b57eac-b617717f_0
2025-10-09T21:44:48.458396Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d75b7439-9fa066b9-99b57eac-b617717f_0] Write session: try to update token
2025-10-09T21:44:48.458418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d75b7439-9fa066b9-99b57eac-b617717f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:48.473334Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:48.473352Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:48.473364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341247293867107:4683]
2025-10-09T21:44:48.473419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:48.473478Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:48.473594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:48.473778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:48.474140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:48.474295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:48.474511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:48.474550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:48.485202Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:48.485222Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:48.485225Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:48.485227Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:48.485237Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341247293867107:4683]
2025-10-09T21:44:48.485242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341247293867107:4683]
2025-10-09T21:44:48.485246Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341247293867107:4683]
2025-10-09T21:44:48.485295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:48.485302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.485305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:48.485441Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:48.485695Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:48.485710Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:48.485714Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:48.491174Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 383 sessionId: src_id|d75b7439-9fa066b9-99b57eac-b617717f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:48.493753Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:48.511564Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341247655678582:4549]
2025-10-09T21:44:48.511919Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 382, pipe: [1:7559341247655678582:4549]
2025-10-09T21:44:48.691092Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:48.691601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:48.691611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:48.691635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341247655678582:4549], now have 1 active actors on pipe
2025-10-09T21:44:48.691851Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:48.692483Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341247655678582:4549]
2025-10-09T21:44:48.692515Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:48.693083Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d75b7439-9fa066b9-99b57eac-b617717f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:48.693093Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d75b7439-9fa066b9-99b57eac-b617717f_0] Write session will restart in 0.005079s
2025-10-09T21:44:48.693115Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d75b7439-9fa066b9-99b57eac-b617717f_0] Write session: Do CDS request
2025-10-09T21:44:48.693122Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d75b7439-9fa066b9-99b57eac-b617717f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:48.692670Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:48.692758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 383 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d75b7439-9fa066b9-99b57eac-b617717f_0
2025-10-09T21:44:48.692947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 383 sessionId: src_id|d75b7439-9fa066b9-99b57eac-b617717f_0 is DEAD
2025-10-09T21:44:48.699595Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d75b7439-9fa066b9-99b57eac-b617717f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:48.700120Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:48.700137Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 384
2025-10-09T21:44:48.700317Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 384 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:48.700354Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 384 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:48.700367Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=384 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:48.700371Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:48.700863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 384 sessionId:  describe result for acl check
2025-10-09T21:44:48.700903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:48.700905Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:48.700907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:48.700913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341247655678597:5079] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:48.701520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341247655678597:5079] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:48.704823Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341247655678597:5079] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:48.709488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341247655678597:5079] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:48.709509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341247655678597:5079] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:48.709514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341247655678597:5079] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:48.709524Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 384 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:48.726783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:48.730294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:48.730806Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:48.730822Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:48.730855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:48.730951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:48.731005Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341247293867315:4685]
2025-10-09T21:44:48.731724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:48.732484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:48.732557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:48.732952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:48.733023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:48.733025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:48.733027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:48.733028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:48.733029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:48.733031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:48.733032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.733034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.733036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.733044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:48.733054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:48.733058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:48.733062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:48.733066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:48.733070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:48.733074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.733079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.733083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.733090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:48.733099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:48.733106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:48.733112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:48.733117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:48.733122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:48.733127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:48.733133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:48.733138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:48.733143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:48.733148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:48.733153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:48.733159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.733177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.733181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.733186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.733191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.733197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.733228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:48.741629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:48.743602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:48.743618Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:48.743621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:48.743626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:48.743633Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 383 [2:7559341247293867315:4685]
2025-10-09T21:44:48.743650Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:48.743672Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:48.743675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:48.743679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:48.743681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:48.743721Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:48.743730Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:48.743738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:48.743808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.743851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.743853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.743855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:48.743959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:48.743968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.743984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:48.743997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:48.744022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:48.744051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341247655678657:5079], now have 1 active actors on pipe
2025-10-09T21:44:48.744060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341247655678585:2432], now have 1 active actors on pipe
2025-10-09T21:44:48.744068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341247293867309:4685]
2025-10-09T21:44:48.744071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341247293867309:4685]
2025-10-09T21:44:48.744073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341247293867309:4685]
2025-10-09T21:44:48.744087Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:48.744104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:48.744107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:48.744111Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:48.744113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:48.744126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:48.744129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:48.744279Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 383
2025-10-09T21:44:48.744799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341247293867309:4685]
2025-10-09T21:44:48.744819Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:48.744890Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:48.744960Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 383
2025-10-09T21:44:48.745098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:48.745112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:48.745157Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e1d93dc2-b6e5eb4c-3789a8c9-24feb8e8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:48.745172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:48.745182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:48.745188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:48.745199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:48.745219Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:48.745246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:48.745454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:48.745479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:48.745500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:48.745622Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 384 partition: 0 MaxSeqNo: 25 sessionId: src_id|e1d93dc2-b6e5eb4c-3789a8c9-24feb8e8_0
2025-10-09T21:44:48.746072Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d75b7439-9fa066b9-99b57eac-b617717f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e1d93dc2-b6e5eb4c-3789a8c9-24feb8e8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:48.748880Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1d93dc2-b6e5eb4c-3789a8c9-24feb8e8_0] Write session: try to update token
2025-10-09T21:44:48.748902Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1d93dc2-b6e5eb4c-3789a8c9-24feb8e8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:48.761165Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:48.761180Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:48.761189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341247293867309:4685]
2025-10-09T21:44:48.761233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:48.761414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:48.761593Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:48.761668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:48.761822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:48.762002Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:48.762277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:48.762344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:48.776142Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 384 sessionId: src_id|e1d93dc2-b6e5eb4c-3789a8c9-24feb8e8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:48.778820Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:48.779019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:48.779044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:48.779087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:48.782165Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:48.782179Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:48.782182Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:48.782183Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:48.782194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341247293867309:4685]
2025-10-09T21:44:48.782198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341247293867309:4685]
2025-10-09T21:44:48.782201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341247293867309:4685]
2025-10-09T21:44:48.782250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:48.782251Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:48.782257Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:48.782257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:48.782260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:48.782260Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:48.782420Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:48.779947Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:48.792565Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341247655678798:4549]
2025-10-09T21:44:48.797081Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 383, pipe: [1:7559341247655678798:4549]
2025-10-09T21:44:48.953887Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:48.955580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341247655678798:4549], now have 1 active actors on pipe
2025-10-09T21:44:48.964851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:48.964879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:48.965052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:48.965160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:48.965233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:48.965303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:48.965350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:48.965355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:48.965425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:48.965479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:48.965536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:48.965613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:48.965659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:48.965661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:48.965726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:48.965796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:48.965864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:48.965929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:48.965972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:48.965976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:48.966067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:48.966127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:48.966187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:48.966248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:48.966291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:48.966295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:48.966379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:48.966439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:48.966510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:48.966573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:48.966636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:48.966650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:48.966720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:48.966775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:48.966834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:48.966891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:48.966931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:48.966948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:48.967013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:48.967073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:48.967138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:48.967202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:48.967242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:48.967245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:48.967318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:48.967376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:48.967433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:48.967494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:48.967533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:48.967536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:48.967601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:48.967660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:48.967720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:48.967782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:48.967825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:48.967828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:48.967897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:48.967958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:48.968017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:48.968078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:48.968187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:48.968191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:48.968904Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:48.968921Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:48.968952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 384 reason: error at write: Disconnected sessionId: src_id|e1d93dc2-b6e5eb4c-3789a8c9-24feb8e8_0
2025-10-09T21:44:48.969151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 384 sessionId: src_id|e1d93dc2-b6e5eb4c-3789a8c9-24feb8e8_0 is DEAD
2025-10-09T21:44:48.969245Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:48.969293Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341247655678798:4549]
2025-10-09T21:44:48.969986Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:48.972201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1d93dc2-b6e5eb4c-3789a8c9-24feb8e8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:48.972213Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1d93dc2-b6e5eb4c-3789a8c9-24feb8e8_0] Write session will restart in 0.009261s
2025-10-09T21:44:48.972237Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1d93dc2-b6e5eb4c-3789a8c9-24feb8e8_0] Write session: Do CDS request
2025-10-09T21:44:48.972244Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1d93dc2-b6e5eb4c-3789a8c9-24feb8e8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:48.982601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1d93dc2-b6e5eb4c-3789a8c9-24feb8e8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:48.982906Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:48.982928Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 385
2025-10-09T21:44:48.986172Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 385 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:48.986229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 385 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:48.986242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=385 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:48.986258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:48.987584Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 385 sessionId:  describe result for acl check
2025-10-09T21:44:48.987622Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:48.987629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:48.987630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:48.987637Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341247655678869:5084] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:48.988220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341247655678869:5084] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:48.998921Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341247655678869:5084] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:49.007558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341247655678869:5084] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:49.007576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341247655678869:5084] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:49.007579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341247655678869:5084] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:49.007587Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 385 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:49.013227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:49.016645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:49.016907Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:49.016923Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:49.016935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:49.016994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:49.017040Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341251588834816:4687]
2025-10-09T21:44:49.017707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:49.018137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:49.018191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:49.018551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:49.018605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:49.018607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:49.018608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:49.018609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:49.018611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:49.018613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:49.018614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.018616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.018617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.018623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:49.018636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:49.018641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:49.018646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:49.018651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:49.018656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:49.018661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.018665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.018669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.018677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:49.018686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:49.018693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:49.018699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:49.018705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:49.018712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:49.018717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:49.018723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:49.018728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:49.018734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:49.018739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:49.018745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:49.018750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.018756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.018761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.018767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.018772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.018778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.018805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:49.030423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:49.031543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:49.031559Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:49.031562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:49.031583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:49.031591Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 384 [2:7559341251588834816:4687]
2025-10-09T21:44:49.031607Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:49.031629Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:49.031633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.031637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.031640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.031661Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:49.031673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:49.031681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:49.031763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.031769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.031772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.031776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:49.032170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:49.032181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.032198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.032212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.032233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:49.032258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341247655678809:2432], now have 1 active actors on pipe
2025-10-09T21:44:49.032261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341251950646302:5084], now have 1 active actors on pipe
2025-10-09T21:44:49.032270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341251588834811:4687]
2025-10-09T21:44:49.032273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341251588834811:4687]
2025-10-09T21:44:49.032276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341251588834811:4687]
2025-10-09T21:44:49.032290Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:49.032307Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:49.032325Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:49.032333Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:49.032342Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:49.032345Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:49.032348Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:49.032377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341251588834811:4687]
2025-10-09T21:44:49.032380Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:49.032386Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:49.032329Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 384
2025-10-09T21:44:49.032376Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 384
2025-10-09T21:44:49.032772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:49.032788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:49.032819Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8d322634-cd086613-4a67d767-ceea229e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:49.032829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.032839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.032843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:49.032854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.032872Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:49.032894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:49.033309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:49.033318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:49.033333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:49.034212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 385 partition: 0 MaxSeqNo: 25 sessionId: src_id|8d322634-cd086613-4a67d767-ceea229e_0
2025-10-09T21:44:49.036653Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1d93dc2-b6e5eb4c-3789a8c9-24feb8e8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8d322634-cd086613-4a67d767-ceea229e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:49.039348Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8d322634-cd086613-4a67d767-ceea229e_0] Write session: try to update token
2025-10-09T21:44:49.039379Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8d322634-cd086613-4a67d767-ceea229e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:49.047850Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:49.047865Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:49.047875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341251588834811:4687]
2025-10-09T21:44:49.047925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:49.047939Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:49.048092Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:49.048222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:49.048390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:49.048516Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:49.048701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:49.048750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:49.068556Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341251950646318:4549]
2025-10-09T21:44:49.067810Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:49.067827Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:49.067830Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:49.067832Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:49.067847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341251588834811:4687]
2025-10-09T21:44:49.067851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341251588834811:4687]
2025-10-09T21:44:49.067855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341251588834811:4687]
2025-10-09T21:44:49.067902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:49.067910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.067913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:49.068046Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:49.068647Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:49.068653Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:49.068657Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:49.070336Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 384, pipe: [1:7559341251950646318:4549]
2025-10-09T21:44:49.073647Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 385 sessionId: src_id|8d322634-cd086613-4a67d767-ceea229e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:49.076026Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:49.245710Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:49.246734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341251950646318:4549], now have 1 active actors on pipe
2025-10-09T21:44:49.246773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:49.246780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:49.247040Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:49.247599Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:49.247663Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:49.247746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 385 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8d322634-cd086613-4a67d767-ceea229e_0
2025-10-09T21:44:49.247947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 385 sessionId: src_id|8d322634-cd086613-4a67d767-ceea229e_0 is DEAD
2025-10-09T21:44:49.248130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d322634-cd086613-4a67d767-ceea229e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:49.248140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d322634-cd086613-4a67d767-ceea229e_0] Write session will restart in 0.008231s
2025-10-09T21:44:49.248165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d322634-cd086613-4a67d767-ceea229e_0] Write session: Do CDS request
2025-10-09T21:44:49.248172Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d322634-cd086613-4a67d767-ceea229e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:49.248146Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341251950646318:4549]
2025-10-09T21:44:49.258473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8d322634-cd086613-4a67d767-ceea229e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:49.260110Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:49.260132Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 386
2025-10-09T21:44:49.260328Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 386 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:49.260366Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 386 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:49.260379Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=386 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:49.260383Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:49.260861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 386 sessionId:  describe result for acl check
2025-10-09T21:44:49.260951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:49.260959Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:49.260961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:49.260968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341251950646374:5095] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:49.261703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341251950646374:5095] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:49.270220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341251950646374:5095] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:49.274806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:49.275045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341251950646374:5095] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:49.275066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341251950646374:5095] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:49.275071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341251950646374:5095] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:49.275081Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 386 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:49.276426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:49.276684Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:49.276695Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:49.276709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:49.276782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:49.276826Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341251588835064:4702]
2025-10-09T21:44:49.277436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:49.279416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:49.279486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:49.279855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:49.279916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:49.279918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:49.279919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:49.279920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:49.279921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:49.279922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:49.279923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.279925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.279926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.279932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:49.279943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:49.279947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:49.279951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:49.279955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:49.279958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:49.279961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.279964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.279968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.279977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:49.279985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:49.279991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:49.279996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:49.280001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:49.280005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:49.280009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:49.280015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:49.280020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:49.280025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:49.280030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:49.280037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:49.280041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.280046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.280050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.280055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.280059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.280064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.280088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:49.302431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:49.304126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:49.304152Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:49.304155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:49.304159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:49.304168Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 385 [2:7559341251588835064:4702]
2025-10-09T21:44:49.304184Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:49.304205Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:49.304209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.304212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.304215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.304235Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:49.304242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:49.304250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:49.304319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.304321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.304323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.304326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:49.304412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:49.304415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.304431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.304444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.304460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:49.304481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341251588835059:4702]
2025-10-09T21:44:49.304484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341251588835059:4702]
2025-10-09T21:44:49.304487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341251588835059:4702]
2025-10-09T21:44:49.304500Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:49.304516Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:49.304533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:49.304536Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:49.304583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341251588835059:4702]
2025-10-09T21:44:49.304586Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:49.305032Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:49.305041Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:49.305044Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:49.305051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:49.305405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341251950646548:5095], now have 1 active actors on pipe
2025-10-09T21:44:49.305416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341251950646346:2432], now have 1 active actors on pipe
2025-10-09T21:44:49.305526Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 385
2025-10-09T21:44:49.305787Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 385
2025-10-09T21:44:49.305959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:49.305973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:49.306005Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3c7af55d-62a96bf2-a81274f2-1797853c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:49.306016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.306053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.306057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:49.306070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.306089Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:49.306112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:49.307948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:49.307971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:49.308001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:49.308178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 386 partition: 0 MaxSeqNo: 25 sessionId: src_id|3c7af55d-62a96bf2-a81274f2-1797853c_0
2025-10-09T21:44:49.308634Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d322634-cd086613-4a67d767-ceea229e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3c7af55d-62a96bf2-a81274f2-1797853c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:49.311630Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c7af55d-62a96bf2-a81274f2-1797853c_0] Write session: try to update token
2025-10-09T21:44:49.311654Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c7af55d-62a96bf2-a81274f2-1797853c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:49.328829Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:49.328847Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:49.328850Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:49.328851Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:49.328862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341251588835059:4702]
2025-10-09T21:44:49.328868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341251588835059:4702]
2025-10-09T21:44:49.328872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341251588835059:4702]
2025-10-09T21:44:49.328926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:49.328938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.328944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:49.328946Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:49.328952Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:49.328956Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:49.329083Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:49.334630Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 386 sessionId: src_id|3c7af55d-62a96bf2-a81274f2-1797853c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:49.337276Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:49.350106Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341251950646560:4549]
2025-10-09T21:44:49.354114Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 385, pipe: [1:7559341251950646560:4549]
2025-10-09T21:44:49.500887Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:49.501177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:49.501466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:49.501526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341251950646560:4549], now have 1 active actors on pipe
2025-10-09T21:44:49.501764Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:49.502575Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341251950646560:4549]
2025-10-09T21:44:49.502627Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:49.502726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 386 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3c7af55d-62a96bf2-a81274f2-1797853c_0
2025-10-09T21:44:49.502957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 386 sessionId: src_id|3c7af55d-62a96bf2-a81274f2-1797853c_0 is DEAD
2025-10-09T21:44:49.503111Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:49.503503Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c7af55d-62a96bf2-a81274f2-1797853c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:49.503513Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c7af55d-62a96bf2-a81274f2-1797853c_0] Write session will restart in 0.005926s
2025-10-09T21:44:49.503544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c7af55d-62a96bf2-a81274f2-1797853c_0] Write session: Do CDS request
2025-10-09T21:44:49.503550Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c7af55d-62a96bf2-a81274f2-1797853c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:49.510622Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c7af55d-62a96bf2-a81274f2-1797853c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:49.510726Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:49.510749Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 387
2025-10-09T21:44:49.510973Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 387 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:49.511020Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 387 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:49.511026Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=387 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:49.511030Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:49.514606Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 387 sessionId:  describe result for acl check
2025-10-09T21:44:49.514661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:49.514663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:49.514665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:49.514671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341251950646573:5100] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:49.515346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341251950646573:5100] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:49.519048Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341251950646573:5100] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:49.531640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341251950646573:5100] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:49.531661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341251950646573:5100] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:49.531665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341251950646573:5100] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:49.531714Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 387 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:49.545648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:49.550316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:49.550588Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:49.550600Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:49.550613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:49.550686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:49.551110Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341251588835274:4705]
2025-10-09T21:44:49.552240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:49.552809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:49.552891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:49.553316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:49.553383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:49.553389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:49.553390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:49.553391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:49.553393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:49.553394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:49.553396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.553397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.553399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.553406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:49.553417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:49.553421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:49.553425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:49.553429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:49.553433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:49.553438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.553442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.553446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.553454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:49.553463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:49.553470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:49.553476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:49.553482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:49.553487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:49.553491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:49.553497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:49.553502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:49.553507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:49.553512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:49.553519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:49.553523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.553529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.553533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.553539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.553543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.553549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.553577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:49.563495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:49.564626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:49.564633Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:49.564636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:49.564641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:49.564650Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 386 [2:7559341251588835274:4705]
2025-10-09T21:44:49.565179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 386
2025-10-09T21:44:49.564667Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:49.564690Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:49.564693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.565215Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 386
2025-10-09T21:44:49.564697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.564699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.564721Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:49.564727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:49.564736Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:49.564809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.564811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.564814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.564817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:49.564897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:49.564900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.564913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.564925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.564940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:49.564961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341251588835268:4705]
2025-10-09T21:44:49.564964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341251588835268:4705]
2025-10-09T21:44:49.564966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341251588835268:4705]
2025-10-09T21:44:49.564975Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:49.564990Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:49.564993Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:49.564996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341251950646613:5100], now have 1 active actors on pipe
2025-10-09T21:44:49.564999Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:49.565003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341251950646563:2432], now have 1 active actors on pipe
2025-10-09T21:44:49.565009Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:49.565025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:49.565028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:49.565072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341251588835268:4705]
2025-10-09T21:44:49.565085Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:49.565119Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:49.565381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:49.565398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:49.565435Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|79cedd92-883da364-ba4b7612-b1175944_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:49.565449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.565459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.565476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:49.565488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.565502Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:49.565521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:49.565711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:49.565718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:49.565732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:49.565830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 387 partition: 0 MaxSeqNo: 25 sessionId: src_id|79cedd92-883da364-ba4b7612-b1175944_0
2025-10-09T21:44:49.567320Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c7af55d-62a96bf2-a81274f2-1797853c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|79cedd92-883da364-ba4b7612-b1175944_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:49.569804Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|79cedd92-883da364-ba4b7612-b1175944_0] Write session: try to update token
2025-10-09T21:44:49.569825Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|79cedd92-883da364-ba4b7612-b1175944_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:49.587971Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:49.587990Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:49.587993Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:49.587995Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:49.588005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341251588835268:4705]
2025-10-09T21:44:49.588009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341251588835268:4705]
2025-10-09T21:44:49.588013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341251588835268:4705]
2025-10-09T21:44:49.588059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:49.588068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.588071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:49.588071Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:49.588078Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:49.588081Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:49.588201Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:49.602584Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341251950646781:4549]
2025-10-09T21:44:49.604645Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 386, pipe: [1:7559341251950646781:4549]
2025-10-09T21:44:49.619393Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 387 sessionId: src_id|79cedd92-883da364-ba4b7612-b1175944_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:49.623125Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:49.756374Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:49.756828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341251950646781:4549], now have 1 active actors on pipe
2025-10-09T21:44:49.756863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:49.756874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:49.757658Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:49.758184Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79cedd92-883da364-ba4b7612-b1175944_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:49.758196Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79cedd92-883da364-ba4b7612-b1175944_0] Write session will restart in 0.006065s
2025-10-09T21:44:49.758220Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79cedd92-883da364-ba4b7612-b1175944_0] Write session: Do CDS request
2025-10-09T21:44:49.758226Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79cedd92-883da364-ba4b7612-b1175944_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:49.757692Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:49.757761Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341251950646781:4549]
2025-10-09T21:44:49.757782Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:49.757785Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:49.757855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 387 reason: error at write: Disconnected sessionId: src_id|79cedd92-883da364-ba4b7612-b1175944_0
2025-10-09T21:44:49.758050Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 387 sessionId: src_id|79cedd92-883da364-ba4b7612-b1175944_0 is DEAD
2025-10-09T21:44:49.765769Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|79cedd92-883da364-ba4b7612-b1175944_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:49.765806Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:49.765838Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 388
2025-10-09T21:44:49.766068Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 388 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:49.766113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 388 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:49.766119Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=388 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:49.766123Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:49.766602Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 388 sessionId:  describe result for acl check
2025-10-09T21:44:49.766639Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:49.766641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:49.766646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:49.766653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341251950646792:5104] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:49.767260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341251950646792:5104] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:49.770721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341251950646792:5104] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:49.776982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341251950646792:5104] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:49.777003Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341251950646792:5104] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:49.777007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341251950646792:5104] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:49.777018Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 388 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:49.788653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:49.791186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:49.791393Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:49.791403Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:49.791416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:49.791469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:49.791513Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341251588835482:4707]
2025-10-09T21:44:49.792138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:49.792645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:49.792750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:49.793098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:49.793152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:49.793155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:49.793156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:49.793158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:49.793159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:49.793160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:49.793162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.793163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.793164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.793170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:49.793181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:49.793185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:49.793189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:49.793193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:49.793197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:49.793201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.793204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.793207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.793214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:49.793223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:49.793229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:49.793234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:49.793239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:49.793245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:49.793250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:49.793255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:49.793260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:49.793266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:49.793271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:49.793277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:49.793282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.793286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.793290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.793295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.793300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.793305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.793335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:49.800436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:49.802268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:49.802281Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:49.802284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:49.802288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:49.802296Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 387 [2:7559341251588835482:4707]
2025-10-09T21:44:49.802323Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:49.802343Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:49.802346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.802350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.802352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.802371Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:49.802380Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:49.802390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:49.802452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.802460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.802462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.802465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:49.802537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:49.802540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.802566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:49.802582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:49.802597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:49.802622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341251950646784:2432], now have 1 active actors on pipe
2025-10-09T21:44:49.802625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341251950646861:5104], now have 1 active actors on pipe
2025-10-09T21:44:49.802633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341251588835477:4707]
2025-10-09T21:44:49.802636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341251588835477:4707]
2025-10-09T21:44:49.802638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341251588835477:4707]
2025-10-09T21:44:49.802653Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:49.802666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:49.802669Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:49.802671Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:49.802673Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:49.802684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:49.802687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:49.802733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341251588835477:4707]
2025-10-09T21:44:49.802746Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:49.802770Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:49.803065Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 387
2025-10-09T21:44:49.803126Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 387
2025-10-09T21:44:49.803218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:49.803233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:49.803266Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d47e3efe-823442ea-d57f1b5a-d3118f14_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:49.803291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.803295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.803298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:49.803309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.803322Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:49.803341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:49.803595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:49.803608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:49.803641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:49.803812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 388 partition: 0 MaxSeqNo: 25 sessionId: src_id|d47e3efe-823442ea-d57f1b5a-d3118f14_0
2025-10-09T21:44:49.804262Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79cedd92-883da364-ba4b7612-b1175944_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d47e3efe-823442ea-d57f1b5a-d3118f14_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:49.806888Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d47e3efe-823442ea-d57f1b5a-d3118f14_0] Write session: try to update token
2025-10-09T21:44:49.806907Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d47e3efe-823442ea-d57f1b5a-d3118f14_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:49.822018Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:49.822050Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:49.822061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341251588835477:4707]
2025-10-09T21:44:49.822116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:49.822295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:49.822455Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:49.822597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:49.822737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:49.822932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:49.822979Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:49.823097Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:49.830081Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 388 sessionId: src_id|d47e3efe-823442ea-d57f1b5a-d3118f14_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:49.832779Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:49.833697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:49.833719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:49.833741Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:49.833745Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:49.833747Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:49.833749Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:49.833758Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341251588835477:4707]
2025-10-09T21:44:49.833769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341251588835477:4707]
2025-10-09T21:44:49.833774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341251588835477:4707]
2025-10-09T21:44:49.833833Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:49.833834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:49.833842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:49.833844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:49.833845Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:49.833849Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:49.834075Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:49.858088Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341251950647002:4549]
2025-10-09T21:44:49.858514Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 387, pipe: [1:7559341251950647002:4549]
2025-10-09T21:44:50.041119Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:50.041435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:50.041553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341251950647002:4549], now have 1 active actors on pipe
2025-10-09T21:44:50.041754Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:50.046146Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:50.046211Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341251950647002:4549]
2025-10-09T21:44:50.071792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:50.073887Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:50.073906Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:50.073934Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 388 reason: error at write: Disconnected sessionId: src_id|d47e3efe-823442ea-d57f1b5a-d3118f14_0
2025-10-09T21:44:50.074325Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d47e3efe-823442ea-d57f1b5a-d3118f14_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:50.074338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d47e3efe-823442ea-d57f1b5a-d3118f14_0] Write session will restart in 0.009058s
2025-10-09T21:44:50.074373Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d47e3efe-823442ea-d57f1b5a-d3118f14_0] Write session: Do CDS request
2025-10-09T21:44:50.074381Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d47e3efe-823442ea-d57f1b5a-d3118f14_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:50.078200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 388 sessionId: src_id|d47e3efe-823442ea-d57f1b5a-d3118f14_0 is DEAD
2025-10-09T21:44:50.081754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:50.082053Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:50.082062Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:50.082077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:50.082146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:50.082501Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341255883802990:4709]
2025-10-09T21:44:50.083093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:50.084805Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d47e3efe-823442ea-d57f1b5a-d3118f14_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:50.084577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:50.084643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:50.085145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:50.085223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:50.085231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:50.085232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:50.085234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:50.085235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:50.085237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:50.085238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.085239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.085240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.085246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:50.085340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:50.085351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:50.085355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:50.085359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:50.085364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:50.085368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.085373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.085378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.085388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:50.085401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:50.085407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:50.085413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:50.085418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:50.085424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:50.085429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:50.085433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:50.085440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:50.085446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:50.085450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:50.085454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:50.085458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.085463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.085468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.085474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.085479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.085484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.085530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:50.086101Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:50.086121Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 389
2025-10-09T21:44:50.086312Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 389 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:50.086365Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 389 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:50.086377Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=389 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:50.086382Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:50.086947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 389 sessionId:  describe result for acl check
2025-10-09T21:44:50.087000Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:50.087007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:50.087009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:50.087015Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341256245614489:5109] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:50.087593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341256245614489:5109] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:50.091337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341256245614489:5109] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:50.094931Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341256245614489:5109] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:50.094949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341256245614489:5109] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:50.094953Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341256245614489:5109] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:50.094970Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 389 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:50.096355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:50.097741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:50.097759Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:50.097762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:50.097766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:50.097773Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 388 [2:7559341255883802990:4709]
2025-10-09T21:44:50.097789Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:50.097807Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:50.097810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.097814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.097816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.097837Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:50.097849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:50.097857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:50.097919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.097928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.097931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.097934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:50.098011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:50.098020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.098052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.098063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.098077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:50.098100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341255883802983:4709]
2025-10-09T21:44:50.098108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341255883802983:4709]
2025-10-09T21:44:50.098111Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341255883802983:4709]
2025-10-09T21:44:50.098120Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:50.098131Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:50.098137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:50.098142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:50.098144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:50.098153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:50.098156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:50.098168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341256245614522:5109], now have 1 active actors on pipe
2025-10-09T21:44:50.098172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341256245614314:2432], now have 1 active actors on pipe
2025-10-09T21:44:50.098314Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 388
2025-10-09T21:44:50.098361Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 388
2025-10-09T21:44:50.098212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341255883802983:4709]
2025-10-09T21:44:50.098225Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:50.098484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:50.098493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:50.098524Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:50.098532Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d1b27416-b9d91c9c-e51f785d-796f8c32_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:50.098542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.098547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.098550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:50.098563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.098588Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:50.098609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:50.099250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:50.099260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:50.099285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:50.099393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 389 partition: 0 MaxSeqNo: 25 sessionId: src_id|d1b27416-b9d91c9c-e51f785d-796f8c32_0
2025-10-09T21:44:50.099851Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d47e3efe-823442ea-d57f1b5a-d3118f14_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d1b27416-b9d91c9c-e51f785d-796f8c32_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:50.102540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1b27416-b9d91c9c-e51f785d-796f8c32_0] Write session: try to update token
2025-10-09T21:44:50.102566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1b27416-b9d91c9c-e51f785d-796f8c32_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:50.115872Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:50.115888Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:50.115895Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:50.115897Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:50.115909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341255883802983:4709]
2025-10-09T21:44:50.115913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341255883802983:4709]
2025-10-09T21:44:50.115917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341255883802983:4709]
2025-10-09T21:44:50.115964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:50.115970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.115972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:50.116083Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:50.116089Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:50.116093Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:50.116150Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:50.124612Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 389 sessionId: src_id|d1b27416-b9d91c9c-e51f785d-796f8c32_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:50.127312Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:50.146559Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341256245614530:4549]
2025-10-09T21:44:50.147008Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 388, pipe: [1:7559341256245614530:4549]
2025-10-09T21:44:50.402435Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:50.402865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:50.403334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:50.403401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341256245614530:4549], now have 1 active actors on pipe
2025-10-09T21:44:50.404145Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:50.404976Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1b27416-b9d91c9c-e51f785d-796f8c32_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:50.404984Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1b27416-b9d91c9c-e51f785d-796f8c32_0] Write session will restart in 0.005465s
2025-10-09T21:44:50.405009Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1b27416-b9d91c9c-e51f785d-796f8c32_0] Write session: Do CDS request
2025-10-09T21:44:50.405017Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1b27416-b9d91c9c-e51f785d-796f8c32_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:50.403832Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:50.403851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:50.403927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 389 reason: error at write: Disconnected sessionId: src_id|d1b27416-b9d91c9c-e51f785d-796f8c32_0
2025-10-09T21:44:50.404126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 389 sessionId: src_id|d1b27416-b9d91c9c-e51f785d-796f8c32_0 is DEAD
2025-10-09T21:44:50.404216Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341256245614530:4549]
2025-10-09T21:44:50.404234Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:50.411969Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1b27416-b9d91c9c-e51f785d-796f8c32_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:50.414381Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:50.414400Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 390
2025-10-09T21:44:50.414709Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 390 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:50.414744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 390 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:50.414750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=390 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:50.414755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:50.418538Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 390 sessionId:  describe result for acl check
2025-10-09T21:44:50.418588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:50.418590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:50.418592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:50.418599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341256245614586:5120] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:50.419135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341256245614586:5120] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:50.434122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341256245614586:5120] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:50.437170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:50.439834Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341256245614586:5120] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:50.439853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341256245614586:5120] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:50.439857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341256245614586:5120] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:50.439868Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 390 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:50.441025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:50.441258Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:50.441270Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:50.441284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:50.441344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:50.441388Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341255883803243:4725]
2025-10-09T21:44:50.441993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:50.442350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:50.442396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:50.442688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:50.442736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:50.442742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:50.442743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:50.442744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:50.442745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:50.442746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:50.442747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.442748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.442749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.442754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:50.442763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:50.442766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:50.442769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:50.442772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:50.442775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:50.442778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.442781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.442784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.442791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:50.442798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:50.442803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:50.442807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:50.442811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:50.442815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:50.442818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:50.442822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:50.442826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:50.442830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:50.442833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:50.442837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:50.442842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.442846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.442849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.442854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.442858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.442861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.442887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:50.450399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:50.451830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:50.451847Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:50.451849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:50.451854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:50.451863Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 389 [2:7559341255883803243:4725]
2025-10-09T21:44:50.451881Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:50.451903Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:50.451907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.451911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.451913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.451936Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:50.451944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:50.451951Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:50.452018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.452027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.452029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.452032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:50.452112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:50.452120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.452135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.452153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.452169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:50.452191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341255883803237:4725]
2025-10-09T21:44:50.452193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341255883803237:4725]
2025-10-09T21:44:50.452195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341255883803237:4725]
2025-10-09T21:44:50.452205Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:50.452215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:50.452217Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:50.452220Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:50.452220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341256245614765:5120], now have 1 active actors on pipe
2025-10-09T21:44:50.452228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341256245614559:2432], now have 1 active actors on pipe
2025-10-09T21:44:50.452234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:50.452248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:50.452251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:50.452317Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 389
2025-10-09T21:44:50.452367Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 389
2025-10-09T21:44:50.452297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341255883803237:4725]
2025-10-09T21:44:50.452311Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:50.452316Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:50.452553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:50.452577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:50.452609Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f3e785a4-9f2d0119-7f166e3b-9a984d01_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:50.452627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.452632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.452635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:50.452647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.452660Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:50.452680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:50.453034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:50.453047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:50.453070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:50.453191Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 390 partition: 0 MaxSeqNo: 25 sessionId: src_id|f3e785a4-9f2d0119-7f166e3b-9a984d01_0
2025-10-09T21:44:50.454116Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1b27416-b9d91c9c-e51f785d-796f8c32_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f3e785a4-9f2d0119-7f166e3b-9a984d01_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:50.456786Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3e785a4-9f2d0119-7f166e3b-9a984d01_0] Write session: try to update token
2025-10-09T21:44:50.456805Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3e785a4-9f2d0119-7f166e3b-9a984d01_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:50.468710Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:50.468726Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:50.468729Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:50.468731Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:50.468740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341255883803237:4725]
2025-10-09T21:44:50.468744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341255883803237:4725]
2025-10-09T21:44:50.468747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341255883803237:4725]
2025-10-09T21:44:50.468794Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:50.468804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:50.468806Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:50.468810Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:50.468811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.468813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:50.469007Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:50.483282Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 390 sessionId: src_id|f3e785a4-9f2d0119-7f166e3b-9a984d01_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:50.485793Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:50.504195Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341256245614777:4549]
2025-10-09T21:44:50.504661Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 389, pipe: [1:7559341256245614777:4549]
2025-10-09T21:44:50.637411Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:50.637886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:50.637905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:50.637928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341256245614777:4549], now have 1 active actors on pipe
2025-10-09T21:44:50.638177Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:50.638890Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3e785a4-9f2d0119-7f166e3b-9a984d01_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:50.638903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3e785a4-9f2d0119-7f166e3b-9a984d01_0] Write session will restart in 0.005318s
2025-10-09T21:44:50.638928Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3e785a4-9f2d0119-7f166e3b-9a984d01_0] Write session: Do CDS request
2025-10-09T21:44:50.638934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3e785a4-9f2d0119-7f166e3b-9a984d01_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:50.638223Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:50.638237Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:50.638241Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:50.638287Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341256245614777:4549]
2025-10-09T21:44:50.638327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 390 reason: error at write: Disconnected sessionId: src_id|f3e785a4-9f2d0119-7f166e3b-9a984d01_0
2025-10-09T21:44:50.638527Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 390 sessionId: src_id|f3e785a4-9f2d0119-7f166e3b-9a984d01_0 is DEAD
2025-10-09T21:44:50.645774Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3e785a4-9f2d0119-7f166e3b-9a984d01_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:50.645916Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:50.645937Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 391
2025-10-09T21:44:50.646104Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 391 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:50.646157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 391 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:50.646163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=391 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:50.646167Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:50.646805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 391 sessionId:  describe result for acl check
2025-10-09T21:44:50.646846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:50.646848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:50.646850Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:50.646856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341256245614795:5125] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:50.647473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341256245614795:5125] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:50.652464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341256245614795:5125] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:50.660480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341256245614795:5125] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:50.660501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341256245614795:5125] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:50.660506Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341256245614795:5125] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:50.660517Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 391 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:50.678323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:50.680068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:50.680492Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:50.680504Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:50.680520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:50.680593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:50.680635Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341255883803456:4727]
2025-10-09T21:44:50.681242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:50.681659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:50.681728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:50.682096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:50.682181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:50.682183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:50.682184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:50.682185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:50.682187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:50.682189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:50.682190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.682191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.682193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.682200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:50.682211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:50.682215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:50.682219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:50.682222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:50.682227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:50.682231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.682234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.682237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.682248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:50.682256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:50.682262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:50.682267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:50.682275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:50.682280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:50.682284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:50.682290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:50.682294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:50.682313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:50.682317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:50.682322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:50.682327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.682331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.682334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.682338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.682342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.682346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.682373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:50.694367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:50.695938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:50.695946Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:50.695949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:50.695954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:50.695963Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 390 [2:7559341255883803456:4727]
2025-10-09T21:44:50.695979Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:50.696003Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:50.696006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.696009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.696011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.696031Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:50.696038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:50.696045Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:50.696119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.696121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.696123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.696126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:50.696202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:50.696205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.696220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.696232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.696251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:50.696272Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341255883803451:4727]
2025-10-09T21:44:50.696276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341255883803451:4727]
2025-10-09T21:44:50.696279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341255883803451:4727]
2025-10-09T21:44:50.696301Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:50.696304Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:50.696308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:50.696310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:50.696319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:50.696334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:50.696338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:50.696350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341256245614902:5125], now have 1 active actors on pipe
2025-10-09T21:44:50.696354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341256245614782:2432], now have 1 active actors on pipe
2025-10-09T21:44:50.696395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341255883803451:4727]
2025-10-09T21:44:50.696405Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:50.696751Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 390
2025-10-09T21:44:50.696787Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 390
2025-10-09T21:44:50.697146Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:50.697420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:50.697431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:50.697465Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|96f6e922-9d53fabd-4a253e4a-3eb33182_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:50.697475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.697480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.697482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:50.697495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.697509Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:50.697528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:50.698263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:50.698275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:50.698316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:50.698500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 391 partition: 0 MaxSeqNo: 25 sessionId: src_id|96f6e922-9d53fabd-4a253e4a-3eb33182_0
2025-10-09T21:44:50.701332Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3e785a4-9f2d0119-7f166e3b-9a984d01_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|96f6e922-9d53fabd-4a253e4a-3eb33182_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:50.704114Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96f6e922-9d53fabd-4a253e4a-3eb33182_0] Write session: try to update token
2025-10-09T21:44:50.704134Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96f6e922-9d53fabd-4a253e4a-3eb33182_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:50.723328Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:50.723347Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:50.723350Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:50.723353Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:50.723364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341255883803451:4727]
2025-10-09T21:44:50.723369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341255883803451:4727]
2025-10-09T21:44:50.723373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341255883803451:4727]
2025-10-09T21:44:50.723420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:50.723428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.723431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:50.723594Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:50.727406Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:50.727421Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:50.727426Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:50.730255Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 391 sessionId: src_id|96f6e922-9d53fabd-4a253e4a-3eb33182_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:50.732851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:50.741077Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341256245615002:4549]
2025-10-09T21:44:50.742196Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 390, pipe: [1:7559341256245615002:4549]
2025-10-09T21:44:50.872308Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:44:50.872343Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 4
2025-10-09T21:44:50.920187Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:50.920552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:50.920563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:50.920587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341256245615002:4549], now have 1 active actors on pipe
2025-10-09T21:44:50.921170Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:50.921624Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:50.921711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 391 reason: status is not ok: tablet will be restarted right now sessionId: src_id|96f6e922-9d53fabd-4a253e4a-3eb33182_0
2025-10-09T21:44:50.921899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 391 sessionId: src_id|96f6e922-9d53fabd-4a253e4a-3eb33182_0 is DEAD
2025-10-09T21:44:50.922089Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:50.922107Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 4
2025-10-09T21:44:50.922136Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341256245615002:4549]
2025-10-09T21:44:50.922383Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96f6e922-9d53fabd-4a253e4a-3eb33182_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:50.922398Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96f6e922-9d53fabd-4a253e4a-3eb33182_0] Write session will restart in 0.007271s
2025-10-09T21:44:50.922421Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96f6e922-9d53fabd-4a253e4a-3eb33182_0] Write session: Do CDS request
2025-10-09T21:44:50.922427Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96f6e922-9d53fabd-4a253e4a-3eb33182_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:50.930574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96f6e922-9d53fabd-4a253e4a-3eb33182_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:50.931047Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:50.931067Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 392
2025-10-09T21:44:50.931198Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 392 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:50.931228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 392 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:50.931234Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=392 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:50.931239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:50.931959Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 392 sessionId:  describe result for acl check
2025-10-09T21:44:50.931997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:50.931999Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:50.932001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:50.932010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341256245615024:5129] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:50.932604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341256245615024:5129] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:50.940516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341256245615024:5129] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:50.947621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341256245615024:5129] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:50.947677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341256245615024:5129] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:50.947683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341256245615024:5129] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:50.947693Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 392 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:50.956221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:50.960145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:50.960423Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:50.960436Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:50.960450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:50.960523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:50.960572Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341255883803670:4730]
2025-10-09T21:44:50.961176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:50.961826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:50.961885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:50.962256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:50.962360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:50.962362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:50.962363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:50.962364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:50.962366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:50.962367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:50.962368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.962370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.962371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.962377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:50.962388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:50.962391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:50.962395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:50.962398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:50.962402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:50.962406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.962410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.962413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.962422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:50.962431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:50.962437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:50.962442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:50.962447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:50.962452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:50.962456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:50.962462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:50.962467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:50.962485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:50.962490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:50.962496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:50.962502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.962507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.962513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.962519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.962523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.962527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.962559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:50.970652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:50.972143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:50.972161Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:50.972164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:50.972168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:50.972177Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 391 [2:7559341255883803670:4730]
2025-10-09T21:44:50.972193Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:50.972213Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:50.972217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.972220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.972223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.972257Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:50.972269Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:50.972277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:50.972350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.972357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.972359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.972362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:50.972449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:50.972458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.972472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:50.972483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:50.972499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:50.972666Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 391
2025-10-09T21:44:50.972522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341255883803664:4730]
2025-10-09T21:44:50.972530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341255883803664:4730]
2025-10-09T21:44:50.972532Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341255883803664:4730]
2025-10-09T21:44:50.972547Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:50.972564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341256245615012:2432], now have 1 active actors on pipe
2025-10-09T21:44:50.972568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341256245615204:5129], now have 1 active actors on pipe
2025-10-09T21:44:50.972669Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 391
2025-10-09T21:44:50.972577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:50.972603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:50.972607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:50.972640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:50.972644Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:50.972647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:50.972651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341255883803664:4730]
2025-10-09T21:44:50.972654Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:50.972671Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:50.972779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:50.972786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:50.972881Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:44:50.972921Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|166bd021-f804f2c0-ae45a26a-4f5aa374_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:50.972930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.972940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.972943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:50.972955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.972966Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:50.972989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:50.973268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:50.973286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:50.973307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:50.973269Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 4 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:44:50.973307Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:44:50.973432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 392 partition: 0 MaxSeqNo: 25 sessionId: src_id|166bd021-f804f2c0-ae45a26a-4f5aa374_0
2025-10-09T21:44:50.973798Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96f6e922-9d53fabd-4a253e4a-3eb33182_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|166bd021-f804f2c0-ae45a26a-4f5aa374_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:50.976551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|166bd021-f804f2c0-ae45a26a-4f5aa374_0] Write session: try to update token
2025-10-09T21:44:50.976571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|166bd021-f804f2c0-ae45a26a-4f5aa374_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:50.997321Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 392 sessionId: src_id|166bd021-f804f2c0-ae45a26a-4f5aa374_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:50.998652Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:50.998667Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:50.998670Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:50.998672Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:50.998681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341255883803664:4730]
2025-10-09T21:44:50.998686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341255883803664:4730]
2025-10-09T21:44:50.998690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341255883803664:4730]
2025-10-09T21:44:50.998743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:50.998747Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:50.998752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:50.998754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:50.998764Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:50.998770Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:50.998898Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:50.999931Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:51.023878Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341260540582534:4549]
2025-10-09T21:44:51.025838Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 391, pipe: [1:7559341260540582534:4549]
2025-10-09T21:44:51.166498Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:51.166929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:51.166941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:51.166993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341260540582534:4549], now have 1 active actors on pipe
2025-10-09T21:44:51.168246Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|166bd021-f804f2c0-ae45a26a-4f5aa374_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:51.168263Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|166bd021-f804f2c0-ae45a26a-4f5aa374_0] Write session will restart in 0.006175s
2025-10-09T21:44:51.168290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|166bd021-f804f2c0-ae45a26a-4f5aa374_0] Write session: Do CDS request
2025-10-09T21:44:51.168297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|166bd021-f804f2c0-ae45a26a-4f5aa374_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:51.168150Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:51.167481Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341260540582534:4549]
2025-10-09T21:44:51.167503Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:51.167517Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:51.167530Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:51.167592Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 392 reason: error at write: Disconnected sessionId: src_id|166bd021-f804f2c0-ae45a26a-4f5aa374_0
2025-10-09T21:44:51.167806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 392 sessionId: src_id|166bd021-f804f2c0-ae45a26a-4f5aa374_0 is DEAD
2025-10-09T21:44:51.175675Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|166bd021-f804f2c0-ae45a26a-4f5aa374_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:51.178577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:51.178602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 393
2025-10-09T21:44:51.178799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 393 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:51.178847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 393 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:51.178853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=393 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:51.178863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:51.179691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 393 sessionId:  describe result for acl check
2025-10-09T21:44:51.179764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:51.179777Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:51.179782Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:51.179799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341260540582560:5134] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:51.180668Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341260540582560:5134] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:51.187173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341260540582560:5134] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:51.191423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341260540582560:5134] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:51.191445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341260540582560:5134] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:51.191450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341260540582560:5134] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:51.191459Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 393 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:51.195973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:51.198009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:51.198510Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:51.198534Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:51.198550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:51.198625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:51.198666Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341260178771184:4734]
2025-10-09T21:44:51.199291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:51.201596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:51.201697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:51.202126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:51.202193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:51.202194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:51.202196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:51.202197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:51.202199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:51.202200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:51.202202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.202206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.202208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.202214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:51.202225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:51.202228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:51.202233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:51.202237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:51.202241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:51.202245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.202250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.202254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.202264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:51.202274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:51.202281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:51.202288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:51.202307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:51.202313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:51.202318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:51.202325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:51.202331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:51.202336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:51.202342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:51.202346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:51.202352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.202357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.202370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.202375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.202380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.202386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.202418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:51.216486Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 392
2025-10-09T21:44:51.216506Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 392
2025-10-09T21:44:51.214630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:51.215928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:51.215936Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:51.215938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:51.215944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:51.215953Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 392 [2:7559341260178771184:4734]
2025-10-09T21:44:51.215969Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:51.215992Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:51.215996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:51.215999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:51.216005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:51.216026Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:51.216033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:51.216041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:51.216113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.216115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.216117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.216119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:51.216224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:51.216227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.216243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.216254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.216273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:51.216295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341260178771179:4734]
2025-10-09T21:44:51.216300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341260178771179:4734]
2025-10-09T21:44:51.216302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341260178771179:4734]
2025-10-09T21:44:51.216311Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:51.216328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:51.216344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:51.216347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:51.216361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:51.216365Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:51.216368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:51.216392Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341260178771179:4734]
2025-10-09T21:44:51.216395Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:51.216406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341260540582743:5134], now have 1 active actors on pipe
2025-10-09T21:44:51.216410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341260540582540:2432], now have 1 active actors on pipe
2025-10-09T21:44:51.216411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:51.216918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:51.216930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:51.216977Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9d9c7e1-278a2632-db22b514-384c0dda_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:51.216987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:51.216992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:51.216995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:51.217007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:51.217020Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:51.217042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:51.217312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:51.217325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:51.217358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:51.220619Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 393 partition: 0 MaxSeqNo: 25 sessionId: src_id|9d9c7e1-278a2632-db22b514-384c0dda_0
2025-10-09T21:44:51.223210Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|166bd021-f804f2c0-ae45a26a-4f5aa374_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9d9c7e1-278a2632-db22b514-384c0dda_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:51.225922Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d9c7e1-278a2632-db22b514-384c0dda_0] Write session: try to update token
2025-10-09T21:44:51.225949Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d9c7e1-278a2632-db22b514-384c0dda_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:51.237433Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:51.237454Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:51.237456Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:51.237458Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:51.237469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341260178771179:4734]
2025-10-09T21:44:51.237473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341260178771179:4734]
2025-10-09T21:44:51.237476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341260178771179:4734]
2025-10-09T21:44:51.237528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:51.237535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.237538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:51.237694Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:51.238401Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:51.238408Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:51.238412Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:51.267545Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341260540582761:4549]
2025-10-09T21:44:51.268352Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 392, pipe: [1:7559341260540582761:4549]
2025-10-09T21:44:51.278965Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 393 sessionId: src_id|9d9c7e1-278a2632-db22b514-384c0dda_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:51.281480Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:51.410682Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:51.411067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341260540582761:4549], now have 1 active actors on pipe
2025-10-09T21:44:51.411102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:51.411112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:51.411314Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:51.412865Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:51.412882Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:51.412887Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341260540582761:4549]
2025-10-09T21:44:51.412907Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:51.412959Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 393 reason: error at write: Disconnected sessionId: src_id|9d9c7e1-278a2632-db22b514-384c0dda_0
2025-10-09T21:44:51.413171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 393 sessionId: src_id|9d9c7e1-278a2632-db22b514-384c0dda_0 is DEAD
2025-10-09T21:44:51.418277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d9c7e1-278a2632-db22b514-384c0dda_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:51.418308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d9c7e1-278a2632-db22b514-384c0dda_0] Write session will restart in 0.009696s
2025-10-09T21:44:51.418338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d9c7e1-278a2632-db22b514-384c0dda_0] Write session: Do CDS request
2025-10-09T21:44:51.418345Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d9c7e1-278a2632-db22b514-384c0dda_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:51.430103Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d9c7e1-278a2632-db22b514-384c0dda_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:51.431429Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:51.431457Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 394
2025-10-09T21:44:51.431793Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 394 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:51.431832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 394 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:51.431840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=394 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:51.431845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:51.432428Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 394 sessionId:  describe result for acl check
2025-10-09T21:44:51.432471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:51.432474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:51.432476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:51.432483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341260540582800:5145] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:51.433068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341260540582800:5145] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:51.446120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341260540582800:5145] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:51.463651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341260540582800:5145] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:51.463676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341260540582800:5145] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:51.463681Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341260540582800:5145] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:51.463693Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 394 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:51.483109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:51.490368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:51.490661Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:51.490666Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:51.490678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:51.490742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:51.490786Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341260178771441:4749]
2025-10-09T21:44:51.491420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:51.493056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:51.493129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:51.493927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:51.494007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:51.494008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:51.494009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:51.494010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:51.494012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:51.494013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:51.494014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.494015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.494017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.494023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:51.494049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:51.494053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:51.494057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:51.494060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:51.494063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:51.494066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.494069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.494072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.494083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:51.494091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:51.494097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:51.494102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:51.494106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:51.494111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:51.494116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:51.494121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:51.494127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:51.494133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:51.494137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:51.494143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:51.494148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.494154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.494159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.494164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.494169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.494175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.494206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:51.504653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:51.505739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:51.505747Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:51.505751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:51.505757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:51.505766Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 393 [2:7559341260178771441:4749]
2025-10-09T21:44:51.505782Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:51.505805Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:51.505808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:51.505812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:51.505815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:51.505836Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:51.505842Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:51.505851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:51.505919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.505922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.505924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.505927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:51.506009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:51.506013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.506045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.506056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.506069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:51.506093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341260178771435:4749]
2025-10-09T21:44:51.506096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341260178771435:4749]
2025-10-09T21:44:51.506098Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341260178771435:4749]
2025-10-09T21:44:51.506113Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:51.506129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:51.506146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:51.506149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:51.506200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341260178771435:4749]
2025-10-09T21:44:51.506202Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:51.507316Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 393
2025-10-09T21:44:51.507352Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 393
2025-10-09T21:44:51.506463Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:51.506472Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:51.506474Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:51.506480Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:51.508984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341260540582932:5145], now have 1 active actors on pipe
2025-10-09T21:44:51.509000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341260540582789:2432], now have 1 active actors on pipe
2025-10-09T21:44:51.509022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:51.509029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:51.509085Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|57143e88-afc70811-4b8058f5-58a90032_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:51.509098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:51.509103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:51.509107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:51.509120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:51.509136Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:51.509158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:51.509777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:51.509792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:51.509823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:51.510059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 394 partition: 0 MaxSeqNo: 25 sessionId: src_id|57143e88-afc70811-4b8058f5-58a90032_0
2025-10-09T21:44:51.510541Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d9c7e1-278a2632-db22b514-384c0dda_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|57143e88-afc70811-4b8058f5-58a90032_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:51.512989Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341260540583009:4549]
2025-10-09T21:44:51.513100Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57143e88-afc70811-4b8058f5-58a90032_0] Write session: try to update token
2025-10-09T21:44:51.513116Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57143e88-afc70811-4b8058f5-58a90032_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:51.513939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341260540583009:4549], now have 1 active actors on pipe
2025-10-09T21:44:51.514151Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 393, pipe: [1:7559341260540583009:4549]
2025-10-09T21:44:51.530756Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:51.530781Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:51.530784Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:51.530787Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:51.530798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341260178771435:4749]
2025-10-09T21:44:51.530802Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341260178771435:4749]
2025-10-09T21:44:51.530806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341260178771435:4749]
2025-10-09T21:44:51.530856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:51.530865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.530867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:51.531000Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:51.531165Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:51.531172Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:51.531185Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:51.550383Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 394 sessionId: src_id|57143e88-afc70811-4b8058f5-58a90032_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:51.552833Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:51.832927Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:51.833393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:51.833409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:51.834194Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:51.837756Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:51.837854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 394 reason: status is not ok: tablet will be restarted right now sessionId: src_id|57143e88-afc70811-4b8058f5-58a90032_0
2025-10-09T21:44:51.838511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57143e88-afc70811-4b8058f5-58a90032_0] Counters: { Errors: 0 CurrentSessionLifetimeMs: 327 BytesWritten: 56320040 MessagesWritten: 25 BytesWrittenCompressed: 0 BytesInflightUncompressed: 22528020 BytesInflightCompressed: 0 BytesInflightTotal: 18022416 MessagesInflight: 10 }
2025-10-09T21:44:51.838543Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57143e88-afc70811-4b8058f5-58a90032_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:51.838548Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57143e88-afc70811-4b8058f5-58a90032_0] Write session will restart in 0.009796s
2025-10-09T21:44:51.838572Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57143e88-afc70811-4b8058f5-58a90032_0] Write session: Do CDS request
2025-10-09T21:44:51.838579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57143e88-afc70811-4b8058f5-58a90032_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:51.838050Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 394 sessionId: src_id|57143e88-afc70811-4b8058f5-58a90032_0 is DEAD
2025-10-09T21:44:51.838230Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:51.838261Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341260540583009:4549]
2025-10-09T21:44:51.849677Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57143e88-afc70811-4b8058f5-58a90032_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:51.849798Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:51.849820Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 395
2025-10-09T21:44:51.850769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 395 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:51.850815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 395 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:51.850822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=395 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:51.850827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:51.851769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 395 sessionId:  describe result for acl check
2025-10-09T21:44:51.851819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:51.851829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:51.851831Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:51.851838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341260540583051:5158] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:51.852432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341260540583051:5158] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:51.856181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341260540583051:5158] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:51.864122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341260540583051:5158] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:51.864145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341260540583051:5158] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:51.864148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341260540583051:5158] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:51.864158Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 395 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:51.877075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:51.880865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:51.881121Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:51.881134Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:51.881149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:51.881221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:51.881265Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341260178771658:4751]
2025-10-09T21:44:51.881828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:51.886380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:51.886467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:51.886849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:51.886917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:51.886919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:51.886920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:51.886921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:51.886922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:51.886924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:51.886925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.886926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.886927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.886933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:51.886945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:51.886950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:51.886954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:51.886958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:51.886962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:51.886966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.886970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.886974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.886981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:51.886989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:51.886995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:51.887002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:51.887006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:51.887012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:51.887017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:51.887022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:51.887027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:51.887032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:51.887037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:51.887042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:51.887048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.887053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.887058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.887065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.887070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.887074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.887101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:51.903466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:51.904565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:51.904572Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:51.904574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:51.904582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:51.904593Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 394 [2:7559341260178771658:4751]
2025-10-09T21:44:51.904609Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:51.904630Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:51.904633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:51.904636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:51.904639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:51.904657Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:51.904663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:51.904672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:51.904739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.904741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.904743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.904746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:51.904821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:51.904823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.904837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:51.904848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:51.904864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:51.904885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341260178771653:4751]
2025-10-09T21:44:51.904887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341260178771653:4751]
2025-10-09T21:44:51.904889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341260178771653:4751]
2025-10-09T21:44:51.904907Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:51.904921Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:51.904934Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:51.904937Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:51.904982Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341260178771653:4751]
2025-10-09T21:44:51.904984Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:51.905195Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:51.905201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:51.905202Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:51.905208Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:51.905650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341260540583171:5158], now have 1 active actors on pipe
2025-10-09T21:44:51.905666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341260540583030:2432], now have 1 active actors on pipe
2025-10-09T21:44:51.908107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 394
2025-10-09T21:44:51.908144Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 394
2025-10-09T21:44:51.908488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:51.908507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:51.908545Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b3ccfc41-1a42735a-9841f5fc-e22cb329_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:51.908562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:51.908568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:51.908571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:51.908586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:51.908604Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:51.908628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:51.909031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:51.909038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:51.909054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:51.909205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 395 partition: 0 MaxSeqNo: 25 sessionId: src_id|b3ccfc41-1a42735a-9841f5fc-e22cb329_0
2025-10-09T21:44:51.909645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57143e88-afc70811-4b8058f5-58a90032_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b3ccfc41-1a42735a-9841f5fc-e22cb329_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:51.912286Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3ccfc41-1a42735a-9841f5fc-e22cb329_0] Write session: try to update token
2025-10-09T21:44:51.912305Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3ccfc41-1a42735a-9841f5fc-e22cb329_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:51.929107Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:51.929124Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:51.929127Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:51.929128Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:51.929138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341260178771653:4751]
2025-10-09T21:44:51.929142Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341260178771653:4751]
2025-10-09T21:44:51.929146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341260178771653:4751]
2025-10-09T21:44:51.929195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:51.929208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:51.929211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:51.929350Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:51.931050Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:51.931064Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:51.931068Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:51.938364Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341260540583255:4549]
2025-10-09T21:44:51.939133Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 394, pipe: [1:7559341260540583255:4549]
2025-10-09T21:44:51.943410Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 395 sessionId: src_id|b3ccfc41-1a42735a-9841f5fc-e22cb329_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:51.946208Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:52.100063Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:52.100749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341260540583255:4549], now have 1 active actors on pipe
2025-10-09T21:44:52.100818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:52.100825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:52.101247Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:52.101936Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3ccfc41-1a42735a-9841f5fc-e22cb329_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:52.101949Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3ccfc41-1a42735a-9841f5fc-e22cb329_0] Write session will restart in 0.008677s
2025-10-09T21:44:52.101994Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3ccfc41-1a42735a-9841f5fc-e22cb329_0] Write session: Do CDS request
2025-10-09T21:44:52.102002Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3ccfc41-1a42735a-9841f5fc-e22cb329_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:52.101206Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:52.101225Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:52.101304Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 395 reason: error at write: Disconnected sessionId: src_id|b3ccfc41-1a42735a-9841f5fc-e22cb329_0
2025-10-09T21:44:52.101523Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 395 sessionId: src_id|b3ccfc41-1a42735a-9841f5fc-e22cb329_0 is DEAD
2025-10-09T21:44:52.101668Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:52.101701Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341260540583255:4549]
2025-10-09T21:44:52.114574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3ccfc41-1a42735a-9841f5fc-e22cb329_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:52.118633Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:52.118656Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 396
2025-10-09T21:44:52.118848Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 396 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:52.118888Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 396 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:52.118896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=396 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:52.118901Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:52.121462Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 396 sessionId:  describe result for acl check
2025-10-09T21:44:52.121504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:52.121505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:52.121507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:52.121515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341264835550635:5163] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:52.122159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341264835550635:5163] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:52.131983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341264835550635:5163] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:52.139617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341264835550635:5163] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:52.139639Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341264835550635:5163] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:52.139644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341264835550635:5163] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:52.139654Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 396 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:52.142574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:52.146633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:52.147050Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:52.147064Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:52.147081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:52.147173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:52.147230Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341264473739174:4754]
2025-10-09T21:44:52.147835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:52.148410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:52.148478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:52.148867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:52.148943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:52.148944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:52.148946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:52.148947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:52.148948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:52.148949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:52.148951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.148952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.148953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.148960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:52.148971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:52.148976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:52.148980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:52.148985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:52.148990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:52.148994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.148998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.149002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.149014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:52.149025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:52.149031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:52.149037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:52.149042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:52.149048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:52.149053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:52.149059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:52.149064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:52.149069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:52.149073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:52.149078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:52.149082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.149086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.149092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.149096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.149101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.149105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.149136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:52.163651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:52.165213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:52.165228Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:52.165231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:52.165236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:52.165245Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 395 [2:7559341264473739174:4754]
2025-10-09T21:44:52.165263Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:52.165286Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:52.165289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.165292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.165295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.165316Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:52.165323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:52.165330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:52.165400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.165411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.165413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.165416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:52.165492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:52.165495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.165511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.165523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.165541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:52.165562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341264473739167:4754]
2025-10-09T21:44:52.165565Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341264473739167:4754]
2025-10-09T21:44:52.165567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341264473739167:4754]
2025-10-09T21:44:52.165583Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:52.165598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:52.165619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:52.165631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:52.165673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341264473739167:4754]
2025-10-09T21:44:52.165681Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:52.166134Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:52.166145Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:52.166147Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:52.166218Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:52.166881Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 395
2025-10-09T21:44:52.166921Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 395
2025-10-09T21:44:52.168510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341264835550771:5163], now have 1 active actors on pipe
2025-10-09T21:44:52.168528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341264835550560:2432], now have 1 active actors on pipe
2025-10-09T21:44:52.168561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:52.168574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:52.168624Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|426394f-ca38159f-e9ccaaa5-3cf72a89_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:52.168639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.168645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.168648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:52.168664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.168694Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:52.168720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:52.170407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:52.170422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:52.170454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:52.170650Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 396 partition: 0 MaxSeqNo: 25 sessionId: src_id|426394f-ca38159f-e9ccaaa5-3cf72a89_0
2025-10-09T21:44:52.171190Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3ccfc41-1a42735a-9841f5fc-e22cb329_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|426394f-ca38159f-e9ccaaa5-3cf72a89_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:52.176432Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|426394f-ca38159f-e9ccaaa5-3cf72a89_0] Write session: try to update token
2025-10-09T21:44:52.176457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|426394f-ca38159f-e9ccaaa5-3cf72a89_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:52.199663Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:52.199688Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:52.199691Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:52.199694Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:52.199706Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341264473739167:4754]
2025-10-09T21:44:52.199710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341264473739167:4754]
2025-10-09T21:44:52.199714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341264473739167:4754]
2025-10-09T21:44:52.199759Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:52.199764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:52.199772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.199775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:52.199763Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:52.199786Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:52.199922Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:52.202023Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341264835550787:4549]
2025-10-09T21:44:52.202842Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 395, pipe: [1:7559341264835550787:4549]
2025-10-09T21:44:52.214257Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 396 sessionId: src_id|426394f-ca38159f-e9ccaaa5-3cf72a89_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:52.216970Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:52.387268Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:52.387711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341264835550787:4549], now have 1 active actors on pipe
2025-10-09T21:44:52.387767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:52.387775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:52.388312Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:52.390594Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341264835550787:4549]
2025-10-09T21:44:52.390635Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:52.390707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 396 reason: status is not ok: tablet will be restarted right now sessionId: src_id|426394f-ca38159f-e9ccaaa5-3cf72a89_0
2025-10-09T21:44:52.390891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 396 sessionId: src_id|426394f-ca38159f-e9ccaaa5-3cf72a89_0 is DEAD
2025-10-09T21:44:52.391118Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:52.394169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|426394f-ca38159f-e9ccaaa5-3cf72a89_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:52.394183Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|426394f-ca38159f-e9ccaaa5-3cf72a89_0] Write session will restart in 0.008859s
2025-10-09T21:44:52.394228Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|426394f-ca38159f-e9ccaaa5-3cf72a89_0] Write session: Do CDS request
2025-10-09T21:44:52.394235Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|426394f-ca38159f-e9ccaaa5-3cf72a89_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:52.404657Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|426394f-ca38159f-e9ccaaa5-3cf72a89_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:52.404754Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:52.404775Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 397
2025-10-09T21:44:52.407809Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 397 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:52.407865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 397 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:52.407871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=397 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:52.407875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:52.410136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 397 sessionId:  describe result for acl check
2025-10-09T21:44:52.410187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:52.410193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:52.410196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:52.410203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341264835550857:5173] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:52.410803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341264835550857:5173] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:52.417873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341264835550857:5173] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:52.425867Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341264835550857:5173] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:52.425890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341264835550857:5173] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:52.425894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341264835550857:5173] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:52.425903Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 397 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:52.437589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:52.440019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:52.440278Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:52.440291Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:52.440306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:52.440393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:52.440443Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341264473739431:4767]
2025-10-09T21:44:52.441036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:52.441356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:52.441408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:52.441759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:52.441829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:52.441836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:52.441838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:52.441839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:52.441840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:52.441841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:52.441842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.441843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.441844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.441850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:52.441861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:52.441865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:52.441869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:52.441873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:52.441878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:52.441882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.441886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.441889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.441897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:52.441907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:52.441913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:52.441918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:52.441923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:52.441929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:52.441935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:52.441940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:52.441949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:52.441954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:52.441958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:52.441967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:52.441972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.441977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.441981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.441986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.441993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.441998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.442025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:52.450470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:52.451701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:52.451723Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:52.451726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:52.451731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:52.451739Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 396 [2:7559341264473739431:4767]
2025-10-09T21:44:52.452090Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:52.452122Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:52.452125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.452128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.452130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.452152Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:52.452162Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:52.452172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:52.452245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.452253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.452256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.452259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:52.452330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:52.452333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.452345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.452354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.452368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:52.452388Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341264473739426:4767]
2025-10-09T21:44:52.452396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341264473739426:4767]
2025-10-09T21:44:52.452398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341264473739426:4767]
2025-10-09T21:44:52.452409Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:52.452416Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:52.452418Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:52.452420Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:52.452426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:52.452656Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 396
2025-10-09T21:44:52.452440Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:52.452447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:52.452458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341264835550976:5173], now have 1 active actors on pipe
2025-10-09T21:44:52.452461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341264835550808:2432], now have 1 active actors on pipe
2025-10-09T21:44:52.452500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341264473739426:4767]
2025-10-09T21:44:52.452506Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:52.452569Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:52.452941Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 396
2025-10-09T21:44:52.452813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:52.452824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:52.452852Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5a7e2a6c-d205f22c-26dc7f78-f96ab63e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:52.452863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.452867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.452870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:52.452881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.452892Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:52.452910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:52.453258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:52.453265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:52.453284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:52.453492Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 397 partition: 0 MaxSeqNo: 25 sessionId: src_id|5a7e2a6c-d205f22c-26dc7f78-f96ab63e_0
2025-10-09T21:44:52.453855Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|426394f-ca38159f-e9ccaaa5-3cf72a89_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5a7e2a6c-d205f22c-26dc7f78-f96ab63e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:52.456698Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a7e2a6c-d205f22c-26dc7f78-f96ab63e_0] Write session: try to update token
2025-10-09T21:44:52.456719Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a7e2a6c-d205f22c-26dc7f78-f96ab63e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:52.480176Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:52.480195Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:52.480198Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:52.480200Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:52.480209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341264473739426:4767]
2025-10-09T21:44:52.480213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341264473739426:4767]
2025-10-09T21:44:52.480217Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341264473739426:4767]
2025-10-09T21:44:52.480263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:52.480264Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:52.480269Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:52.480270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.480273Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:52.480273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:52.480457Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:52.484200Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 397 sessionId: src_id|5a7e2a6c-d205f22c-26dc7f78-f96ab63e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:52.486770Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:52.489752Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341264835551035:4549]
2025-10-09T21:44:52.490140Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 396, pipe: [1:7559341264835551035:4549]
2025-10-09T21:44:52.642937Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:52.643540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:52.643559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:52.643586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341264835551035:4549], now have 1 active actors on pipe
2025-10-09T21:44:52.644357Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341264835551035:4549]
2025-10-09T21:44:52.644402Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:52.644405Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:52.644464Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 397 reason: error at write: Disconnected sessionId: src_id|5a7e2a6c-d205f22c-26dc7f78-f96ab63e_0
2025-10-09T21:44:52.644660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 397 sessionId: src_id|5a7e2a6c-d205f22c-26dc7f78-f96ab63e_0 is DEAD
2025-10-09T21:44:52.644002Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:52.644837Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a7e2a6c-d205f22c-26dc7f78-f96ab63e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:52.644794Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:52.644854Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a7e2a6c-d205f22c-26dc7f78-f96ab63e_0] Write session will restart in 0.006927s
2025-10-09T21:44:52.644909Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a7e2a6c-d205f22c-26dc7f78-f96ab63e_0] Write session: Do CDS request
2025-10-09T21:44:52.644916Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a7e2a6c-d205f22c-26dc7f78-f96ab63e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:52.652685Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a7e2a6c-d205f22c-26dc7f78-f96ab63e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:52.652771Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:52.652792Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 398
2025-10-09T21:44:52.652976Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 398 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:52.653018Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 398 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:52.653032Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=398 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:52.653037Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:52.653416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 398 sessionId:  describe result for acl check
2025-10-09T21:44:52.653453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:52.653455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:52.653457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:52.653463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341264835551053:5179] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:52.654008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341264835551053:5179] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:52.660374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341264835551053:5179] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:52.666825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341264835551053:5179] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:52.666844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341264835551053:5179] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:52.666848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341264835551053:5179] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:52.666858Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 398 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:52.675487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:52.678670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:52.679099Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:52.679125Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:52.679145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:52.679241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:52.679294Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341264473739651:4771]
2025-10-09T21:44:52.679840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:52.680584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:52.680651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:52.680968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:52.681050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:52.681051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:52.681052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:52.681053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:52.681054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:52.681055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:52.681056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.681057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.681058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.681064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:52.681075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:52.681079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:52.681082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:52.681086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:52.681089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:52.681093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.681096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.681099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.681107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:52.681118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:52.681123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:52.681129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:52.681134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:52.681138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:52.681142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:52.681146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:52.681151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:52.681155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:52.681159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:52.681163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:52.681168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.681173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.681177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.681181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.681185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.681191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.681217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:52.689077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:52.690539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:52.690556Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:52.690559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:52.690563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:52.690571Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 397 [2:7559341264473739651:4771]
2025-10-09T21:44:52.690587Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:52.690608Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:52.690611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.690614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.690616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.690636Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:52.690648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:52.690656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:52.690738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.690745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.690747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.690749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:52.690827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:52.690836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.690864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.690880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.690897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:52.690978Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 397
2025-10-09T21:44:52.691015Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 397
2025-10-09T21:44:52.690933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341264835551148:5179], now have 1 active actors on pipe
2025-10-09T21:44:52.690941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341264835551040:2432], now have 1 active actors on pipe
2025-10-09T21:44:52.690949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341264473739646:4771]
2025-10-09T21:44:52.690951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341264473739646:4771]
2025-10-09T21:44:52.690953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341264473739646:4771]
2025-10-09T21:44:52.690965Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:52.690979Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:52.691001Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:52.691002Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:52.691004Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:52.691005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:52.691006Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:52.691064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:52.691069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:52.691093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341264473739646:4771]
2025-10-09T21:44:52.691101Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:52.691110Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:52.691131Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c983b548-65365a3-349f1c94-3fd8debf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:52.691143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.691147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.691152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:52.691163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.691177Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:52.691208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:52.691508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:52.691523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:52.691546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:52.691648Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 398 partition: 0 MaxSeqNo: 25 sessionId: src_id|c983b548-65365a3-349f1c94-3fd8debf_0
2025-10-09T21:44:52.692114Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a7e2a6c-d205f22c-26dc7f78-f96ab63e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c983b548-65365a3-349f1c94-3fd8debf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:52.694891Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c983b548-65365a3-349f1c94-3fd8debf_0] Write session: try to update token
2025-10-09T21:44:52.694912Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c983b548-65365a3-349f1c94-3fd8debf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:52.711084Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:52.711106Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:52.711108Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:52.711110Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:52.711121Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341264473739646:4771]
2025-10-09T21:44:52.711125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341264473739646:4771]
2025-10-09T21:44:52.711130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341264473739646:4771]
2025-10-09T21:44:52.711178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:52.711187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.711190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:52.711337Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:52.712023Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 398 sessionId: src_id|c983b548-65365a3-349f1c94-3fd8debf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:52.714115Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:52.714128Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:52.714133Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:52.714593Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:52.745104Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341264835551267:4549]
2025-10-09T21:44:52.746276Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 397, pipe: [1:7559341264835551267:4549]
2025-10-09T21:44:52.786120Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] Counters: { Errors: 3 CurrentSessionLifetimeMs: 121237 BytesRead: 24780812 MessagesRead: 11 BytesReadCompressed: 24780812 BytesInflightUncompressed: 2252802 BytesInflightCompressed: 0 BytesInflightTotal: 2252802 MessagesInflight: 1 }
2025-10-09T21:44:52.870394Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:52.870876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:52.870893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:52.870943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341264835551267:4549], now have 1 active actors on pipe
2025-10-09T21:44:52.871312Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:52.871340Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:52.871411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 398 reason: error at write: Disconnected sessionId: src_id|c983b548-65365a3-349f1c94-3fd8debf_0
2025-10-09T21:44:52.871615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 398 sessionId: src_id|c983b548-65365a3-349f1c94-3fd8debf_0 is DEAD
2025-10-09T21:44:52.871657Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341264835551267:4549]
2025-10-09T21:44:52.871805Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c983b548-65365a3-349f1c94-3fd8debf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:52.871817Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c983b548-65365a3-349f1c94-3fd8debf_0] Write session will restart in 0.009758s
2025-10-09T21:44:52.871843Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c983b548-65365a3-349f1c94-3fd8debf_0] Write session: Do CDS request
2025-10-09T21:44:52.871849Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c983b548-65365a3-349f1c94-3fd8debf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:52.871681Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:52.872299Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:52.882579Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c983b548-65365a3-349f1c94-3fd8debf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:52.882916Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:52.882936Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 399
2025-10-09T21:44:52.883141Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 399 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:52.883204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 399 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:52.883225Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=399 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:52.883233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:52.883693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 399 sessionId:  describe result for acl check
2025-10-09T21:44:52.883743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:52.883745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:52.883746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:52.883752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341264835551328:5183] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:52.884353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341264835551328:5183] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:52.888163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341264835551328:5183] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:52.892197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341264835551328:5183] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:52.892217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341264835551328:5183] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:52.892221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341264835551328:5183] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:52.892233Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 399 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:52.894432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:52.897474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:52.897940Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:52.897955Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:52.897971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:52.898073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:52.898133Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341264473739871:4773]
2025-10-09T21:44:52.898769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:52.899241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:52.899304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:52.899641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:52.899736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:52.899743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:52.899744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:52.899746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:52.899747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:52.899748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:52.899750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.899751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.899752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.899759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:52.899770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:52.899775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:52.899778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:52.899782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:52.899786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:52.899790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.899794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.899798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.899806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:52.899815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:52.899821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:52.899827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:52.899844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:52.899850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:52.899854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:52.899861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:52.899865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:52.899871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:52.899875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:52.899880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:52.899887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.899892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.899897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.899902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.899907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.899912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.899942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:52.908151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:52.909422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:52.909445Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:52.909447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:52.909452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:52.909460Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 398 [2:7559341264473739871:4773]
2025-10-09T21:44:52.909476Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:52.909495Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:52.909498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.909501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.909503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.909521Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:52.909533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:52.909541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:52.909617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.909625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.909627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.909630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:52.909702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:52.909710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.909723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:52.909738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:52.909756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:52.909786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341264835551274:2432], now have 1 active actors on pipe
2025-10-09T21:44:52.909794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341264835551486:5183], now have 1 active actors on pipe
2025-10-09T21:44:52.909803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341264473739866:4773]
2025-10-09T21:44:52.909811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341264473739866:4773]
2025-10-09T21:44:52.909813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341264473739866:4773]
2025-10-09T21:44:52.909828Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:52.909842Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:52.909852Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:52.909856Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:52.909858Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:52.909861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:52.909865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:52.909881Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 398
2025-10-09T21:44:52.909908Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341264473739866:4773]
2025-10-09T21:44:52.909929Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 398
2025-10-09T21:44:52.909921Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:52.909928Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:52.911958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:52.911977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:52.912033Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1a6c440d-c058e4d-e274b2ca-81525097_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:52.912052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.912073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.912076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:52.912089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.912110Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:52.912157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:52.912410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:52.912437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:52.912461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:52.913210Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c983b548-65365a3-349f1c94-3fd8debf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1a6c440d-c058e4d-e274b2ca-81525097_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:52.912773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 399 partition: 0 MaxSeqNo: 25 sessionId: src_id|1a6c440d-c058e4d-e274b2ca-81525097_0
2025-10-09T21:44:52.915828Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a6c440d-c058e4d-e274b2ca-81525097_0] Write session: try to update token
2025-10-09T21:44:52.915849Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a6c440d-c058e4d-e274b2ca-81525097_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:52.934684Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 399 sessionId: src_id|1a6c440d-c058e4d-e274b2ca-81525097_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:52.937333Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:52.940127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:52.940154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:52.940178Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:52.940181Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:52.940192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341264473739866:4773]
2025-10-09T21:44:52.940255Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:52.940255Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:52.940428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:52.940555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:52.940682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:52.940812Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:52.940988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:52.941037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:52.941062Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:52.941158Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:52.951256Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:52.951278Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:52.951280Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:52.951287Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:52.951298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341264473739866:4773]
2025-10-09T21:44:52.951301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341264473739866:4773]
2025-10-09T21:44:52.951305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341264473739866:4773]
2025-10-09T21:44:52.951353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:52.951359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:52.951361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:52.951508Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:52.951999Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:52.952011Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:52.952016Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:52.972012Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341264835551510:4549]
2025-10-09T21:44:52.972481Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 398, pipe: [1:7559341264835551510:4549]
2025-10-09T21:44:53.117608Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:53.127848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:53.127877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:53.128052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:53.128123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:53.128190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:53.128269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:53.128666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:53.128672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:53.128745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:53.128807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:53.128871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:53.128939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:53.128981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:53.128985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:53.129056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:53.129123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:53.129192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:53.129253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:53.129289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:53.129293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:53.129363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:53.129429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:53.129491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:53.129554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:53.129592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:53.129595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:53.129668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:53.129729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:53.129795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:53.129859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:53.129908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:53.129911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:53.129976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:53.130538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:53.132410Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341264835551510:4549]
2025-10-09T21:44:53.132461Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:53.132468Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:53.132500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 399 reason: error at write: Disconnected sessionId: src_id|1a6c440d-c058e4d-e274b2ca-81525097_0
2025-10-09T21:44:53.132701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 399 sessionId: src_id|1a6c440d-c058e4d-e274b2ca-81525097_0 is DEAD
2025-10-09T21:44:53.132852Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a6c440d-c058e4d-e274b2ca-81525097_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:53.132863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a6c440d-c058e4d-e274b2ca-81525097_0] Write session will restart in 0.009262s
2025-10-09T21:44:53.132897Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a6c440d-c058e4d-e274b2ca-81525097_0] Write session: Do CDS request
2025-10-09T21:44:53.132903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a6c440d-c058e4d-e274b2ca-81525097_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:53.132851Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:53.130618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:53.130688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:53.130729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:53.130734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:53.130808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:53.130866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:53.130944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:53.131006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:53.131044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:53.131048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:53.131119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:53.131184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:53.131245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:53.131307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:53.131366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:53.131370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:53.131438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:53.131490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:53.131551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:53.131614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:53.131652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:53.131655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:53.131720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:53.131779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:53.131841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:53.131905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:53.131950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:53.131954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:53.132029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341264835551510:4549], now have 1 active actors on pipe
2025-10-09T21:44:53.132783Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:53.143872Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a6c440d-c058e4d-e274b2ca-81525097_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:53.146344Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:53.146363Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 400
2025-10-09T21:44:53.146621Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 400 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:53.146663Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 400 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:53.146670Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=400 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:53.146674Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:53.147192Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 400 sessionId:  describe result for acl check
2025-10-09T21:44:53.147245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:53.147247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:53.147249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:53.147255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341269130518865:5188] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:53.147831Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341269130518865:5188] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:53.152521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341269130518865:5188] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:53.157579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341269130518865:5188] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:53.157615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341269130518865:5188] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:53.157619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341269130518865:5188] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:53.157632Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 400 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:53.160455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:53.161347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:53.161578Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:53.161589Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:53.161602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:53.161672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:53.161719Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341268768707390:4776]
2025-10-09T21:44:53.162357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:53.164247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:53.164305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:53.164671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:53.164742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:53.164748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:53.164750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:53.164751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:53.164752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:53.164754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:53.164755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.164756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.164757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.164764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:53.164774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:53.164778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:53.164781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:53.164784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:53.164789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:53.164792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.164796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.164799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.164808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:53.164816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:53.164823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:53.164827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:53.164831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:53.164836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:53.164840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:53.164845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:53.164850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:53.164854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:53.164858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:53.164863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:53.164868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.164874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.164879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.164885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.164889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.164894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.164923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:53.173596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:53.175164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:53.175183Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:53.175186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:53.175191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:53.175200Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 399 [2:7559341268768707390:4776]
2025-10-09T21:44:53.175217Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:53.175238Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:53.175242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.175245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.175247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.175269Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:53.175276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:53.175284Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:53.175357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.175360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.175362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.175365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:53.175445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:53.175448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.175462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.175474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.175490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:53.175513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341268768707385:4776]
2025-10-09T21:44:53.175516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341268768707385:4776]
2025-10-09T21:44:53.175518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341268768707385:4776]
2025-10-09T21:44:53.175534Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:53.175549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:53.175563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:53.175566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:53.175610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341268768707385:4776]
2025-10-09T21:44:53.175612Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:53.175670Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:53.175673Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:53.175675Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:53.175682Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:53.176258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341269130518812:2432], now have 1 active actors on pipe
2025-10-09T21:44:53.176265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341269130519023:5188], now have 1 active actors on pipe
2025-10-09T21:44:53.179678Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 399
2025-10-09T21:44:53.179723Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 399
2025-10-09T21:44:53.180687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:53.180704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:53.180744Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7557287e-36f8b8ac-9682501a-def6436a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:53.180764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.180769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.180772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:53.180787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.180806Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:53.180830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:53.181206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:53.181216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:53.181232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:53.181398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 400 partition: 0 MaxSeqNo: 25 sessionId: src_id|7557287e-36f8b8ac-9682501a-def6436a_0
2025-10-09T21:44:53.181879Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a6c440d-c058e4d-e274b2ca-81525097_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7557287e-36f8b8ac-9682501a-def6436a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:53.184454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7557287e-36f8b8ac-9682501a-def6436a_0] Write session: try to update token
2025-10-09T21:44:53.184475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7557287e-36f8b8ac-9682501a-def6436a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:53.199405Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:53.199419Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:53.199424Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:53.199426Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:53.199435Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341268768707385:4776]
2025-10-09T21:44:53.199440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341268768707385:4776]
2025-10-09T21:44:53.199443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341268768707385:4776]
2025-10-09T21:44:53.199492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:53.199499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.199502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:53.199663Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:53.202977Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:53.202987Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:53.202991Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:53.221332Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 400 sessionId: src_id|7557287e-36f8b8ac-9682501a-def6436a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:53.223915Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:53.232659Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341269130519042:4549]
2025-10-09T21:44:53.233105Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 399, pipe: [1:7559341269130519042:4549]
2025-10-09T21:44:53.371348Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:53.371648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:53.371659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:53.371684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341269130519042:4549], now have 1 active actors on pipe
2025-10-09T21:44:53.372957Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:53.372989Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:53.373019Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341269130519042:4549]
2025-10-09T21:44:53.373041Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:53.373062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 400 reason: error at write: Disconnected sessionId: src_id|7557287e-36f8b8ac-9682501a-def6436a_0
2025-10-09T21:44:53.373265Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 400 sessionId: src_id|7557287e-36f8b8ac-9682501a-def6436a_0 is DEAD
2025-10-09T21:44:53.373441Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7557287e-36f8b8ac-9682501a-def6436a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:53.373455Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7557287e-36f8b8ac-9682501a-def6436a_0] Write session will restart in 0.006774s
2025-10-09T21:44:53.373483Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7557287e-36f8b8ac-9682501a-def6436a_0] Write session: Do CDS request
2025-10-09T21:44:53.373493Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7557287e-36f8b8ac-9682501a-def6436a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:53.373643Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:53.381692Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7557287e-36f8b8ac-9682501a-def6436a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:53.381762Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:53.381784Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 401
2025-10-09T21:44:53.381946Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 401 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:53.381977Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 401 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:53.381982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=401 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:53.381986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:53.382632Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 401 sessionId:  describe result for acl check
2025-10-09T21:44:53.382663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:53.382665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:53.382667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:53.382673Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341269130519059:5192] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:53.383345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341269130519059:5192] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:53.386568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341269130519059:5192] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:53.393541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341269130519059:5192] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:53.393558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341269130519059:5192] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:53.393562Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341269130519059:5192] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:53.393573Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 401 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:53.407626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:53.418314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:53.418633Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:53.418650Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:53.418664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:53.418737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:53.418791Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341268768707644:4785]
2025-10-09T21:44:53.419433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:53.420002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:53.420090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:53.420428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:53.420488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:53.420489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:53.420491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:53.420492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:53.420493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:53.420494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:53.420495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.420496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.420497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.420503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:53.420515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:53.420519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:53.420524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:53.420528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:53.420533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:53.420537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.420541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.420545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.420554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:53.420563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:53.420570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:53.420576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:53.420582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:53.420587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:53.420591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:53.420597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:53.420648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:53.420654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:53.420658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:53.420665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:53.420670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.420675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.420679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.420684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.420690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.420696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.420727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:53.430135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:53.431232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:53.431253Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:53.431255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:53.431259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:53.431267Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 400 [2:7559341268768707644:4785]
2025-10-09T21:44:53.431283Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:53.431303Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:53.431306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.431309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.431312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.431332Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:53.431339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:53.431361Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:53.431431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.431439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.431441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.431445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:53.431518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:53.431527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.431538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.431549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.431565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:53.431586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341268768707639:4785]
2025-10-09T21:44:53.431588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341268768707639:4785]
2025-10-09T21:44:53.431591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341268768707639:4785]
2025-10-09T21:44:53.431604Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:53.431620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341269130519045:2432], now have 1 active actors on pipe
2025-10-09T21:44:53.431623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341269130519161:5192], now have 1 active actors on pipe
2025-10-09T21:44:53.431632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:53.431647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:53.431650Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:53.431689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341268768707639:4785]
2025-10-09T21:44:53.431691Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:53.431733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:53.431736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:53.431740Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:53.431747Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:53.431764Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 400
2025-10-09T21:44:53.431819Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 400
2025-10-09T21:44:53.431997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:53.432010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:53.432056Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|16c49afd-66b7a99d-89a0e5b3-b893014b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:53.432071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.432075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.432078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:53.432089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.432101Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:53.432121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:53.432373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:53.432386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:53.432409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:53.432540Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 401 partition: 0 MaxSeqNo: 25 sessionId: src_id|16c49afd-66b7a99d-89a0e5b3-b893014b_0
2025-10-09T21:44:53.433155Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7557287e-36f8b8ac-9682501a-def6436a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|16c49afd-66b7a99d-89a0e5b3-b893014b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:53.436475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16c49afd-66b7a99d-89a0e5b3-b893014b_0] Write session: try to update token
2025-10-09T21:44:53.436502Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16c49afd-66b7a99d-89a0e5b3-b893014b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:53.450996Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:53.451016Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:53.451018Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:53.451020Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:53.451031Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341268768707639:4785]
2025-10-09T21:44:53.451035Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341268768707639:4785]
2025-10-09T21:44:53.451039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341268768707639:4785]
2025-10-09T21:44:53.451085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:53.451093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.451095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:53.451277Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:53.451407Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:53.451421Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:53.451424Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:53.464170Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 401 sessionId: src_id|16c49afd-66b7a99d-89a0e5b3-b893014b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:53.466845Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:53.473273Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341269130519287:4549]
2025-10-09T21:44:53.474002Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 400, pipe: [1:7559341269130519287:4549]
2025-10-09T21:44:53.616503Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:53.616536Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341269130519287:4549]
2025-10-09T21:44:53.616568Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:53.616570Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:44:53.616571Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:53.616639Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 401 reason: error at write: Disconnected sessionId: src_id|16c49afd-66b7a99d-89a0e5b3-b893014b_0
2025-10-09T21:44:53.616711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 401 sessionId: src_id|16c49afd-66b7a99d-89a0e5b3-b893014b_0 is DEAD
2025-10-09T21:44:53.616808Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 4 consumer shared/user session shared/user_1_4_257779208761094781_v1 is DEAD
2025-10-09T21:44:53.616932Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:44:53.617088Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:44:53.617094Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:44:53.617104Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.009134s
2025-10-09T21:44:53.615422Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:53.616145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:53.616156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:53.616179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341269130519287:4549], now have 1 active actors on pipe
2025-10-09T21:44:53.616822Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:53.617232Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341148871415703:4545] disconnected.
2025-10-09T21:44:53.617256Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341148871415703:4545] disconnected; active server actors: 1
2025-10-09T21:44:53.617259Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341148871415703:4545] client user disconnected session shared/user_1_4_257779208761094781_v1
2025-10-09T21:44:53.617702Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16c49afd-66b7a99d-89a0e5b3-b893014b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:53.617712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16c49afd-66b7a99d-89a0e5b3-b893014b_0] Write session will restart in 0.006872s
2025-10-09T21:44:53.617749Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16c49afd-66b7a99d-89a0e5b3-b893014b_0] Write session: Do CDS request
2025-10-09T21:44:53.617756Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16c49afd-66b7a99d-89a0e5b3-b893014b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:53.625781Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16c49afd-66b7a99d-89a0e5b3-b893014b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:53.626168Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:53.626187Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 402
2025-10-09T21:44:53.627379Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:44:53.630146Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 402 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:53.630172Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:44:53.630176Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 5
2025-10-09T21:44:53.630204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 402 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:53.630217Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=402 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:53.630221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:53.630401Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 5 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:44:53.630475Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:44:53.630575Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 auth for : user
2025-10-09T21:44:53.630729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 402 sessionId:  describe result for acl check
2025-10-09T21:44:53.630765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:53.630767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:53.630769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:53.630775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341269130519388:5204] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:53.630804Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 Handle describe topics response
2025-10-09T21:44:53.630832Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 auth is DEAD
2025-10-09T21:44:53.630846Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:44:53.631103Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:44:53.631234Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_5_1413312754900983240_v1
2025-10-09T21:44:53.631400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341269130519388:5204] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:53.631512Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341269130519391:5205] connected; active server actors: 1
2025-10-09T21:44:53.631522Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559341269130519391:5205] session shared/user_1_5_1413312754900983240_v1
2025-10-09T21:44:53.631532Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:44:53.631543Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:44:53.631549Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_5_1413312754900983240_v1" (Sender=[1:7559341269130519376:5205], Pipe=[1:7559341269130519391:5205], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:44:53.631553Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:44:53.631563Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:44:53.631572Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_5_1413312754900983240_v1" (Sender=[1:7559341269130519376:5205], Pipe=[1:7559341269130519391:5205], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:44:53.631582Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_5_1413312754900983240_v1" sender [1:7559341269130519376:5205] lock partition 0 for ReadingSession "shared/user_1_5_1413312754900983240_v1" (Sender=[1:7559341269130519376:5205], Pipe=[1:7559341269130519391:5205], Partitions=[], ActiveFamilyCount=1) generation 1 step 5
2025-10-09T21:44:53.631591Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:44:53.631596Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000029s
2025-10-09T21:44:53.631805Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 5 Session: "shared/user_1_5_1413312754900983240_v1" ClientId: "user" PipeClient { RawX1: 7559341269130519391 RawX2: 4503603922342997 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:44:53.631837Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:44:53.640386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341269130519388:5204] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:53.646101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:53.648995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:53.649251Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:53.649262Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:53.649275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:53.649336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:53.649475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341269130519388:5204] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:53.649497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341269130519388:5204] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:53.649501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341269130519388:5204] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:53.649513Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 402 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:53.649662Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341268768707875:4793]
2025-10-09T21:44:53.650340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:53.650811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:53.650888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:53.651262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:53.651325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:53.651327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:53.651328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:53.651330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:53.651331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:53.651332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:53.651334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.651335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.651336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.651343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:53.651355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:53.651360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:53.651364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:53.651369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:53.651373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:53.651377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.651380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.651384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.651405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:53.651413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:53.651421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:53.651427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:53.651432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:53.651437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:53.651442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:53.651453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:53.651457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:53.651462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:53.651468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:53.651474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:53.651485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.651490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.651495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.651500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.651509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.651513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.651539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:53.659177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:53.661171Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 401
2025-10-09T21:44:53.661217Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 401, pipe: [1:7559341269130519405:5210]
2025-10-09T21:44:53.661238Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 401
2025-10-09T21:44:53.660227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:53.660244Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:53.660247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:53.660252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:53.660260Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 401 [2:7559341268768707875:4793]
2025-10-09T21:44:53.660277Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:53.660298Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:53.660301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.660305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.660307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.660326Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:53.660332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:53.660339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:53.660404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.660406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.660408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.660411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:53.660485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:53.660488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.660501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.660512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.660527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:53.660549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341268768707869:4793]
2025-10-09T21:44:53.660551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341268768707869:4793]
2025-10-09T21:44:53.660553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341268768707869:4793]
2025-10-09T21:44:53.660566Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:53.660581Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:53.660595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:53.660598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:53.660637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341268768707869:4793]
2025-10-09T21:44:53.660639Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:53.660727Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:53.660729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:53.660731Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:53.660736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:53.661948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341269130519526:5204], now have 1 active actors on pipe
2025-10-09T21:44:53.661960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341269130519405:5210], now have 1 active actors on pipe
2025-10-09T21:44:53.661963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341269130519301:2432], now have 1 active actors on pipe
2025-10-09T21:44:53.661986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:53.661992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:53.662009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:53.662017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:53.662023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_5_1413312754900983240_v1 on pipe: [1:7559341269130519405:5210]
2025-10-09T21:44:53.662075Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_5_1413312754900983240_v1:1 with generation 401
2025-10-09T21:44:53.662083Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|26cf8269-9158e9d3-718aca94-99ec48c7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:53.662093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.662097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.662100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:53.662112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.662127Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:53.662138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.662143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.662145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:53.662160Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_5_1413312754900983240_v1
2025-10-09T21:44:53.662163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.662193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:53.662222Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:44:53.662666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:53.662673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:53.662689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:53.662846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 402 partition: 0 MaxSeqNo: 25 sessionId: src_id|26cf8269-9158e9d3-718aca94-99ec48c7_0
2025-10-09T21:44:53.663261Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16c49afd-66b7a99d-89a0e5b3-b893014b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|26cf8269-9158e9d3-718aca94-99ec48c7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:53.666168Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26cf8269-9158e9d3-718aca94-99ec48c7_0] Write session: try to update token
2025-10-09T21:44:53.666190Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26cf8269-9158e9d3-718aca94-99ec48c7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:53.666521Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046293661 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:44:53.666547Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:44:53.666585Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 sending to client partition status
2025-10-09T21:44:53.666240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:44:53.666276Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:44:53.666300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.666303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.666306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.666316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:44:53.666344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:44:53.681895Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 402 sessionId: src_id|26cf8269-9158e9d3-718aca94-99ec48c7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:53.679401Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:53.679421Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:53.679424Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:53.679426Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:53.679437Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341268768707869:4793]
2025-10-09T21:44:53.679442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341268768707869:4793]
2025-10-09T21:44:53.679446Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341268768707869:4793]
2025-10-09T21:44:53.679495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:53.679503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.679506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:53.679644Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:53.682278Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:53.682305Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:53.682308Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:53.684372Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:53.850679Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:53.851055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:53.851066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:53.851574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_5_1413312754900983240_v1
2025-10-09T21:44:53.851591Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:53.852185Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_5_1413312754900983240_v1
2025-10-09T21:44:53.854090Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341269130519405:5210]
2025-10-09T21:44:53.854125Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:53.854168Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:53.854245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 402 reason: status is not ok: tablet will be restarted right now sessionId: src_id|26cf8269-9158e9d3-718aca94-99ec48c7_0
2025-10-09T21:44:53.854439Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 402 sessionId: src_id|26cf8269-9158e9d3-718aca94-99ec48c7_0 is DEAD
2025-10-09T21:44:53.854664Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26cf8269-9158e9d3-718aca94-99ec48c7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:53.854675Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26cf8269-9158e9d3-718aca94-99ec48c7_0] Write session will restart in 0.009760s
2025-10-09T21:44:53.854703Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26cf8269-9158e9d3-718aca94-99ec48c7_0] Write session: Do CDS request
2025-10-09T21:44:53.854709Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26cf8269-9158e9d3-718aca94-99ec48c7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:53.866045Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26cf8269-9158e9d3-718aca94-99ec48c7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:53.866123Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:53.866143Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 403
2025-10-09T21:44:53.867012Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 403 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:53.867068Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 403 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:53.867079Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=403 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:53.867083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:53.867595Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 403 sessionId:  describe result for acl check
2025-10-09T21:44:53.867641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:53.867644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:53.867646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:53.867653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341269130519554:5212] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:53.868237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341269130519554:5212] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:53.872287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341269130519554:5212] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:53.878122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341269130519554:5212] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:53.878144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341269130519554:5212] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:53.878148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341269130519554:5212] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:53.878158Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 403 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:53.894414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:53.897340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:53.897603Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:53.897616Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:53.897629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:53.897701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:53.898114Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341268768708103:4795]
2025-10-09T21:44:53.898720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:53.899172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:53.899248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:53.899608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:53.899682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:53.899691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:53.899693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:53.899694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:53.899695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:53.899696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:53.899698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.899699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.899700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.899707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:53.899718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:53.899722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:53.899725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:53.899729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:53.899733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:53.899736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.899740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.899743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.899751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:53.899759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:53.899780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:53.899786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:53.899789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:53.899794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:53.899801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:53.899807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:53.899811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:53.899816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:53.899821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:53.899826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:53.899831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.899837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.899841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.899846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.899851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.899856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.899896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:53.912342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:53.915085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:53.915103Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:53.915106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:53.915111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:53.915121Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 402 [2:7559341268768708103:4795]
2025-10-09T21:44:53.915143Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:53.915164Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:53.915167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.915170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.915173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.915195Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:53.915202Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:53.915210Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:53.915291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.915302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.915304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.915307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:53.915679Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 402
2025-10-09T21:44:53.915734Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 402
2025-10-09T21:44:53.915390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:53.915509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.915526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:53.915539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:53.915557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:53.915583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341269130519640:5212], now have 1 active actors on pipe
2025-10-09T21:44:53.915587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341269130519542:2432], now have 1 active actors on pipe
2025-10-09T21:44:53.915596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341268768708098:4795]
2025-10-09T21:44:53.915601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341268768708098:4795]
2025-10-09T21:44:53.915604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341268768708098:4795]
2025-10-09T21:44:53.915618Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:53.915635Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:53.915650Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:53.915654Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:53.915714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341268768708098:4795]
2025-10-09T21:44:53.915716Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:53.915759Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:53.915763Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:53.915765Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:53.915772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:53.916081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:53.916090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:53.916119Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|52dba9c5-6a856586-63228634-ade36f2e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:53.916130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.916135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.916138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:53.916151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.916165Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:53.916184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:53.916590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:53.916606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:53.916630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:53.916760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 403 partition: 0 MaxSeqNo: 25 sessionId: src_id|52dba9c5-6a856586-63228634-ade36f2e_0
2025-10-09T21:44:53.917194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26cf8269-9158e9d3-718aca94-99ec48c7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|52dba9c5-6a856586-63228634-ade36f2e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:53.919879Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52dba9c5-6a856586-63228634-ade36f2e_0] Write session: try to update token
2025-10-09T21:44:53.919900Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52dba9c5-6a856586-63228634-ade36f2e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:53.935058Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:53.935080Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:53.935084Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:53.935086Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:53.935096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341268768708098:4795]
2025-10-09T21:44:53.935101Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341268768708098:4795]
2025-10-09T21:44:53.935105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341268768708098:4795]
2025-10-09T21:44:53.935154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:53.935167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:53.935169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:53.935309Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:53.938126Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:53.938142Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:53.938147Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:53.941941Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 403 sessionId: src_id|52dba9c5-6a856586-63228634-ade36f2e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:53.945964Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:53.954006Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341269130519777:5210]
2025-10-09T21:44:53.955222Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 402, pipe: [1:7559341269130519777:5210]
2025-10-09T21:44:54.103758Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:54.104463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:54.104483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:54.104531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341269130519777:5210], now have 1 active actors on pipe
2025-10-09T21:44:54.104773Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341269130519777:5210]
2025-10-09T21:44:54.104829Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:54.104832Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:54.104902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 403 reason: error at write: Disconnected sessionId: src_id|52dba9c5-6a856586-63228634-ade36f2e_0
2025-10-09T21:44:54.105088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 403 sessionId: src_id|52dba9c5-6a856586-63228634-ade36f2e_0 is DEAD
2025-10-09T21:44:54.105216Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:54.104793Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:54.105469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52dba9c5-6a856586-63228634-ade36f2e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:54.105479Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52dba9c5-6a856586-63228634-ade36f2e_0] Write session will restart in 0.007535s
2025-10-09T21:44:54.105519Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52dba9c5-6a856586-63228634-ade36f2e_0] Write session: Do CDS request
2025-10-09T21:44:54.105526Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52dba9c5-6a856586-63228634-ade36f2e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:54.114580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52dba9c5-6a856586-63228634-ade36f2e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:54.115024Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:54.115046Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 404
2025-10-09T21:44:54.115323Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 404 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:54.115388Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 404 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:54.115410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=404 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:54.115420Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:54.115963Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 404 sessionId:  describe result for acl check
2025-10-09T21:44:54.116009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:54.116032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:54.116039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:54.116045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341273425487099:5217] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:54.116668Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341273425487099:5217] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:54.125270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341273425487099:5217] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:54.131597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341273425487099:5217] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:54.131619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341273425487099:5217] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:54.131623Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341273425487099:5217] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:54.131674Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 404 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:54.139202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:54.143825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:54.144105Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:54.144119Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:54.144137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:54.144212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:54.144267Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341273063675626:4798]
2025-10-09T21:44:54.145285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:54.145750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:54.145820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:54.146215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:54.146318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:54.146320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:54.146321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:54.146323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:54.146324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:54.146325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:54.146327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.146328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.146329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.146335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:54.146348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:54.146352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:54.146357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:54.146361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:54.146365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:54.146369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.146374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.146378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.146386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:54.146395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:54.146403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:54.146408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:54.146412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:54.146418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:54.146422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:54.146428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:54.146433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:54.146439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:54.146444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:54.146450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:54.146468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.146481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.146486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.146491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.146499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.146504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.146546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:54.157717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:54.159201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:54.159220Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:54.159223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:54.159228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:54.159237Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 403 [2:7559341273063675626:4798]
2025-10-09T21:44:54.159268Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:54.159289Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:54.159292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.159296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.159298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.159319Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:54.159325Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:54.159333Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:54.159403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.159405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.159407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.159410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:54.159487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:54.159490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.159505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.159517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.159536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:54.159558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341273063675620:4798]
2025-10-09T21:44:54.159560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341273063675620:4798]
2025-10-09T21:44:54.159562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341273063675620:4798]
2025-10-09T21:44:54.159580Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:54.159595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:54.159610Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:54.159613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:54.159640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:54.159644Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:54.159646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:54.159660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341273063675620:4798]
2025-10-09T21:44:54.159663Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:54.159735Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:54.159841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341273425487079:2432], now have 1 active actors on pipe
2025-10-09T21:44:54.159847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341273425487265:5217], now have 1 active actors on pipe
2025-10-09T21:44:54.160289Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 403
2025-10-09T21:44:54.160303Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 403
2025-10-09T21:44:54.160704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:54.160721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:54.160755Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|412cca8a-3b6f383-9722dba8-78d99314_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:54.160766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.160771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.160774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:54.160787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.160802Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:54.160821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:54.161335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:54.161349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:54.161371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:54.161538Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 404 partition: 0 MaxSeqNo: 25 sessionId: src_id|412cca8a-3b6f383-9722dba8-78d99314_0
2025-10-09T21:44:54.161970Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52dba9c5-6a856586-63228634-ade36f2e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|412cca8a-3b6f383-9722dba8-78d99314_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:54.164689Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|412cca8a-3b6f383-9722dba8-78d99314_0] Write session: try to update token
2025-10-09T21:44:54.164713Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|412cca8a-3b6f383-9722dba8-78d99314_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:54.186227Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 404 sessionId: src_id|412cca8a-3b6f383-9722dba8-78d99314_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:54.188808Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:54.189440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:54.189465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:54.189518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:54.189791Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:54.202683Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:54.202704Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:54.202707Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:54.202709Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:54.202720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341273063675620:4798]
2025-10-09T21:44:54.202725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341273063675620:4798]
2025-10-09T21:44:54.202732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341273063675620:4798]
2025-10-09T21:44:54.202781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:54.202798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.202801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:54.204998Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341273425487312:5210]
2025-10-09T21:44:54.205841Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 403, pipe: [1:7559341273425487312:5210]
2025-10-09T21:44:54.202951Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:54.204284Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:54.204290Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:54.204294Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:54.429302Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:54.429557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341273425487312:5210], now have 1 active actors on pipe
2025-10-09T21:44:54.437652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:54.437684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:54.437850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.437916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.437981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.438063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.438103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.438108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.438172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.438242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.438326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.438398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.438435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.438439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.438505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.438564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.438624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.438682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.438719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.438722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.438786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.438844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.438902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.438958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.438996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.438999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.439077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.439139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.439204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.439264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.439311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.439314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.439377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.439429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.439483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.439541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.439581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.439583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.439644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.439697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.439756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.439814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.439855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.439859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.439919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.439975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.440034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.440092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.440137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.440140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.440205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.440262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.440318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.440375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.440417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.440420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.440480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.440537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.440591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.440647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.440687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.440689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.441148Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:54.441215Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341273425487312:5210]
2025-10-09T21:44:54.441243Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:54.441246Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:54.441276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 404 reason: error at write: Disconnected sessionId: src_id|412cca8a-3b6f383-9722dba8-78d99314_0
2025-10-09T21:44:54.441472Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 404 sessionId: src_id|412cca8a-3b6f383-9722dba8-78d99314_0 is DEAD
2025-10-09T21:44:54.442069Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:54.443468Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|412cca8a-3b6f383-9722dba8-78d99314_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:54.443479Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|412cca8a-3b6f383-9722dba8-78d99314_0] Write session will restart in 0.006638s
2025-10-09T21:44:54.443504Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|412cca8a-3b6f383-9722dba8-78d99314_0] Write session: Do CDS request
2025-10-09T21:44:54.443511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|412cca8a-3b6f383-9722dba8-78d99314_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:54.451674Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|412cca8a-3b6f383-9722dba8-78d99314_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:54.451798Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:54.451818Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 405
2025-10-09T21:44:54.452120Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 405 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:54.452156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 405 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:54.452161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=405 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:54.452165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:54.452623Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 405 sessionId:  describe result for acl check
2025-10-09T21:44:54.452663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:54.452671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:54.452673Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:54.452679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341273425487329:5221] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:54.453245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341273425487329:5221] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:54.459115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341273425487329:5221] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:54.468075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341273425487329:5221] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:54.468097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341273425487329:5221] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:54.468101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341273425487329:5221] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:54.468110Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 405 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:54.499899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:54.503388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:54.503716Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:54.503729Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:54.503744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:54.503814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:54.503853Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341273063675885:4807]
2025-10-09T21:44:54.504453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:54.504924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:54.504974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:54.505283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:54.505333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:54.505334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:54.505336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:54.505356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:54.505357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:54.505358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:54.505359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.505361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.505362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.505368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:54.505379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:54.505384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:54.505388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:54.505391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:54.505396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:54.505400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.505404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.505408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.505415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:54.505424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:54.505430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:54.505435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:54.505440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:54.505446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:54.505450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:54.505457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:54.505462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:54.505467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:54.505472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:54.505491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:54.505496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.505501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.505505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.505511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.505516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.505521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.505552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:54.513036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:54.514168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:54.514189Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:54.514191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:54.514196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:54.514203Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 404 [2:7559341273063675885:4807]
2025-10-09T21:44:54.514220Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:54.514239Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:54.514243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.514246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.514249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.514269Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:54.514275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:54.514296Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:54.514378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.514380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.514382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.514386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:54.514460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:54.514463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.514478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.514490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.514506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:54.514526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341273063675879:4807]
2025-10-09T21:44:54.514528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341273063675879:4807]
2025-10-09T21:44:54.514530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341273063675879:4807]
2025-10-09T21:44:54.514545Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:54.514561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:54.514576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:54.514578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:54.514622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341273063675879:4807]
2025-10-09T21:44:54.514625Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:54.514780Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:54.514782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:54.514784Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:54.514790Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:54.515413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341273425487414:5221], now have 1 active actors on pipe
2025-10-09T21:44:54.515418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341273425487315:2432], now have 1 active actors on pipe
2025-10-09T21:44:54.518126Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 404
2025-10-09T21:44:54.518161Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 404
2025-10-09T21:44:54.519673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:54.519692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:54.519734Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f545517e-4fa6e899-f6b91a36-4ee3019_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:54.519748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.519760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.519764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:54.519777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.519797Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:54.519821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:54.523638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:54.523660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:54.523703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:54.525009Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|412cca8a-3b6f383-9722dba8-78d99314_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f545517e-4fa6e899-f6b91a36-4ee3019_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:54.523964Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 405 partition: 0 MaxSeqNo: 25 sessionId: src_id|f545517e-4fa6e899-f6b91a36-4ee3019_0
2025-10-09T21:44:54.527641Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f545517e-4fa6e899-f6b91a36-4ee3019_0] Write session: try to update token
2025-10-09T21:44:54.527660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f545517e-4fa6e899-f6b91a36-4ee3019_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:54.534972Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:54.534990Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:54.534992Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:54.534994Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:54.535003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341273063675879:4807]
2025-10-09T21:44:54.535007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341273063675879:4807]
2025-10-09T21:44:54.535011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341273063675879:4807]
2025-10-09T21:44:54.535057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:54.535069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.535077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:54.535203Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:54.538609Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:54.538627Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:54.538632Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:54.541819Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341273425487578:5210]
2025-10-09T21:44:54.548377Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 404, pipe: [1:7559341273425487578:5210]
2025-10-09T21:44:54.549710Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 405 sessionId: src_id|f545517e-4fa6e899-f6b91a36-4ee3019_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:54.552201Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:54.697211Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:54.698108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341273425487578:5210], now have 1 active actors on pipe
2025-10-09T21:44:54.698144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:54.698151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:54.698390Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:54.698520Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:54.698522Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:54.698531Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:54.698563Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341273425487578:5210]
2025-10-09T21:44:54.698602Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 405 reason: error at write: Disconnected sessionId: src_id|f545517e-4fa6e899-f6b91a36-4ee3019_0
2025-10-09T21:44:54.698789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 405 sessionId: src_id|f545517e-4fa6e899-f6b91a36-4ee3019_0 is DEAD
2025-10-09T21:44:54.699188Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f545517e-4fa6e899-f6b91a36-4ee3019_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:54.699200Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f545517e-4fa6e899-f6b91a36-4ee3019_0] Write session will restart in 0.007936s
2025-10-09T21:44:54.699227Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f545517e-4fa6e899-f6b91a36-4ee3019_0] Write session: Do CDS request
2025-10-09T21:44:54.699233Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f545517e-4fa6e899-f6b91a36-4ee3019_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:54.708618Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f545517e-4fa6e899-f6b91a36-4ee3019_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:54.708846Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:54.708860Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 406
2025-10-09T21:44:54.709020Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 406 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:54.709058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 406 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:54.709068Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=406 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:54.709073Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:54.710016Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 406 sessionId:  describe result for acl check
2025-10-09T21:44:54.710074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:54.710080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:54.710082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:54.710088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341273425487634:5233] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:54.710658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341273425487634:5233] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:54.714748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341273425487634:5233] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:54.717660Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341273425487634:5233] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:54.717674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341273425487634:5233] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:54.717677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341273425487634:5233] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:54.717691Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 406 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:54.719420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:54.721928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:54.722199Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:54.722213Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:54.722227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:54.722317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:54.722363Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341273063676121:4815]
2025-10-09T21:44:54.723004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:54.723354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:54.723421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:54.723757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:54.723810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:54.723817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:54.723818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:54.723820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:54.723821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:54.723822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:54.723823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.723825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.723826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.723831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:54.723843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:54.723846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:54.723849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:54.723853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:54.723857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:54.723861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.723864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.723869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.723888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:54.723897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:54.723904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:54.723909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:54.723914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:54.723919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:54.723924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:54.723929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:54.723934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:54.723939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:54.723944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:54.723950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:54.723955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.723959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.723964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.723969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.723976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.723981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.724009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:54.732832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:54.734055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:54.734068Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:54.734072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:54.734076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:54.734084Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 405 [2:7559341273063676121:4815]
2025-10-09T21:44:54.734099Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:54.734119Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:54.734122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.734125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.734128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.734147Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:54.734153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:54.734160Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:54.734238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.734246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.734248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.734251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:54.734349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:54.734358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.734374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.734391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.734414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:54.734443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341273425487589:2432], now have 1 active actors on pipe
2025-10-09T21:44:54.734453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341273425487810:5233], now have 1 active actors on pipe
2025-10-09T21:44:54.734462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341273063676116:4815]
2025-10-09T21:44:54.734464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341273063676116:4815]
2025-10-09T21:44:54.734466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341273063676116:4815]
2025-10-09T21:44:54.734479Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:54.734497Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:54.734508Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:54.734512Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:54.734514Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:54.734517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:54.734521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:54.734563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341273063676116:4815]
2025-10-09T21:44:54.734580Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:54.734589Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:54.734795Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 405
2025-10-09T21:44:54.734843Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 405
2025-10-09T21:44:54.734977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:54.734994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:54.735029Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|70ee1716-27ca6248-b4572f12-e6c7cdcf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:54.735040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.735045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.735048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:54.735059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.735074Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:54.735091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:54.735334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:54.735347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:54.735364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:54.735480Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 406 partition: 0 MaxSeqNo: 25 sessionId: src_id|70ee1716-27ca6248-b4572f12-e6c7cdcf_0
2025-10-09T21:44:54.735866Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f545517e-4fa6e899-f6b91a36-4ee3019_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|70ee1716-27ca6248-b4572f12-e6c7cdcf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:54.738591Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70ee1716-27ca6248-b4572f12-e6c7cdcf_0] Write session: try to update token
2025-10-09T21:44:54.738610Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70ee1716-27ca6248-b4572f12-e6c7cdcf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:54.747396Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:54.747414Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:54.747425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341273063676116:4815]
2025-10-09T21:44:54.747478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:54.747658Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:54.747831Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:54.747981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:54.748127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:54.748335Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:54.748385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:54.748501Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:54.756338Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 406 sessionId: src_id|70ee1716-27ca6248-b4572f12-e6c7cdcf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:54.759040Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:54.759821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:54.759847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:54.759921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:54.760257Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:54.765822Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:54.765843Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:54.765845Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:54.765885Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:54.765896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341273063676116:4815]
2025-10-09T21:44:54.765900Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341273063676116:4815]
2025-10-09T21:44:54.765904Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341273063676116:4815]
2025-10-09T21:44:54.765953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:54.765960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.765964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:54.766129Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:54.770084Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:54.770103Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:54.770107Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:54.798860Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341273425487827:5210]
2025-10-09T21:44:54.799233Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 405, pipe: [1:7559341273425487827:5210]
2025-10-09T21:44:54.926090Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:54.934296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:54.934327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:54.934500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.934562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.934627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.934694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.934739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.934749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.934817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.934882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.934939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.934998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.935044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.935048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.935116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.935171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.935264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.935327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.935365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.935368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.935436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.935498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.935559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.935616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.935657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.935660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.935726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.935781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.935845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.935903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.935946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.935949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.936016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.936069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.938127Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:54.936128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.938194Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341273425487827:5210]
2025-10-09T21:44:54.936187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.936232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.936235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.936298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.936350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.936408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.936463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.936506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.936510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.936571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.936624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.936685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.936743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.936781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.936784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.938238Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:54.938247Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:54.938292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 406 reason: error at write: Disconnected sessionId: src_id|70ee1716-27ca6248-b4572f12-e6c7cdcf_0
2025-10-09T21:44:54.936849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.936909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.936968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.937027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.937069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.937072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.937142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:54.937199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:54.937260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:54.937321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:54.937362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:54.937365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:54.937444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341273425487827:5210], now have 1 active actors on pipe
2025-10-09T21:44:54.938663Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70ee1716-27ca6248-b4572f12-e6c7cdcf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:54.938681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70ee1716-27ca6248-b4572f12-e6c7cdcf_0] Write session will restart in 0.006299s
2025-10-09T21:44:54.938706Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70ee1716-27ca6248-b4572f12-e6c7cdcf_0] Write session: Do CDS request
2025-10-09T21:44:54.938712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70ee1716-27ca6248-b4572f12-e6c7cdcf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:54.938780Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:54.938503Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 406 sessionId: src_id|70ee1716-27ca6248-b4572f12-e6c7cdcf_0 is DEAD
2025-10-09T21:44:54.946675Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70ee1716-27ca6248-b4572f12-e6c7cdcf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:54.946731Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:54.946748Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 407
2025-10-09T21:44:54.947102Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 407 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:54.947157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 407 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:54.947169Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=407 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:54.947173Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:54.947600Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 407 sessionId:  describe result for acl check
2025-10-09T21:44:54.947638Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:54.947642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:54.947646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:54.947652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341273425487850:5238] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:54.948203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341273425487850:5238] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:54.951907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341273425487850:5238] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:54.958918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341273425487850:5238] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:54.958938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341273425487850:5238] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:54.958942Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341273425487850:5238] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:54.958952Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 407 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:54.968930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:54.971988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:54.972230Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:54.972239Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:54.972253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:54.972312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:54.972357Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341273063676349:4817]
2025-10-09T21:44:54.972927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:54.973323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:54.973391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:54.973766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:54.973833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:54.973834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:54.973835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:54.973836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:54.973837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:54.973838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:54.973839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.973840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.973841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.973847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:54.973858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:54.973861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:54.973864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:54.973867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:54.973871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:54.973874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.973878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.973881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.973888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:54.973896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:54.973901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:54.973906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:54.973909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:54.973914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:54.973917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:54.973922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:54.973926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:54.973931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:54.973934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:54.973939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:54.973943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.973947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.973951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.973955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.973959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.973963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.973990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:54.982155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:54.983347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:54.983354Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:54.983358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:54.983363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:54.983371Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 406 [2:7559341273063676349:4817]
2025-10-09T21:44:54.983406Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:54.983430Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:54.983434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.983438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.983441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.983461Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:54.983468Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:54.983477Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:54.983560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.983562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.983564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.983567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:54.983649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:54.983655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:54.983669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:54.983681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:54.983696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:54.983719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341273063676344:4817]
2025-10-09T21:44:54.983722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341273063676344:4817]
2025-10-09T21:44:54.983724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341273063676344:4817]
2025-10-09T21:44:54.983747Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:54.983763Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:54.983767Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:54.983770Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:54.983772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:54.983778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:54.983781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:54.983794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341273425487979:5238], now have 1 active actors on pipe
2025-10-09T21:44:54.983797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341273425487834:2432], now have 1 active actors on pipe
2025-10-09T21:44:54.983838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341273063676344:4817]
2025-10-09T21:44:54.983848Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:54.983875Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:54.984269Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 406
2025-10-09T21:44:54.984303Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 406
2025-10-09T21:44:54.986169Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 407 partition: 0 MaxSeqNo: 25 sessionId: src_id|2d54a822-cadbf868-21f4d43e-52369a6b_0
2025-10-09T21:44:54.987025Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70ee1716-27ca6248-b4572f12-e6c7cdcf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2d54a822-cadbf868-21f4d43e-52369a6b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:54.985020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:54.985037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:54.985069Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2d54a822-cadbf868-21f4d43e-52369a6b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:54.985080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.985087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.985090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:54.985103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.985118Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:54.985136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:54.985799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:54.985807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:54.985830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:54.989525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d54a822-cadbf868-21f4d43e-52369a6b_0] Write session: try to update token
2025-10-09T21:44:54.989543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d54a822-cadbf868-21f4d43e-52369a6b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:55.000711Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:55.000731Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:55.000733Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:55.000735Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:55.000744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341273063676344:4817]
2025-10-09T21:44:55.000748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341273063676344:4817]
2025-10-09T21:44:55.000752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341273063676344:4817]
2025-10-09T21:44:55.000797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:55.000808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.000812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:55.000943Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:55.003004Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:55.003022Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:55.003026Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:55.019983Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 407 sessionId: src_id|2d54a822-cadbf868-21f4d43e-52369a6b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:55.022619Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:55.037966Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341277720455371:5210]
2025-10-09T21:44:55.038663Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 406, pipe: [1:7559341277720455371:5210]
2025-10-09T21:44:55.172123Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:55.172754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:55.172774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:55.172798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341277720455371:5210], now have 1 active actors on pipe
2025-10-09T21:44:55.173017Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:55.173054Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:55.173069Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:55.173142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 407 reason: error at write: Disconnected sessionId: src_id|2d54a822-cadbf868-21f4d43e-52369a6b_0
2025-10-09T21:44:55.173257Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:55.173306Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341277720455371:5210]
2025-10-09T21:44:55.173339Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 407 sessionId: src_id|2d54a822-cadbf868-21f4d43e-52369a6b_0 is DEAD
2025-10-09T21:44:55.173565Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d54a822-cadbf868-21f4d43e-52369a6b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:55.173575Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d54a822-cadbf868-21f4d43e-52369a6b_0] Write session will restart in 0.009822s
2025-10-09T21:44:55.173603Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d54a822-cadbf868-21f4d43e-52369a6b_0] Write session: Do CDS request
2025-10-09T21:44:55.173610Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d54a822-cadbf868-21f4d43e-52369a6b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:55.184575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d54a822-cadbf868-21f4d43e-52369a6b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:55.185172Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:55.185188Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 408
2025-10-09T21:44:55.185375Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 408 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:55.185411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 408 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:55.185417Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=408 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:55.185421Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:55.185929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 408 sessionId:  describe result for acl check
2025-10-09T21:44:55.185966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:55.185969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:55.185971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:55.185977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341277720455406:5242] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:55.186617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341277720455406:5242] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:55.190359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341277720455406:5242] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:55.195307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341277720455406:5242] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:55.195322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341277720455406:5242] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:55.195327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341277720455406:5242] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:55.195343Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 408 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:55.207646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:55.211876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:55.212124Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:55.212136Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:55.212150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:55.212233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:55.212274Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341277358643877:4820]
2025-10-09T21:44:55.212892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:55.213293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:55.213345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:55.213672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:55.213740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:55.213747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:55.213748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:55.213749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:55.213750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:55.213752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:55.213753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.213754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.213755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.213761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:55.213774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:55.213777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:55.213782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:55.213786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:55.213791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:55.213795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.213804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.213808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.213816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:55.213829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:55.213838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:55.213845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:55.213850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:55.213856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:55.213861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:55.213866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:55.213875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:55.213880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:55.213885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:55.213890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:55.213894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.213899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.213903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.213909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.213915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.213921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.213955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:55.221555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:55.222707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:55.222726Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:55.222729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:55.222733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:55.222742Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 407 [2:7559341277358643877:4820]
2025-10-09T21:44:55.222759Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:55.222780Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:55.222783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.222786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.222789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.222810Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:55.222817Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:55.222825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:55.222894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.222902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.222905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.222908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:55.223192Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 407
2025-10-09T21:44:55.223252Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 407
2025-10-09T21:44:55.222985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:55.222993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.223007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.223019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.223035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:55.223059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341277720455373:2432], now have 1 active actors on pipe
2025-10-09T21:44:55.223062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341277720455494:5242], now have 1 active actors on pipe
2025-10-09T21:44:55.223070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341277358643871:4820]
2025-10-09T21:44:55.223084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341277358643871:4820]
2025-10-09T21:44:55.223087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341277358643871:4820]
2025-10-09T21:44:55.223096Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:55.223112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:55.223112Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:55.223116Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:55.223118Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:55.223126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:55.223129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:55.223174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341277358643871:4820]
2025-10-09T21:44:55.223183Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:55.223241Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:55.223824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:55.223837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:55.223884Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|74137f7c-7257415d-fdee362d-1c273d41_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:55.223898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.223903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.223906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:55.223917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.223936Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:55.223965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:55.224242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:55.224256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:55.224276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:55.224402Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 408 partition: 0 MaxSeqNo: 25 sessionId: src_id|74137f7c-7257415d-fdee362d-1c273d41_0
2025-10-09T21:44:55.224790Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d54a822-cadbf868-21f4d43e-52369a6b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|74137f7c-7257415d-fdee362d-1c273d41_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:55.227478Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74137f7c-7257415d-fdee362d-1c273d41_0] Write session: try to update token
2025-10-09T21:44:55.227500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74137f7c-7257415d-fdee362d-1c273d41_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:55.240586Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:55.240605Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:55.240607Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:55.240610Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:55.240641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341277358643871:4820]
2025-10-09T21:44:55.240646Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341277358643871:4820]
2025-10-09T21:44:55.240651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341277358643871:4820]
2025-10-09T21:44:55.240702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:55.240716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.240718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:55.240856Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:55.241163Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:55.241177Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:55.241181Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:55.257553Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 408 sessionId: src_id|74137f7c-7257415d-fdee362d-1c273d41_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:55.260304Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:55.273508Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341277720455610:5210]
2025-10-09T21:44:55.273993Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 407, pipe: [1:7559341277720455610:5210]
2025-10-09T21:44:55.412829Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:55.413601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:55.413623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:55.413647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341277720455610:5210], now have 1 active actors on pipe
2025-10-09T21:44:55.414004Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:55.414161Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:55.414211Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:55.414299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 408 reason: status is not ok: tablet will be restarted right now sessionId: src_id|74137f7c-7257415d-fdee362d-1c273d41_0
2025-10-09T21:44:55.414386Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341277720455610:5210]
2025-10-09T21:44:55.414504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 408 sessionId: src_id|74137f7c-7257415d-fdee362d-1c273d41_0 is DEAD
2025-10-09T21:44:55.414647Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74137f7c-7257415d-fdee362d-1c273d41_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:55.414671Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74137f7c-7257415d-fdee362d-1c273d41_0] Write session will restart in 0.005637s
2025-10-09T21:44:55.414698Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74137f7c-7257415d-fdee362d-1c273d41_0] Write session: Do CDS request
2025-10-09T21:44:55.414705Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74137f7c-7257415d-fdee362d-1c273d41_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:55.421936Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74137f7c-7257415d-fdee362d-1c273d41_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:55.423351Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:55.423368Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 409
2025-10-09T21:44:55.424035Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 409 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:55.424093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 409 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:55.424099Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=409 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:55.424104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:55.424547Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 409 sessionId:  describe result for acl check
2025-10-09T21:44:55.424577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:55.424579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:55.424580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:55.424587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341277720455626:5246] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:55.425152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341277720455626:5246] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:55.432993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341277720455626:5246] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:55.447111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341277720455626:5246] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:55.447130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341277720455626:5246] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:55.447133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341277720455626:5246] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:55.447149Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 409 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:55.452214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:55.454787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:55.455037Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:55.455050Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:55.455064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:55.455134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:55.455180Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341277358644108:4823]
2025-10-09T21:44:55.455791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:55.456170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:55.456241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:55.456633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:55.456726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:55.456737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:55.456738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:55.456739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:55.456741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:55.456742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:55.456743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.456744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.456745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.456751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:55.456762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:55.456766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:55.456770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:55.456773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:55.456777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:55.456781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.456784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.456788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.456797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:55.456805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:55.456810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:55.456816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:55.456833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:55.456839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:55.456843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:55.456848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:55.456853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:55.456857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:55.456864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:55.456868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:55.456873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.456879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.456884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.456889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.456894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.456898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.456925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:55.466536Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 408
2025-10-09T21:44:55.466569Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 408
2025-10-09T21:44:55.464379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:55.466095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:55.466101Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:55.466104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:55.466108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:55.466116Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 408 [2:7559341277358644108:4823]
2025-10-09T21:44:55.466132Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:55.466150Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:55.466154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.466157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.466159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.466180Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:55.466186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:55.466197Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:55.466266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.466268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.466270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.466272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:55.466361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:55.466363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.466375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.466384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.466399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:55.466419Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341277358644103:4823]
2025-10-09T21:44:55.466422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341277358644103:4823]
2025-10-09T21:44:55.466424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341277358644103:4823]
2025-10-09T21:44:55.466432Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:55.466447Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:55.466451Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:55.466451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341277720455818:5246], now have 1 active actors on pipe
2025-10-09T21:44:55.466453Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:55.466454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341277720455613:2432], now have 1 active actors on pipe
2025-10-09T21:44:55.466463Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:55.466477Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:55.466480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:55.466521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341277358644103:4823]
2025-10-09T21:44:55.466531Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:55.466564Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:55.467027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:55.467042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:55.467077Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|71e859fd-2859463b-dfe39981-411ef30f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:55.467091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.467099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.467102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:55.467117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.467135Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:55.467152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:55.467544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 409 partition: 0 MaxSeqNo: 25 sessionId: src_id|71e859fd-2859463b-dfe39981-411ef30f_0
2025-10-09T21:44:55.467404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:55.467414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:55.467430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:55.467976Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74137f7c-7257415d-fdee362d-1c273d41_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|71e859fd-2859463b-dfe39981-411ef30f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:55.470663Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|71e859fd-2859463b-dfe39981-411ef30f_0] Write session: try to update token
2025-10-09T21:44:55.470681Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|71e859fd-2859463b-dfe39981-411ef30f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:55.485437Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:55.485453Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:55.485455Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:55.485458Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:55.485468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341277358644103:4823]
2025-10-09T21:44:55.485472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341277358644103:4823]
2025-10-09T21:44:55.485476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341277358644103:4823]
2025-10-09T21:44:55.485522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:55.485522Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:55.485529Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:55.485529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.485532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:55.485548Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:55.485683Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:55.492390Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 409 sessionId: src_id|71e859fd-2859463b-dfe39981-411ef30f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:55.495238Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:55.514599Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341277720455850:5210]
2025-10-09T21:44:55.514942Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 408, pipe: [1:7559341277720455850:5210]
2025-10-09T21:44:55.646898Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:55.647434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:55.647453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:55.647477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341277720455850:5210], now have 1 active actors on pipe
2025-10-09T21:44:55.647716Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:55.647763Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:55.647777Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:55.647863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 409 reason: error at write: Disconnected sessionId: src_id|71e859fd-2859463b-dfe39981-411ef30f_0
2025-10-09T21:44:55.648047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 409 sessionId: src_id|71e859fd-2859463b-dfe39981-411ef30f_0 is DEAD
2025-10-09T21:44:55.648136Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:55.648183Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341277720455850:5210]
2025-10-09T21:44:55.648381Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71e859fd-2859463b-dfe39981-411ef30f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:55.648393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71e859fd-2859463b-dfe39981-411ef30f_0] Write session will restart in 0.008441s
2025-10-09T21:44:55.648421Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71e859fd-2859463b-dfe39981-411ef30f_0] Write session: Do CDS request
2025-10-09T21:44:55.648426Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71e859fd-2859463b-dfe39981-411ef30f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:55.658630Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|71e859fd-2859463b-dfe39981-411ef30f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:55.662213Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:55.662231Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 410
2025-10-09T21:44:55.666101Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 410 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:55.666158Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 410 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:55.666164Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=410 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:55.666169Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:55.666657Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 410 sessionId:  describe result for acl check
2025-10-09T21:44:55.666697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:55.666703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:55.666705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:55.666712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341277720456004:5258] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:55.667291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341277720456004:5258] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:55.670191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341277720456004:5258] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:55.676303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341277720456004:5258] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:55.676317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341277720456004:5258] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:55.676321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341277720456004:5258] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:55.676331Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 410 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:55.675991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:55.677113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:55.677410Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:55.677414Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:55.677427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:55.677489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:55.677525Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341277358644377:4837]
2025-10-09T21:44:55.678113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:55.679038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:55.679092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:55.679404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:55.679469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:55.679476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:55.679478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:55.679479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:55.679480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:55.679481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:55.679483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.679484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.679485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.679491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:55.679501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:55.679505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:55.679509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:55.679512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:55.679515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:55.679518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.679521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.679525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.679532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:55.679540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:55.679545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:55.679550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:55.679559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:55.679563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:55.679567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:55.679572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:55.679577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:55.679582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:55.679591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:55.679595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:55.679599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.679609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.679613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.679618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.679626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.679632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.679658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:55.689175Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 409
2025-10-09T21:44:55.689208Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 409
2025-10-09T21:44:55.686942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:55.688425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:55.688430Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:55.688432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:55.688437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:55.688446Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 409 [2:7559341277358644377:4837]
2025-10-09T21:44:55.688459Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:55.688478Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:55.688482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.688485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.688487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.688504Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:55.688510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:55.688517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:55.688587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.688589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.688592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.688595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:55.688669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:55.688672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.688684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.688696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.688713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:55.688735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341277720456104:5258], now have 1 active actors on pipe
2025-10-09T21:44:55.688738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341277720455880:2432], now have 1 active actors on pipe
2025-10-09T21:44:55.688746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341277358644371:4837]
2025-10-09T21:44:55.688749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341277358644371:4837]
2025-10-09T21:44:55.688751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341277358644371:4837]
2025-10-09T21:44:55.688761Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:55.688775Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:55.688789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:55.688792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:55.688807Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:55.688810Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:55.688811Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:55.688840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341277358644371:4837]
2025-10-09T21:44:55.688847Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:55.688852Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:55.689600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:55.689606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:55.689638Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7981338c-fb556eda-44457bc0-587e28b6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:55.689647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.689651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.689654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:55.689672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.689685Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:55.689705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:55.690554Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 410 partition: 0 MaxSeqNo: 25 sessionId: src_id|7981338c-fb556eda-44457bc0-587e28b6_0
2025-10-09T21:44:55.690968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71e859fd-2859463b-dfe39981-411ef30f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7981338c-fb556eda-44457bc0-587e28b6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:55.693492Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7981338c-fb556eda-44457bc0-587e28b6_0] Write session: try to update token
2025-10-09T21:44:55.693510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7981338c-fb556eda-44457bc0-587e28b6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:55.690362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:55.690384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:55.690403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:55.705907Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:55.705930Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:55.705933Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:55.705935Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:55.705954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341277358644371:4837]
2025-10-09T21:44:55.705960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341277358644371:4837]
2025-10-09T21:44:55.705964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341277358644371:4837]
2025-10-09T21:44:55.706013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:55.706017Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:55.706022Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:55.706026Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:55.706048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.706051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:55.706238Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:55.709993Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 410 sessionId: src_id|7981338c-fb556eda-44457bc0-587e28b6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:55.712634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:55.748593Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341277720456116:5210]
2025-10-09T21:44:55.749062Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 409, pipe: [1:7559341277720456116:5210]
2025-10-09T21:44:55.878091Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:55.878413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:55.878424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:55.878474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341277720456116:5210], now have 1 active actors on pipe
2025-10-09T21:44:55.879589Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341277720456116:5210]
2025-10-09T21:44:55.879620Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:55.879623Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:55.879679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 410 reason: error at write: Disconnected sessionId: src_id|7981338c-fb556eda-44457bc0-587e28b6_0
2025-10-09T21:44:55.879866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 410 sessionId: src_id|7981338c-fb556eda-44457bc0-587e28b6_0 is DEAD
2025-10-09T21:44:55.880164Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:55.881351Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7981338c-fb556eda-44457bc0-587e28b6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:55.881383Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7981338c-fb556eda-44457bc0-587e28b6_0] Write session will restart in 0.009602s
2025-10-09T21:44:55.881415Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7981338c-fb556eda-44457bc0-587e28b6_0] Write session: Do CDS request
2025-10-09T21:44:55.881425Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7981338c-fb556eda-44457bc0-587e28b6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:55.881119Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:55.892865Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7981338c-fb556eda-44457bc0-587e28b6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:55.892937Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:55.892956Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 411
2025-10-09T21:44:55.893138Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 411 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:55.893185Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 411 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:55.893196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=411 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:55.893200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:55.894501Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 411 sessionId:  describe result for acl check
2025-10-09T21:44:55.894543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:55.894544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:55.894546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:55.894553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341277720456129:5262] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:55.895127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341277720456129:5262] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:55.906252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341277720456129:5262] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:55.915311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341277720456129:5262] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:55.915333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341277720456129:5262] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:55.915337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341277720456129:5262] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:55.915346Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 411 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:55.961394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:55.968153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:55.968413Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:55.968418Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:55.968431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:55.968492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:55.968533Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341277358644610:4839]
2025-10-09T21:44:55.969130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:55.970983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:55.971050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:55.971441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:55.971507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:55.971508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:55.971509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:55.971511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:55.971512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:55.971513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:55.971515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.971516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.971517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.971524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:55.971536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:55.971541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:55.971545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:55.971550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:55.971554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:55.971558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.971562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.971566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.971574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:55.971583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:55.971590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:55.971595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:55.971601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:55.971606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:55.971611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:55.971616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:55.971622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:55.971628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:55.971633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:55.971638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:55.971644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.971650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.971655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.971661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.971667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.971672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.971703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:55.982001Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341277720456360:5210]
2025-10-09T21:44:55.981540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:55.982615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:55.982622Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:55.982625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:55.982630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:55.982638Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 410 [2:7559341277358644610:4839]
2025-10-09T21:44:55.982653Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:55.982673Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:55.982676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.982680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.982682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.982702Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:55.982708Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:55.982715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:55.982780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.982782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.982784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.982787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:55.982862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:55.982865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:55.982880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:55.982892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:55.982909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:55.982930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341277358644604:4839]
2025-10-09T21:44:55.982932Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341277358644604:4839]
2025-10-09T21:44:55.982936Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341277358644604:4839]
2025-10-09T21:44:55.982946Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:55.982964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:55.982979Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:55.982982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:55.983022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341277358644604:4839]
2025-10-09T21:44:55.983025Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:55.983418Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:55.983431Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:55.983434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:55.983433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341277720456171:5262], now have 1 active actors on pipe
2025-10-09T21:44:55.983440Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:55.983441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341277720456121:2432], now have 1 active actors on pipe
2025-10-09T21:44:55.983444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341277720456360:5210], now have 1 active actors on pipe
2025-10-09T21:44:55.983513Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 410
2025-10-09T21:44:55.983555Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 410
2025-10-09T21:44:55.983574Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 410, pipe: [1:7559341277720456360:5210]
2025-10-09T21:44:55.983778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:55.983793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:55.983828Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|551acd57-1aa8aa3a-6597985f-cc0b9168_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:55.983838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.983843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.983846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:55.983858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.983872Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:55.983891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:55.989724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:55.989745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:55.989783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:55.990469Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 411 partition: 0 MaxSeqNo: 25 sessionId: src_id|551acd57-1aa8aa3a-6597985f-cc0b9168_0
2025-10-09T21:44:55.990979Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7981338c-fb556eda-44457bc0-587e28b6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|551acd57-1aa8aa3a-6597985f-cc0b9168_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:55.993400Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|551acd57-1aa8aa3a-6597985f-cc0b9168_0] Write session: try to update token
2025-10-09T21:44:55.993420Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|551acd57-1aa8aa3a-6597985f-cc0b9168_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:56.000536Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:56.000560Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:56.000563Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:56.000566Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:56.000576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341277358644604:4839]
2025-10-09T21:44:56.000580Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341277358644604:4839]
2025-10-09T21:44:56.000584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341277358644604:4839]
2025-10-09T21:44:56.000633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:56.000641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.000644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:56.000796Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:56.006872Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:56.006897Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:56.006903Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:56.012209Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 411 sessionId: src_id|551acd57-1aa8aa3a-6597985f-cc0b9168_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:56.014845Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:56.169941Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:56.171081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:56.171094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:56.171360Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:56.172007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|551acd57-1aa8aa3a-6597985f-cc0b9168_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:56.172018Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|551acd57-1aa8aa3a-6597985f-cc0b9168_0] Write session will restart in 0.008551s
2025-10-09T21:44:56.172052Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|551acd57-1aa8aa3a-6597985f-cc0b9168_0] Write session: Do CDS request
2025-10-09T21:44:56.172059Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|551acd57-1aa8aa3a-6597985f-cc0b9168_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:56.171525Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341277720456360:5210]
2025-10-09T21:44:56.171560Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:56.171563Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:56.171635Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 411 reason: error at write: Disconnected sessionId: src_id|551acd57-1aa8aa3a-6597985f-cc0b9168_0
2025-10-09T21:44:56.171858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 411 sessionId: src_id|551acd57-1aa8aa3a-6597985f-cc0b9168_0 is DEAD
2025-10-09T21:44:56.172048Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:56.181542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|551acd57-1aa8aa3a-6597985f-cc0b9168_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:56.181937Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:56.181950Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 412
2025-10-09T21:44:56.182815Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 412 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:56.182864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 412 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:56.182869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=412 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:56.182875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:56.183475Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 412 sessionId:  describe result for acl check
2025-10-09T21:44:56.183510Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:56.183512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:56.183514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:56.183520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341282015423679:5267] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:56.184116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341282015423679:5267] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:56.191661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341282015423679:5267] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:56.199335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341282015423679:5267] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:56.199355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341282015423679:5267] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:56.199359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341282015423679:5267] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:56.199370Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 412 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:56.221923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:56.227367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:56.227698Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:56.227712Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:56.227726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:56.227793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:56.227840Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341281653612142:4842]
2025-10-09T21:44:56.228467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:56.230456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:56.230525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:56.230912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:56.230972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:56.230978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:56.230980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:56.230982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:56.230984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:56.230985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:56.230987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.230988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.230990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.230997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:56.231009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:56.231014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:56.231019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:56.231023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:56.231028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:56.231033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.231037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.231041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.231050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:56.231060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:56.231068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:56.231073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:56.231079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:56.231085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:56.231090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:56.231095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:56.231101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:56.231107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:56.231111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:56.231117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:56.231122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.231129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.231133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.231139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.231144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.231150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.231184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:56.247115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:56.248534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:56.248540Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:56.248543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:56.248548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:56.248555Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 411 [2:7559341281653612142:4842]
2025-10-09T21:44:56.248570Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:56.248591Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:56.248594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:56.248598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:56.248600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:56.248616Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:56.248623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:56.248630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:56.248692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.248694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.248697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.248700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:56.248777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:56.248781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.248796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.248810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.248829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:56.248851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341281653612137:4842]
2025-10-09T21:44:56.248854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341281653612137:4842]
2025-10-09T21:44:56.248856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341281653612137:4842]
2025-10-09T21:44:56.248875Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:56.248894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:56.248909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:56.248913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:56.248944Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:56.248948Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:56.248951Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:56.248956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341281653612137:4842]
2025-10-09T21:44:56.248960Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:56.249062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:56.249219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341282015423668:2432], now have 1 active actors on pipe
2025-10-09T21:44:56.249234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341282015423781:5267], now have 1 active actors on pipe
2025-10-09T21:44:56.251968Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 411
2025-10-09T21:44:56.252027Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 411
2025-10-09T21:44:56.255568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:56.255604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:56.255664Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|84708ac7-148db6d7-1467c52a-1c0badce_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:56.255679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:56.255685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:56.255690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:56.255719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:56.255739Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:56.255773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:56.259878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:56.259899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:56.259937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:56.266842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 412 partition: 0 MaxSeqNo: 25 sessionId: src_id|84708ac7-148db6d7-1467c52a-1c0badce_0
2025-10-09T21:44:56.267276Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|551acd57-1aa8aa3a-6597985f-cc0b9168_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|84708ac7-148db6d7-1467c52a-1c0badce_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:56.269958Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|84708ac7-148db6d7-1467c52a-1c0badce_0] Write session: try to update token
2025-10-09T21:44:56.269979Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|84708ac7-148db6d7-1467c52a-1c0badce_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:56.271869Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341282015423909:5210]
2025-10-09T21:44:56.272831Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 411, pipe: [1:7559341282015423909:5210]
2025-10-09T21:44:56.275580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341282015423909:5210], now have 1 active actors on pipe
2025-10-09T21:44:56.275606Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:56.275609Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:56.275619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341281653612137:4842]
2025-10-09T21:44:56.275670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:56.275677Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:56.275843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:56.275987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:56.276129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:56.276271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:56.276476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:56.276513Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:56.293277Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 412 sessionId: src_id|84708ac7-148db6d7-1467c52a-1c0badce_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:56.289387Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:56.289407Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:56.289409Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:56.289411Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:56.289424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341281653612137:4842]
2025-10-09T21:44:56.289427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341281653612137:4842]
2025-10-09T21:44:56.289435Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341281653612137:4842]
2025-10-09T21:44:56.289489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:56.289496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.289501Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:56.289503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:56.289509Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:56.289514Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:56.289641Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:56.296572Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:56.469503Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:56.470508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:56.470528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:56.470874Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:56.471945Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84708ac7-148db6d7-1467c52a-1c0badce_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:56.471956Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84708ac7-148db6d7-1467c52a-1c0badce_0] Write session will restart in 0.009148s
2025-10-09T21:44:56.471979Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84708ac7-148db6d7-1467c52a-1c0badce_0] Write session: Do CDS request
2025-10-09T21:44:56.471986Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84708ac7-148db6d7-1467c52a-1c0badce_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:56.471316Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:56.471378Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:56.471394Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:56.471464Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 412 reason: error at write: Disconnected sessionId: src_id|84708ac7-148db6d7-1467c52a-1c0badce_0
2025-10-09T21:44:56.471627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 412 sessionId: src_id|84708ac7-148db6d7-1467c52a-1c0badce_0 is DEAD
2025-10-09T21:44:56.471747Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341282015423909:5210]
2025-10-09T21:44:56.482640Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|84708ac7-148db6d7-1467c52a-1c0badce_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:56.482659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:56.482674Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 413
2025-10-09T21:44:56.482919Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 413 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:56.482955Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 413 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:56.482967Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=413 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:56.482971Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:56.483394Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 413 sessionId:  describe result for acl check
2025-10-09T21:44:56.483433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:56.483435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:56.483437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:56.483443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341282015423927:5271] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:56.483987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341282015423927:5271] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:56.490457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341282015423927:5271] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:56.497431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341282015423927:5271] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:56.497448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341282015423927:5271] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:56.497452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341282015423927:5271] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:56.497463Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 413 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:56.512577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:56.513174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:56.513367Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:56.513371Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:56.513382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:56.513431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:56.513466Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341281653612379:4845]
2025-10-09T21:44:56.514080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:56.514508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:56.514545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:56.514854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:56.514901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:56.514902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:56.514903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:56.514905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:56.514906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:56.514907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:56.514909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.514910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.514911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.514916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:56.514927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:56.514931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:56.514935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:56.514939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:56.514943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:56.514947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.514951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.514955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.514962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:56.514970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:56.514976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:56.514981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:56.514985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:56.514990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:56.514994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:56.514999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:56.515003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:56.515008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:56.515013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:56.515018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:56.515023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.515028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.515033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.515038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.515043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.515048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.515076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:56.523013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:56.524460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:56.524467Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:56.524469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:56.524473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:56.524481Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 412 [2:7559341281653612379:4845]
2025-10-09T21:44:56.524497Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:56.524514Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:56.524516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:56.524520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:56.524522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:56.524539Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:56.524544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:56.524552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:56.524616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.524618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.524620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.524622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:56.524696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:56.524699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.524713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.524723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.524739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:56.524760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341281653612374:4845]
2025-10-09T21:44:56.524762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341281653612374:4845]
2025-10-09T21:44:56.524764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341281653612374:4845]
2025-10-09T21:44:56.524776Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:56.524791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:56.524804Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:56.524807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:56.524849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341281653612374:4845]
2025-10-09T21:44:56.524851Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:56.525158Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:56.525166Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:56.525168Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:56.525173Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:56.526054Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 412
2025-10-09T21:44:56.526086Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 412
2025-10-09T21:44:56.528861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341282015424069:5271], now have 1 active actors on pipe
2025-10-09T21:44:56.528877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341282015423913:2432], now have 1 active actors on pipe
2025-10-09T21:44:56.528898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:56.528905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:56.528946Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e4ff4e13-cb6f8487-efd7d881-e65c6d1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:56.528959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:56.528964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:56.528967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:56.528980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:56.528999Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:56.529021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:56.530611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:56.530625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:56.530654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:56.530926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 413 partition: 0 MaxSeqNo: 25 sessionId: src_id|e4ff4e13-cb6f8487-efd7d881-e65c6d1_0
2025-10-09T21:44:56.531537Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84708ac7-148db6d7-1467c52a-1c0badce_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e4ff4e13-cb6f8487-efd7d881-e65c6d1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:56.534209Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4ff4e13-cb6f8487-efd7d881-e65c6d1_0] Write session: try to update token
2025-10-09T21:44:56.534229Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4ff4e13-cb6f8487-efd7d881-e65c6d1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:56.555016Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:56.555035Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:56.555037Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:56.555038Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:56.555050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341281653612374:4845]
2025-10-09T21:44:56.555056Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341281653612374:4845]
2025-10-09T21:44:56.555059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341281653612374:4845]
2025-10-09T21:44:56.555103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:56.555110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.555112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:56.555243Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:56.558817Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:56.558844Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:56.558848Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:56.560661Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 413 sessionId: src_id|e4ff4e13-cb6f8487-efd7d881-e65c6d1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:56.563103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:56.571967Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341282015424157:5210]
2025-10-09T21:44:56.574956Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 412, pipe: [1:7559341282015424157:5210]
2025-10-09T21:44:56.727989Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:56.728526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:56.728537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:56.728560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341282015424157:5210], now have 1 active actors on pipe
2025-10-09T21:44:56.729040Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341282015424157:5210]
2025-10-09T21:44:56.729076Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:56.729127Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:56.729136Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:56.729205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 413 reason: error at write: Disconnected sessionId: src_id|e4ff4e13-cb6f8487-efd7d881-e65c6d1_0
2025-10-09T21:44:56.729400Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 413 sessionId: src_id|e4ff4e13-cb6f8487-efd7d881-e65c6d1_0 is DEAD
2025-10-09T21:44:56.729557Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4ff4e13-cb6f8487-efd7d881-e65c6d1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:56.729568Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4ff4e13-cb6f8487-efd7d881-e65c6d1_0] Write session will restart in 0.005909s
2025-10-09T21:44:56.729594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4ff4e13-cb6f8487-efd7d881-e65c6d1_0] Write session: Do CDS request
2025-10-09T21:44:56.729603Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4ff4e13-cb6f8487-efd7d881-e65c6d1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:56.729084Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:56.738650Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4ff4e13-cb6f8487-efd7d881-e65c6d1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:56.739038Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:56.739065Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 414
2025-10-09T21:44:56.742330Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 414 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:56.742381Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 414 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:56.742402Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=414 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:56.742406Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:56.743227Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 414 sessionId:  describe result for acl check
2025-10-09T21:44:56.743273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:56.743279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:56.743282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:56.743288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341282015424202:5285] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:56.743855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341282015424202:5285] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:56.767349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341282015424202:5285] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:56.783364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341282015424202:5285] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:56.783381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341282015424202:5285] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:56.783387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341282015424202:5285] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:56.783397Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 414 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:56.794668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:56.802864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:56.803117Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:56.803121Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:56.803133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:56.803187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:56.803239Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341281653612652:4859]
2025-10-09T21:44:56.804421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:56.804902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:56.804946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:56.805312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:56.805355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:56.805357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:56.805358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:56.805359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:56.805360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:56.805361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:56.805362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.805364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.805365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.805371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:56.805382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:56.805385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:56.805389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:56.805393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:56.805398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:56.805401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.805405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.805409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.805415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:56.805424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:56.805430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:56.805435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:56.805439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:56.805445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:56.805448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:56.805453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:56.805458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:56.805463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:56.805467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:56.805472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:56.805477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.805482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.805486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.805491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.805495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.805500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.805527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:56.818818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:56.819985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:56.819992Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:56.819995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:56.820000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:56.820008Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 413 [2:7559341281653612652:4859]
2025-10-09T21:44:56.820022Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:56.820041Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:56.820045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:56.820049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:56.820051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:56.820070Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:56.820077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:56.820084Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:56.820146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.820148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.820150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.820153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:56.820230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:56.820232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.820245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:56.820258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:56.820275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:56.820309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341281653612646:4859]
2025-10-09T21:44:56.820312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341281653612646:4859]
2025-10-09T21:44:56.820314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341281653612646:4859]
2025-10-09T21:44:56.820326Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:56.820341Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:56.820357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:56.820360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:56.820399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341281653612646:4859]
2025-10-09T21:44:56.820402Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:56.820702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:56.820710Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:56.820711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:56.820717Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:56.821436Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 413
2025-10-09T21:44:56.821478Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 413
2025-10-09T21:44:56.823143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341282015424187:2432], now have 1 active actors on pipe
2025-10-09T21:44:56.823159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341282015424356:5285], now have 1 active actors on pipe
2025-10-09T21:44:56.823178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:56.823185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:56.823224Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e360f580-6e60cb2c-fdffbb21-f18f7b59_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:56.823238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:56.823242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:56.823245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:56.823259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:56.823276Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:56.823305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:56.823631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:56.823640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:56.823657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:56.824235Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4ff4e13-cb6f8487-efd7d881-e65c6d1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e360f580-6e60cb2c-fdffbb21-f18f7b59_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:56.826841Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e360f580-6e60cb2c-fdffbb21-f18f7b59_0] Write session: try to update token
2025-10-09T21:44:56.826861Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e360f580-6e60cb2c-fdffbb21-f18f7b59_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:56.823786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 414 partition: 0 MaxSeqNo: 25 sessionId: src_id|e360f580-6e60cb2c-fdffbb21-f18f7b59_0
2025-10-09T21:44:56.829378Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341282015424427:5210]
2025-10-09T21:44:56.829886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341282015424427:5210], now have 1 active actors on pipe
2025-10-09T21:44:56.838116Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 413, pipe: [1:7559341282015424427:5210]
2025-10-09T21:44:56.845546Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:56.845571Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:56.845574Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:56.845576Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:56.845587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341281653612646:4859]
2025-10-09T21:44:56.845591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341281653612646:4859]
2025-10-09T21:44:56.845595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341281653612646:4859]
2025-10-09T21:44:56.845631Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:56.845635Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:56.845638Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:56.845639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:56.845646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:56.845649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:56.845793Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:56.858704Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 414 sessionId: src_id|e360f580-6e60cb2c-fdffbb21-f18f7b59_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:56.861109Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:57.012336Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:57.012658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:57.012669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:57.013150Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:57.013164Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:57.013169Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:57.013221Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341282015424427:5210]
2025-10-09T21:44:57.013238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 414 reason: error at write: Disconnected sessionId: src_id|e360f580-6e60cb2c-fdffbb21-f18f7b59_0
2025-10-09T21:44:57.013764Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:57.015768Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e360f580-6e60cb2c-fdffbb21-f18f7b59_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:57.015780Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e360f580-6e60cb2c-fdffbb21-f18f7b59_0] Write session will restart in 0.007728s
2025-10-09T21:44:57.015804Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e360f580-6e60cb2c-fdffbb21-f18f7b59_0] Write session: Do CDS request
2025-10-09T21:44:57.015811Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e360f580-6e60cb2c-fdffbb21-f18f7b59_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:57.013436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 414 sessionId: src_id|e360f580-6e60cb2c-fdffbb21-f18f7b59_0 is DEAD
2025-10-09T21:44:57.026540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e360f580-6e60cb2c-fdffbb21-f18f7b59_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:57.030262Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:57.030289Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 415
2025-10-09T21:44:57.030485Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 415 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:57.030520Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 415 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:57.030530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=415 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:57.030534Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:57.031640Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 415 sessionId:  describe result for acl check
2025-10-09T21:44:57.031676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:57.031682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:57.031683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:57.031690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341286310391793:5290] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:57.032210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341286310391793:5290] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:57.036815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341286310391793:5290] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:57.047019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:57.051346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:57.051594Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:57.051604Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:57.051617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:57.051670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:57.051739Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341285948580184:4861]
2025-10-09T21:44:57.052317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:57.053354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:57.053407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:57.053718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:57.053756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:57.053757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:57.053758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:57.053759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:57.053761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:57.053762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:57.053763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.053764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.053766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.053771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:57.053780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:57.053784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:57.053787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:57.053790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:57.053793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:57.053796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.053799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.053802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.053809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:57.053817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:57.053822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:57.053827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:57.053830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:57.053835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:57.053839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:57.053843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:57.053847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:57.053851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:57.053855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:57.053859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:57.053864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.053869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.053874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.053878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.053882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.053887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.053914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:57.051709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341286310391793:5290] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:57.051726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341286310391793:5290] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:57.051730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341286310391793:5290] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:57.051740Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 415 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:57.062312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:57.063452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:57.063458Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:57.063461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:57.063466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:57.063474Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 414 [2:7559341285948580184:4861]
2025-10-09T21:44:57.063488Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:57.063509Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:57.063512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.063515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.063518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.063536Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:57.063542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:57.063549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:57.063612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.063614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.063616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.063619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:57.063699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:57.063701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.063712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.063722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.063736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:57.063757Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341285948580179:4861]
2025-10-09T21:44:57.063762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341285948580179:4861]
2025-10-09T21:44:57.063764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341285948580179:4861]
2025-10-09T21:44:57.064070Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:57.064086Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:57.064099Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:57.064101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:57.064144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341285948580179:4861]
2025-10-09T21:44:57.064146Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:57.064154Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:57.064158Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:57.064160Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:57.064168Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:57.064988Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 414
2025-10-09T21:44:57.065032Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 414
2025-10-09T21:44:57.065843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341286310391739:2432], now have 1 active actors on pipe
2025-10-09T21:44:57.065857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341286310391968:5290], now have 1 active actors on pipe
2025-10-09T21:44:57.065880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:57.065885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:57.065918Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c83f2802-60fbeae7-f7709846-6c25f9d1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:57.065927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.065932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.065935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:57.065948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.065973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:57.065992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:57.066437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:57.066451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:57.066474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:57.066957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 415 partition: 0 MaxSeqNo: 25 sessionId: src_id|c83f2802-60fbeae7-f7709846-6c25f9d1_0
2025-10-09T21:44:57.070159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e360f580-6e60cb2c-fdffbb21-f18f7b59_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c83f2802-60fbeae7-f7709846-6c25f9d1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:57.072901Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c83f2802-60fbeae7-f7709846-6c25f9d1_0] Write session: try to update token
2025-10-09T21:44:57.072924Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c83f2802-60fbeae7-f7709846-6c25f9d1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:57.085330Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:57.085351Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:57.085353Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:57.085355Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:57.085366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341285948580179:4861]
2025-10-09T21:44:57.085371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341285948580179:4861]
2025-10-09T21:44:57.085375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341285948580179:4861]
2025-10-09T21:44:57.085425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:57.085440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.085443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:57.085460Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:57.085465Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:57.085469Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:57.085589Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:57.095182Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 415 sessionId: src_id|c83f2802-60fbeae7-f7709846-6c25f9d1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:57.097763Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:57.113620Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341286310391980:5210]
2025-10-09T21:44:57.114224Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 414, pipe: [1:7559341286310391980:5210]
2025-10-09T21:44:57.410522Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:57.411132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:57.411148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:57.411194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341286310391980:5210], now have 1 active actors on pipe
2025-10-09T21:44:57.411586Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:57.411498Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341286310391980:5210]
2025-10-09T21:44:57.411528Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:57.411569Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:57.411571Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:57.411627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 415 reason: error at write: Disconnected sessionId: src_id|c83f2802-60fbeae7-f7709846-6c25f9d1_0
2025-10-09T21:44:57.411805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 415 sessionId: src_id|c83f2802-60fbeae7-f7709846-6c25f9d1_0 is DEAD
2025-10-09T21:44:57.412233Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c83f2802-60fbeae7-f7709846-6c25f9d1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:57.412242Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c83f2802-60fbeae7-f7709846-6c25f9d1_0] Write session will restart in 0.008305s
2025-10-09T21:44:57.412268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c83f2802-60fbeae7-f7709846-6c25f9d1_0] Write session: Do CDS request
2025-10-09T21:44:57.412275Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c83f2802-60fbeae7-f7709846-6c25f9d1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:57.421607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c83f2802-60fbeae7-f7709846-6c25f9d1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:57.421825Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:57.421842Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 416
2025-10-09T21:44:57.422193Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 416 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:57.422225Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 416 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:57.422231Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=416 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:57.422234Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:57.423032Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 416 sessionId:  describe result for acl check
2025-10-09T21:44:57.423076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:57.423082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:57.423085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:57.423093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341286310391998:5294] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:57.423670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341286310391998:5294] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:57.428401Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341286310391998:5294] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:57.432231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341286310391998:5294] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:57.432248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341286310391998:5294] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:57.432252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341286310391998:5294] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:57.432262Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 416 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:57.456230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:57.459121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:57.459360Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:57.459380Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:57.459395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:57.459460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:57.459507Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341285948580427:4865]
2025-10-09T21:44:57.460166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:57.462013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:57.462091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:57.462473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:57.462521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:57.462523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:57.462524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:57.462525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:57.462527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:57.462528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:57.462529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.462530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.462531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.462537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:57.462547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:57.462551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:57.462555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:57.462558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:57.462562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:57.462566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.462570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.462573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.462580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:57.462588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:57.462594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:57.462599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:57.462603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:57.462608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:57.462612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:57.462617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:57.462621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:57.462626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:57.462629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:57.462634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:57.462640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.462645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.462649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.462653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.462656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.462661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.462689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:57.473676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:57.475633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:57.475645Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:57.475649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:57.475653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:57.475660Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 415 [2:7559341285948580427:4865]
2025-10-09T21:44:57.475674Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:57.475692Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:57.475695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.475699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.475701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.475720Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:57.475726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:57.475733Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:57.475791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.475795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.475797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.475799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:57.475870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:57.475872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.475884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.475896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.475913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:57.475933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341285948580422:4865]
2025-10-09T21:44:57.475935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341285948580422:4865]
2025-10-09T21:44:57.475937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341285948580422:4865]
2025-10-09T21:44:57.475947Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:57.475960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:57.475971Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:57.475975Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:57.475977Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:57.475985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:57.475988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:57.476039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341286310391985:2432], now have 1 active actors on pipe
2025-10-09T21:44:57.476042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341286310392075:5294], now have 1 active actors on pipe
2025-10-09T21:44:57.476044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341285948580422:4865]
2025-10-09T21:44:57.476051Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:57.476081Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:57.478458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:57.478475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:57.478511Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7815ae1-7fe2e113-717172b7-a342d3c9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:57.478523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.478530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.478534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:57.478546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.478574Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:57.478601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:57.479474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:57.479482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:57.479502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:57.476849Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 415
2025-10-09T21:44:57.476899Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 415
2025-10-09T21:44:57.479647Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 416 partition: 0 MaxSeqNo: 25 sessionId: src_id|7815ae1-7fe2e113-717172b7-a342d3c9_0
2025-10-09T21:44:57.481475Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c83f2802-60fbeae7-f7709846-6c25f9d1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7815ae1-7fe2e113-717172b7-a342d3c9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:57.484147Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7815ae1-7fe2e113-717172b7-a342d3c9_0] Write session: try to update token
2025-10-09T21:44:57.484167Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7815ae1-7fe2e113-717172b7-a342d3c9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:57.498540Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:57.498558Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:57.498561Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:57.498563Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:57.498574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341285948580422:4865]
2025-10-09T21:44:57.498578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341285948580422:4865]
2025-10-09T21:44:57.498582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341285948580422:4865]
2025-10-09T21:44:57.498628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:57.498635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.498637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:57.498775Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:57.500098Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:57.500108Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:57.500111Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:57.508025Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 416 sessionId: src_id|7815ae1-7fe2e113-717172b7-a342d3c9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:57.510611Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:57.511392Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341286310392229:5210]
2025-10-09T21:44:57.512452Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 415, pipe: [1:7559341286310392229:5210]
2025-10-09T21:44:57.674804Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:57.675052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:57.675067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:57.675091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341286310392229:5210], now have 1 active actors on pipe
2025-10-09T21:44:57.675687Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:57.675708Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341286310392229:5210]
2025-10-09T21:44:57.675735Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:57.675776Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:57.675855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 416 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7815ae1-7fe2e113-717172b7-a342d3c9_0
2025-10-09T21:44:57.676065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 416 sessionId: src_id|7815ae1-7fe2e113-717172b7-a342d3c9_0 is DEAD
2025-10-09T21:44:57.678437Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7815ae1-7fe2e113-717172b7-a342d3c9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:57.678451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7815ae1-7fe2e113-717172b7-a342d3c9_0] Write session will restart in 0.008512s
2025-10-09T21:44:57.678479Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7815ae1-7fe2e113-717172b7-a342d3c9_0] Write session: Do CDS request
2025-10-09T21:44:57.678498Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7815ae1-7fe2e113-717172b7-a342d3c9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:57.688607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7815ae1-7fe2e113-717172b7-a342d3c9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:57.691320Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:57.691336Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 417
2025-10-09T21:44:57.691509Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 417 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:57.691543Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 417 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:57.691548Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=417 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:57.691552Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:57.692237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 417 sessionId:  describe result for acl check
2025-10-09T21:44:57.692285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:57.692287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:57.692289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:57.692295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341286310392303:5305] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:57.692849Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341286310392303:5305] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:57.696899Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341286310392303:5305] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:57.701578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341286310392303:5305] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:57.701601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341286310392303:5305] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:57.701605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341286310392303:5305] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:57.701614Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 417 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:57.707789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:57.710981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:57.711224Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:57.711237Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:57.711259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:57.711325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:57.711388Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341285948580696:4877]
2025-10-09T21:44:57.712048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:57.712943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:57.713000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:57.713387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:57.713449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:57.713455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:57.713456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:57.713457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:57.713458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:57.713459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:57.713460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.713461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.713462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.713467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:57.713476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:57.713479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:57.713483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:57.713486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:57.713489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:57.713492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.713495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.713499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.713506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:57.713513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:57.713519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:57.713524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:57.713527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:57.713531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:57.713535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:57.713539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:57.713543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:57.713562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:57.713575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:57.713580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:57.713584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.713589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.713592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.713597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.713600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.713605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.713631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:57.721138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:57.722252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:57.722268Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:57.722271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:57.722290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:57.722299Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 416 [2:7559341285948580696:4877]
2025-10-09T21:44:57.722313Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:57.722333Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:57.722342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.722346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.722348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.722367Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:57.722379Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:57.722387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:57.722455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.722464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.722466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.722469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:57.722534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:57.722543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.722556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.722566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.722583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:57.722611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341285948580691:4877]
2025-10-09T21:44:57.722619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341285948580691:4877]
2025-10-09T21:44:57.722621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341285948580691:4877]
2025-10-09T21:44:57.722639Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:57.722660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341286310392250:2432], now have 1 active actors on pipe
2025-10-09T21:44:57.722666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341286310392431:5305], now have 1 active actors on pipe
2025-10-09T21:44:57.722672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:57.722683Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:57.722687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:57.722688Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:57.722689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:57.722690Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:57.722725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341285948580691:4877]
2025-10-09T21:44:57.722733Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:57.722736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:57.722949Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 416
2025-10-09T21:44:57.723001Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 416
2025-10-09T21:44:57.723264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:57.723277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:57.723305Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b925accc-6572e2ac-95c567ce-2c5d145b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:57.723314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.723319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.723321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:57.723332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.723347Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:57.723364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:57.723575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:57.723583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:57.723601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:57.723732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 417 partition: 0 MaxSeqNo: 25 sessionId: src_id|b925accc-6572e2ac-95c567ce-2c5d145b_0
2025-10-09T21:44:57.724240Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7815ae1-7fe2e113-717172b7-a342d3c9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b925accc-6572e2ac-95c567ce-2c5d145b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:57.727099Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b925accc-6572e2ac-95c567ce-2c5d145b_0] Write session: try to update token
2025-10-09T21:44:57.727121Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b925accc-6572e2ac-95c567ce-2c5d145b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:57.739728Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:57.739746Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:57.739749Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:57.739752Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:57.739762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341285948580691:4877]
2025-10-09T21:44:57.739766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341285948580691:4877]
2025-10-09T21:44:57.739770Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341285948580691:4877]
2025-10-09T21:44:57.739816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:57.739824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.739827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:57.739981Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:57.745339Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:57.745354Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:57.745358Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:57.747487Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 417 sessionId: src_id|b925accc-6572e2ac-95c567ce-2c5d145b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:57.749936Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:57.775668Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341286310392502:5210]
2025-10-09T21:44:57.775978Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 416, pipe: [1:7559341286310392502:5210]
2025-10-09T21:44:57.912625Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:57.912907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:57.912919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:57.912966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341286310392502:5210], now have 1 active actors on pipe
2025-10-09T21:44:57.913337Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:57.913352Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341286310392502:5210]
2025-10-09T21:44:57.913375Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:57.913377Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:57.913427Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 417 reason: error at write: Disconnected sessionId: src_id|b925accc-6572e2ac-95c567ce-2c5d145b_0
2025-10-09T21:44:57.913807Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b925accc-6572e2ac-95c567ce-2c5d145b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:57.913822Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b925accc-6572e2ac-95c567ce-2c5d145b_0] Write session will restart in 0.006298s
2025-10-09T21:44:57.913849Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b925accc-6572e2ac-95c567ce-2c5d145b_0] Write session: Do CDS request
2025-10-09T21:44:57.913855Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b925accc-6572e2ac-95c567ce-2c5d145b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:57.913590Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 417 sessionId: src_id|b925accc-6572e2ac-95c567ce-2c5d145b_0 is DEAD
2025-10-09T21:44:57.913674Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:57.922596Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b925accc-6572e2ac-95c567ce-2c5d145b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:57.922909Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:57.922921Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 418
2025-10-09T21:44:57.923054Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 418 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:57.923140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 418 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:57.923146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=418 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:57.923151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:57.923574Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 418 sessionId:  describe result for acl check
2025-10-09T21:44:57.923603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:57.923605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:57.923607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:57.923612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341286310392522:5310] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:57.924236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341286310392522:5310] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:57.929066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341286310392522:5310] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:57.934518Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341286310392522:5310] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:57.934532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341286310392522:5310] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:57.934536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341286310392522:5310] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:57.934546Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 418 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:57.958054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:57.960944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:57.963419Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:57.963432Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:57.963452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:57.963539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:57.963599Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341285948580943:4881]
2025-10-09T21:44:57.967421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:57.967923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:57.968001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:57.968336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:57.968417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:57.968418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:57.968419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:57.968420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:57.968422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:57.968423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:57.968424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.968426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.968427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.968434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:57.968445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:57.968449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:57.968453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:57.968457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:57.968460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:57.968464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.968468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.968472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.968480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:57.968490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:57.968496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:57.968502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:57.968509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:57.968515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:57.968519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:57.968525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:57.968530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:57.968535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:57.968539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:57.968544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:57.968549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.968554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.968558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.968564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.968568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.968576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.968619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:57.977277Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 417
2025-10-09T21:44:57.977335Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 417
2025-10-09T21:44:57.975648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:57.976876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:57.976882Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:57.976884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:57.976889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:57.976896Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 417 [2:7559341285948580943:4881]
2025-10-09T21:44:57.976911Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:57.976933Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:57.976937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.976940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.976942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.976962Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:57.976968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:57.976975Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:57.977039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.977041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.977043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.977046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:57.977120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:57.977123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:57.977139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:57.977148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:57.977164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:57.977184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341285948580938:4881]
2025-10-09T21:44:57.977187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341285948580938:4881]
2025-10-09T21:44:57.977188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341285948580938:4881]
2025-10-09T21:44:57.977197Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:57.977205Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:57.977207Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:57.977209Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:57.977212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:57.977226Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:57.977230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:57.977242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341286310392507:2432], now have 1 active actors on pipe
2025-10-09T21:44:57.977245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341286310392576:5310], now have 1 active actors on pipe
2025-10-09T21:44:57.977282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341285948580938:4881]
2025-10-09T21:44:57.977291Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:57.977310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:57.977628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:57.977649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:57.977681Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c040dc73-f8836ba8-adab3272-3391135a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:57.977689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.977694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.977697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:57.977710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.977722Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:57.977742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:57.979368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:57.979392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:57.979415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:57.979580Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 418 partition: 0 MaxSeqNo: 25 sessionId: src_id|c040dc73-f8836ba8-adab3272-3391135a_0
2025-10-09T21:44:57.979969Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b925accc-6572e2ac-95c567ce-2c5d145b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c040dc73-f8836ba8-adab3272-3391135a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:57.982637Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c040dc73-f8836ba8-adab3272-3391135a_0] Write session: try to update token
2025-10-09T21:44:57.982658Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c040dc73-f8836ba8-adab3272-3391135a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:57.998004Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:57.998023Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:57.998048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341285948580938:4881]
2025-10-09T21:44:57.998100Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:57.998115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:57.998312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:57.998482Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:57.998616Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:57.998764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:57.998977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:57.999028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:58.006564Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:58.006585Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:58.006588Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:58.006590Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:58.006602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341285948580938:4881]
2025-10-09T21:44:58.006606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341285948580938:4881]
2025-10-09T21:44:58.006610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341285948580938:4881]
2025-10-09T21:44:58.006656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:58.006671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.006674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:58.006678Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:58.006699Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:58.006703Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:58.006830Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:58.009409Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 418 sessionId: src_id|c040dc73-f8836ba8-adab3272-3391135a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:58.012317Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:58.013142Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341290605360057:5210]
2025-10-09T21:44:58.013450Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 417, pipe: [1:7559341290605360057:5210]
2025-10-09T21:44:58.198675Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:58.199273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:58.199293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:58.199319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341290605360057:5210], now have 1 active actors on pipe
2025-10-09T21:44:58.199555Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341290605360057:5210]
2025-10-09T21:44:58.199585Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:58.199587Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:58.199618Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:58.199622Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:58.199694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 418 reason: error at write: Disconnected sessionId: src_id|c040dc73-f8836ba8-adab3272-3391135a_0
2025-10-09T21:44:58.199921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 418 sessionId: src_id|c040dc73-f8836ba8-adab3272-3391135a_0 is DEAD
2025-10-09T21:44:58.200143Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c040dc73-f8836ba8-adab3272-3391135a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:58.200157Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c040dc73-f8836ba8-adab3272-3391135a_0] Write session will restart in 0.007537s
2025-10-09T21:44:58.200189Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c040dc73-f8836ba8-adab3272-3391135a_0] Write session: Do CDS request
2025-10-09T21:44:58.200196Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c040dc73-f8836ba8-adab3272-3391135a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:58.208711Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c040dc73-f8836ba8-adab3272-3391135a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:58.208930Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:58.208966Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 419
2025-10-09T21:44:58.209144Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 419 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:58.209183Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 419 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:58.209189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=419 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:58.209194Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:58.209747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 419 sessionId:  describe result for acl check
2025-10-09T21:44:58.209786Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:58.209794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:58.209797Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:58.209804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341290605360073:5315] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:58.210445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341290605360073:5315] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:58.213558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341290605360073:5315] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:58.219740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341290605360073:5315] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:58.219758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341290605360073:5315] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:58.219763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341290605360073:5315] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:58.219773Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 419 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:58.232647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:58.235906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:58.236296Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:58.236310Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:58.236325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:58.236397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:58.236461Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341290243548482:4884]
2025-10-09T21:44:58.237117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:58.237522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:58.237602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:58.237957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:58.238057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:58.238064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:58.238066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:58.238067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:58.238068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:58.238069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:58.238070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.238072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.238073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.238080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:58.238092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:58.238096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:58.238099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:58.238103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:58.238108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:58.238112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.238115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.238118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.238126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:58.238134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:58.238140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:58.238146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:58.238153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:58.238158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:58.238162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:58.238167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:58.238172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:58.238177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:58.238181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:58.238186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:58.238190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.238196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.238208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.238213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.238218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.238224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.238257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:58.250254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:58.251898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:58.251917Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:58.251920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:58.251925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:58.251933Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 418 [2:7559341290243548482:4884]
2025-10-09T21:44:58.251967Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:58.251989Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:58.251993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:58.251996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:58.251999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:58.252019Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:58.252025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:58.252033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:58.252102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.252112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.252114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.252118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:58.252194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:58.252197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.252211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.252225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.252249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:58.252275Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341290243548477:4884]
2025-10-09T21:44:58.252284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341290243548477:4884]
2025-10-09T21:44:58.252287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341290243548477:4884]
2025-10-09T21:44:58.252301Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:58.252321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:58.252340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:58.252353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:58.252400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341290243548477:4884]
2025-10-09T21:44:58.252408Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:58.252537Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:58.252547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:58.252549Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:58.252555Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:58.252849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341290605360175:5315], now have 1 active actors on pipe
2025-10-09T21:44:58.252875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341290605360060:2432], now have 1 active actors on pipe
2025-10-09T21:44:58.254511Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 418
2025-10-09T21:44:58.254573Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 418
2025-10-09T21:44:58.254862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:58.254878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:58.254911Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|144af381-b4d87b87-9afb55c-ac310083_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:58.254929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:58.254939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:58.254942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:58.254956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:58.254972Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:58.254997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:58.257358Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 419 partition: 0 MaxSeqNo: 25 sessionId: src_id|144af381-b4d87b87-9afb55c-ac310083_0
2025-10-09T21:44:58.257101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:58.257116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:58.257145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:58.257906Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c040dc73-f8836ba8-adab3272-3391135a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|144af381-b4d87b87-9afb55c-ac310083_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:58.260456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|144af381-b4d87b87-9afb55c-ac310083_0] Write session: try to update token
2025-10-09T21:44:58.260477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|144af381-b4d87b87-9afb55c-ac310083_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:58.273323Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:58.273343Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:58.273346Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:58.273348Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:58.273359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341290243548477:4884]
2025-10-09T21:44:58.273364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341290243548477:4884]
2025-10-09T21:44:58.273369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341290243548477:4884]
2025-10-09T21:44:58.273418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:58.273426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.273425Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:58.273429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:58.273432Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:58.273436Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:58.273561Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:58.288526Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 419 sessionId: src_id|144af381-b4d87b87-9afb55c-ac310083_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:58.290952Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:58.299877Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341290605360308:5210]
2025-10-09T21:44:58.300315Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 418, pipe: [1:7559341290605360308:5210]
2025-10-09T21:44:58.456890Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:58.457454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:58.457464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:58.457486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341290605360308:5210], now have 1 active actors on pipe
2025-10-09T21:44:58.459056Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|144af381-b4d87b87-9afb55c-ac310083_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:58.459068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|144af381-b4d87b87-9afb55c-ac310083_0] Write session will restart in 0.007726s
2025-10-09T21:44:58.459092Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|144af381-b4d87b87-9afb55c-ac310083_0] Write session: Do CDS request
2025-10-09T21:44:58.459098Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|144af381-b4d87b87-9afb55c-ac310083_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:58.460459Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:58.458400Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:58.458412Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:58.458473Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 419 reason: error at write: Disconnected sessionId: src_id|144af381-b4d87b87-9afb55c-ac310083_0
2025-10-09T21:44:58.458648Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 419 sessionId: src_id|144af381-b4d87b87-9afb55c-ac310083_0 is DEAD
2025-10-09T21:44:58.458778Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:58.458818Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341290605360308:5210]
2025-10-09T21:44:58.470582Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|144af381-b4d87b87-9afb55c-ac310083_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:58.470622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:58.470638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 420
2025-10-09T21:44:58.470866Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 420 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:58.470909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 420 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:58.470918Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=420 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:58.470922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:58.471432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 420 sessionId:  describe result for acl check
2025-10-09T21:44:58.471486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:58.471493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:58.471495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:58.471501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341290605360324:5319] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:58.472173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341290605360324:5319] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:58.474588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341290605360324:5319] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:58.480666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341290605360324:5319] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:58.480683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341290605360324:5319] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:58.480688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341290605360324:5319] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:58.480698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 420 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:58.500544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:58.501499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:58.501704Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:58.501713Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:58.501725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:58.501774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:58.501815Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341290243548725:4887]
2025-10-09T21:44:58.502483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:58.503023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:58.503080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:58.503591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:58.503651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:58.503652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:58.503654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:58.503655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:58.503656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:58.503658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:58.503659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.503661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.503662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.503667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:58.503678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:58.503682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:58.503687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:58.503691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:58.503695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:58.503699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.503703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.503707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.503715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:58.503723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:58.503739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:58.503745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:58.503751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:58.503756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:58.503761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:58.503767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:58.503771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:58.503777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:58.503781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:58.503787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:58.503792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.503798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.503802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.503808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.503813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.503818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.503849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:58.512447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:58.513669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:58.513676Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:58.513679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:58.513684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:58.513692Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 419 [2:7559341290243548725:4887]
2025-10-09T21:44:58.513707Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:58.513728Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:58.513732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:58.513736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:58.513739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:58.513760Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:58.513765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:58.513774Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:58.513835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.513838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.513840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.513843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:58.513921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:58.513924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.513939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.513951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.513969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:58.513990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341290243548720:4887]
2025-10-09T21:44:58.513993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341290243548720:4887]
2025-10-09T21:44:58.513995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341290243548720:4887]
2025-10-09T21:44:58.514006Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:58.514024Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:58.514100Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:58.514104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:58.514144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341290243548720:4887]
2025-10-09T21:44:58.514148Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:58.514878Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:58.514886Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:58.514889Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:58.514896Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:58.515323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341290605360313:2432], now have 1 active actors on pipe
2025-10-09T21:44:58.515340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341290605360430:5319], now have 1 active actors on pipe
2025-10-09T21:44:58.516653Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 419
2025-10-09T21:44:58.516700Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 419
2025-10-09T21:44:58.522625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:58.522642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:58.522692Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b811498c-7d6b069-d0e43f88-cdeef908_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:58.522709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:58.522718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:58.522721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:58.522737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:58.522758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:58.522781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:58.523934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:58.523950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:58.523978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:58.524136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 420 partition: 0 MaxSeqNo: 25 sessionId: src_id|b811498c-7d6b069-d0e43f88-cdeef908_0
2025-10-09T21:44:58.524642Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|144af381-b4d87b87-9afb55c-ac310083_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b811498c-7d6b069-d0e43f88-cdeef908_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:58.527471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b811498c-7d6b069-d0e43f88-cdeef908_0] Write session: try to update token
2025-10-09T21:44:58.527498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b811498c-7d6b069-d0e43f88-cdeef908_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:58.542098Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:58.542115Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:58.542125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341290243548720:4887]
2025-10-09T21:44:58.542174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:58.542354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:58.542488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:58.542621Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:58.542743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:58.542916Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:58.542950Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:58.546721Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:58.559481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341290605360562:5210], now have 1 active actors on pipe
2025-10-09T21:44:58.558995Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341290605360562:5210]
2025-10-09T21:44:58.559650Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 419, pipe: [1:7559341290605360562:5210]
2025-10-09T21:44:58.569070Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 420 sessionId: src_id|b811498c-7d6b069-d0e43f88-cdeef908_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:58.571763Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:58.572666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:58.572686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:58.572727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:58.578195Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:58.585027Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:58.585043Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:58.585046Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:58.585048Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:58.585059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341290243548720:4887]
2025-10-09T21:44:58.585064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341290243548720:4887]
2025-10-09T21:44:58.585068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341290243548720:4887]
2025-10-09T21:44:58.585112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:58.585120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.585123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:58.585276Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:58.589282Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:58.589294Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:58.589297Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:58.810363Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:58.819093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:58.819133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:58.819326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:58.819418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:58.819483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:58.819547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:58.819595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:58.819601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:58.819679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:58.819737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:58.819797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:58.819860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:58.819899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:58.819903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:58.819972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:58.820036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:58.820096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:58.820160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:58.820201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:58.820204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:58.820278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:58.820343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:58.820415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:58.820473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:58.820511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:58.820513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:58.820579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:58.820639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:58.820700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:58.820761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:58.820802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:58.820805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:58.820868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:58.820963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:58.821032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:58.821107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:58.821148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:58.821151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:58.821215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:58.821268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:58.821326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:58.821381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:58.821421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:58.821423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:58.821484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:58.821538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:58.821596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:58.821653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:58.821693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:58.821695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:58.821760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:58.821815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:58.821876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:58.821937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:58.821981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:58.821984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:58.822071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:58.822126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:58.822186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:58.822247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:58.822312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:58.822315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:58.824183Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:58.824254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 420 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b811498c-7d6b069-d0e43f88-cdeef908_0
2025-10-09T21:44:58.824476Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 420 sessionId: src_id|b811498c-7d6b069-d0e43f88-cdeef908_0 is DEAD
2025-10-09T21:44:58.824615Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b811498c-7d6b069-d0e43f88-cdeef908_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:44:58.824641Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b811498c-7d6b069-d0e43f88-cdeef908_0] Write session will restart in 0.008061s
2025-10-09T21:44:58.824667Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b811498c-7d6b069-d0e43f88-cdeef908_0] Write session: Do CDS request
2025-10-09T21:44:58.824673Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b811498c-7d6b069-d0e43f88-cdeef908_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:58.824665Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341290605360562:5210]
2025-10-09T21:44:58.824690Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:58.823528Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:58.835045Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b811498c-7d6b069-d0e43f88-cdeef908_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:58.835436Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:58.835453Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 421
2025-10-09T21:44:58.835583Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 421 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:58.835616Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 421 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:58.835623Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=421 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:58.835635Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:58.836285Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 421 sessionId:  describe result for acl check
2025-10-09T21:44:58.836328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:58.836331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:58.836333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:58.836350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341290605360613:5331] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:58.836953Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341290605360613:5331] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:58.841915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341290605360613:5331] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:58.848546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341290605360613:5331] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:58.848568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341290605360613:5331] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:58.848573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341290605360613:5331] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:58.848583Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 421 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:58.856395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:58.859510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:58.859786Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:58.859795Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:58.859810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:58.859897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:58.859950Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341290243549006:4901]
2025-10-09T21:44:58.860594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:58.860979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:58.861045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:58.861406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:58.861461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:58.861463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:58.861464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:58.861465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:58.861467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:58.861468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:58.861470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.861471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.861473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.861478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:58.861490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:58.861495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:58.861499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:58.861504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:58.861509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:58.861513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.861518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.861522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.861529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:58.861538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:58.861544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:58.861550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:58.861555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:58.861561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:58.861566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:58.861572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:58.861577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:58.861583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:58.861588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:58.861594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:58.861612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.861618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.861623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.861628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.861633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.861640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.861671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:58.870649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:58.872179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:58.872186Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:58.872189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:58.872195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:58.872203Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 420 [2:7559341290243549006:4901]
2025-10-09T21:44:58.872220Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:58.872242Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:58.872246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:58.872249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:58.872251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:58.872270Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:58.872280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:58.872288Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:58.872353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.872355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.872357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.872360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:58.872438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:58.872441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.872454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:58.872466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:58.872483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:58.872503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341290243549001:4901]
2025-10-09T21:44:58.872506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341290243549001:4901]
2025-10-09T21:44:58.872508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341290243549001:4901]
2025-10-09T21:44:58.872522Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:58.872539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:58.872550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:58.872552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:58.872587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341290243549001:4901]
2025-10-09T21:44:58.872589Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:58.873499Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 420
2025-10-09T21:44:58.873538Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 420
2025-10-09T21:44:58.873132Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:58.873141Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:58.873144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:58.873151Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:58.874140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341290605360761:5331], now have 1 active actors on pipe
2025-10-09T21:44:58.874152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341290605360590:2432], now have 1 active actors on pipe
2025-10-09T21:44:58.874170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:58.874176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:58.874211Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a5f098b4-11f427e5-ab7c637c-12fe4e4d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:58.874221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:58.874225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:58.874228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:58.874240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:58.874256Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:58.874286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:58.876274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:58.876287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:58.876326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:58.877404Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 421 partition: 0 MaxSeqNo: 25 sessionId: src_id|a5f098b4-11f427e5-ab7c637c-12fe4e4d_0
2025-10-09T21:44:58.877833Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b811498c-7d6b069-d0e43f88-cdeef908_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a5f098b4-11f427e5-ab7c637c-12fe4e4d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:58.880499Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a5f098b4-11f427e5-ab7c637c-12fe4e4d_0] Write session: try to update token
2025-10-09T21:44:58.880519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a5f098b4-11f427e5-ab7c637c-12fe4e4d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:58.891213Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:58.891234Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:58.891236Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:58.891238Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:58.891249Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341290243549001:4901]
2025-10-09T21:44:58.891255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341290243549001:4901]
2025-10-09T21:44:58.891260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341290243549001:4901]
2025-10-09T21:44:58.891321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:58.891329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:58.891332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:58.891566Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:58.894322Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:58.894328Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:58.894335Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:58.899964Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 421 sessionId: src_id|a5f098b4-11f427e5-ab7c637c-12fe4e4d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:58.902918Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:58.925069Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341290605360844:5210]
2025-10-09T21:44:58.926280Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 420, pipe: [1:7559341290605360844:5210]
2025-10-09T21:44:59.076388Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:59.078461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:59.078482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:59.078531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341290605360844:5210], now have 1 active actors on pipe
2025-10-09T21:44:59.081777Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:59.082118Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341290605360844:5210]
2025-10-09T21:44:59.082160Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:59.082163Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:59.082229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 421 reason: error at write: Disconnected sessionId: src_id|a5f098b4-11f427e5-ab7c637c-12fe4e4d_0
2025-10-09T21:44:59.082456Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:59.082495Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 421 sessionId: src_id|a5f098b4-11f427e5-ab7c637c-12fe4e4d_0 is DEAD
2025-10-09T21:44:59.082626Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a5f098b4-11f427e5-ab7c637c-12fe4e4d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:59.082638Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a5f098b4-11f427e5-ab7c637c-12fe4e4d_0] Write session will restart in 0.008549s
2025-10-09T21:44:59.082674Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a5f098b4-11f427e5-ab7c637c-12fe4e4d_0] Write session: Do CDS request
2025-10-09T21:44:59.082680Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a5f098b4-11f427e5-ab7c637c-12fe4e4d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:59.094500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a5f098b4-11f427e5-ab7c637c-12fe4e4d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:59.094803Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:59.094819Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 422
2025-10-09T21:44:59.094922Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 422 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:59.094950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 422 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:59.094956Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=422 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:59.094960Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:59.098662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 422 sessionId:  describe result for acl check
2025-10-09T21:44:59.098711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:59.098713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:59.098716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:59.098722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341294900328163:5336] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:59.099305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341294900328163:5336] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:59.103708Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341294900328163:5336] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:59.108135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341294900328163:5336] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:59.108151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341294900328163:5336] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:59.108155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341294900328163:5336] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:59.108164Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 422 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:59.126215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:59.129378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:59.129628Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:59.129638Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:59.129651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:59.129711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:59.129763Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341294538516547:4903]
2025-10-09T21:44:59.130405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:59.130874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:59.130933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:59.131274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:59.131322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:59.131324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:59.131325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:59.131326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:59.131327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:59.131329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:59.131330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.131331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.131332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.131337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:59.131349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:59.131354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:59.131358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:59.131362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:59.131366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:59.131370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.131375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.131379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.131387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:59.131395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:59.131400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:59.131406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:59.131411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:59.131416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:59.131420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:59.131425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:59.131430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:59.131436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:59.131442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:59.131447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:59.131452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.131457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.131461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.131467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.131471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.131476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.131508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:59.139443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:59.140562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:59.140581Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:59.140584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:59.140589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:59.140597Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 421 [2:7559341294538516547:4903]
2025-10-09T21:44:59.140614Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:59.140637Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:59.140648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:59.140651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:59.140654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:59.140674Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:59.140686Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:59.140694Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:59.140761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.140769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.140771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.140774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:59.140855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:59.140865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.140878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.140888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.140904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:59.140927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341294900328238:5336], now have 1 active actors on pipe
2025-10-09T21:44:59.140931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341294900328150:2432], now have 1 active actors on pipe
2025-10-09T21:44:59.140938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341294538516542:4903]
2025-10-09T21:44:59.140941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341294538516542:4903]
2025-10-09T21:44:59.140943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341294538516542:4903]
2025-10-09T21:44:59.140957Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:59.140972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:59.140980Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:59.140987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:59.140989Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:59.140991Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:59.140992Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:59.141033Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341294538516542:4903]
2025-10-09T21:44:59.141042Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:59.141053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:59.141071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:59.141082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:59.141122Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|31b226f6-5043d612-9aa1008e-371366bd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:59.141137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:59.141141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:59.141144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:59.141013Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 421
2025-10-09T21:44:59.141027Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 421
2025-10-09T21:44:59.141156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:59.141172Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:59.141200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:59.141416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:59.141428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:59.141448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:59.141578Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 422 partition: 0 MaxSeqNo: 25 sessionId: src_id|31b226f6-5043d612-9aa1008e-371366bd_0
2025-10-09T21:44:59.142000Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a5f098b4-11f427e5-ab7c637c-12fe4e4d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|31b226f6-5043d612-9aa1008e-371366bd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:59.144663Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|31b226f6-5043d612-9aa1008e-371366bd_0] Write session: try to update token
2025-10-09T21:44:59.144682Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|31b226f6-5043d612-9aa1008e-371366bd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:59.160406Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:59.160425Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:59.160427Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:59.160429Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:59.160440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341294538516542:4903]
2025-10-09T21:44:59.160447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341294538516542:4903]
2025-10-09T21:44:59.160452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341294538516542:4903]
2025-10-09T21:44:59.160492Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:59.160497Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:59.160499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:59.160503Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:59.160507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.160510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:59.160648Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:59.163085Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 422 sessionId: src_id|31b226f6-5043d612-9aa1008e-371366bd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:59.165678Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:59.181595Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341294900328400:5210]
2025-10-09T21:44:59.188537Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 421, pipe: [1:7559341294900328400:5210]
2025-10-09T21:44:59.411024Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:59.411627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:59.411639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:59.411701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341294900328400:5210], now have 1 active actors on pipe
2025-10-09T21:44:59.412364Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:59.412389Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:59.412446Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 422 reason: error at write: Disconnected sessionId: src_id|31b226f6-5043d612-9aa1008e-371366bd_0
2025-10-09T21:44:59.412138Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:59.412648Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 422 sessionId: src_id|31b226f6-5043d612-9aa1008e-371366bd_0 is DEAD
2025-10-09T21:44:59.412802Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:59.412838Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341294900328400:5210]
2025-10-09T21:44:59.412900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31b226f6-5043d612-9aa1008e-371366bd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:59.412913Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31b226f6-5043d612-9aa1008e-371366bd_0] Write session will restart in 0.009387s
2025-10-09T21:44:59.412940Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31b226f6-5043d612-9aa1008e-371366bd_0] Write session: Do CDS request
2025-10-09T21:44:59.412947Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31b226f6-5043d612-9aa1008e-371366bd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:59.426858Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|31b226f6-5043d612-9aa1008e-371366bd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:59.427222Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:59.427238Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 423
2025-10-09T21:44:59.427371Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 423 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:59.427406Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 423 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:59.427411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=423 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:59.427415Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:59.428266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 423 sessionId:  describe result for acl check
2025-10-09T21:44:59.428305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:59.428307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:59.428308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:59.428314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341294900328419:5340] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:59.428871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341294900328419:5340] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:59.444638Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341294900328419:5340] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:59.450051Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341294900328419:5340] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:59.450069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341294900328419:5340] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:59.450074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341294900328419:5340] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:59.450088Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 423 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:59.456647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:59.459947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:59.460248Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:59.460259Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:59.460273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:59.460343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:59.460391Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341294538516794:4907]
2025-10-09T21:44:59.460970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:59.461359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:59.461411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:59.462017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:59.462120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:59.462127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:59.462128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:59.462129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:59.462130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:59.462131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:59.462132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.462134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.462135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.462141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:59.462152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:59.462155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:59.462159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:59.462162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:59.462166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:59.462169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.462172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.462176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.462184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:59.462194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:59.462200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:59.462205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:59.462209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:59.462213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:59.462217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:59.462222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:59.462226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:59.462231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:59.462234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:59.462239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:59.462244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.462248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.462252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.462256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.462260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.462265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.462302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:59.469235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:59.470648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:59.470662Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:59.470665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:59.470668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:59.470678Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 422 [2:7559341294538516794:4907]
2025-10-09T21:44:59.470695Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:59.470718Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:59.470721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:59.470724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:59.470726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:59.470746Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:59.470756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:59.470764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:59.470825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.470833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.470835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.470838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:59.470908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:59.470916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.470927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.470937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.470952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:59.470974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341294900328559:5340], now have 1 active actors on pipe
2025-10-09T21:44:59.470982Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341294538516789:4907]
2025-10-09T21:44:59.470984Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341294538516789:4907]
2025-10-09T21:44:59.470986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341294538516789:4907]
2025-10-09T21:44:59.471000Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:59.471010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341294900328404:2432], now have 1 active actors on pipe
2025-10-09T21:44:59.471019Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:59.471022Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:59.471026Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:59.471028Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:59.471032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:59.471036Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:59.471069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341294538516789:4907]
2025-10-09T21:44:59.471076Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:59.471084Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:59.474073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:59.474090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:59.474139Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|851e6650-9f4391ce-b048ecd7-59ebda5a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:59.473957Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 422
2025-10-09T21:44:59.474157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:59.473993Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 422
2025-10-09T21:44:59.474167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:59.474171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:59.474184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:59.474200Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:59.474224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:59.478604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:59.478618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:59.478650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:59.479693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 423 partition: 0 MaxSeqNo: 25 sessionId: src_id|851e6650-9f4391ce-b048ecd7-59ebda5a_0
2025-10-09T21:44:59.480174Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31b226f6-5043d612-9aa1008e-371366bd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|851e6650-9f4391ce-b048ecd7-59ebda5a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:59.482649Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|851e6650-9f4391ce-b048ecd7-59ebda5a_0] Write session: try to update token
2025-10-09T21:44:59.482666Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|851e6650-9f4391ce-b048ecd7-59ebda5a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:59.493910Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:44:59.493925Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:59.493937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341294538516789:4907]
2025-10-09T21:44:59.493991Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:44:59.494200Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:44:59.494372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:59.494518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:59.494663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:44:59.494849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:44:59.494893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:44:59.495203Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:44:59.513336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341294900328655:5210], now have 1 active actors on pipe
2025-10-09T21:44:59.512892Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341294900328655:5210]
2025-10-09T21:44:59.513485Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 422, pipe: [1:7559341294900328655:5210]
2025-10-09T21:44:59.513494Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 423 sessionId: src_id|851e6650-9f4391ce-b048ecd7-59ebda5a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:59.516178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:59.516195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:59.516234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:59.515875Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:59.518421Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:44:59.525657Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:59.525679Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:59.525682Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:59.525685Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:59.525697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341294538516789:4907]
2025-10-09T21:44:59.525702Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341294538516789:4907]
2025-10-09T21:44:59.525707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341294538516789:4907]
2025-10-09T21:44:59.525758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:59.525767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.525770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:59.525944Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:59.529868Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:59.529882Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:59.529885Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:59.718461Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:59.726875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:59.726902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:59.727071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:59.727129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:59.727189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:59.727246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:59.727288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:59.727296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:59.727365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:59.727421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:59.727482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:59.727544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:59.727583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:59.727586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:59.727648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:59.727702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:59.727756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:59.727813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:59.727848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:59.727851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:59.727918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:59.727975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:59.728035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:59.728093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:59.728138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:59.728141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:59.728211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:59.728265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:59.728324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:59.728402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:59.728442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:59.728445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:59.728509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:59.728566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:59.728627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:59.728690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:59.728729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:59.728733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:59.728797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:59.728852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:59.728908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:59.728965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:59.729004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:59.729007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:59.729068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:59.729123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:59.729181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:59.729238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:59.729277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:59.729280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:59.729339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:59.729396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:59.729455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:59.729515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:59.729559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:59.729561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:59.729629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:44:59.729681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:44:59.729738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:44:59.729799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:44:59.729933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:44:59.729936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:44:59.730761Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341294900328655:5210]
2025-10-09T21:44:59.730897Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:59.730900Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:59.730931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 423 reason: error at write: Disconnected sessionId: src_id|851e6650-9f4391ce-b048ecd7-59ebda5a_0
2025-10-09T21:44:59.731116Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 423 sessionId: src_id|851e6650-9f4391ce-b048ecd7-59ebda5a_0 is DEAD
2025-10-09T21:44:59.731261Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:59.731582Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:59.734298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|851e6650-9f4391ce-b048ecd7-59ebda5a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:59.734312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|851e6650-9f4391ce-b048ecd7-59ebda5a_0] Write session will restart in 0.005414s
2025-10-09T21:44:59.734350Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|851e6650-9f4391ce-b048ecd7-59ebda5a_0] Write session: Do CDS request
2025-10-09T21:44:59.734358Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|851e6650-9f4391ce-b048ecd7-59ebda5a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:59.740568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|851e6650-9f4391ce-b048ecd7-59ebda5a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:59.740695Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:59.740711Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 424
2025-10-09T21:44:59.740881Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 424 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:59.740932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 424 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:59.740937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=424 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:59.740942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:59.741479Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 424 sessionId:  describe result for acl check
2025-10-09T21:44:59.741515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:59.741517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:59.741519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:59.741525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341294900328673:5345] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:59.742103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341294900328673:5345] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:59.749229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341294900328673:5345] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:59.757161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341294900328673:5345] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:59.757177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341294900328673:5345] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:59.757181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341294900328673:5345] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:59.757192Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 424 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:59.772815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:59.775973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:59.776211Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:59.776216Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:44:59.776230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:59.776290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:59.776329Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341294538517053:4909]
2025-10-09T21:44:59.776902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:44:59.777487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:44:59.777529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:44:59.777857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:44:59.777906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:59.777908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:59.777909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:59.777910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:59.777911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:59.777912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:59.777913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.777915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.777916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.777921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:59.777930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:59.777934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:59.777937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:59.777940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:59.777944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:59.777947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.777950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.777955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.777962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:59.777971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:44:59.777980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:59.777986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:44:59.777991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:59.777997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:44:59.778001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:59.778005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:44:59.778009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:59.778014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:44:59.778018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:59.778022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:44:59.778049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.778054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.778058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.778061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.778065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.778069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.778095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:44:59.785955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:44:59.787335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:44:59.787341Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:44:59.787344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:59.787348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:59.787355Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 423 [2:7559341294538517053:4909]
2025-10-09T21:44:59.787367Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:44:59.787388Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:44:59.787391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:59.787394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:59.787396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:59.787413Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:44:59.787418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:59.787426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:44:59.787480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.787482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.787484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.787486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:44:59.787560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:44:59.787563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.787575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:44:59.787586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:44:59.787602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:44:59.787622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341294538517047:4909]
2025-10-09T21:44:59.787624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341294538517047:4909]
2025-10-09T21:44:59.787626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341294538517047:4909]
2025-10-09T21:44:59.787639Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:44:59.787654Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:44:59.787668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:44:59.787672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:44:59.787711Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341294538517047:4909]
2025-10-09T21:44:59.787714Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:44:59.788053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:44:59.788060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:44:59.788062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:44:59.788068Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:44:59.788632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341294900328661:2432], now have 1 active actors on pipe
2025-10-09T21:44:59.788640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341294900328791:5345], now have 1 active actors on pipe
2025-10-09T21:44:59.789786Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 423
2025-10-09T21:44:59.789826Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 423
2025-10-09T21:44:59.789946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:59.789968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:59.789999Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dd2724ae-c9ea9913-3473f644-a1d495f2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:59.790011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:59.790017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:59.790020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:59.790043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:59.790058Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:59.790076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:59.790749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:59.790762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:59.790782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:59.794117Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 424 partition: 0 MaxSeqNo: 25 sessionId: src_id|dd2724ae-c9ea9913-3473f644-a1d495f2_0
2025-10-09T21:44:59.794553Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|851e6650-9f4391ce-b048ecd7-59ebda5a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dd2724ae-c9ea9913-3473f644-a1d495f2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:59.797086Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dd2724ae-c9ea9913-3473f644-a1d495f2_0] Write session: try to update token
2025-10-09T21:44:59.797106Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dd2724ae-c9ea9913-3473f644-a1d495f2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:44:59.810403Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:44:59.810420Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:44:59.810422Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:44:59.810423Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:44:59.810431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341294538517047:4909]
2025-10-09T21:44:59.810435Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341294538517047:4909]
2025-10-09T21:44:59.810440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341294538517047:4909]
2025-10-09T21:44:59.810483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:44:59.810490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:44:59.810492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:44:59.810624Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:44:59.813645Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:44:59.813658Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:44:59.813661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:44:59.814750Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 424 sessionId: src_id|dd2724ae-c9ea9913-3473f644-a1d495f2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:59.817272Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:59.830769Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341294900328926:5210]
2025-10-09T21:44:59.831254Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 423, pipe: [1:7559341294900328926:5210]
2025-10-09T21:44:59.981682Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:44:59.982022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:59.982050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:59.982098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341294900328926:5210], now have 1 active actors on pipe
2025-10-09T21:44:59.982976Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:44:59.986117Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd2724ae-c9ea9913-3473f644-a1d495f2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:44:59.986132Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd2724ae-c9ea9913-3473f644-a1d495f2_0] Write session will restart in 0.009791s
2025-10-09T21:44:59.982862Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:44:59.982901Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:44:59.982905Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:44:59.986158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd2724ae-c9ea9913-3473f644-a1d495f2_0] Write session: Do CDS request
2025-10-09T21:44:59.982916Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341294900328926:5210]
2025-10-09T21:44:59.982967Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 424 reason: error at write: Disconnected sessionId: src_id|dd2724ae-c9ea9913-3473f644-a1d495f2_0
2025-10-09T21:44:59.983167Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 424 sessionId: src_id|dd2724ae-c9ea9913-3473f644-a1d495f2_0 is DEAD
2025-10-09T21:44:59.986166Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd2724ae-c9ea9913-3473f644-a1d495f2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:44:59.997597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dd2724ae-c9ea9913-3473f644-a1d495f2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:59.997719Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:59.997747Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 425
2025-10-09T21:44:59.998569Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 425 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:59.998619Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 425 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:44:59.998626Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=425 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:59.998640Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:59.999179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 425 sessionId:  describe result for acl check
2025-10-09T21:44:59.999211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:59.999213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:59.999214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:59.999220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341294900329076:5357] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:59.999795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341294900329076:5357] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:00.004973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341294900329076:5357] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:00.008103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:00.011272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:00.011506Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:00.011516Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:00.011528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:00.011578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:00.011614Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341298833484619:4923]
2025-10-09T21:45:00.012229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:00.012944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:00.012989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:00.013320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:00.013360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:00.013361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:00.013362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:00.013363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:00.013365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:00.013366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:00.013367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.013368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.013370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.013375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:00.013386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:00.013391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:00.013395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:00.013398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:00.013402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:00.013406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.013410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.013414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.013422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:00.013431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:00.013437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:00.013442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:00.013446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:00.013451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:00.013456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:00.013461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:00.013466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:00.013491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:00.013495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:00.013500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:00.013506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.013511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.013516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.013521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.013524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.013531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.013562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:00.014428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341294900329076:5357] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:00.014450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341294900329076:5357] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:00.014454Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341294900329076:5357] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:00.014465Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 425 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:00.024705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:00.026428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:00.026434Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:00.026437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:00.026441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:00.026449Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 424 [2:7559341298833484619:4923]
2025-10-09T21:45:00.026466Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:00.026488Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:00.026492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.026495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.026497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.026516Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:00.026522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:00.026529Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:00.026594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.026596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.026598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.026601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:00.026678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:00.026681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.026695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.026707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.026722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:00.026742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341298833484614:4923]
2025-10-09T21:45:00.026745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341298833484614:4923]
2025-10-09T21:45:00.026747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341298833484614:4923]
2025-10-09T21:45:00.026755Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:00.026769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:00.026783Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:00.026786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:00.026801Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:00.026804Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:00.026807Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:00.026825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341298833484614:4923]
2025-10-09T21:45:00.026828Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:00.026831Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:00.026841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341299195296490:5357], now have 1 active actors on pipe
2025-10-09T21:45:00.026845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341294900328948:2432], now have 1 active actors on pipe
2025-10-09T21:45:00.027672Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 424
2025-10-09T21:45:00.027706Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 424
2025-10-09T21:45:00.028403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:00.028421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:00.028463Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|380dc606-ba143f60-160846cf-68756ec5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:00.028480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.028490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.028493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:00.028507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.028529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:00.028557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:00.029102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:00.029117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:00.029139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:00.030312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 425 partition: 0 MaxSeqNo: 25 sessionId: src_id|380dc606-ba143f60-160846cf-68756ec5_0
2025-10-09T21:45:00.031484Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd2724ae-c9ea9913-3473f644-a1d495f2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|380dc606-ba143f60-160846cf-68756ec5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:00.034083Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|380dc606-ba143f60-160846cf-68756ec5_0] Write session: try to update token
2025-10-09T21:45:00.034109Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|380dc606-ba143f60-160846cf-68756ec5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:00.046983Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:00.046998Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:00.047001Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:00.047003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:00.047014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341298833484614:4923]
2025-10-09T21:45:00.047019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341298833484614:4923]
2025-10-09T21:45:00.047023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341298833484614:4923]
2025-10-09T21:45:00.047076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:00.047084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.047087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:00.047223Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:00.051444Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:00.051458Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:00.051463Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:00.055282Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 425 sessionId: src_id|380dc606-ba143f60-160846cf-68756ec5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:00.057816Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:00.083179Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341299195296498:5210]
2025-10-09T21:45:00.083535Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 424, pipe: [1:7559341299195296498:5210]
2025-10-09T21:45:00.223493Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:00.224110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:00.224127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:00.224177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341299195296498:5210], now have 1 active actors on pipe
2025-10-09T21:45:00.225146Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|380dc606-ba143f60-160846cf-68756ec5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:00.225160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|380dc606-ba143f60-160846cf-68756ec5_0] Write session will restart in 0.005359s
2025-10-09T21:45:00.224690Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:00.225184Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|380dc606-ba143f60-160846cf-68756ec5_0] Write session: Do CDS request
2025-10-09T21:45:00.225190Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|380dc606-ba143f60-160846cf-68756ec5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:00.224711Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:00.224727Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:00.224788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 425 reason: error at write: Disconnected sessionId: src_id|380dc606-ba143f60-160846cf-68756ec5_0
2025-10-09T21:45:00.224974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 425 sessionId: src_id|380dc606-ba143f60-160846cf-68756ec5_0 is DEAD
2025-10-09T21:45:00.225110Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341299195296498:5210]
2025-10-09T21:45:00.225131Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:00.231537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|380dc606-ba143f60-160846cf-68756ec5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:00.231669Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:00.231685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 426
2025-10-09T21:45:00.231934Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 426 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:00.231980Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 426 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:00.232009Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=426 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:00.232014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:00.232518Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 426 sessionId:  describe result for acl check
2025-10-09T21:45:00.232565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:00.232572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:00.232574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:00.232581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341299195296523:5361] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:00.233200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341299195296523:5361] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:00.238220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341299195296523:5361] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:00.243364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341299195296523:5361] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:00.243384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341299195296523:5361] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:00.243388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341299195296523:5361] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:00.243399Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 426 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:00.252707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:00.256089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:00.256355Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:00.256371Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:00.256386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:00.256456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:00.256512Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341298833484870:4926]
2025-10-09T21:45:00.257152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:00.257778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:00.257856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:00.258298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:00.258378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:00.258386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:00.258387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:00.258388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:00.258390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:00.258391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:00.258393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.258394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.258396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.258402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:00.258414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:00.258418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:00.258422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:00.258440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:00.258448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:00.258452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.258456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.258459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.258469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:00.258482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:00.258489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:00.258495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:00.258499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:00.258509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:00.258514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:00.258524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:00.258529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:00.258539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:00.258544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:00.258550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:00.258554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.258558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.258563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.258574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.258581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.258586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.258614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:00.267912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:00.269217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:00.269242Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:00.269246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:00.269252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:00.269260Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 425 [2:7559341298833484870:4926]
2025-10-09T21:45:00.269279Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:00.269302Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:00.269311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.269315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.269318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.269340Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:00.269355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:00.269363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:00.269440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.269449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.269452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.269455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:00.269542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:00.269550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.269565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.269583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.269608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:00.269634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341298833484865:4926]
2025-10-09T21:45:00.269645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341298833484865:4926]
2025-10-09T21:45:00.269647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341298833484865:4926]
2025-10-09T21:45:00.269658Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:00.269668Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:00.269675Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:00.269677Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:00.269680Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:00.269696Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:00.269701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:00.269713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341299195296646:5361], now have 1 active actors on pipe
2025-10-09T21:45:00.269717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341299195296506:2432], now have 1 active actors on pipe
2025-10-09T21:45:00.269789Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 425
2025-10-09T21:45:00.269768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341298833484865:4926]
2025-10-09T21:45:00.269783Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:00.269792Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:00.269880Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 425
2025-10-09T21:45:00.270005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:00.270017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:00.270071Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6805e4ab-e580baec-636d7dcd-6d8a6115_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:00.270087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.270092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.270095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:00.270111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.270127Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:00.270150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:00.271028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:00.271042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:00.271066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:00.271198Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 426 partition: 0 MaxSeqNo: 25 sessionId: src_id|6805e4ab-e580baec-636d7dcd-6d8a6115_0
2025-10-09T21:45:00.274363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|380dc606-ba143f60-160846cf-68756ec5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6805e4ab-e580baec-636d7dcd-6d8a6115_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:00.277138Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6805e4ab-e580baec-636d7dcd-6d8a6115_0] Write session: try to update token
2025-10-09T21:45:00.277166Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6805e4ab-e580baec-636d7dcd-6d8a6115_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:00.286338Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:00.286355Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:00.286367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341298833484865:4926]
2025-10-09T21:45:00.286422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:00.286448Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:00.286622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:00.286785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:00.286944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:00.287124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:00.287539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:00.287595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:00.310499Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 426 sessionId: src_id|6805e4ab-e580baec-636d7dcd-6d8a6115_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:00.313082Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:00.311522Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:00.311545Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:00.311547Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:00.311549Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:00.311561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341298833484865:4926]
2025-10-09T21:45:00.311566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341298833484865:4926]
2025-10-09T21:45:00.311571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341298833484865:4926]
2025-10-09T21:45:00.311620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:00.311628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.311631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:00.311786Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:00.314300Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:00.314307Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:00.314312Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:00.325389Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341299195296763:5210]
2025-10-09T21:45:00.334383Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 425, pipe: [1:7559341299195296763:5210]
2025-10-09T21:45:00.509607Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:00.510720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:00.510737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:00.510763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341299195296763:5210], now have 1 active actors on pipe
2025-10-09T21:45:00.511366Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:00.516506Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:00.516601Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 426 reason: status is not ok: tablet will be restarted right now sessionId: src_id|6805e4ab-e580baec-636d7dcd-6d8a6115_0
2025-10-09T21:45:00.516786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 426 sessionId: src_id|6805e4ab-e580baec-636d7dcd-6d8a6115_0 is DEAD
2025-10-09T21:45:00.516972Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341299195296763:5210]
2025-10-09T21:45:00.516993Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:00.522121Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6805e4ab-e580baec-636d7dcd-6d8a6115_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:00.522137Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6805e4ab-e580baec-636d7dcd-6d8a6115_0] Write session will restart in 0.005487s
2025-10-09T21:45:00.522165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6805e4ab-e580baec-636d7dcd-6d8a6115_0] Write session: Do CDS request
2025-10-09T21:45:00.522173Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6805e4ab-e580baec-636d7dcd-6d8a6115_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:00.530489Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6805e4ab-e580baec-636d7dcd-6d8a6115_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:00.530859Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:00.530878Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 427
2025-10-09T21:45:00.531048Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 427 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:00.531076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 427 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:00.531082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=427 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:00.531085Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:00.531511Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 427 sessionId:  describe result for acl check
2025-10-09T21:45:00.531549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:00.531551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:00.531554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:00.531560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341299195296794:5366] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:00.532106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341299195296794:5366] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:00.546840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341299195296794:5366] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:00.562576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341299195296794:5366] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:00.562595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341299195296794:5366] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:00.562599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341299195296794:5366] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:00.562610Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 427 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:00.573472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:00.578484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:00.578675Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:00.578683Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:00.578692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:00.578739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:00.578790Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341298833485123:4929]
2025-10-09T21:45:00.579294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:00.590242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:00.590357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:00.590748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:00.590817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:00.590823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:00.590824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:00.590825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:00.590826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:00.590828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:00.590829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.590831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.590832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.590838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:00.590849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:00.590853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:00.590857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:00.590860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:00.590864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:00.590868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.590871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.590874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.590886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:00.590894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:00.590903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:00.590909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:00.590914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:00.590919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:00.590925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:00.590929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:00.590935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:00.590939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:00.590943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:00.590947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:00.590953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.590957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.590961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.590967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.590971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.590975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.591004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:00.598434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:00.599560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:00.599582Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:00.599585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:00.599590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:00.599598Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 426 [2:7559341298833485123:4929]
2025-10-09T21:45:00.599614Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:00.599634Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:00.599642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.599645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.599648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.599666Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:00.599674Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:00.599687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:00.600281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.600290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.600292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.600295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:00.600374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:00.600384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.600398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.600410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.600427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:00.600451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341298833485118:4929]
2025-10-09T21:45:00.600458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341298833485118:4929]
2025-10-09T21:45:00.600460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341298833485118:4929]
2025-10-09T21:45:00.600469Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:00.600485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341299195296991:5366], now have 1 active actors on pipe
2025-10-09T21:45:00.600487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:00.600490Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:00.600491Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:00.600493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341299195296772:2432], now have 1 active actors on pipe
2025-10-09T21:45:00.600503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:00.600518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:00.600521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:00.600574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341298833485118:4929]
2025-10-09T21:45:00.600587Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:00.600972Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:00.601121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:00.601132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:00.601175Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9114c05e-cbdc4fc2-5f2d4a46-4bd1ec23_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:00.601190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.601194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.601197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:00.601209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.601222Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:00.601247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:00.600833Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 426
2025-10-09T21:45:00.600870Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 426
2025-10-09T21:45:00.602115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:00.602128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:00.602149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:00.602413Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 427 partition: 0 MaxSeqNo: 25 sessionId: src_id|9114c05e-cbdc4fc2-5f2d4a46-4bd1ec23_0
2025-10-09T21:45:00.606224Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6805e4ab-e580baec-636d7dcd-6d8a6115_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9114c05e-cbdc4fc2-5f2d4a46-4bd1ec23_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:00.608832Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9114c05e-cbdc4fc2-5f2d4a46-4bd1ec23_0] Write session: try to update token
2025-10-09T21:45:00.608855Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9114c05e-cbdc4fc2-5f2d4a46-4bd1ec23_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:00.618583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341299195297025:5210], now have 1 active actors on pipe
2025-10-09T21:45:00.618262Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341299195297025:5210]
2025-10-09T21:45:00.618568Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 426, pipe: [1:7559341299195297025:5210]
2025-10-09T21:45:00.629006Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:00.629028Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:00.629031Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:00.629033Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:00.629043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341298833485118:4929]
2025-10-09T21:45:00.629049Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341298833485118:4929]
2025-10-09T21:45:00.629053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341298833485118:4929]
2025-10-09T21:45:00.629100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:00.629108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.629110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:00.629259Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:00.634623Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:00.634635Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:00.634639Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:00.643107Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 427 sessionId: src_id|9114c05e-cbdc4fc2-5f2d4a46-4bd1ec23_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:00.646351Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:00.795193Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:00.795831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:00.795842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:00.796138Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:00.796249Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341299195297025:5210]
2025-10-09T21:45:00.796277Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:00.796280Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:00.796337Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 427 reason: error at write: Disconnected sessionId: src_id|9114c05e-cbdc4fc2-5f2d4a46-4bd1ec23_0
2025-10-09T21:45:00.796529Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 427 sessionId: src_id|9114c05e-cbdc4fc2-5f2d4a46-4bd1ec23_0 is DEAD
2025-10-09T21:45:00.796647Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:00.796903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9114c05e-cbdc4fc2-5f2d4a46-4bd1ec23_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:00.796910Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9114c05e-cbdc4fc2-5f2d4a46-4bd1ec23_0] Write session will restart in 0.008019s
2025-10-09T21:45:00.796934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9114c05e-cbdc4fc2-5f2d4a46-4bd1ec23_0] Write session: Do CDS request
2025-10-09T21:45:00.796943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9114c05e-cbdc4fc2-5f2d4a46-4bd1ec23_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:00.806541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9114c05e-cbdc4fc2-5f2d4a46-4bd1ec23_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:00.810177Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:00.810197Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 428
2025-10-09T21:45:00.810672Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 428 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:00.810709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 428 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:00.810715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=428 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:00.810719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:00.811361Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 428 sessionId:  describe result for acl check
2025-10-09T21:45:00.811396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:00.811398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:00.811400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:00.811406Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341299195297045:5370] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:00.811831Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341299195297045:5370] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:00.815761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341299195297045:5370] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:00.823343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341299195297045:5370] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:00.823359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341299195297045:5370] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:00.823364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341299195297045:5370] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:00.823373Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 428 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:00.839785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:00.849201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:00.849468Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:00.849472Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:00.849488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:00.849551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:00.849589Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341298833485385:4931]
2025-10-09T21:45:00.850170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:00.851004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:00.851074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:00.851413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:00.851485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:00.851486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:00.851487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:00.851488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:00.851490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:00.851491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:00.851492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.851493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.851495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.851501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:00.851511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:00.851515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:00.851520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:00.851525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:00.851529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:00.851533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.851536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.851540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.851546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:00.851554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:00.851560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:00.851566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:00.851571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:00.851576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:00.851580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:00.851586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:00.851590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:00.851596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:00.851601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:00.851607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:00.851613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.851619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.851624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.851629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.851635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.851640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.851671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:00.865500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:00.866873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:00.866880Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:00.866882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:00.866887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:00.866894Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 427 [2:7559341298833485385:4931]
2025-10-09T21:45:00.866909Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:00.866930Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:00.866933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.866936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.866938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.866956Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:00.866962Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:00.866970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:00.867035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.867037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.867038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.867041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:00.867117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:00.867119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.867133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:00.867143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:00.867158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:00.867180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341299195297162:5370], now have 1 active actors on pipe
2025-10-09T21:45:00.867183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341299195297030:2432], now have 1 active actors on pipe
2025-10-09T21:45:00.867192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341298833485379:4931]
2025-10-09T21:45:00.867193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341298833485379:4931]
2025-10-09T21:45:00.867195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341298833485379:4931]
2025-10-09T21:45:00.867210Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:00.867222Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:00.867224Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:00.867224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:00.867226Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:00.867237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:00.867240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:00.867277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341298833485379:4931]
2025-10-09T21:45:00.867284Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:00.867288Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:00.870696Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 427
2025-10-09T21:45:00.870728Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 427
2025-10-09T21:45:00.870905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:00.870922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:00.870954Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|85dceaed-9872c99a-99f0a8f6-113c6f6d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:00.870966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.870972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.870975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:00.870988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.871005Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:00.871024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:00.874322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:00.874336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:00.874379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:00.874590Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 428 partition: 0 MaxSeqNo: 25 sessionId: src_id|85dceaed-9872c99a-99f0a8f6-113c6f6d_0
2025-10-09T21:45:00.876325Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9114c05e-cbdc4fc2-5f2d4a46-4bd1ec23_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|85dceaed-9872c99a-99f0a8f6-113c6f6d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:00.878882Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85dceaed-9872c99a-99f0a8f6-113c6f6d_0] Write session: try to update token
2025-10-09T21:45:00.878899Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85dceaed-9872c99a-99f0a8f6-113c6f6d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:00.898123Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341299195297300:5210]
2025-10-09T21:45:00.898605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341299195297300:5210], now have 1 active actors on pipe
2025-10-09T21:45:00.898671Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 427, pipe: [1:7559341299195297300:5210]
2025-10-09T21:45:00.907544Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:00.907563Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:00.907565Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:00.907567Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:00.907577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341298833485379:4931]
2025-10-09T21:45:00.907581Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341298833485379:4931]
2025-10-09T21:45:00.907584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341298833485379:4931]
2025-10-09T21:45:00.907628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:00.907635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:00.907638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:00.907787Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:00.910114Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:00.910131Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:00.910135Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:00.913341Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 428 sessionId: src_id|85dceaed-9872c99a-99f0a8f6-113c6f6d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:00.915916Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:01.082781Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:01.083410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:01.083423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:01.083969Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:01.084525Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341299195297300:5210]
2025-10-09T21:45:01.084557Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:01.084595Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:01.084668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 428 reason: status is not ok: tablet will be restarted right now sessionId: src_id|85dceaed-9872c99a-99f0a8f6-113c6f6d_0
2025-10-09T21:45:01.084865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 428 sessionId: src_id|85dceaed-9872c99a-99f0a8f6-113c6f6d_0 is DEAD
2025-10-09T21:45:01.086517Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85dceaed-9872c99a-99f0a8f6-113c6f6d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:01.086527Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85dceaed-9872c99a-99f0a8f6-113c6f6d_0] Write session will restart in 0.007098s
2025-10-09T21:45:01.086551Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85dceaed-9872c99a-99f0a8f6-113c6f6d_0] Write session: Do CDS request
2025-10-09T21:45:01.086558Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85dceaed-9872c99a-99f0a8f6-113c6f6d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:01.098522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85dceaed-9872c99a-99f0a8f6-113c6f6d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:01.098845Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:01.098861Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 429
2025-10-09T21:45:01.099024Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 429 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:01.099053Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 429 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:01.099062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=429 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:01.099065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:01.099547Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 429 sessionId:  describe result for acl check
2025-10-09T21:45:01.099582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:01.099583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:01.099585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:01.099591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341303490264653:5382] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:01.100117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341303490264653:5382] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:01.110295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341303490264653:5382] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:01.120767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341303490264653:5382] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:01.120788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341303490264653:5382] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:01.120792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341303490264653:5382] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:01.120802Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 429 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:01.124975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:01.128036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:01.128278Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:01.128283Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:01.128296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:01.128355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:01.128412Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341303128452959:4945]
2025-10-09T21:45:01.129041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:01.129396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:01.129436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:01.129733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:01.129768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:01.129770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:01.129771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:01.129772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:01.129774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:01.129775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:01.129776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.129777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.129778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.129782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:01.129792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:01.129795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:01.129799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:01.129802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:01.129805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:01.129808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.129812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.129815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.129823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:01.129843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:01.129848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:01.129853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:01.129858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:01.129862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:01.129866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:01.129870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:01.129874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:01.129879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:01.129882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:01.129886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:01.129891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.129895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.129898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.129903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.129906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.129911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.129936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:01.138050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:01.140126Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 428
2025-10-09T21:45:01.140176Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 428
2025-10-09T21:45:01.141540Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 429 partition: 0 MaxSeqNo: 25 sessionId: src_id|404195c8-587e5bc1-12c4c00a-e93c3620_0
2025-10-09T21:45:01.141987Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85dceaed-9872c99a-99f0a8f6-113c6f6d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|404195c8-587e5bc1-12c4c00a-e93c3620_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:01.139381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:01.139398Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:01.139401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:01.139407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:01.139415Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 428 [2:7559341303128452959:4945]
2025-10-09T21:45:01.139431Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:01.139452Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:01.139455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:01.139458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:01.139461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:01.139481Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:01.139487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:01.139494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:01.139556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.139558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.139560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.139563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:01.139644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:01.139647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.139659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.139671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.139687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:01.139710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341303490264620:2432], now have 1 active actors on pipe
2025-10-09T21:45:01.139713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341303490264846:5382], now have 1 active actors on pipe
2025-10-09T21:45:01.139720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341303128452954:4945]
2025-10-09T21:45:01.139722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341303128452954:4945]
2025-10-09T21:45:01.139725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341303128452954:4945]
2025-10-09T21:45:01.139733Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:01.139744Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:01.139751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:01.139751Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:01.139753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:01.139766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:01.139769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:01.139807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341303128452954:4945]
2025-10-09T21:45:01.139818Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:01.140274Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:01.140721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:01.140728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:01.140760Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|404195c8-587e5bc1-12c4c00a-e93c3620_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:01.140783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:01.140788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:01.140791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:01.140804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:01.140816Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:01.140846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:01.141378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:01.141383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:01.141402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:01.144814Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|404195c8-587e5bc1-12c4c00a-e93c3620_0] Write session: try to update token
2025-10-09T21:45:01.144831Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|404195c8-587e5bc1-12c4c00a-e93c3620_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:01.159578Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:01.159598Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:01.159601Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:01.159603Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:01.159612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341303128452954:4945]
2025-10-09T21:45:01.159617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341303128452954:4945]
2025-10-09T21:45:01.159628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341303128452954:4945]
2025-10-09T21:45:01.159670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:01.159678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.159681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:01.159819Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:01.162684Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:01.162698Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:01.162702Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:01.179357Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 429 sessionId: src_id|404195c8-587e5bc1-12c4c00a-e93c3620_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:01.181884Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:01.186251Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341303490264877:5210]
2025-10-09T21:45:01.186515Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 428, pipe: [1:7559341303490264877:5210]
2025-10-09T21:45:01.394704Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:01.394994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:01.395006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:01.395030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341303490264877:5210], now have 1 active actors on pipe
2025-10-09T21:45:01.396450Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|404195c8-587e5bc1-12c4c00a-e93c3620_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:01.396462Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|404195c8-587e5bc1-12c4c00a-e93c3620_0] Write session will restart in 0.009661s
2025-10-09T21:45:01.396489Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|404195c8-587e5bc1-12c4c00a-e93c3620_0] Write session: Do CDS request
2025-10-09T21:45:01.396495Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|404195c8-587e5bc1-12c4c00a-e93c3620_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:01.395742Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:01.395791Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341303490264877:5210]
2025-10-09T21:45:01.395814Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:01.395817Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:01.395872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 429 reason: error at write: Disconnected sessionId: src_id|404195c8-587e5bc1-12c4c00a-e93c3620_0
2025-10-09T21:45:01.396070Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 429 sessionId: src_id|404195c8-587e5bc1-12c4c00a-e93c3620_0 is DEAD
2025-10-09T21:45:01.403087Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:01.407576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|404195c8-587e5bc1-12c4c00a-e93c3620_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:01.408162Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:01.408178Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 430
2025-10-09T21:45:01.408325Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 430 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:01.408357Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 430 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:01.408363Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=430 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:01.408367Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:01.408957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 430 sessionId:  describe result for acl check
2025-10-09T21:45:01.408987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:01.408988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:01.408990Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:01.408996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341303490264893:5386] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:01.409492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341303490264893:5386] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:01.413148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341303490264893:5386] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:01.425194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341303490264893:5386] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:01.425212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341303490264893:5386] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:01.425216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341303490264893:5386] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:01.425226Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 430 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:01.448028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:01.452546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:01.452791Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:01.452796Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:01.452809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:01.452868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:01.452923Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341303128453214:4949]
2025-10-09T21:45:01.453516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:01.454318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:01.454378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:01.454716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:01.454777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:01.454778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:01.454780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:01.454781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:01.454782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:01.454784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:01.454785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.454787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.454788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.454794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:01.454806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:01.454810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:01.454814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:01.454819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:01.454824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:01.454828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.454832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.454836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.454844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:01.454852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:01.454858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:01.454864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:01.454869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:01.454874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:01.454878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:01.454883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:01.454888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:01.454893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:01.454898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:01.454902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:01.454908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.454913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.454918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.454924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.454929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.454935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.454989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:01.464776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:01.466013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:01.466020Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:01.466023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:01.466047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:01.466056Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 429 [2:7559341303128453214:4949]
2025-10-09T21:45:01.466073Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:01.466095Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:01.466099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:01.466103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:01.466106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:01.466125Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:01.466131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:01.466149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:01.467531Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 429
2025-10-09T21:45:01.467576Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 429
2025-10-09T21:45:01.472563Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 430 partition: 0 MaxSeqNo: 25 sessionId: src_id|bfe7e37a-16ec0e16-aedc727-5141f25f_0
2025-10-09T21:45:01.472959Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|404195c8-587e5bc1-12c4c00a-e93c3620_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bfe7e37a-16ec0e16-aedc727-5141f25f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:01.466216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.466224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.466226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.466229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:01.466320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:01.466324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.466338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.466352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.466370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:01.466391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341303128453209:4949]
2025-10-09T21:45:01.466395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341303128453209:4949]
2025-10-09T21:45:01.466397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341303128453209:4949]
2025-10-09T21:45:01.466414Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:01.466432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:01.466447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:01.466451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:01.466496Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341303128453209:4949]
2025-10-09T21:45:01.466499Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:01.466948Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:01.466951Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:01.466954Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:01.466961Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:01.468612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341303490264881:2432], now have 1 active actors on pipe
2025-10-09T21:45:01.468619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341303490265000:5386], now have 1 active actors on pipe
2025-10-09T21:45:01.468641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:01.468647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:01.468681Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bfe7e37a-16ec0e16-aedc727-5141f25f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:01.468693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:01.468699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:01.468702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:01.468715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:01.468731Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:01.468749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:01.469030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:01.469037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:01.469073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:01.475527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bfe7e37a-16ec0e16-aedc727-5141f25f_0] Write session: try to update token
2025-10-09T21:45:01.475545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bfe7e37a-16ec0e16-aedc727-5141f25f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:01.490885Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:01.490906Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:01.490908Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:01.490911Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:01.490922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341303128453209:4949]
2025-10-09T21:45:01.490927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341303128453209:4949]
2025-10-09T21:45:01.490931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341303128453209:4949]
2025-10-09T21:45:01.490980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:01.490988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.490991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:01.491133Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:01.494650Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:01.494671Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:01.494676Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:01.495740Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341303490265139:5210]
2025-10-09T21:45:01.496138Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 429, pipe: [1:7559341303490265139:5210]
2025-10-09T21:45:01.506236Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 430 sessionId: src_id|bfe7e37a-16ec0e16-aedc727-5141f25f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:01.508790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:01.690155Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:01.690917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341303490265139:5210], now have 1 active actors on pipe
2025-10-09T21:45:01.690954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:01.690962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:01.692244Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:01.692307Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfe7e37a-16ec0e16-aedc727-5141f25f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:01.692338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfe7e37a-16ec0e16-aedc727-5141f25f_0] Write session will restart in 0.006113s
2025-10-09T21:45:01.692364Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfe7e37a-16ec0e16-aedc727-5141f25f_0] Write session: Do CDS request
2025-10-09T21:45:01.692370Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfe7e37a-16ec0e16-aedc727-5141f25f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:01.691509Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:01.691571Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:01.691574Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:01.691644Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 430 reason: error at write: Disconnected sessionId: src_id|bfe7e37a-16ec0e16-aedc727-5141f25f_0
2025-10-09T21:45:01.691832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 430 sessionId: src_id|bfe7e37a-16ec0e16-aedc727-5141f25f_0 is DEAD
2025-10-09T21:45:01.691958Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341303490265139:5210]
2025-10-09T21:45:01.699700Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bfe7e37a-16ec0e16-aedc727-5141f25f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:01.700039Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:01.700060Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 431
2025-10-09T21:45:01.700203Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 431 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:01.700246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 431 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:01.700253Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=431 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:01.700258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:01.700796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 431 sessionId:  describe result for acl check
2025-10-09T21:45:01.700838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:01.700844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:01.700851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:01.700860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341303490265161:5393] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:01.701428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341303490265161:5393] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:01.708290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341303490265161:5393] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:01.715250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341303490265161:5393] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:01.715269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341303490265161:5393] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:01.715274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341303490265161:5393] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:01.715308Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 431 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:01.736102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:01.740988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:01.741285Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:01.741291Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:01.741306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:01.741382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:01.741427Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341303128453466:4951]
2025-10-09T21:45:01.742047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:01.742647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:01.742719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:01.743071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:01.743141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:01.743150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:01.743152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:01.743154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:01.743155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:01.743157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:01.743158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.743160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.743161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.743168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:01.743179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:01.743183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:01.743187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:01.743192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:01.743197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:01.743201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.743206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.743211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.743219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:01.743229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:01.743237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:01.743243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:01.743251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:01.743257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:01.743263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:01.743269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:01.743274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:01.743280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:01.743286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:01.743292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:01.743298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.743304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.743309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.743315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.743320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.743327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.743363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:01.752894Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 430
2025-10-09T21:45:01.752927Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 430
2025-10-09T21:45:01.750955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:01.752074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:01.752083Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:01.752086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:01.752092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:01.752101Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 430 [2:7559341303128453466:4951]
2025-10-09T21:45:01.752117Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:01.752138Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:01.752141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:01.752145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:01.752147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:01.752168Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:01.752174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:01.752182Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:01.752250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.752253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.752256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.752259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:01.752346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:01.752349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.752364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:01.752374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:01.752391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:01.752414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341303128453461:4951]
2025-10-09T21:45:01.752418Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341303128453461:4951]
2025-10-09T21:45:01.752421Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341303128453461:4951]
2025-10-09T21:45:01.752443Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:01.752460Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:01.752476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:01.752480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:01.752497Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:01.752501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:01.752504Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:01.752526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341303128453461:4951]
2025-10-09T21:45:01.752529Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:01.752596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:01.752673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341303490265212:5393], now have 1 active actors on pipe
2025-10-09T21:45:01.752680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341303490265148:2432], now have 1 active actors on pipe
2025-10-09T21:45:01.753047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:01.753056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:01.753109Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|438c4e98-33d7db8e-c926227d-7f3b30b8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:01.753120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:01.753125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:01.753128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:01.753155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:01.753171Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:01.753203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:01.753389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:01.753402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:01.753434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:01.753598Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 431 partition: 0 MaxSeqNo: 25 sessionId: src_id|438c4e98-33d7db8e-c926227d-7f3b30b8_0
2025-10-09T21:45:01.754008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfe7e37a-16ec0e16-aedc727-5141f25f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|438c4e98-33d7db8e-c926227d-7f3b30b8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:01.756504Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|438c4e98-33d7db8e-c926227d-7f3b30b8_0] Write session: try to update token
2025-10-09T21:45:01.756520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|438c4e98-33d7db8e-c926227d-7f3b30b8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:01.771079Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:01.771101Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:01.771104Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:01.771106Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:01.771116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341303128453461:4951]
2025-10-09T21:45:01.771121Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341303128453461:4951]
2025-10-09T21:45:01.771125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341303128453461:4951]
2025-10-09T21:45:01.771176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:01.771183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:01.771186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:01.771323Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:01.771499Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:01.771515Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:01.771519Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:01.773098Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 431 sessionId: src_id|438c4e98-33d7db8e-c926227d-7f3b30b8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:01.775582Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:01.792269Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341303490265408:5210]
2025-10-09T21:45:01.792572Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 430, pipe: [1:7559341303490265408:5210]
2025-10-09T21:45:01.944009Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:01.944618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:01.944629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:01.944676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341303490265408:5210], now have 1 active actors on pipe
2025-10-09T21:45:01.945020Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:01.946008Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341303490265408:5210]
2025-10-09T21:45:01.946063Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:01.946132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 431 reason: status is not ok: tablet will be restarted right now sessionId: src_id|438c4e98-33d7db8e-c926227d-7f3b30b8_0
2025-10-09T21:45:01.946335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 431 sessionId: src_id|438c4e98-33d7db8e-c926227d-7f3b30b8_0 is DEAD
2025-10-09T21:45:01.946521Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:01.946587Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|438c4e98-33d7db8e-c926227d-7f3b30b8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:01.946599Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|438c4e98-33d7db8e-c926227d-7f3b30b8_0] Write session will restart in 0.006730s
2025-10-09T21:45:01.946626Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|438c4e98-33d7db8e-c926227d-7f3b30b8_0] Write session: Do CDS request
2025-10-09T21:45:01.946633Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|438c4e98-33d7db8e-c926227d-7f3b30b8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:01.957942Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|438c4e98-33d7db8e-c926227d-7f3b30b8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:01.958349Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:01.958367Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 432
2025-10-09T21:45:01.958633Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 432 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:01.958668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 432 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:01.958674Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=432 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:01.958677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:01.959708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 432 sessionId:  describe result for acl check
2025-10-09T21:45:01.959753Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:01.959755Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:01.959757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:01.959763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341303490265439:5403] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:01.960324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341303490265439:5403] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:01.964424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341303490265439:5403] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:01.992950Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341303490265439:5403] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:01.992968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341303490265439:5403] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:01.992972Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341303490265439:5403] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:01.992982Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 432 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:02.016408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:02.017519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:02.018005Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:02.018010Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:02.018024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:02.018111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:02.018152Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341307423421053:4963]
2025-10-09T21:45:02.018799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:02.019464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:02.019515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:02.019851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:02.019894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:02.019895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:02.019896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:02.019898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:02.019899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:02.019901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:02.019902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.019903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.019905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.019922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:02.019934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:02.019938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:02.019943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:02.019947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:02.019951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:02.019955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.019959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.019964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.019981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:02.019990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:02.019995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:02.020002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:02.020008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:02.020014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:02.020018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:02.020024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:02.020028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:02.020034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:02.020038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:02.020044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:02.020049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.020054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.020061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.020066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.020071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.020078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.020109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:02.029133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:02.030884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:02.030901Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:02.030903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:02.030908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:02.030917Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 431 [2:7559341307423421053:4963]
2025-10-09T21:45:02.030932Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:02.030953Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:02.030957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.030961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.030963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.030984Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:02.030998Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:02.031006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:02.031078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.031080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.031083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.031086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:02.031170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:02.031173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.031190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.031202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.031219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:02.031241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341307423421048:4963]
2025-10-09T21:45:02.031244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341307423421048:4963]
2025-10-09T21:45:02.031247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341307423421048:4963]
2025-10-09T21:45:02.031264Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:02.031281Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:02.031296Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:02.031299Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:02.031344Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341307423421048:4963]
2025-10-09T21:45:02.031346Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:02.031714Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:02.031722Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:02.031723Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:02.031729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:02.032687Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 431
2025-10-09T21:45:02.032721Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 431
2025-10-09T21:45:02.033026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341303490265566:5403], now have 1 active actors on pipe
2025-10-09T21:45:02.033036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341303490265430:2432], now have 1 active actors on pipe
2025-10-09T21:45:02.033051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:02.033057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:02.033095Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|90ef3ead-78fe28c3-f7e3bc45-ea912503_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:02.033107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.033111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.033115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:02.033128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.033143Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:02.033163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:02.033896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:02.033906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:02.033923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:02.034093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 432 partition: 0 MaxSeqNo: 25 sessionId: src_id|90ef3ead-78fe28c3-f7e3bc45-ea912503_0
2025-10-09T21:45:02.034594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|438c4e98-33d7db8e-c926227d-7f3b30b8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|90ef3ead-78fe28c3-f7e3bc45-ea912503_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:02.037502Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90ef3ead-78fe28c3-f7e3bc45-ea912503_0] Write session: try to update token
2025-10-09T21:45:02.037529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90ef3ead-78fe28c3-f7e3bc45-ea912503_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:02.043651Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341307785232995:5210]
2025-10-09T21:45:02.044514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341307785232995:5210], now have 1 active actors on pipe
2025-10-09T21:45:02.044563Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 431, pipe: [1:7559341307785232995:5210]
2025-10-09T21:45:02.051834Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:02.051854Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:02.051856Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:02.051858Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:02.051868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341307423421048:4963]
2025-10-09T21:45:02.051878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341307423421048:4963]
2025-10-09T21:45:02.051881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341307423421048:4963]
2025-10-09T21:45:02.051933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:02.051941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.051944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:02.052040Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:02.052047Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:02.052051Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:02.052093Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:02.064857Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 432 sessionId: src_id|90ef3ead-78fe28c3-f7e3bc45-ea912503_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:02.067772Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:02.226382Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:02.226887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:02.226904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:02.227230Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:02.227642Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:02.227742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 432 reason: status is not ok: tablet will be restarted right now sessionId: src_id|90ef3ead-78fe28c3-f7e3bc45-ea912503_0
2025-10-09T21:45:02.227956Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 432 sessionId: src_id|90ef3ead-78fe28c3-f7e3bc45-ea912503_0 is DEAD
2025-10-09T21:45:02.227968Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341307785232995:5210]
2025-10-09T21:45:02.227988Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:02.228133Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90ef3ead-78fe28c3-f7e3bc45-ea912503_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:02.228145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90ef3ead-78fe28c3-f7e3bc45-ea912503_0] Write session will restart in 0.006125s
2025-10-09T21:45:02.228170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90ef3ead-78fe28c3-f7e3bc45-ea912503_0] Write session: Do CDS request
2025-10-09T21:45:02.228177Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90ef3ead-78fe28c3-f7e3bc45-ea912503_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:02.236621Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90ef3ead-78fe28c3-f7e3bc45-ea912503_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:02.236796Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:02.236814Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 433
2025-10-09T21:45:02.237079Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 433 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:02.237131Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 433 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:02.237137Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=433 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:02.237140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:02.237687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 433 sessionId:  describe result for acl check
2025-10-09T21:45:02.237726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:02.237728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:02.237730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:02.237737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341307785233014:5409] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:02.238448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341307785233014:5409] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:02.241818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341307785233014:5409] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:02.250300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341307785233014:5409] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:02.250322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341307785233014:5409] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:02.250326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341307785233014:5409] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:02.250337Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 433 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:02.281535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:02.285595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:02.285863Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:02.285878Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:02.285892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:02.285975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:02.286024Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341307423421312:4968]
2025-10-09T21:45:02.286672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:02.287402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:02.287481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:02.287845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:02.288006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:02.288008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:02.288010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:02.288012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:02.288013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:02.288014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:02.288016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.288017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.288018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.288026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:02.288037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:02.288042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:02.288046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:02.288050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:02.288054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:02.288057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.288061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.288065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.288074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:02.288083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:02.288099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:02.288104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:02.288110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:02.288115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:02.288120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:02.288125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:02.288130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:02.288136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:02.288141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:02.288146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:02.288152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.288157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.288161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.288166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.288171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.288176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.288206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:02.302008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:02.303553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:02.303574Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:02.303576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:02.303582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:02.303592Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 432 [2:7559341307423421312:4968]
2025-10-09T21:45:02.303608Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:02.303632Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:02.303640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.303644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.303647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.303668Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:02.303679Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:02.303692Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:02.303763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.303770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.303772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.303775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:02.303853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:02.303861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.303907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.303924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.303948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:02.303977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341307785233000:2432], now have 1 active actors on pipe
2025-10-09T21:45:02.303986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341307785233120:5409], now have 1 active actors on pipe
2025-10-09T21:45:02.303995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341307423421305:4968]
2025-10-09T21:45:02.303998Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341307423421305:4968]
2025-10-09T21:45:02.304000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341307423421305:4968]
2025-10-09T21:45:02.304037Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:02.304054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:02.304058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:02.304063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:02.304065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:02.304047Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 432
2025-10-09T21:45:02.304076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:02.304069Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 432
2025-10-09T21:45:02.304080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:02.304160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:02.304171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:02.304186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341307423421305:4968]
2025-10-09T21:45:02.304196Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:02.304233Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:02.304235Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b9f8ee7c-ea10ef1e-cd98b00f-a83fa83c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:02.304242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.304246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.304249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:02.304262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.304274Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:02.304312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:02.304661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:02.304670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:02.304699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:02.304895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 433 partition: 0 MaxSeqNo: 25 sessionId: src_id|b9f8ee7c-ea10ef1e-cd98b00f-a83fa83c_0
2025-10-09T21:45:02.305299Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90ef3ead-78fe28c3-f7e3bc45-ea912503_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b9f8ee7c-ea10ef1e-cd98b00f-a83fa83c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:02.308110Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b9f8ee7c-ea10ef1e-cd98b00f-a83fa83c_0] Write session: try to update token
2025-10-09T21:45:02.308134Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b9f8ee7c-ea10ef1e-cd98b00f-a83fa83c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:02.324809Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:02.324828Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:02.324839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341307423421305:4968]
2025-10-09T21:45:02.324897Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:02.324900Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:02.325148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:02.325310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:02.325467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:02.325620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:02.325843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:02.325896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:02.328335Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341307785233261:5210]
2025-10-09T21:45:02.328736Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 432, pipe: [1:7559341307785233261:5210]
2025-10-09T21:45:02.328677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341307785233261:5210], now have 1 active actors on pipe
2025-10-09T21:45:02.337686Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 433 sessionId: src_id|b9f8ee7c-ea10ef1e-cd98b00f-a83fa83c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:02.340555Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:02.340622Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:02.340642Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:02.340645Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:02.340647Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:02.340658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341307423421305:4968]
2025-10-09T21:45:02.340661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341307423421305:4968]
2025-10-09T21:45:02.340666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341307423421305:4968]
2025-10-09T21:45:02.340711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:02.340719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.340721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:02.340790Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:02.340806Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:02.340810Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:02.340858Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:02.504816Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:02.505315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:02.505332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:02.505613Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:02.505615Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341307785233261:5210]
2025-10-09T21:45:02.505632Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:02.505637Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:02.505665Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:02.505702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 433 reason: error at write: Disconnected sessionId: src_id|b9f8ee7c-ea10ef1e-cd98b00f-a83fa83c_0
2025-10-09T21:45:02.505882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 433 sessionId: src_id|b9f8ee7c-ea10ef1e-cd98b00f-a83fa83c_0 is DEAD
2025-10-09T21:45:02.506083Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9f8ee7c-ea10ef1e-cd98b00f-a83fa83c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:02.506110Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9f8ee7c-ea10ef1e-cd98b00f-a83fa83c_0] Write session will restart in 0.006245s
2025-10-09T21:45:02.506137Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9f8ee7c-ea10ef1e-cd98b00f-a83fa83c_0] Write session: Do CDS request
2025-10-09T21:45:02.506157Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9f8ee7c-ea10ef1e-cd98b00f-a83fa83c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:02.513612Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b9f8ee7c-ea10ef1e-cd98b00f-a83fa83c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:02.513723Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:02.513742Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 434
2025-10-09T21:45:02.513865Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 434 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:02.513897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 434 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:02.513908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=434 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:02.513912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:02.514536Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 434 sessionId:  describe result for acl check
2025-10-09T21:45:02.514577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:02.514583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:02.514585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:02.514592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341307785233278:5413] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:02.515207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341307785233278:5413] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:02.518979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341307785233278:5413] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:02.524794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341307785233278:5413] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:02.524811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341307785233278:5413] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:02.524815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341307785233278:5413] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:02.524825Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 434 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:02.533037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:02.536331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:02.536568Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:02.536580Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:02.536594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:02.536656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:02.536699Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341307423421567:4971]
2025-10-09T21:45:02.537287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:02.537716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:02.537786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:02.538178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:02.538247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:02.538254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:02.538256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:02.538267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:02.538268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:02.538269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:02.538271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.538272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.538273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.538280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:02.538290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:02.538294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:02.538298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:02.538302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:02.538306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:02.538310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.538313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.538316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.538324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:02.538334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:02.538339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:02.538345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:02.538349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:02.538354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:02.538358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:02.538363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:02.538368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:02.538382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:02.538387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:02.538393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:02.538398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.538404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.538408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.538413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.538417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.538423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.538454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:02.545959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:02.547141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:02.547160Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:02.547163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:02.547168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:02.547176Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 433 [2:7559341307423421567:4971]
2025-10-09T21:45:02.547194Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:02.547594Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 433
2025-10-09T21:45:02.547643Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 433
2025-10-09T21:45:02.547215Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:02.547219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.547223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.547225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.547244Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:02.547251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:02.547259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:02.547332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.547334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.547336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.547339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:02.547416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:02.547419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.547433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.547444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.547459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:02.547480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341307423421562:4971]
2025-10-09T21:45:02.547483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341307423421562:4971]
2025-10-09T21:45:02.547485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341307423421562:4971]
2025-10-09T21:45:02.547494Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:02.547510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:02.547516Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:02.547520Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:02.547523Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:02.547525Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:02.547528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:02.547540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341307785233264:2432], now have 1 active actors on pipe
2025-10-09T21:45:02.547544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341307785233410:5413], now have 1 active actors on pipe
2025-10-09T21:45:02.547585Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341307423421562:4971]
2025-10-09T21:45:02.547589Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:02.547619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:02.547995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:02.548009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:02.548042Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c85cfebb-cab8920c-920fd267-1d704c1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:02.548052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.548056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.548058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:02.548070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.548083Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:02.548103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:02.548420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:02.548431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:02.548447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:02.548706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 434 partition: 0 MaxSeqNo: 25 sessionId: src_id|c85cfebb-cab8920c-920fd267-1d704c1_0
2025-10-09T21:45:02.549149Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9f8ee7c-ea10ef1e-cd98b00f-a83fa83c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c85cfebb-cab8920c-920fd267-1d704c1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:02.552269Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c85cfebb-cab8920c-920fd267-1d704c1_0] Write session: try to update token
2025-10-09T21:45:02.552294Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c85cfebb-cab8920c-920fd267-1d704c1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:02.565409Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:02.565433Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:02.565436Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:02.565438Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:02.565448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341307423421562:4971]
2025-10-09T21:45:02.565452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341307423421562:4971]
2025-10-09T21:45:02.565457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341307423421562:4971]
2025-10-09T21:45:02.565503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:02.565511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.565514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:02.565517Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:02.565523Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:02.565527Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:02.565651Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:02.569167Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 434 sessionId: src_id|c85cfebb-cab8920c-920fd267-1d704c1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:02.571882Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:02.605794Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341307785233529:5210]
2025-10-09T21:45:02.606168Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 433, pipe: [1:7559341307785233529:5210]
2025-10-09T21:45:02.735588Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:02.736123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:02.736140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:02.736200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341307785233529:5210], now have 1 active actors on pipe
2025-10-09T21:45:02.736520Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341307785233529:5210]
2025-10-09T21:45:02.736559Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:02.736599Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:02.737053Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c85cfebb-cab8920c-920fd267-1d704c1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:02.737065Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c85cfebb-cab8920c-920fd267-1d704c1_0] Write session will restart in 0.006926s
2025-10-09T21:45:02.737100Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c85cfebb-cab8920c-920fd267-1d704c1_0] Write session: Do CDS request
2025-10-09T21:45:02.736691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 434 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c85cfebb-cab8920c-920fd267-1d704c1_0
2025-10-09T21:45:02.737107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c85cfebb-cab8920c-920fd267-1d704c1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:02.736887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 434 sessionId: src_id|c85cfebb-cab8920c-920fd267-1d704c1_0 is DEAD
2025-10-09T21:45:02.736455Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:02.745585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c85cfebb-cab8920c-920fd267-1d704c1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:02.745671Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:02.745688Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 435
2025-10-09T21:45:02.746223Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 435 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:02.746288Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 435 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:02.746299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=435 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:02.746303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:02.746835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 435 sessionId:  describe result for acl check
2025-10-09T21:45:02.746888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:02.746890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:02.746894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:02.746901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341307785233546:5418] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:02.747493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341307785233546:5418] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:02.751546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341307785233546:5418] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:02.756372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341307785233546:5418] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:02.756399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341307785233546:5418] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:02.756405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341307785233546:5418] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:02.756416Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 435 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:02.765345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:02.768687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:02.768924Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:02.768932Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:02.768944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:02.769018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:02.769065Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341307423421825:4973]
2025-10-09T21:45:02.769674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:02.770076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:02.770142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:02.770509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:02.770578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:02.770585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:02.770587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:02.770588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:02.770589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:02.770590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:02.770592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.770593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.770594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.770600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:02.770611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:02.770615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:02.770619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:02.770623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:02.770627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:02.770631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.770634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.770638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.770646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:02.770654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:02.770661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:02.770667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:02.770672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:02.770677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:02.770683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:02.770688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:02.770692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:02.770697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:02.770701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:02.770705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:02.770710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.770714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.770719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.770723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.770728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.770732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.770763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:02.778504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:02.780160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:02.780166Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:02.780168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:02.780172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:02.780180Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 434 [2:7559341307423421825:4973]
2025-10-09T21:45:02.780200Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:02.780220Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:02.780223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.780226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.780229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.780247Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:02.780253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:02.780261Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:02.780334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.780336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.780338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.780342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:02.780430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:02.780433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.780448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:02.780459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:02.780475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:02.780497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341307423421819:4973]
2025-10-09T21:45:02.780500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341307423421819:4973]
2025-10-09T21:45:02.780502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341307423421819:4973]
2025-10-09T21:45:02.780517Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:02.780533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:02.780548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:02.780551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:02.780596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341307423421819:4973]
2025-10-09T21:45:02.780599Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:02.780934Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:02.780943Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:02.780948Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:02.780955Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:02.781331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341307785233532:2432], now have 1 active actors on pipe
2025-10-09T21:45:02.781340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341307785233665:5418], now have 1 active actors on pipe
2025-10-09T21:45:02.781304Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 434
2025-10-09T21:45:02.781315Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 434
2025-10-09T21:45:02.781723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:02.781738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:02.781771Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8b6b7269-2d77212a-996b6d24-d0983c36_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:02.781782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.781787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.781790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:02.781802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.781814Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:02.781831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:02.782093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:02.782114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:02.782139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:02.782299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 435 partition: 0 MaxSeqNo: 25 sessionId: src_id|8b6b7269-2d77212a-996b6d24-d0983c36_0
2025-10-09T21:45:02.786160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c85cfebb-cab8920c-920fd267-1d704c1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8b6b7269-2d77212a-996b6d24-d0983c36_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:02.788842Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b6b7269-2d77212a-996b6d24-d0983c36_0] Write session: try to update token
2025-10-09T21:45:02.788864Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b6b7269-2d77212a-996b6d24-d0983c36_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:02.797325Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:02.797342Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:02.797345Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:02.797347Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:02.797357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341307423421819:4973]
2025-10-09T21:45:02.797361Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341307423421819:4973]
2025-10-09T21:45:02.797366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341307423421819:4973]
2025-10-09T21:45:02.797412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:02.797420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:02.797422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:02.797427Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:02.797434Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:02.797438Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:02.797589Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:02.814044Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 435 sessionId: src_id|8b6b7269-2d77212a-996b6d24-d0983c36_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:02.816767Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:02.836711Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341307785233794:5210]
2025-10-09T21:45:02.837048Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 434, pipe: [1:7559341307785233794:5210]
2025-10-09T21:45:02.989654Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:02.989975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:02.989992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:02.990016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341307785233794:5210], now have 1 active actors on pipe
2025-10-09T21:45:02.990816Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:02.990882Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:02.990893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:02.990909Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341307785233794:5210]
2025-10-09T21:45:02.990965Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 435 reason: error at write: Disconnected sessionId: src_id|8b6b7269-2d77212a-996b6d24-d0983c36_0
2025-10-09T21:45:02.991191Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 435 sessionId: src_id|8b6b7269-2d77212a-996b6d24-d0983c36_0 is DEAD
2025-10-09T21:45:02.990893Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:02.994190Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b6b7269-2d77212a-996b6d24-d0983c36_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:02.994207Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b6b7269-2d77212a-996b6d24-d0983c36_0] Write session will restart in 0.008739s
2025-10-09T21:45:02.994236Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b6b7269-2d77212a-996b6d24-d0983c36_0] Write session: Do CDS request
2025-10-09T21:45:02.994242Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b6b7269-2d77212a-996b6d24-d0983c36_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:03.006562Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b6b7269-2d77212a-996b6d24-d0983c36_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:03.006628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:03.006645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 436
2025-10-09T21:45:03.006943Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 436 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:03.007001Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 436 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:03.007013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=436 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:03.007017Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:03.007444Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 436 sessionId:  describe result for acl check
2025-10-09T21:45:03.007484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:03.007490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:03.007492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:03.007498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341312080201129:5425] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:03.008042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341312080201129:5425] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:03.017294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341312080201129:5425] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:03.022912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341312080201129:5425] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:03.022931Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341312080201129:5425] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:03.022935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341312080201129:5425] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:03.022946Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 436 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:03.030828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:03.034977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:03.035300Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:03.035304Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:03.035318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:03.035391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:03.035433Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341311718389407:4985]
2025-10-09T21:45:03.036054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:03.036528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:03.036583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:03.036956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:03.037012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:03.037014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:03.037015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:03.037017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:03.037018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:03.037020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:03.037021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.037022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.037036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.037044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:03.037055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:03.037059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:03.037064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:03.037069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:03.037073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:03.037077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.037082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.037085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.037094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:03.037102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:03.037110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:03.037117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:03.037122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:03.037127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:03.037132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:03.037138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:03.037143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:03.037149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:03.037154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:03.037160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:03.037166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.037172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.037176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.037182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.037187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.037193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.037225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:03.045306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:03.046531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:03.046553Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:03.046556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:03.046560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:03.046568Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 435 [2:7559341311718389407:4985]
2025-10-09T21:45:03.046583Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:03.046603Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:03.046607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.046610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.046613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.046634Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:03.046648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:03.046655Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:03.046727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.046736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.046738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.046741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:03.047071Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 435
2025-10-09T21:45:03.047116Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 435
2025-10-09T21:45:03.047195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:03.047207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.047223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.047236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.047255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:03.047279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341307785233805:2432], now have 1 active actors on pipe
2025-10-09T21:45:03.047288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341312080201324:5425], now have 1 active actors on pipe
2025-10-09T21:45:03.047309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:03.047314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:03.047332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341311718389402:4985]
2025-10-09T21:45:03.047339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341311718389402:4985]
2025-10-09T21:45:03.047341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341311718389402:4985]
2025-10-09T21:45:03.047354Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:03.047369Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:03.047371Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:03.047377Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:03.047379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:03.047386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:03.047389Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:03.047411Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|475438e2-ecb1cecc-135e0a1c-4d653add_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:03.047419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.047423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.047425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:03.047437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.047455Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:03.047503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341311718389402:4985]
2025-10-09T21:45:03.047516Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:03.047530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:03.047557Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:03.047692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:03.047700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:03.047716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:03.047896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 436 partition: 0 MaxSeqNo: 25 sessionId: src_id|475438e2-ecb1cecc-135e0a1c-4d653add_0
2025-10-09T21:45:03.048343Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b6b7269-2d77212a-996b6d24-d0983c36_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|475438e2-ecb1cecc-135e0a1c-4d653add_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:03.050927Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|475438e2-ecb1cecc-135e0a1c-4d653add_0] Write session: try to update token
2025-10-09T21:45:03.050952Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|475438e2-ecb1cecc-135e0a1c-4d653add_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:03.071520Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:03.071540Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:03.071553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341311718389402:4985]
2025-10-09T21:45:03.071607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:03.071616Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:03.071792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:03.071947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:03.072097Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:03.072248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:03.072572Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:03.072645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:03.079459Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 436 sessionId: src_id|475438e2-ecb1cecc-135e0a1c-4d653add_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:03.082289Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:03.083282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:03.083308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:03.083354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:03.083530Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:03.091184Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341312080201388:5210]
2025-10-09T21:45:03.091475Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 435, pipe: [1:7559341312080201388:5210]
2025-10-09T21:45:03.091221Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:03.091239Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:03.091242Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:03.091244Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:03.091255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341311718389402:4985]
2025-10-09T21:45:03.091259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341311718389402:4985]
2025-10-09T21:45:03.091263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341311718389402:4985]
2025-10-09T21:45:03.091311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:03.091320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.091322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:03.091323Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:03.091327Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:03.091348Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:03.091486Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:03.260876Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:03.261290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341312080201388:5210], now have 1 active actors on pipe
2025-10-09T21:45:03.268945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:03.268975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:03.269151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.269217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.269275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.269332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.269374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.269380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.269447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.269504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.269563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.269624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.269664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.269671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.269736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.269791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.269848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.269908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.269946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.269949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.273519Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|475438e2-ecb1cecc-135e0a1c-4d653add_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:03.273532Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|475438e2-ecb1cecc-135e0a1c-4d653add_0] Write session will restart in 0.008802s
2025-10-09T21:45:03.273556Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|475438e2-ecb1cecc-135e0a1c-4d653add_0] Write session: Do CDS request
2025-10-09T21:45:03.273562Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|475438e2-ecb1cecc-135e0a1c-4d653add_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:03.273075Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341312080201388:5210]
2025-10-09T21:45:03.273117Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:03.273120Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:03.273163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 436 reason: error at write: Disconnected sessionId: src_id|475438e2-ecb1cecc-135e0a1c-4d653add_0
2025-10-09T21:45:03.273369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 436 sessionId: src_id|475438e2-ecb1cecc-135e0a1c-4d653add_0 is DEAD
2025-10-09T21:45:03.273648Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:03.270150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.270217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.270288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.270348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.270389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.270393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.270468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.270530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.270594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.270661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.270706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.270710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.270777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.270837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.270900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.270959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.271003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.271006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.271078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.271235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.271421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.271706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.271758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.271761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.271823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.271878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.271935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.271991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.272100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.272103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.272166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.272220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.272281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.272340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.272383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.272386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.272454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.272515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.272575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.272639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.272745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.272749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.274227Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:03.283644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|475438e2-ecb1cecc-135e0a1c-4d653add_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:03.283773Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:03.283791Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 437
2025-10-09T21:45:03.284032Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 437 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:03.284066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 437 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:03.284072Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=437 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:03.284076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:03.284963Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 437 sessionId:  describe result for acl check
2025-10-09T21:45:03.285006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:03.285014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:03.285019Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:03.285026Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341312080201432:5434] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:03.285628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341312080201432:5434] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:03.290907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341312080201432:5434] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:03.295789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341312080201432:5434] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:03.295807Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341312080201432:5434] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:03.295812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341312080201432:5434] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:03.296005Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 437 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:03.304706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:03.307863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:03.308135Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:03.308145Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:03.308157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:03.308218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:03.308264Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341311718389678:4990]
2025-10-09T21:45:03.308865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:03.309344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:03.309395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:03.309793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:03.309858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:03.309860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:03.309861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:03.309863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:03.309864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:03.309865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:03.309866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.309867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.309868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.309873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:03.309884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:03.309888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:03.309892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:03.309895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:03.309899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:03.309902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.309906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.309909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.309917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:03.309925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:03.309931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:03.309936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:03.309940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:03.309945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:03.309949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:03.309954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:03.309959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:03.309963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:03.309967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:03.309971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:03.309975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.309980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.309984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.310005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.310009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.310014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.310056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:03.320991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:03.322473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:03.322489Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:03.322492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:03.322496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:03.322506Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 436 [2:7559341311718389678:4990]
2025-10-09T21:45:03.322520Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:03.322540Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:03.322543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.322546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.322550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.322569Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:03.322574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:03.322582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:03.322644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.322646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.322648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.322650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:03.322722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:03.322729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.322740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.322750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.322763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:03.322785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341311718389673:4990]
2025-10-09T21:45:03.322787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341311718389673:4990]
2025-10-09T21:45:03.322789Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341311718389673:4990]
2025-10-09T21:45:03.322806Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:03.322823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:03.322836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:03.322839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:03.322881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341311718389673:4990]
2025-10-09T21:45:03.322883Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:03.322935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341312080201574:5434], now have 1 active actors on pipe
2025-10-09T21:45:03.323012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341312080201391:2432], now have 1 active actors on pipe
2025-10-09T21:45:03.323057Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:03.323060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:03.323065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:03.323072Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:03.323190Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 436
2025-10-09T21:45:03.323225Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 436
2025-10-09T21:45:03.323263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:03.323280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:03.323312Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|aacbf77f-918f7596-dc5f42c6-ae7a393a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:03.323328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.323332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.323336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:03.323356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.323369Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:03.323391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:03.323768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:03.323778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:03.323792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:03.324112Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 437 partition: 0 MaxSeqNo: 25 sessionId: src_id|aacbf77f-918f7596-dc5f42c6-ae7a393a_0
2025-10-09T21:45:03.324520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|475438e2-ecb1cecc-135e0a1c-4d653add_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|aacbf77f-918f7596-dc5f42c6-ae7a393a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:03.330861Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aacbf77f-918f7596-dc5f42c6-ae7a393a_0] Write session: try to update token
2025-10-09T21:45:03.330885Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aacbf77f-918f7596-dc5f42c6-ae7a393a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:03.354553Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:03.354570Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:03.354581Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341311718389673:4990]
2025-10-09T21:45:03.354653Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:03.356006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:03.356475Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:03.357452Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 437 sessionId: src_id|aacbf77f-918f7596-dc5f42c6-ae7a393a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:03.356626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:03.356776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:03.356920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:03.357130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:03.357190Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:03.357194Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:03.357196Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:03.357199Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:03.357208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341311718389673:4990]
2025-10-09T21:45:03.357212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341311718389673:4990]
2025-10-09T21:45:03.357217Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341311718389673:4990]
2025-10-09T21:45:03.357265Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:03.357287Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:03.357290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:03.357294Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:03.357295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.357297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:03.357298Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:03.357436Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:03.360361Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:03.373332Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341312080201657:5210]
2025-10-09T21:45:03.373774Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 436, pipe: [1:7559341312080201657:5210]
2025-10-09T21:45:03.535549Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:03.536339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:03.536360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:03.536386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341312080201657:5210], now have 1 active actors on pipe
2025-10-09T21:45:03.538380Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:03.538963Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aacbf77f-918f7596-dc5f42c6-ae7a393a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:03.538974Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aacbf77f-918f7596-dc5f42c6-ae7a393a_0] Write session will restart in 0.006669s
2025-10-09T21:45:03.538998Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aacbf77f-918f7596-dc5f42c6-ae7a393a_0] Write session: Do CDS request
2025-10-09T21:45:03.539006Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aacbf77f-918f7596-dc5f42c6-ae7a393a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:03.538429Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:03.538496Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341312080201657:5210]
2025-10-09T21:45:03.538525Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:03.538530Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:03.538608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 437 reason: error at write: Disconnected sessionId: src_id|aacbf77f-918f7596-dc5f42c6-ae7a393a_0
2025-10-09T21:45:03.538804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 437 sessionId: src_id|aacbf77f-918f7596-dc5f42c6-ae7a393a_0 is DEAD
2025-10-09T21:45:03.547710Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aacbf77f-918f7596-dc5f42c6-ae7a393a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:03.548172Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:03.548191Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 438
2025-10-09T21:45:03.550296Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 438 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:03.550345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 438 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:03.550355Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=438 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:03.550370Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:03.551674Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 438 sessionId:  describe result for acl check
2025-10-09T21:45:03.551713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:03.551717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:03.551718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:03.551725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341312080201675:5439] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:03.552191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341312080201675:5439] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:03.556523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341312080201675:5439] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:03.562093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341312080201675:5439] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:03.562111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341312080201675:5439] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:03.562114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341312080201675:5439] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:03.562124Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 438 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:03.590437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:03.597831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:03.598131Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:03.598136Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:03.598152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:03.598236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:03.598295Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341311718389938:4993]
2025-10-09T21:45:03.598942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:03.599358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:03.599412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:03.599763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:03.599828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:03.599829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:03.599831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:03.599832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:03.599834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:03.599836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:03.599837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.599838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.599840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.599845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:03.599855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:03.599859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:03.599863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:03.599867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:03.599870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:03.599873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.599877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.599881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.599889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:03.599896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:03.599903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:03.599908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:03.599913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:03.599919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:03.599924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:03.599930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:03.599936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:03.599941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:03.599957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:03.599962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:03.599968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.599976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.599981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.599987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.599992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.599997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.600031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:03.609036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:03.611068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:03.611077Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:03.611080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:03.611085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:03.611092Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 437 [2:7559341311718389938:4993]
2025-10-09T21:45:03.611123Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:03.611143Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:03.611147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.611150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.611152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.611172Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:03.611178Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:03.611186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:03.611255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.611258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.611260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.611642Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 437
2025-10-09T21:45:03.611681Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 437
2025-10-09T21:45:03.611263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:03.611340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:03.611343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.611357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.611368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.611384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:03.611406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341311718389933:4993]
2025-10-09T21:45:03.611409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341311718389933:4993]
2025-10-09T21:45:03.611410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341311718389933:4993]
2025-10-09T21:45:03.611418Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:03.611436Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:03.611444Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:03.611448Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:03.611450Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:03.611451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:03.611454Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:03.611467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341312080201739:5439], now have 1 active actors on pipe
2025-10-09T21:45:03.611507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341312080201662:2432], now have 1 active actors on pipe
2025-10-09T21:45:03.611509Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341311718389933:4993]
2025-10-09T21:45:03.611519Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:03.611562Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:03.612161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:03.612178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:03.612212Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5cf3b80-227e38b2-49401ad8-5bb465df_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:03.612225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.612230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.612233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:03.612245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.612271Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:03.612295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:03.612711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:03.612726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:03.612747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:03.612994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 438 partition: 0 MaxSeqNo: 25 sessionId: src_id|5cf3b80-227e38b2-49401ad8-5bb465df_0
2025-10-09T21:45:03.613402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aacbf77f-918f7596-dc5f42c6-ae7a393a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5cf3b80-227e38b2-49401ad8-5bb465df_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:03.616250Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5cf3b80-227e38b2-49401ad8-5bb465df_0] Write session: try to update token
2025-10-09T21:45:03.616273Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5cf3b80-227e38b2-49401ad8-5bb465df_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:03.637520Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 438 sessionId: src_id|5cf3b80-227e38b2-49401ad8-5bb465df_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:03.638567Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341312080201927:5210]
2025-10-09T21:45:03.638985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341312080201927:5210], now have 1 active actors on pipe
2025-10-09T21:45:03.640474Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 437, pipe: [1:7559341312080201927:5210]
2025-10-09T21:45:03.640550Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:03.640757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:03.640776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:03.640830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:03.640938Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:03.648650Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:03.648671Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:03.648674Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:03.648675Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:03.648684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341311718389933:4993]
2025-10-09T21:45:03.648688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341311718389933:4993]
2025-10-09T21:45:03.648692Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341311718389933:4993]
2025-10-09T21:45:03.648738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:03.648745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.648747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:03.648767Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:03.648787Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:03.648792Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:03.648903Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:03.811458Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:03.819739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:03.819767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:03.819941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.819997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.820052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.820107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.820149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.820154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.820217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.820276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.820332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.820394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.820434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.820437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.820497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.820550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.820604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.820659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.820702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.820705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.820774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.820832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.820892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.820949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.820993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.820996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.821060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.821117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.821177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.821248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.821288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.821291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.821352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.821406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.821461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.821516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.821553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.821556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.821616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.821667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.821723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.821779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.821818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.821820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.821882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.821934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.821993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.826113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.826181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.826184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.826268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.826346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.826402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.826457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.826494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.826496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.826582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:03.826634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:03.826690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:03.826746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:03.826785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:03.826787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:03.828882Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5cf3b80-227e38b2-49401ad8-5bb465df_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:03.828895Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5cf3b80-227e38b2-49401ad8-5bb465df_0] Write session will restart in 0.009489s
2025-10-09T21:45:03.828921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5cf3b80-227e38b2-49401ad8-5bb465df_0] Write session: Do CDS request
2025-10-09T21:45:03.828927Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5cf3b80-227e38b2-49401ad8-5bb465df_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:03.828612Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:03.828436Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:03.828450Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:03.828493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 438 reason: error at write: Disconnected sessionId: src_id|5cf3b80-227e38b2-49401ad8-5bb465df_0
2025-10-09T21:45:03.828711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 438 sessionId: src_id|5cf3b80-227e38b2-49401ad8-5bb465df_0 is DEAD
2025-10-09T21:45:03.829254Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341312080201927:5210]
2025-10-09T21:45:03.829270Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:03.841238Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5cf3b80-227e38b2-49401ad8-5bb465df_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:03.841379Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:03.841401Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 439
2025-10-09T21:45:03.841583Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 439 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:03.841631Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 439 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:03.841637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=439 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:03.841642Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:03.842154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 439 sessionId:  describe result for acl check
2025-10-09T21:45:03.842200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:03.842202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:03.842204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:03.842211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341312080201965:5443] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:03.842843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341312080201965:5443] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:03.851693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341312080201965:5443] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:03.856369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341312080201965:5443] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:03.856390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341312080201965:5443] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:03.856394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341312080201965:5443] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:03.856419Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 439 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:03.868807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:03.872464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:03.872732Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:03.872737Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:03.872749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:03.872808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:03.872848Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341311718390198:4995]
2025-10-09T21:45:03.873472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:03.874001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:03.874089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:03.874468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:03.874521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:03.874522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:03.874523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:03.874525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:03.874526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:03.874527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:03.874528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.874529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.874530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.874536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:03.874546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:03.874550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:03.874554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:03.874558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:03.874562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:03.874566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.874570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.874574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.874581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:03.874590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:03.874597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:03.874602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:03.874607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:03.874613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:03.874617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:03.874621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:03.874626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:03.874630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:03.874634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:03.874639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:03.874646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.874650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.874654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.874660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.874663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.874668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.874697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:03.883655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:03.884743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:03.884749Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:03.884752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:03.884757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:03.884766Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 438 [2:7559341311718390198:4995]
2025-10-09T21:45:03.884785Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:03.884806Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:03.884809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.884813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.884815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.884837Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:03.884842Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:03.884851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:03.885379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.885383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.885385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.885388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:03.885469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:03.885472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.885486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:03.885497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:03.885513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:03.885534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341311718390193:4995]
2025-10-09T21:45:03.885536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341311718390193:4995]
2025-10-09T21:45:03.885539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341311718390193:4995]
2025-10-09T21:45:03.885553Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:03.885568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:03.885583Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:03.885586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:03.885630Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341311718390193:4995]
2025-10-09T21:45:03.885633Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:03.886163Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 438
2025-10-09T21:45:03.886166Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 438
2025-10-09T21:45:03.886024Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:03.886043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:03.886045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:03.886052Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:03.886458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341312080202084:5443], now have 1 active actors on pipe
2025-10-09T21:45:03.886468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341312080201932:2432], now have 1 active actors on pipe
2025-10-09T21:45:03.886482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:03.886488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:03.886523Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f08edc5e-930d6ff9-5c40c8f3-def1cc59_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:03.886534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.886539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.886542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:03.886555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.886568Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:03.886587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:03.886940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:03.886948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:03.886964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:03.887142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 439 partition: 0 MaxSeqNo: 25 sessionId: src_id|f08edc5e-930d6ff9-5c40c8f3-def1cc59_0
2025-10-09T21:45:03.887704Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5cf3b80-227e38b2-49401ad8-5bb465df_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f08edc5e-930d6ff9-5c40c8f3-def1cc59_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:03.890352Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f08edc5e-930d6ff9-5c40c8f3-def1cc59_0] Write session: try to update token
2025-10-09T21:45:03.890379Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f08edc5e-930d6ff9-5c40c8f3-def1cc59_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:03.903263Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:03.903285Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:03.903288Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:03.903290Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:03.903300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341311718390193:4995]
2025-10-09T21:45:03.903305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341311718390193:4995]
2025-10-09T21:45:03.903311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341311718390193:4995]
2025-10-09T21:45:03.903358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:03.903366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:03.903368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:03.903520Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:03.906735Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:03.906752Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:03.906756Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:03.915738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 439 sessionId: src_id|f08edc5e-930d6ff9-5c40c8f3-def1cc59_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:03.919197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:03.929447Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341312080202205:5210]
2025-10-09T21:45:03.929783Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 438, pipe: [1:7559341312080202205:5210]
2025-10-09T21:45:04.108517Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:04.109116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:04.109127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:04.109152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341312080202205:5210], now have 1 active actors on pipe
2025-10-09T21:45:04.109806Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:04.109820Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:04.109875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 439 reason: error at write: Disconnected sessionId: src_id|f08edc5e-930d6ff9-5c40c8f3-def1cc59_0
2025-10-09T21:45:04.110078Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 439 sessionId: src_id|f08edc5e-930d6ff9-5c40c8f3-def1cc59_0 is DEAD
2025-10-09T21:45:04.110364Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:04.110403Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341312080202205:5210]
2025-10-09T21:45:04.110321Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:04.111268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f08edc5e-930d6ff9-5c40c8f3-def1cc59_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:04.111279Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f08edc5e-930d6ff9-5c40c8f3-def1cc59_0] Write session will restart in 0.006302s
2025-10-09T21:45:04.111306Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f08edc5e-930d6ff9-5c40c8f3-def1cc59_0] Write session: Do CDS request
2025-10-09T21:45:04.111313Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f08edc5e-930d6ff9-5c40c8f3-def1cc59_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:04.118732Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f08edc5e-930d6ff9-5c40c8f3-def1cc59_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:04.119002Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:04.119020Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 440
2025-10-09T21:45:04.119183Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 440 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:04.119243Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 440 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:04.119256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=440 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:04.119261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:04.119702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 440 sessionId:  describe result for acl check
2025-10-09T21:45:04.119741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:04.119744Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:04.119745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:04.119751Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341316375169536:5454] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:04.120293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341316375169536:5454] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:04.123920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341316375169536:5454] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:04.126615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341316375169536:5454] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:04.126631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341316375169536:5454] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:04.126635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341316375169536:5454] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:04.126645Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 440 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:04.145792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:04.150267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:04.150529Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:04.150540Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:04.150553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:04.150608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:04.150654Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341316013357796:5008]
2025-10-09T21:45:04.151201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:04.151589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:04.151637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:04.151995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:04.152044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:04.152050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:04.152051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:04.152053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:04.152054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:04.152055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:04.152056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.152058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.152059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.152063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:04.152075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:04.152079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:04.152083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:04.152087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:04.152090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:04.152093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.152096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.152100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.152109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:04.152116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:04.152122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:04.152129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:04.152134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:04.152139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:04.152149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:04.152153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:04.152157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:04.152162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:04.152167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:04.152172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:04.152179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.152184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.152196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.152200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.152205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.152210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.152241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:04.160232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:04.161824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:04.161845Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:04.161847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:04.161851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:04.161859Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 439 [2:7559341316013357796:5008]
2025-10-09T21:45:04.161875Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:04.161895Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:04.161898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.161901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.161904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.161923Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:04.161929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:04.161936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:04.162021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.162022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.162024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.162027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:04.162125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:04.162134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.162146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.162157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.162172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:04.162197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341316013357791:5008]
2025-10-09T21:45:04.162199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341316013357791:5008]
2025-10-09T21:45:04.162202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341316013357791:5008]
2025-10-09T21:45:04.162209Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:04.162223Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:04.162225Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:04.162226Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:04.162238Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:04.162241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:04.162244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:04.162268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341316375169577:5454], now have 1 active actors on pipe
2025-10-09T21:45:04.162375Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 439
2025-10-09T21:45:04.162272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341316375169525:2432], now have 1 active actors on pipe
2025-10-09T21:45:04.162312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341316013357791:5008]
2025-10-09T21:45:04.162364Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 439
2025-10-09T21:45:04.162323Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:04.162362Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:04.162661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:04.162672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:04.162719Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ba64efb4-b5e82c03-9ada87f8-3acaf7d0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:04.162728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.162732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.162735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:04.162749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.162762Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:04.162779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:04.166380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:04.166400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:04.166455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:04.166680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 440 partition: 0 MaxSeqNo: 25 sessionId: src_id|ba64efb4-b5e82c03-9ada87f8-3acaf7d0_0
2025-10-09T21:45:04.170167Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f08edc5e-930d6ff9-5c40c8f3-def1cc59_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ba64efb4-b5e82c03-9ada87f8-3acaf7d0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:04.172899Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ba64efb4-b5e82c03-9ada87f8-3acaf7d0_0] Write session: try to update token
2025-10-09T21:45:04.172928Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ba64efb4-b5e82c03-9ada87f8-3acaf7d0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:04.178988Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:04.179016Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:04.179020Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:04.179022Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:04.179032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341316013357791:5008]
2025-10-09T21:45:04.179038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341316013357791:5008]
2025-10-09T21:45:04.179042Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341316013357791:5008]
2025-10-09T21:45:04.179092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:04.179100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.179103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:04.179240Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:04.182111Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:04.182132Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:04.182136Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:04.198873Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 440 sessionId: src_id|ba64efb4-b5e82c03-9ada87f8-3acaf7d0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:04.201439Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:04.210438Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341316375169797:5210]
2025-10-09T21:45:04.210825Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 439, pipe: [1:7559341316375169797:5210]
2025-10-09T21:45:04.345192Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:04.346431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:04.346451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:04.346476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341316375169797:5210], now have 1 active actors on pipe
2025-10-09T21:45:04.346724Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:04.347061Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:04.347128Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341316375169797:5210]
2025-10-09T21:45:04.347155Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:04.347240Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 440 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ba64efb4-b5e82c03-9ada87f8-3acaf7d0_0
2025-10-09T21:45:04.347488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 440 sessionId: src_id|ba64efb4-b5e82c03-9ada87f8-3acaf7d0_0 is DEAD
2025-10-09T21:45:04.347596Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba64efb4-b5e82c03-9ada87f8-3acaf7d0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:04.347608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba64efb4-b5e82c03-9ada87f8-3acaf7d0_0] Write session will restart in 0.006211s
2025-10-09T21:45:04.347636Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba64efb4-b5e82c03-9ada87f8-3acaf7d0_0] Write session: Do CDS request
2025-10-09T21:45:04.347644Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba64efb4-b5e82c03-9ada87f8-3acaf7d0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:04.354579Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ba64efb4-b5e82c03-9ada87f8-3acaf7d0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:04.354654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:04.354671Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 441
2025-10-09T21:45:04.354870Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 441 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:04.354915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 441 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:04.354934Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=441 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:04.354938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:04.355381Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 441 sessionId:  describe result for acl check
2025-10-09T21:45:04.355423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:04.355429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:04.355431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:04.355437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341316375169814:5458] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:04.355992Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341316375169814:5458] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:04.359109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341316375169814:5458] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:04.365304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341316375169814:5458] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:04.365329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341316375169814:5458] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:04.365332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341316375169814:5458] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:04.365342Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 441 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:04.372992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:04.375856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:04.376070Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:04.376079Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:04.376091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:04.376140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:04.376181Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341316013358059:5013]
2025-10-09T21:45:04.376712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:04.377032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:04.377086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:04.377385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:04.377429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:04.377435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:04.377436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:04.377437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:04.377438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:04.377439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:04.377440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.377441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.377442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.377447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:04.377457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:04.377460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:04.377465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:04.377468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:04.377472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:04.377475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.377478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.377481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.377487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:04.377495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:04.377500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:04.377504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:04.377508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:04.377512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:04.377516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:04.377520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:04.377524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:04.377529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:04.377534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:04.377539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:04.377548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.377553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.377556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.377561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.377566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.377570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.377593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:04.386600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:04.388335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:04.388354Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:04.388357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:04.388361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:04.388369Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 440 [2:7559341316013358059:5013]
2025-10-09T21:45:04.388382Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:04.388403Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:04.388406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.388409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.388411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.388429Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:04.388436Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:04.388442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:04.388522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.388531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.388532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.388535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:04.388607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:04.388609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.388623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.388632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.388645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:04.388670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341316013358054:5013]
2025-10-09T21:45:04.388678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341316013358054:5013]
2025-10-09T21:45:04.388679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341316013358054:5013]
2025-10-09T21:45:04.388690Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:04.388705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:04.388724Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:04.388729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:04.388731Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:04.388732Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:04.388735Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:04.388772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341316013358054:5013]
2025-10-09T21:45:04.388785Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:04.388799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341316375169800:2432], now have 1 active actors on pipe
2025-10-09T21:45:04.388807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341316375169962:5458], now have 1 active actors on pipe
2025-10-09T21:45:04.388808Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:04.388855Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 440
2025-10-09T21:45:04.388914Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 440
2025-10-09T21:45:04.389075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:04.389089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:04.389121Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1b0e9a9a-bd9e7714-234cba3a-e59dc879_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:04.389136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.389141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.389143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:04.389165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.389180Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:04.389203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:04.389429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:04.389439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:04.389454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:04.394348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 441 partition: 0 MaxSeqNo: 25 sessionId: src_id|1b0e9a9a-bd9e7714-234cba3a-e59dc879_0
2025-10-09T21:45:04.394838Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba64efb4-b5e82c03-9ada87f8-3acaf7d0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1b0e9a9a-bd9e7714-234cba3a-e59dc879_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:04.397653Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b0e9a9a-bd9e7714-234cba3a-e59dc879_0] Write session: try to update token
2025-10-09T21:45:04.397683Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b0e9a9a-bd9e7714-234cba3a-e59dc879_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:04.409392Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:04.409412Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:04.409415Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:04.409416Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:04.409426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341316013358054:5013]
2025-10-09T21:45:04.409430Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341316013358054:5013]
2025-10-09T21:45:04.409433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341316013358054:5013]
2025-10-09T21:45:04.409474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:04.409481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.409483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:04.409617Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:04.410172Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:04.410187Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:04.410191Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:04.420462Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 441 sessionId: src_id|1b0e9a9a-bd9e7714-234cba3a-e59dc879_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:04.423190Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:04.447144Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341316375170070:5210]
2025-10-09T21:45:04.447600Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 440, pipe: [1:7559341316375170070:5210]
2025-10-09T21:45:04.577608Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:04.577928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:04.577947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:04.577969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341316375170070:5210], now have 1 active actors on pipe
2025-10-09T21:45:04.578687Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341316375170070:5210]
2025-10-09T21:45:04.578725Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:04.578759Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:04.578762Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:04.578823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 441 reason: error at write: Disconnected sessionId: src_id|1b0e9a9a-bd9e7714-234cba3a-e59dc879_0
2025-10-09T21:45:04.579001Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 441 sessionId: src_id|1b0e9a9a-bd9e7714-234cba3a-e59dc879_0 is DEAD
2025-10-09T21:45:04.579335Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:04.582493Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b0e9a9a-bd9e7714-234cba3a-e59dc879_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:04.582505Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b0e9a9a-bd9e7714-234cba3a-e59dc879_0] Write session will restart in 0.006706s
2025-10-09T21:45:04.582536Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b0e9a9a-bd9e7714-234cba3a-e59dc879_0] Write session: Do CDS request
2025-10-09T21:45:04.582544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b0e9a9a-bd9e7714-234cba3a-e59dc879_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:04.590674Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b0e9a9a-bd9e7714-234cba3a-e59dc879_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:04.590739Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:04.590755Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 442
2025-10-09T21:45:04.590899Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 442 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:04.590946Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 442 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:04.590957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=442 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:04.590961Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:04.592154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 442 sessionId:  describe result for acl check
2025-10-09T21:45:04.592195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:04.592197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:04.592203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:04.592216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341316375170099:5463] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:04.592926Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341316375170099:5463] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:04.599690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341316375170099:5463] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:04.605636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341316375170099:5463] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:04.605659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341316375170099:5463] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:04.605663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341316375170099:5463] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:04.605673Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 442 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:04.613997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:04.618605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:04.618872Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:04.618877Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:04.618890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:04.618959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:04.619001Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341316013358323:5015]
2025-10-09T21:45:04.619597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:04.620202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:04.620258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:04.620634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:04.620689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:04.620691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:04.620692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:04.620693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:04.620695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:04.620696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:04.620697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.620699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.620700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.620706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:04.620718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:04.620733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:04.620737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:04.620740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:04.620744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:04.620747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.620751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.620755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.620763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:04.620772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:04.620779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:04.620793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:04.620798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:04.620802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:04.620808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:04.620813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:04.620818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:04.620822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:04.620827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:04.620833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:04.620840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.620845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.620851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.620856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.620864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.620871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.620905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:04.630867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 441
2025-10-09T21:45:04.630905Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 441
2025-10-09T21:45:04.628591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:04.630398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:04.630405Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:04.630409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:04.630414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:04.630422Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 441 [2:7559341316013358323:5015]
2025-10-09T21:45:04.630438Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:04.630457Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:04.630460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.630463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.630466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.630486Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:04.630492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:04.630500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:04.630569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.630571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.630573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.630576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:04.630657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:04.630660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.630675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.630685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.630699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:04.630720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341316013358318:5015]
2025-10-09T21:45:04.630723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341316013358318:5015]
2025-10-09T21:45:04.630725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341316013358318:5015]
2025-10-09T21:45:04.630734Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:04.630750Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:04.630751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:04.630753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:04.630755Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:04.630766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:04.630769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:04.630781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341316375170260:5463], now have 1 active actors on pipe
2025-10-09T21:45:04.630785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341316375170075:2432], now have 1 active actors on pipe
2025-10-09T21:45:04.630823Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341316013358318:5015]
2025-10-09T21:45:04.630832Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:04.630835Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:04.631284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:04.631291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:04.631336Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|edf38c7c-5f0a6605-5d1692de-99a1d2a0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:04.631346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.631367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.631370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:04.631382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.631396Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:04.631417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:04.631787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:04.631806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:04.631825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:04.632637Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b0e9a9a-bd9e7714-234cba3a-e59dc879_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|edf38c7c-5f0a6605-5d1692de-99a1d2a0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:04.632027Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 442 partition: 0 MaxSeqNo: 25 sessionId: src_id|edf38c7c-5f0a6605-5d1692de-99a1d2a0_0
2025-10-09T21:45:04.635578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|edf38c7c-5f0a6605-5d1692de-99a1d2a0_0] Write session: try to update token
2025-10-09T21:45:04.635601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|edf38c7c-5f0a6605-5d1692de-99a1d2a0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:04.649359Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:04.649379Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:04.649392Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:04.649395Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:04.649405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341316013358318:5015]
2025-10-09T21:45:04.649410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341316013358318:5015]
2025-10-09T21:45:04.649414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341316013358318:5015]
2025-10-09T21:45:04.649455Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:04.649462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:04.649467Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:04.649470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.649471Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:04.649473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:04.649628Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:04.662308Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 442 sessionId: src_id|edf38c7c-5f0a6605-5d1692de-99a1d2a0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:04.665140Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:04.680756Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341316375170345:5210]
2025-10-09T21:45:04.681265Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 441, pipe: [1:7559341316375170345:5210]
2025-10-09T21:45:04.819173Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:04.819767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:04.819786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:04.819832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341316375170345:5210], now have 1 active actors on pipe
2025-10-09T21:45:04.820057Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341316375170345:5210]
2025-10-09T21:45:04.820107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:04.820110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:04.820198Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 442 reason: error at write: Disconnected sessionId: src_id|edf38c7c-5f0a6605-5d1692de-99a1d2a0_0
2025-10-09T21:45:04.820389Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:04.820401Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 442 sessionId: src_id|edf38c7c-5f0a6605-5d1692de-99a1d2a0_0 is DEAD
2025-10-09T21:45:04.822157Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edf38c7c-5f0a6605-5d1692de-99a1d2a0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:04.822171Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edf38c7c-5f0a6605-5d1692de-99a1d2a0_0] Write session will restart in 0.006961s
2025-10-09T21:45:04.822195Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edf38c7c-5f0a6605-5d1692de-99a1d2a0_0] Write session: Do CDS request
2025-10-09T21:45:04.822205Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edf38c7c-5f0a6605-5d1692de-99a1d2a0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:04.822345Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:04.834531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|edf38c7c-5f0a6605-5d1692de-99a1d2a0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:04.838133Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:04.838151Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 443
2025-10-09T21:45:04.842151Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 443 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:04.842211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 443 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:04.842221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=443 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:04.842236Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:04.844040Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 443 sessionId:  describe result for acl check
2025-10-09T21:45:04.844082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:04.844090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:04.844092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:04.844099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341316375170403:5467] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:04.844679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341316375170403:5467] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:04.849403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341316375170403:5467] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:04.855318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341316375170403:5467] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:04.855334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341316375170403:5467] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:04.855338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341316375170403:5467] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:04.855349Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 443 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:04.874428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:04.878942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:04.879208Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:04.879222Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:04.879235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:04.879294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:04.879337Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341316013358588:5017]
2025-10-09T21:45:04.879940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:04.880430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:04.880483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:04.880807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:04.880852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:04.880854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:04.880855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:04.880856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:04.880857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:04.880858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:04.880860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.880861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.880862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.880868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:04.880879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:04.880883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:04.880887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:04.880891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:04.880895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:04.880898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.880902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.880906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.880913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:04.880921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:04.880927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:04.880933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:04.880937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:04.880942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:04.880946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:04.880951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:04.880956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:04.880960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:04.880965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:04.880970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:04.880975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.880980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.880984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.880988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.880996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.881002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.881027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:04.888318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:04.889798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:04.889814Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:04.889816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:04.889820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:04.889825Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 442 [2:7559341316013358588:5017]
2025-10-09T21:45:04.889838Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:04.889856Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:04.889858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.889860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.889861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.889876Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:04.889880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:04.889887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:04.889952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.889962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.889964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.889968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:04.890091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:04.890101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.890114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:04.890135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:04.890152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:04.890176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341316375170352:2432], now have 1 active actors on pipe
2025-10-09T21:45:04.890184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341316375170478:5467], now have 1 active actors on pipe
2025-10-09T21:45:04.890193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341316013358582:5017]
2025-10-09T21:45:04.890197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341316013358582:5017]
2025-10-09T21:45:04.890199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341316013358582:5017]
2025-10-09T21:45:04.890213Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:04.890229Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:04.890233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:04.890233Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:04.890235Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:04.890248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:04.890270Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:04.890311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341316013358582:5017]
2025-10-09T21:45:04.890322Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:04.890360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:04.890585Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 442
2025-10-09T21:45:04.890632Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 442
2025-10-09T21:45:04.890886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:04.890901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:04.890942Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|36bed955-d5827cfb-5b52db48-731ef6d1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:04.890956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.890961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.890963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:04.890974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.890997Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:04.891017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:04.891350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:04.891363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:04.891382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:04.891563Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 443 partition: 0 MaxSeqNo: 25 sessionId: src_id|36bed955-d5827cfb-5b52db48-731ef6d1_0
2025-10-09T21:45:04.892008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edf38c7c-5f0a6605-5d1692de-99a1d2a0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|36bed955-d5827cfb-5b52db48-731ef6d1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:04.894826Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36bed955-d5827cfb-5b52db48-731ef6d1_0] Write session: try to update token
2025-10-09T21:45:04.894844Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36bed955-d5827cfb-5b52db48-731ef6d1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:04.910529Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:04.910549Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:04.910553Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:04.910555Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:04.910566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341316013358582:5017]
2025-10-09T21:45:04.910571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341316013358582:5017]
2025-10-09T21:45:04.910575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341316013358582:5017]
2025-10-09T21:45:04.910626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:04.910638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:04.910641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:04.910801Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:04.913291Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:04.913312Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:04.913317Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:04.915912Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 443 sessionId: src_id|36bed955-d5827cfb-5b52db48-731ef6d1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:04.918939Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:04.920493Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341316375170629:5210]
2025-10-09T21:45:04.920891Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 442, pipe: [1:7559341316375170629:5210]
2025-10-09T21:45:05.088079Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:05.088597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:05.088616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:05.088639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341316375170629:5210], now have 1 active actors on pipe
2025-10-09T21:45:05.088917Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:05.089434Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:05.089483Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:05.089554Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 443 reason: status is not ok: tablet will be restarted right now sessionId: src_id|36bed955-d5827cfb-5b52db48-731ef6d1_0
2025-10-09T21:45:05.089755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 443 sessionId: src_id|36bed955-d5827cfb-5b52db48-731ef6d1_0 is DEAD
2025-10-09T21:45:05.089925Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36bed955-d5827cfb-5b52db48-731ef6d1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:05.089934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36bed955-d5827cfb-5b52db48-731ef6d1_0] Write session will restart in 0.007866s
2025-10-09T21:45:05.089971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36bed955-d5827cfb-5b52db48-731ef6d1_0] Write session: Do CDS request
2025-10-09T21:45:05.089979Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36bed955-d5827cfb-5b52db48-731ef6d1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:05.089922Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341316375170629:5210]
2025-10-09T21:45:05.099603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36bed955-d5827cfb-5b52db48-731ef6d1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:05.099801Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:05.099818Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 444
2025-10-09T21:45:05.099974Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 444 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:05.100016Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 444 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:05.100021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=444 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:05.100025Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:05.100455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 444 sessionId:  describe result for acl check
2025-10-09T21:45:05.100486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:05.100488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:05.100490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:05.100496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341320670137948:5472] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:05.101084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341320670137948:5472] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:05.105103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341320670137948:5472] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:05.111956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341320670137948:5472] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:05.111972Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341320670137948:5472] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:05.111976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341320670137948:5472] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:05.111994Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 444 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:05.125090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:05.129354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:05.129591Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:05.129603Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:05.129615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:05.129671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:05.129714Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341320308326181:5025]
2025-10-09T21:45:05.130375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:05.130772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:05.130816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:05.131137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:05.131175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:05.131176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:05.131178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:05.131179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:05.131180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:05.131181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:05.131182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.131183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.131184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.131189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:05.131199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:05.131202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:05.131206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:05.131209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:05.131213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:05.131217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.131220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.131224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.131229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:05.131237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:05.131243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:05.131248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:05.131252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:05.131256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:05.131260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:05.131266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:05.131270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:05.131275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:05.131279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:05.131283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:05.131288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.131292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.131296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.131301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.131306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.131310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.131336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:05.142506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:05.144166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:05.144173Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:05.144175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:05.144187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:05.144195Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 443 [2:7559341320308326181:5025]
2025-10-09T21:45:05.144210Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:05.144230Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:05.144234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.144237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.144239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.144258Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:05.144264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:05.144272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:05.144336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.144338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.144340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.144343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:05.144420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:05.144422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.144434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.144445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.144462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:05.144482Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341320308326165:5025]
2025-10-09T21:45:05.144485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341320308326165:5025]
2025-10-09T21:45:05.144487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341320308326165:5025]
2025-10-09T21:45:05.144495Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:05.144510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341320670137936:2432], now have 1 active actors on pipe
2025-10-09T21:45:05.144514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341320670138038:5472], now have 1 active actors on pipe
2025-10-09T21:45:05.144506Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:05.144521Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:05.144522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:05.144523Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:05.144536Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:05.144539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:05.144580Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341320308326165:5025]
2025-10-09T21:45:05.144593Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:05.144639Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:05.145516Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 443
2025-10-09T21:45:05.145562Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 443
2025-10-09T21:45:05.147250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:05.147279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:05.147320Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dcd71243-9b78fdac-862b8974-e2cd7c92_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:05.147352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.147357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.147361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:05.147373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.147391Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:05.147412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:05.154733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:05.154754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:05.154791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:05.154969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 444 partition: 0 MaxSeqNo: 25 sessionId: src_id|dcd71243-9b78fdac-862b8974-e2cd7c92_0
2025-10-09T21:45:05.155355Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36bed955-d5827cfb-5b52db48-731ef6d1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dcd71243-9b78fdac-862b8974-e2cd7c92_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:05.158156Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dcd71243-9b78fdac-862b8974-e2cd7c92_0] Write session: try to update token
2025-10-09T21:45:05.158177Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dcd71243-9b78fdac-862b8974-e2cd7c92_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:05.177906Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:05.177923Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:05.177934Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341320308326165:5025]
2025-10-09T21:45:05.177986Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:05.178221Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:05.178392Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:05.178555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:05.178714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:05.178933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:05.178973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:05.179194Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:05.185900Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 444 sessionId: src_id|dcd71243-9b78fdac-862b8974-e2cd7c92_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:05.188733Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:05.189908Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341320670138220:5210]
2025-10-09T21:45:05.190261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:05.190284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:05.191159Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:05.190332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:05.192065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341320670138220:5210], now have 1 active actors on pipe
2025-10-09T21:45:05.192076Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:05.192080Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:05.192081Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:05.192083Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:05.192091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341320308326165:5025]
2025-10-09T21:45:05.192094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341320308326165:5025]
2025-10-09T21:45:05.192097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341320308326165:5025]
2025-10-09T21:45:05.192140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:05.192148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.192150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:05.192157Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:05.192163Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:05.192167Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:05.192287Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:05.194099Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 443, pipe: [1:7559341320670138220:5210]
2025-10-09T21:45:05.402553Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:05.410737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:05.410766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:05.410929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:05.410987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:05.411056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:05.411118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:05.411297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:05.411305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:05.411372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:05.411432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:05.411492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:05.411557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:05.411601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:05.411608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:05.411679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:05.411733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:05.411788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:05.411844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:05.411884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:05.411888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:05.411956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:05.412015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:05.412079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:05.412147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:05.412187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:05.412190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:05.412260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:05.412320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:05.412387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:05.412446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:05.412493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:05.412496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:05.412557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:05.412615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:05.412675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:05.412740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:05.412784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:05.412787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:05.412852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:05.412906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:05.412967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:05.413024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:05.413060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:05.413063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:05.413126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:05.413182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:05.413242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:05.413309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:05.413348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:05.413351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:05.413414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:05.413471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:05.413527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:05.413589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:05.413628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:05.413631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:05.413696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:05.413749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:05.413812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:05.413870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:05.414207Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:05.413907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:05.414221Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341320670138220:5210]
2025-10-09T21:45:05.413910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:05.414262Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:05.414265Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:05.414296Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 444 reason: error at write: Disconnected sessionId: src_id|dcd71243-9b78fdac-862b8974-e2cd7c92_0
2025-10-09T21:45:05.414958Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dcd71243-9b78fdac-862b8974-e2cd7c92_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:05.414969Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dcd71243-9b78fdac-862b8974-e2cd7c92_0] Write session will restart in 0.008253s
2025-10-09T21:45:05.414994Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dcd71243-9b78fdac-862b8974-e2cd7c92_0] Write session: Do CDS request
2025-10-09T21:45:05.415000Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dcd71243-9b78fdac-862b8974-e2cd7c92_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:05.414526Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 444 sessionId: src_id|dcd71243-9b78fdac-862b8974-e2cd7c92_0 is DEAD
2025-10-09T21:45:05.415112Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:05.424585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dcd71243-9b78fdac-862b8974-e2cd7c92_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:05.424948Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:05.424972Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 445
2025-10-09T21:45:05.425274Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 445 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:05.425323Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 445 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:05.425328Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=445 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:05.425332Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:05.426019Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 445 sessionId:  describe result for acl check
2025-10-09T21:45:05.426069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:05.426075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:05.426077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:05.426084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341320670138256:5483] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:05.426620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341320670138256:5483] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:05.429580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341320670138256:5483] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:05.434296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341320670138256:5483] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:05.434314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341320670138256:5483] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:05.434318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341320670138256:5483] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:05.434349Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 445 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:05.443312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:05.453684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:05.453976Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:05.453989Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:05.454003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:05.454086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:05.454135Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341320308326459:5035]
2025-10-09T21:45:05.454828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:05.455300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:05.455373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:05.455713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:05.455793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:05.455802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:05.455804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:05.455805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:05.455806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:05.455807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:05.455808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.455809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.455811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.455818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:05.455829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:05.455832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:05.455836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:05.455840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:05.455844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:05.455848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.455851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.455855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.455863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:05.455871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:05.455878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:05.455891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:05.455896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:05.455901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:05.455904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:05.455909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:05.455913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:05.455918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:05.455922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:05.455926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:05.455931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.455935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.455939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.455944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.455949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.455953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.455998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:05.473491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:05.476136Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 444
2025-10-09T21:45:05.476173Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 444
2025-10-09T21:45:05.474682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:05.474702Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:05.474705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:05.474711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:05.474718Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 444 [2:7559341320308326459:5035]
2025-10-09T21:45:05.474735Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:05.474756Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:05.474759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.474763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.474766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.474787Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:05.474793Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:05.474803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:05.474872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.474875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.474878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.474881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:05.474961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:05.474965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.474980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.474993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.475010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:05.475031Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341320308326454:5035]
2025-10-09T21:45:05.475034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341320308326454:5035]
2025-10-09T21:45:05.475037Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341320308326454:5035]
2025-10-09T21:45:05.475049Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:05.475065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:05.475084Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:05.475088Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:05.475132Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341320308326454:5035]
2025-10-09T21:45:05.475135Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:05.475579Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:05.475582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:05.475585Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:05.475593Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:05.475933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341320670138423:5483], now have 1 active actors on pipe
2025-10-09T21:45:05.475941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341320670138233:2432], now have 1 active actors on pipe
2025-10-09T21:45:05.476555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:05.476563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:05.476607Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7ae40fd0-6455d549-9cc67232-2d9483f8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:05.476619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.476623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.476626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:05.476641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.476655Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:05.476686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:05.486377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:05.486404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:05.486453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:05.487250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dcd71243-9b78fdac-862b8974-e2cd7c92_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7ae40fd0-6455d549-9cc67232-2d9483f8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:05.486764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 445 partition: 0 MaxSeqNo: 25 sessionId: src_id|7ae40fd0-6455d549-9cc67232-2d9483f8_0
2025-10-09T21:45:05.491858Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ae40fd0-6455d549-9cc67232-2d9483f8_0] Write session: try to update token
2025-10-09T21:45:05.491880Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ae40fd0-6455d549-9cc67232-2d9483f8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:05.500467Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:05.500489Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:05.500492Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:05.500494Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:05.500504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341320308326454:5035]
2025-10-09T21:45:05.500508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341320308326454:5035]
2025-10-09T21:45:05.500511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341320308326454:5035]
2025-10-09T21:45:05.500566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:05.500570Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:05.500574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.500577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:05.500579Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:05.500584Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:05.500725Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:05.514475Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341320670138506:5210]
2025-10-09T21:45:05.521556Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 444, pipe: [1:7559341320670138506:5210]
2025-10-09T21:45:05.524971Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 445 sessionId: src_id|7ae40fd0-6455d549-9cc67232-2d9483f8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:05.529679Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:05.674099Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:05.674693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341320670138506:5210], now have 1 active actors on pipe
2025-10-09T21:45:05.675040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:05.675054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:05.675364Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:05.675394Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341320670138506:5210]
2025-10-09T21:45:05.675417Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:05.675421Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:05.675425Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:05.675502Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 445 reason: error at write: Disconnected sessionId: src_id|7ae40fd0-6455d549-9cc67232-2d9483f8_0
2025-10-09T21:45:05.675688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 445 sessionId: src_id|7ae40fd0-6455d549-9cc67232-2d9483f8_0 is DEAD
2025-10-09T21:45:05.676503Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ae40fd0-6455d549-9cc67232-2d9483f8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:05.676514Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ae40fd0-6455d549-9cc67232-2d9483f8_0] Write session will restart in 0.006699s
2025-10-09T21:45:05.676541Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ae40fd0-6455d549-9cc67232-2d9483f8_0] Write session: Do CDS request
2025-10-09T21:45:05.676548Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ae40fd0-6455d549-9cc67232-2d9483f8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:05.684596Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ae40fd0-6455d549-9cc67232-2d9483f8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:05.686500Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:05.686521Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 446
2025-10-09T21:45:05.686758Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 446 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:05.686812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 446 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:05.686826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=446 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:05.686830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:05.687659Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 446 sessionId:  describe result for acl check
2025-10-09T21:45:05.687702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:05.687704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:05.687706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:05.687716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341320670138532:5488] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:05.688378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341320670138532:5488] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:05.692168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341320670138532:5488] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:05.696907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341320670138532:5488] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:05.696927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341320670138532:5488] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:05.696932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341320670138532:5488] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:05.696943Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 446 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:05.710224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:05.711570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:05.711826Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:05.711838Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:05.711853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:05.711915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:05.711964Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341320308326725:5037]
2025-10-09T21:45:05.712566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:05.712952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:05.713030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:05.713415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:05.713489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:05.713491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:05.713492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:05.713493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:05.713494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:05.713496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:05.713497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.713498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.713499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.713506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:05.713517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:05.713521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:05.713525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:05.713529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:05.713533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:05.713536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.713539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.713543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.713551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:05.713560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:05.713566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:05.713573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:05.713580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:05.713585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:05.713589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:05.713596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:05.713601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:05.713607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:05.713611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:05.713617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:05.713623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.713629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.713634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.713639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.713644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.713650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.713681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:05.723684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:05.725342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:05.725364Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:05.725366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:05.725371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:05.725382Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 445 [2:7559341320308326725:5037]
2025-10-09T21:45:05.725398Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:05.725421Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:05.725424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.725428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.725430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.725451Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:05.725466Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:05.725473Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:05.725547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.725550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.725552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.725555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:05.725636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:05.725645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.725660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.725674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.725693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:05.725720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341320308326720:5037]
2025-10-09T21:45:05.725723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341320308326720:5037]
2025-10-09T21:45:05.725725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341320308326720:5037]
2025-10-09T21:45:05.725736Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:05.725751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:05.725760Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:05.725763Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:05.725765Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:05.725767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:05.725771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:05.725782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341320670138680:5488], now have 1 active actors on pipe
2025-10-09T21:45:05.725826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341320670138511:2432], now have 1 active actors on pipe
2025-10-09T21:45:05.725835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341320308326720:5037]
2025-10-09T21:45:05.725843Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:05.726239Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:05.728715Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 445
2025-10-09T21:45:05.728754Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 445
2025-10-09T21:45:05.729125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:05.729143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:05.729178Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3a7e116c-37c90a7e-bf49bd6f-1fe588a5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:05.729204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.729211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.729215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:05.729231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.729250Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:05.729280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:05.732777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:05.732800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:05.732846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:05.733134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 446 partition: 0 MaxSeqNo: 25 sessionId: src_id|3a7e116c-37c90a7e-bf49bd6f-1fe588a5_0
2025-10-09T21:45:05.733620Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ae40fd0-6455d549-9cc67232-2d9483f8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3a7e116c-37c90a7e-bf49bd6f-1fe588a5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:05.736373Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a7e116c-37c90a7e-bf49bd6f-1fe588a5_0] Write session: try to update token
2025-10-09T21:45:05.736396Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a7e116c-37c90a7e-bf49bd6f-1fe588a5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:05.744000Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:05.744020Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:05.744023Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:05.744025Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:05.744036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341320308326720:5037]
2025-10-09T21:45:05.744040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341320308326720:5037]
2025-10-09T21:45:05.744045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341320308326720:5037]
2025-10-09T21:45:05.744092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:05.744101Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:05.744108Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:05.744108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.744111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:05.744115Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:05.744258Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:05.760409Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 446 sessionId: src_id|3a7e116c-37c90a7e-bf49bd6f-1fe588a5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:05.763077Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:05.775590Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341320670138786:5210]
2025-10-09T21:45:05.775936Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 445, pipe: [1:7559341320670138786:5210]
2025-10-09T21:45:05.907843Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:05.908534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:05.908555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:05.908579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341320670138786:5210], now have 1 active actors on pipe
2025-10-09T21:45:05.909132Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:05.909086Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:05.909103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:05.909161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 446 reason: error at write: Disconnected sessionId: src_id|3a7e116c-37c90a7e-bf49bd6f-1fe588a5_0
2025-10-09T21:45:05.909380Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 446 sessionId: src_id|3a7e116c-37c90a7e-bf49bd6f-1fe588a5_0 is DEAD
2025-10-09T21:45:05.909511Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:05.909547Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341320670138786:5210]
2025-10-09T21:45:05.910350Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a7e116c-37c90a7e-bf49bd6f-1fe588a5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:05.910363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a7e116c-37c90a7e-bf49bd6f-1fe588a5_0] Write session will restart in 0.009697s
2025-10-09T21:45:05.910391Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a7e116c-37c90a7e-bf49bd6f-1fe588a5_0] Write session: Do CDS request
2025-10-09T21:45:05.910398Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a7e116c-37c90a7e-bf49bd6f-1fe588a5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:05.921579Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a7e116c-37c90a7e-bf49bd6f-1fe588a5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:05.921936Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:05.921954Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 447
2025-10-09T21:45:05.923089Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 447 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:05.923147Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 447 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:05.923153Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=447 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:05.923157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:05.923791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 447 sessionId:  describe result for acl check
2025-10-09T21:45:05.923833Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:05.923835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:05.923837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:05.923844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341320670138809:5492] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:05.924421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341320670138809:5492] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:05.933700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341320670138809:5492] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:05.942234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341320670138809:5492] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:05.942266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341320670138809:5492] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:05.942270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341320670138809:5492] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:05.942280Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 447 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:05.962573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:05.963422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:05.963637Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:05.963641Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:05.963653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:05.963711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:05.963752Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341320308326994:5039]
2025-10-09T21:45:05.964353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:05.965608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:05.965674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:05.966384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:05.966476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:05.966482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:05.966483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:05.966484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:05.966485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:05.966486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:05.966487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.966489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.966490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.966498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:05.966510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:05.966513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:05.966517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:05.966521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:05.966526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:05.966530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.966533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.966537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.966545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:05.966554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:05.966565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:05.966571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:05.966576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:05.966585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:05.966590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:05.966596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:05.966601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:05.966606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:05.966611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:05.966616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:05.966624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.966630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.966642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.966647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.966651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.966656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.966689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:05.975378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:05.977093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:05.977115Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:05.977119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:05.977124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:05.977132Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 446 [2:7559341320308326994:5039]
2025-10-09T21:45:05.977149Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:05.977171Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:05.977180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.977184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.977186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.977206Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:05.977212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:05.977225Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:05.977297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.977308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.977310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.977313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:05.977390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:05.977399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:05.977414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:05.977424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:05.977440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:05.977466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341320308326989:5039]
2025-10-09T21:45:05.977476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341320308326989:5039]
2025-10-09T21:45:05.977478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341320308326989:5039]
2025-10-09T21:45:05.977535Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:05.977542Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:05.977544Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:05.977930Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 446
2025-10-09T21:45:05.977972Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 446
2025-10-09T21:45:05.978169Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:05.978204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:05.978224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:05.978228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:05.978242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341320670138981:5492], now have 1 active actors on pipe
2025-10-09T21:45:05.978264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341320670138791:2432], now have 1 active actors on pipe
2025-10-09T21:45:05.978283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:05.978289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:05.978360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341320308326989:5039]
2025-10-09T21:45:05.978371Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:05.978391Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|da770a3d-676c400b-afd1bf5c-85f02cb8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:05.978401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.978405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.978408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:05.978421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.978443Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:05.978492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:05.978624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:05.978864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:05.978883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:05.978906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:05.979113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 447 partition: 0 MaxSeqNo: 25 sessionId: src_id|da770a3d-676c400b-afd1bf5c-85f02cb8_0
2025-10-09T21:45:05.979663Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a7e116c-37c90a7e-bf49bd6f-1fe588a5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|da770a3d-676c400b-afd1bf5c-85f02cb8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:05.982323Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da770a3d-676c400b-afd1bf5c-85f02cb8_0] Write session: try to update token
2025-10-09T21:45:05.982345Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da770a3d-676c400b-afd1bf5c-85f02cb8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:06.010053Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341324965106369:5210]
2025-10-09T21:45:06.010684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341324965106369:5210], now have 1 active actors on pipe
2025-10-09T21:45:06.010863Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 446, pipe: [1:7559341324965106369:5210]
2025-10-09T21:45:06.012042Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 447 sessionId: src_id|da770a3d-676c400b-afd1bf5c-85f02cb8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:06.014732Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:06.015195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:06.015224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:06.015270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:06.019447Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:06.019471Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:06.019474Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:06.019475Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:06.019486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341320308326989:5039]
2025-10-09T21:45:06.019491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341320308326989:5039]
2025-10-09T21:45:06.019495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341320308326989:5039]
2025-10-09T21:45:06.019549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:06.019558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.019567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:06.019719Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:06.015496Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:06.022166Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:06.022186Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:06.022195Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:06.199213Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:06.207855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:06.207883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:06.208050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:06.208116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:06.208188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:06.208249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:06.208294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:06.208300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:06.208368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:06.208429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:06.208494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:06.208584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:06.208627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:06.208630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:06.208697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:06.208754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:06.208813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:06.208870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:06.208911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:06.208914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:06.208986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:06.209048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:06.209110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:06.209170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:06.209214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:06.209217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:06.209286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:06.209344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:06.209403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:06.209463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:06.209504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:06.209507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:06.209573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:06.209630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:06.209690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:06.209752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:06.209792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:06.209795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:06.209858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:06.209913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:06.209972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:06.210239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:06.210303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:06.210306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:06.210367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:06.210431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:06.210486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:06.210544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:06.210603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:06.210606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:06.210669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:06.210723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:06.210781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:06.210840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:06.210881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:06.210885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:06.210950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:06.211005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:06.211064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:06.211123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:06.211165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:06.211168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:06.212563Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:06.212579Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:06.212619Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 447 reason: error at write: Disconnected sessionId: src_id|da770a3d-676c400b-afd1bf5c-85f02cb8_0
2025-10-09T21:45:06.212842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 447 sessionId: src_id|da770a3d-676c400b-afd1bf5c-85f02cb8_0 is DEAD
2025-10-09T21:45:06.213004Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da770a3d-676c400b-afd1bf5c-85f02cb8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:06.213017Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da770a3d-676c400b-afd1bf5c-85f02cb8_0] Write session will restart in 0.008049s
2025-10-09T21:45:06.213044Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da770a3d-676c400b-afd1bf5c-85f02cb8_0] Write session: Do CDS request
2025-10-09T21:45:06.213050Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da770a3d-676c400b-afd1bf5c-85f02cb8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:06.212709Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:06.213235Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:06.213280Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341324965106369:5210]
2025-10-09T21:45:06.222739Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da770a3d-676c400b-afd1bf5c-85f02cb8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:06.222811Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:06.222827Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 448
2025-10-09T21:45:06.223006Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 448 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:06.223052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 448 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:06.223057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=448 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:06.223061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:06.223512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 448 sessionId:  describe result for acl check
2025-10-09T21:45:06.223549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:06.223551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:06.223553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:06.223559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341324965106414:5503] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:06.224092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341324965106414:5503] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:06.227570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341324965106414:5503] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:06.235823Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341324965106414:5503] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:06.235852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341324965106414:5503] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:06.235856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341324965106414:5503] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:06.235867Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 448 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:06.238106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:06.241558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:06.241817Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:06.241831Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:06.241844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:06.241919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:06.241966Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341324603294591:5052]
2025-10-09T21:45:06.242630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:06.243004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:06.243050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:06.243426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:06.243488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:06.243495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:06.243496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:06.243497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:06.243498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:06.243499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:06.243501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.243502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.243503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.243509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:06.243518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:06.243523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:06.243527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:06.243531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:06.243536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:06.243539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.243543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.243547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.243554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:06.243562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:06.243569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:06.243574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:06.243578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:06.243584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:06.243588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:06.243594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:06.243598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:06.243604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:06.243608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:06.243613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:06.243619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.243625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.243629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.243637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.243641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.243647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.243672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:06.251149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:06.252620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:06.252638Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:06.252642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:06.252647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:06.252655Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 447 [2:7559341324603294591:5052]
2025-10-09T21:45:06.252670Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:06.252690Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:06.252694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:06.252697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:06.252700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:06.252720Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:06.252726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:06.252741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:06.252812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.252820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.252822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.252825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:06.252906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:06.252913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.252928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.252944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.252962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:06.252984Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341324603294585:5052]
2025-10-09T21:45:06.252987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341324603294585:5052]
2025-10-09T21:45:06.252989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341324603294585:5052]
2025-10-09T21:45:06.253000Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:06.253014Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:06.253033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:06.253040Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:06.253062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:06.253073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:06.253075Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:06.253084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341324603294585:5052]
2025-10-09T21:45:06.253092Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:06.253094Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:06.253103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341324965106649:5503], now have 1 active actors on pipe
2025-10-09T21:45:06.253114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341324965106391:2432], now have 1 active actors on pipe
2025-10-09T21:45:06.253421Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 447
2025-10-09T21:45:06.253455Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 447
2025-10-09T21:45:06.254331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:06.254349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:06.254391Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5313878d-bbdf3b54-f37ae7f-c4708e6a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:06.254411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:06.254418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:06.254420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:06.254433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:06.254457Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:06.254484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:06.255512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:06.255526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:06.255553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:06.255694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 448 partition: 0 MaxSeqNo: 25 sessionId: src_id|5313878d-bbdf3b54-f37ae7f-c4708e6a_0
2025-10-09T21:45:06.256166Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da770a3d-676c400b-afd1bf5c-85f02cb8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5313878d-bbdf3b54-f37ae7f-c4708e6a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:06.258882Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5313878d-bbdf3b54-f37ae7f-c4708e6a_0] Write session: try to update token
2025-10-09T21:45:06.258907Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5313878d-bbdf3b54-f37ae7f-c4708e6a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:06.268884Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:06.268902Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:06.268913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341324603294585:5052]
2025-10-09T21:45:06.268965Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:06.268999Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:06.269159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:06.269321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:06.269457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:06.269611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:06.269851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:06.269905Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:06.285299Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:06.285320Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:06.285323Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:06.285325Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:06.285336Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341324603294585:5052]
2025-10-09T21:45:06.285340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341324603294585:5052]
2025-10-09T21:45:06.285345Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341324603294585:5052]
2025-10-09T21:45:06.285398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:06.285406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.285409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:06.285418Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:06.285424Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:06.285428Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:06.285561Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:06.298424Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 448 sessionId: src_id|5313878d-bbdf3b54-f37ae7f-c4708e6a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:06.301131Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:06.313224Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341324965106672:5210]
2025-10-09T21:45:06.313698Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 447, pipe: [1:7559341324965106672:5210]
2025-10-09T21:45:06.465604Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:06.466000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:06.466020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:06.466057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341324965106672:5210], now have 1 active actors on pipe
2025-10-09T21:45:06.466315Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:06.466420Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:06.466446Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:06.466512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 448 reason: error at write: Disconnected sessionId: src_id|5313878d-bbdf3b54-f37ae7f-c4708e6a_0
2025-10-09T21:45:06.466742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 448 sessionId: src_id|5313878d-bbdf3b54-f37ae7f-c4708e6a_0 is DEAD
2025-10-09T21:45:06.466763Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:06.466812Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341324965106672:5210]
2025-10-09T21:45:06.470222Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5313878d-bbdf3b54-f37ae7f-c4708e6a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:06.470235Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5313878d-bbdf3b54-f37ae7f-c4708e6a_0] Write session will restart in 0.005239s
2025-10-09T21:45:06.470273Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5313878d-bbdf3b54-f37ae7f-c4708e6a_0] Write session: Do CDS request
2025-10-09T21:45:06.470280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5313878d-bbdf3b54-f37ae7f-c4708e6a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:06.476626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5313878d-bbdf3b54-f37ae7f-c4708e6a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:06.476871Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:06.476899Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 449
2025-10-09T21:45:06.477036Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 449 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:06.477082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 449 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:06.477095Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=449 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:06.477099Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:06.477674Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 449 sessionId:  describe result for acl check
2025-10-09T21:45:06.477714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:06.477721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:06.477723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:06.477730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341324965106689:5508] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:06.478321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341324965106689:5508] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:06.483134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341324965106689:5508] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:06.487341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341324965106689:5508] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:06.487358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341324965106689:5508] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:06.487364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341324965106689:5508] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:06.487374Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 449 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:06.496313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:06.499529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:06.499870Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:06.499881Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:06.499896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:06.499976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:06.500033Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341324603294872:5057]
2025-10-09T21:45:06.500674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:06.501160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:06.501229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:06.501578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:06.501632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:06.501641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:06.501642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:06.501644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:06.501645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:06.501646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:06.501648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.501649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.501650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.501656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:06.501667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:06.501671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:06.501675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:06.501679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:06.501683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:06.501687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.501691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.501694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.501701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:06.501708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:06.501714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:06.501719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:06.501724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:06.501731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:06.501736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:06.501741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:06.501746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:06.501751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:06.501755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:06.501760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:06.501766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.501770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.501774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.501779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.501783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.501789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.501820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:06.509137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:06.510310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:06.510332Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:06.510335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:06.510339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:06.510347Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 448 [2:7559341324603294872:5057]
2025-10-09T21:45:06.510363Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:06.510386Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:06.510390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:06.510395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:06.510398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:06.510419Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:06.510431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:06.510440Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:06.510517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.510527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.510529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.510532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:06.510621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:06.510629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.510646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.510666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.510712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:06.510743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341324965106799:5508], now have 1 active actors on pipe
2025-10-09T21:45:06.510752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341324965106675:2432], now have 1 active actors on pipe
2025-10-09T21:45:06.510774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341324603294867:5057]
2025-10-09T21:45:06.510781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341324603294867:5057]
2025-10-09T21:45:06.510784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341324603294867:5057]
2025-10-09T21:45:06.510798Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:06.510813Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:06.510817Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:06.510819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:06.510820Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:06.510804Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 448
2025-10-09T21:45:06.510839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:06.510846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:06.510857Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 448
2025-10-09T21:45:06.510890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341324603294867:5057]
2025-10-09T21:45:06.510912Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:06.510934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:06.510946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:06.510962Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:06.511007Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d1dc2f7-94cab9c2-99e594ee-b14e0a15_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:06.511025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:06.511029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:06.511032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:06.511045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:06.511057Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:06.511085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:06.511375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:06.511386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:06.511404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:06.511510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 449 partition: 0 MaxSeqNo: 25 sessionId: src_id|d1dc2f7-94cab9c2-99e594ee-b14e0a15_0
2025-10-09T21:45:06.511955Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5313878d-bbdf3b54-f37ae7f-c4708e6a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d1dc2f7-94cab9c2-99e594ee-b14e0a15_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:06.514670Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1dc2f7-94cab9c2-99e594ee-b14e0a15_0] Write session: try to update token
2025-10-09T21:45:06.514691Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1dc2f7-94cab9c2-99e594ee-b14e0a15_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:06.528815Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:06.528841Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:06.528844Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:06.528846Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:06.528860Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341324603294867:5057]
2025-10-09T21:45:06.528865Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341324603294867:5057]
2025-10-09T21:45:06.528883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341324603294867:5057]
2025-10-09T21:45:06.528934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:06.528952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.528951Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:06.528957Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:06.528962Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:06.528962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:06.529115Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:06.534274Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 449 sessionId: src_id|d1dc2f7-94cab9c2-99e594ee-b14e0a15_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:06.537117Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:06.567202Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341324965106954:5210]
2025-10-09T21:45:06.567526Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 448, pipe: [1:7559341324965106954:5210]
2025-10-09T21:45:06.687609Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:06.688771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:06.688784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:06.688831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341324965106954:5210], now have 1 active actors on pipe
2025-10-09T21:45:06.689381Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:06.689397Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:06.689957Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1dc2f7-94cab9c2-99e594ee-b14e0a15_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:06.689968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1dc2f7-94cab9c2-99e594ee-b14e0a15_0] Write session will restart in 0.006008s
2025-10-09T21:45:06.689993Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1dc2f7-94cab9c2-99e594ee-b14e0a15_0] Write session: Do CDS request
2025-10-09T21:45:06.690000Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1dc2f7-94cab9c2-99e594ee-b14e0a15_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:06.689457Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 449 reason: error at write: Disconnected sessionId: src_id|d1dc2f7-94cab9c2-99e594ee-b14e0a15_0
2025-10-09T21:45:06.689650Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 449 sessionId: src_id|d1dc2f7-94cab9c2-99e594ee-b14e0a15_0 is DEAD
2025-10-09T21:45:06.689804Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341324965106954:5210]
2025-10-09T21:45:06.689818Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:06.690433Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:06.702288Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1dc2f7-94cab9c2-99e594ee-b14e0a15_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:06.702721Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:06.702751Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 450
2025-10-09T21:45:06.703014Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 450 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:06.703052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 450 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:06.703058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=450 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:06.703061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:06.703866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 450 sessionId:  describe result for acl check
2025-10-09T21:45:06.703912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:06.703914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:06.703915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:06.703922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341324965106994:5515] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:06.704518Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341324965106994:5515] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:06.715626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341324965106994:5515] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:06.721873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341324965106994:5515] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:06.721892Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341324965106994:5515] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:06.721896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341324965106994:5515] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:06.721908Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 450 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:06.725728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:06.729494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:06.729757Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:06.729768Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:06.729782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:06.729860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:06.729908Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341324603295143:5059]
2025-10-09T21:45:06.730631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:06.731060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:06.731133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:06.731534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:06.731603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:06.731611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:06.731612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:06.731613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:06.731615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:06.731616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:06.731617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.731618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.731619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.731626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:06.731638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:06.731641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:06.731646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:06.731650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:06.731654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:06.731658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.731662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.731666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.731674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:06.731684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:06.731690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:06.731695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:06.731700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:06.731706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:06.731711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:06.731717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:06.731722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:06.731727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:06.731733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:06.731738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:06.731749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.731754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.731758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.731763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.731767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.731772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.731801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:06.742428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:06.744309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:06.744338Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:06.744340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:06.744346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:06.744355Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 449 [2:7559341324603295143:5059]
2025-10-09T21:45:06.744370Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:06.744395Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:06.744398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:06.744400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:06.744403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:06.744423Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:06.744433Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:06.744440Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:06.744523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.744531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.744533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.744537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:06.744619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:06.744628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.744642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.744652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.744667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:06.744691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341324603295138:5059]
2025-10-09T21:45:06.744694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341324603295138:5059]
2025-10-09T21:45:06.744696Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341324603295138:5059]
2025-10-09T21:45:06.744712Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:06.744728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341324965106960:2432], now have 1 active actors on pipe
2025-10-09T21:45:06.744732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341324965107211:5515], now have 1 active actors on pipe
2025-10-09T21:45:06.744742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:06.744757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:06.744766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:06.744769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:06.744772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:06.744774Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:06.744809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341324603295138:5059]
2025-10-09T21:45:06.744818Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:06.744835Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:06.744845Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 449
2025-10-09T21:45:06.744896Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 449
2025-10-09T21:45:06.745113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:06.745128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:06.745162Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a266ab2a-161b5374-71e785b7-8d1da4be_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:06.745172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:06.745176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:06.745179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:06.745202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:06.745215Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:06.745244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:06.746318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:06.746346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:06.746374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:06.746579Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 450 partition: 0 MaxSeqNo: 25 sessionId: src_id|a266ab2a-161b5374-71e785b7-8d1da4be_0
2025-10-09T21:45:06.746996Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1dc2f7-94cab9c2-99e594ee-b14e0a15_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a266ab2a-161b5374-71e785b7-8d1da4be_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:06.749489Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a266ab2a-161b5374-71e785b7-8d1da4be_0] Write session: try to update token
2025-10-09T21:45:06.749507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a266ab2a-161b5374-71e785b7-8d1da4be_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:06.761832Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:06.761857Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:06.761860Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:06.761862Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:06.761872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341324603295138:5059]
2025-10-09T21:45:06.761877Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341324603295138:5059]
2025-10-09T21:45:06.761882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341324603295138:5059]
2025-10-09T21:45:06.764564Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 450 sessionId: src_id|a266ab2a-161b5374-71e785b7-8d1da4be_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:06.761934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:06.761939Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:06.761946Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:06.761950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.761951Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:06.761954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:06.762120Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:06.767726Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:06.789904Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341324965107238:5210]
2025-10-09T21:45:06.790270Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 449, pipe: [1:7559341324965107238:5210]
2025-10-09T21:45:06.922373Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:06.922929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:06.922950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:06.923004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341324965107238:5210], now have 1 active actors on pipe
2025-10-09T21:45:06.923545Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341324965107238:5210]
2025-10-09T21:45:06.923582Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:06.923615Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:06.923341Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:06.923700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 450 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a266ab2a-161b5374-71e785b7-8d1da4be_0
2025-10-09T21:45:06.923903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 450 sessionId: src_id|a266ab2a-161b5374-71e785b7-8d1da4be_0 is DEAD
2025-10-09T21:45:06.924059Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a266ab2a-161b5374-71e785b7-8d1da4be_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:06.924069Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a266ab2a-161b5374-71e785b7-8d1da4be_0] Write session will restart in 0.007181s
2025-10-09T21:45:06.924096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a266ab2a-161b5374-71e785b7-8d1da4be_0] Write session: Do CDS request
2025-10-09T21:45:06.924103Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a266ab2a-161b5374-71e785b7-8d1da4be_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:06.932612Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a266ab2a-161b5374-71e785b7-8d1da4be_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:06.932716Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:06.932734Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 451
2025-10-09T21:45:06.932881Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 451 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:06.932931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 451 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:06.932942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=451 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:06.932945Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:06.934121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 451 sessionId:  describe result for acl check
2025-10-09T21:45:06.934180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:06.934186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:06.934189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:06.934194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341324965107260:5519] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:06.934918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341324965107260:5519] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:06.938900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341324965107260:5519] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:06.944827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341324965107260:5519] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:06.944844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341324965107260:5519] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:06.944855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341324965107260:5519] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:06.944865Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 451 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:06.951201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:06.956080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:06.956366Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:06.956377Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:06.956391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:06.956452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:06.956499Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341324603295416:5061]
2025-10-09T21:45:06.957153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:06.957521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:06.957581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:06.957898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:06.957951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:06.957958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:06.957959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:06.957960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:06.957961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:06.957963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:06.957964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.957965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.957966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.957971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:06.957982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:06.957986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:06.957989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:06.957993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:06.957997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:06.958000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.958004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.958009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.958016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:06.958025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:06.958049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:06.958054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:06.958059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:06.958064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:06.958068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:06.958073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:06.958077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:06.958081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:06.958085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:06.958090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:06.958094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.958099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.958103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.958107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.958111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.958117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.958147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:06.965425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:06.966658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:06.967635Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 450
2025-10-09T21:45:06.967646Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 450
2025-10-09T21:45:06.966675Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:06.966678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:06.966683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:06.966691Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 450 [2:7559341324603295416:5061]
2025-10-09T21:45:06.966706Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:06.966726Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:06.966729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:06.966733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:06.966736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:06.966755Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:06.966761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:06.966770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:06.966851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.966854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.966856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.966859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:06.966940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:06.966942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.966956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:06.966965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:06.966979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:06.967001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341324603295411:5061]
2025-10-09T21:45:06.967003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341324603295411:5061]
2025-10-09T21:45:06.967005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341324603295411:5061]
2025-10-09T21:45:06.967022Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:06.967037Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:06.967051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:06.967055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:06.967097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341324603295411:5061]
2025-10-09T21:45:06.967100Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:06.967321Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:06.967324Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:06.967326Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:06.967332Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:06.967556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341324965107426:5519], now have 1 active actors on pipe
2025-10-09T21:45:06.967563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341324965107246:2432], now have 1 active actors on pipe
2025-10-09T21:45:06.967735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:06.967742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:06.967773Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4a02c70-780aa9da-89e8477c-f390f470_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:06.967782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:06.967786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:06.967789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:06.967801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:06.967815Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:06.967830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:06.968197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:06.968209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:06.968228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:06.968390Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 451 partition: 0 MaxSeqNo: 25 sessionId: src_id|4a02c70-780aa9da-89e8477c-f390f470_0
2025-10-09T21:45:06.968806Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a266ab2a-161b5374-71e785b7-8d1da4be_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4a02c70-780aa9da-89e8477c-f390f470_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:06.971414Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a02c70-780aa9da-89e8477c-f390f470_0] Write session: try to update token
2025-10-09T21:45:06.971435Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a02c70-780aa9da-89e8477c-f390f470_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:06.983571Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:06.983595Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:06.983598Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:06.983600Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:06.983611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341324603295411:5061]
2025-10-09T21:45:06.983615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341324603295411:5061]
2025-10-09T21:45:06.983619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341324603295411:5061]
2025-10-09T21:45:06.983664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:06.983676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:06.983677Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:06.983679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:06.983682Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:06.983689Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:06.983840Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:06.986389Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 451 sessionId: src_id|4a02c70-780aa9da-89e8477c-f390f470_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:06.989069Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:07.023843Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341329260074827:5210]
2025-10-09T21:45:07.024163Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 450, pipe: [1:7559341329260074827:5210]
2025-10-09T21:45:07.041946Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: CLIENT_DEADLINE_EXCEEDED. Description: [ { <main>: Error: GRpc error: (4): Deadline Exceeded } { <main>: Error: Grpc error response on endpoint localhost:14704 } ]
2025-10-09T21:45:07.147796Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:07.148617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:07.148630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:07.148679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341329260074827:5210], now have 1 active actors on pipe
2025-10-09T21:45:07.149233Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:07.149269Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:07.149278Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:07.149284Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:07.149322Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341329260074827:5210]
2025-10-09T21:45:07.149356Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 451 reason: error at write: Disconnected sessionId: src_id|4a02c70-780aa9da-89e8477c-f390f470_0
2025-10-09T21:45:07.149717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 451 sessionId: src_id|4a02c70-780aa9da-89e8477c-f390f470_0 is DEAD
2025-10-09T21:45:07.149869Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a02c70-780aa9da-89e8477c-f390f470_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:07.149881Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a02c70-780aa9da-89e8477c-f390f470_0] Write session will restart in 0.007380s
2025-10-09T21:45:07.149920Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a02c70-780aa9da-89e8477c-f390f470_0] Write session: Do CDS request
2025-10-09T21:45:07.149926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a02c70-780aa9da-89e8477c-f390f470_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:07.158702Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a02c70-780aa9da-89e8477c-f390f470_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:07.159248Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:07.159268Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 452
2025-10-09T21:45:07.159448Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 452 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:07.159502Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 452 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:07.159513Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=452 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:07.159518Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:07.160141Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 452 sessionId:  describe result for acl check
2025-10-09T21:45:07.160179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:07.160182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:07.160184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:07.160190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341329260074844:5524] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:07.160819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341329260074844:5524] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:07.166049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341329260074844:5524] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:07.176774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341329260074844:5524] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:07.176797Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341329260074844:5524] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:07.176801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341329260074844:5524] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:07.176812Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 452 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:07.208454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:07.212859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:07.213157Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:07.213162Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:07.213177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:07.213244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:07.213281Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341328898263000:5064]
2025-10-09T21:45:07.213896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:07.215003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:07.215078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:07.215464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:07.215520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:07.215521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:07.215523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:07.215524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:07.215525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:07.215527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:07.215528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.215529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.215530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.215537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:07.215547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:07.215552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:07.215556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:07.215560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:07.215564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:07.215568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.215572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.215576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.215582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:07.215590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:07.215597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:07.215604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:07.215608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:07.215613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:07.215617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:07.216095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:07.216107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:07.216112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:07.216117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:07.216123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:07.216128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.216134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.216138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.216152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.216157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.216162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.216195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:07.224473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:07.226134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:07.226148Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:07.226151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:07.226157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:07.226164Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 451 [2:7559341328898263000:5064]
2025-10-09T21:45:07.226178Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:07.226197Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:07.226200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:07.226204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:07.226206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:07.226224Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:07.226230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:07.226237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:07.226328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.226331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.226332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.226335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:07.226929Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 451
2025-10-09T21:45:07.226974Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 451
2025-10-09T21:45:07.227295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:07.227305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.227322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.227357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.227372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:07.227383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341329260074829:2432], now have 1 active actors on pipe
2025-10-09T21:45:07.227386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341329260074963:5524], now have 1 active actors on pipe
2025-10-09T21:45:07.227406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:07.227411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:07.227455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341328898262994:5064]
2025-10-09T21:45:07.227457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341328898262994:5064]
2025-10-09T21:45:07.227459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341328898262994:5064]
2025-10-09T21:45:07.227472Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:07.227500Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|413a1d4f-2d3292db-38fd0af1-26af853c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:07.227510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:07.227515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:07.227517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:07.227528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:07.227542Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:07.227556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:07.227580Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:07.227584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:07.227635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:07.227651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341328898262994:5064]
2025-10-09T21:45:07.227655Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:07.227826Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:07.227833Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:07.227835Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:07.227840Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:07.228147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:07.228154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:07.228173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:07.228356Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 452 partition: 0 MaxSeqNo: 25 sessionId: src_id|413a1d4f-2d3292db-38fd0af1-26af853c_0
2025-10-09T21:45:07.228792Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a02c70-780aa9da-89e8477c-f390f470_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|413a1d4f-2d3292db-38fd0af1-26af853c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:07.231454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|413a1d4f-2d3292db-38fd0af1-26af853c_0] Write session: try to update token
2025-10-09T21:45:07.231479Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|413a1d4f-2d3292db-38fd0af1-26af853c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:07.247101Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341329260075108:5210]
2025-10-09T21:45:07.247624Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 451, pipe: [1:7559341329260075108:5210]
2025-10-09T21:45:07.248577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341329260075108:5210], now have 1 active actors on pipe
2025-10-09T21:45:07.248607Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:07.248611Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:07.248613Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:07.248614Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:07.248621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341328898262994:5064]
2025-10-09T21:45:07.248626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341328898262994:5064]
2025-10-09T21:45:07.248629Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341328898262994:5064]
2025-10-09T21:45:07.248681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:07.248688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.248697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:07.248884Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:07.250090Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:07.250111Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:07.250116Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:07.271817Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 452 sessionId: src_id|413a1d4f-2d3292db-38fd0af1-26af853c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:07.274427Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:07.418375Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:07.419122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:07.419142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:07.419960Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:07.422554Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341329260075108:5210]
2025-10-09T21:45:07.422584Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:07.422620Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:07.422735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 452 reason: status is not ok: tablet will be restarted right now sessionId: src_id|413a1d4f-2d3292db-38fd0af1-26af853c_0
2025-10-09T21:45:07.422920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 452 sessionId: src_id|413a1d4f-2d3292db-38fd0af1-26af853c_0 is DEAD
2025-10-09T21:45:07.424630Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|413a1d4f-2d3292db-38fd0af1-26af853c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:07.424640Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|413a1d4f-2d3292db-38fd0af1-26af853c_0] Write session will restart in 0.006132s
2025-10-09T21:45:07.424666Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|413a1d4f-2d3292db-38fd0af1-26af853c_0] Write session: Do CDS request
2025-10-09T21:45:07.424672Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|413a1d4f-2d3292db-38fd0af1-26af853c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:07.431511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|413a1d4f-2d3292db-38fd0af1-26af853c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:07.434462Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:07.434481Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 453
2025-10-09T21:45:07.434633Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 453 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:07.434670Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 453 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:07.434675Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=453 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:07.434680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:07.435378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 453 sessionId:  describe result for acl check
2025-10-09T21:45:07.435429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:07.435431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:07.435433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:07.435439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341329260075152:5535] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:07.436006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341329260075152:5535] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:07.441045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341329260075152:5535] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:07.446300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341329260075152:5535] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:07.446319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341329260075152:5535] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:07.446323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341329260075152:5535] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:07.446333Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 453 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:07.457095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:07.460666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:07.460907Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:07.460918Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:07.460932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:07.460987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:07.461032Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341328898263302:5079]
2025-10-09T21:45:07.461630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:07.462102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:07.462162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:07.462501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:07.462557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:07.462563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:07.462565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:07.462566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:07.462568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:07.462569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:07.462570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.462571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.462573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.462578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:07.462589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:07.462592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:07.462596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:07.462599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:07.462603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:07.462606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.462610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.462613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.462621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:07.462628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:07.462635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:07.462640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:07.462644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:07.462650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:07.462654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:07.462659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:07.462664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:07.462669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:07.462674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:07.462680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:07.462685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.462690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.462694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.462699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.462711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.462716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.462750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:07.469839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:07.471335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:07.471354Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:07.471357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:07.471363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:07.471371Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 452 [2:7559341328898263302:5079]
2025-10-09T21:45:07.471386Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:07.471407Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:07.471410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:07.471414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:07.471416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:07.471447Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:07.471458Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:07.471467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:07.471535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.471542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.471546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.471548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:07.471625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:07.471634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.471649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.471669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.471704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:07.471731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341328898263296:5079]
2025-10-09T21:45:07.471733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341328898263296:5079]
2025-10-09T21:45:07.471735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341328898263296:5079]
2025-10-09T21:45:07.471744Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:07.471764Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:07.471770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:07.471776Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:07.471778Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:07.471787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:07.471790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:07.471803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341329260075137:2432], now have 1 active actors on pipe
2025-10-09T21:45:07.471810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341329260075326:5535], now have 1 active actors on pipe
2025-10-09T21:45:07.471850Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341328898263296:5079]
2025-10-09T21:45:07.471863Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:07.471880Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:07.471877Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 452
2025-10-09T21:45:07.471942Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 452
2025-10-09T21:45:07.472149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:07.472163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:07.472194Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|11755c58-cfa29e3f-7d1e7a0b-20cfcadb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:07.472208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:07.472217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:07.472220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:07.472231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:07.472248Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:07.472278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:07.472561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:07.472589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:07.472611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:07.472744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 453 partition: 0 MaxSeqNo: 25 sessionId: src_id|11755c58-cfa29e3f-7d1e7a0b-20cfcadb_0
2025-10-09T21:45:07.473178Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|413a1d4f-2d3292db-38fd0af1-26af853c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|11755c58-cfa29e3f-7d1e7a0b-20cfcadb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:07.475729Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|11755c58-cfa29e3f-7d1e7a0b-20cfcadb_0] Write session: try to update token
2025-10-09T21:45:07.475764Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|11755c58-cfa29e3f-7d1e7a0b-20cfcadb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:07.487620Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:07.487648Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:07.487659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341328898263296:5079]
2025-10-09T21:45:07.487713Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:07.487911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:07.488065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:07.488206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:07.488355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:07.488550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:07.488591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:07.488847Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:07.492020Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 453 sessionId: src_id|11755c58-cfa29e3f-7d1e7a0b-20cfcadb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:07.494647Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:07.495440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:07.495465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:07.495506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:07.496321Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:07.506765Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:07.506787Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:07.506791Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:07.506793Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:07.506803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341328898263296:5079]
2025-10-09T21:45:07.506807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341328898263296:5079]
2025-10-09T21:45:07.506810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341328898263296:5079]
2025-10-09T21:45:07.506854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:07.506862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.506864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:07.507002Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:07.507279Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:07.507300Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:07.507304Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:07.522571Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341329260075419:5210]
2025-10-09T21:45:07.523094Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 452, pipe: [1:7559341329260075419:5210]
2025-10-09T21:45:07.669183Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:07.677656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:07.677694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:07.677871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:07.677932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:07.678000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:07.678086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:07.678135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:07.678146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:07.681144Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:07.681208Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341329260075419:5210]
2025-10-09T21:45:07.681238Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:07.681245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:07.681273Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 453 reason: error at write: Disconnected sessionId: src_id|11755c58-cfa29e3f-7d1e7a0b-20cfcadb_0
2025-10-09T21:45:07.681484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 453 sessionId: src_id|11755c58-cfa29e3f-7d1e7a0b-20cfcadb_0 is DEAD
2025-10-09T21:45:07.681663Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11755c58-cfa29e3f-7d1e7a0b-20cfcadb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:07.681677Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11755c58-cfa29e3f-7d1e7a0b-20cfcadb_0] Write session will restart in 0.005001s
2025-10-09T21:45:07.681704Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11755c58-cfa29e3f-7d1e7a0b-20cfcadb_0] Write session: Do CDS request
2025-10-09T21:45:07.681722Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11755c58-cfa29e3f-7d1e7a0b-20cfcadb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:07.678227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:07.678301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:07.678355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:07.678413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:07.678455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:07.678459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:07.678520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:07.678573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:07.678630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:07.678694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:07.678737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:07.678740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:07.678807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:07.678871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:07.678931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:07.678988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:07.679034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:07.679038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:07.679129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:07.679185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:07.679241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:07.679299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:07.679338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:07.679341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:07.679405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:07.679458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:07.679516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:07.679573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:07.679617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:07.679619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:07.679681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:07.679734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:07.679795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:07.679858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:07.679898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:07.679901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:07.679967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:07.680022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:07.680079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:07.680133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:07.680178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:07.680181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:07.680247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:07.680300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:07.680359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:07.680416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:07.680458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:07.680463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:07.680526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:07.680579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:07.680635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:07.680690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:07.680729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:07.680732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:07.680804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341329260075419:5210], now have 1 active actors on pipe
2025-10-09T21:45:07.681774Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:07.687574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|11755c58-cfa29e3f-7d1e7a0b-20cfcadb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:07.687877Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:07.687897Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 454
2025-10-09T21:45:07.688154Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 454 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:07.688213Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 454 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:07.688223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=454 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:07.688228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:07.688981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 454 sessionId:  describe result for acl check
2025-10-09T21:45:07.689018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:07.689020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:07.689021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:07.689028Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341329260075431:5540] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:07.689668Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341329260075431:5540] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:07.694643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341329260075431:5540] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:07.703659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341329260075431:5540] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:07.703680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341329260075431:5540] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:07.703684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341329260075431:5540] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:07.703694Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 454 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:07.735729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:07.742117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:07.742417Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:07.742433Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:07.742449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:07.742519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:07.742568Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341328898263576:5081]
2025-10-09T21:45:07.743336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:07.746599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:07.746674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:07.747069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:07.747127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:07.747129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:07.747130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:07.747132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:07.747133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:07.747135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:07.747136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.747138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.747139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.747146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:07.747158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:07.747162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:07.747166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:07.747170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:07.747175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:07.747179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.747183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.747187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.747196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:07.747204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:07.747210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:07.747215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:07.747221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:07.747226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:07.747231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:07.747236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:07.747241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:07.747246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:07.747250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:07.747255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:07.747261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.747266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.747271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.747275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.747280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.747288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.747320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:07.758582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:07.760091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:07.760110Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:07.760113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:07.760118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:07.760126Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 453 [2:7559341328898263576:5081]
2025-10-09T21:45:07.760145Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:07.760166Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:07.760170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:07.760173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:07.760176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:07.760196Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:07.760206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:07.760214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:07.760285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.760287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.760290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.760293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:07.760375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:07.760379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.760395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:07.760409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:07.760428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:07.760449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341328898263570:5081]
2025-10-09T21:45:07.760452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341328898263570:5081]
2025-10-09T21:45:07.760455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341328898263570:5081]
2025-10-09T21:45:07.760470Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:07.760485Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:07.760501Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:07.760505Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:07.760550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341328898263570:5081]
2025-10-09T21:45:07.760553Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:07.760667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:07.760670Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:07.760672Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:07.760677Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:07.761094Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 453
2025-10-09T21:45:07.761146Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 453
2025-10-09T21:45:07.763843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341329260075482:5540], now have 1 active actors on pipe
2025-10-09T21:45:07.763867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341329260075424:2432], now have 1 active actors on pipe
2025-10-09T21:45:07.763901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:07.763914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:07.763952Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|df6e6d74-c75a701d-d4c8f929-13169a67_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:07.763973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:07.763985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:07.763996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:07.764012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:07.764031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:07.764051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:07.768073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:07.768110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:07.768147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:07.768869Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11755c58-cfa29e3f-7d1e7a0b-20cfcadb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|df6e6d74-c75a701d-d4c8f929-13169a67_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:07.768358Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 454 partition: 0 MaxSeqNo: 25 sessionId: src_id|df6e6d74-c75a701d-d4c8f929-13169a67_0
2025-10-09T21:45:07.771606Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df6e6d74-c75a701d-d4c8f929-13169a67_0] Write session: try to update token
2025-10-09T21:45:07.771626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df6e6d74-c75a701d-d4c8f929-13169a67_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:07.781289Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341329260075704:5210]
2025-10-09T21:45:07.781731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341329260075704:5210], now have 1 active actors on pipe
2025-10-09T21:45:07.781831Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 453, pipe: [1:7559341329260075704:5210]
2025-10-09T21:45:07.788850Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:07.788872Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:07.788874Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:07.788876Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:07.788888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341328898263570:5081]
2025-10-09T21:45:07.788893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341328898263570:5081]
2025-10-09T21:45:07.788897Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341328898263570:5081]
2025-10-09T21:45:07.788946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:07.788953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:07.788956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:07.788969Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:07.788976Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:07.788980Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:07.789104Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:07.798022Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 454 sessionId: src_id|df6e6d74-c75a701d-d4c8f929-13169a67_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:07.800733Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:07.974019Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:07.974757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:07.974769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:07.975044Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:07.978606Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:07.978698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 454 reason: status is not ok: tablet will be restarted right now sessionId: src_id|df6e6d74-c75a701d-d4c8f929-13169a67_0
2025-10-09T21:45:07.979222Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df6e6d74-c75a701d-d4c8f929-13169a67_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:07.979244Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df6e6d74-c75a701d-d4c8f929-13169a67_0] Write session will restart in 0.006660s
2025-10-09T21:45:07.979268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df6e6d74-c75a701d-d4c8f929-13169a67_0] Write session: Do CDS request
2025-10-09T21:45:07.979286Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df6e6d74-c75a701d-d4c8f929-13169a67_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:07.978899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 454 sessionId: src_id|df6e6d74-c75a701d-d4c8f929-13169a67_0 is DEAD
2025-10-09T21:45:07.979042Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:07.979077Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341329260075704:5210]
2025-10-09T21:45:07.988014Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df6e6d74-c75a701d-d4c8f929-13169a67_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:07.990523Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:07.990543Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 455
2025-10-09T21:45:07.990836Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 455 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:07.990870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 455 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:07.990876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=455 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:07.990880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:07.991416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 455 sessionId:  describe result for acl check
2025-10-09T21:45:07.991450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:07.991452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:07.991454Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:07.991461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341329260075726:5545] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:07.992062Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341329260075726:5545] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:07.994650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341329260075726:5545] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:07.999448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341329260075726:5545] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:07.999467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341329260075726:5545] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:07.999471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341329260075726:5545] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:07.999480Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 455 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:08.027344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:08.034443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:08.034727Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:08.034732Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:08.034747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:08.034818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:08.034860Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341333193231149:5083]
2025-10-09T21:45:08.036232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:08.037039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:08.037109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:08.037635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:08.037688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:08.037690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:08.037691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:08.037692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:08.037694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:08.037695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:08.037696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.037697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.037698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.037705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:08.037717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:08.037721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:08.037725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:08.037729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:08.037733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:08.037736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.037740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.037744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.037752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:08.037760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:08.037767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:08.037773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:08.037778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:08.037783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:08.037788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:08.037793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:08.037799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:08.037804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:08.037809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:08.037815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:08.037820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.037825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.037830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.037837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.037841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.037847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.037876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:08.053076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:08.054877Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 454
2025-10-09T21:45:08.054929Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 454
2025-10-09T21:45:08.054374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:08.054395Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:08.054398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:08.054404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:08.054412Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 454 [2:7559341333193231149:5083]
2025-10-09T21:45:08.054432Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:08.054454Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:08.054457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:08.054461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:08.054464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:08.054486Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:08.054492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:08.054501Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:08.054581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.054584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.054586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.054589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:08.054676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:08.054679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.054693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.054705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.054720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:08.054743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341333193231143:5083]
2025-10-09T21:45:08.054746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341333193231143:5083]
2025-10-09T21:45:08.054748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341333193231143:5083]
2025-10-09T21:45:08.054757Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:08.054770Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:08.054773Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:08.054774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341329260075772:5545], now have 1 active actors on pipe
2025-10-09T21:45:08.054775Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:08.054779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341329260075717:2432], now have 1 active actors on pipe
2025-10-09T21:45:08.054788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:08.054804Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:08.054807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:08.054851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341333193231143:5083]
2025-10-09T21:45:08.054859Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:08.054869Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:08.055269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:08.055282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:08.055317Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cdcb370f-7c2bba3b-744ae878-d2706f0f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:08.055326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:08.055332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:08.055335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:08.055346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:08.055359Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:08.055378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:08.055628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:08.055639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:08.055654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:08.055758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 455 partition: 0 MaxSeqNo: 25 sessionId: src_id|cdcb370f-7c2bba3b-744ae878-d2706f0f_0
2025-10-09T21:45:08.056282Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df6e6d74-c75a701d-d4c8f929-13169a67_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cdcb370f-7c2bba3b-744ae878-d2706f0f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:08.059358Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cdcb370f-7c2bba3b-744ae878-d2706f0f_0] Write session: try to update token
2025-10-09T21:45:08.059382Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cdcb370f-7c2bba3b-744ae878-d2706f0f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:08.075396Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:08.075416Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:08.075428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341333193231143:5083]
2025-10-09T21:45:08.075484Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:08.075668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:08.075821Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:08.075969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:08.076118Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:08.076322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:08.076367Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:08.076850Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:08.079064Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341333555043296:5210]
2025-10-09T21:45:08.079793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341333555043296:5210], now have 1 active actors on pipe
2025-10-09T21:45:08.079839Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 454, pipe: [1:7559341333555043296:5210]
2025-10-09T21:45:08.081327Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 455 sessionId: src_id|cdcb370f-7c2bba3b-744ae878-d2706f0f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:08.084107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:08.084544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:08.084566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:08.084611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:08.084936Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:08.087142Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:08.087160Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:08.087163Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:08.087165Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:08.087176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341333193231143:5083]
2025-10-09T21:45:08.087180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341333193231143:5083]
2025-10-09T21:45:08.087184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341333193231143:5083]
2025-10-09T21:45:08.087230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:08.087239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.087242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:08.087387Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:08.090411Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:08.090427Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:08.090431Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:08.262880Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:08.271564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:08.271595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:08.271758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:08.271830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:08.271906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:08.271973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:08.272022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:08.272032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:08.272104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:08.272170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:08.272257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:08.272323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:08.272370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:08.272374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:08.272440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:08.272495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:08.272552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:08.272607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:08.272651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:08.272654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:08.272719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:08.272779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:08.272840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:08.272901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:08.272945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:08.272948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:08.273018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:08.273080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:08.273139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:08.273203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:08.273241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:08.273244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:08.273311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:08.273373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:08.273435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:08.273496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:08.273537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:08.273543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:08.273607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:08.273666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:08.273725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:08.273786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:08.273827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:08.273830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:08.273898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:08.273953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:08.274017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:08.274280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:08.274332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:08.274340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:08.274405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:08.275713Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cdcb370f-7c2bba3b-744ae878-d2706f0f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:08.275724Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cdcb370f-7c2bba3b-744ae878-d2706f0f_0] Write session will restart in 0.005643s
2025-10-09T21:45:08.275749Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cdcb370f-7c2bba3b-744ae878-d2706f0f_0] Write session: Do CDS request
2025-10-09T21:45:08.275765Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cdcb370f-7c2bba3b-744ae878-d2706f0f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:08.274473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:08.274543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:08.274608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:08.274652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:08.274655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:08.274727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:08.274802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:08.274863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:08.274922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:08.274966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:08.274969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:08.276208Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:08.275287Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:08.275303Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:08.275310Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341333555043296:5210]
2025-10-09T21:45:08.275334Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 455 reason: error at write: Disconnected sessionId: src_id|cdcb370f-7c2bba3b-744ae878-d2706f0f_0
2025-10-09T21:45:08.275338Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:08.275568Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 455 sessionId: src_id|cdcb370f-7c2bba3b-744ae878-d2706f0f_0 is DEAD
2025-10-09T21:45:08.282529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cdcb370f-7c2bba3b-744ae878-d2706f0f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:08.282682Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:08.282712Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 456
2025-10-09T21:45:08.282837Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 456 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:08.282882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 456 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:08.282887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=456 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:08.282890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:08.283413Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 456 sessionId:  describe result for acl check
2025-10-09T21:45:08.283442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:08.283443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:08.283445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:08.283451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341333555043312:5549] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:08.284016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341333555043312:5549] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:08.287501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341333555043312:5549] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:08.291517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341333555043312:5549] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:08.291533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341333555043312:5549] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:08.291538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341333555043312:5549] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:08.291546Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 456 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:08.309750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:08.313430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:08.313700Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:08.313716Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:08.313731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:08.313801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:08.313860Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341333193231461:5092]
2025-10-09T21:45:08.314467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:08.315049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:08.315131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:08.315522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:08.315588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:08.315596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:08.315598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:08.315599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:08.315600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:08.315602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:08.315603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.315604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.315606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.315612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:08.315622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:08.315627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:08.315631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:08.315634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:08.315638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:08.315642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.315646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.315650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.315668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:08.315677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:08.315684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:08.315689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:08.315694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:08.315699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:08.315703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:08.315708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:08.315712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:08.315717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:08.315721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:08.315726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:08.315730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.315734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.315738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.315742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.315746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.315751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.315780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:08.323465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:08.324546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:08.324552Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:08.324555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:08.324560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:08.324568Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 455 [2:7559341333193231461:5092]
2025-10-09T21:45:08.324582Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:08.324601Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:08.324605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:08.325482Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 455
2025-10-09T21:45:08.325529Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 455
2025-10-09T21:45:08.324608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:08.324611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:08.324630Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:08.324636Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:08.324644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:08.324706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.324709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.324711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.324714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:08.324788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:08.324791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.324817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.324830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.324848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:08.324881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341333193231455:5092]
2025-10-09T21:45:08.324884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341333193231455:5092]
2025-10-09T21:45:08.324886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341333193231455:5092]
2025-10-09T21:45:08.324901Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:08.324918Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:08.324933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:08.324936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:08.324975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341333193231455:5092]
2025-10-09T21:45:08.324977Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:08.325136Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:08.325138Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:08.325140Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:08.325146Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:08.325306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341333555043299:2432], now have 1 active actors on pipe
2025-10-09T21:45:08.325314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341333555043381:5549], now have 1 active actors on pipe
2025-10-09T21:45:08.325821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:08.325845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:08.325884Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b6cf81e9-8abe6964-2f00190-66c22e97_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:08.325899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:08.325908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:08.325911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:08.325936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:08.325955Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:08.325973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:08.326264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:08.326276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:08.326294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:08.326436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 456 partition: 0 MaxSeqNo: 25 sessionId: src_id|b6cf81e9-8abe6964-2f00190-66c22e97_0
2025-10-09T21:45:08.326819Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cdcb370f-7c2bba3b-744ae878-d2706f0f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b6cf81e9-8abe6964-2f00190-66c22e97_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:08.329402Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6cf81e9-8abe6964-2f00190-66c22e97_0] Write session: try to update token
2025-10-09T21:45:08.329417Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6cf81e9-8abe6964-2f00190-66c22e97_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:08.343020Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:08.343041Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:08.343043Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:08.343045Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:08.343056Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341333193231455:5092]
2025-10-09T21:45:08.343061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341333193231455:5092]
2025-10-09T21:45:08.343065Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341333193231455:5092]
2025-10-09T21:45:08.343111Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:08.343129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:08.343130Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:08.343135Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:08.343136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.343139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:08.343273Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:08.348304Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 456 sessionId: src_id|b6cf81e9-8abe6964-2f00190-66c22e97_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:08.350950Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:08.376128Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341333555043605:5210]
2025-10-09T21:45:08.377237Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 455, pipe: [1:7559341333555043605:5210]
2025-10-09T21:45:08.512298Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:08.513006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:08.513029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:08.513097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341333555043605:5210], now have 1 active actors on pipe
2025-10-09T21:45:08.513393Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341333555043605:5210]
2025-10-09T21:45:08.513426Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:08.513461Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:08.513464Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:08.513522Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 456 reason: error at write: Disconnected sessionId: src_id|b6cf81e9-8abe6964-2f00190-66c22e97_0
2025-10-09T21:45:08.513716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 456 sessionId: src_id|b6cf81e9-8abe6964-2f00190-66c22e97_0 is DEAD
2025-10-09T21:45:08.513908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6cf81e9-8abe6964-2f00190-66c22e97_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:08.513919Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6cf81e9-8abe6964-2f00190-66c22e97_0] Write session will restart in 0.009815s
2025-10-09T21:45:08.513943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6cf81e9-8abe6964-2f00190-66c22e97_0] Write session: Do CDS request
2025-10-09T21:45:08.513953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6cf81e9-8abe6964-2f00190-66c22e97_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:08.513944Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:08.524586Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6cf81e9-8abe6964-2f00190-66c22e97_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:08.524813Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:08.524831Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 457
2025-10-09T21:45:08.525185Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 457 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:08.525226Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 457 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:08.525237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=457 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:08.525241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:08.526413Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 457 sessionId:  describe result for acl check
2025-10-09T21:45:08.526451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:08.526453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:08.526456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:08.526462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341333555043633:5560] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:08.527155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341333555043633:5560] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:08.532903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341333555043633:5560] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:08.541102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341333555043633:5560] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:08.541124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341333555043633:5560] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:08.541128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341333555043633:5560] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:08.541139Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 457 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:08.553075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:08.557402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:08.557637Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:08.557641Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:08.557653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:08.557703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:08.557742Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341333193231747:5101]
2025-10-09T21:45:08.558312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:08.561861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:08.561928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:08.562305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:08.562369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:08.562370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:08.562371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:08.562372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:08.562373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:08.562374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:08.562375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.562377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.562378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.562385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:08.562395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:08.562398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:08.562402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:08.562406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:08.562409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:08.562413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.562416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.562421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.562430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:08.562440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:08.562446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:08.562451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:08.562456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:08.562461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:08.562466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:08.562472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:08.562476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:08.562481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:08.562486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:08.562492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:08.562496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.562501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.562505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.562510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.562514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.562521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.562549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:08.572182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:08.573322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:08.573328Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:08.573331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:08.573335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:08.573342Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 456 [2:7559341333193231747:5101]
2025-10-09T21:45:08.573356Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:08.573377Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:08.573381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:08.573383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:08.573386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:08.573406Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:08.573412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:08.573419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:08.573481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.573483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.573485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.573488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:08.573563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:08.573566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.573578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.573587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.573601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:08.573622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341333193231742:5101]
2025-10-09T21:45:08.573624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341333193231742:5101]
2025-10-09T21:45:08.573626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341333193231742:5101]
2025-10-09T21:45:08.573634Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:08.573648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:08.573663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:08.573662Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:08.573665Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:08.573666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:08.573669Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:08.573706Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341333193231742:5101]
2025-10-09T21:45:08.573709Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:08.573712Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:08.573720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341333555043609:2432], now have 1 active actors on pipe
2025-10-09T21:45:08.573724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341333555043772:5560], now have 1 active actors on pipe
2025-10-09T21:45:08.575347Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 456
2025-10-09T21:45:08.575358Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 456
2025-10-09T21:45:08.575835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:08.575852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:08.575885Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ad5a53ee-6192e9cc-50fe6220-7aa27386_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:08.575897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:08.575902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:08.575905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:08.575917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:08.575932Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:08.575950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:08.576290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:08.576298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:08.576315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:08.576897Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6cf81e9-8abe6964-2f00190-66c22e97_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ad5a53ee-6192e9cc-50fe6220-7aa27386_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:08.579684Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad5a53ee-6192e9cc-50fe6220-7aa27386_0] Write session: try to update token
2025-10-09T21:45:08.579707Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad5a53ee-6192e9cc-50fe6220-7aa27386_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:08.576488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 457 partition: 0 MaxSeqNo: 25 sessionId: src_id|ad5a53ee-6192e9cc-50fe6220-7aa27386_0
2025-10-09T21:45:08.594959Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:08.594980Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:08.594982Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:08.594984Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:08.594993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341333193231742:5101]
2025-10-09T21:45:08.594997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341333193231742:5101]
2025-10-09T21:45:08.595001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341333193231742:5101]
2025-10-09T21:45:08.595044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:08.595051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.595054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:08.595193Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:08.598227Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:08.598256Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:08.598259Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:08.604129Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 457 sessionId: src_id|ad5a53ee-6192e9cc-50fe6220-7aa27386_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:08.606928Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:08.613628Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341333555043896:5210]
2025-10-09T21:45:08.613968Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 456, pipe: [1:7559341333555043896:5210]
2025-10-09T21:45:08.767469Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:08.767761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:08.767776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:08.767825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341333555043896:5210], now have 1 active actors on pipe
2025-10-09T21:45:08.768060Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341333555043896:5210]
2025-10-09T21:45:08.768105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:08.768088Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:08.768108Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:08.768196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 457 reason: error at write: Disconnected sessionId: src_id|ad5a53ee-6192e9cc-50fe6220-7aa27386_0
2025-10-09T21:45:08.768556Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad5a53ee-6192e9cc-50fe6220-7aa27386_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:08.768576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad5a53ee-6192e9cc-50fe6220-7aa27386_0] Write session will restart in 0.007931s
2025-10-09T21:45:08.768600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad5a53ee-6192e9cc-50fe6220-7aa27386_0] Write session: Do CDS request
2025-10-09T21:45:08.768607Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad5a53ee-6192e9cc-50fe6220-7aa27386_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:08.768386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 457 sessionId: src_id|ad5a53ee-6192e9cc-50fe6220-7aa27386_0 is DEAD
2025-10-09T21:45:08.768631Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:08.777676Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad5a53ee-6192e9cc-50fe6220-7aa27386_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:08.777754Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:08.777769Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 458
2025-10-09T21:45:08.779151Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 458 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:08.779196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 458 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:08.779201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=458 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:08.779207Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:08.780014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 458 sessionId:  describe result for acl check
2025-10-09T21:45:08.780047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:08.780048Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:08.780050Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:08.780068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341333555043931:5565] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:08.780606Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341333555043931:5565] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:08.789123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341333555043931:5565] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:08.795774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341333555043931:5565] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:08.795792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341333555043931:5565] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:08.795797Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341333555043931:5565] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:08.795807Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 458 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:08.805632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:08.810453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:08.810749Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:08.810754Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:08.810768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:08.810828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:08.810870Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341333193232027:5103]
2025-10-09T21:45:08.811509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:08.812008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:08.812076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:08.812391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:08.812450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:08.812456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:08.812457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:08.812458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:08.812459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:08.812460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:08.812462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.812463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.812464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.812469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:08.812479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:08.812482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:08.812486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:08.812489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:08.812492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:08.812495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.812498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.812501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.812509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:08.812517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:08.812523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:08.812527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:08.812530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:08.812534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:08.812538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:08.812542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:08.812546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:08.812551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:08.812555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:08.812559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:08.812563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.812567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.812571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.812575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.812581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.812585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.812610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:08.819853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:08.821590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:08.821597Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:08.821600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:08.821605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:08.821613Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 457 [2:7559341333193232027:5103]
2025-10-09T21:45:08.821629Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:08.821649Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:08.821653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:08.821656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:08.821658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:08.821677Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:08.821683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:08.821691Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:08.821779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.821781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.821786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.821790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:08.821872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:08.821875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.821891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:08.821904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:08.821921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:08.821944Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341333193232021:5103]
2025-10-09T21:45:08.821946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341333193232021:5103]
2025-10-09T21:45:08.821949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341333193232021:5103]
2025-10-09T21:45:08.821964Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:08.821981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:08.821996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:08.822000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:08.822064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341333193232021:5103]
2025-10-09T21:45:08.822067Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:08.822405Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:08.822416Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:08.822419Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:08.822426Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:08.822627Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 457
2025-10-09T21:45:08.822630Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 457
2025-10-09T21:45:08.822873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341333555044110:5565], now have 1 active actors on pipe
2025-10-09T21:45:08.822896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341333555043899:2432], now have 1 active actors on pipe
2025-10-09T21:45:08.822918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:08.822924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:08.822964Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7669ec4a-a769af26-b215e5d-e6a5e9b7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:08.822975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:08.822980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:08.822984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:08.822995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:08.823008Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:08.823026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:08.823398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:08.823413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:08.823436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:08.823564Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 458 partition: 0 MaxSeqNo: 25 sessionId: src_id|7669ec4a-a769af26-b215e5d-e6a5e9b7_0
2025-10-09T21:45:08.824046Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad5a53ee-6192e9cc-50fe6220-7aa27386_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7669ec4a-a769af26-b215e5d-e6a5e9b7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:08.826692Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7669ec4a-a769af26-b215e5d-e6a5e9b7_0] Write session: try to update token
2025-10-09T21:45:08.826709Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7669ec4a-a769af26-b215e5d-e6a5e9b7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:08.841543Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:08.841560Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:08.841561Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:08.841563Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:08.841573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341333193232021:5103]
2025-10-09T21:45:08.841578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341333193232021:5103]
2025-10-09T21:45:08.841581Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341333193232021:5103]
2025-10-09T21:45:08.841629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:08.841637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:08.841640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:08.841773Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:08.842128Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:08.842134Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:08.842139Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:08.845209Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 458 sessionId: src_id|7669ec4a-a769af26-b215e5d-e6a5e9b7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:08.847798Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:08.868114Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341333555044185:5210]
2025-10-09T21:45:08.868926Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 457, pipe: [1:7559341333555044185:5210]
2025-10-09T21:45:09.004952Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:09.005235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:09.005251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:09.005297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341333555044185:5210], now have 1 active actors on pipe
2025-10-09T21:45:09.005528Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:09.006044Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7669ec4a-a769af26-b215e5d-e6a5e9b7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:09.006053Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7669ec4a-a769af26-b215e5d-e6a5e9b7_0] Write session will restart in 0.009485s
2025-10-09T21:45:09.006077Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7669ec4a-a769af26-b215e5d-e6a5e9b7_0] Write session: Do CDS request
2025-10-09T21:45:09.006083Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7669ec4a-a769af26-b215e5d-e6a5e9b7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:09.005545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:09.005560Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:09.005619Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341333555044185:5210]
2025-10-09T21:45:09.005637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 458 reason: error at write: Disconnected sessionId: src_id|7669ec4a-a769af26-b215e5d-e6a5e9b7_0
2025-10-09T21:45:09.005640Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:09.005831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 458 sessionId: src_id|7669ec4a-a769af26-b215e5d-e6a5e9b7_0 is DEAD
2025-10-09T21:45:09.016779Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7669ec4a-a769af26-b215e5d-e6a5e9b7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:09.017560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:09.017587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 459
2025-10-09T21:45:09.017864Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 459 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:09.017905Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 459 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:09.017916Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=459 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:09.017920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:09.018589Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 459 sessionId:  describe result for acl check
2025-10-09T21:45:09.018630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:09.018632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:09.018634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:09.018640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341337850011530:5570] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:09.019240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341337850011530:5570] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:09.022308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341337850011530:5570] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:09.027560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341337850011530:5570] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:09.027577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341337850011530:5570] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:09.027582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341337850011530:5570] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:09.027592Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 459 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:09.034544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:09.037847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:09.038118Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:09.038130Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:09.038145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:09.038207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:09.038260Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341337488199602:5105]
2025-10-09T21:45:09.038923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:09.039513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:09.039602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:09.040052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:09.040126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:09.040138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:09.040139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:09.040140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:09.040142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:09.040143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:09.040144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.040145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.040147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.040154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:09.040166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:09.040171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:09.040175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:09.040179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:09.040183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:09.040187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.040191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.040194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.040203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:09.040212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:09.040219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:09.040225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:09.040231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:09.040237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:09.040241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:09.040247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:09.040252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:09.040258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:09.040265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:09.040270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:09.040278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.040284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.040289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.040295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.040300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.040305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.040335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:09.048012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:09.049700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:09.049719Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:09.049722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:09.049727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:09.049734Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 458 [2:7559341337488199602:5105]
2025-10-09T21:45:09.049748Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:09.049768Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:09.049771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:09.049774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:09.049776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:09.049795Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:09.049811Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:09.049823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:09.049886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.049894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.049896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.049899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:09.049976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:09.049984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.049998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.050051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.050074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:09.050101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341337850011676:5570], now have 1 active actors on pipe
2025-10-09T21:45:09.050104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341337850011495:2432], now have 1 active actors on pipe
2025-10-09T21:45:09.050113Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341337488199597:5105]
2025-10-09T21:45:09.050114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341337488199597:5105]
2025-10-09T21:45:09.050116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341337488199597:5105]
2025-10-09T21:45:09.050131Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:09.050147Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:09.050150Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:09.050155Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:09.050157Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:09.050161Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:09.050164Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:09.050205Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341337488199597:5105]
2025-10-09T21:45:09.050214Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:09.050230Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:09.050195Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 458
2025-10-09T21:45:09.050227Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 458
2025-10-09T21:45:09.050286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:09.050298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:09.050334Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|85d81a6a-b721505d-33699ddb-c932396b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:09.050348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:09.050353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:09.050356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:09.050369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:09.050379Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:09.050398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:09.050632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:09.050647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:09.050672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:09.050820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 459 partition: 0 MaxSeqNo: 25 sessionId: src_id|85d81a6a-b721505d-33699ddb-c932396b_0
2025-10-09T21:45:09.051250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7669ec4a-a769af26-b215e5d-e6a5e9b7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|85d81a6a-b721505d-33699ddb-c932396b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:09.053942Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85d81a6a-b721505d-33699ddb-c932396b_0] Write session: try to update token
2025-10-09T21:45:09.053964Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85d81a6a-b721505d-33699ddb-c932396b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:09.063856Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:09.063875Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:09.063885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341337488199597:5105]
2025-10-09T21:45:09.063935Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:09.063936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:09.064114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:09.064264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:09.064412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:09.064553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:09.064847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:09.064898Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:09.074503Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 459 sessionId: src_id|85d81a6a-b721505d-33699ddb-c932396b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:09.077164Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:09.077445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:09.077476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:09.077527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:09.080333Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:09.080355Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:09.080357Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:09.080360Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:09.080371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341337488199597:5105]
2025-10-09T21:45:09.080375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341337488199597:5105]
2025-10-09T21:45:09.080379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341337488199597:5105]
2025-10-09T21:45:09.080428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:09.080431Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:09.080438Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:09.080441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.080443Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:09.080445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:09.080627Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:09.077730Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:09.105653Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341337850011779:5210]
2025-10-09T21:45:09.105902Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 458, pipe: [1:7559341337850011779:5210]
2025-10-09T21:45:09.253926Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:09.267309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:09.267342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:09.267496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.267568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.267635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.267700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.267751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.267756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.267832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.267886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.267947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.268010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.268052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.268055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.268123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.268182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.268244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.268306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.268352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.268356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.268420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.268496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.268552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.268611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.268653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.268656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.268725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.268787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.268851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.268914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.268959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.268963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.269035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.269098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.269159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.269225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.269275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.269278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.269350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.269411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.269478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.269547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.269596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.269599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.269674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.269734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.269797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.269858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.269901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.269903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.269970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.270389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.270465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.271307Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341337850011779:5210]
2025-10-09T21:45:09.271360Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:09.271364Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:09.271396Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 459 reason: error at write: Disconnected sessionId: src_id|85d81a6a-b721505d-33699ddb-c932396b_0
2025-10-09T21:45:09.271614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 459 sessionId: src_id|85d81a6a-b721505d-33699ddb-c932396b_0 is DEAD
2025-10-09T21:45:09.271790Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85d81a6a-b721505d-33699ddb-c932396b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:09.271798Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85d81a6a-b721505d-33699ddb-c932396b_0] Write session will restart in 0.007885s
2025-10-09T21:45:09.271821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85d81a6a-b721505d-33699ddb-c932396b_0] Write session: Do CDS request
2025-10-09T21:45:09.271827Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85d81a6a-b721505d-33699ddb-c932396b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:09.271884Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:09.270531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.270581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.270583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.270644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.270702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.270764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.270822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.270861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.270864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.270939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341337850011779:5210], now have 1 active actors on pipe
2025-10-09T21:45:09.272173Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:09.280612Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85d81a6a-b721505d-33699ddb-c932396b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:09.282368Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:09.282385Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 460
2025-10-09T21:45:09.282571Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 460 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:09.282604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 460 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:09.282610Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=460 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:09.282613Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:09.284192Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 460 sessionId:  describe result for acl check
2025-10-09T21:45:09.284232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:09.284233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:09.284236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:09.284242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341337850011796:5574] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:09.284852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341337850011796:5574] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:09.291948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341337850011796:5574] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:09.299570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341337850011796:5574] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:09.299593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341337850011796:5574] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:09.299598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341337850011796:5574] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:09.299609Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 460 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:09.330915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:09.337394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:09.337668Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:09.337673Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:09.337687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:09.337977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:09.338018Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341337488199901:5108]
2025-10-09T21:45:09.338658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:09.339041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:09.339133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:09.339510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:09.339580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:09.339586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:09.339588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:09.339589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:09.339590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:09.339591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:09.339592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.339593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.339594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.339600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:09.339610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:09.339614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:09.339619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:09.339622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:09.339627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:09.339631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.339634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.339638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.339648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:09.339657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:09.339669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:09.339674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:09.339696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:09.339706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:09.339710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:09.339715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:09.339726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:09.339731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:09.339735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:09.339741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:09.339747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.339752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.339756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.339762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.339766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.339772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.339806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:09.352083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:09.353639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:09.353658Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:09.353661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:09.353665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:09.353672Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 459 [2:7559341337488199901:5108]
2025-10-09T21:45:09.353685Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:09.353703Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:09.353706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:09.353709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:09.353711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:09.353732Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:09.353738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:09.353744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:09.353804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.353807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.353808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.353811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:09.353888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:09.353897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.353910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.353922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.353939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:09.353960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341337488199896:5108]
2025-10-09T21:45:09.353968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341337488199896:5108]
2025-10-09T21:45:09.353970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341337488199896:5108]
2025-10-09T21:45:09.353982Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:09.353992Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:09.353995Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:09.353997Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:09.353997Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:09.354012Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:09.354015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:09.354026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341337850011876:5574], now have 1 active actors on pipe
2025-10-09T21:45:09.354048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341337850011782:2432], now have 1 active actors on pipe
2025-10-09T21:45:09.354154Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 459
2025-10-09T21:45:09.354192Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 459
2025-10-09T21:45:09.354094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341337488199896:5108]
2025-10-09T21:45:09.354108Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:09.354344Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:09.354530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:09.354544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:09.354577Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9747aed8-827745e2-82685896-f971bb7e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:09.354593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:09.354597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:09.354599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:09.354610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:09.354622Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:09.354646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:09.355185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:09.355200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:09.355222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:09.355410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 460 partition: 0 MaxSeqNo: 25 sessionId: src_id|9747aed8-827745e2-82685896-f971bb7e_0
2025-10-09T21:45:09.355846Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85d81a6a-b721505d-33699ddb-c932396b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9747aed8-827745e2-82685896-f971bb7e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:09.358528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9747aed8-827745e2-82685896-f971bb7e_0] Write session: try to update token
2025-10-09T21:45:09.358551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9747aed8-827745e2-82685896-f971bb7e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:09.371488Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341337850012072:5210]
2025-10-09T21:45:09.375968Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 459, pipe: [1:7559341337850012072:5210]
2025-10-09T21:45:09.379456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341337850012072:5210], now have 1 active actors on pipe
2025-10-09T21:45:09.379486Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:09.380028Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:09.380054Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341337488199896:5108]
2025-10-09T21:45:09.380121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:09.380309Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:09.380461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:09.380600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:09.380733Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:09.382201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:09.382271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:09.382435Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:09.389035Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 460 sessionId: src_id|9747aed8-827745e2-82685896-f971bb7e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:09.391583Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:09.391822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:09.391846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:09.391890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:09.392067Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:09.399861Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:09.399882Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:09.399885Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:09.399889Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:09.399901Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341337488199896:5108]
2025-10-09T21:45:09.399904Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341337488199896:5108]
2025-10-09T21:45:09.399908Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341337488199896:5108]
2025-10-09T21:45:09.399961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:09.399968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.399971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:09.400034Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:09.400041Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:09.400048Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:09.400108Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:09.604344Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:09.613020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:09.613054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:09.613225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.613442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.613525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.616448Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:09.616462Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:09.616470Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:09.616495Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 460 reason: error at write: Disconnected sessionId: src_id|9747aed8-827745e2-82685896-f971bb7e_0
2025-10-09T21:45:09.616537Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341337850012072:5210]
2025-10-09T21:45:09.613587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.616717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 460 sessionId: src_id|9747aed8-827745e2-82685896-f971bb7e_0 is DEAD
2025-10-09T21:45:09.618183Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9747aed8-827745e2-82685896-f971bb7e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:09.618197Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9747aed8-827745e2-82685896-f971bb7e_0] Write session will restart in 0.008036s
2025-10-09T21:45:09.618224Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9747aed8-827745e2-82685896-f971bb7e_0] Write session: Do CDS request
2025-10-09T21:45:09.618231Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9747aed8-827745e2-82685896-f971bb7e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:09.613638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.613644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.613708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.613762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.613818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.613881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.613919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.613922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.613982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.614057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.614111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.614167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.614204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.614207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.614291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.614348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.614427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.614485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.614528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.614531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.614597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.614651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.614714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.614774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.614813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.614816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.614879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.614931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.614988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.615044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.615089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.615092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.615156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.615210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.615269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.615327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.615364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.615367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.615429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.615481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.615538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.615594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.615633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.615636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.615697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.615749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.615804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.615859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.615899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.615902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.615963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:09.616017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:09.616074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:09.616131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:09.616173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:09.616176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:09.618021Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:09.632469Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9747aed8-827745e2-82685896-f971bb7e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:09.632941Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:09.632957Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 461
2025-10-09T21:45:09.633142Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 461 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:09.633177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 461 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:09.633183Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=461 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:09.633188Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:09.633633Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 461 sessionId:  describe result for acl check
2025-10-09T21:45:09.633675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:09.633681Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:09.633683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:09.633693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341337850012135:5586] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:09.634294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341337850012135:5586] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:09.646912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341337850012135:5586] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:09.651988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341337850012135:5586] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:09.652008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341337850012135:5586] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:09.652012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341337850012135:5586] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:09.652022Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 461 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:09.659195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:09.660241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:09.660464Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:09.660479Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:09.660493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:09.660551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:09.660615Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341337488200209:5123]
2025-10-09T21:45:09.661234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:09.664242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:09.664349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:09.665005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:09.665078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:09.665079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:09.665080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:09.665081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:09.665083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:09.665084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:09.665085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.665086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.665088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.665094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:09.665104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:09.665108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:09.665113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:09.665117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:09.665121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:09.665125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.665129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.665133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.665141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:09.665151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:09.665158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:09.665164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:09.665168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:09.665173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:09.665177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:09.665182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:09.665186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:09.665192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:09.665196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:09.665201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:09.665206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.665212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.665216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.665221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.665226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.665233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.665263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:09.675668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:09.676754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:09.676770Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:09.676773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:09.676778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:09.676787Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 460 [2:7559341337488200209:5123]
2025-10-09T21:45:09.676806Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:09.676831Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:09.676840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:09.676843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:09.676846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:09.676866Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:09.676876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:09.676884Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:09.676950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.676958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.676961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.676964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:09.677043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:09.677051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.677066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.677084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.677104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:09.677134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341337488200203:5123]
2025-10-09T21:45:09.677141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341337488200203:5123]
2025-10-09T21:45:09.677144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341337488200203:5123]
2025-10-09T21:45:09.677154Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:09.677170Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:09.677174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:09.677184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:09.677187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:09.677190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:09.677193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:09.677206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341337850012329:5586], now have 1 active actors on pipe
2025-10-09T21:45:09.677210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341337850012102:2432], now have 1 active actors on pipe
2025-10-09T21:45:09.677261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341337488200203:5123]
2025-10-09T21:45:09.677274Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:09.677280Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:09.678714Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 460
2025-10-09T21:45:09.678743Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 460
2025-10-09T21:45:09.679126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:09.679145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:09.679182Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|27a70683-326128aa-306217d6-d60810ac_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:09.679195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:09.679201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:09.679204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:09.679226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:09.679244Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:09.679267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:09.683036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:09.683067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:09.683103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:09.688591Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 461 partition: 0 MaxSeqNo: 25 sessionId: src_id|27a70683-326128aa-306217d6-d60810ac_0
2025-10-09T21:45:09.688974Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9747aed8-827745e2-82685896-f971bb7e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|27a70683-326128aa-306217d6-d60810ac_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:09.691547Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|27a70683-326128aa-306217d6-d60810ac_0] Write session: try to update token
2025-10-09T21:45:09.691571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|27a70683-326128aa-306217d6-d60810ac_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:09.699413Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:09.699432Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:09.699434Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:09.699436Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:09.699446Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341337488200203:5123]
2025-10-09T21:45:09.699451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341337488200203:5123]
2025-10-09T21:45:09.699455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341337488200203:5123]
2025-10-09T21:45:09.699500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:09.699500Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:09.699508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.699510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:09.699516Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:09.699520Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:09.699658Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:09.716034Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 461 sessionId: src_id|27a70683-326128aa-306217d6-d60810ac_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:09.718263Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341337850012392:5210]
2025-10-09T21:45:09.718615Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:09.718954Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 460, pipe: [1:7559341337850012392:5210]
2025-10-09T21:45:09.876259Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:09.876568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:09.876591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:09.876617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341337850012392:5210], now have 1 active actors on pipe
2025-10-09T21:45:09.876907Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:09.876914Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341337850012392:5210]
2025-10-09T21:45:09.876924Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:09.876952Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:09.876999Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 461 reason: error at write: Disconnected sessionId: src_id|27a70683-326128aa-306217d6-d60810ac_0
2025-10-09T21:45:09.877224Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 461 sessionId: src_id|27a70683-326128aa-306217d6-d60810ac_0 is DEAD
2025-10-09T21:45:09.876848Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:09.877616Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27a70683-326128aa-306217d6-d60810ac_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:09.877626Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27a70683-326128aa-306217d6-d60810ac_0] Write session will restart in 0.009658s
2025-10-09T21:45:09.877652Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27a70683-326128aa-306217d6-d60810ac_0] Write session: Do CDS request
2025-10-09T21:45:09.877658Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27a70683-326128aa-306217d6-d60810ac_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:09.890565Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|27a70683-326128aa-306217d6-d60810ac_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:09.893731Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:09.893754Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 462
2025-10-09T21:45:09.894003Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 462 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:09.894077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 462 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:09.894083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=462 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:09.894087Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:09.894566Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 462 sessionId:  describe result for acl check
2025-10-09T21:45:09.894614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:09.894616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:09.894618Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:09.894626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341337850012495:5590] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:09.895209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341337850012495:5590] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:09.898924Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341337850012495:5590] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:09.903373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341337850012495:5590] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:09.903394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341337850012495:5590] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:09.903398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341337850012495:5590] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:09.903408Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 462 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:09.908055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:09.911150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:09.911420Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:09.911430Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:09.911444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:09.911503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:09.911552Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341337488200492:5125]
2025-10-09T21:45:09.912200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:09.912606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:09.912669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:09.913020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:09.913090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:09.913096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:09.913097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:09.913098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:09.913100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:09.913100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:09.913101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.913102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.913104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.913109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:09.913120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:09.913125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:09.913129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:09.913133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:09.913138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:09.913142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.913145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.913148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.913158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:09.913166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:09.913173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:09.913179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:09.913183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:09.913188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:09.913193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:09.913199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:09.913204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:09.913209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:09.913212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:09.913216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:09.913222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.913227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.913231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.913235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.913240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.913244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.913273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:09.920796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:09.921930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:09.921950Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:09.921952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:09.921957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:09.921965Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 461 [2:7559341337488200492:5125]
2025-10-09T21:45:09.921980Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:09.922001Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:09.922004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:09.922008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:09.922010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:09.922049Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:09.922064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:09.922072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:09.922137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.922142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.922145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.922147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:09.922221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:09.922231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.922259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:09.922276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:09.922296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:09.922317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341337488200487:5125]
2025-10-09T21:45:09.922320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341337488200487:5125]
2025-10-09T21:45:09.922322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341337488200487:5125]
2025-10-09T21:45:09.922337Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:09.922353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:09.922368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:09.922371Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:09.922387Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:09.922390Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:09.922393Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:09.922413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341337488200487:5125]
2025-10-09T21:45:09.922416Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:09.922494Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:09.922645Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 461
2025-10-09T21:45:09.922654Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 461
2025-10-09T21:45:09.922716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341337850012395:2432], now have 1 active actors on pipe
2025-10-09T21:45:09.922743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341337850012652:5590], now have 1 active actors on pipe
2025-10-09T21:45:09.922826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:09.922839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:09.922870Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5b93d536-ad8f82b8-b891c28d-b32235cd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:09.922879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:09.922883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:09.922885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:09.922898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:09.922911Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:09.922942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:09.923135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:09.923149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:09.923173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:09.923296Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 462 partition: 0 MaxSeqNo: 25 sessionId: src_id|5b93d536-ad8f82b8-b891c28d-b32235cd_0
2025-10-09T21:45:09.923690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27a70683-326128aa-306217d6-d60810ac_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5b93d536-ad8f82b8-b891c28d-b32235cd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:09.926223Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b93d536-ad8f82b8-b891c28d-b32235cd_0] Write session: try to update token
2025-10-09T21:45:09.926252Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b93d536-ad8f82b8-b891c28d-b32235cd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:09.939782Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:09.939802Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:09.939805Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:09.939807Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:09.939817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341337488200487:5125]
2025-10-09T21:45:09.939822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341337488200487:5125]
2025-10-09T21:45:09.939826Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341337488200487:5125]
2025-10-09T21:45:09.939878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:09.939877Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:09.939884Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:09.939887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:09.939888Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:09.939889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:09.940022Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:09.946574Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 462 sessionId: src_id|5b93d536-ad8f82b8-b891c28d-b32235cd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:09.949050Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:09.977095Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341337850012690:5210]
2025-10-09T21:45:09.977507Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 461, pipe: [1:7559341337850012690:5210]
2025-10-09T21:45:10.111222Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:10.112086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:10.112104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:10.112152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341337850012690:5210], now have 1 active actors on pipe
2025-10-09T21:45:10.113379Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341337850012690:5210]
2025-10-09T21:45:10.113417Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:10.114063Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b93d536-ad8f82b8-b891c28d-b32235cd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:10.114077Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b93d536-ad8f82b8-b891c28d-b32235cd_0] Write session will restart in 0.005015s
2025-10-09T21:45:10.114101Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b93d536-ad8f82b8-b891c28d-b32235cd_0] Write session: Do CDS request
2025-10-09T21:45:10.114107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b93d536-ad8f82b8-b891c28d-b32235cd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:10.113454Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:10.113462Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:10.113528Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 462 reason: error at write: Disconnected sessionId: src_id|5b93d536-ad8f82b8-b891c28d-b32235cd_0
2025-10-09T21:45:10.113711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 462 sessionId: src_id|5b93d536-ad8f82b8-b891c28d-b32235cd_0 is DEAD
2025-10-09T21:45:10.120626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b93d536-ad8f82b8-b891c28d-b32235cd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:10.120695Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:10.120711Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 463
2025-10-09T21:45:10.119963Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:10.120925Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 463 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:10.120959Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 463 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:10.120969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=463 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:10.120973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:10.121649Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 463 sessionId:  describe result for acl check
2025-10-09T21:45:10.121686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:10.121692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:10.121694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:10.121700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341342144980005:5595] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:10.122349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341342144980005:5595] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:10.131295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341342144980005:5595] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:10.139471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341342144980005:5595] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:10.139491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341342144980005:5595] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:10.139495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341342144980005:5595] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:10.139506Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 463 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:10.156671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:10.163425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:10.163753Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:10.163769Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:10.163785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:10.163875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:10.163925Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341341783168075:5128]
2025-10-09T21:45:10.164583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:10.166611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:10.166697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:10.167129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:10.167209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:10.167210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:10.167211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:10.167213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:10.167214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:10.167216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:10.167217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.167218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.167220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.167227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:10.167239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:10.167243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:10.167247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:10.167251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:10.167256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:10.167260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.167264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.167267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.167276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:10.167286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:10.167292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:10.167298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:10.167304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:10.167309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:10.167314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:10.167320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:10.167325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:10.167330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:10.167335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:10.167341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:10.167346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.167353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.167360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.167366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.167371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.167377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.167412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:10.177795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:10.179841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:10.179869Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:10.179871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:10.179875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:10.179884Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 462 [2:7559341341783168075:5128]
2025-10-09T21:45:10.179905Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:10.179927Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:10.179930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:10.179934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:10.179936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:10.179957Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:10.179964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:10.179971Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:10.180051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.180060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.180063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.180066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:10.180142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:10.180154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.180169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.180182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.180206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:10.180233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341342144979992:2432], now have 1 active actors on pipe
2025-10-09T21:45:10.180242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341342144980104:5595], now have 1 active actors on pipe
2025-10-09T21:45:10.180250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341341783168070:5128]
2025-10-09T21:45:10.180253Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341341783168070:5128]
2025-10-09T21:45:10.180255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341341783168070:5128]
2025-10-09T21:45:10.180272Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:10.180286Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:10.180288Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:10.180289Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:10.180292Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:10.180305Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:10.180309Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:10.180352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341341783168070:5128]
2025-10-09T21:45:10.180364Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:10.180380Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:10.180406Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 462
2025-10-09T21:45:10.180466Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 462
2025-10-09T21:45:10.180734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:10.180750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:10.180784Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|395273fa-7152504a-31a78fbc-63562c98_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:10.180800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:10.180805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:10.180808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:10.180820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:10.180841Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:10.180859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:10.181282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:10.181298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:10.181322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:10.182049Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b93d536-ad8f82b8-b891c28d-b32235cd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|395273fa-7152504a-31a78fbc-63562c98_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:10.184670Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|395273fa-7152504a-31a78fbc-63562c98_0] Write session: try to update token
2025-10-09T21:45:10.184691Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|395273fa-7152504a-31a78fbc-63562c98_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:10.181567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 463 partition: 0 MaxSeqNo: 25 sessionId: src_id|395273fa-7152504a-31a78fbc-63562c98_0
2025-10-09T21:45:10.197070Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:10.197093Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:10.197103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341341783168070:5128]
2025-10-09T21:45:10.197156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:10.197339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:10.197489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:10.197627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:10.197767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:10.197986Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:10.198049Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:10.198430Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:10.204898Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 463 sessionId: src_id|395273fa-7152504a-31a78fbc-63562c98_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:10.207650Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:10.208045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:10.208069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:10.208112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:10.208236Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:10.213625Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341342144980284:5210]
2025-10-09T21:45:10.214229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341342144980284:5210], now have 1 active actors on pipe
2025-10-09T21:45:10.214311Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 462, pipe: [1:7559341342144980284:5210]
2025-10-09T21:45:10.231704Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:10.231726Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:10.231728Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:10.231730Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:10.231742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341341783168070:5128]
2025-10-09T21:45:10.231746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341341783168070:5128]
2025-10-09T21:45:10.231750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341341783168070:5128]
2025-10-09T21:45:10.231809Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:10.231817Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:10.231820Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:10.232131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:10.232158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.232162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:10.232322Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:10.400642Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:10.412697Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:10.412758Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:10.412761Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:10.412802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 463 reason: error at write: Disconnected sessionId: src_id|395273fa-7152504a-31a78fbc-63562c98_0
2025-10-09T21:45:10.412972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 463 sessionId: src_id|395273fa-7152504a-31a78fbc-63562c98_0 is DEAD
2025-10-09T21:45:10.413062Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341342144980284:5210]
2025-10-09T21:45:10.409190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:10.409211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:10.409371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:10.409424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:10.409478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:10.409539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:10.409581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:10.409586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:10.409658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:10.409711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:10.409768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:10.409834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:10.409872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:10.409875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:10.409939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:10.409996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:10.410077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:10.410140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:10.410177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:10.410179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:10.410260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:10.410316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:10.410370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:10.410427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:10.410467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:10.410470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:10.410535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:10.410593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:10.410651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:10.410715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:10.410761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:10.410763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:10.410829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:10.410884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:10.410939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:10.411002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:10.411043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:10.411046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:10.411113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:10.411173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:10.411237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:10.411301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:10.411345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:10.411349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:10.411415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:10.411472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:10.411533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:10.411600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:10.411641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:10.411644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:10.411711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:10.411765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:10.411822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:10.411880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:10.411919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:10.411922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:10.411983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:10.412044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:10.412106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:10.412169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:10.412209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:10.412212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:10.413561Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:10.414771Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|395273fa-7152504a-31a78fbc-63562c98_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:10.414783Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|395273fa-7152504a-31a78fbc-63562c98_0] Write session will restart in 0.007723s
2025-10-09T21:45:10.414808Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|395273fa-7152504a-31a78fbc-63562c98_0] Write session: Do CDS request
2025-10-09T21:45:10.414816Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|395273fa-7152504a-31a78fbc-63562c98_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:10.423728Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|395273fa-7152504a-31a78fbc-63562c98_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:10.426382Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:10.426400Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 464
2025-10-09T21:45:10.426815Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 464 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:10.426855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 464 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:10.426866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=464 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:10.426870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:10.427305Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 464 sessionId:  describe result for acl check
2025-10-09T21:45:10.427344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:10.427347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:10.427348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:10.427354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341342144980323:5599] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:10.427892Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341342144980323:5599] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:10.434389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341342144980323:5599] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:10.439532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341342144980323:5599] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:10.439550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341342144980323:5599] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:10.439556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341342144980323:5599] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:10.439566Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 464 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:10.451223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:10.455791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:10.456095Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:10.456107Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:10.456121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:10.456202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:10.456247Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341341783168392:5141]
2025-10-09T21:45:10.456851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:10.457319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:10.457393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:10.457750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:10.457802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:10.457804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:10.457805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:10.457806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:10.457808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:10.457810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:10.457811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.457834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.457835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.457841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:10.457854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:10.457859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:10.457864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:10.457879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:10.457884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:10.457889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.457893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.457898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.457906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:10.457920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:10.457928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:10.457934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:10.457942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:10.457948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:10.457953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:10.457959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:10.457965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:10.457970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:10.457975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:10.457981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:10.457986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.457992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.457997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.458003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.458008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.458015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.458066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:10.467305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:10.468749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:10.468767Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:10.468770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:10.468775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:10.468782Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 463 [2:7559341341783168392:5141]
2025-10-09T21:45:10.468796Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:10.468816Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:10.468819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:10.468822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:10.468825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:10.468843Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:10.468852Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:10.468867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:10.468929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.468940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.468943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.468946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:10.469029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:10.469038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.469054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.469068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.469087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:10.469111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341342144980290:2432], now have 1 active actors on pipe
2025-10-09T21:45:10.469123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341341783168387:5141]
2025-10-09T21:45:10.469126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341341783168387:5141]
2025-10-09T21:45:10.469128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341341783168387:5141]
2025-10-09T21:45:10.469145Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:10.469157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341342144980468:5599], now have 1 active actors on pipe
2025-10-09T21:45:10.469171Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:10.469174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:10.469175Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:10.469177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:10.469191Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:10.469195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:10.469239Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341341783168387:5141]
2025-10-09T21:45:10.469254Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:10.469323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:10.469652Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 463
2025-10-09T21:45:10.469701Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 463
2025-10-09T21:45:10.470230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:10.470260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:10.470306Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e3118e5b-3931593c-c8a7b39e-4b7b3841_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:10.470316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:10.470321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:10.470324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:10.470337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:10.470352Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:10.470371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:10.470813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:10.470824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:10.470843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:10.471147Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 464 partition: 0 MaxSeqNo: 25 sessionId: src_id|e3118e5b-3931593c-c8a7b39e-4b7b3841_0
2025-10-09T21:45:10.471557Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|395273fa-7152504a-31a78fbc-63562c98_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e3118e5b-3931593c-c8a7b39e-4b7b3841_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:10.474281Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3118e5b-3931593c-c8a7b39e-4b7b3841_0] Write session: try to update token
2025-10-09T21:45:10.474299Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3118e5b-3931593c-c8a7b39e-4b7b3841_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:10.486640Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:10.486664Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:10.486666Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:10.486669Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:10.486679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341341783168387:5141]
2025-10-09T21:45:10.486683Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341341783168387:5141]
2025-10-09T21:45:10.486687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341341783168387:5141]
2025-10-09T21:45:10.486733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:10.486741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.486744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:10.486890Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:10.489053Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:10.489082Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:10.489086Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:10.506284Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 464 sessionId: src_id|e3118e5b-3931593c-c8a7b39e-4b7b3841_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:10.508943Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:10.513147Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341342144980602:5210]
2025-10-09T21:45:10.513471Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 463, pipe: [1:7559341342144980602:5210]
2025-10-09T21:45:10.657588Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:10.658431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:10.658455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:10.658479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341342144980602:5210], now have 1 active actors on pipe
2025-10-09T21:45:10.658729Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:10.658768Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341342144980602:5210]
2025-10-09T21:45:10.659408Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:10.659489Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 464 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e3118e5b-3931593c-c8a7b39e-4b7b3841_0
2025-10-09T21:45:10.659700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 464 sessionId: src_id|e3118e5b-3931593c-c8a7b39e-4b7b3841_0 is DEAD
2025-10-09T21:45:10.659845Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3118e5b-3931593c-c8a7b39e-4b7b3841_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:10.659873Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3118e5b-3931593c-c8a7b39e-4b7b3841_0] Write session will restart in 0.008816s
2025-10-09T21:45:10.659900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3118e5b-3931593c-c8a7b39e-4b7b3841_0] Write session: Do CDS request
2025-10-09T21:45:10.659907Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3118e5b-3931593c-c8a7b39e-4b7b3841_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:10.659906Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:10.669597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3118e5b-3931593c-c8a7b39e-4b7b3841_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:10.670358Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:10.670374Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 465
2025-10-09T21:45:10.670556Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 465 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:10.670617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 465 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:10.670623Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=465 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:10.670627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:10.671501Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 465 sessionId:  describe result for acl check
2025-10-09T21:45:10.671540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:10.671548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:10.671550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:10.671559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341342144980630:5611] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:10.672113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341342144980630:5611] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:10.675508Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341342144980630:5611] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:10.679236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341342144980630:5611] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:10.679258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341342144980630:5611] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:10.679262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341342144980630:5611] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:10.679271Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 465 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:10.691949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:10.695365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:10.695621Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:10.695634Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:10.695647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:10.695704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:10.695745Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341341783168686:5145]
2025-10-09T21:45:10.696377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:10.696846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:10.696898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:10.697243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:10.697306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:10.697313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:10.697314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:10.697315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:10.697317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:10.697318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:10.697319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.697320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.697321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.697327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:10.697338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:10.697342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:10.697347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:10.697351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:10.697355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:10.697358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.697361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.697364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.697373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:10.697381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:10.697386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:10.697391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:10.697395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:10.697417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:10.697430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:10.697436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:10.697440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:10.697445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:10.697448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:10.697454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:10.697458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.697462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.697466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.697471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.697475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.697479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.697505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:10.704857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:10.706120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:10.706134Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:10.706137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:10.706142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:10.706154Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 464 [2:7559341341783168686:5145]
2025-10-09T21:45:10.706170Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:10.706193Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:10.706200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:10.706204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:10.706206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:10.706227Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:10.706251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:10.706268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:10.706339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.706348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.706350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.706353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:10.706432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:10.706441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.706454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.706465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.706485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:10.706509Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341341783168681:5145]
2025-10-09T21:45:10.706518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341341783168681:5145]
2025-10-09T21:45:10.706520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341341783168681:5145]
2025-10-09T21:45:10.706530Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:10.706547Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:10.706547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:10.706551Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:10.706553Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:10.706562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:10.706565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:10.706577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341342144980746:5611], now have 1 active actors on pipe
2025-10-09T21:45:10.706580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341342144980606:2432], now have 1 active actors on pipe
2025-10-09T21:45:10.706623Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341341783168681:5145]
2025-10-09T21:45:10.706636Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:10.706640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:10.706692Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 464
2025-10-09T21:45:10.706908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:10.706922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:10.706961Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e8b19bff-16bae11d-e0a876e1-bdcbbd60_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:10.706975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:10.706980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:10.706984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:10.706996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:10.707008Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:10.707002Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 464
2025-10-09T21:45:10.707025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:10.707421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:10.707432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:10.707451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:10.707570Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 465 partition: 0 MaxSeqNo: 25 sessionId: src_id|e8b19bff-16bae11d-e0a876e1-bdcbbd60_0
2025-10-09T21:45:10.708021Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3118e5b-3931593c-c8a7b39e-4b7b3841_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e8b19bff-16bae11d-e0a876e1-bdcbbd60_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:10.710810Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8b19bff-16bae11d-e0a876e1-bdcbbd60_0] Write session: try to update token
2025-10-09T21:45:10.710833Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8b19bff-16bae11d-e0a876e1-bdcbbd60_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:10.723688Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:10.723713Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:10.723716Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:10.723718Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:10.723729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341341783168681:5145]
2025-10-09T21:45:10.723734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341341783168681:5145]
2025-10-09T21:45:10.723739Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341341783168681:5145]
2025-10-09T21:45:10.723787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:10.723795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.723798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:10.723798Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:10.723804Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:10.723808Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:10.723938Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:10.728422Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 465 sessionId: src_id|e8b19bff-16bae11d-e0a876e1-bdcbbd60_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:10.731093Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:10.758886Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341342144980899:5210]
2025-10-09T21:45:10.759483Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 464, pipe: [1:7559341342144980899:5210]
2025-10-09T21:45:10.906734Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:10.907071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:10.907084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:10.907134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341342144980899:5210], now have 1 active actors on pipe
2025-10-09T21:45:10.907400Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:10.908875Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8b19bff-16bae11d-e0a876e1-bdcbbd60_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:10.908886Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8b19bff-16bae11d-e0a876e1-bdcbbd60_0] Write session will restart in 0.006297s
2025-10-09T21:45:10.908918Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8b19bff-16bae11d-e0a876e1-bdcbbd60_0] Write session: Do CDS request
2025-10-09T21:45:10.908924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8b19bff-16bae11d-e0a876e1-bdcbbd60_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:10.908202Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341342144980899:5210]
2025-10-09T21:45:10.908238Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:10.908302Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 465 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e8b19bff-16bae11d-e0a876e1-bdcbbd60_0
2025-10-09T21:45:10.908495Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 465 sessionId: src_id|e8b19bff-16bae11d-e0a876e1-bdcbbd60_0 is DEAD
2025-10-09T21:45:10.908622Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:10.918528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8b19bff-16bae11d-e0a876e1-bdcbbd60_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:10.922201Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:10.922228Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 466
2025-10-09T21:45:10.926167Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 466 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:10.926224Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 466 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:10.926246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=466 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:10.926250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:10.929629Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 466 sessionId:  describe result for acl check
2025-10-09T21:45:10.929679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:10.929681Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:10.929683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:10.929690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341342144980933:5615] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:10.930346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341342144980933:5615] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:10.942902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341342144980933:5615] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:10.951755Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341342144980933:5615] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:10.951775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341342144980933:5615] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:10.951779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341342144980933:5615] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:10.951789Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 466 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:10.974085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:10.979275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:10.979845Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:10.979850Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:10.979869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:10.979933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:10.979965Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341341783168974:5147]
2025-10-09T21:45:10.980426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:10.981821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:10.981854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:10.982193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:10.982288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:10.982289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:10.982290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:10.982291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:10.982292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:10.982294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:10.982295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.982296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.982298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.982303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:10.982314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:10.982318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:10.982323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:10.982327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:10.982331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:10.982335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.982338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.982341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.982349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:10.982357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:10.982364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:10.982368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:10.982372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:10.982378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:10.982384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:10.982389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:10.982393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:10.982397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:10.982402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:10.982408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:10.982413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.982417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:10.982421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.982426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:10.982431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.982436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:10.982467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:11.001804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:11.005463Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 465
2025-10-09T21:45:11.005507Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 465
2025-10-09T21:45:11.003498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:11.003517Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:11.003521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:11.003526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:11.003535Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 465 [2:7559341341783168974:5147]
2025-10-09T21:45:11.003551Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:11.003573Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:11.003577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:11.003581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:11.003585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:11.003607Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:11.003614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:11.004053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:11.004151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.004154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.004157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.004160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:11.004244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:11.004247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.004263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.004278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.004299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:11.004322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341341783168968:5147]
2025-10-09T21:45:11.004325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341341783168968:5147]
2025-10-09T21:45:11.004327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341341783168968:5147]
2025-10-09T21:45:11.004345Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:11.004362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:11.004380Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:11.004384Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:11.004428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341341783168968:5147]
2025-10-09T21:45:11.004431Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:11.004568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:11.004571Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:11.004583Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:11.004589Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:11.004976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341342144981082:5615], now have 1 active actors on pipe
2025-10-09T21:45:11.004980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341342144980906:2432], now have 1 active actors on pipe
2025-10-09T21:45:11.006067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:11.006075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:11.006121Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a60ec30e-59c460e-da8738ee-87807014_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:11.006132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:11.006137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:11.006141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:11.006152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:11.006168Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:11.006201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:11.009695Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341346439948503:5210]
2025-10-09T21:45:11.009762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 466 partition: 0 MaxSeqNo: 25 sessionId: src_id|a60ec30e-59c460e-da8738ee-87807014_0
2025-10-09T21:45:11.007402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:11.007415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:11.007435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:11.010319Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8b19bff-16bae11d-e0a876e1-bdcbbd60_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a60ec30e-59c460e-da8738ee-87807014_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:11.012824Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a60ec30e-59c460e-da8738ee-87807014_0] Write session: try to update token
2025-10-09T21:45:11.012843Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a60ec30e-59c460e-da8738ee-87807014_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:11.011386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341346439948503:5210], now have 1 active actors on pipe
2025-10-09T21:45:11.011327Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 465, pipe: [1:7559341346439948503:5210]
2025-10-09T21:45:11.029006Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:11.029031Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:11.029034Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:11.029036Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:11.029047Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341341783168968:5147]
2025-10-09T21:45:11.029051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341341783168968:5147]
2025-10-09T21:45:11.029055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341341783168968:5147]
2025-10-09T21:45:11.029108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:11.029116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.029119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:11.029263Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:11.030093Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:11.030102Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:11.030107Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:11.047305Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 466 sessionId: src_id|a60ec30e-59c460e-da8738ee-87807014_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:11.049849Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:11.225437Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:11.226642Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:11.226741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 466 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a60ec30e-59c460e-da8738ee-87807014_0
2025-10-09T21:45:11.226765Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341346439948503:5210]
2025-10-09T21:45:11.226791Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:11.226950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 466 sessionId: src_id|a60ec30e-59c460e-da8738ee-87807014_0 is DEAD
2025-10-09T21:45:11.229698Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a60ec30e-59c460e-da8738ee-87807014_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:11.229710Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a60ec30e-59c460e-da8738ee-87807014_0] Write session will restart in 0.009375s
2025-10-09T21:45:11.229735Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a60ec30e-59c460e-da8738ee-87807014_0] Write session: Do CDS request
2025-10-09T21:45:11.229743Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a60ec30e-59c460e-da8738ee-87807014_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:11.226163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:11.226186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:11.226470Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:11.240630Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a60ec30e-59c460e-da8738ee-87807014_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:11.240738Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:11.240772Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 467
2025-10-09T21:45:11.240918Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 467 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:11.240955Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 467 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:11.240962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=467 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:11.240966Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:11.241652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 467 sessionId:  describe result for acl check
2025-10-09T21:45:11.241711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:11.241716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:11.241718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:11.241724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341346439948542:5620] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:11.242386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341346439948542:5620] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:11.247454Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341346439948542:5620] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:11.251969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341346439948542:5620] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:11.251988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341346439948542:5620] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:11.251992Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341346439948542:5620] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:11.252003Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 467 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:11.270796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:11.271402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:11.271611Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:11.271615Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:11.271629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:11.271686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:11.271727Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341346078136560:5150]
2025-10-09T21:45:11.272376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:11.273081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:11.273137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:11.273471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:11.273543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:11.273544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:11.273545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:11.273546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:11.273548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:11.273549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:11.273550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.273551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.273552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.273558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:11.273569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:11.273574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:11.273578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:11.273582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:11.273585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:11.273590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.273593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.273596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.273604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:11.273613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:11.273620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:11.273627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:11.273632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:11.273637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:11.273641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:11.273646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:11.273651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:11.273656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:11.273661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:11.273665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:11.273669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.273674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.273679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.273684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.273688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.273694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.273725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:11.282756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:11.283877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:11.283886Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:11.283890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:11.283895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:11.283905Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 466 [2:7559341346078136560:5150]
2025-10-09T21:45:11.283921Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:11.283944Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:11.283948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:11.283952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:11.283954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:11.283975Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:11.283983Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:11.283992Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:11.284064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.284067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.284070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.284073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:11.284159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:11.284163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.284178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.284189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.284205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:11.284228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341346078136554:5150]
2025-10-09T21:45:11.284231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341346078136554:5150]
2025-10-09T21:45:11.284234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341346078136554:5150]
2025-10-09T21:45:11.284244Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:11.284260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:11.284277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:11.284278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:11.284280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:11.284282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:11.284284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:11.284330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341346439948508:2432], now have 1 active actors on pipe
2025-10-09T21:45:11.284334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341346439948693:5620], now have 1 active actors on pipe
2025-10-09T21:45:11.284337Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341346078136554:5150]
2025-10-09T21:45:11.284346Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:11.284602Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:11.284981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:11.284986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:11.285017Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4567440-246240b2-41d5cd8-b3166b93_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:11.285027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:11.285031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:11.285033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:11.285047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:11.285061Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:11.285080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:11.284695Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 466
2025-10-09T21:45:11.284740Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 466
2025-10-09T21:45:11.288782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 467 partition: 0 MaxSeqNo: 25 sessionId: src_id|4567440-246240b2-41d5cd8-b3166b93_0
2025-10-09T21:45:11.289177Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a60ec30e-59c460e-da8738ee-87807014_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4567440-246240b2-41d5cd8-b3166b93_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:11.288567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:11.288591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:11.288630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:11.291768Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4567440-246240b2-41d5cd8-b3166b93_0] Write session: try to update token
2025-10-09T21:45:11.291787Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4567440-246240b2-41d5cd8-b3166b93_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:11.311029Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:11.311050Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:11.311053Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:11.311055Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:11.311066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341346078136554:5150]
2025-10-09T21:45:11.311071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341346078136554:5150]
2025-10-09T21:45:11.311075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341346078136554:5150]
2025-10-09T21:45:11.311120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:11.311127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.311129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:11.311269Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:11.314465Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:11.314485Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:11.314490Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:11.320839Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 467 sessionId: src_id|4567440-246240b2-41d5cd8-b3166b93_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:11.323457Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:11.326783Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341346439948803:5210]
2025-10-09T21:45:11.327086Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 466, pipe: [1:7559341346439948803:5210]
2025-10-09T21:45:11.497391Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:11.498585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:11.498606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:11.498633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341346439948803:5210], now have 1 active actors on pipe
2025-10-09T21:45:11.498920Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:11.498960Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:11.499006Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341346439948803:5210]
2025-10-09T21:45:11.499010Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:11.499083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 467 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4567440-246240b2-41d5cd8-b3166b93_0
2025-10-09T21:45:11.499267Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 467 sessionId: src_id|4567440-246240b2-41d5cd8-b3166b93_0 is DEAD
2025-10-09T21:45:11.499410Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4567440-246240b2-41d5cd8-b3166b93_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:11.499421Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4567440-246240b2-41d5cd8-b3166b93_0] Write session will restart in 0.006956s
2025-10-09T21:45:11.499458Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4567440-246240b2-41d5cd8-b3166b93_0] Write session: Do CDS request
2025-10-09T21:45:11.499469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4567440-246240b2-41d5cd8-b3166b93_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:11.507637Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4567440-246240b2-41d5cd8-b3166b93_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:11.509254Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:11.509273Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 468
2025-10-09T21:45:11.509419Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 468 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:11.509470Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 468 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:11.509476Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=468 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:11.509479Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:11.509877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 468 sessionId:  describe result for acl check
2025-10-09T21:45:11.509916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:11.509918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:11.509920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:11.509926Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341346439948837:5628] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:11.510476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341346439948837:5628] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:11.516789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341346439948837:5628] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:11.520340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341346439948837:5628] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:11.520359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341346439948837:5628] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:11.520363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341346439948837:5628] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:11.520372Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 468 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:11.528935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:11.532079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:11.532410Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:11.532423Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:11.532436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:11.532499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:11.532555Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341346078136883:5163]
2025-10-09T21:45:11.533143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:11.533792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:11.533856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:11.534197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:11.534272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:11.534274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:11.534275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:11.534276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:11.534277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:11.534279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:11.534280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.534281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.534282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.534288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:11.534297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:11.534300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:11.534304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:11.534307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:11.534311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:11.534314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.534317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.534321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.534328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:11.534335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:11.534341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:11.534346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:11.534351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:11.534356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:11.534361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:11.534366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:11.534369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:11.534373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:11.534377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:11.534382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:11.534387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.534391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.534397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.534401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.534405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.534410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.534436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:11.542092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:11.543262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:11.543282Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:11.543284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:11.543289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:11.543296Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 467 [2:7559341346078136883:5163]
2025-10-09T21:45:11.543311Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:11.543330Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:11.543334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:11.543338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:11.543341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:11.543359Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:11.543374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:11.543383Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:11.543444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.543454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.543456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.543458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:11.543792Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 467
2025-10-09T21:45:11.543855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 467
2025-10-09T21:45:11.544070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:11.544088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.544101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.544113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.544129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:11.544153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341346439948809:2432], now have 1 active actors on pipe
2025-10-09T21:45:11.544156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341346439949030:5628], now have 1 active actors on pipe
2025-10-09T21:45:11.544180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:11.544195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:11.544214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341346078136878:5163]
2025-10-09T21:45:11.544222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341346078136878:5163]
2025-10-09T21:45:11.544224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341346078136878:5163]
2025-10-09T21:45:11.544240Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:11.544257Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:11.544275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:11.544286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:11.544307Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3ace7e90-6fe9012e-8decaa8d-d13cf10d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:11.544315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:11.544319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:11.544326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:11.544337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:11.544350Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:11.544359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:11.544366Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:11.544367Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:11.544396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341346078136878:5163]
2025-10-09T21:45:11.544409Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:11.544424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:11.544427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:11.545700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:11.545714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:11.545737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:11.545898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 468 partition: 0 MaxSeqNo: 25 sessionId: src_id|3ace7e90-6fe9012e-8decaa8d-d13cf10d_0
2025-10-09T21:45:11.546365Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4567440-246240b2-41d5cd8-b3166b93_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3ace7e90-6fe9012e-8decaa8d-d13cf10d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:11.548900Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3ace7e90-6fe9012e-8decaa8d-d13cf10d_0] Write session: try to update token
2025-10-09T21:45:11.548918Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3ace7e90-6fe9012e-8decaa8d-d13cf10d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:11.565055Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:11.565076Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:11.565087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341346078136878:5163]
2025-10-09T21:45:11.565137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:11.565300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:11.565437Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:11.565572Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:11.565716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:11.565894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:11.565936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:11.566387Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:11.566963Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 468 sessionId: src_id|3ace7e90-6fe9012e-8decaa8d-d13cf10d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:11.569399Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:11.569636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:11.569654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:11.569693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:11.569914Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:11.589846Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:11.589878Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:11.589881Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:11.589883Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:11.589894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341346078136878:5163]
2025-10-09T21:45:11.589898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341346078136878:5163]
2025-10-09T21:45:11.589902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341346078136878:5163]
2025-10-09T21:45:11.589941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:11.589948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.589951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:11.590091Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:11.593394Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:11.593409Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:11.593412Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:11.599100Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341346439949127:5210]
2025-10-09T21:45:11.601565Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 467, pipe: [1:7559341346439949127:5210]
2025-10-09T21:45:11.791550Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:11.792357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341346439949127:5210], now have 1 active actors on pipe
2025-10-09T21:45:11.801291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:11.801320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:11.801491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:11.801547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:11.801604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:11.801660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:11.801701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:11.801707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:11.805347Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341346439949127:5210]
2025-10-09T21:45:11.805390Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:11.805444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:11.805454Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:11.805496Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 468 reason: error at write: Disconnected sessionId: src_id|3ace7e90-6fe9012e-8decaa8d-d13cf10d_0
2025-10-09T21:45:11.802342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:11.802409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:11.802463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:11.802526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:11.802575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:11.802579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:11.802644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:11.802695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:11.802748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:11.802803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:11.802851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:11.802855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:11.802921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:11.802979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:11.803052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:11.803107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:11.803146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:11.803150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:11.803216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:11.803269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:11.803344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:11.803407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:11.803451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:11.803455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:11.803519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:11.803573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:11.803633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:11.803696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:11.803745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:11.805708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 468 sessionId: src_id|3ace7e90-6fe9012e-8decaa8d-d13cf10d_0 is DEAD
2025-10-09T21:45:11.803749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:11.803812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:11.803867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:11.803927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:11.803999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:11.804045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:11.804048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:11.804114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:11.804171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:11.804229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:11.804287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:11.804330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:11.804333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:11.804399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:11.804454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:11.804510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:11.804569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:11.804623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:11.804627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:11.804691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:11.804746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:11.804804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:11.804860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:11.804901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:11.804904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:11.805910Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ace7e90-6fe9012e-8decaa8d-d13cf10d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:11.805917Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ace7e90-6fe9012e-8decaa8d-d13cf10d_0] Write session will restart in 0.009994s
2025-10-09T21:45:11.805941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ace7e90-6fe9012e-8decaa8d-d13cf10d_0] Write session: Do CDS request
2025-10-09T21:45:11.805946Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ace7e90-6fe9012e-8decaa8d-d13cf10d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:11.806516Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:11.818597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3ace7e90-6fe9012e-8decaa8d-d13cf10d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:11.819051Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:11.819067Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 469
2025-10-09T21:45:11.819325Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 469 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:11.819374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 469 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:11.819384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=469 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:11.819398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:11.823527Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 469 sessionId:  describe result for acl check
2025-10-09T21:45:11.823574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:11.823576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:11.823578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:11.823585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341346439949158:5638] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:11.824169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341346439949158:5638] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:11.836405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341346439949158:5638] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:11.844280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341346439949158:5638] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:11.844301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341346439949158:5638] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:11.844305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341346439949158:5638] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:11.844315Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 469 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:11.863498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:11.867661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:11.867927Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:11.867932Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:11.867945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:11.868003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:11.868043Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341346078137182:5167]
2025-10-09T21:45:11.868619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:11.869672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:11.869746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:11.870119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:11.870186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:11.870192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:11.870193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:11.870195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:11.870196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:11.870197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:11.870199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.870200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.870201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.870208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:11.870219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:11.870223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:11.870226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:11.870241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:11.870245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:11.870249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.870254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.870257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.870265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:11.870274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:11.870280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:11.870285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:11.870301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:11.870307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:11.870312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:11.870317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:11.870322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:11.870327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:11.870332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:11.870338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:11.870343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.870348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.870352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.870358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.870362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.870368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.870397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:11.881572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:11.883742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:11.883760Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:11.883763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:11.883768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:11.883776Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 468 [2:7559341346078137182:5167]
2025-10-09T21:45:11.883793Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:11.883813Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:11.883816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:11.883820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:11.883822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:11.883842Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:11.883854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:11.883862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:11.883937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.883946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.883948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.883951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:11.884035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:11.884042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.884058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:11.884071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:11.884086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:11.884107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341346078137177:5167]
2025-10-09T21:45:11.884109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341346078137177:5167]
2025-10-09T21:45:11.884111Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341346078137177:5167]
2025-10-09T21:45:11.884121Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:11.884136Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:11.884137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:11.884139Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:11.884142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:11.884153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:11.884156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:11.884167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341346439949272:5638], now have 1 active actors on pipe
2025-10-09T21:45:11.884172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341346439949134:2432], now have 1 active actors on pipe
2025-10-09T21:45:11.884214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341346078137177:5167]
2025-10-09T21:45:11.884222Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:11.884557Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:11.884732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:11.884743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:11.884790Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|73815607-9e91e723-e868387d-e1faf2ae_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:11.884801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:11.884806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:11.884810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:11.884845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:11.884865Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:11.884888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:11.884308Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 468
2025-10-09T21:45:11.884345Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 468
2025-10-09T21:45:11.886217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:11.886243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:11.886273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:11.886819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 469 partition: 0 MaxSeqNo: 25 sessionId: src_id|73815607-9e91e723-e868387d-e1faf2ae_0
2025-10-09T21:45:11.890166Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ace7e90-6fe9012e-8decaa8d-d13cf10d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|73815607-9e91e723-e868387d-e1faf2ae_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:11.892895Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73815607-9e91e723-e868387d-e1faf2ae_0] Write session: try to update token
2025-10-09T21:45:11.892916Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73815607-9e91e723-e868387d-e1faf2ae_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:11.905910Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341346439949430:5210]
2025-10-09T21:45:11.906712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341346439949430:5210], now have 1 active actors on pipe
2025-10-09T21:45:11.906899Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 468, pipe: [1:7559341346439949430:5210]
2025-10-09T21:45:11.908852Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:11.908871Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:11.908874Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:11.908876Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:11.908885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341346078137177:5167]
2025-10-09T21:45:11.908890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341346078137177:5167]
2025-10-09T21:45:11.908893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341346078137177:5167]
2025-10-09T21:45:11.908938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:11.908946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:11.908949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:11.908954Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:11.908966Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:11.908970Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:11.909084Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:11.920936Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 469 sessionId: src_id|73815607-9e91e723-e868387d-e1faf2ae_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:11.923584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:12.116654Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:12.117002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:12.117021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:12.118478Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:12.118519Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:12.118535Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:12.118598Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 469 reason: error at write: Disconnected sessionId: src_id|73815607-9e91e723-e868387d-e1faf2ae_0
2025-10-09T21:45:12.118786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 469 sessionId: src_id|73815607-9e91e723-e868387d-e1faf2ae_0 is DEAD
2025-10-09T21:45:12.118935Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:12.118968Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341346439949430:5210]
2025-10-09T21:45:12.122001Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73815607-9e91e723-e868387d-e1faf2ae_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:12.122014Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73815607-9e91e723-e868387d-e1faf2ae_0] Write session will restart in 0.008701s
2025-10-09T21:45:12.122053Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73815607-9e91e723-e868387d-e1faf2ae_0] Write session: Do CDS request
2025-10-09T21:45:12.122065Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73815607-9e91e723-e868387d-e1faf2ae_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:12.134494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73815607-9e91e723-e868387d-e1faf2ae_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:12.138225Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:12.138259Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 470
2025-10-09T21:45:12.138487Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 470 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:12.138519Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 470 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:12.138524Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=470 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:12.138528Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:12.139156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 470 sessionId:  describe result for acl check
2025-10-09T21:45:12.139197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:12.139199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:12.139200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:12.139208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341350734916780:5643] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:12.139794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341350734916780:5643] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:12.147375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341350734916780:5643] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:12.155100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341350734916780:5643] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:12.155118Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341350734916780:5643] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:12.155123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341350734916780:5643] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:12.155133Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 470 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:12.168150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:12.171554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:12.171854Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:12.171864Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:12.171878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:12.171943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:12.172002Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341350373104772:5170]
2025-10-09T21:45:12.172652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:12.174429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:12.174510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:12.174933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:12.175058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:12.175060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:12.175062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:12.175063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:12.175064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:12.175065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:12.175066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.175068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.175069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.175076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:12.175087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:12.175091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:12.175095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:12.175099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:12.175103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:12.175107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.175111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.175115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.175126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:12.175135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:12.175143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:12.175148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:12.175154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:12.175159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:12.175163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:12.175168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:12.175175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:12.175180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:12.175187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:12.175193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:12.175199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.175205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.175209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.175214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.175219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.175224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.175258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:12.186588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:12.187857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:12.187875Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:12.187878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:12.187882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:12.187890Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 469 [2:7559341350373104772:5170]
2025-10-09T21:45:12.187907Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:12.187928Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:12.187934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:12.187946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:12.187948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:12.187969Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:12.187976Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:12.187983Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:12.188051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.188053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.188055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.188059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:12.188134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:12.188137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.188150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.188172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.188188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:12.188209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341350373104767:5170]
2025-10-09T21:45:12.188212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341350373104767:5170]
2025-10-09T21:45:12.188214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341350373104767:5170]
2025-10-09T21:45:12.188231Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:12.188249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:12.188264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:12.188267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:12.188311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341350373104767:5170]
2025-10-09T21:45:12.188313Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:12.188317Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:12.188321Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:12.188324Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:12.188331Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:12.188718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341350734916907:5643], now have 1 active actors on pipe
2025-10-09T21:45:12.188727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341350734916740:2432], now have 1 active actors on pipe
2025-10-09T21:45:12.189545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:12.189575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:12.189618Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cc07367c-5eb65f5b-da3474af-e5b59f76_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:12.189643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:12.189652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:12.189655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:12.189869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:12.189893Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:12.189922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:12.189225Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 469
2025-10-09T21:45:12.189260Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 469
2025-10-09T21:45:12.190212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:12.190226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:12.190261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:12.190381Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 470 partition: 0 MaxSeqNo: 25 sessionId: src_id|cc07367c-5eb65f5b-da3474af-e5b59f76_0
2025-10-09T21:45:12.194399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73815607-9e91e723-e868387d-e1faf2ae_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cc07367c-5eb65f5b-da3474af-e5b59f76_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:12.197106Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc07367c-5eb65f5b-da3474af-e5b59f76_0] Write session: try to update token
2025-10-09T21:45:12.197129Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc07367c-5eb65f5b-da3474af-e5b59f76_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:12.208068Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:12.208091Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:12.208094Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:12.208097Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:12.208109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341350373104767:5170]
2025-10-09T21:45:12.208124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341350373104767:5170]
2025-10-09T21:45:12.208129Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341350373104767:5170]
2025-10-09T21:45:12.208185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:12.208200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.208203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:12.208236Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:12.208244Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:12.208248Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:12.208367Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:12.218216Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 470 sessionId: src_id|cc07367c-5eb65f5b-da3474af-e5b59f76_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:12.220663Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:12.220748Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341350734917039:5210]
2025-10-09T21:45:12.221487Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 469, pipe: [1:7559341350734917039:5210]
2025-10-09T21:45:12.405580Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:12.406776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:12.406796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:12.406820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341350734917039:5210], now have 1 active actors on pipe
2025-10-09T21:45:12.407303Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:12.407864Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc07367c-5eb65f5b-da3474af-e5b59f76_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:12.407875Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc07367c-5eb65f5b-da3474af-e5b59f76_0] Write session will restart in 0.009323s
2025-10-09T21:45:12.407900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc07367c-5eb65f5b-da3474af-e5b59f76_0] Write session: Do CDS request
2025-10-09T21:45:12.407906Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc07367c-5eb65f5b-da3474af-e5b59f76_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:12.407222Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341350734917039:5210]
2025-10-09T21:45:12.407264Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:12.407267Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:12.407330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 470 reason: error at write: Disconnected sessionId: src_id|cc07367c-5eb65f5b-da3474af-e5b59f76_0
2025-10-09T21:45:12.407548Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 470 sessionId: src_id|cc07367c-5eb65f5b-da3474af-e5b59f76_0 is DEAD
2025-10-09T21:45:12.407685Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:12.419651Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc07367c-5eb65f5b-da3474af-e5b59f76_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:12.419874Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:12.419891Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 471
2025-10-09T21:45:12.420139Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 471 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:12.420171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 471 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:12.420177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=471 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:12.420181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:12.420768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 471 sessionId:  describe result for acl check
2025-10-09T21:45:12.420800Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:12.420802Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:12.420804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:12.420810Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341350734917056:5647] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:12.421378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341350734917056:5647] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:12.428616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341350734917056:5647] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:12.438437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341350734917056:5647] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:12.438459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341350734917056:5647] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:12.438464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341350734917056:5647] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:12.438475Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 471 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:12.465731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:12.468987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:12.469238Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:12.469251Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:12.469264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:12.469323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:12.469373Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341350373105069:5173]
2025-10-09T21:45:12.470061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:12.473095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:12.473188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:12.473543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:12.473605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:12.473606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:12.473607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:12.473608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:12.473610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:12.473611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:12.473613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.473614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.473615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.473624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:12.473635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:12.473638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:12.473643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:12.473646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:12.473650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:12.473654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.473657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.473661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.473670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:12.473679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:12.473686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:12.473691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:12.473697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:12.473701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:12.473705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:12.473710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:12.473715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:12.473720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:12.473724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:12.473729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:12.473735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.473741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.473745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.473749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.473756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.473761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.473789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:12.496102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:12.497447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:12.497476Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:12.497479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:12.497484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:12.497491Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 470 [2:7559341350373105069:5173]
2025-10-09T21:45:12.497506Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:12.497529Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:12.497533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:12.497536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:12.497542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:12.497564Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:12.497571Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:12.497579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:12.497653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.497655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.497657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.497660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:12.497747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:12.497757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.497772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.497792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.497821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:12.497845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341350373105060:5173]
2025-10-09T21:45:12.497852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341350373105060:5173]
2025-10-09T21:45:12.497854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341350373105060:5173]
2025-10-09T21:45:12.497866Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:12.497883Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:12.497898Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:12.497901Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:12.497950Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341350373105060:5173]
2025-10-09T21:45:12.497959Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:12.498484Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:12.498510Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:12.498513Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:12.498520Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:12.498799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341350734917149:5647], now have 1 active actors on pipe
2025-10-09T21:45:12.498814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341350734917042:2432], now have 1 active actors on pipe
2025-10-09T21:45:12.499673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:12.499690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:12.499735Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7b865d1e-8fedff89-9054187-c2e01200_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:12.499747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:12.499752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:12.499756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:12.499768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:12.499796Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:12.499819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:12.500972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:12.500996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:12.501028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:12.499081Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 470
2025-10-09T21:45:12.499120Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 470
2025-10-09T21:45:12.504002Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc07367c-5eb65f5b-da3474af-e5b59f76_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7b865d1e-8fedff89-9054187-c2e01200_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:12.503352Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 471 partition: 0 MaxSeqNo: 25 sessionId: src_id|7b865d1e-8fedff89-9054187-c2e01200_0
2025-10-09T21:45:12.507084Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b865d1e-8fedff89-9054187-c2e01200_0] Write session: try to update token
2025-10-09T21:45:12.507106Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b865d1e-8fedff89-9054187-c2e01200_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:12.509225Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341350734917339:5210]
2025-10-09T21:45:12.510371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341350734917339:5210], now have 1 active actors on pipe
2025-10-09T21:45:12.511507Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 470, pipe: [1:7559341350734917339:5210]
2025-10-09T21:45:12.529849Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:12.529872Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:12.529874Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:12.529877Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:12.529887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341350373105060:5173]
2025-10-09T21:45:12.529892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341350373105060:5173]
2025-10-09T21:45:12.529896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341350373105060:5173]
2025-10-09T21:45:12.529955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:12.529955Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:12.529961Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:12.529965Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:12.529965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.529968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:12.530144Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:12.550239Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 471 sessionId: src_id|7b865d1e-8fedff89-9054187-c2e01200_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:12.553139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:12.697487Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:12.697802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:12.697820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:12.698093Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:12.698215Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341350734917339:5210]
2025-10-09T21:45:12.698291Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:12.698295Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:12.698356Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 471 reason: error at write: Disconnected sessionId: src_id|7b865d1e-8fedff89-9054187-c2e01200_0
2025-10-09T21:45:12.698552Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 471 sessionId: src_id|7b865d1e-8fedff89-9054187-c2e01200_0 is DEAD
2025-10-09T21:45:12.698690Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:12.702302Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b865d1e-8fedff89-9054187-c2e01200_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:12.702315Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b865d1e-8fedff89-9054187-c2e01200_0] Write session will restart in 0.007195s
2025-10-09T21:45:12.702341Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b865d1e-8fedff89-9054187-c2e01200_0] Write session: Do CDS request
2025-10-09T21:45:12.702351Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b865d1e-8fedff89-9054187-c2e01200_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:12.710657Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b865d1e-8fedff89-9054187-c2e01200_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:12.710775Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:12.710794Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 472
2025-10-09T21:45:12.712623Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 472 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:12.712669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 472 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:12.712688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=472 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:12.712691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:12.714044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 472 sessionId:  describe result for acl check
2025-10-09T21:45:12.714081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:12.714083Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:12.714085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:12.714092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341350734917434:5659] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:12.714706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341350734917434:5659] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:12.718318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341350734917434:5659] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:12.723457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341350734917434:5659] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:12.723475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341350734917434:5659] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:12.723480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341350734917434:5659] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:12.723491Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 472 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:12.736657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:12.739604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:12.739865Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:12.739880Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:12.739895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:12.739960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:12.740023Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341350373105397:5187]
2025-10-09T21:45:12.740631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:12.741072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:12.741151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:12.741520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:12.741583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:12.741590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:12.741591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:12.741593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:12.741594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:12.741596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:12.741597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.741598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.741600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.741606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:12.741617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:12.741620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:12.741624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:12.741627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:12.741630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:12.741634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.741648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.741652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.741660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:12.741667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:12.741673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:12.741679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:12.741683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:12.741688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:12.741692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:12.741697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:12.741702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:12.741707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:12.741712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:12.741717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:12.741732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.741745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.741749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.741754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.741759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.741765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.741792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:12.749773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:12.751479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:12.751495Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:12.751498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:12.751502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:12.751509Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 471 [2:7559341350373105397:5187]
2025-10-09T21:45:12.751526Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:12.751545Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:12.751549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:12.751552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:12.751554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:12.751574Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:12.751580Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:12.751590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:12.751669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.751677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.751678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.751681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:12.751762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:12.752137Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 471
2025-10-09T21:45:12.751772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.751786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:12.752137Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 471
2025-10-09T21:45:12.751796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:12.751811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:12.751832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341350373105392:5187]
2025-10-09T21:45:12.751835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341350373105392:5187]
2025-10-09T21:45:12.751837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341350373105392:5187]
2025-10-09T21:45:12.751852Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:12.751867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:12.751881Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:12.751884Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:12.751897Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:12.751901Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:12.751904Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:12.751928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341350373105392:5187]
2025-10-09T21:45:12.751932Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:12.751998Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:12.752054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341350734917540:5659], now have 1 active actors on pipe
2025-10-09T21:45:12.752061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341350734917372:2432], now have 1 active actors on pipe
2025-10-09T21:45:12.752462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:12.752488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:12.752532Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cf82ccd2-9440a653-348a96c3-b5c2b07a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:12.752550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:12.752554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:12.752557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:12.752569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:12.752598Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:12.752623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:12.752908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:12.752923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:12.752954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:12.753098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 472 partition: 0 MaxSeqNo: 25 sessionId: src_id|cf82ccd2-9440a653-348a96c3-b5c2b07a_0
2025-10-09T21:45:12.753645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b865d1e-8fedff89-9054187-c2e01200_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cf82ccd2-9440a653-348a96c3-b5c2b07a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:12.756398Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf82ccd2-9440a653-348a96c3-b5c2b07a_0] Write session: try to update token
2025-10-09T21:45:12.756418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf82ccd2-9440a653-348a96c3-b5c2b07a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:12.772011Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:12.772029Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:12.772040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341350373105392:5187]
2025-10-09T21:45:12.772090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:12.772251Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:12.772340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:12.772484Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:12.772632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:12.772779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:12.772987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:12.773034Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:12.771239Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 472 sessionId: src_id|cf82ccd2-9440a653-348a96c3-b5c2b07a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:12.773855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:12.774048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:12.774065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:12.774099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:12.774215Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:12.787673Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:12.787693Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:12.787696Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:12.787698Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:12.787712Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341350373105392:5187]
2025-10-09T21:45:12.787716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341350373105392:5187]
2025-10-09T21:45:12.787720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341350373105392:5187]
2025-10-09T21:45:12.787754Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:12.787757Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:12.787761Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:12.787761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:12.787768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:12.787771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:12.787912Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:12.794634Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341350734917670:5210]
2025-10-09T21:45:12.796035Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 471, pipe: [1:7559341350734917670:5210]
2025-10-09T21:45:12.952470Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:12.953055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341350734917670:5210], now have 1 active actors on pipe
2025-10-09T21:45:12.961185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:12.961218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:12.961390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:12.961461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:12.961534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:12.961605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:12.961653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:12.961665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:12.961738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:12.961800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:12.961868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:12.961932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:12.961974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:12.961978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:12.964920Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341350734917670:5210]
2025-10-09T21:45:12.964960Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:12.964980Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:12.964982Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:12.965014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 472 reason: error at write: Disconnected sessionId: src_id|cf82ccd2-9440a653-348a96c3-b5c2b07a_0
2025-10-09T21:45:12.965240Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 472 sessionId: src_id|cf82ccd2-9440a653-348a96c3-b5c2b07a_0 is DEAD
2025-10-09T21:45:12.965430Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf82ccd2-9440a653-348a96c3-b5c2b07a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:12.965453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf82ccd2-9440a653-348a96c3-b5c2b07a_0] Write session will restart in 0.005300s
2025-10-09T21:45:12.965480Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf82ccd2-9440a653-348a96c3-b5c2b07a_0] Write session: Do CDS request
2025-10-09T21:45:12.965487Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf82ccd2-9440a653-348a96c3-b5c2b07a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:12.962170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:12.962246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:12.962305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:12.962366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:12.962408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:12.962411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:12.962489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:12.962554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:12.962624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:12.962686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:12.962727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:12.962731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:12.962801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:12.962864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:12.962924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:12.962992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:12.963034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:12.963038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:12.963111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:12.963176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:12.963243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:12.963306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:12.963350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:12.963353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:12.963420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:12.963484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:12.963546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:12.963608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:12.963653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:12.963657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:12.963728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:12.963789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:12.963849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:12.963913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:12.963953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:12.963956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:12.964022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:12.964079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:12.964142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:12.964208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:12.964251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:12.964254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:12.964323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:12.964380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:12.964441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:12.964503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:12.964543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:12.964547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:12.965784Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:12.971602Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf82ccd2-9440a653-348a96c3-b5c2b07a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:12.971845Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:12.971862Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 473
2025-10-09T21:45:12.972026Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 473 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:12.972072Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 473 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:12.972079Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=473 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:12.972098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:12.972509Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 473 sessionId:  describe result for acl check
2025-10-09T21:45:12.972545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:12.972552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:12.972554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:12.972560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341350734917695:5664] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:12.973140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341350734917695:5664] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:12.976716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341350734917695:5664] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:12.983447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341350734917695:5664] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:12.983484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341350734917695:5664] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:12.983490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341350734917695:5664] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:12.983500Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 473 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:12.996749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:13.000440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:13.000732Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:13.000745Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:13.000759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:13.000844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:13.000894Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341354668072988:5189]
2025-10-09T21:45:13.001525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:13.001960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:13.002096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:13.002498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:13.002562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:13.002563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:13.002565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:13.002566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:13.002567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:13.002569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:13.002570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.002571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.002572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.002579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:13.002592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:13.002596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:13.002600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:13.002604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:13.002608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:13.002612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.002617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.002621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.002630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:13.002640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:13.002646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:13.002651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:13.002656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:13.002662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:13.002666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:13.002672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:13.002677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:13.002682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:13.002710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:13.002715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:13.002722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.002728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.002733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.002738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.002742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.002748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.002782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:13.010173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:13.011678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:13.011699Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:13.011702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:13.011708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:13.011716Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 472 [2:7559341354668072988:5189]
2025-10-09T21:45:13.011735Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:13.011758Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:13.011761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:13.011765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:13.011769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:13.011790Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:13.011807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:13.011816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:13.011885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.011896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.011899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.011902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:13.012001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:13.012010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.012026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.012044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.012066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:13.012095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341350373105687:5189]
2025-10-09T21:45:13.012102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341350373105687:5189]
2025-10-09T21:45:13.012104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341350373105687:5189]
2025-10-09T21:45:13.012113Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:13.012133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:13.012149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:13.012158Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:13.012181Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:13.012194Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:13.012196Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:13.012205Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341350373105687:5189]
2025-10-09T21:45:13.012208Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:13.012214Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:13.012234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341350734917784:5664], now have 1 active actors on pipe
2025-10-09T21:45:13.012238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341350734917680:2432], now have 1 active actors on pipe
2025-10-09T21:45:13.012341Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 472
2025-10-09T21:45:13.012394Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 472
2025-10-09T21:45:13.012421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:13.012438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:13.012488Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7a4d7448-a50781c5-dd2ee55f-48a1315_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:13.012502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:13.012509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:13.012516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:13.012544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:13.012564Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:13.012602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:13.012836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:13.012862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:13.012887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:13.012986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 473 partition: 0 MaxSeqNo: 25 sessionId: src_id|7a4d7448-a50781c5-dd2ee55f-48a1315_0
2025-10-09T21:45:13.013450Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf82ccd2-9440a653-348a96c3-b5c2b07a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7a4d7448-a50781c5-dd2ee55f-48a1315_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:13.016141Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a4d7448-a50781c5-dd2ee55f-48a1315_0] Write session: try to update token
2025-10-09T21:45:13.016168Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a4d7448-a50781c5-dd2ee55f-48a1315_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:13.027704Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:13.027722Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:13.027732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341350373105687:5189]
2025-10-09T21:45:13.027783Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:13.027795Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:13.027972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:13.028142Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:13.028289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:13.028441Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:13.028621Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:13.028672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:13.035917Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 473 sessionId: src_id|7a4d7448-a50781c5-dd2ee55f-48a1315_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:13.038711Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:13.038978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:13.039003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:13.039062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:13.039623Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:13.042087Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:13.042105Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:13.042108Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:13.042110Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:13.042122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341350373105687:5189]
2025-10-09T21:45:13.042126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341350373105687:5189]
2025-10-09T21:45:13.042130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341350373105687:5189]
2025-10-09T21:45:13.042186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:13.042186Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:13.042194Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:13.042198Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:13.042198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.042202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:13.042368Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:13.064997Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341355029885280:5210]
2025-10-09T21:45:13.065466Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 472, pipe: [1:7559341355029885280:5210]
2025-10-09T21:45:13.207733Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:13.216783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:13.216815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:13.217003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.217073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.217136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.217205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.220692Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:13.220708Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:13.220742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 473 reason: error at write: Disconnected sessionId: src_id|7a4d7448-a50781c5-dd2ee55f-48a1315_0
2025-10-09T21:45:13.220933Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 473 sessionId: src_id|7a4d7448-a50781c5-dd2ee55f-48a1315_0 is DEAD
2025-10-09T21:45:13.217355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.217366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.217433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.217489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.217543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.221104Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a4d7448-a50781c5-dd2ee55f-48a1315_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:13.221126Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a4d7448-a50781c5-dd2ee55f-48a1315_0] Write session will restart in 0.006203s
2025-10-09T21:45:13.221151Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a4d7448-a50781c5-dd2ee55f-48a1315_0] Write session: Do CDS request
2025-10-09T21:45:13.221158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a4d7448-a50781c5-dd2ee55f-48a1315_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:13.221151Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:13.221204Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341355029885280:5210]
2025-10-09T21:45:13.217605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.217640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.217644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.217708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.217758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.217815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.217894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.217934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.217937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.218006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.218092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.218156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.218214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.218268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.218271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.218342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.218402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.218462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.218526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.218567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.218570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.218638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.218699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.218769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.218832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.218877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.218880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.218944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.219003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.219071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.219128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.219179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.219182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.219245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.219296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.219355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.219502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.219612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.219615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.219755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.219811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.219870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.219932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.219970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.219973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.220047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.220106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.220168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.220229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.220293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.220296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.220387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341355029885280:5210], now have 1 active actors on pipe
2025-10-09T21:45:13.220677Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:13.228764Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a4d7448-a50781c5-dd2ee55f-48a1315_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:13.228813Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:13.228832Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 474
2025-10-09T21:45:13.229025Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 474 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:13.229065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 474 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:13.229076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=474 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:13.229080Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:13.229535Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 474 sessionId:  describe result for acl check
2025-10-09T21:45:13.229571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:13.229573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:13.229575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:13.229581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341355029885296:5668] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:13.230163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341355029885296:5668] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:13.233512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341355029885296:5668] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:13.237884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341355029885296:5668] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:13.237905Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341355029885296:5668] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:13.237910Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341355029885296:5668] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:13.237920Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 474 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:13.249608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:13.254022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:13.254338Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:13.254353Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:13.254367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:13.254433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:13.254477Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341354668073285:5192]
2025-10-09T21:45:13.255128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:13.255547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:13.255609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:13.256634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:13.256716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:13.256723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:13.256724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:13.256725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:13.256726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:13.256728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:13.256729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.256730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.256731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.256737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:13.256750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:13.256754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:13.256759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:13.256763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:13.256768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:13.256771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.256776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.256779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.256787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:13.256801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:13.256812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:13.256818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:13.256822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:13.256828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:13.256832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:13.256838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:13.256843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:13.256848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:13.256852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:13.256857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:13.256863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.256867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.256872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.256879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.256884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.256889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.256919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:13.265338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:13.266525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:13.266541Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:13.266544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:13.266548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:13.266556Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 473 [2:7559341354668073285:5192]
2025-10-09T21:45:13.266573Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:13.266595Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:13.266598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:13.266602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:13.266604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:13.266635Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:13.266642Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:13.266649Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:13.266711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.266720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.266722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.266725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:13.266797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:13.266805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.266818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.266834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.266858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:13.266880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341355029885283:2432], now have 1 active actors on pipe
2025-10-09T21:45:13.266889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341355029885388:5668], now have 1 active actors on pipe
2025-10-09T21:45:13.266897Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341354668073280:5192]
2025-10-09T21:45:13.266900Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341354668073280:5192]
2025-10-09T21:45:13.266902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341354668073280:5192]
2025-10-09T21:45:13.266914Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:13.266929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:13.266930Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:13.266933Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:13.266935Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:13.266951Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:13.266954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:13.266957Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 473
2025-10-09T21:45:13.266981Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 473
2025-10-09T21:45:13.266996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341354668073280:5192]
2025-10-09T21:45:13.267005Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:13.267006Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:13.267039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:13.267044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:13.267081Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|612f7307-12ec9642-3998760f-47cb44c8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:13.267096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:13.267099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:13.267102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:13.267116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:13.267131Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:13.267149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:13.267513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:13.267526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:13.267544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:13.267712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 474 partition: 0 MaxSeqNo: 25 sessionId: src_id|612f7307-12ec9642-3998760f-47cb44c8_0
2025-10-09T21:45:13.268138Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a4d7448-a50781c5-dd2ee55f-48a1315_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|612f7307-12ec9642-3998760f-47cb44c8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:13.270752Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|612f7307-12ec9642-3998760f-47cb44c8_0] Write session: try to update token
2025-10-09T21:45:13.270774Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|612f7307-12ec9642-3998760f-47cb44c8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:13.283499Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:13.283519Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:13.283522Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:13.283525Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:13.283536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341354668073280:5192]
2025-10-09T21:45:13.283554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341354668073280:5192]
2025-10-09T21:45:13.283558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341354668073280:5192]
2025-10-09T21:45:13.283610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:13.283612Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:13.283616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.283619Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:13.283619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:13.283623Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:13.283775Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:13.292312Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 474 sessionId: src_id|612f7307-12ec9642-3998760f-47cb44c8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:13.295061Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:13.320988Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341355029885585:5210]
2025-10-09T21:45:13.321369Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 473, pipe: [1:7559341355029885585:5210]
2025-10-09T21:45:13.453947Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:13.454693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:13.454712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:13.454760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341355029885585:5210], now have 1 active actors on pipe
2025-10-09T21:45:13.455284Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:13.455298Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341355029885585:5210]
2025-10-09T21:45:13.455309Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:13.455346Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:13.455349Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:13.455414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 474 reason: error at write: Disconnected sessionId: src_id|612f7307-12ec9642-3998760f-47cb44c8_0
2025-10-09T21:45:13.455598Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 474 sessionId: src_id|612f7307-12ec9642-3998760f-47cb44c8_0 is DEAD
2025-10-09T21:45:13.455774Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|612f7307-12ec9642-3998760f-47cb44c8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:13.455786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|612f7307-12ec9642-3998760f-47cb44c8_0] Write session will restart in 0.005651s
2025-10-09T21:45:13.455814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|612f7307-12ec9642-3998760f-47cb44c8_0] Write session: Do CDS request
2025-10-09T21:45:13.455821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|612f7307-12ec9642-3998760f-47cb44c8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:13.463168Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|612f7307-12ec9642-3998760f-47cb44c8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:13.463246Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:13.463262Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 475
2025-10-09T21:45:13.463522Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 475 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:13.463559Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 475 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:13.463565Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=475 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:13.463568Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:13.464132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 475 sessionId:  describe result for acl check
2025-10-09T21:45:13.464172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:13.464179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:13.464181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:13.464187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341355029885602:5672] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:13.464863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341355029885602:5672] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:13.467755Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341355029885602:5672] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:13.472193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341355029885602:5672] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:13.472209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341355029885602:5672] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:13.472213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341355029885602:5672] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:13.472221Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 475 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:13.488468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:13.489217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:13.489487Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:13.489497Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:13.489512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:13.489586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:13.489641Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341354668073584:5195]
2025-10-09T21:45:13.490318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:13.490719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:13.490791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:13.491461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:13.491524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:13.491531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:13.491532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:13.491534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:13.491535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:13.491536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:13.491538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.491539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.491540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.491547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:13.491559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:13.491563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:13.491568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:13.491573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:13.491578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:13.491582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.491587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.491591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.491601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:13.491610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:13.491617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:13.491623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:13.491629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:13.491634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:13.491639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:13.491647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:13.491657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:13.491664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:13.491669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:13.491675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:13.491680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.491704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.491709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.491715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.491720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.491725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.491764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:13.499363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:13.500696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:13.500714Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:13.500718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:13.500722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:13.500730Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 474 [2:7559341354668073584:5195]
2025-10-09T21:45:13.500747Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:13.500769Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:13.500772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:13.500775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:13.500778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:13.500809Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:13.500815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:13.500823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:13.500897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.500905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.500908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.500911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:13.500990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:13.500997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.501013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.501034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.501060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:13.501120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341355029885718:5672], now have 1 active actors on pipe
2025-10-09T21:45:13.501131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341355029885588:2432], now have 1 active actors on pipe
2025-10-09T21:45:13.501140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341354668073579:5195]
2025-10-09T21:45:13.501143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341354668073579:5195]
2025-10-09T21:45:13.501145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341354668073579:5195]
2025-10-09T21:45:13.501161Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:13.501180Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:13.501189Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:13.501179Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 474
2025-10-09T21:45:13.501198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:13.501201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:13.501185Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 474
2025-10-09T21:45:13.501201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:13.501203Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:13.501244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341354668073579:5195]
2025-10-09T21:45:13.501258Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:13.501267Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:13.501282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:13.501287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:13.501339Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2a68f3-ef50d956-5caa1262-d79e1583_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:13.501355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:13.501359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:13.501361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:13.501374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:13.501395Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:13.501416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:13.501919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 475 partition: 0 MaxSeqNo: 25 sessionId: src_id|2a68f3-ef50d956-5caa1262-d79e1583_0
2025-10-09T21:45:13.501729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:13.501741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:13.501760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:13.502394Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|612f7307-12ec9642-3998760f-47cb44c8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2a68f3-ef50d956-5caa1262-d79e1583_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:13.504892Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a68f3-ef50d956-5caa1262-d79e1583_0] Write session: try to update token
2025-10-09T21:45:13.504910Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a68f3-ef50d956-5caa1262-d79e1583_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:13.518649Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:13.518671Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:13.518674Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:13.518677Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:13.518686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341354668073579:5195]
2025-10-09T21:45:13.518692Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341354668073579:5195]
2025-10-09T21:45:13.518696Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341354668073579:5195]
2025-10-09T21:45:13.518744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:13.518752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.518755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:13.518895Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:13.522695Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:13.522714Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:13.522719Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:13.529323Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 475 sessionId: src_id|2a68f3-ef50d956-5caa1262-d79e1583_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:13.532258Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:13.555450Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341355029885894:5210]
2025-10-09T21:45:13.555781Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 474, pipe: [1:7559341355029885894:5210]
2025-10-09T21:45:13.691281Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a68f3-ef50d956-5caa1262-d79e1583_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:13.691297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a68f3-ef50d956-5caa1262-d79e1583_0] Write session will restart in 0.007016s
2025-10-09T21:45:13.691320Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a68f3-ef50d956-5caa1262-d79e1583_0] Write session: Do CDS request
2025-10-09T21:45:13.691326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a68f3-ef50d956-5caa1262-d79e1583_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:13.690794Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:13.690842Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341355029885894:5210]
2025-10-09T21:45:13.690866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:13.690868Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:13.690923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 475 reason: error at write: Disconnected sessionId: src_id|2a68f3-ef50d956-5caa1262-d79e1583_0
2025-10-09T21:45:13.691205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 475 sessionId: src_id|2a68f3-ef50d956-5caa1262-d79e1583_0 is DEAD
2025-10-09T21:45:13.688898Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:13.689604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:13.689615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:13.689638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341355029885894:5210], now have 1 active actors on pipe
2025-10-09T21:45:13.690236Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:13.699571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a68f3-ef50d956-5caa1262-d79e1583_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:13.699901Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:13.699920Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 476
2025-10-09T21:45:13.700072Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 476 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:13.700117Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 476 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:13.700123Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=476 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:13.700133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:13.700617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 476 sessionId:  describe result for acl check
2025-10-09T21:45:13.700655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:13.700661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:13.700663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:13.700670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341355029885936:5684] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:13.701356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341355029885936:5684] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:13.704593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341355029885936:5684] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:13.711848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341355029885936:5684] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:13.711869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341355029885936:5684] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:13.711872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341355029885936:5684] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:13.711882Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 476 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:13.728254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:13.731769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:13.732033Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:13.732044Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:13.732056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:13.732155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:13.732200Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341354668073918:5209]
2025-10-09T21:45:13.732829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:13.733213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:13.733272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:13.733625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:13.733689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:13.733697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:13.733698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:13.733699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:13.733701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:13.733702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:13.733704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.733705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.733707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.733713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:13.733737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:13.733742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:13.733746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:13.733751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:13.733755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:13.733759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.733764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.733768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.733776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:13.733785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:13.733791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:13.733798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:13.733803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:13.733809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:13.733814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:13.733819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:13.733825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:13.733831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:13.733835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:13.733841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:13.733847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.733853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.733868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.733881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.733887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.733893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.733931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:13.741255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:13.742571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:13.742593Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:13.742597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:13.742601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:13.742610Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 475 [2:7559341354668073918:5209]
2025-10-09T21:45:13.742627Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:13.742649Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:13.742655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:13.742659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:13.742661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:13.742681Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:13.742697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:13.742706Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:13.742775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.742788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.742790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.742794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:13.742873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:13.742884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.742900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:13.742915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:13.742935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:13.742958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341355029886028:5684], now have 1 active actors on pipe
2025-10-09T21:45:13.742976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341354668073913:5209]
2025-10-09T21:45:13.742979Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341354668073913:5209]
2025-10-09T21:45:13.742982Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341354668073913:5209]
2025-10-09T21:45:13.742991Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:13.743007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341355029885921:2432], now have 1 active actors on pipe
2025-10-09T21:45:13.743013Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:13.743015Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:13.743019Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:13.743021Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:13.743030Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:13.743034Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:13.743075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341354668073913:5209]
2025-10-09T21:45:13.743085Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:13.743141Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:13.743352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:13.743364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:13.743398Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|94716216-8c471692-12bc06c-eb74b224_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:13.743407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:13.743424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:13.743427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:13.743439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:13.743451Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:13.743471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:13.743883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:13.743898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:13.743920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:13.743201Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 475
2025-10-09T21:45:13.743197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 475
2025-10-09T21:45:13.744049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 476 partition: 0 MaxSeqNo: 25 sessionId: src_id|94716216-8c471692-12bc06c-eb74b224_0
2025-10-09T21:45:13.744536Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a68f3-ef50d956-5caa1262-d79e1583_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|94716216-8c471692-12bc06c-eb74b224_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:13.747187Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94716216-8c471692-12bc06c-eb74b224_0] Write session: try to update token
2025-10-09T21:45:13.747209Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94716216-8c471692-12bc06c-eb74b224_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:13.756649Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:13.756671Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:13.756682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341354668073913:5209]
2025-10-09T21:45:13.756744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:13.756933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:13.757080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:13.757219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:13.757361Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:13.757551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:13.757601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:13.757817Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:13.767335Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 476 sessionId: src_id|94716216-8c471692-12bc06c-eb74b224_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:13.770183Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:13.770473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:13.770498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:13.770557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:13.770754Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:13.774271Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:13.774293Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:13.774296Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:13.774298Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:13.774309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341354668073913:5209]
2025-10-09T21:45:13.774313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341354668073913:5209]
2025-10-09T21:45:13.774317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341354668073913:5209]
2025-10-09T21:45:13.774362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:13.774370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:13.774374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:13.774503Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:13.774888Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:13.774894Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:13.774898Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:13.791695Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341355029886223:5210]
2025-10-09T21:45:13.794271Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 475, pipe: [1:7559341355029886223:5210]
2025-10-09T21:45:13.941530Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:13.950060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:13.950090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:13.950269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.950330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.950385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.950440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.950487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.950493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.950557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.950608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.950663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.950723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.950768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.950770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.950831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.950882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.950937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.950996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.951034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.951036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.951100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.951157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.951211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.951265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.951309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.951312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.951376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.951431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.951487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.951545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.951582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.951585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.951650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.951703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.951755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.951812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.951854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.951856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.951916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.951968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.952026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.952081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.952118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.952121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.952180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.952230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.952283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.952337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.952377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.952380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.952440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.952498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.952554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.952612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.952651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.952653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.952714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:13.952766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:13.952824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:13.952882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:13.952921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:13.952923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:13.952991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341355029886223:5210], now have 1 active actors on pipe
2025-10-09T21:45:13.954250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94716216-8c471692-12bc06c-eb74b224_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:13.954263Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94716216-8c471692-12bc06c-eb74b224_0] Write session will restart in 0.007955s
2025-10-09T21:45:13.954290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94716216-8c471692-12bc06c-eb74b224_0] Write session: Do CDS request
2025-10-09T21:45:13.954310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94716216-8c471692-12bc06c-eb74b224_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:13.953790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:13.953805Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:13.953836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 476 reason: error at write: Disconnected sessionId: src_id|94716216-8c471692-12bc06c-eb74b224_0
2025-10-09T21:45:13.954076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 476 sessionId: src_id|94716216-8c471692-12bc06c-eb74b224_0 is DEAD
2025-10-09T21:45:13.954259Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341355029886223:5210]
2025-10-09T21:45:13.954273Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:13.955222Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:13.965350Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94716216-8c471692-12bc06c-eb74b224_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:13.965784Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:13.965810Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 477
2025-10-09T21:45:13.966411Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 477 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:13.966454Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 477 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:13.966460Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=477 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:13.966464Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:13.967693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 477 sessionId:  describe result for acl check
2025-10-09T21:45:13.967748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:13.967750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:13.967751Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:13.967757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341355029886246:5688] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:13.968352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341355029886246:5688] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:13.972504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341355029886246:5688] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:13.980451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341355029886246:5688] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:13.980471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341355029886246:5688] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:13.980476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341355029886246:5688] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:13.980486Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 477 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:14.001551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:14.006366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:14.006637Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:14.006649Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:14.006664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:14.006731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:14.006780Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341358963041513:5211]
2025-10-09T21:45:14.007442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:14.008108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:14.008189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:14.008579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:14.008647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:14.008655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:14.008657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:14.008658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:14.008659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:14.008660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:14.008662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.008663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.008665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.008671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:14.008682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:14.008686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:14.008690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:14.008694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:14.008698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:14.008702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.008706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.008710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.008719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:14.008728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:14.008737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:14.008743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:14.008748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:14.008770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:14.008774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:14.008785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:14.008790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:14.008795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:14.008805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:14.008810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:14.008821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.008827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.008838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.008843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.008848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.008853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.008889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:14.016078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:14.017231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:14.017252Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:14.017255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:14.017260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:14.017267Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 476 [2:7559341358963041513:5211]
2025-10-09T21:45:14.017283Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:14.017306Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:14.017316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:14.017320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:14.017323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:14.017345Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:14.017360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:14.017373Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:14.017450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.017458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.017461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.017464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:14.017587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:14.017597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.017612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.017624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.017640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:14.017667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341355029886232:2432], now have 1 active actors on pipe
2025-10-09T21:45:14.017671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341355029886345:5688], now have 1 active actors on pipe
2025-10-09T21:45:14.017681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341358963041508:5211]
2025-10-09T21:45:14.017684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341358963041508:5211]
2025-10-09T21:45:14.017687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341358963041508:5211]
2025-10-09T21:45:14.017705Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:14.017722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:14.017738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:14.017742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:14.017796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341358963041508:5211]
2025-10-09T21:45:14.017799Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:14.018253Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 476
2025-10-09T21:45:14.018358Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 476
2025-10-09T21:45:14.018872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:14.018882Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:14.018885Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:14.018894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:14.019409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:14.019421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:14.019463Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|75150896-1ada765-fe8ca886-131507a3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:14.019476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:14.019481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:14.019485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:14.019500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:14.019516Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:14.019535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:14.022730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:14.022746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:14.022777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:14.023649Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 477 partition: 0 MaxSeqNo: 25 sessionId: src_id|75150896-1ada765-fe8ca886-131507a3_0
2025-10-09T21:45:14.024201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94716216-8c471692-12bc06c-eb74b224_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|75150896-1ada765-fe8ca886-131507a3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:14.026845Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|75150896-1ada765-fe8ca886-131507a3_0] Write session: try to update token
2025-10-09T21:45:14.026871Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|75150896-1ada765-fe8ca886-131507a3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:14.037179Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:14.037199Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:14.037202Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:14.037204Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:14.037214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341358963041508:5211]
2025-10-09T21:45:14.037219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341358963041508:5211]
2025-10-09T21:45:14.037223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341358963041508:5211]
2025-10-09T21:45:14.037270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:14.037280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.037289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:14.037428Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:14.041292Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:14.041311Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:14.041315Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:14.054147Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341359324853836:5210]
2025-10-09T21:45:14.054504Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 476, pipe: [1:7559341359324853836:5210]
2025-10-09T21:45:14.064341Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 477 sessionId: src_id|75150896-1ada765-fe8ca886-131507a3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:14.066911Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:14.215161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:14.215177Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:14.215255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 477 reason: error at write: Disconnected sessionId: src_id|75150896-1ada765-fe8ca886-131507a3_0
2025-10-09T21:45:14.215527Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341359324853836:5210]
2025-10-09T21:45:14.215600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|75150896-1ada765-fe8ca886-131507a3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:14.215613Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|75150896-1ada765-fe8ca886-131507a3_0] Write session will restart in 0.006457s
2025-10-09T21:45:14.215637Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|75150896-1ada765-fe8ca886-131507a3_0] Write session: Do CDS request
2025-10-09T21:45:14.215643Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|75150896-1ada765-fe8ca886-131507a3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:14.215570Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:14.215879Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 477 sessionId: src_id|75150896-1ada765-fe8ca886-131507a3_0 is DEAD
2025-10-09T21:45:14.214440Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:14.214770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341359324853836:5210], now have 1 active actors on pipe
2025-10-09T21:45:14.214799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:14.214808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:14.216171Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:14.223623Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|75150896-1ada765-fe8ca886-131507a3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:14.223734Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:14.223752Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 478
2025-10-09T21:45:14.224064Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 478 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:14.224121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 478 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:14.224133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=478 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:14.224136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:14.224657Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 478 sessionId:  describe result for acl check
2025-10-09T21:45:14.224707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:14.224709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:14.224714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:14.224720Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341359324853853:5693] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:14.225267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341359324853853:5693] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:14.229049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341359324853853:5693] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:14.234346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341359324853853:5693] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:14.234365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341359324853853:5693] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:14.234370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341359324853853:5693] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:14.234381Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 478 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:14.248507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:14.253501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:14.253762Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:14.253773Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:14.253785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:14.253840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:14.253893Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341358963041814:5214]
2025-10-09T21:45:14.254603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:14.255073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:14.255122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:14.255489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:14.255533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:14.255535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:14.255536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:14.255537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:14.255538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:14.255540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:14.255541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.255542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.255544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.255549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:14.255561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:14.255566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:14.255571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:14.255575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:14.255579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:14.255583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.255587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.255591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.255599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:14.255608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:14.255614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:14.255620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:14.255625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:14.255630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:14.255634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:14.255639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:14.255644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:14.255649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:14.255654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:14.255661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:14.255667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.255672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.255677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.255683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.255688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.255693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.255724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:14.264569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:14.265661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:14.265679Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:14.265683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:14.265687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:14.265694Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 477 [2:7559341358963041814:5214]
2025-10-09T21:45:14.265710Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:14.265735Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:14.265742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:14.265745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:14.265748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:14.265767Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:14.265778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:14.265786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:14.265853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.265860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.265863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.265866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:14.265957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:14.265966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.265979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.265991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.266010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:14.266087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341359324853965:5693], now have 1 active actors on pipe
2025-10-09T21:45:14.266091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341359324853839:2432], now have 1 active actors on pipe
2025-10-09T21:45:14.266100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341358963041809:5214]
2025-10-09T21:45:14.266105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341358963041809:5214]
2025-10-09T21:45:14.266107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341358963041809:5214]
2025-10-09T21:45:14.266120Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:14.266258Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 477
2025-10-09T21:45:14.266308Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 477
2025-10-09T21:45:14.266135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:14.266155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:14.266159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:14.266203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341358963041809:5214]
2025-10-09T21:45:14.266206Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:14.266476Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:14.266480Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:14.266482Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:14.266488Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:14.266694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:14.266701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:14.266737Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|37b54916-6f997770-8800db6a-5a6c9e48_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:14.266747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:14.266751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:14.266755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:14.266766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:14.266778Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:14.266796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:14.267594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:14.267608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:14.267631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:14.267780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 478 partition: 0 MaxSeqNo: 25 sessionId: src_id|37b54916-6f997770-8800db6a-5a6c9e48_0
2025-10-09T21:45:14.268193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|75150896-1ada765-fe8ca886-131507a3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|37b54916-6f997770-8800db6a-5a6c9e48_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:14.270947Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|37b54916-6f997770-8800db6a-5a6c9e48_0] Write session: try to update token
2025-10-09T21:45:14.270966Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|37b54916-6f997770-8800db6a-5a6c9e48_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:14.287761Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:14.287782Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:14.287784Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:14.287786Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:14.287796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341358963041809:5214]
2025-10-09T21:45:14.287800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341358963041809:5214]
2025-10-09T21:45:14.287803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341358963041809:5214]
2025-10-09T21:45:14.287847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:14.287859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.287862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:14.287873Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:14.287879Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:14.287883Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:14.288001Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:14.287491Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 478 sessionId: src_id|37b54916-6f997770-8800db6a-5a6c9e48_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:14.290148Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:14.315817Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341359324854145:5210]
2025-10-09T21:45:14.316312Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 477, pipe: [1:7559341359324854145:5210]
2025-10-09T21:45:14.516010Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:14.517005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:14.517017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:14.517066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341359324854145:5210], now have 1 active actors on pipe
2025-10-09T21:45:14.518014Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341359324854145:5210]
2025-10-09T21:45:14.518090Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:14.518099Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:14.518104Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:14.518162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 478 reason: error at write: Disconnected sessionId: src_id|37b54916-6f997770-8800db6a-5a6c9e48_0
2025-10-09T21:45:14.518359Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 478 sessionId: src_id|37b54916-6f997770-8800db6a-5a6c9e48_0 is DEAD
2025-10-09T21:45:14.518696Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37b54916-6f997770-8800db6a-5a6c9e48_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:14.518708Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37b54916-6f997770-8800db6a-5a6c9e48_0] Write session will restart in 0.008863s
2025-10-09T21:45:14.518732Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37b54916-6f997770-8800db6a-5a6c9e48_0] Write session: Do CDS request
2025-10-09T21:45:14.518741Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37b54916-6f997770-8800db6a-5a6c9e48_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:14.518981Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:14.530589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|37b54916-6f997770-8800db6a-5a6c9e48_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:14.531267Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:14.531288Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 479
2025-10-09T21:45:14.531767Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 479 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:14.531803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 479 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:14.531809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=479 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:14.531813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:14.532289Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 479 sessionId:  describe result for acl check
2025-10-09T21:45:14.532319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:14.532320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:14.532322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:14.532328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341359324854163:5697] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:14.532858Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341359324854163:5697] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:14.537238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341359324854163:5697] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:14.555708Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341359324854163:5697] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:14.555724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341359324854163:5697] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:14.555729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341359324854163:5697] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:14.555738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 479 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:14.596003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:14.600368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:14.600873Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:14.600877Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:14.600890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:14.600947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:14.600987Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341358963042130:5217]
2025-10-09T21:45:14.601528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:14.602079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:14.602125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:14.602434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:14.602498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:14.602499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:14.602500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:14.602501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:14.602502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:14.602503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:14.602505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.602506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.602507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.602513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:14.602521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:14.602525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:14.602529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:14.602532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:14.602536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:14.602540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.602543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.602547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.602554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:14.602561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:14.602568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:14.602572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:14.602575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:14.602580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:14.602583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:14.602587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:14.602591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:14.602595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:14.602599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:14.602603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:14.602608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.602612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.602616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.602621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.602625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.602630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.602654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:14.611507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:14.612962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:14.612968Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:14.612970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:14.612975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:14.612982Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 478 [2:7559341358963042130:5217]
2025-10-09T21:45:14.612995Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:14.613014Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:14.613018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:14.613021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:14.613024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:14.613041Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:14.613048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:14.613055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:14.613115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.613117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.613119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.613122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:14.613195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:14.613197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.613209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.613221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.613238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:14.613258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341358963042124:5217]
2025-10-09T21:45:14.613260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341358963042124:5217]
2025-10-09T21:45:14.613263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341358963042124:5217]
2025-10-09T21:45:14.613275Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:14.613290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:14.613304Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:14.613306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:14.613343Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341358963042124:5217]
2025-10-09T21:45:14.613345Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:14.614044Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 478
2025-10-09T21:45:14.614089Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 478
2025-10-09T21:45:14.613648Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:14.613655Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:14.613657Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:14.613663Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:14.614544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341359324854148:2432], now have 1 active actors on pipe
2025-10-09T21:45:14.614553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341359324854280:5697], now have 1 active actors on pipe
2025-10-09T21:45:14.614573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:14.614578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:14.614606Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|176bad09-a88fe4e8-7cde2802-ac36a38d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:14.614619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:14.614623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:14.614626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:14.614637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:14.614651Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:14.614667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:14.615277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:14.615289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:14.615308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:14.615470Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 479 partition: 0 MaxSeqNo: 25 sessionId: src_id|176bad09-a88fe4e8-7cde2802-ac36a38d_0
2025-10-09T21:45:14.617602Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37b54916-6f997770-8800db6a-5a6c9e48_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|176bad09-a88fe4e8-7cde2802-ac36a38d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:14.618420Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341359324854456:5210]
2025-10-09T21:45:14.619439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341359324854456:5210], now have 1 active actors on pipe
2025-10-09T21:45:14.619490Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 478, pipe: [1:7559341359324854456:5210]
2025-10-09T21:45:14.620205Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|176bad09-a88fe4e8-7cde2802-ac36a38d_0] Write session: try to update token
2025-10-09T21:45:14.620222Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|176bad09-a88fe4e8-7cde2802-ac36a38d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:14.630497Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:14.630934Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:14.630943Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:14.630945Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:14.630954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341358963042124:5217]
2025-10-09T21:45:14.630958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341358963042124:5217]
2025-10-09T21:45:14.630963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341358963042124:5217]
2025-10-09T21:45:14.630995Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:14.631000Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:14.631003Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:14.631010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:14.631017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.631020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:14.631153Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:14.646951Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 479 sessionId: src_id|176bad09-a88fe4e8-7cde2802-ac36a38d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:14.649578Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:14.798838Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:14.799887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:14.799899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:14.800327Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:14.800681Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:14.800728Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:14.800731Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:14.800786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 479 reason: error at write: Disconnected sessionId: src_id|176bad09-a88fe4e8-7cde2802-ac36a38d_0
2025-10-09T21:45:14.800968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 479 sessionId: src_id|176bad09-a88fe4e8-7cde2802-ac36a38d_0 is DEAD
2025-10-09T21:45:14.801105Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341359324854456:5210]
2025-10-09T21:45:14.801348Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|176bad09-a88fe4e8-7cde2802-ac36a38d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:14.801356Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|176bad09-a88fe4e8-7cde2802-ac36a38d_0] Write session will restart in 0.005938s
2025-10-09T21:45:14.801381Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|176bad09-a88fe4e8-7cde2802-ac36a38d_0] Write session: Do CDS request
2025-10-09T21:45:14.801390Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|176bad09-a88fe4e8-7cde2802-ac36a38d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:14.808623Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|176bad09-a88fe4e8-7cde2802-ac36a38d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:14.810240Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:14.810261Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 480
2025-10-09T21:45:14.810435Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 480 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:14.810465Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 480 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:14.810477Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=480 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:14.810480Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:14.811099Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 480 sessionId:  describe result for acl check
2025-10-09T21:45:14.811146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:14.811148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:14.811150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:14.811155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341359324854502:5709] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:14.811733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341359324854502:5709] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:14.819774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341359324854502:5709] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:14.828002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341359324854502:5709] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:14.828021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341359324854502:5709] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:14.828026Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341359324854502:5709] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:14.828037Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 480 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:14.859815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:14.863449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:14.863646Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:14.863654Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:14.863663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:14.863714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:14.863748Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341358963042458:5231]
2025-10-09T21:45:14.864133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:14.864493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:14.864536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:14.864791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:14.864847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:14.864848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:14.864849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:14.864850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:14.864850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:14.864851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:14.864852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.864852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.864853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.864856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:14.864864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:14.864866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:14.864868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:14.864870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:14.864872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:14.864874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.864876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.864878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.864884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:14.864889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:14.864893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:14.864897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:14.864899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:14.864902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:14.864904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:14.864907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:14.864911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:14.864914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:14.864916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:14.864919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:14.864922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.864924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.864927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.864931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.864935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.864940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.864968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:14.873129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:14.874383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:14.874401Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:14.874404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:14.874409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:14.874415Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 479 [2:7559341358963042458:5231]
2025-10-09T21:45:14.874432Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:14.874453Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:14.874457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:14.874471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:14.874473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:14.874492Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:14.874499Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:14.874506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:14.874576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.874582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.874584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.874587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:14.875758Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 479
2025-10-09T21:45:14.875802Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 479
2025-10-09T21:45:14.874661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:14.874668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.874683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:14.874697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:14.874713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:14.874733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341358963042453:5231]
2025-10-09T21:45:14.874735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341358963042453:5231]
2025-10-09T21:45:14.874738Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341358963042453:5231]
2025-10-09T21:45:14.874750Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:14.874764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:14.874779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:14.874782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:14.874836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341358963042453:5231]
2025-10-09T21:45:14.874838Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:14.874984Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:14.874987Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:14.874989Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:14.874995Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:14.876813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341359324854490:2432], now have 1 active actors on pipe
2025-10-09T21:45:14.876821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341359324854572:5709], now have 1 active actors on pipe
2025-10-09T21:45:14.876839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:14.876845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:14.876880Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|12c9bbd9-5f4a0e2a-ce285b2c-979f2017_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:14.876890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:14.876895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:14.876898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:14.876910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:14.876924Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:14.876943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:14.877293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:14.877300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:14.877315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:14.877436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 480 partition: 0 MaxSeqNo: 25 sessionId: src_id|12c9bbd9-5f4a0e2a-ce285b2c-979f2017_0
2025-10-09T21:45:14.877815Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|176bad09-a88fe4e8-7cde2802-ac36a38d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|12c9bbd9-5f4a0e2a-ce285b2c-979f2017_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:14.880727Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12c9bbd9-5f4a0e2a-ce285b2c-979f2017_0] Write session: try to update token
2025-10-09T21:45:14.880749Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12c9bbd9-5f4a0e2a-ce285b2c-979f2017_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:14.895185Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:14.895202Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:14.895205Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:14.895208Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:14.895221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341358963042453:5231]
2025-10-09T21:45:14.895227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341358963042453:5231]
2025-10-09T21:45:14.895232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341358963042453:5231]
2025-10-09T21:45:14.895281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:14.895290Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:14.895295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:14.895297Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:14.895298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:14.895300Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:14.895448Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:14.901012Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341359324854799:5210]
2025-10-09T21:45:14.901561Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 479, pipe: [1:7559341359324854799:5210]
2025-10-09T21:45:14.902887Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 480 sessionId: src_id|12c9bbd9-5f4a0e2a-ce285b2c-979f2017_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:14.905512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:15.073024Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:15.073874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341359324854799:5210], now have 1 active actors on pipe
2025-10-09T21:45:15.073908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:15.073917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:15.074928Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:15.074976Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341359324854799:5210]
2025-10-09T21:45:15.074996Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:15.075003Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:15.075057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 480 reason: error at write: Disconnected sessionId: src_id|12c9bbd9-5f4a0e2a-ce285b2c-979f2017_0
2025-10-09T21:45:15.075249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 480 sessionId: src_id|12c9bbd9-5f4a0e2a-ce285b2c-979f2017_0 is DEAD
2025-10-09T21:45:15.075675Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:15.078285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12c9bbd9-5f4a0e2a-ce285b2c-979f2017_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:15.078308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12c9bbd9-5f4a0e2a-ce285b2c-979f2017_0] Write session will restart in 0.007265s
2025-10-09T21:45:15.078337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12c9bbd9-5f4a0e2a-ce285b2c-979f2017_0] Write session: Do CDS request
2025-10-09T21:45:15.078344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12c9bbd9-5f4a0e2a-ce285b2c-979f2017_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:15.086647Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12c9bbd9-5f4a0e2a-ce285b2c-979f2017_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:15.086748Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:15.086768Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 481
2025-10-09T21:45:15.090289Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 481 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:15.090344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 481 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:15.090364Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=481 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:15.090373Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:15.091112Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 481 sessionId:  describe result for acl check
2025-10-09T21:45:15.091151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:15.091156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:15.091158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:15.091164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341363619822186:5714] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:15.091706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341363619822186:5714] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:15.094545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341363619822186:5714] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:15.099049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341363619822186:5714] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:15.099067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341363619822186:5714] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:15.099071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341363619822186:5714] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:15.099081Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 481 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:15.104660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:15.107706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:15.107979Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:15.107990Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:15.108020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:15.108101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:15.108145Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341363258010055:5233]
2025-10-09T21:45:15.108761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:15.109176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:15.109239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:15.109596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:15.109663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:15.109673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:15.109675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:15.109676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:15.109677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:15.109678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:15.109680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.109682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.109683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.109689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:15.109701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:15.109706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:15.109711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:15.109716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:15.109720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:15.109724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.109728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.109733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.109742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:15.109751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:15.109758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:15.109763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:15.109767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:15.109771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:15.109775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:15.109779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:15.109784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:15.109789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:15.109794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:15.109801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:15.109806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.109813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.109817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.109822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.109828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.109834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.109865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:15.117633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:15.119032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:15.119048Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:15.119051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:15.119056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:15.119064Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 480 [2:7559341363258010055:5233]
2025-10-09T21:45:15.119081Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:15.119104Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:15.119107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:15.119111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:15.119114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:15.119135Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:15.119147Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:15.119170Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:15.119241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.119249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.119251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.119254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:15.119339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:15.119345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.119359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.119371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.119394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:15.119422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341363258010050:5233]
2025-10-09T21:45:15.119424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341363258010050:5233]
2025-10-09T21:45:15.119427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341363258010050:5233]
2025-10-09T21:45:15.119436Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:15.119628Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 480
2025-10-09T21:45:15.119674Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 480
2025-10-09T21:45:15.119455Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:15.119454Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:15.119458Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:15.119460Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:15.119472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:15.119475Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:15.119489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341363619822354:5714], now have 1 active actors on pipe
2025-10-09T21:45:15.119492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341363619822111:2432], now have 1 active actors on pipe
2025-10-09T21:45:15.119536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341363258010050:5233]
2025-10-09T21:45:15.119545Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:15.119578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:15.119847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:15.119854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:15.119889Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|62c9ee72-4143fd6e-41e5796-c0ac5927_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:15.119902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:15.119906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:15.119909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:15.119922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:15.119948Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:15.119968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:15.120323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:15.120351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:15.120378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:15.120605Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 481 partition: 0 MaxSeqNo: 25 sessionId: src_id|62c9ee72-4143fd6e-41e5796-c0ac5927_0
2025-10-09T21:45:15.121073Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12c9bbd9-5f4a0e2a-ce285b2c-979f2017_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|62c9ee72-4143fd6e-41e5796-c0ac5927_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:15.123846Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62c9ee72-4143fd6e-41e5796-c0ac5927_0] Write session: try to update token
2025-10-09T21:45:15.123872Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62c9ee72-4143fd6e-41e5796-c0ac5927_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:15.138386Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:15.138399Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:15.138401Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:15.138403Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:15.138413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341363258010050:5233]
2025-10-09T21:45:15.138417Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341363258010050:5233]
2025-10-09T21:45:15.138421Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341363258010050:5233]
2025-10-09T21:45:15.138467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:15.138474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.138476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:15.138484Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:15.138492Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:15.138496Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:15.138734Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:15.144607Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 481 sessionId: src_id|62c9ee72-4143fd6e-41e5796-c0ac5927_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:15.147366Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:15.174894Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341363619822417:5210]
2025-10-09T21:45:15.175173Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 480, pipe: [1:7559341363619822417:5210]
2025-10-09T21:45:15.338019Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:15.339079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:15.339094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:15.339155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341363619822417:5210], now have 1 active actors on pipe
2025-10-09T21:45:15.340369Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62c9ee72-4143fd6e-41e5796-c0ac5927_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:15.340379Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62c9ee72-4143fd6e-41e5796-c0ac5927_0] Write session will restart in 0.008931s
2025-10-09T21:45:15.340412Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62c9ee72-4143fd6e-41e5796-c0ac5927_0] Write session: Do CDS request
2025-10-09T21:45:15.340418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62c9ee72-4143fd6e-41e5796-c0ac5927_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:15.339950Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341363619822417:5210]
2025-10-09T21:45:15.339982Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:15.339985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:15.340042Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 481 reason: error at write: Disconnected sessionId: src_id|62c9ee72-4143fd6e-41e5796-c0ac5927_0
2025-10-09T21:45:15.340218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 481 sessionId: src_id|62c9ee72-4143fd6e-41e5796-c0ac5927_0 is DEAD
2025-10-09T21:45:15.340526Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:15.340718Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:15.350924Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:15.350943Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 482
2025-10-09T21:45:15.351205Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62c9ee72-4143fd6e-41e5796-c0ac5927_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:15.351444Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 482 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:15.351482Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 482 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:15.351488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=482 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:15.351492Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:15.351942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 482 sessionId:  describe result for acl check
2025-10-09T21:45:15.351974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:15.351976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:15.351978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:15.351984Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341363619822435:5720] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:15.352759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341363619822435:5720] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:15.358710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341363619822435:5720] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:15.366486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341363619822435:5720] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:15.366505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341363619822435:5720] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:15.366510Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341363619822435:5720] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:15.366520Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 482 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:15.386285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:15.393573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:15.393859Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:15.393871Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:15.393888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:15.393953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:15.394141Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341363258010363:5237]
2025-10-09T21:45:15.394806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:15.395353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:15.395406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:15.395776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:15.395843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:15.395849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:15.395850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:15.395852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:15.395854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:15.395855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:15.395857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.395858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.395860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.395866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:15.395879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:15.395884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:15.395889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:15.395893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:15.395898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:15.395903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.395907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.395912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.395920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:15.395929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:15.395937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:15.395954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:15.395960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:15.395965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:15.395970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:15.395976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:15.395983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:15.395989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:15.395993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:15.396000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:15.396005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.396011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.396016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.396022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.396028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.396034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.396069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:15.409434Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 481
2025-10-09T21:45:15.409500Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 481
2025-10-09T21:45:15.407038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:15.408809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:15.408819Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:15.408822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:15.408828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:15.408836Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 481 [2:7559341363258010363:5237]
2025-10-09T21:45:15.408855Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:15.408878Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:15.408883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:15.408887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:15.408889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:15.408909Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:15.408916Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:15.408924Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:15.408996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.408999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.409001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.409004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:15.409087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:15.409093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.409110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.409122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.409139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:15.409162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341363258010358:5237]
2025-10-09T21:45:15.409165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341363258010358:5237]
2025-10-09T21:45:15.409167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341363258010358:5237]
2025-10-09T21:45:15.409176Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:15.409187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:15.409190Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:15.409193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:15.409193Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:15.409208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:15.409211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:15.409224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341363619822423:2432], now have 1 active actors on pipe
2025-10-09T21:45:15.409228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341363619822574:5720], now have 1 active actors on pipe
2025-10-09T21:45:15.409271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341363258010358:5237]
2025-10-09T21:45:15.409285Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:15.409653Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:15.409878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:15.409889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:15.409923Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d9e27423-a7c22156-d1e1aed0-3db6174b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:15.409933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:15.409938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:15.409941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:15.409954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:15.409968Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:15.409988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:15.411136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:15.411154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:15.411180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:15.411361Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 482 partition: 0 MaxSeqNo: 25 sessionId: src_id|d9e27423-a7c22156-d1e1aed0-3db6174b_0
2025-10-09T21:45:15.411793Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62c9ee72-4143fd6e-41e5796-c0ac5927_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d9e27423-a7c22156-d1e1aed0-3db6174b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:15.414468Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d9e27423-a7c22156-d1e1aed0-3db6174b_0] Write session: try to update token
2025-10-09T21:45:15.414488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d9e27423-a7c22156-d1e1aed0-3db6174b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:15.428985Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:15.429005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:15.429009Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:15.429011Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:15.429023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341363258010358:5237]
2025-10-09T21:45:15.429028Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341363258010358:5237]
2025-10-09T21:45:15.429032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341363258010358:5237]
2025-10-09T21:45:15.429084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:15.429092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.429094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:15.429247Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:15.431316Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:15.431323Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:15.431328Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:15.433005Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 482 sessionId: src_id|d9e27423-a7c22156-d1e1aed0-3db6174b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:15.435643Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:15.441480Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341363619822735:5210]
2025-10-09T21:45:15.442935Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 481, pipe: [1:7559341363619822735:5210]
2025-10-09T21:45:15.676273Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:15.676549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:15.676560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:15.676608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341363619822735:5210], now have 1 active actors on pipe
2025-10-09T21:45:15.677097Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341363619822735:5210]
2025-10-09T21:45:15.677143Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:15.677190Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:15.677193Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:15.677247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 482 reason: error at write: Disconnected sessionId: src_id|d9e27423-a7c22156-d1e1aed0-3db6174b_0
2025-10-09T21:45:15.677464Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 482 sessionId: src_id|d9e27423-a7c22156-d1e1aed0-3db6174b_0 is DEAD
2025-10-09T21:45:15.677737Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:15.682122Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9e27423-a7c22156-d1e1aed0-3db6174b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:15.682140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9e27423-a7c22156-d1e1aed0-3db6174b_0] Write session will restart in 0.006439s
2025-10-09T21:45:15.682168Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9e27423-a7c22156-d1e1aed0-3db6174b_0] Write session: Do CDS request
2025-10-09T21:45:15.682175Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9e27423-a7c22156-d1e1aed0-3db6174b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:15.690554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d9e27423-a7c22156-d1e1aed0-3db6174b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:15.690706Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:15.690732Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 483
2025-10-09T21:45:15.691662Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 483 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:15.691721Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 483 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:15.691730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=483 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:15.691734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:15.692253Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 483 sessionId:  describe result for acl check
2025-10-09T21:45:15.692295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:15.692298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:15.692301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:15.692308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341363619822763:5725] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:15.692878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341363619822763:5725] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:15.698560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341363619822763:5725] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:15.702868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341363619822763:5725] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:15.702901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341363619822763:5725] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:15.702906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341363619822763:5725] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:15.702917Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 483 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:15.730114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:15.730648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:15.730884Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:15.730896Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:15.730909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:15.730958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:15.731004Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341363258010704:5247]
2025-10-09T21:45:15.731581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:15.732100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:15.732145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:15.732444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:15.732499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:15.732500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:15.732502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:15.732503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:15.732504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:15.732506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:15.732507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.732508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.732509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.732514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:15.732525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:15.732529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:15.732533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:15.732536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:15.732540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:15.732543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.732546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.732549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.732556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:15.732564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:15.732570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:15.732575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:15.732579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:15.732583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:15.732587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:15.732591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:15.732595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:15.732599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:15.732603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:15.732608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:15.732612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.732617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.732621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.732625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.732629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.732634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.732659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:15.744608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:15.746630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:15.746648Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:15.746651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:15.746656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:15.746665Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 482 [2:7559341363258010704:5247]
2025-10-09T21:45:15.746682Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:15.746704Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:15.746708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:15.746712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:15.746714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:15.746738Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:15.746943Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:15.746957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:15.747036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.747045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.747048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.747055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:15.747138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:15.747146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.747163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:15.747196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:15.747221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:15.747243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341363258010699:5247]
2025-10-09T21:45:15.747260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341363258010699:5247]
2025-10-09T21:45:15.747263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341363258010699:5247]
2025-10-09T21:45:15.747281Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:15.747308Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:15.747326Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:15.747334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:15.747364Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:15.747372Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:15.747374Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:15.747383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341363258010699:5247]
2025-10-09T21:45:15.747387Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:15.747437Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:15.747529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341363619822837:5725], now have 1 active actors on pipe
2025-10-09T21:45:15.747543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341363619822744:2432], now have 1 active actors on pipe
2025-10-09T21:45:15.747849Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 482
2025-10-09T21:45:15.747861Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 482
2025-10-09T21:45:15.748209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:15.748221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:15.748254Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b0512617-d25e7429-abc32d84-b2b21b0f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:15.748275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:15.748284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:15.748287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:15.748301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:15.748319Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:15.748338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:15.748578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:15.748589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:15.748614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:15.748737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 483 partition: 0 MaxSeqNo: 25 sessionId: src_id|b0512617-d25e7429-abc32d84-b2b21b0f_0
2025-10-09T21:45:15.750184Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9e27423-a7c22156-d1e1aed0-3db6174b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b0512617-d25e7429-abc32d84-b2b21b0f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:15.753169Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0512617-d25e7429-abc32d84-b2b21b0f_0] Write session: try to update token
2025-10-09T21:45:15.753195Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0512617-d25e7429-abc32d84-b2b21b0f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:15.761635Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:15.761652Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:15.761665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341363258010699:5247]
2025-10-09T21:45:15.761710Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:15.761726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:15.761915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:15.762089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:15.762249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:15.762393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:15.762592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:15.762642Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:15.778826Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:15.779022Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:15.779031Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:15.779033Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:15.779044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341363258010699:5247]
2025-10-09T21:45:15.779048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341363258010699:5247]
2025-10-09T21:45:15.779053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341363258010699:5247]
2025-10-09T21:45:15.779113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:15.779122Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:15.779127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:15.779129Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:15.779129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:15.779133Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:15.779562Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:15.782245Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341363619823069:5210]
2025-10-09T21:45:15.785839Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 482, pipe: [1:7559341363619823069:5210]
2025-10-09T21:45:15.785862Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 483 sessionId: src_id|b0512617-d25e7429-abc32d84-b2b21b0f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:15.788506Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:15.959637Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:15.960170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341363619823069:5210], now have 1 active actors on pipe
2025-10-09T21:45:15.960211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:15.960220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:15.960466Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:15.960480Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:15.960487Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:15.960538Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341363619823069:5210]
2025-10-09T21:45:15.960539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 483 reason: error at write: Disconnected sessionId: src_id|b0512617-d25e7429-abc32d84-b2b21b0f_0
2025-10-09T21:45:15.960725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 483 sessionId: src_id|b0512617-d25e7429-abc32d84-b2b21b0f_0 is DEAD
2025-10-09T21:45:15.960871Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0512617-d25e7429-abc32d84-b2b21b0f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:15.960883Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0512617-d25e7429-abc32d84-b2b21b0f_0] Write session will restart in 0.007359s
2025-10-09T21:45:15.960907Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0512617-d25e7429-abc32d84-b2b21b0f_0] Write session: Do CDS request
2025-10-09T21:45:15.960927Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0512617-d25e7429-abc32d84-b2b21b0f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:15.960495Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:15.969602Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0512617-d25e7429-abc32d84-b2b21b0f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:15.969675Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:15.969691Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 484
2025-10-09T21:45:15.969803Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 484 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:15.969834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 484 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:15.969845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=484 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:15.969848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:15.974527Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 484 sessionId:  describe result for acl check
2025-10-09T21:45:15.974584Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:15.974587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:15.974589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:15.974595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341363619823113:5736] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:15.975163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341363619823113:5736] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:15.985590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341363619823113:5736] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:15.990423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341363619823113:5736] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:15.990442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341363619823113:5736] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:15.990446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341363619823113:5736] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:15.990456Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 484 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:16.003512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:16.008482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:16.008859Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:16.008872Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:16.008888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:16.008962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:16.009008Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341367552978322:5255]
2025-10-09T21:45:16.009620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:16.009996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:16.010087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:16.010483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:16.010568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.010574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.010576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.010577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.010578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.010580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.010581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.010583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.010584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.010591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.010604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.010608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.010613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.010617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.010621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.010625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.010629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.010634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.010643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.010652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.010658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.010664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.010675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.010681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.010686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.010692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.010698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.010704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.010709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.010714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.010726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.010732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.010737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.010744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.010749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.010756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.010794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:16.019246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:16.020860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:16.020882Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:16.020886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:16.020891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:16.020900Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 483 [2:7559341367552978322:5255]
2025-10-09T21:45:16.020918Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:16.020977Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:16.020987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:16.020991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:16.020993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:16.021015Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:16.021022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:16.021031Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:16.021105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.021113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.021116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.021119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:16.021205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:16.021214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.021246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.021263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.021288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:16.021312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341367552978312:5255]
2025-10-09T21:45:16.021319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341367552978312:5255]
2025-10-09T21:45:16.021322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341367552978312:5255]
2025-10-09T21:45:16.021342Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:16.021366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:16.021390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:16.021393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:16.021440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341367552978312:5255]
2025-10-09T21:45:16.021449Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:16.021490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341363619823248:5736], now have 1 active actors on pipe
2025-10-09T21:45:16.021498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341363619823091:2432], now have 1 active actors on pipe
2025-10-09T21:45:16.021644Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:16.021653Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:16.021656Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:16.021663Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:16.021636Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 483
2025-10-09T21:45:16.021647Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 483
2025-10-09T21:45:16.022021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:16.022050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:16.022093Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|30f6dc89-6c3ddf93-e6874998-a3801841_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:16.022111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:16.022115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:16.022118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:16.022132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:16.022147Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:16.022174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:16.022476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:16.022498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:16.022574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:16.022780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 484 partition: 0 MaxSeqNo: 25 sessionId: src_id|30f6dc89-6c3ddf93-e6874998-a3801841_0
2025-10-09T21:45:16.023255Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0512617-d25e7429-abc32d84-b2b21b0f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|30f6dc89-6c3ddf93-e6874998-a3801841_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:16.025849Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|30f6dc89-6c3ddf93-e6874998-a3801841_0] Write session: try to update token
2025-10-09T21:45:16.025870Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|30f6dc89-6c3ddf93-e6874998-a3801841_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:16.040341Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:16.040363Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:16.040365Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:16.040367Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:16.040378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341367552978312:5255]
2025-10-09T21:45:16.040382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341367552978312:5255]
2025-10-09T21:45:16.040385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341367552978312:5255]
2025-10-09T21:45:16.040437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:16.040445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.040448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:16.040592Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:16.042191Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 484 sessionId: src_id|30f6dc89-6c3ddf93-e6874998-a3801841_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:16.043696Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:16.043710Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:16.043714Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:16.044759Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:16.060645Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341367914790704:5210]
2025-10-09T21:45:16.061478Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 483, pipe: [1:7559341367914790704:5210]
2025-10-09T21:45:16.209675Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:16.210167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:16.210185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:16.210246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341367914790704:5210], now have 1 active actors on pipe
2025-10-09T21:45:16.210515Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:16.210631Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:16.210648Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:16.210722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 484 reason: error at write: Disconnected sessionId: src_id|30f6dc89-6c3ddf93-e6874998-a3801841_0
2025-10-09T21:45:16.210934Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 484 sessionId: src_id|30f6dc89-6c3ddf93-e6874998-a3801841_0 is DEAD
2025-10-09T21:45:16.211058Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:16.211085Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341367914790704:5210]
2025-10-09T21:45:16.211304Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30f6dc89-6c3ddf93-e6874998-a3801841_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:16.211318Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30f6dc89-6c3ddf93-e6874998-a3801841_0] Write session will restart in 0.008337s
2025-10-09T21:45:16.211346Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30f6dc89-6c3ddf93-e6874998-a3801841_0] Write session: Do CDS request
2025-10-09T21:45:16.211354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30f6dc89-6c3ddf93-e6874998-a3801841_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:16.220974Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|30f6dc89-6c3ddf93-e6874998-a3801841_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:16.221103Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:16.221123Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 485
2025-10-09T21:45:16.221632Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 485 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:16.221688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 485 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:16.221694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=485 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:16.221698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:16.222593Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 485 sessionId:  describe result for acl check
2025-10-09T21:45:16.222716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:16.222726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:16.222728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:16.222738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341367914790720:5741] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:16.223600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341367914790720:5741] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:16.226854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341367914790720:5741] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:16.232653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:16.232898Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341367914790720:5741] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:16.232917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341367914790720:5741] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:16.232922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341367914790720:5741] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:16.232932Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 485 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:16.233320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:16.233565Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:16.233575Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:16.233587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:16.233643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:16.233690Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341367552978383:5258]
2025-10-09T21:45:16.234371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:16.234792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:16.234883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:16.235249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:16.235330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.235336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.235337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.235339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.235340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.235341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.235343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.235344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.235345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.235351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.235363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.235367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.235371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.235374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.235378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.235393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.235396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.235399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.235407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.235415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.235421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.235427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.235432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.235437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.235441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.235446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.235450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.235455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.235459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.235465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.235470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.235476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.235480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.235485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.235489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.235509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.235537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:16.243045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:16.244718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:16.244732Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:16.244735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:16.244740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:16.244747Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 484 [2:7559341367552978383:5258]
2025-10-09T21:45:16.244768Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:16.244792Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:16.244795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:16.244798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:16.244800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:16.244819Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:16.244825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:16.244833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:16.244904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.244912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.244914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.244917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:16.244994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:16.245001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.245016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.245029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.245046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:16.245069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341367552978377:5258]
2025-10-09T21:45:16.245077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341367552978377:5258]
2025-10-09T21:45:16.245079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341367552978377:5258]
2025-10-09T21:45:16.245094Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:16.245117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:16.245154Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:16.245167Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:16.245216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341367552978377:5258]
2025-10-09T21:45:16.245224Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:16.245244Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:16.245247Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:16.245248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341367914790769:5741], now have 1 active actors on pipe
2025-10-09T21:45:16.245249Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:16.245252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341367914790707:2432], now have 1 active actors on pipe
2025-10-09T21:45:16.245257Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:16.245995Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 484
2025-10-09T21:45:16.246057Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 484
2025-10-09T21:45:16.246128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:16.246142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:16.246175Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|57f70714-a2a88321-6975e74d-d7b15bc7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:16.246186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:16.246191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:16.246194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:16.246232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:16.246250Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:16.246273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:16.247565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:16.247578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:16.247607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:16.247803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 485 partition: 0 MaxSeqNo: 25 sessionId: src_id|57f70714-a2a88321-6975e74d-d7b15bc7_0
2025-10-09T21:45:16.248388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30f6dc89-6c3ddf93-e6874998-a3801841_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|57f70714-a2a88321-6975e74d-d7b15bc7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:16.251035Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57f70714-a2a88321-6975e74d-d7b15bc7_0] Write session: try to update token
2025-10-09T21:45:16.251057Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57f70714-a2a88321-6975e74d-d7b15bc7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:16.276034Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:16.276051Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:16.276053Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:16.276055Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:16.276067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341367552978377:5258]
2025-10-09T21:45:16.276071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341367552978377:5258]
2025-10-09T21:45:16.276074Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341367552978377:5258]
2025-10-09T21:45:16.276116Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:16.276118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:16.276126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.276128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:16.276129Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:16.276133Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:16.273495Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 485 sessionId: src_id|57f70714-a2a88321-6975e74d-d7b15bc7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:16.276278Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:16.276260Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:16.311059Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341367914790781:5210]
2025-10-09T21:45:16.311386Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 484, pipe: [1:7559341367914790781:5210]
2025-10-09T21:45:16.443635Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:16.443937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:16.443948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:16.443995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341367914790781:5210], now have 1 active actors on pipe
2025-10-09T21:45:16.444414Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:16.444700Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:16.444714Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:16.444772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 485 reason: error at write: Disconnected sessionId: src_id|57f70714-a2a88321-6975e74d-d7b15bc7_0
2025-10-09T21:45:16.444965Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 485 sessionId: src_id|57f70714-a2a88321-6975e74d-d7b15bc7_0 is DEAD
2025-10-09T21:45:16.445095Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:16.445126Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341367914790781:5210]
2025-10-09T21:45:16.445434Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57f70714-a2a88321-6975e74d-d7b15bc7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:16.445444Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57f70714-a2a88321-6975e74d-d7b15bc7_0] Write session will restart in 0.008421s
2025-10-09T21:45:16.445469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57f70714-a2a88321-6975e74d-d7b15bc7_0] Write session: Do CDS request
2025-10-09T21:45:16.445476Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57f70714-a2a88321-6975e74d-d7b15bc7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:16.455642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57f70714-a2a88321-6975e74d-d7b15bc7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:16.455723Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:16.455743Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 486
2025-10-09T21:45:16.455943Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 486 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:16.455995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 486 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:16.456002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=486 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:16.456005Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:16.456531Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 486 sessionId:  describe result for acl check
2025-10-09T21:45:16.456574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:16.456582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:16.456584Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:16.456589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341367914790797:5745] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:16.457216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341367914790797:5745] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:16.459845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341367914790797:5745] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:16.463575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:16.463352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341367914790797:5745] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:16.463369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341367914790797:5745] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:16.463373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341367914790797:5745] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:16.463382Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 486 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:16.466627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:16.466917Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:16.466926Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:16.466940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:16.467010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:16.467061Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341367552978444:5261]
2025-10-09T21:45:16.467672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:16.468089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:16.468139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:16.468527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:16.468590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.468597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.468598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.468600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.468602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.468603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.468604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.468606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.468607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.468613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.468624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.468629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.468639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.468653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.468657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.468660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.468669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.468671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.468680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.468692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.468698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.468703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.468712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.468716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.468720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.468725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.468734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.468739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.468746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.468750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.468756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.468762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.468767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.468772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.468780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.468784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.468822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:16.476355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:16.477853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:16.477868Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:16.477871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:16.477875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:16.477882Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 485 [2:7559341367552978444:5261]
2025-10-09T21:45:16.477899Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:16.477920Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:16.477923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:16.477926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:16.477929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:16.477949Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:16.477959Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:16.477985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:16.478084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.478092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.478096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.478099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:16.478177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:16.478186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.478200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.478245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.478267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:16.478436Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 485
2025-10-09T21:45:16.478497Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 485
2025-10-09T21:45:16.478294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341367914790784:2432], now have 1 active actors on pipe
2025-10-09T21:45:16.478304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341367914790836:5745], now have 1 active actors on pipe
2025-10-09T21:45:16.478313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341367552978438:5261]
2025-10-09T21:45:16.478316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341367552978438:5261]
2025-10-09T21:45:16.478318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341367552978438:5261]
2025-10-09T21:45:16.478336Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:16.478352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:16.478367Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:16.478368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:16.478370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:16.478380Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:16.478382Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:16.478413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341367552978438:5261]
2025-10-09T21:45:16.478421Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:16.478427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:16.478598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:16.478625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:16.478705Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|34317392-c1e00d5a-4a89b3bf-d16e662c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:16.478719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:16.478724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:16.478727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:16.478738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:16.478757Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:16.478795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:16.479073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:16.479082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:16.479105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:16.479283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 486 partition: 0 MaxSeqNo: 25 sessionId: src_id|34317392-c1e00d5a-4a89b3bf-d16e662c_0
2025-10-09T21:45:16.479691Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57f70714-a2a88321-6975e74d-d7b15bc7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|34317392-c1e00d5a-4a89b3bf-d16e662c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:16.482549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|34317392-c1e00d5a-4a89b3bf-d16e662c_0] Write session: try to update token
2025-10-09T21:45:16.482572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|34317392-c1e00d5a-4a89b3bf-d16e662c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:16.501398Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:16.501418Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:16.501429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341367552978438:5261]
2025-10-09T21:45:16.502301Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 486 sessionId: src_id|34317392-c1e00d5a-4a89b3bf-d16e662c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:16.501491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:16.501505Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:16.501677Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:16.501835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:16.501989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:16.502153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:16.502379Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:16.502426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:16.505036Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:16.505909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:16.505931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:16.505973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:16.506127Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:16.512700Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:16.512716Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:16.512718Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:16.512721Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:16.512732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341367552978438:5261]
2025-10-09T21:45:16.512736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341367552978438:5261]
2025-10-09T21:45:16.512739Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341367552978438:5261]
2025-10-09T21:45:16.512788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:16.512795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.512798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:16.512798Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:16.512805Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:16.512809Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:16.512944Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:16.546826Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341367914790850:5210]
2025-10-09T21:45:16.548203Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 485, pipe: [1:7559341367914790850:5210]
2025-10-09T21:45:16.676386Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:16.685036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:16.685070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:16.685243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:16.685309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:16.685371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:16.685432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:16.685474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:16.685479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:16.685556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:16.685616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:16.685675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:16.685735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:16.685778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:16.685780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:16.685844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:16.685898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:16.685955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:16.686011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:16.686159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:16.686168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:16.686246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:16.686309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:16.686371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:16.686452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:16.686500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:16.686509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:16.686583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:16.686646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:16.686715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:16.688740Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:16.688814Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341367914790850:5210]
2025-10-09T21:45:16.688843Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:16.688846Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:16.688873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 486 reason: error at write: Disconnected sessionId: src_id|34317392-c1e00d5a-4a89b3bf-d16e662c_0
2025-10-09T21:45:16.689080Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 486 sessionId: src_id|34317392-c1e00d5a-4a89b3bf-d16e662c_0 is DEAD
2025-10-09T21:45:16.686784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:16.686835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:16.686838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:16.686905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:16.686967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:16.687027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:16.687091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:16.687135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:16.687138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:16.687203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:16.687260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:16.687324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:16.687391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:16.687434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:16.687437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:16.687507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:16.687563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:16.687623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:16.687685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:16.687727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:16.687730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:16.687792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:16.687847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:16.687905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:16.687967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:16.688007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:16.688010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:16.688075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:16.688130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:16.688189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:16.688247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:16.688287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:16.688289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:16.688355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341367914790850:5210], now have 1 active actors on pipe
2025-10-09T21:45:16.689534Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:16.696204Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34317392-c1e00d5a-4a89b3bf-d16e662c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:16.696219Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34317392-c1e00d5a-4a89b3bf-d16e662c_0] Write session will restart in 0.008434s
2025-10-09T21:45:16.696243Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34317392-c1e00d5a-4a89b3bf-d16e662c_0] Write session: Do CDS request
2025-10-09T21:45:16.696249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34317392-c1e00d5a-4a89b3bf-d16e662c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:16.705000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:16.705573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|34317392-c1e00d5a-4a89b3bf-d16e662c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:16.705665Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:16.705684Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 487
2025-10-09T21:45:16.706191Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 487 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:16.706241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 487 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:16.706246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=487 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:16.706250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:16.706902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 487 sessionId:  describe result for acl check
2025-10-09T21:45:16.706945Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:16.706955Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:16.706957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:16.706963Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341367914790879:5752] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:16.707548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341367914790879:5752] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:16.709137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:16.709390Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:16.709401Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:16.709414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:16.709487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:16.709537Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341367552978514:5267]
2025-10-09T21:45:16.710122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:16.710933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:16.710986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:16.710974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341367914790879:5752] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:16.711359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:16.711417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.711420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.711421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.711422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.711423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.711424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.711425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.711426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.711427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.711432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.711441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.711444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.711448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.711452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.711456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.711459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.711461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.711464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.711471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.711478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.711483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.711488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.711492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.711496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.711499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.711503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.711506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.711510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.711514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.711518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.711522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.711526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.711531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.711536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.711541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.711545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.711570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:16.722135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341367914790879:5752] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:16.722157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341367914790879:5752] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:16.722162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341367914790879:5752] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:16.722172Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 487 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:16.723494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:16.724810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:16.724823Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:16.724825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:16.724829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:16.724836Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 486 [2:7559341367552978514:5267]
2025-10-09T21:45:16.724850Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:16.724867Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:16.724871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:16.724874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:16.724876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:16.724895Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:16.724900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:16.724907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:16.724966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.724973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.724975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.724978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:16.725046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:16.725054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.725065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.725075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.725095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:16.725118Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341367552978508:5267]
2025-10-09T21:45:16.725124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341367552978508:5267]
2025-10-09T21:45:16.725126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341367552978508:5267]
2025-10-09T21:45:16.725133Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:16.725139Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:16.725141Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:16.725143Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:16.725151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341367914790916:5752], now have 1 active actors on pipe
2025-10-09T21:45:16.725157Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:16.725170Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:16.725176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:16.725182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341367914790858:2432], now have 1 active actors on pipe
2025-10-09T21:45:16.725218Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341367552978508:5267]
2025-10-09T21:45:16.725229Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:16.725250Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 486
2025-10-09T21:45:16.725235Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:16.725255Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 486
2025-10-09T21:45:16.725451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:16.725462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:16.725490Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8e3c3922-5a6ab02-e897a3c6-f9c6bbc0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:16.725503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:16.725507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:16.725510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:16.725521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:16.725539Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:16.725554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:16.725742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:16.725774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:16.725797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:16.725922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 487 partition: 0 MaxSeqNo: 25 sessionId: src_id|8e3c3922-5a6ab02-e897a3c6-f9c6bbc0_0
2025-10-09T21:45:16.726447Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34317392-c1e00d5a-4a89b3bf-d16e662c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8e3c3922-5a6ab02-e897a3c6-f9c6bbc0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:16.729032Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3c3922-5a6ab02-e897a3c6-f9c6bbc0_0] Write session: try to update token
2025-10-09T21:45:16.729048Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3c3922-5a6ab02-e897a3c6-f9c6bbc0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:16.743208Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:16.743226Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:16.743229Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:16.743231Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:16.743240Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341367552978508:5267]
2025-10-09T21:45:16.743245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341367552978508:5267]
2025-10-09T21:45:16.743249Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341367552978508:5267]
2025-10-09T21:45:16.743296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:16.743304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.743307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:16.743315Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:16.743321Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:16.743325Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:16.743449Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:16.749273Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 487 sessionId: src_id|8e3c3922-5a6ab02-e897a3c6-f9c6bbc0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:16.751857Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:16.789925Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341367914790925:5210]
2025-10-09T21:45:16.790314Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 486, pipe: [1:7559341367914790925:5210]
2025-10-09T21:45:16.904242Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:16.905053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:16.905068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:16.905118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341367914790925:5210], now have 1 active actors on pipe
2025-10-09T21:45:16.905356Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:16.905958Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3c3922-5a6ab02-e897a3c6-f9c6bbc0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:16.905972Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3c3922-5a6ab02-e897a3c6-f9c6bbc0_0] Write session will restart in 0.009472s
2025-10-09T21:45:16.906001Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3c3922-5a6ab02-e897a3c6-f9c6bbc0_0] Write session: Do CDS request
2025-10-09T21:45:16.906008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3c3922-5a6ab02-e897a3c6-f9c6bbc0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:16.905496Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341367914790925:5210]
2025-10-09T21:45:16.905531Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:16.905614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 487 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8e3c3922-5a6ab02-e897a3c6-f9c6bbc0_0
2025-10-09T21:45:16.905800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 487 sessionId: src_id|8e3c3922-5a6ab02-e897a3c6-f9c6bbc0_0 is DEAD
2025-10-09T21:45:16.905947Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:16.916653Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3c3922-5a6ab02-e897a3c6-f9c6bbc0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:16.916713Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:16.916732Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 488
2025-10-09T21:45:16.916935Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 488 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:16.916999Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 488 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:16.917009Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=488 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:16.917014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:16.917210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:16.917564Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 488 sessionId:  describe result for acl check
2025-10-09T21:45:16.917604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:16.917606Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:16.917608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:16.917614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341367914790977:5763] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:16.918192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341367914790977:5763] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:16.921059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341367914790977:5763] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:16.920817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:16.921088Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:16.921093Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:16.921108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:16.921188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:16.921228Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341367552978600:5277]
2025-10-09T21:45:16.921826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:16.922257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:16.922308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:16.922642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:16.922689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.922690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.922691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.922693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.922694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.922695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.922696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.922698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.922699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.922704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.922714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.922718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.922722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.922725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.922730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.922733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.922736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.922740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.922746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.922754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:16.922760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.922765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:16.922769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.922774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:16.922778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.922783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:16.922788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.922793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:16.922796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.922801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:16.922805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.922810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.922814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.922819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.922822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.922827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.922855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:16.931421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341367914790977:5763] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:16.931442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341367914790977:5763] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:16.931447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341367914790977:5763] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:16.931457Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 488 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:16.933715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:16.935526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:16.935542Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:16.935545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:16.935550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:16.935557Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 487 [2:7559341367552978600:5277]
2025-10-09T21:45:16.935571Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:16.935591Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:16.935595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:16.935598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:16.935600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:16.935620Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:16.935626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:16.935633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:16.935700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.935705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.935709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.935712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:16.935790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:16.935797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.935811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:16.935824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:16.935846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:16.935869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341367914791012:5763], now have 1 active actors on pipe
2025-10-09T21:45:16.935879Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341367552978594:5277]
2025-10-09T21:45:16.935881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341367552978594:5277]
2025-10-09T21:45:16.935883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341367552978594:5277]
2025-10-09T21:45:16.935897Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:16.935911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341367914790956:2432], now have 1 active actors on pipe
2025-10-09T21:45:16.935920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:16.935926Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:16.935930Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:16.935932Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:16.935934Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:16.935937Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:16.935978Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341367552978594:5277]
2025-10-09T21:45:16.935987Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:16.935997Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:16.936054Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 487
2025-10-09T21:45:16.936106Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 487
2025-10-09T21:45:16.936284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:16.936298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:16.936334Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|652136ef-b3835684-ab0329ab-1c5c211e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:16.936344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:16.936348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:16.936350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:16.936362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:16.936384Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:16.936406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:16.936646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:16.936657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:16.936691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:16.937303Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3c3922-5a6ab02-e897a3c6-f9c6bbc0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|652136ef-b3835684-ab0329ab-1c5c211e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:16.936852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 488 partition: 0 MaxSeqNo: 25 sessionId: src_id|652136ef-b3835684-ab0329ab-1c5c211e_0
2025-10-09T21:45:16.940311Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|652136ef-b3835684-ab0329ab-1c5c211e_0] Write session: try to update token
2025-10-09T21:45:16.940330Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|652136ef-b3835684-ab0329ab-1c5c211e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:16.964630Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 488 sessionId: src_id|652136ef-b3835684-ab0329ab-1c5c211e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:16.964210Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:16.964228Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:16.964241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341367552978594:5277]
2025-10-09T21:45:16.964296Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:16.964301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:16.964495Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:16.964647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:16.964801Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:16.964953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:16.965162Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:16.965220Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:16.967529Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:16.967551Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:16.967554Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:16.967556Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:16.967565Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341367552978594:5277]
2025-10-09T21:45:16.967569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341367552978594:5277]
2025-10-09T21:45:16.967573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341367552978594:5277]
2025-10-09T21:45:16.967553Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:16.967613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:16.967620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:16.967623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:16.967622Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:16.967628Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:16.967632Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:16.967770Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:17.006235Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341372209758321:5210]
2025-10-09T21:45:17.010771Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 487, pipe: [1:7559341372209758321:5210]
2025-10-09T21:45:17.155549Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:17.156008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:17.156026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:17.156052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341372209758321:5210], now have 1 active actors on pipe
2025-10-09T21:45:17.156317Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341372209758321:5210]
2025-10-09T21:45:17.156347Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:17.156348Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:17.156351Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:17.156418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 488 reason: error at write: Disconnected sessionId: src_id|652136ef-b3835684-ab0329ab-1c5c211e_0
2025-10-09T21:45:17.156671Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 488 sessionId: src_id|652136ef-b3835684-ab0329ab-1c5c211e_0 is DEAD
2025-10-09T21:45:17.156367Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:17.158291Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|652136ef-b3835684-ab0329ab-1c5c211e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:17.158304Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|652136ef-b3835684-ab0329ab-1c5c211e_0] Write session will restart in 0.009812s
2025-10-09T21:45:17.158334Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|652136ef-b3835684-ab0329ab-1c5c211e_0] Write session: Do CDS request
2025-10-09T21:45:17.158357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|652136ef-b3835684-ab0329ab-1c5c211e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:17.169665Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|652136ef-b3835684-ab0329ab-1c5c211e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:17.169723Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:17.169742Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 489
2025-10-09T21:45:17.169971Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 489 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:17.170016Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 489 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:17.170022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=489 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:17.170026Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:17.170688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:17.170475Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 489 sessionId:  describe result for acl check
2025-10-09T21:45:17.170514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:17.170518Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:17.170520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:17.170527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341372209758349:5768] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:17.171144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341372209758349:5768] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:17.174473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:17.174783Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:17.174788Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:17.174803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:17.174899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:17.174944Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341371847945959:5280]
2025-10-09T21:45:17.175613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:17.176312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341372209758349:5768] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:17.176116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:17.176188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:17.176579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:17.176633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:17.176635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:17.176636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:17.176637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:17.176638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:17.176640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:17.176641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.176642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.176643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.176651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:17.176662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:17.176666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:17.176671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:17.176690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:17.176695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:17.176699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.176702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.176706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.176714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:17.176724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:17.176730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:17.176735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:17.176739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:17.176743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:17.176747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:17.176753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:17.176758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:17.176763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:17.176771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:17.176778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:17.176783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.176788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.176793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.176799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.176805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.176823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.176851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:17.184077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341372209758349:5768] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:17.184098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341372209758349:5768] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:17.184102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341372209758349:5768] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:17.184113Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 489 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:17.184382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:17.185642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:17.185655Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:17.185657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:17.185662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:17.185670Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 488 [2:7559341371847945959:5280]
2025-10-09T21:45:17.185686Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:17.185709Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:17.185720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:17.185724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:17.185726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:17.185745Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:17.185752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:17.185759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:17.185825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.185834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.185836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.185839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:17.185910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:17.185917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.185930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.185940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.185962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:17.185983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341372209758388:5768], now have 1 active actors on pipe
2025-10-09T21:45:17.185986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341372209758327:2432], now have 1 active actors on pipe
2025-10-09T21:45:17.185994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341371847945954:5280]
2025-10-09T21:45:17.186000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341371847945954:5280]
2025-10-09T21:45:17.186003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341371847945954:5280]
2025-10-09T21:45:17.186014Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:17.186046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:17.186055Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:17.186055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:17.186058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:17.186069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:17.186072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:17.186070Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 488
2025-10-09T21:45:17.186100Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 488
2025-10-09T21:45:17.186112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341371847945954:5280]
2025-10-09T21:45:17.186121Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:17.186123Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:17.186177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:17.186182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:17.186230Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9ede47b0-323f60c5-ed458723-66a20f6b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:17.186243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:17.186246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:17.186249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:17.186260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:17.186270Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:17.186292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:17.186641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:17.186651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:17.186668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:17.186783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 489 partition: 0 MaxSeqNo: 25 sessionId: src_id|9ede47b0-323f60c5-ed458723-66a20f6b_0
2025-10-09T21:45:17.187209Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|652136ef-b3835684-ab0329ab-1c5c211e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9ede47b0-323f60c5-ed458723-66a20f6b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:17.189871Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ede47b0-323f60c5-ed458723-66a20f6b_0] Write session: try to update token
2025-10-09T21:45:17.189891Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ede47b0-323f60c5-ed458723-66a20f6b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:17.202477Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:17.202494Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:17.202501Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:17.202504Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:17.202514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341371847945954:5280]
2025-10-09T21:45:17.202518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341371847945954:5280]
2025-10-09T21:45:17.202522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341371847945954:5280]
2025-10-09T21:45:17.202574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:17.202581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.202584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:17.202727Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:17.205519Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:17.205533Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:17.205538Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:17.204701Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 489 sessionId: src_id|9ede47b0-323f60c5-ed458723-66a20f6b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:17.207327Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:17.256773Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341372209758396:5210]
2025-10-09T21:45:17.257133Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 488, pipe: [1:7559341372209758396:5210]
2025-10-09T21:45:17.375872Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ede47b0-323f60c5-ed458723-66a20f6b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:17.375888Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ede47b0-323f60c5-ed458723-66a20f6b_0] Write session will restart in 0.009646s
2025-10-09T21:45:17.375411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:17.375416Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341372209758396:5210]
2025-10-09T21:45:17.375425Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:17.375458Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:17.375497Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 489 reason: error at write: Disconnected sessionId: src_id|9ede47b0-323f60c5-ed458723-66a20f6b_0
2025-10-09T21:45:17.375924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ede47b0-323f60c5-ed458723-66a20f6b_0] Write session: Do CDS request
2025-10-09T21:45:17.375937Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ede47b0-323f60c5-ed458723-66a20f6b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:17.375693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 489 sessionId: src_id|9ede47b0-323f60c5-ed458723-66a20f6b_0 is DEAD
2025-10-09T21:45:17.374596Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:17.374900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:17.374912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:17.374962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341372209758396:5210], now have 1 active actors on pipe
2025-10-09T21:45:17.375832Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:17.386600Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ede47b0-323f60c5-ed458723-66a20f6b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:17.386719Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:17.386737Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 490
2025-10-09T21:45:17.386874Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 490 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:17.386910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 490 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:17.386922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=490 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:17.386926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:17.387338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 490 sessionId:  describe result for acl check
2025-10-09T21:45:17.387373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:17.387381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:17.387383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:17.387390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341372209758420:5772] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:17.388019Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341372209758420:5772] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:17.387692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:17.390796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:17.391020Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:17.391032Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:17.391045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:17.391117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341372209758420:5772] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:17.391105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:17.391152Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341371847946022:5283]
2025-10-09T21:45:17.391811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:17.392198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:17.392263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:17.392622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:17.392672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:17.392680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:17.392682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:17.392683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:17.392685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:17.392686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:17.392688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.392689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.392690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.392695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:17.392707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:17.392711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:17.392718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:17.392722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:17.392727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:17.392731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.392736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.392740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.392747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:17.392756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:17.392763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:17.392769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:17.392774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:17.392780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:17.392785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:17.392791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:17.392796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:17.392801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:17.392806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:17.392811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:17.392818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.392824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.392829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.392834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.392840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.392845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.392876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:17.398764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341372209758420:5772] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:17.398784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341372209758420:5772] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:17.398788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341372209758420:5772] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:17.398809Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 490 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:17.400181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:17.401860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:17.401877Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:17.401880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:17.401886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:17.401894Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 489 [2:7559341371847946022:5283]
2025-10-09T21:45:17.401909Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:17.401932Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:17.401935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:17.401938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:17.401941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:17.401962Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:17.401977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:17.401985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:17.402070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.402077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.402079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.402082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:17.402167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:17.402175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.402191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.402205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.402240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:17.402270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341372209758460:5772], now have 1 active actors on pipe
2025-10-09T21:45:17.402279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341372209758398:2432], now have 1 active actors on pipe
2025-10-09T21:45:17.402288Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341371847946016:5283]
2025-10-09T21:45:17.402290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341371847946016:5283]
2025-10-09T21:45:17.402302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341371847946016:5283]
2025-10-09T21:45:17.402315Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:17.402329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:17.402330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:17.402334Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:17.402337Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:17.402352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:17.402355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:17.402401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341371847946016:5283]
2025-10-09T21:45:17.402410Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:17.402413Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:17.402444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:17.402452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:17.402488Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|72f47bac-11a7a492-6e011cc5-8ef5015_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:17.402505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:17.402509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:17.402512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:17.402523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:17.402538Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:17.402567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:17.402347Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 489
2025-10-09T21:45:17.402349Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 489
2025-10-09T21:45:17.402756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:17.402769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:17.402801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:17.402910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 490 partition: 0 MaxSeqNo: 25 sessionId: src_id|72f47bac-11a7a492-6e011cc5-8ef5015_0
2025-10-09T21:45:17.403399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ede47b0-323f60c5-ed458723-66a20f6b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|72f47bac-11a7a492-6e011cc5-8ef5015_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:17.406243Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72f47bac-11a7a492-6e011cc5-8ef5015_0] Write session: try to update token
2025-10-09T21:45:17.406267Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72f47bac-11a7a492-6e011cc5-8ef5015_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:17.418170Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:17.418186Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:17.418189Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:17.418191Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:17.418201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341371847946016:5283]
2025-10-09T21:45:17.418206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341371847946016:5283]
2025-10-09T21:45:17.418210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341371847946016:5283]
2025-10-09T21:45:17.418269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:17.418278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.418281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:17.418422Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:17.419014Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:17.419021Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:17.419025Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:17.426562Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 490 sessionId: src_id|72f47bac-11a7a492-6e011cc5-8ef5015_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:17.429108Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:17.475840Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341372209758468:5210]
2025-10-09T21:45:17.477775Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 489, pipe: [1:7559341372209758468:5210]
2025-10-09T21:45:17.614542Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:17.615222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:17.615241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:17.615266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341372209758468:5210], now have 1 active actors on pipe
2025-10-09T21:45:17.618479Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:17.619286Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:17.619304Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:17.619354Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341372209758468:5210]
2025-10-09T21:45:17.619398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 490 reason: status is not ok: tablet will be restarted right now sessionId: src_id|72f47bac-11a7a492-6e011cc5-8ef5015_0
2025-10-09T21:45:17.619612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 490 sessionId: src_id|72f47bac-11a7a492-6e011cc5-8ef5015_0 is DEAD
2025-10-09T21:45:17.622633Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72f47bac-11a7a492-6e011cc5-8ef5015_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:17.622646Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72f47bac-11a7a492-6e011cc5-8ef5015_0] Write session will restart in 0.009789s
2025-10-09T21:45:17.622680Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72f47bac-11a7a492-6e011cc5-8ef5015_0] Write session: Do CDS request
2025-10-09T21:45:17.622686Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72f47bac-11a7a492-6e011cc5-8ef5015_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:17.632212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:17.634730Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72f47bac-11a7a492-6e011cc5-8ef5015_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:17.635114Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:17.635131Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 491
2025-10-09T21:45:17.635539Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 491 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:17.635576Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 491 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:17.635582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=491 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:17.635586Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:17.635567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:17.636105Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 491 sessionId:  describe result for acl check
2025-10-09T21:45:17.636142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:17.636144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:17.636146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:17.636152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341372209758506:5777] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:17.636722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341372209758506:5777] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:17.635803Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:17.635817Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:17.635830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:17.635890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:17.635933Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341371847946086:5285]
2025-10-09T21:45:17.636561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:17.637343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:17.637402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:17.637749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:17.637810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:17.637819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:17.637821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:17.637822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:17.637823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:17.637824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:17.637825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.637826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.637827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.637833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:17.637844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:17.637849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:17.637854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:17.637858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:17.637863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:17.637867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.637872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.637876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.637883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:17.637892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:17.637899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:17.637905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:17.637911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:17.637916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:17.637923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:17.637929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:17.637936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:17.637941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:17.637958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:17.637964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:17.637969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.637974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.637979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.637984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.637989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.637995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.638026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:17.640130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341372209758506:5777] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:17.644560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341372209758506:5777] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:17.644582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341372209758506:5777] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:17.644586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341372209758506:5777] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:17.644596Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 491 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:17.645780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:17.647017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:17.647034Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:17.647037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:17.647042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:17.647050Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 490 [2:7559341371847946086:5285]
2025-10-09T21:45:17.647065Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:17.647087Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:17.647091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:17.647095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:17.647098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:17.647117Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:17.647123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:17.647131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:17.647203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.647205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.647207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.647211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:17.647290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:17.647293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.647308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.647321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.647340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:17.647362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341371847946081:5285]
2025-10-09T21:45:17.647364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341371847946081:5285]
2025-10-09T21:45:17.647367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341371847946081:5285]
2025-10-09T21:45:17.647384Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:17.647400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:17.647417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:17.647420Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:17.647441Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:17.647444Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:17.647446Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:17.647461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341371847946081:5285]
2025-10-09T21:45:17.647464Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:17.647549Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:17.648002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341372209758473:2432], now have 1 active actors on pipe
2025-10-09T21:45:17.648007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341372209758543:5777], now have 1 active actors on pipe
2025-10-09T21:45:17.649337Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 490
2025-10-09T21:45:17.649379Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 490
2025-10-09T21:45:17.653292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:17.653313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:17.653365Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f07c0397-5258a12a-33f264e-c151783e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:17.653384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:17.653395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:17.653398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:17.653413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:17.653434Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:17.653458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:17.653769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:17.653783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:17.653814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:17.654019Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 491 partition: 0 MaxSeqNo: 25 sessionId: src_id|f07c0397-5258a12a-33f264e-c151783e_0
2025-10-09T21:45:17.655323Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72f47bac-11a7a492-6e011cc5-8ef5015_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f07c0397-5258a12a-33f264e-c151783e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:17.658152Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f07c0397-5258a12a-33f264e-c151783e_0] Write session: try to update token
2025-10-09T21:45:17.658173Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f07c0397-5258a12a-33f264e-c151783e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:17.667020Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:17.667036Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:17.667039Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:17.667041Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:17.667052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341371847946081:5285]
2025-10-09T21:45:17.667057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341371847946081:5285]
2025-10-09T21:45:17.667061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341371847946081:5285]
2025-10-09T21:45:17.667113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:17.667121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.667124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:17.667127Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:17.667132Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:17.667136Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:17.667257Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:17.690090Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 491 sessionId: src_id|f07c0397-5258a12a-33f264e-c151783e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:17.693202Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:17.719650Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341372209758551:5210]
2025-10-09T21:45:17.720974Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 490, pipe: [1:7559341372209758551:5210]
2025-10-09T21:45:17.839630Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:17.840264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:17.840275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:17.840300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341372209758551:5210], now have 1 active actors on pipe
2025-10-09T21:45:17.841018Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:17.843601Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341372209758551:5210]
2025-10-09T21:45:17.843645Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:17.843715Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:17.843731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 491 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f07c0397-5258a12a-33f264e-c151783e_0
2025-10-09T21:45:17.843933Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 491 sessionId: src_id|f07c0397-5258a12a-33f264e-c151783e_0 is DEAD
2025-10-09T21:45:17.844537Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f07c0397-5258a12a-33f264e-c151783e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:17.844551Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f07c0397-5258a12a-33f264e-c151783e_0] Write session will restart in 0.006344s
2025-10-09T21:45:17.844593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f07c0397-5258a12a-33f264e-c151783e_0] Write session: Do CDS request
2025-10-09T21:45:17.844600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f07c0397-5258a12a-33f264e-c151783e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:17.852604Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f07c0397-5258a12a-33f264e-c151783e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:17.852732Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:17.852750Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 492
2025-10-09T21:45:17.852923Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 492 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:17.852968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 492 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:17.852974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=492 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:17.852997Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:17.854177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 492 sessionId:  describe result for acl check
2025-10-09T21:45:17.854225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:17.854227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:17.854229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:17.854243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341372209758578:5781] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:17.854827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341372209758578:5781] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:17.857879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:17.862548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341372209758578:5781] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:17.864361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:17.864599Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:17.864604Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:17.864615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:17.864667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:17.864703Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341371847946180:5297]
2025-10-09T21:45:17.865291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:17.866305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:17.866370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:17.866705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:17.866758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:17.866759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:17.866760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:17.866761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:17.866762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:17.866765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:17.866767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.866768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.866769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.866776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:17.866787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:17.866791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:17.866794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:17.866797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:17.866801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:17.866804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.866808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.866811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.866819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:17.866829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:17.866835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:17.866840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:17.866844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:17.866848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:17.866852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:17.866857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:17.866862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:17.866867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:17.866871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:17.866875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:17.866881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.866884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.866888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.866892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.866897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.866902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.866928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:17.875967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341372209758578:5781] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:17.875989Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341372209758578:5781] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:17.875993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341372209758578:5781] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:17.876003Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 492 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:17.877583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:17.878727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:17.878734Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:17.878738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:17.878743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:17.878752Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 491 [2:7559341371847946180:5297]
2025-10-09T21:45:17.878767Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:17.878787Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:17.878791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:17.878794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:17.878797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:17.878816Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:17.878821Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:17.878829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:17.878895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.878897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.878899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.878903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:17.878984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:17.878987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.879001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:17.879012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:17.879029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:17.879048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341371847946175:5297]
2025-10-09T21:45:17.879053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341371847946175:5297]
2025-10-09T21:45:17.879055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341371847946175:5297]
2025-10-09T21:45:17.879062Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:17.879076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:17.879091Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:17.879094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:17.879133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341371847946175:5297]
2025-10-09T21:45:17.879135Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:17.879368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:17.879378Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:17.879380Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:17.879386Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:17.880346Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 491
2025-10-09T21:45:17.880393Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 491
2025-10-09T21:45:17.881610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341372209758556:2432], now have 1 active actors on pipe
2025-10-09T21:45:17.881623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341372209758616:5781], now have 1 active actors on pipe
2025-10-09T21:45:17.883242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:17.883252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:17.883303Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|96235e4e-144d3010-9afc23b7-e558a56_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:17.883324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:17.883331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:17.883334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:17.883349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:17.883367Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:17.883391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:17.887296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:17.887310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:17.887346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:17.888419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 492 partition: 0 MaxSeqNo: 25 sessionId: src_id|96235e4e-144d3010-9afc23b7-e558a56_0
2025-10-09T21:45:17.888834Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f07c0397-5258a12a-33f264e-c151783e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|96235e4e-144d3010-9afc23b7-e558a56_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:17.891391Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96235e4e-144d3010-9afc23b7-e558a56_0] Write session: try to update token
2025-10-09T21:45:17.891410Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96235e4e-144d3010-9afc23b7-e558a56_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:17.897482Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:17.897496Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:17.897506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341371847946175:5297]
2025-10-09T21:45:17.897563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:17.897743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:17.897921Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:17.898095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:17.898256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:17.898449Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:17.898488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:17.898797Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:17.918399Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:17.918415Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:17.918417Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:17.918419Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:17.918432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341371847946175:5297]
2025-10-09T21:45:17.918437Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341371847946175:5297]
2025-10-09T21:45:17.918440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341371847946175:5297]
2025-10-09T21:45:17.918484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:17.918491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:17.918494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:17.918630Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:17.922372Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:17.922383Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:17.922387Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:17.943402Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 492 sessionId: src_id|96235e4e-144d3010-9afc23b7-e558a56_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:17.945800Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:17.950100Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341372209758641:5210]
2025-10-09T21:45:17.950794Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 491, pipe: [1:7559341372209758641:5210]
2025-10-09T21:45:18.111547Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:18.112274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:18.112286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:18.112314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341372209758641:5210], now have 1 active actors on pipe
2025-10-09T21:45:18.112745Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:18.112782Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341372209758641:5210]
2025-10-09T21:45:18.112828Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:18.112833Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:18.112911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 492 reason: error at write: Disconnected sessionId: src_id|96235e4e-144d3010-9afc23b7-e558a56_0
2025-10-09T21:45:18.113175Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 492 sessionId: src_id|96235e4e-144d3010-9afc23b7-e558a56_0 is DEAD
2025-10-09T21:45:18.112875Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:18.113661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96235e4e-144d3010-9afc23b7-e558a56_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:18.113674Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96235e4e-144d3010-9afc23b7-e558a56_0] Write session will restart in 0.007437s
2025-10-09T21:45:18.113715Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96235e4e-144d3010-9afc23b7-e558a56_0] Write session: Do CDS request
2025-10-09T21:45:18.113723Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96235e4e-144d3010-9afc23b7-e558a56_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:18.122606Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96235e4e-144d3010-9afc23b7-e558a56_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:18.126101Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:18.126121Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 493
2025-10-09T21:45:18.127785Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 493 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:18.127838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 493 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:18.127844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=493 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:18.127849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:18.128286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 493 sessionId:  describe result for acl check
2025-10-09T21:45:18.128318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:18.128321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:18.128323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:18.128329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341376504725983:5793] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:18.128934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341376504725983:5793] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:18.134779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:18.136038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341376504725983:5793] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:18.135817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:18.136030Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:18.136035Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:18.136047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:18.136135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:18.136176Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341376142913552:5301]
2025-10-09T21:45:18.136813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:18.138065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:18.138139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:18.138689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:18.138760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:18.138767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:18.138768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:18.138770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:18.138771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:18.138772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:18.138774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.138775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.138776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.138783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:18.138794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:18.138798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:18.138802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:18.138806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:18.138810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:18.138813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.138817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.138821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.138833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:18.138841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:18.138847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:18.138862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:18.138866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:18.138871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:18.138875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:18.138880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:18.138887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:18.138896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:18.138900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:18.138905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:18.138910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.138915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.138919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.138923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.138928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.138932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.138968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:18.139665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341376504725983:5793] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:18.139685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341376504725983:5793] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:18.139688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341376504725983:5793] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:18.139700Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 493 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:18.151889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:18.153165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:18.153184Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:18.153188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:18.153193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:18.153203Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 492 [2:7559341376142913552:5301]
2025-10-09T21:45:18.153223Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:18.153250Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:18.153254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:18.153258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:18.153260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:18.153287Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:18.153294Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:18.153302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:18.153376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.153384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.153387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.153390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:18.153473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:18.153481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.153495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.153507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.153523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:18.153550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341376142913547:5301]
2025-10-09T21:45:18.153564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341376142913547:5301]
2025-10-09T21:45:18.153566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341376142913547:5301]
2025-10-09T21:45:18.153582Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:18.153599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:18.153614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:18.153622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:18.153669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341376142913547:5301]
2025-10-09T21:45:18.153676Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:18.153773Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:18.153784Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:18.153786Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:18.153794Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:18.154237Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 492
2025-10-09T21:45:18.154277Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 492
2025-10-09T21:45:18.155209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341376504726022:5793], now have 1 active actors on pipe
2025-10-09T21:45:18.155224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341376504725956:2432], now have 1 active actors on pipe
2025-10-09T21:45:18.155252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:18.155266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:18.155819Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|88b34245-9b55ab56-40b368b8-ecfd6cee_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:18.155843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:18.155848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:18.155852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:18.155866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:18.155884Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:18.155909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:18.156338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:18.156350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:18.156371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:18.156525Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 493 partition: 0 MaxSeqNo: 25 sessionId: src_id|88b34245-9b55ab56-40b368b8-ecfd6cee_0
2025-10-09T21:45:18.157016Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96235e4e-144d3010-9afc23b7-e558a56_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|88b34245-9b55ab56-40b368b8-ecfd6cee_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:18.159808Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88b34245-9b55ab56-40b368b8-ecfd6cee_0] Write session: try to update token
2025-10-09T21:45:18.159831Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88b34245-9b55ab56-40b368b8-ecfd6cee_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:18.176076Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:18.176093Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:18.176096Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:18.176099Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:18.176110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341376142913547:5301]
2025-10-09T21:45:18.176115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341376142913547:5301]
2025-10-09T21:45:18.176119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341376142913547:5301]
2025-10-09T21:45:18.176172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:18.176172Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:18.176179Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:18.176180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.176182Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:18.176183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:18.176337Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:18.186432Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 493 sessionId: src_id|88b34245-9b55ab56-40b368b8-ecfd6cee_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:18.189002Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:18.213116Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341376504726029:5210]
2025-10-09T21:45:18.213484Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 492, pipe: [1:7559341376504726029:5210]
2025-10-09T21:45:18.347480Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:18.347788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:18.347803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:18.347825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341376504726029:5210], now have 1 active actors on pipe
2025-10-09T21:45:18.348124Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:18.348142Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:18.348150Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341376504726029:5210]
2025-10-09T21:45:18.348180Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:18.348203Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 493 reason: error at write: Disconnected sessionId: src_id|88b34245-9b55ab56-40b368b8-ecfd6cee_0
2025-10-09T21:45:18.348389Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 493 sessionId: src_id|88b34245-9b55ab56-40b368b8-ecfd6cee_0 is DEAD
2025-10-09T21:45:18.348894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88b34245-9b55ab56-40b368b8-ecfd6cee_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:18.348912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88b34245-9b55ab56-40b368b8-ecfd6cee_0] Write session will restart in 0.005018s
2025-10-09T21:45:18.348147Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:18.348937Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88b34245-9b55ab56-40b368b8-ecfd6cee_0] Write session: Do CDS request
2025-10-09T21:45:18.348943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88b34245-9b55ab56-40b368b8-ecfd6cee_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:18.355567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88b34245-9b55ab56-40b368b8-ecfd6cee_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:18.355702Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:18.355719Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 494
2025-10-09T21:45:18.355916Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 494 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:18.355967Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 494 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:18.355978Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=494 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:18.355982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:18.356450Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 494 sessionId:  describe result for acl check
2025-10-09T21:45:18.356487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:18.356494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:18.356496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:18.356502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341376504726045:5797] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:18.357075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341376504726045:5797] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:18.359685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341376504726045:5797] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:18.361913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:18.365758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:18.365985Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:18.365996Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:18.366008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:18.366264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:18.366311Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341376142913619:5305]
2025-10-09T21:45:18.366953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:18.367334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:18.367397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:18.367765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:18.367825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:18.367831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:18.367832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:18.367834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:18.367835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:18.367837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:18.367838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.367840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.367841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.367846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:18.367858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:18.367863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:18.367867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:18.367871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:18.367875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:18.367879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.367882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.367886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.367892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:18.367900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:18.367906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:18.367911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:18.367917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:18.367923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:18.367927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:18.367933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:18.367938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:18.367944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:18.367947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:18.367953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:18.367958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.367965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.367976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.367982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.367987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.367992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.368034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:18.366424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341376504726045:5797] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:18.366441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341376504726045:5797] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:18.366446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341376504726045:5797] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:18.366456Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 494 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:18.376704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:18.379144Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 493
2025-10-09T21:45:18.379182Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 493
2025-10-09T21:45:18.381370Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 494 partition: 0 MaxSeqNo: 25 sessionId: src_id|e89462df-e4675270-9fbe5933-ec018c5_0
2025-10-09T21:45:18.377914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:18.377932Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:18.377935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:18.377940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:18.377948Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 493 [2:7559341376142913619:5305]
2025-10-09T21:45:18.377965Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:18.377987Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:18.377991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:18.377994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:18.377996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:18.378018Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:18.378025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:18.378059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:18.378132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.378135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.378137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.378140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:18.378230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:18.378233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.378246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.378256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.378273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:18.378295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341376142913613:5305]
2025-10-09T21:45:18.378298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341376142913613:5305]
2025-10-09T21:45:18.378300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341376142913613:5305]
2025-10-09T21:45:18.378313Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:18.378328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:18.378343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:18.378347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:18.378364Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:18.378367Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:18.378373Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:18.378389Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341376142913613:5305]
2025-10-09T21:45:18.378392Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:18.378394Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:18.378403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341376504726032:2432], now have 1 active actors on pipe
2025-10-09T21:45:18.378407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341376504726093:5797], now have 1 active actors on pipe
2025-10-09T21:45:18.379562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:18.379571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:18.379614Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e89462df-e4675270-9fbe5933-ec018c5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:18.379633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:18.379638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:18.379640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:18.379655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:18.379669Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:18.379698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:18.381006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:18.381012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:18.381048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:18.384009Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88b34245-9b55ab56-40b368b8-ecfd6cee_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e89462df-e4675270-9fbe5933-ec018c5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:18.386617Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e89462df-e4675270-9fbe5933-ec018c5_0] Write session: try to update token
2025-10-09T21:45:18.386644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e89462df-e4675270-9fbe5933-ec018c5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:18.406901Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:18.406920Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:18.406924Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:18.406927Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:18.406938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341376142913613:5305]
2025-10-09T21:45:18.406944Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341376142913613:5305]
2025-10-09T21:45:18.406949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341376142913613:5305]
2025-10-09T21:45:18.406997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:18.407005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.407008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:18.407158Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:18.409691Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:18.409707Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:18.409711Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:18.420625Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 494 sessionId: src_id|e89462df-e4675270-9fbe5933-ec018c5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:18.423165Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:18.450192Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341376504726106:5210]
2025-10-09T21:45:18.451179Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 493, pipe: [1:7559341376504726106:5210]
2025-10-09T21:45:18.575886Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:18.576192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:18.576202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:18.576224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341376504726106:5210], now have 1 active actors on pipe
2025-10-09T21:45:18.576689Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:18.576769Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341376504726106:5210]
2025-10-09T21:45:18.576933Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:18.576942Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:18.577004Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 494 reason: error at write: Disconnected sessionId: src_id|e89462df-e4675270-9fbe5933-ec018c5_0
2025-10-09T21:45:18.577215Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 494 sessionId: src_id|e89462df-e4675270-9fbe5933-ec018c5_0 is DEAD
2025-10-09T21:45:18.576742Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:18.578122Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e89462df-e4675270-9fbe5933-ec018c5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:18.578137Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e89462df-e4675270-9fbe5933-ec018c5_0] Write session will restart in 0.006760s
2025-10-09T21:45:18.578163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e89462df-e4675270-9fbe5933-ec018c5_0] Write session: Do CDS request
2025-10-09T21:45:18.578170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e89462df-e4675270-9fbe5933-ec018c5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:18.590506Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e89462df-e4675270-9fbe5933-ec018c5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:18.590860Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:18.590878Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 495
2025-10-09T21:45:18.591164Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 495 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:18.591202Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 495 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:18.591208Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=495 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:18.591222Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:18.594093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 495 sessionId:  describe result for acl check
2025-10-09T21:45:18.594130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:18.594132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:18.594133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:18.594139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341376504726139:5802] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:18.594694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341376504726139:5802] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:18.595740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:18.598077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341376504726139:5802] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:18.600386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:18.600588Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:18.600592Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:18.600602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:18.600655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:18.600692Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341376142913687:5307]
2025-10-09T21:45:18.601256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:18.601589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341376504726139:5802] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:18.601601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341376504726139:5802] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:18.601604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341376504726139:5802] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:18.601613Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 495 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:18.601881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:18.601926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:18.602263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:18.602323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:18.602324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:18.602325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:18.602326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:18.602327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:18.602329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:18.602329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.602331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.602332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.602337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:18.602348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:18.602351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:18.602354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:18.602358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:18.602361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:18.602365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.602369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.602375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.602383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:18.602391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:18.602397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:18.602402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:18.602406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:18.602412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:18.602416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:18.602421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:18.602425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:18.602430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:18.602434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:18.602439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:18.602445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.602451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.602455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.602460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.602464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.602469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.602498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:18.612909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:18.614235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:18.614242Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:18.614245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:18.614250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:18.614258Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 494 [2:7559341376142913687:5307]
2025-10-09T21:45:18.614275Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:18.614296Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:18.614300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:18.614303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:18.614305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:18.614327Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:18.614334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:18.614342Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:18.614405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.614408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.614410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.614412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:18.614491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:18.614494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.614509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.614520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.614537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:18.614558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341376142913681:5307]
2025-10-09T21:45:18.614561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341376142913681:5307]
2025-10-09T21:45:18.614563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341376142913681:5307]
2025-10-09T21:45:18.614578Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:18.614593Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:18.614609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:18.614612Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:18.614656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341376142913681:5307]
2025-10-09T21:45:18.614659Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:18.615084Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:18.615091Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:18.615094Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:18.615101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:18.616548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341376504726175:5802], now have 1 active actors on pipe
2025-10-09T21:45:18.616564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341376504726111:2432], now have 1 active actors on pipe
2025-10-09T21:45:18.616814Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 494
2025-10-09T21:45:18.616850Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 494
2025-10-09T21:45:18.617201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:18.617212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:18.617251Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f5731a4d-a4d6b75b-ca814c5d-6fd8ff65_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:18.617263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:18.617268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:18.617271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:18.617283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:18.617297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:18.617318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:18.618189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:18.618201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:18.618243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:18.618440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 495 partition: 0 MaxSeqNo: 25 sessionId: src_id|f5731a4d-a4d6b75b-ca814c5d-6fd8ff65_0
2025-10-09T21:45:18.618985Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e89462df-e4675270-9fbe5933-ec018c5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f5731a4d-a4d6b75b-ca814c5d-6fd8ff65_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:18.621733Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5731a4d-a4d6b75b-ca814c5d-6fd8ff65_0] Write session: try to update token
2025-10-09T21:45:18.621759Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5731a4d-a4d6b75b-ca814c5d-6fd8ff65_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:18.634982Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:18.635000Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:18.635003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:18.635005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:18.635016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341376142913681:5307]
2025-10-09T21:45:18.635034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341376142913681:5307]
2025-10-09T21:45:18.635038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341376142913681:5307]
2025-10-09T21:45:18.635083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:18.635090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.635092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:18.635228Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:18.636176Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:18.636186Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:18.636190Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:18.658002Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 495 sessionId: src_id|f5731a4d-a4d6b75b-ca814c5d-6fd8ff65_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:18.660709Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:18.678103Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341376504726185:5210]
2025-10-09T21:45:18.678599Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 494, pipe: [1:7559341376504726185:5210]
2025-10-09T21:45:18.844520Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:18.844773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:18.844783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:18.844830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341376504726185:5210], now have 1 active actors on pipe
2025-10-09T21:45:18.845337Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:18.846195Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:18.846197Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341376504726185:5210]
2025-10-09T21:45:18.846230Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:18.846293Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 495 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f5731a4d-a4d6b75b-ca814c5d-6fd8ff65_0
2025-10-09T21:45:18.846482Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 495 sessionId: src_id|f5731a4d-a4d6b75b-ca814c5d-6fd8ff65_0 is DEAD
2025-10-09T21:45:18.846875Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5731a4d-a4d6b75b-ca814c5d-6fd8ff65_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:18.846885Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5731a4d-a4d6b75b-ca814c5d-6fd8ff65_0] Write session will restart in 0.008312s
2025-10-09T21:45:18.846911Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5731a4d-a4d6b75b-ca814c5d-6fd8ff65_0] Write session: Do CDS request
2025-10-09T21:45:18.846918Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5731a4d-a4d6b75b-ca814c5d-6fd8ff65_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:18.856556Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5731a4d-a4d6b75b-ca814c5d-6fd8ff65_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:18.857073Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:18.857089Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 496
2025-10-09T21:45:18.857301Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 496 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:18.857331Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 496 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:18.857336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=496 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:18.857340Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:18.857827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 496 sessionId:  describe result for acl check
2025-10-09T21:45:18.857855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:18.857857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:18.857858Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:18.857864Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341376504726202:5806] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:18.858481Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341376504726202:5806] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:18.864771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341376504726202:5806] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:18.869168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:18.873079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:18.873534Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:18.873540Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:18.873555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:18.873626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:18.873665Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341376142913768:5315]
2025-10-09T21:45:18.875791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341376504726202:5806] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:18.875809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341376504726202:5806] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:18.875813Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341376504726202:5806] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:18.875824Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 496 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:18.874320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:18.875293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:18.875347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:18.875678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:18.875740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:18.875742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:18.875742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:18.875744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:18.875746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:18.875747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:18.875748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.875749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.875750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.875756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:18.875767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:18.875771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:18.875775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:18.875779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:18.875784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:18.875788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.875792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.875796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.875802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:18.875810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:18.875817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:18.875823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:18.875827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:18.875832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:18.875837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:18.875843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:18.875848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:18.875854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:18.875858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:18.875866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:18.875871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.875876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.875881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.875887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.875892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.875897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.875928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:18.885240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:18.887053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:18.887069Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:18.887071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:18.887075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:18.887083Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 495 [2:7559341376142913768:5315]
2025-10-09T21:45:18.887098Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:18.887117Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:18.887120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:18.887124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:18.887126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:18.887145Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:18.887152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:18.887160Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:18.887300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.887306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.887308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.887311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:18.887391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:18.887394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.887421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:18.887433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:18.887454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:18.887478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341376142913762:5315]
2025-10-09T21:45:18.887481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341376142913762:5315]
2025-10-09T21:45:18.887483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341376142913762:5315]
2025-10-09T21:45:18.887498Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:18.887514Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:18.887533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:18.887536Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:18.887551Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:18.887555Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:18.887567Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:18.887579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341376142913762:5315]
2025-10-09T21:45:18.887582Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:18.887656Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:18.887882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341376504726188:2432], now have 1 active actors on pipe
2025-10-09T21:45:18.887888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341376504726255:5806], now have 1 active actors on pipe
2025-10-09T21:45:18.887964Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 495
2025-10-09T21:45:18.888004Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 495
2025-10-09T21:45:18.888107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:18.888117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:18.888145Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|783205f0-f1b9e235-21f6ba49-a8e3b24_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:18.888153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:18.888159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:18.888162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:18.888173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:18.888186Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:18.888202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:18.888481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:18.888488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:18.888505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:18.889363Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 496 partition: 0 MaxSeqNo: 25 sessionId: src_id|783205f0-f1b9e235-21f6ba49-a8e3b24_0
2025-10-09T21:45:18.891152Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5731a4d-a4d6b75b-ca814c5d-6fd8ff65_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|783205f0-f1b9e235-21f6ba49-a8e3b24_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:18.894617Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|783205f0-f1b9e235-21f6ba49-a8e3b24_0] Write session: try to update token
2025-10-09T21:45:18.894638Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|783205f0-f1b9e235-21f6ba49-a8e3b24_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:18.913904Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:18.913927Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:18.913930Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:18.913932Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:18.913943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341376142913762:5315]
2025-10-09T21:45:18.913951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341376142913762:5315]
2025-10-09T21:45:18.913955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341376142913762:5315]
2025-10-09T21:45:18.914003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:18.914010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:18.914013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:18.914187Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:18.915111Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:18.915120Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:18.915126Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:18.925633Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 496 sessionId: src_id|783205f0-f1b9e235-21f6ba49-a8e3b24_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:18.928178Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:18.946536Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341376504726269:5210]
2025-10-09T21:45:18.947193Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 495, pipe: [1:7559341376504726269:5210]
2025-10-09T21:45:19.133850Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:19.138458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:19.138479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:19.138527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341376504726269:5210], now have 1 active actors on pipe
2025-10-09T21:45:19.138916Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:19.138990Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:19.138994Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:19.139072Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 496 reason: error at write: Disconnected sessionId: src_id|783205f0-f1b9e235-21f6ba49-a8e3b24_0
2025-10-09T21:45:19.139283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 496 sessionId: src_id|783205f0-f1b9e235-21f6ba49-a8e3b24_0 is DEAD
2025-10-09T21:45:19.139460Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|783205f0-f1b9e235-21f6ba49-a8e3b24_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:19.139472Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|783205f0-f1b9e235-21f6ba49-a8e3b24_0] Write session will restart in 0.005869s
2025-10-09T21:45:19.139471Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341376504726269:5210]
2025-10-09T21:45:19.139497Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|783205f0-f1b9e235-21f6ba49-a8e3b24_0] Write session: Do CDS request
2025-10-09T21:45:19.139507Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|783205f0-f1b9e235-21f6ba49-a8e3b24_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:19.140533Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:19.146589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|783205f0-f1b9e235-21f6ba49-a8e3b24_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:19.146767Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:19.146801Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 497
2025-10-09T21:45:19.147101Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 497 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:19.147155Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 497 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:19.147167Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=497 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:19.147170Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:19.147972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 497 sessionId:  describe result for acl check
2025-10-09T21:45:19.148014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:19.148033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:19.148039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:19.148046Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341380799693609:5818] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:19.148682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341380799693609:5818] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:19.152404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341380799693609:5818] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:19.157952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:19.158716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:19.158912Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:19.158916Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:19.158929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:19.158981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:19.159018Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341380437881161:5324]
2025-10-09T21:45:19.159652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:19.161193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:19.161246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:19.161594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:19.161639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:19.161641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:19.161642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:19.161643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:19.161645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:19.161646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:19.161647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.161648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.161649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.161655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:19.161667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:19.161671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:19.161676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:19.161680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:19.161684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:19.161689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.161693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.161697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.161705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:19.161717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:19.161724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:19.161730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:19.161736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:19.161742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:19.161746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:19.161752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:19.161758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:19.161763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:19.161768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:19.161773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:19.161779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.161784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.161788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.161793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.161799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.161804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.161838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:19.162923Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341380799693609:5818] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:19.162944Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341380799693609:5818] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:19.162949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341380799693609:5818] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:19.162960Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 497 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:19.171476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:19.172885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:19.172892Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:19.172894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:19.172898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:19.172906Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 496 [2:7559341380437881161:5324]
2025-10-09T21:45:19.172922Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:19.172945Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:19.172949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:19.172953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:19.172955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:19.172976Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:19.172986Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:19.172993Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:19.173061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.173064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.173066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.173068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:19.173151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:19.173154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.173168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.173181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.173198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:19.173219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341380437881155:5324]
2025-10-09T21:45:19.173223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341380437881155:5324]
2025-10-09T21:45:19.173225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341380437881155:5324]
2025-10-09T21:45:19.173241Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:19.173260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:19.173277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:19.173280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:19.173324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341380437881155:5324]
2025-10-09T21:45:19.173326Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:19.173831Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:19.173837Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:19.173840Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:19.173847Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:19.174254Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 496
2025-10-09T21:45:19.174290Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 496
2025-10-09T21:45:19.174319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341380799693663:5818], now have 1 active actors on pipe
2025-10-09T21:45:19.174328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341380799693598:2432], now have 1 active actors on pipe
2025-10-09T21:45:19.174344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:19.174350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:19.174380Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dbfd66ca-b404defb-74255ca9-277a8a18_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:19.174389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:19.174395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:19.174398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:19.174410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:19.174423Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:19.174458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:19.174855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:19.174862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:19.174880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:19.175551Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|783205f0-f1b9e235-21f6ba49-a8e3b24_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dbfd66ca-b404defb-74255ca9-277a8a18_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:19.175047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 497 partition: 0 MaxSeqNo: 25 sessionId: src_id|dbfd66ca-b404defb-74255ca9-277a8a18_0
2025-10-09T21:45:19.178726Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dbfd66ca-b404defb-74255ca9-277a8a18_0] Write session: try to update token
2025-10-09T21:45:19.178750Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dbfd66ca-b404defb-74255ca9-277a8a18_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:19.194345Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:19.194360Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:19.194362Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:19.194365Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:19.194375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341380437881155:5324]
2025-10-09T21:45:19.194381Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341380437881155:5324]
2025-10-09T21:45:19.194385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341380437881155:5324]
2025-10-09T21:45:19.194432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:19.194441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.194439Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:19.194443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:19.194446Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:19.194452Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:19.194590Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:19.198078Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 497 sessionId: src_id|dbfd66ca-b404defb-74255ca9-277a8a18_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:19.200926Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:19.239742Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341380799693672:5210]
2025-10-09T21:45:19.242378Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 496, pipe: [1:7559341380799693672:5210]
2025-10-09T21:45:19.362385Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:19.362673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:19.362689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:19.362804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341380799693672:5210], now have 1 active actors on pipe
2025-10-09T21:45:19.363083Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:19.366306Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:19.366401Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 497 reason: status is not ok: tablet will be restarted right now sessionId: src_id|dbfd66ca-b404defb-74255ca9-277a8a18_0
2025-10-09T21:45:19.366593Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 497 sessionId: src_id|dbfd66ca-b404defb-74255ca9-277a8a18_0 is DEAD
2025-10-09T21:45:19.366813Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341380799693672:5210]
2025-10-09T21:45:19.366829Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:19.367178Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbfd66ca-b404defb-74255ca9-277a8a18_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:19.367191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbfd66ca-b404defb-74255ca9-277a8a18_0] Write session will restart in 0.007047s
2025-10-09T21:45:19.367236Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbfd66ca-b404defb-74255ca9-277a8a18_0] Write session: Do CDS request
2025-10-09T21:45:19.367244Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbfd66ca-b404defb-74255ca9-277a8a18_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:19.375659Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dbfd66ca-b404defb-74255ca9-277a8a18_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:19.378223Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:19.378243Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 498
2025-10-09T21:45:19.378428Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 498 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:19.378477Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 498 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:19.378492Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=498 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:19.378496Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:19.379341Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 498 sessionId:  describe result for acl check
2025-10-09T21:45:19.379382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:19.379384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:19.379386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:19.379392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341380799693703:5822] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:19.380065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341380799693703:5822] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:19.383198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341380799693703:5822] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:19.390187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341380799693703:5822] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:19.390219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341380799693703:5822] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:19.390223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341380799693703:5822] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:19.390234Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 498 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:19.395081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:19.400186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:19.400513Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:19.400527Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:19.400542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:19.400637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:19.400691Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341380437881233:5327]
2025-10-09T21:45:19.401297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:19.402437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:19.402514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:19.402872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:19.402939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:19.402940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:19.402941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:19.402942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:19.402943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:19.402944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:19.402945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.402947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.402948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.402955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:19.402966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:19.402969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:19.402973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:19.402976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:19.402980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:19.402984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.402988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.402992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.403000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:19.403009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:19.403016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:19.403021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:19.403026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:19.403031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:19.403035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:19.403041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:19.403045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:19.403051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:19.403058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:19.403064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:19.403071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.403076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.403081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.403086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.403092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.403096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.403126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:19.411313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:19.413080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:19.413097Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:19.413100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:19.413105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:19.413115Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 497 [2:7559341380437881233:5327]
2025-10-09T21:45:19.413130Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:19.413151Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:19.413155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:19.413159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:19.413161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:19.413180Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:19.413186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:19.413195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:19.413262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.413270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.413274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.413277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:19.413365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:19.413368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.413381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.413395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.413413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:19.413442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341380437881228:5327]
2025-10-09T21:45:19.413449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341380437881228:5327]
2025-10-09T21:45:19.413451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341380437881228:5327]
2025-10-09T21:45:19.413462Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:19.413476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:19.413496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:19.413503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:19.413544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341380437881228:5327]
2025-10-09T21:45:19.413551Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:19.413714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341380799693674:2432], now have 1 active actors on pipe
2025-10-09T21:45:19.413727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341380799693737:5822], now have 1 active actors on pipe
2025-10-09T21:45:19.413759Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:19.413769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:19.413772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:19.413789Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:19.414729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:19.414742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:19.414794Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fadef10c-a4da578-8b926bd8-df467409_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:19.414810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:19.414817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:19.414820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:19.414832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:19.414846Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:19.414866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:19.416044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:19.416056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:19.416084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:19.417235Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbfd66ca-b404defb-74255ca9-277a8a18_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fadef10c-a4da578-8b926bd8-df467409_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:19.414099Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 497
2025-10-09T21:45:19.414110Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 497
2025-10-09T21:45:19.416321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 498 partition: 0 MaxSeqNo: 25 sessionId: src_id|fadef10c-a4da578-8b926bd8-df467409_0
2025-10-09T21:45:19.420223Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fadef10c-a4da578-8b926bd8-df467409_0] Write session: try to update token
2025-10-09T21:45:19.420243Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fadef10c-a4da578-8b926bd8-df467409_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:19.430134Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:19.430148Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:19.430151Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:19.430155Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:19.430165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341380437881228:5327]
2025-10-09T21:45:19.430170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341380437881228:5327]
2025-10-09T21:45:19.430173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341380437881228:5327]
2025-10-09T21:45:19.430235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:19.430246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.430249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:19.430320Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:19.430327Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:19.430331Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:19.430396Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:19.437592Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 498 sessionId: src_id|fadef10c-a4da578-8b926bd8-df467409_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:19.440395Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:19.466987Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341380799693753:5210]
2025-10-09T21:45:19.467318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 497, pipe: [1:7559341380799693753:5210]
2025-10-09T21:45:19.591126Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:19.591691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:19.591706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:19.591754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341380799693753:5210], now have 1 active actors on pipe
2025-10-09T21:45:19.592348Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:19.592065Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:19.592122Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:19.592604Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fadef10c-a4da578-8b926bd8-df467409_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:19.592646Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fadef10c-a4da578-8b926bd8-df467409_0] Write session will restart in 0.008223s
2025-10-09T21:45:19.592125Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:19.592134Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341380799693753:5210]
2025-10-09T21:45:19.592204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 498 reason: error at write: Disconnected sessionId: src_id|fadef10c-a4da578-8b926bd8-df467409_0
2025-10-09T21:45:19.592417Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 498 sessionId: src_id|fadef10c-a4da578-8b926bd8-df467409_0 is DEAD
2025-10-09T21:45:19.592684Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fadef10c-a4da578-8b926bd8-df467409_0] Write session: Do CDS request
2025-10-09T21:45:19.592716Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fadef10c-a4da578-8b926bd8-df467409_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:19.604048Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fadef10c-a4da578-8b926bd8-df467409_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:19.604281Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:19.604300Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 499
2025-10-09T21:45:19.604460Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 499 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:19.604493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 499 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:19.604500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=499 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:19.604504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:19.604380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:19.604992Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 499 sessionId:  describe result for acl check
2025-10-09T21:45:19.605030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:19.605037Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:19.605038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:19.605044Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341380799693790:5827] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:19.605601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341380799693790:5827] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:19.607957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341380799693790:5827] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:19.608490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:19.608715Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:19.608727Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:19.608740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:19.608807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:19.608849Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341380437881304:5329]
2025-10-09T21:45:19.609432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:19.609862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:19.609910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:19.610264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:19.610333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:19.610339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:19.610340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:19.610341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:19.610343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:19.610345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:19.610347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.610348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.610349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.610354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:19.610364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:19.610368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:19.610372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:19.610376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:19.610380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:19.610384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.610387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.610391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.610398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:19.610405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:19.610412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:19.610417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:19.610421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:19.610426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:19.610430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:19.610435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:19.610439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:19.610444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:19.610447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:19.610453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:19.610458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.610464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.610470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.610477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.610482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.610487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.610515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:19.611274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341380799693790:5827] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:19.611298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341380799693790:5827] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:19.611302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341380799693790:5827] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:19.611311Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 499 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:19.617294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:19.618904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:19.618920Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:19.618923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:19.618928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:19.618936Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 498 [2:7559341380437881304:5329]
2025-10-09T21:45:19.618953Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:19.618975Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:19.618979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:19.618983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:19.618986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:19.619005Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:19.619011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:19.619018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:19.619083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.619093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.619096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.619099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:19.619189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:19.619191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.619205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.619217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.619233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:19.619254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341380437881299:5329]
2025-10-09T21:45:19.619256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341380437881299:5329]
2025-10-09T21:45:19.619259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341380437881299:5329]
2025-10-09T21:45:19.619270Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:19.619282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:19.619285Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:19.619287Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:19.619288Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:19.619303Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:19.619306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:19.619318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341380799693829:5827], now have 1 active actors on pipe
2025-10-09T21:45:19.619322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341380799693758:2432], now have 1 active actors on pipe
2025-10-09T21:45:19.619362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341380437881299:5329]
2025-10-09T21:45:19.619379Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:19.619403Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:19.620667Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 498
2025-10-09T21:45:19.620676Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 498
2025-10-09T21:45:19.620944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:19.620964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:19.621012Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|36dae66-ae7a501f-32c3e70f-788d7f3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:19.621042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:19.621049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:19.621052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:19.621065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:19.621088Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:19.621128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:19.621461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:19.621472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:19.621491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:19.621730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 499 partition: 0 MaxSeqNo: 25 sessionId: src_id|36dae66-ae7a501f-32c3e70f-788d7f3_0
2025-10-09T21:45:19.625571Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fadef10c-a4da578-8b926bd8-df467409_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|36dae66-ae7a501f-32c3e70f-788d7f3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:19.628054Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36dae66-ae7a501f-32c3e70f-788d7f3_0] Write session: try to update token
2025-10-09T21:45:19.628074Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36dae66-ae7a501f-32c3e70f-788d7f3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:19.639563Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:19.639583Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:19.639586Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:19.639588Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:19.639598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341380437881299:5329]
2025-10-09T21:45:19.639603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341380437881299:5329]
2025-10-09T21:45:19.639606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341380437881299:5329]
2025-10-09T21:45:19.639645Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:19.639651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:19.639658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.639660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:19.639665Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:19.639669Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:19.639788Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:19.650152Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 499 sessionId: src_id|36dae66-ae7a501f-32c3e70f-788d7f3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:19.652671Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:19.692308Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341380799693836:5210]
2025-10-09T21:45:19.692758Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 498, pipe: [1:7559341380799693836:5210]
2025-10-09T21:45:19.831252Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:19.831946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:19.831968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:19.831994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341380799693836:5210], now have 1 active actors on pipe
2025-10-09T21:45:19.832269Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:19.833090Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341380799693836:5210]
2025-10-09T21:45:19.833127Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:19.833196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 499 reason: status is not ok: tablet will be restarted right now sessionId: src_id|36dae66-ae7a501f-32c3e70f-788d7f3_0
2025-10-09T21:45:19.833384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 499 sessionId: src_id|36dae66-ae7a501f-32c3e70f-788d7f3_0 is DEAD
2025-10-09T21:45:19.833526Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:19.834479Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36dae66-ae7a501f-32c3e70f-788d7f3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:19.834501Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36dae66-ae7a501f-32c3e70f-788d7f3_0] Write session will restart in 0.006588s
2025-10-09T21:45:19.834529Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36dae66-ae7a501f-32c3e70f-788d7f3_0] Write session: Do CDS request
2025-10-09T21:45:19.834538Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36dae66-ae7a501f-32c3e70f-788d7f3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:19.844207Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36dae66-ae7a501f-32c3e70f-788d7f3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:19.847326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:19.850111Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:19.850131Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 500
2025-10-09T21:45:19.850359Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 500 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:19.850410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 500 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:19.850416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=500 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:19.850420Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:19.850962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 500 sessionId:  describe result for acl check
2025-10-09T21:45:19.851003Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:19.851009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:19.851011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:19.851017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341380799693872:5831] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:19.851318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:19.851565Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:19.851575Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:19.851589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:19.851633Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341380799693872:5831] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:19.851656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:19.851704Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341380437881375:5331]
2025-10-09T21:45:19.852320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:19.852696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:19.852754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:19.853142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:19.853214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:19.853219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:19.853221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:19.853222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:19.853224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:19.853225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:19.853226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.853227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.853229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.853234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:19.853246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:19.853252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:19.853256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:19.853259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:19.853264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:19.853267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.853271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.853275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.853282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:19.853291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:19.853297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:19.853302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:19.853306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:19.853311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:19.853318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:19.853323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:19.853327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:19.853332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:19.853336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:19.853341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:19.853345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.853351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.853355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.853360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.853381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.853386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.853416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:19.861149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:19.861422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341380799693872:5831] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:19.862817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:19.862835Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:19.862838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:19.862842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:19.862848Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 499 [2:7559341380437881375:5331]
2025-10-09T21:45:19.862862Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:19.862884Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:19.862887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:19.862890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:19.862892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:19.862911Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:19.862917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:19.862924Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:19.862990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.862999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.863001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.863004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:19.863076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:19.863078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.863092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:19.863105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:19.863124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:19.863145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341380437881370:5331]
2025-10-09T21:45:19.863148Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341380437881370:5331]
2025-10-09T21:45:19.863150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341380437881370:5331]
2025-10-09T21:45:19.863169Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:19.863184Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:19.863206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:19.863209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:19.863214Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:19.863217Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:19.863221Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:19.863257Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341380437881370:5331]
2025-10-09T21:45:19.863266Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:19.863297Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:19.863521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341380799693839:2432], now have 1 active actors on pipe
2025-10-09T21:45:19.865086Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 499
2025-10-09T21:45:19.865376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341380799693872:5831] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:19.865394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341380799693872:5831] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:19.865399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341380799693872:5831] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:19.865410Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 500 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:19.866411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341380799693920:5831], now have 1 active actors on pipe
2025-10-09T21:45:19.867764Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 499
2025-10-09T21:45:19.868529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:19.868558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:19.868595Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fdf75dd0-d5fc2e77-35e5b785-46287bd7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:19.868649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:19.868661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:19.868665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:19.868680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:19.868700Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:19.868739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:19.869626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:19.869638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:19.869660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:19.869898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 500 partition: 0 MaxSeqNo: 25 sessionId: src_id|fdf75dd0-d5fc2e77-35e5b785-46287bd7_0
2025-10-09T21:45:19.874388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36dae66-ae7a501f-32c3e70f-788d7f3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fdf75dd0-d5fc2e77-35e5b785-46287bd7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:19.877267Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf75dd0-d5fc2e77-35e5b785-46287bd7_0] Write session: try to update token
2025-10-09T21:45:19.877291Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf75dd0-d5fc2e77-35e5b785-46287bd7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:19.882508Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:19.882524Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:19.882527Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:19.882529Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:19.882539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341380437881370:5331]
2025-10-09T21:45:19.882544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341380437881370:5331]
2025-10-09T21:45:19.882548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341380437881370:5331]
2025-10-09T21:45:19.882593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:19.882601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:19.882603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:19.882606Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:19.882613Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:19.882616Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:19.882739Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:19.893685Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 500 sessionId: src_id|fdf75dd0-d5fc2e77-35e5b785-46287bd7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:19.896271Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:19.933397Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341380799693927:5210]
2025-10-09T21:45:19.933667Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 499, pipe: [1:7559341380799693927:5210]
2025-10-09T21:45:20.047310Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:20.048443Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:20.048454Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341380799693927:5210]
2025-10-09T21:45:20.048482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:20.048485Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:20.048561Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 500 reason: error at write: Disconnected sessionId: src_id|fdf75dd0-d5fc2e77-35e5b785-46287bd7_0
2025-10-09T21:45:20.048819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 500 sessionId: src_id|fdf75dd0-d5fc2e77-35e5b785-46287bd7_0 is DEAD
2025-10-09T21:45:20.049029Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf75dd0-d5fc2e77-35e5b785-46287bd7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:20.049040Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf75dd0-d5fc2e77-35e5b785-46287bd7_0] Write session will restart in 0.006763s
2025-10-09T21:45:20.049068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf75dd0-d5fc2e77-35e5b785-46287bd7_0] Write session: Do CDS request
2025-10-09T21:45:20.048090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:20.049076Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf75dd0-d5fc2e77-35e5b785-46287bd7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:20.048109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:20.048135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341380799693927:5210], now have 1 active actors on pipe
2025-10-09T21:45:20.048598Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:20.056602Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf75dd0-d5fc2e77-35e5b785-46287bd7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:20.056829Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:20.056848Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 501
2025-10-09T21:45:20.057210Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 501 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:20.057263Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 501 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:20.057269Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=501 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:20.057273Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:20.057814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 501 sessionId:  describe result for acl check
2025-10-09T21:45:20.057866Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:20.057868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:20.057870Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:20.057876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341385094661260:5842] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:20.058483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341385094661260:5842] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:20.061408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341385094661260:5842] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:20.065008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341385094661260:5842] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:20.065025Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341385094661260:5842] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:20.065029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341385094661260:5842] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:20.065039Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 501 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:20.069610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:20.070530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:20.070935Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:20.070950Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:20.070966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:20.071059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:20.071117Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341384732848775:5343]
2025-10-09T21:45:20.071860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:20.072323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:20.072393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:20.072781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:20.072842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:20.072850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:20.072851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:20.072852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:20.072853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:20.072856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:20.072858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.072859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.072860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.072866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:20.072874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:20.072878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:20.072881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:20.072884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:20.072887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:20.072891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.072895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.072898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.072905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:20.072912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:20.072922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:20.072928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:20.072933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:20.072938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:20.072943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:20.072948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:20.072953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:20.072958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:20.072963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:20.072969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:20.072973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.072978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.072983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.072989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.072993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.072998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.073028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:20.080881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:20.082016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:20.082023Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:20.082025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:20.082054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:20.082063Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 500 [2:7559341384732848775:5343]
2025-10-09T21:45:20.082078Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:20.082098Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:20.082102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:20.082105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:20.082108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:20.082128Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:20.082135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:20.082143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:20.082224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.082226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.082555Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 500
2025-10-09T21:45:20.082228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.082230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:20.082308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:20.082602Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 500
2025-10-09T21:45:20.082311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.082327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.082340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.082357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:20.082378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341384732848770:5343]
2025-10-09T21:45:20.082383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341384732848770:5343]
2025-10-09T21:45:20.082385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341384732848770:5343]
2025-10-09T21:45:20.082394Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:20.082412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341385094661313:5842], now have 1 active actors on pipe
2025-10-09T21:45:20.082422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:20.082421Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:20.082425Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:20.082428Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:20.082435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:20.082439Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:20.082446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341385094661234:2432], now have 1 active actors on pipe
2025-10-09T21:45:20.082485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341384732848770:5343]
2025-10-09T21:45:20.082491Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:20.082500Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:20.082764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:20.082774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:20.082804Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|29109080-5e7e8dae-b6ba980a-c6741c07_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:20.082813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:20.082817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:20.082820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:20.082831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:20.082843Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:20.082866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:20.083126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:20.083135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:20.083158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:20.083252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 501 partition: 0 MaxSeqNo: 25 sessionId: src_id|29109080-5e7e8dae-b6ba980a-c6741c07_0
2025-10-09T21:45:20.083694Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf75dd0-d5fc2e77-35e5b785-46287bd7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|29109080-5e7e8dae-b6ba980a-c6741c07_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:20.086394Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29109080-5e7e8dae-b6ba980a-c6741c07_0] Write session: try to update token
2025-10-09T21:45:20.086415Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29109080-5e7e8dae-b6ba980a-c6741c07_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:20.095273Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:20.095292Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:20.095302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341384732848770:5343]
2025-10-09T21:45:20.095352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:20.095350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:20.095524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:20.095668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:20.095805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:20.095942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:20.096153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:20.096201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:20.107290Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 501 sessionId: src_id|29109080-5e7e8dae-b6ba980a-c6741c07_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:20.109904Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:20.110162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:20.110185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:20.111568Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:20.111580Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:20.111583Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:20.111585Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:20.111595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341384732848770:5343]
2025-10-09T21:45:20.111599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341384732848770:5343]
2025-10-09T21:45:20.111604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341384732848770:5343]
2025-10-09T21:45:20.111648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:20.111669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:20.111680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.111683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:20.111707Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:20.111716Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:20.111720Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:20.111802Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:20.111817Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:20.150350Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341385094661336:5210]
2025-10-09T21:45:20.154557Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 500, pipe: [1:7559341385094661336:5210]
2025-10-09T21:45:20.279397Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:20.287613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:20.287640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:20.287812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.287869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.287934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.287990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.288035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.288045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.288108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.288160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.288220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.288285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.288324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.288327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.288392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.288443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.288500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.288557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.288595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.288597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.288661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.288720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.288775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.288832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.288872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.288875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.288945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.289005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.289069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.289126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.289174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.289178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.289242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.289296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.289347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.289400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.289439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.289441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.289506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.289563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.289625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.289683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.289721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.289723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.289786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.289837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.289894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.289950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.289989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.289992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.291262Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:20.291325Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:20.291333Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:20.291364Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 501 reason: error at write: Disconnected sessionId: src_id|29109080-5e7e8dae-b6ba980a-c6741c07_0
2025-10-09T21:45:20.291581Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 501 sessionId: src_id|29109080-5e7e8dae-b6ba980a-c6741c07_0 is DEAD
2025-10-09T21:45:20.291632Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341385094661336:5210]
2025-10-09T21:45:20.291676Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:45:20.291735Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29109080-5e7e8dae-b6ba980a-c6741c07_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:20.291758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29109080-5e7e8dae-b6ba980a-c6741c07_0] Write session will restart in 0.009311s
2025-10-09T21:45:20.291783Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29109080-5e7e8dae-b6ba980a-c6741c07_0] Write session: Do CDS request
2025-10-09T21:45:20.291790Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29109080-5e7e8dae-b6ba980a-c6741c07_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:20.291765Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 5 consumer shared/user session shared/user_1_5_1413312754900983240_v1 is DEAD
2025-10-09T21:45:20.292077Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:45:20.292131Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341269130519391:5205] disconnected.
2025-10-09T21:45:20.292146Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341269130519391:5205] disconnected; active server actors: 1
2025-10-09T21:45:20.292150Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341269130519391:5205] client user disconnected session shared/user_1_5_1413312754900983240_v1
2025-10-09T21:45:20.292191Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:45:20.292198Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:45:20.292206Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.009365s
2025-10-09T21:45:20.290322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.290389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.290452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.290512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.290556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.290559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.290627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.290684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.290747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.290805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.290845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.290848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.290961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341385094661336:5210], now have 1 active actors on pipe
2025-10-09T21:45:20.292082Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:20.304642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:20.306253Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:45:20.306524Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:20.306578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29109080-5e7e8dae-b6ba980a-c6741c07_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:20.306524Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:45:20.306530Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 6
2025-10-09T21:45:20.306543Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 502
2025-10-09T21:45:20.306727Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 6 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:45:20.306790Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:45:20.306840Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 502 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:20.306887Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 auth for : user
2025-10-09T21:45:20.306892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 502 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:20.306898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=502 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:20.306902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:20.307162Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 Handle describe topics response
2025-10-09T21:45:20.307203Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 auth is DEAD
2025-10-09T21:45:20.307227Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:45:20.307430Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 502 sessionId:  describe result for acl check
2025-10-09T21:45:20.307459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:20.307461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:20.307465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:20.307470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341385094661377:5848] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:20.307494Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:45:20.307622Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341385094661378:5847] connected; active server actors: 1
2025-10-09T21:45:20.307635Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559341385094661378:5847] session shared/user_1_6_6774085051053870161_v1
2025-10-09T21:45:20.307645Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:45:20.307671Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:45:20.307678Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_6_6774085051053870161_v1" (Sender=[1:7559341385094661372:5847], Pipe=[1:7559341385094661378:5847], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:45:20.307683Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:45:20.307694Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:45:20.307704Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_6_6774085051053870161_v1" (Sender=[1:7559341385094661372:5847], Pipe=[1:7559341385094661378:5847], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:45:20.307716Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_6_6774085051053870161_v1" sender [1:7559341385094661372:5847] lock partition 0 for ReadingSession "shared/user_1_6_6774085051053870161_v1" (Sender=[1:7559341385094661372:5847], Pipe=[1:7559341385094661378:5847], Partitions=[], ActiveFamilyCount=1) generation 1 step 6
2025-10-09T21:45:20.307726Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:45:20.307731Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000034s
2025-10-09T21:45:20.307968Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_6_6774085051053870161_v1
2025-10-09T21:45:20.307947Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 6 Session: "shared/user_1_6_6774085051053870161_v1" ClientId: "user" PipeClient { RawX1: 7559341385094661378 RawX2: 4503603922343639 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:45:20.307969Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:45:20.308077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:20.308057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341385094661377:5848] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:20.308318Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:20.308333Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:20.308347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:20.308406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:20.308469Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341384732848862:5348]
2025-10-09T21:45:20.309011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:20.309362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:20.309428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:20.309751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:20.309811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:20.309812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:20.309813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:20.309814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:20.309816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:20.309817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:20.309818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.309819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.309820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.309826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:20.309836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:20.309841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:20.309845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:20.309849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:20.309853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:20.309856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.309859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.309864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.309872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:20.309880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:20.309886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:20.309891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:20.309895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:20.309900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:20.309904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:20.309908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:20.309913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:20.309917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:20.309921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:20.309927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:20.309932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.309937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.309941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.309946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.309950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.309955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.309982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:20.311829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341385094661377:5848] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:20.317302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341385094661377:5848] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:20.317322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341385094661377:5848] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:20.317346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341385094661377:5848] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:20.317357Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 502 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:20.318167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:20.319638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:20.319658Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:20.319660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:20.319665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:20.319675Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 501 [2:7559341384732848862:5348]
2025-10-09T21:45:20.319692Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:20.319711Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:20.319715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:20.319718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:20.319721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:20.319743Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:20.319749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:20.319756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:20.319816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.319826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.319828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.319830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:20.319907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:20.319910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.319925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.319938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.319957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:20.319978Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341384732848854:5348]
2025-10-09T21:45:20.319980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341384732848854:5348]
2025-10-09T21:45:20.319982Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341384732848854:5348]
2025-10-09T21:45:20.319998Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:20.320022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:20.320036Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:20.320038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:20.320053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:20.320062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:20.320065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:20.320082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341384732848854:5348]
2025-10-09T21:45:20.320086Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:20.320164Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:20.320310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341385094661339:2432], now have 1 active actors on pipe
2025-10-09T21:45:20.320318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341385094661420:5848], now have 1 active actors on pipe
2025-10-09T21:45:20.320322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341385094661382:5852], now have 1 active actors on pipe
2025-10-09T21:45:20.320356Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 501
2025-10-09T21:45:20.320369Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 501
2025-10-09T21:45:20.320399Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 501, pipe: [1:7559341385094661382:5852]
2025-10-09T21:45:20.320471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:20.320493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:20.320515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:20.320518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:20.320525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_6_6774085051053870161_v1 on pipe: [1:7559341385094661382:5852]
2025-10-09T21:45:20.320571Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|decb524c-700987a8-6c054dc5-70fd59f1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:20.320580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:20.320585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:20.320587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:20.320600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:20.320613Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:20.320627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:20.320632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:20.320634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:20.320651Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_6_6774085051053870161_v1
2025-10-09T21:45:20.320654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:20.320686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:20.320707Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:45:20.320854Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_6_6774085051053870161_v1:1 with generation 501
2025-10-09T21:45:20.320967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:20.320973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:20.321006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:20.321134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 502 partition: 0 MaxSeqNo: 25 sessionId: src_id|decb524c-700987a8-6c054dc5-70fd59f1_0
2025-10-09T21:45:20.321568Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29109080-5e7e8dae-b6ba980a-c6741c07_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|decb524c-700987a8-6c054dc5-70fd59f1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:20.324180Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|decb524c-700987a8-6c054dc5-70fd59f1_0] Write session: try to update token
2025-10-09T21:45:20.324201Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|decb524c-700987a8-6c054dc5-70fd59f1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:20.325772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:45:20.325809Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:45:20.325820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:20.325823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:20.325825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:20.325846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:45:20.325866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:45:20.326069Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046320320 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:45:20.326095Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:45:20.326180Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 sending to client partition status
2025-10-09T21:45:20.335839Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:20.335856Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:20.335860Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:20.335861Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:20.335875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341384732848854:5348]
2025-10-09T21:45:20.335880Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341384732848854:5348]
2025-10-09T21:45:20.335884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341384732848854:5348]
2025-10-09T21:45:20.335929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:20.335937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.335939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:20.336077Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:20.337842Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:20.337857Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:20.337861Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:20.343415Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 502 sessionId: src_id|decb524c-700987a8-6c054dc5-70fd59f1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:20.346128Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:20.504831Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:20.505457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:20.505480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:20.505743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_6_6774085051053870161_v1
2025-10-09T21:45:20.505763Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:20.505760Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:20.505802Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:20.505814Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341385094661382:5852]
2025-10-09T21:45:20.505816Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:20.505801Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_6_6774085051053870161_v1
2025-10-09T21:45:20.505889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 502 reason: error at write: Disconnected sessionId: src_id|decb524c-700987a8-6c054dc5-70fd59f1_0
2025-10-09T21:45:20.506126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 502 sessionId: src_id|decb524c-700987a8-6c054dc5-70fd59f1_0 is DEAD
2025-10-09T21:45:20.506298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|decb524c-700987a8-6c054dc5-70fd59f1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:20.506311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|decb524c-700987a8-6c054dc5-70fd59f1_0] Write session will restart in 0.006725s
2025-10-09T21:45:20.506337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|decb524c-700987a8-6c054dc5-70fd59f1_0] Write session: Do CDS request
2025-10-09T21:45:20.506343Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|decb524c-700987a8-6c054dc5-70fd59f1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:20.514684Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|decb524c-700987a8-6c054dc5-70fd59f1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:20.514780Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:20.514820Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 503
2025-10-09T21:45:20.514978Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 503 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:20.515025Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 503 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:20.515037Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=503 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:20.515043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:20.515601Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 503 sessionId:  describe result for acl check
2025-10-09T21:45:20.515640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:20.515647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:20.515649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:20.515657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341385094661450:5855] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:20.516249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341385094661450:5855] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:20.518844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341385094661450:5855] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:20.521991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:20.522255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341385094661450:5855] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:20.522269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341385094661450:5855] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:20.522272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341385094661450:5855] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:20.522280Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 503 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:20.522779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:20.522990Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:20.522999Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:20.523012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:20.523067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:20.523107Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341384732848940:5351]
2025-10-09T21:45:20.523725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:20.524164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:20.524233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:20.524564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:20.524623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:20.524624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:20.524625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:20.524626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:20.524628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:20.524629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:20.524630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.524631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.524632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.524637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:20.524647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:20.524650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:20.524653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:20.524656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:20.524660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:20.524663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.524666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.524669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.524676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:20.524684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:20.524691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:20.524695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:20.524700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:20.524704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:20.524708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:20.524714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:20.524718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:20.524723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:20.524729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:20.524735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:20.524739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.524744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.524747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.524752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.524756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.524761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.524789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:20.536080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:20.537204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:20.537221Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:20.537224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:20.537230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:20.537237Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 502 [2:7559341384732848940:5351]
2025-10-09T21:45:20.537256Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:20.537277Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:20.537281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:20.537284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:20.537287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:20.537307Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:20.537329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:20.537342Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:20.537413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.537422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.537424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.537427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:20.537503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:20.537508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.537523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.537535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.537552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:20.537574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341384732848934:5351]
2025-10-09T21:45:20.537577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341384732848934:5351]
2025-10-09T21:45:20.537580Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341384732848934:5351]
2025-10-09T21:45:20.537588Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:20.537605Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:20.537619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:20.537622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:20.537635Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:20.537638Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:20.537640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:20.537679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341384732848934:5351]
2025-10-09T21:45:20.537694Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:20.537705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341385094661501:5855], now have 1 active actors on pipe
2025-10-09T21:45:20.537713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341385094661430:2432], now have 1 active actors on pipe
2025-10-09T21:45:20.537801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 502
2025-10-09T21:45:20.537853Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 502
2025-10-09T21:45:20.537755Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:20.538070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:20.538086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:20.538312Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|570e37cc-26914054-7610c1b6-4c6369ee_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:20.538329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:20.538333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:20.538336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:20.538347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:20.538371Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:20.538392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:20.539512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 503 partition: 0 MaxSeqNo: 25 sessionId: src_id|570e37cc-26914054-7610c1b6-4c6369ee_0
2025-10-09T21:45:20.539370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:20.539380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:20.539401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:20.539944Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|decb524c-700987a8-6c054dc5-70fd59f1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|570e37cc-26914054-7610c1b6-4c6369ee_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:20.542454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|570e37cc-26914054-7610c1b6-4c6369ee_0] Write session: try to update token
2025-10-09T21:45:20.542473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|570e37cc-26914054-7610c1b6-4c6369ee_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:20.559590Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 503 sessionId: src_id|570e37cc-26914054-7610c1b6-4c6369ee_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:20.562184Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:20.560211Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:20.560227Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:20.560241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341384732848934:5351]
2025-10-09T21:45:20.560292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:20.560488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:20.560494Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:20.560631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:20.560763Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:20.560892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:20.561092Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:20.561131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:20.562419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:20.562436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:20.562472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:20.562586Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:20.567780Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:20.567798Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:20.567801Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:20.567803Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:20.567813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341384732848934:5351]
2025-10-09T21:45:20.567817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341384732848934:5351]
2025-10-09T21:45:20.567824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341384732848934:5351]
2025-10-09T21:45:20.567867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:20.567874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.567876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:20.568008Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:20.570216Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:20.570231Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:20.570235Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:20.606225Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341385094661515:5852]
2025-10-09T21:45:20.606647Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 502, pipe: [1:7559341385094661515:5852]
2025-10-09T21:45:20.731963Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:20.745836Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341385094661515:5852]
2025-10-09T21:45:20.745878Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:20.745936Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:20.745938Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:20.745981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 503 reason: error at write: Disconnected sessionId: src_id|570e37cc-26914054-7610c1b6-4c6369ee_0
2025-10-09T21:45:20.742249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:20.742276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:20.742450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.742511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.742573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.742635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.742681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.742690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.742760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.742814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.742874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.742935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.742983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.742987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.743051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.743104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.743161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.743221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.743261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.743267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.743338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.743401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.743465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.743529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.743568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.743572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.743642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.743701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.743761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.743818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.743859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.743863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.743925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.743977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.744053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.744113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.744154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.744157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.744221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.744282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.744343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.744399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.744440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.744443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.744519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.744576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.744634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.744692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.744732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.744735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.744800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.744855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.744913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.744971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.745012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.745014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.745083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:20.745137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:20.745194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:20.745258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:20.745299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:20.745303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:20.745398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341385094661515:5852], now have 1 active actors on pipe
2025-10-09T21:45:20.745963Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:20.749303Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|570e37cc-26914054-7610c1b6-4c6369ee_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:20.749316Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|570e37cc-26914054-7610c1b6-4c6369ee_0] Write session will restart in 0.009116s
2025-10-09T21:45:20.749341Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|570e37cc-26914054-7610c1b6-4c6369ee_0] Write session: Do CDS request
2025-10-09T21:45:20.749348Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|570e37cc-26914054-7610c1b6-4c6369ee_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:20.746252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 503 sessionId: src_id|570e37cc-26914054-7610c1b6-4c6369ee_0 is DEAD
2025-10-09T21:45:20.759619Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|570e37cc-26914054-7610c1b6-4c6369ee_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:20.759822Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:20.759850Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 504
2025-10-09T21:45:20.760069Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 504 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:20.760116Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 504 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:20.760122Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=504 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:20.760127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:20.760577Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 504 sessionId:  describe result for acl check
2025-10-09T21:45:20.760615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:20.760617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:20.760619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:20.760624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341385094661537:5860] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:20.761166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341385094661537:5860] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:20.764244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341385094661537:5860] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:20.764689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:20.768126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341385094661537:5860] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:20.768142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341385094661537:5860] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:20.768146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341385094661537:5860] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:20.768154Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 504 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:20.767959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:20.768198Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:20.768202Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:20.768216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:20.768278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:20.768313Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341384732849015:5353]
2025-10-09T21:45:20.768916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:20.769278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:20.769335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:20.769677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:20.769739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:20.769748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:20.769749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:20.769751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:20.769752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:20.769753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:20.769755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.769756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.769757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.769764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:20.769776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:20.769780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:20.769785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:20.769789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:20.769793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:20.769798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.769802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.769805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.769814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:20.769824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:20.769831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:20.769837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:20.769841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:20.769847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:20.769852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:20.769857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:20.769862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:20.769868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:20.769873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:20.769878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:20.769883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.769888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.769893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.769898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.769903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.769908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.769939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:20.781547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:20.782674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:20.782686Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:20.782689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:20.782694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:20.782702Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 503 [2:7559341384732849015:5353]
2025-10-09T21:45:20.782718Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:20.782738Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:20.782741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:20.782744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:20.782746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:20.782768Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:20.782774Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:20.782791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:20.782856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.782863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.782865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.782868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:20.782942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:20.782950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.782963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:20.782979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:20.783007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:20.783036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341385094661589:5860], now have 1 active actors on pipe
2025-10-09T21:45:20.783040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341385094661518:2432], now have 1 active actors on pipe
2025-10-09T21:45:20.783048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341384732849009:5353]
2025-10-09T21:45:20.783055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341384732849009:5353]
2025-10-09T21:45:20.783060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341384732849009:5353]
2025-10-09T21:45:20.783073Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:20.783088Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:20.783093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:20.783096Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:20.783098Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:20.783108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:20.783111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:20.783155Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341384732849009:5353]
2025-10-09T21:45:20.783166Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:20.783137Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 503
2025-10-09T21:45:20.783171Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:20.783137Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 503
2025-10-09T21:45:20.783380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:20.783393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:20.783427Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b0440f01-cd02dafe-7e9a1b27-a0a17231_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:20.783447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:20.783451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:20.783454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:20.783466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:20.783478Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:20.783495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:20.783749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:20.783759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:20.783777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:20.785007Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 504 partition: 0 MaxSeqNo: 25 sessionId: src_id|b0440f01-cd02dafe-7e9a1b27-a0a17231_0
2025-10-09T21:45:20.785481Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|570e37cc-26914054-7610c1b6-4c6369ee_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b0440f01-cd02dafe-7e9a1b27-a0a17231_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:20.788113Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0440f01-cd02dafe-7e9a1b27-a0a17231_0] Write session: try to update token
2025-10-09T21:45:20.788149Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0440f01-cd02dafe-7e9a1b27-a0a17231_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:20.797029Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:20.797045Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:20.797056Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341384732849009:5353]
2025-10-09T21:45:20.797102Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:20.797168Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:20.797286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:20.797618Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:20.797758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:20.797915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:20.798150Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:20.798195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:20.811179Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 504 sessionId: src_id|b0440f01-cd02dafe-7e9a1b27-a0a17231_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:20.812395Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:20.812413Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:20.812416Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:20.812419Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:20.812429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341384732849009:5353]
2025-10-09T21:45:20.812432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341384732849009:5353]
2025-10-09T21:45:20.812436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341384732849009:5353]
2025-10-09T21:45:20.812487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:20.812496Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:20.812499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:20.812502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:20.812503Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:20.812509Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:20.812633Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:20.813816Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:20.846452Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341385094661605:5852]
2025-10-09T21:45:20.846827Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 503, pipe: [1:7559341385094661605:5852]
2025-10-09T21:45:20.872880Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:45:20.872909Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 5
2025-10-09T21:45:21.000256Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:21.000698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:21.002125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:21.002554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341385094661605:5852], now have 1 active actors on pipe
2025-10-09T21:45:21.002917Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:21.005004Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:21.005006Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341385094661605:5852]
2025-10-09T21:45:21.005020Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:21.005044Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:21.005057Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 5
2025-10-09T21:45:21.005083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 504 reason: error at write: Disconnected sessionId: src_id|b0440f01-cd02dafe-7e9a1b27-a0a17231_0
2025-10-09T21:45:21.005260Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 504 sessionId: src_id|b0440f01-cd02dafe-7e9a1b27-a0a17231_0 is DEAD
2025-10-09T21:45:21.006113Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0440f01-cd02dafe-7e9a1b27-a0a17231_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:21.006129Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0440f01-cd02dafe-7e9a1b27-a0a17231_0] Write session will restart in 0.006269s
2025-10-09T21:45:21.006153Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0440f01-cd02dafe-7e9a1b27-a0a17231_0] Write session: Do CDS request
2025-10-09T21:45:21.006160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0440f01-cd02dafe-7e9a1b27-a0a17231_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:21.013590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0440f01-cd02dafe-7e9a1b27-a0a17231_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:21.013853Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:21.013869Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 505
2025-10-09T21:45:21.014069Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 505 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:21.014116Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 505 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:21.014122Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=505 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:21.014127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:21.014703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 505 sessionId:  describe result for acl check
2025-10-09T21:45:21.014751Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:21.014753Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:21.014755Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:21.014762Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341389389628922:5865] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:21.015357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341389389628922:5865] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:21.021876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341389389628922:5865] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:21.033206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341389389628922:5865] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:21.033228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341389389628922:5865] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:21.033232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341389389628922:5865] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:21.033244Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 505 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:21.043663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:21.047902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:21.048176Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:21.048181Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:21.048194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:21.048261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:21.048302Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341389027816420:5366]
2025-10-09T21:45:21.048988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:21.049840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:21.049907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:21.050319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:21.050397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:21.050398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:21.050400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:21.050401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:21.050403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:21.050404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:21.050405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.050407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.050408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.050414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:21.050425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:21.050431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:21.050436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:21.050440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:21.050445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:21.050463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.050467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.050471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.050480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:21.050488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:21.050495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:21.050501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:21.050510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:21.050516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:21.050521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:21.050527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:21.050532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:21.050538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:21.050545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:21.050551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:21.050556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.050562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.050566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.050572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.050577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.050583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.050616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:21.064100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:21.065950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:21.065967Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:21.065969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:21.065977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:21.065985Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 504 [2:7559341389027816420:5366]
2025-10-09T21:45:21.066001Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:21.066023Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:21.066026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:21.066056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:21.066058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:21.066079Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:21.066087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:21.066101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:21.066174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.066179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.066181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.066183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:21.066511Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 504
2025-10-09T21:45:21.066542Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 504
2025-10-09T21:45:21.067127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:21.067139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.067156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.067166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.067182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:21.067264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341389389628912:2432], now have 1 active actors on pipe
2025-10-09T21:45:21.067272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341389389628981:5865], now have 1 active actors on pipe
2025-10-09T21:45:21.067312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:21.067322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:21.067356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341389027816414:5366]
2025-10-09T21:45:21.067362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341389027816414:5366]
2025-10-09T21:45:21.067367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341389027816414:5366]
2025-10-09T21:45:21.067379Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:21.067439Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:45:21.067488Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|48a32d3f-25298cdf-3719db73-8107f9c7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:21.067499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:21.067503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:21.067506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:21.067527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:21.067540Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:21.067554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:21.067569Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:21.067573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:21.067640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:21.067667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341389027816414:5366]
2025-10-09T21:45:21.067670Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:21.067699Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:21.067703Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:21.067705Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:21.067714Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:21.067948Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 5 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:45:21.068000Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:45:21.068053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:21.068063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:21.068085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:21.068818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 505 partition: 0 MaxSeqNo: 25 sessionId: src_id|48a32d3f-25298cdf-3719db73-8107f9c7_0
2025-10-09T21:45:21.069219Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0440f01-cd02dafe-7e9a1b27-a0a17231_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|48a32d3f-25298cdf-3719db73-8107f9c7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:21.071792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48a32d3f-25298cdf-3719db73-8107f9c7_0] Write session: try to update token
2025-10-09T21:45:21.071812Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48a32d3f-25298cdf-3719db73-8107f9c7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:21.089448Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 505 sessionId: src_id|48a32d3f-25298cdf-3719db73-8107f9c7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:21.092457Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:21.093209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:21.093232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:21.093254Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:21.093257Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:21.093266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341389027816414:5366]
2025-10-09T21:45:21.093323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:21.093497Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:21.093638Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:21.093775Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:21.093911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:21.094163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:21.094197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:21.094234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:21.094796Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:21.095170Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:21.103949Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341389389628999:5852]
2025-10-09T21:45:21.104423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341389389628999:5852], now have 1 active actors on pipe
2025-10-09T21:45:21.104592Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 504, pipe: [1:7559341389389628999:5852]
2025-10-09T21:45:21.108805Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:21.108823Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:21.108826Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:21.108828Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:21.108839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341389027816414:5366]
2025-10-09T21:45:21.108842Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341389027816414:5366]
2025-10-09T21:45:21.108846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341389027816414:5366]
2025-10-09T21:45:21.108893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:21.108901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.108904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:21.109080Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:21.109090Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:21.109093Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:21.109279Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:21.324001Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:21.332940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:21.332970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:21.333151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:21.333213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:21.333279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:21.333340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:21.333671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:21.333677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:21.333746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:21.333805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:21.333868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:21.333930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:21.333975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:21.333978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:21.334323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:21.334388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:21.334451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:21.334517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:21.334568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:21.334571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:21.334644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:21.334707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:21.334772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:21.334831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:21.334877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:21.334881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:21.334950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:21.335010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:21.335075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:21.335132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:21.335181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:21.335184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:21.335252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:21.335312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:21.335375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:21.335440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:21.335490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:21.335494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:21.335563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:21.335615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:21.335673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:21.335733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:21.335777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:21.335780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:21.335845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:21.335900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:21.335956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:21.336014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:21.336056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:21.336059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:21.336121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:21.336173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:21.336229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:21.336286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:21.336327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:21.336331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:21.336395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:21.336449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:21.336505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:21.336562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:21.336605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:21.336608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:21.338934Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:21.338989Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341389389628999:5852]
2025-10-09T21:45:21.339014Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:21.339016Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:21.339046Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 505 reason: error at write: Disconnected sessionId: src_id|48a32d3f-25298cdf-3719db73-8107f9c7_0
2025-10-09T21:45:21.339262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 505 sessionId: src_id|48a32d3f-25298cdf-3719db73-8107f9c7_0 is DEAD
2025-10-09T21:45:21.340114Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:21.342587Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48a32d3f-25298cdf-3719db73-8107f9c7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:21.342599Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48a32d3f-25298cdf-3719db73-8107f9c7_0] Write session will restart in 0.006273s
2025-10-09T21:45:21.342627Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48a32d3f-25298cdf-3719db73-8107f9c7_0] Write session: Do CDS request
2025-10-09T21:45:21.342634Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48a32d3f-25298cdf-3719db73-8107f9c7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:21.350964Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48a32d3f-25298cdf-3719db73-8107f9c7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:21.351033Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:21.351051Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 506
2025-10-09T21:45:21.351282Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 506 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:21.351344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 506 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:21.351355Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=506 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:21.351359Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:21.351886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 506 sessionId:  describe result for acl check
2025-10-09T21:45:21.351920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:21.351923Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:21.351925Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:21.351932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341389389629044:5876] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:21.352536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341389389629044:5876] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:21.357261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341389389629044:5876] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:21.367001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341389389629044:5876] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:21.367020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341389389629044:5876] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:21.367024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341389389629044:5876] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:21.367035Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 506 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:21.368975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:21.380604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:21.380874Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:21.380878Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:21.380891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:21.380950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:21.380993Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341389027816512:5373]
2025-10-09T21:45:21.381609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:21.385735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:21.385831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:21.386258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:21.386335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:21.386336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:21.386338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:21.386339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:21.386340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:21.386342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:21.386343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.386345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.386346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.386353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:21.386365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:21.386368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:21.386372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:21.386376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:21.386379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:21.386383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.386387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.386390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.386399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:21.386409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:21.386416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:21.386421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:21.386425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:21.386432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:21.386437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:21.386441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:21.386451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:21.386485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:21.386490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:21.386495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:21.386499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.386504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.386508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.386512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.386516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.386521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.386548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:21.398474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:21.400151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:21.400158Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:21.400160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:21.400165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:21.400173Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 505 [2:7559341389027816512:5373]
2025-10-09T21:45:21.400192Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:21.400234Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:21.400238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:21.400241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:21.400244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:21.400266Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:21.400272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:21.400280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:21.400346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.400348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.400350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.400353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:21.400434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:21.400437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.400453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.400464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.400479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:21.400502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341389027816505:5373]
2025-10-09T21:45:21.400505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341389027816505:5373]
2025-10-09T21:45:21.400507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341389027816505:5373]
2025-10-09T21:45:21.400521Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:21.400537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:21.400552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:21.400555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:21.400601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341389027816505:5373]
2025-10-09T21:45:21.400604Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:21.402132Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:21.402144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:21.402147Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:21.402155Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:21.402448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341389389629102:5876], now have 1 active actors on pipe
2025-10-09T21:45:21.402455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341389389629031:2432], now have 1 active actors on pipe
2025-10-09T21:45:21.403069Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 505
2025-10-09T21:45:21.403106Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 505
2025-10-09T21:45:21.403497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:21.403534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:21.403585Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|57d4511f-e65bea79-f0698e73-ba37935d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:21.403598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:21.403608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:21.403612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:21.403631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:21.403653Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:21.403682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:21.407326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:21.407340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:21.407388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:21.412615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 506 partition: 0 MaxSeqNo: 25 sessionId: src_id|57d4511f-e65bea79-f0698e73-ba37935d_0
2025-10-09T21:45:21.413042Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48a32d3f-25298cdf-3719db73-8107f9c7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|57d4511f-e65bea79-f0698e73-ba37935d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:21.415646Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57d4511f-e65bea79-f0698e73-ba37935d_0] Write session: try to update token
2025-10-09T21:45:21.415669Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57d4511f-e65bea79-f0698e73-ba37935d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:21.428295Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:21.428313Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:21.428316Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:21.428318Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:21.428328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341389027816505:5373]
2025-10-09T21:45:21.428333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341389027816505:5373]
2025-10-09T21:45:21.428337Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341389027816505:5373]
2025-10-09T21:45:21.428383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:21.428384Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:21.428390Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:21.428390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.428393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:21.428394Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:21.428532Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:21.439572Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 506 sessionId: src_id|57d4511f-e65bea79-f0698e73-ba37935d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:21.439859Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341389389629115:5852]
2025-10-09T21:45:21.440627Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 505, pipe: [1:7559341389389629115:5852]
2025-10-09T21:45:21.442139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:21.753733Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:21.754298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341389389629115:5852], now have 1 active actors on pipe
2025-10-09T21:45:21.754343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:21.754351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:21.754934Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:21.757979Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57d4511f-e65bea79-f0698e73-ba37935d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:21.757993Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57d4511f-e65bea79-f0698e73-ba37935d_0] Write session will restart in 0.008045s
2025-10-09T21:45:21.758019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57d4511f-e65bea79-f0698e73-ba37935d_0] Write session: Do CDS request
2025-10-09T21:45:21.758026Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57d4511f-e65bea79-f0698e73-ba37935d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:21.757389Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341389389629115:5852]
2025-10-09T21:45:21.757431Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:21.757512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 506 reason: status is not ok: tablet will be restarted right now sessionId: src_id|57d4511f-e65bea79-f0698e73-ba37935d_0
2025-10-09T21:45:21.757706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 506 sessionId: src_id|57d4511f-e65bea79-f0698e73-ba37935d_0 is DEAD
2025-10-09T21:45:21.757936Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:21.769319Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57d4511f-e65bea79-f0698e73-ba37935d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:21.769659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:21.769676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 507
2025-10-09T21:45:21.770129Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 507 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:21.770165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 507 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:21.770171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=507 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:21.770175Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:21.771472Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 507 sessionId:  describe result for acl check
2025-10-09T21:45:21.771514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:21.771516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:21.771517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:21.771524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341389389629157:5883] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:21.772117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341389389629157:5883] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:21.784093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341389389629157:5883] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:21.787897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341389389629157:5883] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:21.787913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341389389629157:5883] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:21.787918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341389389629157:5883] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:21.787928Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 507 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:21.790483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:21.794109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:21.794369Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:21.794373Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:21.794387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:21.794446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:21.794487Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341389027816593:5375]
2025-10-09T21:45:21.795037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:21.795462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:21.795503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:21.795841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:21.795888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:21.795890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:21.795891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:21.795892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:21.795894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:21.795898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:21.795900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.795901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.795902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.795907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:21.795920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:21.795924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:21.795928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:21.795932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:21.795936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:21.795941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.795945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.795948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.795956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:21.795964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:21.795970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:21.795975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:21.795979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:21.795985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:21.795989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:21.795994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:21.796000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:21.796005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:21.796010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:21.796015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:21.796020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.796026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.796031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.796036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.796041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.796046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.796080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:21.804754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:21.806126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:21.806134Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:21.806137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:21.806143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:21.806152Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 506 [2:7559341389027816593:5375]
2025-10-09T21:45:21.806169Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:21.806194Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:21.806209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:21.806213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:21.806216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:21.806236Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:21.806242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:21.806251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:21.806322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.806324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.806326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.806329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:21.806406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:21.806408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.806423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:21.806432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:21.806448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:21.806470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341389027816587:5375]
2025-10-09T21:45:21.806477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341389027816587:5375]
2025-10-09T21:45:21.806480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341389027816587:5375]
2025-10-09T21:45:21.806490Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:21.806507Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:21.806521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:21.806524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:21.806566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341389027816587:5375]
2025-10-09T21:45:21.806569Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:21.807116Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:21.807122Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:21.807125Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:21.807132Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:21.807949Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 506
2025-10-09T21:45:21.807991Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 506
2025-10-09T21:45:21.808716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341389389629126:2432], now have 1 active actors on pipe
2025-10-09T21:45:21.808728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341389389629198:5883], now have 1 active actors on pipe
2025-10-09T21:45:21.808748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:21.808754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:21.808785Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d49a7ee-192a6285-2e023113-9cdf57c7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:21.808796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:21.808802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:21.808806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:21.808819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:21.808834Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:21.808852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:21.817063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:21.817085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:21.817130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:21.817338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 507 partition: 0 MaxSeqNo: 25 sessionId: src_id|d49a7ee-192a6285-2e023113-9cdf57c7_0
2025-10-09T21:45:21.817840Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57d4511f-e65bea79-f0698e73-ba37935d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d49a7ee-192a6285-2e023113-9cdf57c7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:21.820351Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d49a7ee-192a6285-2e023113-9cdf57c7_0] Write session: try to update token
2025-10-09T21:45:21.820374Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d49a7ee-192a6285-2e023113-9cdf57c7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:21.835060Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:21.835080Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:21.835083Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:21.835085Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:21.835098Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341389027816587:5375]
2025-10-09T21:45:21.835103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341389027816587:5375]
2025-10-09T21:45:21.835107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341389027816587:5375]
2025-10-09T21:45:21.835159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:21.835168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:21.835170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:21.835306Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:21.838246Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:21.838260Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:21.838264Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:21.854825Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 507 sessionId: src_id|d49a7ee-192a6285-2e023113-9cdf57c7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:21.857621Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:21.859307Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341389389629212:5852]
2025-10-09T21:45:21.859877Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 506, pipe: [1:7559341389389629212:5852]
2025-10-09T21:45:22.023761Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:22.024096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:22.024106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:22.024152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341389389629212:5852], now have 1 active actors on pipe
2025-10-09T21:45:22.026574Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:22.026647Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341389389629212:5852]
2025-10-09T21:45:22.026680Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:22.026683Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:22.026744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 507 reason: error at write: Disconnected sessionId: src_id|d49a7ee-192a6285-2e023113-9cdf57c7_0
2025-10-09T21:45:22.026938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 507 sessionId: src_id|d49a7ee-192a6285-2e023113-9cdf57c7_0 is DEAD
2025-10-09T21:45:22.027459Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d49a7ee-192a6285-2e023113-9cdf57c7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:22.027467Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d49a7ee-192a6285-2e023113-9cdf57c7_0] Write session will restart in 0.009606s
2025-10-09T21:45:22.027492Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d49a7ee-192a6285-2e023113-9cdf57c7_0] Write session: Do CDS request
2025-10-09T21:45:22.027503Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d49a7ee-192a6285-2e023113-9cdf57c7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:22.030740Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:22.040449Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d49a7ee-192a6285-2e023113-9cdf57c7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:22.042190Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:22.042225Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 508
2025-10-09T21:45:22.043947Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 508 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:22.044000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 508 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:22.044008Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=508 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:22.044012Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:22.045228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 508 sessionId:  describe result for acl check
2025-10-09T21:45:22.045271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:22.045273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:22.045275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:22.045284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341393684596537:5888] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:22.045830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341393684596537:5888] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:22.052654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341393684596537:5888] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:22.060871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341393684596537:5888] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:22.060889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341393684596537:5888] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:22.060893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341393684596537:5888] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:22.060902Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 508 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:22.074955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:22.076961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:22.077472Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:22.077478Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:22.077497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:22.077581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:22.077624Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341393322783982:5383]
2025-10-09T21:45:22.078275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:22.082192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:22.082297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:22.082679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:22.082773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:22.082774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:22.082775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:22.082777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:22.082778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:22.082779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:22.082780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.082782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.082784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.082791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:22.082801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:22.082806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:22.082809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:22.082813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:22.082830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:22.082834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.082837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.082842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.082851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:22.082860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:22.082867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:22.082873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:22.082880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:22.082885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:22.082889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:22.082895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:22.082899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:22.082904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:22.082911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:22.082917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:22.082922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.082927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.082931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.082936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.082940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.082947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.082975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:22.095453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:22.096580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:22.096594Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:22.096596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:22.096600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:22.096607Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 507 [2:7559341393322783982:5383]
2025-10-09T21:45:22.096621Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:22.096641Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:22.096647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:22.096651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:22.096654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:22.096674Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:22.096685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:22.096692Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:22.096751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.096758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.096759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.096762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:22.096834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:22.096837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.096850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.096860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.096876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:22.096899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341393684596593:5888], now have 1 active actors on pipe
2025-10-09T21:45:22.096907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341393322783977:5383]
2025-10-09T21:45:22.096910Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341393322783977:5383]
2025-10-09T21:45:22.096912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341393322783977:5383]
2025-10-09T21:45:22.096922Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:22.096931Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:22.096934Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:22.096938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:22.096939Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:22.096953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:22.096957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:22.096963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341393684596524:2432], now have 1 active actors on pipe
2025-10-09T21:45:22.097006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341393322783977:5383]
2025-10-09T21:45:22.097017Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:22.097293Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 507
2025-10-09T21:45:22.097324Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 507
2025-10-09T21:45:22.098576Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 508 partition: 0 MaxSeqNo: 25 sessionId: src_id|caae9f8c-5db86754-9279bc2a-997cb922_0
2025-10-09T21:45:22.099147Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d49a7ee-192a6285-2e023113-9cdf57c7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|caae9f8c-5db86754-9279bc2a-997cb922_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:22.097492Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:22.097973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:22.097980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:22.098014Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|caae9f8c-5db86754-9279bc2a-997cb922_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:22.098024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:22.098043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:22.098047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:22.098058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:22.098071Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:22.098090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:22.098377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:22.098380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:22.098403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:22.101869Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|caae9f8c-5db86754-9279bc2a-997cb922_0] Write session: try to update token
2025-10-09T21:45:22.101888Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|caae9f8c-5db86754-9279bc2a-997cb922_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:22.122716Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:22.122734Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:22.122737Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:22.122739Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:22.122748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341393322783977:5383]
2025-10-09T21:45:22.122752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341393322783977:5383]
2025-10-09T21:45:22.122756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341393322783977:5383]
2025-10-09T21:45:22.122800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:22.122808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.122810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:22.122951Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:22.126550Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:22.126561Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:22.126565Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:22.126545Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341393684596611:5852]
2025-10-09T21:45:22.130118Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 508 sessionId: src_id|caae9f8c-5db86754-9279bc2a-997cb922_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:22.131001Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 507, pipe: [1:7559341393684596611:5852]
2025-10-09T21:45:22.132840Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:22.321461Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:22.323863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341393684596611:5852], now have 1 active actors on pipe
2025-10-09T21:45:22.323907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:22.323915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:22.324502Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:22.324571Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341393684596611:5852]
2025-10-09T21:45:22.324601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:22.324604Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:22.324667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 508 reason: error at write: Disconnected sessionId: src_id|caae9f8c-5db86754-9279bc2a-997cb922_0
2025-10-09T21:45:22.324870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 508 sessionId: src_id|caae9f8c-5db86754-9279bc2a-997cb922_0 is DEAD
2025-10-09T21:45:22.325538Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|caae9f8c-5db86754-9279bc2a-997cb922_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:22.325548Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|caae9f8c-5db86754-9279bc2a-997cb922_0] Write session will restart in 0.008817s
2025-10-09T21:45:22.325572Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|caae9f8c-5db86754-9279bc2a-997cb922_0] Write session: Do CDS request
2025-10-09T21:45:22.325582Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|caae9f8c-5db86754-9279bc2a-997cb922_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:22.325155Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:22.336948Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|caae9f8c-5db86754-9279bc2a-997cb922_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:22.337016Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:22.337035Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 509
2025-10-09T21:45:22.337765Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 509 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:22.337817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 509 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:22.337823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=509 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:22.337827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:22.338580Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 509 sessionId:  describe result for acl check
2025-10-09T21:45:22.338629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:22.338631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:22.338633Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:22.338640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341393684596651:5899] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:22.339266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341393684596651:5899] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:22.344362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341393684596651:5899] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:22.351399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341393684596651:5899] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:22.351421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341393684596651:5899] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:22.351426Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341393684596651:5899] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:22.351436Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 509 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:22.358220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:22.359821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:22.360219Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:22.360224Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:22.360240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:22.360312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:22.360353Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341393322784093:5393]
2025-10-09T21:45:22.360958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:22.361561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:22.361629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:22.361999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:22.362074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:22.362080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:22.362081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:22.362083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:22.362084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:22.362086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:22.362087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.362088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.362090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.362096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:22.362108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:22.362112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:22.362116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:22.362121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:22.362125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:22.362129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.362133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.362146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.362154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:22.362163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:22.362170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:22.362176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:22.362186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:22.362192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:22.362210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:22.362216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:22.362221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:22.362226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:22.362233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:22.362248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:22.362253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.362265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.362269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.362274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.362279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.362284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.362327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:22.372465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:22.373703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:22.373729Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:22.373732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:22.373737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:22.373747Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 508 [2:7559341393322784093:5393]
2025-10-09T21:45:22.373764Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:22.373790Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:22.373794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:22.373797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:22.373799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:22.373820Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:22.373827Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:22.373843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:22.373926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.373937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.373938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.373941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:22.374508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:22.374521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.374539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.374553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.374572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:22.374598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341393322784088:5393]
2025-10-09T21:45:22.374601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341393322784088:5393]
2025-10-09T21:45:22.374603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341393322784088:5393]
2025-10-09T21:45:22.374619Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:22.374639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:22.374655Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:22.374658Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:22.374702Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341393322784088:5393]
2025-10-09T21:45:22.374705Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:22.375066Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:22.375069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:22.375071Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:22.375080Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:22.375505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341393684596711:5899], now have 1 active actors on pipe
2025-10-09T21:45:22.375509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341393684596639:2432], now have 1 active actors on pipe
2025-10-09T21:45:22.376419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:22.376428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:22.376477Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|63caca82-c4270fa0-b348b8cb-21a26b6d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:22.376489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:22.376494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:22.376497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:22.376511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:22.376529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:22.376550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:22.376038Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 508
2025-10-09T21:45:22.376074Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 508
2025-10-09T21:45:22.380175Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 509 partition: 0 MaxSeqNo: 25 sessionId: src_id|63caca82-c4270fa0-b348b8cb-21a26b6d_0
2025-10-09T21:45:22.379966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:22.379983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:22.380022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:22.381294Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|caae9f8c-5db86754-9279bc2a-997cb922_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|63caca82-c4270fa0-b348b8cb-21a26b6d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:22.384266Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63caca82-c4270fa0-b348b8cb-21a26b6d_0] Write session: try to update token
2025-10-09T21:45:22.384289Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63caca82-c4270fa0-b348b8cb-21a26b6d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:22.398916Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:22.398932Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:22.398935Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:22.398937Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:22.398948Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341393322784088:5393]
2025-10-09T21:45:22.398955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341393322784088:5393]
2025-10-09T21:45:22.398963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341393322784088:5393]
2025-10-09T21:45:22.399013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:22.399022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.399025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:22.399170Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:22.402259Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:22.402282Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:22.402295Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:22.410219Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 509 sessionId: src_id|63caca82-c4270fa0-b348b8cb-21a26b6d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:22.413290Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:22.424298Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341393684596725:5852]
2025-10-09T21:45:22.424789Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 508, pipe: [1:7559341393684596725:5852]
2025-10-09T21:45:22.599008Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:22.599505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:22.599516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:22.599565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341393684596725:5852], now have 1 active actors on pipe
2025-10-09T21:45:22.600411Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:22.600706Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63caca82-c4270fa0-b348b8cb-21a26b6d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:22.600718Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63caca82-c4270fa0-b348b8cb-21a26b6d_0] Write session will restart in 0.007334s
2025-10-09T21:45:22.600745Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63caca82-c4270fa0-b348b8cb-21a26b6d_0] Write session: Do CDS request
2025-10-09T21:45:22.600751Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63caca82-c4270fa0-b348b8cb-21a26b6d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:22.600244Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:22.600260Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:22.600282Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:22.600323Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341393684596725:5852]
2025-10-09T21:45:22.600328Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 509 reason: error at write: Disconnected sessionId: src_id|63caca82-c4270fa0-b348b8cb-21a26b6d_0
2025-10-09T21:45:22.600522Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 509 sessionId: src_id|63caca82-c4270fa0-b348b8cb-21a26b6d_0 is DEAD
2025-10-09T21:45:22.612685Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63caca82-c4270fa0-b348b8cb-21a26b6d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:22.614215Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:22.614233Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 510
2025-10-09T21:45:22.618416Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 510 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:22.618467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 510 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:22.618473Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=510 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:22.618478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:22.618998Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 510 sessionId:  describe result for acl check
2025-10-09T21:45:22.619038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:22.619040Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:22.619041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:22.619048Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341393684596771:5904] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:22.619939Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341393684596771:5904] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:22.628071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:22.628633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:22.628803Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:22.628808Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:22.628821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:22.628871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:22.628908Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341393322784175:5395]
2025-10-09T21:45:22.629494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:22.630607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:22.630679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:22.631022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:22.631079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:22.631080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:22.631081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:22.631083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:22.631084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:22.631086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:22.631087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.631089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.631090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.631096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:22.631108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:22.631112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:22.631115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:22.631119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:22.631123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:22.631127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.631131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.631135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.631143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:22.631152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:22.631157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:22.631163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:22.631167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:22.631171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:22.631175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:22.631179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:22.631184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:22.631189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:22.631193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:22.631198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:22.631202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.631207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.631210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.631215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.631220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.631226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.631257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:22.634928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341393684596771:5904] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:22.641535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:22.643002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:22.643009Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:22.643013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:22.643019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:22.643028Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 509 [2:7559341393322784175:5395]
2025-10-09T21:45:22.643044Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:22.643067Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:22.643070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:22.643073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:22.643077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:22.643095Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:22.643101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:22.643108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:22.643178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.643180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.643182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.643185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:22.643261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:22.643265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.643278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.643288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.643303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:22.643324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341393322784169:5395]
2025-10-09T21:45:22.643327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341393322784169:5395]
2025-10-09T21:45:22.643332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341393322784169:5395]
2025-10-09T21:45:22.643347Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:22.643362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:22.643376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:22.643379Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:22.643425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341393322784169:5395]
2025-10-09T21:45:22.643428Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:22.644865Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:22.644872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:22.644874Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:22.644883Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:22.646115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341393684596730:2432], now have 1 active actors on pipe
2025-10-09T21:45:22.647303Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 509
2025-10-09T21:45:22.649545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341393684596771:5904] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:22.649565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341393684596771:5904] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:22.649569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341393684596771:5904] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:22.649581Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 510 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:22.654290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341393684596815:5904], now have 1 active actors on pipe
2025-10-09T21:45:22.654411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 509
2025-10-09T21:45:22.654562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:22.654575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:22.654610Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c3f89fb9-de6101a6-3b65fb69-350791b8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:22.654627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:22.654633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:22.654636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:22.654651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:22.654675Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:22.654702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:22.654956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:22.654967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:22.654982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:22.655108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 510 partition: 0 MaxSeqNo: 25 sessionId: src_id|c3f89fb9-de6101a6-3b65fb69-350791b8_0
2025-10-09T21:45:22.658921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63caca82-c4270fa0-b348b8cb-21a26b6d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c3f89fb9-de6101a6-3b65fb69-350791b8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:22.661409Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f89fb9-de6101a6-3b65fb69-350791b8_0] Write session: try to update token
2025-10-09T21:45:22.661433Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f89fb9-de6101a6-3b65fb69-350791b8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:22.669959Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:22.669979Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:22.669982Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:22.669984Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:22.669994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341393322784169:5395]
2025-10-09T21:45:22.670000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341393322784169:5395]
2025-10-09T21:45:22.670004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341393322784169:5395]
2025-10-09T21:45:22.670101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:22.670109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.670113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:22.670266Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:22.675448Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:22.675463Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:22.675468Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:22.690232Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 510 sessionId: src_id|c3f89fb9-de6101a6-3b65fb69-350791b8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:22.692834Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:22.701115Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341393684596820:5852]
2025-10-09T21:45:22.701936Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 509, pipe: [1:7559341393684596820:5852]
2025-10-09T21:45:22.863201Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:22.863488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:22.863498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:22.863547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341393684596820:5852], now have 1 active actors on pipe
2025-10-09T21:45:22.864015Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:22.863971Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:22.863987Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:22.864062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 510 reason: error at write: Disconnected sessionId: src_id|c3f89fb9-de6101a6-3b65fb69-350791b8_0
2025-10-09T21:45:22.864257Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 510 sessionId: src_id|c3f89fb9-de6101a6-3b65fb69-350791b8_0 is DEAD
2025-10-09T21:45:22.864457Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341393684596820:5852]
2025-10-09T21:45:22.864471Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:22.866375Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f89fb9-de6101a6-3b65fb69-350791b8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:22.866386Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f89fb9-de6101a6-3b65fb69-350791b8_0] Write session will restart in 0.009237s
2025-10-09T21:45:22.866415Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f89fb9-de6101a6-3b65fb69-350791b8_0] Write session: Do CDS request
2025-10-09T21:45:22.866421Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f89fb9-de6101a6-3b65fb69-350791b8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:22.878922Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f89fb9-de6101a6-3b65fb69-350791b8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:22.884377Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:22.884395Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 511
2025-10-09T21:45:22.884594Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 511 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:22.884630Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 511 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:22.884636Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=511 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:22.884641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:22.885311Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 511 sessionId:  describe result for acl check
2025-10-09T21:45:22.885339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:22.885342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:22.885343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:22.885353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341393684596865:5908] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:22.885906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341393684596865:5908] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:22.888116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:22.889017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341393684596865:5908] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:22.891204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:22.891415Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:22.891419Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:22.891431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:22.891481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:22.891515Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341393322784258:5397]
2025-10-09T21:45:22.892083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:22.892550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:22.892613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:22.892954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:22.892993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:22.892995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:22.892996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:22.892997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:22.892999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:22.893000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:22.893001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.893003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.893004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.893010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:22.893023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:22.893028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:22.893032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:22.893037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:22.893041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:22.893045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.893050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.893054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.893061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:22.893070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:22.893076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:22.893082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:22.893087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:22.893092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:22.893097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:22.893103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:22.893108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:22.893114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:22.893118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:22.893124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:22.893131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.893136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.893141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.893147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.893152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.893158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.893189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:22.893887Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341393684596865:5908] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:22.893904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341393684596865:5908] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:22.893908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341393684596865:5908] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:22.893918Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 511 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:22.901527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:22.902942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:22.902963Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:22.902967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:22.902972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:22.902984Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 510 [2:7559341393322784258:5397]
2025-10-09T21:45:22.903004Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:22.903027Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:22.903030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:22.903034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:22.903036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:22.903057Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:22.903064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:22.903073Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:22.903152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.903155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.903157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.903160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:22.903242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:22.903250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.903265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:22.903278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:22.903302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:22.903328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341393322784253:5397]
2025-10-09T21:45:22.903330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341393322784253:5397]
2025-10-09T21:45:22.903333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341393322784253:5397]
2025-10-09T21:45:22.903343Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:22.903353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:22.903356Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:22.903361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:22.903361Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:22.903376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:22.903378Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:22.903390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341393684596823:2432], now have 1 active actors on pipe
2025-10-09T21:45:22.903419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341393684596906:5908], now have 1 active actors on pipe
2025-10-09T21:45:22.903459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341393322784253:5397]
2025-10-09T21:45:22.903474Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:22.903485Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:22.904113Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 510
2025-10-09T21:45:22.904156Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 510
2025-10-09T21:45:22.904702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:22.904717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:22.904765Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|86a66e91-b0cd9fa8-3d59fc10-d4c18cba_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:22.904778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:22.904786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:22.904789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:22.904802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:22.904833Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:22.904857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:22.905354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:22.905366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:22.905387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:22.905548Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 511 partition: 0 MaxSeqNo: 25 sessionId: src_id|86a66e91-b0cd9fa8-3d59fc10-d4c18cba_0
2025-10-09T21:45:22.906520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f89fb9-de6101a6-3b65fb69-350791b8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|86a66e91-b0cd9fa8-3d59fc10-d4c18cba_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:22.908958Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86a66e91-b0cd9fa8-3d59fc10-d4c18cba_0] Write session: try to update token
2025-10-09T21:45:22.908976Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86a66e91-b0cd9fa8-3d59fc10-d4c18cba_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:22.920562Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:22.920577Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:22.920580Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:22.920582Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:22.920593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341393322784253:5397]
2025-10-09T21:45:22.920598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341393322784253:5397]
2025-10-09T21:45:22.920602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341393322784253:5397]
2025-10-09T21:45:22.920651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:22.920658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:22.920661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:22.920672Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:22.920680Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:22.920684Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:22.920805Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:22.929884Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 511 sessionId: src_id|86a66e91-b0cd9fa8-3d59fc10-d4c18cba_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:22.932801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:22.964156Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341393684596916:5852]
2025-10-09T21:45:22.964497Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 510, pipe: [1:7559341393684596916:5852]
2025-10-09T21:45:23.227889Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:23.230344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:23.230362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:23.230387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341393684596916:5852], now have 1 active actors on pipe
2025-10-09T21:45:23.231023Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:23.233258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86a66e91-b0cd9fa8-3d59fc10-d4c18cba_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:23.233279Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86a66e91-b0cd9fa8-3d59fc10-d4c18cba_0] Write session will restart in 0.009271s
2025-10-09T21:45:23.233304Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86a66e91-b0cd9fa8-3d59fc10-d4c18cba_0] Write session: Do CDS request
2025-10-09T21:45:23.233322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86a66e91-b0cd9fa8-3d59fc10-d4c18cba_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:23.231344Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:23.231402Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:23.231405Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:23.231467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 511 reason: error at write: Disconnected sessionId: src_id|86a66e91-b0cd9fa8-3d59fc10-d4c18cba_0
2025-10-09T21:45:23.231668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 511 sessionId: src_id|86a66e91-b0cd9fa8-3d59fc10-d4c18cba_0 is DEAD
2025-10-09T21:45:23.231802Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341393684596916:5852]
2025-10-09T21:45:23.243798Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86a66e91-b0cd9fa8-3d59fc10-d4c18cba_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:23.244007Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:23.244031Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 512
2025-10-09T21:45:23.244370Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 512 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:23.244408Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 512 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:23.244411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=512 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:23.244415Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:23.245092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 512 sessionId:  describe result for acl check
2025-10-09T21:45:23.245136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:23.245141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:23.245143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:23.245150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341397979564233:5913] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:23.245745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341397979564233:5913] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:23.255139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341397979564233:5913] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:23.260431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341397979564233:5913] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:23.260457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341397979564233:5913] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:23.260463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341397979564233:5913] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:23.260474Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 512 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:23.278150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:23.279009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:23.279202Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:23.279207Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:23.279219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:23.279267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:23.279306Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341397617751672:5410]
2025-10-09T21:45:23.279955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:23.282140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:23.282225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:23.282606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:23.282651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:23.282652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:23.282654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:23.282655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:23.282657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:23.282660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:23.282662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.282663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.282665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.282672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:23.282684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:23.282689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:23.282693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:23.282698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:23.282703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:23.282708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.282713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.282717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.282725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:23.282735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:23.282742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:23.282747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:23.282753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:23.282759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:23.282763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:23.282769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:23.282775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:23.282781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:23.282787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:23.282793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:23.282799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.282805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.282809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.282815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.282820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.282826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.282861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:23.294359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:23.295917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:23.295925Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:23.295928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:23.295933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:23.295941Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 511 [2:7559341397617751672:5410]
2025-10-09T21:45:23.295958Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:23.295978Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:23.295981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:23.295985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:23.295987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:23.296006Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:23.296013Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:23.296021Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:23.296090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.296093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.296095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.296098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:23.296177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:23.296180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.296195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.296209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.296228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:23.296251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341397617751666:5410]
2025-10-09T21:45:23.296253Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341397617751666:5410]
2025-10-09T21:45:23.296256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341397617751666:5410]
2025-10-09T21:45:23.296271Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:23.296285Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:23.296298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:23.296302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:23.296358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341397617751666:5410]
2025-10-09T21:45:23.296361Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:23.299156Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 511
2025-10-09T21:45:23.299199Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 511
2025-10-09T21:45:23.298294Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:23.298305Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:23.298308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:23.298316Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:23.309085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341397979564224:2432], now have 1 active actors on pipe
2025-10-09T21:45:23.309102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341397979564286:5913], now have 1 active actors on pipe
2025-10-09T21:45:23.309128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:23.309135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:23.309177Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|390fa1d0-10ecab73-f1d25a11-714fee67_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:23.309190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:23.309196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:23.309199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:23.309212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:23.309232Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:23.309254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:23.310094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:23.310105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:23.310127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:23.310325Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 512 partition: 0 MaxSeqNo: 25 sessionId: src_id|390fa1d0-10ecab73-f1d25a11-714fee67_0
2025-10-09T21:45:23.310746Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86a66e91-b0cd9fa8-3d59fc10-d4c18cba_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|390fa1d0-10ecab73-f1d25a11-714fee67_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:23.314815Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|390fa1d0-10ecab73-f1d25a11-714fee67_0] Write session: try to update token
2025-10-09T21:45:23.314842Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|390fa1d0-10ecab73-f1d25a11-714fee67_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:23.328232Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:23.328248Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:23.328251Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:23.328254Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:23.328268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341397617751666:5410]
2025-10-09T21:45:23.328273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341397617751666:5410]
2025-10-09T21:45:23.328276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341397617751666:5410]
2025-10-09T21:45:23.328322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:23.328330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.328332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:23.328471Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:23.334101Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341397979564325:5852]
2025-10-09T21:45:23.334662Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 511, pipe: [1:7559341397979564325:5852]
2025-10-09T21:45:23.334265Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:23.334277Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:23.334281Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:23.342527Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 512 sessionId: src_id|390fa1d0-10ecab73-f1d25a11-714fee67_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:23.345061Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:23.534458Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:23.534737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341397979564325:5852], now have 1 active actors on pipe
2025-10-09T21:45:23.534770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:23.534777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:23.535062Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:23.535562Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|390fa1d0-10ecab73-f1d25a11-714fee67_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:23.535572Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|390fa1d0-10ecab73-f1d25a11-714fee67_0] Write session will restart in 0.005019s
2025-10-09T21:45:23.535606Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|390fa1d0-10ecab73-f1d25a11-714fee67_0] Write session: Do CDS request
2025-10-09T21:45:23.535612Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|390fa1d0-10ecab73-f1d25a11-714fee67_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:23.535086Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341397979564325:5852]
2025-10-09T21:45:23.535117Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:23.535157Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:23.535160Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:23.535228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 512 reason: error at write: Disconnected sessionId: src_id|390fa1d0-10ecab73-f1d25a11-714fee67_0
2025-10-09T21:45:23.535571Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 512 sessionId: src_id|390fa1d0-10ecab73-f1d25a11-714fee67_0 is DEAD
2025-10-09T21:45:23.541542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|390fa1d0-10ecab73-f1d25a11-714fee67_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:23.541660Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:23.541677Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 513
2025-10-09T21:45:23.543148Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 513 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:23.543203Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 513 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:23.543209Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=513 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:23.543212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:23.545616Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 513 sessionId:  describe result for acl check
2025-10-09T21:45:23.545665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:23.545667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:23.545669Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:23.545676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341397979564349:5924] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:23.546323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341397979564349:5924] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:23.552283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341397979564349:5924] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:23.563107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341397979564349:5924] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:23.563126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341397979564349:5924] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:23.563130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341397979564349:5924] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:23.563141Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 513 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:23.576272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:23.581425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:23.581690Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:23.581695Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:23.581709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:23.581778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:23.581822Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341397617751768:5415]
2025-10-09T21:45:23.582455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:23.586191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:23.586293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:23.586676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:23.586730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:23.586731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:23.586732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:23.586734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:23.586735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:23.586736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:23.586737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.586739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.586740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.586746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:23.586758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:23.586763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:23.586767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:23.586772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:23.586775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:23.586779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.586783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.586787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.586795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:23.586804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:23.586812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:23.586818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:23.586823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:23.586828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:23.586833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:23.586838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:23.586842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:23.586848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:23.586853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:23.586861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:23.586866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.586871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.586877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.586882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.586888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.586893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.586923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:23.599644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:23.600743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:23.600762Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:23.600764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:23.600769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:23.600777Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 512 [2:7559341397617751768:5415]
2025-10-09T21:45:23.600792Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:23.600815Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:23.600818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:23.600821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:23.600823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:23.600843Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:23.600849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:23.600857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:23.600930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.600932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.600934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.600937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:23.601013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:23.601016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.601346Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 512
2025-10-09T21:45:23.601411Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 512
2025-10-09T21:45:23.601028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.601045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.601058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:23.601078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341397617751762:5415]
2025-10-09T21:45:23.601080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341397617751762:5415]
2025-10-09T21:45:23.601082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341397617751762:5415]
2025-10-09T21:45:23.601090Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:23.601104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:23.601117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:23.601120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:23.601160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341397617751762:5415]
2025-10-09T21:45:23.601162Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:23.601176Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:23.601180Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:23.601182Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:23.601188Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:23.601205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341397979564415:5924], now have 1 active actors on pipe
2025-10-09T21:45:23.601208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341397979564341:2432], now have 1 active actors on pipe
2025-10-09T21:45:23.601610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:23.601619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:23.601651Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|81041e85-54f57370-cd4bcf7-b42f64fa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:23.601661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:23.601666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:23.601669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:23.601680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:23.601693Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:23.601710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:23.602151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:23.602157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:23.602172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:23.602387Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 513 partition: 0 MaxSeqNo: 25 sessionId: src_id|81041e85-54f57370-cd4bcf7-b42f64fa_0
2025-10-09T21:45:23.603005Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|390fa1d0-10ecab73-f1d25a11-714fee67_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|81041e85-54f57370-cd4bcf7-b42f64fa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:23.605766Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|81041e85-54f57370-cd4bcf7-b42f64fa_0] Write session: try to update token
2025-10-09T21:45:23.605786Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|81041e85-54f57370-cd4bcf7-b42f64fa_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:23.620153Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:23.620173Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:23.620176Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:23.620178Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:23.620190Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341397617751762:5415]
2025-10-09T21:45:23.620196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341397617751762:5415]
2025-10-09T21:45:23.620201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341397617751762:5415]
2025-10-09T21:45:23.620257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:23.620645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.620653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:23.620845Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:23.620851Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:23.620855Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:23.620905Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:23.624934Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 513 sessionId: src_id|81041e85-54f57370-cd4bcf7-b42f64fa_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:23.627576Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:23.635802Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341397979564433:5852]
2025-10-09T21:45:23.636346Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 512, pipe: [1:7559341397979564433:5852]
2025-10-09T21:45:23.790868Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:23.791552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:23.791565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:23.791592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341397979564433:5852], now have 1 active actors on pipe
2025-10-09T21:45:23.792043Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:23.792059Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:23.792131Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 513 reason: error at write: Disconnected sessionId: src_id|81041e85-54f57370-cd4bcf7-b42f64fa_0
2025-10-09T21:45:23.792322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 513 sessionId: src_id|81041e85-54f57370-cd4bcf7-b42f64fa_0 is DEAD
2025-10-09T21:45:23.792475Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:23.792515Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341397979564433:5852]
2025-10-09T21:45:23.792466Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:23.793418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|81041e85-54f57370-cd4bcf7-b42f64fa_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:23.793430Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|81041e85-54f57370-cd4bcf7-b42f64fa_0] Write session will restart in 0.008150s
2025-10-09T21:45:23.793457Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|81041e85-54f57370-cd4bcf7-b42f64fa_0] Write session: Do CDS request
2025-10-09T21:45:23.793463Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|81041e85-54f57370-cd4bcf7-b42f64fa_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:23.802680Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|81041e85-54f57370-cd4bcf7-b42f64fa_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:23.802823Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:23.802850Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 514
2025-10-09T21:45:23.803116Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 514 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:23.803150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 514 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:23.803157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=514 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:23.803160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:23.804994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 514 sessionId:  describe result for acl check
2025-10-09T21:45:23.805038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:23.805041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:23.805043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:23.805050Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341397979564451:5929] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:23.805605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341397979564451:5929] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:23.824118Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341397979564451:5929] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:23.825628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:23.829984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:23.830289Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:23.830300Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:23.830316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:23.830424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341397979564451:5929] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:23.830445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341397979564451:5929] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:23.830450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341397979564451:5929] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:23.830460Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 514 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:23.830382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:23.830423Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341397617751852:5417]
2025-10-09T21:45:23.831008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:23.831527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:23.831590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:23.831990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:23.832053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:23.832059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:23.832060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:23.832061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:23.832063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:23.832067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:23.832069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.832070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.832072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.832077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:23.832090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:23.832095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:23.832099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:23.832111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:23.832115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:23.832118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.832122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.832126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.832135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:23.832143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:23.832149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:23.832163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:23.832169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:23.832174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:23.832178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:23.832183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:23.832187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:23.832193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:23.832197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:23.832202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:23.832207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.832211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.832215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.832220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.832224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.832229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.832259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:23.840167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:23.841424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:23.841441Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:23.841445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:23.841449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:23.841460Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 513 [2:7559341397617751852:5417]
2025-10-09T21:45:23.841474Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:23.841496Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:23.841499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:23.841503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:23.841506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:23.841525Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:23.841531Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:23.841539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:23.841605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.841610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.841612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.841615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:23.841685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:23.841687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.841700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:23.841709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:23.841722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:23.841745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341397617751847:5417]
2025-10-09T21:45:23.841753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341397617751847:5417]
2025-10-09T21:45:23.841754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341397617751847:5417]
2025-10-09T21:45:23.841763Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:23.841781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:23.841781Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:23.841784Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:23.841786Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:23.841795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:23.841798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:23.841847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341397979564517:5929], now have 1 active actors on pipe
2025-10-09T21:45:23.841857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341397979564436:2432], now have 1 active actors on pipe
2025-10-09T21:45:23.841859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341397617751847:5417]
2025-10-09T21:45:23.841868Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:23.841894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:23.842701Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 513
2025-10-09T21:45:23.842739Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 513
2025-10-09T21:45:23.843828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:23.843844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:23.843883Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f52f1da0-a4535c0e-648b06b4-ee4e6bf2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:23.843905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:23.843910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:23.843913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:23.843925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:23.843941Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:23.843971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:23.844487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:23.844503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:23.844525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:23.844718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 514 partition: 0 MaxSeqNo: 25 sessionId: src_id|f52f1da0-a4535c0e-648b06b4-ee4e6bf2_0
2025-10-09T21:45:23.845166Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|81041e85-54f57370-cd4bcf7-b42f64fa_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f52f1da0-a4535c0e-648b06b4-ee4e6bf2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:23.847775Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f52f1da0-a4535c0e-648b06b4-ee4e6bf2_0] Write session: try to update token
2025-10-09T21:45:23.847794Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f52f1da0-a4535c0e-648b06b4-ee4e6bf2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:23.859788Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:23.859803Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:23.859805Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:23.859807Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:23.859817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341397617751847:5417]
2025-10-09T21:45:23.859821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341397617751847:5417]
2025-10-09T21:45:23.859825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341397617751847:5417]
2025-10-09T21:45:23.859869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:23.859876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:23.859881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:23.860015Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:23.861643Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:23.861653Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:23.861658Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:23.894251Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 514 sessionId: src_id|f52f1da0-a4535c0e-648b06b4-ee4e6bf2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:23.894363Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341397979564530:5852]
2025-10-09T21:45:23.895112Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 513, pipe: [1:7559341397979564530:5852]
2025-10-09T21:45:23.896741Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:24.028188Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:24.028504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341397979564530:5852], now have 1 active actors on pipe
2025-10-09T21:45:24.028534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:24.028542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:24.028827Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:24.028944Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341397979564530:5852]
2025-10-09T21:45:24.028983Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:24.028985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:24.029055Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 514 reason: error at write: Disconnected sessionId: src_id|f52f1da0-a4535c0e-648b06b4-ee4e6bf2_0
2025-10-09T21:45:24.029242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 514 sessionId: src_id|f52f1da0-a4535c0e-648b06b4-ee4e6bf2_0 is DEAD
2025-10-09T21:45:24.029375Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:24.031656Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f52f1da0-a4535c0e-648b06b4-ee4e6bf2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:24.031675Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f52f1da0-a4535c0e-648b06b4-ee4e6bf2_0] Write session will restart in 0.006785s
2025-10-09T21:45:24.031707Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f52f1da0-a4535c0e-648b06b4-ee4e6bf2_0] Write session: Do CDS request
2025-10-09T21:45:24.031729Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f52f1da0-a4535c0e-648b06b4-ee4e6bf2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:24.042152Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f52f1da0-a4535c0e-648b06b4-ee4e6bf2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:24.046223Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:24.046243Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 515
2025-10-09T21:45:24.046465Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 515 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:24.046504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 515 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:24.046511Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=515 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:24.046516Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:24.047016Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 515 sessionId:  describe result for acl check
2025-10-09T21:45:24.047058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:24.047061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:24.047063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:24.047070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341402274531885:5934] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:24.047680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341402274531885:5934] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:24.056228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:24.061991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:24.062293Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:24.062305Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:24.062319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:24.062387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:24.062434Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341401912719236:5419]
2025-10-09T21:45:24.062613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341402274531885:5934] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:24.063051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:24.064026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:24.064090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:24.064455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:24.064505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:24.064515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:24.064516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:24.064518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:24.064519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:24.064520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:24.064522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.064523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.064524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.064530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:24.064541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:24.064545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:24.064549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:24.064553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:24.064558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:24.064561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.064565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.064569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.064577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:24.064586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:24.064593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:24.064599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:24.064605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:24.064610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:24.064615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:24.064620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:24.064625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:24.064630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:24.064634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:24.064639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:24.064645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.064650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.064656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.064661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.064666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.064674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.064704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:24.071729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341402274531885:5934] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:24.071748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341402274531885:5934] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:24.071752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341402274531885:5934] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:24.071762Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 515 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:24.079174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:24.080711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:24.080727Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:24.080730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:24.080735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:24.080743Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 514 [2:7559341401912719236:5419]
2025-10-09T21:45:24.080761Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:24.080788Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:24.080798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:24.080802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:24.080804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:24.080826Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:24.080832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:24.080841Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:24.080918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.080928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.080930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.080933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:24.081012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:24.081021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.081035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.081045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.081060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:24.081081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341401912719230:5419]
2025-10-09T21:45:24.081089Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341401912719230:5419]
2025-10-09T21:45:24.081091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341401912719230:5419]
2025-10-09T21:45:24.081102Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:24.081118Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:24.081119Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:24.081122Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:24.081133Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:24.081134Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:24.081137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:24.081149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341402274531922:5934], now have 1 active actors on pipe
2025-10-09T21:45:24.081193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341402274531839:2432], now have 1 active actors on pipe
2025-10-09T21:45:24.081196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341401912719230:5419]
2025-10-09T21:45:24.081203Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:24.081379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:24.081483Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 514
2025-10-09T21:45:24.081518Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 514
2025-10-09T21:45:24.081710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:24.081726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:24.081777Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7015f8af-ec4c52c1-f6bf52bd-f1a04418_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:24.081792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:24.081797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:24.081800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:24.081814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:24.081838Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:24.081865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:24.082207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:24.082227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:24.082248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:24.082665Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 515 partition: 0 MaxSeqNo: 25 sessionId: src_id|7015f8af-ec4c52c1-f6bf52bd-f1a04418_0
2025-10-09T21:45:24.083102Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f52f1da0-a4535c0e-648b06b4-ee4e6bf2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7015f8af-ec4c52c1-f6bf52bd-f1a04418_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:24.085953Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7015f8af-ec4c52c1-f6bf52bd-f1a04418_0] Write session: try to update token
2025-10-09T21:45:24.085974Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7015f8af-ec4c52c1-f6bf52bd-f1a04418_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:24.104177Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:24.104195Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:24.104197Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:24.104199Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:24.104215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341401912719230:5419]
2025-10-09T21:45:24.104220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341401912719230:5419]
2025-10-09T21:45:24.104223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341401912719230:5419]
2025-10-09T21:45:24.104270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:24.104284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.104293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:24.104446Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:24.105235Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:24.105248Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:24.105252Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:24.115446Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 515 sessionId: src_id|7015f8af-ec4c52c1-f6bf52bd-f1a04418_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:24.118286Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:24.129301Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341402274531931:5852]
2025-10-09T21:45:24.129845Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 514, pipe: [1:7559341402274531931:5852]
2025-10-09T21:45:24.284843Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:24.285132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:24.285147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:24.285172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341402274531931:5852], now have 1 active actors on pipe
2025-10-09T21:45:24.285394Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:24.285454Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:24.285481Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:24.285486Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:24.285524Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341402274531931:5852]
2025-10-09T21:45:24.285571Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 515 reason: error at write: Disconnected sessionId: src_id|7015f8af-ec4c52c1-f6bf52bd-f1a04418_0
2025-10-09T21:45:24.285770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 515 sessionId: src_id|7015f8af-ec4c52c1-f6bf52bd-f1a04418_0 is DEAD
2025-10-09T21:45:24.285953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7015f8af-ec4c52c1-f6bf52bd-f1a04418_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:24.285967Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7015f8af-ec4c52c1-f6bf52bd-f1a04418_0] Write session will restart in 0.009779s
2025-10-09T21:45:24.286011Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7015f8af-ec4c52c1-f6bf52bd-f1a04418_0] Write session: Do CDS request
2025-10-09T21:45:24.286019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7015f8af-ec4c52c1-f6bf52bd-f1a04418_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:24.298491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7015f8af-ec4c52c1-f6bf52bd-f1a04418_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:24.298854Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:24.298871Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 516
2025-10-09T21:45:24.299046Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 516 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:24.299078Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 516 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:24.299083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=516 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:24.299087Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:24.299573Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 516 sessionId:  describe result for acl check
2025-10-09T21:45:24.299616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:24.299618Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:24.299619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:24.299625Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341402274531949:5938] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:24.300215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341402274531949:5938] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:24.303523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341402274531949:5938] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:24.315361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341402274531949:5938] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:24.315382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341402274531949:5938] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:24.315387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341402274531949:5938] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:24.315397Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 516 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:24.315681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:24.321905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:24.322202Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:24.322213Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:24.322227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:24.322297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:24.322346Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341401912719359:5432]
2025-10-09T21:45:24.322946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:24.323527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:24.323585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:24.323910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:24.323968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:24.323970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:24.323971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:24.323972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:24.323973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:24.323974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:24.323975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.323977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.323978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.323984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:24.323994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:24.323998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:24.324001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:24.324005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:24.324009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:24.324012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.324015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.324019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.324026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:24.324035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:24.324041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:24.324047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:24.324052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:24.324057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:24.324064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:24.324069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:24.324073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:24.324077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:24.324081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:24.324085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:24.324090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.324095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.324099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.324104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.324108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.324111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.324156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:24.337915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:24.340426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:24.340437Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:24.340440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:24.340445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:24.340454Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 515 [2:7559341401912719359:5432]
2025-10-09T21:45:24.340470Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:24.340492Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:24.340497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:24.340500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:24.340503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:24.340526Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:24.340532Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:24.340539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:24.340611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.340613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.340614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.340617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:24.340706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:24.340709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.340724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.340738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.340755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:24.340776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341401912719353:5432]
2025-10-09T21:45:24.340779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341401912719353:5432]
2025-10-09T21:45:24.340781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341401912719353:5432]
2025-10-09T21:45:24.340790Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:24.340807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:24.340821Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:24.340824Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:24.340840Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:24.340844Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:24.340847Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:24.340870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341401912719353:5432]
2025-10-09T21:45:24.340874Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:24.340945Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:24.342471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341402274531936:2432], now have 1 active actors on pipe
2025-10-09T21:45:24.342488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341402274532015:5938], now have 1 active actors on pipe
2025-10-09T21:45:24.342526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:24.342534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:24.342581Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|20602584-8effbfc5-66675f93-5c3ab273_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:24.342593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:24.342598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:24.342602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:24.342621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:24.342647Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:24.342667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:24.341260Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 515
2025-10-09T21:45:24.341308Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 515
2025-10-09T21:45:24.346867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:24.346887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:24.346926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:24.347254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 516 partition: 0 MaxSeqNo: 25 sessionId: src_id|20602584-8effbfc5-66675f93-5c3ab273_0
2025-10-09T21:45:24.347654Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7015f8af-ec4c52c1-f6bf52bd-f1a04418_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|20602584-8effbfc5-66675f93-5c3ab273_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:24.350142Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|20602584-8effbfc5-66675f93-5c3ab273_0] Write session: try to update token
2025-10-09T21:45:24.350163Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|20602584-8effbfc5-66675f93-5c3ab273_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:24.372033Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:24.372053Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:24.372063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341401912719353:5432]
2025-10-09T21:45:24.372119Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:24.372226Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:24.372346Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:24.372510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:24.372652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:24.372784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:24.372957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:24.372995Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:24.389289Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341402274532029:5852]
2025-10-09T21:45:24.390229Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 516 sessionId: src_id|20602584-8effbfc5-66675f93-5c3ab273_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:24.392955Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:24.395394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:24.395413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:24.395438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341402274532029:5852], now have 1 active actors on pipe
2025-10-09T21:45:24.395480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:24.396166Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 515, pipe: [1:7559341402274532029:5852]
2025-10-09T21:45:24.396196Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:24.404619Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:24.404639Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:24.404641Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:24.404643Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:24.404654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341401912719353:5432]
2025-10-09T21:45:24.404661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341401912719353:5432]
2025-10-09T21:45:24.404664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341401912719353:5432]
2025-10-09T21:45:24.404713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:24.404718Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:24.404720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.404723Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:24.404723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:24.404730Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:24.404861Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:24.596195Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:24.605338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:24.605366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:24.605543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:24.605600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:24.605659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:24.605718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:24.605755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:24.605760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:24.605826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:24.605889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:24.605947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:24.606015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:24.606841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:24.606851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:24.606920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:24.606981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:24.607041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:24.607122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:24.607160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:24.607163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:24.607234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:24.607292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:24.607351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:24.607416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:24.607459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:24.607462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:24.607532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:24.607590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:24.607651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:24.607710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:24.607752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:24.607755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:24.607824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:24.607878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:24.607932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:24.607991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:24.608028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:24.608032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:24.608096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:24.608149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:24.608207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:24.608264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:24.608302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:24.608305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:24.608375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:24.608427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:24.608486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:24.608544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:24.608587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:24.608590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:24.608653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:24.608708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:24.608766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:24.608825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:24.608864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:24.608867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:24.608931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:24.608986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:24.609044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:24.609103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:24.609142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:24.609145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:24.610679Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:24.610694Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:24.610726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 516 reason: error at write: Disconnected sessionId: src_id|20602584-8effbfc5-66675f93-5c3ab273_0
2025-10-09T21:45:24.611045Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20602584-8effbfc5-66675f93-5c3ab273_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:24.611055Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20602584-8effbfc5-66675f93-5c3ab273_0] Write session will restart in 0.009381s
2025-10-09T21:45:24.611079Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20602584-8effbfc5-66675f93-5c3ab273_0] Write session: Do CDS request
2025-10-09T21:45:24.611085Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20602584-8effbfc5-66675f93-5c3ab273_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:24.610910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 516 sessionId: src_id|20602584-8effbfc5-66675f93-5c3ab273_0 is DEAD
2025-10-09T21:45:24.611073Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341402274532029:5852]
2025-10-09T21:45:24.611087Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:24.610935Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:24.621855Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|20602584-8effbfc5-66675f93-5c3ab273_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:24.623631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:24.623649Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 517
2025-10-09T21:45:24.623850Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 517 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:24.623886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 517 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:24.623892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=517 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:24.623895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:24.624692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 517 sessionId:  describe result for acl check
2025-10-09T21:45:24.624731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:24.624733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:24.624735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:24.624742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341402274532106:5950] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:24.625287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341402274532106:5950] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:24.637501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:24.638884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341402274532106:5950] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:24.640127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:24.640363Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:24.640368Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:24.640381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:24.640451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:24.640491Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341401912719456:5437]
2025-10-09T21:45:24.641107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:24.641659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:24.641703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:24.641991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:24.642078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:24.642079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:24.642080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:24.642081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:24.642082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:24.642084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:24.642085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.642086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.642087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.642091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:24.642101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:24.642105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:24.642108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:24.642111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:24.642114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:24.642117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.642120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.642123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.642131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:24.642138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:24.642144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:24.642148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:24.642156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:24.642160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:24.642165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:24.642169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:24.642174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:24.642178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:24.642181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:24.642185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:24.642200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.642204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.642208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.642212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.642217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.642221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.642248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:24.650533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341402274532106:5950] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:24.650552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341402274532106:5950] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:24.650555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341402274532106:5950] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:24.650562Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 517 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:24.651120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:24.652428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:24.652434Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:24.652437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:24.652441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:24.652449Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 516 [2:7559341401912719456:5437]
2025-10-09T21:45:24.652465Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:24.652482Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:24.652486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:24.652490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:24.652492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:24.652510Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:24.652516Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:24.652523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:24.652587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.652591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.652594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.652597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:24.652672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:24.652675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.652689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.652702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.652721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:24.652741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341401912719451:5437]
2025-10-09T21:45:24.652744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341401912719451:5437]
2025-10-09T21:45:24.652747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341401912719451:5437]
2025-10-09T21:45:24.652756Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:24.652773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:24.652790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:24.652793Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:24.652799Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:24.652802Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:24.652804Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:24.652830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341401912719451:5437]
2025-10-09T21:45:24.652833Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:24.652854Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:24.652870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341402274532059:2432], now have 1 active actors on pipe
2025-10-09T21:45:24.652875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341402274532143:5950], now have 1 active actors on pipe
2025-10-09T21:45:24.653187Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 516
2025-10-09T21:45:24.653223Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 516
2025-10-09T21:45:24.653961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:24.653977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:24.654020Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5b058b67-66f470c3-2323ef2b-367353cf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:24.654043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:24.654048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:24.654051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:24.654062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:24.654076Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:24.654107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:24.655061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:24.655068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:24.655090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:24.655329Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 517 partition: 0 MaxSeqNo: 25 sessionId: src_id|5b058b67-66f470c3-2323ef2b-367353cf_0
2025-10-09T21:45:24.659178Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20602584-8effbfc5-66675f93-5c3ab273_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5b058b67-66f470c3-2323ef2b-367353cf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:24.662517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b058b67-66f470c3-2323ef2b-367353cf_0] Write session: try to update token
2025-10-09T21:45:24.662542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b058b67-66f470c3-2323ef2b-367353cf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:24.672009Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:24.672031Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:24.672034Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:24.672036Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:24.672049Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341401912719451:5437]
2025-10-09T21:45:24.672053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341401912719451:5437]
2025-10-09T21:45:24.672057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341401912719451:5437]
2025-10-09T21:45:24.672106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:24.672113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.672115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:24.672116Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:24.672122Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:24.672129Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:24.672251Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:24.691760Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 517 sessionId: src_id|5b058b67-66f470c3-2323ef2b-367353cf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:24.694332Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:24.710408Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341402274532154:5852]
2025-10-09T21:45:24.710896Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 516, pipe: [1:7559341402274532154:5852]
2025-10-09T21:45:24.896872Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:24.897440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:24.897458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:24.897481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341402274532154:5852], now have 1 active actors on pipe
2025-10-09T21:45:24.897855Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:24.898307Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:24.898355Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:24.898444Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 517 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5b058b67-66f470c3-2323ef2b-367353cf_0
2025-10-09T21:45:24.898630Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 517 sessionId: src_id|5b058b67-66f470c3-2323ef2b-367353cf_0 is DEAD
2025-10-09T21:45:24.898774Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341402274532154:5852]
2025-10-09T21:45:24.898790Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b058b67-66f470c3-2323ef2b-367353cf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:24.898901Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b058b67-66f470c3-2323ef2b-367353cf_0] Write session will restart in 0.008234s
2025-10-09T21:45:24.898931Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b058b67-66f470c3-2323ef2b-367353cf_0] Write session: Do CDS request
2025-10-09T21:45:24.898938Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b058b67-66f470c3-2323ef2b-367353cf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:24.910822Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b058b67-66f470c3-2323ef2b-367353cf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:24.911242Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:24.911261Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 518
2025-10-09T21:45:24.912354Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 518 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:24.912409Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 518 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:24.912415Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=518 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:24.912419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:24.916385Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 518 sessionId:  describe result for acl check
2025-10-09T21:45:24.916441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:24.916443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:24.916445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:24.916452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341402274532177:5954] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:24.917028Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341402274532177:5954] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:24.928503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341402274532177:5954] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:24.934536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341402274532177:5954] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:24.934558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341402274532177:5954] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:24.934563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341402274532177:5954] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:24.934573Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 518 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:24.947407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:24.947639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:24.947824Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:24.947828Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:24.947842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:24.947900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:24.947943Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341401912719549:5439]
2025-10-09T21:45:24.948567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:24.952357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:24.952443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:24.952881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:24.952959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:24.952960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:24.952961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:24.952962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:24.952963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:24.952965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:24.952966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.952967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.952968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.952974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:24.952985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:24.952989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:24.952992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:24.952996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:24.953001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:24.953004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.953008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.953011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.953021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:24.953030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:24.953037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:24.953043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:24.953052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:24.953056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:24.953060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:24.953064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:24.953069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:24.953074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:24.953078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:24.953083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:24.953088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.953096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.953100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.953104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.953110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.953114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.953147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:24.964107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:24.965634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:24.965643Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:24.965646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:24.965650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:24.965659Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 517 [2:7559341401912719549:5439]
2025-10-09T21:45:24.965674Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:24.965696Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:24.965699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:24.965703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:24.965705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:24.965728Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:24.965734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:24.965743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:24.965814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.965816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.965818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.965825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:24.965908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:24.965911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.965926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:24.965935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:24.965949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:24.965973Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341401912719544:5439]
2025-10-09T21:45:24.965976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341401912719544:5439]
2025-10-09T21:45:24.965977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341401912719544:5439]
2025-10-09T21:45:24.965993Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:24.966012Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:24.966025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:24.966044Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:24.966092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341401912719544:5439]
2025-10-09T21:45:24.966094Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:24.966851Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:24.966864Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:24.966866Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:24.966873Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:24.967221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341402274532163:2432], now have 1 active actors on pipe
2025-10-09T21:45:24.967228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341402274532247:5954], now have 1 active actors on pipe
2025-10-09T21:45:24.968138Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 517
2025-10-09T21:45:24.968187Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 517
2025-10-09T21:45:24.970349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:24.970365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:24.970404Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|542135ee-2530f2cc-ddfe372a-a1a16d70_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:24.970427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:24.970434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:24.970438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:24.970452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:24.970471Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:24.970500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:24.971191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:24.971200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:24.971221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:24.971411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 518 partition: 0 MaxSeqNo: 25 sessionId: src_id|542135ee-2530f2cc-ddfe372a-a1a16d70_0
2025-10-09T21:45:24.971861Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b058b67-66f470c3-2323ef2b-367353cf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|542135ee-2530f2cc-ddfe372a-a1a16d70_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:24.974450Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|542135ee-2530f2cc-ddfe372a-a1a16d70_0] Write session: try to update token
2025-10-09T21:45:24.974472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|542135ee-2530f2cc-ddfe372a-a1a16d70_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:24.995548Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:24.995565Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:24.995569Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:24.995571Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:24.995588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341401912719544:5439]
2025-10-09T21:45:24.995595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341401912719544:5439]
2025-10-09T21:45:24.995600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341401912719544:5439]
2025-10-09T21:45:24.995654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:24.995664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:24.995667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:24.995929Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:24.998309Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:24.998325Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:24.998332Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:24.998679Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341402274532265:5852]
2025-10-09T21:45:24.999368Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 517, pipe: [1:7559341402274532265:5852]
2025-10-09T21:45:25.000408Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 518 sessionId: src_id|542135ee-2530f2cc-ddfe372a-a1a16d70_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:25.003148Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:25.191676Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:25.192230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341402274532265:5852], now have 1 active actors on pipe
2025-10-09T21:45:25.192262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:25.192269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:25.193257Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341402274532265:5852]
2025-10-09T21:45:25.193289Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:25.193320Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:25.193322Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:25.192975Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:25.193379Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 518 reason: error at write: Disconnected sessionId: src_id|542135ee-2530f2cc-ddfe372a-a1a16d70_0
2025-10-09T21:45:25.193575Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 518 sessionId: src_id|542135ee-2530f2cc-ddfe372a-a1a16d70_0 is DEAD
2025-10-09T21:45:25.194616Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|542135ee-2530f2cc-ddfe372a-a1a16d70_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:25.194627Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|542135ee-2530f2cc-ddfe372a-a1a16d70_0] Write session will restart in 0.005214s
2025-10-09T21:45:25.194655Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|542135ee-2530f2cc-ddfe372a-a1a16d70_0] Write session: Do CDS request
2025-10-09T21:45:25.194661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|542135ee-2530f2cc-ddfe372a-a1a16d70_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:25.201629Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|542135ee-2530f2cc-ddfe372a-a1a16d70_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:25.202315Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:25.202343Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 519
2025-10-09T21:45:25.202503Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 519 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:25.202539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 519 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:25.202551Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=519 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:25.202556Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:25.203312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 519 sessionId:  describe result for acl check
2025-10-09T21:45:25.203359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:25.203366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:25.203368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:25.203375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341406569499576:5959] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:25.203949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341406569499576:5959] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:25.209622Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341406569499576:5959] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:25.214971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341406569499576:5959] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:25.214992Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341406569499576:5959] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:25.214996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341406569499576:5959] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:25.215008Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 519 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:25.217189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:25.223438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:25.223716Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:25.223738Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:25.223752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:25.223812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:25.223879Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341406207686939:5442]
2025-10-09T21:45:25.224371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:25.224914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:25.224968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:25.225321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:25.225378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:25.225384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:25.225386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:25.225388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:25.225389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:25.225391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:25.225393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.225394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.225395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.225401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:25.225412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:25.225416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:25.225420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:25.225424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:25.225428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:25.225432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.225436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.225440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.225448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:25.225457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:25.225465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:25.225472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:25.225477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:25.225485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:25.225490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:25.225495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:25.225500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:25.225506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:25.225510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:25.225515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:25.225521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.225526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.225530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.225535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.225541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.225545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.225574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:25.233138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:25.234276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:25.234292Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:25.234295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:25.234300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:25.234309Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 518 [2:7559341406207686939:5442]
2025-10-09T21:45:25.234325Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:25.234348Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:25.234352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:25.234355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:25.234358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:25.234379Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:25.234386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:25.234394Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:25.234466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.234468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.234470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.234473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:25.234563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:25.234566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.234579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.234589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.234607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:25.234630Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341406207686934:5442]
2025-10-09T21:45:25.234632Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341406207686934:5442]
2025-10-09T21:45:25.234634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341406207686934:5442]
2025-10-09T21:45:25.234643Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:25.234654Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:25.234662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341406569499651:5959], now have 1 active actors on pipe
2025-10-09T21:45:25.234663Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:25.234669Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:25.234674Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:25.234682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:25.234685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:25.234691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341406569499568:2432], now have 1 active actors on pipe
2025-10-09T21:45:25.234735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341406207686934:5442]
2025-10-09T21:45:25.234747Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:25.234787Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:25.235012Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 518
2025-10-09T21:45:25.235050Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 518
2025-10-09T21:45:25.235366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:25.235378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:25.235417Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|608abe13-4790c478-4f775182-accdc6a4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:25.235428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:25.235433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:25.235436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:25.235451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:25.235465Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:25.235485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:25.235792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:25.235802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:25.235822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:25.235920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 519 partition: 0 MaxSeqNo: 25 sessionId: src_id|608abe13-4790c478-4f775182-accdc6a4_0
2025-10-09T21:45:25.236484Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|542135ee-2530f2cc-ddfe372a-a1a16d70_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|608abe13-4790c478-4f775182-accdc6a4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:25.239311Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|608abe13-4790c478-4f775182-accdc6a4_0] Write session: try to update token
2025-10-09T21:45:25.239331Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|608abe13-4790c478-4f775182-accdc6a4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:25.254609Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:25.254626Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:25.254629Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:25.254631Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:25.254640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341406207686934:5442]
2025-10-09T21:45:25.254645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341406207686934:5442]
2025-10-09T21:45:25.254649Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341406207686934:5442]
2025-10-09T21:45:25.254700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:25.254703Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:25.254707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.254709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:25.254710Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:25.254714Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:25.254852Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:25.269620Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 519 sessionId: src_id|608abe13-4790c478-4f775182-accdc6a4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:25.272381Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:25.293420Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341406569499666:5852]
2025-10-09T21:45:25.295097Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 518, pipe: [1:7559341406569499666:5852]
2025-10-09T21:45:25.437538Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:25.437902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:25.437913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:25.437936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341406569499666:5852], now have 1 active actors on pipe
2025-10-09T21:45:25.438463Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:25.439229Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341406569499666:5852]
2025-10-09T21:45:25.439264Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:25.439861Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|608abe13-4790c478-4f775182-accdc6a4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:25.439456Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:25.439874Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|608abe13-4790c478-4f775182-accdc6a4_0] Write session will restart in 0.008266s
2025-10-09T21:45:25.439524Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 519 reason: status is not ok: tablet will be restarted right now sessionId: src_id|608abe13-4790c478-4f775182-accdc6a4_0
2025-10-09T21:45:25.439711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 519 sessionId: src_id|608abe13-4790c478-4f775182-accdc6a4_0 is DEAD
2025-10-09T21:45:25.439898Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|608abe13-4790c478-4f775182-accdc6a4_0] Write session: Do CDS request
2025-10-09T21:45:25.439918Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|608abe13-4790c478-4f775182-accdc6a4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:25.449573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|608abe13-4790c478-4f775182-accdc6a4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:25.449716Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:25.449746Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 520
2025-10-09T21:45:25.450018Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 520 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:25.450069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 520 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:25.450075Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=520 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:25.450079Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:25.450643Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 520 sessionId:  describe result for acl check
2025-10-09T21:45:25.450676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:25.450678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:25.450679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:25.450685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341406569499683:5963] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:25.451304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341406569499683:5963] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:25.457176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341406569499683:5963] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:25.463172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341406569499683:5963] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:25.463192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341406569499683:5963] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:25.463197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341406569499683:5963] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:25.463208Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 520 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:25.469213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:25.470469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:25.470758Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:25.470769Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:25.470783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:25.470841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:25.470884Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341406207687062:5455]
2025-10-09T21:45:25.471546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:25.472096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:25.472146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:25.472470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:25.472526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:25.472534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:25.472536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:25.472537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:25.472538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:25.472540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:25.472541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.472542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.472543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.472549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:25.472560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:25.472564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:25.472568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:25.472574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:25.472578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:25.472582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.472586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.472590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.472597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:25.472605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:25.472611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:25.472616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:25.472620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:25.472624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:25.472628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:25.472632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:25.472636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:25.472641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:25.472645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:25.472650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:25.472655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.472660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.472665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.472669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.472673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.472678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.472705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:25.480713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:25.482585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:25.482601Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:25.482603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:25.482608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:25.482619Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 519 [2:7559341406207687062:5455]
2025-10-09T21:45:25.482635Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:25.482659Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:25.482681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:25.482684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:25.482686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:25.482709Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:25.482715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:25.482723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:25.482790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.482792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.482794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.482797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:25.482871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:25.482874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.482887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.482899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.482914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:25.482936Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341406207687057:5455]
2025-10-09T21:45:25.482938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341406207687057:5455]
2025-10-09T21:45:25.482941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341406207687057:5455]
2025-10-09T21:45:25.482951Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:25.482966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:25.482980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:25.482983Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:25.482994Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:25.482997Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:25.483002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:25.483030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341406207687057:5455]
2025-10-09T21:45:25.483046Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:25.483053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:25.483057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341406569499669:2432], now have 1 active actors on pipe
2025-10-09T21:45:25.483061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341406569499754:5963], now have 1 active actors on pipe
2025-10-09T21:45:25.483314Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 519
2025-10-09T21:45:25.483372Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 519
2025-10-09T21:45:25.483496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:25.483507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:25.483541Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ff2b9c51-8a5288af-576544ff-8f692284_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:25.483554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:25.483558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:25.483561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:25.483587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:25.483600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:25.483618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:25.484655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 520 partition: 0 MaxSeqNo: 25 sessionId: src_id|ff2b9c51-8a5288af-576544ff-8f692284_0
2025-10-09T21:45:25.485078Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|608abe13-4790c478-4f775182-accdc6a4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ff2b9c51-8a5288af-576544ff-8f692284_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:25.484296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:25.484308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:25.484339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:25.487747Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff2b9c51-8a5288af-576544ff-8f692284_0] Write session: try to update token
2025-10-09T21:45:25.487765Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff2b9c51-8a5288af-576544ff-8f692284_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:25.505576Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:25.505593Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:25.505596Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:25.505598Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:25.505608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341406207687057:5455]
2025-10-09T21:45:25.505613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341406207687057:5455]
2025-10-09T21:45:25.505617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341406207687057:5455]
2025-10-09T21:45:25.505668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:25.505676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.505678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:25.505684Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:25.505689Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:25.505703Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:25.505827Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:25.512073Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 520 sessionId: src_id|ff2b9c51-8a5288af-576544ff-8f692284_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:25.515157Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:25.539334Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341406569499782:5852]
2025-10-09T21:45:25.540732Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 519, pipe: [1:7559341406569499782:5852]
2025-10-09T21:45:25.776141Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:25.776671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:25.776681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:25.776725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341406569499782:5852], now have 1 active actors on pipe
2025-10-09T21:45:25.777196Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:25.777213Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:25.777293Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 520 reason: error at write: Disconnected sessionId: src_id|ff2b9c51-8a5288af-576544ff-8f692284_0
2025-10-09T21:45:25.777451Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341406569499782:5852]
2025-10-09T21:45:25.777479Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:25.777484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 520 sessionId: src_id|ff2b9c51-8a5288af-576544ff-8f692284_0 is DEAD
2025-10-09T21:45:25.777634Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff2b9c51-8a5288af-576544ff-8f692284_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:25.777646Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff2b9c51-8a5288af-576544ff-8f692284_0] Write session will restart in 0.008908s
2025-10-09T21:45:25.777670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff2b9c51-8a5288af-576544ff-8f692284_0] Write session: Do CDS request
2025-10-09T21:45:25.777681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff2b9c51-8a5288af-576544ff-8f692284_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:25.777893Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:25.787622Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff2b9c51-8a5288af-576544ff-8f692284_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:25.787707Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:25.787727Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 521
2025-10-09T21:45:25.787900Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 521 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:25.787935Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 521 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:25.787942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=521 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:25.787946Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:25.788991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 521 sessionId:  describe result for acl check
2025-10-09T21:45:25.789032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:25.789041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:25.789042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:25.789049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341406569499812:5975] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:25.789593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341406569499812:5975] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:25.803934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341406569499812:5975] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:25.814429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:25.815560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:25.816126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341406569499812:5975] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:25.816146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341406569499812:5975] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:25.816150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341406569499812:5975] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:25.816161Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 521 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:25.816173Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:25.816184Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:25.816201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:25.816284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:25.816345Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341406207687163:5459]
2025-10-09T21:45:25.816903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:25.817381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:25.817433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:25.817745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:25.817810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:25.817811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:25.817812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:25.817814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:25.817815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:25.817816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:25.817817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.817819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.817820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.817826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:25.817840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:25.817844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:25.817848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:25.817851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:25.817855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:25.817858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.817862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.817866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.817873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:25.817881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:25.817889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:25.817895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:25.817900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:25.817905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:25.817910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:25.817915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:25.817919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:25.817923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:25.817928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:25.817932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:25.817937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.817941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.817946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.817951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.817955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.817961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.817987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:25.826316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:25.827430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:25.827439Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:25.827442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:25.827447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:25.827460Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 520 [2:7559341406207687163:5459]
2025-10-09T21:45:25.827477Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:25.827499Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:25.827502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:25.827506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:25.827509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:25.827529Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:25.827534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:25.827542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:25.827611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.827613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.827616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.827619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:25.827707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:25.827710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.827727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:25.827740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:25.827758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:25.827779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341406207687158:5459]
2025-10-09T21:45:25.827784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341406207687158:5459]
2025-10-09T21:45:25.827786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341406207687158:5459]
2025-10-09T21:45:25.827795Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:25.827812Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:25.827828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:25.827831Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:25.827874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341406207687158:5459]
2025-10-09T21:45:25.827877Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:25.828294Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:25.828301Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:25.828304Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:25.828310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:25.829169Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 520
2025-10-09T21:45:25.829212Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 520
2025-10-09T21:45:25.834184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341406569499799:2432], now have 1 active actors on pipe
2025-10-09T21:45:25.834219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341406569499887:5975], now have 1 active actors on pipe
2025-10-09T21:45:25.834258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:25.834267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:25.834310Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|21b41a36-2e93bb07-91fb1319-df36534_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:25.834328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:25.834334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:25.834337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:25.834352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:25.834373Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:25.834400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:25.837252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:25.837278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:25.837319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:25.837525Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 521 partition: 0 MaxSeqNo: 25 sessionId: src_id|21b41a36-2e93bb07-91fb1319-df36534_0
2025-10-09T21:45:25.838100Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff2b9c51-8a5288af-576544ff-8f692284_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|21b41a36-2e93bb07-91fb1319-df36534_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:25.840931Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21b41a36-2e93bb07-91fb1319-df36534_0] Write session: try to update token
2025-10-09T21:45:25.840975Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21b41a36-2e93bb07-91fb1319-df36534_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:25.854624Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:25.854642Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:25.854644Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:25.854646Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:25.854658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341406207687158:5459]
2025-10-09T21:45:25.854662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341406207687158:5459]
2025-10-09T21:45:25.854666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341406207687158:5459]
2025-10-09T21:45:25.854712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:25.854720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:25.854723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:25.854864Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:25.857815Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:25.857825Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:25.857829Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:25.877788Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341406569499900:5852]
2025-10-09T21:45:25.878763Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 520, pipe: [1:7559341406569499900:5852]
2025-10-09T21:45:25.891977Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 521 sessionId: src_id|21b41a36-2e93bb07-91fb1319-df36534_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:25.894815Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:26.061457Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:26.061954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341406569499900:5852], now have 1 active actors on pipe
2025-10-09T21:45:26.062014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:26.062022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:26.063912Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:26.064731Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341406569499900:5852]
2025-10-09T21:45:26.064763Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:26.064798Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:26.064801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:26.064858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 521 reason: error at write: Disconnected sessionId: src_id|21b41a36-2e93bb07-91fb1319-df36534_0
2025-10-09T21:45:26.065041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 521 sessionId: src_id|21b41a36-2e93bb07-91fb1319-df36534_0 is DEAD
2025-10-09T21:45:26.065487Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21b41a36-2e93bb07-91fb1319-df36534_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:26.065500Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21b41a36-2e93bb07-91fb1319-df36534_0] Write session will restart in 0.005016s
2025-10-09T21:45:26.065536Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21b41a36-2e93bb07-91fb1319-df36534_0] Write session: Do CDS request
2025-10-09T21:45:26.065543Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21b41a36-2e93bb07-91fb1319-df36534_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:26.074022Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21b41a36-2e93bb07-91fb1319-df36534_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:26.078457Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:26.078486Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 522
2025-10-09T21:45:26.078884Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 522 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:26.078927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 522 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:26.078937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=522 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:26.078941Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:26.079448Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 522 sessionId:  describe result for acl check
2025-10-09T21:45:26.079482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:26.079485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:26.079488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:26.079496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341410864467217:5980] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:26.080295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341410864467217:5980] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:26.089136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341410864467217:5980] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:26.096766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341410864467217:5980] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:26.096784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341410864467217:5980] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:26.096789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341410864467217:5980] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:26.096799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 522 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:26.104688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:26.104879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:26.105051Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:26.105055Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:26.105068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:26.105122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:26.105163Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341410502654555:5461]
2025-10-09T21:45:26.105802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:26.106485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:26.106545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:26.106883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:26.106946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:26.106948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:26.106949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:26.106950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:26.106952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:26.106953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:26.106954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.106955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.106957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.106963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:26.106974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:26.106978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:26.106982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:26.106986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:26.106990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:26.106995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.106999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.107003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.107011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:26.107020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:26.107026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:26.107032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:26.107037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:26.107043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:26.107048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:26.107053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:26.107058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:26.107062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:26.107066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:26.107071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:26.107079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.107085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.107089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.107095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.107100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.107106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.107154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:26.115744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:26.116827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:26.116836Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:26.116839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:26.116844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:26.116852Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 521 [2:7559341410502654555:5461]
2025-10-09T21:45:26.116870Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:26.116890Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:26.116894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:26.116898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:26.116900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:26.116920Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:26.116925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:26.116933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:26.117015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.117017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.117022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.117025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:26.117106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:26.117109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.117125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.117136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.117156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:26.117177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341410502654550:5461]
2025-10-09T21:45:26.117180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341410502654550:5461]
2025-10-09T21:45:26.117183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341410502654550:5461]
2025-10-09T21:45:26.117198Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:26.117214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:26.117230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:26.117233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:26.117277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341410502654550:5461]
2025-10-09T21:45:26.117280Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:26.118485Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 521
2025-10-09T21:45:26.118521Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 521
2025-10-09T21:45:26.117582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:26.117590Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:26.117593Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:26.117599Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:26.119242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341410864467296:5980], now have 1 active actors on pipe
2025-10-09T21:45:26.119252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341410864467209:2432], now have 1 active actors on pipe
2025-10-09T21:45:26.119268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:26.119274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:26.119309Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ee09386c-c6bcbbb5-3e3e59c4-8a8a2511_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:26.119320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:26.119326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:26.119330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:26.119342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:26.119357Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:26.119377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:26.119705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:26.119713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:26.119729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:26.119907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 522 partition: 0 MaxSeqNo: 25 sessionId: src_id|ee09386c-c6bcbbb5-3e3e59c4-8a8a2511_0
2025-10-09T21:45:26.120373Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21b41a36-2e93bb07-91fb1319-df36534_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ee09386c-c6bcbbb5-3e3e59c4-8a8a2511_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:26.126279Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee09386c-c6bcbbb5-3e3e59c4-8a8a2511_0] Write session: try to update token
2025-10-09T21:45:26.126304Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee09386c-c6bcbbb5-3e3e59c4-8a8a2511_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:26.157413Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 522 sessionId: src_id|ee09386c-c6bcbbb5-3e3e59c4-8a8a2511_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:26.160781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:26.160799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:26.162019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:26.162063Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:26.162069Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:26.162071Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:26.162072Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:26.162080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341410502654550:5461]
2025-10-09T21:45:26.162084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341410502654550:5461]
2025-10-09T21:45:26.162087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341410502654550:5461]
2025-10-09T21:45:26.162123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:26.162130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.162132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:26.162281Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:26.160149Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:26.161892Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341410864467308:5852]
2025-10-09T21:45:26.170589Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:26.170599Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:26.170603Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:26.171058Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:26.171064Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 521, pipe: [1:7559341410864467308:5852]
2025-10-09T21:45:26.377435Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:26.377887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341410864467308:5852], now have 1 active actors on pipe
2025-10-09T21:45:26.398142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:26.398165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:26.398351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:26.398402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:26.398516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:26.398577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:26.398631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:26.398636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:26.398698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:26.398753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:26.398811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:26.398869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:26.398906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:26.398909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:26.398970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:26.399024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:26.399082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:26.399141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:26.399183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:26.399186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:26.399256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:26.399318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:26.399376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:26.399434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:26.399472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:26.399475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:26.399539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:26.399596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:26.399657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:26.399715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:26.399755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:26.399758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:26.399821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:26.399872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:26.399928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:26.399988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:26.400033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:26.400036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:26.400106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:26.400159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:26.400241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:26.400299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:26.400342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:26.400345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:26.400407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:26.400561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:26.400618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:26.400676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:26.400717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:26.400720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:26.400783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:26.400837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:26.400894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:26.400951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:26.400989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:26.400991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:26.401055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:26.401110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:26.401165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:26.401225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:26.401263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:26.401265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:26.403215Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341410864467308:5852]
2025-10-09T21:45:26.403252Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:26.403255Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:26.403282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 522 reason: error at write: Disconnected sessionId: src_id|ee09386c-c6bcbbb5-3e3e59c4-8a8a2511_0
2025-10-09T21:45:26.403484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 522 sessionId: src_id|ee09386c-c6bcbbb5-3e3e59c4-8a8a2511_0 is DEAD
2025-10-09T21:45:26.403610Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:26.403989Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee09386c-c6bcbbb5-3e3e59c4-8a8a2511_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:26.403999Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee09386c-c6bcbbb5-3e3e59c4-8a8a2511_0] Write session will restart in 0.006516s
2025-10-09T21:45:26.404023Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee09386c-c6bcbbb5-3e3e59c4-8a8a2511_0] Write session: Do CDS request
2025-10-09T21:45:26.404030Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee09386c-c6bcbbb5-3e3e59c4-8a8a2511_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:26.406581Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:26.415450Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee09386c-c6bcbbb5-3e3e59c4-8a8a2511_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:26.415811Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:26.415830Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 523
2025-10-09T21:45:26.416010Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 523 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:26.416042Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 523 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:26.416048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=523 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:26.416051Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:26.416676Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 523 sessionId:  describe result for acl check
2025-10-09T21:45:26.416720Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:26.416722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:26.416724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:26.416731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341410864467331:5984] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:26.417271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341410864467331:5984] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:26.428838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341410864467331:5984] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:26.429296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:26.433204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:26.433459Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:26.433464Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:26.433476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:26.433535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:26.433573Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341410502654685:5472]
2025-10-09T21:45:26.434159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:26.435848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:26.435902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:26.436216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:26.436264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:26.436265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:26.436266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:26.436268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:26.436269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:26.436270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:26.436272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.436275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.436276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.436281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:26.436292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:26.436296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:26.436300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:26.436303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:26.436307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:26.436311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.436314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.436318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.436325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:26.436332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:26.436338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:26.436343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:26.436348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:26.436353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:26.436358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:26.436363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:26.436367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:26.436371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:26.436375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:26.436379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:26.436384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.436389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.436393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.436399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.436403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.436408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.436437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:26.435365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341410864467331:5984] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:26.435382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341410864467331:5984] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:26.435387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341410864467331:5984] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:26.435397Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 523 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:26.444860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:26.445933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:26.445943Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:26.445945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:26.445950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:26.445957Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 522 [2:7559341410502654685:5472]
2025-10-09T21:45:26.445973Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:26.445994Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:26.445998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:26.446001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:26.446003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:26.446022Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:26.446050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:26.446058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:26.446125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.446129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.446131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.446134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:26.446231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:26.446234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.446247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.446257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.446274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:26.446295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341410502654680:5472]
2025-10-09T21:45:26.446297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341410502654680:5472]
2025-10-09T21:45:26.446299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341410502654680:5472]
2025-10-09T21:45:26.446315Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:26.446332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:26.446346Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:26.446350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:26.446390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341410502654680:5472]
2025-10-09T21:45:26.446393Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:26.446659Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:26.446666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:26.446668Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:26.446674Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:26.447422Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 522
2025-10-09T21:45:26.447460Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 522
2025-10-09T21:45:26.447847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341410864467407:5984], now have 1 active actors on pipe
2025-10-09T21:45:26.447857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341410864467315:2432], now have 1 active actors on pipe
2025-10-09T21:45:26.447876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:26.447882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:26.447917Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|45178189-5c3007f4-7cd10f9a-1573734d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:26.447926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:26.447932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:26.447935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:26.447948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:26.447963Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:26.447982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:26.451522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:26.451537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:26.451574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:26.452279Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee09386c-c6bcbbb5-3e3e59c4-8a8a2511_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|45178189-5c3007f4-7cd10f9a-1573734d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:26.455026Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45178189-5c3007f4-7cd10f9a-1573734d_0] Write session: try to update token
2025-10-09T21:45:26.455048Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45178189-5c3007f4-7cd10f9a-1573734d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:26.451808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 523 partition: 0 MaxSeqNo: 25 sessionId: src_id|45178189-5c3007f4-7cd10f9a-1573734d_0
2025-10-09T21:45:26.487568Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 523 sessionId: src_id|45178189-5c3007f4-7cd10f9a-1573734d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:26.490086Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:26.489646Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:26.489665Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:26.489668Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:26.489670Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:26.489681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341410502654680:5472]
2025-10-09T21:45:26.489685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341410502654680:5472]
2025-10-09T21:45:26.489689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341410502654680:5472]
2025-10-09T21:45:26.489737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:26.489744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.489747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:26.489890Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:26.492279Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:26.492295Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:26.492302Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:26.506130Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341410864467417:5852]
2025-10-09T21:45:26.508203Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 522, pipe: [1:7559341410864467417:5852]
2025-10-09T21:45:26.665187Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:26.666887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:26.666903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:26.666950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341410864467417:5852], now have 1 active actors on pipe
2025-10-09T21:45:26.670090Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45178189-5c3007f4-7cd10f9a-1573734d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:26.670103Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45178189-5c3007f4-7cd10f9a-1573734d_0] Write session will restart in 0.007600s
2025-10-09T21:45:26.670128Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45178189-5c3007f4-7cd10f9a-1573734d_0] Write session: Do CDS request
2025-10-09T21:45:26.670136Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45178189-5c3007f4-7cd10f9a-1573734d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:26.669786Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:26.669286Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:26.669302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:26.669362Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 523 reason: error at write: Disconnected sessionId: src_id|45178189-5c3007f4-7cd10f9a-1573734d_0
2025-10-09T21:45:26.669530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 523 sessionId: src_id|45178189-5c3007f4-7cd10f9a-1573734d_0 is DEAD
2025-10-09T21:45:26.669696Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341410864467417:5852]
2025-10-09T21:45:26.669710Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:26.686826Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45178189-5c3007f4-7cd10f9a-1573734d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:26.687517Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:26.687538Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 524
2025-10-09T21:45:26.687719Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 524 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:26.687755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 524 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:26.687763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=524 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:26.687779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:26.688256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 524 sessionId:  describe result for acl check
2025-10-09T21:45:26.688289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:26.688291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:26.688294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:26.688300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341410864467493:5997] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:26.688866Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341410864467493:5997] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:26.691674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:26.692796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341410864467493:5997] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:26.696609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:26.696863Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:26.696875Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:26.696889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:26.696973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:26.697016Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341410502654781:5477]
2025-10-09T21:45:26.697147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341410864467493:5997] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:26.697166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341410864467493:5997] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:26.697170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341410864467493:5997] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:26.697180Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 524 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:26.697599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:26.698076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:26.698143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:26.698528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:26.698591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:26.698592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:26.698593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:26.698594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:26.698596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:26.698597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:26.698598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.698599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.698600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.698606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:26.698616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:26.698619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:26.698623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:26.698628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:26.698632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:26.698638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.698642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.698646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.698653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:26.698663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:26.698669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:26.698676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:26.698681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:26.698686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:26.698692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:26.698698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:26.698703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:26.698708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:26.698713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:26.698720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:26.698725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.698729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.698733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.698738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.698746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.698751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.698780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:26.707055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:26.708124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:26.708142Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:26.708145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:26.708149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:26.708155Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 523 [2:7559341410502654781:5477]
2025-10-09T21:45:26.708170Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:26.708202Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:26.708210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:26.708214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:26.708216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:26.708235Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:26.708245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:26.708253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:26.708311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.708318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.708323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.708326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:26.708397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:26.708404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.708418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.708435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.708457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:26.708484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341410864467530:5997], now have 1 active actors on pipe
2025-10-09T21:45:26.708492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341410864467438:2432], now have 1 active actors on pipe
2025-10-09T21:45:26.708500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341410502654775:5477]
2025-10-09T21:45:26.708506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341410502654775:5477]
2025-10-09T21:45:26.708509Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341410502654775:5477]
2025-10-09T21:45:26.708532Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:26.708551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:26.708572Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:26.708579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:26.708619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341410502654775:5477]
2025-10-09T21:45:26.708626Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:26.708621Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 523
2025-10-09T21:45:26.708666Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 523
2025-10-09T21:45:26.708657Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:26.708665Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:26.708667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:26.708683Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:26.708913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:26.708925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:26.708957Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e5022818-69173a55-481ee11d-2bb2f6c9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:26.708967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:26.708973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:26.708976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:26.708987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:26.709000Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:26.709017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:26.710203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:26.710215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:26.710237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:26.711296Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 524 partition: 0 MaxSeqNo: 25 sessionId: src_id|e5022818-69173a55-481ee11d-2bb2f6c9_0
2025-10-09T21:45:26.714473Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45178189-5c3007f4-7cd10f9a-1573734d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e5022818-69173a55-481ee11d-2bb2f6c9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:26.717010Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5022818-69173a55-481ee11d-2bb2f6c9_0] Write session: try to update token
2025-10-09T21:45:26.717032Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5022818-69173a55-481ee11d-2bb2f6c9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:26.727216Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:26.727230Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:26.727233Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:26.727239Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:26.727250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341410502654775:5477]
2025-10-09T21:45:26.727254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341410502654775:5477]
2025-10-09T21:45:26.727258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341410502654775:5477]
2025-10-09T21:45:26.727307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:26.727314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.727317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:26.727461Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:26.728761Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:26.728769Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:26.728774Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:26.749613Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 524 sessionId: src_id|e5022818-69173a55-481ee11d-2bb2f6c9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:26.752209Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:26.770364Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341410864467551:5852]
2025-10-09T21:45:26.778452Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 523, pipe: [1:7559341410864467551:5852]
2025-10-09T21:45:26.920078Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:26.920602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:26.920617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:26.920640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341410864467551:5852], now have 1 active actors on pipe
2025-10-09T21:45:26.920939Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:26.920970Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:26.921050Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 524 reason: error at write: Disconnected sessionId: src_id|e5022818-69173a55-481ee11d-2bb2f6c9_0
2025-10-09T21:45:26.921261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 524 sessionId: src_id|e5022818-69173a55-481ee11d-2bb2f6c9_0 is DEAD
2025-10-09T21:45:26.921109Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:26.921476Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341410864467551:5852]
2025-10-09T21:45:26.921495Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:26.922226Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5022818-69173a55-481ee11d-2bb2f6c9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:26.922240Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5022818-69173a55-481ee11d-2bb2f6c9_0] Write session will restart in 0.007000s
2025-10-09T21:45:26.922267Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5022818-69173a55-481ee11d-2bb2f6c9_0] Write session: Do CDS request
2025-10-09T21:45:26.922273Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5022818-69173a55-481ee11d-2bb2f6c9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:26.930681Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5022818-69173a55-481ee11d-2bb2f6c9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:26.931275Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:26.931294Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 525
2025-10-09T21:45:26.931449Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 525 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:26.931492Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 525 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:26.931498Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=525 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:26.931503Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:26.932220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 525 sessionId:  describe result for acl check
2025-10-09T21:45:26.932259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:26.932266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:26.932267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:26.932273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341410864467566:6002] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:26.932804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341410864467566:6002] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:26.937749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341410864467566:6002] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:26.942836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341410864467566:6002] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:26.942856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341410864467566:6002] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:26.942860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341410864467566:6002] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:26.942870Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 525 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:26.945084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:26.952126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:26.952700Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:26.952714Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:26.952734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:26.952844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:26.952892Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341410502654888:5481]
2025-10-09T21:45:26.953464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:26.954011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:26.954087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:26.954412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:26.954472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:26.954474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:26.954477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:26.954478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:26.954480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:26.954481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:26.954482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.954483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.954485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.954491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:26.954501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:26.954505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:26.954509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:26.954512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:26.954516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:26.954520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.954524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.954527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.954535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:26.954544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:26.954550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:26.954555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:26.954559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:26.954579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:26.954585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:26.954590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:26.954595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:26.954600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:26.954605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:26.954610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:26.954616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.954621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.954625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.954632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.954637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.954643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.954674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:26.964188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:26.965311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:26.965318Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:26.965321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:26.965327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:26.965335Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 524 [2:7559341410502654888:5481]
2025-10-09T21:45:26.965351Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:26.965374Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:26.965378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:26.965381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:26.965384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:26.965404Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:26.965410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:26.965418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:26.965488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.965491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.965493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.965497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:26.965584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:26.965587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:26.965604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:26.965614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:26.965629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:26.965651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341410502654882:5481]
2025-10-09T21:45:26.965654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341410502654882:5481]
2025-10-09T21:45:26.965656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341410502654882:5481]
2025-10-09T21:45:26.965671Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:26.965687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:26.965701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:26.965704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:26.965748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341410502654882:5481]
2025-10-09T21:45:26.965751Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:26.966023Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:26.966092Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:26.966095Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:26.966102Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:26.966944Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 524
2025-10-09T21:45:26.966986Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 524
2025-10-09T21:45:26.967448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341410864467554:2432], now have 1 active actors on pipe
2025-10-09T21:45:26.967457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341410864467643:6002], now have 1 active actors on pipe
2025-10-09T21:45:26.967476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:26.967481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:26.967511Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|92f7e89e-a639a97d-affa6c52-360fba58_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:26.967522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:26.967526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:26.967529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:26.967543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:26.967557Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:26.967585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:26.968290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:26.968298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:26.968317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:26.968484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 525 partition: 0 MaxSeqNo: 25 sessionId: src_id|92f7e89e-a639a97d-affa6c52-360fba58_0
2025-10-09T21:45:26.968906Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5022818-69173a55-481ee11d-2bb2f6c9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|92f7e89e-a639a97d-affa6c52-360fba58_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:26.971466Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92f7e89e-a639a97d-affa6c52-360fba58_0] Write session: try to update token
2025-10-09T21:45:26.971486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92f7e89e-a639a97d-affa6c52-360fba58_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:27.011346Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:27.011363Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:27.011365Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:27.011367Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:27.011381Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341410502654882:5481]
2025-10-09T21:45:27.011387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341410502654882:5481]
2025-10-09T21:45:27.011390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341410502654882:5481]
2025-10-09T21:45:27.011441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:27.011442Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:27.011449Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:27.011453Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:27.011455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.011458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:27.011585Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:27.019571Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 525 sessionId: src_id|92f7e89e-a639a97d-affa6c52-360fba58_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:27.022163Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:27.022265Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341415159434962:5852]
2025-10-09T21:45:27.024543Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 524, pipe: [1:7559341415159434962:5852]
2025-10-09T21:45:27.179556Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:27.180533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:27.180558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:27.180584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341415159434962:5852], now have 1 active actors on pipe
2025-10-09T21:45:27.180910Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:27.181916Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92f7e89e-a639a97d-affa6c52-360fba58_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:27.181941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92f7e89e-a639a97d-affa6c52-360fba58_0] Write session will restart in 0.007378s
2025-10-09T21:45:27.181966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92f7e89e-a639a97d-affa6c52-360fba58_0] Write session: Do CDS request
2025-10-09T21:45:27.181972Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92f7e89e-a639a97d-affa6c52-360fba58_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:27.181116Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:27.181171Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:27.181174Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:27.181191Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341415159434962:5852]
2025-10-09T21:45:27.181238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 525 reason: error at write: Disconnected sessionId: src_id|92f7e89e-a639a97d-affa6c52-360fba58_0
2025-10-09T21:45:27.181456Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 525 sessionId: src_id|92f7e89e-a639a97d-affa6c52-360fba58_0 is DEAD
2025-10-09T21:45:27.193159Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92f7e89e-a639a97d-affa6c52-360fba58_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:27.193545Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:27.193567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 526
2025-10-09T21:45:27.193780Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 526 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:27.193819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 526 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:27.193826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=526 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:27.193833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:27.194481Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 526 sessionId:  describe result for acl check
2025-10-09T21:45:27.194528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:27.194531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:27.194535Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:27.194543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341415159435019:6007] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:27.195162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341415159435019:6007] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:27.199096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341415159435019:6007] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:27.207766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:27.208790Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341415159435019:6007] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:27.208821Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341415159435019:6007] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:27.208826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341415159435019:6007] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:27.208836Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 526 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:27.210950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:27.211235Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:27.211247Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:27.211261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:27.211345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:27.211391Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341414797622284:5483]
2025-10-09T21:45:27.212013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:27.212465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:27.212535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:27.212912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:27.212973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:27.212979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:27.212981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:27.212982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:27.212984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:27.212985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:27.212987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.212988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.212989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.212995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:27.213007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:27.213011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:27.213015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:27.213019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:27.213024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:27.213028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.213032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.213036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.213045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:27.213053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:27.213059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:27.213065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:27.213070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:27.213075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:27.213080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:27.213085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:27.213090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:27.213097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:27.213102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:27.213108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:27.213119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.213124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.213128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.213133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.213137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.213141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.213173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:27.220739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:27.222617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:27.222644Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:27.222647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:27.222652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:27.222660Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 525 [2:7559341414797622284:5483]
2025-10-09T21:45:27.222677Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:27.222702Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:27.222706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:27.222711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:27.222714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:27.222737Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:27.222749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:27.222758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:27.222831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.222838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.222841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.222844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:27.222931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:27.222939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.222956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.222973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.222995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:27.223018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341414797622278:5483]
2025-10-09T21:45:27.223021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341414797622278:5483]
2025-10-09T21:45:27.223023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341414797622278:5483]
2025-10-09T21:45:27.223034Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:27.223051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:27.223067Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:27.223070Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:27.223072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:27.223072Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:27.223075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:27.223128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341415159434965:2432], now have 1 active actors on pipe
2025-10-09T21:45:27.223133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341415159435060:6007], now have 1 active actors on pipe
2025-10-09T21:45:27.223136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341414797622278:5483]
2025-10-09T21:45:27.223144Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:27.223231Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 525
2025-10-09T21:45:27.223283Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 525
2025-10-09T21:45:27.223427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:27.223719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:27.223726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:27.223761Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c4f7fbde-3d81d7a2-10a47d0f-67997860_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:27.223771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:27.223776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:27.223779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:27.223793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:27.223806Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:27.223824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:27.224391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:27.224403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:27.224433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:27.224656Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 526 partition: 0 MaxSeqNo: 25 sessionId: src_id|c4f7fbde-3d81d7a2-10a47d0f-67997860_0
2025-10-09T21:45:27.225162Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92f7e89e-a639a97d-affa6c52-360fba58_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c4f7fbde-3d81d7a2-10a47d0f-67997860_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:27.227873Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4f7fbde-3d81d7a2-10a47d0f-67997860_0] Write session: try to update token
2025-10-09T21:45:27.227893Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4f7fbde-3d81d7a2-10a47d0f-67997860_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:27.246938Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:27.246952Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:27.246954Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:27.246956Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:27.246966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341414797622278:5483]
2025-10-09T21:45:27.246972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341414797622278:5483]
2025-10-09T21:45:27.246976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341414797622278:5483]
2025-10-09T21:45:27.247024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:27.247031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.247034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:27.247167Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:27.247800Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:27.247826Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:27.247831Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:27.263994Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 526 sessionId: src_id|c4f7fbde-3d81d7a2-10a47d0f-67997860_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:27.266476Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:27.281246Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341415159435071:5852]
2025-10-09T21:45:27.281942Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 525, pipe: [1:7559341415159435071:5852]
2025-10-09T21:45:27.434945Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:27.436053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:27.436067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:27.436090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341415159435071:5852], now have 1 active actors on pipe
2025-10-09T21:45:27.436308Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:27.438288Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4f7fbde-3d81d7a2-10a47d0f-67997860_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:27.438301Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4f7fbde-3d81d7a2-10a47d0f-67997860_0] Write session will restart in 0.009649s
2025-10-09T21:45:27.437405Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341415159435071:5852]
2025-10-09T21:45:27.437447Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:27.437491Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:27.437566Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 526 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c4f7fbde-3d81d7a2-10a47d0f-67997860_0
2025-10-09T21:45:27.438333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4f7fbde-3d81d7a2-10a47d0f-67997860_0] Write session: Do CDS request
2025-10-09T21:45:27.437756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 526 sessionId: src_id|c4f7fbde-3d81d7a2-10a47d0f-67997860_0 is DEAD
2025-10-09T21:45:27.438342Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4f7fbde-3d81d7a2-10a47d0f-67997860_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:27.450603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4f7fbde-3d81d7a2-10a47d0f-67997860_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:27.451258Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:27.451277Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 527
2025-10-09T21:45:27.452422Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 527 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:27.452462Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 527 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:27.452467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=527 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:27.452471Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:27.453893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 527 sessionId:  describe result for acl check
2025-10-09T21:45:27.453930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:27.453932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:27.453934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:27.453940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341415159435123:6011] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:27.454511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341415159435123:6011] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:27.462137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341415159435123:6011] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:27.470711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:27.470930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:27.471098Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:27.471102Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:27.471120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:27.471176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:27.471214Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341414797622398:5493]
2025-10-09T21:45:27.471797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:27.471920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341415159435123:6011] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:27.471938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341415159435123:6011] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:27.471942Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341415159435123:6011] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:27.471951Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 527 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:27.472726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:27.472778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:27.473118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:27.473175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:27.473176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:27.473177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:27.473179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:27.473180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:27.473181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:27.473183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.473184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.473185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.473191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:27.473202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:27.473206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:27.473210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:27.473214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:27.473219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:27.473223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.473226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.473230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.473237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:27.473245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:27.473252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:27.473257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:27.473262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:27.473267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:27.473272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:27.473279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:27.473283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:27.473287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:27.473291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:27.473296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:27.473301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.473307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.473311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.473316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.473321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.473326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.473356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:27.483655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:27.484774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:27.484780Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:27.484783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:27.484787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:27.484795Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 526 [2:7559341414797622398:5493]
2025-10-09T21:45:27.484814Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:27.484834Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:27.484837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:27.484840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:27.484843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:27.484863Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:27.484869Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:27.484876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:27.484940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.484942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.484944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.484947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:27.485020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:27.485023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.485034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.485045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.485059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:27.485080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341414797622392:5493]
2025-10-09T21:45:27.485082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341414797622392:5493]
2025-10-09T21:45:27.485084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341414797622392:5493]
2025-10-09T21:45:27.485091Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:27.485102Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:27.485105Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:27.485106Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:27.485107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:27.485119Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:27.485123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:27.485134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341415159435168:6011], now have 1 active actors on pipe
2025-10-09T21:45:27.485137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341415159435074:2432], now have 1 active actors on pipe
2025-10-09T21:45:27.485176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341414797622392:5493]
2025-10-09T21:45:27.485184Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:27.485628Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 526
2025-10-09T21:45:27.485668Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 526
2025-10-09T21:45:27.485767Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:27.486211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:27.486225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:27.486261Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7bbfd455-7e010461-b39ff331-d9fbdde9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:27.486272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:27.486277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:27.486280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:27.486291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:27.486304Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:27.486320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:27.486857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:27.486864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:27.486884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:27.489777Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4f7fbde-3d81d7a2-10a47d0f-67997860_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7bbfd455-7e010461-b39ff331-d9fbdde9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:27.489256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 527 partition: 0 MaxSeqNo: 25 sessionId: src_id|7bbfd455-7e010461-b39ff331-d9fbdde9_0
2025-10-09T21:45:27.492371Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7bbfd455-7e010461-b39ff331-d9fbdde9_0] Write session: try to update token
2025-10-09T21:45:27.492388Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7bbfd455-7e010461-b39ff331-d9fbdde9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:27.508188Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:27.508207Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:27.508210Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:27.508212Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:27.508221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341414797622392:5493]
2025-10-09T21:45:27.508226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341414797622392:5493]
2025-10-09T21:45:27.508233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341414797622392:5493]
2025-10-09T21:45:27.508282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:27.508289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.508292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:27.508430Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:27.510806Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:27.510818Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:27.510821Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:27.527387Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 527 sessionId: src_id|7bbfd455-7e010461-b39ff331-d9fbdde9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:27.529805Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:27.539801Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341415159435181:5852]
2025-10-09T21:45:27.544062Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 526, pipe: [1:7559341415159435181:5852]
2025-10-09T21:45:27.717973Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:27.718984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:27.719002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:27.719051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341415159435181:5852], now have 1 active actors on pipe
2025-10-09T21:45:27.719676Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:27.720540Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:27.721230Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bbfd455-7e010461-b39ff331-d9fbdde9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:27.721241Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bbfd455-7e010461-b39ff331-d9fbdde9_0] Write session will restart in 0.009706s
2025-10-09T21:45:27.721264Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bbfd455-7e010461-b39ff331-d9fbdde9_0] Write session: Do CDS request
2025-10-09T21:45:27.721270Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bbfd455-7e010461-b39ff331-d9fbdde9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:27.720592Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341415159435181:5852]
2025-10-09T21:45:27.720619Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:27.720687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 527 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7bbfd455-7e010461-b39ff331-d9fbdde9_0
2025-10-09T21:45:27.720867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 527 sessionId: src_id|7bbfd455-7e010461-b39ff331-d9fbdde9_0 is DEAD
2025-10-09T21:45:27.732734Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7bbfd455-7e010461-b39ff331-d9fbdde9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:27.733108Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:27.733127Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 528
2025-10-09T21:45:27.733454Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 528 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:27.733494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 528 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:27.733500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=528 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:27.733503Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:27.735043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 528 sessionId:  describe result for acl check
2025-10-09T21:45:27.735085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:27.735087Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:27.735089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:27.735095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341415159435241:6022] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:27.735652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341415159435241:6022] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:27.745075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:27.749344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:27.749580Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:27.749592Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:27.749605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:27.749665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:27.749944Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341414797622515:5499]
2025-10-09T21:45:27.750513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:27.751150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:27.751199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:27.751507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:27.751549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:27.751550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:27.751552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:27.751553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:27.751554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:27.751555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:27.751556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.751557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.751558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.751562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:27.751571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:27.751574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:27.751578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:27.751580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:27.751584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:27.751586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.751589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.751593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.751599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:27.751607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:27.751612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:27.751616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:27.751620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:27.751626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:27.751630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:27.751634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:27.751638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:27.751643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:27.751647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:27.751651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:27.751654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.751658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.751679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.751683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.751686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.751692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.751719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:27.754897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341415159435241:6022] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:27.764868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:27.766143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:27.766159Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:27.766162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:27.766166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:27.766172Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 527 [2:7559341414797622515:5499]
2025-10-09T21:45:27.766200Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:27.766219Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:27.766222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:27.766224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:27.766227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:27.766245Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:27.766252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:27.766258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:27.766318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.766324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.766326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.766329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:27.766399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:27.766407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.766418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:27.766429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:27.766442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:27.766463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341414797622509:5499]
2025-10-09T21:45:27.766465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341414797622509:5499]
2025-10-09T21:45:27.766467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341414797622509:5499]
2025-10-09T21:45:27.766480Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:27.766496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:27.766509Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:27.766512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:27.766549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341414797622509:5499]
2025-10-09T21:45:27.766555Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:27.766867Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:27.766877Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:27.766879Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:27.766885Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:27.767040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341415159435200:2432], now have 1 active actors on pipe
2025-10-09T21:45:27.767948Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 527
2025-10-09T21:45:27.775127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341415159435241:6022] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:27.775148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341415159435241:6022] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:27.775151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341415159435241:6022] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:27.775162Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 528 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:27.776720Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 527
2025-10-09T21:45:27.777452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341415159435310:6022], now have 1 active actors on pipe
2025-10-09T21:45:27.777476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:27.777484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:27.777518Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3c934593-48ae0db6-e72e078d-44d83520_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:27.777537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:27.777544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:27.777548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:27.777561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:27.777583Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:27.777614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:27.779068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:27.779081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:27.779106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:27.781509Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bbfd455-7e010461-b39ff331-d9fbdde9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3c934593-48ae0db6-e72e078d-44d83520_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:27.780839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 528 partition: 0 MaxSeqNo: 25 sessionId: src_id|3c934593-48ae0db6-e72e078d-44d83520_0
2025-10-09T21:45:27.784405Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c934593-48ae0db6-e72e078d-44d83520_0] Write session: try to update token
2025-10-09T21:45:27.784428Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c934593-48ae0db6-e72e078d-44d83520_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:27.795041Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:27.795060Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:27.795063Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:27.795066Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:27.795079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341414797622509:5499]
2025-10-09T21:45:27.795084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341414797622509:5499]
2025-10-09T21:45:27.795088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341414797622509:5499]
2025-10-09T21:45:27.795136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:27.795144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:27.795147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:27.795293Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:27.796722Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:27.796741Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:27.796745Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:27.804433Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 528 sessionId: src_id|3c934593-48ae0db6-e72e078d-44d83520_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:27.807947Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:27.820777Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341415159435316:5852]
2025-10-09T21:45:27.821317Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 527, pipe: [1:7559341415159435316:5852]
2025-10-09T21:45:28.000390Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:28.000802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:28.000812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:28.000837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341415159435316:5852], now have 1 active actors on pipe
2025-10-09T21:45:28.001790Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:28.001996Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:28.002091Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:28.002094Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:28.002168Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 528 reason: error at write: Disconnected sessionId: src_id|3c934593-48ae0db6-e72e078d-44d83520_0
2025-10-09T21:45:28.002377Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 528 sessionId: src_id|3c934593-48ae0db6-e72e078d-44d83520_0 is DEAD
2025-10-09T21:45:28.002552Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341415159435316:5852]
2025-10-09T21:45:28.003543Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c934593-48ae0db6-e72e078d-44d83520_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:28.003557Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c934593-48ae0db6-e72e078d-44d83520_0] Write session will restart in 0.007300s
2025-10-09T21:45:28.003586Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c934593-48ae0db6-e72e078d-44d83520_0] Write session: Do CDS request
2025-10-09T21:45:28.003593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c934593-48ae0db6-e72e078d-44d83520_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:28.012626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c934593-48ae0db6-e72e078d-44d83520_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:28.012750Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:28.012769Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 529
2025-10-09T21:45:28.013039Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 529 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:28.013076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 529 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:28.013092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=529 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:28.013097Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:28.013718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 529 sessionId:  describe result for acl check
2025-10-09T21:45:28.013755Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:28.013757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:28.013759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:28.013765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341419454402632:6028] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:28.014414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341419454402632:6028] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:28.020730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341419454402632:6028] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:28.031850Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341419454402632:6028] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:28.031870Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341419454402632:6028] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:28.031876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341419454402632:6028] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:28.031886Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 529 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:28.045095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:28.051821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:28.052282Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:28.052287Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:28.052303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:28.052390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:28.052430Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341419092589921:5503]
2025-10-09T21:45:28.053039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:28.053726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:28.053771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:28.054116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:28.054167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:28.054168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:28.054170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:28.054171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:28.054172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:28.054173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:28.054174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.054176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.054177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.054208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:28.054220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:28.054224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:28.054229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:28.054233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:28.054237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:28.054241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.054245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.054248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.054257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:28.054266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:28.054274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:28.054289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:28.054296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:28.054302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:28.054306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:28.054311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:28.054316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:28.054321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:28.054326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:28.054331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:28.054337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.054343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.054347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.054352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.054357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.054362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.054395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:28.076453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:28.077662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:28.077669Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:28.077672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:28.077678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:28.077686Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 528 [2:7559341419092589921:5503]
2025-10-09T21:45:28.077704Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:28.077724Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:28.077728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:28.077732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:28.077737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:28.077755Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:28.077761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:28.077769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:28.077844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.077859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.077861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.077864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:28.077976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:28.077985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.078000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.078013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.078047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:28.078072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341419454402711:6028], now have 1 active actors on pipe
2025-10-09T21:45:28.078077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341419454402623:2432], now have 1 active actors on pipe
2025-10-09T21:45:28.078086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341419092589915:5503]
2025-10-09T21:45:28.078088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341419092589915:5503]
2025-10-09T21:45:28.078090Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341419092589915:5503]
2025-10-09T21:45:28.078112Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:28.078134Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:28.078151Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:28.078155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:28.078225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341419092589915:5503]
2025-10-09T21:45:28.078229Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:28.078296Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 528
2025-10-09T21:45:28.078333Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 528
2025-10-09T21:45:28.078291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:28.078298Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:28.078300Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:28.078307Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:28.078572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:28.078580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:28.078614Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cb1a664e-c9c0010b-7c414aa5-4c9bf22f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:28.078625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:28.078629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:28.078632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:28.078644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:28.078659Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:28.078679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:28.082264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:28.082280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:28.082318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:28.085677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 529 partition: 0 MaxSeqNo: 25 sessionId: src_id|cb1a664e-c9c0010b-7c414aa5-4c9bf22f_0
2025-10-09T21:45:28.088744Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c934593-48ae0db6-e72e078d-44d83520_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cb1a664e-c9c0010b-7c414aa5-4c9bf22f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:28.091447Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cb1a664e-c9c0010b-7c414aa5-4c9bf22f_0] Write session: try to update token
2025-10-09T21:45:28.091470Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cb1a664e-c9c0010b-7c414aa5-4c9bf22f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:28.100935Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341419454402732:5852]
2025-10-09T21:45:28.102783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341419454402732:5852], now have 1 active actors on pipe
2025-10-09T21:45:28.102770Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 528, pipe: [1:7559341419454402732:5852]
2025-10-09T21:45:28.115737Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:28.115756Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:28.115759Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:28.115762Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:28.115776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341419092589915:5503]
2025-10-09T21:45:28.115781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341419092589915:5503]
2025-10-09T21:45:28.115787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341419092589915:5503]
2025-10-09T21:45:28.115828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:28.115835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.115838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:28.115971Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:28.118677Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:28.118689Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:28.118693Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:28.123722Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 529 sessionId: src_id|cb1a664e-c9c0010b-7c414aa5-4c9bf22f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:28.126425Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:28.391175Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:28.391676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:28.391689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:28.393899Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:28.393914Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:28.393973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 529 reason: error at write: Disconnected sessionId: src_id|cb1a664e-c9c0010b-7c414aa5-4c9bf22f_0
2025-10-09T21:45:28.394105Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:28.394154Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341419454402732:5852]
2025-10-09T21:45:28.394221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 529 sessionId: src_id|cb1a664e-c9c0010b-7c414aa5-4c9bf22f_0 is DEAD
2025-10-09T21:45:28.394679Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb1a664e-c9c0010b-7c414aa5-4c9bf22f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:28.394693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb1a664e-c9c0010b-7c414aa5-4c9bf22f_0] Write session will restart in 0.008278s
2025-10-09T21:45:28.394720Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb1a664e-c9c0010b-7c414aa5-4c9bf22f_0] Write session: Do CDS request
2025-10-09T21:45:28.394739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb1a664e-c9c0010b-7c414aa5-4c9bf22f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:28.398345Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:28.407828Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cb1a664e-c9c0010b-7c414aa5-4c9bf22f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:28.407872Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:28.407890Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 530
2025-10-09T21:45:28.408090Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 530 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:28.408132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 530 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:28.408139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=530 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:28.408143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:28.408594Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 530 sessionId:  describe result for acl check
2025-10-09T21:45:28.408634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:28.408641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:28.408643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:28.408651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341419454402763:6032] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:28.409200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341419454402763:6032] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:28.412059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341419454402763:6032] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:28.413952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:28.418354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341419454402763:6032] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:28.418374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341419454402763:6032] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:28.418378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341419454402763:6032] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:28.418389Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 530 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:28.418405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:28.418665Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:28.418669Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:28.418683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:28.418759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:28.418796Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341419092590028:5507]
2025-10-09T21:45:28.419390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:28.419832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:28.419894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:28.420252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:28.420323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:28.420329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:28.420331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:28.420332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:28.420333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:28.420334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:28.420335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.420336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.420337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.420343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:28.420367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:28.420372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:28.420376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:28.420380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:28.420383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:28.420388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.420392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.420396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.420404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:28.420412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:28.420419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:28.420424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:28.420430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:28.420442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:28.420446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:28.420451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:28.420455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:28.420460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:28.420464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:28.420469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:28.420474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.420479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.420483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.420488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.420492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.420497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.420525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:28.428415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:28.429750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:28.429764Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:28.429766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:28.429771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:28.429778Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 529 [2:7559341419092590028:5507]
2025-10-09T21:45:28.429795Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:28.429816Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:28.429824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:28.429829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:28.429832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:28.429850Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:28.429861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:28.429869Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:28.429935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.429943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.429945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.429948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:28.430054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:28.430062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.430076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.430101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.430122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:28.430149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341419454402837:6032], now have 1 active actors on pipe
2025-10-09T21:45:28.430153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341419454402739:2432], now have 1 active actors on pipe
2025-10-09T21:45:28.430161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341419092590022:5507]
2025-10-09T21:45:28.430164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341419092590022:5507]
2025-10-09T21:45:28.430166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341419092590022:5507]
2025-10-09T21:45:28.430255Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 529
2025-10-09T21:45:28.430287Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 529
2025-10-09T21:45:28.430192Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:28.430211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:28.430228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:28.430232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:28.430274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341419092590022:5507]
2025-10-09T21:45:28.430277Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:28.430341Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:28.430346Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:28.430351Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:28.430361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:28.430728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:28.430743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:28.430788Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|80db3e21-fcbcfdb9-b1a6b952-40545d77_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:28.431060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:28.431071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:28.431074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:28.431085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:28.431098Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:28.431119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:28.431398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:28.431408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:28.431427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:28.431565Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 530 partition: 0 MaxSeqNo: 25 sessionId: src_id|80db3e21-fcbcfdb9-b1a6b952-40545d77_0
2025-10-09T21:45:28.434277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb1a664e-c9c0010b-7c414aa5-4c9bf22f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|80db3e21-fcbcfdb9-b1a6b952-40545d77_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:28.436735Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80db3e21-fcbcfdb9-b1a6b952-40545d77_0] Write session: try to update token
2025-10-09T21:45:28.436756Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80db3e21-fcbcfdb9-b1a6b952-40545d77_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:28.449308Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:28.449324Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:28.449327Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:28.449329Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:28.449341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341419092590022:5507]
2025-10-09T21:45:28.449347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341419092590022:5507]
2025-10-09T21:45:28.449350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341419092590022:5507]
2025-10-09T21:45:28.449398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:28.449407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.449409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:28.449554Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:28.454352Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:28.454372Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:28.454377Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:28.459384Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 530 sessionId: src_id|80db3e21-fcbcfdb9-b1a6b952-40545d77_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:28.461982Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:28.494367Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341419454402848:5852]
2025-10-09T21:45:28.496895Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 529, pipe: [1:7559341419454402848:5852]
2025-10-09T21:45:28.669680Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:28.670006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:28.670023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:28.671226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341419454402848:5852], now have 1 active actors on pipe
2025-10-09T21:45:28.674549Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:28.679621Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80db3e21-fcbcfdb9-b1a6b952-40545d77_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:28.679634Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80db3e21-fcbcfdb9-b1a6b952-40545d77_0] Write session will restart in 0.005799s
2025-10-09T21:45:28.679661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80db3e21-fcbcfdb9-b1a6b952-40545d77_0] Write session: Do CDS request
2025-10-09T21:45:28.679667Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80db3e21-fcbcfdb9-b1a6b952-40545d77_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:28.678990Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:28.679051Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:28.679135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 530 reason: status is not ok: tablet will be restarted right now sessionId: src_id|80db3e21-fcbcfdb9-b1a6b952-40545d77_0
2025-10-09T21:45:28.679177Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341419454402848:5852]
2025-10-09T21:45:28.679333Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 530 sessionId: src_id|80db3e21-fcbcfdb9-b1a6b952-40545d77_0 is DEAD
2025-10-09T21:45:28.690517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80db3e21-fcbcfdb9-b1a6b952-40545d77_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:28.691782Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:28.691802Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 531
2025-10-09T21:45:28.692008Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 531 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:28.692040Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 531 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:28.692046Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=531 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:28.692050Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:28.695228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 531 sessionId:  describe result for acl check
2025-10-09T21:45:28.695275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:28.695277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:28.695282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:28.695288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341419454402863:6037] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:28.695842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341419454402863:6037] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:28.707760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341419454402863:6037] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:28.716210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341419454402863:6037] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:28.716229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341419454402863:6037] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:28.716234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341419454402863:6037] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:28.716245Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 531 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:28.724999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:28.730295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:28.730593Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:28.730598Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:28.730611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:28.730684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:28.730722Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341419092590162:5519]
2025-10-09T21:45:28.731300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:28.731746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:28.731807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:28.732175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:28.732223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:28.732224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:28.732225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:28.732227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:28.732228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:28.732230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:28.732231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.732232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.732233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.732239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:28.732249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:28.732254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:28.732257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:28.732272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:28.732276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:28.732280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.732284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.732286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.732294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:28.732301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:28.732307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:28.732313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:28.732318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:28.732323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:28.732328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:28.732333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:28.732337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:28.732345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:28.732349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:28.732355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:28.732359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.732364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.732380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.732386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.732391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.732396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.732426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:28.741971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:28.743713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:28.743732Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:28.743734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:28.743738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:28.743745Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 530 [2:7559341419092590162:5519]
2025-10-09T21:45:28.743763Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:28.743783Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:28.743786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:28.743790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:28.743792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:28.743814Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:28.743820Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:28.743828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:28.743892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.743900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.743902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.743904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:28.743978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:28.743988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.744001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:28.744012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:28.744028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:28.744048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341419092590157:5519]
2025-10-09T21:45:28.744051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341419092590157:5519]
2025-10-09T21:45:28.744053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341419092590157:5519]
2025-10-09T21:45:28.744068Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:28.744081Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:28.744096Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:28.744099Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:28.744138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341419092590157:5519]
2025-10-09T21:45:28.744140Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:28.744487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:28.744495Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:28.744497Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:28.744502Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:28.744824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341419454402904:6037], now have 1 active actors on pipe
2025-10-09T21:45:28.744832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341419454402855:2432], now have 1 active actors on pipe
2025-10-09T21:45:28.747560Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 530
2025-10-09T21:45:28.747600Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 530
2025-10-09T21:45:28.748027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:28.748041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:28.748090Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9525478-92ef1a1-eced8aac-4a58a6f8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:28.748107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:28.748114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:28.748121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:28.748135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:28.748157Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:28.748200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:28.748489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:28.748502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:28.748524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:28.748625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 531 partition: 0 MaxSeqNo: 25 sessionId: src_id|9525478-92ef1a1-eced8aac-4a58a6f8_0
2025-10-09T21:45:28.749014Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80db3e21-fcbcfdb9-b1a6b952-40545d77_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9525478-92ef1a1-eced8aac-4a58a6f8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:28.751645Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9525478-92ef1a1-eced8aac-4a58a6f8_0] Write session: try to update token
2025-10-09T21:45:28.751664Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9525478-92ef1a1-eced8aac-4a58a6f8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:28.767444Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:28.767465Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:28.767468Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:28.767470Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:28.767480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341419092590157:5519]
2025-10-09T21:45:28.767486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341419092590157:5519]
2025-10-09T21:45:28.767490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341419092590157:5519]
2025-10-09T21:45:28.767538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:28.767545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:28.767548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:28.767713Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:28.770474Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:28.770482Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:28.770486Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:28.780289Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341419454402966:5852]
2025-10-09T21:45:28.782421Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 530, pipe: [1:7559341419454402966:5852]
2025-10-09T21:45:28.790085Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 531 sessionId: src_id|9525478-92ef1a1-eced8aac-4a58a6f8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:28.792645Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:28.998637Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:28.999058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341419454402966:5852], now have 1 active actors on pipe
2025-10-09T21:45:28.999092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:28.999100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:28.999965Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:28.999985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:28.999713Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:29.000049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 531 reason: error at write: Disconnected sessionId: src_id|9525478-92ef1a1-eced8aac-4a58a6f8_0
2025-10-09T21:45:29.000244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 531 sessionId: src_id|9525478-92ef1a1-eced8aac-4a58a6f8_0 is DEAD
2025-10-09T21:45:29.000376Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341419454402966:5852]
2025-10-09T21:45:29.000392Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:29.006386Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9525478-92ef1a1-eced8aac-4a58a6f8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:29.006405Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9525478-92ef1a1-eced8aac-4a58a6f8_0] Write session will restart in 0.005807s
2025-10-09T21:45:29.006434Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9525478-92ef1a1-eced8aac-4a58a6f8_0] Write session: Do CDS request
2025-10-09T21:45:29.006441Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9525478-92ef1a1-eced8aac-4a58a6f8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:29.014655Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9525478-92ef1a1-eced8aac-4a58a6f8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:29.017648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:29.017672Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 532
2025-10-09T21:45:29.017831Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 532 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:29.017866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 532 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:29.017877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=532 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:29.017881Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:29.018336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 532 sessionId:  describe result for acl check
2025-10-09T21:45:29.018374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:29.018377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:29.018379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:29.018386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341423749370312:6049] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:29.018975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341423749370312:6049] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:29.030559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341423749370312:6049] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:29.034748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:29.045071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341423749370312:6049] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:29.045095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341423749370312:6049] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:29.045099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341423749370312:6049] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:29.045110Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 532 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:29.044825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:29.045107Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:29.045111Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:29.045124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:29.045188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:29.045231Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341423387557570:5523]
2025-10-09T21:45:29.045800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:29.046395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:29.046472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:29.046833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:29.047142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:29.047154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:29.047155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:29.047157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:29.047158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:29.047160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:29.047161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.047162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.047164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.047171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:29.047183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:29.047187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:29.047191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:29.047195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:29.047199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:29.047204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.047208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.047212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.047224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:29.047233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:29.047240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:29.047245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:29.047251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:29.047256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:29.047260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:29.047267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:29.047272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:29.047278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:29.047283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:29.047288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:29.047294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.047299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.047304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.047309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.047314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.047319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.047349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:29.056469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:29.058062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:29.058078Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:29.058081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:29.058085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:29.058094Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 531 [2:7559341423387557570:5523]
2025-10-09T21:45:29.058115Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:29.058137Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:29.058141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:29.058145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:29.058148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:29.058168Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:29.058174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:29.058196Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:29.058273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.058275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.058278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.058281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:29.058532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:29.058536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.058551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.058564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.058581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:29.058606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341423387557565:5523]
2025-10-09T21:45:29.058609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341423387557565:5523]
2025-10-09T21:45:29.058611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341423387557565:5523]
2025-10-09T21:45:29.058625Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:29.058643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:29.058660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:29.058664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:29.058708Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341423387557565:5523]
2025-10-09T21:45:29.058710Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:29.059049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:29.059057Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:29.059060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:29.059078Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:29.059131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341423749370294:2432], now have 1 active actors on pipe
2025-10-09T21:45:29.059145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341423749370400:6049], now have 1 active actors on pipe
2025-10-09T21:45:29.059610Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 531
2025-10-09T21:45:29.059655Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 531
2025-10-09T21:45:29.060232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:29.060259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:29.060295Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ef4a2bef-4231027e-7c0c1042-4cf7b4f1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:29.060307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:29.060311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:29.060314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:29.060328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:29.060356Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:29.060381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:29.060899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:29.060915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:29.060958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:29.061123Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 532 partition: 0 MaxSeqNo: 25 sessionId: src_id|ef4a2bef-4231027e-7c0c1042-4cf7b4f1_0
2025-10-09T21:45:29.066153Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9525478-92ef1a1-eced8aac-4a58a6f8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ef4a2bef-4231027e-7c0c1042-4cf7b4f1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:29.068872Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef4a2bef-4231027e-7c0c1042-4cf7b4f1_0] Write session: try to update token
2025-10-09T21:45:29.068897Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef4a2bef-4231027e-7c0c1042-4cf7b4f1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:29.081357Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:29.081376Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:29.081379Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:29.081381Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:29.081389Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341423387557565:5523]
2025-10-09T21:45:29.081393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341423387557565:5523]
2025-10-09T21:45:29.081397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341423387557565:5523]
2025-10-09T21:45:29.081444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:29.081452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.081455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:29.081595Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:29.082090Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:29.082097Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:29.082101Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:29.100096Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 532 sessionId: src_id|ef4a2bef-4231027e-7c0c1042-4cf7b4f1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:29.100322Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341423749370412:5852]
2025-10-09T21:45:29.100938Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 531, pipe: [1:7559341423749370412:5852]
2025-10-09T21:45:29.102624Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:29.265471Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:29.266801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:29.266816Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:29.266878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 532 reason: error at write: Disconnected sessionId: src_id|ef4a2bef-4231027e-7c0c1042-4cf7b4f1_0
2025-10-09T21:45:29.267066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 532 sessionId: src_id|ef4a2bef-4231027e-7c0c1042-4cf7b4f1_0 is DEAD
2025-10-09T21:45:29.267279Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef4a2bef-4231027e-7c0c1042-4cf7b4f1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:29.267291Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef4a2bef-4231027e-7c0c1042-4cf7b4f1_0] Write session will restart in 0.006438s
2025-10-09T21:45:29.267317Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef4a2bef-4231027e-7c0c1042-4cf7b4f1_0] Write session: Do CDS request
2025-10-09T21:45:29.267324Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef4a2bef-4231027e-7c0c1042-4cf7b4f1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:29.267266Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:29.267355Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341423749370412:5852]
2025-10-09T21:45:29.265761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341423749370412:5852], now have 1 active actors on pipe
2025-10-09T21:45:29.265796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:29.265804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:29.268089Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:29.274767Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef4a2bef-4231027e-7c0c1042-4cf7b4f1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:29.275690Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:29.275706Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 533
2025-10-09T21:45:29.275900Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 533 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:29.275946Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 533 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:29.275958Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=533 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:29.275962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:29.276674Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 533 sessionId:  describe result for acl check
2025-10-09T21:45:29.276723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:29.276730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:29.276732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:29.276738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341423749370429:6053] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:29.277702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341423749370429:6053] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:29.280529Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341423749370429:6053] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:29.287276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:29.287446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341423749370429:6053] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:29.287463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341423749370429:6053] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:29.287467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341423749370429:6053] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:29.287479Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 533 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:29.288265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:29.288484Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:29.288494Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:29.288507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:29.288561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:29.288617Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341423387557679:5526]
2025-10-09T21:45:29.289238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:29.289711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:29.289775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:29.290393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:29.290470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:29.290476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:29.290477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:29.290478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:29.290480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:29.290481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:29.290482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.290483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.290484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.290490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:29.290501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:29.290505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:29.290509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:29.290512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:29.290517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:29.290520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.290524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.290529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.290537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:29.290548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:29.290554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:29.290561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:29.290566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:29.290572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:29.290579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:29.290584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:29.290590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:29.290595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:29.290600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:29.290605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:29.290610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.290614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.290618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.290624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.290629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.290635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.290668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:29.301206Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 532
2025-10-09T21:45:29.301224Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 532
2025-10-09T21:45:29.299129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:29.300528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:29.300535Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:29.300537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:29.300542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:29.300550Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 532 [2:7559341423387557679:5526]
2025-10-09T21:45:29.300568Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:29.300589Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:29.300593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:29.300596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:29.300599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:29.300617Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:29.300623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:29.300631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:29.300868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.300871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.300873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.300875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:29.300956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:29.300959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.300979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.300993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.301013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:29.301036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341423387557673:5526]
2025-10-09T21:45:29.301038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341423387557673:5526]
2025-10-09T21:45:29.301040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341423387557673:5526]
2025-10-09T21:45:29.301051Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:29.301065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:29.301080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:29.301084Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:29.301087Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:29.301091Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:29.301093Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:29.301125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341423387557673:5526]
2025-10-09T21:45:29.301128Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:29.301130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:29.301153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341423749370415:2432], now have 1 active actors on pipe
2025-10-09T21:45:29.301157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341423749370513:6053], now have 1 active actors on pipe
2025-10-09T21:45:29.302049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:29.302060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:29.302093Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ac550ea5-77a8d734-d7d8278b-7e7dd1c6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:29.302104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:29.302108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:29.302110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:29.302136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:29.302148Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:29.302166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:29.302628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:29.302635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:29.302653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:29.302828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 533 partition: 0 MaxSeqNo: 25 sessionId: src_id|ac550ea5-77a8d734-d7d8278b-7e7dd1c6_0
2025-10-09T21:45:29.303223Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef4a2bef-4231027e-7c0c1042-4cf7b4f1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ac550ea5-77a8d734-d7d8278b-7e7dd1c6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:29.305642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac550ea5-77a8d734-d7d8278b-7e7dd1c6_0] Write session: try to update token
2025-10-09T21:45:29.305662Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac550ea5-77a8d734-d7d8278b-7e7dd1c6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:29.320930Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 533 sessionId: src_id|ac550ea5-77a8d734-d7d8278b-7e7dd1c6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:29.323486Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:29.322627Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:29.322644Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:29.322646Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:29.322648Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:29.322658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341423387557673:5526]
2025-10-09T21:45:29.322663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341423387557673:5526]
2025-10-09T21:45:29.322667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341423387557673:5526]
2025-10-09T21:45:29.322720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:29.322728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.322730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:29.322881Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:29.324751Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:29.324765Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:29.324769Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:29.367580Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341423749370526:5852]
2025-10-09T21:45:29.369019Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 532, pipe: [1:7559341423749370526:5852]
2025-10-09T21:45:29.492975Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:29.493765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:29.493783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:29.493830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341423749370526:5852], now have 1 active actors on pipe
2025-10-09T21:45:29.496436Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:29.497523Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341423749370526:5852]
2025-10-09T21:45:29.497554Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:29.497592Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:29.497687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 533 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ac550ea5-77a8d734-d7d8278b-7e7dd1c6_0
2025-10-09T21:45:29.497878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 533 sessionId: src_id|ac550ea5-77a8d734-d7d8278b-7e7dd1c6_0 is DEAD
2025-10-09T21:45:29.500103Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac550ea5-77a8d734-d7d8278b-7e7dd1c6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:29.500114Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac550ea5-77a8d734-d7d8278b-7e7dd1c6_0] Write session will restart in 0.009694s
2025-10-09T21:45:29.500143Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac550ea5-77a8d734-d7d8278b-7e7dd1c6_0] Write session: Do CDS request
2025-10-09T21:45:29.500149Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac550ea5-77a8d734-d7d8278b-7e7dd1c6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:29.510611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac550ea5-77a8d734-d7d8278b-7e7dd1c6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:29.513245Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:29.513263Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 534
2025-10-09T21:45:29.513765Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 534 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:29.513804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 534 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:29.513810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=534 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:29.513815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:29.516960Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 534 sessionId:  describe result for acl check
2025-10-09T21:45:29.517004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:29.517006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:29.517011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:29.517018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341423749370537:6057] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:29.517614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341423749370537:6057] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:29.527000Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341423749370537:6057] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:29.533927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341423749370537:6057] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:29.533948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341423749370537:6057] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:29.533952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341423749370537:6057] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:29.533961Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 534 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:29.545257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:29.550243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:29.550520Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:29.550526Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:29.550540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:29.550612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:29.550649Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341423387557786:5529]
2025-10-09T21:45:29.551230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:29.555103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:29.555227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:29.555802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:29.555867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:29.555868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:29.555870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:29.555871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:29.555873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:29.555874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:29.555875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.555877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.555878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.555885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:29.555897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:29.555901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:29.555906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:29.555910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:29.555915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:29.555919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.555923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.555927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.555937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:29.555946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:29.555953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:29.555960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:29.555966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:29.555973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:29.555978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:29.555984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:29.555989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:29.555996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:29.556000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:29.556006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:29.556012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.556018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.556022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.556028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.556034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.556039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.556069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:29.565370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:29.566853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:29.566861Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:29.566864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:29.566869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:29.566877Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 533 [2:7559341423387557786:5529]
2025-10-09T21:45:29.566892Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:29.566913Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:29.566916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:29.566919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:29.566922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:29.566941Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:29.566948Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:29.566955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:29.567022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.567024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.567026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.567029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:29.567110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:29.567113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.567130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:29.567141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:29.567158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:29.567178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341423387557781:5529]
2025-10-09T21:45:29.567181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341423387557781:5529]
2025-10-09T21:45:29.567183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341423387557781:5529]
2025-10-09T21:45:29.567192Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:29.567207Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:29.567221Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:29.567224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:29.567237Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:29.567241Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:29.567244Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:29.567267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341423387557781:5529]
2025-10-09T21:45:29.567270Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:29.567281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341423749370627:6057], now have 1 active actors on pipe
2025-10-09T21:45:29.567285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341423749370529:2432], now have 1 active actors on pipe
2025-10-09T21:45:29.567328Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:29.570089Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 533
2025-10-09T21:45:29.570125Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 533
2025-10-09T21:45:29.570502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:29.570515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:29.570564Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e52357c4-10e4e604-b30a183b-7bd65237_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:29.570576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:29.570582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:29.570586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:29.570600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:29.570617Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:29.570640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:29.571369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:29.571382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:29.571416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:29.574138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 534 partition: 0 MaxSeqNo: 25 sessionId: src_id|e52357c4-10e4e604-b30a183b-7bd65237_0
2025-10-09T21:45:29.574726Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac550ea5-77a8d734-d7d8278b-7e7dd1c6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e52357c4-10e4e604-b30a183b-7bd65237_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:29.577130Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e52357c4-10e4e604-b30a183b-7bd65237_0] Write session: try to update token
2025-10-09T21:45:29.577146Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e52357c4-10e4e604-b30a183b-7bd65237_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:29.602110Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341423749370643:5852]
2025-10-09T21:45:29.602678Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 533, pipe: [1:7559341423749370643:5852]
2025-10-09T21:45:29.604549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341423749370643:5852], now have 1 active actors on pipe
2025-10-09T21:45:29.614196Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 534 sessionId: src_id|e52357c4-10e4e604-b30a183b-7bd65237_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:29.616570Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:29.618159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:29.618195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:29.618239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:29.621108Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:29.622608Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:29.622624Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:29.622628Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:29.622629Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:29.622644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341423387557781:5529]
2025-10-09T21:45:29.622648Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341423387557781:5529]
2025-10-09T21:45:29.622652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341423387557781:5529]
2025-10-09T21:45:29.622696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:29.622703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:29.622705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:29.622839Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:29.626279Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:29.626290Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:29.626294Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:29.973100Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:29.996133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:29.996159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:29.996338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:29.996394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:29.996449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:29.996508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:29.996714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:29.996722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:29.996786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:29.996841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:29.996897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:29.996958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:29.997001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:29.997015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:29.997077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:29.997138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:29.997195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:29.997253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:29.997296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:29.997299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:29.997392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:29.997451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:29.997509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:29.997567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:29.997609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:29.997612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:29.997677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:29.997731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:29.997790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:29.997851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:29.997890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:29.997893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:29.997955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:29.998009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:30.005973Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:30.005998Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:30.006042Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 534 reason: error at write: Disconnected sessionId: src_id|e52357c4-10e4e604-b30a183b-7bd65237_0
2025-10-09T21:45:30.002470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:30.002545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:30.002591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:30.002596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:30.002675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:30.002732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:30.002793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:30.002849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:30.002891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:30.002897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:30.002963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:30.003019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:30.003100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:30.003159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:30.003202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:30.003206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:30.003280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:30.003336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:30.003392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:30.003453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:30.003498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:30.003502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:30.003570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:30.003629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:30.003691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:30.003751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:30.003793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:30.003796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:30.006661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e52357c4-10e4e604-b30a183b-7bd65237_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:30.006670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e52357c4-10e4e604-b30a183b-7bd65237_0] Write session will restart in 0.008918s
2025-10-09T21:45:30.006694Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e52357c4-10e4e604-b30a183b-7bd65237_0] Write session: Do CDS request
2025-10-09T21:45:30.006700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e52357c4-10e4e604-b30a183b-7bd65237_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:30.006254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 534 sessionId: src_id|e52357c4-10e4e604-b30a183b-7bd65237_0 is DEAD
2025-10-09T21:45:30.006417Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341423749370643:5852]
2025-10-09T21:45:30.006432Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:30.006903Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:30.016573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e52357c4-10e4e604-b30a183b-7bd65237_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:30.018227Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:30.018248Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 535
2025-10-09T21:45:30.021419Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 535 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:30.021494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 535 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:30.021505Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=535 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:30.021511Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:30.026952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 535 sessionId:  describe result for acl check
2025-10-09T21:45:30.027008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:30.027010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:30.027013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:30.027020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341428044337992:6070] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:30.027636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341428044337992:6070] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:30.032445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341428044337992:6070] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:30.039817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341428044337992:6070] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:30.039841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341428044337992:6070] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:30.039846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341428044337992:6070] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:30.039857Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 535 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:30.056350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:30.057309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:30.057545Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:30.057557Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:30.057571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:30.057649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:30.057695Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341427682525230:5543]
2025-10-09T21:45:30.058324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:30.058772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:30.058818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:30.059140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:30.059191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:30.059192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:30.059193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:30.059194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:30.059195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:30.059196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:30.059197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.059198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.059199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.059205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:30.059215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:30.059219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:30.059223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:30.059226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:30.059229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:30.059233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.059236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.059239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.059248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:30.059256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:30.059262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:30.059267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:30.059271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:30.059275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:30.059281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:30.059286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:30.059289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:30.059294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:30.059298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:30.059302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:30.059311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.059316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.059320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.059324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.059328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.059333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.059359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:30.068841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:30.070537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:30.070545Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:30.070548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:30.070552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:30.070561Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 534 [2:7559341427682525230:5543]
2025-10-09T21:45:30.070579Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:30.070600Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:30.070604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:30.070607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:30.070609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:30.070630Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:30.070637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:30.070645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:30.070718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.070720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.070726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.070729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:30.070814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:30.070817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.070832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.070842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.070857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:30.070878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341427682525225:5543]
2025-10-09T21:45:30.070881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341427682525225:5543]
2025-10-09T21:45:30.070882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341427682525225:5543]
2025-10-09T21:45:30.070891Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:30.070907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:30.070921Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:30.070923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:30.070924Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:30.070927Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:30.070929Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:30.070977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341428044337979:2432], now have 1 active actors on pipe
2025-10-09T21:45:30.070981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341428044338029:6070], now have 1 active actors on pipe
2025-10-09T21:45:30.070986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341427682525225:5543]
2025-10-09T21:45:30.070989Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:30.070992Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:30.071960Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 534
2025-10-09T21:45:30.072009Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 534
2025-10-09T21:45:30.075925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:30.075945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:30.075991Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|57a99fa3-725e3c63-7a9d6c20-e1d4a8f1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:30.076014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:30.076020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:30.076024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:30.076041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:30.076061Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:30.076094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:30.078812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:30.078836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:30.078873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:30.079150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 535 partition: 0 MaxSeqNo: 25 sessionId: src_id|57a99fa3-725e3c63-7a9d6c20-e1d4a8f1_0
2025-10-09T21:45:30.082460Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e52357c4-10e4e604-b30a183b-7bd65237_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|57a99fa3-725e3c63-7a9d6c20-e1d4a8f1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:30.085432Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57a99fa3-725e3c63-7a9d6c20-e1d4a8f1_0] Write session: try to update token
2025-10-09T21:45:30.085455Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57a99fa3-725e3c63-7a9d6c20-e1d4a8f1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:30.097725Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:30.097746Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:30.097750Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:30.097759Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:30.097770Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341427682525225:5543]
2025-10-09T21:45:30.097778Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341427682525225:5543]
2025-10-09T21:45:30.097783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341427682525225:5543]
2025-10-09T21:45:30.097838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:30.097848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.097852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:30.098102Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:30.098220Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:30.098235Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:30.098239Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:30.106101Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341428044338096:5852]
2025-10-09T21:45:30.115245Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 534, pipe: [1:7559341428044338096:5852]
2025-10-09T21:45:30.133077Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 535 sessionId: src_id|57a99fa3-725e3c63-7a9d6c20-e1d4a8f1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:30.135896Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:30.380901Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:30.381385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341428044338096:5852], now have 1 active actors on pipe
2025-10-09T21:45:30.381435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:30.381443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:30.381897Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:30.381913Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:30.381973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 535 reason: error at write: Disconnected sessionId: src_id|57a99fa3-725e3c63-7a9d6c20-e1d4a8f1_0
2025-10-09T21:45:30.382161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 535 sessionId: src_id|57a99fa3-725e3c63-7a9d6c20-e1d4a8f1_0 is DEAD
2025-10-09T21:45:30.382330Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341428044338096:5852]
2025-10-09T21:45:30.382344Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:30.381961Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:30.382511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57a99fa3-725e3c63-7a9d6c20-e1d4a8f1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:30.382529Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57a99fa3-725e3c63-7a9d6c20-e1d4a8f1_0] Write session will restart in 0.005749s
2025-10-09T21:45:30.382559Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57a99fa3-725e3c63-7a9d6c20-e1d4a8f1_0] Write session: Do CDS request
2025-10-09T21:45:30.382566Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57a99fa3-725e3c63-7a9d6c20-e1d4a8f1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:30.389533Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57a99fa3-725e3c63-7a9d6c20-e1d4a8f1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:30.389877Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:30.389894Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 536
2025-10-09T21:45:30.390070Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 536 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:30.390104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 536 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:30.390110Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=536 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:30.390113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:30.390658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 536 sessionId:  describe result for acl check
2025-10-09T21:45:30.390690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:30.390692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:30.390694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:30.390700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341428044338108:6074] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:30.391268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341428044338108:6074] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:30.396957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341428044338108:6074] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:30.410487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341428044338108:6074] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:30.410509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341428044338108:6074] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:30.410513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341428044338108:6074] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:30.410523Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 536 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:30.418800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:30.430743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:30.431018Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:30.431022Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:30.431036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:30.431095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:30.431137Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341427682525344:5547]
2025-10-09T21:45:30.431746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:30.432251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:30.432302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:30.432657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:30.432701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:30.432703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:30.432704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:30.432706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:30.432707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:30.432709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:30.432710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.432714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.432716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.432722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:30.432734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:30.432738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:30.432742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:30.432747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:30.432751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:30.432756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.432760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.432765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.432775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:30.432784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:30.432790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:30.432796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:30.432801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:30.432806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:30.432811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:30.432816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:30.432820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:30.432824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:30.432827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:30.432832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:30.432838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.432843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.432847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.432852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.432858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.432863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.432896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:30.447907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:30.449755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:30.449772Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:30.449774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:30.449779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:30.449787Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 535 [2:7559341427682525344:5547]
2025-10-09T21:45:30.449803Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:30.449824Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:30.449827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:30.449830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:30.449832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:30.449853Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:30.449860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:30.449868Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:30.449938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.449951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.449953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.449956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:30.450048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:30.450054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.450069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.450083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.450107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:30.450134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341427682525337:5547]
2025-10-09T21:45:30.450137Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341427682525337:5547]
2025-10-09T21:45:30.450139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341427682525337:5547]
2025-10-09T21:45:30.450154Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:30.450169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:30.450199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:30.450202Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:30.450250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341427682525337:5547]
2025-10-09T21:45:30.450253Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:30.450349Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:30.450351Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:30.450353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:30.450358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:30.451112Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 535
2025-10-09T21:45:30.451147Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 535
2025-10-09T21:45:30.451585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341428044338202:6074], now have 1 active actors on pipe
2025-10-09T21:45:30.451596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341428044338101:2432], now have 1 active actors on pipe
2025-10-09T21:45:30.451612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:30.451619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:30.451655Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bd291934-103bfa23-19b742fd-afcdda24_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:30.451665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:30.451670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:30.451673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:30.451702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:30.451716Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:30.451737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:30.452079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:30.452090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:30.452108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:30.452205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 536 partition: 0 MaxSeqNo: 25 sessionId: src_id|bd291934-103bfa23-19b742fd-afcdda24_0
2025-10-09T21:45:30.452648Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57a99fa3-725e3c63-7a9d6c20-e1d4a8f1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bd291934-103bfa23-19b742fd-afcdda24_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:30.455282Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd291934-103bfa23-19b742fd-afcdda24_0] Write session: try to update token
2025-10-09T21:45:30.455301Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd291934-103bfa23-19b742fd-afcdda24_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:30.477641Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:30.477659Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:30.477661Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:30.477663Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:30.477673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341427682525337:5547]
2025-10-09T21:45:30.477679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341427682525337:5547]
2025-10-09T21:45:30.477682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341427682525337:5547]
2025-10-09T21:45:30.477729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:30.477740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.477743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:30.477872Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:30.479875Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 536 sessionId: src_id|bd291934-103bfa23-19b742fd-afcdda24_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:30.482504Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:30.482647Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:30.482660Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:30.482663Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:30.482703Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341428044338220:5852]
2025-10-09T21:45:30.483795Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 535, pipe: [1:7559341428044338220:5852]
2025-10-09T21:45:30.657082Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:30.657376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:30.657390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:30.657415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341428044338220:5852], now have 1 active actors on pipe
2025-10-09T21:45:30.657702Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:30.657675Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:30.657690Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:30.657758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 536 reason: error at write: Disconnected sessionId: src_id|bd291934-103bfa23-19b742fd-afcdda24_0
2025-10-09T21:45:30.657951Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 536 sessionId: src_id|bd291934-103bfa23-19b742fd-afcdda24_0 is DEAD
2025-10-09T21:45:30.657978Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:30.658011Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341428044338220:5852]
2025-10-09T21:45:30.661513Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd291934-103bfa23-19b742fd-afcdda24_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:30.661524Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd291934-103bfa23-19b742fd-afcdda24_0] Write session will restart in 0.006241s
2025-10-09T21:45:30.661552Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd291934-103bfa23-19b742fd-afcdda24_0] Write session: Do CDS request
2025-10-09T21:45:30.661557Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd291934-103bfa23-19b742fd-afcdda24_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:30.668694Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd291934-103bfa23-19b742fd-afcdda24_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:30.673048Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:30.673066Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 537
2025-10-09T21:45:30.673312Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 537 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:30.673356Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 537 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:30.673364Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=537 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:30.673369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:30.673817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 537 sessionId:  describe result for acl check
2025-10-09T21:45:30.673846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:30.673848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:30.673849Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:30.673856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341428044338294:6079] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:30.674464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341428044338294:6079] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:30.679486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341428044338294:6079] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:30.683478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:30.690932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:30.691492Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:30.691505Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:30.691522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:30.691601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:30.691645Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341427682525453:5549]
2025-10-09T21:45:30.691095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341428044338294:6079] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:30.691117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341428044338294:6079] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:30.691120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341428044338294:6079] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:30.691130Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 537 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:30.692302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:30.692900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:30.692961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:30.693330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:30.693408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:30.693415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:30.693416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:30.693417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:30.693419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:30.693420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:30.693421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.693423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.693424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.693432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:30.693453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:30.693458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:30.693462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:30.693467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:30.693471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:30.693475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.693478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.693481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.693492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:30.693503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:30.693509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:30.693514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:30.693518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:30.693524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:30.693528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:30.693533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:30.693538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:30.693542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:30.693546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:30.693552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:30.693557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.693562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.693567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.693572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.693577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.693582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.693613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:30.701019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:30.702645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:30.702662Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:30.702665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:30.702670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:30.702679Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 536 [2:7559341427682525453:5549]
2025-10-09T21:45:30.702699Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:30.702721Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:30.702725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:30.702728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:30.702731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:30.702757Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:30.702765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:30.702773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:30.702848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.702850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.702852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.703190Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 536
2025-10-09T21:45:30.702854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:30.702952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:30.702955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.702970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.702980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.702995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:30.703017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341427682525447:5549]
2025-10-09T21:45:30.703020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341427682525447:5549]
2025-10-09T21:45:30.703022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341427682525447:5549]
2025-10-09T21:45:30.703031Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:30.703047Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:30.703057Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:30.703060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:30.703061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:30.703064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:30.703065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:30.703114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341428044338226:2432], now have 1 active actors on pipe
2025-10-09T21:45:30.703117Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341427682525447:5549]
2025-10-09T21:45:30.703126Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:30.703131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341428044338330:6079], now have 1 active actors on pipe
2025-10-09T21:45:30.703203Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:30.703231Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 536
2025-10-09T21:45:30.703445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:30.703457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:30.703487Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|56c02990-1b695991-790bfde1-56487c43_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:30.703499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:30.703509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:30.703512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:30.703524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:30.703540Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:30.703556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:30.703912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:30.703924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:30.703954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:30.704101Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 537 partition: 0 MaxSeqNo: 25 sessionId: src_id|56c02990-1b695991-790bfde1-56487c43_0
2025-10-09T21:45:30.704598Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd291934-103bfa23-19b742fd-afcdda24_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|56c02990-1b695991-790bfde1-56487c43_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:30.707408Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56c02990-1b695991-790bfde1-56487c43_0] Write session: try to update token
2025-10-09T21:45:30.707429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56c02990-1b695991-790bfde1-56487c43_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:30.728247Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 537 sessionId: src_id|56c02990-1b695991-790bfde1-56487c43_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:30.726092Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:30.726108Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:30.726111Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:30.726113Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:30.726124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341427682525447:5549]
2025-10-09T21:45:30.726129Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341427682525447:5549]
2025-10-09T21:45:30.726133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341427682525447:5549]
2025-10-09T21:45:30.726192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:30.726212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.726214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:30.726376Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:30.730219Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:30.730230Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:30.730235Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:30.731278Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:30.758137Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341428044338347:5852]
2025-10-09T21:45:30.758976Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 536, pipe: [1:7559341428044338347:5852]
2025-10-09T21:45:30.920440Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:30.920942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:30.920953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:30.921000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341428044338347:5852], now have 1 active actors on pipe
2025-10-09T21:45:30.921435Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:30.921807Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341428044338347:5852]
2025-10-09T21:45:30.921814Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:30.921851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:30.921937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 537 reason: status is not ok: tablet will be restarted right now sessionId: src_id|56c02990-1b695991-790bfde1-56487c43_0
2025-10-09T21:45:30.922151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 537 sessionId: src_id|56c02990-1b695991-790bfde1-56487c43_0 is DEAD
2025-10-09T21:45:30.924770Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56c02990-1b695991-790bfde1-56487c43_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:30.924786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56c02990-1b695991-790bfde1-56487c43_0] Write session will restart in 0.009178s
2025-10-09T21:45:30.924808Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56c02990-1b695991-790bfde1-56487c43_0] Write session: Do CDS request
2025-10-09T21:45:30.924813Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56c02990-1b695991-790bfde1-56487c43_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:30.935561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56c02990-1b695991-790bfde1-56487c43_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:30.935744Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:30.935763Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 538
2025-10-09T21:45:30.935925Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 538 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:30.935970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 538 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:30.935981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=538 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:30.935985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:30.936681Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 538 sessionId:  describe result for acl check
2025-10-09T21:45:30.936723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:30.936732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:30.936736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:30.936742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341428044338415:6083] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:30.937328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341428044338415:6083] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:30.939704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341428044338415:6083] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:30.944335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:30.951056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341428044338415:6083] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:30.951077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341428044338415:6083] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:30.951082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341428044338415:6083] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:30.951093Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 538 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:30.952146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:30.952619Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:30.952634Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:30.952652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:30.952738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:30.952795Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341427682525594:5561]
2025-10-09T21:45:30.953374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:30.954560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:30.954633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:30.955164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:30.955237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:30.955238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:30.955239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:30.955241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:30.955242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:30.955243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:30.955245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.955246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.955247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.955255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:30.955268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:30.955272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:30.955276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:30.955280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:30.955285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:30.955289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.955293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.955297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.955306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:30.955318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:30.955326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:30.955332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:30.955336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:30.955342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:30.955346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:30.955351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:30.955357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:30.955362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:30.955367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:30.955372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:30.955381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.955385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.955390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.955394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.955399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.955404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.955430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:30.965599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:30.966813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:30.966833Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:30.966836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:30.966840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:30.966849Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 537 [2:7559341427682525594:5561]
2025-10-09T21:45:30.966865Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:30.966885Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:30.966889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:30.966892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:30.966895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:30.966915Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:30.966925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:30.966933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:30.966999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.967002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.967004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.967008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:30.967087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:30.967091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.967106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:30.967120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:30.967139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:30.967161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341427682525588:5561]
2025-10-09T21:45:30.967164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341427682525588:5561]
2025-10-09T21:45:30.967167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341427682525588:5561]
2025-10-09T21:45:30.967177Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:30.967192Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:30.967210Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:30.967213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:30.967257Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341427682525588:5561]
2025-10-09T21:45:30.967260Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:30.968626Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:30.968634Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:30.968638Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:30.968646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:30.969115Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 537
2025-10-09T21:45:30.969245Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 537
2025-10-09T21:45:30.969645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341428044338458:6083], now have 1 active actors on pipe
2025-10-09T21:45:30.969655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341428044338356:2432], now have 1 active actors on pipe
2025-10-09T21:45:30.969673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:30.969679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:30.969715Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c02292ce-20059beb-e637385e-6ba1886e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:30.969731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:30.969737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:30.969741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:30.969754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:30.969770Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:30.969792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:30.970090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:30.970099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:30.970117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:30.970312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 538 partition: 0 MaxSeqNo: 25 sessionId: src_id|c02292ce-20059beb-e637385e-6ba1886e_0
2025-10-09T21:45:30.970711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56c02990-1b695991-790bfde1-56487c43_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c02292ce-20059beb-e637385e-6ba1886e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:30.973274Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c02292ce-20059beb-e637385e-6ba1886e_0] Write session: try to update token
2025-10-09T21:45:30.973296Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c02292ce-20059beb-e637385e-6ba1886e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:30.991201Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:30.991215Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:30.991217Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:30.991220Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:30.991229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341427682525588:5561]
2025-10-09T21:45:30.991234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341427682525588:5561]
2025-10-09T21:45:30.991238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341427682525588:5561]
2025-10-09T21:45:30.991285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:30.991292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:30.991295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:30.991424Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:30.991942Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:30.991949Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:30.991965Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:30.999241Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 538 sessionId: src_id|c02292ce-20059beb-e637385e-6ba1886e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:31.001796Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:31.021787Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341432339305784:5852]
2025-10-09T21:45:31.022356Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 537, pipe: [1:7559341432339305784:5852]
2025-10-09T21:45:31.217021Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:31.222576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:31.222597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:31.222648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341432339305784:5852], now have 1 active actors on pipe
2025-10-09T21:45:31.223086Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:31.223040Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:31.223097Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341432339305784:5852]
2025-10-09T21:45:31.223110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:31.223113Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:31.223188Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 538 reason: error at write: Disconnected sessionId: src_id|c02292ce-20059beb-e637385e-6ba1886e_0
2025-10-09T21:45:31.223386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 538 sessionId: src_id|c02292ce-20059beb-e637385e-6ba1886e_0 is DEAD
2025-10-09T21:45:31.226315Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c02292ce-20059beb-e637385e-6ba1886e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:31.226328Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c02292ce-20059beb-e637385e-6ba1886e_0] Write session will restart in 0.009362s
2025-10-09T21:45:31.226357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c02292ce-20059beb-e637385e-6ba1886e_0] Write session: Do CDS request
2025-10-09T21:45:31.226363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c02292ce-20059beb-e637385e-6ba1886e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:31.238526Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c02292ce-20059beb-e637385e-6ba1886e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:31.238872Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:31.238891Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 539
2025-10-09T21:45:31.239026Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 539 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:31.239060Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 539 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:31.239065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=539 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:31.239069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:31.239528Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 539 sessionId:  describe result for acl check
2025-10-09T21:45:31.239567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:31.239569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:31.239571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:31.239580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341432339305818:6095] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:31.240133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341432339305818:6095] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:31.249601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341432339305818:6095] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:31.260318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341432339305818:6095] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:31.260338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341432339305818:6095] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:31.260342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341432339305818:6095] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:31.260353Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 539 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:31.270366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:31.274816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:31.275452Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:31.275465Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:31.275483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:31.275571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:31.275620Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341431977493011:5566]
2025-10-09T21:45:31.276582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:31.281477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:31.281585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:31.281966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:31.282064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:31.282074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:31.282076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:31.282077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:31.282079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:31.282080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:31.282082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.282083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.282084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.282092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:31.282103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:31.282116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:31.282120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:31.282124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:31.282128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:31.282131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.282135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.282139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.282148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:31.282157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:31.282186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:31.282191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:31.282196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:31.282202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:31.282207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:31.282212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:31.282228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:31.282235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:31.282239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:31.282244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:31.282249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.282254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.282262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.282268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.282276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.282281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.282311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:31.296565Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 538
2025-10-09T21:45:31.296598Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 538
2025-10-09T21:45:31.294065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:31.295169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:31.295176Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:31.295179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:31.295184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:31.295194Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 538 [2:7559341431977493011:5566]
2025-10-09T21:45:31.295211Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:31.295232Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:31.295236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:31.295240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:31.295243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:31.295263Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:31.295269Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:31.295278Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:31.295343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.295346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.295348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.295352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:31.295437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:31.295440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.295454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.295469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.295488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:31.295509Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341431977493005:5566]
2025-10-09T21:45:31.295513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341431977493005:5566]
2025-10-09T21:45:31.295515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341431977493005:5566]
2025-10-09T21:45:31.295531Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:31.295547Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:31.295564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:31.295568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:31.295610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341431977493005:5566]
2025-10-09T21:45:31.295613Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:31.295724Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:31.295726Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:31.295729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:31.295735Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:31.300744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341432339305907:6095], now have 1 active actors on pipe
2025-10-09T21:45:31.300761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341432339305803:2432], now have 1 active actors on pipe
2025-10-09T21:45:31.300780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:31.300787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:31.300832Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|47ade955-abeebfe7-6342e95-d5e6989d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:31.300851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:31.300857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:31.300860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:31.300874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:31.300893Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:31.300916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:31.301260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:31.301272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:31.301286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:31.301412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 539 partition: 0 MaxSeqNo: 25 sessionId: src_id|47ade955-abeebfe7-6342e95-d5e6989d_0
2025-10-09T21:45:31.301892Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c02292ce-20059beb-e637385e-6ba1886e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|47ade955-abeebfe7-6342e95-d5e6989d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:31.304367Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|47ade955-abeebfe7-6342e95-d5e6989d_0] Write session: try to update token
2025-10-09T21:45:31.304389Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|47ade955-abeebfe7-6342e95-d5e6989d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:31.323444Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341432339305920:5852]
2025-10-09T21:45:31.324432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341432339305920:5852], now have 1 active actors on pipe
2025-10-09T21:45:31.324546Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 538, pipe: [1:7559341432339305920:5852]
2025-10-09T21:45:31.332343Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 539 sessionId: src_id|47ade955-abeebfe7-6342e95-d5e6989d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:31.334667Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:31.337343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:31.337364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:31.337409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:31.339221Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:31.340536Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:31.340553Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:31.340560Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:31.340562Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:31.340571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341431977493005:5566]
2025-10-09T21:45:31.340575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341431977493005:5566]
2025-10-09T21:45:31.340578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341431977493005:5566]
2025-10-09T21:45:31.340619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:31.340626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.340629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:31.340765Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:31.343946Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:31.343953Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:31.343958Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:31.559617Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:31.569320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:31.569346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:31.569515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:31.569571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:31.569627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:31.569682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:31.569724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:31.569729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:31.569794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:31.569849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:31.569925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:31.569985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:31.570021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:31.570024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:31.570107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:31.570163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:31.570233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:31.570291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:31.570334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:31.570337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:31.570410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:31.570466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:31.570527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:31.570582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:31.570621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:31.570623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:31.570686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:31.570740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:31.570797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:31.570857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:31.570896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:31.570899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:31.570962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:31.571017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:31.571074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:31.571129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:31.571171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:31.571174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:31.571236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:31.571311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:31.571372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:31.571427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:31.571468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:31.571470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:31.571530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:31.571581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:31.571638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:31.571693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:31.571733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:31.571736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:31.571804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:31.571858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:31.571912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:31.571969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:31.572009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:31.572013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:31.572076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:31.572131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:31.572189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:31.572246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:31.572286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:31.572289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:31.573414Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:31.574160Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:31.574232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 539 reason: status is not ok: tablet will be restarted right now sessionId: src_id|47ade955-abeebfe7-6342e95-d5e6989d_0
2025-10-09T21:45:31.574432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 539 sessionId: src_id|47ade955-abeebfe7-6342e95-d5e6989d_0 is DEAD
2025-10-09T21:45:31.574571Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341432339305920:5852]
2025-10-09T21:45:31.574585Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:31.578583Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|47ade955-abeebfe7-6342e95-d5e6989d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:31.578608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|47ade955-abeebfe7-6342e95-d5e6989d_0] Write session will restart in 0.009075s
2025-10-09T21:45:31.578634Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|47ade955-abeebfe7-6342e95-d5e6989d_0] Write session: Do CDS request
2025-10-09T21:45:31.578642Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|47ade955-abeebfe7-6342e95-d5e6989d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:31.593205Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|47ade955-abeebfe7-6342e95-d5e6989d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:31.593651Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:31.593680Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 540
2025-10-09T21:45:31.594481Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 540 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:31.594522Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 540 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:31.594529Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=540 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:31.594533Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:31.595102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 540 sessionId:  describe result for acl check
2025-10-09T21:45:31.595143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:31.595145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:31.595148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:31.595154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341432339306000:6100] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:31.595669Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341432339306000:6100] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:31.600457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:31.600177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341432339306000:6100] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:31.605999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:31.606311Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:31.606326Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:31.606340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:31.606416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:31.606467Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341431977493124:5569]
2025-10-09T21:45:31.607047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:31.607973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:31.608039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:31.608377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:31.608442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:31.608449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:31.608451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:31.608452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:31.608454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:31.608455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:31.608456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.608458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.608459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.608465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:31.608479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:31.608485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:31.608490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:31.608493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:31.608498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:31.608502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.608506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.608510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.608518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:31.608531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:31.608541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:31.608546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:31.608554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:31.608560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:31.608568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:31.608572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:31.608577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:31.608585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:31.608589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:31.608593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:31.608599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.608608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.608612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.608616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.608621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.608626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.608654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:31.609626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341432339306000:6100] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:31.609645Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341432339306000:6100] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:31.609649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341432339306000:6100] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:31.609659Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 540 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:31.617778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:31.619462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:31.619469Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:31.619471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:31.619476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:31.619484Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 539 [2:7559341431977493124:5569]
2025-10-09T21:45:31.619503Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:31.619525Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:31.619528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:31.619531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:31.619534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:31.619551Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:31.619562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:31.619569Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:31.619632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.619634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.619635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.619638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:31.619717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:31.619720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.619733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.619745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.619762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:31.619782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341431977493118:5569]
2025-10-09T21:45:31.619784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341431977493118:5569]
2025-10-09T21:45:31.619786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341431977493118:5569]
2025-10-09T21:45:31.619800Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:31.619816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:31.619829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:31.619832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:31.619879Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341431977493118:5569]
2025-10-09T21:45:31.619882Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:31.620283Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:31.620291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:31.620293Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:31.620299Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:31.621217Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 539
2025-10-09T21:45:31.621256Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 539
2025-10-09T21:45:31.623140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341432339306041:6100], now have 1 active actors on pipe
2025-10-09T21:45:31.623152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341432339305927:2432], now have 1 active actors on pipe
2025-10-09T21:45:31.623173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:31.623181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:31.623219Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|64297608-a2515251-a6901c11-896c2c6e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:31.623235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:31.623241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:31.623244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:31.623257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:31.623274Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:31.623296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:31.623637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:31.623643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:31.623657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:31.623788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 540 partition: 0 MaxSeqNo: 25 sessionId: src_id|64297608-a2515251-a6901c11-896c2c6e_0
2025-10-09T21:45:31.624236Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|47ade955-abeebfe7-6342e95-d5e6989d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|64297608-a2515251-a6901c11-896c2c6e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:31.626977Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|64297608-a2515251-a6901c11-896c2c6e_0] Write session: try to update token
2025-10-09T21:45:31.627000Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|64297608-a2515251-a6901c11-896c2c6e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:31.642757Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:31.642771Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:31.642775Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:31.642776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:31.642788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341431977493118:5569]
2025-10-09T21:45:31.642792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341431977493118:5569]
2025-10-09T21:45:31.642796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341431977493118:5569]
2025-10-09T21:45:31.642845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:31.642853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.642855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:31.643050Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:31.645876Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:31.645886Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:31.645890Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:31.657380Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 540 sessionId: src_id|64297608-a2515251-a6901c11-896c2c6e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:31.659771Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:31.674322Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341432339306052:5852]
2025-10-09T21:45:31.674993Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 539, pipe: [1:7559341432339306052:5852]
2025-10-09T21:45:31.831973Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:31.832644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:31.832656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:31.832706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341432339306052:5852], now have 1 active actors on pipe
2025-10-09T21:45:31.837401Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:31.837651Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341432339306052:5852]
2025-10-09T21:45:31.837698Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:31.837701Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:31.837767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 540 reason: error at write: Disconnected sessionId: src_id|64297608-a2515251-a6901c11-896c2c6e_0
2025-10-09T21:45:31.837961Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 540 sessionId: src_id|64297608-a2515251-a6901c11-896c2c6e_0 is DEAD
2025-10-09T21:45:31.837998Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:31.838489Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|64297608-a2515251-a6901c11-896c2c6e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:31.838501Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|64297608-a2515251-a6901c11-896c2c6e_0] Write session will restart in 0.007705s
2025-10-09T21:45:31.838540Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|64297608-a2515251-a6901c11-896c2c6e_0] Write session: Do CDS request
2025-10-09T21:45:31.838548Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|64297608-a2515251-a6901c11-896c2c6e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:31.850481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|64297608-a2515251-a6901c11-896c2c6e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:31.854418Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:31.854439Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 541
2025-10-09T21:45:31.858264Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 541 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:31.858320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 541 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:31.858326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=541 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:31.858331Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:31.858851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 541 sessionId:  describe result for acl check
2025-10-09T21:45:31.858889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:31.858891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:31.858893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:31.858900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341432339306070:6106] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:31.859480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341432339306070:6106] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:31.869489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341432339306070:6106] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:31.874420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:31.885315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:31.885584Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:31.885589Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:31.885603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:31.885670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:31.885710Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341431977493249:5571]
2025-10-09T21:45:31.886353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:31.888734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:31.888791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:31.889151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:31.889193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:31.889198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:31.889199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:31.889200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:31.889201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:31.889203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:31.889204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.889205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.889207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.889213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:31.887862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341432339306070:6106] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:31.887883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341432339306070:6106] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:31.887887Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341432339306070:6106] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:31.887898Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 541 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:31.889226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:31.889230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:31.889235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:31.889239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:31.889244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:31.889249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.889253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.889257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.889264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:31.889273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:31.889279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:31.889285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:31.889290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:31.889295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:31.889300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:31.889305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:31.889310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:31.889317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:31.889321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:31.889327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:31.889332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.889337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.889345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.889364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.889370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.889375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.889409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:31.900686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:31.902427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:31.902437Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:31.902440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:31.902446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:31.902455Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 540 [2:7559341431977493249:5571]
2025-10-09T21:45:31.902478Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:31.902500Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:31.902504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:31.902508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:31.902512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:31.902532Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:31.902539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:31.902548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:31.902617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.902619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.902621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.902624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:31.902719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:31.902722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.902739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:31.902753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:31.902773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:31.902796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341431977493234:5571]
2025-10-09T21:45:31.902798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341431977493234:5571]
2025-10-09T21:45:31.902800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341431977493234:5571]
2025-10-09T21:45:31.902816Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:31.902831Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:31.902848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:31.902851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:31.902900Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341431977493234:5571]
2025-10-09T21:45:31.902903Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:31.904671Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 540
2025-10-09T21:45:31.904716Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 540
2025-10-09T21:45:31.903278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:31.903285Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:31.903287Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:31.903294Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:31.913734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341432339306166:6106], now have 1 active actors on pipe
2025-10-09T21:45:31.913751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341432339306055:2432], now have 1 active actors on pipe
2025-10-09T21:45:31.913783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:31.913791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:31.913837Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2dceceb0-26404f2-1fa5207a-dda5b399_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:31.913851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:31.913857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:31.913860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:31.913874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:31.913897Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:31.913920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:31.914389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:31.914398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:31.914417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:31.914580Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 541 partition: 0 MaxSeqNo: 25 sessionId: src_id|2dceceb0-26404f2-1fa5207a-dda5b399_0
2025-10-09T21:45:31.915070Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|64297608-a2515251-a6901c11-896c2c6e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2dceceb0-26404f2-1fa5207a-dda5b399_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:31.917649Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2dceceb0-26404f2-1fa5207a-dda5b399_0] Write session: try to update token
2025-10-09T21:45:31.917668Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2dceceb0-26404f2-1fa5207a-dda5b399_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:31.937805Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341432339306174:5852]
2025-10-09T21:45:31.938684Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 540, pipe: [1:7559341432339306174:5852]
2025-10-09T21:45:31.938526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341432339306174:5852], now have 1 active actors on pipe
2025-10-09T21:45:31.963925Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 541 sessionId: src_id|2dceceb0-26404f2-1fa5207a-dda5b399_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:31.966562Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:31.966866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:31.966886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:31.966933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:31.967081Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:31.975431Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:31.975451Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:31.975453Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:31.975455Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:31.975467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341431977493234:5571]
2025-10-09T21:45:31.975472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341431977493234:5571]
2025-10-09T21:45:31.975476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341431977493234:5571]
2025-10-09T21:45:31.975527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:31.975533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:31.975536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:31.975680Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:31.978205Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:31.978218Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:31.978222Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:32.215831Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:32.228087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:32.228117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:32.228293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:32.228360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:32.228439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:32.228512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:32.230408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:32.230424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:32.230515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:32.230600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:32.230666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:32.230729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:32.230787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:32.230790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:32.230867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:32.230933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:32.231000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:32.231064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:32.231113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:32.231117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:32.231196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:32.231268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:32.231337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:32.233906Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341432339306174:5852]
2025-10-09T21:45:32.233950Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:32.233952Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:32.233982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 541 reason: error at write: Disconnected sessionId: src_id|2dceceb0-26404f2-1fa5207a-dda5b399_0
2025-10-09T21:45:32.234219Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 541 sessionId: src_id|2dceceb0-26404f2-1fa5207a-dda5b399_0 is DEAD
2025-10-09T21:45:32.234290Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:32.234370Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2dceceb0-26404f2-1fa5207a-dda5b399_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:32.234382Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2dceceb0-26404f2-1fa5207a-dda5b399_0] Write session will restart in 0.007250s
2025-10-09T21:45:32.234408Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2dceceb0-26404f2-1fa5207a-dda5b399_0] Write session: Do CDS request
2025-10-09T21:45:32.234415Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2dceceb0-26404f2-1fa5207a-dda5b399_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:32.231409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:32.231464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:32.231467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:32.231536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:32.231596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:32.231655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:32.231730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:32.231774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:32.231777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:32.231846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:32.231899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:32.231954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:32.232013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:32.232064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:32.232067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:32.232132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:32.232185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:32.232247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:32.232307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:32.232352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:32.232354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:32.232423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:32.232478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:32.232532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:32.232589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:32.232628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:32.232631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:32.232693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:32.232749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:32.232811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:32.232872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:32.232912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:32.232915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:32.232978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:32.233035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:32.233096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:32.233155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:32.233196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:32.233198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:32.234562Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:32.243397Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2dceceb0-26404f2-1fa5207a-dda5b399_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:32.244165Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:32.244185Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 542
2025-10-09T21:45:32.244511Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 542 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:32.244570Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 542 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:32.244577Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=542 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:32.244582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:32.245102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 542 sessionId:  describe result for acl check
2025-10-09T21:45:32.245141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:32.245143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:32.245144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:32.245152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341436634273518:6118] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:32.245746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341436634273518:6118] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:32.251452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341436634273518:6118] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:32.258854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341436634273518:6118] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:32.258874Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341436634273518:6118] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:32.258878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341436634273518:6118] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:32.258888Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 542 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:32.285914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:32.287986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:32.288248Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:32.288254Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:32.288269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:32.288347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:32.288388Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341436272460689:5586]
2025-10-09T21:45:32.288977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:32.289430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:32.289482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:32.289808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:32.289880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:32.289882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:32.289883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:32.289884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:32.289885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:32.289886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:32.289888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:32.289889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:32.289890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:32.289895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:32.289906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:32.289909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:32.289913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:32.289918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:32.289922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:32.289926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:32.289929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:32.289933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:32.289941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:32.289950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:32.289956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:32.289962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:32.289967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:32.289972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:32.289976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:32.289982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:32.289986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:32.289991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:32.289995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:32.290000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:32.290005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:32.290009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:32.290014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:32.290020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:32.290043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:32.290049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:32.290076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:32.298674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:32.299928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:32.299936Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:32.299939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:32.299944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:32.299952Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 541 [2:7559341436272460689:5586]
2025-10-09T21:45:32.299971Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:32.299994Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:32.299998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:32.300002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:32.300005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:32.300026Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:32.300033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:32.300041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:32.300114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:32.300116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:32.300118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:32.300122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:32.300214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:32.300217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:32.300232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:32.300244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:32.300260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:32.300282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341436272460684:5586]
2025-10-09T21:45:32.300285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341436272460684:5586]
2025-10-09T21:45:32.300287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341436272460684:5586]
2025-10-09T21:45:32.300305Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:32.300323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:32.300337Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:32.300340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:32.300384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341436272460684:5586]
2025-10-09T21:45:32.300387Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:32.300444Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:32.300447Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:32.300449Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:32.300455Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:32.301273Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 541
2025-10-09T21:45:32.301323Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 541
2025-10-09T21:45:32.302343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341436634273511:2432], now have 1 active actors on pipe
2025-10-09T21:45:32.302356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341436634273570:6118], now have 1 active actors on pipe
2025-10-09T21:45:32.302381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:32.302403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:32.302446Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3736a39f-d7b365ae-2d4f9c9f-208ae102_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:32.302468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:32.302474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:32.302477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:32.302492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:32.302511Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:32.302539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:32.303597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:32.303611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:32.303637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:32.305425Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 542 partition: 0 MaxSeqNo: 25 sessionId: src_id|3736a39f-d7b365ae-2d4f9c9f-208ae102_0
2025-10-09T21:45:32.310273Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2dceceb0-26404f2-1fa5207a-dda5b399_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3736a39f-d7b365ae-2d4f9c9f-208ae102_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:32.313014Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3736a39f-d7b365ae-2d4f9c9f-208ae102_0] Write session: try to update token
2025-10-09T21:45:32.313043Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3736a39f-d7b365ae-2d4f9c9f-208ae102_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:32.333318Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:32.333336Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:32.333338Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:32.333341Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:32.333351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341436272460684:5586]
2025-10-09T21:45:32.333355Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341436272460684:5586]
2025-10-09T21:45:32.333359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341436272460684:5586]
2025-10-09T21:45:32.333407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:32.333415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:32.333417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:32.333555Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:32.334845Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341436634273632:5852]
2025-10-09T21:45:32.335736Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:32.335755Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:32.335759Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:32.335993Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 541, pipe: [1:7559341436634273632:5852]
2025-10-09T21:45:32.368216Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 542 sessionId: src_id|3736a39f-d7b365ae-2d4f9c9f-208ae102_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:32.370890Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:32.595125Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:32.595714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341436634273632:5852], now have 1 active actors on pipe
2025-10-09T21:45:32.595779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:32.595792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:32.597273Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:32.597721Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:32.597783Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341436634273632:5852]
2025-10-09T21:45:32.597809Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:32.597816Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:32.597869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 542 reason: error at write: Disconnected sessionId: src_id|3736a39f-d7b365ae-2d4f9c9f-208ae102_0
2025-10-09T21:45:32.598238Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3736a39f-d7b365ae-2d4f9c9f-208ae102_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:32.598252Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3736a39f-d7b365ae-2d4f9c9f-208ae102_0] Write session will restart in 0.005353s
2025-10-09T21:45:32.598276Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3736a39f-d7b365ae-2d4f9c9f-208ae102_0] Write session: Do CDS request
2025-10-09T21:45:32.598284Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3736a39f-d7b365ae-2d4f9c9f-208ae102_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:32.600793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 542 sessionId: src_id|3736a39f-d7b365ae-2d4f9c9f-208ae102_0 is DEAD
2025-10-09T21:45:32.606548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3736a39f-d7b365ae-2d4f9c9f-208ae102_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:32.606870Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:32.606888Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 543
2025-10-09T21:45:32.607064Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 543 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:32.607095Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 543 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:32.607101Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=543 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:32.607105Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:32.609099Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 543 sessionId:  describe result for acl check
2025-10-09T21:45:32.609144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:32.609146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:32.609148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:32.609153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341436634273648:6123] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:32.609685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341436634273648:6123] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:32.625835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341436634273648:6123] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:32.631799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341436634273648:6123] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:32.631817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341436634273648:6123] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:32.631821Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341436634273648:6123] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:32.631831Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 543 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:32.636663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:32.644013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:32.644643Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:32.644658Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:32.644676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:32.644763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:32.644811Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341436272460806:5589]
2025-10-09T21:45:32.645417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:32.650686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:32.650777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:32.651182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:32.651235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:32.651237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:32.651238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:32.651240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:32.651241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:32.651242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:32.651244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:32.651245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:32.651246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:32.651253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:32.651266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:32.651270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:32.651274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:32.651278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:32.651283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:32.651286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:32.651290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:32.651294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:32.651302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:32.651311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:32.651318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:32.651323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:32.651329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:32.651335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:32.651339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:32.651345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:32.651349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:32.651357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:32.651362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:32.651368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:32.651372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:32.651378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:32.651382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:32.651387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:32.651392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:32.651397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:32.651433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:32.678641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:32.679852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:32.679860Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:32.679863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:32.679868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:32.679885Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 542 [2:7559341436272460806:5589]
2025-10-09T21:45:32.679905Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:32.679929Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:32.679932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:32.679936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:32.679938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:32.679958Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:32.679965Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:32.679974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:32.680440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:32.680443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:32.680445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:32.680448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:32.680532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:32.680534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:32.680549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:32.680558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:32.680575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:32.680596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341436272460800:5589]
2025-10-09T21:45:32.680598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341436272460800:5589]
2025-10-09T21:45:32.680600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341436272460800:5589]
2025-10-09T21:45:32.680614Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:32.680629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:32.680642Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:32.680645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:32.680687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341436272460800:5589]
2025-10-09T21:45:32.680689Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:32.681784Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 542
2025-10-09T21:45:32.681819Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 542
2025-10-09T21:45:32.681403Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:32.681411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:32.681413Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:32.681421Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:32.682318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341436634273746:6123], now have 1 active actors on pipe
2025-10-09T21:45:32.682327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341436634273639:2432], now have 1 active actors on pipe
2025-10-09T21:45:32.682342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:32.682349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:32.682383Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|da471604-b37e1f75-ec4e330e-60eb5c46_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:32.682394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:32.682401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:32.682404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:32.682417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:32.682432Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:32.682453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:32.683100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:32.683108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:32.683127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:32.683283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 543 partition: 0 MaxSeqNo: 25 sessionId: src_id|da471604-b37e1f75-ec4e330e-60eb5c46_0
2025-10-09T21:45:32.683677Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3736a39f-d7b365ae-2d4f9c9f-208ae102_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|da471604-b37e1f75-ec4e330e-60eb5c46_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:32.686312Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da471604-b37e1f75-ec4e330e-60eb5c46_0] Write session: try to update token
2025-10-09T21:45:32.686330Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da471604-b37e1f75-ec4e330e-60eb5c46_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:32.694626Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341436634273759:5852]
2025-10-09T21:45:32.696421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341436634273759:5852], now have 1 active actors on pipe
2025-10-09T21:45:32.696547Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 542, pipe: [1:7559341436634273759:5852]
2025-10-09T21:45:32.721088Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 543 sessionId: src_id|da471604-b37e1f75-ec4e330e-60eb5c46_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:32.723558Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:32.724634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:32.724460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:32.724484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:32.724528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:32.731575Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:32.731597Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:32.731600Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:32.731602Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:32.731612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341436272460800:5589]
2025-10-09T21:45:32.731616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341436272460800:5589]
2025-10-09T21:45:32.731620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341436272460800:5589]
2025-10-09T21:45:32.731674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:32.731675Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:32.731681Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:32.731682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:32.731685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:32.731686Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:32.731845Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:32.988542Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:32.997651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:32.997681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:32.997857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:32.997935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:32.998026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:32.998669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:32.998714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:32.998720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:32.998798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:32.998879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:32.998947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:32.999019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:32.999235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:32.999238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:32.999321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:32.999384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:32.999447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:32.999514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:32.999560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:32.999562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:32.999636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:32.999701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:32.999764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:32.999829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:32.999871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:32.999873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:32.999943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:33.000012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:33.000077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:33.000140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:33.000182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:33.000185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:33.000268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:33.000334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:33.000401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:33.000464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:33.000677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:33.000680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:33.000747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:33.000809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:33.000871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:33.000937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:33.001202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:33.001205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:33.001271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:33.001333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:33.001397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:33.001467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:33.001744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:33.001747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:33.001814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:33.001873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:33.001937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:33.002003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:33.002765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:33.002774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:33.005073Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:33.005125Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:33.005128Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:33.005158Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 543 reason: error at write: Disconnected sessionId: src_id|da471604-b37e1f75-ec4e330e-60eb5c46_0
2025-10-09T21:45:33.005358Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 543 sessionId: src_id|da471604-b37e1f75-ec4e330e-60eb5c46_0 is DEAD
2025-10-09T21:45:33.005481Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341436634273759:5852]
2025-10-09T21:45:33.005819Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da471604-b37e1f75-ec4e330e-60eb5c46_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:33.005829Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da471604-b37e1f75-ec4e330e-60eb5c46_0] Write session will restart in 0.006904s
2025-10-09T21:45:33.005853Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da471604-b37e1f75-ec4e330e-60eb5c46_0] Write session: Do CDS request
2025-10-09T21:45:33.005859Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da471604-b37e1f75-ec4e330e-60eb5c46_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:33.002847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:33.002918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:33.002992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:33.003067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:33.003112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:33.003115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:33.006386Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:33.013931Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da471604-b37e1f75-ec4e330e-60eb5c46_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:33.014915Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:33.014930Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 544
2025-10-09T21:45:33.015639Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 544 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:33.015672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 544 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:33.015678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=544 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:33.015682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:33.016208Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 544 sessionId:  describe result for acl check
2025-10-09T21:45:33.016238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:33.016240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:33.016241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:33.016247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341440929241076:6127] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:33.016752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341440929241076:6127] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:33.028955Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341440929241076:6127] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:33.039476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341440929241076:6127] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:33.039493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341440929241076:6127] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:33.039497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341440929241076:6127] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:33.039507Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 544 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:33.063467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:33.070663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:33.071077Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:33.071081Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:33.071094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:33.071149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:33.071186Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341440567428231:5595]
2025-10-09T21:45:33.071743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:33.078099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:33.078203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:33.078562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:33.078606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:33.078607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:33.078608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:33.078609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:33.078611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:33.078612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:33.078613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.078614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.078616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.078622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:33.078632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:33.078635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:33.078641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:33.078644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:33.078647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:33.078651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.078654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.078657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.078664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:33.078673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:33.078678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:33.078682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:33.078686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:33.078690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:33.078697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:33.078702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:33.078706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:33.078710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:33.078714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:33.078718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:33.078723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.078728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.078731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.078735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.078739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.078743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.078768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:33.094917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:33.096557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:33.096563Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:33.096565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:33.096570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:33.096577Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 543 [2:7559341440567428231:5595]
2025-10-09T21:45:33.096596Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:33.096618Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:33.096621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:33.096625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:33.096627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:33.096645Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:33.096651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:33.096659Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:33.096726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.096728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.096730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.096733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:33.096814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:33.096817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.096829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.096842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.096860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:33.096881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341440567428225:5595]
2025-10-09T21:45:33.096883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341440567428225:5595]
2025-10-09T21:45:33.096885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341440567428225:5595]
2025-10-09T21:45:33.096900Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:33.096915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:33.096930Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:33.096933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:33.096976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341440567428225:5595]
2025-10-09T21:45:33.096978Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:33.098279Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 543
2025-10-09T21:45:33.098314Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 543
2025-10-09T21:45:33.097361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:33.097366Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:33.097368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:33.097374Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:33.099245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341440929241136:6127], now have 1 active actors on pipe
2025-10-09T21:45:33.099253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341440929241069:2432], now have 1 active actors on pipe
2025-10-09T21:45:33.099275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:33.099281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:33.099317Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ad2946de-6ec0bd4a-43bb91a3-6d1dbb2a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:33.099328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:33.099332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:33.099335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:33.099347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:33.099362Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:33.099383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:33.099703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:33.099715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:33.099727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:33.099877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 544 partition: 0 MaxSeqNo: 25 sessionId: src_id|ad2946de-6ec0bd4a-43bb91a3-6d1dbb2a_0
2025-10-09T21:45:33.100375Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da471604-b37e1f75-ec4e330e-60eb5c46_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ad2946de-6ec0bd4a-43bb91a3-6d1dbb2a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:33.102375Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341440929241193:5852]
2025-10-09T21:45:33.103116Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad2946de-6ec0bd4a-43bb91a3-6d1dbb2a_0] Write session: try to update token
2025-10-09T21:45:33.103135Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad2946de-6ec0bd4a-43bb91a3-6d1dbb2a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:33.106864Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 543, pipe: [1:7559341440929241193:5852]
2025-10-09T21:45:33.110422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341440929241193:5852], now have 1 active actors on pipe
2025-10-09T21:45:33.154958Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:33.154974Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:33.154976Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:33.154978Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:33.154986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341440567428225:5595]
2025-10-09T21:45:33.154993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341440567428225:5595]
2025-10-09T21:45:33.154996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341440567428225:5595]
2025-10-09T21:45:33.155039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:33.155045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.155048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:33.155223Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:33.157660Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:33.157671Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:33.157674Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:33.190401Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 544 sessionId: src_id|ad2946de-6ec0bd4a-43bb91a3-6d1dbb2a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:33.193087Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:33.337616Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:33.338072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:33.338083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:33.342562Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:33.342661Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:33.342685Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:33.342765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 544 reason: error at write: Disconnected sessionId: src_id|ad2946de-6ec0bd4a-43bb91a3-6d1dbb2a_0
2025-10-09T21:45:33.342959Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 544 sessionId: src_id|ad2946de-6ec0bd4a-43bb91a3-6d1dbb2a_0 is DEAD
2025-10-09T21:45:33.343101Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:33.343591Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341440929241193:5852]
2025-10-09T21:45:33.346279Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad2946de-6ec0bd4a-43bb91a3-6d1dbb2a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:33.346287Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad2946de-6ec0bd4a-43bb91a3-6d1dbb2a_0] Write session will restart in 0.006968s
2025-10-09T21:45:33.346311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad2946de-6ec0bd4a-43bb91a3-6d1dbb2a_0] Write session: Do CDS request
2025-10-09T21:45:33.346320Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad2946de-6ec0bd4a-43bb91a3-6d1dbb2a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:33.354511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad2946de-6ec0bd4a-43bb91a3-6d1dbb2a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:33.354729Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:33.354745Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 545
2025-10-09T21:45:33.354919Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 545 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:33.354963Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 545 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:33.354969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=545 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:33.354973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:33.356350Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 545 sessionId:  describe result for acl check
2025-10-09T21:45:33.356390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:33.356392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:33.356394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:33.356400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341440929241251:6139] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:33.356960Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341440929241251:6139] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:33.369741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341440929241251:6139] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:33.373697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341440929241251:6139] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:33.373724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341440929241251:6139] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:33.373728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341440929241251:6139] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:33.373737Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 545 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:33.374781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:33.379797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:33.380017Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:33.380022Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:33.380036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:33.380100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:33.380210Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341440567428377:5607]
2025-10-09T21:45:33.380799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:33.381335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:33.381382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:33.381673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:33.381720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:33.381721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:33.381722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:33.381723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:33.381724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:33.381725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:33.381726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.381727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.381728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.381733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:33.381743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:33.381746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:33.381749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:33.381753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:33.381756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:33.381759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.381762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.381765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.381772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:33.381780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:33.381785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:33.381790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:33.381793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:33.381798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:33.381802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:33.381806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:33.381810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:33.381815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:33.381819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:33.381822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:33.381826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.381830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.381834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.381838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.381842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.381845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.381870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:33.392702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:33.393834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:33.393853Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:33.393856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:33.393861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:33.393869Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 544 [2:7559341440567428377:5607]
2025-10-09T21:45:33.393885Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:33.393904Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:33.393915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:33.393918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:33.393921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:33.393940Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:33.393947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:33.393955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:33.394021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.394049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.394051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.394055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:33.394131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:33.394135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.394148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.394158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.394189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:33.394212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341440567428372:5607]
2025-10-09T21:45:33.394215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341440567428372:5607]
2025-10-09T21:45:33.394218Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341440567428372:5607]
2025-10-09T21:45:33.394234Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:33.394252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:33.394267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:33.394271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:33.394314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341440567428372:5607]
2025-10-09T21:45:33.394317Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:33.394922Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:33.394932Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:33.394935Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:33.394943Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:33.395246Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 544
2025-10-09T21:45:33.395256Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 544
2025-10-09T21:45:33.396217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341440929241335:6139], now have 1 active actors on pipe
2025-10-09T21:45:33.396229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341440929241227:2432], now have 1 active actors on pipe
2025-10-09T21:45:33.396254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:33.396260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:33.396292Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|756083a0-4b5b07f9-939f6755-18f4c86c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:33.396303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:33.396308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:33.396310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:33.396322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:33.396339Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:33.396361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:33.397477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:33.397492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:33.397535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:33.398140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad2946de-6ec0bd4a-43bb91a3-6d1dbb2a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|756083a0-4b5b07f9-939f6755-18f4c86c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:33.397660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 545 partition: 0 MaxSeqNo: 25 sessionId: src_id|756083a0-4b5b07f9-939f6755-18f4c86c_0
2025-10-09T21:45:33.400811Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|756083a0-4b5b07f9-939f6755-18f4c86c_0] Write session: try to update token
2025-10-09T21:45:33.400829Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|756083a0-4b5b07f9-939f6755-18f4c86c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:33.416422Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:33.416440Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:33.416442Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:33.416444Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:33.416454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341440567428372:5607]
2025-10-09T21:45:33.416461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341440567428372:5607]
2025-10-09T21:45:33.416466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341440567428372:5607]
2025-10-09T21:45:33.416517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:33.416524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.416527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:33.416671Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:33.418250Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:33.418262Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:33.418266Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:33.423875Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 545 sessionId: src_id|756083a0-4b5b07f9-939f6755-18f4c86c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:33.426537Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:33.446135Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341440929241349:5852]
2025-10-09T21:45:33.450113Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 544, pipe: [1:7559341440929241349:5852]
2025-10-09T21:45:33.589457Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:33.590620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:33.590637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:33.590661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341440929241349:5852], now have 1 active actors on pipe
2025-10-09T21:45:33.592260Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:33.592262Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:33.592277Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:33.592314Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341440929241349:5852]
2025-10-09T21:45:33.592351Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 545 reason: error at write: Disconnected sessionId: src_id|756083a0-4b5b07f9-939f6755-18f4c86c_0
2025-10-09T21:45:33.592581Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 545 sessionId: src_id|756083a0-4b5b07f9-939f6755-18f4c86c_0 is DEAD
2025-10-09T21:45:33.592283Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:33.592759Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|756083a0-4b5b07f9-939f6755-18f4c86c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:33.592772Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|756083a0-4b5b07f9-939f6755-18f4c86c_0] Write session will restart in 0.008873s
2025-10-09T21:45:33.592813Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|756083a0-4b5b07f9-939f6755-18f4c86c_0] Write session: Do CDS request
2025-10-09T21:45:33.592820Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|756083a0-4b5b07f9-939f6755-18f4c86c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:33.603025Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|756083a0-4b5b07f9-939f6755-18f4c86c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:33.603160Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:33.603179Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 546
2025-10-09T21:45:33.603382Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 546 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:33.603418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 546 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:33.603424Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=546 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:33.603428Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:33.603897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 546 sessionId:  describe result for acl check
2025-10-09T21:45:33.603939Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:33.603946Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:33.603948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:33.603955Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341440929241362:6144] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:33.604538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341440929241362:6144] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:33.607280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341440929241362:6144] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:33.611693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341440929241362:6144] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:33.611711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341440929241362:6144] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:33.611715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341440929241362:6144] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:33.611734Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 546 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:33.615592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:33.620302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:33.620556Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:33.620561Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:33.620574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:33.620633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:33.620685Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341440567428495:5609]
2025-10-09T21:45:33.621281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:33.621655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:33.621709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:33.622056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:33.622117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:33.622128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:33.622129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:33.622130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:33.622132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:33.622133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:33.622134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.622135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.622136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.622143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:33.622176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:33.622181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:33.622185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:33.622188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:33.622192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:33.622195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.622199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.622202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.622210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:33.622218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:33.622225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:33.622230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:33.622235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:33.622240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:33.622244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:33.622247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:33.622251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:33.622256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:33.622260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:33.622264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:33.622269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.622276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.622288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.622292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.622297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.622301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.622327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:33.629089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:33.631419Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 545
2025-10-09T21:45:33.631429Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 545
2025-10-09T21:45:33.630965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:33.630981Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:33.630983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:33.630987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:33.630994Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 545 [2:7559341440567428495:5609]
2025-10-09T21:45:33.631009Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:33.631032Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:33.631035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:33.631038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:33.631040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:33.631059Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:33.631065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:33.631073Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:33.631162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.631166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.631167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.631173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:33.631249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:33.631252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.631266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.631278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.631296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:33.631317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341440567428490:5609]
2025-10-09T21:45:33.631321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341440567428490:5609]
2025-10-09T21:45:33.631323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341440567428490:5609]
2025-10-09T21:45:33.631331Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:33.631347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:33.631349Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:33.631353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:33.631355Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:33.631362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:33.631365Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:33.631377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341440929241436:6144], now have 1 active actors on pipe
2025-10-09T21:45:33.631381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341440929241352:2432], now have 1 active actors on pipe
2025-10-09T21:45:33.632143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 546 partition: 0 MaxSeqNo: 25 sessionId: src_id|160a8e48-9ef7a453-b8b0dbee-39ada6a0_0
2025-10-09T21:45:33.631420Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341440567428490:5609]
2025-10-09T21:45:33.631430Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:33.631434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:33.631660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:33.631668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:33.631712Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|160a8e48-9ef7a453-b8b0dbee-39ada6a0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:33.631725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:33.631729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:33.631731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:33.631743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:33.631755Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:33.631774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:33.631994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:33.631999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:33.632014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:33.632608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|756083a0-4b5b07f9-939f6755-18f4c86c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|160a8e48-9ef7a453-b8b0dbee-39ada6a0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:33.635553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|160a8e48-9ef7a453-b8b0dbee-39ada6a0_0] Write session: try to update token
2025-10-09T21:45:33.635574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|160a8e48-9ef7a453-b8b0dbee-39ada6a0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:33.652255Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:33.652272Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:33.652275Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:33.652277Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:33.652289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341440567428490:5609]
2025-10-09T21:45:33.652294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341440567428490:5609]
2025-10-09T21:45:33.652297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341440567428490:5609]
2025-10-09T21:45:33.652344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:33.652353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.652356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:33.652500Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:33.654304Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:33.654316Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:33.654320Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:33.658706Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 546 sessionId: src_id|160a8e48-9ef7a453-b8b0dbee-39ada6a0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:33.661532Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:33.692450Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341440929241479:5852]
2025-10-09T21:45:33.692916Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 545, pipe: [1:7559341440929241479:5852]
2025-10-09T21:45:33.805572Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:33.806250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:33.806265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:33.806314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341440929241479:5852], now have 1 active actors on pipe
2025-10-09T21:45:33.806668Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341440929241479:5852]
2025-10-09T21:45:33.806711Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:33.806724Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:33.806728Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:33.806801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 546 reason: error at write: Disconnected sessionId: src_id|160a8e48-9ef7a453-b8b0dbee-39ada6a0_0
2025-10-09T21:45:33.806993Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 546 sessionId: src_id|160a8e48-9ef7a453-b8b0dbee-39ada6a0_0 is DEAD
2025-10-09T21:45:33.807168Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|160a8e48-9ef7a453-b8b0dbee-39ada6a0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:33.807180Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|160a8e48-9ef7a453-b8b0dbee-39ada6a0_0] Write session will restart in 0.009763s
2025-10-09T21:45:33.807219Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|160a8e48-9ef7a453-b8b0dbee-39ada6a0_0] Write session: Do CDS request
2025-10-09T21:45:33.807229Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|160a8e48-9ef7a453-b8b0dbee-39ada6a0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:33.807180Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:33.818631Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|160a8e48-9ef7a453-b8b0dbee-39ada6a0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:33.822224Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:33.822244Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 547
2025-10-09T21:45:33.822437Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 547 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:33.822481Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 547 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:33.822489Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=547 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:33.822493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:33.822936Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 547 sessionId:  describe result for acl check
2025-10-09T21:45:33.822967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:33.822969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:33.822972Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:33.822979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341440929241506:6148] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:33.823534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341440929241506:6148] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:33.829869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341440929241506:6148] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:33.839246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341440929241506:6148] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:33.839272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341440929241506:6148] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:33.839276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341440929241506:6148] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:33.839285Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 547 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:33.841103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:33.846138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:33.846416Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:33.846420Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:33.846431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:33.846482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:33.846518Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341440567428613:5611]
2025-10-09T21:45:33.847027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:33.848726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:33.848782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:33.849097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:33.849145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:33.849151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:33.849152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:33.849154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:33.849155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:33.849156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:33.849157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.849158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.849159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.849164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:33.849173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:33.849176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:33.849180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:33.849183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:33.849187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:33.849190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.849193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.849196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.849203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:33.849210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:33.849216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:33.849221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:33.849224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:33.849229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:33.849232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:33.849240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:33.849244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:33.849249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:33.849256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:33.849260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:33.849264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.849272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.849276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.849280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.849284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.849288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.849312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:33.864005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:33.865671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:33.866645Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:33.866649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:33.866654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:33.866662Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 546 [2:7559341440567428613:5611]
2025-10-09T21:45:33.866680Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:33.866701Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:33.866705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:33.866709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:33.866711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:33.866732Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:33.866738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:33.866745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:33.866864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.866867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.866869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.866875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:33.866976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:33.866979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.866994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:33.867006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:33.867022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:33.867043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341440567428607:5611]
2025-10-09T21:45:33.867046Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341440567428607:5611]
2025-10-09T21:45:33.867048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341440567428607:5611]
2025-10-09T21:45:33.867063Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:33.867802Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 546
2025-10-09T21:45:33.867835Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 546
2025-10-09T21:45:33.867080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:33.867101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:33.867104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:33.867132Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:33.867136Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:33.867139Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:33.867146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341440567428607:5611]
2025-10-09T21:45:33.867149Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:33.867240Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:33.867475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341440929241482:2432], now have 1 active actors on pipe
2025-10-09T21:45:33.867479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341440929241595:6148], now have 1 active actors on pipe
2025-10-09T21:45:33.868294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:33.868307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:33.868338Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4f1ecb69-43cd70a7-8eac97f2-fb3788e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:33.868348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:33.868352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:33.868355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:33.868365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:33.868394Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:33.868412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:33.874816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:33.874844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:33.874880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:33.875060Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 547 partition: 0 MaxSeqNo: 25 sessionId: src_id|4f1ecb69-43cd70a7-8eac97f2-fb3788e_0
2025-10-09T21:45:33.878406Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|160a8e48-9ef7a453-b8b0dbee-39ada6a0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4f1ecb69-43cd70a7-8eac97f2-fb3788e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:33.880936Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4f1ecb69-43cd70a7-8eac97f2-fb3788e_0] Write session: try to update token
2025-10-09T21:45:33.880951Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4f1ecb69-43cd70a7-8eac97f2-fb3788e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:33.886480Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:33.886494Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:33.886497Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:33.886499Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:33.886514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341440567428607:5611]
2025-10-09T21:45:33.886518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341440567428607:5611]
2025-10-09T21:45:33.886521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341440567428607:5611]
2025-10-09T21:45:33.886568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:33.886575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:33.886578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:33.886711Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:33.890231Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:33.890240Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:33.890245Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:33.907067Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341440929241609:5852]
2025-10-09T21:45:33.907715Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 546, pipe: [1:7559341440929241609:5852]
2025-10-09T21:45:33.912126Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 547 sessionId: src_id|4f1ecb69-43cd70a7-8eac97f2-fb3788e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:33.915135Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:34.065503Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:34.066227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341440929241609:5852], now have 1 active actors on pipe
2025-10-09T21:45:34.066265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:34.066273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:34.066629Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:34.067203Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341440929241609:5852]
2025-10-09T21:45:34.067236Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:34.067272Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:34.067356Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 547 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4f1ecb69-43cd70a7-8eac97f2-fb3788e_0
2025-10-09T21:45:34.067547Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 547 sessionId: src_id|4f1ecb69-43cd70a7-8eac97f2-fb3788e_0 is DEAD
2025-10-09T21:45:34.070581Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f1ecb69-43cd70a7-8eac97f2-fb3788e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:34.070599Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f1ecb69-43cd70a7-8eac97f2-fb3788e_0] Write session will restart in 0.009037s
2025-10-09T21:45:34.070631Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f1ecb69-43cd70a7-8eac97f2-fb3788e_0] Write session: Do CDS request
2025-10-09T21:45:34.070638Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f1ecb69-43cd70a7-8eac97f2-fb3788e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:34.080595Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4f1ecb69-43cd70a7-8eac97f2-fb3788e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:34.082479Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:34.082499Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 548
2025-10-09T21:45:34.082679Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 548 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:34.082712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 548 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:34.082718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=548 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:34.082722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:34.083288Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 548 sessionId:  describe result for acl check
2025-10-09T21:45:34.083331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:34.083340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:34.083342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:34.083348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341445224208932:6153] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:34.083931Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341445224208932:6153] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:34.089280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341445224208932:6153] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:34.095001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341445224208932:6153] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:34.095022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341445224208932:6153] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:34.095027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341445224208932:6153] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:34.095036Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 548 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:34.095883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:34.100607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:34.100864Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:34.100869Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:34.100883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:34.100949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:34.100990Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341444862396029:5613]
2025-10-09T21:45:34.101619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:34.102995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:34.103060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:34.103418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:34.103480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:34.103481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:34.103483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:34.103484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:34.103486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:34.103487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:34.103488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.103490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.103491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.103497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:34.103508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:34.103513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:34.103517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:34.103521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:34.103526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:34.103530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.103534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.103538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.103546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:34.103556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:34.103564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:34.103570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:34.103576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:34.103582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:34.103586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:34.103592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:34.103597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:34.103602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:34.103607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:34.103613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:34.103619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.103625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.103630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.103635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.103640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.103645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.103674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:34.115496Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 547
2025-10-09T21:45:34.115839Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 547
2025-10-09T21:45:34.113259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:34.114946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:34.114954Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:34.114957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:34.114963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:34.114973Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 547 [2:7559341444862396029:5613]
2025-10-09T21:45:34.114991Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:34.115015Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:34.115019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:34.115023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:34.115026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:34.115048Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:34.115055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:34.115064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:34.115141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.115144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.115146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.115149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:34.115233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:34.115237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.115253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.115264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.115282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:34.115307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341444862396024:5613]
2025-10-09T21:45:34.115313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341444862396024:5613]
2025-10-09T21:45:34.115315Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341444862396024:5613]
2025-10-09T21:45:34.115325Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:34.115341Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:34.115352Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:34.115356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:34.115359Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:34.115369Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:34.115372Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:34.115407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341444862396024:5613]
2025-10-09T21:45:34.115414Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:34.115414Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:34.117347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341445224209033:6153], now have 1 active actors on pipe
2025-10-09T21:45:34.117404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341445224208918:2432], now have 1 active actors on pipe
2025-10-09T21:45:34.117448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:34.117468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:34.117551Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|83697088-47e3cdad-73141829-8d38f9b2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:34.117564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:34.117571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:34.117574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:34.117587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:34.117616Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:34.117641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:34.118150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:34.118175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:34.118198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:34.118516Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 548 partition: 0 MaxSeqNo: 25 sessionId: src_id|83697088-47e3cdad-73141829-8d38f9b2_0
2025-10-09T21:45:34.121720Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f1ecb69-43cd70a7-8eac97f2-fb3788e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|83697088-47e3cdad-73141829-8d38f9b2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:34.124613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|83697088-47e3cdad-73141829-8d38f9b2_0] Write session: try to update token
2025-10-09T21:45:34.124646Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|83697088-47e3cdad-73141829-8d38f9b2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:34.157087Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 548 sessionId: src_id|83697088-47e3cdad-73141829-8d38f9b2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:34.159532Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:34.160081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:34.160104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:34.160150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:34.160430Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:34.167562Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341445224209043:5852]
2025-10-09T21:45:34.168570Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 547, pipe: [1:7559341445224209043:5852]
2025-10-09T21:45:34.170116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341445224209043:5852], now have 1 active actors on pipe
2025-10-09T21:45:34.171396Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:34.171401Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:34.171403Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:34.171407Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:34.171420Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341444862396024:5613]
2025-10-09T21:45:34.171424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341444862396024:5613]
2025-10-09T21:45:34.171427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341444862396024:5613]
2025-10-09T21:45:34.171477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:34.171484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.171486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:34.171624Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:34.174141Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:34.174150Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:34.174154Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:34.380649Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:34.389685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:34.389711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:34.389887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:34.389941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:34.389997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:34.390259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:34.390570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:34.390576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:34.390647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:34.390701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:34.390755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:34.390811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:34.390852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:34.390855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:34.390916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:34.390970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:34.391027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:34.391088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:34.391127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:34.391130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:34.391197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:34.391256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:34.391315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:34.391373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:34.391411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:34.391414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:34.391479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:34.391536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:34.391599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:34.391655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:34.391694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:34.391697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:34.391779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:34.391831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:34.391883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:34.391936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:34.391975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:34.391980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:34.392049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:34.392099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:34.392155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:34.392209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:34.392246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:34.392249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:34.392309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:34.392363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:34.392418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:34.392473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:34.392513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:34.392515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:34.392578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:34.392630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:34.392687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:34.392745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:34.392784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:34.392786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:34.392848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:34.392899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:34.392953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:34.393012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:34.393050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:34.393052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:34.394595Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:34.398870Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83697088-47e3cdad-73141829-8d38f9b2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:34.398884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83697088-47e3cdad-73141829-8d38f9b2_0] Write session will restart in 0.006546s
2025-10-09T21:45:34.398910Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83697088-47e3cdad-73141829-8d38f9b2_0] Write session: Do CDS request
2025-10-09T21:45:34.398917Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83697088-47e3cdad-73141829-8d38f9b2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:34.398417Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:34.398488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 548 reason: status is not ok: tablet will be restarted right now sessionId: src_id|83697088-47e3cdad-73141829-8d38f9b2_0
2025-10-09T21:45:34.398682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 548 sessionId: src_id|83697088-47e3cdad-73141829-8d38f9b2_0 is DEAD
2025-10-09T21:45:34.398822Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:34.398859Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341445224209043:5852]
2025-10-09T21:45:34.406792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|83697088-47e3cdad-73141829-8d38f9b2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:34.410110Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:34.410142Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 549
2025-10-09T21:45:34.410401Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 549 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:34.410434Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 549 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:34.410440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=549 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:34.410444Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:34.412439Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 549 sessionId:  describe result for acl check
2025-10-09T21:45:34.412485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:34.412486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:34.412488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:34.412494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341445224209073:6163] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:34.413028Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341445224209073:6163] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:34.417830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341445224209073:6163] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:34.430567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341445224209073:6163] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:34.430592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341445224209073:6163] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:34.430596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341445224209073:6163] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:34.430607Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 549 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:34.444003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:34.448292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:34.448559Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:34.448564Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:34.448578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:34.448641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:34.448695Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341444862396184:5627]
2025-10-09T21:45:34.449305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:34.451809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:34.451881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:34.452266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:34.452310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:34.452311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:34.452313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:34.452314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:34.452315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:34.452317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:34.452318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.452319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.452321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.452327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:34.452339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:34.452343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:34.452347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:34.452352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:34.452356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:34.452360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.452364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.452368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.452376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:34.452385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:34.452391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:34.452396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:34.452404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:34.452409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:34.452413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:34.452418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:34.452424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:34.452429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:34.452433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:34.452438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:34.452444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.452449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.452453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.452458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.452463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.452469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.452500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:34.466912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:34.468533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:34.468541Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:34.468543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:34.468547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:34.468554Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 548 [2:7559341444862396184:5627]
2025-10-09T21:45:34.468569Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:34.468592Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:34.468596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:34.468599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:34.468602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:34.468622Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:34.468628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:34.468635Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:34.468703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.468705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.468707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.468710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:34.468790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:34.468792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.468806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.468816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.468832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:34.468853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341444862396178:5627]
2025-10-09T21:45:34.468855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341444862396178:5627]
2025-10-09T21:45:34.468856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341444862396178:5627]
2025-10-09T21:45:34.468864Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:34.468881Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:34.468887Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:34.468891Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:34.468893Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:34.468896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:34.468899Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:34.468911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341445224209064:2432], now have 1 active actors on pipe
2025-10-09T21:45:34.468914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341445224209166:6163], now have 1 active actors on pipe
2025-10-09T21:45:34.468956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341444862396178:5627]
2025-10-09T21:45:34.468958Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:34.468962Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:34.469896Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 548
2025-10-09T21:45:34.469941Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 548
2025-10-09T21:45:34.474204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:34.474223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:34.474266Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c1c718f4-153a10c8-5020acb5-1aa0cdc3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:34.474280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:34.474285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:34.474288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:34.474302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:34.474321Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:34.474341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:34.477741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:34.477762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:34.477816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:34.478128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 549 partition: 0 MaxSeqNo: 25 sessionId: src_id|c1c718f4-153a10c8-5020acb5-1aa0cdc3_0
2025-10-09T21:45:34.478582Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83697088-47e3cdad-73141829-8d38f9b2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c1c718f4-153a10c8-5020acb5-1aa0cdc3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:34.481279Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1c718f4-153a10c8-5020acb5-1aa0cdc3_0] Write session: try to update token
2025-10-09T21:45:34.481301Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1c718f4-153a10c8-5020acb5-1aa0cdc3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:34.489239Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:34.489256Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:34.489259Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:34.489261Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:34.489271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341444862396178:5627]
2025-10-09T21:45:34.489276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341444862396178:5627]
2025-10-09T21:45:34.489282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341444862396178:5627]
2025-10-09T21:45:34.489330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:34.489337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.489339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:34.489476Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:34.493841Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:34.493855Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:34.493859Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:34.498828Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341445224209199:5852]
2025-10-09T21:45:34.500679Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 549 sessionId: src_id|c1c718f4-153a10c8-5020acb5-1aa0cdc3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:34.503640Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:34.503653Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 548, pipe: [1:7559341445224209199:5852]
2025-10-09T21:45:34.764328Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:34.764799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341445224209199:5852], now have 1 active actors on pipe
2025-10-09T21:45:34.764832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:34.764842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:34.765147Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:34.766146Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:34.766218Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341445224209199:5852]
2025-10-09T21:45:34.766239Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:34.766310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 549 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c1c718f4-153a10c8-5020acb5-1aa0cdc3_0
2025-10-09T21:45:34.766489Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 549 sessionId: src_id|c1c718f4-153a10c8-5020acb5-1aa0cdc3_0 is DEAD
2025-10-09T21:45:34.768199Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1c718f4-153a10c8-5020acb5-1aa0cdc3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:34.768211Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1c718f4-153a10c8-5020acb5-1aa0cdc3_0] Write session will restart in 0.009403s
2025-10-09T21:45:34.768240Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1c718f4-153a10c8-5020acb5-1aa0cdc3_0] Write session: Do CDS request
2025-10-09T21:45:34.768247Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1c718f4-153a10c8-5020acb5-1aa0cdc3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:34.778628Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1c718f4-153a10c8-5020acb5-1aa0cdc3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:34.778744Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:34.778763Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 550
2025-10-09T21:45:34.779097Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 550 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:34.779152Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 550 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:34.779163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=550 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:34.779167Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:34.779774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 550 sessionId:  describe result for acl check
2025-10-09T21:45:34.779818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:34.779821Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:34.779824Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:34.779830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341445224209234:6169] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:34.780397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341445224209234:6169] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:34.784456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341445224209234:6169] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:34.791956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341445224209234:6169] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:34.791974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341445224209234:6169] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:34.791978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341445224209234:6169] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:34.791988Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 550 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:34.802212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:34.804139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:34.804515Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:34.804528Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:34.804545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:34.804632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:34.804688Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341444862396315:5631]
2025-10-09T21:45:34.805347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:34.805821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:34.805887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:34.806268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:34.806335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:34.806337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:34.806338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:34.806339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:34.806341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:34.806342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:34.806344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.806345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.806347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.806354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:34.806365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:34.806370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:34.806374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:34.806378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:34.806383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:34.806387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.806390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.806395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.806403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:34.806412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:34.806436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:34.806442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:34.806449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:34.806454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:34.806458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:34.806465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:34.806469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:34.806473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:34.806481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:34.806486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:34.806492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.806497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.806502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.806507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.806513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.806517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.806548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:34.815558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:34.817367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:34.817387Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:34.817390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:34.817395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:34.817403Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 549 [2:7559341444862396315:5631]
2025-10-09T21:45:34.817417Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:34.817440Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:34.817444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:34.817447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:34.817449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:34.817471Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:34.817478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:34.817489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:34.817573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.817581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.817583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.817586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:34.817930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:34.817941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.817957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:34.818286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:34.818304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:34.818297Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 549
2025-10-09T21:45:34.818334Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 549
2025-10-09T21:45:34.818341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341445224209324:6169], now have 1 active actors on pipe
2025-10-09T21:45:34.818349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341445224209207:2432], now have 1 active actors on pipe
2025-10-09T21:45:34.818365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341444862396310:5631]
2025-10-09T21:45:34.818368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341444862396310:5631]
2025-10-09T21:45:34.818370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341444862396310:5631]
2025-10-09T21:45:34.818385Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:34.818403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:34.818425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:34.818430Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:34.818433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:34.818434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:34.818435Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:34.818490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341444862396310:5631]
2025-10-09T21:45:34.818503Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:34.818511Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:34.818528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:34.818545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:34.818611Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d2565bc0-d14c75cb-62dc9a59-655c9fc9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:34.818621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:34.818627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:34.818631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:34.818642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:34.818653Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:34.818676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:34.819091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:34.819104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:34.819125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:34.819250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 550 partition: 0 MaxSeqNo: 25 sessionId: src_id|d2565bc0-d14c75cb-62dc9a59-655c9fc9_0
2025-10-09T21:45:34.820070Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1c718f4-153a10c8-5020acb5-1aa0cdc3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d2565bc0-d14c75cb-62dc9a59-655c9fc9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:34.822845Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d2565bc0-d14c75cb-62dc9a59-655c9fc9_0] Write session: try to update token
2025-10-09T21:45:34.822867Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d2565bc0-d14c75cb-62dc9a59-655c9fc9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:34.841331Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:34.841348Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:34.841351Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:34.841353Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:34.841364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341444862396310:5631]
2025-10-09T21:45:34.841368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341444862396310:5631]
2025-10-09T21:45:34.841372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341444862396310:5631]
2025-10-09T21:45:34.841419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:34.841430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:34.841433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:34.841569Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:34.845512Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:34.845525Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:34.845529Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:34.848506Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 550 sessionId: src_id|d2565bc0-d14c75cb-62dc9a59-655c9fc9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:34.851118Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:34.865398Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341445224209340:5852]
2025-10-09T21:45:34.865865Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 549, pipe: [1:7559341445224209340:5852]
2025-10-09T21:45:35.037261Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:35.038017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:35.038058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:35.038117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341445224209340:5852], now have 1 active actors on pipe
2025-10-09T21:45:35.040762Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d2565bc0-d14c75cb-62dc9a59-655c9fc9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:35.040775Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d2565bc0-d14c75cb-62dc9a59-655c9fc9_0] Write session will restart in 0.006589s
2025-10-09T21:45:35.040798Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d2565bc0-d14c75cb-62dc9a59-655c9fc9_0] Write session: Do CDS request
2025-10-09T21:45:35.040805Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d2565bc0-d14c75cb-62dc9a59-655c9fc9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:35.041636Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:35.039318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341445224209340:5852]
2025-10-09T21:45:35.039360Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:35.039364Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:35.039424Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 550 reason: error at write: Disconnected sessionId: src_id|d2565bc0-d14c75cb-62dc9a59-655c9fc9_0
2025-10-09T21:45:35.039614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 550 sessionId: src_id|d2565bc0-d14c75cb-62dc9a59-655c9fc9_0 is DEAD
2025-10-09T21:45:35.039757Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:35.050743Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d2565bc0-d14c75cb-62dc9a59-655c9fc9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:35.051161Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:35.051181Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 551
2025-10-09T21:45:35.051373Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 551 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:35.051403Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 551 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:35.051410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=551 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:35.051414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:35.051831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 551 sessionId:  describe result for acl check
2025-10-09T21:45:35.051874Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:35.051876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:35.051877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:35.051884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341449519176665:6174] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:35.052487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341449519176665:6174] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:35.058321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341449519176665:6174] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:35.066708Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341449519176665:6174] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:35.066731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341449519176665:6174] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:35.066735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341449519176665:6174] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:35.066746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 551 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:35.080808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:35.082686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:35.083413Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:35.083427Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:35.083446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:35.083532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:35.083582Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341449157363733:5633]
2025-10-09T21:45:35.084210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:35.085045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:35.085104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:35.085457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:35.085539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:35.085540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:35.085541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:35.085543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:35.085544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:35.085545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:35.085547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.085548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.085549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.085555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:35.085565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:35.085569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:35.085573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:35.085576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:35.085580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:35.085583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.085586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.085590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.085597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:35.085606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:35.085612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:35.085618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:35.085622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:35.085627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:35.085631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:35.085636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:35.085640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:35.085648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:35.085652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:35.085657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:35.085661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.085668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.085672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.085677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.085681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.085685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.085714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:35.096193Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 550
2025-10-09T21:45:35.096234Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 550
2025-10-09T21:45:35.094416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:35.095655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:35.095663Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:35.095666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:35.095672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:35.095681Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 550 [2:7559341449157363733:5633]
2025-10-09T21:45:35.095697Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:35.095721Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:35.095725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:35.095728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:35.095731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:35.095752Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:35.095758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:35.095766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:35.095842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.095845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.095847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.095850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:35.095937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:35.095940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.095956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.095968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.095987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:35.096008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341449157363728:5633]
2025-10-09T21:45:35.096011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341449157363728:5633]
2025-10-09T21:45:35.096013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341449157363728:5633]
2025-10-09T21:45:35.096022Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:35.096041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:35.096053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:35.096057Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:35.096059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:35.096063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:35.096063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:35.096075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341449519176750:6174], now have 1 active actors on pipe
2025-10-09T21:45:35.096080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341449519176652:2432], now have 1 active actors on pipe
2025-10-09T21:45:35.096123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341449157363728:5633]
2025-10-09T21:45:35.096134Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:35.096150Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:35.096661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:35.096675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:35.096713Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e9bdff9b-5f76c8fe-1ebe149a-f049cfb1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:35.096723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:35.096727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:35.096730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:35.096742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:35.096756Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:35.096777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:35.097857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:35.097870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:35.097897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:35.098142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 551 partition: 0 MaxSeqNo: 25 sessionId: src_id|e9bdff9b-5f76c8fe-1ebe149a-f049cfb1_0
2025-10-09T21:45:35.102284Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d2565bc0-d14c75cb-62dc9a59-655c9fc9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e9bdff9b-5f76c8fe-1ebe149a-f049cfb1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:35.105019Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9bdff9b-5f76c8fe-1ebe149a-f049cfb1_0] Write session: try to update token
2025-10-09T21:45:35.105049Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9bdff9b-5f76c8fe-1ebe149a-f049cfb1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:35.119091Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:35.119109Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:35.119112Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:35.119114Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:35.119124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341449157363728:5633]
2025-10-09T21:45:35.119144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341449157363728:5633]
2025-10-09T21:45:35.119149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341449157363728:5633]
2025-10-09T21:45:35.119217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:35.119225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.119228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:35.119381Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:35.120041Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:35.120052Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:35.120055Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:35.131931Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 551 sessionId: src_id|e9bdff9b-5f76c8fe-1ebe149a-f049cfb1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:35.134722Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:35.139317Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341449519176781:5852]
2025-10-09T21:45:35.139620Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 550, pipe: [1:7559341449519176781:5852]
2025-10-09T21:45:35.340632Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:35.341206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:35.341218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:35.341242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341449519176781:5852], now have 1 active actors on pipe
2025-10-09T21:45:35.341569Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:35.342222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:35.342274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:35.342318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341449519176781:5852]
2025-10-09T21:45:35.342319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 551 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e9bdff9b-5f76c8fe-1ebe149a-f049cfb1_0
2025-10-09T21:45:35.342511Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 551 sessionId: src_id|e9bdff9b-5f76c8fe-1ebe149a-f049cfb1_0 is DEAD
2025-10-09T21:45:35.348601Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9bdff9b-5f76c8fe-1ebe149a-f049cfb1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:35.348618Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9bdff9b-5f76c8fe-1ebe149a-f049cfb1_0] Write session will restart in 0.009314s
2025-10-09T21:45:35.348648Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9bdff9b-5f76c8fe-1ebe149a-f049cfb1_0] Write session: Do CDS request
2025-10-09T21:45:35.348654Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9bdff9b-5f76c8fe-1ebe149a-f049cfb1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:35.362588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9bdff9b-5f76c8fe-1ebe149a-f049cfb1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:35.363124Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:35.363142Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 552
2025-10-09T21:45:35.363345Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 552 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:35.363383Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 552 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:35.363389Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=552 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:35.363392Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:35.364032Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 552 sessionId:  describe result for acl check
2025-10-09T21:45:35.364064Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:35.364066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:35.364068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:35.364073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341449519176797:6178] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:35.364594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341449519176797:6178] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:35.369940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341449519176797:6178] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:35.379808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341449519176797:6178] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:35.379831Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341449519176797:6178] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:35.379836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341449519176797:6178] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:35.379847Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 552 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:35.383088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:35.390319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:35.390611Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:35.390629Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:35.390646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:35.390724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:35.390765Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341449157363874:5637]
2025-10-09T21:45:35.391373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:35.391950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:35.392004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:35.392329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:35.392375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:35.392377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:35.392379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:35.392380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:35.392381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:35.392383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:35.392384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.392386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.392388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.392392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:35.392403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:35.392406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:35.392410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:35.392413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:35.392417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:35.392420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.392424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.392427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.392435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:35.392443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:35.392449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:35.392454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:35.392458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:35.392463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:35.392467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:35.392471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:35.392475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:35.392480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:35.392483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:35.392489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:35.392496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.392501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.392505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.392509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.392513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.392518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.392545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:35.403066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:35.405413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:35.405422Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:35.405424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:35.405428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:35.405437Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 551 [2:7559341449157363874:5637]
2025-10-09T21:45:35.405451Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:35.405471Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:35.405474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:35.405478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:35.405480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:35.405499Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:35.405506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:35.405512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:35.405680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.405682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.405687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.405690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:35.405765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:35.405767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.405781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.405791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.405806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:35.405826Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341449157363869:5637]
2025-10-09T21:45:35.405828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341449157363869:5637]
2025-10-09T21:45:35.405830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341449157363869:5637]
2025-10-09T21:45:35.405846Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:35.405861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:35.405874Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:35.405876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:35.405917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341449157363869:5637]
2025-10-09T21:45:35.405920Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:35.406256Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:35.406270Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:35.406282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:35.406290Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:35.406635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341449519176900:6178], now have 1 active actors on pipe
2025-10-09T21:45:35.406646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341449519176784:2432], now have 1 active actors on pipe
2025-10-09T21:45:35.407874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 551
2025-10-09T21:45:35.407915Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 551
2025-10-09T21:45:35.408971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:35.408997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:35.409031Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e38becf1-fa10f347-1419e39b-d6a8bf68_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:35.409043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:35.409054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:35.409056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:35.409069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:35.409092Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:35.409116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:35.410832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:35.410856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:35.410886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:35.414726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 552 partition: 0 MaxSeqNo: 25 sessionId: src_id|e38becf1-fa10f347-1419e39b-d6a8bf68_0
2025-10-09T21:45:35.415187Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9bdff9b-5f76c8fe-1ebe149a-f049cfb1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e38becf1-fa10f347-1419e39b-d6a8bf68_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:35.417893Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e38becf1-fa10f347-1419e39b-d6a8bf68_0] Write session: try to update token
2025-10-09T21:45:35.417918Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e38becf1-fa10f347-1419e39b-d6a8bf68_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:35.439245Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:35.439261Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:35.439264Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:35.439266Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:35.439276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341449157363869:5637]
2025-10-09T21:45:35.439281Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341449157363869:5637]
2025-10-09T21:45:35.439285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341449157363869:5637]
2025-10-09T21:45:35.439337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:35.439344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.439347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:35.442401Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341449519176916:5852]
2025-10-09T21:45:35.439490Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:35.442363Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:35.442372Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:35.442375Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:35.442897Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 551, pipe: [1:7559341449519176916:5852]
2025-10-09T21:45:35.462086Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 552 sessionId: src_id|e38becf1-fa10f347-1419e39b-d6a8bf68_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:35.464635Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:35.612648Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:35.614019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341449519176916:5852], now have 1 active actors on pipe
2025-10-09T21:45:35.614099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:35.614107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:35.614669Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:35.618343Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341449519176916:5852]
2025-10-09T21:45:35.618385Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:35.618471Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 552 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e38becf1-fa10f347-1419e39b-d6a8bf68_0
2025-10-09T21:45:35.618624Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:35.618661Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 552 sessionId: src_id|e38becf1-fa10f347-1419e39b-d6a8bf68_0 is DEAD
2025-10-09T21:45:35.618905Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e38becf1-fa10f347-1419e39b-d6a8bf68_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:35.618922Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e38becf1-fa10f347-1419e39b-d6a8bf68_0] Write session will restart in 0.005476s
2025-10-09T21:45:35.618953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e38becf1-fa10f347-1419e39b-d6a8bf68_0] Write session: Do CDS request
2025-10-09T21:45:35.618960Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e38becf1-fa10f347-1419e39b-d6a8bf68_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:35.626799Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e38becf1-fa10f347-1419e39b-d6a8bf68_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:35.630266Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:35.630292Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 553
2025-10-09T21:45:35.630550Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 553 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:35.630582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 553 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:35.630589Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=553 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:35.630594Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:35.634402Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 553 sessionId:  describe result for acl check
2025-10-09T21:45:35.634462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:35.634465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:35.634467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:35.634473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341449519176953:6190] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:35.635289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341449519176953:6190] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:35.640862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341449519176953:6190] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:35.648521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341449519176953:6190] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:35.648543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341449519176953:6190] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:35.648547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341449519176953:6190] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:35.648558Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 553 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:35.660761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:35.666198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:35.666474Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:35.666478Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:35.666492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:35.666559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:35.666603Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341449157364023:5651]
2025-10-09T21:45:35.667192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:35.667919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:35.667975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:35.668299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:35.668354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:35.668355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:35.668357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:35.668358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:35.668359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:35.668360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:35.668362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.668363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.668365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.668370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:35.668384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:35.668387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:35.668392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:35.668396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:35.668400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:35.668404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.668407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.668411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.668419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:35.668427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:35.668454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:35.668460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:35.668465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:35.668469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:35.668474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:35.668479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:35.668484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:35.668489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:35.668494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:35.668500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:35.668505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.668519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.668524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.668528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.668535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.668540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.668570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:35.679545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:35.681215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:35.681231Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:35.681234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:35.681239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:35.681247Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 552 [2:7559341449157364023:5651]
2025-10-09T21:45:35.681264Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:35.681285Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:35.681288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:35.681292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:35.681294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:35.681313Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:35.681320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:35.681332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:35.681398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.681400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.681403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.681406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:35.681481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:35.681484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.681499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.681511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.681528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:35.681550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341449157364017:5651]
2025-10-09T21:45:35.681566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341449157364017:5651]
2025-10-09T21:45:35.681568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341449157364017:5651]
2025-10-09T21:45:35.681584Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:35.681600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:35.681616Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:35.681619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:35.681662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341449157364017:5651]
2025-10-09T21:45:35.681665Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:35.682026Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:35.682049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:35.682051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:35.682059Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:35.682516Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 552
2025-10-09T21:45:35.682546Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 552
2025-10-09T21:45:35.682813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341449519176945:2432], now have 1 active actors on pipe
2025-10-09T21:45:35.682823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341449519177011:6190], now have 1 active actors on pipe
2025-10-09T21:45:35.682844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:35.682850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:35.682884Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a44e5021-ae393ff4-d653541d-5f7c51a9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:35.682895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:35.682900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:35.682904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:35.682917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:35.682931Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:35.682951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:35.683250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:35.683271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:35.683293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:35.683500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 553 partition: 0 MaxSeqNo: 25 sessionId: src_id|a44e5021-ae393ff4-d653541d-5f7c51a9_0
2025-10-09T21:45:35.683921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e38becf1-fa10f347-1419e39b-d6a8bf68_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a44e5021-ae393ff4-d653541d-5f7c51a9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:35.686626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a44e5021-ae393ff4-d653541d-5f7c51a9_0] Write session: try to update token
2025-10-09T21:45:35.686644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a44e5021-ae393ff4-d653541d-5f7c51a9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:35.700788Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:35.700814Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:35.700816Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:35.700819Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:35.700829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341449157364017:5651]
2025-10-09T21:45:35.700833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341449157364017:5651]
2025-10-09T21:45:35.700837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341449157364017:5651]
2025-10-09T21:45:35.700886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:35.700895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.700901Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:35.700904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:35.700909Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:35.700913Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:35.701043Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:35.705868Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 553 sessionId: src_id|a44e5021-ae393ff4-d653541d-5f7c51a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:35.708874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:35.718574Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341449519177077:5852]
2025-10-09T21:45:35.722335Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 552, pipe: [1:7559341449519177077:5852]
2025-10-09T21:45:35.879726Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:35.880002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:35.880021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:35.880068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341449519177077:5852], now have 1 active actors on pipe
2025-10-09T21:45:35.880416Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:35.882928Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a44e5021-ae393ff4-d653541d-5f7c51a9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:35.882941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a44e5021-ae393ff4-d653541d-5f7c51a9_0] Write session will restart in 0.009149s
2025-10-09T21:45:35.882966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a44e5021-ae393ff4-d653541d-5f7c51a9_0] Write session: Do CDS request
2025-10-09T21:45:35.882973Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a44e5021-ae393ff4-d653541d-5f7c51a9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:35.882384Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:35.882459Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:35.882562Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 553 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a44e5021-ae393ff4-d653541d-5f7c51a9_0
2025-10-09T21:45:35.882737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 553 sessionId: src_id|a44e5021-ae393ff4-d653541d-5f7c51a9_0 is DEAD
2025-10-09T21:45:35.882944Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341449519177077:5852]
2025-10-09T21:45:35.894750Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a44e5021-ae393ff4-d653541d-5f7c51a9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:35.895725Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:35.895746Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 554
2025-10-09T21:45:35.896079Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 554 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:35.896132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 554 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:35.896144Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=554 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:35.896148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:35.896993Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 554 sessionId:  describe result for acl check
2025-10-09T21:45:35.897043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:35.897045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:35.897047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:35.897053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341449519177107:6194] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:35.897609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341449519177107:6194] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:35.902205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341449519177107:6194] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:35.909580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341449519177107:6194] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:35.909601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341449519177107:6194] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:35.909606Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341449519177107:6194] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:35.909617Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 554 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:35.916388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:35.917225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:35.917435Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:35.917443Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:35.917455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:35.917503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:35.917545Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341449157364149:5653]
2025-10-09T21:45:35.918095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:35.918656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:35.918700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:35.918997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:35.919044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:35.919047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:35.919048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:35.919049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:35.919050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:35.919051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:35.919052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.919053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.919055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.919060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:35.919069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:35.919073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:35.919076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:35.919080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:35.919084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:35.919086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.919090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.919093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.919100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:35.919107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:35.919112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:35.919116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:35.919120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:35.919124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:35.919127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:35.919132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:35.919135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:35.919140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:35.919143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:35.919148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:35.919152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.919157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.919160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.919166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.919170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.919174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.919200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:35.927621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:35.929754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:35.929771Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:35.929774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:35.929778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:35.929788Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 553 [2:7559341449157364149:5653]
2025-10-09T21:45:35.929803Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:35.929824Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:35.929835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:35.929839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:35.929841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:35.929860Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:35.929870Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:35.929878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:35.929943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.929949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.929950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.929954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:35.930046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:35.930053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.930068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:35.930080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:35.930102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:35.930854Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 553
2025-10-09T21:45:35.930887Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 553
2025-10-09T21:45:35.930124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341449157364144:5653]
2025-10-09T21:45:35.930132Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341449157364144:5653]
2025-10-09T21:45:35.930134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341449157364144:5653]
2025-10-09T21:45:35.930144Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:35.930173Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:35.930179Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:35.930182Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:35.930185Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:35.930190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:35.930193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:35.930205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341449519177202:6194], now have 1 active actors on pipe
2025-10-09T21:45:35.930209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341449519177083:2432], now have 1 active actors on pipe
2025-10-09T21:45:35.930250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341449157364144:5653]
2025-10-09T21:45:35.930256Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:35.930660Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:35.934206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:35.934224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:35.934266Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f3c68a9d-1fa9b9da-99382558-caf7e37b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:35.934285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:35.934292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:35.934295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:35.934310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:35.934328Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:35.934350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:35.934727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:35.934736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:35.934754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:35.934881Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 554 partition: 0 MaxSeqNo: 25 sessionId: src_id|f3c68a9d-1fa9b9da-99382558-caf7e37b_0
2025-10-09T21:45:35.935307Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a44e5021-ae393ff4-d653541d-5f7c51a9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f3c68a9d-1fa9b9da-99382558-caf7e37b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:35.938130Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3c68a9d-1fa9b9da-99382558-caf7e37b_0] Write session: try to update token
2025-10-09T21:45:35.938151Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3c68a9d-1fa9b9da-99382558-caf7e37b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:35.969396Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 554 sessionId: src_id|f3c68a9d-1fa9b9da-99382558-caf7e37b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:35.972094Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:35.973193Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:35.973207Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:35.973210Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:35.973211Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:35.973220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341449157364144:5653]
2025-10-09T21:45:35.973225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341449157364144:5653]
2025-10-09T21:45:35.973230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341449157364144:5653]
2025-10-09T21:45:35.973273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:35.973279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:35.973281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:35.973412Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:35.974760Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:35.974773Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:35.974777Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:35.982937Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341449519177219:5852]
2025-10-09T21:45:35.984318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 553, pipe: [1:7559341449519177219:5852]
2025-10-09T21:45:36.148073Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:36.148352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:36.148367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:36.148391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341449519177219:5852], now have 1 active actors on pipe
2025-10-09T21:45:36.149153Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:36.149262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 554 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f3c68a9d-1fa9b9da-99382558-caf7e37b_0
2025-10-09T21:45:36.149467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 554 sessionId: src_id|f3c68a9d-1fa9b9da-99382558-caf7e37b_0 is DEAD
2025-10-09T21:45:36.149651Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341449519177219:5852]
2025-10-09T21:45:36.149666Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:36.148678Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:36.150297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3c68a9d-1fa9b9da-99382558-caf7e37b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:36.150310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3c68a9d-1fa9b9da-99382558-caf7e37b_0] Write session will restart in 0.005561s
2025-10-09T21:45:36.150339Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3c68a9d-1fa9b9da-99382558-caf7e37b_0] Write session: Do CDS request
2025-10-09T21:45:36.150345Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3c68a9d-1fa9b9da-99382558-caf7e37b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:36.162285Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3c68a9d-1fa9b9da-99382558-caf7e37b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:36.162753Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:36.162770Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 555
2025-10-09T21:45:36.166311Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 555 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:36.166368Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 555 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:36.166375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=555 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:36.166389Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:36.167139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 555 sessionId:  describe result for acl check
2025-10-09T21:45:36.167182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:36.167189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:36.167190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:36.167196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341453814144548:6199] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:36.167751Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341453814144548:6199] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:36.172513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341453814144548:6199] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:36.180734Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341453814144548:6199] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:36.180754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341453814144548:6199] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:36.180759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341453814144548:6199] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:36.180768Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 555 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:36.183353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:36.187223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:36.187444Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:36.187449Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:36.187460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:36.187509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:36.187546Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341453452331572:5655]
2025-10-09T21:45:36.188080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:36.188517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:36.188558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:36.189082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:36.189141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:36.189142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:36.189143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:36.189144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:36.189145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:36.189146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:36.189147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.189148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.189149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.189155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:36.189166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:36.189169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:36.189173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:36.189177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:36.189182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:36.189185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.189188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.189191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.189199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:36.189207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:36.189212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:36.189217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:36.189221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:36.189227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:36.189231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:36.189235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:36.189260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:36.189264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:36.189268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:36.189273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:36.189277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.189282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.189286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.189291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.189295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.189298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.189326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:36.198623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:36.199904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:36.199921Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:36.199924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:36.199932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:36.199940Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 554 [2:7559341453452331572:5655]
2025-10-09T21:45:36.199957Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:36.199980Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:36.199987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:36.199991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:36.199994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:36.200014Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:36.200021Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:36.200029Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:36.200122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.200124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.200126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.200128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:36.200229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:36.200240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.200254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.200265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.200282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:36.200312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341453452331566:5655]
2025-10-09T21:45:36.200314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341453452331566:5655]
2025-10-09T21:45:36.200316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341453452331566:5655]
2025-10-09T21:45:36.200325Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:36.200343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:36.200348Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:36.200351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:36.200351Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:36.200365Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:36.200369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:36.200382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341453814144645:6199], now have 1 active actors on pipe
2025-10-09T21:45:36.200386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341453814144523:2432], now have 1 active actors on pipe
2025-10-09T21:45:36.200429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341453452331566:5655]
2025-10-09T21:45:36.200442Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:36.200472Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:36.201318Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 554
2025-10-09T21:45:36.201355Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 554
2025-10-09T21:45:36.202558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:36.202577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:36.202619Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9e7ec36d-c57026fe-88e17076-459dc89_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:36.202635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:36.202658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:36.202662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:36.202676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:36.202702Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:36.202725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:36.203683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:36.203708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:36.203735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:36.203877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 555 partition: 0 MaxSeqNo: 25 sessionId: src_id|9e7ec36d-c57026fe-88e17076-459dc89_0
2025-10-09T21:45:36.206512Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3c68a9d-1fa9b9da-99382558-caf7e37b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9e7ec36d-c57026fe-88e17076-459dc89_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:36.209155Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7ec36d-c57026fe-88e17076-459dc89_0] Write session: try to update token
2025-10-09T21:45:36.209177Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7ec36d-c57026fe-88e17076-459dc89_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:36.227654Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:36.227672Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:36.227675Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:36.227677Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:36.227687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341453452331566:5655]
2025-10-09T21:45:36.227693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341453452331566:5655]
2025-10-09T21:45:36.227697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341453452331566:5655]
2025-10-09T21:45:36.227744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:36.227752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.227755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:36.227896Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:36.230728Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:36.230742Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:36.230747Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:36.247906Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 555 sessionId: src_id|9e7ec36d-c57026fe-88e17076-459dc89_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:36.249851Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341453814144658:5852]
2025-10-09T21:45:36.250407Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:36.250487Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 554, pipe: [1:7559341453814144658:5852]
2025-10-09T21:45:36.407564Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:36.408373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341453814144658:5852], now have 1 active actors on pipe
2025-10-09T21:45:36.408405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:36.408412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:36.409562Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:36.409593Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:36.409671Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 555 reason: error at write: Disconnected sessionId: src_id|9e7ec36d-c57026fe-88e17076-459dc89_0
2025-10-09T21:45:36.409878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 555 sessionId: src_id|9e7ec36d-c57026fe-88e17076-459dc89_0 is DEAD
2025-10-09T21:45:36.410020Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7ec36d-c57026fe-88e17076-459dc89_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:36.410041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7ec36d-c57026fe-88e17076-459dc89_0] Write session will restart in 0.006477s
2025-10-09T21:45:36.410068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7ec36d-c57026fe-88e17076-459dc89_0] Write session: Do CDS request
2025-10-09T21:45:36.410075Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7ec36d-c57026fe-88e17076-459dc89_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:36.409528Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:36.410122Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341453814144658:5852]
2025-10-09T21:45:36.410150Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:36.420567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7ec36d-c57026fe-88e17076-459dc89_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:36.420767Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:36.420789Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 556
2025-10-09T21:45:36.421458Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 556 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:36.421507Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 556 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:36.421513Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=556 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:36.421527Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:36.422077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 556 sessionId:  describe result for acl check
2025-10-09T21:45:36.422117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:36.422124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:36.422125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:36.422131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341453814144668:6203] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:36.422695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341453814144668:6203] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:36.425768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341453814144668:6203] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:36.429708Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341453814144668:6203] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:36.429724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341453814144668:6203] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:36.429728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341453814144668:6203] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:36.429737Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 556 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:36.463773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:36.464705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:36.464908Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:36.464912Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:36.464924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:36.464977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:36.465015Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341453452331717:5659]
2025-10-09T21:45:36.465618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:36.466524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:36.466583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:36.466954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:36.467023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:36.467024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:36.467025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:36.467026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:36.467027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:36.467028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:36.467030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.467034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.467035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.467042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:36.467052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:36.467056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:36.467059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:36.467062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:36.467066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:36.467070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.467073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.467076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.467084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:36.467093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:36.467099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:36.467103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:36.467108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:36.467112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:36.467116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:36.467121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:36.467125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:36.467129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:36.467133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:36.467137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:36.467142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.467147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.467151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.467156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.467161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.467165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.467195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:36.478243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:36.479449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:36.479456Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:36.479460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:36.479465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:36.479473Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 555 [2:7559341453452331717:5659]
2025-10-09T21:45:36.479491Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:36.479513Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:36.479517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:36.479520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:36.479523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:36.479545Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:36.479551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:36.479559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:36.479635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.479637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.479639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.479642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:36.479724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:36.479729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.479743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.479754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.479769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:36.479792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341453452331711:5659]
2025-10-09T21:45:36.479795Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341453452331711:5659]
2025-10-09T21:45:36.479796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341453452331711:5659]
2025-10-09T21:45:36.479817Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:36.479832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:36.479846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:36.479849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:36.479891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341453452331711:5659]
2025-10-09T21:45:36.479894Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:36.481111Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 555
2025-10-09T21:45:36.481163Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 555
2025-10-09T21:45:36.480276Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:36.480283Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:36.480286Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:36.480292Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:36.482672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341453814144661:2432], now have 1 active actors on pipe
2025-10-09T21:45:36.482684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341453814144709:6203], now have 1 active actors on pipe
2025-10-09T21:45:36.482713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:36.482720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:36.482766Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|31bf19e4-5ffb68f0-49cf0db-fa8bbab5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:36.482778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:36.482783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:36.482786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:36.482799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:36.482816Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:36.482837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:36.483693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:36.483705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:36.483727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:36.483914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 556 partition: 0 MaxSeqNo: 25 sessionId: src_id|31bf19e4-5ffb68f0-49cf0db-fa8bbab5_0
2025-10-09T21:45:36.486256Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7ec36d-c57026fe-88e17076-459dc89_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|31bf19e4-5ffb68f0-49cf0db-fa8bbab5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:36.489058Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|31bf19e4-5ffb68f0-49cf0db-fa8bbab5_0] Write session: try to update token
2025-10-09T21:45:36.489081Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|31bf19e4-5ffb68f0-49cf0db-fa8bbab5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:36.510449Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341453814144797:5852]
2025-10-09T21:45:36.508821Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:36.508840Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:36.508842Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:36.508845Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:36.508855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341453452331711:5659]
2025-10-09T21:45:36.508860Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341453452331711:5659]
2025-10-09T21:45:36.508866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341453452331711:5659]
2025-10-09T21:45:36.508909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:36.508915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.508917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:36.509058Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:36.509276Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:36.509281Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:36.509285Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:36.514627Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 555, pipe: [1:7559341453814144797:5852]
2025-10-09T21:45:36.525569Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 556 sessionId: src_id|31bf19e4-5ffb68f0-49cf0db-fa8bbab5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:36.528261Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:36.708283Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:36.708851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341453814144797:5852], now have 1 active actors on pipe
2025-10-09T21:45:36.708886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:36.708895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:36.711174Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:36.711234Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:36.711237Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:36.711304Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 556 reason: error at write: Disconnected sessionId: src_id|31bf19e4-5ffb68f0-49cf0db-fa8bbab5_0
2025-10-09T21:45:36.711517Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 556 sessionId: src_id|31bf19e4-5ffb68f0-49cf0db-fa8bbab5_0 is DEAD
2025-10-09T21:45:36.711712Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341453814144797:5852]
2025-10-09T21:45:36.712122Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31bf19e4-5ffb68f0-49cf0db-fa8bbab5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:36.712142Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31bf19e4-5ffb68f0-49cf0db-fa8bbab5_0] Write session will restart in 0.005008s
2025-10-09T21:45:36.712171Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31bf19e4-5ffb68f0-49cf0db-fa8bbab5_0] Write session: Do CDS request
2025-10-09T21:45:36.712181Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31bf19e4-5ffb68f0-49cf0db-fa8bbab5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:36.712885Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:36.718669Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|31bf19e4-5ffb68f0-49cf0db-fa8bbab5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:36.718745Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:36.718764Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 557
2025-10-09T21:45:36.719214Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 557 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:36.719258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 557 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:36.719274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=557 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:36.719279Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:36.720033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 557 sessionId:  describe result for acl check
2025-10-09T21:45:36.720074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:36.720076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:36.720078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:36.720083Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341453814144839:6217] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:36.720674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341453814144839:6217] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:36.725163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341453814144839:6217] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:36.731521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341453814144839:6217] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:36.731542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341453814144839:6217] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:36.731546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341453814144839:6217] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:36.731557Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 557 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:36.743518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:36.750847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:36.751133Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:36.751144Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:36.751157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:36.751247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:36.751292Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341453452331871:5673]
2025-10-09T21:45:36.751843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:36.752449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:36.752519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:36.752924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:36.752999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:36.753008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:36.753010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:36.753011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:36.753013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:36.753014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:36.753015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.753016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.753017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.753024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:36.753037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:36.753042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:36.753046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:36.753050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:36.753054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:36.753059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.753063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.753067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.753075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:36.753084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:36.753091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:36.753098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:36.753103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:36.753109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:36.753114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:36.753119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:36.753126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:36.753132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:36.753136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:36.753142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:36.753146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.753152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.753156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.753162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.753166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.753171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.753201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:36.762236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:36.763969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:36.763977Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:36.763979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:36.763984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:36.763992Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 556 [2:7559341453452331871:5673]
2025-10-09T21:45:36.764008Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:36.764028Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:36.764032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:36.764035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:36.764037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:36.764056Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:36.764063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:36.764072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:36.764153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.764156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.764158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.764160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:36.764237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:36.764239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.764252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:36.764261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:36.764274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:36.764295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341453452331865:5673]
2025-10-09T21:45:36.764298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341453452331865:5673]
2025-10-09T21:45:36.764300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341453452331865:5673]
2025-10-09T21:45:36.764311Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:36.764329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:36.764336Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:36.764338Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:36.764343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:36.764344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:36.764347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:36.764394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341453814144904:6217], now have 1 active actors on pipe
2025-10-09T21:45:36.764397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341453814144830:2432], now have 1 active actors on pipe
2025-10-09T21:45:36.764399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341453452331865:5673]
2025-10-09T21:45:36.764409Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:36.764438Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:36.764829Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 556
2025-10-09T21:45:36.764837Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 556
2025-10-09T21:45:36.766080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:36.766093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:36.766134Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5ff3875-7dca8ebc-2bf3fc86-16d9d80f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:36.766145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:36.766150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:36.766161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:36.766174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:36.766196Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:36.766215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:36.769418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:36.769438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:36.769469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:36.769730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 557 partition: 0 MaxSeqNo: 25 sessionId: src_id|5ff3875-7dca8ebc-2bf3fc86-16d9d80f_0
2025-10-09T21:45:36.770335Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31bf19e4-5ffb68f0-49cf0db-fa8bbab5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5ff3875-7dca8ebc-2bf3fc86-16d9d80f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:36.773668Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ff3875-7dca8ebc-2bf3fc86-16d9d80f_0] Write session: try to update token
2025-10-09T21:45:36.773693Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ff3875-7dca8ebc-2bf3fc86-16d9d80f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:36.785815Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:36.785839Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:36.785842Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:36.785845Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:36.785861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341453452331865:5673]
2025-10-09T21:45:36.785865Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341453452331865:5673]
2025-10-09T21:45:36.785871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341453452331865:5673]
2025-10-09T21:45:36.785927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:36.785937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:36.785948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:36.785951Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:36.785958Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:36.785962Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:36.786127Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:36.812032Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341453814144966:5852]
2025-10-09T21:45:36.812358Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 556, pipe: [1:7559341453814144966:5852]
2025-10-09T21:45:36.813811Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 557 sessionId: src_id|5ff3875-7dca8ebc-2bf3fc86-16d9d80f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:36.816703Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:37.172047Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:37.172682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341453814144966:5852], now have 1 active actors on pipe
2025-10-09T21:45:37.172716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:37.172723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:37.174750Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:37.175287Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ff3875-7dca8ebc-2bf3fc86-16d9d80f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:37.175299Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ff3875-7dca8ebc-2bf3fc86-16d9d80f_0] Write session will restart in 0.009108s
2025-10-09T21:45:37.175322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ff3875-7dca8ebc-2bf3fc86-16d9d80f_0] Write session: Do CDS request
2025-10-09T21:45:37.175327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ff3875-7dca8ebc-2bf3fc86-16d9d80f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:37.174641Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:37.174659Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:37.174664Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341453814144966:5852]
2025-10-09T21:45:37.174689Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:37.174731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 557 reason: error at write: Disconnected sessionId: src_id|5ff3875-7dca8ebc-2bf3fc86-16d9d80f_0
2025-10-09T21:45:37.174986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 557 sessionId: src_id|5ff3875-7dca8ebc-2bf3fc86-16d9d80f_0 is DEAD
2025-10-09T21:45:37.186849Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ff3875-7dca8ebc-2bf3fc86-16d9d80f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:37.189609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:37.189627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 558
2025-10-09T21:45:37.189811Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 558 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:37.189847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 558 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:37.189853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=558 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:37.189858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:37.190282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 558 sessionId:  describe result for acl check
2025-10-09T21:45:37.190321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:37.190325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:37.190327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:37.190333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341458109112367:6222] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:37.190883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341458109112367:6222] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:37.195307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:37.195659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341458109112367:6222] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:37.200302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:37.200537Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:37.200550Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:37.200563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:37.200623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:37.200673Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341457747299297:5675]
2025-10-09T21:45:37.201301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:37.202252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:37.202318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:37.202694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341458109112367:6222] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:37.202716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341458109112367:6222] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:37.202720Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341458109112367:6222] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:37.202731Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 558 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:37.202667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:37.202742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:37.202743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:37.202744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:37.202745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:37.202746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:37.202747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:37.202748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.202749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.202752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.202758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:37.202768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:37.202771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:37.202775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:37.202778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:37.202782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:37.202786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.202790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.202794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.202802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:37.202811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:37.202817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:37.202823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:37.202827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:37.202831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:37.202835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:37.202840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:37.202844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:37.202849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:37.202854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:37.202859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:37.202864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.202871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.202875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.202880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.202885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.202891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.202922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:37.210751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:37.211829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:37.211834Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:37.211840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:37.211844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:37.211852Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 557 [2:7559341457747299297:5675]
2025-10-09T21:45:37.211868Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:37.211888Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:37.211892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:37.211895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:37.211897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:37.211917Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:37.211923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:37.211931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:37.212000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.212002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.212004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.212006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:37.212090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:37.212093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.212104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.212115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.212130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:37.212151Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341457747299292:5675]
2025-10-09T21:45:37.212154Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341457747299292:5675]
2025-10-09T21:45:37.212156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341457747299292:5675]
2025-10-09T21:45:37.212175Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:37.212192Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:37.212207Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:37.212209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:37.212253Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341457747299292:5675]
2025-10-09T21:45:37.212256Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:37.212461Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:37.212471Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:37.212474Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:37.212481Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:37.213532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341458109112282:2432], now have 1 active actors on pipe
2025-10-09T21:45:37.213545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341458109112409:6222], now have 1 active actors on pipe
2025-10-09T21:45:37.214179Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 557
2025-10-09T21:45:37.214232Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 557
2025-10-09T21:45:37.214364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:37.214386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:37.214422Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|12713136-e9f20f8d-a9b71b08-95f54811_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:37.214445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:37.214451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:37.214454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:37.214468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:37.214485Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:37.214504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:37.214753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:37.214760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:37.214781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:37.215321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ff3875-7dca8ebc-2bf3fc86-16d9d80f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|12713136-e9f20f8d-a9b71b08-95f54811_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:37.218114Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12713136-e9f20f8d-a9b71b08-95f54811_0] Write session: try to update token
2025-10-09T21:45:37.218134Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12713136-e9f20f8d-a9b71b08-95f54811_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:37.214929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 558 partition: 0 MaxSeqNo: 25 sessionId: src_id|12713136-e9f20f8d-a9b71b08-95f54811_0
2025-10-09T21:45:37.237215Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:37.237236Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:37.237240Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:37.237242Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:37.237252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341457747299292:5675]
2025-10-09T21:45:37.237257Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341457747299292:5675]
2025-10-09T21:45:37.237260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341457747299292:5675]
2025-10-09T21:45:37.237315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:37.237322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.237325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:37.237490Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:37.239400Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:37.239406Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:37.239411Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:37.250539Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 558 sessionId: src_id|12713136-e9f20f8d-a9b71b08-95f54811_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:37.253857Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:37.274887Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341458109112419:5852]
2025-10-09T21:45:37.275251Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 557, pipe: [1:7559341458109112419:5852]
2025-10-09T21:45:37.457090Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:37.459111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:37.459128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:37.459153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341458109112419:5852], now have 1 active actors on pipe
2025-10-09T21:45:37.459427Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:37.462802Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:37.462867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:37.462965Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 558 reason: status is not ok: tablet will be restarted right now sessionId: src_id|12713136-e9f20f8d-a9b71b08-95f54811_0
2025-10-09T21:45:37.463153Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 558 sessionId: src_id|12713136-e9f20f8d-a9b71b08-95f54811_0 is DEAD
2025-10-09T21:45:37.463302Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341458109112419:5852]
2025-10-09T21:45:37.466418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12713136-e9f20f8d-a9b71b08-95f54811_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:37.466431Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12713136-e9f20f8d-a9b71b08-95f54811_0] Write session will restart in 0.008361s
2025-10-09T21:45:37.466458Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12713136-e9f20f8d-a9b71b08-95f54811_0] Write session: Do CDS request
2025-10-09T21:45:37.466464Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12713136-e9f20f8d-a9b71b08-95f54811_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:37.478319Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12713136-e9f20f8d-a9b71b08-95f54811_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:37.478664Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:37.478683Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 559
2025-10-09T21:45:37.482340Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 559 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:37.482396Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 559 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:37.482402Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=559 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:37.482406Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:37.482804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 559 sessionId:  describe result for acl check
2025-10-09T21:45:37.482837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:37.482839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:37.482841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:37.482847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341458109112454:6226] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:37.483429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341458109112454:6226] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:37.490325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341458109112454:6226] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:37.496131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341458109112454:6226] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:37.496148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341458109112454:6226] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:37.496154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341458109112454:6226] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:37.496164Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 559 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:37.507329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:37.508444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:37.508680Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:37.508684Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:37.508695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:37.508744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:37.508782Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341457747299433:5679]
2025-10-09T21:45:37.509303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:37.509793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:37.509831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:37.510125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:37.510192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:37.510193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:37.510195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:37.510196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:37.510197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:37.510198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:37.510199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.510201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.510202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.510207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:37.510218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:37.510222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:37.510225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:37.510229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:37.510232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:37.510236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.510239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.510242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.510249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:37.510257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:37.510262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:37.510266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:37.510269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:37.510275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:37.510279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:37.510283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:37.510287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:37.510290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:37.510294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:37.510298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:37.510301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.510305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.510309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.510313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.510316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.510321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.510348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:37.537774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:37.539111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:37.539132Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:37.539134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:37.539139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:37.539147Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 558 [2:7559341457747299433:5679]
2025-10-09T21:45:37.539162Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:37.539181Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:37.539185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:37.539189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:37.539191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:37.539210Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:37.539217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:37.539225Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:37.539288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.539300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.539302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.539305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:37.539379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:37.539388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.539402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.539413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.539433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:37.539455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341457747299427:5679]
2025-10-09T21:45:37.539463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341457747299427:5679]
2025-10-09T21:45:37.539465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341457747299427:5679]
2025-10-09T21:45:37.539481Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:37.539496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:37.539512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:37.539515Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:37.539558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341457747299427:5679]
2025-10-09T21:45:37.539561Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:37.539878Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:37.539880Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:37.539883Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:37.539890Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:37.540061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341458109112533:6226], now have 1 active actors on pipe
2025-10-09T21:45:37.540069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341458109112424:2432], now have 1 active actors on pipe
2025-10-09T21:45:37.540199Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 558
2025-10-09T21:45:37.540239Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 558
2025-10-09T21:45:37.541236Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 559 partition: 0 MaxSeqNo: 25 sessionId: src_id|ef234feb-82554458-cfc1d8bb-8cb5b3e5_0
2025-10-09T21:45:37.541669Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12713136-e9f20f8d-a9b71b08-95f54811_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ef234feb-82554458-cfc1d8bb-8cb5b3e5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:37.544261Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef234feb-82554458-cfc1d8bb-8cb5b3e5_0] Write session: try to update token
2025-10-09T21:45:37.544282Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef234feb-82554458-cfc1d8bb-8cb5b3e5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:37.540591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:37.540607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:37.540647Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ef234feb-82554458-cfc1d8bb-8cb5b3e5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:37.540658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:37.540662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:37.540665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:37.540677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:37.540692Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:37.540709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:37.540968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:37.540974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:37.541014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:37.566091Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341458109112559:5852]
2025-10-09T21:45:37.568436Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 558, pipe: [1:7559341458109112559:5852]
2025-10-09T21:45:37.568364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341458109112559:5852], now have 1 active actors on pipe
2025-10-09T21:45:37.574281Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:37.574297Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:37.574300Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:37.574302Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:37.574310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341457747299427:5679]
2025-10-09T21:45:37.574314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341457747299427:5679]
2025-10-09T21:45:37.574319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341457747299427:5679]
2025-10-09T21:45:37.574361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:37.574368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.574370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:37.574512Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:37.576177Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 559 sessionId: src_id|ef234feb-82554458-cfc1d8bb-8cb5b3e5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:37.578691Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:37.579649Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:37.579663Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:37.579667Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:37.772374Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:37.773066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:37.773084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:37.773579Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:37.773818Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:37.773834Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:37.773893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 559 reason: error at write: Disconnected sessionId: src_id|ef234feb-82554458-cfc1d8bb-8cb5b3e5_0
2025-10-09T21:45:37.774087Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:37.774127Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341458109112559:5852]
2025-10-09T21:45:37.774441Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef234feb-82554458-cfc1d8bb-8cb5b3e5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:37.774453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef234feb-82554458-cfc1d8bb-8cb5b3e5_0] Write session will restart in 0.006273s
2025-10-09T21:45:37.774484Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef234feb-82554458-cfc1d8bb-8cb5b3e5_0] Write session: Do CDS request
2025-10-09T21:45:37.774491Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef234feb-82554458-cfc1d8bb-8cb5b3e5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:37.776204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 559 sessionId: src_id|ef234feb-82554458-cfc1d8bb-8cb5b3e5_0 is DEAD
2025-10-09T21:45:37.782162Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef234feb-82554458-cfc1d8bb-8cb5b3e5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:37.782247Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:37.782261Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 560
2025-10-09T21:45:37.782496Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 560 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:37.782544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 560 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:37.782555Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=560 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:37.782560Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:37.785622Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 560 sessionId:  describe result for acl check
2025-10-09T21:45:37.785662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:37.785664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:37.785668Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:37.785673Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341458109112605:6238] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:37.786303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341458109112605:6238] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:37.795021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341458109112605:6238] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:37.802076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341458109112605:6238] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:37.802103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341458109112605:6238] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:37.802107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341458109112605:6238] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:37.802119Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 560 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:37.806598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:37.814475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:37.814750Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:37.814756Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:37.814770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:37.814839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:37.814883Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341457747299603:5693]
2025-10-09T21:45:37.815502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:37.821359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:37.821452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:37.821837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:37.821917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:37.821918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:37.821920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:37.821922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:37.821923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:37.821924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:37.821925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.821927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.821928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.821936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:37.821949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:37.821953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:37.821957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:37.821961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:37.821965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:37.821969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.821973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.821977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.821988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:37.821997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:37.822004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:37.822009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:37.822014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:37.822020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:37.822024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:37.822045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:37.822049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:37.822055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:37.822059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:37.822064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:37.822070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.822075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.822080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.822086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.822090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.822098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.822132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:37.838566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:37.840148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:37.840156Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:37.840159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:37.840164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:37.840171Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 559 [2:7559341457747299603:5693]
2025-10-09T21:45:37.840187Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:37.840209Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:37.840212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:37.840216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:37.840219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:37.840238Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:37.840245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:37.840254Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:37.840326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.840329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.840331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.840335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:37.840421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:37.840424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.840441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:37.840454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:37.840473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:37.840495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341457747299598:5693]
2025-10-09T21:45:37.840546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341457747299598:5693]
2025-10-09T21:45:37.840550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341457747299598:5693]
2025-10-09T21:45:37.840566Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:37.840586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:37.840605Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:37.840608Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:37.840623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341458109112712:6238], now have 1 active actors on pipe
2025-10-09T21:45:37.840627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341458109112591:2432], now have 1 active actors on pipe
2025-10-09T21:45:37.840670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341457747299598:5693]
2025-10-09T21:45:37.840673Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:37.841260Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:37.841265Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:37.841269Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:37.841278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:37.840927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 559
2025-10-09T21:45:37.840963Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 559
2025-10-09T21:45:37.846014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:37.846048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:37.846096Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|19029193-b78056df-cf10f510-fda978b9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:37.846109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:37.846115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:37.846118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:37.846134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:37.846165Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:37.846188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:37.848458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:37.848728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 560 partition: 0 MaxSeqNo: 25 sessionId: src_id|19029193-b78056df-cf10f510-fda978b9_0
2025-10-09T21:45:37.848468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:37.848504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:37.849158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef234feb-82554458-cfc1d8bb-8cb5b3e5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|19029193-b78056df-cf10f510-fda978b9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:37.851845Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19029193-b78056df-cf10f510-fda978b9_0] Write session: try to update token
2025-10-09T21:45:37.851868Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19029193-b78056df-cf10f510-fda978b9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:37.870430Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:37.870449Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:37.870452Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:37.873558Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341458109112729:5852]
2025-10-09T21:45:37.870454Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:37.870463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341457747299598:5693]
2025-10-09T21:45:37.870469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341457747299598:5693]
2025-10-09T21:45:37.870473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341457747299598:5693]
2025-10-09T21:45:37.870526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:37.870534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:37.870536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:37.870669Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:37.874764Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 559, pipe: [1:7559341458109112729:5852]
2025-10-09T21:45:37.874478Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:37.874493Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:37.874497Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:37.883561Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 560 sessionId: src_id|19029193-b78056df-cf10f510-fda978b9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:37.886250Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:38.086766Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:38.087074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341458109112729:5852], now have 1 active actors on pipe
2025-10-09T21:45:38.087109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:38.087119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:38.089696Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:38.088161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:38.088179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:38.088241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 560 reason: error at write: Disconnected sessionId: src_id|19029193-b78056df-cf10f510-fda978b9_0
2025-10-09T21:45:38.088439Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 560 sessionId: src_id|19029193-b78056df-cf10f510-fda978b9_0 is DEAD
2025-10-09T21:45:38.088591Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:38.088627Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341458109112729:5852]
2025-10-09T21:45:38.090203Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19029193-b78056df-cf10f510-fda978b9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:38.090215Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19029193-b78056df-cf10f510-fda978b9_0] Write session will restart in 0.009308s
2025-10-09T21:45:38.090245Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19029193-b78056df-cf10f510-fda978b9_0] Write session: Do CDS request
2025-10-09T21:45:38.090251Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19029193-b78056df-cf10f510-fda978b9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:38.101973Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19029193-b78056df-cf10f510-fda978b9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:38.102148Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:38.102183Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 561
2025-10-09T21:45:38.106472Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 561 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:38.106527Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 561 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:38.106534Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=561 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:38.106538Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:38.110514Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 561 sessionId:  describe result for acl check
2025-10-09T21:45:38.110570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:38.110572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:38.110575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:38.110582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341462404080071:6243] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:38.111182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341462404080071:6243] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:38.129872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341462404080071:6243] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:38.130541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:38.130762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:38.131587Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:38.131592Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:38.131611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:38.131686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:38.131732Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341462042267031:5695]
2025-10-09T21:45:38.132328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:38.132888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:38.132942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:38.133295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:38.133352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:38.133353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:38.133355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:38.133357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:38.133358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:38.133359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:38.133361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.133362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.133364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.133370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:38.133383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:38.133387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:38.133392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:38.133396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:38.133400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:38.133405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.133409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.133413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.133421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:38.133430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:38.133438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:38.133443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:38.133450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:38.133456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:38.133460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:38.133466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:38.133472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:38.133477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:38.133483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:38.133489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:38.133497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.133503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.133508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.133514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.133519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.133524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.133555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:38.144553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341462404080071:6243] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:38.144577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341462404080071:6243] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:38.144582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341462404080071:6243] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:38.144593Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 561 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:38.146843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:38.148078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:38.148086Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:38.148090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:38.148096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:38.148104Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 560 [2:7559341462042267031:5695]
2025-10-09T21:45:38.148122Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:38.148145Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:38.148149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:38.148153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:38.148156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:38.148178Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:38.148185Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:38.148194Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:38.148276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.148279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.148284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.148287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:38.148380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:38.148383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.148400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.148411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.148428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:38.148451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341462042267026:5695]
2025-10-09T21:45:38.148454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341462042267026:5695]
2025-10-09T21:45:38.148456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341462042267026:5695]
2025-10-09T21:45:38.148472Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:38.148490Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:38.148506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:38.148509Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:38.148557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341462042267026:5695]
2025-10-09T21:45:38.148560Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:38.148866Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:38.148875Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:38.148877Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:38.148884Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:38.149250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341462404080172:6243], now have 1 active actors on pipe
2025-10-09T21:45:38.149257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341462404080038:2432], now have 1 active actors on pipe
2025-10-09T21:45:38.150217Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 560
2025-10-09T21:45:38.150256Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 560
2025-10-09T21:45:38.154207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:38.154230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:38.154276Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|55f65dbb-9eba8464-d38c2bbf-d83c8a2e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:38.154292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:38.154298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:38.154302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:38.154320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:38.154344Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:38.154369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:38.158063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:38.158086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:38.158134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:38.158852Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19029193-b78056df-cf10f510-fda978b9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|55f65dbb-9eba8464-d38c2bbf-d83c8a2e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:38.161514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55f65dbb-9eba8464-d38c2bbf-d83c8a2e_0] Write session: try to update token
2025-10-09T21:45:38.161539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55f65dbb-9eba8464-d38c2bbf-d83c8a2e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:38.158387Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 561 partition: 0 MaxSeqNo: 25 sessionId: src_id|55f65dbb-9eba8464-d38c2bbf-d83c8a2e_0
2025-10-09T21:45:38.189489Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341462404080178:5852]
2025-10-09T21:45:38.188574Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:38.188593Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:38.188596Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:38.188602Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:38.188612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341462042267026:5695]
2025-10-09T21:45:38.188617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341462042267026:5695]
2025-10-09T21:45:38.188623Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341462042267026:5695]
2025-10-09T21:45:38.188671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:38.188678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.188681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:38.188822Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:38.194304Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:38.194318Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:38.194322Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:38.194547Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 560, pipe: [1:7559341462404080178:5852]
2025-10-09T21:45:38.209075Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 561 sessionId: src_id|55f65dbb-9eba8464-d38c2bbf-d83c8a2e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:38.211791Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:38.478447Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:38.479163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341462404080178:5852], now have 1 active actors on pipe
2025-10-09T21:45:38.479198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:38.479205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:38.479653Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:38.479952Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341462404080178:5852]
2025-10-09T21:45:38.480005Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:38.480008Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:38.480670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55f65dbb-9eba8464-d38c2bbf-d83c8a2e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:38.480693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55f65dbb-9eba8464-d38c2bbf-d83c8a2e_0] Write session will restart in 0.007736s
2025-10-09T21:45:38.480065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 561 reason: error at write: Disconnected sessionId: src_id|55f65dbb-9eba8464-d38c2bbf-d83c8a2e_0
2025-10-09T21:45:38.480255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 561 sessionId: src_id|55f65dbb-9eba8464-d38c2bbf-d83c8a2e_0 is DEAD
2025-10-09T21:45:38.480432Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:38.480716Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55f65dbb-9eba8464-d38c2bbf-d83c8a2e_0] Write session: Do CDS request
2025-10-09T21:45:38.480795Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55f65dbb-9eba8464-d38c2bbf-d83c8a2e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:38.489748Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55f65dbb-9eba8464-d38c2bbf-d83c8a2e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:38.489894Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:38.489930Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 562
2025-10-09T21:45:38.490113Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 562 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:38.490163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 562 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:38.490171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=562 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:38.490175Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:38.490598Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 562 sessionId:  describe result for acl check
2025-10-09T21:45:38.490648Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:38.490650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:38.490652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:38.490657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341462404080191:6247] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:38.491234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341462404080191:6247] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:38.495288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341462404080191:6247] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:38.503390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341462404080191:6247] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:38.503409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341462404080191:6247] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:38.503415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341462404080191:6247] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:38.503426Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 562 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:38.519015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:38.524694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:38.524979Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:38.524984Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:38.525000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:38.525077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:38.525123Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341462042267170:5699]
2025-10-09T21:45:38.525717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:38.526866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:38.526929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:38.527316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:38.527378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:38.527380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:38.527381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:38.527383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:38.527384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:38.527386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:38.527387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.527391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.527393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.527399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:38.527410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:38.527414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:38.527418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:38.527421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:38.527426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:38.527429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.527433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.527437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.527445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:38.527453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:38.527460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:38.527465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:38.527469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:38.527474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:38.527478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:38.527483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:38.527489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:38.527494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:38.527498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:38.527503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:38.527508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.527513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.527517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.527522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.527529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.527534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.527563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:38.536463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:38.538119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:38.538131Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:38.538135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:38.538139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:38.538162Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 561 [2:7559341462042267170:5699]
2025-10-09T21:45:38.538183Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:38.538208Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:38.538212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:38.538215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:38.538217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:38.538239Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:38.538246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:38.538254Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:38.538332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.538334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.538336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.538338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:38.538428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:38.538431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.538447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.538456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.538471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:38.538494Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341462042267164:5699]
2025-10-09T21:45:38.538497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341462042267164:5699]
2025-10-09T21:45:38.538499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341462042267164:5699]
2025-10-09T21:45:38.538559Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:38.538563Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:38.538565Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:38.539379Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 561
2025-10-09T21:45:38.539441Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 561
2025-10-09T21:45:38.540027Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:38.540068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:38.540260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:38.540265Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:38.540285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341462404080184:2432], now have 1 active actors on pipe
2025-10-09T21:45:38.540290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341462404080271:6247], now have 1 active actors on pipe
2025-10-09T21:45:38.540340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341462042267164:5699]
2025-10-09T21:45:38.540347Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:38.540559Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:38.541136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:38.541147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:38.541189Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|15384702-568ec84c-78910bec-d7a20c3d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:38.541202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:38.541207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:38.541211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:38.541225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:38.541244Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:38.541265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:38.541768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:38.541777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:38.541806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:38.541986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 562 partition: 0 MaxSeqNo: 25 sessionId: src_id|15384702-568ec84c-78910bec-d7a20c3d_0
2025-10-09T21:45:38.542875Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55f65dbb-9eba8464-d38c2bbf-d83c8a2e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|15384702-568ec84c-78910bec-d7a20c3d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:38.545520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15384702-568ec84c-78910bec-d7a20c3d_0] Write session: try to update token
2025-10-09T21:45:38.545543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15384702-568ec84c-78910bec-d7a20c3d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:38.571472Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:38.571492Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:38.571495Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:38.571497Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:38.571506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341462042267164:5699]
2025-10-09T21:45:38.571510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341462042267164:5699]
2025-10-09T21:45:38.571514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341462042267164:5699]
2025-10-09T21:45:38.571562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:38.571572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.571576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:38.571719Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:38.572825Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:38.572832Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:38.572836Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:38.578990Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 562 sessionId: src_id|15384702-568ec84c-78910bec-d7a20c3d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:38.581694Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:38.581781Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341462404080325:5852]
2025-10-09T21:45:38.590410Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 561, pipe: [1:7559341462404080325:5852]
2025-10-09T21:45:38.811964Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:38.812474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:38.812491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:38.812529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341462404080325:5852], now have 1 active actors on pipe
2025-10-09T21:45:38.813015Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:38.813660Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:38.813715Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341462404080325:5852]
2025-10-09T21:45:38.813735Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:38.813819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 562 reason: status is not ok: tablet will be restarted right now sessionId: src_id|15384702-568ec84c-78910bec-d7a20c3d_0
2025-10-09T21:45:38.814010Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 562 sessionId: src_id|15384702-568ec84c-78910bec-d7a20c3d_0 is DEAD
2025-10-09T21:45:38.814362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15384702-568ec84c-78910bec-d7a20c3d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:38.814375Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15384702-568ec84c-78910bec-d7a20c3d_0] Write session will restart in 0.005912s
2025-10-09T21:45:38.814411Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15384702-568ec84c-78910bec-d7a20c3d_0] Write session: Do CDS request
2025-10-09T21:45:38.814418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15384702-568ec84c-78910bec-d7a20c3d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:38.822752Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15384702-568ec84c-78910bec-d7a20c3d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:38.823687Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:38.823708Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 563
2025-10-09T21:45:38.824122Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 563 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:38.824162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 563 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:38.824168Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=563 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:38.824172Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:38.824790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 563 sessionId:  describe result for acl check
2025-10-09T21:45:38.824827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:38.824829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:38.824830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:38.824839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341462404080368:6259] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:38.826126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341462404080368:6259] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:38.829914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341462404080368:6259] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:38.836053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341462404080368:6259] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:38.836079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341462404080368:6259] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:38.836084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341462404080368:6259] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:38.836095Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 563 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:38.852793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:38.852993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:38.853204Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:38.853214Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:38.853227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:38.853276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:38.853314Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341462042267343:5713]
2025-10-09T21:45:38.853968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:38.854582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:38.854627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:38.854955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:38.855009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:38.855014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:38.855016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:38.855017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:38.855018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:38.855020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:38.855021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.855022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.855024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.855029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:38.855042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:38.855046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:38.855051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:38.855057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:38.855062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:38.855082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.855086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.855090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.855097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:38.855106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:38.855113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:38.855119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:38.855123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:38.855129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:38.855133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:38.855139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:38.855145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:38.855151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:38.855156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:38.855162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:38.855167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.855173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.855177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.855183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.855188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.855193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.855227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:38.863620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:38.865246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:38.865266Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:38.865270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:38.865275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:38.865284Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 562 [2:7559341462042267343:5713]
2025-10-09T21:45:38.865301Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:38.865321Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:38.865327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:38.865331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:38.865333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:38.865352Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:38.865364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:38.865372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:38.865443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.865451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.865453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.865456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:38.865534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:38.865542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.865557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:38.865571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:38.865595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:38.865620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341462042267338:5713]
2025-10-09T21:45:38.865628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341462042267338:5713]
2025-10-09T21:45:38.865631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341462042267338:5713]
2025-10-09T21:45:38.865644Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:38.865664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:38.865684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:38.865691Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:38.865731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341462042267338:5713]
2025-10-09T21:45:38.865739Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:38.866702Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 562
2025-10-09T21:45:38.866745Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 562
2025-10-09T21:45:38.867947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 563 partition: 0 MaxSeqNo: 25 sessionId: src_id|89d661e1-a5da413e-954eff4c-976ea5b1_0
2025-10-09T21:45:38.868310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15384702-568ec84c-78910bec-d7a20c3d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|89d661e1-a5da413e-954eff4c-976ea5b1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:38.865931Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:38.865939Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:38.865941Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:38.865948Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:38.866920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341462404080355:2432], now have 1 active actors on pipe
2025-10-09T21:45:38.866927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341462404080428:6259], now have 1 active actors on pipe
2025-10-09T21:45:38.867390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:38.867397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:38.867428Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|89d661e1-a5da413e-954eff4c-976ea5b1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:38.867439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:38.867444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:38.867447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:38.867459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:38.867474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:38.867492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:38.867808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:38.867812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:38.867826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:38.874351Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|89d661e1-a5da413e-954eff4c-976ea5b1_0] Write session: try to update token
2025-10-09T21:45:38.874381Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|89d661e1-a5da413e-954eff4c-976ea5b1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:38.886557Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:38.886576Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:38.886579Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:38.886581Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:38.886590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341462042267338:5713]
2025-10-09T21:45:38.886595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341462042267338:5713]
2025-10-09T21:45:38.886599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341462042267338:5713]
2025-10-09T21:45:38.886648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:38.886656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:38.886659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:38.886799Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:38.890374Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:38.890385Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:38.890389Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:38.904667Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 563 sessionId: src_id|89d661e1-a5da413e-954eff4c-976ea5b1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:38.907268Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:38.914455Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341462404080498:5852]
2025-10-09T21:45:38.914972Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 562, pipe: [1:7559341462404080498:5852]
2025-10-09T21:45:39.150495Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:39.150780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:39.150792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:39.150838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341462404080498:5852], now have 1 active actors on pipe
2025-10-09T21:45:39.151962Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89d661e1-a5da413e-954eff4c-976ea5b1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:39.151975Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89d661e1-a5da413e-954eff4c-976ea5b1_0] Write session will restart in 0.007140s
2025-10-09T21:45:39.152002Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89d661e1-a5da413e-954eff4c-976ea5b1_0] Write session: Do CDS request
2025-10-09T21:45:39.152008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89d661e1-a5da413e-954eff4c-976ea5b1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:39.152038Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:39.151444Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:39.151504Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341462404080498:5852]
2025-10-09T21:45:39.151526Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:39.151529Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:39.151605Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 563 reason: error at write: Disconnected sessionId: src_id|89d661e1-a5da413e-954eff4c-976ea5b1_0
2025-10-09T21:45:39.151801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 563 sessionId: src_id|89d661e1-a5da413e-954eff4c-976ea5b1_0 is DEAD
2025-10-09T21:45:39.160665Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|89d661e1-a5da413e-954eff4c-976ea5b1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:39.164808Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:39.164828Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 564
2025-10-09T21:45:39.165043Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 564 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:39.165076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 564 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:39.165083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=564 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:39.165087Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:39.165896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 564 sessionId:  describe result for acl check
2025-10-09T21:45:39.165938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:39.165941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:39.165943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:39.165949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341466699047823:6264] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:39.166559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341466699047823:6264] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:39.172034Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341466699047823:6264] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:39.180217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341466699047823:6264] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:39.180236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341466699047823:6264] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:39.180241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341466699047823:6264] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:39.180252Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 564 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:39.184974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:39.186139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:39.186331Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:39.186340Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:39.186353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:39.186397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:39.186443Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341466337234775:5715]
2025-10-09T21:45:39.186992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:39.187455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:39.187493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:39.187775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:39.187811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:39.187812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:39.187813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:39.187814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:39.187815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:39.187817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:39.187818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.187819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.187820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.187824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:39.187835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:39.187838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:39.187842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:39.187845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:39.187850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:39.187853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.187856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.187859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.187865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:39.187874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:39.187879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:39.187883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:39.187886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:39.187892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:39.187896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:39.187900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:39.187904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:39.187908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:39.187912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:39.187916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:39.187921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.187925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.187929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.187934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.187939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.187943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.187968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:39.200427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:39.201526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:39.201531Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:39.201533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:39.201552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:39.201559Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 563 [2:7559341466337234775:5715]
2025-10-09T21:45:39.201579Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:39.201599Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:39.201603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:39.201605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:39.201608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:39.201626Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:39.201632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:39.201638Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:39.201698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.201702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.201704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.201707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:39.201775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:39.201777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.201788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.201798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.201821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:39.201842Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341466337234769:5715]
2025-10-09T21:45:39.201845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341466337234769:5715]
2025-10-09T21:45:39.201847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341466337234769:5715]
2025-10-09T21:45:39.201860Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:39.201875Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:39.201889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:39.201893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:39.201932Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341466337234769:5715]
2025-10-09T21:45:39.201934Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:39.206116Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:39.206130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:39.206133Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:39.206140Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:39.206447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341466699047919:6264], now have 1 active actors on pipe
2025-10-09T21:45:39.206455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341466699047807:2432], now have 1 active actors on pipe
2025-10-09T21:45:39.206785Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 563
2025-10-09T21:45:39.206826Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 563
2025-10-09T21:45:39.206897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:39.206914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:39.206959Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b7ee4192-e9cccb9e-a65ce3c2-baa2fe41_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:39.206976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:39.206986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:39.206990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:39.207023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:39.207042Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:39.207078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:39.208140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:39.208154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:39.208191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:39.209224Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89d661e1-a5da413e-954eff4c-976ea5b1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b7ee4192-e9cccb9e-a65ce3c2-baa2fe41_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:39.208509Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 564 partition: 0 MaxSeqNo: 25 sessionId: src_id|b7ee4192-e9cccb9e-a65ce3c2-baa2fe41_0
2025-10-09T21:45:39.212108Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7ee4192-e9cccb9e-a65ce3c2-baa2fe41_0] Write session: try to update token
2025-10-09T21:45:39.212131Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7ee4192-e9cccb9e-a65ce3c2-baa2fe41_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:39.228709Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:39.228729Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:39.228733Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:39.228735Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:39.228745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341466337234769:5715]
2025-10-09T21:45:39.228750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341466337234769:5715]
2025-10-09T21:45:39.228753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341466337234769:5715]
2025-10-09T21:45:39.228804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:39.228812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.228815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:39.228824Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:39.228839Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:39.228843Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:39.228964Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:39.251509Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341466699047952:5852]
2025-10-09T21:45:39.255121Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 564 sessionId: src_id|b7ee4192-e9cccb9e-a65ce3c2-baa2fe41_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:39.257847Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:39.258349Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 563, pipe: [1:7559341466699047952:5852]
2025-10-09T21:45:39.552042Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:39.552769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341466699047952:5852], now have 1 active actors on pipe
2025-10-09T21:45:39.552798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:39.552806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:39.553078Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:39.553520Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:39.553573Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341466699047952:5852]
2025-10-09T21:45:39.553591Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:39.553634Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 564 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b7ee4192-e9cccb9e-a65ce3c2-baa2fe41_0
2025-10-09T21:45:39.553832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 564 sessionId: src_id|b7ee4192-e9cccb9e-a65ce3c2-baa2fe41_0 is DEAD
2025-10-09T21:45:39.558144Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7ee4192-e9cccb9e-a65ce3c2-baa2fe41_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:39.558174Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7ee4192-e9cccb9e-a65ce3c2-baa2fe41_0] Write session will restart in 0.009879s
2025-10-09T21:45:39.558206Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7ee4192-e9cccb9e-a65ce3c2-baa2fe41_0] Write session: Do CDS request
2025-10-09T21:45:39.558214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7ee4192-e9cccb9e-a65ce3c2-baa2fe41_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:39.569654Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7ee4192-e9cccb9e-a65ce3c2-baa2fe41_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:39.570380Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:39.570398Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 565
2025-10-09T21:45:39.574229Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 565 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:39.574303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 565 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:39.574309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=565 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:39.574314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:39.574909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 565 sessionId:  describe result for acl check
2025-10-09T21:45:39.574941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:39.574943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:39.574945Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:39.574951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341466699047990:6268] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:39.575539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341466699047990:6268] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:39.581007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341466699047990:6268] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:39.584518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:39.584161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341466699047990:6268] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:39.584180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341466699047990:6268] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:39.584184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341466699047990:6268] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:39.584194Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 565 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:39.589586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:39.589858Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:39.589862Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:39.589876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:39.589936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:39.589977Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341466337234916:5719]
2025-10-09T21:45:39.590590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:39.591016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:39.591073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:39.591425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:39.591494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:39.591496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:39.591497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:39.591498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:39.591500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:39.591501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:39.591502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.591503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.591504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.591510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:39.591521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:39.591526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:39.591530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:39.591534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:39.591538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:39.591542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.591545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.591548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.591556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:39.591565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:39.591571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:39.591576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:39.591581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:39.591586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:39.591589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:39.591594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:39.591597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:39.591601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:39.591605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:39.591610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:39.592187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.592214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.592220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.592226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.592232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.592239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.592274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:39.603172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:39.604285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:39.604304Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:39.604308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:39.604313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:39.604321Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 564 [2:7559341466337234916:5719]
2025-10-09T21:45:39.604339Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:39.604362Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:39.604365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:39.604369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:39.604373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:39.604396Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:39.604403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:39.604412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:39.604498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.604501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.604503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.604507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:39.605141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:39.605144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.605161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.605175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.605197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:39.605221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341466337234909:5719]
2025-10-09T21:45:39.605224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341466337234909:5719]
2025-10-09T21:45:39.605226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341466337234909:5719]
2025-10-09T21:45:39.605243Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:39.605260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:39.605278Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:39.605282Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:39.605327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341466337234909:5719]
2025-10-09T21:45:39.605330Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:39.605729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:39.605735Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:39.605737Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:39.605743Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:39.608229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341466699048088:6268], now have 1 active actors on pipe
2025-10-09T21:45:39.608241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341466699047957:2432], now have 1 active actors on pipe
2025-10-09T21:45:39.608267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:39.608275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:39.608318Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|399f8829-baf8303e-53d52b43-1355b86_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:39.608332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:39.608347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:39.608351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:39.608368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:39.608388Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:39.608413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:39.609012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:39.609030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:39.609049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:39.609924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7ee4192-e9cccb9e-a65ce3c2-baa2fe41_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|399f8829-baf8303e-53d52b43-1355b86_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:39.607881Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 564
2025-10-09T21:45:39.607922Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 564
2025-10-09T21:45:39.609248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 565 partition: 0 MaxSeqNo: 25 sessionId: src_id|399f8829-baf8303e-53d52b43-1355b86_0
2025-10-09T21:45:39.612916Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|399f8829-baf8303e-53d52b43-1355b86_0] Write session: try to update token
2025-10-09T21:45:39.612939Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|399f8829-baf8303e-53d52b43-1355b86_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:39.631722Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:39.631742Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:39.631745Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:39.631747Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:39.631759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341466337234909:5719]
2025-10-09T21:45:39.631764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341466337234909:5719]
2025-10-09T21:45:39.631774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341466337234909:5719]
2025-10-09T21:45:39.631822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:39.631830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.631833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:39.631985Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:39.634603Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:39.634616Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:39.634620Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:39.649480Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 565 sessionId: src_id|399f8829-baf8303e-53d52b43-1355b86_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:39.652510Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:39.654186Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341466699048103:5852]
2025-10-09T21:45:39.654402Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 564, pipe: [1:7559341466699048103:5852]
2025-10-09T21:45:39.823571Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:39.824050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:39.824061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:39.824083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341466699048103:5852], now have 1 active actors on pipe
2025-10-09T21:45:39.824411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:39.824427Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:39.824498Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 565 reason: error at write: Disconnected sessionId: src_id|399f8829-baf8303e-53d52b43-1355b86_0
2025-10-09T21:45:39.824423Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:39.824703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 565 sessionId: src_id|399f8829-baf8303e-53d52b43-1355b86_0 is DEAD
2025-10-09T21:45:39.824851Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341466699048103:5852]
2025-10-09T21:45:39.824865Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:39.826272Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|399f8829-baf8303e-53d52b43-1355b86_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:39.826285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|399f8829-baf8303e-53d52b43-1355b86_0] Write session will restart in 0.006234s
2025-10-09T21:45:39.826314Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|399f8829-baf8303e-53d52b43-1355b86_0] Write session: Do CDS request
2025-10-09T21:45:39.826320Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|399f8829-baf8303e-53d52b43-1355b86_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:39.838571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|399f8829-baf8303e-53d52b43-1355b86_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:39.842264Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:39.842286Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 566
2025-10-09T21:45:39.845231Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 566 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:39.845293Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 566 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:39.845305Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=566 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:39.845310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:39.845958Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 566 sessionId:  describe result for acl check
2025-10-09T21:45:39.846002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:39.846005Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:39.846006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:39.846013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341466699048148:6279] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:39.846639Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341466699048148:6279] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:39.853716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341466699048148:6279] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:39.854669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:39.862239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341466699048148:6279] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:39.862261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341466699048148:6279] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:39.862265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341466699048148:6279] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:39.862276Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 566 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:39.862117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:39.862404Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:39.862408Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:39.862420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:39.862490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:39.862534Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341466337235083:5731]
2025-10-09T21:45:39.863138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:39.863489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:39.863567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:39.863938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:39.864396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:39.864416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:39.864418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:39.864419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:39.864420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:39.864421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:39.864423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.864424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.864425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.864432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:39.864443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:39.864447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:39.864451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:39.864455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:39.864459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:39.864463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.864467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.864471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.864480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:39.864488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:39.864494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:39.864500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:39.864504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:39.864509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:39.864513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:39.864519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:39.864524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:39.864529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:39.864534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:39.864539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:39.864546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.864551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.864556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.864561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.864566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.864571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.864597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:39.873541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:39.874718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:39.874738Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:39.874741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:39.874746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:39.874756Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 565 [2:7559341466337235083:5731]
2025-10-09T21:45:39.874772Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:39.874791Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:39.874794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:39.874798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:39.874801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:39.874820Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:39.874826Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:39.874835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:39.874897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.874910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.874912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.874915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:39.875765Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 565
2025-10-09T21:45:39.875814Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 565
2025-10-09T21:45:39.876183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:39.876195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.876212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:39.876228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:39.876257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:39.876288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341466699048122:2432], now have 1 active actors on pipe
2025-10-09T21:45:39.876292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341466699048253:6279], now have 1 active actors on pipe
2025-10-09T21:45:39.876310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:39.876322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:39.876345Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341466337235078:5731]
2025-10-09T21:45:39.876348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341466337235078:5731]
2025-10-09T21:45:39.876350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341466337235078:5731]
2025-10-09T21:45:39.876365Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:39.876383Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:39.876399Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:39.876405Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:39.876426Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|249bc129-369fb619-a99af9b3-2454e8c5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:39.876436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:39.876440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:39.876443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:39.876454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:39.876473Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:39.876521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341466337235078:5731]
2025-10-09T21:45:39.876530Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:39.876545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:39.876624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:39.876634Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:39.876637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:39.876644Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:39.877878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 566 partition: 0 MaxSeqNo: 25 sessionId: src_id|249bc129-369fb619-a99af9b3-2454e8c5_0
2025-10-09T21:45:39.877271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:39.877293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:39.877316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:39.882172Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|399f8829-baf8303e-53d52b43-1355b86_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|249bc129-369fb619-a99af9b3-2454e8c5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:39.884731Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|249bc129-369fb619-a99af9b3-2454e8c5_0] Write session: try to update token
2025-10-09T21:45:39.884753Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|249bc129-369fb619-a99af9b3-2454e8c5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:39.901122Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:39.901141Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:39.901145Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:39.901147Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:39.901156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341466337235078:5731]
2025-10-09T21:45:39.901161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341466337235078:5731]
2025-10-09T21:45:39.901165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341466337235078:5731]
2025-10-09T21:45:39.901220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:39.901227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:39.901230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:39.901377Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:39.904763Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:39.904779Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:39.904783Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:39.919859Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 566 sessionId: src_id|249bc129-369fb619-a99af9b3-2454e8c5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:39.922390Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:39.924976Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341466699048279:5852]
2025-10-09T21:45:39.925903Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 565, pipe: [1:7559341466699048279:5852]
2025-10-09T21:45:40.216501Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:40.217438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:40.217459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:40.217884Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341466699048279:5852]
2025-10-09T21:45:40.217926Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:40.217928Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:40.218013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 566 reason: error at write: Disconnected sessionId: src_id|249bc129-369fb619-a99af9b3-2454e8c5_0
2025-10-09T21:45:40.218243Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 566 sessionId: src_id|249bc129-369fb619-a99af9b3-2454e8c5_0 is DEAD
2025-10-09T21:45:40.218425Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:40.217486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341466699048279:5852], now have 1 active actors on pipe
2025-10-09T21:45:40.220396Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|249bc129-369fb619-a99af9b3-2454e8c5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:40.220409Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|249bc129-369fb619-a99af9b3-2454e8c5_0] Write session will restart in 0.007403s
2025-10-09T21:45:40.220471Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|249bc129-369fb619-a99af9b3-2454e8c5_0] Write session: Do CDS request
2025-10-09T21:45:40.220481Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|249bc129-369fb619-a99af9b3-2454e8c5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:40.224623Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:40.229777Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|249bc129-369fb619-a99af9b3-2454e8c5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:40.229853Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:40.229873Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 567
2025-10-09T21:45:40.230344Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 567 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:40.230410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 567 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:40.230416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=567 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:40.230423Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:40.231454Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 567 sessionId:  describe result for acl check
2025-10-09T21:45:40.231492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:40.231493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:40.231495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:40.231501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341470994015601:6285] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:40.232146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341470994015601:6285] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:40.243379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341470994015601:6285] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:40.250331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341470994015601:6285] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:40.250352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341470994015601:6285] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:40.250356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341470994015601:6285] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:40.250367Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 567 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:40.268640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:40.272175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:40.272522Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:40.272530Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:40.272545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:40.272633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:40.272686Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341470632202527:5735]
2025-10-09T21:45:40.273319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:40.274529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:40.274593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:40.274948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:40.275012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:40.275013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:40.275015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:40.275016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:40.275018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:40.275019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:40.275020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.275022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.275023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.275030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:40.275041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:40.275045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:40.275050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:40.275055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:40.275059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:40.275064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.275068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.275071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.275082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:40.275090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:40.275096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:40.275101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:40.275107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:40.275113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:40.275118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:40.275124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:40.275130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:40.275134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:40.275139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:40.275144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:40.275148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.275153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.275157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.275162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.275166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.275170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.275199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:40.285360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:40.288239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:40.288247Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:40.288250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:40.288255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:40.288264Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 566 [2:7559341470632202527:5735]
2025-10-09T21:45:40.288285Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:40.288306Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:40.288310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:40.288314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:40.288316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:40.288338Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:40.288345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:40.288353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:40.288775Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 566
2025-10-09T21:45:40.288816Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 566
2025-10-09T21:45:40.288432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.288435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.288438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.288441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:40.288527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:40.288530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.288545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.288556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.288572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:40.288593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341470632202522:5735]
2025-10-09T21:45:40.288600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341470632202522:5735]
2025-10-09T21:45:40.288602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341470632202522:5735]
2025-10-09T21:45:40.288611Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:40.288628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:40.288636Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:40.288642Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:40.288643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:40.288646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:40.288646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:40.288697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341470994015688:6285], now have 1 active actors on pipe
2025-10-09T21:45:40.288701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341470994015587:2432], now have 1 active actors on pipe
2025-10-09T21:45:40.288703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341470632202522:5735]
2025-10-09T21:45:40.288716Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:40.288726Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:40.294272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:40.294294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:40.294345Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1a73b555-827ed7d3-a395a2af-e6f16792_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:40.294360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:40.294365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:40.294369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:40.294384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:40.294406Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:40.294429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:40.298083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:40.298104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:40.298158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:40.298384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 567 partition: 0 MaxSeqNo: 25 sessionId: src_id|1a73b555-827ed7d3-a395a2af-e6f16792_0
2025-10-09T21:45:40.298848Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|249bc129-369fb619-a99af9b3-2454e8c5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1a73b555-827ed7d3-a395a2af-e6f16792_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:40.317845Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341470994015733:5852]
2025-10-09T21:45:40.315854Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:40.315872Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:40.315875Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:40.315881Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:40.315890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341470632202522:5735]
2025-10-09T21:45:40.315895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341470632202522:5735]
2025-10-09T21:45:40.315899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341470632202522:5735]
2025-10-09T21:45:40.315949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:40.315955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.315956Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:40.315958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:40.315962Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:40.315966Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:40.316093Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:40.320273Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 566, pipe: [1:7559341470994015733:5852]
2025-10-09T21:45:40.326631Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a73b555-827ed7d3-a395a2af-e6f16792_0] Write session: try to update token
2025-10-09T21:45:40.326658Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a73b555-827ed7d3-a395a2af-e6f16792_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:40.393322Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 567 sessionId: src_id|1a73b555-827ed7d3-a395a2af-e6f16792_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:40.395915Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:40.591487Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:40.592362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341470994015733:5852], now have 1 active actors on pipe
2025-10-09T21:45:40.592449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:40.592467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:40.592814Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341470994015733:5852]
2025-10-09T21:45:40.592864Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:40.592867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:40.592927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 567 reason: error at write: Disconnected sessionId: src_id|1a73b555-827ed7d3-a395a2af-e6f16792_0
2025-10-09T21:45:40.593289Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a73b555-827ed7d3-a395a2af-e6f16792_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:40.593302Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a73b555-827ed7d3-a395a2af-e6f16792_0] Write session will restart in 0.008495s
2025-10-09T21:45:40.593328Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a73b555-827ed7d3-a395a2af-e6f16792_0] Write session: Do CDS request
2025-10-09T21:45:40.593336Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a73b555-827ed7d3-a395a2af-e6f16792_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:40.593122Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 567 sessionId: src_id|1a73b555-827ed7d3-a395a2af-e6f16792_0 is DEAD
2025-10-09T21:45:40.593259Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:40.595905Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:40.602642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a73b555-827ed7d3-a395a2af-e6f16792_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:40.603126Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:40.603144Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 568
2025-10-09T21:45:40.603508Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 568 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:40.603556Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 568 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:40.603562Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=568 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:40.603566Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:40.603989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 568 sessionId:  describe result for acl check
2025-10-09T21:45:40.604025Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:40.604027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:40.604032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:40.604038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341470994015745:6289] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:40.604627Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341470994015745:6289] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:40.608860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341470994015745:6289] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:40.619505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341470994015745:6289] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:40.619525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341470994015745:6289] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:40.619529Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341470994015745:6289] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:40.619540Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 568 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:40.673809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:40.673996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:40.674208Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:40.674212Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:40.674228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:40.674292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:40.674331Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341470632202671:5739]
2025-10-09T21:45:40.674941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:40.675474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:40.675547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:40.675893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:40.675965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:40.675966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:40.675968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:40.675969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:40.675971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:40.675972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:40.675973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.675974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.675976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.675981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:40.675992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:40.675995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:40.676008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:40.676011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:40.676015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:40.676018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.676022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.676025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.676032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:40.676040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:40.676047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:40.676053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:40.676057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:40.676073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:40.676077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:40.676083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:40.676087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:40.676093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:40.676097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:40.676102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:40.676109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.676115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.676119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.676124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.676140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.676146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.676173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:40.692115Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 567
2025-10-09T21:45:40.692119Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 567
2025-10-09T21:45:40.692960Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341470994015884:5852]
2025-10-09T21:45:40.693564Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 567, pipe: [1:7559341470994015884:5852]
2025-10-09T21:45:40.693915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 568 partition: 0 MaxSeqNo: 25 sessionId: src_id|6400edec-f74d706c-ffbe17a-f2e0464b_0
2025-10-09T21:45:40.689088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:40.690304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:40.690325Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:40.690328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:40.690333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:40.690341Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 567 [2:7559341470632202671:5739]
2025-10-09T21:45:40.690357Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:40.690391Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:40.690395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:40.690398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:40.690401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:40.690419Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:40.690426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:40.690447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:40.690521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.690524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.690525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.690528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:40.691144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:40.691148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.691166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.691177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.691194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:40.691219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341470632202666:5739]
2025-10-09T21:45:40.691221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341470632202666:5739]
2025-10-09T21:45:40.691222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341470632202666:5739]
2025-10-09T21:45:40.691244Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:40.691259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:40.691275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:40.691277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:40.691320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341470632202666:5739]
2025-10-09T21:45:40.691323Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:40.691718Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:40.691723Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:40.691725Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:40.691731Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:40.693128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341470994015788:6289], now have 1 active actors on pipe
2025-10-09T21:45:40.693133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341470994015738:2432], now have 1 active actors on pipe
2025-10-09T21:45:40.693163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:40.693169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:40.693203Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6400edec-f74d706c-ffbe17a-f2e0464b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:40.693213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:40.693219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:40.693222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:40.693235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:40.693250Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:40.693269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:40.693535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341470994015884:5852], now have 1 active actors on pipe
2025-10-09T21:45:40.693772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:40.693776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:40.693796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:40.695254Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a73b555-827ed7d3-a395a2af-e6f16792_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6400edec-f74d706c-ffbe17a-f2e0464b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:40.697767Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6400edec-f74d706c-ffbe17a-f2e0464b_0] Write session: try to update token
2025-10-09T21:45:40.697789Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6400edec-f74d706c-ffbe17a-f2e0464b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:40.738297Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:40.738313Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:40.738315Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:40.738317Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:40.738327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341470632202666:5739]
2025-10-09T21:45:40.738331Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341470632202666:5739]
2025-10-09T21:45:40.738334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341470632202666:5739]
2025-10-09T21:45:40.738386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:40.738393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.738396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:40.738616Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:40.742411Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:40.742425Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:40.742429Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:40.774604Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 568 sessionId: src_id|6400edec-f74d706c-ffbe17a-f2e0464b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:40.777063Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:40.919044Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:40.919582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:40.919593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:40.919805Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:40.922391Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:40.922481Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 568 reason: status is not ok: tablet will be restarted right now sessionId: src_id|6400edec-f74d706c-ffbe17a-f2e0464b_0
2025-10-09T21:45:40.922661Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 568 sessionId: src_id|6400edec-f74d706c-ffbe17a-f2e0464b_0 is DEAD
2025-10-09T21:45:40.922792Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:40.922824Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341470994015884:5852]
2025-10-09T21:45:40.923245Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6400edec-f74d706c-ffbe17a-f2e0464b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:40.923254Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6400edec-f74d706c-ffbe17a-f2e0464b_0] Write session will restart in 0.008569s
2025-10-09T21:45:40.923278Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6400edec-f74d706c-ffbe17a-f2e0464b_0] Write session: Do CDS request
2025-10-09T21:45:40.923284Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6400edec-f74d706c-ffbe17a-f2e0464b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:40.938570Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6400edec-f74d706c-ffbe17a-f2e0464b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:40.939855Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:40.939873Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 569
2025-10-09T21:45:40.940071Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 569 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:40.940116Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 569 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:40.940122Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=569 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:40.940127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:40.940595Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 569 sessionId:  describe result for acl check
2025-10-09T21:45:40.940629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:40.940631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:40.940633Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:40.940639Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341470994015921:6300] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:40.941202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341470994015921:6300] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:40.950006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341470994015921:6300] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:40.955237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341470994015921:6300] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:40.955257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341470994015921:6300] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:40.955261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341470994015921:6300] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:40.955271Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 569 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:40.962679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:40.969128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:40.969576Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:40.969580Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:40.969595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:40.969665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:40.969704Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341470632202842:5751]
2025-10-09T21:45:40.970285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:40.981621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:40.981725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:40.982173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:40.982238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:40.982240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:40.982241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:40.982243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:40.982244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:40.982246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:40.982247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.982249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.982250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.982258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:40.982270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:40.982274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:40.982277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:40.982281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:40.982286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:40.982290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.982294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.982298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.982308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:40.982317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:40.982323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:40.982330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:40.982337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:40.982343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:40.982348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:40.982353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:40.982358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:40.982363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:40.982368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:40.982374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:40.982380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.982385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.982389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.982397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.982402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.982407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.982438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:40.993774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:40.995294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:40.995301Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:40.995304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:40.995308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:40.995316Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 568 [2:7559341470632202842:5751]
2025-10-09T21:45:40.995330Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:40.995351Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:40.995354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:40.995358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:40.995359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:40.995378Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:40.995384Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:40.995392Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:40.995464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.995466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.995468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.995471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:40.995562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:40.995565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:40.995580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:40.995590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:40.995604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:40.995626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341470632202836:5751]
2025-10-09T21:45:40.995629Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341470632202836:5751]
2025-10-09T21:45:40.995631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341470632202836:5751]
2025-10-09T21:45:40.995650Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:40.995665Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:40.995679Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:40.995682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:40.995726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341470632202836:5751]
2025-10-09T21:45:40.995728Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:40.996394Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:40.996397Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:40.996399Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:40.996405Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:40.996858Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 568
2025-10-09T21:45:40.996897Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 568
2025-10-09T21:45:40.997101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341470994016022:6300], now have 1 active actors on pipe
2025-10-09T21:45:40.997110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341470994015900:2432], now have 1 active actors on pipe
2025-10-09T21:45:40.997578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:40.997592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:40.997628Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f291add1-fc76309e-615e698b-72d36bea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:40.997638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:40.997643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:40.997646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:40.997666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:40.997680Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:40.997700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:41.002527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:41.002540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:41.002574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:41.002768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 569 partition: 0 MaxSeqNo: 25 sessionId: src_id|f291add1-fc76309e-615e698b-72d36bea_0
2025-10-09T21:45:41.006165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6400edec-f74d706c-ffbe17a-f2e0464b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f291add1-fc76309e-615e698b-72d36bea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:41.008802Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f291add1-fc76309e-615e698b-72d36bea_0] Write session: try to update token
2025-10-09T21:45:41.008828Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f291add1-fc76309e-615e698b-72d36bea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:41.022961Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341475288983352:5852]
2025-10-09T21:45:41.023093Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:41.023110Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:41.023112Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:41.023114Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:41.023124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341470632202836:5751]
2025-10-09T21:45:41.023129Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341470632202836:5751]
2025-10-09T21:45:41.023133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341470632202836:5751]
2025-10-09T21:45:41.023182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:41.023190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.023192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:41.023338Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:41.023477Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:41.023484Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:41.023488Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:41.028360Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 568, pipe: [1:7559341475288983352:5852]
2025-10-09T21:45:41.052082Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 569 sessionId: src_id|f291add1-fc76309e-615e698b-72d36bea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:41.054505Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:41.232988Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:41.233553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341475288983352:5852], now have 1 active actors on pipe
2025-10-09T21:45:41.233609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:41.233617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:41.235515Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341475288983352:5852]
2025-10-09T21:45:41.235554Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:41.235556Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:41.235618Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 569 reason: error at write: Disconnected sessionId: src_id|f291add1-fc76309e-615e698b-72d36bea_0
2025-10-09T21:45:41.235811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 569 sessionId: src_id|f291add1-fc76309e-615e698b-72d36bea_0 is DEAD
2025-10-09T21:45:41.236073Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:41.237216Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:41.237499Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f291add1-fc76309e-615e698b-72d36bea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:41.237510Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f291add1-fc76309e-615e698b-72d36bea_0] Write session will restart in 0.009367s
2025-10-09T21:45:41.237540Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f291add1-fc76309e-615e698b-72d36bea_0] Write session: Do CDS request
2025-10-09T21:45:41.237547Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f291add1-fc76309e-615e698b-72d36bea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:41.250563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f291add1-fc76309e-615e698b-72d36bea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:41.250916Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:41.250935Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 570
2025-10-09T21:45:41.251103Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 570 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:41.251133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 570 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:41.251139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=570 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:41.251142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:41.252447Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 570 sessionId:  describe result for acl check
2025-10-09T21:45:41.252494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:41.252496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:41.252498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:41.252506Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341475288983447:6306] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:41.253111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341475288983447:6306] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:41.260417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341475288983447:6306] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:41.261521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:41.263693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:41.263911Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:41.263921Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:41.263930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:41.263974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:41.264340Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341474927170288:5755]
2025-10-09T21:45:41.265451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:41.265952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:41.266006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:41.266398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:41.266481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:41.266482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:41.266484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:41.266485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:41.266487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:41.266488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:41.266489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.266491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.266492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.266499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:41.266531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:41.266536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:41.266540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:41.266545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:41.266549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:41.266554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.266558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.266562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.266571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:41.266580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:41.266587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:41.266593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:41.266598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:41.266605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:41.266610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:41.266615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:41.266620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:41.266626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:41.266632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:41.266638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:41.266644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.266650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.266655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.266660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.266665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.266670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.266697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:41.274681Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341475288983447:6306] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:41.274705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341475288983447:6306] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:41.274709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341475288983447:6306] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:41.274720Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 570 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:41.277816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:41.279547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:41.279553Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:41.279555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:41.279559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:41.279565Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 569 [2:7559341474927170288:5755]
2025-10-09T21:45:41.279578Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:41.279595Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:41.279597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:41.279600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:41.279602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:41.279617Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:41.279623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:41.279629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:41.279681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.279682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.279683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.279686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:41.279749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:41.279751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.279760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.279766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.279776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:41.279791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341474927170283:5755]
2025-10-09T21:45:41.279808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341474927170283:5755]
2025-10-09T21:45:41.279810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341474927170283:5755]
2025-10-09T21:45:41.279829Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:41.279845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:41.279857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:41.279861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:41.279903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341474927170283:5755]
2025-10-09T21:45:41.279906Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:41.280681Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:41.280690Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:41.280693Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:41.280699Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:41.281692Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 569
2025-10-09T21:45:41.281726Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 569
2025-10-09T21:45:41.294757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341475288983511:6306], now have 1 active actors on pipe
2025-10-09T21:45:41.294774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341475288983371:2432], now have 1 active actors on pipe
2025-10-09T21:45:41.294804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:41.294812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:41.294864Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e720745-8e6439f7-d4ed2730-18297240_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:41.294877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:41.294883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:41.294886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:41.294900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:41.294919Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:41.294947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:41.295444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:41.295453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:41.295469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:41.295625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 570 partition: 0 MaxSeqNo: 25 sessionId: src_id|e720745-8e6439f7-d4ed2730-18297240_0
2025-10-09T21:45:41.296098Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f291add1-fc76309e-615e698b-72d36bea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e720745-8e6439f7-d4ed2730-18297240_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:41.298856Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e720745-8e6439f7-d4ed2730-18297240_0] Write session: try to update token
2025-10-09T21:45:41.298877Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e720745-8e6439f7-d4ed2730-18297240_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:41.306990Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:41.307006Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:41.307008Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:41.307010Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:41.307025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341474927170283:5755]
2025-10-09T21:45:41.307030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341474927170283:5755]
2025-10-09T21:45:41.307033Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341474927170283:5755]
2025-10-09T21:45:41.307078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:41.307086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.307088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:41.307261Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:41.307910Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:41.307916Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:41.307920Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:41.334226Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341475288983520:5852]
2025-10-09T21:45:41.334656Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 569, pipe: [1:7559341475288983520:5852]
2025-10-09T21:45:41.336268Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 570 sessionId: src_id|e720745-8e6439f7-d4ed2730-18297240_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:41.338969Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:41.516222Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:41.516898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341475288983520:5852], now have 1 active actors on pipe
2025-10-09T21:45:41.516929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:41.516937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:41.517340Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:41.517371Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:41.517453Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 570 reason: error at write: Disconnected sessionId: src_id|e720745-8e6439f7-d4ed2730-18297240_0
2025-10-09T21:45:41.517418Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:41.517634Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 570 sessionId: src_id|e720745-8e6439f7-d4ed2730-18297240_0 is DEAD
2025-10-09T21:45:41.517680Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341475288983520:5852]
2025-10-09T21:45:41.517696Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:41.522196Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e720745-8e6439f7-d4ed2730-18297240_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:41.522211Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e720745-8e6439f7-d4ed2730-18297240_0] Write session will restart in 0.009053s
2025-10-09T21:45:41.522240Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e720745-8e6439f7-d4ed2730-18297240_0] Write session: Do CDS request
2025-10-09T21:45:41.522248Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e720745-8e6439f7-d4ed2730-18297240_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:41.533776Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e720745-8e6439f7-d4ed2730-18297240_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:41.534226Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:41.534246Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 571
2025-10-09T21:45:41.534393Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 571 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:41.534429Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 571 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:41.534435Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=571 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:41.534439Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:41.538535Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 571 sessionId:  describe result for acl check
2025-10-09T21:45:41.538595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:41.538597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:41.538598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:41.538605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341475288983533:6310] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:41.539221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341475288983533:6310] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:41.544773Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341475288983533:6310] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:41.551495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341475288983533:6310] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:41.551515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341475288983533:6310] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:41.551521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341475288983533:6310] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:41.551532Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 571 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:41.569123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:41.578528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:41.578822Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:41.578832Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:41.578845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:41.578915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:41.578960Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341474927170436:5759]
2025-10-09T21:45:41.579565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:41.580785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:41.580846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:41.581208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:41.581261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:41.581267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:41.581269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:41.581270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:41.581271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:41.581272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:41.581274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.581275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.581276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.581281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:41.581292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:41.581296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:41.581300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:41.581304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:41.581308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:41.581312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.581316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.581320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.581328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:41.581336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:41.581342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:41.581347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:41.581352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:41.581357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:41.581362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:41.581367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:41.581372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:41.581377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:41.581381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:41.581386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:41.581391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.581396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.581400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.581405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.581412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.581425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.581450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:41.590971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:41.592713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:41.592730Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:41.592732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:41.592736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:41.592745Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 570 [2:7559341474927170436:5759]
2025-10-09T21:45:41.592775Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:41.592799Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:41.592808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:41.592811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:41.592813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:41.592834Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:41.592847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:41.592855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:41.592930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.592937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.592938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.592941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:41.593015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:41.593022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.593034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.593045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.593062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:41.593083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341474927170430:5759]
2025-10-09T21:45:41.593085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341474927170430:5759]
2025-10-09T21:45:41.593087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341474927170430:5759]
2025-10-09T21:45:41.593095Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:41.593109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:41.593123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:41.593126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:41.593127Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:41.593130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:41.593132Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:41.593170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341474927170430:5759]
2025-10-09T21:45:41.593173Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:41.593175Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:41.593185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341475288983523:2432], now have 1 active actors on pipe
2025-10-09T21:45:41.593189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341475288983581:6310], now have 1 active actors on pipe
2025-10-09T21:45:41.594232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:41.594246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:41.594288Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|be3a0534-4b22a711-67f7316-1ca9e5c6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:41.594299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:41.594304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:41.594307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:41.594320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:41.594333Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:41.594354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:41.595132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:41.595142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:41.595160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:41.593527Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 570
2025-10-09T21:45:41.593569Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 570
2025-10-09T21:45:41.595339Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 571 partition: 0 MaxSeqNo: 25 sessionId: src_id|be3a0534-4b22a711-67f7316-1ca9e5c6_0
2025-10-09T21:45:41.598134Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e720745-8e6439f7-d4ed2730-18297240_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|be3a0534-4b22a711-67f7316-1ca9e5c6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:41.600813Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be3a0534-4b22a711-67f7316-1ca9e5c6_0] Write session: try to update token
2025-10-09T21:45:41.600837Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be3a0534-4b22a711-67f7316-1ca9e5c6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:41.618971Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:41.618991Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:41.618993Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:41.618995Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:41.619006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341474927170430:5759]
2025-10-09T21:45:41.619012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341474927170430:5759]
2025-10-09T21:45:41.619015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341474927170430:5759]
2025-10-09T21:45:41.620958Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341475288983674:5852]
2025-10-09T21:45:41.619062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:41.619070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.619072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:41.619216Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:41.622367Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:41.622381Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:41.622385Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:41.622592Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 570, pipe: [1:7559341475288983674:5852]
2025-10-09T21:45:41.657681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 571 sessionId: src_id|be3a0534-4b22a711-67f7316-1ca9e5c6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:41.660259Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:41.875847Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:41.876403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341475288983674:5852], now have 1 active actors on pipe
2025-10-09T21:45:41.876460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:41.876467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:41.878463Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:41.879628Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341475288983674:5852]
2025-10-09T21:45:41.879660Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:41.879729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 571 reason: status is not ok: tablet will be restarted right now sessionId: src_id|be3a0534-4b22a711-67f7316-1ca9e5c6_0
2025-10-09T21:45:41.879914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 571 sessionId: src_id|be3a0534-4b22a711-67f7316-1ca9e5c6_0 is DEAD
2025-10-09T21:45:41.880045Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:41.882297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be3a0534-4b22a711-67f7316-1ca9e5c6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:41.882310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be3a0534-4b22a711-67f7316-1ca9e5c6_0] Write session will restart in 0.008974s
2025-10-09T21:45:41.882340Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be3a0534-4b22a711-67f7316-1ca9e5c6_0] Write session: Do CDS request
2025-10-09T21:45:41.882347Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be3a0534-4b22a711-67f7316-1ca9e5c6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:41.894578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be3a0534-4b22a711-67f7316-1ca9e5c6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:41.894907Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:41.894927Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 572
2025-10-09T21:45:41.895133Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 572 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:41.895190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 572 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:41.895196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=572 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:41.895200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:41.895920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 572 sessionId:  describe result for acl check
2025-10-09T21:45:41.895958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:41.895962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:41.895965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:41.895970Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341475288983696:6317] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:41.896522Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341475288983696:6317] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:41.917056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341475288983696:6317] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:41.926923Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341475288983696:6317] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:41.926943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341475288983696:6317] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:41.926949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341475288983696:6317] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:41.926961Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 572 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:41.937142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:41.937350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:41.937562Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:41.937566Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:41.937580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:41.937631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:41.937670Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341474927170593:5767]
2025-10-09T21:45:41.938336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:41.939185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:41.939259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:41.939750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:41.939867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:41.939869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:41.939871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:41.939872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:41.939874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:41.939875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:41.939876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.939878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.939879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.939886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:41.939897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:41.939902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:41.939906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:41.939911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:41.939915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:41.939919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.939923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.939927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.939935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:41.939943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:41.939949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:41.939954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:41.939958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:41.939963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:41.939967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:41.939972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:41.939976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:41.939981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:41.939988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:41.939992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:41.939997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.940001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.940005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.940010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.940015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.940020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.940047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:41.950557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:41.953699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:41.953723Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:41.953725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:41.953729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:41.953736Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 571 [2:7559341474927170593:5767]
2025-10-09T21:45:41.953751Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:41.953772Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:41.953776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:41.953779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:41.953781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:41.953801Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:41.953810Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:41.953824Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:41.953891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.953894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.953895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.953899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:41.953970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:41.953974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.953989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:41.954002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:41.954019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:41.954060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341474927170588:5767]
2025-10-09T21:45:41.954062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341474927170588:5767]
2025-10-09T21:45:41.954065Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341474927170588:5767]
2025-10-09T21:45:41.954078Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:41.954095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:41.954111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:41.954114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:41.954176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341474927170588:5767]
2025-10-09T21:45:41.954186Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:41.954534Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:41.954544Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:41.954546Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:41.954552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:41.956467Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 571
2025-10-09T21:45:41.956503Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 571
2025-10-09T21:45:41.956891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341475288983823:6317], now have 1 active actors on pipe
2025-10-09T21:45:41.956901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341475288983684:2432], now have 1 active actors on pipe
2025-10-09T21:45:41.956918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:41.956925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:41.956961Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4091cc99-199b05dd-b453ae1-ad29e00_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:41.956974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:41.956980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:41.956983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:41.957001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:41.957019Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:41.957040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:41.957748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:41.957758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:41.957777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:41.957979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 572 partition: 0 MaxSeqNo: 25 sessionId: src_id|4091cc99-199b05dd-b453ae1-ad29e00_0
2025-10-09T21:45:41.958455Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be3a0534-4b22a711-67f7316-1ca9e5c6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4091cc99-199b05dd-b453ae1-ad29e00_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:41.961001Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4091cc99-199b05dd-b453ae1-ad29e00_0] Write session: try to update token
2025-10-09T21:45:41.961026Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4091cc99-199b05dd-b453ae1-ad29e00_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:41.979164Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341475288983838:5852]
2025-10-09T21:45:41.980933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341475288983838:5852], now have 1 active actors on pipe
2025-10-09T21:45:41.985078Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 571, pipe: [1:7559341475288983838:5852]
2025-10-09T21:45:41.991408Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:41.991426Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:41.991428Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:41.991430Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:41.991440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341474927170588:5767]
2025-10-09T21:45:41.991448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341474927170588:5767]
2025-10-09T21:45:41.991451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341474927170588:5767]
2025-10-09T21:45:41.991493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:41.991500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:41.991503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:41.991641Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:41.997936Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:41.997949Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:41.997952Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:41.997951Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 572 sessionId: src_id|4091cc99-199b05dd-b453ae1-ad29e00_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:42.000608Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:42.222818Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:42.223434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:42.223444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:42.226088Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:42.226562Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341475288983838:5852]
2025-10-09T21:45:42.226601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:42.226677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 572 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4091cc99-199b05dd-b453ae1-ad29e00_0
2025-10-09T21:45:42.226879Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 572 sessionId: src_id|4091cc99-199b05dd-b453ae1-ad29e00_0 is DEAD
2025-10-09T21:45:42.227018Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:42.230437Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4091cc99-199b05dd-b453ae1-ad29e00_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:42.230450Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4091cc99-199b05dd-b453ae1-ad29e00_0] Write session will restart in 0.007525s
2025-10-09T21:45:42.230478Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4091cc99-199b05dd-b453ae1-ad29e00_0] Write session: Do CDS request
2025-10-09T21:45:42.230486Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4091cc99-199b05dd-b453ae1-ad29e00_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:42.239728Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4091cc99-199b05dd-b453ae1-ad29e00_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:42.240129Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:42.240148Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 573
2025-10-09T21:45:42.240244Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 573 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:42.240272Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 573 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:42.240278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=573 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:42.240282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:42.240752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 573 sessionId:  describe result for acl check
2025-10-09T21:45:42.240800Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:42.240803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:42.240804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:42.240811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341479583951210:6329] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:42.241393Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341479583951210:6329] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:42.244981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341479583951210:6329] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:42.250005Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341479583951210:6329] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:42.250023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341479583951210:6329] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:42.250027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341479583951210:6329] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:42.250052Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 573 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:42.251066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:42.259207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:42.259472Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:42.259484Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:42.259498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:42.259562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:42.259617Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341479222138059:5775]
2025-10-09T21:45:42.260276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:42.263180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:42.263255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:42.263654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:42.263711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:42.263713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:42.263714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:42.263715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:42.263716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:42.263718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:42.263719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.263720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.263723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.263729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:42.263742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:42.263746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:42.263751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:42.263756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:42.263761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:42.263765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.263769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.263774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.263784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:42.263793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:42.263799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:42.263805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:42.263811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:42.263817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:42.263823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:42.263828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:42.263833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:42.263839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:42.263843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:42.263849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:42.263855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.263872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.263877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.263884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.263891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.263897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.263932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:42.276943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:42.278629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:42.278637Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:42.278640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:42.278644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:42.278652Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 572 [2:7559341479222138059:5775]
2025-10-09T21:45:42.278670Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:42.278692Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:42.278696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:42.278700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:42.278702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:42.278722Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:42.278729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:42.278738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:42.278805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.278807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.278809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.278812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:42.278897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:42.278900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.278916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.278928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.278944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:42.278965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341479222138052:5775]
2025-10-09T21:45:42.278968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341479222138052:5775]
2025-10-09T21:45:42.278971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341479222138052:5775]
2025-10-09T21:45:42.278986Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:42.279000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:42.279015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:42.279018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:42.279059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341479222138052:5775]
2025-10-09T21:45:42.279062Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:42.279380Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:42.279388Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:42.279392Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:42.279399Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:42.279770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341479583951170:2432], now have 1 active actors on pipe
2025-10-09T21:45:42.279775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341479583951306:6329], now have 1 active actors on pipe
2025-10-09T21:45:42.279853Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 572
2025-10-09T21:45:42.279919Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 572
2025-10-09T21:45:42.280473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:42.280485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:42.280520Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6e0206ae-73054915-714bd45c-b167c1b9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:42.280530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:42.280534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:42.280537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:42.280548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:42.280562Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:42.280591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:42.280995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:42.281008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:42.281030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:42.281170Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 573 partition: 0 MaxSeqNo: 25 sessionId: src_id|6e0206ae-73054915-714bd45c-b167c1b9_0
2025-10-09T21:45:42.281590Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4091cc99-199b05dd-b453ae1-ad29e00_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6e0206ae-73054915-714bd45c-b167c1b9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:42.284229Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e0206ae-73054915-714bd45c-b167c1b9_0] Write session: try to update token
2025-10-09T21:45:42.284249Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e0206ae-73054915-714bd45c-b167c1b9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:42.300372Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:42.300392Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:42.300394Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:42.300396Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:42.300407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341479222138052:5775]
2025-10-09T21:45:42.300412Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341479222138052:5775]
2025-10-09T21:45:42.300415Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341479222138052:5775]
2025-10-09T21:45:42.300465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:42.300469Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:42.300472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.300476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:42.300476Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:42.300480Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:42.300623Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:42.312575Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 573 sessionId: src_id|6e0206ae-73054915-714bd45c-b167c1b9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:42.315533Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:42.326901Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341479583951319:5852]
2025-10-09T21:45:42.327891Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 572, pipe: [1:7559341479583951319:5852]
2025-10-09T21:45:42.499534Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:42.500452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:42.500466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:42.500514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341479583951319:5852], now have 1 active actors on pipe
2025-10-09T21:45:42.501254Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:42.501347Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341479583951319:5852]
2025-10-09T21:45:42.501379Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:42.501392Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:42.501475Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 573 reason: error at write: Disconnected sessionId: src_id|6e0206ae-73054915-714bd45c-b167c1b9_0
2025-10-09T21:45:42.502076Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:42.501711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 573 sessionId: src_id|6e0206ae-73054915-714bd45c-b167c1b9_0 is DEAD
2025-10-09T21:45:42.502462Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e0206ae-73054915-714bd45c-b167c1b9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:42.502475Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e0206ae-73054915-714bd45c-b167c1b9_0] Write session will restart in 0.008058s
2025-10-09T21:45:42.502512Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e0206ae-73054915-714bd45c-b167c1b9_0] Write session: Do CDS request
2025-10-09T21:45:42.502519Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e0206ae-73054915-714bd45c-b167c1b9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:42.515443Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e0206ae-73054915-714bd45c-b167c1b9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:42.522234Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:42.522254Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 574
2025-10-09T21:45:42.522446Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 574 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:42.522487Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 574 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:42.522498Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=574 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:42.522502Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:42.523595Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 574 sessionId:  describe result for acl check
2025-10-09T21:45:42.523630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:42.523631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:42.523632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:42.523637Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341479583951346:6333] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:42.524100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341479583951346:6333] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:42.534186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341479583951346:6333] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:42.540325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341479583951346:6333] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:42.540350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341479583951346:6333] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:42.540355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341479583951346:6333] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:42.540366Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 574 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:42.553774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:42.555877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:42.556369Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:42.556374Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:42.556390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:42.556463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:42.556503Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341479222138209:5779]
2025-10-09T21:45:42.557137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:42.557978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:42.558055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:42.558428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:42.558493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:42.558494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:42.558495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:42.558497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:42.558499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:42.558500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:42.558502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.558503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.558504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.558511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:42.558523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:42.558527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:42.558531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:42.558534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:42.558537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:42.558540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.558543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.558546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.558554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:42.558563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:42.558570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:42.558574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:42.558578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:42.558584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:42.558588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:42.558593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:42.558597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:42.558601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:42.558605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:42.558629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:42.558634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.558638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.558642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.558646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.558650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.558656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.558682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:42.569020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:42.571296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:42.571303Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:42.571306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:42.571315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:42.571324Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 573 [2:7559341479222138209:5779]
2025-10-09T21:45:42.571340Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:42.571361Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:42.571365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:42.571368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:42.571371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:42.571389Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:42.571396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:42.571404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:42.571480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.571483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.571485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.571488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:42.571582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:42.571586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.571601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.571614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.571630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:42.571651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341479222138204:5779]
2025-10-09T21:45:42.571653Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341479222138204:5779]
2025-10-09T21:45:42.571655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341479222138204:5779]
2025-10-09T21:45:42.571674Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:42.571690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:42.571707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:42.571710Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:42.571754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341479222138204:5779]
2025-10-09T21:45:42.571756Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:42.572905Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 573
2025-10-09T21:45:42.572948Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 573
2025-10-09T21:45:42.572175Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:42.572181Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:42.572183Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:42.572189Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:42.573270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341479583951322:2432], now have 1 active actors on pipe
2025-10-09T21:45:42.573276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341479583951442:6333], now have 1 active actors on pipe
2025-10-09T21:45:42.573593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:42.573603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:42.573646Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1f21e30c-65e8b089-4efda33b-a69fc23b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:42.573656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:42.573661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:42.573663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:42.573676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:42.573690Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:42.573708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:42.574196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 574 partition: 0 MaxSeqNo: 25 sessionId: src_id|1f21e30c-65e8b089-4efda33b-a69fc23b_0
2025-10-09T21:45:42.574012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:42.574019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:42.574065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:42.578406Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e0206ae-73054915-714bd45c-b167c1b9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1f21e30c-65e8b089-4efda33b-a69fc23b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:42.590458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1f21e30c-65e8b089-4efda33b-a69fc23b_0] Write session: try to update token
2025-10-09T21:45:42.590494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1f21e30c-65e8b089-4efda33b-a69fc23b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:42.595521Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:42.595540Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:42.595542Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:42.595544Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:42.595555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341479222138204:5779]
2025-10-09T21:45:42.595559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341479222138204:5779]
2025-10-09T21:45:42.595563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341479222138204:5779]
2025-10-09T21:45:42.595609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:42.595617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.595619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:42.595768Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:42.598524Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:42.598535Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:42.598539Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:42.601493Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341479583951476:5852]
2025-10-09T21:45:42.602736Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 573, pipe: [1:7559341479583951476:5852]
2025-10-09T21:45:42.655514Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 574 sessionId: src_id|1f21e30c-65e8b089-4efda33b-a69fc23b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:42.657979Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:42.799651Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:42.800074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341479583951476:5852], now have 1 active actors on pipe
2025-10-09T21:45:42.800130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:42.800138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:42.802324Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:42.803357Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341479583951476:5852]
2025-10-09T21:45:42.803393Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:42.803428Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:42.803505Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 574 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1f21e30c-65e8b089-4efda33b-a69fc23b_0
2025-10-09T21:45:42.803693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 574 sessionId: src_id|1f21e30c-65e8b089-4efda33b-a69fc23b_0 is DEAD
2025-10-09T21:45:42.804089Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1f21e30c-65e8b089-4efda33b-a69fc23b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:42.804101Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1f21e30c-65e8b089-4efda33b-a69fc23b_0] Write session will restart in 0.009034s
2025-10-09T21:45:42.804128Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1f21e30c-65e8b089-4efda33b-a69fc23b_0] Write session: Do CDS request
2025-10-09T21:45:42.804135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1f21e30c-65e8b089-4efda33b-a69fc23b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:42.817102Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1f21e30c-65e8b089-4efda33b-a69fc23b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:42.817478Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:42.817497Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 575
2025-10-09T21:45:42.817766Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 575 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:42.817800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 575 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:42.817806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=575 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:42.817810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:42.818430Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 575 sessionId:  describe result for acl check
2025-10-09T21:45:42.818470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:42.818476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:42.818478Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:42.818487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341479583951495:6338] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:42.819093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341479583951495:6338] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:42.829151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341479583951495:6338] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:42.839534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341479583951495:6338] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:42.839554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341479583951495:6338] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:42.839558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341479583951495:6338] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:42.839567Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 575 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:42.847001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:42.852272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:42.852515Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:42.852526Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:42.852539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:42.852607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:42.852648Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341479222138356:5781]
2025-10-09T21:45:42.853185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:42.853529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:42.853581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:42.853840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:42.854279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:42.854286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:42.854287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:42.854288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:42.854290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:42.854291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:42.854292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.854294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.854295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.854302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:42.854313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:42.854318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:42.854323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:42.854327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:42.854332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:42.854336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.854340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.854344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.854356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:42.854367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:42.854374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:42.854380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:42.854384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:42.854390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:42.854394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:42.854400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:42.854404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:42.854410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:42.854414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:42.854425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:42.854430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.854436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.854440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.854446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.854456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.854463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.854497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:42.861533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:42.862787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:42.862802Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:42.862805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:42.862809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:42.862818Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 574 [2:7559341479222138356:5781]
2025-10-09T21:45:42.862838Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:42.862861Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:42.862864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:42.862867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:42.862871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:42.862891Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:42.862898Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:42.862906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:42.862969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.862979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.862981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.862984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:42.863367Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 574
2025-10-09T21:45:42.863413Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 574
2025-10-09T21:45:42.863064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:42.863073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.863087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:42.863096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:42.863112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:42.863133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341479222138351:5781]
2025-10-09T21:45:42.863135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341479222138351:5781]
2025-10-09T21:45:42.863138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341479222138351:5781]
2025-10-09T21:45:42.863149Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:42.863154Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:42.863156Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:42.863160Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:42.863164Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:42.863178Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:42.863181Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:42.863193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341479583951591:6338], now have 1 active actors on pipe
2025-10-09T21:45:42.863196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341479583951481:2432], now have 1 active actors on pipe
2025-10-09T21:45:42.863235Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341479222138351:5781]
2025-10-09T21:45:42.863243Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:42.863627Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:42.867014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:42.867030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:42.867069Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9f2909ca-9ea769f2-1a6059fc-cb1eb204_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:42.867082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:42.867088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:42.867091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:42.867106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:42.867123Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:42.867153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:42.868914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:42.868927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:42.868965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:42.869194Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 575 partition: 0 MaxSeqNo: 25 sessionId: src_id|9f2909ca-9ea769f2-1a6059fc-cb1eb204_0
2025-10-09T21:45:42.869733Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1f21e30c-65e8b089-4efda33b-a69fc23b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9f2909ca-9ea769f2-1a6059fc-cb1eb204_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:42.872316Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f2909ca-9ea769f2-1a6059fc-cb1eb204_0] Write session: try to update token
2025-10-09T21:45:42.872340Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f2909ca-9ea769f2-1a6059fc-cb1eb204_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:42.887997Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:42.888017Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:42.888020Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:42.888022Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:42.888032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341479222138351:5781]
2025-10-09T21:45:42.888037Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341479222138351:5781]
2025-10-09T21:45:42.888041Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341479222138351:5781]
2025-10-09T21:45:42.888086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:42.888095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:42.888097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:42.888276Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:42.891255Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:42.891273Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:42.891277Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:42.905744Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341479583951635:5852]
2025-10-09T21:45:42.907518Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 574, pipe: [1:7559341479583951635:5852]
2025-10-09T21:45:42.922156Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 575 sessionId: src_id|9f2909ca-9ea769f2-1a6059fc-cb1eb204_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:42.924879Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:43.133997Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:43.139587Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:43.139600Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:43.139670Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 575 reason: error at write: Disconnected sessionId: src_id|9f2909ca-9ea769f2-1a6059fc-cb1eb204_0
2025-10-09T21:45:43.139853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 575 sessionId: src_id|9f2909ca-9ea769f2-1a6059fc-cb1eb204_0 is DEAD
2025-10-09T21:45:43.140082Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:43.140116Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341479583951635:5852]
2025-10-09T21:45:43.142431Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f2909ca-9ea769f2-1a6059fc-cb1eb204_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:43.142445Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f2909ca-9ea769f2-1a6059fc-cb1eb204_0] Write session will restart in 0.006081s
2025-10-09T21:45:43.142469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f2909ca-9ea769f2-1a6059fc-cb1eb204_0] Write session: Do CDS request
2025-10-09T21:45:43.142476Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f2909ca-9ea769f2-1a6059fc-cb1eb204_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:43.138728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341479583951635:5852], now have 1 active actors on pipe
2025-10-09T21:45:43.138793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:43.138799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:43.140999Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:43.150611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f2909ca-9ea769f2-1a6059fc-cb1eb204_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:43.150942Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:43.150958Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 576
2025-10-09T21:45:43.151166Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 576 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:43.151196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 576 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:43.151204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=576 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:43.151208Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:43.151828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 576 sessionId:  describe result for acl check
2025-10-09T21:45:43.151863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:43.151865Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:43.151866Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:43.151872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341483878918959:6343] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:43.152414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341483878918959:6343] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:43.159577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341483878918959:6343] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:43.170711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341483878918959:6343] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:43.170734Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341483878918959:6343] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:43.170738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341483878918959:6343] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:43.170748Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 576 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:43.180717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:43.187808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:43.188101Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:43.188105Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:43.188122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:43.188194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:43.188238Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341483517105830:5793]
2025-10-09T21:45:43.188745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:43.193583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:43.193663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:43.194084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:43.194185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:43.194192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:43.194194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:43.194195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:43.194197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:43.194198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:43.194199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.194201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.194202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.194210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:43.194221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:43.194225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:43.194230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:43.194234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:43.194238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:43.194242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.194245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.194249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.194258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:43.194268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:43.194275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:43.194280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:43.194286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:43.194292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:43.194296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:43.194302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:43.194307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:43.194312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:43.194316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:43.194321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:43.194326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.194331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.194336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.194341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.194348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.194360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.194390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:43.203988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:43.205074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:43.205092Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:43.205096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:43.205101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:43.205110Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 575 [2:7559341483517105830:5793]
2025-10-09T21:45:43.205130Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:43.205154Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:43.205158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:43.205161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:43.205165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:43.205183Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:43.205195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:43.205204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:43.205277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.205285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.205287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.205290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:43.205370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:43.205372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.205385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.205398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.205416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:43.205437Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341483517105824:5793]
2025-10-09T21:45:43.205439Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341483517105824:5793]
2025-10-09T21:45:43.205441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341483517105824:5793]
2025-10-09T21:45:43.205451Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:43.205463Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:43.205467Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:43.205467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:43.205468Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:43.205481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:43.205484Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:43.205497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341483878918947:2432], now have 1 active actors on pipe
2025-10-09T21:45:43.205501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341483878919081:6343], now have 1 active actors on pipe
2025-10-09T21:45:43.205547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341483517105824:5793]
2025-10-09T21:45:43.205553Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:43.205629Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 575
2025-10-09T21:45:43.205680Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 575
2025-10-09T21:45:43.205846Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:43.207234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:43.207248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:43.207282Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|db59552c-7bf007a0-834af885-549bf4a9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:43.207294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:43.207298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:43.207302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:43.207314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:43.207330Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:43.207348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:43.207848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:43.207871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:43.207891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:43.208190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 576 partition: 0 MaxSeqNo: 25 sessionId: src_id|db59552c-7bf007a0-834af885-549bf4a9_0
2025-10-09T21:45:43.209807Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f2909ca-9ea769f2-1a6059fc-cb1eb204_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|db59552c-7bf007a0-834af885-549bf4a9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:43.212280Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db59552c-7bf007a0-834af885-549bf4a9_0] Write session: try to update token
2025-10-09T21:45:43.212300Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db59552c-7bf007a0-834af885-549bf4a9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:43.244507Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341483878919111:5852]
2025-10-09T21:45:43.244555Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 576 sessionId: src_id|db59552c-7bf007a0-834af885-549bf4a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:43.247118Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:43.251244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:43.251268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:43.251295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341483878919111:5852], now have 1 active actors on pipe
2025-10-09T21:45:43.251325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:43.255579Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 575, pipe: [1:7559341483878919111:5852]
2025-10-09T21:45:43.255625Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:43.279281Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:43.279301Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:43.279304Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:43.279306Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:43.279317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341483517105824:5793]
2025-10-09T21:45:43.279321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341483517105824:5793]
2025-10-09T21:45:43.279324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341483517105824:5793]
2025-10-09T21:45:43.279370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:43.279377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.279380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:43.279594Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:43.282331Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:43.282342Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:43.282346Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:43.547872Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:43.564969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:43.565005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:43.565199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.565264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.565346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.565421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.565791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.565806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.565878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.565946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.566006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.570187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.570243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.570249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.570326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.570382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.570440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.570504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.570642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.570645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.570754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.570850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.570913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.571009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.571064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.571068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.571173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.571267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.571363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.571425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.571481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.571484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.571566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.571621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.571704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.571763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.571822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.571828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.571890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.571946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.572006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.572061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.572105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.572108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.572173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.572229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.572286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.572343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.572405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.572408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.572470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.572526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.572589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.572650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.572714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.572717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.572786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.572846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.572907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.572968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.573030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.573032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.573951Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:43.573970Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:43.573974Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:43.574020Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 576 reason: error at write: Disconnected sessionId: src_id|db59552c-7bf007a0-834af885-549bf4a9_0
2025-10-09T21:45:43.574021Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341483878919111:5852]
2025-10-09T21:45:43.574273Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 576 sessionId: src_id|db59552c-7bf007a0-834af885-549bf4a9_0 is DEAD
2025-10-09T21:45:43.574022Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:43.574675Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db59552c-7bf007a0-834af885-549bf4a9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:43.574685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db59552c-7bf007a0-834af885-549bf4a9_0] Write session will restart in 0.009903s
2025-10-09T21:45:43.574726Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db59552c-7bf007a0-834af885-549bf4a9_0] Write session: Do CDS request
2025-10-09T21:45:43.574733Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db59552c-7bf007a0-834af885-549bf4a9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:43.585607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db59552c-7bf007a0-834af885-549bf4a9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:43.586226Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:43.586246Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 577
2025-10-09T21:45:43.586390Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 577 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:43.586419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 577 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:43.586425Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=577 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:43.586432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:43.586880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 577 sessionId:  describe result for acl check
2025-10-09T21:45:43.586925Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:43.586927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:43.586929Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:43.586935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341483878919140:6354] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:43.587561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341483878919140:6354] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:43.590851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341483878919140:6354] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:43.606396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341483878919140:6354] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:43.606417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341483878919140:6354] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:43.606422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341483878919140:6354] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:43.606433Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 577 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:43.644436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:43.648945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:43.649218Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:43.649229Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:43.649243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:43.649308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:43.649356Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341483517105991:5799]
2025-10-09T21:45:43.649988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:43.650453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:43.650528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:43.650883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:43.650944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:43.650946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:43.650947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:43.650949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:43.650950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:43.650952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:43.650953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.650954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.650956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.650962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:43.650974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:43.650978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:43.650983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:43.650987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:43.650992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:43.650996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.651000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.651005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.651015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:43.651024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:43.651030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:43.651036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:43.651043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:43.651049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:43.651055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:43.651061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:43.651066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:43.651072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:43.651077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:43.651083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:43.651088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.651095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.651100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.651106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.651114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.651119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.651153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:43.665297Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 576
2025-10-09T21:45:43.663230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:43.664781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:43.664787Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:43.664790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:43.664794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:43.664802Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 576 [2:7559341483517105991:5799]
2025-10-09T21:45:43.664818Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:43.664840Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:43.664844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:43.664848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:43.664851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:43.664870Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:43.664876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:43.664884Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:43.664947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.664950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.664952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.664955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:43.665036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:43.665039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.665053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.665066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.665085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:43.665107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341483517105986:5799]
2025-10-09T21:45:43.665109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341483517105986:5799]
2025-10-09T21:45:43.665430Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 576
2025-10-09T21:45:43.665114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341483517105986:5799]
2025-10-09T21:45:43.665122Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:43.665136Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:43.665146Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:43.665150Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:43.665152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:43.665152Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:43.665155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:43.665195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341483517105986:5799]
2025-10-09T21:45:43.665205Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:43.665216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341483878919130:2432], now have 1 active actors on pipe
2025-10-09T21:45:43.665220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341483878919203:6354], now have 1 active actors on pipe
2025-10-09T21:45:43.665493Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:43.665953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:43.665967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:43.666001Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b5701112-dc2659d6-6403e5af-577b6dcf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:43.666010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:43.666015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:43.666017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:43.666043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:43.666056Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:43.666074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:43.666644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:43.666651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:43.666667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:43.666806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 577 partition: 0 MaxSeqNo: 25 sessionId: src_id|b5701112-dc2659d6-6403e5af-577b6dcf_0
2025-10-09T21:45:43.667204Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db59552c-7bf007a0-834af885-549bf4a9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b5701112-dc2659d6-6403e5af-577b6dcf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:43.669715Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5701112-dc2659d6-6403e5af-577b6dcf_0] Write session: try to update token
2025-10-09T21:45:43.669734Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5701112-dc2659d6-6403e5af-577b6dcf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:43.674392Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341483878919286:5852]
2025-10-09T21:45:43.674800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341483878919286:5852], now have 1 active actors on pipe
2025-10-09T21:45:43.674937Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 576, pipe: [1:7559341483878919286:5852]
2025-10-09T21:45:43.688737Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:43.688755Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:43.688766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341483517105986:5799]
2025-10-09T21:45:43.688823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:43.688823Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:43.689030Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:43.689164Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:43.690231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:43.690365Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:43.690541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:43.690583Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:43.690686Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 577 sessionId: src_id|b5701112-dc2659d6-6403e5af-577b6dcf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:43.693043Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:43.693676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:43.693693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:43.693732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:43.693836Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:43.700799Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:43.700821Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:43.700824Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:43.700826Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:43.700836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341483517105986:5799]
2025-10-09T21:45:43.700841Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341483517105986:5799]
2025-10-09T21:45:43.700845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341483517105986:5799]
2025-10-09T21:45:43.700889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:43.700902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.700905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:43.700905Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:43.700912Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:43.700916Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:43.701080Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:43.870789Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:43.878954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:43.878982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:43.879219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.879285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.879355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.879416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.879471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.879478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.879542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.879599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.879652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.879742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.879784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.879789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.879854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.879904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.879958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.880015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.880060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.880064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.880125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.880181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.880237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.880323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.880365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.880368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.880433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.880488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.880544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.880601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.880641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.880644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.880712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.880771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.880828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.880882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.880920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.880923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.880990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.881043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.881100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.881155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.881193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.881196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.881256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.881305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.881356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.881420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.881460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.881463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.881542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.881598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.881656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.881713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.881752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.881777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.881837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.881897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.881956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.882017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.882305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.882320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.882777Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341483878919286:5852]
2025-10-09T21:45:43.882781Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:43.882823Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:43.882826Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:43.882873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 577 reason: error at write: Disconnected sessionId: src_id|b5701112-dc2659d6-6403e5af-577b6dcf_0
2025-10-09T21:45:43.883092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 577 sessionId: src_id|b5701112-dc2659d6-6403e5af-577b6dcf_0 is DEAD
2025-10-09T21:45:43.883379Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5701112-dc2659d6-6403e5af-577b6dcf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:43.883407Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5701112-dc2659d6-6403e5af-577b6dcf_0] Write session will restart in 0.007470s
2025-10-09T21:45:43.883433Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5701112-dc2659d6-6403e5af-577b6dcf_0] Write session: Do CDS request
2025-10-09T21:45:43.883439Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5701112-dc2659d6-6403e5af-577b6dcf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:43.883511Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:43.892671Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5701112-dc2659d6-6403e5af-577b6dcf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:43.892929Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:43.892962Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 578
2025-10-09T21:45:43.893128Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 578 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:43.893166Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 578 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:43.893179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=578 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:43.893183Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:43.893712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 578 sessionId:  describe result for acl check
2025-10-09T21:45:43.893752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:43.893760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:43.893763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:43.893770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341483878919306:6359] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:43.894580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341483878919306:6359] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:43.899761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341483878919306:6359] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:43.903559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:43.911271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:43.911599Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:43.911609Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:43.911622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:43.911415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341483878919306:6359] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:43.911435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341483878919306:6359] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:43.911439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341483878919306:6359] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:43.911450Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 578 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:43.911688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:43.911736Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341483517106140:5801]
2025-10-09T21:45:43.912346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:43.912966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:43.913047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:43.913415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:43.913479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:43.913486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:43.913487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:43.913488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:43.913489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:43.913490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:43.913491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.913492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.913494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.913499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:43.913509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:43.913512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:43.913515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:43.913519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:43.913523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:43.913526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.913530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.913533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.913541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:43.913548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:43.913554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:43.913558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:43.913565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:43.913570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:43.913574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:43.913578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:43.913582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:43.913586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:43.913598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:43.913603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:43.913607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.913612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.913615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.913619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.913624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.913629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.913655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:43.924061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:43.925212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:43.925231Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:43.925235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:43.925240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:43.925249Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 577 [2:7559341483517106140:5801]
2025-10-09T21:45:43.925265Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:43.925283Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:43.925287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:43.925290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:43.925294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:43.925315Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:43.925328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:43.925341Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:43.925408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.925416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.925418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.925421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:43.925495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:43.925506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:43.925519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:43.925533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:43.925552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:43.925574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341483878919292:2432], now have 1 active actors on pipe
2025-10-09T21:45:43.925583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341483517106135:5801]
2025-10-09T21:45:43.925587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341483517106135:5801]
2025-10-09T21:45:43.925589Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341483517106135:5801]
2025-10-09T21:45:43.925675Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 577
2025-10-09T21:45:43.925734Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 577
2025-10-09T21:45:43.925603Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:43.925611Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:43.925622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:43.925632Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:43.925636Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:43.925637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:43.925641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:43.925648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341483878919437:6359], now have 1 active actors on pipe
2025-10-09T21:45:43.925686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341483517106135:5801]
2025-10-09T21:45:43.925694Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:43.925733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:43.926077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:43.926096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:43.926129Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|50451f2-fcaadf9-ee46b1c4-8e50deca_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:43.926169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:43.926173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:43.926176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:43.926189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:43.926218Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:43.926237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:43.927083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:43.927096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:43.927119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:43.927509Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 578 partition: 0 MaxSeqNo: 25 sessionId: src_id|50451f2-fcaadf9-ee46b1c4-8e50deca_0
2025-10-09T21:45:43.927967Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5701112-dc2659d6-6403e5af-577b6dcf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|50451f2-fcaadf9-ee46b1c4-8e50deca_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:43.930486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|50451f2-fcaadf9-ee46b1c4-8e50deca_0] Write session: try to update token
2025-10-09T21:45:43.930507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|50451f2-fcaadf9-ee46b1c4-8e50deca_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:43.951998Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:43.952017Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:43.952027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341483517106135:5801]
2025-10-09T21:45:43.952075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:43.952248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:43.952382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:43.952507Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:43.952637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:43.952820Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:43.952857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:43.953532Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:43.956778Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 578 sessionId: src_id|50451f2-fcaadf9-ee46b1c4-8e50deca_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:43.959489Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:43.960577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:43.960603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:43.960656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:43.960760Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:43.986596Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341483878919452:5852]
2025-10-09T21:45:43.999092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:43.999119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:43.999306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.999373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.999441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.999506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.999558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.999564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.999636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.999694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:43.999752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:43.999816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:43.999862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:43.999865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:43.999933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:43.999992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:44.000051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:44.000111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:44.000152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:44.000155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:44.000226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:44.000286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:44.000344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:44.000410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:44.000449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:44.000452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:44.000536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:44.000593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:44.000654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:44.000726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:44.000772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:44.000775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:44.000839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:44.000893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:44.000953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:44.001010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:44.001133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:44.001136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:44.001210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:44.001271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:44.001343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:44.001407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:44.001447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:44.001450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:44.001515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:44.001574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:44.001636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:44.002112Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 577, pipe: [1:7559341483878919452:5852]
2025-10-09T21:45:44.002236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:44.002376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:44.002385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:44.002453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:44.002517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:44.002582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:44.002655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:44.002782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:44.002786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:44.002851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:44.002910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:44.002977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:44.003042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:44.003100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:44.003103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:44.003204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341483878919452:5852], now have 1 active actors on pipe
2025-10-09T21:45:44.003214Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:45:44.003278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:44.003298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:44.003301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (50)
2025-10-09T21:45:44.003338Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 0
2025-10-09T21:45:44.003353Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 1
2025-10-09T21:45:44.003356Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 2
2025-10-09T21:45:44.003361Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 3
2025-10-09T21:45:44.003363Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 4
2025-10-09T21:45:44.003789Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 26 partNo 4 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 2253230 count 1 nextOffset 26 batches 5
2025-10-09T21:45:44.003809Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 0
2025-10-09T21:45:44.003817Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 1
2025-10-09T21:45:44.003820Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 2
2025-10-09T21:45:44.003823Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 3
2025-10-09T21:45:44.003826Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 4
2025-10-09T21:45:44.004627Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 27 partNo 4 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 4506428 count 2 nextOffset 27 batches 9
2025-10-09T21:45:44.004643Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 0
2025-10-09T21:45:44.004649Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 1
2025-10-09T21:45:44.004652Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 2
2025-10-09T21:45:44.004655Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 3
2025-10-09T21:45:44.004659Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 4
2025-10-09T21:45:44.005169Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 28 partNo 4 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 6759626 count 3 nextOffset 28 batches 13
2025-10-09T21:45:44.005186Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 0
2025-10-09T21:45:44.005192Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 1
2025-10-09T21:45:44.005195Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 2
2025-10-09T21:45:44.005199Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 3
2025-10-09T21:45:44.008131Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1380: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 29 partNo 3 result is x0000000000_00000000000000000025_00000_0000000003_00015? size 8295638
2025-10-09T21:45:44.008158Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 4
2025-10-09T21:45:44.008175Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1098: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000025_00000_0000000003_00015? new key d0000000000_00000000000000000025_00000_0000000003_00015? size 8295638 WTime 1760046344008
2025-10-09T21:45:44.008848Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 29 partNo 4 FormedBlobsCount 1 NewHead: Offset 28 PartNo 3 PackedSize 717145 count 1 nextOffset 29 batches 2
2025-10-09T21:45:44.008872Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 0
2025-10-09T21:45:44.008879Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 1
2025-10-09T21:45:44.008883Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 2
2025-10-09T21:45:44.008888Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 3
2025-10-09T21:45:44.008893Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 4
2025-10-09T21:45:44.009454Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 30 partNo 4 FormedBlobsCount 0 NewHead: Offset 28 PartNo 3 PackedSize 2970343 count 2 nextOffset 30 batches 6
2025-10-09T21:45:44.009480Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 0
2025-10-09T21:45:44.009486Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 1
2025-10-09T21:45:44.009488Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 2
2025-10-09T21:45:44.009491Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 3
2025-10-09T21:45:44.009496Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 4
2025-10-09T21:45:44.010630Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 31 partNo 4 FormedBlobsCount 0 NewHead: Offset 28 PartNo 3 PackedSize 5223541 count 3 nextOffset 31 batches 10
2025-10-09T21:45:44.010658Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 0
2025-10-09T21:45:44.010668Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 1
2025-10-09T21:45:44.010672Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 2
2025-10-09T21:45:44.010675Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 3
2025-10-09T21:45:44.010679Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 4
2025-10-09T21:45:44.011500Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 32 partNo 4 FormedBlobsCount 0 NewHead: Offset 28 PartNo 3 PackedSize 7476739 count 4 nextOffset 32 batches 14
2025-10-09T21:45:44.011526Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 0
2025-10-09T21:45:44.011531Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 1
2025-10-09T21:45:44.012691Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1380: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 33 partNo 1 result is x0000000000_00000000000000000028_00003_0000000004_00014? size 7988760
2025-10-09T21:45:44.012713Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 2
2025-10-09T21:45:44.012719Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 3
2025-10-09T21:45:44.012723Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 4
2025-10-09T21:45:44.012735Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1098: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000028_00003_0000000004_00014? new key d0000000000_00000000000000000028_00003_0000000004_00014? size 7988760 WTime 1760046344012
2025-10-09T21:45:44.013041Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 33 partNo 4 FormedBlobsCount 1 NewHead: Offset 32 PartNo 1 PackedSize 1741199 count 1 nextOffset 33 batches 4
2025-10-09T21:45:44.013056Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 0
2025-10-09T21:45:44.013062Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 1
2025-10-09T21:45:44.013065Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 2
2025-10-09T21:45:44.013072Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 3
2025-10-09T21:45:44.013076Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 4
2025-10-09T21:45:44.013756Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 34 partNo 4 FormedBlobsCount 0 NewHead: Offset 32 PartNo 1 PackedSize 3994397 count 2 nextOffset 34 batches 8
2025-10-09T21:45:44.013773Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 0
2025-10-09T21:45:44.013779Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 1
2025-10-09T21:45:44.013787Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 2
2025-10-09T21:45:44.013791Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 3
2025-10-09T21:45:44.013794Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 4
2025-10-09T21:45:44.014877Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 35 partNo 4 FormedBlobsCount 0 NewHead: Offset 32 PartNo 1 PackedSize 6247595 count 3 nextOffset 35 batches 12
2025-10-09T21:45:44.014899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:44.015014Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 32,3 HeadOffset 25 endOffset 25 curOffset 35 d0000000000_00000000000000000032_00001_0000000003_00011? size 6247585 WTime 1760046344014
2025-10-09T21:45:44.020700Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:45:44.020732Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 25 partNo 0 count 3 size 8295638
2025-10-09T21:45:44.020741Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 28 partNo 3 count 4 size 7988760
2025-10-09T21:45:44.020751Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 32 partNo 1 count 3 size 6247585
2025-10-09T21:45:44.020764Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:44.020767Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:44.020769Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:44.020774Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:44.020782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341483517106135:5801]
2025-10-09T21:45:44.020786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341483517106135:5801]
2025-10-09T21:45:44.020789Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341483517106135:5801]
2025-10-09T21:45:44.020848Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:44.020852Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:44.020855Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:44.020877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:44.020882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.020885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:44.021029Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:44.191578Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:44.193093Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:44.194075Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:44.194151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 578 reason: status is not ok: tablet will be restarted right now sessionId: src_id|50451f2-fcaadf9-ee46b1c4-8e50deca_0
2025-10-09T21:45:44.194228Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:44.194262Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341483878919452:5852]
2025-10-09T21:45:44.194350Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 578 sessionId: src_id|50451f2-fcaadf9-ee46b1c4-8e50deca_0 is DEAD
2025-10-09T21:45:44.194764Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50451f2-fcaadf9-ee46b1c4-8e50deca_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:44.194779Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50451f2-fcaadf9-ee46b1c4-8e50deca_0] Write session will restart in 0.005992s
2025-10-09T21:45:44.194806Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50451f2-fcaadf9-ee46b1c4-8e50deca_0] Write session: Do CDS request
2025-10-09T21:45:44.194813Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50451f2-fcaadf9-ee46b1c4-8e50deca_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:44.201672Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|50451f2-fcaadf9-ee46b1c4-8e50deca_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:44.204067Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:44.204086Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 579
2025-10-09T21:45:44.204311Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 579 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:44.204345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 579 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:44.204351Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=579 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:44.204356Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:44.204904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 579 sessionId:  describe result for acl check
2025-10-09T21:45:44.204951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:44.204953Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:44.204955Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:44.204963Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341488173886767:6364] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:44.205552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341488173886767:6364] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:44.209566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341488173886767:6364] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:44.220741Z node 2 :BS_PROXY_PUT ERROR: dsproxy_put_impl.cpp:72: [a7910bdf135e2514] Result# TEvPutResult {Id# [72075186224037892:577:1:2:1:8295638:0] Status# BLOCKED StatusFlags# { } ErrorReason# "Got VPutResult status# BLOCKED from VDiskId# [82000000:1:0:0:0]" ApproximateFreeSpaceShare# 0} GroupId# 2181038080 Marker# BPP12
2025-10-09T21:45:44.222938Z node 1 :BS_VDISK_PUT ERROR: blobstorage_skeleton.cpp:577: PDiskId# 1 VDISK[82000000:_:0:0:0]: (2181038080) TEvVPut: failed to pass the Hull check; id# [72075186224037892:577:1:2:2:7988760:1] status# {Status# BLOCKED} Marker# BSVS03
2025-10-09T21:45:44.225460Z node 1 :BS_VDISK_PUT ERROR: blobstorage_skeleton.cpp:577: PDiskId# 1 VDISK[82000000:_:0:0:0]: (2181038080) TEvVPut: failed to pass the Hull check; id# [72075186224037892:577:1:2:3:6247585:1] status# {Status# BLOCKED} Marker# BSVS03
2025-10-09T21:45:44.230216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341488173886767:6364] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:44.230235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341488173886767:6364] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:44.230240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341488173886767:6364] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:44.230251Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 579 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:44.248020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:44.255147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:44.255435Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:44.255440Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:44.255452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:44.255516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:44.255560Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341487812073622:5813]
2025-10-09T21:45:44.256184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:44.257177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:44.257249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:44.257616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:44.257667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:44.257668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:44.257670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:44.257671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:44.257673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:44.257674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:44.257676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.257677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.257678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.257684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:44.257697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:44.257700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:44.257704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:44.257708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:44.257712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:44.257716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.257720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.257724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.257733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:44.257742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:44.257748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:44.257755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:44.257760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:44.257765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:44.257770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:44.257776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:44.257781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:44.257786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:44.257790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:44.257794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:44.257799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.257805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.257811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.257819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.257824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.257829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.257856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:44.269204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:44.270821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:44.270841Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:44.270844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:44.270849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:44.270856Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 578 [2:7559341487812073622:5813]
2025-10-09T21:45:44.270873Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:44.270895Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:44.270899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:44.270902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:44.270905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:44.270927Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:44.270934Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:44.270942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:44.271011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.271021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.271023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.271026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:44.271107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:44.271110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.271124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.271136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.271153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:44.271175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341487812073617:5813]
2025-10-09T21:45:44.271178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341487812073617:5813]
2025-10-09T21:45:44.271180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341487812073617:5813]
2025-10-09T21:45:44.271196Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:44.271211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:44.271227Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:44.271230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:44.271276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341487812073617:5813]
2025-10-09T21:45:44.271279Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:44.272278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:44.272282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:44.272285Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:44.272292Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:44.272293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341488173886760:2432], now have 1 active actors on pipe
2025-10-09T21:45:44.272301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341488173886835:6364], now have 1 active actors on pipe
2025-10-09T21:45:44.273096Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 578
2025-10-09T21:45:44.273145Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 578
2025-10-09T21:45:44.273445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:44.273472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:44.273512Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c4e0809a-3b510094-c2143a4f-8c716039_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:44.273539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:44.273550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:44.273554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:44.273570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:44.273594Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:44.273621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:44.273952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:44.273962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:44.273986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:44.274178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 579 partition: 0 MaxSeqNo: 25 sessionId: src_id|c4e0809a-3b510094-c2143a4f-8c716039_0
2025-10-09T21:45:44.274639Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50451f2-fcaadf9-ee46b1c4-8e50deca_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c4e0809a-3b510094-c2143a4f-8c716039_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:44.277385Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4e0809a-3b510094-c2143a4f-8c716039_0] Write session: try to update token
2025-10-09T21:45:44.277404Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4e0809a-3b510094-c2143a4f-8c716039_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:44.295553Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341488173886917:5852]
2025-10-09T21:45:44.301545Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 578, pipe: [1:7559341488173886917:5852]
2025-10-09T21:45:44.301634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341488173886917:5852], now have 1 active actors on pipe
2025-10-09T21:45:44.304008Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:44.304022Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:44.304025Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:44.304027Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:44.304036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341487812073617:5813]
2025-10-09T21:45:44.304041Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341487812073617:5813]
2025-10-09T21:45:44.304045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341487812073617:5813]
2025-10-09T21:45:44.304094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:44.304101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.304104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:44.304246Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:44.307141Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:44.307156Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:44.307162Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:44.310782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 579 sessionId: src_id|c4e0809a-3b510094-c2143a4f-8c716039_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:44.313444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:44.490490Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:44.491203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:44.491221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:44.491616Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:44.492177Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:44.491698Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341488173886917:5852]
2025-10-09T21:45:44.491726Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:44.491731Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:44.491792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 579 reason: error at write: Disconnected sessionId: src_id|c4e0809a-3b510094-c2143a4f-8c716039_0
2025-10-09T21:45:44.491982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 579 sessionId: src_id|c4e0809a-3b510094-c2143a4f-8c716039_0 is DEAD
2025-10-09T21:45:44.492910Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4e0809a-3b510094-c2143a4f-8c716039_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:44.492921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4e0809a-3b510094-c2143a4f-8c716039_0] Write session will restart in 0.006462s
2025-10-09T21:45:44.492946Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4e0809a-3b510094-c2143a4f-8c716039_0] Write session: Do CDS request
2025-10-09T21:45:44.492956Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4e0809a-3b510094-c2143a4f-8c716039_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:44.500620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4e0809a-3b510094-c2143a4f-8c716039_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:44.503249Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:44.503267Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 580
2025-10-09T21:45:44.503470Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 580 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:44.503509Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 580 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:44.503520Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=580 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:44.503524Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:44.504072Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 580 sessionId:  describe result for acl check
2025-10-09T21:45:44.504111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:44.504114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:44.504116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:44.504121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341488173886962:6375] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:44.504621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341488173886962:6375] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:44.509877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341488173886962:6375] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:44.514853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341488173886962:6375] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:44.514876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341488173886962:6375] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:44.514880Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341488173886962:6375] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:44.514891Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 580 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:44.529678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:44.537548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:44.537822Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:44.537827Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:44.537841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:44.537908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:44.537950Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341487812073786:5819]
2025-10-09T21:45:44.538553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:44.542981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:44.543072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:44.543460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:44.543541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:44.543542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:44.543544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:44.543545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:44.543546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:44.543548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:44.543549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.543551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.543552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.543558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:44.543569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:44.543572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:44.543576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:44.543580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:44.543583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:44.543587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.543590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.543594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.543603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:44.543611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:44.543618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:44.543623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:44.543627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:44.543633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:44.543636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:44.543640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:44.543645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:44.543649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:44.543653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:44.543659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:44.543667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.543671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.543675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.543680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.543684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.543688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.543713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:44.554724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:44.555908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:44.555928Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:44.555931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:44.555936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:44.555943Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 579 [2:7559341487812073786:5819]
2025-10-09T21:45:44.555962Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:44.555982Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:44.555986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:44.555990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:44.555993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:44.556014Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:44.556021Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:44.556035Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:44.556101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.556103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.556105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.556108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:44.556188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:44.556191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.556204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.556217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.556235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:44.556256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341487812073780:5819]
2025-10-09T21:45:44.556258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341487812073780:5819]
2025-10-09T21:45:44.556260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341487812073780:5819]
2025-10-09T21:45:44.556280Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:44.556298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:44.556313Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:44.556316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:44.556348Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:44.556352Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:44.556357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341487812073780:5819]
2025-10-09T21:45:44.556358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:44.556360Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:44.556368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:44.556637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341488173887031:6375], now have 1 active actors on pipe
2025-10-09T21:45:44.556649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341488173886947:2432], now have 1 active actors on pipe
2025-10-09T21:45:44.556827Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 579
2025-10-09T21:45:44.556873Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 579
2025-10-09T21:45:44.557107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:44.557124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:44.557156Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d5b47a1b-d92deb3f-15e61d3d-362dde90_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:44.557170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:44.557174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:44.557179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:44.557205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:44.557223Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:44.557245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:44.557556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:44.557571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:44.557605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:44.558223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 580 partition: 0 MaxSeqNo: 25 sessionId: src_id|d5b47a1b-d92deb3f-15e61d3d-362dde90_0
2025-10-09T21:45:44.558807Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4e0809a-3b510094-c2143a4f-8c716039_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d5b47a1b-d92deb3f-15e61d3d-362dde90_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:44.561687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d5b47a1b-d92deb3f-15e61d3d-362dde90_0] Write session: try to update token
2025-10-09T21:45:44.561712Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d5b47a1b-d92deb3f-15e61d3d-362dde90_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:44.581382Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 580 sessionId: src_id|d5b47a1b-d92deb3f-15e61d3d-362dde90_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:44.583973Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:44.582790Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:44.582803Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:44.582805Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:44.582807Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:44.582816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341487812073780:5819]
2025-10-09T21:45:44.582820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341487812073780:5819]
2025-10-09T21:45:44.582824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341487812073780:5819]
2025-10-09T21:45:44.582867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:44.582874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.582876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:44.583047Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:44.586386Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:44.586397Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:44.586400Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:44.594522Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341488173887108:5852]
2025-10-09T21:45:44.598480Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 579, pipe: [1:7559341488173887108:5852]
2025-10-09T21:45:44.819687Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:44.820218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:44.820230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:44.820256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341488173887108:5852], now have 1 active actors on pipe
2025-10-09T21:45:44.821970Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:44.822024Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341488173887108:5852]
2025-10-09T21:45:44.822066Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:44.822072Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:44.822669Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d5b47a1b-d92deb3f-15e61d3d-362dde90_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:44.822693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d5b47a1b-d92deb3f-15e61d3d-362dde90_0] Write session will restart in 0.008071s
2025-10-09T21:45:44.822717Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d5b47a1b-d92deb3f-15e61d3d-362dde90_0] Write session: Do CDS request
2025-10-09T21:45:44.822723Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d5b47a1b-d92deb3f-15e61d3d-362dde90_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:44.822139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 580 reason: error at write: Disconnected sessionId: src_id|d5b47a1b-d92deb3f-15e61d3d-362dde90_0
2025-10-09T21:45:44.822309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 580 sessionId: src_id|d5b47a1b-d92deb3f-15e61d3d-362dde90_0 is DEAD
2025-10-09T21:45:44.823993Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:44.831654Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d5b47a1b-d92deb3f-15e61d3d-362dde90_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:44.835078Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:44.835102Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 581
2025-10-09T21:45:44.835332Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 581 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:44.835390Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 581 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:44.835403Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=581 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:44.835408Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:44.836239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 581 sessionId:  describe result for acl check
2025-10-09T21:45:44.836282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:44.836283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:44.836290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:44.836297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341488173887126:6380] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:44.836892Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341488173887126:6380] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:44.840742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341488173887126:6380] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:44.845829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341488173887126:6380] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:44.845849Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341488173887126:6380] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:44.845852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341488173887126:6380] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:44.845870Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 581 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:44.878934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:44.884882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:44.885128Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:44.885131Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:44.885145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:44.885210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:44.885247Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341487812073940:5821]
2025-10-09T21:45:44.885822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:44.886435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:44.886486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:44.886827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:44.886869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:44.886871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:44.886872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:44.886874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:44.886876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:44.886877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:44.886878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.886880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.886881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.886887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:44.886900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:44.886904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:44.886909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:44.886913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:44.886917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:44.886922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.886925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.886929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.886936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:44.886944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:44.886950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:44.886955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:44.886959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:44.886964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:44.886968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:44.886973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:44.886977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:44.886982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:44.886986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:44.886991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:44.886997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.887001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.887018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.887023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.887028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.887033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.887062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:44.897956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:44.899816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:44.899826Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:44.899830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:44.899837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:44.899847Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 580 [2:7559341487812073940:5821]
2025-10-09T21:45:44.899868Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:44.899897Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:44.899901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:44.899906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:44.899908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:44.899934Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:44.899943Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:44.899953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:44.900045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.900048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.900050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.900053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:44.900147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:44.900152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.900166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:44.900176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:44.900192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:44.900214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341487812073935:5821]
2025-10-09T21:45:44.900216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341487812073935:5821]
2025-10-09T21:45:44.900218Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341487812073935:5821]
2025-10-09T21:45:44.900230Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:44.900255Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:44.900272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:44.900276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:44.900333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341487812073935:5821]
2025-10-09T21:45:44.900338Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:44.902302Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 580
2025-10-09T21:45:44.902358Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 580
2025-10-09T21:45:44.901389Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:44.901405Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:44.901408Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:44.901418Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:44.906758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341488173887117:2432], now have 1 active actors on pipe
2025-10-09T21:45:44.906776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341488173887165:6380], now have 1 active actors on pipe
2025-10-09T21:45:44.906808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:44.906815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:44.906857Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|51105383-c51d3a20-aa41b0cc-b7a72d77_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:44.906874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:44.906880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:44.906883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:44.906899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:44.906918Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:44.906941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:44.908541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:44.908560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:44.908600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:44.908794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 581 partition: 0 MaxSeqNo: 25 sessionId: src_id|51105383-c51d3a20-aa41b0cc-b7a72d77_0
2025-10-09T21:45:44.909313Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d5b47a1b-d92deb3f-15e61d3d-362dde90_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|51105383-c51d3a20-aa41b0cc-b7a72d77_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:44.912061Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51105383-c51d3a20-aa41b0cc-b7a72d77_0] Write session: try to update token
2025-10-09T21:45:44.912097Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51105383-c51d3a20-aa41b0cc-b7a72d77_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:44.924541Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341488173887281:5852]
2025-10-09T21:45:44.936311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341488173887281:5852], now have 1 active actors on pipe
2025-10-09T21:45:44.938624Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 580, pipe: [1:7559341488173887281:5852]
2025-10-09T21:45:44.944208Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 581 sessionId: src_id|51105383-c51d3a20-aa41b0cc-b7a72d77_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:44.946873Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:44.950646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:44.950668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:44.950710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:44.951681Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:44.968361Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:44.968379Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:44.968381Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:44.968383Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:44.968394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341487812073935:5821]
2025-10-09T21:45:44.968398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341487812073935:5821]
2025-10-09T21:45:44.968402Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341487812073935:5821]
2025-10-09T21:45:44.968443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:44.968450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:44.968452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:44.968622Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:44.971734Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:44.971741Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:44.971745Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:45.530524Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:45.547510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:45.547540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:45.547716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:45.547769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:45.547825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:45.547882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:45.548180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:45.548187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:45.548252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:45.548310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:45.548370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:45.548435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:45.548474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:45.548477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:45.548546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:45.548600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:45.548656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:45.548731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:45.548770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:45.548773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:45.548840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:45.548898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:45.548955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:45.549010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:45.549047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:45.549050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:45.549119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:45.549177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:45.549234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:45.549298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:45.549356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:45.549359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:45.549422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:45.549472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:45.549525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:45.549579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:45.549635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:45.549638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:45.549702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:45.549751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:45.549809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:45.549868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:45.549906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:45.549909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:45.549973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:45.570075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:45.570185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:45.570250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:45.570297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:45.570303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:45.570378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:45.570432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:45.570487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:45.570546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:45.570676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:45.570680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:45.570777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:45.570829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:45.570885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:45.570941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:45.571004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:45.571010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:45.572120Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:45.575121Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51105383-c51d3a20-aa41b0cc-b7a72d77_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:45.575135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51105383-c51d3a20-aa41b0cc-b7a72d77_0] Write session will restart in 0.005885s
2025-10-09T21:45:45.575159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51105383-c51d3a20-aa41b0cc-b7a72d77_0] Write session: Do CDS request
2025-10-09T21:45:45.575166Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51105383-c51d3a20-aa41b0cc-b7a72d77_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:45.574306Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341488173887281:5852]
2025-10-09T21:45:45.574340Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:45.574378Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:45.574424Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 581 reason: status is not ok: tablet will be restarted right now sessionId: src_id|51105383-c51d3a20-aa41b0cc-b7a72d77_0
2025-10-09T21:45:45.574614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 581 sessionId: src_id|51105383-c51d3a20-aa41b0cc-b7a72d77_0 is DEAD
2025-10-09T21:45:45.582764Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51105383-c51d3a20-aa41b0cc-b7a72d77_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:45.586205Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:45.586224Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 582
2025-10-09T21:45:45.586451Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 582 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:45.586482Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 582 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:45.586488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=582 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:45.586491Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:45.590002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 582 sessionId:  describe result for acl check
2025-10-09T21:45:45.590071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:45.590077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:45.590079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:45.590085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341492468854634:6392] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:45.590857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341492468854634:6392] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:45.603904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341492468854634:6392] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:45.630007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341492468854634:6392] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:45.630043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341492468854634:6392] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:45.630048Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341492468854634:6392] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:45.630059Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 582 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:45.632014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:45.638502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:45.638799Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:45.638804Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:45.638817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:45.638886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:45.638928Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341492107041437:5837]
2025-10-09T21:45:45.639499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:45.642797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:45.642887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:45.643297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:45.643370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:45.643372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:45.643374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:45.643375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:45.643376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:45.643378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:45.643389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:45.643390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:45.643392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:45.643399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:45.643410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:45.643413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:45.643417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:45.643421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:45.643425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:45.643429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:45.643433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:45.643437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:45.643450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:45.643469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:45.643482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:45.643488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:45.643492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:45.643498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:45.643502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:45.643507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:45.643512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:45.643517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:45.643522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:45.643527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:45.643533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:45.643538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:45.643542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:45.643547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:45.643553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:45.643557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:45.643597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:45.654233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:45.656032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:45.656040Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:45.656043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:45.656047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:45.656056Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 581 [2:7559341492107041437:5837]
2025-10-09T21:45:45.656070Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:45.656091Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:45.656095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:45.656098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:45.656100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:45.656118Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:45.656124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:45.656133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:45.656206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:45.656208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:45.656210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:45.656214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:45.656292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:45.656295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:45.656310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:45.656321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:45.656337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:45.656360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341492107041431:5837]
2025-10-09T21:45:45.656362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341492107041431:5837]
2025-10-09T21:45:45.656364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341492107041431:5837]
2025-10-09T21:45:45.656391Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:45.658111Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 581
2025-10-09T21:45:45.658168Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 581
2025-10-09T21:45:45.656406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:45.656425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:45.656429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:45.656474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341492107041431:5837]
2025-10-09T21:45:45.656477Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:45.657227Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:45.657230Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:45.657233Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:45.657240Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:45.663015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341492468854628:2432], now have 1 active actors on pipe
2025-10-09T21:45:45.663036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341492468854777:6392], now have 1 active actors on pipe
2025-10-09T21:45:45.663067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:45.663075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:45.663116Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dbeb982e-468781e0-ff406e1e-afb20c6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:45.663130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:45.663136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:45.663140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:45.663154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:45.663174Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:45.663196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:45.666771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 582 partition: 0 MaxSeqNo: 25 sessionId: src_id|dbeb982e-468781e0-ff406e1e-afb20c6_0
2025-10-09T21:45:45.666565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:45.666582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:45.666617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:45.667192Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51105383-c51d3a20-aa41b0cc-b7a72d77_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dbeb982e-468781e0-ff406e1e-afb20c6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:45.669729Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dbeb982e-468781e0-ff406e1e-afb20c6_0] Write session: try to update token
2025-10-09T21:45:45.669750Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dbeb982e-468781e0-ff406e1e-afb20c6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:45.678099Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341492468854788:5852]
2025-10-09T21:45:45.682328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341492468854788:5852], now have 1 active actors on pipe
2025-10-09T21:45:45.682235Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 581, pipe: [1:7559341492468854788:5852]
2025-10-09T21:45:45.697768Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:45.702141Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:45.702158Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:45.702161Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:45.702173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341492107041431:5837]
2025-10-09T21:45:45.702179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341492107041431:5837]
2025-10-09T21:45:45.702184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341492107041431:5837]
2025-10-09T21:45:45.702268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:45.702277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:45.702280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:45.702445Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:45.706226Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:45.706241Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:45.706245Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:45.728531Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 582 sessionId: src_id|dbeb982e-468781e0-ff406e1e-afb20c6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:45.747161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:45.929427Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:45.930116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:45.930143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:45.930833Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:45.931193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbeb982e-468781e0-ff406e1e-afb20c6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:45.931205Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbeb982e-468781e0-ff406e1e-afb20c6_0] Write session will restart in 0.009002s
2025-10-09T21:45:45.931230Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbeb982e-468781e0-ff406e1e-afb20c6_0] Write session: Do CDS request
2025-10-09T21:45:45.931237Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbeb982e-468781e0-ff406e1e-afb20c6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:45.930471Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:45.930526Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:45.930528Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:45.930587Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 582 reason: error at write: Disconnected sessionId: src_id|dbeb982e-468781e0-ff406e1e-afb20c6_0
2025-10-09T21:45:45.930688Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341492468854788:5852]
2025-10-09T21:45:45.930780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 582 sessionId: src_id|dbeb982e-468781e0-ff406e1e-afb20c6_0 is DEAD
2025-10-09T21:45:45.941680Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dbeb982e-468781e0-ff406e1e-afb20c6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:45.942246Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:45.942265Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 583
2025-10-09T21:45:45.942450Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 583 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:45.942497Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 583 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:45.942510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=583 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:45.942514Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:45.943048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 583 sessionId:  describe result for acl check
2025-10-09T21:45:45.943092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:45.943096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:45.943098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:45.943103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341492468854809:6397] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:45.943690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341492468854809:6397] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:45.946471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341492468854809:6397] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:45.955298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341492468854809:6397] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:45.955321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341492468854809:6397] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:45.955326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341492468854809:6397] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:45.955337Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 583 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:45.962345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:45.967273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:45.967537Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:45.967549Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:45.967562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:45.967621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:45.967669Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341492107041592:5839]
2025-10-09T21:45:45.968172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:45.968612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:45.968676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:45.969033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:45.969098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:45.969106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:45.969107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:45.969109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:45.969110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:45.969112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:45.969113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:45.969114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:45.969115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:45.969121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:45.969133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:45.969138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:45.969143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:45.969147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:45.969151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:45.969156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:45.969161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:45.969165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:45.969173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:45.969182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:45.969189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:45.969195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:45.969202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:45.969208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:45.969213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:45.969219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:45.969224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:45.969230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:45.969235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:45.969253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:45.969264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:45.969269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:45.969275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:45.969280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:45.969291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:45.969297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:45.969335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:45.978207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:45.979324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:45.979331Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:45.979334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:45.979338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:45.979346Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 582 [2:7559341492107041592:5839]
2025-10-09T21:45:45.979361Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:45.979384Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:45.979388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:45.979391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:45.979394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:45.979413Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:45.979419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:45.979426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:45.979494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:45.979495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:45.979497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:45.979500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:45.979582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:45.979585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:45.979596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:45.979605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:45.979619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:45.979640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341492107041587:5839]
2025-10-09T21:45:45.979642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341492107041587:5839]
2025-10-09T21:45:45.979644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341492107041587:5839]
2025-10-09T21:45:45.979663Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:45.979678Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:45.979690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:45.979693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:45.979732Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:45.979734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341492107041587:5839]
2025-10-09T21:45:45.979736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:45.979736Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:45.979742Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:45.979748Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:45.980022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341492468854904:6397], now have 1 active actors on pipe
2025-10-09T21:45:45.980030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341492468854794:2432], now have 1 active actors on pipe
2025-10-09T21:45:45.980184Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 582
2025-10-09T21:45:45.980186Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 582
2025-10-09T21:45:45.980326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:45.980341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:45.980375Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|14c864ad-999ad064-a9515ec-22ea26b0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:45.980400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:45.980409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:45.980412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:45.980423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:45.980435Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:45.980452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:45.981075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:45.981089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:45.981111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:45.981232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 583 partition: 0 MaxSeqNo: 25 sessionId: src_id|14c864ad-999ad064-a9515ec-22ea26b0_0
2025-10-09T21:45:45.981675Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbeb982e-468781e0-ff406e1e-afb20c6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|14c864ad-999ad064-a9515ec-22ea26b0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:45.984632Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14c864ad-999ad064-a9515ec-22ea26b0_0] Write session: try to update token
2025-10-09T21:45:45.984653Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14c864ad-999ad064-a9515ec-22ea26b0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:46.001193Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:46.001212Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:46.001215Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:46.001217Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:46.001226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341492107041587:5839]
2025-10-09T21:45:46.001230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341492107041587:5839]
2025-10-09T21:45:46.001234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341492107041587:5839]
2025-10-09T21:45:46.001282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:46.001291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.001294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:46.001430Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:46.004205Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:46.004220Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:46.004224Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:46.010738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 583 sessionId: src_id|14c864ad-999ad064-a9515ec-22ea26b0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:46.013772Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:46.030628Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341496763822256:5852]
2025-10-09T21:45:46.031247Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 582, pipe: [1:7559341496763822256:5852]
2025-10-09T21:45:46.195161Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341496763822256:5852]
2025-10-09T21:45:46.195217Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:46.195219Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:46.195295Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 583 reason: error at write: Disconnected sessionId: src_id|14c864ad-999ad064-a9515ec-22ea26b0_0
2025-10-09T21:45:46.195398Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:46.195478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 583 sessionId: src_id|14c864ad-999ad064-a9515ec-22ea26b0_0 is DEAD
2025-10-09T21:45:46.194246Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:46.194773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:46.194785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:46.194831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341496763822256:5852], now have 1 active actors on pipe
2025-10-09T21:45:46.196248Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:46.198311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14c864ad-999ad064-a9515ec-22ea26b0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:46.198324Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14c864ad-999ad064-a9515ec-22ea26b0_0] Write session will restart in 0.009608s
2025-10-09T21:45:46.198351Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14c864ad-999ad064-a9515ec-22ea26b0_0] Write session: Do CDS request
2025-10-09T21:45:46.198358Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14c864ad-999ad064-a9515ec-22ea26b0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:46.209589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14c864ad-999ad064-a9515ec-22ea26b0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:46.209969Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:46.209989Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 584
2025-10-09T21:45:46.210227Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 584 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:46.210266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 584 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:46.210278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=584 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:46.210282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:46.210934Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 584 sessionId:  describe result for acl check
2025-10-09T21:45:46.210983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:46.210993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:46.210994Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:46.211001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341496763822305:6402] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:46.211560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341496763822305:6402] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:46.217104Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341496763822305:6402] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:46.223281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341496763822305:6402] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:46.223302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341496763822305:6402] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:46.223307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341496763822305:6402] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:46.223318Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 584 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:46.239082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:46.239754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:46.240002Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:46.240006Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:46.240019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:46.240267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:46.240392Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341496402009044:5841]
2025-10-09T21:45:46.241056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:46.241600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:46.241646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:46.241956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:46.242011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:46.242012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:46.242014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:46.242016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:46.242017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:46.242018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:46.242019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.242021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.242022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.242051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:46.242065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:46.242070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:46.242075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:46.242079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:46.242084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:46.242089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.242094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.242098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.242108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:46.242117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:46.242124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:46.242144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:46.242150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:46.242156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:46.242161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:46.242167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:46.242173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:46.242179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:46.242184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:46.242191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:46.242196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.242203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.242207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.242213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.242221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.242227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.242265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:46.252459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:46.254264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:46.254270Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:46.254273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:46.254278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:46.254287Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 583 [2:7559341496402009044:5841]
2025-10-09T21:45:46.254305Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:46.254326Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:46.254330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:46.254334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:46.254336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:46.254355Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:46.254362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:46.254369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:46.254438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.254442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.254444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.254447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:46.254526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:46.254529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.254545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.254559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.254580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:46.254601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341496402009038:5841]
2025-10-09T21:45:46.254604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341496402009038:5841]
2025-10-09T21:45:46.254606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341496402009038:5841]
2025-10-09T21:45:46.254616Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:46.254632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:46.254648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:46.254652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:46.254664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341496763822410:6402], now have 1 active actors on pipe
2025-10-09T21:45:46.254680Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:46.254683Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:46.254685Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:46.254703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341496402009038:5841]
2025-10-09T21:45:46.254706Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:46.254710Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:46.254711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341496763822264:2432], now have 1 active actors on pipe
2025-10-09T21:45:46.255353Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 583
2025-10-09T21:45:46.255386Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 583
2025-10-09T21:45:46.259965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:46.259986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:46.260034Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3025699f-d34d3ae6-eb70c989-d76c60be_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:46.260059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:46.260066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:46.260069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:46.260083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:46.260103Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:46.260139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:46.260658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:46.260666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:46.260684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:46.260809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 584 partition: 0 MaxSeqNo: 25 sessionId: src_id|3025699f-d34d3ae6-eb70c989-d76c60be_0
2025-10-09T21:45:46.262316Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14c864ad-999ad064-a9515ec-22ea26b0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3025699f-d34d3ae6-eb70c989-d76c60be_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:46.265105Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3025699f-d34d3ae6-eb70c989-d76c60be_0] Write session: try to update token
2025-10-09T21:45:46.265128Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3025699f-d34d3ae6-eb70c989-d76c60be_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:46.281643Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:46.281660Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:46.281662Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:46.281665Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:46.281676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341496402009038:5841]
2025-10-09T21:45:46.281680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341496402009038:5841]
2025-10-09T21:45:46.281684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341496402009038:5841]
2025-10-09T21:45:46.281735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:46.281748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.281750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:46.281902Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:46.282139Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:46.282152Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:46.282156Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:46.287155Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 584 sessionId: src_id|3025699f-d34d3ae6-eb70c989-d76c60be_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:46.289831Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:46.295324Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341496763822428:5852]
2025-10-09T21:45:46.298581Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 583, pipe: [1:7559341496763822428:5852]
2025-10-09T21:45:46.474657Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:46.475210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:46.475222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:46.475247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341496763822428:5852], now have 1 active actors on pipe
2025-10-09T21:45:46.476042Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341496763822428:5852]
2025-10-09T21:45:46.476072Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:46.476103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:46.476106Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:46.476160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 584 reason: error at write: Disconnected sessionId: src_id|3025699f-d34d3ae6-eb70c989-d76c60be_0
2025-10-09T21:45:46.476343Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 584 sessionId: src_id|3025699f-d34d3ae6-eb70c989-d76c60be_0 is DEAD
2025-10-09T21:45:46.476911Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3025699f-d34d3ae6-eb70c989-d76c60be_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:46.476923Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3025699f-d34d3ae6-eb70c989-d76c60be_0] Write session will restart in 0.005248s
2025-10-09T21:45:46.476945Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3025699f-d34d3ae6-eb70c989-d76c60be_0] Write session: Do CDS request
2025-10-09T21:45:46.476954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3025699f-d34d3ae6-eb70c989-d76c60be_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:46.478424Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:46.486373Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3025699f-d34d3ae6-eb70c989-d76c60be_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:46.489739Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:46.489758Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 585
2025-10-09T21:45:46.489940Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 585 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:46.489973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 585 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:46.489983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=585 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:46.489987Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:46.490665Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 585 sessionId:  describe result for acl check
2025-10-09T21:45:46.490698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:46.490700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:46.490702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:46.490707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341496763822461:6412] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:46.491166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341496763822461:6412] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:46.497179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341496763822461:6412] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:46.499844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341496763822461:6412] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:46.499868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341496763822461:6412] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:46.499872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341496763822461:6412] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:46.499881Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 585 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:46.512570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:46.517492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:46.517731Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:46.517742Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:46.517755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:46.517811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:46.517859Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341496402009243:5855]
2025-10-09T21:45:46.518494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:46.519131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:46.519179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:46.519529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:46.519574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:46.519576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:46.519577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:46.519579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:46.519580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:46.519582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:46.519583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.519585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.519586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.519592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:46.519604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:46.519608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:46.519613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:46.519617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:46.519622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:46.519627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.519631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.519636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.519644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:46.519653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:46.519660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:46.519666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:46.519672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:46.519678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:46.519683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:46.519689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:46.519694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:46.519700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:46.519705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:46.519711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:46.519717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.519723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.519728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.519734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.519739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.519745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.519778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:46.538816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:46.540267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:46.540273Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:46.540277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:46.540283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:46.540291Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 584 [2:7559341496402009243:5855]
2025-10-09T21:45:46.540307Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:46.540328Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:46.540332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:46.540335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:46.540337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:46.540358Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:46.540365Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:46.540373Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:46.540452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.540454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.540457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.540460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:46.540538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:46.540541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.540556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.540569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.540588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:46.540608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341496402009238:5855]
2025-10-09T21:45:46.540612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341496402009238:5855]
2025-10-09T21:45:46.540615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341496402009238:5855]
2025-10-09T21:45:46.540631Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:46.540646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:46.540664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:46.540667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:46.540703Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:46.540709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341496402009238:5855]
2025-10-09T21:45:46.540712Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:46.540715Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:46.540717Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:46.540725Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:46.541430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341496763822447:2432], now have 1 active actors on pipe
2025-10-09T21:45:46.541444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341496763822509:6412], now have 1 active actors on pipe
2025-10-09T21:45:46.541402Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 584
2025-10-09T21:45:46.541447Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 584
2025-10-09T21:45:46.542850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:46.542879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:46.542916Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|52280a0a-7a13559c-833992a2-61ddb729_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:46.542927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:46.542933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:46.542936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:46.542958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:46.542973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:46.542993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:46.547500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 585 partition: 0 MaxSeqNo: 25 sessionId: src_id|52280a0a-7a13559c-833992a2-61ddb729_0
2025-10-09T21:45:46.548137Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3025699f-d34d3ae6-eb70c989-d76c60be_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|52280a0a-7a13559c-833992a2-61ddb729_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:46.546635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:46.546657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:46.546693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:46.550932Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52280a0a-7a13559c-833992a2-61ddb729_0] Write session: try to update token
2025-10-09T21:45:46.550949Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52280a0a-7a13559c-833992a2-61ddb729_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:46.571721Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:46.571742Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:46.571745Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:46.571747Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:46.571757Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341496402009238:5855]
2025-10-09T21:45:46.571763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341496402009238:5855]
2025-10-09T21:45:46.571766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341496402009238:5855]
2025-10-09T21:45:46.571856Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:46.571864Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:46.571869Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:46.571894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:46.571901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.571904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:46.572054Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:46.572979Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 585 sessionId: src_id|52280a0a-7a13559c-833992a2-61ddb729_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:46.575871Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:46.575950Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341496763822618:5852]
2025-10-09T21:45:46.576368Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 584, pipe: [1:7559341496763822618:5852]
2025-10-09T21:45:46.650367Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: CLIENT_DEADLINE_EXCEEDED. Description: [ { <main>: Error: GRpc error: (4): Deadline Exceeded } { <main>: Error: Grpc error response on endpoint localhost:14704 } ]
2025-10-09T21:45:46.743114Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:46.743820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:46.743832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:46.743859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341496763822618:5852], now have 1 active actors on pipe
2025-10-09T21:45:46.747618Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52280a0a-7a13559c-833992a2-61ddb729_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:46.747633Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52280a0a-7a13559c-833992a2-61ddb729_0] Write session will restart in 0.009226s
2025-10-09T21:45:46.747659Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52280a0a-7a13559c-833992a2-61ddb729_0] Write session: Do CDS request
2025-10-09T21:45:46.746675Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:46.746868Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:46.746885Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:46.746963Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 585 reason: error at write: Disconnected sessionId: src_id|52280a0a-7a13559c-833992a2-61ddb729_0
2025-10-09T21:45:46.747156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 585 sessionId: src_id|52280a0a-7a13559c-833992a2-61ddb729_0 is DEAD
2025-10-09T21:45:46.747300Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:46.747333Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341496763822618:5852]
2025-10-09T21:45:46.747666Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52280a0a-7a13559c-833992a2-61ddb729_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:46.759351Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52280a0a-7a13559c-833992a2-61ddb729_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:46.762244Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:46.762268Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 586
2025-10-09T21:45:46.762506Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 586 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:46.762554Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 586 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:46.762560Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=586 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:46.762564Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:46.763302Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 586 sessionId:  describe result for acl check
2025-10-09T21:45:46.763338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:46.763340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:46.763341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:46.763347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341496763822636:6420] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:46.763855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341496763822636:6420] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:46.769342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341496763822636:6420] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:46.782655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341496763822636:6420] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:46.782680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341496763822636:6420] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:46.782685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341496763822636:6420] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:46.782713Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 586 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:46.811240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:46.826105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:46.826411Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:46.826425Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:46.826439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:46.826518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:46.826574Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341496402009401:5859]
2025-10-09T21:45:46.827185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:46.828122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:46.828184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:46.828578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:46.828645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:46.828646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:46.828647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:46.828648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:46.828648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:46.828650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:46.828651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.828651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.828652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.828657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:46.828664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:46.828667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:46.828669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:46.828671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:46.828673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:46.828675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.828677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.828680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.828687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:46.828692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:46.828699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:46.828706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:46.828710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:46.828714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:46.828719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:46.828723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:46.828728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:46.828734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:46.828738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:46.828743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:46.828750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.828756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.828761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.828767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.828772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.828778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.828810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:46.837979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:46.839973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:46.839993Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:46.839996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:46.840001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:46.840010Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 585 [2:7559341496402009401:5859]
2025-10-09T21:45:46.840026Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:46.840049Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:46.840053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:46.840056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:46.840059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:46.840081Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:46.840088Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:46.840095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:46.840194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.840196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.840198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.840201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:46.840282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:46.840285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.840300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:46.840311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:46.840326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:46.840347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341496402009396:5859]
2025-10-09T21:45:46.840349Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341496402009396:5859]
2025-10-09T21:45:46.840351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341496402009396:5859]
2025-10-09T21:45:46.840367Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:46.840382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:46.840395Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:46.840399Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:46.840456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341496402009396:5859]
2025-10-09T21:45:46.840460Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:46.840829Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:46.840843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:46.840846Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:46.840853Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:46.841550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341496763822693:6420], now have 1 active actors on pipe
2025-10-09T21:45:46.841567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341496763822626:2432], now have 1 active actors on pipe
2025-10-09T21:45:46.841650Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 585
2025-10-09T21:45:46.841699Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 585
2025-10-09T21:45:46.842317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:46.842335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:46.842380Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3333811e-52e155ac-792341a7-114de444_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:46.842398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:46.842403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:46.842405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:46.842422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:46.842445Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:46.842470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:46.842788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:46.842802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:46.842828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:46.843396Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52280a0a-7a13559c-833992a2-61ddb729_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3333811e-52e155ac-792341a7-114de444_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:46.846136Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3333811e-52e155ac-792341a7-114de444_0] Write session: try to update token
2025-10-09T21:45:46.846155Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3333811e-52e155ac-792341a7-114de444_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:46.842945Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 586 partition: 0 MaxSeqNo: 25 sessionId: src_id|3333811e-52e155ac-792341a7-114de444_0
2025-10-09T21:45:46.848895Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341496763822790:5852]
2025-10-09T21:45:46.849494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341496763822790:5852], now have 1 active actors on pipe
2025-10-09T21:45:46.849596Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 585, pipe: [1:7559341496763822790:5852]
2025-10-09T21:45:46.873760Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:46.873783Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:46.873786Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:46.873788Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:46.873797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341496402009396:5859]
2025-10-09T21:45:46.873802Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341496402009396:5859]
2025-10-09T21:45:46.873806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341496402009396:5859]
2025-10-09T21:45:46.873856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:46.873856Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:46.873862Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:46.873863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:46.873866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:46.873877Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:46.874442Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:46.881281Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 586 sessionId: src_id|3333811e-52e155ac-792341a7-114de444_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:46.883809Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:47.083273Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:47.083591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:47.083601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:47.084372Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341496763822790:5852]
2025-10-09T21:45:47.084411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:47.084414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:47.084473Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 586 reason: error at write: Disconnected sessionId: src_id|3333811e-52e155ac-792341a7-114de444_0
2025-10-09T21:45:47.084650Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 586 sessionId: src_id|3333811e-52e155ac-792341a7-114de444_0 is DEAD
2025-10-09T21:45:47.084869Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:47.084419Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:47.086224Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3333811e-52e155ac-792341a7-114de444_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:47.086237Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3333811e-52e155ac-792341a7-114de444_0] Write session will restart in 0.008827s
2025-10-09T21:45:47.086260Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3333811e-52e155ac-792341a7-114de444_0] Write session: Do CDS request
2025-10-09T21:45:47.086269Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3333811e-52e155ac-792341a7-114de444_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:47.096603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3333811e-52e155ac-792341a7-114de444_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:47.096798Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:47.096820Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 587
2025-10-09T21:45:47.097049Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 587 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:47.097097Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 587 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:47.097103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=587 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:47.097107Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:47.097672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 587 sessionId:  describe result for acl check
2025-10-09T21:45:47.097723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:47.097725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:47.097727Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:47.097733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341501058790119:6425] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:47.098517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341501058790119:6425] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:47.103564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341501058790119:6425] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:47.116239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341501058790119:6425] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:47.116276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341501058790119:6425] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:47.116281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341501058790119:6425] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:47.116292Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 587 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:47.117055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:47.121756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:47.122049Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:47.122061Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:47.122075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:47.122155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:47.122209Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341500696976856:5861]
2025-10-09T21:45:47.122847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:47.123243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:47.123322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:47.123706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:47.123777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:47.123785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:47.123787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:47.123788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:47.123789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:47.123790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:47.123791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.123792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.123794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.123800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:47.123810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:47.123814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:47.123818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:47.123822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:47.123826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:47.123830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.123833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.123837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.123846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:47.123856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:47.123863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:47.123870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:47.123875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:47.123880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:47.123885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:47.123889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:47.123895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:47.123900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:47.123905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:47.123911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:47.123916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.123922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.123927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.123933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.123938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.123943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.123975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:47.131729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:47.132936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:47.132955Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:47.132957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:47.132962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:47.132969Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 586 [2:7559341500696976856:5861]
2025-10-09T21:45:47.132995Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:47.133021Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:47.133032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:47.133036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:47.133038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:47.133060Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:47.133072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:47.133079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:47.133152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.133160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.133162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.133165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:47.133245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:47.133258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.133276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.133288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.133308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:47.133336Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341500696976849:5861]
2025-10-09T21:45:47.133347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341500696976849:5861]
2025-10-09T21:45:47.133349Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341500696976849:5861]
2025-10-09T21:45:47.133370Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:47.133395Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:47.133412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:47.133416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:47.133472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341500696976849:5861]
2025-10-09T21:45:47.133483Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:47.133974Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 586
2025-10-09T21:45:47.133983Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 586
2025-10-09T21:45:47.133681Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:47.133693Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:47.133696Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:47.133703Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:47.133887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341501058790105:2432], now have 1 active actors on pipe
2025-10-09T21:45:47.133896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341501058790258:6425], now have 1 active actors on pipe
2025-10-09T21:45:47.135995Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3333811e-52e155ac-792341a7-114de444_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|12f3e8ca-1905873c-99aa9051-5009bb3a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:47.134574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:47.134600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:47.134635Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|12f3e8ca-1905873c-99aa9051-5009bb3a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:47.134658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:47.134663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:47.134667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:47.134680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:47.134696Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:47.134717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:47.135393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:47.135401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:47.135437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:47.135564Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 587 partition: 0 MaxSeqNo: 25 sessionId: src_id|12f3e8ca-1905873c-99aa9051-5009bb3a_0
2025-10-09T21:45:47.139263Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12f3e8ca-1905873c-99aa9051-5009bb3a_0] Write session: try to update token
2025-10-09T21:45:47.139286Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12f3e8ca-1905873c-99aa9051-5009bb3a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:47.154854Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:47.154873Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:47.154876Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:47.154879Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:47.154889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341500696976849:5861]
2025-10-09T21:45:47.154895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341500696976849:5861]
2025-10-09T21:45:47.154899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341500696976849:5861]
2025-10-09T21:45:47.154948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:47.154961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.154962Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:47.154964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:47.154968Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:47.154972Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:47.155196Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:47.173242Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 587 sessionId: src_id|12f3e8ca-1905873c-99aa9051-5009bb3a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:47.176069Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:47.185710Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341501058790271:5852]
2025-10-09T21:45:47.186255Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 586, pipe: [1:7559341501058790271:5852]
2025-10-09T21:45:47.325828Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:47.326801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:47.326823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:47.326847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341501058790271:5852], now have 1 active actors on pipe
2025-10-09T21:45:47.327177Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341501058790271:5852]
2025-10-09T21:45:47.327216Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:47.327253Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:47.327256Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:47.327335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 587 reason: error at write: Disconnected sessionId: src_id|12f3e8ca-1905873c-99aa9051-5009bb3a_0
2025-10-09T21:45:47.327535Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 587 sessionId: src_id|12f3e8ca-1905873c-99aa9051-5009bb3a_0 is DEAD
2025-10-09T21:45:47.328159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12f3e8ca-1905873c-99aa9051-5009bb3a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:47.328171Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12f3e8ca-1905873c-99aa9051-5009bb3a_0] Write session will restart in 0.005407s
2025-10-09T21:45:47.328197Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12f3e8ca-1905873c-99aa9051-5009bb3a_0] Write session: Do CDS request
2025-10-09T21:45:47.328207Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12f3e8ca-1905873c-99aa9051-5009bb3a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:47.327576Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:47.334717Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12f3e8ca-1905873c-99aa9051-5009bb3a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:47.338403Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:47.338430Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 588
2025-10-09T21:45:47.342112Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 588 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:47.342203Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 588 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:47.342210Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=588 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:47.342215Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:47.342706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 588 sessionId:  describe result for acl check
2025-10-09T21:45:47.342744Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:47.342746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:47.342748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:47.342755Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341501058790283:6429] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:47.343355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341501058790283:6429] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:47.347390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341501058790283:6429] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:47.351307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341501058790283:6429] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:47.351325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341501058790283:6429] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:47.351331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341501058790283:6429] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:47.351341Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 588 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:47.369844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:47.374321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:47.374576Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:47.374589Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:47.374604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:47.374673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:47.374724Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341500696977015:5863]
2025-10-09T21:45:47.375381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:47.379203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:47.379285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:47.379687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:47.379736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:47.379738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:47.379739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:47.379740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:47.379742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:47.379744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:47.379745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.379746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.379748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.379755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:47.379767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:47.379772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:47.379776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:47.379781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:47.379785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:47.379789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.379794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.379798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.379809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:47.379819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:47.379826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:47.379833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:47.379838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:47.379843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:47.379848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:47.379854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:47.379860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:47.379865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:47.379870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:47.379876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:47.379882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.379888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.379893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.379899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.379905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.379911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.379944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:47.400626Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 587
2025-10-09T21:45:47.400678Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 587
2025-10-09T21:45:47.398855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:47.400081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:47.400093Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:47.400097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:47.400103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:47.400112Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 587 [2:7559341500696977015:5863]
2025-10-09T21:45:47.400133Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:47.400157Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:47.400162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:47.400165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:47.400167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:47.400187Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:47.400194Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:47.400202Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:47.400271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.400273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.400275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.400278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:47.400362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:47.400365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.400381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.400394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.400412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:47.400436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341500696977009:5863]
2025-10-09T21:45:47.400438Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341500696977009:5863]
2025-10-09T21:45:47.400440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341500696977009:5863]
2025-10-09T21:45:47.400449Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:47.400474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341501058790274:2432], now have 1 active actors on pipe
2025-10-09T21:45:47.400474Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:47.400478Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:47.400480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:47.400483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:47.400495Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:47.400499Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:47.400505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341501058790322:6429], now have 1 active actors on pipe
2025-10-09T21:45:47.400553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341500696977009:5863]
2025-10-09T21:45:47.400568Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:47.400606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:47.401028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:47.401039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:47.401070Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a97179e6-ec00a4b-b0f7312-127617fe_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:47.401079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:47.401086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:47.401089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:47.401100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:47.401112Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:47.401131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:47.402705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:47.402723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:47.402751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:47.403100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 588 partition: 0 MaxSeqNo: 25 sessionId: src_id|a97179e6-ec00a4b-b0f7312-127617fe_0
2025-10-09T21:45:47.403461Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12f3e8ca-1905873c-99aa9051-5009bb3a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a97179e6-ec00a4b-b0f7312-127617fe_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:47.406112Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a97179e6-ec00a4b-b0f7312-127617fe_0] Write session: try to update token
2025-10-09T21:45:47.406145Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a97179e6-ec00a4b-b0f7312-127617fe_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:47.425254Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341501058790440:5852]
2025-10-09T21:45:47.425541Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 587, pipe: [1:7559341501058790440:5852]
2025-10-09T21:45:47.425611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341501058790440:5852], now have 1 active actors on pipe
2025-10-09T21:45:47.429910Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 588 sessionId: src_id|a97179e6-ec00a4b-b0f7312-127617fe_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:47.432541Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:47.432838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:47.432861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:47.432919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:47.433221Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:47.437445Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:47.437463Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:47.437466Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:47.437468Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:47.437477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341500696977009:5863]
2025-10-09T21:45:47.437481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341500696977009:5863]
2025-10-09T21:45:47.437485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341500696977009:5863]
2025-10-09T21:45:47.437531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:47.437541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.437544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:47.437679Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:47.438809Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:47.438823Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:47.438827Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:47.650591Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:47.668342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:47.668429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:47.668609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:47.668665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:47.668743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:47.668805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:47.669365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:47.669374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:47.669439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:47.669490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:47.669545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:47.669600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:47.669640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:47.669643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:47.669708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:47.669785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:47.669874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:47.669965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:47.670003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:47.670005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:47.670140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:47.670201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:47.670262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:47.670330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:47.670379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:47.670386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:47.670450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:47.670552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:47.670609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:47.670667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:47.670707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:47.670710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:47.670787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:47.670839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:47.670892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:47.670947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:47.670987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:47.670990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:47.671094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:47.671241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:47.671297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:47.671370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:47.671426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:47.671429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:47.671552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:47.671695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:47.671750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:47.671807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:47.671845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:47.671848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:47.671911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:47.671963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:47.672021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:47.672079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:47.672136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:47.672138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:47.672200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:47.672295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:47.672393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:47.672510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:47.672564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:47.672566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:47.674880Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:47.677115Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a97179e6-ec00a4b-b0f7312-127617fe_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:47.677127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a97179e6-ec00a4b-b0f7312-127617fe_0] Write session will restart in 0.005999s
2025-10-09T21:45:47.677153Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a97179e6-ec00a4b-b0f7312-127617fe_0] Write session: Do CDS request
2025-10-09T21:45:47.677160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a97179e6-ec00a4b-b0f7312-127617fe_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:47.675310Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:47.675345Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341501058790440:5852]
2025-10-09T21:45:47.675381Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:47.675395Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 588 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a97179e6-ec00a4b-b0f7312-127617fe_0
2025-10-09T21:45:47.675637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 588 sessionId: src_id|a97179e6-ec00a4b-b0f7312-127617fe_0 is DEAD
2025-10-09T21:45:47.685307Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a97179e6-ec00a4b-b0f7312-127617fe_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:47.685435Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:47.685454Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 589
2025-10-09T21:45:47.685721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 589 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:47.685762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 589 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:47.685768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=589 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:47.685772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:47.686410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 589 sessionId:  describe result for acl check
2025-10-09T21:45:47.686470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:47.686477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:47.686479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:47.686486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341501058790486:6441] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:47.687088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341501058790486:6441] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:47.691741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341501058790486:6441] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:47.702293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:47.703158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341501058790486:6441] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:47.703180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341501058790486:6441] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:47.703185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341501058790486:6441] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:47.703210Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 589 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:47.707540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:47.707808Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:47.707818Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:47.707842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:47.707915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:47.707966Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341500696977218:5879]
2025-10-09T21:45:47.708619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:47.709009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:47.709085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:47.709492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:47.709548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:47.709549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:47.709551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:47.709552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:47.709553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:47.709555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:47.709556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.709557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.709558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.709564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:47.709575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:47.709579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:47.709583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:47.709587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:47.709591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:47.709595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.709599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.709602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.709611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:47.709620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:47.709626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:47.709633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:47.709637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:47.709642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:47.709661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:47.709667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:47.709672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:47.709677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:47.709682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:47.709688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:47.709693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.709697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.710450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.710467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.710474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.710481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.710525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:47.718638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:47.720239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:47.720255Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:47.720257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:47.720262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:47.720269Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 588 [2:7559341500696977218:5879]
2025-10-09T21:45:47.720288Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:47.720307Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:47.720311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:47.720314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:47.720317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:47.720336Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:47.720342Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:47.720350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:47.720429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.720440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.720442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.720445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:47.720516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:47.720519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.720532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.720543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.720561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:47.720588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341500696977212:5879]
2025-10-09T21:45:47.720590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341500696977212:5879]
2025-10-09T21:45:47.720593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341500696977212:5879]
2025-10-09T21:45:47.720607Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:47.720627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341501058790629:6441], now have 1 active actors on pipe
2025-10-09T21:45:47.720630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341501058790472:2432], now have 1 active actors on pipe
2025-10-09T21:45:47.720639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:47.720653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:47.720657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:47.720700Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341500696977212:5879]
2025-10-09T21:45:47.720703Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:47.720707Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 588
2025-10-09T21:45:47.720713Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 588
2025-10-09T21:45:47.720974Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:47.720988Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:47.720990Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:47.720998Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:47.721075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:47.721081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:47.721112Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bca61523-7cfaaa3b-3ed5da67-d94450bc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:47.721122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:47.721127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:47.721129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:47.721141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:47.721155Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:47.721172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:47.721877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 589 partition: 0 MaxSeqNo: 25 sessionId: src_id|bca61523-7cfaaa3b-3ed5da67-d94450bc_0
2025-10-09T21:45:47.721656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:47.721667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:47.721686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:47.722377Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a97179e6-ec00a4b-b0f7312-127617fe_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bca61523-7cfaaa3b-3ed5da67-d94450bc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:47.724915Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bca61523-7cfaaa3b-3ed5da67-d94450bc_0] Write session: try to update token
2025-10-09T21:45:47.724933Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bca61523-7cfaaa3b-3ed5da67-d94450bc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:47.743780Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:47.743797Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:47.743807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341500696977212:5879]
2025-10-09T21:45:47.743869Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:47.743874Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:47.744100Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:47.744269Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:47.744423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:47.744568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:47.744825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:47.744879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:47.753986Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:47.754008Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:47.754010Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:47.754013Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:47.754023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341500696977212:5879]
2025-10-09T21:45:47.754027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341500696977212:5879]
2025-10-09T21:45:47.754072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341500696977212:5879]
2025-10-09T21:45:47.755673Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 589 sessionId: src_id|bca61523-7cfaaa3b-3ed5da67-d94450bc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:47.754118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:47.754147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.754146Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:47.754151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:47.754153Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:47.754158Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:47.754292Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:47.758267Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:47.775493Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341501058790641:5852]
2025-10-09T21:45:47.775867Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 588, pipe: [1:7559341501058790641:5852]
2025-10-09T21:45:47.925670Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:47.930781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:47.930805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:47.930838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341501058790641:5852], now have 1 active actors on pipe
2025-10-09T21:45:47.931136Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:47.931211Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341501058790641:5852]
2025-10-09T21:45:47.931249Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:47.931251Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:47.931316Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 589 reason: error at write: Disconnected sessionId: src_id|bca61523-7cfaaa3b-3ed5da67-d94450bc_0
2025-10-09T21:45:47.931500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 589 sessionId: src_id|bca61523-7cfaaa3b-3ed5da67-d94450bc_0 is DEAD
2025-10-09T21:45:47.931683Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:47.934289Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bca61523-7cfaaa3b-3ed5da67-d94450bc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:47.934302Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bca61523-7cfaaa3b-3ed5da67-d94450bc_0] Write session will restart in 0.007221s
2025-10-09T21:45:47.934330Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bca61523-7cfaaa3b-3ed5da67-d94450bc_0] Write session: Do CDS request
2025-10-09T21:45:47.934338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bca61523-7cfaaa3b-3ed5da67-d94450bc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:47.946585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bca61523-7cfaaa3b-3ed5da67-d94450bc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:47.946715Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:47.946736Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 590
2025-10-09T21:45:47.946966Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 590 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:47.947000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 590 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:47.947005Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=590 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:47.947009Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:47.947771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 590 sessionId:  describe result for acl check
2025-10-09T21:45:47.947814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:47.947816Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:47.947818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:47.947823Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341501058790702:6445] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:47.948470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341501058790702:6445] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:47.954817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341501058790702:6445] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:47.956633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:47.957889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:47.958146Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:47.958158Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:47.958182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:47.958246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:47.958302Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341500696977379:5881]
2025-10-09T21:45:47.958899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:47.961341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:47.961418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:47.961783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:47.961853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:47.961861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:47.961862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:47.961863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:47.961865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:47.961866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:47.961868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.961869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.961871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.961877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:47.961889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:47.961893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:47.961897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:47.961853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341501058790702:6445] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:47.961875Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341501058790702:6445] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:47.961879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341501058790702:6445] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:47.961901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:47.961890Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 590 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:47.961911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:47.961915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.961918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.961922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.961931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:47.961939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:47.961946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:47.961951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:47.961956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:47.961961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:47.961965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:47.961971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:47.961976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:47.961981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:47.961986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:47.961997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:47.962001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.962007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.962012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.962017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.962024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.962154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.962195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:47.969919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:47.971491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:47.971499Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:47.971502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:47.971506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:47.971514Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 589 [2:7559341500696977379:5881]
2025-10-09T21:45:47.971532Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:47.971552Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:47.971556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:47.971559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:47.971562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:47.971580Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:47.971587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:47.971595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:47.971668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.971670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.971672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.971674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:47.971755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:47.971758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.971771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:47.971781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:47.971795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:47.971813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341500696977373:5881]
2025-10-09T21:45:47.971815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341500696977373:5881]
2025-10-09T21:45:47.971817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341500696977373:5881]
2025-10-09T21:45:47.971827Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:47.971840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:47.971855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:47.971857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:47.971896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341500696977373:5881]
2025-10-09T21:45:47.971898Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:47.971939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341501058790644:2432], now have 1 active actors on pipe
2025-10-09T21:45:47.971943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341501058790806:6445], now have 1 active actors on pipe
2025-10-09T21:45:47.971954Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:47.971958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:47.971960Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:47.971968Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:47.972435Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 589
2025-10-09T21:45:47.972505Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 589
2025-10-09T21:45:47.973007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:47.973023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:47.973056Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|edcccfe6-61b38c77-3226d2d3-641f7750_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:47.973066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:47.973071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:47.973074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:47.973087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:47.973100Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:47.973118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:47.973782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:47.973793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:47.973818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:47.973974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 590 partition: 0 MaxSeqNo: 25 sessionId: src_id|edcccfe6-61b38c77-3226d2d3-641f7750_0
2025-10-09T21:45:47.974449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bca61523-7cfaaa3b-3ed5da67-d94450bc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|edcccfe6-61b38c77-3226d2d3-641f7750_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:47.976938Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|edcccfe6-61b38c77-3226d2d3-641f7750_0] Write session: try to update token
2025-10-09T21:45:47.976957Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|edcccfe6-61b38c77-3226d2d3-641f7750_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:47.988656Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:47.988678Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:47.988681Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:47.988683Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:47.988693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341500696977373:5881]
2025-10-09T21:45:47.988697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341500696977373:5881]
2025-10-09T21:45:47.988702Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341500696977373:5881]
2025-10-09T21:45:47.988758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:47.988767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:47.988770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:47.988771Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:47.988777Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:47.988782Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:47.988928Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:47.999053Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 590 sessionId: src_id|edcccfe6-61b38c77-3226d2d3-641f7750_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:48.001522Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:48.031521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341505353758115:5852]
2025-10-09T21:45:48.031910Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 589, pipe: [1:7559341505353758115:5852]
2025-10-09T21:45:48.155435Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:48.155932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:48.155949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:48.155997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341505353758115:5852], now have 1 active actors on pipe
2025-10-09T21:45:48.156321Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:48.156886Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341505353758115:5852]
2025-10-09T21:45:48.156888Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:48.156930Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:48.157005Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 590 reason: status is not ok: tablet will be restarted right now sessionId: src_id|edcccfe6-61b38c77-3226d2d3-641f7750_0
2025-10-09T21:45:48.157219Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 590 sessionId: src_id|edcccfe6-61b38c77-3226d2d3-641f7750_0 is DEAD
2025-10-09T21:45:48.157384Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edcccfe6-61b38c77-3226d2d3-641f7750_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:48.157395Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edcccfe6-61b38c77-3226d2d3-641f7750_0] Write session will restart in 0.009648s
2025-10-09T21:45:48.157436Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edcccfe6-61b38c77-3226d2d3-641f7750_0] Write session: Do CDS request
2025-10-09T21:45:48.157443Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edcccfe6-61b38c77-3226d2d3-641f7750_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:48.169069Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|edcccfe6-61b38c77-3226d2d3-641f7750_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:48.169156Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:48.169177Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 591
2025-10-09T21:45:48.169379Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 591 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:48.169427Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 591 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:48.169438Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=591 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:48.169457Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:48.169945Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 591 sessionId:  describe result for acl check
2025-10-09T21:45:48.169986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:48.169988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:48.169990Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:48.169997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341505353758138:6450] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:48.170624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341505353758138:6450] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:48.173681Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341505353758138:6450] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:48.178522Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341505353758138:6450] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:48.178544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341505353758138:6450] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:48.178548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341505353758138:6450] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:48.178558Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 591 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:48.187038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:48.192569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:48.192852Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:48.192857Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:48.192872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:48.192944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:48.192983Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341504991944838:5883]
2025-10-09T21:45:48.193588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:48.194083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:48.194175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:48.194625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:48.194699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:48.194700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:48.194701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:48.194703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:48.194704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:48.194706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:48.194707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.194708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.194709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.194716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:48.194727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:48.194732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:48.194736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:48.194740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:48.194744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:48.194748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.194752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.194757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.194766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:48.194789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:48.194796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:48.194802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:48.194807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:48.194812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:48.194816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:48.194821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:48.194826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:48.194831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:48.194835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:48.194840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:48.194845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.194850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.194853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.194858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.194862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.194883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.194912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:48.202704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:48.204283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:48.204301Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:48.204304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:48.204308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:48.204316Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 590 [2:7559341504991944838:5883]
2025-10-09T21:45:48.204332Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:48.204363Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:48.204369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:48.204373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:48.204375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:48.204395Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:48.204407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:48.204414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:48.204491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.204503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.204508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.204511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:48.204596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:48.204604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.204620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.204639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.204657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:48.204685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341504991944833:5883]
2025-10-09T21:45:48.204692Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341504991944833:5883]
2025-10-09T21:45:48.204695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341504991944833:5883]
2025-10-09T21:45:48.204706Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:48.204719Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:48.204722Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:48.204725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:48.204733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:48.204741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:48.204750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:48.204764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341505353758123:2432], now have 1 active actors on pipe
2025-10-09T21:45:48.204773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341505353758280:6450], now have 1 active actors on pipe
2025-10-09T21:45:48.204814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341504991944833:5883]
2025-10-09T21:45:48.204823Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:48.204813Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 590
2025-10-09T21:45:48.204839Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 590
2025-10-09T21:45:48.204853Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:48.205121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:48.205136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:48.205182Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1377e2a8-f3222aaf-6e8f64bd-5c200227_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:48.205197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:48.205201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:48.205204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:48.205217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:48.205230Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:48.205249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:48.205475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:48.205488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:48.205515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:48.205628Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 591 partition: 0 MaxSeqNo: 25 sessionId: src_id|1377e2a8-f3222aaf-6e8f64bd-5c200227_0
2025-10-09T21:45:48.206046Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edcccfe6-61b38c77-3226d2d3-641f7750_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1377e2a8-f3222aaf-6e8f64bd-5c200227_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:48.208555Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1377e2a8-f3222aaf-6e8f64bd-5c200227_0] Write session: try to update token
2025-10-09T21:45:48.208575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1377e2a8-f3222aaf-6e8f64bd-5c200227_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:48.226808Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:48.226829Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:48.226831Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:48.226833Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:48.226843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341504991944833:5883]
2025-10-09T21:45:48.226848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341504991944833:5883]
2025-10-09T21:45:48.226864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341504991944833:5883]
2025-10-09T21:45:48.226915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:48.226923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.226926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:48.226991Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:48.227002Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:48.227007Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:48.227072Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:48.234636Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 591 sessionId: src_id|1377e2a8-f3222aaf-6e8f64bd-5c200227_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:48.237256Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:48.257297Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341505353758293:5852]
2025-10-09T21:45:48.257679Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 590, pipe: [1:7559341505353758293:5852]
2025-10-09T21:45:48.403173Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:48.403688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:48.403710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:48.403770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341505353758293:5852], now have 1 active actors on pipe
2025-10-09T21:45:48.404220Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:48.404105Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341505353758293:5852]
2025-10-09T21:45:48.404141Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:48.404146Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:48.404149Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:48.404218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 591 reason: error at write: Disconnected sessionId: src_id|1377e2a8-f3222aaf-6e8f64bd-5c200227_0
2025-10-09T21:45:48.404412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 591 sessionId: src_id|1377e2a8-f3222aaf-6e8f64bd-5c200227_0 is DEAD
2025-10-09T21:45:48.406282Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1377e2a8-f3222aaf-6e8f64bd-5c200227_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:48.406294Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1377e2a8-f3222aaf-6e8f64bd-5c200227_0] Write session will restart in 0.007354s
2025-10-09T21:45:48.406338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1377e2a8-f3222aaf-6e8f64bd-5c200227_0] Write session: Do CDS request
2025-10-09T21:45:48.406346Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1377e2a8-f3222aaf-6e8f64bd-5c200227_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:48.418307Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1377e2a8-f3222aaf-6e8f64bd-5c200227_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:48.418980Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:48.419009Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 592
2025-10-09T21:45:48.419179Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 592 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:48.419210Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 592 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:48.419216Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=592 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:48.419221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:48.420087Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 592 sessionId:  describe result for acl check
2025-10-09T21:45:48.420124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:48.420126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:48.420128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:48.420135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341505353758363:6454] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:48.420738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341505353758363:6454] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:48.429374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341505353758363:6454] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:48.433981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341505353758363:6454] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:48.434001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341505353758363:6454] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:48.434006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341505353758363:6454] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:48.434016Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 592 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:48.435210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:48.435992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:48.436232Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:48.436237Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:48.436252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:48.436317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:48.436363Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341504991945006:5887]
2025-10-09T21:45:48.436989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:48.441121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:48.441224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:48.441618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:48.441710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:48.441712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:48.441713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:48.441714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:48.441715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:48.441716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:48.441717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.441718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.441720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.441727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:48.441738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:48.441742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:48.441745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:48.441748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:48.441752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:48.441755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.441759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.441763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.441772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:48.441781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:48.441787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:48.441792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:48.441796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:48.441802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:48.441807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:48.441812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:48.441817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:48.441822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:48.441826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:48.441830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:48.441838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.441842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.441847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.441852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.441857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.441862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.441892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:48.454547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:48.455659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:48.455672Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:48.455675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:48.455680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:48.455687Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 591 [2:7559341504991945006:5887]
2025-10-09T21:45:48.455702Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:48.455724Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:48.455727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:48.455731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:48.455733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:48.455752Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:48.455767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:48.455774Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:48.455838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.455846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.455848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.455851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:48.455934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:48.455944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.455958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.455968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.455984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:48.456004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341504991945000:5887]
2025-10-09T21:45:48.456012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341504991945000:5887]
2025-10-09T21:45:48.456014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341504991945000:5887]
2025-10-09T21:45:48.456029Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:48.456048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:48.456063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:48.456072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:48.456093Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:48.456105Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:48.456107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:48.456115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341504991945000:5887]
2025-10-09T21:45:48.456119Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:48.456142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:48.456166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341505353758296:2432], now have 1 active actors on pipe
2025-10-09T21:45:48.456176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341505353758456:6454], now have 1 active actors on pipe
2025-10-09T21:45:48.456302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 591
2025-10-09T21:45:48.456308Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 591
2025-10-09T21:45:48.456791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:48.456806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:48.456841Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|df801d5e-14b86e27-151b9214-51a823df_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:48.456855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:48.456860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:48.456863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:48.456874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:48.456904Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:48.456923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:48.457472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:48.457484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:48.457553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:48.457764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 592 partition: 0 MaxSeqNo: 25 sessionId: src_id|df801d5e-14b86e27-151b9214-51a823df_0
2025-10-09T21:45:48.462185Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1377e2a8-f3222aaf-6e8f64bd-5c200227_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|df801d5e-14b86e27-151b9214-51a823df_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:48.464648Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df801d5e-14b86e27-151b9214-51a823df_0] Write session: try to update token
2025-10-09T21:45:48.464685Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df801d5e-14b86e27-151b9214-51a823df_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:48.469863Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:48.469880Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:48.469889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341504991945000:5887]
2025-10-09T21:45:48.469950Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:48.470164Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:48.470310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:48.470456Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:48.470606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:48.470941Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:48.471260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:48.471308Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:48.487391Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 592 sessionId: src_id|df801d5e-14b86e27-151b9214-51a823df_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:48.489825Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:48.492079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:48.492109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:48.492133Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:48.492137Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:48.492139Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:48.492141Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:48.492153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341504991945000:5887]
2025-10-09T21:45:48.492156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341504991945000:5887]
2025-10-09T21:45:48.492159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341504991945000:5887]
2025-10-09T21:45:48.492217Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:48.492219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:48.492226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.492229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:48.492230Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:48.492236Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:48.492365Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:48.503584Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341505353758468:5852]
2025-10-09T21:45:48.503944Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 591, pipe: [1:7559341505353758468:5852]
2025-10-09T21:45:48.897024Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:48.897586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:48.897630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341505353758468:5852], now have 1 active actors on pipe
2025-10-09T21:45:48.897908Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:48.898018Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:48.898048Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:48.898161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 592 reason: error at write: Disconnected sessionId: src_id|df801d5e-14b86e27-151b9214-51a823df_0
2025-10-09T21:45:48.898288Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341505353758468:5852]
2025-10-09T21:45:48.898314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 592 sessionId: src_id|df801d5e-14b86e27-151b9214-51a823df_0 is DEAD
2025-10-09T21:45:48.898326Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:48.898498Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df801d5e-14b86e27-151b9214-51a823df_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:48.898510Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df801d5e-14b86e27-151b9214-51a823df_0] Write session will restart in 0.005889s
2025-10-09T21:45:48.898537Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df801d5e-14b86e27-151b9214-51a823df_0] Write session: Do CDS request
2025-10-09T21:45:48.898546Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df801d5e-14b86e27-151b9214-51a823df_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:48.905597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df801d5e-14b86e27-151b9214-51a823df_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:48.906071Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:48.906101Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 593
2025-10-09T21:45:48.906512Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 593 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:48.906567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 593 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:48.906573Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=593 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:48.906577Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:48.907092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 593 sessionId:  describe result for acl check
2025-10-09T21:45:48.907124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:48.907126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:48.907128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:48.907133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341505353758508:6466] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:48.907689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341505353758508:6466] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:48.914154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341505353758508:6466] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:48.923469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341505353758508:6466] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:48.923491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341505353758508:6466] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:48.923494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341505353758508:6466] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:48.923505Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 593 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:48.944324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:48.946764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:48.947534Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:48.947547Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:48.947564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:48.947658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:48.947713Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341504991945210:5901]
2025-10-09T21:45:48.948431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:48.953069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:48.953154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:48.953544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:48.953618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:48.953619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:48.953621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:48.953622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:48.953623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:48.953625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:48.953626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.953627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.953629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.953636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:48.953648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:48.953652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:48.953656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:48.953661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:48.953665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:48.953671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.953675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.953679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.953689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:48.953700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:48.953707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:48.953713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:48.953718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:48.953723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:48.953731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:48.953736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:48.953740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:48.953746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:48.953750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:48.953755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:48.953761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.953766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.953771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.953775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.953780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.953785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.953817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:48.969027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:48.970532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:48.970538Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:48.970540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:48.970545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:48.970553Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 592 [2:7559341504991945210:5901]
2025-10-09T21:45:48.970572Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:48.970594Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:48.970598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:48.970604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:48.970607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:48.970627Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:48.970633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:48.970640Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:48.970708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.970710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.970712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.970715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:48.970798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:48.970801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:48.970815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:48.970829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:48.970849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:48.970870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341504991945205:5901]
2025-10-09T21:45:48.970872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341504991945205:5901]
2025-10-09T21:45:48.970874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341504991945205:5901]
2025-10-09T21:45:48.970888Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:48.970903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:48.970920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:48.970923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:48.970966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341504991945205:5901]
2025-10-09T21:45:48.970969Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:48.972551Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 592
2025-10-09T21:45:48.972595Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 592
2025-10-09T21:45:48.971416Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:48.971426Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:48.971428Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:48.971434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:48.972682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341505353758500:2432], now have 1 active actors on pipe
2025-10-09T21:45:48.972689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341505353758612:6466], now have 1 active actors on pipe
2025-10-09T21:45:48.973101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:48.973133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:48.973168Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f0a042e6-3c7970e7-fd3a5d42-d65d9b23_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:48.973179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:48.973184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:48.973189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:48.973212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:48.973228Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:48.973246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:48.973735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:48.973744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:48.973761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:48.974415Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df801d5e-14b86e27-151b9214-51a823df_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f0a042e6-3c7970e7-fd3a5d42-d65d9b23_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:48.973916Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 593 partition: 0 MaxSeqNo: 25 sessionId: src_id|f0a042e6-3c7970e7-fd3a5d42-d65d9b23_0
2025-10-09T21:45:48.977069Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0a042e6-3c7970e7-fd3a5d42-d65d9b23_0] Write session: try to update token
2025-10-09T21:45:48.977093Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0a042e6-3c7970e7-fd3a5d42-d65d9b23_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:48.997724Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341505353758674:5852]
2025-10-09T21:45:48.998231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341505353758674:5852], now have 1 active actors on pipe
2025-10-09T21:45:48.998525Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 592, pipe: [1:7559341505353758674:5852]
2025-10-09T21:45:49.004597Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 593 sessionId: src_id|f0a042e6-3c7970e7-fd3a5d42-d65d9b23_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:49.007472Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:49.007368Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:49.007386Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:49.007388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:49.007390Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:49.007399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341504991945205:5901]
2025-10-09T21:45:49.007403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341504991945205:5901]
2025-10-09T21:45:49.007406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341504991945205:5901]
2025-10-09T21:45:49.007445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:49.007451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.007453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:49.007587Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:49.010606Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:49.010621Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:49.010625Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:49.188083Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:49.188388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:49.188399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:49.188909Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:49.188904Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341505353758674:5852]
2025-10-09T21:45:49.188927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:49.188934Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:49.189005Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 593 reason: error at write: Disconnected sessionId: src_id|f0a042e6-3c7970e7-fd3a5d42-d65d9b23_0
2025-10-09T21:45:49.189209Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 593 sessionId: src_id|f0a042e6-3c7970e7-fd3a5d42-d65d9b23_0 is DEAD
2025-10-09T21:45:49.189207Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:49.190423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0a042e6-3c7970e7-fd3a5d42-d65d9b23_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:49.190435Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0a042e6-3c7970e7-fd3a5d42-d65d9b23_0] Write session will restart in 0.005775s
2025-10-09T21:45:49.190475Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0a042e6-3c7970e7-fd3a5d42-d65d9b23_0] Write session: Do CDS request
2025-10-09T21:45:49.190486Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0a042e6-3c7970e7-fd3a5d42-d65d9b23_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:49.197594Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0a042e6-3c7970e7-fd3a5d42-d65d9b23_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:49.198195Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:49.198216Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 594
2025-10-09T21:45:49.198412Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 594 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:49.198451Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 594 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:49.198457Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=594 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:49.198462Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:49.199027Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 594 sessionId:  describe result for acl check
2025-10-09T21:45:49.199072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:49.199074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:49.199076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:49.199082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341509648725994:6471] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:49.199660Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341509648725994:6471] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:49.208891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341509648725994:6471] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:49.213264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341509648725994:6471] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:49.213283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341509648725994:6471] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:49.213287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341509648725994:6471] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:49.213297Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 594 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:49.227383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:49.236799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:49.237111Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:49.237122Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:49.237138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:49.237224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:49.237276Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341509286912671:5903]
2025-10-09T21:45:49.237893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:49.243210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:49.243292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:49.243695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:49.243767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:49.243773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:49.243774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:49.243775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:49.243777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:49.243778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:49.243780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.243781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.243783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.243789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:49.243800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:49.243805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:49.243809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:49.243814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:49.243818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:49.243822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.243827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.243831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.243840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:49.243849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:49.243857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:49.243863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:49.243869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:49.243875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:49.243880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:49.243885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:49.243891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:49.243896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:49.243902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:49.243908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:49.243914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.243919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.243930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.243935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.243939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.243946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.243977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:49.263125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:49.264702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:49.264717Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:49.264719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:49.264724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:49.264731Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 593 [2:7559341509286912671:5903]
2025-10-09T21:45:49.264746Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:49.264765Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:49.264768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:49.264771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:49.264773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:49.264791Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:49.264797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:49.264805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:49.264879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.264881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.264883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.264885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:49.264957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:49.264960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.264974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.264985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.264999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:49.265021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341509286912666:5903]
2025-10-09T21:45:49.265024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341509286912666:5903]
2025-10-09T21:45:49.265026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341509286912666:5903]
2025-10-09T21:45:49.265040Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:49.265056Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:49.265071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:49.265074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:49.265115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341509286912666:5903]
2025-10-09T21:45:49.265118Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:49.266579Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 593
2025-10-09T21:45:49.266621Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 593
2025-10-09T21:45:49.266308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:49.266317Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:49.266319Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:49.266324Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:49.266476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341509648726101:6471], now have 1 active actors on pipe
2025-10-09T21:45:49.266483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341509648725980:2432], now have 1 active actors on pipe
2025-10-09T21:45:49.266983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:49.266997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:49.267035Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f140a791-6db8e507-8bc9223c-d2f67d6d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:49.267047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:49.267051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:49.267054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:49.267079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:49.267096Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:49.267118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:49.269128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:49.269137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:49.269168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:49.269466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 594 partition: 0 MaxSeqNo: 25 sessionId: src_id|f140a791-6db8e507-8bc9223c-d2f67d6d_0
2025-10-09T21:45:49.269894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0a042e6-3c7970e7-fd3a5d42-d65d9b23_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f140a791-6db8e507-8bc9223c-d2f67d6d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:49.272593Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f140a791-6db8e507-8bc9223c-d2f67d6d_0] Write session: try to update token
2025-10-09T21:45:49.272617Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f140a791-6db8e507-8bc9223c-d2f67d6d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:49.286932Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341509648726152:5852]
2025-10-09T21:45:49.294689Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 593, pipe: [1:7559341509648726152:5852]
2025-10-09T21:45:49.294690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341509648726152:5852], now have 1 active actors on pipe
2025-10-09T21:45:49.296793Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:49.296809Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:49.296812Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:49.296814Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:49.296824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341509286912666:5903]
2025-10-09T21:45:49.296830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341509286912666:5903]
2025-10-09T21:45:49.296834Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341509286912666:5903]
2025-10-09T21:45:49.296881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:49.296889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.296891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:49.297034Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:49.302691Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:49.302722Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:49.302727Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:49.311873Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 594 sessionId: src_id|f140a791-6db8e507-8bc9223c-d2f67d6d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:49.314650Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:49.481019Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:49.481631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:49.481650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:49.481920Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:49.481954Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:49.481958Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:49.481977Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341509648726152:5852]
2025-10-09T21:45:49.482041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 594 reason: error at write: Disconnected sessionId: src_id|f140a791-6db8e507-8bc9223c-d2f67d6d_0
2025-10-09T21:45:49.481955Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:49.482289Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 594 sessionId: src_id|f140a791-6db8e507-8bc9223c-d2f67d6d_0 is DEAD
2025-10-09T21:45:49.483611Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f140a791-6db8e507-8bc9223c-d2f67d6d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:49.483623Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f140a791-6db8e507-8bc9223c-d2f67d6d_0] Write session will restart in 0.009333s
2025-10-09T21:45:49.483647Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f140a791-6db8e507-8bc9223c-d2f67d6d_0] Write session: Do CDS request
2025-10-09T21:45:49.483656Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f140a791-6db8e507-8bc9223c-d2f67d6d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:49.494580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f140a791-6db8e507-8bc9223c-d2f67d6d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:49.498551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:49.498572Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 595
2025-10-09T21:45:49.498837Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 595 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:49.498905Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 595 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:49.498912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=595 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:49.498916Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:49.499481Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 595 sessionId:  describe result for acl check
2025-10-09T21:45:49.499520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:49.499522Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:49.499523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:49.499530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341509648726222:6475] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:49.500118Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341509648726222:6475] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:49.504794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341509648726222:6475] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:49.509860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341509648726222:6475] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:49.509882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341509648726222:6475] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:49.509886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341509648726222:6475] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:49.509896Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 595 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:49.510261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:49.515425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:49.515704Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:49.515716Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:49.515729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:49.515789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:49.515834Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341509286912842:5907]
2025-10-09T21:45:49.516402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:49.516914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:49.516971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:49.517309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:49.517380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:49.517388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:49.517389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:49.517391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:49.517392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:49.517393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:49.517395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.517397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.517398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.517403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:49.517415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:49.517419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:49.517423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:49.517427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:49.517431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:49.517434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.517438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.517442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.517451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:49.517459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:49.517465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:49.517472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:49.517476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:49.517482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:49.517486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:49.517491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:49.517496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:49.517502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:49.517506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:49.517511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:49.517516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.517521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.517527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.517532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.517551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.517564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.517594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:49.525647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:49.527547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:49.527568Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:49.527571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:49.527575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:49.527584Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 594 [2:7559341509286912842:5907]
2025-10-09T21:45:49.527600Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:49.527623Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:49.527626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:49.527630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:49.527635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:49.527656Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:49.527669Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:49.527676Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:49.527747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.527755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.527758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.527761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:49.527836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:49.527843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.527858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.527868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.527881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:49.527905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341509648726157:2432], now have 1 active actors on pipe
2025-10-09T21:45:49.527915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341509648726320:6475], now have 1 active actors on pipe
2025-10-09T21:45:49.527923Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341509286912835:5907]
2025-10-09T21:45:49.527925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341509286912835:5907]
2025-10-09T21:45:49.527927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341509286912835:5907]
2025-10-09T21:45:49.527934Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:49.527946Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:49.527948Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:49.527950Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:49.527951Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:49.527965Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:49.527968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:49.528012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341509286912835:5907]
2025-10-09T21:45:49.528025Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:49.527993Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 594
2025-10-09T21:45:49.528037Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:49.528063Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 594
2025-10-09T21:45:49.528305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:49.528319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:49.528347Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7763f9b5-59485619-5d25fd7-aa3cb701_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:49.528358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:49.528362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:49.528365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:49.528376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:49.528394Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:49.528413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:49.528878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:49.528890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:49.528910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:49.529919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 595 partition: 0 MaxSeqNo: 25 sessionId: src_id|7763f9b5-59485619-5d25fd7-aa3cb701_0
2025-10-09T21:45:49.531181Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f140a791-6db8e507-8bc9223c-d2f67d6d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7763f9b5-59485619-5d25fd7-aa3cb701_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:49.533879Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7763f9b5-59485619-5d25fd7-aa3cb701_0] Write session: try to update token
2025-10-09T21:45:49.533900Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7763f9b5-59485619-5d25fd7-aa3cb701_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:49.544438Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:49.544455Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:49.544458Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:49.544460Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:49.544469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341509286912835:5907]
2025-10-09T21:45:49.544474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341509286912835:5907]
2025-10-09T21:45:49.544477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341509286912835:5907]
2025-10-09T21:45:49.544526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:49.544533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.544536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:49.544542Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:49.544548Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:49.544551Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:49.544670Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:49.553892Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 595 sessionId: src_id|7763f9b5-59485619-5d25fd7-aa3cb701_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:49.556631Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:49.578648Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341509648726333:5852]
2025-10-09T21:45:49.579119Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 594, pipe: [1:7559341509648726333:5852]
2025-10-09T21:45:49.701075Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:49.701441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:49.701460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:49.701485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341509648726333:5852], now have 1 active actors on pipe
2025-10-09T21:45:49.701852Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:49.701787Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:49.701830Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:49.701843Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:49.701851Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341509648726333:5852]
2025-10-09T21:45:49.701918Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 595 reason: error at write: Disconnected sessionId: src_id|7763f9b5-59485619-5d25fd7-aa3cb701_0
2025-10-09T21:45:49.702323Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7763f9b5-59485619-5d25fd7-aa3cb701_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:49.702343Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7763f9b5-59485619-5d25fd7-aa3cb701_0] Write session will restart in 0.005947s
2025-10-09T21:45:49.702367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7763f9b5-59485619-5d25fd7-aa3cb701_0] Write session: Do CDS request
2025-10-09T21:45:49.702377Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7763f9b5-59485619-5d25fd7-aa3cb701_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:49.702160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 595 sessionId: src_id|7763f9b5-59485619-5d25fd7-aa3cb701_0 is DEAD
2025-10-09T21:45:49.709644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7763f9b5-59485619-5d25fd7-aa3cb701_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:49.709752Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:49.709771Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 596
2025-10-09T21:45:49.710063Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 596 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:49.710127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 596 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:49.710138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=596 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:49.710143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:49.710654Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 596 sessionId:  describe result for acl check
2025-10-09T21:45:49.710694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:49.710702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:49.710704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:49.710710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341509648726375:6487] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:49.711264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341509648726375:6487] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:49.713595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341509648726375:6487] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:49.719751Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341509648726375:6487] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:49.719772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341509648726375:6487] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:49.719776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341509648726375:6487] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:49.720175Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 596 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:49.724458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:49.729882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:49.730199Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:49.730212Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:49.730227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:49.730296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:49.730343Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341509286913047:5921]
2025-10-09T21:45:49.730991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:49.731434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:49.731489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:49.731852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:49.731918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:49.731924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:49.731926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:49.731928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:49.731929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:49.731931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:49.731933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.731934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.731936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.731943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:49.731955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:49.731959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:49.731963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:49.731967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:49.731972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:49.731975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.731979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.731983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.731991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:49.731999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:49.732007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:49.732013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:49.732019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:49.732026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:49.732031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:49.732037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:49.732043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:49.732048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:49.732062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:49.732067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:49.732074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.732080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.732085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.732090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.732097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.732103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.732149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:49.740333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:49.741484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:49.741504Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:49.741507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:49.741511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:49.741519Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 595 [2:7559341509286913047:5921]
2025-10-09T21:45:49.741538Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:49.741561Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:49.741565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:49.741568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:49.741571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:49.741591Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:49.741598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:49.741606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:49.741685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.741697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.741700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.741703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:49.741782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:49.741789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.741802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:49.741818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:49.741834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:49.741866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341509648726362:2432], now have 1 active actors on pipe
2025-10-09T21:45:49.741870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341509648726505:6487], now have 1 active actors on pipe
2025-10-09T21:45:49.741879Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341509286913042:5921]
2025-10-09T21:45:49.741882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341509286913042:5921]
2025-10-09T21:45:49.741884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341509286913042:5921]
2025-10-09T21:45:49.741898Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:49.741919Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:49.741935Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:49.741938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:49.741941Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:49.741944Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:49.741958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:49.741980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341509286913042:5921]
2025-10-09T21:45:49.741984Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:49.741986Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:49.743760Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 595
2025-10-09T21:45:49.743889Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 595
2025-10-09T21:45:49.743982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:49.743998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:49.744046Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|effb4a47-83ccfb64-931dc0a4-679ce042_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:49.744063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:49.744072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:49.744076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:49.744091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:49.744109Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:49.744133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:49.744581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:49.744607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:49.744627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:49.744816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 596 partition: 0 MaxSeqNo: 25 sessionId: src_id|effb4a47-83ccfb64-931dc0a4-679ce042_0
2025-10-09T21:45:49.745264Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7763f9b5-59485619-5d25fd7-aa3cb701_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|effb4a47-83ccfb64-931dc0a4-679ce042_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:49.748081Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|effb4a47-83ccfb64-931dc0a4-679ce042_0] Write session: try to update token
2025-10-09T21:45:49.748103Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|effb4a47-83ccfb64-931dc0a4-679ce042_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:49.756950Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:49.756970Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:49.756982Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341509286913042:5921]
2025-10-09T21:45:49.757032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:49.757220Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:49.757369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:49.757518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:49.757690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:49.757883Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:49.757925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:49.758135Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:49.768049Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 596 sessionId: src_id|effb4a47-83ccfb64-931dc0a4-679ce042_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:49.770693Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:49.771513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:49.771540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:49.771575Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:49.771585Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:49.771587Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:49.771589Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:49.771599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341509286913042:5921]
2025-10-09T21:45:49.772084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341509286913042:5921]
2025-10-09T21:45:49.772096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341509286913042:5921]
2025-10-09T21:45:49.772161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:49.772172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:49.772174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:49.772227Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:49.772241Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:49.772245Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:49.772316Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:49.801977Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341509648726535:5852]
2025-10-09T21:45:49.802524Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 595, pipe: [1:7559341509648726535:5852]
2025-10-09T21:45:49.976652Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:49.977210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:49.977257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341509648726535:5852], now have 1 active actors on pipe
2025-10-09T21:45:49.977821Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:49.977887Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341509648726535:5852]
2025-10-09T21:45:49.977913Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:49.977916Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:49.977928Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:49.978013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 596 reason: error at write: Disconnected sessionId: src_id|effb4a47-83ccfb64-931dc0a4-679ce042_0
2025-10-09T21:45:49.978195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 596 sessionId: src_id|effb4a47-83ccfb64-931dc0a4-679ce042_0 is DEAD
2025-10-09T21:45:49.978514Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|effb4a47-83ccfb64-931dc0a4-679ce042_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:49.978525Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|effb4a47-83ccfb64-931dc0a4-679ce042_0] Write session will restart in 0.005795s
2025-10-09T21:45:49.978547Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|effb4a47-83ccfb64-931dc0a4-679ce042_0] Write session: Do CDS request
2025-10-09T21:45:49.978556Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|effb4a47-83ccfb64-931dc0a4-679ce042_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:49.985606Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|effb4a47-83ccfb64-931dc0a4-679ce042_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:49.986083Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:49.986103Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 597
2025-10-09T21:45:49.986290Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 597 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:49.986338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 597 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:49.986344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=597 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:49.986348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:49.986876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 597 sessionId:  describe result for acl check
2025-10-09T21:45:49.986913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:49.986920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:49.986922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:49.986928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341509648726554:6491] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:49.987459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341509648726554:6491] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:49.991535Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341509648726554:6491] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:49.995287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341509648726554:6491] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:49.995307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341509648726554:6491] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:49.995311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341509648726554:6491] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:49.995321Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 597 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:50.017596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:50.024345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:50.024667Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:50.024681Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:50.024707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:50.024773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:50.024826Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341513581880512:5923]
2025-10-09T21:45:50.025847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:50.026372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:50.026448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:50.026839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:50.026912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:50.026914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:50.026915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:50.026916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:50.026918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:50.026919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:50.026920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.026921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.026923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.026930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:50.026941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:50.026946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:50.026949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:50.026953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:50.026957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:50.026961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.026966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.026969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.026978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:50.026986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:50.026993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:50.027117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:50.027124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:50.027130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:50.027137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:50.027143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:50.027148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:50.027154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:50.027187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:50.027199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:50.027205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.027210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.027214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.027219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.027223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.027228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.027259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:50.039365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:50.041431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:50.041452Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:50.041455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:50.041460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:50.041469Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 596 [2:7559341513581880512:5923]
2025-10-09T21:45:50.041488Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:50.041512Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:50.041516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:50.041521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:50.041523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:50.041546Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:50.041553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:50.041561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:50.041641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.041643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.041645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.041648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:50.041759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:50.041771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.041786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.041796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.041813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:50.041837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341513581880506:5923]
2025-10-09T21:45:50.041846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341513581880506:5923]
2025-10-09T21:45:50.041849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341513581880506:5923]
2025-10-09T21:45:50.041866Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:50.041883Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:50.041898Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:50.041902Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:50.041927Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:50.041936Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:50.041938Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:50.041954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341513581880506:5923]
2025-10-09T21:45:50.041965Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:50.042417Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 596
2025-10-09T21:45:50.042466Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 596
2025-10-09T21:45:50.042064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:50.042639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341509648726586:6491], now have 1 active actors on pipe
2025-10-09T21:45:50.042653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341509648726541:2432], now have 1 active actors on pipe
2025-10-09T21:45:50.042680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:50.042686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:50.042720Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9eb40f88-1748ff2d-b1506d76-bb0f6a91_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:50.042736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:50.042745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:50.042748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:50.042761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:50.042778Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:50.042800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:50.043024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:50.043034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:50.043051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:50.043480Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 597 partition: 0 MaxSeqNo: 25 sessionId: src_id|9eb40f88-1748ff2d-b1506d76-bb0f6a91_0
2025-10-09T21:45:50.044045Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|effb4a47-83ccfb64-931dc0a4-679ce042_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9eb40f88-1748ff2d-b1506d76-bb0f6a91_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:50.048473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9eb40f88-1748ff2d-b1506d76-bb0f6a91_0] Write session: try to update token
2025-10-09T21:45:50.048500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9eb40f88-1748ff2d-b1506d76-bb0f6a91_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:50.068510Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:50.068534Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:50.068537Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:50.068539Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:50.068551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341513581880506:5923]
2025-10-09T21:45:50.068556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341513581880506:5923]
2025-10-09T21:45:50.068562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341513581880506:5923]
2025-10-09T21:45:50.068621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:50.068630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.068633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:50.068901Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:50.069216Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:50.069223Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:50.069231Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:50.080111Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341513943694018:5852]
2025-10-09T21:45:50.080584Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 596, pipe: [1:7559341513943694018:5852]
2025-10-09T21:45:50.088700Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 597 sessionId: src_id|9eb40f88-1748ff2d-b1506d76-bb0f6a91_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:50.091173Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:50.247659Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:50.247959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341513943694018:5852], now have 1 active actors on pipe
2025-10-09T21:45:50.247994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:50.248001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:50.248570Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341513943694018:5852]
2025-10-09T21:45:50.248625Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:50.248628Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:50.248692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 597 reason: error at write: Disconnected sessionId: src_id|9eb40f88-1748ff2d-b1506d76-bb0f6a91_0
2025-10-09T21:45:50.248865Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:50.248891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 597 sessionId: src_id|9eb40f88-1748ff2d-b1506d76-bb0f6a91_0 is DEAD
2025-10-09T21:45:50.249220Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9eb40f88-1748ff2d-b1506d76-bb0f6a91_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:50.249233Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9eb40f88-1748ff2d-b1506d76-bb0f6a91_0] Write session will restart in 0.007785s
2025-10-09T21:45:50.249258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9eb40f88-1748ff2d-b1506d76-bb0f6a91_0] Write session: Do CDS request
2025-10-09T21:45:50.249269Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9eb40f88-1748ff2d-b1506d76-bb0f6a91_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:50.249413Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:50.258557Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9eb40f88-1748ff2d-b1506d76-bb0f6a91_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:50.259005Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:50.259026Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 598
2025-10-09T21:45:50.259223Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 598 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:50.259260Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 598 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:50.259271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=598 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:50.259274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:50.259805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 598 sessionId:  describe result for acl check
2025-10-09T21:45:50.259856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:50.259864Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:50.259866Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:50.259872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341513943694033:6496] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:50.260448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341513943694033:6496] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:50.263298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341513943694033:6496] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:50.266922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341513943694033:6496] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:50.266951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341513943694033:6496] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:50.266955Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341513943694033:6496] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:50.266970Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 598 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:50.283517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:50.289413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:50.289702Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:50.289715Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:50.289729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:50.289789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:50.289840Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341513581880682:5925]
2025-10-09T21:45:50.290531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:50.291675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:50.291751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:50.292166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:50.292239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:50.292248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:50.292249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:50.292250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:50.292251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:50.292252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:50.292253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.292255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.292256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.292262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:50.292273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:50.292277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:50.292280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:50.292284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:50.292287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:50.292290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.292293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.292297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.292305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:50.292312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:50.292318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:50.292323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:50.292332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:50.292336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:50.292340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:50.292344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:50.292349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:50.292353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:50.292356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:50.292360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:50.292364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.292368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.292372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.292376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.292380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.292384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.292415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:50.305361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:50.307132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:50.307141Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:50.307144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:50.307149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:50.307158Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 597 [2:7559341513581880682:5925]
2025-10-09T21:45:50.307177Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:50.307201Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:50.307205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:50.307208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:50.307211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:50.307231Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:50.307238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:50.307247Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:50.307323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.307328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.307330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.307333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:50.307422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:50.307425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.307442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.307455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.307474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:50.307497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341513581880676:5925]
2025-10-09T21:45:50.307500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341513581880676:5925]
2025-10-09T21:45:50.307502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341513581880676:5925]
2025-10-09T21:45:50.307559Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:50.307561Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:50.307564Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:50.308355Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 597
2025-10-09T21:45:50.308357Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 597
2025-10-09T21:45:50.308130Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:50.308165Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:50.308183Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:50.308653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:50.308677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341513943694093:6496], now have 1 active actors on pipe
2025-10-09T21:45:50.308681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341513943694021:2432], now have 1 active actors on pipe
2025-10-09T21:45:50.308761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:50.308768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:50.308785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341513581880676:5925]
2025-10-09T21:45:50.308799Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:50.308811Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:50.308854Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7c533148-3b156fc5-c09f75b8-41d3044d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:50.308866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:50.308871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:50.308875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:50.308889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:50.308904Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:50.308934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:50.313278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:50.313303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:50.313347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:50.313590Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 598 partition: 0 MaxSeqNo: 25 sessionId: src_id|7c533148-3b156fc5-c09f75b8-41d3044d_0
2025-10-09T21:45:50.314077Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9eb40f88-1748ff2d-b1506d76-bb0f6a91_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7c533148-3b156fc5-c09f75b8-41d3044d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:50.317050Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c533148-3b156fc5-c09f75b8-41d3044d_0] Write session: try to update token
2025-10-09T21:45:50.317073Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c533148-3b156fc5-c09f75b8-41d3044d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:50.328051Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:50.328071Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:50.328074Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:50.328077Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:50.328087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341513581880676:5925]
2025-10-09T21:45:50.328091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341513581880676:5925]
2025-10-09T21:45:50.328095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341513581880676:5925]
2025-10-09T21:45:50.328144Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:50.328146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:50.328151Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:50.328152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.328155Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:50.328155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:50.328293Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:50.348910Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341513943694199:5852]
2025-10-09T21:45:50.350725Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 597, pipe: [1:7559341513943694199:5852]
2025-10-09T21:45:50.350748Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 598 sessionId: src_id|7c533148-3b156fc5-c09f75b8-41d3044d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:50.353340Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:50.525012Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:50.525029Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:50.525107Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 598 reason: error at write: Disconnected sessionId: src_id|7c533148-3b156fc5-c09f75b8-41d3044d_0
2025-10-09T21:45:50.525578Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 598 sessionId: src_id|7c533148-3b156fc5-c09f75b8-41d3044d_0 is DEAD
2025-10-09T21:45:50.525776Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341513943694199:5852]
2025-10-09T21:45:50.525797Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:50.523664Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:50.524424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341513943694199:5852], now have 1 active actors on pipe
2025-10-09T21:45:50.524452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:50.524459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:50.525446Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:50.526157Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c533148-3b156fc5-c09f75b8-41d3044d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:50.526170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c533148-3b156fc5-c09f75b8-41d3044d_0] Write session will restart in 0.008923s
2025-10-09T21:45:50.526198Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c533148-3b156fc5-c09f75b8-41d3044d_0] Write session: Do CDS request
2025-10-09T21:45:50.526205Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c533148-3b156fc5-c09f75b8-41d3044d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:50.538706Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c533148-3b156fc5-c09f75b8-41d3044d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:50.539217Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:50.539239Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 599
2025-10-09T21:45:50.539437Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 599 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:50.539476Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 599 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:50.539488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=599 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:50.539493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:50.539981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 599 sessionId:  describe result for acl check
2025-10-09T21:45:50.540024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:50.540027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:50.540029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:50.540036Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341513943694213:6500] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:50.540678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341513943694213:6500] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:50.549702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341513943694213:6500] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:50.557568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341513943694213:6500] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:50.557589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341513943694213:6500] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:50.557593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341513943694213:6500] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:50.557603Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 599 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:50.579184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:50.579389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:50.579680Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:50.579685Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:50.579703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:50.579775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:50.579816Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341513581880856:5929]
2025-10-09T21:45:50.580391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:50.581070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:50.581129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:50.581440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:50.581503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:50.581504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:50.581505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:50.581507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:50.581508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:50.581509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:50.581510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.581511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.581513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.581519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:50.581529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:50.581532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:50.581536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:50.581539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:50.581543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:50.581547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.581550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.581553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.581561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:50.581569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:50.581575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:50.581581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:50.581585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:50.581589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:50.581593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:50.581598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:50.581602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:50.581606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:50.581610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:50.581615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:50.581619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.581623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.581627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.581631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.581636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.581641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.581687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:50.593677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:50.595488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:50.595495Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:50.595498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:50.595503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:50.595514Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 598 [2:7559341513581880856:5929]
2025-10-09T21:45:50.595529Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:50.595550Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:50.595554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:50.595557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:50.595559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:50.595579Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:50.595585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:50.595592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:50.595658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.595660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.595663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.595665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:50.595745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:50.595748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.595761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.595771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.595786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:50.595807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341513581880851:5929]
2025-10-09T21:45:50.595809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341513581880851:5929]
2025-10-09T21:45:50.595811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341513581880851:5929]
2025-10-09T21:45:50.595827Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:50.595842Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:50.595856Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:50.595859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:50.595902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341513581880851:5929]
2025-10-09T21:45:50.595905Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:50.596183Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:50.596191Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:50.596196Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:50.596202Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:50.596966Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 598
2025-10-09T21:45:50.597015Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 598
2025-10-09T21:45:50.598709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 599 partition: 0 MaxSeqNo: 25 sessionId: src_id|985dbef2-8c2fe76-24dfa786-c1ad629e_0
2025-10-09T21:45:50.599143Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c533148-3b156fc5-c09f75b8-41d3044d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|985dbef2-8c2fe76-24dfa786-c1ad629e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:50.598139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341513943694204:2432], now have 1 active actors on pipe
2025-10-09T21:45:50.598155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341513943694265:6500], now have 1 active actors on pipe
2025-10-09T21:45:50.598176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:50.598186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:50.598225Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|985dbef2-8c2fe76-24dfa786-c1ad629e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:50.598265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:50.598270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:50.598272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:50.598285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:50.598301Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:50.598321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:50.598560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:50.598565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:50.598593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:50.601811Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|985dbef2-8c2fe76-24dfa786-c1ad629e_0] Write session: try to update token
2025-10-09T21:45:50.601832Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|985dbef2-8c2fe76-24dfa786-c1ad629e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:50.613553Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:50.613572Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:50.613575Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:50.613577Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:50.613587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341513581880851:5929]
2025-10-09T21:45:50.613592Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341513581880851:5929]
2025-10-09T21:45:50.613596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341513581880851:5929]
2025-10-09T21:45:50.613644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:50.613653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.613655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:50.613799Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:50.620225Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:50.620241Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:50.620246Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:50.625499Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 599 sessionId: src_id|985dbef2-8c2fe76-24dfa786-c1ad629e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:50.625758Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341513943694385:5852]
2025-10-09T21:45:50.628076Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:50.628175Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 598, pipe: [1:7559341513943694385:5852]
2025-10-09T21:45:50.795943Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:50.796186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341513943694385:5852], now have 1 active actors on pipe
2025-10-09T21:45:50.796220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:50.796228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:50.796697Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:50.796765Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341513943694385:5852]
2025-10-09T21:45:50.796787Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:50.796795Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:50.796847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 599 reason: error at write: Disconnected sessionId: src_id|985dbef2-8c2fe76-24dfa786-c1ad629e_0
2025-10-09T21:45:50.797033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 599 sessionId: src_id|985dbef2-8c2fe76-24dfa786-c1ad629e_0 is DEAD
2025-10-09T21:45:50.797311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|985dbef2-8c2fe76-24dfa786-c1ad629e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:50.797321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|985dbef2-8c2fe76-24dfa786-c1ad629e_0] Write session will restart in 0.006258s
2025-10-09T21:45:50.797344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|985dbef2-8c2fe76-24dfa786-c1ad629e_0] Write session: Do CDS request
2025-10-09T21:45:50.797353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|985dbef2-8c2fe76-24dfa786-c1ad629e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:50.796676Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:50.804844Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|985dbef2-8c2fe76-24dfa786-c1ad629e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:50.804832Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:50.804857Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 600
2025-10-09T21:45:50.805443Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 600 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:50.805499Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 600 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:50.805506Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=600 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:50.805510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:50.805938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 600 sessionId:  describe result for acl check
2025-10-09T21:45:50.805971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:50.805973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:50.805975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:50.805980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341513943694424:6512] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:50.806630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341513943694424:6512] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:50.809372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341513943694424:6512] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:50.813564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341513943694424:6512] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:50.813585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341513943694424:6512] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:50.813590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341513943694424:6512] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:50.813600Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 600 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:50.826487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:50.835165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:50.835450Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:50.835454Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:50.835467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:50.835528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:50.835572Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341513581881066:5943]
2025-10-09T21:45:50.836250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:50.836811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:50.836888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:50.837230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:50.837286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:50.837288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:50.837289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:50.837290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:50.837292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:50.837293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:50.837295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.837296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.837298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.837304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:50.837315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:50.837320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:50.837324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:50.837345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:50.837349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:50.837352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.837356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.837360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.837368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:50.837376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:50.837382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:50.837387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:50.837390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:50.837395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:50.837399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:50.837404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:50.837408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:50.837412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:50.837416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:50.837422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:50.837426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.837431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.837438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.837442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.837447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.837455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.837484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:50.852020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:50.853397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:50.853403Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:50.853406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:50.853412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:50.853420Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 599 [2:7559341513581881066:5943]
2025-10-09T21:45:50.853436Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:50.853458Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:50.853462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:50.853466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:50.853469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:50.853488Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:50.853494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:50.853503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:50.853569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.853572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.853574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.853577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:50.853656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:50.853660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.853675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:50.853689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:50.853708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:50.853732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341513581881061:5943]
2025-10-09T21:45:50.853735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341513581881061:5943]
2025-10-09T21:45:50.853737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341513581881061:5943]
2025-10-09T21:45:50.853751Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:50.853766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:50.853782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:50.853785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:50.853824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341513581881061:5943]
2025-10-09T21:45:50.853827Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:50.854195Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:50.854202Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:50.854204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:50.854210Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:50.854759Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 599
2025-10-09T21:45:50.854805Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 599
2025-10-09T21:45:50.858278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341513943694415:2432], now have 1 active actors on pipe
2025-10-09T21:45:50.858295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341513943694472:6512], now have 1 active actors on pipe
2025-10-09T21:45:50.858322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:50.858328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:50.858368Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|97fd7873-199e0d53-b1fb9410-6bd0e37a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:50.858380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:50.858386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:50.858390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:50.858407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:50.858427Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:50.858448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:50.862093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:50.862130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:50.862172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:50.862897Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|985dbef2-8c2fe76-24dfa786-c1ad629e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|97fd7873-199e0d53-b1fb9410-6bd0e37a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:50.862434Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 600 partition: 0 MaxSeqNo: 25 sessionId: src_id|97fd7873-199e0d53-b1fb9410-6bd0e37a_0
2025-10-09T21:45:50.865481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97fd7873-199e0d53-b1fb9410-6bd0e37a_0] Write session: try to update token
2025-10-09T21:45:50.865502Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97fd7873-199e0d53-b1fb9410-6bd0e37a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:50.878301Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:45:50.878345Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 6
2025-10-09T21:45:50.879872Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 6 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:45:50.879927Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:45:50.879409Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:50.879427Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:50.879430Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:50.879432Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:50.879444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341513581881061:5943]
2025-10-09T21:45:50.879452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341513581881061:5943]
2025-10-09T21:45:50.879456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341513581881061:5943]
2025-10-09T21:45:50.879525Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:50.879533Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:50.879537Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:50.879578Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:45:50.879603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:50.879610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:50.879612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:50.879742Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:50.894225Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 600 sessionId: src_id|97fd7873-199e0d53-b1fb9410-6bd0e37a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:50.896830Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:50.897189Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341513943694598:5852]
2025-10-09T21:45:50.897625Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 599, pipe: [1:7559341513943694598:5852]
2025-10-09T21:45:51.137384Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341513943694598:5852]
2025-10-09T21:45:51.138130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97fd7873-199e0d53-b1fb9410-6bd0e37a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:51.138143Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97fd7873-199e0d53-b1fb9410-6bd0e37a_0] Write session will restart in 0.007876s
2025-10-09T21:45:51.137424Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:51.137436Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 0
2025-10-09T21:45:51.137467Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:51.137470Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:51.137529Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 600 reason: error at write: Disconnected sessionId: src_id|97fd7873-199e0d53-b1fb9410-6bd0e37a_0
2025-10-09T21:45:51.137727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 600 sessionId: src_id|97fd7873-199e0d53-b1fb9410-6bd0e37a_0 is DEAD
2025-10-09T21:45:51.136252Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:51.136548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:51.136559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:51.136585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341513943694598:5852], now have 1 active actors on pipe
2025-10-09T21:45:51.138190Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97fd7873-199e0d53-b1fb9410-6bd0e37a_0] Write session: Do CDS request
2025-10-09T21:45:51.138233Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97fd7873-199e0d53-b1fb9410-6bd0e37a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:51.142263Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:51.147838Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97fd7873-199e0d53-b1fb9410-6bd0e37a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:51.148868Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:51.148893Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 601
2025-10-09T21:45:51.149043Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 601 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:51.149085Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 601 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:51.149097Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=601 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:51.149101Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:51.150810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 601 sessionId:  describe result for acl check
2025-10-09T21:45:51.150848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:51.150850Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:51.150855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:51.150863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341518238661920:6517] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:51.151487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341518238661920:6517] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:51.159093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341518238661920:6517] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:51.167363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341518238661920:6517] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:51.167386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341518238661920:6517] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:51.167390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341518238661920:6517] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:51.167401Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 601 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:51.196101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:51.201928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:51.202224Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:51.202238Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:51.202254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:51.202332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:51.202378Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341517876848536:5946]
2025-10-09T21:45:51.202960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:51.203467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:51.203526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:51.203837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:51.203901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:51.203902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:51.203903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:51.203905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:51.203906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:51.203907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:51.203908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.203909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.203910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.203916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:51.203927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:51.203931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:51.203934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:51.203938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:51.203942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:51.203945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.203949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.203952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.203961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:51.203970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:51.203977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:51.203983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:51.203988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:51.203994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:51.203998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:51.204004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:51.204011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:51.204016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:51.204020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:51.204026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:51.204031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.204036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.204041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.204046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.204051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.204055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.204082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:51.211818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:51.213325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:51.213348Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:51.213350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:51.213355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:51.213363Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 600 [2:7559341517876848536:5946]
2025-10-09T21:45:51.213380Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:51.213401Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:51.213404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:51.213407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:51.213410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:51.213429Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:51.213437Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:51.213445Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:51.213515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.213525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.213527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.213530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:51.213606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:51.213613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.213637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.213648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.213666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:51.213687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341517876848530:5946]
2025-10-09T21:45:51.213690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341517876848530:5946]
2025-10-09T21:45:51.213692Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341517876848530:5946]
2025-10-09T21:45:51.213708Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:51.213723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:51.213737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:51.213741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:51.213783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341517876848530:5946]
2025-10-09T21:45:51.213787Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:51.213796Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:51.213799Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:51.213804Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:51.213811Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:51.214276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341518238661958:6517], now have 1 active actors on pipe
2025-10-09T21:45:51.214280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341518238661912:2432], now have 1 active actors on pipe
2025-10-09T21:45:51.214537Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 600
2025-10-09T21:45:51.214594Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 600
2025-10-09T21:45:51.214654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:51.214671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:51.214739Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b837e31b-7f983a53-5d2c9dc6-2daff95a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:51.214764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:51.214768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:51.214771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:51.214784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:51.214799Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:51.214867Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:45:51.214905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:51.215126Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 7 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:45:51.215158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:51.215171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:51.215163Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:45:51.215209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:51.215328Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 601 partition: 0 MaxSeqNo: 25 sessionId: src_id|b837e31b-7f983a53-5d2c9dc6-2daff95a_0
2025-10-09T21:45:51.217497Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97fd7873-199e0d53-b1fb9410-6bd0e37a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b837e31b-7f983a53-5d2c9dc6-2daff95a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:51.220092Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b837e31b-7f983a53-5d2c9dc6-2daff95a_0] Write session: try to update token
2025-10-09T21:45:51.220114Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b837e31b-7f983a53-5d2c9dc6-2daff95a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:51.236030Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:51.236050Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:51.236052Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:51.236055Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:51.236065Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341517876848530:5946]
2025-10-09T21:45:51.236071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341517876848530:5946]
2025-10-09T21:45:51.236075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341517876848530:5946]
2025-10-09T21:45:51.236138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:51.236145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.236148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:51.236375Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:51.237136Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341518238662097:5852]
2025-10-09T21:45:51.237579Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:51.237600Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:51.237604Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:51.238177Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 600, pipe: [1:7559341518238662097:5852]
2025-10-09T21:45:51.243933Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 601 sessionId: src_id|b837e31b-7f983a53-5d2c9dc6-2daff95a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:51.247029Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:51.424872Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:51.424877Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341518238662097:5852]
2025-10-09T21:45:51.424890Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:51.424897Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:51.424938Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:45:51.424973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 601 reason: error at write: Disconnected sessionId: src_id|b837e31b-7f983a53-5d2c9dc6-2daff95a_0
2025-10-09T21:45:51.425051Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 6 consumer shared/user session shared/user_1_6_6774085051053870161_v1 is DEAD
2025-10-09T21:45:51.425075Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 601 sessionId: src_id|b837e31b-7f983a53-5d2c9dc6-2daff95a_0 is DEAD
2025-10-09T21:45:51.425195Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341385094661378:5847] disconnected.
2025-10-09T21:45:51.425217Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341385094661378:5847] disconnected; active server actors: 1
2025-10-09T21:45:51.425227Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341385094661378:5847] client user disconnected session shared/user_1_6_6774085051053870161_v1
2025-10-09T21:45:51.424176Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:51.424441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341518238662097:5852], now have 1 active actors on pipe
2025-10-09T21:45:51.424474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:51.424480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:51.425123Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:51.426499Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:45:51.426655Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:45:51.426662Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:45:51.426668Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.009632s
2025-10-09T21:45:51.426499Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b837e31b-7f983a53-5d2c9dc6-2daff95a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:51.426795Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b837e31b-7f983a53-5d2c9dc6-2daff95a_0] Write session will restart in 0.009604s
2025-10-09T21:45:51.426820Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b837e31b-7f983a53-5d2c9dc6-2daff95a_0] Write session: Do CDS request
2025-10-09T21:45:51.426827Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b837e31b-7f983a53-5d2c9dc6-2daff95a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:51.437764Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b837e31b-7f983a53-5d2c9dc6-2daff95a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:51.437904Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:45:51.438373Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:51.438393Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 602
2025-10-09T21:45:51.438424Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:45:51.438427Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 7
2025-10-09T21:45:51.438631Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 602 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:51.438668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 602 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:51.438679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=602 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:51.438683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:51.438723Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 7 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:45:51.438772Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:45:51.438925Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 auth for : user
2025-10-09T21:45:51.439695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 602 sessionId:  describe result for acl check
2025-10-09T21:45:51.439737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:51.439743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:51.439745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:51.439750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341518238662121:6521] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:51.440372Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 Handle describe topics response
2025-10-09T21:45:51.440402Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 auth is DEAD
2025-10-09T21:45:51.440420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341518238662121:6521] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:51.440454Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:45:51.440712Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:45:51.440881Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_7_8824138098741025035_v1
2025-10-09T21:45:51.441342Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341518238662124:6522] connected; active server actors: 1
2025-10-09T21:45:51.441456Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559341518238662124:6522] session shared/user_1_7_8824138098741025035_v1
2025-10-09T21:45:51.441467Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:45:51.441478Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:45:51.441484Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_7_8824138098741025035_v1" (Sender=[1:7559341518238662116:6522], Pipe=[1:7559341518238662124:6522], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:45:51.441487Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:45:51.441507Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:45:51.441516Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_7_8824138098741025035_v1" (Sender=[1:7559341518238662116:6522], Pipe=[1:7559341518238662124:6522], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:45:51.441525Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_7_8824138098741025035_v1" sender [1:7559341518238662116:6522] lock partition 0 for ReadingSession "shared/user_1_7_8824138098741025035_v1" (Sender=[1:7559341518238662116:6522], Pipe=[1:7559341518238662124:6522], Partitions=[], ActiveFamilyCount=1) generation 1 step 7
2025-10-09T21:45:51.441537Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:45:51.441541Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000032s
2025-10-09T21:45:51.441798Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 7 Session: "shared/user_1_7_8824138098741025035_v1" ClientId: "user" PipeClient { RawX1: 7559341518238662124 RawX2: 4503603922344314 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:45:51.441817Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:45:51.446464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341518238662121:6521] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:51.451142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341518238662121:6521] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:51.451165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341518238662121:6521] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:51.451170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341518238662121:6521] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:51.451186Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 602 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:51.457204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:51.466899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:51.467198Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:51.467203Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:51.467216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:51.467284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:51.467323Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341517876848716:5951]
2025-10-09T21:45:51.467978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:51.470292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:51.470384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:51.470803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:51.470910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:51.470912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:51.470913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:51.470915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:51.470917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:51.470918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:51.470919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.470921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.470922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.470932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:51.470944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:51.470949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:51.470953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:51.470957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:51.470962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:51.470965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.470969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.470973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.470982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:51.470991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:51.470998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:51.471003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:51.471009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:51.471015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:51.471022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:51.471028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:51.471036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:51.471041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:51.471045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:51.471051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:51.471057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.471062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.471066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.471072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.471076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.471081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.471110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:51.485833Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 601
2025-10-09T21:45:51.485887Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 601
2025-10-09T21:45:51.485917Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 601, pipe: [1:7559341518238662143:6527]
2025-10-09T21:45:51.482920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:51.484438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:51.484445Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:51.484448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:51.484453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:51.484463Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 601 [2:7559341517876848716:5951]
2025-10-09T21:45:51.484479Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:51.484501Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:51.484505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:51.484508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:51.484510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:51.484529Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:51.484535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:51.484542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:51.484604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.484606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.484608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.484610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:51.484693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:51.484696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.484711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.484722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.484737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:51.484757Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341517876848711:5951]
2025-10-09T21:45:51.484759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341517876848711:5951]
2025-10-09T21:45:51.484761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341517876848711:5951]
2025-10-09T21:45:51.484770Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:51.484787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:51.484803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:51.484806Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:51.484844Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:51.484845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341517876848711:5951]
2025-10-09T21:45:51.484848Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:51.484858Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:51.484860Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:51.484868Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:51.486324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341518238662143:6527], now have 1 active actors on pipe
2025-10-09T21:45:51.486336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341518238662103:2432], now have 1 active actors on pipe
2025-10-09T21:45:51.486339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341518238662260:6521], now have 1 active actors on pipe
2025-10-09T21:45:51.486365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:51.486370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:51.486383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_7_8824138098741025035_v1 on pipe: [1:7559341518238662143:6527]
2025-10-09T21:45:51.486400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:51.486402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:51.486417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:51.486424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:51.486426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:51.486448Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_7_8824138098741025035_v1
2025-10-09T21:45:51.486451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:51.486488Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|56e95fea-ac2e47d3-5b2bccbe-79a148a1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:51.486494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:45:51.486499Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:45:51.486646Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_7_8824138098741025035_v1:1 with generation 601
2025-10-09T21:45:51.498230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:45:51.498272Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:45:51.498285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:51.498294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:51.498298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:51.498313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:51.498335Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:51.498345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:45:51.498363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:45:51.498383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:51.502450Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046351482 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:45:51.502474Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:45:51.502506Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 sending to client partition status
2025-10-09T21:45:51.508730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 602 partition: 0 MaxSeqNo: 25 sessionId: src_id|56e95fea-ac2e47d3-5b2bccbe-79a148a1_0
2025-10-09T21:45:51.506151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:51.506178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:51.506222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:51.511619Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:51.511639Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:51.511641Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:51.511643Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:51.511656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341517876848711:5951]
2025-10-09T21:45:51.511660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341517876848711:5951]
2025-10-09T21:45:51.511668Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341517876848711:5951]
2025-10-09T21:45:51.511713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:51.511720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.511722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:51.511858Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:51.514170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b837e31b-7f983a53-5d2c9dc6-2daff95a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|56e95fea-ac2e47d3-5b2bccbe-79a148a1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:51.514378Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:51.514393Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:51.514399Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:51.516633Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56e95fea-ac2e47d3-5b2bccbe-79a148a1_0] Write session: try to update token
2025-10-09T21:45:51.516652Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56e95fea-ac2e47d3-5b2bccbe-79a148a1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:51.542928Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 602 sessionId: src_id|56e95fea-ac2e47d3-5b2bccbe-79a148a1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:51.545385Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:51.702963Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:51.703593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:51.703610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:51.703928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_7_8824138098741025035_v1
2025-10-09T21:45:51.703945Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:51.703978Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_7_8824138098741025035_v1
2025-10-09T21:45:51.707665Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:51.707688Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341518238662143:6527]
2025-10-09T21:45:51.707720Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:51.707822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 602 reason: status is not ok: tablet will be restarted right now sessionId: src_id|56e95fea-ac2e47d3-5b2bccbe-79a148a1_0
2025-10-09T21:45:51.708038Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 602 sessionId: src_id|56e95fea-ac2e47d3-5b2bccbe-79a148a1_0 is DEAD
2025-10-09T21:45:51.708290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56e95fea-ac2e47d3-5b2bccbe-79a148a1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:51.708303Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56e95fea-ac2e47d3-5b2bccbe-79a148a1_0] Write session will restart in 0.008545s
2025-10-09T21:45:51.708334Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56e95fea-ac2e47d3-5b2bccbe-79a148a1_0] Write session: Do CDS request
2025-10-09T21:45:51.708342Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56e95fea-ac2e47d3-5b2bccbe-79a148a1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:51.718706Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56e95fea-ac2e47d3-5b2bccbe-79a148a1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:51.719093Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:51.719124Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 603
2025-10-09T21:45:51.719265Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 603 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:51.719314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 603 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:51.719322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=603 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:51.719325Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:51.726638Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 603 sessionId:  describe result for acl check
2025-10-09T21:45:51.726697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:51.726699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:51.726701Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:51.726708Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341518238662317:6538] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:51.727340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341518238662317:6538] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:51.729889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341518238662317:6538] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:51.734303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341518238662317:6538] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:51.734323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341518238662317:6538] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:51.734327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341518238662317:6538] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:51.734338Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 603 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:51.762532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:51.768354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:51.768649Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:51.768654Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:51.768667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:51.768757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:51.769124Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341517876848905:5953]
2025-10-09T21:45:51.769693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:51.770327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:51.770396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:51.770759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:51.770819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:51.770820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:51.770821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:51.770822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:51.770823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:51.770824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:51.770825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.770827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.770828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.770834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:51.770843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:51.770847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:51.770866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:51.770870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:51.770874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:51.770877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.770880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.770883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.770891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:51.770899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:51.770905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:51.770910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:51.770914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:51.770919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:51.770922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:51.770927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:51.770932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:51.770937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:51.770941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:51.770945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:51.770949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.770954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.770960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.770964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.770968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.770973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.771005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:51.780994Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 602
2025-10-09T21:45:51.779392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:51.780505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:51.780511Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:51.780514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:51.780519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:51.780527Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 602 [2:7559341517876848905:5953]
2025-10-09T21:45:51.780542Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:51.780563Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:51.780567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:51.780571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:51.780575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:51.780594Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:51.780601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:51.780609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:51.780677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.780679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.780681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.780684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:51.780760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:51.780764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.780777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:51.780791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:51.780812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:51.780833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341517876848900:5953]
2025-10-09T21:45:51.780837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341517876848900:5953]
2025-10-09T21:45:51.780839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341517876848900:5953]
2025-10-09T21:45:51.780848Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:51.780859Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:51.780863Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:51.780865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:51.780865Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:51.780882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:51.780885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:51.780898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341518238662356:6538], now have 1 active actors on pipe
2025-10-09T21:45:51.780902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341518238662309:2432], now have 1 active actors on pipe
2025-10-09T21:45:51.780941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341517876848900:5953]
2025-10-09T21:45:51.780952Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:51.781001Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:51.782746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:51.782764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:51.782812Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|452e2c35-d2f13fa1-25965b-762c39a2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:51.782829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:51.782834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:51.782846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:51.782859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:51.782880Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:51.782909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:51.783654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:51.783670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:51.783693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:51.781049Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 602
2025-10-09T21:45:51.783875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 603 partition: 0 MaxSeqNo: 25 sessionId: src_id|452e2c35-d2f13fa1-25965b-762c39a2_0
2025-10-09T21:45:51.786643Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56e95fea-ac2e47d3-5b2bccbe-79a148a1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|452e2c35-d2f13fa1-25965b-762c39a2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:51.789326Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|452e2c35-d2f13fa1-25965b-762c39a2_0] Write session: try to update token
2025-10-09T21:45:51.789344Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|452e2c35-d2f13fa1-25965b-762c39a2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:51.804101Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:51.804122Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:51.804125Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:51.804128Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:51.804137Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341517876848900:5953]
2025-10-09T21:45:51.804143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341517876848900:5953]
2025-10-09T21:45:51.804146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341517876848900:5953]
2025-10-09T21:45:51.804193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:51.804199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:51.804201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:51.804332Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:51.807869Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:51.807892Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:51.807896Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:51.808152Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341518238662513:6527]
2025-10-09T21:45:51.808807Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 602, pipe: [1:7559341518238662513:6527]
2025-10-09T21:45:51.815264Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 603 sessionId: src_id|452e2c35-d2f13fa1-25965b-762c39a2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:51.818065Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:51.992828Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:51.993371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341518238662513:6527], now have 1 active actors on pipe
2025-10-09T21:45:51.993427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:51.993434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:51.993984Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:51.994000Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:51.994078Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 603 reason: error at write: Disconnected sessionId: src_id|452e2c35-d2f13fa1-25965b-762c39a2_0
2025-10-09T21:45:51.994080Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:51.994133Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341518238662513:6527]
2025-10-09T21:45:51.994758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|452e2c35-d2f13fa1-25965b-762c39a2_0] Counters: { Errors: 0 CurrentSessionLifetimeMs: 208 BytesWritten: 56320040 MessagesWritten: 25 BytesWrittenCompressed: 0 BytesInflightUncompressed: 22528020 BytesInflightCompressed: 0 BytesInflightTotal: 18022416 MessagesInflight: 10 }
2025-10-09T21:45:51.994789Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|452e2c35-d2f13fa1-25965b-762c39a2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:51.994793Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|452e2c35-d2f13fa1-25965b-762c39a2_0] Write session will restart in 0.009872s
2025-10-09T21:45:51.994818Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|452e2c35-d2f13fa1-25965b-762c39a2_0] Write session: Do CDS request
2025-10-09T21:45:51.994824Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|452e2c35-d2f13fa1-25965b-762c39a2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:51.994284Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:51.994290Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 603 sessionId: src_id|452e2c35-d2f13fa1-25965b-762c39a2_0 is DEAD
2025-10-09T21:45:52.006563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|452e2c35-d2f13fa1-25965b-762c39a2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:52.010332Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:52.010360Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 604
2025-10-09T21:45:52.014186Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 604 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:52.014238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 604 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:52.014245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=604 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:52.014250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:52.014803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 604 sessionId:  describe result for acl check
2025-10-09T21:45:52.014846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:52.014849Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:52.014850Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:52.014857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341522533629926:6549] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:52.015455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341522533629926:6549] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:52.018460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341522533629926:6549] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:52.020278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:52.022022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341522533629926:6549] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:52.022061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341522533629926:6549] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:52.022066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341522533629926:6549] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:52.022079Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 604 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:52.025610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:52.025894Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:52.025907Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:52.025921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:52.025978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:52.026026Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341522171816403:5967]
2025-10-09T21:45:52.026646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:52.027157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:52.027224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:52.027565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:52.027613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:52.027614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:52.027615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:52.027617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:52.027618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:52.027619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:52.027620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.027622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.027623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.027630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:52.027641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:52.027646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:52.027650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:52.027654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:52.027658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:52.027662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.027667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.027671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.027679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:52.027687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:52.027693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:52.027698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:52.027702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:52.027708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:52.027712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:52.027717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:52.027725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:52.027730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:52.027735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:52.027740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:52.027746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.027752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.027756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.027762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.027766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.027772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.027802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:52.038783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:52.039907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:52.039926Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:52.039928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:52.039933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:52.039941Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 603 [2:7559341522171816403:5967]
2025-10-09T21:45:52.039956Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:52.039979Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:52.039982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:52.039986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:52.039989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:52.040010Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:52.040016Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:52.040024Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:52.040104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.040114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.040117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.040120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:52.040198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:52.040201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.040217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.040226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.040241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:52.040267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341522171816397:5967]
2025-10-09T21:45:52.040277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341522171816397:5967]
2025-10-09T21:45:52.040279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341522171816397:5967]
2025-10-09T21:45:52.040305Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:52.040322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341518238662523:2432], now have 1 active actors on pipe
2025-10-09T21:45:52.040325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341522533629988:6549], now have 1 active actors on pipe
2025-10-09T21:45:52.040325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:52.040329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:52.040333Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:52.040334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:52.040350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:52.040353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:52.040395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341522171816397:5967]
2025-10-09T21:45:52.040408Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:52.040430Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:52.043101Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 603
2025-10-09T21:45:52.043103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 603
2025-10-09T21:45:52.044066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:52.044084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:52.044127Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e24056dc-c51a176b-ae6c5cbe-72c21346_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:52.044142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:52.044148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:52.044164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:52.044181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:52.044201Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:52.044229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:52.044424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:52.044438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:52.044472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:52.044623Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 604 partition: 0 MaxSeqNo: 25 sessionId: src_id|e24056dc-c51a176b-ae6c5cbe-72c21346_0
2025-10-09T21:45:52.046401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|452e2c35-d2f13fa1-25965b-762c39a2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e24056dc-c51a176b-ae6c5cbe-72c21346_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:52.049076Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e24056dc-c51a176b-ae6c5cbe-72c21346_0] Write session: try to update token
2025-10-09T21:45:52.049096Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e24056dc-c51a176b-ae6c5cbe-72c21346_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:52.062919Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:52.062937Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:52.062940Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:52.062942Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:52.062954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341522171816397:5967]
2025-10-09T21:45:52.062959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341522171816397:5967]
2025-10-09T21:45:52.062963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341522171816397:5967]
2025-10-09T21:45:52.063007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:52.063016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.063023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:52.063171Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:52.066635Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:52.066651Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:52.066655Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:52.067799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 604 sessionId: src_id|e24056dc-c51a176b-ae6c5cbe-72c21346_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:52.070407Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:52.094279Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341522533630004:6527]
2025-10-09T21:45:52.095343Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 603, pipe: [1:7559341522533630004:6527]
2025-10-09T21:45:52.232385Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:52.233023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:52.233045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:52.233097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341522533630004:6527], now have 1 active actors on pipe
2025-10-09T21:45:52.233390Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:52.233436Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:52.233499Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341522533630004:6527]
2025-10-09T21:45:52.233530Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:52.233624Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 604 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e24056dc-c51a176b-ae6c5cbe-72c21346_0
2025-10-09T21:45:52.233831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 604 sessionId: src_id|e24056dc-c51a176b-ae6c5cbe-72c21346_0 is DEAD
2025-10-09T21:45:52.234070Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e24056dc-c51a176b-ae6c5cbe-72c21346_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:52.234082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e24056dc-c51a176b-ae6c5cbe-72c21346_0] Write session will restart in 0.006636s
2025-10-09T21:45:52.234131Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e24056dc-c51a176b-ae6c5cbe-72c21346_0] Write session: Do CDS request
2025-10-09T21:45:52.234138Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e24056dc-c51a176b-ae6c5cbe-72c21346_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:52.241626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e24056dc-c51a176b-ae6c5cbe-72c21346_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:52.241746Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:52.241776Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 605
2025-10-09T21:45:52.241892Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 605 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:52.241935Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 605 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:52.241945Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=605 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:52.241949Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:52.242513Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 605 sessionId:  describe result for acl check
2025-10-09T21:45:52.242558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:52.242560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:52.242562Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:52.242568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341522533630021:6554] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:52.243138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341522533630021:6554] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:52.245996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341522533630021:6554] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:52.250556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341522533630021:6554] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:52.250578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341522533630021:6554] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:52.250583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341522533630021:6554] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:52.250594Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 605 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:52.264023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:52.265049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:52.265307Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:52.265317Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:52.265331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:52.265422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:52.265487Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341522171816580:5969]
2025-10-09T21:45:52.266417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:52.267545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:52.267609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:52.267997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:52.268069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:52.268071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:52.268072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:52.268073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:52.268075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:52.268076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:52.268077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.268079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.268081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.268087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:52.268101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:52.268105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:52.268111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:52.268115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:52.268119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:52.268124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.268128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.268132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.268142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:52.268151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:52.268157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:52.268163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:52.268167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:52.268173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:52.268178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:52.268183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:52.268188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:52.268193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:52.268197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:52.268203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:52.268210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.268215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.268219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.268225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.268229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.268235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.268277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:52.276544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:52.278125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:52.278145Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:52.278148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:52.278164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:52.278174Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 604 [2:7559341522171816580:5969]
2025-10-09T21:45:52.278191Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:52.278218Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:52.278221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:52.278225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:52.278228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:52.278249Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:52.278267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:52.278276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:52.278355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.278361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.278363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.278366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:52.278454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:52.278457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.278472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.278483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.278498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:52.278522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341522171816575:5969]
2025-10-09T21:45:52.278524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341522171816575:5969]
2025-10-09T21:45:52.278526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341522171816575:5969]
2025-10-09T21:45:52.278542Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:52.278557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:52.278571Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:52.278574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:52.278638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341522171816575:5969]
2025-10-09T21:45:52.278641Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:52.278957Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:52.278971Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:52.278973Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:52.278980Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:52.279442Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 604
2025-10-09T21:45:52.279448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341522533630126:6554], now have 1 active actors on pipe
2025-10-09T21:45:52.279496Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 604
2025-10-09T21:45:52.279461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341522533630007:2432], now have 1 active actors on pipe
2025-10-09T21:45:52.279633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:52.279643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:52.279678Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b83e7502-8c688945-4c7d1bf7-dafda860_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:52.279689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:52.279694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:52.279697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:52.279709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:52.279730Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:52.279753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:52.280585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:52.280598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:52.280622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:52.280786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 605 partition: 0 MaxSeqNo: 25 sessionId: src_id|b83e7502-8c688945-4c7d1bf7-dafda860_0
2025-10-09T21:45:52.281191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e24056dc-c51a176b-ae6c5cbe-72c21346_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b83e7502-8c688945-4c7d1bf7-dafda860_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:52.283841Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b83e7502-8c688945-4c7d1bf7-dafda860_0] Write session: try to update token
2025-10-09T21:45:52.283864Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b83e7502-8c688945-4c7d1bf7-dafda860_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:52.300926Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:52.300944Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:52.300947Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:52.300949Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:52.300959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341522171816575:5969]
2025-10-09T21:45:52.300968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341522171816575:5969]
2025-10-09T21:45:52.300972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341522171816575:5969]
2025-10-09T21:45:52.301019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:52.301027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.301029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:52.301061Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:52.301067Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:52.301072Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:52.301182Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:52.308045Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 605 sessionId: src_id|b83e7502-8c688945-4c7d1bf7-dafda860_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:52.310678Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:52.333814Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341522533630192:6527]
2025-10-09T21:45:52.334153Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 604, pipe: [1:7559341522533630192:6527]
2025-10-09T21:45:52.466580Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:52.467201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:52.467216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:52.467280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341522533630192:6527], now have 1 active actors on pipe
2025-10-09T21:45:52.467583Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:52.468322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b83e7502-8c688945-4c7d1bf7-dafda860_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:52.468334Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b83e7502-8c688945-4c7d1bf7-dafda860_0] Write session will restart in 0.005462s
2025-10-09T21:45:52.468357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b83e7502-8c688945-4c7d1bf7-dafda860_0] Write session: Do CDS request
2025-10-09T21:45:52.468364Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b83e7502-8c688945-4c7d1bf7-dafda860_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:52.467681Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341522533630192:6527]
2025-10-09T21:45:52.467683Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:52.467688Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:52.467737Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:52.467771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 605 reason: error at write: Disconnected sessionId: src_id|b83e7502-8c688945-4c7d1bf7-dafda860_0
2025-10-09T21:45:52.467973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 605 sessionId: src_id|b83e7502-8c688945-4c7d1bf7-dafda860_0 is DEAD
2025-10-09T21:45:52.474576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b83e7502-8c688945-4c7d1bf7-dafda860_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:52.474950Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:52.474970Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 606
2025-10-09T21:45:52.475150Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 606 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:52.475209Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 606 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:52.475215Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=606 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:52.475220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:52.476885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 606 sessionId:  describe result for acl check
2025-10-09T21:45:52.476932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:52.476942Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:52.476944Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:52.476951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341522533630202:6558] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:52.477541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341522533630202:6558] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:52.480788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341522533630202:6558] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:52.487398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341522533630202:6558] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:52.487418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341522533630202:6558] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:52.487422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341522533630202:6558] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:52.487433Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 606 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:52.499926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:52.504780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:52.505063Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:52.505078Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:52.505090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:52.505156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:52.505202Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341522171816761:5973]
2025-10-09T21:45:52.505836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:52.506331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:52.506405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:52.506802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:52.506859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:52.506866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:52.506867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:52.506868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:52.506870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:52.506871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:52.506872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.506874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.506875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.506881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:52.506894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:52.506898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:52.506903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:52.506907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:52.506911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:52.506915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.506919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.506923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.506933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:52.506942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:52.506953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:52.506959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:52.506964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:52.506974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:52.506979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:52.506989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:52.506994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:52.507006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:52.507010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:52.507016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:52.507023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.507029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.507038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.507043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.507052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.507062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.507104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:52.516664Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 605
2025-10-09T21:45:52.516701Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 605
2025-10-09T21:45:52.514440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:52.515549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:52.515556Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:52.515559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:52.515564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:52.515571Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 605 [2:7559341522171816761:5973]
2025-10-09T21:45:52.515587Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:52.515610Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:52.515614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:52.515617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:52.515620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:52.515640Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:52.515646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:52.515654Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:52.515718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.515721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.515723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.515726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:52.515806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:52.515808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.515822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.515831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.515845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:52.515867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341522171816756:5973]
2025-10-09T21:45:52.515871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341522171816756:5973]
2025-10-09T21:45:52.515873Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341522171816756:5973]
2025-10-09T21:45:52.515890Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:52.515905Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:52.515920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:52.515923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:52.515967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341522171816756:5973]
2025-10-09T21:45:52.515970Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:52.515972Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:52.515979Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:52.515981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:52.515988Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:52.516206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341522533630260:6558], now have 1 active actors on pipe
2025-10-09T21:45:52.516211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341522533630194:2432], now have 1 active actors on pipe
2025-10-09T21:45:52.517002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:52.517020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:52.517057Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e1aa5ca2-ceec6406-9eebb9cd-2115b3fd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:52.517067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:52.517071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:52.517075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:52.517087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:52.517101Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:52.517121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:52.518261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:52.518290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:52.518316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:52.518467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 606 partition: 0 MaxSeqNo: 25 sessionId: src_id|e1aa5ca2-ceec6406-9eebb9cd-2115b3fd_0
2025-10-09T21:45:52.518917Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b83e7502-8c688945-4c7d1bf7-dafda860_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e1aa5ca2-ceec6406-9eebb9cd-2115b3fd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:52.521605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1aa5ca2-ceec6406-9eebb9cd-2115b3fd_0] Write session: try to update token
2025-10-09T21:45:52.521631Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1aa5ca2-ceec6406-9eebb9cd-2115b3fd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:52.540593Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:52.540610Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:52.540613Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:52.540615Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:52.540626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341522171816756:5973]
2025-10-09T21:45:52.540633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341522171816756:5973]
2025-10-09T21:45:52.540640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341522171816756:5973]
2025-10-09T21:45:52.540686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:52.540693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.540695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:52.540860Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:52.544582Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:52.544597Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:52.544605Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:52.551361Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 606 sessionId: src_id|e1aa5ca2-ceec6406-9eebb9cd-2115b3fd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:52.553814Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:52.567869Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341522533630380:6527]
2025-10-09T21:45:52.568534Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 605, pipe: [1:7559341522533630380:6527]
2025-10-09T21:45:52.782852Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:52.783776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:52.783787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:52.783835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341522533630380:6527], now have 1 active actors on pipe
2025-10-09T21:45:52.784126Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:52.784563Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341522533630380:6527]
2025-10-09T21:45:52.784598Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:52.784642Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:52.784683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 606 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e1aa5ca2-ceec6406-9eebb9cd-2115b3fd_0
2025-10-09T21:45:52.784867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 606 sessionId: src_id|e1aa5ca2-ceec6406-9eebb9cd-2115b3fd_0 is DEAD
2025-10-09T21:45:52.788035Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1aa5ca2-ceec6406-9eebb9cd-2115b3fd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:52.788054Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1aa5ca2-ceec6406-9eebb9cd-2115b3fd_0] Write session will restart in 0.008148s
2025-10-09T21:45:52.788080Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1aa5ca2-ceec6406-9eebb9cd-2115b3fd_0] Write session: Do CDS request
2025-10-09T21:45:52.788086Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1aa5ca2-ceec6406-9eebb9cd-2115b3fd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:52.797596Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1aa5ca2-ceec6406-9eebb9cd-2115b3fd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:52.797666Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:52.797685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 607
2025-10-09T21:45:52.797790Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 607 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:52.797818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 607 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:52.797825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=607 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:52.797829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:52.798266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 607 sessionId:  describe result for acl check
2025-10-09T21:45:52.798299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:52.798304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:52.798307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:52.798313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341522533630400:6563] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:52.798889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341522533630400:6563] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:52.802626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341522533630400:6563] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:52.811389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341522533630400:6563] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:52.811421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341522533630400:6563] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:52.811425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341522533630400:6563] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:52.811436Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 607 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:52.823003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:52.824032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:52.824292Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:52.824304Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:52.824317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:52.824390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:52.824434Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341522171816940:5975]
2025-10-09T21:45:52.825091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:52.825426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:52.825478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:52.825866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:52.825930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:52.825937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:52.825938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:52.825939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:52.825940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:52.825941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:52.825941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.825943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.825944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.825948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:52.825957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:52.825961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:52.825964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:52.825968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:52.825972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:52.825976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.825980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.825984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.825992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:52.826001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:52.826008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:52.826014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:52.826027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:52.826048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:52.826052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:52.826058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:52.826068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:52.826073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:52.826076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:52.826081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:52.826087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.826093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.826097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.826101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.826115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.826120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.826144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:52.833435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:52.835007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:52.835021Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:52.835024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:52.835028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:52.835035Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 606 [2:7559341522171816940:5975]
2025-10-09T21:45:52.835051Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:52.835073Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:52.835076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:52.835080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:52.835082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:52.835112Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:52.835122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:52.835130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:52.835213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.835219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.835221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.835224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:52.835311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:52.835318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.835332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:52.835348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:52.835367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:52.835390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341522171816935:5975]
2025-10-09T21:45:52.835396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341522171816935:5975]
2025-10-09T21:45:52.835399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341522171816935:5975]
2025-10-09T21:45:52.835413Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:52.835432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:52.835450Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:52.835452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:52.835495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341522171816935:5975]
2025-10-09T21:45:52.835502Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:52.835653Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:52.835664Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:52.835666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:52.835674Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:52.835672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341522533630528:6563], now have 1 active actors on pipe
2025-10-09T21:45:52.835681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341522533630385:2432], now have 1 active actors on pipe
2025-10-09T21:45:52.836223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:52.836232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:52.836280Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|42521173-be27904-8be62cce-97f246cf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:52.836292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:52.836297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:52.836299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:52.836312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:52.836347Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:52.836369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:52.836686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:52.836698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:52.836731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:52.837318Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1aa5ca2-ceec6406-9eebb9cd-2115b3fd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|42521173-be27904-8be62cce-97f246cf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:52.836008Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 606
2025-10-09T21:45:52.836042Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 606
2025-10-09T21:45:52.836907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 607 partition: 0 MaxSeqNo: 25 sessionId: src_id|42521173-be27904-8be62cce-97f246cf_0
2025-10-09T21:45:52.840218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42521173-be27904-8be62cce-97f246cf_0] Write session: try to update token
2025-10-09T21:45:52.840237Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42521173-be27904-8be62cce-97f246cf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:52.852705Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:52.852723Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:52.852725Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:52.852726Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:52.852735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341522171816935:5975]
2025-10-09T21:45:52.852738Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341522171816935:5975]
2025-10-09T21:45:52.852741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341522171816935:5975]
2025-10-09T21:45:52.852785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:52.852784Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:52.852789Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:52.852792Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:52.852795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:52.852798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:52.852948Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:52.857846Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 607 sessionId: src_id|42521173-be27904-8be62cce-97f246cf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:52.860249Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:52.877211Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] Counters: { Errors: 6 CurrentSessionLifetimeMs: 181328 BytesRead: 24780812 MessagesRead: 11 BytesReadCompressed: 24780812 BytesInflightUncompressed: 2252802 BytesInflightCompressed: 0 BytesInflightTotal: 2252802 MessagesInflight: 1 }
2025-10-09T21:45:52.884699Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341522533630594:6527]
2025-10-09T21:45:52.885085Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 606, pipe: [1:7559341522533630594:6527]
2025-10-09T21:45:53.011439Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:53.011887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:53.011899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:53.011951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341522533630594:6527], now have 1 active actors on pipe
2025-10-09T21:45:53.012265Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:53.013058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42521173-be27904-8be62cce-97f246cf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:53.013069Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42521173-be27904-8be62cce-97f246cf_0] Write session will restart in 0.009006s
2025-10-09T21:45:53.013097Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42521173-be27904-8be62cce-97f246cf_0] Write session: Do CDS request
2025-10-09T21:45:53.013114Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42521173-be27904-8be62cce-97f246cf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:53.012583Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:53.012675Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 607 reason: status is not ok: tablet will be restarted right now sessionId: src_id|42521173-be27904-8be62cce-97f246cf_0
2025-10-09T21:45:53.012887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 607 sessionId: src_id|42521173-be27904-8be62cce-97f246cf_0 is DEAD
2025-10-09T21:45:53.013048Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:53.013085Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341522533630594:6527]
2025-10-09T21:45:53.023909Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42521173-be27904-8be62cce-97f246cf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:53.024703Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:53.024723Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 608
2025-10-09T21:45:53.024851Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 608 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:53.024880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 608 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:53.024887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=608 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:53.024891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:53.025394Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 608 sessionId:  describe result for acl check
2025-10-09T21:45:53.025441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:53.025443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:53.025445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:53.025452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341526828597920:6574] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:53.026263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341526828597920:6574] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:53.029765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341526828597920:6574] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:53.033364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341526828597920:6574] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:53.033385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341526828597920:6574] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:53.033389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341526828597920:6574] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:53.033400Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 608 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:53.041176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:53.041840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:53.042088Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:53.042099Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:53.042134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:53.042227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:53.042288Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341526466784453:5989]
2025-10-09T21:45:53.042886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:53.043411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:53.043475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:53.043831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:53.043895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:53.043897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:53.043898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:53.043899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:53.043901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:53.043902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:53.043904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.043906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.043908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.043914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:53.043925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:53.043928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:53.043932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:53.043936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:53.043940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:53.043944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.043948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.043952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.043960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:53.043969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:53.043978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:53.043983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:53.043988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:53.043992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:53.043996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:53.044001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:53.044006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:53.044011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:53.044016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:53.044036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:53.044042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.044048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.044053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.044059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.044063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.044068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.044095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:53.054901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:53.056646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:53.056671Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:53.056673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:53.056678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:53.056687Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 607 [2:7559341526466784453:5989]
2025-10-09T21:45:53.056704Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:53.056727Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:53.056730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:53.056734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:53.056737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:53.056756Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:53.056769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:53.056777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:53.056864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.056872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.056874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.056877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:53.056969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:53.056978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.056994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.057007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.057023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:53.057046Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341526466784448:5989]
2025-10-09T21:45:53.057048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341526466784448:5989]
2025-10-09T21:45:53.057051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341526466784448:5989]
2025-10-09T21:45:53.057063Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:53.057080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:53.057082Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:53.057086Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:53.057089Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:53.057104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:53.057111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:53.057124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341526828598070:6574], now have 1 active actors on pipe
2025-10-09T21:45:53.057133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341526828597900:2432], now have 1 active actors on pipe
2025-10-09T21:45:53.057177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341526466784448:5989]
2025-10-09T21:45:53.057193Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:53.057189Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 607
2025-10-09T21:45:53.057220Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 607
2025-10-09T21:45:53.057291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:53.057568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:53.057575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:53.057608Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|57fafd27-191babc4-164b15f2-5645195c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:53.057625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:53.057631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:53.057634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:53.057649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:53.057663Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:53.057694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:53.058140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:53.058151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:53.058175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:53.058375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 608 partition: 0 MaxSeqNo: 25 sessionId: src_id|57fafd27-191babc4-164b15f2-5645195c_0
2025-10-09T21:45:53.059167Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42521173-be27904-8be62cce-97f246cf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|57fafd27-191babc4-164b15f2-5645195c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:53.061907Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57fafd27-191babc4-164b15f2-5645195c_0] Write session: try to update token
2025-10-09T21:45:53.061932Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57fafd27-191babc4-164b15f2-5645195c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:53.079466Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:53.079489Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:53.079502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341526466784448:5989]
2025-10-09T21:45:53.079560Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:53.079562Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:53.079984Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:53.080203Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:53.080374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:53.080548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:53.080798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:53.080845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:53.084391Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 608 sessionId: src_id|57fafd27-191babc4-164b15f2-5645195c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:53.087208Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:53.087761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:53.087801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:53.087858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:53.089337Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:53.089354Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:53.089356Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:53.089358Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:53.089369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341526466784448:5989]
2025-10-09T21:45:53.089373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341526466784448:5989]
2025-10-09T21:45:53.089377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341526466784448:5989]
2025-10-09T21:45:53.089421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:53.089431Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:53.089436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.089437Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:53.089438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:53.089444Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:53.089579Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:53.087982Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:53.113092Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341526828598089:6527]
2025-10-09T21:45:53.113577Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 607, pipe: [1:7559341526828598089:6527]
2025-10-09T21:45:53.263964Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:53.264511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341526828598089:6527], now have 1 active actors on pipe
2025-10-09T21:45:53.273092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:53.273124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:53.273308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:53.273374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:53.273441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:53.273506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:53.273558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:53.273572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:53.273642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:53.273699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:53.273758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:53.273824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:53.273861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:53.273864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:53.273932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:53.273989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:53.274219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:53.274286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:53.274328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:53.274335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:53.274427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:53.274487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:53.274550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:53.274611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:53.274685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:53.274688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:53.274754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:53.274811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:53.274871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:53.274931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:53.274973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:53.274976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:53.275041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:53.275094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:53.275150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:53.275203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:53.275265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:53.275268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:53.275334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:53.275390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:53.275458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:53.275520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:53.275564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:53.275567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:53.275630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:53.275693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:53.275766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:53.275831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:53.275875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:53.275878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:53.275951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:53.276024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:53.276090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:53.276154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:53.276207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:53.276210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:53.276281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:53.276345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:53.276405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:53.276466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:53.276508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:53.276511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:53.277795Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:53.279034Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57fafd27-191babc4-164b15f2-5645195c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:53.279047Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57fafd27-191babc4-164b15f2-5645195c_0] Write session will restart in 0.009581s
2025-10-09T21:45:53.279074Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57fafd27-191babc4-164b15f2-5645195c_0] Write session: Do CDS request
2025-10-09T21:45:53.279081Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57fafd27-191babc4-164b15f2-5645195c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:53.278633Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:53.278691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 608 reason: status is not ok: tablet will be restarted right now sessionId: src_id|57fafd27-191babc4-164b15f2-5645195c_0
2025-10-09T21:45:53.278904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 608 sessionId: src_id|57fafd27-191babc4-164b15f2-5645195c_0 is DEAD
2025-10-09T21:45:53.279370Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:53.279414Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341526828598089:6527]
2025-10-09T21:45:53.289661Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57fafd27-191babc4-164b15f2-5645195c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:53.289715Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:53.289738Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 609
2025-10-09T21:45:53.289931Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 609 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:53.289984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 609 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:53.289990Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=609 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:53.289993Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:53.290667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 609 sessionId:  describe result for acl check
2025-10-09T21:45:53.290706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:53.290708Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:53.290710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:53.290716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341526828598107:6579] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:53.291299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341526828598107:6579] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:53.295124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341526828598107:6579] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:53.303045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341526828598107:6579] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:53.303063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341526828598107:6579] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:53.303068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341526828598107:6579] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:53.303078Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 609 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:53.311304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:53.317780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:53.318076Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:53.318080Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:53.318094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:53.318173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:53.318211Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341526466784634:5991]
2025-10-09T21:45:53.318824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:53.319691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:53.319751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:53.320029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:53.320095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:53.320096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:53.320098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:53.320099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:53.320100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:53.320102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:53.320103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.320105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.320106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.320111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:53.320118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:53.320122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:53.320127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:53.320131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:53.320134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:53.320137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.320140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.320145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.320153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:53.320164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:53.320172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:53.320178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:53.320182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:53.320188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:53.320192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:53.320198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:53.320203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:53.320207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:53.320209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:53.320212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:53.320216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.320219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.320222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.320224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.320227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.320230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.320248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:53.328694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:53.330384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:53.330399Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:53.330401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:53.330406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:53.330414Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 608 [2:7559341526466784634:5991]
2025-10-09T21:45:53.330430Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:53.330450Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:53.330458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:53.330461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:53.330463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:53.330483Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:53.330492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:53.330501Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:53.330568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.330571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.330573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.330576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:53.330639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:53.330641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.330650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.330656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.330665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:53.330681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341526466784628:5991]
2025-10-09T21:45:53.330682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341526466784628:5991]
2025-10-09T21:45:53.330684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341526466784628:5991]
2025-10-09T21:45:53.330693Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:53.330704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:53.330714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:53.330716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:53.330751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341526466784628:5991]
2025-10-09T21:45:53.330753Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:53.330999Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:53.331003Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:53.331005Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:53.331012Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:53.331262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341526828598265:6579], now have 1 active actors on pipe
2025-10-09T21:45:53.331269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341526828598092:2432], now have 1 active actors on pipe
2025-10-09T21:45:53.331617Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 608
2025-10-09T21:45:53.331655Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 608
2025-10-09T21:45:53.332443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:53.332457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:53.332495Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8924207f-b6e58209-1061bf74-62831cbf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:53.332506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:53.332511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:53.332524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:53.332536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:53.332552Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:53.332580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:53.333184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:53.333194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:53.333213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:53.333328Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 609 partition: 0 MaxSeqNo: 25 sessionId: src_id|8924207f-b6e58209-1061bf74-62831cbf_0
2025-10-09T21:45:53.333703Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57fafd27-191babc4-164b15f2-5645195c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8924207f-b6e58209-1061bf74-62831cbf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:53.336310Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8924207f-b6e58209-1061bf74-62831cbf_0] Write session: try to update token
2025-10-09T21:45:53.336327Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8924207f-b6e58209-1061bf74-62831cbf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:53.351014Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:53.351046Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:53.351049Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:53.351051Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:53.351062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341526466784628:5991]
2025-10-09T21:45:53.351066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341526466784628:5991]
2025-10-09T21:45:53.351071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341526466784628:5991]
2025-10-09T21:45:53.351120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:53.351121Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:53.351128Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:53.351128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.351131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:53.351137Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:53.351341Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:53.361020Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 609 sessionId: src_id|8924207f-b6e58209-1061bf74-62831cbf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:53.363861Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:53.379498Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341526828598281:6527]
2025-10-09T21:45:53.382625Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 608, pipe: [1:7559341526828598281:6527]
2025-10-09T21:45:53.538964Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:53.539777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:53.539791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:53.539844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341526828598281:6527], now have 1 active actors on pipe
2025-10-09T21:45:53.540601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:53.540623Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:53.540351Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:53.541367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8924207f-b6e58209-1061bf74-62831cbf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:53.541379Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8924207f-b6e58209-1061bf74-62831cbf_0] Write session will restart in 0.009668s
2025-10-09T21:45:53.541405Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8924207f-b6e58209-1061bf74-62831cbf_0] Write session: Do CDS request
2025-10-09T21:45:53.541410Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8924207f-b6e58209-1061bf74-62831cbf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:53.540692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 609 reason: error at write: Disconnected sessionId: src_id|8924207f-b6e58209-1061bf74-62831cbf_0
2025-10-09T21:45:53.540920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 609 sessionId: src_id|8924207f-b6e58209-1061bf74-62831cbf_0 is DEAD
2025-10-09T21:45:53.541094Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:53.541128Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341526828598281:6527]
2025-10-09T21:45:53.554730Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8924207f-b6e58209-1061bf74-62831cbf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:53.555890Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:53.555910Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 610
2025-10-09T21:45:53.556580Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 610 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:53.556624Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 610 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:53.556631Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=610 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:53.556638Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:53.557132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 610 sessionId:  describe result for acl check
2025-10-09T21:45:53.557173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:53.557175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:53.557178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:53.557184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341526828598298:6583] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:53.557771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341526828598298:6583] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:53.567596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341526828598298:6583] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:53.589891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:53.594619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341526828598298:6583] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:53.594644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341526828598298:6583] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:53.594648Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341526828598298:6583] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:53.594660Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 610 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:53.600056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:53.600332Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:53.600337Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:53.600353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:53.600426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:53.600476Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341526466784819:5995]
2025-10-09T21:45:53.601106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:53.603204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:53.603285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:53.603674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:53.603736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:53.603737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:53.603738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:53.603740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:53.603741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:53.603742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:53.603744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.603745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.603746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.603754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:53.603765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:53.603770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:53.603775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:53.603779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:53.603783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:53.603787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.603791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.603795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.603804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:53.603812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:53.603820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:53.603826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:53.603832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:53.603837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:53.603842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:53.603848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:53.603854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:53.603860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:53.603865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:53.603871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:53.603876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.603881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.603885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.603890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.603895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.603901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.603931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:53.613653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:53.615386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:53.615416Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:53.615419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:53.615424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:53.615433Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 609 [2:7559341526466784819:5995]
2025-10-09T21:45:53.615453Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:53.615476Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:53.615480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:53.615484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:53.615487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:53.615507Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:53.615514Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:53.615522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:53.615602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.615605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.615607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.615610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:53.615820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:53.615823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.615839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.615850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.615866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:53.615890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341526466784812:5995]
2025-10-09T21:45:53.615893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341526466784812:5995]
2025-10-09T21:45:53.615896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341526466784812:5995]
2025-10-09T21:45:53.615911Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:53.615929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:53.615946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:53.615950Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:53.615998Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341526466784812:5995]
2025-10-09T21:45:53.616000Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:53.616181Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:53.616185Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:53.616187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:53.616194Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:53.617359Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 609
2025-10-09T21:45:53.617406Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 609
2025-10-09T21:45:53.617935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341526828598284:2432], now have 1 active actors on pipe
2025-10-09T21:45:53.617947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341526828598460:6583], now have 1 active actors on pipe
2025-10-09T21:45:53.617980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:53.617987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:53.618022Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|843b58f3-1fd1e5f9-1387d6d0-1fceb14d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:53.618047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:53.618052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:53.618056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:53.618070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:53.618089Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:53.618123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:53.618671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:53.618689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:53.618712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:53.618887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 610 partition: 0 MaxSeqNo: 25 sessionId: src_id|843b58f3-1fd1e5f9-1387d6d0-1fceb14d_0
2025-10-09T21:45:53.619307Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8924207f-b6e58209-1061bf74-62831cbf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|843b58f3-1fd1e5f9-1387d6d0-1fceb14d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:53.621796Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|843b58f3-1fd1e5f9-1387d6d0-1fceb14d_0] Write session: try to update token
2025-10-09T21:45:53.621824Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|843b58f3-1fd1e5f9-1387d6d0-1fceb14d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:53.636947Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:53.636967Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:53.636970Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:53.636972Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:53.636983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341526466784812:5995]
2025-10-09T21:45:53.636988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341526466784812:5995]
2025-10-09T21:45:53.636993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341526466784812:5995]
2025-10-09T21:45:53.637041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:53.637049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.637052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:53.637241Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:53.642025Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341526828598474:6527]
2025-10-09T21:45:53.643480Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:53.643494Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:53.643498Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:53.643962Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 609, pipe: [1:7559341526828598474:6527]
2025-10-09T21:45:53.660632Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 610 sessionId: src_id|843b58f3-1fd1e5f9-1387d6d0-1fceb14d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:53.663389Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:53.842516Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:53.842835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341526828598474:6527], now have 1 active actors on pipe
2025-10-09T21:45:53.842886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:53.842893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:53.843327Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:53.847216Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:53.847336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 610 reason: status is not ok: tablet will be restarted right now sessionId: src_id|843b58f3-1fd1e5f9-1387d6d0-1fceb14d_0
2025-10-09T21:45:53.847538Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 610 sessionId: src_id|843b58f3-1fd1e5f9-1387d6d0-1fceb14d_0 is DEAD
2025-10-09T21:45:53.847731Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341526828598474:6527]
2025-10-09T21:45:53.847746Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:53.850324Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|843b58f3-1fd1e5f9-1387d6d0-1fceb14d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:53.850338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|843b58f3-1fd1e5f9-1387d6d0-1fceb14d_0] Write session will restart in 0.007565s
2025-10-09T21:45:53.850368Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|843b58f3-1fd1e5f9-1387d6d0-1fceb14d_0] Write session: Do CDS request
2025-10-09T21:45:53.850374Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|843b58f3-1fd1e5f9-1387d6d0-1fceb14d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:53.859585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|843b58f3-1fd1e5f9-1387d6d0-1fceb14d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:53.859716Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:53.859736Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 611
2025-10-09T21:45:53.860025Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 611 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:53.860069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 611 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:53.860077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=611 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:53.860081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:53.860489Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 611 sessionId:  describe result for acl check
2025-10-09T21:45:53.860528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:53.860530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:53.860532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:53.860539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341526828598498:6594] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:53.861117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341526828598498:6594] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:53.864561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341526828598498:6594] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:53.868947Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341526828598498:6594] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:53.868966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341526828598498:6594] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:53.868970Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341526828598498:6594] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:53.868979Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 611 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:53.883862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:53.888226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:53.888503Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:53.888508Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:53.888521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:53.888589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:53.888643Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341526466785002:5997]
2025-10-09T21:45:53.889237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:53.889894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:53.889957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:53.890314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:53.890385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:53.890392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:53.890394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:53.890395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:53.890396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:53.890397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:53.890398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.890399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.890400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.890406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:53.890416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:53.890420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:53.890424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:53.890428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:53.890431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:53.890435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.890438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.890441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.890450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:53.890458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:53.890463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:53.890468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:53.890472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:53.890477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:53.890481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:53.890485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:53.890489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:53.890493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:53.890497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:53.890502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:53.890510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.890514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.890519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.890524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.890533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.890539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.890564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:53.898326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:53.899880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:53.899902Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:53.899904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:53.899908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:53.899915Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 610 [2:7559341526466785002:5997]
2025-10-09T21:45:53.899930Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:53.899952Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:53.899955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:53.899958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:53.899961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:53.899981Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:53.899991Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:53.899999Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:53.900066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.900073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.900075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.900077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:53.900620Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 610
2025-10-09T21:45:53.900663Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 610
2025-10-09T21:45:53.900151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:53.900158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.900171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:53.900181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:53.900194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:53.900215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341526466784997:5997]
2025-10-09T21:45:53.900217Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341526466784997:5997]
2025-10-09T21:45:53.900220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341526466784997:5997]
2025-10-09T21:45:53.900229Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:53.900243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:53.900243Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:53.900245Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:53.900248Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:53.900255Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:53.900257Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:53.900268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341526828598478:2432], now have 1 active actors on pipe
2025-10-09T21:45:53.900271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341526828598539:6594], now have 1 active actors on pipe
2025-10-09T21:45:53.900309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341526466784997:5997]
2025-10-09T21:45:53.900317Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:53.900813Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:53.902010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:53.902054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:53.902091Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1ad4e3c8-339d3bdc-877176ad-e04f904f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:53.902120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:53.902126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:53.902129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:53.902141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:53.902159Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:53.902179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:53.903885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:53.903902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:53.903935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:53.904093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 611 partition: 0 MaxSeqNo: 25 sessionId: src_id|1ad4e3c8-339d3bdc-877176ad-e04f904f_0
2025-10-09T21:45:53.904661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|843b58f3-1fd1e5f9-1387d6d0-1fceb14d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1ad4e3c8-339d3bdc-877176ad-e04f904f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:53.907872Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad4e3c8-339d3bdc-877176ad-e04f904f_0] Write session: try to update token
2025-10-09T21:45:53.907896Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad4e3c8-339d3bdc-877176ad-e04f904f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:53.928575Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:53.928595Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:53.928597Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:53.928598Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:53.928606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341526466784997:5997]
2025-10-09T21:45:53.928609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341526466784997:5997]
2025-10-09T21:45:53.928615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341526466784997:5997]
2025-10-09T21:45:53.928654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:53.928661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:53.928663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:53.928804Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:53.930224Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:53.930238Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:53.930242Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:53.938955Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 611 sessionId: src_id|1ad4e3c8-339d3bdc-877176ad-e04f904f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:53.941643Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:53.950161Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341526828598681:6527]
2025-10-09T21:45:53.951768Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 610, pipe: [1:7559341526828598681:6527]
2025-10-09T21:45:54.104883Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:54.105841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:54.105862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:54.105911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341526828598681:6527], now have 1 active actors on pipe
2025-10-09T21:45:54.106359Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:54.106820Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:54.106921Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:54.106952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 611 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1ad4e3c8-339d3bdc-877176ad-e04f904f_0
2025-10-09T21:45:54.106962Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341526828598681:6527]
2025-10-09T21:45:54.107171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 611 sessionId: src_id|1ad4e3c8-339d3bdc-877176ad-e04f904f_0 is DEAD
2025-10-09T21:45:54.110318Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad4e3c8-339d3bdc-877176ad-e04f904f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:54.110333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad4e3c8-339d3bdc-877176ad-e04f904f_0] Write session will restart in 0.008659s
2025-10-09T21:45:54.110364Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad4e3c8-339d3bdc-877176ad-e04f904f_0] Write session: Do CDS request
2025-10-09T21:45:54.110372Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad4e3c8-339d3bdc-877176ad-e04f904f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:54.122543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad4e3c8-339d3bdc-877176ad-e04f904f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:54.122770Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:54.122786Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 612
2025-10-09T21:45:54.122958Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 612 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:54.122995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 612 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:54.123002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=612 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:54.123006Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:54.124379Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 612 sessionId:  describe result for acl check
2025-10-09T21:45:54.124430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:54.124432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:54.124434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:54.124440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341531123566014:6600] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:54.125109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341531123566014:6600] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:54.149769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341531123566014:6600] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:54.154956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:54.156784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341531123566014:6600] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:54.156807Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341531123566014:6600] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:54.156811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341531123566014:6600] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:54.156823Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 612 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:54.163070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:54.163645Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:54.163661Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:54.163677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:54.163750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:54.163799Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341530761752520:6011]
2025-10-09T21:45:54.164431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:54.164967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:54.165030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:54.165366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:54.165440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:54.165441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:54.165443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:54.165444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:54.165445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:54.165447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:54.165448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.165449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.165451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.165458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:54.165470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:54.165474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:54.165477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:54.165481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:54.165486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:54.165490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.165494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.165498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.165507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:54.165517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:54.165523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:54.165529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:54.165535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:54.165540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:54.165544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:54.165549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:54.165553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:54.165575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:54.165580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:54.165586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:54.165590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.165595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.165600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.165605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.165610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.165615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.165644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:54.179212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:54.180403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:54.180410Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:54.180413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:54.180419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:54.180428Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 611 [2:7559341530761752520:6011]
2025-10-09T21:45:54.180444Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:54.180467Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:54.180471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:54.180475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:54.180477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:54.180498Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:54.180504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:54.180513Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:54.180591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.180593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.180596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.180600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:54.180684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:54.180687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.180700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.180711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.180726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:54.180751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341530761752513:6011]
2025-10-09T21:45:54.180754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341530761752513:6011]
2025-10-09T21:45:54.180756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341530761752513:6011]
2025-10-09T21:45:54.180774Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:54.180790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:54.180805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:54.180808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:54.180829Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:54.180833Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:54.180836Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:54.180855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341530761752513:6011]
2025-10-09T21:45:54.180859Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:54.180943Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:54.181130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341531123566178:6600], now have 1 active actors on pipe
2025-10-09T21:45:54.181149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341531123565999:2432], now have 1 active actors on pipe
2025-10-09T21:45:54.181250Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 611
2025-10-09T21:45:54.181300Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 611
2025-10-09T21:45:54.185925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:54.185946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:54.185987Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|99c017f7-21e0408e-5a89d89c-dfac284b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:54.186001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:54.186007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:54.186010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:54.186026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:54.186059Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:54.186081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:54.189140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:54.189159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:54.189198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:54.189480Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 612 partition: 0 MaxSeqNo: 25 sessionId: src_id|99c017f7-21e0408e-5a89d89c-dfac284b_0
2025-10-09T21:45:54.189894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad4e3c8-339d3bdc-877176ad-e04f904f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|99c017f7-21e0408e-5a89d89c-dfac284b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:54.192734Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99c017f7-21e0408e-5a89d89c-dfac284b_0] Write session: try to update token
2025-10-09T21:45:54.192758Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99c017f7-21e0408e-5a89d89c-dfac284b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:54.200729Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:54.200750Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:54.200752Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:54.200755Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:54.200765Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341530761752513:6011]
2025-10-09T21:45:54.200770Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341530761752513:6011]
2025-10-09T21:45:54.200775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341530761752513:6011]
2025-10-09T21:45:54.200827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:54.200835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.200837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:54.200837Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:54.200845Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:54.200849Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:54.201004Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:54.206755Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341531123566191:6527]
2025-10-09T21:45:54.207308Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 611, pipe: [1:7559341531123566191:6527]
2025-10-09T21:45:54.218461Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 612 sessionId: src_id|99c017f7-21e0408e-5a89d89c-dfac284b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:54.220957Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:54.384564Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:54.384901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341531123566191:6527], now have 1 active actors on pipe
2025-10-09T21:45:54.384969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:54.384978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:54.385734Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99c017f7-21e0408e-5a89d89c-dfac284b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:54.385746Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99c017f7-21e0408e-5a89d89c-dfac284b_0] Write session will restart in 0.008227s
2025-10-09T21:45:54.385780Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99c017f7-21e0408e-5a89d89c-dfac284b_0] Write session: Do CDS request
2025-10-09T21:45:54.385786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99c017f7-21e0408e-5a89d89c-dfac284b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:54.385281Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:54.385297Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:54.385301Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:54.385349Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341531123566191:6527]
2025-10-09T21:45:54.385369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 612 reason: error at write: Disconnected sessionId: src_id|99c017f7-21e0408e-5a89d89c-dfac284b_0
2025-10-09T21:45:54.385571Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 612 sessionId: src_id|99c017f7-21e0408e-5a89d89c-dfac284b_0 is DEAD
2025-10-09T21:45:54.385619Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:54.397714Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99c017f7-21e0408e-5a89d89c-dfac284b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:54.398203Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:54.398237Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 613
2025-10-09T21:45:54.398575Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 613 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:54.398633Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 613 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:54.398640Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=613 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:54.398644Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:54.399351Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 613 sessionId:  describe result for acl check
2025-10-09T21:45:54.399384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:54.399386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:54.399387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:54.399402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341531123566235:6604] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:54.399996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341531123566235:6604] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:54.404103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341531123566235:6604] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:54.409946Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341531123566235:6604] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:54.409965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341531123566235:6604] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:54.409969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341531123566235:6604] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:54.409981Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 613 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:54.411940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:54.416878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:54.417171Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:54.417176Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:54.417190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:54.417257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:54.417297Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341530761752708:6013]
2025-10-09T21:45:54.417906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:54.418308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:54.418389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:54.418788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:54.418864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:54.418866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:54.418867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:54.418868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:54.418869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:54.418870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:54.418871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.418873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.418875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.418880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:54.418892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:54.418895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:54.418899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:54.418903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:54.418907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:54.418910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.418913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.418917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.418927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:54.418936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:54.418942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:54.418947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:54.418954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:54.418960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:54.418965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:54.418970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:54.418975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:54.418980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:54.418984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:54.418988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:54.418993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.419005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.419010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.419014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.419019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.419024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.419052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:54.429375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:54.431940Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 612
2025-10-09T21:45:54.431954Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 612
2025-10-09T21:45:54.430698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:54.430718Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:54.430722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:54.430726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:54.430734Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 612 [2:7559341530761752708:6013]
2025-10-09T21:45:54.430751Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:54.430775Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:54.430778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:54.430782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:54.430785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:54.430805Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:54.430812Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:54.430821Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:54.431622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.431626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.431629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.431631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:54.431733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:54.431736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.431751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.431762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.431779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:54.431803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341530761752703:6013]
2025-10-09T21:45:54.431807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341530761752703:6013]
2025-10-09T21:45:54.431809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341530761752703:6013]
2025-10-09T21:45:54.431828Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:54.431845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341531123566371:6604], now have 1 active actors on pipe
2025-10-09T21:45:54.431849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341531123566194:2432], now have 1 active actors on pipe
2025-10-09T21:45:54.431859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:54.431875Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:54.431879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:54.431920Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341530761752703:6013]
2025-10-09T21:45:54.431923Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:54.431985Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:54.431990Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:54.431992Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:54.432002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:54.432126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:54.432134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:54.432183Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|da1c1aa7-4164cded-e177d87f-71902296_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:54.432193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:54.432199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:54.432202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:54.432217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:54.432230Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:54.432263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:54.432696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:54.432710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:54.432731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:54.432826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 613 partition: 0 MaxSeqNo: 25 sessionId: src_id|da1c1aa7-4164cded-e177d87f-71902296_0
2025-10-09T21:45:54.433258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99c017f7-21e0408e-5a89d89c-dfac284b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|da1c1aa7-4164cded-e177d87f-71902296_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:54.436036Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da1c1aa7-4164cded-e177d87f-71902296_0] Write session: try to update token
2025-10-09T21:45:54.436058Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da1c1aa7-4164cded-e177d87f-71902296_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:54.449012Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:54.449033Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:54.449036Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:54.449038Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:54.449048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341530761752703:6013]
2025-10-09T21:45:54.449053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341530761752703:6013]
2025-10-09T21:45:54.449057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341530761752703:6013]
2025-10-09T21:45:54.449103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:54.449110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.449112Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:54.449119Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:54.449122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:54.449123Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:54.449267Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:54.455058Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 613 sessionId: src_id|da1c1aa7-4164cded-e177d87f-71902296_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:54.457722Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:54.485252Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341531123566386:6527]
2025-10-09T21:45:54.485744Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 612, pipe: [1:7559341531123566386:6527]
2025-10-09T21:45:54.606727Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:54.607010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:54.607027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:54.607074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341531123566386:6527], now have 1 active actors on pipe
2025-10-09T21:45:54.607745Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341531123566386:6527]
2025-10-09T21:45:54.607784Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:54.607853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 613 reason: status is not ok: tablet will be restarted right now sessionId: src_id|da1c1aa7-4164cded-e177d87f-71902296_0
2025-10-09T21:45:54.608035Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 613 sessionId: src_id|da1c1aa7-4164cded-e177d87f-71902296_0 is DEAD
2025-10-09T21:45:54.608862Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da1c1aa7-4164cded-e177d87f-71902296_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:54.608874Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da1c1aa7-4164cded-e177d87f-71902296_0] Write session will restart in 0.008969s
2025-10-09T21:45:54.608899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da1c1aa7-4164cded-e177d87f-71902296_0] Write session: Do CDS request
2025-10-09T21:45:54.608906Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da1c1aa7-4164cded-e177d87f-71902296_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:54.608166Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:54.607380Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:54.622892Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da1c1aa7-4164cded-e177d87f-71902296_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:54.623355Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:54.623374Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 614
2025-10-09T21:45:54.623560Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 614 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:54.623591Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 614 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:54.623608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=614 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:54.623612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:54.624317Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 614 sessionId:  describe result for acl check
2025-10-09T21:45:54.624351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:54.624354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:54.624356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:54.624362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341531123566430:6608] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:54.624854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341531123566430:6608] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:54.632158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341531123566430:6608] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:54.639417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341531123566430:6608] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:54.639442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341531123566430:6608] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:54.639447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341531123566430:6608] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:54.639458Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 614 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:54.645228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:54.649220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:54.649829Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:54.649834Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:54.649850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:54.649928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:54.649973Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341530761752893:6017]
2025-10-09T21:45:54.650576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:54.650986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:54.651040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:54.651354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:54.651407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:54.651408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:54.651410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:54.651411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:54.651412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:54.651413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:54.651415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.651416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.651418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.651424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:54.651436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:54.651440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:54.651444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:54.651449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:54.651452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:54.651456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.651460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.651464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.651472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:54.651481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:54.651486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:54.651492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:54.651497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:54.651502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:54.651507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:54.651512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:54.651517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:54.651522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:54.651527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:54.651533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:54.651538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.651543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.651547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.651553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.651558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.651563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.651592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:54.658955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:54.660024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:54.660030Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:54.660033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:54.660037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:54.660046Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 613 [2:7559341530761752893:6017]
2025-10-09T21:45:54.660063Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:54.660087Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:54.660090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:54.660094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:54.660096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:54.660115Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:54.660121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:54.660129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:54.660197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.660201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.660203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.660207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:54.660290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:54.660293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.660308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.660319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.660335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:54.660356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341530761752887:6017]
2025-10-09T21:45:54.660359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341530761752887:6017]
2025-10-09T21:45:54.660362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341530761752887:6017]
2025-10-09T21:45:54.660377Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:54.660396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341531123566562:6608], now have 1 active actors on pipe
2025-10-09T21:45:54.660399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341531123566389:2432], now have 1 active actors on pipe
2025-10-09T21:45:54.660408Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:54.660425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:54.660429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:54.660471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341530761752887:6017]
2025-10-09T21:45:54.660473Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:54.660613Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:54.660618Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:54.660621Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:54.660628Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:54.660773Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 613
2025-10-09T21:45:54.660776Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 613
2025-10-09T21:45:54.661210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:54.661228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:54.661261Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|eb44beab-ee4b2c0b-47114d20-898a88ff_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:54.661272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:54.661276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:54.661280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:54.661294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:54.661316Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:54.661347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:54.662133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:54.662149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:54.662174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:54.662466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 614 partition: 0 MaxSeqNo: 25 sessionId: src_id|eb44beab-ee4b2c0b-47114d20-898a88ff_0
2025-10-09T21:45:54.663761Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da1c1aa7-4164cded-e177d87f-71902296_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|eb44beab-ee4b2c0b-47114d20-898a88ff_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:54.666574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eb44beab-ee4b2c0b-47114d20-898a88ff_0] Write session: try to update token
2025-10-09T21:45:54.666599Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eb44beab-ee4b2c0b-47114d20-898a88ff_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:54.689328Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:54.689357Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:54.689360Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:54.689362Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:54.689371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341530761752887:6017]
2025-10-09T21:45:54.689375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341530761752887:6017]
2025-10-09T21:45:54.689379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341530761752887:6017]
2025-10-09T21:45:54.689422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:54.689429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.689432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:54.689433Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:54.689439Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:54.689443Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:54.689566Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:54.690096Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 614 sessionId: src_id|eb44beab-ee4b2c0b-47114d20-898a88ff_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:54.692696Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:54.707957Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341531123566585:6527]
2025-10-09T21:45:54.708436Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 613, pipe: [1:7559341531123566585:6527]
2025-10-09T21:45:54.917394Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:54.917957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:54.917975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:54.918022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341531123566585:6527], now have 1 active actors on pipe
2025-10-09T21:45:54.918292Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:54.918778Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341531123566585:6527]
2025-10-09T21:45:54.918818Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:54.918860Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:54.918914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 614 reason: status is not ok: tablet will be restarted right now sessionId: src_id|eb44beab-ee4b2c0b-47114d20-898a88ff_0
2025-10-09T21:45:54.919105Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 614 sessionId: src_id|eb44beab-ee4b2c0b-47114d20-898a88ff_0 is DEAD
2025-10-09T21:45:54.922298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb44beab-ee4b2c0b-47114d20-898a88ff_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:54.922312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb44beab-ee4b2c0b-47114d20-898a88ff_0] Write session will restart in 0.006212s
2025-10-09T21:45:54.922341Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb44beab-ee4b2c0b-47114d20-898a88ff_0] Write session: Do CDS request
2025-10-09T21:45:54.922347Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb44beab-ee4b2c0b-47114d20-898a88ff_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:54.929545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eb44beab-ee4b2c0b-47114d20-898a88ff_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:54.930152Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:54.930173Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 615
2025-10-09T21:45:54.930405Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 615 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:54.930442Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 615 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:54.930449Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=615 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:54.930453Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:54.931098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 615 sessionId:  describe result for acl check
2025-10-09T21:45:54.931136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:54.931138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:54.931140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:54.931147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341531123566605:6613] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:54.932082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341531123566605:6613] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:54.937125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341531123566605:6613] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:54.943392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341531123566605:6613] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:54.943415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341531123566605:6613] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:54.943419Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341531123566605:6613] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:54.943429Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 615 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:54.957018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:54.962387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:54.962673Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:54.962683Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:54.962697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:54.962783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:54.962838Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341530761753081:6019]
2025-10-09T21:45:54.963430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:54.966271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:54.966344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:54.966717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:54.966815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:54.966823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:54.966825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:54.966826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:54.966827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:54.966829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:54.966830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.966831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.966833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.966840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:54.966851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:54.966855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:54.966860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:54.966864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:54.966868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:54.966872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.966875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.966879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.966887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:54.966896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:54.966903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:54.966908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:54.966913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:54.966919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:54.966923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:54.966929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:54.966933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:54.966938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:54.966943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:54.966948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:54.966953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.966959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.966964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.966968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.966974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.966979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.967006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:54.974311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:54.976920Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 614
2025-10-09T21:45:54.976965Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 614
2025-10-09T21:45:54.976243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:54.976262Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:54.976265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:54.976270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:54.976279Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 614 [2:7559341530761753081:6019]
2025-10-09T21:45:54.976294Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:54.976316Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:54.976320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:54.976324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:54.976326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:54.976346Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:54.976352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:54.976359Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:54.976433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.976435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.976437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.976439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:54.976523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:54.976526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.976541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:54.976553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:54.976569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:54.976591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341531123566593:2432], now have 1 active actors on pipe
2025-10-09T21:45:54.976595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341531123566687:6613], now have 1 active actors on pipe
2025-10-09T21:45:54.976603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341530761753075:6019]
2025-10-09T21:45:54.976605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341530761753075:6019]
2025-10-09T21:45:54.976607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341530761753075:6019]
2025-10-09T21:45:54.976625Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:54.976642Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:54.976643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:54.976648Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:54.976650Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:54.976657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:54.976660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:54.976703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341530761753075:6019]
2025-10-09T21:45:54.976712Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:54.976729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:54.977307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:54.977320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:54.977351Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b46b9155-593e30a3-30c874a5-275bd35b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:54.977360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:54.977364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:54.977366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:54.977378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:54.977390Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:54.977407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:54.977740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:54.977746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:54.977761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:54.977932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 615 partition: 0 MaxSeqNo: 25 sessionId: src_id|b46b9155-593e30a3-30c874a5-275bd35b_0
2025-10-09T21:45:54.978401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb44beab-ee4b2c0b-47114d20-898a88ff_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b46b9155-593e30a3-30c874a5-275bd35b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:54.981021Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b46b9155-593e30a3-30c874a5-275bd35b_0] Write session: try to update token
2025-10-09T21:45:54.981041Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b46b9155-593e30a3-30c874a5-275bd35b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:54.994657Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:54.994679Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:54.994681Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:54.994683Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:54.994694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341530761753075:6019]
2025-10-09T21:45:54.994698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341530761753075:6019]
2025-10-09T21:45:54.994702Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341530761753075:6019]
2025-10-09T21:45:54.994757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:54.994757Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:54.994764Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:54.994766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:54.994768Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:54.994769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:54.994910Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:55.012080Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 615 sessionId: src_id|b46b9155-593e30a3-30c874a5-275bd35b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:55.014907Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:55.018748Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341535418534100:6527]
2025-10-09T21:45:55.019267Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 614, pipe: [1:7559341535418534100:6527]
2025-10-09T21:45:55.173253Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:55.173912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:55.173932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:55.173956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341535418534100:6527], now have 1 active actors on pipe
2025-10-09T21:45:55.174320Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:55.174788Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341535418534100:6527]
2025-10-09T21:45:55.174842Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:55.174887Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:55.174969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 615 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b46b9155-593e30a3-30c874a5-275bd35b_0
2025-10-09T21:45:55.175232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 615 sessionId: src_id|b46b9155-593e30a3-30c874a5-275bd35b_0 is DEAD
2025-10-09T21:45:55.175392Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b46b9155-593e30a3-30c874a5-275bd35b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:55.175409Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b46b9155-593e30a3-30c874a5-275bd35b_0] Write session will restart in 0.007904s
2025-10-09T21:45:55.175441Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b46b9155-593e30a3-30c874a5-275bd35b_0] Write session: Do CDS request
2025-10-09T21:45:55.175448Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b46b9155-593e30a3-30c874a5-275bd35b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:55.187486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b46b9155-593e30a3-30c874a5-275bd35b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:55.187903Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:55.187933Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 616
2025-10-09T21:45:55.188179Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 616 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:55.188228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 616 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:55.188239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=616 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:55.188255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:55.188792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 616 sessionId:  describe result for acl check
2025-10-09T21:45:55.188826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:55.188828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:55.188830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:55.188837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341535418534134:6625] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:55.189513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341535418534134:6625] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:55.195035Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341535418534134:6625] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:55.203699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341535418534134:6625] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:55.203722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341535418534134:6625] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:55.203728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341535418534134:6625] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:55.203738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 616 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:55.209390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:55.210338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:55.210585Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:55.210589Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:55.210604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:55.210685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:55.210726Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341535056720603:6033]
2025-10-09T21:45:55.211388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:55.214242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:55.214307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:55.214689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:55.214746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:55.214748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:55.214749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:55.214750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:55.214751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:55.214752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:55.214753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.214754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.214755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.214761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:55.214771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:55.214774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:55.214778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:55.214781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:55.214784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:55.214787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.214791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.214794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.214804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:55.214812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:55.214818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:55.214823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:55.214828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:55.214833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:55.214838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:55.214843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:55.214847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:55.214852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:55.214856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:55.214861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:55.214866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.214870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.214873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.214878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.214882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.214886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.214912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:55.223364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:55.224906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:55.224924Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:55.224927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:55.224933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:55.224941Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 615 [2:7559341535056720603:6033]
2025-10-09T21:45:55.224957Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:55.224977Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:55.224986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:55.224990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:55.224992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:55.225013Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:55.225027Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:55.225035Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:55.225101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.225104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.225106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.225109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:55.225189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:55.225198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.225215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.225234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.225259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:55.225284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341535418534285:6625], now have 1 active actors on pipe
2025-10-09T21:45:55.225288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341535418534119:2432], now have 1 active actors on pipe
2025-10-09T21:45:55.225297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341535056720597:6033]
2025-10-09T21:45:55.225300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341535056720597:6033]
2025-10-09T21:45:55.225302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341535056720597:6033]
2025-10-09T21:45:55.225316Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:55.225483Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 615
2025-10-09T21:45:55.225537Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 615
2025-10-09T21:45:55.225334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:55.225357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:55.225360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:55.225368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:55.225373Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:55.225375Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:55.225404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341535056720597:6033]
2025-10-09T21:45:55.225412Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:55.225419Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:55.225740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:55.225748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:55.225792Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|366e0b2-6d3380cd-3b43e241-71e190cd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:55.225800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:55.225805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:55.225816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:55.225830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:55.225843Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:55.225863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:55.226128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:55.226144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:55.226179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:55.226338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 616 partition: 0 MaxSeqNo: 25 sessionId: src_id|366e0b2-6d3380cd-3b43e241-71e190cd_0
2025-10-09T21:45:55.226667Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b46b9155-593e30a3-30c874a5-275bd35b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|366e0b2-6d3380cd-3b43e241-71e190cd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:55.229276Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|366e0b2-6d3380cd-3b43e241-71e190cd_0] Write session: try to update token
2025-10-09T21:45:55.229298Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|366e0b2-6d3380cd-3b43e241-71e190cd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:55.243958Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:55.243984Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:55.243987Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:55.243989Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:55.243999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341535056720597:6033]
2025-10-09T21:45:55.244004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341535056720597:6033]
2025-10-09T21:45:55.244009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341535056720597:6033]
2025-10-09T21:45:55.244053Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:55.244058Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:55.244061Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:55.244061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:55.244071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.244074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:55.244212Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:55.247828Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 616 sessionId: src_id|366e0b2-6d3380cd-3b43e241-71e190cd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:55.250463Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:55.279412Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341535418534314:6527]
2025-10-09T21:45:55.279844Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 615, pipe: [1:7559341535418534314:6527]
2025-10-09T21:45:55.424645Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:55.424668Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341535418534314:6527]
2025-10-09T21:45:55.424693Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:55.424698Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:55.424769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 616 reason: error at write: Disconnected sessionId: src_id|366e0b2-6d3380cd-3b43e241-71e190cd_0
2025-10-09T21:45:55.423160Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:55.423894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:55.423907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:55.423956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341535418534314:6527], now have 1 active actors on pipe
2025-10-09T21:45:55.424401Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:55.424963Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 616 sessionId: src_id|366e0b2-6d3380cd-3b43e241-71e190cd_0 is DEAD
2025-10-09T21:45:55.425118Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|366e0b2-6d3380cd-3b43e241-71e190cd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:55.425130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|366e0b2-6d3380cd-3b43e241-71e190cd_0] Write session will restart in 0.005229s
2025-10-09T21:45:55.425159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|366e0b2-6d3380cd-3b43e241-71e190cd_0] Write session: Do CDS request
2025-10-09T21:45:55.425175Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|366e0b2-6d3380cd-3b43e241-71e190cd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:55.431576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|366e0b2-6d3380cd-3b43e241-71e190cd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:55.431693Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:55.431720Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 617
2025-10-09T21:45:55.434580Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 617 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:55.434636Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 617 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:55.434642Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=617 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:55.434646Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:55.435140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 617 sessionId:  describe result for acl check
2025-10-09T21:45:55.435183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:55.435185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:55.435187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:55.435193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341535418534331:6629] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:55.435848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341535418534331:6629] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:55.443689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341535418534331:6629] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:55.447961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341535418534331:6629] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:55.447997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341535418534331:6629] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:55.448001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341535418534331:6629] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:55.448012Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 617 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:55.452333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:55.457447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:55.458052Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:55.458067Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:55.458093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:55.458189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:55.458232Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341535056720796:6037]
2025-10-09T21:45:55.459008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:55.459440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:55.459495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:55.459837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:55.459896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:55.459897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:55.459899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:55.459900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:55.459901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:55.459903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:55.459904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.459906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.459907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.459913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:55.459925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:55.459930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:55.459933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:55.459937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:55.459941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:55.459945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.459949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.459952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.459961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:55.459969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:55.459993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:55.459999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:55.460004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:55.460010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:55.460015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:55.460020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:55.460026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:55.460031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:55.460035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:55.460041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:55.460047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.460052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.460057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.460062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.460067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.460073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.460104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:55.468578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:55.470387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:55.470404Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:55.470407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:55.470412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:55.470420Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 616 [2:7559341535056720796:6037]
2025-10-09T21:45:55.470438Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:55.470459Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:55.470468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:55.470471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:55.470473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:55.470530Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:55.470536Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:55.470544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:55.470624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.470633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.470638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.470641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:55.470727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:55.470736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.470750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.470762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.470780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:55.470803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341535056720790:6037]
2025-10-09T21:45:55.470806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341535056720790:6037]
2025-10-09T21:45:55.470808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341535056720790:6037]
2025-10-09T21:45:55.470823Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:55.470846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:55.470861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:55.470865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:55.470867Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:55.470869Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:55.470874Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:55.470906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341535056720790:6037]
2025-10-09T21:45:55.470910Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:55.471391Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 616
2025-10-09T21:45:55.471430Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 616
2025-10-09T21:45:55.470981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:55.471877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341535418534495:6629], now have 1 active actors on pipe
2025-10-09T21:45:55.471887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341535418534317:2432], now have 1 active actors on pipe
2025-10-09T21:45:55.471905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:55.471924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:55.471960Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2ed22367-eb8f7666-775bdad5-83376672_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:55.471977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:55.471982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:55.471985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:55.471999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:55.472021Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:55.472045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:55.472263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:55.472272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:55.472306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:55.472461Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 617 partition: 0 MaxSeqNo: 25 sessionId: src_id|2ed22367-eb8f7666-775bdad5-83376672_0
2025-10-09T21:45:55.472923Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|366e0b2-6d3380cd-3b43e241-71e190cd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2ed22367-eb8f7666-775bdad5-83376672_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:55.475552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed22367-eb8f7666-775bdad5-83376672_0] Write session: try to update token
2025-10-09T21:45:55.475574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed22367-eb8f7666-775bdad5-83376672_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:55.495850Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:55.495868Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:55.495870Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:55.495872Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:55.495880Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341535056720790:6037]
2025-10-09T21:45:55.495885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341535056720790:6037]
2025-10-09T21:45:55.495888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341535056720790:6037]
2025-10-09T21:45:55.495932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:55.495940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.495943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:55.496073Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:55.499105Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:55.499135Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:55.499139Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:55.498415Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 617 sessionId: src_id|2ed22367-eb8f7666-775bdad5-83376672_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:55.500949Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:55.524114Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341535418534514:6527]
2025-10-09T21:45:55.526614Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 616, pipe: [1:7559341535418534514:6527]
2025-10-09T21:45:55.696092Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:55.696401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:55.696419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:55.696467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341535418534514:6527], now have 1 active actors on pipe
2025-10-09T21:45:55.697480Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:55.697085Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341535418534514:6527]
2025-10-09T21:45:55.697126Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:55.697128Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:55.697199Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 617 reason: error at write: Disconnected sessionId: src_id|2ed22367-eb8f7666-775bdad5-83376672_0
2025-10-09T21:45:55.697402Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 617 sessionId: src_id|2ed22367-eb8f7666-775bdad5-83376672_0 is DEAD
2025-10-09T21:45:55.697615Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:55.698924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed22367-eb8f7666-775bdad5-83376672_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:55.698937Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed22367-eb8f7666-775bdad5-83376672_0] Write session will restart in 0.006549s
2025-10-09T21:45:55.698966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed22367-eb8f7666-775bdad5-83376672_0] Write session: Do CDS request
2025-10-09T21:45:55.698972Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed22367-eb8f7666-775bdad5-83376672_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:55.707247Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed22367-eb8f7666-775bdad5-83376672_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:55.710446Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:55.710468Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 618
2025-10-09T21:45:55.714206Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 618 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:55.714270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 618 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:55.714282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=618 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:55.714286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:55.715178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 618 sessionId:  describe result for acl check
2025-10-09T21:45:55.715225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:55.715227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:55.715229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:55.715253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341535418534611:6634] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:55.715855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341535418534611:6634] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:55.722970Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341535418534611:6634] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:55.729506Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341535418534611:6634] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:55.729541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341535418534611:6634] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:55.729545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341535418534611:6634] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:55.729555Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 618 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:55.729611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:55.733815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:55.734128Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:55.734133Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:55.734148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:55.734231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:55.734273Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341535056720985:6039]
2025-10-09T21:45:55.734889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:55.735319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:55.735379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:55.735728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:55.735786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:55.735793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:55.735795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:55.735796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:55.735797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:55.735799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:55.735800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.735801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.735803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.735809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:55.735821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:55.735826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:55.735830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:55.735834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:55.735839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:55.735844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.735848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.735852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.735860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:55.735876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:55.735887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:55.735897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:55.735902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:55.735908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:55.735912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:55.735917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:55.735923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:55.735929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:55.735934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:55.735945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:55.735952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.735961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.735968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.735977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.735982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.735989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.736021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:55.743642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:55.744885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:55.744907Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:55.744910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:55.744915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:55.744926Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 617 [2:7559341535056720985:6039]
2025-10-09T21:45:55.744944Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:55.744966Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:55.744970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:55.744973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:55.744984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:55.745008Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:55.745021Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:55.745029Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:55.745110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.745117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.745120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.745122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:55.745197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:55.745208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.745220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.745245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.745262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:55.745292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341535418534519:2432], now have 1 active actors on pipe
2025-10-09T21:45:55.745302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341535418534703:6634], now have 1 active actors on pipe
2025-10-09T21:45:55.745311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341535056720979:6039]
2025-10-09T21:45:55.745313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341535056720979:6039]
2025-10-09T21:45:55.745315Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341535056720979:6039]
2025-10-09T21:45:55.745332Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:55.745348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:55.745364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:55.745366Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:55.745371Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:55.745372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:55.745373Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:55.745417Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341535056720979:6039]
2025-10-09T21:45:55.745432Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:55.745411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 617
2025-10-09T21:45:55.745422Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 617
2025-10-09T21:45:55.745475Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:55.745558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:55.745576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:55.745619Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f6fcc2f2-b1a3816b-7df5e8e3-36ff06d1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:55.745635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:55.745644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:55.745647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:55.745667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:55.745686Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:55.745710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:55.746347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:55.746360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:55.746382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:55.746697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 618 partition: 0 MaxSeqNo: 25 sessionId: src_id|f6fcc2f2-b1a3816b-7df5e8e3-36ff06d1_0
2025-10-09T21:45:55.747148Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed22367-eb8f7666-775bdad5-83376672_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f6fcc2f2-b1a3816b-7df5e8e3-36ff06d1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:55.750199Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f6fcc2f2-b1a3816b-7df5e8e3-36ff06d1_0] Write session: try to update token
2025-10-09T21:45:55.750222Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f6fcc2f2-b1a3816b-7df5e8e3-36ff06d1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:55.766413Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:55.766435Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:55.766447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341535056720979:6039]
2025-10-09T21:45:55.767193Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 618 sessionId: src_id|f6fcc2f2-b1a3816b-7df5e8e3-36ff06d1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:55.766502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:55.766732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:55.766911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:55.767061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:55.767244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:55.767483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:55.767526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:55.767672Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:55.770075Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:55.770511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:55.770536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:55.770586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:55.770755Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:55.775604Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:55.775622Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:55.775625Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:55.775627Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:55.775639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341535056720979:6039]
2025-10-09T21:45:55.775644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341535056720979:6039]
2025-10-09T21:45:55.775647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341535056720979:6039]
2025-10-09T21:45:55.775696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:55.775705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.775707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:55.775856Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:55.778333Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:55.778355Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:55.778359Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:55.797189Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341535418534715:6527]
2025-10-09T21:45:55.797500Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 617, pipe: [1:7559341535418534715:6527]
2025-10-09T21:45:55.941219Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:55.941714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341535418534715:6527], now have 1 active actors on pipe
2025-10-09T21:45:55.949716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:55.949748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:55.949914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:55.949971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:55.950123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:55.950195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:55.950324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:55.950329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:55.950404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:55.950459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:55.950518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:55.950968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:55.951006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:55.951010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:55.951073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:55.951130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:55.951188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:55.951246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:55.951285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:55.951288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:55.951357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:55.951421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:55.951477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:55.951541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:55.951594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:55.951597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:55.951662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:55.951719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:55.951803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:55.951863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:55.951905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:55.951908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:55.951978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:55.952039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:55.952102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:55.952165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:55.952209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:55.952213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:55.952275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:55.952329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:55.952389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:55.952447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:55.952505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:55.952511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:55.952576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:55.952631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:55.952687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:55.952745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:55.952783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:55.952786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:55.952850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:55.952906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:55.952962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:55.953019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:55.953060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:55.953064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:55.953133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:55.953188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:55.953250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:55.953309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:55.953347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:55.953350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:55.954234Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6fcc2f2-b1a3816b-7df5e8e3-36ff06d1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:55.954247Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6fcc2f2-b1a3816b-7df5e8e3-36ff06d1_0] Write session will restart in 0.006783s
2025-10-09T21:45:55.954273Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6fcc2f2-b1a3816b-7df5e8e3-36ff06d1_0] Write session: Do CDS request
2025-10-09T21:45:55.954281Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6fcc2f2-b1a3816b-7df5e8e3-36ff06d1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:55.953782Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:55.953799Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:55.953825Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:55.953832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 618 reason: error at write: Disconnected sessionId: src_id|f6fcc2f2-b1a3816b-7df5e8e3-36ff06d1_0
2025-10-09T21:45:55.953871Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341535418534715:6527]
2025-10-09T21:45:55.954063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 618 sessionId: src_id|f6fcc2f2-b1a3816b-7df5e8e3-36ff06d1_0 is DEAD
2025-10-09T21:45:55.956535Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:55.962657Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f6fcc2f2-b1a3816b-7df5e8e3-36ff06d1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:55.963400Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:55.963420Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 619
2025-10-09T21:45:55.963800Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 619 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:55.963851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 619 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:55.963859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=619 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:55.963863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:55.964319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 619 sessionId:  describe result for acl check
2025-10-09T21:45:55.964353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:55.964355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:55.964357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:55.964363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341535418534739:6638] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:55.964928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341535418534739:6638] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:55.969492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341535418534739:6638] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:55.973612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341535418534739:6638] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:55.973628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341535418534739:6638] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:55.973633Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341535418534739:6638] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:55.973648Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 619 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:55.981191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:55.982189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:55.982426Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:55.982438Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:55.982451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:55.982514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:55.982553Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341535056721175:6041]
2025-10-09T21:45:55.983187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:55.983542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:55.983591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:55.983915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:55.984011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:55.984023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:55.984025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:55.984026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:55.984028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:55.984030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:55.984031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.984033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.984034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.984042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:55.984054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:55.984066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:55.984070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:55.984083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:55.984088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:55.984099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.984102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.984105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.984116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:55.984129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:55.984143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:55.984157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:55.984170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:55.984183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:55.984188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:55.984193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:55.984199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:55.984204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:55.984212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:55.984217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:55.984224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.984230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.984235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.984240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.984245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.984250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.984298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:55.991699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:55.993233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:55.993253Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:55.993256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:55.993261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:55.993270Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 618 [2:7559341535056721175:6041]
2025-10-09T21:45:55.993286Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:55.993306Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:55.993310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:55.993314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:55.993317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:55.993336Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:55.993351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:55.993366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:55.993435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.993443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.993445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.993448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:55.993519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:55.993527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:55.993542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:55.993560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:55.993576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:55.993606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341535418534725:2432], now have 1 active actors on pipe
2025-10-09T21:45:55.993620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341535056721170:6041]
2025-10-09T21:45:55.993622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341535056721170:6041]
2025-10-09T21:45:55.993624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341535056721170:6041]
2025-10-09T21:45:55.993641Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:55.993652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341535418534905:6638], now have 1 active actors on pipe
2025-10-09T21:45:55.993666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:55.993665Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:55.993670Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:55.993673Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:55.993682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:55.993685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:55.993728Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341535056721170:6041]
2025-10-09T21:45:55.993746Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:55.993798Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:55.993800Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 618
2025-10-09T21:45:55.993858Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 618
2025-10-09T21:45:55.994147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:55.994159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:55.994201Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6e2230d3-d3ac5850-aeb2ab1f-f381ea39_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:55.994213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:55.994223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:55.994225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:55.994250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:55.994270Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:55.994288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:55.994712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:55.994725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:55.994746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:55.994940Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 619 partition: 0 MaxSeqNo: 25 sessionId: src_id|6e2230d3-d3ac5850-aeb2ab1f-f381ea39_0
2025-10-09T21:45:55.995371Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6fcc2f2-b1a3816b-7df5e8e3-36ff06d1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6e2230d3-d3ac5850-aeb2ab1f-f381ea39_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:55.997937Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e2230d3-d3ac5850-aeb2ab1f-f381ea39_0] Write session: try to update token
2025-10-09T21:45:55.997958Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e2230d3-d3ac5850-aeb2ab1f-f381ea39_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:56.019173Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:56.019197Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:56.019199Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:56.019202Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:56.019213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341535056721170:6041]
2025-10-09T21:45:56.019219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341535056721170:6041]
2025-10-09T21:45:56.019223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341535056721170:6041]
2025-10-09T21:45:56.019271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:56.019279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.019282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:56.019284Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:56.019291Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:56.019295Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:56.019432Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:56.018328Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 619 sessionId: src_id|6e2230d3-d3ac5850-aeb2ab1f-f381ea39_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:56.020998Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:56.053547Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341539713502222:6527]
2025-10-09T21:45:56.053876Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 618, pipe: [1:7559341539713502222:6527]
2025-10-09T21:45:56.182054Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:56.182777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:56.182790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:56.182836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341539713502222:6527], now have 1 active actors on pipe
2025-10-09T21:45:56.183099Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:56.183599Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e2230d3-d3ac5850-aeb2ab1f-f381ea39_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:56.183612Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e2230d3-d3ac5850-aeb2ab1f-f381ea39_0] Write session will restart in 0.009533s
2025-10-09T21:45:56.183638Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e2230d3-d3ac5850-aeb2ab1f-f381ea39_0] Write session: Do CDS request
2025-10-09T21:45:56.183644Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e2230d3-d3ac5850-aeb2ab1f-f381ea39_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:56.183173Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:56.183191Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:56.183271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 619 reason: error at write: Disconnected sessionId: src_id|6e2230d3-d3ac5850-aeb2ab1f-f381ea39_0
2025-10-09T21:45:56.183458Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 619 sessionId: src_id|6e2230d3-d3ac5850-aeb2ab1f-f381ea39_0 is DEAD
2025-10-09T21:45:56.183521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341539713502222:6527]
2025-10-09T21:45:56.183539Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:56.194823Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e2230d3-d3ac5850-aeb2ab1f-f381ea39_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:56.195006Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:56.195026Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 620
2025-10-09T21:45:56.195213Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 620 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:56.195254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 620 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:56.195265Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=620 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:56.195269Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:56.195930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 620 sessionId:  describe result for acl check
2025-10-09T21:45:56.195973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:56.195975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:56.195977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:56.195984Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341539713502267:6650] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:56.196590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341539713502267:6650] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:56.205556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341539713502267:6650] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:56.211075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341539713502267:6650] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:56.211098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341539713502267:6650] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:56.211102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341539713502267:6650] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:56.211112Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 620 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:56.215960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:56.227486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:56.227995Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:56.228008Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:56.228025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:56.228099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:56.228148Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341539351688701:6055]
2025-10-09T21:45:56.228799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:56.230359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:56.230435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:56.230820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:56.230870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:56.230872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:56.230873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:56.230875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:56.230877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:56.230878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:56.230880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.230881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.230883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.230891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:56.230904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:56.230909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:56.230914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:56.230918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:56.230923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:56.230928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.230932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.230937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.230947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:56.230956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:56.230963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:56.230969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:56.230975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:56.230982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:56.230987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:56.230994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:56.231000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:56.231006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:56.231011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:56.231017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:56.231024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.231030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.231035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.231041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.231047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.231054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.231088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:56.241303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:56.243018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:56.243039Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:56.243042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:56.243047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:56.243055Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 619 [2:7559341539351688701:6055]
2025-10-09T21:45:56.243070Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:56.243095Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:56.243099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:56.243104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:56.243106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:56.243129Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:56.243142Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:56.243149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:56.244064Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 619
2025-10-09T21:45:56.244120Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 619
2025-10-09T21:45:56.243226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.243234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.243236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.243239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:56.243321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:56.243325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.243340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.243353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.243373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:56.243395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341539351688696:6055]
2025-10-09T21:45:56.243398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341539351688696:6055]
2025-10-09T21:45:56.243400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341539351688696:6055]
2025-10-09T21:45:56.243415Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:56.243431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:56.243448Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:56.243452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:56.243496Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341539351688696:6055]
2025-10-09T21:45:56.243499Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:56.243710Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:56.243713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:56.243715Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:56.243721Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:56.243740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341539713502434:6650], now have 1 active actors on pipe
2025-10-09T21:45:56.243749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341539713502253:2432], now have 1 active actors on pipe
2025-10-09T21:45:56.244619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:56.244635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:56.244680Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3f220f72-af7a8b10-9626c18d-6b84cead_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:56.244692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:56.244697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:56.244700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:56.244714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:56.244729Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:56.244758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:56.246070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:56.246084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:56.246126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:56.246291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 620 partition: 0 MaxSeqNo: 25 sessionId: src_id|3f220f72-af7a8b10-9626c18d-6b84cead_0
2025-10-09T21:45:56.246766Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e2230d3-d3ac5850-aeb2ab1f-f381ea39_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3f220f72-af7a8b10-9626c18d-6b84cead_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:56.249400Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3f220f72-af7a8b10-9626c18d-6b84cead_0] Write session: try to update token
2025-10-09T21:45:56.249425Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3f220f72-af7a8b10-9626c18d-6b84cead_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:56.264138Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:56.264158Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:56.264161Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:56.264164Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:56.264176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341539351688696:6055]
2025-10-09T21:45:56.264180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341539351688696:6055]
2025-10-09T21:45:56.264185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341539351688696:6055]
2025-10-09T21:45:56.264235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:56.264244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.264252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:56.264545Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:56.267590Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:56.267609Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:56.267613Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:56.282134Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 620 sessionId: src_id|3f220f72-af7a8b10-9626c18d-6b84cead_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:56.283455Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341539713502449:6527]
2025-10-09T21:45:56.283918Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 619, pipe: [1:7559341539713502449:6527]
2025-10-09T21:45:56.284816Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:56.457685Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:56.459064Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341539713502449:6527]
2025-10-09T21:45:56.459108Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:56.459111Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:56.459181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 620 reason: error at write: Disconnected sessionId: src_id|3f220f72-af7a8b10-9626c18d-6b84cead_0
2025-10-09T21:45:56.459364Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 620 sessionId: src_id|3f220f72-af7a8b10-9626c18d-6b84cead_0 is DEAD
2025-10-09T21:45:56.459651Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:56.459959Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3f220f72-af7a8b10-9626c18d-6b84cead_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:56.459968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3f220f72-af7a8b10-9626c18d-6b84cead_0] Write session will restart in 0.007328s
2025-10-09T21:45:56.460005Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3f220f72-af7a8b10-9626c18d-6b84cead_0] Write session: Do CDS request
2025-10-09T21:45:56.460012Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3f220f72-af7a8b10-9626c18d-6b84cead_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:56.458586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341539713502449:6527], now have 1 active actors on pipe
2025-10-09T21:45:56.458624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:56.458631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:56.459775Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:56.468812Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3f220f72-af7a8b10-9626c18d-6b84cead_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:56.470136Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:56.470156Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 621
2025-10-09T21:45:56.470667Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 621 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:56.470712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 621 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:56.470718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=621 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:56.470721Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:56.471816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 621 sessionId:  describe result for acl check
2025-10-09T21:45:56.471859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:56.471861Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:56.471863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:56.471869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341539713502467:6654] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:56.472476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341539713502467:6654] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:56.487941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341539713502467:6654] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:56.493712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341539713502467:6654] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:56.493736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341539713502467:6654] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:56.493741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341539713502467:6654] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:56.493751Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 621 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:56.505547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:56.510353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:56.510592Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:56.510597Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:56.510610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:56.510679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:56.510719Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341539351688899:6059]
2025-10-09T21:45:56.511290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:56.514602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:56.514672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:56.515060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:56.515105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:56.515106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:56.515108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:56.515109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:56.515110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:56.515111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:56.515113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.515115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.515117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.515124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:56.515136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:56.515141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:56.515145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:56.515149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:56.515154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:56.515159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.515164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.515168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.515176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:56.515186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:56.515194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:56.515200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:56.515205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:56.515211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:56.515216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:56.515221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:56.515226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:56.515232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:56.515237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:56.515243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:56.515248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.515255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.515260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.515266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.515271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.515277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.515310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:56.527126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:56.528566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:56.528572Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:56.528575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:56.528579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:56.528588Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 620 [2:7559341539351688899:6059]
2025-10-09T21:45:56.528606Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:56.528628Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:56.528632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:56.528635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:56.528637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:56.528655Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:56.528661Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:56.528668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:56.528734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.528736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.528738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.528741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:56.528816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:56.528819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.528833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.528846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.528864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:56.528886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341539351688894:6059]
2025-10-09T21:45:56.528891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341539351688894:6059]
2025-10-09T21:45:56.528893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341539351688894:6059]
2025-10-09T21:45:56.528910Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:56.528927Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:56.528940Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:56.528943Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:56.528988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341539351688894:6059]
2025-10-09T21:45:56.528991Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:56.529233Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:56.529244Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:56.529246Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:56.529252Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:56.529669Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 620
2025-10-09T21:45:56.529705Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 620
2025-10-09T21:45:56.530217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341539713502617:6654], now have 1 active actors on pipe
2025-10-09T21:45:56.530230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341539713502453:2432], now have 1 active actors on pipe
2025-10-09T21:45:56.530245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:56.530251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:56.530286Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|56c10e42-245b8ee2-a627e50a-3a94228_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:56.530296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:56.530302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:56.530305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:56.530317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:56.530331Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:56.530352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:56.534500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:56.534520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:56.534562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:56.534786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 621 partition: 0 MaxSeqNo: 25 sessionId: src_id|56c10e42-245b8ee2-a627e50a-3a94228_0
2025-10-09T21:45:56.535220Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3f220f72-af7a8b10-9626c18d-6b84cead_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|56c10e42-245b8ee2-a627e50a-3a94228_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:56.537712Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56c10e42-245b8ee2-a627e50a-3a94228_0] Write session: try to update token
2025-10-09T21:45:56.537733Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56c10e42-245b8ee2-a627e50a-3a94228_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:56.559871Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341539713502654:6527]
2025-10-09T21:45:56.561456Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 620, pipe: [1:7559341539713502654:6527]
2025-10-09T21:45:56.557878Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:56.557901Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:56.557904Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:56.557905Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:56.557916Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341539351688894:6059]
2025-10-09T21:45:56.557921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341539351688894:6059]
2025-10-09T21:45:56.557925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341539351688894:6059]
2025-10-09T21:45:56.557972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:56.557980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.557982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:56.558155Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:56.559947Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:56.559954Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:56.559958Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:56.571102Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 621 sessionId: src_id|56c10e42-245b8ee2-a627e50a-3a94228_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:56.573577Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:56.924484Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:56.925276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341539713502654:6527], now have 1 active actors on pipe
2025-10-09T21:45:56.925335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:56.925342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:56.925811Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341539713502654:6527]
2025-10-09T21:45:56.925831Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:56.925836Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:56.925842Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:56.925894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 621 reason: error at write: Disconnected sessionId: src_id|56c10e42-245b8ee2-a627e50a-3a94228_0
2025-10-09T21:45:56.925737Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:56.926120Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 621 sessionId: src_id|56c10e42-245b8ee2-a627e50a-3a94228_0 is DEAD
2025-10-09T21:45:56.926945Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56c10e42-245b8ee2-a627e50a-3a94228_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:56.926971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56c10e42-245b8ee2-a627e50a-3a94228_0] Write session will restart in 0.005010s
2025-10-09T21:45:56.926999Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56c10e42-245b8ee2-a627e50a-3a94228_0] Write session: Do CDS request
2025-10-09T21:45:56.927006Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56c10e42-245b8ee2-a627e50a-3a94228_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:56.933540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56c10e42-245b8ee2-a627e50a-3a94228_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:56.933677Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:56.933696Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 622
2025-10-09T21:45:56.933848Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 622 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:56.933899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 622 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:56.933911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=622 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:56.933916Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:56.934353Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 622 sessionId:  describe result for acl check
2025-10-09T21:45:56.934387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:56.934389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:56.934390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:56.934396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341539713502678:6661] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:56.935330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341539713502678:6661] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:56.939294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341539713502678:6661] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:56.944091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341539713502678:6661] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:56.944109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341539713502678:6661] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:56.944114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341539713502678:6661] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:56.944123Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 622 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:56.954287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:56.960327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:56.960722Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:56.960742Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:56.960762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:56.960855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:56.960912Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341539351689093:6061]
2025-10-09T21:45:56.961642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:56.962338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:56.962421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:56.962914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:56.963014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:56.963026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:56.963028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:56.963029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:56.963030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:56.963032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:56.963033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.963035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.963036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.963043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:56.963056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:56.963061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:56.963065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:56.963068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:56.963071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:56.963075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.963079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.963083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.963099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:56.963109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:56.963125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:56.963135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:56.963141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:56.963154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:56.963164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:56.963170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:56.963183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:56.963190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:56.963194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:56.963200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:56.963211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.963215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.963226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.963231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.963236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.963245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.963273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:56.971600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:56.973249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:56.973269Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:56.973271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:56.973275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:56.973287Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 621 [2:7559341539351689093:6061]
2025-10-09T21:45:56.973302Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:56.973324Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:56.973327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:56.973330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:56.973332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:56.973350Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:56.973362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:56.973370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:56.973445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.973454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.973456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.973459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:56.973532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:56.973539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.973554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:56.973565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:56.973585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:56.973607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341539351689088:6061]
2025-10-09T21:45:56.973609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341539351689088:6061]
2025-10-09T21:45:56.973611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341539351689088:6061]
2025-10-09T21:45:56.973620Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:56.973636Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:56.973652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:56.973655Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:56.973696Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341539351689088:6061]
2025-10-09T21:45:56.973698Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:56.973700Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:56.973702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:56.973705Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:56.973711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:56.973728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341539713502791:6661], now have 1 active actors on pipe
2025-10-09T21:45:56.973734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341539713502665:2432], now have 1 active actors on pipe
2025-10-09T21:45:56.974647Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 621
2025-10-09T21:45:56.974685Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 621
2025-10-09T21:45:56.974831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:56.974857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:56.974901Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ed42aaff-3b66932c-5f4d2f3-1aa19f48_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:56.974928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:56.974934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:56.974937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:56.974950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:56.974965Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:56.975004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:56.975555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:56.975576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:56.975597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:56.975706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 622 partition: 0 MaxSeqNo: 25 sessionId: src_id|ed42aaff-3b66932c-5f4d2f3-1aa19f48_0
2025-10-09T21:45:56.976152Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56c10e42-245b8ee2-a627e50a-3a94228_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ed42aaff-3b66932c-5f4d2f3-1aa19f48_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:56.978648Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ed42aaff-3b66932c-5f4d2f3-1aa19f48_0] Write session: try to update token
2025-10-09T21:45:56.978667Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ed42aaff-3b66932c-5f4d2f3-1aa19f48_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:56.991096Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:56.991118Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:56.991121Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:56.991124Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:56.991135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341539351689088:6061]
2025-10-09T21:45:56.991139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341539351689088:6061]
2025-10-09T21:45:56.991143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341539351689088:6061]
2025-10-09T21:45:56.991189Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:56.991195Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:56.991199Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:56.991201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:56.991210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:56.991215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:56.991374Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:56.999606Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 622 sessionId: src_id|ed42aaff-3b66932c-5f4d2f3-1aa19f48_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:57.002248Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:57.027087Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341544008470166:6527]
2025-10-09T21:45:57.032387Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 621, pipe: [1:7559341544008470166:6527]
2025-10-09T21:45:57.206840Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:57.207417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:57.207428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:57.207477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341544008470166:6527], now have 1 active actors on pipe
2025-10-09T21:45:57.210731Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:57.210801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:57.210808Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:57.211787Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed42aaff-3b66932c-5f4d2f3-1aa19f48_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:57.211800Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed42aaff-3b66932c-5f4d2f3-1aa19f48_0] Write session will restart in 0.008781s
2025-10-09T21:45:57.211824Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed42aaff-3b66932c-5f4d2f3-1aa19f48_0] Write session: Do CDS request
2025-10-09T21:45:57.211831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed42aaff-3b66932c-5f4d2f3-1aa19f48_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:57.210867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 622 reason: error at write: Disconnected sessionId: src_id|ed42aaff-3b66932c-5f4d2f3-1aa19f48_0
2025-10-09T21:45:57.211066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 622 sessionId: src_id|ed42aaff-3b66932c-5f4d2f3-1aa19f48_0 is DEAD
2025-10-09T21:45:57.211215Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341544008470166:6527]
2025-10-09T21:45:57.211190Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:57.221598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ed42aaff-3b66932c-5f4d2f3-1aa19f48_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:57.221908Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:57.221938Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 623
2025-10-09T21:45:57.222170Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 623 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:57.222215Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 623 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:57.222221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=623 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:57.222226Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:57.222678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 623 sessionId:  describe result for acl check
2025-10-09T21:45:57.222717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:57.222720Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:57.222722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:57.222729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341544008470212:6673] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:57.223360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341544008470212:6673] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:57.229193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341544008470212:6673] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:57.236509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341544008470212:6673] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:57.236540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341544008470212:6673] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:57.236544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341544008470212:6673] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:57.236556Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 623 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:57.258468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:57.261004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:57.261285Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:57.261290Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:57.261303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:57.261364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:57.261406Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341543646656622:6069]
2025-10-09T21:45:57.262022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:57.262567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:57.262653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:57.263004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:57.263085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:57.263093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:57.263094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:57.263096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:57.263097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:57.263099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:57.263100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.263101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.263102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.263109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:57.263135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:57.263144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:57.263147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:57.263151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:57.263155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:57.263158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.263162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.263165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.263174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:57.263183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:57.263189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:57.263195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:57.263199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:57.263205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:57.263208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:57.263213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:57.263218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:57.263223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:57.263236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:57.263241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:57.263246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.263251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.263255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.263260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.263268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.263275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.263302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:57.274911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:57.276461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:57.276470Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:57.276472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:57.276478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:57.276485Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 622 [2:7559341543646656622:6069]
2025-10-09T21:45:57.276502Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:57.276523Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:57.276526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:57.276529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:57.276532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:57.276552Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:57.276558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:57.276567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:57.276643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.276645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.276647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.276649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:57.276745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:57.276748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.276765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.276775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.276790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:57.276811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341543646656616:6069]
2025-10-09T21:45:57.276814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341543646656616:6069]
2025-10-09T21:45:57.276816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341543646656616:6069]
2025-10-09T21:45:57.276832Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:57.276848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:57.276862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:57.276865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:57.276911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341543646656616:6069]
2025-10-09T21:45:57.276914Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:57.277205Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:57.277215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:57.277217Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:57.277223Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:57.277886Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 622
2025-10-09T21:45:57.277939Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 622
2025-10-09T21:45:57.278666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341544008470187:2432], now have 1 active actors on pipe
2025-10-09T21:45:57.278682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341544008470306:6673], now have 1 active actors on pipe
2025-10-09T21:45:57.278711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:57.278718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:57.278765Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4a7bfc3f-f5d7edc0-82bb6125-eba6c87e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:57.278781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:57.278786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:57.278790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:57.278803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:57.278821Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:57.278839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:57.282738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:57.282761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:57.282802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:57.286575Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 623 partition: 0 MaxSeqNo: 25 sessionId: src_id|4a7bfc3f-f5d7edc0-82bb6125-eba6c87e_0
2025-10-09T21:45:57.287352Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed42aaff-3b66932c-5f4d2f3-1aa19f48_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4a7bfc3f-f5d7edc0-82bb6125-eba6c87e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:57.290749Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a7bfc3f-f5d7edc0-82bb6125-eba6c87e_0] Write session: try to update token
2025-10-09T21:45:57.290775Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a7bfc3f-f5d7edc0-82bb6125-eba6c87e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:57.303992Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:57.304010Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:57.304022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341543646656616:6069]
2025-10-09T21:45:57.304077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:57.304263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:57.304410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:57.304556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:57.304707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:57.304932Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:57.305163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:57.305230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:57.310548Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 623 sessionId: src_id|4a7bfc3f-f5d7edc0-82bb6125-eba6c87e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:57.311480Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341544008470398:6527]
2025-10-09T21:45:57.312118Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 622, pipe: [1:7559341544008470398:6527]
2025-10-09T21:45:57.312100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341544008470398:6527], now have 1 active actors on pipe
2025-10-09T21:45:57.313147Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:57.313502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:57.313517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:57.313565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:57.313691Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:57.318525Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:57.318546Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:57.318548Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:57.318550Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:57.318563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341543646656616:6069]
2025-10-09T21:45:57.318568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341543646656616:6069]
2025-10-09T21:45:57.318571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341543646656616:6069]
2025-10-09T21:45:57.318628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:57.318636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.318639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:57.318789Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:57.318990Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:57.318997Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:57.319001Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:57.486048Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:57.494594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:57.494622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:57.494774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:57.494829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:57.494887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:57.494948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:57.494994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:57.495003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:57.495069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:57.495163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:57.495230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:57.495300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:57.495346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:57.495354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:57.495418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:57.495482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:57.495547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:57.495609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:57.495648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:57.495653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:57.495713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:57.495767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:57.495831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:57.495897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:57.495937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:57.495943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:57.496005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:57.496059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:57.496115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:57.496175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:57.496245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:57.496251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:57.496309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:57.496364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:57.496421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:57.496481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:57.496526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:57.496532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:57.496589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:57.496638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:57.496947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:57.497008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:57.497051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:57.497060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:57.497122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:57.497182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:57.497246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:57.497313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:57.497360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:57.497369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:57.497431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:57.497494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:57.497558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:57.497618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:57.497659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:57.497662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:57.497726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:45:57.497785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:45:57.497866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:45:57.497927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:45:57.497966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:45:57.497968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:45:57.499327Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:57.499903Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:57.499967Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341544008470398:6527]
2025-10-09T21:45:57.499991Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:57.500033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 623 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4a7bfc3f-f5d7edc0-82bb6125-eba6c87e_0
2025-10-09T21:45:57.500240Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 623 sessionId: src_id|4a7bfc3f-f5d7edc0-82bb6125-eba6c87e_0 is DEAD
2025-10-09T21:45:57.500688Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a7bfc3f-f5d7edc0-82bb6125-eba6c87e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:57.500701Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a7bfc3f-f5d7edc0-82bb6125-eba6c87e_0] Write session will restart in 0.009738s
2025-10-09T21:45:57.500736Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a7bfc3f-f5d7edc0-82bb6125-eba6c87e_0] Write session: Do CDS request
2025-10-09T21:45:57.500742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a7bfc3f-f5d7edc0-82bb6125-eba6c87e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:57.511705Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a7bfc3f-f5d7edc0-82bb6125-eba6c87e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:57.512435Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:57.512453Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 624
2025-10-09T21:45:57.513215Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 624 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:57.513254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 624 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:57.513260Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=624 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:57.513266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:57.515204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 624 sessionId:  describe result for acl check
2025-10-09T21:45:57.515245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:57.515247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:57.515250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:57.515256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341544008470441:6677] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:57.515797Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341544008470441:6677] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:57.528054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341544008470441:6677] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:57.538142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:57.538794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341544008470441:6677] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:57.538814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341544008470441:6677] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:57.538818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341544008470441:6677] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:57.538828Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 624 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:57.541411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:57.541594Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:57.541598Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:57.541606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:57.541653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:57.541680Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341543646656822:6079]
2025-10-09T21:45:57.542134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:57.542478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:57.542512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:57.542750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:57.542779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:57.542780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:57.542781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:57.542782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:57.542782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:57.542783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:57.542784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.542784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.542785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.542788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:57.542796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:57.542798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:57.542800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:57.542802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:57.542805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:57.542807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.542808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.542811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.542816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:57.542821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:57.542826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:57.542829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:57.542832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:57.542836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:57.542839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:57.542842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:57.542844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:57.542847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:57.542850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:57.542853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:57.542856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.542859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.542861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.542864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.542867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.542870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.542888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:57.550802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:57.552776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:57.552791Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:57.552793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:57.552799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:57.552809Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 623 [2:7559341543646656822:6079]
2025-10-09T21:45:57.552827Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:57.552847Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:57.552851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:57.552854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:57.552857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:57.552875Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:57.552882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:57.552890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:57.552961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.552963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.552965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.552968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:57.553046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:57.553049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.553061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.553071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.553088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:57.553108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341543646656815:6079]
2025-10-09T21:45:57.553110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341543646656815:6079]
2025-10-09T21:45:57.553112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341543646656815:6079]
2025-10-09T21:45:57.553120Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:57.553133Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:57.553136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341544008470401:2432], now have 1 active actors on pipe
2025-10-09T21:45:57.553137Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:57.553139Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:57.553139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341544008470592:6677], now have 1 active actors on pipe
2025-10-09T21:45:57.553148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:57.553162Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:57.553165Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:57.553204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341543646656815:6079]
2025-10-09T21:45:57.553213Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:57.553220Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:57.553261Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 623
2025-10-09T21:45:57.553314Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 623
2025-10-09T21:45:57.553711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:57.553722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:57.553752Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1ca4de9a-9028f8f0-f76f6703-14c868da_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:57.553761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:57.553766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:57.553769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:57.553781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:57.553793Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:57.553810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:57.558326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:57.558345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:57.558391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:57.561725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 624 partition: 0 MaxSeqNo: 25 sessionId: src_id|1ca4de9a-9028f8f0-f76f6703-14c868da_0
2025-10-09T21:45:57.562173Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a7bfc3f-f5d7edc0-82bb6125-eba6c87e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1ca4de9a-9028f8f0-f76f6703-14c868da_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:57.564907Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ca4de9a-9028f8f0-f76f6703-14c868da_0] Write session: try to update token
2025-10-09T21:45:57.564928Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ca4de9a-9028f8f0-f76f6703-14c868da_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:57.572434Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:57.572451Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:57.572453Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:57.572458Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:57.572468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341543646656815:6079]
2025-10-09T21:45:57.572472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341543646656815:6079]
2025-10-09T21:45:57.572475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341543646656815:6079]
2025-10-09T21:45:57.572520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:57.572533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.572536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:57.572715Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:57.577243Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:57.577261Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:57.577269Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:57.595847Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 624 sessionId: src_id|1ca4de9a-9028f8f0-f76f6703-14c868da_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:57.598650Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:57.600251Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341544008470604:6527]
2025-10-09T21:45:57.600559Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 623, pipe: [1:7559341544008470604:6527]
2025-10-09T21:45:57.805833Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341544008470604:6527]
2025-10-09T21:45:57.805862Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:57.805902Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:57.804343Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:57.805035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:57.805045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:57.805068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341544008470604:6527], now have 1 active actors on pipe
2025-10-09T21:45:57.805563Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:57.805973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 624 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1ca4de9a-9028f8f0-f76f6703-14c868da_0
2025-10-09T21:45:57.806189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 624 sessionId: src_id|1ca4de9a-9028f8f0-f76f6703-14c868da_0 is DEAD
2025-10-09T21:45:57.806605Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ca4de9a-9028f8f0-f76f6703-14c868da_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:57.806617Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ca4de9a-9028f8f0-f76f6703-14c868da_0] Write session will restart in 0.006976s
2025-10-09T21:45:57.806646Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ca4de9a-9028f8f0-f76f6703-14c868da_0] Write session: Do CDS request
2025-10-09T21:45:57.806653Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ca4de9a-9028f8f0-f76f6703-14c868da_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:57.814713Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ca4de9a-9028f8f0-f76f6703-14c868da_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:57.815051Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:57.815077Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 625
2025-10-09T21:45:57.815327Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 625 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:57.815385Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 625 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:57.815407Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=625 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:57.815412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:57.818490Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 625 sessionId:  describe result for acl check
2025-10-09T21:45:57.818538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:57.818545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:57.818546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:57.818553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341544008470622:6682] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:57.819111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341544008470622:6682] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:57.830613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341544008470622:6682] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:57.834584Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341544008470622:6682] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:57.834604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341544008470622:6682] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:57.834610Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341544008470622:6682] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:57.834619Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 625 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:57.839702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:57.848612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:57.848889Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:57.848901Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:57.848926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:57.849011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:57.849058Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341543646657019:6081]
2025-10-09T21:45:57.849661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:57.850168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:57.850233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:57.850732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:57.850830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:57.850839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:57.850840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:57.850841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:57.850843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:57.850844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:57.850846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.850847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.850848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.850854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:57.850866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:57.850871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:57.850875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:57.850879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:57.850883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:57.850887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.850891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.850895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.850914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:57.850923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:57.850929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:57.850934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:57.850939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:57.850944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:57.850949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:57.850953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:57.850958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:57.850965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:57.850970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:57.850975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:57.850979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.850984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.850987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.850992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.850996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.851000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.851029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:57.858584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:57.860319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:57.860335Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:57.860338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:57.860342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:57.860349Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 624 [2:7559341543646657019:6081]
2025-10-09T21:45:57.860364Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:57.860385Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:57.860389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:57.860392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:57.860395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:57.860414Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:57.860420Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:57.860428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:57.860491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.860508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.860510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.860513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:57.860605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:57.860615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.860629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:57.860648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:57.860664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:57.860688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341543646657013:6081]
2025-10-09T21:45:57.860697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341543646657013:6081]
2025-10-09T21:45:57.860699Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341543646657013:6081]
2025-10-09T21:45:57.860716Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:57.860734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:57.860755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:57.860759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:57.860799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341543646657013:6081]
2025-10-09T21:45:57.860802Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:57.860834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341544008470785:6682], now have 1 active actors on pipe
2025-10-09T21:45:57.860843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341544008470609:2432], now have 1 active actors on pipe
2025-10-09T21:45:57.860938Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:57.860951Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:57.860953Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:57.860961Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:57.860962Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 624
2025-10-09T21:45:57.860998Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 624
2025-10-09T21:45:57.861569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:57.861584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:57.861621Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f006266e-f8cfa62c-80595c34-fe8a8e3e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:57.861631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:57.861635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:57.861639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:57.861653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:57.861679Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:57.861698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:57.862057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:57.862064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:57.862126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:57.862322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 625 partition: 0 MaxSeqNo: 25 sessionId: src_id|f006266e-f8cfa62c-80595c34-fe8a8e3e_0
2025-10-09T21:45:57.865556Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ca4de9a-9028f8f0-f76f6703-14c868da_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f006266e-f8cfa62c-80595c34-fe8a8e3e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:57.868081Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f006266e-f8cfa62c-80595c34-fe8a8e3e_0] Write session: try to update token
2025-10-09T21:45:57.868104Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f006266e-f8cfa62c-80595c34-fe8a8e3e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:57.877781Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:57.877800Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:57.877803Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:57.877805Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:57.877814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341543646657013:6081]
2025-10-09T21:45:57.877821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341543646657013:6081]
2025-10-09T21:45:57.877826Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341543646657013:6081]
2025-10-09T21:45:57.877872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:57.877878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:57.877881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:57.877932Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:57.877943Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:57.877946Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:57.878018Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:57.896289Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 625 sessionId: src_id|f006266e-f8cfa62c-80595c34-fe8a8e3e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:57.898870Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:57.905976Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341544008470813:6527]
2025-10-09T21:45:57.906498Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 624, pipe: [1:7559341544008470813:6527]
2025-10-09T21:45:58.042651Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:58.042970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:58.042981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:58.043005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341544008470813:6527], now have 1 active actors on pipe
2025-10-09T21:45:58.043344Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:58.043801Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f006266e-f8cfa62c-80595c34-fe8a8e3e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:58.043814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f006266e-f8cfa62c-80595c34-fe8a8e3e_0] Write session will restart in 0.009613s
2025-10-09T21:45:58.043851Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f006266e-f8cfa62c-80595c34-fe8a8e3e_0] Write session: Do CDS request
2025-10-09T21:45:58.043858Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f006266e-f8cfa62c-80595c34-fe8a8e3e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:58.043334Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:58.043352Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:58.043423Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 625 reason: error at write: Disconnected sessionId: src_id|f006266e-f8cfa62c-80595c34-fe8a8e3e_0
2025-10-09T21:45:58.043442Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341544008470813:6527]
2025-10-09T21:45:58.043461Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:58.043705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 625 sessionId: src_id|f006266e-f8cfa62c-80595c34-fe8a8e3e_0 is DEAD
2025-10-09T21:45:58.054634Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f006266e-f8cfa62c-80595c34-fe8a8e3e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:58.054764Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:58.054785Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 626
2025-10-09T21:45:58.055045Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 626 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:58.055082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 626 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:58.055088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=626 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:58.055115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:58.055648Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 626 sessionId:  describe result for acl check
2025-10-09T21:45:58.055696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:58.055698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:58.055699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:58.055706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341548303438136:6687] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:58.056333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341548303438136:6687] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:58.060179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341548303438136:6687] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:58.064148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341548303438136:6687] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:58.064169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341548303438136:6687] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:58.064173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341548303438136:6687] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:58.064183Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 626 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:58.070117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:58.075327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:58.075619Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:58.075634Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:58.075650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:58.075730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:58.075782Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341547941624512:6083]
2025-10-09T21:45:58.076399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:58.076911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:58.076992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:58.077359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:58.077465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:58.077474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:58.077476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:58.077477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:58.077479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:58.077480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:58.077481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.077482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.077484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.077490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:58.077503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:58.077507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:58.077511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:58.077514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:58.077518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:58.077521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.077524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.077528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.077537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:58.077546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:58.077552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:58.077557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:58.077561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:58.077566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:58.077570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:58.077575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:58.077579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:58.077585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:58.077589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:58.077594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:58.077599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.077605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.077609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.077614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.077619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.077625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.077653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:58.085474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:58.087145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:58.087165Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:58.087167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:58.087171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:58.087178Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 625 [2:7559341547941624512:6083]
2025-10-09T21:45:58.087196Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:58.087218Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:58.087221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:58.087225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:58.087227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:58.087247Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:58.087253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:58.087260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:58.087333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.087335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.087337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.087340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:58.087770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:58.087774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.087789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.087803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.087821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:58.087845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341547941624507:6083]
2025-10-09T21:45:58.087848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341547941624507:6083]
2025-10-09T21:45:58.087850Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341547941624507:6083]
2025-10-09T21:45:58.087870Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:58.087887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:58.087902Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:58.087906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:58.087941Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:58.087945Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:58.087947Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:58.087948Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341547941624507:6083]
2025-10-09T21:45:58.087951Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:58.087954Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:58.088325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341548303438119:2432], now have 1 active actors on pipe
2025-10-09T21:45:58.088330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341548303438277:6687], now have 1 active actors on pipe
2025-10-09T21:45:58.088640Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 625
2025-10-09T21:45:58.088699Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 625
2025-10-09T21:45:58.088907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:58.088933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:58.088974Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ad9e057f-665d24ca-8383ceb-6897f325_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:58.088989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:58.088993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:58.088996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:58.089020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:58.089039Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:58.089062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:58.089514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:58.089530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:58.089552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:58.089701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 626 partition: 0 MaxSeqNo: 25 sessionId: src_id|ad9e057f-665d24ca-8383ceb-6897f325_0
2025-10-09T21:45:58.090211Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f006266e-f8cfa62c-80595c34-fe8a8e3e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ad9e057f-665d24ca-8383ceb-6897f325_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:58.093055Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad9e057f-665d24ca-8383ceb-6897f325_0] Write session: try to update token
2025-10-09T21:45:58.093078Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad9e057f-665d24ca-8383ceb-6897f325_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:58.108382Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:58.108406Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:58.108408Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:58.108410Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:58.108420Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341547941624507:6083]
2025-10-09T21:45:58.108425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341547941624507:6083]
2025-10-09T21:45:58.108429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341547941624507:6083]
2025-10-09T21:45:58.108479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:58.108487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.108490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:58.108630Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:58.114060Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:58.114083Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:58.114087Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:58.124863Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 626 sessionId: src_id|ad9e057f-665d24ca-8383ceb-6897f325_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:58.127822Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:58.143464Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341548303438327:6527]
2025-10-09T21:45:58.144104Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 625, pipe: [1:7559341548303438327:6527]
2025-10-09T21:45:58.277473Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:58.277762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:58.277773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:58.277797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341548303438327:6527], now have 1 active actors on pipe
2025-10-09T21:45:58.278330Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341548303438327:6527]
2025-10-09T21:45:58.278357Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:58.278390Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:58.278393Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:58.278447Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 626 reason: error at write: Disconnected sessionId: src_id|ad9e057f-665d24ca-8383ceb-6897f325_0
2025-10-09T21:45:58.278838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 626 sessionId: src_id|ad9e057f-665d24ca-8383ceb-6897f325_0 is DEAD
2025-10-09T21:45:58.279173Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad9e057f-665d24ca-8383ceb-6897f325_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:58.279189Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad9e057f-665d24ca-8383ceb-6897f325_0] Write session will restart in 0.008090s
2025-10-09T21:45:58.279216Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad9e057f-665d24ca-8383ceb-6897f325_0] Write session: Do CDS request
2025-10-09T21:45:58.279228Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad9e057f-665d24ca-8383ceb-6897f325_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:58.279586Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:58.288567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad9e057f-665d24ca-8383ceb-6897f325_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:58.288961Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:58.288990Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 627
2025-10-09T21:45:58.289290Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 627 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:58.289364Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 627 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:58.289371Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=627 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:58.289376Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:58.289856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 627 sessionId:  describe result for acl check
2025-10-09T21:45:58.289893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:58.289895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:58.289897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:58.289903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341548303438364:6698] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:58.290558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341548303438364:6698] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:58.293031Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341548303438364:6698] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:58.298718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341548303438364:6698] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:58.298739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341548303438364:6698] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:58.298742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341548303438364:6698] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:58.298752Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 627 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:58.314077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:58.319427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:58.319734Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:58.319749Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:58.319765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:58.319843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:58.319891Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341547941624748:6093]
2025-10-09T21:45:58.320494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:58.320986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:58.321062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:58.321425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:58.321498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:58.321508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:58.321509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:58.321510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:58.321512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:58.321513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:58.321515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.321516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.321518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.321524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:58.321536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:58.321539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:58.321561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:58.321566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:58.321570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:58.321574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.321578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.321582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.321590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:58.321598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:58.321604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:58.321612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:58.321622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:58.321627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:58.321631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:58.321637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:58.321646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:58.321664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:58.321673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:58.321678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:58.321683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.321688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.321698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.321703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.321708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.321712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.321744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:58.330135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:58.331320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:58.331326Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:58.331329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:58.331334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:58.331342Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 626 [2:7559341547941624748:6093]
2025-10-09T21:45:58.332033Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 626
2025-10-09T21:45:58.332082Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 626
2025-10-09T21:45:58.331358Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:58.331379Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:58.331383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:58.331387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:58.331389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:58.331409Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:58.331416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:58.331424Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:58.331491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.331492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.331497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.331501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:58.331583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:58.331586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.331601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.331612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.331628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:58.331650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341547941624743:6093]
2025-10-09T21:45:58.331653Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341547941624743:6093]
2025-10-09T21:45:58.331655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341547941624743:6093]
2025-10-09T21:45:58.331664Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:58.331679Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:58.331695Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:58.331698Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:58.331718Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:58.331722Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:58.331724Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:58.331738Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341547941624743:6093]
2025-10-09T21:45:58.331741Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:58.331749Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:58.331765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341548303438445:6698], now have 1 active actors on pipe
2025-10-09T21:45:58.331779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341548303438346:2432], now have 1 active actors on pipe
2025-10-09T21:45:58.332321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:58.332346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:58.332385Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2b56fa45-9621469-315ec407-fb0a2b33_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:58.332401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:58.332406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:58.332408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:58.332420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:58.332433Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:58.332460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:58.333203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:58.333223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:58.333243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:58.335696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 627 partition: 0 MaxSeqNo: 25 sessionId: src_id|2b56fa45-9621469-315ec407-fb0a2b33_0
2025-10-09T21:45:58.336265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad9e057f-665d24ca-8383ceb-6897f325_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2b56fa45-9621469-315ec407-fb0a2b33_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:58.339534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b56fa45-9621469-315ec407-fb0a2b33_0] Write session: try to update token
2025-10-09T21:45:58.339561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b56fa45-9621469-315ec407-fb0a2b33_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:58.350783Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:58.350804Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:58.350807Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:58.350808Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:58.350818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341547941624743:6093]
2025-10-09T21:45:58.350823Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341547941624743:6093]
2025-10-09T21:45:58.350827Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341547941624743:6093]
2025-10-09T21:45:58.350879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:58.350887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.350889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:58.351063Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:58.354504Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:58.354524Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:58.354529Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:58.374420Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 627 sessionId: src_id|2b56fa45-9621469-315ec407-fb0a2b33_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:58.376966Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:58.378735Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341548303438558:6527]
2025-10-09T21:45:58.381160Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 626, pipe: [1:7559341548303438558:6527]
2025-10-09T21:45:58.586443Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:58.587187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:58.587206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:58.587230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341548303438558:6527], now have 1 active actors on pipe
2025-10-09T21:45:58.587532Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:58.587550Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:58.587476Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:58.588238Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b56fa45-9621469-315ec407-fb0a2b33_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:58.588254Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b56fa45-9621469-315ec407-fb0a2b33_0] Write session will restart in 0.006567s
2025-10-09T21:45:58.588282Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b56fa45-9621469-315ec407-fb0a2b33_0] Write session: Do CDS request
2025-10-09T21:45:58.587622Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 627 reason: error at write: Disconnected sessionId: src_id|2b56fa45-9621469-315ec407-fb0a2b33_0
2025-10-09T21:45:58.587828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 627 sessionId: src_id|2b56fa45-9621469-315ec407-fb0a2b33_0 is DEAD
2025-10-09T21:45:58.588084Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:58.588125Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341548303438558:6527]
2025-10-09T21:45:58.588290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b56fa45-9621469-315ec407-fb0a2b33_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:58.598894Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b56fa45-9621469-315ec407-fb0a2b33_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:58.598973Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:58.599003Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 628
2025-10-09T21:45:58.599556Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 628 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:58.599613Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 628 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:58.599624Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=628 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:58.599628Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:58.601003Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 628 sessionId:  describe result for acl check
2025-10-09T21:45:58.601051Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:58.601054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:58.601056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:58.601065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341548303438596:6702] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:58.601699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341548303438596:6702] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:58.607456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341548303438596:6702] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:58.612433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341548303438596:6702] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:58.612459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341548303438596:6702] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:58.612463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341548303438596:6702] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:58.612473Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 628 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:58.616310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:58.624848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:58.625134Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:58.625139Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:58.625153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:58.625226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:58.625264Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341547941624952:6101]
2025-10-09T21:45:58.625891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:58.626441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:58.626504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:58.626849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:58.626925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:58.626934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:58.626935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:58.626937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:58.626939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:58.626940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:58.626941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.626944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.626946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.626952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:58.626965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:58.626969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:58.626973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:58.626977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:58.626981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:58.626985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.626989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.626993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.627001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:58.627009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:58.627016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:58.627021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:58.627026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:58.627031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:58.627035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:58.627040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:58.627045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:58.627051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:58.627056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:58.627062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:58.627067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.627073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.627077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.627082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.627087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.627093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.627122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:58.637211Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 627
2025-10-09T21:45:58.637274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 627
2025-10-09T21:45:58.635423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:58.636574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:58.636585Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:58.636588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:58.636593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:58.636602Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 627 [2:7559341547941624952:6101]
2025-10-09T21:45:58.636619Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:58.636640Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:58.636643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:58.636647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:58.636649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:58.636668Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:58.636674Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:58.636682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:58.636748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.636751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.636752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.636756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:58.636836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:58.636839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.636852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.636865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.636883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:58.636904Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341547941624947:6101]
2025-10-09T21:45:58.636906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341547941624947:6101]
2025-10-09T21:45:58.636908Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341547941624947:6101]
2025-10-09T21:45:58.636926Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:58.636942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:58.636957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:58.636961Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:58.636961Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:58.636967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:58.636970Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:58.637004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341547941624947:6101]
2025-10-09T21:45:58.637007Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:58.637051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:58.637524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341548303438561:2432], now have 1 active actors on pipe
2025-10-09T21:45:58.637530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341548303438753:6702], now have 1 active actors on pipe
2025-10-09T21:45:58.637546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:58.637552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:58.637585Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9ba8d201-4d07f302-f76acd2d-5e30aca5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:58.637595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:58.637599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:58.637602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:58.637613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:58.637625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:58.637643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:58.638403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:58.638420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:58.638440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:58.638718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 628 partition: 0 MaxSeqNo: 25 sessionId: src_id|9ba8d201-4d07f302-f76acd2d-5e30aca5_0
2025-10-09T21:45:58.639300Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b56fa45-9621469-315ec407-fb0a2b33_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9ba8d201-4d07f302-f76acd2d-5e30aca5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:58.642143Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ba8d201-4d07f302-f76acd2d-5e30aca5_0] Write session: try to update token
2025-10-09T21:45:58.642164Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ba8d201-4d07f302-f76acd2d-5e30aca5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:58.660917Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:58.660939Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:58.660942Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:58.660944Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:58.660953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341547941624947:6101]
2025-10-09T21:45:58.660958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341547941624947:6101]
2025-10-09T21:45:58.660962Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341547941624947:6101]
2025-10-09T21:45:58.661012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:58.661015Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:58.661019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.661022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:58.661032Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:58.661036Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:58.661164Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:58.661609Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 628 sessionId: src_id|9ba8d201-4d07f302-f76acd2d-5e30aca5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:58.665060Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:58.688319Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341548303438770:6527]
2025-10-09T21:45:58.689180Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 627, pipe: [1:7559341548303438770:6527]
2025-10-09T21:45:58.833717Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:58.834437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:58.834459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:58.834509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341548303438770:6527], now have 1 active actors on pipe
2025-10-09T21:45:58.834908Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:58.835734Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ba8d201-4d07f302-f76acd2d-5e30aca5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:58.835748Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ba8d201-4d07f302-f76acd2d-5e30aca5_0] Write session will restart in 0.009838s
2025-10-09T21:45:58.835772Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ba8d201-4d07f302-f76acd2d-5e30aca5_0] Write session: Do CDS request
2025-10-09T21:45:58.835779Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ba8d201-4d07f302-f76acd2d-5e30aca5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:58.835185Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:58.835247Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341548303438770:6527]
2025-10-09T21:45:58.835278Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:58.835350Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 628 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9ba8d201-4d07f302-f76acd2d-5e30aca5_0
2025-10-09T21:45:58.835560Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 628 sessionId: src_id|9ba8d201-4d07f302-f76acd2d-5e30aca5_0 is DEAD
2025-10-09T21:45:58.846787Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ba8d201-4d07f302-f76acd2d-5e30aca5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:58.847562Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:58.847581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 629
2025-10-09T21:45:58.847768Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 629 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:58.847829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 629 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:58.847835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=629 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:58.847840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:58.848469Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 629 sessionId:  describe result for acl check
2025-10-09T21:45:58.848512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:58.848519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:58.848521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:58.848528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341548303438787:6707] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:58.849197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341548303438787:6707] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:58.854620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341548303438787:6707] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:58.861248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341548303438787:6707] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:58.861273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341548303438787:6707] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:58.861278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341548303438787:6707] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:58.861288Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 629 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:58.873714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:58.880090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:58.880393Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:58.880407Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:58.880423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:58.880496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:58.880550Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341547941625153:6103]
2025-10-09T21:45:58.881158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:58.882188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:58.882249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:58.882609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:58.882692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:58.882699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:58.882700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:58.882701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:58.882703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:58.882704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:58.882705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.882706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.882708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.882714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:58.882724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:58.882727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:58.882731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:58.882734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:58.882738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:58.882741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.882745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.882749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.882758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:58.882766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:58.882773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:58.882778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:58.882783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:58.882789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:58.882793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:58.882798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:58.882803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:58.882807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:58.882812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:58.882816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:58.882822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.882826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.882830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.882835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.882840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.882845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.882875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:58.892060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:58.893759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:58.893782Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:58.893784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:58.893790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:58.893798Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 628 [2:7559341547941625153:6103]
2025-10-09T21:45:58.893812Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:58.893833Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:58.893837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:58.893840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:58.893842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:58.893865Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:58.893882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:58.893893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:58.893968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.893975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.893977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.893980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:58.894078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:58.894086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.894116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:58.894128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:58.894150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:58.894177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341547941625148:6103]
2025-10-09T21:45:58.894180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341547941625148:6103]
2025-10-09T21:45:58.894182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341547941625148:6103]
2025-10-09T21:45:58.894197Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:58.894220Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:58.894234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:58.894243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:58.894285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341547941625148:6103]
2025-10-09T21:45:58.894293Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:58.894725Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 628
2025-10-09T21:45:58.894776Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 628
2025-10-09T21:45:58.894581Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:58.894590Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:58.894593Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:58.894600Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:58.895055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341548303438864:6707], now have 1 active actors on pipe
2025-10-09T21:45:58.895073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341548303438773:2432], now have 1 active actors on pipe
2025-10-09T21:45:58.895096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:58.895101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:58.895132Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|241bdaab-db749f5a-a95ff716-7f151741_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:58.895148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:58.895153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:58.895156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:58.895168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:58.895183Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:58.895200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:58.895561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:58.895572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:58.895592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:58.895947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 629 partition: 0 MaxSeqNo: 25 sessionId: src_id|241bdaab-db749f5a-a95ff716-7f151741_0
2025-10-09T21:45:58.896414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ba8d201-4d07f302-f76acd2d-5e30aca5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|241bdaab-db749f5a-a95ff716-7f151741_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:58.899960Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|241bdaab-db749f5a-a95ff716-7f151741_0] Write session: try to update token
2025-10-09T21:45:58.899980Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|241bdaab-db749f5a-a95ff716-7f151741_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:58.913418Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:58.913440Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:58.913443Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:58.913445Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:58.913456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341547941625148:6103]
2025-10-09T21:45:58.913461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341547941625148:6103]
2025-10-09T21:45:58.913464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341547941625148:6103]
2025-10-09T21:45:58.913509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:58.913517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:58.913520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:58.913671Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:58.918146Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:58.918154Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:58.918158Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:58.921402Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 629 sessionId: src_id|241bdaab-db749f5a-a95ff716-7f151741_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:58.926246Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:58.935678Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341548303438982:6527]
2025-10-09T21:45:58.936596Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 628, pipe: [1:7559341548303438982:6527]
2025-10-09T21:45:59.082945Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:59.083268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:59.083283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:59.083335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341548303438982:6527], now have 1 active actors on pipe
2025-10-09T21:45:59.083633Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:59.083595Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:59.083596Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:59.083612Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:59.083666Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341548303438982:6527]
2025-10-09T21:45:59.083931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 629 reason: error at write: Disconnected sessionId: src_id|241bdaab-db749f5a-a95ff716-7f151741_0
2025-10-09T21:45:59.084129Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 629 sessionId: src_id|241bdaab-db749f5a-a95ff716-7f151741_0 is DEAD
2025-10-09T21:45:59.084310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|241bdaab-db749f5a-a95ff716-7f151741_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:59.084322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|241bdaab-db749f5a-a95ff716-7f151741_0] Write session will restart in 0.009009s
2025-10-09T21:45:59.084366Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|241bdaab-db749f5a-a95ff716-7f151741_0] Write session: Do CDS request
2025-10-09T21:45:59.084372Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|241bdaab-db749f5a-a95ff716-7f151741_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:59.094668Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|241bdaab-db749f5a-a95ff716-7f151741_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:59.095132Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:59.095152Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 630
2025-10-09T21:45:59.095313Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 630 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:59.095347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 630 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:59.095358Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=630 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:59.095362Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:59.096229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 630 sessionId:  describe result for acl check
2025-10-09T21:45:59.096298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:59.096300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:59.096302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:59.096309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341552598406316:6712] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:59.097182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341552598406316:6712] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:59.100358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341552598406316:6712] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:59.104992Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341552598406316:6712] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:59.105014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341552598406316:6712] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:59.105018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341552598406316:6712] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:59.105030Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 630 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:59.108364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:59.112712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:59.113078Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:59.113090Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:59.113105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:59.113195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:59.113241Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341552236592650:6105]
2025-10-09T21:45:59.113848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:59.114377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:59.114459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:59.114830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:59.114897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:59.114908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:59.114909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:59.114910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:59.114911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:59.114912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:59.114915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.114916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.114918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.114923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:59.114947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:59.114950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:59.114954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:59.114957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:59.114960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:59.114964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.114967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.114971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.114980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:59.114988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:59.114994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:59.114999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:59.115003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:59.115015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:59.115019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:59.115024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:59.115028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:59.115037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:59.115042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:59.115048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:59.115061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.115067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.115071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.115077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.115083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.115088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.115133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:59.123470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:59.125158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:59.125180Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:59.125183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:59.125188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:59.125196Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 629 [2:7559341552236592650:6105]
2025-10-09T21:45:59.125211Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:59.125232Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:59.125236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:59.125240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:59.125242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:59.125262Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:59.125268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:59.125277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:59.125347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.125357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.125359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.125362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:59.125443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:59.125454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.125469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.125480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.125496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:59.125519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341552598406483:6712], now have 1 active actors on pipe
2025-10-09T21:45:59.125522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341552598406289:2432], now have 1 active actors on pipe
2025-10-09T21:45:59.125541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341552236592645:6105]
2025-10-09T21:45:59.125544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341552236592645:6105]
2025-10-09T21:45:59.125548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341552236592645:6105]
2025-10-09T21:45:59.125560Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:59.125576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:59.125591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:59.125599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:59.125622Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:59.125630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:59.125632Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:59.125644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341552236592645:6105]
2025-10-09T21:45:59.125652Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:59.125657Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:59.125645Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 629
2025-10-09T21:45:59.125646Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 629
2025-10-09T21:45:59.125691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:59.125704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:59.125744Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|681ae48a-ef970286-f66f113e-4fd0c787_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:59.125752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:59.125757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:59.125759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:59.125770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:59.125781Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:59.125799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:59.125975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:59.125984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:59.126000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:59.126161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 630 partition: 0 MaxSeqNo: 25 sessionId: src_id|681ae48a-ef970286-f66f113e-4fd0c787_0
2025-10-09T21:45:59.126606Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|241bdaab-db749f5a-a95ff716-7f151741_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|681ae48a-ef970286-f66f113e-4fd0c787_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:59.129411Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|681ae48a-ef970286-f66f113e-4fd0c787_0] Write session: try to update token
2025-10-09T21:45:59.129434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|681ae48a-ef970286-f66f113e-4fd0c787_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:59.141156Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:45:59.141176Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:59.141189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341552236592645:6105]
2025-10-09T21:45:59.141245Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:45:59.141247Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:45:59.141431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:45:59.141595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:59.141751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:59.141912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:45:59.142168Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:45:59.142227Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:45:59.152220Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 630 sessionId: src_id|681ae48a-ef970286-f66f113e-4fd0c787_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:59.154858Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:59.156145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:59.156179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:59.156203Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:59.156205Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:59.156210Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:59.156212Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:59.156221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341552236592645:6105]
2025-10-09T21:45:59.156225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341552236592645:6105]
2025-10-09T21:45:59.156228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341552236592645:6105]
2025-10-09T21:45:59.156283Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:59.156289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:59.156298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.156300Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:59.156301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:59.156304Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:59.156442Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:59.183950Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341552598406498:6527]
2025-10-09T21:45:59.184365Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 629, pipe: [1:7559341552598406498:6527]
2025-10-09T21:45:59.321422Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:59.321750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:59.321795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341552598406498:6527], now have 1 active actors on pipe
2025-10-09T21:45:59.321935Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:59.322062Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:59.322072Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:59.322179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 630 reason: error at write: Disconnected sessionId: src_id|681ae48a-ef970286-f66f113e-4fd0c787_0
2025-10-09T21:45:59.322334Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 630 sessionId: src_id|681ae48a-ef970286-f66f113e-4fd0c787_0 is DEAD
2025-10-09T21:45:59.322491Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341552598406498:6527]
2025-10-09T21:45:59.322518Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:59.323140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|681ae48a-ef970286-f66f113e-4fd0c787_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:59.323158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|681ae48a-ef970286-f66f113e-4fd0c787_0] Write session will restart in 0.005866s
2025-10-09T21:45:59.323187Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|681ae48a-ef970286-f66f113e-4fd0c787_0] Write session: Do CDS request
2025-10-09T21:45:59.323199Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|681ae48a-ef970286-f66f113e-4fd0c787_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:59.334648Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|681ae48a-ef970286-f66f113e-4fd0c787_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:59.334893Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:59.334912Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 631
2025-10-09T21:45:59.337569Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 631 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:59.337620Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 631 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:59.337625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=631 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:59.337629Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:59.338539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 631 sessionId:  describe result for acl check
2025-10-09T21:45:59.338580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:59.338587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:59.338589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:59.338594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341552598406620:6722] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:59.339144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341552598406620:6722] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:59.342250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341552598406620:6722] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:59.345483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341552598406620:6722] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:59.345501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341552598406620:6722] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:59.345505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341552598406620:6722] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:59.345514Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 631 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:59.345864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:59.350176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:59.350463Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:59.350473Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:59.350486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:59.350548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:59.350603Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341552236592891:6113]
2025-10-09T21:45:59.351176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:59.351647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:59.351723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:59.352086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:59.352151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:59.352159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:59.352161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:59.352162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:59.352163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:59.352164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:59.352165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.352166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.352167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.352176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:59.352188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:59.352192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:59.352196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:59.352199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:59.352202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:59.352205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.352212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.352216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.352223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:59.352231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:59.352237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:59.352241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:59.352246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:59.352251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:59.352255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:59.352260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:59.352264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:59.352269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:59.352272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:59.352277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:59.352282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.352286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.352290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.352294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.352299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.352304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.352335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:59.359597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:59.360759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:59.360766Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:59.360769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:59.360773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:59.360781Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 630 [2:7559341552236592891:6113]
2025-10-09T21:45:59.360796Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:59.360816Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:59.360820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:59.360823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:59.360826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:59.360846Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:59.360851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:59.360859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:59.360927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.360929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.360931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.360934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:59.361007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:59.361010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.361024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.361034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.361049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:59.361071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341552236592885:6113]
2025-10-09T21:45:59.361076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341552236592885:6113]
2025-10-09T21:45:59.361078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341552236592885:6113]
2025-10-09T21:45:59.361087Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:59.361104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:59.361108Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:59.361112Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:59.361114Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:59.361119Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:59.361122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:59.361134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341552598406719:6722], now have 1 active actors on pipe
2025-10-09T21:45:59.361172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341552598406514:2432], now have 1 active actors on pipe
2025-10-09T21:45:59.361175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341552236592885:6113]
2025-10-09T21:45:59.361190Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:59.361202Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:59.362134Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 630
2025-10-09T21:45:59.362184Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 630
2025-10-09T21:45:59.362253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:59.362269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:59.362373Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f5e579c2-33467c07-590db7b7-4387b84a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:59.362385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:59.362390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:59.362392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:59.362404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:59.362418Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:59.362437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:59.362804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:59.362818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:59.362840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:59.363033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 631 partition: 0 MaxSeqNo: 25 sessionId: src_id|f5e579c2-33467c07-590db7b7-4387b84a_0
2025-10-09T21:45:59.363517Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|681ae48a-ef970286-f66f113e-4fd0c787_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f5e579c2-33467c07-590db7b7-4387b84a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:59.366923Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5e579c2-33467c07-590db7b7-4387b84a_0] Write session: try to update token
2025-10-09T21:45:59.366947Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5e579c2-33467c07-590db7b7-4387b84a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:59.377081Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:59.377104Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:59.377107Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:59.377109Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:59.377120Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341552236592885:6113]
2025-10-09T21:45:59.377124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341552236592885:6113]
2025-10-09T21:45:59.377128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341552236592885:6113]
2025-10-09T21:45:59.377179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:59.377186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.377189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:59.377199Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:59.377205Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:59.377209Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:59.377346Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:59.384619Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 631 sessionId: src_id|f5e579c2-33467c07-590db7b7-4387b84a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:59.387274Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:59.422132Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341552598406733:6527]
2025-10-09T21:45:59.422587Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 630, pipe: [1:7559341552598406733:6527]
2025-10-09T21:45:59.538191Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:59.538862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:59.538875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:59.538925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341552598406733:6527], now have 1 active actors on pipe
2025-10-09T21:45:59.539227Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:59.540156Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341552598406733:6527]
2025-10-09T21:45:59.540175Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:45:59.540188Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:59.540246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 631 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f5e579c2-33467c07-590db7b7-4387b84a_0
2025-10-09T21:45:59.540454Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 631 sessionId: src_id|f5e579c2-33467c07-590db7b7-4387b84a_0 is DEAD
2025-10-09T21:45:59.542296Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5e579c2-33467c07-590db7b7-4387b84a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:45:59.542309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5e579c2-33467c07-590db7b7-4387b84a_0] Write session will restart in 0.009505s
2025-10-09T21:45:59.542338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5e579c2-33467c07-590db7b7-4387b84a_0] Write session: Do CDS request
2025-10-09T21:45:59.542345Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5e579c2-33467c07-590db7b7-4387b84a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:59.554507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5e579c2-33467c07-590db7b7-4387b84a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:59.554714Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:59.554735Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 632
2025-10-09T21:45:59.556029Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 632 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:59.556076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 632 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:59.556083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=632 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:59.556087Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:59.557431Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 632 sessionId:  describe result for acl check
2025-10-09T21:45:59.557470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:59.557474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:59.557476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:59.557482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341552598406759:6727] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:59.558023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341552598406759:6727] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:59.561666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341552598406759:6727] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:59.574464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:59.574576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341552598406759:6727] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:59.574600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341552598406759:6727] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:59.574604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341552598406759:6727] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:59.574615Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 632 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:59.576032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:59.576265Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:59.576270Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:59.576285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:59.576354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:59.576391Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341552236593098:6123]
2025-10-09T21:45:59.576997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:59.577492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:59.577546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:59.577935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:59.577996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:59.578002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:59.578004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:59.578005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:59.578007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:59.578008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:59.578009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.578010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.578012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.578018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:59.578048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:59.578053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:59.578056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:59.578059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:59.578063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:59.578067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.578070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.578074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.578082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:59.578104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:59.578111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:59.578116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:59.578121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:59.578127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:59.578131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:59.578136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:59.578141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:59.578146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:59.578150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:59.578155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:59.578160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.578164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.578167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.578184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.578188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.578196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.578226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:59.585699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:59.587175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:59.587191Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:59.587204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:59.587209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:59.587216Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 631 [2:7559341552236593098:6123]
2025-10-09T21:45:59.587231Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:59.587256Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:59.587263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:59.587266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:59.587269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:59.587288Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:59.587299Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:59.587306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:59.587359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.587366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.587368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.587370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:59.587443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:59.587450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.587462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.587473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.587494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:59.587813Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 631
2025-10-09T21:45:59.587865Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 631
2025-10-09T21:45:59.587971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341552236593092:6123]
2025-10-09T21:45:59.587979Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341552236593092:6123]
2025-10-09T21:45:59.587981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341552236593092:6123]
2025-10-09T21:45:59.587995Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:59.588012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341552598406934:6727], now have 1 active actors on pipe
2025-10-09T21:45:59.588015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341552598406736:2432], now have 1 active actors on pipe
2025-10-09T21:45:59.588031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:59.588041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:59.588067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:59.588096Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:59.588104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:59.588159Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|80fd7601-8b7868ce-7a5c5f0d-78e09f17_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:59.588171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:59.588175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:59.588178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:59.588189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:59.588200Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:59.588211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341552236593092:6123]
2025-10-09T21:45:59.588213Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:59.588235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:59.588312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:59.588316Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:59.588318Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:59.588324Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:59.589061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:59.589072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:59.589092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:59.589235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 632 partition: 0 MaxSeqNo: 25 sessionId: src_id|80fd7601-8b7868ce-7a5c5f0d-78e09f17_0
2025-10-09T21:45:59.589706Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5e579c2-33467c07-590db7b7-4387b84a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|80fd7601-8b7868ce-7a5c5f0d-78e09f17_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:59.592449Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80fd7601-8b7868ce-7a5c5f0d-78e09f17_0] Write session: try to update token
2025-10-09T21:45:59.592469Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80fd7601-8b7868ce-7a5c5f0d-78e09f17_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:59.609056Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 632 sessionId: src_id|80fd7601-8b7868ce-7a5c5f0d-78e09f17_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:59.607678Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:59.607702Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:59.607705Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:59.607707Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:59.607732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341552236593092:6123]
2025-10-09T21:45:59.607737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341552236593092:6123]
2025-10-09T21:45:59.607741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341552236593092:6123]
2025-10-09T21:45:59.607785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:59.607793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.607795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:59.607838Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:59.607844Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:59.607848Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:59.607935Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:59.611589Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:59.640611Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341552598406947:6527]
2025-10-09T21:45:59.641117Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 631, pipe: [1:7559341552598406947:6527]
2025-10-09T21:45:59.772197Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:45:59.772459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:59.772476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:59.772521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341552598406947:6527], now have 1 active actors on pipe
2025-10-09T21:45:59.772727Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:45:59.772874Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341552598406947:6527]
2025-10-09T21:45:59.772927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:45:59.772930Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:45:59.772996Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 632 reason: error at write: Disconnected sessionId: src_id|80fd7601-8b7868ce-7a5c5f0d-78e09f17_0
2025-10-09T21:45:59.773175Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 632 sessionId: src_id|80fd7601-8b7868ce-7a5c5f0d-78e09f17_0 is DEAD
2025-10-09T21:45:59.773341Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:45:59.773603Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80fd7601-8b7868ce-7a5c5f0d-78e09f17_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:45:59.773611Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80fd7601-8b7868ce-7a5c5f0d-78e09f17_0] Write session will restart in 0.008420s
2025-10-09T21:45:59.773645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80fd7601-8b7868ce-7a5c5f0d-78e09f17_0] Write session: Do CDS request
2025-10-09T21:45:59.773651Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80fd7601-8b7868ce-7a5c5f0d-78e09f17_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:45:59.783728Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80fd7601-8b7868ce-7a5c5f0d-78e09f17_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:59.784121Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:45:59.784141Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 633
2025-10-09T21:45:59.784322Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 633 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:45:59.784384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 633 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:45:59.784410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=633 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:45:59.784414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:45:59.784947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 633 sessionId:  describe result for acl check
2025-10-09T21:45:59.785002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:45:59.785010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:45:59.785011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:45:59.785017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341552598406960:6732] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:45:59.785599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341552598406960:6732] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:45:59.788909Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341552598406960:6732] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:45:59.795412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341552598406960:6732] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:45:59.795432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341552598406960:6732] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:45:59.795436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341552598406960:6732] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:45:59.795446Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 633 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:45:59.805939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:45:59.809931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:45:59.810224Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:45:59.810233Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:45:59.810247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:45:59.810308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:45:59.810356Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341552236593302:6125]
2025-10-09T21:45:59.810961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:45:59.811643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:45:59.811710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:45:59.812083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:45:59.812140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:59.812146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:59.812148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:59.812149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:59.812150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:59.812151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:59.812153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.812155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.812156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.812162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:59.812173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:59.812177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:59.812181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:59.812185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:59.812190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:59.812194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.812197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.812201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.812209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:59.812217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:45:59.812223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:59.812233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:45:59.812238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:59.812243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:45:59.812247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:59.812251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:45:59.812256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:59.812264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:45:59.812267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:59.812282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:45:59.812293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.812297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.812301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.812304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.812309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.812313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.812336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:45:59.819726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:45:59.822003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:45:59.822022Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:45:59.822025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:45:59.822065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:45:59.822074Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 632 [2:7559341552236593302:6125]
2025-10-09T21:45:59.822103Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:45:59.822125Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:45:59.822143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:59.822151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:59.822154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:59.822174Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:45:59.822185Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:45:59.822193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:45:59.822581Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 632
2025-10-09T21:45:59.822590Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 632
2025-10-09T21:45:59.822269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.822276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.822278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.822281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:45:59.822357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:45:59.822361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.822376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:45:59.822389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:45:59.822408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:45:59.822429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341552236593297:6125]
2025-10-09T21:45:59.822432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341552236593297:6125]
2025-10-09T21:45:59.822434Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341552236593297:6125]
2025-10-09T21:45:59.822445Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:45:59.822461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:45:59.822478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:45:59.822481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:45:59.822482Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:45:59.822485Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:45:59.822487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:45:59.822493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341552598407008:6732], now have 1 active actors on pipe
2025-10-09T21:45:59.822497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341552598406952:2432], now have 1 active actors on pipe
2025-10-09T21:45:59.822540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341552236593297:6125]
2025-10-09T21:45:59.822547Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:45:59.822572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:45:59.822832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:59.822842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:59.822874Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e240f676-abb85118-1e27642b-aa4da734_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:45:59.822891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:59.822895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:59.822898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:59.822909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:59.822918Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:45:59.822936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:59.823246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:59.823254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:59.823282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:45:59.823554Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 633 partition: 0 MaxSeqNo: 25 sessionId: src_id|e240f676-abb85118-1e27642b-aa4da734_0
2025-10-09T21:45:59.823915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80fd7601-8b7868ce-7a5c5f0d-78e09f17_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e240f676-abb85118-1e27642b-aa4da734_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:45:59.826738Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e240f676-abb85118-1e27642b-aa4da734_0] Write session: try to update token
2025-10-09T21:45:59.826758Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e240f676-abb85118-1e27642b-aa4da734_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:45:59.838614Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:45:59.838638Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:45:59.838640Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:45:59.838641Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:45:59.838652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341552236593297:6125]
2025-10-09T21:45:59.838656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341552236593297:6125]
2025-10-09T21:45:59.838660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341552236593297:6125]
2025-10-09T21:45:59.838705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:45:59.838712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:45:59.838711Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:45:59.838717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:45:59.838718Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:45:59.838721Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:45:59.838860Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:45:59.843693Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 633 sessionId: src_id|e240f676-abb85118-1e27642b-aa4da734_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:59.846198Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:59.870481Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341552598407167:6527]
2025-10-09T21:45:59.870838Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 632, pipe: [1:7559341552598407167:6527]
2025-10-09T21:46:00.019604Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:00.019906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:00.019917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:00.019970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341552598407167:6527], now have 1 active actors on pipe
2025-10-09T21:46:00.020245Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:00.020683Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341552598407167:6527]
2025-10-09T21:46:00.020726Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:00.020729Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:00.021157Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e240f676-abb85118-1e27642b-aa4da734_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:00.021169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e240f676-abb85118-1e27642b-aa4da734_0] Write session will restart in 0.005750s
2025-10-09T21:46:00.021193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e240f676-abb85118-1e27642b-aa4da734_0] Write session: Do CDS request
2025-10-09T21:46:00.021198Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e240f676-abb85118-1e27642b-aa4da734_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:00.020788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 633 reason: error at write: Disconnected sessionId: src_id|e240f676-abb85118-1e27642b-aa4da734_0
2025-10-09T21:46:00.020983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 633 sessionId: src_id|e240f676-abb85118-1e27642b-aa4da734_0 is DEAD
2025-10-09T21:46:00.021089Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:00.030631Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e240f676-abb85118-1e27642b-aa4da734_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:00.030683Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:00.030703Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 634
2025-10-09T21:46:00.033287Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 634 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:00.033353Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 634 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:00.033360Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=634 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:00.033364Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:00.034159Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 634 sessionId:  describe result for acl check
2025-10-09T21:46:00.034205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:00.034207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:00.034209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:00.034215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341556893374494:6736] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:00.034886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341556893374494:6736] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:00.038176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341556893374494:6736] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:00.047412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341556893374494:6736] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:00.047438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341556893374494:6736] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:00.047442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341556893374494:6736] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:00.047452Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 634 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:00.049455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:00.053732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:00.054053Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:00.054064Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:00.054080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:00.054162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:00.054212Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341556531560804:6127]
2025-10-09T21:46:00.054853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:00.055300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:00.055385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:00.055734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:00.055804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:00.055805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:00.055807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:00.055808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:00.055809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:00.055811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:00.055812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.055813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.055815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.055822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:00.055833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:00.055838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:00.055842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:00.055846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:00.055851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:00.055855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.055859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.055863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.055872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:00.055880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:00.055887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:00.055893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:00.055897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:00.055902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:00.055906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:00.055911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:00.055916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:00.055921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:00.055926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:00.055931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:00.055937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.055942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.055947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.055952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.055957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.055963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.055991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:00.067597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:00.069126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:00.069145Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:00.069148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:00.069153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:00.069160Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 633 [2:7559341556531560804:6127]
2025-10-09T21:46:00.069176Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:00.069198Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:00.069202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:00.069205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:00.069207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:00.069227Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:00.069235Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:00.069253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:00.069323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.069331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.069333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.069336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:00.069414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:00.069422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.069435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.069444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.069460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:00.069481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341556531560799:6127]
2025-10-09T21:46:00.069488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341556531560799:6127]
2025-10-09T21:46:00.069491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341556531560799:6127]
2025-10-09T21:46:00.069506Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:00.069522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:00.069541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:00.069562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:00.069607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341556531560799:6127]
2025-10-09T21:46:00.069613Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:00.069643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:00.069656Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:00.069658Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:00.069669Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:00.069714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341556893374470:2432], now have 1 active actors on pipe
2025-10-09T21:46:00.069881Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 633
2025-10-09T21:46:00.069934Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 633
2025-10-09T21:46:00.070280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341556893374669:6736], now have 1 active actors on pipe
2025-10-09T21:46:00.070317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:00.070328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:00.070374Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|df56db31-8f89495b-ef8f7140-ecf1df33_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:00.070389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:00.070394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:00.070397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:00.070409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:00.070425Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:00.070451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:00.071025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:00.071038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:00.071069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:00.071207Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 634 partition: 0 MaxSeqNo: 25 sessionId: src_id|df56db31-8f89495b-ef8f7140-ecf1df33_0
2025-10-09T21:46:00.074202Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e240f676-abb85118-1e27642b-aa4da734_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|df56db31-8f89495b-ef8f7140-ecf1df33_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:00.076959Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df56db31-8f89495b-ef8f7140-ecf1df33_0] Write session: try to update token
2025-10-09T21:46:00.076987Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df56db31-8f89495b-ef8f7140-ecf1df33_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:00.103300Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 634 sessionId: src_id|df56db31-8f89495b-ef8f7140-ecf1df33_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:00.102979Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:00.102997Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:00.103010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341556531560799:6127]
2025-10-09T21:46:00.103066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:00.103248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:00.103388Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:00.103534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:00.103670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:00.103954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:00.103999Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:00.104164Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:00.105959Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:00.106589Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:00.106388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:00.106411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:00.106457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:00.115552Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:00.115575Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:00.115578Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:00.115580Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:00.115595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341556531560799:6127]
2025-10-09T21:46:00.115599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341556531560799:6127]
2025-10-09T21:46:00.115604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341556531560799:6127]
2025-10-09T21:46:00.115652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:00.115658Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:00.115661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.115664Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:00.115664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:00.115669Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:00.115830Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:00.120852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341556893374687:6527]
2025-10-09T21:46:00.121210Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 633, pipe: [1:7559341556893374687:6527]
2025-10-09T21:46:00.339180Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:00.340659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341556893374687:6527], now have 1 active actors on pipe
2025-10-09T21:46:00.350006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:00.350872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:00.351053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:00.351113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:00.351188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:00.351248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:00.351289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:00.351293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:00.351358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:00.351413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:00.351483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:00.351604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:00.351649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:00.351654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:00.351720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:00.354826Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:00.354842Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:00.354874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 634 reason: error at write: Disconnected sessionId: src_id|df56db31-8f89495b-ef8f7140-ecf1df33_0
2025-10-09T21:46:00.351800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:00.351864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:00.351928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:00.351968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:00.351971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:00.352040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:00.352099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:00.352161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:00.352224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:00.352265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:00.352268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:00.352337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:00.352397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:00.352459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:00.352522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:00.352566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:00.352569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:00.352637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:00.352694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:00.352753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:00.352809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:00.352856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:00.352859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:00.352923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:00.352978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:00.355549Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df56db31-8f89495b-ef8f7140-ecf1df33_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:00.355568Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df56db31-8f89495b-ef8f7140-ecf1df33_0] Write session will restart in 0.005868s
2025-10-09T21:46:00.353042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:00.355592Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df56db31-8f89495b-ef8f7140-ecf1df33_0] Write session: Do CDS request
2025-10-09T21:46:00.355598Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df56db31-8f89495b-ef8f7140-ecf1df33_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:00.353108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:00.353149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:00.353151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:00.353214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:00.353274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:00.353332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:00.353393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:00.353434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:00.353437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:00.353507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:00.353568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:00.353629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:00.353688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:00.353740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:00.353742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:00.353822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:00.353882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:00.353944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:00.354010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:00.354075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:00.354079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:00.355069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 634 sessionId: src_id|df56db31-8f89495b-ef8f7140-ecf1df33_0 is DEAD
2025-10-09T21:46:00.355254Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341556893374687:6527]
2025-10-09T21:46:00.355284Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:00.355751Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:00.362644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df56db31-8f89495b-ef8f7140-ecf1df33_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:00.363309Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:00.363331Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 635
2025-10-09T21:46:00.363524Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 635 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:00.363575Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 635 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:00.363588Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=635 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:00.363594Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:00.364416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 635 sessionId:  describe result for acl check
2025-10-09T21:46:00.364458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:00.364460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:00.364464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:00.364470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341556893374717:6747] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:00.365026Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341556893374717:6747] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:00.372963Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341556893374717:6747] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:00.378960Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341556893374717:6747] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:00.378985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341556893374717:6747] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:00.378989Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341556893374717:6747] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:00.378999Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 635 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:00.389262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:00.390443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:00.390698Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:00.390702Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:00.390716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:00.390794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:00.391086Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341556531561024:6131]
2025-10-09T21:46:00.391701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:00.392137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:00.392188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:00.392570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:00.392621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:00.392623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:00.392625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:00.392626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:00.392627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:00.392628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:00.392629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.392631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.392632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.392638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:00.392648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:00.392652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:00.392657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:00.392660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:00.392664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:00.392668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.392673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.392677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.392684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:00.392693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:00.392699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:00.392705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:00.392710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:00.392717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:00.392721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:00.392727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:00.392731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:00.392736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:00.392741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:00.392746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:00.392752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.392757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.392761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.392766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.392771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.392777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.392804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:00.404055Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 634
2025-10-09T21:46:00.404097Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 634
2025-10-09T21:46:00.402313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:00.403474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:00.403484Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:00.403486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:00.403491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:00.403499Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 634 [2:7559341556531561024:6131]
2025-10-09T21:46:00.403514Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:00.403532Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:00.403536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:00.403540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:00.403542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:00.403561Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:00.403568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:00.403576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:00.403662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.403664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.403666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.403669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:00.403903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:00.403906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.403921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.403934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.403952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:00.403977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341556893374896:6747], now have 1 active actors on pipe
2025-10-09T21:46:00.403980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341556893374698:2432], now have 1 active actors on pipe
2025-10-09T21:46:00.403988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341556531561019:6131]
2025-10-09T21:46:00.403991Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341556531561019:6131]
2025-10-09T21:46:00.403993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341556531561019:6131]
2025-10-09T21:46:00.404006Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:00.404022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:00.404024Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:00.404029Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:00.404031Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:00.404038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:00.404041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:00.404084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341556531561019:6131]
2025-10-09T21:46:00.404087Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:00.404109Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:00.404124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:00.404129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:00.404160Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2b114f1a-b4ae4a1d-d7cce214-bb8b9719_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:00.404168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:00.404172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:00.404175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:00.404186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:00.404197Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:00.404213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:00.405017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:00.405038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:00.405065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:00.405711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 635 partition: 0 MaxSeqNo: 25 sessionId: src_id|2b114f1a-b4ae4a1d-d7cce214-bb8b9719_0
2025-10-09T21:46:00.406186Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df56db31-8f89495b-ef8f7140-ecf1df33_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2b114f1a-b4ae4a1d-d7cce214-bb8b9719_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:00.408963Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b114f1a-b4ae4a1d-d7cce214-bb8b9719_0] Write session: try to update token
2025-10-09T21:46:00.408987Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b114f1a-b4ae4a1d-d7cce214-bb8b9719_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:00.420651Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:00.420671Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:00.420682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341556531561019:6131]
2025-10-09T21:46:00.420732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:00.420932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:00.421022Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:00.421078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:00.421231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:00.421370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:00.421581Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:00.421620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:00.440770Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:00.440795Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:00.440799Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:00.440801Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:00.440815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341556531561019:6131]
2025-10-09T21:46:00.440821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341556531561019:6131]
2025-10-09T21:46:00.440825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341556531561019:6131]
2025-10-09T21:46:00.440876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:00.440883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.440886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:00.441051Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:00.443269Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 635 sessionId: src_id|2b114f1a-b4ae4a1d-d7cce214-bb8b9719_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:00.445873Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:00.447618Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:00.447635Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:00.447639Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:00.455654Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341556893374928:6527]
2025-10-09T21:46:00.456406Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 634, pipe: [1:7559341556893374928:6527]
2025-10-09T21:46:00.640612Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:00.641114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:00.641125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:00.641149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341556893374928:6527], now have 1 active actors on pipe
2025-10-09T21:46:00.642799Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341556893374928:6527]
2025-10-09T21:46:00.642847Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:00.642853Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:00.642917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 635 reason: error at write: Disconnected sessionId: src_id|2b114f1a-b4ae4a1d-d7cce214-bb8b9719_0
2025-10-09T21:46:00.643102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 635 sessionId: src_id|2b114f1a-b4ae4a1d-d7cce214-bb8b9719_0 is DEAD
2025-10-09T21:46:00.643234Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:00.643457Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b114f1a-b4ae4a1d-d7cce214-bb8b9719_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:00.643470Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b114f1a-b4ae4a1d-d7cce214-bb8b9719_0] Write session will restart in 0.008083s
2025-10-09T21:46:00.643497Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b114f1a-b4ae4a1d-d7cce214-bb8b9719_0] Write session: Do CDS request
2025-10-09T21:46:00.643508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b114f1a-b4ae4a1d-d7cce214-bb8b9719_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:00.651975Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:00.655218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b114f1a-b4ae4a1d-d7cce214-bb8b9719_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:00.655759Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:00.655779Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 636
2025-10-09T21:46:00.655971Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 636 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:00.656005Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 636 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:00.656012Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=636 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:00.656017Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:00.658632Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 636 sessionId:  describe result for acl check
2025-10-09T21:46:00.658680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:00.658693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:00.658695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:00.658702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341556893374944:6753] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:00.659295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341556893374944:6753] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:00.675373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341556893374944:6753] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:00.682543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341556893374944:6753] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:00.682566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341556893374944:6753] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:00.682571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341556893374944:6753] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:00.682581Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 636 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:00.707848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:00.711001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:00.711273Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:00.711277Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:00.711291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:00.711358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:00.711401Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341556531561263:6145]
2025-10-09T21:46:00.712093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:00.712683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:00.712738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:00.713068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:00.713123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:00.713124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:00.713126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:00.713127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:00.713128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:00.713130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:00.713131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.713133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.713134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.713139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:00.713151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:00.713155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:00.713158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:00.713161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:00.713165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:00.713169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.713172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.713177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.713185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:00.713193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:00.713200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:00.713206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:00.713212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:00.713218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:00.713222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:00.713228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:00.713232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:00.713238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:00.713243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:00.713249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:00.713255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.713259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.713265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.713272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.713277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.713283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.713313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:00.722996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:00.724199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:00.724206Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:00.724210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:00.724215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:00.724223Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 635 [2:7559341556531561263:6145]
2025-10-09T21:46:00.724240Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:00.724265Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:00.724268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:00.724272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:00.724274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:00.724295Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:00.724302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:00.724310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:00.724383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.724386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.724388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.724391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:00.724474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:00.724477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.724491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:00.724502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:00.724516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:00.724539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341556531561258:6145]
2025-10-09T21:46:00.724542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341556531561258:6145]
2025-10-09T21:46:00.724544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341556531561258:6145]
2025-10-09T21:46:00.724559Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:00.724575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:00.724590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:00.724594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:00.724640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341556531561258:6145]
2025-10-09T21:46:00.724642Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:00.725050Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:00.725061Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:00.725062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:00.725069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:00.725941Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 635
2025-10-09T21:46:00.725977Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 635
2025-10-09T21:46:00.726811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341556893375011:6753], now have 1 active actors on pipe
2025-10-09T21:46:00.726825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341556893374935:2432], now have 1 active actors on pipe
2025-10-09T21:46:00.726842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:00.726848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:00.726887Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8622cadf-67707be1-edb98ee-bb23b0e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:00.726899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:00.726903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:00.726907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:00.726920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:00.726938Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:00.726970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:00.727308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:00.727318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:00.727337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:00.727501Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 636 partition: 0 MaxSeqNo: 25 sessionId: src_id|8622cadf-67707be1-edb98ee-bb23b0e_0
2025-10-09T21:46:00.727890Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b114f1a-b4ae4a1d-d7cce214-bb8b9719_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8622cadf-67707be1-edb98ee-bb23b0e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:00.730611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8622cadf-67707be1-edb98ee-bb23b0e_0] Write session: try to update token
2025-10-09T21:46:00.730631Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8622cadf-67707be1-edb98ee-bb23b0e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:00.740801Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341556893375151:6527]
2025-10-09T21:46:00.741230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341556893375151:6527], now have 1 active actors on pipe
2025-10-09T21:46:00.741358Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 635, pipe: [1:7559341556893375151:6527]
2025-10-09T21:46:00.752303Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:00.752324Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:00.752327Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:00.752329Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:00.752340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341556531561258:6145]
2025-10-09T21:46:00.752811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341556531561258:6145]
2025-10-09T21:46:00.752817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341556531561258:6145]
2025-10-09T21:46:00.752871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:00.752877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:00.752880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:00.753047Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:00.754228Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:00.754241Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:00.754245Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:00.769531Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 636 sessionId: src_id|8622cadf-67707be1-edb98ee-bb23b0e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:00.772531Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:00.978530Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:00.979170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:00.979181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:00.982364Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:00.983470Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8622cadf-67707be1-edb98ee-bb23b0e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:00.983478Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8622cadf-67707be1-edb98ee-bb23b0e_0] Write session will restart in 0.005540s
2025-10-09T21:46:00.983502Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8622cadf-67707be1-edb98ee-bb23b0e_0] Write session: Do CDS request
2025-10-09T21:46:00.983509Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8622cadf-67707be1-edb98ee-bb23b0e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:00.982743Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:00.982828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 636 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8622cadf-67707be1-edb98ee-bb23b0e_0
2025-10-09T21:46:00.983023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 636 sessionId: src_id|8622cadf-67707be1-edb98ee-bb23b0e_0 is DEAD
2025-10-09T21:46:00.983184Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:00.983218Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341556893375151:6527]
2025-10-09T21:46:00.990611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8622cadf-67707be1-edb98ee-bb23b0e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:00.991141Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:00.991159Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 637
2025-10-09T21:46:00.991334Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 637 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:00.991367Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 637 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:00.991374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=637 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:00.991377Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:00.992374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 637 sessionId:  describe result for acl check
2025-10-09T21:46:00.992412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:00.992417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:00.992419Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:00.992424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341556893375177:6757] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:00.992982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341556893375177:6757] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:00.998836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341556893375177:6757] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:01.019334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341556893375177:6757] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:01.019356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341556893375177:6757] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:01.019362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341556893375177:6757] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:01.019374Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 637 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:01.041356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:01.042120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:01.042315Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:01.042319Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:01.042333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:01.042381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:01.042425Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341560826528768:6147]
2025-10-09T21:46:01.043085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:01.044142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:01.044196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:01.044531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:01.044587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:01.044588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:01.044589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:01.044590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:01.044591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:01.044593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:01.044594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.044595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.044596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.044602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:01.044612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:01.044616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:01.044619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:01.044623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:01.044627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:01.044631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.044634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.044638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.044647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:01.044656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:01.044663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:01.044668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:01.044672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:01.044677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:01.044681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:01.044686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:01.044691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:01.044697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:01.044702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:01.044706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:01.044712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.044716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.044721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.044726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.044731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.044736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.044768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:01.060765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:01.062500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:01.062508Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:01.062511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:01.062515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:01.062524Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 636 [2:7559341560826528768:6147]
2025-10-09T21:46:01.062541Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:01.062565Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:01.062569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:01.062573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:01.062575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:01.062595Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:01.062603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:01.062611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:01.062684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.062686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.062688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.062691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:01.062782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:01.062786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.062802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.062812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.062827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:01.062853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341560826528762:6147]
2025-10-09T21:46:01.062856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341560826528762:6147]
2025-10-09T21:46:01.062858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341560826528762:6147]
2025-10-09T21:46:01.062874Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:01.062893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:01.062907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:01.062911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:01.062955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341560826528762:6147]
2025-10-09T21:46:01.062958Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:01.063605Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:01.063616Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:01.063619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:01.063625Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:01.064640Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 636
2025-10-09T21:46:01.064674Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 636
2025-10-09T21:46:01.068368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341561188342603:6757], now have 1 active actors on pipe
2025-10-09T21:46:01.068381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341556893375161:2432], now have 1 active actors on pipe
2025-10-09T21:46:01.068408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:01.068415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:01.068456Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|72527932-6a68424e-a48094dd-6662a5f2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:01.068469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:01.068473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:01.068476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:01.068487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:01.068502Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:01.068522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:01.068841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:01.068846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:01.068857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:01.068985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 637 partition: 0 MaxSeqNo: 25 sessionId: src_id|72527932-6a68424e-a48094dd-6662a5f2_0
2025-10-09T21:46:01.069374Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8622cadf-67707be1-edb98ee-bb23b0e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|72527932-6a68424e-a48094dd-6662a5f2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:01.072218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72527932-6a68424e-a48094dd-6662a5f2_0] Write session: try to update token
2025-10-09T21:46:01.072235Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72527932-6a68424e-a48094dd-6662a5f2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:01.086563Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:01.086583Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:01.086594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341560826528762:6147]
2025-10-09T21:46:01.086647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:01.086647Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:01.086825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:01.086962Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:01.087110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:01.087244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:01.087434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:01.087475Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:01.087654Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341561188342679:6527]
2025-10-09T21:46:01.089382Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 636, pipe: [1:7559341561188342679:6527]
2025-10-09T21:46:01.089470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341561188342679:6527], now have 1 active actors on pipe
2025-10-09T21:46:01.101778Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 637 sessionId: src_id|72527932-6a68424e-a48094dd-6662a5f2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:01.104747Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:01.105170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:01.105194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:01.105237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:01.106388Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:01.116328Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:01.116350Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:01.116352Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:01.116354Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:01.116366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341560826528762:6147]
2025-10-09T21:46:01.116371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341560826528762:6147]
2025-10-09T21:46:01.116378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341560826528762:6147]
2025-10-09T21:46:01.116422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:01.116430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.116433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:01.116578Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:01.118290Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:01.118307Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:01.118310Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:01.320095Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:01.332356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:01.332391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:01.332569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:01.332633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:01.332699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:01.332756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:01.332804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:01.332809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:01.332876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:01.332941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:01.333008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:01.333076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:01.333129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:01.333139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:01.333207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:01.333273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:01.333338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:01.333405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:01.333455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:01.333462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:01.333528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:01.333600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:01.333683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:01.333751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:01.333799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:01.333807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:01.333879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:01.333946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:01.334023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:01.334982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:01.335031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:01.335034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:01.335122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:01.338493Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341561188342679:6527]
2025-10-09T21:46:01.338534Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:01.338580Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 637 reason: status is not ok: tablet will be restarted right now sessionId: src_id|72527932-6a68424e-a48094dd-6662a5f2_0
2025-10-09T21:46:01.338773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 637 sessionId: src_id|72527932-6a68424e-a48094dd-6662a5f2_0 is DEAD
2025-10-09T21:46:01.338892Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:01.339495Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72527932-6a68424e-a48094dd-6662a5f2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:01.339510Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72527932-6a68424e-a48094dd-6662a5f2_0] Write session will restart in 0.006112s
2025-10-09T21:46:01.339540Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72527932-6a68424e-a48094dd-6662a5f2_0] Write session: Do CDS request
2025-10-09T21:46:01.339547Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72527932-6a68424e-a48094dd-6662a5f2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:01.335180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:01.335245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:01.335301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:01.335341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:01.335344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:01.335411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:01.335485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:01.335545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:01.335612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:01.335655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:01.335658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:01.335728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:01.335786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:01.335846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:01.335915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:01.336050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:01.336052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:01.336126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:01.336184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:01.336246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:01.336318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:01.336359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:01.336361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:01.336431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:01.336492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:01.336554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:01.336619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:01.336664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:01.336666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:01.337125Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:01.347817Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72527932-6a68424e-a48094dd-6662a5f2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:01.348803Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:01.348824Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 638
2025-10-09T21:46:01.348995Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 638 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:01.349036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 638 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:01.349042Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=638 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:01.349045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:01.352917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 638 sessionId:  describe result for acl check
2025-10-09T21:46:01.352975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:01.352978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:01.352979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:01.352986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341561188342692:6762] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:01.353632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341561188342692:6762] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:01.373837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341561188342692:6762] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:01.378936Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341561188342692:6762] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:01.378957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341561188342692:6762] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:01.378961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341561188342692:6762] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:01.378972Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 638 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:01.411553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:01.416001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:01.416814Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:01.416819Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:01.416837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:01.416931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:01.416968Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341560826528983:6149]
2025-10-09T21:46:01.417519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:01.418224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:01.418272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:01.418573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:01.418610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:01.418611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:01.418612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:01.418613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:01.418614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:01.418615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:01.418616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.418617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.418618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.418624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:01.418634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:01.418637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:01.418640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:01.418643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:01.418647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:01.418650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.418653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.418656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.418662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:01.418670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:01.418676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:01.418680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:01.418689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:01.418693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:01.418696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:01.418701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:01.418705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:01.418711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:01.418716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:01.418720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:01.418724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.418728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.418732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.418737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.418741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.418745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.418769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:01.434865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:01.436166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:01.436172Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:01.436175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:01.436180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:01.436188Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 637 [2:7559341560826528983:6149]
2025-10-09T21:46:01.436208Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:01.436226Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:01.436230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:01.436233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:01.436236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:01.436253Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:01.436259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:01.436266Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:01.436323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.436326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.436327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.436330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:01.436400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:01.436402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.436414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.436423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.436436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:01.436457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341560826528974:6149]
2025-10-09T21:46:01.436459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341560826528974:6149]
2025-10-09T21:46:01.436461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341560826528974:6149]
2025-10-09T21:46:01.436470Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:01.436483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:01.436496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:01.436500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:01.436538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341560826528974:6149]
2025-10-09T21:46:01.436542Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:01.436615Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:01.436618Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:01.436620Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:01.436626Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:01.437472Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 637
2025-10-09T21:46:01.437509Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 637
2025-10-09T21:46:01.440866Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341561188342911:6527]
2025-10-09T21:46:01.441839Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 637, pipe: [1:7559341561188342911:6527]
2025-10-09T21:46:01.441921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341561188342794:6762], now have 1 active actors on pipe
2025-10-09T21:46:01.441934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341561188342684:2432], now have 1 active actors on pipe
2025-10-09T21:46:01.441952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:01.441958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:01.441978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341561188342911:6527], now have 1 active actors on pipe
2025-10-09T21:46:01.442008Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|12f8bf6a-2204c180-736334f8-1c98053a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:01.442020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:01.442026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:01.442044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:01.442059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:01.442078Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:01.442115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:01.442435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:01.442441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:01.442454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:01.442596Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 638 partition: 0 MaxSeqNo: 25 sessionId: src_id|12f8bf6a-2204c180-736334f8-1c98053a_0
2025-10-09T21:46:01.443024Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72527932-6a68424e-a48094dd-6662a5f2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|12f8bf6a-2204c180-736334f8-1c98053a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:01.445718Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12f8bf6a-2204c180-736334f8-1c98053a_0] Write session: try to update token
2025-10-09T21:46:01.445739Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12f8bf6a-2204c180-736334f8-1c98053a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:01.479120Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:01.479141Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:01.479144Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:01.479150Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:01.479160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341560826528974:6149]
2025-10-09T21:46:01.479164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341560826528974:6149]
2025-10-09T21:46:01.479167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341560826528974:6149]
2025-10-09T21:46:01.479216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:01.479224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.479226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:01.479374Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:01.483006Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:01.483024Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:01.483033Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:01.509779Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 638 sessionId: src_id|12f8bf6a-2204c180-736334f8-1c98053a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:01.512391Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:01.765232Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:01.765598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:01.765619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:01.765892Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341561188342911:6527]
2025-10-09T21:46:01.765905Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:01.765910Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:01.765938Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:01.765983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 638 reason: error at write: Disconnected sessionId: src_id|12f8bf6a-2204c180-736334f8-1c98053a_0
2025-10-09T21:46:01.767179Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:01.766322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 638 sessionId: src_id|12f8bf6a-2204c180-736334f8-1c98053a_0 is DEAD
2025-10-09T21:46:01.770189Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12f8bf6a-2204c180-736334f8-1c98053a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:01.770200Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12f8bf6a-2204c180-736334f8-1c98053a_0] Write session will restart in 0.006059s
2025-10-09T21:46:01.770227Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12f8bf6a-2204c180-736334f8-1c98053a_0] Write session: Do CDS request
2025-10-09T21:46:01.770238Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12f8bf6a-2204c180-736334f8-1c98053a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:01.777512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12f8bf6a-2204c180-736334f8-1c98053a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:01.777806Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:01.777829Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 639
2025-10-09T21:46:01.777978Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 639 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:01.778022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 639 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:01.778045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=639 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:01.778049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:01.778704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 639 sessionId:  describe result for acl check
2025-10-09T21:46:01.778745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:01.778747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:01.778748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:01.778754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341561188342948:6776] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:01.779358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341561188342948:6776] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:01.785325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341561188342948:6776] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:01.795246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341561188342948:6776] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:01.795270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341561188342948:6776] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:01.795274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341561188342948:6776] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:01.795285Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 639 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:01.835890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:01.840735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:01.841008Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:01.841013Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:01.841026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:01.841088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:01.841154Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341560826529233:6165]
2025-10-09T21:46:01.841771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:01.842300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:01.842364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:01.842833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:01.842915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:01.842916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:01.842918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:01.842919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:01.842920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:01.842921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:01.842922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.842923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.842925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.842931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:01.842942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:01.842946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:01.842950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:01.842954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:01.842957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:01.842960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.842964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.842967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.842976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:01.842987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:01.842993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:01.842998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:01.843003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:01.843008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:01.843012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:01.843017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:01.843021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:01.843026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:01.843030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:01.843035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:01.843040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.843045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.843050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.843053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.843057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.843062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.843092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:01.852075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:01.853184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:01.853191Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:01.853194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:01.853199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:01.853208Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 638 [2:7559341560826529233:6165]
2025-10-09T21:46:01.853225Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:01.853245Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:01.853249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:01.853252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:01.853255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:01.853275Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:01.853281Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:01.853289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:01.853358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.853361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.853363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.853366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:01.853441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:01.853444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.853458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:01.853469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:01.853485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:01.853508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341560826529227:6165]
2025-10-09T21:46:01.853510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341560826529227:6165]
2025-10-09T21:46:01.853512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341560826529227:6165]
2025-10-09T21:46:01.853526Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:01.853545Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:01.853563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:01.853566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:01.853610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341560826529227:6165]
2025-10-09T21:46:01.853613Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:01.853863Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:01.853872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:01.853874Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:01.853882Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:01.854470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341561188342937:2432], now have 1 active actors on pipe
2025-10-09T21:46:01.854484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341561188342988:6776], now have 1 active actors on pipe
2025-10-09T21:46:01.854731Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 638
2025-10-09T21:46:01.854771Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 638
2025-10-09T21:46:01.855253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:01.855278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:01.855310Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d319c96b-c1dad0c3-2869f2f-83c84335_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:01.855323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:01.855328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:01.855332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:01.855345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:01.855361Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:01.855379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:01.855962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:01.855972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:01.855991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:01.856141Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 639 partition: 0 MaxSeqNo: 25 sessionId: src_id|d319c96b-c1dad0c3-2869f2f-83c84335_0
2025-10-09T21:46:01.856594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12f8bf6a-2204c180-736334f8-1c98053a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d319c96b-c1dad0c3-2869f2f-83c84335_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:01.859208Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d319c96b-c1dad0c3-2869f2f-83c84335_0] Write session: try to update token
2025-10-09T21:46:01.859226Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d319c96b-c1dad0c3-2869f2f-83c84335_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:01.870148Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341561188343153:6527]
2025-10-09T21:46:01.870724Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 638, pipe: [1:7559341561188343153:6527]
2025-10-09T21:46:01.870793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341561188343153:6527], now have 1 active actors on pipe
2025-10-09T21:46:01.897575Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:01.897595Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:01.897598Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:01.897599Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:01.897610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341560826529227:6165]
2025-10-09T21:46:01.897613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341560826529227:6165]
2025-10-09T21:46:01.897616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341560826529227:6165]
2025-10-09T21:46:01.897660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:01.897667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:01.897670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:01.897809Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:01.901274Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:01.901291Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:01.901295Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:01.899687Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 639 sessionId: src_id|d319c96b-c1dad0c3-2869f2f-83c84335_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:01.902345Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:02.165890Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:02.166355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:02.166371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:02.167195Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341561188343153:6527]
2025-10-09T21:46:02.167230Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:02.167270Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:02.167279Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:02.167342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 639 reason: error at write: Disconnected sessionId: src_id|d319c96b-c1dad0c3-2869f2f-83c84335_0
2025-10-09T21:46:02.166905Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:02.167530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 639 sessionId: src_id|d319c96b-c1dad0c3-2869f2f-83c84335_0 is DEAD
2025-10-09T21:46:02.170207Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d319c96b-c1dad0c3-2869f2f-83c84335_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:02.170221Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d319c96b-c1dad0c3-2869f2f-83c84335_0] Write session will restart in 0.005866s
2025-10-09T21:46:02.170248Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d319c96b-c1dad0c3-2869f2f-83c84335_0] Write session: Do CDS request
2025-10-09T21:46:02.170257Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d319c96b-c1dad0c3-2869f2f-83c84335_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:02.177586Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d319c96b-c1dad0c3-2869f2f-83c84335_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:02.178001Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:02.178042Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 640
2025-10-09T21:46:02.178232Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 640 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:02.178296Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 640 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:02.178307Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=640 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:02.178312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:02.179415Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 640 sessionId:  describe result for acl check
2025-10-09T21:46:02.179467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:02.179469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:02.179471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:02.179487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341565483310483:6781] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:02.180061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341565483310483:6781] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:02.188389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341565483310483:6781] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:02.194589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341565483310483:6781] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:02.194608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341565483310483:6781] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:02.194612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341565483310483:6781] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:02.194630Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 640 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:02.212497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:02.217586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:02.217868Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:02.217881Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:02.217896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:02.217965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:02.218006Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341565121496741:6167]
2025-10-09T21:46:02.218598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:02.219048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:02.219104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:02.219356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:02.219401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:02.219407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:02.219409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:02.219410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:02.219411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:02.219413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:02.219414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:02.219416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:02.219417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:02.219424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:02.219436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:02.219441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:02.219445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:02.219449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:02.219454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:02.219457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:02.219462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:02.219466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:02.219474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:02.219489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:02.219499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:02.219506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:02.219515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:02.219523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:02.219531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:02.219537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:02.219547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:02.219554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:02.219563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:02.219569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:02.219578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:02.219585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:02.219594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:02.219599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:02.219605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:02.219615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:02.219645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:02.229145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:02.230899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:02.230907Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:02.230910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:02.230915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:02.230922Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 639 [2:7559341565121496741:6167]
2025-10-09T21:46:02.230941Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:02.230961Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:02.230964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:02.230966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:02.230968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:02.230984Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:02.230988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:02.230995Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:02.231051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:02.231052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:02.231053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:02.231056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:02.231137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:02.231140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:02.231155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:02.231164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:02.231179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:02.231202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341565121496736:6167]
2025-10-09T21:46:02.231203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341565121496736:6167]
2025-10-09T21:46:02.231205Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341565121496736:6167]
2025-10-09T21:46:02.231213Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:02.231232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:02.231245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:02.231248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:02.231259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341565483310469:2432], now have 1 active actors on pipe
2025-10-09T21:46:02.231263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341565483310556:6781], now have 1 active actors on pipe
2025-10-09T21:46:02.231264Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:02.231269Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:02.231272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:02.231303Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341565121496736:6167]
2025-10-09T21:46:02.231310Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:02.231330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:02.231914Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 639
2025-10-09T21:46:02.231953Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 639
2025-10-09T21:46:02.234756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 640 partition: 0 MaxSeqNo: 25 sessionId: src_id|66635fa9-983a0584-bdeb3677-a44fd64f_0
2025-10-09T21:46:02.235158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d319c96b-c1dad0c3-2869f2f-83c84335_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|66635fa9-983a0584-bdeb3677-a44fd64f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:02.234182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:02.234203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:02.234241Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|66635fa9-983a0584-bdeb3677-a44fd64f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:02.234254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:02.234260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:02.234263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:02.234278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:02.234295Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:02.234315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:02.234612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:02.234616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:02.234632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:02.237969Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66635fa9-983a0584-bdeb3677-a44fd64f_0] Write session: try to update token
2025-10-09T21:46:02.237993Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66635fa9-983a0584-bdeb3677-a44fd64f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:02.261075Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:02.261098Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:02.261101Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:02.261103Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:02.261114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341565121496736:6167]
2025-10-09T21:46:02.261120Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341565121496736:6167]
2025-10-09T21:46:02.261125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341565121496736:6167]
2025-10-09T21:46:02.261176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:02.261183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:02.261186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:02.261341Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:02.262239Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:02.262249Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:02.262253Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:02.272187Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341565483310689:6527]
2025-10-09T21:46:02.275041Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 639, pipe: [1:7559341565483310689:6527]
2025-10-09T21:46:02.284351Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 640 sessionId: src_id|66635fa9-983a0584-bdeb3677-a44fd64f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:02.286951Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:02.527801Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:02.528609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341565483310689:6527], now have 1 active actors on pipe
2025-10-09T21:46:02.528681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:02.528694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:02.529783Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:02.529852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341565483310689:6527]
2025-10-09T21:46:02.529874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:02.529879Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:02.529938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 640 reason: error at write: Disconnected sessionId: src_id|66635fa9-983a0584-bdeb3677-a44fd64f_0
2025-10-09T21:46:02.530154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 640 sessionId: src_id|66635fa9-983a0584-bdeb3677-a44fd64f_0 is DEAD
2025-10-09T21:46:02.530676Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66635fa9-983a0584-bdeb3677-a44fd64f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:02.530689Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66635fa9-983a0584-bdeb3677-a44fd64f_0] Write session will restart in 0.007036s
2025-10-09T21:46:02.530713Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66635fa9-983a0584-bdeb3677-a44fd64f_0] Write session: Do CDS request
2025-10-09T21:46:02.530723Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66635fa9-983a0584-bdeb3677-a44fd64f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:02.530755Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:02.538581Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66635fa9-983a0584-bdeb3677-a44fd64f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:02.539050Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:02.539074Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 641
2025-10-09T21:46:02.539430Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 641 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:02.539467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 641 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:02.539471Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=641 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:02.539475Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:02.539926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 641 sessionId:  describe result for acl check
2025-10-09T21:46:02.539956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:02.539958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:02.539959Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:02.539964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341565483310701:6785] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:02.540582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341565483310701:6785] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:02.548187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341565483310701:6785] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:02.557511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341565483310701:6785] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:02.557532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341565483310701:6785] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:02.557535Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341565483310701:6785] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:02.557546Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 641 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:02.615675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:02.616559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:02.616795Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:02.616799Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:02.616811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:02.616870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:02.616909Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341565121496972:6171]
2025-10-09T21:46:02.617424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:02.617952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:02.618002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:02.618376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:02.618435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:02.618436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:02.618437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:02.618439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:02.618440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:02.618441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:02.618442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:02.618444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:02.618445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:02.618451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:02.618463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:02.618468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:02.618472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:02.618476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:02.618482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:02.618486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:02.618490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:02.618493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:02.618500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:02.618509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:02.618515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:02.618521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:02.618525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:02.618530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:02.618534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:02.618539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:02.618544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:02.618549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:02.618553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:02.618559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:02.618563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:02.618569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:02.618573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:02.618580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:02.618591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:02.618596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:02.618626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:02.629416Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341565483310921:6527]
2025-10-09T21:46:02.634999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:02.636417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:02.636436Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:02.636439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:02.636443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:02.636451Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 640 [2:7559341565121496972:6171]
2025-10-09T21:46:02.636465Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:02.636485Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:02.636499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:02.636503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:02.636505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:02.636524Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:02.636531Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:02.636538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:02.636601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:02.636603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:02.636605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:02.636608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:02.636826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:02.636830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:02.636844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:02.636859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:02.636878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:02.636903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341565121496967:6171]
2025-10-09T21:46:02.636905Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341565121496967:6171]
2025-10-09T21:46:02.636907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341565121496967:6171]
2025-10-09T21:46:02.636923Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:02.636938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:02.636954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:02.636957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:02.636999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341565121496967:6171]
2025-10-09T21:46:02.637003Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:02.637489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:02.637493Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:02.637495Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:02.637502Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:02.638392Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 640, pipe: [1:7559341565483310921:6527]
2025-10-09T21:46:02.638438Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 640
2025-10-09T21:46:02.638456Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 640
2025-10-09T21:46:02.638711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341565483310921:6527], now have 1 active actors on pipe
2025-10-09T21:46:02.638724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341565483310772:6785], now have 1 active actors on pipe
2025-10-09T21:46:02.638727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341565483310693:2432], now have 1 active actors on pipe
2025-10-09T21:46:02.640745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 641 partition: 0 MaxSeqNo: 25 sessionId: src_id|815553e6-71204629-433b5e62-abc4b953_0
2025-10-09T21:46:02.639976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:02.639991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:02.640035Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|815553e6-71204629-433b5e62-abc4b953_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:02.640049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:02.640054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:02.640056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:02.640070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:02.640087Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:02.640108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:02.640596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:02.640600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:02.640616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:02.645758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66635fa9-983a0584-bdeb3677-a44fd64f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|815553e6-71204629-433b5e62-abc4b953_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:02.648528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|815553e6-71204629-433b5e62-abc4b953_0] Write session: try to update token
2025-10-09T21:46:02.648552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|815553e6-71204629-433b5e62-abc4b953_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:02.674444Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:02.674463Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:02.674465Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:02.674468Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:02.674478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341565121496967:6171]
2025-10-09T21:46:02.674483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341565121496967:6171]
2025-10-09T21:46:02.674487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341565121496967:6171]
2025-10-09T21:46:02.674539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:02.674546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:02.674549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:02.674699Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:02.677572Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:02.677589Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:02.677592Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:02.677055Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 641 sessionId: src_id|815553e6-71204629-433b5e62-abc4b953_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:02.679698Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:02.982238Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:02.982686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:02.982697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:02.983091Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:02.986509Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341565483310921:6527]
2025-10-09T21:46:02.986552Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:02.986637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 641 reason: status is not ok: tablet will be restarted right now sessionId: src_id|815553e6-71204629-433b5e62-abc4b953_0
2025-10-09T21:46:02.986820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 641 sessionId: src_id|815553e6-71204629-433b5e62-abc4b953_0 is DEAD
2025-10-09T21:46:02.986946Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:02.987152Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|815553e6-71204629-433b5e62-abc4b953_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:02.987162Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|815553e6-71204629-433b5e62-abc4b953_0] Write session will restart in 0.009169s
2025-10-09T21:46:02.987186Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|815553e6-71204629-433b5e62-abc4b953_0] Write session: Do CDS request
2025-10-09T21:46:02.987192Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|815553e6-71204629-433b5e62-abc4b953_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:02.998563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|815553e6-71204629-433b5e62-abc4b953_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:02.998959Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:02.998979Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 642
2025-10-09T21:46:02.999123Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 642 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:02.999176Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 642 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:02.999183Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=642 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:02.999187Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:02.999610Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 642 sessionId:  describe result for acl check
2025-10-09T21:46:02.999640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:02.999642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:02.999644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:02.999650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341565483310967:6797] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:03.000295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341565483310967:6797] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:03.004942Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341565483310967:6797] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:03.015538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341565483310967:6797] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:03.015560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341565483310967:6797] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:03.015564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341565483310967:6797] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:03.015577Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 642 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:03.029135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:03.036462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:03.036740Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:03.036745Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:03.036758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:03.036811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:03.036852Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341569416464507:6185]
2025-10-09T21:46:03.037457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:03.038369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:03.038423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:03.038789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:03.038833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:03.038834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:03.038836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:03.038837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:03.038838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:03.038839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:03.038841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.038842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.038843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.038848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:03.038858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:03.038862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:03.038865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:03.038869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:03.038872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:03.038875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.038879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.038882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.038890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:03.038898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:03.038906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:03.038911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:03.038917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:03.038922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:03.038925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:03.038930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:03.038934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:03.038938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:03.038942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:03.038946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:03.038951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.038956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.038961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.038965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.038969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.038974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.039002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:03.052319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:03.055881Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 641
2025-10-09T21:46:03.055929Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 641
2025-10-09T21:46:03.054796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:03.054816Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:03.054818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:03.054822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:03.054831Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 641 [2:7559341569416464507:6185]
2025-10-09T21:46:03.054853Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:03.054877Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:03.054880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:03.054884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:03.054886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:03.054908Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:03.054916Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:03.054924Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:03.055002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.055006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.055008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.055011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:03.055099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:03.055102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.055118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.055131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.055149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:03.055173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341569416464501:6185]
2025-10-09T21:46:03.055175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341569416464501:6185]
2025-10-09T21:46:03.055177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341569416464501:6185]
2025-10-09T21:46:03.055191Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:03.055207Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:03.055223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:03.055226Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:03.055274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341569416464501:6185]
2025-10-09T21:46:03.055277Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:03.055370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:03.055372Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:03.055374Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:03.055379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:03.058131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341569778278411:6797], now have 1 active actors on pipe
2025-10-09T21:46:03.058141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341565483310951:2432], now have 1 active actors on pipe
2025-10-09T21:46:03.060850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:03.060876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:03.060930Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cd4a5bbe-513713ad-5566e2ae-c16ddbcd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:03.060943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:03.060949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:03.060953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:03.060970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:03.060990Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:03.061015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:03.062115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:03.062132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:03.062161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:03.064666Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 642 partition: 0 MaxSeqNo: 25 sessionId: src_id|cd4a5bbe-513713ad-5566e2ae-c16ddbcd_0
2025-10-09T21:46:03.065100Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|815553e6-71204629-433b5e62-abc4b953_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cd4a5bbe-513713ad-5566e2ae-c16ddbcd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:03.068056Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd4a5bbe-513713ad-5566e2ae-c16ddbcd_0] Write session: try to update token
2025-10-09T21:46:03.068084Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd4a5bbe-513713ad-5566e2ae-c16ddbcd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:03.086989Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341569778278472:6527]
2025-10-09T21:46:03.089493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341569778278472:6527], now have 1 active actors on pipe
2025-10-09T21:46:03.089629Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 641, pipe: [1:7559341569778278472:6527]
2025-10-09T21:46:03.112470Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:03.112494Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:03.112496Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:03.112499Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:03.112510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341569416464501:6185]
2025-10-09T21:46:03.112515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341569416464501:6185]
2025-10-09T21:46:03.112519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341569416464501:6185]
2025-10-09T21:46:03.112569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:03.112577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.112581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:03.112743Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:03.114330Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:03.114344Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:03.114351Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:03.123931Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 642 sessionId: src_id|cd4a5bbe-513713ad-5566e2ae-c16ddbcd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:03.127130Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:03.356981Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:03.357291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:03.357302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:03.357674Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:03.358255Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:03.358681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd4a5bbe-513713ad-5566e2ae-c16ddbcd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:03.358692Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd4a5bbe-513713ad-5566e2ae-c16ddbcd_0] Write session will restart in 0.008870s
2025-10-09T21:46:03.358716Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd4a5bbe-513713ad-5566e2ae-c16ddbcd_0] Write session: Do CDS request
2025-10-09T21:46:03.358724Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd4a5bbe-513713ad-5566e2ae-c16ddbcd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:03.358341Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 642 reason: status is not ok: tablet will be restarted right now sessionId: src_id|cd4a5bbe-513713ad-5566e2ae-c16ddbcd_0
2025-10-09T21:46:03.358543Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 642 sessionId: src_id|cd4a5bbe-513713ad-5566e2ae-c16ddbcd_0 is DEAD
2025-10-09T21:46:03.358726Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341569778278472:6527]
2025-10-09T21:46:03.358741Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:03.368563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd4a5bbe-513713ad-5566e2ae-c16ddbcd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:03.368964Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:03.369282Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 643
2025-10-09T21:46:03.369729Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 643 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:03.369765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 643 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:03.369771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=643 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:03.369774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:03.370518Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 643 sessionId:  describe result for acl check
2025-10-09T21:46:03.370560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:03.370567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:03.370568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:03.370574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341569778278487:6802] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:03.371177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341569778278487:6802] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:03.380293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341569778278487:6802] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:03.384517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341569778278487:6802] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:03.384533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341569778278487:6802] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:03.384536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341569778278487:6802] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:03.384546Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 643 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:03.414107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:03.417238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:03.417432Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:03.417443Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:03.417456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:03.417511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:03.417556Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341569416464722:6187]
2025-10-09T21:46:03.417963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:03.418377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:03.418424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:03.418703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:03.418759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:03.418765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:03.418766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:03.418768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:03.418769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:03.418771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:03.418772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.418773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.418774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.418780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:03.418791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:03.418795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:03.418798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:03.418803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:03.418807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:03.418811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.418815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.418819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.418827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:03.418835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:03.418842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:03.418847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:03.418852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:03.418858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:03.418868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:03.418874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:03.418883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:03.418888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:03.418897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:03.418903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:03.418912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.418918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.418927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.418933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.418942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.418953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.418986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:03.427862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:03.428985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:03.429003Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:03.429005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:03.429010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:03.429018Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 642 [2:7559341569416464722:6187]
2025-10-09T21:46:03.429036Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:03.429055Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:03.429058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:03.429061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:03.429063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:03.429081Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:03.429093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:03.429105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:03.429166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.429172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.429174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.429177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:03.429688Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 642
2025-10-09T21:46:03.429724Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 642
2025-10-09T21:46:03.429247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:03.429254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.429266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.429275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.429290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:03.429312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341569416464716:6187]
2025-10-09T21:46:03.429314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341569416464716:6187]
2025-10-09T21:46:03.429316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341569416464716:6187]
2025-10-09T21:46:03.429324Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:03.429330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:03.429333Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:03.429335Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:03.429339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:03.429353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:03.429356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:03.429368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341569778278525:6802], now have 1 active actors on pipe
2025-10-09T21:46:03.429371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341569778278478:2432], now have 1 active actors on pipe
2025-10-09T21:46:03.429408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341569416464716:6187]
2025-10-09T21:46:03.429414Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:03.429904Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:03.430584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:03.430599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:03.430636Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|53f2f090-6bf5ab28-57e3ad89-e5018ba_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:03.430647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:03.430652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:03.430655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:03.430667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:03.430682Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:03.430713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:03.431031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:03.431039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:03.431055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:03.431519Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd4a5bbe-513713ad-5566e2ae-c16ddbcd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|53f2f090-6bf5ab28-57e3ad89-e5018ba_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:03.434241Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53f2f090-6bf5ab28-57e3ad89-e5018ba_0] Write session: try to update token
2025-10-09T21:46:03.434255Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53f2f090-6bf5ab28-57e3ad89-e5018ba_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:03.431164Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 643 partition: 0 MaxSeqNo: 25 sessionId: src_id|53f2f090-6bf5ab28-57e3ad89-e5018ba_0
2025-10-09T21:46:03.459630Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341569778278699:6527]
2025-10-09T21:46:03.460638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341569778278699:6527], now have 1 active actors on pipe
2025-10-09T21:46:03.460832Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 642, pipe: [1:7559341569778278699:6527]
2025-10-09T21:46:03.470303Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 643 sessionId: src_id|53f2f090-6bf5ab28-57e3ad89-e5018ba_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:03.473396Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:03.474025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:03.474092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:03.474140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:03.477310Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:03.486740Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:03.486761Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:03.486764Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:03.486766Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:03.486777Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341569416464716:6187]
2025-10-09T21:46:03.486782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341569416464716:6187]
2025-10-09T21:46:03.486786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341569416464716:6187]
2025-10-09T21:46:03.486838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:03.486848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.486851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:03.487008Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:03.492250Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:03.492277Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:03.492281Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:03.733657Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:03.744581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:03.744604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:03.744760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:03.744821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:03.744881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:03.744941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:03.744983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:03.744987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:03.745055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:03.745119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:03.745182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:03.745244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:03.745287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:03.745289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:03.745354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:03.745415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:03.745479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:03.745542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:03.745582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:03.745585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:03.745663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:03.745739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:03.745804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:03.745870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:03.745925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:03.745928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:03.745997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:03.746095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:03.746158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:03.746221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:03.746279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:03.746282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:03.746348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:03.746756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:03.746820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:03.746883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:03.746926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:03.746928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:03.746995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:03.747055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:03.747121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:03.747184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:03.747226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:03.747228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:03.747297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:03.747357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:03.747420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:03.747485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:03.747545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:03.747547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:03.747613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:03.747676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:03.747739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:03.747807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:03.747848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:03.747850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:03.747919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:03.747978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:03.748037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:03.748097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:03.748138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:03.748140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:03.750172Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:03.754653Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341569778278699:6527]
2025-10-09T21:46:03.754695Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:03.754734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 643 reason: status is not ok: tablet will be restarted right now sessionId: src_id|53f2f090-6bf5ab28-57e3ad89-e5018ba_0
2025-10-09T21:46:03.754916Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 643 sessionId: src_id|53f2f090-6bf5ab28-57e3ad89-e5018ba_0 is DEAD
2025-10-09T21:46:03.755034Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:03.758717Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53f2f090-6bf5ab28-57e3ad89-e5018ba_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:03.758728Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53f2f090-6bf5ab28-57e3ad89-e5018ba_0] Write session will restart in 0.008309s
2025-10-09T21:46:03.758752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53f2f090-6bf5ab28-57e3ad89-e5018ba_0] Write session: Do CDS request
2025-10-09T21:46:03.758761Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53f2f090-6bf5ab28-57e3ad89-e5018ba_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:03.773293Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53f2f090-6bf5ab28-57e3ad89-e5018ba_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:03.773722Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:03.773742Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 644
2025-10-09T21:46:03.773889Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 644 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:03.773917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 644 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:03.773922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=644 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:03.773926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:03.775585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 644 sessionId:  describe result for acl check
2025-10-09T21:46:03.775632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:03.775634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:03.775636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:03.775643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341569778278753:6813] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:03.776203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341569778278753:6813] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:03.786281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341569778278753:6813] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:03.804350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341569778278753:6813] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:03.804367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341569778278753:6813] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:03.804370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341569778278753:6813] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:03.804379Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 644 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:03.825144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:03.830479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:03.831113Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:03.831117Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:03.831129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:03.831185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:03.831222Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341569416464964:6197]
2025-10-09T21:46:03.831747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:03.832225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:03.832260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:03.832530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:03.832567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:03.832569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:03.832570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:03.832572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:03.832573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:03.832574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:03.832575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.832576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.832578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.832583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:03.832592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:03.832596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:03.832599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:03.832602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:03.832606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:03.832610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.832614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.832617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.832624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:03.832631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:03.832636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:03.832641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:03.832645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:03.832649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:03.832652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:03.832657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:03.832661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:03.832665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:03.832669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:03.832674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:03.832678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.832684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.832690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.832694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.832697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.832701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.832726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:03.845402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:03.847184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:03.847201Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:03.847203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:03.847207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:03.847214Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 643 [2:7559341569416464964:6197]
2025-10-09T21:46:03.847229Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:03.847259Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:03.847262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:03.847265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:03.847268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:03.847286Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:03.847295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:03.847302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:03.847366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.847369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.847370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.847373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:03.847446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:03.847448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.847460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:03.847470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:03.847484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:03.847504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341569416464959:6197]
2025-10-09T21:46:03.847506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341569416464959:6197]
2025-10-09T21:46:03.847508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341569416464959:6197]
2025-10-09T21:46:03.847522Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:03.847537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:03.847551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:03.847554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:03.847593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341569416464959:6197]
2025-10-09T21:46:03.847595Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:03.848560Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 643
2025-10-09T21:46:03.848600Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 643
2025-10-09T21:46:03.847863Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:03.847870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:03.847872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:03.847878Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:03.850053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341569778278932:6813], now have 1 active actors on pipe
2025-10-09T21:46:03.850070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341569778278723:2432], now have 1 active actors on pipe
2025-10-09T21:46:03.850101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:03.850107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:03.850140Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1bf80b6d-963cb7ba-681e710f-9887a295_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:03.850150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:03.850155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:03.850158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:03.850168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:03.850183Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:03.850202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:03.856548Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 644 partition: 0 MaxSeqNo: 25 sessionId: src_id|1bf80b6d-963cb7ba-681e710f-9887a295_0
2025-10-09T21:46:03.857964Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341569778278950:6527]
2025-10-09T21:46:03.856091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:03.856109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:03.856149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:03.858427Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53f2f090-6bf5ab28-57e3ad89-e5018ba_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1bf80b6d-963cb7ba-681e710f-9887a295_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:03.861166Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 643, pipe: [1:7559341569778278950:6527]
2025-10-09T21:46:03.861317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341569778278950:6527], now have 1 active actors on pipe
2025-10-09T21:46:03.861454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1bf80b6d-963cb7ba-681e710f-9887a295_0] Write session: try to update token
2025-10-09T21:46:03.861469Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1bf80b6d-963cb7ba-681e710f-9887a295_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:03.895465Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:03.895485Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:03.895488Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:03.895489Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:03.895499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341569416464959:6197]
2025-10-09T21:46:03.895504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341569416464959:6197]
2025-10-09T21:46:03.895508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341569416464959:6197]
2025-10-09T21:46:03.895549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:03.895556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:03.895558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:03.895709Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:03.898870Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:03.898888Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:03.898898Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:03.918311Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 644 sessionId: src_id|1bf80b6d-963cb7ba-681e710f-9887a295_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:03.921570Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
E1009 21:46:04.060935712 3624865 chttp2_transport.cc:1143]             ipv4:127.0.0.1:14704: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 2500ms
2025-10-09T21:46:04.213018Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:04.214162Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:04.214180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:04.214249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 644 reason: error at write: Disconnected sessionId: src_id|1bf80b6d-963cb7ba-681e710f-9887a295_0
2025-10-09T21:46:04.214907Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bf80b6d-963cb7ba-681e710f-9887a295_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:04.214920Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bf80b6d-963cb7ba-681e710f-9887a295_0] Write session will restart in 0.005425s
2025-10-09T21:46:04.214944Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bf80b6d-963cb7ba-681e710f-9887a295_0] Write session: Do CDS request
2025-10-09T21:46:04.214951Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bf80b6d-963cb7ba-681e710f-9887a295_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:04.213318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:04.213338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:04.215927Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:04.214439Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 644 sessionId: src_id|1bf80b6d-963cb7ba-681e710f-9887a295_0 is DEAD
2025-10-09T21:46:04.214584Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:04.214621Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341569778278950:6527]
2025-10-09T21:46:04.221550Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1bf80b6d-963cb7ba-681e710f-9887a295_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:04.221985Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:04.222004Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 645
2025-10-09T21:46:04.222201Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 645 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:04.222235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 645 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:04.222241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=645 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:04.222245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:04.225108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 645 sessionId:  describe result for acl check
2025-10-09T21:46:04.225157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:04.225159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:04.225161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:04.225167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341574073246272:6819] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:04.225716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341574073246272:6819] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:04.251819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341574073246272:6819] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:04.258092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341574073246272:6819] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:04.258112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341574073246272:6819] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:04.258117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341574073246272:6819] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:04.258128Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 645 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:04.288146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:04.298237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:04.298543Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:04.298549Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:04.298563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:04.298631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:04.298683Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341573711432495:6205]
2025-10-09T21:46:04.299320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:04.300149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:04.300203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:04.300549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:04.300591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:04.300592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:04.300594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:04.300595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:04.300596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:04.300597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:04.300599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.300600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:04.300602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:04.300608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:04.300618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:04.300622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:04.300626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:04.300630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:04.300634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:04.300638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.300642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:04.300645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:04.300654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:04.300662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:04.300671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:04.300676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:04.300681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:04.300686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:04.300690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:04.300695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:04.300700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:04.300704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:04.300709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:04.300714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:04.300719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.300724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.300744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:04.300750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:04.300754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:04.300759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:04.300789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:04.312516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:04.314205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:04.314212Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:04.314215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:04.314219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:04.314227Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 644 [2:7559341573711432495:6205]
2025-10-09T21:46:04.314242Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:04.314266Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:04.314270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:04.314273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:04.314275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:04.314296Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:04.314302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:04.314310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:04.314382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.314384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:04.314386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:04.314389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:04.314468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:04.314471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.314484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:04.314494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:04.314508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:04.314530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341573711432489:6205]
2025-10-09T21:46:04.314533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341573711432489:6205]
2025-10-09T21:46:04.314535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341573711432489:6205]
2025-10-09T21:46:04.314550Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:04.314566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:04.314583Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:04.314586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:04.314629Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341573711432489:6205]
2025-10-09T21:46:04.314632Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:04.315146Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:04.315158Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:04.315161Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:04.315167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:04.315896Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 644
2025-10-09T21:46:04.315927Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 644
2025-10-09T21:46:04.319494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341574073246366:6819], now have 1 active actors on pipe
2025-10-09T21:46:04.319510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341574073246264:2432], now have 1 active actors on pipe
2025-10-09T21:46:04.319532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:04.319538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:04.319581Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b69b6d6f-a9a3bab8-1dc30155-2f8e35f0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:04.319594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:04.319599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:04.319603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:04.319623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:04.319641Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:04.319663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:04.322708Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341574073246485:6527]
2025-10-09T21:46:04.324724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:04.324742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:04.324776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:04.325944Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 645 partition: 0 MaxSeqNo: 25 sessionId: src_id|b69b6d6f-a9a3bab8-1dc30155-2f8e35f0_0
2025-10-09T21:46:04.326395Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bf80b6d-963cb7ba-681e710f-9887a295_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b69b6d6f-a9a3bab8-1dc30155-2f8e35f0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:04.329276Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b69b6d6f-a9a3bab8-1dc30155-2f8e35f0_0] Write session: try to update token
2025-10-09T21:46:04.329300Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b69b6d6f-a9a3bab8-1dc30155-2f8e35f0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:04.336311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341574073246485:6527], now have 1 active actors on pipe
2025-10-09T21:46:04.336583Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 644, pipe: [1:7559341574073246485:6527]
2025-10-09T21:46:04.363199Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:04.363217Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:04.363220Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:04.363222Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:04.363234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341573711432489:6205]
2025-10-09T21:46:04.363239Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341573711432489:6205]
2025-10-09T21:46:04.363242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341573711432489:6205]
2025-10-09T21:46:04.363286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:04.363295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.363298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:04.363441Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:04.371133Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:04.371150Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:04.371156Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:04.383795Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 645 sessionId: src_id|b69b6d6f-a9a3bab8-1dc30155-2f8e35f0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:04.386666Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:04.569375Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:04.569922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:04.569939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:04.570542Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:04.570557Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:04.570614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 645 reason: error at write: Disconnected sessionId: src_id|b69b6d6f-a9a3bab8-1dc30155-2f8e35f0_0
2025-10-09T21:46:04.570839Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:04.570881Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341574073246485:6527]
2025-10-09T21:46:04.570884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 645 sessionId: src_id|b69b6d6f-a9a3bab8-1dc30155-2f8e35f0_0 is DEAD
2025-10-09T21:46:04.570413Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:04.571541Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b69b6d6f-a9a3bab8-1dc30155-2f8e35f0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:04.571551Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b69b6d6f-a9a3bab8-1dc30155-2f8e35f0_0] Write session will restart in 0.008182s
2025-10-09T21:46:04.571574Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b69b6d6f-a9a3bab8-1dc30155-2f8e35f0_0] Write session: Do CDS request
2025-10-09T21:46:04.571584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b69b6d6f-a9a3bab8-1dc30155-2f8e35f0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:04.582821Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b69b6d6f-a9a3bab8-1dc30155-2f8e35f0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:04.583318Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:04.583337Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 646
2025-10-09T21:46:04.586428Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 646 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:04.586494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 646 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:04.586500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=646 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:04.586505Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:04.587078Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 646 sessionId:  describe result for acl check
2025-10-09T21:46:04.587114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:04.587116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:04.587119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:04.587125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341574073246507:6823] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:04.587682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341574073246507:6823] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:04.595269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341574073246507:6823] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:04.609500Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341574073246507:6823] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:04.609520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341574073246507:6823] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:04.609525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341574073246507:6823] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:04.609535Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 646 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:04.631469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:04.636423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:04.636707Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:04.636717Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:04.636729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:04.636787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:04.636832Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341573711432717:6209]
2025-10-09T21:46:04.637438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:04.638274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:04.638335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:04.638678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:04.639226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:04.639234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:04.639235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:04.639237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:04.639238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:04.639239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:04.639240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.639241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:04.639242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:04.639250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:04.639262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:04.639266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:04.639269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:04.639272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:04.639275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:04.639278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.639281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:04.639285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:04.639294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:04.639304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:04.639310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:04.639315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:04.639319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:04.639325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:04.639329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:04.639334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:04.639338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:04.639344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:04.639348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:04.639352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:04.639358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.639362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.639367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:04.639372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:04.639376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:04.639382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:04.639423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:04.650408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:04.652038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:04.652045Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:04.652048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:04.652053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:04.652061Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 645 [2:7559341573711432717:6209]
2025-10-09T21:46:04.652078Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:04.652101Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:04.652104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:04.652108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:04.652111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:04.652130Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:04.652137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:04.652145Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:04.652219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.652221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:04.652223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:04.652226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:04.652302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:04.652305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.652319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:04.652330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:04.652345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:04.652386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341573711432712:6209]
2025-10-09T21:46:04.652389Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341573711432712:6209]
2025-10-09T21:46:04.652391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341573711432712:6209]
2025-10-09T21:46:04.652400Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:04.652417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:04.652431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:04.652434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:04.652465Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:04.652471Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:04.652473Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:04.652478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341573711432712:6209]
2025-10-09T21:46:04.652482Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:04.652487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:04.652507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341574073246493:2432], now have 1 active actors on pipe
2025-10-09T21:46:04.652512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341574073246595:6823], now have 1 active actors on pipe
2025-10-09T21:46:04.652726Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 645
2025-10-09T21:46:04.652795Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 645
2025-10-09T21:46:04.653877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:04.653894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:04.653928Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8c2776d-b6873453-c38ccc54-2e49b39c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:04.653939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:04.653944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:04.653947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:04.653960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:04.653975Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:04.654010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:04.654695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:04.654709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:04.654730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:04.654921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 646 partition: 0 MaxSeqNo: 25 sessionId: src_id|8c2776d-b6873453-c38ccc54-2e49b39c_0
2025-10-09T21:46:04.658538Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b69b6d6f-a9a3bab8-1dc30155-2f8e35f0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8c2776d-b6873453-c38ccc54-2e49b39c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:04.661366Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c2776d-b6873453-c38ccc54-2e49b39c_0] Write session: try to update token
2025-10-09T21:46:04.661389Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c2776d-b6873453-c38ccc54-2e49b39c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:04.670540Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341574073246719:6527]
2025-10-09T21:46:04.671010Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 645, pipe: [1:7559341574073246719:6527]
2025-10-09T21:46:04.671077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341574073246719:6527], now have 1 active actors on pipe
2025-10-09T21:46:04.684814Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:04.684836Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:04.684839Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:04.684841Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:04.684851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341573711432712:6209]
2025-10-09T21:46:04.684855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341573711432712:6209]
2025-10-09T21:46:04.684860Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341573711432712:6209]
2025-10-09T21:46:04.684904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:04.684912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.684915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:04.685068Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:04.685185Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:04.685191Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:04.685195Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:04.708596Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 646 sessionId: src_id|8c2776d-b6873453-c38ccc54-2e49b39c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:04.711649Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:04.913297Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:04.913770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:04.913790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:04.915387Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c2776d-b6873453-c38ccc54-2e49b39c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:04.915397Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c2776d-b6873453-c38ccc54-2e49b39c_0] Write session will restart in 0.008506s
2025-10-09T21:46:04.914926Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:04.914981Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:04.914984Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:04.915042Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 646 reason: error at write: Disconnected sessionId: src_id|8c2776d-b6873453-c38ccc54-2e49b39c_0
2025-10-09T21:46:04.915221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 646 sessionId: src_id|8c2776d-b6873453-c38ccc54-2e49b39c_0 is DEAD
2025-10-09T21:46:04.915421Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c2776d-b6873453-c38ccc54-2e49b39c_0] Write session: Do CDS request
2025-10-09T21:46:04.915428Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c2776d-b6873453-c38ccc54-2e49b39c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:04.915355Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341574073246719:6527]
2025-10-09T21:46:04.916101Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:04.926842Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c2776d-b6873453-c38ccc54-2e49b39c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:04.927451Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:04.927470Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 647
2025-10-09T21:46:04.927673Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 647 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:04.927707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 647 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:04.927712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=647 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:04.927715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:04.928727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 647 sessionId:  describe result for acl check
2025-10-09T21:46:04.928766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:04.928768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:04.928769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:04.928774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341574073246768:6835] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:04.929285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341574073246768:6835] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:04.936288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341574073246768:6835] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:04.947914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341574073246768:6835] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:04.947934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341574073246768:6835] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:04.947938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341574073246768:6835] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:04.947949Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 647 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:04.972990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:04.978875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:04.979199Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:04.979204Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:04.979220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:04.979291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:04.979333Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341573711432975:6219]
2025-10-09T21:46:04.979942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:04.990428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:04.990558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:04.990963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:04.991035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:04.991037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:04.991039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:04.991040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:04.991042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:04.991043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:04.991044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.991045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:04.991046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:04.991054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:04.991065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:04.991069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:04.991073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:04.991077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:04.991081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:04.991085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.991089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:04.991093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:04.991102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:04.991111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:04.991117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:04.991124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:04.991129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:04.991154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:04.991159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:04.991165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:04.991169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:04.991175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:04.991179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:04.991185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:04.991190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.991196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:04.991200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:04.991205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:04.991210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:04.991215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:04.991247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:05.011668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:05.012842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:05.012850Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:05.012854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:05.012858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:05.012867Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 646 [2:7559341573711432975:6219]
2025-10-09T21:46:05.012886Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:05.012910Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:05.012914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:05.012917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:05.012920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:05.012940Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:05.012946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:05.012955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:05.013023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.013026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:05.013028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:05.013031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:05.013116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:05.013119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.013134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:05.013147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:05.013164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:05.013186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341573711432969:6219]
2025-10-09T21:46:05.013189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341573711432969:6219]
2025-10-09T21:46:05.013191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341573711432969:6219]
2025-10-09T21:46:05.013209Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:05.013226Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:05.013242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:05.013245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:05.013290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341573711432969:6219]
2025-10-09T21:46:05.013293Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:05.015809Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341578368214270:6527]
2025-10-09T21:46:05.015985Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:05.016002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:05.016004Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:05.016012Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:05.016464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341574073246753:2432], now have 1 active actors on pipe
2025-10-09T21:46:05.016475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341574073246888:6835], now have 1 active actors on pipe
2025-10-09T21:46:05.016955Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 646
2025-10-09T21:46:05.017002Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 646
2025-10-09T21:46:05.019824Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c2776d-b6873453-c38ccc54-2e49b39c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ea36a6c7-7fee13de-96d4c146-e9854478_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:05.017491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:05.017508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:05.017539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341578368214270:6527], now have 1 active actors on pipe
2025-10-09T21:46:05.017563Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ea36a6c7-7fee13de-96d4c146-e9854478_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:05.017576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:05.017581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:05.017585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:05.017597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:05.017616Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:05.017638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:05.019106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:05.019113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:05.019141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:05.018449Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 646, pipe: [1:7559341578368214270:6527]
2025-10-09T21:46:05.019344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 647 partition: 0 MaxSeqNo: 25 sessionId: src_id|ea36a6c7-7fee13de-96d4c146-e9854478_0
2025-10-09T21:46:05.022714Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea36a6c7-7fee13de-96d4c146-e9854478_0] Write session: try to update token
2025-10-09T21:46:05.022733Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea36a6c7-7fee13de-96d4c146-e9854478_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:05.045768Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:05.045792Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:05.045794Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:05.045796Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:05.045806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341573711432969:6219]
2025-10-09T21:46:05.045810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341573711432969:6219]
2025-10-09T21:46:05.045814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341573711432969:6219]
2025-10-09T21:46:05.045860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:05.045869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.045872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:05.046021Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:05.047537Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:05.047554Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:05.047558Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:05.064190Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 647 sessionId: src_id|ea36a6c7-7fee13de-96d4c146-e9854478_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:05.067043Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:05.266473Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:05.267167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:05.267178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:05.267461Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:05.267901Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:05.267966Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341578368214270:6527]
2025-10-09T21:46:05.267988Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:05.268071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 647 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ea36a6c7-7fee13de-96d4c146-e9854478_0
2025-10-09T21:46:05.268267Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 647 sessionId: src_id|ea36a6c7-7fee13de-96d4c146-e9854478_0 is DEAD
2025-10-09T21:46:05.268892Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea36a6c7-7fee13de-96d4c146-e9854478_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:05.268903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea36a6c7-7fee13de-96d4c146-e9854478_0] Write session will restart in 0.007932s
2025-10-09T21:46:05.268928Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea36a6c7-7fee13de-96d4c146-e9854478_0] Write session: Do CDS request
2025-10-09T21:46:05.268934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea36a6c7-7fee13de-96d4c146-e9854478_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:05.285849Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea36a6c7-7fee13de-96d4c146-e9854478_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:05.286345Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:05.286364Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 648
2025-10-09T21:46:05.287210Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 648 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:05.287252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 648 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:05.287259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=648 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:05.287262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:05.290161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 648 sessionId:  describe result for acl check
2025-10-09T21:46:05.290209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:05.290211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:05.290212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:05.290219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341578368214324:6840] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:05.290794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341578368214324:6840] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:05.305891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341578368214324:6840] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:05.316326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341578368214324:6840] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:05.316350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341578368214324:6840] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:05.316357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341578368214324:6840] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:05.316368Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 648 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:05.328537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:05.329530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:05.329740Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:05.329744Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:05.329755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:05.329813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:05.329853Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341578006400491:6225]
2025-10-09T21:46:05.330482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:05.330986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:05.331032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:05.331342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:05.331402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:05.331403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:05.331404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:05.331406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:05.331407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:05.331408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:05.331409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.331411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:05.331412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:05.331417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:05.331428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:05.331432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:05.331436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:05.331440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:05.331444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:05.331447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.331451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:05.331455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:05.331463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:05.331471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:05.331479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:05.331484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:05.331489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:05.331494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:05.331499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:05.331517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:05.331522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:05.331527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:05.331532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:05.331537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:05.331542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.331547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.331552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:05.331556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:05.331560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:05.331565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:05.331596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:05.340915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:05.343446Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 647
2025-10-09T21:46:05.343480Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 647
2025-10-09T21:46:05.342141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:05.342156Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:05.342159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:05.342164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:05.342172Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 647 [2:7559341578006400491:6225]
2025-10-09T21:46:05.342187Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:05.342209Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:05.342212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:05.342215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:05.342218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:05.342235Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:05.342242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:05.342251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:05.342321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.342324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:05.342326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:05.342329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:05.342405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:05.342408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.342422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:05.342435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:05.342452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:05.342474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341578006400486:6225]
2025-10-09T21:46:05.342476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341578006400486:6225]
2025-10-09T21:46:05.342478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341578006400486:6225]
2025-10-09T21:46:05.342492Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:05.342508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:05.342524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:05.342528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:05.342568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341578006400486:6225]
2025-10-09T21:46:05.342571Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:05.342728Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:05.342730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:05.342732Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:05.342738Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:05.344098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341578368214498:6840], now have 1 active actors on pipe
2025-10-09T21:46:05.344110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341578368214286:2432], now have 1 active actors on pipe
2025-10-09T21:46:05.344127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:05.344133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:05.344173Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|945a22a2-6b6b9aa7-60329ac7-b899dbd9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:05.344188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:05.344197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:05.344204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:05.344217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:05.344235Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:05.344260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:05.344845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:05.344855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:05.344872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:05.344999Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 648 partition: 0 MaxSeqNo: 25 sessionId: src_id|945a22a2-6b6b9aa7-60329ac7-b899dbd9_0
2025-10-09T21:46:05.345474Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea36a6c7-7fee13de-96d4c146-e9854478_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|945a22a2-6b6b9aa7-60329ac7-b899dbd9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:05.348341Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|945a22a2-6b6b9aa7-60329ac7-b899dbd9_0] Write session: try to update token
2025-10-09T21:46:05.348360Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|945a22a2-6b6b9aa7-60329ac7-b899dbd9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:05.368164Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341578368214511:6527]
2025-10-09T21:46:05.370146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341578368214511:6527], now have 1 active actors on pipe
2025-10-09T21:46:05.370270Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 647, pipe: [1:7559341578368214511:6527]
2025-10-09T21:46:05.410129Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:05.410149Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:05.410152Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:05.410155Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:05.410166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341578006400486:6225]
2025-10-09T21:46:05.410171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341578006400486:6225]
2025-10-09T21:46:05.410175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341578006400486:6225]
2025-10-09T21:46:05.410225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:05.410232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.410235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:05.410392Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:05.414744Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:05.414762Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:05.414765Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:05.420091Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 648 sessionId: src_id|945a22a2-6b6b9aa7-60329ac7-b899dbd9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:05.422900Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:05.625079Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:05.625945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:05.625963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:05.626278Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:05.626788Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:05.626887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 648 reason: status is not ok: tablet will be restarted right now sessionId: src_id|945a22a2-6b6b9aa7-60329ac7-b899dbd9_0
2025-10-09T21:46:05.627056Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 648 sessionId: src_id|945a22a2-6b6b9aa7-60329ac7-b899dbd9_0 is DEAD
2025-10-09T21:46:05.627188Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341578368214511:6527]
2025-10-09T21:46:05.627203Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:05.627579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|945a22a2-6b6b9aa7-60329ac7-b899dbd9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:05.627588Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|945a22a2-6b6b9aa7-60329ac7-b899dbd9_0] Write session will restart in 0.006843s
2025-10-09T21:46:05.627616Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|945a22a2-6b6b9aa7-60329ac7-b899dbd9_0] Write session: Do CDS request
2025-10-09T21:46:05.627622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|945a22a2-6b6b9aa7-60329ac7-b899dbd9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:05.635724Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|945a22a2-6b6b9aa7-60329ac7-b899dbd9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:05.636174Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:05.636193Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 649
2025-10-09T21:46:05.636388Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 649 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:05.636434Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 649 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:05.636440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=649 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:05.636443Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:05.637300Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 649 sessionId:  describe result for acl check
2025-10-09T21:46:05.637340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:05.637349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:05.637351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:05.637357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341578368214529:6844] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:05.637914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341578368214529:6844] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:05.641625Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341578368214529:6844] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:05.652795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341578368214529:6844] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:05.652817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341578368214529:6844] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:05.652821Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341578368214529:6844] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:05.652830Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 649 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:05.678911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:05.679436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:05.679661Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:05.679672Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:05.679687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:05.679751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:05.679792Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341578006400716:6229]
2025-10-09T21:46:05.680361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:05.680851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:05.680895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:05.681258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:05.681326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:05.681327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:05.681328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:05.681330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:05.681331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:05.681332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:05.681333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.681334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:05.681335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:05.681340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:05.681352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:05.681355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:05.681359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:05.681363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:05.681367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:05.681371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.681374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:05.681378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:05.681385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:05.681393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:05.681399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:05.681404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:05.681409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:05.681414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:05.681419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:05.681423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:05.681428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:05.681433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:05.681438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:05.681442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:05.681447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.681451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.681454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:05.681458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:05.681462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:05.681467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:05.681493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:05.691526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:05.693363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:05.693379Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:05.693381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:05.693386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:05.693392Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 648 [2:7559341578006400716:6229]
2025-10-09T21:46:05.693406Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:05.694611Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 648
2025-10-09T21:46:05.694651Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 648
2025-10-09T21:46:05.693424Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:05.693428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:05.693431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:05.693434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:05.693451Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:05.693457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:05.693464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:05.693523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.693528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:05.693530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:05.693532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:05.693611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:05.693615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.693626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:05.693638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:05.693655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:05.693676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341578006400711:6229]
2025-10-09T21:46:05.693679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341578006400711:6229]
2025-10-09T21:46:05.693681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341578006400711:6229]
2025-10-09T21:46:05.693693Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:05.693707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:05.693720Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:05.693723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:05.693763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341578006400711:6229]
2025-10-09T21:46:05.693766Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:05.694006Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:05.694008Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:05.694010Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:05.694016Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:05.698860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341578368214620:6844], now have 1 active actors on pipe
2025-10-09T21:46:05.698878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341578368214517:2432], now have 1 active actors on pipe
2025-10-09T21:46:05.698903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:05.698910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:05.698957Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|af813410-9701ad5a-5607ccf8-2f67812a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:05.698975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:05.699142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:05.699151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:05.699166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:05.699189Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:05.699216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:05.699664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:05.699676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:05.699695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:05.699848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 649 partition: 0 MaxSeqNo: 25 sessionId: src_id|af813410-9701ad5a-5607ccf8-2f67812a_0
2025-10-09T21:46:05.700331Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|945a22a2-6b6b9aa7-60329ac7-b899dbd9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|af813410-9701ad5a-5607ccf8-2f67812a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:05.703540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af813410-9701ad5a-5607ccf8-2f67812a_0] Write session: try to update token
2025-10-09T21:46:05.703559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af813410-9701ad5a-5607ccf8-2f67812a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:05.718682Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:05.718706Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:05.718709Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:05.718711Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:05.718723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341578006400711:6229]
2025-10-09T21:46:05.718729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341578006400711:6229]
2025-10-09T21:46:05.718733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341578006400711:6229]
2025-10-09T21:46:05.718782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:05.718790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:05.718793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:05.718951Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:05.725476Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:05.725495Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:05.725500Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:05.725681Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341578368214748:6527]
2025-10-09T21:46:05.726350Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 648, pipe: [1:7559341578368214748:6527]
2025-10-09T21:46:05.731180Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 649 sessionId: src_id|af813410-9701ad5a-5607ccf8-2f67812a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:05.734271Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:05.951719Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:05.952007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341578368214748:6527], now have 1 active actors on pipe
2025-10-09T21:46:05.952038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:05.952046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:05.956088Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341578368214748:6527]
2025-10-09T21:46:05.956125Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:05.956206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 649 reason: status is not ok: tablet will be restarted right now sessionId: src_id|af813410-9701ad5a-5607ccf8-2f67812a_0
2025-10-09T21:46:05.956391Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 649 sessionId: src_id|af813410-9701ad5a-5607ccf8-2f67812a_0 is DEAD
2025-10-09T21:46:05.956504Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:05.957582Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af813410-9701ad5a-5607ccf8-2f67812a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:05.957594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af813410-9701ad5a-5607ccf8-2f67812a_0] Write session will restart in 0.009602s
2025-10-09T21:46:05.957618Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af813410-9701ad5a-5607ccf8-2f67812a_0] Write session: Do CDS request
2025-10-09T21:46:05.957625Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af813410-9701ad5a-5607ccf8-2f67812a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:05.954443Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:05.968660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af813410-9701ad5a-5607ccf8-2f67812a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:05.969757Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:05.969779Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 650
2025-10-09T21:46:05.969893Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 650 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:05.969920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 650 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:05.969930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=650 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:05.969933Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:05.970372Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 650 sessionId:  describe result for acl check
2025-10-09T21:46:05.970406Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:05.970412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:05.970417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:05.970422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341578368214783:6855] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:05.970889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341578368214783:6855] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:06.005649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341578368214783:6855] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:06.024681Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341578368214783:6855] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:06.024707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341578368214783:6855] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:06.024711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341578368214783:6855] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:06.024721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 650 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:06.037341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:06.037568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:06.037752Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:06.037756Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:06.037770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:06.037818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:06.037857Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341582301368255:6237]
2025-10-09T21:46:06.038550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:06.040737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:06.040817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:06.041185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:06.041251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:06.041252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:06.041254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:06.041255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:06.041256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:06.041258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:06.041259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.041261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.041262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.041269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:06.041281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:06.041285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:06.041290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:06.041294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:06.041299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:06.041303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.041307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.041311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.041319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:06.041329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:06.041335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:06.041342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:06.041347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:06.041353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:06.041358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:06.041366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:06.041371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:06.041377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:06.041381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:06.041387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:06.041398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.041404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.041409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.041415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.041420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.041426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.041459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:06.054050Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 649
2025-10-09T21:46:06.054098Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 649
2025-10-09T21:46:06.052229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:06.053343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:06.053349Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:06.053350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:06.053355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:06.053361Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 649 [2:7559341582301368255:6237]
2025-10-09T21:46:06.053371Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:06.053389Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:06.053391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:06.053394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:06.053396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:06.053412Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:06.053416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:06.053422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:06.053489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.053492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.053494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.053498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:06.053573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:06.053576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.053590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.053600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.053615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:06.053637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341582301368250:6237]
2025-10-09T21:46:06.053639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341582301368250:6237]
2025-10-09T21:46:06.053641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341582301368250:6237]
2025-10-09T21:46:06.053650Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:06.053656Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:06.053658Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:06.053660Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:06.053665Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:06.053679Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:06.053682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:06.053694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341582663182288:6855], now have 1 active actors on pipe
2025-10-09T21:46:06.053697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341578368214770:2432], now have 1 active actors on pipe
2025-10-09T21:46:06.053733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341582301368250:6237]
2025-10-09T21:46:06.053740Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:06.054442Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:06.056064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:06.056083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:06.056123Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|378f5733-dbe8b45c-36b333a6-e6d75571_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:06.056136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:06.056140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:06.056143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:06.056158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:06.056175Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:06.056196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:06.056563Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341582663182303:6527]
2025-10-09T21:46:06.056885Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 649, pipe: [1:7559341582663182303:6527]
2025-10-09T21:46:06.056929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:06.056939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:06.056963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341582663182303:6527], now have 1 active actors on pipe
2025-10-09T21:46:06.056975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:06.057151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 650 partition: 0 MaxSeqNo: 25 sessionId: src_id|378f5733-dbe8b45c-36b333a6-e6d75571_0
2025-10-09T21:46:06.057598Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af813410-9701ad5a-5607ccf8-2f67812a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|378f5733-dbe8b45c-36b333a6-e6d75571_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:06.060531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|378f5733-dbe8b45c-36b333a6-e6d75571_0] Write session: try to update token
2025-10-09T21:46:06.060551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|378f5733-dbe8b45c-36b333a6-e6d75571_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:06.090735Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 650 sessionId: src_id|378f5733-dbe8b45c-36b333a6-e6d75571_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:06.093734Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:06.097134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:06.097158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:06.097182Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:06.097186Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:06.097195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341582301368250:6237]
2025-10-09T21:46:06.097257Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:06.097275Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:06.097430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:06.097568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:06.097704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:06.097839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:06.098016Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:06.102234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:06.102276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:06.102498Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:06.111609Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:06.111627Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:06.111629Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:06.111631Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:06.111641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341582301368250:6237]
2025-10-09T21:46:06.111644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341582301368250:6237]
2025-10-09T21:46:06.111647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341582301368250:6237]
2025-10-09T21:46:06.111690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:06.111697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.111698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:06.111708Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:06.111723Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:06.111727Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:06.111832Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:06.301224Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:06.311547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:06.311582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:06.311761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:06.311835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:06.311903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:06.312183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:06.312228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:06.312243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:06.312322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:06.312380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:06.312438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:06.312504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:06.312545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:06.312547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:06.312614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:06.312674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:06.312761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:06.312837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:06.312880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:06.312882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:06.312957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:06.313026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:06.313105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:06.313171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:06.313216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:06.313219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:06.313289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:06.313354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:06.313422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:06.313487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:06.313533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:06.313535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:06.313604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:06.313671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:06.313733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:06.313795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:06.313840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:06.313843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:06.313909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:06.313978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:06.314082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:06.314145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:06.314187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:06.314189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:06.314261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:06.314325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:06.314390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:06.314455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:06.314502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:06.314504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:06.314576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:06.314640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:06.314710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:06.314782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:06.314828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:06.314831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:06.314903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:06.314969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:06.315034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:06.315102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:06.315148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:06.315151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:06.318563Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341582663182303:6527]
2025-10-09T21:46:06.318613Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:06.318616Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:06.318647Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 650 reason: error at write: Disconnected sessionId: src_id|378f5733-dbe8b45c-36b333a6-e6d75571_0
2025-10-09T21:46:06.318851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 650 sessionId: src_id|378f5733-dbe8b45c-36b333a6-e6d75571_0 is DEAD
2025-10-09T21:46:06.318993Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:06.319871Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|378f5733-dbe8b45c-36b333a6-e6d75571_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:06.319884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|378f5733-dbe8b45c-36b333a6-e6d75571_0] Write session will restart in 0.007223s
2025-10-09T21:46:06.319909Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|378f5733-dbe8b45c-36b333a6-e6d75571_0] Write session: Do CDS request
2025-10-09T21:46:06.319915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|378f5733-dbe8b45c-36b333a6-e6d75571_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:06.320403Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:06.329052Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|378f5733-dbe8b45c-36b333a6-e6d75571_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:06.330651Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:06.330671Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 651
2025-10-09T21:46:06.330892Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 651 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:06.330926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 651 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:06.330932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=651 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:06.330936Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:06.331622Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 651 sessionId:  describe result for acl check
2025-10-09T21:46:06.331667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:06.331669Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:06.331671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:06.331677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341582663182326:6861] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:06.332244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341582663182326:6861] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:06.340380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341582663182326:6861] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:06.348629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341582663182326:6861] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:06.348652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341582663182326:6861] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:06.348657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341582663182326:6861] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:06.348669Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 651 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:06.388461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:06.388878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:06.389074Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:06.389078Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:06.389089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:06.389131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:06.389167Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341582301368496:6245]
2025-10-09T21:46:06.389806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:06.390446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:06.390497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:06.390832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:06.390877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:06.390879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:06.390880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:06.390881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:06.390883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:06.390884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:06.390885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.390887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.390888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.390893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:06.390905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:06.390909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:06.390913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:06.390917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:06.390921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:06.390924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.390927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.390931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.390939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:06.390969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:06.390975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:06.390980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:06.390984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:06.390990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:06.390994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:06.391008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:06.391013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:06.391019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:06.391024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:06.391030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:06.391036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.391041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.391046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.391051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.391056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.391062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.391093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:06.401299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:06.404616Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 650
2025-10-09T21:46:06.404649Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 650
2025-10-09T21:46:06.403228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:06.403244Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:06.403246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:06.403251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:06.403260Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 650 [2:7559341582301368496:6245]
2025-10-09T21:46:06.403275Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:06.403295Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:06.403299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:06.403302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:06.403304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:06.403322Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:06.403329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:06.403336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:06.403400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.403402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.403404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.403407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:06.403482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:06.403485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.403499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.403511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.403528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:06.403550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341582301368491:6245]
2025-10-09T21:46:06.403552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341582301368491:6245]
2025-10-09T21:46:06.403554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341582301368491:6245]
2025-10-09T21:46:06.403567Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:06.403581Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:06.403595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:06.403599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:06.403637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341582301368491:6245]
2025-10-09T21:46:06.403640Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:06.403873Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:06.403875Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:06.403880Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:06.403885Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:06.409249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 651 partition: 0 MaxSeqNo: 25 sessionId: src_id|44dbaaee-28b99504-d91bc7e2-85f0d9f1_0
2025-10-09T21:46:06.408555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341582663182372:6861], now have 1 active actors on pipe
2025-10-09T21:46:06.408573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341582663182314:2432], now have 1 active actors on pipe
2025-10-09T21:46:06.408598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:06.408604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:06.408650Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|44dbaaee-28b99504-d91bc7e2-85f0d9f1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:06.408663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:06.408669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:06.408672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:06.408687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:06.408706Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:06.408730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:06.409100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:06.409105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:06.409123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:06.411447Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|378f5733-dbe8b45c-36b333a6-e6d75571_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|44dbaaee-28b99504-d91bc7e2-85f0d9f1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:06.414670Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|44dbaaee-28b99504-d91bc7e2-85f0d9f1_0] Write session: try to update token
2025-10-09T21:46:06.414694Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|44dbaaee-28b99504-d91bc7e2-85f0d9f1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:06.422847Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341582663182543:6527]
2025-10-09T21:46:06.424044Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 650, pipe: [1:7559341582663182543:6527]
2025-10-09T21:46:06.423660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341582663182543:6527], now have 1 active actors on pipe
2025-10-09T21:46:06.428426Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:06.428452Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:06.428454Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:06.428457Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:06.428466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341582301368491:6245]
2025-10-09T21:46:06.428470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341582301368491:6245]
2025-10-09T21:46:06.428473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341582301368491:6245]
2025-10-09T21:46:06.428516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:06.428529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.428531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:06.428681Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:06.430982Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:06.430997Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:06.431002Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:06.457467Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 651 sessionId: src_id|44dbaaee-28b99504-d91bc7e2-85f0d9f1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:06.460263Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:06.608852Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:06.608872Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:06.608926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 651 reason: error at write: Disconnected sessionId: src_id|44dbaaee-28b99504-d91bc7e2-85f0d9f1_0
2025-10-09T21:46:06.609119Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 651 sessionId: src_id|44dbaaee-28b99504-d91bc7e2-85f0d9f1_0 is DEAD
2025-10-09T21:46:06.609239Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:06.609285Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341582663182543:6527]
2025-10-09T21:46:06.609353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44dbaaee-28b99504-d91bc7e2-85f0d9f1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:06.609377Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44dbaaee-28b99504-d91bc7e2-85f0d9f1_0] Write session will restart in 0.007746s
2025-10-09T21:46:06.609405Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44dbaaee-28b99504-d91bc7e2-85f0d9f1_0] Write session: Do CDS request
2025-10-09T21:46:06.609412Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44dbaaee-28b99504-d91bc7e2-85f0d9f1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:06.607727Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:06.608042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:06.608053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:06.609980Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:06.619248Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|44dbaaee-28b99504-d91bc7e2-85f0d9f1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:06.619832Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:06.619848Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 652
2025-10-09T21:46:06.620227Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 652 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:06.620259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 652 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:06.620265Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=652 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:06.620269Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:06.620728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 652 sessionId:  describe result for acl check
2025-10-09T21:46:06.620761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:06.620763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:06.620765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:06.620770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341582663182562:6865] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:06.621271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341582663182562:6865] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:06.627981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341582663182562:6865] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:06.635877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341582663182562:6865] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:06.635895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341582663182562:6865] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:06.635899Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341582663182562:6865] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:06.635909Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 652 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:06.651330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:06.656057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:06.656283Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:06.656287Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:06.656298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:06.656347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:06.656384Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341582301368724:6249]
2025-10-09T21:46:06.656909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:06.657615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:06.657658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:06.657931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:06.657968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:06.657969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:06.657970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:06.657971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:06.657972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:06.657973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:06.657974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.657975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.657977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.657981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:06.657992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:06.657996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:06.658000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:06.658003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:06.658007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:06.658010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.658014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.658049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.658058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:06.658065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:06.658083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:06.658099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:06.658103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:06.658108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:06.658111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:06.658115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:06.658120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:06.658124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:06.658127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:06.658132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:06.658136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.658140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.658144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.658148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.658152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.658157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.658183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:06.670062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:06.671208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:06.671214Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:06.671217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:06.671222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:06.671230Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 651 [2:7559341582301368724:6249]
2025-10-09T21:46:06.671245Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:06.671265Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:06.671268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:06.671272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:06.671275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:06.671292Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:06.671298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:06.671305Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:06.671365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.671367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.671369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.671372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:06.671448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:06.671450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.671461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.671471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.671484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:06.671505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341582301368718:6249]
2025-10-09T21:46:06.671507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341582301368718:6249]
2025-10-09T21:46:06.671510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341582301368718:6249]
2025-10-09T21:46:06.671523Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:06.671536Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:06.671550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:06.671553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:06.671591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341582301368718:6249]
2025-10-09T21:46:06.671593Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:06.671954Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:06.671957Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:06.671959Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:06.671964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:06.672438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341582663182670:6865], now have 1 active actors on pipe
2025-10-09T21:46:06.672451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341582663182548:2432], now have 1 active actors on pipe
2025-10-09T21:46:06.674742Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 651
2025-10-09T21:46:06.674779Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 651
2025-10-09T21:46:06.678381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:06.678401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:06.678446Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|502fa007-28fec294-29b63f07-221c1610_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:06.678458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:06.678464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:06.678467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:06.678483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:06.678501Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:06.678523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:06.678938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:06.678942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:06.678957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:06.679482Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44dbaaee-28b99504-d91bc7e2-85f0d9f1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|502fa007-28fec294-29b63f07-221c1610_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:06.679077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 652 partition: 0 MaxSeqNo: 25 sessionId: src_id|502fa007-28fec294-29b63f07-221c1610_0
2025-10-09T21:46:06.682537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|502fa007-28fec294-29b63f07-221c1610_0] Write session: try to update token
2025-10-09T21:46:06.682553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|502fa007-28fec294-29b63f07-221c1610_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:06.708640Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 652 sessionId: src_id|502fa007-28fec294-29b63f07-221c1610_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:06.709402Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341582663182783:6527]
2025-10-09T21:46:06.711464Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:06.711809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341582663182783:6527], now have 1 active actors on pipe
2025-10-09T21:46:06.713220Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:06.713224Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:06.713227Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:06.713229Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:06.713237Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341582301368718:6249]
2025-10-09T21:46:06.713242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341582301368718:6249]
2025-10-09T21:46:06.713245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341582301368718:6249]
2025-10-09T21:46:06.713289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:06.713297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.713299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:06.713440Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:06.715785Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:06.715803Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:06.715807Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:06.725081Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 651, pipe: [1:7559341582663182783:6527]
2025-10-09T21:46:06.913295Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:06.914287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:06.914308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:06.915628Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|502fa007-28fec294-29b63f07-221c1610_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:06.915644Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|502fa007-28fec294-29b63f07-221c1610_0] Write session will restart in 0.005728s
2025-10-09T21:46:06.915668Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|502fa007-28fec294-29b63f07-221c1610_0] Write session: Do CDS request
2025-10-09T21:46:06.915674Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|502fa007-28fec294-29b63f07-221c1610_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:06.915152Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341582663182783:6527]
2025-10-09T21:46:06.915191Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:06.915194Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:06.915246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 652 reason: error at write: Disconnected sessionId: src_id|502fa007-28fec294-29b63f07-221c1610_0
2025-10-09T21:46:06.915417Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 652 sessionId: src_id|502fa007-28fec294-29b63f07-221c1610_0 is DEAD
2025-10-09T21:46:06.915544Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:06.916180Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:06.926656Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|502fa007-28fec294-29b63f07-221c1610_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:06.926814Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:06.926833Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 653
2025-10-09T21:46:06.927144Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 653 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:06.927181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 653 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:06.927187Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=653 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:06.927190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:06.928408Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 653 sessionId:  describe result for acl check
2025-10-09T21:46:06.928454Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:06.928458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:06.928460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:06.928465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341582663182798:6872] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:06.929020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341582663182798:6872] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:06.944296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341582663182798:6872] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:06.957904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341582663182798:6872] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:06.957927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341582663182798:6872] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:06.957932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341582663182798:6872] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:06.957943Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 653 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:06.971869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:06.978052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:06.978335Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:06.978341Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:06.978354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:06.978422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:06.978467Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341582301368949:6251]
2025-10-09T21:46:06.979082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:06.979516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:06.979567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:06.979892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:06.979955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:06.979957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:06.979958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:06.979959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:06.979961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:06.979962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:06.979963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.979964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.979966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.979971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:06.979982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:06.979986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:06.979990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:06.979994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:06.979998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:06.980002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.980006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.980009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.980017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:06.980025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:06.980032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:06.980038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:06.980044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:06.980050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:06.980054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:06.980060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:06.980064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:06.980070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:06.980074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:06.980079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:06.980084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.980090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.980094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.980099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.980104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.980110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.980139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:06.989074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:06.990685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:06.990703Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:06.990705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:06.990712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:06.990721Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 652 [2:7559341582301368949:6251]
2025-10-09T21:46:06.990736Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:06.990760Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:06.990763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:06.990766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:06.990769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:06.990789Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:06.990796Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:06.990805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:06.990876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.990878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.990881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.990883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:06.990964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:06.990972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:06.990986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:06.990999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:06.991017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:06.991039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341582301368943:6251]
2025-10-09T21:46:06.991041Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341582301368943:6251]
2025-10-09T21:46:06.991044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341582301368943:6251]
2025-10-09T21:46:06.991061Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:06.991078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:06.991094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:06.991098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:06.991141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341582301368943:6251]
2025-10-09T21:46:06.991143Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:06.991235Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:06.991237Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:06.991239Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:06.991246Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:06.992176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341582663182789:2432], now have 1 active actors on pipe
2025-10-09T21:46:06.992182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341582663182937:6872], now have 1 active actors on pipe
2025-10-09T21:46:06.993211Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 652
2025-10-09T21:46:06.993254Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 652
2025-10-09T21:46:06.994866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 653 partition: 0 MaxSeqNo: 25 sessionId: src_id|25491ba-4c1019f6-34a4df34-ef0f24f2_0
2025-10-09T21:46:06.995402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|502fa007-28fec294-29b63f07-221c1610_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|25491ba-4c1019f6-34a4df34-ef0f24f2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:06.993830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:06.993847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:06.993883Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|25491ba-4c1019f6-34a4df34-ef0f24f2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:06.993896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:06.993900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:06.993903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:06.993926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:06.993944Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:06.993964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:06.994620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:06.994626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:06.994647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:06.998338Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25491ba-4c1019f6-34a4df34-ef0f24f2_0] Write session: try to update token
2025-10-09T21:46:06.998356Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25491ba-4c1019f6-34a4df34-ef0f24f2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:07.014497Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341586958150321:6527]
2025-10-09T21:46:07.015094Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 652, pipe: [1:7559341586958150321:6527]
2025-10-09T21:46:07.014992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341586958150321:6527], now have 1 active actors on pipe
2025-10-09T21:46:07.019466Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:07.019483Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:07.019485Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:07.019487Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:07.019499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341582301368943:6251]
2025-10-09T21:46:07.019504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341582301368943:6251]
2025-10-09T21:46:07.019507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341582301368943:6251]
2025-10-09T21:46:07.019554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:07.019562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:07.019565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:07.019740Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:07.026143Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:07.026176Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:07.026185Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:07.033320Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 653 sessionId: src_id|25491ba-4c1019f6-34a4df34-ef0f24f2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:07.036604Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:07.251463Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:07.251774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:07.251792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:07.252101Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:07.253535Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25491ba-4c1019f6-34a4df34-ef0f24f2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:07.253554Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25491ba-4c1019f6-34a4df34-ef0f24f2_0] Write session will restart in 0.006976s
2025-10-09T21:46:07.253577Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25491ba-4c1019f6-34a4df34-ef0f24f2_0] Write session: Do CDS request
2025-10-09T21:46:07.253584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25491ba-4c1019f6-34a4df34-ef0f24f2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:07.253013Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341586958150321:6527]
2025-10-09T21:46:07.253044Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:07.253077Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:07.253163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 653 reason: status is not ok: tablet will be restarted right now sessionId: src_id|25491ba-4c1019f6-34a4df34-ef0f24f2_0
2025-10-09T21:46:07.253368Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 653 sessionId: src_id|25491ba-4c1019f6-34a4df34-ef0f24f2_0 is DEAD
2025-10-09T21:46:07.263065Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25491ba-4c1019f6-34a4df34-ef0f24f2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:07.266274Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:07.266293Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 654
2025-10-09T21:46:07.266662Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 654 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:07.266699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 654 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:07.266705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=654 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:07.266709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:07.267723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 654 sessionId:  describe result for acl check
2025-10-09T21:46:07.267765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:07.267767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:07.267768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:07.267775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341586958150380:6884] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:07.268346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341586958150380:6884] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:07.275900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341586958150380:6884] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:07.286883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341586958150380:6884] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:07.286906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341586958150380:6884] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:07.286911Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341586958150380:6884] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:07.286921Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 654 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:07.304279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:07.309914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:07.310197Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:07.310210Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:07.310225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:07.310294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:07.310343Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341586596336509:6265]
2025-10-09T21:46:07.310963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:07.311680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:07.311759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:07.312145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:07.312201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:07.312207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:07.312208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:07.312210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:07.312211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:07.312212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:07.312214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:07.312216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:07.312217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:07.312224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:07.312236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:07.312241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:07.312246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:07.312250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:07.312255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:07.312258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:07.312262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:07.312266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:07.312274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:07.312283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:07.312295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:07.312303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:07.312309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:07.312314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:07.312319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:07.312324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:07.312328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:07.312334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:07.312345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:07.312351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:07.312356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:07.312361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:07.312364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:07.312367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:07.312370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:07.312372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:07.312389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:07.323169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:07.324673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:07.324690Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:07.324692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:07.324697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:07.324704Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 653 [2:7559341586596336509:6265]
2025-10-09T21:46:07.324719Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:07.324740Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:07.324749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:07.324752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:07.324755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:07.324772Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:07.324784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:07.324792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:07.324854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:07.324862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:07.324864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:07.324867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:07.324940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:07.324947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:07.324959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:07.324970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:07.324987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:07.325008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341586596336504:6265]
2025-10-09T21:46:07.325017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341586596336504:6265]
2025-10-09T21:46:07.325019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341586596336504:6265]
2025-10-09T21:46:07.325034Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:07.325052Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:07.325073Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:07.325076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:07.325103Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:07.325106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:07.325107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:07.325116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341586596336504:6265]
2025-10-09T21:46:07.325120Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:07.325182Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:07.325624Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 653
2025-10-09T21:46:07.325660Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 653
2025-10-09T21:46:07.325960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341586958150502:6884], now have 1 active actors on pipe
2025-10-09T21:46:07.325969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341586958150357:2432], now have 1 active actors on pipe
2025-10-09T21:46:07.325995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:07.326001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:07.326049Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6e93b059-9d1fe25a-4300c24a-8540c103_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:07.326058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:07.326062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:07.326080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:07.326104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:07.326116Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:07.326133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:07.330513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:07.330531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:07.330563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:07.330712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 654 partition: 0 MaxSeqNo: 25 sessionId: src_id|6e93b059-9d1fe25a-4300c24a-8540c103_0
2025-10-09T21:46:07.331223Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25491ba-4c1019f6-34a4df34-ef0f24f2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6e93b059-9d1fe25a-4300c24a-8540c103_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:07.334226Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e93b059-9d1fe25a-4300c24a-8540c103_0] Write session: try to update token
2025-10-09T21:46:07.334245Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e93b059-9d1fe25a-4300c24a-8540c103_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:07.353777Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341586958150588:6527]
2025-10-09T21:46:07.354257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341586958150588:6527], now have 1 active actors on pipe
2025-10-09T21:46:07.354365Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 653, pipe: [1:7559341586958150588:6527]
2025-10-09T21:46:07.372312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:07.372346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:07.372402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:07.369033Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 654 sessionId: src_id|6e93b059-9d1fe25a-4300c24a-8540c103_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:07.372046Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:07.372715Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:07.379903Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:07.379925Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:07.379928Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:07.379930Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:07.379943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341586596336504:6265]
2025-10-09T21:46:07.379949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341586596336504:6265]
2025-10-09T21:46:07.379953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341586596336504:6265]
2025-10-09T21:46:07.380005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:07.380020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:07.380023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:07.380217Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:07.383354Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:07.383371Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:07.383374Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:07.576026Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:07.587238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:07.587283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:07.587472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:07.587533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:07.587601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:07.587669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:07.587713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:07.587718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:07.587793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:07.587854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:07.587925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:07.587995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:07.588038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:07.588041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:07.588115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:07.588180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:07.588246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:07.588315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:07.588360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:07.588363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:07.588447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:07.588520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:07.588590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:07.588654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:07.588701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:07.588704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:07.588781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:07.588860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:07.589018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:07.589165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:07.589213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:07.589216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:07.589300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:07.589398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:07.589472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:07.589557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:07.589630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:07.589633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:07.589713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:07.589816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:07.589900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:07.590682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:07.590741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:07.590745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:07.590833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:07.590894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:07.590961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:07.591024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:07.591067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:07.591070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:07.591139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:07.591222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:07.591292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:07.591369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:07.591413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:07.591416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:07.591482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:07.591543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:07.591609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:07.591682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:07.591726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:07.591728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:07.594391Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341586958150588:6527]
2025-10-09T21:46:07.594431Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:07.594433Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:07.594461Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 654 reason: error at write: Disconnected sessionId: src_id|6e93b059-9d1fe25a-4300c24a-8540c103_0
2025-10-09T21:46:07.594636Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 654 sessionId: src_id|6e93b059-9d1fe25a-4300c24a-8540c103_0 is DEAD
2025-10-09T21:46:07.595024Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:07.595230Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e93b059-9d1fe25a-4300c24a-8540c103_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:07.595242Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e93b059-9d1fe25a-4300c24a-8540c103_0] Write session will restart in 0.007280s
2025-10-09T21:46:07.595264Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e93b059-9d1fe25a-4300c24a-8540c103_0] Write session: Do CDS request
2025-10-09T21:46:07.595269Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e93b059-9d1fe25a-4300c24a-8540c103_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:07.594287Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:07.603997Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e93b059-9d1fe25a-4300c24a-8540c103_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:07.604394Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:07.604421Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 655
2025-10-09T21:46:07.604647Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 655 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:07.604688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 655 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:07.604699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=655 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:07.604703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:07.606432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 655 sessionId:  describe result for acl check
2025-10-09T21:46:07.606471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:07.606473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:07.606475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:07.606480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341586958150602:6888] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:07.606951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341586958150602:6888] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:07.619086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341586958150602:6888] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:07.628030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341586958150602:6888] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:07.628050Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341586958150602:6888] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:07.628054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341586958150602:6888] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:07.628064Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 655 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:07.645420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:07.655600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:07.656234Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:07.656255Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:07.656275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:07.656362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:07.656413Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341586596336740:6269]
2025-10-09T21:46:07.656963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:07.657475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:07.657531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:07.657853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:07.657909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:07.657917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:07.657918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:07.657920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:07.657921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:07.657922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:07.657924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:07.657925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:07.657926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:07.657932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:07.657944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:07.657949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:07.657953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:07.657958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:07.657962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:07.657967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:07.657971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:07.657975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:07.657982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:07.657992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:07.657999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:07.658004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:07.658008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:07.658014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:07.658018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:07.658023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:07.658039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:07.658044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:07.658050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:07.658055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:07.658062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:07.658085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:07.658089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:07.658095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:07.658101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:07.658107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:07.658137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:07.670786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:07.671953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:07.671974Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:07.671977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:07.671982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:07.671990Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 654 [2:7559341586596336740:6269]
2025-10-09T21:46:07.672004Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:07.672025Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:07.672028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:07.672032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:07.672034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:07.672053Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:07.672063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:07.672069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:07.672134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:07.672136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:07.672138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:07.672141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:07.672213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:07.672216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:07.672227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:07.672238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:07.672251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:07.672273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341586596336734:6269]
2025-10-09T21:46:07.672275Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341586596336734:6269]
2025-10-09T21:46:07.672277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341586596336734:6269]
2025-10-09T21:46:07.672285Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:07.672301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:07.672314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:07.672317Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:07.672360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341586596336734:6269]
2025-10-09T21:46:07.672362Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:07.672367Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:07.672370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:07.672374Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:07.672379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:07.672397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341586958150708:6888], now have 1 active actors on pipe
2025-10-09T21:46:07.672400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341586958150593:2432], now have 1 active actors on pipe
2025-10-09T21:46:07.672740Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 654
2025-10-09T21:46:07.672780Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 654
2025-10-09T21:46:07.675602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:07.675621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:07.675662Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5e59fd0c-4bd3d1a5-2e01e08c-1ee109b1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:07.675677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:07.675682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:07.675685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:07.675699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:07.675716Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:07.675738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:07.676097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:07.676103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:07.676116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:07.676303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 655 partition: 0 MaxSeqNo: 25 sessionId: src_id|5e59fd0c-4bd3d1a5-2e01e08c-1ee109b1_0
2025-10-09T21:46:07.678249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e93b059-9d1fe25a-4300c24a-8540c103_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5e59fd0c-4bd3d1a5-2e01e08c-1ee109b1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:07.681042Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5e59fd0c-4bd3d1a5-2e01e08c-1ee109b1_0] Write session: try to update token
2025-10-09T21:46:07.681062Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5e59fd0c-4bd3d1a5-2e01e08c-1ee109b1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:07.693913Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341586958150829:6527]
2025-10-09T21:46:07.695502Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:07.695523Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:07.695525Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:07.695527Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:07.695536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341586596336734:6269]
2025-10-09T21:46:07.695540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341586596336734:6269]
2025-10-09T21:46:07.695544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341586596336734:6269]
2025-10-09T21:46:07.695590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:07.695597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:07.695599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:07.695730Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:07.698203Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:07.698219Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:07.698226Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:07.699155Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 654, pipe: [1:7559341586958150829:6527]
2025-10-09T21:46:07.719064Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 655 sessionId: src_id|5e59fd0c-4bd3d1a5-2e01e08c-1ee109b1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:07.721870Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:08.030601Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:08.031434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341586958150829:6527], now have 1 active actors on pipe
2025-10-09T21:46:08.031510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:08.031519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:08.032913Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e59fd0c-4bd3d1a5-2e01e08c-1ee109b1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:08.032927Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e59fd0c-4bd3d1a5-2e01e08c-1ee109b1_0] Write session will restart in 0.008672s
2025-10-09T21:46:08.032953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e59fd0c-4bd3d1a5-2e01e08c-1ee109b1_0] Write session: Do CDS request
2025-10-09T21:46:08.032960Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e59fd0c-4bd3d1a5-2e01e08c-1ee109b1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:08.032356Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:08.032373Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:08.032439Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 655 reason: error at write: Disconnected sessionId: src_id|5e59fd0c-4bd3d1a5-2e01e08c-1ee109b1_0
2025-10-09T21:46:08.032640Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 655 sessionId: src_id|5e59fd0c-4bd3d1a5-2e01e08c-1ee109b1_0 is DEAD
2025-10-09T21:46:08.032800Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341586958150829:6527]
2025-10-09T21:46:08.032815Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:08.033255Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:08.042604Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5e59fd0c-4bd3d1a5-2e01e08c-1ee109b1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:08.044799Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:08.044820Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 656
2025-10-09T21:46:08.045233Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 656 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:08.045275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 656 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:08.045282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=656 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:08.045285Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:08.045855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 656 sessionId:  describe result for acl check
2025-10-09T21:46:08.045901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:08.045904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:08.045905Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:08.045911Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341591253118149:6893] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:08.046788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341591253118149:6893] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:08.054829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341591253118149:6893] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:08.071509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341591253118149:6893] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:08.071531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341591253118149:6893] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:08.071536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341591253118149:6893] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:08.071546Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 656 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:08.130996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:08.134433Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341591253118380:6527]
2025-10-09T21:46:08.141012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:08.141299Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:08.141304Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:08.141317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:08.141381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:08.141421Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341590891304278:6271]
2025-10-09T21:46:08.142095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:08.144012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:08.144080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:08.144436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:08.144482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:08.144483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:08.144484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:08.144485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:08.144486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:08.144487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:08.144488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:08.144490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:08.144491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:08.144497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:08.144508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:08.144522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:08.144526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:08.144530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:08.144533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:08.144536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:08.144539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:08.144543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:08.144550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:08.144559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:08.144565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:08.144569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:08.144573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:08.144577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:08.144581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:08.144585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:08.144589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:08.144594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:08.144597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:08.144601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:08.144605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:08.144610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:08.144613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:08.144619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:08.144624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:08.144628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:08.144652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:08.166631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:08.168175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:08.168183Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:08.168186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:08.168190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:08.168198Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 655 [2:7559341590891304278:6271]
2025-10-09T21:46:08.168214Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:08.168235Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:08.168238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:08.168242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:08.168243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:08.168262Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:08.168268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:08.168274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:08.168342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:08.168344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:08.168346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:08.168349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:08.168428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:08.168430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:08.168443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:08.168454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:08.168468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:08.168489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341590891304258:6271]
2025-10-09T21:46:08.168491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341590891304258:6271]
2025-10-09T21:46:08.168493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341590891304258:6271]
2025-10-09T21:46:08.168508Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:08.168524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:08.168538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:08.168541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:08.168582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341590891304258:6271]
2025-10-09T21:46:08.168585Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:08.169141Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:08.169151Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:08.169153Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:08.169161Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:08.169591Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 655
2025-10-09T21:46:08.169630Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 655, pipe: [1:7559341591253118380:6527]
2025-10-09T21:46:08.169650Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 655
2025-10-09T21:46:08.171310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341591253118185:6893], now have 1 active actors on pipe
2025-10-09T21:46:08.171329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341591253118380:6527], now have 1 active actors on pipe
2025-10-09T21:46:08.171332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341591253118140:2432], now have 1 active actors on pipe
2025-10-09T21:46:08.171351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:08.171362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:08.171413Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|abaf73c7-fdac2941-4db670fd-23142dc8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:08.171427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:08.171433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:08.171437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:08.171459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:08.171480Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:08.171505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:08.172000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:08.172011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:08.172029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:08.172197Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 656 partition: 0 MaxSeqNo: 25 sessionId: src_id|abaf73c7-fdac2941-4db670fd-23142dc8_0
2025-10-09T21:46:08.172711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e59fd0c-4bd3d1a5-2e01e08c-1ee109b1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|abaf73c7-fdac2941-4db670fd-23142dc8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:08.175578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|abaf73c7-fdac2941-4db670fd-23142dc8_0] Write session: try to update token
2025-10-09T21:46:08.175604Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|abaf73c7-fdac2941-4db670fd-23142dc8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:08.204411Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 656 sessionId: src_id|abaf73c7-fdac2941-4db670fd-23142dc8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:08.207229Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:08.207996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:08.208018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:08.208058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:08.208235Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:08.254465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:08.254496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:08.254677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:08.254738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:08.254802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:08.254869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:08.254914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:08.254919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:08.254991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:08.255056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:08.255120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:08.255188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:08.255230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:08.255233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:08.255303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:08.255363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:08.255426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:08.255487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:08.255528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:08.255530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:08.255596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:08.255661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:08.255732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:08.255796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:08.256074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:08.256077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:08.256147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:08.256207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:08.256272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:08.256342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:08.256384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:08.256387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:08.256461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:08.256524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:08.256590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:08.256657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:08.256726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:08.256731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:08.256801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:08.256864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:08.256928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:08.256993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:08.257037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:08.257040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:08.257107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:08.257173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:08.257236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:08.257303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:08.257346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:08.257348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:08.257416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:08.257482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:08.257547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:08.257619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:08.257662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:08.257664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:08.257855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:08.257918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:08.257980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:08.310170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:08.310247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:08.310259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:08.310389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:08.310399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:08.310401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:08.310422Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:46:08.310483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:08.310503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:08.310506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (50)
2025-10-09T21:46:08.310545Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 0
2025-10-09T21:46:08.310560Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 1
2025-10-09T21:46:08.310564Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 2
2025-10-09T21:46:08.310567Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 3
2025-10-09T21:46:08.310570Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 4
2025-10-09T21:46:08.311046Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 26 partNo 4 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 2253230 count 1 nextOffset 26 batches 5
2025-10-09T21:46:08.311064Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 0
2025-10-09T21:46:08.311073Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 1
2025-10-09T21:46:08.311076Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 2
2025-10-09T21:46:08.311079Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 3
2025-10-09T21:46:08.311082Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 4
2025-10-09T21:46:08.312368Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 27 partNo 4 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 4506428 count 2 nextOffset 27 batches 9
2025-10-09T21:46:08.312387Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 0
2025-10-09T21:46:08.312396Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 1
2025-10-09T21:46:08.312400Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 2
2025-10-09T21:46:08.312403Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 3
2025-10-09T21:46:08.312407Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 4
2025-10-09T21:46:08.312918Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 28 partNo 4 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 6759626 count 3 nextOffset 28 batches 13
2025-10-09T21:46:08.312933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 0
2025-10-09T21:46:08.312938Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 1
2025-10-09T21:46:08.312941Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 2
2025-10-09T21:46:08.312945Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 3
2025-10-09T21:46:08.423469Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1380: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 29 partNo 3 result is x0000000000_00000000000000000025_00000_0000000003_00015? size 8295638
2025-10-09T21:46:08.423501Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 4
2025-10-09T21:46:08.423517Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1098: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000025_00000_0000000003_00015? new key d0000000000_00000000000000000025_00000_0000000003_00015? size 8295638 WTime 1760046368422
2025-10-09T21:46:08.423805Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 29 partNo 4 FormedBlobsCount 1 NewHead: Offset 28 PartNo 3 PackedSize 717145 count 1 nextOffset 29 batches 2
2025-10-09T21:46:08.423817Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 0
2025-10-09T21:46:08.423825Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 1
2025-10-09T21:46:08.423828Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 2
2025-10-09T21:46:08.423833Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 3
2025-10-09T21:46:08.423836Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 4
2025-10-09T21:46:08.424336Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 30 partNo 4 FormedBlobsCount 0 NewHead: Offset 28 PartNo 3 PackedSize 2970343 count 2 nextOffset 30 batches 6
2025-10-09T21:46:08.424349Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 0
2025-10-09T21:46:08.424354Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 1
2025-10-09T21:46:08.424358Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 2
2025-10-09T21:46:08.424361Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 3
2025-10-09T21:46:08.424365Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 4
2025-10-09T21:46:08.424882Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 31 partNo 4 FormedBlobsCount 0 NewHead: Offset 28 PartNo 3 PackedSize 5223541 count 3 nextOffset 31 batches 10
2025-10-09T21:46:08.424892Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 0
2025-10-09T21:46:08.424898Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 1
2025-10-09T21:46:08.424901Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 2
2025-10-09T21:46:08.424905Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 3
2025-10-09T21:46:08.424909Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 4
2025-10-09T21:46:08.425462Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 32 partNo 4 FormedBlobsCount 0 NewHead: Offset 28 PartNo 3 PackedSize 7476739 count 4 nextOffset 32 batches 14
2025-10-09T21:46:08.425473Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 0
2025-10-09T21:46:08.425478Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 1
2025-10-09T21:46:08.426689Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1380: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob sourceId '\0src_id' seqNo 33 partNo 1 result is x0000000000_00000000000000000028_00003_0000000004_00014? size 7988760
2025-10-09T21:46:08.426698Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 2
2025-10-09T21:46:08.426704Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 3
2025-10-09T21:46:08.426708Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 4
2025-10-09T21:46:08.426717Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1098: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--test-topic' partition 0 old key x0000000000_00000000000000000028_00003_0000000004_00014? new key d0000000000_00000000000000000028_00003_0000000004_00014? size 7988760 WTime 1760046368426
2025-10-09T21:46:08.427082Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 33 partNo 4 FormedBlobsCount 1 NewHead: Offset 32 PartNo 1 PackedSize 1741199 count 1 nextOffset 33 batches 4
2025-10-09T21:46:08.427094Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 0
2025-10-09T21:46:08.427103Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 1
2025-10-09T21:46:08.427106Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 2
2025-10-09T21:46:08.427111Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 3
2025-10-09T21:46:08.427114Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 4
2025-10-09T21:46:08.427636Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 34 partNo 4 FormedBlobsCount 0 NewHead: Offset 32 PartNo 1 PackedSize 3994397 count 2 nextOffset 34 batches 8
2025-10-09T21:46:08.427648Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 0
2025-10-09T21:46:08.427654Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 1
2025-10-09T21:46:08.427658Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 2
2025-10-09T21:46:08.427661Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 3
2025-10-09T21:46:08.427665Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 4
2025-10-09T21:46:08.428173Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 35 partNo 4 FormedBlobsCount 0 NewHead: Offset 32 PartNo 1 PackedSize 6247595 count 3 nextOffset 35 batches 12
2025-10-09T21:46:08.428184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:08.428322Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 32,3 HeadOffset 25 endOffset 25 curOffset 35 d0000000000_00000000000000000032_00001_0000000003_00011? size 6247585 WTime 1760046368427
2025-10-09T21:46:08.586265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:46:08.586294Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:46:08.586325Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 25 partNo 0 count 3 size 8295638
2025-10-09T21:46:08.586335Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 28 partNo 3 count 4 size 7988760
2025-10-09T21:46:08.586343Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 32 partNo 1 count 3 size 6247585
2025-10-09T21:46:08.586360Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:08.586363Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:08.586366Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:08.586369Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:08.586379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341590891304258:6271]
2025-10-09T21:46:08.586384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341590891304258:6271]
2025-10-09T21:46:08.586387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341590891304258:6271]
2025-10-09T21:46:08.586434Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:08.586443Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:08.586447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341590891304258:6271]
2025-10-09T21:46:08.586459Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:08.586466Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:08.586470Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:08.586490Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:08.586513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:08.586522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:08.586525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:08.586686Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:09.050277Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:09.054209Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|abaf73c7-fdac2941-4db670fd-23142dc8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:09.054225Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|abaf73c7-fdac2941-4db670fd-23142dc8_0] Write session will restart in 0.008586s
2025-10-09T21:46:09.054248Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|abaf73c7-fdac2941-4db670fd-23142dc8_0] Write session: Do CDS request
2025-10-09T21:46:09.054255Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|abaf73c7-fdac2941-4db670fd-23142dc8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:09.052061Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:09.052078Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:09.052108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 656 reason: error at write: Disconnected sessionId: src_id|abaf73c7-fdac2941-4db670fd-23142dc8_0
2025-10-09T21:46:09.052288Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 656 sessionId: src_id|abaf73c7-fdac2941-4db670fd-23142dc8_0 is DEAD
2025-10-09T21:46:09.052424Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341591253118380:6527]
2025-10-09T21:46:09.052437Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:09.060099Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:09.064153Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|abaf73c7-fdac2941-4db670fd-23142dc8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:09.066729Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:09.066747Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 657
2025-10-09T21:46:09.067001Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 657 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:09.067030Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 657 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:09.067036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=657 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:09.067040Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:09.067215Z node 1 :BS_VDISK_PUT ERROR: blobstorage_skeleton.cpp:577: PDiskId# 1 VDISK[82000000:_:0:0:0]: (2181038080) TEvVPut: failed to pass the Hull check; id# [72075186224037892:655:1:2:1:8295638:1] status# {Status# BLOCKED} Marker# BSVS03
2025-10-09T21:46:09.068938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 657 sessionId:  describe result for acl check
2025-10-09T21:46:09.068968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:09.068969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:09.068971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:09.068976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341595548085741:6906] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:09.069475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341595548085741:6906] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:09.069966Z node 2 :BS_PROXY_PUT ERROR: dsproxy_put_impl.cpp:72: [200015b4b2c6ef57] Result# TEvPutResult {Id# [72075186224037892:655:1:2:1:8295638:0] Status# BLOCKED StatusFlags# { } ErrorReason# "Got VPutResult status# BLOCKED from VDiskId# [82000000:1:0:0:0]" ApproximateFreeSpaceShare# 0} GroupId# 2181038080 Marker# BPP12
2025-10-09T21:46:09.074203Z node 1 :BS_VDISK_PUT ERROR: blobstorage_skeleton.cpp:577: PDiskId# 1 VDISK[82000000:_:0:0:0]: (2181038080) TEvVPut: failed to pass the Hull check; id# [72075186224037892:655:1:2:2:7988760:1] status# {Status# BLOCKED} Marker# BSVS03
2025-10-09T21:46:09.080453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341595548085741:6906] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:09.096571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341595548085741:6906] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:09.096590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341595548085741:6906] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:09.096594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341595548085741:6906] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:09.096604Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 657 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:09.127416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:09.138984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:09.139269Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:09.139276Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:09.139290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:09.139343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:09.139383Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341595186271839:6287]
2025-10-09T21:46:09.139903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:09.140338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:09.140381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:09.140658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:09.140709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:09.140710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:09.140711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:09.140712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:09.140713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:09.140714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:09.140715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:09.140716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:09.140718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:09.140722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:09.140732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:09.140736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:09.140739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:09.140742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:09.140747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:09.140750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:09.140753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:09.140756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:09.140763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:09.140773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:09.140778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:09.140783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:09.140787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:09.140791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:09.140795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:09.140799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:09.140802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:09.140806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:09.140810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:09.140814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:09.140818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:09.140822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:09.140826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:09.140830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:09.140837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:09.140843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:09.140867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:09.151223Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341595548085968:6527]
2025-10-09T21:46:09.151297Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 656
2025-10-09T21:46:09.151329Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 656
2025-10-09T21:46:09.148699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:09.149955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:09.149962Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:09.149965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:09.149970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:09.149979Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 656 [2:7559341595186271839:6287]
2025-10-09T21:46:09.149995Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:09.150016Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:09.150020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:09.150023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:09.150026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:09.150057Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:09.150076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:09.150084Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:09.150152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:09.150154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:09.150156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:09.150160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:09.150235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:09.150238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:09.150253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:09.150265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:09.150280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:09.150302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341595186271834:6287]
2025-10-09T21:46:09.150304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341595186271834:6287]
2025-10-09T21:46:09.150306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341595186271834:6287]
2025-10-09T21:46:09.150315Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:09.150329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:09.150333Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:09.150343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:09.150345Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:09.150346Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:09.150350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:09.150362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341595548085729:2432], now have 1 active actors on pipe
2025-10-09T21:46:09.150365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341595548085818:6906], now have 1 active actors on pipe
2025-10-09T21:46:09.150404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341595186271834:6287]
2025-10-09T21:46:09.150414Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:09.150501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:09.151449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:09.151457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:09.151475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341595548085968:6527], now have 1 active actors on pipe
2025-10-09T21:46:09.151493Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|84d172a2-aeae54a5-f8dfec51-6aaa1dbd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:09.151502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:09.151507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:09.151509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:09.151521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:09.151535Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:09.151564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:09.151823Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 656, pipe: [1:7559341595548085968:6527]
2025-10-09T21:46:09.152046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:09.152055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:09.152071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:09.152222Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 657 partition: 0 MaxSeqNo: 25 sessionId: src_id|84d172a2-aeae54a5-f8dfec51-6aaa1dbd_0
2025-10-09T21:46:09.154335Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|abaf73c7-fdac2941-4db670fd-23142dc8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|84d172a2-aeae54a5-f8dfec51-6aaa1dbd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:09.157066Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|84d172a2-aeae54a5-f8dfec51-6aaa1dbd_0] Write session: try to update token
2025-10-09T21:46:09.157092Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|84d172a2-aeae54a5-f8dfec51-6aaa1dbd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:09.183241Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:09.183263Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:09.183266Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:09.183268Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:09.183283Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341595186271834:6287]
2025-10-09T21:46:09.183286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341595186271834:6287]
2025-10-09T21:46:09.183290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341595186271834:6287]
2025-10-09T21:46:09.183335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:09.183343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:09.183345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:09.183483Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:09.186773Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:09.186787Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:09.186791Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:09.188854Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 657 sessionId: src_id|84d172a2-aeae54a5-f8dfec51-6aaa1dbd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:09.191510Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:09.844264Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:09.845083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:09.845095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:09.845833Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:09.845851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:09.845910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 657 reason: error at write: Disconnected sessionId: src_id|84d172a2-aeae54a5-f8dfec51-6aaa1dbd_0
2025-10-09T21:46:09.846128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 657 sessionId: src_id|84d172a2-aeae54a5-f8dfec51-6aaa1dbd_0 is DEAD
2025-10-09T21:46:09.846287Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:09.846320Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341595548085968:6527]
2025-10-09T21:46:09.846830Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84d172a2-aeae54a5-f8dfec51-6aaa1dbd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:09.846839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84d172a2-aeae54a5-f8dfec51-6aaa1dbd_0] Write session will restart in 0.008209s
2025-10-09T21:46:09.846862Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84d172a2-aeae54a5-f8dfec51-6aaa1dbd_0] Write session: Do CDS request
2025-10-09T21:46:09.846869Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84d172a2-aeae54a5-f8dfec51-6aaa1dbd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:09.845597Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:09.856512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|84d172a2-aeae54a5-f8dfec51-6aaa1dbd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:09.856914Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:09.856936Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 658
2025-10-09T21:46:09.857558Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 658 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:09.857610Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 658 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:09.857622Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=658 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:09.857626Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:09.858082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 658 sessionId:  describe result for acl check
2025-10-09T21:46:09.858130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:09.858132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:09.858134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:09.858141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341595548086022:6919] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:09.858700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341595548086022:6919] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:09.866722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341595548086022:6919] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:09.876341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341595548086022:6919] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:09.876359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341595548086022:6919] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:09.876363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341595548086022:6919] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:09.876373Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 658 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:09.895730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:09.905597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:09.905871Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:09.905883Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:09.905896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:09.905950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:09.905994Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341595186272114:6303]
2025-10-09T21:46:09.906597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:09.907976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:09.908048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:09.908423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:09.908472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:09.908473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:09.908475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:09.908476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:09.908477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:09.908478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:09.908480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:09.908481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:09.908483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:09.908488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:09.908501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:09.908505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:09.908509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:09.908514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:09.908519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:09.908523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:09.908527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:09.908531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:09.908539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:09.908548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:09.908555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:09.908561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:09.908566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:09.908571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:09.908576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:09.908585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:09.908590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:09.908596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:09.908601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:09.908607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:09.908612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:09.908618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:09.908623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:09.908628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:09.908633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:09.908639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:09.908671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:09.923439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:09.924793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:09.924798Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:09.924800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:09.924804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:09.924810Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 657 [2:7559341595186272114:6303]
2025-10-09T21:46:09.924822Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:09.924838Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:09.924840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:09.924842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:09.924843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:09.924858Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:09.924862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:09.924869Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:09.924917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:09.924919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:09.924920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:09.924922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:09.924983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:09.924985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:09.924995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:09.925000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:09.925010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:09.925026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341595186272108:6303]
2025-10-09T21:46:09.925027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341595186272108:6303]
2025-10-09T21:46:09.925029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341595186272108:6303]
2025-10-09T21:46:09.925039Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:09.925050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:09.925058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:09.925059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:09.925091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341595186272108:6303]
2025-10-09T21:46:09.925093Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:09.926572Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 657
2025-10-09T21:46:09.926639Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 657
2025-10-09T21:46:09.926224Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:09.926234Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:09.926237Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:09.926243Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:09.927674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341595548086014:2432], now have 1 active actors on pipe
2025-10-09T21:46:09.927682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341595548086119:6919], now have 1 active actors on pipe
2025-10-09T21:46:09.927699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:09.927703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:09.927735Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|33ad8d03-32ad9d8d-6d300543-39ab5be3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:09.927743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:09.927746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:09.927748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:09.927757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:09.927769Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:09.927784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:09.928283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 658 partition: 0 MaxSeqNo: 25 sessionId: src_id|33ad8d03-32ad9d8d-6d300543-39ab5be3_0
2025-10-09T21:46:09.928736Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84d172a2-aeae54a5-f8dfec51-6aaa1dbd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|33ad8d03-32ad9d8d-6d300543-39ab5be3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:09.931335Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33ad8d03-32ad9d8d-6d300543-39ab5be3_0] Write session: try to update token
2025-10-09T21:46:09.931354Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33ad8d03-32ad9d8d-6d300543-39ab5be3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:09.928130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:09.928141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:09.928157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:09.945634Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341595548086251:6527]
2025-10-09T21:46:09.946467Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 657, pipe: [1:7559341595548086251:6527]
2025-10-09T21:46:09.946404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341595548086251:6527], now have 1 active actors on pipe
2025-10-09T21:46:09.948975Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:09.948993Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:09.948996Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:09.948998Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:09.949006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341595186272108:6303]
2025-10-09T21:46:09.949010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341595186272108:6303]
2025-10-09T21:46:09.949013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341595186272108:6303]
2025-10-09T21:46:09.949056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:09.949063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:09.949066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:09.949205Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:09.951291Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:09.951297Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:09.951300Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:09.994088Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 658 sessionId: src_id|33ad8d03-32ad9d8d-6d300543-39ab5be3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:09.996739Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:10.156229Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:10.156845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:10.156856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:10.158748Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:10.162133Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341595548086251:6527]
2025-10-09T21:46:10.162171Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:10.162212Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:10.162296Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 658 reason: status is not ok: tablet will be restarted right now sessionId: src_id|33ad8d03-32ad9d8d-6d300543-39ab5be3_0
2025-10-09T21:46:10.162473Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 658 sessionId: src_id|33ad8d03-32ad9d8d-6d300543-39ab5be3_0 is DEAD
2025-10-09T21:46:10.163400Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33ad8d03-32ad9d8d-6d300543-39ab5be3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:10.163414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33ad8d03-32ad9d8d-6d300543-39ab5be3_0] Write session will restart in 0.009738s
2025-10-09T21:46:10.163439Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33ad8d03-32ad9d8d-6d300543-39ab5be3_0] Write session: Do CDS request
2025-10-09T21:46:10.163446Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33ad8d03-32ad9d8d-6d300543-39ab5be3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:10.174717Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33ad8d03-32ad9d8d-6d300543-39ab5be3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:10.175065Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:10.175086Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 659
2025-10-09T21:46:10.175265Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 659 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:10.175299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 659 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:10.175305Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=659 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:10.175309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:10.175771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 659 sessionId:  describe result for acl check
2025-10-09T21:46:10.175811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:10.175813Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:10.175817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:10.175823Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341599843053582:6924] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:10.176340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341599843053582:6924] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:10.182471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341599843053582:6924] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:10.187986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341599843053582:6924] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:10.188007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341599843053582:6924] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:10.188011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341599843053582:6924] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:10.188022Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 659 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:10.204811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:10.209382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:10.209646Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:10.209657Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:10.209668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:10.209718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:10.209763Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341599481239641:6305]
2025-10-09T21:46:10.210339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:10.210680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:10.210750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:10.211094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:10.211153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:10.211154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:10.211155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:10.211156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:10.211158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:10.211160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:10.211161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.211163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.211164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.211169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:10.211180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:10.211183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:10.211188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:10.211192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:10.211196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:10.211199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.211203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.211207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.211215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:10.211223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:10.211229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:10.211234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:10.211240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:10.211245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:10.211248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:10.211254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:10.211258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:10.211262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:10.211266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:10.211271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:10.211277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.211282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.211287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.211293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.211297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.211301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.211330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:10.225539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:10.227691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:10.227715Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:10.227717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:10.227723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:10.227732Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 658 [2:7559341599481239641:6305]
2025-10-09T21:46:10.227748Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:10.227781Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:10.227794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:10.227798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:10.227801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:10.227823Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:10.227836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:10.227845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:10.227917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.227926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.227927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.227931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:10.228017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:10.228025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.228042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.228062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.228087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:10.228113Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341599481239636:6305]
2025-10-09T21:46:10.228121Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341599481239636:6305]
2025-10-09T21:46:10.228123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341599481239636:6305]
2025-10-09T21:46:10.228138Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:10.228162Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:10.228184Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:10.228193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:10.228252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341599481239636:6305]
2025-10-09T21:46:10.228261Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:10.228425Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:10.228434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:10.228437Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:10.228443Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:10.229087Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 658
2025-10-09T21:46:10.229121Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 658
2025-10-09T21:46:10.229284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341599843053668:6924], now have 1 active actors on pipe
2025-10-09T21:46:10.229298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341599843053562:2432], now have 1 active actors on pipe
2025-10-09T21:46:10.229888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:10.229900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:10.229936Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|108e0964-7d68182f-f4891525-599f4247_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:10.229947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:10.229952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:10.229955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:10.229970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:10.229985Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:10.230005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:10.231111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:10.231122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:10.232328Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 659 partition: 0 MaxSeqNo: 25 sessionId: src_id|108e0964-7d68182f-f4891525-599f4247_0
2025-10-09T21:46:10.231145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:10.234950Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33ad8d03-32ad9d8d-6d300543-39ab5be3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|108e0964-7d68182f-f4891525-599f4247_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:10.237625Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|108e0964-7d68182f-f4891525-599f4247_0] Write session: try to update token
2025-10-09T21:46:10.237644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|108e0964-7d68182f-f4891525-599f4247_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:10.260648Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:10.260672Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:10.260674Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:10.260677Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:10.260688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341599481239636:6305]
2025-10-09T21:46:10.260693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341599481239636:6305]
2025-10-09T21:46:10.260698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341599481239636:6305]
2025-10-09T21:46:10.260745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:10.260753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.260757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:10.260907Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:10.262584Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341599843053797:6527]
2025-10-09T21:46:10.262894Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:10.262912Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:10.262917Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:10.263168Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 658, pipe: [1:7559341599843053797:6527]
2025-10-09T21:46:10.277701Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 659 sessionId: src_id|108e0964-7d68182f-f4891525-599f4247_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:10.281024Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:10.461287Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|108e0964-7d68182f-f4891525-599f4247_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:10.461302Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|108e0964-7d68182f-f4891525-599f4247_0] Write session will restart in 0.005062s
2025-10-09T21:46:10.461326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|108e0964-7d68182f-f4891525-599f4247_0] Write session: Do CDS request
2025-10-09T21:46:10.461332Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|108e0964-7d68182f-f4891525-599f4247_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:10.460877Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:10.460895Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:10.460968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 659 reason: error at write: Disconnected sessionId: src_id|108e0964-7d68182f-f4891525-599f4247_0
2025-10-09T21:46:10.461150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 659 sessionId: src_id|108e0964-7d68182f-f4891525-599f4247_0 is DEAD
2025-10-09T21:46:10.461461Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:10.461496Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341599843053797:6527]
2025-10-09T21:46:10.460126Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:10.460376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341599843053797:6527], now have 1 active actors on pipe
2025-10-09T21:46:10.460429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:10.460437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:10.461504Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:10.467566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|108e0964-7d68182f-f4891525-599f4247_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:10.467935Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:10.467956Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 660
2025-10-09T21:46:10.468077Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 660 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:10.468111Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 660 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:10.468124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=660 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:10.468128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:10.468853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 660 sessionId:  describe result for acl check
2025-10-09T21:46:10.468898Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:10.468904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:10.468906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:10.468911Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341599843053814:6928] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:10.469470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341599843053814:6928] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:10.480167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341599843053814:6928] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:10.507503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341599843053814:6928] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:10.507523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341599843053814:6928] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:10.507526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341599843053814:6928] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:10.507535Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 660 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:10.533613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:10.539149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:10.539448Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:10.539452Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:10.539465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:10.539524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:10.539776Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341599481239907:6319]
2025-10-09T21:46:10.540387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:10.540876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:10.540929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:10.541264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:10.541312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:10.541313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:10.541315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:10.541316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:10.541317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:10.541318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:10.541320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.541321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.541322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.541328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:10.541337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:10.541342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:10.541346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:10.541350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:10.541354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:10.541358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.541362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.541367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.541375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:10.541385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:10.541391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:10.541397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:10.541402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:10.541407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:10.541412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:10.541418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:10.541423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:10.541428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:10.541433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:10.541438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:10.541444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.541449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.541454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.541460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.541465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.541471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.541502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:10.550965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:10.552159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:10.552166Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:10.552169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:10.552173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:10.552182Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 659 [2:7559341599481239907:6319]
2025-10-09T21:46:10.552197Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:10.552218Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:10.552222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:10.552225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:10.552228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:10.552248Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:10.552254Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:10.552262Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:10.552326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.552328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.552331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.552334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:10.552418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:10.552422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.552435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.552447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.552465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:10.552485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341599481239901:6319]
2025-10-09T21:46:10.552488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341599481239901:6319]
2025-10-09T21:46:10.552490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341599481239901:6319]
2025-10-09T21:46:10.552506Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:10.552525Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:10.552540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:10.552544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:10.552585Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341599481239901:6319]
2025-10-09T21:46:10.552587Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:10.552818Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:10.552828Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:10.552830Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:10.552836Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:10.554015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341599843053967:6928], now have 1 active actors on pipe
2025-10-09T21:46:10.554045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341599843053803:2432], now have 1 active actors on pipe
2025-10-09T21:46:10.554558Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 659
2025-10-09T21:46:10.554597Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 659
2025-10-09T21:46:10.558055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:10.558087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:10.558131Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c8eff849-6303b71f-11d68659-d0630488_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:10.558143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:10.558150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:10.558153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:10.558168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:10.558186Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:10.558532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:10.564379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:10.564401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:10.564443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:10.566878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341599843054051:6527], now have 1 active actors on pipe
2025-10-09T21:46:10.566184Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341599843054051:6527]
2025-10-09T21:46:10.566276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 660 partition: 0 MaxSeqNo: 25 sessionId: src_id|c8eff849-6303b71f-11d68659-d0630488_0
2025-10-09T21:46:10.567288Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|108e0964-7d68182f-f4891525-599f4247_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c8eff849-6303b71f-11d68659-d0630488_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:10.567334Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 659, pipe: [1:7559341599843054051:6527]
2025-10-09T21:46:10.577083Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8eff849-6303b71f-11d68659-d0630488_0] Write session: try to update token
2025-10-09T21:46:10.577110Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8eff849-6303b71f-11d68659-d0630488_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:10.579994Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:10.580014Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:10.580017Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:10.580019Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:10.580030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341599481239901:6319]
2025-10-09T21:46:10.580035Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341599481239901:6319]
2025-10-09T21:46:10.580039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341599481239901:6319]
2025-10-09T21:46:10.580084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:10.580092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.580096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:10.580246Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:10.583099Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:10.583114Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:10.583118Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:10.600712Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 660 sessionId: src_id|c8eff849-6303b71f-11d68659-d0630488_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:10.603850Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:10.774011Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:10.774716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:10.774733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:10.774999Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:10.775556Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:10.775599Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341599843054051:6527]
2025-10-09T21:46:10.775620Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:10.775654Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 660 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c8eff849-6303b71f-11d68659-d0630488_0
2025-10-09T21:46:10.776230Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8eff849-6303b71f-11d68659-d0630488_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:10.776240Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8eff849-6303b71f-11d68659-d0630488_0] Write session will restart in 0.008231s
2025-10-09T21:46:10.776265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8eff849-6303b71f-11d68659-d0630488_0] Write session: Do CDS request
2025-10-09T21:46:10.776270Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8eff849-6303b71f-11d68659-d0630488_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:10.775878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 660 sessionId: src_id|c8eff849-6303b71f-11d68659-d0630488_0 is DEAD
2025-10-09T21:46:10.785631Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8eff849-6303b71f-11d68659-d0630488_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:10.785815Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:10.785835Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 661
2025-10-09T21:46:10.785976Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 661 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:10.786008Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 661 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:10.786019Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=661 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:10.786023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:10.786527Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 661 sessionId:  describe result for acl check
2025-10-09T21:46:10.786578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:10.786580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:10.786582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:10.786588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341599843054084:6940] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:10.787147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341599843054084:6940] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:10.793072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341599843054084:6940] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:10.805645Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341599843054084:6940] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:10.805674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341599843054084:6940] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:10.805678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341599843054084:6940] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:10.805687Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 661 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:10.827910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:10.829738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:10.829997Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:10.830008Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:10.830022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:10.830112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:10.830157Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341599481240148:6323]
2025-10-09T21:46:10.830793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:10.833569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:10.833647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:10.834267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:10.834358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:10.834366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:10.834367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:10.834368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:10.834369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:10.834370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:10.834371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.834372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.834372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.834379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:10.834389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:10.834392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:10.834396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:10.834398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:10.834402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:10.834404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.834407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.834410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.834421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:10.834429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:10.834435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:10.834439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:10.834442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:10.834447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:10.834450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:10.834455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:10.834459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:10.834463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:10.834467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:10.834471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:10.834475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.834479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.834483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.834486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.834490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.834494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.834519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:10.844740Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 660
2025-10-09T21:46:10.844740Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 660
2025-10-09T21:46:10.843067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:10.844234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:10.844241Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:10.844243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:10.844248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:10.844256Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 660 [2:7559341599481240148:6323]
2025-10-09T21:46:10.844273Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:10.844293Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:10.844297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:10.844301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:10.844303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:10.844321Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:10.844328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:10.844336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:10.844401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.844404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.844406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.844409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:10.844490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:10.844494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.844506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:10.844519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:10.844537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:10.844557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341599481240142:6323]
2025-10-09T21:46:10.844560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341599481240142:6323]
2025-10-09T21:46:10.844562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341599481240142:6323]
2025-10-09T21:46:10.844570Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:10.844588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:10.844604Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:10.844607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:10.844610Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:10.844614Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:10.844616Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:10.844619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341599843054190:6940], now have 1 active actors on pipe
2025-10-09T21:46:10.844623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341599843054070:2432], now have 1 active actors on pipe
2025-10-09T21:46:10.844662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341599481240142:6323]
2025-10-09T21:46:10.844673Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:10.845085Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:10.845186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:10.845194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:10.845236Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|641df4b6-96e3f070-8d2ba33e-31509efb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:10.845245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:10.845249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:10.845251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:10.845262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:10.845284Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:10.845303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:10.845597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:10.845607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:10.845620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:10.845929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 661 partition: 0 MaxSeqNo: 25 sessionId: src_id|641df4b6-96e3f070-8d2ba33e-31509efb_0
2025-10-09T21:46:10.846572Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8eff849-6303b71f-11d68659-d0630488_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|641df4b6-96e3f070-8d2ba33e-31509efb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:10.849272Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|641df4b6-96e3f070-8d2ba33e-31509efb_0] Write session: try to update token
2025-10-09T21:46:10.849291Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|641df4b6-96e3f070-8d2ba33e-31509efb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:10.872190Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:10.872212Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:10.872215Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:10.872218Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:10.872229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341599481240142:6323]
2025-10-09T21:46:10.872235Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341599481240142:6323]
2025-10-09T21:46:10.872239Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341599481240142:6323]
2025-10-09T21:46:10.872282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:10.872290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:10.872293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:10.872451Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:10.875875Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341599843054310:6527]
2025-10-09T21:46:10.876155Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 660, pipe: [1:7559341599843054310:6527]
2025-10-09T21:46:10.875066Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:10.875082Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:10.875087Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:10.882172Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 661 sessionId: src_id|641df4b6-96e3f070-8d2ba33e-31509efb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:10.884834Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:11.051237Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:11.051680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341599843054310:6527], now have 1 active actors on pipe
2025-10-09T21:46:11.051713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:11.051721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:11.052070Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:11.052302Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341599843054310:6527]
2025-10-09T21:46:11.052354Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:11.052358Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:11.052418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 661 reason: error at write: Disconnected sessionId: src_id|641df4b6-96e3f070-8d2ba33e-31509efb_0
2025-10-09T21:46:11.052629Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 661 sessionId: src_id|641df4b6-96e3f070-8d2ba33e-31509efb_0 is DEAD
2025-10-09T21:46:11.052745Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:11.052891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|641df4b6-96e3f070-8d2ba33e-31509efb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:11.052904Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|641df4b6-96e3f070-8d2ba33e-31509efb_0] Write session will restart in 0.006779s
2025-10-09T21:46:11.052933Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|641df4b6-96e3f070-8d2ba33e-31509efb_0] Write session: Do CDS request
2025-10-09T21:46:11.052940Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|641df4b6-96e3f070-8d2ba33e-31509efb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:11.060671Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|641df4b6-96e3f070-8d2ba33e-31509efb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:11.060803Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:11.060823Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 662
2025-10-09T21:46:11.061139Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 662 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:11.061174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 662 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:11.061180Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=662 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:11.061184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:11.061614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 662 sessionId:  describe result for acl check
2025-10-09T21:46:11.061649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:11.061650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:11.061652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:11.061658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341604138021624:6944] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:11.062213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341604138021624:6944] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:11.064635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341604138021624:6944] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:11.074833Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341604138021624:6944] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:11.074855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341604138021624:6944] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:11.074860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341604138021624:6944] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:11.074871Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 662 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:11.089639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:11.094205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:11.094497Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:11.094510Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:11.094524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:11.094614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:11.094661Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341603776207677:6325]
2025-10-09T21:46:11.095293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:11.096000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:11.096079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:11.096488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:11.096769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:11.096778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:11.096779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:11.096781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:11.096782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:11.096783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:11.096785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.096786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.096787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.096795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:11.096807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:11.096812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:11.096815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:11.096820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:11.096824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:11.096828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.096832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.096835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.096844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:11.096852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:11.096859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:11.096864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:11.096869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:11.096874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:11.096878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:11.096883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:11.096889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:11.096894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:11.096898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:11.096902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:11.096908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.096912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.096917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.096922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.096926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.096929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.096957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:11.105511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:11.106754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:11.106782Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:11.106784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:11.106793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:11.106801Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 661 [2:7559341603776207677:6325]
2025-10-09T21:46:11.106817Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:11.106842Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:11.106846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:11.106850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:11.106853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:11.106873Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:11.106880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:11.106887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:11.106954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.106956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.106958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.106961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:11.107038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:11.107048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.107063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.107077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.107097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:11.107124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341603776207672:6325]
2025-10-09T21:46:11.107131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341603776207672:6325]
2025-10-09T21:46:11.107134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341603776207672:6325]
2025-10-09T21:46:11.107150Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:11.107165Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:11.107190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:11.107193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:11.107235Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341603776207672:6325]
2025-10-09T21:46:11.107238Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:11.107326Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:11.107328Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:11.107330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:11.107337Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:11.107653Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 661
2025-10-09T21:46:11.107891Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 661
2025-10-09T21:46:11.107593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341604138021615:2432], now have 1 active actors on pipe
2025-10-09T21:46:11.107610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341604138021720:6944], now have 1 active actors on pipe
2025-10-09T21:46:11.107985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:11.107992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:11.108031Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|23c07053-4e0e4410-a803999b-a084daa5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:11.108039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:11.108045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:11.108048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:11.108059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:11.108083Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:11.108108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:11.108515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:11.108527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:11.108546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:11.108695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 662 partition: 0 MaxSeqNo: 25 sessionId: src_id|23c07053-4e0e4410-a803999b-a084daa5_0
2025-10-09T21:46:11.109164Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|641df4b6-96e3f070-8d2ba33e-31509efb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|23c07053-4e0e4410-a803999b-a084daa5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:11.112019Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23c07053-4e0e4410-a803999b-a084daa5_0] Write session: try to update token
2025-10-09T21:46:11.112042Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23c07053-4e0e4410-a803999b-a084daa5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:11.132680Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 662 sessionId: src_id|23c07053-4e0e4410-a803999b-a084daa5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:11.135384Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:11.135407Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:11.135425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341603776207672:6325]
2025-10-09T21:46:11.135492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:11.135554Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:11.135777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:11.135862Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:11.135975Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:11.137275Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:11.136168Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:11.136351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:11.136558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:11.136610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:11.136623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:11.136645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:11.136687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:11.144099Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:11.144121Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:11.144124Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:11.144126Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:11.144138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341603776207672:6325]
2025-10-09T21:46:11.144143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341603776207672:6325]
2025-10-09T21:46:11.144147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341603776207672:6325]
2025-10-09T21:46:11.144197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:11.144205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.144209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:11.144506Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:11.148797Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:11.148819Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:11.148824Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:11.152377Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341604138021859:6527]
2025-10-09T21:46:11.175455Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 661, pipe: [1:7559341604138021859:6527]
2025-10-09T21:46:11.340986Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:11.341496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341604138021859:6527], now have 1 active actors on pipe
2025-10-09T21:46:11.352428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:11.352461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:11.352643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:11.352726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:11.352787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:11.352856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:11.353311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:11.353318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:11.353392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:11.353451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:11.353514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:11.353589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:11.353635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:11.353639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:11.353711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:11.353776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:11.353842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:11.353910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:11.353955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:11.353958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:11.354099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:11.354166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:11.354231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:11.354298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:11.354353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:11.354357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:11.354432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:11.354499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:11.354566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:11.354636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:11.354679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:11.354682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:11.354759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:11.354817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:11.354878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:11.354940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:11.354989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:11.354993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:11.355063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:11.355124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:11.355194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:11.355260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:11.355303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:11.355305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:11.355378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:11.355439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:11.355499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:11.355559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:11.355602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:11.355606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:11.355675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:11.355741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:11.355802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:11.355872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:11.355914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:11.355917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:11.355989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:11.356047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:11.356108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:11.356172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:11.356215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:11.356218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:11.356674Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:11.356706Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:11.356711Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:11.356731Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341604138021859:6527]
2025-10-09T21:46:11.356748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 662 reason: error at write: Disconnected sessionId: src_id|23c07053-4e0e4410-a803999b-a084daa5_0
2025-10-09T21:46:11.357145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23c07053-4e0e4410-a803999b-a084daa5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:11.357169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23c07053-4e0e4410-a803999b-a084daa5_0] Write session will restart in 0.009296s
2025-10-09T21:46:11.357195Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23c07053-4e0e4410-a803999b-a084daa5_0] Write session: Do CDS request
2025-10-09T21:46:11.357203Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23c07053-4e0e4410-a803999b-a084daa5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:11.356965Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 662 sessionId: src_id|23c07053-4e0e4410-a803999b-a084daa5_0 is DEAD
2025-10-09T21:46:11.357303Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:11.370698Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23c07053-4e0e4410-a803999b-a084daa5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:11.371481Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:11.371503Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 663
2025-10-09T21:46:11.371791Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 663 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:11.371854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 663 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:11.371860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=663 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:11.371864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:11.375675Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 663 sessionId:  describe result for acl check
2025-10-09T21:46:11.375731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:11.375738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:11.375741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:11.375749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341604138021874:6949] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:11.376357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341604138021874:6949] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:11.385662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341604138021874:6949] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:11.414119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341604138021874:6949] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:11.414140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341604138021874:6949] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:11.414145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341604138021874:6949] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:11.414157Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 663 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:11.427102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:11.436557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:11.436821Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:11.436826Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:11.436838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:11.436897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:11.436938Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341603776207912:6327]
2025-10-09T21:46:11.437664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:11.438508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:11.438569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:11.438959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:11.439034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:11.439035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:11.439036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:11.439038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:11.439039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:11.439040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:11.439041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.439043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.439044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.439049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:11.439062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:11.439066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:11.439069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:11.439073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:11.439076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:11.439079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.439082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.439085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.439245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:11.439260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:11.439266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:11.439272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:11.439275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:11.439279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:11.439283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:11.439287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:11.439291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:11.439296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:11.439299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:11.439303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:11.439308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.439312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.439316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.439322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.439326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.439330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.439355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:11.448584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:11.450354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:11.450361Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:11.450364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:11.450371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:11.450380Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 662 [2:7559341603776207912:6327]
2025-10-09T21:46:11.450397Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:11.450420Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:11.450425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:11.450428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:11.450431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:11.450454Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:11.450461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:11.450470Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:11.450544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.450546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.450548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.450551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:11.450636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:11.450640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.450654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.450666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.450684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:11.450707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341603776207906:6327]
2025-10-09T21:46:11.450710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341603776207906:6327]
2025-10-09T21:46:11.450712Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341603776207906:6327]
2025-10-09T21:46:11.450726Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:11.450741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:11.450754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:11.450757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:11.450799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341603776207906:6327]
2025-10-09T21:46:11.450802Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:11.451101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:11.451109Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:11.451111Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:11.451118Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:11.451904Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 662
2025-10-09T21:46:11.451953Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 662
2025-10-09T21:46:11.454599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341604138022053:6949], now have 1 active actors on pipe
2025-10-09T21:46:11.454619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341604138021862:2432], now have 1 active actors on pipe
2025-10-09T21:46:11.454641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:11.454649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:11.454697Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9c6b0b3d-b73beb59-1b974c6a-e002f38_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:11.454709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:11.454714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:11.454717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:11.454731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:11.454750Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:11.454774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:11.456912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:11.456930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:11.456968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:11.457254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 663 partition: 0 MaxSeqNo: 25 sessionId: src_id|9c6b0b3d-b73beb59-1b974c6a-e002f38_0
2025-10-09T21:46:11.457666Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23c07053-4e0e4410-a803999b-a084daa5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9c6b0b3d-b73beb59-1b974c6a-e002f38_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:11.460532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c6b0b3d-b73beb59-1b974c6a-e002f38_0] Write session: try to update token
2025-10-09T21:46:11.460558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c6b0b3d-b73beb59-1b974c6a-e002f38_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:11.463652Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341604138022103:6527]
2025-10-09T21:46:11.465096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341604138022103:6527], now have 1 active actors on pipe
2025-10-09T21:46:11.465318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 662, pipe: [1:7559341604138022103:6527]
2025-10-09T21:46:11.480991Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:11.481013Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:11.481015Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:11.481017Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:11.481026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341603776207906:6327]
2025-10-09T21:46:11.481029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341603776207906:6327]
2025-10-09T21:46:11.481033Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341603776207906:6327]
2025-10-09T21:46:11.481075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:11.481082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.481084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:11.481246Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:11.482475Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 663 sessionId: src_id|9c6b0b3d-b73beb59-1b974c6a-e002f38_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:11.485020Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:11.484154Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:11.484168Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:11.484173Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:11.713512Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:11.714755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:11.714788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:11.715118Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:11.717332Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:11.717427Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 663 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9c6b0b3d-b73beb59-1b974c6a-e002f38_0
2025-10-09T21:46:11.717611Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 663 sessionId: src_id|9c6b0b3d-b73beb59-1b974c6a-e002f38_0 is DEAD
2025-10-09T21:46:11.717770Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341604138022103:6527]
2025-10-09T21:46:11.717785Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:11.717916Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c6b0b3d-b73beb59-1b974c6a-e002f38_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:11.717926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c6b0b3d-b73beb59-1b974c6a-e002f38_0] Write session will restart in 0.006672s
2025-10-09T21:46:11.717953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c6b0b3d-b73beb59-1b974c6a-e002f38_0] Write session: Do CDS request
2025-10-09T21:46:11.717961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c6b0b3d-b73beb59-1b974c6a-e002f38_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:11.729325Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c6b0b3d-b73beb59-1b974c6a-e002f38_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:11.729780Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:11.729800Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 664
2025-10-09T21:46:11.734412Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 664 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:11.734475Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 664 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:11.734482Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=664 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:11.734487Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:11.735086Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 664 sessionId:  describe result for acl check
2025-10-09T21:46:11.735136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:11.735144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:11.735146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:11.735153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341604138022156:6963] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:11.735773Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341604138022156:6963] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:11.753587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341604138022156:6963] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:11.764648Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341604138022156:6963] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:11.764672Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341604138022156:6963] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:11.764677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341604138022156:6963] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:11.764687Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 664 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:11.781169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:11.782253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:11.782475Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:11.782480Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:11.782491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:11.782542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:11.782591Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341603776208190:6343]
2025-10-09T21:46:11.783240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:11.783612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:11.783651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:11.783985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:11.784029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:11.784031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:11.784032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:11.784033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:11.784034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:11.784035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:11.784036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.784037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.784038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.784043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:11.784053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:11.784056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:11.784060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:11.784063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:11.784067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:11.784071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.784074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.784078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.784085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:11.784093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:11.784099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:11.784105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:11.784112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:11.784116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:11.784122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:11.784126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:11.784132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:11.784136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:11.784139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:11.784143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:11.784147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.784152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.784155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.784160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.784164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.784169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.784192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:11.795610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:11.797006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:11.797013Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:11.797015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:11.797020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:11.797028Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 663 [2:7559341603776208190:6343]
2025-10-09T21:46:11.797061Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:11.797081Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:11.797085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:11.797088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:11.797090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:11.797110Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:11.797116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:11.797123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:11.797194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.797196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.797198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.797201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:11.797279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:11.797282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.797298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:11.797309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:11.797327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:11.797350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341603776208185:6343]
2025-10-09T21:46:11.797353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341603776208185:6343]
2025-10-09T21:46:11.797354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341603776208185:6343]
2025-10-09T21:46:11.797368Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:11.797385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:11.797402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:11.797405Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:11.797448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341603776208185:6343]
2025-10-09T21:46:11.797451Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:11.798338Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:11.798351Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:11.798353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:11.798360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:11.800661Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 663
2025-10-09T21:46:11.800702Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 663
2025-10-09T21:46:11.798512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341604138022337:6963], now have 1 active actors on pipe
2025-10-09T21:46:11.798537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341604138022140:2432], now have 1 active actors on pipe
2025-10-09T21:46:11.801111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:11.801122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:11.801163Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ea7651fa-8b79bf45-b43352f6-385b084_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:11.801178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:11.801184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:11.801187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:11.801201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:11.801221Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:11.801243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:11.802702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 664 partition: 0 MaxSeqNo: 25 sessionId: src_id|ea7651fa-8b79bf45-b43352f6-385b084_0
2025-10-09T21:46:11.803181Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c6b0b3d-b73beb59-1b974c6a-e002f38_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ea7651fa-8b79bf45-b43352f6-385b084_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:11.805996Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea7651fa-8b79bf45-b43352f6-385b084_0] Write session: try to update token
2025-10-09T21:46:11.806018Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea7651fa-8b79bf45-b43352f6-385b084_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:11.801810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:11.801827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:11.801850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:11.817079Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341604138022379:6527]
2025-10-09T21:46:11.818237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341604138022379:6527], now have 1 active actors on pipe
2025-10-09T21:46:11.818364Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 663, pipe: [1:7559341604138022379:6527]
2025-10-09T21:46:11.855314Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 664 sessionId: src_id|ea7651fa-8b79bf45-b43352f6-385b084_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:11.858178Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:11.858722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:11.858749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:11.858789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:11.864255Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:11.864278Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:11.864281Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:11.864284Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:11.864293Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341603776208185:6343]
2025-10-09T21:46:11.864304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341603776208185:6343]
2025-10-09T21:46:11.864308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341603776208185:6343]
2025-10-09T21:46:11.864354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:11.864363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:11.864366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:11.864514Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:11.869085Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:11.869104Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:11.869116Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:11.869416Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:12.109252Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:12.121114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:12.121145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:12.121319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:12.121379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:12.121436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:12.121490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:12.122199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:12.122206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:12.122293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:12.122349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:12.122409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:12.122500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:12.122562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:12.122565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:12.122667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:12.122745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:12.122803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:12.122868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:12.122910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:12.122913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:12.122984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:12.123044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:12.123102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:12.123158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:12.123296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:12.123299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:12.123363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:12.123418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:12.123472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:12.123528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:12.123566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:12.123569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:12.123630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:12.123679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:12.123731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:12.123787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:12.123825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:12.123830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:12.123893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:12.123946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:12.124005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:12.124061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:12.124099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:12.124103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:12.124163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:12.124218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:12.124275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:12.124331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:12.124435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:12.124438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:12.124504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:12.124562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:12.124619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:12.124678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:12.124720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:12.124723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:12.124791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:12.124844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:12.124901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:12.124964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:12.125004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:12.125007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:12.125714Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:12.126152Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341604138022379:6527]
2025-10-09T21:46:12.126197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:12.126199Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:12.126231Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 664 reason: error at write: Disconnected sessionId: src_id|ea7651fa-8b79bf45-b43352f6-385b084_0
2025-10-09T21:46:12.126439Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 664 sessionId: src_id|ea7651fa-8b79bf45-b43352f6-385b084_0 is DEAD
2025-10-09T21:46:12.126556Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:12.129727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea7651fa-8b79bf45-b43352f6-385b084_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:12.129743Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea7651fa-8b79bf45-b43352f6-385b084_0] Write session will restart in 0.007681s
2025-10-09T21:46:12.129772Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea7651fa-8b79bf45-b43352f6-385b084_0] Write session: Do CDS request
2025-10-09T21:46:12.129790Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea7651fa-8b79bf45-b43352f6-385b084_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:12.139012Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea7651fa-8b79bf45-b43352f6-385b084_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:12.140412Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:12.140432Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 665
2025-10-09T21:46:12.140632Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 665 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:12.140667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 665 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:12.140673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=665 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:12.140677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:12.141405Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 665 sessionId:  describe result for acl check
2025-10-09T21:46:12.141464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:12.141466Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:12.141468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:12.141473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341608432989701:6968] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:12.142107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341608432989701:6968] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:12.159397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341608432989701:6968] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:12.204848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341608432989701:6968] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:12.204871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341608432989701:6968] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:12.204875Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341608432989701:6968] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:12.204886Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 665 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:12.229329Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341608432989913:6527]
2025-10-09T21:46:12.241796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:12.251163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:12.251480Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:12.251494Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:12.251511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:12.251589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:12.251639Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341608071175726:6345]
2025-10-09T21:46:12.252262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:12.256043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:12.256126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:12.256516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:12.256598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:12.256604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:12.256606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:12.256607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:12.256609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:12.256610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:12.256612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:12.256613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:12.256614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:12.256622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:12.256634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:12.256639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:12.256644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:12.256648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:12.256652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:12.256656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:12.256660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:12.256665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:12.256673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:12.256683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:12.256690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:12.256696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:12.256701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:12.256706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:12.256714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:12.256720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:12.256725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:12.256730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:12.256734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:12.256740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:12.256745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:12.256750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:12.256755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:12.256760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:12.256767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:12.256772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:12.256801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:12.269012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:12.271019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:12.271037Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:12.271040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:12.271045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:12.271052Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 664 [2:7559341608071175726:6345]
2025-10-09T21:46:12.271067Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:12.271090Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:12.271093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:12.271097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:12.271099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:12.271118Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:12.271126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:12.271134Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:12.271208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:12.271210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:12.271212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:12.271216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:12.271293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:12.271296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:12.271313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:12.271325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:12.271343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:12.271365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341608071175720:6345]
2025-10-09T21:46:12.271368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341608071175720:6345]
2025-10-09T21:46:12.271369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341608071175720:6345]
2025-10-09T21:46:12.271385Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:12.271400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:12.271415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:12.271418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:12.271462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341608071175720:6345]
2025-10-09T21:46:12.271464Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:12.273747Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 664
2025-10-09T21:46:12.273794Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 664
2025-10-09T21:46:12.273813Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 664, pipe: [1:7559341608432989913:6527]
2025-10-09T21:46:12.272238Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:12.272250Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:12.272252Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:12.272259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:12.280581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341608432989693:2432], now have 1 active actors on pipe
2025-10-09T21:46:12.280604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341608432989839:6968], now have 1 active actors on pipe
2025-10-09T21:46:12.280607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341608432989913:6527], now have 1 active actors on pipe
2025-10-09T21:46:12.280656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:12.280663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:12.280706Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5ec51960-4545a588-33726aea-6fb829f7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:12.280721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:12.280728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:12.280731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:12.280746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:12.280767Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:12.280789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:12.281786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:12.281803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:12.281828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:12.282050Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 665 partition: 0 MaxSeqNo: 25 sessionId: src_id|5ec51960-4545a588-33726aea-6fb829f7_0
2025-10-09T21:46:12.286306Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea7651fa-8b79bf45-b43352f6-385b084_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5ec51960-4545a588-33726aea-6fb829f7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:12.288950Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ec51960-4545a588-33726aea-6fb829f7_0] Write session: try to update token
2025-10-09T21:46:12.288976Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ec51960-4545a588-33726aea-6fb829f7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:12.313572Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:12.313595Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:12.313598Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:12.313600Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:12.313611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341608071175720:6345]
2025-10-09T21:46:12.313616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341608071175720:6345]
2025-10-09T21:46:12.313620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341608071175720:6345]
2025-10-09T21:46:12.313667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:12.313675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:12.313678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:12.313844Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:12.319365Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:12.319383Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:12.319387Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:12.341918Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 665 sessionId: src_id|5ec51960-4545a588-33726aea-6fb829f7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:12.344566Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:12.676396Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:12.676606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:12.676615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:12.678128Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:12.678185Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:12.678188Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:12.678261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 665 reason: error at write: Disconnected sessionId: src_id|5ec51960-4545a588-33726aea-6fb829f7_0
2025-10-09T21:46:12.678448Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 665 sessionId: src_id|5ec51960-4545a588-33726aea-6fb829f7_0 is DEAD
2025-10-09T21:46:12.678655Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341608432989913:6527]
2025-10-09T21:46:12.682154Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ec51960-4545a588-33726aea-6fb829f7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:12.682170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ec51960-4545a588-33726aea-6fb829f7_0] Write session will restart in 0.009437s
2025-10-09T21:46:12.682196Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ec51960-4545a588-33726aea-6fb829f7_0] Write session: Do CDS request
2025-10-09T21:46:12.682202Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ec51960-4545a588-33726aea-6fb829f7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:12.681277Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:12.696191Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ec51960-4545a588-33726aea-6fb829f7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:12.699711Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:12.699730Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 666
2025-10-09T21:46:12.703546Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 666 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:12.703607Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 666 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:12.703613Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=666 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:12.703619Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:12.704852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 666 sessionId:  describe result for acl check
2025-10-09T21:46:12.704895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:12.704898Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:12.704899Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:12.704906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341608432990001:6979] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:12.705483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341608432990001:6979] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:12.720356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341608432990001:6979] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:12.735680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341608432990001:6979] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:12.735700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341608432990001:6979] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:12.735704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341608432990001:6979] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:12.735714Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 666 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:12.748713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:12.748963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:12.749140Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:12.749143Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:12.749157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:12.749201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:12.749254Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341608071175998:6356]
2025-10-09T21:46:12.749859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:12.755096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:12.755183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:12.755588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:12.755638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:12.755639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:12.755641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:12.755642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:12.755644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:12.755646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:12.755647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:12.755648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:12.755650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:12.755657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:12.755669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:12.755673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:12.755678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:12.755683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:12.755688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:12.755692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:12.755696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:12.755701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:12.755709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:12.755720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:12.755727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:12.755732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:12.755738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:12.755744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:12.755749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:12.755755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:12.755761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:12.755766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:12.755773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:12.755779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:12.755785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:12.755792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:12.755796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:12.755803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:12.755809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:12.755815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:12.755848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:12.766380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:12.767956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:12.767963Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:12.767965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:12.767970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:12.767978Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 665 [2:7559341608071175998:6356]
2025-10-09T21:46:12.767996Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:12.768014Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:12.768018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:12.768021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:12.768024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:12.768046Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:12.768052Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:12.768059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:12.768126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:12.768128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:12.768130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:12.768134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:12.768215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:12.768218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:12.768233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:12.768247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:12.768265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:12.768286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341608071175993:6356]
2025-10-09T21:46:12.768289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341608071175993:6356]
2025-10-09T21:46:12.768291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341608071175993:6356]
2025-10-09T21:46:12.768305Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:12.768320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:12.768335Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:12.768338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:12.768379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341608071175993:6356]
2025-10-09T21:46:12.768382Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:12.769457Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 665
2025-10-09T21:46:12.769500Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 665
2025-10-09T21:46:12.768759Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:12.768768Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:12.768771Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:12.768777Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:12.769924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341608432989944:2432], now have 1 active actors on pipe
2025-10-09T21:46:12.769933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341608432990192:6979], now have 1 active actors on pipe
2025-10-09T21:46:12.769954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:12.769960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:12.769991Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bd75ff2c-f140ba9f-699622cb-3ad0e312_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:12.770001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:12.770005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:12.770008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:12.770021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:12.770071Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:12.770094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:12.770910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:12.770922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:12.770943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:12.771106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 666 partition: 0 MaxSeqNo: 25 sessionId: src_id|bd75ff2c-f140ba9f-699622cb-3ad0e312_0
2025-10-09T21:46:12.773433Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ec51960-4545a588-33726aea-6fb829f7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bd75ff2c-f140ba9f-699622cb-3ad0e312_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:12.776076Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd75ff2c-f140ba9f-699622cb-3ad0e312_0] Write session: try to update token
2025-10-09T21:46:12.776099Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd75ff2c-f140ba9f-699622cb-3ad0e312_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:12.782800Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341608432990212:6527]
2025-10-09T21:46:12.784900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341608432990212:6527], now have 1 active actors on pipe
2025-10-09T21:46:12.784991Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 665, pipe: [1:7559341608432990212:6527]
2025-10-09T21:46:12.796482Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:12.796506Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:12.796508Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:12.796510Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:12.796521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341608071175993:6356]
2025-10-09T21:46:12.796526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341608071175993:6356]
2025-10-09T21:46:12.796530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341608071175993:6356]
2025-10-09T21:46:12.796576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:12.796584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:12.796587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:12.796727Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:12.802103Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:12.802123Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:12.802128Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:12.815393Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 666 sessionId: src_id|bd75ff2c-f140ba9f-699622cb-3ad0e312_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:12.817971Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:13.099739Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:13.100619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:13.100638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:13.101755Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:13.101815Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341608432990212:6527]
2025-10-09T21:46:13.101837Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:13.101839Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:13.101897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 666 reason: error at write: Disconnected sessionId: src_id|bd75ff2c-f140ba9f-699622cb-3ad0e312_0
2025-10-09T21:46:13.102132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 666 sessionId: src_id|bd75ff2c-f140ba9f-699622cb-3ad0e312_0 is DEAD
2025-10-09T21:46:13.101514Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:13.106158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd75ff2c-f140ba9f-699622cb-3ad0e312_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:13.106175Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd75ff2c-f140ba9f-699622cb-3ad0e312_0] Write session will restart in 0.007592s
2025-10-09T21:46:13.106205Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd75ff2c-f140ba9f-699622cb-3ad0e312_0] Write session: Do CDS request
2025-10-09T21:46:13.106213Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd75ff2c-f140ba9f-699622cb-3ad0e312_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:13.114895Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd75ff2c-f140ba9f-699622cb-3ad0e312_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:13.115260Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:13.115282Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 667
2025-10-09T21:46:13.115501Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 667 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:13.115536Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 667 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:13.115542Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=667 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:13.115546Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:13.116221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 667 sessionId:  describe result for acl check
2025-10-09T21:46:13.116264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:13.116266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:13.116268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:13.116275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341612727957542:6985] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:13.116859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341612727957542:6985] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:13.124048Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341612727957542:6985] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:13.129232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341612727957542:6985] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:13.129251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341612727957542:6985] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:13.129255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341612727957542:6985] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:13.129265Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 667 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:13.162278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:13.163559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:13.164158Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:13.164176Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:13.164195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:13.164289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:13.164338Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341612366143541:6363]
2025-10-09T21:46:13.164920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:13.165499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:13.165559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:13.166077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:13.166161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:13.166163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:13.166164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:13.166165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:13.166167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:13.166169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:13.166170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.166172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.166173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.166179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:13.166191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:13.166195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:13.166198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:13.166202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:13.166206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:13.166209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.166213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.166216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.166225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:13.166233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:13.166239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:13.166244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:13.166249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:13.166254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:13.166259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:13.166263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:13.166268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:13.166273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:13.166277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:13.166282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:13.166290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.166294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.166299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.166303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.166307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.166314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.166343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:13.175425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:13.176560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:13.176565Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:13.176568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:13.176573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:13.176581Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 666 [2:7559341612366143541:6363]
2025-10-09T21:46:13.176597Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:13.176617Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:13.176620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:13.176623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:13.176625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:13.176644Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:13.176650Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:13.176658Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:13.176722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.176724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.176726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.176729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:13.176809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:13.176812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.176823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.176832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.176847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:13.176867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341612366143535:6363]
2025-10-09T21:46:13.176870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341612366143535:6363]
2025-10-09T21:46:13.176872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341612366143535:6363]
2025-10-09T21:46:13.176886Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:13.176900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:13.176914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:13.176917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:13.176954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341612366143535:6363]
2025-10-09T21:46:13.176957Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:13.177198Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:13.177201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:13.177203Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:13.177209Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:13.178556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341612727957529:2432], now have 1 active actors on pipe
2025-10-09T21:46:13.178568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341612727957603:6985], now have 1 active actors on pipe
2025-10-09T21:46:13.178591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:13.178596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:13.178627Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c6745b51-81b30c0e-f1eb8bac-cb530332_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:13.178636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:13.178641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:13.178643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:13.178656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:13.178670Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:13.178687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:13.178955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:13.178961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:13.178972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:13.179492Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd75ff2c-f140ba9f-699622cb-3ad0e312_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c6745b51-81b30c0e-f1eb8bac-cb530332_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:13.177925Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 666
2025-10-09T21:46:13.177969Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 666
2025-10-09T21:46:13.179099Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 667 partition: 0 MaxSeqNo: 25 sessionId: src_id|c6745b51-81b30c0e-f1eb8bac-cb530332_0
2025-10-09T21:46:13.182234Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c6745b51-81b30c0e-f1eb8bac-cb530332_0] Write session: try to update token
2025-10-09T21:46:13.182251Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c6745b51-81b30c0e-f1eb8bac-cb530332_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:13.201569Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341612727957773:6527]
2025-10-09T21:46:13.202065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341612727957773:6527], now have 1 active actors on pipe
2025-10-09T21:46:13.202215Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 666, pipe: [1:7559341612727957773:6527]
2025-10-09T21:46:13.229592Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 667 sessionId: src_id|c6745b51-81b30c0e-f1eb8bac-cb530332_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:13.225140Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:13.225162Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:13.225165Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:13.225167Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:13.225177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341612366143535:6363]
2025-10-09T21:46:13.225182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341612366143535:6363]
2025-10-09T21:46:13.225186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341612366143535:6363]
2025-10-09T21:46:13.225233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:13.225241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.225244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:13.225251Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:13.225259Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:13.225264Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:13.225395Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:13.232468Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:13.421986Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:13.423426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:13.423446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:13.424711Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:13.424767Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341612727957773:6527]
2025-10-09T21:46:13.424787Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:13.424790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:13.424848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 667 reason: error at write: Disconnected sessionId: src_id|c6745b51-81b30c0e-f1eb8bac-cb530332_0
2025-10-09T21:46:13.425033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 667 sessionId: src_id|c6745b51-81b30c0e-f1eb8bac-cb530332_0 is DEAD
2025-10-09T21:46:13.425453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6745b51-81b30c0e-f1eb8bac-cb530332_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:13.425468Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6745b51-81b30c0e-f1eb8bac-cb530332_0] Write session will restart in 0.005000s
2025-10-09T21:46:13.425494Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6745b51-81b30c0e-f1eb8bac-cb530332_0] Write session: Do CDS request
2025-10-09T21:46:13.425501Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6745b51-81b30c0e-f1eb8bac-cb530332_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:13.426709Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:13.434633Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c6745b51-81b30c0e-f1eb8bac-cb530332_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:13.434730Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:13.434750Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 668
2025-10-09T21:46:13.435018Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 668 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:13.435055Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 668 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:13.435060Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=668 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:13.435064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:13.435567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 668 sessionId:  describe result for acl check
2025-10-09T21:46:13.435602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:13.435608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:13.435609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:13.435616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341612727957786:6989] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:13.436170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341612727957786:6989] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:13.443483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341612727957786:6989] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:13.456808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341612727957786:6989] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:13.456830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341612727957786:6989] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:13.456834Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341612727957786:6989] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:13.456844Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 668 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:13.483207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:13.489091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:13.489369Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:13.489378Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:13.489391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:13.489455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:13.489498Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341612366143785:6365]
2025-10-09T21:46:13.490002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:13.490461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:13.490514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:13.490816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:13.490887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:13.490888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:13.490889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:13.490890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:13.490891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:13.490892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:13.490894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.490895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.490896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.490902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:13.490914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:13.490918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:13.490922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:13.490925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:13.490929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:13.490932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.490936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.490939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.490947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:13.490956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:13.490961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:13.490966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:13.490971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:13.490976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:13.490980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:13.490988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:13.490994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:13.490998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:13.491002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:13.491007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:13.491011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.491016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.491020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.491024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.491028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.491033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.491059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:13.499718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:13.501440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:13.501459Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:13.501461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:13.501466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:13.501474Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 667 [2:7559341612366143785:6365]
2025-10-09T21:46:13.501489Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:13.501511Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:13.501517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:13.501520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:13.501523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:13.501544Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:13.501550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:13.501557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:13.501627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.501647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.501649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.501652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:13.501734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:13.501737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.501750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.501760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.501775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:13.501797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341612366143779:6365]
2025-10-09T21:46:13.501799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341612366143779:6365]
2025-10-09T21:46:13.501801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341612366143779:6365]
2025-10-09T21:46:13.501817Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:13.501835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:13.501850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:13.501854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:13.501900Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341612366143779:6365]
2025-10-09T21:46:13.501902Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:13.502413Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:13.502424Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:13.502426Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:13.502433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:13.502876Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 667
2025-10-09T21:46:13.502911Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 667
2025-10-09T21:46:13.507209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341612727957836:6989], now have 1 active actors on pipe
2025-10-09T21:46:13.507228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341612727957779:2432], now have 1 active actors on pipe
2025-10-09T21:46:13.507254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:13.507262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:13.507324Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|730f44ce-62d786e6-da1a0ab2-b64834ea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:13.507337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:13.507345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:13.507348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:13.507366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:13.507386Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:13.507410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:13.509960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:13.509978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:13.510009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:13.510195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 668 partition: 0 MaxSeqNo: 25 sessionId: src_id|730f44ce-62d786e6-da1a0ab2-b64834ea_0
2025-10-09T21:46:13.510632Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6745b51-81b30c0e-f1eb8bac-cb530332_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|730f44ce-62d786e6-da1a0ab2-b64834ea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:13.513293Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|730f44ce-62d786e6-da1a0ab2-b64834ea_0] Write session: try to update token
2025-10-09T21:46:13.513310Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|730f44ce-62d786e6-da1a0ab2-b64834ea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:13.524044Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341612727958024:6527]
2025-10-09T21:46:13.524797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341612727958024:6527], now have 1 active actors on pipe
2025-10-09T21:46:13.524961Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 667, pipe: [1:7559341612727958024:6527]
2025-10-09T21:46:13.534797Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:13.534817Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:13.534820Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:13.534821Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:13.534831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341612366143779:6365]
2025-10-09T21:46:13.534835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341612366143779:6365]
2025-10-09T21:46:13.534839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341612366143779:6365]
2025-10-09T21:46:13.534882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:13.534890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.534893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:13.535020Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:13.536645Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:13.536661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:13.536665Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:13.556900Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 668 sessionId: src_id|730f44ce-62d786e6-da1a0ab2-b64834ea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:13.559651Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:13.747803Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:13.748335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:13.748357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:13.748689Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:13.749126Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341612727958024:6527]
2025-10-09T21:46:13.749164Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:13.749233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 668 reason: status is not ok: tablet will be restarted right now sessionId: src_id|730f44ce-62d786e6-da1a0ab2-b64834ea_0
2025-10-09T21:46:13.749426Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 668 sessionId: src_id|730f44ce-62d786e6-da1a0ab2-b64834ea_0 is DEAD
2025-10-09T21:46:13.749552Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:13.750097Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|730f44ce-62d786e6-da1a0ab2-b64834ea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:13.750111Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|730f44ce-62d786e6-da1a0ab2-b64834ea_0] Write session will restart in 0.008484s
2025-10-09T21:46:13.750148Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|730f44ce-62d786e6-da1a0ab2-b64834ea_0] Write session: Do CDS request
2025-10-09T21:46:13.750155Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|730f44ce-62d786e6-da1a0ab2-b64834ea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:13.760672Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|730f44ce-62d786e6-da1a0ab2-b64834ea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:13.760722Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:13.760742Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 669
2025-10-09T21:46:13.761014Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 669 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:13.761059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 669 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:13.761065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=669 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:13.761069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:13.761700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 669 sessionId:  describe result for acl check
2025-10-09T21:46:13.761745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:13.761755Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:13.761760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:13.761766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341612727958046:6994] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:13.762397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341612727958046:6994] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:13.767881Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341612727958046:6994] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:13.775743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341612727958046:6994] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:13.775766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341612727958046:6994] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:13.775772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341612727958046:6994] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:13.775783Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 669 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:13.804475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:13.819248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:13.819518Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:13.819532Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:13.819546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:13.819613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:13.819662Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341612366144056:6379]
2025-10-09T21:46:13.820289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:13.821652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:13.821708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:13.822069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:13.822149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:13.822150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:13.822151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:13.822152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:13.822154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:13.822155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:13.822156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.822157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.822158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.822163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:13.822173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:13.822177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:13.822181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:13.822184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:13.822188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:13.822191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.822194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.822197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.822205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:13.822214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:13.822219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:13.822224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:13.822230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:13.822234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:13.822237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:13.822241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:13.822245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:13.822249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:13.822253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:13.822258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:13.822262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.822266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.822270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.822275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.822279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.822284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.822310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:13.833543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:13.835261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:13.835269Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:13.835271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:13.835276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:13.835283Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 668 [2:7559341612366144056:6379]
2025-10-09T21:46:13.835299Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:13.835318Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:13.835322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:13.835326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:13.835328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:13.835346Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:13.835353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:13.835360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:13.835426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.835428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.835430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.835433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:13.835511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:13.835514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.835529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:13.835541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:13.835557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:13.835579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341612366144050:6379]
2025-10-09T21:46:13.835581Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341612366144050:6379]
2025-10-09T21:46:13.835583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341612366144050:6379]
2025-10-09T21:46:13.835597Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:13.835614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:13.835628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:13.835632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:13.835678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341612366144050:6379]
2025-10-09T21:46:13.835681Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:13.838532Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:13.838541Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:13.838543Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:13.838549Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:13.838781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341612727958032:2432], now have 1 active actors on pipe
2025-10-09T21:46:13.838786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341612727958098:6994], now have 1 active actors on pipe
2025-10-09T21:46:13.841016Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 668
2025-10-09T21:46:13.841065Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 668
2025-10-09T21:46:13.841764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:13.841794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:13.841833Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f4ddf99-5dd0c27d-95a62714-d5fa5dc2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:13.841848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:13.841853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:13.841856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:13.841871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:13.841890Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:13.841914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:13.842638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:13.842652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:13.842673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:13.842975Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 669 partition: 0 MaxSeqNo: 25 sessionId: src_id|f4ddf99-5dd0c27d-95a62714-d5fa5dc2_0
2025-10-09T21:46:13.843494Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|730f44ce-62d786e6-da1a0ab2-b64834ea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f4ddf99-5dd0c27d-95a62714-d5fa5dc2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:13.846218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f4ddf99-5dd0c27d-95a62714-d5fa5dc2_0] Write session: try to update token
2025-10-09T21:46:13.846238Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f4ddf99-5dd0c27d-95a62714-d5fa5dc2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:13.850226Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341612727958299:6527]
2025-10-09T21:46:13.850916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341612727958299:6527], now have 1 active actors on pipe
2025-10-09T21:46:13.851007Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 668, pipe: [1:7559341612727958299:6527]
2025-10-09T21:46:13.868892Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:13.868912Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:13.868915Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:13.868916Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:13.868926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341612366144050:6379]
2025-10-09T21:46:13.868931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341612366144050:6379]
2025-10-09T21:46:13.868935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341612366144050:6379]
2025-10-09T21:46:13.868980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:13.868988Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:13.868989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:13.868992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:13.868995Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:13.868999Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:13.869128Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:13.872394Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 669 sessionId: src_id|f4ddf99-5dd0c27d-95a62714-d5fa5dc2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:13.874986Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:14.039283Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:14.039586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:14.039598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:14.039955Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:14.040581Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:14.040606Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:14.040650Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341612727958299:6527]
2025-10-09T21:46:14.040695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 669 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f4ddf99-5dd0c27d-95a62714-d5fa5dc2_0
2025-10-09T21:46:14.040879Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 669 sessionId: src_id|f4ddf99-5dd0c27d-95a62714-d5fa5dc2_0 is DEAD
2025-10-09T21:46:14.041230Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f4ddf99-5dd0c27d-95a62714-d5fa5dc2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:14.041241Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f4ddf99-5dd0c27d-95a62714-d5fa5dc2_0] Write session will restart in 0.009454s
2025-10-09T21:46:14.041262Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f4ddf99-5dd0c27d-95a62714-d5fa5dc2_0] Write session: Do CDS request
2025-10-09T21:46:14.041269Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f4ddf99-5dd0c27d-95a62714-d5fa5dc2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:14.052946Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f4ddf99-5dd0c27d-95a62714-d5fa5dc2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:14.053419Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:14.053445Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 670
2025-10-09T21:46:14.053938Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 670 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:14.053986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 670 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:14.053992Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=670 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:14.054006Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:14.055358Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 670 sessionId:  describe result for acl check
2025-10-09T21:46:14.055401Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:14.055403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:14.055405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:14.055412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341617022925626:7005] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:14.055991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341617022925626:7005] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:14.062897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341617022925626:7005] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:14.068825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341617022925626:7005] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:14.068847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341617022925626:7005] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:14.068852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341617022925626:7005] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:14.068861Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 670 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:14.076736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:14.082334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:14.082618Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:14.082622Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:14.082635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:14.082711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:14.082750Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341616661111602:6383]
2025-10-09T21:46:14.083334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:14.083862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:14.083920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:14.084270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:14.084333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:14.084340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:14.084341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:14.084342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:14.084343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:14.084345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:14.084346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.084348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.084349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.084355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:14.084365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:14.084370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:14.084375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:14.084379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:14.084384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:14.084388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.084391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.084395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.084403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:14.084411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:14.084419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:14.084424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:14.084435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:14.084440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:14.084445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:14.084450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:14.084473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:14.084484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:14.084488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:14.084493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:14.084499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.084504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.084522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.084531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.084538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.084543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.084572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:14.095506Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 669
2025-10-09T21:46:14.095508Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 669
2025-10-09T21:46:14.093700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:14.094985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:14.094993Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:14.094996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:14.095002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:14.095010Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 669 [2:7559341616661111602:6383]
2025-10-09T21:46:14.095027Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:14.095047Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:14.095051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:14.095054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:14.095056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:14.095075Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:14.095083Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:14.095091Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:14.095167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.095170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.095172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.095175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:14.095266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:14.095269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.095283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.095292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.095308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:14.095329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341616661111597:6383]
2025-10-09T21:46:14.095332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341616661111597:6383]
2025-10-09T21:46:14.095334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341616661111597:6383]
2025-10-09T21:46:14.095343Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:14.095356Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:14.095358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:14.095360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:14.095362Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:14.095372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:14.095376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:14.095427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341617022925767:7005], now have 1 active actors on pipe
2025-10-09T21:46:14.095431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341617022925609:2432], now have 1 active actors on pipe
2025-10-09T21:46:14.095433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341616661111597:6383]
2025-10-09T21:46:14.095446Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:14.095487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:14.095714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:14.095723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:14.095759Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|de74d4f1-1f9ba7ee-d83a38e5-911d3818_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:14.095769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:14.095774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:14.095777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:14.095792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:14.095806Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:14.095823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:14.096292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:14.096317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:14.096338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:14.096699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 670 partition: 0 MaxSeqNo: 25 sessionId: src_id|de74d4f1-1f9ba7ee-d83a38e5-911d3818_0
2025-10-09T21:46:14.097132Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f4ddf99-5dd0c27d-95a62714-d5fa5dc2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|de74d4f1-1f9ba7ee-d83a38e5-911d3818_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:14.099814Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|de74d4f1-1f9ba7ee-d83a38e5-911d3818_0] Write session: try to update token
2025-10-09T21:46:14.099833Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|de74d4f1-1f9ba7ee-d83a38e5-911d3818_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:14.117092Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:14.117115Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:14.117118Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:14.117120Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:14.117130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341616661111597:6383]
2025-10-09T21:46:14.117135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341616661111597:6383]
2025-10-09T21:46:14.117139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341616661111597:6383]
2025-10-09T21:46:14.117187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:14.117193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.117196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:14.117207Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:14.117214Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:14.117218Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:14.117338Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:14.116418Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 670 sessionId: src_id|de74d4f1-1f9ba7ee-d83a38e5-911d3818_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:14.120739Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:14.140880Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341617022925862:6527]
2025-10-09T21:46:14.141139Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 669, pipe: [1:7559341617022925862:6527]
2025-10-09T21:46:14.299052Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:14.300236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:14.300262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:14.300311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341617022925862:6527], now have 1 active actors on pipe
2025-10-09T21:46:14.300555Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:14.300712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:14.300730Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:14.300815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 670 reason: error at write: Disconnected sessionId: src_id|de74d4f1-1f9ba7ee-d83a38e5-911d3818_0
2025-10-09T21:46:14.301003Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341617022925862:6527]
2025-10-09T21:46:14.301032Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 670 sessionId: src_id|de74d4f1-1f9ba7ee-d83a38e5-911d3818_0 is DEAD
2025-10-09T21:46:14.301046Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:14.301149Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de74d4f1-1f9ba7ee-d83a38e5-911d3818_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:14.301159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de74d4f1-1f9ba7ee-d83a38e5-911d3818_0] Write session will restart in 0.006349s
2025-10-09T21:46:14.301193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de74d4f1-1f9ba7ee-d83a38e5-911d3818_0] Write session: Do CDS request
2025-10-09T21:46:14.301199Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de74d4f1-1f9ba7ee-d83a38e5-911d3818_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:14.308573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|de74d4f1-1f9ba7ee-d83a38e5-911d3818_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:14.308696Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:14.308715Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 671
2025-10-09T21:46:14.308920Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 671 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:14.308959Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 671 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:14.308969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=671 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:14.308980Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:14.309429Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 671 sessionId:  describe result for acl check
2025-10-09T21:46:14.309466Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:14.309468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:14.309470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:14.309476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341617022925878:7010] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:14.310214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341617022925878:7010] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:14.317665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341617022925878:7010] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:14.327336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341617022925878:7010] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:14.327357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341617022925878:7010] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:14.327361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341617022925878:7010] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:14.327371Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 671 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:14.335912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:14.341460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:14.341725Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:14.341740Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:14.341753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:14.341824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:14.341874Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341616661111844:6385]
2025-10-09T21:46:14.342534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:14.343064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:14.343130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:14.343486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:14.343557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:14.343565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:14.343566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:14.343568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:14.343569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:14.343571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:14.343572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.343573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.343575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.343580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:14.343593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:14.343598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:14.343602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:14.343607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:14.343612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:14.343616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.343621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.343626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.343636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:14.343645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:14.343659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:14.343665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:14.343683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:14.343694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:14.343698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:14.343704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:14.343713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:14.343719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:14.343729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:14.343735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:14.343741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.343747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.343752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.343758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.343763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.343769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.343800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:14.353727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:14.355293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:14.355316Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:14.355319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:14.355323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:14.355331Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 670 [2:7559341616661111844:6385]
2025-10-09T21:46:14.355347Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:14.355367Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:14.355370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:14.355374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:14.355377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:14.355397Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:14.355404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:14.355415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:14.355483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.355495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.355497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.355499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:14.355576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:14.355579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.355594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.355607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.355625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:14.355654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341616661111838:6385]
2025-10-09T21:46:14.355657Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341616661111838:6385]
2025-10-09T21:46:14.355659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341616661111838:6385]
2025-10-09T21:46:14.355674Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:14.355690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:14.355712Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:14.355721Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:14.355764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341616661111838:6385]
2025-10-09T21:46:14.355767Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:14.355917Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:14.355928Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:14.355930Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:14.355937Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:14.356193Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 670
2025-10-09T21:46:14.356194Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 670
2025-10-09T21:46:14.356104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341617022926070:7010], now have 1 active actors on pipe
2025-10-09T21:46:14.356120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341617022925865:2432], now have 1 active actors on pipe
2025-10-09T21:46:14.356365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:14.356371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:14.356403Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|618f845f-8fbc9aa5-c1ae547f-bf01175c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:14.356420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:14.356425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:14.356428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:14.356440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:14.356453Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:14.356471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:14.356903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:14.356933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:14.356956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:14.357125Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 671 partition: 0 MaxSeqNo: 25 sessionId: src_id|618f845f-8fbc9aa5-c1ae547f-bf01175c_0
2025-10-09T21:46:14.357514Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de74d4f1-1f9ba7ee-d83a38e5-911d3818_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|618f845f-8fbc9aa5-c1ae547f-bf01175c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:14.360224Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|618f845f-8fbc9aa5-c1ae547f-bf01175c_0] Write session: try to update token
2025-10-09T21:46:14.360240Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|618f845f-8fbc9aa5-c1ae547f-bf01175c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:14.377450Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:14.377470Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:14.377472Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:14.377474Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:14.377485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341616661111838:6385]
2025-10-09T21:46:14.377490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341616661111838:6385]
2025-10-09T21:46:14.377494Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341616661111838:6385]
2025-10-09T21:46:14.377539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:14.377547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.377550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:14.377709Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:14.382002Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:14.382044Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:14.382059Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:14.388225Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 671 sessionId: src_id|618f845f-8fbc9aa5-c1ae547f-bf01175c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:14.390981Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:14.419434Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341617022926115:6527]
2025-10-09T21:46:14.420384Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 670, pipe: [1:7559341617022926115:6527]
2025-10-09T21:46:14.567146Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:14.567437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:14.567454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:14.567503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341617022926115:6527], now have 1 active actors on pipe
2025-10-09T21:46:14.567963Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341617022926115:6527]
2025-10-09T21:46:14.568013Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:14.568016Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:14.568442Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|618f845f-8fbc9aa5-c1ae547f-bf01175c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:14.568453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|618f845f-8fbc9aa5-c1ae547f-bf01175c_0] Write session will restart in 0.008858s
2025-10-09T21:46:14.568478Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|618f845f-8fbc9aa5-c1ae547f-bf01175c_0] Write session: Do CDS request
2025-10-09T21:46:14.568484Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|618f845f-8fbc9aa5-c1ae547f-bf01175c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:14.568074Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 671 reason: error at write: Disconnected sessionId: src_id|618f845f-8fbc9aa5-c1ae547f-bf01175c_0
2025-10-09T21:46:14.568267Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 671 sessionId: src_id|618f845f-8fbc9aa5-c1ae547f-bf01175c_0 is DEAD
2025-10-09T21:46:14.568401Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:14.568573Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:14.580119Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|618f845f-8fbc9aa5-c1ae547f-bf01175c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:14.583486Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:14.583504Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 672
2025-10-09T21:46:14.583669Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 672 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:14.583721Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 672 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:14.583732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=672 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:14.583736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:14.584317Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 672 sessionId:  describe result for acl check
2025-10-09T21:46:14.584358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:14.584365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:14.584369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:14.584375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341617022926140:7014] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:14.584866Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341617022926140:7014] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:14.595225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341617022926140:7014] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:14.600034Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341617022926140:7014] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:14.600051Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341617022926140:7014] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:14.600054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341617022926140:7014] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:14.600063Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 672 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:14.607332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:14.612222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:14.612461Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:14.612471Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:14.612484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:14.612532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:14.612572Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341616661112092:6389]
2025-10-09T21:46:14.613083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:14.613604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:14.613667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:14.613998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:14.614100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:14.614101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:14.614102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:14.614103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:14.614104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:14.614105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:14.614107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.614108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.614109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.614115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:14.614127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:14.614130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:14.614133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:14.614136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:14.614140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:14.614143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.614146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.614150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.614160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:14.614169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:14.614176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:14.614182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:14.614187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:14.614202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:14.614206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:14.614211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:14.614216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:14.614221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:14.614226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:14.614231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:14.614239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.614244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.614250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.614254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.614260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.614265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.614292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:14.625403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:14.627144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:14.627160Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:14.627163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:14.627168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:14.627175Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 671 [2:7559341616661112092:6389]
2025-10-09T21:46:14.627194Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:14.627213Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:14.627221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:14.627224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:14.627227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:14.627246Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:14.627256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:14.627263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:14.627335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.627345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.627347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.627350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:14.627650Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 671
2025-10-09T21:46:14.627662Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 671
2025-10-09T21:46:14.627439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:14.627447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.627458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.627469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.627486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:14.627506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341616661112087:6389]
2025-10-09T21:46:14.627509Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341616661112087:6389]
2025-10-09T21:46:14.627511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341616661112087:6389]
2025-10-09T21:46:14.627520Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:14.627529Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:14.627531Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:14.627533Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:14.627540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341617022926314:7014], now have 1 active actors on pipe
2025-10-09T21:46:14.627544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341617022926118:2432], now have 1 active actors on pipe
2025-10-09T21:46:14.627549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:14.627564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:14.627567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:14.627607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341616661112087:6389]
2025-10-09T21:46:14.627615Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:14.627627Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:14.627853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:14.627861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:14.627898Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4a295d71-f4d064f7-f31f3713-8b85459_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:14.627907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:14.627911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:14.627914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:14.627925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:14.627937Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:14.627953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:14.628285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:14.628296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:14.628316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:14.628528Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 672 partition: 0 MaxSeqNo: 25 sessionId: src_id|4a295d71-f4d064f7-f31f3713-8b85459_0
2025-10-09T21:46:14.630380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|618f845f-8fbc9aa5-c1ae547f-bf01175c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4a295d71-f4d064f7-f31f3713-8b85459_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:14.632901Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a295d71-f4d064f7-f31f3713-8b85459_0] Write session: try to update token
2025-10-09T21:46:14.632917Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a295d71-f4d064f7-f31f3713-8b85459_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:14.644542Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:14.644563Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:14.644565Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:14.644568Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:14.644581Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341616661112087:6389]
2025-10-09T21:46:14.644585Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341616661112087:6389]
2025-10-09T21:46:14.644587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341616661112087:6389]
2025-10-09T21:46:14.644633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:14.644639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.644642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:14.644638Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:14.644653Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:14.644657Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:14.644779Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:14.655760Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 672 sessionId: src_id|4a295d71-f4d064f7-f31f3713-8b85459_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:14.658480Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:14.668304Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341617022926370:6527]
2025-10-09T21:46:14.668764Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 671, pipe: [1:7559341617022926370:6527]
2025-10-09T21:46:14.809108Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:14.809569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:14.809583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:14.809604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341617022926370:6527], now have 1 active actors on pipe
2025-10-09T21:46:14.809867Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:14.810347Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341617022926370:6527]
2025-10-09T21:46:14.810398Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:14.810477Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 672 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4a295d71-f4d064f7-f31f3713-8b85459_0
2025-10-09T21:46:14.810662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 672 sessionId: src_id|4a295d71-f4d064f7-f31f3713-8b85459_0 is DEAD
2025-10-09T21:46:14.810780Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:14.811107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a295d71-f4d064f7-f31f3713-8b85459_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:14.811119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a295d71-f4d064f7-f31f3713-8b85459_0] Write session will restart in 0.008103s
2025-10-09T21:46:14.811145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a295d71-f4d064f7-f31f3713-8b85459_0] Write session: Do CDS request
2025-10-09T21:46:14.811151Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a295d71-f4d064f7-f31f3713-8b85459_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:14.820611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a295d71-f4d064f7-f31f3713-8b85459_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:14.820702Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:14.820723Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 673
2025-10-09T21:46:14.820885Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 673 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:14.820939Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 673 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:14.820950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=673 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:14.820954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:14.821491Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 673 sessionId:  describe result for acl check
2025-10-09T21:46:14.821532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:14.821534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:14.821555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:14.821566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341617022926396:7019] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:14.822184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341617022926396:7019] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:14.827257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341617022926396:7019] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:14.832751Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341617022926396:7019] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:14.832770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341617022926396:7019] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:14.832775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341617022926396:7019] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:14.832785Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 673 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:14.854880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:14.859203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:14.859613Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:14.859625Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:14.859640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:14.859706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:14.859752Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341616661112368:6401]
2025-10-09T21:46:14.860331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:14.861109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:14.861188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:14.861594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:14.861671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:14.861678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:14.861679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:14.861680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:14.861682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:14.861683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:14.861684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.861686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.861687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.861693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:14.861721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:14.861726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:14.861730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:14.861733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:14.861737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:14.861742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.861746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.861750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.861759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:14.861768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:14.861775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:14.861781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:14.861786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:14.861791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:14.861795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:14.861808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:14.861813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:14.861818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:14.861822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:14.861828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:14.861833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.861839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.861843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.861849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.861854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.861859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.861887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:14.871674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:14.873393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:14.873407Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:14.873409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:14.873414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:14.873420Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 672 [2:7559341616661112368:6401]
2025-10-09T21:46:14.873434Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:14.873454Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:14.873456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:14.873460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:14.873462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:14.873479Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:14.873494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:14.873501Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:14.873563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.873565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.873568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.873570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:14.873649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:14.873652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.873666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:14.873681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:14.873700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:14.873726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341617022926379:2432], now have 1 active actors on pipe
2025-10-09T21:46:14.873729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341617022926526:7019], now have 1 active actors on pipe
2025-10-09T21:46:14.873738Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341616661112362:6401]
2025-10-09T21:46:14.873741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341616661112362:6401]
2025-10-09T21:46:14.873743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341616661112362:6401]
2025-10-09T21:46:14.873757Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:14.873774Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:14.873803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:14.873808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:14.873852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341616661112362:6401]
2025-10-09T21:46:14.873854Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:14.873899Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:14.873903Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:14.873905Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:14.873923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:14.874546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:14.874560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:14.874600Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|86123528-c4fc5af4-3eb05f5-91f515d8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:14.874613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:14.874617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:14.874622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:14.874634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:14.874651Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:14.874684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:14.873846Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 672
2025-10-09T21:46:14.873911Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 672
2025-10-09T21:46:14.875098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:14.875110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:14.875144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:14.875857Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a295d71-f4d064f7-f31f3713-8b85459_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|86123528-c4fc5af4-3eb05f5-91f515d8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:14.875263Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 673 partition: 0 MaxSeqNo: 25 sessionId: src_id|86123528-c4fc5af4-3eb05f5-91f515d8_0
2025-10-09T21:46:14.878591Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86123528-c4fc5af4-3eb05f5-91f515d8_0] Write session: try to update token
2025-10-09T21:46:14.878610Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86123528-c4fc5af4-3eb05f5-91f515d8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:14.891273Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:14.891290Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:14.891293Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:14.891295Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:14.891305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341616661112362:6401]
2025-10-09T21:46:14.891310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341616661112362:6401]
2025-10-09T21:46:14.891314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341616661112362:6401]
2025-10-09T21:46:14.891358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:14.891365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:14.891366Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:14.891370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:14.891372Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:14.891374Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:14.891514Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:14.905363Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 673 sessionId: src_id|86123528-c4fc5af4-3eb05f5-91f515d8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:14.908626Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:14.910452Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341617022926647:6527]
2025-10-09T21:46:14.911642Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 672, pipe: [1:7559341617022926647:6527]
2025-10-09T21:46:15.073520Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:15.074387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:15.074407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:15.074457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341617022926647:6527], now have 1 active actors on pipe
2025-10-09T21:46:15.078734Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:15.079255Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341617022926647:6527]
2025-10-09T21:46:15.079300Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:15.079355Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:15.079436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 673 reason: status is not ok: tablet will be restarted right now sessionId: src_id|86123528-c4fc5af4-3eb05f5-91f515d8_0
2025-10-09T21:46:15.079626Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 673 sessionId: src_id|86123528-c4fc5af4-3eb05f5-91f515d8_0 is DEAD
2025-10-09T21:46:15.079806Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86123528-c4fc5af4-3eb05f5-91f515d8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:15.079820Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86123528-c4fc5af4-3eb05f5-91f515d8_0] Write session will restart in 0.007852s
2025-10-09T21:46:15.079848Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86123528-c4fc5af4-3eb05f5-91f515d8_0] Write session: Do CDS request
2025-10-09T21:46:15.079855Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86123528-c4fc5af4-3eb05f5-91f515d8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:15.089990Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86123528-c4fc5af4-3eb05f5-91f515d8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:15.094235Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:15.094259Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 674
2025-10-09T21:46:15.094441Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 674 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:15.094481Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 674 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:15.094486Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=674 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:15.094490Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:15.095189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 674 sessionId:  describe result for acl check
2025-10-09T21:46:15.095224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:15.095226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:15.095227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:15.095233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341621317893977:7031] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:15.095771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341621317893977:7031] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:15.099137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341621317893977:7031] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:15.110584Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341621317893977:7031] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:15.110604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341621317893977:7031] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:15.110608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341621317893977:7031] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:15.110619Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 674 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:15.137521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:15.144304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:15.144747Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:15.144752Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:15.144765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:15.144820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:15.144860Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341620956079917:6405]
2025-10-09T21:46:15.145414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:15.146827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:15.146881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:15.147328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:15.147367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:15.147369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:15.147370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:15.147371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:15.147372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:15.147373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:15.147374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.147375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.147376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.147381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:15.147391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:15.147395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:15.147398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:15.147401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:15.147405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:15.147408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.147411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.147414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.147422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:15.147430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:15.147436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:15.147440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:15.147444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:15.147449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:15.147453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:15.147457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:15.147461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:15.147465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:15.147469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:15.147474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:15.147479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.147484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.147487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.147492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.147496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.147501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.147531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:15.159244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:15.160505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:15.160512Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:15.160515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:15.160520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:15.160531Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 673 [2:7559341620956079917:6405]
2025-10-09T21:46:15.160549Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:15.160569Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:15.160573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:15.160576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:15.160579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:15.160599Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:15.160606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:15.160614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:15.160682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.160683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.160686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.160688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:15.160768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:15.160770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.160782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.160792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.160807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:15.160828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341620956079911:6405]
2025-10-09T21:46:15.160831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341620956079911:6405]
2025-10-09T21:46:15.160832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341620956079911:6405]
2025-10-09T21:46:15.160847Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:15.160867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:15.160880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:15.160883Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:15.160929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341620956079911:6405]
2025-10-09T21:46:15.160931Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:15.162401Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 673
2025-10-09T21:46:15.162439Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 673
2025-10-09T21:46:15.161276Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:15.161281Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:15.161284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:15.161290Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:15.162800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341621317894055:7031], now have 1 active actors on pipe
2025-10-09T21:46:15.162807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341621317893965:2432], now have 1 active actors on pipe
2025-10-09T21:46:15.162823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:15.162830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:15.162867Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1c17b66d-e5455be1-fa0e0178-9cb106f7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:15.162877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:15.162882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:15.162885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:15.162898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:15.162925Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:15.162946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:15.163383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:15.163389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:15.163403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:15.163556Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 674 partition: 0 MaxSeqNo: 25 sessionId: src_id|1c17b66d-e5455be1-fa0e0178-9cb106f7_0
2025-10-09T21:46:15.163994Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86123528-c4fc5af4-3eb05f5-91f515d8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1c17b66d-e5455be1-fa0e0178-9cb106f7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:15.167042Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c17b66d-e5455be1-fa0e0178-9cb106f7_0] Write session: try to update token
2025-10-09T21:46:15.167064Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c17b66d-e5455be1-fa0e0178-9cb106f7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:15.182911Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341621317894223:6527]
2025-10-09T21:46:15.183454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341621317894223:6527], now have 1 active actors on pipe
2025-10-09T21:46:15.183708Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 673, pipe: [1:7559341621317894223:6527]
2025-10-09T21:46:15.186292Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:15.186308Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:15.186311Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:15.186313Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:15.186324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341620956079911:6405]
2025-10-09T21:46:15.186328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341620956079911:6405]
2025-10-09T21:46:15.186332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341620956079911:6405]
2025-10-09T21:46:15.186376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:15.186383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.186389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:15.186549Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:15.190273Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:15.190287Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:15.190291Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:15.193154Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 674 sessionId: src_id|1c17b66d-e5455be1-fa0e0178-9cb106f7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:15.196304Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:15.376582Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:15.377329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:15.377340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:15.378538Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341621317894223:6527]
2025-10-09T21:46:15.378562Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:15.378567Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:15.378572Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:15.378708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 674 reason: error at write: Disconnected sessionId: src_id|1c17b66d-e5455be1-fa0e0178-9cb106f7_0
2025-10-09T21:46:15.378888Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 674 sessionId: src_id|1c17b66d-e5455be1-fa0e0178-9cb106f7_0 is DEAD
2025-10-09T21:46:15.379308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c17b66d-e5455be1-fa0e0178-9cb106f7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:15.379319Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c17b66d-e5455be1-fa0e0178-9cb106f7_0] Write session will restart in 0.009434s
2025-10-09T21:46:15.379344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c17b66d-e5455be1-fa0e0178-9cb106f7_0] Write session: Do CDS request
2025-10-09T21:46:15.379351Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c17b66d-e5455be1-fa0e0178-9cb106f7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:15.378738Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:15.394800Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c17b66d-e5455be1-fa0e0178-9cb106f7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:15.395156Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:15.395178Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 675
2025-10-09T21:46:15.395391Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 675 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:15.395422Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 675 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:15.395430Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=675 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:15.395434Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:15.395909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 675 sessionId:  describe result for acl check
2025-10-09T21:46:15.395948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:15.395952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:15.395954Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:15.395960Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341621317894250:7037] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:15.396523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341621317894250:7037] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:15.417602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341621317894250:7037] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:15.420566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:15.427447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:15.427746Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:15.427756Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:15.427770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:15.427836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:15.427886Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341620956080165:6407]
2025-10-09T21:46:15.427797Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341621317894250:7037] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:15.427821Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341621317894250:7037] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:15.427826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341621317894250:7037] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:15.427836Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 675 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:15.428811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:15.429273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:15.429323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:15.430098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:15.430167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:15.430168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:15.430170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:15.430171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:15.430172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:15.430173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:15.430174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.430175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.430176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.430182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:15.430211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:15.430215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:15.430219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:15.430222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:15.430226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:15.430230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.430233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.430237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.430247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:15.430257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:15.430263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:15.430268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:15.430272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:15.430277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:15.430281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:15.430286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:15.430290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:15.430295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:15.430300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:15.430305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:15.430309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.430314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.430318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.430323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.430328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.430333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.430359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:15.451111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:15.452259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:15.452275Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:15.452278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:15.452283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:15.452290Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 674 [2:7559341620956080165:6407]
2025-10-09T21:46:15.452305Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:15.452328Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:15.452335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:15.452339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:15.452341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:15.452359Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:15.452365Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:15.452376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:15.452438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.452444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.452447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.452450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:15.452523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:15.452530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.452541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.452551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.452569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:15.452593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341620956080158:6407]
2025-10-09T21:46:15.452600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341620956080158:6407]
2025-10-09T21:46:15.452602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341620956080158:6407]
2025-10-09T21:46:15.452620Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:15.452636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341621317894233:2432], now have 1 active actors on pipe
2025-10-09T21:46:15.452643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341621317894474:7037], now have 1 active actors on pipe
2025-10-09T21:46:15.452648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:15.452660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:15.452663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:15.452701Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341620956080158:6407]
2025-10-09T21:46:15.452704Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:15.452767Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 674
2025-10-09T21:46:15.452809Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 674
2025-10-09T21:46:15.452948Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:15.452958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:15.452960Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:15.452967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:15.453042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:15.453052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:15.453085Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f8396425-7b28549-426a787b-20c8e19c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:15.453094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:15.453099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:15.453102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:15.453113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:15.453125Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:15.453142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:15.454113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:15.454125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:15.454146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:15.459482Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 675 partition: 0 MaxSeqNo: 25 sessionId: src_id|f8396425-7b28549-426a787b-20c8e19c_0
2025-10-09T21:46:15.462929Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c17b66d-e5455be1-fa0e0178-9cb106f7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f8396425-7b28549-426a787b-20c8e19c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:15.465760Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f8396425-7b28549-426a787b-20c8e19c_0] Write session: try to update token
2025-10-09T21:46:15.465783Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f8396425-7b28549-426a787b-20c8e19c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:15.478285Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341621317894485:6527]
2025-10-09T21:46:15.478669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341621317894485:6527], now have 1 active actors on pipe
2025-10-09T21:46:15.478746Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 674, pipe: [1:7559341621317894485:6527]
2025-10-09T21:46:15.482877Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:15.482894Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:15.482897Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:15.482899Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:15.482909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341620956080158:6407]
2025-10-09T21:46:15.482914Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341620956080158:6407]
2025-10-09T21:46:15.482917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341620956080158:6407]
2025-10-09T21:46:15.482958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:15.482967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.482970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:15.483120Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:15.486787Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:15.486800Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:15.486804Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:15.486096Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 675 sessionId: src_id|f8396425-7b28549-426a787b-20c8e19c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:15.488993Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:15.774238Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:15.775087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:15.775098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:15.775724Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:15.775788Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341621317894485:6527]
2025-10-09T21:46:15.775820Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:15.775822Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:15.775893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 675 reason: error at write: Disconnected sessionId: src_id|f8396425-7b28549-426a787b-20c8e19c_0
2025-10-09T21:46:15.776073Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 675 sessionId: src_id|f8396425-7b28549-426a787b-20c8e19c_0 is DEAD
2025-10-09T21:46:15.776393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f8396425-7b28549-426a787b-20c8e19c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:15.776401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f8396425-7b28549-426a787b-20c8e19c_0] Write session will restart in 0.005092s
2025-10-09T21:46:15.776424Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f8396425-7b28549-426a787b-20c8e19c_0] Write session: Do CDS request
2025-10-09T21:46:15.776442Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f8396425-7b28549-426a787b-20c8e19c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:15.776692Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:15.782963Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f8396425-7b28549-426a787b-20c8e19c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:15.785886Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:15.785911Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 676
2025-10-09T21:46:15.786206Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 676 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:15.786253Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 676 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:15.786269Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=676 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:15.786273Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:15.786713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 676 sessionId:  describe result for acl check
2025-10-09T21:46:15.786766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:15.786769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:15.786770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:15.786776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341621317894512:7042] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:15.787342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341621317894512:7042] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:15.795297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341621317894512:7042] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:15.808301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341621317894512:7042] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:15.808328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341621317894512:7042] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:15.808332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341621317894512:7042] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:15.808343Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 676 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:15.855622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:15.877870Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341621317894743:6527]
2025-10-09T21:46:15.874820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:15.875078Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:15.875083Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:15.875096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:15.875160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:15.875202Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341620956080439:6413]
2025-10-09T21:46:15.875816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:15.878692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:15.878761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:15.879115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:15.879162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:15.879164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:15.879165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:15.879166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:15.879168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:15.879169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:15.879170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.879172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.879173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.879181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:15.879192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:15.879197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:15.879202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:15.879206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:15.879211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:15.879215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.879220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.879224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.879232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:15.879241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:15.879249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:15.879256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:15.879263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:15.879269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:15.879274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:15.879280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:15.879285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:15.879292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:15.879296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:15.879302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:15.879308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.879313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.879318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.879324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.879328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.879334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.879365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:15.899325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:15.905184Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 675
2025-10-09T21:46:15.905231Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 675, pipe: [1:7559341621317894743:6527]
2025-10-09T21:46:15.905257Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 675
2025-10-09T21:46:15.900510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:15.900527Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:15.900531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:15.900536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:15.900544Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 675 [2:7559341620956080439:6413]
2025-10-09T21:46:15.900562Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:15.900587Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:15.900591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:15.900595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:15.900598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:15.900619Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:15.900625Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:15.900634Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:15.900706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.900709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.900711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.900714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:15.900793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:15.900797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.900811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:15.900825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:15.900844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:15.900866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341620956080429:6413]
2025-10-09T21:46:15.900869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341620956080429:6413]
2025-10-09T21:46:15.900871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341620956080429:6413]
2025-10-09T21:46:15.900891Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:15.900908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:15.900925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:15.900928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:15.900970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341620956080429:6413]
2025-10-09T21:46:15.900973Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:15.901414Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:15.901417Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:15.901420Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:15.901427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:15.901824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341621317894498:2432], now have 1 active actors on pipe
2025-10-09T21:46:15.901829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341621317894743:6527], now have 1 active actors on pipe
2025-10-09T21:46:15.901832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341621317894635:7042], now have 1 active actors on pipe
2025-10-09T21:46:15.906150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:15.906164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:15.906202Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|49a3bb39-e2fbf7c4-ab9aa55-4a118380_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:15.906216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:15.906221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:15.906223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:15.906238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:15.906257Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:15.906277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:15.906614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:15.906621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:15.906636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:15.906809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 676 partition: 0 MaxSeqNo: 25 sessionId: src_id|49a3bb39-e2fbf7c4-ab9aa55-4a118380_0
2025-10-09T21:46:15.907223Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f8396425-7b28549-426a787b-20c8e19c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|49a3bb39-e2fbf7c4-ab9aa55-4a118380_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:15.910151Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|49a3bb39-e2fbf7c4-ab9aa55-4a118380_0] Write session: try to update token
2025-10-09T21:46:15.910169Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|49a3bb39-e2fbf7c4-ab9aa55-4a118380_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:15.973649Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 676 sessionId: src_id|49a3bb39-e2fbf7c4-ab9aa55-4a118380_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:15.976198Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:15.977154Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:15.977170Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:15.977172Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:15.977174Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:15.977183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341620956080429:6413]
2025-10-09T21:46:15.977187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341620956080429:6413]
2025-10-09T21:46:15.977191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341620956080429:6413]
2025-10-09T21:46:15.977238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:15.977244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:15.977247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:15.977405Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:15.982598Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:15.982609Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:15.982614Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:16.770585Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:16.771143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:16.771155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:16.771709Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:16.773495Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:16.773562Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341621317894743:6527]
2025-10-09T21:46:16.773586Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:16.773680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 676 reason: status is not ok: tablet will be restarted right now sessionId: src_id|49a3bb39-e2fbf7c4-ab9aa55-4a118380_0
2025-10-09T21:46:16.773847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 676 sessionId: src_id|49a3bb39-e2fbf7c4-ab9aa55-4a118380_0 is DEAD
2025-10-09T21:46:16.778648Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49a3bb39-e2fbf7c4-ab9aa55-4a118380_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:16.778663Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49a3bb39-e2fbf7c4-ab9aa55-4a118380_0] Write session will restart in 0.006025s
2025-10-09T21:46:16.778691Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49a3bb39-e2fbf7c4-ab9aa55-4a118380_0] Write session: Do CDS request
2025-10-09T21:46:16.778697Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49a3bb39-e2fbf7c4-ab9aa55-4a118380_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:16.786885Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|49a3bb39-e2fbf7c4-ab9aa55-4a118380_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:16.787229Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:16.787251Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 677
2025-10-09T21:46:16.787478Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 677 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:16.787513Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 677 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:16.787529Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=677 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:16.787533Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:16.788106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 677 sessionId:  describe result for acl check
2025-10-09T21:46:16.788138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:16.788140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:16.788142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:16.788149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341625612862113:7057] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:16.788720Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341625612862113:7057] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:16.810132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341625612862113:7057] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:16.814404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:16.820387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341625612862113:7057] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:16.820406Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341625612862113:7057] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:16.820410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341625612862113:7057] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:16.820419Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 677 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:16.828133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:16.828921Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:16.828929Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:16.828947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:16.829038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:16.829084Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341625251048014:6429]
2025-10-09T21:46:16.829734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:16.831328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:16.831392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:16.831867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:16.831937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:16.831938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:16.831939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:16.831940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:16.831942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:16.831943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:16.831945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:16.831946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:16.831947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:16.831954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:16.831965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:16.831970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:16.831974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:16.831978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:16.831982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:16.831986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:16.831990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:16.831994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:16.832002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:16.832010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:16.832017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:16.832022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:16.832027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:16.832032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:16.832037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:16.832042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:16.832050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:16.832055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:16.832060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:16.832065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:16.832073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:16.832078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:16.832082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:16.832088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:16.832093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:16.832099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:16.832127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:16.847238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:16.848285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:16.848291Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:16.848294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:16.848299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:16.848307Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 676 [2:7559341625251048014:6429]
2025-10-09T21:46:16.848321Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:16.848341Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:16.848345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:16.848348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:16.848350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:16.848371Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:16.848377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:16.848385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:16.848446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:16.848448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:16.848450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:16.848453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:16.848528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:16.848531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:16.848544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:16.848553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:16.848567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:16.848587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341625251048007:6429]
2025-10-09T21:46:16.848591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341625251048007:6429]
2025-10-09T21:46:16.848593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341625251048007:6429]
2025-10-09T21:46:16.848609Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:16.848624Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:16.848637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:16.848640Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:16.848684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341625251048007:6429]
2025-10-09T21:46:16.848687Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:16.850667Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 676
2025-10-09T21:46:16.850709Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 676
2025-10-09T21:46:16.849839Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:16.849847Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:16.849849Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:16.849855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:16.851777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341625612862343:7057], now have 1 active actors on pipe
2025-10-09T21:46:16.851789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341625612862098:2432], now have 1 active actors on pipe
2025-10-09T21:46:16.851809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:16.851816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:16.851865Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c5724800-fbed0f99-83991171-feb1848c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:16.851877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:16.851883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:16.851886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:16.851900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:16.851918Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:16.851939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:16.858522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:16.858545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:16.858585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:16.858829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 677 partition: 0 MaxSeqNo: 25 sessionId: src_id|c5724800-fbed0f99-83991171-feb1848c_0
2025-10-09T21:46:16.859286Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49a3bb39-e2fbf7c4-ab9aa55-4a118380_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c5724800-fbed0f99-83991171-feb1848c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:16.861921Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c5724800-fbed0f99-83991171-feb1848c_0] Write session: try to update token
2025-10-09T21:46:16.861942Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c5724800-fbed0f99-83991171-feb1848c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:16.874331Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341625612862354:6527]
2025-10-09T21:46:16.878504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341625612862354:6527], now have 1 active actors on pipe
2025-10-09T21:46:16.878629Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 676, pipe: [1:7559341625612862354:6527]
2025-10-09T21:46:16.900300Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 677 sessionId: src_id|c5724800-fbed0f99-83991171-feb1848c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:16.903309Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:16.903688Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:16.903705Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:16.903707Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:16.903709Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:16.903721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341625251048007:6429]
2025-10-09T21:46:16.903726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341625251048007:6429]
2025-10-09T21:46:16.903730Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341625251048007:6429]
2025-10-09T21:46:16.903775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:16.903782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:16.903785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:16.903930Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:16.906542Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:16.906550Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:16.906554Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:17.217027Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:17.217667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:17.217677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:17.218776Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:17.221409Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:17.221494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 677 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c5724800-fbed0f99-83991171-feb1848c_0
2025-10-09T21:46:17.221653Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 677 sessionId: src_id|c5724800-fbed0f99-83991171-feb1848c_0 is DEAD
2025-10-09T21:46:17.221762Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:17.221799Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341625612862354:6527]
2025-10-09T21:46:17.222192Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5724800-fbed0f99-83991171-feb1848c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:17.222207Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5724800-fbed0f99-83991171-feb1848c_0] Write session will restart in 0.006775s
2025-10-09T21:46:17.222241Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5724800-fbed0f99-83991171-feb1848c_0] Write session: Do CDS request
2025-10-09T21:46:17.222249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5724800-fbed0f99-83991171-feb1848c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:17.230661Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c5724800-fbed0f99-83991171-feb1848c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:17.230926Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:17.230948Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 678
2025-10-09T21:46:17.231196Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 678 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:17.231235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 678 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:17.231241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=678 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:17.231245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:17.231751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 678 sessionId:  describe result for acl check
2025-10-09T21:46:17.231794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:17.231796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:17.231798Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:17.231804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341629907829702:7069] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:17.232436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341629907829702:7069] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:17.237331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341629907829702:7069] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:17.248696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341629907829702:7069] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:17.248732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341629907829702:7069] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:17.248736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341629907829702:7069] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:17.248746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 678 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:17.284467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:17.290287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:17.290562Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:17.290566Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:17.290579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:17.290651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:17.290693Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341629546015598:6443]
2025-10-09T21:46:17.291288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:17.291899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:17.291945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:17.292270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:17.292312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:17.292313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:17.292315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:17.292316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:17.292317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:17.292318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:17.292319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.292321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.292322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.292327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:17.292337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:17.292341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:17.292345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:17.292348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:17.292352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:17.292356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.292360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.292363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.292370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:17.292378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:17.292385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:17.292391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:17.292398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:17.292403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:17.292407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:17.292412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:17.292417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:17.292421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:17.292425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:17.292430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:17.292434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.292440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.292444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.292450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.292454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.292458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.292502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:17.301726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:17.303490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:17.303504Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:17.303507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:17.303512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:17.303519Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 677 [2:7559341629546015598:6443]
2025-10-09T21:46:17.303535Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:17.303559Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:17.303562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:17.303565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:17.303568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:17.303588Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:17.303595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:17.303603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:17.303669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.303671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.303674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.303677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:17.303757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:17.303760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.303776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.303789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.303808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:17.303830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341629546015592:6443]
2025-10-09T21:46:17.303832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341629546015592:6443]
2025-10-09T21:46:17.303835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341629546015592:6443]
2025-10-09T21:46:17.303848Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:17.303863Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:17.303879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:17.303882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:17.303925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341629546015592:6443]
2025-10-09T21:46:17.303928Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:17.304020Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:17.304023Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:17.304025Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:17.304031Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:17.305312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341629907829691:2432], now have 1 active actors on pipe
2025-10-09T21:46:17.305322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341629907829774:7069], now have 1 active actors on pipe
2025-10-09T21:46:17.305343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:17.305349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:17.305380Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5ed23ccb-2a9eb5cb-e0e8e3a6-70c9d449_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:17.305403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:17.305409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:17.305412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:17.305425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:17.305441Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:17.305461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:17.305815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:17.305825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:17.305841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:17.306388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5724800-fbed0f99-83991171-feb1848c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5ed23ccb-2a9eb5cb-e0e8e3a6-70c9d449_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:17.308906Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ed23ccb-2a9eb5cb-e0e8e3a6-70c9d449_0] Write session: try to update token
2025-10-09T21:46:17.308927Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ed23ccb-2a9eb5cb-e0e8e3a6-70c9d449_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:17.304646Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 677
2025-10-09T21:46:17.304713Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 677
2025-10-09T21:46:17.305978Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 678 partition: 0 MaxSeqNo: 25 sessionId: src_id|5ed23ccb-2a9eb5cb-e0e8e3a6-70c9d449_0
2025-10-09T21:46:17.322266Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341629907829947:6527]
2025-10-09T21:46:17.322733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341629907829947:6527], now have 1 active actors on pipe
2025-10-09T21:46:17.322953Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 677, pipe: [1:7559341629907829947:6527]
2025-10-09T21:46:17.325580Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:17.325595Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:17.325598Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:17.325600Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:17.325609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341629546015592:6443]
2025-10-09T21:46:17.325613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341629546015592:6443]
2025-10-09T21:46:17.325616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341629546015592:6443]
2025-10-09T21:46:17.325662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:17.325670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.325672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:17.325811Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:17.325838Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:17.325844Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:17.325848Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:17.348573Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 678 sessionId: src_id|5ed23ccb-2a9eb5cb-e0e8e3a6-70c9d449_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:17.351217Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:17.515245Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:17.515963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:17.515975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:17.516856Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341629907829947:6527]
2025-10-09T21:46:17.516892Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:17.516935Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:17.517020Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 678 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5ed23ccb-2a9eb5cb-e0e8e3a6-70c9d449_0
2025-10-09T21:46:17.517199Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 678 sessionId: src_id|5ed23ccb-2a9eb5cb-e0e8e3a6-70c9d449_0 is DEAD
2025-10-09T21:46:17.517543Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ed23ccb-2a9eb5cb-e0e8e3a6-70c9d449_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:17.517551Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ed23ccb-2a9eb5cb-e0e8e3a6-70c9d449_0] Write session will restart in 0.009284s
2025-10-09T21:46:17.517576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ed23ccb-2a9eb5cb-e0e8e3a6-70c9d449_0] Write session: Do CDS request
2025-10-09T21:46:17.517582Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ed23ccb-2a9eb5cb-e0e8e3a6-70c9d449_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:17.516343Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:17.527616Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ed23ccb-2a9eb5cb-e0e8e3a6-70c9d449_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:17.527765Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:17.527783Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 679
2025-10-09T21:46:17.528484Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 679 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:17.528537Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 679 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:17.528543Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=679 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:17.528546Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:17.529302Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 679 sessionId:  describe result for acl check
2025-10-09T21:46:17.529344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:17.529346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:17.529348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:17.529354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341629907829966:7073] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:17.529886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341629907829966:7073] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:17.534635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341629907829966:7073] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:17.540373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341629907829966:7073] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:17.540393Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341629907829966:7073] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:17.540398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341629907829966:7073] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:17.540409Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 679 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:17.587238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:17.587447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:17.588277Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:17.588290Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:17.588308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:17.588387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:17.588432Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341629546015853:6447]
2025-10-09T21:46:17.589008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:17.591279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:17.591353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:17.591708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:17.591782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:17.591783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:17.591784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:17.591785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:17.591786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:17.591787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:17.591788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.591790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.591792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.591798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:17.591808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:17.591812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:17.591815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:17.591818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:17.591823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:17.591826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.591829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.591833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.591841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:17.591850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:17.591856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:17.591862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:17.591867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:17.591872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:17.591877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:17.591881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:17.591886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:17.591892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:17.591897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:17.591903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:17.591908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.591913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.591918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.591923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.591929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.591934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.591965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:17.610420Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 678
2025-10-09T21:46:17.610463Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 678
2025-10-09T21:46:17.607549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:17.608661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:17.608668Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:17.608672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:17.608677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:17.608685Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 678 [2:7559341629546015853:6447]
2025-10-09T21:46:17.608704Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:17.608728Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:17.608732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:17.608736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:17.608738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:17.608759Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:17.608766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:17.608775Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:17.608847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.608850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.608852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.608856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:17.608937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:17.608941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.608958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.608970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.608987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:17.609007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341629546015848:6447]
2025-10-09T21:46:17.609010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341629546015848:6447]
2025-10-09T21:46:17.609013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341629546015848:6447]
2025-10-09T21:46:17.609031Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:17.609047Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:17.609061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:17.609065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:17.609109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341629546015848:6447]
2025-10-09T21:46:17.609112Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:17.609296Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:17.609298Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:17.609302Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:17.609309Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:17.611548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341629907829952:2432], now have 1 active actors on pipe
2025-10-09T21:46:17.611562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341629907830027:7073], now have 1 active actors on pipe
2025-10-09T21:46:17.611600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:17.611608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:17.611649Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|822c3dad-2c176035-e71eec38-417595f0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:17.611672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:17.611678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:17.611681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:17.611696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:17.611715Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:17.611735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:17.616349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:17.616369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:17.616428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:17.616911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 679 partition: 0 MaxSeqNo: 25 sessionId: src_id|822c3dad-2c176035-e71eec38-417595f0_0
2025-10-09T21:46:17.617147Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341629907830209:6527]
2025-10-09T21:46:17.617326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ed23ccb-2a9eb5cb-e0e8e3a6-70c9d449_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|822c3dad-2c176035-e71eec38-417595f0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:17.617650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341629907830209:6527], now have 1 active actors on pipe
2025-10-09T21:46:17.617759Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 678, pipe: [1:7559341629907830209:6527]
2025-10-09T21:46:17.620096Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|822c3dad-2c176035-e71eec38-417595f0_0] Write session: try to update token
2025-10-09T21:46:17.620117Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|822c3dad-2c176035-e71eec38-417595f0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:17.649401Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 679 sessionId: src_id|822c3dad-2c176035-e71eec38-417595f0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:17.651994Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:17.652325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:17.652347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:17.653575Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:17.653580Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:17.653583Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:17.653585Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:17.653594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341629546015848:6447]
2025-10-09T21:46:17.653598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341629546015848:6447]
2025-10-09T21:46:17.653602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341629546015848:6447]
2025-10-09T21:46:17.653654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:17.653657Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:17.653668Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:17.653672Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:17.653675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:17.653683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.653685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:17.653825Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:17.655422Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:17.837898Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:17.847734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:17.847762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:17.847943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:17.847998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:17.848059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:17.848115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:17.848154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:17.848160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:17.848228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:17.848283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:17.848336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:17.848398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:17.848441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:17.848448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:17.848516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:17.848570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:17.848636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:17.848695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:17.848736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:17.848740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:17.848813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:17.848874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:17.848936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:17.848997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:17.849038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:17.849041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:17.849109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:17.849166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:17.849229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:17.849290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:17.849356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:17.849360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:17.849421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:17.849476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:17.849534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:17.849590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:17.849629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:17.849632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:17.849694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:17.849748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:17.849807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:17.849865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:17.849905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:17.849907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:17.849983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:17.850447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:17.850513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:17.850573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:17.850614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:17.850617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:17.850687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:17.850744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:17.850804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:17.850863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:17.850907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:17.850910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:17.850983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:17.851039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:17.851095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:17.851172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:17.851218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:17.851220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:17.852878Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:17.852896Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:17.852946Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 679 reason: error at write: Disconnected sessionId: src_id|822c3dad-2c176035-e71eec38-417595f0_0
2025-10-09T21:46:17.853147Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 679 sessionId: src_id|822c3dad-2c176035-e71eec38-417595f0_0 is DEAD
2025-10-09T21:46:17.853166Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:17.853209Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341629907830209:6527]
2025-10-09T21:46:17.853350Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|822c3dad-2c176035-e71eec38-417595f0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:17.853363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|822c3dad-2c176035-e71eec38-417595f0_0] Write session will restart in 0.008900s
2025-10-09T21:46:17.853388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|822c3dad-2c176035-e71eec38-417595f0_0] Write session: Do CDS request
2025-10-09T21:46:17.853395Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|822c3dad-2c176035-e71eec38-417595f0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:17.853087Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:17.863652Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|822c3dad-2c176035-e71eec38-417595f0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:17.863779Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:17.863799Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 680
2025-10-09T21:46:17.864385Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 680 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:17.864439Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 680 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:17.864451Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=680 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:17.864458Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:17.865589Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 680 sessionId:  describe result for acl check
2025-10-09T21:46:17.865639Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:17.865641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:17.865643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:17.865648Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341629907830225:7078] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:17.866310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341629907830225:7078] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:17.870846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341629907830225:7078] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:17.885676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341629907830225:7078] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:17.885700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341629907830225:7078] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:17.885706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341629907830225:7078] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:17.885717Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 680 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:17.903898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:17.911486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:17.911767Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:17.911771Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:17.911784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:17.911846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:17.911888Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341629546016105:6449]
2025-10-09T21:46:17.912482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:17.912918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:17.912971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:17.913633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:17.913718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:17.913719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:17.913721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:17.913722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:17.913723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:17.913724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:17.913725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.913726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.913728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.913735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:17.913747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:17.913751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:17.913755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:17.913760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:17.913764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:17.913768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.913773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.913776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.913785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:17.913793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:17.913800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:17.913807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:17.913812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:17.913818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:17.913822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:17.913827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:17.913832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:17.913838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:17.913842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:17.913847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:17.913852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.913856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.913861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.913865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.913870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.913875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.913905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:17.922290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:17.923336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:17.923345Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:17.923347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:17.923352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:17.923361Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 679 [2:7559341629546016105:6449]
2025-10-09T21:46:17.923377Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:17.923398Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:17.923402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:17.923406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:17.923408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:17.923428Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:17.923436Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:17.923444Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:17.923513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.923515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.923517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.923520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:17.923603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:17.923606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.923618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:17.923630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:17.923649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:17.923669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341629546016100:6449]
2025-10-09T21:46:17.923672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341629546016100:6449]
2025-10-09T21:46:17.923674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341629546016100:6449]
2025-10-09T21:46:17.923690Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:17.923705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:17.923720Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:17.923723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:17.923766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341629546016100:6449]
2025-10-09T21:46:17.923770Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:17.923773Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:17.923778Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:17.923781Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:17.923788Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:17.924430Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 679
2025-10-09T21:46:17.924503Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 679
2025-10-09T21:46:17.929498Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|822c3dad-2c176035-e71eec38-417595f0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e95e85b3-3a27690e-7287fd46-5d1cddf9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:17.928889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 680 partition: 0 MaxSeqNo: 25 sessionId: src_id|e95e85b3-3a27690e-7287fd46-5d1cddf9_0
2025-10-09T21:46:17.926425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341629907830216:2432], now have 1 active actors on pipe
2025-10-09T21:46:17.926448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341629907830335:7078], now have 1 active actors on pipe
2025-10-09T21:46:17.926469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:17.926476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:17.926516Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e95e85b3-3a27690e-7287fd46-5d1cddf9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:17.926528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:17.926542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:17.926545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:17.926558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:17.926575Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:17.926595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:17.927356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:17.927362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:17.927382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:17.932168Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e95e85b3-3a27690e-7287fd46-5d1cddf9_0] Write session: try to update token
2025-10-09T21:46:17.932187Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e95e85b3-3a27690e-7287fd46-5d1cddf9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:17.945991Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:17.946011Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:17.946014Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:17.946016Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:17.946026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341629546016100:6449]
2025-10-09T21:46:17.946075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341629546016100:6449]
2025-10-09T21:46:17.946080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341629546016100:6449]
2025-10-09T21:46:17.946130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:17.946138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:17.946141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:17.946300Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:17.949751Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:17.949766Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:17.949770Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:17.952120Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 680 sessionId: src_id|e95e85b3-3a27690e-7287fd46-5d1cddf9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:17.954020Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341629907830477:6527]
2025-10-09T21:46:17.954788Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:17.956165Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 679, pipe: [1:7559341629907830477:6527]
2025-10-09T21:46:18.176106Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:18.177236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:18.177247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:18.177274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341629907830477:6527], now have 1 active actors on pipe
2025-10-09T21:46:18.177775Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341629907830477:6527]
2025-10-09T21:46:18.177808Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:18.177855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:18.177862Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:18.177928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 680 reason: error at write: Disconnected sessionId: src_id|e95e85b3-3a27690e-7287fd46-5d1cddf9_0
2025-10-09T21:46:18.178151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 680 sessionId: src_id|e95e85b3-3a27690e-7287fd46-5d1cddf9_0 is DEAD
2025-10-09T21:46:18.178710Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e95e85b3-3a27690e-7287fd46-5d1cddf9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:18.178722Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e95e85b3-3a27690e-7287fd46-5d1cddf9_0] Write session will restart in 0.005909s
2025-10-09T21:46:18.178747Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e95e85b3-3a27690e-7287fd46-5d1cddf9_0] Write session: Do CDS request
2025-10-09T21:46:18.178758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e95e85b3-3a27690e-7287fd46-5d1cddf9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:18.186426Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:18.189148Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e95e85b3-3a27690e-7287fd46-5d1cddf9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:18.193814Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:18.193839Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 681
2025-10-09T21:46:18.194154Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 681 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:18.194194Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 681 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:18.194203Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=681 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:18.194207Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:18.194991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 681 sessionId:  describe result for acl check
2025-10-09T21:46:18.195031Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:18.195033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:18.195035Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:18.195042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341634202797819:7090] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:18.195573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341634202797819:7090] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:18.200455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341634202797819:7090] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:18.211472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341634202797819:7090] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:18.211512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341634202797819:7090] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:18.211516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341634202797819:7090] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:18.211526Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 681 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:18.254525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:18.259422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:18.259670Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:18.259675Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:18.259689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:18.259752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:18.259792Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341633840983692:6463]
2025-10-09T21:46:18.260378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:18.260854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:18.260905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:18.261241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:18.261294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:18.261296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:18.261297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:18.261298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:18.261299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:18.261301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:18.261302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.261304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:18.261305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:18.261311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:18.261322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:18.261327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:18.261331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:18.261335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:18.261340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:18.261344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.261348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:18.261352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:18.261361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:18.261370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:18.261376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:18.261386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:18.261391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:18.261396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:18.261403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:18.261408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:18.261421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:18.261426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:18.261431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:18.261436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:18.261442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.261447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.261451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:18.261456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:18.261462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:18.261467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:18.261500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:18.275239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:18.276754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:18.276761Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:18.276765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:18.276769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:18.276778Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 680 [2:7559341633840983692:6463]
2025-10-09T21:46:18.276795Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:18.276818Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:18.276822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:18.276825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:18.276827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:18.276847Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:18.276854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:18.276861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:18.276937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.276939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:18.276941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:18.276944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:18.277023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:18.277027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.277041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:18.277053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:18.277072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:18.277094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341633840983687:6463]
2025-10-09T21:46:18.277097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341633840983687:6463]
2025-10-09T21:46:18.277100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341633840983687:6463]
2025-10-09T21:46:18.277116Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:18.277133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:18.277149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:18.277152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:18.277197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341633840983687:6463]
2025-10-09T21:46:18.277200Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:18.277340Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341634202798060:6527]
2025-10-09T21:46:18.278411Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 680
2025-10-09T21:46:18.278465Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 680
2025-10-09T21:46:18.277503Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:18.277510Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:18.277512Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:18.277518Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:18.279407Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 680, pipe: [1:7559341634202798060:6527]
2025-10-09T21:46:18.279367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341634202797808:2432], now have 1 active actors on pipe
2025-10-09T21:46:18.279376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341634202797864:7090], now have 1 active actors on pipe
2025-10-09T21:46:18.279401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:18.279406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:18.279420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341634202798060:6527], now have 1 active actors on pipe
2025-10-09T21:46:18.279442Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b50db59d-bb08feb8-44ece800-9ce4af4d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:18.279453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:18.279458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:18.279472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:18.279487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:18.279502Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:18.279521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:18.284648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:18.284667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:18.284709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:18.284974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 681 partition: 0 MaxSeqNo: 25 sessionId: src_id|b50db59d-bb08feb8-44ece800-9ce4af4d_0
2025-10-09T21:46:18.285489Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e95e85b3-3a27690e-7287fd46-5d1cddf9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b50db59d-bb08feb8-44ece800-9ce4af4d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:18.288251Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b50db59d-bb08feb8-44ece800-9ce4af4d_0] Write session: try to update token
2025-10-09T21:46:18.288272Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b50db59d-bb08feb8-44ece800-9ce4af4d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:18.325586Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:18.325601Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:18.325602Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:18.325604Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:18.325612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341633840983687:6463]
2025-10-09T21:46:18.325616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341633840983687:6463]
2025-10-09T21:46:18.325620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341633840983687:6463]
2025-10-09T21:46:18.325663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:18.325670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.325673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:18.325829Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:18.333084Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:18.333098Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:18.333102Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:18.336624Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 681 sessionId: src_id|b50db59d-bb08feb8-44ece800-9ce4af4d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:18.339142Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:18.580594Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:18.581587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:18.581600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:18.583266Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:18.583318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341634202798060:6527]
2025-10-09T21:46:18.583340Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:18.583343Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:18.583402Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 681 reason: error at write: Disconnected sessionId: src_id|b50db59d-bb08feb8-44ece800-9ce4af4d_0
2025-10-09T21:46:18.583591Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 681 sessionId: src_id|b50db59d-bb08feb8-44ece800-9ce4af4d_0 is DEAD
2025-10-09T21:46:18.583989Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b50db59d-bb08feb8-44ece800-9ce4af4d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:18.583997Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b50db59d-bb08feb8-44ece800-9ce4af4d_0] Write session will restart in 0.008610s
2025-10-09T21:46:18.584021Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b50db59d-bb08feb8-44ece800-9ce4af4d_0] Write session: Do CDS request
2025-10-09T21:46:18.584027Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b50db59d-bb08feb8-44ece800-9ce4af4d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:18.583472Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:18.594718Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b50db59d-bb08feb8-44ece800-9ce4af4d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:18.594811Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:18.594833Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 682
2025-10-09T21:46:18.595119Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 682 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:18.595156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 682 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:18.595163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=682 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:18.595167Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:18.595948Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 682 sessionId:  describe result for acl check
2025-10-09T21:46:18.595987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:18.595989Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:18.595991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:18.595997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341634202798079:7094] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:18.596589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341634202798079:7094] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:18.616306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341634202798079:7094] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:18.627190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341634202798079:7094] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:18.627212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341634202798079:7094] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:18.627216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341634202798079:7094] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:18.627226Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 682 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:18.677722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:18.681266Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341634202798323:6527]
2025-10-09T21:46:18.682905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:18.683169Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:18.683174Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:18.683190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:18.683257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:18.683301Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341633840983950:6467]
2025-10-09T21:46:18.683851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:18.684207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:18.684254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:18.684539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:18.684575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:18.684576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:18.684576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:18.684577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:18.684578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:18.684579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:18.684580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.684580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:18.684581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:18.684585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:18.684592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:18.684595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:18.684598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:18.684600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:18.684602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:18.684605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.684607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:18.684609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:18.684614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:18.684619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:18.684624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:18.684628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:18.684631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:18.684634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:18.684637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:18.684640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:18.684643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:18.684646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:18.684649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:18.684652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:18.684655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.684659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.684662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:18.684665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:18.684668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:18.684671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:18.684689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:18.697845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:18.699055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:18.699062Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:18.699064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:18.699068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:18.699075Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 681 [2:7559341633840983950:6467]
2025-10-09T21:46:18.699086Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:18.699107Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:18.699109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:18.699112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:18.699114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:18.699133Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:18.699138Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:18.699144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:18.699201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.699202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:18.699203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:18.699206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:18.699290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:18.699293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.699322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:18.699333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:18.699347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:18.699369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341633840983945:6467]
2025-10-09T21:46:18.699371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341633840983945:6467]
2025-10-09T21:46:18.699373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341633840983945:6467]
2025-10-09T21:46:18.699381Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:18.699400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:18.699405Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:18.699408Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:18.699410Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:18.699414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:18.699418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:18.699468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341634202798323:6527], now have 1 active actors on pipe
2025-10-09T21:46:18.699471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341634202798119:7094], now have 1 active actors on pipe
2025-10-09T21:46:18.699476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341634202798071:2432], now have 1 active actors on pipe
2025-10-09T21:46:18.699478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341633840983945:6467]
2025-10-09T21:46:18.699485Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:18.699806Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 681, pipe: [1:7559341634202798323:6527]
2025-10-09T21:46:18.699852Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 681
2025-10-09T21:46:18.699873Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 681
2025-10-09T21:46:18.700061Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:18.700489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:18.700502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:18.700539Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|46a2e8d1-645f8b53-a1398db-43aee4b4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:18.700550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:18.700556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:18.700559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:18.700573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:18.700588Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:18.700609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:18.701601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:18.701612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:18.701636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:18.704513Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b50db59d-bb08feb8-44ece800-9ce4af4d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|46a2e8d1-645f8b53-a1398db-43aee4b4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:18.704048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 682 partition: 0 MaxSeqNo: 25 sessionId: src_id|46a2e8d1-645f8b53-a1398db-43aee4b4_0
2025-10-09T21:46:18.707594Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46a2e8d1-645f8b53-a1398db-43aee4b4_0] Write session: try to update token
2025-10-09T21:46:18.707620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46a2e8d1-645f8b53-a1398db-43aee4b4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:18.728836Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:18.728853Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:18.728856Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:18.728858Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:18.728868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341633840983945:6467]
2025-10-09T21:46:18.728874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341633840983945:6467]
2025-10-09T21:46:18.728879Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341633840983945:6467]
2025-10-09T21:46:18.728926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:18.728935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.728937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:18.729084Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:18.730185Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:18.730191Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:18.730195Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:18.740626Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 682 sessionId: src_id|46a2e8d1-645f8b53-a1398db-43aee4b4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:18.743469Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:18.927697Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:18.928858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:18.928876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:18.929146Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:18.930840Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46a2e8d1-645f8b53-a1398db-43aee4b4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:18.930874Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46a2e8d1-645f8b53-a1398db-43aee4b4_0] Write session will restart in 0.008518s
2025-10-09T21:46:18.930901Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46a2e8d1-645f8b53-a1398db-43aee4b4_0] Write session: Do CDS request
2025-10-09T21:46:18.930915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46a2e8d1-645f8b53-a1398db-43aee4b4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:18.930105Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341634202798323:6527]
2025-10-09T21:46:18.930144Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:18.930217Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 682 reason: status is not ok: tablet will be restarted right now sessionId: src_id|46a2e8d1-645f8b53-a1398db-43aee4b4_0
2025-10-09T21:46:18.930402Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 682 sessionId: src_id|46a2e8d1-645f8b53-a1398db-43aee4b4_0 is DEAD
2025-10-09T21:46:18.930523Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:18.942164Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46a2e8d1-645f8b53-a1398db-43aee4b4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:18.942588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:18.942621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 683
2025-10-09T21:46:18.942816Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 683 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:18.942880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 683 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:18.942886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=683 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:18.942890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:18.944481Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 683 sessionId:  describe result for acl check
2025-10-09T21:46:18.944529Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:18.944531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:18.944533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:18.944539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341634202798348:7099] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:18.945115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341634202798348:7099] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:18.948493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341634202798348:7099] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:18.961175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341634202798348:7099] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:18.961197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341634202798348:7099] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:18.961201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341634202798348:7099] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:18.961211Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 683 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:18.984417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:18.989889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:18.990183Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:18.990189Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:18.990205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:18.990290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:18.990383Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341633840984205:6469]
2025-10-09T21:46:18.990986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:18.991670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:18.991756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:18.992142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:18.992217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:18.992218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:18.992219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:18.992220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:18.992222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:18.992223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:18.992224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.992226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:18.992227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:18.992233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:18.992246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:18.992250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:18.992253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:18.992256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:18.992260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:18.992264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.992267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:18.992270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:18.992279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:18.992288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:18.992306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:18.992312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:18.992317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:18.992322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:18.992328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:18.992334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:18.992338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:18.992343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:18.992347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:18.992353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:18.992359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.992364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:18.992369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:18.992374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:18.992379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:18.992385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:18.992429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:19.004265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:19.006326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:19.006345Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:19.006347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:19.006352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:19.006361Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 682 [2:7559341633840984205:6469]
2025-10-09T21:46:19.006378Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:19.006401Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:19.006405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:19.006409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:19.006411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:19.006432Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:19.006439Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:19.006447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:19.006529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.006540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:19.006542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:19.006546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:19.006632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:19.006635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.006652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:19.006663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:19.006679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:19.006703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341633840984199:6469]
2025-10-09T21:46:19.006705Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341633840984199:6469]
2025-10-09T21:46:19.006708Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341633840984199:6469]
2025-10-09T21:46:19.006718Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:19.006735Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:19.006752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:19.006755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:19.006772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:19.006779Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:19.006781Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:19.006802Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341633840984199:6469]
2025-10-09T21:46:19.006806Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:19.006810Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:19.006817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341634202798407:7099], now have 1 active actors on pipe
2025-10-09T21:46:19.006823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341634202798338:2432], now have 1 active actors on pipe
2025-10-09T21:46:19.007576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:19.007590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:19.007633Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7846bfb8-9a9c04b6-4cccdaff-1258beca_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:19.007643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:19.007647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:19.007651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:19.007663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:19.007677Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:19.007703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:19.008147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:19.008157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:19.008178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:19.007010Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 682
2025-10-09T21:46:19.007013Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 682
2025-10-09T21:46:19.011446Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46a2e8d1-645f8b53-a1398db-43aee4b4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7846bfb8-9a9c04b6-4cccdaff-1258beca_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:19.014455Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7846bfb8-9a9c04b6-4cccdaff-1258beca_0] Write session: try to update token
2025-10-09T21:46:19.014478Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7846bfb8-9a9c04b6-4cccdaff-1258beca_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:19.009865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 683 partition: 0 MaxSeqNo: 25 sessionId: src_id|7846bfb8-9a9c04b6-4cccdaff-1258beca_0
2025-10-09T21:46:19.029114Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:19.029132Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:19.029135Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:19.029137Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:19.029147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341633840984199:6469]
2025-10-09T21:46:19.029152Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341633840984199:6469]
2025-10-09T21:46:19.029156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341633840984199:6469]
2025-10-09T21:46:19.029204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:19.029212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.029215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:19.029361Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:19.033766Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:19.033781Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:19.033786Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:19.035307Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341638497765903:6527]
2025-10-09T21:46:19.035776Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 682, pipe: [1:7559341638497765903:6527]
2025-10-09T21:46:19.081505Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 683 sessionId: src_id|7846bfb8-9a9c04b6-4cccdaff-1258beca_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:19.084206Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:19.219506Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:19.220118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341638497765903:6527], now have 1 active actors on pipe
2025-10-09T21:46:19.220190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:19.220199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:19.220592Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:19.222918Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341638497765903:6527]
2025-10-09T21:46:19.222959Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:19.223035Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 683 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7846bfb8-9a9c04b6-4cccdaff-1258beca_0
2025-10-09T21:46:19.223248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 683 sessionId: src_id|7846bfb8-9a9c04b6-4cccdaff-1258beca_0 is DEAD
2025-10-09T21:46:19.223415Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:19.223698Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7846bfb8-9a9c04b6-4cccdaff-1258beca_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:19.223711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7846bfb8-9a9c04b6-4cccdaff-1258beca_0] Write session will restart in 0.005108s
2025-10-09T21:46:19.223738Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7846bfb8-9a9c04b6-4cccdaff-1258beca_0] Write session: Do CDS request
2025-10-09T21:46:19.223746Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7846bfb8-9a9c04b6-4cccdaff-1258beca_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:19.229616Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7846bfb8-9a9c04b6-4cccdaff-1258beca_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:19.229787Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:19.229815Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 684
2025-10-09T21:46:19.230107Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 684 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:19.230146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 684 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:19.230151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=684 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:19.230156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:19.230772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 684 sessionId:  describe result for acl check
2025-10-09T21:46:19.230817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:19.230818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:19.230820Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:19.230827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341638497765932:7110] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:19.231609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341638497765932:7110] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:19.236557Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341638497765932:7110] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:19.243474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341638497765932:7110] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:19.243498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341638497765932:7110] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:19.243503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341638497765932:7110] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:19.243513Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 684 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:19.278972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:19.284203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:19.284501Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:19.284512Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:19.284527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:19.284590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:19.284628Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341638135951787:6477]
2025-10-09T21:46:19.285184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:19.285743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:19.285786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:19.286138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:19.286198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:19.286200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:19.286201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:19.286202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:19.286204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:19.286206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:19.286207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.286209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:19.286210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:19.286217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:19.286228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:19.286231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:19.286235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:19.286238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:19.286242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:19.286246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.286250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:19.286256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:19.286263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:19.286271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:19.286277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:19.286283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:19.286291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:19.286296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:19.286300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:19.286304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:19.286308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:19.286312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:19.286315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:19.286320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:19.286325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.286331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.286335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:19.286340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:19.286347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:19.286352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:19.286381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:19.300237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:19.301792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:19.301799Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:19.301801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:19.301806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:19.301814Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 683 [2:7559341638135951787:6477]
2025-10-09T21:46:19.301829Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:19.301850Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:19.301854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:19.301858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:19.301860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:19.301880Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:19.301886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:19.301894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:19.301964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.301967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:19.301970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:19.301973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:19.302090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:19.302093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.302111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:19.302122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:19.302138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:19.302160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341638135951781:6477]
2025-10-09T21:46:19.302163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341638135951781:6477]
2025-10-09T21:46:19.302165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341638135951781:6477]
2025-10-09T21:46:19.302182Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:19.302209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:19.302224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:19.302227Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:19.302274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341638135951781:6477]
2025-10-09T21:46:19.302276Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:19.303412Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:19.303422Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:19.303424Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:19.303431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341638497765972:7110], now have 1 active actors on pipe
2025-10-09T21:46:19.303434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:19.303438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341638497765926:2432], now have 1 active actors on pipe
2025-10-09T21:46:19.303667Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 683
2025-10-09T21:46:19.303707Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 683
2025-10-09T21:46:19.304047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:19.304060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:19.304100Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e1ace49b-9da18601-77518f53-6f95f756_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:19.304112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:19.304117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:19.304120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:19.304134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:19.304161Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:19.304183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:19.304698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:19.304707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:19.304726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:19.304942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 684 partition: 0 MaxSeqNo: 25 sessionId: src_id|e1ace49b-9da18601-77518f53-6f95f756_0
2025-10-09T21:46:19.305409Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7846bfb8-9a9c04b6-4cccdaff-1258beca_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e1ace49b-9da18601-77518f53-6f95f756_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:19.308086Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ace49b-9da18601-77518f53-6f95f756_0] Write session: try to update token
2025-10-09T21:46:19.308105Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ace49b-9da18601-77518f53-6f95f756_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:19.324013Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341638497766195:6527]
2025-10-09T21:46:19.324568Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 683, pipe: [1:7559341638497766195:6527]
2025-10-09T21:46:19.324661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341638497766195:6527], now have 1 active actors on pipe
2025-10-09T21:46:19.343852Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 684 sessionId: src_id|e1ace49b-9da18601-77518f53-6f95f756_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:19.350974Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:19.354360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:19.354385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:19.354428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:19.355880Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:19.357717Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:19.357738Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:19.357740Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:19.357743Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:19.357753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341638135951781:6477]
2025-10-09T21:46:19.357758Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341638135951781:6477]
2025-10-09T21:46:19.357762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341638135951781:6477]
2025-10-09T21:46:19.357813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:19.357821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.357824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:19.357848Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:19.357854Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:19.357858Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:19.357997Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:19.590354Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:19.603645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:19.603673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:19.603896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:19.603966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:19.604048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:19.604117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:19.604677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:19.604684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:19.604757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:19.604819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:19.604883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:19.604949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:19.604991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:19.604994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:19.605063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:19.605129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:19.605213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:19.605287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:19.605332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:19.605336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:19.605410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:19.605477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:19.605544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:19.605611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:19.605661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:19.605664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:19.605737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:19.605802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:19.605868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:19.605940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:19.605981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:19.605984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:19.606088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:19.606148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:19.607103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:19.608453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:19.608495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:19.608498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:19.608569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:19.608633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:19.608699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:19.608762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:19.608805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:19.608808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:19.608879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:19.608936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:19.609002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:19.609069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:19.609128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:19.609131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:19.609206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:19.610328Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:19.610349Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:19.610383Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 684 reason: error at write: Disconnected sessionId: src_id|e1ace49b-9da18601-77518f53-6f95f756_0
2025-10-09T21:46:19.610570Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 684 sessionId: src_id|e1ace49b-9da18601-77518f53-6f95f756_0 is DEAD
2025-10-09T21:46:19.610612Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:19.610649Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341638497766195:6527]
2025-10-09T21:46:19.610933Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ace49b-9da18601-77518f53-6f95f756_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:19.610943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ace49b-9da18601-77518f53-6f95f756_0] Write session will restart in 0.008171s
2025-10-09T21:46:19.610967Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ace49b-9da18601-77518f53-6f95f756_0] Write session: Do CDS request
2025-10-09T21:46:19.610973Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ace49b-9da18601-77518f53-6f95f756_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:19.609296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:19.609373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:19.609444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:19.609488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:19.609494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:19.609570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:19.609632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:19.609699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:19.609763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:19.609806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:19.609809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:19.610789Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:19.620613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ace49b-9da18601-77518f53-6f95f756_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:19.621965Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:19.621985Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 685
2025-10-09T21:46:19.622220Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 685 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:19.622255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 685 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:19.622262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=685 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:19.622266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:19.623290Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 685 sessionId:  describe result for acl check
2025-10-09T21:46:19.623329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:19.623331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:19.623333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:19.623338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341638497766212:7115] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:19.625286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341638497766212:7115] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:19.628365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341638497766212:7115] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:19.639655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341638497766212:7115] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:19.639675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341638497766212:7115] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:19.639678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341638497766212:7115] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:19.639686Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 685 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:19.677181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:19.677385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:19.677570Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:19.677574Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:19.677602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:19.678403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:19.678459Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341638135952056:6487]
2025-10-09T21:46:19.679076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:19.679657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:19.679701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:19.680021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:19.680074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:19.680076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:19.680077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:19.680078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:19.680079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:19.680081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:19.680082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.680083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:19.680085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:19.680091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:19.680104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:19.680108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:19.680113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:19.680117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:19.680121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:19.680125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.680128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:19.680132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:19.680139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:19.680147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:19.680153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:19.680159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:19.680168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:19.680174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:19.680180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:19.680186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:19.680192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:19.680210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:19.680216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:19.680222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:19.680228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.680234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.680238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:19.680245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:19.680251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:19.680256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:19.680287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:19.698804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:19.699860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:19.699877Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:19.699880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:19.699885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:19.699892Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 684 [2:7559341638135952056:6487]
2025-10-09T21:46:19.699907Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:19.699929Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:19.699932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:19.699936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:19.699939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:19.699957Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:19.699963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:19.699971Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:19.700041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.700044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:19.700046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:19.700049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:19.700784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:19.700788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.700803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:19.700814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:19.700831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:19.700855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341638135952051:6487]
2025-10-09T21:46:19.700858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341638135952051:6487]
2025-10-09T21:46:19.700859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341638135952051:6487]
2025-10-09T21:46:19.700876Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:19.700893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:19.700906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:19.700909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:19.700951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341638135952051:6487]
2025-10-09T21:46:19.700953Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:19.701045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:19.701047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:19.701049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:19.701054Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:19.701336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341638497766201:2432], now have 1 active actors on pipe
2025-10-09T21:46:19.701339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341638497766293:7115], now have 1 active actors on pipe
2025-10-09T21:46:19.701820Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 684
2025-10-09T21:46:19.701868Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 684
2025-10-09T21:46:19.705790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:19.705804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:19.705839Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bf98ffe9-ee404538-db27d9e2-b409252c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:19.705851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:19.705856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:19.705859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:19.705875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:19.705892Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:19.705912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:19.706315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:19.706323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:19.706341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:19.706515Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 685 partition: 0 MaxSeqNo: 25 sessionId: src_id|bf98ffe9-ee404538-db27d9e2-b409252c_0
2025-10-09T21:46:19.708432Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341638497766465:6527]
2025-10-09T21:46:19.709508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ace49b-9da18601-77518f53-6f95f756_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bf98ffe9-ee404538-db27d9e2-b409252c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:19.709659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341638497766465:6527], now have 1 active actors on pipe
2025-10-09T21:46:19.709729Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 684, pipe: [1:7559341638497766465:6527]
2025-10-09T21:46:19.712319Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf98ffe9-ee404538-db27d9e2-b409252c_0] Write session: try to update token
2025-10-09T21:46:19.712342Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf98ffe9-ee404538-db27d9e2-b409252c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:19.721312Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:19.721329Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:19.721332Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:19.721334Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:19.721345Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341638135952051:6487]
2025-10-09T21:46:19.721351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341638135952051:6487]
2025-10-09T21:46:19.721355Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341638135952051:6487]
2025-10-09T21:46:19.721404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:19.721412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:19.721415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:19.721561Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:19.726643Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:19.726652Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:19.726655Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:19.781446Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 685 sessionId: src_id|bf98ffe9-ee404538-db27d9e2-b409252c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:19.784255Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:20.013318Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:20.013969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:20.013988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:20.015448Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341638497766465:6527]
2025-10-09T21:46:20.015496Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:20.015546Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:20.015549Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:20.015631Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 685 reason: error at write: Disconnected sessionId: src_id|bf98ffe9-ee404538-db27d9e2-b409252c_0
2025-10-09T21:46:20.015849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 685 sessionId: src_id|bf98ffe9-ee404538-db27d9e2-b409252c_0 is DEAD
2025-10-09T21:46:20.018382Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf98ffe9-ee404538-db27d9e2-b409252c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:20.018396Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf98ffe9-ee404538-db27d9e2-b409252c_0] Write session will restart in 0.008556s
2025-10-09T21:46:20.018436Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf98ffe9-ee404538-db27d9e2-b409252c_0] Write session: Do CDS request
2025-10-09T21:46:20.018443Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf98ffe9-ee404538-db27d9e2-b409252c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:20.017676Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:20.030855Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf98ffe9-ee404538-db27d9e2-b409252c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:20.034603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:20.034626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 686
2025-10-09T21:46:20.034878Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 686 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:20.034941Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 686 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:20.034948Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=686 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:20.034952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:20.037367Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 686 sessionId:  describe result for acl check
2025-10-09T21:46:20.037407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:20.037409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:20.037416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:20.037423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341642792733790:7120] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:20.038108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341642792733790:7120] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:20.062574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341642792733790:7120] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:20.074284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341642792733790:7120] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:20.074309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341642792733790:7120] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:20.074313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341642792733790:7120] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:20.074325Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 686 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:20.093692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:20.099875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:20.100172Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:20.100177Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:20.100192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:20.100256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:20.100302Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341642430919611:6489]
2025-10-09T21:46:20.100953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:20.101606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:20.101660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:20.102024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:20.102145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:20.102147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:20.102148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:20.102150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:20.102151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:20.102153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:20.102154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.102156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.102158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.102165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:20.102178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:20.102183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:20.102188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:20.102193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:20.102197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:20.102202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.102206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.102211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.102219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:20.102229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:20.102237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:20.102243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:20.102249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:20.102255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:20.102260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:20.102267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:20.102272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:20.102278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:20.102283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:20.102289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:20.102294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.102300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.102306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.102312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.102318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.102324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.102359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:20.114607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:20.116240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:20.116255Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:20.116258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:20.116264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:20.116271Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 685 [2:7559341642430919611:6489]
2025-10-09T21:46:20.116288Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:20.116310Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:20.116314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:20.116317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:20.116320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:20.116340Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:20.116347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:20.116355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:20.116429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.116431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.116433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.116436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:20.116521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:20.116524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.116539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.116553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.116569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:20.116591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341642430919606:6489]
2025-10-09T21:46:20.116596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341642430919606:6489]
2025-10-09T21:46:20.116599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341642430919606:6489]
2025-10-09T21:46:20.116614Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:20.116632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:20.116646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:20.116649Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:20.116693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341642430919606:6489]
2025-10-09T21:46:20.116699Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:20.116908Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:20.116914Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:20.116917Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:20.116923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:20.117238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341642792733772:2432], now have 1 active actors on pipe
2025-10-09T21:46:20.117245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341642792733947:7120], now have 1 active actors on pipe
2025-10-09T21:46:20.115633Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341642792734036:6527]
2025-10-09T21:46:20.117005Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 685
2025-10-09T21:46:20.117035Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 685
2025-10-09T21:46:20.118685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:20.118697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:20.118716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341642792734036:6527], now have 1 active actors on pipe
2025-10-09T21:46:20.118742Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|540a3530-d7280cd0-d72543c7-f9bf84ee_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:20.118752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:20.118757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:20.118760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:20.118772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:20.118788Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:20.118810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:20.119301Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 685, pipe: [1:7559341642792734036:6527]
2025-10-09T21:46:20.119941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:20.119951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:20.119989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:20.120242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 686 partition: 0 MaxSeqNo: 25 sessionId: src_id|540a3530-d7280cd0-d72543c7-f9bf84ee_0
2025-10-09T21:46:20.124901Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf98ffe9-ee404538-db27d9e2-b409252c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|540a3530-d7280cd0-d72543c7-f9bf84ee_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:20.127563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|540a3530-d7280cd0-d72543c7-f9bf84ee_0] Write session: try to update token
2025-10-09T21:46:20.127590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|540a3530-d7280cd0-d72543c7-f9bf84ee_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:20.138752Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:20.138769Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:20.138772Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:20.138775Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:20.138785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341642430919606:6489]
2025-10-09T21:46:20.138790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341642430919606:6489]
2025-10-09T21:46:20.138793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341642430919606:6489]
2025-10-09T21:46:20.138840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:20.138847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.138850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:20.138986Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:20.141883Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:20.141894Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:20.141897Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:20.179319Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 686 sessionId: src_id|540a3530-d7280cd0-d72543c7-f9bf84ee_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:20.181854Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:20.360375Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:20.360884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:20.360894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:20.361124Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:20.361253Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:20.361302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:20.361368Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 686 reason: status is not ok: tablet will be restarted right now sessionId: src_id|540a3530-d7280cd0-d72543c7-f9bf84ee_0
2025-10-09T21:46:20.361553Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 686 sessionId: src_id|540a3530-d7280cd0-d72543c7-f9bf84ee_0 is DEAD
2025-10-09T21:46:20.361675Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341642792734036:6527]
2025-10-09T21:46:20.361978Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|540a3530-d7280cd0-d72543c7-f9bf84ee_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:20.361986Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|540a3530-d7280cd0-d72543c7-f9bf84ee_0] Write session will restart in 0.007789s
2025-10-09T21:46:20.362010Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|540a3530-d7280cd0-d72543c7-f9bf84ee_0] Write session: Do CDS request
2025-10-09T21:46:20.362016Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|540a3530-d7280cd0-d72543c7-f9bf84ee_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:20.373588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|540a3530-d7280cd0-d72543c7-f9bf84ee_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:20.374116Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:20.374134Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 687
2025-10-09T21:46:20.374317Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 687 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:20.374347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 687 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:20.374352Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=687 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:20.374356Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:20.374799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 687 sessionId:  describe result for acl check
2025-10-09T21:46:20.374836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:20.374838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:20.374840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:20.374846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341642792734074:7131] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:20.375565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341642792734074:7131] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:20.379843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341642792734074:7131] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:20.389353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341642792734074:7131] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:20.389377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341642792734074:7131] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:20.389381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341642792734074:7131] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:20.389391Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 687 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:20.408859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:20.413380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:20.413916Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:20.413921Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:20.413936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:20.414002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:20.414074Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341642430919907:6501]
2025-10-09T21:46:20.414751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:20.415399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:20.415454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:20.415804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:20.415842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:20.415843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:20.415844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:20.415845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:20.415847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:20.415848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:20.415849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.415850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.415851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.415856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:20.415867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:20.415870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:20.415873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:20.415876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:20.415880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:20.415883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.415886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.415888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.415898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:20.415904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:20.415910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:20.415914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:20.415919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:20.415923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:20.415927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:20.415945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:20.415950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:20.415956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:20.415960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:20.415964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:20.415969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.415973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.415977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.415981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.415987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.415991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.416018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:20.425689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:20.427482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:20.427489Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:20.427492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:20.427497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:20.427506Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 686 [2:7559341642430919907:6501]
2025-10-09T21:46:20.427522Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:20.427543Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:20.427547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:20.427550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:20.427552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:20.427571Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:20.427578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:20.427586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:20.427660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.427663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.427665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.427668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:20.427746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:20.427749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.427764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.427774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.427790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:20.427814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341642430919902:6501]
2025-10-09T21:46:20.427817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341642430919902:6501]
2025-10-09T21:46:20.427820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341642430919902:6501]
2025-10-09T21:46:20.427835Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:20.427853Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:20.427868Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:20.427871Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:20.427915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341642430919902:6501]
2025-10-09T21:46:20.427918Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:20.428336Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:20.428344Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:20.428346Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:20.428353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:20.429149Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 686
2025-10-09T21:46:20.429183Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 686
2025-10-09T21:46:20.430163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341642792734183:7131], now have 1 active actors on pipe
2025-10-09T21:46:20.430185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341642792734063:2432], now have 1 active actors on pipe
2025-10-09T21:46:20.430204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:20.430210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:20.430259Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7502ae72-25723b95-9f846e86-af1391f7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:20.430271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:20.430288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:20.430291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:20.430303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:20.430320Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:20.430349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:20.432148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:20.432163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:20.432196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:20.432547Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 687 partition: 0 MaxSeqNo: 25 sessionId: src_id|7502ae72-25723b95-9f846e86-af1391f7_0
2025-10-09T21:46:20.433031Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|540a3530-d7280cd0-d72543c7-f9bf84ee_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7502ae72-25723b95-9f846e86-af1391f7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:20.435609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7502ae72-25723b95-9f846e86-af1391f7_0] Write session: try to update token
2025-10-09T21:46:20.435632Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7502ae72-25723b95-9f846e86-af1391f7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:20.466113Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341642792734333:6527]
2025-10-09T21:46:20.467783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341642792734333:6527], now have 1 active actors on pipe
2025-10-09T21:46:20.467806Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:20.467809Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:20.467816Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:20.467818Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:20.467827Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341642430919902:6501]
2025-10-09T21:46:20.467831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341642430919902:6501]
2025-10-09T21:46:20.467835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341642430919902:6501]
2025-10-09T21:46:20.467878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:20.467887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.467890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:20.468031Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:20.470119Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 686, pipe: [1:7559341642792734333:6527]
2025-10-09T21:46:20.472803Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:20.472817Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:20.472820Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:20.477615Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 687 sessionId: src_id|7502ae72-25723b95-9f846e86-af1391f7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:20.479992Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:20.650197Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:20.651648Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:20.651669Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:20.651723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 687 reason: error at write: Disconnected sessionId: src_id|7502ae72-25723b95-9f846e86-af1391f7_0
2025-10-09T21:46:20.651902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 687 sessionId: src_id|7502ae72-25723b95-9f846e86-af1391f7_0 is DEAD
2025-10-09T21:46:20.652024Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:20.652062Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341642792734333:6527]
2025-10-09T21:46:20.652299Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7502ae72-25723b95-9f846e86-af1391f7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:20.652312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7502ae72-25723b95-9f846e86-af1391f7_0] Write session will restart in 0.005838s
2025-10-09T21:46:20.652339Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7502ae72-25723b95-9f846e86-af1391f7_0] Write session: Do CDS request
2025-10-09T21:46:20.652346Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7502ae72-25723b95-9f846e86-af1391f7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:20.650775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:20.650791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:20.653091Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:20.662117Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7502ae72-25723b95-9f846e86-af1391f7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:20.662416Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:20.662436Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 688
2025-10-09T21:46:20.662651Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 688 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:20.662688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 688 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:20.662694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=688 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:20.662697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:20.663743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 688 sessionId:  describe result for acl check
2025-10-09T21:46:20.663775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:20.663777Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:20.663779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:20.663784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341642792734349:7136] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:20.664313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341642792734349:7136] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:20.680416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341642792734349:7136] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:20.692215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341642792734349:7136] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:20.692238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341642792734349:7136] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:20.692241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341642792734349:7136] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:20.692251Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 688 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:20.716789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:20.730411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:20.730680Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:20.730692Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:20.730705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:20.730773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:20.730818Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341642430920171:6507]
2025-10-09T21:46:20.731292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:20.731848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:20.731902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:20.732232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:20.732277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:20.732278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:20.732279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:20.732281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:20.732282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:20.732284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:20.732285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.732287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.732288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.732294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:20.732306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:20.732323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:20.732328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:20.732332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:20.732337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:20.732341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.732344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.732348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.732357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:20.732366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:20.732372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:20.732378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:20.732383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:20.732388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:20.732395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:20.732400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:20.732405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:20.732410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:20.732415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:20.732420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:20.732425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.732431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.732435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.732440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.732445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.732450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.732498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:20.740554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:20.742330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:20.742337Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:20.742339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:20.742344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:20.742352Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 687 [2:7559341642430920171:6507]
2025-10-09T21:46:20.742372Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:20.742393Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:20.742396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:20.742399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:20.742402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:20.742420Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:20.742426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:20.742433Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:20.742497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.742499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.742501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.742504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:20.742579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:20.742582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.742594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:20.742604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:20.742618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:20.742639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341642430920165:6507]
2025-10-09T21:46:20.742642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341642430920165:6507]
2025-10-09T21:46:20.742644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341642430920165:6507]
2025-10-09T21:46:20.742652Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:20.742669Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:20.742673Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:20.742676Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:20.742680Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:20.742683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:20.742686Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:20.742727Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341642430920165:6507]
2025-10-09T21:46:20.742736Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:20.742746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341642792734468:7136], now have 1 active actors on pipe
2025-10-09T21:46:20.742752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341642792734339:2432], now have 1 active actors on pipe
2025-10-09T21:46:20.743053Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 687
2025-10-09T21:46:20.743089Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 687
2025-10-09T21:46:20.743184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:20.748549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:20.748568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:20.748616Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|73e51df2-e8e15d71-fb5c5cee-de206bab_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:20.748634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:20.748640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:20.748643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:20.748658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:20.748678Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:20.748703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:20.749108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:20.749115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:20.749130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:20.749372Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 688 partition: 0 MaxSeqNo: 25 sessionId: src_id|73e51df2-e8e15d71-fb5c5cee-de206bab_0
2025-10-09T21:46:20.749865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7502ae72-25723b95-9f846e86-af1391f7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|73e51df2-e8e15d71-fb5c5cee-de206bab_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:20.752612Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341642792734607:6527]
2025-10-09T21:46:20.752726Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73e51df2-e8e15d71-fb5c5cee-de206bab_0] Write session: try to update token
2025-10-09T21:46:20.752745Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73e51df2-e8e15d71-fb5c5cee-de206bab_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:20.753560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341642792734607:6527], now have 1 active actors on pipe
2025-10-09T21:46:20.753748Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 687, pipe: [1:7559341642792734607:6527]
2025-10-09T21:46:20.766909Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:20.766932Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:20.766936Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:20.766939Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:20.766952Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341642430920165:6507]
2025-10-09T21:46:20.766959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341642430920165:6507]
2025-10-09T21:46:20.766966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341642430920165:6507]
2025-10-09T21:46:20.767020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:20.767031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:20.767035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:20.767250Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:20.770948Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:20.770961Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:20.770965Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:20.776595Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 688 sessionId: src_id|73e51df2-e8e15d71-fb5c5cee-de206bab_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:20.779291Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:20.882657Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:46:20.882687Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 7
2025-10-09T21:46:21.008519Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:21.008822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:21.008834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:21.009731Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:21.011218Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:21.011246Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 7
2025-10-09T21:46:21.011284Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341642792734607:6527]
2025-10-09T21:46:21.011308Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:21.011411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 688 reason: status is not ok: tablet will be restarted right now sessionId: src_id|73e51df2-e8e15d71-fb5c5cee-de206bab_0
2025-10-09T21:46:21.011629Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 688 sessionId: src_id|73e51df2-e8e15d71-fb5c5cee-de206bab_0 is DEAD
2025-10-09T21:46:21.013738Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73e51df2-e8e15d71-fb5c5cee-de206bab_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:21.013752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73e51df2-e8e15d71-fb5c5cee-de206bab_0] Write session will restart in 0.008757s
2025-10-09T21:46:21.013781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73e51df2-e8e15d71-fb5c5cee-de206bab_0] Write session: Do CDS request
2025-10-09T21:46:21.013788Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73e51df2-e8e15d71-fb5c5cee-de206bab_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:21.023686Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73e51df2-e8e15d71-fb5c5cee-de206bab_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:21.025140Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:21.025160Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 689
2025-10-09T21:46:21.025385Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 689 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:21.025417Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 689 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:21.025424Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=689 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:21.025428Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:21.028696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 689 sessionId:  describe result for acl check
2025-10-09T21:46:21.028747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:21.028749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:21.028751Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:21.028761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341647087701935:7141] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:21.029364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341647087701935:7141] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:21.045207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341647087701935:7141] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:21.052014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341647087701935:7141] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:21.052041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341647087701935:7141] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:21.052045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341647087701935:7141] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:21.052055Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 689 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:21.107899Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341647087702167:6527]
2025-10-09T21:46:21.120575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:21.129188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:21.129547Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:21.129553Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:21.129568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:21.129657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:21.129709Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341646725887732:6510]
2025-10-09T21:46:21.130435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:21.131636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:21.131700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:21.132081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:21.132152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:21.132164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:21.132165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:21.132166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:21.132168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:21.132169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:21.132171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.132172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.132173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.132180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:21.132192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:21.132196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:21.132201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:21.132205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:21.132209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:21.132213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.132217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.132221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.132230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:21.132239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:21.132249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:21.132256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:21.132261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:21.132268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:21.132273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:21.132279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:21.132285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:21.132291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:21.132296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:21.132302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:21.132308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.132315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.132321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.132327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.132333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.132338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.132370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:21.143785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:21.144958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:21.144965Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:21.144968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:21.144973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:21.144983Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 688 [2:7559341646725887732:6510]
2025-10-09T21:46:21.145000Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:21.145026Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:21.145030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:21.145034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:21.145038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:21.145058Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:21.145065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:21.145074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:21.145151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.145154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.145156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.145159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:21.145248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:21.145251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.145269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.145280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.145296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:21.145319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341646725887726:6510]
2025-10-09T21:46:21.145325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341646725887726:6510]
2025-10-09T21:46:21.145327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341646725887726:6510]
2025-10-09T21:46:21.145345Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:21.145363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:21.145378Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:21.145381Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:21.145427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341646725887726:6510]
2025-10-09T21:46:21.145430Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:21.146855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:21.146865Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:21.146867Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:21.146875Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:21.147729Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 688
2025-10-09T21:46:21.147772Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 688, pipe: [1:7559341647087702167:6527]
2025-10-09T21:46:21.147798Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 688
2025-10-09T21:46:21.152692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341647087701917:2432], now have 1 active actors on pipe
2025-10-09T21:46:21.152708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341647087702167:6527], now have 1 active actors on pipe
2025-10-09T21:46:21.152712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341647087701974:7141], now have 1 active actors on pipe
2025-10-09T21:46:21.152770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:21.152777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:21.152865Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:46:21.152903Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7ea98be7-237b96bc-ff4a2c78-ea1e0ef6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:21.152914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:21.152921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:21.152924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:21.152941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:21.152960Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:21.152981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:21.153487Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 8 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:46:21.153537Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:46:21.154246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:21.154259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:21.154288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:21.154470Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 689 partition: 0 MaxSeqNo: 25 sessionId: src_id|7ea98be7-237b96bc-ff4a2c78-ea1e0ef6_0
2025-10-09T21:46:21.155059Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73e51df2-e8e15d71-fb5c5cee-de206bab_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7ea98be7-237b96bc-ff4a2c78-ea1e0ef6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:21.157965Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ea98be7-237b96bc-ff4a2c78-ea1e0ef6_0] Write session: try to update token
2025-10-09T21:46:21.157993Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ea98be7-237b96bc-ff4a2c78-ea1e0ef6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:21.194207Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 689 sessionId: src_id|7ea98be7-237b96bc-ff4a2c78-ea1e0ef6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:21.196846Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:21.197152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:21.197170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:21.197214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:21.198711Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:21.198730Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:21.198733Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:21.198735Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:21.198744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341646725887726:6510]
2025-10-09T21:46:21.198749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341646725887726:6510]
2025-10-09T21:46:21.198753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341646725887726:6510]
2025-10-09T21:46:21.198796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:21.198805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.198808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:21.198811Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:21.198818Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:21.198822Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:21.198947Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:21.200669Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:21.406757Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:21.422283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:21.422307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:21.422458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:21.422515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:21.422577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:21.422635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:21.422730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:21.422738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:21.422797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:21.422861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:21.422928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:21.422989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:21.423034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:21.423042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:21.423109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:21.423171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:21.423234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:21.423302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:21.423352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:21.423361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:21.423441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:21.423510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:21.423581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:21.423647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:21.423688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:21.423693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:21.433498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:21.433596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:21.433680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:21.433753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:21.433816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:21.433827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:21.433911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:21.433993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:21.434108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:21.434177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:21.434225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:21.434229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:21.434301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:21.434372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:21.434462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:21.434528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:21.434574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:21.434577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:21.434649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:21.434707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:21.434770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:21.434844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:21.434892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:21.434895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:21.434968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:21.435029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:21.435094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:21.435160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:21.435207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:21.435210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:21.435284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:21.435342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:21.435406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:21.435492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:21.435535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:21.435538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:21.436495Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:21.436554Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341647087702167:6527]
2025-10-09T21:46:21.436577Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:21.436580Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:21.436610Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 689 reason: error at write: Disconnected sessionId: src_id|7ea98be7-237b96bc-ff4a2c78-ea1e0ef6_0
2025-10-09T21:46:21.436805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 689 sessionId: src_id|7ea98be7-237b96bc-ff4a2c78-ea1e0ef6_0 is DEAD
2025-10-09T21:46:21.437949Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:21.438212Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ea98be7-237b96bc-ff4a2c78-ea1e0ef6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:21.438222Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ea98be7-237b96bc-ff4a2c78-ea1e0ef6_0] Write session will restart in 0.008893s
2025-10-09T21:46:21.438267Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ea98be7-237b96bc-ff4a2c78-ea1e0ef6_0] Write session: Do CDS request
2025-10-09T21:46:21.438274Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ea98be7-237b96bc-ff4a2c78-ea1e0ef6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:21.448627Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ea98be7-237b96bc-ff4a2c78-ea1e0ef6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:21.449590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:21.449610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 690
2025-10-09T21:46:21.449808Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 690 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:21.449865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 690 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:21.449869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=690 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:21.449873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:21.457778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 690 sessionId:  describe result for acl check
2025-10-09T21:46:21.457831Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:21.457834Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:21.457836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:21.457842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341647087702232:7152] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:21.458464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341647087702232:7152] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:21.470139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341647087702232:7152] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:21.475446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341647087702232:7152] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:21.475468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341647087702232:7152] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:21.475472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341647087702232:7152] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:21.475482Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 690 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:21.503556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:21.511429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:21.511693Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:21.511703Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:21.511716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:21.511784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:21.511836Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341646725888037:6523]
2025-10-09T21:46:21.512464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:21.512914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:21.512964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:21.513301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:21.513354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:21.513362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:21.513363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:21.513364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:21.513366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:21.513367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:21.513369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.513370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.513372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.513377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:21.513388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:21.513393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:21.513397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:21.513401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:21.513405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:21.513409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.513412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.513416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.513425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:21.513433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:21.513439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:21.513444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:21.513448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:21.513454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:21.513458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:21.513464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:21.513468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:21.513473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:21.513477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:21.513482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:21.513487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.513492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.513496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.513500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.513504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.513509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.513534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:21.530062Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 689
2025-10-09T21:46:21.527991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:21.530126Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 689
2025-10-09T21:46:21.529554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:21.529561Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:21.529564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:21.529569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:21.529577Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 689 [2:7559341646725888037:6523]
2025-10-09T21:46:21.529595Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:21.529616Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:21.529620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:21.529623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:21.529626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:21.529644Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:21.529650Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:21.529659Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:21.529723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.529725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.529730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.529733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:21.529815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:21.529818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.529831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.529845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.529864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:21.529885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341646725888031:6523]
2025-10-09T21:46:21.529887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341646725888031:6523]
2025-10-09T21:46:21.529889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341646725888031:6523]
2025-10-09T21:46:21.529897Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:21.529914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:21.529923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:21.529927Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:21.529929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:21.529929Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:21.529933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:21.529980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341647087702297:7152], now have 1 active actors on pipe
2025-10-09T21:46:21.529984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341647087702217:2432], now have 1 active actors on pipe
2025-10-09T21:46:21.529987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341646725888031:6523]
2025-10-09T21:46:21.529994Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:21.530002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:21.530258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:21.530267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:21.530305Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a7d81d95-8328d4d7-d955b7a-87ab2e8a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:21.530334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:21.530340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:21.530343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:21.530356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:21.530369Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:21.530394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:21.530628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:21.530635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:21.530652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:21.530770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 690 partition: 0 MaxSeqNo: 25 sessionId: src_id|a7d81d95-8328d4d7-d955b7a-87ab2e8a_0
2025-10-09T21:46:21.531130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ea98be7-237b96bc-ff4a2c78-ea1e0ef6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a7d81d95-8328d4d7-d955b7a-87ab2e8a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:21.533639Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7d81d95-8328d4d7-d955b7a-87ab2e8a_0] Write session: try to update token
2025-10-09T21:46:21.533661Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7d81d95-8328d4d7-d955b7a-87ab2e8a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:21.536683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341647087702491:6527], now have 1 active actors on pipe
2025-10-09T21:46:21.536309Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341647087702491:6527]
2025-10-09T21:46:21.536751Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 689, pipe: [1:7559341647087702491:6527]
2025-10-09T21:46:21.548612Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:21.548632Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:21.548635Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:21.548637Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:21.548649Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341646725888031:6523]
2025-10-09T21:46:21.548654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341646725888031:6523]
2025-10-09T21:46:21.548658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341646725888031:6523]
2025-10-09T21:46:21.548706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:21.548713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.548716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:21.549458Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:21.553079Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:21.553093Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:21.553098Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:21.557851Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 690 sessionId: src_id|a7d81d95-8328d4d7-d955b7a-87ab2e8a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:21.560679Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:21.764619Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:21.765505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:21.765525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:21.765769Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:21.771159Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341647087702491:6527]
2025-10-09T21:46:21.771207Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:21.771292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 690 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a7d81d95-8328d4d7-d955b7a-87ab2e8a_0
2025-10-09T21:46:21.771940Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7d81d95-8328d4d7-d955b7a-87ab2e8a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:21.771952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7d81d95-8328d4d7-d955b7a-87ab2e8a_0] Write session will restart in 0.008186s
2025-10-09T21:46:21.771976Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7d81d95-8328d4d7-d955b7a-87ab2e8a_0] Write session: Do CDS request
2025-10-09T21:46:21.771983Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7d81d95-8328d4d7-d955b7a-87ab2e8a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:21.771486Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 690 sessionId: src_id|a7d81d95-8328d4d7-d955b7a-87ab2e8a_0 is DEAD
2025-10-09T21:46:21.771634Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:21.782303Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7d81d95-8328d4d7-d955b7a-87ab2e8a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:21.783188Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:21.783208Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 691
2025-10-09T21:46:21.783689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 691 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:21.783733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 691 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:21.783739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=691 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:21.783743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:21.784239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 691 sessionId:  describe result for acl check
2025-10-09T21:46:21.784270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:21.784272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:21.784274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:21.784280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341647087702517:7160] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:21.784937Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341647087702517:7160] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:21.793180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341647087702517:7160] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:21.802916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341647087702517:7160] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:21.802938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341647087702517:7160] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:21.802942Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341647087702517:7160] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:21.802954Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 691 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:21.834919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:21.841760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:21.842052Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:21.842057Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:21.842072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:21.842137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:21.842177Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341646725888300:6529]
2025-10-09T21:46:21.842727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:21.846492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:21.846585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:21.846958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:21.847010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:21.847011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:21.847013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:21.847014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:21.847015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:21.847016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:21.847018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.847019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.847020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.847027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:21.847038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:21.847042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:21.847045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:21.847049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:21.847053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:21.847056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.847059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.847062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.847071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:21.847079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:21.847086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:21.847091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:21.847096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:21.847101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:21.847106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:21.847111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:21.847116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:21.847120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:21.847125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:21.847129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:21.847134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.847138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.847143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.847148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.847153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.847158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.847191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:21.857339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:21.859153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:21.859158Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:21.859161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:21.859165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:21.859173Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 690 [2:7559341646725888300:6529]
2025-10-09T21:46:21.859187Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:21.859207Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:21.859211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:21.859214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:21.859216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:21.859235Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:21.859240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:21.859247Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:21.859310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.859312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.859314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.859316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:21.859386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:21.859389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.859401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:21.859410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:21.859424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:21.859444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341646725888295:6529]
2025-10-09T21:46:21.859446Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341646725888295:6529]
2025-10-09T21:46:21.859448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341646725888295:6529]
2025-10-09T21:46:21.859458Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:21.859472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:21.859483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:21.859487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:21.859488Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:21.859490Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:21.859490Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:21.859535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341647087702594:7160], now have 1 active actors on pipe
2025-10-09T21:46:21.859539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341647087702504:2432], now have 1 active actors on pipe
2025-10-09T21:46:21.859541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341646725888295:6529]
2025-10-09T21:46:21.859543Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:21.859858Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:21.860675Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 690
2025-10-09T21:46:21.860711Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 690
2025-10-09T21:46:21.861677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:21.861702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:21.861751Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4ce2cb54-4186b209-e68a032c-b64dd4c5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:21.861762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:21.861767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:21.861770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:21.861784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:21.861800Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:21.861831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:21.865720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:21.865735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:21.865767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:21.867547Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 691 partition: 0 MaxSeqNo: 25 sessionId: src_id|4ce2cb54-4186b209-e68a032c-b64dd4c5_0
2025-10-09T21:46:21.871200Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341647087702771:6527]
2025-10-09T21:46:21.872072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341647087702771:6527], now have 1 active actors on pipe
2025-10-09T21:46:21.872292Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 690, pipe: [1:7559341647087702771:6527]
2025-10-09T21:46:21.872868Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7d81d95-8328d4d7-d955b7a-87ab2e8a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4ce2cb54-4186b209-e68a032c-b64dd4c5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:21.875381Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ce2cb54-4186b209-e68a032c-b64dd4c5_0] Write session: try to update token
2025-10-09T21:46:21.875403Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ce2cb54-4186b209-e68a032c-b64dd4c5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:21.884860Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:21.884877Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:21.884880Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:21.884882Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:21.884892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341646725888295:6529]
2025-10-09T21:46:21.884896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341646725888295:6529]
2025-10-09T21:46:21.884899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341646725888295:6529]
2025-10-09T21:46:21.884946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:21.884955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:21.884957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:21.885091Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:21.887355Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:21.887369Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:21.887386Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:21.967766Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 691 sessionId: src_id|4ce2cb54-4186b209-e68a032c-b64dd4c5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:21.970371Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:22.077625Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:22.078634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:22.078649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:22.079337Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341647087702771:6527]
2025-10-09T21:46:22.079373Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:22.079376Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:22.079431Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 691 reason: error at write: Disconnected sessionId: src_id|4ce2cb54-4186b209-e68a032c-b64dd4c5_0
2025-10-09T21:46:22.079619Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 691 sessionId: src_id|4ce2cb54-4186b209-e68a032c-b64dd4c5_0 is DEAD
2025-10-09T21:46:22.079752Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:22.082808Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ce2cb54-4186b209-e68a032c-b64dd4c5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:22.082819Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ce2cb54-4186b209-e68a032c-b64dd4c5_0] Write session will restart in 0.006848s
2025-10-09T21:46:22.082842Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ce2cb54-4186b209-e68a032c-b64dd4c5_0] Write session: Do CDS request
2025-10-09T21:46:22.082857Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ce2cb54-4186b209-e68a032c-b64dd4c5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:22.082539Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:22.090793Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ce2cb54-4186b209-e68a032c-b64dd4c5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:22.095159Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:22.095181Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 692
2025-10-09T21:46:22.096774Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 692 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:22.096828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 692 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:22.096834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=692 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:22.096839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:22.097478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 692 sessionId:  describe result for acl check
2025-10-09T21:46:22.097529Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:22.097531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:22.097533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:22.097539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341651382670101:7164] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:22.098166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341651382670101:7164] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:22.110284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341651382670101:7164] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:22.122201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341651382670101:7164] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:22.122224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341651382670101:7164] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:22.122228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341651382670101:7164] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:22.122238Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 692 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:22.146637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:22.152872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:22.153151Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:22.153156Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:22.153171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:22.153237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:22.153712Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341651020855860:6531]
2025-10-09T21:46:22.154350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:22.155296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:22.155359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:22.155710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:22.155774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:22.155775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:22.155777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:22.155778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:22.155780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:22.155781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:22.155782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.155783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.155785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.155791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:22.155803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:22.155807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:22.155811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:22.155815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:22.155818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:22.155822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.155825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.155828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.155836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:22.155845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:22.155851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:22.155856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:22.155861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:22.155866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:22.155870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:22.155876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:22.155881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:22.155886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:22.155891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:22.155898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:22.155903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.155908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.155913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.155918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.155923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.155927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.155959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:22.168085Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 691
2025-10-09T21:46:22.168084Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 691
2025-10-09T21:46:22.165671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:22.167563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:22.167572Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:22.167575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:22.167580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:22.167593Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 691 [2:7559341651020855860:6531]
2025-10-09T21:46:22.167611Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:22.167634Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:22.167639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:22.167642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:22.167646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:22.167667Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:22.167674Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:22.167683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:22.167756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.167758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.167760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.167764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:22.171873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:22.171889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.171910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.171933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.171951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:22.171965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341651382670210:7164], now have 1 active actors on pipe
2025-10-09T21:46:22.171969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341651382670077:2432], now have 1 active actors on pipe
2025-10-09T21:46:22.172003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:22.172010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:22.172047Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341651020855855:6531]
2025-10-09T21:46:22.172050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341651020855855:6531]
2025-10-09T21:46:22.172052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341651020855855:6531]
2025-10-09T21:46:22.172071Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:22.172107Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|74bb36e3-1168a29b-ad71174d-39a4a315_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:22.172120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:22.172125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:22.172128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:22.172142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:22.172161Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:22.172178Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:22.172194Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:22.172198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:22.172255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:22.172281Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341651020855855:6531]
2025-10-09T21:46:22.172284Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:22.172469Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:22.172488Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:22.172491Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:22.172503Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:22.173480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:22.173494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:22.173525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:22.174082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 692 partition: 0 MaxSeqNo: 25 sessionId: src_id|74bb36e3-1168a29b-ad71174d-39a4a315_0
2025-10-09T21:46:22.174866Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ce2cb54-4186b209-e68a032c-b64dd4c5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|74bb36e3-1168a29b-ad71174d-39a4a315_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:22.177473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74bb36e3-1168a29b-ad71174d-39a4a315_0] Write session: try to update token
2025-10-09T21:46:22.177493Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74bb36e3-1168a29b-ad71174d-39a4a315_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:22.178894Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341651382670352:6527]
2025-10-09T21:46:22.179382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341651382670352:6527], now have 1 active actors on pipe
2025-10-09T21:46:22.179475Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 691, pipe: [1:7559341651382670352:6527]
2025-10-09T21:46:22.241424Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 692 sessionId: src_id|74bb36e3-1168a29b-ad71174d-39a4a315_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:22.244163Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:22.246307Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:22.244599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:22.244618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:22.245915Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:22.245920Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:22.245922Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:22.245924Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:22.245934Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341651020855855:6531]
2025-10-09T21:46:22.245939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341651020855855:6531]
2025-10-09T21:46:22.245943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341651020855855:6531]
2025-10-09T21:46:22.245989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:22.246008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:22.246016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.246018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:22.246070Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:22.246076Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:22.246081Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:22.246197Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:22.427921Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:22.437739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:22.438325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:22.438525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:22.438593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:22.438654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:22.438722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:22.438764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:22.438769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:22.438842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:22.438902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:22.438960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:22.439034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:22.439072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:22.439076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:22.439142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:22.439201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:22.439268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:22.439411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:22.439452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:22.439460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:22.439579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:22.439645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:22.439711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:22.439777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:22.439817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:22.439821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:22.439893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:22.439956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:22.440020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:22.440085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:22.440127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:22.440130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:22.440196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:22.440256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:22.440315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:22.440373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:22.440415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:22.440419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:22.440484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:22.440537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:22.440604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:22.440669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:22.440711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:22.440716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:22.440785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:22.440841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:22.440902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:22.440961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:22.441001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:22.441003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:22.441076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:22.441139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:22.441204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:22.441265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:22.441309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:22.441312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:22.441383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:22.441440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:22.441499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:22.441561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:22.441605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:22.441608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:22.442463Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:22.442531Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341651382670352:6527]
2025-10-09T21:46:22.442554Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:22.442560Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:22.442590Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 692 reason: error at write: Disconnected sessionId: src_id|74bb36e3-1168a29b-ad71174d-39a4a315_0
2025-10-09T21:46:22.442775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 692 sessionId: src_id|74bb36e3-1168a29b-ad71174d-39a4a315_0 is DEAD
2025-10-09T21:46:22.443387Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74bb36e3-1168a29b-ad71174d-39a4a315_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:22.443399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74bb36e3-1168a29b-ad71174d-39a4a315_0] Write session will restart in 0.009811s
2025-10-09T21:46:22.443424Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74bb36e3-1168a29b-ad71174d-39a4a315_0] Write session: Do CDS request
2025-10-09T21:46:22.443431Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74bb36e3-1168a29b-ad71174d-39a4a315_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:22.445194Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:22.454676Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74bb36e3-1168a29b-ad71174d-39a4a315_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:22.454741Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:22.454762Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 693
2025-10-09T21:46:22.454947Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 693 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:22.454981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 693 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:22.454994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=693 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:22.454999Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:22.455759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 693 sessionId:  describe result for acl check
2025-10-09T21:46:22.455817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:22.455823Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:22.455825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:22.455831Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341651382670372:7169] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:22.456449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341651382670372:7169] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:22.465461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341651382670372:7169] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:22.473055Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341651382670372:7169] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:22.473078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341651382670372:7169] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:22.473098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341651382670372:7169] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:22.473107Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 693 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:22.511370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:22.513507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:22.513798Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:22.513802Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:22.513816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:22.513897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:22.513952Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341651020856159:6539]
2025-10-09T21:46:22.514572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:22.516991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:22.517093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:22.517551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:22.517612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:22.517613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:22.517615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:22.517616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:22.517617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:22.517618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:22.517619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.517621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.517622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.517629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:22.517639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:22.517642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:22.517646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:22.517668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:22.517672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:22.517676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.517679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.517682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.517690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:22.517699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:22.517705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:22.517710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:22.517715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:22.517720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:22.517725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:22.517729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:22.517733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:22.517738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:22.517742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:22.517747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:22.517752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.517756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.517763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.517768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.517773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.517777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.517805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:22.538355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:22.539410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:22.539427Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:22.539429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:22.539433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:22.539439Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 692 [2:7559341651020856159:6539]
2025-10-09T21:46:22.539453Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:22.539472Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:22.539475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:22.539479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:22.539482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:22.539500Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:22.539504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:22.539510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:22.539559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.539567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.539568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.539570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:22.539635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:22.539638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.539649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.539658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.539671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:22.539692Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341651020856154:6539]
2025-10-09T21:46:22.539694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341651020856154:6539]
2025-10-09T21:46:22.539696Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341651020856154:6539]
2025-10-09T21:46:22.539704Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:22.539718Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:22.539730Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:22.539733Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:22.539737Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:22.539739Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:22.539741Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:22.539745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341651382670442:7169], now have 1 active actors on pipe
2025-10-09T21:46:22.539784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341651020856154:6539]
2025-10-09T21:46:22.539791Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:22.539796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341651382670358:2432], now have 1 active actors on pipe
2025-10-09T21:46:22.539807Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:22.539886Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 692
2025-10-09T21:46:22.539938Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 692
2025-10-09T21:46:22.540154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:22.540168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:22.540201Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|210a5a23-549df96e-74ef63eb-7eda3239_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:22.540216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:22.540224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:22.540227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:22.540239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:22.540251Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:22.540270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:22.543294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:22.543308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:22.543339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:22.544536Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341651382670642:6527]
2025-10-09T21:46:22.545426Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74bb36e3-1168a29b-ad71174d-39a4a315_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|210a5a23-549df96e-74ef63eb-7eda3239_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:22.544979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 693 partition: 0 MaxSeqNo: 25 sessionId: src_id|210a5a23-549df96e-74ef63eb-7eda3239_0
2025-10-09T21:46:22.545775Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 692, pipe: [1:7559341651382670642:6527]
2025-10-09T21:46:22.546694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341651382670642:6527], now have 1 active actors on pipe
2025-10-09T21:46:22.548129Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|210a5a23-549df96e-74ef63eb-7eda3239_0] Write session: try to update token
2025-10-09T21:46:22.548146Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|210a5a23-549df96e-74ef63eb-7eda3239_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:22.576811Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 693 sessionId: src_id|210a5a23-549df96e-74ef63eb-7eda3239_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:22.575911Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:22.575929Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:22.575932Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:22.575934Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:22.575945Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341651020856154:6539]
2025-10-09T21:46:22.575949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341651020856154:6539]
2025-10-09T21:46:22.575953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341651020856154:6539]
2025-10-09T21:46:22.575997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:22.576005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.576008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:22.576159Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:22.578230Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:22.578241Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:22.578246Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:22.579276Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:22.724520Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: CLIENT_DEADLINE_EXCEEDED. Description: [ { <main>: Error: GRpc error: (4): Deadline Exceeded } { <main>: Error: Grpc error response on endpoint localhost:14704 } ]
2025-10-09T21:46:22.769888Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:22.771323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:22.771340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:22.772157Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:22.772181Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:22.772237Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:22.772240Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:22.772300Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 693 reason: error at write: Disconnected sessionId: src_id|210a5a23-549df96e-74ef63eb-7eda3239_0
2025-10-09T21:46:22.772505Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 693 sessionId: src_id|210a5a23-549df96e-74ef63eb-7eda3239_0 is DEAD
2025-10-09T21:46:22.772692Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|210a5a23-549df96e-74ef63eb-7eda3239_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:22.772703Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|210a5a23-549df96e-74ef63eb-7eda3239_0] Write session will restart in 0.007243s
2025-10-09T21:46:22.772727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|210a5a23-549df96e-74ef63eb-7eda3239_0] Write session: Do CDS request
2025-10-09T21:46:22.772734Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|210a5a23-549df96e-74ef63eb-7eda3239_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:22.772692Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341651382670642:6527]
2025-10-09T21:46:22.781916Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|210a5a23-549df96e-74ef63eb-7eda3239_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:22.782625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:22.782647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 694
2025-10-09T21:46:22.783014Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 694 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:22.783052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 694 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:22.783059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=694 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:22.783063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:22.783585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 694 sessionId:  describe result for acl check
2025-10-09T21:46:22.783613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:22.783615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:22.783616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:22.783622Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341651382670670:7181] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:22.784151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341651382670670:7181] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:22.792784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341651382670670:7181] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:22.802790Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341651382670670:7181] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:22.802811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341651382670670:7181] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:22.802815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341651382670670:7181] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:22.802826Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 694 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:22.830186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:22.834996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:22.835650Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:22.835660Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:22.835675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:22.835736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:22.835784Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341651020856433:6549]
2025-10-09T21:46:22.836173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:22.836670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:22.836736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:22.836999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:22.837036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:22.837037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:22.837038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:22.837040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:22.837041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:22.837042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:22.837043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.837044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.837045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.837051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:22.837062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:22.837066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:22.837069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:22.837073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:22.837077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:22.837081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.837085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.837088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.837095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:22.837100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:22.837105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:22.837109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:22.837111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:22.837114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:22.837117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:22.837120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:22.837123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:22.837128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:22.837130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:22.837133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:22.837136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.837139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.837141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.837144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.837146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.837150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.837168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:22.846440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:22.847548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:22.847554Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:22.847557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:22.847562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:22.847570Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 693 [2:7559341651020856433:6549]
2025-10-09T21:46:22.847587Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:22.847608Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:22.847612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:22.847615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:22.847617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:22.847638Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:22.847644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:22.847651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:22.847723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.847725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.847728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.847731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:22.847809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:22.847812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.847826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:22.847838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:22.847855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:22.847878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341651020856427:6549]
2025-10-09T21:46:22.847881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341651020856427:6549]
2025-10-09T21:46:22.847883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341651020856427:6549]
2025-10-09T21:46:22.847898Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:22.847913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:22.847928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:22.847932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:22.847975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341651020856427:6549]
2025-10-09T21:46:22.847978Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:22.848473Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:22.848481Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:22.848483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:22.848489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:22.849267Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 693
2025-10-09T21:46:22.849314Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 693
2025-10-09T21:46:22.849826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341651382670658:2432], now have 1 active actors on pipe
2025-10-09T21:46:22.849836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341651382670760:7181], now have 1 active actors on pipe
2025-10-09T21:46:22.849858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:22.849863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:22.849897Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3db2a4ca-9dcfef2a-3d2117d1-2a13dceb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:22.849908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:22.849913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:22.849916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:22.849933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:22.849949Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:22.849967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:22.850620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:22.850629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:22.850649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:22.850807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 694 partition: 0 MaxSeqNo: 25 sessionId: src_id|3db2a4ca-9dcfef2a-3d2117d1-2a13dceb_0
2025-10-09T21:46:22.851822Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|210a5a23-549df96e-74ef63eb-7eda3239_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3db2a4ca-9dcfef2a-3d2117d1-2a13dceb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:22.854419Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3db2a4ca-9dcfef2a-3d2117d1-2a13dceb_0] Write session: try to update token
2025-10-09T21:46:22.854444Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3db2a4ca-9dcfef2a-3d2117d1-2a13dceb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:22.871052Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341651382670930:6527]
2025-10-09T21:46:22.871550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341651382670930:6527], now have 1 active actors on pipe
2025-10-09T21:46:22.871715Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 693, pipe: [1:7559341651382670930:6527]
2025-10-09T21:46:22.884188Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:22.884205Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:22.884208Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:22.884210Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:22.884223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341651020856427:6549]
2025-10-09T21:46:22.884227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341651020856427:6549]
2025-10-09T21:46:22.884231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341651020856427:6549]
2025-10-09T21:46:22.884279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:22.884285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:22.884288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:22.884429Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:22.887196Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:22.887222Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:22.887226Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:22.902930Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 694 sessionId: src_id|3db2a4ca-9dcfef2a-3d2117d1-2a13dceb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:22.905485Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:23.068739Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:23.069491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:23.069503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:23.071240Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341651382670930:6527]
2025-10-09T21:46:23.071287Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:23.071335Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:23.071343Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:23.071404Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 694 reason: error at write: Disconnected sessionId: src_id|3db2a4ca-9dcfef2a-3d2117d1-2a13dceb_0
2025-10-09T21:46:23.071832Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3db2a4ca-9dcfef2a-3d2117d1-2a13dceb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:23.071855Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3db2a4ca-9dcfef2a-3d2117d1-2a13dceb_0] Write session will restart in 0.007119s
2025-10-09T21:46:23.071879Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3db2a4ca-9dcfef2a-3d2117d1-2a13dceb_0] Write session: Do CDS request
2025-10-09T21:46:23.071886Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3db2a4ca-9dcfef2a-3d2117d1-2a13dceb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:23.071799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 694 sessionId: src_id|3db2a4ca-9dcfef2a-3d2117d1-2a13dceb_0 is DEAD
2025-10-09T21:46:23.072011Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:23.080603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3db2a4ca-9dcfef2a-3d2117d1-2a13dceb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:23.080711Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:23.080746Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 695
2025-10-09T21:46:23.080943Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 695 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:23.080989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 695 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:23.081002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=695 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:23.081006Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:23.081563Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 695 sessionId:  describe result for acl check
2025-10-09T21:46:23.081611Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:23.081613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:23.081615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:23.081621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341655677638245:7185] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:23.082217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341655677638245:7185] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:23.091090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341655677638245:7185] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:23.099113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341655677638245:7185] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:23.099140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341655677638245:7185] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:23.099145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341655677638245:7185] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:23.099156Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 695 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:23.154379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:23.160496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:23.160780Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:23.160785Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:23.160799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:23.160865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:23.160922Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341655315823996:6551]
2025-10-09T21:46:23.161581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:23.162255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:23.162335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:23.162717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:23.162811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:23.162813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:23.162814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:23.162815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:23.162816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:23.162817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:23.162818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.162819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.162821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.162826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:23.162837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:23.162840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:23.162844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:23.162848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:23.162852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:23.162856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.162859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.162864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.162873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:23.162883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:23.162891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:23.162897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:23.162902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:23.162908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:23.162913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:23.162919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:23.162925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:23.162931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:23.162935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:23.162940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:23.162946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.162952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.162958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.162964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.162970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.162976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.163027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:23.172503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:23.171083Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341655677638509:6527]
2025-10-09T21:46:23.174001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:23.174018Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:23.174070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:23.174075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:23.174083Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 694 [2:7559341655315823996:6551]
2025-10-09T21:46:23.174100Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:23.174123Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:23.174133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:23.174137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:23.174140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:23.174162Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:23.174168Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:23.174176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:23.174248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.174251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.174253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.174256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:23.174334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:23.174337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.174357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.174369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.174387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:23.174429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341655315823990:6551]
2025-10-09T21:46:23.174432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341655315823990:6551]
2025-10-09T21:46:23.174434Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341655315823990:6551]
2025-10-09T21:46:23.174450Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:23.174467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:23.174484Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:23.174494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:23.174543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341655315823990:6551]
2025-10-09T21:46:23.174551Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:23.174561Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:23.174565Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:23.174568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:23.174576Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:23.175277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341655677638509:6527], now have 1 active actors on pipe
2025-10-09T21:46:23.175290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341655677638236:2432], now have 1 active actors on pipe
2025-10-09T21:46:23.175294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341655677638293:7185], now have 1 active actors on pipe
2025-10-09T21:46:23.175987Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 694, pipe: [1:7559341655677638509:6527]
2025-10-09T21:46:23.176038Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 694
2025-10-09T21:46:23.176071Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 694
2025-10-09T21:46:23.176291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:23.176320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:23.176371Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2efd2d51-c23f01bb-debfee56-187c9d40_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:23.176383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:23.176389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:23.176392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:23.176407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:23.176426Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:23.176447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:23.177169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:23.177199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:23.177223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:23.177375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 695 partition: 0 MaxSeqNo: 25 sessionId: src_id|2efd2d51-c23f01bb-debfee56-187c9d40_0
2025-10-09T21:46:23.177900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3db2a4ca-9dcfef2a-3d2117d1-2a13dceb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2efd2d51-c23f01bb-debfee56-187c9d40_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:23.180582Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2efd2d51-c23f01bb-debfee56-187c9d40_0] Write session: try to update token
2025-10-09T21:46:23.180605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2efd2d51-c23f01bb-debfee56-187c9d40_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:23.197070Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:23.197090Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:23.197093Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:23.197095Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:23.197107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341655315823990:6551]
2025-10-09T21:46:23.197111Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341655315823990:6551]
2025-10-09T21:46:23.197115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341655315823990:6551]
2025-10-09T21:46:23.197164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:23.197172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.197175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:23.197246Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:23.197256Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:23.197259Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:23.197327Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:23.208353Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 695 sessionId: src_id|2efd2d51-c23f01bb-debfee56-187c9d40_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:23.210989Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:23.392321Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:23.393097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:23.393108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:23.393753Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341655677638509:6527]
2025-10-09T21:46:23.393803Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:23.393852Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:23.393859Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:23.393917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 695 reason: error at write: Disconnected sessionId: src_id|2efd2d51-c23f01bb-debfee56-187c9d40_0
2025-10-09T21:46:23.394146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 695 sessionId: src_id|2efd2d51-c23f01bb-debfee56-187c9d40_0 is DEAD
2025-10-09T21:46:23.394330Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2efd2d51-c23f01bb-debfee56-187c9d40_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:23.394351Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2efd2d51-c23f01bb-debfee56-187c9d40_0] Write session will restart in 0.009116s
2025-10-09T21:46:23.394375Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2efd2d51-c23f01bb-debfee56-187c9d40_0] Write session: Do CDS request
2025-10-09T21:46:23.394381Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2efd2d51-c23f01bb-debfee56-187c9d40_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:23.394005Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:23.404567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2efd2d51-c23f01bb-debfee56-187c9d40_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:23.406536Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:23.406555Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 696
2025-10-09T21:46:23.406964Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 696 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:23.407008Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 696 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:23.407013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=696 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:23.407019Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:23.408180Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 696 sessionId:  describe result for acl check
2025-10-09T21:46:23.408222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:23.408224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:23.408226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:23.408232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341655677638551:7190] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:23.408788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341655677638551:7190] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:23.422439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341655677638551:7190] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:23.438828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341655677638551:7190] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:23.438853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341655677638551:7190] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:23.438857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341655677638551:7190] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:23.438868Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 696 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:23.459226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:23.459945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:23.460200Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:23.460205Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:23.460220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:23.460313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:23.460364Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341655315824264:6553]
2025-10-09T21:46:23.461043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:23.461895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:23.461953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:23.462357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:23.462440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:23.462441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:23.462442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:23.462444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:23.462445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:23.462447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:23.462448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.462449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.462451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.462474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:23.462487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:23.462492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:23.462496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:23.462500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:23.462504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:23.462509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.462513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.462517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.462527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:23.462536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:23.462543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:23.462549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:23.462554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:23.462562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:23.462567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:23.462572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:23.462577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:23.462582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:23.462586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:23.462592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:23.462604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.462610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.462616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.462622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.462627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.462632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.462664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:23.471705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:23.472843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:23.472849Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:23.472853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:23.472858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:23.472867Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 695 [2:7559341655315824264:6553]
2025-10-09T21:46:23.472883Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:23.472907Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:23.472911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:23.472914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:23.472917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:23.472940Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:23.472946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:23.472955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:23.473036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.473038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.473040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.473044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:23.473126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:23.473130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.473144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.473155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.473171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:23.473194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341655315824258:6553]
2025-10-09T21:46:23.473197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341655315824258:6553]
2025-10-09T21:46:23.473199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341655315824258:6553]
2025-10-09T21:46:23.473214Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:23.473233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:23.473248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:23.473251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:23.473294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341655315824258:6553]
2025-10-09T21:46:23.473297Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:23.474132Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:23.474138Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:23.474141Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:23.474148Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:23.475053Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 695
2025-10-09T21:46:23.475089Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 695
2025-10-09T21:46:23.475672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341655677638731:7190], now have 1 active actors on pipe
2025-10-09T21:46:23.475684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341655677638519:2432], now have 1 active actors on pipe
2025-10-09T21:46:23.475707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:23.475715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:23.475754Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|27ba91a4-7377f2e9-82726873-8801551a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:23.475766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:23.475772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:23.475775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:23.475789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:23.475809Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:23.475831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:23.480236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:23.480253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:23.480294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:23.486172Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 696 partition: 0 MaxSeqNo: 25 sessionId: src_id|27ba91a4-7377f2e9-82726873-8801551a_0
2025-10-09T21:46:23.490863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2efd2d51-c23f01bb-debfee56-187c9d40_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|27ba91a4-7377f2e9-82726873-8801551a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:23.493368Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|27ba91a4-7377f2e9-82726873-8801551a_0] Write session: try to update token
2025-10-09T21:46:23.493389Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|27ba91a4-7377f2e9-82726873-8801551a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:23.493651Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341655677638792:6527]
2025-10-09T21:46:23.494431Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 695, pipe: [1:7559341655677638792:6527]
2025-10-09T21:46:23.494489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341655677638792:6527], now have 1 active actors on pipe
2025-10-09T21:46:23.507242Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:23.507262Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:23.507264Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:23.507266Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:23.507276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341655315824258:6553]
2025-10-09T21:46:23.507280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341655315824258:6553]
2025-10-09T21:46:23.507283Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341655315824258:6553]
2025-10-09T21:46:23.507333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:23.507340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.507343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:23.507482Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:23.508418Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:23.508427Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:23.508431Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:23.522978Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 696 sessionId: src_id|27ba91a4-7377f2e9-82726873-8801551a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:23.525467Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:23.696272Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:23.696851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:23.696869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:23.697223Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:23.697240Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:23.697295Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 696 reason: error at write: Disconnected sessionId: src_id|27ba91a4-7377f2e9-82726873-8801551a_0
2025-10-09T21:46:23.697121Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:23.697487Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 696 sessionId: src_id|27ba91a4-7377f2e9-82726873-8801551a_0 is DEAD
2025-10-09T21:46:23.697601Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341655677638792:6527]
2025-10-09T21:46:23.697627Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:23.697906Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27ba91a4-7377f2e9-82726873-8801551a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:23.697917Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27ba91a4-7377f2e9-82726873-8801551a_0] Write session will restart in 0.005300s
2025-10-09T21:46:23.697941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27ba91a4-7377f2e9-82726873-8801551a_0] Write session: Do CDS request
2025-10-09T21:46:23.697952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27ba91a4-7377f2e9-82726873-8801551a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:23.704554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|27ba91a4-7377f2e9-82726873-8801551a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:23.708757Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:23.708781Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 697
2025-10-09T21:46:23.709059Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 697 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:23.709100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 697 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:23.709109Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=697 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:23.709113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:23.709753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 697 sessionId:  describe result for acl check
2025-10-09T21:46:23.709795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:23.709800Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:23.709806Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:23.709813Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341655677638832:7202] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:23.710426Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341655677638832:7202] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:23.717102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341655677638832:7202] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:23.733743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341655677638832:7202] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:23.733767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341655677638832:7202] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:23.733771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341655677638832:7202] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:23.733784Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 697 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:23.779247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:23.781328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:23.781629Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:23.781642Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:23.781658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:23.781743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:23.781795Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341655315824574:6569]
2025-10-09T21:46:23.782844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:23.783929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:23.783998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:23.784345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:23.784430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:23.784432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:23.784433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:23.784435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:23.784436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:23.784438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:23.784439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.784440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.784441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.784448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:23.784461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:23.784464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:23.784468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:23.784472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:23.784476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:23.784480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.784484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.784488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.784496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:23.784504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:23.784510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:23.784516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:23.784520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:23.784526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:23.784531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:23.784536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:23.784544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:23.784549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:23.784555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:23.784560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:23.784565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.784571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.784576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.784580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.784590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.784595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.784622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:23.794332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:23.795861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:23.795880Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:23.795882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:23.795888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:23.795896Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 696 [2:7559341655315824574:6569]
2025-10-09T21:46:23.795913Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:23.795933Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:23.795936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:23.795939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:23.795942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:23.795963Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:23.795970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:23.795981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:23.796089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.796094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.796097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.796100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:23.796189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:23.796192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.796207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:23.796219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:23.796236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:23.796261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341655315824569:6569]
2025-10-09T21:46:23.796264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341655315824569:6569]
2025-10-09T21:46:23.796266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341655315824569:6569]
2025-10-09T21:46:23.796296Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:23.796322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:23.796340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:23.796344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:23.796380Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:23.796384Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:23.796386Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:23.796391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341655315824569:6569]
2025-10-09T21:46:23.796395Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:23.796475Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:23.796704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341655677638881:7202], now have 1 active actors on pipe
2025-10-09T21:46:23.796720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341655677638822:2432], now have 1 active actors on pipe
2025-10-09T21:46:23.796754Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 696
2025-10-09T21:46:23.796793Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 696
2025-10-09T21:46:23.796902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:23.796913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:23.796950Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6853df41-4b7bd763-dd093d80-904b4d4f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:23.796960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:23.796965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:23.796968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:23.796982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:23.796997Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:23.797016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:23.797319Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341655677639096:6527]
2025-10-09T21:46:23.797543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:23.797556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:23.797570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341655677639096:6527], now have 1 active actors on pipe
2025-10-09T21:46:23.797585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:23.797704Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 696, pipe: [1:7559341655677639096:6527]
2025-10-09T21:46:23.797742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 697 partition: 0 MaxSeqNo: 25 sessionId: src_id|6853df41-4b7bd763-dd093d80-904b4d4f_0
2025-10-09T21:46:23.798406Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27ba91a4-7377f2e9-82726873-8801551a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6853df41-4b7bd763-dd093d80-904b4d4f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:23.801311Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6853df41-4b7bd763-dd093d80-904b4d4f_0] Write session: try to update token
2025-10-09T21:46:23.801336Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6853df41-4b7bd763-dd093d80-904b4d4f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:23.829706Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:23.829723Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:23.829726Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:23.829728Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:23.829740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341655315824569:6569]
2025-10-09T21:46:23.829746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341655315824569:6569]
2025-10-09T21:46:23.829750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341655315824569:6569]
2025-10-09T21:46:23.829803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:23.829812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:23.829815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:23.829969Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:23.833570Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:23.833587Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:23.833592Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:23.834208Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 697 sessionId: src_id|6853df41-4b7bd763-dd093d80-904b4d4f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:23.836953Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:24.013387Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:24.015494Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:24.015513Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:24.015573Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 697 reason: error at write: Disconnected sessionId: src_id|6853df41-4b7bd763-dd093d80-904b4d4f_0
2025-10-09T21:46:24.015760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 697 sessionId: src_id|6853df41-4b7bd763-dd093d80-904b4d4f_0 is DEAD
2025-10-09T21:46:24.015957Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6853df41-4b7bd763-dd093d80-904b4d4f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:24.015971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6853df41-4b7bd763-dd093d80-904b4d4f_0] Write session will restart in 0.008581s
2025-10-09T21:46:24.015962Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:24.015997Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6853df41-4b7bd763-dd093d80-904b4d4f_0] Write session: Do CDS request
2025-10-09T21:46:24.016004Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6853df41-4b7bd763-dd093d80-904b4d4f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:24.014705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:24.014724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:24.016003Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341655677639096:6527]
2025-10-09T21:46:24.016554Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:24.025610Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6853df41-4b7bd763-dd093d80-904b4d4f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:24.025952Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:24.025974Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 698
2025-10-09T21:46:24.026318Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 698 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:24.026369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 698 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:24.026375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=698 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:24.026378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:24.027177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 698 sessionId:  describe result for acl check
2025-10-09T21:46:24.027219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:24.027222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:24.027224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:24.027230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341659972606416:7206] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:24.027767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341659972606416:7206] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:24.034583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341659972606416:7206] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:24.051189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341659972606416:7206] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:24.051213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341659972606416:7206] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:24.051218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341659972606416:7206] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:24.051232Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 698 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:24.088875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:24.093093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:24.093816Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:24.093822Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:24.093842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:24.093945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:24.093993Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341659610792139:6571]
2025-10-09T21:46:24.094660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:24.098922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:24.099003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:24.099371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:24.099450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:24.099452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:24.099453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:24.099455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:24.099456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:24.099457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:24.099458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:24.099459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:24.099461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:24.099467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:24.099477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:24.099481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:24.099484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:24.099488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:24.099491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:24.099495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:24.099498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:24.099501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:24.099509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:24.099517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:24.099523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:24.099528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:24.099534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:24.099538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:24.099543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:24.099547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:24.099552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:24.099557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:24.099562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:24.099566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:24.099574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:24.099579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:24.099584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:24.099589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:24.099593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:24.099600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:24.099629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:24.117560Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341659972606679:6527]
2025-10-09T21:46:24.118981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:24.120066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:24.120073Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:24.120076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:24.120081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:24.120089Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 697 [2:7559341659610792139:6571]
2025-10-09T21:46:24.120121Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:24.120141Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:24.120145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:24.120149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:24.120151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:24.120170Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:24.120178Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:24.120186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:24.120252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:24.120255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:24.120257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:24.120261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:24.120340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:24.120343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:24.120357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:24.120371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:24.120389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:24.120411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341659610792133:6571]
2025-10-09T21:46:24.120414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341659610792133:6571]
2025-10-09T21:46:24.120416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341659610792133:6571]
2025-10-09T21:46:24.120434Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:24.120451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:24.120467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:24.120470Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:24.120492Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:24.120511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341659610792133:6571]
2025-10-09T21:46:24.120513Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:24.120514Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:24.120517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:24.120524Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:24.120950Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 697
2025-10-09T21:46:24.120979Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 697
2025-10-09T21:46:24.121001Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 697, pipe: [1:7559341659972606679:6527]
2025-10-09T21:46:24.120824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341659972606523:7206], now have 1 active actors on pipe
2025-10-09T21:46:24.120833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341659972606402:2432], now have 1 active actors on pipe
2025-10-09T21:46:24.120836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341659972606679:6527], now have 1 active actors on pipe
2025-10-09T21:46:24.121290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:24.121296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:24.121329Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7559be7f-24e045d1-318854c8-a5799008_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:24.121337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:24.121341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:24.121344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:24.121355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:24.121367Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:24.121385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:24.123516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:24.123528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:24.123557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:24.123722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 698 partition: 0 MaxSeqNo: 25 sessionId: src_id|7559be7f-24e045d1-318854c8-a5799008_0
2025-10-09T21:46:24.124106Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6853df41-4b7bd763-dd093d80-904b4d4f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7559be7f-24e045d1-318854c8-a5799008_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:24.126805Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7559be7f-24e045d1-318854c8-a5799008_0] Write session: try to update token
2025-10-09T21:46:24.126827Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7559be7f-24e045d1-318854c8-a5799008_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:24.157542Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 698 sessionId: src_id|7559be7f-24e045d1-318854c8-a5799008_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:24.160058Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:24.161274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:24.161297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:24.162656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:24.162683Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:24.162686Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:24.162688Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:24.162690Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:24.162700Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341659610792133:6571]
2025-10-09T21:46:24.162705Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341659610792133:6571]
2025-10-09T21:46:24.162709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341659610792133:6571]
2025-10-09T21:46:24.162750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:24.162757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:24.162760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:24.162896Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:24.166426Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:24.166434Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:24.166438Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:24.167157Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:24.407772Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:24.443709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:24.443740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:24.443940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.444013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.444078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.444147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.444380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.444392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.444461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.444525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.444587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.445069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.445122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.445125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.445196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.445259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.445322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.445384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.445439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.445442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.445537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.445606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.445669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.445731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.445771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.445775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.445841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.445904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.445967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.446063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.446107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.446110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.446180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.446240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.446308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.446370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.446410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.446414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.446479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.446540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.446606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.446672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.446729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.446732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.446794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.446848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.446913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.446976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.447040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.447043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.447118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.447178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.447242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.447307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.447348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.447351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.447423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.447490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.447555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.447622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.447672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.447676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.448642Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:24.448664Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:24.448706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 698 reason: error at write: Disconnected sessionId: src_id|7559be7f-24e045d1-318854c8-a5799008_0
2025-10-09T21:46:24.449106Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7559be7f-24e045d1-318854c8-a5799008_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:24.449119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7559be7f-24e045d1-318854c8-a5799008_0] Write session will restart in 0.006909s
2025-10-09T21:46:24.449144Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7559be7f-24e045d1-318854c8-a5799008_0] Write session: Do CDS request
2025-10-09T21:46:24.449151Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7559be7f-24e045d1-318854c8-a5799008_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:24.448914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 698 sessionId: src_id|7559be7f-24e045d1-318854c8-a5799008_0 is DEAD
2025-10-09T21:46:24.449060Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:24.449068Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341659972606679:6527]
2025-10-09T21:46:24.449130Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:24.457691Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7559be7f-24e045d1-318854c8-a5799008_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:24.458697Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:24.458719Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 699
2025-10-09T21:46:24.458895Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 699 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:24.458934Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 699 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:24.458940Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=699 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:24.458945Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:24.460927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 699 sessionId:  describe result for acl check
2025-10-09T21:46:24.460976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:24.460978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:24.460980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:24.460986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341659972606700:7211] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:24.461567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341659972606700:7211] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:24.465491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341659972606700:7211] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:24.477712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341659972606700:7211] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:24.477740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341659972606700:7211] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:24.477745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341659972606700:7211] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:24.477756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 699 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:24.520497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:24.532320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:24.532638Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:24.532648Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:24.532674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:24.532747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:24.532790Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341659610792416:6573]
2025-10-09T21:46:24.533731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:24.534595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:24.534659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:24.535035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:24.535102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:24.535108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:24.535109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:24.535111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:24.535113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:24.535114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:24.535116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:24.535117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:24.535119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:24.535126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:24.535136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:24.535141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:24.535145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:24.535150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:24.535155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:24.535159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:24.535163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:24.535167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:24.535175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:24.535184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:24.535191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:24.535203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:24.535212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:24.535216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:24.535220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:24.535226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:24.535231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:24.535235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:24.535251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:24.535256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:24.535261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:24.535267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:24.535270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:24.535275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:24.535279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:24.535283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:24.535311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:24.548900Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341659972606964:6527]
2025-10-09T21:46:24.595441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:24.596526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:24.596542Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:24.596544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:24.596549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:24.596557Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 698 [2:7559341659610792416:6573]
2025-10-09T21:46:24.596572Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:24.596592Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:24.596595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:24.596598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:24.596601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:24.596620Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:24.596625Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:24.596633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:24.596698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:24.596701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:24.596703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:24.596706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:24.596781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:24.596784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:24.596797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:24.596807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:24.596823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:24.596845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341659610792410:6573]
2025-10-09T21:46:24.596848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341659610792410:6573]
2025-10-09T21:46:24.596850Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341659610792410:6573]
2025-10-09T21:46:24.596870Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:24.596886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:24.596901Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:24.596904Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:24.596933Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:24.596937Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:24.596939Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:24.596946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341659610792410:6573]
2025-10-09T21:46:24.596950Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:24.597060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:24.597431Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 698, pipe: [1:7559341659972606964:6527]
2025-10-09T21:46:24.597473Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 698
2025-10-09T21:46:24.597492Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 698
2025-10-09T21:46:24.597677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341659972606964:6527], now have 1 active actors on pipe
2025-10-09T21:46:24.597685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341659972606748:7211], now have 1 active actors on pipe
2025-10-09T21:46:24.597689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341659972606690:2432], now have 1 active actors on pipe
2025-10-09T21:46:24.597703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:24.597708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:24.597740Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|77adc470-978d5dea-2e50ea93-f9c89561_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:24.597749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:24.597754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:24.597756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:24.597768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:24.597781Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:24.597799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:24.603588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:24.603607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:24.603646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:24.603814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 699 partition: 0 MaxSeqNo: 25 sessionId: src_id|77adc470-978d5dea-2e50ea93-f9c89561_0
2025-10-09T21:46:24.604301Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7559be7f-24e045d1-318854c8-a5799008_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|77adc470-978d5dea-2e50ea93-f9c89561_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:24.607055Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|77adc470-978d5dea-2e50ea93-f9c89561_0] Write session: try to update token
2025-10-09T21:46:24.607079Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|77adc470-978d5dea-2e50ea93-f9c89561_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:24.632251Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:24.632266Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:24.632276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341659610792410:6573]
2025-10-09T21:46:24.632327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:24.632326Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:24.632502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:24.634354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:24.637674Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 699 sessionId: src_id|77adc470-978d5dea-2e50ea93-f9c89561_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:24.640164Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:24.642401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:24.642552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:24.642739Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:24.642810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:24.642824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:24.642850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:24.643480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:24.643676Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:24.648812Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:24.648828Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:24.648830Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:24.648832Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:24.648841Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341659610792410:6573]
2025-10-09T21:46:24.648845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341659610792410:6573]
2025-10-09T21:46:24.648848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341659610792410:6573]
2025-10-09T21:46:24.648897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:24.648908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:24.648910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:24.649047Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:24.651736Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:24.651744Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:24.651748Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:24.972860Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:24.989579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:24.989605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:24.989786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.989844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.989902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.989964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.990005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.990009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.990114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.990183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.990245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.990309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.990352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.990355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.990425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.990489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.990553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.990617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.990661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.990664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.990739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.990811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.990886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.990979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.991029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.991032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.991110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.991176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.991248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.991317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.991359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.991361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.991435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.991510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.991586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.991652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.991695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.991699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.991771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.991834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.991900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.991966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.992010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.992014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.992085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.992148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.992213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.992279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.992323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.992327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.992399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.992463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.992533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.992604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.994644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.994657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.994723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:24.994776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:24.994835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:24.994898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:24.994937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:24.994940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:24.997289Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:24.998746Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341659972606964:6527]
2025-10-09T21:46:24.998786Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:24.998829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 699 reason: status is not ok: tablet will be restarted right now sessionId: src_id|77adc470-978d5dea-2e50ea93-f9c89561_0
2025-10-09T21:46:24.999031Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 699 sessionId: src_id|77adc470-978d5dea-2e50ea93-f9c89561_0 is DEAD
2025-10-09T21:46:24.999160Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:24.999409Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|77adc470-978d5dea-2e50ea93-f9c89561_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:24.999417Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|77adc470-978d5dea-2e50ea93-f9c89561_0] Write session will restart in 0.008389s
2025-10-09T21:46:24.999442Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|77adc470-978d5dea-2e50ea93-f9c89561_0] Write session: Do CDS request
2025-10-09T21:46:24.999449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|77adc470-978d5dea-2e50ea93-f9c89561_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:25.008616Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|77adc470-978d5dea-2e50ea93-f9c89561_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:25.008961Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:25.008981Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 700
2025-10-09T21:46:25.009168Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 700 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:25.009201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 700 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:25.009209Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=700 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:25.009213Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:25.009648Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 700 sessionId:  describe result for acl check
2025-10-09T21:46:25.009680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:25.009683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:25.009685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:25.009691Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341664267574318:7223] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:25.010334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341664267574318:7223] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:25.020556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341664267574318:7223] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:25.030951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341664267574318:7223] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:25.030976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341664267574318:7223] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:25.030980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341664267574318:7223] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:25.030992Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 700 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:25.086892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:25.088040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:25.088211Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:25.088215Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:25.088227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:25.088271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:25.088308Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341663905760024:6589]
2025-10-09T21:46:25.088907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:25.089517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:25.089556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:25.090068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:25.090139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:25.090141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:25.090142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:25.090143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:25.090144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:25.090145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:25.090147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.090148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.090149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.090155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:25.090167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:25.090172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:25.090176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:25.090180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:25.090184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:25.090189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.090193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.090198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.090224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:25.090232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:25.090238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:25.090242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:25.090247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:25.090252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:25.090256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:25.090261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:25.090265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:25.090270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:25.090274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:25.090279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:25.090284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.090289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.090293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.090298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.090303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.090308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.090337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:25.102207Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341664267574582:6527]
2025-10-09T21:46:25.104380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:25.105523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:25.105530Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:25.105532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:25.105537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:25.105544Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 699 [2:7559341663905760024:6589]
2025-10-09T21:46:25.105561Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:25.105582Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:25.105586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:25.105590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:25.105592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:25.105612Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:25.105619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:25.105627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:25.105695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.105697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.105699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.105702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:25.106339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:25.106343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.106359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.106371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.106390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:25.106413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341663905760018:6589]
2025-10-09T21:46:25.106416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341663905760018:6589]
2025-10-09T21:46:25.106418Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341663905760018:6589]
2025-10-09T21:46:25.106434Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:25.106452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:25.106467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:25.106472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:25.106515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341663905760018:6589]
2025-10-09T21:46:25.106517Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:25.106908Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:25.106915Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:25.106917Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:25.106925Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:25.107756Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 699, pipe: [1:7559341664267574582:6527]
2025-10-09T21:46:25.107804Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 699
2025-10-09T21:46:25.107824Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 699
2025-10-09T21:46:25.107873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341664267574363:7223], now have 1 active actors on pipe
2025-10-09T21:46:25.107883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341659972607012:2432], now have 1 active actors on pipe
2025-10-09T21:46:25.107887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341664267574582:6527], now have 1 active actors on pipe
2025-10-09T21:46:25.108419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:25.108429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:25.108463Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e8c31a4d-2f4d5325-314cfbe1-7bcb5d31_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:25.108473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:25.108478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:25.108482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:25.108495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:25.108511Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:25.108531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:25.108811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:25.108817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:25.108831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:25.108991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 700 partition: 0 MaxSeqNo: 25 sessionId: src_id|e8c31a4d-2f4d5325-314cfbe1-7bcb5d31_0
2025-10-09T21:46:25.110293Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|77adc470-978d5dea-2e50ea93-f9c89561_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e8c31a4d-2f4d5325-314cfbe1-7bcb5d31_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:25.112685Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8c31a4d-2f4d5325-314cfbe1-7bcb5d31_0] Write session: try to update token
2025-10-09T21:46:25.112703Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8c31a4d-2f4d5325-314cfbe1-7bcb5d31_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:25.135033Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:25.135055Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:25.135057Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:25.135059Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:25.135069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341663905760018:6589]
2025-10-09T21:46:25.135074Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341663905760018:6589]
2025-10-09T21:46:25.135078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341663905760018:6589]
2025-10-09T21:46:25.135123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:25.135130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.135133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:25.135274Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:25.138523Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:25.138537Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:25.138541Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:25.174480Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 700 sessionId: src_id|e8c31a4d-2f4d5325-314cfbe1-7bcb5d31_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:25.177087Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:25.346965Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:25.347623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:25.347647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:25.347976Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:25.350285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8c31a4d-2f4d5325-314cfbe1-7bcb5d31_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:25.350298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8c31a4d-2f4d5325-314cfbe1-7bcb5d31_0] Write session will restart in 0.009794s
2025-10-09T21:46:25.350332Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8c31a4d-2f4d5325-314cfbe1-7bcb5d31_0] Write session: Do CDS request
2025-10-09T21:46:25.350339Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8c31a4d-2f4d5325-314cfbe1-7bcb5d31_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:25.348349Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:25.348439Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 700 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e8c31a4d-2f4d5325-314cfbe1-7bcb5d31_0
2025-10-09T21:46:25.348622Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 700 sessionId: src_id|e8c31a4d-2f4d5325-314cfbe1-7bcb5d31_0 is DEAD
2025-10-09T21:46:25.348767Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:25.348802Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341664267574582:6527]
2025-10-09T21:46:25.361621Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8c31a4d-2f4d5325-314cfbe1-7bcb5d31_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:25.361742Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:25.361763Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 701
2025-10-09T21:46:25.362269Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 701 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:25.362329Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 701 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:25.362340Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=701 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:25.362346Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:25.362882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 701 sessionId:  describe result for acl check
2025-10-09T21:46:25.362933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:25.362935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:25.362937Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:25.362943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341664267574612:7228] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:25.363540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341664267574612:7228] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:25.367641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341664267574612:7228] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:25.373787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341664267574612:7228] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:25.373808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341664267574612:7228] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:25.373813Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341664267574612:7228] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:25.373824Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 701 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:25.406242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:25.411325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:25.411567Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:25.411579Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:25.411592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:25.411653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:25.411700Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341663905760297:6591]
2025-10-09T21:46:25.412281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:25.417724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:25.417824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:25.418253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:25.418324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:25.418326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:25.418328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:25.418329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:25.418330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:25.418331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:25.418332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.418334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.418335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.418342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:25.418354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:25.418358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:25.418363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:25.418367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:25.418372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:25.418376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.418380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.418385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.418394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:25.418405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:25.418412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:25.418419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:25.418424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:25.418429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:25.418434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:25.418439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:25.418446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:25.418452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:25.418457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:25.418463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:25.418468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.418474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.418479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.418485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.418491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.418497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.418530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:25.429924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:25.431861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:25.431868Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:25.431871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:25.431877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:25.431885Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 700 [2:7559341663905760297:6591]
2025-10-09T21:46:25.431902Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:25.431924Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:25.431928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:25.431931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:25.431933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:25.431954Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:25.431960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:25.431968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:25.432041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.432043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.432046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.432048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:25.432133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:25.432136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.432151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.432162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.432177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:25.432199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341663905760292:6591]
2025-10-09T21:46:25.432202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341663905760292:6591]
2025-10-09T21:46:25.432204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341663905760292:6591]
2025-10-09T21:46:25.432223Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:25.432240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:25.432255Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:25.432258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:25.432300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341663905760292:6591]
2025-10-09T21:46:25.432303Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:25.432534Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:25.432537Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:25.432539Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:25.432546Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:25.433491Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 700
2025-10-09T21:46:25.433539Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 700
2025-10-09T21:46:25.433878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341664267574597:2432], now have 1 active actors on pipe
2025-10-09T21:46:25.433892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341664267574681:7228], now have 1 active actors on pipe
2025-10-09T21:46:25.433916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:25.433923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:25.433956Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7761ee18-bf8c25ef-a4ee8a41-6143ca81_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:25.433970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:25.433974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:25.433978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:25.433991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:25.434006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:25.434049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:25.434674Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 701 partition: 0 MaxSeqNo: 25 sessionId: src_id|7761ee18-bf8c25ef-a4ee8a41-6143ca81_0
2025-10-09T21:46:25.435097Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8c31a4d-2f4d5325-314cfbe1-7bcb5d31_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7761ee18-bf8c25ef-a4ee8a41-6143ca81_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:25.434536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:25.434546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:25.434563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:25.437665Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7761ee18-bf8c25ef-a4ee8a41-6143ca81_0] Write session: try to update token
2025-10-09T21:46:25.437687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7761ee18-bf8c25ef-a4ee8a41-6143ca81_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:25.449648Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341664267574876:6527]
2025-10-09T21:46:25.450347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341664267574876:6527], now have 1 active actors on pipe
2025-10-09T21:46:25.450449Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 700, pipe: [1:7559341664267574876:6527]
2025-10-09T21:46:25.455136Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:25.455155Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:25.455157Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:25.455159Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:25.455169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341663905760292:6591]
2025-10-09T21:46:25.455173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341663905760292:6591]
2025-10-09T21:46:25.455177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341663905760292:6591]
2025-10-09T21:46:25.455222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:25.455229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.455232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:25.455368Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:25.458088Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:25.458102Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:25.458106Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:25.484122Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 701 sessionId: src_id|7761ee18-bf8c25ef-a4ee8a41-6143ca81_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:25.488974Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:25.655255Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:25.655867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:25.655878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:25.656579Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341664267574876:6527]
2025-10-09T21:46:25.656620Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:25.656622Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:25.656680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 701 reason: error at write: Disconnected sessionId: src_id|7761ee18-bf8c25ef-a4ee8a41-6143ca81_0
2025-10-09T21:46:25.656861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 701 sessionId: src_id|7761ee18-bf8c25ef-a4ee8a41-6143ca81_0 is DEAD
2025-10-09T21:46:25.656920Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:25.656965Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:46:25.657094Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 7 consumer shared/user session shared/user_1_7_8824138098741025035_v1 is DEAD
2025-10-09T21:46:25.657246Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:46:25.657309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7761ee18-bf8c25ef-a4ee8a41-6143ca81_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:25.657322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7761ee18-bf8c25ef-a4ee8a41-6143ca81_0] Write session will restart in 0.007665s
2025-10-09T21:46:25.657346Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7761ee18-bf8c25ef-a4ee8a41-6143ca81_0] Write session: Do CDS request
2025-10-09T21:46:25.657352Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7761ee18-bf8c25ef-a4ee8a41-6143ca81_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:25.657406Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:46:25.657412Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:46:25.657419Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.005616s
2025-10-09T21:46:25.657445Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341518238662124:6522] disconnected.
2025-10-09T21:46:25.657459Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341518238662124:6522] disconnected; active server actors: 1
2025-10-09T21:46:25.657463Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341518238662124:6522] client user disconnected session shared/user_1_7_8824138098741025035_v1
2025-10-09T21:46:25.664207Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:25.664979Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:46:25.665497Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:46:25.665522Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 8
2025-10-09T21:46:25.665847Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 8 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:46:25.665906Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:46:25.666084Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 auth for : user
2025-10-09T21:46:25.666260Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 Handle describe topics response
2025-10-09T21:46:25.666285Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 auth is DEAD
2025-10-09T21:46:25.666304Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:46:25.666567Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:46:25.666672Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_8_189961622288699779_v1
2025-10-09T21:46:25.667239Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7761ee18-bf8c25ef-a4ee8a41-6143ca81_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:25.667276Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:25.667287Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 702
2025-10-09T21:46:25.667326Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341664267574894:7232] connected; active server actors: 1
2025-10-09T21:46:25.667355Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559341664267574894:7232] session shared/user_1_8_189961622288699779_v1
2025-10-09T21:46:25.667366Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:46:25.667378Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:46:25.667385Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_8_189961622288699779_v1" (Sender=[1:7559341664267574891:7232], Pipe=[1:7559341664267574894:7232], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:46:25.667618Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:46:25.667639Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:46:25.667650Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_8_189961622288699779_v1" (Sender=[1:7559341664267574891:7232], Pipe=[1:7559341664267574894:7232], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:46:25.667661Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_8_189961622288699779_v1" sender [1:7559341664267574891:7232] lock partition 0 for ReadingSession "shared/user_1_8_189961622288699779_v1" (Sender=[1:7559341664267574891:7232], Pipe=[1:7559341664267574894:7232], Partitions=[], ActiveFamilyCount=1) generation 1 step 8
2025-10-09T21:46:25.667678Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:46:25.667683Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000040s
2025-10-09T21:46:25.667708Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 702 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:25.667749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 702 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:25.667754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=702 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:25.667758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:25.667924Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 8 Session: "shared/user_1_8_189961622288699779_v1" ClientId: "user" PipeClient { RawX1: 7559341664267574894 RawX2: 4503603922345024 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:46:25.667941Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:46:25.669982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 702 sessionId:  describe result for acl check
2025-10-09T21:46:25.670060Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:25.670062Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:25.670064Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:25.670069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341664267574903:7235] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:25.670630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341664267574903:7235] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:25.678411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341664267574903:7235] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:25.691521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341664267574903:7235] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:25.691544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341664267574903:7235] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:25.691547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341664267574903:7235] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:25.691557Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 702 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:25.724074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:25.726693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:25.726956Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:25.726961Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:25.726977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:25.727051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:25.727090Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341663905760593:6601]
2025-10-09T21:46:25.727708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:25.728344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:25.728398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:25.728731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:25.728798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:25.728799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:25.728800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:25.728801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:25.728803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:25.728804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:25.728805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.728806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.728807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.728814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:25.728824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:25.728828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:25.728832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:25.728836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:25.728840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:25.728843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.728846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.728849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.728856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:25.728866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:25.728872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:25.728878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:25.728882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:25.728887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:25.728892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:25.728896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:25.728903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:25.728908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:25.728912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:25.728917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:25.728922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.728927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.728931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.728935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.728939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.728945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.728973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:25.741996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:25.743698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:25.743706Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:25.743708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:25.743712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:25.743720Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 701 [2:7559341663905760593:6601]
2025-10-09T21:46:25.743737Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:25.743757Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:25.743761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:25.743765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:25.743768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:25.743788Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:25.743794Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:25.743803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:25.743872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.743874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.743877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.743880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:25.743960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:25.743963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.743978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:25.743989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:25.744007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:25.744029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341663905760586:6601]
2025-10-09T21:46:25.744032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341663905760586:6601]
2025-10-09T21:46:25.744034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341663905760586:6601]
2025-10-09T21:46:25.744051Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:25.744069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:25.744085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:25.744088Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:25.744133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341663905760586:6601]
2025-10-09T21:46:25.744136Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:25.744430Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:25.744438Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:25.744440Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:25.744447Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:25.745404Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 701, pipe: [1:7559341664267574899:7236]
2025-10-09T21:46:25.745449Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 701
2025-10-09T21:46:25.745470Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 701
2025-10-09T21:46:25.749243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341664267574899:7236], now have 1 active actors on pipe
2025-10-09T21:46:25.749258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341664267575008:7235], now have 1 active actors on pipe
2025-10-09T21:46:25.749261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341664267574881:2432], now have 1 active actors on pipe
2025-10-09T21:46:25.749295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:25.749302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:25.749321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_8_189961622288699779_v1 on pipe: [1:7559341664267574899:7236]
2025-10-09T21:46:25.749345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:25.749347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:25.749370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:25.749378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:25.749381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:25.749407Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_8_189961622288699779_v1
2025-10-09T21:46:25.749409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:25.749471Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e9f512ab-8fcf06a0-17494d61-84632306_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:25.749479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:46:25.749486Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:46:25.749700Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_8_189961622288699779_v1:1 with generation 701
2025-10-09T21:46:25.755688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:46:25.755724Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:46:25.755737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:25.755745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:25.755749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:25.755764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:25.755782Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:25.755792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:46:25.755809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:46:25.755829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:25.756171Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046385742 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:46:25.756197Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:46:25.756269Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 sending to client partition status
2025-10-09T21:46:25.756907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:25.756922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:25.756950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:25.757139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 702 partition: 0 MaxSeqNo: 25 sessionId: src_id|e9f512ab-8fcf06a0-17494d61-84632306_0
2025-10-09T21:46:25.757550Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7761ee18-bf8c25ef-a4ee8a41-6143ca81_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e9f512ab-8fcf06a0-17494d61-84632306_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:25.760283Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9f512ab-8fcf06a0-17494d61-84632306_0] Write session: try to update token
2025-10-09T21:46:25.760304Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9f512ab-8fcf06a0-17494d61-84632306_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:25.783899Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:25.783922Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:25.783925Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:25.783927Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:25.783936Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341663905760586:6601]
2025-10-09T21:46:25.783941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341663905760586:6601]
2025-10-09T21:46:25.783944Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341663905760586:6601]
2025-10-09T21:46:25.783990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:25.783997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:25.784000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:25.784141Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:25.786330Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:25.786339Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:25.786342Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:25.789997Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 702 sessionId: src_id|e9f512ab-8fcf06a0-17494d61-84632306_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:25.792723Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:26.011142Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:26.011810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:26.011825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:26.012200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_8_189961622288699779_v1
2025-10-09T21:46:26.012220Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:26.012259Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_8_189961622288699779_v1
2025-10-09T21:46:26.012660Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341664267574899:7236]
2025-10-09T21:46:26.012702Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:26.012774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 702 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e9f512ab-8fcf06a0-17494d61-84632306_0
2025-10-09T21:46:26.012962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 702 sessionId: src_id|e9f512ab-8fcf06a0-17494d61-84632306_0 is DEAD
2025-10-09T21:46:26.013123Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:26.013365Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9f512ab-8fcf06a0-17494d61-84632306_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:26.013378Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9f512ab-8fcf06a0-17494d61-84632306_0] Write session will restart in 0.007301s
2025-10-09T21:46:26.013407Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9f512ab-8fcf06a0-17494d61-84632306_0] Write session: Do CDS request
2025-10-09T21:46:26.013413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9f512ab-8fcf06a0-17494d61-84632306_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:26.021611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9f512ab-8fcf06a0-17494d61-84632306_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:26.021708Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:26.021730Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 703
2025-10-09T21:46:26.023055Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 703 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:26.023118Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 703 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:26.023124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=703 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:26.023129Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:26.023582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 703 sessionId:  describe result for acl check
2025-10-09T21:46:26.023620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:26.023622Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:26.023624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:26.023631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341668562542507:7248] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:26.024302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341668562542507:7248] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:26.034761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341668562542507:7248] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:26.062816Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341668562542507:7248] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:26.062842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341668562542507:7248] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:26.062847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341668562542507:7248] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:26.062858Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 703 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:26.112458Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341668562542770:7236]
2025-10-09T21:46:26.115715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:26.123350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:26.123657Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:26.123662Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:26.123677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:26.123743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:26.123786Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341668200728189:6609]
2025-10-09T21:46:26.124411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:26.130265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:26.130378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:26.130824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:26.130891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:26.130893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:26.130894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:26.130896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:26.130897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:26.130898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:26.130900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:26.130901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:26.130902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:26.130910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:26.130923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:26.130927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:26.130931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:26.130936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:26.130941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:26.130945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:26.130949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:26.130954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:26.130964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:26.130975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:26.130983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:26.130989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:26.130995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:26.131001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:26.131006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:26.131012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:26.131017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:26.131022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:26.131026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:26.131031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:26.131041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:26.131047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:26.131052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:26.131059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:26.131063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:26.131069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:26.131099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:26.139764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:26.141350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:26.141367Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:26.141370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:26.141376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:26.141386Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 702 [2:7559341668200728189:6609]
2025-10-09T21:46:26.141403Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:26.141424Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:26.141427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:26.141430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:26.141433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:26.141454Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:26.141460Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:26.141467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:26.141542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:26.141545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:26.141547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:26.141550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:26.141629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:26.141637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:26.141651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:26.141668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:26.141686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:26.141713Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341668200728182:6609]
2025-10-09T21:46:26.141715Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341668200728182:6609]
2025-10-09T21:46:26.141718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341668200728182:6609]
2025-10-09T21:46:26.141727Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:26.141744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:26.141760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:26.141763Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:26.141784Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:26.141789Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:26.141791Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:26.141813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341668200728182:6609]
2025-10-09T21:46:26.141816Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:26.141818Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:26.141827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341668562542498:2432], now have 1 active actors on pipe
2025-10-09T21:46:26.141831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341668562542770:7236], now have 1 active actors on pipe
2025-10-09T21:46:26.141853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341668562542581:7248], now have 1 active actors on pipe
2025-10-09T21:46:26.141907Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 702
2025-10-09T21:46:26.141955Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 702, pipe: [1:7559341668562542770:7236]
2025-10-09T21:46:26.141980Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 702
2025-10-09T21:46:26.142691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:26.142704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:26.142741Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|efa7c3c3-faed7726-38c463ed-a08f87_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:26.142751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:26.142756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:26.142759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:26.142771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:26.142784Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:26.142806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:26.143515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:26.143523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:26.143543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:26.143686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 703 partition: 0 MaxSeqNo: 25 sessionId: src_id|efa7c3c3-faed7726-38c463ed-a08f87_0
2025-10-09T21:46:26.144352Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9f512ab-8fcf06a0-17494d61-84632306_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|efa7c3c3-faed7726-38c463ed-a08f87_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:26.147066Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|efa7c3c3-faed7726-38c463ed-a08f87_0] Write session: try to update token
2025-10-09T21:46:26.147090Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|efa7c3c3-faed7726-38c463ed-a08f87_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:26.162083Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:26.162098Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:26.162101Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:26.162103Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:26.162113Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341668200728182:6609]
2025-10-09T21:46:26.162119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341668200728182:6609]
2025-10-09T21:46:26.162123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341668200728182:6609]
2025-10-09T21:46:26.162170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:26.162178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:26.162180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:26.162329Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:26.166496Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:26.166508Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:26.166512Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:26.193358Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 703 sessionId: src_id|efa7c3c3-faed7726-38c463ed-a08f87_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:26.196033Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:26.433666Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:26.437264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:26.437284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:26.438059Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:26.438121Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341668562542770:7236]
2025-10-09T21:46:26.438147Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:26.438151Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:26.438205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 703 reason: error at write: Disconnected sessionId: src_id|efa7c3c3-faed7726-38c463ed-a08f87_0
2025-10-09T21:46:26.438384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 703 sessionId: src_id|efa7c3c3-faed7726-38c463ed-a08f87_0 is DEAD
2025-10-09T21:46:26.438778Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efa7c3c3-faed7726-38c463ed-a08f87_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:26.438790Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efa7c3c3-faed7726-38c463ed-a08f87_0] Write session will restart in 0.006529s
2025-10-09T21:46:26.438814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efa7c3c3-faed7726-38c463ed-a08f87_0] Write session: Do CDS request
2025-10-09T21:46:26.438821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efa7c3c3-faed7726-38c463ed-a08f87_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:26.440856Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:26.446689Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|efa7c3c3-faed7726-38c463ed-a08f87_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:26.446795Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:26.446818Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 704
2025-10-09T21:46:26.446992Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 704 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:26.447027Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 704 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:26.447034Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=704 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:26.447038Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:26.447523Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 704 sessionId:  describe result for acl check
2025-10-09T21:46:26.447563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:26.447565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:26.447567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:26.447574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341668562542802:7253] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:26.448129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341668562542802:7253] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:26.467923Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341668562542802:7253] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:26.481508Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341668562542802:7253] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:26.481529Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341668562542802:7253] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:26.481533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341668562542802:7253] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:26.481543Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 704 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:26.525680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:26.525879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:26.526089Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:26.526095Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:26.526109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:26.526169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:26.526212Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341668200728470:6611]
2025-10-09T21:46:26.526865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:26.527680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:26.527739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:26.528082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:26.528154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:26.528161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:26.528163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:26.528164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:26.528166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:26.528167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:26.528169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:26.528170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:26.528171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:26.528177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:26.528189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:26.528193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:26.528198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:26.528202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:26.528207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:26.528211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:26.528216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:26.528220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:26.528228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:26.528239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:26.528250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:26.528257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:26.528267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:26.528272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:26.528277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:26.528283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:26.528288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:26.528299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:26.528304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:26.528315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:26.528321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:26.528331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:26.528336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:26.528341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:26.528349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:26.528357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:26.528394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:26.537648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:26.539334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:26.539352Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:26.539355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:26.539360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:26.539367Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 703 [2:7559341668200728470:6611]
2025-10-09T21:46:26.539385Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:26.539406Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:26.539416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:26.539419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:26.539422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:26.539441Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:26.539452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:26.539461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:26.539531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:26.539539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:26.539541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:26.539544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:26.539624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:26.539632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:26.539647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:26.539660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:26.539682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:26.539707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341668200728465:6611]
2025-10-09T21:46:26.539714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341668200728465:6611]
2025-10-09T21:46:26.539717Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341668200728465:6611]
2025-10-09T21:46:26.539730Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:26.539748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:26.539771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:26.539780Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:26.539818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341668200728465:6611]
2025-10-09T21:46:26.539825Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:26.538587Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341668562543068:7236]
2025-10-09T21:46:26.540317Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:26.540327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:26.540330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:26.540337Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:26.541047Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 703
2025-10-09T21:46:26.541083Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 703
2025-10-09T21:46:26.541097Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 703, pipe: [1:7559341668562543068:7236]
2025-10-09T21:46:26.545328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341668562542791:2432], now have 1 active actors on pipe
2025-10-09T21:46:26.545345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341668562542914:7253], now have 1 active actors on pipe
2025-10-09T21:46:26.545349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341668562543068:7236], now have 1 active actors on pipe
2025-10-09T21:46:26.545380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:26.545387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:26.545426Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9aa18654-3aa5c2c6-3092b68-e12d02bd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:26.545443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:26.545453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:26.545456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:26.545469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:26.545503Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:26.545523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:26.548157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:26.548171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:26.548220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:26.548879Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 704 partition: 0 MaxSeqNo: 25 sessionId: src_id|9aa18654-3aa5c2c6-3092b68-e12d02bd_0
2025-10-09T21:46:26.549368Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efa7c3c3-faed7726-38c463ed-a08f87_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9aa18654-3aa5c2c6-3092b68-e12d02bd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:26.551934Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9aa18654-3aa5c2c6-3092b68-e12d02bd_0] Write session: try to update token
2025-10-09T21:46:26.551952Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9aa18654-3aa5c2c6-3092b68-e12d02bd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:26.580190Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:26.580209Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:26.580212Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:26.580214Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:26.580224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341668200728465:6611]
2025-10-09T21:46:26.580228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341668200728465:6611]
2025-10-09T21:46:26.580232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341668200728465:6611]
2025-10-09T21:46:26.580278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:26.580285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:26.580288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:26.580432Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:26.582228Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:26.582238Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:26.582242Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:26.585644Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 704 sessionId: src_id|9aa18654-3aa5c2c6-3092b68-e12d02bd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:26.588216Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:26.944694Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:26.945370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:26.945381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:26.947095Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:26.947160Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:26.947163Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:26.947230Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 704 reason: error at write: Disconnected sessionId: src_id|9aa18654-3aa5c2c6-3092b68-e12d02bd_0
2025-10-09T21:46:26.947435Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 704 sessionId: src_id|9aa18654-3aa5c2c6-3092b68-e12d02bd_0 is DEAD
2025-10-09T21:46:26.947630Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341668562543068:7236]
2025-10-09T21:46:26.948416Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:26.950354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9aa18654-3aa5c2c6-3092b68-e12d02bd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:26.950362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9aa18654-3aa5c2c6-3092b68-e12d02bd_0] Write session will restart in 0.007946s
2025-10-09T21:46:26.950388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9aa18654-3aa5c2c6-3092b68-e12d02bd_0] Write session: Do CDS request
2025-10-09T21:46:26.950398Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9aa18654-3aa5c2c6-3092b68-e12d02bd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:26.962760Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9aa18654-3aa5c2c6-3092b68-e12d02bd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:26.963115Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:26.963137Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 705
2025-10-09T21:46:26.963325Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 705 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:26.963358Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 705 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:26.963365Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=705 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:26.963369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:26.963876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 705 sessionId:  describe result for acl check
2025-10-09T21:46:26.963918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:26.963920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:26.963922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:26.963928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341668562543124:7267] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:26.964508Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341668562543124:7267] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:26.967116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341668562543124:7267] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:26.975975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341668562543124:7267] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:26.975996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341668562543124:7267] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:26.976000Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341668562543124:7267] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:26.976010Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 705 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:27.030828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:27.031050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:27.031538Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:27.031543Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:27.031559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:27.031626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:27.031665Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341672495696081:6627]
2025-10-09T21:46:27.032237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:27.033413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:27.033484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:27.033962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:27.034061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:27.034063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:27.034064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:27.034066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:27.034067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:27.034069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:27.034071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.034073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.034074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.034081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:27.034092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:27.034096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:27.034100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:27.034104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:27.034107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:27.034110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.034114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.034118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.034127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:27.034134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:27.034142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:27.034147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:27.034151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:27.034157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:27.034161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:27.034166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:27.034171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:27.034177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:27.034182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:27.034187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:27.034192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.034197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.034202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.034207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.034212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.034216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.034273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:27.047715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:27.049596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:27.049603Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:27.049606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:27.049612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:27.049621Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 704 [2:7559341672495696081:6627]
2025-10-09T21:46:27.049638Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:27.049660Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:27.049665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:27.049668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:27.049670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:27.049690Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:27.049697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:27.049704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:27.049771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.049773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.049775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.049778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:27.049868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:27.049871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.049887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.049898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.049915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:27.049938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341672495696076:6627]
2025-10-09T21:46:27.049941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341672495696076:6627]
2025-10-09T21:46:27.049943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341672495696076:6627]
2025-10-09T21:46:27.049952Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:27.049969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:27.049974Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:27.049977Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:27.049980Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:27.049983Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:27.049986Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:27.049999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341668562543111:2432], now have 1 active actors on pipe
2025-10-09T21:46:27.050002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341668562543158:7267], now have 1 active actors on pipe
2025-10-09T21:46:27.050552Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341672857510689:7236]
2025-10-09T21:46:27.050607Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 704
2025-10-09T21:46:27.050636Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 704
2025-10-09T21:46:27.051549Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 704, pipe: [1:7559341672857510689:7236]
2025-10-09T21:46:27.053739Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341672495696076:6627]
2025-10-09T21:46:27.053766Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:27.053798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:27.053803Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:27.053808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:27.053835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341672857510689:7236], now have 1 active actors on pipe
2025-10-09T21:46:27.053911Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1b38beb4-c977ae1-52f94f2d-e558eda1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:27.053926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:27.053931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:27.053934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:27.053949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:27.053967Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:27.053993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:27.055041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:27.055053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:27.055079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:27.055276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 705 partition: 0 MaxSeqNo: 25 sessionId: src_id|1b38beb4-c977ae1-52f94f2d-e558eda1_0
2025-10-09T21:46:27.058325Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9aa18654-3aa5c2c6-3092b68-e12d02bd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1b38beb4-c977ae1-52f94f2d-e558eda1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:27.061028Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b38beb4-c977ae1-52f94f2d-e558eda1_0] Write session: try to update token
2025-10-09T21:46:27.061053Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b38beb4-c977ae1-52f94f2d-e558eda1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:27.094581Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:27.094599Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:27.094601Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:27.094603Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:27.094613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341672495696076:6627]
2025-10-09T21:46:27.094617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341672495696076:6627]
2025-10-09T21:46:27.094620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341672495696076:6627]
2025-10-09T21:46:27.094661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:27.094669Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:27.094670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:27.094673Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:27.094677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.094680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:27.094818Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:27.104233Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 705 sessionId: src_id|1b38beb4-c977ae1-52f94f2d-e558eda1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:27.106694Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:27.278730Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:27.279043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:27.279059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:27.279433Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:27.279606Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:27.280279Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b38beb4-c977ae1-52f94f2d-e558eda1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:27.280291Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b38beb4-c977ae1-52f94f2d-e558eda1_0] Write session will restart in 0.007714s
2025-10-09T21:46:27.279682Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341672857510689:7236]
2025-10-09T21:46:27.279717Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:27.280323Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b38beb4-c977ae1-52f94f2d-e558eda1_0] Write session: Do CDS request
2025-10-09T21:46:27.279791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 705 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1b38beb4-c977ae1-52f94f2d-e558eda1_0
2025-10-09T21:46:27.279991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 705 sessionId: src_id|1b38beb4-c977ae1-52f94f2d-e558eda1_0 is DEAD
2025-10-09T21:46:27.280332Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b38beb4-c977ae1-52f94f2d-e558eda1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:27.289620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b38beb4-c977ae1-52f94f2d-e558eda1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:27.289822Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:27.289845Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 706
2025-10-09T21:46:27.290094Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 706 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:27.290136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 706 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:27.290143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=706 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:27.290148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:27.290641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 706 sessionId:  describe result for acl check
2025-10-09T21:46:27.290677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:27.290678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:27.290680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:27.290685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341672857510711:7272] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:27.291275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341672857510711:7272] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:27.293641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341672857510711:7272] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:27.306384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341672857510711:7272] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:27.306409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341672857510711:7272] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:27.306414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341672857510711:7272] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:27.306424Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 706 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:27.343859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:27.352286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:27.352583Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:27.352588Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:27.352603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:27.352690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:27.352732Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341672495696359:6629]
2025-10-09T21:46:27.353320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:27.354227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:27.354280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:27.354624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:27.354697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:27.354698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:27.354700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:27.354701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:27.354703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:27.354704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:27.354705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.354706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.354707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.354713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:27.354724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:27.354727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:27.354730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:27.354734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:27.354737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:27.354742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.354745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.354748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.354756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:27.354764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:27.354770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:27.354776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:27.354780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:27.354786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:27.354791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:27.354796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:27.354800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:27.354805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:27.354811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:27.354816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:27.354821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.354827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.354831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.354837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.354842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.354846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.354874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:27.364196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:27.365804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:27.365810Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:27.365813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:27.365817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:27.365826Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 705 [2:7559341672495696359:6629]
2025-10-09T21:46:27.365841Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:27.365861Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:27.365865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:27.365868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:27.365870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:27.365889Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:27.365894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:27.365902Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:27.365965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.365968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.365970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.365973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:27.369528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:27.369541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.369559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.369572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.369593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:27.369634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341672857510769:7272], now have 1 active actors on pipe
2025-10-09T21:46:27.369638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341672857510703:2432], now have 1 active actors on pipe
2025-10-09T21:46:27.369647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341672495696353:6629]
2025-10-09T21:46:27.369649Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341672495696353:6629]
2025-10-09T21:46:27.369651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341672495696353:6629]
2025-10-09T21:46:27.369665Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:27.369686Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:27.369703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:27.369707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:27.369752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341672495696353:6629]
2025-10-09T21:46:27.369755Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:27.369821Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:27.369826Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:27.369828Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:27.369836Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:27.371408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:27.371422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:27.371456Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|18c91a10-17b21adb-685a11cb-c5daeeda_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:27.371474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:27.371479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:27.371482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:27.371499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:27.371522Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:27.371549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:27.370706Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 705
2025-10-09T21:46:27.370745Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 705
2025-10-09T21:46:27.374715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:27.374730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:27.374764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:27.374953Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 706 partition: 0 MaxSeqNo: 25 sessionId: src_id|18c91a10-17b21adb-685a11cb-c5daeeda_0
2025-10-09T21:46:27.377116Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b38beb4-c977ae1-52f94f2d-e558eda1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|18c91a10-17b21adb-685a11cb-c5daeeda_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:27.379733Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341672857510986:7236]
2025-10-09T21:46:27.380090Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|18c91a10-17b21adb-685a11cb-c5daeeda_0] Write session: try to update token
2025-10-09T21:46:27.380111Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|18c91a10-17b21adb-685a11cb-c5daeeda_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:27.380890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341672857510986:7236], now have 1 active actors on pipe
2025-10-09T21:46:27.381038Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 705, pipe: [1:7559341672857510986:7236]
2025-10-09T21:46:27.410108Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 706 sessionId: src_id|18c91a10-17b21adb-685a11cb-c5daeeda_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:27.412681Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:27.413766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:27.413786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:27.413828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:27.413973Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:27.430469Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:27.430486Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:27.430488Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:27.430490Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:27.430500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341672495696353:6629]
2025-10-09T21:46:27.430504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341672495696353:6629]
2025-10-09T21:46:27.430507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341672495696353:6629]
2025-10-09T21:46:27.430553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:27.430561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.430564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:27.430696Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:27.434516Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:27.434524Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:27.434528Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:27.818679Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:27.844406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:27.844440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:27.844632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:27.844691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:27.844752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:27.844814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:27.844857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:27.844862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:27.844936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:27.844993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:27.845059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:27.845125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:27.845168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:27.845171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:27.845241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:27.845298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:27.845360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:27.845423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:27.845465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:27.845468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:27.845542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:27.845604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:27.845668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:27.845737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:27.845781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:27.845784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:27.845852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:27.845914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:27.845980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:27.850146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:27.850200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:27.850204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:27.850279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:27.850339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:27.850409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:27.850466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:27.850506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:27.850510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:27.850575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:27.850632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:27.850702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:27.850758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:27.850799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:27.850802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:27.850868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:27.850923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:27.850980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:27.851042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:27.851093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:27.851096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:27.851184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:27.851240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:27.851299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:27.853933Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:27.853950Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:27.853983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 706 reason: error at write: Disconnected sessionId: src_id|18c91a10-17b21adb-685a11cb-c5daeeda_0
2025-10-09T21:46:27.854096Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341672857510986:7236]
2025-10-09T21:46:27.854115Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:27.851383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:27.851422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:27.851428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:27.851495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:27.851556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:27.851618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:27.851678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:27.851726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:27.851730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:27.854585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 706 sessionId: src_id|18c91a10-17b21adb-685a11cb-c5daeeda_0 is DEAD
2025-10-09T21:46:27.857080Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18c91a10-17b21adb-685a11cb-c5daeeda_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:27.857093Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18c91a10-17b21adb-685a11cb-c5daeeda_0] Write session will restart in 0.005908s
2025-10-09T21:46:27.857116Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18c91a10-17b21adb-685a11cb-c5daeeda_0] Write session: Do CDS request
2025-10-09T21:46:27.857123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18c91a10-17b21adb-685a11cb-c5daeeda_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:27.859101Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:27.867453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|18c91a10-17b21adb-685a11cb-c5daeeda_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:27.867833Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:27.867853Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 707
2025-10-09T21:46:27.868028Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 707 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:27.868063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 707 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:27.868071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=707 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:27.868075Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:27.869156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 707 sessionId:  describe result for acl check
2025-10-09T21:46:27.869201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:27.869203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:27.869205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:27.869213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341672857511011:7277] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:27.869835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341672857511011:7277] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:27.880515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341672857511011:7277] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:27.888294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341672857511011:7277] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:27.888325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341672857511011:7277] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:27.888329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341672857511011:7277] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:27.888338Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 707 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:27.923911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:27.931267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:27.931650Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:27.931655Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:27.931672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:27.931766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:27.931821Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341672495696657:6633]
2025-10-09T21:46:27.932392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:27.933001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:27.933048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:27.933399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:27.933448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:27.933449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:27.933451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:27.933452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:27.933454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:27.933455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:27.933457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.933458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.933459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.933466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:27.933477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:27.933482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:27.933486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:27.933491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:27.933495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:27.933500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.933504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.933509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.933516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:27.933525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:27.933537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:27.933544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:27.933549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:27.933554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:27.933559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:27.933565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:27.933570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:27.933576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:27.933580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:27.933586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:27.933603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.933609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.933614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.933620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.933626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.933632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.933660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:27.947948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:27.949063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:27.949070Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:27.949073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:27.949077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:27.949085Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 706 [2:7559341672495696657:6633]
2025-10-09T21:46:27.949100Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:27.949121Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:27.949124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:27.949127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:27.949129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:27.949149Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:27.949155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:27.949163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:27.949226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.949228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.949233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.949236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:27.949321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:27.949324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:27.949337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:27.949350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:27.949368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:27.949391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341672495696651:6633]
2025-10-09T21:46:27.949394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341672495696651:6633]
2025-10-09T21:46:27.949397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341672495696651:6633]
2025-10-09T21:46:27.949414Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:27.949429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:27.949445Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:27.949448Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:27.949490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341672495696651:6633]
2025-10-09T21:46:27.949493Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:27.949603Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:27.949607Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:27.949609Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:27.949615Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:27.950329Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 706
2025-10-09T21:46:27.950366Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 706
2025-10-09T21:46:27.952788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341672857511160:7277], now have 1 active actors on pipe
2025-10-09T21:46:27.952804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341672857510997:2432], now have 1 active actors on pipe
2025-10-09T21:46:27.952832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:27.952838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:27.952874Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8bd87b2e-83981b57-b3e33c38-432c03e6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:27.952886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:27.952892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:27.952894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:27.952907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:27.952924Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:27.952941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:27.955191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:27.955207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:27.955226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341672857511293:7236], now have 1 active actors on pipe
2025-10-09T21:46:27.955252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:27.954163Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341672857511293:7236]
2025-10-09T21:46:27.955070Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 706, pipe: [1:7559341672857511293:7236]
2025-10-09T21:46:27.955455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 707 partition: 0 MaxSeqNo: 25 sessionId: src_id|8bd87b2e-83981b57-b3e33c38-432c03e6_0
2025-10-09T21:46:27.958354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18c91a10-17b21adb-685a11cb-c5daeeda_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8bd87b2e-83981b57-b3e33c38-432c03e6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:27.960865Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8bd87b2e-83981b57-b3e33c38-432c03e6_0] Write session: try to update token
2025-10-09T21:46:27.960884Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8bd87b2e-83981b57-b3e33c38-432c03e6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:27.990539Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:27.990640Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:27.990656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341672495696651:6633]
2025-10-09T21:46:27.990725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:27.990904Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:27.991050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:27.991198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:27.991335Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:27.991523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:27.991561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:27.991809Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:28.005377Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 707 sessionId: src_id|8bd87b2e-83981b57-b3e33c38-432c03e6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:28.010748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:28.010786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:28.008181Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:28.012318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:28.012355Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:28.012358Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:28.012360Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:28.012363Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:28.012373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341672495696651:6633]
2025-10-09T21:46:28.012377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341672495696651:6633]
2025-10-09T21:46:28.012385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341672495696651:6633]
2025-10-09T21:46:28.012431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:28.012439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:28.012448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:28.012482Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:28.012489Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:28.012493Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:28.012594Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:28.014152Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:28.300563Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:28.330659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:28.330691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:28.330886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:28.330947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:28.331009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:28.331074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:28.331120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:28.331126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:28.331203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:28.331260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:28.331322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:28.331389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:28.331437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:28.331440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:28.331508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:28.331566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:28.331632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:28.331694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:28.331734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:28.331738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:28.331812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:28.331881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:28.331945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:28.332009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:28.332048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:28.332051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:28.332116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:28.332175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:28.332241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:28.332302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:28.332343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:28.332349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:28.332416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:28.332473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:28.332535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:28.332598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:28.332738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:28.332741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:28.332808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:28.332867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:28.332933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:28.332996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:28.333037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:28.333040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:28.333108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:28.333168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:28.333230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:28.333292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:28.333335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:28.333338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:28.333408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:28.333470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:28.333537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:28.333601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:28.333791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:28.333795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:28.333868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:28.333927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:28.334023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:28.340755Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:28.340772Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:28.340816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 707 reason: error at write: Disconnected sessionId: src_id|8bd87b2e-83981b57-b3e33c38-432c03e6_0
2025-10-09T21:46:28.341000Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:28.341042Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 707 sessionId: src_id|8bd87b2e-83981b57-b3e33c38-432c03e6_0 is DEAD
2025-10-09T21:46:28.341057Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341672857511293:7236]
2025-10-09T21:46:28.342260Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8bd87b2e-83981b57-b3e33c38-432c03e6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:28.342272Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8bd87b2e-83981b57-b3e33c38-432c03e6_0] Write session will restart in 0.006796s
2025-10-09T21:46:28.342298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8bd87b2e-83981b57-b3e33c38-432c03e6_0] Write session: Do CDS request
2025-10-09T21:46:28.342304Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8bd87b2e-83981b57-b3e33c38-432c03e6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:28.340066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:28.340159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:28.340163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:28.341653Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:28.350554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8bd87b2e-83981b57-b3e33c38-432c03e6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:28.354165Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:28.354186Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 708
2025-10-09T21:46:28.354461Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 708 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:28.354494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 708 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:28.354501Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=708 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:28.354505Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:28.354951Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 708 sessionId:  describe result for acl check
2025-10-09T21:46:28.354986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:28.354988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:28.354990Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:28.354995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341677152478627:7289] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:28.355524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341677152478627:7289] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:28.365397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341677152478627:7289] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:28.372521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341677152478627:7289] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:28.372544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341677152478627:7289] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:28.372548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341677152478627:7289] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:28.372558Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 708 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:28.409054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:28.414203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:28.414466Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:28.414471Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:28.414486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:28.414563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:28.414605Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341676790664258:6647]
2025-10-09T21:46:28.415237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:28.415876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:28.415926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:28.416260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:28.416329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:28.416330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:28.416331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:28.416333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:28.416334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:28.416335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:28.416336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:28.416337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:28.416338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:28.416344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:28.416355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:28.416359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:28.416362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:28.416366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:28.416371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:28.416374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:28.416377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:28.416380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:28.416389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:28.416397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:28.416403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:28.416408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:28.416413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:28.416417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:28.416422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:28.416427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:28.416433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:28.416438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:28.416442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:28.416446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:28.416452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:28.416456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:28.416460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:28.416465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:28.416470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:28.416475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:28.416503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:28.424175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:28.425960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:28.425967Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:28.425971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:28.425975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:28.425983Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 707 [2:7559341676790664258:6647]
2025-10-09T21:46:28.425998Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:28.426048Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:28.426052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:28.426057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:28.426059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:28.426080Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:28.426085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:28.426094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:28.426169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:28.426171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:28.426173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:28.426176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:28.426256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:28.426268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:28.426284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:28.426295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:28.426323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:28.426346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341676790664253:6647]
2025-10-09T21:46:28.426356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341676790664253:6647]
2025-10-09T21:46:28.426358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341676790664253:6647]
2025-10-09T21:46:28.426376Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:28.426392Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:28.426414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:28.426420Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:28.426438Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:28.426441Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:28.426443Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:28.426464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341676790664253:6647]
2025-10-09T21:46:28.426473Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:28.426539Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:28.426612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341677152478689:7289], now have 1 active actors on pipe
2025-10-09T21:46:28.426619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341677152478615:2432], now have 1 active actors on pipe
2025-10-09T21:46:28.427065Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 707
2025-10-09T21:46:28.427105Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 707
2025-10-09T21:46:28.427192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:28.427205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:28.427251Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4a34bd7f-7804399f-453f9454-bc15567f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:28.427266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:28.427284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:28.427287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:28.427299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:28.427313Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:28.427330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:28.427817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 708 partition: 0 MaxSeqNo: 25 sessionId: src_id|4a34bd7f-7804399f-453f9454-bc15567f_0
2025-10-09T21:46:28.428253Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8bd87b2e-83981b57-b3e33c38-432c03e6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4a34bd7f-7804399f-453f9454-bc15567f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:28.427534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:28.427544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:28.427575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:28.430792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a34bd7f-7804399f-453f9454-bc15567f_0] Write session: try to update token
2025-10-09T21:46:28.430813Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a34bd7f-7804399f-453f9454-bc15567f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:28.440618Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341677152478901:7236]
2025-10-09T21:46:28.442387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341677152478901:7236], now have 1 active actors on pipe
2025-10-09T21:46:28.442864Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 707, pipe: [1:7559341677152478901:7236]
2025-10-09T21:46:28.451870Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 708 sessionId: src_id|4a34bd7f-7804399f-453f9454-bc15567f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:28.448537Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:28.448556Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:28.448560Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:28.448562Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:28.448572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341676790664253:6647]
2025-10-09T21:46:28.448577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341676790664253:6647]
2025-10-09T21:46:28.448581Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341676790664253:6647]
2025-10-09T21:46:28.448631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:28.448638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:28.448638Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:28.448641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:28.448645Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:28.448648Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:28.448785Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:28.455094Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:28.687694Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:28.687997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:28.688015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:28.688340Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:28.689062Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a34bd7f-7804399f-453f9454-bc15567f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:28.689074Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a34bd7f-7804399f-453f9454-bc15567f_0] Write session will restart in 0.007125s
2025-10-09T21:46:28.689100Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a34bd7f-7804399f-453f9454-bc15567f_0] Write session: Do CDS request
2025-10-09T21:46:28.689106Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a34bd7f-7804399f-453f9454-bc15567f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:28.688644Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:28.688661Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:28.688719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 708 reason: error at write: Disconnected sessionId: src_id|4a34bd7f-7804399f-453f9454-bc15567f_0
2025-10-09T21:46:28.688903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 708 sessionId: src_id|4a34bd7f-7804399f-453f9454-bc15567f_0 is DEAD
2025-10-09T21:46:28.689038Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341677152478901:7236]
2025-10-09T21:46:28.689051Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:28.697637Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a34bd7f-7804399f-453f9454-bc15567f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:28.699143Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:28.699164Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 709
2025-10-09T21:46:28.699343Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 709 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:28.699393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 709 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:28.699407Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=709 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:28.699412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:28.700790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 709 sessionId:  describe result for acl check
2025-10-09T21:46:28.700838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:28.700845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:28.700850Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:28.700857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341677152478920:7293] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:28.701428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341677152478920:7293] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:28.704527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341677152478920:7293] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:28.707766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341677152478920:7293] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:28.707784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341677152478920:7293] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:28.707788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341677152478920:7293] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:28.707797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 709 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:28.745933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:28.746904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:28.747130Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:28.747155Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:28.747169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:28.747244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:28.747292Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341676790664543:6651]
2025-10-09T21:46:28.747933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:28.748438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:28.748519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:28.748872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:28.748936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:28.748937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:28.748939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:28.748940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:28.748942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:28.748943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:28.748944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:28.748945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:28.748947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:28.748953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:28.748964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:28.748968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:28.748972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:28.748976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:28.748991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:28.748995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:28.748999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:28.749003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:28.749012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:28.749020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:28.749027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:28.749032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:28.749036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:28.749041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:28.749046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:28.749051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:28.749056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:28.749061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:28.749065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:28.749069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:28.749074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:28.749081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:28.749085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:28.749089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:28.749093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:28.749098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:28.749143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:28.757928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:28.759889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:28.759909Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:28.759913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:28.759918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:28.759927Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 708 [2:7559341676790664543:6651]
2025-10-09T21:46:28.759944Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:28.759965Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:28.759974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:28.759978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:28.759980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:28.760002Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:28.760018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:28.760026Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:28.760104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:28.760112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:28.760114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:28.760118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:28.760200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:28.760207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:28.760221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:28.760448Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 708
2025-10-09T21:46:28.760492Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 708
2025-10-09T21:46:28.762217Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 709 partition: 0 MaxSeqNo: 25 sessionId: src_id|b7ac23-6015969a-d1a02584-ec7ed271_0
2025-10-09T21:46:28.760231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:28.760251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:28.760274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341676790664538:6651]
2025-10-09T21:46:28.760276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341676790664538:6651]
2025-10-09T21:46:28.760278Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341676790664538:6651]
2025-10-09T21:46:28.760288Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:28.760306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:28.760308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:28.760311Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:28.760314Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:28.760320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:28.760323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:28.760337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341677152478964:7293], now have 1 active actors on pipe
2025-10-09T21:46:28.760340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341677152478906:2432], now have 1 active actors on pipe
2025-10-09T21:46:28.760383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341676790664538:6651]
2025-10-09T21:46:28.760392Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:28.760425Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:28.760682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:28.760690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:28.760740Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b7ac23-6015969a-d1a02584-ec7ed271_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:28.760750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:28.760755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:28.760758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:28.760772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:28.760786Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:28.760823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:28.761989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:28.761994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:28.762026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:28.766443Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a34bd7f-7804399f-453f9454-bc15567f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b7ac23-6015969a-d1a02584-ec7ed271_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:28.769073Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7ac23-6015969a-d1a02584-ec7ed271_0] Write session: try to update token
2025-10-09T21:46:28.769099Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7ac23-6015969a-d1a02584-ec7ed271_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:28.777593Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:28.777611Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:28.777614Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:28.777616Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:28.777625Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341676790664538:6651]
2025-10-09T21:46:28.777630Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341676790664538:6651]
2025-10-09T21:46:28.777634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341676790664538:6651]
2025-10-09T21:46:28.777680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:28.777689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:28.777690Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:28.777692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:28.777695Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:28.777699Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:28.777844Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:28.788059Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 709 sessionId: src_id|b7ac23-6015969a-d1a02584-ec7ed271_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:28.789265Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341677152479198:7236]
2025-10-09T21:46:28.789893Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 708, pipe: [1:7559341677152479198:7236]
2025-10-09T21:46:28.790847Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:28.946882Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:28.948013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341677152479198:7236], now have 1 active actors on pipe
2025-10-09T21:46:28.948058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:28.948067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:28.948777Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:28.948796Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:28.948865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 709 reason: error at write: Disconnected sessionId: src_id|b7ac23-6015969a-d1a02584-ec7ed271_0
2025-10-09T21:46:28.949089Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 709 sessionId: src_id|b7ac23-6015969a-d1a02584-ec7ed271_0 is DEAD
2025-10-09T21:46:28.948813Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:28.949256Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7ac23-6015969a-d1a02584-ec7ed271_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:28.949268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7ac23-6015969a-d1a02584-ec7ed271_0] Write session will restart in 0.006468s
2025-10-09T21:46:28.949293Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:28.949332Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341677152479198:7236]
2025-10-09T21:46:28.949309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7ac23-6015969a-d1a02584-ec7ed271_0] Write session: Do CDS request
2025-10-09T21:46:28.949604Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7ac23-6015969a-d1a02584-ec7ed271_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:28.958538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7ac23-6015969a-d1a02584-ec7ed271_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:28.958630Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:28.958652Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 710
2025-10-09T21:46:28.958927Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 710 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:28.958981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 710 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:28.958987Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=710 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:28.958991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:28.959894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 710 sessionId:  describe result for acl check
2025-10-09T21:46:28.959940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:28.959945Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:28.959947Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:28.959953Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341677152479216:7298] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:28.960380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341677152479216:7298] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:28.969294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341677152479216:7298] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:28.983323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341677152479216:7298] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:28.983346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341677152479216:7298] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:28.983351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341677152479216:7298] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:28.983362Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 710 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:28.994642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:29.001147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:29.001604Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:29.001620Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:29.001636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:29.001708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:29.001756Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341681085632134:6653]
2025-10-09T21:46:29.002362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:29.002878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:29.002926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:29.003262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:29.003315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:29.003322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:29.003323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:29.003324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:29.003325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:29.003326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:29.003327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.003328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.003329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.003335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:29.003345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:29.003348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:29.003352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:29.003355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:29.003359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:29.003363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.003366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.003369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.003376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:29.003384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:29.003391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:29.003396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:29.003400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:29.003410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:29.003414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:29.003419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:29.003423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:29.003428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:29.003432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:29.003437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:29.003442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.003447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.003451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.003456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.003461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.003465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.003493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:29.011490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:29.012573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:29.012592Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:29.012595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:29.012599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:29.012606Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 709 [2:7559341681085632134:6653]
2025-10-09T21:46:29.012621Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:29.012639Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:29.012642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:29.012645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:29.012647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:29.012665Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:29.012671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:29.012677Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:29.012745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.012747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.012749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.012752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:29.012823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:29.012825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.012837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.012847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.012860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:29.012880Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341676790664832:6653]
2025-10-09T21:46:29.012882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341676790664832:6653]
2025-10-09T21:46:29.012884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341676790664832:6653]
2025-10-09T21:46:29.012898Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:29.012913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:29.012926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:29.012929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:29.012971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341676790664832:6653]
2025-10-09T21:46:29.012973Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:29.013063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:29.013065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:29.013067Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:29.013073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:29.013360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341677152479201:2432], now have 1 active actors on pipe
2025-10-09T21:46:29.013367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341677152479393:7298], now have 1 active actors on pipe
2025-10-09T21:46:29.013610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:29.013626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:29.013675Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|96875400-44c078de-7831c224-d5206a27_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:29.013688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:29.013693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:29.013695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:29.013706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:29.013722Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:29.013752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:29.013438Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 709
2025-10-09T21:46:29.013502Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 709
2025-10-09T21:46:29.014374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:29.014389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:29.014412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:29.014544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 710 partition: 0 MaxSeqNo: 25 sessionId: src_id|96875400-44c078de-7831c224-d5206a27_0
2025-10-09T21:46:29.014995Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7ac23-6015969a-d1a02584-ec7ed271_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|96875400-44c078de-7831c224-d5206a27_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:29.017753Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96875400-44c078de-7831c224-d5206a27_0] Write session: try to update token
2025-10-09T21:46:29.017774Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96875400-44c078de-7831c224-d5206a27_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:29.036041Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 710 sessionId: src_id|96875400-44c078de-7831c224-d5206a27_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:29.035470Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:29.035490Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:29.035493Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:29.035495Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:29.035506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341676790664832:6653]
2025-10-09T21:46:29.035511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341676790664832:6653]
2025-10-09T21:46:29.035515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341676790664832:6653]
2025-10-09T21:46:29.035565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:29.035574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.035577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:29.035579Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:29.035587Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:29.035592Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:29.035717Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:29.038806Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:29.046301Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341681447446811:7236]
2025-10-09T21:46:29.046684Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 709, pipe: [1:7559341681447446811:7236]
2025-10-09T21:46:29.308140Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:29.308462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:29.308473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:29.308525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341681447446811:7236], now have 1 active actors on pipe
2025-10-09T21:46:29.310223Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96875400-44c078de-7831c224-d5206a27_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:29.310237Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96875400-44c078de-7831c224-d5206a27_0] Write session will restart in 0.006658s
2025-10-09T21:46:29.310273Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96875400-44c078de-7831c224-d5206a27_0] Write session: Do CDS request
2025-10-09T21:46:29.310281Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96875400-44c078de-7831c224-d5206a27_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:29.309656Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:29.309718Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:29.309721Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:29.309787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 710 reason: error at write: Disconnected sessionId: src_id|96875400-44c078de-7831c224-d5206a27_0
2025-10-09T21:46:29.309983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 710 sessionId: src_id|96875400-44c078de-7831c224-d5206a27_0 is DEAD
2025-10-09T21:46:29.310211Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341681447446811:7236]
2025-10-09T21:46:29.310296Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:29.318665Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96875400-44c078de-7831c224-d5206a27_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:29.319092Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:29.319121Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 711
2025-10-09T21:46:29.319250Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 711 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:29.319282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 711 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:29.319288Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=711 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:29.319293Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:29.319802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 711 sessionId:  describe result for acl check
2025-10-09T21:46:29.319851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:29.319853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:29.319855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:29.319861Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341681447446831:7310] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:29.320487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341681447446831:7310] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:29.324321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341681447446831:7310] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:29.329154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341681447446831:7310] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:29.329172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341681447446831:7310] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:29.329176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341681447446831:7310] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:29.329199Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 711 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:29.367244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:29.368102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:29.368345Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:29.368349Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:29.368362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:29.368433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:29.368475Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341681085632442:6667]
2025-10-09T21:46:29.369102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:29.369549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:29.369605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:29.369945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:29.370022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:29.370044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:29.370046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:29.370047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:29.370049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:29.370050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:29.370051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.370053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.370055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.370061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:29.370072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:29.370076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:29.370081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:29.370085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:29.370089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:29.370094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.370098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.370102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.370111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:29.370119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:29.370127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:29.370134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:29.370139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:29.370144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:29.370149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:29.370154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:29.370159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:29.370164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:29.370168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:29.370173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:29.370178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.370184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.370189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.370194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.370198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.370203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.370230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:29.383981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:29.385369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:29.385391Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:29.385394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:29.385399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:29.385406Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 710 [2:7559341681085632442:6667]
2025-10-09T21:46:29.385424Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:29.385446Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:29.385456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:29.385459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:29.385462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:29.385482Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:29.385489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:29.385496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:29.385564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.385574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.385576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.385579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:29.385672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:29.385680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.385697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.385710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.385729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:29.385754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341681447446821:2432], now have 1 active actors on pipe
2025-10-09T21:46:29.385768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341681085632437:6667]
2025-10-09T21:46:29.385771Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341681085632437:6667]
2025-10-09T21:46:29.385774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341681085632437:6667]
2025-10-09T21:46:29.385783Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:29.385793Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:29.385796Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:29.385798Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:29.385798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:29.385817Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:29.385820Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:29.385827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341681447446878:7310], now have 1 active actors on pipe
2025-10-09T21:46:29.385869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341681085632437:6667]
2025-10-09T21:46:29.385886Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:29.385895Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:29.385899Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 710
2025-10-09T21:46:29.385900Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 710
2025-10-09T21:46:29.386679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:29.386694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:29.386733Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|aff5a394-f4b46d1-dd6e53f1-f6eb4b1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:29.386747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:29.386764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:29.386767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:29.386780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:29.386796Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:29.386817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:29.387211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:29.387222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:29.387242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:29.387436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 711 partition: 0 MaxSeqNo: 25 sessionId: src_id|aff5a394-f4b46d1-dd6e53f1-f6eb4b1_0
2025-10-09T21:46:29.387874Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96875400-44c078de-7831c224-d5206a27_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|aff5a394-f4b46d1-dd6e53f1-f6eb4b1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:29.390562Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aff5a394-f4b46d1-dd6e53f1-f6eb4b1_0] Write session: try to update token
2025-10-09T21:46:29.390584Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aff5a394-f4b46d1-dd6e53f1-f6eb4b1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:29.410277Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341681447447110:7236]
2025-10-09T21:46:29.410604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341681447447110:7236], now have 1 active actors on pipe
2025-10-09T21:46:29.410863Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 710, pipe: [1:7559341681447447110:7236]
2025-10-09T21:46:29.412328Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:29.412345Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:29.412348Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:29.412350Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:29.412360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341681085632437:6667]
2025-10-09T21:46:29.412364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341681085632437:6667]
2025-10-09T21:46:29.412368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341681085632437:6667]
2025-10-09T21:46:29.412419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:29.412426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.412429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:29.412445Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:29.412451Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:29.412456Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:29.412581Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:29.415421Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 711 sessionId: src_id|aff5a394-f4b46d1-dd6e53f1-f6eb4b1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:29.417917Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:29.698974Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:29.700048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:29.700065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:29.702221Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aff5a394-f4b46d1-dd6e53f1-f6eb4b1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:29.702232Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aff5a394-f4b46d1-dd6e53f1-f6eb4b1_0] Write session will restart in 0.007075s
2025-10-09T21:46:29.702257Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aff5a394-f4b46d1-dd6e53f1-f6eb4b1_0] Write session: Do CDS request
2025-10-09T21:46:29.702263Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aff5a394-f4b46d1-dd6e53f1-f6eb4b1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:29.704914Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:29.700525Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:29.700543Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:29.700617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 711 reason: error at write: Disconnected sessionId: src_id|aff5a394-f4b46d1-dd6e53f1-f6eb4b1_0
2025-10-09T21:46:29.700804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 711 sessionId: src_id|aff5a394-f4b46d1-dd6e53f1-f6eb4b1_0 is DEAD
2025-10-09T21:46:29.700993Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341681447447110:7236]
2025-10-09T21:46:29.701008Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:29.710568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aff5a394-f4b46d1-dd6e53f1-f6eb4b1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:29.710747Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:29.710772Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 712
2025-10-09T21:46:29.710904Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 712 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:29.710934Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 712 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:29.710940Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=712 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:29.710944Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:29.711437Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 712 sessionId:  describe result for acl check
2025-10-09T21:46:29.711466Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:29.711468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:29.711470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:29.711476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341681447447129:7314] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:29.712032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341681447447129:7314] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:29.715807Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341681447447129:7314] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:29.723488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341681447447129:7314] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:29.723510Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341681447447129:7314] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:29.723516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341681447447129:7314] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:29.723525Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 712 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:29.765484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:29.775255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:29.775530Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:29.775544Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:29.775556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:29.775618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:29.775661Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341681085632730:6671]
2025-10-09T21:46:29.776348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:29.777137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:29.777196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:29.777565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:29.777623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:29.777629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:29.777630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:29.777631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:29.777633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:29.777634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:29.777635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.777636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.777638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.777644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:29.777654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:29.777659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:29.777664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:29.777669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:29.777674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:29.777678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.777682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.777686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.777694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:29.777703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:29.777709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:29.777718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:29.777723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:29.777736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:29.777746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:29.777751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:29.777756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:29.777761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:29.777766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:29.777775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:29.777783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.777790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.777795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.777800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.777805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.777810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.777843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:29.787860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:29.788997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:29.789004Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:29.789008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:29.789013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:29.789021Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 711 [2:7559341681085632730:6671]
2025-10-09T21:46:29.789038Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:29.789059Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:29.789063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:29.789067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:29.789070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:29.789307Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:29.789317Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:29.789326Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:29.789396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.789398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.789401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.789404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:29.789477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:29.789480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.789498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:29.789509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:29.789527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:29.789548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341681085632724:6671]
2025-10-09T21:46:29.789551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341681085632724:6671]
2025-10-09T21:46:29.789553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341681085632724:6671]
2025-10-09T21:46:29.789568Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:29.789585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:29.789599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:29.789603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:29.789647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341681085632724:6671]
2025-10-09T21:46:29.789650Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:29.789822Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:29.789829Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:29.789832Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:29.789837Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:29.791619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341681447447116:2432], now have 1 active actors on pipe
2025-10-09T21:46:29.791631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341681447447189:7314], now have 1 active actors on pipe
2025-10-09T21:46:29.791652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:29.791658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:29.791694Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|63229c0-de11d25c-eee7490f-c81563c4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:29.791705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:29.791711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:29.791714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:29.791727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:29.791744Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:29.791763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:29.792085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:29.792094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:29.792112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:29.792769Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aff5a394-f4b46d1-dd6e53f1-f6eb4b1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|63229c0-de11d25c-eee7490f-c81563c4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:29.790776Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 711
2025-10-09T21:46:29.790826Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 711
2025-10-09T21:46:29.792345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 712 partition: 0 MaxSeqNo: 25 sessionId: src_id|63229c0-de11d25c-eee7490f-c81563c4_0
2025-10-09T21:46:29.795516Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63229c0-de11d25c-eee7490f-c81563c4_0] Write session: try to update token
2025-10-09T21:46:29.795535Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63229c0-de11d25c-eee7490f-c81563c4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:29.803979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341681447447408:7236], now have 1 active actors on pipe
2025-10-09T21:46:29.803292Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341681447447408:7236]
2025-10-09T21:46:29.804118Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 711, pipe: [1:7559341681447447408:7236]
2025-10-09T21:46:29.827877Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 712 sessionId: src_id|63229c0-de11d25c-eee7490f-c81563c4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:29.830440Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:29.831880Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:29.831897Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:29.831901Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:29.831903Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:29.831912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341681085632724:6671]
2025-10-09T21:46:29.831918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341681085632724:6671]
2025-10-09T21:46:29.831921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341681085632724:6671]
2025-10-09T21:46:29.831975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:29.831983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:29.831987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:29.831991Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:29.831998Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:29.832003Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:29.832133Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:30.059832Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:30.060623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:30.060636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:30.064136Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:30.064192Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:30.064195Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:30.064256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 712 reason: error at write: Disconnected sessionId: src_id|63229c0-de11d25c-eee7490f-c81563c4_0
2025-10-09T21:46:30.064454Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 712 sessionId: src_id|63229c0-de11d25c-eee7490f-c81563c4_0 is DEAD
2025-10-09T21:46:30.064594Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341681447447408:7236]
2025-10-09T21:46:30.066562Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63229c0-de11d25c-eee7490f-c81563c4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:30.066576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63229c0-de11d25c-eee7490f-c81563c4_0] Write session will restart in 0.008866s
2025-10-09T21:46:30.066604Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63229c0-de11d25c-eee7490f-c81563c4_0] Write session: Do CDS request
2025-10-09T21:46:30.066611Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63229c0-de11d25c-eee7490f-c81563c4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:30.066479Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:30.078253Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63229c0-de11d25c-eee7490f-c81563c4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:30.080435Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:30.080455Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 713
2025-10-09T21:46:30.087189Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 713 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:30.087250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 713 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:30.087256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=713 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:30.087261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:30.087978Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 713 sessionId:  describe result for acl check
2025-10-09T21:46:30.088008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:30.088009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:30.088011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:30.088015Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341685742414756:7325] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:30.088545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341685742414756:7325] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:30.092551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341685742414756:7325] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:30.099470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341685742414756:7325] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:30.099489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341685742414756:7325] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:30.099493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341685742414756:7325] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:30.099503Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 713 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:30.147356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:30.158194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:30.158694Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:30.158708Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:30.158726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:30.158815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:30.158864Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341685380600349:6679]
2025-10-09T21:46:30.159505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:30.160386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:30.160446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:30.160817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:30.160874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:30.160876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:30.160877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:30.160879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:30.160880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:30.160882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:30.160883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.160884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.160886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.160893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:30.160904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:30.160909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:30.160913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:30.160917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:30.160922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:30.160926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.160930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.160934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.160942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:30.160951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:30.160958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:30.160964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:30.160969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:30.160974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:30.160979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:30.160984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:30.160990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:30.160995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:30.161000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:30.161006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:30.161011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.161017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.161021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.161026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.161031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.161036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.161064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:30.164995Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341685742415032:7236]
2025-10-09T21:46:30.177412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:30.178635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:30.178642Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:30.178646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:30.178651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:30.178660Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 712 [2:7559341685380600349:6679]
2025-10-09T21:46:30.178677Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:30.178703Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:30.178707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:30.178711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:30.178714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:30.178735Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:30.178743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:30.178751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:30.178823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.178825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.178828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.178832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:30.178917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:30.178920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.178938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.178951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.178972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:30.178997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341685380600343:6679]
2025-10-09T21:46:30.179000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341685380600343:6679]
2025-10-09T21:46:30.179003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341685380600343:6679]
2025-10-09T21:46:30.179020Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:30.179037Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:30.179054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:30.179057Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:30.179101Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341685380600343:6679]
2025-10-09T21:46:30.179105Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:30.179204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:30.179207Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:30.179210Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:30.179220Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:30.180217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341685742414814:7325], now have 1 active actors on pipe
2025-10-09T21:46:30.180232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341685742415032:7236], now have 1 active actors on pipe
2025-10-09T21:46:30.180236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341685742414731:2432], now have 1 active actors on pipe
2025-10-09T21:46:30.180602Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 712
2025-10-09T21:46:30.180639Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 712, pipe: [1:7559341685742415032:7236]
2025-10-09T21:46:30.180660Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 712
2025-10-09T21:46:30.181386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:30.181406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:30.181451Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ff7232c9-92f1cc65-ae48803e-b2f78806_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:30.181470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:30.181494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:30.181502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:30.181518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:30.181538Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:30.181567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:30.182935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:30.182951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:30.182982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:30.183541Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 713 partition: 0 MaxSeqNo: 25 sessionId: src_id|ff7232c9-92f1cc65-ae48803e-b2f78806_0
2025-10-09T21:46:30.186271Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63229c0-de11d25c-eee7490f-c81563c4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ff7232c9-92f1cc65-ae48803e-b2f78806_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:30.189214Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff7232c9-92f1cc65-ae48803e-b2f78806_0] Write session: try to update token
2025-10-09T21:46:30.189239Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff7232c9-92f1cc65-ae48803e-b2f78806_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:30.215012Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:30.215032Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:30.215035Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:30.215037Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:30.215048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341685380600343:6679]
2025-10-09T21:46:30.215053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341685380600343:6679]
2025-10-09T21:46:30.215058Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341685380600343:6679]
2025-10-09T21:46:30.215112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:30.215119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.215122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:30.215271Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:30.218270Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:30.218286Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:30.218290Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:30.232750Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 713 sessionId: src_id|ff7232c9-92f1cc65-ae48803e-b2f78806_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:30.235565Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:30.399685Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:30.400769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:30.400781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:30.401549Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:30.401567Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:30.401625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 713 reason: error at write: Disconnected sessionId: src_id|ff7232c9-92f1cc65-ae48803e-b2f78806_0
2025-10-09T21:46:30.401804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 713 sessionId: src_id|ff7232c9-92f1cc65-ae48803e-b2f78806_0 is DEAD
2025-10-09T21:46:30.401925Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:30.401998Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff7232c9-92f1cc65-ae48803e-b2f78806_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:30.402046Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff7232c9-92f1cc65-ae48803e-b2f78806_0] Write session will restart in 0.009345s
2025-10-09T21:46:30.402075Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff7232c9-92f1cc65-ae48803e-b2f78806_0] Write session: Do CDS request
2025-10-09T21:46:30.402082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff7232c9-92f1cc65-ae48803e-b2f78806_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:30.401290Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:30.402022Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341685742415032:7236]
2025-10-09T21:46:30.412640Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff7232c9-92f1cc65-ae48803e-b2f78806_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:30.414388Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:30.414420Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 714
2025-10-09T21:46:30.414661Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 714 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:30.414700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 714 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:30.414706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=714 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:30.414711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:30.415614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 714 sessionId:  describe result for acl check
2025-10-09T21:46:30.415661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:30.415664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:30.415665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:30.415673Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341685742415050:7331] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:30.416278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341685742415050:7331] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:30.426845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341685742415050:7331] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:30.434894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341685742415050:7331] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:30.434921Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341685742415050:7331] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:30.434926Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341685742415050:7331] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:30.434938Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 714 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:30.469118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:30.474241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:30.474536Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:30.474547Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:30.474562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:30.474637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:30.474687Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341685380600637:6687]
2025-10-09T21:46:30.475310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:30.475790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:30.475875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:30.476254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:30.476316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:30.476326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:30.476327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:30.476328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:30.476330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:30.476331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:30.476333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.476334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.476336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.476341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:30.476353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:30.476357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:30.476362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:30.476367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:30.476371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:30.476375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.476380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.476384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.476394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:30.476420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:30.476434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:30.476440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:30.476446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:30.476452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:30.476457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:30.476477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:30.476488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:30.476494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:30.476499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:30.476506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:30.476513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.476525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.476530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.476536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.476541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.476546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.476579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:30.489763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:30.491779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:30.491800Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:30.491802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:30.491807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:30.491815Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 713 [2:7559341685380600637:6687]
2025-10-09T21:46:30.491831Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:30.491852Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:30.491855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:30.491859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:30.491861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:30.491882Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:30.491891Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:30.491900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:30.491974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.491985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.491987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.491990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:30.492066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:30.492076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.492091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.492103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.492120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:30.492466Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 713
2025-10-09T21:46:30.492509Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 713
2025-10-09T21:46:30.492919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341685380600632:6687]
2025-10-09T21:46:30.492928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341685380600632:6687]
2025-10-09T21:46:30.492930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341685380600632:6687]
2025-10-09T21:46:30.492945Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:30.492964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341685742415101:7331], now have 1 active actors on pipe
2025-10-09T21:46:30.492984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341685742415042:2432], now have 1 active actors on pipe
2025-10-09T21:46:30.493003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:30.493014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:30.493041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:30.493060Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:30.493063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:30.493124Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4d700ed6-7ae79070-6f1ebcda-af60b2c9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:30.493138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:30.493142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:30.493145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:30.493156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:30.493170Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:30.493189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341685380600632:6687]
2025-10-09T21:46:30.493197Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:30.493233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:30.493265Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:30.493270Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:30.493273Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:30.493282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:30.493789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:30.493804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:30.493826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:30.494135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 714 partition: 0 MaxSeqNo: 25 sessionId: src_id|4d700ed6-7ae79070-6f1ebcda-af60b2c9_0
2025-10-09T21:46:30.498345Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff7232c9-92f1cc65-ae48803e-b2f78806_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4d700ed6-7ae79070-6f1ebcda-af60b2c9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:30.501013Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4d700ed6-7ae79070-6f1ebcda-af60b2c9_0] Write session: try to update token
2025-10-09T21:46:30.501038Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4d700ed6-7ae79070-6f1ebcda-af60b2c9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:30.504703Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341685742415337:7236]
2025-10-09T21:46:30.506514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341685742415337:7236], now have 1 active actors on pipe
2025-10-09T21:46:30.506570Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 713, pipe: [1:7559341685742415337:7236]
2025-10-09T21:46:30.521166Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 714 sessionId: src_id|4d700ed6-7ae79070-6f1ebcda-af60b2c9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:30.523991Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:30.524411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:30.524446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:30.524492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:30.524639Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:30.537568Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:30.537588Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:30.537591Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:30.537594Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:30.537603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341685380600632:6687]
2025-10-09T21:46:30.537607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341685380600632:6687]
2025-10-09T21:46:30.537612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341685380600632:6687]
2025-10-09T21:46:30.537663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:30.537671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.537674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:30.537812Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:30.539619Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:30.539636Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:30.539640Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:30.805894Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:30.814888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:30.814921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:30.815101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:30.815173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:30.815235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:30.815292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:30.815332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:30.815338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:30.815407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:30.815461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:30.815518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:30.815582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:30.815623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:30.815627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:30.815697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:30.815750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:30.815809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:30.815869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:30.815908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:30.815911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:30.815987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:30.816043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:30.816103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:30.816163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:30.816204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:30.816207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:30.816274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:30.816328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:30.816394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:30.816453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:30.816493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:30.816497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:30.816570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:30.816629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:30.816682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:30.816745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:30.816785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:30.816788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:30.816850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:30.816903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:30.816961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:30.817021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:30.817062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:30.817065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:30.817129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:30.817181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:30.817242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:30.817300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:30.817341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:30.817344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:30.817408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:30.817463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:30.817524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:30.817583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:30.817620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:30.817623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:30.817687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:30.817743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:30.817802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:30.817863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:30.817905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:30.817909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:30.818771Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d700ed6-7ae79070-6f1ebcda-af60b2c9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:30.818785Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d700ed6-7ae79070-6f1ebcda-af60b2c9_0] Write session will restart in 0.008742s
2025-10-09T21:46:30.818810Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d700ed6-7ae79070-6f1ebcda-af60b2c9_0] Write session: Do CDS request
2025-10-09T21:46:30.818816Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d700ed6-7ae79070-6f1ebcda-af60b2c9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:30.818332Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:30.818361Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:30.818394Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 714 reason: error at write: Disconnected sessionId: src_id|4d700ed6-7ae79070-6f1ebcda-af60b2c9_0
2025-10-09T21:46:30.818471Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341685742415337:7236]
2025-10-09T21:46:30.818588Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 714 sessionId: src_id|4d700ed6-7ae79070-6f1ebcda-af60b2c9_0 is DEAD
2025-10-09T21:46:30.818740Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:30.819720Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:30.831694Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4d700ed6-7ae79070-6f1ebcda-af60b2c9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:30.831974Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:30.831995Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 715
2025-10-09T21:46:30.832258Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 715 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:30.832303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 715 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:30.832326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=715 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:30.832335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:30.832928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 715 sessionId:  describe result for acl check
2025-10-09T21:46:30.832968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:30.832977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:30.832978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:30.832985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341685742415362:7336] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:30.833566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341685742415362:7336] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:30.836878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341685742415362:7336] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:30.841267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341685742415362:7336] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:30.841288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341685742415362:7336] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:30.841293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341685742415362:7336] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:30.841302Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 715 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:30.858404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:30.864600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:30.864852Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:30.864864Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:30.864876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:30.864949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:30.864993Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341685380600930:6691]
2025-10-09T21:46:30.865577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:30.866048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:30.866118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:30.866452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:30.866500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:30.866507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:30.866509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:30.866510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:30.866511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:30.866512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:30.866513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.866515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.866516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.866521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:30.866532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:30.866536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:30.866540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:30.866544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:30.866548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:30.866551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.866555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.866559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.866566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:30.866574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:30.866581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:30.866586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:30.866590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:30.866596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:30.866600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:30.866606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:30.866611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:30.866616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:30.866620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:30.866625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:30.866630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.866635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.866639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.866644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.866660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.866665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.866694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:30.877436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:30.879201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:30.879216Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:30.879219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:30.879224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:30.879232Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 714 [2:7559341685380600930:6691]
2025-10-09T21:46:30.879247Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:30.879268Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:30.879271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:30.879275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:30.879278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:30.879297Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:30.879306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:30.879316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:30.879391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.879400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.879402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.879405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:30.879481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:30.879489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.879503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:30.879513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:30.879528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:30.879555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341685380600925:6691]
2025-10-09T21:46:30.879561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341685380600925:6691]
2025-10-09T21:46:30.879563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341685380600925:6691]
2025-10-09T21:46:30.879576Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:30.879594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341685742415479:7336], now have 1 active actors on pipe
2025-10-09T21:46:30.879603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341685742415347:2432], now have 1 active actors on pipe
2025-10-09T21:46:30.879605Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:30.879608Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:30.879610Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:30.879613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:30.879626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:30.879633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:30.879675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341685380600925:6691]
2025-10-09T21:46:30.879685Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:30.879715Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:30.879689Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 714
2025-10-09T21:46:30.879712Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 714
2025-10-09T21:46:30.879797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:30.879811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:30.879847Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f66fa6e8-fbbff42e-f29e1216-42e326b7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:30.879871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:30.879887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:30.879892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:30.879906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:30.879922Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:30.879952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:30.880127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:30.880137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:30.880157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:30.880270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 715 partition: 0 MaxSeqNo: 25 sessionId: src_id|f66fa6e8-fbbff42e-f29e1216-42e326b7_0
2025-10-09T21:46:30.880657Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d700ed6-7ae79070-6f1ebcda-af60b2c9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f66fa6e8-fbbff42e-f29e1216-42e326b7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:30.883206Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f66fa6e8-fbbff42e-f29e1216-42e326b7_0] Write session: try to update token
2025-10-09T21:46:30.883224Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f66fa6e8-fbbff42e-f29e1216-42e326b7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:30.904089Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 715 sessionId: src_id|f66fa6e8-fbbff42e-f29e1216-42e326b7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:30.906863Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:30.909185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:30.909207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:30.909230Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:30.909234Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:30.909243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341685380600925:6691]
2025-10-09T21:46:30.909304Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:30.909353Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:30.909481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:30.910521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:30.910674Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:30.910823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:30.911020Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:30.911087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:30.911128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:30.912565Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:30.916854Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:30.916869Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:30.916871Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:30.916873Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:30.916883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341685380600925:6691]
2025-10-09T21:46:30.916887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341685380600925:6691]
2025-10-09T21:46:30.916890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341685380600925:6691]
2025-10-09T21:46:30.916934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:30.916941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:30.916944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:30.917074Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:30.918769Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341685742415642:7236]
2025-10-09T21:46:30.919454Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 714, pipe: [1:7559341685742415642:7236]
2025-10-09T21:46:30.919268Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:30.919284Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:30.919287Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:31.291445Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:31.292276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341685742415642:7236], now have 1 active actors on pipe
2025-10-09T21:46:31.317197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:31.317235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:31.317414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:31.317483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:31.317553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:31.317632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:31.326308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:31.326331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:31.329432Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341685742415642:7236]
2025-10-09T21:46:31.329477Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:31.329480Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:31.329511Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 715 reason: error at write: Disconnected sessionId: src_id|f66fa6e8-fbbff42e-f29e1216-42e326b7_0
2025-10-09T21:46:31.329742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 715 sessionId: src_id|f66fa6e8-fbbff42e-f29e1216-42e326b7_0 is DEAD
2025-10-09T21:46:31.329941Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:31.326412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:31.326476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:31.326534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:31.326595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:31.326637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:31.326644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:31.326712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:31.326769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:31.326827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:31.326898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:31.326938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:31.326941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:31.327005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:31.327068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:31.327131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:31.327194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:31.327236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:31.327239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:31.327305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:31.327367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:31.327433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:31.327499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:31.327541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:31.327544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:31.327616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:31.327673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:31.327730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:31.327790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:31.327831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:31.327835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:31.327904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:31.327963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:31.328027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:31.328094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:31.328135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:31.328138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:31.328207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:31.328267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:31.328325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:31.328384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:31.328423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:31.328427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:31.328492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:31.328551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:31.328612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:31.328674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:31.328715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:31.328718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:31.328783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:31.328840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:31.328906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:31.328969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:31.329010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:31.330274Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f66fa6e8-fbbff42e-f29e1216-42e326b7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:31.330288Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f66fa6e8-fbbff42e-f29e1216-42e326b7_0] Write session will restart in 0.005769s
2025-10-09T21:46:31.330312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f66fa6e8-fbbff42e-f29e1216-42e326b7_0] Write session: Do CDS request
2025-10-09T21:46:31.330319Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f66fa6e8-fbbff42e-f29e1216-42e326b7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:31.329013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:31.339023Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f66fa6e8-fbbff42e-f29e1216-42e326b7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:31.339413Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:31.339438Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 716
2025-10-09T21:46:31.339750Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 716 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:31.339785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 716 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:31.339790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=716 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:31.339794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:31.338592Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:31.341958Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 716 sessionId:  describe result for acl check
2025-10-09T21:46:31.342018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:31.342020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:31.342021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:31.342038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341690037382993:7348] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:31.342569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341690037382993:7348] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:31.347115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341690037382993:7348] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:31.352521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341690037382993:7348] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:31.352542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341690037382993:7348] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:31.352546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341690037382993:7348] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:31.352557Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 716 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:31.387524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:31.393815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:31.394129Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:31.394134Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:31.394159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:31.394578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:31.394636Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341689675568554:6705]
2025-10-09T21:46:31.395305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:31.395660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:31.395704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:31.396052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:31.396115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:31.396116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:31.396117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:31.396118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:31.396120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:31.396121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:31.396122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:31.396123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:31.396124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:31.396129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:31.396141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:31.396145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:31.396149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:31.396153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:31.396157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:31.396161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:31.396164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:31.396167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:31.396174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:31.396182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:31.396188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:31.396194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:31.396199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:31.396205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:31.396210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:31.396214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:31.396218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:31.396224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:31.396229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:31.396234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:31.396238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:31.396243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:31.396247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:31.396252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:31.396256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:31.396261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:31.396288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:31.404403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:31.405698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:31.405706Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:31.405709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:31.405714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:31.405723Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 715 [2:7559341689675568554:6705]
2025-10-09T21:46:31.405738Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:31.405761Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:31.405765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:31.405768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:31.405770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:31.405791Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:31.405797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:31.405804Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:31.405878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:31.405880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:31.405882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:31.405885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:31.405967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:31.405970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:31.405983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:31.405994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:31.406022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:31.406064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341689675568548:6705]
2025-10-09T21:46:31.406067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341689675568548:6705]
2025-10-09T21:46:31.406069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341689675568548:6705]
2025-10-09T21:46:31.406088Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:31.406115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341690037383026:7348], now have 1 active actors on pipe
2025-10-09T21:46:31.406125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341690037382978:2432], now have 1 active actors on pipe
2025-10-09T21:46:31.406131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:31.406144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:31.406148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:31.406221Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 715
2025-10-09T21:46:31.406288Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 715
2025-10-09T21:46:31.407114Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 716 partition: 0 MaxSeqNo: 25 sessionId: src_id|c64537c6-aa3b92c4-15f2d6c0-688dd8ec_0
2025-10-09T21:46:31.406188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341689675568548:6705]
2025-10-09T21:46:31.406199Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:31.406338Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:31.406342Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:31.406344Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:31.406350Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:31.406600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:31.406607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:31.406638Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c64537c6-aa3b92c4-15f2d6c0-688dd8ec_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:31.406647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:31.406652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:31.406655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:31.406670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:31.406682Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:31.406699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:31.406920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:31.406927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:31.406967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:31.410402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f66fa6e8-fbbff42e-f29e1216-42e326b7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c64537c6-aa3b92c4-15f2d6c0-688dd8ec_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:31.413180Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c64537c6-aa3b92c4-15f2d6c0-688dd8ec_0] Write session: try to update token
2025-10-09T21:46:31.413205Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c64537c6-aa3b92c4-15f2d6c0-688dd8ec_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:31.429602Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341690037383274:7236]
2025-10-09T21:46:31.429913Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 715, pipe: [1:7559341690037383274:7236]
2025-10-09T21:46:31.427562Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:31.427580Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:31.427583Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:31.427585Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:31.427596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341689675568548:6705]
2025-10-09T21:46:31.427601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341689675568548:6705]
2025-10-09T21:46:31.427604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341689675568548:6705]
2025-10-09T21:46:31.427650Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:31.427654Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:31.427661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:31.427662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:31.427670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:31.427673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:31.427816Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:31.445435Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 716 sessionId: src_id|c64537c6-aa3b92c4-15f2d6c0-688dd8ec_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:31.448200Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:31.602813Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:31.603668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341690037383274:7236], now have 1 active actors on pipe
2025-10-09T21:46:31.603729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:31.603839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:31.604113Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:31.604516Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341690037383274:7236]
2025-10-09T21:46:31.604548Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:31.604629Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 716 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c64537c6-aa3b92c4-15f2d6c0-688dd8ec_0
2025-10-09T21:46:31.604802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 716 sessionId: src_id|c64537c6-aa3b92c4-15f2d6c0-688dd8ec_0 is DEAD
2025-10-09T21:46:31.604926Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:31.605405Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c64537c6-aa3b92c4-15f2d6c0-688dd8ec_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:31.605417Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c64537c6-aa3b92c4-15f2d6c0-688dd8ec_0] Write session will restart in 0.006208s
2025-10-09T21:46:31.605444Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c64537c6-aa3b92c4-15f2d6c0-688dd8ec_0] Write session: Do CDS request
2025-10-09T21:46:31.605450Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c64537c6-aa3b92c4-15f2d6c0-688dd8ec_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:31.614490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c64537c6-aa3b92c4-15f2d6c0-688dd8ec_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:31.614830Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:31.614854Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 717
2025-10-09T21:46:31.615004Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 717 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:31.615040Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 717 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:31.615051Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=717 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:31.615060Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:31.615533Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 717 sessionId:  describe result for acl check
2025-10-09T21:46:31.615575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:31.615577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:31.615579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:31.615585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341690037383286:7352] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:31.616143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341690037383286:7352] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:31.624203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341690037383286:7352] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:31.634749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341690037383286:7352] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:31.634771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341690037383286:7352] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:31.634775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341690037383286:7352] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:31.634786Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 717 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:31.683940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:31.685064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:31.685295Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:31.685309Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:31.685323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:31.685391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:31.685433Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341689675568847:6709]
2025-10-09T21:46:31.686078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:31.686887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:31.686937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:31.687291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:31.687349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:31.687351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:31.687352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:31.687353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:31.687355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:31.687356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:31.687357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:31.687359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:31.687360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:31.687365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:31.687376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:31.687379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:31.687383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:31.687386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:31.687390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:31.687393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:31.687396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:31.687400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:31.687407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:31.687415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:31.687421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:31.687427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:31.687431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:31.687436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:31.687440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:31.687444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:31.687449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:31.687453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:31.687457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:31.687462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:31.687467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:31.687471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:31.687475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:31.687480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:31.687485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:31.687490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:31.687528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:31.696748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:31.698369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:31.698384Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:31.698387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:31.698392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:31.698400Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 716 [2:7559341689675568847:6709]
2025-10-09T21:46:31.698421Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:31.698440Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:31.698443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:31.698447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:31.698448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:31.698467Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:31.698478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:31.698487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:31.698550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:31.698558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:31.698560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:31.698563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:31.698641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:31.698651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:31.698663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:31.698672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:31.698685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:31.698706Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341689675568842:6709]
2025-10-09T21:46:31.698708Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341689675568842:6709]
2025-10-09T21:46:31.698710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341689675568842:6709]
2025-10-09T21:46:31.698718Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:31.698730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:31.698734Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:31.698734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:31.698736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:31.698748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:31.698751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:31.698763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341690037383277:2432], now have 1 active actors on pipe
2025-10-09T21:46:31.698776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341690037383371:7352], now have 1 active actors on pipe
2025-10-09T21:46:31.698815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341689675568842:6709]
2025-10-09T21:46:31.698827Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:31.698855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:31.699258Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 716
2025-10-09T21:46:31.699305Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 716
2025-10-09T21:46:31.699493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:31.699516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:31.699550Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|28e7d6af-3b498977-8e13f7b9-3b3031d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:31.699565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:31.699570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:31.699573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:31.699587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:31.699601Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:31.699619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:31.700548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:31.700561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:31.700601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:31.700857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 717 partition: 0 MaxSeqNo: 25 sessionId: src_id|28e7d6af-3b498977-8e13f7b9-3b3031d_0
2025-10-09T21:46:31.704449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c64537c6-aa3b92c4-15f2d6c0-688dd8ec_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|28e7d6af-3b498977-8e13f7b9-3b3031d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:31.705630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341690037383574:7236], now have 1 active actors on pipe
2025-10-09T21:46:31.704986Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341690037383574:7236]
2025-10-09T21:46:31.705643Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 716, pipe: [1:7559341690037383574:7236]
2025-10-09T21:46:31.707117Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28e7d6af-3b498977-8e13f7b9-3b3031d_0] Write session: try to update token
2025-10-09T21:46:31.707136Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28e7d6af-3b498977-8e13f7b9-3b3031d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:31.729901Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:31.729916Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:31.729919Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:31.729921Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:31.729931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341689675568842:6709]
2025-10-09T21:46:31.729936Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341689675568842:6709]
2025-10-09T21:46:31.729941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341689675568842:6709]
2025-10-09T21:46:31.729989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:31.729996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:31.730009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:31.730171Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:31.730384Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:31.730395Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:31.730400Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:31.741777Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 717 sessionId: src_id|28e7d6af-3b498977-8e13f7b9-3b3031d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:31.744391Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:31.930941Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:31.931908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:31.931925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:31.934548Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:31.934223Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341690037383574:7236]
2025-10-09T21:46:31.934274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:31.934313Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:31.934316Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:31.934379Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 717 reason: error at write: Disconnected sessionId: src_id|28e7d6af-3b498977-8e13f7b9-3b3031d_0
2025-10-09T21:46:31.934562Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 717 sessionId: src_id|28e7d6af-3b498977-8e13f7b9-3b3031d_0 is DEAD
2025-10-09T21:46:31.935045Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28e7d6af-3b498977-8e13f7b9-3b3031d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:31.935058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28e7d6af-3b498977-8e13f7b9-3b3031d_0] Write session will restart in 0.008266s
2025-10-09T21:46:31.935081Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28e7d6af-3b498977-8e13f7b9-3b3031d_0] Write session: Do CDS request
2025-10-09T21:46:31.935091Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28e7d6af-3b498977-8e13f7b9-3b3031d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:31.944571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28e7d6af-3b498977-8e13f7b9-3b3031d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:31.944685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:31.944706Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 718
2025-10-09T21:46:31.944868Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 718 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:31.944908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 718 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:31.944920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=718 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:31.944933Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:31.945455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 718 sessionId:  describe result for acl check
2025-10-09T21:46:31.945495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:31.945497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:31.945499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:31.945505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341690037383590:7359] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:31.946088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341690037383590:7359] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:31.960994Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341690037383590:7359] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:31.971579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341690037383590:7359] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:31.971601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341690037383590:7359] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:31.971605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341690037383590:7359] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:31.971615Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 718 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:32.004230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:32.008531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:32.008806Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:32.008811Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:32.008824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:32.008887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:32.008931Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341693970536433:6711]
2025-10-09T21:46:32.009569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:32.010176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:32.010231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:32.010575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:32.010626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:32.010627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:32.010629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:32.010630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:32.010632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:32.010633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:32.010635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.010636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.010637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.010643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:32.010655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:32.010660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:32.010665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:32.010669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:32.010674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:32.010678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.010683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.010687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.010695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:32.010707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:32.010713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:32.010719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:32.010724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:32.010730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:32.010735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:32.010741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:32.010745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:32.010750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:32.010755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:32.010761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:32.010766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.010771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.010776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.010781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.010787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.010809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.010843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:32.021460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:32.023128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:32.023135Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:32.023139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:32.023144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:32.023152Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 717 [2:7559341693970536433:6711]
2025-10-09T21:46:32.023168Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:32.023190Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:32.023194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:32.023197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:32.023199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:32.023219Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:32.023225Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:32.023236Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:32.023303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.023305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.023308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.023311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:32.023388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:32.023391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.023405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.023417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.023433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:32.023455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341693970536428:6711]
2025-10-09T21:46:32.023458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341693970536428:6711]
2025-10-09T21:46:32.023460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341693970536428:6711]
2025-10-09T21:46:32.023517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:32.023519Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:32.023521Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:32.024023Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:32.024055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:32.024074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:32.024078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:32.024092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341690037383582:2432], now have 1 active actors on pipe
2025-10-09T21:46:32.024097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341690037383647:7359], now have 1 active actors on pipe
2025-10-09T21:46:32.024140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341693970536428:6711]
2025-10-09T21:46:32.024154Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:32.024990Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 717
2025-10-09T21:46:32.025033Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 717
2025-10-09T21:46:32.026396Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:32.026449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:32.026456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:32.026513Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|597e871d-1107704d-e222695f-14edda51_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:32.026524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:32.026529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:32.026533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:32.026545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:32.026571Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:32.026592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:32.030057Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28e7d6af-3b498977-8e13f7b9-3b3031d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|597e871d-1107704d-e222695f-14edda51_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:32.029607Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 718 partition: 0 MaxSeqNo: 25 sessionId: src_id|597e871d-1107704d-e222695f-14edda51_0
2025-10-09T21:46:32.029435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:32.029447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:32.029475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:32.033129Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|597e871d-1107704d-e222695f-14edda51_0] Write session: try to update token
2025-10-09T21:46:32.033146Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|597e871d-1107704d-e222695f-14edda51_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:32.034893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341694332351175:7236], now have 1 active actors on pipe
2025-10-09T21:46:32.034485Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341694332351175:7236]
2025-10-09T21:46:32.034852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 717, pipe: [1:7559341694332351175:7236]
2025-10-09T21:46:32.053409Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:32.053434Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:32.053437Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:32.053440Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:32.053452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341693970536428:6711]
2025-10-09T21:46:32.053459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341693970536428:6711]
2025-10-09T21:46:32.053465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341693970536428:6711]
2025-10-09T21:46:32.053523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:32.053531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.053535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:32.053784Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:32.054222Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:32.054234Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:32.054239Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:32.076331Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 718 sessionId: src_id|597e871d-1107704d-e222695f-14edda51_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:32.079108Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:32.253431Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:32.254489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:32.254505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:32.254877Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:32.254951Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341694332351175:7236]
2025-10-09T21:46:32.254971Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:32.254973Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:32.255045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 718 reason: error at write: Disconnected sessionId: src_id|597e871d-1107704d-e222695f-14edda51_0
2025-10-09T21:46:32.255235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 718 sessionId: src_id|597e871d-1107704d-e222695f-14edda51_0 is DEAD
2025-10-09T21:46:32.254941Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:32.255583Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|597e871d-1107704d-e222695f-14edda51_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:32.255597Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|597e871d-1107704d-e222695f-14edda51_0] Write session will restart in 0.009249s
2025-10-09T21:46:32.255619Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|597e871d-1107704d-e222695f-14edda51_0] Write session: Do CDS request
2025-10-09T21:46:32.255629Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|597e871d-1107704d-e222695f-14edda51_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:32.265550Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|597e871d-1107704d-e222695f-14edda51_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:32.266290Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:32.266310Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 719
2025-10-09T21:46:32.266417Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 719 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:32.266458Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 719 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:32.266465Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=719 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:32.266468Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:32.267005Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 719 sessionId:  describe result for acl check
2025-10-09T21:46:32.267055Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:32.267063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:32.267065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:32.267072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341694332351228:7371] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:32.267681Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341694332351228:7371] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:32.274344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341694332351228:7371] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:32.278042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341694332351228:7371] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:32.278065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341694332351228:7371] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:32.278069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341694332351228:7371] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:32.278079Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 719 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:32.355263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:32.355263Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341694332351498:7236]
2025-10-09T21:46:32.362474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:32.362762Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:32.362779Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:32.362794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:32.362869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:32.362920Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341693970536763:6725]
2025-10-09T21:46:32.363533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:32.363924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:32.363977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:32.364341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:32.364398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:32.364399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:32.364400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:32.364402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:32.364403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:32.364404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:32.364405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.364407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.364408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.364414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:32.364426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:32.364430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:32.364434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:32.364438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:32.364442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:32.364446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.364450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.364455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.364462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:32.364471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:32.364478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:32.364484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:32.364488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:32.364493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:32.364496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:32.364501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:32.364505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:32.364509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:32.364514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:32.364519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:32.364523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.364528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.364532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.364537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.364542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.364548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.364576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:32.371876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:32.373127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:32.373141Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:32.373143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:32.373148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:32.373155Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 718 [2:7559341693970536763:6725]
2025-10-09T21:46:32.373170Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:32.373190Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:32.373194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:32.373197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:32.373199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:32.373217Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:32.373222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:32.373231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:32.373381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.373389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.373391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.373395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:32.373469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:32.373480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.373494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.373507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.373524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:32.373554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341693970536756:6725]
2025-10-09T21:46:32.373557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341693970536756:6725]
2025-10-09T21:46:32.373559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341693970536756:6725]
2025-10-09T21:46:32.373573Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:32.373588Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:32.373591Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:32.373593Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:32.373597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341694332351498:7236], now have 1 active actors on pipe
2025-10-09T21:46:32.373601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341694332351214:2432], now have 1 active actors on pipe
2025-10-09T21:46:32.373643Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 718, pipe: [1:7559341694332351498:7236]
2025-10-09T21:46:32.373666Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 718
2025-10-09T21:46:32.373689Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 718
2025-10-09T21:46:32.373604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341694332351294:7371], now have 1 active actors on pipe
2025-10-09T21:46:32.373612Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:32.373628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:32.373632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:32.373667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341693970536756:6725]
2025-10-09T21:46:32.373674Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:32.373720Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:32.373876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:32.373888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:32.373917Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|abb714bf-d75b5e69-64ae6213-ea8a6585_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:32.373926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:32.373931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:32.373938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:32.373950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:32.373961Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:32.373976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:32.374207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:32.374223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:32.374246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:32.374491Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 719 partition: 0 MaxSeqNo: 25 sessionId: src_id|abb714bf-d75b5e69-64ae6213-ea8a6585_0
2025-10-09T21:46:32.375007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|597e871d-1107704d-e222695f-14edda51_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|abb714bf-d75b5e69-64ae6213-ea8a6585_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:32.377834Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|abb714bf-d75b5e69-64ae6213-ea8a6585_0] Write session: try to update token
2025-10-09T21:46:32.377854Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|abb714bf-d75b5e69-64ae6213-ea8a6585_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:32.413080Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:32.413096Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:32.413107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341693970536756:6725]
2025-10-09T21:46:32.413192Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:32.412580Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 719 sessionId: src_id|abb714bf-d75b5e69-64ae6213-ea8a6585_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:32.415483Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:32.414642Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:32.414841Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:32.414985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:32.415140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:32.415280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:32.415476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:32.415514Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:32.415518Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:32.415521Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:32.415523Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:32.415532Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341693970536756:6725]
2025-10-09T21:46:32.415536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341693970536756:6725]
2025-10-09T21:46:32.415541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341693970536756:6725]
2025-10-09T21:46:32.415589Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:32.415610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:32.415617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.415619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:32.415759Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:32.415907Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:32.415920Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:32.415924Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:32.595823Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:32.596955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:32.596966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:32.597353Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:32.597414Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341694332351498:7236]
2025-10-09T21:46:32.597434Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:32.597437Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:32.597383Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:32.597862Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|abb714bf-d75b5e69-64ae6213-ea8a6585_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:32.597889Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|abb714bf-d75b5e69-64ae6213-ea8a6585_0] Write session will restart in 0.007402s
2025-10-09T21:46:32.597915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|abb714bf-d75b5e69-64ae6213-ea8a6585_0] Write session: Do CDS request
2025-10-09T21:46:32.597921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|abb714bf-d75b5e69-64ae6213-ea8a6585_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:32.597510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 719 reason: error at write: Disconnected sessionId: src_id|abb714bf-d75b5e69-64ae6213-ea8a6585_0
2025-10-09T21:46:32.597707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 719 sessionId: src_id|abb714bf-d75b5e69-64ae6213-ea8a6585_0 is DEAD
2025-10-09T21:46:32.606610Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|abb714bf-d75b5e69-64ae6213-ea8a6585_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:32.606702Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:32.606723Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 720
2025-10-09T21:46:32.606983Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 720 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:32.607047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 720 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:32.607061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=720 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:32.607066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:32.607703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 720 sessionId:  describe result for acl check
2025-10-09T21:46:32.607739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:32.607745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:32.607747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:32.607753Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341694332351528:7375] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:32.608310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341694332351528:7375] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:32.612011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341694332351528:7375] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:32.618760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341694332351528:7375] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:32.618784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341694332351528:7375] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:32.618789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341694332351528:7375] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:32.618801Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 720 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:32.640049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:32.647235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:32.647932Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:32.647937Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:32.647952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:32.648024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:32.648067Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341693970537058:6729]
2025-10-09T21:46:32.648730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:32.649186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:32.649245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:32.649596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:32.649652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:32.649658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:32.649660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:32.649661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:32.649663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:32.649664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:32.649666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.649667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.649669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.649675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:32.649687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:32.649692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:32.649696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:32.649699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:32.649704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:32.649708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.649712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.649717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.649724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:32.649733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:32.649740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:32.649752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:32.649757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:32.649768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:32.649772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:32.649788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:32.649793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:32.649804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:32.649809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:32.649815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:32.649821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.649832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.649837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.649843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.649848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.649854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.649903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:32.658134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:32.659621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:32.659640Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:32.659645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:32.659650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:32.659658Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 719 [2:7559341693970537058:6729]
2025-10-09T21:46:32.659675Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:32.659698Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:32.659701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:32.659705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:32.659707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:32.659728Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:32.659740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:32.659749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:32.659824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.659832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.659835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.659838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:32.659916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:32.659924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.659939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.659950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.659965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:32.659991Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341693970537053:6729]
2025-10-09T21:46:32.660000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341693970537053:6729]
2025-10-09T21:46:32.660003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341693970537053:6729]
2025-10-09T21:46:32.660019Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:32.660040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341694332351514:2432], now have 1 active actors on pipe
2025-10-09T21:46:32.660043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341694332351653:7375], now have 1 active actors on pipe
2025-10-09T21:46:32.660042Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:32.660046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:32.660052Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:32.660053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:32.660068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:32.660071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:32.660114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341693970537053:6729]
2025-10-09T21:46:32.660127Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:32.660133Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:32.660133Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 719
2025-10-09T21:46:32.660178Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 719
2025-10-09T21:46:32.660373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:32.660386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:32.660430Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b9bb08b3-30cb92ed-5b6d92ed-3838bdaa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:32.660442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:32.660446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:32.660448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:32.660460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:32.660472Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:32.660488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:32.660725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:32.660731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:32.660752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:32.660861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 720 partition: 0 MaxSeqNo: 25 sessionId: src_id|b9bb08b3-30cb92ed-5b6d92ed-3838bdaa_0
2025-10-09T21:46:32.661282Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|abb714bf-d75b5e69-64ae6213-ea8a6585_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b9bb08b3-30cb92ed-5b6d92ed-3838bdaa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:32.664319Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b9bb08b3-30cb92ed-5b6d92ed-3838bdaa_0] Write session: try to update token
2025-10-09T21:46:32.664342Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b9bb08b3-30cb92ed-5b6d92ed-3838bdaa_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:32.677967Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:32.677990Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:32.677993Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:32.678011Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:32.678022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341693970537053:6729]
2025-10-09T21:46:32.678027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341693970537053:6729]
2025-10-09T21:46:32.678371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341693970537053:6729]
2025-10-09T21:46:32.678426Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:32.678432Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:32.678444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:32.678450Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:32.678452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.678455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:32.678631Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:32.695736Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 720 sessionId: src_id|b9bb08b3-30cb92ed-5b6d92ed-3838bdaa_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:32.698496Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:32.698594Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341694332351813:7236]
2025-10-09T21:46:32.704375Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 719, pipe: [1:7559341694332351813:7236]
2025-10-09T21:46:32.847412Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:32.848549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:32.848563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:32.848587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341694332351813:7236], now have 1 active actors on pipe
2025-10-09T21:46:32.850278Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:32.855367Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341694332351813:7236]
2025-10-09T21:46:32.855410Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:32.855450Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:32.855530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 720 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b9bb08b3-30cb92ed-5b6d92ed-3838bdaa_0
2025-10-09T21:46:32.855735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 720 sessionId: src_id|b9bb08b3-30cb92ed-5b6d92ed-3838bdaa_0 is DEAD
2025-10-09T21:46:32.855887Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9bb08b3-30cb92ed-5b6d92ed-3838bdaa_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:32.855899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9bb08b3-30cb92ed-5b6d92ed-3838bdaa_0] Write session will restart in 0.006540s
2025-10-09T21:46:32.855925Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9bb08b3-30cb92ed-5b6d92ed-3838bdaa_0] Write session: Do CDS request
2025-10-09T21:46:32.855932Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9bb08b3-30cb92ed-5b6d92ed-3838bdaa_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:32.866684Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b9bb08b3-30cb92ed-5b6d92ed-3838bdaa_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:32.867060Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:32.867082Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 721
2025-10-09T21:46:32.867213Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 721 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:32.867247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 721 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:32.867253Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=721 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:32.867256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:32.869482Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 721 sessionId:  describe result for acl check
2025-10-09T21:46:32.869542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:32.869544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:32.869545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:32.869552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341694332351832:7380] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:32.870167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341694332351832:7380] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:32.875809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341694332351832:7380] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:32.882446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341694332351832:7380] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:32.882472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341694332351832:7380] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:32.882476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341694332351832:7380] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:32.882487Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 721 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:32.898385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:32.905490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:32.905743Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:32.905753Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:32.905766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:32.905832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:32.905879Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341693970537350:6731]
2025-10-09T21:46:32.906505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:32.907145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:32.907197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:32.907547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:32.907591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:32.907592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:32.907593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:32.907594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:32.907595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:32.907597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:32.907598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.907600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.907601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.907607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:32.907618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:32.907622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:32.907626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:32.907630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:32.907634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:32.907638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.907642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.907646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.907654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:32.907663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:32.907670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:32.907676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:32.907680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:32.907685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:32.907690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:32.907695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:32.907700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:32.907705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:32.907710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:32.907715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:32.907720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.907738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.907743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.907748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.907753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.907758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.907796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:32.919775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:32.921625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:32.921642Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:32.921646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:32.921651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:32.921661Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 720 [2:7559341693970537350:6731]
2025-10-09T21:46:32.921675Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:32.921694Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:32.921698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:32.921701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:32.921704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:32.921723Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:32.921734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:32.921740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:32.921813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.921818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.921820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.921823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:32.921898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:32.921901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.921914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:32.921925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:32.921941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:32.921970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341693970537345:6731]
2025-10-09T21:46:32.921978Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341693970537345:6731]
2025-10-09T21:46:32.921980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341693970537345:6731]
2025-10-09T21:46:32.922007Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:32.922230Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:32.922249Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:32.922252Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:32.922315Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:32.922353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:32.922361Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:32.922425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341694332351820:2432], now have 1 active actors on pipe
2025-10-09T21:46:32.922433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341694332351948:7380], now have 1 active actors on pipe
2025-10-09T21:46:32.922436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341693970537345:6731]
2025-10-09T21:46:32.922445Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:32.922729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:32.923191Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 720
2025-10-09T21:46:32.923236Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 720
2025-10-09T21:46:32.923476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:32.923492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:32.923535Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2653908f-beaa79d5-f89e4838-aa0df5ce_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:32.923551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:32.923560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:32.923563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:32.923575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:32.923598Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:32.923622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:32.924156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:32.924169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:32.924189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:32.924365Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 721 partition: 0 MaxSeqNo: 25 sessionId: src_id|2653908f-beaa79d5-f89e4838-aa0df5ce_0
2025-10-09T21:46:32.924857Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9bb08b3-30cb92ed-5b6d92ed-3838bdaa_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2653908f-beaa79d5-f89e4838-aa0df5ce_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:32.927429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2653908f-beaa79d5-f89e4838-aa0df5ce_0] Write session: try to update token
2025-10-09T21:46:32.927450Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2653908f-beaa79d5-f89e4838-aa0df5ce_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:32.945963Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:32.945985Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:32.945988Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:32.945990Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:32.946016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341693970537345:6731]
2025-10-09T21:46:32.946021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341693970537345:6731]
2025-10-09T21:46:32.946025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341693970537345:6731]
2025-10-09T21:46:32.946087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:32.946101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:32.946104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:32.946641Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:32.950653Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:32.950669Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:32.950673Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:32.954530Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341694332352119:7236]
2025-10-09T21:46:32.954910Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 720, pipe: [1:7559341694332352119:7236]
2025-10-09T21:46:32.956243Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 721 sessionId: src_id|2653908f-beaa79d5-f89e4838-aa0df5ce_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:32.977568Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:33.135435Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:33.136335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341694332352119:7236], now have 1 active actors on pipe
2025-10-09T21:46:33.136370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:33.136378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:33.138439Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:33.143155Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:33.143206Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:33.143287Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 721 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2653908f-beaa79d5-f89e4838-aa0df5ce_0
2025-10-09T21:46:33.143475Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 721 sessionId: src_id|2653908f-beaa79d5-f89e4838-aa0df5ce_0 is DEAD
2025-10-09T21:46:33.143614Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341694332352119:7236]
2025-10-09T21:46:33.143943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2653908f-beaa79d5-f89e4838-aa0df5ce_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:33.143957Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2653908f-beaa79d5-f89e4838-aa0df5ce_0] Write session will restart in 0.009777s
2025-10-09T21:46:33.143996Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2653908f-beaa79d5-f89e4838-aa0df5ce_0] Write session: Do CDS request
2025-10-09T21:46:33.144004Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2653908f-beaa79d5-f89e4838-aa0df5ce_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:33.154568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2653908f-beaa79d5-f89e4838-aa0df5ce_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:33.155018Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:33.155040Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 722
2025-10-09T21:46:33.155203Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 722 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:33.155258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 722 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:33.155273Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=722 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:33.155276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:33.155909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 722 sessionId:  describe result for acl check
2025-10-09T21:46:33.155956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:33.155958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:33.155962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:33.155968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341698627319446:7385] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:33.156583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341698627319446:7385] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:33.159417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341698627319446:7385] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:33.164614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341698627319446:7385] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:33.164636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341698627319446:7385] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:33.164640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341698627319446:7385] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:33.164651Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 722 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:33.201035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:33.203308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:33.203567Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:33.203572Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:33.203585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:33.203652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:33.203702Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341698265504940:6733]
2025-10-09T21:46:33.204306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:33.205331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:33.205391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:33.205673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:33.205728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:33.205730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:33.205731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:33.205733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:33.205734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:33.205735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:33.205737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.205738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.205739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.205745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:33.205756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:33.205761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:33.205765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:33.205769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:33.205773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:33.205777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.205781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.205785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.205793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:33.205801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:33.205808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:33.205813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:33.205818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:33.205823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:33.205828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:33.205833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:33.205837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:33.205843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:33.205849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:33.205852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:33.205855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.205858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.205861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.205864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.205867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.205870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.205888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:33.216660Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 721
2025-10-09T21:46:33.216698Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 721
2025-10-09T21:46:33.214828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:33.215963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:33.215972Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:33.215975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:33.215981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:33.215990Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 721 [2:7559341698265504940:6733]
2025-10-09T21:46:33.216007Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:33.216029Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:33.216034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:33.216037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:33.216039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:33.216059Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:33.216066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:33.216074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:33.216146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.216148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.216150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.216153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:33.216236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:33.216239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.216252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.216263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.216278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:33.216300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341698265504934:6733]
2025-10-09T21:46:33.216304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341698265504934:6733]
2025-10-09T21:46:33.216308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341698265504934:6733]
2025-10-09T21:46:33.216317Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:33.216325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:33.216327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:33.216329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:33.216335Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:33.216349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:33.216352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:33.216364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341698627319524:7385], now have 1 active actors on pipe
2025-10-09T21:46:33.216368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341698627319431:2432], now have 1 active actors on pipe
2025-10-09T21:46:33.216406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341698265504934:6733]
2025-10-09T21:46:33.216415Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:33.216878Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:33.217189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:33.217203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:33.217240Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ad85cde1-8a1d52f3-dbb6e1a6-5c18f275_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:33.217251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:33.217255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:33.217258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:33.217270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:33.217294Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:33.217313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:33.217690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:33.217700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:33.217720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:33.217912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 722 partition: 0 MaxSeqNo: 25 sessionId: src_id|ad85cde1-8a1d52f3-dbb6e1a6-5c18f275_0
2025-10-09T21:46:33.218367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2653908f-beaa79d5-f89e4838-aa0df5ce_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ad85cde1-8a1d52f3-dbb6e1a6-5c18f275_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:33.221036Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad85cde1-8a1d52f3-dbb6e1a6-5c18f275_0] Write session: try to update token
2025-10-09T21:46:33.221060Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad85cde1-8a1d52f3-dbb6e1a6-5c18f275_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:33.238472Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:33.238491Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:33.238494Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:33.238496Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:33.238506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341698265504934:6733]
2025-10-09T21:46:33.238511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341698265504934:6733]
2025-10-09T21:46:33.238516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341698265504934:6733]
2025-10-09T21:46:33.238562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:33.238570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.238572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:33.238714Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:33.242475Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:33.242488Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:33.242493Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:33.245191Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341698627319977:7236]
2025-10-09T21:46:33.246546Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 721, pipe: [1:7559341698627319977:7236]
2025-10-09T21:46:33.248460Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 722 sessionId: src_id|ad85cde1-8a1d52f3-dbb6e1a6-5c18f275_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:33.251122Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:33.452524Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:33.453269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341698627319977:7236], now have 1 active actors on pipe
2025-10-09T21:46:33.453314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:33.453325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:33.454323Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:33.455067Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:33.455152Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 722 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ad85cde1-8a1d52f3-dbb6e1a6-5c18f275_0
2025-10-09T21:46:33.455341Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 722 sessionId: src_id|ad85cde1-8a1d52f3-dbb6e1a6-5c18f275_0 is DEAD
2025-10-09T21:46:33.455469Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:33.455507Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341698627319977:7236]
2025-10-09T21:46:33.458132Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad85cde1-8a1d52f3-dbb6e1a6-5c18f275_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:33.458146Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad85cde1-8a1d52f3-dbb6e1a6-5c18f275_0] Write session will restart in 0.009211s
2025-10-09T21:46:33.458174Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad85cde1-8a1d52f3-dbb6e1a6-5c18f275_0] Write session: Do CDS request
2025-10-09T21:46:33.458182Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad85cde1-8a1d52f3-dbb6e1a6-5c18f275_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:33.470575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad85cde1-8a1d52f3-dbb6e1a6-5c18f275_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:33.471994Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:33.472016Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 723
2025-10-09T21:46:33.472177Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 723 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:33.472206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 723 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:33.472212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=723 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:33.472215Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:33.473438Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 723 sessionId:  describe result for acl check
2025-10-09T21:46:33.473472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:33.473474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:33.473476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:33.473483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341698627320036:7396] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:33.474541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341698627320036:7396] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:33.482063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341698627320036:7396] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:33.489697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341698627320036:7396] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:33.489716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341698627320036:7396] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:33.489721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341698627320036:7396] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:33.489731Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 723 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:33.529880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:33.530688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:33.530903Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:33.530908Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:33.530922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:33.530982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:33.531017Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341698265505277:6747]
2025-10-09T21:46:33.531635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:33.532048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:33.532124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:33.532514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:33.532591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:33.532601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:33.532602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:33.532604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:33.532605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:33.532606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:33.532608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.532609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.532610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.532617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:33.532628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:33.532632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:33.532635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:33.532638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:33.532655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:33.532658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.532661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.532665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.532674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:33.532682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:33.532691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:33.532695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:33.532709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:33.532715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:33.532719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:33.532725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:33.532730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:33.532735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:33.532740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:33.532745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:33.532749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.532753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.532756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.532761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.532765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.532770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.532798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:33.540208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:33.542462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:33.542484Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:33.542487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:33.542492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:33.542500Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 722 [2:7559341698265505277:6747]
2025-10-09T21:46:33.542518Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:33.542542Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:33.542545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:33.542549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:33.542551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:33.542572Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:33.542579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:33.542587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:33.542659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.542670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.542672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.542675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:33.542758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:33.542761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.542776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.542787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.542807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:33.542838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341698265505272:6747]
2025-10-09T21:46:33.542846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341698265505272:6747]
2025-10-09T21:46:33.542848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341698265505272:6747]
2025-10-09T21:46:33.542857Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:33.542873Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:33.542888Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:33.542894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:33.542898Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:33.542898Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:33.542900Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:33.542910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341698627320001:2432], now have 1 active actors on pipe
2025-10-09T21:46:33.542914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341698627320165:7396], now have 1 active actors on pipe
2025-10-09T21:46:33.542956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341698265505272:6747]
2025-10-09T21:46:33.542973Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:33.542990Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:33.542990Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 722
2025-10-09T21:46:33.543043Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 722
2025-10-09T21:46:33.543340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:33.543358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:33.543388Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|839b7816-4cd48a03-b74f6a7b-61b8772d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:33.543403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:33.543412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:33.543415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:33.543427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:33.543444Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:33.543481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:33.543767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:33.543776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:33.543793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:33.544068Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 723 partition: 0 MaxSeqNo: 25 sessionId: src_id|839b7816-4cd48a03-b74f6a7b-61b8772d_0
2025-10-09T21:46:33.546145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad85cde1-8a1d52f3-dbb6e1a6-5c18f275_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|839b7816-4cd48a03-b74f6a7b-61b8772d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:33.548591Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|839b7816-4cd48a03-b74f6a7b-61b8772d_0] Write session: try to update token
2025-10-09T21:46:33.548613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|839b7816-4cd48a03-b74f6a7b-61b8772d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:33.556004Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341698627320541:7236]
2025-10-09T21:46:33.556312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341698627320541:7236], now have 1 active actors on pipe
2025-10-09T21:46:33.556311Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 722, pipe: [1:7559341698627320541:7236]
2025-10-09T21:46:33.558509Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:33.558525Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:33.558528Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:33.558530Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:33.558539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341698265505272:6747]
2025-10-09T21:46:33.558543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341698265505272:6747]
2025-10-09T21:46:33.558547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341698265505272:6747]
2025-10-09T21:46:33.558594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:33.558602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.558605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:33.558757Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:33.561977Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:33.562011Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:33.562016Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:33.575203Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 723 sessionId: src_id|839b7816-4cd48a03-b74f6a7b-61b8772d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:33.577521Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:33.720481Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:33.721196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:33.721261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:33.722448Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:33.723447Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|839b7816-4cd48a03-b74f6a7b-61b8772d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:33.723458Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|839b7816-4cd48a03-b74f6a7b-61b8772d_0] Write session will restart in 0.008539s
2025-10-09T21:46:33.723489Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|839b7816-4cd48a03-b74f6a7b-61b8772d_0] Write session: Do CDS request
2025-10-09T21:46:33.723496Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|839b7816-4cd48a03-b74f6a7b-61b8772d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:33.722843Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:33.722901Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341698627320541:7236]
2025-10-09T21:46:33.722934Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:33.723071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 723 reason: status is not ok: tablet will be restarted right now sessionId: src_id|839b7816-4cd48a03-b74f6a7b-61b8772d_0
2025-10-09T21:46:33.723261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 723 sessionId: src_id|839b7816-4cd48a03-b74f6a7b-61b8772d_0 is DEAD
2025-10-09T21:46:33.733614Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|839b7816-4cd48a03-b74f6a7b-61b8772d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:33.734329Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:33.734357Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 724
2025-10-09T21:46:33.738183Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 724 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:33.738256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 724 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:33.738266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=724 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:33.738271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:33.738816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 724 sessionId:  describe result for acl check
2025-10-09T21:46:33.738862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:33.738864Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:33.738865Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:33.738872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341698627320600:7401] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:33.739463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341698627320600:7401] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:33.742472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341698627320600:7401] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:33.745475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341698627320600:7401] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:33.745493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341698627320600:7401] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:33.745498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341698627320600:7401] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:33.745507Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 724 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:33.765083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:33.766372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:33.766610Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:33.766624Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:33.766637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:33.766701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:33.766750Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341698265505572:6751]
2025-10-09T21:46:33.767350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:33.767858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:33.767913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:33.768232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:33.768305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:33.768306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:33.768308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:33.768309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:33.768310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:33.768311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:33.768313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.768314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.768315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.768321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:33.768333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:33.768337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:33.768342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:33.768353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:33.768357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:33.768361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.768365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.768369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.768376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:33.768385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:33.768391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:33.768397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:33.768401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:33.768407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:33.768411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:33.768421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:33.768426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:33.768430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:33.768437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:33.768442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:33.768447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.768457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.768461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.768465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.768471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.768500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.768530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:33.777029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:33.778175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:33.778194Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:33.778197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:33.778202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:33.778210Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 723 [2:7559341698265505572:6751]
2025-10-09T21:46:33.778225Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:33.778247Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:33.778250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:33.778254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:33.778257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:33.778276Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:33.778282Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:33.778290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:33.778362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.778372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.778374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.778377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:33.778458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:33.778467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.778482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:33.778493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:33.778507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:33.778529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341698265505567:6751]
2025-10-09T21:46:33.778531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341698265505567:6751]
2025-10-09T21:46:33.778533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341698265505567:6751]
2025-10-09T21:46:33.778544Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:33.778558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:33.778573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:33.778572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:33.778575Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:33.778576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:33.778578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:33.778626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341698627320546:2432], now have 1 active actors on pipe
2025-10-09T21:46:33.778637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341698627320790:7401], now have 1 active actors on pipe
2025-10-09T21:46:33.778640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341698265505567:6751]
2025-10-09T21:46:33.778670Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:33.778713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:33.778720Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 723
2025-10-09T21:46:33.778778Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 723
2025-10-09T21:46:33.778923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:33.778935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:33.778967Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3101d525-876717be-39bf5a80-1ea70fa1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:33.778976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:33.778981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:33.778984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:33.778996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:33.779009Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:33.779026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:33.779368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:33.779379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:33.779400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:33.779651Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 724 partition: 0 MaxSeqNo: 25 sessionId: src_id|3101d525-876717be-39bf5a80-1ea70fa1_0
2025-10-09T21:46:33.780135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|839b7816-4cd48a03-b74f6a7b-61b8772d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3101d525-876717be-39bf5a80-1ea70fa1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:33.782874Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3101d525-876717be-39bf5a80-1ea70fa1_0] Write session: try to update token
2025-10-09T21:46:33.782895Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3101d525-876717be-39bf5a80-1ea70fa1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:33.795465Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:33.795483Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:33.795496Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341698265505567:6751]
2025-10-09T21:46:33.795548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:33.795553Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:33.796439Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:33.796969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:33.797124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:33.797277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:33.797476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:33.797527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:33.808072Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 724 sessionId: src_id|3101d525-876717be-39bf5a80-1ea70fa1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:33.806640Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:33.806659Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:33.806662Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:33.806664Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:33.806677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341698265505567:6751]
2025-10-09T21:46:33.806682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341698265505567:6751]
2025-10-09T21:46:33.806686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341698265505567:6751]
2025-10-09T21:46:33.806734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:33.806741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:33.806744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:33.806881Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:33.810172Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:33.810199Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:33.810203Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:33.810753Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:33.823261Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341698627321091:7236]
2025-10-09T21:46:33.823691Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 723, pipe: [1:7559341698627321091:7236]
2025-10-09T21:46:33.970106Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:33.971490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:33.971510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:33.971536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341698627321091:7236], now have 1 active actors on pipe
2025-10-09T21:46:33.972271Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:33.972328Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341698627321091:7236]
2025-10-09T21:46:33.972350Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:33.972417Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 724 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3101d525-876717be-39bf5a80-1ea70fa1_0
2025-10-09T21:46:33.972606Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 724 sessionId: src_id|3101d525-876717be-39bf5a80-1ea70fa1_0 is DEAD
2025-10-09T21:46:33.972787Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3101d525-876717be-39bf5a80-1ea70fa1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:33.972801Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3101d525-876717be-39bf5a80-1ea70fa1_0] Write session will restart in 0.009731s
2025-10-09T21:46:33.972825Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3101d525-876717be-39bf5a80-1ea70fa1_0] Write session: Do CDS request
2025-10-09T21:46:33.972833Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3101d525-876717be-39bf5a80-1ea70fa1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:33.971760Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:33.983660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3101d525-876717be-39bf5a80-1ea70fa1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:33.987013Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:33.987039Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 725
2025-10-09T21:46:33.987221Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 725 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:33.987258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 725 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:33.987264Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=725 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:33.987268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:33.987843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 725 sessionId:  describe result for acl check
2025-10-09T21:46:33.987890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:33.987894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:33.987896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:33.987903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341698627321177:7405] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:33.988527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341698627321177:7405] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:33.992133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341698627321177:7405] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:33.996033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341698627321177:7405] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:33.996055Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341698627321177:7405] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:33.996060Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341698627321177:7405] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:33.996071Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 725 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:34.011171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:34.014070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:34.014342Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:34.014356Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:34.014371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:34.014441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:34.014486Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341702560473165:6753]
2025-10-09T21:46:34.015077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:34.015757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:34.015834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:34.016203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:34.016271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:34.016272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:34.016274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:34.016275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:34.016276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:34.016278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:34.016279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.016280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.016281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.016306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:34.016317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:34.016321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:34.016324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:34.016328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:34.016332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:34.016336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.016340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.016344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.016353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:34.016362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:34.016368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:34.016373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:34.016380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:34.016385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:34.016390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:34.016395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:34.016400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:34.016406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:34.016410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:34.016415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:34.016421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.016427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.016431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.016436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.016441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.016446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.016476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:34.027496Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 724
2025-10-09T21:46:34.024814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:34.027000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:34.027007Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:34.027010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:34.027014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:34.027023Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 724 [2:7559341702560473165:6753]
2025-10-09T21:46:34.027568Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 724
2025-10-09T21:46:34.027043Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:34.027068Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:34.027072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:34.027076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:34.027078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:34.027098Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:34.027105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:34.027113Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:34.027194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.027196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.027198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.027201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:34.027290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:34.027293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.027309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.027320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.027336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:34.027361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341698627321097:2432], now have 1 active actors on pipe
2025-10-09T21:46:34.027364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341698627321363:7405], now have 1 active actors on pipe
2025-10-09T21:46:34.027373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341702560473160:6753]
2025-10-09T21:46:34.027375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341702560473160:6753]
2025-10-09T21:46:34.027378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341702560473160:6753]
2025-10-09T21:46:34.027399Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:34.027416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:34.027434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:34.027437Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:34.027482Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341702560473160:6753]
2025-10-09T21:46:34.027485Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:34.027600Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:34.027604Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:34.027606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:34.027624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:34.028046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:34.028064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:34.028097Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|96b64d41-be90a1a9-2a5ac266-cf3b6fb9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:34.028109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:34.028113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:34.028116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:34.028128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:34.028143Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:34.028164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:34.028484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:34.028495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:34.028530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:34.028686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 725 partition: 0 MaxSeqNo: 25 sessionId: src_id|96b64d41-be90a1a9-2a5ac266-cf3b6fb9_0
2025-10-09T21:46:34.029141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3101d525-876717be-39bf5a80-1ea70fa1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|96b64d41-be90a1a9-2a5ac266-cf3b6fb9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:34.031920Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96b64d41-be90a1a9-2a5ac266-cf3b6fb9_0] Write session: try to update token
2025-10-09T21:46:34.031943Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96b64d41-be90a1a9-2a5ac266-cf3b6fb9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:34.044365Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:34.044386Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:34.044389Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:34.044391Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:34.044400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341702560473160:6753]
2025-10-09T21:46:34.044406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341702560473160:6753]
2025-10-09T21:46:34.044410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341702560473160:6753]
2025-10-09T21:46:34.044463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:34.044477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.044480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:34.044479Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:34.044487Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:34.044495Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:34.044646Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:34.054797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 725 sessionId: src_id|96b64d41-be90a1a9-2a5ac266-cf3b6fb9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:34.057522Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:34.073303Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341702922288936:7236]
2025-10-09T21:46:34.074138Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 724, pipe: [1:7559341702922288936:7236]
2025-10-09T21:46:34.292803Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:34.293596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:34.293616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:34.293640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341702922288936:7236], now have 1 active actors on pipe
2025-10-09T21:46:34.293841Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:34.293847Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:34.293863Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:34.293867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:34.293915Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341702922288936:7236]
2025-10-09T21:46:34.293932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 725 reason: error at write: Disconnected sessionId: src_id|96b64d41-be90a1a9-2a5ac266-cf3b6fb9_0
2025-10-09T21:46:34.294179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 725 sessionId: src_id|96b64d41-be90a1a9-2a5ac266-cf3b6fb9_0 is DEAD
2025-10-09T21:46:34.294604Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96b64d41-be90a1a9-2a5ac266-cf3b6fb9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:34.294616Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96b64d41-be90a1a9-2a5ac266-cf3b6fb9_0] Write session will restart in 0.007504s
2025-10-09T21:46:34.294643Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96b64d41-be90a1a9-2a5ac266-cf3b6fb9_0] Write session: Do CDS request
2025-10-09T21:46:34.294650Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96b64d41-be90a1a9-2a5ac266-cf3b6fb9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:34.303663Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96b64d41-be90a1a9-2a5ac266-cf3b6fb9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:34.303738Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:34.303758Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 726
2025-10-09T21:46:34.303909Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 726 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:34.303945Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 726 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:34.303954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=726 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:34.303958Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:34.304421Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 726 sessionId:  describe result for acl check
2025-10-09T21:46:34.304467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:34.304474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:34.304476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:34.304484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341702922288953:7410] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:34.305090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341702922288953:7410] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:34.307277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341702922288953:7410] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:34.311882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341702922288953:7410] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:34.311902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341702922288953:7410] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:34.311907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341702922288953:7410] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:34.311917Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 726 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:34.354845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:34.357401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:34.357627Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:34.357641Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:34.357653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:34.357708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:34.357744Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341702560473476:6755]
2025-10-09T21:46:34.358655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:34.361010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:34.361075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:34.361396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:34.361449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:34.361460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:34.361461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:34.361462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:34.361463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:34.361464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:34.361465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.361466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.361466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.361472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:34.361482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:34.361486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:34.361490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:34.361494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:34.361498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:34.361502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.361505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.361509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.361516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:34.361524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:34.361529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:34.361534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:34.361538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:34.361544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:34.361547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:34.361552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:34.361557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:34.361560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:34.361563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:34.361568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:34.361572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.361575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.361579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.361583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.361586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.361590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.361613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:34.371928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:34.374494Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 725
2025-10-09T21:46:34.374538Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 725
2025-10-09T21:46:34.373900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:34.373921Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:34.373924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:34.373929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:34.373937Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 725 [2:7559341702560473476:6755]
2025-10-09T21:46:34.373951Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:34.373973Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:34.373976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:34.373980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:34.373982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:34.374014Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:34.374021Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:34.374043Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:34.374114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.374117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.374119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.374121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:34.374198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:34.374200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.374215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.374227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.374240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:34.374261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341702560473471:6755]
2025-10-09T21:46:34.374264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341702560473471:6755]
2025-10-09T21:46:34.374266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341702560473471:6755]
2025-10-09T21:46:34.374274Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:34.374288Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:34.374302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:34.374305Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:34.374317Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:34.374321Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:34.374323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:34.374353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341702560473471:6755]
2025-10-09T21:46:34.374356Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:34.374363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:34.374367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341702922288990:7410], now have 1 active actors on pipe
2025-10-09T21:46:34.374370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341702922288946:2432], now have 1 active actors on pipe
2025-10-09T21:46:34.374601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:34.374620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:34.374658Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|66131ba4-f40f4f53-d59322c4-596873dd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:34.374668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:34.374673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:34.374675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:34.374688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:34.374701Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:34.374721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:34.374918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:34.374934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:34.374956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:34.375058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 726 partition: 0 MaxSeqNo: 25 sessionId: src_id|66131ba4-f40f4f53-d59322c4-596873dd_0
2025-10-09T21:46:34.375494Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96b64d41-be90a1a9-2a5ac266-cf3b6fb9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|66131ba4-f40f4f53-d59322c4-596873dd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:34.378387Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66131ba4-f40f4f53-d59322c4-596873dd_0] Write session: try to update token
2025-10-09T21:46:34.378408Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66131ba4-f40f4f53-d59322c4-596873dd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:34.388537Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:34.388554Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:34.388564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341702560473471:6755]
2025-10-09T21:46:34.388613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:34.388655Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:34.388809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:34.388960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:34.389101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:34.389329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:34.389686Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:34.389732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:34.394138Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341702922289512:7236]
2025-10-09T21:46:34.394563Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 725, pipe: [1:7559341702922289512:7236]
2025-10-09T21:46:34.394613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341702922289512:7236], now have 1 active actors on pipe
2025-10-09T21:46:34.395447Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 726 sessionId: src_id|66131ba4-f40f4f53-d59322c4-596873dd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:34.398531Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:34.398790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:34.398811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:34.398862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:34.399038Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:34.402707Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:34.402722Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:34.402724Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:34.402725Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:34.402737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341702560473471:6755]
2025-10-09T21:46:34.402741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341702560473471:6755]
2025-10-09T21:46:34.402744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341702560473471:6755]
2025-10-09T21:46:34.402790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:34.402796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.402798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:34.402826Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:34.402832Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:34.402836Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:34.402952Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:34.579928Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:34.588232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:34.588255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:34.588417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:34.588489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:34.588562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:34.588630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:34.588673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:34.588678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:34.588754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:34.588811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:34.588871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:34.588937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:34.588979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:34.588983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:34.589047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:34.589102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:34.589160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:34.589261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:34.589304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:34.589308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:34.589382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:34.589442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:34.589499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:34.589561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:34.589603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:34.589606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:34.589675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:34.589730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:34.589795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:34.589855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:34.589898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:34.589901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:34.589964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:34.590231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:34.591896Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:34.591908Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341702922289512:7236]
2025-10-09T21:46:34.591914Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:34.591941Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:34.591960Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 726 reason: error at write: Disconnected sessionId: src_id|66131ba4-f40f4f53-d59322c4-596873dd_0
2025-10-09T21:46:34.592165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 726 sessionId: src_id|66131ba4-f40f4f53-d59322c4-596873dd_0 is DEAD
2025-10-09T21:46:34.592936Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66131ba4-f40f4f53-d59322c4-596873dd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:34.592947Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66131ba4-f40f4f53-d59322c4-596873dd_0] Write session will restart in 0.007293s
2025-10-09T21:46:34.592971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66131ba4-f40f4f53-d59322c4-596873dd_0] Write session: Do CDS request
2025-10-09T21:46:34.592977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66131ba4-f40f4f53-d59322c4-596873dd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:34.590306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:34.590367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:34.590409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:34.590413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:34.590477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:34.590531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:34.590592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:34.590648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:34.590687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:34.590690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:34.590755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:34.590818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:34.590877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:34.590940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:34.590980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:34.590983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:34.591051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:34.591107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:34.591168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:34.591226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:34.591266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:34.591271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:34.591341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:34.591395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:34.591456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:34.591516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:34.591564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:34.591567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:34.592857Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:34.601627Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66131ba4-f40f4f53-d59322c4-596873dd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:34.601684Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:34.601703Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 727
2025-10-09T21:46:34.601893Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 727 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:34.601931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 727 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:34.601939Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=727 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:34.601942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:34.602417Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 727 sessionId:  describe result for acl check
2025-10-09T21:46:34.602456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:34.602458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:34.602460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:34.602465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341702922289531:7421] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:34.603053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341702922289531:7421] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:34.606995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341702922289531:7421] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:34.610934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341702922289531:7421] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:34.610955Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341702922289531:7421] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:34.610962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341702922289531:7421] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:34.610972Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 727 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:34.631476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:34.633710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:34.633988Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:34.634010Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:34.634024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:34.634107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:34.634149Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341702560473804:6771]
2025-10-09T21:46:34.634735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:34.635608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:34.635682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:34.636040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:34.636118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:34.636144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:34.636145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:34.636147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:34.636148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:34.636149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:34.636151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.636152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.636153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.636160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:34.636171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:34.636175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:34.636179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:34.636183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:34.636187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:34.636190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.636193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.636196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.636205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:34.636213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:34.636220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:34.636226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:34.636232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:34.636237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:34.636241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:34.636246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:34.636252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:34.636257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:34.636262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:34.636267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:34.636272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.636278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.636282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.636287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.636291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.636297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.636328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:34.644333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:34.646523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:34.646541Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:34.646543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:34.646548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:34.646558Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 726 [2:7559341702560473804:6771]
2025-10-09T21:46:34.646576Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:34.646597Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:34.646601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:34.646605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:34.646607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:34.646628Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:34.646640Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:34.646648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:34.646731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.646734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.646736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.646739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:34.646821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:34.646831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.646846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.646856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.646871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:34.646894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341702560473799:6771]
2025-10-09T21:46:34.646903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341702560473799:6771]
2025-10-09T21:46:34.646906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341702560473799:6771]
2025-10-09T21:46:34.646920Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:34.646935Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:34.646949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:34.646952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:34.646995Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:34.646999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341702560473799:6771]
2025-10-09T21:46:34.646999Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:34.647001Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:34.647002Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:34.647008Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:34.647162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341702922289516:2432], now have 1 active actors on pipe
2025-10-09T21:46:34.647212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341702922289713:7421], now have 1 active actors on pipe
2025-10-09T21:46:34.647726Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 726
2025-10-09T21:46:34.647783Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 726
2025-10-09T21:46:34.648001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:34.648015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:34.648047Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|952febc2-576093f1-6c16206f-439ae94e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:34.648059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:34.648063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:34.648066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:34.648090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:34.648103Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:34.648123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:34.648495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:34.648505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:34.648525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:34.648687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 727 partition: 0 MaxSeqNo: 25 sessionId: src_id|952febc2-576093f1-6c16206f-439ae94e_0
2025-10-09T21:46:34.652962Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66131ba4-f40f4f53-d59322c4-596873dd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|952febc2-576093f1-6c16206f-439ae94e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:34.655661Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|952febc2-576093f1-6c16206f-439ae94e_0] Write session: try to update token
2025-10-09T21:46:34.655683Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|952febc2-576093f1-6c16206f-439ae94e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:34.666855Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:34.666873Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:34.666876Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:34.666878Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:34.666888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341702560473799:6771]
2025-10-09T21:46:34.666892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341702560473799:6771]
2025-10-09T21:46:34.666896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341702560473799:6771]
2025-10-09T21:46:34.666943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:34.666946Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:34.666951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.666953Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:34.666954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:34.666957Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:34.667094Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:34.688076Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 727 sessionId: src_id|952febc2-576093f1-6c16206f-439ae94e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:34.690844Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:34.692091Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341702922290061:7236]
2025-10-09T21:46:34.692503Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 726, pipe: [1:7559341702922290061:7236]
2025-10-09T21:46:34.836637Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:34.836968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:34.836988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:34.837013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341702922290061:7236], now have 1 active actors on pipe
2025-10-09T21:46:34.837374Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:34.837466Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:34.837540Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341702922290061:7236]
2025-10-09T21:46:34.837573Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:34.837577Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:34.837655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 727 reason: error at write: Disconnected sessionId: src_id|952febc2-576093f1-6c16206f-439ae94e_0
2025-10-09T21:46:34.837844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 727 sessionId: src_id|952febc2-576093f1-6c16206f-439ae94e_0 is DEAD
2025-10-09T21:46:34.838317Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|952febc2-576093f1-6c16206f-439ae94e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:34.838329Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|952febc2-576093f1-6c16206f-439ae94e_0] Write session will restart in 0.009053s
2025-10-09T21:46:34.838358Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|952febc2-576093f1-6c16206f-439ae94e_0] Write session: Do CDS request
2025-10-09T21:46:34.838366Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|952febc2-576093f1-6c16206f-439ae94e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:34.848694Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|952febc2-576093f1-6c16206f-439ae94e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:34.850292Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:34.850315Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 728
2025-10-09T21:46:34.851306Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 728 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:34.851355Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 728 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:34.851362Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=728 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:34.851368Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:34.852436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 728 sessionId:  describe result for acl check
2025-10-09T21:46:34.852481Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:34.852484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:34.852485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:34.852492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341702922290102:7426] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:34.853031Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341702922290102:7426] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:34.858309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341702922290102:7426] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:34.861928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341702922290102:7426] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:34.861945Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341702922290102:7426] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:34.861949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341702922290102:7426] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:34.861960Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 728 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:34.885570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:34.887979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:34.888309Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:34.888324Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:34.888338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:34.888396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:34.888437Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341702560474104:6773]
2025-10-09T21:46:34.889056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:34.889615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:34.889662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:34.889969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:34.890051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:34.890052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:34.890053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:34.890055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:34.890056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:34.890057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:34.890059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.890060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.890061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.890066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:34.890077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:34.890080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:34.890083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:34.890087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:34.890090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:34.890093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.890096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.890099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.890106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:34.890114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:34.890120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:34.890124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:34.890128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:34.890133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:34.890136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:34.890141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:34.890145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:34.890149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:34.890152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:34.890156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:34.890161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.890165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.890169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.890173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.890177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.890181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.890207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:34.899788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:34.901737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:34.901757Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:34.901759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:34.901764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:34.901772Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 727 [2:7559341702560474104:6773]
2025-10-09T21:46:34.901789Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:34.901813Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:34.901817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:34.901821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:34.901823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:34.901844Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:34.901851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:34.901858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:34.901935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.901944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.901946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.901949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:34.902066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:34.902076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.902092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:34.902103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:34.902121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:34.902143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341702560474098:6773]
2025-10-09T21:46:34.902146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341702560474098:6773]
2025-10-09T21:46:34.902151Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341702560474098:6773]
2025-10-09T21:46:34.902182Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:34.902190Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:34.902193Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:34.902630Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:34.902759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:34.902781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:34.902784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:34.902809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341702922290236:7426], now have 1 active actors on pipe
2025-10-09T21:46:34.902812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341702922290070:2432], now have 1 active actors on pipe
2025-10-09T21:46:34.902859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341702560474098:6773]
2025-10-09T21:46:34.902797Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 727
2025-10-09T21:46:34.902871Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:34.902856Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 727
2025-10-09T21:46:34.902934Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:34.903852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:34.903864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:34.903900Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|80b1b32d-74e9e937-f9b87193-33f40d8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:34.903919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:34.903925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:34.903928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:34.903941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:34.903959Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:34.903984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:34.904317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:34.904338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:34.904358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:34.904644Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 728 partition: 0 MaxSeqNo: 25 sessionId: src_id|80b1b32d-74e9e937-f9b87193-33f40d8_0
2025-10-09T21:46:34.905102Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|952febc2-576093f1-6c16206f-439ae94e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|80b1b32d-74e9e937-f9b87193-33f40d8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:34.908239Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80b1b32d-74e9e937-f9b87193-33f40d8_0] Write session: try to update token
2025-10-09T21:46:34.908264Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80b1b32d-74e9e937-f9b87193-33f40d8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:34.933083Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:34.933103Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:34.933106Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:34.933108Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:34.933119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341702560474098:6773]
2025-10-09T21:46:34.933124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341702560474098:6773]
2025-10-09T21:46:34.933127Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341702560474098:6773]
2025-10-09T21:46:34.933176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:34.933182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:34.933185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:34.933436Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:34.933448Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:34.933452Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:34.937431Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 728 sessionId: src_id|80b1b32d-74e9e937-f9b87193-33f40d8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:34.933622Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:34.938123Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341702922290618:7236]
2025-10-09T21:46:34.940034Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:34.946151Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 727, pipe: [1:7559341702922290618:7236]
2025-10-09T21:46:35.126911Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:35.127265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:35.127277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:35.127304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341702922290618:7236], now have 1 active actors on pipe
2025-10-09T21:46:35.127589Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:35.128084Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:35.128105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:35.128174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 728 reason: error at write: Disconnected sessionId: src_id|80b1b32d-74e9e937-f9b87193-33f40d8_0
2025-10-09T21:46:35.128351Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 728 sessionId: src_id|80b1b32d-74e9e937-f9b87193-33f40d8_0 is DEAD
2025-10-09T21:46:35.128484Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341702922290618:7236]
2025-10-09T21:46:35.130126Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80b1b32d-74e9e937-f9b87193-33f40d8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:35.130150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80b1b32d-74e9e937-f9b87193-33f40d8_0] Write session will restart in 0.005343s
2025-10-09T21:46:35.128501Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:35.130176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80b1b32d-74e9e937-f9b87193-33f40d8_0] Write session: Do CDS request
2025-10-09T21:46:35.130183Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80b1b32d-74e9e937-f9b87193-33f40d8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:35.138548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80b1b32d-74e9e937-f9b87193-33f40d8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:35.138685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:35.138707Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 729
2025-10-09T21:46:35.138863Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 729 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:35.138899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 729 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:35.138910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=729 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:35.138915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:35.139732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 729 sessionId:  describe result for acl check
2025-10-09T21:46:35.139779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:35.139781Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:35.139783Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:35.139791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341707217257935:7431] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:35.140418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341707217257935:7431] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:35.146560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341707217257935:7431] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:35.150931Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341707217257935:7431] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:35.150954Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341707217257935:7431] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:35.150958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341707217257935:7431] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:35.150969Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 729 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:35.175566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:35.177400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:35.177663Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:35.177668Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:35.177681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:35.177742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:35.177785Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341706855441701:6775]
2025-10-09T21:46:35.178416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:35.178851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:35.178929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:35.179456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:35.179541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:35.179549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:35.179550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:35.179551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:35.179553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:35.179554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:35.179555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.179557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.179558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.179565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:35.179577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:35.179581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:35.179586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:35.179589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:35.179593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:35.179598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.179602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.179606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.179615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:35.179624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:35.179631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:35.179637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:35.179641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:35.179647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:35.179652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:35.179658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:35.179662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:35.179676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:35.179682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:35.179687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:35.179692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.179698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.179702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.179708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.179712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.179719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.179753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:35.188234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:35.189492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:35.189511Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:35.189514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:35.189520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:35.189528Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 728 [2:7559341706855441701:6775]
2025-10-09T21:46:35.189545Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:35.189572Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:35.189582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:35.189586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:35.189589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:35.189610Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:35.189622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:35.189631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:35.189706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.189715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.189718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.189721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:35.189805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:35.189816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.189830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.189841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.189861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:35.189889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341707217257920:2432], now have 1 active actors on pipe
2025-10-09T21:46:35.189899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341707217258045:7431], now have 1 active actors on pipe
2025-10-09T21:46:35.189909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341706855441696:6775]
2025-10-09T21:46:35.189911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341706855441696:6775]
2025-10-09T21:46:35.189914Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341706855441696:6775]
2025-10-09T21:46:35.189929Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:35.189952Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:35.189955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:35.189957Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:35.189959Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:35.189971Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:35.189982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:35.190068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341706855441696:6775]
2025-10-09T21:46:35.190089Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:35.190096Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:35.189978Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 728
2025-10-09T21:46:35.190070Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 728
2025-10-09T21:46:35.190139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:35.190151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:35.190184Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|40132369-6e16d50c-a149d1a5-b2e1e54c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:35.190194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:35.190198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:35.190202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:35.190214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:35.190227Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:35.190245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:35.191378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:35.191397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:35.191426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:35.191617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 729 partition: 0 MaxSeqNo: 25 sessionId: src_id|40132369-6e16d50c-a149d1a5-b2e1e54c_0
2025-10-09T21:46:35.194148Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80b1b32d-74e9e937-f9b87193-33f40d8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|40132369-6e16d50c-a149d1a5-b2e1e54c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:35.196949Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|40132369-6e16d50c-a149d1a5-b2e1e54c_0] Write session: try to update token
2025-10-09T21:46:35.196970Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|40132369-6e16d50c-a149d1a5-b2e1e54c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:35.207016Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:35.207034Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:35.207036Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:35.207039Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:35.207049Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341706855441696:6775]
2025-10-09T21:46:35.207054Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341706855441696:6775]
2025-10-09T21:46:35.207059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341706855441696:6775]
2025-10-09T21:46:35.207112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:35.207121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.207124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:35.207128Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:35.207136Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:35.207141Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:35.207316Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:35.214538Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 729 sessionId: src_id|40132369-6e16d50c-a149d1a5-b2e1e54c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:35.217252Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:35.226658Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341707217258472:7236]
2025-10-09T21:46:35.227061Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 728, pipe: [1:7559341707217258472:7236]
2025-10-09T21:46:35.373970Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:35.374659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:35.374680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:35.374731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341707217258472:7236], now have 1 active actors on pipe
2025-10-09T21:46:35.374957Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:35.374982Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:35.375000Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:35.375004Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:35.375043Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341707217258472:7236]
2025-10-09T21:46:35.375065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 729 reason: error at write: Disconnected sessionId: src_id|40132369-6e16d50c-a149d1a5-b2e1e54c_0
2025-10-09T21:46:35.375266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 729 sessionId: src_id|40132369-6e16d50c-a149d1a5-b2e1e54c_0 is DEAD
2025-10-09T21:46:35.375713Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|40132369-6e16d50c-a149d1a5-b2e1e54c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:35.375726Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|40132369-6e16d50c-a149d1a5-b2e1e54c_0] Write session will restart in 0.008357s
2025-10-09T21:46:35.375753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|40132369-6e16d50c-a149d1a5-b2e1e54c_0] Write session: Do CDS request
2025-10-09T21:46:35.375759Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|40132369-6e16d50c-a149d1a5-b2e1e54c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:35.385618Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|40132369-6e16d50c-a149d1a5-b2e1e54c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:35.385790Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:35.385813Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 730
2025-10-09T21:46:35.386196Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 730 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:35.386258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 730 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:35.386269Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=730 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:35.386274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:35.388746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 730 sessionId:  describe result for acl check
2025-10-09T21:46:35.388782Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:35.388784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:35.388786Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:35.388792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341707217258492:7435] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:35.389320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341707217258492:7435] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:35.393155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341707217258492:7435] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:35.401131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341707217258492:7435] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:35.401157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341707217258492:7435] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:35.401162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341707217258492:7435] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:35.401171Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 730 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:35.439174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:35.440331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:35.440959Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:35.440973Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:35.440992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:35.441070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:35.441111Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341706855442033:6783]
2025-10-09T21:46:35.441700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:35.442242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:35.442301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:35.442637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:35.442695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:35.442697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:35.442698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:35.442699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:35.442701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:35.442702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:35.442704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.442705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.442706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.442712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:35.442724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:35.442728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:35.442732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:35.442737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:35.442742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:35.442746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.442750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.442754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.442763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:35.442771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:35.442778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:35.442784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:35.442789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:35.442794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:35.442799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:35.442804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:35.442809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:35.442813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:35.442818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:35.442823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:35.442829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.442834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.442838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.442844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.442849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.442854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.442882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:35.451388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:35.452902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:35.452921Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:35.452924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:35.452929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:35.452938Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 729 [2:7559341706855442033:6783]
2025-10-09T21:46:35.452955Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:35.452979Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:35.452983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:35.452986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:35.452989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:35.453010Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:35.453017Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:35.453026Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:35.453102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.453104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.453107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.453110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:35.453193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:35.453196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.453212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.453225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.453242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:35.453264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341706855442028:6783]
2025-10-09T21:46:35.453267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341706855442028:6783]
2025-10-09T21:46:35.453270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341706855442028:6783]
2025-10-09T21:46:35.453281Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:35.453297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:35.453316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:35.453320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:35.453367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341706855442028:6783]
2025-10-09T21:46:35.453370Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:35.453501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:35.453506Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:35.453508Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:35.453514Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:35.453904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341707217258475:2432], now have 1 active actors on pipe
2025-10-09T21:46:35.453916Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 729
2025-10-09T21:46:35.453916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341707217258589:7435], now have 1 active actors on pipe
2025-10-09T21:46:35.454005Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 729
2025-10-09T21:46:35.454433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:35.454450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:35.454499Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|de862265-4a87283e-fd12bf54-6ce86c63_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:35.454510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:35.454516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:35.454519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:35.454532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:35.454549Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:35.454579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:35.454885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:35.454900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:35.454937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:35.455041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 730 partition: 0 MaxSeqNo: 25 sessionId: src_id|de862265-4a87283e-fd12bf54-6ce86c63_0
2025-10-09T21:46:35.455508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|40132369-6e16d50c-a149d1a5-b2e1e54c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|de862265-4a87283e-fd12bf54-6ce86c63_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:35.458614Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|de862265-4a87283e-fd12bf54-6ce86c63_0] Write session: try to update token
2025-10-09T21:46:35.458641Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|de862265-4a87283e-fd12bf54-6ce86c63_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:35.470427Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:35.470446Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:35.470458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341706855442028:6783]
2025-10-09T21:46:35.470508Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:35.470514Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:35.470790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:35.470960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:35.471108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:35.471258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:35.471473Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:35.471520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:35.475625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341707217259046:7236], now have 1 active actors on pipe
2025-10-09T21:46:35.474905Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341707217259046:7236]
2025-10-09T21:46:35.475655Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 729, pipe: [1:7559341707217259046:7236]
2025-10-09T21:46:35.487020Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:35.487039Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:35.487042Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:35.487044Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:35.487056Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341706855442028:6783]
2025-10-09T21:46:35.487061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341706855442028:6783]
2025-10-09T21:46:35.487065Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341706855442028:6783]
2025-10-09T21:46:35.487112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:35.487120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.487123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:35.487279Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:35.487452Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:35.487470Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:35.487474Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:35.487898Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 730 sessionId: src_id|de862265-4a87283e-fd12bf54-6ce86c63_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:35.490788Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:35.657721Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:35.658597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:35.658614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:35.659395Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341707217259046:7236]
2025-10-09T21:46:35.659441Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:35.659444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:35.659503Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 730 reason: error at write: Disconnected sessionId: src_id|de862265-4a87283e-fd12bf54-6ce86c63_0
2025-10-09T21:46:35.659692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 730 sessionId: src_id|de862265-4a87283e-fd12bf54-6ce86c63_0 is DEAD
2025-10-09T21:46:35.659873Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:35.660223Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:35.662178Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de862265-4a87283e-fd12bf54-6ce86c63_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:35.662191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de862265-4a87283e-fd12bf54-6ce86c63_0] Write session will restart in 0.008742s
2025-10-09T21:46:35.662215Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de862265-4a87283e-fd12bf54-6ce86c63_0] Write session: Do CDS request
2025-10-09T21:46:35.662224Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de862265-4a87283e-fd12bf54-6ce86c63_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:35.672679Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|de862265-4a87283e-fd12bf54-6ce86c63_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:35.672784Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:35.672806Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 731
2025-10-09T21:46:35.673440Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 731 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:35.673501Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 731 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:35.673508Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=731 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:35.673511Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:35.674098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 731 sessionId:  describe result for acl check
2025-10-09T21:46:35.674135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:35.674137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:35.674138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:35.674144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341707217259138:7446] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:35.674715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341707217259138:7446] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:35.676916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341707217259138:7446] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:35.681007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341707217259138:7446] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:35.681024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341707217259138:7446] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:35.681027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341707217259138:7446] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:35.681037Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 731 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:35.704298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:35.707613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:35.707878Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:35.707883Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:35.707898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:35.707963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:35.708005Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341706855442347:6793]
2025-10-09T21:46:35.708605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:35.709909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:35.710001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:35.710389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:35.710471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:35.710480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:35.710481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:35.710483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:35.710484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:35.710485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:35.710486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.710487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.710488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.710495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:35.710507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:35.710511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:35.710514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:35.710518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:35.710521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:35.710524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.710528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.710532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.710541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:35.710550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:35.710556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:35.710562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:35.710567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:35.710572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:35.710577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:35.710582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:35.710587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:35.710592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:35.710596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:35.710602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:35.710607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.710611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.710616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.710621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.710625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.710633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.710660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:35.726914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:35.728643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:35.728650Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:35.728652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:35.728657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:35.728667Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 730 [2:7559341706855442347:6793]
2025-10-09T21:46:35.728684Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:35.728707Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:35.728712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:35.728715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:35.728717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:35.728738Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:35.728744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:35.728752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:35.728824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.728826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.728829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.728832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:35.728917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:35.728920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.728936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:35.728945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:35.728959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:35.728982Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341706855442341:6793]
2025-10-09T21:46:35.728985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341706855442341:6793]
2025-10-09T21:46:35.728987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341706855442341:6793]
2025-10-09T21:46:35.728996Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:35.729012Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:35.729028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:35.729031Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:35.729042Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:35.729047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:35.729049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:35.729076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341706855442341:6793]
2025-10-09T21:46:35.729079Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:35.729090Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:35.729108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341707217259258:7446], now have 1 active actors on pipe
2025-10-09T21:46:35.729121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341707217259049:2432], now have 1 active actors on pipe
2025-10-09T21:46:35.729631Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 730
2025-10-09T21:46:35.729640Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 730
2025-10-09T21:46:35.729960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:35.729977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:35.730025Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cef1fa23-7c30eba3-6637b5d5-87a226e6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:35.730048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:35.730052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:35.730055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:35.730069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:35.730085Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:35.730117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:35.732029Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 731 partition: 0 MaxSeqNo: 25 sessionId: src_id|cef1fa23-7c30eba3-6637b5d5-87a226e6_0
2025-10-09T21:46:35.731463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:35.731478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:35.731509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:35.734559Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de862265-4a87283e-fd12bf54-6ce86c63_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cef1fa23-7c30eba3-6637b5d5-87a226e6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:35.737229Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cef1fa23-7c30eba3-6637b5d5-87a226e6_0] Write session: try to update token
2025-10-09T21:46:35.737253Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cef1fa23-7c30eba3-6637b5d5-87a226e6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:35.760525Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:35.760546Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:35.760559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341706855442341:6793]
2025-10-09T21:46:35.760618Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:35.760861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:35.761097Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341707217259601:7236]
2025-10-09T21:46:35.761012Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:35.761158Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:35.761289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:35.761387Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:35.761470Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:35.761512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341707217259601:7236], now have 1 active actors on pipe
2025-10-09T21:46:35.761524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:35.761794Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 730, pipe: [1:7559341707217259601:7236]
2025-10-09T21:46:35.775507Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:35.775526Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:35.775529Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:35.775531Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:35.775543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341706855442341:6793]
2025-10-09T21:46:35.775548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341706855442341:6793]
2025-10-09T21:46:35.775552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341706855442341:6793]
2025-10-09T21:46:35.775596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:35.775604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:35.775607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:35.775622Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:35.775635Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:35.775639Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:35.775756Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:35.799037Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 731 sessionId: src_id|cef1fa23-7c30eba3-6637b5d5-87a226e6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:35.801770Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:35.951851Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:35.952661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:35.952673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:35.953401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cef1fa23-7c30eba3-6637b5d5-87a226e6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:35.953413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cef1fa23-7c30eba3-6637b5d5-87a226e6_0] Write session will restart in 0.006663s
2025-10-09T21:46:35.953446Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cef1fa23-7c30eba3-6637b5d5-87a226e6_0] Write session: Do CDS request
2025-10-09T21:46:35.953453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cef1fa23-7c30eba3-6637b5d5-87a226e6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:35.953144Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:35.952955Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:35.952973Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:35.952979Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341707217259601:7236]
2025-10-09T21:46:35.953000Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:35.953036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 731 reason: error at write: Disconnected sessionId: src_id|cef1fa23-7c30eba3-6637b5d5-87a226e6_0
2025-10-09T21:46:35.953224Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 731 sessionId: src_id|cef1fa23-7c30eba3-6637b5d5-87a226e6_0 is DEAD
2025-10-09T21:46:35.961649Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cef1fa23-7c30eba3-6637b5d5-87a226e6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:35.961766Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:35.961798Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 732
2025-10-09T21:46:35.961951Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 732 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:35.962001Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 732 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:35.962019Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=732 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:35.962026Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:35.962539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 732 sessionId:  describe result for acl check
2025-10-09T21:46:35.962576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:35.962579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:35.962581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:35.962587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341707217259622:7451] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:35.963140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341707217259622:7451] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:35.966084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341707217259622:7451] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:35.969697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341707217259622:7451] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:35.969718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341707217259622:7451] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:35.969722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341707217259622:7451] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:35.969732Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 732 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:36.019266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:36.022089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:36.022816Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:36.022821Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:36.022844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:36.022928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:36.022969Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341711150409952:6795]
2025-10-09T21:46:36.023543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:36.023956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:36.024029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:36.024373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:36.024449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.024453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.024454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.024455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.024457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.024458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.024459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.024460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.024462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.024467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.024478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.024481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.024485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.024488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.024491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.024494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.024498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.024501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.024509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.024517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.024525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.024530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.024534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.024539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.024553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.024558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.024562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.024566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.024570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.024574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.024579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.024583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.024587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.024593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.024597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.024601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.024638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:36.035242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:36.036448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:36.036467Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:36.036470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:36.036475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:36.036483Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 731 [2:7559341711150409952:6795]
2025-10-09T21:46:36.036498Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:36.036519Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:36.036523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:36.036527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:36.036529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:36.036549Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:36.036557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:36.036565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:36.036631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.036643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.036645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.036648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:36.036728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:36.036731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.036743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.036755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.036771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:36.036794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341711150409942:6795]
2025-10-09T21:46:36.036797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341711150409942:6795]
2025-10-09T21:46:36.036799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341711150409942:6795]
2025-10-09T21:46:36.036815Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:36.036831Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:36.036847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:36.036850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:36.036894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341711150409942:6795]
2025-10-09T21:46:36.036898Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:36.036996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:36.037008Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:36.037010Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:36.037016Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:36.037387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341707217259609:2432], now have 1 active actors on pipe
2025-10-09T21:46:36.037398Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 731
2025-10-09T21:46:36.037464Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 731
2025-10-09T21:46:36.037410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341707217259655:7451], now have 1 active actors on pipe
2025-10-09T21:46:36.040521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:36.040535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:36.040573Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3935e9a1-8f8fde06-5825fad0-d0946f31_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:36.040586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:36.040592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:36.040594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:36.040609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:36.040628Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:36.040650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:36.042806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:36.042824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:36.042859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:36.043124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 732 partition: 0 MaxSeqNo: 25 sessionId: src_id|3935e9a1-8f8fde06-5825fad0-d0946f31_0
2025-10-09T21:46:36.043627Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cef1fa23-7c30eba3-6637b5d5-87a226e6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3935e9a1-8f8fde06-5825fad0-d0946f31_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:36.046282Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3935e9a1-8f8fde06-5825fad0-d0946f31_0] Write session: try to update token
2025-10-09T21:46:36.046303Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3935e9a1-8f8fde06-5825fad0-d0946f31_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:36.052582Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341711512227456:7236]
2025-10-09T21:46:36.053057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341711512227456:7236], now have 1 active actors on pipe
2025-10-09T21:46:36.053122Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 731, pipe: [1:7559341711512227456:7236]
2025-10-09T21:46:36.055970Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:36.056003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:36.056005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:36.056008Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:36.056018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341711150409942:6795]
2025-10-09T21:46:36.056023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341711150409942:6795]
2025-10-09T21:46:36.056027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341711150409942:6795]
2025-10-09T21:46:36.056078Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:36.056080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:36.056091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.056093Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:36.056093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:36.056108Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:36.056249Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:36.067623Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 732 sessionId: src_id|3935e9a1-8f8fde06-5825fad0-d0946f31_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:36.070501Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:36.240036Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:36.241233Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341711512227456:7236]
2025-10-09T21:46:36.241293Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:36.241315Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:36.241321Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:36.241398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 732 reason: error at write: Disconnected sessionId: src_id|3935e9a1-8f8fde06-5825fad0-d0946f31_0
2025-10-09T21:46:36.240752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:36.240770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:36.241293Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:36.241672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 732 sessionId: src_id|3935e9a1-8f8fde06-5825fad0-d0946f31_0 is DEAD
2025-10-09T21:46:36.241825Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3935e9a1-8f8fde06-5825fad0-d0946f31_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:36.241837Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3935e9a1-8f8fde06-5825fad0-d0946f31_0] Write session will restart in 0.005049s
2025-10-09T21:46:36.241863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3935e9a1-8f8fde06-5825fad0-d0946f31_0] Write session: Do CDS request
2025-10-09T21:46:36.241887Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3935e9a1-8f8fde06-5825fad0-d0946f31_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:36.248604Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3935e9a1-8f8fde06-5825fad0-d0946f31_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:36.248723Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:36.248744Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 733
2025-10-09T21:46:36.248944Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 733 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:36.248988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 733 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:36.248994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=733 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:36.248998Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:36.249523Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 733 sessionId:  describe result for acl check
2025-10-09T21:46:36.249575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:36.249583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:36.249585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:36.249592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341711512227487:7456] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:36.250274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341711512227487:7456] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:36.251116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:36.252185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:36.252430Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:36.252444Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:36.252470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:36.252552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:36.252597Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341711150410012:6797]
2025-10-09T21:46:36.253250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:36.253726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:36.253796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:36.254184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:36.254253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.254254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.254256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.254257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.254258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.254260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.254261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.254262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.254264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.254270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.254282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.254286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.254291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.254295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.254299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.254303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.254307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.254311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.254318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.254327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.254342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.254348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.254353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.254358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.254363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.254369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.254386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.254402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.254413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341711512227487:7456] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:36.254407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.254412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.254418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.254425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.254429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.254434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.254438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.254443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.254472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:36.261392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341711512227487:7456] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:36.261417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341711512227487:7456] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:36.261420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341711512227487:7456] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:36.261431Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 733 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:36.262329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:36.263471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:36.263491Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:36.263495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:36.263500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:36.263508Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 732 [2:7559341711150410012:6797]
2025-10-09T21:46:36.263525Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:36.263546Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:36.263549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:36.263553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:36.263556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:36.263576Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:36.263582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:36.263590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:36.263658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.263669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.263672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.263675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:36.263754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:36.263764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.263779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.263793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.263812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:36.263847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341711150410007:6797]
2025-10-09T21:46:36.263851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341711150410007:6797]
2025-10-09T21:46:36.263856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341711150410007:6797]
2025-10-09T21:46:36.263871Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:36.263886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:36.263904Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:36.263907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:36.263940Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:36.263944Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:36.263947Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:36.263962Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341711150410007:6797]
2025-10-09T21:46:36.263966Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:36.264023Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:36.264174Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 732
2025-10-09T21:46:36.264222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 732
2025-10-09T21:46:36.264235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341711512227470:2432], now have 1 active actors on pipe
2025-10-09T21:46:36.264247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341711512227529:7456], now have 1 active actors on pipe
2025-10-09T21:46:36.264282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:36.264289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:36.264318Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e085c53e-d6d52a14-87711e35-c75128f7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:36.264335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:36.264351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:36.264355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:36.264367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:36.264385Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:36.264423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:36.264701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:36.264716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:36.264737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:36.264890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 733 partition: 0 MaxSeqNo: 25 sessionId: src_id|e085c53e-d6d52a14-87711e35-c75128f7_0
2025-10-09T21:46:36.265307Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3935e9a1-8f8fde06-5825fad0-d0946f31_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e085c53e-d6d52a14-87711e35-c75128f7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:36.267935Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e085c53e-d6d52a14-87711e35-c75128f7_0] Write session: try to update token
2025-10-09T21:46:36.267954Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e085c53e-d6d52a14-87711e35-c75128f7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:36.281433Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:36.281455Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:36.281459Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:36.281461Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:36.281473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341711150410007:6797]
2025-10-09T21:46:36.281479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341711150410007:6797]
2025-10-09T21:46:36.281484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341711150410007:6797]
2025-10-09T21:46:36.281541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:36.281550Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:36.281557Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:36.281559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.281562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:36.281562Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:36.281721Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:36.287514Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 733 sessionId: src_id|e085c53e-d6d52a14-87711e35-c75128f7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:36.290202Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:36.341288Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341711512227537:7236]
2025-10-09T21:46:36.341751Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 732, pipe: [1:7559341711512227537:7236]
2025-10-09T21:46:36.445387Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:36.445812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:36.445822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:36.445863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341711512227537:7236], now have 1 active actors on pipe
2025-10-09T21:46:36.447091Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:36.447207Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:36.448121Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e085c53e-d6d52a14-87711e35-c75128f7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:36.448131Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e085c53e-d6d52a14-87711e35-c75128f7_0] Write session will restart in 0.008051s
2025-10-09T21:46:36.448155Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e085c53e-d6d52a14-87711e35-c75128f7_0] Write session: Do CDS request
2025-10-09T21:46:36.448161Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e085c53e-d6d52a14-87711e35-c75128f7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:36.447237Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:36.447245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:36.447306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 733 reason: error at write: Disconnected sessionId: src_id|e085c53e-d6d52a14-87711e35-c75128f7_0
2025-10-09T21:46:36.447496Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 733 sessionId: src_id|e085c53e-d6d52a14-87711e35-c75128f7_0 is DEAD
2025-10-09T21:46:36.447644Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341711512227537:7236]
2025-10-09T21:46:36.458727Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e085c53e-d6d52a14-87711e35-c75128f7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:36.458911Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:36.458933Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 734
2025-10-09T21:46:36.459084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:36.459133Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 734 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:36.459170Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 734 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:36.459181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=734 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:36.459185Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:36.459616Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 734 sessionId:  describe result for acl check
2025-10-09T21:46:36.459661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:36.459668Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:36.459670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:36.459676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341711512227573:7466] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:36.460233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341711512227573:7466] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:36.460627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:36.460846Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:36.460858Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:36.460870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:36.460934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:36.460983Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341711150410084:6805]
2025-10-09T21:46:36.462467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341711512227573:7466] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:36.461605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:36.461950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:36.462011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:36.462348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:36.462395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.462396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.462397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.462399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.462400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.462402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.462403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.462405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.462409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.462415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.462427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.462431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.462434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.462438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.462442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.462446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.462450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.462454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.462460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.462467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.462473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.462478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.462482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.462488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.462493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.462499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.462503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.462509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.462513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.462517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.462522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.462526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.462530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.462534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.462538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.462543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.462569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:36.469015Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341711512227573:7466] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:36.469036Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341711512227573:7466] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:36.469039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341711512227573:7466] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:36.469049Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 734 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:36.470777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:36.472281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:36.472297Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:36.472303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:36.472309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:36.472317Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 733 [2:7559341711150410084:6805]
2025-10-09T21:46:36.472331Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:36.472351Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:36.472359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:36.472363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:36.472365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:36.472398Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:36.472410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:36.472417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:36.472482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.472490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.472492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.472495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:36.472613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:36.472623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.472639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.472651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.472669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:36.472697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341711150410077:6805]
2025-10-09T21:46:36.472718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341711150410077:6805]
2025-10-09T21:46:36.472721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341711150410077:6805]
2025-10-09T21:46:36.472736Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:36.472752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:36.472768Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:36.472771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:36.472791Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:36.472800Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:36.472802Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:36.472817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341711150410077:6805]
2025-10-09T21:46:36.472821Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:36.472897Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:36.473121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341711512227610:7466], now have 1 active actors on pipe
2025-10-09T21:46:36.473148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341711512227543:2432], now have 1 active actors on pipe
2025-10-09T21:46:36.473295Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 733
2025-10-09T21:46:36.473331Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 733
2025-10-09T21:46:36.473525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:36.473545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:36.473583Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1515c8b0-d1978fc6-edf9b32b-275223e6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:36.473593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:36.473597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:36.473599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:36.473611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:36.473625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:36.473644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:36.473889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:36.473914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:36.473935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:36.474137Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 734 partition: 0 MaxSeqNo: 25 sessionId: src_id|1515c8b0-d1978fc6-edf9b32b-275223e6_0
2025-10-09T21:46:36.474585Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e085c53e-d6d52a14-87711e35-c75128f7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1515c8b0-d1978fc6-edf9b32b-275223e6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:36.477158Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1515c8b0-d1978fc6-edf9b32b-275223e6_0] Write session: try to update token
2025-10-09T21:46:36.477177Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1515c8b0-d1978fc6-edf9b32b-275223e6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:36.489701Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:36.489717Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:36.489719Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:36.489720Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:36.489729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341711150410077:6805]
2025-10-09T21:46:36.489733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341711150410077:6805]
2025-10-09T21:46:36.489736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341711150410077:6805]
2025-10-09T21:46:36.489782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:36.489789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.489792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:36.489935Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:36.494356Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:36.494372Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:36.494376Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:36.495552Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 734 sessionId: src_id|1515c8b0-d1978fc6-edf9b32b-275223e6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:36.498331Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:36.547602Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341711512227628:7236]
2025-10-09T21:46:36.550300Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 733, pipe: [1:7559341711512227628:7236]
2025-10-09T21:46:36.658918Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:36.659458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:36.659479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:36.659527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341711512227628:7236], now have 1 active actors on pipe
2025-10-09T21:46:36.659850Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:36.659947Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341711512227628:7236]
2025-10-09T21:46:36.659972Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:36.659980Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:36.660037Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 734 reason: error at write: Disconnected sessionId: src_id|1515c8b0-d1978fc6-edf9b32b-275223e6_0
2025-10-09T21:46:36.660238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 734 sessionId: src_id|1515c8b0-d1978fc6-edf9b32b-275223e6_0 is DEAD
2025-10-09T21:46:36.660449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1515c8b0-d1978fc6-edf9b32b-275223e6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:36.660465Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1515c8b0-d1978fc6-edf9b32b-275223e6_0] Write session will restart in 0.005646s
2025-10-09T21:46:36.660492Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1515c8b0-d1978fc6-edf9b32b-275223e6_0] Write session: Do CDS request
2025-10-09T21:46:36.660503Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1515c8b0-d1978fc6-edf9b32b-275223e6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:36.660014Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:36.667857Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1515c8b0-d1978fc6-edf9b32b-275223e6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:36.668319Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:36.668338Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 735
2025-10-09T21:46:36.668536Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 735 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:36.668586Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 735 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:36.668592Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=735 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:36.668596Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:36.669183Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 735 sessionId:  describe result for acl check
2025-10-09T21:46:36.669217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:36.669220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:36.669222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:36.669228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341711512227651:7471] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:36.669794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341711512227651:7471] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:36.670002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:36.671187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:36.671400Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:36.671404Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:36.671417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:36.671478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:36.671514Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341711150410170:6815]
2025-10-09T21:46:36.672057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:36.673203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:36.673272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:36.673585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:36.673643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.673650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.673651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.673653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.673654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.673655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.673656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.673658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.673659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.673665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.673675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.673679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.673682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.673685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.673690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.673693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.673697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.673701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.673707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.673715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.673721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.673726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.673731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.673735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.673739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.673744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.673749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.673755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.673760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.673765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.673770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.673779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.673783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.673978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341711512227651:7471] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:36.673787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.673792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.673796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.673822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:36.680916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341711512227651:7471] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:36.680938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341711512227651:7471] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:36.680942Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341711512227651:7471] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:36.680953Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 735 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:36.682748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:36.683895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:36.683917Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:36.683920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:36.683925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:36.683933Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 734 [2:7559341711150410170:6815]
2025-10-09T21:46:36.683949Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:36.683975Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:36.683988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:36.683991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:36.683993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:36.684010Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:36.684023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:36.684031Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:36.684095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.684104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.684106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.684109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:36.684252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:36.684263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.684276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.684287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.684311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:36.684339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341711150410164:6815]
2025-10-09T21:46:36.684349Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341711150410164:6815]
2025-10-09T21:46:36.684351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341711150410164:6815]
2025-10-09T21:46:36.684367Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:36.684387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:36.684406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:36.684414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:36.684460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341711150410164:6815]
2025-10-09T21:46:36.684472Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:36.684813Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:36.684822Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:36.684825Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:36.684831Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:36.685319Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 734
2025-10-09T21:46:36.685362Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 734
2025-10-09T21:46:36.688403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341711512227690:7471], now have 1 active actors on pipe
2025-10-09T21:46:36.688419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341711512227631:2432], now have 1 active actors on pipe
2025-10-09T21:46:36.688452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:36.688461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:36.688504Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4f2f4064-8089ca2d-c129b415-a627180_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:36.688518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:36.688524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:36.688528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:36.688542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:36.688562Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:36.688584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:36.690518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:36.690535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:36.690566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:36.690828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 735 partition: 0 MaxSeqNo: 25 sessionId: src_id|4f2f4064-8089ca2d-c129b415-a627180_0
2025-10-09T21:46:36.691340Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1515c8b0-d1978fc6-edf9b32b-275223e6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4f2f4064-8089ca2d-c129b415-a627180_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:36.694658Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4f2f4064-8089ca2d-c129b415-a627180_0] Write session: try to update token
2025-10-09T21:46:36.694681Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4f2f4064-8089ca2d-c129b415-a627180_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:36.701353Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:36.701376Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:36.701379Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:36.701381Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:36.701391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341711150410164:6815]
2025-10-09T21:46:36.701396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341711150410164:6815]
2025-10-09T21:46:36.701400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341711150410164:6815]
2025-10-09T21:46:36.701449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:36.701456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.701460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:36.701608Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:36.704333Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:36.704348Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:36.704352Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:36.720949Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 735 sessionId: src_id|4f2f4064-8089ca2d-c129b415-a627180_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:36.723656Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:36.760095Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341711512227703:7236]
2025-10-09T21:46:36.760475Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 734, pipe: [1:7559341711512227703:7236]
2025-10-09T21:46:36.916800Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:36.917190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:36.917207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:36.917231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341711512227703:7236], now have 1 active actors on pipe
2025-10-09T21:46:36.917458Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:36.917463Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:36.917468Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341711512227703:7236]
2025-10-09T21:46:36.917493Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:36.917500Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:36.917582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 735 reason: error at write: Disconnected sessionId: src_id|4f2f4064-8089ca2d-c129b415-a627180_0
2025-10-09T21:46:36.917793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 735 sessionId: src_id|4f2f4064-8089ca2d-c129b415-a627180_0 is DEAD
2025-10-09T21:46:36.917982Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f2f4064-8089ca2d-c129b415-a627180_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:36.918009Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f2f4064-8089ca2d-c129b415-a627180_0] Write session will restart in 0.005915s
2025-10-09T21:46:36.918056Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f2f4064-8089ca2d-c129b415-a627180_0] Write session: Do CDS request
2025-10-09T21:46:36.918064Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f2f4064-8089ca2d-c129b415-a627180_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:36.925541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4f2f4064-8089ca2d-c129b415-a627180_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:36.925771Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:36.925794Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 736
2025-10-09T21:46:36.925920Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 736 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:36.925965Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 736 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:36.925971Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=736 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:36.925975Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:36.926660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 736 sessionId:  describe result for acl check
2025-10-09T21:46:36.926702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:36.926711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:36.926713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:36.926721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341711512227713:7478] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:36.927349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341711512227713:7478] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:36.930547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341711512227713:7478] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:36.933452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341711512227713:7478] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:36.933469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341711512227713:7478] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:36.933473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341711512227713:7478] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:36.933483Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 736 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:36.934154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:36.935237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:36.935458Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:36.935474Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:36.935490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:36.935557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:36.935598Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341711150410232:6817]
2025-10-09T21:46:36.936138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:36.936467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:36.936517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:36.936831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:36.936891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.936898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.936899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.936901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.936902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.936904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.936905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.936909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.936910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.936915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.936925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.936928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.936932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.936936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.936939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.936942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.936945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.936949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.936956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.936964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:36.936969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.936974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:36.936978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.936983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:36.936987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.936992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:36.936996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.937001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:36.937006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.937011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:36.937026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.937031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.937047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.937051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.937056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.937060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.937087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:36.945207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:36.946416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:36.946433Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:36.946436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:36.946441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:36.946450Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 735 [2:7559341711150410232:6817]
2025-10-09T21:46:36.946468Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:36.946490Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:36.946494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:36.946498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:36.946500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:36.946523Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:36.946541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:36.946550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:36.946623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.946633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.946635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.946641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:36.946731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:36.946743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.946760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:36.946772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:36.946790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:36.946820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341711150410227:6817]
2025-10-09T21:46:36.946827Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341711150410227:6817]
2025-10-09T21:46:36.946829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341711150410227:6817]
2025-10-09T21:46:36.946847Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:36.946869Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:36.946890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:36.946894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:36.946919Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:36.946939Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:36.946944Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341711150410227:6817]
2025-10-09T21:46:36.946947Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:36.946951Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:36.946958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:36.947387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341711512227762:7478], now have 1 active actors on pipe
2025-10-09T21:46:36.947400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341711512227706:2432], now have 1 active actors on pipe
2025-10-09T21:46:36.947750Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 735
2025-10-09T21:46:36.947788Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 735
2025-10-09T21:46:36.947972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:36.947993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:36.948028Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ece16bb1-f289dff6-70195201-f0ad0718_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:36.948044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:36.948048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:36.948051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:36.948066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:36.948081Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:36.948104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:36.948302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:36.948319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:36.948342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:36.948502Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 736 partition: 0 MaxSeqNo: 25 sessionId: src_id|ece16bb1-f289dff6-70195201-f0ad0718_0
2025-10-09T21:46:36.950135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f2f4064-8089ca2d-c129b415-a627180_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ece16bb1-f289dff6-70195201-f0ad0718_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:36.952949Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ece16bb1-f289dff6-70195201-f0ad0718_0] Write session: try to update token
2025-10-09T21:46:36.952971Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ece16bb1-f289dff6-70195201-f0ad0718_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:36.969952Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:36.969973Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:36.969975Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:36.969979Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:36.970003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341711150410227:6817]
2025-10-09T21:46:36.970008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341711150410227:6817]
2025-10-09T21:46:36.970012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341711150410227:6817]
2025-10-09T21:46:36.970116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:36.970124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:36.970127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:36.970264Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:36.970772Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:36.970786Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:36.970791Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:36.971344Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 736 sessionId: src_id|ece16bb1-f289dff6-70195201-f0ad0718_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:36.973835Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:37.017577Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341715807195071:7236]
2025-10-09T21:46:37.017927Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 735, pipe: [1:7559341715807195071:7236]
2025-10-09T21:46:37.184802Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:37.185121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:37.185137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:37.185186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341715807195071:7236], now have 1 active actors on pipe
2025-10-09T21:46:37.186679Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341715807195071:7236]
2025-10-09T21:46:37.186718Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:37.186721Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:37.186793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 736 reason: error at write: Disconnected sessionId: src_id|ece16bb1-f289dff6-70195201-f0ad0718_0
2025-10-09T21:46:37.186783Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:37.187153Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ece16bb1-f289dff6-70195201-f0ad0718_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:37.187165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ece16bb1-f289dff6-70195201-f0ad0718_0] Write session will restart in 0.007808s
2025-10-09T21:46:37.187193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ece16bb1-f289dff6-70195201-f0ad0718_0] Write session: Do CDS request
2025-10-09T21:46:37.187200Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ece16bb1-f289dff6-70195201-f0ad0718_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:37.186971Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:37.187004Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 736 sessionId: src_id|ece16bb1-f289dff6-70195201-f0ad0718_0 is DEAD
2025-10-09T21:46:37.196568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ece16bb1-f289dff6-70195201-f0ad0718_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:37.196752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:37.197052Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:37.197068Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 737
2025-10-09T21:46:37.197240Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 737 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:37.197280Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 737 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:37.197286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=737 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:37.197290Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:37.197866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 737 sessionId:  describe result for acl check
2025-10-09T21:46:37.197909Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:37.197911Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:37.197913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:37.197920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341715807195105:7483] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:37.198516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341715807195105:7483] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:37.197752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:37.197948Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:37.197952Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:37.197965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:37.198056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:37.198096Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341715445377587:6819]
2025-10-09T21:46:37.198658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:37.199208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:37.199546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:37.199884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:37.199957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:37.199966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:37.199967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:37.199968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:37.199969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:37.199970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:37.199971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.199972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.199974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.199979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:37.199989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:37.199993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:37.199997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:37.200000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:37.200004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:37.200008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.200012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.200015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.200025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:37.200033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:37.200039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:37.200045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:37.200049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:37.200054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:37.200059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:37.200064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:37.200069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:37.200074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:37.200078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:37.200083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:37.200089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.200095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.200099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.200106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.200110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.200114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.200142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:37.206426Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341715807195105:7483] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:37.208575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:37.209704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:37.209721Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:37.209724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:37.209729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:37.209737Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 736 [2:7559341715445377587:6819]
2025-10-09T21:46:37.209751Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:37.209773Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:37.209783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:37.209786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:37.209789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:37.209813Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:37.209819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:37.209826Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:37.209891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.209901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.209903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.209907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:37.209991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:37.209996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.210010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.210023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.210062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:37.210084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341715445377582:6819]
2025-10-09T21:46:37.210087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341715445377582:6819]
2025-10-09T21:46:37.210088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341715445377582:6819]
2025-10-09T21:46:37.210096Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:37.210107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:37.210110Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:37.210112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:37.210113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:37.210126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:37.210130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:37.210141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341715807195082:2432], now have 1 active actors on pipe
2025-10-09T21:46:37.210182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341715445377582:6819]
2025-10-09T21:46:37.210192Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:37.210202Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:37.212819Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 736
2025-10-09T21:46:37.221182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341715807195105:7483] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:37.221203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341715807195105:7483] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:37.221207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341715807195105:7483] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:37.221230Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 737 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:37.221940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341715807195147:7483], now have 1 active actors on pipe
2025-10-09T21:46:37.222064Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 736
2025-10-09T21:46:37.222205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:37.222223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:37.222259Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cef4f6de-528ac877-5f1e2db0-129a447d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:37.222275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:37.222285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:37.222289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:37.222324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:37.222352Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:37.222384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:37.222650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:37.222654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:37.222675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:37.222783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 737 partition: 0 MaxSeqNo: 25 sessionId: src_id|cef4f6de-528ac877-5f1e2db0-129a447d_0
2025-10-09T21:46:37.223168Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ece16bb1-f289dff6-70195201-f0ad0718_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cef4f6de-528ac877-5f1e2db0-129a447d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:37.226009Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cef4f6de-528ac877-5f1e2db0-129a447d_0] Write session: try to update token
2025-10-09T21:46:37.227201Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cef4f6de-528ac877-5f1e2db0-129a447d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:37.228745Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:37.228776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:37.228778Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:37.228780Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:37.228790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341715445377582:6819]
2025-10-09T21:46:37.228796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341715445377582:6819]
2025-10-09T21:46:37.228800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341715445377582:6819]
2025-10-09T21:46:37.228845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:37.228852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.228855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:37.228859Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:37.228865Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:37.228869Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:37.228999Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:37.243697Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 737 sessionId: src_id|cef4f6de-528ac877-5f1e2db0-129a447d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:37.246345Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:37.286400Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341715807195151:7236]
2025-10-09T21:46:37.286830Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 736, pipe: [1:7559341715807195151:7236]
2025-10-09T21:46:37.400044Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:37.400394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:37.400404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:37.400427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341715807195151:7236], now have 1 active actors on pipe
2025-10-09T21:46:37.401958Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:37.401977Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:37.402076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 737 reason: error at write: Disconnected sessionId: src_id|cef4f6de-528ac877-5f1e2db0-129a447d_0
2025-10-09T21:46:37.402759Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cef4f6de-528ac877-5f1e2db0-129a447d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:37.402791Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cef4f6de-528ac877-5f1e2db0-129a447d_0] Write session will restart in 0.006735s
2025-10-09T21:46:37.402816Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cef4f6de-528ac877-5f1e2db0-129a447d_0] Write session: Do CDS request
2025-10-09T21:46:37.402823Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cef4f6de-528ac877-5f1e2db0-129a447d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:37.402272Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 737 sessionId: src_id|cef4f6de-528ac877-5f1e2db0-129a447d_0 is DEAD
2025-10-09T21:46:37.402415Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:37.402450Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341715807195151:7236]
2025-10-09T21:46:37.401692Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:37.414649Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cef4f6de-528ac877-5f1e2db0-129a447d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:37.415032Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:37.415048Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 738
2025-10-09T21:46:37.415223Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 738 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:37.415262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 738 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:37.415271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=738 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:37.415274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:37.415824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 738 sessionId:  describe result for acl check
2025-10-09T21:46:37.415855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:37.415857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:37.415858Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:37.415865Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341715807195179:7487] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:37.416430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341715807195179:7487] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:37.414197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:37.415406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:37.415617Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:37.415621Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:37.415632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:37.415688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:37.415743Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341715445377649:6821]
2025-10-09T21:46:37.416337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:37.416893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:37.416950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:37.417277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:37.417326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:37.417327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:37.417328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:37.417329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:37.417330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:37.417332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:37.417333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.417334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.417336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.417341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:37.417351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:37.417355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:37.417359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:37.417363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:37.417367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:37.417371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.417374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.417378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.417386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:37.417395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:37.417401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:37.417406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:37.417410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:37.417415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:37.417420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:37.417425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:37.417429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:37.417434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:37.417440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:37.417444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:37.417450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.417455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.417459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.417463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.417468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.417472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.417503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:37.421473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341715807195179:7487] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:37.426586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341715807195179:7487] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:37.426605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341715807195179:7487] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:37.426610Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341715807195179:7487] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:37.426619Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 738 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:37.426802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:37.427926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:37.427945Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:37.427948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:37.427952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:37.427960Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 737 [2:7559341715445377649:6821]
2025-10-09T21:46:37.427978Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:37.427997Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:37.428006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:37.428009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:37.428012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:37.428031Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:37.428043Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:37.428051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:37.428112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.428121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.428123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.428125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:37.428197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:37.428205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.428217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.428228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.428258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:37.428284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341715445377644:6821]
2025-10-09T21:46:37.428286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341715445377644:6821]
2025-10-09T21:46:37.428288Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341715445377644:6821]
2025-10-09T21:46:37.428300Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:37.428316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:37.428331Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:37.428334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:37.428365Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:37.428368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:37.428370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:37.428372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341715445377644:6821]
2025-10-09T21:46:37.428375Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:37.428377Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:37.428521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341715807195154:2432], now have 1 active actors on pipe
2025-10-09T21:46:37.428533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341715807195217:7487], now have 1 active actors on pipe
2025-10-09T21:46:37.428662Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 737
2025-10-09T21:46:37.428700Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 737
2025-10-09T21:46:37.429199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:37.429215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:37.429249Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8ad847d-505c653e-71d55225-78b81bf2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:37.429265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:37.429270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:37.429273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:37.429284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:37.429297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:37.429328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:37.429745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:37.429754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:37.429771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:37.429905Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 738 partition: 0 MaxSeqNo: 25 sessionId: src_id|8ad847d-505c653e-71d55225-78b81bf2_0
2025-10-09T21:46:37.430339Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cef4f6de-528ac877-5f1e2db0-129a447d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8ad847d-505c653e-71d55225-78b81bf2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:37.433111Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8ad847d-505c653e-71d55225-78b81bf2_0] Write session: try to update token
2025-10-09T21:46:37.433133Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8ad847d-505c653e-71d55225-78b81bf2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:37.447041Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:37.447056Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:37.447059Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:37.447061Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:37.447070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341715445377644:6821]
2025-10-09T21:46:37.447075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341715445377644:6821]
2025-10-09T21:46:37.447079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341715445377644:6821]
2025-10-09T21:46:37.447130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:37.447138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.447137Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:37.447141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:37.447143Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:37.447149Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:37.447274Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:37.461550Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 738 sessionId: src_id|8ad847d-505c653e-71d55225-78b81bf2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:37.464098Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:37.503484Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341715807195225:7236]
2025-10-09T21:46:37.508798Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 737, pipe: [1:7559341715807195225:7236]
2025-10-09T21:46:37.618020Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:37.619076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:37.619092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:37.619114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341715807195225:7236], now have 1 active actors on pipe
2025-10-09T21:46:37.619496Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:37.619520Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:37.619576Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 738 reason: error at write: Disconnected sessionId: src_id|8ad847d-505c653e-71d55225-78b81bf2_0
2025-10-09T21:46:37.619612Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:37.619767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 738 sessionId: src_id|8ad847d-505c653e-71d55225-78b81bf2_0 is DEAD
2025-10-09T21:46:37.619912Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:37.619953Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341715807195225:7236]
2025-10-09T21:46:37.622229Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ad847d-505c653e-71d55225-78b81bf2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:37.622241Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ad847d-505c653e-71d55225-78b81bf2_0] Write session will restart in 0.009278s
2025-10-09T21:46:37.622268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ad847d-505c653e-71d55225-78b81bf2_0] Write session: Do CDS request
2025-10-09T21:46:37.622274Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ad847d-505c653e-71d55225-78b81bf2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:37.629668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:37.632772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:37.633053Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:37.633064Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:37.633080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:37.633153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:37.633199Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341715445377754:6837]
2025-10-09T21:46:37.633816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:37.634466Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8ad847d-505c653e-71d55225-78b81bf2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:37.634642Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:37.634656Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 739
2025-10-09T21:46:37.634223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:37.634281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:37.634612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:37.634659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:37.634660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:37.634661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:37.634662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:37.634663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:37.634676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:37.634677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.634678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.634679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.634685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:37.634694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:37.634697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:37.634700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:37.634703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:37.634706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:37.634709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.634712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.634715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.634722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:37.634729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:37.634743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:37.634748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:37.634752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:37.634756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:37.634760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:37.634764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:37.634768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:37.634772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:37.634777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:37.634783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:37.634787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.634801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.634804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.634808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.634812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.634815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.634840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:37.635310Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 739 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:37.635352Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 739 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:37.635358Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=739 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:37.635361Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:37.636091Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 739 sessionId:  describe result for acl check
2025-10-09T21:46:37.636127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:37.636128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:37.636130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:37.636135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341715807195281:7498] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:37.636696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341715807195281:7498] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:37.640073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341715807195281:7498] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:37.643847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341715807195281:7498] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:37.643863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341715807195281:7498] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:37.643868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341715807195281:7498] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:37.643878Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 739 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:37.643625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:37.644932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:37.644939Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:37.644942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:37.644946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:37.644953Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 738 [2:7559341715445377754:6837]
2025-10-09T21:46:37.644970Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:37.644988Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:37.644991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:37.644994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:37.644996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:37.645015Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:37.645021Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:37.645027Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:37.645094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.645096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.645100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.645105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:37.645186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:37.645190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.645202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.645214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.645231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:37.645254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341715807195311:7498], now have 1 active actors on pipe
2025-10-09T21:46:37.645267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341715807195251:2432], now have 1 active actors on pipe
2025-10-09T21:46:37.645275Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341715445377747:6837]
2025-10-09T21:46:37.645277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341715445377747:6837]
2025-10-09T21:46:37.645279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341715445377747:6837]
2025-10-09T21:46:37.645291Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:37.645307Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:37.645322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:37.645325Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:37.645368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341715445377747:6837]
2025-10-09T21:46:37.645370Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:37.645860Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 738
2025-10-09T21:46:37.645888Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 738
2025-10-09T21:46:37.646053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:37.646064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:37.646067Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:37.646074Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:37.646781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:37.646796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:37.646846Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|29f1123d-aa9dc594-f636f29d-2643209b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:37.646858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:37.646863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:37.646866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:37.646881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:37.646897Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:37.646918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:37.647327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:37.647337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:37.647366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:37.647617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 739 partition: 0 MaxSeqNo: 25 sessionId: src_id|29f1123d-aa9dc594-f636f29d-2643209b_0
2025-10-09T21:46:37.654225Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ad847d-505c653e-71d55225-78b81bf2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|29f1123d-aa9dc594-f636f29d-2643209b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:37.656525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29f1123d-aa9dc594-f636f29d-2643209b_0] Write session: try to update token
2025-10-09T21:46:37.656539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29f1123d-aa9dc594-f636f29d-2643209b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:37.688389Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:37.688409Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:37.688412Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:37.688414Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:37.688428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341715445377747:6837]
2025-10-09T21:46:37.688433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341715445377747:6837]
2025-10-09T21:46:37.688437Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341715445377747:6837]
2025-10-09T21:46:37.688482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:37.688491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.688494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:37.688642Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:37.690556Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:37.690573Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:37.690577Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:37.691315Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 739 sessionId: src_id|29f1123d-aa9dc594-f636f29d-2643209b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:37.694210Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:37.719520Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341715807195320:7236]
2025-10-09T21:46:37.721109Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 738, pipe: [1:7559341715807195320:7236]
2025-10-09T21:46:37.888078Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:37.888432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:37.888443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:37.888488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341715807195320:7236], now have 1 active actors on pipe
2025-10-09T21:46:37.888760Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:37.889189Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:37.889219Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:37.889264Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341715807195320:7236]
2025-10-09T21:46:37.889290Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 739 reason: status is not ok: tablet will be restarted right now sessionId: src_id|29f1123d-aa9dc594-f636f29d-2643209b_0
2025-10-09T21:46:37.889475Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 739 sessionId: src_id|29f1123d-aa9dc594-f636f29d-2643209b_0 is DEAD
2025-10-09T21:46:37.890114Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29f1123d-aa9dc594-f636f29d-2643209b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:37.890127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29f1123d-aa9dc594-f636f29d-2643209b_0] Write session will restart in 0.005853s
2025-10-09T21:46:37.890158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29f1123d-aa9dc594-f636f29d-2643209b_0] Write session: Do CDS request
2025-10-09T21:46:37.890164Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29f1123d-aa9dc594-f636f29d-2643209b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:37.897612Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29f1123d-aa9dc594-f636f29d-2643209b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:37.897909Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:37.897926Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 740
2025-10-09T21:46:37.898147Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 740 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:37.898188Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 740 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:37.898194Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=740 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:37.898198Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:37.898745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 740 sessionId:  describe result for acl check
2025-10-09T21:46:37.898785Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:37.898787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:37.898789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:37.898795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341715807195341:7503] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:37.899374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341715807195341:7503] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:37.901604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:37.903230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:37.903468Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:37.903479Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:37.903494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:37.903558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:37.903600Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341715445377817:6839]
2025-10-09T21:46:37.904172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:37.904594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:37.904654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:37.905011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:37.905080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:37.905086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:37.905087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:37.905089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:37.905091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:37.905092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:37.905093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.905094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.905096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.905102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:37.905115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:37.905119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:37.905124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:37.905128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:37.905133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:37.905137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.905141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.905143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.905151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:37.905159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:37.905166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:37.905173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:37.905180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:37.905189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:37.905195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:37.905201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:37.905208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:37.905214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:37.905220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:37.905227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:37.905238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.905245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.905252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.905258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.905265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.905269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.905296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:37.905131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341715807195341:7503] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:37.909194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341715807195341:7503] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:37.909203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341715807195341:7503] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:37.909207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341715807195341:7503] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:37.909216Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 740 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:37.915057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:37.916287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:37.916313Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:37.916317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:37.916322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:37.916330Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 739 [2:7559341715445377817:6839]
2025-10-09T21:46:37.916348Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:37.916373Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:37.916386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:37.916389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:37.916392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:37.916413Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:37.916430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:37.916443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:37.916515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.916525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.916529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.916533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:37.916688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:37.916696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.916711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:37.916723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:37.916741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:37.916772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341715807195326:2432], now have 1 active actors on pipe
2025-10-09T21:46:37.916783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341715807195391:7503], now have 1 active actors on pipe
2025-10-09T21:46:37.916790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341715445377812:6839]
2025-10-09T21:46:37.916792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341715445377812:6839]
2025-10-09T21:46:37.916794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341715445377812:6839]
2025-10-09T21:46:37.916809Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:37.916824Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:37.916840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:37.916839Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:37.916843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:37.916845Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:37.916848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:37.916825Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 739
2025-10-09T21:46:37.916870Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 739
2025-10-09T21:46:37.916893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341715445377812:6839]
2025-10-09T21:46:37.916907Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:37.916923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:37.917252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:37.917266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:37.917300Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d48e343c-197cd1f8-9e9a802-e7a278f8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:37.917313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:37.917319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:37.917322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:37.917333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:37.917351Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:37.917373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:37.918284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:37.918297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:37.918320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:37.918528Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 740 partition: 0 MaxSeqNo: 25 sessionId: src_id|d48e343c-197cd1f8-9e9a802-e7a278f8_0
2025-10-09T21:46:37.918966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29f1123d-aa9dc594-f636f29d-2643209b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d48e343c-197cd1f8-9e9a802-e7a278f8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:37.921683Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d48e343c-197cd1f8-9e9a802-e7a278f8_0] Write session: try to update token
2025-10-09T21:46:37.921707Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d48e343c-197cd1f8-9e9a802-e7a278f8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:37.937837Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:37.937866Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:37.937877Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341715445377812:6839]
2025-10-09T21:46:37.937931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:37.938048Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:37.938422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:37.938585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:37.938751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:37.938897Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:37.939108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:37.939159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:37.939570Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 740 sessionId: src_id|d48e343c-197cd1f8-9e9a802-e7a278f8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:37.942222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:37.942455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:37.942473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:37.942514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:37.942666Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:37.945444Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:37.945458Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:37.945461Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:37.945462Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:37.945472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341715445377812:6839]
2025-10-09T21:46:37.945475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341715445377812:6839]
2025-10-09T21:46:37.945478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341715445377812:6839]
2025-10-09T21:46:37.945516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:37.945523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:37.945525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:37.945525Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:37.945531Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:37.945535Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:37.945655Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:37.989375Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341715807195398:7236]
2025-10-09T21:46:37.989795Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 739, pipe: [1:7559341715807195398:7236]
2025-10-09T21:46:38.101904Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:38.110504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:38.110538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:38.110712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:38.110778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:38.110853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:38.110925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:38.110978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:38.110992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:38.111072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:38.111142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:38.111207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:38.111277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:38.111324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:38.111331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:38.111402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:38.111464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:38.111529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:38.111594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:38.115258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d48e343c-197cd1f8-9e9a802-e7a278f8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:38.115276Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d48e343c-197cd1f8-9e9a802-e7a278f8_0] Write session will restart in 0.007127s
2025-10-09T21:46:38.115298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d48e343c-197cd1f8-9e9a802-e7a278f8_0] Write session: Do CDS request
2025-10-09T21:46:38.115303Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d48e343c-197cd1f8-9e9a802-e7a278f8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:38.111638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:38.111650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:38.111722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:38.111783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:38.111845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:38.111916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:38.111959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:38.111962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:38.112036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:38.112095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:38.112159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:38.112222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:38.112266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:38.112268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:38.112337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:38.112397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:38.112458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:38.112517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:38.112558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:38.112560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:38.112632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:38.112691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:38.112756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:38.112818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:38.112861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:38.112864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:38.112928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:38.112983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:38.113043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:38.113102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:38.113143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:38.113145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:38.113207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:38.113260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:38.113322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:38.113381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:38.113422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:38.113425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:38.113510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:38.113575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:38.113642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:38.113702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:38.113744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:38.113747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:38.113810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341715807195398:7236], now have 1 active actors on pipe
2025-10-09T21:46:38.114640Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:38.114696Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:38.114699Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:38.114728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 740 reason: error at write: Disconnected sessionId: src_id|d48e343c-197cd1f8-9e9a802-e7a278f8_0
2025-10-09T21:46:38.114924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 740 sessionId: src_id|d48e343c-197cd1f8-9e9a802-e7a278f8_0 is DEAD
2025-10-09T21:46:38.115073Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341715807195398:7236]
2025-10-09T21:46:38.117392Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:38.123642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d48e343c-197cd1f8-9e9a802-e7a278f8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:38.125773Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:38.125789Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 741
2025-10-09T21:46:38.126472Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 741 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:38.126517Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 741 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:38.126523Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=741 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:38.126528Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:38.127135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 741 sessionId:  describe result for acl check
2025-10-09T21:46:38.127168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:38.127170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:38.127172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:38.127178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341720102162727:7507] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:38.127732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341720102162727:7507] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:38.129584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:38.131700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:38.131939Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:38.131943Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:38.131958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:38.132010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:38.132063Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341719740345178:6841]
2025-10-09T21:46:38.132605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:38.133072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:38.133119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:38.133408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:38.133453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:38.133454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:38.133455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:38.133456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:38.133457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:38.133458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:38.133460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.133461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.133462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.133466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:38.133660Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341720102162727:7507] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:38.133477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:38.133480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:38.133483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:38.133486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:38.133490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:38.133493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.133496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.133500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.133506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:38.133514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:38.133520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:38.133524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:38.133529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:38.133533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:38.133537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:38.133542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:38.133546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:38.133550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:38.133554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:38.133560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:38.133564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.133568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.133571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.133576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.133580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.133583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.133609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:38.141621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341720102162727:7507] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:38.141640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341720102162727:7507] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:38.141645Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341720102162727:7507] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:38.141657Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 741 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:38.142095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:38.143283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:38.143290Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:38.143293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:38.143298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:38.143305Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 740 [2:7559341719740345178:6841]
2025-10-09T21:46:38.143320Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:38.143342Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:38.143345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:38.143349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:38.143352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:38.143371Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:38.143377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:38.143385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:38.143461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.143464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.143466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.143468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:38.143544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:38.143546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.143560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.143569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.143583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:38.143603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341719740345173:6841]
2025-10-09T21:46:38.143606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341719740345173:6841]
2025-10-09T21:46:38.143610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341719740345173:6841]
2025-10-09T21:46:38.143623Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:38.143638Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:38.143652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:38.143655Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:38.143700Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341719740345173:6841]
2025-10-09T21:46:38.143702Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:38.144046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:38.144055Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:38.144057Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:38.144062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:38.144633Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 740
2025-10-09T21:46:38.144678Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 740
2025-10-09T21:46:38.144969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341720102162702:2432], now have 1 active actors on pipe
2025-10-09T21:46:38.144979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341720102162769:7507], now have 1 active actors on pipe
2025-10-09T21:46:38.144996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:38.145003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:38.145049Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|19e0f654-90cdca1f-c7e926b5-44906864_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:38.145060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:38.145065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:38.145067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:38.145078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:38.145094Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:38.145116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:38.147763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:38.147782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:38.147821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:38.148783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 741 partition: 0 MaxSeqNo: 25 sessionId: src_id|19e0f654-90cdca1f-c7e926b5-44906864_0
2025-10-09T21:46:38.150391Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d48e343c-197cd1f8-9e9a802-e7a278f8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|19e0f654-90cdca1f-c7e926b5-44906864_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:38.153059Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19e0f654-90cdca1f-c7e926b5-44906864_0] Write session: try to update token
2025-10-09T21:46:38.153081Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19e0f654-90cdca1f-c7e926b5-44906864_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:38.177862Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:38.177885Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:38.177891Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:38.177893Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:38.177903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341719740345173:6841]
2025-10-09T21:46:38.177909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341719740345173:6841]
2025-10-09T21:46:38.177913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341719740345173:6841]
2025-10-09T21:46:38.177964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:38.177971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.177974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:38.177995Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:38.178002Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:38.178006Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:38.178182Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:38.196727Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 741 sessionId: src_id|19e0f654-90cdca1f-c7e926b5-44906864_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:38.199395Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:38.218658Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341720102162781:7236]
2025-10-09T21:46:38.219116Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 740, pipe: [1:7559341720102162781:7236]
2025-10-09T21:46:38.367471Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:38.367883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:38.367901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:38.367925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341720102162781:7236], now have 1 active actors on pipe
2025-10-09T21:46:38.368231Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341720102162781:7236]
2025-10-09T21:46:38.368275Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:38.368352Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 741 reason: status is not ok: tablet will be restarted right now sessionId: src_id|19e0f654-90cdca1f-c7e926b5-44906864_0
2025-10-09T21:46:38.368141Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:38.368549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 741 sessionId: src_id|19e0f654-90cdca1f-c7e926b5-44906864_0 is DEAD
2025-10-09T21:46:38.368639Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:38.368724Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19e0f654-90cdca1f-c7e926b5-44906864_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:38.368735Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19e0f654-90cdca1f-c7e926b5-44906864_0] Write session will restart in 0.005882s
2025-10-09T21:46:38.368766Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19e0f654-90cdca1f-c7e926b5-44906864_0] Write session: Do CDS request
2025-10-09T21:46:38.368772Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19e0f654-90cdca1f-c7e926b5-44906864_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:38.375594Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19e0f654-90cdca1f-c7e926b5-44906864_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:38.376347Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:38.376363Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 742
2025-10-09T21:46:38.376677Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 742 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:38.376724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 742 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:38.376730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=742 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:38.376734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:38.377162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 742 sessionId:  describe result for acl check
2025-10-09T21:46:38.377193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:38.377195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:38.377196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:38.377202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341720102162809:7512] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:38.377771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341720102162809:7512] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:38.379100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:38.381405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:38.381667Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:38.381680Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:38.381697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:38.381769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:38.381815Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341719740345243:6843]
2025-10-09T21:46:38.382684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:38.383131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:38.383188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:38.381762Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341720102162809:7512] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:38.383525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:38.383607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:38.383608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:38.383609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:38.383610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:38.383611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:38.383612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:38.383612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.383613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.383614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.383621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:38.383631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:38.383634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:38.383638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:38.383642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:38.383647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:38.383651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.383655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.383659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.383666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:38.383674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:38.383680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:38.383686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:38.383690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:38.383695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:38.383699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:38.383703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:38.383708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:38.383713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:38.383720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:38.383725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:38.383730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.383737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.383742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.383747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.383751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.383756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.383785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:38.391643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341720102162809:7512] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:38.391664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341720102162809:7512] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:38.391670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341720102162809:7512] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:38.391682Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 742 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:38.392859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:38.394055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:38.394062Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:38.394065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:38.394070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:38.394078Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 741 [2:7559341719740345243:6843]
2025-10-09T21:46:38.394094Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:38.394117Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:38.394122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:38.394126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:38.394128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:38.394148Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:38.394154Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:38.394162Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:38.394228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.394230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.394234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.394237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:38.394318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:38.394324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.394338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.394349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.394364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:38.394386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341719740345237:6843]
2025-10-09T21:46:38.394388Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341719740345237:6843]
2025-10-09T21:46:38.394390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341719740345237:6843]
2025-10-09T21:46:38.394399Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:38.394413Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:38.394425Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:38.394428Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:38.394431Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:38.394444Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:38.394447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:38.394495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341720102162850:7512], now have 1 active actors on pipe
2025-10-09T21:46:38.394499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341720102162784:2432], now have 1 active actors on pipe
2025-10-09T21:46:38.394502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341719740345237:6843]
2025-10-09T21:46:38.394513Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:38.394523Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:38.395349Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 741
2025-10-09T21:46:38.395353Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 741
2025-10-09T21:46:38.395539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:38.395563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:38.395597Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|90f79510-64bda367-32141b2-d112ef91_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:38.395607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:38.395615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:38.395618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:38.395630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:38.395657Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:38.395677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:38.396018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:38.396027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:38.396045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:38.396252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 742 partition: 0 MaxSeqNo: 25 sessionId: src_id|90f79510-64bda367-32141b2-d112ef91_0
2025-10-09T21:46:38.396751Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19e0f654-90cdca1f-c7e926b5-44906864_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|90f79510-64bda367-32141b2-d112ef91_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:38.400074Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90f79510-64bda367-32141b2-d112ef91_0] Write session: try to update token
2025-10-09T21:46:38.400106Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90f79510-64bda367-32141b2-d112ef91_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:38.413402Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:38.413426Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:38.413437Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341719740345237:6843]
2025-10-09T21:46:38.413492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:38.413499Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:38.413678Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:38.413834Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:38.414011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:38.414207Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:38.414577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:38.414641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:38.424991Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 742 sessionId: src_id|90f79510-64bda367-32141b2-d112ef91_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:38.428081Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:38.428098Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:38.428100Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:38.428103Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:38.428118Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341719740345237:6843]
2025-10-09T21:46:38.428126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341719740345237:6843]
2025-10-09T21:46:38.428131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341719740345237:6843]
2025-10-09T21:46:38.428184Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:38.428189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:38.428196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.428196Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:38.428199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:38.428201Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:38.428339Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:38.428046Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:38.469174Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341720102162858:7236]
2025-10-09T21:46:38.469854Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 741, pipe: [1:7559341720102162858:7236]
2025-10-09T21:46:38.589014Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:38.589497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:38.589508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:38.589531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341720102162858:7236], now have 1 active actors on pipe
2025-10-09T21:46:38.589853Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:38.589867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:38.589937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 742 reason: error at write: Disconnected sessionId: src_id|90f79510-64bda367-32141b2-d112ef91_0
2025-10-09T21:46:38.590096Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:38.590138Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341720102162858:7236]
2025-10-09T21:46:38.590159Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 742 sessionId: src_id|90f79510-64bda367-32141b2-d112ef91_0 is DEAD
2025-10-09T21:46:38.590742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90f79510-64bda367-32141b2-d112ef91_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:38.590753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90f79510-64bda367-32141b2-d112ef91_0] Write session will restart in 0.008834s
2025-10-09T21:46:38.590777Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90f79510-64bda367-32141b2-d112ef91_0] Write session: Do CDS request
2025-10-09T21:46:38.590786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90f79510-64bda367-32141b2-d112ef91_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:38.590515Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:38.598945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:38.600901Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90f79510-64bda367-32141b2-d112ef91_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:38.600972Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:38.600987Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 743
2025-10-09T21:46:38.601184Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 743 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:38.601226Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 743 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:38.601231Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=743 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:38.601235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:38.601827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 743 sessionId:  describe result for acl check
2025-10-09T21:46:38.601885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:38.601887Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:38.601888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:38.601894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341720102162894:7518] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:38.602390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341720102162894:7518] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:38.603105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:38.603378Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:38.603391Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:38.603405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:38.603472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:38.603517Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341719740345326:6853]
2025-10-09T21:46:38.604107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:38.605702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:38.605760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:38.606142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:38.606204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:38.606205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:38.606206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:38.606207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:38.606209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:38.606210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:38.606211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.606214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.606215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.606222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:38.606231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:38.606234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:38.606240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:38.606243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:38.606247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:38.606250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.606254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.606257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.606263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:38.606272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:38.606279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:38.606284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:38.606288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:38.606293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:38.606298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:38.606304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:38.606308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:38.606313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:38.606318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:38.606323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:38.606329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.606334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.606338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.606343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.606348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.606352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.606381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:38.608027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341720102162894:7518] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:38.618278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341720102162894:7518] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:38.618298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341720102162894:7518] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:38.618302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341720102162894:7518] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:38.618313Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 743 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:38.618359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:38.619557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:38.619579Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:38.619581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:38.619585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:38.619592Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 742 [2:7559341719740345326:6853]
2025-10-09T21:46:38.619607Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:38.619626Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:38.619629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:38.619632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:38.619635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:38.619654Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:38.619661Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:38.619668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:38.619750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.619760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.619761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.619764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:38.619840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:38.619847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.619863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.619887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.619911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:38.619933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341720102162861:2432], now have 1 active actors on pipe
2025-10-09T21:46:38.619937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341720102162941:7518], now have 1 active actors on pipe
2025-10-09T21:46:38.619944Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341719740345320:6853]
2025-10-09T21:46:38.619947Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341719740345320:6853]
2025-10-09T21:46:38.619949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341719740345320:6853]
2025-10-09T21:46:38.619961Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:38.619977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:38.619994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:38.619997Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:38.620038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341719740345320:6853]
2025-10-09T21:46:38.620041Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:38.620072Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 742
2025-10-09T21:46:38.620107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 742
2025-10-09T21:46:38.620143Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:38.620163Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:38.620166Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:38.620173Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:38.620431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:38.620442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:38.620484Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|251d0019-54722443-ed19a694-bdfdabcf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:38.620493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:38.620498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:38.620501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:38.620512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:38.620525Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:38.620552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:38.620976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:38.620989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:38.621010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:38.621138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 743 partition: 0 MaxSeqNo: 25 sessionId: src_id|251d0019-54722443-ed19a694-bdfdabcf_0
2025-10-09T21:46:38.621519Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90f79510-64bda367-32141b2-d112ef91_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|251d0019-54722443-ed19a694-bdfdabcf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:38.624229Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|251d0019-54722443-ed19a694-bdfdabcf_0] Write session: try to update token
2025-10-09T21:46:38.624252Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|251d0019-54722443-ed19a694-bdfdabcf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:38.640773Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:38.640792Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:38.640794Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:38.640797Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:38.640806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341719740345320:6853]
2025-10-09T21:46:38.640810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341719740345320:6853]
2025-10-09T21:46:38.640813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341719740345320:6853]
2025-10-09T21:46:38.640861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:38.640869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.640872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:38.640987Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:38.640996Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:38.641012Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:38.641015Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:38.644267Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 743 sessionId: src_id|251d0019-54722443-ed19a694-bdfdabcf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:38.647002Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:38.689813Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341720102162959:7236]
2025-10-09T21:46:38.690273Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 742, pipe: [1:7559341720102162959:7236]
2025-10-09T21:46:38.807597Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:38.807952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:38.807968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:38.808018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341720102162959:7236], now have 1 active actors on pipe
2025-10-09T21:46:38.808301Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:38.809183Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:38.809266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 743 reason: status is not ok: tablet will be restarted right now sessionId: src_id|251d0019-54722443-ed19a694-bdfdabcf_0
2025-10-09T21:46:38.809447Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 743 sessionId: src_id|251d0019-54722443-ed19a694-bdfdabcf_0 is DEAD
2025-10-09T21:46:38.809628Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341720102162959:7236]
2025-10-09T21:46:38.809642Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:38.810238Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|251d0019-54722443-ed19a694-bdfdabcf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:38.810251Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|251d0019-54722443-ed19a694-bdfdabcf_0] Write session will restart in 0.009252s
2025-10-09T21:46:38.810280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|251d0019-54722443-ed19a694-bdfdabcf_0] Write session: Do CDS request
2025-10-09T21:46:38.810287Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|251d0019-54722443-ed19a694-bdfdabcf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:38.819578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:38.820645Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|251d0019-54722443-ed19a694-bdfdabcf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:38.821011Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:38.821028Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 744
2025-10-09T21:46:38.821160Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 744 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:38.821201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 744 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:38.821207Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=744 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:38.821211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:38.821944Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 744 sessionId:  describe result for acl check
2025-10-09T21:46:38.821998Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:38.822000Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:38.822002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:38.822009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341720102162993:7528] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:38.822609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341720102162993:7528] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:38.824983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:38.825256Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:38.825267Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:38.825280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:38.825342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:38.825380Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341719740345419:6861]
2025-10-09T21:46:38.825972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:38.826332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341720102162993:7528] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:38.826420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:38.826466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:38.826760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:38.826810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:38.826815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:38.826817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:38.826818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:38.826819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:38.826823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:38.826825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.826826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.826828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.826833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:38.826843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:38.826847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:38.826850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:38.826854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:38.826857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:38.826862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.826865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.826868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.826876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:38.826884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:38.826890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:38.826895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:38.826899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:38.826904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:38.826907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:38.826912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:38.826916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:38.826921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:38.826926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:38.826931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:38.826935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.826940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.826944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.826949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.826953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.826964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.826992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:38.833798Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341720102162993:7528] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:38.833832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341720102162993:7528] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:38.833837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341720102162993:7528] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:38.833848Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 744 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:38.834203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:38.835388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:38.835405Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:38.835407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:38.835412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:38.835421Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 743 [2:7559341719740345419:6861]
2025-10-09T21:46:38.835437Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:38.835460Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:38.835471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:38.835474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:38.835477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:38.835498Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:38.835510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:38.835517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:38.835671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.835680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.835682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.835685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:38.835761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:38.835769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.835781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:38.835791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:38.835807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:38.835828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341719740345414:6861]
2025-10-09T21:46:38.835830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341719740345414:6861]
2025-10-09T21:46:38.835832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341719740345414:6861]
2025-10-09T21:46:38.835841Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:38.835856Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:38.835866Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:38.835869Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:38.835870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:38.835872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:38.835873Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:38.835885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341720102163032:7528], now have 1 active actors on pipe
2025-10-09T21:46:38.835888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341720102162964:2432], now have 1 active actors on pipe
2025-10-09T21:46:38.835925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341719740345414:6861]
2025-10-09T21:46:38.835933Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:38.835951Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:38.836240Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 743
2025-10-09T21:46:38.836272Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 743
2025-10-09T21:46:38.836373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:38.836385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:38.836416Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|23953e3d-6ebb2a82-12bb018-31736957_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:38.836425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:38.836431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:38.836433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:38.836445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:38.836457Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:38.836473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:38.836717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:38.836726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:38.836741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:38.836852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 744 partition: 0 MaxSeqNo: 25 sessionId: src_id|23953e3d-6ebb2a82-12bb018-31736957_0
2025-10-09T21:46:38.838293Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|251d0019-54722443-ed19a694-bdfdabcf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|23953e3d-6ebb2a82-12bb018-31736957_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:38.840837Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23953e3d-6ebb2a82-12bb018-31736957_0] Write session: try to update token
2025-10-09T21:46:38.840858Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23953e3d-6ebb2a82-12bb018-31736957_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:38.857176Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:38.857194Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:38.857205Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341719740345414:6861]
2025-10-09T21:46:38.857261Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:38.857268Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:38.857541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:38.857706Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:38.857835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:38.857973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:38.858280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:38.858335Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:38.870001Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 744 sessionId: src_id|23953e3d-6ebb2a82-12bb018-31736957_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:38.871140Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:38.871162Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:38.871164Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:38.871166Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:38.871176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341719740345414:6861]
2025-10-09T21:46:38.871180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341719740345414:6861]
2025-10-09T21:46:38.871183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341719740345414:6861]
2025-10-09T21:46:38.871233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:38.871244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:38.871247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:38.871386Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:38.872868Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:38.872883Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:38.872887Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:38.872676Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:38.910687Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341720102163039:7236]
2025-10-09T21:46:38.911264Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 743, pipe: [1:7559341720102163039:7236]
2025-10-09T21:46:39.034479Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:39.034848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:39.034866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:39.034893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341720102163039:7236], now have 1 active actors on pipe
2025-10-09T21:46:39.035228Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:39.035777Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23953e3d-6ebb2a82-12bb018-31736957_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:39.035791Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23953e3d-6ebb2a82-12bb018-31736957_0] Write session will restart in 0.009790s
2025-10-09T21:46:39.035836Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23953e3d-6ebb2a82-12bb018-31736957_0] Write session: Do CDS request
2025-10-09T21:46:39.035843Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23953e3d-6ebb2a82-12bb018-31736957_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:39.035322Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:39.035337Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:39.035403Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 744 reason: error at write: Disconnected sessionId: src_id|23953e3d-6ebb2a82-12bb018-31736957_0
2025-10-09T21:46:39.035603Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 744 sessionId: src_id|23953e3d-6ebb2a82-12bb018-31736957_0 is DEAD
2025-10-09T21:46:39.035654Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:39.035702Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341720102163039:7236]
2025-10-09T21:46:39.044997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:39.046648Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23953e3d-6ebb2a82-12bb018-31736957_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:39.046714Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:39.046743Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 745
2025-10-09T21:46:39.046934Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 745 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:39.046988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 745 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:39.046998Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=745 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:39.047002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:39.046240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:39.046497Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:39.046502Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:39.046516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:39.046584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:39.046630Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341724035312782:6863]
2025-10-09T21:46:39.047232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:39.047512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 745 sessionId:  describe result for acl check
2025-10-09T21:46:39.047555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:39.047564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:39.047566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:39.047574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341724397130373:7532] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:39.047887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:39.047948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:39.048123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341724397130373:7532] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:39.048282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:39.048348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.048354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.048356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.048357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.048358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.048360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.048361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.048362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.048363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.048369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.048379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.048383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.048387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.048391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.048395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.048399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.048402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.048406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.048426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.048435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.048441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.048447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.048452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.048457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.048462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.048467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.048481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.048486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.048489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.048493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.048498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.048503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.048507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.048512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.048516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.048521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.048548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:39.050649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341724397130373:7532] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:39.056607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341724397130373:7532] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:39.056625Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341724397130373:7532] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:39.056636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341724397130373:7532] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:39.056647Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 745 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:39.057366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:39.058830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:39.058854Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:39.058857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:39.058861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:39.058869Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 744 [2:7559341724035312782:6863]
2025-10-09T21:46:39.058887Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:39.058905Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:39.058909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:39.058912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:39.058917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:39.058935Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:39.058942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:39.058949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:39.059016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.059025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.059027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.059030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:39.059107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:39.059115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.059130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.059148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.059165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:39.059273Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 744
2025-10-09T21:46:39.059308Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 744
2025-10-09T21:46:39.059192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341724035312777:6863]
2025-10-09T21:46:39.059200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341724035312777:6863]
2025-10-09T21:46:39.059202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341724035312777:6863]
2025-10-09T21:46:39.059213Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:39.059224Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:39.059227Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:39.059229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341724397130408:7532], now have 1 active actors on pipe
2025-10-09T21:46:39.059233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341724397130342:2432], now have 1 active actors on pipe
2025-10-09T21:46:39.059242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:39.059242Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:39.059256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:39.059260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:39.059306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341724035312777:6863]
2025-10-09T21:46:39.059309Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:39.059338Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:39.059346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:39.059352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:39.059387Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|905f36eb-795f5496-985020a9-7a0c573e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:39.059395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:39.059399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:39.059401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:39.059412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:39.059422Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:39.059439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:39.059870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:39.059895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:39.059917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:39.060095Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 745 partition: 0 MaxSeqNo: 25 sessionId: src_id|905f36eb-795f5496-985020a9-7a0c573e_0
2025-10-09T21:46:39.060464Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23953e3d-6ebb2a82-12bb018-31736957_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|905f36eb-795f5496-985020a9-7a0c573e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:39.063208Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|905f36eb-795f5496-985020a9-7a0c573e_0] Write session: try to update token
2025-10-09T21:46:39.063229Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|905f36eb-795f5496-985020a9-7a0c573e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:39.077206Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:39.077226Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:39.077237Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341724035312777:6863]
2025-10-09T21:46:39.077283Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:39.077289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:39.077464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:39.077614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:39.077756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:39.077896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:39.078162Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:39.078206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:39.082435Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 745 sessionId: src_id|905f36eb-795f5496-985020a9-7a0c573e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:39.084944Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:39.085271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:39.085306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:39.085354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:39.085574Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:39.093618Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:39.093646Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:39.093649Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:39.093651Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:39.093662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341724035312777:6863]
2025-10-09T21:46:39.093671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341724035312777:6863]
2025-10-09T21:46:39.093675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341724035312777:6863]
2025-10-09T21:46:39.093727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:39.093740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.093740Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:39.093744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:39.093746Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:39.093751Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:39.093890Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:39.135922Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341724397130422:7236]
2025-10-09T21:46:39.136581Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 744, pipe: [1:7559341724397130422:7236]
2025-10-09T21:46:39.264767Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:39.273028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:39.273069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:39.273256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:39.273326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:39.273396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:39.273466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:39.273510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:39.273516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:39.273586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:39.273645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:39.273713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:39.273781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:39.273826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:39.273833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:39.273903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:39.273962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:39.274140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:39.274208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:39.274575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:39.274590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:39.274661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:39.274722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:39.274786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:39.274848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:39.274888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:39.274891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:39.274960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:39.275025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:39.275089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:39.275158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:39.275196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:39.275199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:39.275267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:39.275325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:39.275382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:39.275441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:39.275484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:39.275487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:39.275552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:39.275608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:39.275674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:39.275738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:39.275780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:39.275783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:39.275853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:39.275918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:39.275982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:39.276044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:39.276086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:39.276090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:39.276159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:39.277762Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:39.276218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:39.277772Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:39.276280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:39.276346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:39.277775Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:39.276388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:39.276395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:39.277818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 745 reason: error at write: Disconnected sessionId: src_id|905f36eb-795f5496-985020a9-7a0c573e_0
2025-10-09T21:46:39.276462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:39.277835Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341724397130422:7236]
2025-10-09T21:46:39.276519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:39.276580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:39.276637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:39.276680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:39.276683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:39.276767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341724397130422:7236], now have 1 active actors on pipe
2025-10-09T21:46:39.277811Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:39.278207Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|905f36eb-795f5496-985020a9-7a0c573e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:39.278219Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|905f36eb-795f5496-985020a9-7a0c573e_0] Write session will restart in 0.006337s
2025-10-09T21:46:39.278243Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|905f36eb-795f5496-985020a9-7a0c573e_0] Write session: Do CDS request
2025-10-09T21:46:39.278252Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|905f36eb-795f5496-985020a9-7a0c573e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:39.278108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 745 sessionId: src_id|905f36eb-795f5496-985020a9-7a0c573e_0 is DEAD
2025-10-09T21:46:39.285713Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|905f36eb-795f5496-985020a9-7a0c573e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:39.285842Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:39.285861Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 746
2025-10-09T21:46:39.286252Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 746 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:39.286310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 746 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:39.286323Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=746 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:39.286327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:39.286899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 746 sessionId:  describe result for acl check
2025-10-09T21:46:39.286936Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:39.286938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:39.286940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:39.286948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341724397130435:7537] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:39.287524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341724397130435:7537] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:39.291892Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341724397130435:7537] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:39.295263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341724397130435:7537] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:39.295280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341724397130435:7537] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:39.295284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341724397130435:7537] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:39.295293Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 746 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:39.295932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:39.297391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:39.297608Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:39.297622Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:39.297637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:39.297700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:39.297752Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341724035312854:6865]
2025-10-09T21:46:39.298344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:39.298701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:39.298770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:39.299118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:39.299191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.299192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.299193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.299195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.299196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.299197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.299199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.299200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.299202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.299207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.299218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.299222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.299226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.299230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.299235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.299239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.299242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.299247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.299254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.299263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.299270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.299275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.299280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.299285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.299289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.299297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.299301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.299307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.299311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.299316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.299321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.299327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.299332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.299336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.299341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.299346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.299375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:39.307634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:39.308904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:39.308926Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:39.308929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:39.308934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:39.308945Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 745 [2:7559341724035312854:6865]
2025-10-09T21:46:39.308963Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:39.308986Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:39.308991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:39.308994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:39.308998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:39.309017Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:39.309024Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:39.309031Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:39.309099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.309111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.309112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.309115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:39.309197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:39.309205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.309221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.309234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.309252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:39.309276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341724035312847:6865]
2025-10-09T21:46:39.309278Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341724035312847:6865]
2025-10-09T21:46:39.309280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341724035312847:6865]
2025-10-09T21:46:39.309296Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:39.309311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:39.309327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:39.309330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:39.309373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341724035312847:6865]
2025-10-09T21:46:39.309381Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:39.311429Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:39.311443Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:39.311445Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:39.311469Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:39.311669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341724397130427:2432], now have 1 active actors on pipe
2025-10-09T21:46:39.311677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341724397130493:7537], now have 1 active actors on pipe
2025-10-09T21:46:39.312080Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 745
2025-10-09T21:46:39.312138Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 745
2025-10-09T21:46:39.312658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:39.312675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:39.312715Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6a1ce32d-feba661a-721e42c5-4d351cbf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:39.312729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:39.312736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:39.312738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:39.312753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:39.312771Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:39.312795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:39.313062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:39.313077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:39.313102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:39.313215Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 746 partition: 0 MaxSeqNo: 25 sessionId: src_id|6a1ce32d-feba661a-721e42c5-4d351cbf_0
2025-10-09T21:46:39.313738Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|905f36eb-795f5496-985020a9-7a0c573e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6a1ce32d-feba661a-721e42c5-4d351cbf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:39.316425Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6a1ce32d-feba661a-721e42c5-4d351cbf_0] Write session: try to update token
2025-10-09T21:46:39.316444Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6a1ce32d-feba661a-721e42c5-4d351cbf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:39.331103Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:39.331124Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:39.331127Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:39.331128Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:39.331139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341724035312847:6865]
2025-10-09T21:46:39.331144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341724035312847:6865]
2025-10-09T21:46:39.331147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341724035312847:6865]
2025-10-09T21:46:39.331194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:39.331202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.331206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:39.331220Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:39.331227Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:39.331231Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:39.331334Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:39.334376Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 746 sessionId: src_id|6a1ce32d-feba661a-721e42c5-4d351cbf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:39.337120Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:39.377973Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341724397130507:7236]
2025-10-09T21:46:39.378630Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 745, pipe: [1:7559341724397130507:7236]
2025-10-09T21:46:39.498218Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:39.498560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:39.498579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:39.498627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341724397130507:7236], now have 1 active actors on pipe
2025-10-09T21:46:39.498925Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:39.498981Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341724397130507:7236]
2025-10-09T21:46:39.499000Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:39.499003Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:39.499064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 746 reason: error at write: Disconnected sessionId: src_id|6a1ce32d-feba661a-721e42c5-4d351cbf_0
2025-10-09T21:46:39.498910Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:39.499238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 746 sessionId: src_id|6a1ce32d-feba661a-721e42c5-4d351cbf_0 is DEAD
2025-10-09T21:46:39.502125Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6a1ce32d-feba661a-721e42c5-4d351cbf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:39.502138Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6a1ce32d-feba661a-721e42c5-4d351cbf_0] Write session will restart in 0.005235s
2025-10-09T21:46:39.502173Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6a1ce32d-feba661a-721e42c5-4d351cbf_0] Write session: Do CDS request
2025-10-09T21:46:39.502180Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6a1ce32d-feba661a-721e42c5-4d351cbf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:39.511981Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6a1ce32d-feba661a-721e42c5-4d351cbf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:39.512205Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:39.512224Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 747
2025-10-09T21:46:39.512378Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 747 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:39.512425Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 747 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:39.512431Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=747 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:39.512436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:39.513248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 747 sessionId:  describe result for acl check
2025-10-09T21:46:39.513303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:39.513305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:39.513307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:39.513315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341724397130543:7541] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:39.513892Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341724397130543:7541] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:39.513850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:39.515181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:39.515415Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:39.515420Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:39.515433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:39.515495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:39.515537Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341724035312923:6867]
2025-10-09T21:46:39.516207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:39.516667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:39.516733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:39.517053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:39.517115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.517121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.517122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.517123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.517124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.517126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.517127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.517129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.517130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.517136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.517147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.517150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.517154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.517159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.517162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.517166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.517170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.517174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.517180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.517187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.517192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.517197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.517203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.517208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.517212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.517217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.517220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.516996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341724397130543:7541] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:39.517225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.517228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.517233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.517237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.517241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.517245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.517250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.517255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.517259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.517286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:39.521975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341724397130543:7541] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:39.522211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341724397130543:7541] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:39.522219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341724397130543:7541] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:39.522233Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 747 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:39.525040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:39.526674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:39.526690Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:39.526693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:39.526698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:39.526705Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 746 [2:7559341724035312923:6867]
2025-10-09T21:46:39.526723Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:39.526742Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:39.526745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:39.526749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:39.526756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:39.526784Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:39.526790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:39.526797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:39.526878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.526889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.526891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.526893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:39.526977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:39.526986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.527001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.527014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.527031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:39.527058Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341724035312918:6867]
2025-10-09T21:46:39.527065Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341724035312918:6867]
2025-10-09T21:46:39.527067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341724035312918:6867]
2025-10-09T21:46:39.527076Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:39.527086Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:39.527092Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:39.527095Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:39.527097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:39.527107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:39.527120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:39.527133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341724397130510:2432], now have 1 active actors on pipe
2025-10-09T21:46:39.527136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341724397130581:7541], now have 1 active actors on pipe
2025-10-09T21:46:39.527178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341724035312918:6867]
2025-10-09T21:46:39.527192Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:39.527195Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 746
2025-10-09T21:46:39.527242Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 746
2025-10-09T21:46:39.527288Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:39.527457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:39.527470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:39.527503Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c53a770e-e9e073f6-99bb5a9f-63d9ebd7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:39.527511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:39.527516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:39.527518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:39.527530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:39.527540Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:39.527569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:39.527926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:39.527938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:39.527965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:39.528306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 747 partition: 0 MaxSeqNo: 25 sessionId: src_id|c53a770e-e9e073f6-99bb5a9f-63d9ebd7_0
2025-10-09T21:46:39.529418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6a1ce32d-feba661a-721e42c5-4d351cbf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c53a770e-e9e073f6-99bb5a9f-63d9ebd7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:39.532260Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c53a770e-e9e073f6-99bb5a9f-63d9ebd7_0] Write session: try to update token
2025-10-09T21:46:39.532281Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c53a770e-e9e073f6-99bb5a9f-63d9ebd7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:39.543758Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:39.543781Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:39.543785Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:39.543787Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:39.543796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341724035312918:6867]
2025-10-09T21:46:39.543801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341724035312918:6867]
2025-10-09T21:46:39.543805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341724035312918:6867]
2025-10-09T21:46:39.543855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:39.543864Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:39.543867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.543870Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:39.543871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:39.543879Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:39.544017Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:39.560695Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 747 sessionId: src_id|c53a770e-e9e073f6-99bb5a9f-63d9ebd7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:39.563517Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:39.600683Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341724397130589:7236]
2025-10-09T21:46:39.601159Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 746, pipe: [1:7559341724397130589:7236]
2025-10-09T21:46:39.712156Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:39.712543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:39.712556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:39.712585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341724397130589:7236], now have 1 active actors on pipe
2025-10-09T21:46:39.713052Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341724397130589:7236]
2025-10-09T21:46:39.713083Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:39.713556Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c53a770e-e9e073f6-99bb5a9f-63d9ebd7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:39.713071Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:39.713572Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c53a770e-e9e073f6-99bb5a9f-63d9ebd7_0] Write session will restart in 0.006945s
2025-10-09T21:46:39.713646Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c53a770e-e9e073f6-99bb5a9f-63d9ebd7_0] Write session: Do CDS request
2025-10-09T21:46:39.713657Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c53a770e-e9e073f6-99bb5a9f-63d9ebd7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:39.713132Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:39.713139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:39.713208Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 747 reason: error at write: Disconnected sessionId: src_id|c53a770e-e9e073f6-99bb5a9f-63d9ebd7_0
2025-10-09T21:46:39.713400Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 747 sessionId: src_id|c53a770e-e9e073f6-99bb5a9f-63d9ebd7_0 is DEAD
2025-10-09T21:46:39.721690Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c53a770e-e9e073f6-99bb5a9f-63d9ebd7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:39.721945Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:39.721963Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 748
2025-10-09T21:46:39.722165Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 748 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:39.722199Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 748 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:39.722206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=748 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:39.722209Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:39.722668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 748 sessionId:  describe result for acl check
2025-10-09T21:46:39.722700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:39.722702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:39.722706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:39.722712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341724397130616:7551] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:39.723264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341724397130616:7551] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:39.726061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341724397130616:7551] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:39.735307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341724397130616:7551] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:39.735325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341724397130616:7551] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:39.735329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341724397130616:7551] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:39.735339Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 748 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:39.742185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:39.743958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:39.744228Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:39.744232Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:39.744247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:39.744324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:39.744367Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341724035313029:6881]
2025-10-09T21:46:39.744949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:39.745687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:39.745744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:39.746162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:39.746225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.746226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.746228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.746229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.746231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.746232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.746234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.746235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.746237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.746242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.746254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.746258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.746262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.746266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.746270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.746274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.746278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.746281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.746289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.746297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.746304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.746310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.746314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.746319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.746322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.746327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.746332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.746336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.746340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.746346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.746351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.746355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.746360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.746365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.746369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.746374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.746409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:39.756736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:39.758336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:39.758358Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:39.758361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:39.758367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:39.758377Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 747 [2:7559341724035313029:6881]
2025-10-09T21:46:39.758396Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:39.758422Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:39.758425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:39.758429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:39.758432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:39.758452Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:39.758460Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:39.758470Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:39.758546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.758549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.758554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.758558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:39.758647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:39.758650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.758666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.758677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.758693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:39.758716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341724035313023:6881]
2025-10-09T21:46:39.758718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341724035313023:6881]
2025-10-09T21:46:39.758720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341724035313023:6881]
2025-10-09T21:46:39.758729Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:39.758743Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:39.758745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:39.758748Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:39.758751Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:39.758760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:39.758764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:39.758777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341724397130608:2432], now have 1 active actors on pipe
2025-10-09T21:46:39.758780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341724397130656:7551], now have 1 active actors on pipe
2025-10-09T21:46:39.758822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341724035313023:6881]
2025-10-09T21:46:39.758831Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:39.758844Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:39.758811Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 747
2025-10-09T21:46:39.758871Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 747
2025-10-09T21:46:39.761330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 748 partition: 0 MaxSeqNo: 25 sessionId: src_id|a6538b69-e5e26b32-f8e7a8b6-401ed2d1_0
2025-10-09T21:46:39.761800Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c53a770e-e9e073f6-99bb5a9f-63d9ebd7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a6538b69-e5e26b32-f8e7a8b6-401ed2d1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:39.759169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:39.759185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:39.759220Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a6538b69-e5e26b32-f8e7a8b6-401ed2d1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:39.759244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:39.759249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:39.759252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:39.759264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:39.759277Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:39.759306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:39.761108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:39.761118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:39.761153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:39.764588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6538b69-e5e26b32-f8e7a8b6-401ed2d1_0] Write session: try to update token
2025-10-09T21:46:39.764606Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6538b69-e5e26b32-f8e7a8b6-401ed2d1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:39.778183Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:39.778202Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:39.778206Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:39.778208Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:39.778221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341724035313023:6881]
2025-10-09T21:46:39.778226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341724035313023:6881]
2025-10-09T21:46:39.778230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341724035313023:6881]
2025-10-09T21:46:39.778280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:39.778288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.778290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:39.778347Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:39.778354Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:39.778359Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:39.778449Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:39.800316Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 748 sessionId: src_id|a6538b69-e5e26b32-f8e7a8b6-401ed2d1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:39.803123Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:39.813074Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341724397130696:7236]
2025-10-09T21:46:39.813709Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 747, pipe: [1:7559341724397130696:7236]
2025-10-09T21:46:39.960775Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:39.961088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:39.961104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:39.961129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341724397130696:7236], now have 1 active actors on pipe
2025-10-09T21:46:39.961437Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:39.962451Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:39.962464Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:39.962506Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:39.962524Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 748 reason: error at write: Disconnected sessionId: src_id|a6538b69-e5e26b32-f8e7a8b6-401ed2d1_0
2025-10-09T21:46:39.962961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6538b69-e5e26b32-f8e7a8b6-401ed2d1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:39.962981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6538b69-e5e26b32-f8e7a8b6-401ed2d1_0] Write session will restart in 0.005999s
2025-10-09T21:46:39.962558Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341724397130696:7236]
2025-10-09T21:46:39.962721Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 748 sessionId: src_id|a6538b69-e5e26b32-f8e7a8b6-401ed2d1_0 is DEAD
2025-10-09T21:46:39.963007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6538b69-e5e26b32-f8e7a8b6-401ed2d1_0] Write session: Do CDS request
2025-10-09T21:46:39.963044Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6538b69-e5e26b32-f8e7a8b6-401ed2d1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:39.970536Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6538b69-e5e26b32-f8e7a8b6-401ed2d1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:39.977373Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:39.977391Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 749
2025-10-09T21:46:39.978048Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 749 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:39.978091Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 749 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:39.978098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=749 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:39.978102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:39.978553Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 749 sessionId:  describe result for acl check
2025-10-09T21:46:39.978587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:39.978589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:39.978590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:39.978597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341724397130718:7557] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:39.979205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341724397130718:7557] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:39.983884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:39.986389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:39.987314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341724397130718:7557] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:39.986685Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:39.986697Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:39.986713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:39.986789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:39.986833Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341724035313109:6885]
2025-10-09T21:46:39.987413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:39.989432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:39.989505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:39.989909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:39.989959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.989966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.989968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.989969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.989970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.989972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.989973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.989987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.989989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.989996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.990006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.990010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.990014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.990018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.990022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.990026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.990046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.990049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.990059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.990068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:39.990074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.990079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:39.990084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.990088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:39.990092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.990097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:39.990102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.990106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:39.990112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.990117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:39.990122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.990127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:39.990132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.990137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:39.990142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.990147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:39.990176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:39.996517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341724397130718:7557] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:39.996553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341724397130718:7557] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:39.996558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341724397130718:7557] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:39.996569Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 749 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:39.999936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:40.001133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:40.001141Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:40.001144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:40.001149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:40.001157Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 748 [2:7559341724035313109:6885]
2025-10-09T21:46:40.001176Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:40.001198Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:40.001202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:40.001205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:40.001208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:40.001228Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:40.001235Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:40.001241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:40.001314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.001316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.001333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.001336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:40.001415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:40.001418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.001432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.001445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.001464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:40.001487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341724035313104:6885]
2025-10-09T21:46:40.001489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341724035313104:6885]
2025-10-09T21:46:40.001491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341724035313104:6885]
2025-10-09T21:46:40.001501Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:40.001518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:40.001520Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:40.001524Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:40.001526Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:40.001534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:40.001538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:40.001550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341724397130773:7557], now have 1 active actors on pipe
2025-10-09T21:46:40.001553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341724397130702:2432], now have 1 active actors on pipe
2025-10-09T21:46:40.001596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341724035313104:6885]
2025-10-09T21:46:40.001605Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:40.001619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:40.003920Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 748
2025-10-09T21:46:40.003956Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 748
2025-10-09T21:46:40.004567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:40.004590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:40.004631Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fe74966d-6a3621f5-f4235937-9e670877_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:40.004646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:40.004652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:40.004656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:40.004670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:40.004689Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:40.004715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:40.005718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 749 partition: 0 MaxSeqNo: 25 sessionId: src_id|fe74966d-6a3621f5-f4235937-9e670877_0
2025-10-09T21:46:40.006363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6538b69-e5e26b32-f8e7a8b6-401ed2d1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fe74966d-6a3621f5-f4235937-9e670877_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:40.005411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:40.005434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:40.005463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:40.009277Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe74966d-6a3621f5-f4235937-9e670877_0] Write session: try to update token
2025-10-09T21:46:40.009300Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe74966d-6a3621f5-f4235937-9e670877_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:40.024066Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:40.024088Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:40.024091Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:40.024093Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:40.024104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341724035313104:6885]
2025-10-09T21:46:40.024112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341724035313104:6885]
2025-10-09T21:46:40.024116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341724035313104:6885]
2025-10-09T21:46:40.024162Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:40.024166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:40.024170Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:40.024174Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:40.024174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.024177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:40.024321Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:40.060489Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 749 sessionId: src_id|fe74966d-6a3621f5-f4235937-9e670877_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:40.062382Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341728692098080:7236]
2025-10-09T21:46:40.063449Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:40.068088Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 748, pipe: [1:7559341728692098080:7236]
2025-10-09T21:46:40.201723Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:40.202258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341728692098080:7236], now have 1 active actors on pipe
2025-10-09T21:46:40.202295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:40.202303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:40.202537Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:40.203095Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:40.203179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 749 reason: status is not ok: tablet will be restarted right now sessionId: src_id|fe74966d-6a3621f5-f4235937-9e670877_0
2025-10-09T21:46:40.203378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 749 sessionId: src_id|fe74966d-6a3621f5-f4235937-9e670877_0 is DEAD
2025-10-09T21:46:40.203479Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341728692098080:7236]
2025-10-09T21:46:40.203497Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:40.203774Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe74966d-6a3621f5-f4235937-9e670877_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:40.203785Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe74966d-6a3621f5-f4235937-9e670877_0] Write session will restart in 0.007079s
2025-10-09T21:46:40.203826Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe74966d-6a3621f5-f4235937-9e670877_0] Write session: Do CDS request
2025-10-09T21:46:40.203833Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe74966d-6a3621f5-f4235937-9e670877_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:40.212621Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe74966d-6a3621f5-f4235937-9e670877_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:40.215070Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:40.215087Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 750
2025-10-09T21:46:40.215927Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 750 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:40.215970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 750 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:40.215977Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=750 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:40.215981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:40.216819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 750 sessionId:  describe result for acl check
2025-10-09T21:46:40.216884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:40.216886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:40.216888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:40.216894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341728692098098:7562] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:40.217458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341728692098098:7562] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:40.221527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341728692098098:7562] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:40.226496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341728692098098:7562] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:40.226516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341728692098098:7562] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:40.226521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341728692098098:7562] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:40.226533Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 750 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:40.229380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:40.231286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:40.231541Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:40.231554Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:40.231569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:40.231649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:40.231701Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341728330280479:6887]
2025-10-09T21:46:40.232295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:40.234830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:40.234909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:40.235287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:40.235368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:40.235374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:40.235376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:40.235377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:40.235378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:40.235380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:40.235381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.235383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.235384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.235391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:40.235403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:40.235407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:40.235411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:40.235416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:40.235420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:40.235423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.235427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.235431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.235439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:40.235448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:40.235455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:40.235461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:40.235466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:40.235474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:40.235478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:40.235495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:40.235506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:40.235510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:40.235517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:40.235522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:40.235530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.235536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.235540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.235544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.235550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.235561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.235600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:40.243489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:40.244671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:40.244689Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:40.244693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:40.244698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:40.244706Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 749 [2:7559341728330280479:6887]
2025-10-09T21:46:40.244724Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:40.244746Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:40.244756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:40.244760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:40.244763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:40.244784Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:40.244796Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:40.244805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:40.244876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.244885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.244891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.244895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:40.244973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:40.244976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.244990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.245001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.245014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:40.245037Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341728330280474:6887]
2025-10-09T21:46:40.245038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341728330280474:6887]
2025-10-09T21:46:40.245040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341728330280474:6887]
2025-10-09T21:46:40.245048Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:40.245064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:40.245077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:40.245081Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:40.245125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341728330280474:6887]
2025-10-09T21:46:40.245128Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:40.245130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:40.245133Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:40.245136Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:40.245140Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:40.245167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341728692098146:7562], now have 1 active actors on pipe
2025-10-09T21:46:40.245170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341728692098088:2432], now have 1 active actors on pipe
2025-10-09T21:46:40.245285Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 749
2025-10-09T21:46:40.245319Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 749
2025-10-09T21:46:40.245836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:40.245851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:40.245889Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7744ee89-63e93b36-5ae480c0-b6e6a789_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:40.245904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:40.245909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:40.245911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:40.245928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:40.245954Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:40.245990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:40.246851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:40.246861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:40.246885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:40.247064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 750 partition: 0 MaxSeqNo: 25 sessionId: src_id|7744ee89-63e93b36-5ae480c0-b6e6a789_0
2025-10-09T21:46:40.247501Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe74966d-6a3621f5-f4235937-9e670877_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7744ee89-63e93b36-5ae480c0-b6e6a789_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:40.250266Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7744ee89-63e93b36-5ae480c0-b6e6a789_0] Write session: try to update token
2025-10-09T21:46:40.250287Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7744ee89-63e93b36-5ae480c0-b6e6a789_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:40.264047Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:40.264062Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:40.264065Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:40.264067Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:40.264077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341728330280474:6887]
2025-10-09T21:46:40.264083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341728330280474:6887]
2025-10-09T21:46:40.264086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341728330280474:6887]
2025-10-09T21:46:40.264134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:40.264152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.264155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:40.264211Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:40.264225Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:40.264229Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:40.264293Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:40.272864Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 750 sessionId: src_id|7744ee89-63e93b36-5ae480c0-b6e6a789_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:40.275545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:40.301654Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341728692098170:7236]
2025-10-09T21:46:40.302081Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 749, pipe: [1:7559341728692098170:7236]
2025-10-09T21:46:40.436744Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:40.437075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:40.437088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:40.437128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341728692098170:7236], now have 1 active actors on pipe
2025-10-09T21:46:40.437302Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:40.437489Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341728692098170:7236]
2025-10-09T21:46:40.437658Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:40.437746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 750 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7744ee89-63e93b36-5ae480c0-b6e6a789_0
2025-10-09T21:46:40.437950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 750 sessionId: src_id|7744ee89-63e93b36-5ae480c0-b6e6a789_0 is DEAD
2025-10-09T21:46:40.437957Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:40.438120Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7744ee89-63e93b36-5ae480c0-b6e6a789_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:40.438132Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7744ee89-63e93b36-5ae480c0-b6e6a789_0] Write session will restart in 0.008354s
2025-10-09T21:46:40.438160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7744ee89-63e93b36-5ae480c0-b6e6a789_0] Write session: Do CDS request
2025-10-09T21:46:40.438167Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7744ee89-63e93b36-5ae480c0-b6e6a789_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:40.447944Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7744ee89-63e93b36-5ae480c0-b6e6a789_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:40.451160Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:40.451176Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 751
2025-10-09T21:46:40.451370Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 751 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:40.451409Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 751 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:40.451415Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=751 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:40.451420Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:40.451958Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 751 sessionId:  describe result for acl check
2025-10-09T21:46:40.451989Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:40.451991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:40.451992Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:40.451999Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341728692098187:7566] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:40.452576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341728692098187:7566] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:40.455052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341728692098187:7566] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:40.459794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341728692098187:7566] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:40.459811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341728692098187:7566] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:40.459815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341728692098187:7566] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:40.459824Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 751 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:40.463372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:40.464671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:40.464920Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:40.464924Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:40.464936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:40.464993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:40.465035Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341728330280554:6889]
2025-10-09T21:46:40.465637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:40.465954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:40.466009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:40.466323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:40.466377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:40.466378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:40.466379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:40.466380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:40.466382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:40.466383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:40.466384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.466385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.466386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.466391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:40.466401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:40.466405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:40.466408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:40.466412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:40.466417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:40.466420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.466423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.466427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.466435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:40.466444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:40.466450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:40.466455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:40.466460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:40.466467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:40.466472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:40.466477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:40.466481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:40.466488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:40.466492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:40.466496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:40.466500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.466505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.466509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.466514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.466518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.466524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.466553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:40.484218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:40.485453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:40.485467Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:40.485470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:40.485474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:40.485481Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 750 [2:7559341728330280554:6889]
2025-10-09T21:46:40.485498Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:40.485519Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:40.485524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:40.485529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:40.485531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:40.485550Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:40.485562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:40.485569Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:40.485639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.485647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.485649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.485652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:40.485725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:40.485732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.485745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.485762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.485782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:40.485809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341728692098231:7566], now have 1 active actors on pipe
2025-10-09T21:46:40.485822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341728692098173:2432], now have 1 active actors on pipe
2025-10-09T21:46:40.485830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341728330280548:6889]
2025-10-09T21:46:40.485832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341728330280548:6889]
2025-10-09T21:46:40.485834Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341728330280548:6889]
2025-10-09T21:46:40.485848Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:40.485862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:40.485879Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:40.485882Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:40.485882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:40.485884Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:40.485885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:40.485928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341728330280548:6889]
2025-10-09T21:46:40.485939Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:40.485991Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:40.490189Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 750
2025-10-09T21:46:40.490230Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 750
2025-10-09T21:46:40.490552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:40.490571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:40.490614Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7e670091-665f65e8-82ca79c9-bb3fec6a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:40.490640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:40.490649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:40.490652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:40.490665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:40.490696Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:40.490719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:40.494351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:40.494371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:40.494419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:40.494684Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 751 partition: 0 MaxSeqNo: 25 sessionId: src_id|7e670091-665f65e8-82ca79c9-bb3fec6a_0
2025-10-09T21:46:40.497047Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7744ee89-63e93b36-5ae480c0-b6e6a789_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7e670091-665f65e8-82ca79c9-bb3fec6a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:40.499699Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e670091-665f65e8-82ca79c9-bb3fec6a_0] Write session: try to update token
2025-10-09T21:46:40.499720Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e670091-665f65e8-82ca79c9-bb3fec6a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:40.520352Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 751 sessionId: src_id|7e670091-665f65e8-82ca79c9-bb3fec6a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:40.523767Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:40.536219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:40.536242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:40.536288Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:40.536297Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:40.536305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341728330280548:6889]
2025-10-09T21:46:40.537598Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341728692098257:7236]
2025-10-09T21:46:40.537631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:40.537789Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:40.537819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:40.537957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:40.537952Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 750, pipe: [1:7559341728692098257:7236]
2025-10-09T21:46:40.542165Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:40.542329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:40.542690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:40.542750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:40.542784Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:40.542803Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:40.542807Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:40.542810Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:40.542820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341728330280548:6889]
2025-10-09T21:46:40.542830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341728330280548:6889]
2025-10-09T21:46:40.542835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341728330280548:6889]
2025-10-09T21:46:40.542879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341728692098257:7236], now have 1 active actors on pipe
2025-10-09T21:46:40.542880Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:40.542885Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:40.542889Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:40.542895Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:40.542920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:40.542926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.542928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:40.543067Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:40.542874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:40.710472Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:40.718944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:40.718975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:40.719143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:40.719214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:40.719277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:40.719337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:40.719755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:40.719761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:40.719834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:40.719891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:40.719957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:40.720018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:40.720057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:40.720060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:40.720121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:40.720176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:40.720236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:40.720294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:40.720332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:40.720335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:40.720408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:40.720468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:40.720531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:40.720597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:40.720639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:40.720642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:40.720715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:40.720780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:40.720838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:40.720898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:40.720938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:40.720941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:40.721013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:40.721075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:40.721157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:40.721222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:40.721267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:40.721270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:40.721344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:40.721397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:40.721460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:40.721520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:40.721567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:40.721570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:40.721636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:40.721692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:40.721752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:40.721811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:40.721850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:40.721853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:40.721920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:40.721994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:40.722366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:40.722439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:40.722481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:40.722484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:40.722559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:40.722617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:40.722675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:40.722734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:40.722782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:40.722786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:40.723895Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341728692098257:7236]
2025-10-09T21:46:40.723930Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:40.723933Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:40.723961Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 751 reason: error at write: Disconnected sessionId: src_id|7e670091-665f65e8-82ca79c9-bb3fec6a_0
2025-10-09T21:46:40.724172Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 751 sessionId: src_id|7e670091-665f65e8-82ca79c9-bb3fec6a_0 is DEAD
2025-10-09T21:46:40.724288Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:40.725844Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e670091-665f65e8-82ca79c9-bb3fec6a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:40.725857Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e670091-665f65e8-82ca79c9-bb3fec6a_0] Write session will restart in 0.005355s
2025-10-09T21:46:40.725884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e670091-665f65e8-82ca79c9-bb3fec6a_0] Write session: Do CDS request
2025-10-09T21:46:40.725891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e670091-665f65e8-82ca79c9-bb3fec6a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:40.723941Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:40.732562Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e670091-665f65e8-82ca79c9-bb3fec6a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:40.732712Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:40.732741Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 752
2025-10-09T21:46:40.732876Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 752 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:40.732903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 752 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:40.732908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=752 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:40.732912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:40.733483Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 752 sessionId:  describe result for acl check
2025-10-09T21:46:40.733527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:40.733529Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:40.733536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:40.733543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341728692098275:7570] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:40.734211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341728692098275:7570] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:40.737200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341728692098275:7570] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:40.742195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341728692098275:7570] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:40.742211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341728692098275:7570] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:40.742216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341728692098275:7570] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:40.742225Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 752 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:40.742978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:40.744272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:40.744491Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:40.744496Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:40.744507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:40.744554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:40.744587Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341728330280646:6899]
2025-10-09T21:46:40.745134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:40.745679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:40.745726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:40.746042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:40.746087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:40.746088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:40.746089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:40.746090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:40.746091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:40.746092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:40.746094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.746095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.746096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.746100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:40.746110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:40.746113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:40.746117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:40.746120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:40.746123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:40.746127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.746130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.746133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.746140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:40.746147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:40.746154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:40.746158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:40.746162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:40.746166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:40.746170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:40.746174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:40.746177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:40.746181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:40.746185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:40.746192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:40.746198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.746202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.746206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.746211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.746215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.746219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.746244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:40.755808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:40.756938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:40.756957Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:40.756962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:40.756967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:40.756974Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 751 [2:7559341728330280646:6899]
2025-10-09T21:46:40.756989Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:40.757012Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:40.757024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:40.757028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:40.757031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:40.757051Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:40.757060Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:40.757068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:40.757130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.757138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.757140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.757143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:40.757219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:40.757228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.757241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:40.757255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:40.757274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:40.757299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341728330280641:6899]
2025-10-09T21:46:40.757302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341728330280641:6899]
2025-10-09T21:46:40.757304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341728330280641:6899]
2025-10-09T21:46:40.757312Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:40.757326Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:40.757342Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:40.757345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:40.757386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341728330280641:6899]
2025-10-09T21:46:40.757388Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:40.757388Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:40.757390Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:40.757393Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:40.757398Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:40.757737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341728692098262:2432], now have 1 active actors on pipe
2025-10-09T21:46:40.757750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341728692098335:7570], now have 1 active actors on pipe
2025-10-09T21:46:40.758012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:40.758026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:40.758085Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b0cf3ab6-787a2452-13731d03-f3321288_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:40.758113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:40.758123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:40.758126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:40.758138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:40.758150Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:40.758168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:40.758908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:40.758921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:40.758944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:40.759538Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e670091-665f65e8-82ca79c9-bb3fec6a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b0cf3ab6-787a2452-13731d03-f3321288_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:40.757885Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 751
2025-10-09T21:46:40.757939Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 751
2025-10-09T21:46:40.759123Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 752 partition: 0 MaxSeqNo: 25 sessionId: src_id|b0cf3ab6-787a2452-13731d03-f3321288_0
2025-10-09T21:46:40.763258Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0cf3ab6-787a2452-13731d03-f3321288_0] Write session: try to update token
2025-10-09T21:46:40.763279Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0cf3ab6-787a2452-13731d03-f3321288_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:40.779283Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:40.779307Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:40.779312Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:40.779314Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:40.779323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341728330280641:6899]
2025-10-09T21:46:40.779327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341728330280641:6899]
2025-10-09T21:46:40.779330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341728330280641:6899]
2025-10-09T21:46:40.779368Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:40.779371Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:40.779375Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:40.779377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:40.779385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:40.779387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:40.779533Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:40.786214Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 752 sessionId: src_id|b0cf3ab6-787a2452-13731d03-f3321288_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:40.789030Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:40.822933Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341728692098370:7236]
2025-10-09T21:46:40.823311Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 751, pipe: [1:7559341728692098370:7236]
2025-10-09T21:46:40.958825Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:40.959199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:40.959212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:40.959261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341728692098370:7236], now have 1 active actors on pipe
2025-10-09T21:46:40.962370Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:40.964106Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341728692098370:7236]
2025-10-09T21:46:40.964144Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:40.964678Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0cf3ab6-787a2452-13731d03-f3321288_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:40.964689Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0cf3ab6-787a2452-13731d03-f3321288_0] Write session will restart in 0.005400s
2025-10-09T21:46:40.964719Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0cf3ab6-787a2452-13731d03-f3321288_0] Write session: Do CDS request
2025-10-09T21:46:40.964725Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0cf3ab6-787a2452-13731d03-f3321288_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:40.964183Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:40.964270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 752 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b0cf3ab6-787a2452-13731d03-f3321288_0
2025-10-09T21:46:40.964463Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 752 sessionId: src_id|b0cf3ab6-787a2452-13731d03-f3321288_0 is DEAD
2025-10-09T21:46:40.973199Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0cf3ab6-787a2452-13731d03-f3321288_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:40.973340Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:40.973353Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 753
2025-10-09T21:46:40.973534Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 753 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:40.973570Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 753 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:40.973581Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=753 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:40.973585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:40.974392Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 753 sessionId:  describe result for acl check
2025-10-09T21:46:40.974435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:40.974437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:40.974439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:40.974445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341728692098382:7582] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:40.975085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341728692098382:7582] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:40.981149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341728692098382:7582] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:40.987740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341728692098382:7582] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:40.987758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341728692098382:7582] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:40.987762Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341728692098382:7582] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:40.987774Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 753 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:40.998144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:41.001460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:41.001719Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:41.001723Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:41.001737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:41.001803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:41.001852Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341728330280748:6907]
2025-10-09T21:46:41.002601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:41.003465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:41.003524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:41.003859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:41.003938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:41.003940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:41.003941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:41.003942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:41.003943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:41.003945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:41.003946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.003948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.003949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.003955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:41.003966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:41.003970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:41.003973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:41.003976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:41.003980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:41.003984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.003987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.003990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.003998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:41.004005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:41.004011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:41.004017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:41.004021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:41.004027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:41.004030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:41.004035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:41.004040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:41.004047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:41.004052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:41.004056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:41.004061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.004066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.004070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.004075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.004082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.004089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.004116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:41.015469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:41.016507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:41.016513Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:41.016515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:41.016519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:41.016525Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 752 [2:7559341728330280748:6907]
2025-10-09T21:46:41.016537Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:41.016554Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:41.016557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:41.016559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:41.016561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:41.016576Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:41.016580Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:41.016587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:41.016645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.016647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.016649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.016653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:41.016716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:41.016719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.016728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.016734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.016744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:41.016763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341728330280743:6907]
2025-10-09T21:46:41.016766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341728330280743:6907]
2025-10-09T21:46:41.016770Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341728330280743:6907]
2025-10-09T21:46:41.016786Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:41.016800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:41.016815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:41.016818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:41.016864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341728330280743:6907]
2025-10-09T21:46:41.016867Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:41.017187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:41.017193Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:41.017194Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:41.017199Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:41.017404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341728692098422:7582], now have 1 active actors on pipe
2025-10-09T21:46:41.017408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341728692098376:2432], now have 1 active actors on pipe
2025-10-09T21:46:41.018645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:41.018658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:41.018699Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1694a23b-6fd81668-dc642ed2-fa31a40f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:41.018714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:41.018721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:41.018725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:41.018738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:41.018758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:41.018782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:41.019098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:41.019106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:41.019123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:41.018248Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 752
2025-10-09T21:46:41.018274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 752
2025-10-09T21:46:41.019319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 753 partition: 0 MaxSeqNo: 25 sessionId: src_id|1694a23b-6fd81668-dc642ed2-fa31a40f_0
2025-10-09T21:46:41.019761Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0cf3ab6-787a2452-13731d03-f3321288_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1694a23b-6fd81668-dc642ed2-fa31a40f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:41.022418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1694a23b-6fd81668-dc642ed2-fa31a40f_0] Write session: try to update token
2025-10-09T21:46:41.022439Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1694a23b-6fd81668-dc642ed2-fa31a40f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:41.043238Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 753 sessionId: src_id|1694a23b-6fd81668-dc642ed2-fa31a40f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:41.046092Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:41.046996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:41.047023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:41.048513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:41.048552Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:41.048556Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:41.048558Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:41.048561Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:41.048572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341728330280743:6907]
2025-10-09T21:46:41.048576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341728330280743:6907]
2025-10-09T21:46:41.048581Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341728330280743:6907]
2025-10-09T21:46:41.048624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:41.048625Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:41.048630Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:41.048632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.048635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:41.048636Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:41.048784Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:41.055781Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:41.064663Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341732987065758:7236]
2025-10-09T21:46:41.065037Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 752, pipe: [1:7559341732987065758:7236]
2025-10-09T21:46:41.265277Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:41.265722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341732987065758:7236], now have 1 active actors on pipe
2025-10-09T21:46:41.274588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:41.274618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:41.274796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:41.274853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:41.274914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:41.274972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:41.275021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:41.275026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:41.275095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:41.275148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:41.275208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:41.275271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:41.275311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:41.275314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:41.275378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:41.275433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:41.275491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:41.275553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:41.275597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:41.275600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:41.275670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:41.275730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:41.275792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:41.275857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:41.275896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:41.275899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:41.275966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:41.276026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:41.276086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:41.276143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:41.276184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:41.276187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:41.276249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:41.276306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:41.276363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:41.276423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:41.276467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:41.276470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:41.276536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:41.276592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:41.276652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:41.278747Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1694a23b-6fd81668-dc642ed2-fa31a40f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:41.278767Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1694a23b-6fd81668-dc642ed2-fa31a40f_0] Write session will restart in 0.008510s
2025-10-09T21:46:41.278793Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1694a23b-6fd81668-dc642ed2-fa31a40f_0] Write session: Do CDS request
2025-10-09T21:46:41.278799Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1694a23b-6fd81668-dc642ed2-fa31a40f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:41.276710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:41.276750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:41.276753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:41.276817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:41.276869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:41.276927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:41.276987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:41.277027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:41.277029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:41.277094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:41.277151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:41.277211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:41.277276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:41.277319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:41.277322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:41.277388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:41.277447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:41.277506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:41.277564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:41.277604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:41.277606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:41.278254Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:41.278125Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341732987065758:7236]
2025-10-09T21:46:41.278168Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:41.278211Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:41.278214Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:41.278246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 753 reason: error at write: Disconnected sessionId: src_id|1694a23b-6fd81668-dc642ed2-fa31a40f_0
2025-10-09T21:46:41.278442Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 753 sessionId: src_id|1694a23b-6fd81668-dc642ed2-fa31a40f_0 is DEAD
2025-10-09T21:46:41.289266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:41.290456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1694a23b-6fd81668-dc642ed2-fa31a40f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:41.290578Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:41.290593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 754
2025-10-09T21:46:41.290763Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 754 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:41.290810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 754 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:41.290816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=754 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:41.290820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:41.291323Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 754 sessionId:  describe result for acl check
2025-10-09T21:46:41.291362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:41.291364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:41.291365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:41.291371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341732987065808:7587] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:41.290802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:41.291025Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:41.291030Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:41.291044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:41.291112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:41.291157Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341732625248121:6909]
2025-10-09T21:46:41.291759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:41.292157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:41.292234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:41.292404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341732987065808:7587] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:41.292618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:41.292694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:41.292700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:41.292701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:41.292703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:41.292704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:41.292705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:41.292706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.292707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.292709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.292715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:41.292726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:41.292730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:41.292734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:41.292737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:41.292741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:41.292745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.292748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.292752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.292759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:41.292768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:41.292773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:41.292778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:41.292783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:41.292788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:41.292792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:41.292797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:41.292801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:41.292806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:41.292811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:41.292815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:41.292822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.292827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.292832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.292837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.292841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.292846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.292873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:41.295218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341732987065808:7587] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:41.299997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341732987065808:7587] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:41.300012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341732987065808:7587] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:41.300017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341732987065808:7587] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:41.300028Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 754 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:41.300501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:41.301588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:41.301602Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:41.301605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:41.301610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:41.301617Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 753 [2:7559341732625248121:6909]
2025-10-09T21:46:41.301632Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:41.301650Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:41.301654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:41.301657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:41.301660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:41.301677Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:41.301683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:41.301690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:41.301748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.301761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.301763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.301766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:41.301840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:41.301849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.301862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.301874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.301893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:41.301917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341732987065845:7587], now have 1 active actors on pipe
2025-10-09T21:46:41.301921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341732987065766:2432], now have 1 active actors on pipe
2025-10-09T21:46:41.301930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341732625248114:6909]
2025-10-09T21:46:41.301933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341732625248114:6909]
2025-10-09T21:46:41.301935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341732625248114:6909]
2025-10-09T21:46:41.301952Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:41.301989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:41.302008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:41.302012Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:41.302069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341732625248114:6909]
2025-10-09T21:46:41.302078Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:41.302117Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:41.302126Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:41.302137Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:41.302145Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:41.302151Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 753
2025-10-09T21:46:41.302190Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 753
2025-10-09T21:46:41.302792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:41.302808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:41.302843Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|32b29223-98973d18-85601c20-55c84cda_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:41.302871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:41.302881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:41.302884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:41.302895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:41.302928Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:41.302955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:41.310124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:41.310149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:41.310188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:41.310626Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 754 partition: 0 MaxSeqNo: 25 sessionId: src_id|32b29223-98973d18-85601c20-55c84cda_0
2025-10-09T21:46:41.311097Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1694a23b-6fd81668-dc642ed2-fa31a40f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|32b29223-98973d18-85601c20-55c84cda_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:41.313710Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32b29223-98973d18-85601c20-55c84cda_0] Write session: try to update token
2025-10-09T21:46:41.313731Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32b29223-98973d18-85601c20-55c84cda_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:41.322180Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:41.322198Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:41.322201Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:41.322202Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:41.322214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341732625248114:6909]
2025-10-09T21:46:41.322221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341732625248114:6909]
2025-10-09T21:46:41.322225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341732625248114:6909]
2025-10-09T21:46:41.322274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:41.322281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.322285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:41.322434Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:41.322609Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:41.322617Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:41.322622Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:41.336705Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 754 sessionId: src_id|32b29223-98973d18-85601c20-55c84cda_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:41.339173Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:41.378330Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341732987065854:7236]
2025-10-09T21:46:41.379734Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 753, pipe: [1:7559341732987065854:7236]
2025-10-09T21:46:41.531364Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:41.531919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:41.531931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:41.531957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341732987065854:7236], now have 1 active actors on pipe
2025-10-09T21:46:41.534409Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:41.535081Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:41.535137Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341732987065854:7236]
2025-10-09T21:46:41.535159Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:41.535228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 754 reason: status is not ok: tablet will be restarted right now sessionId: src_id|32b29223-98973d18-85601c20-55c84cda_0
2025-10-09T21:46:41.535416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 754 sessionId: src_id|32b29223-98973d18-85601c20-55c84cda_0 is DEAD
2025-10-09T21:46:41.542176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32b29223-98973d18-85601c20-55c84cda_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:41.542194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32b29223-98973d18-85601c20-55c84cda_0] Write session will restart in 0.009890s
2025-10-09T21:46:41.542224Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32b29223-98973d18-85601c20-55c84cda_0] Write session: Do CDS request
2025-10-09T21:46:41.542231Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32b29223-98973d18-85601c20-55c84cda_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:41.553701Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32b29223-98973d18-85601c20-55c84cda_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:41.553842Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:41.553858Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 755
2025-10-09T21:46:41.554147Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 755 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:41.554180Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 755 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:41.554186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=755 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:41.554190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:41.555108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 755 sessionId:  describe result for acl check
2025-10-09T21:46:41.555146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:41.555147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:41.555149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:41.555162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341732987065898:7591] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:41.555731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341732987065898:7591] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:41.553852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:41.555887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:41.556113Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:41.556117Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:41.556130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:41.556206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:41.556246Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341732625248200:6911]
2025-10-09T21:46:41.556779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:41.557455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:41.557501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:41.557865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:41.557921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:41.557922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:41.557923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:41.557924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:41.557925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:41.557926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:41.557927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.557929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.557930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.557935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:41.557955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:41.557959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:41.557962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:41.557966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:41.557985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:41.557989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.557991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.557994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.558001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:41.558008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:41.558014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:41.558019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:41.558024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:41.558044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:41.558048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:41.558052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:41.558056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:41.558060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:41.558064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:41.558069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:41.558073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.558077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.558081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.558085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.558090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.558095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.558123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:41.558846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341732987065898:7591] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:41.565337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341732987065898:7591] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:41.565353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341732987065898:7591] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:41.565356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341732987065898:7591] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:41.565367Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 755 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:41.574618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:41.575701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:41.575723Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:41.575726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:41.575731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:41.575738Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 754 [2:7559341732625248200:6911]
2025-10-09T21:46:41.575756Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:41.575777Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:41.575826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:41.575838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:41.575841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:41.575862Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:41.575878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:41.575891Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:41.575983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.575991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.575993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.575996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:41.576135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:41.576143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.576158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.576175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.576199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:41.576232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341732987065857:2432], now have 1 active actors on pipe
2025-10-09T21:46:41.576236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341732987065936:7591], now have 1 active actors on pipe
2025-10-09T21:46:41.576244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341732625248195:6911]
2025-10-09T21:46:41.576246Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341732625248195:6911]
2025-10-09T21:46:41.576248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341732625248195:6911]
2025-10-09T21:46:41.576266Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:41.576289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:41.576304Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:41.576304Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:41.576308Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:41.576308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:41.576310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:41.576347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341732625248195:6911]
2025-10-09T21:46:41.576354Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:41.576387Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:41.576928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:41.576942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:41.576778Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 754
2025-10-09T21:46:41.576815Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 754
2025-10-09T21:46:41.576984Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b40af535-4ad216e5-908cdadf-99f361ea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:41.576999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:41.577004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:41.577006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:41.577019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:41.577032Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:41.577060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:41.577537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:41.577552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:41.577570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:41.577713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 755 partition: 0 MaxSeqNo: 25 sessionId: src_id|b40af535-4ad216e5-908cdadf-99f361ea_0
2025-10-09T21:46:41.578151Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32b29223-98973d18-85601c20-55c84cda_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b40af535-4ad216e5-908cdadf-99f361ea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:41.580724Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b40af535-4ad216e5-908cdadf-99f361ea_0] Write session: try to update token
2025-10-09T21:46:41.580743Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b40af535-4ad216e5-908cdadf-99f361ea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:41.598934Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 755 sessionId: src_id|b40af535-4ad216e5-908cdadf-99f361ea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:41.594251Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:41.594270Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:41.594272Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:41.594274Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:41.594284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341732625248195:6911]
2025-10-09T21:46:41.594289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341732625248195:6911]
2025-10-09T21:46:41.594293Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341732625248195:6911]
2025-10-09T21:46:41.594338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:41.594349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.594352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:41.594493Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:41.599837Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:41.599853Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:41.599858Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:41.601509Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:41.635376Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341732987065943:7236]
2025-10-09T21:46:41.636683Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 754, pipe: [1:7559341732987065943:7236]
2025-10-09T21:46:41.757211Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:41.757597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:41.757616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:41.757667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341732987065943:7236], now have 1 active actors on pipe
2025-10-09T21:46:41.757967Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:41.758023Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341732987065943:7236]
2025-10-09T21:46:41.758058Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:41.758061Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:41.758124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 755 reason: error at write: Disconnected sessionId: src_id|b40af535-4ad216e5-908cdadf-99f361ea_0
2025-10-09T21:46:41.758313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 755 sessionId: src_id|b40af535-4ad216e5-908cdadf-99f361ea_0 is DEAD
2025-10-09T21:46:41.758731Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b40af535-4ad216e5-908cdadf-99f361ea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:41.758742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b40af535-4ad216e5-908cdadf-99f361ea_0] Write session will restart in 0.005467s
2025-10-09T21:46:41.758771Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b40af535-4ad216e5-908cdadf-99f361ea_0] Write session: Do CDS request
2025-10-09T21:46:41.758781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b40af535-4ad216e5-908cdadf-99f361ea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:41.757958Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:41.765727Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b40af535-4ad216e5-908cdadf-99f361ea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:41.765878Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:41.765894Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 756
2025-10-09T21:46:41.766166Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 756 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:41.766202Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 756 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:41.766208Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=756 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:41.766212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:41.766667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 756 sessionId:  describe result for acl check
2025-10-09T21:46:41.766702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:41.766704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:41.766706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:41.766713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341732987065966:7598] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:41.767317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341732987065966:7598] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:41.770437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341732987065966:7598] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:41.771799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:41.773564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:41.773763Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:41.773767Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:41.773780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:41.773837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:41.773873Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341732625248283:6915]
2025-10-09T21:46:41.774105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341732987065966:7598] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:41.774126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341732987065966:7598] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:41.774132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341732987065966:7598] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:41.774142Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 756 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:41.774495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:41.775054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:41.775098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:41.775463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:41.775503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:41.775504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:41.775505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:41.775506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:41.775507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:41.775509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:41.775510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.775511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.775512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.775518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:41.775528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:41.775532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:41.775535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:41.775540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:41.775543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:41.775547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.775552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.775555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.775562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:41.775579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:41.775584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:41.775591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:41.775597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:41.775602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:41.775606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:41.775611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:41.775616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:41.775620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:41.775626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:41.775640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:41.775647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.775652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.775656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.775660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.775664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.775669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.775697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:41.783948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:41.785147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:41.785166Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:41.785169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:41.785174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:41.785182Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 755 [2:7559341732625248283:6915]
2025-10-09T21:46:41.785198Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:41.785233Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:41.785241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:41.785245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:41.785247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:41.785268Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:41.785274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:41.785284Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:41.785351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.785359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.785361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.785364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:41.785432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:41.785442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.785454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:41.785467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:41.785484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:41.785510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341732987065951:2432], now have 1 active actors on pipe
2025-10-09T21:46:41.785529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341732987066027:7598], now have 1 active actors on pipe
2025-10-09T21:46:41.785537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341732625248278:6915]
2025-10-09T21:46:41.785539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341732625248278:6915]
2025-10-09T21:46:41.785541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341732625248278:6915]
2025-10-09T21:46:41.785553Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:41.785568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:41.785582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:41.785584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:41.785590Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:41.785593Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:41.785596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:41.785590Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 755
2025-10-09T21:46:41.785626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341732625248278:6915]
2025-10-09T21:46:41.785634Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:41.785637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:41.785653Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 755
2025-10-09T21:46:41.785733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:41.785740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:41.785767Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e0a7eaf3-c18dc3d0-69b4ca49-ed6e8edb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:41.785780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:41.785789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:41.785791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:41.785803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:41.785816Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:41.785835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:41.786145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:41.786159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:41.786179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:41.786318Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 756 partition: 0 MaxSeqNo: 25 sessionId: src_id|e0a7eaf3-c18dc3d0-69b4ca49-ed6e8edb_0
2025-10-09T21:46:41.786712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b40af535-4ad216e5-908cdadf-99f361ea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e0a7eaf3-c18dc3d0-69b4ca49-ed6e8edb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:41.789481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e0a7eaf3-c18dc3d0-69b4ca49-ed6e8edb_0] Write session: try to update token
2025-10-09T21:46:41.789503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e0a7eaf3-c18dc3d0-69b4ca49-ed6e8edb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:41.804117Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:41.804139Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:41.804141Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:41.804143Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:41.804156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341732625248278:6915]
2025-10-09T21:46:41.804160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341732625248278:6915]
2025-10-09T21:46:41.804163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341732625248278:6915]
2025-10-09T21:46:41.804211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:41.804220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:41.804223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:41.804356Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:41.806136Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:41.806152Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:41.806156Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:41.807469Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 756 sessionId: src_id|e0a7eaf3-c18dc3d0-69b4ca49-ed6e8edb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:41.810102Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:41.858226Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341732987066053:7236]
2025-10-09T21:46:41.858662Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 755, pipe: [1:7559341732987066053:7236]
2025-10-09T21:46:42.007545Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:42.008048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:42.008059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:42.008107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341732987066053:7236], now have 1 active actors on pipe
2025-10-09T21:46:42.010395Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:42.010887Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341732987066053:7236]
2025-10-09T21:46:42.010935Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:42.010938Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:42.011016Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 756 reason: error at write: Disconnected sessionId: src_id|e0a7eaf3-c18dc3d0-69b4ca49-ed6e8edb_0
2025-10-09T21:46:42.011223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 756 sessionId: src_id|e0a7eaf3-c18dc3d0-69b4ca49-ed6e8edb_0 is DEAD
2025-10-09T21:46:42.011224Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:42.014175Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0a7eaf3-c18dc3d0-69b4ca49-ed6e8edb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:42.014191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0a7eaf3-c18dc3d0-69b4ca49-ed6e8edb_0] Write session will restart in 0.006385s
2025-10-09T21:46:42.014222Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0a7eaf3-c18dc3d0-69b4ca49-ed6e8edb_0] Write session: Do CDS request
2025-10-09T21:46:42.014230Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0a7eaf3-c18dc3d0-69b4ca49-ed6e8edb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:42.020953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:42.021642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e0a7eaf3-c18dc3d0-69b4ca49-ed6e8edb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:42.021692Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:42.021706Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 757
2025-10-09T21:46:42.021888Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 757 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:42.021944Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 757 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:42.021951Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=757 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:42.021955Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:42.022499Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 757 sessionId:  describe result for acl check
2025-10-09T21:46:42.022540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:42.022542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:42.022543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:42.022550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341737282033410:7609] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:42.022980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:42.023244Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:42.023256Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:42.023272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:42.023186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341737282033410:7609] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:42.023359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:42.023408Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341736920215697:6929]
2025-10-09T21:46:42.024162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:42.024965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:42.025023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:42.025345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:42.025409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.025416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.025417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.025418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.025420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.025421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.025422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.025425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.025428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.025433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.025442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.025446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.025450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.025453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.025457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.025460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.025463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.025467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.025473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.025482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.025487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.025492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.025495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.025502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.025506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.025510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.025513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.025518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.025521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.025526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.025530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.025534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.025537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.025542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.025545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.025550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.025579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:42.025960Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341737282033410:7609] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:42.035169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341737282033410:7609] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:42.035189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341737282033410:7609] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:42.035193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341737282033410:7609] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:42.035204Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 757 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:42.035625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:42.036960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:42.036982Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:42.036985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:42.036990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:42.036998Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 756 [2:7559341736920215697:6929]
2025-10-09T21:46:42.037015Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:42.037038Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:42.037047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:42.037051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:42.037054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:42.037074Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:42.037080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:42.037089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:42.037157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.037165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.037171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.037174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:42.037251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:42.037260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.037273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.037284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.037306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:42.037336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341737282033445:7609], now have 1 active actors on pipe
2025-10-09T21:46:42.037351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341736920215692:6929]
2025-10-09T21:46:42.037358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341736920215692:6929]
2025-10-09T21:46:42.037360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341736920215692:6929]
2025-10-09T21:46:42.037374Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:42.037388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341737282033366:2432], now have 1 active actors on pipe
2025-10-09T21:46:42.037397Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:42.037401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:42.037402Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:42.037405Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:42.037415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:42.037418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:42.037463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341736920215692:6929]
2025-10-09T21:46:42.037477Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:42.037457Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 756
2025-10-09T21:46:42.037473Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 756
2025-10-09T21:46:42.037483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:42.037513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:42.037518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:42.037553Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1b9b4eeb-86fc1caa-5ca7544f-1ba8a6f4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:42.037567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:42.037571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:42.037573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:42.037585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:42.037596Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:42.037613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:42.037803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:42.037827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:42.037845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:42.038055Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 757 partition: 0 MaxSeqNo: 25 sessionId: src_id|1b9b4eeb-86fc1caa-5ca7544f-1ba8a6f4_0
2025-10-09T21:46:42.038474Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0a7eaf3-c18dc3d0-69b4ca49-ed6e8edb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1b9b4eeb-86fc1caa-5ca7544f-1ba8a6f4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:42.041327Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b9b4eeb-86fc1caa-5ca7544f-1ba8a6f4_0] Write session: try to update token
2025-10-09T21:46:42.041354Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b9b4eeb-86fc1caa-5ca7544f-1ba8a6f4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:42.065314Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:42.065341Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:42.065353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341736920215692:6929]
2025-10-09T21:46:42.065415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:42.065422Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:42.065610Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:42.065776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:42.065933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:42.066398Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:42.066626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:42.066677Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:42.068119Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 757 sessionId: src_id|1b9b4eeb-86fc1caa-5ca7544f-1ba8a6f4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:42.070866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:42.071344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:42.071366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:42.071411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:42.071709Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:42.076786Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:42.076810Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:42.076813Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:42.076815Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:42.076829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341736920215692:6929]
2025-10-09T21:46:42.076833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341736920215692:6929]
2025-10-09T21:46:42.076837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341736920215692:6929]
2025-10-09T21:46:42.076885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:42.076891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.076894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:42.077031Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:42.078363Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:42.078378Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:42.078382Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:42.110991Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341737282033456:7236]
2025-10-09T21:46:42.111523Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 756, pipe: [1:7559341737282033456:7236]
2025-10-09T21:46:42.281135Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:42.289545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:42.289580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:42.289766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:42.289828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:42.289892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:42.289957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:42.290543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:42.290567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:42.290645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:42.290722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:42.290798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:42.290872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:42.293808Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:42.293858Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341737282033456:7236]
2025-10-09T21:46:42.293864Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:42.293867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:42.293899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 757 reason: error at write: Disconnected sessionId: src_id|1b9b4eeb-86fc1caa-5ca7544f-1ba8a6f4_0
2025-10-09T21:46:42.290919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:42.290932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:42.290999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:42.291060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:42.291126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:42.291190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:42.291231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:42.291235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:42.291307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:42.291377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:42.291440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:42.291504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:42.291543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:42.291547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:42.291615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:42.291677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:42.291743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:42.291809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:42.291853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:42.291857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:42.291925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:42.291983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:42.292045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:42.294134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 757 sessionId: src_id|1b9b4eeb-86fc1caa-5ca7544f-1ba8a6f4_0 is DEAD
2025-10-09T21:46:42.292109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:42.292148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:42.292151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:42.292224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:42.292285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:42.292356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:42.292419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:42.292459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:42.292462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:42.292527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:42.292587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:42.292652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:42.292716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:42.292758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:42.292761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:42.292836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:42.292896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:42.292964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:42.293032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:42.293075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:42.293078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:42.293150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:42.293211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:42.293276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:42.293341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:42.293384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:42.293387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:42.293458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341737282033456:7236], now have 1 active actors on pipe
2025-10-09T21:46:42.294227Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:42.294301Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b9b4eeb-86fc1caa-5ca7544f-1ba8a6f4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:42.294313Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b9b4eeb-86fc1caa-5ca7544f-1ba8a6f4_0] Write session will restart in 0.008843s
2025-10-09T21:46:42.294351Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b9b4eeb-86fc1caa-5ca7544f-1ba8a6f4_0] Write session: Do CDS request
2025-10-09T21:46:42.294358Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b9b4eeb-86fc1caa-5ca7544f-1ba8a6f4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:42.304645Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b9b4eeb-86fc1caa-5ca7544f-1ba8a6f4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:42.304732Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:42.304749Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 758
2025-10-09T21:46:42.305000Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 758 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:42.305048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 758 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:42.305054Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=758 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:42.305058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:42.305624Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 758 sessionId:  describe result for acl check
2025-10-09T21:46:42.305663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:42.305671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:42.305673Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:42.305680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341737282033508:7614] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:42.306799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341737282033508:7614] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:42.306844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:42.307997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:42.308282Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:42.308286Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:42.308300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:42.308377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:42.308419Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341736920215780:6931]
2025-10-09T21:46:42.309053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:42.309496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:42.309547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:42.309936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:42.310023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.310048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.310050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.310051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.310052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.310053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.310054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.310056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.310057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.310064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.310076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.310079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.310083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.310087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.310091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.310095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.310098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.310102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.310110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.310119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.310135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.310146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.310151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.310156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.310160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.310165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.310174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.310181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.310185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.310190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.310194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.310199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.310203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.310208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.310212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.310217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.310245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:42.313641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341737282033508:7614] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:42.317788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341737282033508:7614] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:42.317809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341737282033508:7614] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:42.317814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341737282033508:7614] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:42.317825Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 758 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:42.317340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:42.318482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:42.318489Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:42.318492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:42.318498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:42.318506Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 757 [2:7559341736920215780:6931]
2025-10-09T21:46:42.318520Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:42.318539Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:42.318543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:42.318547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:42.318549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:42.318569Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:42.318575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:42.318582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:42.318650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.318652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.318654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.318657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:42.318733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:42.318738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.318752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.318765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.318784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:42.318805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341736920215773:6931]
2025-10-09T21:46:42.318807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341736920215773:6931]
2025-10-09T21:46:42.318810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341736920215773:6931]
2025-10-09T21:46:42.318822Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:42.318838Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:42.318854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:42.318858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:42.318862Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:42.318867Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:42.318870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:42.318897Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341736920215773:6931]
2025-10-09T21:46:42.318900Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:42.318911Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:42.318925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341737282033465:2432], now have 1 active actors on pipe
2025-10-09T21:46:42.318930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341737282033548:7614], now have 1 active actors on pipe
2025-10-09T21:46:42.319473Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 757
2025-10-09T21:46:42.319517Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 757
2025-10-09T21:46:42.319631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:42.319644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:42.319674Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2375cc29-f7e9cea0-ff91970e-8f9efae5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:42.319683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:42.319687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:42.319690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:42.319701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:42.319713Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:42.319729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:42.319989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:42.319999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:42.320017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:42.320132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 758 partition: 0 MaxSeqNo: 25 sessionId: src_id|2375cc29-f7e9cea0-ff91970e-8f9efae5_0
2025-10-09T21:46:42.320577Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b9b4eeb-86fc1caa-5ca7544f-1ba8a6f4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2375cc29-f7e9cea0-ff91970e-8f9efae5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:42.323255Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2375cc29-f7e9cea0-ff91970e-8f9efae5_0] Write session: try to update token
2025-10-09T21:46:42.323276Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2375cc29-f7e9cea0-ff91970e-8f9efae5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:42.341373Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:42.341397Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:42.341400Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:42.341402Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:42.341413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341736920215773:6931]
2025-10-09T21:46:42.341418Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341736920215773:6931]
2025-10-09T21:46:42.341422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341736920215773:6931]
2025-10-09T21:46:42.341473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:42.341483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.341486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:42.341516Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:42.341534Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:42.341542Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:42.341659Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:42.345406Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 758 sessionId: src_id|2375cc29-f7e9cea0-ff91970e-8f9efae5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:42.348297Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:42.394420Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341737282033556:7236]
2025-10-09T21:46:42.394910Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 757, pipe: [1:7559341737282033556:7236]
2025-10-09T21:46:42.510362Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:42.511310Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:42.511386Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341737282033556:7236]
2025-10-09T21:46:42.511393Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:42.511398Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:42.511514Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 758 reason: error at write: Disconnected sessionId: src_id|2375cc29-f7e9cea0-ff91970e-8f9efae5_0
2025-10-09T21:46:42.511730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 758 sessionId: src_id|2375cc29-f7e9cea0-ff91970e-8f9efae5_0 is DEAD
2025-10-09T21:46:42.511903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2375cc29-f7e9cea0-ff91970e-8f9efae5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:42.511926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2375cc29-f7e9cea0-ff91970e-8f9efae5_0] Write session will restart in 0.009381s
2025-10-09T21:46:42.511952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2375cc29-f7e9cea0-ff91970e-8f9efae5_0] Write session: Do CDS request
2025-10-09T21:46:42.511959Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2375cc29-f7e9cea0-ff91970e-8f9efae5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:42.511007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:42.511028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:42.511078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341737282033556:7236], now have 1 active actors on pipe
2025-10-09T21:46:42.512544Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:42.522577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2375cc29-f7e9cea0-ff91970e-8f9efae5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:42.523041Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:42.523059Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 759
2025-10-09T21:46:42.523543Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 759 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:42.523590Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 759 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:42.523596Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=759 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:42.523600Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:42.524250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 759 sessionId:  describe result for acl check
2025-10-09T21:46:42.524297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:42.524308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:42.524310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:42.524318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341737282033593:7618] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:42.524938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341737282033593:7618] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:42.528679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:42.530047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341737282033593:7618] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:42.530562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:42.530818Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:42.530831Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:42.530846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:42.530911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:42.530960Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341736920215862:6933]
2025-10-09T21:46:42.531543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:42.532119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:42.532200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:42.532531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:42.532591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.532594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.532595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.532597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.532598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.532599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.532600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.532602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.532603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.532609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.532619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.532623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.532627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.532630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.532633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.532636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.532640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.532643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.532651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.532659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.532665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.532671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.532675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.532680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.532686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.532691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.532695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.532699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.532704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.532709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.532714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.532719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.532724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.532728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.532733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.532738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.532781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:42.533799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341737282033593:7618] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:42.533823Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341737282033593:7618] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:42.533827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341737282033593:7618] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:42.533838Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 759 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:42.541360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:42.542582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:42.542605Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:42.542609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:42.542614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:42.542622Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 758 [2:7559341736920215862:6933]
2025-10-09T21:46:42.542644Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:42.542666Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:42.542670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:42.542674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:42.542676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:42.542697Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:42.542704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:42.542711Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:42.542787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.542798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.542800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.542803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:42.542884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:42.542894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.542908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.542919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.542933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:42.542955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341736920215856:6933]
2025-10-09T21:46:42.542958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341736920215856:6933]
2025-10-09T21:46:42.542960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341736920215856:6933]
2025-10-09T21:46:42.542970Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:42.542985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:42.542996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:42.542999Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:42.543002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:42.543007Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:42.543011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:42.543066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341737282033643:7618], now have 1 active actors on pipe
2025-10-09T21:46:42.543073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341737282033558:2432], now have 1 active actors on pipe
2025-10-09T21:46:42.543076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341736920215856:6933]
2025-10-09T21:46:42.543086Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:42.543096Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:42.543167Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 758
2025-10-09T21:46:42.543221Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 758
2025-10-09T21:46:42.543368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:42.543380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:42.543410Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|250c1f25-d2e6bd7c-3f34a525-ec73c8b9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:42.543418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:42.543423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:42.543426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:42.543440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:42.543452Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:42.543472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:42.544013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:42.544026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:42.544049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:42.544242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 759 partition: 0 MaxSeqNo: 25 sessionId: src_id|250c1f25-d2e6bd7c-3f34a525-ec73c8b9_0
2025-10-09T21:46:42.544696Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2375cc29-f7e9cea0-ff91970e-8f9efae5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|250c1f25-d2e6bd7c-3f34a525-ec73c8b9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:42.547409Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|250c1f25-d2e6bd7c-3f34a525-ec73c8b9_0] Write session: try to update token
2025-10-09T21:46:42.547434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|250c1f25-d2e6bd7c-3f34a525-ec73c8b9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:42.563183Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:42.563204Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:42.563207Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:42.563209Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:42.563220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341736920215856:6933]
2025-10-09T21:46:42.563226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341736920215856:6933]
2025-10-09T21:46:42.563230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341736920215856:6933]
2025-10-09T21:46:42.563289Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:42.563293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:42.563297Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:42.563302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.563302Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:42.563305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:42.563505Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:42.568779Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 759 sessionId: src_id|250c1f25-d2e6bd7c-3f34a525-ec73c8b9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:42.571469Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:42.612381Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341737282033650:7236]
2025-10-09T21:46:42.612802Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 758, pipe: [1:7559341737282033650:7236]
2025-10-09T21:46:42.735160Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:42.735577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:42.735595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:42.735645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341737282033650:7236], now have 1 active actors on pipe
2025-10-09T21:46:42.735890Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:42.736701Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|250c1f25-d2e6bd7c-3f34a525-ec73c8b9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:42.736723Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|250c1f25-d2e6bd7c-3f34a525-ec73c8b9_0] Write session will restart in 0.009123s
2025-10-09T21:46:42.736749Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|250c1f25-d2e6bd7c-3f34a525-ec73c8b9_0] Write session: Do CDS request
2025-10-09T21:46:42.736756Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|250c1f25-d2e6bd7c-3f34a525-ec73c8b9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:42.736225Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:42.736251Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:42.736316Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 759 reason: error at write: Disconnected sessionId: src_id|250c1f25-d2e6bd7c-3f34a525-ec73c8b9_0
2025-10-09T21:46:42.736541Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 759 sessionId: src_id|250c1f25-d2e6bd7c-3f34a525-ec73c8b9_0 is DEAD
2025-10-09T21:46:42.736750Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:42.736785Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341737282033650:7236]
2025-10-09T21:46:42.747647Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|250c1f25-d2e6bd7c-3f34a525-ec73c8b9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:42.747771Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:42.747788Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 760
2025-10-09T21:46:42.748019Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 760 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:42.748063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 760 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:42.748070Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=760 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:42.748075Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:42.748612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 760 sessionId:  describe result for acl check
2025-10-09T21:46:42.748674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:42.748676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:42.748678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:42.748686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341737282033701:7623] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:42.749327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341737282033701:7623] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:42.748152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:42.751208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:42.751448Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:42.751462Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:42.751477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:42.751546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:42.751594Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341736920215949:6937]
2025-10-09T21:46:42.752156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:42.752359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341737282033701:7623] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:42.752721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:42.752797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:42.753168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:42.753223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.753233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.753235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.753236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.753237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.753239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.753240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.753241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.753243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.753249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.753260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.753264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.753270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.753276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.753281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.753285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.753289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.753293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.753299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.753309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.753316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.753321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.753326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.753332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.753336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.753342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.753347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.753352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.753357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.753362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.753367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.753373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.753377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.753383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.753388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.753393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.753424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:42.761635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341737282033701:7623] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:42.761658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341737282033701:7623] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:42.761663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341737282033701:7623] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:42.761675Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 760 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:42.762466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:42.764027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:42.764046Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:42.764050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:42.764054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:42.764066Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 759 [2:7559341736920215949:6937]
2025-10-09T21:46:42.764092Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:42.764112Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:42.764115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:42.764119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:42.764122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:42.764140Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:42.764151Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:42.764158Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:42.764247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.764255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.764257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.764260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:42.764339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:42.764347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.764360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.764376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.764397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:42.764421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341737282033655:2432], now have 1 active actors on pipe
2025-10-09T21:46:42.764430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341737282033739:7623], now have 1 active actors on pipe
2025-10-09T21:46:42.764438Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341736920215942:6937]
2025-10-09T21:46:42.764440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341736920215942:6937]
2025-10-09T21:46:42.764442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341736920215942:6937]
2025-10-09T21:46:42.764455Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:42.764478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:42.764482Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:42.764493Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:42.764495Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:42.764495Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:42.764498Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:42.764541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341736920215942:6937]
2025-10-09T21:46:42.764555Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:42.764558Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 759
2025-10-09T21:46:42.764585Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 759
2025-10-09T21:46:42.764580Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:42.764794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:42.764809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:42.764850Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a77ec426-a2bbea07-8152202f-9d806107_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:42.764865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:42.764871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:42.764873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:42.764884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:42.764915Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:42.764935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:42.765202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:42.765217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:42.765238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:42.765501Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 760 partition: 0 MaxSeqNo: 25 sessionId: src_id|a77ec426-a2bbea07-8152202f-9d806107_0
2025-10-09T21:46:42.765842Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|250c1f25-d2e6bd7c-3f34a525-ec73c8b9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a77ec426-a2bbea07-8152202f-9d806107_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:42.769327Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a77ec426-a2bbea07-8152202f-9d806107_0] Write session: try to update token
2025-10-09T21:46:42.769350Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a77ec426-a2bbea07-8152202f-9d806107_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:42.785266Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:42.785290Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:42.785302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341736920215942:6937]
2025-10-09T21:46:42.785368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:42.785560Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:42.785717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:42.785879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:42.786096Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:42.786175Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:42.786405Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:42.786453Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:42.793872Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:42.793899Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:42.793902Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:42.793909Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:42.793919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341736920215942:6937]
2025-10-09T21:46:42.793924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341736920215942:6937]
2025-10-09T21:46:42.793929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341736920215942:6937]
2025-10-09T21:46:42.793992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:42.794000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.794003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:42.794182Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:42.795717Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:42.795736Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:42.795741Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:42.797746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 760 sessionId: src_id|a77ec426-a2bbea07-8152202f-9d806107_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:42.801044Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:42.837007Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341737282033747:7236]
2025-10-09T21:46:42.837550Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 759, pipe: [1:7559341737282033747:7236]
2025-10-09T21:46:42.962847Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:42.963159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:42.963176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:42.963201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341737282033747:7236], now have 1 active actors on pipe
2025-10-09T21:46:42.963453Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:42.963458Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:42.963498Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341737282033747:7236]
2025-10-09T21:46:42.963521Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:42.963524Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:42.963607Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 760 reason: error at write: Disconnected sessionId: src_id|a77ec426-a2bbea07-8152202f-9d806107_0
2025-10-09T21:46:42.963800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 760 sessionId: src_id|a77ec426-a2bbea07-8152202f-9d806107_0 is DEAD
2025-10-09T21:46:42.963918Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a77ec426-a2bbea07-8152202f-9d806107_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:42.963933Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a77ec426-a2bbea07-8152202f-9d806107_0] Write session will restart in 0.008550s
2025-10-09T21:46:42.963963Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a77ec426-a2bbea07-8152202f-9d806107_0] Write session: Do CDS request
2025-10-09T21:46:42.963968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a77ec426-a2bbea07-8152202f-9d806107_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:42.973564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a77ec426-a2bbea07-8152202f-9d806107_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:42.973713Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:42.973740Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 761
2025-10-09T21:46:42.973869Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 761 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:42.973909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 761 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:42.973921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=761 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:42.973925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:42.974414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 761 sessionId:  describe result for acl check
2025-10-09T21:46:42.974454Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:42.974463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:42.974465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:42.974471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341737282033786:7634] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:42.975066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341737282033786:7634] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:42.977952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341737282033786:7634] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:42.981290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:42.984995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:42.985269Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:42.985282Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:42.985298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:42.985376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:42.985426Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341736920216072:6951]
2025-10-09T21:46:42.985090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341737282033786:7634] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:42.985116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341737282033786:7634] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:42.985119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341737282033786:7634] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:42.985130Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 761 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:42.986049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:42.986427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:42.986481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:42.986848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:42.986923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.986930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.986931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.986933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.986934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.986935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.986937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.986938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.986940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.986947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.986957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.986962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.986966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.986970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.986974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.986977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.986981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.986985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.986994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.987002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:42.987010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.987017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:42.987022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.987027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:42.987031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.987035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:42.987038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.987043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:42.987047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.987051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:42.987062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.987067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.987074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.987080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.987084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.987090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.987128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:42.994801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:42.995952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:42.995969Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:42.995973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:42.995977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:42.995984Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 760 [2:7559341736920216072:6951]
2025-10-09T21:46:42.995999Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:42.996020Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:42.996031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:42.996037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:42.996039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:42.996058Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:42.996074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:42.996085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:42.996149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.996159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.996161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.996163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:42.996241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:42.996250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:42.996262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:42.996271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:42.996286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:42.996307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341736920216066:6951]
2025-10-09T21:46:42.996310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341736920216066:6951]
2025-10-09T21:46:42.996312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341736920216066:6951]
2025-10-09T21:46:42.996321Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:42.996328Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:42.996331Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:42.996333Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:42.996342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341737282033852:7634], now have 1 active actors on pipe
2025-10-09T21:46:42.996349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341737282033774:2432], now have 1 active actors on pipe
2025-10-09T21:46:42.996354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:42.996370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:42.996374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:42.996416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341736920216066:6951]
2025-10-09T21:46:42.996429Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:42.996437Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:42.996442Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 760
2025-10-09T21:46:42.996444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 760
2025-10-09T21:46:42.996487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:42.996497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:42.996526Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|27503130-a8993d84-bdc994de-1400d70e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:42.996539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:42.996542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:42.996546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:42.996556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:42.996566Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:42.996581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:42.996953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:42.996963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:42.996991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:42.997135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 761 partition: 0 MaxSeqNo: 25 sessionId: src_id|27503130-a8993d84-bdc994de-1400d70e_0
2025-10-09T21:46:42.997541Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a77ec426-a2bbea07-8152202f-9d806107_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|27503130-a8993d84-bdc994de-1400d70e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:43.000454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|27503130-a8993d84-bdc994de-1400d70e_0] Write session: try to update token
2025-10-09T21:46:43.000474Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|27503130-a8993d84-bdc994de-1400d70e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:43.010136Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:43.010160Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:43.010170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341736920216066:6951]
2025-10-09T21:46:43.010222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:43.010228Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:43.010413Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:43.010564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:43.010712Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:43.010864Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:43.011069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:43.011122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:43.019033Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 761 sessionId: src_id|27503130-a8993d84-bdc994de-1400d70e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:43.021996Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:43.022307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:43.022347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:43.022391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:43.022668Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:43.025047Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:43.025067Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:43.025071Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:43.025073Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:43.025090Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341736920216066:6951]
2025-10-09T21:46:43.025093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341736920216066:6951]
2025-10-09T21:46:43.025097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341736920216066:6951]
2025-10-09T21:46:43.025145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:43.025150Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:43.025157Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:43.025158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.025161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:43.025162Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:43.025304Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:43.063830Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341741577001161:7236]
2025-10-09T21:46:43.064350Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 760, pipe: [1:7559341741577001161:7236]
2025-10-09T21:46:43.191483Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:43.199751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:43.199784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:43.199961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.200032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.200091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.200155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.200200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.200205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.200271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.200325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.200385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.200452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.200492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.200496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.200561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.200614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.200669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.200726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.200768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.200771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.200841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.200903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.200970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.201027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.201073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.201077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.201147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.201202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.201269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.201336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.201374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.201377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.201441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.201493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.201549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.201609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.201651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.201654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.201719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.201775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.201835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.201891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.201936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.201939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.202022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.202285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.202355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.202413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.203448Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341741577001161:7236]
2025-10-09T21:46:43.203453Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:43.203466Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:43.203481Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:43.203500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 761 reason: error at write: Disconnected sessionId: src_id|27503130-a8993d84-bdc994de-1400d70e_0
2025-10-09T21:46:43.203697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 761 sessionId: src_id|27503130-a8993d84-bdc994de-1400d70e_0 is DEAD
2025-10-09T21:46:43.203865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27503130-a8993d84-bdc994de-1400d70e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:43.203876Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27503130-a8993d84-bdc994de-1400d70e_0] Write session will restart in 0.007344s
2025-10-09T21:46:43.202454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.202465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.202534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.202596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.203903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27503130-a8993d84-bdc994de-1400d70e_0] Write session: Do CDS request
2025-10-09T21:46:43.202658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.202717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.203909Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27503130-a8993d84-bdc994de-1400d70e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:43.202759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.202763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.202828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.202892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.202953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.203016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.203055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.203059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.203126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341741577001161:7236], now have 1 active actors on pipe
2025-10-09T21:46:43.204371Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:43.212638Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|27503130-a8993d84-bdc994de-1400d70e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:43.212701Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:43.212715Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 762
2025-10-09T21:46:43.212954Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 762 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:43.213010Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 762 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:43.213022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=762 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:43.213027Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:43.213479Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 762 sessionId:  describe result for acl check
2025-10-09T21:46:43.213518Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:43.213526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:43.213528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:43.213534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341741577001214:7639] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:43.214163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341741577001214:7639] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:43.215751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:43.217087Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341741577001214:7639] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:43.217193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:43.217402Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:43.217413Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:43.217425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:43.217488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:43.217547Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341741215183453:6953]
2025-10-09T21:46:43.218197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:43.218661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:43.218730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:43.219107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:43.219168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:43.219174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:43.219175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:43.219176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:43.219177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:43.219178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:43.219179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.219180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.219182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.219187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:43.219199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:43.219203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:43.219208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:43.219212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:43.219215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:43.219220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.219224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.219228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.219236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:43.219244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:43.219251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:43.219256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:43.219262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:43.219268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:43.219273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:43.219282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:43.219287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:43.219295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:43.219299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:43.219307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:43.219313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.219323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.219328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.219335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.219340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.219346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.219374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:43.220379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341741577001214:7639] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:43.220394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341741577001214:7639] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:43.220399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341741577001214:7639] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:43.220408Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 762 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:43.227589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:43.228880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:43.228900Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:43.228903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:43.228908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:43.228917Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 761 [2:7559341741215183453:6953]
2025-10-09T21:46:43.228931Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:43.228958Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:43.228961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:43.228965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:43.228968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:43.228989Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:43.228998Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:43.229005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:43.229091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.229100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.229102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.229105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:43.229182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:43.229191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.229205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.229216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.229233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:43.229261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341741215183447:6953]
2025-10-09T21:46:43.229269Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341741215183447:6953]
2025-10-09T21:46:43.229271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341741215183447:6953]
2025-10-09T21:46:43.229286Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:43.229306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:43.229323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:43.229326Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:43.229355Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:43.229359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:43.229362Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:43.229371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341741215183447:6953]
2025-10-09T21:46:43.229374Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:43.229449Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:43.229517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341741577001172:2432], now have 1 active actors on pipe
2025-10-09T21:46:43.229531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341741577001259:7639], now have 1 active actors on pipe
2025-10-09T21:46:43.229609Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 761
2025-10-09T21:46:43.229687Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 761
2025-10-09T21:46:43.229683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:43.229698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:43.229730Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1fa0f5d3-2476ccf1-88a22038-618866c5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:43.229754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:43.229763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:43.229765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:43.229778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:43.229795Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:43.229817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:43.230103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:43.230119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:43.230140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:43.230335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 762 partition: 0 MaxSeqNo: 25 sessionId: src_id|1fa0f5d3-2476ccf1-88a22038-618866c5_0
2025-10-09T21:46:43.230844Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27503130-a8993d84-bdc994de-1400d70e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1fa0f5d3-2476ccf1-88a22038-618866c5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:43.234247Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1fa0f5d3-2476ccf1-88a22038-618866c5_0] Write session: try to update token
2025-10-09T21:46:43.234268Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1fa0f5d3-2476ccf1-88a22038-618866c5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:43.243500Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:43.243521Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:43.243532Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341741215183447:6953]
2025-10-09T21:46:43.243582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:43.243586Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:43.243776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:43.243955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:43.244127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:43.244274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:43.244490Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:43.244539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:43.253233Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 762 sessionId: src_id|1fa0f5d3-2476ccf1-88a22038-618866c5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:43.256088Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:43.256566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:43.256585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:43.256640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:43.256737Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:43.261848Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:43.261870Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:43.261873Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:43.261875Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:43.261886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341741215183447:6953]
2025-10-09T21:46:43.261891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341741215183447:6953]
2025-10-09T21:46:43.261895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341741215183447:6953]
2025-10-09T21:46:43.261943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:43.261951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.261955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:43.261960Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:43.261979Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:43.261983Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:43.262139Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:43.306145Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341741577001266:7236]
2025-10-09T21:46:43.306477Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 761, pipe: [1:7559341741577001266:7236]
2025-10-09T21:46:43.423205Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:43.431633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:43.431666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:43.431841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.431906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.431975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.432035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.432083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.432098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.432172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.432237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.432297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.432369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.432413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.432420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.432486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.432551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.432612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.432675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.432722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.432726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.432786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.432851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.432914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.432973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.433014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.433018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.433084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.433139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.433199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.433256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.433297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.433300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.433363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.433420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.433477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.433531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.433574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.433577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.433640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.433694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.433754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.433813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.433857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.433860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.433931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.434000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.435296Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:43.435316Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:43.435321Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:43.435356Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 762 reason: error at write: Disconnected sessionId: src_id|1fa0f5d3-2476ccf1-88a22038-618866c5_0
2025-10-09T21:46:43.435358Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341741577001266:7236]
2025-10-09T21:46:43.435717Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fa0f5d3-2476ccf1-88a22038-618866c5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:43.435728Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fa0f5d3-2476ccf1-88a22038-618866c5_0] Write session will restart in 0.005609s
2025-10-09T21:46:43.435752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fa0f5d3-2476ccf1-88a22038-618866c5_0] Write session: Do CDS request
2025-10-09T21:46:43.435758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fa0f5d3-2476ccf1-88a22038-618866c5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:43.435570Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 762 sessionId: src_id|1fa0f5d3-2476ccf1-88a22038-618866c5_0 is DEAD
2025-10-09T21:46:43.434183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.434248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.434288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.434291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.434354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.434415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.434477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.434539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.434580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.434583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.434647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:43.434699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:43.434755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:43.434817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:43.434858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:43.434861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:43.434937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341741577001266:7236], now have 1 active actors on pipe
2025-10-09T21:46:43.435977Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:43.442664Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1fa0f5d3-2476ccf1-88a22038-618866c5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:43.443124Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:43.443139Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 763
2025-10-09T21:46:43.443426Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 763 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:43.443483Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 763 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:43.443494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=763 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:43.443498Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:43.444466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 763 sessionId:  describe result for acl check
2025-10-09T21:46:43.444517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:43.444525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:43.444527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:43.444533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341741577001293:7643] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:43.445123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341741577001293:7643] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:43.449428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341741577001293:7643] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:43.450611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:43.453345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:43.453600Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:43.453615Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:43.453628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:43.453695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:43.453789Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341741215183539:6955]
2025-10-09T21:46:43.454517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:43.454977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:43.455081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:43.455447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:43.455516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:43.455522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:43.455523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:43.455524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:43.455526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:43.455527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:43.455529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.455530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.455531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.455537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:43.455548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:43.455552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:43.455556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:43.455560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:43.455564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:43.455568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.455572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.455576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.455584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:43.455593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:43.455599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:43.455605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:43.455609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:43.455614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:43.455618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:43.455623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:43.455628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:43.455633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:43.455637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:43.455643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:43.455647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.455653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.455657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.455662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.455667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.455674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.455702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:43.455814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341741577001293:7643] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:43.455829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341741577001293:7643] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:43.455832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341741577001293:7643] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:43.455853Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 763 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:43.464335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:43.465618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:43.465642Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:43.465645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:43.465650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:43.465660Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 762 [2:7559341741215183539:6955]
2025-10-09T21:46:43.465678Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:43.465699Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:43.465703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:43.465708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:43.465710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:43.465736Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:43.465742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:43.465750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:43.465825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.465828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.465830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.465833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:43.465920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:43.465923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.465941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.465953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.465991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:43.466012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341741215183534:6955]
2025-10-09T21:46:43.466015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341741215183534:6955]
2025-10-09T21:46:43.466017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341741215183534:6955]
2025-10-09T21:46:43.466052Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:43.466068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:43.466083Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:43.466086Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:43.466134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341741215183534:6955]
2025-10-09T21:46:43.466137Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:43.466142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:43.466147Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:43.466150Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:43.466156Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:43.466528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341741577001269:2432], now have 1 active actors on pipe
2025-10-09T21:46:43.466535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341741577001355:7643], now have 1 active actors on pipe
2025-10-09T21:46:43.467157Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 762
2025-10-09T21:46:43.467208Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 762
2025-10-09T21:46:43.467411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:43.467427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:43.467480Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d93b0e3b-7181fde-f2939eee-1f3832fb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:43.467492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:43.467497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:43.467500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:43.467513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:43.467529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:43.467553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:43.467837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:43.467845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:43.467868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:43.468035Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 763 partition: 0 MaxSeqNo: 25 sessionId: src_id|d93b0e3b-7181fde-f2939eee-1f3832fb_0
2025-10-09T21:46:43.468419Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fa0f5d3-2476ccf1-88a22038-618866c5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d93b0e3b-7181fde-f2939eee-1f3832fb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:43.471356Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d93b0e3b-7181fde-f2939eee-1f3832fb_0] Write session: try to update token
2025-10-09T21:46:43.471380Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d93b0e3b-7181fde-f2939eee-1f3832fb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:43.486011Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:43.486044Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:43.486048Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:43.486050Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:43.486061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341741215183534:6955]
2025-10-09T21:46:43.486067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341741215183534:6955]
2025-10-09T21:46:43.486070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341741215183534:6955]
2025-10-09T21:46:43.486120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:43.486131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.486131Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:43.486134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:43.486138Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:43.486142Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:43.486281Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:43.505749Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 763 sessionId: src_id|d93b0e3b-7181fde-f2939eee-1f3832fb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:43.508999Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:43.535398Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341741577001365:7236]
2025-10-09T21:46:43.535913Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 762, pipe: [1:7559341741577001365:7236]
2025-10-09T21:46:43.714970Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:43.715527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:43.715538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:43.715563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341741577001365:7236], now have 1 active actors on pipe
2025-10-09T21:46:43.716036Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:43.716711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d93b0e3b-7181fde-f2939eee-1f3832fb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:43.716724Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d93b0e3b-7181fde-f2939eee-1f3832fb_0] Write session will restart in 0.007343s
2025-10-09T21:46:43.716749Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d93b0e3b-7181fde-f2939eee-1f3832fb_0] Write session: Do CDS request
2025-10-09T21:46:43.716755Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d93b0e3b-7181fde-f2939eee-1f3832fb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:43.715974Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:43.716035Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:43.716038Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:43.716114Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 763 reason: error at write: Disconnected sessionId: src_id|d93b0e3b-7181fde-f2939eee-1f3832fb_0
2025-10-09T21:46:43.716304Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 763 sessionId: src_id|d93b0e3b-7181fde-f2939eee-1f3832fb_0 is DEAD
2025-10-09T21:46:43.716456Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341741577001365:7236]
2025-10-09T21:46:43.725952Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d93b0e3b-7181fde-f2939eee-1f3832fb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:43.726256Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:43.726284Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 764
2025-10-09T21:46:43.726427Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 764 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:43.726461Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 764 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:43.726468Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=764 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:43.726472Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:43.727012Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 764 sessionId:  describe result for acl check
2025-10-09T21:46:43.727047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:43.727050Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:43.727052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:43.727058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341741577001415:7647] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:43.727653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341741577001415:7647] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:43.729658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:43.731138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:43.731361Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:43.731372Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:43.731385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:43.731460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:43.731506Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341741215183631:6959]
2025-10-09T21:46:43.731799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341741577001415:7647] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:43.732169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:43.732815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:43.732887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:43.733230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:43.733293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:43.733294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:43.733296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:43.733297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:43.733299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:43.733300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:43.733301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.733302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.733303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.733308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:43.733321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:43.733325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:43.733329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:43.733332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:43.733335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:43.733339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.733342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.733345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.733355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:43.733363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:43.733368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:43.733373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:43.733377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:43.733383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:43.733386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:43.733391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:43.733395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:43.733400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:43.733431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:43.733436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:43.733442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.733448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.733452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.733456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.733461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.733466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.733496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:43.741430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341741577001415:7647] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:43.741449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341741577001415:7647] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:43.741453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341741577001415:7647] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:43.741464Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 764 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:43.742066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:43.743177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:43.743196Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:43.743199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:43.743204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:43.743211Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 763 [2:7559341741215183631:6959]
2025-10-09T21:46:43.743229Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:43.743249Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:43.743252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:43.743256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:43.743258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:43.743277Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:43.743283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:43.743291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:43.743355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.743357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.743359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.743363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:43.743438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:43.743441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.743454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.743468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.743487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:43.743508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341741215183625:6959]
2025-10-09T21:46:43.743511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341741215183625:6959]
2025-10-09T21:46:43.743513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341741215183625:6959]
2025-10-09T21:46:43.743530Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:43.743546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:43.743561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:43.743565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:43.743592Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:43.743596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:43.743599Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:43.743606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341741215183625:6959]
2025-10-09T21:46:43.743609Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:43.743693Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:43.743858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341741577001368:2432], now have 1 active actors on pipe
2025-10-09T21:46:43.743886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341741577001458:7647], now have 1 active actors on pipe
2025-10-09T21:46:43.744314Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 763
2025-10-09T21:46:43.744350Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 763
2025-10-09T21:46:43.744461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:43.744478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:43.744511Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f5611f3c-f0eacac0-cb787035-2af6a175_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:43.744520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:43.744530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:43.744532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:43.744545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:43.744568Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:43.744591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:43.744845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:43.744861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:43.744897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:43.745096Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 764 partition: 0 MaxSeqNo: 25 sessionId: src_id|f5611f3c-f0eacac0-cb787035-2af6a175_0
2025-10-09T21:46:43.745530Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d93b0e3b-7181fde-f2939eee-1f3832fb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f5611f3c-f0eacac0-cb787035-2af6a175_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:43.748344Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5611f3c-f0eacac0-cb787035-2af6a175_0] Write session: try to update token
2025-10-09T21:46:43.748367Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5611f3c-f0eacac0-cb787035-2af6a175_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:43.764407Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:43.764429Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:43.764432Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:43.764434Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:43.764445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341741215183625:6959]
2025-10-09T21:46:43.764456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341741215183625:6959]
2025-10-09T21:46:43.764462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341741215183625:6959]
2025-10-09T21:46:43.764521Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:43.764527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:43.764528Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:43.764532Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:43.764535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.764541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:43.764738Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:43.770010Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 764 sessionId: src_id|f5611f3c-f0eacac0-cb787035-2af6a175_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:43.772662Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:43.812475Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341741577001466:7236]
2025-10-09T21:46:43.812920Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 763, pipe: [1:7559341741577001466:7236]
2025-10-09T21:46:43.929749Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:43.930021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:43.930164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:43.930226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341741577001466:7236], now have 1 active actors on pipe
2025-10-09T21:46:43.930635Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:43.930531Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341741577001466:7236]
2025-10-09T21:46:43.930559Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:43.930606Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:43.930608Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:43.930666Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 764 reason: error at write: Disconnected sessionId: src_id|f5611f3c-f0eacac0-cb787035-2af6a175_0
2025-10-09T21:46:43.930849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 764 sessionId: src_id|f5611f3c-f0eacac0-cb787035-2af6a175_0 is DEAD
2025-10-09T21:46:43.931252Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5611f3c-f0eacac0-cb787035-2af6a175_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:43.931265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5611f3c-f0eacac0-cb787035-2af6a175_0] Write session will restart in 0.006064s
2025-10-09T21:46:43.931301Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5611f3c-f0eacac0-cb787035-2af6a175_0] Write session: Do CDS request
2025-10-09T21:46:43.931307Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5611f3c-f0eacac0-cb787035-2af6a175_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:43.938673Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5611f3c-f0eacac0-cb787035-2af6a175_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:43.938792Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:43.938808Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 765
2025-10-09T21:46:43.939077Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 765 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:43.939114Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 765 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:43.939128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=765 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:43.939131Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:43.939596Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 765 sessionId:  describe result for acl check
2025-10-09T21:46:43.939630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:43.939635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:43.939637Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:43.939642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341741577001491:7658] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:43.940177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341741577001491:7658] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:43.943453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341741577001491:7658] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:43.947399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:43.948963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:43.949176Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:43.949187Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:43.949198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:43.949244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:43.949279Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341741215183733:6967]
2025-10-09T21:46:43.949379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341741577001491:7658] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:43.949395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341741577001491:7658] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:43.949399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341741577001491:7658] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:43.949420Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 765 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:43.949837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:43.950677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:43.950745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:43.951140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:43.951225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:43.951227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:43.951239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:43.951241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:43.951241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:43.951242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:43.951244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.951244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.951245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.951251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:43.951260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:43.951264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:43.951267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:43.951270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:43.951273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:43.951278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.951281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.951284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.951291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:43.951298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:43.951304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:43.951308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:43.951312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:43.951316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:43.951320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:43.951324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:43.951328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:43.951332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:43.951336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:43.951340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:43.951344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.951348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.951351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.951356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.951360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.951364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.951391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:43.961388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:43.962980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:43.963002Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:43.963005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:43.963009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:43.963016Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 764 [2:7559341741215183733:6967]
2025-10-09T21:46:43.963030Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:43.963050Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:43.963054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:43.963058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:43.963060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:43.963079Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:43.963091Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:43.963100Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:43.963171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.963184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.963186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.963190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:43.963265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:43.963273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.963288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:43.963300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:43.963316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:43.963339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341741577001473:2432], now have 1 active actors on pipe
2025-10-09T21:46:43.963346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341741577001567:7658], now have 1 active actors on pipe
2025-10-09T21:46:43.963354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341741215183727:6967]
2025-10-09T21:46:43.963360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341741215183727:6967]
2025-10-09T21:46:43.963362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341741215183727:6967]
2025-10-09T21:46:43.963375Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:43.963429Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 764
2025-10-09T21:46:43.963482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 764
2025-10-09T21:46:43.963391Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:43.963391Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:43.963394Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:43.963396Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:43.963405Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:43.963408Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:43.963450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341741215183727:6967]
2025-10-09T21:46:43.963453Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:43.963457Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:43.963986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:43.964018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:43.964048Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bd526b89-77861cff-682616fe-4ccbd8fa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:43.964066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:43.964071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:43.964074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:43.964086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:43.964099Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:43.964117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:43.964398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:43.964408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:43.964424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:43.964573Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 765 partition: 0 MaxSeqNo: 25 sessionId: src_id|bd526b89-77861cff-682616fe-4ccbd8fa_0
2025-10-09T21:46:43.964970Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5611f3c-f0eacac0-cb787035-2af6a175_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bd526b89-77861cff-682616fe-4ccbd8fa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:43.967622Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd526b89-77861cff-682616fe-4ccbd8fa_0] Write session: try to update token
2025-10-09T21:46:43.967640Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd526b89-77861cff-682616fe-4ccbd8fa_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:43.979827Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:43.979847Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:43.979850Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:43.979852Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:43.979861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341741215183727:6967]
2025-10-09T21:46:43.979869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341741215183727:6967]
2025-10-09T21:46:43.979872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341741215183727:6967]
2025-10-09T21:46:43.979916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:43.979928Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:43.979929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:43.979932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:43.979933Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:43.979937Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:43.980068Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:43.987440Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 765 sessionId: src_id|bd526b89-77861cff-682616fe-4ccbd8fa_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:43.990056Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:44.030497Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341745871968887:7236]
2025-10-09T21:46:44.030852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 764, pipe: [1:7559341745871968887:7236]
2025-10-09T21:46:44.151300Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:44.151635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:44.151651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:44.151711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341745871968887:7236], now have 1 active actors on pipe
2025-10-09T21:46:44.152072Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:44.152082Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:44.152085Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:44.152143Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341745871968887:7236]
2025-10-09T21:46:44.152160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 765 reason: error at write: Disconnected sessionId: src_id|bd526b89-77861cff-682616fe-4ccbd8fa_0
2025-10-09T21:46:44.152359Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 765 sessionId: src_id|bd526b89-77861cff-682616fe-4ccbd8fa_0 is DEAD
2025-10-09T21:46:44.152560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd526b89-77861cff-682616fe-4ccbd8fa_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:44.152514Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:44.152586Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd526b89-77861cff-682616fe-4ccbd8fa_0] Write session will restart in 0.007909s
2025-10-09T21:46:44.152616Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd526b89-77861cff-682616fe-4ccbd8fa_0] Write session: Do CDS request
2025-10-09T21:46:44.152622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd526b89-77861cff-682616fe-4ccbd8fa_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:44.161616Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd526b89-77861cff-682616fe-4ccbd8fa_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:44.162442Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:44.162463Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 766
2025-10-09T21:46:44.162720Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 766 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:44.162775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 766 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:44.162786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=766 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:44.162790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:44.163432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 766 sessionId:  describe result for acl check
2025-10-09T21:46:44.163488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:44.163490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:44.163492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:44.163498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341745871968912:7664] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:44.164075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341745871968912:7664] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:44.168303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341745871968912:7664] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:44.171769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341745871968912:7664] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:44.171786Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341745871968912:7664] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:44.171791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341745871968912:7664] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:44.171802Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 766 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:44.172891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:44.174286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:44.174534Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:44.174546Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:44.174560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:44.174634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:44.174692Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341745510151144:6975]
2025-10-09T21:46:44.175299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:44.175764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:44.175830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:44.176194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:44.176250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:44.176252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:44.176253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:44.176254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:44.176256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:44.176257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:44.176258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.176259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.176260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.176266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:44.176278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:44.176283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:44.176287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:44.176291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:44.176295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:44.176299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.176303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.176307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.176314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:44.176323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:44.176329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:44.176336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:44.176341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:44.176347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:44.176351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:44.176356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:44.176362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:44.176366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:44.176371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:44.176376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:44.176382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.176387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.176391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.176396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.176401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.176406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.176437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:44.184676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:44.186567Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 765
2025-10-09T21:46:44.186600Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 765
2025-10-09T21:46:44.185898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:44.185919Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:44.185922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:44.185927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:44.185935Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 765 [2:7559341745510151144:6975]
2025-10-09T21:46:44.185950Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:44.185983Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:44.185989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:44.185993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:44.185996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:44.186015Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:44.186020Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:44.186056Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:44.186125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.186127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.186130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.186133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:44.186220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:44.186223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.186238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.186250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.186268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:44.186293Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341745510151139:6975]
2025-10-09T21:46:44.186295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341745510151139:6975]
2025-10-09T21:46:44.186297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341745510151139:6975]
2025-10-09T21:46:44.186310Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:44.186324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:44.186338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:44.186341Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:44.186370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:44.186373Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:44.186376Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:44.186383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341745510151139:6975]
2025-10-09T21:46:44.186388Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:44.186403Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:44.186415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341745871968982:7664], now have 1 active actors on pipe
2025-10-09T21:46:44.186425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341745871968897:2432], now have 1 active actors on pipe
2025-10-09T21:46:44.186952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:44.186963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:44.186997Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a20e9296-f3460e6c-32d776b-84ee7807_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:44.187007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:44.187012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:44.187015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:44.187026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:44.187038Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:44.187057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:44.192856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:44.192883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:44.192929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:44.193110Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 766 partition: 0 MaxSeqNo: 25 sessionId: src_id|a20e9296-f3460e6c-32d776b-84ee7807_0
2025-10-09T21:46:44.193520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd526b89-77861cff-682616fe-4ccbd8fa_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a20e9296-f3460e6c-32d776b-84ee7807_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:44.196593Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a20e9296-f3460e6c-32d776b-84ee7807_0] Write session: try to update token
2025-10-09T21:46:44.196616Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a20e9296-f3460e6c-32d776b-84ee7807_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:44.202626Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:44.202646Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:44.202648Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:44.202651Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:44.202662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341745510151139:6975]
2025-10-09T21:46:44.202666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341745510151139:6975]
2025-10-09T21:46:44.202670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341745510151139:6975]
2025-10-09T21:46:44.202718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:44.202727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.202730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:44.202871Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:44.203835Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:44.203850Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:44.203854Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:44.221958Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 766 sessionId: src_id|a20e9296-f3460e6c-32d776b-84ee7807_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:44.224645Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:44.252157Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341745871968996:7236]
2025-10-09T21:46:44.254611Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 765, pipe: [1:7559341745871968996:7236]
2025-10-09T21:46:44.377740Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:44.378096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:44.378114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:44.378162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341745871968996:7236], now have 1 active actors on pipe
2025-10-09T21:46:44.378663Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341745871968996:7236]
2025-10-09T21:46:44.378709Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:44.378415Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:44.378784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 766 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a20e9296-f3460e6c-32d776b-84ee7807_0
2025-10-09T21:46:44.379122Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a20e9296-f3460e6c-32d776b-84ee7807_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:44.379134Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a20e9296-f3460e6c-32d776b-84ee7807_0] Write session will restart in 0.005294s
2025-10-09T21:46:44.379160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a20e9296-f3460e6c-32d776b-84ee7807_0] Write session: Do CDS request
2025-10-09T21:46:44.379166Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a20e9296-f3460e6c-32d776b-84ee7807_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:44.378961Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 766 sessionId: src_id|a20e9296-f3460e6c-32d776b-84ee7807_0 is DEAD
2025-10-09T21:46:44.379314Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:44.385564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a20e9296-f3460e6c-32d776b-84ee7807_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:44.386052Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:44.386068Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 767
2025-10-09T21:46:44.386424Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 767 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:44.386473Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 767 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:44.386479Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=767 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:44.386483Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:44.387134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 767 sessionId:  describe result for acl check
2025-10-09T21:46:44.387177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:44.387179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:44.387181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:44.387187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341745871969023:7668] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:44.387764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341745871969023:7668] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:44.391754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341745871969023:7668] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:44.392308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:44.393631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:44.393829Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:44.393833Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:44.393845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:44.393896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:44.393935Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341745510151233:6977]
2025-10-09T21:46:44.394575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:44.395079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:44.395151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:44.395541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:44.395636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:44.395648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:44.395649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:44.395651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:44.395652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:44.395653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:44.395655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.395656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.395657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.395664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:44.395676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:44.395680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:44.395684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:44.395688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:44.395692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:44.395696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.395700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.395704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.395713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:44.395722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:44.395735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:44.395741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:44.395751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:44.395756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:44.395761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:44.395766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:44.395770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:44.395782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:44.395786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:44.395796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:44.395801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.395809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.395813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.395818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.395823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.395828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.395882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:44.395928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341745871969023:7668] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:44.395946Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341745871969023:7668] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:44.395949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341745871969023:7668] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:44.395968Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 767 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:44.406256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:44.407400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:44.407408Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:44.407412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:44.407417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:44.407424Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 766 [2:7559341745510151233:6977]
2025-10-09T21:46:44.407442Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:44.407462Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:44.407466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:44.407470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:44.407474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:44.407493Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:44.407500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:44.407508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:44.407577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.407580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.407582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.407588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:44.407667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:44.407669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.407684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.407697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.407729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:44.407752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341745510151228:6977]
2025-10-09T21:46:44.407755Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341745510151228:6977]
2025-10-09T21:46:44.407757Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341745510151228:6977]
2025-10-09T21:46:44.407772Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:44.407789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:44.407806Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:44.407809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:44.407853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341745510151228:6977]
2025-10-09T21:46:44.407855Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:44.408638Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 766
2025-10-09T21:46:44.408641Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 766
2025-10-09T21:46:44.408148Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:44.408158Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:44.408161Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:44.408167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:44.409071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341745871969088:7668], now have 1 active actors on pipe
2025-10-09T21:46:44.409082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341745871968999:2432], now have 1 active actors on pipe
2025-10-09T21:46:44.409097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:44.409104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:44.409140Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d17c08f4-592ab931-44831b75-431dd649_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:44.409151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:44.409155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:44.409158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:44.409171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:44.409185Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:44.409204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:44.409553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:44.409562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:44.409580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:44.409743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 767 partition: 0 MaxSeqNo: 25 sessionId: src_id|d17c08f4-592ab931-44831b75-431dd649_0
2025-10-09T21:46:44.410273Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a20e9296-f3460e6c-32d776b-84ee7807_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d17c08f4-592ab931-44831b75-431dd649_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:44.412908Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d17c08f4-592ab931-44831b75-431dd649_0] Write session: try to update token
2025-10-09T21:46:44.412927Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d17c08f4-592ab931-44831b75-431dd649_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:44.426490Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:44.426511Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:44.426515Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:44.426517Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:44.426531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341745510151228:6977]
2025-10-09T21:46:44.426535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341745510151228:6977]
2025-10-09T21:46:44.426539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341745510151228:6977]
2025-10-09T21:46:44.426583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:44.426590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.426595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:44.426732Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:44.430371Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:44.430386Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:44.430390Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:44.443521Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 767 sessionId: src_id|d17c08f4-592ab931-44831b75-431dd649_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:44.446109Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:44.479020Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341745871969097:7236]
2025-10-09T21:46:44.479461Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 766, pipe: [1:7559341745871969097:7236]
2025-10-09T21:46:44.603352Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:44.603661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:44.603671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:44.603694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341745871969097:7236], now have 1 active actors on pipe
2025-10-09T21:46:44.604026Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:44.604460Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:44.605866Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d17c08f4-592ab931-44831b75-431dd649_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:44.605879Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d17c08f4-592ab931-44831b75-431dd649_0] Write session will restart in 0.007179s
2025-10-09T21:46:44.605906Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d17c08f4-592ab931-44831b75-431dd649_0] Write session: Do CDS request
2025-10-09T21:46:44.605912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d17c08f4-592ab931-44831b75-431dd649_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:44.604544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 767 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d17c08f4-592ab931-44831b75-431dd649_0
2025-10-09T21:46:44.604739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 767 sessionId: src_id|d17c08f4-592ab931-44831b75-431dd649_0 is DEAD
2025-10-09T21:46:44.605035Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341745871969097:7236]
2025-10-09T21:46:44.605050Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:44.614559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d17c08f4-592ab931-44831b75-431dd649_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:44.618149Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:44.618166Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 768
2025-10-09T21:46:44.618706Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 768 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:44.618745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 768 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:44.618751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=768 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:44.618756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:44.619875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 768 sessionId:  describe result for acl check
2025-10-09T21:46:44.619915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:44.619920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:44.619921Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:44.619928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341745871969143:7672] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:44.620487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341745871969143:7672] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:44.628599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:44.631734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:44.631995Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:44.631999Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:44.632013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:44.632073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:44.632115Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341745510151329:6981]
2025-10-09T21:46:44.632709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:44.633610Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341745871969143:7672] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:44.634303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:44.634366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:44.634712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:44.634777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:44.634778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:44.634779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:44.634780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:44.634782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:44.634783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:44.634784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.634785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.634787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.634793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:44.634804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:44.634808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:44.634811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:44.634814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:44.634818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:44.634821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.634825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.634827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.634835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:44.634843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:44.634849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:44.634853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:44.634857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:44.634861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:44.634865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:44.634870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:44.634874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:44.634878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:44.634882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:44.634886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:44.634891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.634896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.634900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.634904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.634909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.634914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.634940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:44.641624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341745871969143:7672] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:44.641642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341745871969143:7672] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:44.641646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341745871969143:7672] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:44.641656Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 768 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:44.645592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:44.647390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:44.647410Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:44.647412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:44.647417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:44.647426Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 767 [2:7559341745510151329:6981]
2025-10-09T21:46:44.647446Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:44.647468Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:44.647472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:44.647476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:44.647478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:44.647498Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:44.647505Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:44.647514Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:44.647585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.647592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.647594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.647597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:44.647677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:44.647681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.647697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.648397Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 767
2025-10-09T21:46:44.648449Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 767
2025-10-09T21:46:44.647712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.647730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:44.647751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341745510151324:6981]
2025-10-09T21:46:44.647756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341745510151324:6981]
2025-10-09T21:46:44.647758Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341745510151324:6981]
2025-10-09T21:46:44.647779Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:44.647828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:44.647844Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:44.647846Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:44.647847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:44.647850Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:44.647853Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:44.647894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341745510151324:6981]
2025-10-09T21:46:44.647898Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:44.647948Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:44.648525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341745871969100:2432], now have 1 active actors on pipe
2025-10-09T21:46:44.648530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341745871969193:7672], now have 1 active actors on pipe
2025-10-09T21:46:44.648746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:44.648761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:44.648795Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f26711c8-6c4cda12-3d623408-67c2b7ef_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:44.648804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:44.648810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:44.648813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:44.648827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:44.648842Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:44.648865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:44.649473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:44.649485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:44.649515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:44.653472Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 768 partition: 0 MaxSeqNo: 25 sessionId: src_id|f26711c8-6c4cda12-3d623408-67c2b7ef_0
2025-10-09T21:46:44.653889Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d17c08f4-592ab931-44831b75-431dd649_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f26711c8-6c4cda12-3d623408-67c2b7ef_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:44.656717Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f26711c8-6c4cda12-3d623408-67c2b7ef_0] Write session: try to update token
2025-10-09T21:46:44.656740Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f26711c8-6c4cda12-3d623408-67c2b7ef_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:44.674671Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:44.674692Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:44.674694Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:44.674696Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:44.674706Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341745510151324:6981]
2025-10-09T21:46:44.674710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341745510151324:6981]
2025-10-09T21:46:44.674714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341745510151324:6981]
2025-10-09T21:46:44.674761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:44.674769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.674772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:44.674913Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:44.676572Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 768 sessionId: src_id|f26711c8-6c4cda12-3d623408-67c2b7ef_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:44.679019Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:44.678350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:44.678367Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:44.678380Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:44.705221Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341745871969201:7236]
2025-10-09T21:46:44.706366Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 767, pipe: [1:7559341745871969201:7236]
2025-10-09T21:46:44.856779Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:44.857103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:44.857118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:44.857167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341745871969201:7236], now have 1 active actors on pipe
2025-10-09T21:46:44.858045Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f26711c8-6c4cda12-3d623408-67c2b7ef_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:44.857679Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:44.858058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f26711c8-6c4cda12-3d623408-67c2b7ef_0] Write session will restart in 0.008092s
2025-10-09T21:46:44.858085Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f26711c8-6c4cda12-3d623408-67c2b7ef_0] Write session: Do CDS request
2025-10-09T21:46:44.858092Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f26711c8-6c4cda12-3d623408-67c2b7ef_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:44.857483Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:44.857560Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341745871969201:7236]
2025-10-09T21:46:44.857581Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:44.857585Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:44.857673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 768 reason: error at write: Disconnected sessionId: src_id|f26711c8-6c4cda12-3d623408-67c2b7ef_0
2025-10-09T21:46:44.857865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 768 sessionId: src_id|f26711c8-6c4cda12-3d623408-67c2b7ef_0 is DEAD
2025-10-09T21:46:44.867654Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f26711c8-6c4cda12-3d623408-67c2b7ef_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:44.867846Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:44.867863Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 769
2025-10-09T21:46:44.868034Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 769 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:44.868081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 769 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:44.868092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=769 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:44.868096Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:44.868526Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 769 sessionId:  describe result for acl check
2025-10-09T21:46:44.868564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:44.868566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:44.868568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:44.868574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341745871969262:7677] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:44.869192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341745871969262:7677] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:44.870724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:44.872309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:44.872533Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:44.872545Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:44.872559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:44.872615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:44.872666Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341745510151422:6983]
2025-10-09T21:46:44.873255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:44.873202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341745871969262:7677] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:44.873650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:44.873723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:44.874095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:44.874164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:44.874166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:44.874167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:44.874169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:44.874170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:44.874171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:44.874172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.874174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.874175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.874181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:44.874202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:44.874205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:44.874209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:44.874212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:44.874216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:44.874219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.874223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.874227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.874235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:44.874244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:44.874251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:44.874255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:44.874261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:44.874266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:44.874272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:44.874277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:44.874282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:44.874287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:44.874293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:44.874298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:44.874303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.874308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.874311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.874316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.874320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.874325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.874354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:44.880599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341745871969262:7677] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:44.880619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341745871969262:7677] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:44.880623Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341745871969262:7677] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:44.880635Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 769 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:44.882145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:44.884241Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 768
2025-10-09T21:46:44.883545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:44.883554Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:44.884279Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 768
2025-10-09T21:46:44.883558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:44.883563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:44.883571Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 768 [2:7559341745510151422:6983]
2025-10-09T21:46:44.883586Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:44.883607Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:44.883611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:44.883614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:44.883618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:44.883638Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:44.883644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:44.883654Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:44.883728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.883730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.883732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.883735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:44.883818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:44.883821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.883835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:44.883845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:44.883859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:44.883882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341745510151417:6983]
2025-10-09T21:46:44.883885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341745510151417:6983]
2025-10-09T21:46:44.883887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341745510151417:6983]
2025-10-09T21:46:44.883896Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:44.883911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:44.883921Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:44.883925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:44.883926Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:44.883928Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:44.883928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:44.883981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341745871969306:7677], now have 1 active actors on pipe
2025-10-09T21:46:44.883984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341745871969209:2432], now have 1 active actors on pipe
2025-10-09T21:46:44.883987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341745510151417:6983]
2025-10-09T21:46:44.883997Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:44.884037Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:44.884473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:44.884480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:44.884513Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|390b13fb-5b1bb393-2d072a1c-5966a342_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:44.884544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:44.884551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:44.884554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:44.884567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:44.884586Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:44.884606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:44.885024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:44.885034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:44.885056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:44.885200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 769 partition: 0 MaxSeqNo: 25 sessionId: src_id|390b13fb-5b1bb393-2d072a1c-5966a342_0
2025-10-09T21:46:44.885683Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f26711c8-6c4cda12-3d623408-67c2b7ef_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|390b13fb-5b1bb393-2d072a1c-5966a342_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:44.888421Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|390b13fb-5b1bb393-2d072a1c-5966a342_0] Write session: try to update token
2025-10-09T21:46:44.888445Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|390b13fb-5b1bb393-2d072a1c-5966a342_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:44.902911Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:44.902934Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:44.902937Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:44.902940Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:44.902952Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341745510151417:6983]
2025-10-09T21:46:44.902956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341745510151417:6983]
2025-10-09T21:46:44.902964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341745510151417:6983]
2025-10-09T21:46:44.903014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:44.903020Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:44.903022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:44.903026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:44.903027Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:44.903031Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:44.903200Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:44.906193Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 769 sessionId: src_id|390b13fb-5b1bb393-2d072a1c-5966a342_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:44.909081Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:44.958770Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341745871969314:7236]
2025-10-09T21:46:44.959284Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 768, pipe: [1:7559341745871969314:7236]
2025-10-09T21:46:45.106952Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:45.107289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:45.107306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:45.107357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341745871969314:7236], now have 1 active actors on pipe
2025-10-09T21:46:45.107607Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:45.108193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|390b13fb-5b1bb393-2d072a1c-5966a342_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:45.108205Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|390b13fb-5b1bb393-2d072a1c-5966a342_0] Write session will restart in 0.008572s
2025-10-09T21:46:45.108230Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|390b13fb-5b1bb393-2d072a1c-5966a342_0] Write session: Do CDS request
2025-10-09T21:46:45.108237Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|390b13fb-5b1bb393-2d072a1c-5966a342_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:45.107699Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341745871969314:7236]
2025-10-09T21:46:45.107753Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:45.107854Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:45.107865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 769 reason: status is not ok: tablet will be restarted right now sessionId: src_id|390b13fb-5b1bb393-2d072a1c-5966a342_0
2025-10-09T21:46:45.108057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 769 sessionId: src_id|390b13fb-5b1bb393-2d072a1c-5966a342_0 is DEAD
2025-10-09T21:46:45.118628Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|390b13fb-5b1bb393-2d072a1c-5966a342_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:45.119114Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:45.119133Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 770
2025-10-09T21:46:45.119487Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 770 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:45.119528Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 770 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:45.119546Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=770 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:45.119550Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:45.120609Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 770 sessionId:  describe result for acl check
2025-10-09T21:46:45.120647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:45.120650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:45.120651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:45.120659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341750166936689:7688] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:45.121276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341750166936689:7688] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:45.128123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341750166936689:7688] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:45.130545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:45.133011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341750166936689:7688] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:45.133029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341750166936689:7688] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:45.133033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341750166936689:7688] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:45.133045Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 770 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:45.134166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:45.134427Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:45.134443Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:45.134457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:45.134527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:45.134582Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341749805118850:6997]
2025-10-09T21:46:45.135178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:45.135903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:45.135982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:45.136323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:45.136402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:45.136404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:45.136405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:45.136407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:45.136411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:45.136412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:45.136414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.136415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.136416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.136423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:45.136435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:45.136439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:45.136443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:45.136447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:45.136452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:45.136455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.136458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.136461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.136470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:45.136478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:45.136484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:45.136489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:45.136495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:45.136500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:45.136505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:45.136511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:45.136515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:45.136521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:45.136525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:45.136529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:45.136534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.136539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.136543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.136548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.136553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.136560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.136593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:45.146302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:45.147522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:45.147540Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:45.147544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:45.147548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:45.147556Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 769 [2:7559341749805118850:6997]
2025-10-09T21:46:45.147578Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:45.147601Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:45.147604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:45.147607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:45.147610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:45.147632Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:45.147637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:45.147646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:45.147724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.147730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.147732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.147735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:45.147821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:45.147829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.147845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.147859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.147873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:45.147902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341749805118844:6997]
2025-10-09T21:46:45.147904Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341749805118844:6997]
2025-10-09T21:46:45.147909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341749805118844:6997]
2025-10-09T21:46:45.147919Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:45.147934Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:45.147950Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:45.147960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:45.148008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341749805118844:6997]
2025-10-09T21:46:45.148010Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:45.148313Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:45.148321Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:45.148323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:45.148329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:45.148923Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 769
2025-10-09T21:46:45.148971Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 769
2025-10-09T21:46:45.150723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341750166936643:2432], now have 1 active actors on pipe
2025-10-09T21:46:45.150738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341750166936730:7688], now have 1 active actors on pipe
2025-10-09T21:46:45.150765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:45.150772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:45.150810Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|25bb1790-33d126cd-8b2171-95292715_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:45.150822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:45.150827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:45.150830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:45.150844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:45.150863Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:45.150882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:45.151293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:45.151304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:45.151320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:45.151437Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 770 partition: 0 MaxSeqNo: 25 sessionId: src_id|25bb1790-33d126cd-8b2171-95292715_0
2025-10-09T21:46:45.151836Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|390b13fb-5b1bb393-2d072a1c-5966a342_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|25bb1790-33d126cd-8b2171-95292715_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:45.154704Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25bb1790-33d126cd-8b2171-95292715_0] Write session: try to update token
2025-10-09T21:46:45.154730Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25bb1790-33d126cd-8b2171-95292715_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:45.169566Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:45.169589Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:45.169592Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:45.169595Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:45.169606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341749805118844:6997]
2025-10-09T21:46:45.169610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341749805118844:6997]
2025-10-09T21:46:45.169622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341749805118844:6997]
2025-10-09T21:46:45.169672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:45.169679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.169682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:45.169690Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:45.169712Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:45.169726Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:45.169823Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:45.189538Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 770 sessionId: src_id|25bb1790-33d126cd-8b2171-95292715_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:45.192338Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:45.210113Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341750166936749:7236]
2025-10-09T21:46:45.210638Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 769, pipe: [1:7559341750166936749:7236]
2025-10-09T21:46:45.354476Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:45.354944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:45.354962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:45.354987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341750166936749:7236], now have 1 active actors on pipe
2025-10-09T21:46:45.355204Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:45.355712Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341750166936749:7236]
2025-10-09T21:46:45.355742Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:45.355777Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:45.355858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 770 reason: status is not ok: tablet will be restarted right now sessionId: src_id|25bb1790-33d126cd-8b2171-95292715_0
2025-10-09T21:46:45.356036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 770 sessionId: src_id|25bb1790-33d126cd-8b2171-95292715_0 is DEAD
2025-10-09T21:46:45.356481Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25bb1790-33d126cd-8b2171-95292715_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:45.356492Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25bb1790-33d126cd-8b2171-95292715_0] Write session will restart in 0.008917s
2025-10-09T21:46:45.356516Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25bb1790-33d126cd-8b2171-95292715_0] Write session: Do CDS request
2025-10-09T21:46:45.356527Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25bb1790-33d126cd-8b2171-95292715_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:45.366593Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25bb1790-33d126cd-8b2171-95292715_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:45.366776Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:45.366802Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 771
2025-10-09T21:46:45.367014Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 771 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:45.367065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 771 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:45.367071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=771 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:45.367075Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:45.369206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 771 sessionId:  describe result for acl check
2025-10-09T21:46:45.369250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:45.369264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:45.369266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:45.369272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341750166936777:7695] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:45.369889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341750166936777:7695] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:45.376304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341750166936777:7695] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:45.380464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:45.380757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341750166936777:7695] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:45.380775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341750166936777:7695] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:45.380779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341750166936777:7695] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:45.380789Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 771 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:45.383134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:45.383382Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:45.383395Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:45.383408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:45.383474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:45.383528Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341749805118945:6999]
2025-10-09T21:46:45.384083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:45.384550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:45.384629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:45.384983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:45.385055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:45.385057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:45.385058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:45.385059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:45.385061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:45.385062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:45.385063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.385065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.385066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.385072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:45.385083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:45.385088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:45.385093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:45.385097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:45.385102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:45.385106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.385111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.385115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.385123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:45.385132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:45.385140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:45.385147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:45.385152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:45.385158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:45.385163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:45.385169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:45.385175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:45.385181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:45.385185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:45.385194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:45.385200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.385206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.385211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.385217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.385222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.385228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.385265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:45.394065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:45.395641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:45.395658Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:45.395661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:45.395665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:45.395674Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 770 [2:7559341749805118945:6999]
2025-10-09T21:46:45.395691Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:45.395712Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:45.395716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:45.395720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:45.395723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:45.395744Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:45.395760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:45.395768Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:45.395847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.395856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.395859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.395862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:45.395945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:45.395954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.396311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.396334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.396359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:45.396392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341750166936843:7695], now have 1 active actors on pipe
2025-10-09T21:46:45.396401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341750166936752:2432], now have 1 active actors on pipe
2025-10-09T21:46:45.396411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341749805118939:6999]
2025-10-09T21:46:45.396418Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341749805118939:6999]
2025-10-09T21:46:45.396420Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341749805118939:6999]
2025-10-09T21:46:45.396629Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 770
2025-10-09T21:46:45.396434Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:45.396457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:45.396472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:45.396476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:45.396523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341749805118939:6999]
2025-10-09T21:46:45.396526Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:45.396583Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:45.396587Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:45.396589Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:45.396596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:45.396676Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 770
2025-10-09T21:46:45.397571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:45.397587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:45.397626Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|88b9f6e3-e598ad1f-27f57bb0-e96ebeb9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:45.397655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:45.397668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:45.397675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:45.397699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:45.397721Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:45.397746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:45.398539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 771 partition: 0 MaxSeqNo: 25 sessionId: src_id|88b9f6e3-e598ad1f-27f57bb0-e96ebeb9_0
2025-10-09T21:46:45.399769Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25bb1790-33d126cd-8b2171-95292715_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|88b9f6e3-e598ad1f-27f57bb0-e96ebeb9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:45.398364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:45.398378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:45.398398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:45.402583Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88b9f6e3-e598ad1f-27f57bb0-e96ebeb9_0] Write session: try to update token
2025-10-09T21:46:45.402609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88b9f6e3-e598ad1f-27f57bb0-e96ebeb9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:45.415389Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:45.415410Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:45.415423Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341749805118939:6999]
2025-10-09T21:46:45.415480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:45.415662Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:45.415809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:45.415942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:45.416083Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:45.416306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:45.416350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:45.417520Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:45.429782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 771 sessionId: src_id|88b9f6e3-e598ad1f-27f57bb0-e96ebeb9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:45.432512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:45.433189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:45.433213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:45.433258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:45.436367Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:45.436390Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:45.436397Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:45.436399Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:45.436411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341749805118939:6999]
2025-10-09T21:46:45.436419Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341749805118939:6999]
2025-10-09T21:46:45.436423Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341749805118939:6999]
2025-10-09T21:46:45.436470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:45.436478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.436481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:45.436647Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:45.436846Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:45.436853Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:45.436857Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:45.437937Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:45.454349Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341750166936857:7236]
2025-10-09T21:46:45.454932Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 770, pipe: [1:7559341750166936857:7236]
2025-10-09T21:46:45.613140Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:45.613458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341750166936857:7236], now have 1 active actors on pipe
2025-10-09T21:46:45.621347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:45.621381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:45.621558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.621629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.621694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.621753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.621794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.621799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.621876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.621931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.622012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.622452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.622495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.622498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.622561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.622617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.622671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.622731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.622768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.622771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.622833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.622889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.622943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.623000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.623039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.623042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.623107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.623163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.623222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.623288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.623335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.623338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.623404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.623459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.623515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.623570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.623607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.623610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.623678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.623728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.623788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.623842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.623880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.623882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.623944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.623996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.624055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.624112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.624151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.624154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.624215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.624269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.624325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.624380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.624419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.624425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.624486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.624540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.624595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.624656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.624696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.624701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.625383Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341750166936857:7236]
2025-10-09T21:46:45.625386Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:45.625391Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:45.625427Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 771 reason: error at write: Disconnected sessionId: src_id|88b9f6e3-e598ad1f-27f57bb0-e96ebeb9_0
2025-10-09T21:46:45.625428Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:45.625641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 771 sessionId: src_id|88b9f6e3-e598ad1f-27f57bb0-e96ebeb9_0 is DEAD
2025-10-09T21:46:45.626104Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:45.626447Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88b9f6e3-e598ad1f-27f57bb0-e96ebeb9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:45.626465Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88b9f6e3-e598ad1f-27f57bb0-e96ebeb9_0] Write session will restart in 0.008200s
2025-10-09T21:46:45.626493Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88b9f6e3-e598ad1f-27f57bb0-e96ebeb9_0] Write session: Do CDS request
2025-10-09T21:46:45.626502Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88b9f6e3-e598ad1f-27f57bb0-e96ebeb9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:45.638601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88b9f6e3-e598ad1f-27f57bb0-e96ebeb9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:45.642607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:45.642622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 772
2025-10-09T21:46:45.642901Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 772 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:45.642931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 772 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:45.642937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=772 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:45.642940Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:45.643057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:45.644991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:45.645230Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:45.645244Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:45.645258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:45.645321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:45.645368Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341749805119048:7005]
2025-10-09T21:46:45.645938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:45.644052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 772 sessionId:  describe result for acl check
2025-10-09T21:46:45.644096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:45.644098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:45.644100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:45.644107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341750166936923:7699] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:45.644728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341750166936923:7699] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:45.646762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:45.646818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:45.647174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:45.647242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:45.647243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:45.647244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:45.647245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:45.647247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:45.647248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:45.647249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.647251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.647252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.647258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:45.647270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:45.647274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:45.647277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:45.647281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:45.647284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:45.647287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.647290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.647293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.647300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:45.647309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:45.647315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:45.647320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:45.647324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:45.647329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:45.647333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:45.647339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:45.647344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:45.647348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:45.647351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:45.647356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:45.647361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.647366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.647370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.647375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.647379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.647384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.647423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:45.647856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341750166936923:7699] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:45.656553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341750166936923:7699] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:45.656574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341750166936923:7699] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:45.656579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341750166936923:7699] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:45.656590Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 772 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:45.656378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:45.657598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:45.657606Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:45.657609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:45.657614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:45.657622Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 771 [2:7559341749805119048:7005]
2025-10-09T21:46:45.657640Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:45.657666Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:45.657670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:45.657674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:45.657676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:45.657698Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:45.657707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:45.657715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:45.657783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.657785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.657787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.657790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:45.657860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:45.657863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.657877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.657887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.657903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:45.657926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341750166936960:7699], now have 1 active actors on pipe
2025-10-09T21:46:45.657929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341750166936864:2432], now have 1 active actors on pipe
2025-10-09T21:46:45.657937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341749805119043:7005]
2025-10-09T21:46:45.657939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341749805119043:7005]
2025-10-09T21:46:45.657940Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341749805119043:7005]
2025-10-09T21:46:45.657973Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:45.657990Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:45.658004Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:45.658011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:45.658068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341749805119043:7005]
2025-10-09T21:46:45.658070Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:45.658074Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:45.658077Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:45.658080Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:45.658086Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:45.660696Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88b9f6e3-e598ad1f-27f57bb0-e96ebeb9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ee385540-b223b53e-805a0305-f7dff589_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:45.658266Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 771
2025-10-09T21:46:45.658297Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 771
2025-10-09T21:46:45.660219Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 772 partition: 0 MaxSeqNo: 25 sessionId: src_id|ee385540-b223b53e-805a0305-f7dff589_0
2025-10-09T21:46:45.659278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:45.659298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:45.659333Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ee385540-b223b53e-805a0305-f7dff589_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:45.659353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:45.659358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:45.659360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:45.659373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:45.659389Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:45.659409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:45.659767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:45.659772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:45.659794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:45.663489Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee385540-b223b53e-805a0305-f7dff589_0] Write session: try to update token
2025-10-09T21:46:45.663512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee385540-b223b53e-805a0305-f7dff589_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:45.673501Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:45.673525Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:45.673535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341749805119043:7005]
2025-10-09T21:46:45.673594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:45.673598Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:45.673781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:45.673948Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:45.674130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:45.674273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:45.674473Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:45.674517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:45.687258Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 772 sessionId: src_id|ee385540-b223b53e-805a0305-f7dff589_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:45.690269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:45.690312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:45.690358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:45.689883Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:45.690473Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:45.698680Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:45.698702Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:45.698705Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:45.698707Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:45.698718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341749805119043:7005]
2025-10-09T21:46:45.698722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341749805119043:7005]
2025-10-09T21:46:45.698725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341749805119043:7005]
2025-10-09T21:46:45.698787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:45.698788Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:45.698795Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:45.698797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.698799Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:45.698800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:45.698930Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:45.725698Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341750166936969:7236]
2025-10-09T21:46:45.726370Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 771, pipe: [1:7559341750166936969:7236]
2025-10-09T21:46:45.862653Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:45.870856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:45.870883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:45.871051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.871111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.871187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.871246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.871316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.871322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.871396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.871451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.871513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.871948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.871987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.871990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.872058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.872110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.872166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.872225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.872265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.872268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.872337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.872395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.872454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.872517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.872561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.872563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.872628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.872686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.872745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.872803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.872849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.872855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.872923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.872981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.873037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.873095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.873132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.873134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.873197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.873250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.873312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.873366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.873410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.873412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.873473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.873527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.873581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.873632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.873672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.873675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.873739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.873796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.873855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.873918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.873975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.873978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.874429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:45.874504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:45.874567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:45.874628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:45.874684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:45.874687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:45.874753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341750166936969:7236], now have 1 active actors on pipe
2025-10-09T21:46:45.875441Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:45.876083Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee385540-b223b53e-805a0305-f7dff589_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:45.876094Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee385540-b223b53e-805a0305-f7dff589_0] Write session will restart in 0.008874s
2025-10-09T21:46:45.876119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee385540-b223b53e-805a0305-f7dff589_0] Write session: Do CDS request
2025-10-09T21:46:45.876126Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee385540-b223b53e-805a0305-f7dff589_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:45.875523Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341750166936969:7236]
2025-10-09T21:46:45.875560Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:45.875565Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:45.875594Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 772 reason: error at write: Disconnected sessionId: src_id|ee385540-b223b53e-805a0305-f7dff589_0
2025-10-09T21:46:45.875805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 772 sessionId: src_id|ee385540-b223b53e-805a0305-f7dff589_0 is DEAD
2025-10-09T21:46:45.875942Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:45.886618Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee385540-b223b53e-805a0305-f7dff589_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:45.891012Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:45.891029Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 773
2025-10-09T21:46:45.891159Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 773 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:45.891188Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 773 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:45.891194Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=773 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:45.891197Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:45.892025Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 773 sessionId:  describe result for acl check
2025-10-09T21:46:45.892056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:45.892058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:45.892060Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:45.892067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341750166937032:7704] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:45.892615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341750166937032:7704] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:45.896436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:45.899109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:45.899344Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:45.899349Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:45.899362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:45.899419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:45.899458Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341749805119144:7007]
2025-10-09T21:46:45.900076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:45.903058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:45.903136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:45.903498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:45.903567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:45.903573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:45.903574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:45.903576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:45.903578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:45.903580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:45.903581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.903582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.903583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.903590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:45.903601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:45.903605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:45.903608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:45.903613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:45.903617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:45.903621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.903625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.903629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.903637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:45.903646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:45.903652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:45.903658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:45.903663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:45.903669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:45.903683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:45.903688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:45.903693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:45.903698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:45.903702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:45.903709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:45.903721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.903727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.903732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.903738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.903742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.903748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.903787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:45.908527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341750166937032:7704] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:45.912059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:45.913157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:45.913174Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:45.913178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:45.913185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:45.913193Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 772 [2:7559341749805119144:7007]
2025-10-09T21:46:45.913206Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:45.913225Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:45.913229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:45.913232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:45.913235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:45.913253Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:45.913259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:45.913266Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:45.913322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.913331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.913333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.913336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:45.913407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:45.913410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.913423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:45.913435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:45.913453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:45.913486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341749805119139:7007]
2025-10-09T21:46:45.913489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341749805119139:7007]
2025-10-09T21:46:45.913491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341749805119139:7007]
2025-10-09T21:46:45.913507Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:45.913525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341750166936974:2432], now have 1 active actors on pipe
2025-10-09T21:46:45.913547Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:45.913561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:45.913564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:45.913604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341749805119139:7007]
2025-10-09T21:46:45.913606Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:45.913768Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 772
2025-10-09T21:46:45.913982Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:45.913993Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:45.913997Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:45.914005Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:45.918266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341750166937032:7704] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:45.918283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341750166937032:7704] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:45.918287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341750166937032:7704] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:45.918297Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 773 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:45.919274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341750166937075:7704], now have 1 active actors on pipe
2025-10-09T21:46:45.919355Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 772
2025-10-09T21:46:45.919491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:45.919504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:45.919538Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|aa417884-986ab213-9e769a9e-6d1e13d9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:45.919556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:45.919561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:45.919564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:45.919578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:45.919597Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:45.919617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:45.920250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:45.920279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:45.920316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:45.920452Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 773 partition: 0 MaxSeqNo: 25 sessionId: src_id|aa417884-986ab213-9e769a9e-6d1e13d9_0
2025-10-09T21:46:45.920865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee385540-b223b53e-805a0305-f7dff589_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|aa417884-986ab213-9e769a9e-6d1e13d9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:45.923452Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aa417884-986ab213-9e769a9e-6d1e13d9_0] Write session: try to update token
2025-10-09T21:46:45.923472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aa417884-986ab213-9e769a9e-6d1e13d9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:45.933040Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:45.933062Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:45.933065Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:45.933067Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:45.933078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341749805119139:7007]
2025-10-09T21:46:45.933083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341749805119139:7007]
2025-10-09T21:46:45.933087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341749805119139:7007]
2025-10-09T21:46:45.933138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:45.933146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:45.933149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:45.933279Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:45.933561Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:45.933572Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:45.933576Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:45.941636Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 773 sessionId: src_id|aa417884-986ab213-9e769a9e-6d1e13d9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:45.944263Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:45.975313Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341750166937079:7236]
2025-10-09T21:46:45.975680Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 772, pipe: [1:7559341750166937079:7236]
2025-10-09T21:46:46.101632Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:46.101945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:46.101976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:46.101999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341750166937079:7236], now have 1 active actors on pipe
2025-10-09T21:46:46.102223Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:46.102562Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:46.102613Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:46.102701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 773 reason: status is not ok: tablet will be restarted right now sessionId: src_id|aa417884-986ab213-9e769a9e-6d1e13d9_0
2025-10-09T21:46:46.102915Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341750166937079:7236]
2025-10-09T21:46:46.102928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 773 sessionId: src_id|aa417884-986ab213-9e769a9e-6d1e13d9_0 is DEAD
2025-10-09T21:46:46.103130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa417884-986ab213-9e769a9e-6d1e13d9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:46.103141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa417884-986ab213-9e769a9e-6d1e13d9_0] Write session will restart in 0.005350s
2025-10-09T21:46:46.103176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa417884-986ab213-9e769a9e-6d1e13d9_0] Write session: Do CDS request
2025-10-09T21:46:46.103184Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa417884-986ab213-9e769a9e-6d1e13d9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:46.109653Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aa417884-986ab213-9e769a9e-6d1e13d9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:46.109781Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:46.109803Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 774
2025-10-09T21:46:46.111051Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 774 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:46.111103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 774 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:46.111109Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=774 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:46.111115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:46.111569Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 774 sessionId:  describe result for acl check
2025-10-09T21:46:46.111624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:46.111632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:46.111634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:46.111641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341754461904422:7714] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:46.112232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341754461904422:7714] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:46.114795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341754461904422:7714] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:46.117249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:46.119597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341754461904422:7714] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:46.119614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341754461904422:7714] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:46.119618Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341754461904422:7714] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:46.119639Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 774 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:46.119436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:46.119695Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:46.119700Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:46.119714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:46.119777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:46.119815Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341754100086568:7019]
2025-10-09T21:46:46.120427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:46.120824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:46.120885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:46.121252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:46.121316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:46.121322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:46.121323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:46.121325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:46.121326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:46.121327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:46.121328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.121330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.121331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.121337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:46.121349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:46.121354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:46.121358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:46.121362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:46.121367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:46.121371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.121375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.121380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.121386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:46.121394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:46.121400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:46.121406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:46.121411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:46.121422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:46.121431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:46.121437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:46.121442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:46.121448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:46.121457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:46.121463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:46.121468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.121473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.121478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.121483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.121490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.121495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.121528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:46.130771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:46.132082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:46.132101Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:46.132104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:46.132109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:46.132116Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 773 [2:7559341754100086568:7019]
2025-10-09T21:46:46.132132Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:46.132155Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:46.132158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:46.132161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:46.132164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:46.132183Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:46.132199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:46.132208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:46.132279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.132287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.132290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.132293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:46.132370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:46.132379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.132393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.132407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.132430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:46.132458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341754461904488:7714], now have 1 active actors on pipe
2025-10-09T21:46:46.132471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341754100086563:7019]
2025-10-09T21:46:46.132474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341754100086563:7019]
2025-10-09T21:46:46.132476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341754100086563:7019]
2025-10-09T21:46:46.132489Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:46.132500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341754461904388:2432], now have 1 active actors on pipe
2025-10-09T21:46:46.132510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:46.132526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:46.132530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:46.132548Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:46.132562Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 773
2025-10-09T21:46:46.132552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:46.132599Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 773
2025-10-09T21:46:46.132554Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:46.132575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341754100086563:7019]
2025-10-09T21:46:46.132579Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:46.132581Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:46.132844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:46.132861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:46.132893Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5252d2ca-83b4fb4f-6f737960-2fb702f3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:46.132902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:46.132908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:46.132912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:46.132924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:46.132947Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:46.132965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:46.133273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:46.133290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:46.133313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:46.133439Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 774 partition: 0 MaxSeqNo: 25 sessionId: src_id|5252d2ca-83b4fb4f-6f737960-2fb702f3_0
2025-10-09T21:46:46.133902Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa417884-986ab213-9e769a9e-6d1e13d9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5252d2ca-83b4fb4f-6f737960-2fb702f3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:46.136668Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5252d2ca-83b4fb4f-6f737960-2fb702f3_0] Write session: try to update token
2025-10-09T21:46:46.136689Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5252d2ca-83b4fb4f-6f737960-2fb702f3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:46.150400Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:46.150423Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:46.150425Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:46.150427Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:46.150436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341754100086563:7019]
2025-10-09T21:46:46.150440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341754100086563:7019]
2025-10-09T21:46:46.150443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341754100086563:7019]
2025-10-09T21:46:46.150490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:46.150496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.150499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:46.150516Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:46.150532Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:46.150537Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:46.150642Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:46.161518Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 774 sessionId: src_id|5252d2ca-83b4fb4f-6f737960-2fb702f3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:46.164672Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:46.202923Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341754461904514:7236]
2025-10-09T21:46:46.203272Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 773, pipe: [1:7559341754461904514:7236]
2025-10-09T21:46:46.320247Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:46.320619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:46.320631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:46.320654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341754461904514:7236], now have 1 active actors on pipe
2025-10-09T21:46:46.320961Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:46.320969Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:46.320973Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:46.321015Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341754461904514:7236]
2025-10-09T21:46:46.321035Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 774 reason: error at write: Disconnected sessionId: src_id|5252d2ca-83b4fb4f-6f737960-2fb702f3_0
2025-10-09T21:46:46.320893Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:46.321252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 774 sessionId: src_id|5252d2ca-83b4fb4f-6f737960-2fb702f3_0 is DEAD
2025-10-09T21:46:46.321421Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5252d2ca-83b4fb4f-6f737960-2fb702f3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:46.321434Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5252d2ca-83b4fb4f-6f737960-2fb702f3_0] Write session will restart in 0.008759s
2025-10-09T21:46:46.321473Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5252d2ca-83b4fb4f-6f737960-2fb702f3_0] Write session: Do CDS request
2025-10-09T21:46:46.321480Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5252d2ca-83b4fb4f-6f737960-2fb702f3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:46.331610Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5252d2ca-83b4fb4f-6f737960-2fb702f3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:46.331721Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:46.331738Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 775
2025-10-09T21:46:46.332986Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 775 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:46.333033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 775 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:46.333039Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=775 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:46.333043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:46.334004Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 775 sessionId:  describe result for acl check
2025-10-09T21:46:46.334057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:46.334061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:46.334063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:46.334069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341754461904569:7720] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:46.334611Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341754461904569:7720] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:46.337469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:46.338489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341754461904569:7720] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:46.342121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341754461904569:7720] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:46.342136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341754461904569:7720] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:46.342140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341754461904569:7720] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:46.342151Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 775 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:46.342758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:46.343026Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:46.343040Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:46.343055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:46.343134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:46.343183Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341754100086676:7023]
2025-10-09T21:46:46.343828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:46.347413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:46.347494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:46.347883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:46.347938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:46.347939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:46.347941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:46.347942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:46.347943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:46.347945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:46.347946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.347947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.347949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.347956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:46.347967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:46.347971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:46.347975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:46.347978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:46.347982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:46.347985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.347989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.347992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.348000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:46.348008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:46.348015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:46.348020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:46.348025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:46.348030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:46.348035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:46.348040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:46.348045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:46.348051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:46.348056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:46.348061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:46.348070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.348075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.348080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.348085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.348090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.348095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.348127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:46.359486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:46.360584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:46.360592Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:46.360595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:46.360601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:46.360609Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 774 [2:7559341754100086676:7023]
2025-10-09T21:46:46.360624Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:46.360647Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:46.360651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:46.360654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:46.360656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:46.360675Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:46.360682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:46.360693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:46.360764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.360766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.360771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.360773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:46.360862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:46.360864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.360878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.360890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.360909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:46.360930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341754100086670:7023]
2025-10-09T21:46:46.360933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341754100086670:7023]
2025-10-09T21:46:46.360935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341754100086670:7023]
2025-10-09T21:46:46.360951Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:46.360967Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:46.360981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:46.360985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:46.361029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341754100086670:7023]
2025-10-09T21:46:46.361032Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:46.361036Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:46.361038Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:46.361040Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:46.361042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341754461904612:7720], now have 1 active actors on pipe
2025-10-09T21:46:46.361046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341754461904517:2432], now have 1 active actors on pipe
2025-10-09T21:46:46.361046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:46.361828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:46.361844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:46.361883Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4920cbba-8503b225-5ef8550a-be9f2831_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:46.361893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:46.361897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:46.361900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:46.361913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:46.361926Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:46.361945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:46.361472Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 774
2025-10-09T21:46:46.361510Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 774
2025-10-09T21:46:46.362321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:46.362340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:46.362360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:46.362486Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 775 partition: 0 MaxSeqNo: 25 sessionId: src_id|4920cbba-8503b225-5ef8550a-be9f2831_0
2025-10-09T21:46:46.362888Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5252d2ca-83b4fb4f-6f737960-2fb702f3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4920cbba-8503b225-5ef8550a-be9f2831_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:46.365444Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4920cbba-8503b225-5ef8550a-be9f2831_0] Write session: try to update token
2025-10-09T21:46:46.365466Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4920cbba-8503b225-5ef8550a-be9f2831_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:46.378726Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:46.378746Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:46.378749Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:46.378754Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:46.378764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341754100086670:7023]
2025-10-09T21:46:46.378769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341754100086670:7023]
2025-10-09T21:46:46.378773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341754100086670:7023]
2025-10-09T21:46:46.378819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:46.378828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.378830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:46.378998Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:46.386323Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:46.386350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:46.386355Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:46.387069Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 775 sessionId: src_id|4920cbba-8503b225-5ef8550a-be9f2831_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:46.389613Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:46.420495Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341754461904625:7236]
2025-10-09T21:46:46.426611Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 774, pipe: [1:7559341754461904625:7236]
2025-10-09T21:46:46.546636Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:46.547103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:46.547114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:46.547165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341754461904625:7236], now have 1 active actors on pipe
2025-10-09T21:46:46.547426Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:46.548048Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4920cbba-8503b225-5ef8550a-be9f2831_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:46.548059Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4920cbba-8503b225-5ef8550a-be9f2831_0] Write session will restart in 0.006116s
2025-10-09T21:46:46.548083Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4920cbba-8503b225-5ef8550a-be9f2831_0] Write session: Do CDS request
2025-10-09T21:46:46.548089Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4920cbba-8503b225-5ef8550a-be9f2831_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:46.547563Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341754461904625:7236]
2025-10-09T21:46:46.547599Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:46.547691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 775 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4920cbba-8503b225-5ef8550a-be9f2831_0
2025-10-09T21:46:46.547897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 775 sessionId: src_id|4920cbba-8503b225-5ef8550a-be9f2831_0 is DEAD
2025-10-09T21:46:46.548078Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:46.556546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4920cbba-8503b225-5ef8550a-be9f2831_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:46.556665Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:46.556681Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 776
2025-10-09T21:46:46.556821Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 776 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:46.556856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 776 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:46.556862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=776 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:46.556871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:46.557902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 776 sessionId:  describe result for acl check
2025-10-09T21:46:46.557939Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:46.557941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:46.557943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:46.557949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341754461904687:7724] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:46.558548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341754461904687:7724] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:46.559326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:46.560940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:46.561158Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:46.561168Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:46.561179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:46.561228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:46.561262Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341754100086774:7025]
2025-10-09T21:46:46.561632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341754461904687:7724] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:46.561921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:46.562256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:46.562314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:46.562671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:46.562748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:46.562754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:46.562756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:46.562757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:46.562758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:46.562758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:46.562760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.562761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.562762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.562767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:46.562776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:46.562780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:46.562783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:46.562786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:46.562790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:46.562794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.562798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.562802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.562809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:46.562818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:46.562825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:46.562830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:46.562846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:46.562852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:46.562857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:46.562862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:46.562868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:46.562873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:46.562877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:46.562883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:46.562888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.562895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.562900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.562904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.562909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.562914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.562943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:46.568319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341754461904687:7724] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:46.568338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341754461904687:7724] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:46.568341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341754461904687:7724] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:46.568351Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 776 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:46.569795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
E1009 21:46:46.570762578 3615253 chttp2_transport.cc:1143]             ipv6:%5B::1%5D:14704: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 2500ms
2025-10-09T21:46:46.571261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:46.571281Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:46.571283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:46.571288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:46.571295Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 775 [2:7559341754100086774:7025]
2025-10-09T21:46:46.571310Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:46.571331Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:46.571334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:46.571337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:46.571340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:46.571359Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:46.571371Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:46.571378Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:46.571445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.571452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.571454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.571457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:46.571531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:46.571537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.571550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.571561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.571578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:46.571606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341754100086768:7025]
2025-10-09T21:46:46.571613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341754100086768:7025]
2025-10-09T21:46:46.571616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341754100086768:7025]
2025-10-09T21:46:46.571631Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:46.571646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:46.571647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341754461904628:2432], now have 1 active actors on pipe
2025-10-09T21:46:46.571649Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:46.571650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341754461904729:7724], now have 1 active actors on pipe
2025-10-09T21:46:46.571651Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:46.571658Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:46.571713Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 775
2025-10-09T21:46:46.571719Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 775
2025-10-09T21:46:46.571672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:46.571682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:46.571724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341754100086768:7025]
2025-10-09T21:46:46.571732Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:46.571749Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:46.571864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:46.571888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:46.571919Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a6e9539b-99bc7951-94edf6b3-e6394840_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:46.571938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:46.571949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:46.571952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:46.571963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:46.571985Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:46.572005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:46.572167Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: TRANSPORT_UNAVAILABLE. Description: [ { <main>: Error: GRpc error: (14): Too many pings } { <main>: Error: Grpc error response on endpoint localhost:14704 } ]
2025-10-09T21:46:46.572363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:46.572377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:46.572398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:46.572552Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 776 partition: 0 MaxSeqNo: 25 sessionId: src_id|a6e9539b-99bc7951-94edf6b3-e6394840_0
2025-10-09T21:46:46.574368Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4920cbba-8503b225-5ef8550a-be9f2831_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a6e9539b-99bc7951-94edf6b3-e6394840_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:46.577033Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6e9539b-99bc7951-94edf6b3-e6394840_0] Write session: try to update token
2025-10-09T21:46:46.577052Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6e9539b-99bc7951-94edf6b3-e6394840_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:46.586904Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:46.586927Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:46.586929Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:46.586931Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:46.586941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341754100086768:7025]
2025-10-09T21:46:46.586946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341754100086768:7025]
2025-10-09T21:46:46.586950Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341754100086768:7025]
2025-10-09T21:46:46.586997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:46.587005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.587010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:46.587091Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:46.587107Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:46.587111Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:46.587148Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:46.598446Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 776 sessionId: src_id|a6e9539b-99bc7951-94edf6b3-e6394840_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:46.600981Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:46.647792Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341754461904737:7236]
2025-10-09T21:46:46.648341Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 775, pipe: [1:7559341754461904737:7236]
2025-10-09T21:46:46.751288Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:46.752196Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341754461904737:7236]
2025-10-09T21:46:46.752261Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:46.752323Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:46.752326Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:46.752408Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 776 reason: error at write: Disconnected sessionId: src_id|a6e9539b-99bc7951-94edf6b3-e6394840_0
2025-10-09T21:46:46.752602Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 776 sessionId: src_id|a6e9539b-99bc7951-94edf6b3-e6394840_0 is DEAD
2025-10-09T21:46:46.751804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:46.751823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:46.751848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341754461904737:7236], now have 1 active actors on pipe
2025-10-09T21:46:46.752127Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:46.753038Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6e9539b-99bc7951-94edf6b3-e6394840_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:46.753052Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6e9539b-99bc7951-94edf6b3-e6394840_0] Write session will restart in 0.006991s
2025-10-09T21:46:46.753088Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6e9539b-99bc7951-94edf6b3-e6394840_0] Write session: Do CDS request
2025-10-09T21:46:46.753096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6e9539b-99bc7951-94edf6b3-e6394840_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:46.764061Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6e9539b-99bc7951-94edf6b3-e6394840_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:46.764183Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:46.764200Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 777
2025-10-09T21:46:46.764373Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 777 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:46.764413Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 777 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:46.764420Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=777 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:46.764424Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:46.764822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 777 sessionId:  describe result for acl check
2025-10-09T21:46:46.764861Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:46.764863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:46.764865Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:46.764871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341754461904806:7731] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:46.765500Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341754461904806:7731] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:46.766947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:46.768056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341754461904806:7731] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:46.768677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:46.769039Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:46.769051Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:46.769067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:46.769165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:46.769231Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341754100086879:7029]
2025-10-09T21:46:46.769781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:46.770182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:46.770244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:46.770626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:46.770692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:46.770702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:46.770703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:46.770706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:46.770707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:46.770708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:46.770710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.770711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.770712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.770717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:46.770727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:46.770740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:46.770743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:46.770746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:46.770750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:46.770754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.770757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.770761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.770770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:46.770778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:46.770784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:46.770790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:46.770795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:46.770808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:46.770812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:46.770818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:46.770823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:46.770828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:46.770833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:46.770838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:46.770843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.770848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.770853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.770858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.770872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.770877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.770910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:46.771013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341754461904806:7731] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:46.771030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341754461904806:7731] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:46.771033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341754461904806:7731] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:46.771047Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 777 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:46.778412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:46.780113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:46.780136Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:46.780139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:46.780144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:46.780152Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 776 [2:7559341754100086879:7029]
2025-10-09T21:46:46.780171Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:46.780195Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:46.780199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:46.780203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:46.780205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:46.780225Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:46.780241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:46.780249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:46.780328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.780337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.780340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.780343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:46.780417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:46.780426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.780441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:46.780452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:46.780468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:46.780498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341754461904745:2432], now have 1 active actors on pipe
2025-10-09T21:46:46.780508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341754461904844:7731], now have 1 active actors on pipe
2025-10-09T21:46:46.780516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341754100086874:7029]
2025-10-09T21:46:46.780519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341754100086874:7029]
2025-10-09T21:46:46.780521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341754100086874:7029]
2025-10-09T21:46:46.780538Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:46.780554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:46.780569Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:46.780572Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:46.780569Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 776
2025-10-09T21:46:46.780625Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 776
2025-10-09T21:46:46.780617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341754100086874:7029]
2025-10-09T21:46:46.780625Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:46.780637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:46.780639Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:46.780641Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:46.780648Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:46.780866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:46.780879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:46.780927Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d97ea41f-9cf9ed37-781f00a8-bbbf6c9d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:46.780942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:46.780946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:46.780953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:46.780965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:46.780992Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:46.781011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:46.781281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:46.781292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:46.781313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:46.781706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 777 partition: 0 MaxSeqNo: 25 sessionId: src_id|d97ea41f-9cf9ed37-781f00a8-bbbf6c9d_0
2025-10-09T21:46:46.782297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6e9539b-99bc7951-94edf6b3-e6394840_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d97ea41f-9cf9ed37-781f00a8-bbbf6c9d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:46.791430Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d97ea41f-9cf9ed37-781f00a8-bbbf6c9d_0] Write session: try to update token
2025-10-09T21:46:46.791455Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d97ea41f-9cf9ed37-781f00a8-bbbf6c9d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:46.799752Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:46.799779Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:46.799783Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:46.799785Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:46.799800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341754100086874:7029]
2025-10-09T21:46:46.799804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341754100086874:7029]
2025-10-09T21:46:46.799811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341754100086874:7029]
2025-10-09T21:46:46.799872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:46.799880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:46.799889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:46.800375Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:46.802150Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:46.802172Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:46.802177Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:46.812203Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 777 sessionId: src_id|d97ea41f-9cf9ed37-781f00a8-bbbf6c9d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:46.814827Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:46.852563Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341754461904854:7236]
2025-10-09T21:46:46.852955Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 776, pipe: [1:7559341754461904854:7236]
2025-10-09T21:46:46.980817Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:46.981375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:46.981397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:46.981420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341754461904854:7236], now have 1 active actors on pipe
2025-10-09T21:46:46.981623Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:46.981766Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:46.981873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 777 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d97ea41f-9cf9ed37-781f00a8-bbbf6c9d_0
2025-10-09T21:46:46.982113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 777 sessionId: src_id|d97ea41f-9cf9ed37-781f00a8-bbbf6c9d_0 is DEAD
2025-10-09T21:46:46.982179Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341754461904854:7236]
2025-10-09T21:46:46.982206Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:46.982282Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d97ea41f-9cf9ed37-781f00a8-bbbf6c9d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:46.982293Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d97ea41f-9cf9ed37-781f00a8-bbbf6c9d_0] Write session will restart in 0.009562s
2025-10-09T21:46:46.982320Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d97ea41f-9cf9ed37-781f00a8-bbbf6c9d_0] Write session: Do CDS request
2025-10-09T21:46:46.982326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d97ea41f-9cf9ed37-781f00a8-bbbf6c9d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:46.993641Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d97ea41f-9cf9ed37-781f00a8-bbbf6c9d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:46.993733Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:46.993749Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 778
2025-10-09T21:46:46.997428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:46.998351Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 778 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:46.998412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 778 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:46.998419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=778 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:46.998423Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:46.999180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:46.999394Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:46.999404Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:46.999417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:46.999470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:46.999520Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341754100086980:7031]
2025-10-09T21:46:47.000092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:47.000512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:47.000571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:47.000919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:47.000995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.001002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.001003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.001005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.001006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.001007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.001009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.001010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.001011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.001016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.001027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.001031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.001035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.001038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.001042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.001046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.001049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.001053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.001061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.001070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.001078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.001083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.001088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.001092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.001096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.001101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.001105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.001110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.001116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.001121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.001126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.001131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.001135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.001152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.001156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.001161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.001189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:47.002430Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 778 sessionId:  describe result for acl check
2025-10-09T21:46:47.002474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:47.002477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:47.002479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:47.002484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341758756872225:7735] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:47.003062Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341758756872225:7735] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:47.006930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341758756872225:7735] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:47.012849Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341758756872225:7735] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:47.012868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341758756872225:7735] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:47.012872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341758756872225:7735] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:47.012883Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 778 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:47.013834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:47.015233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:47.015253Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:47.015257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:47.015262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:47.015268Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 777 [2:7559341754100086980:7031]
2025-10-09T21:46:47.015286Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:47.015305Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:47.015316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:47.015322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:47.015325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:47.015345Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:47.015351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:47.015359Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:47.015423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.015432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.015434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.015438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:47.015526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:47.015535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.015550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.015568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.015585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:47.015607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341754100086975:7031]
2025-10-09T21:46:47.015614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341754100086975:7031]
2025-10-09T21:46:47.015617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341754100086975:7031]
2025-10-09T21:46:47.015630Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:47.015646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:47.015667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:47.015675Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:47.015675Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:47.015679Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:47.015681Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:47.015717Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341754100086975:7031]
2025-10-09T21:46:47.015726Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:47.015766Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:47.016024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341754461904860:2432], now have 1 active actors on pipe
2025-10-09T21:46:47.016037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341758756872255:7735], now have 1 active actors on pipe
2025-10-09T21:46:47.016213Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 777
2025-10-09T21:46:47.016261Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 777
2025-10-09T21:46:47.016670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:47.016684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:47.016714Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e8023ced-e5cafbc8-c4e8b720-d5dda898_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:47.016733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:47.016738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:47.016741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:47.016753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:47.016767Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:47.016786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:47.017057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:47.017064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:47.017081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:47.017566Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 778 partition: 0 MaxSeqNo: 25 sessionId: src_id|e8023ced-e5cafbc8-c4e8b720-d5dda898_0
2025-10-09T21:46:47.022686Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d97ea41f-9cf9ed37-781f00a8-bbbf6c9d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e8023ced-e5cafbc8-c4e8b720-d5dda898_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:47.025418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8023ced-e5cafbc8-c4e8b720-d5dda898_0] Write session: try to update token
2025-10-09T21:46:47.025447Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8023ced-e5cafbc8-c4e8b720-d5dda898_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:47.034911Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:47.034934Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:47.034937Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:47.034939Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:47.034951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341754100086975:7031]
2025-10-09T21:46:47.034959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341754100086975:7031]
2025-10-09T21:46:47.034963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341754100086975:7031]
2025-10-09T21:46:47.035015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:47.035022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.035026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:47.035174Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:47.038091Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:47.038099Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:47.038104Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:47.052617Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 778 sessionId: src_id|e8023ced-e5cafbc8-c4e8b720-d5dda898_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:47.055233Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:47.082500Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341758756872267:7236]
2025-10-09T21:46:47.082830Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 777, pipe: [1:7559341758756872267:7236]
2025-10-09T21:46:47.210601Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:47.210980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:47.210991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:47.211016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341758756872267:7236], now have 1 active actors on pipe
2025-10-09T21:46:47.211289Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:47.211362Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341758756872267:7236]
2025-10-09T21:46:47.211358Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:47.211389Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:47.211392Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:47.211464Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 778 reason: error at write: Disconnected sessionId: src_id|e8023ced-e5cafbc8-c4e8b720-d5dda898_0
2025-10-09T21:46:47.211824Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8023ced-e5cafbc8-c4e8b720-d5dda898_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:47.211835Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8023ced-e5cafbc8-c4e8b720-d5dda898_0] Write session will restart in 0.008264s
2025-10-09T21:46:47.211858Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8023ced-e5cafbc8-c4e8b720-d5dda898_0] Write session: Do CDS request
2025-10-09T21:46:47.211867Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8023ced-e5cafbc8-c4e8b720-d5dda898_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:47.211656Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 778 sessionId: src_id|e8023ced-e5cafbc8-c4e8b720-d5dda898_0 is DEAD
2025-10-09T21:46:47.221651Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8023ced-e5cafbc8-c4e8b720-d5dda898_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:47.221726Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:47.221744Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 779
2025-10-09T21:46:47.221890Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 779 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:47.221923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 779 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:47.221929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=779 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:47.221933Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:47.222470Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 779 sessionId:  describe result for acl check
2025-10-09T21:46:47.222512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:47.222519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:47.222521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:47.222528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341758756872324:7747] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:47.223121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341758756872324:7747] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:47.228236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341758756872324:7747] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:47.228448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:47.229989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:47.230212Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:47.230224Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:47.230237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:47.230292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:47.230329Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341758395054416:7045]
2025-10-09T21:46:47.230972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:47.231604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341758756872324:7747] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:47.231621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341758756872324:7747] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:47.231625Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341758756872324:7747] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:47.231635Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 779 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:47.231935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:47.231991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:47.232364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:47.232418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.232425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.232426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.232428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.232429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.232430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.232432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.232433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.232434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.232441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.232452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.232457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.232461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.232466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.232470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.232474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.232478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.232482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.232490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.232498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.232505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.232517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.232522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.232530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.232535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.232541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.232547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.232552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.232558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.232564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.232570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.232577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.232582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.232587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.232593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.232598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.232632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:47.241902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:47.243109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:47.243130Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:47.243134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:47.243139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:47.243148Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 778 [2:7559341758395054416:7045]
2025-10-09T21:46:47.243164Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:47.243187Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:47.243190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:47.243194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:47.243196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:47.243216Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:47.243222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:47.243231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:47.243379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.243393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.243397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.243399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:47.243486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:47.243498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.243512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.243524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.243542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:47.243564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341758395054411:7045]
2025-10-09T21:46:47.243571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341758395054411:7045]
2025-10-09T21:46:47.243573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341758395054411:7045]
2025-10-09T21:46:47.243588Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:47.243609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341758756872398:7747], now have 1 active actors on pipe
2025-10-09T21:46:47.243608Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:47.243612Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:47.243614Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:47.243622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341758756872300:2432], now have 1 active actors on pipe
2025-10-09T21:46:47.243631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:47.243645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:47.243648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:47.243693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341758395054411:7045]
2025-10-09T21:46:47.243703Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:47.243842Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:47.244273Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 778
2025-10-09T21:46:47.244310Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 778
2025-10-09T21:46:47.246120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:47.246145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:47.246188Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8c7e021a-4cc35eb7-ae5c857f-3366a273_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:47.246200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:47.246206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:47.246210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:47.246226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:47.246244Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:47.246266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:47.250089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:47.250112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:47.250152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:47.250806Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8023ced-e5cafbc8-c4e8b720-d5dda898_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8c7e021a-4cc35eb7-ae5c857f-3366a273_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:47.250322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 779 partition: 0 MaxSeqNo: 25 sessionId: src_id|8c7e021a-4cc35eb7-ae5c857f-3366a273_0
2025-10-09T21:46:47.253661Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c7e021a-4cc35eb7-ae5c857f-3366a273_0] Write session: try to update token
2025-10-09T21:46:47.253683Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c7e021a-4cc35eb7-ae5c857f-3366a273_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:47.262250Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:47.262270Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:47.262273Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:47.262275Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:47.262286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341758395054411:7045]
2025-10-09T21:46:47.262291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341758395054411:7045]
2025-10-09T21:46:47.262295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341758395054411:7045]
2025-10-09T21:46:47.262343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:47.262350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.262353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:47.262457Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:47.262473Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:47.262478Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:47.262510Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:47.269587Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 779 sessionId: src_id|8c7e021a-4cc35eb7-ae5c857f-3366a273_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:47.272308Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:47.311422Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341758756872408:7236]
2025-10-09T21:46:47.311863Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 778, pipe: [1:7559341758756872408:7236]
2025-10-09T21:46:47.428492Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:47.428987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:47.429013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:47.429063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341758756872408:7236], now have 1 active actors on pipe
2025-10-09T21:46:47.429398Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:47.429460Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:47.429464Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341758756872408:7236]
2025-10-09T21:46:47.429476Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:47.429503Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:47.429554Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 779 reason: error at write: Disconnected sessionId: src_id|8c7e021a-4cc35eb7-ae5c857f-3366a273_0
2025-10-09T21:46:47.429884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c7e021a-4cc35eb7-ae5c857f-3366a273_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:47.429892Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c7e021a-4cc35eb7-ae5c857f-3366a273_0] Write session will restart in 0.007597s
2025-10-09T21:46:47.429918Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c7e021a-4cc35eb7-ae5c857f-3366a273_0] Write session: Do CDS request
2025-10-09T21:46:47.429927Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c7e021a-4cc35eb7-ae5c857f-3366a273_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:47.429745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 779 sessionId: src_id|8c7e021a-4cc35eb7-ae5c857f-3366a273_0 is DEAD
2025-10-09T21:46:47.438605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c7e021a-4cc35eb7-ae5c857f-3366a273_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:47.438813Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:47.438842Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 780
2025-10-09T21:46:47.438996Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 780 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:47.439051Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 780 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:47.439065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=780 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:47.439069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:47.439602Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 780 sessionId:  describe result for acl check
2025-10-09T21:46:47.439648Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:47.439655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:47.439657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:47.439664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341758756872424:7751] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:47.440311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341758756872424:7751] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:47.444170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341758756872424:7751] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:47.447105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:47.449086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:47.449332Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:47.449344Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:47.449356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:47.449423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:47.449469Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341758395054520:7047]
2025-10-09T21:46:47.449343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341758756872424:7751] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:47.449368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341758756872424:7751] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:47.449373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341758756872424:7751] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:47.449384Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 780 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:47.450200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:47.450768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:47.450830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:47.451225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:47.451284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.451285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.451287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.451288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.451290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.451291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.451293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.451295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.451296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.451302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.451315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.451320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.451338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.451342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.451347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.451352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.451357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.451361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.451369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.451379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.451386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.451392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.451397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.451404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.451409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.451415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.451420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.451427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.451432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.451439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.451446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.451452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.451457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.451463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.451469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.451475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.451510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:47.461029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:47.462494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:47.462513Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:47.462516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:47.462521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:47.462530Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 779 [2:7559341758395054520:7047]
2025-10-09T21:46:47.462550Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:47.462572Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:47.462582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:47.462586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:47.462589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:47.462611Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:47.462617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:47.462626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:47.462696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.462704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.462707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.462710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:47.462797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:47.462806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.462822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.462853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.462896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:47.462923Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341758395054515:7047]
2025-10-09T21:46:47.462925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341758395054515:7047]
2025-10-09T21:46:47.462928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341758395054515:7047]
2025-10-09T21:46:47.462938Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:47.462954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:47.462972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:47.462981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:47.462983Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:47.462987Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:47.462989Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:47.463028Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341758395054515:7047]
2025-10-09T21:46:47.463042Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:47.463054Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:47.463056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341758756872411:2432], now have 1 active actors on pipe
2025-10-09T21:46:47.463060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341758756872512:7751], now have 1 active actors on pipe
2025-10-09T21:46:47.463135Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 779
2025-10-09T21:46:47.463184Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 779
2025-10-09T21:46:47.463501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:47.463519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:47.463556Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|949ed13e-386675dd-a9308aa3-af4628d3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:47.463576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:47.463580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:47.463584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:47.463596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:47.463611Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:47.463649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:47.463913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:47.463930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:47.463949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:47.464091Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 780 partition: 0 MaxSeqNo: 25 sessionId: src_id|949ed13e-386675dd-a9308aa3-af4628d3_0
2025-10-09T21:46:47.464561Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c7e021a-4cc35eb7-ae5c857f-3366a273_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|949ed13e-386675dd-a9308aa3-af4628d3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:47.467646Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|949ed13e-386675dd-a9308aa3-af4628d3_0] Write session: try to update token
2025-10-09T21:46:47.467674Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|949ed13e-386675dd-a9308aa3-af4628d3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:47.485578Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:47.485601Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:47.485605Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:47.485607Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:47.485620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341758395054515:7047]
2025-10-09T21:46:47.485625Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341758395054515:7047]
2025-10-09T21:46:47.485633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341758395054515:7047]
2025-10-09T21:46:47.485687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:47.485695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.485704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:47.485888Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:47.488888Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:47.488908Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:47.488913Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:47.496292Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 780 sessionId: src_id|949ed13e-386675dd-a9308aa3-af4628d3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:47.499013Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:47.528989Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341758756872523:7236]
2025-10-09T21:46:47.529518Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 779, pipe: [1:7559341758756872523:7236]
2025-10-09T21:46:47.657747Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:47.658428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:47.658457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:47.658527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341758756872523:7236], now have 1 active actors on pipe
2025-10-09T21:46:47.659110Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:47.659667Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|949ed13e-386675dd-a9308aa3-af4628d3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:47.659685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|949ed13e-386675dd-a9308aa3-af4628d3_0] Write session will restart in 0.009787s
2025-10-09T21:46:47.659723Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|949ed13e-386675dd-a9308aa3-af4628d3_0] Write session: Do CDS request
2025-10-09T21:46:47.659731Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|949ed13e-386675dd-a9308aa3-af4628d3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:47.658982Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:47.659024Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341758756872523:7236]
2025-10-09T21:46:47.659065Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:47.659073Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:47.659205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 780 reason: error at write: Disconnected sessionId: src_id|949ed13e-386675dd-a9308aa3-af4628d3_0
2025-10-09T21:46:47.659450Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 780 sessionId: src_id|949ed13e-386675dd-a9308aa3-af4628d3_0 is DEAD
2025-10-09T21:46:47.670643Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|949ed13e-386675dd-a9308aa3-af4628d3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:47.670841Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:47.670864Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 781
2025-10-09T21:46:47.671060Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 781 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:47.671123Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 781 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:47.671132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=781 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:47.671136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:47.671948Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 781 sessionId:  describe result for acl check
2025-10-09T21:46:47.671998Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:47.672007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:47.672010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:47.672017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341758756872591:7755] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:47.672833Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341758756872591:7755] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:47.673602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:47.675756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:47.676020Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:47.676045Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:47.676062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:47.676132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:47.676176Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341758395054628:7051]
2025-10-09T21:46:47.676523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341758756872591:7755] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:47.676792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:47.677154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:47.677213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:47.677572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:47.677644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.677652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.677653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.677655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.677656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.677657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.677659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.677660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.677661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.677667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.677678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.677681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.677685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.677689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.677692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.677697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.677700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.677704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.677712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.677721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.677728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.677733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.677737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.677742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.677746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.677750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.677755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.677759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.677763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.677767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.677771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.677776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.677779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.677784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.677788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.677792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.677818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:47.685614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341758756872591:7755] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:47.685635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341758756872591:7755] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:47.685643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341758756872591:7755] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:47.685653Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 781 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:47.686768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:47.688621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:47.688643Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:47.688646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:47.688651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:47.688660Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 780 [2:7559341758395054628:7051]
2025-10-09T21:46:47.688675Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:47.688698Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:47.688702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:47.688706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:47.688709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:47.688730Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:47.688737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:47.688745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:47.689068Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 780
2025-10-09T21:46:47.688820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.688831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.688833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.688835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:47.688923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:47.688926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.688941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.688952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.688970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:47.688992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341758756872526:2432], now have 1 active actors on pipe
2025-10-09T21:46:47.688995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341758756872632:7755], now have 1 active actors on pipe
2025-10-09T21:46:47.689004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341758395054622:7051]
2025-10-09T21:46:47.689006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341758395054622:7051]
2025-10-09T21:46:47.689008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341758395054622:7051]
2025-10-09T21:46:47.689023Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:47.689038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:47.689052Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:47.689055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:47.689069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:47.689074Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:47.689076Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:47.689102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341758395054622:7051]
2025-10-09T21:46:47.689105Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:47.689117Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:47.689118Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 780
2025-10-09T21:46:47.689516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:47.689531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:47.689567Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|42fbfff0-38cd3da9-15d599f5-b3ce9453_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:47.689576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:47.689591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:47.689598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:47.689611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:47.689625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:47.689645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:47.689870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:47.689887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:47.689909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:47.690133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 781 partition: 0 MaxSeqNo: 25 sessionId: src_id|42fbfff0-38cd3da9-15d599f5-b3ce9453_0
2025-10-09T21:46:47.690683Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|949ed13e-386675dd-a9308aa3-af4628d3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|42fbfff0-38cd3da9-15d599f5-b3ce9453_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:47.695948Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42fbfff0-38cd3da9-15d599f5-b3ce9453_0] Write session: try to update token
2025-10-09T21:46:47.695973Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42fbfff0-38cd3da9-15d599f5-b3ce9453_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:47.711120Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:47.711147Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:47.711159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341758395054622:7051]
2025-10-09T21:46:47.711222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:47.711225Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:47.711413Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:47.711569Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:47.711707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:47.711866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:47.712064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:47.712112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:47.721508Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:47.721533Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:47.721536Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:47.721538Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:47.721550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341758395054622:7051]
2025-10-09T21:46:47.721555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341758395054622:7051]
2025-10-09T21:46:47.721559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341758395054622:7051]
2025-10-09T21:46:47.721608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:47.721614Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:47.721616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.721619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:47.721626Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:47.721630Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:47.721770Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:47.726372Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 781 sessionId: src_id|42fbfff0-38cd3da9-15d599f5-b3ce9453_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:47.728942Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:47.759107Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341758756872642:7236]
2025-10-09T21:46:47.759554Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 780, pipe: [1:7559341758756872642:7236]
2025-10-09T21:46:47.884243Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:47.884694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:47.884711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:47.884735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341758756872642:7236], now have 1 active actors on pipe
2025-10-09T21:46:47.885121Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:47.885123Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:47.885127Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:47.885189Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341758756872642:7236]
2025-10-09T21:46:47.885214Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 781 reason: error at write: Disconnected sessionId: src_id|42fbfff0-38cd3da9-15d599f5-b3ce9453_0
2025-10-09T21:46:47.885412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 781 sessionId: src_id|42fbfff0-38cd3da9-15d599f5-b3ce9453_0 is DEAD
2025-10-09T21:46:47.885582Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42fbfff0-38cd3da9-15d599f5-b3ce9453_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:47.885595Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42fbfff0-38cd3da9-15d599f5-b3ce9453_0] Write session will restart in 0.006487s
2025-10-09T21:46:47.885624Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42fbfff0-38cd3da9-15d599f5-b3ce9453_0] Write session: Do CDS request
2025-10-09T21:46:47.885634Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42fbfff0-38cd3da9-15d599f5-b3ce9453_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:47.885239Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:47.894650Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42fbfff0-38cd3da9-15d599f5-b3ce9453_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:47.894784Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:47.894801Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 782
2025-10-09T21:46:47.895070Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 782 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:47.895110Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 782 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:47.895118Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=782 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:47.895122Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:47.895651Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 782 sessionId:  describe result for acl check
2025-10-09T21:46:47.895686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:47.895702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:47.895703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:47.895710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341758756872711:7760] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:47.896334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341758756872711:7760] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:47.896458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:47.897899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:47.898209Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:47.898222Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:47.898237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:47.898310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:47.898356Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341758395054733:7053]
2025-10-09T21:46:47.898925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:47.899403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:47.899137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341758756872711:7760] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:47.899471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:47.899848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:47.899915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.899922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.899923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.899925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.899926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.899927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.899928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.899930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.899932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.899937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.899949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.899954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.899957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.899960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.899964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.899967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.899970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.899973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.899981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.899989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:47.899995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.900001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:47.900006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.900012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:47.900016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.900021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:47.900027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.900037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:47.900042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.900052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:47.900058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.900068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.900072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.900082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.900087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.900092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.900126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:47.907254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341758756872711:7760] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:47.907275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341758756872711:7760] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:47.907296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341758756872711:7760] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:47.907307Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 782 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:47.907758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:47.908909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:47.908930Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:47.908934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:47.908938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:47.908945Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 781 [2:7559341758395054733:7053]
2025-10-09T21:46:47.908963Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:47.908983Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:47.908987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:47.908990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:47.908993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:47.909013Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:47.909026Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:47.909033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:47.909101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.909113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.909114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.909117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:47.909211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:47.909214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.909227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:47.909238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:47.909254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:47.909274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341758395054727:7053]
2025-10-09T21:46:47.909277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341758395054727:7053]
2025-10-09T21:46:47.909279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341758395054727:7053]
2025-10-09T21:46:47.909293Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:47.909309Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:47.909325Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:47.909328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:47.909372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341758395054727:7053]
2025-10-09T21:46:47.909375Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:47.909575Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:47.909587Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:47.909589Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:47.909595Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:47.909820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341758756872645:2432], now have 1 active actors on pipe
2025-10-09T21:46:47.909836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341758756872751:7760], now have 1 active actors on pipe
2025-10-09T21:46:47.909870Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 781
2025-10-09T21:46:47.909877Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 781
2025-10-09T21:46:47.909981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:47.909998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:47.910052Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|770bf78c-816d05ab-d60219fa-ad4b8c41_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:47.910068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:47.910073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:47.910076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:47.910088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:47.910100Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:47.910118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:47.910350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:47.910354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:47.910380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:47.910494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 782 partition: 0 MaxSeqNo: 25 sessionId: src_id|770bf78c-816d05ab-d60219fa-ad4b8c41_0
2025-10-09T21:46:47.910919Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42fbfff0-38cd3da9-15d599f5-b3ce9453_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|770bf78c-816d05ab-d60219fa-ad4b8c41_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:47.913455Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|770bf78c-816d05ab-d60219fa-ad4b8c41_0] Write session: try to update token
2025-10-09T21:46:47.913474Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|770bf78c-816d05ab-d60219fa-ad4b8c41_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:47.926079Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:47.926098Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:47.926104Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:47.926106Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:47.926118Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341758395054727:7053]
2025-10-09T21:46:47.926122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341758395054727:7053]
2025-10-09T21:46:47.926125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341758395054727:7053]
2025-10-09T21:46:47.926173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:47.926188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:47.926190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:47.926321Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:47.926326Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:47.926342Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:47.926346Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:47.929333Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 782 sessionId: src_id|770bf78c-816d05ab-d60219fa-ad4b8c41_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:47.932007Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:47.980167Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341758756872759:7236]
2025-10-09T21:46:47.980530Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 781, pipe: [1:7559341758756872759:7236]
2025-10-09T21:46:48.121370Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341758756872759:7236]
2025-10-09T21:46:48.121381Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:48.121411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:48.121414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:48.121474Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 782 reason: error at write: Disconnected sessionId: src_id|770bf78c-816d05ab-d60219fa-ad4b8c41_0
2025-10-09T21:46:48.120418Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:48.120852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:48.120865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:48.120916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341758756872759:7236], now have 1 active actors on pipe
2025-10-09T21:46:48.121382Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:48.121702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 782 sessionId: src_id|770bf78c-816d05ab-d60219fa-ad4b8c41_0 is DEAD
2025-10-09T21:46:48.121863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|770bf78c-816d05ab-d60219fa-ad4b8c41_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:48.121875Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|770bf78c-816d05ab-d60219fa-ad4b8c41_0] Write session will restart in 0.008283s
2025-10-09T21:46:48.121902Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|770bf78c-816d05ab-d60219fa-ad4b8c41_0] Write session: Do CDS request
2025-10-09T21:46:48.121908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|770bf78c-816d05ab-d60219fa-ad4b8c41_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:48.131647Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|770bf78c-816d05ab-d60219fa-ad4b8c41_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:48.131801Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:48.131819Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 783
2025-10-09T21:46:48.131937Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 783 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:48.131972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 783 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:48.131979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=783 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:48.131983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:48.132662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 783 sessionId:  describe result for acl check
2025-10-09T21:46:48.132703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:48.132705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:48.132708Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:48.132714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341763051840087:7764] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:48.133321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341763051840087:7764] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:48.138710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341763051840087:7764] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:48.144432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:48.145449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:48.145686Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:48.145697Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:48.145711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:48.145767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:48.145816Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341762690022136:7055]
2025-10-09T21:46:48.146405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:48.147347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:48.147410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:48.147774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:48.147867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:48.147869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:48.147870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:48.147872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:48.147873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:48.147874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:48.147876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.147877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.147880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.147888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:48.147899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:48.147904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:48.147908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:48.147911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:48.147915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:48.147918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.147922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.147928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.147937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:48.147948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:48.147955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:48.147960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:48.147965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:48.147970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:48.147974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:48.147978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:48.147983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:48.147988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:48.147993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:48.147998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:48.148007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.148013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.148020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.148026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.148031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.148036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.148071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:48.148211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341763051840087:7764] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:48.148234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341763051840087:7764] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:48.148239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341763051840087:7764] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:48.148250Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 783 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:48.158269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:48.159609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:48.159616Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:48.159619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:48.159625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:48.159632Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 782 [2:7559341762690022136:7055]
2025-10-09T21:46:48.159651Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:48.159672Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:48.159675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:48.159679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:48.159682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:48.159701Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:48.159709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:48.159717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:48.159776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.159779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.159781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.159784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:48.159862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:48.159865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.159882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.159892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.159908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:48.159928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341762690022131:7055]
2025-10-09T21:46:48.159931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341762690022131:7055]
2025-10-09T21:46:48.159933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341762690022131:7055]
2025-10-09T21:46:48.159946Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:48.159963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:48.159978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:48.159981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:48.160024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341762690022131:7055]
2025-10-09T21:46:48.160027Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:48.166147Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:48.166170Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:48.166177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:48.166185Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:48.167233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341763051840063:2432], now have 1 active actors on pipe
2025-10-09T21:46:48.167253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341763051840170:7764], now have 1 active actors on pipe
2025-10-09T21:46:48.170098Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 782
2025-10-09T21:46:48.170145Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 782
2025-10-09T21:46:48.174342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:48.174371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:48.174423Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c6089f3-cac43987-f1b37541-d4141af4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:48.174447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:48.174456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:48.174462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:48.174479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:48.174500Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:48.174533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:48.178191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:48.178218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:48.178262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:48.178457Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 783 partition: 0 MaxSeqNo: 25 sessionId: src_id|c6089f3-cac43987-f1b37541-d4141af4_0
2025-10-09T21:46:48.182175Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|770bf78c-816d05ab-d60219fa-ad4b8c41_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c6089f3-cac43987-f1b37541-d4141af4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:48.184911Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c6089f3-cac43987-f1b37541-d4141af4_0] Write session: try to update token
2025-10-09T21:46:48.184944Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c6089f3-cac43987-f1b37541-d4141af4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:48.195509Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:48.195534Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:48.195537Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:48.195539Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:48.195549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341762690022131:7055]
2025-10-09T21:46:48.195556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341762690022131:7055]
2025-10-09T21:46:48.195560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341762690022131:7055]
2025-10-09T21:46:48.195611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:48.195619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.195622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:48.195773Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:48.195792Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:48.195800Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:48.195803Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:48.221691Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341763051840197:7236]
2025-10-09T21:46:48.222201Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 782, pipe: [1:7559341763051840197:7236]
2025-10-09T21:46:48.228954Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 783 sessionId: src_id|c6089f3-cac43987-f1b37541-d4141af4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:48.231611Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:48.398804Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:48.399176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341763051840197:7236], now have 1 active actors on pipe
2025-10-09T21:46:48.399212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:48.399225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:48.399702Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341763051840197:7236]
2025-10-09T21:46:48.399709Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:48.399749Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:48.399752Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:48.399826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 783 reason: error at write: Disconnected sessionId: src_id|c6089f3-cac43987-f1b37541-d4141af4_0
2025-10-09T21:46:48.400045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 783 sessionId: src_id|c6089f3-cac43987-f1b37541-d4141af4_0 is DEAD
2025-10-09T21:46:48.399754Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:48.400253Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6089f3-cac43987-f1b37541-d4141af4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:48.400265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6089f3-cac43987-f1b37541-d4141af4_0] Write session will restart in 0.005054s
2025-10-09T21:46:48.400306Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6089f3-cac43987-f1b37541-d4141af4_0] Write session: Do CDS request
2025-10-09T21:46:48.400313Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6089f3-cac43987-f1b37541-d4141af4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:48.406739Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c6089f3-cac43987-f1b37541-d4141af4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:48.406879Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:48.406904Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 784
2025-10-09T21:46:48.407197Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 784 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:48.407237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 784 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:48.407244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=784 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:48.407248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:48.407921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 784 sessionId:  describe result for acl check
2025-10-09T21:46:48.407959Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:48.407961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:48.407963Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:48.407969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341763051840222:7776] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:48.408588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341763051840222:7776] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:48.418715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341763051840222:7776] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:48.424954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:48.426554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341763051840222:7776] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:48.426579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341763051840222:7776] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:48.426583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341763051840222:7776] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:48.426593Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 784 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:48.426714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:48.426992Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:48.426997Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:48.427011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:48.427083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:48.427128Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341762690022281:7069]
2025-10-09T21:46:48.427748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:48.430653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:48.430731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:48.431086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:48.431140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:48.431141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:48.431142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:48.431144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:48.431145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:48.431146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:48.431147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.431149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.431150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.431157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:48.431168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:48.431172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:48.431176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:48.431180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:48.431184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:48.431187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.431191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.431194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.431203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:48.431211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:48.431218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:48.431223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:48.431228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:48.431233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:48.431238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:48.431243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:48.431249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:48.431253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:48.431257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:48.431263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:48.431268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.431272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.431277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.431283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.431288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.431293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.431321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:48.440847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:48.442026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:48.442063Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:48.442066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:48.442071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:48.442078Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 783 [2:7559341762690022281:7069]
2025-10-09T21:46:48.442094Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:48.442114Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:48.442117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:48.442121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:48.442123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:48.442142Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:48.442156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:48.442164Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:48.442237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.442239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.442241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.442244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:48.442472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:48.442474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.442488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.442500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.442518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:48.442541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341763051840209:2432], now have 1 active actors on pipe
2025-10-09T21:46:48.442545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341763051840313:7776], now have 1 active actors on pipe
2025-10-09T21:46:48.442553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341762690022275:7069]
2025-10-09T21:46:48.442556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341762690022275:7069]
2025-10-09T21:46:48.442558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341762690022275:7069]
2025-10-09T21:46:48.442571Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:48.442586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:48.442592Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:48.442598Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:48.442600Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:48.442601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:48.442604Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:48.442645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341762690022275:7069]
2025-10-09T21:46:48.442801Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 783
2025-10-09T21:46:48.442653Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:48.442696Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:48.442856Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 783
2025-10-09T21:46:48.443277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:48.443294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:48.443337Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|da57ef73-eb17232a-2e5f4333-b895ffff_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:48.443347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:48.443351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:48.443354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:48.443366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:48.443381Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:48.443401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:48.443724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:48.443734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:48.443752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:48.443915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 784 partition: 0 MaxSeqNo: 25 sessionId: src_id|da57ef73-eb17232a-2e5f4333-b895ffff_0
2025-10-09T21:46:48.444331Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6089f3-cac43987-f1b37541-d4141af4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|da57ef73-eb17232a-2e5f4333-b895ffff_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:48.446997Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da57ef73-eb17232a-2e5f4333-b895ffff_0] Write session: try to update token
2025-10-09T21:46:48.447021Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da57ef73-eb17232a-2e5f4333-b895ffff_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:48.462510Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:48.462531Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:48.462534Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:48.462536Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:48.462547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341762690022275:7069]
2025-10-09T21:46:48.462552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341762690022275:7069]
2025-10-09T21:46:48.462556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341762690022275:7069]
2025-10-09T21:46:48.462606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:48.462637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.462640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:48.462787Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:48.466160Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:48.466188Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:48.466192Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:48.468056Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 784 sessionId: src_id|da57ef73-eb17232a-2e5f4333-b895ffff_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:48.472044Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:48.499682Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341763051840325:7236]
2025-10-09T21:46:48.500253Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 783, pipe: [1:7559341763051840325:7236]
2025-10-09T21:46:48.640716Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:48.641193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:48.641215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:48.641265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341763051840325:7236], now have 1 active actors on pipe
2025-10-09T21:46:48.641928Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341763051840325:7236]
2025-10-09T21:46:48.641968Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:48.642002Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:48.642005Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:48.642080Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 784 reason: error at write: Disconnected sessionId: src_id|da57ef73-eb17232a-2e5f4333-b895ffff_0
2025-10-09T21:46:48.642277Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 784 sessionId: src_id|da57ef73-eb17232a-2e5f4333-b895ffff_0 is DEAD
2025-10-09T21:46:48.646106Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:48.646280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da57ef73-eb17232a-2e5f4333-b895ffff_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:48.646292Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da57ef73-eb17232a-2e5f4333-b895ffff_0] Write session will restart in 0.008745s
2025-10-09T21:46:48.646330Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da57ef73-eb17232a-2e5f4333-b895ffff_0] Write session: Do CDS request
2025-10-09T21:46:48.646338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da57ef73-eb17232a-2e5f4333-b895ffff_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:48.656584Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da57ef73-eb17232a-2e5f4333-b895ffff_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:48.658383Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:48.658399Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 785
2025-10-09T21:46:48.658612Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 785 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:48.658646Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 785 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:48.658652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=785 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:48.658656Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:48.659161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 785 sessionId:  describe result for acl check
2025-10-09T21:46:48.659192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:48.659194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:48.659195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:48.659202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341763051840398:7780] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:48.659724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341763051840398:7780] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:48.662021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:48.662290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:48.662457Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:48.662460Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:48.662473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:48.662527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:48.662568Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341762690022394:7073]
2025-10-09T21:46:48.663184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:48.664950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:48.665014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:48.665351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:48.665397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:48.665399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:48.665400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:48.665401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:48.665403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:48.665404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:48.665405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.665407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.665408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.665414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:48.665426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:48.665430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:48.665434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:48.665438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:48.665442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:48.665446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.665450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.665454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.665461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:48.665470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:48.665476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:48.665481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:48.665486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:48.665492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:48.665496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:48.665501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:48.665518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:48.665524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:48.665528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:48.665534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:48.665539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.665545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.665550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.665555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.665560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.665566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.665595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:48.667702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341763051840398:7780] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:48.673452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341763051840398:7780] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:48.673472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341763051840398:7780] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:48.673477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341763051840398:7780] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:48.673488Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 785 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:48.673793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:48.675043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:48.675062Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:48.675065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:48.675070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:48.675078Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 784 [2:7559341762690022394:7073]
2025-10-09T21:46:48.675215Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:48.675234Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:48.675237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:48.675240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:48.675242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:48.675261Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:48.675268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:48.675274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:48.675336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.675338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.675340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.675343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:48.675416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:48.675420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.675435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.675447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.675461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:48.675482Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341762690022388:7073]
2025-10-09T21:46:48.675485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341762690022388:7073]
2025-10-09T21:46:48.675488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341762690022388:7073]
2025-10-09T21:46:48.675496Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:48.675511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:48.675525Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:48.675529Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:48.675569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341762690022388:7073]
2025-10-09T21:46:48.675572Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:48.675678Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:48.675679Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:48.675683Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:48.675689Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:48.675824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341763051840328:2432], now have 1 active actors on pipe
2025-10-09T21:46:48.675844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341763051840436:7780], now have 1 active actors on pipe
2025-10-09T21:46:48.676081Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 784
2025-10-09T21:46:48.676137Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 784
2025-10-09T21:46:48.676230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:48.676244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:48.676277Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1b387d73-7f580aa7-4517bd9d-fa636662_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:48.676300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:48.676310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:48.676313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:48.676327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:48.676346Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:48.676366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:48.679840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:48.679858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:48.679892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:48.680605Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da57ef73-eb17232a-2e5f4333-b895ffff_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1b387d73-7f580aa7-4517bd9d-fa636662_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:48.680133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 785 partition: 0 MaxSeqNo: 25 sessionId: src_id|1b387d73-7f580aa7-4517bd9d-fa636662_0
2025-10-09T21:46:48.683540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b387d73-7f580aa7-4517bd9d-fa636662_0] Write session: try to update token
2025-10-09T21:46:48.683558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b387d73-7f580aa7-4517bd9d-fa636662_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:48.694544Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:48.694568Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:48.694570Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:48.694573Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:48.694583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341762690022388:7073]
2025-10-09T21:46:48.694589Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341762690022388:7073]
2025-10-09T21:46:48.694593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341762690022388:7073]
2025-10-09T21:46:48.694643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:48.694656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.694661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:48.694813Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:48.698146Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:48.698165Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:48.698170Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:48.707921Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 785 sessionId: src_id|1b387d73-7f580aa7-4517bd9d-fa636662_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:48.710571Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:48.741853Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341763051840447:7236]
2025-10-09T21:46:48.742241Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 784, pipe: [1:7559341763051840447:7236]
2025-10-09T21:46:48.874787Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:48.875159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:48.875170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:48.875198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341763051840447:7236], now have 1 active actors on pipe
2025-10-09T21:46:48.875521Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:48.877907Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:48.878048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 785 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1b387d73-7f580aa7-4517bd9d-fa636662_0
2025-10-09T21:46:48.878256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 785 sessionId: src_id|1b387d73-7f580aa7-4517bd9d-fa636662_0 is DEAD
2025-10-09T21:46:48.878534Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:48.878838Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b387d73-7f580aa7-4517bd9d-fa636662_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:48.878850Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b387d73-7f580aa7-4517bd9d-fa636662_0] Write session will restart in 0.005209s
2025-10-09T21:46:48.878877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b387d73-7f580aa7-4517bd9d-fa636662_0] Write session: Do CDS request
2025-10-09T21:46:48.878884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b387d73-7f580aa7-4517bd9d-fa636662_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:48.878579Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341763051840447:7236]
2025-10-09T21:46:48.885584Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b387d73-7f580aa7-4517bd9d-fa636662_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:48.886132Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:48.886150Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 786
2025-10-09T21:46:48.886323Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 786 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:48.886360Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 786 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:48.886370Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=786 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:48.886374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:48.886863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 786 sessionId:  describe result for acl check
2025-10-09T21:46:48.886900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:48.886902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:48.886904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:48.886911Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341763051840465:7785] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:48.887504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341763051840465:7785] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:48.891024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341763051840465:7785] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:48.898890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341763051840465:7785] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:48.898912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341763051840465:7785] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:48.898916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341763051840465:7785] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:48.898927Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 786 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:48.901782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:48.904400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:48.904662Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:48.904674Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:48.904686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:48.904741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:48.904787Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341762690022504:7075]
2025-10-09T21:46:48.905362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:48.905848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:48.905913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:48.906261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:48.906325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:48.906327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:48.906328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:48.906329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:48.906331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:48.906334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:48.906335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.906337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.906337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.906343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:48.906353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:48.906357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:48.906360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:48.906364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:48.906368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:48.906371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.906374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.906377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.906385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:48.906392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:48.906398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:48.906402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:48.906406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:48.906411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:48.906415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:48.906419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:48.906426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:48.906431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:48.906435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:48.906440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:48.906444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.906449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.906454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.906458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.906462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.906467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.906493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:48.915903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:48.917134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:48.917156Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:48.917160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:48.917165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:48.917174Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 785 [2:7559341762690022504:7075]
2025-10-09T21:46:48.917191Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:48.917216Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:48.917220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:48.917224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:48.917226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:48.917248Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:48.917261Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:48.917270Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:48.917349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.917358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.917361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.917364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:48.917975Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 785
2025-10-09T21:46:48.918011Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 785
2025-10-09T21:46:48.917445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:48.917453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.917470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:48.917482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:48.917501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:48.917523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341762690022499:7075]
2025-10-09T21:46:48.917526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341762690022499:7075]
2025-10-09T21:46:48.917529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341762690022499:7075]
2025-10-09T21:46:48.917539Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:48.917547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:48.917552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:48.917555Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:48.917558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341763051840552:7785], now have 1 active actors on pipe
2025-10-09T21:46:48.917563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341763051840449:2432], now have 1 active actors on pipe
2025-10-09T21:46:48.917570Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:48.917588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:48.917592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:48.917637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341762690022499:7075]
2025-10-09T21:46:48.917645Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:48.918084Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:48.920139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:48.920156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:48.920197Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|91d9a737-6205dff1-a1bcb6d-ed73e321_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:48.920210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:48.920216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:48.920219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:48.920232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:48.920252Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:48.920274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:48.920917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:48.920927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:48.920946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:48.921135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 786 partition: 0 MaxSeqNo: 25 sessionId: src_id|91d9a737-6205dff1-a1bcb6d-ed73e321_0
2025-10-09T21:46:48.921513Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b387d73-7f580aa7-4517bd9d-fa636662_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|91d9a737-6205dff1-a1bcb6d-ed73e321_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:48.924147Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91d9a737-6205dff1-a1bcb6d-ed73e321_0] Write session: try to update token
2025-10-09T21:46:48.924171Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91d9a737-6205dff1-a1bcb6d-ed73e321_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:48.940901Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:48.940921Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:48.940924Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:48.940926Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:48.940937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341762690022499:7075]
2025-10-09T21:46:48.940941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341762690022499:7075]
2025-10-09T21:46:48.940945Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341762690022499:7075]
2025-10-09T21:46:48.940990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:48.940998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:48.941000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:48.941135Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:48.944930Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:48.944952Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:48.944958Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:48.953231Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 786 sessionId: src_id|91d9a737-6205dff1-a1bcb6d-ed73e321_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:48.955855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:48.979498Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341763051840567:7236]
2025-10-09T21:46:48.980635Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 785, pipe: [1:7559341763051840567:7236]
2025-10-09T21:46:49.229815Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91d9a737-6205dff1-a1bcb6d-ed73e321_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:49.229832Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91d9a737-6205dff1-a1bcb6d-ed73e321_0] Write session will restart in 0.008118s
2025-10-09T21:46:49.229858Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91d9a737-6205dff1-a1bcb6d-ed73e321_0] Write session: Do CDS request
2025-10-09T21:46:49.229865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91d9a737-6205dff1-a1bcb6d-ed73e321_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:49.228151Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:49.228514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:49.228526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:49.228576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341763051840567:7236], now have 1 active actors on pipe
2025-10-09T21:46:49.228808Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:49.229029Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:49.229153Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 786 reason: status is not ok: tablet will be restarted right now sessionId: src_id|91d9a737-6205dff1-a1bcb6d-ed73e321_0
2025-10-09T21:46:49.229356Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 786 sessionId: src_id|91d9a737-6205dff1-a1bcb6d-ed73e321_0 is DEAD
2025-10-09T21:46:49.229524Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341763051840567:7236]
2025-10-09T21:46:49.229540Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:49.239641Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91d9a737-6205dff1-a1bcb6d-ed73e321_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:49.239837Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:49.239858Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 787
2025-10-09T21:46:49.240417Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 787 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:49.240468Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 787 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:49.240474Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=787 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:49.240477Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:49.240993Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 787 sessionId:  describe result for acl check
2025-10-09T21:46:49.241037Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:49.241045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:49.241047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:49.241052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341767346807891:7790] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:49.241666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341767346807891:7790] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:49.255934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341767346807891:7790] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:49.265927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:49.268493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:49.268780Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:49.268784Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:49.268797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:49.268858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:49.268899Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341766984989924:7077]
2025-10-09T21:46:49.269546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:49.273969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:49.274213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341767346807891:7790] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:49.274233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341767346807891:7790] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:49.274237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341767346807891:7790] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:49.274247Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 787 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:49.274071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:49.274431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:49.274484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:49.274485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:49.274487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:49.274488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:49.274489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:49.274490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:49.274491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.274493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.274494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.274500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:49.274510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:49.274514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:49.274518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:49.274521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:49.274525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:49.274529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.274532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.274535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.274543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:49.274551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:49.274557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:49.274563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:49.274567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:49.274571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:49.274576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:49.274581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:49.274585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:49.274590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:49.274594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:49.274600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:49.274605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.274610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.274617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.274622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.274626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.274631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.274662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:49.294200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:49.295477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:49.295484Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:49.295487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:49.295492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:49.295502Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 786 [2:7559341766984989924:7077]
2025-10-09T21:46:49.295519Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:49.295540Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:49.295544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:49.295547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:49.295550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:49.295572Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:49.295578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:49.295586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:49.295656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.295658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.295661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.295665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:49.295750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:49.295753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.295768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.295782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.295803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:49.295825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341766984989918:7077]
2025-10-09T21:46:49.295828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341766984989918:7077]
2025-10-09T21:46:49.295830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341766984989918:7077]
2025-10-09T21:46:49.295846Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:49.295862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:49.295872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:49.295877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:49.295876Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:49.295881Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:49.295882Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:49.295893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341767346807995:7790], now have 1 active actors on pipe
2025-10-09T21:46:49.295897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341767346807876:2432], now have 1 active actors on pipe
2025-10-09T21:46:49.295938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341766984989918:7077]
2025-10-09T21:46:49.295950Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:49.295963Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:49.299479Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 786
2025-10-09T21:46:49.299515Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 786
2025-10-09T21:46:49.299639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:49.299658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:49.299696Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|adb0e320-7e014703-5b80ccbf-f09a0d8f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:49.299709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:49.299716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:49.299719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:49.299733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:49.299751Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:49.299772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:49.303563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:49.303587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:49.303622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:49.304619Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 787 partition: 0 MaxSeqNo: 25 sessionId: src_id|adb0e320-7e014703-5b80ccbf-f09a0d8f_0
2025-10-09T21:46:49.305026Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91d9a737-6205dff1-a1bcb6d-ed73e321_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|adb0e320-7e014703-5b80ccbf-f09a0d8f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:49.307736Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|adb0e320-7e014703-5b80ccbf-f09a0d8f_0] Write session: try to update token
2025-10-09T21:46:49.307758Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|adb0e320-7e014703-5b80ccbf-f09a0d8f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:49.327827Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 787 sessionId: src_id|adb0e320-7e014703-5b80ccbf-f09a0d8f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:49.329656Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341767346808015:7236]
2025-10-09T21:46:49.329886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341767346808015:7236], now have 1 active actors on pipe
2025-10-09T21:46:49.329934Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 786, pipe: [1:7559341767346808015:7236]
2025-10-09T21:46:49.330325Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:49.330938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:49.330961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:49.331014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:49.331224Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:49.336296Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:49.336317Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:49.336320Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:49.336322Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:49.336333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341766984989918:7077]
2025-10-09T21:46:49.336338Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341766984989918:7077]
2025-10-09T21:46:49.336342Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341766984989918:7077]
2025-10-09T21:46:49.336387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:49.336395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.336398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:49.336538Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:49.338595Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:49.338620Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:49.338625Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:49.499384Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:49.507946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:49.507978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:49.508160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:49.508219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:49.508277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:49.508335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:49.508377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:49.508383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:49.508448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:49.508501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:49.508560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:49.508664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:49.508702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:49.508707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:49.508774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:49.508861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:49.508942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:49.509024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:49.509062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:49.509065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:49.509169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:49.509510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:49.509608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:49.509665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:49.509704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:49.509707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:49.509778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:49.509842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:49.509896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:49.509990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:49.510261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:49.510273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:49.510338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:49.510413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:49.510471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:49.510545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:49.510603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:49.510606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:49.510675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:49.510729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:49.510788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:49.510845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:49.510885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:49.510891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:49.510956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:49.511017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:49.511075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:49.511153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:49.511257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:49.511259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:49.511322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:49.511377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:49.511432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:49.511492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:49.511540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:49.511543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:49.511609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:49.511663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:49.511719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:49.511775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:49.511899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:49.511902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:49.513870Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:49.513922Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341767346808015:7236]
2025-10-09T21:46:49.513945Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:49.513948Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:49.513990Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 787 reason: error at write: Disconnected sessionId: src_id|adb0e320-7e014703-5b80ccbf-f09a0d8f_0
2025-10-09T21:46:49.513648Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:49.514198Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 787 sessionId: src_id|adb0e320-7e014703-5b80ccbf-f09a0d8f_0 is DEAD
2025-10-09T21:46:49.514384Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adb0e320-7e014703-5b80ccbf-f09a0d8f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:49.514395Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adb0e320-7e014703-5b80ccbf-f09a0d8f_0] Write session will restart in 0.005612s
2025-10-09T21:46:49.514419Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adb0e320-7e014703-5b80ccbf-f09a0d8f_0] Write session: Do CDS request
2025-10-09T21:46:49.514425Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adb0e320-7e014703-5b80ccbf-f09a0d8f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:49.522768Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|adb0e320-7e014703-5b80ccbf-f09a0d8f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:49.522889Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:49.522909Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 788
2025-10-09T21:46:49.523269Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 788 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:49.523310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 788 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:49.523316Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=788 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:49.523320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:49.524106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 788 sessionId:  describe result for acl check
2025-10-09T21:46:49.524155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:49.524175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:49.524178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:49.524186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341767346808027:7801] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:49.524794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341767346808027:7801] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:49.531974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341767346808027:7801] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:49.539538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341767346808027:7801] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:49.539561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341767346808027:7801] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:49.539564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341767346808027:7801] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:49.539575Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 788 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:49.553495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:49.557810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:49.558196Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:49.558202Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:49.558219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:49.558310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:49.558354Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341766984990061:7091]
2025-10-09T21:46:49.558950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:49.559504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:49.559569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:49.559908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:49.559965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:49.559971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:49.559972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:49.559974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:49.559975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:49.559976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:49.559977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.559978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.559980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.559997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:49.560008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:49.560012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:49.560017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:49.560020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:49.560024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:49.560028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.560031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.560035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.560043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:49.560052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:49.560065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:49.560071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:49.560075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:49.560080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:49.560084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:49.560090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:49.560094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:49.560098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:49.560104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:49.560108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:49.560113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.560119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.560123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.560129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.560133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.560139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.560168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:49.569089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:49.570388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:49.570394Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:49.570397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:49.570401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:49.570409Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 787 [2:7559341766984990061:7091]
2025-10-09T21:46:49.570425Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:49.570446Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:49.570450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:49.570454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:49.570456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:49.570475Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:49.570482Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:49.570491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:49.570562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.570564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.570565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.570568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:49.570652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:49.570654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.570669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.570680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.570694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:49.570717Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341766984990054:7091]
2025-10-09T21:46:49.570720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341766984990054:7091]
2025-10-09T21:46:49.570722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341766984990054:7091]
2025-10-09T21:46:49.570730Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:49.570746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:49.570760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:49.570763Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:49.570791Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:49.570794Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:49.570797Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:49.570808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341766984990054:7091]
2025-10-09T21:46:49.570811Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:49.570829Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:49.570834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341767346808066:7801], now have 1 active actors on pipe
2025-10-09T21:46:49.570837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341767346808020:2432], now have 1 active actors on pipe
2025-10-09T21:46:49.571003Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 787
2025-10-09T21:46:49.571040Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 787
2025-10-09T21:46:49.571367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:49.571382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:49.571416Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|24c30ed6-540c0f39-50946afb-2321e31b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:49.571426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:49.571430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:49.571433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:49.571445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:49.571457Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:49.571475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:49.571719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:49.571736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:49.571760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:49.571871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 788 partition: 0 MaxSeqNo: 25 sessionId: src_id|24c30ed6-540c0f39-50946afb-2321e31b_0
2025-10-09T21:46:49.572263Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adb0e320-7e014703-5b80ccbf-f09a0d8f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|24c30ed6-540c0f39-50946afb-2321e31b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:49.574830Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|24c30ed6-540c0f39-50946afb-2321e31b_0] Write session: try to update token
2025-10-09T21:46:49.574849Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|24c30ed6-540c0f39-50946afb-2321e31b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:49.590448Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:49.590471Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:49.590474Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:49.590476Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:49.590486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341766984990054:7091]
2025-10-09T21:46:49.590490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341766984990054:7091]
2025-10-09T21:46:49.590493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341766984990054:7091]
2025-10-09T21:46:49.590537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:49.590550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.590552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:49.590689Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:49.593266Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:49.593286Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:49.593291Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:49.595509Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 788 sessionId: src_id|24c30ed6-540c0f39-50946afb-2321e31b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:49.598119Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:49.613872Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341767346808142:7236]
2025-10-09T21:46:49.614259Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 787, pipe: [1:7559341767346808142:7236]
2025-10-09T21:46:49.792496Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24c30ed6-540c0f39-50946afb-2321e31b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:49.792511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24c30ed6-540c0f39-50946afb-2321e31b_0] Write session will restart in 0.005478s
2025-10-09T21:46:49.792536Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24c30ed6-540c0f39-50946afb-2321e31b_0] Write session: Do CDS request
2025-10-09T21:46:49.792022Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341767346808142:7236]
2025-10-09T21:46:49.792062Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:49.792065Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:49.792124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 788 reason: error at write: Disconnected sessionId: src_id|24c30ed6-540c0f39-50946afb-2321e31b_0
2025-10-09T21:46:49.792317Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:49.792333Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 788 sessionId: src_id|24c30ed6-540c0f39-50946afb-2321e31b_0 is DEAD
2025-10-09T21:46:49.792543Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24c30ed6-540c0f39-50946afb-2321e31b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:49.791210Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:49.791535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:49.791547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:49.791599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341767346808142:7236], now have 1 active actors on pipe
2025-10-09T21:46:49.794290Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:49.799680Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|24c30ed6-540c0f39-50946afb-2321e31b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:49.799789Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:49.799807Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 789
2025-10-09T21:46:49.800079Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 789 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:49.800130Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 789 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:49.800137Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=789 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:49.800141Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:49.800664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 789 sessionId:  describe result for acl check
2025-10-09T21:46:49.800702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:49.800704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:49.800706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:49.800712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341767346808160:7806] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:49.801333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341767346808160:7806] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:49.805414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341767346808160:7806] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:49.809800Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341767346808160:7806] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:49.809816Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341767346808160:7806] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:49.809819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341767346808160:7806] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:49.809829Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 789 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:49.819986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:49.826269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:49.826529Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:49.826545Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:49.826557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:49.826610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:49.826657Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341766984990178:7095]
2025-10-09T21:46:49.827234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:49.834210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:49.834309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:49.834701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:49.834746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:49.834747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:49.834749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:49.834750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:49.834752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:49.834753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:49.834754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.834756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.834757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.834764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:49.834775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:49.834779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:49.834783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:49.834787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:49.834791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:49.834796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.834800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.834803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.834811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:49.834821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:49.834827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:49.834833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:49.834838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:49.834844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:49.834848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:49.834853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:49.834858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:49.834864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:49.834868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:49.834874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:49.834879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.834885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.834890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.834895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.834901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.834906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.834939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:49.846949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:49.848074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:49.848081Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:49.848085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:49.848090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:49.848100Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 788 [2:7559341766984990178:7095]
2025-10-09T21:46:49.848119Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:49.848143Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:49.848146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:49.848150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:49.848153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:49.848172Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:49.848179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:49.848187Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:49.848255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.848257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.848259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.848262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:49.848345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:49.848349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.848362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:49.848374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:49.848389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:49.848411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341766984990172:7095]
2025-10-09T21:46:49.848414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341766984990172:7095]
2025-10-09T21:46:49.848416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341766984990172:7095]
2025-10-09T21:46:49.848432Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:49.848452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:49.848468Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:49.848474Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:49.848521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341766984990172:7095]
2025-10-09T21:46:49.848524Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:49.850646Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 788
2025-10-09T21:46:49.850701Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 788
2025-10-09T21:46:49.850272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:49.850288Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:49.850291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:49.850298Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:49.850588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341767346808147:2432], now have 1 active actors on pipe
2025-10-09T21:46:49.850595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341767346808227:7806], now have 1 active actors on pipe
2025-10-09T21:46:49.851049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:49.851072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:49.851126Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|492e1305-e5c258c4-1d91897-a10722a2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:49.851139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:49.851145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:49.851148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:49.851162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:49.851181Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:49.851212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:49.851962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:49.851977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:49.852001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:49.852172Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 789 partition: 0 MaxSeqNo: 25 sessionId: src_id|492e1305-e5c258c4-1d91897-a10722a2_0
2025-10-09T21:46:49.852640Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24c30ed6-540c0f39-50946afb-2321e31b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|492e1305-e5c258c4-1d91897-a10722a2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:49.855311Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|492e1305-e5c258c4-1d91897-a10722a2_0] Write session: try to update token
2025-10-09T21:46:49.855331Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|492e1305-e5c258c4-1d91897-a10722a2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:49.878904Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 789 sessionId: src_id|492e1305-e5c258c4-1d91897-a10722a2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:49.881930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:49.881969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:49.882027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:49.881537Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:49.882172Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:49.888243Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:49.888268Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:49.888271Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:49.888273Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:49.888282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341766984990172:7095]
2025-10-09T21:46:49.888287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341766984990172:7095]
2025-10-09T21:46:49.888291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341766984990172:7095]
2025-10-09T21:46:49.888336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:49.888344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:49.888346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:49.888508Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:49.890114Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:49.890130Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:49.890135Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:49.892331Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341767346808269:7236]
2025-10-09T21:46:49.892753Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 788, pipe: [1:7559341767346808269:7236]
2025-10-09T21:46:50.058008Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:50.058352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341767346808269:7236], now have 1 active actors on pipe
2025-10-09T21:46:50.067012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:50.067048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:50.067235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:50.067306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:50.067374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:50.067451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:50.067494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:50.067500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:50.067574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:50.067634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:50.067696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:50.067759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:50.067801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:50.067806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:50.067880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:50.067936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:50.067995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:50.068058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:50.068098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:50.068101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:50.068224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:50.068284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:50.068346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:50.068434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:50.068475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:50.068478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:50.068545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:50.068645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:50.068713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:50.068776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:50.068816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:50.068820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:50.068883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:50.068938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:50.068995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:50.069055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:50.069096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:50.069099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:50.069167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:50.069222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:50.069283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:50.069349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:50.069394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:50.069396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:50.069466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:50.069570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:50.069627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:50.069690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:50.069755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:50.069758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:50.069828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:50.069894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:50.069982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:50.070380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:50.070436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:50.070445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:50.070509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:50.070571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:50.070642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:50.070715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:50.070753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:50.070763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:50.072424Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:50.074134Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|492e1305-e5c258c4-1d91897-a10722a2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:50.074148Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|492e1305-e5c258c4-1d91897-a10722a2_0] Write session will restart in 0.005155s
2025-10-09T21:46:50.074174Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|492e1305-e5c258c4-1d91897-a10722a2_0] Write session: Do CDS request
2025-10-09T21:46:50.074181Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|492e1305-e5c258c4-1d91897-a10722a2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:50.072476Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341767346808269:7236]
2025-10-09T21:46:50.072516Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:50.072557Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:50.072561Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:50.072604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 789 reason: error at write: Disconnected sessionId: src_id|492e1305-e5c258c4-1d91897-a10722a2_0
2025-10-09T21:46:50.072848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 789 sessionId: src_id|492e1305-e5c258c4-1d91897-a10722a2_0 is DEAD
2025-10-09T21:46:50.080640Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|492e1305-e5c258c4-1d91897-a10722a2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:50.082177Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:50.082195Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 790
2025-10-09T21:46:50.082588Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 790 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:50.082627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 790 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:50.082634Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=790 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:50.082639Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:50.086486Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 790 sessionId:  describe result for acl check
2025-10-09T21:46:50.086540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:50.086544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:50.086546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:50.086556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341771641775590:7810] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:50.087186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341771641775590:7810] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:50.096660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:50.093995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341771641775590:7810] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:50.097476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:50.097705Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:50.097709Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:50.097722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:50.097774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:50.097812Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341771279957586:7097]
2025-10-09T21:46:50.098393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:50.099034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:50.099085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:50.099420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:50.099470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:50.099471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:50.099472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:50.099473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:50.099474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:50.099476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:50.099477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.099478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.099479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.099485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:50.099496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:50.099500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:50.099504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:50.099508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:50.099512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:50.099515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.099519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.099522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.099530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:50.099538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:50.099545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:50.099563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:50.099568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:50.099573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:50.099577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:50.099582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:50.099586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:50.099592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:50.099596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:50.099601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:50.099605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.099609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.099613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.099618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.099622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.099627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.099655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:50.100056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341771641775590:7810] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:50.100073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341771641775590:7810] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:50.100078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341771641775590:7810] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:50.100088Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 790 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:50.110006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:50.111245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:50.111268Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:50.111272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:50.111277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:50.111286Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 789 [2:7559341771279957586:7097]
2025-10-09T21:46:50.111304Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:50.111328Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:50.111331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:50.111335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:50.111338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:50.111360Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:50.111366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:50.111375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:50.112401Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 789
2025-10-09T21:46:50.112462Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 789
2025-10-09T21:46:50.111450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.111461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.111464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.111468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:50.111553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:50.111557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.111573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.111587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.111606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:50.111628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341771279957581:7097]
2025-10-09T21:46:50.111633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341771279957581:7097]
2025-10-09T21:46:50.111635Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341771279957581:7097]
2025-10-09T21:46:50.111651Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:50.111667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:50.111685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:50.111689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:50.111735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341771279957581:7097]
2025-10-09T21:46:50.111738Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:50.111979Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:50.111981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:50.111984Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:50.111991Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:50.112855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341771641775572:2432], now have 1 active actors on pipe
2025-10-09T21:46:50.112871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341771641775689:7810], now have 1 active actors on pipe
2025-10-09T21:46:50.112903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:50.112914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:50.112961Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c3358479-99ee9070-5d8ee1b1-a3670013_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:50.112971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:50.112976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:50.112980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:50.112993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:50.113010Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:50.113028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:50.113397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:50.113412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:50.113435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:50.113748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 790 partition: 0 MaxSeqNo: 25 sessionId: src_id|c3358479-99ee9070-5d8ee1b1-a3670013_0
2025-10-09T21:46:50.114657Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|492e1305-e5c258c4-1d91897-a10722a2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c3358479-99ee9070-5d8ee1b1-a3670013_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:50.117621Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3358479-99ee9070-5d8ee1b1-a3670013_0] Write session: try to update token
2025-10-09T21:46:50.117651Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3358479-99ee9070-5d8ee1b1-a3670013_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:50.133098Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:50.133120Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:50.133123Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:50.133125Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:50.133134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341771279957581:7097]
2025-10-09T21:46:50.133142Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341771279957581:7097]
2025-10-09T21:46:50.133150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341771279957581:7097]
2025-10-09T21:46:50.133198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:50.133198Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:50.133204Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:50.133209Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:50.133210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.133212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:50.133353Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:50.144112Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 790 sessionId: src_id|c3358479-99ee9070-5d8ee1b1-a3670013_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:50.146726Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:50.171912Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341771641775702:7236]
2025-10-09T21:46:50.172277Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 789, pipe: [1:7559341771641775702:7236]
2025-10-09T21:46:50.305259Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:50.305805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:50.305825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:50.305875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341771641775702:7236], now have 1 active actors on pipe
2025-10-09T21:46:50.306148Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:50.306168Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341771641775702:7236]
2025-10-09T21:46:50.306196Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:50.306199Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:50.306287Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 790 reason: error at write: Disconnected sessionId: src_id|c3358479-99ee9070-5d8ee1b1-a3670013_0
2025-10-09T21:46:50.306499Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 790 sessionId: src_id|c3358479-99ee9070-5d8ee1b1-a3670013_0 is DEAD
2025-10-09T21:46:50.306746Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3358479-99ee9070-5d8ee1b1-a3670013_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:50.306765Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3358479-99ee9070-5d8ee1b1-a3670013_0] Write session will restart in 0.006239s
2025-10-09T21:46:50.306793Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3358479-99ee9070-5d8ee1b1-a3670013_0] Write session: Do CDS request
2025-10-09T21:46:50.306800Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3358479-99ee9070-5d8ee1b1-a3670013_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:50.306532Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:50.318456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3358479-99ee9070-5d8ee1b1-a3670013_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:50.318608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:50.318636Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 791
2025-10-09T21:46:50.318848Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 791 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:50.318896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 791 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:50.318901Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=791 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:50.318906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:50.319401Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 791 sessionId:  describe result for acl check
2025-10-09T21:46:50.319431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:50.319433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:50.319435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:50.319442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341771641775721:7815] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:50.320273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341771641775721:7815] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:50.325996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341771641775721:7815] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:50.326324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:50.328002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:50.328233Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:50.328243Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:50.328257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:50.328331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:50.328368Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341771279957705:7099]
2025-10-09T21:46:50.328925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:50.329607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:50.329679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:50.330077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:50.330145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:50.330153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:50.330154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:50.330155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:50.330156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:50.330157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:50.330158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.330159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.330160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.330166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:50.330176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:50.330180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:50.330184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:50.330189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:50.330193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:50.330196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.330200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.330204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.330211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:50.330221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:50.330227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:50.330232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:50.330237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:50.330244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:50.330261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:50.330271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:50.330277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:50.330282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:50.330286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:50.330291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:50.330296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.330301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.330305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.330309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.330321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.330326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.330355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:50.332556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341771641775721:7815] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:50.332574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341771641775721:7815] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:50.332579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341771641775721:7815] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:50.332589Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 791 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:50.340921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:50.342251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:50.342272Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:50.342292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:50.342298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:50.342306Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 790 [2:7559341771279957705:7099]
2025-10-09T21:46:50.342322Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:50.342346Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:50.342349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:50.342353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:50.342355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:50.342377Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:50.342382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:50.342391Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:50.342462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.342464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.342466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.342469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:50.342553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:50.342556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.342572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.342587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.342608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:50.342631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341771279957697:7099]
2025-10-09T21:46:50.342634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341771279957697:7099]
2025-10-09T21:46:50.342636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341771279957697:7099]
2025-10-09T21:46:50.342644Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:50.342663Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:50.342665Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:50.342666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:50.342668Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:50.342682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:50.342685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:50.342697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341771641775828:7815], now have 1 active actors on pipe
2025-10-09T21:46:50.342703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341771641775705:2432], now have 1 active actors on pipe
2025-10-09T21:46:50.342746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341771279957697:7099]
2025-10-09T21:46:50.342757Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:50.342764Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:50.342843Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 790
2025-10-09T21:46:50.342887Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 790
2025-10-09T21:46:50.343154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:50.343172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:50.343204Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cf123cf6-1012a747-a661aca9-a951a237_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:50.343216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:50.343221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:50.343223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:50.343236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:50.343253Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:50.343271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:50.343490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:50.343504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:50.343526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:50.343631Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 791 partition: 0 MaxSeqNo: 25 sessionId: src_id|cf123cf6-1012a747-a661aca9-a951a237_0
2025-10-09T21:46:50.344040Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3358479-99ee9070-5d8ee1b1-a3670013_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cf123cf6-1012a747-a661aca9-a951a237_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:50.351672Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf123cf6-1012a747-a661aca9-a951a237_0] Write session: try to update token
2025-10-09T21:46:50.351696Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf123cf6-1012a747-a661aca9-a951a237_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:50.357873Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:50.357892Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:50.357901Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341771279957697:7099]
2025-10-09T21:46:50.357958Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:50.357984Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:50.358206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:50.358383Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:50.358541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:50.358731Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:50.358963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:50.359033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:50.377917Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:50.377936Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:50.377938Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:50.377942Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:50.377966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341771279957697:7099]
2025-10-09T21:46:50.377970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341771279957697:7099]
2025-10-09T21:46:50.377974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341771279957697:7099]
2025-10-09T21:46:50.378019Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:50.378021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:50.378024Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:50.378040Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:50.378111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.378120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:50.378255Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:50.380016Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 791 sessionId: src_id|cf123cf6-1012a747-a661aca9-a951a237_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:50.388184Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:50.406507Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341771641775851:7236]
2025-10-09T21:46:50.406856Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 790, pipe: [1:7559341771641775851:7236]
2025-10-09T21:46:50.588437Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:50.588744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:50.588763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:50.588788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341771641775851:7236], now have 1 active actors on pipe
2025-10-09T21:46:50.589066Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:50.589584Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341771641775851:7236]
2025-10-09T21:46:50.589618Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:50.589810Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:50.589935Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 791 reason: status is not ok: tablet will be restarted right now sessionId: src_id|cf123cf6-1012a747-a661aca9-a951a237_0
2025-10-09T21:46:50.590162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 791 sessionId: src_id|cf123cf6-1012a747-a661aca9-a951a237_0 is DEAD
2025-10-09T21:46:50.594126Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf123cf6-1012a747-a661aca9-a951a237_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:50.594142Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf123cf6-1012a747-a661aca9-a951a237_0] Write session will restart in 0.007684s
2025-10-09T21:46:50.594171Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf123cf6-1012a747-a661aca9-a951a237_0] Write session: Do CDS request
2025-10-09T21:46:50.594178Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf123cf6-1012a747-a661aca9-a951a237_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:50.606677Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf123cf6-1012a747-a661aca9-a951a237_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:50.606932Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:50.606949Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 792
2025-10-09T21:46:50.607144Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 792 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:50.607189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 792 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:50.607195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=792 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:50.607202Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:50.608243Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 792 sessionId:  describe result for acl check
2025-10-09T21:46:50.608283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:50.608287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:50.608288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:50.608295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341771641775933:7826] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:50.608866Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341771641775933:7826] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:50.612734Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341771641775933:7826] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:50.615127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:50.617178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:50.617434Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:50.617449Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:50.617465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:50.617538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:50.617600Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341771279957859:7115]
2025-10-09T21:46:50.617790Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341771641775933:7826] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:50.617808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341771641775933:7826] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:50.617813Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341771641775933:7826] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:50.617822Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 792 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:50.618232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:50.618627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:50.618686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:50.619044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:50.619102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:50.619103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:50.619104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:50.619105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:50.619107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:50.619108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:50.619110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.619111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.619112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.619119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:50.619130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:50.619135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:50.619139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:50.619144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:50.619148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:50.619152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.619156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.619160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.619167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:50.619176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:50.619183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:50.619189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:50.619194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:50.619200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:50.619204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:50.619210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:50.619215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:50.619221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:50.619226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:50.619232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:50.619238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.619244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.619249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.619254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.619261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.619265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.619296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:50.626877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:50.628570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:50.628587Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:50.628590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:50.628595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:50.628602Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 791 [2:7559341771279957859:7115]
2025-10-09T21:46:50.628619Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:50.628640Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:50.628648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:50.628651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:50.628653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:50.628765Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:50.628777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:50.628785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:50.628855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.628857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.628859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.628862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:50.628936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:50.628939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.628952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.628965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.628983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:50.629017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341771279957854:7115]
2025-10-09T21:46:50.629020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341771279957854:7115]
2025-10-09T21:46:50.629022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341771279957854:7115]
2025-10-09T21:46:50.629037Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:50.629064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:50.629079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:50.629085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:50.629101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:50.629105Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:50.629107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:50.629127Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341771279957854:7115]
2025-10-09T21:46:50.629131Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:50.629226Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:50.629384Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 791
2025-10-09T21:46:50.629394Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 791
2025-10-09T21:46:50.629449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341771641775968:7826], now have 1 active actors on pipe
2025-10-09T21:46:50.629461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341771641775856:2432], now have 1 active actors on pipe
2025-10-09T21:46:50.629615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:50.629624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:50.629660Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ff098357-b0b219cf-dbc9eacf-9026ba86_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:50.629670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:50.629675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:50.629678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:50.629690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:50.629703Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:50.629724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:50.630320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:50.630332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:50.630355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:50.630585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 792 partition: 0 MaxSeqNo: 25 sessionId: src_id|ff098357-b0b219cf-dbc9eacf-9026ba86_0
2025-10-09T21:46:50.630997Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf123cf6-1012a747-a661aca9-a951a237_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ff098357-b0b219cf-dbc9eacf-9026ba86_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:50.633765Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff098357-b0b219cf-dbc9eacf-9026ba86_0] Write session: try to update token
2025-10-09T21:46:50.633786Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff098357-b0b219cf-dbc9eacf-9026ba86_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:50.651531Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:50.651553Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:50.651556Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:50.651558Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:50.651569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341771279957854:7115]
2025-10-09T21:46:50.651575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341771279957854:7115]
2025-10-09T21:46:50.651580Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341771279957854:7115]
2025-10-09T21:46:50.651634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:50.651640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.651643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:50.651787Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:50.652423Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:50.652430Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:50.652433Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:50.664409Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 792 sessionId: src_id|ff098357-b0b219cf-dbc9eacf-9026ba86_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:50.667435Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:50.689526Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341771641775980:7236]
2025-10-09T21:46:50.690612Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 791, pipe: [1:7559341771641775980:7236]
2025-10-09T21:46:50.827984Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:50.828887Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:50.828922Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:50.828927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:50.828954Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341771641775980:7236]
2025-10-09T21:46:50.828998Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 792 reason: error at write: Disconnected sessionId: src_id|ff098357-b0b219cf-dbc9eacf-9026ba86_0
2025-10-09T21:46:50.828282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:50.828302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:50.828348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341771641775980:7236], now have 1 active actors on pipe
2025-10-09T21:46:50.828799Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:50.829387Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff098357-b0b219cf-dbc9eacf-9026ba86_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:50.829397Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff098357-b0b219cf-dbc9eacf-9026ba86_0] Write session will restart in 0.008983s
2025-10-09T21:46:50.829435Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff098357-b0b219cf-dbc9eacf-9026ba86_0] Write session: Do CDS request
2025-10-09T21:46:50.829445Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff098357-b0b219cf-dbc9eacf-9026ba86_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:50.829210Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 792 sessionId: src_id|ff098357-b0b219cf-dbc9eacf-9026ba86_0 is DEAD
2025-10-09T21:46:50.840061Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff098357-b0b219cf-dbc9eacf-9026ba86_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:50.840548Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:50.840566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 793
2025-10-09T21:46:50.840716Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 793 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:50.840762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 793 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:50.840768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=793 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:50.840773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:50.841638Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 793 sessionId:  describe result for acl check
2025-10-09T21:46:50.841686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:50.841688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:50.841689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:50.841696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341771641776041:7831] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:50.842425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341771641776041:7831] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:50.848404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:50.849019Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341771641776041:7831] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:50.850964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:50.851206Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:50.851218Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:50.851229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:50.851300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:50.851348Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341771279957975:7117]
2025-10-09T21:46:50.851950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:50.852343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:50.852391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:50.852696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:50.852752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:50.852761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:50.852762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:50.852764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:50.852765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:50.852766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:50.852767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.852768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.852770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.852775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:50.852785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:50.852789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:50.852793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:50.852797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:50.852800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:50.852804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.852808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.852811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.852818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:50.852826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:50.852832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:50.852838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:50.852842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:50.852847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:50.852852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:50.852864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:50.852869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:50.852873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:50.852878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:50.852882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:50.852887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.852892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.852896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.852900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.852905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.852910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.852939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:50.853199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341771641776041:7831] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:50.853219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341771641776041:7831] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:50.853224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341771641776041:7831] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:50.853234Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 793 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:50.864860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:50.866009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:50.866049Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:50.866052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:50.866058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:50.866066Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 792 [2:7559341771279957975:7117]
2025-10-09T21:46:50.866082Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:50.866106Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:50.866116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:50.866119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:50.866122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:50.866143Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:50.866153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:50.866161Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:50.866233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.866236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.866240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.866244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:50.866320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:50.866328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.866340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:50.866350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:50.866367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:50.866389Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341771279957970:7117]
2025-10-09T21:46:50.866391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341771279957970:7117]
2025-10-09T21:46:50.866394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341771279957970:7117]
2025-10-09T21:46:50.866402Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:50.866418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:50.866424Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:50.866427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:50.866429Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:50.866439Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:50.866443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:50.866504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341771641776105:7831], now have 1 active actors on pipe
2025-10-09T21:46:50.866515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341771641775988:2432], now have 1 active actors on pipe
2025-10-09T21:46:50.866518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341771279957970:7117]
2025-10-09T21:46:50.866526Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:50.866876Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:50.869271Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 792
2025-10-09T21:46:50.869272Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 792
2025-10-09T21:46:50.869620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:50.869658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:50.869694Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|88486117-6974cfc2-7d2333ca-5209efff_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:50.869712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:50.869727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:50.869730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:50.869755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:50.869775Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:50.869802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:50.870426Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 793 partition: 0 MaxSeqNo: 25 sessionId: src_id|88486117-6974cfc2-7d2333ca-5209efff_0
2025-10-09T21:46:50.870940Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff098357-b0b219cf-dbc9eacf-9026ba86_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|88486117-6974cfc2-7d2333ca-5209efff_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:50.870174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:50.870189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:50.870221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:50.873598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88486117-6974cfc2-7d2333ca-5209efff_0] Write session: try to update token
2025-10-09T21:46:50.873619Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88486117-6974cfc2-7d2333ca-5209efff_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:50.882580Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:46:50.882608Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 8
2025-10-09T21:46:50.883004Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:46:50.883433Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 9 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:46:50.883491Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:46:50.889450Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:50.889466Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:50.889477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341771279957970:7117]
2025-10-09T21:46:50.889528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:50.889577Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:50.889710Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:50.889869Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:50.890047Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:50.890194Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:50.890396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:50.890463Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:50.900644Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 793 sessionId: src_id|88486117-6974cfc2-7d2333ca-5209efff_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:50.903227Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:50.903868Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:50.903638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:50.903674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:50.903721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:50.910124Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:50.910144Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:50.910147Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:50.910149Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:50.910160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341771279957970:7117]
2025-10-09T21:46:50.910165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341771279957970:7117]
2025-10-09T21:46:50.910169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341771279957970:7117]
2025-10-09T21:46:50.910219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:50.910226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:50.910229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:50.910230Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:50.910236Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:50.910240Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:50.910465Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:50.929443Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341771641776116:7236]
2025-10-09T21:46:50.931152Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 792, pipe: [1:7559341771641776116:7236]
2025-10-09T21:46:51.087538Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:51.098097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:51.098131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:51.098319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:51.098389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:51.098454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:51.098519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:51.099323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:51.099341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:51.099414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:51.099469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:51.099529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:51.099597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:51.099639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:51.099642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:51.099708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:51.099774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:51.099839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:51.099895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:51.099937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:51.099940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:51.100003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:51.100089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:51.100147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:51.100201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:51.100333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:51.100337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:51.100411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:51.100482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:51.100548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:51.100617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:51.100681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:51.100684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:51.100757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:51.100818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:51.100878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:51.100937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:51.100978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:51.100981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:51.101049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:51.101106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:51.101176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:51.101239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:51.101376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:51.101379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:51.101442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:51.101495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:51.101552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:51.101616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:51.101658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:51.101661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:51.101727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:51.101781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:51.101840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:51.101896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:51.102018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:51.102021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:51.102488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:51.102560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:51.102627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:51.102689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:51.102828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:51.102831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:51.102900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341771641776116:7236], now have 1 active actors on pipe
2025-10-09T21:46:51.103632Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:51.104003Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:51.104061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 793 reason: status is not ok: tablet will be restarted right now sessionId: src_id|88486117-6974cfc2-7d2333ca-5209efff_0
2025-10-09T21:46:51.104414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88486117-6974cfc2-7d2333ca-5209efff_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:51.104423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88486117-6974cfc2-7d2333ca-5209efff_0] Write session will restart in 0.006959s
2025-10-09T21:46:51.104449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88486117-6974cfc2-7d2333ca-5209efff_0] Write session: Do CDS request
2025-10-09T21:46:51.104456Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88486117-6974cfc2-7d2333ca-5209efff_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:51.104256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 793 sessionId: src_id|88486117-6974cfc2-7d2333ca-5209efff_0 is DEAD
2025-10-09T21:46:51.104393Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:51.104405Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 0
2025-10-09T21:46:51.104433Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341771641776116:7236]
2025-10-09T21:46:51.112611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88486117-6974cfc2-7d2333ca-5209efff_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:51.112993Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:51.113011Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 794
2025-10-09T21:46:51.113175Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 794 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:51.113210Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 794 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:51.113222Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=794 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:51.113229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:51.113730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 794 sessionId:  describe result for acl check
2025-10-09T21:46:51.113766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:51.113768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:51.113770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:51.113777Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341775936743436:7835] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:51.114532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341775936743436:7835] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:51.120621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341775936743436:7835] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:51.126022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:51.126156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341775936743436:7835] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:51.126176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341775936743436:7835] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:51.126181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341775936743436:7835] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:51.126192Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 794 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:51.128013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:51.128243Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:51.128255Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:51.128267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:51.128316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:51.128359Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341775574925392:7120]
2025-10-09T21:46:51.128928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:51.129342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:51.129404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:51.129763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:51.129820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:51.129827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:51.129829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:51.129830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:51.129832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:51.129833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:51.129834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.129836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.129837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.129843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:51.129853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:51.129857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:51.129861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:51.129864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:51.129868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:51.129872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.129875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.129878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.129886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:51.129894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:51.129901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:51.129908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:51.129913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:51.129917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:51.129921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:51.129926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:51.129931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:51.129936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:51.129965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:51.129970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:51.129975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.129979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.129984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.129989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.129994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.129999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.130025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:51.143363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:51.144932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:51.144941Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:51.144944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:51.144950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:51.144959Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 793 [2:7559341775574925392:7120]
2025-10-09T21:46:51.144976Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:51.145003Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:51.145008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:51.145011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:51.145014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:51.145037Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:51.145045Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:51.145054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:51.145137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.145144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.145147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.145150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:51.145266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:51.145271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.145286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.145299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.145317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:51.145346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341775574925386:7120]
2025-10-09T21:46:51.145349Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341775574925386:7120]
2025-10-09T21:46:51.145352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341775574925386:7120]
2025-10-09T21:46:51.145375Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:51.145398Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:51.145417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:51.145421Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:51.145432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:51.145437Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:51.145440Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:51.145479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341775574925386:7120]
2025-10-09T21:46:51.145485Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:51.145546Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:51.145807Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 793
2025-10-09T21:46:51.145852Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 793
2025-10-09T21:46:51.146104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341775936743534:7835], now have 1 active actors on pipe
2025-10-09T21:46:51.146120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341775936743421:2432], now have 1 active actors on pipe
2025-10-09T21:46:51.146148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:51.146155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:51.146205Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|189b64d-f23bdd42-ef7ba8d3-6ceac2b1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:51.146220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:51.146227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:51.146231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:51.146243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:51.146273Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:51.146330Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:46:51.146365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:51.146771Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 10 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:46:51.146813Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:46:51.146826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:51.146846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:51.146883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:51.147021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 794 partition: 0 MaxSeqNo: 25 sessionId: src_id|189b64d-f23bdd42-ef7ba8d3-6ceac2b1_0
2025-10-09T21:46:51.147441Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88486117-6974cfc2-7d2333ca-5209efff_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|189b64d-f23bdd42-ef7ba8d3-6ceac2b1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:51.150022Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|189b64d-f23bdd42-ef7ba8d3-6ceac2b1_0] Write session: try to update token
2025-10-09T21:46:51.150066Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|189b64d-f23bdd42-ef7ba8d3-6ceac2b1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:51.174424Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:51.174447Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:51.174450Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:51.174452Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:51.174469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341775574925386:7120]
2025-10-09T21:46:51.174479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341775574925386:7120]
2025-10-09T21:46:51.174484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341775574925386:7120]
2025-10-09T21:46:51.174535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:51.174550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.174553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:51.174678Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:51.176572Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:51.176590Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:51.176599Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:51.188692Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 794 sessionId: src_id|189b64d-f23bdd42-ef7ba8d3-6ceac2b1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:51.191429Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:51.204805Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341775936743556:7236]
2025-10-09T21:46:51.205418Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 793, pipe: [1:7559341775936743556:7236]
2025-10-09T21:46:51.346728Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:51.347156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:51.347168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:51.347197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341775936743556:7236], now have 1 active actors on pipe
2025-10-09T21:46:51.348100Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341775936743556:7236]
2025-10-09T21:46:51.348135Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:51.348138Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:51.348217Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 794 reason: error at write: Disconnected sessionId: src_id|189b64d-f23bdd42-ef7ba8d3-6ceac2b1_0
2025-10-09T21:46:51.348409Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 794 sessionId: src_id|189b64d-f23bdd42-ef7ba8d3-6ceac2b1_0 is DEAD
2025-10-09T21:46:51.348553Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:51.349482Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:51.353124Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|189b64d-f23bdd42-ef7ba8d3-6ceac2b1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:51.353139Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|189b64d-f23bdd42-ef7ba8d3-6ceac2b1_0] Write session will restart in 0.005466s
2025-10-09T21:46:51.353182Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|189b64d-f23bdd42-ef7ba8d3-6ceac2b1_0] Write session: Do CDS request
2025-10-09T21:46:51.353190Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|189b64d-f23bdd42-ef7ba8d3-6ceac2b1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:51.362406Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|189b64d-f23bdd42-ef7ba8d3-6ceac2b1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:51.362856Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:51.362888Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 795
2025-10-09T21:46:51.363075Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 795 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:51.363121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 795 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:51.363134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=795 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:51.363138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:51.363707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 795 sessionId:  describe result for acl check
2025-10-09T21:46:51.363750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:51.363752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:51.363754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:51.363770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341775936743587:7840] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:51.364359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341775936743587:7840] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:51.369630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341775936743587:7840] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:51.377598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:51.381536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341775936743587:7840] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:51.381557Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341775936743587:7840] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:51.381561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341775936743587:7840] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:51.381572Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 795 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:51.380623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:51.380928Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:51.380933Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:51.380948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:51.381016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:51.381077Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341775574925510:7123]
2025-10-09T21:46:51.381679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:51.382072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:51.382133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:51.382467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:51.382532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:51.382533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:51.382535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:51.382535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:51.382537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:51.382538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:51.382539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.382540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.382541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.382547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:51.382556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:51.382560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:51.382563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:51.382567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:51.382571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:51.382574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.382578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.382581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.382588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:51.382597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:51.382603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:51.382608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:51.382612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:51.382617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:51.382621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:51.382627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:51.382632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:51.382636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:51.382640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:51.382644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:51.382648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.382653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.382656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.382661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.382665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.382670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.382707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:51.393311Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 794
2025-10-09T21:46:51.391375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:51.393355Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 794
2025-10-09T21:46:51.392882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:51.392889Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:51.392892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:51.392898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:51.392905Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 794 [2:7559341775574925510:7123]
2025-10-09T21:46:51.392922Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:51.392939Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:51.392943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:51.392947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:51.392949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:51.392970Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:51.392976Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:51.392984Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:51.393052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.393054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.393056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.393059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:51.393149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:51.393152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.393169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.393182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.393200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:51.393222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341775936743678:7840], now have 1 active actors on pipe
2025-10-09T21:46:51.393231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341775574925504:7123]
2025-10-09T21:46:51.393233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341775574925504:7123]
2025-10-09T21:46:51.393235Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341775574925504:7123]
2025-10-09T21:46:51.393250Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:51.393261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341775936743562:2432], now have 1 active actors on pipe
2025-10-09T21:46:51.393270Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:51.393274Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:51.393277Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:51.393280Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:51.393286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:51.393290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:51.393328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341775574925504:7123]
2025-10-09T21:46:51.393339Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:51.394058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:51.394232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:51.394251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:51.394289Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|618f7efb-f64404be-c0050521-a575484f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:51.394306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:51.394311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:51.394314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:51.394325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:51.394339Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:51.394358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:51.395457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:51.395470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:51.395493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:51.395669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 795 partition: 0 MaxSeqNo: 25 sessionId: src_id|618f7efb-f64404be-c0050521-a575484f_0
2025-10-09T21:46:51.396102Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|189b64d-f23bdd42-ef7ba8d3-6ceac2b1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|618f7efb-f64404be-c0050521-a575484f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:51.398945Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|618f7efb-f64404be-c0050521-a575484f_0] Write session: try to update token
2025-10-09T21:46:51.398966Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|618f7efb-f64404be-c0050521-a575484f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:51.414291Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:51.414313Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:51.414315Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:51.414320Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:51.414329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341775574925504:7123]
2025-10-09T21:46:51.414335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341775574925504:7123]
2025-10-09T21:46:51.414344Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341775574925504:7123]
2025-10-09T21:46:51.414390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:51.414396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.414399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:51.414605Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:51.415209Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:51.415222Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:51.415226Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:51.430530Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 795 sessionId: src_id|618f7efb-f64404be-c0050521-a575484f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:51.433024Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:51.449117Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341775936743711:7236]
2025-10-09T21:46:51.449544Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 794, pipe: [1:7559341775936743711:7236]
2025-10-09T21:46:51.592704Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:51.593134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:51.593152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:51.593177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341775936743711:7236], now have 1 active actors on pipe
2025-10-09T21:46:51.593742Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:51.593402Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:51.593829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 795 reason: status is not ok: tablet will be restarted right now sessionId: src_id|618f7efb-f64404be-c0050521-a575484f_0
2025-10-09T21:46:51.593879Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341775936743711:7236]
2025-10-09T21:46:51.593895Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:51.594042Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 795 sessionId: src_id|618f7efb-f64404be-c0050521-a575484f_0 is DEAD
2025-10-09T21:46:51.598127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|618f7efb-f64404be-c0050521-a575484f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:51.598142Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|618f7efb-f64404be-c0050521-a575484f_0] Write session will restart in 0.005041s
2025-10-09T21:46:51.598172Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|618f7efb-f64404be-c0050521-a575484f_0] Write session: Do CDS request
2025-10-09T21:46:51.598179Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|618f7efb-f64404be-c0050521-a575484f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:51.606285Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|618f7efb-f64404be-c0050521-a575484f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:51.607016Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:51.607030Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 796
2025-10-09T21:46:51.607316Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 796 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:51.607345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 796 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:51.607350Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=796 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:51.607352Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:51.608170Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 796 sessionId:  describe result for acl check
2025-10-09T21:46:51.608199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:51.608200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:51.608202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:51.608206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341775936743757:7851] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:51.608919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341775936743757:7851] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:51.614415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341775936743757:7851] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:51.625724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341775936743757:7851] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:51.625746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341775936743757:7851] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:51.625751Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341775936743757:7851] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:51.625762Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 796 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:51.629265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:51.633017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:51.633552Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:51.633558Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:51.633577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:51.633665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:51.633710Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341775574925673:7139]
2025-10-09T21:46:51.634238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:51.634766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:51.634830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:51.635198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:51.635258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:51.635264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:51.635265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:51.635267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:51.635269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:51.635270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:51.635272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.635273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.635275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.635281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:51.635294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:51.635303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:51.635314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:51.635323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:51.635328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:51.635336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.635341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.635344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.635352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:51.635365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:51.635372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:51.635377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:51.635382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:51.635388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:51.635398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:51.635404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:51.635416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:51.635426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:51.635431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:51.635435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:51.635441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.635446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.635451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.635455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.635460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.635465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.635494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:51.648298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:51.649831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:51.649852Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:51.649856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:51.649860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:51.649868Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 795 [2:7559341775574925673:7139]
2025-10-09T21:46:51.649885Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:51.649908Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:51.649916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:51.649919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:51.649922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:51.649941Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:51.649964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:51.649972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:51.650078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.650085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.650088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.650090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:51.650179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:51.650182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.650200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.650210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.650226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:51.650247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341775574925667:7139]
2025-10-09T21:46:51.650249Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341775574925667:7139]
2025-10-09T21:46:51.650251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341775574925667:7139]
2025-10-09T21:46:51.650267Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:51.650286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:51.650301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:51.650304Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:51.650350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341775574925667:7139]
2025-10-09T21:46:51.650352Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:51.650354Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:51.650354Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:51.650357Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:51.650363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:51.651914Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 795
2025-10-09T21:46:51.651949Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 795
2025-10-09T21:46:51.655162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341775936743829:7851], now have 1 active actors on pipe
2025-10-09T21:46:51.655180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341775936743714:2432], now have 1 active actors on pipe
2025-10-09T21:46:51.655199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:51.655206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:51.655250Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b46feae7-603c866c-99128b3e-9aa530e2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:51.655267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:51.655280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:51.655289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:51.655305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:51.655326Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:51.655351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:51.658447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:51.658467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:51.658501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:51.662297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 796 partition: 0 MaxSeqNo: 25 sessionId: src_id|b46feae7-603c866c-99128b3e-9aa530e2_0
2025-10-09T21:46:51.664540Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|618f7efb-f64404be-c0050521-a575484f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b46feae7-603c866c-99128b3e-9aa530e2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:51.667519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b46feae7-603c866c-99128b3e-9aa530e2_0] Write session: try to update token
2025-10-09T21:46:51.667546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b46feae7-603c866c-99128b3e-9aa530e2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:51.680602Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:51.680634Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:51.680637Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:51.680639Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:51.680649Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341775574925667:7139]
2025-10-09T21:46:51.680654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341775574925667:7139]
2025-10-09T21:46:51.680662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341775574925667:7139]
2025-10-09T21:46:51.680719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:51.680731Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:51.680734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.680737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:51.680738Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:51.680743Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:51.680949Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:51.696871Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341775936743849:7236]
2025-10-09T21:46:51.701210Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 796 sessionId: src_id|b46feae7-603c866c-99128b3e-9aa530e2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:51.701239Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 795, pipe: [1:7559341775936743849:7236]
2025-10-09T21:46:51.704014Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:51.853766Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:51.854211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341775936743849:7236], now have 1 active actors on pipe
2025-10-09T21:46:51.854245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:51.854252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:51.855691Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341775936743849:7236]
2025-10-09T21:46:51.855724Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:51.856297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b46feae7-603c866c-99128b3e-9aa530e2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:51.856307Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b46feae7-603c866c-99128b3e-9aa530e2_0] Write session will restart in 0.007352s
2025-10-09T21:46:51.856326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b46feae7-603c866c-99128b3e-9aa530e2_0] Write session: Do CDS request
2025-10-09T21:46:51.856332Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b46feae7-603c866c-99128b3e-9aa530e2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:51.854962Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:51.855786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 796 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b46feae7-603c866c-99128b3e-9aa530e2_0
2025-10-09T21:46:51.855947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 796 sessionId: src_id|b46feae7-603c866c-99128b3e-9aa530e2_0 is DEAD
2025-10-09T21:46:51.856085Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:51.865606Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b46feae7-603c866c-99128b3e-9aa530e2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:51.866008Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:51.866542Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 797
2025-10-09T21:46:51.866785Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 797 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:51.866832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 797 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:51.866843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=797 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:51.866847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:51.867277Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 797 sessionId:  describe result for acl check
2025-10-09T21:46:51.867314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:51.867320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:51.867322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:51.867329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341775936743873:7864] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:51.867932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341775936743873:7864] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:51.872920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341775936743873:7864] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:51.876692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341775936743873:7864] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:51.876710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341775936743873:7864] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:51.876713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341775936743873:7864] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:51.876723Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 797 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:51.884760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:51.887864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:51.888094Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:51.888105Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:51.888118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:51.888179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:51.888222Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341775574925793:7141]
2025-10-09T21:46:51.888821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:51.889232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:51.889281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:51.889609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:51.889666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:51.889673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:51.889674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:51.889676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:51.889677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:51.889678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:51.889680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.889681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.889683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.889689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:51.889700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:51.889704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:51.889708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:51.889712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:51.889716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:51.889720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.889724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.889726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.889734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:51.889741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:51.889747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:51.889752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:51.889756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:51.889762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:51.889773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:51.889778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:51.889783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:51.889788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:51.889794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:51.889799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:51.889804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.889809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.889813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.889818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.889823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.889828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.889858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:51.897664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:51.898992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:51.898998Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:51.899000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:51.899004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:51.899011Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 796 [2:7559341775574925793:7141]
2025-10-09T21:46:51.899022Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:51.899039Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:51.899042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:51.899045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:51.899046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:51.899061Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:51.899065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:51.899071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:51.899120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.899122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.899126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.899128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:51.899190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:51.899192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.899201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:51.899207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:51.899217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:51.899233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341775574925788:7141]
2025-10-09T21:46:51.899234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341775574925788:7141]
2025-10-09T21:46:51.899236Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341775574925788:7141]
2025-10-09T21:46:51.899242Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:51.899254Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:51.899264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:51.899266Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:51.899270Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:51.899273Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:51.899276Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:51.899299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341775574925788:7141]
2025-10-09T21:46:51.899302Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:51.899312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341775936743860:2432], now have 1 active actors on pipe
2025-10-09T21:46:51.899314Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:51.899315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341775936743918:7864], now have 1 active actors on pipe
2025-10-09T21:46:51.900090Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 796
2025-10-09T21:46:51.900129Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 796
2025-10-09T21:46:51.901597Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 797 partition: 0 MaxSeqNo: 25 sessionId: src_id|23c2133f-4a53a87-c84689c2-5ff7c1d4_0
2025-10-09T21:46:51.902008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b46feae7-603c866c-99128b3e-9aa530e2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|23c2133f-4a53a87-c84689c2-5ff7c1d4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:51.901009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:51.901034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:51.901072Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|23c2133f-4a53a87-c84689c2-5ff7c1d4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:51.901083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:51.901088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:51.901091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:51.901104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:51.901120Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:51.901138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:51.901439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:51.901445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:51.901465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:51.904874Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23c2133f-4a53a87-c84689c2-5ff7c1d4_0] Write session: try to update token
2025-10-09T21:46:51.904895Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23c2133f-4a53a87-c84689c2-5ff7c1d4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:51.917695Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:51.917716Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:51.917720Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:51.917722Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:51.917733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341775574925788:7141]
2025-10-09T21:46:51.917742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341775574925788:7141]
2025-10-09T21:46:51.917746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341775574925788:7141]
2025-10-09T21:46:51.917793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:51.917800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:51.917803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:51.917805Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:51.917812Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:51.917815Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:51.918883Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:51.933782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 797 sessionId: src_id|23c2133f-4a53a87-c84689c2-5ff7c1d4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:51.936800Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:51.955710Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341775936743988:7236]
2025-10-09T21:46:51.956173Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 796, pipe: [1:7559341775936743988:7236]
2025-10-09T21:46:52.082972Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:52.083498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:52.083518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:52.083544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341775936743988:7236], now have 1 active actors on pipe
2025-10-09T21:46:52.083907Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:52.084295Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23c2133f-4a53a87-c84689c2-5ff7c1d4_0] Counters: { Errors: 0 CurrentSessionLifetimeMs: 182 BytesWritten: 56320040 MessagesWritten: 25 BytesWrittenCompressed: 0 BytesInflightUncompressed: 22528020 BytesInflightCompressed: 0 BytesInflightTotal: 18022416 MessagesInflight: 10 }
2025-10-09T21:46:52.084344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23c2133f-4a53a87-c84689c2-5ff7c1d4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:52.084350Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23c2133f-4a53a87-c84689c2-5ff7c1d4_0] Write session will restart in 0.008527s
2025-10-09T21:46:52.084372Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23c2133f-4a53a87-c84689c2-5ff7c1d4_0] Write session: Do CDS request
2025-10-09T21:46:52.084379Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23c2133f-4a53a87-c84689c2-5ff7c1d4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:52.083830Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:52.083886Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341775936743988:7236]
2025-10-09T21:46:52.083893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:52.083896Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:52.083960Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 797 reason: error at write: Disconnected sessionId: src_id|23c2133f-4a53a87-c84689c2-5ff7c1d4_0
2025-10-09T21:46:52.084154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 797 sessionId: src_id|23c2133f-4a53a87-c84689c2-5ff7c1d4_0 is DEAD
2025-10-09T21:46:52.095041Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23c2133f-4a53a87-c84689c2-5ff7c1d4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:52.098195Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:52.098213Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 798
2025-10-09T21:46:52.098488Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 798 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:52.098538Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 798 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:52.098544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=798 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:52.098549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:52.099297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 798 sessionId:  describe result for acl check
2025-10-09T21:46:52.099337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:52.099340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:52.099341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:52.099348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341780231711362:7868] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:52.099928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341780231711362:7868] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:52.102408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341780231711362:7868] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:52.104202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:52.105925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:52.106207Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:52.106218Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:52.106232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:52.106295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:52.106348Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341779869893210:7143]
2025-10-09T21:46:52.106991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:52.107500Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341780231711362:7868] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:52.107517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341780231711362:7868] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:52.107521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341780231711362:7868] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:52.107530Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 798 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:52.107549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:52.107631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:52.107990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:52.108045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:52.108052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:52.108053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:52.108054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:52.108055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:52.108057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:52.108058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.108059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.108060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.108066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:52.108078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:52.108082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:52.108087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:52.108091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:52.108095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:52.108099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.108103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.108107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.108130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:52.108143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:52.108150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:52.108156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:52.108161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:52.108166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:52.108171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:52.108177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:52.108183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:52.108188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:52.108194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:52.108200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:52.108205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.108210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.108216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.108221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.108227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.108234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.108265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:52.119067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:52.120427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:52.120450Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:52.120453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:52.120458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:52.120466Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 797 [2:7559341779869893210:7143]
2025-10-09T21:46:52.120483Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:52.120506Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:52.120510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:52.120513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:52.120516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:52.120536Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:52.120543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:52.120551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:52.120636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.120645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.120648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.120651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:52.120740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:52.120747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.120762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.120775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.120793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:52.120817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341779869893203:7143]
2025-10-09T21:46:52.120828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341779869893203:7143]
2025-10-09T21:46:52.120830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341779869893203:7143]
2025-10-09T21:46:52.120848Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:52.120870Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:52.120893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:52.120901Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:52.120910Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:52.120923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:52.120925Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:52.120949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341779869893203:7143]
2025-10-09T21:46:52.120953Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:52.121192Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 797
2025-10-09T21:46:52.121234Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 797
2025-10-09T21:46:52.121017Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:52.121113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341780231711411:7868], now have 1 active actors on pipe
2025-10-09T21:46:52.121119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341780231711291:2432], now have 1 active actors on pipe
2025-10-09T21:46:52.121260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:52.121265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:52.121297Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e92e47e0-336695c7-6e3f6049-eea281b8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:52.121318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:52.121323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:52.121326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:52.121337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:52.121351Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:52.121374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:52.121894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:52.121909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:52.121928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:52.122119Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 798 partition: 0 MaxSeqNo: 25 sessionId: src_id|e92e47e0-336695c7-6e3f6049-eea281b8_0
2025-10-09T21:46:52.122546Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23c2133f-4a53a87-c84689c2-5ff7c1d4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e92e47e0-336695c7-6e3f6049-eea281b8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:52.125153Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e92e47e0-336695c7-6e3f6049-eea281b8_0] Write session: try to update token
2025-10-09T21:46:52.125173Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e92e47e0-336695c7-6e3f6049-eea281b8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:52.139165Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:52.139189Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:52.139192Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:52.139194Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:52.139205Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341779869893203:7143]
2025-10-09T21:46:52.139209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341779869893203:7143]
2025-10-09T21:46:52.139214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341779869893203:7143]
2025-10-09T21:46:52.139264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:52.139265Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:52.139272Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:52.139275Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:52.139276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.139279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:52.139412Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:52.146156Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 798 sessionId: src_id|e92e47e0-336695c7-6e3f6049-eea281b8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:52.148871Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:52.184076Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341780231711427:7236]
2025-10-09T21:46:52.184475Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 797, pipe: [1:7559341780231711427:7236]
2025-10-09T21:46:52.330041Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:52.330341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:52.330352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:52.330402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341780231711427:7236], now have 1 active actors on pipe
2025-10-09T21:46:52.333521Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:52.334778Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e92e47e0-336695c7-6e3f6049-eea281b8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:52.334791Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e92e47e0-336695c7-6e3f6049-eea281b8_0] Write session will restart in 0.007351s
2025-10-09T21:46:52.334816Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e92e47e0-336695c7-6e3f6049-eea281b8_0] Write session: Do CDS request
2025-10-09T21:46:52.334823Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e92e47e0-336695c7-6e3f6049-eea281b8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:52.333976Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341780231711427:7236]
2025-10-09T21:46:52.334009Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:52.334111Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 798 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e92e47e0-336695c7-6e3f6049-eea281b8_0
2025-10-09T21:46:52.334306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 798 sessionId: src_id|e92e47e0-336695c7-6e3f6049-eea281b8_0 is DEAD
2025-10-09T21:46:52.334455Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:52.343871Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e92e47e0-336695c7-6e3f6049-eea281b8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:52.346396Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:52.346412Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 799
2025-10-09T21:46:52.346650Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 799 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:52.346686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 799 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:52.346696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=799 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:52.346700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:52.347547Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 799 sessionId:  describe result for acl check
2025-10-09T21:46:52.347590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:52.347592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:52.347593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:52.347602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341780231711442:7873] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:52.348148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341780231711442:7873] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:52.355825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341780231711442:7873] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:52.361712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341780231711442:7873] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:52.361730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341780231711442:7873] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:52.361735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341780231711442:7873] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:52.361745Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 799 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:52.367264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:52.369731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:52.370008Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:52.370020Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:52.370047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:52.370115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:52.370162Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341779869893333:7145]
2025-10-09T21:46:52.370836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:52.371253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:52.371314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:52.371694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:52.371750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:52.371756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:52.371758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:52.371759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:52.371761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:52.371762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:52.371763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.371765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.371766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.371771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:52.371783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:52.371787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:52.371791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:52.371795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:52.371799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:52.371802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.371805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.371808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.371816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:52.371825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:52.371831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:52.371836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:52.371841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:52.371846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:52.371851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:52.371856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:52.371861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:52.371866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:52.371871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:52.371876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:52.371882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.371895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.371900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.371905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.371910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.371916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.371947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:52.380357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:52.381471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:52.381488Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:52.381491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:52.381496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:52.381504Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 798 [2:7559341779869893333:7145]
2025-10-09T21:46:52.381521Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:52.381542Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:52.381547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:52.381550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:52.381552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:52.381588Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:52.381593Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:52.381601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:52.381679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.381687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.381689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.381692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:52.381769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:52.381777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.381790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.381803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.381817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:52.381838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341779869893328:7145]
2025-10-09T21:46:52.381844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341779869893328:7145]
2025-10-09T21:46:52.381846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341779869893328:7145]
2025-10-09T21:46:52.381855Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:52.381874Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:52.381875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341780231711545:7873], now have 1 active actors on pipe
2025-10-09T21:46:52.381878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341780231711430:2432], now have 1 active actors on pipe
2025-10-09T21:46:52.381887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:52.381889Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:52.381894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:52.381899Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:52.381903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:52.381955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341779869893328:7145]
2025-10-09T21:46:52.381964Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:52.382010Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:52.382280Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 798
2025-10-09T21:46:52.382319Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 798
2025-10-09T21:46:52.382500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:52.382518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:52.382551Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5e63bae6-555cce9b-6968502a-ab029890_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:52.382568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:52.382572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:52.382575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:52.382587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:52.382600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:52.382620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:52.383379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:52.383393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:52.383419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:52.383617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 799 partition: 0 MaxSeqNo: 25 sessionId: src_id|5e63bae6-555cce9b-6968502a-ab029890_0
2025-10-09T21:46:52.384017Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e92e47e0-336695c7-6e3f6049-eea281b8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5e63bae6-555cce9b-6968502a-ab029890_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:52.386719Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5e63bae6-555cce9b-6968502a-ab029890_0] Write session: try to update token
2025-10-09T21:46:52.386742Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5e63bae6-555cce9b-6968502a-ab029890_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:52.397926Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:52.397959Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:52.397968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341779869893328:7145]
2025-10-09T21:46:52.398022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:52.398197Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:52.398843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:52.399150Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:52.399308Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:52.399466Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:52.399690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:52.399753Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:52.408120Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 799 sessionId: src_id|5e63bae6-555cce9b-6968502a-ab029890_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:52.410793Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:52.411018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:52.411041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:52.411094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:52.411266Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:52.413060Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:52.413079Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:52.413082Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:52.413083Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:52.413094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341779869893328:7145]
2025-10-09T21:46:52.413098Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341779869893328:7145]
2025-10-09T21:46:52.413106Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341779869893328:7145]
2025-10-09T21:46:52.413155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:52.413167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.413170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:52.413307Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:52.416453Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:52.416479Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:52.416488Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:52.434828Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341780231711561:7236]
2025-10-09T21:46:52.441998Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 798, pipe: [1:7559341780231711561:7236]
2025-10-09T21:46:52.573658Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:52.574560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341780231711561:7236], now have 1 active actors on pipe
2025-10-09T21:46:52.582449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:52.582477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:52.582649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:52.582703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:52.582761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:52.582816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:52.582855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:52.582861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:52.582922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:52.582973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:52.583029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:52.583089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:52.583134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:52.583137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:52.583198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:52.583251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:52.583304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:52.583398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:52.583519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:52.583522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:52.583582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:52.583634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:52.583688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:52.583742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:52.583781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:52.583783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:52.583851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:52.583904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:52.583962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:52.584020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:52.584078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:52.584083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:52.584145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:52.584202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:52.584275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:52.584330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:52.584382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:52.584385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:52.584444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:52.584512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:52.584570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:52.584624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:52.584660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:52.584666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:52.584749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:52.586218Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:52.586276Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:52.586279Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:52.584799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:52.584849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:52.584900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:52.584960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:52.584962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:52.585024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:52.585076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:52.585131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:52.585185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:52.585526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:52.585530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:52.585592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:52.585643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:52.585699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:52.585754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:52.585808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:52.586975Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e63bae6-555cce9b-6968502a-ab029890_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:52.586991Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e63bae6-555cce9b-6968502a-ab029890_0] Write session will restart in 0.008274s
2025-10-09T21:46:52.587016Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e63bae6-555cce9b-6968502a-ab029890_0] Write session: Do CDS request
2025-10-09T21:46:52.587022Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e63bae6-555cce9b-6968502a-ab029890_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:52.585811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:52.586635Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:52.586319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 799 reason: error at write: Disconnected sessionId: src_id|5e63bae6-555cce9b-6968502a-ab029890_0
2025-10-09T21:46:52.586536Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 799 sessionId: src_id|5e63bae6-555cce9b-6968502a-ab029890_0 is DEAD
2025-10-09T21:46:52.586743Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341780231711561:7236]
2025-10-09T21:46:52.596549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5e63bae6-555cce9b-6968502a-ab029890_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:52.596776Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:52.596798Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 800
2025-10-09T21:46:52.597017Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 800 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:52.597066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 800 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:52.597085Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=800 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:52.597098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:52.597820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 800 sessionId:  describe result for acl check
2025-10-09T21:46:52.597882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:52.597885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:52.597887Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:52.597895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341780231711617:7884] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:52.598507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341780231711617:7884] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:52.605974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341780231711617:7884] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:52.608141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:52.610199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:52.610431Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:52.610442Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:52.610456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:52.610519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:52.610564Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341779869893498:7161]
2025-10-09T21:46:52.611154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:52.610608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341780231711617:7884] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:52.610627Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341780231711617:7884] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:52.610631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341780231711617:7884] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:52.610642Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 800 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:52.611559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:52.611607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:52.611978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:52.612063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:52.612072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:52.612073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:52.612074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:52.612076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:52.612077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:52.612078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.612080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.612081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.612087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:52.612099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:52.612103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:52.612107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:52.612111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:52.612116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:52.612120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.612124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.612128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.612136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:52.612145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:52.612151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:52.612157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:52.612167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:52.612172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:52.612177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:52.612182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:52.612186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:52.612191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:52.612195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:52.612200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:52.612205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.612218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.612222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.612227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.612232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.612238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.612267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:52.620196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:52.621589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:52.621609Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:52.621612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:52.621617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:52.621625Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 799 [2:7559341779869893498:7161]
2025-10-09T21:46:52.621641Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:52.621662Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:52.621666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:52.621669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:52.621671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:52.621691Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:52.621705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:52.621713Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:52.622271Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 799
2025-10-09T21:46:52.622282Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 799
2025-10-09T21:46:52.621780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.621787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.621790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.621793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:52.621867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:52.621870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.621884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.621896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.623254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 800 partition: 0 MaxSeqNo: 25 sessionId: src_id|52a8540d-506c2b6-e7f4e292-7d1280fd_0
2025-10-09T21:46:52.621913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:52.621934Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341779869893492:7161]
2025-10-09T21:46:52.621936Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341779869893492:7161]
2025-10-09T21:46:52.621939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341779869893492:7161]
2025-10-09T21:46:52.621959Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:52.621976Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:52.621991Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:52.621992Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:52.621994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:52.621995Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:52.622000Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:52.622006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341780231711588:2432], now have 1 active actors on pipe
2025-10-09T21:46:52.622064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341780231711704:7884], now have 1 active actors on pipe
2025-10-09T21:46:52.622067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341779869893492:7161]
2025-10-09T21:46:52.622076Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:52.622086Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:52.622532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:52.622541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:52.622577Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|52a8540d-506c2b6-e7f4e292-7d1280fd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:52.622588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:52.622593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:52.622595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:52.622609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:52.622637Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:52.622655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:52.623025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:52.623032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:52.623054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:52.623707Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e63bae6-555cce9b-6968502a-ab029890_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|52a8540d-506c2b6-e7f4e292-7d1280fd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:52.626561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52a8540d-506c2b6-e7f4e292-7d1280fd_0] Write session: try to update token
2025-10-09T21:46:52.626584Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52a8540d-506c2b6-e7f4e292-7d1280fd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:52.643132Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:52.643153Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:52.643156Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:52.643158Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:52.643169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341779869893492:7161]
2025-10-09T21:46:52.643173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341779869893492:7161]
2025-10-09T21:46:52.643177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341779869893492:7161]
2025-10-09T21:46:52.643223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:52.643230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.643232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:52.643361Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:52.646297Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:52.646315Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:52.646320Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:52.650277Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 800 sessionId: src_id|52a8540d-506c2b6-e7f4e292-7d1280fd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:52.653107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:52.686868Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341780231711718:7236]
2025-10-09T21:46:52.687226Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 799, pipe: [1:7559341780231711718:7236]
2025-10-09T21:46:52.805336Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:52.805788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:52.805807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:52.805855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341780231711718:7236], now have 1 active actors on pipe
2025-10-09T21:46:52.806243Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341780231711718:7236]
2025-10-09T21:46:52.806274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:52.806111Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:52.806308Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:52.806395Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 800 reason: status is not ok: tablet will be restarted right now sessionId: src_id|52a8540d-506c2b6-e7f4e292-7d1280fd_0
2025-10-09T21:46:52.806565Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 800 sessionId: src_id|52a8540d-506c2b6-e7f4e292-7d1280fd_0 is DEAD
2025-10-09T21:46:52.810139Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52a8540d-506c2b6-e7f4e292-7d1280fd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:52.810150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52a8540d-506c2b6-e7f4e292-7d1280fd_0] Write session will restart in 0.007403s
2025-10-09T21:46:52.810176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52a8540d-506c2b6-e7f4e292-7d1280fd_0] Write session: Do CDS request
2025-10-09T21:46:52.810181Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52a8540d-506c2b6-e7f4e292-7d1280fd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:52.818537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52a8540d-506c2b6-e7f4e292-7d1280fd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:52.820580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:52.820597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 801
2025-10-09T21:46:52.820848Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 801 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:52.820894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 801 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:52.820902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=801 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:52.820905Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:52.821491Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 801 sessionId:  describe result for acl check
2025-10-09T21:46:52.821527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:52.821529Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:52.821531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:52.821537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341780231711746:7889] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:52.822176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341780231711746:7889] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:52.826854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341780231711746:7889] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:52.838752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341780231711746:7889] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:52.838774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341780231711746:7889] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:52.838778Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341780231711746:7889] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:52.838789Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 801 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:52.838189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:52.841720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:52.841978Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:52.841983Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:52.841997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:52.842079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:52.842134Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341779869893623:7163]
2025-10-09T21:46:52.842715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:52.843092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:52.843154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:52.843475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:52.843526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:52.843527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:52.843528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:52.843530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:52.843531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:52.843532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:52.843533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.843535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.843536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.843541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:52.843552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:52.843555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:52.843560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:52.843564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:52.843568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:52.843571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.843574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.843578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.843586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:52.843594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:52.843600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:52.843606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:52.843612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:52.843617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:52.843622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:52.843627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:52.843632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:52.843637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:52.843641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:52.843646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:52.843651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.843656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.843660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.843665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.843670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.843674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.843705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:52.851570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:52.852758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:52.852783Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:52.852786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:52.852791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:52.852800Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 800 [2:7559341779869893623:7163]
2025-10-09T21:46:52.852814Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:52.852836Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:52.852839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:52.852843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:52.852846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:52.852866Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:52.852880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:52.852889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:52.852960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.852969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.852971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.852975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:52.853052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:52.853060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.853072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:52.853085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:52.853102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:52.853122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341779869893617:7163]
2025-10-09T21:46:52.853125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341779869893617:7163]
2025-10-09T21:46:52.853128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341779869893617:7163]
2025-10-09T21:46:52.853139Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:52.853154Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:52.853167Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:52.853171Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:52.853171Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:52.853174Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:52.853176Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:52.853213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341779869893617:7163]
2025-10-09T21:46:52.853228Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:52.853240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341780231711842:7889], now have 1 active actors on pipe
2025-10-09T21:46:52.853247Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:52.853249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341780231711721:2432], now have 1 active actors on pipe
2025-10-09T21:46:52.853299Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 800
2025-10-09T21:46:52.853335Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 800
2025-10-09T21:46:52.853628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:52.853642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:52.853678Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1041179c-b62c8716-4daf2a9e-1862e16_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:52.853707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:52.853712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:52.853715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:52.853727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:52.853740Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:52.853770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:52.854045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:52.854056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:52.854072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:52.854198Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 801 partition: 0 MaxSeqNo: 25 sessionId: src_id|1041179c-b62c8716-4daf2a9e-1862e16_0
2025-10-09T21:46:52.858561Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52a8540d-506c2b6-e7f4e292-7d1280fd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1041179c-b62c8716-4daf2a9e-1862e16_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:52.861157Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1041179c-b62c8716-4daf2a9e-1862e16_0] Write session: try to update token
2025-10-09T21:46:52.861178Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1041179c-b62c8716-4daf2a9e-1862e16_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:52.872788Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:52.872810Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:52.872822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341779869893617:7163]
2025-10-09T21:46:52.872876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:52.873244Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:52.873573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:52.873724Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:52.873898Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:52.874072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:52.874407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:52.874467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:52.885542Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:52.885564Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:52.885567Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:52.885568Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:52.885579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341779869893617:7163]
2025-10-09T21:46:52.885583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341779869893617:7163]
2025-10-09T21:46:52.885587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341779869893617:7163]
2025-10-09T21:46:52.885633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:52.885640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:52.885643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:52.885649Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:52.885660Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:52.885666Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:52.885784Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:52.887438Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 801 sessionId: src_id|1041179c-b62c8716-4daf2a9e-1862e16_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:52.889665Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:52.906526Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341780231711855:7236]
2025-10-09T21:46:52.906831Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 800, pipe: [1:7559341780231711855:7236]
2025-10-09T21:46:52.978232Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] Counters: { Errors: 7 CurrentSessionLifetimeMs: 241429 BytesRead: 24780812 MessagesRead: 11 BytesReadCompressed: 24780812 BytesInflightUncompressed: 2252802 BytesInflightCompressed: 0 BytesInflightTotal: 2252802 MessagesInflight: 1 }
2025-10-09T21:46:53.052102Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:53.052439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:53.052456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:53.052482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341780231711855:7236], now have 1 active actors on pipe
2025-10-09T21:46:53.052790Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:53.052810Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341780231711855:7236]
2025-10-09T21:46:53.052857Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:53.052950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 801 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1041179c-b62c8716-4daf2a9e-1862e16_0
2025-10-09T21:46:53.053138Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:53.053170Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 801 sessionId: src_id|1041179c-b62c8716-4daf2a9e-1862e16_0 is DEAD
2025-10-09T21:46:53.053181Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:46:53.053344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1041179c-b62c8716-4daf2a9e-1862e16_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:53.053359Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1041179c-b62c8716-4daf2a9e-1862e16_0] Write session will restart in 0.007781s
2025-10-09T21:46:53.053347Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 8 consumer shared/user session shared/user_1_8_189961622288699779_v1 is DEAD
2025-10-09T21:46:53.053394Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1041179c-b62c8716-4daf2a9e-1862e16_0] Write session: Do CDS request
2025-10-09T21:46:53.053402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1041179c-b62c8716-4daf2a9e-1862e16_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:53.053503Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341664267574894:7232] disconnected.
2025-10-09T21:46:53.053515Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341664267574894:7232] disconnected; active server actors: 1
2025-10-09T21:46:53.053519Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341664267574894:7232] client user disconnected session shared/user_1_8_189961622288699779_v1
2025-10-09T21:46:53.053519Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:46:53.053700Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:46:53.053706Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:46:53.053714Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.008609s
2025-10-09T21:46:53.062609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1041179c-b62c8716-4daf2a9e-1862e16_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:53.063350Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:46:53.063213Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:53.063233Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 802
2025-10-09T21:46:53.063410Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 802 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:53.063447Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 802 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:53.063455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=802 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:53.063469Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:53.063757Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:46:53.063768Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 9
2025-10-09T21:46:53.064069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 802 sessionId:  describe result for acl check
2025-10-09T21:46:53.064111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:53.064113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:53.064115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:53.064121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341784526679175:7893] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:53.064161Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 9 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:46:53.064229Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:46:53.064334Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 auth for : user
2025-10-09T21:46:53.064525Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 Handle describe topics response
2025-10-09T21:46:53.064553Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 auth is DEAD
2025-10-09T21:46:53.064567Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:46:53.064740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341784526679175:7893] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:53.064931Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_9_1758451654357684628_v1
2025-10-09T21:46:53.064834Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:46:53.065171Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341784526679193:7895] connected; active server actors: 1
2025-10-09T21:46:53.065192Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559341784526679193:7895] session shared/user_1_9_1758451654357684628_v1
2025-10-09T21:46:53.065203Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:46:53.065215Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:46:53.065222Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_9_1758451654357684628_v1" (Sender=[1:7559341784526679173:7895], Pipe=[1:7559341784526679193:7895], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:46:53.065227Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:46:53.065238Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:46:53.065248Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_9_1758451654357684628_v1" (Sender=[1:7559341784526679173:7895], Pipe=[1:7559341784526679193:7895], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:46:53.065260Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_9_1758451654357684628_v1" sender [1:7559341784526679173:7895] lock partition 0 for ReadingSession "shared/user_1_9_1758451654357684628_v1" (Sender=[1:7559341784526679173:7895], Pipe=[1:7559341784526679193:7895], Partitions=[], ActiveFamilyCount=1) generation 1 step 9
2025-10-09T21:46:53.065293Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:46:53.065301Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000060s
2025-10-09T21:46:53.065560Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 9 Session: "shared/user_1_9_1758451654357684628_v1" ClientId: "user" PipeClient { RawX1: 7559341784526679193 RawX2: 4503603922345687 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:46:53.065590Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:46:53.069908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341784526679175:7893] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:53.071497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:53.073465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:53.073760Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:53.073773Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:53.073789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:53.073867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:53.073922Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341784164861044:7165]
2025-10-09T21:46:53.074560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:53.075071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:53.075144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:53.075511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:53.075589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:53.075590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:53.075592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:53.075593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:53.075595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:53.075596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:53.075597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.075599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.075600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.075607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:53.075632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:53.075636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:53.075639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:53.075643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:53.075647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:53.075651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.075654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.075658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.075667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:53.075676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:53.075684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:53.075690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:53.075694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:53.075699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:53.075703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:53.075709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:53.075713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:53.075719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:53.075724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:53.075729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:53.075737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.075743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.075748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.075754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.075759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.075764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.075797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:53.076126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341784526679175:7893] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:53.076149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341784526679175:7893] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:53.076152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341784526679175:7893] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:53.076163Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 802 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:53.084420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:53.086179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:53.086201Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:53.086204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:53.086210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:53.086218Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 801 [2:7559341784164861044:7165]
2025-10-09T21:46:53.086232Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:53.086259Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:53.086264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:53.086267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:53.086270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:53.086292Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:53.086298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:53.086308Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:53.086388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.086400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.086403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.086407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:53.086497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:53.086506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.086523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.086536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.086554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:53.086577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341784164861039:7165]
2025-10-09T21:46:53.086580Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341784164861039:7165]
2025-10-09T21:46:53.086583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341784164861039:7165]
2025-10-09T21:46:53.086596Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:53.086614Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:53.086615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341784526679205:7899], now have 1 active actors on pipe
2025-10-09T21:46:53.086617Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:53.086620Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:53.086623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341784526679289:7893], now have 1 active actors on pipe
2025-10-09T21:46:53.086626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341784526679158:2432], now have 1 active actors on pipe
2025-10-09T21:46:53.086632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:53.086646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:53.086650Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:53.086699Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341784164861039:7165]
2025-10-09T21:46:53.086706Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:53.086744Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:53.087061Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 801, pipe: [1:7559341784526679205:7899]
2025-10-09T21:46:53.087096Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 801
2025-10-09T21:46:53.087118Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 801
2025-10-09T21:46:53.087453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:53.087474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:53.087503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_9_1758451654357684628_v1 on pipe: [1:7559341784526679205:7899]
2025-10-09T21:46:53.087541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:53.087548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:53.087564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:53.087568Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_9_1758451654357684628_v1:1 with generation 801
2025-10-09T21:46:53.087576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:53.087579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:53.087603Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_9_1758451654357684628_v1
2025-10-09T21:46:53.087611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:53.087656Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5ee50ebd-542e1432-b46ac537-aa7dc599_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:53.087669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:46:53.087674Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:46:53.092365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:46:53.092413Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:46:53.092430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:53.092438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:53.092442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:53.092466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:53.092487Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:53.092497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:46:53.092515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:46:53.092533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:53.092728Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046413086 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:46:53.092753Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:46:53.092833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:53.092853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:53.092817Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 sending to client partition status
2025-10-09T21:46:53.092898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:53.093028Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 802 partition: 0 MaxSeqNo: 25 sessionId: src_id|5ee50ebd-542e1432-b46ac537-aa7dc599_0
2025-10-09T21:46:53.093393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1041179c-b62c8716-4daf2a9e-1862e16_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5ee50ebd-542e1432-b46ac537-aa7dc599_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:53.096339Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ee50ebd-542e1432-b46ac537-aa7dc599_0] Write session: try to update token
2025-10-09T21:46:53.096364Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ee50ebd-542e1432-b46ac537-aa7dc599_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:53.114936Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:53.114962Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:53.114965Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:53.114968Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:53.114984Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341784164861039:7165]
2025-10-09T21:46:53.114989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341784164861039:7165]
2025-10-09T21:46:53.114993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341784164861039:7165]
2025-10-09T21:46:53.115051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:53.115066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.115068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:53.115105Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:53.115122Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:53.115126Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:53.115238Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:53.120437Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 802 sessionId: src_id|5ee50ebd-542e1432-b46ac537-aa7dc599_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:53.123405Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:53.286665Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:53.286990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:53.287010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:53.287391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_9_1758451654357684628_v1
2025-10-09T21:46:53.287389Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:53.287390Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341784526679205:7899]
2025-10-09T21:46:53.287415Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:53.287405Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:53.287413Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:53.287485Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_9_1758451654357684628_v1
2025-10-09T21:46:53.287478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 802 reason: error at write: Disconnected sessionId: src_id|5ee50ebd-542e1432-b46ac537-aa7dc599_0
2025-10-09T21:46:53.287664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 802 sessionId: src_id|5ee50ebd-542e1432-b46ac537-aa7dc599_0 is DEAD
2025-10-09T21:46:53.290321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ee50ebd-542e1432-b46ac537-aa7dc599_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:53.290337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ee50ebd-542e1432-b46ac537-aa7dc599_0] Write session will restart in 0.006518s
2025-10-09T21:46:53.290363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ee50ebd-542e1432-b46ac537-aa7dc599_0] Write session: Do CDS request
2025-10-09T21:46:53.290370Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ee50ebd-542e1432-b46ac537-aa7dc599_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:53.298625Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ee50ebd-542e1432-b46ac537-aa7dc599_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:53.298885Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:53.298904Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 803
2025-10-09T21:46:53.299025Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 803 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:53.299058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 803 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:53.299065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=803 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:53.299069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:53.299649Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 803 sessionId:  describe result for acl check
2025-10-09T21:46:53.299687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:53.299689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:53.299693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:53.299702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341784526679341:7902] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:53.300281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341784526679341:7902] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:53.303713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341784526679341:7902] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:53.308134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341784526679341:7902] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:53.308152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341784526679341:7902] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:53.308155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341784526679341:7902] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:53.308165Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 803 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:53.307810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:53.310003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:53.310251Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:53.310266Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:53.310280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:53.310346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:53.310388Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341784164861174:7167]
2025-10-09T21:46:53.310978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:53.313697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:53.313757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:53.314177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:53.314265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:53.314267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:53.314268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:53.314269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:53.314270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:53.314272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:53.314273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.314274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.314275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.314282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:53.314293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:53.314297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:53.314301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:53.314305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:53.314309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:53.314313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.314316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.314320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.314328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:53.314337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:53.314343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:53.314348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:53.314353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:53.314359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:53.314364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:53.314369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:53.314374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:53.314380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:53.314387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:53.314394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:53.314398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.314404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.314409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.314414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.314419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.314425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.314455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:53.323174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:53.324410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:53.324433Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:53.324436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:53.324441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:53.324450Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 802 [2:7559341784164861174:7167]
2025-10-09T21:46:53.324467Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:53.324489Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:53.324495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:53.324499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:53.324501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:53.324523Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:53.324535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:53.324544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:53.324622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.324633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.324635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.324638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:53.324723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:53.324734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.324747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.324765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.324788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:53.324813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341784164861167:7167]
2025-10-09T21:46:53.324820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341784164861167:7167]
2025-10-09T21:46:53.324822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341784164861167:7167]
2025-10-09T21:46:53.324832Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:53.324845Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:53.324849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:53.324849Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:53.324852Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:53.324866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:53.324870Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:53.324882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341784526679308:2432], now have 1 active actors on pipe
2025-10-09T21:46:53.324892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341784526679431:7902], now have 1 active actors on pipe
2025-10-09T21:46:53.324932Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341784164861167:7167]
2025-10-09T21:46:53.324948Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:53.324961Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 802
2025-10-09T21:46:53.325019Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 802
2025-10-09T21:46:53.325249Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:53.325661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:53.325675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:53.325707Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f7bc55c2-5046fbb9-cd9269d3-b4f49493_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:53.325734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:53.325738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:53.325742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:53.325754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:53.325772Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:53.325795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:53.326010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:53.326017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:53.326050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:53.326147Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 803 partition: 0 MaxSeqNo: 25 sessionId: src_id|f7bc55c2-5046fbb9-cd9269d3-b4f49493_0
2025-10-09T21:46:53.326575Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ee50ebd-542e1432-b46ac537-aa7dc599_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f7bc55c2-5046fbb9-cd9269d3-b4f49493_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:53.329298Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7bc55c2-5046fbb9-cd9269d3-b4f49493_0] Write session: try to update token
2025-10-09T21:46:53.329316Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7bc55c2-5046fbb9-cd9269d3-b4f49493_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:53.345920Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 803 sessionId: src_id|f7bc55c2-5046fbb9-cd9269d3-b4f49493_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:53.348595Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:53.351165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:53.351196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:53.351225Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:53.351229Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:53.351239Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341784164861167:7167]
2025-10-09T21:46:53.351308Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:53.351311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:53.351522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:53.351675Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:53.351819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:53.351964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:53.352209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:53.352261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:53.352292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:53.352441Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:53.355688Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:53.355705Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:53.355708Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:53.355710Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:53.355720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341784164861167:7167]
2025-10-09T21:46:53.355726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341784164861167:7167]
2025-10-09T21:46:53.355734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341784164861167:7167]
2025-10-09T21:46:53.355778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:53.355787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.355799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:53.355821Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:53.355828Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:53.355832Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:53.355941Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:53.387768Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341784526679444:7899]
2025-10-09T21:46:53.388184Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 802, pipe: [1:7559341784526679444:7899]
2025-10-09T21:46:53.536499Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:53.545195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:53.545224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:53.545383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:53.545444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:53.545510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:53.545573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:53.545618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:53.545624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:53.545704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:53.545760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:53.545823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:53.545895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:53.545935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:53.545938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:53.546022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:53.549472Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341784526679444:7899]
2025-10-09T21:46:53.549501Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:53.549560Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:53.549567Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:53.549593Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 803 reason: error at write: Disconnected sessionId: src_id|f7bc55c2-5046fbb9-cd9269d3-b4f49493_0
2025-10-09T21:46:53.549786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 803 sessionId: src_id|f7bc55c2-5046fbb9-cd9269d3-b4f49493_0 is DEAD
2025-10-09T21:46:53.550024Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7bc55c2-5046fbb9-cd9269d3-b4f49493_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:53.550054Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7bc55c2-5046fbb9-cd9269d3-b4f49493_0] Write session will restart in 0.007835s
2025-10-09T21:46:53.546217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:53.550082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7bc55c2-5046fbb9-cd9269d3-b4f49493_0] Write session: Do CDS request
2025-10-09T21:46:53.546286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:53.546344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:53.550090Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7bc55c2-5046fbb9-cd9269d3-b4f49493_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:53.546384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:53.546388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:53.546459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:53.546518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:53.546576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:53.546638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:53.546686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:53.546689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:53.546765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:53.546825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:53.546888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:53.546950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:53.546992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:53.546997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:53.547068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:53.547127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:53.547187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:53.547272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:53.547317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:53.547319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:53.547387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:53.547447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:53.547550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:53.547613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:53.547784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:53.547788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:53.547855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:53.547913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:53.547976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:53.548035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:53.548079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:53.548082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:53.548147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:53.548204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:53.548265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:53.548327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:53.548371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:53.548373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:53.548443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:53.548497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:53.548557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:53.548613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:53.548787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:53.548791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:53.548861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341784526679444:7899], now have 1 active actors on pipe
2025-10-09T21:46:53.550316Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:53.559657Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7bc55c2-5046fbb9-cd9269d3-b4f49493_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:53.559712Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:53.559729Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 804
2025-10-09T21:46:53.559867Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 804 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:53.559895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 804 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:53.559901Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=804 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:53.559905Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:53.560467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 804 sessionId:  describe result for acl check
2025-10-09T21:46:53.560502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:53.560504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:53.560505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:53.560511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341784526679487:7913] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:53.561089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341784526679487:7913] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:53.564499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341784526679487:7913] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:53.570714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341784526679487:7913] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:53.570732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341784526679487:7913] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:53.570736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341784526679487:7913] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:53.570748Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 804 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:53.571040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:53.573117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:53.573365Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:53.573378Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:53.573391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:53.573454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:53.573500Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341784164861343:7181]
2025-10-09T21:46:53.574113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:53.574561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:53.574627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:53.574944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:53.575012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:53.575019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:53.575020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:53.575021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:53.575022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:53.575023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:53.575024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.575025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.575026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.575032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:53.575044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:53.575048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:53.575052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:53.575055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:53.575059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:53.575062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.575064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.575068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.575076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:53.575084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:53.575091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:53.575097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:53.575101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:53.575105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:53.575109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:53.575113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:53.575117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:53.575123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:53.575126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:53.575131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:53.575137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.575141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.575146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.575150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.575155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.575159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.575188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:53.582567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:53.584006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:53.584027Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:53.584030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:53.584035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:53.584521Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 803
2025-10-09T21:46:53.584581Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 803
2025-10-09T21:46:53.584043Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 803 [2:7559341784164861343:7181]
2025-10-09T21:46:53.584057Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:53.584078Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:53.584082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:53.584085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:53.584087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:53.584110Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:53.584115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:53.584123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:53.584194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.584196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.584199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.584201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:53.584281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:53.584284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.584298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.584309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.584384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:53.584409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341784526679593:7913], now have 1 active actors on pipe
2025-10-09T21:46:53.584412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341784526679471:2432], now have 1 active actors on pipe
2025-10-09T21:46:53.584420Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341784164861338:7181]
2025-10-09T21:46:53.584436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341784164861338:7181]
2025-10-09T21:46:53.584438Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341784164861338:7181]
2025-10-09T21:46:53.584451Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:53.584467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:53.584491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:53.584495Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:53.584542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341784164861338:7181]
2025-10-09T21:46:53.584545Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:53.584544Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:53.584550Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:53.584552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:53.584559Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:53.584740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:53.584750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:53.584793Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c39c13b1-cc7a6cb5-70411aa8-a97dcb6b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:53.584805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:53.584810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:53.584813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:53.584826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:53.584839Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:53.584859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:53.587159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:53.587189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:53.587253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:53.587405Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 804 partition: 0 MaxSeqNo: 25 sessionId: src_id|c39c13b1-cc7a6cb5-70411aa8-a97dcb6b_0
2025-10-09T21:46:53.587929Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7bc55c2-5046fbb9-cd9269d3-b4f49493_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c39c13b1-cc7a6cb5-70411aa8-a97dcb6b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:53.590664Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c39c13b1-cc7a6cb5-70411aa8-a97dcb6b_0] Write session: try to update token
2025-10-09T21:46:53.590686Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c39c13b1-cc7a6cb5-70411aa8-a97dcb6b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:53.603418Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:53.603441Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:53.603445Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:53.603447Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:53.603457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341784164861338:7181]
2025-10-09T21:46:53.603463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341784164861338:7181]
2025-10-09T21:46:53.603466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341784164861338:7181]
2025-10-09T21:46:53.603514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:53.603527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.603530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:53.603684Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:53.603912Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:53.603925Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:53.603932Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:53.608448Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 804 sessionId: src_id|c39c13b1-cc7a6cb5-70411aa8-a97dcb6b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:53.611006Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:53.647887Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341784526679605:7899]
2025-10-09T21:46:53.648242Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 803, pipe: [1:7559341784526679605:7899]
2025-10-09T21:46:53.773108Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341784526679605:7899]
2025-10-09T21:46:53.773137Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:53.773179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:53.773182Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:53.773244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 804 reason: error at write: Disconnected sessionId: src_id|c39c13b1-cc7a6cb5-70411aa8-a97dcb6b_0
2025-10-09T21:46:53.773446Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 804 sessionId: src_id|c39c13b1-cc7a6cb5-70411aa8-a97dcb6b_0 is DEAD
2025-10-09T21:46:53.773878Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c39c13b1-cc7a6cb5-70411aa8-a97dcb6b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:53.773891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c39c13b1-cc7a6cb5-70411aa8-a97dcb6b_0] Write session will restart in 0.006437s
2025-10-09T21:46:53.773915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c39c13b1-cc7a6cb5-70411aa8-a97dcb6b_0] Write session: Do CDS request
2025-10-09T21:46:53.773923Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c39c13b1-cc7a6cb5-70411aa8-a97dcb6b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:53.772417Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:53.772728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:53.772740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:53.772789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341784526679605:7899], now have 1 active actors on pipe
2025-10-09T21:46:53.773831Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:53.781888Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c39c13b1-cc7a6cb5-70411aa8-a97dcb6b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:53.782072Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:53.782090Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 805
2025-10-09T21:46:53.782238Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 805 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:53.782273Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 805 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:53.782280Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=805 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:53.782286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:53.782837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 805 sessionId:  describe result for acl check
2025-10-09T21:46:53.782883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:53.782886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:53.782888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:53.782895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341784526679625:7918] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:53.783478Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341784526679625:7918] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:53.788001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341784526679625:7918] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:53.792921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:53.793055Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341784526679625:7918] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:53.793071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341784526679625:7918] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:53.793075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341784526679625:7918] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:53.793088Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 805 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:53.795362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:53.795640Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:53.795645Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:53.795661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:53.795732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:53.795769Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341784164861472:7185]
2025-10-09T21:46:53.796358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:53.800234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:53.800324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:53.800751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:53.800819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:53.800829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:53.800831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:53.800833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:53.800834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:53.800835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:53.800837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.800838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.800839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.800847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:53.800858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:53.800862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:53.800866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:53.800870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:53.800873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:53.800877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.800881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.800884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.800894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:53.800904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:53.800911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:53.800917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:53.800922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:53.800928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:53.800933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:53.800939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:53.800945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:53.800951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:53.800957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:53.800963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:53.800969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.800975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.800981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.800987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.800992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.800998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.801046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:53.808373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:53.809531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:53.809551Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:53.809555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:53.809560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:53.809570Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 804 [2:7559341784164861472:7185]
2025-10-09T21:46:53.809586Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:53.809607Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:53.809613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:53.809617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:53.809620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:53.809639Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:53.809650Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:53.809660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:53.809752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.809759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.809761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.809764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:53.809846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:53.809855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.809868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:53.809881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:53.809903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:53.809933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341784526679610:2432], now have 1 active actors on pipe
2025-10-09T21:46:53.809937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341784526679734:7918], now have 1 active actors on pipe
2025-10-09T21:46:53.809961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341784164861466:7185]
2025-10-09T21:46:53.809971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341784164861466:7185]
2025-10-09T21:46:53.809973Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341784164861466:7185]
2025-10-09T21:46:53.810001Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:53.810046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:53.810068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:53.810072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:53.810116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341784164861466:7185]
2025-10-09T21:46:53.810126Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:53.810129Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 804
2025-10-09T21:46:53.810148Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:53.810173Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:53.810176Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:53.810183Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:53.810186Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 804
2025-10-09T21:46:53.810416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:53.810430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:53.810463Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|399c4d1c-e2632a92-cc82c2ac-9203c803_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:53.810476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:53.810482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:53.810485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:53.810497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:53.810509Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:53.810526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:53.810859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:53.810872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:53.810893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:53.810995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 805 partition: 0 MaxSeqNo: 25 sessionId: src_id|399c4d1c-e2632a92-cc82c2ac-9203c803_0
2025-10-09T21:46:53.811377Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c39c13b1-cc7a6cb5-70411aa8-a97dcb6b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|399c4d1c-e2632a92-cc82c2ac-9203c803_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:53.813952Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|399c4d1c-e2632a92-cc82c2ac-9203c803_0] Write session: try to update token
2025-10-09T21:46:53.813970Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|399c4d1c-e2632a92-cc82c2ac-9203c803_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:53.825352Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:53.825373Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:53.825376Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:53.825378Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:53.825387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341784164861466:7185]
2025-10-09T21:46:53.825392Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341784164861466:7185]
2025-10-09T21:46:53.825396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341784164861466:7185]
2025-10-09T21:46:53.825444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:53.825445Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:53.825451Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:53.825452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:53.825455Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:53.825455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:53.825601Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:53.832185Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 805 sessionId: src_id|399c4d1c-e2632a92-cc82c2ac-9203c803_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:53.835064Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:53.873170Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341784526679748:7899]
2025-10-09T21:46:53.873616Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 804, pipe: [1:7559341784526679748:7899]
2025-10-09T21:46:53.989486Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:53.989918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:53.989935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:53.989995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341784526679748:7899], now have 1 active actors on pipe
2025-10-09T21:46:53.990296Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:53.990352Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341784526679748:7899]
2025-10-09T21:46:53.990370Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:53.990377Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:53.990406Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:53.990440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 805 reason: error at write: Disconnected sessionId: src_id|399c4d1c-e2632a92-cc82c2ac-9203c803_0
2025-10-09T21:46:53.990619Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 805 sessionId: src_id|399c4d1c-e2632a92-cc82c2ac-9203c803_0 is DEAD
2025-10-09T21:46:53.990796Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|399c4d1c-e2632a92-cc82c2ac-9203c803_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:53.990805Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|399c4d1c-e2632a92-cc82c2ac-9203c803_0] Write session will restart in 0.006047s
2025-10-09T21:46:53.990833Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|399c4d1c-e2632a92-cc82c2ac-9203c803_0] Write session: Do CDS request
2025-10-09T21:46:53.990839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|399c4d1c-e2632a92-cc82c2ac-9203c803_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:53.999131Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|399c4d1c-e2632a92-cc82c2ac-9203c803_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:53.999341Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:53.999364Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 806
2025-10-09T21:46:53.999564Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 806 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:53.999596Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 806 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:53.999611Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=806 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:53.999615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:54.000023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 806 sessionId:  describe result for acl check
2025-10-09T21:46:54.000050Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:54.000051Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:54.000056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:54.000062Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341784526679759:7922] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:54.000582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341784526679759:7922] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:54.003034Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341784526679759:7922] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:54.007634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341784526679759:7922] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:54.007651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341784526679759:7922] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:54.007656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341784526679759:7922] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:54.007666Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 806 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:54.015959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:54.018150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:54.018411Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:54.018427Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:54.018441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:54.018514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:54.018565Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341788459828897:7187]
2025-10-09T21:46:54.019189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:54.019638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:54.019699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:54.020067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:54.020125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:54.020132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:54.020133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:54.020134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:54.020136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:54.020137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:54.020139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.020140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.020142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.020148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:54.020159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:54.020164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:54.020168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:54.020172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:54.020176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:54.020179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.020183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.020187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.020195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:54.020204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:54.020210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:54.020215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:54.020221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:54.020227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:54.020231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:54.020237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:54.020241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:54.020247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:54.020251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:54.020257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:54.020262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.020267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.020272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.020279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.020284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.020289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.020321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:54.028467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:54.029626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:54.029633Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:54.029637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:54.029642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:54.029650Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 805 [2:7559341788459828897:7187]
2025-10-09T21:46:54.029669Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:54.029689Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:54.029693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:54.029697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:54.029699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:54.029719Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:54.029727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:54.029735Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:54.029804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.029807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.029810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.029813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:54.029894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:54.029897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.029910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.029921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.029937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:54.029971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341788459828892:7187]
2025-10-09T21:46:54.029974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341788459828892:7187]
2025-10-09T21:46:54.029976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341788459828892:7187]
2025-10-09T21:46:54.029986Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:54.030004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341788821647096:7922], now have 1 active actors on pipe
2025-10-09T21:46:54.030013Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:54.030044Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:54.030027Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:54.030048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:54.030051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:54.030054Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:54.030092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341788459828892:7187]
2025-10-09T21:46:54.030101Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:54.030107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341784526679752:2432], now have 1 active actors on pipe
2025-10-09T21:46:54.030122Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:54.030209Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 805
2025-10-09T21:46:54.030275Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 805
2025-10-09T21:46:54.030382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:54.030396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:54.030430Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bac3f605-a33cf577-cab15601-d05a1d0b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:54.030443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:54.030448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:54.030451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:54.030464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:54.030479Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:54.030505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:54.030834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:54.030844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:54.030864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:54.030983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 806 partition: 0 MaxSeqNo: 25 sessionId: src_id|bac3f605-a33cf577-cab15601-d05a1d0b_0
2025-10-09T21:46:54.031399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|399c4d1c-e2632a92-cc82c2ac-9203c803_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bac3f605-a33cf577-cab15601-d05a1d0b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:54.034198Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bac3f605-a33cf577-cab15601-d05a1d0b_0] Write session: try to update token
2025-10-09T21:46:54.034218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bac3f605-a33cf577-cab15601-d05a1d0b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:54.044275Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:54.044297Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:54.044308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341788459828892:7187]
2025-10-09T21:46:54.044360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:54.044361Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:54.044539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:54.044689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:54.044829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:54.044976Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:54.045176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:54.045223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:54.056048Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 806 sessionId: src_id|bac3f605-a33cf577-cab15601-d05a1d0b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:54.058917Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:54.059505Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:54.059274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:54.059299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:54.059351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:54.065304Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:54.065325Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:54.065327Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:54.065330Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:54.065341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341788459828892:7187]
2025-10-09T21:46:54.065346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341788459828892:7187]
2025-10-09T21:46:54.065351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341788459828892:7187]
2025-10-09T21:46:54.065399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:54.065406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.065409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:54.065544Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:54.069957Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:54.069979Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:54.069983Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:54.087161Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341788821647187:7899]
2025-10-09T21:46:54.087741Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 805, pipe: [1:7559341788821647187:7899]
2025-10-09T21:46:54.248557Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:54.249000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341788821647187:7899], now have 1 active actors on pipe
2025-10-09T21:46:54.256946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:54.256980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:54.257146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.257213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.257278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.257336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.257380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.257390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.257458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.257518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.257582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.257644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.257696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.257704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.257764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.257826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.257894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.257967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.258011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.258015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.258174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.258238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.258296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.258360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.258404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.258413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.258478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.258539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.258605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.258671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.258786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.258795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.258862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.260646Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:54.260768Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341788821647187:7899]
2025-10-09T21:46:54.260841Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:54.260850Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:54.260879Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 806 reason: error at write: Disconnected sessionId: src_id|bac3f605-a33cf577-cab15601-d05a1d0b_0
2025-10-09T21:46:54.261209Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bac3f605-a33cf577-cab15601-d05a1d0b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:54.261232Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bac3f605-a33cf577-cab15601-d05a1d0b_0] Write session will restart in 0.005620s
2025-10-09T21:46:54.261257Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bac3f605-a33cf577-cab15601-d05a1d0b_0] Write session: Do CDS request
2025-10-09T21:46:54.261263Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bac3f605-a33cf577-cab15601-d05a1d0b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:54.261153Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 806 sessionId: src_id|bac3f605-a33cf577-cab15601-d05a1d0b_0 is DEAD
2025-10-09T21:46:54.258923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.258999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.259092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.259131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.259135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.259233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.259288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.259349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.259402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.259439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.259441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.259503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.259554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.259605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.259660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.259701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.259704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.259772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.259829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.259890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.259952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.259995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.259998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.260064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.260119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.260178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.260240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.260302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.260305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.261517Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:54.268707Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bac3f605-a33cf577-cab15601-d05a1d0b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:54.269035Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:54.269052Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 807
2025-10-09T21:46:54.269352Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 807 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:54.269395Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 807 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:54.269401Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=807 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:54.269406Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:54.269930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 807 sessionId:  describe result for acl check
2025-10-09T21:46:54.269994Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:54.270001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:54.270003Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:54.270010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341788821647206:7927] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:54.270632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341788821647206:7927] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:54.273696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341788821647206:7927] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:54.277096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341788821647206:7927] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:54.277126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341788821647206:7927] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:54.277131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341788821647206:7927] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:54.277142Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 807 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:54.282423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:54.284429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:54.284696Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:54.284711Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:54.284725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:54.284794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:54.284840Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341788459829028:7189]
2025-10-09T21:46:54.285420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:54.285785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:54.285856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:54.286250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:54.286342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:54.286350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:54.286351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:54.286353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:54.286354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:54.286355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:54.286356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.286357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.286358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.286364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:54.286376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:54.286380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:54.286383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:54.286386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:54.286391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:54.286395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.286398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.286402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.286410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:54.286419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:54.286425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:54.286431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:54.286436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:54.286441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:54.286446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:54.286451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:54.286456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:54.286461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:54.286465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:54.286471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:54.286476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.286481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.286485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.286489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.286494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.286499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.286529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:54.295078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:54.296520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:54.296541Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:54.296544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:54.296549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:54.296557Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 806 [2:7559341788459829028:7189]
2025-10-09T21:46:54.296573Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:54.296595Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:54.296599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:54.296602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:54.296605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:54.296624Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:54.296631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:54.296639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:54.296709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.296713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.296716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.296719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:54.296805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:54.296815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.296832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.296847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.296865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:54.296889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341788821647251:7927], now have 1 active actors on pipe
2025-10-09T21:46:54.296900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341788821647195:2432], now have 1 active actors on pipe
2025-10-09T21:46:54.296908Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341788459829023:7189]
2025-10-09T21:46:54.296910Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341788459829023:7189]
2025-10-09T21:46:54.296913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341788459829023:7189]
2025-10-09T21:46:54.296927Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:54.296944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:54.296952Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:54.296959Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:54.296963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:54.296972Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:54.296975Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:54.297007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341788459829023:7189]
2025-10-09T21:46:54.297010Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:54.296969Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 806
2025-10-09T21:46:54.296989Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 806
2025-10-09T21:46:54.297032Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:54.297044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:54.297049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:54.297092Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|df70d09a-421e7d4-da8af007-778f327d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:54.297100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:54.297104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:54.297107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:54.297117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:54.297128Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:54.297151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:54.297596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:54.297610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:54.297632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:54.298008Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 807 partition: 0 MaxSeqNo: 25 sessionId: src_id|df70d09a-421e7d4-da8af007-778f327d_0
2025-10-09T21:46:54.298422Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bac3f605-a33cf577-cab15601-d05a1d0b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|df70d09a-421e7d4-da8af007-778f327d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:54.301017Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df70d09a-421e7d4-da8af007-778f327d_0] Write session: try to update token
2025-10-09T21:46:54.301039Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df70d09a-421e7d4-da8af007-778f327d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:54.312829Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:54.312853Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:54.312859Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:54.312861Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:54.312872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341788459829023:7189]
2025-10-09T21:46:54.312876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341788459829023:7189]
2025-10-09T21:46:54.312880Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341788459829023:7189]
2025-10-09T21:46:54.312929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:54.312942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.312942Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:54.312945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:54.312949Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:54.312954Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:54.313086Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:54.320809Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 807 sessionId: src_id|df70d09a-421e7d4-da8af007-778f327d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:54.323435Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:54.361228Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341788821647335:7899]
2025-10-09T21:46:54.361767Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 806, pipe: [1:7559341788821647335:7899]
2025-10-09T21:46:54.492685Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:54.493118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:54.493130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:54.493155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341788821647335:7899], now have 1 active actors on pipe
2025-10-09T21:46:54.493552Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:54.493563Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:54.493583Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:54.493640Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 807 reason: error at write: Disconnected sessionId: src_id|df70d09a-421e7d4-da8af007-778f327d_0
2025-10-09T21:46:54.493846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 807 sessionId: src_id|df70d09a-421e7d4-da8af007-778f327d_0 is DEAD
2025-10-09T21:46:54.493863Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341788821647335:7899]
2025-10-09T21:46:54.493891Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:54.494242Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df70d09a-421e7d4-da8af007-778f327d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:54.494253Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df70d09a-421e7d4-da8af007-778f327d_0] Write session will restart in 0.005504s
2025-10-09T21:46:54.494278Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df70d09a-421e7d4-da8af007-778f327d_0] Write session: Do CDS request
2025-10-09T21:46:54.494285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df70d09a-421e7d4-da8af007-778f327d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:54.500670Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df70d09a-421e7d4-da8af007-778f327d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:54.500732Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:54.500748Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 808
2025-10-09T21:46:54.500936Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 808 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:54.500982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 808 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:54.500988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=808 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:54.500992Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:54.501445Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 808 sessionId:  describe result for acl check
2025-10-09T21:46:54.501481Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:54.501482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:54.501484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:54.501490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341788821647344:7931] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:54.502101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341788821647344:7931] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:54.506407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341788821647344:7931] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:54.510498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341788821647344:7931] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:54.510516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341788821647344:7931] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:54.510520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341788821647344:7931] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:54.510530Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 808 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:54.527747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:54.530051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:54.530280Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:54.530294Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:54.530307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:54.530376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:54.530427Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341788459829159:7191]
2025-10-09T21:46:54.530994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:54.531840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:54.531896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:54.532214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:54.532276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:54.532285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:54.532286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:54.532287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:54.532288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:54.532290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:54.532291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.532292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.532293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.532299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:54.532310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:54.532314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:54.532318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:54.532322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:54.532326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:54.532329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.532333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.532337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.532344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:54.532352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:54.532359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:54.532364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:54.532368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:54.532372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:54.532375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:54.532379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:54.532384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:54.532389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:54.532394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:54.532399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:54.532405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.532411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.532415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.532420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.532426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.532431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.532459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:54.541513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:54.542632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:54.542650Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:54.542653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:54.542658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:54.542666Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 807 [2:7559341788459829159:7191]
2025-10-09T21:46:54.542684Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:54.542704Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:54.542714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:54.542718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:54.542721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:54.542740Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:54.542746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:54.542754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:54.542820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.542829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.542831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.542835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:54.542908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:54.542911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.542923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.542934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.542948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:54.542969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341788821647398:7931], now have 1 active actors on pipe
2025-10-09T21:46:54.542977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341788459829153:7191]
2025-10-09T21:46:54.542980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341788459829153:7191]
2025-10-09T21:46:54.542982Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341788459829153:7191]
2025-10-09T21:46:54.542992Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:54.543003Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:54.543006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341788821647337:2432], now have 1 active actors on pipe
2025-10-09T21:46:54.543006Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:54.543008Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:54.543015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:54.543028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:54.543032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:54.543073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341788459829153:7191]
2025-10-09T21:46:54.543082Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:54.543092Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:54.543299Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 807
2025-10-09T21:46:54.543334Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 807
2025-10-09T21:46:54.543449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:54.543461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:54.543491Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d9c53739-a56b3bc9-b125886c-7f26c2ca_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:54.543513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:54.543517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:54.543520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:54.543535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:54.543548Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:54.543565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:54.544137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:54.544148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:54.544167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:54.544330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 808 partition: 0 MaxSeqNo: 25 sessionId: src_id|d9c53739-a56b3bc9-b125886c-7f26c2ca_0
2025-10-09T21:46:54.544722Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df70d09a-421e7d4-da8af007-778f327d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d9c53739-a56b3bc9-b125886c-7f26c2ca_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:54.547240Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d9c53739-a56b3bc9-b125886c-7f26c2ca_0] Write session: try to update token
2025-10-09T21:46:54.547257Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d9c53739-a56b3bc9-b125886c-7f26c2ca_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:54.561109Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:54.561130Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:54.561139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341788459829153:7191]
2025-10-09T21:46:54.561190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:54.561384Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:54.561519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:54.561737Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:54.562551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:54.562705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:54.562902Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:54.562945Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:54.569855Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 808 sessionId: src_id|d9c53739-a56b3bc9-b125886c-7f26c2ca_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:54.572442Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:54.576124Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:54.574459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:54.574483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:54.575941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:54.575975Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:54.575979Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:54.575981Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:54.575983Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:54.575994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341788459829153:7191]
2025-10-09T21:46:54.575998Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341788459829153:7191]
2025-10-09T21:46:54.576001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341788459829153:7191]
2025-10-09T21:46:54.576040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:54.576041Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:54.576047Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:54.576049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.576051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:54.576051Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:54.576224Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:54.594547Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341788821647491:7899]
2025-10-09T21:46:54.598502Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 807, pipe: [1:7559341788821647491:7899]
2025-10-09T21:46:54.797480Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:54.799108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341788821647491:7899], now have 1 active actors on pipe
2025-10-09T21:46:54.808737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:54.808772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:54.808949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.809013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.809086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.809150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.809202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.809207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.809281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.809344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.809405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.809468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.809507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.809510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.809577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.809634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.809690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.809750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.809793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.809797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.809867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.809926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.810002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.810084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.810124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.810127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.810196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.810253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.810308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.810367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.812778Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:54.812861Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:54.812862Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341788821647491:7899]
2025-10-09T21:46:54.812871Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:54.812907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 808 reason: error at write: Disconnected sessionId: src_id|d9c53739-a56b3bc9-b125886c-7f26c2ca_0
2025-10-09T21:46:54.810405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.810412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.810476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.810532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.810593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.810657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.810699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.810702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.810771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.810825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.810885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.810941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.811046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.811049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.811113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.811166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.811226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.811283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.811323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.811339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.811404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.811458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.811518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.811583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.811627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.811630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.811701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:54.811756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:54.811815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:54.811872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:54.811914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:54.811917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:54.813258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9c53739-a56b3bc9-b125886c-7f26c2ca_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:54.813294Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9c53739-a56b3bc9-b125886c-7f26c2ca_0] Write session will restart in 0.005218s
2025-10-09T21:46:54.813321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9c53739-a56b3bc9-b125886c-7f26c2ca_0] Write session: Do CDS request
2025-10-09T21:46:54.813327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9c53739-a56b3bc9-b125886c-7f26c2ca_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:54.813555Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 808 sessionId: src_id|d9c53739-a56b3bc9-b125886c-7f26c2ca_0 is DEAD
2025-10-09T21:46:54.813912Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:54.819592Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d9c53739-a56b3bc9-b125886c-7f26c2ca_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:54.819714Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:54.819731Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 809
2025-10-09T21:46:54.819841Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 809 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:54.819869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 809 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:54.819874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=809 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:54.819879Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:54.820418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 809 sessionId:  describe result for acl check
2025-10-09T21:46:54.820454Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:54.820456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:54.820458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:54.820464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341788821647523:7943] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:54.821002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341788821647523:7943] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:54.824388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341788821647523:7943] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:54.828405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341788821647523:7943] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:54.828420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341788821647523:7943] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:54.828424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341788821647523:7943] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:54.828434Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 809 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:54.831057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:54.832684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:54.832906Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:54.832918Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:54.832930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:54.832982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:54.833030Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341788459829335:7207]
2025-10-09T21:46:54.833569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:54.833997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:54.834058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:54.834378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:54.834425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:54.834426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:54.834428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:54.834429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:54.834430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:54.834431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:54.834432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.834433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.834434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.834440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:54.834451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:54.834454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:54.834458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:54.834462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:54.834465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:54.834469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.834472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.834475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.834482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:54.834490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:54.834495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:54.834500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:54.834504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:54.834508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:54.834513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:54.834517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:54.834521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:54.834526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:54.834530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:54.834535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:54.834540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.834544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.834548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.834552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.834556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.834561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.834588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:54.843786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:54.845471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:54.845479Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:54.845481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:54.845485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:54.845494Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 808 [2:7559341788459829335:7207]
2025-10-09T21:46:54.845510Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:54.845531Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:54.845535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:54.845538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:54.845540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:54.845559Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:54.845565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:54.845573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:54.845647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.845649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.845651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.845653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:54.845736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:54.845739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.845750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:54.845759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:54.845771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:54.845792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341788459829329:7207]
2025-10-09T21:46:54.845794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341788459829329:7207]
2025-10-09T21:46:54.845796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341788459829329:7207]
2025-10-09T21:46:54.845804Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:54.845819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:54.845827Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:54.845831Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:54.845832Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:54.845833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:54.845835Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:54.845844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341788821647639:7943], now have 1 active actors on pipe
2025-10-09T21:46:54.845847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341788821647511:2432], now have 1 active actors on pipe
2025-10-09T21:46:54.845885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341788459829329:7207]
2025-10-09T21:46:54.845895Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:54.845951Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:54.845915Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 808
2025-10-09T21:46:54.845987Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 808
2025-10-09T21:46:54.846924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 809 partition: 0 MaxSeqNo: 25 sessionId: src_id|69c45706-f90cb2ce-cc796b72-318aba87_0
2025-10-09T21:46:54.847351Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9c53739-a56b3bc9-b125886c-7f26c2ca_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|69c45706-f90cb2ce-cc796b72-318aba87_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:54.849969Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|69c45706-f90cb2ce-cc796b72-318aba87_0] Write session: try to update token
2025-10-09T21:46:54.849990Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|69c45706-f90cb2ce-cc796b72-318aba87_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:54.846299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:54.846325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:54.846360Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|69c45706-f90cb2ce-cc796b72-318aba87_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:54.846373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:54.846389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:54.846391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:54.846403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:54.846415Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:54.846434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:54.846763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:54.846767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:54.846783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:54.862825Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:54.862864Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:54.862867Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:54.862870Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:54.862880Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341788459829329:7207]
2025-10-09T21:46:54.862885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341788459829329:7207]
2025-10-09T21:46:54.862890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341788459829329:7207]
2025-10-09T21:46:54.862937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:54.862945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:54.862948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:54.863080Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:54.866017Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:54.866049Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:54.866053Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:54.879156Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 809 sessionId: src_id|69c45706-f90cb2ce-cc796b72-318aba87_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:54.881579Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:54.911542Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341788821647655:7899]
2025-10-09T21:46:54.912165Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 808, pipe: [1:7559341788821647655:7899]
2025-10-09T21:46:55.074958Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:55.075363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:55.075376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:55.075399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341788821647655:7899], now have 1 active actors on pipe
2025-10-09T21:46:55.075736Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:55.076249Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341788821647655:7899]
2025-10-09T21:46:55.076280Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:55.076317Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:55.076402Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 809 reason: status is not ok: tablet will be restarted right now sessionId: src_id|69c45706-f90cb2ce-cc796b72-318aba87_0
2025-10-09T21:46:55.076619Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 809 sessionId: src_id|69c45706-f90cb2ce-cc796b72-318aba87_0 is DEAD
2025-10-09T21:46:55.078361Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69c45706-f90cb2ce-cc796b72-318aba87_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:55.078375Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69c45706-f90cb2ce-cc796b72-318aba87_0] Write session will restart in 0.008942s
2025-10-09T21:46:55.078420Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69c45706-f90cb2ce-cc796b72-318aba87_0] Write session: Do CDS request
2025-10-09T21:46:55.078429Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69c45706-f90cb2ce-cc796b72-318aba87_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:55.088881Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|69c45706-f90cb2ce-cc796b72-318aba87_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:55.090510Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:55.090531Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 810
2025-10-09T21:46:55.090749Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 810 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:55.090806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 810 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:55.090813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=810 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:55.090829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:55.091384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 810 sessionId:  describe result for acl check
2025-10-09T21:46:55.091438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:55.091441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:55.091442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:55.091449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341793116615037:7947] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:55.092178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341793116615037:7947] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:55.095223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:55.096682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341793116615037:7947] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:55.097461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:55.097733Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:55.097747Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:55.097764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:55.097848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:55.097889Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341792754796763:7209]
2025-10-09T21:46:55.098551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:55.099125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:55.099207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:55.099555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:55.099630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:55.099632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:55.099633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:55.099634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:55.099635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:55.099637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:55.099638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.099640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.099641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.099647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:55.099658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:55.099662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:55.099666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:55.099670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:55.099674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:55.099677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.099680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.099683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.099691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:55.099699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:55.099705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:55.099710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:55.099715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:55.099721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:55.099724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:55.099729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:55.099733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:55.099739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:55.099742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:55.099748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:55.099753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.099759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.099764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.099769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.099773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.099779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.099808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:55.101333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341793116615037:7947] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:55.101354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341793116615037:7947] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:55.101358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341793116615037:7947] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:55.101369Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 810 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:55.109627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:55.111988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:55.112012Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:55.112016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:55.112021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:55.112033Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 809 [2:7559341792754796763:7209]
2025-10-09T21:46:55.112053Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:55.112089Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:55.112093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:55.112097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:55.112100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:55.112121Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:55.112130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:55.112138Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:55.112217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.112231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.112233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.112236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:55.112324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:55.112334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.112349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.112361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.112388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:55.112410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341792754796758:7209]
2025-10-09T21:46:55.112413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341792754796758:7209]
2025-10-09T21:46:55.112416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341792754796758:7209]
2025-10-09T21:46:55.112426Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:55.112443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:55.112457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:55.112461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:55.112510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341792754796758:7209]
2025-10-09T21:46:55.112513Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:55.112516Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:55.112520Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:55.112522Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:55.112530Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:55.112664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341793116614956:2432], now have 1 active actors on pipe
2025-10-09T21:46:55.112669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341793116615092:7947], now have 1 active actors on pipe
2025-10-09T21:46:55.112966Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 809
2025-10-09T21:46:55.112966Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 809
2025-10-09T21:46:55.113391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:55.113412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:55.113455Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b8400b13-11dc4191-b551bcbb-1941c492_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:55.113470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:55.113485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:55.113487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:55.113500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:55.113514Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:55.113546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:55.113859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:55.113886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:55.113913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:55.114088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 810 partition: 0 MaxSeqNo: 25 sessionId: src_id|b8400b13-11dc4191-b551bcbb-1941c492_0
2025-10-09T21:46:55.114510Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69c45706-f90cb2ce-cc796b72-318aba87_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b8400b13-11dc4191-b551bcbb-1941c492_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:55.124123Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b8400b13-11dc4191-b551bcbb-1941c492_0] Write session: try to update token
2025-10-09T21:46:55.124157Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b8400b13-11dc4191-b551bcbb-1941c492_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:55.131556Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:55.131591Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:55.131594Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:55.131597Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:55.131613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341792754796758:7209]
2025-10-09T21:46:55.131619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341792754796758:7209]
2025-10-09T21:46:55.131624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341792754796758:7209]
2025-10-09T21:46:55.131686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:55.131694Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:55.131704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.131707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:55.131709Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:55.131714Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:55.132050Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:55.155354Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 810 sessionId: src_id|b8400b13-11dc4191-b551bcbb-1941c492_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:55.157949Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:55.176156Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341793116615106:7899]
2025-10-09T21:46:55.176649Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 809, pipe: [1:7559341793116615106:7899]
2025-10-09T21:46:55.345603Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:55.350125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:55.350157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:55.350184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341793116615106:7899], now have 1 active actors on pipe
2025-10-09T21:46:55.354254Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:55.354284Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:55.354314Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:55.354318Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:55.354359Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341793116615106:7899]
2025-10-09T21:46:55.354388Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 810 reason: error at write: Disconnected sessionId: src_id|b8400b13-11dc4191-b551bcbb-1941c492_0
2025-10-09T21:46:55.354741Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8400b13-11dc4191-b551bcbb-1941c492_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:55.354753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8400b13-11dc4191-b551bcbb-1941c492_0] Write session will restart in 0.008239s
2025-10-09T21:46:55.354778Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8400b13-11dc4191-b551bcbb-1941c492_0] Write session: Do CDS request
2025-10-09T21:46:55.354786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8400b13-11dc4191-b551bcbb-1941c492_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:55.354599Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 810 sessionId: src_id|b8400b13-11dc4191-b551bcbb-1941c492_0 is DEAD
2025-10-09T21:46:55.364612Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b8400b13-11dc4191-b551bcbb-1941c492_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:55.364756Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:55.364773Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 811
2025-10-09T21:46:55.364941Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 811 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:55.364978Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 811 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:55.364984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=811 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:55.364987Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:55.365476Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 811 sessionId:  describe result for acl check
2025-10-09T21:46:55.365517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:55.365525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:55.365527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:55.365534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341793116615133:7952] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:55.366189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341793116615133:7952] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:55.371329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341793116615133:7952] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:55.374822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341793116615133:7952] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:55.374839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341793116615133:7952] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:55.374843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341793116615133:7952] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:55.374853Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 811 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:55.377448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:55.380157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:55.380398Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:55.380403Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:55.380416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:55.380493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:55.380534Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341792754796899:7211]
2025-10-09T21:46:55.381164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:55.381702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:55.381754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:55.382094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:55.382161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:55.382162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:55.382163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:55.382164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:55.382165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:55.382166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:55.382167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.382169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.382170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.382175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:55.382186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:55.382190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:55.382193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:55.382197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:55.382200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:55.382204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.382207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.382210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.382219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:55.382226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:55.382233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:55.382238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:55.382243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:55.382248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:55.382252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:55.382264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:55.382268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:55.382285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:55.382290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:55.382296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:55.382302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.382307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.382311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.382316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.382321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.382328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.382355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:55.397770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:55.399053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:55.399079Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:55.399084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:55.399089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:55.399098Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 810 [2:7559341792754796899:7211]
2025-10-09T21:46:55.399115Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:55.399137Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:55.399140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:55.399144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:55.399147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:55.399168Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:55.399181Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:55.399259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:55.399329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.399332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.399334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.399338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:55.399446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:55.399449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.399463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.399485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.399505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:55.399528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341792754796894:7211]
2025-10-09T21:46:55.399532Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341792754796894:7211]
2025-10-09T21:46:55.399535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341792754796894:7211]
2025-10-09T21:46:55.399549Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:55.399568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:55.400078Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 810
2025-10-09T21:46:55.399582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:55.400099Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 810
2025-10-09T21:46:55.399586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:55.399632Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341792754796894:7211]
2025-10-09T21:46:55.399635Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:55.399738Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:55.399742Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:55.399744Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:55.399750Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:55.400225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341793116615109:2432], now have 1 active actors on pipe
2025-10-09T21:46:55.400236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341793116615237:7952], now have 1 active actors on pipe
2025-10-09T21:46:55.400428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:55.400442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:55.400479Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c4786ef4-6880f974-eb3a4712-63195a7d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:55.400496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:55.400501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:55.400504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:55.400516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:55.400529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:55.400550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:55.400869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:55.400877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:55.400894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:55.400999Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 811 partition: 0 MaxSeqNo: 25 sessionId: src_id|c4786ef4-6880f974-eb3a4712-63195a7d_0
2025-10-09T21:46:55.401407Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8400b13-11dc4191-b551bcbb-1941c492_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c4786ef4-6880f974-eb3a4712-63195a7d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:55.404072Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4786ef4-6880f974-eb3a4712-63195a7d_0] Write session: try to update token
2025-10-09T21:46:55.404094Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4786ef4-6880f974-eb3a4712-63195a7d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:55.415008Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:55.415029Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:55.415039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341792754796894:7211]
2025-10-09T21:46:55.415089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:55.415091Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:55.415333Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:55.415480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:55.415631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:55.415771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:55.415963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:55.416007Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:55.421964Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 811 sessionId: src_id|c4786ef4-6880f974-eb3a4712-63195a7d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:55.424497Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:55.424747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:55.424774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:55.424829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:55.424958Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:55.433190Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:55.433211Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:55.433214Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:55.433234Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:55.433247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341792754796894:7211]
2025-10-09T21:46:55.433252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341792754796894:7211]
2025-10-09T21:46:55.433257Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341792754796894:7211]
2025-10-09T21:46:55.433308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:55.433323Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:55.433330Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:55.433334Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:55.433358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.433361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:55.433547Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:55.458486Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341793116615252:7899]
2025-10-09T21:46:55.459027Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 810, pipe: [1:7559341793116615252:7899]
2025-10-09T21:46:55.592417Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:55.600645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:55.600683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:55.600851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:55.600923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:55.600989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:55.601047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:55.601095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:55.601100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:55.601165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:55.601233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:55.601296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:55.601362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:55.601405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:55.601408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:55.601516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:55.601575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:55.601629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:55.601686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:55.601722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:55.601725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:55.601790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:55.601883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:55.601979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:55.602155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:55.602291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:55.602298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:55.602363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:55.602428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:55.602523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:55.602594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:55.604407Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:55.604421Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:55.604464Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 811 reason: error at write: Disconnected sessionId: src_id|c4786ef4-6880f974-eb3a4712-63195a7d_0
2025-10-09T21:46:55.604531Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:55.604579Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341793116615252:7899]
2025-10-09T21:46:55.604684Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 811 sessionId: src_id|c4786ef4-6880f974-eb3a4712-63195a7d_0 is DEAD
2025-10-09T21:46:55.604808Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4786ef4-6880f974-eb3a4712-63195a7d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:55.604820Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4786ef4-6880f974-eb3a4712-63195a7d_0] Write session will restart in 0.005057s
2025-10-09T21:46:55.604845Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4786ef4-6880f974-eb3a4712-63195a7d_0] Write session: Do CDS request
2025-10-09T21:46:55.604851Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4786ef4-6880f974-eb3a4712-63195a7d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:55.602640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:55.602650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:55.602721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:55.602777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:55.602834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:55.602894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:55.602934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:55.602937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:55.602998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:55.603049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:55.603108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:55.603162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:55.603215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:55.603217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:55.603281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:55.603333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:55.603388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:55.603444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:55.603481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:55.603484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:55.603547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:55.603600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:55.603652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:55.603707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:55.603745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:55.603747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:55.603806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:55.603865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:55.603919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:55.603974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:55.604014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:55.604017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:55.604089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341793116615252:7899], now have 1 active actors on pipe
2025-10-09T21:46:55.605124Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:55.611544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4786ef4-6880f974-eb3a4712-63195a7d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:55.611702Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:55.611724Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 812
2025-10-09T21:46:55.611841Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 812 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:55.611885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 812 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:55.611897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=812 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:55.611902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:55.612606Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 812 sessionId:  describe result for acl check
2025-10-09T21:46:55.612643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:55.612663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:55.612665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:55.612672Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341793116615295:7956] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:55.613337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341793116615295:7956] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:55.617002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341793116615295:7956] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:55.619048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:55.620752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:55.621012Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:55.621025Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:55.621038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:55.621013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341793116615295:7956] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:55.621029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341793116615295:7956] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:55.621032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341793116615295:7956] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:55.621041Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 812 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:55.621109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:55.621156Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341792754797046:7215]
2025-10-09T21:46:55.621701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:55.622136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:55.622197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:55.622532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:55.622584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:55.622589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:55.622590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:55.622592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:55.622593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:55.622594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:55.622596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.622597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.622598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.622612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:55.622623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:55.622632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:55.622635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:55.622638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:55.622641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:55.622644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.622648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.622652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.622659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:55.622668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:55.622674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:55.622679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:55.622683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:55.622689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:55.622693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:55.622697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:55.622701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:55.622717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:55.622721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:55.622730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:55.622735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.622739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.622743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.622749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.622753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.622757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.622784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:55.629444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:55.630596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:55.630611Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:55.630614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:55.630618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:55.630625Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 811 [2:7559341792754797046:7215]
2025-10-09T21:46:55.630640Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:55.630659Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:55.630663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:55.630666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:55.630669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:55.630690Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:55.630696Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:55.630714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:55.630783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.630793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.630794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.630798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:55.630874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:55.630877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.630890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.630899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.630915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:55.630938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341792754797037:7215]
2025-10-09T21:46:55.630941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341792754797037:7215]
2025-10-09T21:46:55.630943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341792754797037:7215]
2025-10-09T21:46:55.630953Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:55.630968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:55.630969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:55.630973Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:55.630975Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:55.630982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:55.630986Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:55.631027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341792754797037:7215]
2025-10-09T21:46:55.631031Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:55.631032Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:55.631042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341793116615387:7956], now have 1 active actors on pipe
2025-10-09T21:46:55.631045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341793116615255:2432], now have 1 active actors on pipe
2025-10-09T21:46:55.631193Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 811
2025-10-09T21:46:55.631247Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 811
2025-10-09T21:46:55.631311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:55.631325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:55.631361Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3275b071-28a99069-48bec11a-e01c7420_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:55.631377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:55.631381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:55.631384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:55.631411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:55.631430Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:55.631459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:55.631695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:55.631706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:55.631727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:55.631859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 812 partition: 0 MaxSeqNo: 25 sessionId: src_id|3275b071-28a99069-48bec11a-e01c7420_0
2025-10-09T21:46:55.632310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4786ef4-6880f974-eb3a4712-63195a7d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3275b071-28a99069-48bec11a-e01c7420_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:55.635206Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3275b071-28a99069-48bec11a-e01c7420_0] Write session: try to update token
2025-10-09T21:46:55.635229Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3275b071-28a99069-48bec11a-e01c7420_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:55.646446Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:55.646463Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:55.646466Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:55.646468Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:55.646491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341792754797037:7215]
2025-10-09T21:46:55.646496Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341792754797037:7215]
2025-10-09T21:46:55.646500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341792754797037:7215]
2025-10-09T21:46:55.646546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:55.646557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.646559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:55.646701Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:55.650325Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:55.650340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:55.650344Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:55.651938Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 812 sessionId: src_id|3275b071-28a99069-48bec11a-e01c7420_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:55.654621Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:55.705366Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341793116615422:7899]
2025-10-09T21:46:55.706367Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 811, pipe: [1:7559341793116615422:7899]
2025-10-09T21:46:55.811837Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:55.812219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:55.812236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:55.812286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341793116615422:7899], now have 1 active actors on pipe
2025-10-09T21:46:55.812884Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:55.813318Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3275b071-28a99069-48bec11a-e01c7420_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:55.813336Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3275b071-28a99069-48bec11a-e01c7420_0] Write session will restart in 0.008377s
2025-10-09T21:46:55.812901Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:55.812959Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 812 reason: error at write: Disconnected sessionId: src_id|3275b071-28a99069-48bec11a-e01c7420_0
2025-10-09T21:46:55.813038Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:55.813371Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3275b071-28a99069-48bec11a-e01c7420_0] Write session: Do CDS request
2025-10-09T21:46:55.813377Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3275b071-28a99069-48bec11a-e01c7420_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:55.813079Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341793116615422:7899]
2025-10-09T21:46:55.813148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 812 sessionId: src_id|3275b071-28a99069-48bec11a-e01c7420_0 is DEAD
2025-10-09T21:46:55.812895Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:55.822725Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3275b071-28a99069-48bec11a-e01c7420_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:55.823450Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:55.823467Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 813
2025-10-09T21:46:55.823602Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 813 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:55.823637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 813 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:55.823643Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=813 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:55.823646Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:55.824081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 813 sessionId:  describe result for acl check
2025-10-09T21:46:55.824117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:55.824121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:55.824123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:55.824129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341793116615436:7968] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:55.824866Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341793116615436:7968] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:55.827254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341793116615436:7968] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:55.830830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341793116615436:7968] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:55.830843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341793116615436:7968] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:55.830847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341793116615436:7968] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:55.830857Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 813 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:55.842944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:55.844469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:55.844672Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:55.844676Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:55.844689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:55.844740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:55.844783Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341792754797213:7229]
2025-10-09T21:46:55.845392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:55.845734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:55.845772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:55.846133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:55.846184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:55.846185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:55.846187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:55.846188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:55.846189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:55.846191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:55.846192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.846194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.846195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.846200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:55.846213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:55.846218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:55.846222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:55.846226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:55.846231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:55.846235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.846239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.846243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.846251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:55.846260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:55.846266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:55.846272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:55.846277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:55.846283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:55.846288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:55.846293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:55.846298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:55.846304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:55.846309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:55.846315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:55.846321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.846328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.846343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.846349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.846354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.846359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.846390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:55.854978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:55.856496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:55.856505Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:55.856508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:55.856513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:55.856521Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 812 [2:7559341792754797213:7229]
2025-10-09T21:46:55.856536Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:55.856559Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:55.856562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:55.856565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:55.856568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:55.856587Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:55.856593Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:55.856602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:55.856679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.856681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.856683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.856687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:55.856774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:55.856777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.856790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:55.856801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:55.856816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:55.856840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341792754797208:7229]
2025-10-09T21:46:55.856842Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341792754797208:7229]
2025-10-09T21:46:55.856845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341792754797208:7229]
2025-10-09T21:46:55.856862Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:55.856882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:55.856897Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:55.856901Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:55.856944Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341792754797208:7229]
2025-10-09T21:46:55.856948Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:55.859642Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:55.859654Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:55.859657Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:55.859665Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:55.860620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341793116615499:7968], now have 1 active actors on pipe
2025-10-09T21:46:55.860633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341793116615426:2432], now have 1 active actors on pipe
2025-10-09T21:46:55.862025Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 812
2025-10-09T21:46:55.862078Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 812
2025-10-09T21:46:55.862825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:55.862850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:55.862891Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2b06cf8a-843447b3-30017ee7-588a1e40_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:55.862909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:55.862915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:55.862929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:55.862944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:55.862966Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:55.863004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:55.863507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:55.863519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:55.863541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:55.863682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 813 partition: 0 MaxSeqNo: 25 sessionId: src_id|2b06cf8a-843447b3-30017ee7-588a1e40_0
2025-10-09T21:46:55.864085Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3275b071-28a99069-48bec11a-e01c7420_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2b06cf8a-843447b3-30017ee7-588a1e40_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:55.866706Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b06cf8a-843447b3-30017ee7-588a1e40_0] Write session: try to update token
2025-10-09T21:46:55.866726Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b06cf8a-843447b3-30017ee7-588a1e40_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:55.882731Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:55.882747Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:55.882754Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:55.882756Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:55.882766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341792754797208:7229]
2025-10-09T21:46:55.882770Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341792754797208:7229]
2025-10-09T21:46:55.882774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341792754797208:7229]
2025-10-09T21:46:55.882821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:55.882827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:55.882829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:55.882822Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:55.882835Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:55.882838Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:55.882964Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:55.895911Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 813 sessionId: src_id|2b06cf8a-843447b3-30017ee7-588a1e40_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:55.898564Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:55.915779Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341793116615576:7899]
2025-10-09T21:46:55.916555Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 812, pipe: [1:7559341793116615576:7899]
2025-10-09T21:46:56.056806Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:56.057119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:56.057136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:56.057160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341793116615576:7899], now have 1 active actors on pipe
2025-10-09T21:46:56.057416Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341793116615576:7899]
2025-10-09T21:46:56.057453Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:56.057456Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:56.057517Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 813 reason: error at write: Disconnected sessionId: src_id|2b06cf8a-843447b3-30017ee7-588a1e40_0
2025-10-09T21:46:56.057597Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:56.057715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 813 sessionId: src_id|2b06cf8a-843447b3-30017ee7-588a1e40_0 is DEAD
2025-10-09T21:46:56.057777Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:56.058247Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b06cf8a-843447b3-30017ee7-588a1e40_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:56.058259Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b06cf8a-843447b3-30017ee7-588a1e40_0] Write session will restart in 0.009158s
2025-10-09T21:46:56.058288Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b06cf8a-843447b3-30017ee7-588a1e40_0] Write session: Do CDS request
2025-10-09T21:46:56.058295Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b06cf8a-843447b3-30017ee7-588a1e40_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:56.068602Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b06cf8a-843447b3-30017ee7-588a1e40_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:56.068988Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:56.069015Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 814
2025-10-09T21:46:56.069196Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 814 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:56.069230Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 814 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:56.069235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=814 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:56.069239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:56.070010Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 814 sessionId:  describe result for acl check
2025-10-09T21:46:56.070071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:56.070073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:56.070074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:56.070081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341797411582932:7972] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:56.070730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341797411582932:7972] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:56.074554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341797411582932:7972] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:56.078191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:56.080261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:56.080491Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:56.080502Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:56.080514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:56.080569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:56.080614Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341797049764646:7231]
2025-10-09T21:46:56.081235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:56.081651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:56.081711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:56.078257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341797411582932:7972] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:56.078275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341797411582932:7972] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:56.078279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341797411582932:7972] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:56.078290Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 814 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:56.082111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:56.082186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:56.082193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:56.082194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:56.082195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:56.082197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:56.082198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:56.082199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.082200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.082202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.082208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:56.082219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:56.082224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:56.082228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:56.082232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:56.082236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:56.082240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.082244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.082248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.082257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:56.082266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:56.082273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:56.082279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:56.082285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:56.082291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:56.082296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:56.082302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:56.082306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:56.082311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:56.082315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:56.082320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:56.082325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.082330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.082334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.082338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.082343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.082348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.082389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:56.090828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:56.092055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:56.092074Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:56.092077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:56.092083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:56.092091Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 813 [2:7559341797049764646:7231]
2025-10-09T21:46:56.092110Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:56.092133Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:56.092141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:56.092145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:56.092148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:56.092168Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:56.092180Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:56.092189Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:56.092256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.092266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.092268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.092271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:56.092355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:56.092363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.092377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.092389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.092406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:56.092427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341797049764640:7231]
2025-10-09T21:46:56.092430Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341797049764640:7231]
2025-10-09T21:46:56.092433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341797049764640:7231]
2025-10-09T21:46:56.092449Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:56.092465Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:56.092481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:56.092484Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:56.092526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341797049764640:7231]
2025-10-09T21:46:56.092530Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:56.092659Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:56.092661Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:56.092664Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:56.092671Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:56.092923Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 813
2025-10-09T21:46:56.092948Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 813
2025-10-09T21:46:56.093078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341797411583010:7972], now have 1 active actors on pipe
2025-10-09T21:46:56.093088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341797411582880:2432], now have 1 active actors on pipe
2025-10-09T21:46:56.093107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:56.093113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:56.093143Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1828788e-36d2d70f-7d47d35a-f07780a2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:56.093158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:56.093163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:56.093166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:56.093177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:56.093190Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:56.093208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:56.093647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:56.093671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:56.093693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:56.094149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 814 partition: 0 MaxSeqNo: 25 sessionId: src_id|1828788e-36d2d70f-7d47d35a-f07780a2_0
2025-10-09T21:46:56.094646Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b06cf8a-843447b3-30017ee7-588a1e40_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1828788e-36d2d70f-7d47d35a-f07780a2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:56.097251Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1828788e-36d2d70f-7d47d35a-f07780a2_0] Write session: try to update token
2025-10-09T21:46:56.097271Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1828788e-36d2d70f-7d47d35a-f07780a2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:56.111303Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:56.111319Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:56.111322Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:56.111324Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:56.111334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341797049764640:7231]
2025-10-09T21:46:56.111339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341797049764640:7231]
2025-10-09T21:46:56.111344Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341797049764640:7231]
2025-10-09T21:46:56.111389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:56.111398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.111401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:56.111638Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:56.111707Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:56.111718Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:56.111723Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:56.120265Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 814 sessionId: src_id|1828788e-36d2d70f-7d47d35a-f07780a2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:56.123031Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:56.157677Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341797411583029:7899]
2025-10-09T21:46:56.158114Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 813, pipe: [1:7559341797411583029:7899]
2025-10-09T21:46:56.289602Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:56.290702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:56.290722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:56.290747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341797411583029:7899], now have 1 active actors on pipe
2025-10-09T21:46:56.291384Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:56.291482Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:56.291403Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:56.291405Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341797411583029:7899]
2025-10-09T21:46:56.291439Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:56.291489Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 814 reason: error at write: Disconnected sessionId: src_id|1828788e-36d2d70f-7d47d35a-f07780a2_0
2025-10-09T21:46:56.291717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 814 sessionId: src_id|1828788e-36d2d70f-7d47d35a-f07780a2_0 is DEAD
2025-10-09T21:46:56.292700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1828788e-36d2d70f-7d47d35a-f07780a2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:56.292714Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1828788e-36d2d70f-7d47d35a-f07780a2_0] Write session will restart in 0.009947s
2025-10-09T21:46:56.292758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1828788e-36d2d70f-7d47d35a-f07780a2_0] Write session: Do CDS request
2025-10-09T21:46:56.292765Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1828788e-36d2d70f-7d47d35a-f07780a2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:56.303672Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1828788e-36d2d70f-7d47d35a-f07780a2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:56.303813Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:56.303831Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 815
2025-10-09T21:46:56.304061Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 815 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:56.304101Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 815 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:56.304106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=815 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:56.304110Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:56.304793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 815 sessionId:  describe result for acl check
2025-10-09T21:46:56.304838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:56.304841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:56.304844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:56.304851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341797411583091:7977] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:56.305569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341797411583091:7977] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:56.309342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341797411583091:7977] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:56.309389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:56.311171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:56.311437Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:56.311447Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:56.311468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:56.311534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:56.311573Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341797049764785:7233]
2025-10-09T21:46:56.312175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:56.312583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:56.312637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:56.312801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341797411583091:7977] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:56.312817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341797411583091:7977] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:56.312821Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341797411583091:7977] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:56.312831Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 815 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:56.312981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:56.313035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:56.313041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:56.313043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:56.313044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:56.313045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:56.313046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:56.313047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.313048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.313049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.313054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:56.313064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:56.313067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:56.313071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:56.313074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:56.313078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:56.313081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.313084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.313088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.313095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:56.313103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:56.313108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:56.313113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:56.313118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:56.313122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:56.313126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:56.313131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:56.313135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:56.313139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:56.313143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:56.313148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:56.313153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.313158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.313162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.313166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.313170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.313174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.313199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:56.320860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:56.322555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:56.322576Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:56.322579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:56.322583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:56.322592Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 814 [2:7559341797049764785:7233]
2025-10-09T21:46:56.322608Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:56.322640Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:56.322648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:56.322652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:56.322655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:56.322675Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:56.322681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:56.322689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:56.322784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.322791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.322793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.322797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:56.322876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:56.322886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.322899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.322916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.322933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:56.322957Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341797049764780:7233]
2025-10-09T21:46:56.322959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341797049764780:7233]
2025-10-09T21:46:56.322962Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341797049764780:7233]
2025-10-09T21:46:56.322971Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:56.322983Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:56.322989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:56.322992Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:56.322997Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:56.323006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:56.323010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:56.323022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341797411583173:7977], now have 1 active actors on pipe
2025-10-09T21:46:56.323025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341797411583035:2432], now have 1 active actors on pipe
2025-10-09T21:46:56.323065Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341797049764780:7233]
2025-10-09T21:46:56.323081Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:56.323087Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 814
2025-10-09T21:46:56.323091Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 814
2025-10-09T21:46:56.323115Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:56.323273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:56.323290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:56.323334Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d297d474-f5333ee6-e14d5e86-427e2237_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:56.323348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:56.323357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:56.323360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:56.323372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:56.323397Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:56.323421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:56.323747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:56.323758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:56.323778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:56.323876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 815 partition: 0 MaxSeqNo: 25 sessionId: src_id|d297d474-f5333ee6-e14d5e86-427e2237_0
2025-10-09T21:46:56.324329Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1828788e-36d2d70f-7d47d35a-f07780a2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d297d474-f5333ee6-e14d5e86-427e2237_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:56.329130Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d297d474-f5333ee6-e14d5e86-427e2237_0] Write session: try to update token
2025-10-09T21:46:56.329160Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d297d474-f5333ee6-e14d5e86-427e2237_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:56.350362Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:56.350377Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:56.350380Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:56.350383Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:56.350394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341797049764780:7233]
2025-10-09T21:46:56.350400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341797049764780:7233]
2025-10-09T21:46:56.350408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341797049764780:7233]
2025-10-09T21:46:56.350458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:56.350466Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:56.350468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.350471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:56.350472Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:56.350493Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:56.350623Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:56.359135Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 815 sessionId: src_id|d297d474-f5333ee6-e14d5e86-427e2237_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:56.361779Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:56.391659Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341797411583183:7899]
2025-10-09T21:46:56.391995Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 814, pipe: [1:7559341797411583183:7899]
2025-10-09T21:46:56.522490Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:56.523590Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:56.523606Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:56.522872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:56.522889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:56.522944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341797411583183:7899], now have 1 active actors on pipe
2025-10-09T21:46:56.523347Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:56.523658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 815 reason: error at write: Disconnected sessionId: src_id|d297d474-f5333ee6-e14d5e86-427e2237_0
2025-10-09T21:46:56.523862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 815 sessionId: src_id|d297d474-f5333ee6-e14d5e86-427e2237_0 is DEAD
2025-10-09T21:46:56.524013Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341797411583183:7899]
2025-10-09T21:46:56.524033Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:56.524056Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d297d474-f5333ee6-e14d5e86-427e2237_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:56.524069Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d297d474-f5333ee6-e14d5e86-427e2237_0] Write session will restart in 0.008281s
2025-10-09T21:46:56.524097Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d297d474-f5333ee6-e14d5e86-427e2237_0] Write session: Do CDS request
2025-10-09T21:46:56.524104Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d297d474-f5333ee6-e14d5e86-427e2237_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:56.533964Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d297d474-f5333ee6-e14d5e86-427e2237_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:56.534245Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:56.534264Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 816
2025-10-09T21:46:56.534441Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 816 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:56.534482Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 816 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:56.534488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=816 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:56.534493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:56.535254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 816 sessionId:  describe result for acl check
2025-10-09T21:46:56.535297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:56.535299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:56.535301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:56.535308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341797411583199:7981] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:56.535935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341797411583199:7981] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:56.540056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341797411583199:7981] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:56.554712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:56.556534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:56.556787Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:56.556791Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:56.556804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:56.556864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:56.556927Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341797049764924:7235]
2025-10-09T21:46:56.557613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:56.558153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:56.558200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:56.558529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:56.558628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:56.558630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:56.558631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:56.558633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:56.558634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:56.558635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:56.558637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.558639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.558640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.558648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:56.558661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:56.558666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:56.558671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:56.558675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:56.558681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:56.558685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.558689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.558693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.558704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:56.558714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:56.558720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:56.558727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:56.558736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:56.558742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:56.558747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:56.558753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:56.558759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:56.558764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:56.558770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:56.558776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:56.558783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.558789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.558794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.558800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.558805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.558811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.558844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:56.562551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341797411583199:7981] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:56.562568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341797411583199:7981] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:56.562573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341797411583199:7981] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:56.562582Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 816 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:56.570720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:56.572858Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 815
2025-10-09T21:46:56.572903Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 815
2025-10-09T21:46:56.571817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:56.571837Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:56.571840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:56.571846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:56.571854Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 815 [2:7559341797049764924:7235]
2025-10-09T21:46:56.571870Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:56.571892Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:56.571895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:56.571899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:56.571902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:56.571922Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:56.571929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:56.571936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:56.572009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.572012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.572017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.572024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:56.572107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:56.572111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.572124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.572136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.572151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:56.572173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341797049764919:7235]
2025-10-09T21:46:56.572180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341797049764919:7235]
2025-10-09T21:46:56.572182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341797049764919:7235]
2025-10-09T21:46:56.572199Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:56.572216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:56.572233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:56.572237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:56.572280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341797049764919:7235]
2025-10-09T21:46:56.572283Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:56.572427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:56.572431Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:56.572433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:56.572439Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:56.572966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341797411583325:7981], now have 1 active actors on pipe
2025-10-09T21:46:56.572970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341797411583186:2432], now have 1 active actors on pipe
2025-10-09T21:46:56.577169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:56.577184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:56.577230Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|61272018-4ade1566-1b32faa3-3661d14a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:56.577242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:56.577247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:56.577251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:56.577266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:56.577285Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:56.577306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:56.578208Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d297d474-f5333ee6-e14d5e86-427e2237_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|61272018-4ade1566-1b32faa3-3661d14a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:56.577605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:56.577613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:56.577629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:56.577763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 816 partition: 0 MaxSeqNo: 25 sessionId: src_id|61272018-4ade1566-1b32faa3-3661d14a_0
2025-10-09T21:46:56.580932Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|61272018-4ade1566-1b32faa3-3661d14a_0] Write session: try to update token
2025-10-09T21:46:56.580949Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|61272018-4ade1566-1b32faa3-3661d14a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:56.599170Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:56.599187Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:56.599190Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:56.599192Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:56.599202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341797049764919:7235]
2025-10-09T21:46:56.599212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341797049764919:7235]
2025-10-09T21:46:56.599216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341797049764919:7235]
2025-10-09T21:46:56.599264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:56.599270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.599273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:56.599414Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:56.602679Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:56.602693Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:56.602697Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:56.605589Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 816 sessionId: src_id|61272018-4ade1566-1b32faa3-3661d14a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:56.607998Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:56.624109Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341797411583333:7899]
2025-10-09T21:46:56.624971Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 815, pipe: [1:7559341797411583333:7899]
2025-10-09T21:46:56.785170Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:56.785408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:56.785417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:56.785457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341797411583333:7899], now have 1 active actors on pipe
2025-10-09T21:46:56.785737Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:56.786142Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:56.786161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:56.786248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 816 reason: error at write: Disconnected sessionId: src_id|61272018-4ade1566-1b32faa3-3661d14a_0
2025-10-09T21:46:56.786461Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 816 sessionId: src_id|61272018-4ade1566-1b32faa3-3661d14a_0 is DEAD
2025-10-09T21:46:56.786620Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:56.786657Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341797411583333:7899]
2025-10-09T21:46:56.790447Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61272018-4ade1566-1b32faa3-3661d14a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:56.790456Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61272018-4ade1566-1b32faa3-3661d14a_0] Write session will restart in 0.007968s
2025-10-09T21:46:56.790484Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61272018-4ade1566-1b32faa3-3661d14a_0] Write session: Do CDS request
2025-10-09T21:46:56.790492Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61272018-4ade1566-1b32faa3-3661d14a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:56.799683Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|61272018-4ade1566-1b32faa3-3661d14a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:56.805065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:56.803311Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:56.803331Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 817
2025-10-09T21:46:56.806445Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 817 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:56.806500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 817 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:56.806506Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=817 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:56.806509Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:56.806937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 817 sessionId:  describe result for acl check
2025-10-09T21:46:56.806977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:56.806989Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:56.806991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:56.806997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341797411583469:7995] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:56.807594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341797411583469:7995] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:56.808658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:56.808930Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:56.808935Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:56.808948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:56.809015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:56.809072Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341797049765107:7249]
2025-10-09T21:46:56.809671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:56.810177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:56.810234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:56.810591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:56.810647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:56.810648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:56.810649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:56.810651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:56.810652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:56.810654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:56.810655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.810656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.810658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.810665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:56.810676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:56.810680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:56.810684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:56.810688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:56.810693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:56.810696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.810700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.810703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.810711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:56.810719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:56.810725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:56.810731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:56.810736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:56.810742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:56.810747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:56.810959Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341797411583469:7995] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:56.810752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:56.810760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:56.810766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:56.810772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:56.810778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:56.810783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.810788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.810793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.810798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.810805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.810809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.810840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:56.818590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341797411583469:7995] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:56.818612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341797411583469:7995] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:56.818621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341797411583469:7995] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:56.818644Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 817 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:56.818956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:56.820283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:56.820296Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:56.820299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:56.820304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:56.820312Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 816 [2:7559341797049765107:7249]
2025-10-09T21:46:56.820329Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:56.820349Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:56.820352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:56.820356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:56.820359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:56.820379Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:56.820385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:56.820400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:56.820468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.820477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.820479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.820481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:56.820558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:56.820565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.820580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:56.820592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:56.820607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:56.820639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341797411583505:7995], now have 1 active actors on pipe
2025-10-09T21:46:56.820648Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341797049765102:7249]
2025-10-09T21:46:56.820650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341797049765102:7249]
2025-10-09T21:46:56.820652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341797049765102:7249]
2025-10-09T21:46:56.820665Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:56.820678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341797411583357:2432], now have 1 active actors on pipe
2025-10-09T21:46:56.820686Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:56.820688Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:56.820693Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:56.820695Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:56.820703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:56.820706Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:56.820750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341797049765102:7249]
2025-10-09T21:46:56.820759Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:56.820745Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 816
2025-10-09T21:46:56.820753Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 816
2025-10-09T21:46:56.820810Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:56.821071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:56.821085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:56.821135Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|52a57518-50f690a5-3dc6e734-b614dd41_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:56.821151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:56.821155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:56.821158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:56.821170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:56.821183Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:56.821199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:56.821421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:56.821430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:56.821447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:56.821616Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 817 partition: 0 MaxSeqNo: 25 sessionId: src_id|52a57518-50f690a5-3dc6e734-b614dd41_0
2025-10-09T21:46:56.823821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61272018-4ade1566-1b32faa3-3661d14a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|52a57518-50f690a5-3dc6e734-b614dd41_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:56.826634Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52a57518-50f690a5-3dc6e734-b614dd41_0] Write session: try to update token
2025-10-09T21:46:56.826656Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52a57518-50f690a5-3dc6e734-b614dd41_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:56.845219Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:56.845235Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:56.845238Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:56.845240Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:56.845252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341797049765102:7249]
2025-10-09T21:46:56.845257Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341797049765102:7249]
2025-10-09T21:46:56.845261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341797049765102:7249]
2025-10-09T21:46:56.845314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:56.845321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:56.845324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:56.845335Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:56.845343Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:56.845348Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:56.845551Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:56.856681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 817 sessionId: src_id|52a57518-50f690a5-3dc6e734-b614dd41_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:56.859520Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:56.887268Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341797411583513:7899]
2025-10-09T21:46:56.888188Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 816, pipe: [1:7559341797411583513:7899]
2025-10-09T21:46:57.013876Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:57.014305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:57.014323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:57.014347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341797411583513:7899], now have 1 active actors on pipe
2025-10-09T21:46:57.014677Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:57.017159Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:57.017179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:57.017262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 817 reason: error at write: Disconnected sessionId: src_id|52a57518-50f690a5-3dc6e734-b614dd41_0
2025-10-09T21:46:57.017478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 817 sessionId: src_id|52a57518-50f690a5-3dc6e734-b614dd41_0 is DEAD
2025-10-09T21:46:57.017742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52a57518-50f690a5-3dc6e734-b614dd41_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:57.017754Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52a57518-50f690a5-3dc6e734-b614dd41_0] Write session will restart in 0.008100s
2025-10-09T21:46:57.017780Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52a57518-50f690a5-3dc6e734-b614dd41_0] Write session: Do CDS request
2025-10-09T21:46:57.017786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52a57518-50f690a5-3dc6e734-b614dd41_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:57.017777Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:57.017855Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341797411583513:7899]
2025-10-09T21:46:57.027530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52a57518-50f690a5-3dc6e734-b614dd41_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:57.027916Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:57.027941Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 818
2025-10-09T21:46:57.028076Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 818 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:57.028112Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 818 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:57.028122Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=818 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:57.028126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:57.028585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 818 sessionId:  describe result for acl check
2025-10-09T21:46:57.028631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:57.028633Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:57.028635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:57.028641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341801706550849:7999] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:57.029240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341801706550849:7999] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:57.032430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341801706550849:7999] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:57.036394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:57.038318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:57.038559Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:57.038564Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:57.038577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:57.038632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:57.038675Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341801344732544:7253]
2025-10-09T21:46:57.039323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:57.038975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341801706550849:7999] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:57.038997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341801706550849:7999] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:57.039001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341801706550849:7999] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:57.039013Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 818 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:57.039976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:57.040028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:57.040391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:57.040449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:57.040450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:57.040451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:57.040453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:57.040454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:57.040455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:57.040456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.040458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.040459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.040465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:57.040476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:57.040480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:57.040485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:57.040489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:57.040493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:57.040497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.040501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.040505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.040513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:57.040523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:57.040529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:57.040534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:57.040539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:57.040544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:57.040549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:57.040554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:57.040559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:57.040564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:57.040568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:57.040573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:57.040579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.040584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.040588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.040595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.040600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.040605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.040637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:57.053858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:57.055270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:57.055289Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:57.055292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:57.055297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:57.055309Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 817 [2:7559341801344732544:7253]
2025-10-09T21:46:57.055325Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:57.055346Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:57.055350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:57.055354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:57.055356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:57.055377Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:57.055384Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:57.055401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:57.055467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.055469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.055470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.055474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:57.055558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:57.055568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.055583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.055593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.055608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:57.055631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341801344732539:7253]
2025-10-09T21:46:57.055634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341801344732539:7253]
2025-10-09T21:46:57.055636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341801344732539:7253]
2025-10-09T21:46:57.055645Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:57.055662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341801706550950:7999], now have 1 active actors on pipe
2025-10-09T21:46:57.055662Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:57.055666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341801706550815:2432], now have 1 active actors on pipe
2025-10-09T21:46:57.055666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:57.055674Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:57.055676Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:57.055701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:57.055705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:57.055749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341801344732539:7253]
2025-10-09T21:46:57.055764Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:57.055888Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 817
2025-10-09T21:46:57.055892Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 817
2025-10-09T21:46:57.055806Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:57.056133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:57.056144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:57.056190Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b5417cd7-2ed0ac24-d9d91a83-2c73ded7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:57.056200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:57.056204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:57.056208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:57.056220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:57.056242Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:57.056263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:57.056594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:57.056602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:57.056622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:57.056901Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 818 partition: 0 MaxSeqNo: 25 sessionId: src_id|b5417cd7-2ed0ac24-d9d91a83-2c73ded7_0
2025-10-09T21:46:57.057277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52a57518-50f690a5-3dc6e734-b614dd41_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b5417cd7-2ed0ac24-d9d91a83-2c73ded7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:57.059948Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5417cd7-2ed0ac24-d9d91a83-2c73ded7_0] Write session: try to update token
2025-10-09T21:46:57.059971Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5417cd7-2ed0ac24-d9d91a83-2c73ded7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:57.074955Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:57.074972Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:57.074974Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:57.074976Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:57.074987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341801344732539:7253]
2025-10-09T21:46:57.074992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341801344732539:7253]
2025-10-09T21:46:57.074996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341801344732539:7253]
2025-10-09T21:46:57.075044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:57.075052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.075055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:57.075192Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:57.077554Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:57.077562Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:57.077566Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:57.079639Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 818 sessionId: src_id|b5417cd7-2ed0ac24-d9d91a83-2c73ded7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:57.082912Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:57.118323Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341801706550965:7899]
2025-10-09T21:46:57.120235Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 817, pipe: [1:7559341801706550965:7899]
2025-10-09T21:46:57.265502Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:57.265888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:57.265905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:57.265966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341801706550965:7899], now have 1 active actors on pipe
2025-10-09T21:46:57.266221Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:57.266692Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341801706550965:7899]
2025-10-09T21:46:57.266729Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:57.266830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 818 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b5417cd7-2ed0ac24-d9d91a83-2c73ded7_0
2025-10-09T21:46:57.267031Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 818 sessionId: src_id|b5417cd7-2ed0ac24-d9d91a83-2c73ded7_0 is DEAD
2025-10-09T21:46:57.267154Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5417cd7-2ed0ac24-d9d91a83-2c73ded7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:57.267165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5417cd7-2ed0ac24-d9d91a83-2c73ded7_0] Write session will restart in 0.006015s
2025-10-09T21:46:57.267188Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5417cd7-2ed0ac24-d9d91a83-2c73ded7_0] Write session: Do CDS request
2025-10-09T21:46:57.267195Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5417cd7-2ed0ac24-d9d91a83-2c73ded7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:57.267083Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:57.274629Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5417cd7-2ed0ac24-d9d91a83-2c73ded7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:57.274736Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:57.274753Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 819
2025-10-09T21:46:57.275012Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 819 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:57.275052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 819 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:57.275058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=819 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:57.275062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:57.275670Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 819 sessionId:  describe result for acl check
2025-10-09T21:46:57.275713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:57.275723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:57.275725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:57.275731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341801706550986:8004] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:57.276402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341801706550986:8004] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:57.279546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341801706550986:8004] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:57.284653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341801706550986:8004] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:57.284667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341801706550986:8004] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:57.284671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341801706550986:8004] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:57.284681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 819 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:57.290303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:57.297131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:57.297421Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:57.297433Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:57.297450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:57.297530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:57.297570Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341801344732688:7255]
2025-10-09T21:46:57.298193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:57.298655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:57.298717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:57.299076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:57.299144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:57.299145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:57.299146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:57.299148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:57.299149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:57.299150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:57.299152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.299153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.299154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.299160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:57.299170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:57.299174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:57.299178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:57.299181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:57.299185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:57.299189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.299192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.299195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.299204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:57.299212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:57.299218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:57.299223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:57.299228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:57.299233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:57.299237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:57.299242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:57.299246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:57.299250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:57.299254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:57.299259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:57.299263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.299269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.299272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.299277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.299283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.299287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.299315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:57.313789Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 818
2025-10-09T21:46:57.313865Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 818
2025-10-09T21:46:57.310894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:57.311992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:57.311999Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:57.312003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:57.312007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:57.312015Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 818 [2:7559341801344732688:7255]
2025-10-09T21:46:57.312034Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:57.312057Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:57.312061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:57.312064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:57.312067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:57.312086Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:57.312093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:57.312101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:57.312171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.312173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.312176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.312179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:57.312259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:57.312262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.312276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.312287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.312302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:57.312325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341801344732683:7255]
2025-10-09T21:46:57.312327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341801344732683:7255]
2025-10-09T21:46:57.312330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341801344732683:7255]
2025-10-09T21:46:57.312348Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:57.312364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:57.312379Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:57.312383Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:57.312425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341801344732683:7255]
2025-10-09T21:46:57.312428Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:57.312704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:57.312706Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:57.312708Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:57.312714Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:57.314328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341801706550973:2432], now have 1 active actors on pipe
2025-10-09T21:46:57.314339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341801706551068:8004], now have 1 active actors on pipe
2025-10-09T21:46:57.314358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:57.314365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:57.314402Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6d29b5f3-fb63d540-1dee7e2f-39db549a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:57.314414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:57.314420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:57.314423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:57.314436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:57.314464Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:57.314488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:57.315120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:57.315129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:57.315150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:57.315402Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 819 partition: 0 MaxSeqNo: 25 sessionId: src_id|6d29b5f3-fb63d540-1dee7e2f-39db549a_0
2025-10-09T21:46:57.315871Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5417cd7-2ed0ac24-d9d91a83-2c73ded7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6d29b5f3-fb63d540-1dee7e2f-39db549a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:57.318655Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6d29b5f3-fb63d540-1dee7e2f-39db549a_0] Write session: try to update token
2025-10-09T21:46:57.318678Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6d29b5f3-fb63d540-1dee7e2f-39db549a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:57.340359Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 819 sessionId: src_id|6d29b5f3-fb63d540-1dee7e2f-39db549a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:57.342965Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:57.341037Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:57.341053Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:57.341056Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:57.341058Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:57.341067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341801344732683:7255]
2025-10-09T21:46:57.341072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341801344732683:7255]
2025-10-09T21:46:57.341076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341801344732683:7255]
2025-10-09T21:46:57.341124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:57.341132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.341134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:57.341334Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:57.343980Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:57.343985Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:57.343988Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:57.367739Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341801706551125:7899]
2025-10-09T21:46:57.368110Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 818, pipe: [1:7559341801706551125:7899]
2025-10-09T21:46:57.576704Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:57.577263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:57.577280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:57.577332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341801706551125:7899], now have 1 active actors on pipe
2025-10-09T21:46:57.577752Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341801706551125:7899]
2025-10-09T21:46:57.577793Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:57.577796Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:57.577863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 819 reason: error at write: Disconnected sessionId: src_id|6d29b5f3-fb63d540-1dee7e2f-39db549a_0
2025-10-09T21:46:57.578079Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:57.578092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 819 sessionId: src_id|6d29b5f3-fb63d540-1dee7e2f-39db549a_0 is DEAD
2025-10-09T21:46:57.577748Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:57.582292Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d29b5f3-fb63d540-1dee7e2f-39db549a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:57.582305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d29b5f3-fb63d540-1dee7e2f-39db549a_0] Write session will restart in 0.009304s
2025-10-09T21:46:57.582342Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d29b5f3-fb63d540-1dee7e2f-39db549a_0] Write session: Do CDS request
2025-10-09T21:46:57.582349Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d29b5f3-fb63d540-1dee7e2f-39db549a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:57.592671Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6d29b5f3-fb63d540-1dee7e2f-39db549a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:57.594558Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:57.594577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 820
2025-10-09T21:46:57.594858Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 820 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:57.594901Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 820 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:57.594907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=820 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:57.594914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:57.595322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 820 sessionId:  describe result for acl check
2025-10-09T21:46:57.595359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:57.595362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:57.595363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:57.595370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341801706551142:8008] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:57.595916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341801706551142:8008] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:57.600985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341801706551142:8008] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:57.605753Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341801706551142:8008] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:57.605772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341801706551142:8008] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:57.605777Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341801706551142:8008] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:57.605787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 820 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:57.612561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:57.614734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:57.614986Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:57.614997Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:57.615021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:57.615089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:57.615131Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341801344732836:7259]
2025-10-09T21:46:57.615726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:57.616148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:57.616209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:57.616594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:57.616692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:57.616700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:57.616701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:57.616702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:57.616703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:57.616704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:57.616705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.616706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.616708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.616714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:57.616725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:57.616729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:57.616732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:57.616736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:57.616739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:57.616743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.616747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.616750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.616758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:57.616766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:57.616773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:57.616778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:57.616783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:57.616803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:57.616813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:57.616818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:57.616823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:57.616828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:57.616832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:57.616838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:57.616843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.616849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.616853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.616859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.616863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.616869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.616902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:57.624407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:57.625785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:57.625803Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:57.625806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:57.625810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:57.625818Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 819 [2:7559341801344732836:7259]
2025-10-09T21:46:57.625837Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:57.625858Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:57.625861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:57.625864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:57.625866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:57.625886Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:57.625894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:57.625901Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:57.625977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.625979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.625980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.625983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:57.626078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:57.626080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.626093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.626103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.626118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:57.626140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341801344732831:7259]
2025-10-09T21:46:57.626142Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341801344732831:7259]
2025-10-09T21:46:57.626144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341801344732831:7259]
2025-10-09T21:46:57.626153Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:57.626167Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:57.626178Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:57.626182Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:57.626184Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:57.626184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:57.626187Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:57.626233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341801344732831:7259]
2025-10-09T21:46:57.626241Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:57.626252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341801706551128:2432], now have 1 active actors on pipe
2025-10-09T21:46:57.626255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341801706551205:8008], now have 1 active actors on pipe
2025-10-09T21:46:57.626300Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:57.626522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:57.626530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:57.626569Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|aeb65b63-6ad9a153-25d033ee-c9337108_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:57.626579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:57.626584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:57.626588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:57.626601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:57.626617Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:57.626636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:57.626877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:57.626885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:57.626908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:57.626357Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 819
2025-10-09T21:46:57.626368Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 819
2025-10-09T21:46:57.627041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 820 partition: 0 MaxSeqNo: 25 sessionId: src_id|aeb65b63-6ad9a153-25d033ee-c9337108_0
2025-10-09T21:46:57.630622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d29b5f3-fb63d540-1dee7e2f-39db549a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|aeb65b63-6ad9a153-25d033ee-c9337108_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:57.633280Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aeb65b63-6ad9a153-25d033ee-c9337108_0] Write session: try to update token
2025-10-09T21:46:57.633308Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aeb65b63-6ad9a153-25d033ee-c9337108_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:57.648440Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:57.648459Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:57.648471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341801344732831:7259]
2025-10-09T21:46:57.648528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:57.648529Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:57.648771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:57.648918Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:57.649100Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:57.649239Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:57.649611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:57.649667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:57.659414Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 820 sessionId: src_id|aeb65b63-6ad9a153-25d033ee-c9337108_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:57.662116Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:57.660075Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:57.660091Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:57.660094Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:57.660096Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:57.660107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341801344732831:7259]
2025-10-09T21:46:57.660112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341801344732831:7259]
2025-10-09T21:46:57.660116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341801344732831:7259]
2025-10-09T21:46:57.660169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:57.660174Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:57.660177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.660181Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:57.660181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:57.660185Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:57.660313Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:57.677838Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341801706551280:7899]
2025-10-09T21:46:57.682169Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 819, pipe: [1:7559341801706551280:7899]
2025-10-09T21:46:57.868442Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:57.868875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:57.868885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:57.877469Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341801706551280:7899]
2025-10-09T21:46:57.877522Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:57.877524Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:57.877607Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 820 reason: error at write: Disconnected sessionId: src_id|aeb65b63-6ad9a153-25d033ee-c9337108_0
2025-10-09T21:46:57.877820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 820 sessionId: src_id|aeb65b63-6ad9a153-25d033ee-c9337108_0 is DEAD
2025-10-09T21:46:57.878009Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:57.878289Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aeb65b63-6ad9a153-25d033ee-c9337108_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:57.878302Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aeb65b63-6ad9a153-25d033ee-c9337108_0] Write session will restart in 0.006088s
2025-10-09T21:46:57.878326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aeb65b63-6ad9a153-25d033ee-c9337108_0] Write session: Do CDS request
2025-10-09T21:46:57.878333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aeb65b63-6ad9a153-25d033ee-c9337108_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:57.876880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341801706551280:7899], now have 1 active actors on pipe
2025-10-09T21:46:57.878163Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:57.885731Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aeb65b63-6ad9a153-25d033ee-c9337108_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:57.885909Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:57.885940Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 821
2025-10-09T21:46:57.886130Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 821 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:57.886163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 821 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:57.886168Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=821 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:57.886172Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:57.886704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 821 sessionId:  describe result for acl check
2025-10-09T21:46:57.886740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:57.886742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:57.886744Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:57.886750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341801706551315:8019] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:57.887321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341801706551315:8019] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:57.893333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341801706551315:8019] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:57.899053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341801706551315:8019] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:57.899075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341801706551315:8019] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:57.899082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341801706551315:8019] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:57.899092Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 821 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:57.903538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:57.906098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:57.906343Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:57.906353Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:57.906367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:57.906436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:57.906478Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341801344733018:7271]
2025-10-09T21:46:57.907073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:57.907997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:57.908053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:57.908382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:57.908449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:57.908450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:57.908451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:57.908452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:57.908453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:57.908455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:57.908456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.908457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.908458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.908465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:57.908476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:57.908480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:57.908484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:57.908487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:57.908490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:57.908494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.908497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.908500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.908508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:57.908516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:57.908525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:57.908531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:57.908535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:57.908540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:57.908545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:57.908549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:57.908554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:57.908558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:57.908563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:57.908584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:57.908590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.908594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.908599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.908603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.908608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.908612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.908638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:57.918574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:57.919795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:57.919815Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:57.919819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:57.919824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:57.919832Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 820 [2:7559341801344733018:7271]
2025-10-09T21:46:57.919848Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:57.919870Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:57.919873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:57.919877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:57.919880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:57.919901Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:57.919907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:57.919916Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:57.919994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.920003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.920006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.920009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:57.921083Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 820
2025-10-09T21:46:57.921122Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 820
2025-10-09T21:46:57.920272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:57.920281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.920294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:57.920306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:57.920326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:57.920347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341801344733012:7271]
2025-10-09T21:46:57.920351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341801344733012:7271]
2025-10-09T21:46:57.920353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341801344733012:7271]
2025-10-09T21:46:57.920373Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:57.920390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:57.920406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:57.920409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:57.920454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341801344733012:7271]
2025-10-09T21:46:57.920457Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:57.920620Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:57.920622Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:57.920624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:57.920630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:57.921727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341801706551445:8019], now have 1 active actors on pipe
2025-10-09T21:46:57.921736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341801706551302:2432], now have 1 active actors on pipe
2025-10-09T21:46:57.921752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:57.921761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:57.921795Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|18dc25c-4bf2261c-e4b70494-9685417e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:57.921806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:57.921811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:57.921813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:57.921825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:57.921838Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:57.921857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:57.923439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:57.923451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:57.923479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:57.923755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 821 partition: 0 MaxSeqNo: 25 sessionId: src_id|18dc25c-4bf2261c-e4b70494-9685417e_0
2025-10-09T21:46:57.924311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aeb65b63-6ad9a153-25d033ee-c9337108_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|18dc25c-4bf2261c-e4b70494-9685417e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:57.927584Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|18dc25c-4bf2261c-e4b70494-9685417e_0] Write session: try to update token
2025-10-09T21:46:57.927613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|18dc25c-4bf2261c-e4b70494-9685417e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:57.941284Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:57.941300Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:57.941302Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:57.941304Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:57.941314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341801344733012:7271]
2025-10-09T21:46:57.941318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341801344733012:7271]
2025-10-09T21:46:57.941321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341801344733012:7271]
2025-10-09T21:46:57.941373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:57.941380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:57.941383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:57.941388Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:57.941395Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:57.941400Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:57.941514Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:57.952306Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 821 sessionId: src_id|18dc25c-4bf2261c-e4b70494-9685417e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:57.954988Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:57.974846Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341801706551463:7899]
2025-10-09T21:46:57.980526Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 820, pipe: [1:7559341801706551463:7899]
2025-10-09T21:46:58.113702Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:58.114153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:58.114168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:58.114193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341801706551463:7899], now have 1 active actors on pipe
2025-10-09T21:46:58.115011Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:58.115029Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:58.115089Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 821 reason: error at write: Disconnected sessionId: src_id|18dc25c-4bf2261c-e4b70494-9685417e_0
2025-10-09T21:46:58.115298Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 821 sessionId: src_id|18dc25c-4bf2261c-e4b70494-9685417e_0 is DEAD
2025-10-09T21:46:58.114679Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:58.115452Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18dc25c-4bf2261c-e4b70494-9685417e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:58.115474Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18dc25c-4bf2261c-e4b70494-9685417e_0] Write session will restart in 0.006849s
2025-10-09T21:46:58.115466Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:58.115500Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18dc25c-4bf2261c-e4b70494-9685417e_0] Write session: Do CDS request
2025-10-09T21:46:58.115507Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18dc25c-4bf2261c-e4b70494-9685417e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:58.115509Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341801706551463:7899]
2025-10-09T21:46:58.123881Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|18dc25c-4bf2261c-e4b70494-9685417e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:58.124185Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:58.124205Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 822
2025-10-09T21:46:58.124528Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 822 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:58.124573Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 822 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:58.124579Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=822 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:58.124594Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:58.126613Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 822 sessionId:  describe result for acl check
2025-10-09T21:46:58.126672Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:58.126674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:58.126676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:58.126683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341806001518772:8024] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:58.127299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341806001518772:8024] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:58.134225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341806001518772:8024] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:58.143365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341806001518772:8024] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:58.143384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341806001518772:8024] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:58.143387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341806001518772:8024] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:58.143398Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 822 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:58.150009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:58.152194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:58.152439Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:58.152444Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:58.152457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:58.152519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:58.152557Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341805639700460:7275]
2025-10-09T21:46:58.153190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:58.153708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:58.153757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:58.154135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:58.154209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:58.154210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:58.154211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:58.154212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:58.154214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:58.154215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:58.154216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.154217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.154218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.154224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:58.154235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:58.154240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:58.154244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:58.154248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:58.154252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:58.154256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.154259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.154263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.154273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:58.154284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:58.154289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:58.154295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:58.154299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:58.154304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:58.154308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:58.154313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:58.154317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:58.154322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:58.154327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:58.154332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:58.154336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.154341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.154346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.154350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.154354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.154359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.154385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:58.164623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:58.165868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:58.165883Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:58.165887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:58.165892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:58.166186Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 821 [2:7559341805639700460:7275]
2025-10-09T21:46:58.166213Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:58.166235Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:58.166239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:58.166243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:58.166245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:58.166264Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:58.166276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:58.166284Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:58.166358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.166367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.166370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.166373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:58.166458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:58.166461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.166476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.166488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.166505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:58.166527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341805639700455:7275]
2025-10-09T21:46:58.166531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341805639700455:7275]
2025-10-09T21:46:58.166533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341805639700455:7275]
2025-10-09T21:46:58.166542Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:58.166554Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:58.166558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341806001518764:2432], now have 1 active actors on pipe
2025-10-09T21:46:58.166558Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:58.166560Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:58.166562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341806001518845:8024], now have 1 active actors on pipe
2025-10-09T21:46:58.166571Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:58.166587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:58.166590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:58.166638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341805639700455:7275]
2025-10-09T21:46:58.166649Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:58.166717Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:58.167068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:58.167081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:58.167128Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|52fc1cfd-7443c27c-78f06000-cead6092_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:58.167145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:58.167151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:58.167169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:58.167186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:58.167205Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:58.167229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:58.167563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:58.167578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:58.167620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:58.166879Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 821
2025-10-09T21:46:58.166925Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 821
2025-10-09T21:46:58.167738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 822 partition: 0 MaxSeqNo: 25 sessionId: src_id|52fc1cfd-7443c27c-78f06000-cead6092_0
2025-10-09T21:46:58.171548Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18dc25c-4bf2261c-e4b70494-9685417e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|52fc1cfd-7443c27c-78f06000-cead6092_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:58.174463Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52fc1cfd-7443c27c-78f06000-cead6092_0] Write session: try to update token
2025-10-09T21:46:58.174492Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52fc1cfd-7443c27c-78f06000-cead6092_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:58.186938Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:58.186955Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:58.186964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341805639700455:7275]
2025-10-09T21:46:58.187014Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:58.187013Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:58.187345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:58.187660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:58.187811Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:58.187949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:58.188146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:58.188194Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:58.192136Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 822 sessionId: src_id|52fc1cfd-7443c27c-78f06000-cead6092_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:58.195023Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:58.195432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:58.195453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:58.195494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:58.195674Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:58.205226Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:58.205244Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:58.205250Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:58.205253Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:58.205264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341805639700455:7275]
2025-10-09T21:46:58.205268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341805639700455:7275]
2025-10-09T21:46:58.205272Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341805639700455:7275]
2025-10-09T21:46:58.205327Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:58.205333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:58.205336Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:58.205340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:58.205340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.205343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:58.205536Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:58.214422Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341806001518925:7899]
2025-10-09T21:46:58.215293Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 821, pipe: [1:7559341806001518925:7899]
2025-10-09T21:46:58.367673Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:58.368014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341806001518925:7899], now have 1 active actors on pipe
2025-10-09T21:46:58.375752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:58.375779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:58.375948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.376013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.376082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.376155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.376207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.376217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.376290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.376353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.376417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.376494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.376539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.376541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.376623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.376698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.376765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.376826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.376869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.376872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.376955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.377020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.377107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.377178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.377231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.377235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.377346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.377406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.377465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.377565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.377685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.377688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.377757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.377855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.377958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.378017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.378196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.378207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.378273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.378338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.378402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.378460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.378529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.378534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.378599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.378660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.378734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.378800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.378846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.378853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.378918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.378977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.379041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.379107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.379200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.379208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.379276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.379334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.379418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.379486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.379553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.379563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.380861Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:58.380990Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:58.381077Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341806001518925:7899]
2025-10-09T21:46:58.381101Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:58.381143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 822 reason: status is not ok: tablet will be restarted right now sessionId: src_id|52fc1cfd-7443c27c-78f06000-cead6092_0
2025-10-09T21:46:58.381354Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 822 sessionId: src_id|52fc1cfd-7443c27c-78f06000-cead6092_0 is DEAD
2025-10-09T21:46:58.382127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52fc1cfd-7443c27c-78f06000-cead6092_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:58.382139Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52fc1cfd-7443c27c-78f06000-cead6092_0] Write session will restart in 0.009967s
2025-10-09T21:46:58.382162Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52fc1cfd-7443c27c-78f06000-cead6092_0] Write session: Do CDS request
2025-10-09T21:46:58.382171Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52fc1cfd-7443c27c-78f06000-cead6092_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:58.393674Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52fc1cfd-7443c27c-78f06000-cead6092_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:58.393740Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:58.393757Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 823
2025-10-09T21:46:58.393981Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 823 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:58.394021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 823 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:58.394048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=823 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:58.394052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:58.394617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 823 sessionId:  describe result for acl check
2025-10-09T21:46:58.394668Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:58.394678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:58.394680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:58.394686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341806001518974:8029] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:58.395277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341806001518974:8029] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:58.400067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341806001518974:8029] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:58.400258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:58.401846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:58.402094Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:58.402104Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:58.402118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:58.402183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:58.402232Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341805639700605:7277]
2025-10-09T21:46:58.402855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:58.403242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341806001518974:8029] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:58.403260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341806001518974:8029] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:58.403264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341806001518974:8029] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:58.403273Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 823 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:58.403304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:58.403364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:58.403721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:58.403772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:58.403773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:58.403775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:58.403776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:58.403778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:58.403779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:58.403780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.403782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.403783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.403788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:58.403800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:58.403804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:58.403809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:58.403814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:58.403818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:58.403822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.403826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.403830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.403838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:58.403847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:58.403854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:58.403859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:58.403864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:58.403869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:58.403873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:58.403879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:58.403884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:58.403890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:58.403894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:58.403899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:58.403906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.403912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.403917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.403922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.403928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.403933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.403962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:58.413264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:58.414604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:58.414619Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:58.414622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:58.414626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:58.414633Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 822 [2:7559341805639700605:7277]
2025-10-09T21:46:58.414646Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:58.414668Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:58.414672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:58.414675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:58.414677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:58.414696Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:58.414701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:58.414710Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:58.414774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.414780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.414782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.414785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:58.414873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:58.414880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.414894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.414909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.414926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:58.414958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341806001519083:8029], now have 1 active actors on pipe
2025-10-09T21:46:58.414962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341806001518928:2432], now have 1 active actors on pipe
2025-10-09T21:46:58.414972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341805639700600:7277]
2025-10-09T21:46:58.414975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341805639700600:7277]
2025-10-09T21:46:58.414977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341805639700600:7277]
2025-10-09T21:46:58.414991Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:58.415010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:58.415179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 822
2025-10-09T21:46:58.415217Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 822
2025-10-09T21:46:58.415023Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:58.415025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:58.415028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:58.415034Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:58.415036Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:58.415073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341805639700600:7277]
2025-10-09T21:46:58.415082Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:58.415217Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:58.415702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:58.415721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:58.415760Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|950acc50-d06095f7-b0714105-e884c256_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:58.415769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:58.415773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:58.415776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:58.415789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:58.415801Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:58.415819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:58.416057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:58.416066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:58.416084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:58.416244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 823 partition: 0 MaxSeqNo: 25 sessionId: src_id|950acc50-d06095f7-b0714105-e884c256_0
2025-10-09T21:46:58.416622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52fc1cfd-7443c27c-78f06000-cead6092_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|950acc50-d06095f7-b0714105-e884c256_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:58.419214Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|950acc50-d06095f7-b0714105-e884c256_0] Write session: try to update token
2025-10-09T21:46:58.419234Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|950acc50-d06095f7-b0714105-e884c256_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:58.438910Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 823 sessionId: src_id|950acc50-d06095f7-b0714105-e884c256_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:58.439920Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:58.439937Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:58.439941Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:58.439943Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:58.439953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341805639700600:7277]
2025-10-09T21:46:58.439958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341805639700600:7277]
2025-10-09T21:46:58.439962Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341805639700600:7277]
2025-10-09T21:46:58.440011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:58.440011Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:58.440017Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:58.440018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.440021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:58.440114Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:58.440166Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:58.441417Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:58.481190Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341806001519092:7899]
2025-10-09T21:46:58.481548Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 822, pipe: [1:7559341806001519092:7899]
2025-10-09T21:46:58.595120Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:58.595521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:58.595533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:58.595580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341806001519092:7899], now have 1 active actors on pipe
2025-10-09T21:46:58.595978Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:58.596271Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:58.596294Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:58.596367Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 823 reason: error at write: Disconnected sessionId: src_id|950acc50-d06095f7-b0714105-e884c256_0
2025-10-09T21:46:58.596665Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 823 sessionId: src_id|950acc50-d06095f7-b0714105-e884c256_0 is DEAD
2025-10-09T21:46:58.596678Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:58.596719Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341806001519092:7899]
2025-10-09T21:46:58.602145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|950acc50-d06095f7-b0714105-e884c256_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:58.602160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|950acc50-d06095f7-b0714105-e884c256_0] Write session will restart in 0.008885s
2025-10-09T21:46:58.602187Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|950acc50-d06095f7-b0714105-e884c256_0] Write session: Do CDS request
2025-10-09T21:46:58.602193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|950acc50-d06095f7-b0714105-e884c256_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:58.612603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|950acc50-d06095f7-b0714105-e884c256_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:58.612707Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:58.612724Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 824
2025-10-09T21:46:58.612869Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 824 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:58.612898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 824 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:58.612903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=824 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:58.612908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:58.613430Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 824 sessionId:  describe result for acl check
2025-10-09T21:46:58.613473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:58.613475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:58.613478Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:58.613484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341806001519203:8033] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:58.614132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341806001519203:8033] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:58.616153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:58.617940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:58.618218Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:58.618228Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:58.618242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:58.618306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:58.618348Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341805639700757:7281]
2025-10-09T21:46:58.618946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:58.619405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341806001519203:8033] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:58.619343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:58.619401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:58.619898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:58.620000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:58.620001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:58.620002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:58.620003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:58.620005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:58.620006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:58.620007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.620010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.620011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.620017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:58.620027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:58.620030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:58.620034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:58.620037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:58.620040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:58.620043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.620047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.620050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.620058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:58.620066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:58.620084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:58.620089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:58.620098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:58.620104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:58.620108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:58.620114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:58.620119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:58.620124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:58.620128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:58.620132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:58.620136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.620140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.620144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.620148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.620153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.620158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.620196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:58.625740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341806001519203:8033] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:58.625764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341806001519203:8033] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:58.625768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341806001519203:8033] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:58.625779Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 824 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:58.629799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:58.630987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:58.631009Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:58.631012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:58.631017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:58.631025Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 823 [2:7559341805639700757:7281]
2025-10-09T21:46:58.631040Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:58.631062Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:58.631065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:58.631069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:58.631071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:58.631091Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:58.631104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:58.631112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:58.631174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.631184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.631186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.631189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:58.631269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:58.631278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.631292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.631302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.631317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:58.631342Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341805639700752:7281]
2025-10-09T21:46:58.631346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341805639700752:7281]
2025-10-09T21:46:58.631348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341805639700752:7281]
2025-10-09T21:46:58.631365Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:58.631382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:58.631397Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:58.631401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:58.631445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341805639700752:7281]
2025-10-09T21:46:58.631447Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:58.631451Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:58.631454Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:58.631455Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:58.631462Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:58.631466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341806001519243:8033], now have 1 active actors on pipe
2025-10-09T21:46:58.631470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341806001519095:2432], now have 1 active actors on pipe
2025-10-09T21:46:58.631537Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 823
2025-10-09T21:46:58.631549Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 823
2025-10-09T21:46:58.631745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:58.631758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:58.631803Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|85aedeab-17cb4611-3a30d13-9df3a79c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:58.631812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:58.631816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:58.631819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:58.631830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:58.631842Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:58.631861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:58.632319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 824 partition: 0 MaxSeqNo: 25 sessionId: src_id|85aedeab-17cb4611-3a30d13-9df3a79c_0
2025-10-09T21:46:58.632175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:58.632188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:58.632212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:58.632716Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|950acc50-d06095f7-b0714105-e884c256_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|85aedeab-17cb4611-3a30d13-9df3a79c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:58.635631Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85aedeab-17cb4611-3a30d13-9df3a79c_0] Write session: try to update token
2025-10-09T21:46:58.635652Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85aedeab-17cb4611-3a30d13-9df3a79c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:58.652330Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:58.652349Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:58.652360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341805639700752:7281]
2025-10-09T21:46:58.652409Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:58.652413Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:58.652624Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:58.652779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:58.652930Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:58.653074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:58.653290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:58.653336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:58.661507Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 824 sessionId: src_id|85aedeab-17cb4611-3a30d13-9df3a79c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:58.664431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:58.664451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:58.664503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:58.663976Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:58.664622Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:58.671683Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:58.671702Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:58.671705Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:58.671707Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:58.671720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341805639700752:7281]
2025-10-09T21:46:58.671724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341805639700752:7281]
2025-10-09T21:46:58.671728Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341805639700752:7281]
2025-10-09T21:46:58.671778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:58.671787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.671790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:58.671961Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:58.674618Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:58.674634Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:58.674638Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:58.697644Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341806001519252:7899]
2025-10-09T21:46:58.698522Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 823, pipe: [1:7559341806001519252:7899]
2025-10-09T21:46:58.835440Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:58.835718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341806001519252:7899], now have 1 active actors on pipe
2025-10-09T21:46:58.843731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:58.843756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:58.843925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.843984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.844054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.844121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.844170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.844175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.844250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.844313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.844380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.844444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.844483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.844486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.844551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.844608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.844667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.844725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.844769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.844771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.844839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.844899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.844954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.845012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.845050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.845052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.845115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.845208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.845295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.845701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.845870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.845874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.845959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.846019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.850127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.850206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.850353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.850366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.850434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.850487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.850550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.850609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.850667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.850671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.850736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.850792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.850848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.850913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.850965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.850967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.851041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.851099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.851155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.851216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.851276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.851279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.851345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:58.851400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:58.851460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:58.851516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:58.851553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:58.851556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:58.852835Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:58.854191Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341806001519252:7899]
2025-10-09T21:46:58.854191Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:58.854210Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:58.854225Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:58.854271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 824 reason: error at write: Disconnected sessionId: src_id|85aedeab-17cb4611-3a30d13-9df3a79c_0
2025-10-09T21:46:58.854512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 824 sessionId: src_id|85aedeab-17cb4611-3a30d13-9df3a79c_0 is DEAD
2025-10-09T21:46:58.854758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85aedeab-17cb4611-3a30d13-9df3a79c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:58.854769Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85aedeab-17cb4611-3a30d13-9df3a79c_0] Write session will restart in 0.006950s
2025-10-09T21:46:58.854794Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85aedeab-17cb4611-3a30d13-9df3a79c_0] Write session: Do CDS request
2025-10-09T21:46:58.854801Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85aedeab-17cb4611-3a30d13-9df3a79c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:58.865687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85aedeab-17cb4611-3a30d13-9df3a79c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:58.866211Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:58.866230Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 825
2025-10-09T21:46:58.866445Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 825 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:58.866494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 825 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:58.866500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=825 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:58.866504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:58.866997Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 825 sessionId:  describe result for acl check
2025-10-09T21:46:58.867039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:58.867044Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:58.867046Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:58.867052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341806001519272:8038] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:58.867657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341806001519272:8038] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:58.870636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341806001519272:8038] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:58.877410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341806001519272:8038] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:58.877440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341806001519272:8038] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:58.877444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341806001519272:8038] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:58.877455Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 825 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:58.892534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:58.894506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:58.894752Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:58.894763Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:58.894776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:58.894837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:58.894877Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341805639700936:7293]
2025-10-09T21:46:58.895459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:58.895852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:58.895905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:58.896231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:58.896281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:58.896282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:58.896283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:58.896284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:58.896286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:58.896287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:58.896288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.896289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.896290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.896296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:58.896308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:58.896312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:58.896316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:58.896320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:58.896324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:58.896328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.896332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.896336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.896343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:58.896353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:58.896358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:58.896364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:58.896368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:58.896374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:58.896378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:58.896384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:58.896388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:58.896393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:58.896397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:58.896402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:58.896406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.896411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.896415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.896419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.896422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.896427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.896455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:58.905053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:58.906547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:58.906554Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:58.906557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:58.906562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:58.906570Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 824 [2:7559341805639700936:7293]
2025-10-09T21:46:58.906586Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:58.906606Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:58.906610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:58.906613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:58.906615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:58.906635Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:58.906642Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:58.906652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:58.906720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.906722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.906725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.906727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:58.906807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:58.906810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.906826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:58.906838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:58.906857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:58.906878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341805639700931:7293]
2025-10-09T21:46:58.906881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341805639700931:7293]
2025-10-09T21:46:58.906883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341805639700931:7293]
2025-10-09T21:46:58.906891Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:58.906908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:58.906912Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:58.906916Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:58.906919Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:58.906923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:58.906926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:58.906940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341806001519316:8038], now have 1 active actors on pipe
2025-10-09T21:46:58.906944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341806001519260:2432], now have 1 active actors on pipe
2025-10-09T21:46:58.906984Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341805639700931:7293]
2025-10-09T21:46:58.906996Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:58.907631Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:58.907885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:58.907897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:58.907931Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c4fa4121-1b7c36f7-7c000afe-c4218c88_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:58.907942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:58.907953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:58.907956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:58.907968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:58.907985Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:58.908010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:58.908346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:58.908356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:58.908377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:58.909035Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85aedeab-17cb4611-3a30d13-9df3a79c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c4fa4121-1b7c36f7-7c000afe-c4218c88_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:58.907409Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 824
2025-10-09T21:46:58.907458Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 824
2025-10-09T21:46:58.908538Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 825 partition: 0 MaxSeqNo: 25 sessionId: src_id|c4fa4121-1b7c36f7-7c000afe-c4218c88_0
2025-10-09T21:46:58.911687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4fa4121-1b7c36f7-7c000afe-c4218c88_0] Write session: try to update token
2025-10-09T21:46:58.911710Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4fa4121-1b7c36f7-7c000afe-c4218c88_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:58.923178Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:58.923199Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:58.923201Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:58.923203Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:58.923212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341805639700931:7293]
2025-10-09T21:46:58.923217Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341805639700931:7293]
2025-10-09T21:46:58.923222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341805639700931:7293]
2025-10-09T21:46:58.923267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:58.923279Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:58.923282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:58.923284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:58.923286Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:58.923289Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:58.923422Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:58.938241Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 825 sessionId: src_id|c4fa4121-1b7c36f7-7c000afe-c4218c88_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:58.940887Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:58.954223Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341806001519435:7899]
2025-10-09T21:46:58.954598Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 824, pipe: [1:7559341806001519435:7899]
2025-10-09T21:46:59.107559Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:59.107919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:59.107937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:59.107962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341806001519435:7899], now have 1 active actors on pipe
2025-10-09T21:46:59.108596Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:59.108684Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341806001519435:7899]
2025-10-09T21:46:59.108720Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:59.108725Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:59.108792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 825 reason: error at write: Disconnected sessionId: src_id|c4fa4121-1b7c36f7-7c000afe-c4218c88_0
2025-10-09T21:46:59.108815Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:59.108993Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 825 sessionId: src_id|c4fa4121-1b7c36f7-7c000afe-c4218c88_0 is DEAD
2025-10-09T21:46:59.109161Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4fa4121-1b7c36f7-7c000afe-c4218c88_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:59.109172Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4fa4121-1b7c36f7-7c000afe-c4218c88_0] Write session will restart in 0.009241s
2025-10-09T21:46:59.109212Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4fa4121-1b7c36f7-7c000afe-c4218c88_0] Write session: Do CDS request
2025-10-09T21:46:59.109220Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4fa4121-1b7c36f7-7c000afe-c4218c88_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:59.120352Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4fa4121-1b7c36f7-7c000afe-c4218c88_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:59.122221Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:59.122243Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 826
2025-10-09T21:46:59.122428Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 826 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:59.122468Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 826 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:59.122474Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=826 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:59.122480Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:59.123015Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 826 sessionId:  describe result for acl check
2025-10-09T21:46:59.123057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:59.123059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:59.123061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:59.123067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341810296486749:8049] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:59.123704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341810296486749:8049] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:59.126525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341810296486749:8049] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:59.142543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341810296486749:8049] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:59.142562Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341810296486749:8049] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:59.142566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341810296486749:8049] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:59.142577Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 826 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:59.150944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:59.158780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:59.159070Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:59.159075Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:59.159093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:59.159175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:59.159239Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341809934668390:7297]
2025-10-09T21:46:59.159864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:59.166267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:59.166382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:59.166806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:59.166882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:59.166884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:59.166885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:59.166886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:59.166888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:59.166890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:59.166891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.166893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.166894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.166902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:59.166914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:59.166919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:59.166923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:59.166928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:59.166932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:59.166936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.166941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.166945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.166953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:59.166962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:59.166969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:59.166974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:59.166982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:59.166989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:59.166994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:59.167000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:59.167005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:59.167013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:59.167019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:59.167026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:59.167032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.167037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.167042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.167048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.167053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.167059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.167095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:59.192649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:59.194093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:59.194112Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:59.194116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:59.194121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:59.194129Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 825 [2:7559341809934668390:7297]
2025-10-09T21:46:59.194145Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:59.194169Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:59.194173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:59.194177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:59.194179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:59.194202Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:59.194209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:59.194217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:59.194300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.194303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.194305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.194307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:59.194402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:59.194405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.194420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.194429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.194443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:59.194465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341809934668385:7297]
2025-10-09T21:46:59.194469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341809934668385:7297]
2025-10-09T21:46:59.194471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341809934668385:7297]
2025-10-09T21:46:59.194488Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:59.194508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:59.194522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:59.194526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:59.194576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341809934668385:7297]
2025-10-09T21:46:59.194578Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:59.194816Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:59.194821Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:59.194823Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:59.194829Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:59.195160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341810296486880:8049], now have 1 active actors on pipe
2025-10-09T21:46:59.195169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341810296486740:2432], now have 1 active actors on pipe
2025-10-09T21:46:59.195476Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 825
2025-10-09T21:46:59.195477Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 825
2025-10-09T21:46:59.196195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:59.196208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:59.196245Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|312c3888-6773fe12-947c8802-e64abff1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:59.196257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:59.196264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:59.196267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:59.196281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:59.196299Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:59.196320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:59.196825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:59.196832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:59.196853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:59.197014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 826 partition: 0 MaxSeqNo: 25 sessionId: src_id|312c3888-6773fe12-947c8802-e64abff1_0
2025-10-09T21:46:59.197559Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4fa4121-1b7c36f7-7c000afe-c4218c88_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|312c3888-6773fe12-947c8802-e64abff1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:59.210390Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|312c3888-6773fe12-947c8802-e64abff1_0] Write session: try to update token
2025-10-09T21:46:59.210420Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|312c3888-6773fe12-947c8802-e64abff1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:59.214274Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341810296486901:7899]
2025-10-09T21:46:59.215673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341810296486901:7899], now have 1 active actors on pipe
2025-10-09T21:46:59.215919Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 825, pipe: [1:7559341810296486901:7899]
2025-10-09T21:46:59.219532Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:59.219549Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:59.219552Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:59.219554Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:59.219565Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341809934668385:7297]
2025-10-09T21:46:59.219574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341809934668385:7297]
2025-10-09T21:46:59.219577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341809934668385:7297]
2025-10-09T21:46:59.219628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:59.219638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.219641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:59.219660Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:59.219667Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:59.219671Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:59.219796Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:59.232770Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 826 sessionId: src_id|312c3888-6773fe12-947c8802-e64abff1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:59.243744Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:59.392106Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:59.392445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:59.392463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:59.392795Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:59.393277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|312c3888-6773fe12-947c8802-e64abff1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:59.393290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|312c3888-6773fe12-947c8802-e64abff1_0] Write session will restart in 0.008801s
2025-10-09T21:46:59.393316Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|312c3888-6773fe12-947c8802-e64abff1_0] Write session: Do CDS request
2025-10-09T21:46:59.393322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|312c3888-6773fe12-947c8802-e64abff1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:59.392819Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341810296486901:7899]
2025-10-09T21:46:59.392824Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:59.392829Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:59.392861Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:59.392892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 826 reason: error at write: Disconnected sessionId: src_id|312c3888-6773fe12-947c8802-e64abff1_0
2025-10-09T21:46:59.393100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 826 sessionId: src_id|312c3888-6773fe12-947c8802-e64abff1_0 is DEAD
2025-10-09T21:46:59.403596Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|312c3888-6773fe12-947c8802-e64abff1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:59.403685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:59.403703Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 827
2025-10-09T21:46:59.403855Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 827 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:59.403894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 827 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:59.403900Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=827 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:59.403917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:59.404358Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 827 sessionId:  describe result for acl check
2025-10-09T21:46:59.404391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:59.404392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:59.404396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:59.404403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341810296486921:8054] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:59.405122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341810296486921:8054] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:59.408017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341810296486921:8054] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:59.412743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341810296486921:8054] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:59.412761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341810296486921:8054] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:59.412765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341810296486921:8054] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:59.412775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 827 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:59.419337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:59.421251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:59.421471Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:59.421482Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:59.421494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:59.421561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:59.421610Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341809934668540:7299]
2025-10-09T21:46:59.422161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:59.422544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:59.422608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:59.423026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:59.423091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:59.423096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:59.423098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:59.423099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:59.423100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:59.423101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:59.423102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.423104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.423105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.423111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:59.423122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:59.423125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:59.423129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:59.423132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:59.423136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:59.423139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.423143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.423164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.423172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:59.423179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:59.423186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:59.423192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:59.423196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:59.423202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:59.423206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:59.423211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:59.423217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:59.423222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:59.423226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:59.423232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:59.423236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.423241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.423246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.423251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.423256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.423261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.423294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:59.431508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:59.432720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:59.432739Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:59.432743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:59.432748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:59.432756Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 826 [2:7559341809934668540:7299]
2025-10-09T21:46:59.432773Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:59.432796Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:59.432800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:59.432803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:59.432805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:59.432825Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:59.432833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:59.432846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:59.432926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.432933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.432936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.432939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:59.433018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:59.433027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.433041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.433052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.433068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:59.433094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341809934668535:7299]
2025-10-09T21:46:59.433096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341809934668535:7299]
2025-10-09T21:46:59.433098Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341809934668535:7299]
2025-10-09T21:46:59.433114Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:59.433132Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:59.433147Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:59.433150Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:59.433196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341809934668535:7299]
2025-10-09T21:46:59.433200Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:59.433339Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:59.433342Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:59.433344Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:59.433353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:59.433474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341810296486906:2432], now have 1 active actors on pipe
2025-10-09T21:46:59.433487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341810296487005:8054], now have 1 active actors on pipe
2025-10-09T21:46:59.433776Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 826
2025-10-09T21:46:59.433824Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 826
2025-10-09T21:46:59.434382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:59.434396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:59.434428Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1fe96581-ac4ebf9a-2acf06c2-4f5b4f24_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:59.434439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:59.434443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:59.434446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:59.434457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:59.434473Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:59.434496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:59.435482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:59.435494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:59.435528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:59.435743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 827 partition: 0 MaxSeqNo: 25 sessionId: src_id|1fe96581-ac4ebf9a-2acf06c2-4f5b4f24_0
2025-10-09T21:46:59.436194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|312c3888-6773fe12-947c8802-e64abff1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1fe96581-ac4ebf9a-2acf06c2-4f5b4f24_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:59.438701Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1fe96581-ac4ebf9a-2acf06c2-4f5b4f24_0] Write session: try to update token
2025-10-09T21:46:59.438722Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1fe96581-ac4ebf9a-2acf06c2-4f5b4f24_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:59.453532Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 827 sessionId: src_id|1fe96581-ac4ebf9a-2acf06c2-4f5b4f24_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:59.456699Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:59.455881Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:59.455899Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:59.455909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341809934668535:7299]
2025-10-09T21:46:59.455965Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:59.456117Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:59.456139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:59.456282Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:59.456446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:59.456584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:59.456769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:59.456806Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:59.456919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:59.456930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:59.456966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:46:59.458305Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:59.469219Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:59.469237Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:59.469239Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:59.469242Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:59.469254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341809934668535:7299]
2025-10-09T21:46:59.469259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341809934668535:7299]
2025-10-09T21:46:59.469262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341809934668535:7299]
2025-10-09T21:46:59.469312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:59.469319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.469322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:59.469322Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:59.469328Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:59.469332Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:59.469460Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:59.494106Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341810296487065:7899]
2025-10-09T21:46:59.495976Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 826, pipe: [1:7559341810296487065:7899]
2025-10-09T21:46:59.630010Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:59.630502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341810296487065:7899], now have 1 active actors on pipe
2025-10-09T21:46:59.638619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:59.638649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:59.638826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:59.638889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:59.638958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:59.639033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:59.639567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:59.639586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:59.639657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:59.639720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:59.639783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:59.640037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:59.640087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:59.640096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:59.640163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:59.640228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:59.640297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:59.640362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:59.640405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:59.640408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:59.640476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:59.640535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:59.640601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:59.640665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:59.640715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:59.640723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:59.640792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:59.640852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:59.640920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:59.640993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:59.641036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:59.641039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:59.641110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:59.641165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:59.641228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:59.641289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:59.641331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:59.641333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:59.641400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:59.641482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:59.641546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:59.641605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:59.641664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:59.641670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:59.641733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:59.641784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:59.641843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:59.641901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:59.641954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:59.641959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:59.642126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:59.642245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:59.642313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:59.642379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:59.642434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:59.642438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:59.642503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:46:59.642562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:46:59.642625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:46:59.642694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:46:59.642740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:46:59.642747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:46:59.643039Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:59.643100Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341810296487065:7899]
2025-10-09T21:46:59.643127Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:46:59.643136Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:46:59.643179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 827 reason: error at write: Disconnected sessionId: src_id|1fe96581-ac4ebf9a-2acf06c2-4f5b4f24_0
2025-10-09T21:46:59.643388Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 827 sessionId: src_id|1fe96581-ac4ebf9a-2acf06c2-4f5b4f24_0 is DEAD
2025-10-09T21:46:59.643543Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fe96581-ac4ebf9a-2acf06c2-4f5b4f24_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:46:59.643553Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fe96581-ac4ebf9a-2acf06c2-4f5b4f24_0] Write session will restart in 0.006871s
2025-10-09T21:46:59.643577Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fe96581-ac4ebf9a-2acf06c2-4f5b4f24_0] Write session: Do CDS request
2025-10-09T21:46:59.643584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fe96581-ac4ebf9a-2acf06c2-4f5b4f24_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:59.643273Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:59.651720Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1fe96581-ac4ebf9a-2acf06c2-4f5b4f24_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:59.654179Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:59.654198Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 828
2025-10-09T21:46:59.654364Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 828 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:59.654423Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 828 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:59.654428Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=828 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:59.654433Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:59.654915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 828 sessionId:  describe result for acl check
2025-10-09T21:46:59.654958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:59.654965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:59.654967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:59.654973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341810296487117:8058] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:59.655595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341810296487117:8058] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:59.660407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:59.660672Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341810296487117:8058] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:59.663048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:59.663380Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:59.663392Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:59.663408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:59.663511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:59.663558Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341809934668695:7303]
2025-10-09T21:46:59.664171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:59.664237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341810296487117:8058] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:59.664256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341810296487117:8058] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:59.664260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341810296487117:8058] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:59.664270Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 828 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:59.664512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:59.664564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:59.664929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:59.664989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:59.664994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:59.664996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:59.664997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:59.664998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:59.664999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:59.665000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.665001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.665002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.665008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:59.665019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:59.665023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:59.665027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:59.665031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:59.665035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:59.665039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.665043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.665047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.665056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:59.665063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:59.665070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:59.665076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:59.665081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:59.665086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:59.665091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:59.665095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:59.665099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:59.665103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:59.665107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:59.665112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:59.665117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.665122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.665127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.665133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.665140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.665147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.665186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:59.673653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:59.674796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:59.674815Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:59.674818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:59.674824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:59.674832Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 827 [2:7559341809934668695:7303]
2025-10-09T21:46:59.674848Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:59.674870Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:59.674874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:59.674878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:59.674881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:59.674901Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:59.674913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:59.674922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:59.675000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.675009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.675011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.675014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:59.675102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:59.675113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.675127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.675141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.675159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:59.675181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341809934668689:7303]
2025-10-09T21:46:59.675188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341809934668689:7303]
2025-10-09T21:46:59.675194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341809934668689:7303]
2025-10-09T21:46:59.675212Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:59.675230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341810296487219:8058], now have 1 active actors on pipe
2025-10-09T21:46:59.675240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341810296487068:2432], now have 1 active actors on pipe
2025-10-09T21:46:59.675252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:59.675273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:59.675277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:59.675321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341809934668689:7303]
2025-10-09T21:46:59.675333Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:59.675384Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:59.675388Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:59.675391Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:59.675401Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:59.675401Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 827
2025-10-09T21:46:59.675403Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 827
2025-10-09T21:46:59.675638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:59.675653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:59.675695Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cb06954a-6d55ccae-ecbd98bc-b49029cf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:59.675710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:59.675714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:59.675717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:59.675729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:59.675755Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:59.675782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:59.676048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:59.676068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:59.676088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:59.676226Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 828 partition: 0 MaxSeqNo: 25 sessionId: src_id|cb06954a-6d55ccae-ecbd98bc-b49029cf_0
2025-10-09T21:46:59.676655Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fe96581-ac4ebf9a-2acf06c2-4f5b4f24_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cb06954a-6d55ccae-ecbd98bc-b49029cf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:59.679351Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cb06954a-6d55ccae-ecbd98bc-b49029cf_0] Write session: try to update token
2025-10-09T21:46:59.679371Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cb06954a-6d55ccae-ecbd98bc-b49029cf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:59.699572Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:46:59.699588Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:59.699601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341809934668689:7303]
2025-10-09T21:46:59.699657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:46:59.699675Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:46:59.699849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:46:59.699999Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:59.700144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:59.700289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:46:59.700488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:46:59.700539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:46:59.703441Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 828 sessionId: src_id|cb06954a-6d55ccae-ecbd98bc-b49029cf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:59.704598Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:59.704615Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:59.704617Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:59.704620Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:59.704631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341809934668689:7303]
2025-10-09T21:46:59.704636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341809934668689:7303]
2025-10-09T21:46:59.704640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341809934668689:7303]
2025-10-09T21:46:59.704687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:59.704686Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:59.704693Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:59.704696Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:59.704699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.704702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:59.704844Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:59.706436Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:59.743325Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341810296487229:7899]
2025-10-09T21:46:59.743865Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 827, pipe: [1:7559341810296487229:7899]
2025-10-09T21:46:59.875252Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:46:59.875621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:59.875641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:59.875666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341810296487229:7899], now have 1 active actors on pipe
2025-10-09T21:46:59.875919Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:46:59.876804Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb06954a-6d55ccae-ecbd98bc-b49029cf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:46:59.876816Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb06954a-6d55ccae-ecbd98bc-b49029cf_0] Write session will restart in 0.008321s
2025-10-09T21:46:59.876845Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb06954a-6d55ccae-ecbd98bc-b49029cf_0] Write session: Do CDS request
2025-10-09T21:46:59.876851Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb06954a-6d55ccae-ecbd98bc-b49029cf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:46:59.876333Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341810296487229:7899]
2025-10-09T21:46:59.876365Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:46:59.876435Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 828 reason: status is not ok: tablet will be restarted right now sessionId: src_id|cb06954a-6d55ccae-ecbd98bc-b49029cf_0
2025-10-09T21:46:59.876625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 828 sessionId: src_id|cb06954a-6d55ccae-ecbd98bc-b49029cf_0 is DEAD
2025-10-09T21:46:59.876808Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:46:59.887192Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cb06954a-6d55ccae-ecbd98bc-b49029cf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:59.887658Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:46:59.887676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 829
2025-10-09T21:46:59.887881Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 829 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:46:59.887926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 829 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:46:59.887931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=829 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:46:59.887936Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:46:59.888493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 829 sessionId:  describe result for acl check
2025-10-09T21:46:59.888529Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:46:59.888531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:46:59.888533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:46:59.888540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341810296487300:8063] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:46:59.889100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341810296487300:8063] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:46:59.896480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341810296487300:8063] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:46:59.897522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:46:59.900913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:46:59.901180Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:46:59.901184Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:46:59.901199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:46:59.901270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:46:59.901308Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341809934668860:7311]
2025-10-09T21:46:59.901900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:46:59.902379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:46:59.902439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:46:59.902787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:46:59.902851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:59.902853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:59.902854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:59.902855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:59.902857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:59.902858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:59.902859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.902861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.902862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.902869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:59.902878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:59.902882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:59.902886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:59.902890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:59.902894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:59.902897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.902900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.902904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.902911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:59.902920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:46:59.902927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:59.902931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:46:59.902935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:59.902941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:46:59.902945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:59.902951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:46:59.902956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:59.902962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:46:59.902966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:59.902972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:46:59.902976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.902981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.902987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.902992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.902997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.903002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.903030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:46:59.909744Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341810296487300:8063] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:46:59.909769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341810296487300:8063] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:46:59.910163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:46:59.909773Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341810296487300:8063] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:46:59.909786Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 829 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:46:59.911532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:46:59.911546Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:46:59.911549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:46:59.911553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:46:59.911560Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 828 [2:7559341809934668860:7311]
2025-10-09T21:46:59.911575Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:46:59.911593Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:46:59.911596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:59.911600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:59.911602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:59.911621Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:46:59.911635Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:46:59.911643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:46:59.911705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.911712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.911715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.911718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:46:59.911789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:46:59.911799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.911812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:46:59.911822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:46:59.911840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:46:59.911865Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341809934668855:7311]
2025-10-09T21:46:59.911867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341809934668855:7311]
2025-10-09T21:46:59.911869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341809934668855:7311]
2025-10-09T21:46:59.911879Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:46:59.911889Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:46:59.911892Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:46:59.911894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:46:59.911894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:46:59.911908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:46:59.911912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:46:59.911924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341810296487388:8063], now have 1 active actors on pipe
2025-10-09T21:46:59.911927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341810296487234:2432], now have 1 active actors on pipe
2025-10-09T21:46:59.911965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341809934668855:7311]
2025-10-09T21:46:59.911971Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:46:59.912005Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:46:59.911989Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 828
2025-10-09T21:46:59.912011Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 828
2025-10-09T21:46:59.912255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:59.912271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:59.912326Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7e817af2-e4f45346-4450951a-46a27b05_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:46:59.912339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:46:59.912344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:46:59.912347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:46:59.912369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:46:59.912384Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:46:59.912404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:59.912676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:46:59.912688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:46:59.912710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:46:59.912844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 829 partition: 0 MaxSeqNo: 25 sessionId: src_id|7e817af2-e4f45346-4450951a-46a27b05_0
2025-10-09T21:46:59.914616Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb06954a-6d55ccae-ecbd98bc-b49029cf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7e817af2-e4f45346-4450951a-46a27b05_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:46:59.917152Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e817af2-e4f45346-4450951a-46a27b05_0] Write session: try to update token
2025-10-09T21:46:59.917174Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e817af2-e4f45346-4450951a-46a27b05_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:46:59.929053Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:46:59.929070Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:46:59.929072Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:46:59.929074Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:46:59.929085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341809934668855:7311]
2025-10-09T21:46:59.929090Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341809934668855:7311]
2025-10-09T21:46:59.929094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341809934668855:7311]
2025-10-09T21:46:59.929144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:46:59.929146Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:46:59.929152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:46:59.929153Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:46:59.929156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:46:59.929157Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:46:59.929287Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:46:59.936652Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 829 sessionId: src_id|7e817af2-e4f45346-4450951a-46a27b05_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:46:59.939257Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:46:59.976823Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341810296487409:7899]
2025-10-09T21:46:59.977163Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 828, pipe: [1:7559341810296487409:7899]
2025-10-09T21:47:00.093362Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:00.093709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:00.093727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:00.093776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341810296487409:7899], now have 1 active actors on pipe
2025-10-09T21:47:00.094075Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:00.094244Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341810296487409:7899]
2025-10-09T21:47:00.094276Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:00.094314Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:00.094317Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:00.094393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 829 reason: error at write: Disconnected sessionId: src_id|7e817af2-e4f45346-4450951a-46a27b05_0
2025-10-09T21:47:00.094582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 829 sessionId: src_id|7e817af2-e4f45346-4450951a-46a27b05_0 is DEAD
2025-10-09T21:47:00.094726Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e817af2-e4f45346-4450951a-46a27b05_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:00.094753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e817af2-e4f45346-4450951a-46a27b05_0] Write session will restart in 0.007326s
2025-10-09T21:47:00.094784Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e817af2-e4f45346-4450951a-46a27b05_0] Write session: Do CDS request
2025-10-09T21:47:00.094790Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e817af2-e4f45346-4450951a-46a27b05_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:00.103792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e817af2-e4f45346-4450951a-46a27b05_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:00.104886Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:00.104902Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 830
2025-10-09T21:47:00.105148Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 830 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:00.105204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 830 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:00.105211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=830 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:00.105215Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:00.106247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 830 sessionId:  describe result for acl check
2025-10-09T21:47:00.106304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:00.106306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:00.106308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:00.106314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341814591454770:8074] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:00.106918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341814591454770:8074] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:00.110926Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341814591454770:8074] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:00.115021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:00.115840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341814591454770:8074] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:00.115864Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341814591454770:8074] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:00.115869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341814591454770:8074] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:00.115879Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 830 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:00.118229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:00.118498Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:00.118510Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:00.118524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:00.118597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:00.118644Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341814229636335:7319]
2025-10-09T21:47:00.119342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:00.122848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:00.122951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:00.123413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:00.123491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:00.123500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:00.123502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:00.123503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:00.123505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:00.123507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:00.123508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.123509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.123511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.123520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:00.123532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:00.123536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:00.123540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:00.123543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:00.123548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:00.123552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.123558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.123563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.123573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:00.123583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:00.123591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:00.123598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:00.123603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:00.123608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:00.123614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:00.123621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:00.123634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:00.123641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:00.123646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:00.123653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:00.123659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.123665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.123670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.123676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.123682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.123690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.123725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:00.133035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:00.134260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:00.134281Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:00.134284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:00.134290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:00.134298Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 829 [2:7559341814229636335:7319]
2025-10-09T21:47:00.134316Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:00.134339Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:00.134342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:00.134346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:00.134350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:00.134370Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:00.134376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:00.134385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:00.134463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.134476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.134478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.134481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:00.134561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:00.134572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.134588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.134602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.134623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:00.134645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341814229636328:7319]
2025-10-09T21:47:00.134648Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341814229636328:7319]
2025-10-09T21:47:00.134651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341814229636328:7319]
2025-10-09T21:47:00.134662Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:00.134673Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:00.134676Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:00.134678Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:00.134679Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:00.134695Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:00.134698Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:00.134711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341814591454869:8074], now have 1 active actors on pipe
2025-10-09T21:47:00.134716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341814591454722:2432], now have 1 active actors on pipe
2025-10-09T21:47:00.134773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341814229636328:7319]
2025-10-09T21:47:00.134783Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:00.134891Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 829
2025-10-09T21:47:00.134928Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 829
2025-10-09T21:47:00.134994Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:00.135362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:00.135376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:00.135414Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b53faee3-ac40234c-651b22c4-cb36d691_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:00.135430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:00.135435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:00.135443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:00.135454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:00.135473Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:00.135705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:00.137102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:00.137116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:00.137146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:00.137308Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 830 partition: 0 MaxSeqNo: 25 sessionId: src_id|b53faee3-ac40234c-651b22c4-cb36d691_0
2025-10-09T21:47:00.137753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e817af2-e4f45346-4450951a-46a27b05_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b53faee3-ac40234c-651b22c4-cb36d691_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:00.140580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b53faee3-ac40234c-651b22c4-cb36d691_0] Write session: try to update token
2025-10-09T21:47:00.140605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b53faee3-ac40234c-651b22c4-cb36d691_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:00.150646Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:00.150665Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:00.150668Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:00.150670Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:00.150681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341814229636328:7319]
2025-10-09T21:47:00.150687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341814229636328:7319]
2025-10-09T21:47:00.150694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341814229636328:7319]
2025-10-09T21:47:00.150748Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:00.150754Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:00.150754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:00.150761Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:00.150766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.150770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:00.150979Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:00.160941Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 830 sessionId: src_id|b53faee3-ac40234c-651b22c4-cb36d691_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:00.163628Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:00.194346Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341814591454888:7899]
2025-10-09T21:47:00.195203Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 829, pipe: [1:7559341814591454888:7899]
2025-10-09T21:47:00.317687Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:00.318017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:00.318050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:00.318074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341814591454888:7899], now have 1 active actors on pipe
2025-10-09T21:47:00.318467Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341814591454888:7899]
2025-10-09T21:47:00.318498Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:00.318324Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:00.319027Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:00.319112Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 830 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b53faee3-ac40234c-651b22c4-cb36d691_0
2025-10-09T21:47:00.319292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 830 sessionId: src_id|b53faee3-ac40234c-651b22c4-cb36d691_0 is DEAD
2025-10-09T21:47:00.319463Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b53faee3-ac40234c-651b22c4-cb36d691_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:00.319483Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b53faee3-ac40234c-651b22c4-cb36d691_0] Write session will restart in 0.007549s
2025-10-09T21:47:00.319509Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b53faee3-ac40234c-651b22c4-cb36d691_0] Write session: Do CDS request
2025-10-09T21:47:00.319515Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b53faee3-ac40234c-651b22c4-cb36d691_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:00.328591Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b53faee3-ac40234c-651b22c4-cb36d691_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:00.328849Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:00.328866Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 831
2025-10-09T21:47:00.329000Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 831 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:00.329044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 831 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:00.329050Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=831 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:00.329053Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:00.329527Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 831 sessionId:  describe result for acl check
2025-10-09T21:47:00.329569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:00.329577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:00.329579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:00.329585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341814591454915:8079] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:00.330205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341814591454915:8079] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:00.336244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341814591454915:8079] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:00.339759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:00.339675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341814591454915:8079] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:00.339695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341814591454915:8079] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:00.339699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341814591454915:8079] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:00.339709Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 831 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:00.341939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:00.342195Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:00.342207Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:00.342219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:00.342277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:00.342323Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341814229636490:7321]
2025-10-09T21:47:00.342987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:00.344187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:00.344251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:00.344641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:00.344702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:00.344708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:00.344709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:00.344710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:00.344712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:00.344713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:00.344714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.344716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.344717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.344724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:00.344735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:00.344740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:00.344744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:00.344748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:00.344752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:00.344756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.344760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.344765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.344774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:00.344783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:00.344789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:00.344795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:00.344803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:00.344808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:00.344813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:00.344819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:00.344824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:00.344830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:00.344837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:00.344843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:00.344849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.344861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.344865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.344870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.344875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.344881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.344909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:00.353507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:00.355022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:00.355048Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:00.355051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:00.355057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:00.355065Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 830 [2:7559341814229636490:7321]
2025-10-09T21:47:00.355082Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:00.355105Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:00.355109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:00.355112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:00.355115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:00.355138Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:00.355155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:00.355174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:00.355249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.355251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.355253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.355256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:00.355351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:00.355354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.355369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.355380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.355395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:00.355416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341814229636484:7321]
2025-10-09T21:47:00.355419Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341814229636484:7321]
2025-10-09T21:47:00.355422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341814229636484:7321]
2025-10-09T21:47:00.355430Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:00.355440Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:00.355443Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:00.355445Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:00.355445Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:00.355460Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:00.355463Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:00.355476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341814591454889:2432], now have 1 active actors on pipe
2025-10-09T21:47:00.355521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341814591455039:8079], now have 1 active actors on pipe
2025-10-09T21:47:00.355524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341814229636484:7321]
2025-10-09T21:47:00.355532Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:00.355749Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:00.355786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:00.355793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:00.355821Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1ebfe6b5-2cf26321-11383c4c-8dfb84cc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:00.355836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:00.355841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:00.355844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:00.355855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:00.355867Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:00.355889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:00.356184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:00.356193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:00.356214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:00.357684Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b53faee3-ac40234c-651b22c4-cb36d691_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1ebfe6b5-2cf26321-11383c4c-8dfb84cc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:00.355650Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 830
2025-10-09T21:47:00.356029Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 830
2025-10-09T21:47:00.356356Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 831 partition: 0 MaxSeqNo: 25 sessionId: src_id|1ebfe6b5-2cf26321-11383c4c-8dfb84cc_0
2025-10-09T21:47:00.364317Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ebfe6b5-2cf26321-11383c4c-8dfb84cc_0] Write session: try to update token
2025-10-09T21:47:00.364344Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ebfe6b5-2cf26321-11383c4c-8dfb84cc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:00.377777Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:00.377796Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:00.377799Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:00.377801Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:00.377813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341814229636484:7321]
2025-10-09T21:47:00.377820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341814229636484:7321]
2025-10-09T21:47:00.377825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341814229636484:7321]
2025-10-09T21:47:00.377874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:00.377889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.377900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:00.378081Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:00.379066Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:00.379086Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:00.379090Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:00.392768Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 831 sessionId: src_id|1ebfe6b5-2cf26321-11383c4c-8dfb84cc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:00.395514Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:00.422115Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341814591455053:7899]
2025-10-09T21:47:00.422539Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 830, pipe: [1:7559341814591455053:7899]
2025-10-09T21:47:00.552219Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:00.552758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:00.552776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:00.552801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341814591455053:7899], now have 1 active actors on pipe
2025-10-09T21:47:00.553127Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:00.553780Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341814591455053:7899]
2025-10-09T21:47:00.553818Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:00.553863Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:00.553976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 831 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1ebfe6b5-2cf26321-11383c4c-8dfb84cc_0
2025-10-09T21:47:00.554191Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 831 sessionId: src_id|1ebfe6b5-2cf26321-11383c4c-8dfb84cc_0 is DEAD
2025-10-09T21:47:00.554563Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ebfe6b5-2cf26321-11383c4c-8dfb84cc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:00.554574Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ebfe6b5-2cf26321-11383c4c-8dfb84cc_0] Write session will restart in 0.005439s
2025-10-09T21:47:00.554601Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ebfe6b5-2cf26321-11383c4c-8dfb84cc_0] Write session: Do CDS request
2025-10-09T21:47:00.554608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ebfe6b5-2cf26321-11383c4c-8dfb84cc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:00.561676Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ebfe6b5-2cf26321-11383c4c-8dfb84cc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:00.561966Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:00.561983Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 832
2025-10-09T21:47:00.562149Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 832 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:00.562194Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 832 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:00.562204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=832 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:00.562207Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:00.562778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 832 sessionId:  describe result for acl check
2025-10-09T21:47:00.562832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:00.562840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:00.562841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:00.562848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341814591455075:8083] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:00.563453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341814591455075:8083] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:00.568018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341814591455075:8083] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:00.573067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341814591455075:8083] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:00.573086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341814591455075:8083] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:00.573090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341814591455075:8083] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:00.573100Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 832 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:00.577540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:00.580772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:00.581085Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:00.581096Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:00.581112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:00.581184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:00.581221Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341814229636652:7323]
2025-10-09T21:47:00.581816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:00.582397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:00.582464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:00.582835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:00.582914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:00.582921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:00.582922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:00.582923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:00.582925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:00.582926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:00.582927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.582928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.582929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.582936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:00.582947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:00.582951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:00.582954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:00.582958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:00.582961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:00.582965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.582968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.582972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.582980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:00.582989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:00.582997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:00.583003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:00.583008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:00.583015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:00.583020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:00.583025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:00.583031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:00.583035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:00.583040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:00.583044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:00.583050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.583055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.583058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.583063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.583068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.583073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.583106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:00.590936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:00.592530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:00.592548Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:00.592550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:00.592555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:00.592563Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 831 [2:7559341814229636652:7323]
2025-10-09T21:47:00.592583Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:00.592608Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:00.592611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:00.592616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:00.592618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:00.592638Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:00.592645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:00.592653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:00.592728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.592739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.592741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.592745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:00.592832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:00.592842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.592858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.592868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.592884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:00.592907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341814229636647:7323]
2025-10-09T21:47:00.592909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341814229636647:7323]
2025-10-09T21:47:00.592912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341814229636647:7323]
2025-10-09T21:47:00.592922Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:00.592939Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:00.592947Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:00.592950Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:00.592952Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:00.592954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:00.592958Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:00.592971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341814591455060:2432], now have 1 active actors on pipe
2025-10-09T21:47:00.592976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341814591455178:8083], now have 1 active actors on pipe
2025-10-09T21:47:00.593021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341814229636647:7323]
2025-10-09T21:47:00.593038Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:00.593043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:00.593059Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 831
2025-10-09T21:47:00.593068Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 831
2025-10-09T21:47:00.593672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 832 partition: 0 MaxSeqNo: 25 sessionId: src_id|54d8d274-5b8d6113-44ebae5a-f5bcd988_0
2025-10-09T21:47:00.594261Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ebfe6b5-2cf26321-11383c4c-8dfb84cc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|54d8d274-5b8d6113-44ebae5a-f5bcd988_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:00.593174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:00.593200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:00.593234Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|54d8d274-5b8d6113-44ebae5a-f5bcd988_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:00.593243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:00.593249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:00.593251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:00.593263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:00.593275Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:00.593312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:00.593511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:00.593529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:00.593551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:00.596873Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54d8d274-5b8d6113-44ebae5a-f5bcd988_0] Write session: try to update token
2025-10-09T21:47:00.596895Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54d8d274-5b8d6113-44ebae5a-f5bcd988_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:00.616903Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:00.616918Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:00.616932Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341814229636647:7323]
2025-10-09T21:47:00.616988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:00.616992Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:00.617169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:00.617322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:00.618077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:00.618258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:00.618482Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:00.618540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:00.621575Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 832 sessionId: src_id|54d8d274-5b8d6113-44ebae5a-f5bcd988_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:00.624162Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:00.624652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:00.624681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:00.624732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:00.625249Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:00.629608Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:00.629626Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:00.629629Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:00.629631Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:00.629641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341814229636647:7323]
2025-10-09T21:47:00.629646Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341814229636647:7323]
2025-10-09T21:47:00.629649Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341814229636647:7323]
2025-10-09T21:47:00.629699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:00.629716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.629719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:00.629750Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:00.629765Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:00.629769Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:00.629856Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:00.654133Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341814591455225:7899]
2025-10-09T21:47:00.654696Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 831, pipe: [1:7559341814591455225:7899]
2025-10-09T21:47:00.802426Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:00.802877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341814591455225:7899], now have 1 active actors on pipe
2025-10-09T21:47:00.811548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:00.811582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:00.811757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:00.811816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:00.811872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:00.811929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:00.812215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:00.812222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:00.812287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:00.812349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:00.812408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:00.812469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:00.812509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:00.812512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:00.812578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:00.812632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:00.812685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:00.812742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:00.812790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:00.812793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:00.812854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:00.813200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:00.813297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:00.813352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:00.813465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:00.813469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:00.813539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:00.813600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:00.813693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:00.813756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:00.813821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:00.813824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:00.813889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:00.813976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:00.814349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:00.814450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:00.814517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:00.814520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:00.814616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:00.814670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:00.814727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:00.814826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:00.814881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:00.814883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:00.815013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:00.815066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:00.815121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:00.815196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:00.815234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:00.815237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:00.815305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:00.815358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:00.815414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:00.815475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:00.815513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:00.815516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:00.815580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:00.815634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:00.815699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:00.815764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:00.815890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:00.815895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:00.816687Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341814591455225:7899]
2025-10-09T21:47:00.816743Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:00.816763Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:00.816773Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:00.816804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 832 reason: error at write: Disconnected sessionId: src_id|54d8d274-5b8d6113-44ebae5a-f5bcd988_0
2025-10-09T21:47:00.816992Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 832 sessionId: src_id|54d8d274-5b8d6113-44ebae5a-f5bcd988_0 is DEAD
2025-10-09T21:47:00.816835Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:00.818330Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54d8d274-5b8d6113-44ebae5a-f5bcd988_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:00.818343Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54d8d274-5b8d6113-44ebae5a-f5bcd988_0] Write session will restart in 0.005439s
2025-10-09T21:47:00.818369Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54d8d274-5b8d6113-44ebae5a-f5bcd988_0] Write session: Do CDS request
2025-10-09T21:47:00.818375Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54d8d274-5b8d6113-44ebae5a-f5bcd988_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:00.824600Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54d8d274-5b8d6113-44ebae5a-f5bcd988_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:00.825194Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:00.825221Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 833
2025-10-09T21:47:00.825403Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 833 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:00.825463Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 833 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:00.825471Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=833 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:00.825475Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:00.826071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 833 sessionId:  describe result for acl check
2025-10-09T21:47:00.826106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:00.826108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:00.826110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:00.826116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341814591455237:8088] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:00.826729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341814591455237:8088] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:00.829954Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341814591455237:8088] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:00.839263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341814591455237:8088] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:00.839296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341814591455237:8088] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:00.839301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341814591455237:8088] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:00.839313Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 833 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:00.850212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:00.852875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:00.853131Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:00.853135Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:00.853150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:00.853223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:00.853264Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341814229636808:7327]
2025-10-09T21:47:00.853849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:00.854393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:00.854448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:00.854817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:00.854871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:00.854872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:00.854873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:00.854874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:00.854875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:00.854876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:00.854877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.854878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.854879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.854884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:00.854894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:00.854898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:00.854902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:00.854906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:00.854910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:00.854913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.854916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.854919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.854928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:00.854936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:00.854942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:00.854949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:00.854952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:00.854957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:00.854963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:00.854969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:00.854974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:00.854979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:00.854985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:00.854990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:00.854997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.855002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.855008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.855013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.855018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.855024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.855054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:00.863163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:00.864330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:00.864348Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:00.864351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:00.864356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:00.864377Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 832 [2:7559341814229636808:7327]
2025-10-09T21:47:00.864394Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:00.864417Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:00.864425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:00.864429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:00.864432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:00.864453Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:00.864465Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:00.864473Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:00.864547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.864559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.864561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.864564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:00.864647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:00.864656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.864670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:00.864681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:00.864698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:00.864720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341814229636803:7327]
2025-10-09T21:47:00.864722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341814229636803:7327]
2025-10-09T21:47:00.864725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341814229636803:7327]
2025-10-09T21:47:00.864735Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:00.864752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:00.864767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:00.864771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:00.864816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341814229636803:7327]
2025-10-09T21:47:00.864819Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:00.865461Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:00.865469Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:00.865471Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:00.865478Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:00.865879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341814591455230:2432], now have 1 active actors on pipe
2025-10-09T21:47:00.865890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341814591455279:8088], now have 1 active actors on pipe
2025-10-09T21:47:00.866089Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 832
2025-10-09T21:47:00.866142Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 832
2025-10-09T21:47:00.866428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:00.866440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:00.866484Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d5bf3b4e-fbe97cff-4037fe58-5edf8e73_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:00.866496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:00.866501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:00.866504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:00.866517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:00.866531Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:00.866563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:00.867064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:00.867074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:00.867095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:00.867267Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 833 partition: 0 MaxSeqNo: 25 sessionId: src_id|d5bf3b4e-fbe97cff-4037fe58-5edf8e73_0
2025-10-09T21:47:00.867660Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54d8d274-5b8d6113-44ebae5a-f5bcd988_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d5bf3b4e-fbe97cff-4037fe58-5edf8e73_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:00.870269Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d5bf3b4e-fbe97cff-4037fe58-5edf8e73_0] Write session: try to update token
2025-10-09T21:47:00.870287Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d5bf3b4e-fbe97cff-4037fe58-5edf8e73_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:00.882751Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:00.882771Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:00.882774Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:00.882776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:00.882788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341814229636803:7327]
2025-10-09T21:47:00.882793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341814229636803:7327]
2025-10-09T21:47:00.882798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341814229636803:7327]
2025-10-09T21:47:00.882851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:00.882851Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:00.882858Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:00.882859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:00.882862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:00.882862Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:00.883022Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:00.892316Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 833 sessionId: src_id|d5bf3b4e-fbe97cff-4037fe58-5edf8e73_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:00.895005Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:00.916777Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341814591455396:7899]
2025-10-09T21:47:00.917322Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 832, pipe: [1:7559341814591455396:7899]
2025-10-09T21:47:01.075762Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:01.076211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:01.076227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:01.076253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341814591455396:7899], now have 1 active actors on pipe
2025-10-09T21:47:01.076499Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:01.078678Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d5bf3b4e-fbe97cff-4037fe58-5edf8e73_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:01.078155Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341814591455396:7899]
2025-10-09T21:47:01.078186Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:01.078221Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:01.078297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 833 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d5bf3b4e-fbe97cff-4037fe58-5edf8e73_0
2025-10-09T21:47:01.078693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d5bf3b4e-fbe97cff-4037fe58-5edf8e73_0] Write session will restart in 0.005256s
2025-10-09T21:47:01.078495Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 833 sessionId: src_id|d5bf3b4e-fbe97cff-4037fe58-5edf8e73_0 is DEAD
2025-10-09T21:47:01.078742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d5bf3b4e-fbe97cff-4037fe58-5edf8e73_0] Write session: Do CDS request
2025-10-09T21:47:01.078750Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d5bf3b4e-fbe97cff-4037fe58-5edf8e73_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:01.085599Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d5bf3b4e-fbe97cff-4037fe58-5edf8e73_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:01.085755Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:01.085777Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 834
2025-10-09T21:47:01.085942Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 834 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:01.085995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 834 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:01.086000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=834 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:01.086004Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:01.086561Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 834 sessionId:  describe result for acl check
2025-10-09T21:47:01.086607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:01.086615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:01.086618Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:01.086628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341818886422717:8098] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:01.087210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341818886422717:8098] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:01.099122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341818886422717:8098] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:01.107889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341818886422717:8098] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:01.107919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341818886422717:8098] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:01.107923Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341818886422717:8098] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:01.107933Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 834 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:01.126997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:01.129693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:01.129967Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:01.129979Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:01.129993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:01.130071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:01.130113Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341818524604300:7339]
2025-10-09T21:47:01.130720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:01.131231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:01.131296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:01.131624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:01.131689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:01.131690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:01.131692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:01.131694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:01.131695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:01.131696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:01.131698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.131699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.131700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.131707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:01.131719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:01.131724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:01.131729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:01.131733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:01.131737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:01.131741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.131745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.131749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.131756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:01.131764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:01.131772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:01.131778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:01.131783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:01.131788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:01.131793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:01.131798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:01.131803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:01.131809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:01.131812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:01.131817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:01.131822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.131826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.131831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.131836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.131864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.131871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.131898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:01.145628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:01.147181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:01.147202Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:01.147205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:01.147210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:01.147217Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 833 [2:7559341818524604300:7339]
2025-10-09T21:47:01.147235Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:01.147255Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:01.147258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:01.147262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:01.147264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:01.147284Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:01.147291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:01.147298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:01.147366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.147375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.147378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.147381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:01.147456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:01.147465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.147479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.147495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.147519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:01.147545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341818886422769:8098], now have 1 active actors on pipe
2025-10-09T21:47:01.147555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341818886422712:2432], now have 1 active actors on pipe
2025-10-09T21:47:01.147564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341818524604294:7339]
2025-10-09T21:47:01.147567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341818524604294:7339]
2025-10-09T21:47:01.147569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341818524604294:7339]
2025-10-09T21:47:01.147582Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:01.147652Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 833
2025-10-09T21:47:01.147598Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:01.147600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:01.147659Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 833
2025-10-09T21:47:01.147601Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:01.147603Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:01.147616Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:01.147619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:01.147670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341818524604294:7339]
2025-10-09T21:47:01.147678Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:01.147685Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:01.147825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:01.147852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:01.147891Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8fc3d0be-f59b9f87-7d7f3356-3a3e96a8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:01.147907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:01.147912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:01.147914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:01.147944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:01.147963Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:01.148004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:01.148210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:01.148224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:01.148248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:01.148878Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d5bf3b4e-fbe97cff-4037fe58-5edf8e73_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8fc3d0be-f59b9f87-7d7f3356-3a3e96a8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:01.151640Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8fc3d0be-f59b9f87-7d7f3356-3a3e96a8_0] Write session: try to update token
2025-10-09T21:47:01.151661Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8fc3d0be-f59b9f87-7d7f3356-3a3e96a8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:01.148388Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 834 partition: 0 MaxSeqNo: 25 sessionId: src_id|8fc3d0be-f59b9f87-7d7f3356-3a3e96a8_0
2025-10-09T21:47:01.166757Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:01.166776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:01.166779Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:01.166781Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:01.166792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341818524604294:7339]
2025-10-09T21:47:01.166796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341818524604294:7339]
2025-10-09T21:47:01.166800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341818524604294:7339]
2025-10-09T21:47:01.166851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:01.166857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.166860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:01.167023Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:01.169175Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:01.169194Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:01.169199Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:01.170714Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 834 sessionId: src_id|8fc3d0be-f59b9f87-7d7f3356-3a3e96a8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:01.173301Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:01.178902Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341818886422893:7899]
2025-10-09T21:47:01.181142Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 833, pipe: [1:7559341818886422893:7899]
2025-10-09T21:47:01.404188Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:01.404659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:01.404675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:01.404703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341818886422893:7899], now have 1 active actors on pipe
2025-10-09T21:47:01.404992Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:01.405103Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341818886422893:7899]
2025-10-09T21:47:01.405274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:01.405430Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:01.405519Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 834 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8fc3d0be-f59b9f87-7d7f3356-3a3e96a8_0
2025-10-09T21:47:01.405840Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fc3d0be-f59b9f87-7d7f3356-3a3e96a8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:01.405854Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fc3d0be-f59b9f87-7d7f3356-3a3e96a8_0] Write session will restart in 0.009240s
2025-10-09T21:47:01.405878Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fc3d0be-f59b9f87-7d7f3356-3a3e96a8_0] Write session: Do CDS request
2025-10-09T21:47:01.405884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fc3d0be-f59b9f87-7d7f3356-3a3e96a8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:01.405715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 834 sessionId: src_id|8fc3d0be-f59b9f87-7d7f3356-3a3e96a8_0 is DEAD
2025-10-09T21:47:01.416670Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8fc3d0be-f59b9f87-7d7f3356-3a3e96a8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:01.416802Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:01.416820Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 835
2025-10-09T21:47:01.416939Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 835 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:01.416972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 835 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:01.416982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=835 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:01.416987Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:01.417517Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 835 sessionId:  describe result for acl check
2025-10-09T21:47:01.417562Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:01.417569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:01.417571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:01.417577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341818886422971:8104] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:01.418219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341818886422971:8104] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:01.421464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341818886422971:8104] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:01.422736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:01.425078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:01.425341Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:01.425353Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:01.425368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:01.425421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341818886422971:8104] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:01.425439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341818886422971:8104] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:01.425444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341818886422971:8104] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:01.425454Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 835 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:01.425449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:01.425495Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341818524604457:7343]
2025-10-09T21:47:01.426144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:01.426600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:01.426663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:01.427200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:01.427273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:01.427293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:01.427294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:01.427295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:01.427297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:01.427298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:01.427299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.427301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.427302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.427308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:01.427320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:01.427323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:01.427327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:01.427330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:01.427333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:01.427337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.427340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.427343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.427351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:01.427359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:01.427365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:01.427371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:01.427376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:01.427381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:01.427385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:01.427390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:01.427396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:01.427403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:01.427407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:01.427412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:01.427418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.427423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.427427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.427431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.427435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.427441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.427469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:01.434535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:01.435729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:01.435744Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:01.435747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:01.435751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:01.435759Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 834 [2:7559341818524604457:7343]
2025-10-09T21:47:01.435775Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:01.435795Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:01.435798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:01.435802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:01.435804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:01.435838Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:01.435849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:01.435856Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:01.435928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.435935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.435937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.435940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:01.436014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:01.436021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.436034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.436048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.436067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:01.436090Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341818524604452:7343]
2025-10-09T21:47:01.436096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341818524604452:7343]
2025-10-09T21:47:01.436099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341818524604452:7343]
2025-10-09T21:47:01.436107Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:01.436124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:01.436125Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:01.436128Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:01.436131Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:01.436142Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:01.436145Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:01.436156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341818886422898:2432], now have 1 active actors on pipe
2025-10-09T21:47:01.436160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341818886423054:8104], now have 1 active actors on pipe
2025-10-09T21:47:01.436210Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 834
2025-10-09T21:47:01.436226Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 834
2025-10-09T21:47:01.436210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341818524604452:7343]
2025-10-09T21:47:01.436227Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:01.436282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:01.436352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:01.436361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:01.436395Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|edd54d5d-9dbdfcbf-94320b76-6096eab2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:01.436406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:01.436411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:01.436413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:01.436426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:01.436439Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:01.436479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:01.436703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:01.436714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:01.436738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:01.436848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 835 partition: 0 MaxSeqNo: 25 sessionId: src_id|edd54d5d-9dbdfcbf-94320b76-6096eab2_0
2025-10-09T21:47:01.437233Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fc3d0be-f59b9f87-7d7f3356-3a3e96a8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|edd54d5d-9dbdfcbf-94320b76-6096eab2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:01.440078Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|edd54d5d-9dbdfcbf-94320b76-6096eab2_0] Write session: try to update token
2025-10-09T21:47:01.440099Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|edd54d5d-9dbdfcbf-94320b76-6096eab2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:01.449585Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:01.449602Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:01.449613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341818524604452:7343]
2025-10-09T21:47:01.449676Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:01.449678Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:01.449882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:01.450081Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:01.450230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:01.450413Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:01.450696Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:01.450751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:01.455896Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 835 sessionId: src_id|edd54d5d-9dbdfcbf-94320b76-6096eab2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:01.458638Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:01.459158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:01.459179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:01.459233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:01.459339Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:01.465071Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:01.465090Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:01.465092Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:01.465095Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:01.465110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341818524604452:7343]
2025-10-09T21:47:01.465115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341818524604452:7343]
2025-10-09T21:47:01.465120Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341818524604452:7343]
2025-10-09T21:47:01.465171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:01.465179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.465182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:01.465180Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:01.465187Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:01.465192Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:01.465324Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:01.504026Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341818886423065:7899]
2025-10-09T21:47:01.504559Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 834, pipe: [1:7559341818886423065:7899]
2025-10-09T21:47:01.628180Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:01.636149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:01.636180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:01.636359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:01.636440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:01.636510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:01.636575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:01.636619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:01.636625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:01.636701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:01.636763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:01.636827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:01.636896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:01.636934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:01.636938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:01.637009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:01.637070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:01.637133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:01.637193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:01.637234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:01.637237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:01.637310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:01.637375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:01.637436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:01.637497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:01.637536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:01.637539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:01.637611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:01.637671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:01.637730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:01.637793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:01.637833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:01.637836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:01.637944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:01.638003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:01.638237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:01.639963Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:01.639971Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341818886423065:7899]
2025-10-09T21:47:01.640000Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:01.640002Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:01.640044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 835 reason: error at write: Disconnected sessionId: src_id|edd54d5d-9dbdfcbf-94320b76-6096eab2_0
2025-10-09T21:47:01.640246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 835 sessionId: src_id|edd54d5d-9dbdfcbf-94320b76-6096eab2_0 is DEAD
2025-10-09T21:47:01.638313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:01.638358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:01.638361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:01.638429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:01.638483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:01.638544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:01.638605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:01.638671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:01.638676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:01.638739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:01.638794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:01.638853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:01.638918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:01.638979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:01.638984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:01.639051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:01.639103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:01.639162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:01.639221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:01.639265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:01.639268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:01.639336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:01.639391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:01.639453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:01.639513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:01.639559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:01.639562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:01.639630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341818886423065:7899], now have 1 active actors on pipe
2025-10-09T21:47:01.640709Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:01.642120Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edd54d5d-9dbdfcbf-94320b76-6096eab2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:01.642134Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edd54d5d-9dbdfcbf-94320b76-6096eab2_0] Write session will restart in 0.008302s
2025-10-09T21:47:01.642159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edd54d5d-9dbdfcbf-94320b76-6096eab2_0] Write session: Do CDS request
2025-10-09T21:47:01.642165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edd54d5d-9dbdfcbf-94320b76-6096eab2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:01.651861Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|edd54d5d-9dbdfcbf-94320b76-6096eab2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:01.652016Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:01.652033Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 836
2025-10-09T21:47:01.652431Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 836 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:01.652478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 836 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:01.652493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=836 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:01.652497Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:01.653031Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 836 sessionId:  describe result for acl check
2025-10-09T21:47:01.653068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:01.653070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:01.653072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:01.653079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341818886423124:8108] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:01.653688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341818886423124:8108] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:01.659257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341818886423124:8108] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:01.660581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:01.662949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:01.663193Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:01.663204Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:01.663215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:01.663275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:01.663309Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341818524604622:7347]
2025-10-09T21:47:01.663457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341818886423124:8108] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:01.663476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341818886423124:8108] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:01.663480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341818886423124:8108] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:01.663490Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 836 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:01.663887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:01.664326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:01.664393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:01.664750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:01.664816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:01.664823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:01.664825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:01.664826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:01.664827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:01.664828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:01.664829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.664830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.664832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.664837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:01.664847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:01.664852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:01.664856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:01.664859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:01.664864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:01.664868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.664871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.664875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.664882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:01.664891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:01.664896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:01.664900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:01.664906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:01.664911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:01.664915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:01.664920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:01.664925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:01.664930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:01.664935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:01.664939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:01.664944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.664949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.664953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.664957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.664963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.664968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.664993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:01.680014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:01.681475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:01.681488Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:01.681491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:01.681495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:01.681503Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 835 [2:7559341818524604622:7347]
2025-10-09T21:47:01.681522Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:01.681543Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:01.681546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:01.681551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:01.681553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:01.681572Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:01.681580Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:01.681595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:01.681675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.681683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.681686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.681689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:01.681787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:01.681796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.681812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.681825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.681842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:01.681868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341818524604617:7347]
2025-10-09T21:47:01.681871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341818524604617:7347]
2025-10-09T21:47:01.681873Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341818524604617:7347]
2025-10-09T21:47:01.681885Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:01.681904Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:01.681915Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:01.681930Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:01.681933Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:01.681933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:01.681938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:01.681952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341818886423224:8108], now have 1 active actors on pipe
2025-10-09T21:47:01.681957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341818886423068:2432], now have 1 active actors on pipe
2025-10-09T21:47:01.682004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341818524604617:7347]
2025-10-09T21:47:01.682019Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:01.682077Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:01.682056Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 835
2025-10-09T21:47:01.682112Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 835
2025-10-09T21:47:01.682405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:01.682418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:01.682453Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2d8ab2e6-ef7a6bda-bdeafc97-c3848536_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:01.682463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:01.682468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:01.682471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:01.682483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:01.682498Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:01.682518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:01.683025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:01.683037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:01.683058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:01.683183Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 836 partition: 0 MaxSeqNo: 25 sessionId: src_id|2d8ab2e6-ef7a6bda-bdeafc97-c3848536_0
2025-10-09T21:47:01.683586Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|edd54d5d-9dbdfcbf-94320b76-6096eab2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2d8ab2e6-ef7a6bda-bdeafc97-c3848536_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:01.686266Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d8ab2e6-ef7a6bda-bdeafc97-c3848536_0] Write session: try to update token
2025-10-09T21:47:01.686286Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d8ab2e6-ef7a6bda-bdeafc97-c3848536_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:01.697049Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:01.697070Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:01.697082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341818524604617:7347]
2025-10-09T21:47:01.697139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:01.697154Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:01.697321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:01.697461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:01.697646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:01.697778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:01.697977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:01.698016Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:01.708885Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 836 sessionId: src_id|2d8ab2e6-ef7a6bda-bdeafc97-c3848536_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:01.711614Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:01.712109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:01.712134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:01.712155Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:01.712159Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:01.712162Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:01.712164Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:01.712174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341818524604617:7347]
2025-10-09T21:47:01.712179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341818524604617:7347]
2025-10-09T21:47:01.712183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341818524604617:7347]
2025-10-09T21:47:01.712228Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:01.712234Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:01.712237Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:01.712242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:01.712250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.712253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:01.712386Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:01.740106Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341818886423240:7899]
2025-10-09T21:47:01.740458Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 835, pipe: [1:7559341818886423240:7899]
2025-10-09T21:47:01.878857Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:01.879492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:01.879550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341818886423240:7899], now have 1 active actors on pipe
2025-10-09T21:47:01.879686Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:01.879790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:01.879803Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:01.879890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 836 reason: error at write: Disconnected sessionId: src_id|2d8ab2e6-ef7a6bda-bdeafc97-c3848536_0
2025-10-09T21:47:01.880025Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 836 sessionId: src_id|2d8ab2e6-ef7a6bda-bdeafc97-c3848536_0 is DEAD
2025-10-09T21:47:01.880091Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341818886423240:7899]
2025-10-09T21:47:01.880123Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:01.880415Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d8ab2e6-ef7a6bda-bdeafc97-c3848536_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:01.880431Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d8ab2e6-ef7a6bda-bdeafc97-c3848536_0] Write session will restart in 0.007165s
2025-10-09T21:47:01.880474Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d8ab2e6-ef7a6bda-bdeafc97-c3848536_0] Write session: Do CDS request
2025-10-09T21:47:01.880486Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d8ab2e6-ef7a6bda-bdeafc97-c3848536_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:01.890708Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d8ab2e6-ef7a6bda-bdeafc97-c3848536_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:01.890888Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:01.890908Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 837
2025-10-09T21:47:01.891098Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 837 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:01.891141Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 837 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:01.891152Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=837 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:01.891157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:01.892403Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 837 sessionId:  describe result for acl check
2025-10-09T21:47:01.892444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:01.892450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:01.892452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:01.892457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341818886423312:8115] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:01.893027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341818886423312:8115] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:01.896702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341818886423312:8115] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:01.898514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:01.901184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:01.901288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341818886423312:8115] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:01.901306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341818886423312:8115] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:01.901310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341818886423312:8115] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:01.901321Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 837 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:01.901410Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:01.901421Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:01.901433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:01.901486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:01.901525Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341818524604782:7349]
2025-10-09T21:47:01.902144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:01.902773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:01.902815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:01.903134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:01.903182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:01.903183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:01.903184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:01.903185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:01.903187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:01.903188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:01.903189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.903190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.903191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.903197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:01.903207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:01.903210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:01.903214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:01.903217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:01.903221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:01.903224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.903227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.903230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.903238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:01.903246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:01.903252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:01.903256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:01.903261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:01.903265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:01.903269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:01.903274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:01.903278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:01.903282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:01.903286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:01.903303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:01.903309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.903314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.903317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.903322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.903325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.903329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.903353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:01.910455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:01.911539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:01.911554Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:01.911557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:01.911562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:01.911569Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 836 [2:7559341818524604782:7349]
2025-10-09T21:47:01.911584Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:01.911605Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:01.911608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:01.911611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:01.911613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:01.911633Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:01.911639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:01.911648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:01.911715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.911717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.911719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.911722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:01.911796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:01.911807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.911819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:01.911828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:01.911842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:01.911863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341818524604776:7349]
2025-10-09T21:47:01.911865Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341818524604776:7349]
2025-10-09T21:47:01.911867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341818524604776:7349]
2025-10-09T21:47:01.911877Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:01.911892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:01.911912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:01.911915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:01.911923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:01.912068Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 836
2025-10-09T21:47:01.911926Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:01.911929Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:01.911957Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341818524604776:7349]
2025-10-09T21:47:01.911960Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:01.911962Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:01.911971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341818886423245:2432], now have 1 active actors on pipe
2025-10-09T21:47:01.911974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341818886423402:8115], now have 1 active actors on pipe
2025-10-09T21:47:01.912139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 836
2025-10-09T21:47:01.912279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:01.912294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:01.912335Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|33d4c641-b35db07-9700861-a9a299ce_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:01.912352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:01.912361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:01.912364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:01.912376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:01.912394Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:01.912420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:01.912773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 837 partition: 0 MaxSeqNo: 25 sessionId: src_id|33d4c641-b35db07-9700861-a9a299ce_0
2025-10-09T21:47:01.912648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:01.912661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:01.912683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:01.913163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d8ab2e6-ef7a6bda-bdeafc97-c3848536_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|33d4c641-b35db07-9700861-a9a299ce_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:01.915924Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33d4c641-b35db07-9700861-a9a299ce_0] Write session: try to update token
2025-10-09T21:47:01.915947Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33d4c641-b35db07-9700861-a9a299ce_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:01.928773Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:01.928793Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:01.928795Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:01.928797Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:01.928807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341818524604776:7349]
2025-10-09T21:47:01.928812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341818524604776:7349]
2025-10-09T21:47:01.928817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341818524604776:7349]
2025-10-09T21:47:01.928862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:01.928870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:01.928872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:01.928923Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:01.928937Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:01.928942Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:01.929020Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:01.934383Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 837 sessionId: src_id|33d4c641-b35db07-9700861-a9a299ce_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:01.936956Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:01.981404Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341818886423414:7899]
2025-10-09T21:47:01.983742Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 836, pipe: [1:7559341818886423414:7899]
2025-10-09T21:47:02.094120Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:02.094611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:02.094628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:02.094677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341818886423414:7899], now have 1 active actors on pipe
2025-10-09T21:47:02.095881Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33d4c641-b35db07-9700861-a9a299ce_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:02.095893Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33d4c641-b35db07-9700861-a9a299ce_0] Write session will restart in 0.009044s
2025-10-09T21:47:02.095922Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33d4c641-b35db07-9700861-a9a299ce_0] Write session: Do CDS request
2025-10-09T21:47:02.095929Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33d4c641-b35db07-9700861-a9a299ce_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:02.096180Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:02.095394Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:02.095450Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:02.095453Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:02.095515Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 837 reason: error at write: Disconnected sessionId: src_id|33d4c641-b35db07-9700861-a9a299ce_0
2025-10-09T21:47:02.095704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 837 sessionId: src_id|33d4c641-b35db07-9700861-a9a299ce_0 is DEAD
2025-10-09T21:47:02.095861Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341818886423414:7899]
2025-10-09T21:47:02.106602Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33d4c641-b35db07-9700861-a9a299ce_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:02.106765Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:02.106786Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 838
2025-10-09T21:47:02.111136Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 838 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:02.111193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 838 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:02.111200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=838 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:02.111205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:02.111608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 838 sessionId:  describe result for acl check
2025-10-09T21:47:02.111643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:02.111645Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:02.111646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:02.111655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341823181390761:8121] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:02.112502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341823181390761:8121] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:02.115449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341823181390761:8121] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:02.123362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341823181390761:8121] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:02.123382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341823181390761:8121] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:02.123387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341823181390761:8121] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:02.123398Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 838 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:02.133651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:02.135292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:02.135531Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:02.135541Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:02.135553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:02.135619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:02.135656Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341822819572271:7361]
2025-10-09T21:47:02.136268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:02.137004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:02.137076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:02.137458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:02.137546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:02.137547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:02.137549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:02.137550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:02.137551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:02.137553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:02.137554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.137556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.137558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.137565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:02.137576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:02.137579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:02.137583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:02.137587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:02.137590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:02.137594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.137598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.137601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.137608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:02.137616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:02.137623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:02.137628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:02.137633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:02.137638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:02.137642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:02.137646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:02.137651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:02.137656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:02.137660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:02.137666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:02.137672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.137677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.137681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.137686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.137691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.137696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.137727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:02.154336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:02.155869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:02.155891Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:02.155894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:02.155899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:02.155907Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 837 [2:7559341822819572271:7361]
2025-10-09T21:47:02.155924Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:02.155944Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:02.155955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:02.155959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:02.155961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:02.155982Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:02.155991Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:02.155999Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:02.156082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.156090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.156093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.156096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:02.156189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:02.156192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.156206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.156217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.156234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:02.156257Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341822819572266:7361]
2025-10-09T21:47:02.156260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341822819572266:7361]
2025-10-09T21:47:02.156263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341822819572266:7361]
2025-10-09T21:47:02.156283Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:02.156301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:02.156315Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:02.156319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:02.156320Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:02.156322Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:02.156323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:02.156386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341822819572266:7361]
2025-10-09T21:47:02.156397Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:02.156603Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:02.156633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341823181390872:8121], now have 1 active actors on pipe
2025-10-09T21:47:02.156640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341823181390732:2432], now have 1 active actors on pipe
2025-10-09T21:47:02.159055Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 837
2025-10-09T21:47:02.159094Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 837
2025-10-09T21:47:02.160595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:02.160616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:02.160666Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fa48e9ab-7f8dccec-a1a9b6a-cc01d089_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:02.160681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:02.160687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:02.160690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:02.160706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:02.160727Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:02.160757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:02.164151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:02.164171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:02.164210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:02.164418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 838 partition: 0 MaxSeqNo: 25 sessionId: src_id|fa48e9ab-7f8dccec-a1a9b6a-cc01d089_0
2025-10-09T21:47:02.164845Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33d4c641-b35db07-9700861-a9a299ce_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fa48e9ab-7f8dccec-a1a9b6a-cc01d089_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:02.167642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fa48e9ab-7f8dccec-a1a9b6a-cc01d089_0] Write session: try to update token
2025-10-09T21:47:02.167668Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fa48e9ab-7f8dccec-a1a9b6a-cc01d089_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:02.183976Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:02.183997Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:02.184000Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:02.184001Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:02.184011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341822819572266:7361]
2025-10-09T21:47:02.184020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341822819572266:7361]
2025-10-09T21:47:02.184023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341822819572266:7361]
2025-10-09T21:47:02.184075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:02.184099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.184102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:02.184248Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:02.184943Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:02.184949Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:02.184953Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:02.195019Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 838 sessionId: src_id|fa48e9ab-7f8dccec-a1a9b6a-cc01d089_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:02.196339Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341823181390931:7899]
2025-10-09T21:47:02.197822Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 837, pipe: [1:7559341823181390931:7899]
2025-10-09T21:47:02.197909Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:02.417200Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:02.417558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341823181390931:7899], now have 1 active actors on pipe
2025-10-09T21:47:02.417594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:02.417601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:02.417898Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:02.418044Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341823181390931:7899]
2025-10-09T21:47:02.418084Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:02.418101Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:02.418105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:02.418168Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 838 reason: error at write: Disconnected sessionId: src_id|fa48e9ab-7f8dccec-a1a9b6a-cc01d089_0
2025-10-09T21:47:02.418388Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 838 sessionId: src_id|fa48e9ab-7f8dccec-a1a9b6a-cc01d089_0 is DEAD
2025-10-09T21:47:02.422079Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa48e9ab-7f8dccec-a1a9b6a-cc01d089_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:02.422096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa48e9ab-7f8dccec-a1a9b6a-cc01d089_0] Write session will restart in 0.007676s
2025-10-09T21:47:02.422127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa48e9ab-7f8dccec-a1a9b6a-cc01d089_0] Write session: Do CDS request
2025-10-09T21:47:02.422134Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa48e9ab-7f8dccec-a1a9b6a-cc01d089_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:02.431823Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fa48e9ab-7f8dccec-a1a9b6a-cc01d089_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:02.432266Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:02.432285Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 839
2025-10-09T21:47:02.432551Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 839 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:02.432585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 839 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:02.432591Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=839 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:02.432595Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:02.433092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 839 sessionId:  describe result for acl check
2025-10-09T21:47:02.433132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:02.433134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:02.433136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:02.433141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341823181390948:8131] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:02.433738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341823181390948:8131] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:02.438064Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341823181390948:8131] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:02.446933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341823181390948:8131] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:02.446953Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341823181390948:8131] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:02.446957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341823181390948:8131] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:02.446968Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 839 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:02.450793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:02.453618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:02.453884Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:02.453890Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:02.453904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:02.453992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:02.454052Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341822819572442:7365]
2025-10-09T21:47:02.454657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:02.455042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:02.455105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:02.455461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:02.455541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:02.455546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:02.455547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:02.455549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:02.455550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:02.455551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:02.455553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.455554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.455555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.455562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:02.455573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:02.455576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:02.455581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:02.455585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:02.455589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:02.455593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.455597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.455600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.455608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:02.455617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:02.455624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:02.455631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:02.455643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:02.455648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:02.455652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:02.455659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:02.455663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:02.455668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:02.455672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:02.455678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:02.455684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.455689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.455694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.455699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.455703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.455708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.455737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:02.463239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:02.464488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:02.464504Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:02.464508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:02.464514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:02.464522Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 838 [2:7559341822819572442:7365]
2025-10-09T21:47:02.464538Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:02.464560Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:02.464564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:02.464568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:02.464571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:02.464592Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:02.464604Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:02.464613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:02.464689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.464696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.464698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.464701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:02.464781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:02.464789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.464803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.464819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.464840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:02.464867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341822819572437:7365]
2025-10-09T21:47:02.464874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341822819572437:7365]
2025-10-09T21:47:02.464876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341822819572437:7365]
2025-10-09T21:47:02.464892Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:02.464914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:02.464930Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:02.464933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:02.464936Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:02.464939Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:02.464942Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:02.464993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341822819572437:7365]
2025-10-09T21:47:02.465002Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:02.465060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:02.465219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341823181391067:8131], now have 1 active actors on pipe
2025-10-09T21:47:02.465227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341823181390934:2432], now have 1 active actors on pipe
2025-10-09T21:47:02.465363Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 838
2025-10-09T21:47:02.465383Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 838
2025-10-09T21:47:02.465669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:02.465682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:02.465719Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1c0c6284-75f1205d-76f43b01-c7851906_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:02.465730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:02.465735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:02.465738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:02.465750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:02.465765Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:02.465785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:02.467014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:02.467031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:02.467068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:02.467253Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 839 partition: 0 MaxSeqNo: 25 sessionId: src_id|1c0c6284-75f1205d-76f43b01-c7851906_0
2025-10-09T21:47:02.467668Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa48e9ab-7f8dccec-a1a9b6a-cc01d089_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1c0c6284-75f1205d-76f43b01-c7851906_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:02.470465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c0c6284-75f1205d-76f43b01-c7851906_0] Write session: try to update token
2025-10-09T21:47:02.470490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c0c6284-75f1205d-76f43b01-c7851906_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:02.482929Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:02.482948Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:02.482951Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:02.482953Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:02.482963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341822819572437:7365]
2025-10-09T21:47:02.482968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341822819572437:7365]
2025-10-09T21:47:02.482974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341822819572437:7365]
2025-10-09T21:47:02.483022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:02.483030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.483033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:02.483038Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:02.483045Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:02.483049Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:02.483170Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:02.488131Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 839 sessionId: src_id|1c0c6284-75f1205d-76f43b01-c7851906_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:02.490787Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:02.513686Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341823181391105:7899]
2025-10-09T21:47:02.514137Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 838, pipe: [1:7559341823181391105:7899]
2025-10-09T21:47:02.646401Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:02.646803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:02.646815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:02.646862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341823181391105:7899], now have 1 active actors on pipe
2025-10-09T21:47:02.647229Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:02.647118Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:02.647133Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341823181391105:7899]
2025-10-09T21:47:02.647596Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c0c6284-75f1205d-76f43b01-c7851906_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:02.647609Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c0c6284-75f1205d-76f43b01-c7851906_0] Write session will restart in 0.009680s
2025-10-09T21:47:02.647134Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:02.647152Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:02.647215Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 839 reason: error at write: Disconnected sessionId: src_id|1c0c6284-75f1205d-76f43b01-c7851906_0
2025-10-09T21:47:02.647403Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 839 sessionId: src_id|1c0c6284-75f1205d-76f43b01-c7851906_0 is DEAD
2025-10-09T21:47:02.647637Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c0c6284-75f1205d-76f43b01-c7851906_0] Write session: Do CDS request
2025-10-09T21:47:02.647645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c0c6284-75f1205d-76f43b01-c7851906_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:02.658591Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c0c6284-75f1205d-76f43b01-c7851906_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:02.658838Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:02.658855Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 840
2025-10-09T21:47:02.658994Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 840 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:02.659032Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 840 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:02.659038Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=840 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:02.659041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:02.659572Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 840 sessionId:  describe result for acl check
2025-10-09T21:47:02.659625Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:02.659634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:02.659636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:02.659643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341823181391129:8135] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:02.660283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341823181391129:8135] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:02.666164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341823181391129:8135] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:02.669393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:02.669759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341823181391129:8135] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:02.669775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341823181391129:8135] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:02.669779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341823181391129:8135] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:02.669788Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 840 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:02.671563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:02.671807Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:02.671820Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:02.671835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:02.671900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:02.671948Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341822819572609:7369]
2025-10-09T21:47:02.672568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:02.672943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:02.673013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:02.673387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:02.673461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:02.673469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:02.673470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:02.673471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:02.673472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:02.673473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:02.673474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.673475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.673476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.673481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:02.673491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:02.673503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:02.673507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:02.673510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:02.673514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:02.673517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.673521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.673525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.673532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:02.673540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:02.673546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:02.673550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:02.673555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:02.673559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:02.673563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:02.673568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:02.673573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:02.673578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:02.673582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:02.673586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:02.673591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.673595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.673599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.673605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.673609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.673614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.673643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:02.680774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:02.682226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:02.682243Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:02.682246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:02.682251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:02.682261Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 839 [2:7559341822819572609:7369]
2025-10-09T21:47:02.682276Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:02.682299Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:02.682307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:02.682312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:02.682314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:02.682334Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:02.682342Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:02.682349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:02.682423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.682432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.682434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.682438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:02.682529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:02.682540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.682553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.682563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.682578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:02.682600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341822819572603:7369]
2025-10-09T21:47:02.682603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341822819572603:7369]
2025-10-09T21:47:02.682605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341822819572603:7369]
2025-10-09T21:47:02.682621Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:02.682638Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:02.682653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:02.682662Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:02.682707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341822819572603:7369]
2025-10-09T21:47:02.682710Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:02.683251Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:02.683260Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:02.683263Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:02.683269Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:02.683437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341823181391108:2432], now have 1 active actors on pipe
2025-10-09T21:47:02.683459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341823181391269:8135], now have 1 active actors on pipe
2025-10-09T21:47:02.683771Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 839
2025-10-09T21:47:02.683820Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 839
2025-10-09T21:47:02.684617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:02.684632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:02.684665Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7cf3dab4-26582e0f-e03e9e56-1063fa3b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:02.684677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:02.684682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:02.684685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:02.684698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:02.684727Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:02.684751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:02.685151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:02.685162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:02.685183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:02.685350Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 840 partition: 0 MaxSeqNo: 25 sessionId: src_id|7cf3dab4-26582e0f-e03e9e56-1063fa3b_0
2025-10-09T21:47:02.685744Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c0c6284-75f1205d-76f43b01-c7851906_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7cf3dab4-26582e0f-e03e9e56-1063fa3b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:02.688248Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7cf3dab4-26582e0f-e03e9e56-1063fa3b_0] Write session: try to update token
2025-10-09T21:47:02.688267Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7cf3dab4-26582e0f-e03e9e56-1063fa3b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:02.700500Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:02.700517Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:02.700520Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:02.700522Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:02.700533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341822819572603:7369]
2025-10-09T21:47:02.700538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341822819572603:7369]
2025-10-09T21:47:02.700543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341822819572603:7369]
2025-10-09T21:47:02.700596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:02.700609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.700616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:02.700766Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:02.703212Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:02.703231Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:02.703236Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:02.707628Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 840 sessionId: src_id|7cf3dab4-26582e0f-e03e9e56-1063fa3b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:02.710233Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:02.747160Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341823181391281:7899]
2025-10-09T21:47:02.750270Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 839, pipe: [1:7559341823181391281:7899]
2025-10-09T21:47:02.865350Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:02.865668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:02.865678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:02.865725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341823181391281:7899], now have 1 active actors on pipe
2025-10-09T21:47:02.865992Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:02.866541Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:02.866604Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:02.866692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 840 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7cf3dab4-26582e0f-e03e9e56-1063fa3b_0
2025-10-09T21:47:02.866908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 840 sessionId: src_id|7cf3dab4-26582e0f-e03e9e56-1063fa3b_0 is DEAD
2025-10-09T21:47:02.867137Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341823181391281:7899]
2025-10-09T21:47:02.867366Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cf3dab4-26582e0f-e03e9e56-1063fa3b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:02.867381Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cf3dab4-26582e0f-e03e9e56-1063fa3b_0] Write session will restart in 0.007111s
2025-10-09T21:47:02.867407Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cf3dab4-26582e0f-e03e9e56-1063fa3b_0] Write session: Do CDS request
2025-10-09T21:47:02.867414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cf3dab4-26582e0f-e03e9e56-1063fa3b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:02.876261Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7cf3dab4-26582e0f-e03e9e56-1063fa3b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:02.876697Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:02.876713Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 841
2025-10-09T21:47:02.876895Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 841 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:02.876946Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 841 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:02.876953Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=841 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:02.876957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:02.877445Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 841 sessionId:  describe result for acl check
2025-10-09T21:47:02.877490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:02.877492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:02.877494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:02.877500Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341823181391301:8140] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:02.878165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341823181391301:8140] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:02.883218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341823181391301:8140] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:02.888427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341823181391301:8140] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:02.888449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341823181391301:8140] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:02.888456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341823181391301:8140] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:02.888469Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 841 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:02.891494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:02.894105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:02.894403Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:02.894407Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:02.894434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:02.894504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:02.894560Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341822819572773:7371]
2025-10-09T21:47:02.895157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:02.895541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:02.895595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:02.895947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:02.896000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:02.896001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:02.896002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:02.896003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:02.896005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:02.896006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:02.896007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.896008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.896009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.896015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:02.896026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:02.896030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:02.896034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:02.896037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:02.896041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:02.896045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.896049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.896052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.896060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:02.896070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:02.896078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:02.896084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:02.896088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:02.896094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:02.896098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:02.896104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:02.896108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:02.896113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:02.896120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:02.896126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:02.896135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.896140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.896146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.896151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.896157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.896162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.896192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:02.904441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:02.905980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:02.905999Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:02.906002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:02.906006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:02.906014Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 840 [2:7559341822819572773:7371]
2025-10-09T21:47:02.906047Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:02.906069Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:02.906073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:02.906077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:02.906080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:02.906098Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:02.906112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:02.906119Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:02.906195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.906202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.906204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.906207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:02.906284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:02.906292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.906308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:02.906321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:02.906338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:02.906361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341823181391286:2432], now have 1 active actors on pipe
2025-10-09T21:47:02.906365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341823181391445:8140], now have 1 active actors on pipe
2025-10-09T21:47:02.906374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341822819572767:7371]
2025-10-09T21:47:02.906382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341822819572767:7371]
2025-10-09T21:47:02.906384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341822819572767:7371]
2025-10-09T21:47:02.906393Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:02.906410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:02.906420Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:02.906424Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:02.906427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:02.906429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:02.906433Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:02.906476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341822819572767:7371]
2025-10-09T21:47:02.906485Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:02.906490Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:02.906515Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 840
2025-10-09T21:47:02.906517Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 840
2025-10-09T21:47:02.906790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:02.906807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:02.906841Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|35037688-5c6fa6e5-ba46ffb6-221b7f88_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:02.906864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:02.906874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:02.906877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:02.906890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:02.906909Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:02.906932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:02.907400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:02.907410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:02.907444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:02.907690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 841 partition: 0 MaxSeqNo: 25 sessionId: src_id|35037688-5c6fa6e5-ba46ffb6-221b7f88_0
2025-10-09T21:47:02.908106Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cf3dab4-26582e0f-e03e9e56-1063fa3b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|35037688-5c6fa6e5-ba46ffb6-221b7f88_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:02.910785Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|35037688-5c6fa6e5-ba46ffb6-221b7f88_0] Write session: try to update token
2025-10-09T21:47:02.910810Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|35037688-5c6fa6e5-ba46ffb6-221b7f88_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:02.924770Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:02.924789Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:02.924792Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:02.924794Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:02.924804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341822819572767:7371]
2025-10-09T21:47:02.924812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341822819572767:7371]
2025-10-09T21:47:02.924816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341822819572767:7371]
2025-10-09T21:47:02.924865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:02.924873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:02.924876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:02.925016Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:02.927178Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:02.927193Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:02.927198Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:02.930342Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 841 sessionId: src_id|35037688-5c6fa6e5-ba46ffb6-221b7f88_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:02.933060Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:02.966800Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341823181391459:7899]
2025-10-09T21:47:02.967132Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 840, pipe: [1:7559341823181391459:7899]
2025-10-09T21:47:03.094988Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:03.095846Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341823181391459:7899]
2025-10-09T21:47:03.095877Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:03.095889Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:03.095893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:03.095948Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 841 reason: error at write: Disconnected sessionId: src_id|35037688-5c6fa6e5-ba46ffb6-221b7f88_0
2025-10-09T21:47:03.096146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 841 sessionId: src_id|35037688-5c6fa6e5-ba46ffb6-221b7f88_0 is DEAD
2025-10-09T21:47:03.095371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:03.095388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:03.095434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341823181391459:7899], now have 1 active actors on pipe
2025-10-09T21:47:03.096748Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:03.098179Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35037688-5c6fa6e5-ba46ffb6-221b7f88_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:03.098198Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35037688-5c6fa6e5-ba46ffb6-221b7f88_0] Write session will restart in 0.005614s
2025-10-09T21:47:03.098232Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35037688-5c6fa6e5-ba46ffb6-221b7f88_0] Write session: Do CDS request
2025-10-09T21:47:03.098239Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35037688-5c6fa6e5-ba46ffb6-221b7f88_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:03.104607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|35037688-5c6fa6e5-ba46ffb6-221b7f88_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:03.104727Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:03.104761Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 842
2025-10-09T21:47:03.104941Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 842 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:03.104985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 842 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:03.104992Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=842 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:03.105000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:03.105504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 842 sessionId:  describe result for acl check
2025-10-09T21:47:03.105554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:03.105559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:03.105561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:03.105569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341827476358784:8144] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:03.106256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341827476358784:8144] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:03.111994Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341827476358784:8144] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:03.117011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341827476358784:8144] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:03.117033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341827476358784:8144] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:03.117037Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341827476358784:8144] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:03.117048Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 842 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:03.121215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:03.123717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:03.123985Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:03.123999Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:03.124013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:03.124088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:03.124136Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341827114540265:7380]
2025-10-09T21:47:03.124761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:03.125424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:03.125498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:03.126115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:03.126199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:03.126200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:03.126202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:03.126203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:03.126205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:03.126207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:03.126208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.126209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.126211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.126218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:03.126231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:03.126235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:03.126240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:03.126244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:03.126248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:03.126253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.126257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.126262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.126270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:03.126280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:03.126287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:03.126293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:03.126307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:03.126313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:03.126318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:03.126324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:03.126329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:03.126334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:03.126341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:03.126347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:03.126353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.126359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.126363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.126368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.126375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.126381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.126412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:03.135136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:03.136201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:03.136211Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:03.136215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:03.136219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:03.136229Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 841 [2:7559341827114540265:7380]
2025-10-09T21:47:03.136247Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:03.136270Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:03.136273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:03.136277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:03.136280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:03.136298Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:03.136305Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:03.136313Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:03.136379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.136382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.136384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.136387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:03.136469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:03.136472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.136486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.136498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.136527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:03.136549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341827114540260:7380]
2025-10-09T21:47:03.136553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341827114540260:7380]
2025-10-09T21:47:03.136556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341827114540260:7380]
2025-10-09T21:47:03.136565Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:03.136580Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:03.136591Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:03.136594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:03.136597Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:03.136597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:03.136599Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:03.136649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341827476358920:8144], now have 1 active actors on pipe
2025-10-09T21:47:03.136653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341827476358761:2432], now have 1 active actors on pipe
2025-10-09T21:47:03.136655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341827114540260:7380]
2025-10-09T21:47:03.136663Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:03.136691Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:03.137368Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 841
2025-10-09T21:47:03.137406Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 841
2025-10-09T21:47:03.137734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:03.137752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:03.137792Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|642ad0a9-599e8846-70e33872-1eae2b98_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:03.137811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:03.137816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:03.137819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:03.137832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:03.137847Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:03.137869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:03.138192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:03.138208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:03.138242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:03.138527Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 842 partition: 0 MaxSeqNo: 25 sessionId: src_id|642ad0a9-599e8846-70e33872-1eae2b98_0
2025-10-09T21:47:03.139028Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35037688-5c6fa6e5-ba46ffb6-221b7f88_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|642ad0a9-599e8846-70e33872-1eae2b98_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:03.142539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|642ad0a9-599e8846-70e33872-1eae2b98_0] Write session: try to update token
2025-10-09T21:47:03.142564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|642ad0a9-599e8846-70e33872-1eae2b98_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:03.155959Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:03.155979Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:03.155982Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:03.155984Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:03.155993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341827114540260:7380]
2025-10-09T21:47:03.155999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341827114540260:7380]
2025-10-09T21:47:03.156003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341827114540260:7380]
2025-10-09T21:47:03.156050Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:03.156053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:03.156057Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:03.156061Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:03.156063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.156066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:03.156214Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:03.166750Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 842 sessionId: src_id|642ad0a9-599e8846-70e33872-1eae2b98_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:03.169353Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:03.196168Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341827476358954:7899]
2025-10-09T21:47:03.196521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 841, pipe: [1:7559341827476358954:7899]
2025-10-09T21:47:03.324847Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:03.325271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:03.325292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:03.325316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341827476358954:7899], now have 1 active actors on pipe
2025-10-09T21:47:03.325552Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:03.326066Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:03.326181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 842 reason: status is not ok: tablet will be restarted right now sessionId: src_id|642ad0a9-599e8846-70e33872-1eae2b98_0
2025-10-09T21:47:03.326186Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341827476358954:7899]
2025-10-09T21:47:03.326209Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:03.326401Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 842 sessionId: src_id|642ad0a9-599e8846-70e33872-1eae2b98_0 is DEAD
2025-10-09T21:47:03.326722Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|642ad0a9-599e8846-70e33872-1eae2b98_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:03.326748Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|642ad0a9-599e8846-70e33872-1eae2b98_0] Write session will restart in 0.005895s
2025-10-09T21:47:03.326776Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|642ad0a9-599e8846-70e33872-1eae2b98_0] Write session: Do CDS request
2025-10-09T21:47:03.326783Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|642ad0a9-599e8846-70e33872-1eae2b98_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:03.333626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|642ad0a9-599e8846-70e33872-1eae2b98_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:03.333711Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:03.333729Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 843
2025-10-09T21:47:03.333878Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 843 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:03.333911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 843 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:03.333930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=843 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:03.333939Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:03.334505Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 843 sessionId:  describe result for acl check
2025-10-09T21:47:03.334544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:03.334547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:03.334550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:03.334555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341827476358991:8156] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:03.335125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341827476358991:8156] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:03.338086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341827476358991:8156] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:03.341033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341827476358991:8156] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:03.341048Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341827476358991:8156] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:03.341052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341827476358991:8156] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:03.341070Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 843 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:03.341517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:03.343421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:03.343641Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:03.343654Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:03.343666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:03.343727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:03.343766Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341827114540439:7387]
2025-10-09T21:47:03.344267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:03.344623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:03.344690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:03.345034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:03.345106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:03.345113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:03.345114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:03.345115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:03.345116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:03.345117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:03.345118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.345119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.345120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.345125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:03.345136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:03.345140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:03.345144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:03.345147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:03.345150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:03.345153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.345157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.345160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.345168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:03.345175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:03.345181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:03.345186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:03.345190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:03.345195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:03.345198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:03.345203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:03.345208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:03.345213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:03.345217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:03.345221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:03.345226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.345230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.345235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.345240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.345244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.345261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.345294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:03.352907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:03.354298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:03.354314Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:03.354317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:03.354322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:03.354331Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 842 [2:7559341827114540439:7387]
2025-10-09T21:47:03.354346Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:03.354368Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:03.354371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:03.354375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:03.354378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:03.354399Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:03.354404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:03.354412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:03.354481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.354493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.354495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.354500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:03.354576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:03.354584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.354599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.354615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.354638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:03.354661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341827476359127:8156], now have 1 active actors on pipe
2025-10-09T21:47:03.354678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341827114540434:7387]
2025-10-09T21:47:03.354681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341827114540434:7387]
2025-10-09T21:47:03.354683Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341827114540434:7387]
2025-10-09T21:47:03.354698Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:03.354710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341827476358967:2432], now have 1 active actors on pipe
2025-10-09T21:47:03.354716Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:03.354720Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:03.354720Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:03.354722Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:03.354735Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:03.354739Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:03.354784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341827114540434:7387]
2025-10-09T21:47:03.354796Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:03.354815Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:03.355004Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 842
2025-10-09T21:47:03.355129Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 842
2025-10-09T21:47:03.355253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:03.355267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:03.355318Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2e66378d-371b2096-d315d153-a43f9fd8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:03.355334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:03.355340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:03.355343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:03.355356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:03.355376Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:03.355414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:03.355836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:03.355850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:03.355872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:03.356069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 843 partition: 0 MaxSeqNo: 25 sessionId: src_id|2e66378d-371b2096-d315d153-a43f9fd8_0
2025-10-09T21:47:03.356525Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|642ad0a9-599e8846-70e33872-1eae2b98_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2e66378d-371b2096-d315d153-a43f9fd8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:03.359317Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e66378d-371b2096-d315d153-a43f9fd8_0] Write session: try to update token
2025-10-09T21:47:03.359335Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e66378d-371b2096-d315d153-a43f9fd8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:03.372119Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:03.372136Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:03.372139Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:03.372154Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:03.372166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341827114540434:7387]
2025-10-09T21:47:03.372171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341827114540434:7387]
2025-10-09T21:47:03.372174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341827114540434:7387]
2025-10-09T21:47:03.372223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:03.372230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.372232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:03.372233Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:03.372242Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:03.372246Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:03.372388Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:03.375057Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 843 sessionId: src_id|2e66378d-371b2096-d315d153-a43f9fd8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:03.377627Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:03.426024Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341827476359141:7899]
2025-10-09T21:47:03.426543Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 842, pipe: [1:7559341827476359141:7899]
2025-10-09T21:47:03.535128Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:03.535528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:03.535546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:03.535597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341827476359141:7899], now have 1 active actors on pipe
2025-10-09T21:47:03.536064Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341827476359141:7899]
2025-10-09T21:47:03.536103Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:03.536105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:03.536110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:03.536171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 843 reason: error at write: Disconnected sessionId: src_id|2e66378d-371b2096-d315d153-a43f9fd8_0
2025-10-09T21:47:03.536388Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 843 sessionId: src_id|2e66378d-371b2096-d315d153-a43f9fd8_0 is DEAD
2025-10-09T21:47:03.536718Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e66378d-371b2096-d315d153-a43f9fd8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:03.536731Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e66378d-371b2096-d315d153-a43f9fd8_0] Write session will restart in 0.009816s
2025-10-09T21:47:03.536756Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e66378d-371b2096-d315d153-a43f9fd8_0] Write session: Do CDS request
2025-10-09T21:47:03.536765Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e66378d-371b2096-d315d153-a43f9fd8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:03.536104Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:03.547762Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e66378d-371b2096-d315d153-a43f9fd8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:03.550178Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:03.550197Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 844
2025-10-09T21:47:03.550380Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 844 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:03.550418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 844 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:03.550424Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=844 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:03.550428Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:03.551484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 844 sessionId:  describe result for acl check
2025-10-09T21:47:03.551524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:03.551526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:03.551528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:03.551534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341827476359159:8160] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:03.552139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341827476359159:8160] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:03.557793Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341827476359159:8160] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:03.574174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341827476359159:8160] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:03.574196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341827476359159:8160] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:03.574200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341827476359159:8160] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:03.574213Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 844 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:03.575367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:03.580576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:03.580873Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:03.580885Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:03.580899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:03.580976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:03.581027Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341827114540611:7389]
2025-10-09T21:47:03.581619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:03.582370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:03.582768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:03.583126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:03.583184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:03.583186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:03.583187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:03.583189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:03.583189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:03.583191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:03.583192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.583193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.583194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.583200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:03.583211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:03.583214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:03.583219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:03.583222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:03.583225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:03.583228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.583231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.583234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.583242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:03.583250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:03.583257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:03.583262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:03.583267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:03.583273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:03.583278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:03.583282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:03.583286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:03.583290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:03.583294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:03.583299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:03.583304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.583309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.583313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.583317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.583324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.583329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.583357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:03.593568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:03.595813Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 843
2025-10-09T21:47:03.595980Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 843
2025-10-09T21:47:03.597714Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 844 partition: 0 MaxSeqNo: 25 sessionId: src_id|f99fb3-9683daed-9ba0bc3a-6cf5959_0
2025-10-09T21:47:03.598196Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e66378d-371b2096-d315d153-a43f9fd8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f99fb3-9683daed-9ba0bc3a-6cf5959_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:03.595078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:03.595097Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:03.595099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:03.595104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:03.595112Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 843 [2:7559341827114540611:7389]
2025-10-09T21:47:03.595129Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:03.595151Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:03.595155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:03.595158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:03.595160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:03.595182Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:03.595189Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:03.595197Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:03.595275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.595277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.595280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.595283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:03.595366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:03.595369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.595385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.595395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.595409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:03.595433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341827114540600:7389]
2025-10-09T21:47:03.595436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341827114540600:7389]
2025-10-09T21:47:03.595438Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341827114540600:7389]
2025-10-09T21:47:03.595454Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:03.595471Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:03.595485Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:03.595488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:03.595535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341827114540600:7389]
2025-10-09T21:47:03.595537Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:03.595584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:03.595593Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:03.595596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:03.595604Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:03.595667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341827476359143:2432], now have 1 active actors on pipe
2025-10-09T21:47:03.595672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341827476359306:8160], now have 1 active actors on pipe
2025-10-09T21:47:03.596075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:03.596083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:03.596128Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f99fb3-9683daed-9ba0bc3a-6cf5959_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:03.596138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:03.596143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:03.596146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:03.596171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:03.596185Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:03.596205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:03.597530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:03.597538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:03.597568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:03.601095Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f99fb3-9683daed-9ba0bc3a-6cf5959_0] Write session: try to update token
2025-10-09T21:47:03.601118Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f99fb3-9683daed-9ba0bc3a-6cf5959_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:03.613718Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:03.613738Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:03.613750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341827114540600:7389]
2025-10-09T21:47:03.613806Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:03.614017Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:03.614242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:03.614438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:03.614618Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:03.614851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:03.614901Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:03.617011Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:03.623977Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 844 sessionId: src_id|f99fb3-9683daed-9ba0bc3a-6cf5959_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:03.626747Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:03.627257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:03.627286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:03.627468Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:03.627337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:03.635469Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341827476359319:7899]
2025-10-09T21:47:03.635951Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 843, pipe: [1:7559341827476359319:7899]
2025-10-09T21:47:03.635939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341827476359319:7899], now have 1 active actors on pipe
2025-10-09T21:47:03.640216Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:03.640233Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:03.640236Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:03.640239Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:03.640250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341827114540600:7389]
2025-10-09T21:47:03.640258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341827114540600:7389]
2025-10-09T21:47:03.640264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341827114540600:7389]
2025-10-09T21:47:03.640310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:03.640316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.640319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:03.640467Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:03.640923Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:03.640930Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:03.640935Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:03.808309Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:03.816866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:03.816899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:03.817079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:03.817138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:03.817198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:03.817263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:03.817306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:03.817311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:03.817381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:03.817446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:03.817513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:03.817582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:03.817622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:03.817624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:03.817693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:03.817748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:03.817812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:03.817871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:03.817931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:03.817934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:03.818017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:03.821070Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:03.821076Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341827476359319:7899]
2025-10-09T21:47:03.821106Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:03.821109Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:03.821140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 844 reason: error at write: Disconnected sessionId: src_id|f99fb3-9683daed-9ba0bc3a-6cf5959_0
2025-10-09T21:47:03.818465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:03.818538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:03.818624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:03.818663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:03.818666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:03.818734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:03.818794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:03.818855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:03.818921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:03.818963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:03.818966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:03.819034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:03.819093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:03.819153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:03.819212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:03.819251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:03.819257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:03.821434Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 844 sessionId: src_id|f99fb3-9683daed-9ba0bc3a-6cf5959_0 is DEAD
2025-10-09T21:47:03.819358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:03.819450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:03.819544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:03.819610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:03.819660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:03.819663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:03.819732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:03.819790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:03.819850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:03.819908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:03.819950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:03.819953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:03.820023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:03.820080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:03.820140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:03.820203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:03.820249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:03.820252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:03.820317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:03.820376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:03.820437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:03.820497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:03.820538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:03.820541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:03.821651Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f99fb3-9683daed-9ba0bc3a-6cf5959_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:03.821661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f99fb3-9683daed-9ba0bc3a-6cf5959_0] Write session will restart in 0.009724s
2025-10-09T21:47:03.821695Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f99fb3-9683daed-9ba0bc3a-6cf5959_0] Write session: Do CDS request
2025-10-09T21:47:03.821702Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f99fb3-9683daed-9ba0bc3a-6cf5959_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:03.821703Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:03.832650Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f99fb3-9683daed-9ba0bc3a-6cf5959_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:03.832722Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:03.832742Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 845
2025-10-09T21:47:03.833033Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 845 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:03.833074Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 845 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:03.833080Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=845 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:03.833084Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:03.833508Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 845 sessionId:  describe result for acl check
2025-10-09T21:47:03.833549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:03.833556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:03.833558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:03.833564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341827476359358:8165] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:03.834248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341827476359358:8165] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:03.841857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341827476359358:8165] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:03.847224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341827476359358:8165] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:03.847244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341827476359358:8165] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:03.847248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341827476359358:8165] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:03.847259Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 845 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:03.851745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:03.857194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:03.857438Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:03.857442Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:03.857455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:03.857507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:03.857562Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341827114540779:7393]
2025-10-09T21:47:03.858170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:03.858649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:03.858719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:03.859066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:03.859146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:03.859153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:03.859154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:03.859156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:03.859157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:03.859159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:03.859160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.859161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.859163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.859170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:03.859181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:03.859186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:03.859191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:03.859195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:03.859199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:03.859203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.859207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.859212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.859221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:03.859230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:03.859236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:03.859242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:03.859247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:03.859253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:03.859258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:03.859264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:03.859275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:03.859281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:03.859300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:03.859311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:03.859317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.859327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.859333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.859338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.859343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.859348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.859377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:03.877842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:03.879047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:03.879055Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:03.879060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:03.879065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:03.879074Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 844 [2:7559341827114540779:7393]
2025-10-09T21:47:03.879088Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:03.879112Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:03.879115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:03.879118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:03.879121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:03.879144Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:03.879150Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:03.879160Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:03.879235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.879237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.879238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.879241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:03.879329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:03.879331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.879346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:03.879355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:03.879370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:03.879393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341827476359488:8165], now have 1 active actors on pipe
2025-10-09T21:47:03.879396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341827476359324:2432], now have 1 active actors on pipe
2025-10-09T21:47:03.879404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341827114540773:7393]
2025-10-09T21:47:03.879407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341827114540773:7393]
2025-10-09T21:47:03.879408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341827114540773:7393]
2025-10-09T21:47:03.879418Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:03.879435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:03.879442Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:03.879446Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:03.879448Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:03.879450Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:03.879453Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:03.879498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341827114540773:7393]
2025-10-09T21:47:03.879508Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:03.879572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:03.880075Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 844
2025-10-09T21:47:03.880122Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 844
2025-10-09T21:47:03.881182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:03.881200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:03.881239Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d952ea18-a98e7240-db1578f4-50f17cc1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:03.881252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:03.881266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:03.881269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:03.881282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:03.881297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:03.881319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:03.882343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:03.882356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:03.882381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:03.883122Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f99fb3-9683daed-9ba0bc3a-6cf5959_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d952ea18-a98e7240-db1578f4-50f17cc1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:03.885723Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d952ea18-a98e7240-db1578f4-50f17cc1_0] Write session: try to update token
2025-10-09T21:47:03.885746Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d952ea18-a98e7240-db1578f4-50f17cc1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:03.882664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 845 partition: 0 MaxSeqNo: 25 sessionId: src_id|d952ea18-a98e7240-db1578f4-50f17cc1_0
2025-10-09T21:47:03.906747Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:03.906764Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:03.906766Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:03.906768Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:03.906779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341827114540773:7393]
2025-10-09T21:47:03.906783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341827114540773:7393]
2025-10-09T21:47:03.906787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341827114540773:7393]
2025-10-09T21:47:03.906831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:03.906837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:03.906840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:03.906841Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:03.906847Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:03.906851Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:03.906989Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:03.921229Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341827476359502:7899]
2025-10-09T21:47:03.921890Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 844, pipe: [1:7559341827476359502:7899]
2025-10-09T21:47:03.923327Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 845 sessionId: src_id|d952ea18-a98e7240-db1578f4-50f17cc1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:03.925934Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:04.074923Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:04.075502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341827476359502:7899], now have 1 active actors on pipe
2025-10-09T21:47:04.075556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:04.075563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:04.075901Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:04.076847Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d952ea18-a98e7240-db1578f4-50f17cc1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:04.076859Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d952ea18-a98e7240-db1578f4-50f17cc1_0] Write session will restart in 0.008969s
2025-10-09T21:47:04.076884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d952ea18-a98e7240-db1578f4-50f17cc1_0] Write session: Do CDS request
2025-10-09T21:47:04.076891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d952ea18-a98e7240-db1578f4-50f17cc1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:04.076311Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341827476359502:7899]
2025-10-09T21:47:04.076347Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:04.076431Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 845 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d952ea18-a98e7240-db1578f4-50f17cc1_0
2025-10-09T21:47:04.076623Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 845 sessionId: src_id|d952ea18-a98e7240-db1578f4-50f17cc1_0 is DEAD
2025-10-09T21:47:04.076679Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:04.088272Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d952ea18-a98e7240-db1578f4-50f17cc1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:04.090224Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:04.090254Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 846
2025-10-09T21:47:04.094175Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 846 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:04.094234Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 846 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:04.094240Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=846 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:04.094245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:04.094791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 846 sessionId:  describe result for acl check
2025-10-09T21:47:04.094830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:04.094832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:04.094834Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:04.094841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341831771326912:8169] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:04.095441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341831771326912:8169] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:04.098194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341831771326912:8169] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:04.099772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:04.103124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341831771326912:8169] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:04.103142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341831771326912:8169] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:04.103146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341831771326912:8169] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:04.103155Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 846 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:04.100745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:04.100972Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:04.100976Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:04.100991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:04.101051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:04.101087Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341831409508243:7395]
2025-10-09T21:47:04.101708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:04.102458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:04.102505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:04.102838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:04.102883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:04.102885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:04.102886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:04.102887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:04.102888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:04.102889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:04.102890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.102891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.102892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.102898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:04.102909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:04.102912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:04.102915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:04.102918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:04.102921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:04.102924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.102927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.102930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.102936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:04.102944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:04.102949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:04.102954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:04.102957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:04.102961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:04.102965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:04.102969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:04.102973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:04.102978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:04.102981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:04.102986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:04.102992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.102996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.103000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.103004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.103011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.103015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.103039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:04.116027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:04.117433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:04.117452Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:04.117455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:04.117584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:04.117599Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 845 [2:7559341831409508243:7395]
2025-10-09T21:47:04.117614Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:04.117634Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:04.117638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:04.117641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:04.117644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:04.117665Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:04.117672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:04.117679Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:04.117749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.117758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.117761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.117765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:04.117839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:04.117848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.117862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.117875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.117893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:04.117926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341831409508238:7395]
2025-10-09T21:47:04.117928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341831409508238:7395]
2025-10-09T21:47:04.117930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341831409508238:7395]
2025-10-09T21:47:04.117939Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:04.117955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:04.117967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:04.117971Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:04.117971Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:04.117973Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:04.117975Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:04.118019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341831409508238:7395]
2025-10-09T21:47:04.118051Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:04.118065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341831771326973:8169], now have 1 active actors on pipe
2025-10-09T21:47:04.118078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341831771326801:2432], now have 1 active actors on pipe
2025-10-09T21:47:04.118103Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:04.118515Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 845
2025-10-09T21:47:04.118553Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 845
2025-10-09T21:47:04.119219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:04.119232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:04.119277Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c29938a6-56246eb2-c2f16158-8553ce43_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:04.119287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:04.119291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:04.119294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:04.119306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:04.119321Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:04.119347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:04.119830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:04.119839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:04.119868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:04.120093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 846 partition: 0 MaxSeqNo: 25 sessionId: src_id|c29938a6-56246eb2-c2f16158-8553ce43_0
2025-10-09T21:47:04.122324Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d952ea18-a98e7240-db1578f4-50f17cc1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c29938a6-56246eb2-c2f16158-8553ce43_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:04.124922Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c29938a6-56246eb2-c2f16158-8553ce43_0] Write session: try to update token
2025-10-09T21:47:04.124945Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c29938a6-56246eb2-c2f16158-8553ce43_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:04.137089Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:04.137110Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:04.137114Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:04.137116Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:04.137129Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341831409508238:7395]
2025-10-09T21:47:04.137134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341831409508238:7395]
2025-10-09T21:47:04.137138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341831409508238:7395]
2025-10-09T21:47:04.137187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:04.137195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.137198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:04.137318Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:04.137330Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:04.137335Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:04.137342Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:04.146780Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 846 sessionId: src_id|c29938a6-56246eb2-c2f16158-8553ce43_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:04.149254Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:04.176152Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341831771326987:7899]
2025-10-09T21:47:04.176484Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 845, pipe: [1:7559341831771326987:7899]
2025-10-09T21:47:04.303501Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:04.303905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:04.303915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:04.303939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341831771326987:7899], now have 1 active actors on pipe
2025-10-09T21:47:04.304193Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:04.304469Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:04.304561Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 846 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c29938a6-56246eb2-c2f16158-8553ce43_0
2025-10-09T21:47:04.304766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 846 sessionId: src_id|c29938a6-56246eb2-c2f16158-8553ce43_0 is DEAD
2025-10-09T21:47:04.304924Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341831771326987:7899]
2025-10-09T21:47:04.304945Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:04.305125Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c29938a6-56246eb2-c2f16158-8553ce43_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:04.305138Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c29938a6-56246eb2-c2f16158-8553ce43_0] Write session will restart in 0.008059s
2025-10-09T21:47:04.305180Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c29938a6-56246eb2-c2f16158-8553ce43_0] Write session: Do CDS request
2025-10-09T21:47:04.305187Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c29938a6-56246eb2-c2f16158-8553ce43_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:04.314585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c29938a6-56246eb2-c2f16158-8553ce43_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:04.317504Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:04.317523Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 847
2025-10-09T21:47:04.317723Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 847 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:04.317777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 847 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:04.317790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=847 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:04.317795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:04.322810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 847 sessionId:  describe result for acl check
2025-10-09T21:47:04.322872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:04.322877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:04.322879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:04.322885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341831771327084:8181] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:04.323502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341831771327084:8181] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:04.327533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341831771327084:8181] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:04.331269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341831771327084:8181] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:04.331285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341831771327084:8181] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:04.331289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341831771327084:8181] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:04.331299Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 847 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:04.339205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:04.344756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:04.345180Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:04.345193Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:04.345209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:04.345305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:04.345351Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341831409508453:7407]
2025-10-09T21:47:04.345998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:04.350194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:04.350293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:04.350717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:04.350812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:04.350814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:04.350815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:04.350817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:04.350818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:04.350820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:04.350821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.350823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.350824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.350833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:04.350845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:04.350849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:04.350853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:04.350857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:04.350861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:04.350866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.350870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.350875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.350884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:04.350893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:04.350900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:04.350906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:04.350914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:04.350919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:04.350924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:04.350929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:04.350933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:04.350938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:04.350944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:04.350950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:04.350955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.350959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.350963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.350968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.350973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.350978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.351006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:04.358546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:04.359775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:04.359792Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:04.359795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:04.359802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:04.359811Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 846 [2:7559341831409508453:7407]
2025-10-09T21:47:04.359827Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:04.359849Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:04.359853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:04.359857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:04.359860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:04.359880Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:04.359887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:04.359896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:04.359968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.359978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.359980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.359982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:04.360250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:04.360262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.360278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.360292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.360313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:04.360342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341831771327141:8181], now have 1 active actors on pipe
2025-10-09T21:47:04.360345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341831771327003:2432], now have 1 active actors on pipe
2025-10-09T21:47:04.360323Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 846
2025-10-09T21:47:04.360355Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341831409508447:7407]
2025-10-09T21:47:04.360359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341831409508447:7407]
2025-10-09T21:47:04.360331Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 846
2025-10-09T21:47:04.360361Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341831409508447:7407]
2025-10-09T21:47:04.360371Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:04.360385Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:04.360392Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:04.360393Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:04.360395Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:04.360407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:04.360410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:04.360431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:04.360436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:04.360493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341831409508447:7407]
2025-10-09T21:47:04.360508Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:04.360517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:04.360527Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1c730042-5a1dec66-defa5c32-f4645d0d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:04.360535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:04.360539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:04.360543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:04.360555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:04.360572Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:04.360619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:04.360823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:04.360833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:04.360851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:04.361053Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 847 partition: 0 MaxSeqNo: 25 sessionId: src_id|1c730042-5a1dec66-defa5c32-f4645d0d_0
2025-10-09T21:47:04.361476Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c29938a6-56246eb2-c2f16158-8553ce43_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1c730042-5a1dec66-defa5c32-f4645d0d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:04.364319Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c730042-5a1dec66-defa5c32-f4645d0d_0] Write session: try to update token
2025-10-09T21:47:04.364343Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c730042-5a1dec66-defa5c32-f4645d0d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:04.373825Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:04.373847Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:04.373859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341831409508447:7407]
2025-10-09T21:47:04.373924Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:04.373928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:04.374169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:04.374418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:04.374643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:04.374877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:04.375146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:04.375210Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:04.383762Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 847 sessionId: src_id|1c730042-5a1dec66-defa5c32-f4645d0d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:04.386460Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:04.388048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:04.388073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:04.388104Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:04.388107Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:04.388113Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:04.388115Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:04.388125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341831409508447:7407]
2025-10-09T21:47:04.388129Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341831409508447:7407]
2025-10-09T21:47:04.388135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341831409508447:7407]
2025-10-09T21:47:04.388191Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:04.388204Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:04.388209Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:04.388214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:04.388223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.388225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:04.388370Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:04.404971Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341831771327192:7899]
2025-10-09T21:47:04.405320Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 846, pipe: [1:7559341831771327192:7899]
2025-10-09T21:47:04.548201Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:04.548506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:04.548547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341831771327192:7899], now have 1 active actors on pipe
2025-10-09T21:47:04.549256Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:04.549349Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:04.549356Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:04.549433Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 847 reason: error at write: Disconnected sessionId: src_id|1c730042-5a1dec66-defa5c32-f4645d0d_0
2025-10-09T21:47:04.549583Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 847 sessionId: src_id|1c730042-5a1dec66-defa5c32-f4645d0d_0 is DEAD
2025-10-09T21:47:04.549771Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c730042-5a1dec66-defa5c32-f4645d0d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:04.549757Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341831771327192:7899]
2025-10-09T21:47:04.549784Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c730042-5a1dec66-defa5c32-f4645d0d_0] Write session will restart in 0.005350s
2025-10-09T21:47:04.549779Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:04.549810Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c730042-5a1dec66-defa5c32-f4645d0d_0] Write session: Do CDS request
2025-10-09T21:47:04.549820Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c730042-5a1dec66-defa5c32-f4645d0d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:04.556617Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c730042-5a1dec66-defa5c32-f4645d0d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:04.557585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:04.557603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 848
2025-10-09T21:47:04.558850Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 848 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:04.558894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 848 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:04.558903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=848 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:04.558908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:04.560268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 848 sessionId:  describe result for acl check
2025-10-09T21:47:04.560315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:04.560317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:04.560319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:04.560325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341831771327206:8185] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:04.560930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341831771327206:8185] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:04.564285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341831771327206:8185] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:04.572042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341831771327206:8185] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:04.572059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341831771327206:8185] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:04.572063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341831771327206:8185] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:04.572073Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 848 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:04.581094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:04.583418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:04.583742Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:04.583754Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:04.583768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:04.583831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:04.583867Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341831409508629:7411]
2025-10-09T21:47:04.584446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:04.584808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:04.584888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:04.585254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:04.585312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:04.585320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:04.585321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:04.585322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:04.585323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:04.585324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:04.585325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.585326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.585328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.585334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:04.585345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:04.585349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:04.585352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:04.585356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:04.585359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:04.585363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.585367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.585371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.585379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:04.585386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:04.585392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:04.585397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:04.585402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:04.585407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:04.585411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:04.585416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:04.585422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:04.585427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:04.585430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:04.585435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:04.585439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.585443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.585447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.585451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.585456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.585461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.585488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:04.592519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:04.593736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:04.593756Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:04.593758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:04.593763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:04.593775Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 847 [2:7559341831409508629:7411]
2025-10-09T21:47:04.593790Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:04.593810Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:04.593814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:04.593817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:04.593820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:04.593839Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:04.593850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:04.593857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:04.593932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.593939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.593941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.593943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:04.594018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:04.594026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.594060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.594073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.594090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:04.594119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341831771327290:8185], now have 1 active actors on pipe
2025-10-09T21:47:04.594123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341831771327195:2432], now have 1 active actors on pipe
2025-10-09T21:47:04.594131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341831409508623:7411]
2025-10-09T21:47:04.594133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341831409508623:7411]
2025-10-09T21:47:04.594136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341831409508623:7411]
2025-10-09T21:47:04.594151Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:04.594175Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:04.594182Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:04.594189Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:04.594190Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:04.594192Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:04.594192Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:04.594231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341831409508623:7411]
2025-10-09T21:47:04.594243Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:04.594269Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:04.598154Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 847
2025-10-09T21:47:04.598196Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 847
2025-10-09T21:47:04.598316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:04.598348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:04.598386Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5dcb493b-aa39ee52-3895ec5a-4b86e6ed_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:04.598403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:04.598412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:04.598416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:04.598430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:04.598467Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:04.598493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:04.598800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:04.598814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:04.598832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:04.599508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c730042-5a1dec66-defa5c32-f4645d0d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5dcb493b-aa39ee52-3895ec5a-4b86e6ed_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:04.598985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 848 partition: 0 MaxSeqNo: 25 sessionId: src_id|5dcb493b-aa39ee52-3895ec5a-4b86e6ed_0
2025-10-09T21:47:04.602170Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5dcb493b-aa39ee52-3895ec5a-4b86e6ed_0] Write session: try to update token
2025-10-09T21:47:04.602189Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5dcb493b-aa39ee52-3895ec5a-4b86e6ed_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:04.614089Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:04.614107Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:04.614121Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341831409508623:7411]
2025-10-09T21:47:04.614179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:04.614199Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:04.614400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:04.614540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:04.614681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:04.614826Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:04.615003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:04.615040Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:04.621003Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 848 sessionId: src_id|5dcb493b-aa39ee52-3895ec5a-4b86e6ed_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:04.622818Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:04.622837Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:04.622840Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:04.622847Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:04.622858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341831409508623:7411]
2025-10-09T21:47:04.622862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341831409508623:7411]
2025-10-09T21:47:04.622865Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341831409508623:7411]
2025-10-09T21:47:04.622917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:04.622928Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:04.622934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.622934Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:04.622936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:04.622943Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:04.623103Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:04.623665Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:04.649203Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341831771327375:7899]
2025-10-09T21:47:04.649466Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 847, pipe: [1:7559341831771327375:7899]
2025-10-09T21:47:04.783472Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:04.783873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:04.783886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:04.783911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341831771327375:7899], now have 1 active actors on pipe
2025-10-09T21:47:04.784165Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:04.784528Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:04.784541Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:04.784605Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 848 reason: error at write: Disconnected sessionId: src_id|5dcb493b-aa39ee52-3895ec5a-4b86e6ed_0
2025-10-09T21:47:04.784791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 848 sessionId: src_id|5dcb493b-aa39ee52-3895ec5a-4b86e6ed_0 is DEAD
2025-10-09T21:47:04.784990Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5dcb493b-aa39ee52-3895ec5a-4b86e6ed_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:04.785002Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5dcb493b-aa39ee52-3895ec5a-4b86e6ed_0] Write session will restart in 0.007960s
2025-10-09T21:47:04.785034Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5dcb493b-aa39ee52-3895ec5a-4b86e6ed_0] Write session: Do CDS request
2025-10-09T21:47:04.785041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5dcb493b-aa39ee52-3895ec5a-4b86e6ed_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:04.785014Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341831771327375:7899]
2025-10-09T21:47:04.785042Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:04.794655Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5dcb493b-aa39ee52-3895ec5a-4b86e6ed_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:04.794742Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:04.794762Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 849
2025-10-09T21:47:04.794926Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 849 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:04.794966Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 849 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:04.794976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=849 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:04.794980Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:04.795512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 849 sessionId:  describe result for acl check
2025-10-09T21:47:04.795569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:04.795579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:04.795581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:04.795588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341831771327408:8190] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:04.796368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341831771327408:8190] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:04.800061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341831771327408:8190] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:04.804091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341831771327408:8190] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:04.804107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341831771327408:8190] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:04.804111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341831771327408:8190] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:04.804130Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 849 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:04.806589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:04.809214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:04.809615Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:04.809628Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:04.809645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:04.809719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:04.809771Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341831409508802:7415]
2025-10-09T21:47:04.810450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:04.811106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:04.811184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:04.811526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:04.811618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:04.811620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:04.811621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:04.811622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:04.811623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:04.811624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:04.811626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.811627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.811628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.811635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:04.811647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:04.811652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:04.811656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:04.811660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:04.811665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:04.811669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.811674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.811677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.811686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:04.811695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:04.811701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:04.811708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:04.811715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:04.811720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:04.811724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:04.811730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:04.811734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:04.811741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:04.811745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:04.811751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:04.811756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.811761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.811779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.811784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.811789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.811794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.811825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:04.819363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:04.820682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:04.820698Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:04.820700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:04.820710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:04.820718Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 848 [2:7559341831409508802:7415]
2025-10-09T21:47:04.820734Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:04.820756Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:04.820765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:04.820768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:04.820770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:04.820788Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:04.820800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:04.820808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:04.820875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.820884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.820887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.820889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:04.820966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:04.820973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.820987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:04.821000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:04.821015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:04.821052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341831409508797:7415]
2025-10-09T21:47:04.821059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341831409508797:7415]
2025-10-09T21:47:04.821061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341831409508797:7415]
2025-10-09T21:47:04.821071Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:04.821089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:04.821096Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:04.821100Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:04.821102Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:04.821109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:04.821112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:04.821125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341831771327385:2432], now have 1 active actors on pipe
2025-10-09T21:47:04.821129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341831771327547:8190], now have 1 active actors on pipe
2025-10-09T21:47:04.821169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341831409508797:7415]
2025-10-09T21:47:04.821182Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:04.821202Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 848
2025-10-09T21:47:04.821224Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 848
2025-10-09T21:47:04.821353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:04.821471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:04.821482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:04.821527Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e064e27d-e30b6aee-98193f91-49bb2fcc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:04.821540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:04.821544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:04.821547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:04.821569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:04.821581Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:04.821597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:04.821815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:04.821824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:04.821841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:04.822021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 849 partition: 0 MaxSeqNo: 25 sessionId: src_id|e064e27d-e30b6aee-98193f91-49bb2fcc_0
2025-10-09T21:47:04.822481Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5dcb493b-aa39ee52-3895ec5a-4b86e6ed_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e064e27d-e30b6aee-98193f91-49bb2fcc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:04.826272Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e064e27d-e30b6aee-98193f91-49bb2fcc_0] Write session: try to update token
2025-10-09T21:47:04.826296Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e064e27d-e30b6aee-98193f91-49bb2fcc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:04.837756Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:04.837772Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:04.837775Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:04.837777Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:04.837787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341831409508797:7415]
2025-10-09T21:47:04.837792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341831409508797:7415]
2025-10-09T21:47:04.837797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341831409508797:7415]
2025-10-09T21:47:04.837843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:04.837853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:04.837856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:04.837858Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:04.837865Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:04.837870Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:04.838007Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:04.845006Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 849 sessionId: src_id|e064e27d-e30b6aee-98193f91-49bb2fcc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:04.851127Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:04.885253Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341831771327566:7899]
2025-10-09T21:47:04.885713Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 848, pipe: [1:7559341831771327566:7899]
2025-10-09T21:47:05.005946Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:05.006535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:05.006549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:05.006573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341831771327566:7899], now have 1 active actors on pipe
2025-10-09T21:47:05.006859Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:05.006960Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:05.007035Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341831771327566:7899]
2025-10-09T21:47:05.007059Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:05.007062Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:05.007144Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 849 reason: error at write: Disconnected sessionId: src_id|e064e27d-e30b6aee-98193f91-49bb2fcc_0
2025-10-09T21:47:05.007354Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 849 sessionId: src_id|e064e27d-e30b6aee-98193f91-49bb2fcc_0 is DEAD
2025-10-09T21:47:05.007883Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e064e27d-e30b6aee-98193f91-49bb2fcc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:05.007896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e064e27d-e30b6aee-98193f91-49bb2fcc_0] Write session will restart in 0.005308s
2025-10-09T21:47:05.007927Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e064e27d-e30b6aee-98193f91-49bb2fcc_0] Write session: Do CDS request
2025-10-09T21:47:05.007936Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e064e27d-e30b6aee-98193f91-49bb2fcc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:05.014620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e064e27d-e30b6aee-98193f91-49bb2fcc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:05.018204Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:05.018227Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 850
2025-10-09T21:47:05.018512Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 850 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:05.018551Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 850 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:05.018558Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=850 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:05.018562Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:05.019014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 850 sessionId:  describe result for acl check
2025-10-09T21:47:05.019059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:05.019066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:05.019068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:05.019074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341836066294936:8194] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:05.019680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341836066294936:8194] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:05.023488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341836066294936:8194] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:05.025019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:05.025949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:05.026185Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:05.026196Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:05.026208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:05.026260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:05.026307Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341835704476270:7417]
2025-10-09T21:47:05.026874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:05.027527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:05.027570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:05.027907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:05.027958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:05.027964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:05.027965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:05.027967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:05.027968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:05.027969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:05.027970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.027971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.027972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.027978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:05.027987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:05.027991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:05.027994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:05.027998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:05.028001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:05.028004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.028008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.028011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.028018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:05.028026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:05.028032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:05.028037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:05.028041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:05.028047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:05.028050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:05.028055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:05.028059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:05.028063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:05.028067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:05.028071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:05.028076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.028064Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341836066294936:8194] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:05.028080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.028084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.028080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341836066294936:8194] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:05.028088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.028084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341836066294936:8194] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:05.028093Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 850 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:05.028092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.028097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.028123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:05.035898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:05.038247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:05.038266Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:05.038268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:05.038273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:05.038281Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 849 [2:7559341835704476270:7417]
2025-10-09T21:47:05.038298Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:05.038323Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:05.038327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:05.038331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:05.038334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:05.038354Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:05.038372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:05.038380Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:05.038452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.038460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.038462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.038465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:05.038553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:05.038556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.038571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.038584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.038602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:05.038625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341836066294866:2432], now have 1 active actors on pipe
2025-10-09T21:47:05.038629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341836066295038:8194], now have 1 active actors on pipe
2025-10-09T21:47:05.038637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341835704476264:7417]
2025-10-09T21:47:05.038641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341835704476264:7417]
2025-10-09T21:47:05.038643Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341835704476264:7417]
2025-10-09T21:47:05.038655Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:05.038677Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:05.038704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:05.038726Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:05.038729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:05.038746Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 849
2025-10-09T21:47:05.038760Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 849
2025-10-09T21:47:05.038892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:05.038903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:05.038970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:05.038980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:05.038994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341835704476264:7417]
2025-10-09T21:47:05.039004Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:05.039012Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:05.039043Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ad98ad68-ee3a568e-4a4aaa92-7e1ca288_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:05.039051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:05.039062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:05.039065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:05.039074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:05.039085Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:05.039109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:05.039280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:05.039289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:05.039307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:05.039541Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 850 partition: 0 MaxSeqNo: 25 sessionId: src_id|ad98ad68-ee3a568e-4a4aaa92-7e1ca288_0
2025-10-09T21:47:05.040023Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e064e27d-e30b6aee-98193f91-49bb2fcc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ad98ad68-ee3a568e-4a4aaa92-7e1ca288_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:05.048755Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad98ad68-ee3a568e-4a4aaa92-7e1ca288_0] Write session: try to update token
2025-10-09T21:47:05.048782Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad98ad68-ee3a568e-4a4aaa92-7e1ca288_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:05.055795Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:05.055817Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:05.055819Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:05.055821Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:05.055832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341835704476264:7417]
2025-10-09T21:47:05.055837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341835704476264:7417]
2025-10-09T21:47:05.055841Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341835704476264:7417]
2025-10-09T21:47:05.055894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:05.055902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.055911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:05.056093Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:05.058576Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:05.058590Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:05.058595Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:05.079763Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 850 sessionId: src_id|ad98ad68-ee3a568e-4a4aaa92-7e1ca288_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:05.082571Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:05.107131Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341836066295051:7899]
2025-10-09T21:47:05.107589Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 849, pipe: [1:7559341836066295051:7899]
2025-10-09T21:47:05.228352Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad98ad68-ee3a568e-4a4aaa92-7e1ca288_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:05.228370Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad98ad68-ee3a568e-4a4aaa92-7e1ca288_0] Write session will restart in 0.007564s
2025-10-09T21:47:05.227757Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341836066295051:7899]
2025-10-09T21:47:05.228399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad98ad68-ee3a568e-4a4aaa92-7e1ca288_0] Write session: Do CDS request
2025-10-09T21:47:05.228406Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad98ad68-ee3a568e-4a4aaa92-7e1ca288_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:05.227809Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:05.227851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:05.227854Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:05.227919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 850 reason: error at write: Disconnected sessionId: src_id|ad98ad68-ee3a568e-4a4aaa92-7e1ca288_0
2025-10-09T21:47:05.228117Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 850 sessionId: src_id|ad98ad68-ee3a568e-4a4aaa92-7e1ca288_0 is DEAD
2025-10-09T21:47:05.226642Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:05.227076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:05.227086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:05.227111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341836066295051:7899], now have 1 active actors on pipe
2025-10-09T21:47:05.228247Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:05.237603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad98ad68-ee3a568e-4a4aaa92-7e1ca288_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:05.238086Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:05.238105Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 851
2025-10-09T21:47:05.238395Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 851 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:05.238432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 851 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:05.238438Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=851 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:05.238442Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:05.239187Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 851 sessionId:  describe result for acl check
2025-10-09T21:47:05.239229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:05.239232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:05.239234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:05.239240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341836066295069:8199] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:05.239838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341836066295069:8199] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:05.243292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341836066295069:8199] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:05.260916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341836066295069:8199] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:05.260941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341836066295069:8199] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:05.260946Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341836066295069:8199] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:05.260957Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 851 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:05.270659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:05.277439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:05.277760Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:05.277765Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:05.277778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:05.277844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:05.277887Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341835704476476:7425]
2025-10-09T21:47:05.278529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:05.280599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:05.280681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:05.281123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:05.281176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:05.281178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:05.281179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:05.281181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:05.281183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:05.281184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:05.281186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.281187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.281189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.281195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:05.281208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:05.281213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:05.281217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:05.281222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:05.281227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:05.281231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.281236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.281240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.281249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:05.281260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:05.281267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:05.281274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:05.281279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:05.281285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:05.281292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:05.281297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:05.281303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:05.281309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:05.281314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:05.281320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:05.281326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.281331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.281337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.281343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.281349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.281355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.281411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:05.289827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:05.291263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:05.291285Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:05.291288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:05.291293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:05.291302Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 850 [2:7559341835704476476:7425]
2025-10-09T21:47:05.291319Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:05.291341Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:05.291344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:05.291347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:05.291350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:05.291370Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:05.291377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:05.291386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:05.291462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.291471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.291473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.291477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:05.291564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:05.291572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.291588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.291601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.291621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:05.291650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341836066295059:2432], now have 1 active actors on pipe
2025-10-09T21:47:05.291655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341836066295189:8199], now have 1 active actors on pipe
2025-10-09T21:47:05.291663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341835704476470:7425]
2025-10-09T21:47:05.291666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341835704476470:7425]
2025-10-09T21:47:05.291669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341835704476470:7425]
2025-10-09T21:47:05.291685Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:05.291708Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:05.291728Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:05.291730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:05.291732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:05.291733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:05.291737Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:05.291782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341835704476470:7425]
2025-10-09T21:47:05.291795Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:05.291795Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 850
2025-10-09T21:47:05.291835Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 850
2025-10-09T21:47:05.291896Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:05.292094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:05.292104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:05.292141Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ddc36b44-61d7f6e2-8ce9acb6-8fc134d2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:05.292152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:05.292157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:05.292172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:05.292185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:05.292202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:05.292236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:05.292690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:05.292703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:05.292725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:05.293075Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 851 partition: 0 MaxSeqNo: 25 sessionId: src_id|ddc36b44-61d7f6e2-8ce9acb6-8fc134d2_0
2025-10-09T21:47:05.293564Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad98ad68-ee3a568e-4a4aaa92-7e1ca288_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ddc36b44-61d7f6e2-8ce9acb6-8fc134d2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:05.296619Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ddc36b44-61d7f6e2-8ce9acb6-8fc134d2_0] Write session: try to update token
2025-10-09T21:47:05.296646Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ddc36b44-61d7f6e2-8ce9acb6-8fc134d2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:05.309394Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:05.309411Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:05.309414Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:05.309416Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:05.309426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341835704476470:7425]
2025-10-09T21:47:05.309432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341835704476470:7425]
2025-10-09T21:47:05.309437Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341835704476470:7425]
2025-10-09T21:47:05.309492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:05.309493Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:05.309499Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:05.309500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.309504Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:05.309503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:05.309734Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:05.327658Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341836066295248:7899]
2025-10-09T21:47:05.328047Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 850, pipe: [1:7559341836066295248:7899]
2025-10-09T21:47:05.331534Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 851 sessionId: src_id|ddc36b44-61d7f6e2-8ce9acb6-8fc134d2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:05.334003Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:05.509153Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:05.509621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341836066295248:7899], now have 1 active actors on pipe
2025-10-09T21:47:05.509659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:05.509667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:05.509942Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:05.509942Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341836066295248:7899]
2025-10-09T21:47:05.509971Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:05.509973Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:05.510100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 851 reason: error at write: Disconnected sessionId: src_id|ddc36b44-61d7f6e2-8ce9acb6-8fc134d2_0
2025-10-09T21:47:05.510524Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 851 sessionId: src_id|ddc36b44-61d7f6e2-8ce9acb6-8fc134d2_0 is DEAD
2025-10-09T21:47:05.510884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddc36b44-61d7f6e2-8ce9acb6-8fc134d2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:05.510899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddc36b44-61d7f6e2-8ce9acb6-8fc134d2_0] Write session will restart in 0.006516s
2025-10-09T21:47:05.510926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddc36b44-61d7f6e2-8ce9acb6-8fc134d2_0] Write session: Do CDS request
2025-10-09T21:47:05.510935Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddc36b44-61d7f6e2-8ce9acb6-8fc134d2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:05.510641Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:05.518674Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ddc36b44-61d7f6e2-8ce9acb6-8fc134d2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:05.518805Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:05.518826Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 852
2025-10-09T21:47:05.518966Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 852 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:05.519003Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 852 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:05.519014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=852 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:05.519018Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:05.519419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 852 sessionId:  describe result for acl check
2025-10-09T21:47:05.519447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:05.519449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:05.519451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:05.519458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341836066295284:8210] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:05.520004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341836066295284:8210] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:05.524921Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341836066295284:8210] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:05.529059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341836066295284:8210] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:05.529078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341836066295284:8210] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:05.529082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341836066295284:8210] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:05.529092Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 852 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:05.533260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:05.535869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:05.536157Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:05.536171Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:05.536185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:05.536255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:05.536301Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341835704476659:7433]
2025-10-09T21:47:05.536950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:05.537463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:05.537538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:05.537926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:05.538025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:05.538047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:05.538048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:05.538050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:05.538051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:05.538052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:05.538054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.538055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.538057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.538064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:05.538075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:05.538079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:05.538083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:05.538087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:05.538091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:05.538095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.538099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.538103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.538112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:05.538123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:05.538130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:05.538136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:05.538141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:05.538147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:05.538152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:05.538158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:05.538163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:05.538168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:05.538173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:05.538180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:05.538185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.538191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.538196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.538201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.538207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.538212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.538245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:05.554907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:05.556319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:05.556335Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:05.556338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:05.556342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:05.556349Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 851 [2:7559341835704476659:7433]
2025-10-09T21:47:05.556365Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:05.556386Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:05.556390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:05.556394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:05.556396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:05.556416Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:05.556423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:05.556435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:05.556506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.556513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.556515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.556517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:05.556591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:05.556598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.556613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.556625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.556641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:05.556666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341835704476654:7433]
2025-10-09T21:47:05.556669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341835704476654:7433]
2025-10-09T21:47:05.556671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341835704476654:7433]
2025-10-09T21:47:05.556682Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:05.556705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:05.556730Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:05.556738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:05.556751Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:05.556760Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:05.556762Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:05.556782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341835704476654:7433]
2025-10-09T21:47:05.556791Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:05.556792Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:05.556803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341836066295406:8210], now have 1 active actors on pipe
2025-10-09T21:47:05.556811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341836066295269:2432], now have 1 active actors on pipe
2025-10-09T21:47:05.556863Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 851
2025-10-09T21:47:05.556899Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 851
2025-10-09T21:47:05.557209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:05.557221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:05.557258Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|60a965a3-4d5748ac-93b65d16-e21c89f1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:05.557271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:05.557276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:05.557279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:05.557289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:05.557309Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:05.557329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:05.557666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:05.557673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:05.557692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:05.557870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 852 partition: 0 MaxSeqNo: 25 sessionId: src_id|60a965a3-4d5748ac-93b65d16-e21c89f1_0
2025-10-09T21:47:05.559569Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddc36b44-61d7f6e2-8ce9acb6-8fc134d2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|60a965a3-4d5748ac-93b65d16-e21c89f1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:05.562158Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|60a965a3-4d5748ac-93b65d16-e21c89f1_0] Write session: try to update token
2025-10-09T21:47:05.562177Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|60a965a3-4d5748ac-93b65d16-e21c89f1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:05.574530Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:05.574552Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:05.574555Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:05.574556Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:05.574566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341835704476654:7433]
2025-10-09T21:47:05.574571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341835704476654:7433]
2025-10-09T21:47:05.574574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341835704476654:7433]
2025-10-09T21:47:05.574625Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:05.574632Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:05.574634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:05.574637Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:05.574641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.574644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:05.574781Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:05.584429Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 852 sessionId: src_id|60a965a3-4d5748ac-93b65d16-e21c89f1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:05.587061Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:05.610262Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341836066295454:7899]
2025-10-09T21:47:05.610563Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 851, pipe: [1:7559341836066295454:7899]
2025-10-09T21:47:05.754590Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:05.755440Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341836066295454:7899]
2025-10-09T21:47:05.755497Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:05.756120Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:05.756133Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:05.756201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 852 reason: error at write: Disconnected sessionId: src_id|60a965a3-4d5748ac-93b65d16-e21c89f1_0
2025-10-09T21:47:05.756377Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 852 sessionId: src_id|60a965a3-4d5748ac-93b65d16-e21c89f1_0 is DEAD
2025-10-09T21:47:05.756543Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60a965a3-4d5748ac-93b65d16-e21c89f1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:05.755024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:05.755041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:05.755063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341836066295454:7899], now have 1 active actors on pipe
2025-10-09T21:47:05.756205Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:05.756557Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60a965a3-4d5748ac-93b65d16-e21c89f1_0] Write session will restart in 0.005540s
2025-10-09T21:47:05.756617Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60a965a3-4d5748ac-93b65d16-e21c89f1_0] Write session: Do CDS request
2025-10-09T21:47:05.756625Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60a965a3-4d5748ac-93b65d16-e21c89f1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:05.763625Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|60a965a3-4d5748ac-93b65d16-e21c89f1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:05.763743Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:05.763761Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 853
2025-10-09T21:47:05.764066Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 853 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:05.764120Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 853 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:05.764134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=853 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:05.764138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:05.764712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 853 sessionId:  describe result for acl check
2025-10-09T21:47:05.764750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:05.764752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:05.764753Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:05.764759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341836066295483:8215] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:05.765667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341836066295483:8215] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:05.771159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341836066295483:8215] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:05.775296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341836066295483:8215] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:05.775313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341836066295483:8215] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:05.775318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341836066295483:8215] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:05.775328Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 853 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:05.775939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:05.778553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:05.778901Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:05.778914Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:05.778930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:05.779000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:05.779044Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341835704476840:7437]
2025-10-09T21:47:05.779739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:05.780097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:05.780162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:05.780486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:05.780556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:05.780562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:05.780564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:05.780565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:05.780566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:05.780567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:05.780569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.780570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.780572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.780577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:05.780589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:05.780594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:05.780598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:05.780602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:05.780606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:05.780609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.780613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.780617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.780626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:05.780633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:05.780639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:05.780645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:05.780649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:05.780654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:05.780658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:05.780664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:05.780670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:05.780675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:05.780680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:05.780697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:05.780707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.780712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.780716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.780720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.780725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.780729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.780760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:05.788192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:05.789621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:05.789634Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:05.789637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:05.789642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:05.789650Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 852 [2:7559341835704476840:7437]
2025-10-09T21:47:05.789665Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:05.789685Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:05.789691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:05.789696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:05.789698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:05.789717Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:05.789723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:05.789730Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:05.789788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.789798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.789801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.789804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:05.789881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:05.789890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.789905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:05.789930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:05.789949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:05.789971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341835704476835:7437]
2025-10-09T21:47:05.789974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341835704476835:7437]
2025-10-09T21:47:05.789976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341835704476835:7437]
2025-10-09T21:47:05.789994Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:05.790018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:05.790063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:05.790073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:05.790074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:05.790075Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:05.790077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:05.790124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341835704476835:7437]
2025-10-09T21:47:05.790138Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:05.790159Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:05.790184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341836066295631:8215], now have 1 active actors on pipe
2025-10-09T21:47:05.790188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341836066295457:2432], now have 1 active actors on pipe
2025-10-09T21:47:05.790331Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 852
2025-10-09T21:47:05.790371Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 852
2025-10-09T21:47:05.790459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:05.790473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:05.790509Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|20e34b0c-24892dc8-a8d44dda-5432c39f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:05.790518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:05.790529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:05.790532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:05.790542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:05.790555Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:05.790573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:05.790852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:05.790866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:05.790889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:05.791061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 853 partition: 0 MaxSeqNo: 25 sessionId: src_id|20e34b0c-24892dc8-a8d44dda-5432c39f_0
2025-10-09T21:47:05.791521Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60a965a3-4d5748ac-93b65d16-e21c89f1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|20e34b0c-24892dc8-a8d44dda-5432c39f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:05.794049Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|20e34b0c-24892dc8-a8d44dda-5432c39f_0] Write session: try to update token
2025-10-09T21:47:05.794068Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|20e34b0c-24892dc8-a8d44dda-5432c39f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:05.805630Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:05.805650Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:05.805654Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:05.805656Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:05.805670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341835704476835:7437]
2025-10-09T21:47:05.805677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341835704476835:7437]
2025-10-09T21:47:05.805681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341835704476835:7437]
2025-10-09T21:47:05.805734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:05.805741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:05.805744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:05.805756Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:05.805770Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:05.805775Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:05.805901Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:05.809884Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 853 sessionId: src_id|20e34b0c-24892dc8-a8d44dda-5432c39f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:05.812858Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:05.855498Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341836066295647:7899]
2025-10-09T21:47:05.855835Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 852, pipe: [1:7559341836066295647:7899]
2025-10-09T21:47:05.965563Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:05.965920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:05.965940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:05.965991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341836066295647:7899], now have 1 active actors on pipe
2025-10-09T21:47:05.966402Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:05.967065Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20e34b0c-24892dc8-a8d44dda-5432c39f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:05.967077Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20e34b0c-24892dc8-a8d44dda-5432c39f_0] Write session will restart in 0.009197s
2025-10-09T21:47:05.967119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20e34b0c-24892dc8-a8d44dda-5432c39f_0] Write session: Do CDS request
2025-10-09T21:47:05.967127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20e34b0c-24892dc8-a8d44dda-5432c39f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:05.966605Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:05.966622Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:05.966703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 853 reason: error at write: Disconnected sessionId: src_id|20e34b0c-24892dc8-a8d44dda-5432c39f_0
2025-10-09T21:47:05.966910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 853 sessionId: src_id|20e34b0c-24892dc8-a8d44dda-5432c39f_0 is DEAD
2025-10-09T21:47:05.967066Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:05.967110Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341836066295647:7899]
2025-10-09T21:47:05.977830Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|20e34b0c-24892dc8-a8d44dda-5432c39f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:05.977890Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:05.977905Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 854
2025-10-09T21:47:05.978113Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 854 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:05.978149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 854 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:05.978156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=854 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:05.978160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:05.978590Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 854 sessionId:  describe result for acl check
2025-10-09T21:47:05.978632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:05.978639Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:05.978641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:05.978647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341836066295661:8219] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:05.979223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341836066295661:8219] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:05.981354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341836066295661:8219] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:05.990297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341836066295661:8219] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:05.990316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341836066295661:8219] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:05.990320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341836066295661:8219] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:05.990330Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 854 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:05.999220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:06.002182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:06.003108Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:06.003124Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:06.003150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:06.003276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:06.003332Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341839999444313:7439]
2025-10-09T21:47:06.003953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:06.004510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:06.004575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:06.004943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:06.005029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:06.005037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:06.005038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:06.005040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:06.005042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:06.005043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:06.005044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.005045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.005047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.005054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:06.005065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:06.005070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:06.005075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:06.005079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:06.005082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:06.005086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.005090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.005094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.005102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:06.005111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:06.005117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:06.005122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:06.005127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:06.005133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:06.005137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:06.005143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:06.005151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:06.005157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:06.005162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:06.005167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:06.005173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.005179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.005184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.005189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.005194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.005200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.005232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:06.013533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:06.014855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:06.014874Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:06.014877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:06.014882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:06.014890Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 853 [2:7559341839999444313:7439]
2025-10-09T21:47:06.014906Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:06.014932Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:06.014935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:06.014939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:06.014942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:06.014963Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:06.014970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:06.014978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:06.015056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.015059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.015061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.015065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:06.015148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:06.015151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.015167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.015180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.015197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:06.015221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341835704477012:7439]
2025-10-09T21:47:06.015227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341835704477012:7439]
2025-10-09T21:47:06.015229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341835704477012:7439]
2025-10-09T21:47:06.015244Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:06.015260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:06.015277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:06.015280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:06.015300Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:06.015305Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:06.015308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:06.015324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341835704477012:7439]
2025-10-09T21:47:06.015327Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:06.015356Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:06.015396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341836066295763:8219], now have 1 active actors on pipe
2025-10-09T21:47:06.015408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341836066295652:2432], now have 1 active actors on pipe
2025-10-09T21:47:06.015712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 853
2025-10-09T21:47:06.015742Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 853
2025-10-09T21:47:06.016063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:06.016076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:06.016114Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|415e8f6a-aa462bba-3001d385-33ab9d61_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:06.016126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:06.016131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:06.016134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:06.016157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:06.016171Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:06.016187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:06.016429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:06.016435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:06.016455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:06.016585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 854 partition: 0 MaxSeqNo: 25 sessionId: src_id|415e8f6a-aa462bba-3001d385-33ab9d61_0
2025-10-09T21:47:06.017119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20e34b0c-24892dc8-a8d44dda-5432c39f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|415e8f6a-aa462bba-3001d385-33ab9d61_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:06.020319Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|415e8f6a-aa462bba-3001d385-33ab9d61_0] Write session: try to update token
2025-10-09T21:47:06.020339Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|415e8f6a-aa462bba-3001d385-33ab9d61_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:06.035228Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:06.035247Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:06.035251Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:06.035253Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:06.035266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341835704477012:7439]
2025-10-09T21:47:06.035271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341835704477012:7439]
2025-10-09T21:47:06.035277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341835704477012:7439]
2025-10-09T21:47:06.035324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:06.035325Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:06.035331Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:06.035335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.035336Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:06.035338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:06.035471Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:06.041170Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 854 sessionId: src_id|415e8f6a-aa462bba-3001d385-33ab9d61_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:06.043710Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:06.066419Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341840361263132:7899]
2025-10-09T21:47:06.066752Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 853, pipe: [1:7559341840361263132:7899]
2025-10-09T21:47:06.210547Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:06.210911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:06.210928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:06.210979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341840361263132:7899], now have 1 active actors on pipe
2025-10-09T21:47:06.211242Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:06.211374Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341840361263132:7899]
2025-10-09T21:47:06.211512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:06.211521Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:06.211534Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:06.211590Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 854 reason: error at write: Disconnected sessionId: src_id|415e8f6a-aa462bba-3001d385-33ab9d61_0
2025-10-09T21:47:06.211825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 854 sessionId: src_id|415e8f6a-aa462bba-3001d385-33ab9d61_0 is DEAD
2025-10-09T21:47:06.211961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|415e8f6a-aa462bba-3001d385-33ab9d61_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:06.211973Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|415e8f6a-aa462bba-3001d385-33ab9d61_0] Write session will restart in 0.006157s
2025-10-09T21:47:06.212001Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|415e8f6a-aa462bba-3001d385-33ab9d61_0] Write session: Do CDS request
2025-10-09T21:47:06.212008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|415e8f6a-aa462bba-3001d385-33ab9d61_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:06.219576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|415e8f6a-aa462bba-3001d385-33ab9d61_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:06.219839Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:06.219854Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 855
2025-10-09T21:47:06.220103Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 855 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:06.220145Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 855 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:06.220156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=855 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:06.220161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:06.220604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 855 sessionId:  describe result for acl check
2025-10-09T21:47:06.220642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:06.220648Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:06.220649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:06.220655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341840361263167:8224] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:06.221196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341840361263167:8224] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:06.226188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341840361263167:8224] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:06.230681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:06.231015Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341840361263167:8224] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:06.231035Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341840361263167:8224] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:06.231039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341840361263167:8224] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:06.231055Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 855 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:06.233835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:06.234208Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:06.234214Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:06.234229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:06.234298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:06.234345Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341839999444491:7441]
2025-10-09T21:47:06.234919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:06.238470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:06.238549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:06.238927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:06.238988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:06.238993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:06.238994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:06.238995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:06.238997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:06.238998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:06.238999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.239000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.239002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.239007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:06.239018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:06.239021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:06.239024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:06.239028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:06.239031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:06.239035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.239038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.239041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.239048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:06.239056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:06.239061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:06.239066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:06.239070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:06.239074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:06.239077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:06.239082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:06.239086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:06.239090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:06.239094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:06.239099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:06.239103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.239108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.239112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.239116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.239122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.239126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.239155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:06.246204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:06.247379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:06.247395Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:06.247398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:06.247404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:06.247412Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 854 [2:7559341839999444491:7441]
2025-10-09T21:47:06.247449Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:06.247473Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:06.247481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:06.247484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:06.247487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:06.247508Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:06.247523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:06.247533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:06.247614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.247627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.247629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.247632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:06.247727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:06.247735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.247748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.247764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.247784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:06.247808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341840361263143:2432], now have 1 active actors on pipe
2025-10-09T21:47:06.247817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341840361263318:8224], now have 1 active actors on pipe
2025-10-09T21:47:06.247824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341839999444485:7441]
2025-10-09T21:47:06.247831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341839999444485:7441]
2025-10-09T21:47:06.247833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341839999444485:7441]
2025-10-09T21:47:06.247848Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:06.247864Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:06.247880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:06.247889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:06.247899Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 854
2025-10-09T21:47:06.247943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341839999444485:7441]
2025-10-09T21:47:06.247951Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 854
2025-10-09T21:47:06.247951Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:06.248043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:06.248050Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:06.248052Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:06.248058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:06.248225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:06.248238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:06.248268Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|608caccf-4da70b56-2d577027-228973d3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:06.248282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:06.248293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:06.248301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:06.248312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:06.248331Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:06.248348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:06.248566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:06.248577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:06.248600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:06.248863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 855 partition: 0 MaxSeqNo: 25 sessionId: src_id|608caccf-4da70b56-2d577027-228973d3_0
2025-10-09T21:47:06.249258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|415e8f6a-aa462bba-3001d385-33ab9d61_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|608caccf-4da70b56-2d577027-228973d3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:06.251921Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|608caccf-4da70b56-2d577027-228973d3_0] Write session: try to update token
2025-10-09T21:47:06.251941Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|608caccf-4da70b56-2d577027-228973d3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:06.264813Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:06.264831Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:06.264834Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:06.264836Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:06.264846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341839999444485:7441]
2025-10-09T21:47:06.264851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341839999444485:7441]
2025-10-09T21:47:06.264854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341839999444485:7441]
2025-10-09T21:47:06.264903Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:06.264909Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:06.264913Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:06.265167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:06.265183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.265187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:06.265325Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:06.268082Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 855 sessionId: src_id|608caccf-4da70b56-2d577027-228973d3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:06.270635Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:06.311853Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341840361263330:7899]
2025-10-09T21:47:06.312255Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 854, pipe: [1:7559341840361263330:7899]
2025-10-09T21:47:06.431789Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:06.432421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:06.432433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:06.432485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341840361263330:7899], now have 1 active actors on pipe
2025-10-09T21:47:06.433421Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:06.433435Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:06.433490Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 855 reason: error at write: Disconnected sessionId: src_id|608caccf-4da70b56-2d577027-228973d3_0
2025-10-09T21:47:06.433096Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:06.433686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 855 sessionId: src_id|608caccf-4da70b56-2d577027-228973d3_0 is DEAD
2025-10-09T21:47:06.433877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|608caccf-4da70b56-2d577027-228973d3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:06.433890Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|608caccf-4da70b56-2d577027-228973d3_0] Write session will restart in 0.009666s
2025-10-09T21:47:06.433932Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|608caccf-4da70b56-2d577027-228973d3_0] Write session: Do CDS request
2025-10-09T21:47:06.433939Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|608caccf-4da70b56-2d577027-228973d3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:06.433856Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341840361263330:7899]
2025-10-09T21:47:06.433877Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:06.446533Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|608caccf-4da70b56-2d577027-228973d3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:06.446966Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:06.446983Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 856
2025-10-09T21:47:06.447123Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 856 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:06.447154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 856 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:06.447159Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=856 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:06.447163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:06.447772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 856 sessionId:  describe result for acl check
2025-10-09T21:47:06.447820Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:06.447822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:06.447824Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:06.447830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341840361263410:8235] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:06.448441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341840361263410:8235] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:06.453501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341840361263410:8235] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:06.457731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341840361263410:8235] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:06.457748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341840361263410:8235] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:06.457751Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341840361263410:8235] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:06.457759Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 856 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:06.457830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:06.461111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:06.461499Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:06.461512Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:06.461528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:06.461608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:06.461657Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341839999444709:7453]
2025-10-09T21:47:06.462310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:06.463494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:06.463572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:06.463943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:06.464014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:06.464020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:06.464021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:06.464022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:06.464024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:06.464025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:06.464026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.464028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.464029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.464036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:06.464047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:06.464051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:06.464055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:06.464059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:06.464063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:06.464066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.464070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.464074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.464082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:06.464091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:06.464097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:06.464103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:06.464110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:06.464116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:06.464126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:06.464130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:06.464135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:06.464139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:06.464143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:06.464149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:06.464159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.464164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.464180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.464185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.464190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.464195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.464226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:06.473928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:06.475883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:06.475907Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:06.475910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:06.475915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:06.475922Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 855 [2:7559341839999444709:7453]
2025-10-09T21:47:06.475937Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:06.475958Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:06.475962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:06.475965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:06.475968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:06.475988Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:06.476003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:06.476011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:06.476084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.476091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.476094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.476097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:06.476185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:06.476194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.476209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.476225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.476241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:06.476264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341839999444702:7453]
2025-10-09T21:47:06.476273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341839999444702:7453]
2025-10-09T21:47:06.476279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341839999444702:7453]
2025-10-09T21:47:06.476295Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:06.476320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:06.476343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:06.476353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:06.476399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341839999444702:7453]
2025-10-09T21:47:06.476405Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:06.476651Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:06.476659Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:06.476662Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:06.476669Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:06.477308Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 855
2025-10-09T21:47:06.477354Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 855
2025-10-09T21:47:06.477962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341840361263351:2432], now have 1 active actors on pipe
2025-10-09T21:47:06.477973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341840361263530:8235], now have 1 active actors on pipe
2025-10-09T21:47:06.477994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:06.478000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:06.478053Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4fc10b68-fc2f1712-6a8d9c28-f049f90b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:06.478064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:06.478069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:06.478072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:06.478085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:06.478101Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:06.478121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:06.478612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:06.478628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:06.478650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:06.479286Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|608caccf-4da70b56-2d577027-228973d3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4fc10b68-fc2f1712-6a8d9c28-f049f90b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:06.478844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 856 partition: 0 MaxSeqNo: 25 sessionId: src_id|4fc10b68-fc2f1712-6a8d9c28-f049f90b_0
2025-10-09T21:47:06.489891Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4fc10b68-fc2f1712-6a8d9c28-f049f90b_0] Write session: try to update token
2025-10-09T21:47:06.489934Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4fc10b68-fc2f1712-6a8d9c28-f049f90b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:06.496775Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:06.496795Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:06.496798Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:06.496800Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:06.496811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341839999444702:7453]
2025-10-09T21:47:06.496816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341839999444702:7453]
2025-10-09T21:47:06.496821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341839999444702:7453]
2025-10-09T21:47:06.496872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:06.496879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.496882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:06.496965Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:06.496971Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:06.496975Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:06.497076Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:06.517259Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 856 sessionId: src_id|4fc10b68-fc2f1712-6a8d9c28-f049f90b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:06.519943Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:06.534273Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341840361263544:7899]
2025-10-09T21:47:06.534852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 855, pipe: [1:7559341840361263544:7899]
2025-10-09T21:47:06.659413Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:06.659819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:06.659830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:06.659853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341840361263544:7899], now have 1 active actors on pipe
2025-10-09T21:47:06.660114Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:06.660143Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341840361263544:7899]
2025-10-09T21:47:06.660180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:06.660183Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:06.660254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 856 reason: error at write: Disconnected sessionId: src_id|4fc10b68-fc2f1712-6a8d9c28-f049f90b_0
2025-10-09T21:47:06.660438Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 856 sessionId: src_id|4fc10b68-fc2f1712-6a8d9c28-f049f90b_0 is DEAD
2025-10-09T21:47:06.660467Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:06.660685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4fc10b68-fc2f1712-6a8d9c28-f049f90b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:06.660696Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4fc10b68-fc2f1712-6a8d9c28-f049f90b_0] Write session will restart in 0.005391s
2025-10-09T21:47:06.660739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4fc10b68-fc2f1712-6a8d9c28-f049f90b_0] Write session: Do CDS request
2025-10-09T21:47:06.660746Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4fc10b68-fc2f1712-6a8d9c28-f049f90b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:06.667591Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4fc10b68-fc2f1712-6a8d9c28-f049f90b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:06.667705Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:06.667721Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 857
2025-10-09T21:47:06.668019Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 857 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:06.668074Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 857 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:06.668080Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=857 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:06.668084Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:06.668523Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 857 sessionId:  describe result for acl check
2025-10-09T21:47:06.668555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:06.668557Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:06.668558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:06.668564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341840361263563:8239] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:06.669113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341840361263563:8239] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:06.673218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341840361263563:8239] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:06.682154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341840361263563:8239] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:06.682174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341840361263563:8239] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:06.682177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341840361263563:8239] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:06.682188Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 857 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:06.690552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:06.693260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:06.693448Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:06.693457Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:06.693467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:06.693508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:06.693550Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341839999444894:7459]
2025-10-09T21:47:06.693948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:06.694409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:06.694455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:06.694763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:06.694818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:06.694819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:06.694820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:06.694821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:06.694822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:06.694823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:06.694824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.694825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.694826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.694832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:06.694842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:06.694847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:06.694850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:06.694853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:06.694857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:06.694861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.694864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.694868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.694875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:06.694883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:06.694890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:06.694894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:06.694901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:06.694906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:06.694909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:06.694914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:06.694918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:06.694922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:06.694926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:06.694932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:06.694937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.694941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.694945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.694950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.694954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.694959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.694986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:06.704558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:06.705997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:06.706016Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:06.706018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:06.706023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:06.706044Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 856 [2:7559341839999444894:7459]
2025-10-09T21:47:06.706061Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:06.706084Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:06.706087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:06.706090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:06.706093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:06.706114Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:06.706120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:06.706128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:06.707286Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 856
2025-10-09T21:47:06.707319Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 856
2025-10-09T21:47:06.706198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.706206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.706207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.706210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:06.706284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:06.706286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.706299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:06.706311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:06.706328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:06.706351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341839999444888:7459]
2025-10-09T21:47:06.706353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341839999444888:7459]
2025-10-09T21:47:06.706355Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341839999444888:7459]
2025-10-09T21:47:06.706368Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:06.706383Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:06.706398Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:06.706401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:06.706448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341839999444888:7459]
2025-10-09T21:47:06.706451Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:06.706658Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:06.706661Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:06.706663Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:06.706669Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:06.711390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341840361263682:8239], now have 1 active actors on pipe
2025-10-09T21:47:06.711406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341840361263547:2432], now have 1 active actors on pipe
2025-10-09T21:47:06.711432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:06.711440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:06.711486Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ae2c8edc-2e02a1c-b4098115-34396c45_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:06.711500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:06.711506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:06.711509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:06.711525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:06.711545Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:06.711567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:06.714714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:06.714739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:06.714785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:06.714991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 857 partition: 0 MaxSeqNo: 25 sessionId: src_id|ae2c8edc-2e02a1c-b4098115-34396c45_0
2025-10-09T21:47:06.718227Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4fc10b68-fc2f1712-6a8d9c28-f049f90b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ae2c8edc-2e02a1c-b4098115-34396c45_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:06.721092Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae2c8edc-2e02a1c-b4098115-34396c45_0] Write session: try to update token
2025-10-09T21:47:06.721118Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae2c8edc-2e02a1c-b4098115-34396c45_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:06.738965Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:06.738980Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:06.738992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341839999444888:7459]
2025-10-09T21:47:06.739050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:06.739236Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:06.739376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:06.739508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:06.739639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:06.739860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:06.739908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:06.740062Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:06.754326Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 857 sessionId: src_id|ae2c8edc-2e02a1c-b4098115-34396c45_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:06.757245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:06.757455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:06.757477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:06.757527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:06.760482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341840361263742:7899], now have 1 active actors on pipe
2025-10-09T21:47:06.758389Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:06.759610Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341840361263742:7899]
2025-10-09T21:47:06.760437Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 856, pipe: [1:7559341840361263742:7899]
2025-10-09T21:47:06.766960Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:06.766983Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:06.766987Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:06.766989Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:06.767002Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341839999444888:7459]
2025-10-09T21:47:06.767006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341839999444888:7459]
2025-10-09T21:47:06.767012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341839999444888:7459]
2025-10-09T21:47:06.767068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:06.767077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:06.767080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:06.767266Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:06.770258Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:06.770279Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:06.770283Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:07.031914Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:07.040344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:07.040377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:07.040563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:07.040633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:07.040701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:07.040769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:07.040898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:07.040903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:07.040980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:07.041043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:07.041108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:07.041175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:07.041215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:07.041218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:07.041357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:07.041463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:07.041546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:07.041670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:07.041718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:07.041721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:07.041802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:07.041895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:07.041974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:07.044826Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:07.044827Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341840361263742:7899]
2025-10-09T21:47:07.044843Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:07.044858Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:07.044876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 857 reason: error at write: Disconnected sessionId: src_id|ae2c8edc-2e02a1c-b4098115-34396c45_0
2025-10-09T21:47:07.045389Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae2c8edc-2e02a1c-b4098115-34396c45_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:07.045403Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae2c8edc-2e02a1c-b4098115-34396c45_0] Write session will restart in 0.007092s
2025-10-09T21:47:07.045427Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae2c8edc-2e02a1c-b4098115-34396c45_0] Write session: Do CDS request
2025-10-09T21:47:07.045433Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae2c8edc-2e02a1c-b4098115-34396c45_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:07.042214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:07.042267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:07.042271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:07.042345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:07.042446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:07.042542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:07.042605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:07.042645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:07.042649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:07.042711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:07.042770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:07.042831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:07.042894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:07.042939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:07.042942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:07.043010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:07.043065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:07.043125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:07.043187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:07.043232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:07.043235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:07.043297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:07.043353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:07.043410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:07.043468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:07.043635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:07.043638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:07.043710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:07.043766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:07.043828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:07.043887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:07.043929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:07.043933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:07.043999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:07.044055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:07.044118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:07.044180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:07.044314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:07.044317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:07.045520Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:07.045197Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 857 sessionId: src_id|ae2c8edc-2e02a1c-b4098115-34396c45_0 is DEAD
2025-10-09T21:47:07.053606Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae2c8edc-2e02a1c-b4098115-34396c45_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:07.058184Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:07.058201Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 858
2025-10-09T21:47:07.058398Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 858 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:07.058432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 858 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:07.058439Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=858 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:07.058443Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:07.058894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 858 sessionId:  describe result for acl check
2025-10-09T21:47:07.058933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:07.058940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:07.058942Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:07.058949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341844656231113:8246] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:07.059510Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341844656231113:8246] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:07.063779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341844656231113:8246] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:07.070878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:07.071075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341844656231113:8246] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:07.071093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341844656231113:8246] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:07.071097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341844656231113:8246] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:07.071107Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 858 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:07.074560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:07.074875Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:07.074889Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:07.074907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:07.074988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:07.075034Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341844294412372:7461]
2025-10-09T21:47:07.075694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:07.076199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:07.076268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:07.076643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:07.076722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:07.076729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:07.076731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:07.076732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:07.076733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:07.076734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:07.076735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.076737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.076738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.076744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:07.076756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:07.076759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:07.076763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:07.076767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:07.076770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:07.076774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.076777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.076781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.076790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:07.076801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:07.076813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:07.076819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:07.076825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:07.076830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:07.076835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:07.076841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:07.076847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:07.076853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:07.076860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:07.076865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:07.076871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.076881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.076886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.076896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.076900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.076905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.076937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:07.088397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:07.089476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:07.089500Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:07.089503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:07.089508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:07.089516Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 857 [2:7559341844294412372:7461]
2025-10-09T21:47:07.089530Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:07.089567Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:07.089576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:07.089579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:07.089582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:07.089603Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:07.089618Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:07.089626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:07.089698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.089707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.089709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.089713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:07.089788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:07.089795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.089807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.089818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.089840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:07.089870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341844294412366:7461]
2025-10-09T21:47:07.089873Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341844294412366:7461]
2025-10-09T21:47:07.089875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341844294412366:7461]
2025-10-09T21:47:07.089893Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:07.089924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341844656231221:8246], now have 1 active actors on pipe
2025-10-09T21:47:07.089936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341844656231045:2432], now have 1 active actors on pipe
2025-10-09T21:47:07.089945Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:07.089959Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:07.089965Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:07.089968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:07.089969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:07.089973Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:07.090009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341844294412366:7461]
2025-10-09T21:47:07.090021Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:07.090061Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 857
2025-10-09T21:47:07.090092Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 857
2025-10-09T21:47:07.090084Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:07.090942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:07.090956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:07.090993Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bb733357-c6330e93-a8b94ab7-9ebd3e42_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:07.091002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:07.091007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:07.091010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:07.091021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:07.091040Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:07.091064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:07.091431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:07.091443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:07.091472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:07.091694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 858 partition: 0 MaxSeqNo: 25 sessionId: src_id|bb733357-c6330e93-a8b94ab7-9ebd3e42_0
2025-10-09T21:47:07.094336Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae2c8edc-2e02a1c-b4098115-34396c45_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bb733357-c6330e93-a8b94ab7-9ebd3e42_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:07.097029Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb733357-c6330e93-a8b94ab7-9ebd3e42_0] Write session: try to update token
2025-10-09T21:47:07.097050Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb733357-c6330e93-a8b94ab7-9ebd3e42_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:07.108665Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:07.108690Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:07.108694Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:07.108697Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:07.108709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341844294412366:7461]
2025-10-09T21:47:07.108716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341844294412366:7461]
2025-10-09T21:47:07.108722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341844294412366:7461]
2025-10-09T21:47:07.108779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:07.108788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.108791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:07.109004Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:07.109595Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:07.109623Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:07.109628Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:07.127352Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 858 sessionId: src_id|bb733357-c6330e93-a8b94ab7-9ebd3e42_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:07.130081Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:07.145259Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341844656231236:7899]
2025-10-09T21:47:07.145816Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 857, pipe: [1:7559341844656231236:7899]
2025-10-09T21:47:07.291456Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:07.291961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:07.291981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:07.292005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341844656231236:7899], now have 1 active actors on pipe
2025-10-09T21:47:07.292376Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341844656231236:7899]
2025-10-09T21:47:07.292415Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:07.292418Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:07.292262Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:07.292471Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 858 reason: error at write: Disconnected sessionId: src_id|bb733357-c6330e93-a8b94ab7-9ebd3e42_0
2025-10-09T21:47:07.292655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 858 sessionId: src_id|bb733357-c6330e93-a8b94ab7-9ebd3e42_0 is DEAD
2025-10-09T21:47:07.292774Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:07.293935Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb733357-c6330e93-a8b94ab7-9ebd3e42_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:07.293946Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb733357-c6330e93-a8b94ab7-9ebd3e42_0] Write session will restart in 0.005245s
2025-10-09T21:47:07.293974Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb733357-c6330e93-a8b94ab7-9ebd3e42_0] Write session: Do CDS request
2025-10-09T21:47:07.293980Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb733357-c6330e93-a8b94ab7-9ebd3e42_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:07.302424Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb733357-c6330e93-a8b94ab7-9ebd3e42_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:07.302568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:07.302584Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 859
2025-10-09T21:47:07.302780Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 859 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:07.302819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 859 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:07.302826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=859 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:07.302830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:07.303531Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 859 sessionId:  describe result for acl check
2025-10-09T21:47:07.303570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:07.303574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:07.303576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:07.303583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341844656231277:8251] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:07.304054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341844656231277:8251] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:07.315118Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341844656231277:8251] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:07.318357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:07.323005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:07.322953Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341844656231277:8251] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:07.322973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341844656231277:8251] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:07.322977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341844656231277:8251] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:07.322987Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 859 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:07.323272Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:07.323286Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:07.323300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:07.323368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:07.323406Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341844294412567:7463]
2025-10-09T21:47:07.324019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:07.325241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:07.325299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:07.325660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:07.325714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:07.325716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:07.325717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:07.325718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:07.325719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:07.325721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:07.325722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.325724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.325725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.325731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:07.325742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:07.325745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:07.325749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:07.325753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:07.325757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:07.325761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.325765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.325770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.325778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:07.325786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:07.325792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:07.325798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:07.325802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:07.325808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:07.325812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:07.325817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:07.325822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:07.325827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:07.325832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:07.325838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:07.325844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.325850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.325855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.325860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.325864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.325870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.325901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:07.334821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:07.336254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:07.336275Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:07.336277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:07.336282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:07.336289Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 858 [2:7559341844294412567:7463]
2025-10-09T21:47:07.336304Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:07.336324Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:07.336327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:07.336331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:07.336334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:07.336759Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 858
2025-10-09T21:47:07.336801Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 858
2025-10-09T21:47:07.336353Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:07.336372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:07.336380Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:07.336443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.336445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.336448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.336451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:07.336530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:07.336533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.336546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.336558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.336576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:07.336599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341844294412562:7463]
2025-10-09T21:47:07.336602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341844294412562:7463]
2025-10-09T21:47:07.336604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341844294412562:7463]
2025-10-09T21:47:07.336613Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:07.336628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:07.336644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:07.336647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:07.336653Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:07.336657Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:07.336660Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:07.336697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341844656231423:8251], now have 1 active actors on pipe
2025-10-09T21:47:07.336701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341844656231244:2432], now have 1 active actors on pipe
2025-10-09T21:47:07.336704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341844294412562:7463]
2025-10-09T21:47:07.336710Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:07.336745Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:07.336788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:07.336794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:07.336821Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e1cf8d8f-496f9a03-9f7c24a5-1d3d949c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:07.336830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:07.336834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:07.336837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:07.336848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:07.336858Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:07.336876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:07.337311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:07.337323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:07.337353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:07.337564Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 859 partition: 0 MaxSeqNo: 25 sessionId: src_id|e1cf8d8f-496f9a03-9f7c24a5-1d3d949c_0
2025-10-09T21:47:07.338262Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb733357-c6330e93-a8b94ab7-9ebd3e42_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e1cf8d8f-496f9a03-9f7c24a5-1d3d949c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:07.340865Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1cf8d8f-496f9a03-9f7c24a5-1d3d949c_0] Write session: try to update token
2025-10-09T21:47:07.340883Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1cf8d8f-496f9a03-9f7c24a5-1d3d949c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:07.351352Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:07.351370Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:07.351380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341844294412562:7463]
2025-10-09T21:47:07.351432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:07.351609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:07.351787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:07.351922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:07.352065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:07.352265Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:07.352314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:07.352545Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:07.365192Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 859 sessionId: src_id|e1cf8d8f-496f9a03-9f7c24a5-1d3d949c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:07.366654Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:07.366672Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:07.366674Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:07.366677Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:07.366687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341844294412562:7463]
2025-10-09T21:47:07.366691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341844294412562:7463]
2025-10-09T21:47:07.366695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341844294412562:7463]
2025-10-09T21:47:07.366747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:07.366757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.366760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:07.366763Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:07.366771Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:07.366795Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:07.366900Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:07.367906Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:07.392513Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341844656231436:7899]
2025-10-09T21:47:07.401818Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 858, pipe: [1:7559341844656231436:7899]
2025-10-09T21:47:07.537472Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:07.537581Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341844656231436:7899]
2025-10-09T21:47:07.537605Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:07.537828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 859 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e1cf8d8f-496f9a03-9f7c24a5-1d3d949c_0
2025-10-09T21:47:07.538059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 859 sessionId: src_id|e1cf8d8f-496f9a03-9f7c24a5-1d3d949c_0 is DEAD
2025-10-09T21:47:07.535890Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:07.536297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:07.536310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:07.536336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341844656231436:7899], now have 1 active actors on pipe
2025-10-09T21:47:07.536666Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:07.542143Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1cf8d8f-496f9a03-9f7c24a5-1d3d949c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:07.542156Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1cf8d8f-496f9a03-9f7c24a5-1d3d949c_0] Write session will restart in 0.006775s
2025-10-09T21:47:07.542185Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1cf8d8f-496f9a03-9f7c24a5-1d3d949c_0] Write session: Do CDS request
2025-10-09T21:47:07.542192Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1cf8d8f-496f9a03-9f7c24a5-1d3d949c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:07.554491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1cf8d8f-496f9a03-9f7c24a5-1d3d949c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:07.555157Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:07.555174Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 860
2025-10-09T21:47:07.558218Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 860 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:07.558274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 860 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:07.558281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=860 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:07.558290Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:07.559246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 860 sessionId:  describe result for acl check
2025-10-09T21:47:07.559285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:07.559287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:07.559288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:07.559294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341844656231489:8262] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:07.559811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341844656231489:8262] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:07.562848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341844656231489:8262] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:07.570876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341844656231489:8262] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:07.570893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341844656231489:8262] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:07.570896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341844656231489:8262] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:07.570906Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 860 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:07.572522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:07.575510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:07.575743Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:07.575748Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:07.575760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:07.575816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:07.575851Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341844294412776:7477]
2025-10-09T21:47:07.576436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:07.577023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:07.577062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:07.577381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:07.577423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:07.577424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:07.577425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:07.577425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:07.577426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:07.577428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:07.577429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.577430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.577431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.577437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:07.577448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:07.577452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:07.577456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:07.577459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:07.577463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:07.577467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.577471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.577475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.577481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:07.577489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:07.577495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:07.577500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:07.577504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:07.577509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:07.577513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:07.577518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:07.577523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:07.577528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:07.577532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:07.577538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:07.577542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.577547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.577551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.577557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.577561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.577577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.577607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:07.591863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:07.592980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:07.592987Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:07.592990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:07.592995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:07.593003Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 859 [2:7559341844294412776:7477]
2025-10-09T21:47:07.593020Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:07.593043Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:07.593047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:07.593051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:07.593070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:07.593089Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:07.593096Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:07.593103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:07.593169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.593171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.593173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.593176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:07.593255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:07.593258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.593271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.593282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.593300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:07.593319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341844294412771:7477]
2025-10-09T21:47:07.593322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341844294412771:7477]
2025-10-09T21:47:07.593325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341844294412771:7477]
2025-10-09T21:47:07.593333Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:07.593343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:07.593348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:07.593358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:07.593360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:07.593363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:07.593367Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:07.593380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341844656231638:8262], now have 1 active actors on pipe
2025-10-09T21:47:07.593384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341844656231466:2432], now have 1 active actors on pipe
2025-10-09T21:47:07.593423Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341844294412771:7477]
2025-10-09T21:47:07.593431Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:07.593452Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:07.594243Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 859
2025-10-09T21:47:07.594276Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 859
2025-10-09T21:47:07.594377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:07.594389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:07.594417Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e31c2a65-6e2c1a97-83d71793-377b98cb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:07.594430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:07.594434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:07.594436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:07.594445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:07.594455Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:07.594469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:07.597615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:07.597623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:07.597652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:07.598158Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 860 partition: 0 MaxSeqNo: 25 sessionId: src_id|e31c2a65-6e2c1a97-83d71793-377b98cb_0
2025-10-09T21:47:07.602575Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1cf8d8f-496f9a03-9f7c24a5-1d3d949c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e31c2a65-6e2c1a97-83d71793-377b98cb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:07.605107Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e31c2a65-6e2c1a97-83d71793-377b98cb_0] Write session: try to update token
2025-10-09T21:47:07.605128Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e31c2a65-6e2c1a97-83d71793-377b98cb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:07.609337Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:07.609359Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:07.609370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341844294412771:7477]
2025-10-09T21:47:07.609421Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:07.609602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:07.609744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:07.609874Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:07.610015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:07.610237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:07.610279Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:07.610418Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:07.631628Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:07.631647Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:07.631649Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:07.631651Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:07.631663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341844294412771:7477]
2025-10-09T21:47:07.631667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341844294412771:7477]
2025-10-09T21:47:07.631671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341844294412771:7477]
2025-10-09T21:47:07.631716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:07.631722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.631725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:07.631864Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:07.635195Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:07.635211Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:07.635215Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:07.639834Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341844656231653:7899]
2025-10-09T21:47:07.642457Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 859, pipe: [1:7559341844656231653:7899]
2025-10-09T21:47:07.643367Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 860 sessionId: src_id|e31c2a65-6e2c1a97-83d71793-377b98cb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:07.645952Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:07.824656Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:07.825047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341844656231653:7899], now have 1 active actors on pipe
2025-10-09T21:47:07.825079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:07.825086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:07.826253Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:07.826311Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341844656231653:7899]
2025-10-09T21:47:07.826333Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:07.826335Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:07.826396Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 860 reason: error at write: Disconnected sessionId: src_id|e31c2a65-6e2c1a97-83d71793-377b98cb_0
2025-10-09T21:47:07.826589Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 860 sessionId: src_id|e31c2a65-6e2c1a97-83d71793-377b98cb_0 is DEAD
2025-10-09T21:47:07.827057Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e31c2a65-6e2c1a97-83d71793-377b98cb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:07.827068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e31c2a65-6e2c1a97-83d71793-377b98cb_0] Write session will restart in 0.007212s
2025-10-09T21:47:07.827094Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e31c2a65-6e2c1a97-83d71793-377b98cb_0] Write session: Do CDS request
2025-10-09T21:47:07.827104Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e31c2a65-6e2c1a97-83d71793-377b98cb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:07.828005Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:07.836046Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e31c2a65-6e2c1a97-83d71793-377b98cb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:07.836366Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:07.836383Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 861
2025-10-09T21:47:07.836622Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 861 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:07.836655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 861 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:07.836661Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=861 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:07.836664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:07.837174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 861 sessionId:  describe result for acl check
2025-10-09T21:47:07.837209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:07.837211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:07.837213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:07.837218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341844656231675:8267] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:07.837757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341844656231675:8267] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:07.842595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341844656231675:8267] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:07.850442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341844656231675:8267] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:07.850465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341844656231675:8267] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:07.850469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341844656231675:8267] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:07.850479Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 861 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:07.856782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:07.858630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:07.858887Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:07.858899Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:07.858914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:07.858992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:07.859040Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341844294412965:7481]
2025-10-09T21:47:07.859615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:07.864598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:07.864707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:07.865176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:07.865271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:07.865273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:07.865275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:07.865276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:07.865278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:07.865279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:07.865281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.865282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.865284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.865292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:07.865305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:07.865310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:07.865314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:07.865319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:07.865324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:07.865328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.865332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.865336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.865345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:07.865356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:07.865363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:07.865369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:07.865374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:07.865379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:07.865386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:07.865392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:07.865400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:07.865406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:07.865410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:07.865416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:07.865421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.865426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.865431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.865437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.865442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.865447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.865477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:07.876645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:07.877668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:07.877684Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:07.877687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:07.877692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:07.877700Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 860 [2:7559341844294412965:7481]
2025-10-09T21:47:07.877718Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:07.877739Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:07.877742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:07.877745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:07.877747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:07.877769Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:07.877774Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:07.877783Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:07.877848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.877850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.877852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.877855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:07.877942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:07.877950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.877962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:07.877972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:07.877986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:07.878010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341844656231798:8267], now have 1 active actors on pipe
2025-10-09T21:47:07.878018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341844294412960:7481]
2025-10-09T21:47:07.878027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341844294412960:7481]
2025-10-09T21:47:07.878046Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341844294412960:7481]
2025-10-09T21:47:07.878061Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:07.878073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341844656231659:2432], now have 1 active actors on pipe
2025-10-09T21:47:07.878083Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:07.878095Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:07.878100Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:07.878101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:07.878102Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:07.878104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:07.878148Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341844294412960:7481]
2025-10-09T21:47:07.878156Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:07.878167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:07.878329Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 860
2025-10-09T21:47:07.878360Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 860
2025-10-09T21:47:07.878448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:07.878459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:07.878491Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3a18f81f-6efdbf80-5500868-88c5dab5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:07.878502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:07.878505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:07.878508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:07.878520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:07.878533Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:07.878550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:07.880040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:07.880052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:07.880075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:07.880294Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 861 partition: 0 MaxSeqNo: 25 sessionId: src_id|3a18f81f-6efdbf80-5500868-88c5dab5_0
2025-10-09T21:47:07.880715Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e31c2a65-6e2c1a97-83d71793-377b98cb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3a18f81f-6efdbf80-5500868-88c5dab5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:07.883436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a18f81f-6efdbf80-5500868-88c5dab5_0] Write session: try to update token
2025-10-09T21:47:07.883459Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a18f81f-6efdbf80-5500868-88c5dab5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:07.901477Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:07.901497Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:07.901500Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:07.901502Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:07.901514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341844294412960:7481]
2025-10-09T21:47:07.901519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341844294412960:7481]
2025-10-09T21:47:07.901522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341844294412960:7481]
2025-10-09T21:47:07.901573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:07.901581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:07.901583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:07.901590Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:07.901598Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:07.901603Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:07.901726Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:07.928041Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341844656231853:7899]
2025-10-09T21:47:07.928088Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 861 sessionId: src_id|3a18f81f-6efdbf80-5500868-88c5dab5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:07.930800Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:07.930883Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 860, pipe: [1:7559341844656231853:7899]
2025-10-09T21:47:08.097097Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:08.097570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341844656231853:7899], now have 1 active actors on pipe
2025-10-09T21:47:08.097626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:08.097639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:08.098610Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:08.099756Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:08.099827Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341844656231853:7899]
2025-10-09T21:47:08.099858Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:08.099942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 861 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3a18f81f-6efdbf80-5500868-88c5dab5_0
2025-10-09T21:47:08.100156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 861 sessionId: src_id|3a18f81f-6efdbf80-5500868-88c5dab5_0 is DEAD
2025-10-09T21:47:08.104018Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a18f81f-6efdbf80-5500868-88c5dab5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:08.104034Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a18f81f-6efdbf80-5500868-88c5dab5_0] Write session will restart in 0.005486s
2025-10-09T21:47:08.104062Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a18f81f-6efdbf80-5500868-88c5dab5_0] Write session: Do CDS request
2025-10-09T21:47:08.104083Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a18f81f-6efdbf80-5500868-88c5dab5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:08.110566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a18f81f-6efdbf80-5500868-88c5dab5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:08.111173Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:08.111192Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 862
2025-10-09T21:47:08.111411Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 862 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:08.111444Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 862 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:08.111455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=862 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:08.111459Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:08.112584Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 862 sessionId:  describe result for acl check
2025-10-09T21:47:08.112624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:08.112626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:08.112628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:08.112634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341848951199164:8271] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:08.113353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341848951199164:8271] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:08.122370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341848951199164:8271] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:08.136287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341848951199164:8271] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:08.136310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341848951199164:8271] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:08.136315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341848951199164:8271] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:08.136326Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 862 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:08.159661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:08.163090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:08.163376Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:08.163389Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:08.163405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:08.163480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:08.163530Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341848589380447:7483]
2025-10-09T21:47:08.164146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:08.165003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:08.165071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:08.165434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:08.165543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:08.165544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:08.165546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:08.165547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:08.165549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:08.165550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:08.165552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.165553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.165555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.165561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:08.165573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:08.165578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:08.165582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:08.165586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:08.165590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:08.165594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.165597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.165601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.165609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:08.165618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:08.165627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:08.165633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:08.165638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:08.165643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:08.165648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:08.165653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:08.165660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:08.165665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:08.165669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:08.165676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:08.165682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.165687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.165691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.165696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.165701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.165706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.165736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:08.194161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:08.195892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:08.195914Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:08.195917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:08.195923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:08.195932Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 861 [2:7559341848589380447:7483]
2025-10-09T21:47:08.195950Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:08.195972Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:08.195976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:08.195980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:08.195982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:08.196006Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:08.196013Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:08.196021Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:08.196104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.196106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.196110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.196114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:08.196204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:08.196208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.196224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.196234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.196248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:08.196271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341848589380441:7483]
2025-10-09T21:47:08.196274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341848589380441:7483]
2025-10-09T21:47:08.196276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341848589380441:7483]
2025-10-09T21:47:08.196298Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:08.196314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:08.196329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:08.196332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:08.196387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341848589380441:7483]
2025-10-09T21:47:08.196390Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:08.196411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:08.196420Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:08.196424Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:08.196432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:08.197809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341848951199157:2432], now have 1 active actors on pipe
2025-10-09T21:47:08.197823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341848951199243:8271], now have 1 active actors on pipe
2025-10-09T21:47:08.198461Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 861
2025-10-09T21:47:08.198518Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 861
2025-10-09T21:47:08.198541Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341848951199355:7899]
2025-10-09T21:47:08.199245Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 861, pipe: [1:7559341848951199355:7899]
2025-10-09T21:47:08.200735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:08.200754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:08.200776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341848951199355:7899], now have 1 active actors on pipe
2025-10-09T21:47:08.200806Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6ba90b7a-f9149121-d70c65c7-73f60872_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:08.200820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:08.200827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:08.200830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:08.200847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:08.200868Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:08.200897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:08.201236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:08.201251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:08.201275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:08.201476Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 862 partition: 0 MaxSeqNo: 25 sessionId: src_id|6ba90b7a-f9149121-d70c65c7-73f60872_0
2025-10-09T21:47:08.202484Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a18f81f-6efdbf80-5500868-88c5dab5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6ba90b7a-f9149121-d70c65c7-73f60872_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:08.205641Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ba90b7a-f9149121-d70c65c7-73f60872_0] Write session: try to update token
2025-10-09T21:47:08.205667Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ba90b7a-f9149121-d70c65c7-73f60872_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:08.224859Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:08.224880Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:08.224882Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:08.224885Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:08.224896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341848589380441:7483]
2025-10-09T21:47:08.224901Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341848589380441:7483]
2025-10-09T21:47:08.224904Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341848589380441:7483]
2025-10-09T21:47:08.224953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:08.224961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.224964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:08.225134Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:08.229464Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:08.229478Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:08.229483Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:08.239286Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 862 sessionId: src_id|6ba90b7a-f9149121-d70c65c7-73f60872_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:08.242003Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:08.451105Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:08.451493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:08.451504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:08.451766Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:08.451799Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:08.451804Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:08.451819Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341848951199355:7899]
2025-10-09T21:47:08.451875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 862 reason: error at write: Disconnected sessionId: src_id|6ba90b7a-f9149121-d70c65c7-73f60872_0
2025-10-09T21:47:08.451755Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:08.452058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 862 sessionId: src_id|6ba90b7a-f9149121-d70c65c7-73f60872_0 is DEAD
2025-10-09T21:47:08.452194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ba90b7a-f9149121-d70c65c7-73f60872_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:08.452207Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ba90b7a-f9149121-d70c65c7-73f60872_0] Write session will restart in 0.005773s
2025-10-09T21:47:08.452246Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ba90b7a-f9149121-d70c65c7-73f60872_0] Write session: Do CDS request
2025-10-09T21:47:08.452255Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ba90b7a-f9149121-d70c65c7-73f60872_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:08.459523Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ba90b7a-f9149121-d70c65c7-73f60872_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:08.459924Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:08.459952Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 863
2025-10-09T21:47:08.460200Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 863 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:08.460247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 863 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:08.460254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=863 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:08.460258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:08.460655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 863 sessionId:  describe result for acl check
2025-10-09T21:47:08.460685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:08.460687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:08.460689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:08.460694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341848951199387:8276] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:08.461264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341848951199387:8276] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:08.471379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:08.472419Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341848951199387:8276] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:08.473406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:08.473638Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:08.473646Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:08.473658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:08.473724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:08.473767Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341848589380663:7495]
2025-10-09T21:47:08.474385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:08.474829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341848951199387:8276] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:08.474844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341848951199387:8276] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:08.474847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341848951199387:8276] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:08.474856Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 863 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:08.474876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:08.474940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:08.475286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:08.475330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:08.475331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:08.475332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:08.475334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:08.475335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:08.475336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:08.475337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.475339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.475340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.475345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:08.475357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:08.475361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:08.475365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:08.475369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:08.475374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:08.475378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.475382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.475386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.475393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:08.475401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:08.475407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:08.475413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:08.475418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:08.475423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:08.475428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:08.475433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:08.475437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:08.475442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:08.475447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:08.475452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:08.475457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.475463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.475467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.475473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.475479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.475485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.475515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:08.483870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:08.485467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:08.485484Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:08.485487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:08.485491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:08.485500Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 862 [2:7559341848589380663:7495]
2025-10-09T21:47:08.485514Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:08.485533Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:08.485536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:08.485540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:08.485542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:08.485913Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 862
2025-10-09T21:47:08.485919Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 862
2025-10-09T21:47:08.485559Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:08.485573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:08.485580Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:08.485647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.485650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.485652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.485655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:08.485729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:08.485732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.485745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.485757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.485774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:08.485798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341848951199546:8276], now have 1 active actors on pipe
2025-10-09T21:47:08.485801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341848951199363:2432], now have 1 active actors on pipe
2025-10-09T21:47:08.485809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341848589380658:7495]
2025-10-09T21:47:08.485811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341848589380658:7495]
2025-10-09T21:47:08.485814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341848589380658:7495]
2025-10-09T21:47:08.485830Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:08.485845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:08.485859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:08.485862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:08.485915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341848589380658:7495]
2025-10-09T21:47:08.485917Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:08.486058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:08.486061Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:08.486063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:08.486069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:08.486402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:08.486418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:08.486451Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|378c1c8d-bb65d5ec-efa4a394-900c738a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:08.486474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:08.486480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:08.486487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:08.486499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:08.486514Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:08.486536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:08.489883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:08.489896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:08.489932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:08.490190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 863 partition: 0 MaxSeqNo: 25 sessionId: src_id|378c1c8d-bb65d5ec-efa4a394-900c738a_0
2025-10-09T21:47:08.490608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ba90b7a-f9149121-d70c65c7-73f60872_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|378c1c8d-bb65d5ec-efa4a394-900c738a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:08.493007Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|378c1c8d-bb65d5ec-efa4a394-900c738a_0] Write session: try to update token
2025-10-09T21:47:08.493025Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|378c1c8d-bb65d5ec-efa4a394-900c738a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:08.504003Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:08.504026Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:08.504028Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:08.504030Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:08.504039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341848589380658:7495]
2025-10-09T21:47:08.504043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341848589380658:7495]
2025-10-09T21:47:08.504046Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341848589380658:7495]
2025-10-09T21:47:08.504090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:08.504098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.504100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:08.504127Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:08.504141Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:08.504144Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:08.504226Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:08.522267Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 863 sessionId: src_id|378c1c8d-bb65d5ec-efa4a394-900c738a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:08.524884Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:08.552364Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341848951199578:7899]
2025-10-09T21:47:08.552771Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 862, pipe: [1:7559341848951199578:7899]
2025-10-09T21:47:08.699425Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:08.699811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:08.699821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:08.699845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341848951199578:7899], now have 1 active actors on pipe
2025-10-09T21:47:08.700157Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:08.701254Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|378c1c8d-bb65d5ec-efa4a394-900c738a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:08.701265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|378c1c8d-bb65d5ec-efa4a394-900c738a_0] Write session will restart in 0.006330s
2025-10-09T21:47:08.701290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|378c1c8d-bb65d5ec-efa4a394-900c738a_0] Write session: Do CDS request
2025-10-09T21:47:08.701297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|378c1c8d-bb65d5ec-efa4a394-900c738a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:08.700504Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341848951199578:7899]
2025-10-09T21:47:08.700561Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:08.700633Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 863 reason: status is not ok: tablet will be restarted right now sessionId: src_id|378c1c8d-bb65d5ec-efa4a394-900c738a_0
2025-10-09T21:47:08.700827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 863 sessionId: src_id|378c1c8d-bb65d5ec-efa4a394-900c738a_0 is DEAD
2025-10-09T21:47:08.701003Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:08.708576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|378c1c8d-bb65d5ec-efa4a394-900c738a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:08.708673Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:08.708690Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 864
2025-10-09T21:47:08.708829Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 864 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:08.708866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 864 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:08.708886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=864 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:08.708894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:08.709389Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 864 sessionId:  describe result for acl check
2025-10-09T21:47:08.709430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:08.709436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:08.709437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:08.709443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341848951199596:8287] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:08.710097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341848951199596:8287] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:08.713499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341848951199596:8287] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:08.720867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:08.717023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341848951199596:8287] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:08.717042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341848951199596:8287] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:08.717046Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341848951199596:8287] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:08.717056Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 864 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:08.723114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:08.723352Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:08.723356Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:08.723367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:08.723422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:08.723458Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341848589380863:7501]
2025-10-09T21:47:08.724032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:08.724496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:08.724555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:08.724891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:08.724964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:08.724971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:08.724972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:08.724974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:08.724975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:08.724976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:08.724977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.724978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.724979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.724985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:08.724996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:08.725000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:08.725005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:08.725009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:08.725013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:08.725017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.725021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.725025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.725033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:08.725041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:08.725048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:08.725053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:08.725057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:08.725062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:08.725066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:08.725072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:08.725077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:08.725082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:08.725092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:08.725097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:08.725102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.725107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.725111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.725116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.725121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.725126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.725167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:08.734606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:08.736490Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 863
2025-10-09T21:47:08.735981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:08.736000Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:08.736003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:08.736008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:08.736016Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 863 [2:7559341848589380863:7501]
2025-10-09T21:47:08.736035Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:08.736059Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:08.736062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:08.736065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:08.736068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:08.736087Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:08.736096Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:08.736104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:08.736174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.736176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.736178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.736181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:08.736263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:08.736266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.736281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.736293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.736309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:08.736330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341848589380858:7501]
2025-10-09T21:47:08.736332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341848589380858:7501]
2025-10-09T21:47:08.736336Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341848589380858:7501]
2025-10-09T21:47:08.736353Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:08.736370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341848951199745:8287], now have 1 active actors on pipe
2025-10-09T21:47:08.736373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341848951199581:2432], now have 1 active actors on pipe
2025-10-09T21:47:08.736382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:08.736396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:08.736400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:08.736412Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:08.736417Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:08.736419Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:08.736444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341848589380858:7501]
2025-10-09T21:47:08.736447Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:08.736449Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:08.736747Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 863
2025-10-09T21:47:08.736925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:08.736941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:08.736979Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8a40bc08-1f3d65b5-7e271486-4be1f752_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:08.736993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:08.736998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:08.737000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:08.737013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:08.737026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:08.737044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:08.737409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:08.737436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:08.737459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:08.737606Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 864 partition: 0 MaxSeqNo: 25 sessionId: src_id|8a40bc08-1f3d65b5-7e271486-4be1f752_0
2025-10-09T21:47:08.738003Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|378c1c8d-bb65d5ec-efa4a394-900c738a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8a40bc08-1f3d65b5-7e271486-4be1f752_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:08.740548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a40bc08-1f3d65b5-7e271486-4be1f752_0] Write session: try to update token
2025-10-09T21:47:08.740567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a40bc08-1f3d65b5-7e271486-4be1f752_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:08.756027Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:08.756047Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:08.756057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341848589380858:7501]
2025-10-09T21:47:08.756105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:08.756214Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:08.756289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:08.756429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:08.756564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:08.756695Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:08.756885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:08.756930Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:08.760558Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 864 sessionId: src_id|8a40bc08-1f3d65b5-7e271486-4be1f752_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:08.763022Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:08.763336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:08.763358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:08.763408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:08.763480Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:08.773044Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:08.773065Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:08.773068Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:08.773070Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:08.773082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341848589380858:7501]
2025-10-09T21:47:08.773086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341848589380858:7501]
2025-10-09T21:47:08.773090Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341848589380858:7501]
2025-10-09T21:47:08.773148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:08.773162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.773164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:08.773171Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:08.773177Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:08.773184Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:08.773314Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:08.800146Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341848951199779:7899]
2025-10-09T21:47:08.800424Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 863, pipe: [1:7559341848951199779:7899]
2025-10-09T21:47:08.930645Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:08.938726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:08.938756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:08.938948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:08.939013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:08.939074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:08.939154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:08.939205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:08.939210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:08.939282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:08.939341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:08.939403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:08.939467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:08.939516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:08.939524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:08.939589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:08.939692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:08.939753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:08.939814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:08.939861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:08.939869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:08.939937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:08.940009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:08.940072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:08.940138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:08.940414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:08.940425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:08.940529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:08.940600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:08.940664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:08.940741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:08.940793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:08.940800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:08.940894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:08.940956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:08.941015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:08.941073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:08.941123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:08.941133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:08.941202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:08.941264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:08.941335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:08.941398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:08.941445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:08.941455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:08.941521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:08.941581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:08.941641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:08.941707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:08.941753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:08.941762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:08.941833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:08.941889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:08.941961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:08.942021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:08.942366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:08.942377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:08.942457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:08.942530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:08.942597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:08.942668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:08.942711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:08.942714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:08.942794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341848951199779:7899], now have 1 active actors on pipe
2025-10-09T21:47:08.946363Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:08.946995Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341848951199779:7899]
2025-10-09T21:47:08.947029Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:08.947038Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:08.947079Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 864 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8a40bc08-1f3d65b5-7e271486-4be1f752_0
2025-10-09T21:47:08.947274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 864 sessionId: src_id|8a40bc08-1f3d65b5-7e271486-4be1f752_0 is DEAD
2025-10-09T21:47:08.950097Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a40bc08-1f3d65b5-7e271486-4be1f752_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:08.950111Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a40bc08-1f3d65b5-7e271486-4be1f752_0] Write session will restart in 0.006658s
2025-10-09T21:47:08.950137Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a40bc08-1f3d65b5-7e271486-4be1f752_0] Write session: Do CDS request
2025-10-09T21:47:08.950145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a40bc08-1f3d65b5-7e271486-4be1f752_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:08.957734Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a40bc08-1f3d65b5-7e271486-4be1f752_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:08.957963Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:08.957997Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 865
2025-10-09T21:47:08.958275Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 865 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:08.958326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 865 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:08.958339Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=865 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:08.958348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:08.958985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 865 sessionId:  describe result for acl check
2025-10-09T21:47:08.959029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:08.959036Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:08.959038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:08.959045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341848951199793:8292] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:08.959637Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341848951199793:8292] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:08.963448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341848951199793:8292] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:08.968187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341848951199793:8292] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:08.968206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341848951199793:8292] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:08.968210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341848951199793:8292] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:08.968218Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 865 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:08.982138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:08.984627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:08.984874Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:08.984887Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:08.984900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:08.984986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:08.985026Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341848589381052:7503]
2025-10-09T21:47:08.985618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:08.985962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:08.986056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:08.986408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:08.986462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:08.986469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:08.986471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:08.986472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:08.986473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:08.986474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:08.986476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.986477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.986478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.986484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:08.986496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:08.986500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:08.986504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:08.986508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:08.986512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:08.986516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.986520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.986525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.986533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:08.986541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:08.986547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:08.986553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:08.986558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:08.986564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:08.986569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:08.986576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:08.986587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:08.986593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:08.986597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:08.986603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:08.986608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.986615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.986634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.986639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.986644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.986650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.986678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:08.994991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:08.996318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:08.996333Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:08.996336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:08.996341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:08.996348Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 864 [2:7559341848589381052:7503]
2025-10-09T21:47:08.996363Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:08.996381Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:08.996384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:08.996388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:08.996390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:08.996407Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:08.996413Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:08.996425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:08.996486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.996499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.996501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.996503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:08.996577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:08.996584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:08.996597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:08.996609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:08.996629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:08.996655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341848589381047:7503]
2025-10-09T21:47:08.996662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341848589381047:7503]
2025-10-09T21:47:08.996665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341848589381047:7503]
2025-10-09T21:47:08.996673Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:08.996688Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:08.996702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:08.996711Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:08.996727Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:08.996731Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:08.996733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:08.996751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341848589381047:7503]
2025-10-09T21:47:08.996755Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:08.996764Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:08.996765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341848951199782:2432], now have 1 active actors on pipe
2025-10-09T21:47:08.996769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341848951199828:8292], now have 1 active actors on pipe
2025-10-09T21:47:08.996856Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 864
2025-10-09T21:47:08.996867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 864
2025-10-09T21:47:08.997092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:08.997104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:08.997144Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2577c4a5-42d40bfb-c77c5a65-c060285b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:08.997158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:08.997162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:08.997164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:08.997176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:08.997188Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:08.997203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:08.997404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:08.997420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:08.997446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:08.997576Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 865 partition: 0 MaxSeqNo: 25 sessionId: src_id|2577c4a5-42d40bfb-c77c5a65-c060285b_0
2025-10-09T21:47:08.997978Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a40bc08-1f3d65b5-7e271486-4be1f752_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2577c4a5-42d40bfb-c77c5a65-c060285b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:09.000631Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2577c4a5-42d40bfb-c77c5a65-c060285b_0] Write session: try to update token
2025-10-09T21:47:09.000653Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2577c4a5-42d40bfb-c77c5a65-c060285b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:09.015315Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:09.015335Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:09.015337Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:09.015339Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:09.015349Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341848589381047:7503]
2025-10-09T21:47:09.015354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341848589381047:7503]
2025-10-09T21:47:09.015358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341848589381047:7503]
2025-10-09T21:47:09.015410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:09.015419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.015422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:09.015459Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:09.015473Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:09.015476Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:09.015570Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:09.017689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 865 sessionId: src_id|2577c4a5-42d40bfb-c77c5a65-c060285b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:09.020459Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:09.047301Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341853246167276:7899]
2025-10-09T21:47:09.047632Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 864, pipe: [1:7559341853246167276:7899]
2025-10-09T21:47:09.188244Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:09.188688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:09.188703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:09.188754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341853246167276:7899], now have 1 active actors on pipe
2025-10-09T21:47:09.188975Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:09.189405Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:09.189449Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:09.189502Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341853246167276:7899]
2025-10-09T21:47:09.189518Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 865 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2577c4a5-42d40bfb-c77c5a65-c060285b_0
2025-10-09T21:47:09.189711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 865 sessionId: src_id|2577c4a5-42d40bfb-c77c5a65-c060285b_0 is DEAD
2025-10-09T21:47:09.189863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2577c4a5-42d40bfb-c77c5a65-c060285b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:09.189889Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2577c4a5-42d40bfb-c77c5a65-c060285b_0] Write session will restart in 0.007563s
2025-10-09T21:47:09.189925Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2577c4a5-42d40bfb-c77c5a65-c060285b_0] Write session: Do CDS request
2025-10-09T21:47:09.189932Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2577c4a5-42d40bfb-c77c5a65-c060285b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:09.198667Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2577c4a5-42d40bfb-c77c5a65-c060285b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:09.198779Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:09.198799Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 866
2025-10-09T21:47:09.199048Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 866 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:09.199095Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 866 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:09.199103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=866 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:09.199107Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:09.199732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 866 sessionId:  describe result for acl check
2025-10-09T21:47:09.199777Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:09.199779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:09.199781Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:09.199787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341853246167320:8297] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:09.200387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341853246167320:8297] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:09.205298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341853246167320:8297] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:09.207890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:09.208488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341853246167320:8297] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:09.208508Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341853246167320:8297] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:09.208513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341853246167320:8297] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:09.208523Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 866 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:09.210128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:09.210360Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:09.210372Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:09.210388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:09.210451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:09.210492Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341852884348537:7505]
2025-10-09T21:47:09.211054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:09.211784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:09.211848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:09.212195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:09.212250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:09.212258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:09.212259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:09.212260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:09.212262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:09.212263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:09.212264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.212265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.212267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.212272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:09.212283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:09.212288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:09.212292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:09.212296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:09.212300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:09.212303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.212307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.212310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.212318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:09.212325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:09.212331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:09.212337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:09.212341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:09.212347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:09.212351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:09.212357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:09.212362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:09.212367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:09.212371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:09.212376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:09.212383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.212405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.212418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.212424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.212429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.212434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.212476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:09.221075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:09.222203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:09.222222Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:09.222225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:09.222229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:09.222239Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 865 [2:7559341852884348537:7505]
2025-10-09T21:47:09.222256Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:09.222278Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:09.222281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:09.222284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:09.222287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:09.222307Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:09.222313Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:09.222321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:09.222388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.222390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.222392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.222395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:09.222469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:09.222471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.222483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.222504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.222523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:09.222546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341853246167287:2432], now have 1 active actors on pipe
2025-10-09T21:47:09.222550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341853246167473:8297], now have 1 active actors on pipe
2025-10-09T21:47:09.222558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341852884348531:7505]
2025-10-09T21:47:09.222560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341852884348531:7505]
2025-10-09T21:47:09.222563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341852884348531:7505]
2025-10-09T21:47:09.222578Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:09.222595Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:09.222597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:09.222606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:09.222609Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:09.222611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:09.222615Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:09.222655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341852884348531:7505]
2025-10-09T21:47:09.222664Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:09.222666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:09.222705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:09.222715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:09.222599Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 865
2025-10-09T21:47:09.222663Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 865
2025-10-09T21:47:09.222746Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|62f2dcb3-5d5afef5-e3a8e33b-11e11009_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:09.222758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:09.222763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:09.222765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:09.222777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:09.222787Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:09.222802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:09.222991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:09.223001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:09.223018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:09.223148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 866 partition: 0 MaxSeqNo: 25 sessionId: src_id|62f2dcb3-5d5afef5-e3a8e33b-11e11009_0
2025-10-09T21:47:09.223675Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2577c4a5-42d40bfb-c77c5a65-c060285b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|62f2dcb3-5d5afef5-e3a8e33b-11e11009_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:09.226332Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62f2dcb3-5d5afef5-e3a8e33b-11e11009_0] Write session: try to update token
2025-10-09T21:47:09.226356Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62f2dcb3-5d5afef5-e3a8e33b-11e11009_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:09.236050Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:09.236076Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:09.236087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341852884348531:7505]
2025-10-09T21:47:09.236135Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:09.236138Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:09.236339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:09.236480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:09.236680Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:09.242173Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:09.242460Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:09.242533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:09.244715Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 866 sessionId: src_id|62f2dcb3-5d5afef5-e3a8e33b-11e11009_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:09.247288Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:09.247497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:09.247520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:09.247565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:09.247695Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:09.250644Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:09.250667Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:09.250671Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:09.250674Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:09.250684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341852884348531:7505]
2025-10-09T21:47:09.250689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341852884348531:7505]
2025-10-09T21:47:09.250694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341852884348531:7505]
2025-10-09T21:47:09.250748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:09.250761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.250764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:09.250764Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:09.250773Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:09.250777Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:09.250958Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:09.289994Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341853246167485:7899]
2025-10-09T21:47:09.290320Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 865, pipe: [1:7559341853246167485:7899]
2025-10-09T21:47:09.413595Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:09.431564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:09.431592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:09.431817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.431886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.431954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.432022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.432063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.432079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.432157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.432219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.432283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.432354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.432405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.432408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.432471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.432567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.432624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.432676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.432729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.432733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.432798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.432852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.432911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.432972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.433013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.433016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.433083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.433178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.433238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.433300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.433413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.433417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.433484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.433542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.433598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.433655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.433695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.433697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.433760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.433814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.433873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.433944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.434003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.434006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.434243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.434304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.434360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.434419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.434521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.434525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.434587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.434639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.434695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.434753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.434793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.434796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.434863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.434920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.434982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.435043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.435083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.435086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.435150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341853246167485:7899], now have 1 active actors on pipe
2025-10-09T21:47:09.437031Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:09.437056Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:09.437088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 866 reason: error at write: Disconnected sessionId: src_id|62f2dcb3-5d5afef5-e3a8e33b-11e11009_0
2025-10-09T21:47:09.437298Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:09.437343Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341853246167485:7899]
2025-10-09T21:47:09.436704Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:09.437439Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 866 sessionId: src_id|62f2dcb3-5d5afef5-e3a8e33b-11e11009_0 is DEAD
2025-10-09T21:47:09.437541Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62f2dcb3-5d5afef5-e3a8e33b-11e11009_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:09.437555Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62f2dcb3-5d5afef5-e3a8e33b-11e11009_0] Write session will restart in 0.009453s
2025-10-09T21:47:09.437585Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62f2dcb3-5d5afef5-e3a8e33b-11e11009_0] Write session: Do CDS request
2025-10-09T21:47:09.437592Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62f2dcb3-5d5afef5-e3a8e33b-11e11009_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:09.448624Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62f2dcb3-5d5afef5-e3a8e33b-11e11009_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:09.448741Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:09.448759Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 867
2025-10-09T21:47:09.448887Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 867 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:09.448924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 867 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:09.448934Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=867 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:09.448938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:09.449418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 867 sessionId:  describe result for acl check
2025-10-09T21:47:09.449451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:09.449453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:09.449455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:09.449460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341853246167503:8301] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:09.450051Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341853246167503:8301] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:09.452832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341853246167503:8301] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:09.461985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341853246167503:8301] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:09.462004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341853246167503:8301] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:09.462008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341853246167503:8301] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:09.462019Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 867 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:09.472312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:09.475474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:09.475704Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:09.475709Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:09.475722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:09.475783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:09.475818Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341852884348741:7509]
2025-10-09T21:47:09.476402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:09.476965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:09.477009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:09.477326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:09.477372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:09.477373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:09.477374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:09.477375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:09.477376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:09.477377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:09.477378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.477380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.477381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.477385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:09.477396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:09.477399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:09.477403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:09.477407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:09.477410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:09.477414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.477417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.477421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.477429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:09.477436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:09.477442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:09.477447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:09.477451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:09.477456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:09.477459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:09.477464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:09.477468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:09.477473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:09.477478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:09.477482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:09.477488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.477493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.477497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.477502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.477506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.477511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.477537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:09.488108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:09.489470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:09.489476Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:09.489479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:09.489484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:09.489492Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 866 [2:7559341852884348741:7509]
2025-10-09T21:47:09.489507Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:09.489526Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:09.489529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:09.489533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:09.489535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:09.489556Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:09.489562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:09.489571Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:09.489636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.489638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.489640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.489643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:09.489735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:09.489738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.489752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.489764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.489780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:09.489801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341852884348735:7509]
2025-10-09T21:47:09.489803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341852884348735:7509]
2025-10-09T21:47:09.489805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341852884348735:7509]
2025-10-09T21:47:09.489821Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:09.489835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:09.489850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:09.489852Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:09.489894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341852884348735:7509]
2025-10-09T21:47:09.489897Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:09.490047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:09.490050Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:09.490052Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:09.490060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:09.490263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341853246167609:8301], now have 1 active actors on pipe
2025-10-09T21:47:09.490273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341853246167488:2432], now have 1 active actors on pipe
2025-10-09T21:47:09.490674Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 866
2025-10-09T21:47:09.490719Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 866
2025-10-09T21:47:09.490814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:09.490824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:09.490858Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e0c659d4-994eb899-a053d83c-6114770d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:09.490867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:09.490871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:09.490875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:09.490887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:09.490900Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:09.490919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:09.491304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:09.491312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:09.491329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:09.491765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 867 partition: 0 MaxSeqNo: 25 sessionId: src_id|e0c659d4-994eb899-a053d83c-6114770d_0
2025-10-09T21:47:09.492195Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62f2dcb3-5d5afef5-e3a8e33b-11e11009_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e0c659d4-994eb899-a053d83c-6114770d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:09.500060Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e0c659d4-994eb899-a053d83c-6114770d_0] Write session: try to update token
2025-10-09T21:47:09.500087Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e0c659d4-994eb899-a053d83c-6114770d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:09.507405Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:09.507428Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:09.507431Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:09.507433Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:09.507442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341852884348735:7509]
2025-10-09T21:47:09.507447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341852884348735:7509]
2025-10-09T21:47:09.507450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341852884348735:7509]
2025-10-09T21:47:09.507496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:09.507503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.507508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:09.507508Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:09.507515Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:09.507519Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:09.507636Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:09.524919Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 867 sessionId: src_id|e0c659d4-994eb899-a053d83c-6114770d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:09.532191Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:09.537309Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341853246167688:7899]
2025-10-09T21:47:09.538318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 866, pipe: [1:7559341853246167688:7899]
2025-10-09T21:47:09.675611Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:09.675938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:09.675957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:09.675982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341853246167688:7899], now have 1 active actors on pipe
2025-10-09T21:47:09.676198Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:09.676251Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:09.676271Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:09.676278Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:09.676318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341853246167688:7899]
2025-10-09T21:47:09.676379Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 867 reason: error at write: Disconnected sessionId: src_id|e0c659d4-994eb899-a053d83c-6114770d_0
2025-10-09T21:47:09.676969Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0c659d4-994eb899-a053d83c-6114770d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:09.676981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0c659d4-994eb899-a053d83c-6114770d_0] Write session will restart in 0.009241s
2025-10-09T21:47:09.676589Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 867 sessionId: src_id|e0c659d4-994eb899-a053d83c-6114770d_0 is DEAD
2025-10-09T21:47:09.677016Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0c659d4-994eb899-a053d83c-6114770d_0] Write session: Do CDS request
2025-10-09T21:47:09.677044Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0c659d4-994eb899-a053d83c-6114770d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:09.687569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e0c659d4-994eb899-a053d83c-6114770d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:09.687650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:09.687667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 868
2025-10-09T21:47:09.688008Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 868 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:09.688052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 868 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:09.688059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=868 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:09.688062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:09.689073Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 868 sessionId:  describe result for acl check
2025-10-09T21:47:09.689119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:09.689121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:09.689123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:09.689131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341853246167760:8312] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:09.689749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341853246167760:8312] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:09.697695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341853246167760:8312] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:09.702989Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341853246167760:8312] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:09.703012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341853246167760:8312] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:09.703016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341853246167760:8312] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:09.703026Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 868 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:09.703017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:09.705742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:09.706006Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:09.706020Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:09.706049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:09.706118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:09.706166Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341852884348961:7523]
2025-10-09T21:47:09.706768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:09.707117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:09.707171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:09.707514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:09.707583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:09.707591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:09.707592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:09.707593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:09.707595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:09.707596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:09.707597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.707598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.707599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.707604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:09.707614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:09.707618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:09.707622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:09.707625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:09.707629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:09.707632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.707636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.707640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.707648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:09.707656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:09.707664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:09.707669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:09.707674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:09.707679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:09.707683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:09.707689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:09.707693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:09.707699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:09.707703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:09.707708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:09.707714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.707719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.707723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.707728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.707734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.707739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.707770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:09.714981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:09.716618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:09.716640Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:09.716643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:09.716648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:09.716658Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 867 [2:7559341852884348961:7523]
2025-10-09T21:47:09.716674Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:09.716698Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:09.716701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:09.716704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:09.716707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:09.716728Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:09.716745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:09.716753Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:09.716820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.716829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.716830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.716834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:09.716915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:09.716925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.716940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.716958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.716982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:09.717006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341852884348955:7523]
2025-10-09T21:47:09.717017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341852884348955:7523]
2025-10-09T21:47:09.717020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341852884348955:7523]
2025-10-09T21:47:09.717038Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:09.717053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:09.717056Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:09.717059Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:09.717060Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:09.717076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:09.717085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:09.717130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341852884348955:7523]
2025-10-09T21:47:09.717139Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:09.717146Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:09.717151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341853246167899:8312], now have 1 active actors on pipe
2025-10-09T21:47:09.717155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341853246167715:2432], now have 1 active actors on pipe
2025-10-09T21:47:09.717260Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 867
2025-10-09T21:47:09.717357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:09.717372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:09.717405Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3259cfd2-7bf865e9-674e1ce2-56137bd2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:09.717419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:09.717424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:09.717427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:09.717450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:09.717467Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:09.717491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:09.717682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:09.717700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:09.717717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:09.717737Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 867
2025-10-09T21:47:09.717857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 868 partition: 0 MaxSeqNo: 25 sessionId: src_id|3259cfd2-7bf865e9-674e1ce2-56137bd2_0
2025-10-09T21:47:09.718295Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0c659d4-994eb899-a053d83c-6114770d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3259cfd2-7bf865e9-674e1ce2-56137bd2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:09.720994Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3259cfd2-7bf865e9-674e1ce2-56137bd2_0] Write session: try to update token
2025-10-09T21:47:09.721017Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3259cfd2-7bf865e9-674e1ce2-56137bd2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:09.736098Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 868 sessionId: src_id|3259cfd2-7bf865e9-674e1ce2-56137bd2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:09.739184Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:09.740554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:09.740580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:09.740604Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:09.740607Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:09.740616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341852884348955:7523]
2025-10-09T21:47:09.740658Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:09.740679Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:09.740859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:09.741008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:09.741156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:09.741305Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:09.741509Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:09.741558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:09.741586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:09.741790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:09.750681Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:09.750701Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:09.750704Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:09.750707Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:09.750719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341852884348955:7523]
2025-10-09T21:47:09.750723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341852884348955:7523]
2025-10-09T21:47:09.750727Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341852884348955:7523]
2025-10-09T21:47:09.750786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:09.750791Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:09.750793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.750796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:09.750797Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:09.750805Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:09.750943Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:09.776463Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341853246167915:7899]
2025-10-09T21:47:09.776955Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 867, pipe: [1:7559341853246167915:7899]
2025-10-09T21:47:09.912101Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:09.920916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:09.920951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:09.921133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.921209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.921272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.921342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.921396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.921400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.921470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.921524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.921584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.921647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.921691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.921694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.921762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.921827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.921887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.921966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.922013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.922017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.922220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.922287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.922355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.922422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.922472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.922475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.922555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.924741Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:09.924758Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:09.924768Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:09.924797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 868 reason: error at write: Disconnected sessionId: src_id|3259cfd2-7bf865e9-674e1ce2-56137bd2_0
2025-10-09T21:47:09.924828Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341853246167915:7899]
2025-10-09T21:47:09.925010Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 868 sessionId: src_id|3259cfd2-7bf865e9-674e1ce2-56137bd2_0 is DEAD
2025-10-09T21:47:09.925220Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3259cfd2-7bf865e9-674e1ce2-56137bd2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:09.925233Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3259cfd2-7bf865e9-674e1ce2-56137bd2_0] Write session will restart in 0.009719s
2025-10-09T21:47:09.925258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3259cfd2-7bf865e9-674e1ce2-56137bd2_0] Write session: Do CDS request
2025-10-09T21:47:09.925265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3259cfd2-7bf865e9-674e1ce2-56137bd2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:09.922621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.922698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.922760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.922807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.922810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.922897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.922954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.923017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.923078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.923123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.923129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.923201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.923255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.923319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.923379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.923422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.923429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.923500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.923556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.923614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.923677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.923744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.923746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.923821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.923879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.923937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.923996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.924040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.924043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.924110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:09.924169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:09.924233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:09.924300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:09.924348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:09.924351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:09.924426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341853246167915:7899], now have 1 active actors on pipe
2025-10-09T21:47:09.925556Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:09.936614Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3259cfd2-7bf865e9-674e1ce2-56137bd2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:09.936882Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:09.936905Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 869
2025-10-09T21:47:09.937183Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 869 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:09.937251Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 869 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:09.937258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=869 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:09.937263Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:09.938061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 869 sessionId:  describe result for acl check
2025-10-09T21:47:09.938104Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:09.938111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:09.938113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:09.938120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341853246168007:8317] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:09.938717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341853246168007:8317] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:09.942581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341853246168007:8317] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:09.943122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:09.946338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:09.946699Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:09.946710Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:09.946726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:09.946797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:09.946846Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341852884349153:7525]
2025-10-09T21:47:09.946800Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341853246168007:8317] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:09.946818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341853246168007:8317] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:09.946831Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341853246168007:8317] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:09.946842Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 869 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:09.947477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:09.947841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:09.947926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:09.948280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:09.948345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:09.948352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:09.948353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:09.948354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:09.948356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:09.948357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:09.948358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.948359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.948360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.948367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:09.948377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:09.948381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:09.948385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:09.948388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:09.948392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:09.948396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.948400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.948403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.948411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:09.948419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:09.948425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:09.948430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:09.948435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:09.948440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:09.948444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:09.948449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:09.948454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:09.948459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:09.948464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:09.948470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:09.948475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.948480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.948484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.948490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.948496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.948501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.948535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:09.957621Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 868
2025-10-09T21:47:09.957682Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 868
2025-10-09T21:47:09.956121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:09.957192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:09.957199Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:09.957203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:09.957208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:09.957217Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 868 [2:7559341852884349153:7525]
2025-10-09T21:47:09.957235Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:09.957256Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:09.957259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:09.957263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:09.957266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:09.957286Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:09.957293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:09.957300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:09.957363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.957365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.957368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.957371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:09.957450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:09.957455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.957467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:09.957479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:09.957495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:09.957517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341853246167917:2432], now have 1 active actors on pipe
2025-10-09T21:47:09.957521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341853246168108:8317], now have 1 active actors on pipe
2025-10-09T21:47:09.957530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341852884349148:7525]
2025-10-09T21:47:09.957532Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341852884349148:7525]
2025-10-09T21:47:09.957534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341852884349148:7525]
2025-10-09T21:47:09.957548Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:09.957558Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:09.957561Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:09.957563Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:09.957564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:09.957579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:09.957582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:09.957624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341852884349148:7525]
2025-10-09T21:47:09.957633Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:09.957661Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:09.957830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:09.957835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:09.958230Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 869 partition: 0 MaxSeqNo: 25 sessionId: src_id|b541fc77-85c464eb-1a55b561-3b526b6a_0
2025-10-09T21:47:09.957862Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b541fc77-85c464eb-1a55b561-3b526b6a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:09.957870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:09.957875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:09.957878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:09.957889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:09.957913Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:09.957928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:09.958103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:09.958107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:09.958126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:09.958667Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3259cfd2-7bf865e9-674e1ce2-56137bd2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b541fc77-85c464eb-1a55b561-3b526b6a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:09.961250Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b541fc77-85c464eb-1a55b561-3b526b6a_0] Write session: try to update token
2025-10-09T21:47:09.961273Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b541fc77-85c464eb-1a55b561-3b526b6a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:09.972842Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:09.972864Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:09.972874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341852884349148:7525]
2025-10-09T21:47:09.972926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:09.973108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:09.973260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:09.973411Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:09.973556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:09.973759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:09.973810Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:09.973917Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:09.975762Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 869 sessionId: src_id|b541fc77-85c464eb-1a55b561-3b526b6a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:09.978333Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:09.980519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:09.980545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:09.980590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:09.980709Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:09.988098Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:09.988115Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:09.988117Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:09.988119Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:09.988130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341852884349148:7525]
2025-10-09T21:47:09.988135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341852884349148:7525]
2025-10-09T21:47:09.988138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341852884349148:7525]
2025-10-09T21:47:09.988185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:09.988192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:09.988195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:09.988332Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:09.988390Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:09.988402Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:09.988406Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:10.025256Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341857541135416:7899]
2025-10-09T21:47:10.025610Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 868, pipe: [1:7559341857541135416:7899]
2025-10-09T21:47:10.151888Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:10.160431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:10.160469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:10.160645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:10.160710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:10.160775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:10.160840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:10.160892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:10.160904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:10.160970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:10.161038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:10.161101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:10.161168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:10.161219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:10.161223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:10.161290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:10.161345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:10.161409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:10.161471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:10.161512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:10.161515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:10.161581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:10.161636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:10.161740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:10.161803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:10.161845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:10.161848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:10.161931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:10.161990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:10.162183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:10.162261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:10.162313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:10.162322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:10.162394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:10.164265Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:10.164282Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:10.164323Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 869 reason: error at write: Disconnected sessionId: src_id|b541fc77-85c464eb-1a55b561-3b526b6a_0
2025-10-09T21:47:10.164549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 869 sessionId: src_id|b541fc77-85c464eb-1a55b561-3b526b6a_0 is DEAD
2025-10-09T21:47:10.164695Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b541fc77-85c464eb-1a55b561-3b526b6a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:10.164708Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b541fc77-85c464eb-1a55b561-3b526b6a_0] Write session will restart in 0.009849s
2025-10-09T21:47:10.164735Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b541fc77-85c464eb-1a55b561-3b526b6a_0] Write session: Do CDS request
2025-10-09T21:47:10.164742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b541fc77-85c464eb-1a55b561-3b526b6a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:10.164774Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341857541135416:7899]
2025-10-09T21:47:10.164801Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:10.162458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:10.162524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:10.162586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:10.162626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:10.162629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:10.162697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:10.162753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:10.162813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:10.162872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:10.162915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:10.162919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:10.162991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:10.163046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:10.163105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:10.163166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:10.163211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:10.163214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:10.163278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:10.163338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:10.163399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:10.163460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:10.163504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:10.163507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:10.163574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:10.163631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:10.163692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:10.163753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:10.163872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:10.163875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:10.163948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341857541135416:7899], now have 1 active actors on pipe
2025-10-09T21:47:10.165106Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:10.175629Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b541fc77-85c464eb-1a55b561-3b526b6a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:10.176134Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:10.176155Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 870
2025-10-09T21:47:10.176594Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 870 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:10.176648Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 870 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:10.176655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=870 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:10.176658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:10.177702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 870 sessionId:  describe result for acl check
2025-10-09T21:47:10.177747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:10.177749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:10.177750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:10.177756Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341857541135440:8322] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:10.178384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341857541135440:8322] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:10.186674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341857541135440:8322] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:10.198921Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341857541135440:8322] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:10.198944Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341857541135440:8322] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:10.198954Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341857541135440:8322] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:10.198966Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 870 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:10.200110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:10.202682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:10.202988Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:10.203004Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:10.203021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:10.203099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:10.203148Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341857179316643:7527]
2025-10-09T21:47:10.203738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:10.204264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:10.204316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:10.204674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:10.204727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:10.204733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:10.204735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:10.204736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:10.204737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:10.204738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:10.204740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.204741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.204742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.204748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:10.204761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:10.204766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:10.204770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:10.204774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:10.204778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:10.204783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.204787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.204791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.204798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:10.204807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:10.204814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:10.204819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:10.204825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:10.204831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:10.204835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:10.204841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:10.204845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:10.204850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:10.204854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:10.204858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:10.204862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.204867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.204871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.204876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.204881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.204886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.204921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:10.213647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:10.214842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:10.214865Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:10.214869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:10.214874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:10.214883Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 869 [2:7559341857179316643:7527]
2025-10-09T21:47:10.214900Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:10.214924Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:10.214927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:10.214932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:10.214935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:10.214957Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:10.214972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:10.214980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:10.215062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.215071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.215074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.215078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:10.215167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:10.215177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.215193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.215206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.215226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:10.215254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341857179316638:7527]
2025-10-09T21:47:10.215256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341857179316638:7527]
2025-10-09T21:47:10.215259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341857179316638:7527]
2025-10-09T21:47:10.215273Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:10.215291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:10.215306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:10.215310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:10.215362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341857179316638:7527]
2025-10-09T21:47:10.215371Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:10.215537Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:10.215542Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:10.215544Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:10.215550Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:10.216093Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 869
2025-10-09T21:47:10.216147Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 869
2025-10-09T21:47:10.216680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341857541135421:2432], now have 1 active actors on pipe
2025-10-09T21:47:10.216692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341857541135616:8322], now have 1 active actors on pipe
2025-10-09T21:47:10.216713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:10.216720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:10.216757Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f6f0fdb9-f40c05b3-366a2f76-62bd53a2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:10.216769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:10.216774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:10.216777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:10.216791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:10.216808Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:10.216827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:10.217156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:10.217164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:10.217180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:10.217329Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 870 partition: 0 MaxSeqNo: 25 sessionId: src_id|f6f0fdb9-f40c05b3-366a2f76-62bd53a2_0
2025-10-09T21:47:10.217812Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b541fc77-85c464eb-1a55b561-3b526b6a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f6f0fdb9-f40c05b3-366a2f76-62bd53a2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:10.220597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f6f0fdb9-f40c05b3-366a2f76-62bd53a2_0] Write session: try to update token
2025-10-09T21:47:10.220622Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f6f0fdb9-f40c05b3-366a2f76-62bd53a2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:10.235903Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:10.235922Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:10.235925Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:10.235927Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:10.235936Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341857179316638:7527]
2025-10-09T21:47:10.235940Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341857179316638:7527]
2025-10-09T21:47:10.235945Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341857179316638:7527]
2025-10-09T21:47:10.235997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:10.236005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.236008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:10.236152Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:10.236828Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:10.236836Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:10.236839Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:10.242526Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 870 sessionId: src_id|f6f0fdb9-f40c05b3-366a2f76-62bd53a2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:10.245139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:10.264750Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341857541135629:7899]
2025-10-09T21:47:10.265689Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 869, pipe: [1:7559341857541135629:7899]
2025-10-09T21:47:10.415232Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:10.415559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:10.415576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:10.415625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341857541135629:7899], now have 1 active actors on pipe
2025-10-09T21:47:10.416588Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:10.416595Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341857541135629:7899]
2025-10-09T21:47:10.416607Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:10.416625Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:10.416678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 870 reason: error at write: Disconnected sessionId: src_id|f6f0fdb9-f40c05b3-366a2f76-62bd53a2_0
2025-10-09T21:47:10.416913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 870 sessionId: src_id|f6f0fdb9-f40c05b3-366a2f76-62bd53a2_0 is DEAD
2025-10-09T21:47:10.417028Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:10.417079Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6f0fdb9-f40c05b3-366a2f76-62bd53a2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:10.417092Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6f0fdb9-f40c05b3-366a2f76-62bd53a2_0] Write session will restart in 0.008657s
2025-10-09T21:47:10.417130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6f0fdb9-f40c05b3-366a2f76-62bd53a2_0] Write session: Do CDS request
2025-10-09T21:47:10.417138Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6f0fdb9-f40c05b3-366a2f76-62bd53a2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:10.430609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f6f0fdb9-f40c05b3-366a2f76-62bd53a2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:10.434438Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:10.434458Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 871
2025-10-09T21:47:10.434647Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 871 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:10.434686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 871 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:10.434693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=871 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:10.434697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:10.438497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:10.438395Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 871 sessionId:  describe result for acl check
2025-10-09T21:47:10.438443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:10.438445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:10.438447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:10.438453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341857541135788:8326] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:10.439014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341857541135788:8326] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:10.441295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:10.441530Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:10.441545Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:10.441557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:10.441622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:10.441661Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341857179316835:7529]
2025-10-09T21:47:10.442264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:10.442884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341857541135788:8326] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:10.442950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:10.443014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:10.443364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:10.443447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:10.443453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:10.443455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:10.443456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:10.443457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:10.443459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:10.443460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.443462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.443463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.443470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:10.443481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:10.443485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:10.443490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:10.443494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:10.443498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:10.443502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.443505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.443509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.443518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:10.443526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:10.443534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:10.443539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:10.443547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:10.443551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:10.443555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:10.443560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:10.443564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:10.443569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:10.443573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:10.443579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:10.443584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.443589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.443593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.443598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.443602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.443607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.443632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:10.452052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341857541135788:8326] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:10.452073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341857541135788:8326] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:10.452078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341857541135788:8326] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:10.452090Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 871 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:10.452381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:10.453632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:10.453650Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:10.453653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:10.453658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:10.453666Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 870 [2:7559341857179316835:7529]
2025-10-09T21:47:10.453682Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:10.453704Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:10.453713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:10.453716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:10.453719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:10.453741Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:10.453751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:10.453761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:10.453831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.453842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.453844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.453848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:10.453942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:10.453951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.453966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.453979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.453995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:10.454019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341857179316830:7529]
2025-10-09T21:47:10.454022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341857179316830:7529]
2025-10-09T21:47:10.454024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341857179316830:7529]
2025-10-09T21:47:10.454319Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 870
2025-10-09T21:47:10.454352Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 870
2025-10-09T21:47:10.457076Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:10.457114Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:10.457118Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:10.457123Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:10.457149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:10.457765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:10.457780Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:10.457818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341857541135827:8326], now have 1 active actors on pipe
2025-10-09T21:47:10.457830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341857541135632:2432], now have 1 active actors on pipe
2025-10-09T21:47:10.457873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:10.457888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:10.457995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341857179316830:7529]
2025-10-09T21:47:10.458009Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:10.458048Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f3f7dd56-319367bf-87fea3f4-5b2c319f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:10.458062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:10.458072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:10.458077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:10.458092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:10.458112Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:10.458170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:10.458248Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:10.458602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:10.458631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:10.458843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 871 partition: 0 MaxSeqNo: 25 sessionId: src_id|f3f7dd56-319367bf-87fea3f4-5b2c319f_0
2025-10-09T21:47:10.458656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:10.462469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6f0fdb9-f40c05b3-366a2f76-62bd53a2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f3f7dd56-319367bf-87fea3f4-5b2c319f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:10.465278Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3f7dd56-319367bf-87fea3f4-5b2c319f_0] Write session: try to update token
2025-10-09T21:47:10.465298Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3f7dd56-319367bf-87fea3f4-5b2c319f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:10.474312Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:10.474328Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:10.474341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341857179316830:7529]
2025-10-09T21:47:10.474393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:10.474454Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:10.474574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:10.474729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:10.474882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:10.475017Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:10.475582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:10.475626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:10.500674Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:10.500696Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:10.500699Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:10.500701Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:10.500713Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341857179316830:7529]
2025-10-09T21:47:10.500719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341857179316830:7529]
2025-10-09T21:47:10.500723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341857179316830:7529]
2025-10-09T21:47:10.500775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:10.500792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.500795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:10.500949Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:10.502283Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:10.502295Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:10.502300Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:10.506083Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 871 sessionId: src_id|f3f7dd56-319367bf-87fea3f4-5b2c319f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:10.508701Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:10.516735Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341857541135835:7899]
2025-10-09T21:47:10.517169Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 870, pipe: [1:7559341857541135835:7899]
2025-10-09T21:47:10.661871Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:10.662223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:10.662238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:10.662262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341857541135835:7899], now have 1 active actors on pipe
2025-10-09T21:47:10.662552Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:10.662600Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341857541135835:7899]
2025-10-09T21:47:10.662620Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:10.662695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 871 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f3f7dd56-319367bf-87fea3f4-5b2c319f_0
2025-10-09T21:47:10.662972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 871 sessionId: src_id|f3f7dd56-319367bf-87fea3f4-5b2c319f_0 is DEAD
2025-10-09T21:47:10.662485Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:10.663075Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3f7dd56-319367bf-87fea3f4-5b2c319f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:10.663094Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3f7dd56-319367bf-87fea3f4-5b2c319f_0] Write session will restart in 0.008125s
2025-10-09T21:47:10.663119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3f7dd56-319367bf-87fea3f4-5b2c319f_0] Write session: Do CDS request
2025-10-09T21:47:10.663125Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3f7dd56-319367bf-87fea3f4-5b2c319f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:10.672612Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3f7dd56-319367bf-87fea3f4-5b2c319f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:10.672832Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:10.672861Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 872
2025-10-09T21:47:10.673067Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 872 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:10.673109Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 872 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:10.673119Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=872 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:10.673123Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:10.673633Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 872 sessionId:  describe result for acl check
2025-10-09T21:47:10.673670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:10.673676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:10.673678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:10.673684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341857541135877:8337] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:10.674272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341857541135877:8337] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:10.680889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341857541135877:8337] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:10.685205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341857541135877:8337] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:10.685227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341857541135877:8337] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:10.685233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341857541135877:8337] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:10.685244Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 872 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:10.690055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:10.693741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:10.694009Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:10.694024Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:10.694057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:10.694175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:10.694243Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341857179317074:7545]
2025-10-09T21:47:10.694821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:10.695177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:10.695241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:10.695595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:10.695665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:10.695666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:10.695667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:10.695669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:10.695670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:10.695679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:10.695681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.695682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.695683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.695688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:10.695699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:10.695703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:10.695707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:10.695710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:10.695714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:10.695718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.695722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.695725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.695732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:10.695739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:10.695745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:10.695750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:10.695755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:10.695761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:10.695765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:10.695770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:10.695775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:10.695787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:10.695791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:10.695795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:10.695800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.695807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.695811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.695817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.695821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.695826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.695859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:10.703233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:10.704638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:10.704657Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:10.704660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:10.704666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:10.704673Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 871 [2:7559341857179317074:7545]
2025-10-09T21:47:10.704688Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:10.704703Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:10.704705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:10.704708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:10.704710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:10.704725Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:10.704731Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:10.704739Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:10.704811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.704823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.704824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.704827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:10.704894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:10.704900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.704910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.704916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.704926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:10.704943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341857179317069:7545]
2025-10-09T21:47:10.704951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341857179317069:7545]
2025-10-09T21:47:10.704953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341857179317069:7545]
2025-10-09T21:47:10.704959Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:10.704973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:10.704975Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:10.704978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:10.704981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:10.704982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:10.704985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:10.704999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341857541135992:8337], now have 1 active actors on pipe
2025-10-09T21:47:10.705002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341857541135853:2432], now have 1 active actors on pipe
2025-10-09T21:47:10.705040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341857179317069:7545]
2025-10-09T21:47:10.705055Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:10.705073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:10.705080Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 871
2025-10-09T21:47:10.705087Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 871
2025-10-09T21:47:10.705256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:10.705272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:10.705321Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3dd379d8-5e08b03f-999cc933-3452407d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:10.705331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:10.705336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:10.705338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:10.705361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:10.705374Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:10.705393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:10.705627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:10.705635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:10.705653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:10.705767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 872 partition: 0 MaxSeqNo: 25 sessionId: src_id|3dd379d8-5e08b03f-999cc933-3452407d_0
2025-10-09T21:47:10.706217Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3f7dd56-319367bf-87fea3f4-5b2c319f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3dd379d8-5e08b03f-999cc933-3452407d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:10.708919Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3dd379d8-5e08b03f-999cc933-3452407d_0] Write session: try to update token
2025-10-09T21:47:10.708941Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3dd379d8-5e08b03f-999cc933-3452407d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:10.721572Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:10.721597Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:10.721600Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:10.721603Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:10.721613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341857179317069:7545]
2025-10-09T21:47:10.721618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341857179317069:7545]
2025-10-09T21:47:10.721623Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341857179317069:7545]
2025-10-09T21:47:10.721677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:10.721685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.721688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:10.721690Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:10.721696Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:10.721700Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:10.721824Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:10.730369Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 872 sessionId: src_id|3dd379d8-5e08b03f-999cc933-3452407d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:10.732905Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:10.763012Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341857541136066:7899]
2025-10-09T21:47:10.763293Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 871, pipe: [1:7559341857541136066:7899]
2025-10-09T21:47:10.896549Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:10.896866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:10.896883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:10.896932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341857541136066:7899], now have 1 active actors on pipe
2025-10-09T21:47:10.897232Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341857541136066:7899]
2025-10-09T21:47:10.897238Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:10.897242Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:10.897276Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:10.897314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 872 reason: error at write: Disconnected sessionId: src_id|3dd379d8-5e08b03f-999cc933-3452407d_0
2025-10-09T21:47:10.897389Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:10.897507Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 872 sessionId: src_id|3dd379d8-5e08b03f-999cc933-3452407d_0 is DEAD
2025-10-09T21:47:10.897686Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3dd379d8-5e08b03f-999cc933-3452407d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:10.897698Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3dd379d8-5e08b03f-999cc933-3452407d_0] Write session will restart in 0.009145s
2025-10-09T21:47:10.897724Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3dd379d8-5e08b03f-999cc933-3452407d_0] Write session: Do CDS request
2025-10-09T21:47:10.897730Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3dd379d8-5e08b03f-999cc933-3452407d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:10.908615Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3dd379d8-5e08b03f-999cc933-3452407d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:10.908733Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:10.908750Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 873
2025-10-09T21:47:10.908863Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 873 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:10.908896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 873 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:10.908920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=873 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:10.908923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:10.909359Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 873 sessionId:  describe result for acl check
2025-10-09T21:47:10.909400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:10.909407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:10.909409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:10.909414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341857541136086:8342] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:10.910001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341857541136086:8342] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:10.913591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341857541136086:8342] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:10.918685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341857541136086:8342] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:10.918701Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341857541136086:8342] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:10.918705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341857541136086:8342] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:10.918717Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 873 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:10.920698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:10.923296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:10.923513Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:10.923524Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:10.923537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:10.923594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:10.923650Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341857179317271:7547]
2025-10-09T21:47:10.924232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:10.924642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:10.924702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:10.925024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:10.925096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:10.925103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:10.925104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:10.925106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:10.925107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:10.925108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:10.925109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.925110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.925112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.925118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:10.925129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:10.925133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:10.925137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:10.925142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:10.925146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:10.925150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.925154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.925158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.925165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:10.925173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:10.925179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:10.925185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:10.925189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:10.925195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:10.925199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:10.925204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:10.925209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:10.925214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:10.925219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:10.925225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:10.925229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.925235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.925239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.925244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.925248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.925254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.925284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:10.932890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:10.934734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:10.934753Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:10.934756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:10.934762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:10.934770Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 872 [2:7559341857179317271:7547]
2025-10-09T21:47:10.934790Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:10.934810Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:10.934813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:10.934817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:10.934819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:10.934839Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:10.934855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:10.934862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:10.934926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.934934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.934936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.934939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:10.935014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:10.935022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.935035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:10.935051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:10.935066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:10.935088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341857179317264:7547]
2025-10-09T21:47:10.935096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341857179317264:7547]
2025-10-09T21:47:10.935100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341857179317264:7547]
2025-10-09T21:47:10.935108Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:10.935120Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:10.935123Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:10.935124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:10.935125Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:10.935139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:10.935148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:10.935161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341857541136069:2432], now have 1 active actors on pipe
2025-10-09T21:47:10.935165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341857541136263:8342], now have 1 active actors on pipe
2025-10-09T21:47:10.935212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341857179317264:7547]
2025-10-09T21:47:10.935225Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:10.935246Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:10.935303Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 872
2025-10-09T21:47:10.935351Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 872
2025-10-09T21:47:10.935699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:10.935709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:10.935740Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|38aa8ce-92c7ddc2-1b0968a7-55bc9a63_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:10.935750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:10.935754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:10.935757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:10.935769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:10.935783Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:10.935801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:10.936129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:10.936142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:10.936163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:10.936306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 873 partition: 0 MaxSeqNo: 25 sessionId: src_id|38aa8ce-92c7ddc2-1b0968a7-55bc9a63_0
2025-10-09T21:47:10.936736Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3dd379d8-5e08b03f-999cc933-3452407d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|38aa8ce-92c7ddc2-1b0968a7-55bc9a63_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:10.939281Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38aa8ce-92c7ddc2-1b0968a7-55bc9a63_0] Write session: try to update token
2025-10-09T21:47:10.939303Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38aa8ce-92c7ddc2-1b0968a7-55bc9a63_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:10.949562Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:10.949579Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:10.949582Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:10.949584Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:10.949597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341857179317264:7547]
2025-10-09T21:47:10.949601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341857179317264:7547]
2025-10-09T21:47:10.949605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341857179317264:7547]
2025-10-09T21:47:10.949654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:10.949662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:10.949665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:10.949709Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:10.949724Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:10.949729Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:10.949932Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:10.961939Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 873 sessionId: src_id|38aa8ce-92c7ddc2-1b0968a7-55bc9a63_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:10.964504Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:10.997987Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341857541136278:7899]
2025-10-09T21:47:10.998560Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 872, pipe: [1:7559341857541136278:7899]
2025-10-09T21:47:11.112066Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:11.112412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:11.112425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:11.112450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341857541136278:7899], now have 1 active actors on pipe
2025-10-09T21:47:11.114374Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:11.118168Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:11.118231Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341857541136278:7899]
2025-10-09T21:47:11.118258Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:11.118350Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 873 reason: status is not ok: tablet will be restarted right now sessionId: src_id|38aa8ce-92c7ddc2-1b0968a7-55bc9a63_0
2025-10-09T21:47:11.118548Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 873 sessionId: src_id|38aa8ce-92c7ddc2-1b0968a7-55bc9a63_0 is DEAD
2025-10-09T21:47:11.122376Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38aa8ce-92c7ddc2-1b0968a7-55bc9a63_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:11.122392Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38aa8ce-92c7ddc2-1b0968a7-55bc9a63_0] Write session will restart in 0.006306s
2025-10-09T21:47:11.122423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38aa8ce-92c7ddc2-1b0968a7-55bc9a63_0] Write session: Do CDS request
2025-10-09T21:47:11.122430Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38aa8ce-92c7ddc2-1b0968a7-55bc9a63_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:11.129627Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38aa8ce-92c7ddc2-1b0968a7-55bc9a63_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:11.129856Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:11.129873Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 874
2025-10-09T21:47:11.130051Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 874 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:11.130081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 874 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:11.130088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=874 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:11.130092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:11.130898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 874 sessionId:  describe result for acl check
2025-10-09T21:47:11.130940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:11.130948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:11.130950Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:11.130956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341861836103631:8346] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:11.131598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341861836103631:8346] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:11.140635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341861836103631:8346] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:11.145589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341861836103631:8346] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:11.145609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341861836103631:8346] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:11.145614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341861836103631:8346] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:11.145625Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 874 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:11.146066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:11.148711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:11.148998Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:11.149009Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:11.149023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:11.149087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:11.149144Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341861474284764:7549]
2025-10-09T21:47:11.149732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:11.150567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:11.150644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:11.151013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:11.151064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:11.151066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:11.151067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:11.151068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:11.151070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:11.151072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:11.151073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.151074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.151075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.151082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:11.151094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:11.151098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:11.151103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:11.151107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:11.151112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:11.151116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.151119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.151124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.151132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:11.151142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:11.151149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:11.151155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:11.151160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:11.151166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:11.151172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:11.151179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:11.151184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:11.151190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:11.151195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:11.151201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:11.151206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.151212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.151218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.151224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.151231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.151237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.151271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:11.159126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:11.160322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:11.160339Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:11.160342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:11.160349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:11.160356Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 873 [2:7559341861474284764:7549]
2025-10-09T21:47:11.160388Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:11.160411Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:11.160422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:11.160426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:11.160428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:11.160451Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:11.160458Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:11.160467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:11.160536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.160545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.160548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.160551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:11.160639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:11.160648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.160664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.160677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.160696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:11.160724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341861474284759:7549]
2025-10-09T21:47:11.160732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341861474284759:7549]
2025-10-09T21:47:11.160735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341861474284759:7549]
2025-10-09T21:47:11.160744Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:11.160762Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:11.160768Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:11.160772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:11.160774Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:11.160778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:11.160781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:11.160795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341861836103582:2432], now have 1 active actors on pipe
2025-10-09T21:47:11.160807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341861836103777:8346], now have 1 active actors on pipe
2025-10-09T21:47:11.160848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341861474284759:7549]
2025-10-09T21:47:11.160865Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:11.160888Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 873
2025-10-09T21:47:11.160893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 873
2025-10-09T21:47:11.161006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:11.161023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:11.161062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:11.161070Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b3c24f9-ae2cda94-c59dbfd0-936eb15e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:11.161082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:11.161098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:11.161101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:11.161115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:11.161130Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:11.161150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:11.161351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:11.161362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:11.161393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:11.161564Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 874 partition: 0 MaxSeqNo: 25 sessionId: src_id|b3c24f9-ae2cda94-c59dbfd0-936eb15e_0
2025-10-09T21:47:11.162157Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38aa8ce-92c7ddc2-1b0968a7-55bc9a63_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b3c24f9-ae2cda94-c59dbfd0-936eb15e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:11.164893Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3c24f9-ae2cda94-c59dbfd0-936eb15e_0] Write session: try to update token
2025-10-09T21:47:11.164914Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3c24f9-ae2cda94-c59dbfd0-936eb15e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:11.177774Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:11.177801Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:11.177804Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:11.177807Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:11.177817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341861474284759:7549]
2025-10-09T21:47:11.177822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341861474284759:7549]
2025-10-09T21:47:11.177827Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341861474284759:7549]
2025-10-09T21:47:11.177888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:11.177910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.177913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:11.178091Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:11.182207Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:11.182229Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:11.182234Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:11.185387Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 874 sessionId: src_id|b3c24f9-ae2cda94-c59dbfd0-936eb15e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:11.188098Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:11.218180Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341861836103795:7899]
2025-10-09T21:47:11.222371Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 873, pipe: [1:7559341861836103795:7899]
2025-10-09T21:47:11.348307Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:11.348987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:11.348999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:11.349024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341861836103795:7899], now have 1 active actors on pipe
2025-10-09T21:47:11.349259Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:11.349265Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341861836103795:7899]
2025-10-09T21:47:11.349299Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:11.349302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:11.349366Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 874 reason: error at write: Disconnected sessionId: src_id|b3c24f9-ae2cda94-c59dbfd0-936eb15e_0
2025-10-09T21:47:11.349540Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 874 sessionId: src_id|b3c24f9-ae2cda94-c59dbfd0-936eb15e_0 is DEAD
2025-10-09T21:47:11.349271Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:11.349689Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3c24f9-ae2cda94-c59dbfd0-936eb15e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:11.349700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3c24f9-ae2cda94-c59dbfd0-936eb15e_0] Write session will restart in 0.008090s
2025-10-09T21:47:11.349720Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3c24f9-ae2cda94-c59dbfd0-936eb15e_0] Write session: Do CDS request
2025-10-09T21:47:11.349733Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3c24f9-ae2cda94-c59dbfd0-936eb15e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:11.358549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3c24f9-ae2cda94-c59dbfd0-936eb15e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:11.358669Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:11.358688Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 875
2025-10-09T21:47:11.358817Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 875 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:11.358846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 875 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:11.358856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=875 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:11.358860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:11.359374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 875 sessionId:  describe result for acl check
2025-10-09T21:47:11.359406Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:11.359416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:11.359418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:11.359424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341861836103805:8351] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:11.360000Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341861836103805:8351] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:11.363196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341861836103805:8351] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:11.365967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341861836103805:8351] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:11.365984Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341861836103805:8351] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:11.365988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341861836103805:8351] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:11.365998Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 875 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:11.384625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:11.387205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:11.387437Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:11.387449Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:11.387464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:11.387532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:11.387580Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341861474284962:7551]
2025-10-09T21:47:11.388175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:11.388558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:11.388615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:11.388960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:11.389030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:11.389037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:11.389038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:11.389039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:11.389040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:11.389041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:11.389043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.389044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.389045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.389050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:11.389062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:11.389066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:11.389070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:11.389074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:11.389079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:11.389083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.389087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.389090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.389097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:11.389106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:11.389113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:11.389119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:11.389124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:11.389128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:11.389133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:11.389138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:11.389143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:11.389147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:11.389151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:11.389156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:11.389161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.389166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.389171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.389182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.389188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.389197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.389229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:11.396513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:11.397660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:11.397676Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:11.397679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:11.397684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:11.397693Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 874 [2:7559341861474284962:7551]
2025-10-09T21:47:11.397711Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:11.397733Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:11.397736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:11.397739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:11.397742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:11.397761Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:11.397775Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:11.397786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:11.397848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.397856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.397858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.397861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:11.397952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:11.397961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.397975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.397987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.398009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:11.398076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341861836103840:8351], now have 1 active actors on pipe
2025-10-09T21:47:11.398085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341861836103798:2432], now have 1 active actors on pipe
2025-10-09T21:47:11.398094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341861474284957:7551]
2025-10-09T21:47:11.398096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341861474284957:7551]
2025-10-09T21:47:11.398099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341861474284957:7551]
2025-10-09T21:47:11.398114Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:11.398139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:11.398156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:11.398159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:11.398173Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:11.398184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:11.398187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:11.398201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341861474284957:7551]
2025-10-09T21:47:11.398213Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:11.398215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:11.398286Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 874
2025-10-09T21:47:11.398285Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 874
2025-10-09T21:47:11.398474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:11.398486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:11.398517Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5fb80e94-36240f78-8a6ce25f-d9c7b8e2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:11.398526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:11.398530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:11.398533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:11.398555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:11.398568Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:11.398586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:11.398877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:11.398904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:11.398932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:11.399058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 875 partition: 0 MaxSeqNo: 25 sessionId: src_id|5fb80e94-36240f78-8a6ce25f-d9c7b8e2_0
2025-10-09T21:47:11.399650Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3c24f9-ae2cda94-c59dbfd0-936eb15e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5fb80e94-36240f78-8a6ce25f-d9c7b8e2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:11.402320Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5fb80e94-36240f78-8a6ce25f-d9c7b8e2_0] Write session: try to update token
2025-10-09T21:47:11.402341Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5fb80e94-36240f78-8a6ce25f-d9c7b8e2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:11.414009Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:11.414048Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:11.414050Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:11.414052Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:11.414063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341861474284957:7551]
2025-10-09T21:47:11.414067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341861474284957:7551]
2025-10-09T21:47:11.414072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341861474284957:7551]
2025-10-09T21:47:11.414119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:11.414132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.414135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:11.414137Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:11.414144Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:11.414148Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:11.414302Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:11.417654Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 875 sessionId: src_id|5fb80e94-36240f78-8a6ce25f-d9c7b8e2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:11.420123Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:11.449558Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341861836104006:7899]
2025-10-09T21:47:11.449879Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 874, pipe: [1:7559341861836104006:7899]
2025-10-09T21:47:11.579713Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:11.580016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:11.580035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:11.580081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341861836104006:7899], now have 1 active actors on pipe
2025-10-09T21:47:11.580441Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:11.581082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fb80e94-36240f78-8a6ce25f-d9c7b8e2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:11.581094Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fb80e94-36240f78-8a6ce25f-d9c7b8e2_0] Write session will restart in 0.006373s
2025-10-09T21:47:11.581117Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fb80e94-36240f78-8a6ce25f-d9c7b8e2_0] Write session: Do CDS request
2025-10-09T21:47:11.581123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fb80e94-36240f78-8a6ce25f-d9c7b8e2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:11.580678Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:11.580773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 875 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5fb80e94-36240f78-8a6ce25f-d9c7b8e2_0
2025-10-09T21:47:11.580894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 875 sessionId: src_id|5fb80e94-36240f78-8a6ce25f-d9c7b8e2_0 is DEAD
2025-10-09T21:47:11.581055Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:11.581083Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341861836104006:7899]
2025-10-09T21:47:11.588601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5fb80e94-36240f78-8a6ce25f-d9c7b8e2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:11.588678Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:11.588696Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 876
2025-10-09T21:47:11.588825Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 876 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:11.588859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 876 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:11.588865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=876 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:11.588869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:11.589472Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 876 sessionId:  describe result for acl check
2025-10-09T21:47:11.589511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:11.589513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:11.589516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:11.589523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341861836104021:8355] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:11.590132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341861836104021:8355] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:11.593734Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341861836104021:8355] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:11.602341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341861836104021:8355] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:11.602359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341861836104021:8355] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:11.602364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341861836104021:8355] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:11.602373Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 876 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:11.618980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:11.622001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:11.622279Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:11.622284Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:11.622296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:11.622351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:11.622389Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341861474285196:7559]
2025-10-09T21:47:11.623048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:11.623460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:11.623513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:11.623854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:11.623917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:11.623918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:11.623919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:11.623921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:11.623922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:11.623923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:11.623924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.623925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.623926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.623933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:11.623943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:11.623948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:11.623952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:11.623957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:11.623961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:11.623965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.623968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.623972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.623979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:11.623988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:11.623995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:11.624001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:11.624006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:11.624012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:11.624016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:11.624021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:11.624026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:11.624031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:11.624036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:11.624041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:11.624047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.624053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.624057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.624063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.624067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.624072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.624103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:11.631510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:11.633046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:11.633065Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:11.633068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:11.633073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:11.633080Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 875 [2:7559341861474285196:7559]
2025-10-09T21:47:11.633096Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:11.633116Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:11.633124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:11.633127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:11.633130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:11.633148Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:11.633155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:11.633167Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:11.633234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.633246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.633248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.633251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:11.633327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:11.633336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.633350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.633362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.633380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:11.633405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341861474285191:7559]
2025-10-09T21:47:11.633413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341861474285191:7559]
2025-10-09T21:47:11.633415Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341861474285191:7559]
2025-10-09T21:47:11.633424Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:11.633439Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:11.633456Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:11.633457Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:11.633459Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:11.633462Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:11.633465Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:11.633472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341861836104103:8355], now have 1 active actors on pipe
2025-10-09T21:47:11.633476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341861836104008:2432], now have 1 active actors on pipe
2025-10-09T21:47:11.633517Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341861474285191:7559]
2025-10-09T21:47:11.633533Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:11.633568Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 875
2025-10-09T21:47:11.633579Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 875
2025-10-09T21:47:11.633627Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:11.633928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:11.633955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:11.633986Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2236cece-1670f0cf-c6bd9ab9-ef7c8a6e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:11.634001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:11.634008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:11.634010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:11.634023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:11.634055Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:11.634080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:11.634694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 876 partition: 0 MaxSeqNo: 25 sessionId: src_id|2236cece-1670f0cf-c6bd9ab9-ef7c8a6e_0
2025-10-09T21:47:11.634474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:11.634489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:11.634515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:11.635236Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fb80e94-36240f78-8a6ce25f-d9c7b8e2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2236cece-1670f0cf-c6bd9ab9-ef7c8a6e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:11.637812Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2236cece-1670f0cf-c6bd9ab9-ef7c8a6e_0] Write session: try to update token
2025-10-09T21:47:11.637832Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2236cece-1670f0cf-c6bd9ab9-ef7c8a6e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:11.655348Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 876 sessionId: src_id|2236cece-1670f0cf-c6bd9ab9-ef7c8a6e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:11.655376Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:11.655393Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:11.655396Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:11.655398Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:11.655407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341861474285191:7559]
2025-10-09T21:47:11.655411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341861474285191:7559]
2025-10-09T21:47:11.655415Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341861474285191:7559]
2025-10-09T21:47:11.655463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:11.655474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.655477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:11.655556Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:11.655571Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:11.655575Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:11.655619Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:11.657832Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:11.679854Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341861836104230:7899]
2025-10-09T21:47:11.680150Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 875, pipe: [1:7559341861836104230:7899]
2025-10-09T21:47:11.816977Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:11.817273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:11.817288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:11.817334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341861836104230:7899], now have 1 active actors on pipe
2025-10-09T21:47:11.817553Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:11.818112Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2236cece-1670f0cf-c6bd9ab9-ef7c8a6e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:11.818133Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2236cece-1670f0cf-c6bd9ab9-ef7c8a6e_0] Write session will restart in 0.006568s
2025-10-09T21:47:11.818158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2236cece-1670f0cf-c6bd9ab9-ef7c8a6e_0] Write session: Do CDS request
2025-10-09T21:47:11.818164Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2236cece-1670f0cf-c6bd9ab9-ef7c8a6e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:11.817599Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:11.817654Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:11.817766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 876 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2236cece-1670f0cf-c6bd9ab9-ef7c8a6e_0
2025-10-09T21:47:11.817851Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341861836104230:7899]
2025-10-09T21:47:11.817954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 876 sessionId: src_id|2236cece-1670f0cf-c6bd9ab9-ef7c8a6e_0 is DEAD
2025-10-09T21:47:11.825580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2236cece-1670f0cf-c6bd9ab9-ef7c8a6e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:11.825677Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:11.825694Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 877
2025-10-09T21:47:11.825807Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 877 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:11.825835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 877 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:11.825841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=877 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:11.825845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:11.826364Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 877 sessionId:  describe result for acl check
2025-10-09T21:47:11.826407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:11.826409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:11.826411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:11.826418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341861836104265:8369] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:11.827020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341861836104265:8369] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:11.830115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341861836104265:8369] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:11.834511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341861836104265:8369] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:11.834528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341861836104265:8369] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:11.834534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341861836104265:8369] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:11.834548Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 877 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:11.838649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:11.841580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:11.841839Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:11.841852Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:11.841867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:11.841950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:11.842017Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341861474285404:7569]
2025-10-09T21:47:11.842641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:11.842979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:11.843027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:11.843378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:11.843434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:11.843441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:11.843442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:11.843444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:11.843445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:11.843446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:11.843447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.843448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.843450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.843455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:11.843466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:11.843470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:11.843475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:11.843479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:11.843483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:11.843487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.843491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.843495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.843503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:11.843511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:11.843517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:11.843523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:11.843528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:11.843533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:11.843542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:11.843547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:11.843552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:11.843557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:11.843561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:11.843567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:11.843573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.843578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.843582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.843587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.843593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.843598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.843629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:11.853494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:11.854788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:11.854810Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:11.854814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:11.854819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:11.854826Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 876 [2:7559341861474285404:7569]
2025-10-09T21:47:11.854842Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:11.854864Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:11.854868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:11.854872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:11.854874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:11.854894Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:11.854904Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:11.854913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:11.854983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.854993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.854996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.855000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:11.855081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:11.855089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.855103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:11.855121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:11.855147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:11.855171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341861836104398:8369], now have 1 active actors on pipe
2025-10-09T21:47:11.855175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341861836104249:2432], now have 1 active actors on pipe
2025-10-09T21:47:11.855184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341861474285398:7569]
2025-10-09T21:47:11.855187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341861474285398:7569]
2025-10-09T21:47:11.855189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341861474285398:7569]
2025-10-09T21:47:11.855203Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:11.855220Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:11.855223Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:11.855226Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:11.855227Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:11.855243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:11.855250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:11.855293Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341861474285398:7569]
2025-10-09T21:47:11.855310Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:11.855281Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 876
2025-10-09T21:47:11.855284Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 876
2025-10-09T21:47:11.855335Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:11.855528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:11.855540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:11.855572Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4a563bdf-5ae44377-6037f652-cb94f0d8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:11.855585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:11.855590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:11.855593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:11.855605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:11.855616Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:11.855640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:11.855876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:11.855889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:11.855910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:11.856002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 877 partition: 0 MaxSeqNo: 25 sessionId: src_id|4a563bdf-5ae44377-6037f652-cb94f0d8_0
2025-10-09T21:47:11.856389Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2236cece-1670f0cf-c6bd9ab9-ef7c8a6e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4a563bdf-5ae44377-6037f652-cb94f0d8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:11.859272Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a563bdf-5ae44377-6037f652-cb94f0d8_0] Write session: try to update token
2025-10-09T21:47:11.859291Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a563bdf-5ae44377-6037f652-cb94f0d8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:11.874067Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 877 sessionId: src_id|4a563bdf-5ae44377-6037f652-cb94f0d8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:11.874761Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:11.874790Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:11.874802Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341861474285398:7569]
2025-10-09T21:47:11.874863Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:11.874913Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:11.875094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:11.875273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:11.875510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:11.875692Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:11.875936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:11.875999Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:11.876559Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:11.876782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:11.876810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:11.876851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:11.876962Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:11.884680Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:11.884709Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:11.884712Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:11.884715Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:11.884728Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341861474285398:7569]
2025-10-09T21:47:11.884733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341861474285398:7569]
2025-10-09T21:47:11.884739Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341861474285398:7569]
2025-10-09T21:47:11.884799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:11.884807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:11.884808Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:11.884815Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:11.884818Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:11.884823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:11.884981Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:11.918171Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341861836104456:7899]
2025-10-09T21:47:11.918718Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 876, pipe: [1:7559341861836104456:7899]
2025-10-09T21:47:12.054701Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:12.063416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:12.063454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:12.063648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:12.063712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:12.063777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:12.063842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:12.063889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:12.063895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:12.063974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:12.064039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:12.064109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:12.064174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:12.064302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:12.064306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:12.064376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:12.064436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:12.064496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:12.064558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:12.064599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:12.064603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:12.064679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:12.064751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:12.064853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:12.064914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:12.065050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:12.065054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:12.065150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:12.065212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:12.065298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:12.065364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:12.065427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:12.065431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:12.065521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:12.065575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:12.065653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:12.065715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:12.065770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:12.065773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:12.065860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:12.065966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:12.066288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:12.066388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:12.066447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:12.066450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:12.066546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:12.066600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:12.066658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:12.066759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:12.066810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:12.066813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:12.066880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:12.066962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:12.067021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:12.067086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:12.067151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:12.067154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:12.067249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:12.067310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:12.067375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:12.067436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:12.067500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:12.067504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:12.067580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341861836104456:7899], now have 1 active actors on pipe
2025-10-09T21:47:12.070448Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:12.070925Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a563bdf-5ae44377-6037f652-cb94f0d8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:12.070939Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a563bdf-5ae44377-6037f652-cb94f0d8_0] Write session will restart in 0.008890s
2025-10-09T21:47:12.070974Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a563bdf-5ae44377-6037f652-cb94f0d8_0] Write session: Do CDS request
2025-10-09T21:47:12.070980Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a563bdf-5ae44377-6037f652-cb94f0d8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:12.070490Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341861836104456:7899]
2025-10-09T21:47:12.070533Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:12.070536Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:12.070576Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 877 reason: error at write: Disconnected sessionId: src_id|4a563bdf-5ae44377-6037f652-cb94f0d8_0
2025-10-09T21:47:12.070765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 877 sessionId: src_id|4a563bdf-5ae44377-6037f652-cb94f0d8_0 is DEAD
2025-10-09T21:47:12.070915Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:12.081572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a563bdf-5ae44377-6037f652-cb94f0d8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:12.081688Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:12.081708Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 878
2025-10-09T21:47:12.081821Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 878 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:12.081852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 878 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:12.081859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=878 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:12.081863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:12.082852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 878 sessionId:  describe result for acl check
2025-10-09T21:47:12.082902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:12.082903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:12.082905Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:12.082912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341866131071770:8373] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:12.083491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341866131071770:8373] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:12.089387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341866131071770:8373] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:12.098166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341866131071770:8373] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:12.098187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341866131071770:8373] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:12.098191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341866131071770:8373] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:12.098217Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 878 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:12.108944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:12.111456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:12.111728Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:12.111741Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:12.111754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:12.111816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:12.111863Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341865769252902:7571]
2025-10-09T21:47:12.112497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:12.112926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:12.112990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:12.113483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:12.113572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:12.113573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:12.113575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:12.113576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:12.113577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:12.113578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:12.113580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.113581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.113582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.113589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:12.113600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:12.113603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:12.113608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:12.113612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:12.113617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:12.113621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.113624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.113628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.113636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:12.113644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:12.113651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:12.113657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:12.113661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:12.113667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:12.113671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:12.113675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:12.113680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:12.113685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:12.113689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:12.113694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:12.113699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.113705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.113709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.113715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.113719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.113724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.113750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:12.122359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:12.123538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:12.123544Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:12.123546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:12.123550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:12.123558Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 877 [2:7559341865769252902:7571]
2025-10-09T21:47:12.123571Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:12.123589Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:12.123592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:12.123594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:12.123596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:12.123614Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:12.123621Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:12.123628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:12.123698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.123699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.123701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.123704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:12.123770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:12.123773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.123789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.123800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.123817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:12.123840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341865769252897:7571]
2025-10-09T21:47:12.123843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341865769252897:7571]
2025-10-09T21:47:12.123845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341865769252897:7571]
2025-10-09T21:47:12.123854Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:12.124012Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 877
2025-10-09T21:47:12.124129Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 877
2025-10-09T21:47:12.123868Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:12.123878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:12.123878Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:12.123880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:12.123882Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:12.123885Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:12.123921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341866131071757:2432], now have 1 active actors on pipe
2025-10-09T21:47:12.123924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341865769252897:7571]
2025-10-09T21:47:12.123931Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:12.123937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341866131071855:8373], now have 1 active actors on pipe
2025-10-09T21:47:12.124033Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:12.124234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:12.124242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:12.124272Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|98b3dac6-780b747c-c5c4a325-4814e735_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:12.124297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:12.124301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:12.124303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:12.124312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:12.124323Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:12.124337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:12.124522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:12.124528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:12.124541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:12.124738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 878 partition: 0 MaxSeqNo: 25 sessionId: src_id|98b3dac6-780b747c-c5c4a325-4814e735_0
2025-10-09T21:47:12.127105Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a563bdf-5ae44377-6037f652-cb94f0d8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|98b3dac6-780b747c-c5c4a325-4814e735_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:12.129790Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98b3dac6-780b747c-c5c4a325-4814e735_0] Write session: try to update token
2025-10-09T21:47:12.129813Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98b3dac6-780b747c-c5c4a325-4814e735_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:12.144300Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:12.144324Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:12.144327Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:12.144329Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:12.144340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341865769252897:7571]
2025-10-09T21:47:12.144345Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341865769252897:7571]
2025-10-09T21:47:12.144349Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341865769252897:7571]
2025-10-09T21:47:12.144399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:12.144408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.144416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:12.144556Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:12.147580Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:12.147597Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:12.147601Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:12.148028Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 878 sessionId: src_id|98b3dac6-780b747c-c5c4a325-4814e735_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:12.150753Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:12.170726Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341866131071971:7899]
2025-10-09T21:47:12.171243Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 877, pipe: [1:7559341866131071971:7899]
2025-10-09T21:47:12.325511Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:12.326110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:12.326126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:12.326190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341866131071971:7899], now have 1 active actors on pipe
2025-10-09T21:47:12.326411Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:12.327297Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:12.327358Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341866131071971:7899]
2025-10-09T21:47:12.327382Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:12.327451Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 878 reason: status is not ok: tablet will be restarted right now sessionId: src_id|98b3dac6-780b747c-c5c4a325-4814e735_0
2025-10-09T21:47:12.327637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 878 sessionId: src_id|98b3dac6-780b747c-c5c4a325-4814e735_0 is DEAD
2025-10-09T21:47:12.331452Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98b3dac6-780b747c-c5c4a325-4814e735_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:12.331465Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98b3dac6-780b747c-c5c4a325-4814e735_0] Write session will restart in 0.007317s
2025-10-09T21:47:12.331496Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98b3dac6-780b747c-c5c4a325-4814e735_0] Write session: Do CDS request
2025-10-09T21:47:12.331503Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98b3dac6-780b747c-c5c4a325-4814e735_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:12.339607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98b3dac6-780b747c-c5c4a325-4814e735_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:12.339781Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:12.339800Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 879
2025-10-09T21:47:12.342433Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 879 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:12.342491Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 879 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:12.342505Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=879 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:12.342515Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:12.343198Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 879 sessionId:  describe result for acl check
2025-10-09T21:47:12.343241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:12.343244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:12.343245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:12.343252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341866131071995:8378] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:12.343860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341866131071995:8378] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:12.347233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341866131071995:8378] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:12.350937Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341866131071995:8378] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:12.350963Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341866131071995:8378] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:12.350969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341866131071995:8378] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:12.350979Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 879 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:12.357060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:12.359410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:12.359660Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:12.359672Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:12.359686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:12.359753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:12.359794Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341865769253105:7573]
2025-10-09T21:47:12.360403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:12.360783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:12.360839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:12.361196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:12.361266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:12.361274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:12.361276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:12.361277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:12.361278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:12.361280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:12.361281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.361282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.361284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.361290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:12.361301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:12.361305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:12.361309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:12.361313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:12.361317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:12.361321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.361325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.361329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.361338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:12.361347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:12.361353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:12.361359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:12.361364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:12.361369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:12.361373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:12.361379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:12.361384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:12.361389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:12.361393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:12.361398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:12.361403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.361408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.361412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.361417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.361422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.361427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.361458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:12.368786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:12.369964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:12.369979Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:12.369982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:12.369988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:12.369995Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 878 [2:7559341865769253105:7573]
2025-10-09T21:47:12.370012Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:12.370053Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:12.370062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:12.370066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:12.370068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:12.370090Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:12.370105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:12.370113Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:12.370183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.370191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.370194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.370197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:12.370279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:12.370291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.370303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.370313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.370329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:12.370350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341865769253100:7573]
2025-10-09T21:47:12.370353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341865769253100:7573]
2025-10-09T21:47:12.370355Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341865769253100:7573]
2025-10-09T21:47:12.370370Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:12.370388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341866131071979:2432], now have 1 active actors on pipe
2025-10-09T21:47:12.370391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341866131072087:8378], now have 1 active actors on pipe
2025-10-09T21:47:12.370390Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:12.370394Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:12.370397Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:12.370400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:12.370414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:12.370417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:12.370462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341865769253100:7573]
2025-10-09T21:47:12.370477Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:12.370483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:12.370587Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 878
2025-10-09T21:47:12.370652Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 878
2025-10-09T21:47:12.370970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:12.370986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:12.371020Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3e63baa4-6601a9b4-b421e848-e4403d32_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:12.371030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:12.371041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:12.371045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:12.371068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:12.371084Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:12.371107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:12.371552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:12.371562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:12.371591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:12.371765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 879 partition: 0 MaxSeqNo: 25 sessionId: src_id|3e63baa4-6601a9b4-b421e848-e4403d32_0
2025-10-09T21:47:12.372269Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98b3dac6-780b747c-c5c4a325-4814e735_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3e63baa4-6601a9b4-b421e848-e4403d32_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:12.374757Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3e63baa4-6601a9b4-b421e848-e4403d32_0] Write session: try to update token
2025-10-09T21:47:12.374776Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3e63baa4-6601a9b4-b421e848-e4403d32_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:12.387884Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:12.387905Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:12.387907Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:12.387910Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:12.387921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341865769253100:7573]
2025-10-09T21:47:12.387926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341865769253100:7573]
2025-10-09T21:47:12.387930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341865769253100:7573]
2025-10-09T21:47:12.387978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:12.387986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.387989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:12.388070Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:12.388083Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:12.388090Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:12.388145Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:12.402853Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 879 sessionId: src_id|3e63baa4-6601a9b4-b421e848-e4403d32_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:12.405392Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:12.427533Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341866131072191:7899]
2025-10-09T21:47:12.427951Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 878, pipe: [1:7559341866131072191:7899]
2025-10-09T21:47:12.559865Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:12.560856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:12.560879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:12.560906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341866131072191:7899], now have 1 active actors on pipe
2025-10-09T21:47:12.561144Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:12.561218Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:12.561289Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341866131072191:7899]
2025-10-09T21:47:12.561311Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:12.561327Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:12.561398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 879 reason: error at write: Disconnected sessionId: src_id|3e63baa4-6601a9b4-b421e848-e4403d32_0
2025-10-09T21:47:12.561598Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 879 sessionId: src_id|3e63baa4-6601a9b4-b421e848-e4403d32_0 is DEAD
2025-10-09T21:47:12.561941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e63baa4-6601a9b4-b421e848-e4403d32_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:12.561953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e63baa4-6601a9b4-b421e848-e4403d32_0] Write session will restart in 0.009788s
2025-10-09T21:47:12.561980Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e63baa4-6601a9b4-b421e848-e4403d32_0] Write session: Do CDS request
2025-10-09T21:47:12.561987Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e63baa4-6601a9b4-b421e848-e4403d32_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:12.572641Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3e63baa4-6601a9b4-b421e848-e4403d32_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:12.573166Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:12.573186Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 880
2025-10-09T21:47:12.573359Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 880 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:12.573394Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 880 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:12.573400Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=880 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:12.573405Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:12.574985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 880 sessionId:  describe result for acl check
2025-10-09T21:47:12.575035Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:12.575038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:12.575040Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:12.575045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341866131072210:8382] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:12.575660Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341866131072210:8382] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:12.579193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341866131072210:8382] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:12.590697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341866131072210:8382] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:12.590722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341866131072210:8382] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:12.590726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341866131072210:8382] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:12.590738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 880 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:12.630961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:12.636773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:12.637059Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:12.637072Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:12.637088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:12.637161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:12.637208Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341865769253326:7575]
2025-10-09T21:47:12.637828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:12.642260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:12.642363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:12.642741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:12.642816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:12.642826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:12.642828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:12.642829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:12.642830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:12.642832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:12.642833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.642834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.642836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.642843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:12.642853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:12.642857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:12.642862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:12.642865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:12.642871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:12.642874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.642879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.642883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.642891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:12.642900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:12.642907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:12.642913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:12.642917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:12.642925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:12.642929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:12.642935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:12.642948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:12.642953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:12.642958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:12.642963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:12.642968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.642973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.642978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.642983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.642987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.642992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.643023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:12.660434Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341866131072409:7899]
2025-10-09T21:47:12.663237Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 879
2025-10-09T21:47:12.663270Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 879, pipe: [1:7559341866131072409:7899]
2025-10-09T21:47:12.663291Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 879
2025-10-09T21:47:12.661502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:12.662852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:12.662858Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:12.662861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:12.662866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:12.662874Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 879 [2:7559341865769253326:7575]
2025-10-09T21:47:12.662889Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:12.662908Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:12.662912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:12.662915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:12.662918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:12.662939Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:12.662946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:12.662953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:12.663014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.663017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.663019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.663024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:12.663125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:12.663128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.663142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.663155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.663173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:12.663196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341866131072270:8382], now have 1 active actors on pipe
2025-10-09T21:47:12.663200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341866131072409:7899], now have 1 active actors on pipe
2025-10-09T21:47:12.663204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341866131072193:2432], now have 1 active actors on pipe
2025-10-09T21:47:12.663212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341865769253320:7575]
2025-10-09T21:47:12.663215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341865769253320:7575]
2025-10-09T21:47:12.663217Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341865769253320:7575]
2025-10-09T21:47:12.663231Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:12.663247Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:12.663266Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:12.663269Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:12.663310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341865769253320:7575]
2025-10-09T21:47:12.663313Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:12.663404Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:12.663408Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:12.663410Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:12.663418Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:12.664399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:12.664415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:12.664451Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2b6c0891-90b8b113-effa715c-2c69e9cd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:12.664467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:12.664476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:12.664479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:12.664491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:12.664510Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:12.664538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:12.666180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:12.666197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:12.666227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:12.666789Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e63baa4-6601a9b4-b421e848-e4403d32_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2b6c0891-90b8b113-effa715c-2c69e9cd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:12.669351Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b6c0891-90b8b113-effa715c-2c69e9cd_0] Write session: try to update token
2025-10-09T21:47:12.669370Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b6c0891-90b8b113-effa715c-2c69e9cd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:12.666395Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 880 partition: 0 MaxSeqNo: 25 sessionId: src_id|2b6c0891-90b8b113-effa715c-2c69e9cd_0
2025-10-09T21:47:12.686457Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:12.686479Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:12.686482Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:12.686484Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:12.686494Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341865769253320:7575]
2025-10-09T21:47:12.686499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341865769253320:7575]
2025-10-09T21:47:12.686502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341865769253320:7575]
2025-10-09T21:47:12.686545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:12.686554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.686557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:12.686770Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:12.689500Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 880 sessionId: src_id|2b6c0891-90b8b113-effa715c-2c69e9cd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:12.690132Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:12.690151Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:12.690155Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:12.692180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:12.890972Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:12.891361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:12.891379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:12.891852Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:12.891869Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:12.891942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 880 reason: error at write: Disconnected sessionId: src_id|2b6c0891-90b8b113-effa715c-2c69e9cd_0
2025-10-09T21:47:12.891657Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:12.892152Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 880 sessionId: src_id|2b6c0891-90b8b113-effa715c-2c69e9cd_0 is DEAD
2025-10-09T21:47:12.892154Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341866131072409:7899]
2025-10-09T21:47:12.892459Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b6c0891-90b8b113-effa715c-2c69e9cd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:12.892469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b6c0891-90b8b113-effa715c-2c69e9cd_0] Write session will restart in 0.005352s
2025-10-09T21:47:12.892494Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b6c0891-90b8b113-effa715c-2c69e9cd_0] Write session: Do CDS request
2025-10-09T21:47:12.892502Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b6c0891-90b8b113-effa715c-2c69e9cd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:12.892437Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:12.899697Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b6c0891-90b8b113-effa715c-2c69e9cd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:12.899824Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:12.899844Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 881
2025-10-09T21:47:12.900100Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 881 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:12.900154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 881 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:12.900165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=881 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:12.900169Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:12.900766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 881 sessionId:  describe result for acl check
2025-10-09T21:47:12.900802Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:12.900804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:12.900805Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:12.900812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341866131072459:8394] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:12.901356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341866131072459:8394] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:12.905848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341866131072459:8394] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:12.913135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341866131072459:8394] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:12.913154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341866131072459:8394] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:12.913157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341866131072459:8394] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:12.913168Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 881 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:12.920044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:12.922579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:12.922851Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:12.922863Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:12.922876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:12.922943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:12.922994Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341865769253555:7591]
2025-10-09T21:47:12.923616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:12.924275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:12.924343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:12.924693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:12.924755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:12.924757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:12.924758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:12.924759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:12.924761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:12.924762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:12.924763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.924765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.924766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.924773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:12.924785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:12.924790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:12.924794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:12.924798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:12.924803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:12.924806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.924811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.924815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.924822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:12.924831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:12.924838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:12.924844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:12.924849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:12.924855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:12.924859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:12.924864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:12.924873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:12.924878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:12.924883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:12.924888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:12.924893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.924898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.924902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.924907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.924912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.924933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.924964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:12.933425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:12.934646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:12.934667Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:12.934671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:12.934676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:12.934689Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 880 [2:7559341865769253555:7591]
2025-10-09T21:47:12.934706Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:12.934728Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:12.934737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:12.934741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:12.934744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:12.934764Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:12.934775Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:12.934782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:12.934859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.934866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.934868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.934871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:12.934952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:12.934961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.934975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:12.934988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:12.935004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:12.935027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341865769253550:7591]
2025-10-09T21:47:12.935030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341865769253550:7591]
2025-10-09T21:47:12.935032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341865769253550:7591]
2025-10-09T21:47:12.935048Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:12.935065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:12.935085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:12.935094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:12.935120Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:12.935123Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:12.935125Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:12.935140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341865769253550:7591]
2025-10-09T21:47:12.935143Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:12.935174Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:12.935393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341866131072597:8394], now have 1 active actors on pipe
2025-10-09T21:47:12.935408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341866131072445:2432], now have 1 active actors on pipe
2025-10-09T21:47:12.935673Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 880
2025-10-09T21:47:12.935675Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 880
2025-10-09T21:47:12.935784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:12.935801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:12.935848Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7d50b3-14bdab47-79591f85-4bbef214_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:12.935864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:12.935869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:12.935872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:12.935884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:12.935898Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:12.935934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:12.936172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:12.936181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:12.936207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:12.936297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 881 partition: 0 MaxSeqNo: 25 sessionId: src_id|7d50b3-14bdab47-79591f85-4bbef214_0
2025-10-09T21:47:12.936687Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b6c0891-90b8b113-effa715c-2c69e9cd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7d50b3-14bdab47-79591f85-4bbef214_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:12.939326Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7d50b3-14bdab47-79591f85-4bbef214_0] Write session: try to update token
2025-10-09T21:47:12.939349Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7d50b3-14bdab47-79591f85-4bbef214_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:12.953740Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 881 sessionId: src_id|7d50b3-14bdab47-79591f85-4bbef214_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:12.956414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:12.956641Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:12.956664Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:12.956674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341865769253550:7591]
2025-10-09T21:47:12.956722Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:12.956723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:12.956734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:12.956772Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:12.956985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:12.957120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:12.957260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:12.957381Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:12.957572Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:12.957619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:12.957645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:12.957806Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:12.963425Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:12.963445Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:12.963447Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:12.963449Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:12.963459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341865769253550:7591]
2025-10-09T21:47:12.963463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341865769253550:7591]
2025-10-09T21:47:12.963466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341865769253550:7591]
2025-10-09T21:47:12.963510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:12.963516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:12.963519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:12.963660Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:12.966298Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:12.966309Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:12.966314Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:12.992345Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341866131072659:7899]
2025-10-09T21:47:12.992633Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 880, pipe: [1:7559341866131072659:7899]
2025-10-09T21:47:13.226537Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:13.227148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341866131072659:7899], now have 1 active actors on pipe
2025-10-09T21:47:13.235390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:13.235421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:13.235597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:13.235655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:13.235719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:13.235784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:13.235838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:13.235843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:13.235913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:13.235974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:13.236036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:13.236102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:13.236140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:13.236143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:13.236209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:13.236268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:13.236330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:13.236395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:13.236436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:13.236439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:13.236511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:13.236572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:13.236633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:13.236697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:13.236739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:13.236742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:13.236812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:13.236869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:13.236926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:13.236990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:13.237093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:13.237097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:13.237164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:13.237221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:13.237283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:13.237344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:13.237383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:13.237389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:13.237452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:13.237507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:13.237565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:13.237624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:13.237666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:13.237669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:13.237737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:13.237791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:13.237847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:13.237925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:13.237966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:13.237969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:13.238305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:13.238371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:13.238435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:13.238497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:13.238541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:13.238544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:13.238615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:13.238675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:13.238735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:13.238803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:13.238845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:13.238848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:13.240761Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:13.240573Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341866131072659:7899]
2025-10-09T21:47:13.240625Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:13.240628Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:13.240662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 881 reason: error at write: Disconnected sessionId: src_id|7d50b3-14bdab47-79591f85-4bbef214_0
2025-10-09T21:47:13.240864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 881 sessionId: src_id|7d50b3-14bdab47-79591f85-4bbef214_0 is DEAD
2025-10-09T21:47:13.240929Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:13.242368Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d50b3-14bdab47-79591f85-4bbef214_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:13.242389Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d50b3-14bdab47-79591f85-4bbef214_0] Write session will restart in 0.006491s
2025-10-09T21:47:13.242415Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d50b3-14bdab47-79591f85-4bbef214_0] Write session: Do CDS request
2025-10-09T21:47:13.242422Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d50b3-14bdab47-79591f85-4bbef214_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:13.251118Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7d50b3-14bdab47-79591f85-4bbef214_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:13.254442Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:13.254471Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 882
2025-10-09T21:47:13.254829Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 882 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:13.254896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 882 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:13.254903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=882 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:13.254908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:13.255798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 882 sessionId:  describe result for acl check
2025-10-09T21:47:13.255839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:13.255841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:13.255843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:13.255851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341870426040042:8399] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:13.256488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341870426040042:8399] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:13.263962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341870426040042:8399] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:13.269101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341870426040042:8399] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:13.269124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341870426040042:8399] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:13.269129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341870426040042:8399] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:13.269140Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 882 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:13.269103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:13.271977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:13.272228Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:13.272244Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:13.272257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:13.272322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:13.272373Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341870064221058:7593]
2025-10-09T21:47:13.273009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:13.273414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:13.273493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:13.273871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:13.273950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:13.273956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:13.273957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:13.273958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:13.273959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:13.273960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:13.273961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.273963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.273964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.273970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:13.273993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:13.273997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:13.274001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:13.274004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:13.274009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:13.274013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.274016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.274020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.274056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:13.274065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:13.274072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:13.274078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:13.274089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:13.274095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:13.274100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:13.274105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:13.274115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:13.274120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:13.274125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:13.274130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:13.274135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.274139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.274144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.274149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.274156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.274167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.274196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:13.281991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:13.283141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:13.283165Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:13.283168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:13.283174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:13.283181Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 881 [2:7559341870064221058:7593]
2025-10-09T21:47:13.283200Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:13.283241Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:13.283249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:13.283253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:13.283256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:13.283279Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:13.283290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:13.283299Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:13.283377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.283386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.283388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.283391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:13.283478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:13.283489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.283503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.283514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.283531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:13.283556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341870426039966:2432], now have 1 active actors on pipe
2025-10-09T21:47:13.283560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341870426040168:8399], now have 1 active actors on pipe
2025-10-09T21:47:13.283568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341870064221052:7593]
2025-10-09T21:47:13.283571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341870064221052:7593]
2025-10-09T21:47:13.283573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341870064221052:7593]
2025-10-09T21:47:13.283590Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:13.283610Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:13.283622Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:13.283627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:13.283630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:13.283634Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:13.283637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:13.283677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341870064221052:7593]
2025-10-09T21:47:13.283692Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:13.283702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:13.285012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:13.285030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:13.285084Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9fe7df01-cbbce035-615d96a5-65247220_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:13.285100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:13.285105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:13.285110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:13.285137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:13.285154Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:13.285180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:13.285511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:13.285519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:13.285536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:13.284626Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 881
2025-10-09T21:47:13.284670Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 881
2025-10-09T21:47:13.285743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 882 partition: 0 MaxSeqNo: 25 sessionId: src_id|9fe7df01-cbbce035-615d96a5-65247220_0
2025-10-09T21:47:13.286252Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d50b3-14bdab47-79591f85-4bbef214_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9fe7df01-cbbce035-615d96a5-65247220_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:13.288847Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9fe7df01-cbbce035-615d96a5-65247220_0] Write session: try to update token
2025-10-09T21:47:13.288869Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9fe7df01-cbbce035-615d96a5-65247220_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:13.301990Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:13.302009Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:13.302012Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:13.302014Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:13.302026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341870064221052:7593]
2025-10-09T21:47:13.302047Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341870064221052:7593]
2025-10-09T21:47:13.302052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341870064221052:7593]
2025-10-09T21:47:13.302101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:13.302111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.302114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:13.302113Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:13.302120Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:13.302124Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:13.302252Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:13.309701Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 882 sessionId: src_id|9fe7df01-cbbce035-615d96a5-65247220_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:13.312349Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:13.341025Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341870426040181:7899]
2025-10-09T21:47:13.343930Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 881, pipe: [1:7559341870426040181:7899]
2025-10-09T21:47:13.483217Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:13.484014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:13.484034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:13.484086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341870426040181:7899], now have 1 active actors on pipe
2025-10-09T21:47:13.484321Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:13.484331Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341870426040181:7899]
2025-10-09T21:47:13.484350Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:13.484350Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:13.484414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 882 reason: error at write: Disconnected sessionId: src_id|9fe7df01-cbbce035-615d96a5-65247220_0
2025-10-09T21:47:13.484618Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 882 sessionId: src_id|9fe7df01-cbbce035-615d96a5-65247220_0 is DEAD
2025-10-09T21:47:13.484987Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9fe7df01-cbbce035-615d96a5-65247220_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:13.485000Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9fe7df01-cbbce035-615d96a5-65247220_0] Write session will restart in 0.009923s
2025-10-09T21:47:13.485025Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9fe7df01-cbbce035-615d96a5-65247220_0] Write session: Do CDS request
2025-10-09T21:47:13.485035Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9fe7df01-cbbce035-615d96a5-65247220_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:13.485108Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:13.496555Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9fe7df01-cbbce035-615d96a5-65247220_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:13.496678Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:13.496697Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 883
2025-10-09T21:47:13.496926Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 883 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:13.496980Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 883 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:13.496994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=883 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:13.496997Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:13.497619Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 883 sessionId:  describe result for acl check
2025-10-09T21:47:13.497659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:13.497662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:13.497663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:13.497670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341870426040227:8403] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:13.498297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341870426040227:8403] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:13.503290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341870426040227:8403] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:13.506943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341870426040227:8403] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:13.506962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341870426040227:8403] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:13.506966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341870426040227:8403] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:13.506977Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 883 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:13.508233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:13.510301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:13.510523Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:13.510531Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:13.510545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:13.510607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:13.510655Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341870064221264:7595]
2025-10-09T21:47:13.511238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:13.511572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:13.511627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:13.511965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:13.512033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:13.512045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:13.512046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:13.512047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:13.512048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:13.512049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:13.512051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.512052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.512054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.512059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:13.512071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:13.512075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:13.512078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:13.512082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:13.512086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:13.512089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.512092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.512096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.512104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:13.512111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:13.512118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:13.512123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:13.512127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:13.512132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:13.512136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:13.512141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:13.512145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:13.512150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:13.512156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:13.512161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:13.512165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.512170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.512174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.512179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.512183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.512188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.512217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:13.518917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:13.520036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:13.520054Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:13.520058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:13.520062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:13.520071Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 882 [2:7559341870064221264:7595]
2025-10-09T21:47:13.520086Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:13.520107Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:13.520110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:13.520114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:13.520116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:13.520135Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:13.520148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:13.520162Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:13.520231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.520236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.520239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.520241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:13.520325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:13.520333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.520345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.520354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.520369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:13.520396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341870064221259:7595]
2025-10-09T21:47:13.520399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341870064221259:7595]
2025-10-09T21:47:13.520401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341870064221259:7595]
2025-10-09T21:47:13.520409Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:13.520418Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:13.520426Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:13.520428Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:13.520433Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:13.520448Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:13.520451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:13.520464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341870426040184:2432], now have 1 active actors on pipe
2025-10-09T21:47:13.520467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341870426040385:8403], now have 1 active actors on pipe
2025-10-09T21:47:13.520505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341870064221259:7595]
2025-10-09T21:47:13.520517Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:13.520537Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 882
2025-10-09T21:47:13.520573Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 882
2025-10-09T21:47:13.520618Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:13.520855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:13.520870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:13.520902Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c03d1258-f18732d5-8121b2de-cff67a2a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:13.520915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:13.520924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:13.520926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:13.520942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:13.520959Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:13.520976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:13.521198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:13.521207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:13.521224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:13.521331Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 883 partition: 0 MaxSeqNo: 25 sessionId: src_id|c03d1258-f18732d5-8121b2de-cff67a2a_0
2025-10-09T21:47:13.521753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9fe7df01-cbbce035-615d96a5-65247220_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c03d1258-f18732d5-8121b2de-cff67a2a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:13.524303Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c03d1258-f18732d5-8121b2de-cff67a2a_0] Write session: try to update token
2025-10-09T21:47:13.524323Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c03d1258-f18732d5-8121b2de-cff67a2a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:13.540452Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:13.540474Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:13.540477Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:13.540479Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:13.540489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341870064221259:7595]
2025-10-09T21:47:13.540494Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341870064221259:7595]
2025-10-09T21:47:13.540502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341870064221259:7595]
2025-10-09T21:47:13.540552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:13.540563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.540566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:13.540567Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:13.540588Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:13.540607Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:13.540702Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:13.542935Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 883 sessionId: src_id|c03d1258-f18732d5-8121b2de-cff67a2a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:13.545483Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:13.583646Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341870426040399:7899]
2025-10-09T21:47:13.584032Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 882, pipe: [1:7559341870426040399:7899]
2025-10-09T21:47:13.709212Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:13.709794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:13.709805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:13.709851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341870426040399:7899], now have 1 active actors on pipe
2025-10-09T21:47:13.710199Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:13.710273Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341870426040399:7899]
2025-10-09T21:47:13.710292Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:13.710294Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:13.710356Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 883 reason: error at write: Disconnected sessionId: src_id|c03d1258-f18732d5-8121b2de-cff67a2a_0
2025-10-09T21:47:13.710147Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:13.710547Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 883 sessionId: src_id|c03d1258-f18732d5-8121b2de-cff67a2a_0 is DEAD
2025-10-09T21:47:13.713321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c03d1258-f18732d5-8121b2de-cff67a2a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:13.713339Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c03d1258-f18732d5-8121b2de-cff67a2a_0] Write session will restart in 0.008921s
2025-10-09T21:47:13.713373Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c03d1258-f18732d5-8121b2de-cff67a2a_0] Write session: Do CDS request
2025-10-09T21:47:13.713381Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c03d1258-f18732d5-8121b2de-cff67a2a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:13.723627Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c03d1258-f18732d5-8121b2de-cff67a2a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:13.726411Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:13.726429Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 884
2025-10-09T21:47:13.726827Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 884 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:13.726880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 884 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:13.726885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=884 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:13.726889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:13.727534Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 884 sessionId:  describe result for acl check
2025-10-09T21:47:13.727577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:13.727579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:13.727580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:13.727587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341870426040472:8407] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:13.728120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341870426040472:8407] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:13.732996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341870426040472:8407] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:13.736594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341870426040472:8407] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:13.736610Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341870426040472:8407] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:13.736615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341870426040472:8407] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:13.736625Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 884 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:13.738065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:13.740486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:13.740668Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:13.740679Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:13.740691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:13.740748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:13.740787Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341870064221505:7605]
2025-10-09T21:47:13.741170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:13.741748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:13.741807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:13.742199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:13.742256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:13.742262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:13.742264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:13.742265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:13.742266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:13.742267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:13.742268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.742270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.742271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.742277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:13.742288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:13.742292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:13.742296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:13.742301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:13.742305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:13.742309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.742313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.742317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.742324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:13.742332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:13.742339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:13.742345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:13.742349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:13.742354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:13.742365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:13.742370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:13.742375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:13.742380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:13.742384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:13.742401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:13.742406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.742411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.742415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.742421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.742425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.742431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.742458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:13.754220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:13.755306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:13.755323Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:13.755326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:13.755331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:13.755339Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 883 [2:7559341870064221505:7605]
2025-10-09T21:47:13.755355Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:13.755375Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:13.755379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:13.755382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:13.755385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:13.755404Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:13.755410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:13.755418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:13.755484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.755486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.755488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.755492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:13.755572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:13.755575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.755588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:13.755600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:13.755618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:13.755640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341870426040402:2432], now have 1 active actors on pipe
2025-10-09T21:47:13.755644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341870426040603:8407], now have 1 active actors on pipe
2025-10-09T21:47:13.755652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341870064221487:7605]
2025-10-09T21:47:13.755655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341870064221487:7605]
2025-10-09T21:47:13.755656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341870064221487:7605]
2025-10-09T21:47:13.755670Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:13.755684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:13.755699Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:13.755703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:13.755747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341870064221487:7605]
2025-10-09T21:47:13.755749Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:13.755815Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:13.755819Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:13.755821Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:13.755830Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:13.755974Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 883
2025-10-09T21:47:13.756019Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 883
2025-10-09T21:47:13.756776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:13.756851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:13.756885Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|67825002-ce6259fb-4cb6bc61-a6c7f6c3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:13.756896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:13.756902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:13.756905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:13.756920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:13.756935Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:13.756955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:13.757608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 884 partition: 0 MaxSeqNo: 25 sessionId: src_id|67825002-ce6259fb-4cb6bc61-a6c7f6c3_0
2025-10-09T21:47:13.757374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:13.757388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:13.757412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:13.758013Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c03d1258-f18732d5-8121b2de-cff67a2a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|67825002-ce6259fb-4cb6bc61-a6c7f6c3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:13.760584Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|67825002-ce6259fb-4cb6bc61-a6c7f6c3_0] Write session: try to update token
2025-10-09T21:47:13.760603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|67825002-ce6259fb-4cb6bc61-a6c7f6c3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:13.776002Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:13.776019Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:13.776022Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:13.776024Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:13.776033Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341870064221487:7605]
2025-10-09T21:47:13.776038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341870064221487:7605]
2025-10-09T21:47:13.776045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341870064221487:7605]
2025-10-09T21:47:13.776102Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:13.776128Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:13.776136Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:13.776291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:13.776306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:13.776308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:13.776585Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:13.788846Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 884 sessionId: src_id|67825002-ce6259fb-4cb6bc61-a6c7f6c3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:13.791299Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:13.810426Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341870426040621:7899]
2025-10-09T21:47:13.811169Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 883, pipe: [1:7559341870426040621:7899]
2025-10-09T21:47:14.010657Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:14.011136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:14.011148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:14.011173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341870426040621:7899], now have 1 active actors on pipe
2025-10-09T21:47:14.014630Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:14.014644Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:14.014708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 884 reason: error at write: Disconnected sessionId: src_id|67825002-ce6259fb-4cb6bc61-a6c7f6c3_0
2025-10-09T21:47:14.014890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 884 sessionId: src_id|67825002-ce6259fb-4cb6bc61-a6c7f6c3_0 is DEAD
2025-10-09T21:47:14.015101Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341870426040621:7899]
2025-10-09T21:47:14.015116Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:14.015465Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|67825002-ce6259fb-4cb6bc61-a6c7f6c3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:14.015488Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|67825002-ce6259fb-4cb6bc61-a6c7f6c3_0] Write session will restart in 0.006817s
2025-10-09T21:47:14.015518Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|67825002-ce6259fb-4cb6bc61-a6c7f6c3_0] Write session: Do CDS request
2025-10-09T21:47:14.015530Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|67825002-ce6259fb-4cb6bc61-a6c7f6c3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:14.017150Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:14.026771Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|67825002-ce6259fb-4cb6bc61-a6c7f6c3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:14.027140Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:14.027155Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 885
2025-10-09T21:47:14.027362Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 885 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:14.027417Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 885 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:14.027428Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=885 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:14.027432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:14.028334Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 885 sessionId:  describe result for acl check
2025-10-09T21:47:14.028384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:14.028386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:14.028388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:14.028394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341874721007959:8419] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:14.028967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341874721007959:8419] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:14.032049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341874721007959:8419] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:14.036868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341874721007959:8419] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:14.036886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341874721007959:8419] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:14.036890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341874721007959:8419] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:14.036903Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 885 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:14.051952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:14.055276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:14.055555Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:14.055570Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:14.055584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:14.055651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:14.055702Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341874359189018:7613]
2025-10-09T21:47:14.056327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:14.057114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:14.057180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:14.057541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:14.057616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:14.057627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:14.057629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:14.057630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:14.057632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:14.057633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:14.057634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.057635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.057637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.057643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:14.057654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:14.057659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:14.057663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:14.057667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:14.057671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:14.057675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.057679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.057683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.057701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:14.057709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:14.057721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:14.057728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:14.057734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:14.057739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:14.057744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:14.057750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:14.057754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:14.057758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:14.057763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:14.057769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:14.057775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.057780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.057785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.057790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.057795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.057800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.057829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:14.066589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:14.067692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:14.067711Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:14.067714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:14.067719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:14.067726Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 884 [2:7559341874359189018:7613]
2025-10-09T21:47:14.067745Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:14.067766Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:14.067770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:14.067773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:14.067775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:14.067795Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:14.067802Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:14.067809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:14.067880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.067889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.067891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.067895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:14.067975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:14.067980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.067991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.068001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.068022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:14.068043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341874359189013:7613]
2025-10-09T21:47:14.068052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341874359189013:7613]
2025-10-09T21:47:14.068055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341874359189013:7613]
2025-10-09T21:47:14.068069Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:14.068085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:14.068100Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:14.068104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:14.068148Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341874359189013:7613]
2025-10-09T21:47:14.068156Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:14.068343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:14.068359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:14.068363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:14.068369Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:14.068712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 884
2025-10-09T21:47:14.068712Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 884
2025-10-09T21:47:14.068724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341874721007995:8419], now have 1 active actors on pipe
2025-10-09T21:47:14.068737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341874721007944:2432], now have 1 active actors on pipe
2025-10-09T21:47:14.068982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:14.068995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:14.069025Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6d2ac02f-cab8e46f-f475cc6-ce54c65b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:14.069034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:14.069047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:14.069050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:14.069063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:14.069077Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:14.069096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:14.069294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:14.069306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:14.069324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:14.069451Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 885 partition: 0 MaxSeqNo: 25 sessionId: src_id|6d2ac02f-cab8e46f-f475cc6-ce54c65b_0
2025-10-09T21:47:14.069851Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|67825002-ce6259fb-4cb6bc61-a6c7f6c3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6d2ac02f-cab8e46f-f475cc6-ce54c65b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:14.072425Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6d2ac02f-cab8e46f-f475cc6-ce54c65b_0] Write session: try to update token
2025-10-09T21:47:14.072442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6d2ac02f-cab8e46f-f475cc6-ce54c65b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:14.086828Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:14.086853Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:14.086855Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:14.086857Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:14.086867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341874359189013:7613]
2025-10-09T21:47:14.086871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341874359189013:7613]
2025-10-09T21:47:14.086875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341874359189013:7613]
2025-10-09T21:47:14.086929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:14.086937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.086936Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:14.086939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:14.086941Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:14.086945Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:14.087078Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:14.099487Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 885 sessionId: src_id|6d2ac02f-cab8e46f-f475cc6-ce54c65b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:14.102136Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:14.114116Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341874721008167:7899]
2025-10-09T21:47:14.115377Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 884, pipe: [1:7559341874721008167:7899]
2025-10-09T21:47:14.310381Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:14.310983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:14.310994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:14.311019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341874721008167:7899], now have 1 active actors on pipe
2025-10-09T21:47:14.311741Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:14.311756Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:14.311817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 885 reason: error at write: Disconnected sessionId: src_id|6d2ac02f-cab8e46f-f475cc6-ce54c65b_0
2025-10-09T21:47:14.312084Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:14.312128Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341874721008167:7899]
2025-10-09T21:47:14.311696Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:14.312824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 885 sessionId: src_id|6d2ac02f-cab8e46f-f475cc6-ce54c65b_0 is DEAD
2025-10-09T21:47:14.314816Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d2ac02f-cab8e46f-f475cc6-ce54c65b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:14.314828Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d2ac02f-cab8e46f-f475cc6-ce54c65b_0] Write session will restart in 0.009946s
2025-10-09T21:47:14.314856Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d2ac02f-cab8e46f-f475cc6-ce54c65b_0] Write session: Do CDS request
2025-10-09T21:47:14.314863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d2ac02f-cab8e46f-f475cc6-ce54c65b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:14.325530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6d2ac02f-cab8e46f-f475cc6-ce54c65b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:14.327134Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:14.327154Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 886
2025-10-09T21:47:14.327464Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 886 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:14.327511Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 886 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:14.327525Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=886 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:14.327535Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:14.328443Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 886 sessionId:  describe result for acl check
2025-10-09T21:47:14.328493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:14.328504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:14.328505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:14.328512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341874721008202:8424] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:14.329131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341874721008202:8424] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:14.332238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341874721008202:8424] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:14.339521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341874721008202:8424] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:14.339547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341874721008202:8424] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:14.339550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341874721008202:8424] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:14.339561Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 886 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:14.354280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:14.354585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:14.354765Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:14.354769Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:14.354781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:14.354823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:14.354866Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341874359189228:7615]
2025-10-09T21:47:14.355510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:14.356108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:14.356156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:14.356478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:14.356522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:14.356524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:14.356525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:14.356526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:14.356528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:14.356529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:14.356530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.356531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.356532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.356537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:14.356549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:14.356553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:14.356556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:14.356561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:14.356565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:14.356568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.356572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.356576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.356583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:14.356591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:14.356597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:14.356603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:14.356607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:14.356612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:14.356616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:14.356622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:14.356626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:14.356631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:14.356636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:14.356641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:14.356646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.356651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.356656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.356661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.356665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.356670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.356700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:14.375136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:14.376324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:14.376342Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:14.376345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:14.376351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:14.376359Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 885 [2:7559341874359189228:7615]
2025-10-09T21:47:14.376376Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:14.376398Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:14.376402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:14.376405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:14.376408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:14.376429Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:14.376435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:14.376442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:14.376505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.376508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.376510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.376513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:14.376595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:14.376598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.376611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.376624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.376641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:14.376666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341874359189222:7615]
2025-10-09T21:47:14.376668Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341874359189222:7615]
2025-10-09T21:47:14.376673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341874359189222:7615]
2025-10-09T21:47:14.376690Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:14.376708Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:14.376722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:14.376725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:14.376769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341874359189222:7615]
2025-10-09T21:47:14.376771Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:14.378158Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 885
2025-10-09T21:47:14.378205Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 885
2025-10-09T21:47:14.377359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:14.377367Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:14.377368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:14.377375Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:14.377577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341874721008175:2432], now have 1 active actors on pipe
2025-10-09T21:47:14.377585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341874721008324:8424], now have 1 active actors on pipe
2025-10-09T21:47:14.378377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:14.378385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:14.378418Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|74fc7d53-bef2ce1-d6213e5c-5fa3d767_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:14.378428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:14.378434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:14.378438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:14.378451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:14.378465Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:14.378484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:14.381413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:14.381429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:14.381468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:14.381759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 886 partition: 0 MaxSeqNo: 25 sessionId: src_id|74fc7d53-bef2ce1-d6213e5c-5fa3d767_0
2025-10-09T21:47:14.382173Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d2ac02f-cab8e46f-f475cc6-ce54c65b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|74fc7d53-bef2ce1-d6213e5c-5fa3d767_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:14.384731Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74fc7d53-bef2ce1-d6213e5c-5fa3d767_0] Write session: try to update token
2025-10-09T21:47:14.384748Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74fc7d53-bef2ce1-d6213e5c-5fa3d767_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:14.399838Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:14.399856Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:14.399860Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:14.399862Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:14.399872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341874359189222:7615]
2025-10-09T21:47:14.399878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341874359189222:7615]
2025-10-09T21:47:14.399882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341874359189222:7615]
2025-10-09T21:47:14.399930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:14.399937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.399940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:14.400070Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:14.407147Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 886 sessionId: src_id|74fc7d53-bef2ce1-d6213e5c-5fa3d767_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:14.405349Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:14.405369Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:14.405374Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:14.409650Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:14.412530Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341874721008395:7899]
2025-10-09T21:47:14.412991Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 885, pipe: [1:7559341874721008395:7899]
2025-10-09T21:47:14.552497Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:14.552778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:14.552794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:14.552842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341874721008395:7899], now have 1 active actors on pipe
2025-10-09T21:47:14.553137Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:14.554293Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74fc7d53-bef2ce1-d6213e5c-5fa3d767_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:14.554305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74fc7d53-bef2ce1-d6213e5c-5fa3d767_0] Write session will restart in 0.005167s
2025-10-09T21:47:14.554327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74fc7d53-bef2ce1-d6213e5c-5fa3d767_0] Write session: Do CDS request
2025-10-09T21:47:14.554333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74fc7d53-bef2ce1-d6213e5c-5fa3d767_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:14.553808Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341874721008395:7899]
2025-10-09T21:47:14.553835Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:14.553876Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:14.553955Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 886 reason: status is not ok: tablet will be restarted right now sessionId: src_id|74fc7d53-bef2ce1-d6213e5c-5fa3d767_0
2025-10-09T21:47:14.554144Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 886 sessionId: src_id|74fc7d53-bef2ce1-d6213e5c-5fa3d767_0 is DEAD
2025-10-09T21:47:14.560553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74fc7d53-bef2ce1-d6213e5c-5fa3d767_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:14.560761Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:14.560776Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 887
2025-10-09T21:47:14.562267Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 887 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:14.562302Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 887 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:14.562308Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=887 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:14.562312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:14.562860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 887 sessionId:  describe result for acl check
2025-10-09T21:47:14.562891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:14.562897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:14.562898Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:14.562904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341874721008422:8428] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:14.563383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341874721008422:8428] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:14.578690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341874721008422:8428] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:14.580945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:14.584064Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341874721008422:8428] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:14.584081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341874721008422:8428] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:14.584085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341874721008422:8428] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:14.584094Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 887 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:14.583505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:14.583715Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:14.583719Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:14.583729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:14.583772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:14.583805Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341874359189437:7617]
2025-10-09T21:47:14.584327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:14.584786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:14.584819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:14.585124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:14.585161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:14.585162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:14.585163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:14.585164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:14.585165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:14.585166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:14.585167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.585168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.585169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.585173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:14.585183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:14.585186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:14.585190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:14.585193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:14.585197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:14.585199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.585202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.585205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.585213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:14.585220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:14.585225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:14.585230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:14.585235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:14.585239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:14.585243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:14.585247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:14.585252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:14.585256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:14.585259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:14.585264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:14.585269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.585273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.585277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.585281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.585287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.585291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.585317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:14.603454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:14.604915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:14.604932Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:14.604935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:14.604940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:14.604947Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 886 [2:7559341874359189437:7617]
2025-10-09T21:47:14.604963Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:14.604985Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:14.604992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:14.604996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:14.604998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:14.605016Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:14.605026Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:14.605033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:14.605100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.605106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.605108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.605111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:14.605181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:14.605188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.605201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.605210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.605228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:14.605252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341874359189431:7617]
2025-10-09T21:47:14.605259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341874359189431:7617]
2025-10-09T21:47:14.605261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341874359189431:7617]
2025-10-09T21:47:14.605271Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:14.605281Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:14.605286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:14.605298Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:14.605298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:14.605301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:14.605304Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:14.605314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341874721008400:2432], now have 1 active actors on pipe
2025-10-09T21:47:14.605318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341874721008604:8428], now have 1 active actors on pipe
2025-10-09T21:47:14.605356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341874359189431:7617]
2025-10-09T21:47:14.605369Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:14.605386Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:14.605578Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 886
2025-10-09T21:47:14.605621Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 886
2025-10-09T21:47:14.606135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:14.606148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:14.606191Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3c1214a5-a2c6bdb5-99e8a16b-238ea56b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:14.606202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:14.606207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:14.606210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:14.606231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:14.606244Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:14.606262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:14.606861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:14.606872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:14.606893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:14.607291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 887 partition: 0 MaxSeqNo: 25 sessionId: src_id|3c1214a5-a2c6bdb5-99e8a16b-238ea56b_0
2025-10-09T21:47:14.610010Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74fc7d53-bef2ce1-d6213e5c-5fa3d767_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3c1214a5-a2c6bdb5-99e8a16b-238ea56b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:14.612742Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c1214a5-a2c6bdb5-99e8a16b-238ea56b_0] Write session: try to update token
2025-10-09T21:47:14.612760Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c1214a5-a2c6bdb5-99e8a16b-238ea56b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:14.631440Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:14.631461Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:14.631464Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:14.631466Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:14.631476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341874359189431:7617]
2025-10-09T21:47:14.631481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341874359189431:7617]
2025-10-09T21:47:14.631485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341874359189431:7617]
2025-10-09T21:47:14.631534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:14.631542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.631545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:14.631701Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:14.632986Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:14.632993Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:14.632997Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:14.637395Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 887 sessionId: src_id|3c1214a5-a2c6bdb5-99e8a16b-238ea56b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:14.640460Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:14.654386Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341874721008617:7899]
2025-10-09T21:47:14.654696Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 886, pipe: [1:7559341874721008617:7899]
2025-10-09T21:47:14.811868Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341874721008617:7899]
2025-10-09T21:47:14.811904Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:14.811934Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:14.811936Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:14.811993Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 887 reason: error at write: Disconnected sessionId: src_id|3c1214a5-a2c6bdb5-99e8a16b-238ea56b_0
2025-10-09T21:47:14.812172Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 887 sessionId: src_id|3c1214a5-a2c6bdb5-99e8a16b-238ea56b_0 is DEAD
2025-10-09T21:47:14.812780Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c1214a5-a2c6bdb5-99e8a16b-238ea56b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:14.812792Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c1214a5-a2c6bdb5-99e8a16b-238ea56b_0] Write session will restart in 0.007233s
2025-10-09T21:47:14.812815Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c1214a5-a2c6bdb5-99e8a16b-238ea56b_0] Write session: Do CDS request
2025-10-09T21:47:14.812821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c1214a5-a2c6bdb5-99e8a16b-238ea56b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:14.810885Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:14.811190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:14.811199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:14.811244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341874721008617:7899], now have 1 active actors on pipe
2025-10-09T21:47:14.812214Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:14.822518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c1214a5-a2c6bdb5-99e8a16b-238ea56b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:14.823347Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:14.823364Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 888
2025-10-09T21:47:14.824202Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 888 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:14.824239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 888 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:14.824245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=888 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:14.824248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:14.825943Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 888 sessionId:  describe result for acl check
2025-10-09T21:47:14.825977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:14.825979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:14.825983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:14.825990Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341874721008664:8433] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:14.826500Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341874721008664:8433] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:14.837614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341874721008664:8433] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:14.843872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341874721008664:8433] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:14.843891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341874721008664:8433] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:14.843894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341874721008664:8433] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:14.843904Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 888 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:14.848116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:14.851054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:14.851251Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:14.851255Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:14.851266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:14.851308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:14.851342Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341874359189685:7631]
2025-10-09T21:47:14.851867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:14.852234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:14.852266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:14.852530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:14.852565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:14.852566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:14.852567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:14.852568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:14.852569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:14.852570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:14.852571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.852572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.852573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.852577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:14.852586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:14.852589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:14.852603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:14.852607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:14.852611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:14.852614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.852618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.852621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.852627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:14.852634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:14.852639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:14.852644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:14.852649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:14.852654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:14.852657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:14.852662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:14.852665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:14.852670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:14.852673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:14.852677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:14.852681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.852685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.852689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.852692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.852696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.852700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.852722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:14.863504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:14.864949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:14.864954Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:14.864957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:14.864962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:14.864970Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 887 [2:7559341874359189685:7631]
2025-10-09T21:47:14.864983Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:14.865001Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:14.865004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:14.865008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:14.865010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:14.865028Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:14.865034Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:14.865042Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:14.865098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.865101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.865103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.865105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:14.865175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:14.865178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.865189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:14.865212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:14.865228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:14.865247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341874359189679:7631]
2025-10-09T21:47:14.865250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341874359189679:7631]
2025-10-09T21:47:14.865252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341874359189679:7631]
2025-10-09T21:47:14.865263Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:14.865276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:14.865290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:14.865292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:14.865328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341874359189679:7631]
2025-10-09T21:47:14.865331Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:14.866026Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 887
2025-10-09T21:47:14.866080Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 887
2025-10-09T21:47:14.865600Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:14.865607Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:14.865609Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:14.865616Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:14.866323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341874721008833:8433], now have 1 active actors on pipe
2025-10-09T21:47:14.866335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341874721008629:2432], now have 1 active actors on pipe
2025-10-09T21:47:14.866590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:14.866599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:14.866631Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|49b9d751-1fcf0a09-fca1d31d-72e70d72_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:14.866641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:14.866646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:14.866649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:14.866662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:14.866675Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:14.866692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:14.866973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:14.866978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:14.866991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:14.867128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 888 partition: 0 MaxSeqNo: 25 sessionId: src_id|49b9d751-1fcf0a09-fca1d31d-72e70d72_0
2025-10-09T21:47:14.867503Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c1214a5-a2c6bdb5-99e8a16b-238ea56b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|49b9d751-1fcf0a09-fca1d31d-72e70d72_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:14.870174Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|49b9d751-1fcf0a09-fca1d31d-72e70d72_0] Write session: try to update token
2025-10-09T21:47:14.870191Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|49b9d751-1fcf0a09-fca1d31d-72e70d72_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:14.885569Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 888 sessionId: src_id|49b9d751-1fcf0a09-fca1d31d-72e70d72_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:14.888086Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:14.889144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:14.889168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:14.889189Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:14.889196Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:14.889199Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:14.889201Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:14.889209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341874359189679:7631]
2025-10-09T21:47:14.889212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341874359189679:7631]
2025-10-09T21:47:14.889216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341874359189679:7631]
2025-10-09T21:47:14.889257Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:14.889269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:14.889275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:14.889277Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:14.889277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:14.889281Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:14.889418Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:14.911724Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341874721008862:7899]
2025-10-09T21:47:14.912139Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 887, pipe: [1:7559341874721008862:7899]
2025-10-09T21:47:15.058835Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:15.059250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:15.059317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341874721008862:7899], now have 1 active actors on pipe
2025-10-09T21:47:15.061173Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49b9d751-1fcf0a09-fca1d31d-72e70d72_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:15.061183Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49b9d751-1fcf0a09-fca1d31d-72e70d72_0] Write session will restart in 0.009174s
2025-10-09T21:47:15.061209Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49b9d751-1fcf0a09-fca1d31d-72e70d72_0] Write session: Do CDS request
2025-10-09T21:47:15.061216Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49b9d751-1fcf0a09-fca1d31d-72e70d72_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:15.060403Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:15.060500Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:15.060503Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:15.060582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 888 reason: error at write: Disconnected sessionId: src_id|49b9d751-1fcf0a09-fca1d31d-72e70d72_0
2025-10-09T21:47:15.060729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 888 sessionId: src_id|49b9d751-1fcf0a09-fca1d31d-72e70d72_0 is DEAD
2025-10-09T21:47:15.060886Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341874721008862:7899]
2025-10-09T21:47:15.060899Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:15.074500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|49b9d751-1fcf0a09-fca1d31d-72e70d72_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:15.074863Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:15.074882Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 889
2025-10-09T21:47:15.075025Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 889 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:15.075053Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 889 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:15.075060Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=889 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:15.075064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:15.076079Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 889 sessionId:  describe result for acl check
2025-10-09T21:47:15.076121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:15.076123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:15.076125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:15.076133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341879015976207:8444] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:15.076674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341879015976207:8444] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:15.087808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341879015976207:8444] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:15.092445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:15.094524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:15.094801Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:15.094814Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:15.094827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:15.094886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:15.094932Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341878654157200:7635]
2025-10-09T21:47:15.095526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:15.097424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:15.097493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:15.097711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341879015976207:8444] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:15.097729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341879015976207:8444] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:15.097733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341879015976207:8444] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:15.097744Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 889 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:15.097863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:15.097946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.097952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.097953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.097955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.097956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.097957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.097958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.097960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.097961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.097966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.097977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.097981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.097985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.097989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.097992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.097996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.097999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.098002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.098010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.098025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.098048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.098058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.098066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.098071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.098074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.098084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.098093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.098097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.098102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.098107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.098111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.098116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.098120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.098124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.098128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.098132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.098158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:15.109054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:15.110716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:15.110723Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:15.110726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:15.110731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:15.110739Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 888 [2:7559341878654157200:7635]
2025-10-09T21:47:15.110753Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:15.110771Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:15.110774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:15.110779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:15.110781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:15.110802Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:15.110808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:15.110817Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:15.110885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.110888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.110890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.110893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:15.110967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:15.110971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.110986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.110999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.111017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:15.111037Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341878654157194:7635]
2025-10-09T21:47:15.111039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341878654157194:7635]
2025-10-09T21:47:15.111042Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341878654157194:7635]
2025-10-09T21:47:15.111051Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:15.111070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:15.111085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:15.111089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:15.111134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341878654157194:7635]
2025-10-09T21:47:15.111137Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:15.111568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:15.111584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:15.111586Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:15.111594Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:15.112046Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 888
2025-10-09T21:47:15.112089Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 888
2025-10-09T21:47:15.112300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341879015976383:8444], now have 1 active actors on pipe
2025-10-09T21:47:15.112317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341879015976172:2432], now have 1 active actors on pipe
2025-10-09T21:47:15.112332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:15.112338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:15.112373Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c67191ea-ba9c2c8b-b35af69d-dbe84800_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:15.112390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:15.112401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:15.112408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:15.112420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:15.112439Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:15.112462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:15.112993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:15.113004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:15.113022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:15.113184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 889 partition: 0 MaxSeqNo: 25 sessionId: src_id|c67191ea-ba9c2c8b-b35af69d-dbe84800_0
2025-10-09T21:47:15.113552Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49b9d751-1fcf0a09-fca1d31d-72e70d72_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c67191ea-ba9c2c8b-b35af69d-dbe84800_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:15.116618Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c67191ea-ba9c2c8b-b35af69d-dbe84800_0] Write session: try to update token
2025-10-09T21:47:15.116642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c67191ea-ba9c2c8b-b35af69d-dbe84800_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:15.133926Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:15.133947Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:15.133950Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:15.133952Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:15.133964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341878654157194:7635]
2025-10-09T21:47:15.133968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341878654157194:7635]
2025-10-09T21:47:15.133972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341878654157194:7635]
2025-10-09T21:47:15.134021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:15.134042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.134045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:15.134201Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:15.138498Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:15.138517Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:15.138523Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:15.144972Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 889 sessionId: src_id|c67191ea-ba9c2c8b-b35af69d-dbe84800_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:15.147530Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:15.161397Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341879015976397:7899]
2025-10-09T21:47:15.163008Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 888, pipe: [1:7559341879015976397:7899]
2025-10-09T21:47:15.326477Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:15.326963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:15.326984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:15.327034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341879015976397:7899], now have 1 active actors on pipe
2025-10-09T21:47:15.327365Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:15.327376Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341879015976397:7899]
2025-10-09T21:47:15.327419Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:15.327422Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:15.327480Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 889 reason: error at write: Disconnected sessionId: src_id|c67191ea-ba9c2c8b-b35af69d-dbe84800_0
2025-10-09T21:47:15.327668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 889 sessionId: src_id|c67191ea-ba9c2c8b-b35af69d-dbe84800_0 is DEAD
2025-10-09T21:47:15.327682Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:15.327821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c67191ea-ba9c2c8b-b35af69d-dbe84800_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:15.327831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c67191ea-ba9c2c8b-b35af69d-dbe84800_0] Write session will restart in 0.008386s
2025-10-09T21:47:15.327858Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c67191ea-ba9c2c8b-b35af69d-dbe84800_0] Write session: Do CDS request
2025-10-09T21:47:15.327865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c67191ea-ba9c2c8b-b35af69d-dbe84800_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:15.337647Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c67191ea-ba9c2c8b-b35af69d-dbe84800_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:15.337880Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:15.337909Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 890
2025-10-09T21:47:15.338058Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 890 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:15.338096Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 890 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:15.338116Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=890 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:15.338121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:15.338824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 890 sessionId:  describe result for acl check
2025-10-09T21:47:15.338869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:15.338871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:15.338872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:15.338878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341879015976438:8449] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:15.339443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341879015976438:8449] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:15.343484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341879015976438:8449] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:15.347978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341879015976438:8449] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:15.347996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341879015976438:8449] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:15.348001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341879015976438:8449] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:15.348015Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 890 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:15.349485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:15.350491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:15.350836Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:15.350850Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:15.350866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:15.350940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:15.351001Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341878654157415:7637]
2025-10-09T21:47:15.351597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:15.352415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:15.352477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:15.352868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:15.352919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.352926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.352927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.352928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.352930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.352931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.352932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.352933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.352934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.352940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.352951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.352956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.352960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.352964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.352969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.352972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.352976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.352980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.352988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.352997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.353020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.353026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.353031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.353036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.353041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.353046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.353052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.353057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.353061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.353067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.353073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.353079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.353083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.353089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.353094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.353099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.353128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:15.360648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:15.362047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:15.362063Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:15.362066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:15.362071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:15.362079Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 889 [2:7559341878654157415:7637]
2025-10-09T21:47:15.362115Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:15.362139Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:15.362148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:15.362152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:15.362154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:15.362174Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:15.362185Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:15.362193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:15.362262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.362271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.362273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.362276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:15.362359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:15.362367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.362381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.362392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.362412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:15.362442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341879015976613:8449], now have 1 active actors on pipe
2025-10-09T21:47:15.362451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341878654157410:7637]
2025-10-09T21:47:15.362453Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341878654157410:7637]
2025-10-09T21:47:15.362455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341878654157410:7637]
2025-10-09T21:47:15.362470Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:15.362480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341879015976405:2432], now have 1 active actors on pipe
2025-10-09T21:47:15.362490Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:15.362494Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:15.362495Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:15.362496Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:15.362510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:15.362518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:15.362540Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 889
2025-10-09T21:47:15.362562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341878654157410:7637]
2025-10-09T21:47:15.362580Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:15.362542Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 889
2025-10-09T21:47:15.362604Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:15.362635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:15.362640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:15.362669Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|55edfda2-22142352-883e665f-f31c4be1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:15.362683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:15.362688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:15.362691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:15.362703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:15.362720Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:15.362789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:15.363100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:15.363110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:15.363127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:15.363242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 890 partition: 0 MaxSeqNo: 25 sessionId: src_id|55edfda2-22142352-883e665f-f31c4be1_0
2025-10-09T21:47:15.363634Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c67191ea-ba9c2c8b-b35af69d-dbe84800_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|55edfda2-22142352-883e665f-f31c4be1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:15.366358Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55edfda2-22142352-883e665f-f31c4be1_0] Write session: try to update token
2025-10-09T21:47:15.366378Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55edfda2-22142352-883e665f-f31c4be1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:15.379169Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:15.379191Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:15.379194Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:15.379196Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:15.379206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341878654157410:7637]
2025-10-09T21:47:15.379211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341878654157410:7637]
2025-10-09T21:47:15.379215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341878654157410:7637]
2025-10-09T21:47:15.379264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:15.379268Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:15.379272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.379274Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:15.379275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:15.379278Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:15.379414Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:15.381851Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 890 sessionId: src_id|55edfda2-22142352-883e665f-f31c4be1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:15.384577Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:15.427344Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341879015976628:7899]
2025-10-09T21:47:15.427666Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 889, pipe: [1:7559341879015976628:7899]
2025-10-09T21:47:15.539855Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:15.540362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:15.540379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:15.540432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341879015976628:7899], now have 1 active actors on pipe
2025-10-09T21:47:15.540664Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:15.541372Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:15.541378Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:15.541442Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341879015976628:7899]
2025-10-09T21:47:15.541454Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 890 reason: status is not ok: tablet will be restarted right now sessionId: src_id|55edfda2-22142352-883e665f-f31c4be1_0
2025-10-09T21:47:15.541652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 890 sessionId: src_id|55edfda2-22142352-883e665f-f31c4be1_0 is DEAD
2025-10-09T21:47:15.542056Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55edfda2-22142352-883e665f-f31c4be1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:15.542068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55edfda2-22142352-883e665f-f31c4be1_0] Write session will restart in 0.009853s
2025-10-09T21:47:15.542096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55edfda2-22142352-883e665f-f31c4be1_0] Write session: Do CDS request
2025-10-09T21:47:15.542103Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55edfda2-22142352-883e665f-f31c4be1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:15.553579Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55edfda2-22142352-883e665f-f31c4be1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:15.554016Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:15.554050Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 891
2025-10-09T21:47:15.554208Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 891 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:15.554245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 891 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:15.554251Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=891 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:15.554261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:15.554792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 891 sessionId:  describe result for acl check
2025-10-09T21:47:15.554829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:15.554835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:15.554837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:15.554842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341879015976693:8453] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:15.555372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341879015976693:8453] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:15.560181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341879015976693:8453] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:15.563090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:15.563920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341879015976693:8453] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:15.563939Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341879015976693:8453] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:15.563942Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341879015976693:8453] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:15.563967Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 891 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:15.565379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:15.565621Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:15.565632Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:15.565644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:15.565709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:15.565756Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341878654157633:7641]
2025-10-09T21:47:15.566355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:15.566922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:15.566973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:15.567301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:15.567369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.567376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.567377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.567378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.567379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.567380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.567381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.567383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.567384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.567390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.567401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.567405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.567409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.567413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.567417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.567420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.567423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.567427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.567435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.567443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.567449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.567454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.567459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.567464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.567467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.567472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.567476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.567481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.567505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.567516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.567522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.567531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.567536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.567541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.567550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.567556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.567587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:15.574841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:15.576291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:15.576312Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:15.576315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:15.576319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:15.576328Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 890 [2:7559341878654157633:7641]
2025-10-09T21:47:15.576344Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:15.576366Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:15.576370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:15.576374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:15.576376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:15.576394Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:15.576406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:15.576414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:15.576480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.576488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.576491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.576495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:15.576573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:15.576580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.576595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.576611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.576632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:15.576656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341879015976634:2432], now have 1 active actors on pipe
2025-10-09T21:47:15.576665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341879015976846:8453], now have 1 active actors on pipe
2025-10-09T21:47:15.576674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341878654157627:7641]
2025-10-09T21:47:15.576681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341878654157627:7641]
2025-10-09T21:47:15.576683Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341878654157627:7641]
2025-10-09T21:47:15.576697Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:15.576716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:15.576726Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:15.576730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:15.576731Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:15.576738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:15.576741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:15.576781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341878654157627:7641]
2025-10-09T21:47:15.576785Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:15.576733Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 890
2025-10-09T21:47:15.576788Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:15.576773Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 890
2025-10-09T21:47:15.576808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:15.576813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:15.576869Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|23c30bf2-7f5c5ca7-aa8e26ce-5b55d552_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:15.576884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:15.576889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:15.576891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:15.576902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:15.576912Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:15.576940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:15.577400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:15.577411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:15.577430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:15.577564Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 891 partition: 0 MaxSeqNo: 25 sessionId: src_id|23c30bf2-7f5c5ca7-aa8e26ce-5b55d552_0
2025-10-09T21:47:15.577957Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55edfda2-22142352-883e665f-f31c4be1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|23c30bf2-7f5c5ca7-aa8e26ce-5b55d552_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:15.580511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23c30bf2-7f5c5ca7-aa8e26ce-5b55d552_0] Write session: try to update token
2025-10-09T21:47:15.580531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23c30bf2-7f5c5ca7-aa8e26ce-5b55d552_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:15.592862Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:15.592884Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:15.592887Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:15.592890Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:15.592900Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341878654157627:7641]
2025-10-09T21:47:15.592909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341878654157627:7641]
2025-10-09T21:47:15.592914Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341878654157627:7641]
2025-10-09T21:47:15.592963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:15.592971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.592974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:15.592975Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:15.592982Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:15.592986Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:15.593115Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:15.599832Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 891 sessionId: src_id|23c30bf2-7f5c5ca7-aa8e26ce-5b55d552_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:15.602619Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:15.641845Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341879015976859:7899]
2025-10-09T21:47:15.642222Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 890, pipe: [1:7559341879015976859:7899]
2025-10-09T21:47:15.763112Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:15.763627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:15.763645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:15.763670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341879015976859:7899], now have 1 active actors on pipe
2025-10-09T21:47:15.764074Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:15.764089Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341879015976859:7899]
2025-10-09T21:47:15.764115Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:15.764205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 891 reason: status is not ok: tablet will be restarted right now sessionId: src_id|23c30bf2-7f5c5ca7-aa8e26ce-5b55d552_0
2025-10-09T21:47:15.763914Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:15.764395Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 891 sessionId: src_id|23c30bf2-7f5c5ca7-aa8e26ce-5b55d552_0 is DEAD
2025-10-09T21:47:15.764573Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23c30bf2-7f5c5ca7-aa8e26ce-5b55d552_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:15.764582Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23c30bf2-7f5c5ca7-aa8e26ce-5b55d552_0] Write session will restart in 0.005729s
2025-10-09T21:47:15.764612Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23c30bf2-7f5c5ca7-aa8e26ce-5b55d552_0] Write session: Do CDS request
2025-10-09T21:47:15.764619Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23c30bf2-7f5c5ca7-aa8e26ce-5b55d552_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:15.771650Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23c30bf2-7f5c5ca7-aa8e26ce-5b55d552_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:15.771746Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:15.771764Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 892
2025-10-09T21:47:15.771967Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 892 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:15.772015Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 892 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:15.772028Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=892 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:15.772041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:15.772568Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 892 sessionId:  describe result for acl check
2025-10-09T21:47:15.772605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:15.772611Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:15.772614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:15.772620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341879015976876:8458] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:15.773195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341879015976876:8458] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:15.777416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341879015976876:8458] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:15.781034Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341879015976876:8458] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:15.781053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341879015976876:8458] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:15.781057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341879015976876:8458] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:15.781066Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 892 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:15.786025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:15.788663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:15.789122Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:15.789156Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:15.789175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:15.789263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:15.789309Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341878654157854:7645]
2025-10-09T21:47:15.789923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:15.790326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:15.790390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:15.790720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:15.790765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.790773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.790775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.790776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.790777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.790778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.790779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.790780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.790781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.790788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.790799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.790803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.790807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.790811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.790815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.790819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.790822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.790826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.790835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.790844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.790851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.790868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.790873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.790879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.790883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.790889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.790896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.790902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.790911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.790917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.790922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.790927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.790936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.790941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.790945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.790950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.790978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:15.797576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:15.799167Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 891
2025-10-09T21:47:15.799217Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 891
2025-10-09T21:47:15.798700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:15.798718Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:15.798721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:15.798726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:15.798733Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 891 [2:7559341878654157854:7645]
2025-10-09T21:47:15.798749Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:15.798769Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:15.798772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:15.798776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:15.798779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:15.798799Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:15.798806Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:15.798814Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:15.798880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.798883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.798885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.798887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:15.798962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:15.798964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.798976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.798987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.799002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:15.799022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341878654157849:7645]
2025-10-09T21:47:15.799025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341878654157849:7645]
2025-10-09T21:47:15.799027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341878654157849:7645]
2025-10-09T21:47:15.799036Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:15.799047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:15.799050Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:15.799053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:15.799055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341879015976995:8458], now have 1 active actors on pipe
2025-10-09T21:47:15.799062Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:15.799076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:15.799079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:15.799084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341879015976861:2432], now have 1 active actors on pipe
2025-10-09T21:47:15.799121Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341878654157849:7645]
2025-10-09T21:47:15.799130Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:15.799138Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:15.799271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:15.799277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:15.799305Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|504a3923-feee4578-ccc8ce27-ccb396d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:15.799313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:15.799317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:15.799319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:15.799330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:15.799341Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:15.799363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:15.799578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:15.799589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:15.799608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:15.799781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 892 partition: 0 MaxSeqNo: 25 sessionId: src_id|504a3923-feee4578-ccc8ce27-ccb396d_0
2025-10-09T21:47:15.802149Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23c30bf2-7f5c5ca7-aa8e26ce-5b55d552_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|504a3923-feee4578-ccc8ce27-ccb396d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:15.804780Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|504a3923-feee4578-ccc8ce27-ccb396d_0] Write session: try to update token
2025-10-09T21:47:15.804800Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|504a3923-feee4578-ccc8ce27-ccb396d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:15.813812Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:15.813843Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:15.813846Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:15.813850Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:15.813860Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341878654157849:7645]
2025-10-09T21:47:15.813863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341878654157849:7645]
2025-10-09T21:47:15.813868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341878654157849:7645]
2025-10-09T21:47:15.813927Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:15.813929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:15.813936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.813939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:15.813941Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:15.813945Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:15.814146Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:15.821818Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 892 sessionId: src_id|504a3923-feee4578-ccc8ce27-ccb396d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:15.824445Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:15.863064Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341879015977088:7899]
2025-10-09T21:47:15.863435Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 891, pipe: [1:7559341879015977088:7899]
2025-10-09T21:47:15.974180Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:15.974755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:15.974773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:15.974797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341879015977088:7899], now have 1 active actors on pipe
2025-10-09T21:47:15.975054Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:15.975067Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:15.975136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 892 reason: error at write: Disconnected sessionId: src_id|504a3923-feee4578-ccc8ce27-ccb396d_0
2025-10-09T21:47:15.975323Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 892 sessionId: src_id|504a3923-feee4578-ccc8ce27-ccb396d_0 is DEAD
2025-10-09T21:47:15.975042Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:15.975570Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341879015977088:7899]
2025-10-09T21:47:15.975600Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:15.975704Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|504a3923-feee4578-ccc8ce27-ccb396d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:15.975713Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|504a3923-feee4578-ccc8ce27-ccb396d_0] Write session will restart in 0.005075s
2025-10-09T21:47:15.975738Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|504a3923-feee4578-ccc8ce27-ccb396d_0] Write session: Do CDS request
2025-10-09T21:47:15.975745Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|504a3923-feee4578-ccc8ce27-ccb396d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:15.981622Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|504a3923-feee4578-ccc8ce27-ccb396d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:15.982048Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:15.982064Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 893
2025-10-09T21:47:15.982216Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 893 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:15.982247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 893 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:15.982253Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=893 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:15.982257Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:15.982682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 893 sessionId:  describe result for acl check
2025-10-09T21:47:15.982710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:15.982712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:15.982714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:15.982720Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341879015977122:8468] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:15.983275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341879015977122:8468] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:15.986587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341879015977122:8468] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:15.991292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341879015977122:8468] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:15.991310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341879015977122:8468] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:15.991313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341879015977122:8468] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:15.991322Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 893 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:15.995224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:15.997739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:15.998266Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:15.998277Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:15.998296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:15.998377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:15.998428Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341878654158108:7657]
2025-10-09T21:47:15.999026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:15.999504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:15.999556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:15.999892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:15.999943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.999962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.999969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:15.999970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:15.999971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:15.999973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:15.999974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:15.999975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:15.999976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:15.999983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:15.999992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:15.999996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.000001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.000005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.000009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.000013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.000016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.000019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.000027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.000036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.000041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.000046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.000051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.000056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.000060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.000065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.000069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.000074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.000079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.000083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.000090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.000094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.000099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.000103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.000108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.000113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.000139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:16.007151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:16.008229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:16.008248Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:16.008263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:16.008268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:16.008276Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 892 [2:7559341878654158108:7657]
2025-10-09T21:47:16.008293Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:16.008315Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:16.008325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:16.008330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:16.008333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:16.008354Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:16.008361Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:16.008370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:16.008439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.008449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.008451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.008454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:16.008544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:16.008553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.008568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.008583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.008602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:16.008624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341879015977107:2432], now have 1 active actors on pipe
2025-10-09T21:47:16.008628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341879015977274:8468], now have 1 active actors on pipe
2025-10-09T21:47:16.008636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341878654158103:7657]
2025-10-09T21:47:16.008642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341878654158103:7657]
2025-10-09T21:47:16.008644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341878654158103:7657]
2025-10-09T21:47:16.008658Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:16.008676Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:16.008693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:16.008697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:16.008740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341878654158103:7657]
2025-10-09T21:47:16.008749Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:16.008685Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 892
2025-10-09T21:47:16.008734Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 892
2025-10-09T21:47:16.008817Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:16.008826Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:16.008829Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:16.008847Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:16.008911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:16.008923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:16.008953Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a0d7b66d-c0b39ad8-fac6c29d-5ace47b2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:16.008961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:16.008966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:16.008969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:16.008980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:16.008992Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:16.009009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:16.009285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:16.009295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:16.009311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:16.009425Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 893 partition: 0 MaxSeqNo: 25 sessionId: src_id|a0d7b66d-c0b39ad8-fac6c29d-5ace47b2_0
2025-10-09T21:47:16.009836Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|504a3923-feee4578-ccc8ce27-ccb396d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a0d7b66d-c0b39ad8-fac6c29d-5ace47b2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:16.012855Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a0d7b66d-c0b39ad8-fac6c29d-5ace47b2_0] Write session: try to update token
2025-10-09T21:47:16.012880Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a0d7b66d-c0b39ad8-fac6c29d-5ace47b2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:16.024562Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:16.024585Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:16.024588Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:16.024590Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:16.024600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341878654158103:7657]
2025-10-09T21:47:16.024604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341878654158103:7657]
2025-10-09T21:47:16.024608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341878654158103:7657]
2025-10-09T21:47:16.024660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:16.024668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.024670Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:16.024671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:16.024678Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:16.024683Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:16.024812Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:16.033623Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 893 sessionId: src_id|a0d7b66d-c0b39ad8-fac6c29d-5ace47b2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:16.036593Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:16.075550Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341883310944636:7899]
2025-10-09T21:47:16.075919Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 892, pipe: [1:7559341883310944636:7899]
2025-10-09T21:47:16.226917Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:16.227275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:16.227293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:16.227343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341883310944636:7899], now have 1 active actors on pipe
2025-10-09T21:47:16.227761Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341883310944636:7899]
2025-10-09T21:47:16.227793Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:16.227837Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:16.227846Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:16.227909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 893 reason: error at write: Disconnected sessionId: src_id|a0d7b66d-c0b39ad8-fac6c29d-5ace47b2_0
2025-10-09T21:47:16.227612Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:16.228101Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 893 sessionId: src_id|a0d7b66d-c0b39ad8-fac6c29d-5ace47b2_0 is DEAD
2025-10-09T21:47:16.228270Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0d7b66d-c0b39ad8-fac6c29d-5ace47b2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:16.228281Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0d7b66d-c0b39ad8-fac6c29d-5ace47b2_0] Write session will restart in 0.009760s
2025-10-09T21:47:16.228308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0d7b66d-c0b39ad8-fac6c29d-5ace47b2_0] Write session: Do CDS request
2025-10-09T21:47:16.228314Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0d7b66d-c0b39ad8-fac6c29d-5ace47b2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:16.239725Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a0d7b66d-c0b39ad8-fac6c29d-5ace47b2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:16.241593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:16.241612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 894
2025-10-09T21:47:16.241769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 894 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:16.241827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 894 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:16.241838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=894 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:16.241842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:16.242445Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 894 sessionId:  describe result for acl check
2025-10-09T21:47:16.242487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:16.242489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:16.242491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:16.242498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341883310944689:8474] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:16.243070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341883310944689:8474] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:16.246943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341883310944689:8474] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:16.251391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341883310944689:8474] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:16.251410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341883310944689:8474] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:16.251414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341883310944689:8474] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:16.251425Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 894 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:16.257768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:16.260666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:16.260910Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:16.260918Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:16.260932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:16.260987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:16.261028Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341882949125622:7661]
2025-10-09T21:47:16.261748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:16.262358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:16.262417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:16.262786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:16.262850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.262851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.262852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.262854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.262855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.262856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.262858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.262859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.262860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.262867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.262877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.262881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.262884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.262888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.262891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.262895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.262899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.262902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.262912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.262921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.262927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.262933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.262937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.262943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.262947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.262953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.262958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.262962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.262967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.262973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.262978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.262983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.262988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.262992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.262998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.263003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.263031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:16.270106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:16.271587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:16.271606Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:16.271609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:16.271614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:16.271621Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 893 [2:7559341882949125622:7661]
2025-10-09T21:47:16.271636Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:16.271655Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:16.271659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:16.271663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:16.271665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:16.271684Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:16.271698Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:16.271708Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:16.271775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.271783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.271785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.271787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:16.271866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:16.271874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.271886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.271898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.271919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:16.271944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341883310944803:8474], now have 1 active actors on pipe
2025-10-09T21:47:16.271953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341883310944647:2432], now have 1 active actors on pipe
2025-10-09T21:47:16.271961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341882949125617:7661]
2025-10-09T21:47:16.271964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341882949125617:7661]
2025-10-09T21:47:16.271966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341882949125617:7661]
2025-10-09T21:47:16.271982Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:16.271997Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:16.272006Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:16.272011Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:16.272013Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:16.272013Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:16.272016Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:16.272068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341882949125617:7661]
2025-10-09T21:47:16.272076Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:16.272095Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:16.272289Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 893
2025-10-09T21:47:16.272367Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 893
2025-10-09T21:47:16.272403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:16.272418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:16.272452Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|efdfa375-b9c07a5e-619b5109-258ed42d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:16.272467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:16.272471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:16.272474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:16.272485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:16.272500Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:16.272520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:16.272834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:16.272850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:16.272875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:16.273058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 894 partition: 0 MaxSeqNo: 25 sessionId: src_id|efdfa375-b9c07a5e-619b5109-258ed42d_0
2025-10-09T21:47:16.273506Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0d7b66d-c0b39ad8-fac6c29d-5ace47b2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|efdfa375-b9c07a5e-619b5109-258ed42d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:16.276274Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|efdfa375-b9c07a5e-619b5109-258ed42d_0] Write session: try to update token
2025-10-09T21:47:16.276294Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|efdfa375-b9c07a5e-619b5109-258ed42d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:16.287660Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:16.287683Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:16.287686Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:16.287688Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:16.287698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341882949125617:7661]
2025-10-09T21:47:16.287704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341882949125617:7661]
2025-10-09T21:47:16.287708Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341882949125617:7661]
2025-10-09T21:47:16.287760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:16.287768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.287772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:16.287774Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:16.287781Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:16.287785Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:16.287927Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:16.294296Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 894 sessionId: src_id|efdfa375-b9c07a5e-619b5109-258ed42d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:16.296918Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:16.326417Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341883310944874:7899]
2025-10-09T21:47:16.326950Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 893, pipe: [1:7559341883310944874:7899]
2025-10-09T21:47:16.448301Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:16.448596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:16.448615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:16.448661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341883310944874:7899], now have 1 active actors on pipe
2025-10-09T21:47:16.448908Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:16.448910Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:16.448925Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:16.448971Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341883310944874:7899]
2025-10-09T21:47:16.448985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 894 reason: error at write: Disconnected sessionId: src_id|efdfa375-b9c07a5e-619b5109-258ed42d_0
2025-10-09T21:47:16.448975Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:16.449183Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 894 sessionId: src_id|efdfa375-b9c07a5e-619b5109-258ed42d_0 is DEAD
2025-10-09T21:47:16.449431Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efdfa375-b9c07a5e-619b5109-258ed42d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:16.449443Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efdfa375-b9c07a5e-619b5109-258ed42d_0] Write session will restart in 0.007758s
2025-10-09T21:47:16.449472Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efdfa375-b9c07a5e-619b5109-258ed42d_0] Write session: Do CDS request
2025-10-09T21:47:16.449478Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efdfa375-b9c07a5e-619b5109-258ed42d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:16.458574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|efdfa375-b9c07a5e-619b5109-258ed42d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:16.458685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:16.458701Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 895
2025-10-09T21:47:16.458849Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 895 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:16.458886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 895 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:16.458892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=895 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:16.458896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:16.459551Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 895 sessionId:  describe result for acl check
2025-10-09T21:47:16.459588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:16.459594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:16.459596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:16.459613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341883310944932:8478] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:16.460151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341883310944932:8478] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:16.463631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341883310944932:8478] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:16.466576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341883310944932:8478] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:16.466593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341883310944932:8478] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:16.466597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341883310944932:8478] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:16.466607Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 895 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:16.467115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:16.469338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:16.469673Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:16.469684Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:16.469697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:16.469758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:16.469801Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341882949125841:7663]
2025-10-09T21:47:16.470428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:16.470750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:16.470801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:16.471124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:16.471175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.471181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.471182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.471183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.471185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.471186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.471187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.471189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.471190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.471196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.471207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.471211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.471215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.471218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.471222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.471226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.471230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.471234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.471241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.471249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.471256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.471262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.471266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.471271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.471276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.471281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.471287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.471293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.471297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.471303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.471308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.471313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.471318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.471323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.471330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.471338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.471371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:16.478206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:16.479631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:16.479647Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:16.479650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:16.479655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:16.479663Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 894 [2:7559341882949125841:7663]
2025-10-09T21:47:16.479681Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:16.479702Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:16.479706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:16.479709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:16.479711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:16.479731Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:16.479738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:16.479749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:16.479819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.479827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.479830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.479833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:16.479908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:16.479918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.479931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.479944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.479961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:16.479986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341882949125836:7663]
2025-10-09T21:47:16.479994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341882949125836:7663]
2025-10-09T21:47:16.479996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341882949125836:7663]
2025-10-09T21:47:16.480005Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:16.480020Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:16.480032Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:16.480035Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:16.480038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:16.480045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:16.480047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:16.480085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341883310945093:8478], now have 1 active actors on pipe
2025-10-09T21:47:16.480094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341883310944878:2432], now have 1 active actors on pipe
2025-10-09T21:47:16.480097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341882949125836:7663]
2025-10-09T21:47:16.480105Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:16.480142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:16.480169Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 894
2025-10-09T21:47:16.480170Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 894
2025-10-09T21:47:16.480255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:16.480269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:16.480303Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|80e83494-2e2697a0-17e18961-55743f85_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:16.480319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:16.480323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:16.480326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:16.480337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:16.480347Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:16.480365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:16.480569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:16.480596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:16.480620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:16.480759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 895 partition: 0 MaxSeqNo: 25 sessionId: src_id|80e83494-2e2697a0-17e18961-55743f85_0
2025-10-09T21:47:16.481140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efdfa375-b9c07a5e-619b5109-258ed42d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|80e83494-2e2697a0-17e18961-55743f85_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:16.483871Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80e83494-2e2697a0-17e18961-55743f85_0] Write session: try to update token
2025-10-09T21:47:16.483892Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80e83494-2e2697a0-17e18961-55743f85_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:16.494834Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:16.494855Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:16.494857Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:16.494860Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:16.494870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341882949125836:7663]
2025-10-09T21:47:16.494874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341882949125836:7663]
2025-10-09T21:47:16.494878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341882949125836:7663]
2025-10-09T21:47:16.494925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:16.494933Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:16.494939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.494941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:16.494942Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:16.494946Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:16.495097Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:16.500169Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 895 sessionId: src_id|80e83494-2e2697a0-17e18961-55743f85_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:16.502830Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:16.549129Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341883310945105:7899]
2025-10-09T21:47:16.549498Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 894, pipe: [1:7559341883310945105:7899]
2025-10-09T21:47:16.654774Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:16.655209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:16.655231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:16.655282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341883310945105:7899], now have 1 active actors on pipe
2025-10-09T21:47:16.655562Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:16.655577Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:16.655600Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341883310945105:7899]
2025-10-09T21:47:16.655627Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:16.655699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 895 reason: error at write: Disconnected sessionId: src_id|80e83494-2e2697a0-17e18961-55743f85_0
2025-10-09T21:47:16.655764Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:16.655898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 895 sessionId: src_id|80e83494-2e2697a0-17e18961-55743f85_0 is DEAD
2025-10-09T21:47:16.656070Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80e83494-2e2697a0-17e18961-55743f85_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:16.656082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80e83494-2e2697a0-17e18961-55743f85_0] Write session will restart in 0.005903s
2025-10-09T21:47:16.656110Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80e83494-2e2697a0-17e18961-55743f85_0] Write session: Do CDS request
2025-10-09T21:47:16.656117Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80e83494-2e2697a0-17e18961-55743f85_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:16.663563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80e83494-2e2697a0-17e18961-55743f85_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:16.663667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:16.663684Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 896
2025-10-09T21:47:16.663841Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 896 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:16.663879Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 896 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:16.663887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=896 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:16.663891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:16.664421Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 896 sessionId:  describe result for acl check
2025-10-09T21:47:16.664461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:16.664466Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:16.664469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:16.664476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341883310945125:8482] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:16.665080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341883310945125:8482] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:16.668548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341883310945125:8482] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:16.674735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341883310945125:8482] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:16.674757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341883310945125:8482] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:16.674761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341883310945125:8482] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:16.674771Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 896 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:16.676425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:16.678780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:16.679030Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:16.679042Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:16.679055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:16.679117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:16.679180Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341882949126064:7667]
2025-10-09T21:47:16.679781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:16.680209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:16.680287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:16.680661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:16.680716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.680725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.680726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.680727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.680729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.680730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.680731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.680732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.680734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.680740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.680750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.680755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.680759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.680764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.680768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.680773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.680777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.680782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.680790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.680799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.680821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.680827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.680831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.680839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.680849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.680855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.680860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.680866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.680871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.680878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.680884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.680895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.680900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.680909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.680914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.680920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.680955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:16.687711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:16.689152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:16.689166Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:16.689169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:16.689174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:16.689183Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 895 [2:7559341882949126064:7667]
2025-10-09T21:47:16.689202Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:16.689226Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:16.689235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:16.689238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:16.689241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:16.689262Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:16.689272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:16.689280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:16.689349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.689357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.689359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.689363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:16.689444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:16.689451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.689463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.689482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.689497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:16.689519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341882949126059:7667]
2025-10-09T21:47:16.689527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341882949126059:7667]
2025-10-09T21:47:16.689529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341882949126059:7667]
2025-10-09T21:47:16.689545Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:16.689565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341883310945322:8482], now have 1 active actors on pipe
2025-10-09T21:47:16.689567Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:16.689570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:16.689572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:16.689573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341883310945108:2432], now have 1 active actors on pipe
2025-10-09T21:47:16.689582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:16.689596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:16.689599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:16.689645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341882949126059:7667]
2025-10-09T21:47:16.689628Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 895
2025-10-09T21:47:16.689652Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:16.689670Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:16.689637Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 895
2025-10-09T21:47:16.689686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:16.689699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:16.689739Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1f3d1786-65afd537-452fd665-1d0be5d1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:16.689752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:16.689757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:16.689760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:16.689771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:16.689785Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:16.689814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:16.690158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:16.690166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:16.690196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:16.690295Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 896 partition: 0 MaxSeqNo: 25 sessionId: src_id|1f3d1786-65afd537-452fd665-1d0be5d1_0
2025-10-09T21:47:16.690752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80e83494-2e2697a0-17e18961-55743f85_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1f3d1786-65afd537-452fd665-1d0be5d1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:16.693235Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1f3d1786-65afd537-452fd665-1d0be5d1_0] Write session: try to update token
2025-10-09T21:47:16.693252Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1f3d1786-65afd537-452fd665-1d0be5d1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:16.705108Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:16.705133Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:16.710049Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 896 sessionId: src_id|1f3d1786-65afd537-452fd665-1d0be5d1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:16.705136Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:16.705138Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:16.705149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341882949126059:7667]
2025-10-09T21:47:16.705153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341882949126059:7667]
2025-10-09T21:47:16.705158Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341882949126059:7667]
2025-10-09T21:47:16.705210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:16.705217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.705220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:16.705237Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:16.705244Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:16.705248Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:16.705376Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:16.712860Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:16.755954Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341883310945339:7899]
2025-10-09T21:47:16.756296Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 895, pipe: [1:7559341883310945339:7899]
2025-10-09T21:47:16.868710Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:16.869101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:16.869122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:16.869168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341883310945339:7899], now have 1 active actors on pipe
2025-10-09T21:47:16.869371Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:16.869447Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341883310945339:7899]
2025-10-09T21:47:16.870020Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1f3d1786-65afd537-452fd665-1d0be5d1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:16.870056Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1f3d1786-65afd537-452fd665-1d0be5d1_0] Write session will restart in 0.006231s
2025-10-09T21:47:16.869489Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:16.869590Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 896 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1f3d1786-65afd537-452fd665-1d0be5d1_0
2025-10-09T21:47:16.870087Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1f3d1786-65afd537-452fd665-1d0be5d1_0] Write session: Do CDS request
2025-10-09T21:47:16.869598Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:16.869793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 896 sessionId: src_id|1f3d1786-65afd537-452fd665-1d0be5d1_0 is DEAD
2025-10-09T21:47:16.870095Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1f3d1786-65afd537-452fd665-1d0be5d1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:16.877630Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1f3d1786-65afd537-452fd665-1d0be5d1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:16.877729Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:16.877759Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 897
2025-10-09T21:47:16.877936Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 897 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:16.877988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 897 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:16.878001Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=897 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:16.878005Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:16.878539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 897 sessionId:  describe result for acl check
2025-10-09T21:47:16.878587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:16.878597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:16.878599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:16.878606Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341883310945370:8489] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:16.879245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341883310945370:8489] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:16.882879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341883310945370:8489] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:16.886966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341883310945370:8489] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:16.886983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341883310945370:8489] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:16.886987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341883310945370:8489] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:16.886996Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 897 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:16.888196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:16.890562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:16.890847Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:16.890862Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:16.890878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:16.890957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:16.891000Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341882949126297:7675]
2025-10-09T21:47:16.891611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:16.892022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:16.892083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:16.892456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:16.892522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.892530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.892531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.892532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.892534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.892535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.892536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.892538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.892539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.892545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.892556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.892560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.892565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.892569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.892573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.892577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.892582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.892586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.892594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.892603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:16.892610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.892616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:16.892628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.892639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:16.892644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.892662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:16.892672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.892678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:16.892682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.892688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:16.892693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.892699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.892709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.892714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.892724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.892729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.892762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:16.899989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:16.901433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:16.901452Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:16.901454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:16.901460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:16.901467Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 896 [2:7559341882949126297:7675]
2025-10-09T21:47:16.901483Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:16.901502Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:16.901505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:16.901509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:16.901512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:16.901532Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:16.901543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:16.901550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:16.901616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.901624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.901626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.901629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:16.901703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:16.901714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.901726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:16.901737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:16.901752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:16.901915Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 896
2025-10-09T21:47:16.901963Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 896
2025-10-09T21:47:16.901775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341882949126292:7675]
2025-10-09T21:47:16.901782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341882949126292:7675]
2025-10-09T21:47:16.901784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341882949126292:7675]
2025-10-09T21:47:16.901798Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:16.901809Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:16.901811Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:16.901813Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:16.901815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341883310945558:8489], now have 1 active actors on pipe
2025-10-09T21:47:16.901818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341883310945344:2432], now have 1 active actors on pipe
2025-10-09T21:47:16.901826Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:16.901842Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:16.901845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:16.901898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341882949126292:7675]
2025-10-09T21:47:16.901904Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:16.901922Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:16.901946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:16.901951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:16.901980Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9bd2fbc7-c3ebad73-bd4660ae-c68e49fe_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:16.901989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:16.901992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:16.901995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:16.902007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:16.902018Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:16.902058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:16.902253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:16.902265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:16.902281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:16.902386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 897 partition: 0 MaxSeqNo: 25 sessionId: src_id|9bd2fbc7-c3ebad73-bd4660ae-c68e49fe_0
2025-10-09T21:47:16.902766Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1f3d1786-65afd537-452fd665-1d0be5d1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9bd2fbc7-c3ebad73-bd4660ae-c68e49fe_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:16.905436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9bd2fbc7-c3ebad73-bd4660ae-c68e49fe_0] Write session: try to update token
2025-10-09T21:47:16.905468Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9bd2fbc7-c3ebad73-bd4660ae-c68e49fe_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:16.917257Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:16.917280Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:16.917283Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:16.917285Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:16.917296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341882949126292:7675]
2025-10-09T21:47:16.917301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341882949126292:7675]
2025-10-09T21:47:16.917304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341882949126292:7675]
2025-10-09T21:47:16.917352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:16.917359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:16.917362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:16.917375Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:16.917390Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:16.917398Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:16.917499Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:16.921805Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 897 sessionId: src_id|9bd2fbc7-c3ebad73-bd4660ae-c68e49fe_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:16.924612Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:16.969445Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341883310945574:7899]
2025-10-09T21:47:16.969969Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 896, pipe: [1:7559341883310945574:7899]
2025-10-09T21:47:17.087246Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:17.087828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:17.087852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:17.087908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341883310945574:7899], now have 1 active actors on pipe
2025-10-09T21:47:17.088647Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:17.090167Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:17.090269Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 897 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9bd2fbc7-c3ebad73-bd4660ae-c68e49fe_0
2025-10-09T21:47:17.090475Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 897 sessionId: src_id|9bd2fbc7-c3ebad73-bd4660ae-c68e49fe_0 is DEAD
2025-10-09T21:47:17.090541Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341883310945574:7899]
2025-10-09T21:47:17.090574Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:17.090633Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9bd2fbc7-c3ebad73-bd4660ae-c68e49fe_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:17.090644Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9bd2fbc7-c3ebad73-bd4660ae-c68e49fe_0] Write session will restart in 0.007758s
2025-10-09T21:47:17.090675Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9bd2fbc7-c3ebad73-bd4660ae-c68e49fe_0] Write session: Do CDS request
2025-10-09T21:47:17.090682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9bd2fbc7-c3ebad73-bd4660ae-c68e49fe_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:17.099682Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9bd2fbc7-c3ebad73-bd4660ae-c68e49fe_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:17.099790Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:17.099809Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 898
2025-10-09T21:47:17.099919Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 898 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:17.099966Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 898 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:17.099973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=898 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:17.099977Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:17.100903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 898 sessionId:  describe result for acl check
2025-10-09T21:47:17.100951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:17.100957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:17.100959Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:17.100966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341887605912940:8500] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:17.101530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341887605912940:8500] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:17.104674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341887605912940:8500] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:17.108568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341887605912940:8500] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:17.108593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341887605912940:8500] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:17.108596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341887605912940:8500] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:17.108607Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 898 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:17.110990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:17.113449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:17.113727Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:17.113738Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:17.113752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:17.113814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:17.113868Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341887244093840:7683]
2025-10-09T21:47:17.114475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:17.114898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:17.114974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:17.115613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:17.115702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.115709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.115710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.115712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.115713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.115714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.115715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.115716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.115718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.115723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.115734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.115739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.115743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.115747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.115752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.115756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.115760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.115763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.115772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.115780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.115787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.115792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.115797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.115803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.115808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.115814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.115818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.115824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.115829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.115834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.115838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.115843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.115848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.115853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.115858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.115871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.115904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:17.123619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:17.124736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:17.124757Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:17.124761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:17.124766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:17.124775Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 897 [2:7559341887244093840:7683]
2025-10-09T21:47:17.124790Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:17.124813Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:17.124817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:17.124821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:17.124823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:17.124845Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:17.124852Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:17.124860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:17.124936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.124939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.124941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.124944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:17.125036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:17.125040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.125054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.125065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.125082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:17.125104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341887244093835:7683]
2025-10-09T21:47:17.125107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341887244093835:7683]
2025-10-09T21:47:17.125109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341887244093835:7683]
2025-10-09T21:47:17.125118Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:17.125135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:17.125144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:17.125148Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:17.125150Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:17.125151Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:17.125155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:17.125168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341887605913108:8500], now have 1 active actors on pipe
2025-10-09T21:47:17.125172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341887605912897:2432], now have 1 active actors on pipe
2025-10-09T21:47:17.125213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341887244093835:7683]
2025-10-09T21:47:17.125223Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:17.125238Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:17.125751Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 897
2025-10-09T21:47:17.125796Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 897
2025-10-09T21:47:17.126567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:17.126588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:17.126628Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|950a8c52-3e8dd7b-f044c69f-d64339c5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:17.126640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:17.126645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:17.126648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:17.126661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:17.126689Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:17.126715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:17.127211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:17.127228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:17.127255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:17.127413Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 898 partition: 0 MaxSeqNo: 25 sessionId: src_id|950a8c52-3e8dd7b-f044c69f-d64339c5_0
2025-10-09T21:47:17.127843Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9bd2fbc7-c3ebad73-bd4660ae-c68e49fe_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|950a8c52-3e8dd7b-f044c69f-d64339c5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:17.130673Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|950a8c52-3e8dd7b-f044c69f-d64339c5_0] Write session: try to update token
2025-10-09T21:47:17.130694Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|950a8c52-3e8dd7b-f044c69f-d64339c5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:17.145123Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:17.145141Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:17.145142Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:17.145144Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:17.145153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341887244093835:7683]
2025-10-09T21:47:17.145157Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341887244093835:7683]
2025-10-09T21:47:17.145160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341887244093835:7683]
2025-10-09T21:47:17.145204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:17.145210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.145214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:17.145353Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:17.148969Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:17.148987Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:17.148991Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:17.152242Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 898 sessionId: src_id|950a8c52-3e8dd7b-f044c69f-d64339c5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:17.155156Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:17.190799Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341887605913133:7899]
2025-10-09T21:47:17.191147Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 897, pipe: [1:7559341887605913133:7899]
2025-10-09T21:47:17.307071Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:17.307522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:17.307533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:17.307577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341887605913133:7899], now have 1 active actors on pipe
2025-10-09T21:47:17.307954Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341887605913133:7899]
2025-10-09T21:47:17.307975Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:17.307980Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:17.308004Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:17.308049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 898 reason: error at write: Disconnected sessionId: src_id|950a8c52-3e8dd7b-f044c69f-d64339c5_0
2025-10-09T21:47:17.307923Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:17.308424Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|950a8c52-3e8dd7b-f044c69f-d64339c5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:17.308440Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|950a8c52-3e8dd7b-f044c69f-d64339c5_0] Write session will restart in 0.005856s
2025-10-09T21:47:17.308467Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|950a8c52-3e8dd7b-f044c69f-d64339c5_0] Write session: Do CDS request
2025-10-09T21:47:17.308476Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|950a8c52-3e8dd7b-f044c69f-d64339c5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:17.308245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 898 sessionId: src_id|950a8c52-3e8dd7b-f044c69f-d64339c5_0 is DEAD
2025-10-09T21:47:17.315648Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|950a8c52-3e8dd7b-f044c69f-d64339c5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:17.315760Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:17.315777Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 899
2025-10-09T21:47:17.315982Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 899 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:17.316039Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 899 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:17.316049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=899 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:17.316053Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:17.316575Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 899 sessionId:  describe result for acl check
2025-10-09T21:47:17.316618Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:17.316620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:17.316622Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:17.316629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341887605913151:8505] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:17.317212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341887605913151:8505] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:17.321058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341887605913151:8505] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:17.325383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341887605913151:8505] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:17.325404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341887605913151:8505] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:17.325408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341887605913151:8505] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:17.325419Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 899 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:17.331219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:17.335085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:17.335362Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:17.335367Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:17.335381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:17.335453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:17.335493Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341887244094062:7685]
2025-10-09T21:47:17.336089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:17.336808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:17.336869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:17.337236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:17.337590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.337592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.337593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.337595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.337596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.337598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.337599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.337600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.337602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.337609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.337622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.337626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.337631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.337635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.337638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.337642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.337646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.337651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.337661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.337670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.337679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.337685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.337689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.337694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.337699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.337704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.337710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.337716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.337720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.337726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.337731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.337736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.337742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.337747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.337751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.337757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.337788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:17.345954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:17.347099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:17.347127Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:17.347130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:17.347136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:17.347144Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 898 [2:7559341887244094062:7685]
2025-10-09T21:47:17.347160Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:17.347180Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:17.347183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:17.347187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:17.347190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:17.347212Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:17.347218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:17.347227Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:17.347292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.347298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.347301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.347304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:17.347385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:17.347394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.347409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.347421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.347438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:17.347459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341887244094057:7685]
2025-10-09T21:47:17.347462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341887244094057:7685]
2025-10-09T21:47:17.347464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341887244094057:7685]
2025-10-09T21:47:17.347473Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:17.347490Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:17.347493Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:17.347494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341887605913289:8505], now have 1 active actors on pipe
2025-10-09T21:47:17.347496Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:17.347498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341887605913136:2432], now have 1 active actors on pipe
2025-10-09T21:47:17.347504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:17.347518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:17.347522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:17.347566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341887244094057:7685]
2025-10-09T21:47:17.347578Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:17.347571Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 898
2025-10-09T21:47:17.347622Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 898
2025-10-09T21:47:17.347614Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:17.347791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:17.347798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:17.347834Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6647f5cc-54394acc-2c8ec6f5-8f484d63_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:17.347844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:17.347849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:17.347852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:17.347872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:17.347891Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:17.347915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:17.348179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:17.348195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:17.348217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:17.348312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 899 partition: 0 MaxSeqNo: 25 sessionId: src_id|6647f5cc-54394acc-2c8ec6f5-8f484d63_0
2025-10-09T21:47:17.348728Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|950a8c52-3e8dd7b-f044c69f-d64339c5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6647f5cc-54394acc-2c8ec6f5-8f484d63_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:17.351233Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6647f5cc-54394acc-2c8ec6f5-8f484d63_0] Write session: try to update token
2025-10-09T21:47:17.351256Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6647f5cc-54394acc-2c8ec6f5-8f484d63_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:17.359824Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:17.359846Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:17.359857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341887244094057:7685]
2025-10-09T21:47:17.359909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:17.360093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:17.360240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:17.360387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:17.360523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:17.360714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:17.360762Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:17.360882Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:17.371055Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 899 sessionId: src_id|6647f5cc-54394acc-2c8ec6f5-8f484d63_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:17.373493Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:17.374130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:17.374156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:17.374180Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:17.374183Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:17.374186Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:17.374188Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:17.374199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341887244094057:7685]
2025-10-09T21:47:17.374202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341887244094057:7685]
2025-10-09T21:47:17.374206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341887244094057:7685]
2025-10-09T21:47:17.374267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:17.374283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.374285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:17.374416Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:17.378382Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:17.378400Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:17.378404Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:17.407737Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341887605913367:7899]
2025-10-09T21:47:17.408105Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 898, pipe: [1:7559341887605913367:7899]
2025-10-09T21:47:17.533118Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:17.533699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:17.533749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341887605913367:7899], now have 1 active actors on pipe
2025-10-09T21:47:17.533917Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:17.533957Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341887605913367:7899]
2025-10-09T21:47:17.533979Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:17.534461Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6647f5cc-54394acc-2c8ec6f5-8f484d63_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:17.534473Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6647f5cc-54394acc-2c8ec6f5-8f484d63_0] Write session will restart in 0.006442s
2025-10-09T21:47:17.534498Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6647f5cc-54394acc-2c8ec6f5-8f484d63_0] Write session: Do CDS request
2025-10-09T21:47:17.534505Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6647f5cc-54394acc-2c8ec6f5-8f484d63_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:17.534055Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:17.534064Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:17.534157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 899 reason: error at write: Disconnected sessionId: src_id|6647f5cc-54394acc-2c8ec6f5-8f484d63_0
2025-10-09T21:47:17.534320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 899 sessionId: src_id|6647f5cc-54394acc-2c8ec6f5-8f484d63_0 is DEAD
2025-10-09T21:47:17.546444Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:17.546463Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 900
2025-10-09T21:47:17.549041Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6647f5cc-54394acc-2c8ec6f5-8f484d63_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:17.549867Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 900 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:17.549953Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 900 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:17.549968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=900 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:17.549973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:17.550638Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 900 sessionId:  describe result for acl check
2025-10-09T21:47:17.550687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:17.550695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:17.550697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:17.550704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341887605913478:8509] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:17.551308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341887605913478:8509] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:17.555207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341887605913478:8509] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:17.558067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:17.558601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341887605913478:8509] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:17.558620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341887605913478:8509] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:17.558625Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341887605913478:8509] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:17.558634Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 900 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:17.561021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:17.561315Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:17.561327Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:17.561340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:17.561412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:17.561467Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341887244094285:7687]
2025-10-09T21:47:17.562135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:17.562580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:17.562645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:17.563054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:17.563129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.563135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.563137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.563138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.563139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.563140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.563142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.563143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.563144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.563152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.563165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.563170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.563180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.563184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.563189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.563193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.563197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.563201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.563211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.563228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.563239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.563250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.563255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.563261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.563264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.563270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.563275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.563280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.563295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.563305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.563311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.563321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.563326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.563335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.563341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.563352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.563388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:17.570718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:17.572806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:17.572825Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:17.572828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:17.572833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:17.572842Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 899 [2:7559341887244094285:7687]
2025-10-09T21:47:17.572857Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:17.572878Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:17.572887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:17.572891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:17.572894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:17.572916Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:17.572927Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:17.572935Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:17.573006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.573014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.573016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.573020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:17.573255Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 899
2025-10-09T21:47:17.573262Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 899
2025-10-09T21:47:17.573415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:17.573424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.573438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.573450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.573467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:17.573496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341887605913591:8509], now have 1 active actors on pipe
2025-10-09T21:47:17.573505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341887605913370:2432], now have 1 active actors on pipe
2025-10-09T21:47:17.573530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:17.573545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:17.573564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341887244094279:7687]
2025-10-09T21:47:17.573566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341887244094279:7687]
2025-10-09T21:47:17.573568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341887244094279:7687]
2025-10-09T21:47:17.573581Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:17.573597Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:17.573603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:17.573608Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:17.573611Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:17.573617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:17.573620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:17.573641Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7247e4b4-cbc92417-70cc52d8-d5b37944_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:17.573654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:17.573663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:17.573666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:17.573677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:17.573691Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:17.573736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341887244094279:7687]
2025-10-09T21:47:17.573750Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:17.573756Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:17.573766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:17.573958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:17.573966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:17.573981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:17.574089Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 900 partition: 0 MaxSeqNo: 25 sessionId: src_id|7247e4b4-cbc92417-70cc52d8-d5b37944_0
2025-10-09T21:47:17.574570Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6647f5cc-54394acc-2c8ec6f5-8f484d63_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7247e4b4-cbc92417-70cc52d8-d5b37944_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:17.582880Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7247e4b4-cbc92417-70cc52d8-d5b37944_0] Write session: try to update token
2025-10-09T21:47:17.582906Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7247e4b4-cbc92417-70cc52d8-d5b37944_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:17.589060Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:17.589081Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:17.589084Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:17.589086Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:17.589099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341887244094279:7687]
2025-10-09T21:47:17.589105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341887244094279:7687]
2025-10-09T21:47:17.589109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341887244094279:7687]
2025-10-09T21:47:17.589164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:17.589171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.589174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:17.589546Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:17.594113Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:17.594136Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:17.594142Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:17.608802Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 900 sessionId: src_id|7247e4b4-cbc92417-70cc52d8-d5b37944_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:17.611448Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:17.634021Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341887605913603:7899]
2025-10-09T21:47:17.634354Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 899, pipe: [1:7559341887605913603:7899]
2025-10-09T21:47:17.751419Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:17.751986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:17.752019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:17.752044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341887605913603:7899], now have 1 active actors on pipe
2025-10-09T21:47:17.752273Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:17.752306Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:17.752320Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:17.752394Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 900 reason: error at write: Disconnected sessionId: src_id|7247e4b4-cbc92417-70cc52d8-d5b37944_0
2025-10-09T21:47:17.752403Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:17.752451Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341887605913603:7899]
2025-10-09T21:47:17.752593Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 900 sessionId: src_id|7247e4b4-cbc92417-70cc52d8-d5b37944_0 is DEAD
2025-10-09T21:47:17.752754Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7247e4b4-cbc92417-70cc52d8-d5b37944_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:17.752766Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7247e4b4-cbc92417-70cc52d8-d5b37944_0] Write session will restart in 0.008261s
2025-10-09T21:47:17.752795Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7247e4b4-cbc92417-70cc52d8-d5b37944_0] Write session: Do CDS request
2025-10-09T21:47:17.752801Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7247e4b4-cbc92417-70cc52d8-d5b37944_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:17.762632Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7247e4b4-cbc92417-70cc52d8-d5b37944_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:17.762805Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:17.762824Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 901
2025-10-09T21:47:17.763005Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 901 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:17.763052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 901 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:17.763063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=901 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:17.763068Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:17.763693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 901 sessionId:  describe result for acl check
2025-10-09T21:47:17.763740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:17.763749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:17.763751Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:17.763757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341887605913658:8513] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:17.764442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341887605913658:8513] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:17.767407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341887605913658:8513] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:17.770851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341887605913658:8513] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:17.770876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341887605913658:8513] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:17.770880Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341887605913658:8513] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:17.770889Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 901 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:17.771809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:17.774194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:17.774467Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:17.774480Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:17.774493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:17.774553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:17.774594Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341887244094514:7691]
2025-10-09T21:47:17.775226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:17.775661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:17.775727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:17.776117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:17.776198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.776205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.776207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.776208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.776209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.776210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.776211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.776212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.776213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.776220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.776233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.776237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.776241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.776245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.776249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.776253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.776257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.776260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.776269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.776277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.776285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.776291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.776296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.776301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.776305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.776310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.776315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.776326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.776332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.776337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.776348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.776353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.776358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.776363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.776369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.776374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.776403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:17.783524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:17.785166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:17.785186Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:17.785189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:17.785193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:17.785201Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 900 [2:7559341887244094514:7691]
2025-10-09T21:47:17.785220Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:17.785243Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:17.785246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:17.785251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:17.785259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:17.785280Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:17.785291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:17.785303Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:17.785377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.785385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.785387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.785389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:17.785477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:17.785485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.785500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.785512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.785528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:17.785553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341887605913825:8513], now have 1 active actors on pipe
2025-10-09T21:47:17.785561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341887605913605:2432], now have 1 active actors on pipe
2025-10-09T21:47:17.785569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341887244094509:7691]
2025-10-09T21:47:17.785572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341887244094509:7691]
2025-10-09T21:47:17.785574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341887244094509:7691]
2025-10-09T21:47:17.785590Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:17.785607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:17.785624Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:17.785624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:17.785627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:17.785628Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:17.785631Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:17.785678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341887244094509:7691]
2025-10-09T21:47:17.785688Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:17.785691Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:17.785656Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 900
2025-10-09T21:47:17.785693Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 900
2025-10-09T21:47:17.785720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:17.785734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:17.785772Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a11b2ddc-7c1a3957-8789c87e-37440ca7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:17.785784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:17.785790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:17.785793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:17.785804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:17.785824Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:17.785845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:17.786160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:17.786176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:17.786197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:17.786318Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 901 partition: 0 MaxSeqNo: 25 sessionId: src_id|a11b2ddc-7c1a3957-8789c87e-37440ca7_0
2025-10-09T21:47:17.786742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7247e4b4-cbc92417-70cc52d8-d5b37944_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a11b2ddc-7c1a3957-8789c87e-37440ca7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:17.789464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a11b2ddc-7c1a3957-8789c87e-37440ca7_0] Write session: try to update token
2025-10-09T21:47:17.789489Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a11b2ddc-7c1a3957-8789c87e-37440ca7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:17.801343Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:17.801368Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:17.801371Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:17.801374Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:17.801385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341887244094509:7691]
2025-10-09T21:47:17.801390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341887244094509:7691]
2025-10-09T21:47:17.801396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341887244094509:7691]
2025-10-09T21:47:17.801463Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:17.801468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:17.801470Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:17.801474Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:17.801475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.801478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:17.801634Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:17.808964Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 901 sessionId: src_id|a11b2ddc-7c1a3957-8789c87e-37440ca7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:17.811685Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:17.852726Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341887605913841:7899]
2025-10-09T21:47:17.853128Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 900, pipe: [1:7559341887605913841:7899]
2025-10-09T21:47:17.961677Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:17.961695Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:17.961767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 901 reason: error at write: Disconnected sessionId: src_id|a11b2ddc-7c1a3957-8789c87e-37440ca7_0
2025-10-09T21:47:17.961870Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:17.961946Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341887605913841:7899]
2025-10-09T21:47:17.961971Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:47:17.961989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 901 sessionId: src_id|a11b2ddc-7c1a3957-8789c87e-37440ca7_0 is DEAD
2025-10-09T21:47:17.960717Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:17.961246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:17.961257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:17.961305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341887605913841:7899], now have 1 active actors on pipe
2025-10-09T21:47:17.961638Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:17.962130Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 9 consumer shared/user session shared/user_1_9_1758451654357684628_v1 is DEAD
2025-10-09T21:47:17.962260Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a11b2ddc-7c1a3957-8789c87e-37440ca7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:17.962268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a11b2ddc-7c1a3957-8789c87e-37440ca7_0] Write session will restart in 0.007605s
2025-10-09T21:47:17.962309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a11b2ddc-7c1a3957-8789c87e-37440ca7_0] Write session: Do CDS request
2025-10-09T21:47:17.962316Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a11b2ddc-7c1a3957-8789c87e-37440ca7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:17.962547Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341784526679193:7895] disconnected.
2025-10-09T21:47:17.962560Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341784526679193:7895] disconnected; active server actors: 1
2025-10-09T21:47:17.962564Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341784526679193:7895] client user disconnected session shared/user_1_9_1758451654357684628_v1
2025-10-09T21:47:17.964394Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:47:17.964576Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:47:17.964582Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:47:17.964592Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.008886s
2025-10-09T21:47:17.971644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a11b2ddc-7c1a3957-8789c87e-37440ca7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:17.971738Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:17.971766Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 902
2025-10-09T21:47:17.971944Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 902 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:17.971979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 902 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:17.971985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=902 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:17.971988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:17.972615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 902 sessionId:  describe result for acl check
2025-10-09T21:47:17.972651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:17.972653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:17.972656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:17.972662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341887605913907:8518] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:17.973218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341887605913907:8518] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:17.977398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341887605913907:8518] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:17.978401Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:47:17.978642Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:47:17.978653Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 10
2025-10-09T21:47:17.978916Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 10 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:47:17.978969Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:47:17.979073Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 auth for : user
2025-10-09T21:47:17.979716Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 Handle describe topics response
2025-10-09T21:47:17.979744Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 auth is DEAD
2025-10-09T21:47:17.979760Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:47:17.980038Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:47:17.980157Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341887605914025:8522] connected; active server actors: 1
2025-10-09T21:47:17.980187Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_10_16382700389732983519_v1
2025-10-09T21:47:17.980171Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559341887605914025:8522] session shared/user_1_10_16382700389732983519_v1
2025-10-09T21:47:17.980180Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:47:17.980193Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:47:17.980199Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_10_16382700389732983519_v1" (Sender=[1:7559341887605913998:8522], Pipe=[1:7559341887605914025:8522], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:47:17.980203Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:47:17.980213Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:47:17.980228Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_10_16382700389732983519_v1" (Sender=[1:7559341887605913998:8522], Pipe=[1:7559341887605914025:8522], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:47:17.980237Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_10_16382700389732983519_v1" sender [1:7559341887605913998:8522] lock partition 0 for ReadingSession "shared/user_1_10_16382700389732983519_v1" (Sender=[1:7559341887605913998:8522], Pipe=[1:7559341887605914025:8522], Partitions=[], ActiveFamilyCount=1) generation 1 step 10
2025-10-09T21:47:17.980250Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:47:17.980253Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000031s
2025-10-09T21:47:17.980481Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 10 Session: "shared/user_1_10_16382700389732983519_v1" ClientId: "user" PipeClient { RawX1: 7559341887605914025 RawX2: 4503603922346314 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:47:17.980497Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:47:17.981862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341887605913907:8518] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:17.981887Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341887605913907:8518] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:17.981892Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341887605913907:8518] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:17.981900Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 902 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:17.983321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:17.985891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:17.986204Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:17.986216Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:17.986231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:17.986306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:17.986365Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341887244094770:7703]
2025-10-09T21:47:17.986965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:17.987402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:17.987486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:17.987840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:17.987927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.987933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.987934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.987935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.987937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.987938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.987939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.987941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.987942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.987949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.987958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.987962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.987965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.987968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.987972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.987975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.987978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.987981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.987990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.987997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:17.988003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.988008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:17.988012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.988017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:17.988022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.988027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:17.988031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.988036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:17.988040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.988045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:17.988050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.988055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.988058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.988063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.988068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.988073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.988099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:17.996050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:17.998205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:17.998231Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:17.998233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:17.998239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:17.998248Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 901 [2:7559341887244094770:7703]
2025-10-09T21:47:17.998266Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:17.998350Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:17.998353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:17.998357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:17.998359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:17.998381Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:17.998386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:17.998394Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:17.998488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.998490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.998492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.998495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:17.998709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:17.998711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:17.998728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:17.998737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:17.998751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:17.998775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341887605913847:2432], now have 1 active actors on pipe
2025-10-09T21:47:17.998778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341887605914071:8518], now have 1 active actors on pipe
2025-10-09T21:47:17.998781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341887605914040:8525], now have 1 active actors on pipe
2025-10-09T21:47:17.998789Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341887244094765:7703]
2025-10-09T21:47:17.998791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341887244094765:7703]
2025-10-09T21:47:17.998792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341887244094765:7703]
2025-10-09T21:47:17.998810Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:17.998828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:17.998839Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:17.998843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:17.998844Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:17.998845Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:17.998847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:17.998897Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341887244094765:7703]
2025-10-09T21:47:17.998906Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:17.998909Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:17.999098Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 901
2025-10-09T21:47:17.999159Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 901
2025-10-09T21:47:17.999193Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 901, pipe: [1:7559341887605914040:8525]
2025-10-09T21:47:17.999454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:17.999476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:17.999505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:17.999507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:17.999515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_10_16382700389732983519_v1 on pipe: [1:7559341887605914040:8525]
2025-10-09T21:47:17.999549Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f5feda24-502beddc-a52f6547-2fdeceeb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:17.999560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:17.999566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:17.999569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:17.999581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:17.999595Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:17.999610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:17.999624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:17.999626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:17.999649Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_10_16382700389732983519_v1
2025-10-09T21:47:17.999671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:17.999704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:17.999728Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:47:17.999896Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_10_16382700389732983519_v1:1 with generation 901
2025-10-09T21:47:17.999995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:18.000000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:18.000018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:18.000091Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 902 partition: 0 MaxSeqNo: 25 sessionId: src_id|f5feda24-502beddc-a52f6547-2fdeceeb_0
2025-10-09T21:47:18.000522Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a11b2ddc-7c1a3957-8789c87e-37440ca7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f5feda24-502beddc-a52f6547-2fdeceeb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:18.005265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:47:18.005312Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:47:18.005327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:18.005329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:18.005332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:18.005362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:47:18.005380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:47:18.005534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5feda24-502beddc-a52f6547-2fdeceeb_0] Write session: try to update token
2025-10-09T21:47:18.005558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5feda24-502beddc-a52f6547-2fdeceeb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:18.005602Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046437999 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:47:18.005651Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:47:18.005696Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 sending to client partition status
2025-10-09T21:47:18.019980Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:18.020006Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:18.020017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341887244094765:7703]
2025-10-09T21:47:18.020076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:18.020082Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:18.020267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:18.020438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:18.020596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:18.020760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:18.021108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:18.021164Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:18.023705Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 902 sessionId: src_id|f5feda24-502beddc-a52f6547-2fdeceeb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:18.026444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:18.026892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:18.026916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:18.026962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:18.027098Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:18.030430Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:18.030453Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:18.030455Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:18.030457Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:18.030468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341887244094765:7703]
2025-10-09T21:47:18.030473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341887244094765:7703]
2025-10-09T21:47:18.030477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341887244094765:7703]
2025-10-09T21:47:18.030524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:18.030536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.030539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:18.030583Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:18.030588Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:18.030602Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:18.030748Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:18.197294Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:18.205447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:18.205487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:18.205666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.205736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.205808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.205897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.206197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.206212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.206284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.206357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.206426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.206501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.206547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.206558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.206622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.206684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.206744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.206853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.206896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.206905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.207009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.207124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.207195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.207308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.207450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.207458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.207528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.207608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.207743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.207811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.207853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.207860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.207925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.207984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.208045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.208149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.208199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.208213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.208279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.208339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.208408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.208476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.208524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.208527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.208595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.208658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.208723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.208786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.208827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.208830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.208893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.208954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.209017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.209082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.209131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.209140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.209204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.209263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.209331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.209401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.209464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.209471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.209905Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:18.209983Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341887605914040:8525]
2025-10-09T21:47:18.210015Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:18.210017Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:18.210071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 902 reason: error at write: Disconnected sessionId: src_id|f5feda24-502beddc-a52f6547-2fdeceeb_0
2025-10-09T21:47:18.210284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 902 sessionId: src_id|f5feda24-502beddc-a52f6547-2fdeceeb_0 is DEAD
2025-10-09T21:47:18.210455Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5feda24-502beddc-a52f6547-2fdeceeb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:18.210468Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5feda24-502beddc-a52f6547-2fdeceeb_0] Write session will restart in 0.006581s
2025-10-09T21:47:18.210498Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5feda24-502beddc-a52f6547-2fdeceeb_0] Write session: Do CDS request
2025-10-09T21:47:18.210505Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5feda24-502beddc-a52f6547-2fdeceeb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:18.210530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_10_16382700389732983519_v1
2025-10-09T21:47:18.210549Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:18.210574Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_10_16382700389732983519_v1
2025-10-09T21:47:18.218807Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5feda24-502beddc-a52f6547-2fdeceeb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:18.218880Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:18.218913Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 903
2025-10-09T21:47:18.219094Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 903 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:18.219141Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 903 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:18.219155Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=903 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:18.219159Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:18.219685Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 903 sessionId:  describe result for acl check
2025-10-09T21:47:18.219729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:18.219735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:18.219737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:18.219758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341891900881433:8534] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:18.220354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341891900881433:8534] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:18.224695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341891900881433:8534] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:18.228172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341891900881433:8534] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:18.228194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341891900881433:8534] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:18.228198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341891900881433:8534] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:18.228209Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 903 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:18.230289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:18.232435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:18.232691Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:18.232708Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:18.232737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:18.232813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:18.232871Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341891539062302:7707]
2025-10-09T21:47:18.233489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:18.233862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:18.233927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:18.234280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:18.234356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:18.234364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:18.234366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:18.234367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:18.234368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:18.234369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:18.234370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.234372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.234373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.234378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:18.234390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:18.234394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:18.234398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:18.234402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:18.234407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:18.234410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.234414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.234417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.234425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:18.234434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:18.234440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:18.234446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:18.234450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:18.234455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:18.234460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:18.234465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:18.234470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:18.234476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:18.234481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:18.234486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:18.234492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.234497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.234501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.234506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.234511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.234516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.234543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:18.242662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:18.244160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:18.244185Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:18.244187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:18.244192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:18.244200Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 902 [2:7559341891539062302:7707]
2025-10-09T21:47:18.244215Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:18.244238Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:18.244241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:18.244244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:18.244247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:18.244268Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:18.244281Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:18.244289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:18.244354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.244363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.244365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.244368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:18.244448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:18.244457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.244471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.244486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.244507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:18.244529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341891539062297:7707]
2025-10-09T21:47:18.244531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341891539062297:7707]
2025-10-09T21:47:18.244533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341891539062297:7707]
2025-10-09T21:47:18.244544Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:18.244562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:18.244563Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:18.244566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:18.244568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:18.244576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:18.244579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:18.244591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341891900881417:2432], now have 1 active actors on pipe
2025-10-09T21:47:18.244595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341891900881634:8534], now have 1 active actors on pipe
2025-10-09T21:47:18.244796Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 902
2025-10-09T21:47:18.244828Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 902
2025-10-09T21:47:18.244639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341891539062297:7707]
2025-10-09T21:47:18.244654Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:18.244661Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:18.244863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:18.244873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:18.244907Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2f598f7d-c091073f-2fe8f07d-17b467a5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:18.244936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:18.244941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:18.244944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:18.244956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:18.244969Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:18.244988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:18.245175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:18.245187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:18.245366Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 903 partition: 0 MaxSeqNo: 25 sessionId: src_id|2f598f7d-c091073f-2fe8f07d-17b467a5_0
2025-10-09T21:47:18.245211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:18.245801Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5feda24-502beddc-a52f6547-2fdeceeb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2f598f7d-c091073f-2fe8f07d-17b467a5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:18.248477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2f598f7d-c091073f-2fe8f07d-17b467a5_0] Write session: try to update token
2025-10-09T21:47:18.248497Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2f598f7d-c091073f-2fe8f07d-17b467a5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:18.261635Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:18.261659Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:18.261662Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:18.261664Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:18.261674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341891539062297:7707]
2025-10-09T21:47:18.261678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341891539062297:7707]
2025-10-09T21:47:18.261681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341891539062297:7707]
2025-10-09T21:47:18.261728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:18.261728Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:18.261734Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:18.261738Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:18.261740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.261742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:18.261940Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:18.269533Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 903 sessionId: src_id|2f598f7d-c091073f-2fe8f07d-17b467a5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:18.272276Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:18.310390Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341891900881651:8525]
2025-10-09T21:47:18.310788Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 902, pipe: [1:7559341891900881651:8525]
2025-10-09T21:47:18.414388Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:18.414739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:18.414756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:18.414783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341891900881651:8525], now have 1 active actors on pipe
2025-10-09T21:47:18.415186Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:18.415203Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:18.415230Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341891900881651:8525]
2025-10-09T21:47:18.415255Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:18.415331Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 903 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2f598f7d-c091073f-2fe8f07d-17b467a5_0
2025-10-09T21:47:18.415529Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 903 sessionId: src_id|2f598f7d-c091073f-2fe8f07d-17b467a5_0 is DEAD
2025-10-09T21:47:18.415684Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f598f7d-c091073f-2fe8f07d-17b467a5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:18.415696Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f598f7d-c091073f-2fe8f07d-17b467a5_0] Write session will restart in 0.007522s
2025-10-09T21:47:18.415725Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f598f7d-c091073f-2fe8f07d-17b467a5_0] Write session: Do CDS request
2025-10-09T21:47:18.415731Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f598f7d-c091073f-2fe8f07d-17b467a5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:18.425640Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2f598f7d-c091073f-2fe8f07d-17b467a5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:18.425840Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:18.425856Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 904
2025-10-09T21:47:18.426008Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 904 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:18.426065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 904 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:18.426075Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=904 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:18.426079Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:18.426603Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 904 sessionId:  describe result for acl check
2025-10-09T21:47:18.426655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:18.426664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:18.426667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:18.426674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341891900881697:8538] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:18.427265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341891900881697:8538] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:18.430978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341891900881697:8538] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:18.434511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341891900881697:8538] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:18.434531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341891900881697:8538] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:18.434536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341891900881697:8538] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:18.434552Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 904 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:18.436890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:18.439384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:18.439651Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:18.439669Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:18.439683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:18.439773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:18.439827Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341891539062530:7709]
2025-10-09T21:47:18.440427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:18.440907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:18.440973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:18.441311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:18.441384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:18.441391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:18.441393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:18.441395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:18.441396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:18.441397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:18.441398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.441399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.441400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.441406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:18.441416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:18.441420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:18.441424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:18.441427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:18.441431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:18.441434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.441438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.441441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.441448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:18.441457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:18.441464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:18.441469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:18.441474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:18.441479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:18.441484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:18.441489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:18.441494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:18.441499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:18.441504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:18.441509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:18.441514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.441520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.441524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.441529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.441534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.441539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.441572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:18.449027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:18.450575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:18.450596Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:18.450599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:18.450603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:18.450610Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 903 [2:7559341891539062530:7709]
2025-10-09T21:47:18.450625Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:18.450645Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:18.450649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:18.450652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:18.450655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:18.450674Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:18.450682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:18.450695Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:18.450763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.450780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.450787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.450790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:18.450864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:18.450871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.450885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.450893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.450903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:18.450922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341891539062525:7709]
2025-10-09T21:47:18.450927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341891539062525:7709]
2025-10-09T21:47:18.450929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341891539062525:7709]
2025-10-09T21:47:18.450935Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:18.450948Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:18.450958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:18.450961Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:18.450963Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:18.450966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:18.450969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:18.450983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341891900881652:2432], now have 1 active actors on pipe
2025-10-09T21:47:18.450991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341891900881873:8538], now have 1 active actors on pipe
2025-10-09T21:47:18.451030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341891539062525:7709]
2025-10-09T21:47:18.451042Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:18.451049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:18.451084Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 903
2025-10-09T21:47:18.451086Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 903
2025-10-09T21:47:18.451122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:18.451134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:18.451159Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|782c0cba-6d2f2fb9-3a1ab79d-c6ee7652_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:18.451169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:18.451172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:18.451173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:18.451182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:18.451194Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:18.451214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:18.451401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:18.451421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:18.451437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:18.451590Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 904 partition: 0 MaxSeqNo: 25 sessionId: src_id|782c0cba-6d2f2fb9-3a1ab79d-c6ee7652_0
2025-10-09T21:47:18.452013Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f598f7d-c091073f-2fe8f07d-17b467a5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|782c0cba-6d2f2fb9-3a1ab79d-c6ee7652_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:18.454758Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|782c0cba-6d2f2fb9-3a1ab79d-c6ee7652_0] Write session: try to update token
2025-10-09T21:47:18.454781Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|782c0cba-6d2f2fb9-3a1ab79d-c6ee7652_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:18.463022Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:18.463040Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:18.463051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341891539062525:7709]
2025-10-09T21:47:18.463093Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:18.463099Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:18.463267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:18.463419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:18.463553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:18.463695Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:18.463894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:18.463941Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:18.469971Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 904 sessionId: src_id|782c0cba-6d2f2fb9-3a1ab79d-c6ee7652_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:18.472486Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:18.472691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:18.472715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:18.472759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:18.472853Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:18.477008Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:18.477028Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:18.477031Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:18.477033Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:18.477044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341891539062525:7709]
2025-10-09T21:47:18.477048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341891539062525:7709]
2025-10-09T21:47:18.477051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341891539062525:7709]
2025-10-09T21:47:18.477093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:18.477101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.477100Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:18.477104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:18.477106Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:18.477110Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:18.477243Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:18.515180Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341891900881889:8525]
2025-10-09T21:47:18.515473Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 903, pipe: [1:7559341891900881889:8525]
2025-10-09T21:47:18.633958Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:18.642137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:18.642171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:18.642343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.642409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.642482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.642552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.642602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.642612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.642678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.642744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.642812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.642885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.642929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.642941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.643006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.645446Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:18.645464Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341891900881889:8525]
2025-10-09T21:47:18.645506Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:18.645509Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:18.645540Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 904 reason: error at write: Disconnected sessionId: src_id|782c0cba-6d2f2fb9-3a1ab79d-c6ee7652_0
2025-10-09T21:47:18.645773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 904 sessionId: src_id|782c0cba-6d2f2fb9-3a1ab79d-c6ee7652_0 is DEAD
2025-10-09T21:47:18.645953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|782c0cba-6d2f2fb9-3a1ab79d-c6ee7652_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:18.645965Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|782c0cba-6d2f2fb9-3a1ab79d-c6ee7652_0] Write session will restart in 0.005348s
2025-10-09T21:47:18.645995Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|782c0cba-6d2f2fb9-3a1ab79d-c6ee7652_0] Write session: Do CDS request
2025-10-09T21:47:18.646001Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|782c0cba-6d2f2fb9-3a1ab79d-c6ee7652_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:18.643072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.643138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.643195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.643236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.643240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.643308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.643371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.643426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.643486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.643527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.643533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.643596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.643652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.643707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.643768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.643832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.643837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.643898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.643950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.644006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.644061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.644098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.644100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.644159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.644209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.644265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.644316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.644358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.644360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.644417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.644469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.644525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.644577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.644611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.644614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.644679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.644730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.644784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.644836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.644870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.644873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.644936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:18.644987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:18.645041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:18.645096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:18.645133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:18.645136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:18.645203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341891900881889:8525], now have 1 active actors on pipe
2025-10-09T21:47:18.646503Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:18.652582Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|782c0cba-6d2f2fb9-3a1ab79d-c6ee7652_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:18.652652Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:18.652666Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 905
2025-10-09T21:47:18.652879Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 905 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:18.652923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 905 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:18.652943Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=905 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:18.652947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:18.653397Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 905 sessionId:  describe result for acl check
2025-10-09T21:47:18.653435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:18.653443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:18.653445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:18.653450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341891900881911:8542] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:18.653989Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341891900881911:8542] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:18.657992Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341891900881911:8542] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:18.662320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341891900881911:8542] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:18.662336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341891900881911:8542] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:18.662340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341891900881911:8542] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:18.662354Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 905 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:18.664066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:18.666424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:18.666667Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:18.666678Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:18.666691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:18.666746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:18.666795Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341891539062762:7713]
2025-10-09T21:47:18.667382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:18.667682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:18.667723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:18.668027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:18.668069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:18.668070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:18.668071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:18.668072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:18.668073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:18.668075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:18.668076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.668077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.668078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.668083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:18.668093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:18.668097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:18.668100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:18.668103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:18.668107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:18.668110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.668114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.668117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.668125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:18.668148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:18.668154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:18.668159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:18.668163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:18.668168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:18.668171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:18.668176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:18.668180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:18.668185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:18.668188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:18.668193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:18.668197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.668202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.668206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.668211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.668215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.668220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.668246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:18.675085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:18.676470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:18.676492Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:18.676495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:18.676500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:18.676508Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 904 [2:7559341891539062762:7713]
2025-10-09T21:47:18.676526Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:18.676547Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:18.676550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:18.676554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:18.676556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:18.676576Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:18.676588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:18.676597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:18.676664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.676671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.676673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.676676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:18.676761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:18.676772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.676785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.676796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.676810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:18.677030Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 904
2025-10-09T21:47:18.677085Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 904
2025-10-09T21:47:18.676834Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341891539062757:7713]
2025-10-09T21:47:18.676844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341891539062757:7713]
2025-10-09T21:47:18.676846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341891539062757:7713]
2025-10-09T21:47:18.676856Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:18.676868Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:18.676871Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:18.676873Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:18.676877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:18.676892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:18.676895Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:18.676909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341891900881892:2432], now have 1 active actors on pipe
2025-10-09T21:47:18.676913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341891900882115:8542], now have 1 active actors on pipe
2025-10-09T21:47:18.676956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341891539062757:7713]
2025-10-09T21:47:18.676967Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:18.676986Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:18.677181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:18.677196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:18.677230Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fa8056ee-c80e6b1c-3f79a8f9-e33ca5f0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:18.677239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:18.677249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:18.677251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:18.677264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:18.677282Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:18.677300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:18.677512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:18.677528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:18.677551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:18.677686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 905 partition: 0 MaxSeqNo: 25 sessionId: src_id|fa8056ee-c80e6b1c-3f79a8f9-e33ca5f0_0
2025-10-09T21:47:18.678128Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|782c0cba-6d2f2fb9-3a1ab79d-c6ee7652_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fa8056ee-c80e6b1c-3f79a8f9-e33ca5f0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:18.680764Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fa8056ee-c80e6b1c-3f79a8f9-e33ca5f0_0] Write session: try to update token
2025-10-09T21:47:18.680785Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fa8056ee-c80e6b1c-3f79a8f9-e33ca5f0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:18.692573Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:18.692596Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:18.692599Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:18.692601Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:18.692611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341891539062757:7713]
2025-10-09T21:47:18.692615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341891539062757:7713]
2025-10-09T21:47:18.692620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341891539062757:7713]
2025-10-09T21:47:18.692675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:18.692684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.692692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:18.692720Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:18.692737Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:18.692741Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:18.692831Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:18.696815Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 905 sessionId: src_id|fa8056ee-c80e6b1c-3f79a8f9-e33ca5f0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:18.699495Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:18.745565Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341891900882129:8525]
2025-10-09T21:47:18.745860Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 904, pipe: [1:7559341891900882129:8525]
2025-10-09T21:47:18.854987Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:18.855501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:18.855522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:18.855568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341891900882129:8525], now have 1 active actors on pipe
2025-10-09T21:47:18.856078Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:18.855791Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341891900882129:8525]
2025-10-09T21:47:18.855825Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:18.855871Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:18.855874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:18.855938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 905 reason: error at write: Disconnected sessionId: src_id|fa8056ee-c80e6b1c-3f79a8f9-e33ca5f0_0
2025-10-09T21:47:18.856145Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 905 sessionId: src_id|fa8056ee-c80e6b1c-3f79a8f9-e33ca5f0_0 is DEAD
2025-10-09T21:47:18.856579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa8056ee-c80e6b1c-3f79a8f9-e33ca5f0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:18.856595Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa8056ee-c80e6b1c-3f79a8f9-e33ca5f0_0] Write session will restart in 0.005099s
2025-10-09T21:47:18.856624Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa8056ee-c80e6b1c-3f79a8f9-e33ca5f0_0] Write session: Do CDS request
2025-10-09T21:47:18.856631Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa8056ee-c80e6b1c-3f79a8f9-e33ca5f0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:18.862520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fa8056ee-c80e6b1c-3f79a8f9-e33ca5f0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:18.862600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:18.862616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 906
2025-10-09T21:47:18.862731Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 906 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:18.862765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 906 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:18.862772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=906 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:18.862775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:18.863258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 906 sessionId:  describe result for acl check
2025-10-09T21:47:18.863298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:18.863305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:18.863307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:18.863314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341891900882149:8547] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:18.863878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341891900882149:8547] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:18.866961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341891900882149:8547] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:18.870061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341891900882149:8547] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:18.870079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341891900882149:8547] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:18.870093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341891900882149:8547] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:18.870103Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 906 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:18.873838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:18.876393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:18.876650Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:18.876661Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:18.876674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:18.876747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:18.876814Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341891539062991:7715]
2025-10-09T21:47:18.877442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:18.877831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:18.877917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:18.878261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:18.878333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:18.878341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:18.878342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:18.878343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:18.878345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:18.878346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:18.878347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.878349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.878350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.878356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:18.878366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:18.878369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:18.878374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:18.878377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:18.878381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:18.878384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.878387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.878391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.878399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:18.878407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:18.878413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:18.878418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:18.878423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:18.878428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:18.878439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:18.878444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:18.878448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:18.878454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:18.878462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:18.878468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:18.878473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.878483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.878488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.878493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.878502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.878507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.878548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:18.886235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:18.887428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:18.887450Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:18.887453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:18.887458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:18.887466Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 905 [2:7559341891539062991:7715]
2025-10-09T21:47:18.887482Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:18.887506Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:18.887509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:18.887513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:18.887516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:18.887537Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:18.887553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:18.887562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:18.887635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.887643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.887645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.887649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:18.887731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:18.887741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.887754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:18.887765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:18.887781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:18.887811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341891900882134:2432], now have 1 active actors on pipe
2025-10-09T21:47:18.887820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341891900882315:8547], now have 1 active actors on pipe
2025-10-09T21:47:18.887829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341891539062986:7715]
2025-10-09T21:47:18.887832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341891539062986:7715]
2025-10-09T21:47:18.887834Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341891539062986:7715]
2025-10-09T21:47:18.887848Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:18.887865Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:18.887868Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:18.887870Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:18.887870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:18.887886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:18.887889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:18.887879Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 905
2025-10-09T21:47:18.887885Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 905
2025-10-09T21:47:18.887939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341891539062986:7715]
2025-10-09T21:47:18.887952Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:18.887970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:18.887975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:18.887986Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:18.888026Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f75d0402-7e505292-78f46f04-10bbc814_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:18.888039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:18.888044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:18.888052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:18.888064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:18.888080Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:18.888108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:18.888393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:18.888408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:18.888430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:18.888544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 906 partition: 0 MaxSeqNo: 25 sessionId: src_id|f75d0402-7e505292-78f46f04-10bbc814_0
2025-10-09T21:47:18.888986Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa8056ee-c80e6b1c-3f79a8f9-e33ca5f0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f75d0402-7e505292-78f46f04-10bbc814_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:18.891522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f75d0402-7e505292-78f46f04-10bbc814_0] Write session: try to update token
2025-10-09T21:47:18.891542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f75d0402-7e505292-78f46f04-10bbc814_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:18.903789Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:18.903824Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:18.903827Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:18.903829Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:18.903839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341891539062986:7715]
2025-10-09T21:47:18.903847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341891539062986:7715]
2025-10-09T21:47:18.903852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341891539062986:7715]
2025-10-09T21:47:18.903895Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:18.903896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:18.903904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:18.903907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:18.903908Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:18.903916Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:18.904046Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:18.907048Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 906 sessionId: src_id|f75d0402-7e505292-78f46f04-10bbc814_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:18.909757Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:18.956037Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341891900882373:8525]
2025-10-09T21:47:18.956379Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 905, pipe: [1:7559341891900882373:8525]
2025-10-09T21:47:19.070090Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:19.070640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:19.070660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:19.070710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341891900882373:8525], now have 1 active actors on pipe
2025-10-09T21:47:19.070961Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341891900882373:8525]
2025-10-09T21:47:19.070976Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:19.071004Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:19.071007Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:19.071009Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:19.071072Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 906 reason: error at write: Disconnected sessionId: src_id|f75d0402-7e505292-78f46f04-10bbc814_0
2025-10-09T21:47:19.071292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 906 sessionId: src_id|f75d0402-7e505292-78f46f04-10bbc814_0 is DEAD
2025-10-09T21:47:19.071525Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f75d0402-7e505292-78f46f04-10bbc814_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:19.071538Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f75d0402-7e505292-78f46f04-10bbc814_0] Write session will restart in 0.009825s
2025-10-09T21:47:19.071568Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f75d0402-7e505292-78f46f04-10bbc814_0] Write session: Do CDS request
2025-10-09T21:47:19.071575Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f75d0402-7e505292-78f46f04-10bbc814_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:19.082682Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f75d0402-7e505292-78f46f04-10bbc814_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:19.082836Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:19.082855Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 907
2025-10-09T21:47:19.083013Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 907 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:19.083072Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 907 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:19.083084Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=907 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:19.083090Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:19.083633Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 907 sessionId:  describe result for acl check
2025-10-09T21:47:19.083682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:19.083685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:19.083687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:19.083693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341896195849696:8551] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:19.084351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341896195849696:8551] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:19.088800Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341896195849696:8551] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:19.094060Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341896195849696:8551] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:19.094083Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341896195849696:8551] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:19.094087Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341896195849696:8551] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:19.094098Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 907 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:19.104030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:19.107151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:19.107473Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:19.107485Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:19.107500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:19.107568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:19.107619Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341895834030548:7727]
2025-10-09T21:47:19.108208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:19.108944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:19.109006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:19.109408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:19.109469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:19.109470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:19.109472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:19.109473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:19.109474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:19.109475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:19.109476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.109477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.109478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.109485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:19.109495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:19.109499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:19.109503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:19.109507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:19.109511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:19.109515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.109520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.109523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.109535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:19.109545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:19.109552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:19.109558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:19.109563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:19.109569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:19.109576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:19.109581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:19.109586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:19.109592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:19.109597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:19.109603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:19.109610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.109616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.109620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.109626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.109632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.109638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.109671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:19.121063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:19.122677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:19.122700Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:19.122703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:19.122708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:19.122716Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 906 [2:7559341895834030548:7727]
2025-10-09T21:47:19.122732Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:19.122754Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:19.122766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:19.122769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:19.122771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:19.122793Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:19.122803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:19.122811Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:19.122883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.122886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.122888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.122891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:19.122977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:19.122980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.122996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.123006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.123021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:19.123043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341895834030543:7727]
2025-10-09T21:47:19.123045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341895834030543:7727]
2025-10-09T21:47:19.123047Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341895834030543:7727]
2025-10-09T21:47:19.123060Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:19.123073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:19.123076Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:19.123078Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:19.123080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341896195849840:8551], now have 1 active actors on pipe
2025-10-09T21:47:19.123086Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:19.123100Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:19.123103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:19.123109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341896195849673:2432], now have 1 active actors on pipe
2025-10-09T21:47:19.123166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341895834030543:7727]
2025-10-09T21:47:19.123180Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:19.123188Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:19.123416Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 906
2025-10-09T21:47:19.123471Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 906
2025-10-09T21:47:19.124688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:19.124706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:19.124743Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|99f5ee09-5bc96525-5f22d52e-ffdc7386_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:19.124768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:19.124774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:19.124777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:19.124792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:19.124810Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:19.124830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:19.125141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:19.125152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:19.125173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:19.125347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 907 partition: 0 MaxSeqNo: 25 sessionId: src_id|99f5ee09-5bc96525-5f22d52e-ffdc7386_0
2025-10-09T21:47:19.126310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f75d0402-7e505292-78f46f04-10bbc814_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|99f5ee09-5bc96525-5f22d52e-ffdc7386_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:19.129150Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99f5ee09-5bc96525-5f22d52e-ffdc7386_0] Write session: try to update token
2025-10-09T21:47:19.129177Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99f5ee09-5bc96525-5f22d52e-ffdc7386_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:19.145143Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:19.145166Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:19.145169Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:19.145171Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:19.145181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341895834030543:7727]
2025-10-09T21:47:19.145185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341895834030543:7727]
2025-10-09T21:47:19.145189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341895834030543:7727]
2025-10-09T21:47:19.145238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:19.145247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.145250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:19.145431Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:19.150320Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:19.150338Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:19.150343Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:19.152389Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 907 sessionId: src_id|99f5ee09-5bc96525-5f22d52e-ffdc7386_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:19.155167Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:19.171100Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341896195849926:8525]
2025-10-09T21:47:19.172366Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 906, pipe: [1:7559341896195849926:8525]
2025-10-09T21:47:19.325626Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:19.326257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:19.326275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:19.326321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341896195849926:8525], now have 1 active actors on pipe
2025-10-09T21:47:19.326565Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:19.327193Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:19.327278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 907 reason: status is not ok: tablet will be restarted right now sessionId: src_id|99f5ee09-5bc96525-5f22d52e-ffdc7386_0
2025-10-09T21:47:19.327472Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 907 sessionId: src_id|99f5ee09-5bc96525-5f22d52e-ffdc7386_0 is DEAD
2025-10-09T21:47:19.327530Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341896195849926:8525]
2025-10-09T21:47:19.327563Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:19.327824Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99f5ee09-5bc96525-5f22d52e-ffdc7386_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:19.327834Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99f5ee09-5bc96525-5f22d52e-ffdc7386_0] Write session will restart in 0.005403s
2025-10-09T21:47:19.327862Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99f5ee09-5bc96525-5f22d52e-ffdc7386_0] Write session: Do CDS request
2025-10-09T21:47:19.327870Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99f5ee09-5bc96525-5f22d52e-ffdc7386_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:19.334569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99f5ee09-5bc96525-5f22d52e-ffdc7386_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:19.335314Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:19.335331Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 908
2025-10-09T21:47:19.335495Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 908 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:19.335530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 908 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:19.335535Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=908 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:19.335539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:19.336048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 908 sessionId:  describe result for acl check
2025-10-09T21:47:19.336092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:19.336099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:19.336101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:19.336107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341896195849958:8563] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:19.336675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341896195849958:8563] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:19.340012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341896195849958:8563] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:19.346826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341896195849958:8563] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:19.346847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341896195849958:8563] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:19.346852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341896195849958:8563] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:19.346863Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 908 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:19.358352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:19.361870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:19.362169Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:19.362183Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:19.362198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:19.362262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:19.362302Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341895834030787:7731]
2025-10-09T21:47:19.362963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:19.366588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:19.366663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:19.366993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:19.367026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:19.367027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:19.367028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:19.367028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:19.367029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:19.367030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:19.367031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.367031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.367032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.367036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:19.367044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:19.367046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:19.367049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:19.367051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:19.367053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:19.367055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.367057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.367059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.367065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:19.367070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:19.367077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:19.367080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:19.367083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:19.367087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:19.367089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:19.367092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:19.367095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:19.367098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:19.367101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:19.367104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:19.367107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.367109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.367112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.367115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.367118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.367121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.367141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:19.376312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:19.377589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:19.377608Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:19.377611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:19.377616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:19.377624Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 907 [2:7559341895834030787:7731]
2025-10-09T21:47:19.377639Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:19.377662Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:19.377665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:19.377669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:19.377672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:19.377692Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:19.377705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:19.377713Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:19.377775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.377781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.377784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.377787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:19.377856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:19.377863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.377891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.377907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.377929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:19.377954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341895834030782:7731]
2025-10-09T21:47:19.377960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341895834030782:7731]
2025-10-09T21:47:19.377963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341895834030782:7731]
2025-10-09T21:47:19.377976Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:19.377995Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:19.378016Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:19.378023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:19.378061Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:19.378066Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:19.378069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:19.378079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341895834030782:7731]
2025-10-09T21:47:19.378084Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:19.378166Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:19.378275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341896195850058:8563], now have 1 active actors on pipe
2025-10-09T21:47:19.378292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341896195849946:2432], now have 1 active actors on pipe
2025-10-09T21:47:19.378552Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 907
2025-10-09T21:47:19.378551Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 907
2025-10-09T21:47:19.378707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:19.378726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:19.378776Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|95fe7de-3d1f7adb-86db851b-7d980b32_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:19.378790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:19.378797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:19.378800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:19.378812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:19.378832Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:19.378850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:19.379395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:19.379411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:19.379433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:19.379995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 908 partition: 0 MaxSeqNo: 25 sessionId: src_id|95fe7de-3d1f7adb-86db851b-7d980b32_0
2025-10-09T21:47:19.380486Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99f5ee09-5bc96525-5f22d52e-ffdc7386_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|95fe7de-3d1f7adb-86db851b-7d980b32_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:19.383050Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|95fe7de-3d1f7adb-86db851b-7d980b32_0] Write session: try to update token
2025-10-09T21:47:19.383072Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|95fe7de-3d1f7adb-86db851b-7d980b32_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:19.399536Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:19.399558Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:19.399560Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:19.399561Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:19.399572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341895834030782:7731]
2025-10-09T21:47:19.399576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341895834030782:7731]
2025-10-09T21:47:19.399580Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341895834030782:7731]
2025-10-09T21:47:19.399630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:19.399641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.399643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:19.399784Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:19.402100Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:19.402118Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:19.402122Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:19.413440Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 908 sessionId: src_id|95fe7de-3d1f7adb-86db851b-7d980b32_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:19.416228Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:19.427557Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341896195850186:8525]
2025-10-09T21:47:19.428420Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 907, pipe: [1:7559341896195850186:8525]
2025-10-09T21:47:19.560951Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:19.561251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:19.561262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:19.561311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341896195850186:8525], now have 1 active actors on pipe
2025-10-09T21:47:19.562790Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:19.564179Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95fe7de-3d1f7adb-86db851b-7d980b32_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:19.564191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95fe7de-3d1f7adb-86db851b-7d980b32_0] Write session will restart in 0.009353s
2025-10-09T21:47:19.564215Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95fe7de-3d1f7adb-86db851b-7d980b32_0] Write session: Do CDS request
2025-10-09T21:47:19.564221Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95fe7de-3d1f7adb-86db851b-7d980b32_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:19.563720Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:19.563805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 908 reason: status is not ok: tablet will be restarted right now sessionId: src_id|95fe7de-3d1f7adb-86db851b-7d980b32_0
2025-10-09T21:47:19.563986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 908 sessionId: src_id|95fe7de-3d1f7adb-86db851b-7d980b32_0 is DEAD
2025-10-09T21:47:19.564115Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:19.564147Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341896195850186:8525]
2025-10-09T21:47:19.575614Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|95fe7de-3d1f7adb-86db851b-7d980b32_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:19.577121Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:19.577139Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 909
2025-10-09T21:47:19.577390Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 909 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:19.577437Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 909 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:19.577448Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=909 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:19.577452Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:19.578537Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 909 sessionId:  describe result for acl check
2025-10-09T21:47:19.578584Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:19.578591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:19.578592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:19.578599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341896195850253:8567] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:19.579189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341896195850253:8567] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:19.582688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341896195850253:8567] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:19.588124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341896195850253:8567] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:19.588139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341896195850253:8567] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:19.588143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341896195850253:8567] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:19.588152Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 909 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:19.591577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:19.594354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:19.594613Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:19.594625Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:19.594638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:19.594695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:19.594737Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341895834031020:7733]
2025-10-09T21:47:19.595353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:19.595696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:19.595739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:19.596074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:19.596129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:19.596130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:19.596132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:19.596133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:19.596134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:19.596135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:19.596136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.596137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.596138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.596144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:19.596155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:19.596158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:19.596162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:19.596166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:19.596169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:19.596173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.596176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.596178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.596186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:19.596195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:19.596200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:19.596204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:19.596208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:19.596213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:19.596217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:19.596221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:19.596226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:19.596231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:19.596235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:19.596239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:19.596243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.596249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.596253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.596258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.596262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.596266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.596297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:19.605961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:19.607574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:19.607598Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:19.607601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:19.607605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:19.607617Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 908 [2:7559341895834031020:7733]
2025-10-09T21:47:19.607633Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:19.607653Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:19.607657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:19.607660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:19.607662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:19.607683Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:19.607695Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:19.607710Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:19.607783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.607790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.607792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.607795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:19.607873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:19.607881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.607894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.607905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.607921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:19.607951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341896195850189:2432], now have 1 active actors on pipe
2025-10-09T21:47:19.607966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341895834031015:7733]
2025-10-09T21:47:19.607968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341895834031015:7733]
2025-10-09T21:47:19.607970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341895834031015:7733]
2025-10-09T21:47:19.607988Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:19.608072Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 908
2025-10-09T21:47:19.608075Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 908
2025-10-09T21:47:19.608005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341896195850414:8567], now have 1 active actors on pipe
2025-10-09T21:47:19.608018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:19.608032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:19.608035Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:19.608064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:19.608069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:19.608073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:19.608086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341895834031015:7733]
2025-10-09T21:47:19.608089Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:19.608091Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:19.608327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:19.608342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:19.608385Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|53dacc76-6d36a6db-488ac35b-35dbeb76_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:19.608395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:19.608401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:19.608404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:19.608416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:19.608430Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:19.608449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:19.608723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:19.608736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:19.608770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:19.609152Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 909 partition: 0 MaxSeqNo: 25 sessionId: src_id|53dacc76-6d36a6db-488ac35b-35dbeb76_0
2025-10-09T21:47:19.609575Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95fe7de-3d1f7adb-86db851b-7d980b32_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|53dacc76-6d36a6db-488ac35b-35dbeb76_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:19.612425Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53dacc76-6d36a6db-488ac35b-35dbeb76_0] Write session: try to update token
2025-10-09T21:47:19.612447Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53dacc76-6d36a6db-488ac35b-35dbeb76_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:19.631481Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:19.631506Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:19.631518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341895834031015:7733]
2025-10-09T21:47:19.631577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:19.631763Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:19.631920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:19.632069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:19.632218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:19.632425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:19.632481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:19.632874Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:19.635894Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 909 sessionId: src_id|53dacc76-6d36a6db-488ac35b-35dbeb76_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:19.638497Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:19.638823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:19.638846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:19.638892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:19.642954Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:19.646239Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:19.646259Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:19.646262Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:19.646264Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:19.646276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341895834031015:7733]
2025-10-09T21:47:19.646282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341895834031015:7733]
2025-10-09T21:47:19.646286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341895834031015:7733]
2025-10-09T21:47:19.646335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:19.646349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.646352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:19.646510Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:19.653471Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:19.653490Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:19.653494Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:19.665410Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341896195850430:8525]
2025-10-09T21:47:19.674264Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 908, pipe: [1:7559341896195850430:8525]
2025-10-09T21:47:19.808108Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:19.808478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341896195850430:8525], now have 1 active actors on pipe
2025-10-09T21:47:19.816870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:19.816908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:19.817097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:19.817164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:19.817230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:19.817293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:19.817837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:19.817844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:19.817936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:19.817990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:19.818168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:19.818244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:19.818394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:19.818403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:19.818472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:19.818527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:19.818589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:19.818651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:19.818774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:19.818779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:19.818852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:19.818919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:19.818981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:19.819043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:19.819085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:19.819089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:19.819182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:19.819244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:19.819305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:19.819365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:19.819410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:19.819413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:19.819484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:19.819543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:19.819600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:19.819664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:19.819704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:19.819707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:19.819768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:19.819822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:19.819889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:19.819947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:19.820008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:19.820011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:19.820076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:19.820137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:19.820196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:19.820258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:19.821072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:19.821077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:19.821151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:19.821209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:19.821266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:19.821329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:19.821432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:19.821435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:19.821502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:19.821560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:19.821618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:19.821680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:19.821719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:19.821722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:19.822307Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:19.822343Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:19.822346Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:19.822372Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341896195850430:8525]
2025-10-09T21:47:19.822379Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 909 reason: error at write: Disconnected sessionId: src_id|53dacc76-6d36a6db-488ac35b-35dbeb76_0
2025-10-09T21:47:19.822588Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 909 sessionId: src_id|53dacc76-6d36a6db-488ac35b-35dbeb76_0 is DEAD
2025-10-09T21:47:19.823530Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:19.826344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53dacc76-6d36a6db-488ac35b-35dbeb76_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:19.826356Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53dacc76-6d36a6db-488ac35b-35dbeb76_0] Write session will restart in 0.009645s
2025-10-09T21:47:19.826379Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53dacc76-6d36a6db-488ac35b-35dbeb76_0] Write session: Do CDS request
2025-10-09T21:47:19.826385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53dacc76-6d36a6db-488ac35b-35dbeb76_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:19.837950Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53dacc76-6d36a6db-488ac35b-35dbeb76_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:19.838081Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:19.838098Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 910
2025-10-09T21:47:19.838594Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 910 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:19.838634Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 910 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:19.838640Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=910 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:19.838645Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:19.839074Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 910 sessionId:  describe result for acl check
2025-10-09T21:47:19.839114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:19.839116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:19.839118Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:19.839124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341896195850537:8572] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:19.839710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341896195850537:8572] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:19.843112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341896195850537:8572] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:19.847282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341896195850537:8572] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:19.847304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341896195850537:8572] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:19.847309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341896195850537:8572] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:19.847320Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 910 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:19.849202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:19.850070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:19.850316Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:19.850326Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:19.850338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:19.850397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:19.850436Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341895834031257:7737]
2025-10-09T21:47:19.851026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:19.852533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:19.852606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:19.853181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:19.853252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:19.853258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:19.853259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:19.853260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:19.853261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:19.853262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:19.853264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.853265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.853266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.853272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:19.853282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:19.853285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:19.853289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:19.853292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:19.853296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:19.853300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.853303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.853306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.853314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:19.853323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:19.853329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:19.853334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:19.853339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:19.853344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:19.853348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:19.853353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:19.853357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:19.853374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:19.853384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:19.853390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:19.853394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.853401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.853411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.853416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.853422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.853427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.853455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:19.862098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:19.863571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:19.863588Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:19.863590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:19.863595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:19.863602Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 909 [2:7559341895834031257:7737]
2025-10-09T21:47:19.863617Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:19.863639Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:19.863642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:19.863645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:19.863647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:19.863669Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:19.863681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:19.863688Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:19.863758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.863769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.863771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.863774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:19.863846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:19.863849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.863862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:19.863872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:19.863887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:19.863907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341895834031252:7737]
2025-10-09T21:47:19.863910Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341895834031252:7737]
2025-10-09T21:47:19.863911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341895834031252:7737]
2025-10-09T21:47:19.863921Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:19.863936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:19.863947Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:19.863950Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:19.863951Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:19.863953Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:19.863954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:19.864005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341896195850435:2432], now have 1 active actors on pipe
2025-10-09T21:47:19.864018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341896195850662:8572], now have 1 active actors on pipe
2025-10-09T21:47:19.864020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341895834031252:7737]
2025-10-09T21:47:19.864029Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:19.864065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:19.864266Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 909
2025-10-09T21:47:19.864409Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 909
2025-10-09T21:47:19.864495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:19.864512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:19.864545Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|962f9141-39121066-2f6828e6-9e09cdac_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:19.864559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:19.864571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:19.864573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:19.864586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:19.864602Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:19.864619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:19.864818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:19.864833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:19.864856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:19.865677Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53dacc76-6d36a6db-488ac35b-35dbeb76_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|962f9141-39121066-2f6828e6-9e09cdac_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:19.865278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 910 partition: 0 MaxSeqNo: 25 sessionId: src_id|962f9141-39121066-2f6828e6-9e09cdac_0
2025-10-09T21:47:19.869120Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|962f9141-39121066-2f6828e6-9e09cdac_0] Write session: try to update token
2025-10-09T21:47:19.869140Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|962f9141-39121066-2f6828e6-9e09cdac_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:19.883293Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:19.883314Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:19.883317Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:19.883319Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:19.883329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341895834031252:7737]
2025-10-09T21:47:19.883334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341895834031252:7737]
2025-10-09T21:47:19.883337Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341895834031252:7737]
2025-10-09T21:47:19.883386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:19.883386Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:19.883397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:19.883400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:19.883404Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:19.883409Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:19.883540Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:19.886905Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 910 sessionId: src_id|962f9141-39121066-2f6828e6-9e09cdac_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:19.889615Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:19.922677Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341896195850677:8525]
2025-10-09T21:47:19.923029Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 909, pipe: [1:7559341896195850677:8525]
2025-10-09T21:47:20.044603Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:20.044964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:20.044980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:20.045024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341896195850677:8525], now have 1 active actors on pipe
2025-10-09T21:47:20.045319Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341896195850677:8525]
2025-10-09T21:47:20.045326Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:20.045355Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:20.045357Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:20.045419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 910 reason: error at write: Disconnected sessionId: src_id|962f9141-39121066-2f6828e6-9e09cdac_0
2025-10-09T21:47:20.045584Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 910 sessionId: src_id|962f9141-39121066-2f6828e6-9e09cdac_0 is DEAD
2025-10-09T21:47:20.045972Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|962f9141-39121066-2f6828e6-9e09cdac_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:20.045984Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|962f9141-39121066-2f6828e6-9e09cdac_0] Write session will restart in 0.007122s
2025-10-09T21:47:20.046008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|962f9141-39121066-2f6828e6-9e09cdac_0] Write session: Do CDS request
2025-10-09T21:47:20.046017Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|962f9141-39121066-2f6828e6-9e09cdac_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:20.046244Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:20.054631Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|962f9141-39121066-2f6828e6-9e09cdac_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:20.054756Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:20.054773Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 911
2025-10-09T21:47:20.055042Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 911 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:20.055091Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 911 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:20.055099Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=911 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:20.055103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:20.055536Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 911 sessionId:  describe result for acl check
2025-10-09T21:47:20.055570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:20.055571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:20.055573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:20.055579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341900490818012:8576] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:20.056138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341900490818012:8576] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:20.058483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341900490818012:8576] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:20.062871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341900490818012:8576] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:20.062888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341900490818012:8576] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:20.062892Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341900490818012:8576] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:20.062910Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 911 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:20.069198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:20.071748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:20.071990Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:20.072000Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:20.072012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:20.072062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:20.072097Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341900128998814:7745]
2025-10-09T21:47:20.072678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:20.073033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:20.073084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:20.073408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:20.073476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:20.073478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:20.073479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:20.073483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:20.073484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:20.073485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:20.073486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.073487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.073488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.073494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:20.073504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:20.073508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:20.073512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:20.073516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:20.073521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:20.073525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.073529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.073533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.073541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:20.073550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:20.073558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:20.073563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:20.073568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:20.073573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:20.073577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:20.073582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:20.073587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:20.073592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:20.073596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:20.073602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:20.073606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.073611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.073617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.073621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.073625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.073631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.073660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:20.082137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:20.083476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:20.083495Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:20.083498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:20.083503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:20.083512Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 910 [2:7559341900128998814:7745]
2025-10-09T21:47:20.083527Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:20.083548Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:20.083555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:20.083559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:20.083561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:20.083582Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:20.083588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:20.083595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:20.083670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.083674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.083676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.083679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:20.083752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:20.083754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.083765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.083775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.083790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:20.083810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341900128998799:7745]
2025-10-09T21:47:20.083812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341900128998799:7745]
2025-10-09T21:47:20.083995Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 910
2025-10-09T21:47:20.083814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341900128998799:7745]
2025-10-09T21:47:20.083822Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:20.083836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:20.084001Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 910
2025-10-09T21:47:20.083850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:20.083852Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:20.083857Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:20.083861Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:20.083863Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:20.083891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341900128998799:7745]
2025-10-09T21:47:20.083894Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:20.083896Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:20.083907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341900490817978:2432], now have 1 active actors on pipe
2025-10-09T21:47:20.083911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341900490818128:8576], now have 1 active actors on pipe
2025-10-09T21:47:20.084179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:20.084192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:20.084219Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|59208e45-4c874e9c-33aa4697-a845f54_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:20.084233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:20.084237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:20.084240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:20.084249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:20.084259Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:20.084276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:20.084466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:20.084472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:20.084484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:20.084641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 911 partition: 0 MaxSeqNo: 25 sessionId: src_id|59208e45-4c874e9c-33aa4697-a845f54_0
2025-10-09T21:47:20.085093Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|962f9141-39121066-2f6828e6-9e09cdac_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|59208e45-4c874e9c-33aa4697-a845f54_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:20.088009Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|59208e45-4c874e9c-33aa4697-a845f54_0] Write session: try to update token
2025-10-09T21:47:20.088029Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|59208e45-4c874e9c-33aa4697-a845f54_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:20.102590Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:20.102615Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:20.102618Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:20.102620Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:20.102630Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341900128998799:7745]
2025-10-09T21:47:20.102634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341900128998799:7745]
2025-10-09T21:47:20.102638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341900128998799:7745]
2025-10-09T21:47:20.102684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:20.102692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.102695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:20.102699Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:20.102716Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:20.102720Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:20.102845Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:20.105614Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 911 sessionId: src_id|59208e45-4c874e9c-33aa4697-a845f54_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:20.109827Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:20.145421Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341900490818225:8525]
2025-10-09T21:47:20.145738Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 910, pipe: [1:7559341900490818225:8525]
2025-10-09T21:47:20.276533Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:20.276810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:20.277316Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341900490818225:8525]
2025-10-09T21:47:20.277340Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:20.276828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:20.276869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341900490818225:8525], now have 1 active actors on pipe
2025-10-09T21:47:20.277012Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:20.277432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 911 reason: status is not ok: tablet will be restarted right now sessionId: src_id|59208e45-4c874e9c-33aa4697-a845f54_0
2025-10-09T21:47:20.277639Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 911 sessionId: src_id|59208e45-4c874e9c-33aa4697-a845f54_0 is DEAD
2025-10-09T21:47:20.277786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|59208e45-4c874e9c-33aa4697-a845f54_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:20.277794Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|59208e45-4c874e9c-33aa4697-a845f54_0] Write session will restart in 0.009131s
2025-10-09T21:47:20.277818Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|59208e45-4c874e9c-33aa4697-a845f54_0] Write session: Do CDS request
2025-10-09T21:47:20.277825Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|59208e45-4c874e9c-33aa4697-a845f54_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:20.277838Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:20.288618Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|59208e45-4c874e9c-33aa4697-a845f54_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:20.288766Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:20.288783Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 912
2025-10-09T21:47:20.289054Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 912 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:20.289089Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 912 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:20.289096Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=912 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:20.289099Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:20.289567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 912 sessionId:  describe result for acl check
2025-10-09T21:47:20.289601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:20.289603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:20.289605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:20.289611Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341900490818270:8588] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:20.290202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341900490818270:8588] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:20.294104Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341900490818270:8588] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:20.298928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341900490818270:8588] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:20.298944Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341900490818270:8588] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:20.298948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341900490818270:8588] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:20.298957Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 912 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:20.331092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:20.333949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:20.334218Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:20.334231Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:20.334244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:20.334299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:20.334346Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341900128999061:7753]
2025-10-09T21:47:20.334990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:20.335478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:20.335524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:20.335850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:20.335909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:20.335910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:20.335912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:20.335913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:20.335914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:20.335915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:20.335916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.335918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.335919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.335925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:20.335936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:20.335941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:20.335944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:20.335948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:20.335952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:20.335956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.335960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.335963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.335972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:20.335981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:20.335990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:20.335995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:20.335999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:20.336005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:20.336009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:20.336014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:20.336019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:20.336024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:20.336029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:20.336035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:20.336040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.336045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.336050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.336055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.336063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.336068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.336095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:20.349365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:20.351476Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 911
2025-10-09T21:47:20.351484Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 911
2025-10-09T21:47:20.350679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:20.350700Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:20.350703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:20.350711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:20.350719Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 911 [2:7559341900128999061:7753]
2025-10-09T21:47:20.350735Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:20.350760Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:20.350763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:20.350767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:20.350770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:20.350793Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:20.350801Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:20.350809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:20.350900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.350903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.350905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.350907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:20.350997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:20.351000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.351014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.351025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.351041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:20.351062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341900128999055:7753]
2025-10-09T21:47:20.351065Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341900128999055:7753]
2025-10-09T21:47:20.351069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341900128999055:7753]
2025-10-09T21:47:20.351078Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:20.351094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:20.351099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:20.351103Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:20.351105Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:20.351110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:20.351114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:20.351175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341900490818257:2432], now have 1 active actors on pipe
2025-10-09T21:47:20.351178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341900128999055:7753]
2025-10-09T21:47:20.351194Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:20.351200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341900490818323:8588], now have 1 active actors on pipe
2025-10-09T21:47:20.351202Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:20.354303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:20.354320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:20.354363Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|634389de-e349ffb8-2791cb68-9baa3c20_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:20.354375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:20.354380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:20.354384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:20.354396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:20.354415Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:20.354436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:20.354899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 912 partition: 0 MaxSeqNo: 25 sessionId: src_id|634389de-e349ffb8-2791cb68-9baa3c20_0
2025-10-09T21:47:20.355288Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|59208e45-4c874e9c-33aa4697-a845f54_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|634389de-e349ffb8-2791cb68-9baa3c20_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:20.357812Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|634389de-e349ffb8-2791cb68-9baa3c20_0] Write session: try to update token
2025-10-09T21:47:20.357830Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|634389de-e349ffb8-2791cb68-9baa3c20_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:20.354724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:20.354739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:20.354770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:20.377431Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341900490818497:8525]
2025-10-09T21:47:20.378512Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 911, pipe: [1:7559341900490818497:8525]
2025-10-09T21:47:20.378818Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 912 sessionId: src_id|634389de-e349ffb8-2791cb68-9baa3c20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:20.381545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:20.377960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341900490818497:8525], now have 1 active actors on pipe
2025-10-09T21:47:20.381631Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:20.381636Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:20.381647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341900128999055:7753]
2025-10-09T21:47:20.381695Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:20.381697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:20.381887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:20.382048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:20.382188Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:20.382347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:20.382995Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:20.383050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:20.383061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:20.383084Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:20.383125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:20.383229Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:20.402961Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:20.402986Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:20.402990Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:20.402992Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:20.403004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341900128999055:7753]
2025-10-09T21:47:20.403009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341900128999055:7753]
2025-10-09T21:47:20.403013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341900128999055:7753]
2025-10-09T21:47:20.403060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:20.403069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.403072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:20.403209Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:20.406096Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:20.406112Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:20.406116Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:20.570236Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:20.578983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:20.579015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:20.579189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.579253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.579320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.579387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.579438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.579452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.579523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.579588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.579651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.579723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.579772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.579775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.579835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.579891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.579955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.580020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.580068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.580077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.580141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.580206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.580261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.580321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.580362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.580366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.580426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.580477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.580569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.580626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.580668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.580672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.580737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.580791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.580846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.580902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.580954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.580958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.581019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.581073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.581132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.581189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.581229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.581232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.581294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.581353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.581412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.581468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.581508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.581511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.581576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.581629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.581690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.581749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.581790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.581796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.581861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.581930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.581994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.582313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.582369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.582378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.582784Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341900490818497:8525]
2025-10-09T21:47:20.582817Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:20.582871Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:20.582882Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:20.582915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 912 reason: error at write: Disconnected sessionId: src_id|634389de-e349ffb8-2791cb68-9baa3c20_0
2025-10-09T21:47:20.583417Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 912 sessionId: src_id|634389de-e349ffb8-2791cb68-9baa3c20_0 is DEAD
2025-10-09T21:47:20.583594Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:20.590267Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|634389de-e349ffb8-2791cb68-9baa3c20_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:20.590282Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|634389de-e349ffb8-2791cb68-9baa3c20_0] Write session will restart in 0.009270s
2025-10-09T21:47:20.590307Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|634389de-e349ffb8-2791cb68-9baa3c20_0] Write session: Do CDS request
2025-10-09T21:47:20.590313Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|634389de-e349ffb8-2791cb68-9baa3c20_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:20.600608Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|634389de-e349ffb8-2791cb68-9baa3c20_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:20.600789Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:20.600812Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 913
2025-10-09T21:47:20.600983Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 913 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:20.601022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 913 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:20.601028Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=913 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:20.601033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:20.601794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 913 sessionId:  describe result for acl check
2025-10-09T21:47:20.601832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:20.601841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:20.601842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:20.601849Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341900490818591:8592] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:20.602468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341900490818591:8592] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:20.606859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341900490818591:8592] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:20.610909Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341900490818591:8592] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:20.610928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341900490818591:8592] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:20.610935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341900490818591:8592] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:20.610960Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 913 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:20.613290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:20.616239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:20.616509Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:20.616528Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:20.616544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:20.616638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:20.616713Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341900128999302:7755]
2025-10-09T21:47:20.617308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:20.617714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:20.617773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:20.618170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:20.618244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:20.618253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:20.618254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:20.618255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:20.618256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:20.618258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:20.618259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.618260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.618262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.618269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:20.618282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:20.618286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:20.618291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:20.618296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:20.618300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:20.618305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.618310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.618314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.618323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:20.618333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:20.618340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:20.618346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:20.618358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:20.618364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:20.618370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:20.618376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:20.618381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:20.618387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:20.618393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:20.618399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:20.618404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.618415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.618420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.618431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.618436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.618446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.618476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:20.626360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:20.627997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:20.628485Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 912
2025-10-09T21:47:20.628538Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 912
2025-10-09T21:47:20.628004Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:20.628006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:20.628011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:20.628020Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 912 [2:7559341900128999302:7755]
2025-10-09T21:47:20.628040Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:20.628064Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:20.628067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:20.628071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:20.628073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:20.628093Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:20.628104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:20.628111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:20.628183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.628185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.628188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.628191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:20.628283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:20.628286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.628302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.628313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.628329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:20.628352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341900128999296:7755]
2025-10-09T21:47:20.628354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341900128999296:7755]
2025-10-09T21:47:20.628356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341900128999296:7755]
2025-10-09T21:47:20.628366Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:20.628378Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:20.628382Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:20.628383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341900490818502:2432], now have 1 active actors on pipe
2025-10-09T21:47:20.628384Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:20.628393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:20.628408Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:20.628411Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:20.628418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341900490818732:8592], now have 1 active actors on pipe
2025-10-09T21:47:20.628462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341900128999296:7755]
2025-10-09T21:47:20.628478Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:20.628487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:20.628680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:20.628690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:20.628734Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9fc6a06b-a387a2e7-324f3666-25780c36_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:20.628745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:20.628751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:20.628753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:20.628767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:20.628780Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:20.628819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:20.629032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:20.629043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:20.629057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:20.629187Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 913 partition: 0 MaxSeqNo: 25 sessionId: src_id|9fc6a06b-a387a2e7-324f3666-25780c36_0
2025-10-09T21:47:20.630327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|634389de-e349ffb8-2791cb68-9baa3c20_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9fc6a06b-a387a2e7-324f3666-25780c36_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:20.633080Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9fc6a06b-a387a2e7-324f3666-25780c36_0] Write session: try to update token
2025-10-09T21:47:20.633102Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9fc6a06b-a387a2e7-324f3666-25780c36_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:20.650297Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 913 sessionId: src_id|9fc6a06b-a387a2e7-324f3666-25780c36_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:20.653511Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:20.656801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:20.656824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:20.656850Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:20.656853Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:20.656863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341900128999296:7755]
2025-10-09T21:47:20.656920Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:20.656928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:20.657114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:20.657446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:20.657599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:20.657745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:20.657961Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:20.658011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:20.658057Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:20.658222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:20.665079Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:20.665096Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:20.665098Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:20.665100Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:20.665111Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341900128999296:7755]
2025-10-09T21:47:20.665115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341900128999296:7755]
2025-10-09T21:47:20.665119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341900128999296:7755]
2025-10-09T21:47:20.665170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:20.665170Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:20.665177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.665180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:20.665184Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:20.665189Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:20.665336Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:20.683038Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341900490818748:8525]
2025-10-09T21:47:20.686521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 912, pipe: [1:7559341900490818748:8525]
2025-10-09T21:47:20.876860Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:20.882683Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:47:20.882714Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 9
2025-10-09T21:47:20.894837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:20.894871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:20.895095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.895162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.895239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.895304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.895345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.895350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.895425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.895479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.895539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.895608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.895651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.895655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.895724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.895785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.895856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.895920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.895965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.895968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.896047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.896113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.896178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.896241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.896285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.896288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.896364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.896424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.896488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.896552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.896597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.896600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.896674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.896731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.896785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.896840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.896905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.896908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.896979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.897036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.897104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.897171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.897212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.897215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.897286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.897344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.897402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.897462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.897509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.897512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.897582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.897642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.897707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.897767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.897818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.897821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.897904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:20.897962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:20.898022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:20.899548Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341900490818748:8525]
2025-10-09T21:47:20.899585Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:20.899600Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 9
2025-10-09T21:47:20.899636Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:20.899643Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:20.899673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 913 reason: error at write: Disconnected sessionId: src_id|9fc6a06b-a387a2e7-324f3666-25780c36_0
2025-10-09T21:47:20.899883Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 913 sessionId: src_id|9fc6a06b-a387a2e7-324f3666-25780c36_0 is DEAD
2025-10-09T21:47:20.900067Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9fc6a06b-a387a2e7-324f3666-25780c36_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:20.900078Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9fc6a06b-a387a2e7-324f3666-25780c36_0] Write session will restart in 0.008481s
2025-10-09T21:47:20.900104Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9fc6a06b-a387a2e7-324f3666-25780c36_0] Write session: Do CDS request
2025-10-09T21:47:20.900110Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9fc6a06b-a387a2e7-324f3666-25780c36_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:20.898455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:20.898585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:20.898588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:20.898674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341900490818748:8525], now have 1 active actors on pipe
2025-10-09T21:47:20.899821Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:20.909628Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9fc6a06b-a387a2e7-324f3666-25780c36_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:20.909817Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:20.909835Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 914
2025-10-09T21:47:20.910185Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 914 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:20.910230Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 914 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:20.910242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=914 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:20.910245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:20.911034Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 914 sessionId:  describe result for acl check
2025-10-09T21:47:20.911074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:20.911080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:20.911082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:20.911088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341900490818780:8597] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:20.912166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341900490818780:8597] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:20.917015Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341900490818780:8597] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:20.921486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341900490818780:8597] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:20.921505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341900490818780:8597] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:20.921510Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341900490818780:8597] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:20.921520Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 914 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:20.924897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:20.927924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:20.928187Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:20.928199Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:20.928214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:20.928287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:20.928337Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341900128999540:7759]
2025-10-09T21:47:20.928960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:20.929326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:20.929381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:20.929750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:20.929828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:20.929830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:20.929831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:20.929833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:20.929834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:20.929835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:20.929836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.929837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.929839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.929845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:20.929857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:20.929862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:20.929867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:20.929883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:20.929887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:20.929891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.929895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.929900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.929907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:20.929917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:20.929924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:20.929930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:20.929935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:20.929940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:20.929947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:20.929952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:20.929956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:20.929962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:20.929968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:20.929974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:20.929979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.929985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.929990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.929996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.930001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.930008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.930068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:20.941990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:20.943128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:20.943144Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:20.943147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:20.943152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:20.943161Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 913 [2:7559341900128999540:7759]
2025-10-09T21:47:20.943178Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:20.943202Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:20.943205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:20.943208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:20.943211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:20.943231Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:20.943244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:20.943252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:20.943324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.943327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.943329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.943345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:20.943423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:20.943425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.943439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:20.943451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:20.943466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:20.943488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341900128999535:7759]
2025-10-09T21:47:20.943490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341900128999535:7759]
2025-10-09T21:47:20.943493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341900128999535:7759]
2025-10-09T21:47:20.943502Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:20.943518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:20.943533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:20.943537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:20.943562Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:20.943566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:20.943568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:20.943581Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341900128999535:7759]
2025-10-09T21:47:20.943585Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:20.943588Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:20.943610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341900490818974:8597], now have 1 active actors on pipe
2025-10-09T21:47:20.943613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341900490818756:2432], now have 1 active actors on pipe
2025-10-09T21:47:20.943800Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 913
2025-10-09T21:47:20.943842Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 913
2025-10-09T21:47:20.944268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:20.944295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:20.944359Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|eaf35af9-eb008186-2b3ff598-e5212791_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:20.944374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:20.944379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:20.944404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:20.944417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:20.944434Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:20.944502Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:47:20.944539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:20.944915Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 11 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:47:20.944968Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:47:20.945220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:20.945232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:20.945267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:20.945576Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 914 partition: 0 MaxSeqNo: 25 sessionId: src_id|eaf35af9-eb008186-2b3ff598-e5212791_0
2025-10-09T21:47:20.946083Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9fc6a06b-a387a2e7-324f3666-25780c36_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|eaf35af9-eb008186-2b3ff598-e5212791_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:20.954573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eaf35af9-eb008186-2b3ff598-e5212791_0] Write session: try to update token
2025-10-09T21:47:20.954603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eaf35af9-eb008186-2b3ff598-e5212791_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:20.963518Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:20.963537Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:20.963539Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:20.963542Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:20.963553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341900128999535:7759]
2025-10-09T21:47:20.963557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341900128999535:7759]
2025-10-09T21:47:20.963560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341900128999535:7759]
2025-10-09T21:47:20.963608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:20.963616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:20.963619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:20.963622Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:20.963637Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:20.963645Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:20.963763Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:20.973437Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 914 sessionId: src_id|eaf35af9-eb008186-2b3ff598-e5212791_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:20.980014Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:20.999893Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341900490819006:8525]
2025-10-09T21:47:21.000260Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 913, pipe: [1:7559341900490819006:8525]
2025-10-09T21:47:21.150917Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:21.151460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:21.151471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:21.151496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341900490819006:8525], now have 1 active actors on pipe
2025-10-09T21:47:21.151789Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:21.152323Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eaf35af9-eb008186-2b3ff598-e5212791_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:21.151793Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:21.151801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:21.151809Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:21.151821Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 0
2025-10-09T21:47:21.151874Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341900490819006:8525]
2025-10-09T21:47:21.151883Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 914 reason: error at write: Disconnected sessionId: src_id|eaf35af9-eb008186-2b3ff598-e5212791_0
2025-10-09T21:47:21.152134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 914 sessionId: src_id|eaf35af9-eb008186-2b3ff598-e5212791_0 is DEAD
2025-10-09T21:47:21.152341Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eaf35af9-eb008186-2b3ff598-e5212791_0] Write session will restart in 0.009122s
2025-10-09T21:47:21.152577Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eaf35af9-eb008186-2b3ff598-e5212791_0] Write session: Do CDS request
2025-10-09T21:47:21.152591Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eaf35af9-eb008186-2b3ff598-e5212791_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:21.162660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eaf35af9-eb008186-2b3ff598-e5212791_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:21.162802Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:21.162845Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 915
2025-10-09T21:47:21.163022Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 915 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:21.163077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 915 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:21.163238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=915 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:21.163261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:21.163875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 915 sessionId:  describe result for acl check
2025-10-09T21:47:21.163916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:21.163918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:21.163923Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:21.163929Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341904785786369:8601] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:21.164547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341904785786369:8601] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:21.171845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341904785786369:8601] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:21.177422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341904785786369:8601] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:21.177444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341904785786369:8601] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:21.177448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341904785786369:8601] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:21.177459Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 915 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:21.180857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:21.183994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:21.184279Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:21.184293Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:21.184309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:21.184380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:21.184430Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341904423967105:7772]
2025-10-09T21:47:21.185052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:21.185631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:21.185718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:21.186226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:21.186309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:21.186310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:21.186312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:21.186314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:21.186315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:21.186318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:21.186319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.186321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.186322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.186331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:21.186344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:21.186349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:21.186354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:21.186357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:21.186375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:21.186378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.186382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.186386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.186396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:21.186405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:21.186412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:21.186417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:21.186422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:21.186428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:21.186433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:21.186440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:21.186445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:21.186450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:21.186454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:21.186460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:21.186466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.186471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.186476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.186485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.186490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.186495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.186529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:21.194875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:21.195980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:21.195986Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:21.195990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:21.195995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:21.196002Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 914 [2:7559341904423967105:7772]
2025-10-09T21:47:21.196020Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:21.196041Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:21.196045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:21.196049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:21.196051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:21.196070Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:21.196077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:21.196084Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:21.196155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.196157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.196160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.196163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:21.196242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:21.196245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.196258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.196268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.196283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:21.196304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341904423967100:7772]
2025-10-09T21:47:21.196308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341904423967100:7772]
2025-10-09T21:47:21.196310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341904423967100:7772]
2025-10-09T21:47:21.196326Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:21.196342Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:21.196357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:21.196361Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:21.196408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341904423967100:7772]
2025-10-09T21:47:21.196411Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:21.196411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:21.196415Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:21.196418Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:21.196425Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:21.196595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341904785786315:2432], now have 1 active actors on pipe
2025-10-09T21:47:21.196601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341904785786542:8601], now have 1 active actors on pipe
2025-10-09T21:47:21.196689Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 914
2025-10-09T21:47:21.196735Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 914
2025-10-09T21:47:21.196866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:21.196881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:21.196949Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7742784c-9842b2cb-7b7172f2-c7aa8f4e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:21.196963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:21.196981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:21.196989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:21.197003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:21.197016Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:21.197078Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:47:21.197116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:21.197365Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 12 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:47:21.197401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:21.197413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:21.197406Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:47:21.197434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:21.197541Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 915 partition: 0 MaxSeqNo: 25 sessionId: src_id|7742784c-9842b2cb-7b7172f2-c7aa8f4e_0
2025-10-09T21:47:21.198085Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eaf35af9-eb008186-2b3ff598-e5212791_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7742784c-9842b2cb-7b7172f2-c7aa8f4e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:21.201023Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7742784c-9842b2cb-7b7172f2-c7aa8f4e_0] Write session: try to update token
2025-10-09T21:47:21.201047Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7742784c-9842b2cb-7b7172f2-c7aa8f4e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:21.214178Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:21.214194Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:21.214198Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:21.214200Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:21.214211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341904423967100:7772]
2025-10-09T21:47:21.214216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341904423967100:7772]
2025-10-09T21:47:21.214220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341904423967100:7772]
2025-10-09T21:47:21.214274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:21.214282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.214285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:21.214350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:21.214360Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:21.214364Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:21.214429Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:21.220540Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 915 sessionId: src_id|7742784c-9842b2cb-7b7172f2-c7aa8f4e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:21.223618Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:21.251835Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341904785786568:8525]
2025-10-09T21:47:21.252388Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 914, pipe: [1:7559341904785786568:8525]
2025-10-09T21:47:21.376968Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:21.377264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:21.377279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:21.377325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341904785786568:8525], now have 1 active actors on pipe
2025-10-09T21:47:21.377591Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:21.377591Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341904785786568:8525]
2025-10-09T21:47:21.377622Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:21.377625Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:21.377697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 915 reason: error at write: Disconnected sessionId: src_id|7742784c-9842b2cb-7b7172f2-c7aa8f4e_0
2025-10-09T21:47:21.377739Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:21.377915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 915 sessionId: src_id|7742784c-9842b2cb-7b7172f2-c7aa8f4e_0 is DEAD
2025-10-09T21:47:21.378084Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7742784c-9842b2cb-7b7172f2-c7aa8f4e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:21.378107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7742784c-9842b2cb-7b7172f2-c7aa8f4e_0] Write session will restart in 0.006688s
2025-10-09T21:47:21.378135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7742784c-9842b2cb-7b7172f2-c7aa8f4e_0] Write session: Do CDS request
2025-10-09T21:47:21.378141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7742784c-9842b2cb-7b7172f2-c7aa8f4e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:21.386578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7742784c-9842b2cb-7b7172f2-c7aa8f4e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:21.386709Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:21.386727Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 916
2025-10-09T21:47:21.386899Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 916 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:21.386958Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 916 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:21.386969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=916 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:21.386973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:21.387422Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 916 sessionId:  describe result for acl check
2025-10-09T21:47:21.387457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:21.387459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:21.387461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:21.387467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341904785786630:8613] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:21.388065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341904785786630:8613] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:21.390837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341904785786630:8613] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:21.395078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341904785786630:8613] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:21.395095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341904785786630:8613] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:21.395099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341904785786630:8613] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:21.395116Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 916 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:21.398797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:21.401191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:21.401435Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:21.401445Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:21.401457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:21.401513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:21.401556Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341904423967353:7777]
2025-10-09T21:47:21.402232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:21.402565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:21.402609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:21.402935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:21.402986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:21.402992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:21.402994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:21.402995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:21.402997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:21.402998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:21.402999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.403001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.403002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.403008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:21.403020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:21.403025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:21.403030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:21.403034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:21.403039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:21.403043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.403048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.403052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.403060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:21.403069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:21.403075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:21.403081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:21.403087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:21.403092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:21.403097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:21.403104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:21.403116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:21.403121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:21.403133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:21.403141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:21.403152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.403158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.403163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.403168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.403173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.403183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.403216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:21.409809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:21.411097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:21.411113Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:21.411117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:21.411122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:21.411130Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 915 [2:7559341904423967353:7777]
2025-10-09T21:47:21.411145Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:21.411164Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:21.411168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:21.411172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:21.411174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:21.411194Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:21.411200Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:21.411208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:21.411608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.411617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.411619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.411622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:21.411696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:21.411704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.411716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.411730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.411751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:21.411778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341904785786599:2432], now have 1 active actors on pipe
2025-10-09T21:47:21.411782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341904785786804:8613], now have 1 active actors on pipe
2025-10-09T21:47:21.411789Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341904423967348:7777]
2025-10-09T21:47:21.411791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341904423967348:7777]
2025-10-09T21:47:21.411794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341904423967348:7777]
2025-10-09T21:47:21.411810Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:21.411827Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:21.411832Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:21.411841Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:21.411844Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:21.411818Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 915
2025-10-09T21:47:21.411834Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 915
2025-10-09T21:47:21.411854Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:21.411861Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:21.411889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341904423967348:7777]
2025-10-09T21:47:21.411901Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:21.411922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:21.411931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:21.411964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:21.411980Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bd515442-31c83a83-c08802cf-95e6962_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:21.411995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:21.411999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:21.412002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:21.412014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:21.412032Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:21.412052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:21.412326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:21.412336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:21.412354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:21.412538Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 916 partition: 0 MaxSeqNo: 25 sessionId: src_id|bd515442-31c83a83-c08802cf-95e6962_0
2025-10-09T21:47:21.412930Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7742784c-9842b2cb-7b7172f2-c7aa8f4e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bd515442-31c83a83-c08802cf-95e6962_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:21.415459Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd515442-31c83a83-c08802cf-95e6962_0] Write session: try to update token
2025-10-09T21:47:21.415480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd515442-31c83a83-c08802cf-95e6962_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:21.426468Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:21.426483Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:21.426486Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:21.426488Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:21.426499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341904423967348:7777]
2025-10-09T21:47:21.426504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341904423967348:7777]
2025-10-09T21:47:21.426508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341904423967348:7777]
2025-10-09T21:47:21.426555Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:21.426558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:21.426562Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:21.426566Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:21.426566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.426570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:21.426722Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:21.431281Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 916 sessionId: src_id|bd515442-31c83a83-c08802cf-95e6962_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:21.433850Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:21.477650Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341904785786844:8525]
2025-10-09T21:47:21.478001Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 915, pipe: [1:7559341904785786844:8525]
2025-10-09T21:47:21.587399Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:21.587681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:21.587695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:21.587741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341904785786844:8525], now have 1 active actors on pipe
2025-10-09T21:47:21.588015Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:21.588004Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341904785786844:8525]
2025-10-09T21:47:21.588030Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:21.588038Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:21.588094Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 916 reason: error at write: Disconnected sessionId: src_id|bd515442-31c83a83-c08802cf-95e6962_0
2025-10-09T21:47:21.588155Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:21.588293Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 916 sessionId: src_id|bd515442-31c83a83-c08802cf-95e6962_0 is DEAD
2025-10-09T21:47:21.588514Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd515442-31c83a83-c08802cf-95e6962_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:21.588527Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd515442-31c83a83-c08802cf-95e6962_0] Write session will restart in 0.006216s
2025-10-09T21:47:21.588554Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd515442-31c83a83-c08802cf-95e6962_0] Write session: Do CDS request
2025-10-09T21:47:21.588560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd515442-31c83a83-c08802cf-95e6962_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:21.595559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd515442-31c83a83-c08802cf-95e6962_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:21.595688Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:21.595706Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 917
2025-10-09T21:47:21.595871Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 917 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:21.595919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 917 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:21.595931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=917 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:21.595935Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:21.596378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 917 sessionId:  describe result for acl check
2025-10-09T21:47:21.596412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:21.596417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:21.596418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:21.596423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341904785786862:8617] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:21.597039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341904785786862:8617] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:21.600200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341904785786862:8617] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:21.604030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341904785786862:8617] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:21.604048Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341904785786862:8617] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:21.604052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341904785786862:8617] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:21.604062Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 917 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:21.607837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:21.610321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:21.610568Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:21.610580Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:21.610593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:21.610667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:21.610712Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341904423967598:7779]
2025-10-09T21:47:21.611318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:21.611614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:21.611657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:21.611973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:21.612026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:21.612036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:21.612037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:21.612038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:21.612039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:21.612040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:21.612041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.612042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.612044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.612050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:21.612062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:21.612066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:21.612069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:21.612073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:21.612078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:21.612082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.612086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.612090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.612097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:21.612106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:21.612111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:21.612116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:21.612120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:21.612127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:21.612131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:21.612137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:21.612142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:21.612147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:21.612151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:21.612156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:21.612163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.612168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.612174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.612178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.612182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.612188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.612219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:21.620161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:21.621295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:21.621312Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:21.621315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:21.621320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:21.621328Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 916 [2:7559341904423967598:7779]
2025-10-09T21:47:21.621347Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:21.621371Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:21.621379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:21.621382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:21.621384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:21.621403Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:21.621409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:21.621419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:21.621493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.621500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.621502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.621504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:21.621575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:21.621582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.621594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.621608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.621622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:21.621649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341904785787040:8617], now have 1 active actors on pipe
2025-10-09T21:47:21.621664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341904423967593:7779]
2025-10-09T21:47:21.621666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341904423967593:7779]
2025-10-09T21:47:21.621668Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341904423967593:7779]
2025-10-09T21:47:21.621682Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:21.621696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341904785786847:2432], now have 1 active actors on pipe
2025-10-09T21:47:21.621704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:21.621707Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:21.621712Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:21.621715Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:21.621720Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:21.621723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:21.621766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341904423967593:7779]
2025-10-09T21:47:21.621778Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:21.621779Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 916
2025-10-09T21:47:21.621800Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 916
2025-10-09T21:47:21.621795Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:21.621861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:21.621885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:21.621917Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5eea1856-4114de29-33e55505-3c7a2f14_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:21.621930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:21.621934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:21.621937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:21.621948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:21.621965Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:21.621983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:21.622181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:21.622192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:21.622207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:21.622318Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 917 partition: 0 MaxSeqNo: 25 sessionId: src_id|5eea1856-4114de29-33e55505-3c7a2f14_0
2025-10-09T21:47:21.622721Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd515442-31c83a83-c08802cf-95e6962_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5eea1856-4114de29-33e55505-3c7a2f14_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:21.625366Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5eea1856-4114de29-33e55505-3c7a2f14_0] Write session: try to update token
2025-10-09T21:47:21.625386Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5eea1856-4114de29-33e55505-3c7a2f14_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:21.640518Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 917 sessionId: src_id|5eea1856-4114de29-33e55505-3c7a2f14_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:21.641797Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:21.641812Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:21.641821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341904423967593:7779]
2025-10-09T21:47:21.641882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:21.641889Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:21.642079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:21.642226Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:21.642386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:21.642522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:21.642725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:21.642774Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:21.642994Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:21.643182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:21.643197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:21.643230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:21.643295Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:21.658227Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:21.658243Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:21.658246Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:21.658249Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:21.658259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341904423967593:7779]
2025-10-09T21:47:21.658263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341904423967593:7779]
2025-10-09T21:47:21.658267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341904423967593:7779]
2025-10-09T21:47:21.658313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:21.658320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.658323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:21.658328Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:21.658335Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:21.658339Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:21.658474Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:21.688087Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341904785787097:8525]
2025-10-09T21:47:21.688397Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 916, pipe: [1:7559341904785787097:8525]
2025-10-09T21:47:21.819325Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:21.828051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:21.828081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:21.828260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:21.828332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:21.828410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:21.828502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:21.829118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:21.829133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:21.829219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:21.829295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:21.829373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:21.829445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:21.829501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:21.829504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:21.829582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:21.829660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:21.829734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:21.829817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:21.829890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:21.829904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:21.829979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:21.830169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:21.830245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:21.830320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:21.830366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:21.830375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:21.830446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:21.830521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:21.830600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:21.830677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:21.830724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:21.830735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:21.830809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:21.830889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:21.830966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:21.831039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:21.831093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:21.831104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:21.831184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:21.831251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:21.831352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:21.831431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:21.831499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:21.831511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:21.831589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:21.833003Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341904785787097:8525]
2025-10-09T21:47:21.833005Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:21.833009Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:21.833036Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:21.833044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 917 reason: error at write: Disconnected sessionId: src_id|5eea1856-4114de29-33e55505-3c7a2f14_0
2025-10-09T21:47:21.833272Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 917 sessionId: src_id|5eea1856-4114de29-33e55505-3c7a2f14_0 is DEAD
2025-10-09T21:47:21.833409Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5eea1856-4114de29-33e55505-3c7a2f14_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:21.833420Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5eea1856-4114de29-33e55505-3c7a2f14_0] Write session will restart in 0.008057s
2025-10-09T21:47:21.833446Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5eea1856-4114de29-33e55505-3c7a2f14_0] Write session: Do CDS request
2025-10-09T21:47:21.833452Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5eea1856-4114de29-33e55505-3c7a2f14_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:21.831664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:21.831738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:21.831804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:21.831941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:21.831944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:21.832022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:21.832095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:21.832157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:21.832228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:21.832272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:21.832275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:21.832354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:21.832416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:21.832487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:21.832560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:21.832608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:21.832612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:21.832692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341904785787097:8525], now have 1 active actors on pipe
2025-10-09T21:47:21.833045Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:21.842609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5eea1856-4114de29-33e55505-3c7a2f14_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:21.842742Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:21.842770Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 918
2025-10-09T21:47:21.842914Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 918 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:21.842950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 918 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:21.842961Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=918 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:21.842965Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:21.843400Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 918 sessionId:  describe result for acl check
2025-10-09T21:47:21.843442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:21.843444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:21.843446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:21.843453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341904785787135:8624] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:21.844067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341904785787135:8624] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:21.848195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341904785787135:8624] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:21.851803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341904785787135:8624] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:21.851820Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341904785787135:8624] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:21.851823Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341904785787135:8624] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:21.851836Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 918 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:21.855374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:21.858490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:21.858848Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:21.858861Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:21.858878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:21.858953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:21.859004Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341904423967840:7783]
2025-10-09T21:47:21.859590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:21.859953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:21.860013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:21.860347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:21.860407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:21.860412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:21.860413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:21.860414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:21.860415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:21.860417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:21.860418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.860430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.860432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.860438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:21.860449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:21.860453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:21.860456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:21.860460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:21.860464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:21.860467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.860470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.860473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.860482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:21.860490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:21.860496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:21.860500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:21.860504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:21.860509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:21.860513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:21.860518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:21.860522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:21.860526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:21.860540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:21.860549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:21.860555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.860560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.860566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.860570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.860575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.860583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.860611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:21.868735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:21.870211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:21.870228Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:21.870230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:21.870235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:21.870242Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 917 [2:7559341904423967840:7783]
2025-10-09T21:47:21.870258Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:21.870279Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:21.870282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:21.870285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:21.870287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:21.870306Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:21.870313Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:21.870328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:21.870395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.870401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.870403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.870407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:21.870487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:21.870495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.870508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:21.870519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:21.870536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:21.870562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341904423967835:7783]
2025-10-09T21:47:21.870564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341904423967835:7783]
2025-10-09T21:47:21.870567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341904423967835:7783]
2025-10-09T21:47:21.870585Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:21.870600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:21.870617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:21.870622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:21.870666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341904423967835:7783]
2025-10-09T21:47:21.870666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:21.870668Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:21.870670Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:21.870675Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:21.870682Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:21.870965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341904785787317:8624], now have 1 active actors on pipe
2025-10-09T21:47:21.870977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341904785787105:2432], now have 1 active actors on pipe
2025-10-09T21:47:21.871182Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 917
2025-10-09T21:47:21.871234Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 917
2025-10-09T21:47:21.871420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:21.871433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:21.871467Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2d3c24e5-f8d4d5e7-6887d986-cda5310e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:21.871482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:21.871490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:21.871492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:21.871504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:21.871518Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:21.871543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:21.872016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:21.872027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:21.872046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:21.872226Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 918 partition: 0 MaxSeqNo: 25 sessionId: src_id|2d3c24e5-f8d4d5e7-6887d986-cda5310e_0
2025-10-09T21:47:21.872658Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5eea1856-4114de29-33e55505-3c7a2f14_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2d3c24e5-f8d4d5e7-6887d986-cda5310e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:21.875263Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d3c24e5-f8d4d5e7-6887d986-cda5310e_0] Write session: try to update token
2025-10-09T21:47:21.875297Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d3c24e5-f8d4d5e7-6887d986-cda5310e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:21.884669Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:21.884683Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:21.884693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341904423967835:7783]
2025-10-09T21:47:21.884742Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:21.884746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:21.884930Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:21.885082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:21.885222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:21.885358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:21.885547Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:21.885593Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:21.891501Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 918 sessionId: src_id|2d3c24e5-f8d4d5e7-6887d986-cda5310e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:21.894049Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:21.894439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:21.894459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:21.894510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:21.894622Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:21.902600Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:21.902614Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:21.902617Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:21.902619Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:21.902632Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341904423967835:7783]
2025-10-09T21:47:21.902636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341904423967835:7783]
2025-10-09T21:47:21.902639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341904423967835:7783]
2025-10-09T21:47:21.902694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:21.902705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:21.902710Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:21.902714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:21.902716Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:21.902723Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:21.902870Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:21.933282Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341904785787356:8525]
2025-10-09T21:47:21.933575Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 917, pipe: [1:7559341904785787356:8525]
2025-10-09T21:47:22.063063Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:22.071431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:22.071464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:22.071648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:22.071722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:22.071793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:22.071852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:22.071892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:22.071898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:22.071964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:22.072022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:22.072079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:22.072148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:22.072189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:22.072192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:22.072260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:22.072318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:22.072380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:22.072449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:22.072487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:22.072490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:22.072561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:22.072627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:22.072689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:22.072758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:22.072800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:22.072803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:22.072874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:22.072935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:22.073004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:22.073073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:22.073121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:22.073125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:22.073192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:22.073256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:22.073314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:22.073375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:22.073419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:22.073422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:22.073488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:22.073546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:22.073607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:22.073666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:22.073706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:22.073709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:22.073777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:22.073834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:22.073912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:22.073974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:22.074016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:22.074023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:22.074332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:22.074396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:22.074464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:22.074527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:22.074573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:22.074582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:22.074651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:22.075226Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341904785787356:8525]
2025-10-09T21:47:22.075234Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:22.075241Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:22.075276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 918 reason: error at write: Disconnected sessionId: src_id|2d3c24e5-f8d4d5e7-6887d986-cda5310e_0
2025-10-09T21:47:22.075282Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:22.075480Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 918 sessionId: src_id|2d3c24e5-f8d4d5e7-6887d986-cda5310e_0 is DEAD
2025-10-09T21:47:22.075887Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d3c24e5-f8d4d5e7-6887d986-cda5310e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:22.075898Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d3c24e5-f8d4d5e7-6887d986-cda5310e_0] Write session will restart in 0.007881s
2025-10-09T21:47:22.075925Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d3c24e5-f8d4d5e7-6887d986-cda5310e_0] Write session: Do CDS request
2025-10-09T21:47:22.075931Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d3c24e5-f8d4d5e7-6887d986-cda5310e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:22.074713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:22.074779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:22.074840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:22.074891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:22.074896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:22.074968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341904785787356:8525], now have 1 active actors on pipe
2025-10-09T21:47:22.076168Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:22.084785Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d3c24e5-f8d4d5e7-6887d986-cda5310e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:22.085643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:22.085665Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 919
2025-10-09T21:47:22.085853Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 919 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:22.085925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 919 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:22.085936Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=919 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:22.085941Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:22.086504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 919 sessionId:  describe result for acl check
2025-10-09T21:47:22.086549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:22.086555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:22.086560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:22.086566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341909080754670:8628] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:22.087189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341909080754670:8628] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:22.092706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341909080754670:8628] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:22.099765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341909080754670:8628] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:22.099787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341909080754670:8628] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:22.099791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341909080754670:8628] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:22.099802Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 919 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:22.107018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:22.112731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:22.112995Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:22.113008Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:22.113021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:22.113081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:22.113126Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341908718935378:7785]
2025-10-09T21:47:22.113770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:22.114225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:22.114282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:22.114628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:22.114698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:22.114706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:22.114707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:22.114708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:22.114709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:22.114710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:22.114712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.114713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.114714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.114720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:22.114732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:22.114737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:22.114742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:22.114746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:22.114751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:22.114756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.114760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.114765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.114775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:22.114783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:22.114790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:22.114803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:22.114807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:22.114813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:22.114818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:22.114822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:22.114826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:22.114832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:22.114836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:22.114840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:22.114847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.114852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.114856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.114861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.114866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.114870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.114900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:22.125403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:22.126950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:22.126956Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:22.126959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:22.126964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:22.126973Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 918 [2:7559341908718935378:7785]
2025-10-09T21:47:22.126987Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:22.127007Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:22.127011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:22.127015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:22.127017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:22.127036Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:22.127046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:22.127056Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:22.127123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.127125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.127127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.127129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:22.127212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:22.127215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.127230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.127243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.127262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:22.127284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341908718935373:7785]
2025-10-09T21:47:22.127286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341908718935373:7785]
2025-10-09T21:47:22.127289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341908718935373:7785]
2025-10-09T21:47:22.127306Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:22.127324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:22.127339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:22.127342Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:22.127383Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:22.127387Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:22.127389Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341908718935373:7785]
2025-10-09T21:47:22.127390Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:22.127392Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:22.127406Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:22.127610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341909080754812:8628], now have 1 active actors on pipe
2025-10-09T21:47:22.127623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341909080754655:2432], now have 1 active actors on pipe
2025-10-09T21:47:22.128321Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 918
2025-10-09T21:47:22.128358Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 918
2025-10-09T21:47:22.128561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:22.128575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:22.128609Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|eff6f2f5-6fe86df-a07f23f8-ab6fc541_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:22.128625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:22.128631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:22.128634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:22.128647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:22.128662Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:22.128680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:22.128941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:22.128951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:22.128968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:22.129205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 919 partition: 0 MaxSeqNo: 25 sessionId: src_id|eff6f2f5-6fe86df-a07f23f8-ab6fc541_0
2025-10-09T21:47:22.129627Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d3c24e5-f8d4d5e7-6887d986-cda5310e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|eff6f2f5-6fe86df-a07f23f8-ab6fc541_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:22.132201Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eff6f2f5-6fe86df-a07f23f8-ab6fc541_0] Write session: try to update token
2025-10-09T21:47:22.132221Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eff6f2f5-6fe86df-a07f23f8-ab6fc541_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:22.145385Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:22.145406Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:22.145408Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:22.145410Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:22.145419Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341908718935373:7785]
2025-10-09T21:47:22.145423Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341908718935373:7785]
2025-10-09T21:47:22.145426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341908718935373:7785]
2025-10-09T21:47:22.145487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:22.145488Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:22.145494Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:22.145498Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:22.145499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.145503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:22.145653Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:22.155494Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 919 sessionId: src_id|eff6f2f5-6fe86df-a07f23f8-ab6fc541_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:22.157976Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:22.173976Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341909080754910:8525]
2025-10-09T21:47:22.175701Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 918, pipe: [1:7559341909080754910:8525]
2025-10-09T21:47:22.308527Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:22.309171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:22.309188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:22.309212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341909080754910:8525], now have 1 active actors on pipe
2025-10-09T21:47:22.309477Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:22.310025Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eff6f2f5-6fe86df-a07f23f8-ab6fc541_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:22.310064Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eff6f2f5-6fe86df-a07f23f8-ab6fc541_0] Write session will restart in 0.005715s
2025-10-09T21:47:22.310090Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eff6f2f5-6fe86df-a07f23f8-ab6fc541_0] Write session: Do CDS request
2025-10-09T21:47:22.310097Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eff6f2f5-6fe86df-a07f23f8-ab6fc541_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:22.309521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341909080754910:8525]
2025-10-09T21:47:22.309570Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:22.309573Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:22.309647Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 919 reason: error at write: Disconnected sessionId: src_id|eff6f2f5-6fe86df-a07f23f8-ab6fc541_0
2025-10-09T21:47:22.309855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 919 sessionId: src_id|eff6f2f5-6fe86df-a07f23f8-ab6fc541_0 is DEAD
2025-10-09T21:47:22.309904Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:22.316554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eff6f2f5-6fe86df-a07f23f8-ab6fc541_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:22.316964Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:22.316981Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 920
2025-10-09T21:47:22.317193Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 920 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:22.317231Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 920 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:22.317237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=920 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:22.317241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:22.318497Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 920 sessionId:  describe result for acl check
2025-10-09T21:47:22.318557Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:22.318563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:22.318565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:22.318571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341909080754926:8633] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:22.319190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341909080754926:8633] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:22.326927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341909080754926:8633] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:22.343820Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341909080754926:8633] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:22.343846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341909080754926:8633] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:22.343850Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341909080754926:8633] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:22.343860Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 920 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:22.352749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:22.357197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:22.357466Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:22.357479Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:22.357493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:22.357554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:22.357596Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341908718935652:7797]
2025-10-09T21:47:22.358294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:22.358757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:22.358805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:22.359134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:22.359190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:22.359197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:22.359199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:22.359200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:22.359201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:22.359202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:22.359204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.359205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.359206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.359212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:22.359222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:22.359225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:22.359229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:22.359234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:22.359238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:22.359241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.359245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.359250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.359256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:22.359265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:22.359271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:22.359277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:22.359281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:22.359287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:22.359292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:22.359296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:22.359300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:22.359304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:22.359309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:22.359314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:22.359319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.359324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.359329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.359334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.359339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.359345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.359373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:22.366725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:22.368474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:22.368488Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:22.368491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:22.368495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:22.368503Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 919 [2:7559341908718935652:7797]
2025-10-09T21:47:22.368518Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:22.368536Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:22.368540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:22.368543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:22.368546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:22.368564Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:22.368570Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:22.368578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:22.368640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.368651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.368653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.368656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:22.368987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:22.368997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.369012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.369026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.369049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:22.369074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341909080755058:8633], now have 1 active actors on pipe
2025-10-09T21:47:22.369079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341909080754917:2432], now have 1 active actors on pipe
2025-10-09T21:47:22.369087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341908718935647:7797]
2025-10-09T21:47:22.369095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341908718935647:7797]
2025-10-09T21:47:22.369097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341908718935647:7797]
2025-10-09T21:47:22.369114Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:22.369133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:22.369136Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:22.369141Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:22.369143Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:22.369150Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:22.369153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:22.369196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341908718935647:7797]
2025-10-09T21:47:22.369209Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:22.369229Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:22.370111Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 919
2025-10-09T21:47:22.370151Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 919
2025-10-09T21:47:22.370410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:22.370425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:22.370462Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|72fac66f-160bed93-8d0b2212-9495ecf3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:22.370473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:22.370478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:22.370481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:22.370495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:22.370510Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:22.370530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:22.374328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:22.374345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:22.374389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:22.374579Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 920 partition: 0 MaxSeqNo: 25 sessionId: src_id|72fac66f-160bed93-8d0b2212-9495ecf3_0
2025-10-09T21:47:22.374981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eff6f2f5-6fe86df-a07f23f8-ab6fc541_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|72fac66f-160bed93-8d0b2212-9495ecf3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:22.377634Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72fac66f-160bed93-8d0b2212-9495ecf3_0] Write session: try to update token
2025-10-09T21:47:22.377652Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72fac66f-160bed93-8d0b2212-9495ecf3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:22.385129Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:22.385145Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:22.385155Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341908718935647:7797]
2025-10-09T21:47:22.385204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:22.385215Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:22.385372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:22.387457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:22.387609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:22.387747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:22.387926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:22.387973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:22.395481Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:22.395498Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:22.395501Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:22.395503Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:22.395513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341908718935647:7797]
2025-10-09T21:47:22.395516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341908718935647:7797]
2025-10-09T21:47:22.395520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341908718935647:7797]
2025-10-09T21:47:22.395565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:22.395572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.395575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:22.395576Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:22.395582Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:22.395585Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:22.395877Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:22.399328Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 920 sessionId: src_id|72fac66f-160bed93-8d0b2212-9495ecf3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:22.401527Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:22.409436Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341909080755192:8525]
2025-10-09T21:47:22.409715Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 919, pipe: [1:7559341909080755192:8525]
2025-10-09T21:47:22.529581Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: CLIENT_DEADLINE_EXCEEDED. Description: [ { <main>: Error: GRpc error: (4): Deadline Exceeded } { <main>: Error: Grpc error response on endpoint localhost:14704 } ]
2025-10-09T21:47:22.555023Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:22.555558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:22.555577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:22.555603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341909080755192:8525], now have 1 active actors on pipe
2025-10-09T21:47:22.556016Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:22.556016Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:22.556031Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:22.556105Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 920 reason: error at write: Disconnected sessionId: src_id|72fac66f-160bed93-8d0b2212-9495ecf3_0
2025-10-09T21:47:22.556327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 920 sessionId: src_id|72fac66f-160bed93-8d0b2212-9495ecf3_0 is DEAD
2025-10-09T21:47:22.556501Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341909080755192:8525]
2025-10-09T21:47:22.556515Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:22.556788Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72fac66f-160bed93-8d0b2212-9495ecf3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:22.556799Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72fac66f-160bed93-8d0b2212-9495ecf3_0] Write session will restart in 0.008048s
2025-10-09T21:47:22.556826Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72fac66f-160bed93-8d0b2212-9495ecf3_0] Write session: Do CDS request
2025-10-09T21:47:22.556832Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72fac66f-160bed93-8d0b2212-9495ecf3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:22.566553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72fac66f-160bed93-8d0b2212-9495ecf3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:22.570108Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:22.570130Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 921
2025-10-09T21:47:22.570338Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 921 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:22.570370Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 921 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:22.570376Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=921 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:22.570380Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:22.571105Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 921 sessionId:  describe result for acl check
2025-10-09T21:47:22.571148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:22.571150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:22.571152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:22.571158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341909080755255:8644] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:22.571707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341909080755255:8644] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:22.574140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341909080755255:8644] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:22.579574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341909080755255:8644] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:22.579593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341909080755255:8644] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:22.579597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341909080755255:8644] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:22.579607Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 921 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:22.583592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:22.586681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:22.586959Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:22.586968Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:22.586982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:22.587046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:22.587090Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341908718935905:7801]
2025-10-09T21:47:22.587670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:22.588057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:22.588105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:22.588428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:22.588487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:22.588494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:22.588496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:22.588497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:22.588499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:22.588501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:22.588502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.588503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.588504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.588510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:22.588521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:22.588526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:22.588529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:22.588533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:22.588537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:22.588541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.588545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.588548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.588558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:22.588566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:22.588572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:22.588578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:22.588582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:22.588587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:22.588592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:22.588597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:22.588601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:22.588606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:22.588611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:22.588615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:22.588621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.588626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.588630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.588635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.588640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.588644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.588677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:22.597770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:22.599295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:22.599313Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:22.599316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:22.599321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:22.599329Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 920 [2:7559341908718935905:7801]
2025-10-09T21:47:22.599347Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:22.599370Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:22.599374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:22.599378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:22.599381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:22.599400Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:22.599407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:22.599415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:22.599483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.599491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.599494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.599497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:22.599603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:22.599611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.599626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.599636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.599651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:22.599674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341908718935900:7801]
2025-10-09T21:47:22.599675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341908718935900:7801]
2025-10-09T21:47:22.599677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341908718935900:7801]
2025-10-09T21:47:22.599686Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:22.599703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:22.599716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:22.599719Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:22.599740Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:22.599743Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:22.599745Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:22.599764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341908718935900:7801]
2025-10-09T21:47:22.599767Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:22.599767Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:22.599791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341909080755409:8644], now have 1 active actors on pipe
2025-10-09T21:47:22.599795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341909080755195:2432], now have 1 active actors on pipe
2025-10-09T21:47:22.600010Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 920
2025-10-09T21:47:22.600043Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 920
2025-10-09T21:47:22.600179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:22.600191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:22.600226Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|54dff67c-6e21dccd-4bc0fe68-91cf0f0e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:22.600236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:22.600240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:22.600243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:22.600257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:22.600269Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:22.600288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:22.600599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:22.600609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:22.600625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:22.600755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 921 partition: 0 MaxSeqNo: 25 sessionId: src_id|54dff67c-6e21dccd-4bc0fe68-91cf0f0e_0
2025-10-09T21:47:22.601132Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72fac66f-160bed93-8d0b2212-9495ecf3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|54dff67c-6e21dccd-4bc0fe68-91cf0f0e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:22.603757Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54dff67c-6e21dccd-4bc0fe68-91cf0f0e_0] Write session: try to update token
2025-10-09T21:47:22.603781Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54dff67c-6e21dccd-4bc0fe68-91cf0f0e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:22.616142Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:22.616158Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:22.616160Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:22.616162Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:22.616172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341908718935900:7801]
2025-10-09T21:47:22.616177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341908718935900:7801]
2025-10-09T21:47:22.616181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341908718935900:7801]
2025-10-09T21:47:22.616231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:22.616239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.616241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:22.616247Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:22.616253Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:22.616267Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:22.616383Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:22.627052Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 921 sessionId: src_id|54dff67c-6e21dccd-4bc0fe68-91cf0f0e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:22.629638Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:22.654705Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341909080755448:8525]
2025-10-09T21:47:22.655060Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 920, pipe: [1:7559341909080755448:8525]
2025-10-09T21:47:22.778612Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:22.779211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:22.779231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:22.779257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341909080755448:8525], now have 1 active actors on pipe
2025-10-09T21:47:22.779703Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:22.779949Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:22.780030Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 921 reason: status is not ok: tablet will be restarted right now sessionId: src_id|54dff67c-6e21dccd-4bc0fe68-91cf0f0e_0
2025-10-09T21:47:22.780226Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 921 sessionId: src_id|54dff67c-6e21dccd-4bc0fe68-91cf0f0e_0 is DEAD
2025-10-09T21:47:22.780376Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:22.780409Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341909080755448:8525]
2025-10-09T21:47:22.782366Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54dff67c-6e21dccd-4bc0fe68-91cf0f0e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:22.782378Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54dff67c-6e21dccd-4bc0fe68-91cf0f0e_0] Write session will restart in 0.008654s
2025-10-09T21:47:22.782406Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54dff67c-6e21dccd-4bc0fe68-91cf0f0e_0] Write session: Do CDS request
2025-10-09T21:47:22.782413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54dff67c-6e21dccd-4bc0fe68-91cf0f0e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:22.794512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54dff67c-6e21dccd-4bc0fe68-91cf0f0e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:22.796239Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:22.796256Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 922
2025-10-09T21:47:22.796421Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 922 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:22.796455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 922 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:22.796466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=922 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:22.796470Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:22.797014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 922 sessionId:  describe result for acl check
2025-10-09T21:47:22.797054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:22.797055Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:22.797058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:22.797066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341909080755509:8649] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:22.797626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341909080755509:8649] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:22.803135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341909080755509:8649] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:22.810154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341909080755509:8649] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:22.810173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341909080755509:8649] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:22.810177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341909080755509:8649] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:22.810187Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 922 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:22.812335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:22.815509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:22.815760Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:22.815765Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:22.815779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:22.815846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:22.815890Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341908718936154:7805]
2025-10-09T21:47:22.816499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:22.819439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:22.819512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:22.820371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:22.820462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:22.820464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:22.820466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:22.820467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:22.820469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:22.820470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:22.820472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.820473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.820475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.820481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:22.820493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:22.820497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:22.820502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:22.820505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:22.820509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:22.820513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.820517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.820521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.820530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:22.820539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:22.820546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:22.820552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:22.820557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:22.820562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:22.820566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:22.820571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:22.820577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:22.820582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:22.820586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:22.820592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:22.820597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.820605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.820610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.820615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.820619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.820624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.820657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:22.830943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:22.832105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:22.832113Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:22.832117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:22.832122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:22.832130Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 921 [2:7559341908718936154:7805]
2025-10-09T21:47:22.832145Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:22.832168Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:22.832172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:22.832176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:22.832178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:22.832571Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 921
2025-10-09T21:47:22.832198Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:22.832616Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 921
2025-10-09T21:47:22.832205Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:22.832212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:22.832281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.832284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.832286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.832289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:22.832370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:22.832373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.832387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:22.832400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:22.832417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:22.832441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341909080755692:8649], now have 1 active actors on pipe
2025-10-09T21:47:22.832444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341909080755453:2432], now have 1 active actors on pipe
2025-10-09T21:47:22.832453Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341908718936149:7805]
2025-10-09T21:47:22.832456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341908718936149:7805]
2025-10-09T21:47:22.832458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341908718936149:7805]
2025-10-09T21:47:22.832472Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:22.832489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:22.832505Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:22.832509Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:22.832552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341908718936149:7805]
2025-10-09T21:47:22.832555Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:22.832604Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:22.832607Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:22.832609Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:22.832617Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:22.833151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:22.833166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:22.833206Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3d3da278-1a9eb519-e8f7a16f-3c614e64_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:22.833220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:22.833229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:22.833233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:22.833246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:22.833263Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:22.833289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:22.833793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:22.833804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:22.833823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:22.833946Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 922 partition: 0 MaxSeqNo: 25 sessionId: src_id|3d3da278-1a9eb519-e8f7a16f-3c614e64_0
2025-10-09T21:47:22.834402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54dff67c-6e21dccd-4bc0fe68-91cf0f0e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3d3da278-1a9eb519-e8f7a16f-3c614e64_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:22.836953Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3da278-1a9eb519-e8f7a16f-3c614e64_0] Write session: try to update token
2025-10-09T21:47:22.836977Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3da278-1a9eb519-e8f7a16f-3c614e64_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:22.851871Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:22.851887Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:22.851889Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:22.851892Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:22.851917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341908718936149:7805]
2025-10-09T21:47:22.851922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341908718936149:7805]
2025-10-09T21:47:22.851927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341908718936149:7805]
2025-10-09T21:47:22.851980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:22.851981Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:22.851987Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:22.851991Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:22.851991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:22.851994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:22.852147Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:22.860407Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 922 sessionId: src_id|3d3da278-1a9eb519-e8f7a16f-3c614e64_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:22.863113Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:22.880545Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341909080755708:8525]
2025-10-09T21:47:22.880939Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 921, pipe: [1:7559341909080755708:8525]
2025-10-09T21:47:23.013190Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:23.013718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:23.013737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:23.013762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341909080755708:8525], now have 1 active actors on pipe
2025-10-09T21:47:23.013998Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:23.014172Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:23.014263Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 922 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3d3da278-1a9eb519-e8f7a16f-3c614e64_0
2025-10-09T21:47:23.014461Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 922 sessionId: src_id|3d3da278-1a9eb519-e8f7a16f-3c614e64_0 is DEAD
2025-10-09T21:47:23.014528Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341909080755708:8525]
2025-10-09T21:47:23.014558Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:23.018305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3da278-1a9eb519-e8f7a16f-3c614e64_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:23.018317Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3da278-1a9eb519-e8f7a16f-3c614e64_0] Write session will restart in 0.006726s
2025-10-09T21:47:23.018347Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3da278-1a9eb519-e8f7a16f-3c614e64_0] Write session: Do CDS request
2025-10-09T21:47:23.018353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3da278-1a9eb519-e8f7a16f-3c614e64_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:23.026604Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3da278-1a9eb519-e8f7a16f-3c614e64_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:23.026993Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:23.027013Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 923
2025-10-09T21:47:23.027143Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 923 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:23.027174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 923 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:23.027180Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=923 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:23.027195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:23.027916Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 923 sessionId:  describe result for acl check
2025-10-09T21:47:23.027958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:23.027966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:23.027968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:23.027976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341913375723043:8653] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:23.028593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341913375723043:8653] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:23.033331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341913375723043:8653] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:23.037615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341913375723043:8653] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:23.037636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341913375723043:8653] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:23.037642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341913375723043:8653] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:23.037652Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 923 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:23.045977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:23.049193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:23.049486Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:23.049494Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:23.049507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:23.049566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:23.049705Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341913013903696:7807]
2025-10-09T21:47:23.050381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:23.054597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:23.054718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:23.055142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:23.055216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:23.055218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:23.055219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:23.055221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:23.055223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:23.055224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:23.055226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.055227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.055229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.055237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:23.055249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:23.055254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:23.055258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:23.055263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:23.055267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:23.055272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.055276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.055280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.055290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:23.055300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:23.055308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:23.055315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:23.055323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:23.055330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:23.055335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:23.055341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:23.055346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:23.055352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:23.055357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:23.055363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:23.055370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.055376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.055381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.055387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.055395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.055401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.055439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:23.065326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:23.066557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:23.066579Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:23.066582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:23.066587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:23.066595Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 922 [2:7559341913013903696:7807]
2025-10-09T21:47:23.066613Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:23.066637Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:23.066641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:23.066645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:23.066648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:23.066684Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:23.066697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:23.066706Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:23.066793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.066804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.066806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.066809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:23.066894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:23.066903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.066918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.066933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.066954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:23.066982Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341913013903691:7807]
2025-10-09T21:47:23.066990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341913013903691:7807]
2025-10-09T21:47:23.066993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341913013903691:7807]
2025-10-09T21:47:23.067008Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:23.067029Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:23.067051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:23.067058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:23.067108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341913013903691:7807]
2025-10-09T21:47:23.067116Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:23.067455Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:23.067463Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:23.067465Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:23.067474Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:23.068052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341913375723009:2432], now have 1 active actors on pipe
2025-10-09T21:47:23.068059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341913375723158:8653], now have 1 active actors on pipe
2025-10-09T21:47:23.068475Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 922
2025-10-09T21:47:23.068526Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 922
2025-10-09T21:47:23.068737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:23.068752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:23.068787Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|99bc1d10-280f0ef8-f8bd414-3e966803_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:23.068801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:23.068806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:23.068809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:23.068824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:23.068842Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:23.068862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:23.069182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:23.069191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:23.069209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:23.069388Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 923 partition: 0 MaxSeqNo: 25 sessionId: src_id|99bc1d10-280f0ef8-f8bd414-3e966803_0
2025-10-09T21:47:23.069965Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3da278-1a9eb519-e8f7a16f-3c614e64_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|99bc1d10-280f0ef8-f8bd414-3e966803_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:23.072578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99bc1d10-280f0ef8-f8bd414-3e966803_0] Write session: try to update token
2025-10-09T21:47:23.072599Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99bc1d10-280f0ef8-f8bd414-3e966803_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:23.090284Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:23.090305Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:23.090307Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:23.090310Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:23.090323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341913013903691:7807]
2025-10-09T21:47:23.090328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341913013903691:7807]
2025-10-09T21:47:23.090333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341913013903691:7807]
2025-10-09T21:47:23.090386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:23.090395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.090398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:23.090541Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:23.094722Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:23.094742Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:23.094746Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:23.099077Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 923 sessionId: src_id|99bc1d10-280f0ef8-f8bd414-3e966803_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:23.101761Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:23.114801Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341913375723264:8525]
2025-10-09T21:47:23.115256Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 922, pipe: [1:7559341913375723264:8525]
2025-10-09T21:47:23.331239Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:23.331753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:23.331763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:23.331788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341913375723264:8525], now have 1 active actors on pipe
2025-10-09T21:47:23.332061Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:23.332892Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341913375723264:8525]
2025-10-09T21:47:23.332933Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:23.332969Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:23.333055Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 923 reason: status is not ok: tablet will be restarted right now sessionId: src_id|99bc1d10-280f0ef8-f8bd414-3e966803_0
2025-10-09T21:47:23.333261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 923 sessionId: src_id|99bc1d10-280f0ef8-f8bd414-3e966803_0 is DEAD
2025-10-09T21:47:23.334652Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99bc1d10-280f0ef8-f8bd414-3e966803_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:23.334665Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99bc1d10-280f0ef8-f8bd414-3e966803_0] Write session will restart in 0.007015s
2025-10-09T21:47:23.334693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99bc1d10-280f0ef8-f8bd414-3e966803_0] Write session: Do CDS request
2025-10-09T21:47:23.334700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99bc1d10-280f0ef8-f8bd414-3e966803_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:23.342941Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99bc1d10-280f0ef8-f8bd414-3e966803_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:23.343055Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:23.343086Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 924
2025-10-09T21:47:23.343186Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 924 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:23.343214Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 924 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:23.343219Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=924 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:23.343223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:23.343791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 924 sessionId:  describe result for acl check
2025-10-09T21:47:23.343824Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:23.343826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:23.343827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:23.343833Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341913375723291:8658] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:23.344418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341913375723291:8658] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:23.346852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341913375723291:8658] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:23.354851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341913375723291:8658] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:23.354879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341913375723291:8658] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:23.354884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341913375723291:8658] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:23.354895Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 924 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:23.362976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:23.371083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:23.371392Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:23.371397Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:23.371414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:23.371493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:23.371534Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341913013903974:7819]
2025-10-09T21:47:23.372137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:23.372531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:23.372606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:23.372979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:23.373037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:23.373043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:23.373044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:23.373046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:23.373047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:23.373048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:23.373050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.373051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.373052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.373058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:23.373070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:23.373074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:23.373078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:23.373082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:23.373087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:23.373091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.373095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.373099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.373107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:23.373115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:23.373121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:23.373127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:23.373133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:23.373138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:23.373144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:23.373154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:23.373159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:23.373164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:23.373170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:23.373180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:23.373189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.373195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.373206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.373211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.373216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.373221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.373262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:23.381784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:23.383442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:23.383458Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:23.383461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:23.383465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:23.383473Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 923 [2:7559341913013903974:7819]
2025-10-09T21:47:23.383490Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:23.383511Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:23.383515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:23.383518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:23.383521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:23.383554Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:23.383565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:23.383574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:23.383645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.383651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.383653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.383656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:23.383736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:23.383742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.383757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.383767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.383782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:23.383807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341913375723431:8658], now have 1 active actors on pipe
2025-10-09T21:47:23.383815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341913013903969:7819]
2025-10-09T21:47:23.383817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341913013903969:7819]
2025-10-09T21:47:23.383819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341913013903969:7819]
2025-10-09T21:47:23.383832Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:23.383843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341913375723275:2432], now have 1 active actors on pipe
2025-10-09T21:47:23.383852Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:23.383866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:23.383873Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:23.383875Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:23.383880Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:23.383882Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:23.383922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341913013903969:7819]
2025-10-09T21:47:23.383931Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:23.383991Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 923
2025-10-09T21:47:23.383992Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 923
2025-10-09T21:47:23.384141Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:23.384673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 924 partition: 0 MaxSeqNo: 25 sessionId: src_id|dda40d11-7728ecaa-dff9f029-e79c12fb_0
2025-10-09T21:47:23.385060Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99bc1d10-280f0ef8-f8bd414-3e966803_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dda40d11-7728ecaa-dff9f029-e79c12fb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:23.384295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:23.384308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:23.384349Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dda40d11-7728ecaa-dff9f029-e79c12fb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:23.384359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:23.384364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:23.384366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:23.384378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:23.384389Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:23.384406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:23.384572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:23.384576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:23.384592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:23.387735Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dda40d11-7728ecaa-dff9f029-e79c12fb_0] Write session: try to update token
2025-10-09T21:47:23.387754Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dda40d11-7728ecaa-dff9f029-e79c12fb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:23.400191Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:23.400206Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:23.400209Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:23.400210Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:23.400219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341913013903969:7819]
2025-10-09T21:47:23.400224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341913013903969:7819]
2025-10-09T21:47:23.400228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341913013903969:7819]
2025-10-09T21:47:23.400280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:23.400280Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:23.400286Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:23.400289Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:23.400295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.400298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:23.400435Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:23.405353Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 924 sessionId: src_id|dda40d11-7728ecaa-dff9f029-e79c12fb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:23.408105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:23.432938Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341913375723543:8525]
2025-10-09T21:47:23.433221Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 923, pipe: [1:7559341913375723543:8525]
2025-10-09T21:47:23.600852Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:23.601131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:23.601143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:23.601192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341913375723543:8525], now have 1 active actors on pipe
2025-10-09T21:47:23.601616Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:23.601632Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:23.601643Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341913375723543:8525]
2025-10-09T21:47:23.601674Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:23.601709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 924 reason: error at write: Disconnected sessionId: src_id|dda40d11-7728ecaa-dff9f029-e79c12fb_0
2025-10-09T21:47:23.601469Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:23.602245Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dda40d11-7728ecaa-dff9f029-e79c12fb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:23.602259Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dda40d11-7728ecaa-dff9f029-e79c12fb_0] Write session will restart in 0.009242s
2025-10-09T21:47:23.602004Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 924 sessionId: src_id|dda40d11-7728ecaa-dff9f029-e79c12fb_0 is DEAD
2025-10-09T21:47:23.602289Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dda40d11-7728ecaa-dff9f029-e79c12fb_0] Write session: Do CDS request
2025-10-09T21:47:23.602307Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dda40d11-7728ecaa-dff9f029-e79c12fb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:23.612521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dda40d11-7728ecaa-dff9f029-e79c12fb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:23.612911Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:23.612928Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 925
2025-10-09T21:47:23.613095Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 925 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:23.613129Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 925 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:23.613135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=925 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:23.613140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:23.613668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 925 sessionId:  describe result for acl check
2025-10-09T21:47:23.613713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:23.613722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:23.613724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:23.613742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341913375723573:8669] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:23.614453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341913375723573:8669] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:23.621065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341913375723573:8669] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:23.625385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341913375723573:8669] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:23.625404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341913375723573:8669] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:23.625408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341913375723573:8669] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:23.625418Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 925 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:23.633582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:23.637577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:23.637840Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:23.637852Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:23.637880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:23.637944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:23.637995Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341913013904235:7825]
2025-10-09T21:47:23.638623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:23.639324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:23.639392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:23.639779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:23.639832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:23.639838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:23.639840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:23.639841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:23.639842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:23.639844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:23.639845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.639847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.639848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.639854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:23.639866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:23.639870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:23.639874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:23.639879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:23.639884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:23.639889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.639893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.639897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.639920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:23.639929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:23.639939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:23.639945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:23.639950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:23.639957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:23.639966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:23.639972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:23.639978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:23.639983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:23.639989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:23.639995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:23.640006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.640012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.640017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.640023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.640029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.640035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.640071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:23.648295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:23.649603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:23.649617Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:23.649619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:23.649624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:23.649631Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 924 [2:7559341913013904235:7825]
2025-10-09T21:47:23.649647Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:23.649668Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:23.649674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:23.649678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:23.649680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:23.649700Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:23.649710Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:23.649717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:23.649783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.649791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.649794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.649797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:23.649885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:23.649892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.649906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.649922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.649942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:23.649968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341913013904230:7825]
2025-10-09T21:47:23.649971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341913013904230:7825]
2025-10-09T21:47:23.649974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341913013904230:7825]
2025-10-09T21:47:23.649984Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:23.649999Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:23.650001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341913375723557:2432], now have 1 active actors on pipe
2025-10-09T21:47:23.650002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:23.650005Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:23.650005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341913375723735:8669], now have 1 active actors on pipe
2025-10-09T21:47:23.650016Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:23.650046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:23.650050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:23.650093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341913013904230:7825]
2025-10-09T21:47:23.650109Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:23.650175Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:23.650456Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 924
2025-10-09T21:47:23.650505Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 924
2025-10-09T21:47:23.650768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:23.650791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:23.650828Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|63a7959b-72bdff1-8170d144-e2895b6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:23.650843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:23.650848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:23.650851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:23.650863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:23.650876Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:23.650899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:23.651277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:23.651286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:23.651304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:23.651465Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 925 partition: 0 MaxSeqNo: 25 sessionId: src_id|63a7959b-72bdff1-8170d144-e2895b6_0
2025-10-09T21:47:23.652028Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dda40d11-7728ecaa-dff9f029-e79c12fb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|63a7959b-72bdff1-8170d144-e2895b6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:23.654583Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63a7959b-72bdff1-8170d144-e2895b6_0] Write session: try to update token
2025-10-09T21:47:23.654602Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63a7959b-72bdff1-8170d144-e2895b6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:23.666532Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:23.666550Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:23.666553Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:23.666556Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:23.666565Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341913013904230:7825]
2025-10-09T21:47:23.666569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341913013904230:7825]
2025-10-09T21:47:23.666573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341913013904230:7825]
2025-10-09T21:47:23.666619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:23.666627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.666645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:23.666670Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:23.666676Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:23.666680Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:23.666820Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:23.674483Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 925 sessionId: src_id|63a7959b-72bdff1-8170d144-e2895b6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:23.677051Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:23.701243Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341913375723815:8525]
2025-10-09T21:47:23.709524Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 924, pipe: [1:7559341913375723815:8525]
2025-10-09T21:47:23.837730Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:23.838196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:23.838213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:23.838267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341913375723815:8525], now have 1 active actors on pipe
2025-10-09T21:47:23.838848Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:23.838910Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:23.838913Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:23.838974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 925 reason: error at write: Disconnected sessionId: src_id|63a7959b-72bdff1-8170d144-e2895b6_0
2025-10-09T21:47:23.839173Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 925 sessionId: src_id|63a7959b-72bdff1-8170d144-e2895b6_0 is DEAD
2025-10-09T21:47:23.839320Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341913375723815:8525]
2025-10-09T21:47:23.839640Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:23.842144Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63a7959b-72bdff1-8170d144-e2895b6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:23.842159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63a7959b-72bdff1-8170d144-e2895b6_0] Write session will restart in 0.009288s
2025-10-09T21:47:23.842188Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63a7959b-72bdff1-8170d144-e2895b6_0] Write session: Do CDS request
2025-10-09T21:47:23.842195Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63a7959b-72bdff1-8170d144-e2895b6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:23.854150Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63a7959b-72bdff1-8170d144-e2895b6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:23.854595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:23.854617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 926
2025-10-09T21:47:23.854746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 926 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:23.854779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 926 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:23.854785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=926 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:23.854790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:23.855572Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 926 sessionId:  describe result for acl check
2025-10-09T21:47:23.855604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:23.855606Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:23.855608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:23.855615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341913375723878:8674] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:23.856168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341913375723878:8674] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:23.863358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341913375723878:8674] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:23.867545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341913375723878:8674] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:23.867568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341913375723878:8674] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:23.867573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341913375723878:8674] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:23.867583Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 926 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:23.869448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:23.872306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:23.872512Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:23.872525Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:23.872536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:23.872584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:23.872625Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341913013904484:7827]
2025-10-09T21:47:23.873253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:23.873618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:23.873689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:23.874072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:23.874146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:23.874154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:23.874155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:23.874157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:23.874158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:23.874159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:23.874161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.874162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.874163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.874169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:23.874180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:23.874183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:23.874187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:23.874190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:23.874194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:23.874198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.874202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.874214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.874224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:23.874231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:23.874236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:23.874241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:23.874244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:23.874250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:23.874256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:23.874261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:23.874266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:23.874271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:23.874275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:23.874281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:23.874300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.874305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.874310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.874315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.874320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.874326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.874355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:23.881394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:23.882671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:23.882692Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:23.882696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:23.882701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:23.882725Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 925 [2:7559341913013904484:7827]
2025-10-09T21:47:23.882740Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:23.882763Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:23.882767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:23.882770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:23.882773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:23.882796Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:23.882810Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:23.882819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:23.882896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.882898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.882901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.882904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:23.882987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:23.882997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.883011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:23.883026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:23.883045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:23.883075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341913013904479:7827]
2025-10-09T21:47:23.883078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341913013904479:7827]
2025-10-09T21:47:23.883080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341913013904479:7827]
2025-10-09T21:47:23.883096Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:23.883116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:23.883136Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:23.883140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:23.883165Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:23.883177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:23.883188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341913013904479:7827]
2025-10-09T21:47:23.883193Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:23.883197Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:23.883201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:23.883478Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 925
2025-10-09T21:47:23.883492Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 925
2025-10-09T21:47:23.883546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341913375723820:2432], now have 1 active actors on pipe
2025-10-09T21:47:23.883557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341913375724064:8674], now have 1 active actors on pipe
2025-10-09T21:47:23.883688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:23.883701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:23.883733Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|32d8a255-a3472ae0-ffdb9234-915bd554_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:23.883746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:23.883751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:23.883754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:23.883766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:23.883780Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:23.883803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:23.884028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:23.884036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:23.884053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:23.884186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 926 partition: 0 MaxSeqNo: 25 sessionId: src_id|32d8a255-a3472ae0-ffdb9234-915bd554_0
2025-10-09T21:47:23.884626Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63a7959b-72bdff1-8170d144-e2895b6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|32d8a255-a3472ae0-ffdb9234-915bd554_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:23.887169Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32d8a255-a3472ae0-ffdb9234-915bd554_0] Write session: try to update token
2025-10-09T21:47:23.887188Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32d8a255-a3472ae0-ffdb9234-915bd554_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:23.901061Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:23.901076Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:23.901079Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:23.901081Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:23.901091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341913013904479:7827]
2025-10-09T21:47:23.901095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341913013904479:7827]
2025-10-09T21:47:23.901100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341913013904479:7827]
2025-10-09T21:47:23.901146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:23.901159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:23.901161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:23.901310Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:23.902840Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:23.902856Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:23.902860Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:23.913219Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 926 sessionId: src_id|32d8a255-a3472ae0-ffdb9234-915bd554_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:23.915738Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:23.942447Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341913375724078:8525]
2025-10-09T21:47:23.943026Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 925, pipe: [1:7559341913375724078:8525]
2025-10-09T21:47:24.082443Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:24.083064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:24.083075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:24.083124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341913375724078:8525], now have 1 active actors on pipe
2025-10-09T21:47:24.086805Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32d8a255-a3472ae0-ffdb9234-915bd554_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:24.086830Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32d8a255-a3472ae0-ffdb9234-915bd554_0] Write session will restart in 0.007471s
2025-10-09T21:47:24.086853Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32d8a255-a3472ae0-ffdb9234-915bd554_0] Write session: Do CDS request
2025-10-09T21:47:24.086860Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32d8a255-a3472ae0-ffdb9234-915bd554_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:24.086106Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:24.086107Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341913375724078:8525]
2025-10-09T21:47:24.086136Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:24.086139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:24.086200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 926 reason: error at write: Disconnected sessionId: src_id|32d8a255-a3472ae0-ffdb9234-915bd554_0
2025-10-09T21:47:24.086392Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 926 sessionId: src_id|32d8a255-a3472ae0-ffdb9234-915bd554_0 is DEAD
2025-10-09T21:47:24.090217Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:24.095658Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32d8a255-a3472ae0-ffdb9234-915bd554_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:24.099251Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:24.099270Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 927
2025-10-09T21:47:24.099636Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 927 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:24.099675Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 927 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:24.099681Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=927 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:24.099685Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:24.100110Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 927 sessionId:  describe result for acl check
2025-10-09T21:47:24.100142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:24.100146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:24.100148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:24.100154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341917670691393:8678] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:24.100734Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341917670691393:8678] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:24.105392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341917670691393:8678] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:24.113077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341917670691393:8678] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:24.113096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341917670691393:8678] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:24.113102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341917670691393:8678] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:24.113113Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 927 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:24.131745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:24.134355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:24.134604Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:24.134618Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:24.134633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:24.134701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:24.134753Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341917308872030:7829]
2025-10-09T21:47:24.135351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:24.135699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:24.135766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:24.136138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:24.136203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:24.136208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:24.136210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:24.136211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:24.136212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:24.136214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:24.136215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.136217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.136218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.136224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:24.136236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:24.136240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:24.136243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:24.136247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:24.136250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:24.136253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.136256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.136260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.136269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:24.136277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:24.136283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:24.136289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:24.136294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:24.136300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:24.136305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:24.136311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:24.136317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:24.136323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:24.136328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:24.136334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:24.136341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.136352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.136357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.136364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.136372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.136378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.136412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:24.144070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:24.145335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:24.145354Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:24.145357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:24.145362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:24.145370Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 926 [2:7559341917308872030:7829]
2025-10-09T21:47:24.145387Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:24.145409Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:24.145420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:24.145423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:24.145426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:24.145446Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:24.145453Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:24.145461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:24.145529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.145539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.145541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.145545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:24.145639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:24.145650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.145666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.145680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.145701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:24.145725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341917670691379:2432], now have 1 active actors on pipe
2025-10-09T21:47:24.145734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341917670691486:8678], now have 1 active actors on pipe
2025-10-09T21:47:24.145742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341917308872024:7829]
2025-10-09T21:47:24.145746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341917308872024:7829]
2025-10-09T21:47:24.145749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341917308872024:7829]
2025-10-09T21:47:24.145763Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:24.145787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:24.145779Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:24.145792Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:24.145794Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:24.145805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:24.145809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:24.145854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341917308872024:7829]
2025-10-09T21:47:24.145876Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:24.146106Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 926
2025-10-09T21:47:24.146116Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 926
2025-10-09T21:47:24.146168Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:24.146276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:24.146285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:24.146319Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b1c6b31d-29ec954f-e573972d-df484fca_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:24.146341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:24.146346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:24.146348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:24.146361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:24.146374Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:24.146392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:24.147082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:24.147092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:24.147114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:24.147290Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 927 partition: 0 MaxSeqNo: 25 sessionId: src_id|b1c6b31d-29ec954f-e573972d-df484fca_0
2025-10-09T21:47:24.147758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32d8a255-a3472ae0-ffdb9234-915bd554_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b1c6b31d-29ec954f-e573972d-df484fca_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:24.150389Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1c6b31d-29ec954f-e573972d-df484fca_0] Write session: try to update token
2025-10-09T21:47:24.150408Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1c6b31d-29ec954f-e573972d-df484fca_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:24.165014Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:24.165031Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:24.165042Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341917308872024:7829]
2025-10-09T21:47:24.165099Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:24.165095Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:24.165279Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:24.165430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:24.165696Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:24.165845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:24.166071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:24.166122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:24.170586Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 927 sessionId: src_id|b1c6b31d-29ec954f-e573972d-df484fca_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:24.173146Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:24.173738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:24.173760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:24.173806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:24.173932Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:24.176306Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:24.176321Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:24.176324Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:24.176326Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:24.176336Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341917308872024:7829]
2025-10-09T21:47:24.176339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341917308872024:7829]
2025-10-09T21:47:24.176343Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341917308872024:7829]
2025-10-09T21:47:24.176385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:24.176394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.176397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:24.176407Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:24.176414Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:24.176418Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:24.176532Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:24.186124Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341917670691642:8525]
2025-10-09T21:47:24.186511Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 926, pipe: [1:7559341917670691642:8525]
2025-10-09T21:47:24.386656Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:24.387176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341917670691642:8525], now have 1 active actors on pipe
2025-10-09T21:47:24.395503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:24.395528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:24.395704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:24.395765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:24.395822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:24.395881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:24.395923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:24.395929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:24.395997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:24.396055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:24.396114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:24.396180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:24.396219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:24.396222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:24.396311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:24.396365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:24.396424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:24.396484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:24.396528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:24.396530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:24.396597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:24.396653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:24.396710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:24.396771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:24.396811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:24.396814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:24.396882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:24.396977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:24.397076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:24.397139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:24.397198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:24.397201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:24.397308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:24.397366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:24.397443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:24.397499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:24.397557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:24.397561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:24.397624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:24.397682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:24.397770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:24.397830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:24.397906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:24.397909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:24.397975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:24.398292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:24.398383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:24.398438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:24.398493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:24.398497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:24.398563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:24.398619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:24.398676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:24.398734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:24.398778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:24.398781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:24.398892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:24.398947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:24.399029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:24.399146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:24.399201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:24.399204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:24.402473Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:24.402890Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:24.402954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 927 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b1c6b31d-29ec954f-e573972d-df484fca_0
2025-10-09T21:47:24.403176Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 927 sessionId: src_id|b1c6b31d-29ec954f-e573972d-df484fca_0 is DEAD
2025-10-09T21:47:24.403278Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:24.403324Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341917670691642:8525]
2025-10-09T21:47:24.403584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1c6b31d-29ec954f-e573972d-df484fca_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:24.403596Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1c6b31d-29ec954f-e573972d-df484fca_0] Write session will restart in 0.006909s
2025-10-09T21:47:24.403622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1c6b31d-29ec954f-e573972d-df484fca_0] Write session: Do CDS request
2025-10-09T21:47:24.403628Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1c6b31d-29ec954f-e573972d-df484fca_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:24.411562Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1c6b31d-29ec954f-e573972d-df484fca_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:24.411788Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:24.411805Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 928
2025-10-09T21:47:24.411984Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 928 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:24.412048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 928 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:24.412058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=928 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:24.412062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:24.412566Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 928 sessionId:  describe result for acl check
2025-10-09T21:47:24.412602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:24.412604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:24.412606Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:24.412612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341917670691658:8683] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:24.413172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341917670691658:8683] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:24.417407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341917670691658:8683] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:24.427197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341917670691658:8683] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:24.427218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341917670691658:8683] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:24.427223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341917670691658:8683] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:24.427233Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 928 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:24.453728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:24.457129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:24.457391Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:24.457404Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:24.457419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:24.457489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:24.457540Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341917308872311:7841]
2025-10-09T21:47:24.458216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:24.458737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:24.458794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:24.459160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:24.459215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:24.459216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:24.459218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:24.459219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:24.459221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:24.459222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:24.459223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.459224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.459226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.459232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:24.459242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:24.459246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:24.459250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:24.459254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:24.459257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:24.459261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.459265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.459269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.459276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:24.459285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:24.459292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:24.459297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:24.459302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:24.459307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:24.459311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:24.459316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:24.459322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:24.459327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:24.459332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:24.459337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:24.459342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.459347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.459352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.459357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.459362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.459367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.459399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:24.469579Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 927
2025-10-09T21:47:24.469631Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 927
2025-10-09T21:47:24.467768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:24.469153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:24.469160Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:24.469163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:24.469167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:24.469174Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 927 [2:7559341917308872311:7841]
2025-10-09T21:47:24.469191Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:24.469210Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:24.469213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:24.469216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:24.469219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:24.469238Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:24.469244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:24.469251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:24.469312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.469314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.469316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.469319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:24.469405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:24.469408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.469421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.469430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.469444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:24.469463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341917308872305:7841]
2025-10-09T21:47:24.469465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341917308872305:7841]
2025-10-09T21:47:24.469467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341917308872305:7841]
2025-10-09T21:47:24.469476Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:24.469485Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:24.469488Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:24.469490Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:24.469493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:24.469508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:24.469512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:24.469524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341917670691651:2432], now have 1 active actors on pipe
2025-10-09T21:47:24.469528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341917670691734:8683], now have 1 active actors on pipe
2025-10-09T21:47:24.469569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341917308872305:7841]
2025-10-09T21:47:24.469573Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:24.469608Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:24.469698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:24.469705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:24.469735Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ac5bc6e0-dd85e145-8c988cad-b79a4ebc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:24.469745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:24.469749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:24.469752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:24.469765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:24.469776Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:24.469804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:24.470069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:24.470083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:24.470105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:24.470588Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 928 partition: 0 MaxSeqNo: 25 sessionId: src_id|ac5bc6e0-dd85e145-8c988cad-b79a4ebc_0
2025-10-09T21:47:24.471021Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1c6b31d-29ec954f-e573972d-df484fca_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ac5bc6e0-dd85e145-8c988cad-b79a4ebc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:24.473603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac5bc6e0-dd85e145-8c988cad-b79a4ebc_0] Write session: try to update token
2025-10-09T21:47:24.473620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac5bc6e0-dd85e145-8c988cad-b79a4ebc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:24.492769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 928 sessionId: src_id|ac5bc6e0-dd85e145-8c988cad-b79a4ebc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:24.495350Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:24.496573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:24.496597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:24.496627Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:24.496630Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:24.496633Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:24.496635Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:24.496643Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341917308872305:7841]
2025-10-09T21:47:24.496647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341917308872305:7841]
2025-10-09T21:47:24.496651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341917308872305:7841]
2025-10-09T21:47:24.496710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:24.496718Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:24.496724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.496725Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:24.496727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:24.496729Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:24.496865Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:24.503545Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341917670691921:8525]
2025-10-09T21:47:24.504601Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 927, pipe: [1:7559341917670691921:8525]
2025-10-09T21:47:24.656244Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:24.656785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:24.656832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341917670691921:8525], now have 1 active actors on pipe
2025-10-09T21:47:24.656973Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:24.660842Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341917670691921:8525]
2025-10-09T21:47:24.660870Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:24.669800Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:24.669815Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:24.669842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 928 reason: error at write: Disconnected sessionId: src_id|ac5bc6e0-dd85e145-8c988cad-b79a4ebc_0
2025-10-09T21:47:24.669994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 928 sessionId: src_id|ac5bc6e0-dd85e145-8c988cad-b79a4ebc_0 is DEAD
2025-10-09T21:47:24.670221Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac5bc6e0-dd85e145-8c988cad-b79a4ebc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:24.670234Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac5bc6e0-dd85e145-8c988cad-b79a4ebc_0] Write session will restart in 0.007771s
2025-10-09T21:47:24.670258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac5bc6e0-dd85e145-8c988cad-b79a4ebc_0] Write session: Do CDS request
2025-10-09T21:47:24.670266Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac5bc6e0-dd85e145-8c988cad-b79a4ebc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:24.679588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac5bc6e0-dd85e145-8c988cad-b79a4ebc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:24.680000Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:24.680017Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 929
2025-10-09T21:47:24.682552Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 929 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:24.682602Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 929 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:24.682608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=929 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:24.682613Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:24.683135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 929 sessionId:  describe result for acl check
2025-10-09T21:47:24.683179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:24.683185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:24.683190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:24.683196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341917670691955:8694] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:24.684211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341917670691955:8694] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:24.690563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341917670691955:8694] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:24.695535Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341917670691955:8694] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:24.695553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341917670691955:8694] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:24.695559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341917670691955:8694] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:24.695569Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 929 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:24.701139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:24.704437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:24.704705Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:24.704717Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:24.704732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:24.704799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:24.704864Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341917308872576:7847]
2025-10-09T21:47:24.705505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:24.705896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:24.705946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:24.706303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:24.706358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:24.706363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:24.706364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:24.706365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:24.706367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:24.706368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:24.706369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.706370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.706371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.706377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:24.706388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:24.706393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:24.706396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:24.706400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:24.706404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:24.706408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.706413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.706417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.706424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:24.706432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:24.706438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:24.706443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:24.706447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:24.706452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:24.706467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:24.706476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:24.706480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:24.706485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:24.706489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:24.706498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:24.706507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.706512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.706516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.706526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.706532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.706537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.706572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:24.714222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:24.715525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:24.715539Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:24.715541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:24.715546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:24.715555Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 928 [2:7559341917308872576:7847]
2025-10-09T21:47:24.715575Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:24.715593Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:24.715597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:24.715600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:24.715604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:24.715625Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:24.715635Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:24.715647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:24.715717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.715726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.715729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.715732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:24.715824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:24.715834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.715847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.715860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.715878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:24.715902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341917308872571:7847]
2025-10-09T21:47:24.715909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341917308872571:7847]
2025-10-09T21:47:24.715911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341917308872571:7847]
2025-10-09T21:47:24.715921Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:24.715932Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:24.715936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:24.715941Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:24.715943Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:24.715952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:24.715955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:24.715967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341917670692140:8694], now have 1 active actors on pipe
2025-10-09T21:47:24.715975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341917670691935:2432], now have 1 active actors on pipe
2025-10-09T21:47:24.716014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341917308872571:7847]
2025-10-09T21:47:24.716027Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:24.716042Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:24.716180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 928
2025-10-09T21:47:24.716184Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 928
2025-10-09T21:47:24.716316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:24.716327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:24.716365Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bf99a6e1-5226742b-810a960-52e96a28_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:24.716374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:24.716378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:24.716380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:24.716392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:24.716404Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:24.716423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:24.716853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:24.716867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:24.716890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:24.717036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 929 partition: 0 MaxSeqNo: 25 sessionId: src_id|bf99a6e1-5226742b-810a960-52e96a28_0
2025-10-09T21:47:24.717471Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac5bc6e0-dd85e145-8c988cad-b79a4ebc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bf99a6e1-5226742b-810a960-52e96a28_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:24.720074Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf99a6e1-5226742b-810a960-52e96a28_0] Write session: try to update token
2025-10-09T21:47:24.720095Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf99a6e1-5226742b-810a960-52e96a28_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:24.732176Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:24.732189Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:24.732191Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:24.732193Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:24.732203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341917308872571:7847]
2025-10-09T21:47:24.732207Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341917308872571:7847]
2025-10-09T21:47:24.732212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341917308872571:7847]
2025-10-09T21:47:24.732259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:24.732263Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:24.732268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.732269Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:24.732271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:24.732273Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:24.732411Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:24.735870Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 929 sessionId: src_id|bf99a6e1-5226742b-810a960-52e96a28_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:24.738540Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:24.761170Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341917670692199:8525]
2025-10-09T21:47:24.761685Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 928, pipe: [1:7559341917670692199:8525]
2025-10-09T21:47:24.900738Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:24.901230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:24.901246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:24.901296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341917670692199:8525], now have 1 active actors on pipe
2025-10-09T21:47:24.901595Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:24.902043Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341917670692199:8525]
2025-10-09T21:47:24.902092Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:24.902170Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 929 reason: status is not ok: tablet will be restarted right now sessionId: src_id|bf99a6e1-5226742b-810a960-52e96a28_0
2025-10-09T21:47:24.902951Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf99a6e1-5226742b-810a960-52e96a28_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:24.902961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf99a6e1-5226742b-810a960-52e96a28_0] Write session will restart in 0.009859s
2025-10-09T21:47:24.902992Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf99a6e1-5226742b-810a960-52e96a28_0] Write session: Do CDS request
2025-10-09T21:47:24.902387Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:24.902387Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 929 sessionId: src_id|bf99a6e1-5226742b-810a960-52e96a28_0 is DEAD
2025-10-09T21:47:24.902999Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf99a6e1-5226742b-810a960-52e96a28_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:24.914620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf99a6e1-5226742b-810a960-52e96a28_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:24.914726Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:24.914744Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 930
2025-10-09T21:47:24.914893Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 930 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:24.914929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 930 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:24.914939Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=930 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:24.914943Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:24.915407Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 930 sessionId:  describe result for acl check
2025-10-09T21:47:24.915453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:24.915460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:24.915462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:24.915468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341917670692280:8699] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:24.916042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341917670692280:8699] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:24.919098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341917670692280:8699] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:24.927385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341917670692280:8699] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:24.927404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341917670692280:8699] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:24.927408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341917670692280:8699] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:24.927418Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 930 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:24.927945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:24.930899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:24.931167Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:24.931179Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:24.931194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:24.931254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:24.931301Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341917308872831:7849]
2025-10-09T21:47:24.931947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:24.932366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:24.932419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:24.932799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:24.932855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:24.932856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:24.932857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:24.932858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:24.932860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:24.932861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:24.932863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.932864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.932866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.932873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:24.932887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:24.932892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:24.932896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:24.932899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:24.932903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:24.932906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.932909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.932912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.932926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:24.932936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:24.932944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:24.932949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:24.932956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:24.932961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:24.932967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:24.932973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:24.932979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:24.932986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:24.932991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:24.932997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:24.933003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.933008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.933011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.933016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.933021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.933028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.933058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:24.941056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:24.942465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:24.942483Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:24.942486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:24.942492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:24.942500Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 929 [2:7559341917308872831:7849]
2025-10-09T21:47:24.942517Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:24.942536Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:24.942540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:24.942544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:24.942546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:24.942566Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:24.942572Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:24.942583Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:24.942660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.942669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.942671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.942674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:24.942755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:24.942766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.942781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:24.942792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:24.942808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:24.942830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341917308872823:7849]
2025-10-09T21:47:24.942837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341917308872823:7849]
2025-10-09T21:47:24.942839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341917308872823:7849]
2025-10-09T21:47:24.942849Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:24.942867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:24.942880Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:24.942883Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:24.942886Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:24.942887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:24.942890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:24.942903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341917670692208:2432], now have 1 active actors on pipe
2025-10-09T21:47:24.942946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341917670692458:8699], now have 1 active actors on pipe
2025-10-09T21:47:24.942953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341917308872823:7849]
2025-10-09T21:47:24.942962Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:24.943204Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 929
2025-10-09T21:47:24.943248Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 929
2025-10-09T21:47:24.943000Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:24.943334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:24.943346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:24.943377Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|42fcb93b-defcf0cf-2ae9e41e-c4b5420a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:24.943385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:24.943389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:24.943392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:24.943404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:24.943427Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:24.943449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:24.947320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:24.947335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:24.947369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:24.949573Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 930 partition: 0 MaxSeqNo: 25 sessionId: src_id|42fcb93b-defcf0cf-2ae9e41e-c4b5420a_0
2025-10-09T21:47:24.950006Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf99a6e1-5226742b-810a960-52e96a28_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|42fcb93b-defcf0cf-2ae9e41e-c4b5420a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:24.952665Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42fcb93b-defcf0cf-2ae9e41e-c4b5420a_0] Write session: try to update token
2025-10-09T21:47:24.952686Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42fcb93b-defcf0cf-2ae9e41e-c4b5420a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:24.961050Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:24.961066Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:24.961069Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:24.961071Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:24.961081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341917308872823:7849]
2025-10-09T21:47:24.961085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341917308872823:7849]
2025-10-09T21:47:24.961088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341917308872823:7849]
2025-10-09T21:47:24.961135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:24.961147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:24.961149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:24.961170Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:24.961185Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:24.961189Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:24.961296Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:24.970075Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 930 sessionId: src_id|42fcb93b-defcf0cf-2ae9e41e-c4b5420a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:24.972583Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:25.001700Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341921965659766:8525]
2025-10-09T21:47:25.002074Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 929, pipe: [1:7559341921965659766:8525]
2025-10-09T21:47:25.125842Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:25.126445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:25.126474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:25.126549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341921965659766:8525], now have 1 active actors on pipe
2025-10-09T21:47:25.126817Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:25.127009Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:25.127056Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:25.127137Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 930 reason: status is not ok: tablet will be restarted right now sessionId: src_id|42fcb93b-defcf0cf-2ae9e41e-c4b5420a_0
2025-10-09T21:47:25.127341Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 930 sessionId: src_id|42fcb93b-defcf0cf-2ae9e41e-c4b5420a_0 is DEAD
2025-10-09T21:47:25.127908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42fcb93b-defcf0cf-2ae9e41e-c4b5420a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:25.127921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42fcb93b-defcf0cf-2ae9e41e-c4b5420a_0] Write session will restart in 0.007452s
2025-10-09T21:47:25.127621Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341921965659766:8525]
2025-10-09T21:47:25.127946Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42fcb93b-defcf0cf-2ae9e41e-c4b5420a_0] Write session: Do CDS request
2025-10-09T21:47:25.127968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42fcb93b-defcf0cf-2ae9e41e-c4b5420a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:25.136666Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42fcb93b-defcf0cf-2ae9e41e-c4b5420a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:25.136774Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:25.136792Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 931
2025-10-09T21:47:25.137228Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 931 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:25.137270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 931 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:25.137282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=931 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:25.137286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:25.138181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 931 sessionId:  describe result for acl check
2025-10-09T21:47:25.138223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:25.138226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:25.138228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:25.138235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341921965659808:8703] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:25.138815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341921965659808:8703] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:25.143876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341921965659808:8703] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:25.149271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341921965659808:8703] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:25.149292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341921965659808:8703] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:25.149296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341921965659808:8703] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:25.149307Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 931 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:25.152659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:25.155812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:25.156119Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:25.156130Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:25.156144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:25.156220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:25.156269Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341921603840381:7851]
2025-10-09T21:47:25.156887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:25.157283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:25.157363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:25.157735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:25.157828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:25.157836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:25.157838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:25.157839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:25.157840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:25.157841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:25.157843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.157844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.157845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.157853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:25.157875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:25.157880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:25.157884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:25.157888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:25.157891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:25.157895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.157899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.157903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.157913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:25.157921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:25.157927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:25.157932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:25.157937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:25.157943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:25.157948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:25.157954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:25.157959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:25.157964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:25.157969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:25.157975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:25.157981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.157986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.157991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.157998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.158003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.158008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.158054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:25.165977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:25.167211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:25.167234Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:25.167238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:25.167243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:25.167251Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 930 [2:7559341921603840381:7851]
2025-10-09T21:47:25.167273Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:25.167297Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:25.167306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:25.167309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:25.167311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:25.167333Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:25.167340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:25.167353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:25.167837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.167851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.167853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.167856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:25.167947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:25.167956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.167972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.167985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.168001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:25.168028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341921965659772:2432], now have 1 active actors on pipe
2025-10-09T21:47:25.168037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341921965659989:8703], now have 1 active actors on pipe
2025-10-09T21:47:25.168047Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341921603840376:7851]
2025-10-09T21:47:25.168049Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341921603840376:7851]
2025-10-09T21:47:25.168052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341921603840376:7851]
2025-10-09T21:47:25.168065Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:25.168082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:25.168099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:25.168103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:25.168103Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:25.168105Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:25.168106Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:25.168152Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341921603840376:7851]
2025-10-09T21:47:25.168156Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:25.168168Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:25.168147Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 930
2025-10-09T21:47:25.168214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:25.168219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:25.168249Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f2a2d143-18e988b7-16e9d82b-574ce869_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:25.168157Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 930
2025-10-09T21:47:25.168256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:25.168260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:25.168263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:25.168274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:25.168287Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:25.168623Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 931 partition: 0 MaxSeqNo: 25 sessionId: src_id|f2a2d143-18e988b7-16e9d82b-574ce869_0
2025-10-09T21:47:25.168303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:25.168510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:25.168514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:25.168531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:25.169051Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42fcb93b-defcf0cf-2ae9e41e-c4b5420a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f2a2d143-18e988b7-16e9d82b-574ce869_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:25.172030Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a2d143-18e988b7-16e9d82b-574ce869_0] Write session: try to update token
2025-10-09T21:47:25.172053Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a2d143-18e988b7-16e9d82b-574ce869_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:25.181697Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:25.181715Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:25.181726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341921603840376:7851]
2025-10-09T21:47:25.181783Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:25.181959Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:25.181975Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:25.182146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:25.182307Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:25.182482Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:25.182690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:25.182741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:25.192354Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 931 sessionId: src_id|f2a2d143-18e988b7-16e9d82b-574ce869_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:25.194960Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:25.195306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:25.195331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:25.195355Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:25.195358Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:25.195361Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:25.195363Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:25.195373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341921603840376:7851]
2025-10-09T21:47:25.195385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341921603840376:7851]
2025-10-09T21:47:25.195389Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341921603840376:7851]
2025-10-09T21:47:25.195429Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:25.195435Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:25.195439Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:25.195446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:25.195454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.195457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:25.195600Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:25.229135Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341921965660041:8525]
2025-10-09T21:47:25.229491Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 930, pipe: [1:7559341921965660041:8525]
2025-10-09T21:47:25.360555Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:25.360852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:25.360892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341921965660041:8525], now have 1 active actors on pipe
2025-10-09T21:47:25.361010Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:25.361058Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:25.361100Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:25.361107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:25.361116Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341921965660041:8525]
2025-10-09T21:47:25.361193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 931 reason: error at write: Disconnected sessionId: src_id|f2a2d143-18e988b7-16e9d82b-574ce869_0
2025-10-09T21:47:25.361581Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a2d143-18e988b7-16e9d82b-574ce869_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:25.361601Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a2d143-18e988b7-16e9d82b-574ce869_0] Write session will restart in 0.008019s
2025-10-09T21:47:25.361626Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a2d143-18e988b7-16e9d82b-574ce869_0] Write session: Do CDS request
2025-10-09T21:47:25.361633Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a2d143-18e988b7-16e9d82b-574ce869_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:25.361437Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 931 sessionId: src_id|f2a2d143-18e988b7-16e9d82b-574ce869_0 is DEAD
2025-10-09T21:47:25.374779Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a2d143-18e988b7-16e9d82b-574ce869_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:25.375248Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:25.375271Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 932
2025-10-09T21:47:25.375433Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 932 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:25.375488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 932 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:25.375493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=932 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:25.375498Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:25.375988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 932 sessionId:  describe result for acl check
2025-10-09T21:47:25.376029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:25.376031Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:25.376033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:25.376039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341921965660090:8708] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:25.376605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341921965660090:8708] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:25.379870Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341921965660090:8708] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:25.386820Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341921965660090:8708] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:25.386840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341921965660090:8708] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:25.386844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341921965660090:8708] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:25.386853Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 932 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:25.389171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:25.392217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:25.392791Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:25.392806Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:25.392820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:25.392882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:25.392916Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341921603840666:7859]
2025-10-09T21:47:25.393473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:25.393829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:25.393902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:25.394268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:25.394342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:25.394348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:25.394350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:25.394362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:25.394363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:25.394364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:25.394366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.394367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.394368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.394375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:25.394387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:25.394391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:25.394396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:25.394405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:25.394409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:25.394413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.394417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.394421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.394429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:25.394437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:25.394443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:25.394454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:25.394458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:25.394464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:25.394469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:25.394474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:25.394482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:25.394487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:25.394492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:25.394497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:25.394502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.394508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.394512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.394517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.394520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.394525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.394554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:25.404908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:25.406005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:25.406021Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:25.406024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:25.406046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:25.406055Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 931 [2:7559341921603840666:7859]
2025-10-09T21:47:25.406072Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:25.406094Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:25.406104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:25.406107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:25.406109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:25.406130Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:25.406138Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:25.406147Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:25.406212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.406220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.406222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.406225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:25.406516Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 931
2025-10-09T21:47:25.406575Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 931
2025-10-09T21:47:25.406307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:25.406316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.406329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.406341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.406357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:25.406381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341921965660044:2432], now have 1 active actors on pipe
2025-10-09T21:47:25.406385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341921965660292:8708], now have 1 active actors on pipe
2025-10-09T21:47:25.406393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341921603840661:7859]
2025-10-09T21:47:25.406396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341921603840661:7859]
2025-10-09T21:47:25.406398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341921603840661:7859]
2025-10-09T21:47:25.406412Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:25.406428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:25.406443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:25.406447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:25.406491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341921603840661:7859]
2025-10-09T21:47:25.406495Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:25.406541Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:25.406553Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:25.406556Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:25.406563Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:25.406718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:25.406724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:25.406756Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4300f60b-a52e095b-7f0205cf-daf4457e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:25.406766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:25.406771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:25.406774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:25.406786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:25.406798Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:25.406816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:25.407171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:25.407182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:25.407202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:25.407766Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a2d143-18e988b7-16e9d82b-574ce869_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4300f60b-a52e095b-7f0205cf-daf4457e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:25.407330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 932 partition: 0 MaxSeqNo: 25 sessionId: src_id|4300f60b-a52e095b-7f0205cf-daf4457e_0
2025-10-09T21:47:25.410574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4300f60b-a52e095b-7f0205cf-daf4457e_0] Write session: try to update token
2025-10-09T21:47:25.410605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4300f60b-a52e095b-7f0205cf-daf4457e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:25.419804Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:25.419819Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:25.419829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341921603840661:7859]
2025-10-09T21:47:25.419879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:25.419883Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:25.420048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:25.420192Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:25.420323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:25.420455Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:25.420637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:25.420683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:25.434159Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:25.434176Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:25.434179Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:25.434181Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:25.434192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341921603840661:7859]
2025-10-09T21:47:25.434195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341921603840661:7859]
2025-10-09T21:47:25.434199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341921603840661:7859]
2025-10-09T21:47:25.434249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:25.434257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.434217Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 932 sessionId: src_id|4300f60b-a52e095b-7f0205cf-daf4457e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:25.434260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:25.434276Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:25.434282Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:25.434287Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:25.434406Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:25.436714Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:25.461913Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341921965660308:8525]
2025-10-09T21:47:25.462484Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 931, pipe: [1:7559341921965660308:8525]
2025-10-09T21:47:25.620029Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:25.620318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:25.620335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:25.620361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341921965660308:8525], now have 1 active actors on pipe
2025-10-09T21:47:25.620681Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:25.621051Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:25.620796Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:25.621561Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4300f60b-a52e095b-7f0205cf-daf4457e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:25.621572Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4300f60b-a52e095b-7f0205cf-daf4457e_0] Write session will restart in 0.005172s
2025-10-09T21:47:25.621150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 932 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4300f60b-a52e095b-7f0205cf-daf4457e_0
2025-10-09T21:47:25.621151Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341921965660308:8525]
2025-10-09T21:47:25.621595Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4300f60b-a52e095b-7f0205cf-daf4457e_0] Write session: Do CDS request
2025-10-09T21:47:25.621366Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 932 sessionId: src_id|4300f60b-a52e095b-7f0205cf-daf4457e_0 is DEAD
2025-10-09T21:47:25.621604Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4300f60b-a52e095b-7f0205cf-daf4457e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:25.629321Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4300f60b-a52e095b-7f0205cf-daf4457e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:25.630361Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:25.630385Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 933
2025-10-09T21:47:25.630647Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 933 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:25.630699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 933 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:25.630707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=933 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:25.630711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:25.634461Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 933 sessionId:  describe result for acl check
2025-10-09T21:47:25.634515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:25.634517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:25.634519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:25.634527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341921965660350:8719] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:25.635187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341921965660350:8719] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:25.644375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341921965660350:8719] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:25.647949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341921965660350:8719] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:25.647959Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341921965660350:8719] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:25.647963Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341921965660350:8719] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:25.647974Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 933 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:25.668566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:25.671592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:25.671832Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:25.671845Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:25.671876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:25.671947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:25.671992Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341921603840935:7869]
2025-10-09T21:47:25.672592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:25.673209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:25.673278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:25.673633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:25.673713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:25.673721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:25.673723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:25.673724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:25.673738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:25.673739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:25.673741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.673742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.673743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.673749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:25.673760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:25.673765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:25.673770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:25.673774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:25.673779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:25.673783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.673787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.673790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.673802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:25.673820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:25.673833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:25.673838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:25.673843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:25.673848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:25.673851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:25.673857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:25.673873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:25.673878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:25.673882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:25.673887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:25.673892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.673896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.673900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.673905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.673911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.673916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.673945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:25.685004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:25.686436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:25.686444Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:25.686447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:25.686451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:25.686460Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 932 [2:7559341921603840935:7869]
2025-10-09T21:47:25.686476Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:25.686497Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:25.686500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:25.686504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:25.686507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:25.686527Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:25.686534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:25.686542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:25.686616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.686621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.686624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.686627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:25.686711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:25.686714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.686729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.686741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.686760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:25.686783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341921603840930:7869]
2025-10-09T21:47:25.686786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341921603840930:7869]
2025-10-09T21:47:25.686788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341921603840930:7869]
2025-10-09T21:47:25.686805Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:25.686821Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:25.686836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:25.686840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:25.686885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341921603840930:7869]
2025-10-09T21:47:25.686888Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:25.687136Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:25.687143Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:25.687145Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:25.687151Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:25.687618Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 932
2025-10-09T21:47:25.687662Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 932
2025-10-09T21:47:25.689050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341921965660336:2432], now have 1 active actors on pipe
2025-10-09T21:47:25.689063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341921965660391:8719], now have 1 active actors on pipe
2025-10-09T21:47:25.691042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:25.691062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:25.691109Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5ef9856b-e4ed16fb-d446b3af-55456d6e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:25.691122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:25.691128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:25.691131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:25.691146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:25.691166Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:25.691190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:25.697274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:25.697297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:25.697343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:25.697586Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 933 partition: 0 MaxSeqNo: 25 sessionId: src_id|5ef9856b-e4ed16fb-d446b3af-55456d6e_0
2025-10-09T21:47:25.698290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4300f60b-a52e095b-7f0205cf-daf4457e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5ef9856b-e4ed16fb-d446b3af-55456d6e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:25.701150Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ef9856b-e4ed16fb-d446b3af-55456d6e_0] Write session: try to update token
2025-10-09T21:47:25.701175Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ef9856b-e4ed16fb-d446b3af-55456d6e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:25.715685Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:25.715705Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:25.715708Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:25.715710Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:25.715721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341921603840930:7869]
2025-10-09T21:47:25.715726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341921603840930:7869]
2025-10-09T21:47:25.715731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341921603840930:7869]
2025-10-09T21:47:25.715780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:25.715788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.715791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:25.715930Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:25.718165Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:25.718172Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:25.718177Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:25.722238Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341921965660599:8525]
2025-10-09T21:47:25.726389Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 932, pipe: [1:7559341921965660599:8525]
2025-10-09T21:47:25.733084Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 933 sessionId: src_id|5ef9856b-e4ed16fb-d446b3af-55456d6e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:25.736088Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:25.901432Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:25.902099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341921965660599:8525], now have 1 active actors on pipe
2025-10-09T21:47:25.902564Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341921965660599:8525]
2025-10-09T21:47:25.902613Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:25.902648Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:25.902651Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:25.902736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 933 reason: error at write: Disconnected sessionId: src_id|5ef9856b-e4ed16fb-d446b3af-55456d6e_0
2025-10-09T21:47:25.902933Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 933 sessionId: src_id|5ef9856b-e4ed16fb-d446b3af-55456d6e_0 is DEAD
2025-10-09T21:47:25.902154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:25.902166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:25.902555Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:25.906283Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ef9856b-e4ed16fb-d446b3af-55456d6e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:25.906297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ef9856b-e4ed16fb-d446b3af-55456d6e_0] Write session will restart in 0.006785s
2025-10-09T21:47:25.906327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ef9856b-e4ed16fb-d446b3af-55456d6e_0] Write session: Do CDS request
2025-10-09T21:47:25.906333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ef9856b-e4ed16fb-d446b3af-55456d6e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:25.918565Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ef9856b-e4ed16fb-d446b3af-55456d6e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:25.922228Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:25.922257Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 934
2025-10-09T21:47:25.922984Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 934 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:25.923034Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 934 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:25.923046Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=934 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:25.923052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:25.926442Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 934 sessionId:  describe result for acl check
2025-10-09T21:47:25.926494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:25.926496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:25.926498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:25.926506Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341921965660824:8724] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:25.927072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341921965660824:8724] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:25.931231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:25.935448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:25.935748Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:25.935756Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:25.935776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:25.935863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:25.935914Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341921603841193:7871]
2025-10-09T21:47:25.936558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:25.937269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:25.937329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:25.937723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:25.937768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:25.937769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:25.937771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:25.937772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:25.937774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:25.937775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:25.937776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.937777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.937778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.937784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:25.937803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:25.937807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:25.937811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:25.937814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:25.937819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:25.937822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.937827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.937830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.937838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:25.937848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:25.937855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:25.937873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:25.937878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:25.937884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:25.937888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:25.937894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:25.937898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:25.937904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:25.937908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:25.937913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:25.937919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.937925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.937930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.937934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.937941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.937945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.937975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:25.948017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341921965660824:8724] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:25.946365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:25.947607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:25.947614Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:25.947617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:25.947622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:25.947630Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 933 [2:7559341921603841193:7871]
2025-10-09T21:47:25.947645Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:25.947667Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:25.947670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:25.947674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:25.947676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:25.947696Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:25.947703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:25.947714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:25.947777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.947779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.947781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.947785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:25.947865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:25.947868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.947878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:25.947890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:25.947907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:25.947928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341921603841186:7871]
2025-10-09T21:47:25.947931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341921603841186:7871]
2025-10-09T21:47:25.947933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341921603841186:7871]
2025-10-09T21:47:25.947947Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:25.947962Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:25.947976Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:25.947980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:25.948020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341921603841186:7871]
2025-10-09T21:47:25.948022Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:25.948422Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:25.948429Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:25.948431Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:25.948437Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:25.948727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341921965660605:2432], now have 1 active actors on pipe
2025-10-09T21:47:25.949495Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 933
2025-10-09T21:47:25.961053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341921965660866:8724], now have 1 active actors on pipe
2025-10-09T21:47:25.960164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341921965660824:8724] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:25.960197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341921965660824:8724] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:25.960202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341921965660824:8724] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:25.960213Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 934 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:25.961063Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 933
2025-10-09T21:47:25.961234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:25.961250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:25.961286Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b6b7b99f-5f5d2f71-851d4fd5-606557bc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:25.961301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:25.961307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:25.961310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:25.961325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:25.961346Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:25.961368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:25.961854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:25.961896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:25.961998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:25.962621Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ef9856b-e4ed16fb-d446b3af-55456d6e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b6b7b99f-5f5d2f71-851d4fd5-606557bc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:25.965315Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6b7b99f-5f5d2f71-851d4fd5-606557bc_0] Write session: try to update token
2025-10-09T21:47:25.965332Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6b7b99f-5f5d2f71-851d4fd5-606557bc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:25.962193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 934 partition: 0 MaxSeqNo: 25 sessionId: src_id|b6b7b99f-5f5d2f71-851d4fd5-606557bc_0
2025-10-09T21:47:25.973915Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:25.973929Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:25.973932Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:25.973934Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:25.973945Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341921603841186:7871]
2025-10-09T21:47:25.973949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341921603841186:7871]
2025-10-09T21:47:25.973955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341921603841186:7871]
2025-10-09T21:47:25.974006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:25.974015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:25.974018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:25.974078Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:25.974084Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:25.974089Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:25.974251Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:25.988390Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 934 sessionId: src_id|b6b7b99f-5f5d2f71-851d4fd5-606557bc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:25.991326Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:26.007747Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341926260628166:8525]
2025-10-09T21:47:26.010987Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 933, pipe: [1:7559341926260628166:8525]
2025-10-09T21:47:26.174312Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:26.175041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:26.175053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:26.175077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341926260628166:8525], now have 1 active actors on pipe
2025-10-09T21:47:26.178605Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:26.178663Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:26.178667Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:26.178737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 934 reason: error at write: Disconnected sessionId: src_id|b6b7b99f-5f5d2f71-851d4fd5-606557bc_0
2025-10-09T21:47:26.178927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 934 sessionId: src_id|b6b7b99f-5f5d2f71-851d4fd5-606557bc_0 is DEAD
2025-10-09T21:47:26.179118Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6b7b99f-5f5d2f71-851d4fd5-606557bc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:26.179131Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6b7b99f-5f5d2f71-851d4fd5-606557bc_0] Write session will restart in 0.006917s
2025-10-09T21:47:26.179157Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6b7b99f-5f5d2f71-851d4fd5-606557bc_0] Write session: Do CDS request
2025-10-09T21:47:26.179163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6b7b99f-5f5d2f71-851d4fd5-606557bc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:26.179139Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341926260628166:8525]
2025-10-09T21:47:26.179044Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:26.189273Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6b7b99f-5f5d2f71-851d4fd5-606557bc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:26.189427Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:26.189447Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 935
2025-10-09T21:47:26.189611Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 935 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:26.189649Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 935 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:26.189655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=935 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:26.189658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:26.190090Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 935 sessionId:  describe result for acl check
2025-10-09T21:47:26.190122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:26.190124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:26.190129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:26.190136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341926260628187:8729] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:26.190686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341926260628187:8729] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:26.194459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341926260628187:8729] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:26.198024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341926260628187:8729] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:26.198052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341926260628187:8729] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:26.198056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341926260628187:8729] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:26.198067Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 935 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:26.229406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:26.233554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:26.233824Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:26.233829Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:26.233843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:26.233926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:26.233969Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341925898808748:7873]
2025-10-09T21:47:26.234593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:26.235103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:26.235158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:26.235499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:26.235571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:26.235572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:26.235574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:26.235575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:26.235576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:26.235577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:26.235578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.235579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.235581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.235588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:26.235600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:26.235604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:26.235609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:26.235612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:26.235616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:26.235620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.235624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.235627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.235637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:26.235646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:26.235652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:26.235658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:26.235664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:26.235670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:26.235675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:26.235681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:26.235685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:26.235691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:26.235695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:26.235701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:26.235706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.235711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.235717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.235722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.235727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.235735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.235766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:26.243392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:26.244868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:26.244874Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:26.244877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:26.244882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:26.244890Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 934 [2:7559341925898808748:7873]
2025-10-09T21:47:26.244912Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:26.244934Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:26.244938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:26.244941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:26.244944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:26.244964Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:26.244970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:26.244978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:26.245046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.245049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.245051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.245054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:26.245137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:26.245140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.245154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.245166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.245184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:26.245209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341925898808742:7873]
2025-10-09T21:47:26.245213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341925898808742:7873]
2025-10-09T21:47:26.245215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341925898808742:7873]
2025-10-09T21:47:26.245233Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:26.245249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:26.245263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:26.245266Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:26.245283Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:26.245295Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:26.245297Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:26.245308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341925898808742:7873]
2025-10-09T21:47:26.245311Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:26.245389Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:26.245538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341926260628175:2432], now have 1 active actors on pipe
2025-10-09T21:47:26.245546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341926260628229:8729], now have 1 active actors on pipe
2025-10-09T21:47:26.245714Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 934
2025-10-09T21:47:26.245807Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 934
2025-10-09T21:47:26.245977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:26.245988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:26.246018Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c818dd80-cd3e70c4-644a8926-e52dfef1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:26.246041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:26.246046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:26.246049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:26.246061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:26.246074Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:26.246095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:26.246502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:26.246511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:26.246533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:26.247266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 935 partition: 0 MaxSeqNo: 25 sessionId: src_id|c818dd80-cd3e70c4-644a8926-e52dfef1_0
2025-10-09T21:47:26.247739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6b7b99f-5f5d2f71-851d4fd5-606557bc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c818dd80-cd3e70c4-644a8926-e52dfef1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:26.250627Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c818dd80-cd3e70c4-644a8926-e52dfef1_0] Write session: try to update token
2025-10-09T21:47:26.250646Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c818dd80-cd3e70c4-644a8926-e52dfef1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:26.264840Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:26.264859Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:26.264862Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:26.264864Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:26.264874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341925898808742:7873]
2025-10-09T21:47:26.264879Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341925898808742:7873]
2025-10-09T21:47:26.264885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341925898808742:7873]
2025-10-09T21:47:26.264941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:26.264949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.264953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:26.265162Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:26.270475Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:26.270484Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:26.270489Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:26.279174Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341926260628447:8525]
2025-10-09T21:47:26.279621Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 934, pipe: [1:7559341926260628447:8525]
2025-10-09T21:47:26.280132Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 935 sessionId: src_id|c818dd80-cd3e70c4-644a8926-e52dfef1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:26.283202Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:26.428801Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:26.429333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341926260628447:8525], now have 1 active actors on pipe
2025-10-09T21:47:26.429359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:26.429366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:26.429668Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:26.429678Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:26.429680Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:26.429737Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341926260628447:8525]
2025-10-09T21:47:26.429751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 935 reason: error at write: Disconnected sessionId: src_id|c818dd80-cd3e70c4-644a8926-e52dfef1_0
2025-10-09T21:47:26.429631Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:26.429961Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 935 sessionId: src_id|c818dd80-cd3e70c4-644a8926-e52dfef1_0 is DEAD
2025-10-09T21:47:26.437520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c818dd80-cd3e70c4-644a8926-e52dfef1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:26.437534Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c818dd80-cd3e70c4-644a8926-e52dfef1_0] Write session will restart in 0.008603s
2025-10-09T21:47:26.437560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c818dd80-cd3e70c4-644a8926-e52dfef1_0] Write session: Do CDS request
2025-10-09T21:47:26.437565Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c818dd80-cd3e70c4-644a8926-e52dfef1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:26.451413Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c818dd80-cd3e70c4-644a8926-e52dfef1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:26.451864Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:26.451881Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 936
2025-10-09T21:47:26.452582Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 936 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:26.452622Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 936 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:26.452628Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=936 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:26.452632Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:26.453654Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 936 sessionId:  describe result for acl check
2025-10-09T21:47:26.453693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:26.453695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:26.453697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:26.453703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341926260628615:8733] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:26.454304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341926260628615:8733] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:26.458883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341926260628615:8733] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:26.463023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341926260628615:8733] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:26.463038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341926260628615:8733] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:26.463042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341926260628615:8733] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:26.463050Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 936 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:26.463334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:26.466903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:26.467232Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:26.467242Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:26.467254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:26.467313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:26.467368Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341925898809037:7881]
2025-10-09T21:47:26.467930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:26.468330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:26.468380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:26.468699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:26.468753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:26.468755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:26.468756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:26.468757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:26.468758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:26.468759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:26.468760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.468761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.468762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.468768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:26.468779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:26.468782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:26.468786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:26.468790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:26.468794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:26.468798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.468802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.468805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.468813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:26.468821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:26.468827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:26.468833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:26.468837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:26.468843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:26.468846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:26.468851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:26.468856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:26.468861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:26.468865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:26.468869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:26.468874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.468879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.468883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.468887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.468892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.468897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.468924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:26.479852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:26.481471Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 935
2025-10-09T21:47:26.481533Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 935
2025-10-09T21:47:26.481019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:26.481037Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:26.481040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:26.481046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:26.481053Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 935 [2:7559341925898809037:7881]
2025-10-09T21:47:26.481067Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:26.481087Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:26.481091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:26.481094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:26.481097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:26.481113Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:26.481117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:26.481123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:26.481182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.481185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.481189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.481192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:26.481268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:26.481271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.481282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.481292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.481309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:26.481331Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341925898809030:7881]
2025-10-09T21:47:26.481333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341925898809030:7881]
2025-10-09T21:47:26.481335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341925898809030:7881]
2025-10-09T21:47:26.481346Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:26.481352Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:26.481355Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:26.481359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:26.481360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:26.481374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:26.481377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:26.481389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341926260628450:2432], now have 1 active actors on pipe
2025-10-09T21:47:26.481392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341926260628705:8733], now have 1 active actors on pipe
2025-10-09T21:47:26.481429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341925898809030:7881]
2025-10-09T21:47:26.481438Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:26.481443Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:26.481747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:26.481754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:26.481799Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|66792023-a36c0931-33a39f16-e4ce0ef6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:26.481814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:26.481819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:26.481822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:26.481835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:26.481877Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:26.481898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:26.482325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:26.482341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:26.482362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:26.482556Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 936 partition: 0 MaxSeqNo: 25 sessionId: src_id|66792023-a36c0931-33a39f16-e4ce0ef6_0
2025-10-09T21:47:26.482958Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c818dd80-cd3e70c4-644a8926-e52dfef1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|66792023-a36c0931-33a39f16-e4ce0ef6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:26.485729Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66792023-a36c0931-33a39f16-e4ce0ef6_0] Write session: try to update token
2025-10-09T21:47:26.485748Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66792023-a36c0931-33a39f16-e4ce0ef6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:26.499636Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:26.499654Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:26.499656Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:26.499659Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:26.499668Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341925898809030:7881]
2025-10-09T21:47:26.499673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341925898809030:7881]
2025-10-09T21:47:26.499677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341925898809030:7881]
2025-10-09T21:47:26.499723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:26.499730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.499734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:26.499734Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:26.499741Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:26.499745Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:26.499884Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:26.510046Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 936 sessionId: src_id|66792023-a36c0931-33a39f16-e4ce0ef6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:26.512822Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:26.530023Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341926260628719:8525]
2025-10-09T21:47:26.530387Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 935, pipe: [1:7559341926260628719:8525]
2025-10-09T21:47:26.668363Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:26.668869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:26.668882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:26.668929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341926260628719:8525], now have 1 active actors on pipe
2025-10-09T21:47:26.669269Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:26.669330Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341926260628719:8525]
2025-10-09T21:47:26.669262Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:26.669348Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:26.669367Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:26.669429Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 936 reason: error at write: Disconnected sessionId: src_id|66792023-a36c0931-33a39f16-e4ce0ef6_0
2025-10-09T21:47:26.669606Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 936 sessionId: src_id|66792023-a36c0931-33a39f16-e4ce0ef6_0 is DEAD
2025-10-09T21:47:26.670226Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66792023-a36c0931-33a39f16-e4ce0ef6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:26.670237Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66792023-a36c0931-33a39f16-e4ce0ef6_0] Write session will restart in 0.009322s
2025-10-09T21:47:26.670262Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66792023-a36c0931-33a39f16-e4ce0ef6_0] Write session: Do CDS request
2025-10-09T21:47:26.670268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66792023-a36c0931-33a39f16-e4ce0ef6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:26.682434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66792023-a36c0931-33a39f16-e4ce0ef6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:26.689484Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:26.689513Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 937
2025-10-09T21:47:26.689715Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 937 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:26.689748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 937 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:26.689754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=937 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:26.689757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:26.690205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 937 sessionId:  describe result for acl check
2025-10-09T21:47:26.690235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:26.690238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:26.690239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:26.690246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341926260628939:8744] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:26.690774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341926260628939:8744] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:26.695290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341926260628939:8744] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:26.700840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:26.703697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341926260628939:8744] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:26.703715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341926260628939:8744] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:26.703719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341926260628939:8744] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:26.703730Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 937 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:26.703057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:26.703588Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:26.703593Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:26.703611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:26.703696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:26.703750Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341925898809310:7891]
2025-10-09T21:47:26.704403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:26.705230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:26.705288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:26.705677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:26.705760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:26.705770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:26.705772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:26.705773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:26.705775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:26.705776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:26.705777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.705779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.705780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.705787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:26.705797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:26.705801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:26.705804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:26.705808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:26.705812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:26.705815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.705819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.705823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.705831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:26.705839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:26.705845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:26.705850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:26.705854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:26.705873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:26.705877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:26.705881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:26.705886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:26.705891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:26.705897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:26.705901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:26.705906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.705911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.705915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.705920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.705927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.705933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.705959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:26.717977Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 936
2025-10-09T21:47:26.718006Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 936
2025-10-09T21:47:26.715869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:26.717512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:26.717519Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:26.717522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:26.717527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:26.717536Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 936 [2:7559341925898809310:7891]
2025-10-09T21:47:26.717556Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:26.717578Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:26.717582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:26.717585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:26.717587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:26.717607Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:26.717614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:26.717622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:26.717693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.717695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.717697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.717701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:26.717784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:26.717788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.717800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.717811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.717828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:26.717851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341926260628746:2432], now have 1 active actors on pipe
2025-10-09T21:47:26.717869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341925898809305:7891]
2025-10-09T21:47:26.717874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341925898809305:7891]
2025-10-09T21:47:26.717876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341925898809305:7891]
2025-10-09T21:47:26.717893Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:26.717903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341926260629002:8744], now have 1 active actors on pipe
2025-10-09T21:47:26.717913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:26.717914Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:26.717918Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:26.717920Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:26.717928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:26.717932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:26.717977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341925898809305:7891]
2025-10-09T21:47:26.717986Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:26.717995Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:26.718359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:26.718375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:26.718425Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2c2fe6ad-fa79287d-904948b8-43cddfa7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:26.718441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:26.718446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:26.718449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:26.718461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:26.718475Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:26.718508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:26.719164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:26.719181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:26.719200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:26.719311Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 937 partition: 0 MaxSeqNo: 25 sessionId: src_id|2c2fe6ad-fa79287d-904948b8-43cddfa7_0
2025-10-09T21:47:26.722241Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66792023-a36c0931-33a39f16-e4ce0ef6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2c2fe6ad-fa79287d-904948b8-43cddfa7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:26.724905Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c2fe6ad-fa79287d-904948b8-43cddfa7_0] Write session: try to update token
2025-10-09T21:47:26.724922Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c2fe6ad-fa79287d-904948b8-43cddfa7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:26.737567Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:26.737586Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:26.737588Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:26.737590Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:26.737600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341925898809305:7891]
2025-10-09T21:47:26.737605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341925898809305:7891]
2025-10-09T21:47:26.737609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341925898809305:7891]
2025-10-09T21:47:26.737651Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:26.737656Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:26.737660Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:26.737693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:26.737701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.737704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:26.737866Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:26.753281Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 937 sessionId: src_id|2c2fe6ad-fa79287d-904948b8-43cddfa7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:26.757564Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:26.769104Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341926260629016:8525]
2025-10-09T21:47:26.769409Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 936, pipe: [1:7559341926260629016:8525]
2025-10-09T21:47:26.906259Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:26.906748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:26.906758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:26.906779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341926260629016:8525], now have 1 active actors on pipe
2025-10-09T21:47:26.907281Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:26.910853Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:26.910919Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341926260629016:8525]
2025-10-09T21:47:26.910948Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:26.911030Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 937 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2c2fe6ad-fa79287d-904948b8-43cddfa7_0
2025-10-09T21:47:26.911569Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 937 sessionId: src_id|2c2fe6ad-fa79287d-904948b8-43cddfa7_0 is DEAD
2025-10-09T21:47:26.914256Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c2fe6ad-fa79287d-904948b8-43cddfa7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:26.914267Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c2fe6ad-fa79287d-904948b8-43cddfa7_0] Write session will restart in 0.009058s
2025-10-09T21:47:26.914309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c2fe6ad-fa79287d-904948b8-43cddfa7_0] Write session: Do CDS request
2025-10-09T21:47:26.914316Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c2fe6ad-fa79287d-904948b8-43cddfa7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:26.926455Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c2fe6ad-fa79287d-904948b8-43cddfa7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:26.926777Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:26.926793Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 938
2025-10-09T21:47:26.926949Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 938 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:26.926979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 938 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:26.926985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=938 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:26.926989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:26.932641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 938 sessionId:  describe result for acl check
2025-10-09T21:47:26.932677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:26.932679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:26.932682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:26.932689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341926260629065:8751] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:26.933223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341926260629065:8751] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:26.940382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341926260629065:8751] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:26.949575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341926260629065:8751] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:26.949592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341926260629065:8751] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:26.949596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341926260629065:8751] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:26.949607Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 938 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:26.967481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:26.977509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:26.977788Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:26.977799Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:26.977812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:26.977911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:26.977960Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341925898809573:7893]
2025-10-09T21:47:26.978591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:26.979001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:26.979059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:26.979428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:26.979494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:26.979502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:26.979504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:26.979505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:26.979506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:26.979508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:26.979509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.979510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.979512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.979519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:26.979530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:26.979535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:26.979539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:26.979544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:26.979548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:26.979552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.979556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.979560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.979567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:26.979577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:26.979582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:26.979587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:26.979591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:26.979595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:26.979599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:26.979603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:26.979608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:26.979612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:26.979615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:26.979620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:26.979627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.979631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.979635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.979640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.979644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.979651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.979676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:26.989953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:26.991490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:26.991507Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:26.991510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:26.991515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:26.991522Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 937 [2:7559341925898809573:7893]
2025-10-09T21:47:26.991540Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:26.991568Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:26.991571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:26.991575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:26.991577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:26.991596Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:26.991603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:26.991610Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:26.991682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.991684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.991686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.991690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:26.991775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:26.991778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:26.991793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:26.991804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:26.991821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:26.991844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341925898809567:7893]
2025-10-09T21:47:26.991847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341925898809567:7893]
2025-10-09T21:47:26.991849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341925898809567:7893]
2025-10-09T21:47:26.991858Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:26.991868Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:26.991871Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:26.991873Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:26.991874Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:26.991889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:26.991892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:26.991905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341926260629021:2432], now have 1 active actors on pipe
2025-10-09T21:47:26.991909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341926260629159:8751], now have 1 active actors on pipe
2025-10-09T21:47:26.991951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341925898809567:7893]
2025-10-09T21:47:26.991961Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:26.992233Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:26.992136Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 937
2025-10-09T21:47:26.992179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 937
2025-10-09T21:47:26.992906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:26.992919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:26.992950Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9b9ccae4-bc12c6fe-8b9dc0ae-ddf04ca8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:26.992960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:26.992965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:26.992968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:26.992981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:26.992994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:26.993015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:26.996545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:26.996558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:26.996591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:27.002330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 938 partition: 0 MaxSeqNo: 25 sessionId: src_id|9b9ccae4-bc12c6fe-8b9dc0ae-ddf04ca8_0
2025-10-09T21:47:27.006144Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c2fe6ad-fa79287d-904948b8-43cddfa7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9b9ccae4-bc12c6fe-8b9dc0ae-ddf04ca8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:27.008861Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9b9ccae4-bc12c6fe-8b9dc0ae-ddf04ca8_0] Write session: try to update token
2025-10-09T21:47:27.008883Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9b9ccae4-bc12c6fe-8b9dc0ae-ddf04ca8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:27.014152Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341930555596588:8525]
2025-10-09T21:47:27.016733Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 937, pipe: [1:7559341930555596588:8525]
2025-10-09T21:47:27.029949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341930555596588:8525], now have 1 active actors on pipe
2025-10-09T21:47:27.029998Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:27.030001Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:27.030010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341925898809567:7893]
2025-10-09T21:47:27.030154Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:27.036377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:27.036641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:27.036880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:27.037117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:27.037356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:27.037697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:27.037757Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:27.037763Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:27.037768Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:27.037771Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:27.037783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341925898809567:7893]
2025-10-09T21:47:27.037791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341925898809567:7893]
2025-10-09T21:47:27.037796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341925898809567:7893]
2025-10-09T21:47:27.037881Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:27.037885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:27.037892Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:27.037896Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:27.042195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:27.042212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.042215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:27.042391Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:27.047266Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 938 sessionId: src_id|9b9ccae4-bc12c6fe-8b9dc0ae-ddf04ca8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:27.049961Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:27.299173Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:27.299545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:27.299555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:27.299791Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:27.300274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:27.300346Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:27.300439Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 938 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9b9ccae4-bc12c6fe-8b9dc0ae-ddf04ca8_0
2025-10-09T21:47:27.300485Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341930555596588:8525]
2025-10-09T21:47:27.300628Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 938 sessionId: src_id|9b9ccae4-bc12c6fe-8b9dc0ae-ddf04ca8_0 is DEAD
2025-10-09T21:47:27.300812Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b9ccae4-bc12c6fe-8b9dc0ae-ddf04ca8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:27.300823Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b9ccae4-bc12c6fe-8b9dc0ae-ddf04ca8_0] Write session will restart in 0.009815s
2025-10-09T21:47:27.300848Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b9ccae4-bc12c6fe-8b9dc0ae-ddf04ca8_0] Write session: Do CDS request
2025-10-09T21:47:27.300855Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b9ccae4-bc12c6fe-8b9dc0ae-ddf04ca8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:27.311659Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9b9ccae4-bc12c6fe-8b9dc0ae-ddf04ca8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:27.311783Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:27.311810Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 939
2025-10-09T21:47:27.311984Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 939 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:27.312038Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 939 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:27.312052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=939 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:27.312057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:27.312627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 939 sessionId:  describe result for acl check
2025-10-09T21:47:27.312680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:27.312688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:27.312690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:27.312697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341930555596619:8756] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:27.313295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341930555596619:8756] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:27.317287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341930555596619:8756] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:27.325410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341930555596619:8756] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:27.325434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341930555596619:8756] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:27.325438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341930555596619:8756] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:27.325449Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 939 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:27.342169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:27.345775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:27.346046Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:27.346056Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:27.346071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:27.346139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:27.346186Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341930193777132:7895]
2025-10-09T21:47:27.346764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:27.347517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:27.347589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:27.348047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:27.348134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:27.348135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:27.348137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:27.348138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:27.348139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:27.348140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:27.348141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.348142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.348144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.348150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:27.348161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:27.348165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:27.348181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:27.348184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:27.348188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:27.348192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.348195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.348199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.348209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:27.348218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:27.348224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:27.348239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:27.348244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:27.348249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:27.348253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:27.348258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:27.348262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:27.348266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:27.348270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:27.348275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:27.348280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.348284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.348288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.348293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.348298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.348305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.348334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:27.359892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:27.361364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:27.361371Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:27.361374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:27.361378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:27.361386Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 938 [2:7559341930193777132:7895]
2025-10-09T21:47:27.361402Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:27.361423Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:27.361427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:27.361430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:27.361432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:27.361451Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:27.361457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:27.361465Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:27.361535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.361538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.361917Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 938
2025-10-09T21:47:27.361962Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 938
2025-10-09T21:47:27.361539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.361542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:27.361627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:27.361630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.361644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.361657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.361674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:27.361695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341930193777127:7895]
2025-10-09T21:47:27.361698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341930193777127:7895]
2025-10-09T21:47:27.361701Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341930193777127:7895]
2025-10-09T21:47:27.361710Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:27.361726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:27.361736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:27.361739Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:27.361741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:27.361742Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:27.361744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:27.361794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341930555596711:8756], now have 1 active actors on pipe
2025-10-09T21:47:27.361798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341930555596603:2432], now have 1 active actors on pipe
2025-10-09T21:47:27.361801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341930193777127:7895]
2025-10-09T21:47:27.361810Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:27.362204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:27.362436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:27.362451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:27.362486Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8f3cd6df-aa7af538-35a16278-680752b0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:27.362496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:27.362501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:27.362504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:27.362517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:27.362529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:27.362551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:27.363328Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 939 partition: 0 MaxSeqNo: 25 sessionId: src_id|8f3cd6df-aa7af538-35a16278-680752b0_0
2025-10-09T21:47:27.363780Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b9ccae4-bc12c6fe-8b9dc0ae-ddf04ca8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8f3cd6df-aa7af538-35a16278-680752b0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:27.363155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:27.363168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:27.363190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:27.366535Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8f3cd6df-aa7af538-35a16278-680752b0_0] Write session: try to update token
2025-10-09T21:47:27.366556Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8f3cd6df-aa7af538-35a16278-680752b0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:27.388481Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:27.388500Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:27.388503Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:27.388505Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:27.388515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341930193777127:7895]
2025-10-09T21:47:27.388520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341930193777127:7895]
2025-10-09T21:47:27.388524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341930193777127:7895]
2025-10-09T21:47:27.388572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:27.388580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.388583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:27.388617Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:27.388630Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:27.388634Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:27.391252Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 939 sessionId: src_id|8f3cd6df-aa7af538-35a16278-680752b0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:27.393968Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:27.388722Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:27.400506Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341930555596875:8525]
2025-10-09T21:47:27.400786Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 938, pipe: [1:7559341930555596875:8525]
2025-10-09T21:47:27.554078Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:27.554606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:27.554618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:27.554644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341930555596875:8525], now have 1 active actors on pipe
2025-10-09T21:47:27.555363Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341930555596875:8525]
2025-10-09T21:47:27.555364Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:27.555369Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:27.555396Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:27.555436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 939 reason: error at write: Disconnected sessionId: src_id|8f3cd6df-aa7af538-35a16278-680752b0_0
2025-10-09T21:47:27.555649Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 939 sessionId: src_id|8f3cd6df-aa7af538-35a16278-680752b0_0 is DEAD
2025-10-09T21:47:27.555398Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:27.557413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8f3cd6df-aa7af538-35a16278-680752b0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:27.557426Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8f3cd6df-aa7af538-35a16278-680752b0_0] Write session will restart in 0.008272s
2025-10-09T21:47:27.557453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8f3cd6df-aa7af538-35a16278-680752b0_0] Write session: Do CDS request
2025-10-09T21:47:27.557459Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8f3cd6df-aa7af538-35a16278-680752b0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:27.566584Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8f3cd6df-aa7af538-35a16278-680752b0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:27.566674Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:27.566700Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 940
2025-10-09T21:47:27.567049Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 940 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:27.567088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 940 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:27.567098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=940 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:27.567102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:27.567632Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 940 sessionId:  describe result for acl check
2025-10-09T21:47:27.567691Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:27.567694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:27.567696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:27.567703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341930555596906:8760] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:27.568326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341930555596906:8760] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:27.571595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341930555596906:8760] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:27.576748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341930555596906:8760] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:27.576769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341930555596906:8760] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:27.576773Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341930555596906:8760] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:27.576783Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 940 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:27.584836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:27.585440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:27.585638Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:27.585646Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:27.585658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:27.585702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:27.585752Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341930193777425:7907]
2025-10-09T21:47:27.586389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:27.588046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:27.588100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:27.588461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:27.588536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:27.588543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:27.588544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:27.588546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:27.588547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:27.588548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:27.588549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.588550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.588552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.588558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:27.588569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:27.588573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:27.588590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:27.588594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:27.588598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:27.588602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.588605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.588609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.588619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:27.588635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:27.588641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:27.588646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:27.588650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:27.588655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:27.588658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:27.588663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:27.588667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:27.588673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:27.588677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:27.588682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:27.588687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.588691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.588695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.588701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.588707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.588712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.588739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:27.596035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:27.597148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:27.597165Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:27.597168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:27.597173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:27.597180Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 939 [2:7559341930193777425:7907]
2025-10-09T21:47:27.597196Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:27.597216Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:27.597220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:27.597223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:27.597226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:27.597245Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:27.597252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:27.597259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:27.597329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.597340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.597343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.597346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:27.597423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:27.597432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.597446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.597460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.597479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:27.597508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341930555596881:2432], now have 1 active actors on pipe
2025-10-09T21:47:27.597512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341930555597048:8760], now have 1 active actors on pipe
2025-10-09T21:47:27.597521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341930193777419:7907]
2025-10-09T21:47:27.597524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341930193777419:7907]
2025-10-09T21:47:27.597527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341930193777419:7907]
2025-10-09T21:47:27.597540Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:27.597557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:27.597561Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:27.597565Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:27.597567Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:27.597579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:27.597582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:27.597626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341930193777419:7907]
2025-10-09T21:47:27.597643Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:27.597612Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 939
2025-10-09T21:47:27.597623Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 939
2025-10-09T21:47:27.597818Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:27.597917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:27.597929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:27.597960Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|112afdfd-34d6263e-f7fcb86c-7116edc6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:27.597973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:27.597983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:27.597996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:27.598008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:27.598023Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:27.598061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:27.598263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:27.598280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:27.598304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:27.598418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 940 partition: 0 MaxSeqNo: 25 sessionId: src_id|112afdfd-34d6263e-f7fcb86c-7116edc6_0
2025-10-09T21:47:27.598795Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8f3cd6df-aa7af538-35a16278-680752b0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|112afdfd-34d6263e-f7fcb86c-7116edc6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:27.601726Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|112afdfd-34d6263e-f7fcb86c-7116edc6_0] Write session: try to update token
2025-10-09T21:47:27.601747Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|112afdfd-34d6263e-f7fcb86c-7116edc6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:27.615095Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:27.615115Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:27.615117Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:27.615120Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:27.615129Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341930193777419:7907]
2025-10-09T21:47:27.615134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341930193777419:7907]
2025-10-09T21:47:27.615139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341930193777419:7907]
2025-10-09T21:47:27.615190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:27.615197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.615201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:27.615351Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:27.617058Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 940 sessionId: src_id|112afdfd-34d6263e-f7fcb86c-7116edc6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:27.618210Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:27.618225Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:27.618233Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:27.619689Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:27.655820Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341930555597163:8525]
2025-10-09T21:47:27.656159Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 939, pipe: [1:7559341930555597163:8525]
2025-10-09T21:47:27.792045Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:27.792862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:27.792873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:27.792928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341930555597163:8525], now have 1 active actors on pipe
2025-10-09T21:47:27.793484Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:27.793544Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341930555597163:8525]
2025-10-09T21:47:27.793572Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:27.793579Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:27.793644Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 940 reason: error at write: Disconnected sessionId: src_id|112afdfd-34d6263e-f7fcb86c-7116edc6_0
2025-10-09T21:47:27.793845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 940 sessionId: src_id|112afdfd-34d6263e-f7fcb86c-7116edc6_0 is DEAD
2025-10-09T21:47:27.793544Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:27.794339Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|112afdfd-34d6263e-f7fcb86c-7116edc6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:27.794350Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|112afdfd-34d6263e-f7fcb86c-7116edc6_0] Write session will restart in 0.007862s
2025-10-09T21:47:27.794380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|112afdfd-34d6263e-f7fcb86c-7116edc6_0] Write session: Do CDS request
2025-10-09T21:47:27.794386Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|112afdfd-34d6263e-f7fcb86c-7116edc6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:27.803604Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|112afdfd-34d6263e-f7fcb86c-7116edc6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:27.804163Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:27.804182Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 941
2025-10-09T21:47:27.804460Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 941 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:27.804499Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 941 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:27.804508Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=941 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:27.804513Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:27.805133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 941 sessionId:  describe result for acl check
2025-10-09T21:47:27.805176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:27.805183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:27.805185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:27.805193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341930555597193:8772] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:27.805791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341930555597193:8772] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:27.808665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341930555597193:8772] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:27.816715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341930555597193:8772] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:27.816737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341930555597193:8772] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:27.816742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341930555597193:8772] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:27.816753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 941 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:27.832820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:27.836317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:27.836623Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:27.836628Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:27.836644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:27.836783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:27.836851Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341930193777702:7913]
2025-10-09T21:47:27.837495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:27.838002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:27.838075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:27.838441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:27.838506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:27.838512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:27.838513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:27.838515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:27.838516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:27.838517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:27.838519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.838520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.838522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.838528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:27.838541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:27.838544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:27.838548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:27.838552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:27.838568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:27.838573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.838576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.838580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.838588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:27.838597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:27.838612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:27.838618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:27.838623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:27.838629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:27.838643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:27.838650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:27.838655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:27.838661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:27.838665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:27.838673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:27.838678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.838684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.838689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.838701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.838710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.838715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.838745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:27.848434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:27.849655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:27.849676Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:27.849679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:27.849684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:27.849693Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 940 [2:7559341930193777702:7913]
2025-10-09T21:47:27.849711Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:27.849736Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:27.849746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:27.849750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:27.849753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:27.849775Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:27.849788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:27.849797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:27.849889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.849899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.849901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.849905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:27.849985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:27.849994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.850012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:27.850027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:27.850072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:27.850102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341930555597293:8772], now have 1 active actors on pipe
2025-10-09T21:47:27.850113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341930555597180:2432], now have 1 active actors on pipe
2025-10-09T21:47:27.850122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341930193777697:7913]
2025-10-09T21:47:27.850124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341930193777697:7913]
2025-10-09T21:47:27.850126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341930193777697:7913]
2025-10-09T21:47:27.850140Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:27.850157Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:27.850172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:27.850176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:27.850222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341930193777697:7913]
2025-10-09T21:47:27.850225Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:27.850286Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:27.850290Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:27.850292Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:27.850300Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:27.850644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:27.850655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:27.850688Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|95226bec-e632fe01-a060e79b-340dd646_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:27.850701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:27.850706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:27.850709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:27.850721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:27.850735Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:27.850753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:27.851028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:27.851035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:27.851057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:27.851685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|112afdfd-34d6263e-f7fcb86c-7116edc6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|95226bec-e632fe01-a060e79b-340dd646_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:27.850169Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 940
2025-10-09T21:47:27.850850Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 940
2025-10-09T21:47:27.851250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 941 partition: 0 MaxSeqNo: 25 sessionId: src_id|95226bec-e632fe01-a060e79b-340dd646_0
2025-10-09T21:47:27.854999Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|95226bec-e632fe01-a060e79b-340dd646_0] Write session: try to update token
2025-10-09T21:47:27.855022Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|95226bec-e632fe01-a060e79b-340dd646_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:27.865349Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:27.865413Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:27.865425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341930193777697:7913]
2025-10-09T21:47:27.865484Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:27.865494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:27.865714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:27.865872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:27.866011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:27.866186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:27.866394Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:27.866438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:27.876074Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 941 sessionId: src_id|95226bec-e632fe01-a060e79b-340dd646_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:27.878764Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:27.879045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:27.879067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:27.880400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:27.880435Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:27.880438Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:27.880441Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:27.880442Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:27.880452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341930193777697:7913]
2025-10-09T21:47:27.880457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341930193777697:7913]
2025-10-09T21:47:27.880460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341930193777697:7913]
2025-10-09T21:47:27.880490Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:27.880494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:27.880495Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:27.880501Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:27.880502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:27.880505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:27.880633Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:27.880794Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:27.894163Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341930555597452:8525]
2025-10-09T21:47:27.898134Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 940, pipe: [1:7559341930555597452:8525]
2025-10-09T21:47:28.051704Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:28.052306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341930555597452:8525], now have 1 active actors on pipe
2025-10-09T21:47:28.060215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:28.060243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:28.060424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.060491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.060558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.060624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.060846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.060860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.060927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.060986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.061040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.061386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.061431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.061440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.061502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.061566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.061626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.061695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.061733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.061736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.061804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.061881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.061942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.061999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.062120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.062129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.062191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.062250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.062310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.062374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.062419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.062430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.062493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.062552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.062615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.062678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.062731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.062734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.062800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.062851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.062910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.062967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.063006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.063009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.063073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.063131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.063192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.063253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.063295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.063298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.063370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.063432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.063495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.063557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.063602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.063605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.063670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.063772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.063834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.063894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.063936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.063939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.064609Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:28.064769Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:28.064826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 941 reason: status is not ok: tablet will be restarted right now sessionId: src_id|95226bec-e632fe01-a060e79b-340dd646_0
2025-10-09T21:47:28.065034Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 941 sessionId: src_id|95226bec-e632fe01-a060e79b-340dd646_0 is DEAD
2025-10-09T21:47:28.065167Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:28.065201Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341930555597452:8525]
2025-10-09T21:47:28.065900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95226bec-e632fe01-a060e79b-340dd646_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:28.065913Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95226bec-e632fe01-a060e79b-340dd646_0] Write session will restart in 0.009064s
2025-10-09T21:47:28.065938Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95226bec-e632fe01-a060e79b-340dd646_0] Write session: Do CDS request
2025-10-09T21:47:28.065945Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95226bec-e632fe01-a060e79b-340dd646_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:28.076667Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|95226bec-e632fe01-a060e79b-340dd646_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:28.077704Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:28.077723Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 942
2025-10-09T21:47:28.077897Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 942 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:28.077938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 942 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:28.077950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=942 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:28.077954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:28.078800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 942 sessionId:  describe result for acl check
2025-10-09T21:47:28.078839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:28.078842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:28.078843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:28.078849Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341934850564806:8776] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:28.079424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341934850564806:8776] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:28.083666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341934850564806:8776] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:28.090078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341934850564806:8776] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:28.090101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341934850564806:8776] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:28.090105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341934850564806:8776] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:28.090115Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 942 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:28.099349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:28.102852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:28.103150Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:28.103161Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:28.103174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:28.103234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:28.103282Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341934488745263:7915]
2025-10-09T21:47:28.103879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:28.104273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:28.104333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:28.104642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:28.104701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:28.104708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:28.104709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:28.104711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:28.104712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:28.104713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:28.104714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.104716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.104717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.104723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:28.104733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:28.104737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:28.104741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:28.104745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:28.104750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:28.104753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.104757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.104761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.104768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:28.104776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:28.104782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:28.104787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:28.104791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:28.104796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:28.104800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:28.104809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:28.104813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:28.104819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:28.104823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:28.104828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:28.104833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.104838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.104842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.104847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.104852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.104857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.104884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:28.112799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:28.114391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:28.114409Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:28.114412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:28.114417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:28.114425Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 941 [2:7559341934488745263:7915]
2025-10-09T21:47:28.114439Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:28.114460Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:28.114464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:28.114467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:28.114470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:28.114489Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:28.114495Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:28.114503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:28.114573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.114581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.114584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.114587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:28.114663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:28.114671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.114684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.114694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.114709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:28.114732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341934488745258:7915]
2025-10-09T21:47:28.114735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341934488745258:7915]
2025-10-09T21:47:28.114738Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341934488745258:7915]
2025-10-09T21:47:28.114751Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:28.114768Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:28.114785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:28.114788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:28.114832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341934488745258:7915]
2025-10-09T21:47:28.114834Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:28.115078Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:28.115088Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:28.115090Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:28.115098Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:28.115182Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 941
2025-10-09T21:47:28.115244Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 941
2025-10-09T21:47:28.115577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341934850564753:2432], now have 1 active actors on pipe
2025-10-09T21:47:28.115588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341934850564946:8776], now have 1 active actors on pipe
2025-10-09T21:47:28.115605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:28.115611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:28.115644Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d181540-f57fa497-15f0d8d2-4cd1d98_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:28.115654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:28.115659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:28.115663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:28.115674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:28.115693Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:28.115715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:28.116354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:28.116367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:28.116391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:28.116651Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 942 partition: 0 MaxSeqNo: 25 sessionId: src_id|d181540-f57fa497-15f0d8d2-4cd1d98_0
2025-10-09T21:47:28.117073Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95226bec-e632fe01-a060e79b-340dd646_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d181540-f57fa497-15f0d8d2-4cd1d98_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:28.119720Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d181540-f57fa497-15f0d8d2-4cd1d98_0] Write session: try to update token
2025-10-09T21:47:28.119740Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d181540-f57fa497-15f0d8d2-4cd1d98_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:28.131272Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:28.131291Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:28.131302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341934488745258:7915]
2025-10-09T21:47:28.131345Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:28.131349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:28.131507Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:28.131641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:28.131773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:28.131905Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:28.132093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:28.132135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:28.139116Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 942 sessionId: src_id|d181540-f57fa497-15f0d8d2-4cd1d98_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:28.142327Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:28.142567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:28.142589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:28.142634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:28.142794Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:28.148404Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:28.148421Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:28.148423Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:28.148425Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:28.148437Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341934488745258:7915]
2025-10-09T21:47:28.148442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341934488745258:7915]
2025-10-09T21:47:28.148447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341934488745258:7915]
2025-10-09T21:47:28.148499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:28.148507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.148509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:28.148524Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:28.148531Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:28.148538Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:28.148654Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:28.165382Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341934850565030:8525]
2025-10-09T21:47:28.165807Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 941, pipe: [1:7559341934850565030:8525]
2025-10-09T21:47:28.316445Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:28.316732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341934850565030:8525], now have 1 active actors on pipe
2025-10-09T21:47:28.324830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:28.324860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:28.325038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.325115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.325191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.325257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.325402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.325407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.325482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.325545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.325612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.325677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.325717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.325720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.325788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.325844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.325916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.325974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.326018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.326021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.326203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.326279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.326347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.326436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.326480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.326490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.326561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.326621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.326683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.326753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.326796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.326804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.328657Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:28.328684Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:28.328692Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:28.328727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 942 reason: error at write: Disconnected sessionId: src_id|d181540-f57fa497-15f0d8d2-4cd1d98_0
2025-10-09T21:47:28.328729Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341934850565030:8525]
2025-10-09T21:47:28.328950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 942 sessionId: src_id|d181540-f57fa497-15f0d8d2-4cd1d98_0 is DEAD
2025-10-09T21:47:28.326867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.326929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.326988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.327050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.327091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.327095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.327157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.327209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.327270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.327331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.327377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.327379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.327443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.327496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.327556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.327616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.327681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.327684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.327748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.327798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.327855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.327916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.328039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.328041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.328103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.328154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.328212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.328319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.328357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.328359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.329591Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:28.330272Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d181540-f57fa497-15f0d8d2-4cd1d98_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:28.330284Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d181540-f57fa497-15f0d8d2-4cd1d98_0] Write session will restart in 0.008400s
2025-10-09T21:47:28.330321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d181540-f57fa497-15f0d8d2-4cd1d98_0] Write session: Do CDS request
2025-10-09T21:47:28.330328Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d181540-f57fa497-15f0d8d2-4cd1d98_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:28.339990Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d181540-f57fa497-15f0d8d2-4cd1d98_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:28.340115Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:28.340136Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 943
2025-10-09T21:47:28.340264Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 943 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:28.340315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 943 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:28.340328Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=943 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:28.340331Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:28.340801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 943 sessionId:  describe result for acl check
2025-10-09T21:47:28.340845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:28.340852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:28.340854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:28.340860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341934850565063:8781] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:28.341443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341934850565063:8781] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:28.345391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341934850565063:8781] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:28.349113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341934850565063:8781] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:28.349132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341934850565063:8781] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:28.349136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341934850565063:8781] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:28.349150Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 943 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:28.358512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:28.361634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:28.361888Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:28.361903Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:28.361919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:28.362004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:28.362071Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341934488745530:7917]
2025-10-09T21:47:28.362683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:28.363008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:28.363054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:28.363394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:28.363473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:28.363481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:28.363482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:28.363483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:28.363484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:28.363486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:28.363487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.363488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.363489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.363496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:28.363506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:28.363510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:28.363514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:28.363517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:28.363522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:28.363526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.363529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.363533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.363540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:28.363548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:28.363555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:28.363560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:28.363565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:28.363570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:28.363574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:28.363579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:28.363584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:28.363589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:28.363593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:28.363598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:28.363603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.363608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.363613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.363618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.363624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.363629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.363660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:28.370586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:28.371868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:28.371883Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:28.371886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:28.371891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:28.371900Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 942 [2:7559341934488745530:7917]
2025-10-09T21:47:28.372274Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:28.372303Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:28.372306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:28.372309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:28.372311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:28.372331Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:28.372344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:28.372352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:28.372417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.372424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.372426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.372429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:28.372498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:28.372505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.372518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.372528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.372547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:28.372571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341934488745525:7917]
2025-10-09T21:47:28.372578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341934488745525:7917]
2025-10-09T21:47:28.372580Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341934488745525:7917]
2025-10-09T21:47:28.372592Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:28.372606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:28.372609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341934850565197:8781], now have 1 active actors on pipe
2025-10-09T21:47:28.372614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341934850565038:2432], now have 1 active actors on pipe
2025-10-09T21:47:28.372616Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:28.372618Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:28.372623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:28.372638Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:28.372641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:28.372683Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 942
2025-10-09T21:47:28.372695Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 942
2025-10-09T21:47:28.372682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341934488745525:7917]
2025-10-09T21:47:28.372695Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:28.372737Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:28.372958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:28.372972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:28.373017Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|367d31fe-ac9b30ad-b38308b6-d6df9909_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:28.373030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:28.373035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:28.373037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:28.373049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:28.373061Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:28.373087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:28.373293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:28.373303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:28.373321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:28.373443Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 943 partition: 0 MaxSeqNo: 25 sessionId: src_id|367d31fe-ac9b30ad-b38308b6-d6df9909_0
2025-10-09T21:47:28.373878Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d181540-f57fa497-15f0d8d2-4cd1d98_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|367d31fe-ac9b30ad-b38308b6-d6df9909_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:28.376458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|367d31fe-ac9b30ad-b38308b6-d6df9909_0] Write session: try to update token
2025-10-09T21:47:28.376478Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|367d31fe-ac9b30ad-b38308b6-d6df9909_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:28.388600Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:28.388617Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:28.388627Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341934488745525:7917]
2025-10-09T21:47:28.388674Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:28.388681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:28.388857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:28.389005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:28.389144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:28.389285Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:28.389491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:28.389540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:28.392227Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 943 sessionId: src_id|367d31fe-ac9b30ad-b38308b6-d6df9909_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:28.394762Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:28.395370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:28.395390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:28.395433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:28.395634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:28.405237Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:28.405254Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:28.405257Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:28.405258Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:28.405269Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341934488745525:7917]
2025-10-09T21:47:28.405273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341934488745525:7917]
2025-10-09T21:47:28.405277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341934488745525:7917]
2025-10-09T21:47:28.405321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:28.405329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.405332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:28.405339Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:28.405346Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:28.405351Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:28.405472Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:28.429073Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341934850565314:8525]
2025-10-09T21:47:28.429508Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 942, pipe: [1:7559341934850565314:8525]
2025-10-09T21:47:28.568122Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:28.576153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:28.576180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:28.576347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.576399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.576450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.576507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.576546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.576551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.576612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.576667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.576724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.576782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.576819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.576822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.576883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.576933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.576990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.577049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.577092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.577094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.577154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.577205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.577259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.577318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.577353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.577356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.577441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.577495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.577550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.577608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.577648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.577650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.577711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.577761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.577814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.577884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.577926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.577929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.577986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.578269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.579646Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341934850565314:8525]
2025-10-09T21:47:28.579660Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:28.579664Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:28.579679Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:28.579805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 943 reason: error at write: Disconnected sessionId: src_id|367d31fe-ac9b30ad-b38308b6-d6df9909_0
2025-10-09T21:47:28.580060Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 943 sessionId: src_id|367d31fe-ac9b30ad-b38308b6-d6df9909_0 is DEAD
2025-10-09T21:47:28.580166Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|367d31fe-ac9b30ad-b38308b6-d6df9909_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:28.580177Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|367d31fe-ac9b30ad-b38308b6-d6df9909_0] Write session will restart in 0.008115s
2025-10-09T21:47:28.580201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|367d31fe-ac9b30ad-b38308b6-d6df9909_0] Write session: Do CDS request
2025-10-09T21:47:28.580208Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|367d31fe-ac9b30ad-b38308b6-d6df9909_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:28.578333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.578396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.578437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.578440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.578507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.578576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.578632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.578691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.578735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.578740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.578797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.578853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.578911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.578974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.579043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.579048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.579109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:28.579168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:28.579223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:28.579282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:28.579330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:28.579333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:28.579404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341934850565314:8525], now have 1 active actors on pipe
2025-10-09T21:47:28.580494Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:28.589577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|367d31fe-ac9b30ad-b38308b6-d6df9909_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:28.589684Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:28.589712Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 944
2025-10-09T21:47:28.590280Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 944 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:28.590322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 944 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:28.590332Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=944 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:28.590336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:28.590793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 944 sessionId:  describe result for acl check
2025-10-09T21:47:28.590839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:28.590841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:28.590843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:28.590849Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341934850565358:8785] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:28.591424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341934850565358:8785] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:28.594507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341934850565358:8785] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:28.606770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341934850565358:8785] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:28.606790Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341934850565358:8785] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:28.606795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341934850565358:8785] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:28.606805Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 944 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:28.607141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:28.612778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:28.613058Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:28.613068Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:28.613081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:28.613145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:28.613207Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341934488745832:7929]
2025-10-09T21:47:28.613833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:28.614670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:28.614732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:28.615103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:28.615154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:28.615161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:28.615162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:28.615163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:28.615165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:28.615166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:28.615167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.615169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.615170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.615175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:28.615186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:28.615189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:28.615193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:28.615197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:28.615200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:28.615204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.615208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.615212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.615220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:28.615228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:28.615234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:28.615240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:28.615244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:28.615249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:28.615256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:28.615261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:28.615265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:28.615271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:28.615275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:28.615280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:28.615285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.615290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.615295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.615301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.615306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.615317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.615347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:28.629082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:28.630361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:28.630378Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:28.630382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:28.630388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:28.630397Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 943 [2:7559341934488745832:7929]
2025-10-09T21:47:28.630413Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:28.630437Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:28.630445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:28.630449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:28.630453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:28.630473Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:28.630479Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:28.630488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:28.630559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.630569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.630573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.630576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:28.630663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:28.630668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.630683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.630697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.630720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:28.630753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341934850565320:2432], now have 1 active actors on pipe
2025-10-09T21:47:28.630764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341934850565581:8785], now have 1 active actors on pipe
2025-10-09T21:47:28.630773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341934488745824:7929]
2025-10-09T21:47:28.630776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341934488745824:7929]
2025-10-09T21:47:28.630779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341934488745824:7929]
2025-10-09T21:47:28.630793Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:28.630809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:28.630817Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:28.630822Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:28.630824Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:28.630825Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:28.630828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:28.630887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341934488745824:7929]
2025-10-09T21:47:28.630901Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:28.630918Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:28.631099Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 943
2025-10-09T21:47:28.631145Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 943
2025-10-09T21:47:28.631391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:28.631412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:28.631447Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|def9b966-d4d41292-bc4755c3-dcc030f8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:28.631462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:28.631473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:28.631476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:28.631490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:28.631508Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:28.631533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:28.631772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:28.631784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:28.631800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:28.631986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 944 partition: 0 MaxSeqNo: 25 sessionId: src_id|def9b966-d4d41292-bc4755c3-dcc030f8_0
2025-10-09T21:47:28.632525Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|367d31fe-ac9b30ad-b38308b6-d6df9909_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|def9b966-d4d41292-bc4755c3-dcc030f8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:28.635167Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|def9b966-d4d41292-bc4755c3-dcc030f8_0] Write session: try to update token
2025-10-09T21:47:28.635185Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|def9b966-d4d41292-bc4755c3-dcc030f8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:28.644177Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:28.644194Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:28.644205Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341934488745824:7929]
2025-10-09T21:47:28.644263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:28.644262Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:28.644439Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:28.644582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:28.644712Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:28.644853Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:28.645055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:28.645105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:28.658608Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:28.658626Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:28.658629Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:28.658631Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:28.658642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341934488745824:7929]
2025-10-09T21:47:28.658646Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341934488745824:7929]
2025-10-09T21:47:28.658650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341934488745824:7929]
2025-10-09T21:47:28.658699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:28.658702Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:28.658706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.658708Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:28.658708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:28.658712Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:28.658850Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:28.660365Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 944 sessionId: src_id|def9b966-d4d41292-bc4755c3-dcc030f8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:28.662832Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:28.680045Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341934850565593:8525]
2025-10-09T21:47:28.680394Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 943, pipe: [1:7559341934850565593:8525]
2025-10-09T21:47:28.820622Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:28.821202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:28.821225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:28.821250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341934850565593:8525], now have 1 active actors on pipe
2025-10-09T21:47:28.821510Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341934850565593:8525]
2025-10-09T21:47:28.821547Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:28.821586Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:28.821588Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:28.821652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 944 reason: error at write: Disconnected sessionId: src_id|def9b966-d4d41292-bc4755c3-dcc030f8_0
2025-10-09T21:47:28.821863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 944 sessionId: src_id|def9b966-d4d41292-bc4755c3-dcc030f8_0 is DEAD
2025-10-09T21:47:28.822080Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|def9b966-d4d41292-bc4755c3-dcc030f8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:28.822101Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|def9b966-d4d41292-bc4755c3-dcc030f8_0] Write session will restart in 0.006014s
2025-10-09T21:47:28.822126Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|def9b966-d4d41292-bc4755c3-dcc030f8_0] Write session: Do CDS request
2025-10-09T21:47:28.822134Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|def9b966-d4d41292-bc4755c3-dcc030f8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:28.823349Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:28.829606Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|def9b966-d4d41292-bc4755c3-dcc030f8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:28.829701Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:28.829719Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 945
2025-10-09T21:47:28.829836Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 945 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:28.829885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 945 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:28.829891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=945 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:28.829899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:28.830401Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 945 sessionId:  describe result for acl check
2025-10-09T21:47:28.830446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:28.830448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:28.830450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:28.830457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341934850565639:8797] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:28.831072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341934850565639:8797] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:28.836814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341934850565639:8797] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:28.843075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341934850565639:8797] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:28.843098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341934850565639:8797] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:28.843103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341934850565639:8797] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:28.843115Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 945 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:28.860193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:28.863144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:28.863382Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:28.863387Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:28.863399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:28.863456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:28.863498Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341934488746108:7935]
2025-10-09T21:47:28.864123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:28.864472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:28.864518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:28.864842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:28.864912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:28.864919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:28.864920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:28.864921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:28.864923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:28.864924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:28.864926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.864927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.864928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.864934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:28.864945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:28.864949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:28.864953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:28.864957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:28.864961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:28.864965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.864968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.864972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.864981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:28.864990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:28.864996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:28.865002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:28.865007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:28.865013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:28.865017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:28.865023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:28.865027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:28.865032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:28.865037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:28.865042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:28.865049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.865055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.865060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.865065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.865069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.865074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.865101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:28.872490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:28.874100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:28.874120Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:28.874123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:28.874127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:28.874136Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 944 [2:7559341934488746108:7935]
2025-10-09T21:47:28.874152Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:28.874174Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:28.874177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:28.874181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:28.874184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:28.874205Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:28.874213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:28.874223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:28.874296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.874304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.874306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.874310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:28.874387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:28.874395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.874412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:28.874424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:28.874440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:28.874464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341934488746103:7935]
2025-10-09T21:47:28.874466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341934488746103:7935]
2025-10-09T21:47:28.874468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341934488746103:7935]
2025-10-09T21:47:28.874483Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:28.874498Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:28.874514Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:28.874517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:28.874564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341934488746103:7935]
2025-10-09T21:47:28.874567Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:28.875738Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:28.875750Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:28.875752Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:28.875761Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:28.876069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341934850565755:8797], now have 1 active actors on pipe
2025-10-09T21:47:28.876085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341934850565624:2432], now have 1 active actors on pipe
2025-10-09T21:47:28.876205Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 944
2025-10-09T21:47:28.876213Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 944
2025-10-09T21:47:28.876468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:28.876497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:28.876537Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b51ca423-c7c64881-844ab391-10e8195_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:28.876554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:28.876559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:28.876562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:28.876575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:28.876592Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:28.876616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:28.876968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:28.876978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:28.876998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:28.877134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 945 partition: 0 MaxSeqNo: 25 sessionId: src_id|b51ca423-c7c64881-844ab391-10e8195_0
2025-10-09T21:47:28.877607Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|def9b966-d4d41292-bc4755c3-dcc030f8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b51ca423-c7c64881-844ab391-10e8195_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:28.880608Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b51ca423-c7c64881-844ab391-10e8195_0] Write session: try to update token
2025-10-09T21:47:28.880635Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b51ca423-c7c64881-844ab391-10e8195_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:28.895885Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:28.895903Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:28.895906Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:28.895908Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:28.895918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341934488746103:7935]
2025-10-09T21:47:28.895923Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341934488746103:7935]
2025-10-09T21:47:28.895927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341934488746103:7935]
2025-10-09T21:47:28.895976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:28.895984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:28.895987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:28.896172Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:28.897233Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:28.897239Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:28.897243Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:28.909141Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 945 sessionId: src_id|b51ca423-c7c64881-844ab391-10e8195_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:28.911936Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:28.921661Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341934850565900:8525]
2025-10-09T21:47:28.923064Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 944, pipe: [1:7559341934850565900:8525]
2025-10-09T21:47:29.063906Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:29.064577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:29.064594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:29.064620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341934850565900:8525], now have 1 active actors on pipe
2025-10-09T21:47:29.064884Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:29.065015Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:29.065031Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:29.065103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 945 reason: error at write: Disconnected sessionId: src_id|b51ca423-c7c64881-844ab391-10e8195_0
2025-10-09T21:47:29.065297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 945 sessionId: src_id|b51ca423-c7c64881-844ab391-10e8195_0 is DEAD
2025-10-09T21:47:29.065314Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:29.065371Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341934850565900:8525]
2025-10-09T21:47:29.066259Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b51ca423-c7c64881-844ab391-10e8195_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:29.066274Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b51ca423-c7c64881-844ab391-10e8195_0] Write session will restart in 0.006600s
2025-10-09T21:47:29.066305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b51ca423-c7c64881-844ab391-10e8195_0] Write session: Do CDS request
2025-10-09T21:47:29.066312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b51ca423-c7c64881-844ab391-10e8195_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:29.074734Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b51ca423-c7c64881-844ab391-10e8195_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:29.078403Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:29.078422Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 946
2025-10-09T21:47:29.078818Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 946 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:29.078857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 946 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:29.078869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=946 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:29.078873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:29.080649Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 946 sessionId:  describe result for acl check
2025-10-09T21:47:29.080683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:29.080685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:29.080686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:29.080691Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341939145533217:8801] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:29.081157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341939145533217:8801] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:29.084895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341939145533217:8801] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:29.096356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341939145533217:8801] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:29.096379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341939145533217:8801] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:29.096383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341939145533217:8801] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:29.096398Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 946 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:29.116322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:29.121211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:29.121490Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:29.121502Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:29.121515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:29.121575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:29.121625Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341938783713673:7937]
2025-10-09T21:47:29.122398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:29.122902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:29.122982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:29.123365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:29.123439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:29.123441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:29.123442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:29.123444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:29.123445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:29.123461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:29.123463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.123464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.123465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.123473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:29.123485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:29.123489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:29.123494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:29.123498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:29.123503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:29.123507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.123510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.123513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.123523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:29.123531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:29.123537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:29.123543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:29.123548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:29.123554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:29.123559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:29.123565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:29.123571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:29.123576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:29.123797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:29.123816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:29.123823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.123829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.123848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.123855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.123861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.123867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.123913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:29.134088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:29.135246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:29.135264Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:29.135268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:29.135273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:29.135281Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 945 [2:7559341938783713673:7937]
2025-10-09T21:47:29.135298Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:29.135320Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:29.135323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:29.135326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:29.135329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:29.135349Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:29.135355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:29.135364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:29.135435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.135444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.135446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.135450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:29.135529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:29.135536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.135550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.135561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.135587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:29.135610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341939145533201:2432], now have 1 active actors on pipe
2025-10-09T21:47:29.135614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341939145533277:8801], now have 1 active actors on pipe
2025-10-09T21:47:29.135621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341938783713667:7937]
2025-10-09T21:47:29.135624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341938783713667:7937]
2025-10-09T21:47:29.135626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341938783713667:7937]
2025-10-09T21:47:29.135640Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:29.135656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:29.135672Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:29.135681Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:29.135684Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:29.135688Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:29.135691Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:29.135736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341938783713667:7937]
2025-10-09T21:47:29.135747Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:29.135800Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:29.135747Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 945
2025-10-09T21:47:29.135803Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 945
2025-10-09T21:47:29.136122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:29.136137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:29.136173Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|646dd5c9-78a9e8a5-dbcf8823-76a6e13a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:29.136185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:29.136192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:29.136195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:29.136207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:29.136220Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:29.136244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:29.136441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:29.136453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:29.136477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:29.136656Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 946 partition: 0 MaxSeqNo: 25 sessionId: src_id|646dd5c9-78a9e8a5-dbcf8823-76a6e13a_0
2025-10-09T21:47:29.137144Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b51ca423-c7c64881-844ab391-10e8195_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|646dd5c9-78a9e8a5-dbcf8823-76a6e13a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:29.139811Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|646dd5c9-78a9e8a5-dbcf8823-76a6e13a_0] Write session: try to update token
2025-10-09T21:47:29.139832Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|646dd5c9-78a9e8a5-dbcf8823-76a6e13a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:29.151143Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:29.151160Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:29.151171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341938783713667:7937]
2025-10-09T21:47:29.151228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:29.151270Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:29.151434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:29.151619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:29.151794Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:29.151964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:29.152191Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:29.152247Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:29.165692Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 946 sessionId: src_id|646dd5c9-78a9e8a5-dbcf8823-76a6e13a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:29.165802Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341939145533482:8525]
2025-10-09T21:47:29.168376Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:29.168729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:29.168751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:29.168804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:29.173351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341939145533482:8525], now have 1 active actors on pipe
2025-10-09T21:47:29.168987Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:29.174537Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 945, pipe: [1:7559341939145533482:8525]
2025-10-09T21:47:29.180627Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:29.180645Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:29.180648Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:29.180650Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:29.180661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341938783713667:7937]
2025-10-09T21:47:29.180666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341938783713667:7937]
2025-10-09T21:47:29.180671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341938783713667:7937]
2025-10-09T21:47:29.180715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:29.180722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.180725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:29.180872Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:29.181961Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:29.181970Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:29.181974Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:29.343449Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:29.351856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:29.351890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:29.352072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.352140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.352207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.352276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.352323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.352328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.352397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.352455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.352515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.352579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.352623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.352626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.352690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.352745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.352836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.352896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.352937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.352940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.353012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.353068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.353151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.353212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.353251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.353254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.353323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.353378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.353432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.353491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.353532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.353536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.353604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.353659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.353719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.353778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.353900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.353906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.353969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.354020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.354265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.354330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.354377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.354386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.354448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.355687Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:29.355699Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:29.355703Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:29.355735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 946 reason: error at write: Disconnected sessionId: src_id|646dd5c9-78a9e8a5-dbcf8823-76a6e13a_0
2025-10-09T21:47:29.355748Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341939145533482:8525]
2025-10-09T21:47:29.355937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 946 sessionId: src_id|646dd5c9-78a9e8a5-dbcf8823-76a6e13a_0 is DEAD
2025-10-09T21:47:29.356181Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|646dd5c9-78a9e8a5-dbcf8823-76a6e13a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:29.356192Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|646dd5c9-78a9e8a5-dbcf8823-76a6e13a_0] Write session will restart in 0.009877s
2025-10-09T21:47:29.356218Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|646dd5c9-78a9e8a5-dbcf8823-76a6e13a_0] Write session: Do CDS request
2025-10-09T21:47:29.356226Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|646dd5c9-78a9e8a5-dbcf8823-76a6e13a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:29.354510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.354573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.354627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.354668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.354670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.354732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.354786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.354838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.354895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.354936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.354939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.355003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.355062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.355118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.355178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.355218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.355221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.356374Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:29.367782Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|646dd5c9-78a9e8a5-dbcf8823-76a6e13a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:29.368161Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:29.368178Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 947
2025-10-09T21:47:29.368330Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 947 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:29.368407Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 947 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:29.368419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=947 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:29.368424Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:29.368832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 947 sessionId:  describe result for acl check
2025-10-09T21:47:29.368868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:29.368870Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:29.368874Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:29.368880Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341939145533562:8806] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:29.369429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341939145533562:8806] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:29.375278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341939145533562:8806] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:29.383069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341939145533562:8806] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:29.383089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341939145533562:8806] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:29.383093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341939145533562:8806] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:29.383103Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 947 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:29.383623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:29.388698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:29.388963Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:29.388972Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:29.388985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:29.389056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:29.389116Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341938783713944:7939]
2025-10-09T21:47:29.389707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:29.390161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:29.390229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:29.390550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:29.390614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:29.390621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:29.390622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:29.390624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:29.390625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:29.390627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:29.390628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.390629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.390631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.390636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:29.390657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:29.390660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:29.390664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:29.390667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:29.390671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:29.390674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.390678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.390681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.390689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:29.390696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:29.390701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:29.390707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:29.390711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:29.390715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:29.390719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:29.390724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:29.390727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:29.390732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:29.390735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:29.390740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:29.390745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.390750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.390753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.390757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.390762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.390766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.390798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:29.397644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:29.398981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:29.398999Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:29.399002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:29.399008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:29.399015Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 946 [2:7559341938783713944:7939]
2025-10-09T21:47:29.399030Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:29.399052Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:29.399056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:29.399060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:29.399062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:29.399083Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:29.399096Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:29.399104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:29.399167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.399174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.399176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.399179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:29.399254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:29.399265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.399280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.399293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.399312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:29.399338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341939145533758:8806], now have 1 active actors on pipe
2025-10-09T21:47:29.399345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341939145533490:2432], now have 1 active actors on pipe
2025-10-09T21:47:29.399353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341938783713937:7939]
2025-10-09T21:47:29.399355Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341938783713937:7939]
2025-10-09T21:47:29.399358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341938783713937:7939]
2025-10-09T21:47:29.399374Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:29.399392Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:29.399411Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:29.399418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:29.399432Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 946
2025-10-09T21:47:29.399462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341938783713937:7939]
2025-10-09T21:47:29.399434Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 946
2025-10-09T21:47:29.399469Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:29.399544Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:29.399554Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:29.399557Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:29.399565Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:29.399615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:29.399630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:29.399678Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a6d9a65c-ef3ba64e-fc878a93-34e01fc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:29.399693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:29.399697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:29.399700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:29.399713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:29.399726Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:29.399745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:29.400019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:29.400029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:29.400047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:29.400245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 947 partition: 0 MaxSeqNo: 25 sessionId: src_id|a6d9a65c-ef3ba64e-fc878a93-34e01fc_0
2025-10-09T21:47:29.400614Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|646dd5c9-78a9e8a5-dbcf8823-76a6e13a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a6d9a65c-ef3ba64e-fc878a93-34e01fc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:29.403145Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6d9a65c-ef3ba64e-fc878a93-34e01fc_0] Write session: try to update token
2025-10-09T21:47:29.403163Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6d9a65c-ef3ba64e-fc878a93-34e01fc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:29.411464Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:29.411482Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:29.411494Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341938783713937:7939]
2025-10-09T21:47:29.411546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:29.411581Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:29.411720Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:29.411872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:29.412015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:29.412160Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:29.412856Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:29.412916Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:29.417248Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 947 sessionId: src_id|a6d9a65c-ef3ba64e-fc878a93-34e01fc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:29.419793Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:29.420111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:29.420131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:29.420183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:29.420307Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:29.425032Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:29.425047Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:29.425050Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:29.425052Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:29.425062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341938783713937:7939]
2025-10-09T21:47:29.425066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341938783713937:7939]
2025-10-09T21:47:29.425071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341938783713937:7939]
2025-10-09T21:47:29.425118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:29.425125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.425127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:29.425180Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:29.425191Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:29.425195Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:29.425264Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:29.456152Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341939145533770:8525]
2025-10-09T21:47:29.456642Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 946, pipe: [1:7559341939145533770:8525]
2025-10-09T21:47:29.581727Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:29.592964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:29.592989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:29.593155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.593405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.593695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.593762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.593804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.593810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.593900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.594185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.594251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.594325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.594559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.594569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.594635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.594906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.595158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.595233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.595272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.595275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.595351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.595556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.595842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.596110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.596151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.596155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.596227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.596282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.596345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.596406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.596460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.596468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.596530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.596606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.599391Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6d9a65c-ef3ba64e-fc878a93-34e01fc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:29.599406Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6d9a65c-ef3ba64e-fc878a93-34e01fc_0] Write session will restart in 0.005068s
2025-10-09T21:47:29.599449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6d9a65c-ef3ba64e-fc878a93-34e01fc_0] Write session: Do CDS request
2025-10-09T21:47:29.599459Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6d9a65c-ef3ba64e-fc878a93-34e01fc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:29.596839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.596901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.596941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.596944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.597008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.597063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.597119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.597187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.597230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.597233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.597297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.597352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.597645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.597740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.597785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.597787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.597869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.597944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.598003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.598085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.598127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.598130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.598196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:29.598253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:29.598311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:29.598376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:29.598424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:29.598426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:29.598486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341939145533770:8525], now have 1 active actors on pipe
2025-10-09T21:47:29.599625Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:29.598803Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341939145533770:8525]
2025-10-09T21:47:29.598851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:29.598855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:29.598886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 947 reason: error at write: Disconnected sessionId: src_id|a6d9a65c-ef3ba64e-fc878a93-34e01fc_0
2025-10-09T21:47:29.599087Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 947 sessionId: src_id|a6d9a65c-ef3ba64e-fc878a93-34e01fc_0 is DEAD
2025-10-09T21:47:29.599088Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:29.605595Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6d9a65c-ef3ba64e-fc878a93-34e01fc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:29.607309Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:29.607326Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 948
2025-10-09T21:47:29.607512Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 948 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:29.607546Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 948 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:29.607554Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=948 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:29.607558Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:29.608065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 948 sessionId:  describe result for acl check
2025-10-09T21:47:29.608114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:29.608122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:29.608123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:29.608132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341939145533789:8810] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:29.608691Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341939145533789:8810] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:29.613136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341939145533789:8810] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:29.617955Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341939145533789:8810] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:29.617972Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341939145533789:8810] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:29.617978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341939145533789:8810] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:29.617988Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 948 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:29.628244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:29.631433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:29.631716Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:29.631730Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:29.631745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:29.631817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:29.631865Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341938783714250:7951]
2025-10-09T21:47:29.632462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:29.632841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:29.632897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:29.633208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:29.633281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:29.633283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:29.633284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:29.633286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:29.633287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:29.633288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:29.633289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.633290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.633292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.633298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:29.633309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:29.633313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:29.633317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:29.633321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:29.633325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:29.633329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.633333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.633337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.633345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:29.633353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:29.633361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:29.633366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:29.633381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:29.633387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:29.633391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:29.633396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:29.633401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:29.633405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:29.633409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:29.633414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:29.633419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.633423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.633426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.633430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.633435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.633439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.633465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:29.640417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:29.641460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:29.641476Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:29.641480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:29.641484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:29.641492Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 947 [2:7559341938783714250:7951]
2025-10-09T21:47:29.641508Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:29.641529Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:29.641537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:29.641541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:29.641543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:29.641562Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:29.641576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:29.641588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:29.641655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.641666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.641667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.641671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:29.641742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:29.641752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.641764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.641775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.641792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:29.641813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341938783714245:7951]
2025-10-09T21:47:29.641820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341938783714245:7951]
2025-10-09T21:47:29.641822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341938783714245:7951]
2025-10-09T21:47:29.641831Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:29.641843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:29.641846Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:29.641858Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:29.641859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:29.641877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:29.641884Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:29.641896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341939145533921:8810], now have 1 active actors on pipe
2025-10-09T21:47:29.641904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341939145533776:2432], now have 1 active actors on pipe
2025-10-09T21:47:29.641943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341938783714245:7951]
2025-10-09T21:47:29.641958Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:29.641998Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:29.642302Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 947
2025-10-09T21:47:29.642350Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 947
2025-10-09T21:47:29.642490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:29.642502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:29.642549Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|336bc2f1-92504687-4bdec56c-bda17bc3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:29.642563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:29.642568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:29.642570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:29.642583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:29.642601Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:29.642622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:29.642859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:29.642867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:29.642884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:29.643028Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 948 partition: 0 MaxSeqNo: 25 sessionId: src_id|336bc2f1-92504687-4bdec56c-bda17bc3_0
2025-10-09T21:47:29.643464Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6d9a65c-ef3ba64e-fc878a93-34e01fc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|336bc2f1-92504687-4bdec56c-bda17bc3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:29.651588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|336bc2f1-92504687-4bdec56c-bda17bc3_0] Write session: try to update token
2025-10-09T21:47:29.651615Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|336bc2f1-92504687-4bdec56c-bda17bc3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:29.661420Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:29.661441Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:29.661454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341938783714245:7951]
2025-10-09T21:47:29.661510Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:29.661512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:29.661689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:29.661835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:29.661987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:29.662341Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:29.662550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:29.662596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:29.671210Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:29.671231Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:29.671234Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:29.671236Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:29.671247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341938783714245:7951]
2025-10-09T21:47:29.671251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341938783714245:7951]
2025-10-09T21:47:29.671255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341938783714245:7951]
2025-10-09T21:47:29.671299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:29.671306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.671309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:29.671307Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:29.671319Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:29.671323Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:29.671439Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:29.684004Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 948 sessionId: src_id|336bc2f1-92504687-4bdec56c-bda17bc3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:29.686677Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:29.698852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341939145534054:8525]
2025-10-09T21:47:29.699318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 947, pipe: [1:7559341939145534054:8525]
2025-10-09T21:47:29.839077Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:29.839690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:29.839710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:29.839735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341939145534054:8525], now have 1 active actors on pipe
2025-10-09T21:47:29.839999Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341939145534054:8525]
2025-10-09T21:47:29.840041Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:29.840088Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:29.840091Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:29.840146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 948 reason: error at write: Disconnected sessionId: src_id|336bc2f1-92504687-4bdec56c-bda17bc3_0
2025-10-09T21:47:29.840348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 948 sessionId: src_id|336bc2f1-92504687-4bdec56c-bda17bc3_0 is DEAD
2025-10-09T21:47:29.840162Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:29.842163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|336bc2f1-92504687-4bdec56c-bda17bc3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:29.842177Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|336bc2f1-92504687-4bdec56c-bda17bc3_0] Write session will restart in 0.006565s
2025-10-09T21:47:29.842207Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|336bc2f1-92504687-4bdec56c-bda17bc3_0] Write session: Do CDS request
2025-10-09T21:47:29.842215Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|336bc2f1-92504687-4bdec56c-bda17bc3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:29.849615Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|336bc2f1-92504687-4bdec56c-bda17bc3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:29.849746Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:29.849763Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 949
2025-10-09T21:47:29.850077Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 949 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:29.850118Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 949 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:29.850131Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=949 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:29.850135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:29.850679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 949 sessionId:  describe result for acl check
2025-10-09T21:47:29.850715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:29.850730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:29.850732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:29.850739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341939145534096:8822] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:29.851299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341939145534096:8822] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:29.854781Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341939145534096:8822] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:29.860555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341939145534096:8822] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:29.860574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341939145534096:8822] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:29.860579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341939145534096:8822] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:29.860592Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 949 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:29.871227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:29.874490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:29.874727Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:29.874736Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:29.874749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:29.874805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:29.874851Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341938783714530:7957]
2025-10-09T21:47:29.875488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:29.876045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:29.876104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:29.876475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:29.876527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:29.876529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:29.876530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:29.876531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:29.876532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:29.876534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:29.876535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.876536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.876537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.876543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:29.876555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:29.876559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:29.876563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:29.876567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:29.876571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:29.876575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.876579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.876583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.876591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:29.876600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:29.876607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:29.876612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:29.876617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:29.876623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:29.876627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:29.876632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:29.876637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:29.876642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:29.876647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:29.876652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:29.876657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.876662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.876667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.876672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.876678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.876683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.876713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:29.884895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:29.886054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:29.886070Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:29.886072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:29.886077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:29.886085Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 948 [2:7559341938783714530:7957]
2025-10-09T21:47:29.886100Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:29.886122Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:29.886130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:29.886134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:29.886137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:29.886157Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:29.886169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:29.886182Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:29.886257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.886260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.886262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.886265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:29.886343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:29.886346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.886359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:29.886371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:29.886387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:29.886407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341938783714525:7957]
2025-10-09T21:47:29.886411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341938783714525:7957]
2025-10-09T21:47:29.886413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341938783714525:7957]
2025-10-09T21:47:29.886429Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:29.886445Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:29.886460Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:29.886464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:29.886510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341938783714525:7957]
2025-10-09T21:47:29.886513Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:29.886542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341939145534210:8822], now have 1 active actors on pipe
2025-10-09T21:47:29.886546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341939145534083:2432], now have 1 active actors on pipe
2025-10-09T21:47:29.886661Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:29.886670Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:29.886672Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:29.886679Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:29.886664Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 948
2025-10-09T21:47:29.886681Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 948
2025-10-09T21:47:29.886912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:29.886925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:29.886961Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|13194a54-3459d62a-cabddb81-dd3facd4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:29.886974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:29.886980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:29.886982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:29.886993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:29.887005Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:29.887025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:29.887443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:29.887451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:29.887471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:29.887571Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 949 partition: 0 MaxSeqNo: 25 sessionId: src_id|13194a54-3459d62a-cabddb81-dd3facd4_0
2025-10-09T21:47:29.890692Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|336bc2f1-92504687-4bdec56c-bda17bc3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|13194a54-3459d62a-cabddb81-dd3facd4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:29.893264Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|13194a54-3459d62a-cabddb81-dd3facd4_0] Write session: try to update token
2025-10-09T21:47:29.893286Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|13194a54-3459d62a-cabddb81-dd3facd4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:29.907556Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:29.907575Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:29.907578Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:29.907581Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:29.907591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341938783714525:7957]
2025-10-09T21:47:29.907596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341938783714525:7957]
2025-10-09T21:47:29.907601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341938783714525:7957]
2025-10-09T21:47:29.907651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:29.907660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:29.907663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:29.907868Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:29.908454Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:29.908460Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:29.908465Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:29.924882Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 949 sessionId: src_id|13194a54-3459d62a-cabddb81-dd3facd4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:29.927487Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:29.940485Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341939145534364:8525]
2025-10-09T21:47:29.942305Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 948, pipe: [1:7559341939145534364:8525]
2025-10-09T21:47:30.081188Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:30.081491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:30.081513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:30.081536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341939145534364:8525], now have 1 active actors on pipe
2025-10-09T21:47:30.081842Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:30.081871Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:30.081875Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:30.081934Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341939145534364:8525]
2025-10-09T21:47:30.081945Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 949 reason: error at write: Disconnected sessionId: src_id|13194a54-3459d62a-cabddb81-dd3facd4_0
2025-10-09T21:47:30.082160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 949 sessionId: src_id|13194a54-3459d62a-cabddb81-dd3facd4_0 is DEAD
2025-10-09T21:47:30.082287Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13194a54-3459d62a-cabddb81-dd3facd4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:30.082296Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13194a54-3459d62a-cabddb81-dd3facd4_0] Write session will restart in 0.008642s
2025-10-09T21:47:30.082322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13194a54-3459d62a-cabddb81-dd3facd4_0] Write session: Do CDS request
2025-10-09T21:47:30.082229Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:30.082356Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13194a54-3459d62a-cabddb81-dd3facd4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:30.092712Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|13194a54-3459d62a-cabddb81-dd3facd4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:30.094175Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:30.094196Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 950
2025-10-09T21:47:30.094405Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 950 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:30.094448Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 950 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:30.094455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=950 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:30.094459Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:30.095438Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 950 sessionId:  describe result for acl check
2025-10-09T21:47:30.095484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:30.095486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:30.095488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:30.095497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341943440501715:8826] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:30.096078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341943440501715:8826] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:30.102546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341943440501715:8826] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:30.108173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341943440501715:8826] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:30.108194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341943440501715:8826] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:30.108198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341943440501715:8826] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:30.108209Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 950 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:30.112701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:30.116278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:30.116607Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:30.116612Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:30.116629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:30.116701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:30.116761Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341943078682099:7959]
2025-10-09T21:47:30.117353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:30.117784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:30.117860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:30.118267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:30.118344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:30.118346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:30.118347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:30.118349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:30.118350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:30.118351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:30.118352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.118354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.118355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.118361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:30.118371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:30.118375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:30.118379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:30.118383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:30.118387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:30.118391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.118395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.118399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.118409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:30.118417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:30.118424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:30.118430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:30.118436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:30.118441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:30.118445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:30.118450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:30.118454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:30.118459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:30.118463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:30.118468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:30.118473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.118479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.118484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.118489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.118494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.118499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.118526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:30.126702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:30.127806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:30.127824Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:30.127827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:30.127832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:30.127841Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 949 [2:7559341943078682099:7959]
2025-10-09T21:47:30.127858Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:30.127880Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:30.127889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:30.127892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:30.127895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:30.127915Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:30.127922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:30.127931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:30.128010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.128018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.128020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.128023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:30.128104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:30.128112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.128128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.128144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.128163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:30.128192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341943440501876:8826], now have 1 active actors on pipe
2025-10-09T21:47:30.128207Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341943078682094:7959]
2025-10-09T21:47:30.128213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341943078682094:7959]
2025-10-09T21:47:30.128216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341943078682094:7959]
2025-10-09T21:47:30.128320Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 949
2025-10-09T21:47:30.128233Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:30.128249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341943440501663:2432], now have 1 active actors on pipe
2025-10-09T21:47:30.128254Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:30.128258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:30.128258Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:30.128261Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:30.128272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:30.128276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:30.128318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341943078682094:7959]
2025-10-09T21:47:30.128327Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:30.128347Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:30.128363Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 949
2025-10-09T21:47:30.128504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:30.128520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:30.128571Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9e7dbbde-1e5dd27a-ee4a237b-f0ecd816_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:30.128586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:30.128591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:30.128595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:30.128608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:30.128635Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:30.128668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:30.129033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:30.129045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:30.129067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:30.129193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 950 partition: 0 MaxSeqNo: 25 sessionId: src_id|9e7dbbde-1e5dd27a-ee4a237b-f0ecd816_0
2025-10-09T21:47:30.129665Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13194a54-3459d62a-cabddb81-dd3facd4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9e7dbbde-1e5dd27a-ee4a237b-f0ecd816_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:30.132310Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7dbbde-1e5dd27a-ee4a237b-f0ecd816_0] Write session: try to update token
2025-10-09T21:47:30.132330Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7dbbde-1e5dd27a-ee4a237b-f0ecd816_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:30.141842Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:30.141872Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:30.141883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341943078682094:7959]
2025-10-09T21:47:30.141938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:30.142126Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:30.142185Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:30.142351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:30.142519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:30.142685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:30.142913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:30.142968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:30.155327Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 950 sessionId: src_id|9e7dbbde-1e5dd27a-ee4a237b-f0ecd816_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:30.158470Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:30.159468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:30.159492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:30.159538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:30.162450Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:30.162469Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:30.162472Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:30.162474Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:30.162484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341943078682094:7959]
2025-10-09T21:47:30.162488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341943078682094:7959]
2025-10-09T21:47:30.162492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341943078682094:7959]
2025-10-09T21:47:30.162541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:30.162544Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:30.162549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.162551Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:30.162552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:30.162558Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:30.162712Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:30.159668Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:30.182261Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341943440501948:8525]
2025-10-09T21:47:30.182559Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 949, pipe: [1:7559341943440501948:8525]
2025-10-09T21:47:30.328275Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:30.328536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341943440501948:8525], now have 1 active actors on pipe
2025-10-09T21:47:30.336399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:30.336433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:30.336603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:30.336659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:30.336717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:30.336776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:30.336815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:30.336824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:30.336897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:30.336948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:30.337002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:30.337061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:30.337101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:30.337105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:30.337169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:30.337220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:30.337275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:30.337333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:30.337373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:30.337376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:30.337552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:30.337706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:30.337867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:30.338022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:30.338354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:30.338377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:30.341712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7dbbde-1e5dd27a-ee4a237b-f0ecd816_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:30.341737Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7dbbde-1e5dd27a-ee4a237b-f0ecd816_0] Write session will restart in 0.006933s
2025-10-09T21:47:30.341764Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7dbbde-1e5dd27a-ee4a237b-f0ecd816_0] Write session: Do CDS request
2025-10-09T21:47:30.341772Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7dbbde-1e5dd27a-ee4a237b-f0ecd816_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:30.338500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:30.338567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:30.338629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:30.338688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:30.338728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:30.338731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:30.338797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:30.338855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:30.338909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:30.338967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:30.339082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:30.339085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:30.339144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:30.339200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:30.339262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:30.339321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:30.339358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:30.339362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:30.339423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:30.339478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:30.339530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:30.339586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:30.339691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:30.339695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:30.339759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:30.339814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:30.339871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:30.339930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:30.339993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:30.339997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:30.340060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:30.340115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:30.340174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:30.340236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:30.340393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:30.340397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:30.341645Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:30.340747Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:30.340786Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341943440501948:8525]
2025-10-09T21:47:30.340807Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:30.340810Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:30.340852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 950 reason: error at write: Disconnected sessionId: src_id|9e7dbbde-1e5dd27a-ee4a237b-f0ecd816_0
2025-10-09T21:47:30.341053Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 950 sessionId: src_id|9e7dbbde-1e5dd27a-ee4a237b-f0ecd816_0 is DEAD
2025-10-09T21:47:30.350678Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7dbbde-1e5dd27a-ee4a237b-f0ecd816_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:30.350783Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:30.350802Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 951
2025-10-09T21:47:30.350984Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 951 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:30.351015Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 951 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:30.351021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=951 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:30.351026Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:30.351621Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 951 sessionId:  describe result for acl check
2025-10-09T21:47:30.351654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:30.351656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:30.351658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:30.351674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341943440501971:8831] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:30.352301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341943440501971:8831] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:30.359509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341943440501971:8831] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:30.363848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341943440501971:8831] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:30.363873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341943440501971:8831] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:30.363877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341943440501971:8831] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:30.363889Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 951 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:30.379178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:30.382460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:30.382773Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:30.382788Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:30.382803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:30.382877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:30.382942Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341943078682374:7961]
2025-10-09T21:47:30.383553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:30.384203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:30.384285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:30.384675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:30.384733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:30.384740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:30.384741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:30.384743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:30.384745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:30.384746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:30.384749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.384751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.384752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.384759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:30.384772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:30.384777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:30.384782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:30.384786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:30.384791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:30.384796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.384801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.384806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.384815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:30.384838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:30.384845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:30.384853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:30.384858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:30.384866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:30.384871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:30.384882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:30.384887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:30.384897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:30.384902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:30.384913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:30.384919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.384929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.384934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.384940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.384946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.384953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.384986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:30.394696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:30.395815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:30.395832Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:30.395836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:30.395842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:30.395850Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 950 [2:7559341943078682374:7961]
2025-10-09T21:47:30.395869Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:30.395893Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:30.395902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:30.395906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:30.395909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:30.395930Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:30.395941Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:30.395950Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:30.396019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.396027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.396029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.396032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:30.396113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:30.396123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.396140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.396156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.396177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:30.396208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341943440502152:8831], now have 1 active actors on pipe
2025-10-09T21:47:30.396217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341943440501956:2432], now have 1 active actors on pipe
2025-10-09T21:47:30.396227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341943078682369:7961]
2025-10-09T21:47:30.396229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341943078682369:7961]
2025-10-09T21:47:30.396232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341943078682369:7961]
2025-10-09T21:47:30.396325Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 950
2025-10-09T21:47:30.396365Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 950
2025-10-09T21:47:30.396264Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:30.396280Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:30.396284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:30.396287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:30.396299Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:30.396307Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:30.396311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:30.396357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341943078682369:7961]
2025-10-09T21:47:30.396364Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:30.396410Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:30.396620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:30.396640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:30.396681Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ce913c2f-54ac28f4-3be682c1-ca1a3008_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:30.396691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:30.396696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:30.396699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:30.396726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:30.396740Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:30.396786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:30.397104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:30.397116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:30.397152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:30.397268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 951 partition: 0 MaxSeqNo: 25 sessionId: src_id|ce913c2f-54ac28f4-3be682c1-ca1a3008_0
2025-10-09T21:47:30.397695Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7dbbde-1e5dd27a-ee4a237b-f0ecd816_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ce913c2f-54ac28f4-3be682c1-ca1a3008_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:30.400379Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ce913c2f-54ac28f4-3be682c1-ca1a3008_0] Write session: try to update token
2025-10-09T21:47:30.400409Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ce913c2f-54ac28f4-3be682c1-ca1a3008_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:30.410052Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:30.410072Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:30.410082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341943078682369:7961]
2025-10-09T21:47:30.410136Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:30.410145Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:30.410329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:30.410476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:30.410626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:30.410773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:30.410971Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:30.411011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:30.423554Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 951 sessionId: src_id|ce913c2f-54ac28f4-3be682c1-ca1a3008_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:30.426301Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:30.426399Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:30.426417Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:30.426419Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:30.426421Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:30.426433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341943078682369:7961]
2025-10-09T21:47:30.426436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341943078682369:7961]
2025-10-09T21:47:30.426440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341943078682369:7961]
2025-10-09T21:47:30.426495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:30.426506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:30.426523Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:30.426529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:30.426529Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:30.426534Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:30.426535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.426538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:30.426712Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:30.440732Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341943440502241:8525]
2025-10-09T21:47:30.441161Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 950, pipe: [1:7559341943440502241:8525]
2025-10-09T21:47:30.588651Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:30.589302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341943440502241:8525], now have 1 active actors on pipe
2025-10-09T21:47:30.589531Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:30.589532Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:30.589535Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:30.589540Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:30.589596Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341943440502241:8525]
2025-10-09T21:47:30.589620Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 951 reason: error at write: Disconnected sessionId: src_id|ce913c2f-54ac28f4-3be682c1-ca1a3008_0
2025-10-09T21:47:30.589825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 951 sessionId: src_id|ce913c2f-54ac28f4-3be682c1-ca1a3008_0 is DEAD
2025-10-09T21:47:30.590014Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce913c2f-54ac28f4-3be682c1-ca1a3008_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:30.590025Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce913c2f-54ac28f4-3be682c1-ca1a3008_0] Write session will restart in 0.007277s
2025-10-09T21:47:30.590066Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce913c2f-54ac28f4-3be682c1-ca1a3008_0] Write session: Do CDS request
2025-10-09T21:47:30.590073Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce913c2f-54ac28f4-3be682c1-ca1a3008_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:30.598656Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ce913c2f-54ac28f4-3be682c1-ca1a3008_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:30.598896Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:30.598915Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 952
2025-10-09T21:47:30.599372Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 952 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:30.599419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 952 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:30.599425Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=952 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:30.599429Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:30.599917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 952 sessionId:  describe result for acl check
2025-10-09T21:47:30.599963Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:30.599971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:30.599973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:30.599980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341943440502263:8835] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:30.600555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341943440502263:8835] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:30.602905Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341943440502263:8835] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:30.608860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341943440502263:8835] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:30.608880Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341943440502263:8835] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:30.608884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341943440502263:8835] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:30.608894Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 952 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:30.626344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:30.627220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:30.627536Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:30.627549Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:30.627570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:30.627635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:30.627676Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341943078682669:7963]
2025-10-09T21:47:30.628491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:30.629915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:30.629980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:30.630350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:30.630417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:30.630418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:30.630419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:30.630420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:30.630422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:30.630423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:30.630424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.630425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.630426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.630432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:30.630443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:30.630446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:30.630449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:30.630453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:30.630457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:30.630460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.630464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.630467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.630475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:30.630483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:30.630489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:30.630494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:30.630499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:30.630503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:30.630508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:30.630513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:30.630518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:30.630523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:30.630527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:30.630531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:30.630536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.630541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.630545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.630550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.630554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.630559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.630586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:30.637447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:30.639227Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 951
2025-10-09T21:47:30.639255Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 951
2025-10-09T21:47:30.638759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:30.638775Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:30.638777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:30.638782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:30.638792Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 951 [2:7559341943078682669:7963]
2025-10-09T21:47:30.638806Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:30.638825Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:30.638828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:30.638832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:30.638834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:30.638853Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:30.638858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:30.638866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:30.638937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.638939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.638941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.638943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:30.639031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:30.639034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.639048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.639060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.639077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:30.639099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341943078682663:7963]
2025-10-09T21:47:30.639101Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341943078682663:7963]
2025-10-09T21:47:30.639103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341943078682663:7963]
2025-10-09T21:47:30.639117Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:30.639131Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:30.639135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341943440502344:8835], now have 1 active actors on pipe
2025-10-09T21:47:30.639137Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:30.639139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341943440502248:2432], now have 1 active actors on pipe
2025-10-09T21:47:30.639139Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:30.639148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:30.639161Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:30.639165Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:30.639207Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341943078682663:7963]
2025-10-09T21:47:30.639218Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:30.639231Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:30.639438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:30.639447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:30.639497Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7b3d1c0d-22eec651-bd9866b8-2822b7a9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:30.639507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:30.639512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:30.639515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:30.639527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:30.639540Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:30.639574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:30.639767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:30.639780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:30.639800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:30.639907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 952 partition: 0 MaxSeqNo: 25 sessionId: src_id|7b3d1c0d-22eec651-bd9866b8-2822b7a9_0
2025-10-09T21:47:30.640284Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce913c2f-54ac28f4-3be682c1-ca1a3008_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7b3d1c0d-22eec651-bd9866b8-2822b7a9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:30.642876Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b3d1c0d-22eec651-bd9866b8-2822b7a9_0] Write session: try to update token
2025-10-09T21:47:30.642898Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b3d1c0d-22eec651-bd9866b8-2822b7a9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:30.655806Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:30.655826Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:30.655829Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:30.655831Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:30.655840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341943078682663:7963]
2025-10-09T21:47:30.655844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341943078682663:7963]
2025-10-09T21:47:30.655849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341943078682663:7963]
2025-10-09T21:47:30.655897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:30.655904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.655907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:30.655909Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:30.655916Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:30.655920Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:30.656040Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:30.671314Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 952 sessionId: src_id|7b3d1c0d-22eec651-bd9866b8-2822b7a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:30.673993Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:30.689548Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341943440502532:8525]
2025-10-09T21:47:30.690294Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 951, pipe: [1:7559341943440502532:8525]
2025-10-09T21:47:30.815078Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:30.816310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:30.816321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:30.816355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341943440502532:8525], now have 1 active actors on pipe
2025-10-09T21:47:30.816587Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:30.816736Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341943440502532:8525]
2025-10-09T21:47:30.816922Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:30.816926Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:30.816976Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:30.816985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 952 reason: error at write: Disconnected sessionId: src_id|7b3d1c0d-22eec651-bd9866b8-2822b7a9_0
2025-10-09T21:47:30.817197Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 952 sessionId: src_id|7b3d1c0d-22eec651-bd9866b8-2822b7a9_0 is DEAD
2025-10-09T21:47:30.818125Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b3d1c0d-22eec651-bd9866b8-2822b7a9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:30.818140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b3d1c0d-22eec651-bd9866b8-2822b7a9_0] Write session will restart in 0.005824s
2025-10-09T21:47:30.818170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b3d1c0d-22eec651-bd9866b8-2822b7a9_0] Write session: Do CDS request
2025-10-09T21:47:30.818178Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b3d1c0d-22eec651-bd9866b8-2822b7a9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:30.826534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b3d1c0d-22eec651-bd9866b8-2822b7a9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:30.826639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:30.826658Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 953
2025-10-09T21:47:30.826820Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 953 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:30.826857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 953 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:30.826868Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=953 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:30.826872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:30.827290Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 953 sessionId:  describe result for acl check
2025-10-09T21:47:30.827344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:30.827352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:30.827354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:30.827360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341943440502552:8840] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:30.827936Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341943440502552:8840] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:30.830743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341943440502552:8840] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:30.842549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341943440502552:8840] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:30.842570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341943440502552:8840] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:30.842575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341943440502552:8840] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:30.842585Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 953 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:30.852855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:30.855444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:30.855720Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:30.855729Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:30.855743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:30.855802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:30.855844Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341943078682962:7977]
2025-10-09T21:47:30.856429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:30.857180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:30.857259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:30.857632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:30.857711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:30.857721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:30.857722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:30.857723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:30.857724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:30.857725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:30.857726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.857727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.857728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.857745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:30.857755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:30.857758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:30.857762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:30.857766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:30.857769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:30.857772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.857776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.857779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.857787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:30.857796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:30.857802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:30.857806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:30.857811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:30.857816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:30.857820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:30.857825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:30.857830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:30.857834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:30.857838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:30.857842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:30.857862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.857866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.857870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.857874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.857878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.857883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.857910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:30.866987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:30.868130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:30.868149Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:30.868152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:30.868156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:30.868163Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 952 [2:7559341943078682962:7977]
2025-10-09T21:47:30.868178Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:30.868198Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:30.868207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:30.868211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:30.868214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:30.868234Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:30.868241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:30.868256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:30.868322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.868330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.868332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.868335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:30.868409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:30.868416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.868428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:30.868443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:30.868460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:30.868487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341943078682957:7977]
2025-10-09T21:47:30.868493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341943078682957:7977]
2025-10-09T21:47:30.868495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341943078682957:7977]
2025-10-09T21:47:30.868504Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:30.868520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341943440502537:2432], now have 1 active actors on pipe
2025-10-09T21:47:30.868526Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:30.868528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341943440502714:8840], now have 1 active actors on pipe
2025-10-09T21:47:30.868530Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:30.868532Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:30.868536Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:30.868548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:30.868552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:30.868594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341943078682957:7977]
2025-10-09T21:47:30.868608Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:30.868645Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:30.868662Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 952
2025-10-09T21:47:30.868726Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 952
2025-10-09T21:47:30.868908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:30.868918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:30.868948Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b672cf84-a7442fbb-8e5aaa7f-788186a2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:30.868961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:30.868965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:30.868968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:30.868980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:30.868993Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:30.869010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:30.869803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:30.869816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:30.869858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:30.870002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 953 partition: 0 MaxSeqNo: 25 sessionId: src_id|b672cf84-a7442fbb-8e5aaa7f-788186a2_0
2025-10-09T21:47:30.873580Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b3d1c0d-22eec651-bd9866b8-2822b7a9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b672cf84-a7442fbb-8e5aaa7f-788186a2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:30.876160Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b672cf84-a7442fbb-8e5aaa7f-788186a2_0] Write session: try to update token
2025-10-09T21:47:30.876182Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b672cf84-a7442fbb-8e5aaa7f-788186a2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:30.884793Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:30.884810Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:30.884813Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:30.884815Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:30.884824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341943078682957:7977]
2025-10-09T21:47:30.884828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341943078682957:7977]
2025-10-09T21:47:30.884832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341943078682957:7977]
2025-10-09T21:47:30.884877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:30.884884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:30.884889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:30.884897Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:30.884911Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:30.884915Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:30.885027Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:30.905904Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 953 sessionId: src_id|b672cf84-a7442fbb-8e5aaa7f-788186a2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:30.908735Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:30.916849Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341943440502845:8525]
2025-10-09T21:47:30.917935Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 952, pipe: [1:7559341943440502845:8525]
2025-10-09T21:47:31.056565Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:31.058219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:31.058246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:31.058274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341943440502845:8525], now have 1 active actors on pipe
2025-10-09T21:47:31.058762Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:31.059115Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:31.059143Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:31.059206Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341943440502845:8525]
2025-10-09T21:47:31.059233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 953 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b672cf84-a7442fbb-8e5aaa7f-788186a2_0
2025-10-09T21:47:31.059457Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 953 sessionId: src_id|b672cf84-a7442fbb-8e5aaa7f-788186a2_0 is DEAD
2025-10-09T21:47:31.059653Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b672cf84-a7442fbb-8e5aaa7f-788186a2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:31.059666Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b672cf84-a7442fbb-8e5aaa7f-788186a2_0] Write session will restart in 0.009134s
2025-10-09T21:47:31.059704Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b672cf84-a7442fbb-8e5aaa7f-788186a2_0] Write session: Do CDS request
2025-10-09T21:47:31.059711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b672cf84-a7442fbb-8e5aaa7f-788186a2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:31.070644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b672cf84-a7442fbb-8e5aaa7f-788186a2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:31.070733Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:31.070752Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 954
2025-10-09T21:47:31.071380Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 954 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:31.071427Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 954 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:31.071433Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=954 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:31.071436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:31.072135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 954 sessionId:  describe result for acl check
2025-10-09T21:47:31.072186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:31.072204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:31.072207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:31.072213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341947735470170:8851] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:31.072834Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341947735470170:8851] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:31.076059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341947735470170:8851] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:31.081234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341947735470170:8851] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:31.081254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341947735470170:8851] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:31.081258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341947735470170:8851] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:31.081267Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 954 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:31.088431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:31.091773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:31.092097Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:31.092114Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:31.092129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:31.092203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:31.092255Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341947373650544:7981]
2025-10-09T21:47:31.092927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:31.093456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:31.093536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:31.093960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:31.094057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:31.094066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:31.094068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:31.094069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:31.094070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:31.094071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:31.094072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.094074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.094075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.094083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:31.094094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:31.094098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:31.094102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:31.094105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:31.094110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:31.094113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.094117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.094121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.094133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:31.094141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:31.094148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:31.094153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:31.094158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:31.094163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:31.094167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:31.094172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:31.094177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:31.094183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:31.094187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:31.094193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:31.094198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.094204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.094208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.094213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.094218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.094224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.094256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:31.104003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:31.107056Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 953
2025-10-09T21:47:31.107110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 953
2025-10-09T21:47:31.108280Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 954 partition: 0 MaxSeqNo: 25 sessionId: src_id|e80098e6-855b307f-8f64abdc-4502a69d_0
2025-10-09T21:47:31.108973Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b672cf84-a7442fbb-8e5aaa7f-788186a2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e80098e6-855b307f-8f64abdc-4502a69d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:31.106124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:31.106142Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:31.106146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:31.106152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:31.106163Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 953 [2:7559341947373650544:7981]
2025-10-09T21:47:31.106179Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:31.106203Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:31.106207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:31.106211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:31.106214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:31.106235Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:31.106242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:31.106250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:31.106337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.106339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.106341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.106345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:31.106438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:31.106441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.106457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.106467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.106482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:31.106505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341947373650538:7981]
2025-10-09T21:47:31.106508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341947373650538:7981]
2025-10-09T21:47:31.106509Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341947373650538:7981]
2025-10-09T21:47:31.106530Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:31.106547Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:31.106563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:31.106566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:31.106616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341947373650538:7981]
2025-10-09T21:47:31.106619Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:31.106644Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:31.106647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:31.106650Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:31.106661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341947735470144:2432], now have 1 active actors on pipe
2025-10-09T21:47:31.106661Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:31.106664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341947735470322:8851], now have 1 active actors on pipe
2025-10-09T21:47:31.107254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:31.107263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:31.107314Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e80098e6-855b307f-8f64abdc-4502a69d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:31.107328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:31.107333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:31.107336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:31.107350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:31.107367Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:31.107389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:31.107899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:31.107907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:31.107942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:31.114259Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e80098e6-855b307f-8f64abdc-4502a69d_0] Write session: try to update token
2025-10-09T21:47:31.114291Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e80098e6-855b307f-8f64abdc-4502a69d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:31.127386Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:31.127402Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:31.127406Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:31.127408Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:31.127417Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341947373650538:7981]
2025-10-09T21:47:31.127423Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341947373650538:7981]
2025-10-09T21:47:31.127427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341947373650538:7981]
2025-10-09T21:47:31.127476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:31.127483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.127483Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:31.127486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:31.127490Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:31.127498Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:31.127633Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:31.144799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 954 sessionId: src_id|e80098e6-855b307f-8f64abdc-4502a69d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:31.147593Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:31.159068Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341947735470434:8525]
2025-10-09T21:47:31.160298Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 953, pipe: [1:7559341947735470434:8525]
2025-10-09T21:47:31.311242Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:31.312108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:31.312123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:31.312152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341947735470434:8525], now have 1 active actors on pipe
2025-10-09T21:47:31.312718Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341947735470434:8525]
2025-10-09T21:47:31.312721Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:31.312771Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:31.312775Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:31.312846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 954 reason: error at write: Disconnected sessionId: src_id|e80098e6-855b307f-8f64abdc-4502a69d_0
2025-10-09T21:47:31.313032Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 954 sessionId: src_id|e80098e6-855b307f-8f64abdc-4502a69d_0 is DEAD
2025-10-09T21:47:31.313506Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:31.313707Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e80098e6-855b307f-8f64abdc-4502a69d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:31.313720Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e80098e6-855b307f-8f64abdc-4502a69d_0] Write session will restart in 0.005488s
2025-10-09T21:47:31.313758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e80098e6-855b307f-8f64abdc-4502a69d_0] Write session: Do CDS request
2025-10-09T21:47:31.313764Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e80098e6-855b307f-8f64abdc-4502a69d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:31.321356Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e80098e6-855b307f-8f64abdc-4502a69d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:31.322357Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:31.322374Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 955
2025-10-09T21:47:31.322566Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 955 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:31.322602Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 955 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:31.322608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=955 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:31.322611Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:31.323168Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 955 sessionId:  describe result for acl check
2025-10-09T21:47:31.323200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:31.323202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:31.323204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:31.323210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341947735470464:8856] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:31.323752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341947735470464:8856] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:31.328673Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341947735470464:8856] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:31.340052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341947735470464:8856] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:31.340076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341947735470464:8856] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:31.340082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341947735470464:8856] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:31.340093Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 955 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:31.362445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:31.366640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:31.366946Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:31.366961Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:31.366976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:31.367051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:31.367100Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341947373650822:7983]
2025-10-09T21:47:31.367718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:31.368432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:31.368490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:31.368842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:31.368907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:31.368913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:31.368915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:31.368916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:31.368918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:31.368919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:31.368920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.368922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.368923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.368929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:31.368941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:31.368945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:31.368950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:31.368954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:31.368958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:31.368963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.368967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.368971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.368979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:31.368988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:31.368995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:31.369000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:31.369004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:31.369009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:31.369015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:31.369020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:31.369025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:31.369031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:31.369035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:31.369052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:31.369064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.369070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.369074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.369080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.369084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.369089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.369118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:31.378715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:31.379875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:31.379895Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:31.379899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:31.379904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:31.379912Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 954 [2:7559341947373650822:7983]
2025-10-09T21:47:31.379933Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:31.379955Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:31.379959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:31.379962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:31.379965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:31.379985Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:31.379992Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:31.380000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:31.380066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.380076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.380079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.380082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:31.380167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:31.380176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.380192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.380210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.380232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:31.380253Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341947373650816:7983]
2025-10-09T21:47:31.380257Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341947373650816:7983]
2025-10-09T21:47:31.380260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341947373650816:7983]
2025-10-09T21:47:31.380269Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:31.380282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:31.380292Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:31.380293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:31.380295Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:31.380310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:31.380322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:31.380334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341947735470574:8856], now have 1 active actors on pipe
2025-10-09T21:47:31.380343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341947735470442:2432], now have 1 active actors on pipe
2025-10-09T21:47:31.380385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341947373650816:7983]
2025-10-09T21:47:31.380401Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:31.380429Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:31.380457Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 954
2025-10-09T21:47:31.380462Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 954
2025-10-09T21:47:31.380725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:31.380736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:31.380766Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e10b582d-470c8889-f159151d-6f4fa3ac_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:31.380779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:31.380785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:31.380794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:31.380807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:31.380818Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:31.380839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:31.381230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:31.381244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:31.381269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:31.381399Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 955 partition: 0 MaxSeqNo: 25 sessionId: src_id|e10b582d-470c8889-f159151d-6f4fa3ac_0
2025-10-09T21:47:31.381799Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e80098e6-855b307f-8f64abdc-4502a69d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e10b582d-470c8889-f159151d-6f4fa3ac_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:31.384623Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e10b582d-470c8889-f159151d-6f4fa3ac_0] Write session: try to update token
2025-10-09T21:47:31.384645Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e10b582d-470c8889-f159151d-6f4fa3ac_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:31.398587Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:31.398604Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:31.398606Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:31.398609Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:31.398621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341947373650816:7983]
2025-10-09T21:47:31.398626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341947373650816:7983]
2025-10-09T21:47:31.398630Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341947373650816:7983]
2025-10-09T21:47:31.398683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:31.398690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.398693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:31.398874Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:31.402096Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:31.402114Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:31.402119Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:31.406992Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 955 sessionId: src_id|e10b582d-470c8889-f159151d-6f4fa3ac_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:31.409621Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:31.412995Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341947735470737:8525]
2025-10-09T21:47:31.413340Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 954, pipe: [1:7559341947735470737:8525]
2025-10-09T21:47:31.564263Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:31.565031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:31.565051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:31.565102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341947735470737:8525], now have 1 active actors on pipe
2025-10-09T21:47:31.565376Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:31.565565Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:31.565569Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:31.565573Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:31.565641Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341947735470737:8525]
2025-10-09T21:47:31.565647Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 955 reason: error at write: Disconnected sessionId: src_id|e10b582d-470c8889-f159151d-6f4fa3ac_0
2025-10-09T21:47:31.565864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 955 sessionId: src_id|e10b582d-470c8889-f159151d-6f4fa3ac_0 is DEAD
2025-10-09T21:47:31.566005Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e10b582d-470c8889-f159151d-6f4fa3ac_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:31.566016Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e10b582d-470c8889-f159151d-6f4fa3ac_0] Write session will restart in 0.006890s
2025-10-09T21:47:31.566058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e10b582d-470c8889-f159151d-6f4fa3ac_0] Write session: Do CDS request
2025-10-09T21:47:31.566065Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e10b582d-470c8889-f159151d-6f4fa3ac_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:31.574589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e10b582d-470c8889-f159151d-6f4fa3ac_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:31.576323Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:31.576341Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 956
2025-10-09T21:47:31.576582Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 956 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:31.576615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 956 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:31.576621Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=956 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:31.576625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:31.577062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 956 sessionId:  describe result for acl check
2025-10-09T21:47:31.577093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:31.577095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:31.577097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:31.577103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341947735470753:8860] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:31.577728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341947735470753:8860] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:31.580001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341947735470753:8860] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:31.589503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341947735470753:8860] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:31.589525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341947735470753:8860] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:31.589529Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341947735470753:8860] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:31.589540Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 956 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:31.616746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:31.619892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:31.620129Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:31.620141Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:31.620154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:31.620206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:31.620254Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341947373651107:7985]
2025-10-09T21:47:31.620893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:31.621305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:31.621365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:31.621735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:31.621796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:31.621802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:31.621803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:31.621805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:31.621806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:31.621807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:31.621809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.621810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.621820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.621825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:31.621837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:31.621841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:31.621855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:31.621860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:31.621864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:31.621868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.621872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.621876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.621893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:31.621902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:31.621908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:31.621914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:31.621924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:31.621934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:31.621939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:31.621944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:31.621949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:31.621953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:31.621960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:31.621964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:31.621974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.621979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.621983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.621988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.621996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.622001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.622046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:31.629395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:31.630625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:31.630643Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:31.630647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:31.630652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:31.630660Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 955 [2:7559341947373651107:7985]
2025-10-09T21:47:31.630678Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:31.630699Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:31.630702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:31.630705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:31.630708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:31.630728Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:31.630735Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:31.630744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:31.630814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.630825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.630827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.630830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:31.630915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:31.630925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.630939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.630950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.630972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:31.631001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341947735470816:8860], now have 1 active actors on pipe
2025-10-09T21:47:31.631010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341947735470740:2432], now have 1 active actors on pipe
2025-10-09T21:47:31.631018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341947373651101:7985]
2025-10-09T21:47:31.631022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341947373651101:7985]
2025-10-09T21:47:31.631025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341947373651101:7985]
2025-10-09T21:47:31.631041Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:31.631064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:31.631076Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:31.631080Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:31.631085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:31.631086Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:31.631089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:31.631134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341947373651101:7985]
2025-10-09T21:47:31.631143Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:31.631357Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 955
2025-10-09T21:47:31.631404Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 955
2025-10-09T21:47:31.631223Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:31.631476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:31.631484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:31.631523Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ae3164ae-e73cbffc-6df1997-f063c5f6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:31.631539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:31.631543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:31.631546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:31.631560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:31.631573Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:31.631610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:31.631849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:31.631862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:31.631886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:31.634329Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 956 partition: 0 MaxSeqNo: 25 sessionId: src_id|ae3164ae-e73cbffc-6df1997-f063c5f6_0
2025-10-09T21:47:31.634760Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e10b582d-470c8889-f159151d-6f4fa3ac_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ae3164ae-e73cbffc-6df1997-f063c5f6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:31.637333Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae3164ae-e73cbffc-6df1997-f063c5f6_0] Write session: try to update token
2025-10-09T21:47:31.637350Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae3164ae-e73cbffc-6df1997-f063c5f6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:31.665424Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341947735471026:8525]
2025-10-09T21:47:31.665921Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 955, pipe: [1:7559341947735471026:8525]
2025-10-09T21:47:31.666108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341947735471026:8525], now have 1 active actors on pipe
2025-10-09T21:47:31.672989Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 956 sessionId: src_id|ae3164ae-e73cbffc-6df1997-f063c5f6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:31.675991Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:31.676419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:31.676444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:31.676488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:31.676584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:31.680892Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:31.680910Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:31.680913Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:31.680915Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:31.680925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341947373651101:7985]
2025-10-09T21:47:31.680930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341947373651101:7985]
2025-10-09T21:47:31.680934Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341947373651101:7985]
2025-10-09T21:47:31.680983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:31.680995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.680998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:31.681013Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:31.681028Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:31.681033Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:31.681144Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:31.846734Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:31.855243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:31.855277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:31.855458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:31.855519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:31.855581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:31.855643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:31.855908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:31.855917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:31.855990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:31.856042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:31.856101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:31.856160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:31.856199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:31.856202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:31.856268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:31.856328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:31.856399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:31.856460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:31.856946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:31.856951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:31.857029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:31.857261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:31.857327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:31.857395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:31.857430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:31.857433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:31.857505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:31.857594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:31.857655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:31.857715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:31.857863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:31.857866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:31.857935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:31.857992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:31.860039Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:31.860102Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:31.860105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:31.860140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 956 reason: error at write: Disconnected sessionId: src_id|ae3164ae-e73cbffc-6df1997-f063c5f6_0
2025-10-09T21:47:31.860348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 956 sessionId: src_id|ae3164ae-e73cbffc-6df1997-f063c5f6_0 is DEAD
2025-10-09T21:47:31.860410Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341947735471026:8525]
2025-10-09T21:47:31.858249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:31.858312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:31.858352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:31.858355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:31.858464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:31.858521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:31.858584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:31.858649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:31.858693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:31.858696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:31.858762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:31.858819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:31.858878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:31.858942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:31.858985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:31.858988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:31.859057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:31.859115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:31.859177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:31.859236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:31.859382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:31.859386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:31.859454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:31.859510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:31.859572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:31.859631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:31.859675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:31.859678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:31.860224Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:31.860781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae3164ae-e73cbffc-6df1997-f063c5f6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:31.860809Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae3164ae-e73cbffc-6df1997-f063c5f6_0] Write session will restart in 0.009145s
2025-10-09T21:47:31.860834Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae3164ae-e73cbffc-6df1997-f063c5f6_0] Write session: Do CDS request
2025-10-09T21:47:31.860841Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae3164ae-e73cbffc-6df1997-f063c5f6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:31.871655Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae3164ae-e73cbffc-6df1997-f063c5f6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:31.871705Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:31.871727Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 957
2025-10-09T21:47:31.871934Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 957 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:31.871991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 957 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:31.871997Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=957 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:31.872001Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:31.872460Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 957 sessionId:  describe result for acl check
2025-10-09T21:47:31.872510Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:31.872512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:31.872514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:31.872520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341947735471048:8867] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:31.873122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341947735471048:8867] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:31.876392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341947735471048:8867] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:31.880157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341947735471048:8867] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:31.880175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341947735471048:8867] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:31.880180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341947735471048:8867] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:31.880190Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 957 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:31.897434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:31.900682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:31.900928Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:31.900940Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:31.900953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:31.901012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:31.901057Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341947373651417:7999]
2025-10-09T21:47:31.901642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:31.902067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:31.902134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:31.902481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:31.902547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:31.902557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:31.902558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:31.902560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:31.902562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:31.902563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:31.902564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.902565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.902567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.902574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:31.902585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:31.902589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:31.902593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:31.902598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:31.902602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:31.902606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.902610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.902614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.902622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:31.902630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:31.902636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:31.902642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:31.902647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:31.902652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:31.902656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:31.902661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:31.902665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:31.902670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:31.902675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:31.902680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:31.902686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.902691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.902696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.902702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.902707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.902712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.902744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:31.911385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:31.912505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:31.912518Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:31.912521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:31.912526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:31.912534Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 956 [2:7559341947373651417:7999]
2025-10-09T21:47:31.912549Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:31.912570Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:31.912574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:31.912577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:31.912580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:31.912600Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:31.912605Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:31.912613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:31.912690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.912698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.912700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.912703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:31.912778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:31.912787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.912800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:31.912812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:31.912829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:31.912856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341947373651412:7999]
2025-10-09T21:47:31.912864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341947373651412:7999]
2025-10-09T21:47:31.912867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341947373651412:7999]
2025-10-09T21:47:31.912876Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:31.912891Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:31.912904Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:31.912907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:31.912911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:31.912914Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:31.912917Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:31.912963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341947735471038:2432], now have 1 active actors on pipe
2025-10-09T21:47:31.912971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341947735471089:8867], now have 1 active actors on pipe
2025-10-09T21:47:31.912974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341947373651412:7999]
2025-10-09T21:47:31.912983Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:31.913021Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:31.913523Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 956
2025-10-09T21:47:31.913569Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 956
2025-10-09T21:47:31.914063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:31.914079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:31.914127Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4b1506d6-9b286cc5-451ee90a-7661ef38_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:31.914143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:31.914152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:31.914155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:31.914168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:31.914194Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:31.914219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:31.914534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:31.914543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:31.914563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:31.914691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 957 partition: 0 MaxSeqNo: 25 sessionId: src_id|4b1506d6-9b286cc5-451ee90a-7661ef38_0
2025-10-09T21:47:31.915093Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae3164ae-e73cbffc-6df1997-f063c5f6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4b1506d6-9b286cc5-451ee90a-7661ef38_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:31.917725Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b1506d6-9b286cc5-451ee90a-7661ef38_0] Write session: try to update token
2025-10-09T21:47:31.917744Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b1506d6-9b286cc5-451ee90a-7661ef38_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:31.926828Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:31.926846Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:31.926858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341947373651412:7999]
2025-10-09T21:47:31.926914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:31.926942Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:31.927092Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:31.927272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:31.927416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:31.927621Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:31.927858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:31.927906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:31.934814Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 957 sessionId: src_id|4b1506d6-9b286cc5-451ee90a-7661ef38_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:31.937327Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:31.937619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:31.937641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:31.937684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:31.937862Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:31.942953Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:31.942972Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:31.942974Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:31.942976Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:31.942987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341947373651412:7999]
2025-10-09T21:47:31.942992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341947373651412:7999]
2025-10-09T21:47:31.942996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341947373651412:7999]
2025-10-09T21:47:31.943045Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:31.943046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:31.943054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:31.943057Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:31.943057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:31.943061Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:31.943208Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:31.959754Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341947735471350:8525]
2025-10-09T21:47:31.960069Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 956, pipe: [1:7559341947735471350:8525]
2025-10-09T21:47:32.114230Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:32.114863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341947735471350:8525], now have 1 active actors on pipe
2025-10-09T21:47:32.134892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:32.134928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:32.135107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.135177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.135236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.135310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.135350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.135356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.135423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.135500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.135565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.135634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.135681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.135684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.135754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.135812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.135869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.135930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.135977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.135980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.136055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.136116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.136177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.136239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.136288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.136291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.136364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.136424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.136487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.136551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.136591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.136595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.136661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.136726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.136787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.136851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.136892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.136898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.136963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.137015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.137075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.137139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.137184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.137188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.137251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.137352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.137447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.137506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.137547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.137549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.137612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.137666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.137728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.137821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.137881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.137885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.137951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.138006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.146116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.146202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.146265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.146278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.146758Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341947735471350:8525]
2025-10-09T21:47:32.146808Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:32.146850Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:32.146863Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:32.146897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 957 reason: error at write: Disconnected sessionId: src_id|4b1506d6-9b286cc5-451ee90a-7661ef38_0
2025-10-09T21:47:32.147091Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 957 sessionId: src_id|4b1506d6-9b286cc5-451ee90a-7661ef38_0 is DEAD
2025-10-09T21:47:32.147780Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:32.150134Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b1506d6-9b286cc5-451ee90a-7661ef38_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:32.150149Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b1506d6-9b286cc5-451ee90a-7661ef38_0] Write session will restart in 0.006335s
2025-10-09T21:47:32.150180Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b1506d6-9b286cc5-451ee90a-7661ef38_0] Write session: Do CDS request
2025-10-09T21:47:32.150187Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b1506d6-9b286cc5-451ee90a-7661ef38_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:32.157641Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b1506d6-9b286cc5-451ee90a-7661ef38_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:32.157775Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:32.157803Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 958
2025-10-09T21:47:32.157934Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 958 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:32.157969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 958 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:32.157976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=958 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:32.157987Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:32.158605Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 958 sessionId:  describe result for acl check
2025-10-09T21:47:32.158645Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:32.158647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:32.158649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:32.158656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341952030438669:8878] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:32.159232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341952030438669:8878] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:32.162278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341952030438669:8878] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:32.169640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341952030438669:8878] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:32.169663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341952030438669:8878] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:32.169669Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341952030438669:8878] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:32.169679Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 958 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:32.185358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:32.189188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:32.189472Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:32.189477Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:32.189490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:32.189551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:32.189591Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341951668619003:8003]
2025-10-09T21:47:32.190284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:32.190692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:32.190748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:32.191124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:32.191173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:32.191174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:32.191175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:32.191176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:32.191178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:32.191179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:32.191181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.191182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.191184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.191190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:32.191202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:32.191207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:32.191212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:32.191217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:32.191221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:32.191226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.191230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.191235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.191243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:32.191252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:32.191259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:32.191266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:32.191271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:32.191277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:32.191282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:32.191288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:32.191294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:32.191300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:32.191305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:32.191311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:32.191317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.191322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.191327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.191333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.191338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.191344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.191375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:32.199645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:32.200880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:32.200895Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:32.200899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:32.200905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:32.200912Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 957 [2:7559341951668619003:8003]
2025-10-09T21:47:32.200929Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:32.200953Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:32.200961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:32.200965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:32.200968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:32.200989Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:32.201000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:32.201008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:32.201096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.201105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.201108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.201110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:32.201190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:32.201197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.201212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.201227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.201248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:32.201274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341951668618997:8003]
2025-10-09T21:47:32.201280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341951668618997:8003]
2025-10-09T21:47:32.201282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341951668618997:8003]
2025-10-09T21:47:32.201292Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:32.201309Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:32.201323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:32.201327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:32.201331Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:32.201331Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:32.201333Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:32.201381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341952030438781:8878], now have 1 active actors on pipe
2025-10-09T21:47:32.201390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341952030438654:2432], now have 1 active actors on pipe
2025-10-09T21:47:32.201393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341951668618997:8003]
2025-10-09T21:47:32.201401Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:32.201422Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:32.201449Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 957
2025-10-09T21:47:32.201450Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 957
2025-10-09T21:47:32.201656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:32.201680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:32.201715Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6690e7cf-34cdb645-4292918-5c4fa2fc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:32.201738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:32.201744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:32.201747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:32.201760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:32.201778Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:32.201811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:32.202317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:32.202330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:32.202350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:32.202449Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 958 partition: 0 MaxSeqNo: 25 sessionId: src_id|6690e7cf-34cdb645-4292918-5c4fa2fc_0
2025-10-09T21:47:32.202888Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b1506d6-9b286cc5-451ee90a-7661ef38_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6690e7cf-34cdb645-4292918-5c4fa2fc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:32.205592Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6690e7cf-34cdb645-4292918-5c4fa2fc_0] Write session: try to update token
2025-10-09T21:47:32.205615Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6690e7cf-34cdb645-4292918-5c4fa2fc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:32.221854Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 958 sessionId: src_id|6690e7cf-34cdb645-4292918-5c4fa2fc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:32.224506Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:32.225410Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:32.225431Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:32.225443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341951668618997:8003]
2025-10-09T21:47:32.225517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:32.225530Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:32.225534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:32.225566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:32.225962Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:32.227914Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:32.226503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:32.226834Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:32.227158Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:32.227608Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:32.227697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:32.227740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:32.239248Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:32.239266Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:32.239269Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:32.239276Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:32.239287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341951668618997:8003]
2025-10-09T21:47:32.239292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341951668618997:8003]
2025-10-09T21:47:32.239300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341951668618997:8003]
2025-10-09T21:47:32.239352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:32.239362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.239365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:32.239468Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:32.239475Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:32.239479Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:32.239517Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:32.247197Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341952030438950:8525]
2025-10-09T21:47:32.247545Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 957, pipe: [1:7559341952030438950:8525]
2025-10-09T21:47:32.415756Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:32.416172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341952030438950:8525], now have 1 active actors on pipe
2025-10-09T21:47:32.424310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:32.424340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:32.424547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.424622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.424693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.424761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.425096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.425103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.425171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.425232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.425290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.425359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.425403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.425407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.425475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.425535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.425594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.425653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.425693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.425695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.425770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.425834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.425917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.425976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.426094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.426098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.426166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.426226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.426299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.426367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.426414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.426422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.426488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.426558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.426630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.426704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.428296Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:32.428311Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:32.428315Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:32.428350Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 958 reason: error at write: Disconnected sessionId: src_id|6690e7cf-34cdb645-4292918-5c4fa2fc_0
2025-10-09T21:47:32.428350Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341952030438950:8525]
2025-10-09T21:47:32.428569Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 958 sessionId: src_id|6690e7cf-34cdb645-4292918-5c4fa2fc_0 is DEAD
2025-10-09T21:47:32.428739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6690e7cf-34cdb645-4292918-5c4fa2fc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:32.428752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6690e7cf-34cdb645-4292918-5c4fa2fc_0] Write session will restart in 0.006753s
2025-10-09T21:47:32.428787Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6690e7cf-34cdb645-4292918-5c4fa2fc_0] Write session: Do CDS request
2025-10-09T21:47:32.428793Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6690e7cf-34cdb645-4292918-5c4fa2fc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:32.426753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.426765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.426829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.426888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.426950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.427011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.427052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.427055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.427119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.427177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.427234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.427292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.427348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.427351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.427415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.427475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.427535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.427594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.427635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.427638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.427709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.427767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.427828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.427895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.427940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.427943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.428722Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:32.436611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6690e7cf-34cdb645-4292918-5c4fa2fc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:32.436739Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:32.436770Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 959
2025-10-09T21:47:32.437043Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 959 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:32.437088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 959 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:32.437094Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=959 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:32.437098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:32.437688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 959 sessionId:  describe result for acl check
2025-10-09T21:47:32.437724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:32.437727Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:32.437731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:32.437738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341952030438968:8883] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:32.438449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341952030438968:8883] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:32.446096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341952030438968:8883] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:32.452641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341952030438968:8883] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:32.452658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341952030438968:8883] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:32.452662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341952030438968:8883] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:32.452672Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 959 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:32.467367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:32.470204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:32.470453Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:32.470459Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:32.470474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:32.470546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:32.470590Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341951668619286:8005]
2025-10-09T21:47:32.471223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:32.471590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:32.471653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:32.472036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:32.472129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:32.472138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:32.472140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:32.472141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:32.472142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:32.472144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:32.472145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.472146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.472147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.472153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:32.472164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:32.472168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:32.472171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:32.472175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:32.472179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:32.472182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.472186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.472189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.472197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:32.472206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:32.472212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:32.472217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:32.472221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:32.472225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:32.472229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:32.472234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:32.472239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:32.472244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:32.472248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:32.472253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:32.472257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.472262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.472266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.472271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.472276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.472289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.472318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:32.480467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:32.481713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:32.481732Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:32.481735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:32.481741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:32.481748Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 958 [2:7559341951668619286:8005]
2025-10-09T21:47:32.481940Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:32.481973Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:32.481977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:32.481981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:32.481984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:32.482006Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:32.482018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:32.482052Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:32.482125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.482135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.482137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.482140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:32.482233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:32.482246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.482261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.482273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.482429Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 958
2025-10-09T21:47:32.482448Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 958
2025-10-09T21:47:32.482291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:32.482323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341952030438953:2432], now have 1 active actors on pipe
2025-10-09T21:47:32.482333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341951668619281:8005]
2025-10-09T21:47:32.482336Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341951668619281:8005]
2025-10-09T21:47:32.482338Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341951668619281:8005]
2025-10-09T21:47:32.482354Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:32.482368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341952030439075:8883], now have 1 active actors on pipe
2025-10-09T21:47:32.482380Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:32.482396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:32.482399Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:32.482445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341951668619281:8005]
2025-10-09T21:47:32.482448Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:32.482449Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:32.482453Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:32.482455Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:32.482463Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:32.482591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:32.482618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:32.482653Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|76ce3097-f767c971-a650a394-24138a93_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:32.482672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:32.482683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:32.482687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:32.482700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:32.482736Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:32.482760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:32.483030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:32.483039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:32.483071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:32.483229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 959 partition: 0 MaxSeqNo: 25 sessionId: src_id|76ce3097-f767c971-a650a394-24138a93_0
2025-10-09T21:47:32.483659Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6690e7cf-34cdb645-4292918-5c4fa2fc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|76ce3097-f767c971-a650a394-24138a93_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:32.486328Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76ce3097-f767c971-a650a394-24138a93_0] Write session: try to update token
2025-10-09T21:47:32.486346Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76ce3097-f767c971-a650a394-24138a93_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:32.501294Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 959 sessionId: src_id|76ce3097-f767c971-a650a394-24138a93_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:32.501431Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:32.501452Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:32.501455Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:32.501458Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:32.501467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341951668619281:8005]
2025-10-09T21:47:32.501471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341951668619281:8005]
2025-10-09T21:47:32.501475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341951668619281:8005]
2025-10-09T21:47:32.501525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:32.501529Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:32.501533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.501536Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:32.501536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:32.501540Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:32.501688Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:32.504006Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:32.528646Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341952030439244:8525]
2025-10-09T21:47:32.528983Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 958, pipe: [1:7559341952030439244:8525]
2025-10-09T21:47:32.666061Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:32.666752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:32.666769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:32.666824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341952030439244:8525], now have 1 active actors on pipe
2025-10-09T21:47:32.667126Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341952030439244:8525]
2025-10-09T21:47:32.667138Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:32.667143Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:32.667165Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:32.667223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 959 reason: error at write: Disconnected sessionId: src_id|76ce3097-f767c971-a650a394-24138a93_0
2025-10-09T21:47:32.667235Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:32.667574Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76ce3097-f767c971-a650a394-24138a93_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:32.667586Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76ce3097-f767c971-a650a394-24138a93_0] Write session will restart in 0.007064s
2025-10-09T21:47:32.667613Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76ce3097-f767c971-a650a394-24138a93_0] Write session: Do CDS request
2025-10-09T21:47:32.667622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76ce3097-f767c971-a650a394-24138a93_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:32.667439Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 959 sessionId: src_id|76ce3097-f767c971-a650a394-24138a93_0 is DEAD
2025-10-09T21:47:32.675611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76ce3097-f767c971-a650a394-24138a93_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:32.675757Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:32.675777Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 960
2025-10-09T21:47:32.675941Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 960 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:32.676000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 960 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:32.676015Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=960 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:32.676020Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:32.676570Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 960 sessionId:  describe result for acl check
2025-10-09T21:47:32.676611Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:32.676613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:32.676615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:32.676622Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341952030439263:8887] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:32.677452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341952030439263:8887] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:32.680767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341952030439263:8887] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:32.685309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341952030439263:8887] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:32.685326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341952030439263:8887] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:32.685331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341952030439263:8887] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:32.685341Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 960 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:32.692883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:32.696426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:32.696694Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:32.696704Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:32.696718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:32.696787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:32.696834Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341951668619573:8009]
2025-10-09T21:47:32.697429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:32.697891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:32.697953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:32.698359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:32.698430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:32.698431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:32.698432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:32.698434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:32.698435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:32.698437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:32.698438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.698439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.698440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.698447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:32.698460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:32.698463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:32.698467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:32.698472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:32.698477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:32.698481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.698485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.698489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.698497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:32.698506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:32.698513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:32.698520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:32.698525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:32.698531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:32.698535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:32.698541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:32.698546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:32.698561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:32.698565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:32.698571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:32.698577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.698582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.698587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.698592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.698597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.698602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.698635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:32.708733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:32.710444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:32.710465Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:32.710468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:32.710473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:32.710481Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 959 [2:7559341951668619573:8009]
2025-10-09T21:47:32.710509Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:32.710533Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:32.710543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:32.710546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:32.710548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:32.710570Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:32.710583Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:32.710591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:32.710666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.710674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.710676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.710679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:32.710758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:32.710766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.710779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.710796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.710813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:32.710845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341952030439415:8887], now have 1 active actors on pipe
2025-10-09T21:47:32.710854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341951668619568:8009]
2025-10-09T21:47:32.710857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341951668619568:8009]
2025-10-09T21:47:32.710859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341951668619568:8009]
2025-10-09T21:47:32.710875Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:32.710891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341952030439247:2432], now have 1 active actors on pipe
2025-10-09T21:47:32.710901Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:32.710907Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:32.710912Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:32.710915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:32.710915Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:32.710918Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:32.710967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341951668619568:8009]
2025-10-09T21:47:32.710977Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:32.711028Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:32.711416Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 959
2025-10-09T21:47:32.711453Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 959
2025-10-09T21:47:32.711513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:32.711526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:32.711571Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a15e4b48-c80d75ab-70f8b491-2f3c76b1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:32.711588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:32.711592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:32.711595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:32.711617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:32.711638Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:32.711660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:32.711809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:32.711817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:32.711831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:32.711925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 960 partition: 0 MaxSeqNo: 25 sessionId: src_id|a15e4b48-c80d75ab-70f8b491-2f3c76b1_0
2025-10-09T21:47:32.712337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76ce3097-f767c971-a650a394-24138a93_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a15e4b48-c80d75ab-70f8b491-2f3c76b1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:32.719865Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a15e4b48-c80d75ab-70f8b491-2f3c76b1_0] Write session: try to update token
2025-10-09T21:47:32.719887Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a15e4b48-c80d75ab-70f8b491-2f3c76b1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:32.733296Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:32.733315Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:32.733325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341951668619568:8009]
2025-10-09T21:47:32.733382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:32.733381Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:32.733801Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:32.733973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:32.734133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:32.734275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:32.734472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:32.734521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:32.736073Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 960 sessionId: src_id|a15e4b48-c80d75ab-70f8b491-2f3c76b1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:32.745305Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:32.745673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:32.745694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:32.745738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:32.745859Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:32.752712Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:32.752730Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:32.752733Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:32.752735Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:32.752745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341951668619568:8009]
2025-10-09T21:47:32.752749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341951668619568:8009]
2025-10-09T21:47:32.752753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341951668619568:8009]
2025-10-09T21:47:32.752802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:32.752809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.752812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:32.752811Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:32.752818Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:32.752821Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:32.752989Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:32.767401Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341952030439540:8525]
2025-10-09T21:47:32.767904Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 959, pipe: [1:7559341952030439540:8525]
2025-10-09T21:47:32.914022Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:32.922702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:32.922733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:32.922903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.922971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.923043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.923115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.923357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.923372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.923442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.923503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.923563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.923629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.923670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.923673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.923737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.923792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.923850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.923912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.923951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.923954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.924020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.924107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.924189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.924248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.924289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.924291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.924360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.924417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.924482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.924545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.924587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.924591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.924655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.924717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.924803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.924864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.924905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.924911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.924974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.925028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.925087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.925147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.925191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.925194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.925257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.925313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.925373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.925435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.925474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.925479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.925543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.925597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.925655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.925717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.925762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.925764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.925827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:32.925897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:32.925955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:32.926013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:32.926272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:32.926284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:32.926352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341952030439540:8525], now have 1 active actors on pipe
2025-10-09T21:47:32.927693Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341952030439540:8525]
2025-10-09T21:47:32.927735Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:32.927738Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:32.927767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 960 reason: error at write: Disconnected sessionId: src_id|a15e4b48-c80d75ab-70f8b491-2f3c76b1_0
2025-10-09T21:47:32.928015Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 960 sessionId: src_id|a15e4b48-c80d75ab-70f8b491-2f3c76b1_0 is DEAD
2025-10-09T21:47:32.928107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a15e4b48-c80d75ab-70f8b491-2f3c76b1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:32.928119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a15e4b48-c80d75ab-70f8b491-2f3c76b1_0] Write session will restart in 0.009723s
2025-10-09T21:47:32.928147Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a15e4b48-c80d75ab-70f8b491-2f3c76b1_0] Write session: Do CDS request
2025-10-09T21:47:32.928153Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a15e4b48-c80d75ab-70f8b491-2f3c76b1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:32.928261Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:32.927602Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:32.939688Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a15e4b48-c80d75ab-70f8b491-2f3c76b1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:32.939864Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:32.939883Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 961
2025-10-09T21:47:32.940077Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 961 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:32.940117Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 961 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:32.940145Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=961 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:32.940149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:32.940983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 961 sessionId:  describe result for acl check
2025-10-09T21:47:32.941024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:32.941027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:32.941029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:32.941036Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341952030439570:8892] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:32.941647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341952030439570:8892] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:32.949901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341952030439570:8892] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:32.956129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341952030439570:8892] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:32.956150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341952030439570:8892] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:32.956154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341952030439570:8892] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:32.956164Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 961 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:32.965093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:32.973112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:32.973423Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:32.973437Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:32.973452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:32.973528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:32.973568Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341951668619889:8021]
2025-10-09T21:47:32.974217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:32.974682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:32.974771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:32.975134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:32.975219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:32.975227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:32.975228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:32.975229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:32.975231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:32.975232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:32.975233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.975234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.975235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.975242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:32.975252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:32.975256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:32.975260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:32.975263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:32.975268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:32.975272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.975276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.975280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.975288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:32.975297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:32.975312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:32.975318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:32.975323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:32.975328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:32.975332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:32.975338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:32.975347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:32.975353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:32.975357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:32.975366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:32.975371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.975376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.975379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.975384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.975388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.975392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.975420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:32.984216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:32.985666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:32.985683Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:32.985686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:32.985691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:32.985697Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 960 [2:7559341951668619889:8021]
2025-10-09T21:47:32.985711Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:32.985731Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:32.985740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:32.985743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:32.985746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:32.985763Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:32.985774Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:32.985783Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:32.985869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.985872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.985874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.985877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:32.985948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:32.985956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:32.985970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:32.985984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:32.986003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:32.986136Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 960
2025-10-09T21:47:32.986180Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 960
2025-10-09T21:47:32.986070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341952030439769:8892], now have 1 active actors on pipe
2025-10-09T21:47:32.986080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341952030439545:2432], now have 1 active actors on pipe
2025-10-09T21:47:32.986088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341951668619883:8021]
2025-10-09T21:47:32.986091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341951668619883:8021]
2025-10-09T21:47:32.986093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341951668619883:8021]
2025-10-09T21:47:32.986107Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:32.986122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:32.986138Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:32.986141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:32.986182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341951668619883:8021]
2025-10-09T21:47:32.986185Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:32.986191Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:32.986196Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:32.986198Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:32.986209Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:32.986402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:32.986413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:32.986441Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b5a6982d-ac196aca-d7ac283d-bceb33ee_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:32.986451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:32.986455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:32.986457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:32.986471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:32.986484Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:32.986501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:32.987272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:32.987283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:32.987302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:32.987569Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 961 partition: 0 MaxSeqNo: 25 sessionId: src_id|b5a6982d-ac196aca-d7ac283d-bceb33ee_0
2025-10-09T21:47:32.987992Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a15e4b48-c80d75ab-70f8b491-2f3c76b1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b5a6982d-ac196aca-d7ac283d-bceb33ee_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:32.990696Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5a6982d-ac196aca-d7ac283d-bceb33ee_0] Write session: try to update token
2025-10-09T21:47:32.990717Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5a6982d-ac196aca-d7ac283d-bceb33ee_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:33.004945Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:33.004964Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:33.004975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341951668619883:8021]
2025-10-09T21:47:33.005027Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:33.005066Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:33.005209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:33.005352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:33.005504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:33.005644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:33.005855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:33.005896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:33.010490Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 961 sessionId: src_id|b5a6982d-ac196aca-d7ac283d-bceb33ee_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:33.013250Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:33.013557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:33.013580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:33.013630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:33.013777Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:33.023688Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:33.023706Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:33.023710Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:33.023712Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:33.023722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341951668619883:8021]
2025-10-09T21:47:33.023726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341951668619883:8021]
2025-10-09T21:47:33.023729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341951668619883:8021]
2025-10-09T21:47:33.023779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:33.023779Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:33.023785Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:33.023790Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:33.023794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.023797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:33.023950Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:33.027808Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341956325407156:8525]
2025-10-09T21:47:33.031735Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 960, pipe: [1:7559341956325407156:8525]
2025-10-09T21:47:33.194602Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:33.195152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341956325407156:8525], now have 1 active actors on pipe
2025-10-09T21:47:33.203038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:33.203069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:33.203248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.203317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.203376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.203440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.203482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.203488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.203563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.203629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.203692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.203758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.203811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.203815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.203885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.203942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.204002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.204066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.204110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.204114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.204185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.204247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.204314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.204379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.204423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.204427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.204502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.204564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.204629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.204694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.204739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.204743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.204812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.204871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.204932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.204996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.205040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.205044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.205113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.205174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.205242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.205311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.205354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.205357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.205460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.205515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.205574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.205636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.205682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.205685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.205754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.205813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.205893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.205956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.205997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.206001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.206346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.206417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.206485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.206556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.206624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.206632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.207047Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:33.207058Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:33.207067Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:33.207105Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341956325407156:8525]
2025-10-09T21:47:33.207126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 961 reason: error at write: Disconnected sessionId: src_id|b5a6982d-ac196aca-d7ac283d-bceb33ee_0
2025-10-09T21:47:33.207452Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 961 sessionId: src_id|b5a6982d-ac196aca-d7ac283d-bceb33ee_0 is DEAD
2025-10-09T21:47:33.207516Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5a6982d-ac196aca-d7ac283d-bceb33ee_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:33.207528Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5a6982d-ac196aca-d7ac283d-bceb33ee_0] Write session will restart in 0.009976s
2025-10-09T21:47:33.207553Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5a6982d-ac196aca-d7ac283d-bceb33ee_0] Write session: Do CDS request
2025-10-09T21:47:33.207560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5a6982d-ac196aca-d7ac283d-bceb33ee_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:33.208090Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:33.219027Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5a6982d-ac196aca-d7ac283d-bceb33ee_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:33.219149Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:33.219167Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 962
2025-10-09T21:47:33.219656Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 962 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:33.219705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 962 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:33.219720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=962 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:33.219724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:33.220587Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 962 sessionId:  describe result for acl check
2025-10-09T21:47:33.220632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:33.220634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:33.220635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:33.220642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341956325407215:8904] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:33.221253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341956325407215:8904] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:33.226131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341956325407215:8904] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:33.233472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341956325407215:8904] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:33.233493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341956325407215:8904] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:33.233496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341956325407215:8904] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:33.233507Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 962 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:33.240160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:33.243635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:33.243884Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:33.243897Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:33.243912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:33.243988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:33.244034Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341955963587479:8025]
2025-10-09T21:47:33.244668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:33.245080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:33.245154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:33.245507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:33.245584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:33.245592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:33.245594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:33.245595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:33.245596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:33.245597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:33.245599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.245600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.245601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.245607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:33.245619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:33.245624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:33.245628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:33.245633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:33.245636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:33.245641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.245644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.245647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.245656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:33.245665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:33.245670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:33.245676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:33.245680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:33.245686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:33.245691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:33.245696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:33.245701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:33.245706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:33.245711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:33.245717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:33.245722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.245728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.245732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.245739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.245744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.245749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.245781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:33.256417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:33.257881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:33.257898Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:33.257901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:33.257905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:33.257917Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 961 [2:7559341955963587479:8025]
2025-10-09T21:47:33.257932Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:33.257953Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:33.257962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:33.257966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:33.257969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:33.257990Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:33.257997Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:33.258006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:33.258086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.258095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.258097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.258100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:33.258183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:33.258192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.258208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.258221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.258239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:33.258263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341955963587474:8025]
2025-10-09T21:47:33.258266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341955963587474:8025]
2025-10-09T21:47:33.258268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341955963587474:8025]
2025-10-09T21:47:33.258282Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:33.258302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:33.258316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:33.258320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:33.258366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341955963587474:8025]
2025-10-09T21:47:33.258369Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:33.258408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341956325407380:8904], now have 1 active actors on pipe
2025-10-09T21:47:33.258412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341956325407170:2432], now have 1 active actors on pipe
2025-10-09T21:47:33.258501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:33.258504Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:33.258505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:33.258512Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:33.258774Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 961
2025-10-09T21:47:33.258778Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 961
2025-10-09T21:47:33.258933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:33.258950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:33.258985Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3cd706de-79fabbbe-441c3781-36be970b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:33.258995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:33.258999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:33.259002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:33.259017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:33.259031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:33.259052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:33.259424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:33.259438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:33.259461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:33.259599Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 962 partition: 0 MaxSeqNo: 25 sessionId: src_id|3cd706de-79fabbbe-441c3781-36be970b_0
2025-10-09T21:47:33.260033Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5a6982d-ac196aca-d7ac283d-bceb33ee_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3cd706de-79fabbbe-441c3781-36be970b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:33.262584Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3cd706de-79fabbbe-441c3781-36be970b_0] Write session: try to update token
2025-10-09T21:47:33.262605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3cd706de-79fabbbe-441c3781-36be970b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:33.279082Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 962 sessionId: src_id|3cd706de-79fabbbe-441c3781-36be970b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:33.281698Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:33.281803Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:33.281823Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:33.281825Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:33.281828Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:33.281838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341955963587474:8025]
2025-10-09T21:47:33.281854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341955963587474:8025]
2025-10-09T21:47:33.281858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341955963587474:8025]
2025-10-09T21:47:33.281898Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:33.281903Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:33.281907Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:33.281914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:33.281926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:33.281950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:33.281957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.281961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:33.282140Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:33.307419Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341956325407464:8525]
2025-10-09T21:47:33.307862Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 961, pipe: [1:7559341956325407464:8525]
2025-10-09T21:47:33.449971Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:33.450701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341956325407464:8525], now have 1 active actors on pipe
2025-10-09T21:47:33.451028Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:33.451033Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:33.451050Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:33.451088Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341956325407464:8525]
2025-10-09T21:47:33.451114Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 962 reason: error at write: Disconnected sessionId: src_id|3cd706de-79fabbbe-441c3781-36be970b_0
2025-10-09T21:47:33.451305Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 962 sessionId: src_id|3cd706de-79fabbbe-441c3781-36be970b_0 is DEAD
2025-10-09T21:47:33.451456Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cd706de-79fabbbe-441c3781-36be970b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:33.451469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cd706de-79fabbbe-441c3781-36be970b_0] Write session will restart in 0.008867s
2025-10-09T21:47:33.451503Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cd706de-79fabbbe-441c3781-36be970b_0] Write session: Do CDS request
2025-10-09T21:47:33.451510Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cd706de-79fabbbe-441c3781-36be970b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:33.451168Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:33.464976Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3cd706de-79fabbbe-441c3781-36be970b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:33.466396Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:33.466416Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 963
2025-10-09T21:47:33.466681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 963 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:33.466721Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 963 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:33.466734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=963 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:33.466739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:33.467403Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 963 sessionId:  describe result for acl check
2025-10-09T21:47:33.467449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:33.467458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:33.467460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:33.467467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341956325407555:8908] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:33.468086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341956325407555:8908] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:33.471411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341956325407555:8908] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:33.475265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341956325407555:8908] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:33.475283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341956325407555:8908] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:33.475287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341956325407555:8908] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:33.475297Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 963 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:33.478746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:33.481854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:33.482108Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:33.482124Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:33.482138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:33.482214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:33.482257Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341955963587765:8027]
2025-10-09T21:47:33.482881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:33.483232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:33.483286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:33.483609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:33.483666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:33.483672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:33.483673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:33.483674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:33.483675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:33.483676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:33.483677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.483678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.483680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.483684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:33.483694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:33.483697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:33.483700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:33.483703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:33.483707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:33.483710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.483713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.483715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.483724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:33.483731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:33.483736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:33.483741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:33.483757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:33.483761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:33.483765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:33.483769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:33.483772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:33.483776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:33.483780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:33.483784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:33.483789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.483794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.483797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.483801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.483805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.483811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.483835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:33.491064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:33.492166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:33.492181Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:33.492185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:33.492190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:33.492197Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 962 [2:7559341955963587765:8027]
2025-10-09T21:47:33.492212Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:33.492233Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:33.492236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:33.492239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:33.492242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:33.492261Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:33.492272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:33.492280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:33.492349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.492357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.492359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.492361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:33.492434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:33.492443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.492455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.492465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.492478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:33.492499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341955963587760:8027]
2025-10-09T21:47:33.492501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341955963587760:8027]
2025-10-09T21:47:33.492503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341955963587760:8027]
2025-10-09T21:47:33.492512Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:33.492527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:33.492540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:33.492542Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:33.492544Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:33.492547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:33.492549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:33.492603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341955963587760:8027]
2025-10-09T21:47:33.492618Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:33.492629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341956325407720:8908], now have 1 active actors on pipe
2025-10-09T21:47:33.492632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341956325407467:2432], now have 1 active actors on pipe
2025-10-09T21:47:33.492640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:33.492662Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 962
2025-10-09T21:47:33.492683Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 962
2025-10-09T21:47:33.492735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:33.492745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:33.492774Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|74df5a06-e39108a4-f8a6ef31-895b736f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:33.492786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:33.492790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:33.492792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:33.492803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:33.492812Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:33.492830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:33.493062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:33.493077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:33.493100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:33.493235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 963 partition: 0 MaxSeqNo: 25 sessionId: src_id|74df5a06-e39108a4-f8a6ef31-895b736f_0
2025-10-09T21:47:33.493638Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cd706de-79fabbbe-441c3781-36be970b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|74df5a06-e39108a4-f8a6ef31-895b736f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:33.496390Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74df5a06-e39108a4-f8a6ef31-895b736f_0] Write session: try to update token
2025-10-09T21:47:33.496410Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74df5a06-e39108a4-f8a6ef31-895b736f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:33.508296Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:33.508311Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:33.508321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341955963587760:8027]
2025-10-09T21:47:33.508368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:33.508377Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:33.508541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:33.508685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:33.508822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:33.508962Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:33.509150Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:33.509194Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:33.512686Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 963 sessionId: src_id|74df5a06-e39108a4-f8a6ef31-895b736f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:33.515431Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:33.515734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:33.515756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:33.515797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:33.515890Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:33.523527Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:33.523541Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:33.523544Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:33.523545Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:33.523556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341955963587760:8027]
2025-10-09T21:47:33.523560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341955963587760:8027]
2025-10-09T21:47:33.523564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341955963587760:8027]
2025-10-09T21:47:33.523610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:33.523617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.523620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:33.523627Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:33.523634Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:33.523639Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:33.523758Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:33.551082Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341956325407763:8525]
2025-10-09T21:47:33.551443Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 962, pipe: [1:7559341956325407763:8525]
2025-10-09T21:47:33.671564Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:33.679389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:33.679417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:33.679581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.679649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.679720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.679784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.679913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.679925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.679991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.680045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.680111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.680181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.680227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.680234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.680335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.680393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.680453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.680518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.680569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.680578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.680650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.680713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.680774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.680841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.680884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.680894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.680961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.681021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.681087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.681147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.681282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.681292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.681356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.681409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.681465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.681523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.681653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.681657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.681719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.681771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.681828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.681898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.681934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.681938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.681998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.682152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.682213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.682274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.682319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.682326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.682385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.682444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.682506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.682568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.682609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.682616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.682678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:33.682738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:33.682799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:33.682861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:33.682903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:33.682912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:33.682976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341956325407763:8525], now have 1 active actors on pipe
2025-10-09T21:47:33.683244Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:33.683253Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341956325407763:8525]
2025-10-09T21:47:33.683260Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:33.683275Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:33.683291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 963 reason: error at write: Disconnected sessionId: src_id|74df5a06-e39108a4-f8a6ef31-895b736f_0
2025-10-09T21:47:33.683591Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 963 sessionId: src_id|74df5a06-e39108a4-f8a6ef31-895b736f_0 is DEAD
2025-10-09T21:47:33.683681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74df5a06-e39108a4-f8a6ef31-895b736f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:33.683693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74df5a06-e39108a4-f8a6ef31-895b736f_0] Write session will restart in 0.005706s
2025-10-09T21:47:33.683719Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74df5a06-e39108a4-f8a6ef31-895b736f_0] Write session: Do CDS request
2025-10-09T21:47:33.683725Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74df5a06-e39108a4-f8a6ef31-895b736f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:33.683923Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:33.690542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74df5a06-e39108a4-f8a6ef31-895b736f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:33.690654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:33.690670Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 964
2025-10-09T21:47:33.690792Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 964 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:33.690827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 964 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:33.690845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=964 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:33.690849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:33.691345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 964 sessionId:  describe result for acl check
2025-10-09T21:47:33.691381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:33.691383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:33.691385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:33.691391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341956325407785:8912] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:33.691916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341956325407785:8912] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:33.694722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341956325407785:8912] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:33.698154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341956325407785:8912] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:33.698171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341956325407785:8912] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:33.698175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341956325407785:8912] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:33.698185Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 964 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:33.707211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:33.709804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:33.710163Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:33.710172Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:33.710186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:33.710250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:33.710294Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341955963588056:8031]
2025-10-09T21:47:33.710790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:33.711096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:33.711137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:33.711423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:33.711467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:33.711468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:33.711469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:33.711470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:33.711471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:33.711472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:33.711473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.711474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.711475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.711479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:33.711489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:33.711492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:33.711495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:33.711498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:33.711501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:33.711504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.711506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.711509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.711515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:33.711523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:33.711528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:33.711533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:33.711536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:33.711540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:33.711543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:33.711548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:33.711552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:33.711556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:33.711559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:33.711563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:33.711568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.711571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.711575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.711578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.711583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.711587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.711610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:33.719496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:33.720682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:33.720700Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:33.720703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:33.720708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:33.720717Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 963 [2:7559341955963588056:8031]
2025-10-09T21:47:33.720734Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:33.720754Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:33.720758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:33.720762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:33.720765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:33.720786Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:33.720793Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:33.720803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:33.720872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.720882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.720884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.720887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:33.720967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:33.720976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.720993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.721002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.721019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:33.721042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341956325407904:8912], now have 1 active actors on pipe
2025-10-09T21:47:33.721053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341956325407765:2432], now have 1 active actors on pipe
2025-10-09T21:47:33.721062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341955963588051:8031]
2025-10-09T21:47:33.721064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341955963588051:8031]
2025-10-09T21:47:33.721066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341955963588051:8031]
2025-10-09T21:47:33.721079Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:33.721095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:33.721106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:33.721108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:33.721109Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:33.721112Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:33.721112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:33.721101Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 963
2025-10-09T21:47:33.721101Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 963
2025-10-09T21:47:33.721157Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341955963588051:8031]
2025-10-09T21:47:33.721166Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:33.721170Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:33.721185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:33.721195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:33.721280Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dde5d2f9-1ed4ff0f-50dbbe64-3b548d0e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:33.721294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:33.721298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:33.721301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:33.721312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:33.721323Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:33.721344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:33.721557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:33.721569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:33.721587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:33.721705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 964 partition: 0 MaxSeqNo: 25 sessionId: src_id|dde5d2f9-1ed4ff0f-50dbbe64-3b548d0e_0
2025-10-09T21:47:33.722116Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74df5a06-e39108a4-f8a6ef31-895b736f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dde5d2f9-1ed4ff0f-50dbbe64-3b548d0e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:33.724644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dde5d2f9-1ed4ff0f-50dbbe64-3b548d0e_0] Write session: try to update token
2025-10-09T21:47:33.724667Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dde5d2f9-1ed4ff0f-50dbbe64-3b548d0e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:33.739105Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 964 sessionId: src_id|dde5d2f9-1ed4ff0f-50dbbe64-3b548d0e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:33.739638Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:33.739659Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:33.739662Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:33.739664Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:33.739673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341955963588051:8031]
2025-10-09T21:47:33.739678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341955963588051:8031]
2025-10-09T21:47:33.739681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341955963588051:8031]
2025-10-09T21:47:33.739732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:33.739737Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:33.739741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.739743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:33.739744Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:33.739748Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:33.739904Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:33.741615Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:33.783507Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341956325408062:8525]
2025-10-09T21:47:33.783773Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 963, pipe: [1:7559341956325408062:8525]
2025-10-09T21:47:33.899301Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:33.899972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:33.899988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:33.900036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341956325408062:8525], now have 1 active actors on pipe
2025-10-09T21:47:33.900246Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:33.900285Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341956325408062:8525]
2025-10-09T21:47:33.900316Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:33.900327Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:33.900331Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:33.900394Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 964 reason: error at write: Disconnected sessionId: src_id|dde5d2f9-1ed4ff0f-50dbbe64-3b548d0e_0
2025-10-09T21:47:33.900604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 964 sessionId: src_id|dde5d2f9-1ed4ff0f-50dbbe64-3b548d0e_0 is DEAD
2025-10-09T21:47:33.900778Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dde5d2f9-1ed4ff0f-50dbbe64-3b548d0e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:33.900789Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dde5d2f9-1ed4ff0f-50dbbe64-3b548d0e_0] Write session will restart in 0.009917s
2025-10-09T21:47:33.900825Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dde5d2f9-1ed4ff0f-50dbbe64-3b548d0e_0] Write session: Do CDS request
2025-10-09T21:47:33.900831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dde5d2f9-1ed4ff0f-50dbbe64-3b548d0e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:33.911589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dde5d2f9-1ed4ff0f-50dbbe64-3b548d0e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:33.911745Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:33.911761Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 965
2025-10-09T21:47:33.911955Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 965 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:33.911989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 965 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:33.912002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=965 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:33.912006Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:33.912458Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 965 sessionId:  describe result for acl check
2025-10-09T21:47:33.912494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:33.912496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:33.912497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:33.912505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341956325408147:8917] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:33.913056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341956325408147:8917] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:33.916304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341956325408147:8917] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:33.919548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341956325408147:8917] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:33.919563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341956325408147:8917] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:33.919567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341956325408147:8917] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:33.919575Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 965 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:33.921317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:33.923957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:33.924177Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:33.924187Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:33.924200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:33.924276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:33.924333Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341955963588375:8043]
2025-10-09T21:47:33.924940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:33.925238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:33.925271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:33.925496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:33.925530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:33.925535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:33.925536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:33.925537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:33.925537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:33.925538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:33.925538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.925539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.925540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.925543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:33.925550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:33.925552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:33.925554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:33.925558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:33.925561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:33.925565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.925568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.925572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.925578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:33.925586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:33.925592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:33.925597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:33.925601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:33.925606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:33.925610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:33.925614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:33.925616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:33.925619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:33.925621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:33.925624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:33.925627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.925630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.925633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.925636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.925639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.925642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.925657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:33.932225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:33.933312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:33.933326Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:33.933329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:33.933333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:33.933341Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 964 [2:7559341955963588375:8043]
2025-10-09T21:47:33.933353Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:33.933374Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:33.933377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:33.933380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:33.933382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:33.933400Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:33.933406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:33.933414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:33.933477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.933485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.933487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.933489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:33.933559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:33.933567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.933579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:33.933590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:33.933607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:33.933630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341956325408069:2432], now have 1 active actors on pipe
2025-10-09T21:47:33.933638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341956325408349:8917], now have 1 active actors on pipe
2025-10-09T21:47:33.933646Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341955963588370:8043]
2025-10-09T21:47:33.933656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341955963588370:8043]
2025-10-09T21:47:33.933658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341955963588370:8043]
2025-10-09T21:47:33.933670Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:33.933684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:33.933685Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:33.933689Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:33.933692Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:33.933702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:33.933705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:33.933697Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 964
2025-10-09T21:47:33.933717Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 964
2025-10-09T21:47:33.933746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341955963588370:8043]
2025-10-09T21:47:33.933758Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:33.933765Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:33.933779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:33.933784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:33.933848Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b2a11caf-c94444be-c1236f80-9a496750_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:33.933864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:33.933868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:33.933871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:33.933881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:33.933892Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:33.933917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:33.934185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:33.934198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:33.934222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:33.934350Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 965 partition: 0 MaxSeqNo: 25 sessionId: src_id|b2a11caf-c94444be-c1236f80-9a496750_0
2025-10-09T21:47:33.935349Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dde5d2f9-1ed4ff0f-50dbbe64-3b548d0e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b2a11caf-c94444be-c1236f80-9a496750_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:33.937834Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b2a11caf-c94444be-c1236f80-9a496750_0] Write session: try to update token
2025-10-09T21:47:33.937865Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b2a11caf-c94444be-c1236f80-9a496750_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:33.948852Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:33.948872Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:33.948875Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:33.948877Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:33.948887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341955963588370:8043]
2025-10-09T21:47:33.948891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341955963588370:8043]
2025-10-09T21:47:33.948894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341955963588370:8043]
2025-10-09T21:47:33.948941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:33.948948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:33.948948Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:33.948951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:33.948955Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:33.948959Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:33.949086Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:33.953385Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 965 sessionId: src_id|b2a11caf-c94444be-c1236f80-9a496750_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:33.956003Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:34.000639Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341960620375662:8525]
2025-10-09T21:47:34.000976Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 964, pipe: [1:7559341960620375662:8525]
2025-10-09T21:47:34.108533Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:34.109396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:34.109417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:34.109467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341960620375662:8525], now have 1 active actors on pipe
2025-10-09T21:47:34.109798Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341960620375662:8525]
2025-10-09T21:47:34.109798Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:34.109854Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:34.109857Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:34.109890Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:34.109926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 965 reason: error at write: Disconnected sessionId: src_id|b2a11caf-c94444be-c1236f80-9a496750_0
2025-10-09T21:47:34.110161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 965 sessionId: src_id|b2a11caf-c94444be-c1236f80-9a496750_0 is DEAD
2025-10-09T21:47:34.110357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b2a11caf-c94444be-c1236f80-9a496750_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:34.110369Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b2a11caf-c94444be-c1236f80-9a496750_0] Write session will restart in 0.009806s
2025-10-09T21:47:34.110399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b2a11caf-c94444be-c1236f80-9a496750_0] Write session: Do CDS request
2025-10-09T21:47:34.110406Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b2a11caf-c94444be-c1236f80-9a496750_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:34.121629Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b2a11caf-c94444be-c1236f80-9a496750_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:34.121758Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:34.121776Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 966
2025-10-09T21:47:34.122063Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 966 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:34.122109Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 966 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:34.122126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=966 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:34.122131Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:34.122751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 966 sessionId:  describe result for acl check
2025-10-09T21:47:34.122797Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:34.122799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:34.122801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:34.122807Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341960620375740:8928] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:34.123408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341960620375740:8928] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:34.127166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341960620375740:8928] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:34.130594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341960620375740:8928] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:34.130610Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341960620375740:8928] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:34.130614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341960620375740:8928] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:34.130624Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 966 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:34.134439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:34.139094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:34.139377Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:34.139386Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:34.139399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:34.139463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:34.139512Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341960258555968:8047]
2025-10-09T21:47:34.140080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:34.140420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:34.140466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:34.140762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:34.140813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:34.140819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:34.140820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:34.140821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:34.140822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:34.140823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:34.140825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.140826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.140827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.140832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:34.140842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:34.140845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:34.140849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:34.140852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:34.140855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:34.140858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.140862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.140865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.140872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:34.140880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:34.140886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:34.140891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:34.140895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:34.140899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:34.140903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:34.140908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:34.140913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:34.140917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:34.140921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:34.140928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:34.140932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.140936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.140940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.140949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.140953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.140957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.140984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:34.148564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:34.150085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:34.150101Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:34.150104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:34.150109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:34.150117Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 965 [2:7559341960258555968:8047]
2025-10-09T21:47:34.150132Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:34.150151Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:34.150154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:34.150159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:34.150161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:34.150180Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:34.150188Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:34.150201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:34.150262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.150270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.150272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.150275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:34.150359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:34.150370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.150384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.150394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.150410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:34.150437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341960620375918:8928], now have 1 active actors on pipe
2025-10-09T21:47:34.150448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341960620375686:2432], now have 1 active actors on pipe
2025-10-09T21:47:34.150456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341960258555963:8047]
2025-10-09T21:47:34.150459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341960258555963:8047]
2025-10-09T21:47:34.150462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341960258555963:8047]
2025-10-09T21:47:34.150476Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:34.150493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:34.150509Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:34.150512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:34.150514Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:34.150519Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:34.150522Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:34.150556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341960258555963:8047]
2025-10-09T21:47:34.150571Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:34.150561Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 965
2025-10-09T21:47:34.150603Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:34.150595Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 965
2025-10-09T21:47:34.150752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:34.150764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:34.150802Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|335093af-25243c09-3b17aa2d-70e594cd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:34.150832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:34.150838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:34.150841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:34.150853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:34.150866Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:34.150889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:34.151337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:34.151349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:34.151371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:34.151510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 966 partition: 0 MaxSeqNo: 25 sessionId: src_id|335093af-25243c09-3b17aa2d-70e594cd_0
2025-10-09T21:47:34.151909Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b2a11caf-c94444be-c1236f80-9a496750_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|335093af-25243c09-3b17aa2d-70e594cd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:34.154554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|335093af-25243c09-3b17aa2d-70e594cd_0] Write session: try to update token
2025-10-09T21:47:34.154577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|335093af-25243c09-3b17aa2d-70e594cd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:34.168168Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:34.168189Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:34.168191Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:34.168194Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:34.168204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341960258555963:8047]
2025-10-09T21:47:34.168209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341960258555963:8047]
2025-10-09T21:47:34.168213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341960258555963:8047]
2025-10-09T21:47:34.168258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:34.168258Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:34.168265Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:34.168269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.168269Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:34.168272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:34.168624Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:34.177183Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 966 sessionId: src_id|335093af-25243c09-3b17aa2d-70e594cd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:34.179796Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:34.209995Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341960620375994:8525]
2025-10-09T21:47:34.210374Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 965, pipe: [1:7559341960620375994:8525]
2025-10-09T21:47:34.336838Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:34.337191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:34.337209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:34.337234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341960620375994:8525], now have 1 active actors on pipe
2025-10-09T21:47:34.337486Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:34.337501Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:34.337533Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:34.337573Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341960620375994:8525]
2025-10-09T21:47:34.337580Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 966 reason: error at write: Disconnected sessionId: src_id|335093af-25243c09-3b17aa2d-70e594cd_0
2025-10-09T21:47:34.337600Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:34.337835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 966 sessionId: src_id|335093af-25243c09-3b17aa2d-70e594cd_0 is DEAD
2025-10-09T21:47:34.337918Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|335093af-25243c09-3b17aa2d-70e594cd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:34.337930Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|335093af-25243c09-3b17aa2d-70e594cd_0] Write session will restart in 0.007283s
2025-10-09T21:47:34.337971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|335093af-25243c09-3b17aa2d-70e594cd_0] Write session: Do CDS request
2025-10-09T21:47:34.337978Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|335093af-25243c09-3b17aa2d-70e594cd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:34.346589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|335093af-25243c09-3b17aa2d-70e594cd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:34.346771Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:34.346797Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 967
2025-10-09T21:47:34.347048Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 967 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:34.347102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 967 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:34.347113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=967 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:34.347117Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:34.347986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 967 sessionId:  describe result for acl check
2025-10-09T21:47:34.348028Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:34.348031Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:34.348033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:34.348040Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341960620376020:8933] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:34.348612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341960620376020:8933] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:34.352345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341960620376020:8933] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:34.356078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341960620376020:8933] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:34.356095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341960620376020:8933] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:34.356098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341960620376020:8933] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:34.356108Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 967 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:34.362217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:34.365587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:34.365848Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:34.365862Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:34.365876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:34.365960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:34.366010Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341960258556258:8049]
2025-10-09T21:47:34.366791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:34.367266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:34.367327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:34.367684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:34.367770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:34.367778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:34.367779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:34.367781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:34.367782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:34.367784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:34.367785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.367786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.367787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.367793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:34.367805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:34.367809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:34.367813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:34.367817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:34.367821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:34.367825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.367839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.367843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.367852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:34.367860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:34.367867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:34.367872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:34.367878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:34.367885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:34.367889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:34.367895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:34.367900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:34.367905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:34.367909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:34.367914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:34.367919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.367924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.367929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.367949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.367953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.367959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.367997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:34.375707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:34.377654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:34.377669Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:34.377672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:34.377676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:34.377684Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 966 [2:7559341960258556258:8049]
2025-10-09T21:47:34.377699Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:34.377718Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:34.377722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:34.377725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:34.377728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:34.377746Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:34.377759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:34.377767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:34.377829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.377849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.377852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.377855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:34.377928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:34.377940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.377953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.377965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.377980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:34.378003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341960620375996:2432], now have 1 active actors on pipe
2025-10-09T21:47:34.378012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341960620376180:8933], now have 1 active actors on pipe
2025-10-09T21:47:34.378019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341960258556253:8049]
2025-10-09T21:47:34.378022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341960258556253:8049]
2025-10-09T21:47:34.378024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341960258556253:8049]
2025-10-09T21:47:34.378055Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:34.378069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:34.378072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:34.378081Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:34.378083Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:34.378087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:34.378091Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:34.378134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341960258556253:8049]
2025-10-09T21:47:34.378170Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:34.378083Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 966
2025-10-09T21:47:34.378089Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 966
2025-10-09T21:47:34.378183Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:34.378191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:34.378196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:34.378254Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9f99bff5-a3b396-8d4ba26d-579c3b41_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:34.378263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:34.378267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:34.378270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:34.378281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:34.378292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:34.378326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:34.378623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:34.378639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:34.378662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:34.378797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 967 partition: 0 MaxSeqNo: 25 sessionId: src_id|9f99bff5-a3b396-8d4ba26d-579c3b41_0
2025-10-09T21:47:34.379241Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|335093af-25243c09-3b17aa2d-70e594cd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9f99bff5-a3b396-8d4ba26d-579c3b41_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:34.381881Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f99bff5-a3b396-8d4ba26d-579c3b41_0] Write session: try to update token
2025-10-09T21:47:34.381900Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f99bff5-a3b396-8d4ba26d-579c3b41_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:34.395106Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:34.395126Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:34.395128Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:34.395131Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:34.395141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341960258556253:8049]
2025-10-09T21:47:34.395147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341960258556253:8049]
2025-10-09T21:47:34.395151Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341960258556253:8049]
2025-10-09T21:47:34.395199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:34.395207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.395207Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:34.395210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:34.395213Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:34.395217Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:34.395346Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:34.403244Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 967 sessionId: src_id|9f99bff5-a3b396-8d4ba26d-579c3b41_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:34.405931Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:34.437939Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341960620376296:8525]
2025-10-09T21:47:34.438258Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 966, pipe: [1:7559341960620376296:8525]
2025-10-09T21:47:34.559762Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:34.560099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:34.560110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:34.560135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341960620376296:8525], now have 1 active actors on pipe
2025-10-09T21:47:34.560398Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:34.560494Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341960620376296:8525]
2025-10-09T21:47:34.560529Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:34.560531Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:34.560597Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 967 reason: error at write: Disconnected sessionId: src_id|9f99bff5-a3b396-8d4ba26d-579c3b41_0
2025-10-09T21:47:34.560889Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f99bff5-a3b396-8d4ba26d-579c3b41_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:34.560899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f99bff5-a3b396-8d4ba26d-579c3b41_0] Write session will restart in 0.005447s
2025-10-09T21:47:34.560922Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f99bff5-a3b396-8d4ba26d-579c3b41_0] Write session: Do CDS request
2025-10-09T21:47:34.560927Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f99bff5-a3b396-8d4ba26d-579c3b41_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:34.560776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 967 sessionId: src_id|9f99bff5-a3b396-8d4ba26d-579c3b41_0 is DEAD
2025-10-09T21:47:34.560909Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:34.567576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f99bff5-a3b396-8d4ba26d-579c3b41_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:34.567806Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:34.567824Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 968
2025-10-09T21:47:34.567993Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 968 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:34.568033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 968 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:34.568044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=968 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:34.568049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:34.568577Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 968 sessionId:  describe result for acl check
2025-10-09T21:47:34.568619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:34.568621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:34.568623Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:34.568629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341960620376314:8937] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:34.569193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341960620376314:8937] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:34.574674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341960620376314:8937] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:34.580961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341960620376314:8937] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:34.580981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341960620376314:8937] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:34.580985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341960620376314:8937] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:34.580996Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 968 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:34.595800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:34.599288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:34.599544Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:34.599556Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:34.599570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:34.599635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:34.599697Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341960258556550:8051]
2025-10-09T21:47:34.600338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:34.600800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:34.600869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:34.601210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:34.601280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:34.601281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:34.601283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:34.601284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:34.601286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:34.601287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:34.601288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.601289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.601291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.601297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:34.601323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:34.601327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:34.601331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:34.601335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:34.601340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:34.601343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.601347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.601351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.601359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:34.601376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:34.601383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:34.601389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:34.601394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:34.601399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:34.601403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:34.601409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:34.601414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:34.601419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:34.601424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:34.601430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:34.601436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.601441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.601445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.601450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.601455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.601459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.601489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:34.609761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:34.611270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:34.611287Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:34.611290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:34.611295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:34.611303Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 967 [2:7559341960258556550:8051]
2025-10-09T21:47:34.611320Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:34.611342Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:34.611344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:34.611347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:34.611348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:34.611363Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:34.611368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:34.611374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:34.611448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.611458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.611461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.611464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:34.611548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:34.611556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.611567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.611577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.611595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:34.611616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341960258556545:8051]
2025-10-09T21:47:34.611618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341960258556545:8051]
2025-10-09T21:47:34.611620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341960258556545:8051]
2025-10-09T21:47:34.611638Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:34.611654Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:34.611670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:34.611673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:34.611695Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:34.611698Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:34.611700Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:34.611715Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341960258556545:8051]
2025-10-09T21:47:34.611719Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:34.611790Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:34.611869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341960620376299:2432], now have 1 active actors on pipe
2025-10-09T21:47:34.611904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341960620376424:8937], now have 1 active actors on pipe
2025-10-09T21:47:34.612018Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 967
2025-10-09T21:47:34.612077Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 967
2025-10-09T21:47:34.612245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:34.612258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:34.612294Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|125c21bb-166cd7bc-d1025c2-b2cdc64f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:34.612309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:34.612313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:34.612316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:34.612338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:34.612356Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:34.612385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:34.612645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:34.612657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:34.612686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:34.612839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 968 partition: 0 MaxSeqNo: 25 sessionId: src_id|125c21bb-166cd7bc-d1025c2-b2cdc64f_0
2025-10-09T21:47:34.613233Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f99bff5-a3b396-8d4ba26d-579c3b41_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|125c21bb-166cd7bc-d1025c2-b2cdc64f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:34.615947Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|125c21bb-166cd7bc-d1025c2-b2cdc64f_0] Write session: try to update token
2025-10-09T21:47:34.615972Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|125c21bb-166cd7bc-d1025c2-b2cdc64f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:34.634192Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:34.634209Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:34.634223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341960258556545:8051]
2025-10-09T21:47:34.634272Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:34.634280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:34.634522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:34.634693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:34.634848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:34.635006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:34.635216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:34.635259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:34.641168Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:34.641189Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:34.641192Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:34.641194Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:34.641206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341960258556545:8051]
2025-10-09T21:47:34.641210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341960258556545:8051]
2025-10-09T21:47:34.641215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341960258556545:8051]
2025-10-09T21:47:34.641264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:34.641265Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:34.641272Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:34.641271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.641275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:34.641276Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:34.641430Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:34.644946Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 968 sessionId: src_id|125c21bb-166cd7bc-d1025c2-b2cdc64f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:34.647434Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:34.660580Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341960620376599:8525]
2025-10-09T21:47:34.660941Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 967, pipe: [1:7559341960620376599:8525]
2025-10-09T21:47:34.805254Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:34.806157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:34.806176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:34.806202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341960620376599:8525], now have 1 active actors on pipe
2025-10-09T21:47:34.806770Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:34.806744Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:34.806827Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341960620376599:8525]
2025-10-09T21:47:34.806855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:34.806862Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:34.806924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 968 reason: error at write: Disconnected sessionId: src_id|125c21bb-166cd7bc-d1025c2-b2cdc64f_0
2025-10-09T21:47:34.807122Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 968 sessionId: src_id|125c21bb-166cd7bc-d1025c2-b2cdc64f_0 is DEAD
2025-10-09T21:47:34.807322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|125c21bb-166cd7bc-d1025c2-b2cdc64f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:34.807347Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|125c21bb-166cd7bc-d1025c2-b2cdc64f_0] Write session will restart in 0.007601s
2025-10-09T21:47:34.807380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|125c21bb-166cd7bc-d1025c2-b2cdc64f_0] Write session: Do CDS request
2025-10-09T21:47:34.807402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|125c21bb-166cd7bc-d1025c2-b2cdc64f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:34.816566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|125c21bb-166cd7bc-d1025c2-b2cdc64f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:34.816885Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:34.816918Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 969
2025-10-09T21:47:34.817105Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 969 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:34.817153Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 969 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:34.817165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=969 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:34.817170Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:34.817669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 969 sessionId:  describe result for acl check
2025-10-09T21:47:34.817713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:34.817715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:34.817717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:34.817723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341960620376632:8942] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:34.818377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341960620376632:8942] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:34.825051Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341960620376632:8942] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:34.831378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341960620376632:8942] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:34.831399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341960620376632:8942] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:34.831404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341960620376632:8942] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:34.831415Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 969 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:34.844829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:34.848082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:34.848416Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:34.848431Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:34.848447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:34.848532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:34.848591Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341960258556860:8055]
2025-10-09T21:47:34.849208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:34.849694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:34.849754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:34.850161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:34.850235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:34.850236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:34.850237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:34.850239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:34.850240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:34.850242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:34.850243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.850244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.850245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.850263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:34.850273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:34.850277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:34.850280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:34.850283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:34.850287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:34.850291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.850294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.850298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.850305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:34.850313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:34.850320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:34.850334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:34.850339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:34.850344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:34.850347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:34.850352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:34.850356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:34.850360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:34.850365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:34.850369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:34.850374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.850379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.850384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.850388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.850394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.850399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.850429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:34.861425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:34.863152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:34.863179Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:34.863182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:34.863187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:34.863195Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 968 [2:7559341960258556860:8055]
2025-10-09T21:47:34.863210Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:34.863230Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:34.863233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:34.863236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:34.863239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:34.863259Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:34.863271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:34.863279Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:34.863676Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 968
2025-10-09T21:47:34.863681Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 968
2025-10-09T21:47:34.863347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.863358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.863360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.863363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:34.863443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:34.863448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.863463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:34.863474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:34.863491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:34.863512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341960258556855:8055]
2025-10-09T21:47:34.863515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341960258556855:8055]
2025-10-09T21:47:34.863517Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341960258556855:8055]
2025-10-09T21:47:34.863526Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:34.863541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:34.863556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:34.863559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:34.863560Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:34.863564Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:34.863566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:34.863601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341960258556855:8055]
2025-10-09T21:47:34.863604Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:34.863611Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:34.863618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341960620376764:8942], now have 1 active actors on pipe
2025-10-09T21:47:34.863621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341960620376608:2432], now have 1 active actors on pipe
2025-10-09T21:47:34.863847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:34.863856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:34.863892Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5aa8fb5c-34806a6e-46d3203a-cf85b568_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:34.863912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:34.863918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:34.863922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:34.863934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:34.863947Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:34.863975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:34.864476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:34.864491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:34.864529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:34.864677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 969 partition: 0 MaxSeqNo: 25 sessionId: src_id|5aa8fb5c-34806a6e-46d3203a-cf85b568_0
2025-10-09T21:47:34.865201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|125c21bb-166cd7bc-d1025c2-b2cdc64f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5aa8fb5c-34806a6e-46d3203a-cf85b568_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:34.868721Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5aa8fb5c-34806a6e-46d3203a-cf85b568_0] Write session: try to update token
2025-10-09T21:47:34.868749Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5aa8fb5c-34806a6e-46d3203a-cf85b568_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:34.882363Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:34.882385Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:34.882395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341960258556855:8055]
2025-10-09T21:47:34.882446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:34.882456Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:34.882635Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:34.882792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:34.882941Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:34.883131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:34.883808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:34.883855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:34.891047Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:34.891066Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:34.891069Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:34.891071Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:34.891085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341960258556855:8055]
2025-10-09T21:47:34.891089Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341960258556855:8055]
2025-10-09T21:47:34.891093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341960258556855:8055]
2025-10-09T21:47:34.891147Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:34.891154Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:34.891158Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:34.891164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:34.891171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:34.891174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:34.891320Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:34.893410Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 969 sessionId: src_id|5aa8fb5c-34806a6e-46d3203a-cf85b568_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:34.896145Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:34.907072Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341960620376911:8525]
2025-10-09T21:47:34.907810Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 968, pipe: [1:7559341960620376911:8525]
2025-10-09T21:47:35.053303Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:35.054088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:35.054111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:35.054139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341960620376911:8525], now have 1 active actors on pipe
2025-10-09T21:47:35.054450Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:35.054444Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341960620376911:8525]
2025-10-09T21:47:35.054471Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:35.054473Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:35.054534Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 969 reason: error at write: Disconnected sessionId: src_id|5aa8fb5c-34806a6e-46d3203a-cf85b568_0
2025-10-09T21:47:35.054734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 969 sessionId: src_id|5aa8fb5c-34806a6e-46d3203a-cf85b568_0 is DEAD
2025-10-09T21:47:35.055099Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5aa8fb5c-34806a6e-46d3203a-cf85b568_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:35.055113Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5aa8fb5c-34806a6e-46d3203a-cf85b568_0] Write session will restart in 0.007836s
2025-10-09T21:47:35.055136Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5aa8fb5c-34806a6e-46d3203a-cf85b568_0] Write session: Do CDS request
2025-10-09T21:47:35.055146Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5aa8fb5c-34806a6e-46d3203a-cf85b568_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:35.054429Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:35.064599Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5aa8fb5c-34806a6e-46d3203a-cf85b568_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:35.065100Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:35.065123Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 970
2025-10-09T21:47:35.065272Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 970 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:35.065333Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 970 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:35.065350Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=970 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:35.065357Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:35.065955Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 970 sessionId:  describe result for acl check
2025-10-09T21:47:35.065998Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:35.066005Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:35.066013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:35.066019Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341964915344259:8946] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:35.066582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341964915344259:8946] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:35.069519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341964915344259:8946] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:35.073828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341964915344259:8946] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:35.073861Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341964915344259:8946] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:35.073864Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341964915344259:8946] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:35.073884Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 970 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:35.080619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:35.083838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:35.084079Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:35.084091Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:35.084103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:35.084159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:35.084202Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341964553524466:8067]
2025-10-09T21:47:35.084783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:35.085153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:35.085219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:35.085546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:35.085598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:35.085604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:35.085606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:35.085607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:35.085608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:35.085609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:35.085611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.085612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.085613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.085618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:35.085628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:35.085633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:35.085637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:35.085641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:35.085646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:35.085650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.085654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.085657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.085664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:35.085671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:35.085677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:35.085686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:35.085690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:35.085697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:35.085701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:35.085706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:35.085712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:35.085717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:35.085721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:35.085729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:35.085734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.085744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.085748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.085753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.085761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.085766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.085792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:35.092823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:35.094081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:35.094098Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:35.094101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:35.094106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:35.094114Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 969 [2:7559341964553524466:8067]
2025-10-09T21:47:35.094129Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:35.094149Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:35.094158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:35.094161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:35.094164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:35.094184Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:35.094190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:35.094198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:35.094271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.094282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.094284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.094287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:35.094358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:35.094367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.094381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.094394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.094563Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 969
2025-10-09T21:47:35.094610Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 969
2025-10-09T21:47:35.094411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:35.094441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341964553524461:8067]
2025-10-09T21:47:35.094443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341964553524461:8067]
2025-10-09T21:47:35.094445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341964553524461:8067]
2025-10-09T21:47:35.094464Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:35.094480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341964915344403:8946], now have 1 active actors on pipe
2025-10-09T21:47:35.094484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341964915344212:2432], now have 1 active actors on pipe
2025-10-09T21:47:35.094489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:35.094493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:35.094494Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:35.094496Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:35.094506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:35.094509Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:35.094551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341964553524461:8067]
2025-10-09T21:47:35.094559Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:35.094565Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:35.094613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:35.094618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:35.094643Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bd771d3c-7be74009-966a3365-893cf299_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:35.094651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:35.094655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:35.094658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:35.094671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:35.094682Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:35.094704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:35.094855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:35.094866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:35.094883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:35.094965Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 970 partition: 0 MaxSeqNo: 25 sessionId: src_id|bd771d3c-7be74009-966a3365-893cf299_0
2025-10-09T21:47:35.095383Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5aa8fb5c-34806a6e-46d3203a-cf85b568_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bd771d3c-7be74009-966a3365-893cf299_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:35.098184Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd771d3c-7be74009-966a3365-893cf299_0] Write session: try to update token
2025-10-09T21:47:35.098204Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd771d3c-7be74009-966a3365-893cf299_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:35.107002Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:35.107017Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:35.107028Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341964553524461:8067]
2025-10-09T21:47:35.107080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:35.107075Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:35.107270Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:35.107429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:35.107667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:35.107877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:35.108190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:35.108241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:35.113072Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 970 sessionId: src_id|bd771d3c-7be74009-966a3365-893cf299_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:35.116797Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:35.117597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:35.117622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:35.117682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:35.117806Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:35.120697Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:35.120713Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:35.120715Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:35.120717Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:35.120727Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341964553524461:8067]
2025-10-09T21:47:35.120731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341964553524461:8067]
2025-10-09T21:47:35.120734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341964553524461:8067]
2025-10-09T21:47:35.120779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:35.120781Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:35.120786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.120788Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:35.120790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:35.120792Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:35.120928Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:35.154687Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341964915344539:8525]
2025-10-09T21:47:35.154974Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 969, pipe: [1:7559341964915344539:8525]
2025-10-09T21:47:35.283022Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:35.291324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:35.291355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:35.291531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:35.291604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:35.291664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:35.291734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:35.292215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:35.292222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:35.292304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:35.292367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:35.292426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:35.292486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:35.292526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:35.292530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:35.292598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:35.292681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:35.292746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:35.292840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:35.292885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:35.292888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:35.292957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:35.293022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:35.293128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:35.293230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:35.293272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:35.293275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:35.293344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:35.293403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:35.293466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:35.293554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:35.293613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:35.293616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:35.293686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:35.293744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:35.293801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:35.293880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:35.293922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:35.293925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:35.293991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:35.295768Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341964915344539:8525]
2025-10-09T21:47:35.295862Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:35.295870Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:35.295909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 970 reason: error at write: Disconnected sessionId: src_id|bd771d3c-7be74009-966a3365-893cf299_0
2025-10-09T21:47:35.296097Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:35.296098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 970 sessionId: src_id|bd771d3c-7be74009-966a3365-893cf299_0 is DEAD
2025-10-09T21:47:35.294250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:35.294318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:35.294377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:35.294423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:35.294426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:35.294490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:35.294542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:35.294597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:35.294655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:35.294700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:35.294705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:35.294766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:35.294822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:35.294880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:35.294944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:35.295063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:35.295066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:35.295134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:35.295191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:35.295247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:35.295310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:35.295434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:35.295437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:35.295522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341964915344539:8525], now have 1 active actors on pipe
2025-10-09T21:47:35.296194Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:35.298339Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd771d3c-7be74009-966a3365-893cf299_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:35.298359Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd771d3c-7be74009-966a3365-893cf299_0] Write session will restart in 0.008235s
2025-10-09T21:47:35.298385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd771d3c-7be74009-966a3365-893cf299_0] Write session: Do CDS request
2025-10-09T21:47:35.298391Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd771d3c-7be74009-966a3365-893cf299_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:35.307665Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd771d3c-7be74009-966a3365-893cf299_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:35.307723Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:35.307742Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 971
2025-10-09T21:47:35.307917Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 971 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:35.307975Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 971 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:35.307986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=971 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:35.307991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:35.308556Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 971 sessionId:  describe result for acl check
2025-10-09T21:47:35.308595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:35.308599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:35.308601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:35.308607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341964915344591:8958] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:35.309143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341964915344591:8958] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:35.312908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341964915344591:8958] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:35.317227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341964915344591:8958] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:35.317247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341964915344591:8958] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:35.317251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341964915344591:8958] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:35.317261Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 971 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:35.321774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:35.324610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:35.324839Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:35.324855Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:35.324869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:35.324942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:35.324987Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341964553524769:8071]
2025-10-09T21:47:35.325579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:35.326015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:35.326106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:35.326457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:35.326518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:35.326524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:35.326526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:35.326527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:35.326528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:35.326530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:35.326531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.326532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.326533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.326538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:35.326549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:35.326553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:35.326556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:35.326559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:35.326563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:35.326566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.326570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.326574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.326582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:35.326590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:35.326597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:35.326602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:35.326608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:35.326613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:35.326617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:35.326621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:35.326627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:35.326632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:35.326637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:35.326642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:35.326646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.326652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.326656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.326660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.326665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.326678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.326708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:35.333912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:35.335493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:35.335506Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:35.335509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:35.335513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:35.335521Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 970 [2:7559341964553524769:8071]
2025-10-09T21:47:35.335535Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:35.335554Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:35.335557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:35.335561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:35.335564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:35.335582Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:35.335588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:35.335596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:35.335667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.335676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.335678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.335681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:35.335750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:35.335758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.335773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.335785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.335800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:35.335825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341964915344547:2432], now have 1 active actors on pipe
2025-10-09T21:47:35.335829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341964915344760:8958], now have 1 active actors on pipe
2025-10-09T21:47:35.335837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341964553524764:8071]
2025-10-09T21:47:35.335840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341964553524764:8071]
2025-10-09T21:47:35.335842Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341964553524764:8071]
2025-10-09T21:47:35.335856Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:35.335872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:35.335875Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:35.335879Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:35.335881Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:35.335888Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:35.335891Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:35.335900Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 970
2025-10-09T21:47:35.335905Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 970
2025-10-09T21:47:35.335933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341964553524764:8071]
2025-10-09T21:47:35.335945Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:35.335947Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:35.335972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:35.335984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:35.336025Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bc81f832-5c670f6a-e8952397-c7fda525_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:35.336039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:35.336042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:35.336045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:35.336056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:35.336066Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:35.336088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:35.336338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:35.336351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:35.336374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:35.336536Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 971 partition: 0 MaxSeqNo: 25 sessionId: src_id|bc81f832-5c670f6a-e8952397-c7fda525_0
2025-10-09T21:47:35.336951Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd771d3c-7be74009-966a3365-893cf299_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bc81f832-5c670f6a-e8952397-c7fda525_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:35.339602Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc81f832-5c670f6a-e8952397-c7fda525_0] Write session: try to update token
2025-10-09T21:47:35.339633Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc81f832-5c670f6a-e8952397-c7fda525_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:35.350674Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:35.350700Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:35.350703Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:35.350717Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:35.350727Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341964553524764:8071]
2025-10-09T21:47:35.350731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341964553524764:8071]
2025-10-09T21:47:35.350736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341964553524764:8071]
2025-10-09T21:47:35.350791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:35.350793Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:35.350800Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:35.350804Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:35.350805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.350808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:35.350971Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:35.356639Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 971 sessionId: src_id|bc81f832-5c670f6a-e8952397-c7fda525_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:35.359231Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:35.395851Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341964915344851:8525]
2025-10-09T21:47:35.396204Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 970, pipe: [1:7559341964915344851:8525]
2025-10-09T21:47:35.514222Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:35.514529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:35.514546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:35.514595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341964915344851:8525], now have 1 active actors on pipe
2025-10-09T21:47:35.514830Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:35.514938Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:35.514942Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341964915344851:8525]
2025-10-09T21:47:35.514957Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:35.514965Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:35.515019Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 971 reason: error at write: Disconnected sessionId: src_id|bc81f832-5c670f6a-e8952397-c7fda525_0
2025-10-09T21:47:35.515217Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 971 sessionId: src_id|bc81f832-5c670f6a-e8952397-c7fda525_0 is DEAD
2025-10-09T21:47:35.515397Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc81f832-5c670f6a-e8952397-c7fda525_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:35.515406Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc81f832-5c670f6a-e8952397-c7fda525_0] Write session will restart in 0.005242s
2025-10-09T21:47:35.515434Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc81f832-5c670f6a-e8952397-c7fda525_0] Write session: Do CDS request
2025-10-09T21:47:35.515440Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc81f832-5c670f6a-e8952397-c7fda525_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:35.521576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc81f832-5c670f6a-e8952397-c7fda525_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:35.521765Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:35.521783Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 972
2025-10-09T21:47:35.521966Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 972 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:35.522007Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 972 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:35.522013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=972 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:35.522017Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:35.522536Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 972 sessionId:  describe result for acl check
2025-10-09T21:47:35.522582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:35.522589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:35.522593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:35.522599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341964915344862:8962] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:35.523154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341964915344862:8962] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:35.525999Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341964915344862:8962] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:35.531471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341964915344862:8962] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:35.531488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341964915344862:8962] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:35.531492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341964915344862:8962] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:35.531501Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 972 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:35.551166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:35.554660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:35.554918Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:35.554929Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:35.554942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:35.554997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:35.555039Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341964553525064:8073]
2025-10-09T21:47:35.555673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:35.556103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:35.556180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:35.556544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:35.556600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:35.556607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:35.556609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:35.556611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:35.556612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:35.556613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:35.556614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.556616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.556617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.556622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:35.556634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:35.556639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:35.556643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:35.556647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:35.556650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:35.556654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.556658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.556662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.556671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:35.556680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:35.556686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:35.556692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:35.556701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:35.556706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:35.556710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:35.556719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:35.556724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:35.556734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:35.556739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:35.556744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:35.556749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.556755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.556760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.556765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.556774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.556780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.556812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:35.565579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:35.567510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:35.567530Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:35.567533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:35.567538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:35.567547Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 971 [2:7559341964553525064:8073]
2025-10-09T21:47:35.567565Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:35.567587Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:35.567592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:35.567595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:35.567597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:35.567619Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:35.567633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:35.567643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:35.567721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.567730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.567731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.567734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:35.567812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:35.567821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.567835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.567850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.567869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:35.567897Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341964553525059:8073]
2025-10-09T21:47:35.567900Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341964553525059:8073]
2025-10-09T21:47:35.567902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341964553525059:8073]
2025-10-09T21:47:35.567918Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:35.567940Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:35.567954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:35.567957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:35.568003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341964553525059:8073]
2025-10-09T21:47:35.568010Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:35.568509Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:35.568520Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:35.568522Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:35.568529Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:35.569107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 971
2025-10-09T21:47:35.569157Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 971
2025-10-09T21:47:35.570997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341964915344914:8962], now have 1 active actors on pipe
2025-10-09T21:47:35.571016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341964915344853:2432], now have 1 active actors on pipe
2025-10-09T21:47:35.571038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:35.571047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:35.571092Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|10d7394b-7b39c4cd-73185746-c558a8bd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:35.571107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:35.571113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:35.571117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:35.571133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:35.571152Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:35.571176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:35.573009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:35.573025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:35.573070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:35.573632Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 972 partition: 0 MaxSeqNo: 25 sessionId: src_id|10d7394b-7b39c4cd-73185746-c558a8bd_0
2025-10-09T21:47:35.574073Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc81f832-5c670f6a-e8952397-c7fda525_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|10d7394b-7b39c4cd-73185746-c558a8bd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:35.577056Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10d7394b-7b39c4cd-73185746-c558a8bd_0] Write session: try to update token
2025-10-09T21:47:35.577078Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10d7394b-7b39c4cd-73185746-c558a8bd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:35.589153Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:35.589169Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:35.589172Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:35.589174Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:35.589185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341964553525059:8073]
2025-10-09T21:47:35.589189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341964553525059:8073]
2025-10-09T21:47:35.589193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341964553525059:8073]
2025-10-09T21:47:35.589238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:35.589241Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:35.589246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.589247Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:35.589249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:35.589254Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:35.589381Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:35.603402Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 972 sessionId: src_id|10d7394b-7b39c4cd-73185746-c558a8bd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:35.606005Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:35.614290Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341964915345158:8525]
2025-10-09T21:47:35.614894Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 971, pipe: [1:7559341964915345158:8525]
2025-10-09T21:47:35.748444Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:35.748980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:35.748991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:35.749018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341964915345158:8525], now have 1 active actors on pipe
2025-10-09T21:47:35.749269Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:35.749311Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341964915345158:8525]
2025-10-09T21:47:35.749354Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:35.749452Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 972 reason: status is not ok: tablet will be restarted right now sessionId: src_id|10d7394b-7b39c4cd-73185746-c558a8bd_0
2025-10-09T21:47:35.749636Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:35.749640Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 972 sessionId: src_id|10d7394b-7b39c4cd-73185746-c558a8bd_0 is DEAD
2025-10-09T21:47:35.749787Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10d7394b-7b39c4cd-73185746-c558a8bd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:35.749798Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10d7394b-7b39c4cd-73185746-c558a8bd_0] Write session will restart in 0.005061s
2025-10-09T21:47:35.749826Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10d7394b-7b39c4cd-73185746-c558a8bd_0] Write session: Do CDS request
2025-10-09T21:47:35.749841Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10d7394b-7b39c4cd-73185746-c558a8bd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:35.756560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10d7394b-7b39c4cd-73185746-c558a8bd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:35.756726Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:35.756753Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 973
2025-10-09T21:47:35.757030Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 973 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:35.757092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 973 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:35.757104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=973 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:35.757108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:35.757585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 973 sessionId:  describe result for acl check
2025-10-09T21:47:35.757627Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:35.757634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:35.757636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:35.757642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341964915345176:8966] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:35.758254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341964915345176:8966] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:35.761706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341964915345176:8966] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:35.771143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341964915345176:8966] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:35.771163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341964915345176:8966] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:35.771167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341964915345176:8966] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:35.771184Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 973 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:35.777647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:35.780456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:35.780692Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:35.780711Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:35.780727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:35.780835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:35.780910Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341964553525365:8077]
2025-10-09T21:47:35.781687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:35.782050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:35.782105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:35.782440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:35.782486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:35.782487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:35.782488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:35.782489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:35.782490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:35.782491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:35.782492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.782493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.782494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.782500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:35.782511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:35.782515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:35.782518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:35.782521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:35.782524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:35.782528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.782531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.782535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.782542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:35.782549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:35.782554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:35.782559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:35.782563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:35.782568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:35.782571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:35.782575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:35.782579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:35.782583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:35.782587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:35.782591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:35.782595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.782598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.782603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.782607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.782611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.782616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.782641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:35.789774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:35.790939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:35.790945Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:35.790948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:35.790953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:35.790961Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 972 [2:7559341964553525365:8077]
2025-10-09T21:47:35.790980Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:35.791000Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:35.791003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:35.791007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:35.791009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:35.791031Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:35.791038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:35.791046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:35.791109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.791112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.791115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.791119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:35.791197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:35.791199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.791213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:35.791223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:35.791242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:35.791263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341964553525360:8077]
2025-10-09T21:47:35.791266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341964553525360:8077]
2025-10-09T21:47:35.791269Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341964553525360:8077]
2025-10-09T21:47:35.791286Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:35.791302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:35.791320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:35.791324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:35.791351Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:35.791356Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:35.791358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:35.791365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341964553525360:8077]
2025-10-09T21:47:35.791369Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:35.791455Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:35.791614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341964915345360:8966], now have 1 active actors on pipe
2025-10-09T21:47:35.791625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341964915345161:2432], now have 1 active actors on pipe
2025-10-09T21:47:35.791778Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 972
2025-10-09T21:47:35.791809Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 972
2025-10-09T21:47:35.791998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:35.792011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:35.792044Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7186bdc7-8c552bcf-4f8c0204-f1906677_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:35.792061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:35.792065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:35.792068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:35.792080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:35.792092Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:35.792111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:35.792402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:35.792411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:35.792442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:35.792551Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 973 partition: 0 MaxSeqNo: 25 sessionId: src_id|7186bdc7-8c552bcf-4f8c0204-f1906677_0
2025-10-09T21:47:35.792939Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10d7394b-7b39c4cd-73185746-c558a8bd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7186bdc7-8c552bcf-4f8c0204-f1906677_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:35.795503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7186bdc7-8c552bcf-4f8c0204-f1906677_0] Write session: try to update token
2025-10-09T21:47:35.795520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7186bdc7-8c552bcf-4f8c0204-f1906677_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:35.808398Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:35.808424Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:35.808429Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:35.808432Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:35.808444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341964553525360:8077]
2025-10-09T21:47:35.808452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341964553525360:8077]
2025-10-09T21:47:35.808458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341964553525360:8077]
2025-10-09T21:47:35.808513Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:35.808517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:35.808519Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:35.808523Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:35.808527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:35.808531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:35.808751Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:35.814355Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 973 sessionId: src_id|7186bdc7-8c552bcf-4f8c0204-f1906677_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:35.817317Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:35.850371Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341964915345471:8525]
2025-10-09T21:47:35.854291Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 972, pipe: [1:7559341964915345471:8525]
2025-10-09T21:47:35.964450Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:35.964984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:35.964999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:35.965044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341964915345471:8525], now have 1 active actors on pipe
2025-10-09T21:47:35.965325Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341964915345471:8525]
2025-10-09T21:47:35.965348Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:35.965353Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:35.965353Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:35.965425Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 973 reason: error at write: Disconnected sessionId: src_id|7186bdc7-8c552bcf-4f8c0204-f1906677_0
2025-10-09T21:47:35.965632Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 973 sessionId: src_id|7186bdc7-8c552bcf-4f8c0204-f1906677_0 is DEAD
2025-10-09T21:47:35.965791Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7186bdc7-8c552bcf-4f8c0204-f1906677_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:35.965803Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7186bdc7-8c552bcf-4f8c0204-f1906677_0] Write session will restart in 0.008604s
2025-10-09T21:47:35.965826Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7186bdc7-8c552bcf-4f8c0204-f1906677_0] Write session: Do CDS request
2025-10-09T21:47:35.965850Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7186bdc7-8c552bcf-4f8c0204-f1906677_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:35.967215Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:35.975695Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7186bdc7-8c552bcf-4f8c0204-f1906677_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:35.976164Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:35.976183Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 974
2025-10-09T21:47:35.976321Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 974 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:35.976367Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 974 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:35.976377Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=974 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:35.976382Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:35.976898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 974 sessionId:  describe result for acl check
2025-10-09T21:47:35.976939Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:35.976949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:35.976951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:35.976957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341964915345488:8971] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:35.977501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341964915345488:8971] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:35.979580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341964915345488:8971] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:35.982715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341964915345488:8971] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:35.982731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341964915345488:8971] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:35.982735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341964915345488:8971] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:35.982743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 974 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:36.002350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:36.005615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:36.005904Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:36.005914Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:36.005928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:36.005998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:36.006060Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341968848492988:8089]
2025-10-09T21:47:36.006689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:36.007137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:36.007202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:36.007556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:36.007618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:36.007619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:36.007620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:36.007622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:36.007623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:36.007624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:36.007625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.007626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.007627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.007633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:36.007644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:36.007648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:36.007652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:36.007656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:36.007660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:36.007664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.007667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.007671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.007680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:36.007688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:36.007694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:36.007700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:36.007706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:36.007711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:36.007715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:36.007719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:36.007724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:36.007728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:36.007747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:36.007753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:36.007758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.007764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.007768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.007775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.007781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.007786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.007821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:36.019316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:36.020984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:36.021000Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:36.021003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:36.021007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:36.021015Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 973 [2:7559341968848492988:8089]
2025-10-09T21:47:36.021044Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:36.021063Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:36.021073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:36.021076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:36.021079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:36.021096Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:36.021103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:36.021110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:36.021175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.021184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.021185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.021188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:36.021264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:36.021274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.021297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.021308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.021323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:36.021349Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341968848492983:8089]
2025-10-09T21:47:36.021356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341968848492983:8089]
2025-10-09T21:47:36.021358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341968848492983:8089]
2025-10-09T21:47:36.021370Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:36.021456Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 973
2025-10-09T21:47:36.021462Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 973
2025-10-09T21:47:36.022409Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 974 partition: 0 MaxSeqNo: 25 sessionId: src_id|db3ca00e-ea05e328-a66d300c-560fdea9_0
2025-10-09T21:47:36.022780Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7186bdc7-8c552bcf-4f8c0204-f1906677_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|db3ca00e-ea05e328-a66d300c-560fdea9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:36.021386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341964915345522:8971], now have 1 active actors on pipe
2025-10-09T21:47:36.021395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341964915345476:2432], now have 1 active actors on pipe
2025-10-09T21:47:36.021395Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:36.021397Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:36.021399Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:36.021404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:36.021419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:36.021422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:36.021466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341968848492983:8089]
2025-10-09T21:47:36.021473Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:36.021479Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:36.021822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:36.021830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:36.021875Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|db3ca00e-ea05e328-a66d300c-560fdea9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:36.021884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:36.021888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:36.021891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:36.021902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:36.021914Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:36.021932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:36.022250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:36.022255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:36.022276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:36.025358Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db3ca00e-ea05e328-a66d300c-560fdea9_0] Write session: try to update token
2025-10-09T21:47:36.025374Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db3ca00e-ea05e328-a66d300c-560fdea9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:36.042375Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:36.042394Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:36.042396Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:36.042399Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:36.042408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341968848492983:8089]
2025-10-09T21:47:36.042413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341968848492983:8089]
2025-10-09T21:47:36.042417Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341968848492983:8089]
2025-10-09T21:47:36.042470Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:36.042471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:36.042480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.042481Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:36.042483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:36.042485Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:36.042670Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:36.050968Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 974 sessionId: src_id|db3ca00e-ea05e328-a66d300c-560fdea9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:36.053518Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:36.062450Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341969210313076:8525]
2025-10-09T21:47:36.062908Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 973, pipe: [1:7559341969210313076:8525]
2025-10-09T21:47:36.215790Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:36.216345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:36.216356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:36.216405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341969210313076:8525], now have 1 active actors on pipe
2025-10-09T21:47:36.216736Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:36.217310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db3ca00e-ea05e328-a66d300c-560fdea9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:36.217321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db3ca00e-ea05e328-a66d300c-560fdea9_0] Write session will restart in 0.005777s
2025-10-09T21:47:36.216909Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:36.216924Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:36.217344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db3ca00e-ea05e328-a66d300c-560fdea9_0] Write session: Do CDS request
2025-10-09T21:47:36.216989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 974 reason: error at write: Disconnected sessionId: src_id|db3ca00e-ea05e328-a66d300c-560fdea9_0
2025-10-09T21:47:36.217350Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db3ca00e-ea05e328-a66d300c-560fdea9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:36.217165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 974 sessionId: src_id|db3ca00e-ea05e328-a66d300c-560fdea9_0 is DEAD
2025-10-09T21:47:36.217224Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341969210313076:8525]
2025-10-09T21:47:36.217244Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:36.224628Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db3ca00e-ea05e328-a66d300c-560fdea9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:36.224773Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:36.224791Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 975
2025-10-09T21:47:36.224955Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 975 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:36.225007Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 975 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:36.225022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=975 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:36.225027Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:36.225863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 975 sessionId:  describe result for acl check
2025-10-09T21:47:36.225909Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:36.225912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:36.225914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:36.225920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341969210313127:8983] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:36.226533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341969210313127:8983] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:36.229157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341969210313127:8983] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:36.236117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341969210313127:8983] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:36.236134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341969210313127:8983] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:36.236138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341969210313127:8983] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:36.236153Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 975 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:36.249104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:36.251698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:36.251904Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:36.251913Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:36.251924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:36.251975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:36.252013Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341968848493294:8093]
2025-10-09T21:47:36.252630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:36.252991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:36.253050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:36.253431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:36.253480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:36.253487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:36.253488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:36.253489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:36.253491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:36.253492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:36.253493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.253494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.253495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.253501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:36.253513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:36.253517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:36.253521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:36.253525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:36.253530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:36.253534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.253538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.253542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.253550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:36.253559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:36.253570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:36.253576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:36.253585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:36.253590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:36.253599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:36.253605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:36.253610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:36.253619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:36.253624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:36.253634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:36.253639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.253648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.253653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.253658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.253663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.253669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.253698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:36.260892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:36.262647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:36.262666Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:36.262668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:36.262673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:36.262680Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 974 [2:7559341968848493294:8093]
2025-10-09T21:47:36.262695Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:36.262717Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:36.262720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:36.262724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:36.262737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:36.262755Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:36.262769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:36.262777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:36.262854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.262866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.262869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.262872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:36.262942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:36.262950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.262965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.262983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.263007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:36.263033Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341968848493289:8093]
2025-10-09T21:47:36.263041Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341968848493289:8093]
2025-10-09T21:47:36.263044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341968848493289:8093]
2025-10-09T21:47:36.263053Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:36.263063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:36.263066Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:36.263068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:36.263068Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:36.263154Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 974
2025-10-09T21:47:36.263085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:36.263091Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:36.263103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341969210313197:8983], now have 1 active actors on pipe
2025-10-09T21:47:36.263107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341969210313112:2432], now have 1 active actors on pipe
2025-10-09T21:47:36.263144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341968848493289:8093]
2025-10-09T21:47:36.263153Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:36.263196Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:36.263339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:36.263351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:36.263379Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c02c8e0c-1cb06293-afd38f54-39357cdf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:36.263394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:36.263403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:36.263406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:36.263395Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 974
2025-10-09T21:47:36.263417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:36.263432Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:36.263447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:36.263983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:36.263996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:36.264019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:36.264178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 975 partition: 0 MaxSeqNo: 25 sessionId: src_id|c02c8e0c-1cb06293-afd38f54-39357cdf_0
2025-10-09T21:47:36.264618Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db3ca00e-ea05e328-a66d300c-560fdea9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c02c8e0c-1cb06293-afd38f54-39357cdf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:36.267309Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c02c8e0c-1cb06293-afd38f54-39357cdf_0] Write session: try to update token
2025-10-09T21:47:36.267325Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c02c8e0c-1cb06293-afd38f54-39357cdf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:36.280262Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:36.280284Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:36.280287Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:36.280289Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:36.280300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341968848493289:8093]
2025-10-09T21:47:36.280305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341968848493289:8093]
2025-10-09T21:47:36.280308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341968848493289:8093]
2025-10-09T21:47:36.280357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:36.280369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.280371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:36.280527Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:36.282088Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:36.282096Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:36.282101Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:36.285228Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 975 sessionId: src_id|c02c8e0c-1cb06293-afd38f54-39357cdf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:36.288172Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:36.317445Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341969210313426:8525]
2025-10-09T21:47:36.317828Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 974, pipe: [1:7559341969210313426:8525]
2025-10-09T21:47:36.458173Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:36.458708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:36.458721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:36.458772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341969210313426:8525], now have 1 active actors on pipe
2025-10-09T21:47:36.459026Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:36.459622Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341969210313426:8525]
2025-10-09T21:47:36.459656Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:36.459677Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:36.459749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 975 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c02c8e0c-1cb06293-afd38f54-39357cdf_0
2025-10-09T21:47:36.459940Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 975 sessionId: src_id|c02c8e0c-1cb06293-afd38f54-39357cdf_0 is DEAD
2025-10-09T21:47:36.460098Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c02c8e0c-1cb06293-afd38f54-39357cdf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:36.460112Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c02c8e0c-1cb06293-afd38f54-39357cdf_0] Write session will restart in 0.008884s
2025-10-09T21:47:36.460139Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c02c8e0c-1cb06293-afd38f54-39357cdf_0] Write session: Do CDS request
2025-10-09T21:47:36.460146Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c02c8e0c-1cb06293-afd38f54-39357cdf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:36.470589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c02c8e0c-1cb06293-afd38f54-39357cdf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:36.470776Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:36.470807Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 976
2025-10-09T21:47:36.470960Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 976 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:36.471000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 976 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:36.471006Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=976 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:36.471013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:36.472097Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 976 sessionId:  describe result for acl check
2025-10-09T21:47:36.472139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:36.472141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:36.472142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:36.472149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341969210313456:8987] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:36.472738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341969210313456:8987] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:36.477663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341969210313456:8987] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:36.484515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341969210313456:8987] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:36.484533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341969210313456:8987] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:36.484537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341969210313456:8987] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:36.484547Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 976 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:36.493209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:36.496732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:36.497020Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:36.497033Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:36.497047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:36.497116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:36.497162Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341968848493595:8095]
2025-10-09T21:47:36.497753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:36.498685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:36.498750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:36.499080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:36.499130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:36.499136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:36.499138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:36.499139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:36.499140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:36.499141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:36.499142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.499143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.499144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.499150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:36.499160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:36.499163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:36.499167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:36.499170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:36.499174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:36.499177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.499180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.499183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.499190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:36.499199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:36.499204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:36.499209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:36.499213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:36.499217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:36.499221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:36.499226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:36.499230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:36.499235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:36.499239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:36.499243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:36.499247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.499251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.499256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.499262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.499268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.499273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.499298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:36.506519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:36.507766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:36.507783Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:36.507785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:36.507790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:36.507800Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 975 [2:7559341968848493595:8095]
2025-10-09T21:47:36.507818Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:36.507839Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:36.507842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:36.507845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:36.507847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:36.507867Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:36.507879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:36.507890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:36.507957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.507963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.507965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.507967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:36.508053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:36.508059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.508071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.508080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.508094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:36.508115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341968848493590:8095]
2025-10-09T21:47:36.508117Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341968848493590:8095]
2025-10-09T21:47:36.508119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341968848493590:8095]
2025-10-09T21:47:36.508128Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:36.508143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:36.508156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:36.508159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:36.508161Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:36.508164Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:36.508167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:36.508202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341968848493590:8095]
2025-10-09T21:47:36.508205Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:36.508212Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:36.508217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341969210313429:2432], now have 1 active actors on pipe
2025-10-09T21:47:36.508220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341969210313635:8987], now have 1 active actors on pipe
2025-10-09T21:47:36.509499Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 975
2025-10-09T21:47:36.509544Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 975
2025-10-09T21:47:36.509973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:36.509986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:36.510016Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9f1ae08c-7a185279-e4bc5e27-5501d8db_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:36.510043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:36.510051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:36.510055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:36.510068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:36.510090Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:36.510119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:36.510458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:36.510468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:36.510492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:36.510601Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 976 partition: 0 MaxSeqNo: 25 sessionId: src_id|9f1ae08c-7a185279-e4bc5e27-5501d8db_0
2025-10-09T21:47:36.511054Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c02c8e0c-1cb06293-afd38f54-39357cdf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9f1ae08c-7a185279-e4bc5e27-5501d8db_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:36.513657Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f1ae08c-7a185279-e4bc5e27-5501d8db_0] Write session: try to update token
2025-10-09T21:47:36.513700Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f1ae08c-7a185279-e4bc5e27-5501d8db_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:36.523469Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:36.523487Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:36.523499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341968848493590:8095]
2025-10-09T21:47:36.523548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:36.523657Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:36.523719Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:36.523856Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:36.524063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:36.524252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:36.524547Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:36.524586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:36.538773Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:36.539209Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:36.539222Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:36.539225Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:36.539235Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341968848493590:8095]
2025-10-09T21:47:36.539239Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341968848493590:8095]
2025-10-09T21:47:36.539242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341968848493590:8095]
2025-10-09T21:47:36.539288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:36.539292Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:36.539297Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:36.539301Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:36.539302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.539305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:36.539440Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:36.539490Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 976 sessionId: src_id|9f1ae08c-7a185279-e4bc5e27-5501d8db_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:36.541895Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:36.559622Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341969210313737:8525]
2025-10-09T21:47:36.560106Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 975, pipe: [1:7559341969210313737:8525]
2025-10-09T21:47:36.729098Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:36.729670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:36.729689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:36.729715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341969210313737:8525], now have 1 active actors on pipe
2025-10-09T21:47:36.729988Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:36.730103Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341969210313737:8525]
2025-10-09T21:47:36.730141Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:36.730175Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:36.730183Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:36.730258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 976 reason: error at write: Disconnected sessionId: src_id|9f1ae08c-7a185279-e4bc5e27-5501d8db_0
2025-10-09T21:47:36.730452Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 976 sessionId: src_id|9f1ae08c-7a185279-e4bc5e27-5501d8db_0 is DEAD
2025-10-09T21:47:36.730643Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f1ae08c-7a185279-e4bc5e27-5501d8db_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:36.730655Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f1ae08c-7a185279-e4bc5e27-5501d8db_0] Write session will restart in 0.007420s
2025-10-09T21:47:36.730684Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f1ae08c-7a185279-e4bc5e27-5501d8db_0] Write session: Do CDS request
2025-10-09T21:47:36.730690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f1ae08c-7a185279-e4bc5e27-5501d8db_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:36.740105Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f1ae08c-7a185279-e4bc5e27-5501d8db_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:36.748459Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:36.748478Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 977
2025-10-09T21:47:36.748739Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 977 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:36.748794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 977 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:36.748806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=977 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:36.748811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:36.749680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 977 sessionId:  describe result for acl check
2025-10-09T21:47:36.749725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:36.749727Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:36.749729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:36.749735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341969210313778:8993] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:36.750333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341969210313778:8993] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:36.755436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341969210313778:8993] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:36.763827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341969210313778:8993] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:36.763848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341969210313778:8993] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:36.763853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341969210313778:8993] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:36.763863Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 977 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:36.770816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:36.774229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:36.774784Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:36.774801Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:36.774830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:36.774918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:36.774966Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341968848493899:8099]
2025-10-09T21:47:36.775562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:36.776016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:36.776080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:36.776407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:36.776452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:36.776459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:36.776460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:36.776462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:36.776463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:36.776464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:36.776465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.776466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.776467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.776481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:36.776491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:36.776495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:36.776498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:36.776501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:36.776505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:36.776508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.776511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.776514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.776522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:36.776529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:36.776535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:36.776540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:36.776544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:36.776549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:36.776555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:36.776559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:36.776564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:36.776568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:36.776572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:36.776577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:36.776582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.776587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.776591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.776595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.776600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.776604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.776631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:36.789034Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 976
2025-10-09T21:47:36.789080Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 976
2025-10-09T21:47:36.787390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:36.788586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:36.788594Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:36.788598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:36.788603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:36.788612Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 976 [2:7559341968848493899:8099]
2025-10-09T21:47:36.788634Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:36.788657Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:36.788661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:36.788665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:36.788668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:36.788689Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:36.788697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:36.788705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:36.788781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.788784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.788786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.788789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:36.789194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:36.789198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.789215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:36.789228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:36.789244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:36.789272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341969210313743:2432], now have 1 active actors on pipe
2025-10-09T21:47:36.789276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341969210313961:8993], now have 1 active actors on pipe
2025-10-09T21:47:36.789284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341968848493894:8099]
2025-10-09T21:47:36.789286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341968848493894:8099]
2025-10-09T21:47:36.789288Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341968848493894:8099]
2025-10-09T21:47:36.789297Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:36.789307Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:36.789312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:36.789314Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:36.789315Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:36.789330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:36.789333Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:36.789357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:36.789362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:36.789414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341968848493894:8099]
2025-10-09T21:47:36.789428Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:36.789437Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:36.789449Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8b2e55e7-b8b45545-1378e7af-5360c05f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:36.789457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:36.789460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:36.789463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:36.789474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:36.789486Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:36.789546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:36.790016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:36.790047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:36.790070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:36.790324Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 977 partition: 0 MaxSeqNo: 25 sessionId: src_id|8b2e55e7-b8b45545-1378e7af-5360c05f_0
2025-10-09T21:47:36.790710Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f1ae08c-7a185279-e4bc5e27-5501d8db_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8b2e55e7-b8b45545-1378e7af-5360c05f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:36.793228Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2e55e7-b8b45545-1378e7af-5360c05f_0] Write session: try to update token
2025-10-09T21:47:36.793247Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2e55e7-b8b45545-1378e7af-5360c05f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:36.811781Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:36.811809Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:36.811812Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:36.811814Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:36.811825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341968848493894:8099]
2025-10-09T21:47:36.811830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341968848493894:8099]
2025-10-09T21:47:36.811835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341968848493894:8099]
2025-10-09T21:47:36.811886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:36.811895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:36.811898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:36.811928Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:36.811944Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:36.811948Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:36.812059Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:36.816165Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 977 sessionId: src_id|8b2e55e7-b8b45545-1378e7af-5360c05f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:36.818900Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:36.830464Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341969210314055:8525]
2025-10-09T21:47:36.830822Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 976, pipe: [1:7559341969210314055:8525]
2025-10-09T21:47:36.982383Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:36.983311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:36.983332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:36.983380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341969210314055:8525], now have 1 active actors on pipe
2025-10-09T21:47:36.983936Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341969210314055:8525]
2025-10-09T21:47:36.983973Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:36.983976Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:36.983986Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:36.984034Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 977 reason: error at write: Disconnected sessionId: src_id|8b2e55e7-b8b45545-1378e7af-5360c05f_0
2025-10-09T21:47:36.984553Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2e55e7-b8b45545-1378e7af-5360c05f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:36.984564Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2e55e7-b8b45545-1378e7af-5360c05f_0] Write session will restart in 0.009369s
2025-10-09T21:47:36.984588Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2e55e7-b8b45545-1378e7af-5360c05f_0] Write session: Do CDS request
2025-10-09T21:47:36.984594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2e55e7-b8b45545-1378e7af-5360c05f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:36.984234Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 977 sessionId: src_id|8b2e55e7-b8b45545-1378e7af-5360c05f_0 is DEAD
2025-10-09T21:47:36.986141Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:36.996331Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2e55e7-b8b45545-1378e7af-5360c05f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:36.998216Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:36.998237Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 978
2025-10-09T21:47:36.998446Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 978 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:36.998483Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 978 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:36.998489Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=978 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:36.998494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:37.000166Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 978 sessionId:  describe result for acl check
2025-10-09T21:47:37.000208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:37.000210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:37.000211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:37.000217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341969210314080:8998] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:37.000779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341969210314080:8998] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:37.006071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341969210314080:8998] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:37.013893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341969210314080:8998] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:37.013917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341969210314080:8998] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:37.013922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341969210314080:8998] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:37.013933Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 978 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:37.026324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:37.029600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:37.029919Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:37.029931Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:37.029944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:37.030011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:37.030074Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341973143461527:8107]
2025-10-09T21:47:37.030721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:37.031097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:37.031148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:37.031556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:37.031613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.031615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.031616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.031617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.031619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.031620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.031622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.031623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.031624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.031631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.031643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.031648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.031652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.031657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.031661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.031665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.031669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.031673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.031683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.031691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.031698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.031703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.031708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.031714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.031718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.031723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.031729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.031734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.031738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.031744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.031749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.031754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.031759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.031764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.031771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.031777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.031808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:37.039094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:37.040688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:37.040708Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:37.040710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:37.040716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:37.040724Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 977 [2:7559341973143461527:8107]
2025-10-09T21:47:37.040741Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:37.040762Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:37.040766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:37.040770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:37.040773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:37.040792Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:37.040798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:37.040807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:37.040883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.040891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.040893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.040897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:37.040990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:37.041001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.041015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.041027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.041052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:37.041076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341969210314061:2432], now have 1 active actors on pipe
2025-10-09T21:47:37.041089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341973505281496:8998], now have 1 active actors on pipe
2025-10-09T21:47:37.041099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341973143461522:8107]
2025-10-09T21:47:37.041101Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341973143461522:8107]
2025-10-09T21:47:37.041104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341973143461522:8107]
2025-10-09T21:47:37.041123Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:37.041141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:37.041158Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:37.041162Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:37.041194Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:37.041199Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:37.041203Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:37.041209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341973143461522:8107]
2025-10-09T21:47:37.041213Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:37.041215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:37.041417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:37.041432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:37.041467Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|58f8b16a-be5133f4-46f8f3b4-a826d064_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:37.041483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:37.041493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:37.041496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:37.041510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:37.041535Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:37.041198Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 977
2025-10-09T21:47:37.041558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:37.041253Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 977
2025-10-09T21:47:37.041998Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 978 partition: 0 MaxSeqNo: 25 sessionId: src_id|58f8b16a-be5133f4-46f8f3b4-a826d064_0
2025-10-09T21:47:37.042361Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2e55e7-b8b45545-1378e7af-5360c05f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|58f8b16a-be5133f4-46f8f3b4-a826d064_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:37.041754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:37.041764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:37.041785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:37.045037Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|58f8b16a-be5133f4-46f8f3b4-a826d064_0] Write session: try to update token
2025-10-09T21:47:37.045058Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|58f8b16a-be5133f4-46f8f3b4-a826d064_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:37.055304Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:37.055324Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:37.055336Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341973143461522:8107]
2025-10-09T21:47:37.055389Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:37.055425Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:37.055627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:37.055779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:37.055932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:37.056080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:37.056278Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:37.056327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:37.065680Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 978 sessionId: src_id|58f8b16a-be5133f4-46f8f3b4-a826d064_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:37.068691Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:37.068929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:37.068951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:37.069004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:37.069115Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:37.070426Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:37.070443Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:37.070446Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:37.070448Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:37.070458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341973143461522:8107]
2025-10-09T21:47:37.070463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341973143461522:8107]
2025-10-09T21:47:37.070466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341973143461522:8107]
2025-10-09T21:47:37.070510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:37.070517Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:37.070521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.070523Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:37.070524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:37.070526Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:37.070673Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:37.083928Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341973505281665:8525]
2025-10-09T21:47:37.086383Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 977, pipe: [1:7559341973505281665:8525]
2025-10-09T21:47:37.235570Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:37.235937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341973505281665:8525], now have 1 active actors on pipe
2025-10-09T21:47:37.243970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:37.244002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:37.244184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:37.244248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:37.244312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:37.244381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:37.244657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:37.244672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:37.244739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:37.244800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:37.244866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:37.244934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:37.244984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:37.244993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:37.245058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:37.245114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:37.245172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:37.245248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:37.245295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:37.245304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:37.245370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:37.245434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:37.245509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:37.245575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:37.245623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:37.245627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:37.245692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:37.245746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:37.245852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:37.245915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:37.246140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:37.246151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:37.246216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:37.246276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:37.246348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:37.246411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:37.246484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:37.246494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:37.246561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:37.246623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:37.246709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:37.246778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:37.246841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:37.246850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:37.246911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:37.246968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:37.247025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:37.247093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:37.247138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:37.247147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:37.247212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:37.248798Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:37.248855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:37.248899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 978 reason: status is not ok: tablet will be restarted right now sessionId: src_id|58f8b16a-be5133f4-46f8f3b4-a826d064_0
2025-10-09T21:47:37.247287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:37.247365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:37.247423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:37.247467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:37.247470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:37.247540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:37.247597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:37.247656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:37.247721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:37.247858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:37.247861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:37.248658Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:37.249212Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|58f8b16a-be5133f4-46f8f3b4-a826d064_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:37.249225Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|58f8b16a-be5133f4-46f8f3b4-a826d064_0] Write session will restart in 0.005771s
2025-10-09T21:47:37.249250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|58f8b16a-be5133f4-46f8f3b4-a826d064_0] Write session: Do CDS request
2025-10-09T21:47:37.249257Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|58f8b16a-be5133f4-46f8f3b4-a826d064_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:37.249095Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 978 sessionId: src_id|58f8b16a-be5133f4-46f8f3b4-a826d064_0 is DEAD
2025-10-09T21:47:37.249180Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341973505281665:8525]
2025-10-09T21:47:37.256624Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|58f8b16a-be5133f4-46f8f3b4-a826d064_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:37.256718Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:37.256737Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 979
2025-10-09T21:47:37.256978Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 979 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:37.257030Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 979 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:37.257036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=979 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:37.257041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:37.257761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 979 sessionId:  describe result for acl check
2025-10-09T21:47:37.257796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:37.257803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:37.257805Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:37.257811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341973505281716:9010] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:37.258402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341973505281716:9010] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:37.261345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341973505281716:9010] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:37.266330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341973505281716:9010] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:37.266348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341973505281716:9010] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:37.266352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341973505281716:9010] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:37.266361Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 979 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:37.281017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:37.284189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:37.284416Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:37.284428Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:37.284442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:37.284513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:37.284555Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341973143461838:8115]
2025-10-09T21:47:37.285153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:37.285493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:37.285555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:37.285918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:37.285980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.285986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.285987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.285988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.285990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.285991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.285993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.285994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.285995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.286000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.286011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.286015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.286020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.286024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.286043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.286047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.286051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.286055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.286063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.286072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.286079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.286085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.286091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.286096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.286101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.286106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.286112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.286117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.286122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.286127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.286132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.286137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.286142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.286147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.286152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.286157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.286190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:37.293218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:37.294674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:37.294689Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:37.294692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:37.294696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:37.294704Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 978 [2:7559341973143461838:8115]
2025-10-09T21:47:37.294719Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:37.294740Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:37.294743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:37.294746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:37.294749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:37.294768Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:37.294779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:37.294787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:37.294856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.294865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.294867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.294870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:37.294944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:37.294952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.294965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.294975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.294992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:37.295020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341973143461833:8115]
2025-10-09T21:47:37.295023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341973143461833:8115]
2025-10-09T21:47:37.295025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341973143461833:8115]
2025-10-09T21:47:37.295034Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:37.295047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:37.295050Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:37.295051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:37.295055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341973505281795:9010], now have 1 active actors on pipe
2025-10-09T21:47:37.295061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:37.295078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:37.295081Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:37.295087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341973505281701:2432], now have 1 active actors on pipe
2025-10-09T21:47:37.295128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341973143461833:8115]
2025-10-09T21:47:37.295145Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:37.295172Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:37.295123Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 978
2025-10-09T21:47:37.295141Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 978
2025-10-09T21:47:37.295208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:37.295217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:37.295246Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|da18671-22b305ec-37dbbb18-befc2921_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:37.295258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:37.295263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:37.295265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:37.295276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:37.295287Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:37.295309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:37.295678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:37.295693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:37.295735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:37.295867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 979 partition: 0 MaxSeqNo: 25 sessionId: src_id|da18671-22b305ec-37dbbb18-befc2921_0
2025-10-09T21:47:37.296288Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|58f8b16a-be5133f4-46f8f3b4-a826d064_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|da18671-22b305ec-37dbbb18-befc2921_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:37.298909Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da18671-22b305ec-37dbbb18-befc2921_0] Write session: try to update token
2025-10-09T21:47:37.298930Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da18671-22b305ec-37dbbb18-befc2921_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:37.308198Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:37.308216Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:37.308226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341973143461833:8115]
2025-10-09T21:47:37.308273Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:37.308276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:37.308467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:37.308605Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:37.308741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:37.308886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:37.309089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:37.309133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:37.321893Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:37.321912Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:37.321915Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:37.321917Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:37.321929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341973143461833:8115]
2025-10-09T21:47:37.321934Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341973143461833:8115]
2025-10-09T21:47:37.321938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341973143461833:8115]
2025-10-09T21:47:37.321991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:37.322004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.322007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:37.322184Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:37.324514Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 979 sessionId: src_id|da18671-22b305ec-37dbbb18-befc2921_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:37.326085Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:37.326106Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:37.326111Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:37.327020Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:37.349616Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341973505282009:8525]
2025-10-09T21:47:37.350142Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 978, pipe: [1:7559341973505282009:8525]
2025-10-09T21:47:37.483660Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:37.483946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:37.483960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:37.483984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341973505282009:8525], now have 1 active actors on pipe
2025-10-09T21:47:37.484238Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:37.484265Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:37.484284Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:37.484364Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 979 reason: error at write: Disconnected sessionId: src_id|da18671-22b305ec-37dbbb18-befc2921_0
2025-10-09T21:47:37.484412Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341973505282009:8525]
2025-10-09T21:47:37.484435Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:37.484692Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da18671-22b305ec-37dbbb18-befc2921_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:37.484703Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da18671-22b305ec-37dbbb18-befc2921_0] Write session will restart in 0.007647s
2025-10-09T21:47:37.484727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da18671-22b305ec-37dbbb18-befc2921_0] Write session: Do CDS request
2025-10-09T21:47:37.484737Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da18671-22b305ec-37dbbb18-befc2921_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:37.484595Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 979 sessionId: src_id|da18671-22b305ec-37dbbb18-befc2921_0 is DEAD
2025-10-09T21:47:37.493607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da18671-22b305ec-37dbbb18-befc2921_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:37.493747Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:37.493766Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 980
2025-10-09T21:47:37.493934Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 980 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:37.493967Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 980 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:37.493982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=980 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:37.493985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:37.494759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 980 sessionId:  describe result for acl check
2025-10-09T21:47:37.494805Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:37.494816Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:37.494819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:37.494825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341973505282038:9014] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:37.495439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341973505282038:9014] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:37.498819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341973505282038:9014] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:37.503042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341973505282038:9014] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:37.503059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341973505282038:9014] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:37.503063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341973505282038:9014] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:37.503071Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 980 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:37.509053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:37.511975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:37.512194Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:37.512205Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:37.512217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:37.512266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:37.512311Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341973143462141:8117]
2025-10-09T21:47:37.512851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:37.513183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:37.513217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:37.513496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:37.513537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.513538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.513539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.513540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.513541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.513542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.513543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.513544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.513545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.513549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.513558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.513561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.513564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.513567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.513570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.513573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.513576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.513579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.513585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.513592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.513597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.513602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.513605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.513610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.513613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.513617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.513621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.513624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.513628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.513632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.513637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.513641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.513645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.513649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.513653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.513656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.513680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:37.521042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:37.522188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:37.522206Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:37.522214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:37.522224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:37.522239Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 979 [2:7559341973143462141:8117]
2025-10-09T21:47:37.522267Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:37.522304Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:37.522312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:37.522320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:37.522326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:37.522349Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:37.522355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:37.522363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:37.522438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.522451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.522453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.522455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:37.522527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:37.522529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.522540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.522549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.522562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:37.522582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341973143462136:8117]
2025-10-09T21:47:37.522584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341973143462136:8117]
2025-10-09T21:47:37.522586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341973143462136:8117]
2025-10-09T21:47:37.522604Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:37.522619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:37.522632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:37.522635Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:37.522637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:37.522642Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:37.522644Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:37.522674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341973143462136:8117]
2025-10-09T21:47:37.522683Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:37.522692Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:37.522710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341973505282213:9014], now have 1 active actors on pipe
2025-10-09T21:47:37.522713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341973505282012:2432], now have 1 active actors on pipe
2025-10-09T21:47:37.522806Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 979
2025-10-09T21:47:37.522955Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 979
2025-10-09T21:47:37.523104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:37.523120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:37.523178Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7ce9ecb1-24eacbbe-a2451fca-12bfdd3e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:37.523195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:37.523201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:37.523204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:37.523215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:37.523235Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:37.523257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:37.523436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:37.523445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:37.523460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:37.523575Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 980 partition: 0 MaxSeqNo: 25 sessionId: src_id|7ce9ecb1-24eacbbe-a2451fca-12bfdd3e_0
2025-10-09T21:47:37.524047Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da18671-22b305ec-37dbbb18-befc2921_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7ce9ecb1-24eacbbe-a2451fca-12bfdd3e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:37.526907Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ce9ecb1-24eacbbe-a2451fca-12bfdd3e_0] Write session: try to update token
2025-10-09T21:47:37.526929Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ce9ecb1-24eacbbe-a2451fca-12bfdd3e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:37.539534Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:37.539552Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:37.539554Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:37.539556Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:37.539566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341973143462136:8117]
2025-10-09T21:47:37.539569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341973143462136:8117]
2025-10-09T21:47:37.539573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341973143462136:8117]
2025-10-09T21:47:37.539616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:37.539624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.539626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:37.539624Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:37.539632Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:37.539646Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:37.539755Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:37.544278Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 980 sessionId: src_id|7ce9ecb1-24eacbbe-a2451fca-12bfdd3e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:37.547403Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:37.584661Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341973505282325:8525]
2025-10-09T21:47:37.585035Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 979, pipe: [1:7559341973505282325:8525]
2025-10-09T21:47:37.685063Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:37.685727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:37.685744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:37.685789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341973505282325:8525], now have 1 active actors on pipe
2025-10-09T21:47:37.685998Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:37.686068Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:37.686074Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341973505282325:8525]
2025-10-09T21:47:37.686101Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:37.686176Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 980 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7ce9ecb1-24eacbbe-a2451fca-12bfdd3e_0
2025-10-09T21:47:37.686359Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 980 sessionId: src_id|7ce9ecb1-24eacbbe-a2451fca-12bfdd3e_0 is DEAD
2025-10-09T21:47:37.686520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ce9ecb1-24eacbbe-a2451fca-12bfdd3e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:37.686545Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ce9ecb1-24eacbbe-a2451fca-12bfdd3e_0] Write session will restart in 0.005788s
2025-10-09T21:47:37.686572Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ce9ecb1-24eacbbe-a2451fca-12bfdd3e_0] Write session: Do CDS request
2025-10-09T21:47:37.686579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ce9ecb1-24eacbbe-a2451fca-12bfdd3e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:37.693564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ce9ecb1-24eacbbe-a2451fca-12bfdd3e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:37.693655Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:37.693682Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 981
2025-10-09T21:47:37.693819Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 981 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:37.693869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 981 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:37.693881Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=981 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:37.693885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:37.694375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 981 sessionId:  describe result for acl check
2025-10-09T21:47:37.694421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:37.694431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:37.694433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:37.694440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341973505282342:9018] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:37.695015Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341973505282342:9018] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:37.697520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341973505282342:9018] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:37.701083Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341973505282342:9018] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:37.701101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341973505282342:9018] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:37.701104Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341973505282342:9018] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:37.701114Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 981 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:37.707878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:37.711199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:37.711907Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:37.711925Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:37.711955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:37.712032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:37.712080Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341973143462458:8121]
2025-10-09T21:47:37.712693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:37.713083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:37.713146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:37.713485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:37.713547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.713553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.713554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.713556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.713557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.713559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.713560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.713561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.713562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.713568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.713580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.713585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.713588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.713592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.713595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.713599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.713602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.713605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.713613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.713621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.713627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.713633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.713638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.713643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.713647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.713652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.713656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.713661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.713665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.713670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.713676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.713688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.713693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.713698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.713702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.713708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.713738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:37.720691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:37.722447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:37.722466Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:37.722470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:37.722475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:37.722483Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 980 [2:7559341973143462458:8121]
2025-10-09T21:47:37.722502Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:37.722525Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:37.722543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:37.722548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:37.722551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:37.722571Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:37.722577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:37.722585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:37.722679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.722688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.722690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.722694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:37.722785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:37.722794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.722809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.722826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.722849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:37.722875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341973505282473:9018], now have 1 active actors on pipe
2025-10-09T21:47:37.722884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341973505282327:2432], now have 1 active actors on pipe
2025-10-09T21:47:37.722893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341973143462444:8121]
2025-10-09T21:47:37.722899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341973143462444:8121]
2025-10-09T21:47:37.722902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341973143462444:8121]
2025-10-09T21:47:37.722918Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:37.722931Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:37.722937Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:37.722940Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:37.722942Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:37.722954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:37.722957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:37.722953Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 980
2025-10-09T21:47:37.722956Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 980
2025-10-09T21:47:37.723001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341973143462444:8121]
2025-10-09T21:47:37.723017Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:37.723024Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:37.723038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:37.723049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:37.723109Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5814966a-408dc2bd-57eb41ec-9e2f53b9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:37.723124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:37.723129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:37.723132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:37.723144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:37.723160Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:37.723192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:37.723381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:37.723391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:37.723408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:37.723503Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 981 partition: 0 MaxSeqNo: 25 sessionId: src_id|5814966a-408dc2bd-57eb41ec-9e2f53b9_0
2025-10-09T21:47:37.723930Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ce9ecb1-24eacbbe-a2451fca-12bfdd3e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5814966a-408dc2bd-57eb41ec-9e2f53b9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:37.726463Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5814966a-408dc2bd-57eb41ec-9e2f53b9_0] Write session: try to update token
2025-10-09T21:47:37.726483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5814966a-408dc2bd-57eb41ec-9e2f53b9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:37.738064Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:37.738084Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:37.738086Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:37.738089Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:37.738099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341973143462444:8121]
2025-10-09T21:47:37.738104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341973143462444:8121]
2025-10-09T21:47:37.738109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341973143462444:8121]
2025-10-09T21:47:37.738154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:37.738157Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:37.738162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.738164Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:37.738164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:37.738168Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:37.738302Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:37.740590Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 981 sessionId: src_id|5814966a-408dc2bd-57eb41ec-9e2f53b9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:37.743219Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:37.786383Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341973505282644:8525]
2025-10-09T21:47:37.786695Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 980, pipe: [1:7559341973505282644:8525]
2025-10-09T21:47:37.899780Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:37.900392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:37.900416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:37.900471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341973505282644:8525], now have 1 active actors on pipe
2025-10-09T21:47:37.900732Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:37.900749Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341973505282644:8525]
2025-10-09T21:47:37.900748Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:37.900753Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:37.900792Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:37.900833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 981 reason: error at write: Disconnected sessionId: src_id|5814966a-408dc2bd-57eb41ec-9e2f53b9_0
2025-10-09T21:47:37.901019Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 981 sessionId: src_id|5814966a-408dc2bd-57eb41ec-9e2f53b9_0 is DEAD
2025-10-09T21:47:37.901192Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5814966a-408dc2bd-57eb41ec-9e2f53b9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:37.901201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5814966a-408dc2bd-57eb41ec-9e2f53b9_0] Write session will restart in 0.006207s
2025-10-09T21:47:37.901228Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5814966a-408dc2bd-57eb41ec-9e2f53b9_0] Write session: Do CDS request
2025-10-09T21:47:37.901235Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5814966a-408dc2bd-57eb41ec-9e2f53b9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:37.908659Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5814966a-408dc2bd-57eb41ec-9e2f53b9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:37.908732Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:37.908753Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 982
2025-10-09T21:47:37.908928Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 982 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:37.908978Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 982 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:37.909000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=982 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:37.909006Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:37.909636Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 982 sessionId:  describe result for acl check
2025-10-09T21:47:37.909682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:37.909684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:37.909686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:37.909692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341973505282665:9023] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:37.910403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341973505282665:9023] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:37.911770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:37.913102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341973505282665:9023] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:37.913442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:37.913684Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:37.913698Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:37.913713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:37.913770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:37.913812Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341973143462515:8123]
2025-10-09T21:47:37.914456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:37.914860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:37.914931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:37.915328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:37.915396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.915406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.915407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.915408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.915410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.915411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.915413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.915414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.915415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.915422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.915434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.915438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.915442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.915446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.915450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.915454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.915457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.915460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.915468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.915476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:37.915485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.915491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:37.915496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.915500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:37.915504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.915510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:37.915516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.915523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:37.915528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.915534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:37.915541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.915546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.915550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.915555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.915560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.915566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.915596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:37.916699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341973505282665:9023] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:37.916723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341973505282665:9023] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:37.916727Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341973505282665:9023] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:37.916740Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 982 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:37.923182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:37.924605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:37.924624Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:37.924627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:37.924632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:37.924640Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 981 [2:7559341973143462515:8123]
2025-10-09T21:47:37.924659Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:37.924681Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:37.924687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:37.924690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:37.924693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:37.924713Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:37.924720Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:37.924727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:37.924816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.924829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.924831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.924833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:37.924912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:37.924921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.924938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:37.924948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:37.924962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:37.924990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341973143462509:8123]
2025-10-09T21:47:37.924994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341973143462509:8123]
2025-10-09T21:47:37.924995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341973143462509:8123]
2025-10-09T21:47:37.925004Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:37.925015Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:37.925019Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:37.925021Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:37.925025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341973505282648:2432], now have 1 active actors on pipe
2025-10-09T21:47:37.925029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341973505282708:9023], now have 1 active actors on pipe
2025-10-09T21:47:37.925034Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:37.925048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:37.925051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:37.925099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341973143462509:8123]
2025-10-09T21:47:37.925115Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:37.925142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:37.925202Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 981
2025-10-09T21:47:37.925282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:37.925299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:37.925269Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 981
2025-10-09T21:47:37.925336Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ad7a38f9-42f57113-7113f9d3-9e9eb06f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:37.925354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:37.925359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:37.925362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:37.925375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:37.925898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 982 partition: 0 MaxSeqNo: 25 sessionId: src_id|ad7a38f9-42f57113-7113f9d3-9e9eb06f_0
2025-10-09T21:47:37.925389Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:37.925434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:37.925674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:37.925678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:37.925707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:37.926341Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5814966a-408dc2bd-57eb41ec-9e2f53b9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ad7a38f9-42f57113-7113f9d3-9e9eb06f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:37.929088Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad7a38f9-42f57113-7113f9d3-9e9eb06f_0] Write session: try to update token
2025-10-09T21:47:37.929114Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad7a38f9-42f57113-7113f9d3-9e9eb06f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:37.939786Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:37.939810Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:37.939822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341973143462509:8123]
2025-10-09T21:47:37.939881Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:37.939894Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:37.940159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:37.940356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:37.940518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:37.940717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:37.940971Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:37.941032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:37.946509Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 982 sessionId: src_id|ad7a38f9-42f57113-7113f9d3-9e9eb06f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:37.949383Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:37.949729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:37.949752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:37.949802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:37.949953Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:37.954626Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:37.954649Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:37.954651Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:37.954653Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:37.954667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341973143462509:8123]
2025-10-09T21:47:37.954671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341973143462509:8123]
2025-10-09T21:47:37.954674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341973143462509:8123]
2025-10-09T21:47:37.954722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:37.954734Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:37.954740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:37.954741Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:37.954743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:37.954745Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:37.954890Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:38.000926Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341977800250011:8525]
2025-10-09T21:47:38.001207Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 981, pipe: [1:7559341977800250011:8525]
2025-10-09T21:47:38.120725Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:38.128681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:38.128712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:38.128888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.128954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.129032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.129115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.129169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.129181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.129254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.129323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.129391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.129459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.129513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.129521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.129590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.129651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.129719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.129842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.129890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.129900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.129972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.130101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.130167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.130233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.130281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.130288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.130357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.130420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.130481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.130552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.130601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.130608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.130671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.130735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.130818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.130883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.130993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.131002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.131068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.131154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.131219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.131281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.131329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.131337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.131400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.131459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.131527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.131592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.131652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.131660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.131721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.131781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.131846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.131908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.131956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.131961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.132025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.132084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.132158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.132223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.132282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.132294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.132370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341977800250011:8525], now have 1 active actors on pipe
2025-10-09T21:47:38.132670Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341977800250011:8525]
2025-10-09T21:47:38.132678Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:38.132683Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:38.132709Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:38.132725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 982 reason: error at write: Disconnected sessionId: src_id|ad7a38f9-42f57113-7113f9d3-9e9eb06f_0
2025-10-09T21:47:38.132969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 982 sessionId: src_id|ad7a38f9-42f57113-7113f9d3-9e9eb06f_0 is DEAD
2025-10-09T21:47:38.133080Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad7a38f9-42f57113-7113f9d3-9e9eb06f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:38.133091Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad7a38f9-42f57113-7113f9d3-9e9eb06f_0] Write session will restart in 0.007291s
2025-10-09T21:47:38.133116Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad7a38f9-42f57113-7113f9d3-9e9eb06f_0] Write session: Do CDS request
2025-10-09T21:47:38.133123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad7a38f9-42f57113-7113f9d3-9e9eb06f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:38.133484Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:38.141596Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad7a38f9-42f57113-7113f9d3-9e9eb06f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:38.141730Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:38.141751Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 983
2025-10-09T21:47:38.141891Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 983 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:38.141925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 983 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:38.141935Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=983 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:38.141939Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:38.142422Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 983 sessionId:  describe result for acl check
2025-10-09T21:47:38.142468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:38.142470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:38.142472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:38.142479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341977800250036:9027] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:38.143072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341977800250036:9027] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:38.144897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:38.145901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341977800250036:9027] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:38.146558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:38.146820Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:38.146833Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:38.146848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:38.146919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:38.146965Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341977438429899:8135]
2025-10-09T21:47:38.147558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:38.148133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:38.148198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:38.148548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:38.148605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:38.148606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:38.148608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:38.148609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:38.148610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:38.148615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:38.148616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.148617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.148618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.148624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:38.148635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:38.148640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:38.148644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:38.148647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:38.148652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:38.148656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.148661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.148665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.148672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:38.148681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:38.148688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:38.148693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:38.148697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:38.148702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:38.148707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:38.148712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:38.148718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:38.148723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:38.148727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:38.148732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:38.148737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.148742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.148747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.148752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.148757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.148763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.148793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:38.149276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341977800250036:9027] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:38.149291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341977800250036:9027] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:38.149296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341977800250036:9027] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:38.149306Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 983 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:38.155793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:38.156967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:38.156990Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:38.156993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:38.156998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:38.157007Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 982 [2:7559341977438429899:8135]
2025-10-09T21:47:38.157026Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:38.157047Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:38.157061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:38.157065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:38.157068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:38.157094Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:38.157100Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:38.157108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:38.157183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.157190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.157192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.157195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:38.157275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:38.157285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.157299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.157310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.157325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:38.157352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341977438429894:8135]
2025-10-09T21:47:38.157362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341977438429894:8135]
2025-10-09T21:47:38.157364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341977438429894:8135]
2025-10-09T21:47:38.157372Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:38.157379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:38.157382Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:38.157384Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:38.157388Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:38.157402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:38.157406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:38.157418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341977800250021:2432], now have 1 active actors on pipe
2025-10-09T21:47:38.157426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341977800250077:9027], now have 1 active actors on pipe
2025-10-09T21:47:38.157468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341977438429894:8135]
2025-10-09T21:47:38.157479Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:38.157485Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:38.157509Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 982
2025-10-09T21:47:38.157515Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 982
2025-10-09T21:47:38.157661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:38.157675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:38.157708Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|53ed3e0c-4c8db658-ad7d8346-30eec371_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:38.157725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:38.157730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:38.157734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:38.157746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:38.157759Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:38.157778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:38.158108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:38.158123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:38.158158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:38.158384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 983 partition: 0 MaxSeqNo: 25 sessionId: src_id|53ed3e0c-4c8db658-ad7d8346-30eec371_0
2025-10-09T21:47:38.158832Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad7a38f9-42f57113-7113f9d3-9e9eb06f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|53ed3e0c-4c8db658-ad7d8346-30eec371_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:38.161536Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53ed3e0c-4c8db658-ad7d8346-30eec371_0] Write session: try to update token
2025-10-09T21:47:38.161561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53ed3e0c-4c8db658-ad7d8346-30eec371_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:38.172702Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:38.172722Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:38.172725Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:38.172727Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:38.172738Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341977438429894:8135]
2025-10-09T21:47:38.172743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341977438429894:8135]
2025-10-09T21:47:38.172750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341977438429894:8135]
2025-10-09T21:47:38.172797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:38.172804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.172809Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:38.172812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:38.172816Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:38.172820Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:38.172947Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:38.176755Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 983 sessionId: src_id|53ed3e0c-4c8db658-ad7d8346-30eec371_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:38.179552Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:38.234115Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341977800250094:8525]
2025-10-09T21:47:38.234867Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 982, pipe: [1:7559341977800250094:8525]
2025-10-09T21:47:38.335053Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:38.335393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:38.335413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:38.335462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341977800250094:8525], now have 1 active actors on pipe
2025-10-09T21:47:38.335700Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:38.335733Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:38.335772Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341977800250094:8525]
2025-10-09T21:47:38.335799Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:38.335890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 983 reason: status is not ok: tablet will be restarted right now sessionId: src_id|53ed3e0c-4c8db658-ad7d8346-30eec371_0
2025-10-09T21:47:38.336088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 983 sessionId: src_id|53ed3e0c-4c8db658-ad7d8346-30eec371_0 is DEAD
2025-10-09T21:47:38.336269Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53ed3e0c-4c8db658-ad7d8346-30eec371_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:38.336280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53ed3e0c-4c8db658-ad7d8346-30eec371_0] Write session will restart in 0.009788s
2025-10-09T21:47:38.336308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53ed3e0c-4c8db658-ad7d8346-30eec371_0] Write session: Do CDS request
2025-10-09T21:47:38.336315Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53ed3e0c-4c8db658-ad7d8346-30eec371_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:38.345036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:38.346882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:38.347104Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:38.347115Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:38.347129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:38.347184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:38.347248Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341977438429964:8139]
2025-10-09T21:47:38.347604Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53ed3e0c-4c8db658-ad7d8346-30eec371_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:38.347705Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:38.347724Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 984
2025-10-09T21:47:38.347870Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 984 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:38.347914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 984 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:38.347887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:38.347928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=984 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:38.347932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:38.348234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:38.348295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:38.348364Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 984 sessionId:  describe result for acl check
2025-10-09T21:47:38.348407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:38.348415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:38.348417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:38.348423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341977800250141:9039] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:38.348684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:38.348766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:38.348776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:38.348778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:38.348779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:38.348780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:38.348781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:38.348782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.348783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.348784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.348790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:38.348803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:38.348809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:38.348813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:38.348817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:38.348821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:38.348825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.348829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.348833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.348839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:38.348848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:38.348854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:38.348860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:38.348863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:38.348869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:38.348875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:38.348880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:38.348884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:38.348888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:38.348892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:38.348898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:38.348908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.348914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.348920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.348925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.348934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.348938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.348965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:38.349014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341977800250141:9039] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:38.351335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341977800250141:9039] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:38.356379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:38.356222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341977800250141:9039] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:38.356244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341977800250141:9039] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:38.356248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341977800250141:9039] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:38.356257Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 984 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:38.357548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:38.357563Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:38.357568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:38.357573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:38.357580Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 983 [2:7559341977438429964:8139]
2025-10-09T21:47:38.357599Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:38.357619Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:38.357621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:38.357625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:38.357627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:38.357645Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:38.357651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:38.357659Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:38.357725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.357735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.357737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.357740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:38.357815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:38.357838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.357854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.357864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.357880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:38.357903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341977800250174:9039], now have 1 active actors on pipe
2025-10-09T21:47:38.357912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341977800250117:2432], now have 1 active actors on pipe
2025-10-09T21:47:38.357920Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341977438429959:8139]
2025-10-09T21:47:38.357922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341977438429959:8139]
2025-10-09T21:47:38.357925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341977438429959:8139]
2025-10-09T21:47:38.357938Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:38.357954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:38.357960Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:38.357969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:38.357971Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:38.357974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:38.357974Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:38.358014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341977438429959:8139]
2025-10-09T21:47:38.358048Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 983
2025-10-09T21:47:38.358082Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 983
2025-10-09T21:47:38.358026Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:38.358077Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:38.358215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:38.358229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:38.358282Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a2ef9d9e-4616ad24-4af018e-4a07b033_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:38.358300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:38.358310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:38.358313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:38.358325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:38.358337Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:38.358362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:38.358614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:38.358630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:38.358653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:38.358799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 984 partition: 0 MaxSeqNo: 25 sessionId: src_id|a2ef9d9e-4616ad24-4af018e-4a07b033_0
2025-10-09T21:47:38.359215Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53ed3e0c-4c8db658-ad7d8346-30eec371_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a2ef9d9e-4616ad24-4af018e-4a07b033_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:38.361824Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a2ef9d9e-4616ad24-4af018e-4a07b033_0] Write session: try to update token
2025-10-09T21:47:38.361861Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a2ef9d9e-4616ad24-4af018e-4a07b033_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:38.373078Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:38.373099Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:38.373110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341977438429959:8139]
2025-10-09T21:47:38.373166Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:38.373167Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:38.373341Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:38.373492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:38.373644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:38.373785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:38.373992Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:38.374053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:38.377027Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 984 sessionId: src_id|a2ef9d9e-4616ad24-4af018e-4a07b033_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:38.379721Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:38.379969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:38.380002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:38.380044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:38.380248Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:38.386763Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:38.386785Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:38.386789Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:38.386791Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:38.386807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341977438429959:8139]
2025-10-09T21:47:38.386811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341977438429959:8139]
2025-10-09T21:47:38.386815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341977438429959:8139]
2025-10-09T21:47:38.386864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:38.386867Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:38.386872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.386874Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:38.386875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:38.386878Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:38.387067Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:38.436123Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341977800250182:8525]
2025-10-09T21:47:38.436613Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 983, pipe: [1:7559341977800250182:8525]
2025-10-09T21:47:38.571645Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:38.580783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:38.580824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:38.581024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.581089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.581171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.581247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.581299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.581306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.581392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.581456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.581537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.581621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.581676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.581680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.581758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.581821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.581909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.581978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.582410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.582432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.582508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.582601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.582684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.582758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.582808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.582811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.582891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.582963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.583030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.583105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.583148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.583151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.583233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.583305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.583372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.583446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.583493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.583496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.583576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.583651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.583812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.583968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.584011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.584014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.584087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.584162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.584244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.584314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.584401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.584404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.584487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.584558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.584632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.584710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.584755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.584758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.584843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:38.584903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:38.584981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:38.585056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:38.585099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:38.585103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:38.585193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341977800250182:8525], now have 1 active actors on pipe
2025-10-09T21:47:38.585939Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341977800250182:8525]
2025-10-09T21:47:38.585991Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:38.585995Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:38.586024Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 984 reason: error at write: Disconnected sessionId: src_id|a2ef9d9e-4616ad24-4af018e-4a07b033_0
2025-10-09T21:47:38.586232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 984 sessionId: src_id|a2ef9d9e-4616ad24-4af018e-4a07b033_0 is DEAD
2025-10-09T21:47:38.586382Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:38.586754Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2ef9d9e-4616ad24-4af018e-4a07b033_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:38.586767Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2ef9d9e-4616ad24-4af018e-4a07b033_0] Write session will restart in 0.005720s
2025-10-09T21:47:38.586793Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2ef9d9e-4616ad24-4af018e-4a07b033_0] Write session: Do CDS request
2025-10-09T21:47:38.586800Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2ef9d9e-4616ad24-4af018e-4a07b033_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:38.586817Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:38.594554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a2ef9d9e-4616ad24-4af018e-4a07b033_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:38.594685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:38.594712Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 985
2025-10-09T21:47:38.594827Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 985 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:38.594859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 985 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:38.594874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=985 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:38.594877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:38.595339Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 985 sessionId:  describe result for acl check
2025-10-09T21:47:38.595383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:38.595401Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:38.595403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:38.595410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341977800250198:9043] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:38.595964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341977800250198:9043] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:38.599594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341977800250198:9043] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:38.598960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:38.600767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:38.600980Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:38.600984Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:38.600999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:38.601062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:38.601102Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341977438430025:8141]
2025-10-09T21:47:38.601753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:38.602320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:38.602384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:38.602785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:38.602842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:38.602850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:38.602851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:38.602853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:38.602854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:38.602855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:38.602856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.602860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.602861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.602868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:38.602878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:38.602882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:38.602886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:38.602890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:38.602894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:38.602898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.602902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.602905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.602928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:38.602940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:38.602949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:38.602954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:38.602964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:38.602969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:38.602976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:38.602981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:38.602988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:38.602993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:38.602997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:38.603015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:38.603020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.603026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.603030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.603036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.603045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.603050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.603081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:38.602927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341977800250198:9043] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:38.602945Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341977800250198:9043] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:38.602949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341977800250198:9043] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:38.602962Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 985 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:38.610391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:38.611514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:38.611529Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:38.611532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:38.611536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:38.611543Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 984 [2:7559341977438430025:8141]
2025-10-09T21:47:38.611559Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:38.611579Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:38.611583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:38.611586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:38.611588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:38.611606Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:38.611619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:38.611626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:38.611685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.611694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.611696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.611698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:38.611764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:38.611767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.611778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.611790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.611805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:38.611824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341977438430019:8141]
2025-10-09T21:47:38.611826Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341977438430019:8141]
2025-10-09T21:47:38.611829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341977438430019:8141]
2025-10-09T21:47:38.611842Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:38.611857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:38.611869Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:38.611872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:38.611913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341977438430019:8141]
2025-10-09T21:47:38.611914Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:38.611935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341977800250185:2432], now have 1 active actors on pipe
2025-10-09T21:47:38.611937Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:38.611940Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:38.611943Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:38.611944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341977800250244:9043], now have 1 active actors on pipe
2025-10-09T21:47:38.611949Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:38.612031Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 984
2025-10-09T21:47:38.612079Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 984
2025-10-09T21:47:38.612208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:38.612232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:38.612259Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4384c829-e1724355-206e5819-83a34e2f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:38.612267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:38.612270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:38.612273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:38.612283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:38.612301Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:38.612326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:38.613609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:38.613626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:38.613652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:38.613781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 985 partition: 0 MaxSeqNo: 25 sessionId: src_id|4384c829-e1724355-206e5819-83a34e2f_0
2025-10-09T21:47:38.614209Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2ef9d9e-4616ad24-4af018e-4a07b033_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4384c829-e1724355-206e5819-83a34e2f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:38.616845Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4384c829-e1724355-206e5819-83a34e2f_0] Write session: try to update token
2025-10-09T21:47:38.616869Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4384c829-e1724355-206e5819-83a34e2f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:38.627791Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:38.627816Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:38.627819Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:38.627821Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:38.627838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341977438430019:8141]
2025-10-09T21:47:38.627842Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341977438430019:8141]
2025-10-09T21:47:38.627849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341977438430019:8141]
2025-10-09T21:47:38.627900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:38.627913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.627916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:38.628084Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:38.630172Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:38.630190Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:38.630195Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:38.637565Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 985 sessionId: src_id|4384c829-e1724355-206e5819-83a34e2f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:38.640091Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:38.686027Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341977800250253:8525]
2025-10-09T21:47:38.691210Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 984, pipe: [1:7559341977800250253:8525]
2025-10-09T21:47:38.812649Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:38.812983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:38.812999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:38.813027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341977800250253:8525], now have 1 active actors on pipe
2025-10-09T21:47:38.813410Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:38.814981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4384c829-e1724355-206e5819-83a34e2f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:38.814993Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4384c829-e1724355-206e5819-83a34e2f_0] Write session will restart in 0.007468s
2025-10-09T21:47:38.815021Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4384c829-e1724355-206e5819-83a34e2f_0] Write session: Do CDS request
2025-10-09T21:47:38.815027Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4384c829-e1724355-206e5819-83a34e2f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:38.814060Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341977800250253:8525]
2025-10-09T21:47:38.814098Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:38.814135Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:38.814223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 985 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4384c829-e1724355-206e5819-83a34e2f_0
2025-10-09T21:47:38.814416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 985 sessionId: src_id|4384c829-e1724355-206e5819-83a34e2f_0 is DEAD
2025-10-09T21:47:38.823613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4384c829-e1724355-206e5819-83a34e2f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:38.824124Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:38.824144Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 986
2025-10-09T21:47:38.824256Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 986 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:38.824291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 986 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:38.824321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=986 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:38.824325Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:38.825283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 986 sessionId:  describe result for acl check
2025-10-09T21:47:38.825336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:38.825346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:38.825348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:38.825355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341977800250282:9048] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:38.825285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:38.825952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341977800250282:9048] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:38.830362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:38.830623Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:38.830630Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:38.830649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:38.830725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:38.830774Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341977438430089:8145]
2025-10-09T21:47:38.831380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:38.831699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:38.831766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:38.832108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:38.832160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:38.832161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:38.832162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:38.832163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:38.832165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:38.832166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:38.832167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.832169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.832170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.832176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:38.832187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:38.832191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:38.832196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:38.832200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:38.832205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:38.832208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.832213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.832217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.832225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:38.832234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:38.832254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:38.832260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:38.832267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:38.832272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:38.832276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:38.832281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:38.832286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:38.832291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:38.832296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:38.832302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:38.832309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.832314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.832319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.832324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.832328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.832333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.832361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:38.839884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341977800250282:9048] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:38.840949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:38.842144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:38.842150Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:38.842153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:38.842157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:38.842165Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 985 [2:7559341977438430089:8145]
2025-10-09T21:47:38.842184Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:38.842207Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:38.842211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:38.842215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:38.842219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:38.842241Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:38.842247Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:38.842255Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:38.842321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.842325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.842327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.842330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:38.842455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:38.842457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.842471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:38.842481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:38.842495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:38.842516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341977438430084:8145]
2025-10-09T21:47:38.842518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341977438430084:8145]
2025-10-09T21:47:38.842520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341977438430084:8145]
2025-10-09T21:47:38.842539Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:38.842554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:38.842567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:38.842571Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:38.842614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341977438430084:8145]
2025-10-09T21:47:38.842616Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:38.842720Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:38.842723Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:38.842726Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:38.842732Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:38.846198Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 985
2025-10-09T21:47:38.847241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341977800250260:2432], now have 1 active actors on pipe
2025-10-09T21:47:38.858548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341977800250282:9048] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:38.858571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341977800250282:9048] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:38.858575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341977800250282:9048] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:38.858587Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 986 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:38.858989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341977800250324:9048], now have 1 active actors on pipe
2025-10-09T21:47:38.859179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 985
2025-10-09T21:47:38.859255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:38.859272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:38.859319Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|33638bf5-82b72b97-7b8a9e39-936c4da1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:38.859338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:38.859352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:38.859355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:38.859371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:38.859390Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:38.859428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:38.859701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:38.859711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:38.859729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:38.860404Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4384c829-e1724355-206e5819-83a34e2f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|33638bf5-82b72b97-7b8a9e39-936c4da1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:38.861214Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:38.861231Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:38.861234Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:38.861236Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:38.861245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341977438430084:8145]
2025-10-09T21:47:38.861250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341977438430084:8145]
2025-10-09T21:47:38.861254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341977438430084:8145]
2025-10-09T21:47:38.861301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:38.861308Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:38.861311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:38.861314Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:38.861316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:38.861319Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:38.861460Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:38.863018Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33638bf5-82b72b97-7b8a9e39-936c4da1_0] Write session: try to update token
2025-10-09T21:47:38.863035Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33638bf5-82b72b97-7b8a9e39-936c4da1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:38.859961Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 986 partition: 0 MaxSeqNo: 25 sessionId: src_id|33638bf5-82b72b97-7b8a9e39-936c4da1_0
2025-10-09T21:47:38.888354Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 986 sessionId: src_id|33638bf5-82b72b97-7b8a9e39-936c4da1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:38.890886Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:38.914250Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341977800250329:8525]
2025-10-09T21:47:38.914785Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 985, pipe: [1:7559341977800250329:8525]
2025-10-09T21:47:39.028708Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:39.029125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:39.029143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:39.029182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341977800250329:8525], now have 1 active actors on pipe
2025-10-09T21:47:39.029455Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:39.029465Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341977800250329:8525]
2025-10-09T21:47:39.029494Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:39.029498Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:39.029504Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:39.029579Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 986 reason: error at write: Disconnected sessionId: src_id|33638bf5-82b72b97-7b8a9e39-936c4da1_0
2025-10-09T21:47:39.029794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 986 sessionId: src_id|33638bf5-82b72b97-7b8a9e39-936c4da1_0 is DEAD
2025-10-09T21:47:39.029963Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33638bf5-82b72b97-7b8a9e39-936c4da1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:39.029976Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33638bf5-82b72b97-7b8a9e39-936c4da1_0] Write session will restart in 0.006222s
2025-10-09T21:47:39.030005Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33638bf5-82b72b97-7b8a9e39-936c4da1_0] Write session: Do CDS request
2025-10-09T21:47:39.030012Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33638bf5-82b72b97-7b8a9e39-936c4da1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:39.037714Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:39.037732Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 987
2025-10-09T21:47:39.037871Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33638bf5-82b72b97-7b8a9e39-936c4da1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:39.038118Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 987 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:39.038157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 987 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:39.038163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=987 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:39.038166Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:39.038745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 987 sessionId:  describe result for acl check
2025-10-09T21:47:39.038787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:39.038795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:39.038797Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:39.038803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341982095217650:9052] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:39.039344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:39.039385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341982095217650:9052] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:39.040851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:39.041061Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:39.041071Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:39.041085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:39.041153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:39.041202Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341981733397445:8147]
2025-10-09T21:47:39.041813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:39.042116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341982095217650:9052] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:39.042378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:39.042443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:39.042942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:39.043015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.043024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.043029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.043030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.043031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.043033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.043034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.043035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.043036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.043043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.043053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.043057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.043061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.043066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.043070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.043073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.043076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.043080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.043088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.043096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.043101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.043106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.043111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.043115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.043119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.043124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.043129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.043133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.043137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.043141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.043146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.043150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.043154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.043160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.043166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.043171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.043206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:39.048745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341982095217650:9052] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:39.048771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341982095217650:9052] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:39.048776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341982095217650:9052] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:39.048787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 987 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:39.050251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:39.051543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:39.051561Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:39.051565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:39.051573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:39.051584Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 986 [2:7559341981733397445:8147]
2025-10-09T21:47:39.051600Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:39.051632Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:39.051642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:39.051646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:39.051649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:39.051670Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:39.051677Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:39.051684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:39.051772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.051782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.051784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.051787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:39.051890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:39.051901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.051916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.051928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.051947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:39.051975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341982095217692:9052], now have 1 active actors on pipe
2025-10-09T21:47:39.051986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341982095217627:2432], now have 1 active actors on pipe
2025-10-09T21:47:39.051994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341981733397440:8147]
2025-10-09T21:47:39.051997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341981733397440:8147]
2025-10-09T21:47:39.052000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341981733397440:8147]
2025-10-09T21:47:39.052014Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:39.052029Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:39.052033Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:39.052035Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:39.052040Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:39.052068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:39.052073Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:39.052066Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 986
2025-10-09T21:47:39.052127Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 986
2025-10-09T21:47:39.052115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341981733397440:8147]
2025-10-09T21:47:39.052130Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:39.052151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:39.052156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:39.052165Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:39.052210Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8438659c-ea916f14-4d75f741-8e831102_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:39.052226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:39.052230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:39.052233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:39.052244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:39.052255Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:39.052278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:39.052504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:39.052516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:39.052554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:39.052648Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 987 partition: 0 MaxSeqNo: 25 sessionId: src_id|8438659c-ea916f14-4d75f741-8e831102_0
2025-10-09T21:47:39.053073Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33638bf5-82b72b97-7b8a9e39-936c4da1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8438659c-ea916f14-4d75f741-8e831102_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:39.055753Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8438659c-ea916f14-4d75f741-8e831102_0] Write session: try to update token
2025-10-09T21:47:39.055775Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8438659c-ea916f14-4d75f741-8e831102_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:39.068766Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:39.068782Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:39.068784Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:39.068786Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:39.068795Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341981733397440:8147]
2025-10-09T21:47:39.068799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341981733397440:8147]
2025-10-09T21:47:39.068805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341981733397440:8147]
2025-10-09T21:47:39.068851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:39.068857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.068859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:39.068874Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:39.068891Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:39.068895Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:39.068996Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:39.075202Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 987 sessionId: src_id|8438659c-ea916f14-4d75f741-8e831102_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:39.077745Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:39.130109Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341982095217703:8525]
2025-10-09T21:47:39.130442Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 986, pipe: [1:7559341982095217703:8525]
2025-10-09T21:47:39.233002Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:39.233428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:39.233445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:39.233497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341982095217703:8525], now have 1 active actors on pipe
2025-10-09T21:47:39.233790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:39.233799Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:39.233811Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:39.233811Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:39.233871Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341982095217703:8525]
2025-10-09T21:47:39.233887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 987 reason: error at write: Disconnected sessionId: src_id|8438659c-ea916f14-4d75f741-8e831102_0
2025-10-09T21:47:39.234113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 987 sessionId: src_id|8438659c-ea916f14-4d75f741-8e831102_0 is DEAD
2025-10-09T21:47:39.234298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8438659c-ea916f14-4d75f741-8e831102_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:39.234310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8438659c-ea916f14-4d75f741-8e831102_0] Write session will restart in 0.008515s
2025-10-09T21:47:39.234350Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8438659c-ea916f14-4d75f741-8e831102_0] Write session: Do CDS request
2025-10-09T21:47:39.234357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8438659c-ea916f14-4d75f741-8e831102_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:39.242695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:39.244381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:39.244580Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:39.244588Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:39.244597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:39.244617Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8438659c-ea916f14-4d75f741-8e831102_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:39.244638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:39.244670Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341981733397537:8159]
2025-10-09T21:47:39.244715Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:39.244734Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 988
2025-10-09T21:47:39.244869Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 988 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:39.244900Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 988 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:39.244911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=988 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:39.244914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:39.245260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:39.245345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 988 sessionId:  describe result for acl check
2025-10-09T21:47:39.245384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:39.245393Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:39.245395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:39.245409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341982095217735:9057] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:39.245550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:39.245597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:39.245942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:39.245997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.246004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.246005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.246006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.246008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.246009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.246010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.246012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.246013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.246019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.246017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341982095217735:9057] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:39.246045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.246049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.246053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.246056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.246060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.246064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.246067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.246070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.246078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.246087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.246094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.246100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.246105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.246111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.246116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.246121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.246126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.246131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.246139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.246145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.246152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.246156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.246160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.246167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.246172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.246176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.246203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:39.248173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341982095217735:9057] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:39.252439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341982095217735:9057] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:39.252464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341982095217735:9057] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:39.252469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341982095217735:9057] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:39.252486Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 988 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:39.252963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:39.254541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:39.254558Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:39.254561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:39.254566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:39.254575Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 987 [2:7559341981733397537:8159]
2025-10-09T21:47:39.254590Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:39.254609Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:39.254617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:39.254621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:39.254624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:39.254643Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:39.254649Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:39.254663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:39.254729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.254736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.254738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.254741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:39.254816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:39.254825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.254838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.254851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.254871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:39.254898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341982095217710:2432], now have 1 active actors on pipe
2025-10-09T21:47:39.254907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341982095217775:9057], now have 1 active actors on pipe
2025-10-09T21:47:39.254915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341981733397532:8159]
2025-10-09T21:47:39.254917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341981733397532:8159]
2025-10-09T21:47:39.254920Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341981733397532:8159]
2025-10-09T21:47:39.254934Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:39.254948Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:39.254953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:39.254958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:39.254960Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:39.254967Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:39.254971Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:39.254967Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 987
2025-10-09T21:47:39.254970Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 987
2025-10-09T21:47:39.255014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341981733397532:8159]
2025-10-09T21:47:39.255025Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:39.255033Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:39.255042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:39.255046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:39.255098Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2a6cf1c9-3f9a517e-ee30eaaf-b02c1037_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:39.255111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:39.255116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:39.255118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:39.255128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:39.255138Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:39.255168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:39.255411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:39.255421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:39.255436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:39.255531Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 988 partition: 0 MaxSeqNo: 25 sessionId: src_id|2a6cf1c9-3f9a517e-ee30eaaf-b02c1037_0
2025-10-09T21:47:39.255935Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8438659c-ea916f14-4d75f741-8e831102_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2a6cf1c9-3f9a517e-ee30eaaf-b02c1037_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:39.258597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a6cf1c9-3f9a517e-ee30eaaf-b02c1037_0] Write session: try to update token
2025-10-09T21:47:39.258619Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a6cf1c9-3f9a517e-ee30eaaf-b02c1037_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:39.270286Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:39.270303Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:39.270306Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:39.270307Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:39.270317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341981733397532:8159]
2025-10-09T21:47:39.270321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341981733397532:8159]
2025-10-09T21:47:39.270325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341981733397532:8159]
2025-10-09T21:47:39.270370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:39.270377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.270379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:39.270380Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:39.270387Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:39.270391Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:39.270526Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:39.272825Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 988 sessionId: src_id|2a6cf1c9-3f9a517e-ee30eaaf-b02c1037_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:39.275451Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:39.334091Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341982095217804:8525]
2025-10-09T21:47:39.334308Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 987, pipe: [1:7559341982095217804:8525]
2025-10-09T21:47:39.428436Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:39.428907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:39.428924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:39.428970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341982095217804:8525], now have 1 active actors on pipe
2025-10-09T21:47:39.429186Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:39.429245Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:39.429313Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:39.429316Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341982095217804:8525]
2025-10-09T21:47:39.429392Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 988 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2a6cf1c9-3f9a517e-ee30eaaf-b02c1037_0
2025-10-09T21:47:39.429586Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 988 sessionId: src_id|2a6cf1c9-3f9a517e-ee30eaaf-b02c1037_0 is DEAD
2025-10-09T21:47:39.429787Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a6cf1c9-3f9a517e-ee30eaaf-b02c1037_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:39.429796Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a6cf1c9-3f9a517e-ee30eaaf-b02c1037_0] Write session will restart in 0.009158s
2025-10-09T21:47:39.429834Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a6cf1c9-3f9a517e-ee30eaaf-b02c1037_0] Write session: Do CDS request
2025-10-09T21:47:39.429854Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a6cf1c9-3f9a517e-ee30eaaf-b02c1037_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:39.437549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:39.439192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:39.439398Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:39.439408Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:39.439420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:39.439474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:39.439519Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341981733397608:8163]
2025-10-09T21:47:39.440125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:39.440466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:39.440523Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a6cf1c9-3f9a517e-ee30eaaf-b02c1037_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:39.440542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:39.440656Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:39.440675Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 989
2025-10-09T21:47:39.440856Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 989 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:39.440890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 989 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:39.440902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=989 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:39.440888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:39.440906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:39.440949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.440956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.440957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.440959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.440960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.440961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.440962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.440964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.440965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.440971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.440981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.440985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.440990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.441005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.441009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.441013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.441017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.441021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.441028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.441039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.441049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.441055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.441064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.441069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.441077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.441081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.441085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.441092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.441101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.441106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.441111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.441117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.441126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.441131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.441136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.441144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.441175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:39.441335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 989 sessionId:  describe result for acl check
2025-10-09T21:47:39.441378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:39.441380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:39.441382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:39.441388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341982095217836:9068] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:39.441970Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341982095217836:9068] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:39.444357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341982095217836:9068] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:39.447591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341982095217836:9068] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:39.447605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341982095217836:9068] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:39.447610Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341982095217836:9068] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:39.447619Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 989 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:39.447792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:39.448845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:39.448859Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:39.448862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:39.448866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:39.448875Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 988 [2:7559341981733397608:8163]
2025-10-09T21:47:39.448890Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:39.448912Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:39.448919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:39.448922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:39.448925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:39.448942Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:39.448957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:39.448964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:39.449023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.449025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.449027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.449030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:39.449102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:39.449110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.449122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.449133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.449154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:39.449179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341981733397603:8163]
2025-10-09T21:47:39.449185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341981733397603:8163]
2025-10-09T21:47:39.449188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341981733397603:8163]
2025-10-09T21:47:39.449208Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:39.449208Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:39.449212Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:39.449215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:39.449230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341982095217807:2432], now have 1 active actors on pipe
2025-10-09T21:47:39.449233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341982095217871:9068], now have 1 active actors on pipe
2025-10-09T21:47:39.449239Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:39.449252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:39.449257Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:39.449272Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 988
2025-10-09T21:47:39.449290Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 988
2025-10-09T21:47:39.449298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341981733397603:8163]
2025-10-09T21:47:39.449307Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:39.449314Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:39.449441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:39.449450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:39.449477Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|37b3b4ee-31c9eb2-d7eb7f46-61bfeb7c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:39.449490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:39.449494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:39.449496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:39.449506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:39.449516Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:39.449529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:39.449897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 989 partition: 0 MaxSeqNo: 25 sessionId: src_id|37b3b4ee-31c9eb2-d7eb7f46-61bfeb7c_0
2025-10-09T21:47:39.450285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a6cf1c9-3f9a517e-ee30eaaf-b02c1037_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|37b3b4ee-31c9eb2-d7eb7f46-61bfeb7c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:39.449739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:39.449755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:39.449776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:39.452992Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|37b3b4ee-31c9eb2-d7eb7f46-61bfeb7c_0] Write session: try to update token
2025-10-09T21:47:39.453011Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|37b3b4ee-31c9eb2-d7eb7f46-61bfeb7c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:39.466772Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:39.466796Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:39.466808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341981733397603:8163]
2025-10-09T21:47:39.466862Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:39.466865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:39.467100Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:39.467273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:39.467418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:39.467528Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 989 sessionId: src_id|37b3b4ee-31c9eb2-d7eb7f46-61bfeb7c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:39.467576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:39.467790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:39.467839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:39.469992Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:39.470215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:39.470234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:39.470413Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:39.470274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:39.474932Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:39.474950Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:39.474956Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:39.474958Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:39.474968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341981733397603:8163]
2025-10-09T21:47:39.474971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341981733397603:8163]
2025-10-09T21:47:39.474974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341981733397603:8163]
2025-10-09T21:47:39.475014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:39.475018Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:39.475024Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:39.475027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.475029Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:39.475030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:39.475166Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:39.529735Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341982095217879:8525]
2025-10-09T21:47:39.530043Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 988, pipe: [1:7559341982095217879:8525]
2025-10-09T21:47:39.632904Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:39.640609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:39.640640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:39.640809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:39.640875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:39.640946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:39.641013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:39.641058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:39.641069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:39.641137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:39.641199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:39.641263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:39.641327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:39.641376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:39.641383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:39.641448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:39.641509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:39.641571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:39.641631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:39.641671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:39.641674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:39.641742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:39.641812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:39.641893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:39.641961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:39.642004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:39.642016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:39.642099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:39.642161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:39.642223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:39.642289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:39.642329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:39.642335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:39.642399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:39.642460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:39.642527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:39.642595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:39.642640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:39.642646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:39.642703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:39.642754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:39.642820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:39.642882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:39.642949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:39.642957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:39.643014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:39.643073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:39.643135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:39.643201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:39.643256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:39.643264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:39.643326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:39.643387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:39.643442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:39.643513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:39.643566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:39.643576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:39.643642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:39.643700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:39.643759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:39.643823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:39.643932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:39.643941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:39.644005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341982095217879:8525], now have 1 active actors on pipe
2025-10-09T21:47:39.644265Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341982095217879:8525]
2025-10-09T21:47:39.644266Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:39.644309Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:39.644313Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:39.644345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 989 reason: error at write: Disconnected sessionId: src_id|37b3b4ee-31c9eb2-d7eb7f46-61bfeb7c_0
2025-10-09T21:47:39.644551Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 989 sessionId: src_id|37b3b4ee-31c9eb2-d7eb7f46-61bfeb7c_0 is DEAD
2025-10-09T21:47:39.644700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37b3b4ee-31c9eb2-d7eb7f46-61bfeb7c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:39.644711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37b3b4ee-31c9eb2-d7eb7f46-61bfeb7c_0] Write session will restart in 0.006256s
2025-10-09T21:47:39.644737Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37b3b4ee-31c9eb2-d7eb7f46-61bfeb7c_0] Write session: Do CDS request
2025-10-09T21:47:39.644743Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37b3b4ee-31c9eb2-d7eb7f46-61bfeb7c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:39.645007Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:39.652581Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|37b3b4ee-31c9eb2-d7eb7f46-61bfeb7c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:39.652697Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:39.652715Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 990
2025-10-09T21:47:39.652869Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 990 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:39.652914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 990 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:39.652920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=990 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:39.652924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:39.653407Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 990 sessionId:  describe result for acl check
2025-10-09T21:47:39.653450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:39.653456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:39.653458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:39.653464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341982095217918:9072] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:39.654027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341982095217918:9072] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:39.654823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:39.656155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341982095217918:9072] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:39.656254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:39.656486Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:39.656496Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:39.656521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:39.656584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:39.656631Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341981733397678:8167]
2025-10-09T21:47:39.657212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:39.657529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:39.657576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:39.657904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:39.657956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.657966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.657968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.657969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.657971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.657972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.657973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.657975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.657976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.657981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.657994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.657998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.658003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.658007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.658011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.658016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.658020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.658025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.658047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.658056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.658065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.658071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.658075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.658081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.658087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.658093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.658098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.658104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.658108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.658114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.658119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.658125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.658129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.658134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.658139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.658145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.658186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:39.658864Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341982095217918:9072] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:39.658882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341982095217918:9072] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:39.658886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341982095217918:9072] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:39.658894Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 990 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:39.664878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:39.666007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:39.666024Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:39.666046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:39.666052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:39.666059Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 989 [2:7559341981733397678:8167]
2025-10-09T21:47:39.666074Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:39.666092Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:39.666100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:39.666103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:39.666107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:39.666125Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:39.666136Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:39.666143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:39.666211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.666219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.666220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.666223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:39.666298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:39.666309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.666321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.666339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.666354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:39.666379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341982095217882:2432], now have 1 active actors on pipe
2025-10-09T21:47:39.666387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341982095217958:9072], now have 1 active actors on pipe
2025-10-09T21:47:39.666395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341981733397672:8167]
2025-10-09T21:47:39.666397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341981733397672:8167]
2025-10-09T21:47:39.666399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341981733397672:8167]
2025-10-09T21:47:39.666414Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:39.666429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:39.666432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:39.666435Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:39.666438Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:39.666447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:39.666420Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 989
2025-10-09T21:47:39.666423Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 989
2025-10-09T21:47:39.667061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 990 partition: 0 MaxSeqNo: 25 sessionId: src_id|813410ee-ab3e8d1a-5da4b23d-aef01519_0
2025-10-09T21:47:39.667441Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37b3b4ee-31c9eb2-d7eb7f46-61bfeb7c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|813410ee-ab3e8d1a-5da4b23d-aef01519_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:39.666451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:39.666491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341981733397672:8167]
2025-10-09T21:47:39.666501Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:39.666514Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:39.666518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:39.666523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:39.666574Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|813410ee-ab3e8d1a-5da4b23d-aef01519_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:39.666583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:39.666587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:39.666589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:39.666600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:39.666610Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:39.666630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:39.666877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:39.666883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:39.666905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:39.670069Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|813410ee-ab3e8d1a-5da4b23d-aef01519_0] Write session: try to update token
2025-10-09T21:47:39.670092Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|813410ee-ab3e8d1a-5da4b23d-aef01519_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:39.681334Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:39.681357Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:39.681359Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:39.681362Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:39.681371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341981733397672:8167]
2025-10-09T21:47:39.681375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341981733397672:8167]
2025-10-09T21:47:39.681379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341981733397672:8167]
2025-10-09T21:47:39.681428Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:39.681431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:39.681434Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:39.681437Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:39.681439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.681442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:39.681584Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:39.683957Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 990 sessionId: src_id|813410ee-ab3e8d1a-5da4b23d-aef01519_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:39.686427Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:39.744276Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341982095217966:8525]
2025-10-09T21:47:39.744563Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 989, pipe: [1:7559341982095217966:8525]
2025-10-09T21:47:39.843232Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:39.843637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:39.843661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:39.843713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341982095217966:8525], now have 1 active actors on pipe
2025-10-09T21:47:39.843946Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:39.843968Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:39.843972Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:39.844005Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341982095217966:8525]
2025-10-09T21:47:39.844036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 990 reason: error at write: Disconnected sessionId: src_id|813410ee-ab3e8d1a-5da4b23d-aef01519_0
2025-10-09T21:47:39.844253Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 990 sessionId: src_id|813410ee-ab3e8d1a-5da4b23d-aef01519_0 is DEAD
2025-10-09T21:47:39.844407Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|813410ee-ab3e8d1a-5da4b23d-aef01519_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:39.844418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|813410ee-ab3e8d1a-5da4b23d-aef01519_0] Write session will restart in 0.009133s
2025-10-09T21:47:39.844455Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|813410ee-ab3e8d1a-5da4b23d-aef01519_0] Write session: Do CDS request
2025-10-09T21:47:39.844465Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|813410ee-ab3e8d1a-5da4b23d-aef01519_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:39.844474Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:39.853095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:39.854518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|813410ee-ab3e8d1a-5da4b23d-aef01519_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:39.854632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:39.854654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 991
2025-10-09T21:47:39.854719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:39.854770Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 991 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:39.854815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 991 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:39.854822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=991 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:39.854833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:39.854949Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:39.854961Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:39.854976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:39.855047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:39.855092Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341981733397742:8169]
2025-10-09T21:47:39.855323Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 991 sessionId:  describe result for acl check
2025-10-09T21:47:39.855364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:39.855366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:39.855367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:39.855372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341982095217999:9077] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:39.855569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:39.855871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341982095217999:9077] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:39.855918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:39.855975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:39.856340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:39.856413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.856422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.856423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.856425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.856426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.856427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.856428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.856430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.856431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.856437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.856447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.856452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.856456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.856461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.856465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.856469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.856473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.856477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.856483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.856491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:39.856499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.856505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:39.856510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.856516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:39.856521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.856526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:39.856530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.856534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:39.856538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.856541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:39.856545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.856551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.856554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.856560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.856564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.856570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.856598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:39.857803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341982095217999:9077] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:39.862704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341982095217999:9077] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:39.862722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341982095217999:9077] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:39.862726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341982095217999:9077] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:39.862735Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 991 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:39.863057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:39.864174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:39.864194Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:39.864196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:39.864200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:39.864207Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 990 [2:7559341981733397742:8169]
2025-10-09T21:47:39.864223Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:39.864241Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:39.864247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:39.864252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:39.864254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:39.864271Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:39.864283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:39.864291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:39.864349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.864356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.864358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.864361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:39.864437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:39.864445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.864458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:39.864470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:39.864488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:39.864512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341981733397737:8169]
2025-10-09T21:47:39.864519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341981733397737:8169]
2025-10-09T21:47:39.864522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341981733397737:8169]
2025-10-09T21:47:39.864535Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:39.864550Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:39.864551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341982095218037:9077], now have 1 active actors on pipe
2025-10-09T21:47:39.864555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341982095217971:2432], now have 1 active actors on pipe
2025-10-09T21:47:39.864561Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:39.864563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:39.864563Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:39.864578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:39.864581Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:39.864618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341981733397737:8169]
2025-10-09T21:47:39.864606Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 990
2025-10-09T21:47:39.864626Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:39.864626Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 990
2025-10-09T21:47:39.864657Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:39.864663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:39.864667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:39.864699Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|11afba70-281633c8-5ba60846-342be858_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:39.864711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:39.864726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:39.864729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:39.864740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:39.864751Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:39.864771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:39.864961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:39.864974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:39.864998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:39.865109Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 991 partition: 0 MaxSeqNo: 25 sessionId: src_id|11afba70-281633c8-5ba60846-342be858_0
2025-10-09T21:47:39.865462Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|813410ee-ab3e8d1a-5da4b23d-aef01519_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|11afba70-281633c8-5ba60846-342be858_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:39.867951Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|11afba70-281633c8-5ba60846-342be858_0] Write session: try to update token
2025-10-09T21:47:39.867970Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|11afba70-281633c8-5ba60846-342be858_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:39.880064Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:39.880083Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:39.880085Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:39.880088Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:39.880098Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341981733397737:8169]
2025-10-09T21:47:39.880103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341981733397737:8169]
2025-10-09T21:47:39.880107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341981733397737:8169]
2025-10-09T21:47:39.880145Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:39.880150Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:39.880150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:39.880153Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:39.880160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:39.880163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:39.880298Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:39.881910Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 991 sessionId: src_id|11afba70-281633c8-5ba60846-342be858_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:39.884263Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:39.944439Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341982095218046:8525]
2025-10-09T21:47:39.944747Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 990, pipe: [1:7559341982095218046:8525]
2025-10-09T21:47:40.041722Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:40.042050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:40.042067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:40.042114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341982095218046:8525], now have 1 active actors on pipe
2025-10-09T21:47:40.042327Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:40.042363Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341982095218046:8525]
2025-10-09T21:47:40.042400Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:40.042400Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:40.042404Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:40.042466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 991 reason: error at write: Disconnected sessionId: src_id|11afba70-281633c8-5ba60846-342be858_0
2025-10-09T21:47:40.042662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 991 sessionId: src_id|11afba70-281633c8-5ba60846-342be858_0 is DEAD
2025-10-09T21:47:40.042799Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11afba70-281633c8-5ba60846-342be858_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:40.042811Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11afba70-281633c8-5ba60846-342be858_0] Write session will restart in 0.006627s
2025-10-09T21:47:40.042839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11afba70-281633c8-5ba60846-342be858_0] Write session: Do CDS request
2025-10-09T21:47:40.042845Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11afba70-281633c8-5ba60846-342be858_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:40.050641Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|11afba70-281633c8-5ba60846-342be858_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:40.050733Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:40.050753Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 992
2025-10-09T21:47:40.050925Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 992 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:40.050932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:40.050985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 992 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:40.050996Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=992 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:40.051000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:40.051440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 992 sessionId:  describe result for acl check
2025-10-09T21:47:40.051483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:40.051491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:40.051492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:40.051499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341986390185373:9081] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:40.052066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341986390185373:9081] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:40.052426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:40.052646Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:40.052658Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:40.052671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:40.052729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:40.052773Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341986028365104:8171]
2025-10-09T21:47:40.053329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:40.053648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:40.053710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:40.054080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:40.054153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.054162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.054164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.054165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.054166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.054168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.054169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.054171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.054172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.054178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.054188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.054191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.054195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.054199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.054203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.054206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.054210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.054214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.054224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.054233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.054241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.054246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.054251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.054255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.054260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.054266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.054271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.054276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.054280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.054286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.054292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.054297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.054302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.054310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.054315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.054323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.054317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341986390185373:9081] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:40.054354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:40.060660Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341986390185373:9081] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:40.060682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341986390185373:9081] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:40.060687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341986390185373:9081] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:40.060701Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 992 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:40.060908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:40.062090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:40.062109Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:40.062112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:40.062116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:40.062124Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 991 [2:7559341986028365104:8171]
2025-10-09T21:47:40.062139Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:40.062158Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:40.062161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:40.062164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:40.062167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:40.062200Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:40.062207Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:40.062218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:40.062281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.062289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.062291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.062294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:40.062393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:40.062401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.062414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.062478Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 991
2025-10-09T21:47:40.062426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.062452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:40.062474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341986390185344:2432], now have 1 active actors on pipe
2025-10-09T21:47:40.062477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341986390185412:9081], now have 1 active actors on pipe
2025-10-09T21:47:40.062485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341986028365099:8171]
2025-10-09T21:47:40.062531Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 991
2025-10-09T21:47:40.062497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341986028365099:8171]
2025-10-09T21:47:40.062500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341986028365099:8171]
2025-10-09T21:47:40.062513Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:40.062522Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:40.062525Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:40.062527Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:40.062528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:40.062543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:40.062546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:40.062594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341986028365099:8171]
2025-10-09T21:47:40.062603Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:40.062619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:40.062629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:40.062628Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:40.062678Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|18f13658-8983cb1e-4bbc81b5-b0bceae2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:40.062693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:40.062696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:40.062699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:40.062709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:40.062719Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:40.062744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:40.062928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:40.062937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:40.062951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:40.063038Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 992 partition: 0 MaxSeqNo: 25 sessionId: src_id|18f13658-8983cb1e-4bbc81b5-b0bceae2_0
2025-10-09T21:47:40.063463Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11afba70-281633c8-5ba60846-342be858_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|18f13658-8983cb1e-4bbc81b5-b0bceae2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:40.066074Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|18f13658-8983cb1e-4bbc81b5-b0bceae2_0] Write session: try to update token
2025-10-09T21:47:40.066097Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|18f13658-8983cb1e-4bbc81b5-b0bceae2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:40.078096Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:40.078116Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:40.078119Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:40.078121Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:40.078130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341986028365099:8171]
2025-10-09T21:47:40.078135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341986028365099:8171]
2025-10-09T21:47:40.078139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341986028365099:8171]
2025-10-09T21:47:40.078186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:40.078191Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:40.078194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.078197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:40.078197Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:40.078202Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:40.078343Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:40.079680Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 992 sessionId: src_id|18f13658-8983cb1e-4bbc81b5-b0bceae2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:40.082243Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:40.142606Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341986390185423:8525]
2025-10-09T21:47:40.142923Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 991, pipe: [1:7559341986390185423:8525]
2025-10-09T21:47:40.236850Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:40.237130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:40.237146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:40.237193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341986390185423:8525], now have 1 active actors on pipe
2025-10-09T21:47:40.237426Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:40.237449Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:40.237437Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341986390185423:8525]
2025-10-09T21:47:40.237474Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:40.237477Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:40.237549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 992 reason: error at write: Disconnected sessionId: src_id|18f13658-8983cb1e-4bbc81b5-b0bceae2_0
2025-10-09T21:47:40.237744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 992 sessionId: src_id|18f13658-8983cb1e-4bbc81b5-b0bceae2_0 is DEAD
2025-10-09T21:47:40.237898Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18f13658-8983cb1e-4bbc81b5-b0bceae2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:40.237910Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18f13658-8983cb1e-4bbc81b5-b0bceae2_0] Write session will restart in 0.009543s
2025-10-09T21:47:40.237943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18f13658-8983cb1e-4bbc81b5-b0bceae2_0] Write session: Do CDS request
2025-10-09T21:47:40.237949Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18f13658-8983cb1e-4bbc81b5-b0bceae2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:40.245452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:40.246875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:40.247045Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:40.247056Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:40.247068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:40.247128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:40.247174Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341986028365201:8183]
2025-10-09T21:47:40.247731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:40.248047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:40.248115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:40.248456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:40.248512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.248517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.248519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.248520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.248521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.248522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.248524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.248525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.248526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.248532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.248544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.248556Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|18f13658-8983cb1e-4bbc81b5-b0bceae2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:40.248548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.248552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.248556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.248560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.248573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.248577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.248580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.248587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.248595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.248602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.248609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.248615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.248620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.248624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.248629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.248616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:40.248633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.248638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.248634Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 993
2025-10-09T21:47:40.248642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.248647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.248652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.248658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.248662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.248667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.248671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.248676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.248701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:40.248816Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 993 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:40.248849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 993 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:40.248859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=993 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:40.248863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:40.249295Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 993 sessionId:  describe result for acl check
2025-10-09T21:47:40.249344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:40.249351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:40.249353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:40.249359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341986390185466:9086] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:40.249930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341986390185466:9086] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:40.252512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341986390185466:9086] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:40.254829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341986390185466:9086] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:40.254840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341986390185466:9086] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:40.254844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341986390185466:9086] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:40.254853Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 993 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:40.255079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:40.256227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:40.256242Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:40.256244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:40.256249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:40.256255Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 992 [2:7559341986028365201:8183]
2025-10-09T21:47:40.256269Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:40.256285Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:40.256288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:40.256293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:40.256295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:40.256311Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:40.256320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:40.256328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:40.256390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.256399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.256401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.256405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:40.256471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:40.256479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.256492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.256509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.256530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:40.256551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341986390185430:2432], now have 1 active actors on pipe
2025-10-09T21:47:40.256560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341986390185500:9086], now have 1 active actors on pipe
2025-10-09T21:47:40.256567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341986028365196:8183]
2025-10-09T21:47:40.256570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341986028365196:8183]
2025-10-09T21:47:40.256572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341986028365196:8183]
2025-10-09T21:47:40.256586Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:40.256595Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:40.256599Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:40.256600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:40.256601Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:40.256582Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 992
2025-10-09T21:47:40.256601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 992
2025-10-09T21:47:40.256615Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:40.256618Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:40.256659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341986028365196:8183]
2025-10-09T21:47:40.256672Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:40.256678Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:40.256689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:40.256694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:40.256743Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8d8e6097-d1b4ec1d-368e4371-87170e3c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:40.256758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:40.256762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:40.256770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:40.256780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:40.256795Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:40.256815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:40.256990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:40.257005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:40.257041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:40.257129Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 993 partition: 0 MaxSeqNo: 25 sessionId: src_id|8d8e6097-d1b4ec1d-368e4371-87170e3c_0
2025-10-09T21:47:40.257495Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18f13658-8983cb1e-4bbc81b5-b0bceae2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8d8e6097-d1b4ec1d-368e4371-87170e3c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:40.260177Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8d8e6097-d1b4ec1d-368e4371-87170e3c_0] Write session: try to update token
2025-10-09T21:47:40.260199Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8d8e6097-d1b4ec1d-368e4371-87170e3c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:40.270725Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:40.270747Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:40.270749Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:40.270754Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:40.270763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341986028365196:8183]
2025-10-09T21:47:40.270768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341986028365196:8183]
2025-10-09T21:47:40.270771Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341986028365196:8183]
2025-10-09T21:47:40.270811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:40.270812Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:40.270818Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:40.270818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.270820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:40.270822Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:40.270951Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:40.273360Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 993 sessionId: src_id|8d8e6097-d1b4ec1d-368e4371-87170e3c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:40.275667Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:40.337615Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341986390185520:8525]
2025-10-09T21:47:40.337854Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 992, pipe: [1:7559341986390185520:8525]
2025-10-09T21:47:40.429718Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:40.430062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:40.430078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:40.430124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341986390185520:8525], now have 1 active actors on pipe
2025-10-09T21:47:40.430356Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:40.430372Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:40.430373Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:40.430420Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:40.430436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 993 reason: error at write: Disconnected sessionId: src_id|8d8e6097-d1b4ec1d-368e4371-87170e3c_0
2025-10-09T21:47:40.430465Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341986390185520:8525]
2025-10-09T21:47:40.430633Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 993 sessionId: src_id|8d8e6097-d1b4ec1d-368e4371-87170e3c_0 is DEAD
2025-10-09T21:47:40.430818Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d8e6097-d1b4ec1d-368e4371-87170e3c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:40.430830Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d8e6097-d1b4ec1d-368e4371-87170e3c_0] Write session will restart in 0.006649s
2025-10-09T21:47:40.430858Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d8e6097-d1b4ec1d-368e4371-87170e3c_0] Write session: Do CDS request
2025-10-09T21:47:40.430864Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d8e6097-d1b4ec1d-368e4371-87170e3c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:40.438538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8d8e6097-d1b4ec1d-368e4371-87170e3c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:40.438645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:40.438619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:40.438637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 994
2025-10-09T21:47:40.438841Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 994 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:40.438882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 994 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:40.438891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=994 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:40.438896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:40.439369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 994 sessionId:  describe result for acl check
2025-10-09T21:47:40.439406Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:40.439412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:40.439414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:40.439419Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341986390185564:9097] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:40.439945Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341986390185564:9097] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:40.440308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:40.440508Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:40.440519Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:40.440532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:40.440595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:40.440636Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341986028365277:8187]
2025-10-09T21:47:40.441302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:40.441561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:40.441603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:40.441946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:40.442004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.442013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.442014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.442015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.442016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.442017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.442018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.442020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.442022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.442044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.442055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.442058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.442063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.442067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.442071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.442074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.442078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.442082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.442089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.442097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.442104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.442111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.442134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.442140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.442144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.442150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.442154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.442159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.442163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.442167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.442172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.442177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.442181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.442185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.442189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.442194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.442223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:40.442362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341986390185564:9097] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:40.448187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341986390185564:9097] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:40.448206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341986390185564:9097] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:40.448210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341986390185564:9097] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:40.448219Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 994 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:40.448604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:40.449832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:40.449845Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:40.449847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:40.449851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:40.449858Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 993 [2:7559341986028365277:8187]
2025-10-09T21:47:40.449874Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:40.449895Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:40.449905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:40.449908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:40.449912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:40.449928Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:40.449938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:40.449946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:40.450005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.450012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.450014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.450017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:40.450106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:40.450115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.450126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.450137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.450153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:40.450174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341986390185534:2432], now have 1 active actors on pipe
2025-10-09T21:47:40.450183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341986390185601:9097], now have 1 active actors on pipe
2025-10-09T21:47:40.450191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341986028365272:8187]
2025-10-09T21:47:40.450196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341986028365272:8187]
2025-10-09T21:47:40.450198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341986028365272:8187]
2025-10-09T21:47:40.450211Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:40.450218Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 993
2025-10-09T21:47:40.450225Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 993
2025-10-09T21:47:40.450226Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:40.450228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:40.450236Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:40.450238Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:40.450242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:40.450246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:40.450279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341986028365272:8187]
2025-10-09T21:47:40.450284Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:40.450284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:40.450300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:40.450305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:40.450356Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bf98d9ca-ede81746-fbc14312-4b38ae7d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:40.450369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:40.450374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:40.450377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:40.450387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:40.450396Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:40.450417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:40.450555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:40.450562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:40.450575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:40.450664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 994 partition: 0 MaxSeqNo: 25 sessionId: src_id|bf98d9ca-ede81746-fbc14312-4b38ae7d_0
2025-10-09T21:47:40.451043Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d8e6097-d1b4ec1d-368e4371-87170e3c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bf98d9ca-ede81746-fbc14312-4b38ae7d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:40.453450Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf98d9ca-ede81746-fbc14312-4b38ae7d_0] Write session: try to update token
2025-10-09T21:47:40.453466Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf98d9ca-ede81746-fbc14312-4b38ae7d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:40.464813Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:40.464832Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:40.464835Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:40.464837Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:40.464847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341986028365272:8187]
2025-10-09T21:47:40.464852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341986028365272:8187]
2025-10-09T21:47:40.464856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341986028365272:8187]
2025-10-09T21:47:40.464902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:40.464902Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:40.464909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.464909Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:40.464912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:40.464913Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:40.465042Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:40.466634Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 994 sessionId: src_id|bf98d9ca-ede81746-fbc14312-4b38ae7d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:40.468920Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:40.530707Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341986390185608:8525]
2025-10-09T21:47:40.530987Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 993, pipe: [1:7559341986390185608:8525]
2025-10-09T21:47:40.619306Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:40.619616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:40.619631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:40.619676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341986390185608:8525], now have 1 active actors on pipe
2025-10-09T21:47:40.619884Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:40.619926Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:40.619932Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:40.619936Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:40.619997Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341986390185608:8525]
2025-10-09T21:47:40.620011Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 994 reason: error at write: Disconnected sessionId: src_id|bf98d9ca-ede81746-fbc14312-4b38ae7d_0
2025-10-09T21:47:40.620204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 994 sessionId: src_id|bf98d9ca-ede81746-fbc14312-4b38ae7d_0 is DEAD
2025-10-09T21:47:40.620354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf98d9ca-ede81746-fbc14312-4b38ae7d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:40.620366Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf98d9ca-ede81746-fbc14312-4b38ae7d_0] Write session will restart in 0.005506s
2025-10-09T21:47:40.620393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf98d9ca-ede81746-fbc14312-4b38ae7d_0] Write session: Do CDS request
2025-10-09T21:47:40.620400Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf98d9ca-ede81746-fbc14312-4b38ae7d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:40.627517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf98d9ca-ede81746-fbc14312-4b38ae7d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:40.627636Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:40.627659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 995
2025-10-09T21:47:40.627794Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 995 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:40.627827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 995 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:40.627838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=995 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:40.627842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:40.628288Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 995 sessionId:  describe result for acl check
2025-10-09T21:47:40.628323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:40.628330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:40.628333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:40.628339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341986390185642:9101] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:40.628437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:40.628888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341986390185642:9101] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:40.630364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:40.630556Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:40.630568Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:40.630581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:40.630660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:40.630709Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341986028365351:8191]
2025-10-09T21:47:40.631132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341986390185642:9101] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:40.631323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:40.631815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:40.631871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:40.632206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:40.632284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.632290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.632292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.632293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.632294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.632295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.632297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.632298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.632299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.632305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.632315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.632318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.632322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.632325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.632329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.632332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.632336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.632340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.632346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.632354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.632361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.632366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.632371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.632376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.632380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.632385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.632389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.632394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.632399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.632407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.632414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.632419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.632430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.632434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.632439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.632446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.632472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:40.637655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341986390185642:9101] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:40.637674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341986390185642:9101] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:40.637678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341986390185642:9101] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:40.637688Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 995 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:40.639023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:40.640253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:40.640276Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:40.640279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:40.640283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:40.640289Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 994 [2:7559341986028365351:8191]
2025-10-09T21:47:40.640303Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:40.640321Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:40.640327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:40.640330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:40.640333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:40.640351Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:40.640361Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:40.640368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:40.640423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.640430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.640432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.640434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:40.640502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:40.640509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.640521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.640536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.640558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:40.640583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341986390185611:2432], now have 1 active actors on pipe
2025-10-09T21:47:40.640592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341986390185682:9101], now have 1 active actors on pipe
2025-10-09T21:47:40.640600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341986028365346:8191]
2025-10-09T21:47:40.640602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341986028365346:8191]
2025-10-09T21:47:40.640618Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 994
2025-10-09T21:47:40.640604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341986028365346:8191]
2025-10-09T21:47:40.640618Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 994
2025-10-09T21:47:40.640616Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:40.640632Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:40.640633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:40.640635Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:40.640637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:40.640649Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:40.640652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:40.640694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341986028365346:8191]
2025-10-09T21:47:40.640704Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:40.640715Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:40.640724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:40.640734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:40.640791Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|26a1edb0-7aeb6380-ca4f57a9-f9555834_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:40.640804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:40.640808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:40.640810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:40.640819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:40.640829Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:40.640848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:40.641019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:40.641028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:40.641044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:40.641145Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 995 partition: 0 MaxSeqNo: 25 sessionId: src_id|26a1edb0-7aeb6380-ca4f57a9-f9555834_0
2025-10-09T21:47:40.641507Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf98d9ca-ede81746-fbc14312-4b38ae7d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|26a1edb0-7aeb6380-ca4f57a9-f9555834_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:40.644013Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26a1edb0-7aeb6380-ca4f57a9-f9555834_0] Write session: try to update token
2025-10-09T21:47:40.644028Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26a1edb0-7aeb6380-ca4f57a9-f9555834_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:40.654947Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:40.654969Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:40.654971Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:40.654973Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:40.654984Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341986028365346:8191]
2025-10-09T21:47:40.654989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341986028365346:8191]
2025-10-09T21:47:40.654993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341986028365346:8191]
2025-10-09T21:47:40.655037Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:40.655040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:40.655050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.655050Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:40.655054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:40.655055Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:40.655185Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:40.657528Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 995 sessionId: src_id|26a1edb0-7aeb6380-ca4f57a9-f9555834_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:40.660245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:40.720163Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341986390185689:8525]
2025-10-09T21:47:40.720658Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 994, pipe: [1:7559341986390185689:8525]
2025-10-09T21:47:40.812372Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:40.812718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:40.812736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:40.812784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341986390185689:8525], now have 1 active actors on pipe
2025-10-09T21:47:40.813014Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:40.813068Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:40.813078Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341986390185689:8525]
2025-10-09T21:47:40.813104Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:40.813180Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 995 reason: status is not ok: tablet will be restarted right now sessionId: src_id|26a1edb0-7aeb6380-ca4f57a9-f9555834_0
2025-10-09T21:47:40.813405Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 995 sessionId: src_id|26a1edb0-7aeb6380-ca4f57a9-f9555834_0 is DEAD
2025-10-09T21:47:40.813560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26a1edb0-7aeb6380-ca4f57a9-f9555834_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:40.813573Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26a1edb0-7aeb6380-ca4f57a9-f9555834_0] Write session will restart in 0.009897s
2025-10-09T21:47:40.813614Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26a1edb0-7aeb6380-ca4f57a9-f9555834_0] Write session: Do CDS request
2025-10-09T21:47:40.813621Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26a1edb0-7aeb6380-ca4f57a9-f9555834_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:40.821674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:40.823419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:40.823643Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:40.823655Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:40.823667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:40.823720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:40.823770Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341986028365421:8193]
2025-10-09T21:47:40.824335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:40.824658Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26a1edb0-7aeb6380-ca4f57a9-f9555834_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:40.824642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:40.824687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:40.824726Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:40.824754Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 996
2025-10-09T21:47:40.824886Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 996 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:40.824930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 996 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:40.824942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=996 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:40.824945Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:40.825009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:40.825069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.825075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.825076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.825078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.825079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.825081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.825082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.825083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.825084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.825090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.825100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.825104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.825107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.825109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.825112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.825115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.825119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.825122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.825133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.825141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:40.825147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.825152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:40.825161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.825166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:40.825174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.825179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:40.825184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.825188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:40.825192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.825198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:40.825203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.825207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.825211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.825221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.825225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.825232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.825267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:40.825386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 996 sessionId:  describe result for acl check
2025-10-09T21:47:40.825422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:40.825427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:40.825429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:40.825435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341986390185730:9106] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:40.826010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341986390185730:9106] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:40.828009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341986390185730:9106] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:40.831187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341986390185730:9106] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:40.831202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341986390185730:9106] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:40.831206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341986390185730:9106] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:40.831216Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 996 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:40.831743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:40.832886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:40.832899Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:40.832902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:40.832907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:40.832914Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 995 [2:7559341986028365421:8193]
2025-10-09T21:47:40.832929Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:40.832947Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:40.832955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:40.832959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:40.832962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:40.832982Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:40.832993Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:40.833000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:40.833069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.833078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.833079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.833082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:40.833154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:40.833162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.833174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:40.833186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:40.833204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:40.833227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341986390185766:9106], now have 1 active actors on pipe
2025-10-09T21:47:40.833235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341986390185694:2432], now have 1 active actors on pipe
2025-10-09T21:47:40.833243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341986028365416:8193]
2025-10-09T21:47:40.833245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341986028365416:8193]
2025-10-09T21:47:40.833247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341986028365416:8193]
2025-10-09T21:47:40.833259Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:40.833307Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 995
2025-10-09T21:47:40.833347Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 995
2025-10-09T21:47:40.833273Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:40.833277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:40.833282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:40.833284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:40.833290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:40.833293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:40.833330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341986028365416:8193]
2025-10-09T21:47:40.833334Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:40.833336Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:40.833362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:40.833367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:40.833401Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5beea595-e037c244-a67fdca8-49e31884_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:40.833409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:40.833413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:40.833416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:40.833428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:40.833437Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:40.833462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:40.833602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:40.833615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:40.833649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:40.833748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 996 partition: 0 MaxSeqNo: 25 sessionId: src_id|5beea595-e037c244-a67fdca8-49e31884_0
2025-10-09T21:47:40.834197Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26a1edb0-7aeb6380-ca4f57a9-f9555834_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5beea595-e037c244-a67fdca8-49e31884_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:40.836762Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5beea595-e037c244-a67fdca8-49e31884_0] Write session: try to update token
2025-10-09T21:47:40.836780Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5beea595-e037c244-a67fdca8-49e31884_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:40.848031Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:40.848053Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:40.848055Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:40.848057Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:40.848068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341986028365416:8193]
2025-10-09T21:47:40.848073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341986028365416:8193]
2025-10-09T21:47:40.848076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341986028365416:8193]
2025-10-09T21:47:40.848124Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:40.848129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:40.848131Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:40.848135Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:40.848136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:40.848139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:40.848282Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:40.851170Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 996 sessionId: src_id|5beea595-e037c244-a67fdca8-49e31884_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:40.853682Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:40.913125Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341986390185773:8525]
2025-10-09T21:47:40.913547Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 995, pipe: [1:7559341986390185773:8525]
2025-10-09T21:47:41.004633Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:41.005009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.005024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.005073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341986390185773:8525], now have 1 active actors on pipe
2025-10-09T21:47:41.005312Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:41.005312Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341986390185773:8525]
2025-10-09T21:47:41.005364Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:41.005373Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:41.005452Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 996 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5beea595-e037c244-a67fdca8-49e31884_0
2025-10-09T21:47:41.005637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 996 sessionId: src_id|5beea595-e037c244-a67fdca8-49e31884_0 is DEAD
2025-10-09T21:47:41.005831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5beea595-e037c244-a67fdca8-49e31884_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:41.005842Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5beea595-e037c244-a67fdca8-49e31884_0] Write session will restart in 0.006462s
2025-10-09T21:47:41.005886Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5beea595-e037c244-a67fdca8-49e31884_0] Write session: Do CDS request
2025-10-09T21:47:41.005894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5beea595-e037c244-a67fdca8-49e31884_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:41.013731Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5beea595-e037c244-a67fdca8-49e31884_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:41.013698Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:41.013719Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 997
2025-10-09T21:47:41.013983Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 997 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:41.014049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 997 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:41.014061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=997 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:41.014066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:41.014529Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 997 sessionId:  describe result for acl check
2025-10-09T21:47:41.014572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:41.014574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:41.014576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:41.014583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341990685153107:9110] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:41.014565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:41.015184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341990685153107:9110] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:41.016269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:41.016486Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:41.016496Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:41.016510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:41.016582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:41.016625Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341990323332788:8195]
2025-10-09T21:47:41.017191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:41.017541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:41.017566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341990685153107:9110] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:41.017618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:41.017971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:41.018046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.018054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.018056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.018057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.018059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.018060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.018061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.018063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.018064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.018071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.018082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.018086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.018090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.018094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.018098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.018101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.018104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.018108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.018116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.018125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.018131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.018137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.018141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.018147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.018151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.018156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.018161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.018166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.018172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.018178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.018183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.018189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.018194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.018198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.018201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.018206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.018235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:41.023844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341990685153107:9110] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:41.023869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341990685153107:9110] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:41.023874Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341990685153107:9110] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:41.023884Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 997 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:41.025130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:41.026396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:41.026414Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:41.026417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:41.026422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:41.026430Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 996 [2:7559341990323332788:8195]
2025-10-09T21:47:41.026447Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:41.026467Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:41.026470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:41.026473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:41.026477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:41.026499Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:41.026512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:41.026520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:41.026597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.026606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.026608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.026611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:41.026688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:41.026695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.026708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.026720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.026737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:41.026767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341990685153144:9110], now have 1 active actors on pipe
2025-10-09T21:47:41.026771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341990685153074:2432], now have 1 active actors on pipe
2025-10-09T21:47:41.026778Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341990323332783:8195]
2025-10-09T21:47:41.026785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341990323332783:8195]
2025-10-09T21:47:41.026787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341990323332783:8195]
2025-10-09T21:47:41.026805Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:41.026826Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:41.026828Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:41.026832Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:41.026834Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:41.026846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:41.026849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:41.026895Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 996
2025-10-09T21:47:41.026903Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 996
2025-10-09T21:47:41.026899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341990323332783:8195]
2025-10-09T21:47:41.026908Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:41.026938Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:41.026951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.026957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.026986Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|496badfa-4f364fba-fad4886d-7fc7d5a9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:41.027000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:41.027004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:41.027006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:41.027018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:41.027034Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:41.027052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:41.027244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.027256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.027290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:41.027390Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 997 partition: 0 MaxSeqNo: 25 sessionId: src_id|496badfa-4f364fba-fad4886d-7fc7d5a9_0
2025-10-09T21:47:41.027771Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5beea595-e037c244-a67fdca8-49e31884_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|496badfa-4f364fba-fad4886d-7fc7d5a9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:41.030456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|496badfa-4f364fba-fad4886d-7fc7d5a9_0] Write session: try to update token
2025-10-09T21:47:41.030475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|496badfa-4f364fba-fad4886d-7fc7d5a9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:41.044761Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:41.044784Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:41.044796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341990323332783:8195]
2025-10-09T21:47:41.044856Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:41.044862Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:41.045064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:41.045223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:41.045220Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 997 sessionId: src_id|496badfa-4f364fba-fad4886d-7fc7d5a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:41.045382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:41.045547Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:41.045765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:41.045813Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:41.047939Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:41.048153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.048172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.048212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:41.048347Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:41.056645Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:41.056664Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:41.056667Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:41.056670Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:41.056681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341990323332783:8195]
2025-10-09T21:47:41.056689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341990323332783:8195]
2025-10-09T21:47:41.056693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341990323332783:8195]
2025-10-09T21:47:41.056739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:41.056747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.056750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:41.056753Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:41.056758Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:41.056762Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:41.056922Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:41.105665Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341990685153152:8525]
2025-10-09T21:47:41.106024Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 996, pipe: [1:7559341990685153152:8525]
2025-10-09T21:47:41.221042Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:41.229583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.229622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.229848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.229932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.230021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.230122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.230399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.230417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.230503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.230578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.230662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.230734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.230795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.230804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.230868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.230936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.233895Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:41.233965Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341990685153152:8525]
2025-10-09T21:47:41.234010Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:41.234017Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:41.231001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.231066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.231108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.234068Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 997 reason: error at write: Disconnected sessionId: src_id|496badfa-4f364fba-fad4886d-7fc7d5a9_0
2025-10-09T21:47:41.231111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.231180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.231242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.231303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.231365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.231767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.231771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.231843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.231898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.231958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.232014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.232054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.232057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.232123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.232179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.232238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.232299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.232359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.232362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.232425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.232484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.232545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.232599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.232643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.232647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.232715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.232767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.232826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.232888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.232942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.232945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.233016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.233075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.233133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.233194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.233232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.233235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.233305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.233370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.233437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.233504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.233546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.233549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.233631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341990685153152:8525], now have 1 active actors on pipe
2025-10-09T21:47:41.234286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 997 sessionId: src_id|496badfa-4f364fba-fad4886d-7fc7d5a9_0 is DEAD
2025-10-09T21:47:41.234451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|496badfa-4f364fba-fad4886d-7fc7d5a9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:41.234473Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|496badfa-4f364fba-fad4886d-7fc7d5a9_0] Write session will restart in 0.005120s
2025-10-09T21:47:41.234500Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|496badfa-4f364fba-fad4886d-7fc7d5a9_0] Write session: Do CDS request
2025-10-09T21:47:41.234507Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|496badfa-4f364fba-fad4886d-7fc7d5a9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:41.234526Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:41.240548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|496badfa-4f364fba-fad4886d-7fc7d5a9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:41.240627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:41.240650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 998
2025-10-09T21:47:41.240917Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 998 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:41.240954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 998 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:41.240967Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=998 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:41.240970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:41.241414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 998 sessionId:  describe result for acl check
2025-10-09T21:47:41.241458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:41.241460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:41.241462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:41.241468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341990685153189:9115] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:41.242068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341990685153189:9115] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:41.243387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:41.245236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:41.245327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341990685153189:9115] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:41.245465Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:41.245478Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:41.245492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:41.245549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:41.245595Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341990323332890:8207]
2025-10-09T21:47:41.246247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:41.246575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:41.246659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:41.247029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:41.247080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.247085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.247086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.247088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.247089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.247090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.247091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.247092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.247093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.247110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.247121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.247125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.247129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.247133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.247138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.247142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.247146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.247150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.247157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.247178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.247190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.247200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.247205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.247210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.247214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.247219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.247224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.247229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.247234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.247245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.247250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.247256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.247264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.247270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.247276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.247281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.247315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:41.248261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341990685153189:9115] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:41.248278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341990685153189:9115] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:41.248282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341990685153189:9115] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:41.248291Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 998 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:41.254329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:41.255525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:41.255542Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:41.255546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:41.255551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:41.255560Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 997 [2:7559341990323332890:8207]
2025-10-09T21:47:41.255577Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:41.255600Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:41.255604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:41.255608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:41.255613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:41.255633Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:41.255645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:41.255654Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:41.255728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.255737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.255741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.255744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:41.255909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:41.255919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.255934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.255946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.255966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:41.255974Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 997
2025-10-09T21:47:41.255985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 997
2025-10-09T21:47:41.255990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341990685153165:2432], now have 1 active actors on pipe
2025-10-09T21:47:41.256001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341990685153236:9115], now have 1 active actors on pipe
2025-10-09T21:47:41.256009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341990323332885:8207]
2025-10-09T21:47:41.256011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341990323332885:8207]
2025-10-09T21:47:41.256013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341990323332885:8207]
2025-10-09T21:47:41.256021Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:41.256029Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:41.256031Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:41.256038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:41.256043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:41.256054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:41.256058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:41.256080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.256085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.256132Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341990323332885:8207]
2025-10-09T21:47:41.256142Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:41.256148Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:41.256166Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ccc35186-6691223f-64ecc3e9-9793e69f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:41.256179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:41.256184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:41.256186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:41.256198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:41.256208Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:41.256247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:41.256456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.256482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.256512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:41.256629Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 998 partition: 0 MaxSeqNo: 25 sessionId: src_id|ccc35186-6691223f-64ecc3e9-9793e69f_0
2025-10-09T21:47:41.257041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|496badfa-4f364fba-fad4886d-7fc7d5a9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ccc35186-6691223f-64ecc3e9-9793e69f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:41.259651Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ccc35186-6691223f-64ecc3e9-9793e69f_0] Write session: try to update token
2025-10-09T21:47:41.259671Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ccc35186-6691223f-64ecc3e9-9793e69f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:41.271416Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:41.271436Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:41.271439Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:41.271441Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:41.271454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341990323332885:8207]
2025-10-09T21:47:41.271458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341990323332885:8207]
2025-10-09T21:47:41.271462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341990323332885:8207]
2025-10-09T21:47:41.271520Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:41.271523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:41.271531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.271534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:41.271559Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:41.271565Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:41.271676Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:41.275271Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 998 sessionId: src_id|ccc35186-6691223f-64ecc3e9-9793e69f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:41.277922Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:41.334018Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341990685153244:8525]
2025-10-09T21:47:41.334310Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 997, pipe: [1:7559341990685153244:8525]
2025-10-09T21:47:41.432660Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:41.432937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.432957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.433004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341990685153244:8525], now have 1 active actors on pipe
2025-10-09T21:47:41.433205Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:41.433243Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:41.433248Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341990685153244:8525]
2025-10-09T21:47:41.433271Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:41.433347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 998 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ccc35186-6691223f-64ecc3e9-9793e69f_0
2025-10-09T21:47:41.433550Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 998 sessionId: src_id|ccc35186-6691223f-64ecc3e9-9793e69f_0 is DEAD
2025-10-09T21:47:41.433718Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ccc35186-6691223f-64ecc3e9-9793e69f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:41.433727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ccc35186-6691223f-64ecc3e9-9793e69f_0] Write session will restart in 0.008085s
2025-10-09T21:47:41.433756Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ccc35186-6691223f-64ecc3e9-9793e69f_0] Write session: Do CDS request
2025-10-09T21:47:41.433762Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ccc35186-6691223f-64ecc3e9-9793e69f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:41.441441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:41.443155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:41.443378Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:41.443388Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:41.443400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:41.443453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:41.443498Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341990323332971:8211]
2025-10-09T21:47:41.443590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ccc35186-6691223f-64ecc3e9-9793e69f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:41.443710Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:41.443729Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 999
2025-10-09T21:47:41.443876Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 999 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:41.443908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 999 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:41.443919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=999 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:41.443923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:41.444097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:41.444342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 999 sessionId:  describe result for acl check
2025-10-09T21:47:41.444379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:41.444381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:41.444382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:41.444390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:41.444388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341990685153310:9126] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:41.444435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:41.444757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:41.444823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.444829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.444833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.444834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.444836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.444837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.444837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.444838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.444840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.444844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.444854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.444859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.444863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.444866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.444871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.444875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.444879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.444883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.444890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.444898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.444904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.444910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.444915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.444925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.444929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.444936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.444945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.444950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.444959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.444943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341990685153310:9126] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:41.444965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.444975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.444981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.444986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.444997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.445002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.445010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.445039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:41.447032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341990685153310:9126] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:41.451451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341990685153310:9126] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:41.451471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341990685153310:9126] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:41.451477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341990685153310:9126] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:41.451491Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 999 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:41.451793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:41.452995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:41.453022Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:41.453026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:41.453032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:41.453042Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 998 [2:7559341990323332971:8211]
2025-10-09T21:47:41.453061Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:41.453083Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:41.453092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:41.453096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:41.453101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:41.453119Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:41.453127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:41.453135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:41.453216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.453226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.453228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.453232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:41.453395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:41.453407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.453422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.453441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.453465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:41.453454Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 998
2025-10-09T21:47:41.453461Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 998
2025-10-09T21:47:41.453496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341990685153344:9126], now have 1 active actors on pipe
2025-10-09T21:47:41.453506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341990685153272:2432], now have 1 active actors on pipe
2025-10-09T21:47:41.453514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341990323332966:8211]
2025-10-09T21:47:41.453521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341990323332966:8211]
2025-10-09T21:47:41.453523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341990323332966:8211]
2025-10-09T21:47:41.453538Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:41.453547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:41.453550Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:41.453552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:41.453558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:41.453573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:41.453581Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:41.453600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.453609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.453663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341990323332966:8211]
2025-10-09T21:47:41.453675Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:41.453680Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:41.453697Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fc2ba68c-d228e1f8-61664d81-2e2e8e3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:41.453706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:41.453715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:41.453718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:41.453728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:41.453743Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:41.453791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:41.453938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.453948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.453962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:41.454063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 999 partition: 0 MaxSeqNo: 25 sessionId: src_id|fc2ba68c-d228e1f8-61664d81-2e2e8e3_0
2025-10-09T21:47:41.454457Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ccc35186-6691223f-64ecc3e9-9793e69f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fc2ba68c-d228e1f8-61664d81-2e2e8e3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:41.457012Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc2ba68c-d228e1f8-61664d81-2e2e8e3_0] Write session: try to update token
2025-10-09T21:47:41.457028Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc2ba68c-d228e1f8-61664d81-2e2e8e3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:41.465699Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:41.465716Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:41.465726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341990323332966:8211]
2025-10-09T21:47:41.465772Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:41.465776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:41.465966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:41.466158Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:41.466303Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:41.466443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:41.466738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:41.466786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:41.471081Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 999 sessionId: src_id|fc2ba68c-d228e1f8-61664d81-2e2e8e3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:41.473394Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:41.473595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.473615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.473654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:41.473731Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:41.478393Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:41.478412Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:41.478415Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:41.478417Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:41.478427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341990323332966:8211]
2025-10-09T21:47:41.478431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341990323332966:8211]
2025-10-09T21:47:41.478436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341990323332966:8211]
2025-10-09T21:47:41.478473Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:41.478477Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:41.478480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:41.478480Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:41.478486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.478489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:41.478626Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:41.533195Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341990685153351:8525]
2025-10-09T21:47:41.533488Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 998, pipe: [1:7559341990685153351:8525]
2025-10-09T21:47:41.633155Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:41.640796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.640826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.640998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.641065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.641131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.641201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.641253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.641266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.641337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.641406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.641471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.641544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.641590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.641597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.641657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.641718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.641782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.641864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.641911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.641920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.641993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.642076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.642142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.642209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.642252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.642260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.642327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.642391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.642457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.642525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.642574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.642582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.642645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.642713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.642774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.642839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.642890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.642902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.642969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.643033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.643104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.643168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.643211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.643219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.643280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.643346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.643411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.643479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.643526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.643533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.643601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.643662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.643732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.643802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.643850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.643858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.643923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:41.643987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:41.644058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:41.644124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:41.644176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:41.644184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:41.644252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341990685153351:8525], now have 1 active actors on pipe
2025-10-09T21:47:41.644470Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341990685153351:8525]
2025-10-09T21:47:41.644494Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:41.644498Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:41.644500Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:41.644527Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 999 reason: error at write: Disconnected sessionId: src_id|fc2ba68c-d228e1f8-61664d81-2e2e8e3_0
2025-10-09T21:47:41.644741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 999 sessionId: src_id|fc2ba68c-d228e1f8-61664d81-2e2e8e3_0 is DEAD
2025-10-09T21:47:41.644923Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc2ba68c-d228e1f8-61664d81-2e2e8e3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:41.644933Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc2ba68c-d228e1f8-61664d81-2e2e8e3_0] Write session will restart in 0.006314s
2025-10-09T21:47:41.644957Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc2ba68c-d228e1f8-61664d81-2e2e8e3_0] Write session: Do CDS request
2025-10-09T21:47:41.644963Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc2ba68c-d228e1f8-61664d81-2e2e8e3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:41.645269Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:41.652565Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc2ba68c-d228e1f8-61664d81-2e2e8e3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:41.652769Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:41.652797Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1000
2025-10-09T21:47:41.652990Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1000 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:41.653033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1000 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:41.653044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1000 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:41.653053Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:41.653321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:41.653547Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1000 sessionId:  describe result for acl check
2025-10-09T21:47:41.653593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:41.653601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:41.653602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:41.653609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341990685153390:9130] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:41.654176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341990685153390:9130] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:41.655016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:41.655204Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:41.655213Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:41.655225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:41.655280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:41.655320Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341990323333051:8215]
2025-10-09T21:47:41.655853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:41.656190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:41.656266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:41.656619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:41.656678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.656684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.656686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.656687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.656688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.656689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.656690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.656691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.656692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.656698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.656707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.656713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.656716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.656720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.656724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.656728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.656732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.656735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.656742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.656705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341990685153390:9130] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:41.656749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.656760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.656765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.656771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.656775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.656779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.656783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.656788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.656792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.656801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.656806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.656810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.656820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.656824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.656829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.656833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.656838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.656865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:41.661783Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341990685153390:9130] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:41.661802Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341990685153390:9130] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:41.661805Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341990685153390:9130] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:41.661815Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1000 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:41.663032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:41.664147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:41.664161Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:41.664163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:41.664167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:41.664173Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 999 [2:7559341990323333051:8215]
2025-10-09T21:47:41.664189Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:41.664205Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:41.664210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:41.664213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:41.664215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:41.664229Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:41.664236Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:41.664242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:41.664293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.664299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.664301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.664304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:41.664362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:41.664367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.664374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.664380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.664392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:41.664408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341990323333046:8215]
2025-10-09T21:47:41.664413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341990323333046:8215]
2025-10-09T21:47:41.664414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341990323333046:8215]
2025-10-09T21:47:41.664420Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:41.664430Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:41.664432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:41.664439Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:41.664442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:41.664451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341990685153354:2432], now have 1 active actors on pipe
2025-10-09T21:47:41.664451Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:41.664453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341990685153436:9130], now have 1 active actors on pipe
2025-10-09T21:47:41.664453Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:41.664481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341990323333046:8215]
2025-10-09T21:47:41.664489Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:41.664503Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 999
2025-10-09T21:47:41.664512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 999
2025-10-09T21:47:41.664508Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:41.664554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.664563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.664584Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9dd46d4a-195cbe3d-576414a7-67c17f30_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:41.664593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:41.664596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:41.664597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:41.664610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:41.664620Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:41.664635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:41.664785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.664800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.664823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:41.664926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1000 partition: 0 MaxSeqNo: 25 sessionId: src_id|9dd46d4a-195cbe3d-576414a7-67c17f30_0
2025-10-09T21:47:41.665302Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc2ba68c-d228e1f8-61664d81-2e2e8e3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9dd46d4a-195cbe3d-576414a7-67c17f30_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:41.667817Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9dd46d4a-195cbe3d-576414a7-67c17f30_0] Write session: try to update token
2025-10-09T21:47:41.667834Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9dd46d4a-195cbe3d-576414a7-67c17f30_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:41.678210Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:41.678230Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:41.678232Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:41.678234Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:41.678249Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341990323333046:8215]
2025-10-09T21:47:41.678254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341990323333046:8215]
2025-10-09T21:47:41.678257Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341990323333046:8215]
2025-10-09T21:47:41.678304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:41.678314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.678316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:41.678345Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:41.678362Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:41.678366Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:41.678454Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:41.681172Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1000 sessionId: src_id|9dd46d4a-195cbe3d-576414a7-67c17f30_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:41.683657Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:41.744614Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341990685153449:8525]
2025-10-09T21:47:41.744937Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 999, pipe: [1:7559341990685153449:8525]
2025-10-09T21:47:41.832722Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:41.833036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.833049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.833086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341990685153449:8525], now have 1 active actors on pipe
2025-10-09T21:47:41.833272Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:41.833310Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:41.833381Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341990685153449:8525]
2025-10-09T21:47:41.833384Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:41.833490Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1000 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9dd46d4a-195cbe3d-576414a7-67c17f30_0
2025-10-09T21:47:41.833692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1000 sessionId: src_id|9dd46d4a-195cbe3d-576414a7-67c17f30_0 is DEAD
2025-10-09T21:47:41.833868Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9dd46d4a-195cbe3d-576414a7-67c17f30_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:41.833880Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9dd46d4a-195cbe3d-576414a7-67c17f30_0] Write session will restart in 0.006216s
2025-10-09T21:47:41.833907Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9dd46d4a-195cbe3d-576414a7-67c17f30_0] Write session: Do CDS request
2025-10-09T21:47:41.833914Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9dd46d4a-195cbe3d-576414a7-67c17f30_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:41.841447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:41.841568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9dd46d4a-195cbe3d-576414a7-67c17f30_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:41.841677Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:41.841715Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1001
2025-10-09T21:47:41.841942Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1001 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:41.841979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1001 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:41.841991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1001 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:41.841997Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:41.842562Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1001 sessionId:  describe result for acl check
2025-10-09T21:47:41.842609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:41.842617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:41.842619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:41.842628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341990685153492:9137] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:41.843112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:41.843313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341990685153492:9137] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:41.843333Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:41.843344Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:41.843357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:41.843414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:41.843462Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341990323333126:8217]
2025-10-09T21:47:41.844098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:41.844382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:41.844416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:41.844720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:41.844774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.844780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.844781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.844783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.844784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.844785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.844786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.844788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.844789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.844794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.844805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.844810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.844815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.844818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.844822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.844826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.844830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.844834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.844840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.844849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:41.844855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.844861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:41.844864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.844867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:41.844869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.844873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:41.844875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.844878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:41.844881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.844884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:41.844887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.844890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.844892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.844895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.844898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.844901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.844918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:41.845514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341990685153492:9137] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:41.851144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341990685153492:9137] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:41.851166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341990685153492:9137] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:41.851170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341990685153492:9137] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:41.851179Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1001 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:41.851384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:41.852446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:41.852465Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:41.852467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:41.852476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:41.852483Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1000 [2:7559341990323333126:8217]
2025-10-09T21:47:41.852496Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:41.852512Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:41.852519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:41.852521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:41.852522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:41.852536Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:41.852543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:41.852549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:41.852594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.852600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.852603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.852605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:41.852677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:41.852685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.852696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:41.852708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:41.852731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:41.852753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341990685153531:9137], now have 1 active actors on pipe
2025-10-09T21:47:41.852760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341990685153454:2432], now have 1 active actors on pipe
2025-10-09T21:47:41.852766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341990323333121:8217]
2025-10-09T21:47:41.852767Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341990323333121:8217]
2025-10-09T21:47:41.852769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341990323333121:8217]
2025-10-09T21:47:41.852783Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:41.852802Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:41.852815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:41.852818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:41.852815Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1000
2025-10-09T21:47:41.852827Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1000
2025-10-09T21:47:41.852828Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:41.852846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341990323333121:8217]
2025-10-09T21:47:41.852848Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:41.852847Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:41.852851Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:41.852871Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:41.852879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.852882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.852905Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3acacce0-ef8d8112-b05674cb-db2f42e4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:41.852915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:41.852917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:41.852919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:41.852926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:41.852933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:41.852948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:41.853133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.853144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.853160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:41.853245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1001 partition: 0 MaxSeqNo: 25 sessionId: src_id|3acacce0-ef8d8112-b05674cb-db2f42e4_0
2025-10-09T21:47:41.853662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9dd46d4a-195cbe3d-576414a7-67c17f30_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3acacce0-ef8d8112-b05674cb-db2f42e4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:41.856229Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3acacce0-ef8d8112-b05674cb-db2f42e4_0] Write session: try to update token
2025-10-09T21:47:41.856248Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3acacce0-ef8d8112-b05674cb-db2f42e4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:41.870421Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1001 sessionId: src_id|3acacce0-ef8d8112-b05674cb-db2f42e4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:41.871650Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:41.871670Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:41.871682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341990323333121:8217]
2025-10-09T21:47:41.871751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:41.871774Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:41.871938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:41.872095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:41.872243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:41.872389Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:41.872593Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:41.872639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:41.873130Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:41.873360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:41.873376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:41.873411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:41.873516Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:41.879251Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:41.879268Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:41.879271Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:41.879273Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:41.879282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341990323333121:8217]
2025-10-09T21:47:41.879287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341990323333121:8217]
2025-10-09T21:47:41.879291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341990323333121:8217]
2025-10-09T21:47:41.879329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:41.879341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:41.879347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:41.879360Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:41.879376Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:41.879381Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:41.879500Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:41.933541Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341990685153539:8525]
2025-10-09T21:47:41.933854Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 1000, pipe: [1:7559341990685153539:8525]
2025-10-09T21:47:42.041402Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:42.049530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:42.049562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:42.049738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:42.049807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:42.049891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:42.049958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:42.050007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:42.050018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:42.050105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:42.050170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:42.050242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:42.050308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:42.050363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:42.050373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:42.050439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:42.050500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:42.050567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:42.050630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:42.050683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:42.050692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:42.050758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:42.050818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:42.050885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:42.050950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:42.050998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:42.051007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:42.051071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:42.051133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:42.051198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:42.051265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:42.051313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:42.051321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:42.051385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:42.051449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:42.051514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:42.051582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:42.051654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:42.051664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:42.051727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:42.051788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:42.051858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:42.051923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:42.051976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:42.051984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:42.052046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:42.052109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:42.052174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:42.052236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:42.052308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:42.052316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:42.052377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:42.052436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:42.052497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:42.052559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:42.052614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:42.052622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:42.052686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:42.052749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:42.052815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:42.052880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:42.052927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:42.052935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:42.053005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341990685153539:8525], now have 1 active actors on pipe
2025-10-09T21:47:42.053226Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341990685153539:8525]
2025-10-09T21:47:42.053255Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:42.053259Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:42.053270Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:42.053292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1001 reason: error at write: Disconnected sessionId: src_id|3acacce0-ef8d8112-b05674cb-db2f42e4_0
2025-10-09T21:47:42.053298Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:47:42.053405Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 10 consumer shared/user session shared/user_1_10_16382700389732983519_v1 is DEAD
2025-10-09T21:47:42.053421Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1001 sessionId: src_id|3acacce0-ef8d8112-b05674cb-db2f42e4_0 is DEAD
2025-10-09T21:47:42.053781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3acacce0-ef8d8112-b05674cb-db2f42e4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:42.053786Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:47:42.053794Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3acacce0-ef8d8112-b05674cb-db2f42e4_0] Write session will restart in 0.007345s
2025-10-09T21:47:42.053829Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3acacce0-ef8d8112-b05674cb-db2f42e4_0] Write session: Do CDS request
2025-10-09T21:47:42.053836Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3acacce0-ef8d8112-b05674cb-db2f42e4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:42.053814Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341887605914025:8522] disconnected.
2025-10-09T21:47:42.053835Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341887605914025:8522] disconnected; active server actors: 1
2025-10-09T21:47:42.053838Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341887605914025:8522] client user disconnected session shared/user_1_10_16382700389732983519_v1
2025-10-09T21:47:42.053877Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:47:42.053892Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:47:42.053899Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.005729s
2025-10-09T21:47:42.054203Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:42.060406Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:47:42.060651Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:47:42.060667Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 11
2025-10-09T21:47:42.060877Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 11 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:47:42.060943Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:47:42.061029Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 auth for : user
2025-10-09T21:47:42.061203Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 Handle describe topics response
2025-10-09T21:47:42.061246Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 auth is DEAD
2025-10-09T21:47:42.061255Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:47:42.061503Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:47:42.061627Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_11_8138003183284567198_v1
2025-10-09T21:47:42.061614Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341994980120875:9141] connected; active server actors: 1
2025-10-09T21:47:42.061633Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559341994980120875:9141] session shared/user_1_11_8138003183284567198_v1
2025-10-09T21:47:42.061644Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:47:42.061655Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:47:42.061661Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_11_8138003183284567198_v1" (Sender=[1:7559341994980120863:9141], Pipe=[1:7559341994980120875:9141], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:47:42.061672Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:47:42.061685Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:47:42.061699Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_11_8138003183284567198_v1" (Sender=[1:7559341994980120863:9141], Pipe=[1:7559341994980120875:9141], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:47:42.061715Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_11_8138003183284567198_v1" sender [1:7559341994980120863:9141] lock partition 0 for ReadingSession "shared/user_1_11_8138003183284567198_v1" (Sender=[1:7559341994980120863:9141], Pipe=[1:7559341994980120875:9141], Partitions=[], ActiveFamilyCount=1) generation 1 step 11
2025-10-09T21:47:42.061734Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:47:42.061744Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000051s
2025-10-09T21:47:42.061960Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 11 Session: "shared/user_1_11_8138003183284567198_v1" ClientId: "user" PipeClient { RawX1: 7559341994980120875 RawX2: 4503603922346933 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:47:42.061990Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:47:42.062453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3acacce0-ef8d8112-b05674cb-db2f42e4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:42.062466Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:42.062478Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1002
2025-10-09T21:47:42.062680Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1002 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:42.062726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1002 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:42.062748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1002 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:42.062756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:42.062844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:42.063145Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1002 sessionId:  describe result for acl check
2025-10-09T21:47:42.063180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:42.063187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:42.063189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:42.063195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341994980120884:9145] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:42.063738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341994980120884:9145] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:42.064146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:42.064338Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:42.064347Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:42.064359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:42.064412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:42.064449Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341994618300499:8219]
2025-10-09T21:47:42.065030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:42.065323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:42.065365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:42.065672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:42.065723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.065730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.065731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.065732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.065733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.065734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.065735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.065737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.065738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.065743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.065754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.065758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.065763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.065766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.065771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.065775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.065779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.065783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.065789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.065797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.065809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.065829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.065833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.065838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.065843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.065848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.065853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.065858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.065861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.065872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.065877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.065883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.065887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.065893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.065899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.065904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.065933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:42.066726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341994980120884:9145] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:42.071863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341994980120884:9145] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:42.071878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341994980120884:9145] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:42.071881Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341994980120884:9145] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:42.071888Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1002 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:42.072198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:42.073263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:42.073280Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:42.073283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:42.073287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:42.073295Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1001 [2:7559341994618300499:8219]
2025-10-09T21:47:42.073308Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:42.073328Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:42.073331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:42.073335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:42.073337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:42.073353Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:42.073359Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:42.073367Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:42.073416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.073422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.073423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.073425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:42.073478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:42.073485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.073492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.073498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.073508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:42.073525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341994980120879:9144], now have 1 active actors on pipe
2025-10-09T21:47:42.073533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341994980120923:9145], now have 1 active actors on pipe
2025-10-09T21:47:42.073540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341994618300494:8219]
2025-10-09T21:47:42.073543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341994618300494:8219]
2025-10-09T21:47:42.073545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341994618300494:8219]
2025-10-09T21:47:42.073553Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:42.073566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:42.073572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:42.073576Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:42.073578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:42.073584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:42.073587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:42.073593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341994980120838:2432], now have 1 active actors on pipe
2025-10-09T21:47:42.073612Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1001
2025-10-09T21:47:42.073626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341994618300494:8219]
2025-10-09T21:47:42.073612Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1001, pipe: [1:7559341994980120879:9144]
2025-10-09T21:47:42.073634Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:42.073646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:42.073657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:42.073662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:42.073652Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1001
2025-10-09T21:47:42.073691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:42.073698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:42.073705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_11_8138003183284567198_v1 on pipe: [1:7559341994980120879:9144]
2025-10-09T21:47:42.073743Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4217a2f4-efff1e81-734be25d-cce73be5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:42.073757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:42.073761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:42.073759Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_11_8138003183284567198_v1:1 with generation 1001
2025-10-09T21:47:42.073763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:42.073772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:42.073783Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:42.073803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:42.073808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:42.073809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:42.073832Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_11_8138003183284567198_v1
2025-10-09T21:47:42.073837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:42.073858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:42.073871Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:47:42.073957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:42.073965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:42.073976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:42.074084Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1002 partition: 0 MaxSeqNo: 25 sessionId: src_id|4217a2f4-efff1e81-734be25d-cce73be5_0
2025-10-09T21:47:42.074450Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3acacce0-ef8d8112-b05674cb-db2f42e4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4217a2f4-efff1e81-734be25d-cce73be5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:42.076978Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4217a2f4-efff1e81-734be25d-cce73be5_0] Write session: try to update token
2025-10-09T21:47:42.076998Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4217a2f4-efff1e81-734be25d-cce73be5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:42.078336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:47:42.078369Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:47:42.078381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:42.078383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:42.078385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:42.078393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:47:42.078406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:47:42.078552Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046462073 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:47:42.078576Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:47:42.078617Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 sending to client partition status
2025-10-09T21:47:42.087977Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:42.087995Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:42.087997Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:42.087999Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:42.088011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341994618300494:8219]
2025-10-09T21:47:42.088016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341994618300494:8219]
2025-10-09T21:47:42.088020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341994618300494:8219]
2025-10-09T21:47:42.088053Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:42.088059Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:42.088062Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:42.088065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:42.088071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.088074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:42.088203Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:42.089837Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1002 sessionId: src_id|4217a2f4-efff1e81-734be25d-cce73be5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:42.092296Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:42.244544Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:42.244822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:42.244837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:42.245080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_11_8138003183284567198_v1
2025-10-09T21:47:42.245094Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:42.245108Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:42.245145Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341994980120879:9144]
2025-10-09T21:47:42.245159Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:42.245231Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1002 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4217a2f4-efff1e81-734be25d-cce73be5_0
2025-10-09T21:47:42.245377Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_11_8138003183284567198_v1
2025-10-09T21:47:42.245409Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1002 sessionId: src_id|4217a2f4-efff1e81-734be25d-cce73be5_0 is DEAD
2025-10-09T21:47:42.245586Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4217a2f4-efff1e81-734be25d-cce73be5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:42.245597Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4217a2f4-efff1e81-734be25d-cce73be5_0] Write session will restart in 0.006832s
2025-10-09T21:47:42.245624Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4217a2f4-efff1e81-734be25d-cce73be5_0] Write session: Do CDS request
2025-10-09T21:47:42.245638Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4217a2f4-efff1e81-734be25d-cce73be5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:42.253627Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4217a2f4-efff1e81-734be25d-cce73be5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:42.253748Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:42.253770Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1003
2025-10-09T21:47:42.254001Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1003 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:42.254094Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1003 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:42.254118Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1003 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:42.254128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:42.254603Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1003 sessionId:  describe result for acl check
2025-10-09T21:47:42.254645Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:42.254653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:42.254655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:42.254661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341994980120956:9150] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:42.255258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341994980120956:9150] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:42.258309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341994980120956:9150] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:42.259536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:42.261211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:42.261417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341994980120956:9150] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:42.261432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341994980120956:9150] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:42.261437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341994980120956:9150] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:42.261416Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:42.261445Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1003 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:42.261427Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:42.261440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:42.261492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:42.261547Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341994618300609:8227]
2025-10-09T21:47:42.262140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:42.262610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:42.262667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:42.262988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:42.263054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.263060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.263061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.263062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.263063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.263065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.263068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.263069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.263070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.263074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.263084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.263088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.263092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.263096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.263100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.263103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.263107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.263111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.263117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.263125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.263131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.263137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.263142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.263147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.263151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.263156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.263160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.263165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.263169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.263174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.263179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.263184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.263188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.263192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.263195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.263200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.263228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:42.270005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:42.272983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:42.273004Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:42.273006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:42.273011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:42.273018Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1002 [2:7559341994618300609:8227]
2025-10-09T21:47:42.273032Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:42.273052Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:42.273062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:42.273066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:42.273067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:42.273087Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:42.273099Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:42.273109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:42.273189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.273199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.273201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.273204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:42.273279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:42.273287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.273301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.273312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.273329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:42.273362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341994980121016:9150], now have 1 active actors on pipe
2025-10-09T21:47:42.273371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341994980120941:2432], now have 1 active actors on pipe
2025-10-09T21:47:42.273380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341994618300604:8227]
2025-10-09T21:47:42.273383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341994618300604:8227]
2025-10-09T21:47:42.273385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341994618300604:8227]
2025-10-09T21:47:42.273403Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:42.273421Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:42.273444Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:42.273447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:42.273463Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:42.273468Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:42.273471Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:42.273481Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1002
2025-10-09T21:47:42.273512Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1002
2025-10-09T21:47:42.273489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341994618300604:8227]
2025-10-09T21:47:42.273502Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:42.273515Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:42.273543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:42.273548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:42.273593Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f7cfd829-e89ab7b9-cceaa8c8-fe0f6c0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:42.273610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:42.273615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:42.273619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:42.273631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:42.273642Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:42.273672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:42.273918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:42.273943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:42.273967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:42.274153Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1003 partition: 0 MaxSeqNo: 25 sessionId: src_id|f7cfd829-e89ab7b9-cceaa8c8-fe0f6c0_0
2025-10-09T21:47:42.274540Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4217a2f4-efff1e81-734be25d-cce73be5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f7cfd829-e89ab7b9-cceaa8c8-fe0f6c0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:42.282552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7cfd829-e89ab7b9-cceaa8c8-fe0f6c0_0] Write session: try to update token
2025-10-09T21:47:42.282576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7cfd829-e89ab7b9-cceaa8c8-fe0f6c0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:42.288335Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:42.288355Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:42.288357Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:42.288359Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:42.288369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341994618300604:8227]
2025-10-09T21:47:42.288374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341994618300604:8227]
2025-10-09T21:47:42.288379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341994618300604:8227]
2025-10-09T21:47:42.288423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:42.288432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.288435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:42.288436Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:42.288442Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:42.288447Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:42.288868Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:42.302884Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1003 sessionId: src_id|f7cfd829-e89ab7b9-cceaa8c8-fe0f6c0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:42.305343Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:42.345176Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341994980121028:9144]
2025-10-09T21:47:42.345483Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 1002, pipe: [1:7559341994980121028:9144]
2025-10-09T21:47:42.448533Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:42.448811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:42.448826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:42.448847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341994980121028:9144], now have 1 active actors on pipe
2025-10-09T21:47:42.449076Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:42.449098Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:42.449115Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:42.449191Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1003 reason: error at write: Disconnected sessionId: src_id|f7cfd829-e89ab7b9-cceaa8c8-fe0f6c0_0
2025-10-09T21:47:42.449407Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1003 sessionId: src_id|f7cfd829-e89ab7b9-cceaa8c8-fe0f6c0_0 is DEAD
2025-10-09T21:47:42.449578Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7cfd829-e89ab7b9-cceaa8c8-fe0f6c0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:42.449587Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7cfd829-e89ab7b9-cceaa8c8-fe0f6c0_0] Write session will restart in 0.008682s
2025-10-09T21:47:42.449604Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7cfd829-e89ab7b9-cceaa8c8-fe0f6c0_0] Write session: Do CDS request
2025-10-09T21:47:42.449608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7cfd829-e89ab7b9-cceaa8c8-fe0f6c0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:42.449601Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:42.449648Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341994980121028:9144]
2025-10-09T21:47:42.458151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:42.459515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7cfd829-e89ab7b9-cceaa8c8-fe0f6c0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:42.459599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:42.459617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1004
2025-10-09T21:47:42.459683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:42.459792Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1004 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:42.459842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1004 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:42.459854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1004 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:42.459857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:42.459889Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:42.459903Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:42.459915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:42.459971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:42.460010Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341994618300686:8233]
2025-10-09T21:47:42.460253Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1004 sessionId:  describe result for acl check
2025-10-09T21:47:42.460287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:42.460295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:42.460297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:42.460302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341994980121084:9160] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:42.460575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:42.460880Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341994980121084:9160] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:42.460941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:42.460985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:42.461302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:42.461356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.461359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.461360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.461361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.461362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.461363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.461364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.461366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.461367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.461372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.461382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.461385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.461390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.461394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.461398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.461402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.461406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.461410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.461416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.461426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.461432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.461437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.461441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.461446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.461450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.461455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.461459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.461464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.461469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.461474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.461479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.461484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.461493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.461499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.461503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.461514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.461541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:42.462839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341994980121084:9160] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:42.468148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:42.468100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341994980121084:9160] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:42.468119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341994980121084:9160] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:42.468122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341994980121084:9160] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:42.468130Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1004 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:42.469429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:42.469447Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:42.469449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:42.469454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:42.469461Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1003 [2:7559341994618300686:8233]
2025-10-09T21:47:42.469478Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:42.469495Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:42.469503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:42.469507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:42.469513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:42.469532Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:42.469542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:42.469554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:42.469620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.469629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.469631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.469634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:42.469707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:42.469716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.469729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.469745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.469766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:42.469794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341994980121130:9160], now have 1 active actors on pipe
2025-10-09T21:47:42.469805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341994980121043:2432], now have 1 active actors on pipe
2025-10-09T21:47:42.469813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341994618300681:8233]
2025-10-09T21:47:42.469815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341994618300681:8233]
2025-10-09T21:47:42.469829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341994618300681:8233]
2025-10-09T21:47:42.469842Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:42.469861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:42.469838Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1003
2025-10-09T21:47:42.469846Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1003
2025-10-09T21:47:42.469880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:42.469885Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:42.469889Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:42.469891Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:42.469891Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:42.469941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:42.469946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:42.469963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341994618300681:8233]
2025-10-09T21:47:42.469972Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:42.469988Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:42.470003Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d6924d9c-8e4eb08a-b9686737-80ddd174_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:42.470012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:42.470021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:42.470023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:42.470055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:42.470071Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:42.470145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:42.470325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:42.470335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:42.470350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:42.470439Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1004 partition: 0 MaxSeqNo: 25 sessionId: src_id|d6924d9c-8e4eb08a-b9686737-80ddd174_0
2025-10-09T21:47:42.470803Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7cfd829-e89ab7b9-cceaa8c8-fe0f6c0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d6924d9c-8e4eb08a-b9686737-80ddd174_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:42.474551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d6924d9c-8e4eb08a-b9686737-80ddd174_0] Write session: try to update token
2025-10-09T21:47:42.474578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d6924d9c-8e4eb08a-b9686737-80ddd174_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:42.484723Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:42.484745Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:42.484747Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:42.484749Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:42.484759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341994618300681:8233]
2025-10-09T21:47:42.484764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341994618300681:8233]
2025-10-09T21:47:42.484768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341994618300681:8233]
2025-10-09T21:47:42.484814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:42.484819Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:42.484821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.484824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:42.484825Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:42.484829Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:42.484957Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:42.489669Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1004 sessionId: src_id|d6924d9c-8e4eb08a-b9686737-80ddd174_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:42.492004Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:42.550020Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341994980121140:9144]
2025-10-09T21:47:42.550346Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 1003, pipe: [1:7559341994980121140:9144]
2025-10-09T21:47:42.645425Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:42.645731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:42.645750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:42.645799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341994980121140:9144], now have 1 active actors on pipe
2025-10-09T21:47:42.646087Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341994980121140:9144]
2025-10-09T21:47:42.646121Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:42.646019Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:42.646480Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:42.646564Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1004 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d6924d9c-8e4eb08a-b9686737-80ddd174_0
2025-10-09T21:47:42.646754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1004 sessionId: src_id|d6924d9c-8e4eb08a-b9686737-80ddd174_0 is DEAD
2025-10-09T21:47:42.646897Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6924d9c-8e4eb08a-b9686737-80ddd174_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:42.646908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6924d9c-8e4eb08a-b9686737-80ddd174_0] Write session will restart in 0.008018s
2025-10-09T21:47:42.646946Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6924d9c-8e4eb08a-b9686737-80ddd174_0] Write session: Do CDS request
2025-10-09T21:47:42.646952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6924d9c-8e4eb08a-b9686737-80ddd174_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:42.656586Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d6924d9c-8e4eb08a-b9686737-80ddd174_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:42.656575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:42.656890Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:42.656910Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1005
2025-10-09T21:47:42.657033Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1005 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:42.657071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1005 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:42.657077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1005 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:42.657081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:42.657579Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1005 sessionId:  describe result for acl check
2025-10-09T21:47:42.657626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:42.657632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:42.657634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:42.657640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341994980121184:9165] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:42.658213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:42.658222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341994980121184:9165] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:42.658410Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:42.658422Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:42.658433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:42.658486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:42.658537Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341994618300779:8239]
2025-10-09T21:47:42.659135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:42.659536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:42.659598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:42.659986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:42.660054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.660061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.660062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.660063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.660066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.660068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.660069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.660070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.660071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.660076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.660086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.660090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.660094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.660098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.660102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.660106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.660110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.660114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.660120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.660128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.660135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.660141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.660152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.660158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.660162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.660168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.660173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.660182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.660187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.660192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.660201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.660206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.660211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.660216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.660220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.660227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.660255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:42.661318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341994980121184:9165] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:42.666295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341994980121184:9165] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:42.666316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341994980121184:9165] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:42.666320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341994980121184:9165] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:42.666330Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1005 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:42.667579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:42.668758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:42.668773Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:42.668776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:42.668781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:42.668789Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1004 [2:7559341994618300779:8239]
2025-10-09T21:47:42.668805Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:42.668823Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:42.668827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:42.668830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:42.668834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:42.668853Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:42.668862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:42.668872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:42.668944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.668953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.668955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.668957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:42.669035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:42.669044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.669055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.669067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.669084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:42.669118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341994980121142:2432], now have 1 active actors on pipe
2025-10-09T21:47:42.669122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341994980121223:9165], now have 1 active actors on pipe
2025-10-09T21:47:42.669130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341994618300773:8239]
2025-10-09T21:47:42.669132Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341994618300773:8239]
2025-10-09T21:47:42.669134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341994618300773:8239]
2025-10-09T21:47:42.669149Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:42.669163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:42.669177Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:42.669186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:42.669225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341994618300773:8239]
2025-10-09T21:47:42.669234Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:42.669308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:42.669325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:42.669329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:42.669338Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:42.669968Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1004
2025-10-09T21:47:42.670019Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1004
2025-10-09T21:47:42.670363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:42.670380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:42.670413Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ef189743-47a20cb0-bb635569-9b5c4a94_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:42.670430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:42.670435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:42.670451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:42.670465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:42.670488Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:42.670513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:42.671691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:42.671704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:42.671727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:42.671907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1005 partition: 0 MaxSeqNo: 25 sessionId: src_id|ef189743-47a20cb0-bb635569-9b5c4a94_0
2025-10-09T21:47:42.672376Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6924d9c-8e4eb08a-b9686737-80ddd174_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ef189743-47a20cb0-bb635569-9b5c4a94_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:42.675058Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef189743-47a20cb0-bb635569-9b5c4a94_0] Write session: try to update token
2025-10-09T21:47:42.675080Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef189743-47a20cb0-bb635569-9b5c4a94_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:42.684422Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:42.684445Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:42.684447Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:42.684449Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:42.684460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341994618300773:8239]
2025-10-09T21:47:42.684469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341994618300773:8239]
2025-10-09T21:47:42.684472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341994618300773:8239]
2025-10-09T21:47:42.684517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:42.684523Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:42.684530Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:42.684531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.684534Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:42.684534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:42.684702Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:42.689763Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1005 sessionId: src_id|ef189743-47a20cb0-bb635569-9b5c4a94_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:42.692321Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:42.746131Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341994980121231:9144]
2025-10-09T21:47:42.746469Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 1004, pipe: [1:7559341994980121231:9144]
2025-10-09T21:47:42.847211Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:42.847657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:42.847679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:42.847728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341994980121231:9144], now have 1 active actors on pipe
2025-10-09T21:47:42.847972Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:42.848002Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:42.848019Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:42.848083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1005 reason: error at write: Disconnected sessionId: src_id|ef189743-47a20cb0-bb635569-9b5c4a94_0
2025-10-09T21:47:42.848280Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1005 sessionId: src_id|ef189743-47a20cb0-bb635569-9b5c4a94_0 is DEAD
2025-10-09T21:47:42.848417Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:42.848426Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341994980121231:9144]
2025-10-09T21:47:42.850274Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef189743-47a20cb0-bb635569-9b5c4a94_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:42.850287Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef189743-47a20cb0-bb635569-9b5c4a94_0] Write session will restart in 0.009918s
2025-10-09T21:47:42.850319Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef189743-47a20cb0-bb635569-9b5c4a94_0] Write session: Do CDS request
2025-10-09T21:47:42.850327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef189743-47a20cb0-bb635569-9b5c4a94_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:42.859087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:42.860795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:42.861114Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:42.861133Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:42.861148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:42.861227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:42.861273Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341994618300859:8241]
2025-10-09T21:47:42.861643Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef189743-47a20cb0-bb635569-9b5c4a94_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:42.861705Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:42.861733Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1006
2025-10-09T21:47:42.861882Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1006 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:42.861920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1006 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:42.861952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:42.861933Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1006 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:42.861937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:42.862313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:42.862369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:42.862425Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1006 sessionId:  describe result for acl check
2025-10-09T21:47:42.862462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:42.862469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:42.862471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:42.862476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341994980121282:9170] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:42.862706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:42.862762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.862770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.862772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.862773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.862774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.862775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.862776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.862777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.862778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.862784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.862793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.862797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.862801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.862805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.862808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.862812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.862816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.862819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.862826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.862836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:42.862842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.862848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:42.862852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.862857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:42.862862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.862868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:42.862872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.862877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:42.862882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.862887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:42.862893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.862898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.862903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.862908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.862912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.862917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.862957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:42.863025Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341994980121282:9170] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:42.865413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341994980121282:9170] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:42.869164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341994980121282:9170] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:42.869180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341994980121282:9170] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:42.869184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341994980121282:9170] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:42.869195Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1006 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:42.869524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:42.870637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:42.870653Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:42.870655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:42.870659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:42.870666Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1005 [2:7559341994618300859:8241]
2025-10-09T21:47:42.870681Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:42.870699Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:42.870709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:42.870713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:42.870715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:42.870733Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:42.870742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:42.870749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:42.870809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.870816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.870818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.870820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:42.870885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:42.870891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.870902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:42.870913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:42.870932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:42.870952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341994980121236:2432], now have 1 active actors on pipe
2025-10-09T21:47:42.870957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341994980121318:9170], now have 1 active actors on pipe
2025-10-09T21:47:42.870965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341994618300853:8241]
2025-10-09T21:47:42.870967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341994618300853:8241]
2025-10-09T21:47:42.870969Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341994618300853:8241]
2025-10-09T21:47:42.870984Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:42.871003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:42.871004Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:42.871008Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:42.871011Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:42.871021Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:42.870997Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1005
2025-10-09T21:47:42.871025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:42.871015Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1005
2025-10-09T21:47:42.871059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341994618300853:8241]
2025-10-09T21:47:42.871068Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:42.871071Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:42.871084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:42.871088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:42.871140Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5ea15c87-11104656-5f1da44f-cf44c9b9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:42.871154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:42.871159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:42.871162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:42.871171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:42.871181Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:42.871200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:42.871348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:42.871355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:42.871369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:42.871496Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1006 partition: 0 MaxSeqNo: 25 sessionId: src_id|5ea15c87-11104656-5f1da44f-cf44c9b9_0
2025-10-09T21:47:42.871856Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef189743-47a20cb0-bb635569-9b5c4a94_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5ea15c87-11104656-5f1da44f-cf44c9b9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:42.874296Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ea15c87-11104656-5f1da44f-cf44c9b9_0] Write session: try to update token
2025-10-09T21:47:42.874317Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ea15c87-11104656-5f1da44f-cf44c9b9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:42.885763Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:42.885786Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:42.885789Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:42.885792Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:42.885802Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341994618300853:8241]
2025-10-09T21:47:42.885808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341994618300853:8241]
2025-10-09T21:47:42.885812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341994618300853:8241]
2025-10-09T21:47:42.885873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:42.885881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:42.885884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:42.885883Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:42.885889Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:42.885903Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:42.886026Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:42.888202Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1006 sessionId: src_id|5ea15c87-11104656-5f1da44f-cf44c9b9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:42.890620Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:42.947848Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341994980121325:9144]
2025-10-09T21:47:42.948140Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 1005, pipe: [1:7559341994980121325:9144]
2025-10-09T21:47:43.049861Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:43.050173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:43.050191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:43.050236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341994980121325:9144], now have 1 active actors on pipe
2025-10-09T21:47:43.050479Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:43.050489Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341994980121325:9144]
2025-10-09T21:47:43.050514Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:43.050517Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:43.050593Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1006 reason: error at write: Disconnected sessionId: src_id|5ea15c87-11104656-5f1da44f-cf44c9b9_0
2025-10-09T21:47:43.050658Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:43.050782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1006 sessionId: src_id|5ea15c87-11104656-5f1da44f-cf44c9b9_0 is DEAD
2025-10-09T21:47:43.050941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ea15c87-11104656-5f1da44f-cf44c9b9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:43.050953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ea15c87-11104656-5f1da44f-cf44c9b9_0] Write session will restart in 0.008449s
2025-10-09T21:47:43.050981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ea15c87-11104656-5f1da44f-cf44c9b9_0] Write session: Do CDS request
2025-10-09T21:47:43.050987Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ea15c87-11104656-5f1da44f-cf44c9b9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:43.060611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ea15c87-11104656-5f1da44f-cf44c9b9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:43.060734Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:43.060756Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1007
2025-10-09T21:47:43.060990Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1007 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:43.061033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1007 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:43.061045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1007 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:43.061049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:43.061504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1007 sessionId:  describe result for acl check
2025-10-09T21:47:43.061547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:43.061555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:43.061556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:43.061563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341999275088666:9174] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:43.062244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341999275088666:9174] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:43.062447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:43.064305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:43.064540Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:43.064552Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:43.064565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:43.064623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:43.064674Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341998913268236:8243]
2025-10-09T21:47:43.064764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341999275088666:9174] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:43.065281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:43.065615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:43.065671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:43.066054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:43.066130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.066138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.066139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.066140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.066141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.066143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.066144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.066145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.066146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.066162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.066173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.066176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.066180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.066183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.066187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.066190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.066193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.066196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.066203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.066211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.066218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.066223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.066228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.066233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.066238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.066243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.066250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.066256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.066260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.066265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.066271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.066276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.066280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.066285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.066290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.066295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.066324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:43.071913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341999275088666:9174] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:43.071935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341999275088666:9174] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:43.071939Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341999275088666:9174] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:43.071949Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1007 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:43.073157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:43.074515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:43.074532Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:43.074535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:43.074539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:43.074548Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1006 [2:7559341998913268236:8243]
2025-10-09T21:47:43.074563Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:43.074583Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:43.074587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:43.074590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:43.074592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:43.074612Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:43.074619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:43.074627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:43.074697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.074706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.074708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.074711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:43.074785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:43.074795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.074809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.074826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.074847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:43.074875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341999275088707:9174], now have 1 active actors on pipe
2025-10-09T21:47:43.074884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341999275088624:2432], now have 1 active actors on pipe
2025-10-09T21:47:43.074892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341998913268230:8243]
2025-10-09T21:47:43.074898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341998913268230:8243]
2025-10-09T21:47:43.074902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341998913268230:8243]
2025-10-09T21:47:43.074919Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:43.074937Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:43.074943Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:43.074955Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:43.074939Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1006
2025-10-09T21:47:43.074957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:43.074958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:43.074960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:43.074960Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1006
2025-10-09T21:47:43.074999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341998913268230:8243]
2025-10-09T21:47:43.075009Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:43.075011Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:43.075047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:43.075057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:43.075113Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fb268450-ee0a6b12-fa6e2dfd-22e397ef_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:43.075129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:43.075136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:43.075138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:43.075150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:43.075164Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:43.075192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:43.075395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:43.075408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:43.075432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:43.075583Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1007 partition: 0 MaxSeqNo: 25 sessionId: src_id|fb268450-ee0a6b12-fa6e2dfd-22e397ef_0
2025-10-09T21:47:43.076009Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ea15c87-11104656-5f1da44f-cf44c9b9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fb268450-ee0a6b12-fa6e2dfd-22e397ef_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:43.078677Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fb268450-ee0a6b12-fa6e2dfd-22e397ef_0] Write session: try to update token
2025-10-09T21:47:43.078696Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fb268450-ee0a6b12-fa6e2dfd-22e397ef_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:43.090198Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:43.090219Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:43.090221Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:43.090223Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:43.090234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341998913268230:8243]
2025-10-09T21:47:43.090242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341998913268230:8243]
2025-10-09T21:47:43.090245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341998913268230:8243]
2025-10-09T21:47:43.090296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:43.090306Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:43.090309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.090312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:43.090312Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:43.090316Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:43.090460Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:43.092781Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1007 sessionId: src_id|fb268450-ee0a6b12-fa6e2dfd-22e397ef_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:43.095303Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:43.150600Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341999275088717:9144]
2025-10-09T21:47:43.150862Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 1006, pipe: [1:7559341999275088717:9144]
2025-10-09T21:47:43.253523Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:43.253933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:43.253946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:43.253994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341999275088717:9144], now have 1 active actors on pipe
2025-10-09T21:47:43.254439Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:43.254472Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341999275088717:9144]
2025-10-09T21:47:43.254497Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:43.254500Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:43.254979Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:43.255432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1007 reason: error at write: Disconnected sessionId: src_id|fb268450-ee0a6b12-fa6e2dfd-22e397ef_0
2025-10-09T21:47:43.255625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1007 sessionId: src_id|fb268450-ee0a6b12-fa6e2dfd-22e397ef_0 is DEAD
2025-10-09T21:47:43.256016Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb268450-ee0a6b12-fa6e2dfd-22e397ef_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:43.256023Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb268450-ee0a6b12-fa6e2dfd-22e397ef_0] Write session will restart in 0.006337s
2025-10-09T21:47:43.256051Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb268450-ee0a6b12-fa6e2dfd-22e397ef_0] Write session: Do CDS request
2025-10-09T21:47:43.256057Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb268450-ee0a6b12-fa6e2dfd-22e397ef_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:43.263584Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fb268450-ee0a6b12-fa6e2dfd-22e397ef_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:43.263708Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:43.263738Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1008
2025-10-09T21:47:43.264003Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1008 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:43.264040Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1008 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:43.264051Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1008 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:43.264055Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:43.264483Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1008 sessionId:  describe result for acl check
2025-10-09T21:47:43.264524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:43.264532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:43.264534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:43.264540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341999275088740:9179] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:43.265099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341999275088740:9179] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:43.267719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341999275088740:9179] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:43.268617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:43.270364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:43.270708Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341999275088740:9179] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:43.270726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341999275088740:9179] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:43.270729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341999275088740:9179] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:43.270738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1008 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:43.270586Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:43.270597Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:43.270611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:43.270666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:43.270757Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341998913268331:8247]
2025-10-09T21:47:43.271286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:43.271831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:43.271886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:43.272232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:43.272308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.272315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.272316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.272317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.272318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.272319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.272320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.272322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.272323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.272329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.272337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.272341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.272344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.272347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.272350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.272354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.272357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.272360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.272368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.272375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.272380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.272385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.272389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.272393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.272398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.272402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.272406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.272411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.272415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.272420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.272424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.272428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.272432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.272437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.272441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.272446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.272473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:43.279428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:43.280514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:43.280528Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:43.280531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:43.280535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:43.280542Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1007 [2:7559341998913268331:8247]
2025-10-09T21:47:43.280569Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:43.280595Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:43.280598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:43.280601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:43.280604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:43.280623Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:43.280629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:43.280636Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:43.280706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.280712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.280714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.280716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:43.280967Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1007
2025-10-09T21:47:43.281007Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1007
2025-10-09T21:47:43.280785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:43.280792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.280803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.280814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.280829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:43.280849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341999275088725:2432], now have 1 active actors on pipe
2025-10-09T21:47:43.280851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341999275088805:9179], now have 1 active actors on pipe
2025-10-09T21:47:43.280859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341998913268326:8247]
2025-10-09T21:47:43.280861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341998913268326:8247]
2025-10-09T21:47:43.280863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341998913268326:8247]
2025-10-09T21:47:43.280875Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:43.280890Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:43.280892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:43.280893Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:43.280896Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:43.280906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:43.280909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:43.280945Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341998913268326:8247]
2025-10-09T21:47:43.280952Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:43.280958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:43.281203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:43.281215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:43.281244Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c8069ab0-bbcfbeb5-5df371f0-43fb8026_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:43.281256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:43.281264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:43.281267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:43.281277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:43.281292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:43.281314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:43.281520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:43.281526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:43.281539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:43.283038Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1008 partition: 0 MaxSeqNo: 25 sessionId: src_id|c8069ab0-bbcfbeb5-5df371f0-43fb8026_0
2025-10-09T21:47:43.283492Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb268450-ee0a6b12-fa6e2dfd-22e397ef_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c8069ab0-bbcfbeb5-5df371f0-43fb8026_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:43.286093Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8069ab0-bbcfbeb5-5df371f0-43fb8026_0] Write session: try to update token
2025-10-09T21:47:43.286112Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8069ab0-bbcfbeb5-5df371f0-43fb8026_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:43.293889Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:43.293906Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:43.293916Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559341998913268326:8247]
2025-10-09T21:47:43.293970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:43.294220Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:43.294366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:43.294508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:43.294682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:43.294866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:43.294908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:43.295051Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:43.302378Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1008 sessionId: src_id|c8069ab0-bbcfbeb5-5df371f0-43fb8026_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:43.304889Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:43.305104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:43.305127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:43.305168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:43.305252Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:43.307453Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:43.307469Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:43.307472Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:43.307474Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:43.307483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341998913268326:8247]
2025-10-09T21:47:43.307486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341998913268326:8247]
2025-10-09T21:47:43.307489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341998913268326:8247]
2025-10-09T21:47:43.307542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:43.307551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.307550Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:43.307554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:43.307556Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:43.307561Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:43.307696Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:43.354905Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341999275088817:9144]
2025-10-09T21:47:43.355217Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 1007, pipe: [1:7559341999275088817:9144]
2025-10-09T21:47:43.471140Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:43.478602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:43.478627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:43.478776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:43.478837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:43.478899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:43.478960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:43.479170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:43.479185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:43.479255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:43.479311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:43.479375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:43.479444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:43.479484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:43.479491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:43.479549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:43.479607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:43.479665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:43.479724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:43.480128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:43.480138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:43.480203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:43.480267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:43.480326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:43.480394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:43.480436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:43.480443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:43.480505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:43.480564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:43.480625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:43.480684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:43.480959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:43.480967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:43.481024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:43.481080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:43.481145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:43.481239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:43.481283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:43.481291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:43.481348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:43.481403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:43.481460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:43.481510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:43.481550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:43.481555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:43.481617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:43.481673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:43.481729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:43.481809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:43.481863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:43.481868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:43.481928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:43.482006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:43.482074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:43.482132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:43.482179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:43.482190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:43.482265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:43.482326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:43.482386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:43.482446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:43.482487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:43.482495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:43.482556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341999275088817:9144], now have 1 active actors on pipe
2025-10-09T21:47:43.482895Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:43.482904Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:43.482974Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:43.483013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1008 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c8069ab0-bbcfbeb5-5df371f0-43fb8026_0
2025-10-09T21:47:43.483283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1008 sessionId: src_id|c8069ab0-bbcfbeb5-5df371f0-43fb8026_0 is DEAD
2025-10-09T21:47:43.483390Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8069ab0-bbcfbeb5-5df371f0-43fb8026_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:43.483401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8069ab0-bbcfbeb5-5df371f0-43fb8026_0] Write session will restart in 0.006141s
2025-10-09T21:47:43.483424Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8069ab0-bbcfbeb5-5df371f0-43fb8026_0] Write session: Do CDS request
2025-10-09T21:47:43.483431Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8069ab0-bbcfbeb5-5df371f0-43fb8026_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:43.483599Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341999275088817:9144]
2025-10-09T21:47:43.490573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8069ab0-bbcfbeb5-5df371f0-43fb8026_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:43.490660Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:43.490699Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1009
2025-10-09T21:47:43.490859Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1009 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:43.490907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1009 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:43.490924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1009 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:43.490929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:43.491340Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1009 sessionId:  describe result for acl check
2025-10-09T21:47:43.491398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:43.491410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:43.491412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:43.491417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341999275088875:9189] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:43.492065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341999275088875:9189] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:43.492333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:43.493845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:43.494099Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:43.494108Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:43.494122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:43.494182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:43.494222Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341998913268431:8257]
2025-10-09T21:47:43.494457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341999275088875:9189] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:43.494763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:43.495064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:43.495124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:43.495441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:43.495502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.495508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.495509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.495510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.495511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.495512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.495513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.495515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.495516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.495521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.495530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.495533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.495537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.495540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.495544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.495547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.495550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.495554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.495561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.495568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.495574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.495578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.495583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.495587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.495590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.495594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.495603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.495607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.495611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.495620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.495624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.495632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.495636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.495640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.495648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.495652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.495675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:43.500556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341999275088875:9189] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:43.500579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341999275088875:9189] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:43.500583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341999275088875:9189] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:43.500598Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1009 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:43.501734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:43.502853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:43.502869Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:43.502871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:43.502875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:43.502881Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1008 [2:7559341998913268431:8257]
2025-10-09T21:47:43.502893Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:43.502909Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:43.502911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:43.502914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:43.502917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:43.502933Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:43.502944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:43.502950Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:43.503003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.503010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.503011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.503013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:43.503130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:43.503137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.503146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.503158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.503173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:43.503198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341999275088832:2432], now have 1 active actors on pipe
2025-10-09T21:47:43.503205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341999275088917:9189], now have 1 active actors on pipe
2025-10-09T21:47:43.503212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341998913268426:8257]
2025-10-09T21:47:43.503213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341998913268426:8257]
2025-10-09T21:47:43.503215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341998913268426:8257]
2025-10-09T21:47:43.503226Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:43.503244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:43.503228Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1008
2025-10-09T21:47:43.503249Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:43.503258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:43.503260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:43.503261Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:43.503264Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:43.503298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341998913268426:8257]
2025-10-09T21:47:43.503306Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:43.503308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:43.503279Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1008
2025-10-09T21:47:43.503332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:43.503341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:43.503370Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|248524b-6741f32c-c567b1a7-3e5b5de_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:43.503381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:43.503385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:43.503389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:43.503400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:43.503410Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:43.503433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:43.503583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:43.503590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:43.503604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:43.503701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1009 partition: 0 MaxSeqNo: 25 sessionId: src_id|248524b-6741f32c-c567b1a7-3e5b5de_0
2025-10-09T21:47:43.504152Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8069ab0-bbcfbeb5-5df371f0-43fb8026_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|248524b-6741f32c-c567b1a7-3e5b5de_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:43.506725Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|248524b-6741f32c-c567b1a7-3e5b5de_0] Write session: try to update token
2025-10-09T21:47:43.506740Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|248524b-6741f32c-c567b1a7-3e5b5de_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:43.518588Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:43.518608Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:43.518610Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:43.518611Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:43.518619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341998913268426:8257]
2025-10-09T21:47:43.518624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341998913268426:8257]
2025-10-09T21:47:43.518628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341998913268426:8257]
2025-10-09T21:47:43.518669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:43.518673Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:43.518676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.518678Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:43.518679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:43.518696Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:43.518825Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:43.521253Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1009 sessionId: src_id|248524b-6741f32c-c567b1a7-3e5b5de_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:43.523700Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:43.583753Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341999275088934:9144]
2025-10-09T21:47:43.584029Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 1008, pipe: [1:7559341999275088934:9144]
2025-10-09T21:47:43.669637Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:43.670007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:43.670024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:43.670082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341999275088934:9144], now have 1 active actors on pipe
2025-10-09T21:47:43.670318Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:43.670313Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:43.670340Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:43.670344Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:43.670371Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341999275088934:9144]
2025-10-09T21:47:43.670435Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1009 reason: error at write: Disconnected sessionId: src_id|248524b-6741f32c-c567b1a7-3e5b5de_0
2025-10-09T21:47:43.670629Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1009 sessionId: src_id|248524b-6741f32c-c567b1a7-3e5b5de_0 is DEAD
2025-10-09T21:47:43.670794Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|248524b-6741f32c-c567b1a7-3e5b5de_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:43.670805Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|248524b-6741f32c-c567b1a7-3e5b5de_0] Write session will restart in 0.005681s
2025-10-09T21:47:43.670832Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|248524b-6741f32c-c567b1a7-3e5b5de_0] Write session: Do CDS request
2025-10-09T21:47:43.670838Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|248524b-6741f32c-c567b1a7-3e5b5de_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:43.677557Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|248524b-6741f32c-c567b1a7-3e5b5de_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:43.677619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:43.677634Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1010
2025-10-09T21:47:43.677783Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1010 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:43.677830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1010 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:43.677840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1010 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:43.677844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:43.678225Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1010 sessionId:  describe result for acl check
2025-10-09T21:47:43.678257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:43.678267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:43.678269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:43.678274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341999275088982:9194] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:43.678816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:43.678829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341999275088982:9194] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:43.680396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:43.680599Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:43.680611Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:43.680632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:43.680692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:43.680746Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341998913268527:8263]
2025-10-09T21:47:43.681075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341999275088982:9194] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:43.681275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:43.681531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:43.681581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:43.681829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:43.681866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.681871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.681871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.681872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.681873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.681873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.681874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.681875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.681875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.681878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.681884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.681887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.681889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.681891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.681894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.681898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.681902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.681905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.681911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.681920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.681926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.681931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.681935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.681940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.681945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.681950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.681954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.681959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.681969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.681974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.681978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.681982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.681986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.681990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.681994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.681999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.682027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:43.687524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341999275088982:9194] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:43.687542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341999275088982:9194] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:43.687546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341999275088982:9194] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:43.687554Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1010 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:43.688098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:43.689361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:43.689375Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:43.689378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:43.689383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:43.689389Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1009 [2:7559341998913268527:8263]
2025-10-09T21:47:43.689402Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:43.689422Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:43.689425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:43.689428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:43.689431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:43.689448Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:43.689457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:43.689464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:43.689526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.689532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.689534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.689537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:43.689601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:43.689608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.689621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.689638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.689657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:43.689682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341999275089023:9194], now have 1 active actors on pipe
2025-10-09T21:47:43.689689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341999275088937:2432], now have 1 active actors on pipe
2025-10-09T21:47:43.689697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341998913268522:8263]
2025-10-09T21:47:43.689702Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341998913268522:8263]
2025-10-09T21:47:43.689704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341998913268522:8263]
2025-10-09T21:47:43.689695Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1009
2025-10-09T21:47:43.689716Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1009
2025-10-09T21:47:43.689716Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:43.689733Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:43.689739Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:43.689743Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:43.689744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:43.689745Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:43.689747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:43.689765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:43.689770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:43.689824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341998913268522:8263]
2025-10-09T21:47:43.689836Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:43.689848Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:43.689854Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6eae099b-25f9a6b8-40a84306-ab91cbfe_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:43.689861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:43.689866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:43.689868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:43.689877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:43.689886Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:43.689924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:43.690169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:43.690177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:43.690190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:43.690260Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1010 partition: 0 MaxSeqNo: 25 sessionId: src_id|6eae099b-25f9a6b8-40a84306-ab91cbfe_0
2025-10-09T21:47:43.690615Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|248524b-6741f32c-c567b1a7-3e5b5de_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6eae099b-25f9a6b8-40a84306-ab91cbfe_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:43.692981Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6eae099b-25f9a6b8-40a84306-ab91cbfe_0] Write session: try to update token
2025-10-09T21:47:43.692999Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6eae099b-25f9a6b8-40a84306-ab91cbfe_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:43.703480Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:43.703499Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:43.703502Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:43.703503Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:43.703513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341998913268522:8263]
2025-10-09T21:47:43.703518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341998913268522:8263]
2025-10-09T21:47:43.703524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341998913268522:8263]
2025-10-09T21:47:43.703569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:43.703577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.703579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:43.703580Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:43.703585Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:43.703589Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:43.703713Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:43.705681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1010 sessionId: src_id|6eae099b-25f9a6b8-40a84306-ab91cbfe_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:43.708130Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:43.770647Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341999275089030:9144]
2025-10-09T21:47:43.770897Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 1009, pipe: [1:7559341999275089030:9144]
2025-10-09T21:47:43.859331Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:43.859625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:43.859643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:43.859688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341999275089030:9144], now have 1 active actors on pipe
2025-10-09T21:47:43.859976Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341999275089030:9144]
2025-10-09T21:47:43.860009Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:43.860031Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:43.860116Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:43.860204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1010 reason: status is not ok: tablet will be restarted right now sessionId: src_id|6eae099b-25f9a6b8-40a84306-ab91cbfe_0
2025-10-09T21:47:43.860397Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1010 sessionId: src_id|6eae099b-25f9a6b8-40a84306-ab91cbfe_0 is DEAD
2025-10-09T21:47:43.860580Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6eae099b-25f9a6b8-40a84306-ab91cbfe_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:43.860592Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6eae099b-25f9a6b8-40a84306-ab91cbfe_0] Write session will restart in 0.009158s
2025-10-09T21:47:43.860621Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6eae099b-25f9a6b8-40a84306-ab91cbfe_0] Write session: Do CDS request
2025-10-09T21:47:43.860628Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6eae099b-25f9a6b8-40a84306-ab91cbfe_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:43.870315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:43.870544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6eae099b-25f9a6b8-40a84306-ab91cbfe_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:43.870640Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:43.870658Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1011
2025-10-09T21:47:43.870797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1011 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:43.870840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1011 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:43.870850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1011 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:43.870854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:43.871335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1011 sessionId:  describe result for acl check
2025-10-09T21:47:43.871369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:43.871376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:43.871378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:43.871384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559341999275089083:9199] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:43.871928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559341999275089083:9199] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:43.872040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:43.872256Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:43.872268Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:43.872280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:43.872337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:43.872384Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559341998913268613:8265]
2025-10-09T21:47:43.872946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:43.873392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:43.873439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:43.873774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:43.873841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.873853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.873854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.873857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.873859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.873860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.873861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.873862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.873863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.873868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.873878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.873881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.873885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.873888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.873892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.873895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.873899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.873902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.873908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.873916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:43.873922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.873927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:43.873932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.873937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:43.873942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.873947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:43.873952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.873957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:43.873961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.873966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:43.873971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.873975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.873979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.873985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.873991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.873996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.874025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:43.874117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559341999275089083:9199] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:43.880463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559341999275089083:9199] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:43.880479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559341999275089083:9199] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:43.880483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559341999275089083:9199] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:43.880491Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1011 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:43.880904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:43.882095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:43.882108Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:43.882110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:43.882113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:43.882118Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1010 [2:7559341998913268613:8265]
2025-10-09T21:47:43.882132Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:43.882146Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:43.882148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:43.882150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:43.882151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:43.882166Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:43.882170Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:43.882176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:43.882223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.882230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.882231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.882233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:43.882286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:43.882294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.882305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:43.882319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:43.882353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:43.882375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341999275089122:9199], now have 1 active actors on pipe
2025-10-09T21:47:43.882383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341999275089035:2432], now have 1 active actors on pipe
2025-10-09T21:47:43.882390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559341998913268607:8265]
2025-10-09T21:47:43.882391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559341998913268607:8265]
2025-10-09T21:47:43.882393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559341998913268607:8265]
2025-10-09T21:47:43.882401Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:43.882411Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:43.882422Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:43.882424Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:43.882426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:43.882432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:43.882434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:43.882429Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1010
2025-10-09T21:47:43.882458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559341998913268607:8265]
2025-10-09T21:47:43.882445Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1010
2025-10-09T21:47:43.882465Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:43.882470Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:43.882490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:43.882501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:43.882540Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|29958daf-96b9e3e2-7337b72d-674f1cdc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:43.882554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:43.882559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:43.882562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:43.882571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:43.882586Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:43.882604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:43.882732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:43.882739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:43.882751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:43.882840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1011 partition: 0 MaxSeqNo: 25 sessionId: src_id|29958daf-96b9e3e2-7337b72d-674f1cdc_0
2025-10-09T21:47:43.883203Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6eae099b-25f9a6b8-40a84306-ab91cbfe_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|29958daf-96b9e3e2-7337b72d-674f1cdc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:43.885639Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29958daf-96b9e3e2-7337b72d-674f1cdc_0] Write session: try to update token
2025-10-09T21:47:43.885654Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29958daf-96b9e3e2-7337b72d-674f1cdc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:43.896884Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:43.896903Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:43.896905Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:43.896907Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:43.896918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559341998913268607:8265]
2025-10-09T21:47:43.896922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559341998913268607:8265]
2025-10-09T21:47:43.896926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559341998913268607:8265]
2025-10-09T21:47:43.896975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:43.896980Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:43.896986Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:43.896989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:43.896990Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:43.896992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:43.897140Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:43.898781Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1011 sessionId: src_id|29958daf-96b9e3e2-7337b72d-674f1cdc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:43.901232Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:43.960078Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559341999275089129:9144]
2025-10-09T21:47:43.960373Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 1010, pipe: [1:7559341999275089129:9144]
2025-10-09T21:47:44.058694Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:44.059369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:44.059390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:44.059438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559341999275089129:9144], now have 1 active actors on pipe
2025-10-09T21:47:44.059837Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:44.059857Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:44.059871Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:44.059858Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:44.059950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1011 reason: error at write: Disconnected sessionId: src_id|29958daf-96b9e3e2-7337b72d-674f1cdc_0
2025-10-09T21:47:44.059968Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559341999275089129:9144]
2025-10-09T21:47:44.060173Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1011 sessionId: src_id|29958daf-96b9e3e2-7337b72d-674f1cdc_0 is DEAD
2025-10-09T21:47:44.060374Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29958daf-96b9e3e2-7337b72d-674f1cdc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:44.060390Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29958daf-96b9e3e2-7337b72d-674f1cdc_0] Write session will restart in 0.007370s
2025-10-09T21:47:44.060427Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29958daf-96b9e3e2-7337b72d-674f1cdc_0] Write session: Do CDS request
2025-10-09T21:47:44.060435Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29958daf-96b9e3e2-7337b72d-674f1cdc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:44.068655Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29958daf-96b9e3e2-7337b72d-674f1cdc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:44.069074Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:44.069096Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1012
2025-10-09T21:47:44.069228Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1012 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:44.069264Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1012 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:44.069270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1012 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:44.069276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:44.070014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1012 sessionId:  describe result for acl check
2025-10-09T21:47:44.070068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:44.070073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:44.070077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:44.070085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342003570056476:9203] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:44.070726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342003570056476:9203] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:44.071274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:44.073764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:44.074078Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:44.074088Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:44.074102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:44.074162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:44.074206Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342003208235995:8267]
2025-10-09T21:47:44.074860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:44.074947Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342003570056476:9203] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:44.075188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:44.075267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:44.075670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:44.075742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.075748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.075749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.075750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.075752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.075753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.075756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.075757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.075759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.075764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.075774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.075778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.075783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.075787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.075791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.075795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.075799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.075803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.075809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.075817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.075839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.075849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.075853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.075859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.075863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.075873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.075877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.075888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.075893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.075901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.075910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.075915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.075920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.075926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.075930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.075936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.075966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:44.081347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342003570056476:9203] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:44.081365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342003570056476:9203] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:44.081369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342003570056476:9203] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:44.081378Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1012 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:44.082692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:44.085007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:44.085023Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:44.085026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:44.085030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:44.085036Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1011 [2:7559342003208235995:8267]
2025-10-09T21:47:44.085049Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:44.085068Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:44.085071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:44.085076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:44.085078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:44.085095Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:44.085107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:44.085114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:44.085176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.085183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.085185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.085189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:44.085256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:44.085264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.085276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.085289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.085307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:44.085333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342003570056518:9203], now have 1 active actors on pipe
2025-10-09T21:47:44.085341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342003570056430:2432], now have 1 active actors on pipe
2025-10-09T21:47:44.085350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342003208235989:8267]
2025-10-09T21:47:44.085352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342003208235989:8267]
2025-10-09T21:47:44.085353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342003208235989:8267]
2025-10-09T21:47:44.085365Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:44.085384Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:44.085399Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:44.085403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:44.085407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:44.085409Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:44.085412Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:44.085415Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1011
2025-10-09T21:47:44.085452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342003208235989:8267]
2025-10-09T21:47:44.085461Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:44.085463Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:44.085460Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1011
2025-10-09T21:47:44.085475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:44.085484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:44.085540Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3a2eac7f-278d3470-126af7e0-eb72839e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:44.085552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:44.085556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:44.085562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:44.085574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:44.085587Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:44.085615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:44.085789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:44.085797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:44.085810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:44.085908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1012 partition: 0 MaxSeqNo: 25 sessionId: src_id|3a2eac7f-278d3470-126af7e0-eb72839e_0
2025-10-09T21:47:44.086305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29958daf-96b9e3e2-7337b72d-674f1cdc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3a2eac7f-278d3470-126af7e0-eb72839e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:44.091246Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a2eac7f-278d3470-126af7e0-eb72839e_0] Write session: try to update token
2025-10-09T21:47:44.091273Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a2eac7f-278d3470-126af7e0-eb72839e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:44.100509Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:44.100529Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:44.100532Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:44.100534Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:44.100544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342003208235989:8267]
2025-10-09T21:47:44.100548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342003208235989:8267]
2025-10-09T21:47:44.100554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342003208235989:8267]
2025-10-09T21:47:44.100596Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:44.100598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:44.100602Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:44.100605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.100608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:44.100619Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:44.100752Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:44.105982Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1012 sessionId: src_id|3a2eac7f-278d3470-126af7e0-eb72839e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:44.108422Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:44.159754Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342003570056528:9144]
2025-10-09T21:47:44.160025Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 1011, pipe: [1:7559342003570056528:9144]
2025-10-09T21:47:44.260581Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:44.260872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:44.260889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:44.260936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342003570056528:9144], now have 1 active actors on pipe
2025-10-09T21:47:44.261164Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:44.261187Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:44.261191Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:44.261227Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342003570056528:9144]
2025-10-09T21:47:44.261253Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1012 reason: error at write: Disconnected sessionId: src_id|3a2eac7f-278d3470-126af7e0-eb72839e_0
2025-10-09T21:47:44.261303Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:44.261446Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1012 sessionId: src_id|3a2eac7f-278d3470-126af7e0-eb72839e_0 is DEAD
2025-10-09T21:47:44.261627Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a2eac7f-278d3470-126af7e0-eb72839e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:44.261640Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a2eac7f-278d3470-126af7e0-eb72839e_0] Write session will restart in 0.006052s
2025-10-09T21:47:44.261678Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a2eac7f-278d3470-126af7e0-eb72839e_0] Write session: Do CDS request
2025-10-09T21:47:44.261684Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a2eac7f-278d3470-126af7e0-eb72839e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:44.268582Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a2eac7f-278d3470-126af7e0-eb72839e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:44.268674Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:44.268690Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1013
2025-10-09T21:47:44.268819Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1013 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:44.268854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1013 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:44.268876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1013 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:44.268880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:44.269295Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1013 sessionId:  describe result for acl check
2025-10-09T21:47:44.269332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:44.269340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:44.269341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:44.269347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342003570056583:9208] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:44.269914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342003570056583:9208] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:44.270294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:44.272121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:44.272322Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:44.272333Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:44.272345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:44.272396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:44.272445Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342003208236082:8269]
2025-10-09T21:47:44.272619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342003570056583:9208] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:44.273019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:44.273396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:44.273454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:44.273801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:44.273862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.273863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.273864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.273866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.273867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.273868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.273869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.273871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.273872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.273877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.273889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.273893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.273897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.273900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.273904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.273907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.273911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.273914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.273921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.273930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.273936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.273941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.273946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.273951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.273976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.273989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.273994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.274003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.274008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.274018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.274022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.274043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.274052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.274057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.274062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.274071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.274099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:44.279497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342003570056583:9208] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:44.279517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342003570056583:9208] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:44.279521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342003570056583:9208] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:44.279532Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1013 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:44.280935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:44.282143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:44.282159Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:44.282161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:44.282166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:44.282177Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1012 [2:7559342003208236082:8269]
2025-10-09T21:47:44.282191Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:44.282209Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:44.282216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:44.282220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:44.282222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:44.282241Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:44.282258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:44.282265Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:44.282327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.282335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.282336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.282339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:44.282410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:44.282418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.282430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.282441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.282456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:44.282477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342003208236076:8269]
2025-10-09T21:47:44.282479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342003208236076:8269]
2025-10-09T21:47:44.282481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342003208236076:8269]
2025-10-09T21:47:44.282490Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:44.282506Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:44.282506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342003570056536:2432], now have 1 active actors on pipe
2025-10-09T21:47:44.282510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342003570056624:9208], now have 1 active actors on pipe
2025-10-09T21:47:44.282516Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:44.282518Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:44.282519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:44.282532Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:44.282535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:44.282570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342003208236076:8269]
2025-10-09T21:47:44.282584Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:44.282586Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:44.282570Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1012
2025-10-09T21:47:44.282621Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1012
2025-10-09T21:47:44.282733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:44.282744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:44.282780Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4dc2e8f5-5d32121d-a5041b36-ea12203f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:44.282794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:44.282802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:44.282805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:44.282815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:44.282829Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:44.282849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:44.283057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:44.283065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:44.283077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:44.283212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1013 partition: 0 MaxSeqNo: 25 sessionId: src_id|4dc2e8f5-5d32121d-a5041b36-ea12203f_0
2025-10-09T21:47:44.286137Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a2eac7f-278d3470-126af7e0-eb72839e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4dc2e8f5-5d32121d-a5041b36-ea12203f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:44.288753Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4dc2e8f5-5d32121d-a5041b36-ea12203f_0] Write session: try to update token
2025-10-09T21:47:44.288768Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4dc2e8f5-5d32121d-a5041b36-ea12203f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:44.300878Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:44.300902Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:44.300905Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:44.300906Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:44.300916Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342003208236076:8269]
2025-10-09T21:47:44.300921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342003208236076:8269]
2025-10-09T21:47:44.300925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342003208236076:8269]
2025-10-09T21:47:44.300969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:44.300976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.300981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:44.300970Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:44.300990Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:44.300994Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:44.301162Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:44.316671Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1013 sessionId: src_id|4dc2e8f5-5d32121d-a5041b36-ea12203f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:44.319268Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:44.359432Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342003570056633:9144]
2025-10-09T21:47:44.359795Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 1012, pipe: [1:7559342003570056633:9144]
2025-10-09T21:47:44.462193Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:44.462553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:44.462569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:44.462593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342003570056633:9144], now have 1 active actors on pipe
2025-10-09T21:47:44.462839Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:44.462980Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342003570056633:9144]
2025-10-09T21:47:44.463018Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:44.463056Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:44.463125Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1013 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4dc2e8f5-5d32121d-a5041b36-ea12203f_0
2025-10-09T21:47:44.463299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1013 sessionId: src_id|4dc2e8f5-5d32121d-a5041b36-ea12203f_0 is DEAD
2025-10-09T21:47:44.463466Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4dc2e8f5-5d32121d-a5041b36-ea12203f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:44.463478Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4dc2e8f5-5d32121d-a5041b36-ea12203f_0] Write session will restart in 0.006689s
2025-10-09T21:47:44.463508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4dc2e8f5-5d32121d-a5041b36-ea12203f_0] Write session: Do CDS request
2025-10-09T21:47:44.463514Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4dc2e8f5-5d32121d-a5041b36-ea12203f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:44.471560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4dc2e8f5-5d32121d-a5041b36-ea12203f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:44.471679Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:44.471700Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1014
2025-10-09T21:47:44.471896Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1014 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:44.471938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1014 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:44.471949Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1014 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:44.471953Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:44.472419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1014 sessionId:  describe result for acl check
2025-10-09T21:47:44.472459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:44.472460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:44.472462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:44.472467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342003570056683:9212] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:44.473044Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342003570056683:9212] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:44.474264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:44.475159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342003570056683:9212] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:44.475929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:44.476142Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:44.476152Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:44.476164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:44.476219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:44.476267Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342003208236200:8281]
2025-10-09T21:47:44.476864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:44.477222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:44.477270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:44.477684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:44.477763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.477770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.477771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.477773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.477774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.477775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.477777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.477778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.477780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.477786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.477796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.477800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.477805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.477809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.477826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.477830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.477834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.477838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.477845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.477854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.477860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.477866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.477870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.477876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.477881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.477887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.477891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.477896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.477900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.477906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.477910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.477915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.477918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.477923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.477927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.477932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.477963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:44.484865Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342003570056683:9212] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:44.484883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342003570056683:9212] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:44.484887Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342003570056683:9212] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:44.484898Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1014 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:44.485263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:44.486448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:44.486467Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:44.486472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:44.486476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:44.486482Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1013 [2:7559342003208236200:8281]
2025-10-09T21:47:44.486494Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:44.486512Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:44.486515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:44.486517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:44.486520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:44.486537Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:44.486543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:44.486550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:44.486608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.486619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.486621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.486624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:44.486691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:44.486699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.486712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.486722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.486736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:44.486756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342003208236194:8281]
2025-10-09T21:47:44.486764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342003208236194:8281]
2025-10-09T21:47:44.486765Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342003208236194:8281]
2025-10-09T21:47:44.486774Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:44.486782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:44.486792Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:44.486795Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:44.486797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:44.486811Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:44.486814Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:44.486826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342003570056635:2432], now have 1 active actors on pipe
2025-10-09T21:47:44.486830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342003570056724:9212], now have 1 active actors on pipe
2025-10-09T21:47:44.486866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342003208236194:8281]
2025-10-09T21:47:44.486874Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:44.486893Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:44.486816Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1013
2025-10-09T21:47:44.486851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1013
2025-10-09T21:47:44.487061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:44.487076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:44.487106Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bc6c3d07-aabe385a-dd85b3e2-1db65498_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:44.487119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:44.487124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:44.487127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:44.487138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:44.487154Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:44.487172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:44.487333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:44.487344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:44.487358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:44.487672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1014 partition: 0 MaxSeqNo: 25 sessionId: src_id|bc6c3d07-aabe385a-dd85b3e2-1db65498_0
2025-10-09T21:47:44.488044Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4dc2e8f5-5d32121d-a5041b36-ea12203f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bc6c3d07-aabe385a-dd85b3e2-1db65498_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:44.490591Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc6c3d07-aabe385a-dd85b3e2-1db65498_0] Write session: try to update token
2025-10-09T21:47:44.490609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc6c3d07-aabe385a-dd85b3e2-1db65498_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:44.501941Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:44.501962Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:44.501965Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:44.501967Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:44.501976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342003208236194:8281]
2025-10-09T21:47:44.501981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342003208236194:8281]
2025-10-09T21:47:44.501984Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342003208236194:8281]
2025-10-09T21:47:44.502044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:44.502051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.502050Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:44.502054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:44.502056Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:44.502059Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:44.502185Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:44.503983Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1014 sessionId: src_id|bc6c3d07-aabe385a-dd85b3e2-1db65498_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:44.506378Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:44.563176Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342003570056755:9144]
2025-10-09T21:47:44.563405Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 1013, pipe: [1:7559342003570056755:9144]
2025-10-09T21:47:44.659032Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:44.659334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:44.659351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:44.659396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342003570056755:9144], now have 1 active actors on pipe
2025-10-09T21:47:44.659668Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:44.660024Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342003570056755:9144]
2025-10-09T21:47:44.660059Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:44.660088Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:44.660156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1014 reason: status is not ok: tablet will be restarted right now sessionId: src_id|bc6c3d07-aabe385a-dd85b3e2-1db65498_0
2025-10-09T21:47:44.660353Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1014 sessionId: src_id|bc6c3d07-aabe385a-dd85b3e2-1db65498_0 is DEAD
2025-10-09T21:47:44.666253Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc6c3d07-aabe385a-dd85b3e2-1db65498_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:44.666268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc6c3d07-aabe385a-dd85b3e2-1db65498_0] Write session will restart in 0.005868s
2025-10-09T21:47:44.666295Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc6c3d07-aabe385a-dd85b3e2-1db65498_0] Write session: Do CDS request
2025-10-09T21:47:44.666300Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc6c3d07-aabe385a-dd85b3e2-1db65498_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:44.673613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc6c3d07-aabe385a-dd85b3e2-1db65498_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:44.673757Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:44.673774Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1015
2025-10-09T21:47:44.673934Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1015 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:44.673970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1015 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:44.673984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1015 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:44.673988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:44.674486Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1015 sessionId:  describe result for acl check
2025-10-09T21:47:44.674525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:44.674531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:44.674533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:44.674539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342003570056786:9223] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:44.675080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342003570056786:9223] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:44.677284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:44.677689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342003570056786:9223] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:44.679081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:44.679271Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:44.679281Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:44.679293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:44.679343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:44.679394Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342003208236302:8287]
2025-10-09T21:47:44.679960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:44.680253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:44.680306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:44.680634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:44.680679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.680692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342003570056786:9223] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:44.680686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.680706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342003570056786:9223] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:44.680710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342003570056786:9223] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:44.680718Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1015 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:44.680687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.680689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.680690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.680691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.680692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.680693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.680694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.680699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.680709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.680713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.680717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.680720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.680724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.680727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.680731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.680734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.680741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.680749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.680755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.680759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.680787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.680792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.680796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.680802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.680806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.680811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.680821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.680826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.680836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.680840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.680844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.680849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.680853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.680867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.680895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:44.688071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:44.689333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:44.689354Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:44.689356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:44.689360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:44.689370Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1014 [2:7559342003208236302:8287]
2025-10-09T21:47:44.689383Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:44.689402Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:44.689410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:44.689414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:44.689418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:44.689438Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:44.689449Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:44.689456Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:44.689520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.689529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.689531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.689533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:44.689607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:44.689616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.689627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.689638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.689653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:44.689673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342003208236297:8287]
2025-10-09T21:47:44.689682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342003208236297:8287]
2025-10-09T21:47:44.689684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342003208236297:8287]
2025-10-09T21:47:44.689692Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:44.689708Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:44.689709Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:44.689712Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:44.689714Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:44.689723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:44.689726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:44.689737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342003570056847:9223], now have 1 active actors on pipe
2025-10-09T21:47:44.689740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342003570056757:2432], now have 1 active actors on pipe
2025-10-09T21:47:44.689778Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342003208236297:8287]
2025-10-09T21:47:44.689794Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:44.689839Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1014
2025-10-09T21:47:44.689845Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1014
2025-10-09T21:47:44.689842Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:44.690156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:44.690170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:44.690206Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a80a08b8-184798fd-7d2e11f0-dc27a0ab_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:44.690223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:44.690238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:44.690247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:44.690260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:44.690273Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:44.690306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:44.690546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:44.690558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:44.690579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:44.690695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1015 partition: 0 MaxSeqNo: 25 sessionId: src_id|a80a08b8-184798fd-7d2e11f0-dc27a0ab_0
2025-10-09T21:47:44.691081Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc6c3d07-aabe385a-dd85b3e2-1db65498_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a80a08b8-184798fd-7d2e11f0-dc27a0ab_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:44.693552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a80a08b8-184798fd-7d2e11f0-dc27a0ab_0] Write session: try to update token
2025-10-09T21:47:44.693571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a80a08b8-184798fd-7d2e11f0-dc27a0ab_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:44.708343Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1015 sessionId: src_id|a80a08b8-184798fd-7d2e11f0-dc27a0ab_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:44.708332Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:44.708354Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:44.708356Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:44.708358Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:44.708368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342003208236297:8287]
2025-10-09T21:47:44.708374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342003208236297:8287]
2025-10-09T21:47:44.708377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342003208236297:8287]
2025-10-09T21:47:44.708427Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:44.708430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:44.708433Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:44.708437Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:44.708438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.708440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:44.708579Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:44.710960Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:44.759815Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342003570056857:9144]
2025-10-09T21:47:44.760221Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 1014, pipe: [1:7559342003570056857:9144]
2025-10-09T21:47:44.871797Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:44.872176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:44.872192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:44.872242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342003570056857:9144], now have 1 active actors on pipe
2025-10-09T21:47:44.872520Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:44.872529Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:44.872545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:44.872580Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342003570056857:9144]
2025-10-09T21:47:44.872489Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:44.872604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1015 reason: error at write: Disconnected sessionId: src_id|a80a08b8-184798fd-7d2e11f0-dc27a0ab_0
2025-10-09T21:47:44.872938Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a80a08b8-184798fd-7d2e11f0-dc27a0ab_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:44.872948Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a80a08b8-184798fd-7d2e11f0-dc27a0ab_0] Write session will restart in 0.005127s
2025-10-09T21:47:44.872979Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a80a08b8-184798fd-7d2e11f0-dc27a0ab_0] Write session: Do CDS request
2025-10-09T21:47:44.872989Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a80a08b8-184798fd-7d2e11f0-dc27a0ab_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:44.872951Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1015 sessionId: src_id|a80a08b8-184798fd-7d2e11f0-dc27a0ab_0 is DEAD
2025-10-09T21:47:44.879638Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a80a08b8-184798fd-7d2e11f0-dc27a0ab_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:44.879670Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:44.879701Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1016
2025-10-09T21:47:44.879880Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1016 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:44.879918Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1016 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:44.879928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1016 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:44.879932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:44.880401Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1016 sessionId:  describe result for acl check
2025-10-09T21:47:44.880442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:44.880448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:44.880450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:44.880456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342003570056916:9228] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:44.880993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342003570056916:9228] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:44.881831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:44.883165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342003570056916:9228] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:44.883578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:44.883798Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:44.883809Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:44.883821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:44.883876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:44.883915Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342003208236392:8289]
2025-10-09T21:47:44.884505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:44.884773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:44.884816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:44.885132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:44.885178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.885183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.885184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.885185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.885186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.885187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.885188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.885189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.885190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.885195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.885205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.885209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.885213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.885217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.885221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.885225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.885229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.885232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.885239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.885247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:44.885253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.885258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:44.885263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.885268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:44.885272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.885277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:44.885282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.885291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:44.885297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.885306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:44.885311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.885315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.885319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.885324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.885331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.885337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.885370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:44.886008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342003570056916:9228] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:44.886024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342003570056916:9228] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:44.886043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342003570056916:9228] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:44.886053Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1016 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:44.891845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:44.893088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:44.893104Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:44.893107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:44.893111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:44.893118Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1015 [2:7559342003208236392:8289]
2025-10-09T21:47:44.893136Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:44.893157Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:44.893168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:44.893171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:44.893174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:44.893195Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:44.893208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:44.893216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:44.893294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.893303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.893305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.893308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:44.893375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:44.893382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.893396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:44.893413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:44.893430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:44.893457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342003570056958:9228], now have 1 active actors on pipe
2025-10-09T21:47:44.893464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342003570056866:2432], now have 1 active actors on pipe
2025-10-09T21:47:44.893473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342003208236387:8289]
2025-10-09T21:47:44.893475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342003208236387:8289]
2025-10-09T21:47:44.893478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342003208236387:8289]
2025-10-09T21:47:44.893490Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:44.893505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:44.893506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:44.893508Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:44.893510Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:44.893521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:44.893524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:44.893562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342003208236387:8289]
2025-10-09T21:47:44.893570Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:44.893572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:44.893523Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1015
2025-10-09T21:47:44.893541Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1015
2025-10-09T21:47:44.893606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:44.893617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:44.893654Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a741c688-3a008264-5e648aa-be0a7114_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:44.893669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:44.893677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:44.893680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:44.893690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:44.893701Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:44.893720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:44.893972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:44.893980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:44.893993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:44.894121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1016 partition: 0 MaxSeqNo: 25 sessionId: src_id|a741c688-3a008264-5e648aa-be0a7114_0
2025-10-09T21:47:44.894523Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a80a08b8-184798fd-7d2e11f0-dc27a0ab_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a741c688-3a008264-5e648aa-be0a7114_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:44.897207Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a741c688-3a008264-5e648aa-be0a7114_0] Write session: try to update token
2025-10-09T21:47:44.897225Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a741c688-3a008264-5e648aa-be0a7114_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:44.905608Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:44.905630Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:44.905641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342003208236387:8289]
2025-10-09T21:47:44.905693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:44.905700Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:44.905880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:44.906020Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:44.906183Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:44.906341Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:44.906734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:44.906779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:44.913122Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1016 sessionId: src_id|a741c688-3a008264-5e648aa-be0a7114_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:44.915698Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:44.915920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:44.915945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:44.915990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:44.916101Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:44.918891Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:44.918910Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:44.918912Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:44.918914Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:44.918927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342003208236387:8289]
2025-10-09T21:47:44.918931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342003208236387:8289]
2025-10-09T21:47:44.918936Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342003208236387:8289]
2025-10-09T21:47:44.918983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:44.918994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:44.918997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:44.919003Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:44.919010Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:44.919018Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:44.919128Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:44.972786Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342003570056968:9144]
2025-10-09T21:47:44.973222Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 1015, pipe: [1:7559342003570056968:9144]
2025-10-09T21:47:45.087104Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:45.095819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.095851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.096030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.096095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.096167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.096237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.096604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.096617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.096690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.096752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.096825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.096896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.096933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.096936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.096999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.097096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.097153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.097207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.097246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.097249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.097319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.097380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.097444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.097502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.097540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.097542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.097614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.097670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.097729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.097788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.097845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.097848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.098076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.098196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.098267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.098332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.098395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.098405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.098481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.098539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.098624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.098687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.098755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.098764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.098823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.098879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.098935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.099004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.099053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.099060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.099122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.099177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.099241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.099306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.099369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.099377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.099441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.099502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.099563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.099629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.099700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.099711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.099782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342003570056968:9144], now have 1 active actors on pipe
2025-10-09T21:47:45.100034Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:45.100041Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342003570056968:9144]
2025-10-09T21:47:45.100051Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:45.100084Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1016 reason: error at write: Disconnected sessionId: src_id|a741c688-3a008264-5e648aa-be0a7114_0
2025-10-09T21:47:45.100106Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:45.100281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1016 sessionId: src_id|a741c688-3a008264-5e648aa-be0a7114_0 is DEAD
2025-10-09T21:47:45.100608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a741c688-3a008264-5e648aa-be0a7114_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:45.100620Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a741c688-3a008264-5e648aa-be0a7114_0] Write session will restart in 0.007047s
2025-10-09T21:47:45.100645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a741c688-3a008264-5e648aa-be0a7114_0] Write session: Do CDS request
2025-10-09T21:47:45.100652Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a741c688-3a008264-5e648aa-be0a7114_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:45.100647Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:45.108624Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a741c688-3a008264-5e648aa-be0a7114_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:45.108711Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:45.108730Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1017
2025-10-09T21:47:45.108859Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1017 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:45.108904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1017 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:45.108911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1017 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:45.108920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:45.109393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1017 sessionId:  describe result for acl check
2025-10-09T21:47:45.109449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:45.109451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:45.109452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:45.109460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342007865024284:9232] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:45.110082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342007865024284:9232] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:45.113451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342007865024284:9232] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:45.114234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:45.116698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:45.116958Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:45.116968Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:45.116980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:45.117037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:45.117081Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342007503203780:8291]
2025-10-09T21:47:45.117120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342007865024284:9232] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:45.117141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342007865024284:9232] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:45.117145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342007865024284:9232] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:45.117156Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1017 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:45.117746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:45.118134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:45.118199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:45.118593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:45.118653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.118661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.118663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.118664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.118666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.118668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.118669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.118670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.118672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.118678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.118690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.118694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.118699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.118704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.118709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.118713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.118717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.118720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.118731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.118739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.118746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.118752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.118756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.118761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.118766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.118771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.118775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.118791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.118801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.118807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.118819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.118825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.118834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.118840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.118845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.118854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.118885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:45.126494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:45.127690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:45.127711Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:45.127713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:45.127718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:45.127726Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1016 [2:7559342007503203780:8291]
2025-10-09T21:47:45.127742Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:45.127766Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:45.127769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:45.127773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:45.127776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:45.127798Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:45.127812Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:45.127820Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:45.127886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.127894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.127896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.127899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:45.127977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:45.127984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.127998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.128016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.128038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:45.128066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342007865024269:2432], now have 1 active actors on pipe
2025-10-09T21:47:45.128074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342007865024358:9232], now have 1 active actors on pipe
2025-10-09T21:47:45.128082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342007503203775:8291]
2025-10-09T21:47:45.128084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342007503203775:8291]
2025-10-09T21:47:45.128086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342007503203775:8291]
2025-10-09T21:47:45.128099Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:45.128117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:45.128126Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:45.128130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:45.128133Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:45.128137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:45.128141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:45.128184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342007503203775:8291]
2025-10-09T21:47:45.128195Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:45.128149Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1016
2025-10-09T21:47:45.128197Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:45.128307Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1016
2025-10-09T21:47:45.128357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.128370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.128401Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8c6be27f-d9d974e3-51d06ada-e7975f47_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:45.128428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:45.128438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:45.128440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:45.128458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:45.128469Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:45.128491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:45.128713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.128722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.128737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:45.128833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1017 partition: 0 MaxSeqNo: 25 sessionId: src_id|8c6be27f-d9d974e3-51d06ada-e7975f47_0
2025-10-09T21:47:45.129230Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a741c688-3a008264-5e648aa-be0a7114_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8c6be27f-d9d974e3-51d06ada-e7975f47_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:45.131808Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c6be27f-d9d974e3-51d06ada-e7975f47_0] Write session: try to update token
2025-10-09T21:47:45.131834Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c6be27f-d9d974e3-51d06ada-e7975f47_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:45.140356Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:45.140380Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:45.140390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342007503203775:8291]
2025-10-09T21:47:45.140440Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:45.140443Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:45.140620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:45.140794Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:45.140942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:45.141092Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:45.141319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:45.141375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:45.145889Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1017 sessionId: src_id|8c6be27f-d9d974e3-51d06ada-e7975f47_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:45.148619Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:45.148849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.148874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.148916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:45.148995Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:45.154056Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:45.154076Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:45.154079Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:45.154082Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:45.154097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342007503203775:8291]
2025-10-09T21:47:45.154101Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342007503203775:8291]
2025-10-09T21:47:45.154108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342007503203775:8291]
2025-10-09T21:47:45.154145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:45.154144Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:45.154151Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:45.154152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.154155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:45.154156Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:45.154293Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:45.200124Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342007865024378:9144]
2025-10-09T21:47:45.200442Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 1016, pipe: [1:7559342007865024378:9144]
2025-10-09T21:47:45.314406Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:45.321837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.321866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.322043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.322117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.322179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.322240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.322287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.322297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.322372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.322444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.322509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.322571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.322614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.322620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.322674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.322729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.322791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.322855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.322900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.322908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.322969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.323031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.323093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.323158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.323203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.323211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.323318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.323378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.323444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.323509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.323579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.323587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.323649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.323706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.323768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.323829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.323872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.323878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.323939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.323998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.324063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.324116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.324158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.324164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.324223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.324280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.324342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.324401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.324445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.324449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.324504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.324557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.324617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.324683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.324723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.324730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.324787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.324842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.324906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.324966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.325022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.325029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.325097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342007865024378:9144], now have 1 active actors on pipe
2025-10-09T21:47:45.325358Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342007865024378:9144]
2025-10-09T21:47:45.325366Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:45.325411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:45.325414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:45.325444Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1017 reason: error at write: Disconnected sessionId: src_id|8c6be27f-d9d974e3-51d06ada-e7975f47_0
2025-10-09T21:47:45.325710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1017 sessionId: src_id|8c6be27f-d9d974e3-51d06ada-e7975f47_0 is DEAD
2025-10-09T21:47:45.325840Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c6be27f-d9d974e3-51d06ada-e7975f47_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:45.325849Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c6be27f-d9d974e3-51d06ada-e7975f47_0] Write session will restart in 0.008264s
2025-10-09T21:47:45.325878Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c6be27f-d9d974e3-51d06ada-e7975f47_0] Write session: Do CDS request
2025-10-09T21:47:45.325884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c6be27f-d9d974e3-51d06ada-e7975f47_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:45.326079Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:45.335578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c6be27f-d9d974e3-51d06ada-e7975f47_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:45.335654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:45.335682Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1018
2025-10-09T21:47:45.336049Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1018 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:45.336101Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1018 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:45.336113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1018 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:45.336117Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:45.336455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:45.336524Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1018 sessionId:  describe result for acl check
2025-10-09T21:47:45.336560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:45.336566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:45.336568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:45.336573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342007865024438:9239] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:45.337499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342007865024438:9239] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:45.338364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:45.338570Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:45.338579Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:45.338591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:45.338646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:45.338688Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342007503203873:8293]
2025-10-09T21:47:45.339317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:45.339591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:45.339634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:45.339946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:45.340006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.340015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.340016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.340017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.340018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.340019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.340021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.340022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.340023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.340029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.340039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.340043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.340047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.340051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.340055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.340058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.340063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.340067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.340074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.340082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.340084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342007865024438:9239] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:45.340088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.340094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.340098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.340103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.340108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.340113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.340117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.340121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.340125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.340129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.340134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.340138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.340142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.340146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.340151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.340156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.340184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:45.346531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342007865024438:9239] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:45.346550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342007865024438:9239] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:45.346555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342007865024438:9239] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:45.346565Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1018 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:45.346894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:45.348005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:45.348024Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:45.348026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:45.348029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:45.348035Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1017 [2:7559342007503203873:8293]
2025-10-09T21:47:45.348046Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:45.348064Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:45.348069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:45.348073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:45.348077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:45.348109Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:45.348130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:45.348137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:45.348184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.348191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.348192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.348194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:45.348249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:45.348254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.348262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.348269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.348282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:45.348303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342007865024380:2432], now have 1 active actors on pipe
2025-10-09T21:47:45.348312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342007865024477:9239], now have 1 active actors on pipe
2025-10-09T21:47:45.348318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342007503203867:8293]
2025-10-09T21:47:45.348319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342007503203867:8293]
2025-10-09T21:47:45.348321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342007503203867:8293]
2025-10-09T21:47:45.348326Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:45.348337Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:45.348339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:45.348340Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:45.348343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:45.348348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:45.348350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:45.348379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342007503203867:8293]
2025-10-09T21:47:45.348385Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:45.348376Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1017
2025-10-09T21:47:45.348399Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1017
2025-10-09T21:47:45.348414Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:45.348435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.348445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.348469Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5fe8a482-f6038b60-9537ce83-adc55959_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:45.348479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:45.348482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:45.348484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:45.348492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:45.348500Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:45.348510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:45.348720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.348727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.348739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:45.348820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1018 partition: 0 MaxSeqNo: 25 sessionId: src_id|5fe8a482-f6038b60-9537ce83-adc55959_0
2025-10-09T21:47:45.349178Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c6be27f-d9d974e3-51d06ada-e7975f47_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5fe8a482-f6038b60-9537ce83-adc55959_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:45.351759Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5fe8a482-f6038b60-9537ce83-adc55959_0] Write session: try to update token
2025-10-09T21:47:45.351779Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5fe8a482-f6038b60-9537ce83-adc55959_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:45.359915Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:45.359940Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:45.359950Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342007503203867:8293]
2025-10-09T21:47:45.360002Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:45.360006Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:45.360296Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:45.360501Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:45.360659Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:45.360802Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:45.360997Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:45.361046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:45.365149Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1018 sessionId: src_id|5fe8a482-f6038b60-9537ce83-adc55959_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:45.367944Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:45.368246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.368271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.368326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:45.368410Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:45.373857Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:45.373874Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:45.373876Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:45.373878Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:45.373888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342007503203867:8293]
2025-10-09T21:47:45.373892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342007503203867:8293]
2025-10-09T21:47:45.373896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342007503203867:8293]
2025-10-09T21:47:45.373945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:45.373951Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:45.373955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.373958Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:45.373958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:45.373961Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:45.374114Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:45.425394Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342007865024485:9144]
2025-10-09T21:47:45.425659Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 1017, pipe: [1:7559342007865024485:9144]
2025-10-09T21:47:45.530519Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:45.538556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.538590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.538766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.538827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.538892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.538958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.539121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.539134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.539203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.539267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.539330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.539395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.539438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.539458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.539539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.539624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.539685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.539739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.539784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.539791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.539870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.539936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.539997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.540063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.540108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.540116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.540184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.540257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.540326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.540392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.540437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.540444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.540511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.540573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.540635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.540700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.540763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.540767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.540829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.540875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.540943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.541004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.541063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.541069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.541130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.541189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.541244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.541298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.541394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.541402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.541459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.541511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.541563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.541624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.541668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.541675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.541733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.541788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.541860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.541921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.541964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.541970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.542052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342007865024485:9144], now have 1 active actors on pipe
2025-10-09T21:47:45.542336Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342007865024485:9144]
2025-10-09T21:47:45.542345Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:45.542381Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:45.542385Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:45.542417Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1018 reason: error at write: Disconnected sessionId: src_id|5fe8a482-f6038b60-9537ce83-adc55959_0
2025-10-09T21:47:45.542639Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1018 sessionId: src_id|5fe8a482-f6038b60-9537ce83-adc55959_0 is DEAD
2025-10-09T21:47:45.542837Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fe8a482-f6038b60-9537ce83-adc55959_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:45.542853Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fe8a482-f6038b60-9537ce83-adc55959_0] Write session will restart in 0.007330s
2025-10-09T21:47:45.542879Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fe8a482-f6038b60-9537ce83-adc55959_0] Write session: Do CDS request
2025-10-09T21:47:45.542885Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fe8a482-f6038b60-9537ce83-adc55959_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:45.543150Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:45.551574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5fe8a482-f6038b60-9537ce83-adc55959_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:45.551737Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:45.551775Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1019
2025-10-09T21:47:45.551961Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1019 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:45.552029Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1019 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:45.552045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1019 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:45.552057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:45.552516Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1019 sessionId:  describe result for acl check
2025-10-09T21:47:45.552556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:45.552558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:45.552560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:45.552566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342007865024546:9243] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:45.553067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:45.553111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342007865024546:9243] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:45.554971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:45.555196Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:45.555207Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:45.555220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:45.555279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:45.555321Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342007503203997:8305]
2025-10-09T21:47:45.555944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:45.556121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342007865024546:9243] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:45.556251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:45.556299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:45.556636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:45.556684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.556690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.556692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.556693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.556694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.556695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.556696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.556698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.556699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.556704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.556715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.556719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.556723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.556727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.556731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.556735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.556739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.556743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.556750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.556760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.556766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.556770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.556774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.556779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.556783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.556787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.556792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.556801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.556805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.556809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.556815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.556824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.556828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.556833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.556842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.556847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.556877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:45.562896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342007865024546:9243] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:45.562917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342007865024546:9243] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:45.562920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342007865024546:9243] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:45.562929Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1019 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:45.564292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:45.565391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:45.565406Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:45.565408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:45.565412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:45.565417Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1018 [2:7559342007503203997:8305]
2025-10-09T21:47:45.565428Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:45.565444Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:45.565447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:45.565450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:45.565453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:45.565472Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:45.565483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:45.565491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:45.565553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.565562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.565564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.565567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:45.565632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:45.565646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.565662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.565684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.565701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:45.565728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342007865024492:2432], now have 1 active actors on pipe
2025-10-09T21:47:45.565736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342007865024590:9243], now have 1 active actors on pipe
2025-10-09T21:47:45.565745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342007503203991:8305]
2025-10-09T21:47:45.565747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342007503203991:8305]
2025-10-09T21:47:45.565749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342007503203991:8305]
2025-10-09T21:47:45.565762Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:45.565778Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:45.565781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:45.565786Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:45.565788Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:45.565760Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1018
2025-10-09T21:47:45.565795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:45.565805Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1018
2025-10-09T21:47:45.565799Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:45.565855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342007503203991:8305]
2025-10-09T21:47:45.565867Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:45.565876Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:45.565883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.565888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.565939Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3a119b3e-c3231635-91cb02a9-8bc083a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:45.565952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:45.565956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:45.565959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:45.565969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:45.565978Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:45.566002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:45.566193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.566207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.566230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:45.566328Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1019 partition: 0 MaxSeqNo: 25 sessionId: src_id|3a119b3e-c3231635-91cb02a9-8bc083a_0
2025-10-09T21:47:45.566727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fe8a482-f6038b60-9537ce83-adc55959_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3a119b3e-c3231635-91cb02a9-8bc083a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:45.569193Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a119b3e-c3231635-91cb02a9-8bc083a_0] Write session: try to update token
2025-10-09T21:47:45.569211Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a119b3e-c3231635-91cb02a9-8bc083a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:45.580440Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:45.580461Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:45.580463Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:45.580465Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:45.580475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342007503203991:8305]
2025-10-09T21:47:45.580480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342007503203991:8305]
2025-10-09T21:47:45.580483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342007503203991:8305]
2025-10-09T21:47:45.580525Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:45.580527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:45.580531Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:45.580535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.580535Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:45.580538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:45.580666Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:45.583907Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1019 sessionId: src_id|3a119b3e-c3231635-91cb02a9-8bc083a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:45.586526Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:45.642521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342007865024617:9144]
2025-10-09T21:47:45.642885Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 1018, pipe: [1:7559342007865024617:9144]
2025-10-09T21:47:45.736729Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:45.737023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.737039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.737088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342007865024617:9144], now have 1 active actors on pipe
2025-10-09T21:47:45.737370Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:45.737372Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342007865024617:9144]
2025-10-09T21:47:45.737398Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:45.737413Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:45.737484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1019 reason: error at write: Disconnected sessionId: src_id|3a119b3e-c3231635-91cb02a9-8bc083a_0
2025-10-09T21:47:45.737672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1019 sessionId: src_id|3a119b3e-c3231635-91cb02a9-8bc083a_0 is DEAD
2025-10-09T21:47:45.737784Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:45.737855Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a119b3e-c3231635-91cb02a9-8bc083a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:45.737879Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a119b3e-c3231635-91cb02a9-8bc083a_0] Write session will restart in 0.008428s
2025-10-09T21:47:45.737913Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a119b3e-c3231635-91cb02a9-8bc083a_0] Write session: Do CDS request
2025-10-09T21:47:45.737921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a119b3e-c3231635-91cb02a9-8bc083a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:45.747576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a119b3e-c3231635-91cb02a9-8bc083a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:45.747680Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:45.747699Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1020
2025-10-09T21:47:45.747841Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1020 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:45.747878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1020 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:45.747885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1020 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:45.747888Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:45.748381Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1020 sessionId:  describe result for acl check
2025-10-09T21:47:45.748433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:45.748441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:45.748444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:45.748451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342007865024637:9254] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:45.749027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342007865024637:9254] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:45.751321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:45.752001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342007865024637:9254] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:45.752946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:45.753172Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:45.753181Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:45.753194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:45.753270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:45.753322Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342007503204104:8311]
2025-10-09T21:47:45.753911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:45.754219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:45.754264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:45.754645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:45.754703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.754709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.754710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.754711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.754713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.754714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.754715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.754716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.754718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.754723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.754734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.754737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.754742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.754746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.754750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.754754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.754759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.754763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.754770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.754779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.754786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.754792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.754798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.754803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.754796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342007865024637:9254] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:45.754807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.754811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342007865024637:9254] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:45.754815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342007865024637:9254] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:45.754813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.754823Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1020 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:45.754818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.754823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.754828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.754833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.754840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.754845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.754850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.754855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.754859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.754865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.754893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:45.761421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:45.762786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:45.762803Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:45.762806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:45.762812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:45.762819Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1019 [2:7559342007503204104:8311]
2025-10-09T21:47:45.762837Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:45.762859Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:45.762862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:45.762866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:45.762869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:45.762889Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:45.762903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:45.762911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:45.762978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.762985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.762987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.762990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:45.763058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:45.763067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.763080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.763092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.763110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:45.763134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342007865024714:9254], now have 1 active actors on pipe
2025-10-09T21:47:45.763138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342007865024620:2432], now have 1 active actors on pipe
2025-10-09T21:47:45.763147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342007503204099:8311]
2025-10-09T21:47:45.763149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342007503204099:8311]
2025-10-09T21:47:45.763151Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342007503204099:8311]
2025-10-09T21:47:45.763168Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:45.763183Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:45.763190Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:45.763197Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:45.763201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:45.763201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:45.763204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:45.763208Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1019
2025-10-09T21:47:45.763229Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1019
2025-10-09T21:47:45.763244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342007503204099:8311]
2025-10-09T21:47:45.763247Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:45.763256Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:45.763287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.763292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.763329Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|784de717-eb996bda-9805e85a-4f6c13cd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:45.763345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:45.763352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:45.763355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:45.763367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:45.763377Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:45.763396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:45.763562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.763570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.763592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:45.763694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1020 partition: 0 MaxSeqNo: 25 sessionId: src_id|784de717-eb996bda-9805e85a-4f6c13cd_0
2025-10-09T21:47:45.764062Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a119b3e-c3231635-91cb02a9-8bc083a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|784de717-eb996bda-9805e85a-4f6c13cd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:45.766620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|784de717-eb996bda-9805e85a-4f6c13cd_0] Write session: try to update token
2025-10-09T21:47:45.766640Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|784de717-eb996bda-9805e85a-4f6c13cd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:45.775274Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:45.775289Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:45.775299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342007503204099:8311]
2025-10-09T21:47:45.775341Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:45.775348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:45.775535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:45.775674Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:45.775806Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:45.775945Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:45.776144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:45.776193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:45.780833Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1020 sessionId: src_id|784de717-eb996bda-9805e85a-4f6c13cd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:45.783599Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:45.783845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.783868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.783913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:45.784044Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:45.788494Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:45.788508Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:45.788510Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:45.788512Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:45.788523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342007503204099:8311]
2025-10-09T21:47:45.788527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342007503204099:8311]
2025-10-09T21:47:45.788534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342007503204099:8311]
2025-10-09T21:47:45.788580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:45.788585Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:45.788591Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:45.788592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.788594Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:45.788595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:45.788740Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:45.837565Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342007865024726:9144]
2025-10-09T21:47:45.837890Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 1019, pipe: [1:7559342007865024726:9144]
2025-10-09T21:47:45.949638Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:45.957784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.957820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.957995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.958097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.958169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.958234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.958281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.958290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.958360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.958422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.958500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.958567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.958616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.958624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.958702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.958765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.958832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.958918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.958975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.958984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.959052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.959119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.959204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.959276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.959318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.959325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.959404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.959471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.959533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.959594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.959639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.959646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.959710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.959766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.959827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.959889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.959936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.959940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.960003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.960062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.960123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.960185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.960247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.960249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.960313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.960369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.960425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.960492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.960531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.960533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.960601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.960654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.960718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.960780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.960827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.960834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.960899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:45.960958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:45.961024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:45.961083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:45.961127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:45.961130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:45.961199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342007865024726:9144], now have 1 active actors on pipe
2025-10-09T21:47:45.961441Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342007865024726:9144]
2025-10-09T21:47:45.961441Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:45.961446Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:45.961478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1020 reason: error at write: Disconnected sessionId: src_id|784de717-eb996bda-9805e85a-4f6c13cd_0
2025-10-09T21:47:45.961483Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:45.961721Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1020 sessionId: src_id|784de717-eb996bda-9805e85a-4f6c13cd_0 is DEAD
2025-10-09T21:47:45.961864Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|784de717-eb996bda-9805e85a-4f6c13cd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:45.961872Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|784de717-eb996bda-9805e85a-4f6c13cd_0] Write session will restart in 0.009089s
2025-10-09T21:47:45.961896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|784de717-eb996bda-9805e85a-4f6c13cd_0] Write session: Do CDS request
2025-10-09T21:47:45.961903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|784de717-eb996bda-9805e85a-4f6c13cd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:45.962268Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:45.972409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:45.972607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|784de717-eb996bda-9805e85a-4f6c13cd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:45.972680Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:45.972707Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1021
2025-10-09T21:47:45.972857Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1021 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:45.972897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1021 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:45.972908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1021 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:45.972912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:45.973345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1021 sessionId:  describe result for acl check
2025-10-09T21:47:45.973387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:45.973394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:45.973396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:45.973402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342007865024791:9259] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:45.973995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342007865024791:9259] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:45.974545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:45.974760Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:45.974769Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:45.974782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:45.974835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:45.974874Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342007503204200:8313]
2025-10-09T21:47:45.975374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:45.975677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:45.975718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:45.976019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:45.976062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.976063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.976064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.976066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.976067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.976068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.976069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.976070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.976071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.976076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.976086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.976090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.976094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.976097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.976101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.976105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.976108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.976112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.976117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.976125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:45.976132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.976137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:45.976142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.976146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:45.976150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.976155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:45.976160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.976165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:45.976169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.976174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:45.976180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.976186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.976194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.976199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.976184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342007865024791:9259] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:45.976205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.976213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.976239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:45.982083Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342007865024791:9259] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:45.982103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342007865024791:9259] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:45.982107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342007865024791:9259] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:45.982117Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1021 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:45.982414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:45.983635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:45.983648Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:45.983651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:45.983655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:45.983664Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1020 [2:7559342007503204200:8313]
2025-10-09T21:47:45.983678Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:45.983696Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:45.983700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:45.983703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:45.983706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:45.983724Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:45.983736Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:45.983743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:45.983802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.983812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.983815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.983818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:45.983898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:45.983907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.983919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:45.983934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:45.983949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:45.983977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342007865024732:2432], now have 1 active actors on pipe
2025-10-09T21:47:45.983987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342007865024826:9259], now have 1 active actors on pipe
2025-10-09T21:47:45.983995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342007503204194:8313]
2025-10-09T21:47:45.983997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342007503204194:8313]
2025-10-09T21:47:45.983999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342007503204194:8313]
2025-10-09T21:47:45.984010Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:45.984024Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:45.984039Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:45.984040Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:45.984044Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:45.984044Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:45.984046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:45.984085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342007503204194:8313]
2025-10-09T21:47:45.984093Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:45.984099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:45.984569Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1020
2025-10-09T21:47:45.984604Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1020
2025-10-09T21:47:45.984749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.984766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.984798Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d4b220df-db006bb1-da12925f-87f3f37d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:45.984807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:45.984812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:45.984814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:45.984826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:45.984838Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:45.984855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:45.985013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:45.985019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:45.985031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:45.985135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1021 partition: 0 MaxSeqNo: 25 sessionId: src_id|d4b220df-db006bb1-da12925f-87f3f37d_0
2025-10-09T21:47:45.985471Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|784de717-eb996bda-9805e85a-4f6c13cd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d4b220df-db006bb1-da12925f-87f3f37d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:45.987837Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d4b220df-db006bb1-da12925f-87f3f37d_0] Write session: try to update token
2025-10-09T21:47:45.987854Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d4b220df-db006bb1-da12925f-87f3f37d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:45.998417Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:45.998432Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:45.998434Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:45.998436Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:45.998446Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342007503204194:8313]
2025-10-09T21:47:45.998450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342007503204194:8313]
2025-10-09T21:47:45.998454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342007503204194:8313]
2025-10-09T21:47:45.998507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:45.998515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:45.998518Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:45.998524Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:45.998525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:45.998528Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:45.998669Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:46.002946Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1021 sessionId: src_id|d4b220df-db006bb1-da12925f-87f3f37d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:46.006025Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:46.061846Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342012159992130:9144]
2025-10-09T21:47:46.062157Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 1020, pipe: [1:7559342012159992130:9144]
2025-10-09T21:47:46.161115Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:46.161449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:46.161465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:46.161512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342012159992130:9144], now have 1 active actors on pipe
2025-10-09T21:47:46.161722Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:46.161838Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:46.161841Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342012159992130:9144]
2025-10-09T21:47:46.161867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:46.161973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1021 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d4b220df-db006bb1-da12925f-87f3f37d_0
2025-10-09T21:47:46.162197Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1021 sessionId: src_id|d4b220df-db006bb1-da12925f-87f3f37d_0 is DEAD
2025-10-09T21:47:46.162355Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4b220df-db006bb1-da12925f-87f3f37d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:46.162374Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4b220df-db006bb1-da12925f-87f3f37d_0] Write session will restart in 0.008301s
2025-10-09T21:47:46.162399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4b220df-db006bb1-da12925f-87f3f37d_0] Write session: Do CDS request
2025-10-09T21:47:46.162406Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4b220df-db006bb1-da12925f-87f3f37d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:46.170753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:46.171578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d4b220df-db006bb1-da12925f-87f3f37d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:46.171702Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:46.171720Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1022
2025-10-09T21:47:46.171854Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1022 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:46.171896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1022 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:46.171906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1022 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:46.171910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:46.172371Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1022 sessionId:  describe result for acl check
2025-10-09T21:47:46.172415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:46.172422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:46.172424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:46.172431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342012159992195:9263] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:46.172487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:46.172693Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:46.172704Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:46.172715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:46.172768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:46.172810Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342011798171591:8315]
2025-10-09T21:47:46.173032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342012159992195:9263] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:46.173410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:46.173714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:46.173769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:46.174110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:46.174166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.174172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.174173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.174174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.174175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.174177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.174178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.174179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.174180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.174185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.174197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.174201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.174205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.174209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.174231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.174239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.174242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.174246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.174257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.174264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.174270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.174275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.174282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.174287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.174291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.174296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.174306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.174311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.174317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.174327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.174332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.174341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.174350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.174356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.174366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.174371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.174405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:46.175399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342012159992195:9263] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:46.180438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342012159992195:9263] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:46.180457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342012159992195:9263] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:46.180461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342012159992195:9263] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:46.180470Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1022 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:46.180832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:46.181934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:46.181953Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:46.181956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:46.181960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:46.181968Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1021 [2:7559342011798171591:8315]
2025-10-09T21:47:46.181983Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:46.182003Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:46.182006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:46.182009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:46.182012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:46.182057Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:46.182068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:46.182076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:46.182140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.182147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.182148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.182150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:46.182209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:46.182217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.182231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.182249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.182266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:46.182293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342012159992135:2432], now have 1 active actors on pipe
2025-10-09T21:47:46.182297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342012159992234:9263], now have 1 active actors on pipe
2025-10-09T21:47:46.182305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342011798171586:8315]
2025-10-09T21:47:46.182307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342011798171586:8315]
2025-10-09T21:47:46.182309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342011798171586:8315]
2025-10-09T21:47:46.182325Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:46.182344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:46.182345Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:46.182349Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:46.182351Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:46.182363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:46.182367Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:46.182406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342011798171586:8315]
2025-10-09T21:47:46.182416Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:46.182384Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1021
2025-10-09T21:47:46.182428Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:46.182387Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1021
2025-10-09T21:47:46.182450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:46.182455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:46.182488Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c0c2f25-9d179a14-db7b7c26-c3e83fcf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:46.182501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:46.182510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:46.182513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:46.182523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:46.182533Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:46.182549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:46.182738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:46.182751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:46.182774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:46.182865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1022 partition: 0 MaxSeqNo: 25 sessionId: src_id|c0c2f25-9d179a14-db7b7c26-c3e83fcf_0
2025-10-09T21:47:46.183243Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4b220df-db006bb1-da12925f-87f3f37d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c0c2f25-9d179a14-db7b7c26-c3e83fcf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:46.185914Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c0c2f25-9d179a14-db7b7c26-c3e83fcf_0] Write session: try to update token
2025-10-09T21:47:46.185937Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c0c2f25-9d179a14-db7b7c26-c3e83fcf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:46.194239Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:46.194250Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:46.194262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342011798171586:8315]
2025-10-09T21:47:46.194300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:46.194312Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:46.194462Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:46.194607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:46.194750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:46.194897Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:46.195068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:46.195101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:46.199871Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1022 sessionId: src_id|c0c2f25-9d179a14-db7b7c26-c3e83fcf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:46.202388Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:46.202720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:46.202736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:46.202770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:46.202840Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:46.207414Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:46.207432Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:46.207434Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:46.207436Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:46.207446Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342011798171586:8315]
2025-10-09T21:47:46.207450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342011798171586:8315]
2025-10-09T21:47:46.207454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342011798171586:8315]
2025-10-09T21:47:46.207497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:46.207509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.207510Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:46.207516Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:46.207520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:46.207520Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:46.207648Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:46.261978Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342012159992246:9144]
2025-10-09T21:47:46.262273Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 1021, pipe: [1:7559342012159992246:9144]
2025-10-09T21:47:46.361779Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:46.369155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:46.369179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:46.369329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:46.369389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:46.369448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:46.369505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:46.369575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:46.369583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:46.369659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:46.369718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:46.369774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:46.369849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:46.369895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:46.369907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:46.369973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:46.370049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:46.370112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:46.370176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:46.370215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:46.370220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:46.370278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:46.370333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:46.370397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:46.370456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:46.370494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:46.370499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:46.370555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:46.370607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:46.370672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:46.370731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:46.370778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:46.370785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:46.370843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:46.370907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:46.370961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:46.371025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:46.371072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:46.371077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:46.371134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:46.371192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:46.371251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:46.371305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:46.371344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:46.371350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:46.371409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:46.371468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:46.371526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:46.371584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:46.371628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:46.371633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:46.371691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:46.371744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:46.371827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:46.371885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:46.371926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:46.371934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:46.371989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:46.372041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:46.372093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:46.372145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:46.372189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:46.372197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:46.372250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342012159992246:9144], now have 1 active actors on pipe
2025-10-09T21:47:46.372482Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:46.372502Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342012159992246:9144]
2025-10-09T21:47:46.372542Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:46.372553Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:46.372582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1022 reason: error at write: Disconnected sessionId: src_id|c0c2f25-9d179a14-db7b7c26-c3e83fcf_0
2025-10-09T21:47:46.372783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1022 sessionId: src_id|c0c2f25-9d179a14-db7b7c26-c3e83fcf_0 is DEAD
2025-10-09T21:47:46.372922Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0c2f25-9d179a14-db7b7c26-c3e83fcf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:46.372933Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0c2f25-9d179a14-db7b7c26-c3e83fcf_0] Write session will restart in 0.008563s
2025-10-09T21:47:46.372962Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0c2f25-9d179a14-db7b7c26-c3e83fcf_0] Write session: Do CDS request
2025-10-09T21:47:46.372968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0c2f25-9d179a14-db7b7c26-c3e83fcf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:46.373103Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:46.382507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c0c2f25-9d179a14-db7b7c26-c3e83fcf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:46.382589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:46.382607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1023
2025-10-09T21:47:46.382721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1023 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:46.382753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1023 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:46.382737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:46.382759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1023 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:46.382765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:46.383186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1023 sessionId:  describe result for acl check
2025-10-09T21:47:46.383223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:46.383229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:46.383230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:46.383236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342012159992309:9268] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:46.383765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342012159992309:9268] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:46.384328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:46.384491Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:46.384499Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:46.384507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:46.384550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:46.384584Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342011798171689:8317]
2025-10-09T21:47:46.384984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:46.385200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:46.385238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:46.385471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:46.385504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.385513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.385514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.385515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.385516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.385517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.385518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.385519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.385520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.385524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.385533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.385536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.385539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.385543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.385547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.385550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.385553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.385556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.385563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.385570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.385577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.385582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.385586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.385591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.385596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.385600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.385604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.385609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.385613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.385618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.385622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.385627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.385631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.385637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.385642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.385647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.385675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:46.385850Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342012159992309:9268] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:46.391402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342012159992309:9268] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:46.391419Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342012159992309:9268] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:46.391423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342012159992309:9268] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:46.391432Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1023 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:46.391725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:46.392826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:46.392841Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:46.392844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:46.392848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:46.392856Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1022 [2:7559342011798171689:8317]
2025-10-09T21:47:46.392869Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:46.392888Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:46.392896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:46.392899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:46.392902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:46.392921Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:46.392930Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:46.392944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:46.392999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.393005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.393006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.393009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:46.393077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:46.393087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.393100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.393121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.393138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:46.393163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342012159992249:2432], now have 1 active actors on pipe
2025-10-09T21:47:46.393170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342012159992348:9268], now have 1 active actors on pipe
2025-10-09T21:47:46.393178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342011798171683:8317]
2025-10-09T21:47:46.393184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342011798171683:8317]
2025-10-09T21:47:46.393186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342011798171683:8317]
2025-10-09T21:47:46.393198Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:46.393212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:46.393227Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:46.393217Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1022
2025-10-09T21:47:46.393228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:46.393220Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1022
2025-10-09T21:47:46.393232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:46.393236Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:46.393238Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:46.393265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342011798171683:8317]
2025-10-09T21:47:46.393267Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:46.393270Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:46.393291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:46.393300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:46.393346Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|11909e76-48bcfcc6-3759361d-165cc68e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:46.393358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:46.393369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:46.393372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:46.393382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:46.393392Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:46.393419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:46.393550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:46.393558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:46.393571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:46.393658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1023 partition: 0 MaxSeqNo: 25 sessionId: src_id|11909e76-48bcfcc6-3759361d-165cc68e_0
2025-10-09T21:47:46.394056Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0c2f25-9d179a14-db7b7c26-c3e83fcf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|11909e76-48bcfcc6-3759361d-165cc68e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:46.396550Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|11909e76-48bcfcc6-3759361d-165cc68e_0] Write session: try to update token
2025-10-09T21:47:46.396566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|11909e76-48bcfcc6-3759361d-165cc68e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:46.407362Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:46.407379Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:46.407381Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:46.407383Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:46.407393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342011798171683:8317]
2025-10-09T21:47:46.407397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342011798171683:8317]
2025-10-09T21:47:46.407401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342011798171683:8317]
2025-10-09T21:47:46.407445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:46.407450Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:46.407453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.407456Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:46.407456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:46.407460Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:46.407595Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:46.410281Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1023 sessionId: src_id|11909e76-48bcfcc6-3759361d-165cc68e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:46.412668Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:46.472935Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342012159992355:9144]
2025-10-09T21:47:46.473224Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 1022, pipe: [1:7559342012159992355:9144]
2025-10-09T21:47:46.564612Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:46.565108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:46.565124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:46.565171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342012159992355:9144], now have 1 active actors on pipe
2025-10-09T21:47:46.565402Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:46.565493Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342012159992355:9144]
2025-10-09T21:47:46.565521Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:46.565525Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:46.565613Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1023 reason: status is not ok: tablet will be restarted right now sessionId: src_id|11909e76-48bcfcc6-3759361d-165cc68e_0
2025-10-09T21:47:46.565830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1023 sessionId: src_id|11909e76-48bcfcc6-3759361d-165cc68e_0 is DEAD
2025-10-09T21:47:46.565995Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11909e76-48bcfcc6-3759361d-165cc68e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:46.566012Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11909e76-48bcfcc6-3759361d-165cc68e_0] Write session will restart in 0.005878s
2025-10-09T21:47:46.566049Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11909e76-48bcfcc6-3759361d-165cc68e_0] Write session: Do CDS request
2025-10-09T21:47:46.566057Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11909e76-48bcfcc6-3759361d-165cc68e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:46.573571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|11909e76-48bcfcc6-3759361d-165cc68e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:46.573707Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:46.573725Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1024
2025-10-09T21:47:46.573866Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1024 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:46.573894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1024 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:46.573904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1024 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:46.573908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:46.574363Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1024 sessionId:  describe result for acl check
2025-10-09T21:47:46.574403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:46.574410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:46.574412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:46.574418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342012159992418:9272] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:46.574907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:46.575031Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342012159992418:9272] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:46.576712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:46.576924Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:46.576936Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:46.576947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:46.576992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:46.577034Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342011798171818:8329]
2025-10-09T21:47:46.577372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342012159992418:9272] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:46.577599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:46.577895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:46.577950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:46.578303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:46.578352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.578358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.578359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.578361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.578362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.578363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.578364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.578365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.578366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.578371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.578381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.578384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.578388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.578391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.578394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.578398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.578401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.578404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.578410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.578418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.578424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.578429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.578433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.578437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.578442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.578448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.578452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.578457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.578461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.578466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.578475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.578480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.578491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.578501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.578505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.578510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.578541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:46.584486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342012159992418:9272] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:46.584508Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342012159992418:9272] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:46.584512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342012159992418:9272] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:46.584522Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1024 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:46.585912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:46.586985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:46.587003Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:46.587005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:46.587009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:46.587015Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1023 [2:7559342011798171818:8329]
2025-10-09T21:47:46.587027Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:46.587042Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:46.587048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:46.587050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:46.587052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:46.587066Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:46.587075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:46.587081Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:46.587142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.587150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.587151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.587153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:46.587219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:46.587229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.587240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.587252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.587269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:46.587292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342012159992459:9272], now have 1 active actors on pipe
2025-10-09T21:47:46.587300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342012159992359:2432], now have 1 active actors on pipe
2025-10-09T21:47:46.587308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342011798171813:8329]
2025-10-09T21:47:46.587310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342011798171813:8329]
2025-10-09T21:47:46.587312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342011798171813:8329]
2025-10-09T21:47:46.587329Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:46.587321Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1023
2025-10-09T21:47:46.587344Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:46.587327Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1023
2025-10-09T21:47:46.587347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:46.587361Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:46.587364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:46.587366Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:46.587369Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:46.587419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:46.587429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:46.587442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342011798171813:8329]
2025-10-09T21:47:46.587448Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:46.587462Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:46.587481Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c95001a9-8f174c49-c0e4db00-2c64de71_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:46.587493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:46.587501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:46.587504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:46.587516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:46.587529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:46.587551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:46.587719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:46.587731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:46.587752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:46.587849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1024 partition: 0 MaxSeqNo: 25 sessionId: src_id|c95001a9-8f174c49-c0e4db00-2c64de71_0
2025-10-09T21:47:46.588268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11909e76-48bcfcc6-3759361d-165cc68e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c95001a9-8f174c49-c0e4db00-2c64de71_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:46.590747Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c95001a9-8f174c49-c0e4db00-2c64de71_0] Write session: try to update token
2025-10-09T21:47:46.590764Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c95001a9-8f174c49-c0e4db00-2c64de71_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:46.604528Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:46.604542Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:46.604545Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:46.604547Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:46.604557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342011798171813:8329]
2025-10-09T21:47:46.604561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342011798171813:8329]
2025-10-09T21:47:46.604565Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342011798171813:8329]
2025-10-09T21:47:46.604614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:46.604624Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:46.604625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.604628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:46.604630Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:46.604634Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:46.604767Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:46.604755Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1024 sessionId: src_id|c95001a9-8f174c49-c0e4db00-2c64de71_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:46.607192Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:46.665512Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342012159992488:9144]
2025-10-09T21:47:46.665792Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 1023, pipe: [1:7559342012159992488:9144]
2025-10-09T21:47:46.762847Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:46.763263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:46.763280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:46.763327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342012159992488:9144], now have 1 active actors on pipe
2025-10-09T21:47:46.763532Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:46.763538Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:46.763557Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:46.763564Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342012159992488:9144]
2025-10-09T21:47:46.763586Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:46.763635Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1024 reason: error at write: Disconnected sessionId: src_id|c95001a9-8f174c49-c0e4db00-2c64de71_0
2025-10-09T21:47:46.763826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1024 sessionId: src_id|c95001a9-8f174c49-c0e4db00-2c64de71_0 is DEAD
2025-10-09T21:47:46.763978Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c95001a9-8f174c49-c0e4db00-2c64de71_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:46.763990Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c95001a9-8f174c49-c0e4db00-2c64de71_0] Write session will restart in 0.007262s
2025-10-09T21:47:46.764018Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c95001a9-8f174c49-c0e4db00-2c64de71_0] Write session: Do CDS request
2025-10-09T21:47:46.764024Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c95001a9-8f174c49-c0e4db00-2c64de71_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:46.772598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c95001a9-8f174c49-c0e4db00-2c64de71_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:46.772746Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:46.772766Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1025
2025-10-09T21:47:46.772899Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1025 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:46.772944Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1025 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:46.772955Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1025 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:46.772959Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:46.773417Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1025 sessionId:  describe result for acl check
2025-10-09T21:47:46.773453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:46.773461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:46.773463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:46.773469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342012159992554:9285] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:46.774010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:46.774080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342012159992554:9285] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:46.775747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:46.775941Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:46.775951Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:46.775963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:46.776012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:46.776051Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342011798171929:8335]
2025-10-09T21:47:46.776495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342012159992554:9285] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:46.776663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:46.776973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:46.777015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:46.777310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:46.777359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.777365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.777366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.777367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.777368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.777369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.777370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.777371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.777372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.777377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.777393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.777397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.777401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.777407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.777413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.777417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.777420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.777424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.777434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.777447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.777457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.777468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.777477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.777489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.777493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.777498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.777513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.777523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.777531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.777538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.777546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.777555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.777563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.777569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.777577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.777583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.777616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:46.783801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342012159992554:9285] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:46.783825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342012159992554:9285] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:46.783829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342012159992554:9285] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:46.783839Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1025 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:46.785438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:46.786560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:46.786574Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:46.786576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:46.786580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:46.786587Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1024 [2:7559342011798171929:8335]
2025-10-09T21:47:46.786600Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:46.786627Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:46.786630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:46.786634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:46.786636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:46.786653Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:46.786663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:46.786671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:46.786731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.786736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.786738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.786741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:46.786825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:46.786833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.786844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.786854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.786868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:46.786893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342012159992595:9285], now have 1 active actors on pipe
2025-10-09T21:47:46.786900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342012159992494:2432], now have 1 active actors on pipe
2025-10-09T21:47:46.786907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342011798171923:8335]
2025-10-09T21:47:46.786909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342011798171923:8335]
2025-10-09T21:47:46.786912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342011798171923:8335]
2025-10-09T21:47:46.786923Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:46.786940Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:46.786944Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:46.786948Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:46.786950Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:46.786956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:46.786960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:46.786962Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1024
2025-10-09T21:47:46.786965Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1024
2025-10-09T21:47:46.786997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342011798171923:8335]
2025-10-09T21:47:46.787007Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:46.787009Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:46.787022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:46.787027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:46.787075Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|be481d31-ec498c7f-c8326e96-45bbce4a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:46.787087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:46.787090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:46.787093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:46.787102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:46.787111Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:46.787128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:46.787308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:46.787315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:46.787328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:46.787398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1025 partition: 0 MaxSeqNo: 25 sessionId: src_id|be481d31-ec498c7f-c8326e96-45bbce4a_0
2025-10-09T21:47:46.787752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c95001a9-8f174c49-c0e4db00-2c64de71_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|be481d31-ec498c7f-c8326e96-45bbce4a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:46.790260Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be481d31-ec498c7f-c8326e96-45bbce4a_0] Write session: try to update token
2025-10-09T21:47:46.790280Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be481d31-ec498c7f-c8326e96-45bbce4a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:46.804081Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:46.804098Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:46.804101Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:46.804103Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:46.804117Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342011798171923:8335]
2025-10-09T21:47:46.804123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342011798171923:8335]
2025-10-09T21:47:46.804128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342011798171923:8335]
2025-10-09T21:47:46.804174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:46.804181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.804184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:46.804187Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:46.804194Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:46.804199Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:46.804337Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:46.805552Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1025 sessionId: src_id|be481d31-ec498c7f-c8326e96-45bbce4a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:46.808268Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:46.863628Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342012159992605:9144]
2025-10-09T21:47:46.863927Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 1024, pipe: [1:7559342012159992605:9144]
2025-10-09T21:47:46.962402Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:46.962775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:46.962792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:46.962842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342012159992605:9144], now have 1 active actors on pipe
2025-10-09T21:47:46.963101Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:46.963111Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:46.963115Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:46.963161Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342012159992605:9144]
2025-10-09T21:47:46.963178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1025 reason: error at write: Disconnected sessionId: src_id|be481d31-ec498c7f-c8326e96-45bbce4a_0
2025-10-09T21:47:46.963397Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1025 sessionId: src_id|be481d31-ec498c7f-c8326e96-45bbce4a_0 is DEAD
2025-10-09T21:47:46.963533Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be481d31-ec498c7f-c8326e96-45bbce4a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:46.963544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be481d31-ec498c7f-c8326e96-45bbce4a_0] Write session will restart in 0.006852s
2025-10-09T21:47:46.963570Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be481d31-ec498c7f-c8326e96-45bbce4a_0] Write session: Do CDS request
2025-10-09T21:47:46.963580Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be481d31-ec498c7f-c8326e96-45bbce4a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:46.963054Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:46.971586Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be481d31-ec498c7f-c8326e96-45bbce4a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:46.971697Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:46.971716Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1026
2025-10-09T21:47:46.971847Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1026 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:46.971897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1026 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:46.971909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1026 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:46.971913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:46.972339Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1026 sessionId:  describe result for acl check
2025-10-09T21:47:46.972378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:46.972385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:46.972388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:46.972394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342012159992671:9290] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:46.972980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342012159992671:9290] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:46.973011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:46.975006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:46.975169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342012159992671:9290] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:46.975250Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:46.975263Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:46.975275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:46.975334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:46.975379Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342011798172030:8337]
2025-10-09T21:47:46.975966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:46.976273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:46.976339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:46.976698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:46.976754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.976759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.976761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.976762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.976764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.976765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.976767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.976768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.976769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.976776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.976786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.976790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.976795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.976799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.976803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.976807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.976811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.976815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.976822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.976830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:46.976836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.976842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:46.976846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.976851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:46.976855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.976859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:46.976864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.976868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:46.976872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.976878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:46.976888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.976893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.976898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.976903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.976912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.976918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.976952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:46.983081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342012159992671:9290] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:46.983105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342012159992671:9290] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:46.983109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342012159992671:9290] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:46.983120Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1026 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:46.985306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:46.986580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:46.986600Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:46.986604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:46.986609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:46.986617Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1025 [2:7559342011798172030:8337]
2025-10-09T21:47:46.986639Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:46.986663Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:46.986670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:46.986673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:46.986676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:46.986698Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:46.986706Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:46.986715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:46.986776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.986781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.986782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.986784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:46.987019Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1025
2025-10-09T21:47:46.987029Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1025
2025-10-09T21:47:46.986949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:46.986957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:46.986971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:46.986981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:46.986997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:46.987021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342012159992610:2432], now have 1 active actors on pipe
2025-10-09T21:47:46.987025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342012159992709:9290], now have 1 active actors on pipe
2025-10-09T21:47:46.987033Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342011798172024:8337]
2025-10-09T21:47:46.987036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342011798172024:8337]
2025-10-09T21:47:46.987038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342011798172024:8337]
2025-10-09T21:47:46.987048Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:46.987060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:46.987064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:46.987065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:46.987066Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:46.987075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:46.987078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:46.987093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:46.987096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:46.987148Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342011798172024:8337]
2025-10-09T21:47:46.987157Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:46.987179Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bb6a7d7e-17bb9b9c-403a1c72-d50fc4d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:46.987179Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:46.987187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:46.987192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:46.987194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:46.987206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:46.987217Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:46.987257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:46.987394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:46.987402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:46.987419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:46.987512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1026 partition: 0 MaxSeqNo: 25 sessionId: src_id|bb6a7d7e-17bb9b9c-403a1c72-d50fc4d_0
2025-10-09T21:47:46.987924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be481d31-ec498c7f-c8326e96-45bbce4a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bb6a7d7e-17bb9b9c-403a1c72-d50fc4d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:46.990697Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb6a7d7e-17bb9b9c-403a1c72-d50fc4d_0] Write session: try to update token
2025-10-09T21:47:46.990720Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb6a7d7e-17bb9b9c-403a1c72-d50fc4d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:47.005887Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1026 sessionId: src_id|bb6a7d7e-17bb9b9c-403a1c72-d50fc4d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:47.008287Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:47.008308Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:47.008319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342011798172024:8337]
2025-10-09T21:47:47.008374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:47.008375Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:47.008582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:47.008595Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:47.008766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:47.008936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:47.009087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:47.009578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:47.009641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:47.009661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:47.009685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:47.009727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:47.009864Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:47.019296Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:47.019321Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:47.019324Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:47.019326Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:47.019337Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342011798172024:8337]
2025-10-09T21:47:47.019340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342011798172024:8337]
2025-10-09T21:47:47.019345Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342011798172024:8337]
2025-10-09T21:47:47.019389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:47.019388Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:47.019393Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:47.019397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.019398Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:47.019399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:47.019534Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:47.063289Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342016454960013:9144]
2025-10-09T21:47:47.063598Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 1025, pipe: [1:7559342016454960013:9144]
2025-10-09T21:47:47.187161Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:47.195233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:47.195264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:47.195438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.195506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.195573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.195636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.195686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.195696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.195765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.195819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.195887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.195953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.196000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.196014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.196078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.196137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.196199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.196260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.196319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.196326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.196414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.196477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.196535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.196630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.196682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.196688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.196750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.196802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.196856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.196908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.196950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.196957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.197018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.197068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.197119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.197171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.197208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.197215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.197276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.197381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.197434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.197492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.197536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.197541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.197592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.197638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.197689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.197741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.197788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.197794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.197857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.197904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.197964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.198022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.198098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.198104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.198160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.198207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.198263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.198320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.198359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.198364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.198418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342016454960013:9144], now have 1 active actors on pipe
2025-10-09T21:47:47.198608Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:47.198626Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:47.198653Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1026 reason: error at write: Disconnected sessionId: src_id|bb6a7d7e-17bb9b9c-403a1c72-d50fc4d_0
2025-10-09T21:47:47.198850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1026 sessionId: src_id|bb6a7d7e-17bb9b9c-403a1c72-d50fc4d_0 is DEAD
2025-10-09T21:47:47.198930Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342016454960013:9144]
2025-10-09T21:47:47.198962Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:47.198999Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb6a7d7e-17bb9b9c-403a1c72-d50fc4d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:47.199011Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb6a7d7e-17bb9b9c-403a1c72-d50fc4d_0] Write session will restart in 0.006682s
2025-10-09T21:47:47.199048Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb6a7d7e-17bb9b9c-403a1c72-d50fc4d_0] Write session: Do CDS request
2025-10-09T21:47:47.199054Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb6a7d7e-17bb9b9c-403a1c72-d50fc4d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:47.199121Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:47.206497Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb6a7d7e-17bb9b9c-403a1c72-d50fc4d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:47.206586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:47.206606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1027
2025-10-09T21:47:47.206743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1027 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:47.206774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1027 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:47.206793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1027 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:47.206802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:47.207321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1027 sessionId:  describe result for acl check
2025-10-09T21:47:47.207361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:47.207368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:47.207370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:47.207376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342016454960082:9295] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:47.207915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342016454960082:9295] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:47.208718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:47.210391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:47.210480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342016454960082:9295] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:47.210596Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:47.210607Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:47.210620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:47.210682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:47.210724Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342016093139426:8339]
2025-10-09T21:47:47.211321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:47.211621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:47.211673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:47.211962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:47.211995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:47.212000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:47.212000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:47.212001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:47.212002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:47.212002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:47.212003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.212003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.212004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.212008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:47.212022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:47.212024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:47.212026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:47.212028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:47.212030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:47.212032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.212034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.212036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.212040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:47.212045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:47.212049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:47.212052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:47.212056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:47.212059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:47.212062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:47.212064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:47.212067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:47.212070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:47.212072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:47.212075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:47.212078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.212081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.212083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.212086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.212089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.212092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.212107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:47.218312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342016454960082:9295] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:47.218332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342016454960082:9295] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:47.218336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342016454960082:9295] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:47.218346Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1027 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:47.219836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:47.220904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:47.220921Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:47.220924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:47.220929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:47.220936Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1026 [2:7559342016093139426:8339]
2025-10-09T21:47:47.220952Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:47.220972Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:47.220982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:47.220986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:47.220988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:47.221008Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:47.221014Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:47.221022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:47.221092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.221102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.221104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.221107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:47.221183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:47.221191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.221203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.221214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.221229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:47.221258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342016454960127:9295], now have 1 active actors on pipe
2025-10-09T21:47:47.221270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342016454960024:2432], now have 1 active actors on pipe
2025-10-09T21:47:47.221278Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342016093139421:8339]
2025-10-09T21:47:47.221280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342016093139421:8339]
2025-10-09T21:47:47.221284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342016093139421:8339]
2025-10-09T21:47:47.221298Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:47.221313Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:47.221314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:47.221316Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:47.221319Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:47.221327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:47.221330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:47.221373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342016093139421:8339]
2025-10-09T21:47:47.221388Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:47.221368Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1026
2025-10-09T21:47:47.221371Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1026
2025-10-09T21:47:47.221410Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:47.221493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:47.221504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:47.221535Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9494cbb2-10fef9cb-e2e81037-d0dfec40_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:47.221546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:47.221550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:47.221553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:47.221564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:47.221576Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:47.221592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:47.221742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:47.221750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:47.221764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:47.221888Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1027 partition: 0 MaxSeqNo: 25 sessionId: src_id|9494cbb2-10fef9cb-e2e81037-d0dfec40_0
2025-10-09T21:47:47.222356Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb6a7d7e-17bb9b9c-403a1c72-d50fc4d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9494cbb2-10fef9cb-e2e81037-d0dfec40_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:47.224910Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9494cbb2-10fef9cb-e2e81037-d0dfec40_0] Write session: try to update token
2025-10-09T21:47:47.224927Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9494cbb2-10fef9cb-e2e81037-d0dfec40_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:47.238606Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:47.238622Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:47.238624Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:47.238626Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:47.238636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342016093139421:8339]
2025-10-09T21:47:47.238641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342016093139421:8339]
2025-10-09T21:47:47.238644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342016093139421:8339]
2025-10-09T21:47:47.238684Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:47.238689Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:47.238690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:47.238694Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:47.238698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.238706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:47.238739Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1027 sessionId: src_id|9494cbb2-10fef9cb-e2e81037-d0dfec40_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:47.238830Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:47.241091Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:47.299348Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342016454960134:9144]
2025-10-09T21:47:47.299650Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 1026, pipe: [1:7559342016454960134:9144]
2025-10-09T21:47:47.391402Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:47.391863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:47.391882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:47.391932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342016454960134:9144], now have 1 active actors on pipe
2025-10-09T21:47:47.392191Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:47.392211Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:47.392229Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:47.392236Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342016454960134:9144]
2025-10-09T21:47:47.392258Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:47.392303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1027 reason: error at write: Disconnected sessionId: src_id|9494cbb2-10fef9cb-e2e81037-d0dfec40_0
2025-10-09T21:47:47.392491Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1027 sessionId: src_id|9494cbb2-10fef9cb-e2e81037-d0dfec40_0 is DEAD
2025-10-09T21:47:47.392684Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9494cbb2-10fef9cb-e2e81037-d0dfec40_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:47.392695Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9494cbb2-10fef9cb-e2e81037-d0dfec40_0] Write session will restart in 0.008424s
2025-10-09T21:47:47.392719Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9494cbb2-10fef9cb-e2e81037-d0dfec40_0] Write session: Do CDS request
2025-10-09T21:47:47.392725Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9494cbb2-10fef9cb-e2e81037-d0dfec40_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:47.400787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:47.402254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:47.402432Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:47.402442Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:47.402454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:47.402508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:47.402545Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342016093139528:8341]
2025-10-09T21:47:47.402580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9494cbb2-10fef9cb-e2e81037-d0dfec40_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:47.402638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:47.402655Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1028
2025-10-09T21:47:47.402835Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1028 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:47.402868Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1028 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:47.402875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1028 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:47.402878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:47.403121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:47.403351Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1028 sessionId:  describe result for acl check
2025-10-09T21:47:47.403374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:47.403389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:47.403397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:47.403399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:47.403407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342016454960204:9299] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:47.403417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:47.403729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:47.403777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:47.403782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:47.403783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:47.403784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:47.403786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:47.403787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:47.403788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.403789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.403790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.403794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:47.403803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:47.403807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:47.403811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:47.403815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:47.403819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:47.403823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.403839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.403842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.403848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:47.403857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:47.403862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:47.403867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:47.403871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:47.403875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:47.403879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:47.403883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:47.403887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:47.403892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:47.403896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:47.403900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:47.403905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.403909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.403914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.403919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.403923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.403933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.403961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:47.403970Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342016454960204:9299] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:47.406182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342016454960204:9299] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:47.411187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342016454960204:9299] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:47.411206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342016454960204:9299] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:47.411211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342016454960204:9299] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:47.411223Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1028 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:47.411611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:47.412839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:47.412852Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:47.412854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:47.412857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:47.412863Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1027 [2:7559342016093139528:8341]
2025-10-09T21:47:47.412876Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:47.412889Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:47.412892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:47.412894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:47.412896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:47.412909Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:47.412915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:47.412924Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:47.412967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.412972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.412974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.412976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:47.413028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:47.413033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.413041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.413047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.413056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:47.413073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342016093139523:8341]
2025-10-09T21:47:47.413077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342016093139523:8341]
2025-10-09T21:47:47.413079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342016093139523:8341]
2025-10-09T21:47:47.413085Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:47.413097Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:47.413097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:47.413100Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:47.413104Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:47.413106Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:47.413108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:47.413116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342016454960241:9299], now have 1 active actors on pipe
2025-10-09T21:47:47.413119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342016454960137:2432], now have 1 active actors on pipe
2025-10-09T21:47:47.413148Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342016093139523:8341]
2025-10-09T21:47:47.413162Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:47.413175Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:47.413165Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1027
2025-10-09T21:47:47.413186Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1027
2025-10-09T21:47:47.413223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:47.413232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:47.413251Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d4c7728f-755063b1-6689e88b-b2b88bc8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:47.413260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:47.413264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:47.413266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:47.413273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:47.413281Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:47.413292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:47.413547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:47.413554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:47.413569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:47.413680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1028 partition: 0 MaxSeqNo: 25 sessionId: src_id|d4c7728f-755063b1-6689e88b-b2b88bc8_0
2025-10-09T21:47:47.414064Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9494cbb2-10fef9cb-e2e81037-d0dfec40_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d4c7728f-755063b1-6689e88b-b2b88bc8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:47.416648Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d4c7728f-755063b1-6689e88b-b2b88bc8_0] Write session: try to update token
2025-10-09T21:47:47.416668Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d4c7728f-755063b1-6689e88b-b2b88bc8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:47.425861Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:47.425880Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:47.425891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342016093139523:8341]
2025-10-09T21:47:47.425939Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:47.425961Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:47.426141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:47.426305Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:47.426459Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:47.426606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:47.426811Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:47.426860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:47.430680Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1028 sessionId: src_id|d4c7728f-755063b1-6689e88b-b2b88bc8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:47.433160Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:47.433380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:47.433399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:47.433446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:47.433530Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:47.439709Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:47.439725Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:47.439729Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:47.439731Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:47.439741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342016093139523:8341]
2025-10-09T21:47:47.439748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342016093139523:8341]
2025-10-09T21:47:47.439755Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342016093139523:8341]
2025-10-09T21:47:47.439805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:47.439812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.439815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:47.439816Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:47.439823Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:47.439827Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:47.439954Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:47.492350Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342016454960248:9144]
2025-10-09T21:47:47.492581Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 1027, pipe: [1:7559342016454960248:9144]
2025-10-09T21:47:47.595436Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:47.603369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:47.603398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:47.603567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.603634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.603701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.603766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.603808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.603814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.603886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.603945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.604009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.604074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.604113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.604116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.604181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.604238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.604295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.604352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.604397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.604400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.604470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.604532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.604622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.604684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.604725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.604728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.604804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.604858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.604915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.604977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.605014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.605017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.605081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.605140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.605197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.605261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.605309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.605312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.605373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.605425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.605481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.605532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.605574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.605577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.605637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.605687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.605740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.605791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.605841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.605844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.605902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.605953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.606007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.606252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.606304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.606308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.606368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.606420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.606480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.606541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.606598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.606606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.606673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342016454960248:9144], now have 1 active actors on pipe
2025-10-09T21:47:47.607399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4c7728f-755063b1-6689e88b-b2b88bc8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:47.607412Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4c7728f-755063b1-6689e88b-b2b88bc8_0] Write session will restart in 0.005940s
2025-10-09T21:47:47.607453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4c7728f-755063b1-6689e88b-b2b88bc8_0] Write session: Do CDS request
2025-10-09T21:47:47.607460Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4c7728f-755063b1-6689e88b-b2b88bc8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:47.606999Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:47.607005Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342016454960248:9144]
2025-10-09T21:47:47.607017Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:47.607036Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:47.607058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1028 reason: error at write: Disconnected sessionId: src_id|d4c7728f-755063b1-6689e88b-b2b88bc8_0
2025-10-09T21:47:47.607258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1028 sessionId: src_id|d4c7728f-755063b1-6689e88b-b2b88bc8_0 is DEAD
2025-10-09T21:47:47.607784Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:47.614591Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d4c7728f-755063b1-6689e88b-b2b88bc8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:47.614675Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:47.614692Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1029
2025-10-09T21:47:47.614979Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1029 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:47.615029Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1029 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:47.615034Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1029 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:47.615037Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:47.615461Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1029 sessionId:  describe result for acl check
2025-10-09T21:47:47.615498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:47.615500Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:47.615504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:47.615509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342016454960274:9303] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:47.616094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342016454960274:9303] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:47.618174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:47.618402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342016454960274:9303] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:47.620010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:47.620211Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:47.620222Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:47.620234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:47.620292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:47.620331Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342016093139665:8353]
2025-10-09T21:47:47.620820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:47.621619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342016454960274:9303] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:47.621635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342016454960274:9303] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:47.621639Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342016454960274:9303] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:47.621649Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1029 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:47.621860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:47.621919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:47.622257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:47.622327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:47.622334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:47.622335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:47.622337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:47.622338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:47.622339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:47.622340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.622341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.622342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.622350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:47.622361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:47.622366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:47.622370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:47.622374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:47.622378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:47.622382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.622387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.622391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.622399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:47.622408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:47.622414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:47.622419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:47.622424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:47.622429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:47.622433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:47.622438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:47.622443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:47.622448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:47.622451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:47.622457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:47.622462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.622467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.622477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.622482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.622487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.622492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.622531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:47.630353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:47.631428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:47.631444Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:47.631447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:47.631451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:47.631459Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1028 [2:7559342016093139665:8353]
2025-10-09T21:47:47.631477Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:47.631497Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:47.631505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:47.631509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:47.631511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:47.631530Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:47.631541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:47.631550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:47.631622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.631630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.631632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.631635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:47.631710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:47.631718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.631730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.631748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.631771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:47.631800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342016454960354:9303], now have 1 active actors on pipe
2025-10-09T21:47:47.631804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342016454960251:2432], now have 1 active actors on pipe
2025-10-09T21:47:47.631812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342016093139659:8353]
2025-10-09T21:47:47.631815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342016093139659:8353]
2025-10-09T21:47:47.631819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342016093139659:8353]
2025-10-09T21:47:47.631834Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:47.631851Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:47.631854Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:47.631855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:47.631856Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:47.631871Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:47.631879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:47.631854Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1028
2025-10-09T21:47:47.631921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342016093139659:8353]
2025-10-09T21:47:47.631934Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:47.631942Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:47.632017Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1028
2025-10-09T21:47:47.632071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:47.632082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:47.632110Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f767248c-e5f08458-4c1805b9-dcd15734_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:47.632124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:47.632128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:47.632131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:47.632142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:47.632153Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:47.632169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:47.632362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:47.632373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:47.632389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:47.632491Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1029 partition: 0 MaxSeqNo: 25 sessionId: src_id|f767248c-e5f08458-4c1805b9-dcd15734_0
2025-10-09T21:47:47.633051Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4c7728f-755063b1-6689e88b-b2b88bc8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f767248c-e5f08458-4c1805b9-dcd15734_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:47.635585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f767248c-e5f08458-4c1805b9-dcd15734_0] Write session: try to update token
2025-10-09T21:47:47.635605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f767248c-e5f08458-4c1805b9-dcd15734_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:47.644333Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:47.644350Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:47.644360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342016093139659:8353]
2025-10-09T21:47:47.644412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:47.644413Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:47.644584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:47.644724Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:47.644858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:47.644988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:47.645179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:47.645221Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:47.652397Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1029 sessionId: src_id|f767248c-e5f08458-4c1805b9-dcd15734_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:47.654897Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:47.655208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:47.655228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:47.655271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:47.655347Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:47.658913Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:47.658940Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:47.658943Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:47.658945Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:47.658963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342016093139659:8353]
2025-10-09T21:47:47.658968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342016093139659:8353]
2025-10-09T21:47:47.658972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342016093139659:8353]
2025-10-09T21:47:47.659016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:47.659026Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:47.659029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.659032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:47.659032Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:47.659035Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:47.659316Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:47.707386Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342016454960386:9144]
2025-10-09T21:47:47.707661Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 1028, pipe: [1:7559342016454960386:9144]
2025-10-09T21:47:47.819839Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:47.827949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:47.827979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:47.828141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.828212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.828276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.828334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.828378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.828387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.828449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.828508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.828570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.828639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.828681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.828688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.828754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.828813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.828874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.828940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.828985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.828993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.829064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.829132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.829196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.829258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.829306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.829317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.829382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.829440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.829507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.829571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.829616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.829624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.829683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.829742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.829798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.829870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.829913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.829920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.829980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.830053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.830115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.830173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.830215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.830221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.830281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.830337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.830390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.830448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.830491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.830498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.830555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.830609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.830666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.830725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.830768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.830772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.830832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:47.830893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:47.830952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:47.831015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:47.831056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:47.831063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:47.831120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342016454960386:9144], now have 1 active actors on pipe
2025-10-09T21:47:47.831320Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:47.831332Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342016454960386:9144]
2025-10-09T21:47:47.831336Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:47.831354Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:47.831368Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1029 reason: error at write: Disconnected sessionId: src_id|f767248c-e5f08458-4c1805b9-dcd15734_0
2025-10-09T21:47:47.831550Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1029 sessionId: src_id|f767248c-e5f08458-4c1805b9-dcd15734_0 is DEAD
2025-10-09T21:47:47.831707Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f767248c-e5f08458-4c1805b9-dcd15734_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:47.831715Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f767248c-e5f08458-4c1805b9-dcd15734_0] Write session will restart in 0.009950s
2025-10-09T21:47:47.831732Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f767248c-e5f08458-4c1805b9-dcd15734_0] Write session: Do CDS request
2025-10-09T21:47:47.831737Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f767248c-e5f08458-4c1805b9-dcd15734_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:47.831975Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:47.839976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:47.841559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:47.841736Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:47.841743Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:47.841752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:47.841794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:47.841840Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342016093139778:8359]
2025-10-09T21:47:47.842297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:47.842554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f767248c-e5f08458-4c1805b9-dcd15734_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:47.842569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:47.842602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:47.842633Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:47.842650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1030
2025-10-09T21:47:47.842781Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1030 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:47.842809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1030 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:47.842819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1030 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:47.842822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:47.842844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:47.842878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:47.842884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:47.842884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:47.842885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:47.842886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:47.842886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:47.842887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.842887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.842888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.842891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:47.842898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:47.842900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:47.842903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:47.842905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:47.842907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:47.842908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.842911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.842912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.842918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:47.842923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:47.842927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:47.842931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:47.842933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:47.842936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:47.842938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:47.842941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:47.842944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:47.842947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:47.842949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:47.842953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:47.842955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.842958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.842961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.842964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.842966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.842969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.842985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:47.843193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1030 sessionId:  describe result for acl check
2025-10-09T21:47:47.843227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:47.843233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:47.843235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:47.843240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342016454960463:9315] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:47.843804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342016454960463:9315] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:47.846074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342016454960463:9315] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:47.850020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:47.850105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342016454960463:9315] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:47.850119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342016454960463:9315] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:47.850122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342016454960463:9315] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:47.850129Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1030 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:47.851055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:47.851068Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:47.851070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:47.851073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:47.851079Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1029 [2:7559342016093139778:8359]
2025-10-09T21:47:47.851089Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:47.851103Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:47.851105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:47.851107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:47.851108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:47.851121Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:47.851126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:47.851131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:47.851174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.851180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.851181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.851183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:47.851236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:47.851242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.851251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:47.851257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:47.851270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:47.851287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342016454960496:9315], now have 1 active actors on pipe
2025-10-09T21:47:47.851292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342016454960390:2432], now have 1 active actors on pipe
2025-10-09T21:47:47.851298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342016093139773:8359]
2025-10-09T21:47:47.851299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342016093139773:8359]
2025-10-09T21:47:47.851301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342016093139773:8359]
2025-10-09T21:47:47.851310Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:47.851320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:47.851332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:47.851334Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:47.851337Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:47.851338Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:47.851340Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:47.851365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342016093139773:8359]
2025-10-09T21:47:47.851373Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:47.851380Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:47.851376Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1029
2025-10-09T21:47:47.851377Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1029
2025-10-09T21:47:47.851416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:47.851423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:47.851446Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|df5eca88-4f3a076d-379a8edd-fb5bbc87_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:47.851455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:47.851458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:47.851459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:47.851466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:47.851474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:47.851484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:47.851668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:47.851683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:47.851709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:47.851796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1030 partition: 0 MaxSeqNo: 25 sessionId: src_id|df5eca88-4f3a076d-379a8edd-fb5bbc87_0
2025-10-09T21:47:47.852145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f767248c-e5f08458-4c1805b9-dcd15734_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|df5eca88-4f3a076d-379a8edd-fb5bbc87_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:47.854816Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df5eca88-4f3a076d-379a8edd-fb5bbc87_0] Write session: try to update token
2025-10-09T21:47:47.854830Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df5eca88-4f3a076d-379a8edd-fb5bbc87_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:47.863414Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:47.863423Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:47.863432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342016093139773:8359]
2025-10-09T21:47:47.863465Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:47.863480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:47.863632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:47.863746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:47.863857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:47.863975Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:47.864141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:47.864172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:47.868503Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1030 sessionId: src_id|df5eca88-4f3a076d-379a8edd-fb5bbc87_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:47.870893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:47.871076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:47.871100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:47.871134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:47.871221Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:47.876400Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:47.876418Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:47.876421Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:47.876423Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:47.876433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342016093139773:8359]
2025-10-09T21:47:47.876438Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342016093139773:8359]
2025-10-09T21:47:47.876441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342016093139773:8359]
2025-10-09T21:47:47.876486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:47.876493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:47.876496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:47.876498Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:47.876506Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:47.876510Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:47.876630Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:47.931641Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342016454960504:9144]
2025-10-09T21:47:47.931880Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 1029, pipe: [1:7559342016454960504:9144]
2025-10-09T21:47:48.032429Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:48.040432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:48.040466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:48.040640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.040708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.040777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.040851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.040897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.040907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.040975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.041039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.041113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.041178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.041228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.041240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.041307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.041367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.041430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.041483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.041535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.041542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.041609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.041670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.041741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.041814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.041857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.041860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.041929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.041989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.042068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.042127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.042165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.042179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.042243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.042306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.042370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.042479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.042544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.042552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.042614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.042671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.042731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.042792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.042905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.042913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.042977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.043044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.043107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.043165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.043208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.043217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.043285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.043349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.043408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.043469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.043514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.043524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.043596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.043659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.043729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.043794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.043840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.043843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.043921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342016454960504:9144], now have 1 active actors on pipe
2025-10-09T21:47:48.044273Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:48.044292Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:48.044338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1030 reason: error at write: Disconnected sessionId: src_id|df5eca88-4f3a076d-379a8edd-fb5bbc87_0
2025-10-09T21:47:48.044558Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1030 sessionId: src_id|df5eca88-4f3a076d-379a8edd-fb5bbc87_0 is DEAD
2025-10-09T21:47:48.044586Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342016454960504:9144]
2025-10-09T21:47:48.044633Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:48.044700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df5eca88-4f3a076d-379a8edd-fb5bbc87_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:48.044712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df5eca88-4f3a076d-379a8edd-fb5bbc87_0] Write session will restart in 0.007125s
2025-10-09T21:47:48.044736Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df5eca88-4f3a076d-379a8edd-fb5bbc87_0] Write session: Do CDS request
2025-10-09T21:47:48.044742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df5eca88-4f3a076d-379a8edd-fb5bbc87_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:48.045098Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:48.053594Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df5eca88-4f3a076d-379a8edd-fb5bbc87_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:48.053648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:48.053669Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1031
2025-10-09T21:47:48.053885Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1031 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:48.053922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1031 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:48.053933Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1031 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:48.053937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:48.054730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1031 sessionId:  describe result for acl check
2025-10-09T21:47:48.054773Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:48.054775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:48.054777Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:48.054783Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342020749927868:9319] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:48.055345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342020749927868:9319] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:48.056209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:48.058595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342020749927868:9319] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:48.058678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:48.058927Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:48.058949Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:48.058965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:48.059038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:48.059101Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342020388107180:8361]
2025-10-09T21:47:48.059717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:48.060021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:48.060067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:48.060387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:48.060452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.060461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.060463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.060464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.060466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.060467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.060468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.060469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.060471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.060477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.060487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.060492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.060496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.060500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.060505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.060509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.060513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.060518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.060525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.060533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.060540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.060546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.060551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.060557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.060562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.060567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.060572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.060578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.060583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.060588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.060594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.060600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.060607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.060612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.060617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.060622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.060649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:48.062573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342020749927868:9319] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:48.062590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342020749927868:9319] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:48.062594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342020749927868:9319] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:48.062604Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1031 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:48.069140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:48.070344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:48.070365Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:48.070368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:48.070373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:48.070381Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1030 [2:7559342020388107180:8361]
2025-10-09T21:47:48.070398Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:48.070420Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:48.070424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:48.070428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:48.070430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:48.070451Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:48.070457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:48.070465Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:48.070540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.070550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.070555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.070558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:48.070638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:48.070649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.070664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.070676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.070692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:48.070723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342020749927803:2432], now have 1 active actors on pipe
2025-10-09T21:47:48.070733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342020749927910:9319], now have 1 active actors on pipe
2025-10-09T21:47:48.070741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342020388107174:8361]
2025-10-09T21:47:48.070743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342020388107174:8361]
2025-10-09T21:47:48.070745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342020388107174:8361]
2025-10-09T21:47:48.070761Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:48.070753Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1030
2025-10-09T21:47:48.070794Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1030
2025-10-09T21:47:48.070778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:48.070784Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:48.070788Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:48.070791Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:48.070803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:48.070807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:48.070872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:48.070877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:48.070891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342020388107174:8361]
2025-10-09T21:47:48.070907Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:48.070948Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ae04f7ac-f00fe3dd-c4cb68-c53c4553_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:48.070957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:48.070959Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:48.070961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:48.070964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:48.070975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:48.070987Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:48.071013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:48.071196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:48.071209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:48.071226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:48.071317Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1031 partition: 0 MaxSeqNo: 25 sessionId: src_id|ae04f7ac-f00fe3dd-c4cb68-c53c4553_0
2025-10-09T21:47:48.071668Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df5eca88-4f3a076d-379a8edd-fb5bbc87_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ae04f7ac-f00fe3dd-c4cb68-c53c4553_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:48.074285Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae04f7ac-f00fe3dd-c4cb68-c53c4553_0] Write session: try to update token
2025-10-09T21:47:48.074304Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae04f7ac-f00fe3dd-c4cb68-c53c4553_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:48.086836Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:48.086856Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:48.086859Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:48.086860Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:48.086876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342020388107174:8361]
2025-10-09T21:47:48.086880Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342020388107174:8361]
2025-10-09T21:47:48.086884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342020388107174:8361]
2025-10-09T21:47:48.086933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:48.086945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.086948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:48.086950Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:48.086956Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:48.086961Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:48.087085Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:48.087643Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1031 sessionId: src_id|ae04f7ac-f00fe3dd-c4cb68-c53c4553_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:48.090119Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:48.144635Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342020749927920:9144]
2025-10-09T21:47:48.144908Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 1030, pipe: [1:7559342020749927920:9144]
2025-10-09T21:47:48.249827Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:48.250239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:48.250256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:48.250303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342020749927920:9144], now have 1 active actors on pipe
2025-10-09T21:47:48.250519Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:48.250525Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:48.250529Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:48.250594Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342020749927920:9144]
2025-10-09T21:47:48.250625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1031 reason: error at write: Disconnected sessionId: src_id|ae04f7ac-f00fe3dd-c4cb68-c53c4553_0
2025-10-09T21:47:48.250701Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:48.250810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1031 sessionId: src_id|ae04f7ac-f00fe3dd-c4cb68-c53c4553_0 is DEAD
2025-10-09T21:47:48.250995Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae04f7ac-f00fe3dd-c4cb68-c53c4553_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:48.251005Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae04f7ac-f00fe3dd-c4cb68-c53c4553_0] Write session will restart in 0.008182s
2025-10-09T21:47:48.251032Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae04f7ac-f00fe3dd-c4cb68-c53c4553_0] Write session: Do CDS request
2025-10-09T21:47:48.251038Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae04f7ac-f00fe3dd-c4cb68-c53c4553_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:48.260657Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae04f7ac-f00fe3dd-c4cb68-c53c4553_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:48.260706Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:48.260727Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1032
2025-10-09T21:47:48.260909Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1032 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:48.260986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1032 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:48.260993Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1032 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:48.260998Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:48.261532Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1032 sessionId:  describe result for acl check
2025-10-09T21:47:48.261586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:48.261588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:48.261590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:48.261597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342020749927994:9324] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:48.262310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342020749927994:9324] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:48.263269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:48.264593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342020749927994:9324] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:48.265194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:48.265384Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:48.265403Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:48.265420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:48.265476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:48.265527Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342020388107287:8363]
2025-10-09T21:47:48.266145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:48.266475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:48.266519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:48.266839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:48.266898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.266905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.266906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.266907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.266909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.266910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.266911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.266912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.266914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.266919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.266929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.266933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.266937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.266940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.266944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.266948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.266951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.266955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.266961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.266969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.266975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.266981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.266986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.266991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.266996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.267000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.267005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.267010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.267014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.267020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.267025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.267030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.267034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.267039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.267044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.267049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.267077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:48.267206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342020749927994:9324] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:48.267230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342020749927994:9324] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:48.267235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342020749927994:9324] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:48.267244Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1032 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:48.274012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:48.275062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:48.275076Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:48.275078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:48.275082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:48.275090Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1031 [2:7559342020388107287:8363]
2025-10-09T21:47:48.275108Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:48.275128Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:48.275130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:48.275134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:48.275136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:48.275153Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:48.275161Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:48.275176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:48.275233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.275242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.275244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.275247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:48.275317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:48.275320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.275331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.275341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.275357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:48.275376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342020388107282:8363]
2025-10-09T21:47:48.275378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342020388107282:8363]
2025-10-09T21:47:48.275380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342020388107282:8363]
2025-10-09T21:47:48.275395Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:48.275410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:48.275422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:48.275425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:48.275441Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:48.275448Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:48.275450Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:48.275461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342020388107282:8363]
2025-10-09T21:47:48.275464Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:48.275549Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:48.275604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342020749928034:9324], now have 1 active actors on pipe
2025-10-09T21:47:48.275612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342020749927928:2432], now have 1 active actors on pipe
2025-10-09T21:47:48.275681Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1031
2025-10-09T21:47:48.275691Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1031
2025-10-09T21:47:48.275791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:48.275857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:48.275888Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|520fcb8b-fb0f642c-227da48e-94ec3762_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:48.275897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:48.275900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:48.275911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:48.275923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:48.275936Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:48.275955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:48.276205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:48.276218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:48.276241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:48.276434Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1032 partition: 0 MaxSeqNo: 25 sessionId: src_id|520fcb8b-fb0f642c-227da48e-94ec3762_0
2025-10-09T21:47:48.276854Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae04f7ac-f00fe3dd-c4cb68-c53c4553_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|520fcb8b-fb0f642c-227da48e-94ec3762_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:48.279577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|520fcb8b-fb0f642c-227da48e-94ec3762_0] Write session: try to update token
2025-10-09T21:47:48.279597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|520fcb8b-fb0f642c-227da48e-94ec3762_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:48.291779Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:48.291799Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:48.291801Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:48.291803Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:48.291814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342020388107282:8363]
2025-10-09T21:47:48.291823Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342020388107282:8363]
2025-10-09T21:47:48.291827Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342020388107282:8363]
2025-10-09T21:47:48.291876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:48.291879Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:48.291885Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:48.291889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.291892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:48.291897Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:48.292025Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:48.294707Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1032 sessionId: src_id|520fcb8b-fb0f642c-227da48e-94ec3762_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:48.297144Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:48.350492Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342020749928044:9144]
2025-10-09T21:47:48.350916Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 1031, pipe: [1:7559342020749928044:9144]
2025-10-09T21:47:48.450346Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:48.450824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:48.450844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:48.450896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342020749928044:9144], now have 1 active actors on pipe
2025-10-09T21:47:48.451189Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:48.451206Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:48.451219Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342020749928044:9144]
2025-10-09T21:47:48.451243Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:48.451279Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1032 reason: error at write: Disconnected sessionId: src_id|520fcb8b-fb0f642c-227da48e-94ec3762_0
2025-10-09T21:47:48.451481Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1032 sessionId: src_id|520fcb8b-fb0f642c-227da48e-94ec3762_0 is DEAD
2025-10-09T21:47:48.451650Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|520fcb8b-fb0f642c-227da48e-94ec3762_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:48.451662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|520fcb8b-fb0f642c-227da48e-94ec3762_0] Write session will restart in 0.007977s
2025-10-09T21:47:48.451689Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|520fcb8b-fb0f642c-227da48e-94ec3762_0] Write session: Do CDS request
2025-10-09T21:47:48.451699Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|520fcb8b-fb0f642c-227da48e-94ec3762_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:48.451700Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:48.460559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|520fcb8b-fb0f642c-227da48e-94ec3762_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:48.460743Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:48.460766Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1033
2025-10-09T21:47:48.460921Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1033 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:48.460972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1033 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:48.460987Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1033 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:48.460991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:48.461470Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1033 sessionId:  describe result for acl check
2025-10-09T21:47:48.461527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:48.461560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:48.461562Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:48.461569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342020749928113:9328] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:48.462251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342020749928113:9328] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:48.463052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:48.464581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342020749928113:9328] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:48.465122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:48.465331Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:48.465340Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:48.465353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:48.465401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:48.465441Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342020388107394:8365]
2025-10-09T21:47:48.466085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:48.466435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:48.466496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:48.466842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:48.466904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.466911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.466913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.466914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.466915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.466916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.466917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.466918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.466919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.466925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.466935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.466939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.466943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.466946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.466950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.466953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.466957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.466960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.466969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.466987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.466994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.466999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.467009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.467015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.467019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.467029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.467033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.467038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.467055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.467065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.467070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.467078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.467086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.467091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.467095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.467100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.467128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:48.467441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342020749928113:9328] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:48.467456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342020749928113:9328] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:48.467461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342020749928113:9328] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:48.467469Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1033 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:48.473728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:48.474840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:48.474857Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:48.474859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:48.474864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:48.474872Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1032 [2:7559342020388107394:8365]
2025-10-09T21:47:48.474886Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:48.474904Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:48.474908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:48.474911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:48.474914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:48.474932Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:48.474939Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:48.474946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:48.475011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.475022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.475024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.475027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:48.475102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:48.475111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.475122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.475133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.475150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:48.475173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342020749928155:9328], now have 1 active actors on pipe
2025-10-09T21:47:48.475182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342020749928046:2432], now have 1 active actors on pipe
2025-10-09T21:47:48.475190Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342020388107388:8365]
2025-10-09T21:47:48.475192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342020388107388:8365]
2025-10-09T21:47:48.475194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342020388107388:8365]
2025-10-09T21:47:48.475206Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:48.475219Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:48.475221Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:48.475228Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:48.475230Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:48.475237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:48.475241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:48.475277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342020388107388:8365]
2025-10-09T21:47:48.475289Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:48.475296Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:48.475346Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1032
2025-10-09T21:47:48.475349Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1032
2025-10-09T21:47:48.475429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:48.475440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:48.475467Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|af6e7987-d4649c1e-5d90d9-85952d0b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:48.475481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:48.475485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:48.475488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:48.475499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:48.475516Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:48.475535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:48.475765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:48.475776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:48.475799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:48.475899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1033 partition: 0 MaxSeqNo: 25 sessionId: src_id|af6e7987-d4649c1e-5d90d9-85952d0b_0
2025-10-09T21:47:48.476306Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|520fcb8b-fb0f642c-227da48e-94ec3762_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|af6e7987-d4649c1e-5d90d9-85952d0b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:48.478863Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af6e7987-d4649c1e-5d90d9-85952d0b_0] Write session: try to update token
2025-10-09T21:47:48.478881Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af6e7987-d4649c1e-5d90d9-85952d0b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:48.488727Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:48.488745Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:48.488754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342020388107388:8365]
2025-10-09T21:47:48.488802Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:48.488806Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:48.488976Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:48.489127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:48.489287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:48.489431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:48.489640Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:48.489684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:48.492614Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1033 sessionId: src_id|af6e7987-d4649c1e-5d90d9-85952d0b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:48.495196Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:48.495472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:48.495493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:48.495535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:48.495644Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:48.501874Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:48.501888Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:48.501891Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:48.501893Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:48.501903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342020388107388:8365]
2025-10-09T21:47:48.501906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342020388107388:8365]
2025-10-09T21:47:48.501910Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342020388107388:8365]
2025-10-09T21:47:48.501952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:48.501964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.501966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:48.501976Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:48.501992Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:48.501997Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:48.502113Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:48.551180Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342020749928163:9144]
2025-10-09T21:47:48.551495Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 1032, pipe: [1:7559342020749928163:9144]
2025-10-09T21:47:48.658260Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:48.666354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:48.666376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:48.666545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.666606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.666666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.666730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.667021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.667033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.667097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.667155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.667212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.667273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.667322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.667330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.667392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.667453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.667509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.667573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.667616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.667625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.667688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.667748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.667819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.667884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.667928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.667935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.667996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.668057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.668115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.668180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.668232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.668240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.668307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.668370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.668432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.668496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.668557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.668568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.668635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.668692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.668756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.668820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.668867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.668876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.668940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.668998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.669060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.669122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.669167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.669176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.669241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.669301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.669361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.669425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.669468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.669475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.669535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:48.669595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:48.669658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:48.669718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:48.669762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:48.669771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:48.669845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342020749928163:9144], now have 1 active actors on pipe
2025-10-09T21:47:48.670159Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342020749928163:9144]
2025-10-09T21:47:48.670168Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:48.670172Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:48.670203Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1033 reason: error at write: Disconnected sessionId: src_id|af6e7987-d4649c1e-5d90d9-85952d0b_0
2025-10-09T21:47:48.670207Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:48.670387Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1033 sessionId: src_id|af6e7987-d4649c1e-5d90d9-85952d0b_0 is DEAD
2025-10-09T21:47:48.670568Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af6e7987-d4649c1e-5d90d9-85952d0b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:48.670584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af6e7987-d4649c1e-5d90d9-85952d0b_0] Write session will restart in 0.008928s
2025-10-09T21:47:48.670612Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af6e7987-d4649c1e-5d90d9-85952d0b_0] Write session: Do CDS request
2025-10-09T21:47:48.670622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af6e7987-d4649c1e-5d90d9-85952d0b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:48.670800Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:48.680602Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af6e7987-d4649c1e-5d90d9-85952d0b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:48.680681Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:48.680711Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1034
2025-10-09T21:47:48.680771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:48.680889Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1034 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:48.680947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1034 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:48.680965Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1034 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:48.680979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:48.681443Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1034 sessionId:  describe result for acl check
2025-10-09T21:47:48.681479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:48.681486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:48.681488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:48.681495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342020749928237:9332] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:48.682082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342020749928237:9332] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:48.682900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:48.683121Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:48.683131Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:48.683143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:48.683199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:48.683239Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342020388107536:8379]
2025-10-09T21:47:48.683840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:48.684158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:48.684209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:48.684524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:48.684587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.684592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.684593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.684594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.684595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.684596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.684597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.684598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.684599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.684604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.684613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.684617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.684620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.684623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.684627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.684630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.684633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.684636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.684643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.684650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.684656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.684660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.684663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.684667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.684689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.684698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.684704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.684709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.684717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.684721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.684726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.684730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.684734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.684737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.684742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.684746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.684770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:48.684839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342020749928237:9332] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:48.690920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342020749928237:9332] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:48.690939Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342020749928237:9332] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:48.690943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342020749928237:9332] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:48.690954Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1034 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:48.691317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:48.692494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:48.692508Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:48.692509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:48.692513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:48.692517Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1033 [2:7559342020388107536:8379]
2025-10-09T21:47:48.692527Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:48.692541Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:48.692543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:48.692546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:48.692547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:48.692561Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:48.692571Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:48.692577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:48.692624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.692630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.692631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.692632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:48.692695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:48.692701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.692709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.692715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.692728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:48.692749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342020749928165:2432], now have 1 active actors on pipe
2025-10-09T21:47:48.692756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342020749928275:9332], now have 1 active actors on pipe
2025-10-09T21:47:48.692762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342020388107530:8379]
2025-10-09T21:47:48.692763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342020388107530:8379]
2025-10-09T21:47:48.692765Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342020388107530:8379]
2025-10-09T21:47:48.692773Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:48.692784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:48.692793Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:48.692795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:48.692807Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:48.692810Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:48.692812Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:48.692826Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342020388107530:8379]
2025-10-09T21:47:48.692829Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:48.692830Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:48.692828Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1033
2025-10-09T21:47:48.692837Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1033
2025-10-09T21:47:48.692943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:48.692955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:48.692984Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c95a24c9-389cfdaf-3d80e85c-5ff3160d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:48.692999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:48.693003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:48.693011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:48.693022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:48.693040Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:48.693055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:48.693199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:48.693208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:48.693221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:48.693310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1034 partition: 0 MaxSeqNo: 25 sessionId: src_id|c95a24c9-389cfdaf-3d80e85c-5ff3160d_0
2025-10-09T21:47:48.693675Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af6e7987-d4649c1e-5d90d9-85952d0b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c95a24c9-389cfdaf-3d80e85c-5ff3160d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:48.696136Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c95a24c9-389cfdaf-3d80e85c-5ff3160d_0] Write session: try to update token
2025-10-09T21:47:48.696157Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c95a24c9-389cfdaf-3d80e85c-5ff3160d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:48.707370Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:48.707387Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:48.707390Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:48.707391Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:48.707400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342020388107530:8379]
2025-10-09T21:47:48.707404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342020388107530:8379]
2025-10-09T21:47:48.707408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342020388107530:8379]
2025-10-09T21:47:48.707451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:48.707451Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:48.707458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.707461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:48.707465Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:48.707469Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:48.707588Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:48.714429Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1034 sessionId: src_id|c95a24c9-389cfdaf-3d80e85c-5ff3160d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:48.717008Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:48.770310Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342020749928305:9144]
2025-10-09T21:47:48.770625Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 1033, pipe: [1:7559342020749928305:9144]
2025-10-09T21:47:48.870386Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:48.870742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:48.870757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:48.870780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342020749928305:9144], now have 1 active actors on pipe
2025-10-09T21:47:48.871003Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:48.871026Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:48.871037Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342020749928305:9144]
2025-10-09T21:47:48.871064Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:48.871154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1034 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c95a24c9-389cfdaf-3d80e85c-5ff3160d_0
2025-10-09T21:47:48.871401Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1034 sessionId: src_id|c95a24c9-389cfdaf-3d80e85c-5ff3160d_0 is DEAD
2025-10-09T21:47:48.871540Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c95a24c9-389cfdaf-3d80e85c-5ff3160d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:48.871552Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c95a24c9-389cfdaf-3d80e85c-5ff3160d_0] Write session will restart in 0.005776s
2025-10-09T21:47:48.871580Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c95a24c9-389cfdaf-3d80e85c-5ff3160d_0] Write session: Do CDS request
2025-10-09T21:47:48.871587Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c95a24c9-389cfdaf-3d80e85c-5ff3160d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:48.878623Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c95a24c9-389cfdaf-3d80e85c-5ff3160d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:48.878700Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:48.878717Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1035
2025-10-09T21:47:48.878895Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1035 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:48.878942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1035 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:48.878947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1035 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:48.878951Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:48.879615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1035 sessionId:  describe result for acl check
2025-10-09T21:47:48.879666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:48.879673Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:48.879675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:48.879680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342020749928350:9344] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:48.880250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342020749928350:9344] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:48.882012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:48.882871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342020749928350:9344] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:48.884105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:48.884308Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:48.884318Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:48.884329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:48.884382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:48.884422Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342020388107653:8383]
2025-10-09T21:47:48.885006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:48.885389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:48.885436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:48.885756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:48.885816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.885822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.885823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.885824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.885826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.885827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.885828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.885829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.885830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.885835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.885847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.885851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.885855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.885859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.885864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.885868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.885872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.885876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.885883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.885891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:48.885897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.885902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:48.885906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.885912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:48.885916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.885921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:48.885926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.885932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:48.885936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.885941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:48.885946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.885952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.885956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.885962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.885966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.885972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.886001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:48.886046Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342020749928350:9344] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:48.886062Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342020749928350:9344] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:48.886066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342020749928350:9344] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:48.886075Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1035 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:48.892612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:48.893916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:48.893932Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:48.893934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:48.893940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:48.893947Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1034 [2:7559342020388107653:8383]
2025-10-09T21:47:48.893961Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:48.893981Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:48.893984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:48.893987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:48.893990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:48.894011Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:48.894022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:48.894068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:48.894137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.894143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.894146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.894148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:48.894218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:48.894226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.894237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:48.894256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:48.894273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:48.894300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342020749928310:2432], now have 1 active actors on pipe
2025-10-09T21:47:48.894308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342020749928419:9344], now have 1 active actors on pipe
2025-10-09T21:47:48.894316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342020388107647:8383]
2025-10-09T21:47:48.894318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342020388107647:8383]
2025-10-09T21:47:48.894320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342020388107647:8383]
2025-10-09T21:47:48.894332Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:48.894353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:48.894360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:48.894364Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:48.894366Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:48.894371Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:48.894375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:48.894414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342020388107647:8383]
2025-10-09T21:47:48.894422Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:48.894396Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1034
2025-10-09T21:47:48.894432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:48.894401Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1034
2025-10-09T21:47:48.894458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:48.894468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:48.894504Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2024748-526d9631-8818fc6d-322335f2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:48.894515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:48.894520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:48.894523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:48.894536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:48.894551Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:48.894572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:48.894706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:48.894714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:48.894728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:48.894804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1035 partition: 0 MaxSeqNo: 25 sessionId: src_id|2024748-526d9631-8818fc6d-322335f2_0
2025-10-09T21:47:48.895201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c95a24c9-389cfdaf-3d80e85c-5ff3160d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2024748-526d9631-8818fc6d-322335f2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:48.897772Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2024748-526d9631-8818fc6d-322335f2_0] Write session: try to update token
2025-10-09T21:47:48.897790Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2024748-526d9631-8818fc6d-322335f2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:48.909060Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:48.909082Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:48.909085Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:48.909087Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:48.909097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342020388107647:8383]
2025-10-09T21:47:48.909103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342020388107647:8383]
2025-10-09T21:47:48.909107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342020388107647:8383]
2025-10-09T21:47:48.909152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:48.909155Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:48.909160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:48.909161Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:48.909163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:48.909165Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:48.909291Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:48.911670Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1035 sessionId: src_id|2024748-526d9631-8818fc6d-322335f2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:48.914154Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:48.970976Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342020749928428:9144]
2025-10-09T21:47:48.971317Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 1034, pipe: [1:7559342020749928428:9144]
2025-10-09T21:47:49.070884Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:49.071190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:49.071202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:49.071251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342020749928428:9144], now have 1 active actors on pipe
2025-10-09T21:47:49.071482Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:49.071700Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:49.071755Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:49.071832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1035 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2024748-526d9631-8818fc6d-322335f2_0
2025-10-09T21:47:49.071901Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342020749928428:9144]
2025-10-09T21:47:49.072054Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1035 sessionId: src_id|2024748-526d9631-8818fc6d-322335f2_0 is DEAD
2025-10-09T21:47:49.072253Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2024748-526d9631-8818fc6d-322335f2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:49.072265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2024748-526d9631-8818fc6d-322335f2_0] Write session will restart in 0.008519s
2025-10-09T21:47:49.072292Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2024748-526d9631-8818fc6d-322335f2_0] Write session: Do CDS request
2025-10-09T21:47:49.072298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2024748-526d9631-8818fc6d-322335f2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:49.081676Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2024748-526d9631-8818fc6d-322335f2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:49.081823Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:49.081842Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1036
2025-10-09T21:47:49.081977Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1036 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:49.082007Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1036 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:49.082013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1036 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:49.082017Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:49.082781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1036 sessionId:  describe result for acl check
2025-10-09T21:47:49.082824Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:49.082830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:49.082832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:49.082838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342025044895798:9348] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:49.083396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342025044895798:9348] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:49.084022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:49.086115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342025044895798:9348] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:49.086309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:49.086519Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:49.086529Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:49.086541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:49.086594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:49.086651Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342024683075059:8385]
2025-10-09T21:47:49.087257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:49.087574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:49.087620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:49.087942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:49.087999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.088008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.088009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.088010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.088011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.088012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.088013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.088014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.088016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.088021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.088032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.088036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.088041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.088044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.088048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.088051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.088056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.088060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.088067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.088075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.088082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.088087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.088091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.088097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.088101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.088106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.088111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.088117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.088121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.088126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.088131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.088153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.088158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.088163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.088168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.088173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.088200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:49.089054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342025044895798:9348] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:49.089067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342025044895798:9348] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:49.089072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342025044895798:9348] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:49.089081Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1036 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:49.094899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:49.096113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:49.096127Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:49.096130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:49.096135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:49.096142Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1035 [2:7559342024683075059:8385]
2025-10-09T21:47:49.096158Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:49.096178Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:49.096187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:49.096190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:49.096193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:49.096211Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:49.096223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:49.096231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:49.096300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.096307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.096309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.096312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:49.096381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:49.096392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.096405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.096418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.096436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:49.096459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342025044895727:2432], now have 1 active actors on pipe
2025-10-09T21:47:49.096469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342025044895838:9348], now have 1 active actors on pipe
2025-10-09T21:47:49.096477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342024683075054:8385]
2025-10-09T21:47:49.096484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342024683075054:8385]
2025-10-09T21:47:49.096486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342024683075054:8385]
2025-10-09T21:47:49.096501Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:49.096515Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:49.096518Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:49.096520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:49.096520Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:49.096537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:49.096540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:49.096580Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342024683075054:8385]
2025-10-09T21:47:49.096587Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:49.096539Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1035
2025-10-09T21:47:49.096600Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:49.096583Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1035
2025-10-09T21:47:49.096896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:49.096915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:49.096951Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3283a697-f8c1e833-f3c04957-33600a92_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:49.096976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:49.096986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:49.096989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:49.097000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:49.097016Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:49.097051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:49.097294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:49.097304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:49.097323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:49.097475Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1036 partition: 0 MaxSeqNo: 25 sessionId: src_id|3283a697-f8c1e833-f3c04957-33600a92_0
2025-10-09T21:47:49.097898Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2024748-526d9631-8818fc6d-322335f2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3283a697-f8c1e833-f3c04957-33600a92_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:49.100638Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3283a697-f8c1e833-f3c04957-33600a92_0] Write session: try to update token
2025-10-09T21:47:49.100660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3283a697-f8c1e833-f3c04957-33600a92_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:49.108696Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:49.108713Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:49.108723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342024683075054:8385]
2025-10-09T21:47:49.108767Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:49.108770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:49.108947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:49.109088Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:49.109217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:49.109355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:49.109827Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:49.109878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:49.114556Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1036 sessionId: src_id|3283a697-f8c1e833-f3c04957-33600a92_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:49.117038Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:49.117461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:49.117483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:49.117540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:49.117652Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:49.122227Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:49.122246Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:49.122249Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:49.122255Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:49.122265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342024683075054:8385]
2025-10-09T21:47:49.122269Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342024683075054:8385]
2025-10-09T21:47:49.122273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342024683075054:8385]
2025-10-09T21:47:49.122321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:49.122321Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:49.122327Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:49.122328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.122331Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:49.122332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:49.122468Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:49.171730Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342025044895849:9144]
2025-10-09T21:47:49.172041Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 1035, pipe: [1:7559342025044895849:9144]
2025-10-09T21:47:49.280498Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:49.287848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:49.287873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:49.288017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:49.288067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:49.288121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:49.288185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:49.288233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:49.288238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:49.288308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:49.288365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:49.288422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:49.288480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:49.288522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:49.288531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:49.288590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:49.288639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:49.288699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:49.288756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:49.288796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:49.288803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:49.288862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:49.288914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:49.288976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:49.289039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:49.289079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:49.289088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:49.289148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:49.289205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:49.289258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:49.289322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:49.289363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:49.289374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:49.289435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:49.289488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:49.289549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:49.289606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:49.289651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:49.289658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:49.289715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:49.289766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:49.289841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:49.289891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:49.289928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:49.289933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:49.289983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:49.290047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:49.290103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:49.290156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:49.290194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:49.290202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:49.290252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:49.290302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:49.290361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:49.290413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:49.290450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:49.290457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:49.290507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:49.290557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:49.290615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:49.290668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:49.290705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:49.290709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:49.290748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342025044895849:9144], now have 1 active actors on pipe
2025-10-09T21:47:49.290957Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:49.290972Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:49.290976Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:49.291008Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1036 reason: error at write: Disconnected sessionId: src_id|3283a697-f8c1e833-f3c04957-33600a92_0
2025-10-09T21:47:49.291013Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342025044895849:9144]
2025-10-09T21:47:49.291203Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1036 sessionId: src_id|3283a697-f8c1e833-f3c04957-33600a92_0 is DEAD
2025-10-09T21:47:49.291370Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3283a697-f8c1e833-f3c04957-33600a92_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:49.291380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3283a697-f8c1e833-f3c04957-33600a92_0] Write session will restart in 0.009273s
2025-10-09T21:47:49.291413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3283a697-f8c1e833-f3c04957-33600a92_0] Write session: Do CDS request
2025-10-09T21:47:49.291420Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3283a697-f8c1e833-f3c04957-33600a92_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:49.291546Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:49.301106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:49.301597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3283a697-f8c1e833-f3c04957-33600a92_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:49.301671Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:49.301697Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1037
2025-10-09T21:47:49.301834Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1037 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:49.301874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1037 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:49.301885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1037 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:49.301889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:49.302308Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1037 sessionId:  describe result for acl check
2025-10-09T21:47:49.302350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:49.302358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:49.302360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:49.302366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342025044895931:9353] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:49.302896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342025044895931:9353] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:49.303153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:49.303341Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:49.303348Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:49.303359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:49.303409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:49.303447Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342024683075171:8387]
2025-10-09T21:47:49.303989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:49.304284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:49.304328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:49.304571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:49.304604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.304610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.304611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.304612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.304613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.304615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.304616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.304617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.304618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.304623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.304632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.304635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.304637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.304639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.304642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.304644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.304646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.304647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.304652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.304657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.304661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.304664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.304675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.304678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.304680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.304683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.304686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.304689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.304692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.304695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.304698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.304701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.304703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.304706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.304710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.304713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.304728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:49.305389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342025044895931:9353] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:49.310543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342025044895931:9353] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:49.310563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342025044895931:9353] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:49.310567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342025044895931:9353] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:49.310576Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1037 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:49.310862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:49.311881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:49.311893Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:49.311896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:49.311900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:49.311907Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1036 [2:7559342024683075171:8387]
2025-10-09T21:47:49.311921Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:49.311938Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:49.311944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:49.311946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:49.311948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:49.311962Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:49.311968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:49.311974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:49.312028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.312034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.312037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.312040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:49.312099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:49.312105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.312118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.312131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.312147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:49.312173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342025044895857:2432], now have 1 active actors on pipe
2025-10-09T21:47:49.312182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342025044895970:9353], now have 1 active actors on pipe
2025-10-09T21:47:49.312190Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342024683075165:8387]
2025-10-09T21:47:49.312193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342024683075165:8387]
2025-10-09T21:47:49.312195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342024683075165:8387]
2025-10-09T21:47:49.312206Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:49.312219Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:49.312196Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1036
2025-10-09T21:47:49.312221Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:49.312222Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:49.312217Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1036
2025-10-09T21:47:49.312224Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:49.312236Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:49.312238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:49.312296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:49.312304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:49.312314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342024683075165:8387]
2025-10-09T21:47:49.312323Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:49.312329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:49.312349Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5a3a3286-4e135ac5-db5ea350-10bb74ca_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:49.312358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:49.312362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:49.312365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:49.312374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:49.312385Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:49.312411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:49.312535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:49.312544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:49.312560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:49.312629Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1037 partition: 0 MaxSeqNo: 25 sessionId: src_id|5a3a3286-4e135ac5-db5ea350-10bb74ca_0
2025-10-09T21:47:49.312952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3283a697-f8c1e833-f3c04957-33600a92_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5a3a3286-4e135ac5-db5ea350-10bb74ca_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:49.315460Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a3a3286-4e135ac5-db5ea350-10bb74ca_0] Write session: try to update token
2025-10-09T21:47:49.315481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a3a3286-4e135ac5-db5ea350-10bb74ca_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:49.326709Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:49.326729Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:49.326732Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:49.326734Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:49.326744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342024683075165:8387]
2025-10-09T21:47:49.326749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342024683075165:8387]
2025-10-09T21:47:49.326753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342024683075165:8387]
2025-10-09T21:47:49.326799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:49.326805Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:49.326810Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:49.326811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.326814Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:49.326824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:49.326954Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:49.328605Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1037 sessionId: src_id|5a3a3286-4e135ac5-db5ea350-10bb74ca_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:49.331325Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:49.391350Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342025044895978:9144]
2025-10-09T21:47:49.391605Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 1036, pipe: [1:7559342025044895978:9144]
2025-10-09T21:47:49.477566Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:49.477877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:49.477892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:49.477930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342025044895978:9144], now have 1 active actors on pipe
2025-10-09T21:47:49.478161Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:49.478146Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342025044895978:9144]
2025-10-09T21:47:49.478152Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:49.478188Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:49.478191Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:49.478246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1037 reason: error at write: Disconnected sessionId: src_id|5a3a3286-4e135ac5-db5ea350-10bb74ca_0
2025-10-09T21:47:49.478435Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1037 sessionId: src_id|5a3a3286-4e135ac5-db5ea350-10bb74ca_0 is DEAD
2025-10-09T21:47:49.478583Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a3a3286-4e135ac5-db5ea350-10bb74ca_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:49.478594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a3a3286-4e135ac5-db5ea350-10bb74ca_0] Write session will restart in 0.005541s
2025-10-09T21:47:49.478622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a3a3286-4e135ac5-db5ea350-10bb74ca_0] Write session: Do CDS request
2025-10-09T21:47:49.478627Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a3a3286-4e135ac5-db5ea350-10bb74ca_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:49.485433Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a3a3286-4e135ac5-db5ea350-10bb74ca_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:49.485610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:49.485627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1038
2025-10-09T21:47:49.485724Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1038 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:49.485780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1038 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:49.485790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1038 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:49.485798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:49.486242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1038 sessionId:  describe result for acl check
2025-10-09T21:47:49.486279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:49.486287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:49.486289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:49.486295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342025044896054:9357] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:49.486384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:49.486850Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342025044896054:9357] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:49.488298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:49.488449Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:49.488455Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:49.488464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:49.488501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:49.488533Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342024683075282:8389]
2025-10-09T21:47:49.488874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:49.489093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:49.489091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342025044896054:9357] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:49.489119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:49.489309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:49.489358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.489363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.489364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.489364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.489365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.489365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.489366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.489367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.489367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.489370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.489376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.489378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.489380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.489382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.489384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.489386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.489388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.489390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.489394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.489399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.489402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.489405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.489408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.489411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.489413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.489416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.489418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.489421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.489424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.489427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.489431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.489433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.489436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.489438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.489442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.489445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.489460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:49.494137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342025044896054:9357] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:49.494153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342025044896054:9357] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:49.494157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342025044896054:9357] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:49.494166Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1038 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:49.495313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:49.496324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:49.496338Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:49.496341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:49.496343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:49.496348Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1037 [2:7559342024683075282:8389]
2025-10-09T21:47:49.496358Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:49.496371Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:49.496373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:49.496376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:49.496377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:49.496390Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:49.496397Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:49.496402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:49.496446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.496451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.496452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.496454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:49.496517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:49.496524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.496531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.496537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.496546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:49.496562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342025044895980:2432], now have 1 active actors on pipe
2025-10-09T21:47:49.496569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342025044896094:9357], now have 1 active actors on pipe
2025-10-09T21:47:49.496575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342024683075277:8389]
2025-10-09T21:47:49.496576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342024683075277:8389]
2025-10-09T21:47:49.496578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342024683075277:8389]
2025-10-09T21:47:49.496586Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:49.496598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:49.496600Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1037
2025-10-09T21:47:49.496605Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:49.496608Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:49.496610Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:49.496632Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1037
2025-10-09T21:47:49.496612Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:49.496614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:49.496638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342024683075277:8389]
2025-10-09T21:47:49.496641Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:49.496643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:49.496665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:49.496671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:49.496700Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|81338e03-6d4c64c2-32e4499b-97cb86ea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:49.496711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:49.496714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:49.496716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:49.496725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:49.496740Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:49.496760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:49.496890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:49.496897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:49.496909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:49.496983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1038 partition: 0 MaxSeqNo: 25 sessionId: src_id|81338e03-6d4c64c2-32e4499b-97cb86ea_0
2025-10-09T21:47:49.497326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a3a3286-4e135ac5-db5ea350-10bb74ca_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|81338e03-6d4c64c2-32e4499b-97cb86ea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:49.499822Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|81338e03-6d4c64c2-32e4499b-97cb86ea_0] Write session: try to update token
2025-10-09T21:47:49.499840Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|81338e03-6d4c64c2-32e4499b-97cb86ea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:49.511358Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:49.511373Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:49.511375Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:49.511376Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:49.511384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342024683075277:8389]
2025-10-09T21:47:49.511387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342024683075277:8389]
2025-10-09T21:47:49.511390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342024683075277:8389]
2025-10-09T21:47:49.511425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:49.511431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.511433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:49.511442Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:49.511448Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:49.511452Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:49.511557Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:49.513454Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1038 sessionId: src_id|81338e03-6d4c64c2-32e4499b-97cb86ea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:49.515877Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:49.578235Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342025044896102:9144]
2025-10-09T21:47:49.578491Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 1037, pipe: [1:7559342025044896102:9144]
2025-10-09T21:47:49.659217Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:49.659688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:49.659709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:49.659757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342025044896102:9144], now have 1 active actors on pipe
2025-10-09T21:47:49.660007Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:49.660011Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:49.660016Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:49.660020Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:49.660074Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342025044896102:9144]
2025-10-09T21:47:49.660088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1038 reason: error at write: Disconnected sessionId: src_id|81338e03-6d4c64c2-32e4499b-97cb86ea_0
2025-10-09T21:47:49.660262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1038 sessionId: src_id|81338e03-6d4c64c2-32e4499b-97cb86ea_0 is DEAD
2025-10-09T21:47:49.660449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|81338e03-6d4c64c2-32e4499b-97cb86ea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:49.660461Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|81338e03-6d4c64c2-32e4499b-97cb86ea_0] Write session will restart in 0.008253s
2025-10-09T21:47:49.660486Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|81338e03-6d4c64c2-32e4499b-97cb86ea_0] Write session: Do CDS request
2025-10-09T21:47:49.660493Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|81338e03-6d4c64c2-32e4499b-97cb86ea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:49.669499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:49.669721Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|81338e03-6d4c64c2-32e4499b-97cb86ea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:49.669793Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:49.669820Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1039
2025-10-09T21:47:49.669951Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1039 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:49.669987Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1039 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:49.669994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1039 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:49.669997Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:49.670387Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1039 sessionId:  describe result for acl check
2025-10-09T21:47:49.670417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:49.670418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:49.670419Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:49.670423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342025044896182:9361] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:49.670845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342025044896182:9361] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:49.671235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:49.671414Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:49.671425Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:49.671436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:49.671480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:49.671520Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342024683075429:8403]
2025-10-09T21:47:49.672001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:49.672303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:49.672363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:49.672696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:49.672757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.672764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.672765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.672766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.672768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.672769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.672770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.672771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.672772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.672777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.672772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342025044896182:9361] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:49.672787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.672791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.672795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.672799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.672803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.672806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.672810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.672813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.672820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.672828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.672834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.672839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.672843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.672848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.672852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.672857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.672862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.672867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.672870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.672877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.672887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.672892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.672901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.672906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.672916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.672921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.672953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:49.678670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342025044896182:9361] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:49.678686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342025044896182:9361] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:49.678693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342025044896182:9361] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:49.678703Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1039 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:49.679040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:49.680139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:49.680156Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:49.680159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:49.680163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:49.680170Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1038 [2:7559342024683075429:8403]
2025-10-09T21:47:49.680184Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:49.680206Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:49.680213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:49.680217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:49.680219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:49.680237Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:49.680249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:49.680256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:49.680314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.680328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.680330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.680333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:49.680402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:49.680409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.680421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.680436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.680452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:49.680479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342025044896106:2432], now have 1 active actors on pipe
2025-10-09T21:47:49.680487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342025044896219:9361], now have 1 active actors on pipe
2025-10-09T21:47:49.680495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342024683075424:8403]
2025-10-09T21:47:49.680501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342024683075424:8403]
2025-10-09T21:47:49.680504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342024683075424:8403]
2025-10-09T21:47:49.680518Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:49.680530Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:49.680534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:49.680539Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:49.680541Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:49.680548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:49.680528Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1038
2025-10-09T21:47:49.680551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:49.680535Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1038
2025-10-09T21:47:49.680583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342024683075424:8403]
2025-10-09T21:47:49.680597Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:49.680603Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:49.680617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:49.680622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:49.680672Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9823db4d-758f321a-83a8fd01-e68c7efa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:49.680686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:49.680690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:49.680693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:49.680703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:49.680717Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:49.680737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:49.680897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:49.680904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:49.680918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:49.680997Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1039 partition: 0 MaxSeqNo: 25 sessionId: src_id|9823db4d-758f321a-83a8fd01-e68c7efa_0
2025-10-09T21:47:49.681353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|81338e03-6d4c64c2-32e4499b-97cb86ea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9823db4d-758f321a-83a8fd01-e68c7efa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:49.683931Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9823db4d-758f321a-83a8fd01-e68c7efa_0] Write session: try to update token
2025-10-09T21:47:49.683962Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9823db4d-758f321a-83a8fd01-e68c7efa_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:49.694602Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:49.694620Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:49.694622Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:49.694624Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:49.694634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342024683075424:8403]
2025-10-09T21:47:49.694638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342024683075424:8403]
2025-10-09T21:47:49.694642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342024683075424:8403]
2025-10-09T21:47:49.694686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:49.694698Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:49.694702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.694703Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:49.694707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:49.694708Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:49.694832Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:49.697512Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1039 sessionId: src_id|9823db4d-758f321a-83a8fd01-e68c7efa_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:49.700059Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:49.760502Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342025044896239:9144]
2025-10-09T21:47:49.760871Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 1038, pipe: [1:7559342025044896239:9144]
2025-10-09T21:47:49.849134Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:49.849631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:49.849648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:49.849694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342025044896239:9144], now have 1 active actors on pipe
2025-10-09T21:47:49.849947Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:49.850012Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342025044896239:9144]
2025-10-09T21:47:49.850111Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:49.850188Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1039 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9823db4d-758f321a-83a8fd01-e68c7efa_0
2025-10-09T21:47:49.849913Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:49.850366Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1039 sessionId: src_id|9823db4d-758f321a-83a8fd01-e68c7efa_0 is DEAD
2025-10-09T21:47:49.850566Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9823db4d-758f321a-83a8fd01-e68c7efa_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:49.850579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9823db4d-758f321a-83a8fd01-e68c7efa_0] Write session will restart in 0.006423s
2025-10-09T21:47:49.850608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9823db4d-758f321a-83a8fd01-e68c7efa_0] Write session: Do CDS request
2025-10-09T21:47:49.850616Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9823db4d-758f321a-83a8fd01-e68c7efa_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:49.861328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:49.862592Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9823db4d-758f321a-83a8fd01-e68c7efa_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:49.862648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:49.862666Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1040
2025-10-09T21:47:49.862799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1040 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:49.862831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1040 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:49.862841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1040 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:49.862844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:49.863250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1040 sessionId:  describe result for acl check
2025-10-09T21:47:49.863292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:49.863298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:49.863300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:49.863306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342025044896334:9373] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:49.863321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:49.863512Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:49.863521Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:49.863533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:49.863595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:49.863638Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342024683075552:8407]
2025-10-09T21:47:49.863850Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342025044896334:9373] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:49.864166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:49.864806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:49.864849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:49.865135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:49.865182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.865187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.865188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.865190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.865191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.865192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.865193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.865194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.865196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.865200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.865210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.865214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.865218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.865222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.865226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.865229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.865232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.865236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.865248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.865255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:49.865261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.865266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:49.865271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.865276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:49.865280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.865285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:49.865290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.865299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:49.865303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.865308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:49.865313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.865318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.865322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.865327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.865331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.865336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.865366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:49.866012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342025044896334:9373] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:49.871322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342025044896334:9373] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:49.871340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342025044896334:9373] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:49.871344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342025044896334:9373] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:49.871354Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1040 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:49.871773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:49.872963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:49.872980Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:49.872983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:49.872988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:49.872994Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1039 [2:7559342024683075552:8407]
2025-10-09T21:47:49.873008Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:49.873023Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:49.873032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:49.873035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:49.873038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:49.873055Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:49.873067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:49.873080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:49.873144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.873151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.873153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.873156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:49.873226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:49.873235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.873247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:49.873260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:49.873277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:49.873305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342025044896369:9373], now have 1 active actors on pipe
2025-10-09T21:47:49.873313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342025044896256:2432], now have 1 active actors on pipe
2025-10-09T21:47:49.873320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342024683075546:8407]
2025-10-09T21:47:49.873322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342024683075546:8407]
2025-10-09T21:47:49.873324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342024683075546:8407]
2025-10-09T21:47:49.873336Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:49.873329Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1039
2025-10-09T21:47:49.873352Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:49.873329Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1039
2025-10-09T21:47:49.873353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:49.873361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:49.873363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:49.873369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:49.873372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:49.873421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:49.873431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:49.873443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342024683075546:8407]
2025-10-09T21:47:49.873455Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:49.873468Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:49.873489Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9c72b8ce-f5ce6dbd-9e96b3fa-fdab5d46_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:49.873502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:49.873505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:49.873508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:49.873518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:49.873533Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:49.873557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:49.873735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:49.873742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:49.873756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:49.873848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1040 partition: 0 MaxSeqNo: 25 sessionId: src_id|9c72b8ce-f5ce6dbd-9e96b3fa-fdab5d46_0
2025-10-09T21:47:49.874244Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9823db4d-758f321a-83a8fd01-e68c7efa_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9c72b8ce-f5ce6dbd-9e96b3fa-fdab5d46_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:49.876618Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c72b8ce-f5ce6dbd-9e96b3fa-fdab5d46_0] Write session: try to update token
2025-10-09T21:47:49.876636Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c72b8ce-f5ce6dbd-9e96b3fa-fdab5d46_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:49.887264Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:49.887280Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:49.887282Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:49.887284Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:49.887295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342024683075546:8407]
2025-10-09T21:47:49.887301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342024683075546:8407]
2025-10-09T21:47:49.887305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342024683075546:8407]
2025-10-09T21:47:49.887347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:49.887349Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:49.887355Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:49.887355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:49.887358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:49.887359Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:49.887499Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:49.890211Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1040 sessionId: src_id|9c72b8ce-f5ce6dbd-9e96b3fa-fdab5d46_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:49.892484Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:49.949924Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342025044896376:9144]
2025-10-09T21:47:49.950218Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 1039, pipe: [1:7559342025044896376:9144]
2025-10-09T21:47:50.041170Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:50.041455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:50.041472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:50.041518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342025044896376:9144], now have 1 active actors on pipe
2025-10-09T21:47:50.041706Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:50.041734Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:50.041748Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:50.041758Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:50.041815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1040 reason: error at write: Disconnected sessionId: src_id|9c72b8ce-f5ce6dbd-9e96b3fa-fdab5d46_0
2025-10-09T21:47:50.041815Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342025044896376:9144]
2025-10-09T21:47:50.042047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1040 sessionId: src_id|9c72b8ce-f5ce6dbd-9e96b3fa-fdab5d46_0 is DEAD
2025-10-09T21:47:50.042178Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c72b8ce-f5ce6dbd-9e96b3fa-fdab5d46_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:50.042189Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c72b8ce-f5ce6dbd-9e96b3fa-fdab5d46_0] Write session will restart in 0.009366s
2025-10-09T21:47:50.042216Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c72b8ce-f5ce6dbd-9e96b3fa-fdab5d46_0] Write session: Do CDS request
2025-10-09T21:47:50.042223Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c72b8ce-f5ce6dbd-9e96b3fa-fdab5d46_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:50.051265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:50.052576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c72b8ce-f5ce6dbd-9e96b3fa-fdab5d46_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:50.052639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:50.052658Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1041
2025-10-09T21:47:50.052797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1041 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:50.052832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1041 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:50.052841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1041 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:50.052845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:50.053088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:50.053282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1041 sessionId:  describe result for acl check
2025-10-09T21:47:50.053295Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:50.053304Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:50.053327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:50.053316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:50.053334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:50.053336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:50.053342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342029339863757:9377] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:50.053370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:50.053408Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342028978042962:8409]
2025-10-09T21:47:50.053915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342029339863757:9377] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:50.054045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:50.054317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:50.054368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:50.054666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:50.054714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.054720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.054721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.054722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.054723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.054724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.054725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.054726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.054727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.054732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.054740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.054743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.054746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.054749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.054753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.054756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.054759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.054762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.054769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.054776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.054781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.054785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.054789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.054793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.054797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.054801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.054805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.054809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.054812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.054817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.054821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.054825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.054828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.054832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.054836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.054840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.054864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:50.055718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342029339863757:9377] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:50.060840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342029339863757:9377] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:50.060858Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342029339863757:9377] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:50.060862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342029339863757:9377] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:50.060872Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1041 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:50.061193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:50.062322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:50.062337Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:50.062340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:50.062344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:50.062350Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1040 [2:7559342028978042962:8409]
2025-10-09T21:47:50.062367Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:50.062384Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:50.062387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:50.062391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:50.062393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:50.062411Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:50.062421Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:50.062429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:50.062487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.062493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.062495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.062497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:50.062571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:50.062579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.062589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.062599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.062618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:50.062643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342029339863679:2432], now have 1 active actors on pipe
2025-10-09T21:47:50.062651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342029339863793:9377], now have 1 active actors on pipe
2025-10-09T21:47:50.062659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342028978042957:8409]
2025-10-09T21:47:50.062665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342028978042957:8409]
2025-10-09T21:47:50.062667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342028978042957:8409]
2025-10-09T21:47:50.062683Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:50.062698Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:50.062704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:50.062708Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:50.062710Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:50.062711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:50.062713Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:50.062711Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1040
2025-10-09T21:47:50.062747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342028978042957:8409]
2025-10-09T21:47:50.062713Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1040
2025-10-09T21:47:50.062754Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:50.062763Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:50.062806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:50.062815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:50.062851Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7094ef02-3803ad3c-70c1131d-3e9eddfd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:50.062862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:50.062865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:50.062867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:50.062876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:50.062889Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:50.062911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:50.063038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:50.063045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:50.063059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:50.063154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1041 partition: 0 MaxSeqNo: 25 sessionId: src_id|7094ef02-3803ad3c-70c1131d-3e9eddfd_0
2025-10-09T21:47:50.063524Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c72b8ce-f5ce6dbd-9e96b3fa-fdab5d46_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7094ef02-3803ad3c-70c1131d-3e9eddfd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:50.066015Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7094ef02-3803ad3c-70c1131d-3e9eddfd_0] Write session: try to update token
2025-10-09T21:47:50.066051Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7094ef02-3803ad3c-70c1131d-3e9eddfd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:50.077002Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:50.077018Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:50.077021Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:50.077022Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:50.077032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342028978042957:8409]
2025-10-09T21:47:50.077036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342028978042957:8409]
2025-10-09T21:47:50.077039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342028978042957:8409]
2025-10-09T21:47:50.077103Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:50.077111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:50.077118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.077120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:50.077123Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:50.077129Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:50.077254Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:50.079221Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1041 sessionId: src_id|7094ef02-3803ad3c-70c1131d-3e9eddfd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:50.081575Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:50.141946Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342029339863804:9144]
2025-10-09T21:47:50.142274Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 1040, pipe: [1:7559342029339863804:9144]
2025-10-09T21:47:50.227011Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:50.227273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:50.227289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:50.227333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342029339863804:9144], now have 1 active actors on pipe
2025-10-09T21:47:50.227543Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:50.227573Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:50.227585Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:50.227601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:50.227631Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342029339863804:9144]
2025-10-09T21:47:50.227663Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1041 reason: error at write: Disconnected sessionId: src_id|7094ef02-3803ad3c-70c1131d-3e9eddfd_0
2025-10-09T21:47:50.227868Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1041 sessionId: src_id|7094ef02-3803ad3c-70c1131d-3e9eddfd_0 is DEAD
2025-10-09T21:47:50.228048Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7094ef02-3803ad3c-70c1131d-3e9eddfd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:50.228060Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7094ef02-3803ad3c-70c1131d-3e9eddfd_0] Write session will restart in 0.009192s
2025-10-09T21:47:50.228088Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7094ef02-3803ad3c-70c1131d-3e9eddfd_0] Write session: Do CDS request
2025-10-09T21:47:50.228095Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7094ef02-3803ad3c-70c1131d-3e9eddfd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:50.237195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:50.238598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7094ef02-3803ad3c-70c1131d-3e9eddfd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:50.238660Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:50.238679Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1042
2025-10-09T21:47:50.238904Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1042 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:50.238949Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1042 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:50.238960Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1042 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:50.238964Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:50.239440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1042 sessionId:  describe result for acl check
2025-10-09T21:47:50.239478Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:50.239485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:50.239486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:50.239493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342029339863891:9382] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:50.239477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:50.239688Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:50.239699Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:50.239710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:50.239762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:50.239802Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342028978043079:8411]
2025-10-09T21:47:50.240071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342029339863891:9382] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:50.240308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:50.240606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:50.240645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:50.240946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:50.240993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.240998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.241000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.241001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.241002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.241003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.241005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.241006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.241007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.241012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.241020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.241023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.241027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.241030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.241033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.241036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.241039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.241042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.241048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.241055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.241060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.241065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.241069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.241073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.241077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.241081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.241085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.241089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.241093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.241097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.241101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.241106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.241110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.241114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.241118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.241122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.241145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:50.242199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342029339863891:9382] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:50.247412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342029339863891:9382] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:50.247434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342029339863891:9382] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:50.247438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342029339863891:9382] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:50.247448Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1042 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:50.247738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:50.249045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:50.249061Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:50.249064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:50.249069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:50.249075Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1041 [2:7559342028978043079:8411]
2025-10-09T21:47:50.249090Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:50.249110Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:50.249114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:50.249117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:50.249119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:50.249138Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:50.249144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:50.249151Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:50.249216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.249227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.249229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.249232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:50.249303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:50.249310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.249321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.249330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.249344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:50.249372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342028978043073:8411]
2025-10-09T21:47:50.249380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342028978043073:8411]
2025-10-09T21:47:50.249384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342028978043073:8411]
2025-10-09T21:47:50.249394Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:50.249405Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:50.249410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:50.249414Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:50.249417Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:50.249425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:50.249428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:50.249439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342029339863811:2432], now have 1 active actors on pipe
2025-10-09T21:47:50.249442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342029339863929:9382], now have 1 active actors on pipe
2025-10-09T21:47:50.249510Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1041
2025-10-09T21:47:50.249568Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1041
2025-10-09T21:47:50.249480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342028978043073:8411]
2025-10-09T21:47:50.249494Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:50.249507Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:50.249705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:50.249712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:50.249741Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9595e618-6b7ae941-44fcb59e-afdfa0c8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:50.249753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:50.249758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:50.249770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:50.249782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:50.249796Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:50.249826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:50.250080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:50.250093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:50.250130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:50.250277Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1042 partition: 0 MaxSeqNo: 25 sessionId: src_id|9595e618-6b7ae941-44fcb59e-afdfa0c8_0
2025-10-09T21:47:50.250631Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7094ef02-3803ad3c-70c1131d-3e9eddfd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9595e618-6b7ae941-44fcb59e-afdfa0c8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:50.253205Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9595e618-6b7ae941-44fcb59e-afdfa0c8_0] Write session: try to update token
2025-10-09T21:47:50.253224Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9595e618-6b7ae941-44fcb59e-afdfa0c8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:50.264633Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:50.264653Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:50.264656Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:50.264658Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:50.264669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342028978043073:8411]
2025-10-09T21:47:50.264674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342028978043073:8411]
2025-10-09T21:47:50.264683Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342028978043073:8411]
2025-10-09T21:47:50.264732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:50.264732Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:50.264738Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:50.264742Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:50.264746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.264749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:50.264922Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:50.267824Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1042 sessionId: src_id|9595e618-6b7ae941-44fcb59e-afdfa0c8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:50.270249Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:50.327810Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342029339863936:9144]
2025-10-09T21:47:50.328060Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 1041, pipe: [1:7559342029339863936:9144]
2025-10-09T21:47:50.420839Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:50.421126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:50.421139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:50.421182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342029339863936:9144], now have 1 active actors on pipe
2025-10-09T21:47:50.421376Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:50.421405Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342029339863936:9144]
2025-10-09T21:47:50.421420Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:50.421444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:50.421508Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1042 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9595e618-6b7ae941-44fcb59e-afdfa0c8_0
2025-10-09T21:47:50.421658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1042 sessionId: src_id|9595e618-6b7ae941-44fcb59e-afdfa0c8_0 is DEAD
2025-10-09T21:47:50.421841Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9595e618-6b7ae941-44fcb59e-afdfa0c8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:50.421850Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9595e618-6b7ae941-44fcb59e-afdfa0c8_0] Write session will restart in 0.008383s
2025-10-09T21:47:50.421878Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9595e618-6b7ae941-44fcb59e-afdfa0c8_0] Write session: Do CDS request
2025-10-09T21:47:50.421885Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9595e618-6b7ae941-44fcb59e-afdfa0c8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:50.431130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:50.431518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9595e618-6b7ae941-44fcb59e-afdfa0c8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:50.431611Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:50.431630Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1043
2025-10-09T21:47:50.431788Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1043 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:50.431827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1043 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:50.431836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1043 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:50.431839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:50.432273Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1043 sessionId:  describe result for acl check
2025-10-09T21:47:50.432312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:50.432320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:50.432321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:50.432327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342029339864019:9386] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:50.432901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342029339864019:9386] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:50.432918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:50.433097Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:50.433111Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:50.433122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:50.433169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:50.433213Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342028978043195:8413]
2025-10-09T21:47:50.433714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:50.434121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:50.434172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:50.434524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:50.434571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.434576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.434578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.434579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.434580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.434581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.434582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.434583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.434585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.434590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.434600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.434604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.434608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.434612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.434617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.434620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.434624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.434628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.434634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.434642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.434648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.434654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.434658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.434667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.434671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.434676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.434681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.434686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.434690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.434695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.434700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.434712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.434717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.434722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.434731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.434736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.434765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:50.435216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342029339864019:9386] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:50.440867Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342029339864019:9386] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:50.440887Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342029339864019:9386] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:50.440890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342029339864019:9386] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:50.440900Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1043 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:50.441233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:50.442353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:50.442372Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:50.442374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:50.442379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:50.442386Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1042 [2:7559342028978043195:8413]
2025-10-09T21:47:50.442399Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:50.442414Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:50.442416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:50.442418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:50.442419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:50.442432Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:50.442442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:50.442448Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:50.442491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.442497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.442499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.442501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:50.442568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:50.442574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.442582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.442589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.442599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:50.442618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342029339863939:2432], now have 1 active actors on pipe
2025-10-09T21:47:50.442621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342029339864058:9386], now have 1 active actors on pipe
2025-10-09T21:47:50.442629Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342028978043190:8413]
2025-10-09T21:47:50.442635Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342028978043190:8413]
2025-10-09T21:47:50.442637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342028978043190:8413]
2025-10-09T21:47:50.442650Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:50.442663Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:50.442667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:50.442668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:50.442670Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:50.442681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:50.442689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:50.442687Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1042
2025-10-09T21:47:50.442689Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1042
2025-10-09T21:47:50.442725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342028978043190:8413]
2025-10-09T21:47:50.442738Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:50.442749Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:50.442767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:50.442772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:50.442808Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f7ce06b0-db8edf5-b44c4c9f-1df5c2e1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:50.442822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:50.442825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:50.442827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:50.442834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:50.442843Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:50.442857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:50.442991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:50.443002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:50.443024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:50.443108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1043 partition: 0 MaxSeqNo: 25 sessionId: src_id|f7ce06b0-db8edf5-b44c4c9f-1df5c2e1_0
2025-10-09T21:47:50.443466Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9595e618-6b7ae941-44fcb59e-afdfa0c8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f7ce06b0-db8edf5-b44c4c9f-1df5c2e1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:50.445770Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7ce06b0-db8edf5-b44c4c9f-1df5c2e1_0] Write session: try to update token
2025-10-09T21:47:50.445788Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7ce06b0-db8edf5-b44c4c9f-1df5c2e1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:50.457165Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:50.457182Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:50.457184Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:50.457187Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:50.457197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342028978043190:8413]
2025-10-09T21:47:50.457201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342028978043190:8413]
2025-10-09T21:47:50.457206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342028978043190:8413]
2025-10-09T21:47:50.457250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:50.457258Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:50.457262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.457265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:50.457265Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:50.457268Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:50.457391Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:50.458661Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1043 sessionId: src_id|f7ce06b0-db8edf5-b44c4c9f-1df5c2e1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:50.460985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:50.521319Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342029339864068:9144]
2025-10-09T21:47:50.521628Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 1042, pipe: [1:7559342029339864068:9144]
2025-10-09T21:47:50.602549Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:50.602998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:50.603013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:50.603057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342029339864068:9144], now have 1 active actors on pipe
2025-10-09T21:47:50.603310Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:50.603328Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:50.603404Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1043 reason: error at write: Disconnected sessionId: src_id|f7ce06b0-db8edf5-b44c4c9f-1df5c2e1_0
2025-10-09T21:47:50.603268Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:50.603544Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:50.603585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1043 sessionId: src_id|f7ce06b0-db8edf5-b44c4c9f-1df5c2e1_0 is DEAD
2025-10-09T21:47:50.603600Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342029339864068:9144]
2025-10-09T21:47:50.604294Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7ce06b0-db8edf5-b44c4c9f-1df5c2e1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:50.604307Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7ce06b0-db8edf5-b44c4c9f-1df5c2e1_0] Write session will restart in 0.005238s
2025-10-09T21:47:50.604334Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7ce06b0-db8edf5-b44c4c9f-1df5c2e1_0] Write session: Do CDS request
2025-10-09T21:47:50.604341Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7ce06b0-db8edf5-b44c4c9f-1df5c2e1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:50.610571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7ce06b0-db8edf5-b44c4c9f-1df5c2e1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:50.610665Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:50.610684Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1044
2025-10-09T21:47:50.610828Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1044 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:50.610861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1044 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:50.610872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1044 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:50.610879Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:50.611422Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1044 sessionId:  describe result for acl check
2025-10-09T21:47:50.611470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:50.611472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:50.611474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:50.611480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342029339864096:9390] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:50.612030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342029339864096:9390] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:50.615226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342029339864096:9390] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:50.614962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:50.616803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:50.616990Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:50.617000Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:50.617019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:50.617072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:50.617114Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342028978043321:8415]
2025-10-09T21:47:50.617709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:50.617863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342029339864096:9390] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:50.617888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342029339864096:9390] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:50.617891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342029339864096:9390] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:50.617900Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1044 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:50.617982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:50.618023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:50.618339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:50.618389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.618395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.618396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.618397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.618398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.618399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.618400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.618401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.618402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.618408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.618418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.618422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.618426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.618430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.618434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.618437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.618441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.618445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.618451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.618458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.618464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.618469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.618474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.618479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.618489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.618494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.618499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.618503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.618508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.618513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.618518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.618523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.618528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.618533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.618537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.618542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.618572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:50.625259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:50.626649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:50.626666Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:50.626668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:50.626673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:50.626681Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1043 [2:7559342028978043321:8415]
2025-10-09T21:47:50.626695Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:50.626714Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:50.626718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:50.626723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:50.626727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:50.626746Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:50.626757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:50.626764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:50.626832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.626839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.626841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.626844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:50.626917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:50.626925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.626940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.626957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.626978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:50.627009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342029339864070:2432], now have 1 active actors on pipe
2025-10-09T21:47:50.627017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342029339864188:9390], now have 1 active actors on pipe
2025-10-09T21:47:50.627027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342028978043313:8415]
2025-10-09T21:47:50.627029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342028978043313:8415]
2025-10-09T21:47:50.627032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342028978043313:8415]
2025-10-09T21:47:50.627046Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:50.627064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:50.627066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:50.627081Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:50.627085Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:50.627088Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:50.627090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:50.627131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342028978043313:8415]
2025-10-09T21:47:50.627138Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:50.627141Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:50.627170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:50.627179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:50.627111Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1043
2025-10-09T21:47:50.627118Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1043
2025-10-09T21:47:50.627219Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b052d087-2772c970-8bf5e2f3-11691d39_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:50.627237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:50.627241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:50.627244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:50.627254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:50.627272Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:50.627297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:50.627474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:50.627482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:50.627499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:50.627652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1044 partition: 0 MaxSeqNo: 25 sessionId: src_id|b052d087-2772c970-8bf5e2f3-11691d39_0
2025-10-09T21:47:50.628150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7ce06b0-db8edf5-b44c4c9f-1df5c2e1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b052d087-2772c970-8bf5e2f3-11691d39_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:50.630709Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b052d087-2772c970-8bf5e2f3-11691d39_0] Write session: try to update token
2025-10-09T21:47:50.630726Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b052d087-2772c970-8bf5e2f3-11691d39_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:50.642119Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:50.642135Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:50.642137Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:50.642139Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:50.642146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342028978043313:8415]
2025-10-09T21:47:50.642149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342028978043313:8415]
2025-10-09T21:47:50.642152Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342028978043313:8415]
2025-10-09T21:47:50.642199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:50.642207Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:50.642210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.642212Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:50.642212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:50.642217Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:50.642351Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:50.645766Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1044 sessionId: src_id|b052d087-2772c970-8bf5e2f3-11691d39_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:50.648162Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:50.703637Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342029339864198:9144]
2025-10-09T21:47:50.703947Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 1043, pipe: [1:7559342029339864198:9144]
2025-10-09T21:47:50.799377Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:50.799681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:50.799695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:50.799736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342029339864198:9144], now have 1 active actors on pipe
2025-10-09T21:47:50.800027Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:50.800243Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:50.800322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1044 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b052d087-2772c970-8bf5e2f3-11691d39_0
2025-10-09T21:47:50.800509Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1044 sessionId: src_id|b052d087-2772c970-8bf5e2f3-11691d39_0 is DEAD
2025-10-09T21:47:50.800678Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b052d087-2772c970-8bf5e2f3-11691d39_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:50.800689Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b052d087-2772c970-8bf5e2f3-11691d39_0] Write session will restart in 0.008745s
2025-10-09T21:47:50.800679Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:50.800715Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b052d087-2772c970-8bf5e2f3-11691d39_0] Write session: Do CDS request
2025-10-09T21:47:50.800723Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b052d087-2772c970-8bf5e2f3-11691d39_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:50.800730Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342029339864198:9144]
2025-10-09T21:47:50.810551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b052d087-2772c970-8bf5e2f3-11691d39_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:50.810640Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:50.810659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1045
2025-10-09T21:47:50.810782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1045 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:50.810821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1045 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:50.810834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1045 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:50.810837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:50.811189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1045 sessionId:  describe result for acl check
2025-10-09T21:47:50.811226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:50.811233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:50.811237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:50.811242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342029339864296:9401] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:50.811568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:50.811775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342029339864296:9401] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:50.813903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:50.814115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342029339864296:9401] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:50.814136Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:50.814147Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:50.814160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:50.814228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:50.814272Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342028978043468:8429]
2025-10-09T21:47:50.814844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:50.815208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:50.815277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:50.815614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:50.815668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.815676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.815677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.815678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.815680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.815680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.815682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.815683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.815684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.815690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.815698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.815702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.815705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.815708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.815712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.815715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.815719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.815722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.815730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.815738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:50.815747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.815752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:50.815759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.815764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:50.815768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.815781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:50.815785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.815790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:50.815801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.815806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:50.815817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.815821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.815825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.815829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.815834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.815839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.815868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:50.819578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342029339864296:9401] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:50.819596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342029339864296:9401] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:50.819600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342029339864296:9401] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:50.819609Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1045 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:50.823380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:50.824717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:50.824742Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:50.824745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:50.824750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:50.824759Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1044 [2:7559342028978043468:8429]
2025-10-09T21:47:50.824773Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:50.824795Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:50.824801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:50.824813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:50.824816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:50.824836Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:50.824853Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:50.824860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:50.824930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.824940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.824942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.824945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:50.825030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:50.825041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.825055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:50.825076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:50.825100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:50.825138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342029339864215:2432], now have 1 active actors on pipe
2025-10-09T21:47:50.825144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342029339864334:9401], now have 1 active actors on pipe
2025-10-09T21:47:50.825162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342028978043463:8429]
2025-10-09T21:47:50.825170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342028978043463:8429]
2025-10-09T21:47:50.825172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342028978043463:8429]
2025-10-09T21:47:50.825190Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:50.825219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:50.825232Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:50.825235Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:50.825237Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:50.825240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:50.825245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:50.825269Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1044
2025-10-09T21:47:50.825312Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1044
2025-10-09T21:47:50.825312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342028978043463:8429]
2025-10-09T21:47:50.825326Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:50.825358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:50.825374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:50.825380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:50.825416Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7db47275-51ccd9a6-14655ddb-7688f470_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:50.825432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:50.825437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:50.825446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:50.825459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:50.825479Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:50.825500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:50.825684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:50.825707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:50.825728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:50.825835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1045 partition: 0 MaxSeqNo: 25 sessionId: src_id|7db47275-51ccd9a6-14655ddb-7688f470_0
2025-10-09T21:47:50.826261Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b052d087-2772c970-8bf5e2f3-11691d39_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7db47275-51ccd9a6-14655ddb-7688f470_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:50.828765Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7db47275-51ccd9a6-14655ddb-7688f470_0] Write session: try to update token
2025-10-09T21:47:50.828785Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7db47275-51ccd9a6-14655ddb-7688f470_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:50.839424Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:50.839444Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:50.839447Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:50.839449Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:50.839460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342028978043463:8429]
2025-10-09T21:47:50.839465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342028978043463:8429]
2025-10-09T21:47:50.839470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342028978043463:8429]
2025-10-09T21:47:50.839517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:50.839531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:50.839534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:50.839551Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:50.839557Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:50.839561Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:50.839726Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:50.844891Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1045 sessionId: src_id|7db47275-51ccd9a6-14655ddb-7688f470_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:50.847404Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:50.882796Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:47:50.882830Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 10
2025-10-09T21:47:50.900747Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342029339864354:9144]
2025-10-09T21:47:50.901041Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 1044, pipe: [1:7559342029339864354:9144]
2025-10-09T21:47:50.994857Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:50.995269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:50.995284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:50.995352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342029339864354:9144], now have 1 active actors on pipe
2025-10-09T21:47:50.995549Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:50.995552Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342029339864354:9144]
2025-10-09T21:47:50.995587Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:50.995592Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:50.995650Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1045 reason: error at write: Disconnected sessionId: src_id|7db47275-51ccd9a6-14655ddb-7688f470_0
2025-10-09T21:47:50.996007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7db47275-51ccd9a6-14655ddb-7688f470_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:50.996019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7db47275-51ccd9a6-14655ddb-7688f470_0] Write session will restart in 0.005780s
2025-10-09T21:47:50.996042Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7db47275-51ccd9a6-14655ddb-7688f470_0] Write session: Do CDS request
2025-10-09T21:47:50.996049Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7db47275-51ccd9a6-14655ddb-7688f470_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:50.995847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1045 sessionId: src_id|7db47275-51ccd9a6-14655ddb-7688f470_0 is DEAD
2025-10-09T21:47:50.996040Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:50.996052Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 10
2025-10-09T21:47:51.002501Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7db47275-51ccd9a6-14655ddb-7688f470_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:51.002599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:51.002617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1046
2025-10-09T21:47:51.002724Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1046 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:51.002764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1046 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:51.002776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1046 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:51.002780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:51.003303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1046 sessionId:  describe result for acl check
2025-10-09T21:47:51.003342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:51.003348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:51.003349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:51.003355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342033634831708:9406] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:51.003919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342033634831708:9406] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:51.005505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:51.006387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342033634831708:9406] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:51.007426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:51.007635Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:51.007645Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:51.007657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:51.007706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:51.007747Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342033273010892:8434]
2025-10-09T21:47:51.008370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:51.009172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342033634831708:9406] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:51.009188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342033634831708:9406] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:51.009193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342033634831708:9406] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:51.009202Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1046 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:51.008701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:51.008743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:51.009050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:51.009100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.009101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.009103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.009104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.009105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.009106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.009107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.009108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.009110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.009114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.009124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.009127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.009131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.009134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.009137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.009140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.009144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.009147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.009154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.009162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.009168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.009174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.009178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.009184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.009188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.009192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.009196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.009201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.009204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.009209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.009215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.009220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.009224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.009228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.009235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.009240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.009265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:51.015963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:51.017175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:51.017192Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:51.017195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:51.017199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:51.017207Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1045 [2:7559342033273010892:8434]
2025-10-09T21:47:51.017221Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:51.017243Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:51.017251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:51.017255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:51.017257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:51.017276Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:51.017286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:51.017294Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:51.017355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.017363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.017365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.017368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:51.017443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:51.017451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.017464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.017476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.017502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:51.017524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342029339864359:2432], now have 1 active actors on pipe
2025-10-09T21:47:51.017533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342033634831772:9406], now have 1 active actors on pipe
2025-10-09T21:47:51.017542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342033273010887:8434]
2025-10-09T21:47:51.017544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342033273010887:8434]
2025-10-09T21:47:51.017546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342033273010887:8434]
2025-10-09T21:47:51.017561Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:51.017581Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:51.017592Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:51.017596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:51.017599Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:51.017602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:51.017605Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:51.017645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342033273010887:8434]
2025-10-09T21:47:51.017607Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1045
2025-10-09T21:47:51.017656Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:51.017659Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:51.017622Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1045
2025-10-09T21:47:51.017700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:51.017710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:51.017817Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:47:51.017861Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dc8b7a7d-b44d0b24-c68b9d03-b182dd35_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:51.017873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:51.017878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:51.017880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:51.017891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:51.017902Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:51.017927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:51.018076Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 13 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:47:51.018105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:51.018118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:51.018117Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:47:51.018141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:51.018221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1046 partition: 0 MaxSeqNo: 25 sessionId: src_id|dc8b7a7d-b44d0b24-c68b9d03-b182dd35_0
2025-10-09T21:47:51.018618Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7db47275-51ccd9a6-14655ddb-7688f470_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dc8b7a7d-b44d0b24-c68b9d03-b182dd35_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:51.021095Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dc8b7a7d-b44d0b24-c68b9d03-b182dd35_0] Write session: try to update token
2025-10-09T21:47:51.021113Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dc8b7a7d-b44d0b24-c68b9d03-b182dd35_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:51.032167Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:51.032185Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:51.032188Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:51.032189Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:51.032199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342033273010887:8434]
2025-10-09T21:47:51.032202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342033273010887:8434]
2025-10-09T21:47:51.032206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342033273010887:8434]
2025-10-09T21:47:51.032249Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:51.032250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:51.032258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.032261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:51.032265Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:51.032269Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:51.032396Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:51.034370Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1046 sessionId: src_id|dc8b7a7d-b44d0b24-c68b9d03-b182dd35_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:51.036965Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:51.095850Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342033634831784:9144]
2025-10-09T21:47:51.096168Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 1045, pipe: [1:7559342033634831784:9144]
2025-10-09T21:47:51.186507Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:51.186832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:51.186846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:51.186892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342033634831784:9144], now have 1 active actors on pipe
2025-10-09T21:47:51.187080Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:51.187163Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:51.187184Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:51.187188Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:51.187228Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342033634831784:9144]
2025-10-09T21:47:51.187261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1046 reason: error at write: Disconnected sessionId: src_id|dc8b7a7d-b44d0b24-c68b9d03-b182dd35_0
2025-10-09T21:47:51.187452Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1046 sessionId: src_id|dc8b7a7d-b44d0b24-c68b9d03-b182dd35_0 is DEAD
2025-10-09T21:47:51.187721Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc8b7a7d-b44d0b24-c68b9d03-b182dd35_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:51.187734Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc8b7a7d-b44d0b24-c68b9d03-b182dd35_0] Write session will restart in 0.009296s
2025-10-09T21:47:51.187762Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc8b7a7d-b44d0b24-c68b9d03-b182dd35_0] Write session: Do CDS request
2025-10-09T21:47:51.187769Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc8b7a7d-b44d0b24-c68b9d03-b182dd35_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:51.198073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:51.198568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dc8b7a7d-b44d0b24-c68b9d03-b182dd35_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:51.198654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:51.198675Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1047
2025-10-09T21:47:51.198827Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1047 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:51.198871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1047 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:51.198882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1047 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:51.198885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:51.199326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1047 sessionId:  describe result for acl check
2025-10-09T21:47:51.199369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:51.199377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:51.199380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:51.199387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342033634831880:9410] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:51.199940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342033634831880:9410] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:51.200567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:51.200768Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:51.200778Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:51.200791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:51.200845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:51.200885Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342033273011015:8437]
2025-10-09T21:47:51.201514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:51.201900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:51.201945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:51.202173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342033634831880:9410] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:51.202279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:51.202325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.202331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.202332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.202334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.202335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.202336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.202337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.202338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.202340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.202345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.202355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.202360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.202364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.202368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.202372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.202376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.202380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.202383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.202390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.202398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.202404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.202414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.202419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.202424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.202433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.202439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.202444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.202450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.202455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.202461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.202466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.202476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.202480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.202485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.202494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.202500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.202534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:51.208776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342033634831880:9410] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:51.208795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342033634831880:9410] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:51.208798Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342033634831880:9410] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:51.208806Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1047 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:51.209096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:51.210281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:51.210295Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:51.210297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:51.210302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:51.210309Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1046 [2:7559342033273011015:8437]
2025-10-09T21:47:51.210324Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:51.210342Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:51.210346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:51.210349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:51.210352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:51.210367Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:51.210375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:51.210383Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:51.210448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.210455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.210458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.210461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:51.210563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:51.210570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.210582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.210600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.210623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:51.210649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342033634831793:2432], now have 1 active actors on pipe
2025-10-09T21:47:51.210657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342033634831921:9410], now have 1 active actors on pipe
2025-10-09T21:47:51.210665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342033273011009:8437]
2025-10-09T21:47:51.210667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342033273011009:8437]
2025-10-09T21:47:51.210669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342033273011009:8437]
2025-10-09T21:47:51.210683Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:51.210695Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1046
2025-10-09T21:47:51.210735Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1046
2025-10-09T21:47:51.210698Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:51.210697Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:51.210701Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:51.210704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:51.210722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:51.210725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:51.210759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342033273011009:8437]
2025-10-09T21:47:51.210762Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:51.210765Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:51.210795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:51.210805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:51.210843Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|998aea4-8cc0d765-aab3b7f1-d4f406d1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:51.210858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:51.210861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:51.210864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:51.210874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:51.210884Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:51.210900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:51.211109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:51.211118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:51.211135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:51.211313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1047 partition: 0 MaxSeqNo: 25 sessionId: src_id|998aea4-8cc0d765-aab3b7f1-d4f406d1_0
2025-10-09T21:47:51.211654Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc8b7a7d-b44d0b24-c68b9d03-b182dd35_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|998aea4-8cc0d765-aab3b7f1-d4f406d1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:51.214238Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|998aea4-8cc0d765-aab3b7f1-d4f406d1_0] Write session: try to update token
2025-10-09T21:47:51.214267Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|998aea4-8cc0d765-aab3b7f1-d4f406d1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:51.225273Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:51.225291Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:51.225297Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:51.225299Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:51.225309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342033273011009:8437]
2025-10-09T21:47:51.225313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342033273011009:8437]
2025-10-09T21:47:51.225318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342033273011009:8437]
2025-10-09T21:47:51.225364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:51.225378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.225375Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:51.225380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:51.225384Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:51.225389Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:51.225530Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:51.229049Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1047 sessionId: src_id|998aea4-8cc0d765-aab3b7f1-d4f406d1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:51.231556Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:51.287295Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342033634831928:9144]
2025-10-09T21:47:51.287641Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 1046, pipe: [1:7559342033634831928:9144]
2025-10-09T21:47:51.371123Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:51.371390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:51.371404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:51.371440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342033634831928:9144], now have 1 active actors on pipe
2025-10-09T21:47:51.371689Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:51.371720Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:51.371746Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:51.371750Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:51.371791Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342033634831928:9144]
2025-10-09T21:47:51.371814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1047 reason: error at write: Disconnected sessionId: src_id|998aea4-8cc0d765-aab3b7f1-d4f406d1_0
2025-10-09T21:47:51.371992Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1047 sessionId: src_id|998aea4-8cc0d765-aab3b7f1-d4f406d1_0 is DEAD
2025-10-09T21:47:51.372150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|998aea4-8cc0d765-aab3b7f1-d4f406d1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:51.372162Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|998aea4-8cc0d765-aab3b7f1-d4f406d1_0] Write session will restart in 0.006053s
2025-10-09T21:47:51.372192Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|998aea4-8cc0d765-aab3b7f1-d4f406d1_0] Write session: Do CDS request
2025-10-09T21:47:51.372197Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|998aea4-8cc0d765-aab3b7f1-d4f406d1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:51.379533Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|998aea4-8cc0d765-aab3b7f1-d4f406d1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:51.379563Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:51.379579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1048
2025-10-09T21:47:51.379732Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1048 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:51.379765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1048 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:51.379775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1048 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:51.379779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:51.380217Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1048 sessionId:  describe result for acl check
2025-10-09T21:47:51.380249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:51.380256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:51.380257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:51.380263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342033634831987:9415] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:51.380899Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342033634831987:9415] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:51.382831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:51.383018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342033634831987:9415] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:51.384482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:51.384664Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:51.384675Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:51.384687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:51.384744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:51.384786Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342033273011138:8439]
2025-10-09T21:47:51.385056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342033634831987:9415] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:51.385084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342033634831987:9415] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:51.385091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342033634831987:9415] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:51.385098Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1048 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:51.385353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:51.385613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:51.385654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:51.385960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:51.386017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.386023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.386024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.386025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.386043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.386044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.386045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.386046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.386048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.386053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.386063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.386067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.386071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.386075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.386079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.386083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.386087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.386090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.386097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.386105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.386111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.386116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.386122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.386127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.386131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.386137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.386142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.386151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.386155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.386160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.386165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.386170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.386174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.386180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.386184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.386189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.386222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:51.393155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:51.394303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:51.394314Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:51.394317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:51.394321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:51.394327Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1047 [2:7559342033273011138:8439]
2025-10-09T21:47:51.394339Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:51.394357Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:51.394365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:51.394368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:51.394371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:51.394386Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:51.394400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:51.394411Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:51.394468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.394475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.394477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.394481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:51.394547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:51.394554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.394565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.394578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.394599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:51.394627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342033634831931:2432], now have 1 active actors on pipe
2025-10-09T21:47:51.394630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342033634832051:9415], now have 1 active actors on pipe
2025-10-09T21:47:51.394638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342033273011132:8439]
2025-10-09T21:47:51.394640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342033273011132:8439]
2025-10-09T21:47:51.394643Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342033273011132:8439]
2025-10-09T21:47:51.394655Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:51.394670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:51.394670Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:51.394674Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:51.394676Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:51.394690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:51.394692Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:51.394688Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1047
2025-10-09T21:47:51.394691Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1047
2025-10-09T21:47:51.394717Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342033273011132:8439]
2025-10-09T21:47:51.394725Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:51.394727Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:51.394748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:51.394753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:51.394788Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|384935bc-cc3e2d62-4ebc169b-cbd06f8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:51.394800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:51.394804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:51.394807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:51.394816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:51.394829Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:51.394850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:51.395003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:51.395012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:51.395027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:51.395120Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1048 partition: 0 MaxSeqNo: 25 sessionId: src_id|384935bc-cc3e2d62-4ebc169b-cbd06f8_0
2025-10-09T21:47:51.395433Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|998aea4-8cc0d765-aab3b7f1-d4f406d1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|384935bc-cc3e2d62-4ebc169b-cbd06f8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:51.397687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|384935bc-cc3e2d62-4ebc169b-cbd06f8_0] Write session: try to update token
2025-10-09T21:47:51.397701Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|384935bc-cc3e2d62-4ebc169b-cbd06f8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:51.410205Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:51.410223Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:51.410226Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:51.410228Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:51.410240Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342033273011132:8439]
2025-10-09T21:47:51.410244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342033273011132:8439]
2025-10-09T21:47:51.410248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342033273011132:8439]
2025-10-09T21:47:51.410287Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:51.410291Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:51.410294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:51.410295Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:51.410301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.410304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:51.410444Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:51.410758Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1048 sessionId: src_id|384935bc-cc3e2d62-4ebc169b-cbd06f8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:51.413236Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:51.472123Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342033634832062:9144]
2025-10-09T21:47:51.472347Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 1047, pipe: [1:7559342033634832062:9144]
2025-10-09T21:47:51.562164Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:51.562538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:51.562556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:51.562603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342033634832062:9144], now have 1 active actors on pipe
2025-10-09T21:47:51.562787Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:51.562820Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:51.562835Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:51.562850Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342033634832062:9144]
2025-10-09T21:47:51.562870Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:51.562898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1048 reason: error at write: Disconnected sessionId: src_id|384935bc-cc3e2d62-4ebc169b-cbd06f8_0
2025-10-09T21:47:51.563091Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1048 sessionId: src_id|384935bc-cc3e2d62-4ebc169b-cbd06f8_0 is DEAD
2025-10-09T21:47:51.563263Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|384935bc-cc3e2d62-4ebc169b-cbd06f8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:51.563274Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|384935bc-cc3e2d62-4ebc169b-cbd06f8_0] Write session will restart in 0.006035s
2025-10-09T21:47:51.563305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|384935bc-cc3e2d62-4ebc169b-cbd06f8_0] Write session: Do CDS request
2025-10-09T21:47:51.563309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|384935bc-cc3e2d62-4ebc169b-cbd06f8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:51.570553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|384935bc-cc3e2d62-4ebc169b-cbd06f8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:51.570615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:51.570634Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1049
2025-10-09T21:47:51.570754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1049 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:51.570786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1049 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:51.570792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1049 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:51.570796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:51.571269Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1049 sessionId:  describe result for acl check
2025-10-09T21:47:51.571308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:51.571311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:51.571312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:51.571318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342033634832130:9419] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:51.571807Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342033634832130:9419] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:51.573052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:51.573985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342033634832130:9419] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:51.574823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:51.575002Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:51.575013Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:51.575025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:51.575077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:51.575117Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342033273011261:8441]
2025-10-09T21:47:51.575556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:51.575816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:51.575856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:51.576109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:51.576140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342033634832130:9419] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:51.576155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.576154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342033634832130:9419] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:51.576157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342033634832130:9419] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:51.576164Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1049 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:51.576162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.576163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.576164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.576165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.576166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.576167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.576168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.576169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.576173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.576183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.576187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.576191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.576195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.576198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.576202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.576206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.576209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.576215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.576222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.576228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.576233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.576237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.576242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.576246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.576250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.576255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.576259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.576263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.576267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.576270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.576273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.576275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.576278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.576281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.576283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.576299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:51.583205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:51.584231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:51.584247Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:51.584250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:51.584254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:51.584261Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1048 [2:7559342033273011261:8441]
2025-10-09T21:47:51.584275Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:51.584293Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:51.584300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:51.584303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:51.584306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:51.584322Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:51.584332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:51.584340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:51.584401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.584407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.584409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.584412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:51.584480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:51.584486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.584498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.584510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.584527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:51.584556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342033634832188:9419], now have 1 active actors on pipe
2025-10-09T21:47:51.584559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342033634832065:2432], now have 1 active actors on pipe
2025-10-09T21:47:51.584566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342033273011255:8441]
2025-10-09T21:47:51.584569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342033273011255:8441]
2025-10-09T21:47:51.584572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342033273011255:8441]
2025-10-09T21:47:51.584585Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:51.584603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:51.584603Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:51.584606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:51.584608Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:51.584623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:51.584626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:51.584659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342033273011255:8441]
2025-10-09T21:47:51.584671Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:51.584676Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:51.584619Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1048
2025-10-09T21:47:51.584623Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1048
2025-10-09T21:47:51.584689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:51.584699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:51.584749Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|155290a7-3df5a591-842c7902-e84849a5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:51.584874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:51.584882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:51.584885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:51.584896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:51.584910Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:51.584938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:51.585076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:51.585086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:51.585100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:51.585194Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1049 partition: 0 MaxSeqNo: 25 sessionId: src_id|155290a7-3df5a591-842c7902-e84849a5_0
2025-10-09T21:47:51.585527Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|384935bc-cc3e2d62-4ebc169b-cbd06f8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|155290a7-3df5a591-842c7902-e84849a5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:51.587974Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|155290a7-3df5a591-842c7902-e84849a5_0] Write session: try to update token
2025-10-09T21:47:51.587994Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|155290a7-3df5a591-842c7902-e84849a5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:51.599912Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:51.599933Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:51.599936Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:51.599938Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:51.599949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342033273011255:8441]
2025-10-09T21:47:51.599954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342033273011255:8441]
2025-10-09T21:47:51.599959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342033273011255:8441]
2025-10-09T21:47:51.599997Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:51.600001Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:51.600002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:51.600004Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:51.600011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.600014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:51.600146Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:51.600802Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1049 sessionId: src_id|155290a7-3df5a591-842c7902-e84849a5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:51.603199Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:51.663088Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342033634832205:9144]
2025-10-09T21:47:51.663350Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 1048, pipe: [1:7559342033634832205:9144]
2025-10-09T21:47:51.753819Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:51.754188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:51.754203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:51.754249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342033634832205:9144], now have 1 active actors on pipe
2025-10-09T21:47:51.754525Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342033634832205:9144]
2025-10-09T21:47:51.754533Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:51.754568Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:51.754572Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:51.754656Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1049 reason: error at write: Disconnected sessionId: src_id|155290a7-3df5a591-842c7902-e84849a5_0
2025-10-09T21:47:51.754724Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:51.754847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1049 sessionId: src_id|155290a7-3df5a591-842c7902-e84849a5_0 is DEAD
2025-10-09T21:47:51.755011Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|155290a7-3df5a591-842c7902-e84849a5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:51.755023Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|155290a7-3df5a591-842c7902-e84849a5_0] Write session will restart in 0.009319s
2025-10-09T21:47:51.755051Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|155290a7-3df5a591-842c7902-e84849a5_0] Write session: Do CDS request
2025-10-09T21:47:51.755058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|155290a7-3df5a591-842c7902-e84849a5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:51.765604Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|155290a7-3df5a591-842c7902-e84849a5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:51.765665Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:51.765685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1050
2025-10-09T21:47:51.765894Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1050 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:51.765932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1050 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:51.765943Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1050 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:51.765947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:51.766407Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1050 sessionId:  describe result for acl check
2025-10-09T21:47:51.766455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:51.766463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:51.766467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:51.766472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342033634832300:9431] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:51.767024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342033634832300:9431] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:51.768250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:51.769270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342033634832300:9431] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:51.770942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:51.771156Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:51.771167Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:51.771178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:51.771234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:51.771292Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342033273011419:8455]
2025-10-09T21:47:51.771855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342033634832300:9431] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:51.771868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:51.771879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342033634832300:9431] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:51.771883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342033634832300:9431] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:51.771891Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1050 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:51.772179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:51.772224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:51.772531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:51.772578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.772585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.772586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.772587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.772588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.772589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.772590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.772591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.772592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.772598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.772608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.772612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.772616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.772620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.772625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.772629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.772633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.772637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.772645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.772652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.772658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.772663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.772668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.772673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.772677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.772683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.772688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.772697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.772702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.772707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.772716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.772721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.772726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.772737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.772741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.772746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.772780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:51.779448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:51.780704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:51.780715Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:51.780717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:51.780720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:51.780726Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1049 [2:7559342033273011419:8455]
2025-10-09T21:47:51.780737Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:51.780754Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:51.780757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:51.780761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:51.780763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:51.780780Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:51.780786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:51.780794Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:51.780848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.780854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.780856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.780858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:51.780932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:51.780944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.780958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.780967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.780987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:51.781014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342033634832334:9431], now have 1 active actors on pipe
2025-10-09T21:47:51.781023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342033634832215:2432], now have 1 active actors on pipe
2025-10-09T21:47:51.781031Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342033273011414:8455]
2025-10-09T21:47:51.781038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342033273011414:8455]
2025-10-09T21:47:51.781040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342033273011414:8455]
2025-10-09T21:47:51.781058Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:51.781078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:51.781087Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:51.781098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:51.781101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:51.781103Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:51.781107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:51.781103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1049
2025-10-09T21:47:51.781134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342033273011414:8455]
2025-10-09T21:47:51.781143Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:51.781109Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1049
2025-10-09T21:47:51.781157Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:51.781184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:51.781196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:51.781225Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f2735640-3708b038-a387d997-61af64c4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:51.781238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:51.781247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:51.781249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:51.781259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:51.781273Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:51.781291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:51.781530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:51.781545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:51.781572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:51.781716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1050 partition: 0 MaxSeqNo: 25 sessionId: src_id|f2735640-3708b038-a387d997-61af64c4_0
2025-10-09T21:47:51.782098Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|155290a7-3df5a591-842c7902-e84849a5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f2735640-3708b038-a387d997-61af64c4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:51.784939Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2735640-3708b038-a387d997-61af64c4_0] Write session: try to update token
2025-10-09T21:47:51.784957Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2735640-3708b038-a387d997-61af64c4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:51.792373Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:51.792387Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:51.792395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342033273011414:8455]
2025-10-09T21:47:51.792432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:51.792444Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:51.792584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:51.792724Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:51.792849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:51.792983Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:51.793179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:51.793228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:51.798524Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1050 sessionId: src_id|f2735640-3708b038-a387d997-61af64c4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:51.800750Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:51.800951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:51.800969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:51.801000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:51.801094Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:51.805318Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:51.805335Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:51.805337Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:51.805340Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:51.805350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342033273011414:8455]
2025-10-09T21:47:51.805354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342033273011414:8455]
2025-10-09T21:47:51.805360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342033273011414:8455]
2025-10-09T21:47:51.805403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:51.805413Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:51.805415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.805418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:51.805419Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:51.805422Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:51.805550Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:51.854534Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342033634832361:9144]
2025-10-09T21:47:51.854816Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 1049, pipe: [1:7559342033634832361:9144]
2025-10-09T21:47:51.963700Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:51.971688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:51.971719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:51.971885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:51.971949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:51.972017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:51.972080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:51.972301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:51.972313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:51.972392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:51.972457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:51.972520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:51.972584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:51.972632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:51.972641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:51.972699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:51.972755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:51.972813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:51.972875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:51.973183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:51.973192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:51.973256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:51.973326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:51.973389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:51.973452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:51.973497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:51.973504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:51.973571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:51.973633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:51.973697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:51.973756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:51.973811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:51.973823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:51.973889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:51.973951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:51.974015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:51.974097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:51.974139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:51.974146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:51.974214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:51.974269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:51.974334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:51.974396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:51.974438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:51.974449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:51.974506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:51.974567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:51.974633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:51.974694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:51.974901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:51.974909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:51.974970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:51.975026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:51.975085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:51.975147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:51.975194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:51.975200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:51.975263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:51.975324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:51.975387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:51.975455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:51.975502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:51.975510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:51.975584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342033634832361:9144], now have 1 active actors on pipe
2025-10-09T21:47:51.975808Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342033634832361:9144]
2025-10-09T21:47:51.975851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:51.975854Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:51.975885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1050 reason: error at write: Disconnected sessionId: src_id|f2735640-3708b038-a387d997-61af64c4_0
2025-10-09T21:47:51.976099Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:51.976089Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1050 sessionId: src_id|f2735640-3708b038-a387d997-61af64c4_0 is DEAD
2025-10-09T21:47:51.976228Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2735640-3708b038-a387d997-61af64c4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:51.976238Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2735640-3708b038-a387d997-61af64c4_0] Write session will restart in 0.006580s
2025-10-09T21:47:51.976262Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2735640-3708b038-a387d997-61af64c4_0] Write session: Do CDS request
2025-10-09T21:47:51.976268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2735640-3708b038-a387d997-61af64c4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:51.976485Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:51.983550Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2735640-3708b038-a387d997-61af64c4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:51.983612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:51.983643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1051
2025-10-09T21:47:51.983761Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1051 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:51.983792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1051 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:51.983802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1051 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:51.983805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:51.984298Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1051 sessionId:  describe result for acl check
2025-10-09T21:47:51.984339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:51.984345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:51.984347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:51.984353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342033634832390:9443] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:51.984887Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342033634832390:9443] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:51.987206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342033634832390:9443] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:51.988928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:51.990961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:51.991180Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:51.991189Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:51.991200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:51.991265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:51.991305Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342033273011553:8459]
2025-10-09T21:47:51.991686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342033634832390:9443] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:51.991703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342033634832390:9443] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:51.991707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342033634832390:9443] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:51.991717Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1051 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:51.991901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:51.992160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:51.992190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:51.992423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:51.992460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.992464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.992465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.992466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.992466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.992467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.992468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.992469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.992469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.992472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.992481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.992485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.992489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.992492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.992497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.992501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.992505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.992508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.992515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.992522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:51.992528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.992533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:51.992538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.992543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:51.992547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.992551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:51.992553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.992556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:51.992558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.992561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:51.992564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.992567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:51.992570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.992573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:51.992576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.992581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:51.992609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:51.999064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:52.000157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:52.000171Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:52.000173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:52.000177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:52.000184Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1050 [2:7559342033273011553:8459]
2025-10-09T21:47:52.000201Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:52.000223Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:52.000231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:52.000234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:52.000237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:52.000257Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:52.000267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:52.000275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:52.000341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.000347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.000349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.000353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:52.000430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:52.000438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.000453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.000469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.000491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:52.000518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342033634832495:9443], now have 1 active actors on pipe
2025-10-09T21:47:52.000525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342033634832375:2432], now have 1 active actors on pipe
2025-10-09T21:47:52.000533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342033273011548:8459]
2025-10-09T21:47:52.000538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342033273011548:8459]
2025-10-09T21:47:52.000541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342033273011548:8459]
2025-10-09T21:47:52.000560Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:52.000579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:52.000590Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:52.000598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:52.000602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:52.000602Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:52.000604Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:52.000642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342033273011548:8459]
2025-10-09T21:47:52.000608Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1050
2025-10-09T21:47:52.000651Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:52.000617Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1050
2025-10-09T21:47:52.000654Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:52.000669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.000674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.000731Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e6a9f2e8-8e7712db-52bc1044-f20d1706_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:52.000742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:52.000746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:52.000749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:52.000761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:52.000776Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:52.000807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:52.000976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.000991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.001016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:52.001106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1051 partition: 0 MaxSeqNo: 25 sessionId: src_id|e6a9f2e8-8e7712db-52bc1044-f20d1706_0
2025-10-09T21:47:52.001480Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2735640-3708b038-a387d997-61af64c4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e6a9f2e8-8e7712db-52bc1044-f20d1706_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:52.004009Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e6a9f2e8-8e7712db-52bc1044-f20d1706_0] Write session: try to update token
2025-10-09T21:47:52.004028Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e6a9f2e8-8e7712db-52bc1044-f20d1706_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:52.015566Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:52.015582Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:52.015584Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:52.015585Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:52.015593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342033273011548:8459]
2025-10-09T21:47:52.015597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342033273011548:8459]
2025-10-09T21:47:52.015600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342033273011548:8459]
2025-10-09T21:47:52.015637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:52.015643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.015643Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:52.015646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:52.015648Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:52.015652Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:52.016430Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:52.020014Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1051 sessionId: src_id|e6a9f2e8-8e7712db-52bc1044-f20d1706_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:52.022577Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:52.075773Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342037929799804:9144]
2025-10-09T21:47:52.076041Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 1050, pipe: [1:7559342037929799804:9144]
2025-10-09T21:47:52.180657Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:52.181052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.181069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.181115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342037929799804:9144], now have 1 active actors on pipe
2025-10-09T21:47:52.181332Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:52.181381Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:52.181385Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:52.181367Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:52.181386Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342037929799804:9144]
2025-10-09T21:47:52.181446Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1051 reason: error at write: Disconnected sessionId: src_id|e6a9f2e8-8e7712db-52bc1044-f20d1706_0
2025-10-09T21:47:52.181647Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1051 sessionId: src_id|e6a9f2e8-8e7712db-52bc1044-f20d1706_0 is DEAD
2025-10-09T21:47:52.181778Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6a9f2e8-8e7712db-52bc1044-f20d1706_0] Counters: { Errors: 0 CurrentSessionLifetimeMs: 180 BytesWritten: 56320040 MessagesWritten: 25 BytesWrittenCompressed: 0 BytesInflightUncompressed: 22528020 BytesInflightCompressed: 0 BytesInflightTotal: 18022416 MessagesInflight: 10 }
2025-10-09T21:47:52.181827Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6a9f2e8-8e7712db-52bc1044-f20d1706_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:52.181833Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6a9f2e8-8e7712db-52bc1044-f20d1706_0] Write session will restart in 0.008157s
2025-10-09T21:47:52.181857Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6a9f2e8-8e7712db-52bc1044-f20d1706_0] Write session: Do CDS request
2025-10-09T21:47:52.181863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6a9f2e8-8e7712db-52bc1044-f20d1706_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:52.191587Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e6a9f2e8-8e7712db-52bc1044-f20d1706_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:52.191682Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:52.191707Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1052
2025-10-09T21:47:52.191839Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1052 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:52.191880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1052 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:52.191892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1052 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:52.191896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:52.192577Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1052 sessionId:  describe result for acl check
2025-10-09T21:47:52.192616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:52.192621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:52.192622Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:52.192628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342037929799897:9447] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:52.192929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:52.193178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342037929799897:9447] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:52.195091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:52.195329Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:52.195340Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:52.195353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:52.195419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:52.195469Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342037567978975:8461]
2025-10-09T21:47:52.195550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342037929799897:9447] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:52.195999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:52.196303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:52.196352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:52.196676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:52.196731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.196737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.196738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.196738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.196739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.196740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.196740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.196741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.196741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.196746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.196753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.196755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.196757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.196760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.196762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.196764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.196766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.196768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.196773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.196779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.196784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.196788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.196791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.196794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.196796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.196799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.196802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.196805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.196808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.196811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.196814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.196818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.196820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.196823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.196827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.196830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.196847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:52.202198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342037929799897:9447] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:52.202220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342037929799897:9447] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:52.202224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342037929799897:9447] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:52.202235Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1052 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:52.203570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:52.204705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:52.204721Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:52.204724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:52.204729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:52.204737Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1051 [2:7559342037567978975:8461]
2025-10-09T21:47:52.204753Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:52.204777Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:52.204785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:52.204788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:52.204791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:52.204813Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:52.204819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:52.204829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:52.204896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.204904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.204907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.204910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:52.204975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:52.204982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.204991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.204997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.205007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:52.205028Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342037567978969:8461]
2025-10-09T21:47:52.205036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342037567978969:8461]
2025-10-09T21:47:52.205038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342037567978969:8461]
2025-10-09T21:47:52.205048Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:52.205070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:52.205070Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:52.205073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:52.205075Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:52.205086Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:52.205088Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:52.205098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342037929799809:2432], now have 1 active actors on pipe
2025-10-09T21:47:52.205101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342037929799942:9447], now have 1 active actors on pipe
2025-10-09T21:47:52.205134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342037567978969:8461]
2025-10-09T21:47:52.205143Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:52.205150Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:52.205173Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1051
2025-10-09T21:47:52.205192Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1051
2025-10-09T21:47:52.205343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.205356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.205388Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|be6ad76d-a1846165-a6a1591b-be7c101c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:52.205401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:52.205409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:52.205412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:52.205424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:52.205437Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:52.205463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:52.205611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.205617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.205630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:52.205707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1052 partition: 0 MaxSeqNo: 25 sessionId: src_id|be6ad76d-a1846165-a6a1591b-be7c101c_0
2025-10-09T21:47:52.206102Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6a9f2e8-8e7712db-52bc1044-f20d1706_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|be6ad76d-a1846165-a6a1591b-be7c101c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:52.208747Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be6ad76d-a1846165-a6a1591b-be7c101c_0] Write session: try to update token
2025-10-09T21:47:52.208766Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be6ad76d-a1846165-a6a1591b-be7c101c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:52.217302Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:52.217317Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:52.217327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342037567978969:8461]
2025-10-09T21:47:52.217375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:52.217383Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:52.217553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:52.217699Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:52.217855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:52.217995Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:52.219166Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:52.219220Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:52.223167Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1052 sessionId: src_id|be6ad76d-a1846165-a6a1591b-be7c101c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:52.225645Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:52.225870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.225891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.225942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:52.226044Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:52.230845Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:52.230863Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:52.230865Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:52.230867Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:52.230880Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342037567978969:8461]
2025-10-09T21:47:52.230884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342037567978969:8461]
2025-10-09T21:47:52.230891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342037567978969:8461]
2025-10-09T21:47:52.230938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:52.230952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.230952Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:52.230955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:52.230957Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:52.230961Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:52.231089Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:52.281575Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342037929799949:9144]
2025-10-09T21:47:52.281855Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 1051, pipe: [1:7559342037929799949:9144]
2025-10-09T21:47:52.382556Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:52.389529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.389559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.389705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.389761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.389827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.389885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.389926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.389935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.390003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.390089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.390148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.390207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.390250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.390256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.390311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.390374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.390429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.390484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.390525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.390532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.390592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.390662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.390722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.390779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.390817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.390822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.390887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.390946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.391000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.391056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.391100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.391108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.391169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.391228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.391281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.391335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.391385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.391388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.391451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.391509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.391576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.391639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.391683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.391691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.391747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.391801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.391860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.391922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.391963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.391971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.392031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.392088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.392148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.392210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.392255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.392261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.392318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.392375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.392438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.392503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.392546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.392553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.392601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342037929799949:9144], now have 1 active actors on pipe
2025-10-09T21:47:52.392831Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:52.392839Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:52.392852Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:52.392886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1052 reason: error at write: Disconnected sessionId: src_id|be6ad76d-a1846165-a6a1591b-be7c101c_0
2025-10-09T21:47:52.392906Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342037929799949:9144]
2025-10-09T21:47:52.393092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1052 sessionId: src_id|be6ad76d-a1846165-a6a1591b-be7c101c_0 is DEAD
2025-10-09T21:47:52.393248Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be6ad76d-a1846165-a6a1591b-be7c101c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:52.393257Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be6ad76d-a1846165-a6a1591b-be7c101c_0] Write session will restart in 0.007097s
2025-10-09T21:47:52.393283Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be6ad76d-a1846165-a6a1591b-be7c101c_0] Write session: Do CDS request
2025-10-09T21:47:52.393289Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be6ad76d-a1846165-a6a1591b-be7c101c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:52.393569Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:52.401548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be6ad76d-a1846165-a6a1591b-be7c101c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:52.401654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:52.401673Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1053
2025-10-09T21:47:52.401824Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1053 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:52.401860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1053 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:52.401872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1053 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:52.401876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:52.402334Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1053 sessionId:  describe result for acl check
2025-10-09T21:47:52.402377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:52.402383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:52.402385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:52.402391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342037929800039:9452] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:52.402936Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342037929800039:9452] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:52.403569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:52.405301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342037929800039:9452] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:52.405586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:52.405827Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:52.405837Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:52.405850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:52.405913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:52.405954Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342037567979102:8463]
2025-10-09T21:47:52.406542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:52.406792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:52.406824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:52.407027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:52.407058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.407062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.407062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.407063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.407064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.407064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.407065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.407066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.407066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.407069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.407075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.407077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.407079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.407081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.407083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.407085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.407087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.407089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.407094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.407099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.407104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.407107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.407109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.407112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.407115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.407119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.407123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.407128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.407132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.407137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.407141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.407146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.407151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.407156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.407159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.407162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.407176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:52.411895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342037929800039:9452] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:52.411914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342037929800039:9452] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:52.411919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342037929800039:9452] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:52.411928Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1053 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:52.413083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:52.414065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:52.414076Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:52.414077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:52.414081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:52.414086Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1052 [2:7559342037567979102:8463]
2025-10-09T21:47:52.414096Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:52.414111Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:52.414113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:52.414115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:52.414116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:52.414129Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:52.414133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:52.414139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:52.414181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.414188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.414189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.414191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:52.414254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:52.414260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.414267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.414273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.414282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:52.414298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342037567979096:8463]
2025-10-09T21:47:52.414303Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342037567979096:8463]
2025-10-09T21:47:52.414305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342037567979096:8463]
2025-10-09T21:47:52.414312Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:52.414322Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:52.414323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342037929799952:2432], now have 1 active actors on pipe
2025-10-09T21:47:52.414329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342037929800081:9452], now have 1 active actors on pipe
2025-10-09T21:47:52.414331Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:52.414332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:52.414334Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:52.414341Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:52.414343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:52.414371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342037567979096:8463]
2025-10-09T21:47:52.414381Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:52.414388Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:52.414399Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1052
2025-10-09T21:47:52.414423Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1052
2025-10-09T21:47:52.414527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.414541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.414570Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|43ea9127-c009d56f-1b45b8af-39bfe210_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:52.414583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:52.414589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:52.414592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:52.414602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:52.414618Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:52.414633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:52.414760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.414768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.414782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:52.414854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1053 partition: 0 MaxSeqNo: 25 sessionId: src_id|43ea9127-c009d56f-1b45b8af-39bfe210_0
2025-10-09T21:47:52.415194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be6ad76d-a1846165-a6a1591b-be7c101c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|43ea9127-c009d56f-1b45b8af-39bfe210_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:52.417629Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|43ea9127-c009d56f-1b45b8af-39bfe210_0] Write session: try to update token
2025-10-09T21:47:52.417647Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|43ea9127-c009d56f-1b45b8af-39bfe210_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:52.428185Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:52.428206Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:52.428209Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:52.428211Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:52.428222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342037567979096:8463]
2025-10-09T21:47:52.428226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342037567979096:8463]
2025-10-09T21:47:52.428230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342037567979096:8463]
2025-10-09T21:47:52.428275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:52.428283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.428286Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:52.428288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:52.428291Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:52.428295Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:52.428442Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:52.430885Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1053 sessionId: src_id|43ea9127-c009d56f-1b45b8af-39bfe210_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:52.433548Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:52.493090Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342037929800088:9144]
2025-10-09T21:47:52.493338Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 1052, pipe: [1:7559342037929800088:9144]
2025-10-09T21:47:52.574771Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:52.575292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.575308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.575354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342037929800088:9144], now have 1 active actors on pipe
2025-10-09T21:47:52.575591Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:52.575658Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:52.575666Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:52.575590Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:52.575725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1053 reason: error at write: Disconnected sessionId: src_id|43ea9127-c009d56f-1b45b8af-39bfe210_0
2025-10-09T21:47:52.575913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1053 sessionId: src_id|43ea9127-c009d56f-1b45b8af-39bfe210_0 is DEAD
2025-10-09T21:47:52.575970Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342037929800088:9144]
2025-10-09T21:47:52.576066Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43ea9127-c009d56f-1b45b8af-39bfe210_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:52.576077Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43ea9127-c009d56f-1b45b8af-39bfe210_0] Write session will restart in 0.009109s
2025-10-09T21:47:52.576103Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43ea9127-c009d56f-1b45b8af-39bfe210_0] Write session: Do CDS request
2025-10-09T21:47:52.576109Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43ea9127-c009d56f-1b45b8af-39bfe210_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:52.586598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|43ea9127-c009d56f-1b45b8af-39bfe210_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:52.586691Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:52.586710Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1054
2025-10-09T21:47:52.586867Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1054 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:52.586914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1054 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:52.586927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1054 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:52.586932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:52.587343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:52.587393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1054 sessionId:  describe result for acl check
2025-10-09T21:47:52.587436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:52.587438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:52.587440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:52.587459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342037929800182:9456] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:52.588057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342037929800182:9456] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:52.589429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:52.589623Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:52.589634Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:52.589645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:52.589694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:52.589734Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342037567979230:8465]
2025-10-09T21:47:52.590311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342037929800182:9456] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:52.590333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:52.590704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:52.590770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:52.591081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:52.591127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.591136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.591137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.591138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.591139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.591140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.591141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.591142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.591143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.591148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.591156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.591159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.591162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.591165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.591168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.591171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.591174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.591177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.591183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.591190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.591196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.591201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.591205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.591209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.591212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.591217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.591220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.591225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.591229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.591233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.591237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.591243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.591246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.591251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.591255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.591259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.591286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:52.594220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342037929800182:9456] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:52.594236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342037929800182:9456] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:52.594240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342037929800182:9456] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:52.594248Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1054 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:52.598618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:52.599680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:52.599698Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:52.599701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:52.599706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:52.599713Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1053 [2:7559342037567979230:8465]
2025-10-09T21:47:52.599727Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:52.599745Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:52.599748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:52.599752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:52.599754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:52.599773Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:52.599784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:52.599793Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:52.599854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.599860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.599863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.599866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:52.599940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:52.599947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.599960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.599979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.600001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:52.600028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342037929800091:2432], now have 1 active actors on pipe
2025-10-09T21:47:52.600036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342037929800221:9456], now have 1 active actors on pipe
2025-10-09T21:47:52.600044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342037567979224:8465]
2025-10-09T21:47:52.600046Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342037567979224:8465]
2025-10-09T21:47:52.600048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342037567979224:8465]
2025-10-09T21:47:52.600060Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:52.600076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:52.600076Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1053
2025-10-09T21:47:52.600095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:52.600090Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1053
2025-10-09T21:47:52.600098Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:52.600103Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:52.600103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:52.600105Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:52.600139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342037567979224:8465]
2025-10-09T21:47:52.600151Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:52.600164Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:52.600171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.600176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.600225Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3b808fc1-85f4cfa5-6d47101a-bee20cbe_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:52.600240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:52.600249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:52.600251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:52.600261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:52.600276Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:52.600297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:52.600503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.600518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.600541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:52.600658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1054 partition: 0 MaxSeqNo: 25 sessionId: src_id|3b808fc1-85f4cfa5-6d47101a-bee20cbe_0
2025-10-09T21:47:52.601029Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43ea9127-c009d56f-1b45b8af-39bfe210_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3b808fc1-85f4cfa5-6d47101a-bee20cbe_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:52.603857Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3b808fc1-85f4cfa5-6d47101a-bee20cbe_0] Write session: try to update token
2025-10-09T21:47:52.603879Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3b808fc1-85f4cfa5-6d47101a-bee20cbe_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:52.614839Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:52.614869Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:52.614872Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:52.614875Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:52.614887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342037567979224:8465]
2025-10-09T21:47:52.614892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342037567979224:8465]
2025-10-09T21:47:52.614896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342037567979224:8465]
2025-10-09T21:47:52.614945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:52.614954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.614957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:52.615104Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:52.617045Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:52.617061Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:52.617066Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:52.619657Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1054 sessionId: src_id|3b808fc1-85f4cfa5-6d47101a-bee20cbe_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:52.622104Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:52.676369Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342037929800229:9144]
2025-10-09T21:47:52.676670Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 1053, pipe: [1:7559342037929800229:9144]
2025-10-09T21:47:52.771710Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:52.771968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.771983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.772028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342037929800229:9144], now have 1 active actors on pipe
2025-10-09T21:47:52.772227Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:52.772210Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:52.772235Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342037929800229:9144]
2025-10-09T21:47:52.772259Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:52.772261Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:52.772319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1054 reason: error at write: Disconnected sessionId: src_id|3b808fc1-85f4cfa5-6d47101a-bee20cbe_0
2025-10-09T21:47:52.772505Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1054 sessionId: src_id|3b808fc1-85f4cfa5-6d47101a-bee20cbe_0 is DEAD
2025-10-09T21:47:52.772673Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b808fc1-85f4cfa5-6d47101a-bee20cbe_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:52.772685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b808fc1-85f4cfa5-6d47101a-bee20cbe_0] Write session will restart in 0.005899s
2025-10-09T21:47:52.772724Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b808fc1-85f4cfa5-6d47101a-bee20cbe_0] Write session: Do CDS request
2025-10-09T21:47:52.772731Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b808fc1-85f4cfa5-6d47101a-bee20cbe_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:52.779536Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3b808fc1-85f4cfa5-6d47101a-bee20cbe_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:52.779606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:52.779623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1055
2025-10-09T21:47:52.779766Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1055 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:52.779825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1055 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:52.779836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1055 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:52.779840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:52.780224Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1055 sessionId:  describe result for acl check
2025-10-09T21:47:52.780267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:52.780273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:52.780276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:52.780283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342037929800289:9460] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:52.780752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342037929800289:9460] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:52.782760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342037929800289:9460] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:52.782763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:52.784658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:52.784832Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:52.784842Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:52.784853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:52.784903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:52.784942Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342037567979393:8479]
2025-10-09T21:47:52.784985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342037929800289:9460] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:52.784996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342037929800289:9460] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:52.784999Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342037929800289:9460] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:52.785005Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1055 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:52.785582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:52.785869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:52.785910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:52.786273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:52.786321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.786326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.786327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.786329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.786330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.786331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.786332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.786334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.786335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.786340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.786352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.786356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.786361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.786365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.786370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.786375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.786379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.786383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.786391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.786400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.786406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.786413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.786417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.786423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.786428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.786434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.786439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.786444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.786449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.786455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.786461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.786472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.786477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.786483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.786488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.786494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.786526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:52.793290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:52.794322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:52.794335Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:52.794338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:52.794342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:52.794349Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1054 [2:7559342037567979393:8479]
2025-10-09T21:47:52.794373Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:52.794392Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:52.794405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:52.794408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:52.794411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:52.794429Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:52.794435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:52.794443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:52.794504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.794511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.794514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.794517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:52.794587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:52.794597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.794612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.794626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.794645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:52.794666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342037929800358:9460], now have 1 active actors on pipe
2025-10-09T21:47:52.794780Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1054
2025-10-09T21:47:52.794784Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1054
2025-10-09T21:47:52.794670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342037929800234:2432], now have 1 active actors on pipe
2025-10-09T21:47:52.794678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342037567979387:8479]
2025-10-09T21:47:52.794680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342037567979387:8479]
2025-10-09T21:47:52.794683Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342037567979387:8479]
2025-10-09T21:47:52.794699Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:52.794714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:52.794721Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:52.794726Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:52.794728Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:52.794729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:52.794732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:52.794762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342037567979387:8479]
2025-10-09T21:47:52.794768Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:52.794797Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:52.794848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.794853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.794886Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1a648d41-7f6bb5b9-fb163aaf-1d0ceebe_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:52.794893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:52.794897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:52.794899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:52.794960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:52.794971Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:52.794989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:52.795170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.795178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.795192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:52.795310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1055 partition: 0 MaxSeqNo: 25 sessionId: src_id|1a648d41-7f6bb5b9-fb163aaf-1d0ceebe_0
2025-10-09T21:47:52.795686Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b808fc1-85f4cfa5-6d47101a-bee20cbe_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1a648d41-7f6bb5b9-fb163aaf-1d0ceebe_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:52.798250Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a648d41-7f6bb5b9-fb163aaf-1d0ceebe_0] Write session: try to update token
2025-10-09T21:47:52.798271Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a648d41-7f6bb5b9-fb163aaf-1d0ceebe_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:52.806417Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:52.806434Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:52.806445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342037567979387:8479]
2025-10-09T21:47:52.806497Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:52.806504Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:52.806690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:52.806846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:52.806998Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:52.807178Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:52.807377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:52.807426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:52.811912Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1055 sessionId: src_id|1a648d41-7f6bb5b9-fb163aaf-1d0ceebe_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:52.814424Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:52.814649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.814673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.814717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:52.814789Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:52.819664Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:52.819690Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:52.819693Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:52.819695Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:52.819703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342037567979387:8479]
2025-10-09T21:47:52.819706Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342037567979387:8479]
2025-10-09T21:47:52.819709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342037567979387:8479]
2025-10-09T21:47:52.819744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:52.819749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.819751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:52.819773Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:52.819798Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:52.819802Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:52.819880Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:52.872446Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342037929800372:9144]
2025-10-09T21:47:52.872729Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 1054, pipe: [1:7559342037929800372:9144]
2025-10-09T21:47:52.972032Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:52.978751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:52.978782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:52.978957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.979018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.979081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.979139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.979180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.979188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.979256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.979311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.979372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.979429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.979472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.979482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.979544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.979597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.979657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.979715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.979754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.979761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.979821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.979883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.979941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.981978Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:52.981995Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:52.981998Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:52.982043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1055 reason: error at write: Disconnected sessionId: src_id|1a648d41-7f6bb5b9-fb163aaf-1d0ceebe_0
2025-10-09T21:47:52.982052Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342037929800372:9144]
2025-10-09T21:47:52.982275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1055 sessionId: src_id|1a648d41-7f6bb5b9-fb163aaf-1d0ceebe_0 is DEAD
2025-10-09T21:47:52.982408Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a648d41-7f6bb5b9-fb163aaf-1d0ceebe_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:52.982420Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a648d41-7f6bb5b9-fb163aaf-1d0ceebe_0] Write session will restart in 0.005405s
2025-10-09T21:47:52.982447Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a648d41-7f6bb5b9-fb163aaf-1d0ceebe_0] Write session: Do CDS request
2025-10-09T21:47:52.982453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a648d41-7f6bb5b9-fb163aaf-1d0ceebe_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:52.980001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.980048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.980051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.980113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.980165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.980224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.980279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.980317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.980319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.980381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.980432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.980488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.980548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.980584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.980587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.980649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.980708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.980766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.980820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.980857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.980860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.980920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.980969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.981023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.981074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.981109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.981111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.981170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.981218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.981271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.981328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.981367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.981369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.981431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:52.981484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:52.981538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:52.981597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:52.981636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:52.981638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:52.981686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342037929800372:9144], now have 1 active actors on pipe
2025-10-09T21:47:52.982710Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:52.989488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a648d41-7f6bb5b9-fb163aaf-1d0ceebe_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:52.989615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:52.989632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1056
2025-10-09T21:47:52.989767Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1056 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:52.989807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1056 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:52.989818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1056 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:52.989822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:52.990253Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1056 sessionId:  describe result for acl check
2025-10-09T21:47:52.990292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:52.990298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:52.990303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:52.990309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342037929800473:9472] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:52.990867Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342037929800473:9472] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:52.991966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:52.993374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342037929800473:9472] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:52.993844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:52.994054Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:52.994064Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:52.994077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:52.994142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:52.994180Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342037567979532:8483]
2025-10-09T21:47:52.994760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:52.995021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:52.995058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:52.995312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:52.995362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.995368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.995369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.995369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.995370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.995371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.995371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.995372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.995372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.995376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.995382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.995385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.995387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.995389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.995391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.995393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.995395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.995396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.995401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.995406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:52.995409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.995413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:52.995415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.995419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:52.995424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.995429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:52.995433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.995436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:52.995441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.995446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:52.995450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.995455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:52.995459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.995464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:52.995468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.995471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:52.995495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:52.995978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342037929800473:9472] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:52.995992Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342037929800473:9472] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:52.995996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342037929800473:9472] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:52.996004Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1056 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:53.001556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:53.002841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:53.002856Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:53.002859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:53.002863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:53.002870Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1055 [2:7559342037567979532:8483]
2025-10-09T21:47:53.002884Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:53.002904Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:53.002912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:53.002915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:53.002917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:53.002934Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:53.002946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:53.002958Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:53.003021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.003030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.003033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.003036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:53.003108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:53.003115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.003128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.003145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.003164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:53.003187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342037929800529:9472], now have 1 active actors on pipe
2025-10-09T21:47:53.003195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342037929800399:2432], now have 1 active actors on pipe
2025-10-09T21:47:53.003203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342037567979526:8483]
2025-10-09T21:47:53.003206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342037567979526:8483]
2025-10-09T21:47:53.003207Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342037567979526:8483]
2025-10-09T21:47:53.003218Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:53.003233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:53.003241Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:53.003244Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:53.003246Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:53.003247Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:53.003251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:53.003255Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1055
2025-10-09T21:47:53.003255Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1055
2025-10-09T21:47:53.003284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342037567979526:8483]
2025-10-09T21:47:53.003296Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:53.003303Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:53.003313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:53.003318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:53.003373Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|871155ee-430ad32e-a59c161-a2074f1c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:53.003384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:53.003393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:53.003395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:53.003406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:53.003420Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:53.003441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:53.003608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:53.003617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:53.003631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:53.003710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1056 partition: 0 MaxSeqNo: 25 sessionId: src_id|871155ee-430ad32e-a59c161-a2074f1c_0
2025-10-09T21:47:53.004035Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a648d41-7f6bb5b9-fb163aaf-1d0ceebe_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|871155ee-430ad32e-a59c161-a2074f1c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:53.006505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|871155ee-430ad32e-a59c161-a2074f1c_0] Write session: try to update token
2025-10-09T21:47:53.006524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|871155ee-430ad32e-a59c161-a2074f1c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:53.017841Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:53.017862Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:53.017865Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:53.017866Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:53.017876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342037567979526:8483]
2025-10-09T21:47:53.017884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342037567979526:8483]
2025-10-09T21:47:53.017888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342037567979526:8483]
2025-10-09T21:47:53.017938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:53.017943Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:53.017949Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:53.017952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.017953Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:53.017955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:53.018116Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:53.020492Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1056 sessionId: src_id|871155ee-430ad32e-a59c161-a2074f1c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:53.023020Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:53.082218Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] Counters: { Errors: 10 CurrentSessionLifetimeMs: 301533 BytesRead: 24780812 MessagesRead: 11 BytesReadCompressed: 24780812 BytesInflightUncompressed: 2252802 BytesInflightCompressed: 0 BytesInflightTotal: 2252802 MessagesInflight: 1 }
2025-10-09T21:47:53.082356Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342042224767832:9144]
2025-10-09T21:47:53.082645Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 1055, pipe: [1:7559342042224767832:9144]
2025-10-09T21:47:53.174135Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:53.174487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:53.174503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:53.174553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342042224767832:9144], now have 1 active actors on pipe
2025-10-09T21:47:53.174775Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:53.174786Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342042224767832:9144]
2025-10-09T21:47:53.174821Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:53.174825Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:53.174831Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:53.174889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1056 reason: error at write: Disconnected sessionId: src_id|871155ee-430ad32e-a59c161-a2074f1c_0
2025-10-09T21:47:53.175104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1056 sessionId: src_id|871155ee-430ad32e-a59c161-a2074f1c_0 is DEAD
2025-10-09T21:47:53.175286Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|871155ee-430ad32e-a59c161-a2074f1c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:53.175298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|871155ee-430ad32e-a59c161-a2074f1c_0] Write session will restart in 0.006641s
2025-10-09T21:47:53.175340Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|871155ee-430ad32e-a59c161-a2074f1c_0] Write session: Do CDS request
2025-10-09T21:47:53.175347Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|871155ee-430ad32e-a59c161-a2074f1c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:53.183634Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|871155ee-430ad32e-a59c161-a2074f1c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:53.183775Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:53.183805Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1057
2025-10-09T21:47:53.183960Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1057 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:53.183994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1057 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:53.184005Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1057 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:53.184009Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:53.184487Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1057 sessionId:  describe result for acl check
2025-10-09T21:47:53.184531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:53.184533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:53.184535Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:53.184541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342042224767914:9476] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:53.185094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342042224767914:9476] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:53.186361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:53.187269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342042224767914:9476] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:53.188322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:53.188535Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:53.188542Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:53.188553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:53.188608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:53.188651Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342041862946959:8485]
2025-10-09T21:47:53.189079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:53.189387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:53.189452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:53.189810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:53.189881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.189886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.189888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.189889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.189890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.189891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.189892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.189894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.189895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.189901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.189912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.189916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.189920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.189923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.189928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.189932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.189935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.189939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.189947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.189955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.189961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.189966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.189971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.189983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.189987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.189998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.190003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.190012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.190017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.190027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.190001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342042224767914:9476] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:53.190015Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342042224767914:9476] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:53.190044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.190019Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342042224767914:9476] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:53.190053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.190041Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1057 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:53.190057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.190063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.190068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.190074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.190108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:53.196690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:53.197757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:53.197774Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:53.197776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:53.197781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:53.197789Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1056 [2:7559342041862946959:8485]
2025-10-09T21:47:53.197813Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:53.197835Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:53.197844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:53.197846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:53.197848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:53.197863Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:53.197867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:53.197873Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:53.197921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.197927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.197929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.197930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:53.197989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:53.197996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.198004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.198013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.198047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:53.198070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342041862946953:8485]
2025-10-09T21:47:53.198077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342041862946953:8485]
2025-10-09T21:47:53.198079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342041862946953:8485]
2025-10-09T21:47:53.198088Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:53.198098Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:53.198104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:53.198111Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:53.198113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:53.198119Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:53.198122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:53.198134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342042224767969:9476], now have 1 active actors on pipe
2025-10-09T21:47:53.198138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342042224767838:2432], now have 1 active actors on pipe
2025-10-09T21:47:53.198175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342041862946953:8485]
2025-10-09T21:47:53.198188Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:53.198201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:53.198237Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1056
2025-10-09T21:47:53.198285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:53.198295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:53.198291Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1056
2025-10-09T21:47:53.198335Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6b0e3d5a-1bcc687b-3f92425d-f9c3e7f9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:53.198348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:53.198352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:53.198355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:53.198368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:53.198379Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:53.198402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:53.198554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:53.198563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:53.198578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:53.198687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1057 partition: 0 MaxSeqNo: 25 sessionId: src_id|6b0e3d5a-1bcc687b-3f92425d-f9c3e7f9_0
2025-10-09T21:47:53.199079Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|871155ee-430ad32e-a59c161-a2074f1c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6b0e3d5a-1bcc687b-3f92425d-f9c3e7f9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:53.201611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b0e3d5a-1bcc687b-3f92425d-f9c3e7f9_0] Write session: try to update token
2025-10-09T21:47:53.201628Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b0e3d5a-1bcc687b-3f92425d-f9c3e7f9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:53.213640Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:53.213664Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:53.213667Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:53.213669Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:53.213679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342041862946953:8485]
2025-10-09T21:47:53.213684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342041862946953:8485]
2025-10-09T21:47:53.213688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342041862946953:8485]
2025-10-09T21:47:53.213735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:53.213742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.213745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:53.213749Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:53.213756Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:53.213759Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:53.213896Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:53.216021Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1057 sessionId: src_id|6b0e3d5a-1bcc687b-3f92425d-f9c3e7f9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:53.218448Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:53.274837Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342042224767983:9144]
2025-10-09T21:47:53.275140Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 1056, pipe: [1:7559342042224767983:9144]
2025-10-09T21:47:53.369502Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:53.369909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:53.369926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:53.369973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342042224767983:9144], now have 1 active actors on pipe
2025-10-09T21:47:53.370204Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:53.370223Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:53.370242Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:53.370313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1057 reason: error at write: Disconnected sessionId: src_id|6b0e3d5a-1bcc687b-3f92425d-f9c3e7f9_0
2025-10-09T21:47:53.370318Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:53.370378Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342042224767983:9144]
2025-10-09T21:47:53.370511Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1057 sessionId: src_id|6b0e3d5a-1bcc687b-3f92425d-f9c3e7f9_0 is DEAD
2025-10-09T21:47:53.370660Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b0e3d5a-1bcc687b-3f92425d-f9c3e7f9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:53.370672Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b0e3d5a-1bcc687b-3f92425d-f9c3e7f9_0] Write session will restart in 0.005876s
2025-10-09T21:47:53.370698Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b0e3d5a-1bcc687b-3f92425d-f9c3e7f9_0] Write session: Do CDS request
2025-10-09T21:47:53.370704Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b0e3d5a-1bcc687b-3f92425d-f9c3e7f9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:53.377548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b0e3d5a-1bcc687b-3f92425d-f9c3e7f9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:53.377663Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:53.377690Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1058
2025-10-09T21:47:53.377821Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1058 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:53.377852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1058 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:53.377863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1058 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:53.377866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:53.378427Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1058 sessionId:  describe result for acl check
2025-10-09T21:47:53.378474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:53.378481Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:53.378483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:53.378490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342042224768024:9481] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:53.379055Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342042224768024:9481] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:53.381644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342042224768024:9481] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:53.385160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342042224768024:9481] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:53.385180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342042224768024:9481] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:53.385183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342042224768024:9481] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:53.385198Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1058 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:53.382827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:53.384766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:53.384968Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:53.384972Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:53.384985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:53.385047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:53.385082Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342041862947091:8487]
2025-10-09T21:47:53.385777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:53.386202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:53.386263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:53.386579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:53.386637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.386647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.386648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.386650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.386651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.386652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.386653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.386654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.386656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.386661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.386671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.386675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.386679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.386682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.386686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.386689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.386693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.386697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.386704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.386712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.386718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.386723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.386727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.386732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.386736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.386741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.386745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.386750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.386754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.386759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.386765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.386770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.386774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.386779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.386784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.386788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.386816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:53.393492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:53.394652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:53.394668Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:53.394671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:53.394675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:53.394682Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1057 [2:7559342041862947091:8487]
2025-10-09T21:47:53.394697Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:53.394717Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:53.394725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:53.394729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:53.394732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:53.394751Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:53.394762Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:53.394769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:53.394988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.394998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.395000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.395003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:53.395077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:53.395086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.395098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.395112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.395135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:53.395159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342042224768116:9481], now have 1 active actors on pipe
2025-10-09T21:47:53.395162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342042224767986:2432], now have 1 active actors on pipe
2025-10-09T21:47:53.395170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342041862947086:8487]
2025-10-09T21:47:53.395173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342041862947086:8487]
2025-10-09T21:47:53.395175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342041862947086:8487]
2025-10-09T21:47:53.395188Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:53.395201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:53.395204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:53.395205Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:53.395206Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:53.395218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:53.395221Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:53.395260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342041862947086:8487]
2025-10-09T21:47:53.395269Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:53.395291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:53.395281Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1057
2025-10-09T21:47:53.395283Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1057
2025-10-09T21:47:53.395346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:53.395359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:53.395394Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1c71c1bf-6a4b51a8-ef85b787-ad5517c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:53.395406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:53.395411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:53.395413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:53.395424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:53.395434Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:53.395454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:53.395614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:53.395623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:53.395638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:53.395730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1058 partition: 0 MaxSeqNo: 25 sessionId: src_id|1c71c1bf-6a4b51a8-ef85b787-ad5517c_0
2025-10-09T21:47:53.396110Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b0e3d5a-1bcc687b-3f92425d-f9c3e7f9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1c71c1bf-6a4b51a8-ef85b787-ad5517c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:53.398602Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c71c1bf-6a4b51a8-ef85b787-ad5517c_0] Write session: try to update token
2025-10-09T21:47:53.398618Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c71c1bf-6a4b51a8-ef85b787-ad5517c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:53.412407Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1058 sessionId: src_id|1c71c1bf-6a4b51a8-ef85b787-ad5517c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:53.412707Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:53.412730Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:53.412741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342041862947086:8487]
2025-10-09T21:47:53.412798Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:53.412803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:53.413050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:53.413242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:53.413421Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:53.413617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:53.413930Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:53.413986Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:53.415200Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:53.415414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:53.415430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:53.415477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:53.415569Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:53.421050Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:53.421064Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:53.421067Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:53.421069Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:53.421079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342041862947086:8487]
2025-10-09T21:47:53.421084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342041862947086:8487]
2025-10-09T21:47:53.421088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342041862947086:8487]
2025-10-09T21:47:53.421137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:53.421147Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:53.421152Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:53.421152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.421155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:53.421156Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:53.421335Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:53.470457Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342042224768127:9144]
2025-10-09T21:47:53.470656Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 1057, pipe: [1:7559342042224768127:9144]
2025-10-09T21:47:53.574249Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:53.582213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:53.582240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:53.582384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:53.582444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:53.582508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:53.582566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:53.582607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:53.582618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:53.582683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:53.582741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:53.582803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:53.582862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:53.582906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:53.582915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:53.582973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:53.583031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:53.583086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:53.583141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:53.583187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:53.583197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:53.583264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:53.583326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:53.583388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:53.583453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:53.583509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:53.583519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:53.583595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:53.583667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:53.583738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:53.583817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:53.583864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:53.583874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:53.583942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:53.584008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:53.584070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:53.584138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:53.584180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:53.584189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:53.584248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:53.584307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:53.584369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:53.584430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:53.584470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:53.584479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:53.584541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:53.584605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:53.584661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:53.584719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:53.584757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:53.584764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:53.584818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:53.584873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:53.584932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:53.584996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:53.585039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:53.585047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:53.585108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:53.585168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:53.585228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:53.585347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:53.585393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:53.585400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:53.585463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342042224768127:9144], now have 1 active actors on pipe
2025-10-09T21:47:53.585681Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:53.585698Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342042224768127:9144]
2025-10-09T21:47:53.585727Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:53.585730Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:53.585761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1058 reason: error at write: Disconnected sessionId: src_id|1c71c1bf-6a4b51a8-ef85b787-ad5517c_0
2025-10-09T21:47:53.585968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1058 sessionId: src_id|1c71c1bf-6a4b51a8-ef85b787-ad5517c_0 is DEAD
2025-10-09T21:47:53.586107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c71c1bf-6a4b51a8-ef85b787-ad5517c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:53.586127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c71c1bf-6a4b51a8-ef85b787-ad5517c_0] Write session will restart in 0.007760s
2025-10-09T21:47:53.586151Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c71c1bf-6a4b51a8-ef85b787-ad5517c_0] Write session: Do CDS request
2025-10-09T21:47:53.586156Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c71c1bf-6a4b51a8-ef85b787-ad5517c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:53.586513Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:53.598277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:53.598533Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c71c1bf-6a4b51a8-ef85b787-ad5517c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:53.598562Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:53.598581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1059
2025-10-09T21:47:53.598785Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1059 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:53.598821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1059 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:53.598831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1059 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:53.598835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:53.599233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1059 sessionId:  describe result for acl check
2025-10-09T21:47:53.599265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:53.599266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:53.599267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:53.599274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342042224768227:9485] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:53.599804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342042224768227:9485] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:53.600239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:53.600416Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:53.600425Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:53.600435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:53.600481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:53.600529Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342041862947224:8489]
2025-10-09T21:47:53.601059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:53.601347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:53.601390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:53.601680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:53.601728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.601734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.601736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.601736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.601738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.601738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.601740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.601741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.601742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.601746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.601756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.601760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.601764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.601767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.601771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.601774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.601777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.601780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.601785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.601803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.601809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.601813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.601817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.601821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.601825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.601830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.601834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.601838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.601841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.601846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.601850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.601855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.601859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.601864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.601869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.601873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.601900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:53.602323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342042224768227:9485] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:53.607739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342042224768227:9485] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:53.607760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342042224768227:9485] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:53.607763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342042224768227:9485] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:53.607774Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1059 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:53.608084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:53.609357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:53.609372Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:53.609374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:53.609379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:53.609385Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1058 [2:7559342041862947224:8489]
2025-10-09T21:47:53.609398Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:53.609414Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:53.609417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:53.609420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:53.609422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:53.609442Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:53.609452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:53.609468Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:53.609527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.609534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.609536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.609538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:53.609606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:53.609612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.609623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.609634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.609656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:53.609677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342042224768265:9485], now have 1 active actors on pipe
2025-10-09T21:47:53.609680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342042224768130:2432], now have 1 active actors on pipe
2025-10-09T21:47:53.609687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342041862947219:8489]
2025-10-09T21:47:53.609689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342041862947219:8489]
2025-10-09T21:47:53.609691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342041862947219:8489]
2025-10-09T21:47:53.609706Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:53.609723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:53.609726Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:53.609730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:53.609732Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:53.609741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:53.609744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:53.609743Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1058
2025-10-09T21:47:53.609753Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1058
2025-10-09T21:47:53.609777Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342041862947219:8489]
2025-10-09T21:47:53.609787Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:53.609801Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:53.609815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:53.609820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:53.609868Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8ac46119-dbb24bcf-d2f7f01d-bfb1b623_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:53.609880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:53.609884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:53.609887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:53.609898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:53.609908Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:53.609933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:53.610138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:53.610153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:53.610176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:53.610287Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1059 partition: 0 MaxSeqNo: 25 sessionId: src_id|8ac46119-dbb24bcf-d2f7f01d-bfb1b623_0
2025-10-09T21:47:53.610636Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c71c1bf-6a4b51a8-ef85b787-ad5517c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8ac46119-dbb24bcf-d2f7f01d-bfb1b623_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:53.613104Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8ac46119-dbb24bcf-d2f7f01d-bfb1b623_0] Write session: try to update token
2025-10-09T21:47:53.613125Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8ac46119-dbb24bcf-d2f7f01d-bfb1b623_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:53.623864Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:53.623883Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:53.623885Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:53.623888Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:53.623898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342041862947219:8489]
2025-10-09T21:47:53.623904Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342041862947219:8489]
2025-10-09T21:47:53.623909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342041862947219:8489]
2025-10-09T21:47:53.623955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:53.623955Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:53.623961Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:53.623965Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:53.623969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.623972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:53.624123Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:53.626488Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1059 sessionId: src_id|8ac46119-dbb24bcf-d2f7f01d-bfb1b623_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:53.628942Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:53.685980Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342042224768273:9144]
2025-10-09T21:47:53.686307Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 1058, pipe: [1:7559342042224768273:9144]
2025-10-09T21:47:53.784072Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:53.784551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:53.784566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:53.784604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342042224768273:9144], now have 1 active actors on pipe
2025-10-09T21:47:53.784836Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:53.784848Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342042224768273:9144]
2025-10-09T21:47:53.784854Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:53.784878Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:53.784928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1059 reason: error at write: Disconnected sessionId: src_id|8ac46119-dbb24bcf-d2f7f01d-bfb1b623_0
2025-10-09T21:47:53.785117Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1059 sessionId: src_id|8ac46119-dbb24bcf-d2f7f01d-bfb1b623_0 is DEAD
2025-10-09T21:47:53.785271Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ac46119-dbb24bcf-d2f7f01d-bfb1b623_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:53.785282Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ac46119-dbb24bcf-d2f7f01d-bfb1b623_0] Write session will restart in 0.008607s
2025-10-09T21:47:53.785306Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ac46119-dbb24bcf-d2f7f01d-bfb1b623_0] Write session: Do CDS request
2025-10-09T21:47:53.785315Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ac46119-dbb24bcf-d2f7f01d-bfb1b623_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:53.785544Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:53.795776Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8ac46119-dbb24bcf-d2f7f01d-bfb1b623_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:53.795843Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:53.795862Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1060
2025-10-09T21:47:53.796015Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1060 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:53.796028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:53.796049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1060 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:53.796060Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1060 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:53.796064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:53.796601Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1060 sessionId:  describe result for acl check
2025-10-09T21:47:53.796638Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:53.796640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:53.796642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:53.796648Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342042224768375:9489] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:53.797215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342042224768375:9489] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:53.798101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:53.798296Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:53.798307Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:53.798318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:53.798376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:53.798414Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342041862947376:8499]
2025-10-09T21:47:53.799021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:53.799035Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342042224768375:9489] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:53.799287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:53.799328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:53.799623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:53.799672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.799677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.799678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.799679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.799680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.799682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.799683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.799684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.799685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.799691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.799700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.799704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.799708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.799712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.799716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.799720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.799724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.799728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.799734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.799742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.799748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.799753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.799758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.799763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.799768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.799776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.799786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.799791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.799799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.799805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.799814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.799820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.799828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.799834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.799842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.799847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.799882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:53.804732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342042224768375:9489] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:53.804750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342042224768375:9489] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:53.804754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342042224768375:9489] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:53.804763Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1060 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:53.805987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:53.807095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:53.807110Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:53.807112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:53.807117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:53.807123Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1059 [2:7559342041862947376:8499]
2025-10-09T21:47:53.807136Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:53.807153Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:53.807156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:53.807159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:53.807161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:53.807177Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:53.807193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:53.807199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:53.807256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.807265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.807267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.807269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:53.807335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:53.807338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.807350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.807368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.807388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:53.807416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342042224768275:2432], now have 1 active actors on pipe
2025-10-09T21:47:53.807420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342042224768412:9489], now have 1 active actors on pipe
2025-10-09T21:47:53.807431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342041862947370:8499]
2025-10-09T21:47:53.807433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342041862947370:8499]
2025-10-09T21:47:53.807435Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342041862947370:8499]
2025-10-09T21:47:53.807447Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:53.807461Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:53.807466Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:53.807469Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:53.807471Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:53.807479Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:53.807483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:53.807466Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1059
2025-10-09T21:47:53.807476Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1059
2025-10-09T21:47:53.807516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342041862947370:8499]
2025-10-09T21:47:53.807527Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:53.807545Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:53.807546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:53.807551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:53.807591Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7e8dcd86-fdac34af-c113fcc0-3a0d19ca_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:53.807604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:53.807608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:53.807611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:53.807622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:53.807649Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:53.807672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:53.807816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:53.807823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:53.807835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:53.807932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1060 partition: 0 MaxSeqNo: 25 sessionId: src_id|7e8dcd86-fdac34af-c113fcc0-3a0d19ca_0
2025-10-09T21:47:53.808292Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ac46119-dbb24bcf-d2f7f01d-bfb1b623_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7e8dcd86-fdac34af-c113fcc0-3a0d19ca_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:53.810694Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e8dcd86-fdac34af-c113fcc0-3a0d19ca_0] Write session: try to update token
2025-10-09T21:47:53.810710Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e8dcd86-fdac34af-c113fcc0-3a0d19ca_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:53.821412Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:53.821434Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:53.821436Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:53.821438Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:53.821449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342041862947370:8499]
2025-10-09T21:47:53.821453Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342041862947370:8499]
2025-10-09T21:47:53.821457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342041862947370:8499]
2025-10-09T21:47:53.821504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:53.821512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.821514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:53.821650Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:53.823475Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1060 sessionId: src_id|7e8dcd86-fdac34af-c113fcc0-3a0d19ca_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:53.825875Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:53.826046Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:53.826063Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:53.826067Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:53.885180Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342042224768421:9144]
2025-10-09T21:47:53.885497Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 1059, pipe: [1:7559342042224768421:9144]
2025-10-09T21:47:53.981012Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:53.981314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:53.981329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:53.981372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342042224768421:9144], now have 1 active actors on pipe
2025-10-09T21:47:53.981560Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:53.981685Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:53.981749Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342042224768421:9144]
2025-10-09T21:47:53.981773Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:53.981872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1060 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7e8dcd86-fdac34af-c113fcc0-3a0d19ca_0
2025-10-09T21:47:53.982062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1060 sessionId: src_id|7e8dcd86-fdac34af-c113fcc0-3a0d19ca_0 is DEAD
2025-10-09T21:47:53.982225Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e8dcd86-fdac34af-c113fcc0-3a0d19ca_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:53.982236Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e8dcd86-fdac34af-c113fcc0-3a0d19ca_0] Write session will restart in 0.006953s
2025-10-09T21:47:53.982263Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e8dcd86-fdac34af-c113fcc0-3a0d19ca_0] Write session: Do CDS request
2025-10-09T21:47:53.982269Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e8dcd86-fdac34af-c113fcc0-3a0d19ca_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:53.990510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e8dcd86-fdac34af-c113fcc0-3a0d19ca_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:53.990601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:53.990618Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1061
2025-10-09T21:47:53.990766Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1061 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:53.990810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1061 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:53.990815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1061 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:53.990819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:53.991278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1061 sessionId:  describe result for acl check
2025-10-09T21:47:53.991317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:53.991319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:53.991321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:53.991326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342042224768465:9501] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:53.991846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342042224768465:9501] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:53.994147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342042224768465:9501] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:53.996178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:53.998212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:53.998456Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:53.998467Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:53.998481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:53.998544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:53.998593Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342041862947537:8505]
2025-10-09T21:47:53.998761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342042224768465:9501] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:53.998781Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342042224768465:9501] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:53.998785Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342042224768465:9501] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:53.998796Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1061 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:53.999169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:53.999489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:53.999548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:53.999909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:53.999959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.999964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.999966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.999967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.999969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:53.999970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:53.999971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:53.999972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:53.999973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:53.999979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:53.999989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:53.999992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:53.999995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:53.999998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.000002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.000006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.000009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.000013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.000020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.000029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.000035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.000040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.000046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.000051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.000056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.000061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.000065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.000070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.000074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.000086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.000091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.000100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.000105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.000110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.000115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.000120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.000155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:54.006891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:54.008027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:54.008046Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:54.008049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:54.008053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:54.008062Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1060 [2:7559342041862947537:8505]
2025-10-09T21:47:54.008078Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:54.008104Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:54.008111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:54.008115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:54.008118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:54.008137Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:54.008149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:54.008157Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:54.008241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.008248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.008251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.008254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:54.008500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:54.008501Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1060
2025-10-09T21:47:54.008512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.008519Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1060
2025-10-09T21:47:54.008527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.008540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.008558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:54.008583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342042224768438:2432], now have 1 active actors on pipe
2025-10-09T21:47:54.008592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342042224768578:9501], now have 1 active actors on pipe
2025-10-09T21:47:54.008617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.008627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.008645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342041862947532:8505]
2025-10-09T21:47:54.008655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342041862947532:8505]
2025-10-09T21:47:54.008657Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342041862947532:8505]
2025-10-09T21:47:54.008673Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:54.008690Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:54.008693Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:54.008696Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:54.008697Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:54.008713Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:54.008722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:54.008743Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|43bda0ff-3ff1b198-7919b957-c9f72150_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:54.008757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:54.008761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:54.008764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:54.008775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:54.008786Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:54.008852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342041862947532:8505]
2025-10-09T21:47:54.008865Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:54.008870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:54.008878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:54.009062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.009077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.009113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:54.009276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1061 partition: 0 MaxSeqNo: 25 sessionId: src_id|43bda0ff-3ff1b198-7919b957-c9f72150_0
2025-10-09T21:47:54.009690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e8dcd86-fdac34af-c113fcc0-3a0d19ca_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|43bda0ff-3ff1b198-7919b957-c9f72150_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:54.012314Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|43bda0ff-3ff1b198-7919b957-c9f72150_0] Write session: try to update token
2025-10-09T21:47:54.012336Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|43bda0ff-3ff1b198-7919b957-c9f72150_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:54.021123Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:54.021143Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:54.021153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342041862947532:8505]
2025-10-09T21:47:54.021202Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:54.021205Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:54.021385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:54.021525Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:54.021663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:54.021813Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:54.022002Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:54.022070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:54.028421Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1061 sessionId: src_id|43bda0ff-3ff1b198-7919b957-c9f72150_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:54.030984Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:54.031240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.031265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.031309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:54.031412Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:54.035134Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:54.035152Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:54.035155Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:54.035157Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:54.035171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342041862947532:8505]
2025-10-09T21:47:54.035175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342041862947532:8505]
2025-10-09T21:47:54.035178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342041862947532:8505]
2025-10-09T21:47:54.035226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:54.035234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.035233Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:54.035237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:54.035241Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:54.035244Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:54.035388Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:54.082222Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342046519735885:9144]
2025-10-09T21:47:54.082557Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 1060, pipe: [1:7559342046519735885:9144]
2025-10-09T21:47:54.199512Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:54.207545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.207576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.207750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.207819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.207884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.207947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.207991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.208003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.208072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.208141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.208217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.208289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.208334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.208345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.208410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.208466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.208526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.208590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.208634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.208641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.208707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.208769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.208830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.208894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.208939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.208946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.209016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.209083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.209143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.209213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.209259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.209267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.209329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.209388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.209451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.209508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.209557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.209564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.209624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.209686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.209751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.209826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.209872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.209879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.209946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.210005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.210095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.210164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.210208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.210216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.210278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.210335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.210400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.210465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.210511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.210517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.210582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.210641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.210703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.210768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.210817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.210824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.210883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342046519735885:9144], now have 1 active actors on pipe
2025-10-09T21:47:54.211106Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:54.211123Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342046519735885:9144]
2025-10-09T21:47:54.211125Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:54.211150Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:54.211157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1061 reason: error at write: Disconnected sessionId: src_id|43bda0ff-3ff1b198-7919b957-c9f72150_0
2025-10-09T21:47:54.211375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1061 sessionId: src_id|43bda0ff-3ff1b198-7919b957-c9f72150_0 is DEAD
2025-10-09T21:47:54.211536Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43bda0ff-3ff1b198-7919b957-c9f72150_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:54.211550Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43bda0ff-3ff1b198-7919b957-c9f72150_0] Write session will restart in 0.008367s
2025-10-09T21:47:54.211576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43bda0ff-3ff1b198-7919b957-c9f72150_0] Write session: Do CDS request
2025-10-09T21:47:54.211582Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43bda0ff-3ff1b198-7919b957-c9f72150_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:54.211981Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:54.221620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|43bda0ff-3ff1b198-7919b957-c9f72150_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:54.221695Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:54.221723Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1062
2025-10-09T21:47:54.221852Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1062 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:54.221897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1062 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:54.221908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1062 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:54.221912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:54.222478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1062 sessionId:  describe result for acl check
2025-10-09T21:47:54.222519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:54.222525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:54.222527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:54.222533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342046519735955:9506] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:54.223214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342046519735955:9506] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:54.225904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:54.226045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342046519735955:9506] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:54.228128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:54.228370Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:54.228380Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:54.228394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:54.228452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:54.228498Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342046157914969:8509]
2025-10-09T21:47:54.228921Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342046519735955:9506] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:54.228939Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342046519735955:9506] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:54.228943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342046519735955:9506] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:54.228953Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1062 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:54.229140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:54.229453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:54.229509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:54.229867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:54.229927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.229933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.229935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.229936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.229938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.229939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.229940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.229942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.229943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.229949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.229959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.229963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.229967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.229971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.229976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.229980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.229984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.229987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.229995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.230004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.230010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.230016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.230026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.230047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.230051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.230057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.230067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.230072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.230076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.230082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.230091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.230097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.230106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.230111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.230120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.230126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.230160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:54.238579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:54.239664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:54.239681Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:54.239684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:54.239690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:54.239698Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1061 [2:7559342046157914969:8509]
2025-10-09T21:47:54.239714Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:54.239735Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:54.239739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:54.239743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:54.239745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:54.239764Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:54.239781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:54.239787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:54.239852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.239859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.239861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.239864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:54.239938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:54.239946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.239955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.239962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.239973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:54.239991Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342046157914964:8509]
2025-10-09T21:47:54.239999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342046157914964:8509]
2025-10-09T21:47:54.240001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342046157914964:8509]
2025-10-09T21:47:54.240010Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:54.240022Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:54.240026Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:54.240027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342046519736030:9506], now have 1 active actors on pipe
2025-10-09T21:47:54.240029Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:54.240036Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:54.240050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:54.240060Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:54.240066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342046519735896:2432], now have 1 active actors on pipe
2025-10-09T21:47:54.240104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342046157914964:8509]
2025-10-09T21:47:54.240117Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:54.240152Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:54.240173Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1061
2025-10-09T21:47:54.240184Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1061
2025-10-09T21:47:54.240234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.240250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.240288Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2cbdf3ad-8a6c0fd0-712a71b5-714de929_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:54.240304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:54.240309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:54.240312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:54.240325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:54.240336Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:54.240355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:54.240621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.240633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.240655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:54.240826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1062 partition: 0 MaxSeqNo: 25 sessionId: src_id|2cbdf3ad-8a6c0fd0-712a71b5-714de929_0
2025-10-09T21:47:54.241246Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43bda0ff-3ff1b198-7919b957-c9f72150_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2cbdf3ad-8a6c0fd0-712a71b5-714de929_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:54.244039Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2cbdf3ad-8a6c0fd0-712a71b5-714de929_0] Write session: try to update token
2025-10-09T21:47:54.244079Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2cbdf3ad-8a6c0fd0-712a71b5-714de929_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:54.255233Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:54.255255Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:54.255257Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:54.255259Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:54.255270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342046157914964:8509]
2025-10-09T21:47:54.255273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342046157914964:8509]
2025-10-09T21:47:54.255278Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342046157914964:8509]
2025-10-09T21:47:54.255326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:54.255334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.255337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:54.255338Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:54.255344Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:54.255349Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:54.255472Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:54.258550Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1062 sessionId: src_id|2cbdf3ad-8a6c0fd0-712a71b5-714de929_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:54.261122Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:54.311141Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342046519736041:9144]
2025-10-09T21:47:54.311477Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 1061, pipe: [1:7559342046519736041:9144]
2025-10-09T21:47:54.409763Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:54.410125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.410142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.410184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342046519736041:9144], now have 1 active actors on pipe
2025-10-09T21:47:54.410394Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:54.410468Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:54.410472Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342046519736041:9144]
2025-10-09T21:47:54.410514Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:54.410592Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1062 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2cbdf3ad-8a6c0fd0-712a71b5-714de929_0
2025-10-09T21:47:54.410800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1062 sessionId: src_id|2cbdf3ad-8a6c0fd0-712a71b5-714de929_0 is DEAD
2025-10-09T21:47:54.410966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2cbdf3ad-8a6c0fd0-712a71b5-714de929_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:54.410979Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2cbdf3ad-8a6c0fd0-712a71b5-714de929_0] Write session will restart in 0.008065s
2025-10-09T21:47:54.411007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2cbdf3ad-8a6c0fd0-712a71b5-714de929_0] Write session: Do CDS request
2025-10-09T21:47:54.411024Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2cbdf3ad-8a6c0fd0-712a71b5-714de929_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:54.420560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2cbdf3ad-8a6c0fd0-712a71b5-714de929_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:54.420688Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:54.420709Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1063
2025-10-09T21:47:54.420925Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1063 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:54.420972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1063 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:54.420984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1063 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:54.420988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:54.421510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1063 sessionId:  describe result for acl check
2025-10-09T21:47:54.421553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:54.421566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:54.421568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:54.421575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342046519736087:9510] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:54.422242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342046519736087:9510] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:54.424503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342046519736087:9510] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:54.425328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:54.427366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:54.427575Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:54.427585Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:54.427598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:54.427653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:54.427699Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342046157915106:8511]
2025-10-09T21:47:54.427655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342046519736087:9510] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:54.427673Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342046519736087:9510] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:54.427677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342046519736087:9510] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:54.427686Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1063 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:54.428289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:54.428645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:54.428710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:54.429042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:54.429099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.429105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.429106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.429108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.429109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.429110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.429111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.429112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.429113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.429120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.429132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.429136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.429140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.429144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.429149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.429153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.429157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.429161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.429169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.429178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.429185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.429191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.429204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.429210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.429215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.429220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.429224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.429229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.429233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.429238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.429242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.429248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.429253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.429258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.429262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.429267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.429296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:54.436167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:54.437382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:54.437395Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:54.437398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:54.437402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:54.437409Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1062 [2:7559342046157915106:8511]
2025-10-09T21:47:54.437427Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:54.437448Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:54.437458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:54.437462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:54.437465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:54.437484Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:54.437493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:54.437500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:54.437560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.437567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.437569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.437573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:54.437648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:54.437655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.437667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.437679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.437701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:54.437727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342046519736179:9510], now have 1 active actors on pipe
2025-10-09T21:47:54.437735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342046519736044:2432], now have 1 active actors on pipe
2025-10-09T21:47:54.437742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342046157915101:8511]
2025-10-09T21:47:54.437745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342046157915101:8511]
2025-10-09T21:47:54.437747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342046157915101:8511]
2025-10-09T21:47:54.437764Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:54.437776Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:54.437784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:54.437754Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1062
2025-10-09T21:47:54.437786Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:54.437800Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:54.437812Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:54.437815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:54.437780Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1062
2025-10-09T21:47:54.437870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.437880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.437897Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342046157915101:8511]
2025-10-09T21:47:54.437908Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:54.437914Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:54.437942Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d83a1bbd-8f0d12ce-661e2e5b-3cee0bdb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:54.437954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:54.437962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:54.437966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:54.437981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:54.437994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:54.438022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:54.438232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.438245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.438268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:54.438432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1063 partition: 0 MaxSeqNo: 25 sessionId: src_id|d83a1bbd-8f0d12ce-661e2e5b-3cee0bdb_0
2025-10-09T21:47:54.438786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2cbdf3ad-8a6c0fd0-712a71b5-714de929_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d83a1bbd-8f0d12ce-661e2e5b-3cee0bdb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:54.441385Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d83a1bbd-8f0d12ce-661e2e5b-3cee0bdb_0] Write session: try to update token
2025-10-09T21:47:54.441403Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d83a1bbd-8f0d12ce-661e2e5b-3cee0bdb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:54.452274Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:54.452291Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:54.452294Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:54.452296Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:54.452305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342046157915101:8511]
2025-10-09T21:47:54.452311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342046157915101:8511]
2025-10-09T21:47:54.452314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342046157915101:8511]
2025-10-09T21:47:54.452362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:54.452361Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:54.452367Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:54.452370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.452371Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:54.452374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:54.452508Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:54.455609Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1063 sessionId: src_id|d83a1bbd-8f0d12ce-661e2e5b-3cee0bdb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:54.458286Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:54.510994Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342046519736191:9144]
2025-10-09T21:47:54.511292Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 1062, pipe: [1:7559342046519736191:9144]
2025-10-09T21:47:54.603965Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:54.604463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.604481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.604525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342046519736191:9144], now have 1 active actors on pipe
2025-10-09T21:47:54.604730Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:54.604810Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:54.604812Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342046519736191:9144]
2025-10-09T21:47:54.604827Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:54.604834Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:54.604887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1063 reason: error at write: Disconnected sessionId: src_id|d83a1bbd-8f0d12ce-661e2e5b-3cee0bdb_0
2025-10-09T21:47:54.605068Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1063 sessionId: src_id|d83a1bbd-8f0d12ce-661e2e5b-3cee0bdb_0 is DEAD
2025-10-09T21:47:54.605214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d83a1bbd-8f0d12ce-661e2e5b-3cee0bdb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:54.605224Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d83a1bbd-8f0d12ce-661e2e5b-3cee0bdb_0] Write session will restart in 0.007668s
2025-10-09T21:47:54.605251Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d83a1bbd-8f0d12ce-661e2e5b-3cee0bdb_0] Write session: Do CDS request
2025-10-09T21:47:54.605257Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d83a1bbd-8f0d12ce-661e2e5b-3cee0bdb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:54.614540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d83a1bbd-8f0d12ce-661e2e5b-3cee0bdb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:54.614663Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:54.614683Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1064
2025-10-09T21:47:54.614812Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1064 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:54.614856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1064 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:54.614866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1064 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:54.614874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:54.615242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:54.615312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1064 sessionId:  describe result for acl check
2025-10-09T21:47:54.615354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:54.615360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:54.615362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:54.615368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342046519736293:9514] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:54.615876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342046519736293:9514] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:54.617164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:54.617323Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:54.617333Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:54.617346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:54.617397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:54.617430Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342046157915245:8513]
2025-10-09T21:47:54.617862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:54.617853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342046519736293:9514] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:54.618136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:54.618168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:54.618408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:54.618457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.618464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.618465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.618466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.618467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.618468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.618469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.618470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.618472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.618476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.618486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.618490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.618494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.618498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.618502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.618505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.618508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.618512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.618517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.618524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.618530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.618535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.618538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.618544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.618548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.618553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.618558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.618563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.618567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.618578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.618583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.618589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.618598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.618602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.618607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.618616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.618644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:54.623650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342046519736293:9514] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:54.623672Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342046519736293:9514] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:54.623676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342046519736293:9514] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:54.623689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1064 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:54.624894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:54.625947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:54.625962Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:54.625964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:54.625967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:54.625972Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1063 [2:7559342046157915245:8513]
2025-10-09T21:47:54.625982Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:54.625996Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:54.625998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:54.626000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:54.626001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:54.626018Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:54.626024Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:54.626054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:54.626103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.626112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.626114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.626116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:54.626328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:54.626341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.626331Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1063
2025-10-09T21:47:54.626353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.626336Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1063
2025-10-09T21:47:54.626365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.626385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:54.626406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342046519736193:2432], now have 1 active actors on pipe
2025-10-09T21:47:54.626416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342046519736333:9514], now have 1 active actors on pipe
2025-10-09T21:47:54.626428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.626435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.626452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342046157915239:8513]
2025-10-09T21:47:54.626457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342046157915239:8513]
2025-10-09T21:47:54.626458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342046157915239:8513]
2025-10-09T21:47:54.626468Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:54.626480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:54.626494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:54.626495Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:54.626498Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:54.626500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:54.626500Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:54.626517Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8802f3f1-4d7e5229-8912eb26-9b60f40f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:54.626525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:54.626534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:54.626538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:54.626549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:54.626563Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:54.626601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342046157915239:8513]
2025-10-09T21:47:54.626612Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:54.626618Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:54.626628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:54.626756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.626764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.626780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:54.626848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1064 partition: 0 MaxSeqNo: 25 sessionId: src_id|8802f3f1-4d7e5229-8912eb26-9b60f40f_0
2025-10-09T21:47:54.627219Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d83a1bbd-8f0d12ce-661e2e5b-3cee0bdb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8802f3f1-4d7e5229-8912eb26-9b60f40f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:54.629641Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8802f3f1-4d7e5229-8912eb26-9b60f40f_0] Write session: try to update token
2025-10-09T21:47:54.629659Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8802f3f1-4d7e5229-8912eb26-9b60f40f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:54.638355Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:54.638371Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:54.638383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342046157915239:8513]
2025-10-09T21:47:54.638430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:54.638429Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:54.638597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:54.638741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:54.638876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:54.639016Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:54.639220Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:54.639260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:54.642723Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1064 sessionId: src_id|8802f3f1-4d7e5229-8912eb26-9b60f40f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:54.645022Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:54.645215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.645234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.645275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:54.645356Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:54.651100Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:54.651116Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:54.651118Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:54.651120Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:54.651131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342046157915239:8513]
2025-10-09T21:47:54.651136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342046157915239:8513]
2025-10-09T21:47:54.651140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342046157915239:8513]
2025-10-09T21:47:54.651183Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:54.651185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:54.651192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.651195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:54.651196Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:54.651200Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:54.651323Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:54.704887Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342046519736341:9144]
2025-10-09T21:47:54.705143Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 1063, pipe: [1:7559342046519736341:9144]
2025-10-09T21:47:54.802575Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:54.810699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.810725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.810875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.810940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.811010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.811074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.811127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.811139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.811210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.811271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.811340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.811409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.811455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.811462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.811528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.811590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.811659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.811720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.811763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.811770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.811834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.811895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.811955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.812019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.812063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.812069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.812133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.812191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.812260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.812325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.812374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.812381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.812443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.812502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.812570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.812630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.812679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.812686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.812751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.812811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.812874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.812934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.812982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.812990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.813053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.813112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.813178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.813245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.813292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.813298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.813360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.813422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.813480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.813545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.813593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.813601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.813666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:54.813726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:54.813796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:54.813855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:54.813898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:54.813907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:54.813965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342046519736341:9144], now have 1 active actors on pipe
2025-10-09T21:47:54.814256Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:54.814302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:54.814309Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:54.814340Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1064 reason: error at write: Disconnected sessionId: src_id|8802f3f1-4d7e5229-8912eb26-9b60f40f_0
2025-10-09T21:47:54.814570Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342046519736341:9144]
2025-10-09T21:47:54.814592Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1064 sessionId: src_id|8802f3f1-4d7e5229-8912eb26-9b60f40f_0 is DEAD
2025-10-09T21:47:54.814711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8802f3f1-4d7e5229-8912eb26-9b60f40f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:54.814722Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8802f3f1-4d7e5229-8912eb26-9b60f40f_0] Write session will restart in 0.006413s
2025-10-09T21:47:54.814746Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8802f3f1-4d7e5229-8912eb26-9b60f40f_0] Write session: Do CDS request
2025-10-09T21:47:54.814752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8802f3f1-4d7e5229-8912eb26-9b60f40f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:54.815044Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:54.823317Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8802f3f1-4d7e5229-8912eb26-9b60f40f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:54.823410Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:54.823425Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1065
2025-10-09T21:47:54.823536Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1065 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:54.823581Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1065 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:54.823592Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1065 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:54.823596Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:54.824152Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1065 sessionId:  describe result for acl check
2025-10-09T21:47:54.824205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:54.824207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:54.824209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:54.824215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342046519736399:9519] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:54.824785Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342046519736399:9519] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:54.828277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342046519736399:9519] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:54.829140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:54.831353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:54.831584Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:54.831598Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:54.831612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:54.831671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:54.831717Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342046157915402:8523]
2025-10-09T21:47:54.832355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:54.832542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342046519736399:9519] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:54.832561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342046519736399:9519] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:54.832565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342046519736399:9519] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:54.832574Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1065 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:54.832657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:54.832704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:54.833027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:54.833075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.833081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.833082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.833084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.833085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.833086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.833087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.833089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.833090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.833095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.833105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.833110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.833114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.833118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.833122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.833126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.833130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.833134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.833140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.833149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:54.833155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.833161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:54.833165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.833171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:54.833175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.833181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:54.833192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.833198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:54.833207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.833213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:54.833223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.833228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.833237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.833243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.833247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.833253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.833286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:54.839998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:54.841103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:54.841122Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:54.841126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:54.841131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:54.841138Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1064 [2:7559342046157915402:8523]
2025-10-09T21:47:54.841153Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:54.841171Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:54.841174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:54.841179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:54.841181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:54.841200Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:54.841211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:54.841219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:54.841293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.841301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.841303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.841306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:54.841388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:54.841401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.841414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:54.841426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:54.841443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:54.841468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342046519736489:9519], now have 1 active actors on pipe
2025-10-09T21:47:54.841476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342046157915397:8523]
2025-10-09T21:47:54.841478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342046157915397:8523]
2025-10-09T21:47:54.841481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342046157915397:8523]
2025-10-09T21:47:54.841495Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:54.841511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342046519736349:2432], now have 1 active actors on pipe
2025-10-09T21:47:54.841524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:54.841540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:54.841540Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:54.841544Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:54.841547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:54.841549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:54.841592Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342046157915397:8523]
2025-10-09T21:47:54.841605Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:54.841583Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1064
2025-10-09T21:47:54.841588Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1064
2025-10-09T21:47:54.841628Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:54.841669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.841679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.841708Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|20d79f9c-7cd1187d-5779b120-643ac01_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:54.841720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:54.841725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:54.841727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:54.841740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:54.841754Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:54.841775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:54.842003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.842018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.842052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:54.842146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1065 partition: 0 MaxSeqNo: 25 sessionId: src_id|20d79f9c-7cd1187d-5779b120-643ac01_0
2025-10-09T21:47:54.842527Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8802f3f1-4d7e5229-8912eb26-9b60f40f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|20d79f9c-7cd1187d-5779b120-643ac01_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:54.845052Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|20d79f9c-7cd1187d-5779b120-643ac01_0] Write session: try to update token
2025-10-09T21:47:54.845076Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|20d79f9c-7cd1187d-5779b120-643ac01_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:54.854557Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:54.854575Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:54.854585Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342046157915397:8523]
2025-10-09T21:47:54.854632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:54.854633Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:54.854808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:54.854954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:54.855085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:54.855225Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:54.855410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:54.855455Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:54.862537Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1065 sessionId: src_id|20d79f9c-7cd1187d-5779b120-643ac01_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:54.864838Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:54.865018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:54.865039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:54.865082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:54.865223Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:54.866765Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:54.866778Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:54.866781Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:54.866783Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:54.866793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342046157915397:8523]
2025-10-09T21:47:54.866797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342046157915397:8523]
2025-10-09T21:47:54.866803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342046157915397:8523]
2025-10-09T21:47:54.866842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:54.866850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:54.866853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:54.866858Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:54.866864Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:54.866880Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:54.866983Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:54.914823Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342046519736499:9144]
2025-10-09T21:47:54.915114Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 1064, pipe: [1:7559342046519736499:9144]
2025-10-09T21:47:55.021904Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:55.030161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:55.030190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:55.030348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.030409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.030472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.030533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.030578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.030589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.030650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.030704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.030763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.030821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.030867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.030874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.030934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.030991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.031052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.031116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.031159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.031168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.031239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.031308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.031377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.031435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.031484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.031492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.031556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.031619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.031684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.031752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.031796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.031803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.031867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.031930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.031990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.032048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.032095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.032104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.032164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.032214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.032268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.032335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.032380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.032387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.032447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.032507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.032570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.032633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.032679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.032688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.032748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.032812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.032876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.032944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.032987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.032994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.033055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.033115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.033174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.033236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.033280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.033288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.033351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342046519736499:9144], now have 1 active actors on pipe
2025-10-09T21:47:55.033593Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:55.033605Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:55.033637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1065 reason: error at write: Disconnected sessionId: src_id|20d79f9c-7cd1187d-5779b120-643ac01_0
2025-10-09T21:47:55.034007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20d79f9c-7cd1187d-5779b120-643ac01_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:55.034018Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20d79f9c-7cd1187d-5779b120-643ac01_0] Write session will restart in 0.006196s
2025-10-09T21:47:55.034061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20d79f9c-7cd1187d-5779b120-643ac01_0] Write session: Do CDS request
2025-10-09T21:47:55.034069Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20d79f9c-7cd1187d-5779b120-643ac01_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:55.033863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1065 sessionId: src_id|20d79f9c-7cd1187d-5779b120-643ac01_0 is DEAD
2025-10-09T21:47:55.034100Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342046519736499:9144]
2025-10-09T21:47:55.034132Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:55.034399Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:55.041572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|20d79f9c-7cd1187d-5779b120-643ac01_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:55.041820Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:55.041834Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1066
2025-10-09T21:47:55.042494Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1066 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:55.042538Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1066 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:55.042548Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1066 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:55.042551Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:55.042944Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1066 sessionId:  describe result for acl check
2025-10-09T21:47:55.042978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:55.042983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:55.042985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:55.042990Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342050814703869:9530] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:55.043562Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342050814703869:9530] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:55.046814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:55.049157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:55.049399Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:55.049410Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:55.049425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:55.049494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:55.049540Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342050452882863:8529]
2025-10-09T21:47:55.049680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342050814703869:9530] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:55.050205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:55.050579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:55.050635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:55.050952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:55.051008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.051016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.051018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.051019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.051020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.051021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.051023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.051024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.051025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.051030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.051040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.051043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.051046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.051049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.051053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.051056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.051059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.051062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.051069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.051076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.051083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.051087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.051091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.051095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.051099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.051103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.051107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.051112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.051115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.051120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.051124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.051129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.051133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.051137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.051141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.051145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.051180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:55.054128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342050814703869:9530] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:55.054146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342050814703869:9530] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:55.054150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342050814703869:9530] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:55.054161Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1066 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:55.057948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:55.059012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:55.059032Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:55.059035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:55.059040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:55.059047Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1065 [2:7559342050452882863:8529]
2025-10-09T21:47:55.059061Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:55.059082Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:55.059085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:55.059088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:55.059090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:55.059109Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:55.059115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:55.059123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:55.059185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.059193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.059195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.059198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:55.059273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:55.059281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.059293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.059309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.059330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:55.059358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342050452882858:8529]
2025-10-09T21:47:55.059361Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342050452882858:8529]
2025-10-09T21:47:55.059363Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342050452882858:8529]
2025-10-09T21:47:55.059379Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:55.059394Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:55.059408Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:55.059416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:55.059457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342050452882858:8529]
2025-10-09T21:47:55.059464Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:55.059492Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:55.059494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342050814703961:9530], now have 1 active actors on pipe
2025-10-09T21:47:55.059498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342050814703813:2432], now have 1 active actors on pipe
2025-10-09T21:47:55.059504Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:55.059506Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:55.059514Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:55.059600Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1065
2025-10-09T21:47:55.059604Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1065
2025-10-09T21:47:55.059680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:55.059691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:55.059721Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|77236130-9103271d-e5e6fd2b-6f5ed79_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:55.059735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:55.059740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:55.059743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:55.059757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:55.059767Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:55.059789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:55.059951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:55.059958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:55.059971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:55.060066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1066 partition: 0 MaxSeqNo: 25 sessionId: src_id|77236130-9103271d-e5e6fd2b-6f5ed79_0
2025-10-09T21:47:55.060443Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20d79f9c-7cd1187d-5779b120-643ac01_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|77236130-9103271d-e5e6fd2b-6f5ed79_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:55.062995Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|77236130-9103271d-e5e6fd2b-6f5ed79_0] Write session: try to update token
2025-10-09T21:47:55.063015Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|77236130-9103271d-e5e6fd2b-6f5ed79_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:55.071755Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:55.071776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:55.071784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342050452882858:8529]
2025-10-09T21:47:55.071826Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:55.071827Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:55.071992Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:55.072143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:55.072278Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:55.072415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:55.072592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:55.072628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:55.077899Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1066 sessionId: src_id|77236130-9103271d-e5e6fd2b-6f5ed79_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:55.080387Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:55.080601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:55.080624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:55.080665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:55.080754Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:55.084986Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:55.085006Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:55.085008Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:55.085010Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:55.085019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342050452882858:8529]
2025-10-09T21:47:55.085023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342050452882858:8529]
2025-10-09T21:47:55.085027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342050452882858:8529]
2025-10-09T21:47:55.085068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:55.085074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.085073Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:55.085077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:55.085079Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:55.085084Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:55.085208Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:55.134432Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342050814703972:9144]
2025-10-09T21:47:55.134702Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 1065, pipe: [1:7559342050814703972:9144]
2025-10-09T21:47:55.234331Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:55.241919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:55.241948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:55.242133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.242197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.242261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.242324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.242367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.242376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.242437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.242494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.242553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.242617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.242661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.242668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.242733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.242790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.242847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.242906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.242947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.242954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.243022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.243084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.243150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.243209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.243251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.243257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.243322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.243382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.243442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.243503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.243545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.243552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.243614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.243672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.243734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.243792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.243837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.243844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.243909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.243965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.244028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.244086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.244127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.244136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.244193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.244252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.244313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.244375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.244420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.244423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.244483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.244544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.244604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.244668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.244709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.244717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.244779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:55.244845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:55.244906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:55.244965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:55.245012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:55.245018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:55.245070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342050814703972:9144], now have 1 active actors on pipe
2025-10-09T21:47:55.245338Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:55.245350Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:55.245355Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:55.245389Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1066 reason: error at write: Disconnected sessionId: src_id|77236130-9103271d-e5e6fd2b-6f5ed79_0
2025-10-09T21:47:55.245400Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342050814703972:9144]
2025-10-09T21:47:55.245600Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1066 sessionId: src_id|77236130-9103271d-e5e6fd2b-6f5ed79_0 is DEAD
2025-10-09T21:47:55.245733Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|77236130-9103271d-e5e6fd2b-6f5ed79_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:55.245745Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|77236130-9103271d-e5e6fd2b-6f5ed79_0] Write session will restart in 0.006230s
2025-10-09T21:47:55.245775Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|77236130-9103271d-e5e6fd2b-6f5ed79_0] Write session: Do CDS request
2025-10-09T21:47:55.245782Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|77236130-9103271d-e5e6fd2b-6f5ed79_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:55.246127Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:55.253474Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|77236130-9103271d-e5e6fd2b-6f5ed79_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:55.253571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:55.253587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1067
2025-10-09T21:47:55.253753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1067 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:55.253815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1067 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:55.253826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1067 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:55.253829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:55.254391Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1067 sessionId:  describe result for acl check
2025-10-09T21:47:55.254426Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:55.254432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:55.254434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:55.254443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342050814704035:9535] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:55.254976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342050814704035:9535] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:55.257783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:55.258001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342050814704035:9535] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:55.259938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:55.260142Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:55.260154Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:55.260167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:55.260223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:55.260264Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342050452883004:8533]
2025-10-09T21:47:55.260621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342050814704035:9535] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:55.260636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342050814704035:9535] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:55.260641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342050814704035:9535] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:55.260654Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1067 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:55.260878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:55.261122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:55.261165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:55.261502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:55.261548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.261554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.261556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.261557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.261558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.261559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.261561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.261562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.261564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.261569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.261580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.261584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.261589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.261593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.261597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.261601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.261605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.261609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.261616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.261624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.261631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.261636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.261641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.261646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.261651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.261656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.261660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.261665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.261670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.261675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.261686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.261691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.261701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.261706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.261711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.261716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.261750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:55.268445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:55.269770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:55.269798Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:55.269802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:55.269807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:55.269814Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1066 [2:7559342050452883004:8533]
2025-10-09T21:47:55.269829Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:55.269851Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:55.269858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:55.269861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:55.269863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:55.269882Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:55.269893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:55.269904Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:55.269970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.269979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.269981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.269984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:55.270078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:55.270087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.270101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.270118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.270136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:55.270162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342050814704118:9535], now have 1 active actors on pipe
2025-10-09T21:47:55.270170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342050814703979:2432], now have 1 active actors on pipe
2025-10-09T21:47:55.270179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342050452882998:8533]
2025-10-09T21:47:55.270181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342050452882998:8533]
2025-10-09T21:47:55.270184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342050452882998:8533]
2025-10-09T21:47:55.270197Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:55.270216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:55.270222Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:55.270226Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:55.270229Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:55.270234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:55.270237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:55.270251Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1066
2025-10-09T21:47:55.270279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342050452882998:8533]
2025-10-09T21:47:55.270288Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:55.270290Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:55.270260Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1066
2025-10-09T21:47:55.270316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:55.270325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:55.270364Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c08c1c81-e7439f95-3bc8669e-e58d292a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:55.270377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:55.270381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:55.270383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:55.270394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:55.270404Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:55.270429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:55.270594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:55.270603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:55.270619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:55.270705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1067 partition: 0 MaxSeqNo: 25 sessionId: src_id|c08c1c81-e7439f95-3bc8669e-e58d292a_0
2025-10-09T21:47:55.271061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|77236130-9103271d-e5e6fd2b-6f5ed79_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c08c1c81-e7439f95-3bc8669e-e58d292a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:55.273512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c08c1c81-e7439f95-3bc8669e-e58d292a_0] Write session: try to update token
2025-10-09T21:47:55.273533Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c08c1c81-e7439f95-3bc8669e-e58d292a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:55.284906Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:55.284929Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:55.284932Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:55.284935Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:55.284945Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342050452882998:8533]
2025-10-09T21:47:55.284949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342050452882998:8533]
2025-10-09T21:47:55.284953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342050452882998:8533]
2025-10-09T21:47:55.284999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:55.285006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.285007Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:55.285010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:55.285013Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:55.285018Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:55.285150Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:55.287184Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1067 sessionId: src_id|c08c1c81-e7439f95-3bc8669e-e58d292a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:55.289619Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:55.345378Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342050814704130:9144]
2025-10-09T21:47:55.345718Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 1066, pipe: [1:7559342050814704130:9144]
2025-10-09T21:47:55.445168Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:55.445558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:55.445576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:55.445622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342050814704130:9144], now have 1 active actors on pipe
2025-10-09T21:47:55.445852Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:55.445881Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:55.445941Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342050814704130:9144]
2025-10-09T21:47:55.446059Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:55.446137Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1067 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c08c1c81-e7439f95-3bc8669e-e58d292a_0
2025-10-09T21:47:55.446315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1067 sessionId: src_id|c08c1c81-e7439f95-3bc8669e-e58d292a_0 is DEAD
2025-10-09T21:47:55.446491Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c08c1c81-e7439f95-3bc8669e-e58d292a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:55.446503Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c08c1c81-e7439f95-3bc8669e-e58d292a_0] Write session will restart in 0.006081s
2025-10-09T21:47:55.446532Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c08c1c81-e7439f95-3bc8669e-e58d292a_0] Write session: Do CDS request
2025-10-09T21:47:55.446539Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c08c1c81-e7439f95-3bc8669e-e58d292a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:55.453589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c08c1c81-e7439f95-3bc8669e-e58d292a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:55.453635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:55.453650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1068
2025-10-09T21:47:55.453822Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1068 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:55.453857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1068 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:55.453867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1068 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:55.453871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:55.454321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1068 sessionId:  describe result for acl check
2025-10-09T21:47:55.454360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:55.454362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:55.454364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:55.454370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342050814704174:9539] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:55.454972Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342050814704174:9539] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:55.457300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342050814704174:9539] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:55.458219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:55.460230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:55.460440Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:55.460452Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:55.460435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342050814704174:9539] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:55.460465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:55.460450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342050814704174:9539] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:55.460454Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342050814704174:9539] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:55.460463Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1068 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:55.460524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:55.460570Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342050452883146:8535]
2025-10-09T21:47:55.461196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:55.461513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:55.461559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:55.461906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:55.461965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.461971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.461973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.461974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.461975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.461977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.461978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.461979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.461981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.461987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.461999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.462003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.462008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.462012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.462017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.462020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.462025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.462046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.462053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.462066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.462077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.462084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.462096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.462102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.462107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.462113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.462118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.462123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.462128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.462134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.462140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.462145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.462150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.462156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.462161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.462167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.462198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:55.469056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:55.470203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:55.470219Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:55.470222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:55.470227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:55.470234Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1067 [2:7559342050452883146:8535]
2025-10-09T21:47:55.470254Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:55.470275Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:55.470284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:55.470288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:55.470290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:55.470309Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:55.470321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:55.470329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:55.470395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.470406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.470409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.470412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:55.470492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:55.470502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.470515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.470528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.470548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:55.470572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342050814704132:2432], now have 1 active actors on pipe
2025-10-09T21:47:55.470576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342050814704271:9539], now have 1 active actors on pipe
2025-10-09T21:47:55.470584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342050452883141:8535]
2025-10-09T21:47:55.470586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342050452883141:8535]
2025-10-09T21:47:55.470598Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1067
2025-10-09T21:47:55.470588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342050452883141:8535]
2025-10-09T21:47:55.470633Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1067
2025-10-09T21:47:55.470604Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:55.470620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:55.470619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:55.470623Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:55.470626Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:55.470638Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:55.470641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:55.470686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342050452883141:8535]
2025-10-09T21:47:55.470690Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:55.470699Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:55.470728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:55.470739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:55.470779Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|188017f6-d3ccc228-bc2313d6-1d581a69_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:55.470792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:55.470802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:55.470809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:55.470820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:55.470835Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:55.470855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:55.471045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:55.471054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:55.471068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:55.471174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1068 partition: 0 MaxSeqNo: 25 sessionId: src_id|188017f6-d3ccc228-bc2313d6-1d581a69_0
2025-10-09T21:47:55.471573Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c08c1c81-e7439f95-3bc8669e-e58d292a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|188017f6-d3ccc228-bc2313d6-1d581a69_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:55.474338Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|188017f6-d3ccc228-bc2313d6-1d581a69_0] Write session: try to update token
2025-10-09T21:47:55.474363Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|188017f6-d3ccc228-bc2313d6-1d581a69_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:55.485623Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:55.485646Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:55.485649Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:55.485651Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:55.485663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342050452883141:8535]
2025-10-09T21:47:55.485668Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342050452883141:8535]
2025-10-09T21:47:55.485672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342050452883141:8535]
2025-10-09T21:47:55.485718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:55.485731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.485738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:55.485739Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:55.485748Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:55.485755Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:55.485888Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:55.488403Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1068 sessionId: src_id|188017f6-d3ccc228-bc2313d6-1d581a69_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:55.491006Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:55.546122Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342050814704284:9144]
2025-10-09T21:47:55.546434Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 1067, pipe: [1:7559342050814704284:9144]
2025-10-09T21:47:55.649925Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:55.650252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:55.650270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:55.650324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342050814704284:9144], now have 1 active actors on pipe
2025-10-09T21:47:55.650574Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:55.650569Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342050814704284:9144]
2025-10-09T21:47:55.650595Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:55.650605Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:55.650608Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:55.650660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1068 reason: error at write: Disconnected sessionId: src_id|188017f6-d3ccc228-bc2313d6-1d581a69_0
2025-10-09T21:47:55.650818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1068 sessionId: src_id|188017f6-d3ccc228-bc2313d6-1d581a69_0 is DEAD
2025-10-09T21:47:55.650999Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|188017f6-d3ccc228-bc2313d6-1d581a69_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:55.651011Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|188017f6-d3ccc228-bc2313d6-1d581a69_0] Write session will restart in 0.009457s
2025-10-09T21:47:55.651038Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|188017f6-d3ccc228-bc2313d6-1d581a69_0] Write session: Do CDS request
2025-10-09T21:47:55.651045Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|188017f6-d3ccc228-bc2313d6-1d581a69_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:55.660350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:55.661510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|188017f6-d3ccc228-bc2313d6-1d581a69_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:55.661581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:55.661593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1069
2025-10-09T21:47:55.661741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1069 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:55.661773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1069 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:55.661779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1069 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:55.661783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:55.662223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:55.662252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1069 sessionId:  describe result for acl check
2025-10-09T21:47:55.662290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:55.662293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:55.662295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:55.662301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342050814704392:9543] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:55.662430Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:55.662443Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:55.662454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:55.662509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:55.662557Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342050452883293:8539]
2025-10-09T21:47:55.662857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342050814704392:9543] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:55.663097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:55.663340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:55.663372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:55.663567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:55.663596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.663600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.663601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.663602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.663602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.663603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.663603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.663604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.663604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.663608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.663614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.663616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.663618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.663620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.663622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.663624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.663626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.663628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.663632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.663638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.663642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.663645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.663647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.663650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.663652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.663655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.663658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.663661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.663664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.663666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.663669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.663673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.663675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.663678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.663681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.663684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.663699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:55.664702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342050814704392:9543] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:55.669222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342050814704392:9543] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:55.669232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342050814704392:9543] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:55.669235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342050814704392:9543] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:55.669242Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1069 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:55.669634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:55.670685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:55.670702Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:55.670704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:55.670709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:55.670715Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1068 [2:7559342050452883293:8539]
2025-10-09T21:47:55.670727Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:55.670746Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:55.670753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:55.670756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:55.670758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:55.670777Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:55.670787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:55.670798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:55.670855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.670861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.670863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.670866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:55.670930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:55.670937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.670950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.670966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.670986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:55.671012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342050814704286:2432], now have 1 active actors on pipe
2025-10-09T21:47:55.671020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342050814704431:9543], now have 1 active actors on pipe
2025-10-09T21:47:55.671022Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1068
2025-10-09T21:47:55.671027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342050452883288:8539]
2025-10-09T21:47:55.671035Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342050452883288:8539]
2025-10-09T21:47:55.671038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342050452883288:8539]
2025-10-09T21:47:55.671047Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:55.671061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:55.671067Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:55.671069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:55.671053Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1068
2025-10-09T21:47:55.671071Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:55.671076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:55.671079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:55.671124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:55.671133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:55.671145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342050452883288:8539]
2025-10-09T21:47:55.671157Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:55.671160Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:55.671188Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8acf84cd-7171ccda-2ac43fdf-2240b3b1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:55.671199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:55.671203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:55.671205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:55.671215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:55.671227Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:55.671251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:55.671349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:55.671356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:55.671370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:55.671432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1069 partition: 0 MaxSeqNo: 25 sessionId: src_id|8acf84cd-7171ccda-2ac43fdf-2240b3b1_0
2025-10-09T21:47:55.671693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|188017f6-d3ccc228-bc2313d6-1d581a69_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8acf84cd-7171ccda-2ac43fdf-2240b3b1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:55.673942Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8acf84cd-7171ccda-2ac43fdf-2240b3b1_0] Write session: try to update token
2025-10-09T21:47:55.673958Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8acf84cd-7171ccda-2ac43fdf-2240b3b1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:55.684550Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:55.684571Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:55.684573Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:55.684575Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:55.684588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342050452883288:8539]
2025-10-09T21:47:55.684593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342050452883288:8539]
2025-10-09T21:47:55.684597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342050452883288:8539]
2025-10-09T21:47:55.684641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:55.684645Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:55.684650Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:55.684654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.684654Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:55.684657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:55.684790Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:55.686059Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1069 sessionId: src_id|8acf84cd-7171ccda-2ac43fdf-2240b3b1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:55.689084Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:55.750911Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342050814704439:9144]
2025-10-09T21:47:55.751186Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 1068, pipe: [1:7559342050814704439:9144]
2025-10-09T21:47:55.841133Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:55.841433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:55.841451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:55.841498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342050814704439:9144], now have 1 active actors on pipe
2025-10-09T21:47:55.841720Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:55.841768Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:55.841802Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342050814704439:9144]
2025-10-09T21:47:55.841829Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:55.841909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1069 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8acf84cd-7171ccda-2ac43fdf-2240b3b1_0
2025-10-09T21:47:55.842115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1069 sessionId: src_id|8acf84cd-7171ccda-2ac43fdf-2240b3b1_0 is DEAD
2025-10-09T21:47:55.842285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8acf84cd-7171ccda-2ac43fdf-2240b3b1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:55.842297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8acf84cd-7171ccda-2ac43fdf-2240b3b1_0] Write session will restart in 0.005966s
2025-10-09T21:47:55.842325Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8acf84cd-7171ccda-2ac43fdf-2240b3b1_0] Write session: Do CDS request
2025-10-09T21:47:55.842331Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8acf84cd-7171ccda-2ac43fdf-2240b3b1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:55.849589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8acf84cd-7171ccda-2ac43fdf-2240b3b1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:55.849699Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:55.849716Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1070
2025-10-09T21:47:55.849898Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1070 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:55.849939Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1070 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:55.849945Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1070 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:55.849950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:55.850497Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1070 sessionId:  describe result for acl check
2025-10-09T21:47:55.850537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:55.850543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:55.850545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:55.850552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342050814704470:9548] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:55.851194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342050814704470:9548] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:55.853667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342050814704470:9548] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:55.855676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:55.857796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:55.857950Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342050814704470:9548] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:55.857966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342050814704470:9548] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:55.857970Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342050814704470:9548] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:55.857983Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1070 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:55.858063Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:55.858076Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:55.858091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:55.858169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:55.858212Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342050452883442:8541]
2025-10-09T21:47:55.858774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:55.859040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:55.859079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:55.859408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:55.859481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.859491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.859492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.859493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.859495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.859497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.859498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.859499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.859500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.859505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.859515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.859519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.859523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.859527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.859531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.859534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.859538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.859542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.859549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.859557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:55.859563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.859568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:55.859572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.859582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:55.859592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.859601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:55.859610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.859621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:55.859629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.859634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:55.859643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.859653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.859661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.859666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.859671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.859675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.859708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:55.866169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:55.867232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:55.867249Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:55.867251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:55.867255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:55.867262Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1069 [2:7559342050452883442:8541]
2025-10-09T21:47:55.867276Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:55.867293Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:55.867301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:55.867304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:55.867307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:55.867326Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:55.867337Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:55.867345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:55.867411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.867417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.867419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.867422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:55.867487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:55.867495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.867508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:55.867524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:55.867546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:55.867568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342050814704587:9548], now have 1 active actors on pipe
2025-10-09T21:47:55.867577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342050814704446:2432], now have 1 active actors on pipe
2025-10-09T21:47:55.867584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342050452883433:8541]
2025-10-09T21:47:55.867591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342050452883433:8541]
2025-10-09T21:47:55.867593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342050452883433:8541]
2025-10-09T21:47:55.867607Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:55.867621Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:55.867627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:55.867631Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:55.867633Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:55.867641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:55.867644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:55.867651Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1069
2025-10-09T21:47:55.867680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342050452883433:8541]
2025-10-09T21:47:55.867660Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1069
2025-10-09T21:47:55.867695Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:55.867697Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:55.867718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:55.867722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:55.867770Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|458ff1bb-d3564e1e-2ac2bd07-a0e1824a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:55.867782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:55.867790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:55.867793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:55.867803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:55.867817Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:55.867838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:55.867959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:55.867966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:55.867979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:55.868064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1070 partition: 0 MaxSeqNo: 25 sessionId: src_id|458ff1bb-d3564e1e-2ac2bd07-a0e1824a_0
2025-10-09T21:47:55.868423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8acf84cd-7171ccda-2ac43fdf-2240b3b1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|458ff1bb-d3564e1e-2ac2bd07-a0e1824a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:55.870846Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|458ff1bb-d3564e1e-2ac2bd07-a0e1824a_0] Write session: try to update token
2025-10-09T21:47:55.870863Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|458ff1bb-d3564e1e-2ac2bd07-a0e1824a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:55.881834Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:55.881853Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:55.881855Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:55.881856Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:55.881867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342050452883433:8541]
2025-10-09T21:47:55.881874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342050452883433:8541]
2025-10-09T21:47:55.881878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342050452883433:8541]
2025-10-09T21:47:55.881922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:55.881933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:55.881936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:55.881955Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:55.881970Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:55.881974Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:55.882472Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:55.899563Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1070 sessionId: src_id|458ff1bb-d3564e1e-2ac2bd07-a0e1824a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:55.908571Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:55.941886Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342050814704599:9144]
2025-10-09T21:47:55.942230Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 1069, pipe: [1:7559342050814704599:9144]
2025-10-09T21:47:56.038875Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:56.039227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.039246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.039274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342050814704599:9144], now have 1 active actors on pipe
2025-10-09T21:47:56.039538Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:56.039536Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342050814704599:9144]
2025-10-09T21:47:56.039565Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:56.039599Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:56.039681Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1070 reason: status is not ok: tablet will be restarted right now sessionId: src_id|458ff1bb-d3564e1e-2ac2bd07-a0e1824a_0
2025-10-09T21:47:56.039888Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1070 sessionId: src_id|458ff1bb-d3564e1e-2ac2bd07-a0e1824a_0 is DEAD
2025-10-09T21:47:56.040092Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|458ff1bb-d3564e1e-2ac2bd07-a0e1824a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:56.040105Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|458ff1bb-d3564e1e-2ac2bd07-a0e1824a_0] Write session will restart in 0.009284s
2025-10-09T21:47:56.040137Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|458ff1bb-d3564e1e-2ac2bd07-a0e1824a_0] Write session: Do CDS request
2025-10-09T21:47:56.040144Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|458ff1bb-d3564e1e-2ac2bd07-a0e1824a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:56.050560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|458ff1bb-d3564e1e-2ac2bd07-a0e1824a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:56.050650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:56.050680Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1071
2025-10-09T21:47:56.053658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:56.054277Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1071 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:56.054329Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1071 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:56.054348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1071 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:56.054357Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:56.054770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1071 sessionId:  describe result for acl check
2025-10-09T21:47:56.054803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:56.054810Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:56.054814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:56.054819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342055109672042:9559] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:56.055374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342055109672042:9559] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:56.056090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:56.056322Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:56.056333Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:56.056346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:56.056405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:56.056457Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342054747850909:8553]
2025-10-09T21:47:56.057087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:56.057410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:56.057448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342055109672042:9559] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:56.057469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:56.057739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:56.057816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.057823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.057825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.057826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.057827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.057828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.057830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.057831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.057832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.057838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.057848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.057852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.057855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.057858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.057862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.057865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.057869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.057872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.057878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.057887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.057893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.057898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.057902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.057907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.057911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.057916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.057920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.057924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.057928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.057934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.057939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.057945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.057950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.057955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.057959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.057964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.057993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:56.063581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342055109672042:9559] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:56.063597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342055109672042:9559] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:56.063601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342055109672042:9559] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:56.063611Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1071 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:56.065928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:56.067343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:56.067363Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:56.067365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:56.067370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:56.067377Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1070 [2:7559342054747850909:8553]
2025-10-09T21:47:56.067391Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:56.067410Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:56.067418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:56.067422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:56.067424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:56.067443Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:56.067454Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:56.067462Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:56.067530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.067537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.067539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.067542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:56.067629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:56.067640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.067654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.067665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.067684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:56.067708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342055109671923:2432], now have 1 active actors on pipe
2025-10-09T21:47:56.067739Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1070
2025-10-09T21:47:56.067783Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1070
2025-10-09T21:47:56.067814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342055109672079:9559], now have 1 active actors on pipe
2025-10-09T21:47:56.067830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342054747850904:8553]
2025-10-09T21:47:56.067833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342054747850904:8553]
2025-10-09T21:47:56.067835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342054747850904:8553]
2025-10-09T21:47:56.067851Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:56.067874Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:56.067874Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:56.067879Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:56.067881Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:56.067892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:56.067895Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:56.067946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342054747850904:8553]
2025-10-09T21:47:56.067964Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:56.067981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:56.067987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.067992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.068050Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b22b1962-8d62f05c-71a47c2c-a02ca8a9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:56.068063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:56.068073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:56.068076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:56.068088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:56.068103Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:56.068129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:56.068363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.068379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.068417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:56.068571Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1071 partition: 0 MaxSeqNo: 25 sessionId: src_id|b22b1962-8d62f05c-71a47c2c-a02ca8a9_0
2025-10-09T21:47:56.069001Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|458ff1bb-d3564e1e-2ac2bd07-a0e1824a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b22b1962-8d62f05c-71a47c2c-a02ca8a9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:56.075323Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b22b1962-8d62f05c-71a47c2c-a02ca8a9_0] Write session: try to update token
2025-10-09T21:47:56.075345Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b22b1962-8d62f05c-71a47c2c-a02ca8a9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:56.086293Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:56.086311Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:56.086313Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:56.086315Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:56.086324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342054747850904:8553]
2025-10-09T21:47:56.086327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342054747850904:8553]
2025-10-09T21:47:56.086330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342054747850904:8553]
2025-10-09T21:47:56.086375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:56.086375Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:56.086380Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:56.086382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.086384Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:56.086385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:56.086533Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:56.089033Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1071 sessionId: src_id|b22b1962-8d62f05c-71a47c2c-a02ca8a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:56.091768Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:56.139545Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342055109672100:9144]
2025-10-09T21:47:56.141045Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 1070, pipe: [1:7559342055109672100:9144]
2025-10-09T21:47:56.249285Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:56.249905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.249922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.249968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342055109672100:9144], now have 1 active actors on pipe
2025-10-09T21:47:56.250206Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:56.250213Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342055109672100:9144]
2025-10-09T21:47:56.250219Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:56.250237Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:56.250281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1071 reason: error at write: Disconnected sessionId: src_id|b22b1962-8d62f05c-71a47c2c-a02ca8a9_0
2025-10-09T21:47:56.250487Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1071 sessionId: src_id|b22b1962-8d62f05c-71a47c2c-a02ca8a9_0 is DEAD
2025-10-09T21:47:56.250636Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b22b1962-8d62f05c-71a47c2c-a02ca8a9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:56.250654Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b22b1962-8d62f05c-71a47c2c-a02ca8a9_0] Write session will restart in 0.005093s
2025-10-09T21:47:56.250632Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:56.250679Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b22b1962-8d62f05c-71a47c2c-a02ca8a9_0] Write session: Do CDS request
2025-10-09T21:47:56.250713Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b22b1962-8d62f05c-71a47c2c-a02ca8a9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:56.256583Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b22b1962-8d62f05c-71a47c2c-a02ca8a9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:56.256645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:56.256661Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1072
2025-10-09T21:47:56.256788Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1072 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:56.256834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1072 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:56.256841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1072 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:56.256846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:56.257326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1072 sessionId:  describe result for acl check
2025-10-09T21:47:56.257361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:56.257363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:56.257365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:56.257371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342055109672122:9565] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:56.257919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342055109672122:9565] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:56.260795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342055109672122:9565] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:56.263182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342055109672122:9565] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:56.263197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342055109672122:9565] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:56.263201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342055109672122:9565] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:56.263208Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1072 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:56.264406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:56.266258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:56.266503Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:56.266512Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:56.266533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:56.266585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:56.266628Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342054747851065:8557]
2025-10-09T21:47:56.267216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:56.267648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:56.267705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:56.268042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:56.268098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.268105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.268106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.268107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.268108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.268109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.268110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.268112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.268113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.268119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.268129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.268132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.268136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.268139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.268142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.268146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.268149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.268153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.268159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.268167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.268172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.268177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.268181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.268185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.268189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.268193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.268197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.268201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.268205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.268210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.268215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.268219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.268229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.268234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.268238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.268243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.268272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:56.274918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:56.276102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:56.276122Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:56.276124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:56.276130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:56.276138Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1071 [2:7559342054747851065:8557]
2025-10-09T21:47:56.276153Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:56.276173Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:56.276176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:56.276180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:56.276183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:56.276202Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:56.276214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:56.276222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:56.276292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.276303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.276305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.276308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:56.276395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:56.276405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.276417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.276428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.276445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:56.276468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342055109672107:2432], now have 1 active actors on pipe
2025-10-09T21:47:56.276477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342055109672249:9565], now have 1 active actors on pipe
2025-10-09T21:47:56.276486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342054747851060:8557]
2025-10-09T21:47:56.276494Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342054747851060:8557]
2025-10-09T21:47:56.276497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342054747851060:8557]
2025-10-09T21:47:56.276510Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:56.276526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:56.276535Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:56.276538Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:56.276540Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:56.276546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:56.276550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:56.276560Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1071
2025-10-09T21:47:56.276573Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1071
2025-10-09T21:47:56.276595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342054747851060:8557]
2025-10-09T21:47:56.276605Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:56.276607Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:56.276639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.276644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.276690Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4b8f692b-b6ca6b15-751ba15e-cd2f6bcb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:56.276703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:56.276707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:56.276714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:56.276725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:56.276741Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:56.276762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:56.276911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.276931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.276955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:56.277077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1072 partition: 0 MaxSeqNo: 25 sessionId: src_id|4b8f692b-b6ca6b15-751ba15e-cd2f6bcb_0
2025-10-09T21:47:56.277491Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b22b1962-8d62f05c-71a47c2c-a02ca8a9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4b8f692b-b6ca6b15-751ba15e-cd2f6bcb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:56.279963Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b8f692b-b6ca6b15-751ba15e-cd2f6bcb_0] Write session: try to update token
2025-10-09T21:47:56.279982Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b8f692b-b6ca6b15-751ba15e-cd2f6bcb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:56.291416Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:56.291437Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:56.291440Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:56.291442Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:56.291451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342054747851060:8557]
2025-10-09T21:47:56.291455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342054747851060:8557]
2025-10-09T21:47:56.291459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342054747851060:8557]
2025-10-09T21:47:56.291507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:56.291522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.291525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:56.291638Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:56.291652Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:56.291655Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:56.291662Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:56.294318Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1072 sessionId: src_id|4b8f692b-b6ca6b15-751ba15e-cd2f6bcb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:56.296681Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:56.350475Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342055109672263:9144]
2025-10-09T21:47:56.350847Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 1071, pipe: [1:7559342055109672263:9144]
2025-10-09T21:47:56.452200Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:56.452525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.452539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.452588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342055109672263:9144], now have 1 active actors on pipe
2025-10-09T21:47:56.452810Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:56.452832Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342055109672263:9144]
2025-10-09T21:47:56.452847Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:56.452876Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:56.452962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1072 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4b8f692b-b6ca6b15-751ba15e-cd2f6bcb_0
2025-10-09T21:47:56.453161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1072 sessionId: src_id|4b8f692b-b6ca6b15-751ba15e-cd2f6bcb_0 is DEAD
2025-10-09T21:47:56.453536Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b8f692b-b6ca6b15-751ba15e-cd2f6bcb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:56.453547Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b8f692b-b6ca6b15-751ba15e-cd2f6bcb_0] Write session will restart in 0.009537s
2025-10-09T21:47:56.453577Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b8f692b-b6ca6b15-751ba15e-cd2f6bcb_0] Write session: Do CDS request
2025-10-09T21:47:56.453583Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b8f692b-b6ca6b15-751ba15e-cd2f6bcb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:56.464649Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b8f692b-b6ca6b15-751ba15e-cd2f6bcb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:56.464705Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:56.464722Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1073
2025-10-09T21:47:56.464995Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1073 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:56.465036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1073 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:56.465045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1073 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:56.465049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:56.465496Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1073 sessionId:  describe result for acl check
2025-10-09T21:47:56.465539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:56.465547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:56.465549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:56.465556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342055109672376:9569] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:56.465760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:56.466181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342055109672376:9569] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:56.467883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:56.468116Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:56.468125Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:56.468137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:56.468200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:56.468156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342055109672376:9569] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:56.468241Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342054747851211:8559]
2025-10-09T21:47:56.468805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:56.469268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:56.469355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:56.469745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:56.469802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.469808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.469810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.469811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.469813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.469814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.469816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.469817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.469818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.469824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.469837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.469842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.469846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.469850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.469854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.469863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.469868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.469872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.469881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.469894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.469905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.469912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.469920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.469926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.469930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.469936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.469940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.469947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.469957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.469963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.469968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.469977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.469982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.469991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.469996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.470005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.470051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:56.472889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342055109672376:9569] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:56.472908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342055109672376:9569] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:56.472912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342055109672376:9569] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:56.472922Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1073 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:56.476909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:56.478189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:56.478206Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:56.478208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:56.478212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:56.478219Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1072 [2:7559342054747851211:8559]
2025-10-09T21:47:56.478229Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:56.478248Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:56.478252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:56.478256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:56.478259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:56.478277Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:56.478286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:56.478292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:56.478341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.478347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.478348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.478350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:56.478408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:56.478414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.478422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.478429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.478443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:56.478465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342054747851206:8559]
2025-10-09T21:47:56.478472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342054747851206:8559]
2025-10-09T21:47:56.478474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342054747851206:8559]
2025-10-09T21:47:56.478488Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:56.478501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:56.478512Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:56.478514Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:56.478517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342055109672414:9569], now have 1 active actors on pipe
2025-10-09T21:47:56.478521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342055109672265:2432], now have 1 active actors on pipe
2025-10-09T21:47:56.478527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:56.478542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:56.478550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:56.478595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342054747851206:8559]
2025-10-09T21:47:56.478603Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:56.478575Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1072
2025-10-09T21:47:56.478617Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:56.478576Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1072
2025-10-09T21:47:56.478650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.478660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.478691Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|155486f9-d85b3148-399edfb7-edfef1c3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:56.478706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:56.478710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:56.478713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:56.478724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:56.478735Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:56.478751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:56.478892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.478901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.478926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:56.479035Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1073 partition: 0 MaxSeqNo: 25 sessionId: src_id|155486f9-d85b3148-399edfb7-edfef1c3_0
2025-10-09T21:47:56.479415Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b8f692b-b6ca6b15-751ba15e-cd2f6bcb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|155486f9-d85b3148-399edfb7-edfef1c3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:56.482459Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|155486f9-d85b3148-399edfb7-edfef1c3_0] Write session: try to update token
2025-10-09T21:47:56.482477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|155486f9-d85b3148-399edfb7-edfef1c3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:56.491275Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:56.491296Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:56.491308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342054747851206:8559]
2025-10-09T21:47:56.491363Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:56.491364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:56.491567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:56.491732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:56.492047Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:56.492283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:56.492651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:56.492710Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:56.497750Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1073 sessionId: src_id|155486f9-d85b3148-399edfb7-edfef1c3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:56.500442Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:56.500683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.500707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.500752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:56.500868Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:56.506777Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:56.506798Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:56.506801Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:56.506804Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:56.506816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342054747851206:8559]
2025-10-09T21:47:56.506821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342054747851206:8559]
2025-10-09T21:47:56.506825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342054747851206:8559]
2025-10-09T21:47:56.506871Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:56.506878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:56.506879Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:56.506883Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:56.506885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.506888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:56.507048Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:56.553231Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342055109672422:9144]
2025-10-09T21:47:56.553550Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 1072, pipe: [1:7559342055109672422:9144]
2025-10-09T21:47:56.669297Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:56.677690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.677726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.677910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.677985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.678075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.678145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.678196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.678207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.678280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.678346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.678417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.678483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.678525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.678532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.678593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.678652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.678722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.678783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.678828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.678833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.678895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.678960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.679031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.679099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.679149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.679158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.679226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.679288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.679357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.679422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.679467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.679475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.679538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.679602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.679664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.679726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.679774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.679782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.679846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.679906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.679971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.680035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.680182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.680192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.680254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.680314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.680383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.680448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.680492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.680495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.680559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.680623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.680681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.680742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.680783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.680788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.680845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.680900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.680967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.681025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.681067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.681072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.681130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342055109672422:9144], now have 1 active actors on pipe
2025-10-09T21:47:56.681387Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:56.681398Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:56.681402Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:56.681429Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1073 reason: error at write: Disconnected sessionId: src_id|155486f9-d85b3148-399edfb7-edfef1c3_0
2025-10-09T21:47:56.681446Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342055109672422:9144]
2025-10-09T21:47:56.681595Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1073 sessionId: src_id|155486f9-d85b3148-399edfb7-edfef1c3_0 is DEAD
2025-10-09T21:47:56.681728Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|155486f9-d85b3148-399edfb7-edfef1c3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:56.681740Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|155486f9-d85b3148-399edfb7-edfef1c3_0] Write session will restart in 0.006573s
2025-10-09T21:47:56.681764Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|155486f9-d85b3148-399edfb7-edfef1c3_0] Write session: Do CDS request
2025-10-09T21:47:56.681771Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|155486f9-d85b3148-399edfb7-edfef1c3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:56.682105Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:56.689492Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|155486f9-d85b3148-399edfb7-edfef1c3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:56.689907Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:56.689919Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1074
2025-10-09T21:47:56.690065Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1074 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:56.690101Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1074 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:56.690107Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1074 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:56.690111Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:56.690549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1074 sessionId:  describe result for acl check
2025-10-09T21:47:56.690590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:56.690596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:56.690598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:56.690603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342055109672483:9573] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:56.691045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342055109672483:9573] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:56.693281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:56.693924Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342055109672483:9573] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:56.695382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:56.695606Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:56.695619Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:56.695631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:56.695686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:56.695728Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342054747851364:8563]
2025-10-09T21:47:56.696108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342055109672483:9573] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:56.696122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342055109672483:9573] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:56.696127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342055109672483:9573] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:56.696135Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1074 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:56.696324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:56.696636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:56.696681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:56.697025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:56.697072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.697079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.697081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.697082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.697083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.697084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.697085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.697086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.697087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.697092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.697103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.697107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.697111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.697114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.697118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.697121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.697125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.697129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.697136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.697144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.697150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.697155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.697160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.697165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.697170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.697176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.697181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.697186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.697195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.697201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.697206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.697216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.697220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.697225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.697235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.697240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.697275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:56.704022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:56.705204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:56.705225Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:56.705228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:56.705233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:56.705241Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1073 [2:7559342054747851364:8563]
2025-10-09T21:47:56.705257Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:56.705276Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:56.705285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:56.705289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:56.705291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:56.705311Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:56.705326Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:56.705334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:56.705387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.705393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.705394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.705396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:56.705463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:56.705473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.705487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.705498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.705514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:56.705541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342055109672425:2432], now have 1 active actors on pipe
2025-10-09T21:47:56.705554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342054747851359:8563]
2025-10-09T21:47:56.705561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342054747851359:8563]
2025-10-09T21:47:56.705564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342054747851359:8563]
2025-10-09T21:47:56.705576Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:56.705587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342055109672571:9573], now have 1 active actors on pipe
2025-10-09T21:47:56.705591Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:56.705595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:56.705602Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:56.705605Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:56.705611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:56.705614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:56.705651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342054747851359:8563]
2025-10-09T21:47:56.705660Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:56.705662Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:56.705636Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1073
2025-10-09T21:47:56.705659Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1073
2025-10-09T21:47:56.705703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.705713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.705740Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|23bbe8dd-65787626-9fb0b197-bcbfc9be_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:56.705752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:56.705760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:56.705763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:56.705773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:56.705800Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:56.705815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:56.706078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.706093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.706127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:56.706258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1074 partition: 0 MaxSeqNo: 25 sessionId: src_id|23bbe8dd-65787626-9fb0b197-bcbfc9be_0
2025-10-09T21:47:56.706659Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|155486f9-d85b3148-399edfb7-edfef1c3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|23bbe8dd-65787626-9fb0b197-bcbfc9be_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:56.709113Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23bbe8dd-65787626-9fb0b197-bcbfc9be_0] Write session: try to update token
2025-10-09T21:47:56.709131Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23bbe8dd-65787626-9fb0b197-bcbfc9be_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:56.717625Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:56.717643Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:56.717654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342054747851359:8563]
2025-10-09T21:47:56.717703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:56.717712Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:56.717911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:56.718079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:56.718223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:56.718364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:56.718562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:56.718600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:56.722592Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1074 sessionId: src_id|23bbe8dd-65787626-9fb0b197-bcbfc9be_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:56.725191Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:56.725425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.725448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.725492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:56.725580Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:56.731511Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:56.731530Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:56.731532Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:56.731534Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:56.731546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342054747851359:8563]
2025-10-09T21:47:56.731550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342054747851359:8563]
2025-10-09T21:47:56.731554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342054747851359:8563]
2025-10-09T21:47:56.731602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:56.731615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.731616Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:56.731618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:56.731622Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:56.731628Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:56.731754Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:56.781576Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342055109672585:9144]
2025-10-09T21:47:56.781899Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 1073, pipe: [1:7559342055109672585:9144]
2025-10-09T21:47:56.889985Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:56.897918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.897945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.898124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.898186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.898253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.898317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.898370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.898384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.898457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.898525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.901424Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342055109672585:9144]
2025-10-09T21:47:56.901430Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:56.901466Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:56.901469Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:56.901499Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1074 reason: error at write: Disconnected sessionId: src_id|23bbe8dd-65787626-9fb0b197-bcbfc9be_0
2025-10-09T21:47:56.901687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1074 sessionId: src_id|23bbe8dd-65787626-9fb0b197-bcbfc9be_0 is DEAD
2025-10-09T21:47:56.901876Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23bbe8dd-65787626-9fb0b197-bcbfc9be_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:56.901888Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23bbe8dd-65787626-9fb0b197-bcbfc9be_0] Write session will restart in 0.005953s
2025-10-09T21:47:56.901912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23bbe8dd-65787626-9fb0b197-bcbfc9be_0] Write session: Do CDS request
2025-10-09T21:47:56.901934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23bbe8dd-65787626-9fb0b197-bcbfc9be_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:56.898592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.898659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.898703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.898706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.898771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.898824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.898889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.898946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.898990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.898993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.899063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.899121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.899177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.899237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.899277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.899280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.899348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.899406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.899541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.899686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.899725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.899728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.899794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.899854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.899913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.899966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.900009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.900011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.900094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.900148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.900203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.900259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.900303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.900306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.900370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.900421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.900474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.900528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.900567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.900570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.900631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.900688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.900741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.900797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.900833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.900836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.900900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:56.900953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:56.901011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:56.901069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:56.901111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:56.901113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:56.901181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342055109672585:9144], now have 1 active actors on pipe
2025-10-09T21:47:56.902392Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:56.909544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23bbe8dd-65787626-9fb0b197-bcbfc9be_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:56.909613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:56.909630Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1075
2025-10-09T21:47:56.909893Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1075 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:56.909928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1075 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:56.909938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1075 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:56.909942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:56.910456Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1075 sessionId:  describe result for acl check
2025-10-09T21:47:56.910495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:56.910497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:56.910499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:56.910505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342055109672635:9580] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:56.911121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342055109672635:9580] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:56.914911Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342055109672635:9580] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:56.915360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:56.918693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342055109672635:9580] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:56.918716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342055109672635:9580] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:56.918719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342055109672635:9580] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:56.918727Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1075 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:56.918225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:56.918582Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:56.918587Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:56.918602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:56.918670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:56.918708Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342054747851526:8567]
2025-10-09T21:47:56.919260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:56.919602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:56.919658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:56.919983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:56.920058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.920064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.920065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.920066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.920067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.920069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.920070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.920071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.920072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.920078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.920088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.920093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.920097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.920101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.920118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.920122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.920126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.920130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.920137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.920145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:56.920152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.920157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:56.920161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.920166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:56.920170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.920175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:56.920180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.920185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:56.920189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.920194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:56.920199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.920204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.920216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.920221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.920225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.920230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.920256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:56.927000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:56.928221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:56.928238Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:56.928241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:56.928246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:56.928255Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1074 [2:7559342054747851526:8567]
2025-10-09T21:47:56.928270Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:56.928289Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:56.928292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:56.928295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:56.928297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:56.928315Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:56.928320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:56.928327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:56.928386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.928388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.928390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.928393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:56.928468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:56.928471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.928482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:56.928492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:56.928506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:56.928527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342054747851521:8567]
2025-10-09T21:47:56.928530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342054747851521:8567]
2025-10-09T21:47:56.928531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342054747851521:8567]
2025-10-09T21:47:56.928540Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:56.928555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:56.928569Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:56.928572Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:56.928581Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:56.928584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:56.928586Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:56.928614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342054747851521:8567]
2025-10-09T21:47:56.928617Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:56.928619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:56.928628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342055109672738:9580], now have 1 active actors on pipe
2025-10-09T21:47:56.928631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342055109672591:2432], now have 1 active actors on pipe
2025-10-09T21:47:56.928694Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1074
2025-10-09T21:47:56.928826Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1074
2025-10-09T21:47:56.928848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.928863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.928890Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5b592d6e-ef903f6c-3e436625-3e9552e5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:56.928903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:56.928912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:56.928914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:56.928927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:56.928939Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:56.928960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:56.929146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:56.929153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:56.929166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:56.929261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1075 partition: 0 MaxSeqNo: 25 sessionId: src_id|5b592d6e-ef903f6c-3e436625-3e9552e5_0
2025-10-09T21:47:56.929634Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23bbe8dd-65787626-9fb0b197-bcbfc9be_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5b592d6e-ef903f6c-3e436625-3e9552e5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:56.932202Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b592d6e-ef903f6c-3e436625-3e9552e5_0] Write session: try to update token
2025-10-09T21:47:56.932221Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b592d6e-ef903f6c-3e436625-3e9552e5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:56.942791Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:56.942814Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:56.942816Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:56.942818Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:56.942827Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342054747851521:8567]
2025-10-09T21:47:56.942830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342054747851521:8567]
2025-10-09T21:47:56.942832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342054747851521:8567]
2025-10-09T21:47:56.942872Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:56.942873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:56.942880Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:56.942882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:56.942885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:56.942886Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:56.943393Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:56.955906Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1075 sessionId: src_id|5b592d6e-ef903f6c-3e436625-3e9552e5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:56.958241Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:57.001766Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342059404640046:9144]
2025-10-09T21:47:57.002136Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 1074, pipe: [1:7559342059404640046:9144]
2025-10-09T21:47:57.103179Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:57.103559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:57.103576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:57.103600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342059404640046:9144], now have 1 active actors on pipe
2025-10-09T21:47:57.103865Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:57.103905Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:57.103918Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:57.103991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1075 reason: error at write: Disconnected sessionId: src_id|5b592d6e-ef903f6c-3e436625-3e9552e5_0
2025-10-09T21:47:57.104202Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1075 sessionId: src_id|5b592d6e-ef903f6c-3e436625-3e9552e5_0 is DEAD
2025-10-09T21:47:57.104398Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342059404640046:9144]
2025-10-09T21:47:57.104431Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:57.104527Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b592d6e-ef903f6c-3e436625-3e9552e5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:57.104538Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b592d6e-ef903f6c-3e436625-3e9552e5_0] Write session will restart in 0.008370s
2025-10-09T21:47:57.104569Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b592d6e-ef903f6c-3e436625-3e9552e5_0] Write session: Do CDS request
2025-10-09T21:47:57.104576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b592d6e-ef903f6c-3e436625-3e9552e5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:57.114763Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b592d6e-ef903f6c-3e436625-3e9552e5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:57.114837Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:57.114854Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1076
2025-10-09T21:47:57.115025Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1076 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:57.115059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1076 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:57.115069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1076 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:57.115075Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:57.115612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1076 sessionId:  describe result for acl check
2025-10-09T21:47:57.115647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:57.115649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:57.115651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:57.115657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342059404640109:9590] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:57.116205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342059404640109:9590] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:57.118664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342059404640109:9590] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:57.119992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:57.124304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:57.124568Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:57.124581Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:57.124596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:57.124668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:57.124730Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342059042818989:8577]
2025-10-09T21:47:57.124787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342059404640109:9590] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:57.124804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342059404640109:9590] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:57.124809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342059404640109:9590] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:57.124818Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1076 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:57.125336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:57.125643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:57.125693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:57.126063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:57.126119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.126126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.126127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.126128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.126130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.126131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.126132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.126134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.126135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.126142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.126153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.126157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.126161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.126165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.126169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.126172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.126176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.126179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.126186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.126194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.126200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.126205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.126209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.126228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.126232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.126237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.126241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.126246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.126251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.126265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.126270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.126275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.126279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.126284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.126290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.126295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.126321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:57.133130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:57.134700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:57.134722Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:57.134725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:57.134729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:57.134736Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1075 [2:7559342059042818989:8577]
2025-10-09T21:47:57.134750Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:57.134767Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:57.134771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:57.134774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:57.134777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:57.134795Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:57.134807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:57.134815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:57.134878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.134886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.134888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.134891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:57.134978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:57.134986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.135000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.135017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.135038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:57.135066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342059404640211:9590], now have 1 active actors on pipe
2025-10-09T21:47:57.135074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342059404640061:2432], now have 1 active actors on pipe
2025-10-09T21:47:57.135082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342059042818984:8577]
2025-10-09T21:47:57.135085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342059042818984:8577]
2025-10-09T21:47:57.135087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342059042818984:8577]
2025-10-09T21:47:57.135103Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:57.135125Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:57.135141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:57.135149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:57.135151Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:57.135155Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:57.135158Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:57.135163Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1075
2025-10-09T21:47:57.135208Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1075
2025-10-09T21:47:57.135187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342059042818984:8577]
2025-10-09T21:47:57.135202Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:57.135246Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:57.135357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:57.135372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:57.135411Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b1e0101a-3301dfc6-997814a9-8e688d85_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:57.135427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:57.135436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:57.135439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:57.135452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:57.135469Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:57.135490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:57.135628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:57.135636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:57.135664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:57.135755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1076 partition: 0 MaxSeqNo: 25 sessionId: src_id|b1e0101a-3301dfc6-997814a9-8e688d85_0
2025-10-09T21:47:57.136115Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b592d6e-ef903f6c-3e436625-3e9552e5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b1e0101a-3301dfc6-997814a9-8e688d85_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:57.138661Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1e0101a-3301dfc6-997814a9-8e688d85_0] Write session: try to update token
2025-10-09T21:47:57.138681Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1e0101a-3301dfc6-997814a9-8e688d85_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:57.148529Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:57.148551Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:57.148561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342059042818984:8577]
2025-10-09T21:47:57.148610Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:57.148620Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:57.148790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:57.148944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:57.149090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:57.149241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:57.149454Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:57.149501Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:57.153694Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1076 sessionId: src_id|b1e0101a-3301dfc6-997814a9-8e688d85_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:57.156268Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:57.156487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:57.156514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:57.156557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:57.156634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:57.162484Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:57.162500Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:57.162503Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:57.162505Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:57.162516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342059042818984:8577]
2025-10-09T21:47:57.162520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342059042818984:8577]
2025-10-09T21:47:57.162524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342059042818984:8577]
2025-10-09T21:47:57.162571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:57.162571Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:57.162578Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:57.162580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.162583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:57.162585Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:57.162720Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:57.204474Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342059404640238:9144]
2025-10-09T21:47:57.209420Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 1075, pipe: [1:7559342059404640238:9144]
2025-10-09T21:47:57.323871Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:57.332282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:57.332319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:57.332498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:57.332563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:57.332625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:57.332688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:57.332738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:57.332748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:57.332813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:57.332871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:57.332939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:57.333004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:57.333145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:57.333155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:57.333220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:57.333310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:57.333431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:57.333533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:57.333585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:57.333592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:57.333658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:57.333726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:57.333807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:57.333917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:57.334373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:57.334388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:57.337188Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:57.337200Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:57.337201Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:57.337233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1076 reason: error at write: Disconnected sessionId: src_id|b1e0101a-3301dfc6-997814a9-8e688d85_0
2025-10-09T21:47:57.337240Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342059404640238:9144]
2025-10-09T21:47:57.337458Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1076 sessionId: src_id|b1e0101a-3301dfc6-997814a9-8e688d85_0 is DEAD
2025-10-09T21:47:57.337637Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1e0101a-3301dfc6-997814a9-8e688d85_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:57.337648Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1e0101a-3301dfc6-997814a9-8e688d85_0] Write session will restart in 0.005618s
2025-10-09T21:47:57.337674Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1e0101a-3301dfc6-997814a9-8e688d85_0] Write session: Do CDS request
2025-10-09T21:47:57.337680Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1e0101a-3301dfc6-997814a9-8e688d85_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:57.334494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:57.334636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:57.334737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:57.334801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:57.334859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:57.334862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:57.334929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:57.334983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:57.335038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:57.335097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:57.335158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:57.335161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:57.335230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:57.335285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:57.335343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:57.335444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:57.335523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:57.335526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:57.335632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:57.335728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:57.335787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:57.335845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:57.335883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:57.335887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:57.335952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:57.336048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:57.336153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:57.336273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:57.336334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:57.336337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:57.336434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:57.336527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:57.336624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:57.336825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:57.336880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:57.336883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:57.336960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342059404640238:9144], now have 1 active actors on pipe
2025-10-09T21:47:57.338400Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:57.344573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1e0101a-3301dfc6-997814a9-8e688d85_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:57.344728Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:57.344745Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1077
2025-10-09T21:47:57.344875Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1077 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:57.344910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1077 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:57.344922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1077 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:57.344927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:57.345420Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1077 sessionId:  describe result for acl check
2025-10-09T21:47:57.345463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:57.345466Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:57.345470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:57.345476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342059404640254:9596] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:57.346111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342059404640254:9596] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:57.348981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342059404640254:9596] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:57.352245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342059404640254:9596] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:57.352262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342059404640254:9596] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:57.352266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342059404640254:9596] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:57.352276Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1077 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:57.353013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:57.355298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:57.355633Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:57.355646Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:57.355660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:57.355742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:57.355788Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342059042819150:8581]
2025-10-09T21:47:57.356398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:57.356724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:57.356782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:57.357112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:57.357168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.357175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.357176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.357177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.357179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.357180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.357181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.357182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.357184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.357189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.357202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.357206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.357211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.357216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.357221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.357224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.357227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.357230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.357238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.357246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.357253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.357266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.357270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.357275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.357279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.357283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.357288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.357293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.357298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.357303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.357308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.357313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.357317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.357323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.357327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.357332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.357361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:57.364367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:57.365505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:57.365529Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:57.365532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:57.365537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:57.365546Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1076 [2:7559342059042819150:8581]
2025-10-09T21:47:57.365563Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:57.365586Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:57.365590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:57.365593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:57.365597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:57.365616Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:57.365628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:57.365636Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:57.365708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.365715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.365717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.365720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:57.365813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:57.365826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.365839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.365849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.365863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:57.365888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342059404640388:9596], now have 1 active actors on pipe
2025-10-09T21:47:57.365897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342059404640241:2432], now have 1 active actors on pipe
2025-10-09T21:47:57.365905Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342059042819145:8581]
2025-10-09T21:47:57.365912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342059042819145:8581]
2025-10-09T21:47:57.365914Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342059042819145:8581]
2025-10-09T21:47:57.365932Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:57.365951Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:57.365966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:57.365967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:57.365970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:57.365972Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:57.365975Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:57.365990Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1076
2025-10-09T21:47:57.365991Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1076
2025-10-09T21:47:57.366013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342059042819145:8581]
2025-10-09T21:47:57.366019Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:57.366022Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:57.366096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:57.366110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:57.366154Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|74192ad2-921cd4f7-a6ef6088-3a8e44c2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:57.366169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:57.366179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:57.366183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:57.366195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:57.366207Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:57.366235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:57.366553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:57.366570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:57.366595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:57.366822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1077 partition: 0 MaxSeqNo: 25 sessionId: src_id|74192ad2-921cd4f7-a6ef6088-3a8e44c2_0
2025-10-09T21:47:57.367262Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1e0101a-3301dfc6-997814a9-8e688d85_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|74192ad2-921cd4f7-a6ef6088-3a8e44c2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:57.370218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74192ad2-921cd4f7-a6ef6088-3a8e44c2_0] Write session: try to update token
2025-10-09T21:47:57.370248Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74192ad2-921cd4f7-a6ef6088-3a8e44c2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:57.382300Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:57.382323Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:57.382326Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:57.382329Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:57.382339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342059042819145:8581]
2025-10-09T21:47:57.382344Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342059042819145:8581]
2025-10-09T21:47:57.382348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342059042819145:8581]
2025-10-09T21:47:57.382401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:57.382409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.382409Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:57.382412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:57.382415Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:57.382418Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:57.382572Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:57.388260Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1077 sessionId: src_id|74192ad2-921cd4f7-a6ef6088-3a8e44c2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:57.390857Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:57.437458Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342059404640401:9144]
2025-10-09T21:47:57.437869Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 1076, pipe: [1:7559342059404640401:9144]
2025-10-09T21:47:57.540834Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:57.541347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:57.541362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:57.541382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342059404640401:9144], now have 1 active actors on pipe
2025-10-09T21:47:57.541614Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:57.541665Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342059404640401:9144]
2025-10-09T21:47:57.541696Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:57.541760Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:57.541856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1077 reason: status is not ok: tablet will be restarted right now sessionId: src_id|74192ad2-921cd4f7-a6ef6088-3a8e44c2_0
2025-10-09T21:47:57.542058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1077 sessionId: src_id|74192ad2-921cd4f7-a6ef6088-3a8e44c2_0 is DEAD
2025-10-09T21:47:57.542285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74192ad2-921cd4f7-a6ef6088-3a8e44c2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:57.542297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74192ad2-921cd4f7-a6ef6088-3a8e44c2_0] Write session will restart in 0.008502s
2025-10-09T21:47:57.542325Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74192ad2-921cd4f7-a6ef6088-3a8e44c2_0] Write session: Do CDS request
2025-10-09T21:47:57.542331Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74192ad2-921cd4f7-a6ef6088-3a8e44c2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:57.551589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74192ad2-921cd4f7-a6ef6088-3a8e44c2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:57.551697Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:57.551714Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1078
2025-10-09T21:47:57.551852Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1078 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:57.551887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1078 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:57.551899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1078 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:57.551903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:57.552357Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1078 sessionId:  describe result for acl check
2025-10-09T21:47:57.552393Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:57.552399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:57.552400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:57.552408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342059404640510:9600] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:57.552949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342059404640510:9600] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:57.554338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:57.555267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342059404640510:9600] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:57.556323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:57.556535Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:57.556546Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:57.556559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:57.556620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:57.556661Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342059042819301:8583]
2025-10-09T21:47:57.557153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:57.557470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:57.557535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:57.557873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:57.557919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.557925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.557926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.557927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.557928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.557929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.557910Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342059404640510:9600] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:57.557925Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342059404640510:9600] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:57.557930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342059404640510:9600] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:57.557939Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1078 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:57.557931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.557932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.557933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.557937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.557947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.557951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.557954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.557958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.557962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.557965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.557969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.557972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.557979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.557987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.557992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.557997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.558002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.558007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.558012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.558017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.558021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.558043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.558049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.558054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.558058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.558063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.558068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.558074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.558078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.558083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.558112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:57.564312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:57.565356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:57.565369Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:57.565372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:57.565377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:57.565384Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1077 [2:7559342059042819301:8583]
2025-10-09T21:47:57.565397Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:57.565417Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:57.565420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:57.565424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:57.565426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:57.565445Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:57.565458Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:57.565471Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:57.565531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.565539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.565541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.565544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:57.565623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:57.565632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.565644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.565656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.565678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:57.565704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342059404640404:2432], now have 1 active actors on pipe
2025-10-09T21:47:57.565712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342059404640556:9600], now have 1 active actors on pipe
2025-10-09T21:47:57.565719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342059042819296:8583]
2025-10-09T21:47:57.565722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342059042819296:8583]
2025-10-09T21:47:57.565724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342059042819296:8583]
2025-10-09T21:47:57.565737Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:57.565751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:57.565770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:57.565774Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:57.565777Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:57.565778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:57.565779Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:57.565773Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1077
2025-10-09T21:47:57.565775Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1077
2025-10-09T21:47:57.565830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342059042819296:8583]
2025-10-09T21:47:57.565848Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:57.565868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:57.565869Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:57.565873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:57.565922Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d4bf4a51-cedfa5ec-347b1199-fd04e0d8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:57.565936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:57.565940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:57.565943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:57.565953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:57.565964Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:57.565983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:57.566206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:57.566214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:57.566228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:57.566316Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1078 partition: 0 MaxSeqNo: 25 sessionId: src_id|d4bf4a51-cedfa5ec-347b1199-fd04e0d8_0
2025-10-09T21:47:57.566696Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74192ad2-921cd4f7-a6ef6088-3a8e44c2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d4bf4a51-cedfa5ec-347b1199-fd04e0d8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:57.569231Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d4bf4a51-cedfa5ec-347b1199-fd04e0d8_0] Write session: try to update token
2025-10-09T21:47:57.569249Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d4bf4a51-cedfa5ec-347b1199-fd04e0d8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:57.579812Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:57.579835Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:57.579838Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:57.579840Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:57.579850Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342059042819296:8583]
2025-10-09T21:47:57.579854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342059042819296:8583]
2025-10-09T21:47:57.579858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342059042819296:8583]
2025-10-09T21:47:57.579904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:57.579916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.579919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:57.579920Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:57.579949Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:57.579968Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:57.580061Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:57.584341Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1078 sessionId: src_id|d4bf4a51-cedfa5ec-347b1199-fd04e0d8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:57.586790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:57.641735Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342059404640566:9144]
2025-10-09T21:47:57.642254Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 1077, pipe: [1:7559342059404640566:9144]
2025-10-09T21:47:57.734715Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:57.735028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:57.735047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:57.735097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342059404640566:9144], now have 1 active actors on pipe
2025-10-09T21:47:57.735337Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:57.735365Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:57.735381Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342059404640566:9144]
2025-10-09T21:47:57.735405Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:57.735489Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1078 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d4bf4a51-cedfa5ec-347b1199-fd04e0d8_0
2025-10-09T21:47:57.735691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1078 sessionId: src_id|d4bf4a51-cedfa5ec-347b1199-fd04e0d8_0 is DEAD
2025-10-09T21:47:57.735831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4bf4a51-cedfa5ec-347b1199-fd04e0d8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:57.735840Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4bf4a51-cedfa5ec-347b1199-fd04e0d8_0] Write session will restart in 0.006212s
2025-10-09T21:47:57.735869Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4bf4a51-cedfa5ec-347b1199-fd04e0d8_0] Write session: Do CDS request
2025-10-09T21:47:57.735875Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4bf4a51-cedfa5ec-347b1199-fd04e0d8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:57.743619Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d4bf4a51-cedfa5ec-347b1199-fd04e0d8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:57.743707Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:57.743723Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1079
2025-10-09T21:47:57.743868Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1079 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:57.743915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1079 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:57.743927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1079 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:57.743931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:57.744457Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1079 sessionId:  describe result for acl check
2025-10-09T21:47:57.744495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:57.744497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:57.744498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:57.744504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342059404640583:9604] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:57.745066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342059404640583:9604] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:57.747799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342059404640583:9604] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:57.751094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342059404640583:9604] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:57.751112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342059404640583:9604] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:57.751117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342059404640583:9604] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:57.751126Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1079 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:57.751635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:57.754014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:57.754290Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:57.754295Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:57.754307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:57.754364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:57.754416Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342059042819459:8587]
2025-10-09T21:47:57.755031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:57.755394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:57.755458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:57.755783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:57.755841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.755852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.755853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.755855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.755856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.755858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.755859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.755860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.755861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.755867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.755879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.755883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.755887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.755892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.755896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.755900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.755904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.755908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.755918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.755926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.755936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.755942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.755947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.755952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.755957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.755962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.755967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.755973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.755977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.755982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.755989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.755994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.755998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.756004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.756009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.756014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.756043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:57.762911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:57.764068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:57.764088Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:57.764091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:57.764096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:57.764103Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1078 [2:7559342059042819459:8587]
2025-10-09T21:47:57.764119Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:57.764139Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:57.764143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:57.764147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:57.764149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:57.764168Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:57.764180Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:57.764188Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:57.764266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.764274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.764276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.764279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:57.764355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:57.764363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.764380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.764397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.764417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:57.764442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342059042819454:8587]
2025-10-09T21:47:57.764450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342059042819454:8587]
2025-10-09T21:47:57.764452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342059042819454:8587]
2025-10-09T21:47:57.764465Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:57.764478Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:57.764482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342059404640718:9604], now have 1 active actors on pipe
2025-10-09T21:47:57.764486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342059404640568:2432], now have 1 active actors on pipe
2025-10-09T21:47:57.764496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:57.764502Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:57.764505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:57.764511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:57.764515Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:57.764555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342059042819454:8587]
2025-10-09T21:47:57.764565Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:57.764567Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1078
2025-10-09T21:47:57.764570Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1078
2025-10-09T21:47:57.764602Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:57.764660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:57.764673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:57.764716Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d58de45b-f5d2ec4e-c68601f7-95e99623_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:57.764729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:57.764733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:57.764736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:57.764748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:57.764764Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:57.764780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:57.764943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:57.764951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:57.764964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:57.765062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1079 partition: 0 MaxSeqNo: 25 sessionId: src_id|d58de45b-f5d2ec4e-c68601f7-95e99623_0
2025-10-09T21:47:57.765467Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4bf4a51-cedfa5ec-347b1199-fd04e0d8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d58de45b-f5d2ec4e-c68601f7-95e99623_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:57.767950Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d58de45b-f5d2ec4e-c68601f7-95e99623_0] Write session: try to update token
2025-10-09T21:47:57.767970Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d58de45b-f5d2ec4e-c68601f7-95e99623_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:57.779623Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:57.779650Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:57.779653Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:57.779655Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:57.779665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342059042819454:8587]
2025-10-09T21:47:57.779670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342059042819454:8587]
2025-10-09T21:47:57.779675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342059042819454:8587]
2025-10-09T21:47:57.779728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:57.779727Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:57.779734Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:57.779736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.779738Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:57.779738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:57.780270Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:57.786768Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1079 sessionId: src_id|d58de45b-f5d2ec4e-c68601f7-95e99623_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:57.789137Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:57.835778Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342059404640733:9144]
2025-10-09T21:47:57.836544Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 1078, pipe: [1:7559342059404640733:9144]
2025-10-09T21:47:57.946133Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:57.946472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:57.946488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:57.946513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342059404640733:9144], now have 1 active actors on pipe
2025-10-09T21:47:57.946744Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:57.946808Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:57.946827Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:57.946852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342059404640733:9144]
2025-10-09T21:47:57.946891Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:57.946896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1079 reason: error at write: Disconnected sessionId: src_id|d58de45b-f5d2ec4e-c68601f7-95e99623_0
2025-10-09T21:47:57.947108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1079 sessionId: src_id|d58de45b-f5d2ec4e-c68601f7-95e99623_0 is DEAD
2025-10-09T21:47:57.947249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d58de45b-f5d2ec4e-c68601f7-95e99623_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:57.947260Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d58de45b-f5d2ec4e-c68601f7-95e99623_0] Write session will restart in 0.009483s
2025-10-09T21:47:57.947290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d58de45b-f5d2ec4e-c68601f7-95e99623_0] Write session: Do CDS request
2025-10-09T21:47:57.947297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d58de45b-f5d2ec4e-c68601f7-95e99623_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:57.957692Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d58de45b-f5d2ec4e-c68601f7-95e99623_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:57.957879Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:57.957898Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1080
2025-10-09T21:47:57.958092Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1080 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:57.958136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1080 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:57.958146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1080 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:57.958150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:57.958641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1080 sessionId:  describe result for acl check
2025-10-09T21:47:57.958678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:57.958686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:57.958688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:57.958694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342059404640791:9609] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:57.959244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342059404640791:9609] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:57.961895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342059404640791:9609] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:57.963354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:57.965920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:57.966183Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:57.966194Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:57.966207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:57.966271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:57.966345Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342059042819617:8589]
2025-10-09T21:47:57.966417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342059404640791:9609] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:57.966435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342059404640791:9609] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:57.966440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342059404640791:9609] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:57.966451Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1080 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:57.966953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:57.967401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:57.967466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:57.967801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:57.967857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.967863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.967864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.967865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.967866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.967867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.967868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.967869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.967870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.967876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.967887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.967890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.967894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.967899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.967902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.967905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.967908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.967911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.967918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.967926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:57.967930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.967935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:57.967938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.967943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:57.967947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.967951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:57.967955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.967959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:57.967969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.967973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:57.967977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.967981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.967988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.967992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.967998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.968007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.968031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:57.975086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:57.976522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:57.976545Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:57.976548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:57.976553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:57.976562Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1079 [2:7559342059042819617:8589]
2025-10-09T21:47:57.976582Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:57.976607Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:57.976619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:57.976623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:57.976625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:57.976645Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:57.976661Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:57.976669Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:57.976738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.976745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.976747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.976749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:57.976825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:57.976831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.976845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:57.976858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:57.976871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:57.976898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342059042819610:8589]
2025-10-09T21:47:57.976903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342059042819610:8589]
2025-10-09T21:47:57.976905Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342059042819610:8589]
2025-10-09T21:47:57.976916Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:57.976939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342059404640888:9609], now have 1 active actors on pipe
2025-10-09T21:47:57.976946Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:57.976947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342059404640735:2432], now have 1 active actors on pipe
2025-10-09T21:47:57.976950Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:57.976952Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:57.976955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:57.976968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:57.976971Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:57.977016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342059042819610:8589]
2025-10-09T21:47:57.977031Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:57.977050Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:57.977077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:57.977086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:57.977034Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1079
2025-10-09T21:47:57.977114Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f6b90be8-5395913d-85b7751a-88e5e803_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:57.977088Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1079
2025-10-09T21:47:57.977129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:57.977132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:57.977134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:57.977146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:57.977156Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:57.977172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:57.977614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:57.977626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:57.977644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:57.977750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1080 partition: 0 MaxSeqNo: 25 sessionId: src_id|f6b90be8-5395913d-85b7751a-88e5e803_0
2025-10-09T21:47:57.978177Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d58de45b-f5d2ec4e-c68601f7-95e99623_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f6b90be8-5395913d-85b7751a-88e5e803_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:57.980786Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f6b90be8-5395913d-85b7751a-88e5e803_0] Write session: try to update token
2025-10-09T21:47:57.980811Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f6b90be8-5395913d-85b7751a-88e5e803_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:57.992552Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:57.992578Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:57.992580Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:57.992583Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:57.992593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342059042819610:8589]
2025-10-09T21:47:57.992597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342059042819610:8589]
2025-10-09T21:47:57.992601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342059042819610:8589]
2025-10-09T21:47:57.992653Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:57.992655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:57.992660Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:57.992664Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:57.992664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:57.992666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:57.992822Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:57.999274Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1080 sessionId: src_id|f6b90be8-5395913d-85b7751a-88e5e803_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:58.001993Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:58.047197Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342063699608195:9144]
2025-10-09T21:47:58.047556Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 1079, pipe: [1:7559342063699608195:9144]
2025-10-09T21:47:58.143842Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:58.144190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:58.144206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:58.144229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342063699608195:9144], now have 1 active actors on pipe
2025-10-09T21:47:58.144394Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:58.144406Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:58.144435Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:58.144478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1080 reason: error at write: Disconnected sessionId: src_id|f6b90be8-5395913d-85b7751a-88e5e803_0
2025-10-09T21:47:58.144677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1080 sessionId: src_id|f6b90be8-5395913d-85b7751a-88e5e803_0 is DEAD
2025-10-09T21:47:58.144805Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6b90be8-5395913d-85b7751a-88e5e803_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:58.144816Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6b90be8-5395913d-85b7751a-88e5e803_0] Write session will restart in 0.008951s
2025-10-09T21:47:58.144841Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6b90be8-5395913d-85b7751a-88e5e803_0] Write session: Do CDS request
2025-10-09T21:47:58.144847Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6b90be8-5395913d-85b7751a-88e5e803_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:58.144861Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342063699608195:9144]
2025-10-09T21:47:58.144880Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:58.154608Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f6b90be8-5395913d-85b7751a-88e5e803_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:58.154894Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:58.154910Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1081
2025-10-09T21:47:58.155065Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1081 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:58.155111Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1081 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:58.155120Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1081 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:58.155124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:58.155494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1081 sessionId:  describe result for acl check
2025-10-09T21:47:58.155530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:58.155545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:58.155546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:58.155552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342063699608329:9619] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:58.156057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342063699608329:9619] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:58.156879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:58.158246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342063699608329:9619] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:58.159049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:58.159254Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:58.159264Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:58.159276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:58.159336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:58.159378Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342063337787094:8601]
2025-10-09T21:47:58.159986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:58.160277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:58.160323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:58.160522Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342063699608329:9619] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:58.160537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342063699608329:9619] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:58.160540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342063699608329:9619] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:58.160548Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1081 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:58.160657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:58.160710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.160716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.160717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.160718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.160719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.160720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.160722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.160723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.160724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.160729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.160739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.160743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.160747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.160751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.160755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.160759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.160763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.160767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.160773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.160781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.160787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.160793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.160801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.160807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.160816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.160821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.160826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.160831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.160836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.160841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.160846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.160855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.160860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.160865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.160874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.160879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.160906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:58.167409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:58.168858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:58.168877Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:58.168879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:58.168884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:58.168891Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1080 [2:7559342063337787094:8601]
2025-10-09T21:47:58.168906Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:58.168924Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:58.168933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:58.168936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:58.168939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:58.168958Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:58.168969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:58.168978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:58.169045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.169053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.169055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.169058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:58.169132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:58.169140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.169153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.169166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.169189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:58.169218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342063699608368:9619], now have 1 active actors on pipe
2025-10-09T21:47:58.169232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342063337787088:8601]
2025-10-09T21:47:58.169235Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342063337787088:8601]
2025-10-09T21:47:58.169237Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342063337787088:8601]
2025-10-09T21:47:58.169251Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:58.169266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342063699608214:2432], now have 1 active actors on pipe
2025-10-09T21:47:58.169274Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:58.169279Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:58.169280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:58.169281Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:58.169295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:58.169303Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:58.169301Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1080
2025-10-09T21:47:58.169335Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1080
2025-10-09T21:47:58.169341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342063337787088:8601]
2025-10-09T21:47:58.169351Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:58.169364Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:58.169388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:58.169401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:58.169440Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d0a76705-86be8c74-fcc40d1b-12152a77_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:58.169452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:58.169456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:58.169464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:58.169475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:58.169486Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:58.169507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:58.169652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:58.169661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:58.169677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:58.169773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1081 partition: 0 MaxSeqNo: 25 sessionId: src_id|d0a76705-86be8c74-fcc40d1b-12152a77_0
2025-10-09T21:47:58.170194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6b90be8-5395913d-85b7751a-88e5e803_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d0a76705-86be8c74-fcc40d1b-12152a77_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:58.172654Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d0a76705-86be8c74-fcc40d1b-12152a77_0] Write session: try to update token
2025-10-09T21:47:58.172677Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d0a76705-86be8c74-fcc40d1b-12152a77_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:58.183956Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:58.183975Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:58.183977Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:58.183978Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:58.183986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342063337787088:8601]
2025-10-09T21:47:58.183989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342063337787088:8601]
2025-10-09T21:47:58.183992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342063337787088:8601]
2025-10-09T21:47:58.184028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:58.184032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.184033Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:58.184034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:58.184038Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:58.184042Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:58.184201Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:58.186826Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1081 sessionId: src_id|d0a76705-86be8c74-fcc40d1b-12152a77_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:58.189348Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:58.244880Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342063699608392:9144]
2025-10-09T21:47:58.245181Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 1080, pipe: [1:7559342063699608392:9144]
2025-10-09T21:47:58.339548Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:58.339795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:58.339807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:58.339847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342063699608392:9144], now have 1 active actors on pipe
2025-10-09T21:47:58.340041Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:58.340086Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:58.340087Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342063699608392:9144]
2025-10-09T21:47:58.340101Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:58.340111Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:58.340162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1081 reason: error at write: Disconnected sessionId: src_id|d0a76705-86be8c74-fcc40d1b-12152a77_0
2025-10-09T21:47:58.340349Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1081 sessionId: src_id|d0a76705-86be8c74-fcc40d1b-12152a77_0 is DEAD
2025-10-09T21:47:58.340520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d0a76705-86be8c74-fcc40d1b-12152a77_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:58.340537Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d0a76705-86be8c74-fcc40d1b-12152a77_0] Write session will restart in 0.009435s
2025-10-09T21:47:58.340565Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d0a76705-86be8c74-fcc40d1b-12152a77_0] Write session: Do CDS request
2025-10-09T21:47:58.340571Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d0a76705-86be8c74-fcc40d1b-12152a77_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:58.351602Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d0a76705-86be8c74-fcc40d1b-12152a77_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:58.351677Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:58.351690Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1082
2025-10-09T21:47:58.351841Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1082 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:58.351875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1082 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:58.351894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1082 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:58.351897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:58.352176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:58.352310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1082 sessionId:  describe result for acl check
2025-10-09T21:47:58.352347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:58.352349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:58.352350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:58.352357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342063699608514:9625] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:58.352904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342063699608514:9625] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:58.354509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:58.354718Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:58.354729Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:58.354742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:58.354795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:58.354838Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342063337787258:8605]
2025-10-09T21:47:58.355383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342063699608514:9625] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:58.355432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:58.355796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:58.355853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:58.356162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:58.356219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.356226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.356227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.356228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.356229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.356230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.356231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.356232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.356233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.356238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.356246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.356250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.356253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.356256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.356260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.356263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.356267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.356270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.356276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.356284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.356289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.356294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.356299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.356303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.356307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.356311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.356315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.356319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.356323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.356327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.356331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.356342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.356346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.356350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.356354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.356358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.356383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:58.361388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342063699608514:9625] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:58.361404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342063699608514:9625] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:58.361411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342063699608514:9625] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:58.361420Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1082 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:58.362690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:58.363786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:58.363805Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:58.363808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:58.363812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:58.363820Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1081 [2:7559342063337787258:8605]
2025-10-09T21:47:58.363833Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:58.364201Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1081
2025-10-09T21:47:58.364240Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1081
2025-10-09T21:47:58.363850Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:58.363854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:58.363857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:58.363859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:58.363876Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:58.363882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:58.363889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:58.363948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.363952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.363953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.363956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:58.364026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:58.364029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.364039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.364049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.364062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:58.364083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342063337787253:8605]
2025-10-09T21:47:58.364085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342063337787253:8605]
2025-10-09T21:47:58.364086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342063337787253:8605]
2025-10-09T21:47:58.364100Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:58.364114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342063699608395:2432], now have 1 active actors on pipe
2025-10-09T21:47:58.364117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342063699608553:9625], now have 1 active actors on pipe
2025-10-09T21:47:58.364125Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:58.364127Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:58.364131Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:58.364134Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:58.364137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:58.364140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:58.364174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342063337787253:8605]
2025-10-09T21:47:58.364181Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:58.364210Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:58.364331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:58.364339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:58.364369Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|645200b6-416973b8-28f757f1-e933efa1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:58.364388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:58.364397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:58.364400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:58.364411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:58.364428Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:58.364446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:58.364592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:58.364600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:58.364614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:58.364758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1082 partition: 0 MaxSeqNo: 25 sessionId: src_id|645200b6-416973b8-28f757f1-e933efa1_0
2025-10-09T21:47:58.365124Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d0a76705-86be8c74-fcc40d1b-12152a77_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|645200b6-416973b8-28f757f1-e933efa1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:58.367520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|645200b6-416973b8-28f757f1-e933efa1_0] Write session: try to update token
2025-10-09T21:47:58.367539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|645200b6-416973b8-28f757f1-e933efa1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:58.375483Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:58.375503Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:58.375513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342063337787253:8605]
2025-10-09T21:47:58.375555Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:58.375558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:58.375728Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:58.375897Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:58.376038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:58.376179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:58.376369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:58.376412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:58.381768Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1082 sessionId: src_id|645200b6-416973b8-28f757f1-e933efa1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:58.384197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:58.384508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:58.384531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:58.384572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:58.384662Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:58.388097Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:58.388117Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:58.388119Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:58.388121Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:58.388130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342063337787253:8605]
2025-10-09T21:47:58.388134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342063337787253:8605]
2025-10-09T21:47:58.388138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342063337787253:8605]
2025-10-09T21:47:58.388183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:58.388183Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:58.388188Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:58.388189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.388192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:58.388192Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:58.388323Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:58.440457Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342063699608560:9144]
2025-10-09T21:47:58.440795Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 1081, pipe: [1:7559342063699608560:9144]
2025-10-09T21:47:58.543073Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:58.550922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:58.550953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:58.551120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:58.551186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:58.551253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:58.551315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:58.551360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:58.551370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:58.551438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:58.551497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:58.551562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:58.551630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:58.551672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:58.551679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:58.551742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:58.551797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:58.551854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:58.551911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:58.551951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:58.551959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:58.552050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:58.552117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:58.552177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:58.552244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:58.552296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:58.552308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:58.552375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:58.552435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:58.552497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:58.552557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:58.552602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:58.552609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:58.552673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:58.552740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:58.552804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:58.552873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:58.552917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:58.552928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:58.552996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:58.553048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:58.553109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:58.553165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:58.553213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:58.553216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:58.553272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:58.553325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:58.553381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:58.553437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:58.553477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:58.553479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:58.553538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:58.553627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:58.553712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:58.553811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:58.553851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:58.553854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:58.553919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:58.553998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:58.554075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:58.554136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:58.554177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:58.554180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:58.554248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342063699608560:9144], now have 1 active actors on pipe
2025-10-09T21:47:58.554496Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:58.554554Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:58.554558Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:58.554961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|645200b6-416973b8-28f757f1-e933efa1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:58.554971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|645200b6-416973b8-28f757f1-e933efa1_0] Write session will restart in 0.008057s
2025-10-09T21:47:58.554997Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|645200b6-416973b8-28f757f1-e933efa1_0] Write session: Do CDS request
2025-10-09T21:47:58.555003Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|645200b6-416973b8-28f757f1-e933efa1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:58.555239Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:58.554592Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1082 reason: error at write: Disconnected sessionId: src_id|645200b6-416973b8-28f757f1-e933efa1_0
2025-10-09T21:47:58.554676Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342063699608560:9144]
2025-10-09T21:47:58.554812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1082 sessionId: src_id|645200b6-416973b8-28f757f1-e933efa1_0 is DEAD
2025-10-09T21:47:58.564616Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|645200b6-416973b8-28f757f1-e933efa1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:58.564659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:58.564672Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1083
2025-10-09T21:47:58.564867Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1083 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:58.564910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1083 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:58.564921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1083 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:58.564926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:58.566471Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1083 sessionId:  describe result for acl check
2025-10-09T21:47:58.566513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:58.566531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:58.566533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:58.566539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342063699608626:9629] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:58.567105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342063699608626:9629] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:58.569933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342063699608626:9629] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:58.570868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:58.572779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:58.573005Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:58.573016Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:58.573028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:58.573085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:58.573126Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342063337787415:8607]
2025-10-09T21:47:58.573179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342063699608626:9629] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:58.573195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342063699608626:9629] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:58.573199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342063699608626:9629] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:58.573208Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1083 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:58.573706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:58.574024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:58.574097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:58.574428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:58.574492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.574499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.574501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.574502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.574503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.574504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.574506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.574507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.574508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.574514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.574524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.574528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.574531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.574535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.574538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.574542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.574545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.574548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.574555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.574563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.574569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.574574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.574578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.574582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.574586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.574590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.574594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.574598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.574601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.574606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.574610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.574614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.574618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.574622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.574626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.574636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.574672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:58.581205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:58.582354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:58.582372Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:58.582375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:58.582379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:58.582386Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1082 [2:7559342063337787415:8607]
2025-10-09T21:47:58.582401Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:58.582420Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:58.582424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:58.582428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:58.582430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:58.582462Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:58.582473Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:58.582481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:58.582546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.582553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.582556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.582558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:58.582635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:58.582645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.582657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.582674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.582695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:58.582724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342063699608717:9629], now have 1 active actors on pipe
2025-10-09T21:47:58.582732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342063699608563:2432], now have 1 active actors on pipe
2025-10-09T21:47:58.582740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342063337787410:8607]
2025-10-09T21:47:58.582747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342063337787410:8607]
2025-10-09T21:47:58.582749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342063337787410:8607]
2025-10-09T21:47:58.582762Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:58.582782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:58.582782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:58.582787Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:58.582789Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:58.582801Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:58.582809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:58.582783Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1082
2025-10-09T21:47:58.582787Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1082
2025-10-09T21:47:58.582861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342063337787410:8607]
2025-10-09T21:47:58.582873Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:58.582879Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:58.582895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:58.582905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:58.582957Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6d7f7475-d8af0b5e-c67f5ee4-5d36c12b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:58.582969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:58.582973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:58.582976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:58.582987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:58.582997Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:58.583021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:58.583183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:58.583199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:58.583222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:58.583332Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1083 partition: 0 MaxSeqNo: 25 sessionId: src_id|6d7f7475-d8af0b5e-c67f5ee4-5d36c12b_0
2025-10-09T21:47:58.583772Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|645200b6-416973b8-28f757f1-e933efa1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6d7f7475-d8af0b5e-c67f5ee4-5d36c12b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:58.586187Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6d7f7475-d8af0b5e-c67f5ee4-5d36c12b_0] Write session: try to update token
2025-10-09T21:47:58.586203Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6d7f7475-d8af0b5e-c67f5ee4-5d36c12b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:58.597403Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:58.597423Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:58.597425Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:58.597427Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:58.597438Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342063337787410:8607]
2025-10-09T21:47:58.597442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342063337787410:8607]
2025-10-09T21:47:58.597447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342063337787410:8607]
2025-10-09T21:47:58.597487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:58.597492Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:58.597496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.597498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:58.597498Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:58.597505Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:58.597622Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:58.600442Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1083 sessionId: src_id|6d7f7475-d8af0b5e-c67f5ee4-5d36c12b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:58.603013Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:58.654968Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342063699608730:9144]
2025-10-09T21:47:58.655337Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 1082, pipe: [1:7559342063699608730:9144]
2025-10-09T21:47:58.756557Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:58.757134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:58.757158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:58.757206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342063699608730:9144], now have 1 active actors on pipe
2025-10-09T21:47:58.757449Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:58.757444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:58.757463Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:58.757520Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342063699608730:9144]
2025-10-09T21:47:58.757530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1083 reason: error at write: Disconnected sessionId: src_id|6d7f7475-d8af0b5e-c67f5ee4-5d36c12b_0
2025-10-09T21:47:58.757548Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:58.757751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1083 sessionId: src_id|6d7f7475-d8af0b5e-c67f5ee4-5d36c12b_0 is DEAD
2025-10-09T21:47:58.757975Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d7f7475-d8af0b5e-c67f5ee4-5d36c12b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:58.757987Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d7f7475-d8af0b5e-c67f5ee4-5d36c12b_0] Write session will restart in 0.006743s
2025-10-09T21:47:58.758015Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d7f7475-d8af0b5e-c67f5ee4-5d36c12b_0] Write session: Do CDS request
2025-10-09T21:47:58.758021Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d7f7475-d8af0b5e-c67f5ee4-5d36c12b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:58.765604Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6d7f7475-d8af0b5e-c67f5ee4-5d36c12b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:58.766014Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:58.766041Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1084
2025-10-09T21:47:58.766157Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1084 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:58.766189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1084 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:58.766198Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1084 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:58.766202Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:58.766634Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1084 sessionId:  describe result for acl check
2025-10-09T21:47:58.766675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:58.766681Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:58.766683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:58.766689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342063699608776:9633] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:58.767251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342063699608776:9633] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:58.769925Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342063699608776:9633] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:58.770723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:58.772718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:58.773121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342063699608776:9633] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:58.773136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342063699608776:9633] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:58.773140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342063699608776:9633] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:58.773150Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1084 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:58.772931Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:58.772947Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:58.772961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:58.773017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:58.773062Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342063337787577:8611]
2025-10-09T21:47:58.773608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:58.773979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:58.774065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:58.774371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:58.774422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.774429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.774430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.774431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.774432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.774433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.774434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.774435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.774436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.774441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.774451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.774454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.774458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.774461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.774465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.774468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.774472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.774475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.774482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.774490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.774496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.774501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.774505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.774511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.774515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.774520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.774525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.774530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.774535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.774540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.774545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.774550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.774555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.774560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.774565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.774570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.774599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:58.781189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:58.782736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:58.782754Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:58.782757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:58.782763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:58.782772Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1083 [2:7559342063337787577:8611]
2025-10-09T21:47:58.782787Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:58.782807Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:58.782819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:58.782823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:58.782826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:58.782844Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:58.782851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:58.782859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:58.782922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.782931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.782934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.782936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:58.783010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:58.783019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.783032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.783050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.783067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:58.783096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342063699608732:2432], now have 1 active actors on pipe
2025-10-09T21:47:58.783105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342063699608887:9633], now have 1 active actors on pipe
2025-10-09T21:47:58.783113Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342063337787572:8611]
2025-10-09T21:47:58.783120Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342063337787572:8611]
2025-10-09T21:47:58.783122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342063337787572:8611]
2025-10-09T21:47:58.783139Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:58.783159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:58.783145Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1083
2025-10-09T21:47:58.783181Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:58.783169Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1083
2025-10-09T21:47:58.783184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:58.783187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:58.783188Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:58.783189Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:58.783249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:58.783260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:58.783275Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342063337787572:8611]
2025-10-09T21:47:58.783288Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:58.783293Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:58.783325Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|29d4ee43-24b5ad49-f43f1b-e4cc794f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:58.783340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:58.783344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:58.783347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:58.783361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:58.783376Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:58.783406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:58.783596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:58.783611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:58.783636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:58.783756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1084 partition: 0 MaxSeqNo: 25 sessionId: src_id|29d4ee43-24b5ad49-f43f1b-e4cc794f_0
2025-10-09T21:47:58.784143Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d7f7475-d8af0b5e-c67f5ee4-5d36c12b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|29d4ee43-24b5ad49-f43f1b-e4cc794f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:58.786638Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29d4ee43-24b5ad49-f43f1b-e4cc794f_0] Write session: try to update token
2025-10-09T21:47:58.786659Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29d4ee43-24b5ad49-f43f1b-e4cc794f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:58.797597Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:58.797619Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:58.797621Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:58.797624Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:58.797633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342063337787572:8611]
2025-10-09T21:47:58.797638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342063337787572:8611]
2025-10-09T21:47:58.797642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342063337787572:8611]
2025-10-09T21:47:58.797685Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:58.797689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:58.797696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.797699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:58.797709Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:58.797713Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:58.797853Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:58.800235Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1084 sessionId: src_id|29d4ee43-24b5ad49-f43f1b-e4cc794f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:58.802757Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:58.857752Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342063699608902:9144]
2025-10-09T21:47:58.858134Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 1083, pipe: [1:7559342063699608902:9144]
2025-10-09T21:47:58.959002Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:58.959285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:58.959300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:58.959344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342063699608902:9144], now have 1 active actors on pipe
2025-10-09T21:47:58.959520Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342063699608902:9144]
2025-10-09T21:47:58.959528Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:58.959532Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:58.959561Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:58.959594Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1084 reason: error at write: Disconnected sessionId: src_id|29d4ee43-24b5ad49-f43f1b-e4cc794f_0
2025-10-09T21:47:58.959782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1084 sessionId: src_id|29d4ee43-24b5ad49-f43f1b-e4cc794f_0 is DEAD
2025-10-09T21:47:58.959771Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:58.959957Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29d4ee43-24b5ad49-f43f1b-e4cc794f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:58.959966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29d4ee43-24b5ad49-f43f1b-e4cc794f_0] Write session will restart in 0.008784s
2025-10-09T21:47:58.959993Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29d4ee43-24b5ad49-f43f1b-e4cc794f_0] Write session: Do CDS request
2025-10-09T21:47:58.959999Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29d4ee43-24b5ad49-f43f1b-e4cc794f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:58.969581Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29d4ee43-24b5ad49-f43f1b-e4cc794f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:58.969704Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:58.969720Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1085
2025-10-09T21:47:58.969836Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1085 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:58.969869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1085 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:58.969877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1085 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:58.969881Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:58.970375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1085 sessionId:  describe result for acl check
2025-10-09T21:47:58.970415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:58.970417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:58.970418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:58.970425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342063699609002:9638] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:58.971033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342063699609002:9638] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:58.972552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:58.973431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342063699609002:9638] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:58.975553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:58.975781Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:58.975792Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:58.975805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:58.975863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:58.975904Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342063337787737:8613]
2025-10-09T21:47:58.976212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342063699609002:9638] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:58.976230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342063699609002:9638] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:58.976233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342063699609002:9638] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:58.976243Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1085 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:58.976465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:58.976764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:58.976807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:58.977103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:58.977151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.977158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.977160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.977161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.977162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.977163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.977164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.977165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.977166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.977171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.977181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.977185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.977189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.977192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.977196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.977200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.977203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.977207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.977213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.977221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:58.977227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.977232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:58.977236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.977241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:58.977245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.977250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:58.977254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.977259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:58.977263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.977268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:58.977272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.977277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.977281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.977286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.977290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.977295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.977321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:58.983890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:58.985000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:58.985020Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:58.985023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:58.985028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:58.985035Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1084 [2:7559342063337787737:8613]
2025-10-09T21:47:58.985050Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:58.985071Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:58.985079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:58.985082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:58.985085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:58.985104Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:58.985111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:58.985118Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:58.985185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.985194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.985197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.985201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:58.985271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:58.985274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:58.985285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:58.985295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:58.985310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:58.985332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342063699609061:9638], now have 1 active actors on pipe
2025-10-09T21:47:58.985335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342063699608907:2432], now have 1 active actors on pipe
2025-10-09T21:47:58.985344Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342063337787732:8613]
2025-10-09T21:47:58.985346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342063337787732:8613]
2025-10-09T21:47:58.985348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342063337787732:8613]
2025-10-09T21:47:58.985363Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:58.985376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:58.985382Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:58.985386Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:58.985388Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:58.985390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:58.985393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:58.985426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342063337787732:8613]
2025-10-09T21:47:58.985433Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:58.985439Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:58.987586Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1084
2025-10-09T21:47:58.987598Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1084
2025-10-09T21:47:58.987721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:58.987738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:58.987787Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|be5111a9-410003e3-60358404-9a61c350_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:58.987805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:58.987810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:58.987813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:58.987826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:58.987842Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:58.987860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:58.988090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:58.988105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:58.988128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:58.988249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1085 partition: 0 MaxSeqNo: 25 sessionId: src_id|be5111a9-410003e3-60358404-9a61c350_0
2025-10-09T21:47:58.988617Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29d4ee43-24b5ad49-f43f1b-e4cc794f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|be5111a9-410003e3-60358404-9a61c350_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:58.991081Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be5111a9-410003e3-60358404-9a61c350_0] Write session: try to update token
2025-10-09T21:47:58.991102Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be5111a9-410003e3-60358404-9a61c350_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:59.002091Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:59.002116Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:59.002118Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:59.002120Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:59.002132Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342063337787732:8613]
2025-10-09T21:47:59.002137Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342063337787732:8613]
2025-10-09T21:47:59.002142Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342063337787732:8613]
2025-10-09T21:47:59.002189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:59.002190Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:59.002195Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:59.002200Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:59.002203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.002206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:59.002351Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:59.005686Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1085 sessionId: src_id|be5111a9-410003e3-60358404-9a61c350_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:59.008216Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:59.059485Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342067994576369:9144]
2025-10-09T21:47:59.059749Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 1084, pipe: [1:7559342067994576369:9144]
2025-10-09T21:47:59.164995Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:59.165295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:59.165313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:59.165360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342067994576369:9144], now have 1 active actors on pipe
2025-10-09T21:47:59.165578Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:59.165587Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342067994576369:9144]
2025-10-09T21:47:59.165589Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:59.165593Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:59.165619Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:59.165653Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1085 reason: error at write: Disconnected sessionId: src_id|be5111a9-410003e3-60358404-9a61c350_0
2025-10-09T21:47:59.165845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1085 sessionId: src_id|be5111a9-410003e3-60358404-9a61c350_0 is DEAD
2025-10-09T21:47:59.166012Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be5111a9-410003e3-60358404-9a61c350_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:59.166023Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be5111a9-410003e3-60358404-9a61c350_0] Write session will restart in 0.009644s
2025-10-09T21:47:59.166066Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be5111a9-410003e3-60358404-9a61c350_0] Write session: Do CDS request
2025-10-09T21:47:59.166072Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be5111a9-410003e3-60358404-9a61c350_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:59.176644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be5111a9-410003e3-60358404-9a61c350_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:59.176671Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:59.176689Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1086
2025-10-09T21:47:59.176882Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1086 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:59.176937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1086 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:59.176948Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1086 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:59.176952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:59.177479Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1086 sessionId:  describe result for acl check
2025-10-09T21:47:59.177516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:59.177518Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:59.177520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:59.177526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342067994576498:9642] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:59.178053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342067994576498:9642] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:59.180110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342067994576498:9642] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:59.180657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:59.182979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:59.183188Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:59.183200Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:59.183213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:59.183269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:59.183333Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342067632755223:8625]
2025-10-09T21:47:59.183349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342067994576498:9642] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:59.183367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342067994576498:9642] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:59.183371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342067994576498:9642] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:59.183381Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1086 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:59.183932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:59.184203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:59.184264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:59.184584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:59.184642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:59.184649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:59.184650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:59.184652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:59.184653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:59.184654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:59.184655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.184657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.184658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.184664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:59.184675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:59.184678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:59.184682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:59.184686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:59.184690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:59.184694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.184698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.184701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.184708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:59.184716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:59.184722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:59.184728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:59.184737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:59.184742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:59.184746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:59.184751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:59.184757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:59.184762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:59.184778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:59.184783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:59.184796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.184801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.184806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.184811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.184816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.184821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.184847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:59.191098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:59.192185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:59.192206Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:59.192208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:59.192212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:59.192219Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1085 [2:7559342067632755223:8625]
2025-10-09T21:47:59.192232Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:59.192250Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:59.192253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:59.192256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:59.192258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:59.192276Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:59.192282Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:59.192289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:59.192346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.192363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.192365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.192368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:59.192441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:59.192450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.192463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.192475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.192492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:59.192521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342067994576377:2432], now have 1 active actors on pipe
2025-10-09T21:47:59.192525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342067994576544:9642], now have 1 active actors on pipe
2025-10-09T21:47:59.192533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342067632755218:8625]
2025-10-09T21:47:59.192540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342067632755218:8625]
2025-10-09T21:47:59.192542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342067632755218:8625]
2025-10-09T21:47:59.192557Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:59.192576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:59.192577Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:59.192581Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:59.192583Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:59.192595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:59.192599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:59.192579Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1085
2025-10-09T21:47:59.192640Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1085
2025-10-09T21:47:59.192637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342067632755218:8625]
2025-10-09T21:47:59.192651Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:59.192665Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:59.192835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:59.192847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:59.192876Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f3629e4c-2f6d5a75-d8a6039a-6cbd98b1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:59.192889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:59.192893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:59.192896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:59.192905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:59.192917Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:59.192933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:59.193103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:59.193115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:59.193128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:59.193235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1086 partition: 0 MaxSeqNo: 25 sessionId: src_id|f3629e4c-2f6d5a75-d8a6039a-6cbd98b1_0
2025-10-09T21:47:59.193623Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be5111a9-410003e3-60358404-9a61c350_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f3629e4c-2f6d5a75-d8a6039a-6cbd98b1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:59.196089Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3629e4c-2f6d5a75-d8a6039a-6cbd98b1_0] Write session: try to update token
2025-10-09T21:47:59.196108Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3629e4c-2f6d5a75-d8a6039a-6cbd98b1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:59.203947Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:47:59.203963Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:59.203974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342067632755218:8625]
2025-10-09T21:47:59.204024Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:47:59.204022Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:47:59.204233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:47:59.204371Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:59.204513Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:59.204653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:47:59.204852Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:47:59.204907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:47:59.209982Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1086 sessionId: src_id|f3629e4c-2f6d5a75-d8a6039a-6cbd98b1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:59.212517Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:59.212902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:59.212925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:59.212968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:47:59.213103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:59.217231Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:59.217250Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:59.217253Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:59.217255Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:59.217267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342067632755218:8625]
2025-10-09T21:47:59.217271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342067632755218:8625]
2025-10-09T21:47:59.217274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342067632755218:8625]
2025-10-09T21:47:59.217321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:59.217336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.217339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:59.217340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:59.217347Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:59.217350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:59.217481Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:59.265799Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342067994576573:9144]
2025-10-09T21:47:59.266114Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 1085, pipe: [1:7559342067994576573:9144]
2025-10-09T21:47:59.373971Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:59.382830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:59.382861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:59.383036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:59.383106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:59.383169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:59.383227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:59.383305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:59.383309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:59.383385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:59.383447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:59.383509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:59.383573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:59.383619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:59.383625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:59.383689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:59.383761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:59.383822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:59.383881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:59.383920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:59.383923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:59.384023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:59.384085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:59.384153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:59.384220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:59.384267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:59.384270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:59.384347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:59.384413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:59.384483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:59.384545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:59.384586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:59.384589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:59.384667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:59.384726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:59.384796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:59.384910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:59.384956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:59.384959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:59.385060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:59.385154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:59.385208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:59.385262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:59.385326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:59.385329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:59.385417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:59.385481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:59.385546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:59.385608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:59.385649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:59.385651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:59.385721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:59.385805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:59.385872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:59.385937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:59.386106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:59.386109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:59.386180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:47:59.386243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:47:59.386308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:47:59.386376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:47:59.386441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:47:59.386450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:47:59.386516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342067994576573:9144], now have 1 active actors on pipe
2025-10-09T21:47:59.386752Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342067994576573:9144]
2025-10-09T21:47:59.386771Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:59.386776Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:59.386783Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:59.386818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1086 reason: error at write: Disconnected sessionId: src_id|f3629e4c-2f6d5a75-d8a6039a-6cbd98b1_0
2025-10-09T21:47:59.387037Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1086 sessionId: src_id|f3629e4c-2f6d5a75-d8a6039a-6cbd98b1_0 is DEAD
2025-10-09T21:47:59.387203Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3629e4c-2f6d5a75-d8a6039a-6cbd98b1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:59.387215Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3629e4c-2f6d5a75-d8a6039a-6cbd98b1_0] Write session will restart in 0.005109s
2025-10-09T21:47:59.387252Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3629e4c-2f6d5a75-d8a6039a-6cbd98b1_0] Write session: Do CDS request
2025-10-09T21:47:59.387258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3629e4c-2f6d5a75-d8a6039a-6cbd98b1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:59.387657Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:59.393588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3629e4c-2f6d5a75-d8a6039a-6cbd98b1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:59.393667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:59.393685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1087
2025-10-09T21:47:59.393894Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1087 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:59.393941Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1087 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:59.393962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1087 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:59.393966Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:59.394541Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1087 sessionId:  describe result for acl check
2025-10-09T21:47:59.394581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:59.394589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:59.394591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:59.394598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342067994576606:9654] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:59.395255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342067994576606:9654] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:59.398325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342067994576606:9654] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:59.399917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:59.402153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:59.402377Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:59.402389Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:59.402421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:59.402493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:59.402539Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342067632755392:8629]
2025-10-09T21:47:59.402562Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342067994576606:9654] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:59.402577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342067994576606:9654] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:59.402581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342067994576606:9654] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:59.402590Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1087 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:59.403050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:59.403384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:59.403417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:59.403650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:59.403681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:59.403685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:59.403686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:59.403687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:59.403688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:59.403688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:59.403689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.403689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.403690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.403693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:59.403700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:59.403702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:59.403705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:59.403707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:59.403709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:59.403711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.403713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.403715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.403720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:59.403725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:59.403730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:59.403733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:59.403736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:59.403739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:59.403741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:59.403745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:59.403748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:59.403751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:59.403753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:59.403757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:59.403759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.403763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.403765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.403768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.403771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.403774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.403795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:59.410639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:59.411738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:59.411753Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:59.411756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:59.411759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:59.411765Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1086 [2:7559342067632755392:8629]
2025-10-09T21:47:59.411776Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:59.411794Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:59.411796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:59.411799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:59.411801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:59.411815Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:59.411819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:59.411828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:59.411876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.411883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.411884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.411886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:59.411956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:59.411964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.411977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.411989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.412002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:59.412024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342067994576575:2432], now have 1 active actors on pipe
2025-10-09T21:47:59.412032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342067994576734:9654], now have 1 active actors on pipe
2025-10-09T21:47:59.412038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342067632755387:8629]
2025-10-09T21:47:59.412040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342067632755387:8629]
2025-10-09T21:47:59.412042Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342067632755387:8629]
2025-10-09T21:47:59.412052Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:59.412067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:59.412071Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:59.412076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:59.412079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:59.412083Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:59.412086Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:59.412110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342067632755387:8629]
2025-10-09T21:47:59.412121Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:59.412137Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:59.412116Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1086
2025-10-09T21:47:59.412183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:59.412138Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1086
2025-10-09T21:47:59.412189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:59.412223Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4711cfb8-16802c62-463e79dc-30cbdadf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:59.412240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:59.412244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:59.412246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:59.412258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:59.412269Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:59.412287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:59.412435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:59.412441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:59.412453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:59.412550Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1087 partition: 0 MaxSeqNo: 25 sessionId: src_id|4711cfb8-16802c62-463e79dc-30cbdadf_0
2025-10-09T21:47:59.412955Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3629e4c-2f6d5a75-d8a6039a-6cbd98b1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4711cfb8-16802c62-463e79dc-30cbdadf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:59.415796Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4711cfb8-16802c62-463e79dc-30cbdadf_0] Write session: try to update token
2025-10-09T21:47:59.415837Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4711cfb8-16802c62-463e79dc-30cbdadf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:59.427277Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:59.427299Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:59.427301Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:59.427304Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:59.427314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342067632755387:8629]
2025-10-09T21:47:59.427319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342067632755387:8629]
2025-10-09T21:47:59.427323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342067632755387:8629]
2025-10-09T21:47:59.427370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:59.427373Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:59.427378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.427379Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:59.427381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:59.427384Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:59.427530Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:59.431175Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1087 sessionId: src_id|4711cfb8-16802c62-463e79dc-30cbdadf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:59.433826Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:59.486989Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342067994576747:9144]
2025-10-09T21:47:59.487473Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 1086, pipe: [1:7559342067994576747:9144]
2025-10-09T21:47:59.587368Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:59.587857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:59.587874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:59.587923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342067994576747:9144], now have 1 active actors on pipe
2025-10-09T21:47:59.588157Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:59.588180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:59.588184Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:59.588213Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342067994576747:9144]
2025-10-09T21:47:59.588266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1087 reason: error at write: Disconnected sessionId: src_id|4711cfb8-16802c62-463e79dc-30cbdadf_0
2025-10-09T21:47:59.588357Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:59.588470Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1087 sessionId: src_id|4711cfb8-16802c62-463e79dc-30cbdadf_0 is DEAD
2025-10-09T21:47:59.588645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4711cfb8-16802c62-463e79dc-30cbdadf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:59.588657Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4711cfb8-16802c62-463e79dc-30cbdadf_0] Write session will restart in 0.007511s
2025-10-09T21:47:59.588684Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4711cfb8-16802c62-463e79dc-30cbdadf_0] Write session: Do CDS request
2025-10-09T21:47:59.588690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4711cfb8-16802c62-463e79dc-30cbdadf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:59.597576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4711cfb8-16802c62-463e79dc-30cbdadf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:59.597620Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:59.597634Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1088
2025-10-09T21:47:59.597807Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1088 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:59.597839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1088 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:59.597851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1088 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:59.597855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:59.598314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1088 sessionId:  describe result for acl check
2025-10-09T21:47:59.598353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:59.598359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:59.598361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:59.598368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342067994576828:9658] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:59.598907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342067994576828:9658] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:59.600798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:59.600995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342067994576828:9658] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:59.602715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:59.602906Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:59.602919Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:59.602931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:59.602984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:59.603022Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342067632755554:8631]
2025-10-09T21:47:59.603309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342067994576828:9658] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:59.603325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342067994576828:9658] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:59.603329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342067994576828:9658] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:59.603337Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1088 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:59.603618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:59.603943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:59.604008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:59.604344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:59.604404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:59.604410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:59.604411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:59.604413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:59.604414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:59.604415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:59.604416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.604417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.604418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.604423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:59.604433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:59.604436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:59.604440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:59.604443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:59.604447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:59.604451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.604454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.604458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.604465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:59.604473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:59.604479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:59.604485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:59.604489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:59.604494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:59.604498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:59.604503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:59.604507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:59.604511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:59.604516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:59.604520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:59.604525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.604530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.604539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.604543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.604547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.604551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.604577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:59.610850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:59.611881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:59.611897Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:59.611900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:59.611906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:59.611913Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1087 [2:7559342067632755554:8631]
2025-10-09T21:47:59.611927Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:59.611945Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:59.611948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:59.611952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:59.611955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:59.611972Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:59.611983Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:59.611990Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:59.612044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.612052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.612054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.612056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:59.612126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:59.612134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.612146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.612160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.612183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:59.612208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342067994576749:2432], now have 1 active actors on pipe
2025-10-09T21:47:59.612217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342067994576909:9658], now have 1 active actors on pipe
2025-10-09T21:47:59.612225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342067632755549:8631]
2025-10-09T21:47:59.612232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342067632755549:8631]
2025-10-09T21:47:59.612235Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342067632755549:8631]
2025-10-09T21:47:59.612247Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:59.612263Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:59.612266Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:59.612276Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:59.612279Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:59.612281Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:59.612285Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:59.612316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342067632755549:8631]
2025-10-09T21:47:59.612325Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:59.612327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:59.612298Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1087
2025-10-09T21:47:59.612303Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1087
2025-10-09T21:47:59.612363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:59.612373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:59.612407Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|84195114-6c8f2cd5-c2d98c49-adde8d90_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:59.612419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:59.612424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:59.612432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:59.612442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:59.612457Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:59.612470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:59.612663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:59.612676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:59.612700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:59.612820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1088 partition: 0 MaxSeqNo: 25 sessionId: src_id|84195114-6c8f2cd5-c2d98c49-adde8d90_0
2025-10-09T21:47:59.613196Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4711cfb8-16802c62-463e79dc-30cbdadf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|84195114-6c8f2cd5-c2d98c49-adde8d90_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:59.615823Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|84195114-6c8f2cd5-c2d98c49-adde8d90_0] Write session: try to update token
2025-10-09T21:47:59.615842Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|84195114-6c8f2cd5-c2d98c49-adde8d90_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:59.626777Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:59.626799Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:59.626802Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:59.626804Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:59.626814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342067632755549:8631]
2025-10-09T21:47:59.626819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342067632755549:8631]
2025-10-09T21:47:59.626823Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342067632755549:8631]
2025-10-09T21:47:59.626849Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:59.626855Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:59.626860Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:59.626864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:59.626871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.626874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:59.627003Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:59.630503Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1088 sessionId: src_id|84195114-6c8f2cd5-c2d98c49-adde8d90_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:59.632928Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:59.688200Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342067994576921:9144]
2025-10-09T21:47:59.688494Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 1087, pipe: [1:7559342067994576921:9144]
2025-10-09T21:47:59.786384Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:59.786863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:59.786878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:59.786920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342067994576921:9144], now have 1 active actors on pipe
2025-10-09T21:47:59.787175Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:59.787187Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:59.787235Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:47:59.787253Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342067994576921:9144]
2025-10-09T21:47:59.787326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1088 reason: status is not ok: tablet will be restarted right now sessionId: src_id|84195114-6c8f2cd5-c2d98c49-adde8d90_0
2025-10-09T21:47:59.787520Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1088 sessionId: src_id|84195114-6c8f2cd5-c2d98c49-adde8d90_0 is DEAD
2025-10-09T21:47:59.787670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84195114-6c8f2cd5-c2d98c49-adde8d90_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:47:59.787681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84195114-6c8f2cd5-c2d98c49-adde8d90_0] Write session will restart in 0.005030s
2025-10-09T21:47:59.787712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84195114-6c8f2cd5-c2d98c49-adde8d90_0] Write session: Do CDS request
2025-10-09T21:47:59.787719Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84195114-6c8f2cd5-c2d98c49-adde8d90_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:47:59.793494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|84195114-6c8f2cd5-c2d98c49-adde8d90_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:59.793574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:47:59.793588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1089
2025-10-09T21:47:59.793753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1089 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:47:59.793798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1089 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:47:59.793804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1089 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:47:59.793808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:47:59.794243Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1089 sessionId:  describe result for acl check
2025-10-09T21:47:59.794272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:47:59.794274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:47:59.794276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:47:59.794281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342067994576938:9662] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:47:59.794883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342067994576938:9662] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:47:59.798018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342067994576938:9662] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:47:59.801339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342067994576938:9662] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:47:59.801354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342067994576938:9662] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:47:59.801358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342067994576938:9662] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:47:59.801367Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1089 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:47:59.801807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:47:59.803561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:47:59.803800Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:47:59.803823Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:47:59.803838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:47:59.803911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:47:59.803959Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342067632755722:8635]
2025-10-09T21:47:59.804556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:47:59.804955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:47:59.805020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:47:59.805365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:47:59.805421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:59.805427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:59.805428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:59.805430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:59.805431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:59.805432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:59.805434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.805435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.805436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.805443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:59.805454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:59.805459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:59.805463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:59.805467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:59.805471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:59.805476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.805480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.805484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.805493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:59.805501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:47:59.805513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:59.805519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:47:59.805530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:59.805537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:47:59.805541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:59.805546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:47:59.805551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:59.805556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:47:59.805560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:59.805570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:47:59.805576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.805586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.805590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.805596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.805601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.805606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.805643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:47:59.814457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:47:59.815613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:47:59.815631Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:47:59.815635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:47:59.815641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:47:59.815648Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1088 [2:7559342067632755722:8635]
2025-10-09T21:47:59.815667Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:47:59.815690Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:47:59.815694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:59.815698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:59.815700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:59.815720Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:47:59.815731Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:47:59.815740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:47:59.815815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.815826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.815828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.815832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:47:59.815909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:47:59.815917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.816090Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1088
2025-10-09T21:47:59.816158Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1088
2025-10-09T21:47:59.815930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:47:59.815945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:47:59.815963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:47:59.815986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342067632755717:8635]
2025-10-09T21:47:59.815988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342067632755717:8635]
2025-10-09T21:47:59.815991Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342067632755717:8635]
2025-10-09T21:47:59.816001Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:47:59.816013Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:47:59.816017Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:47:59.816019Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:47:59.816021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342067994576923:2432], now have 1 active actors on pipe
2025-10-09T21:47:59.816026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342067994577086:9662], now have 1 active actors on pipe
2025-10-09T21:47:59.816031Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:47:59.816048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:47:59.816051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:47:59.816093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342067632755717:8635]
2025-10-09T21:47:59.816102Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:47:59.816117Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:47:59.816363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:59.816371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:59.816401Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4b1aaa27-f3942547-6be2538-4ad6426_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:47:59.816409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:47:59.816414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:47:59.816417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:47:59.816429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:47:59.816440Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:47:59.816457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:59.816621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:59.816626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:59.816640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:47:59.816760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1089 partition: 0 MaxSeqNo: 25 sessionId: src_id|4b1aaa27-f3942547-6be2538-4ad6426_0
2025-10-09T21:47:59.817126Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84195114-6c8f2cd5-c2d98c49-adde8d90_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4b1aaa27-f3942547-6be2538-4ad6426_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:47:59.819769Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b1aaa27-f3942547-6be2538-4ad6426_0] Write session: try to update token
2025-10-09T21:47:59.819789Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b1aaa27-f3942547-6be2538-4ad6426_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:47:59.831171Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:47:59.831186Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:47:59.831189Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:47:59.831191Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:47:59.831201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342067632755717:8635]
2025-10-09T21:47:59.831206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342067632755717:8635]
2025-10-09T21:47:59.831209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342067632755717:8635]
2025-10-09T21:47:59.831260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:47:59.831267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:47:59.831266Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:47:59.831270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:47:59.831272Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:47:59.831278Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:47:59.831438Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:47:59.839912Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1089 sessionId: src_id|4b1aaa27-f3942547-6be2538-4ad6426_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:47:59.842423Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:47:59.890119Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342067994577098:9144]
2025-10-09T21:47:59.890488Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 1088, pipe: [1:7559342067994577098:9144]
2025-10-09T21:47:59.992214Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:47:59.992632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:47:59.992649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:47:59.992674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342067994577098:9144], now have 1 active actors on pipe
2025-10-09T21:47:59.992889Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:47:59.992914Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342067994577098:9144]
2025-10-09T21:47:59.992940Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:47:59.992951Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:47:59.992954Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:47:59.993029Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1089 reason: error at write: Disconnected sessionId: src_id|4b1aaa27-f3942547-6be2538-4ad6426_0
2025-10-09T21:47:59.993208Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1089 sessionId: src_id|4b1aaa27-f3942547-6be2538-4ad6426_0 is DEAD
2025-10-09T21:47:59.993355Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b1aaa27-f3942547-6be2538-4ad6426_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:47:59.993371Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b1aaa27-f3942547-6be2538-4ad6426_0] Write session will restart in 0.008130s
2025-10-09T21:47:59.993391Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b1aaa27-f3942547-6be2538-4ad6426_0] Write session: Do CDS request
2025-10-09T21:47:59.993396Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b1aaa27-f3942547-6be2538-4ad6426_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:00.002609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b1aaa27-f3942547-6be2538-4ad6426_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:00.002716Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:00.002733Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1090
2025-10-09T21:48:00.002926Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1090 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:00.002972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1090 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:00.002985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1090 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:00.002989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:00.003498Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1090 sessionId:  describe result for acl check
2025-10-09T21:48:00.003541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:00.003550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:00.003552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:00.003559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342072289544486:9667] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:00.004157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342072289544486:9667] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:00.006017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:00.006488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342072289544486:9667] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:00.008651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:00.008838Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:00.008847Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:00.008856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:00.008899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:00.008930Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342071927723182:8637]
2025-10-09T21:48:00.009023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342072289544486:9667] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:00.009037Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342072289544486:9667] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:00.009041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342072289544486:9667] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:00.009050Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1090 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:00.009377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:00.009645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:00.009709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:00.009991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:00.010054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.010060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.010062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.010063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.010065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.010065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.010066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.010067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.010068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.010072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.010080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.010082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.010084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.010086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.010088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.010091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.010093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.010095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.010100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.010105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.010110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.010113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.010116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.010119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.010122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.010125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.010128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.010131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.010134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.010137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.010140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.010143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.010146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.010149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.010152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.010155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.010180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:00.016978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:00.018199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:00.018213Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:00.018216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:00.018222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:00.018230Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1089 [2:7559342071927723182:8637]
2025-10-09T21:48:00.018246Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:00.018269Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:00.018276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:00.018280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:00.018283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:00.018304Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:00.018314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:00.018327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:00.018647Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1089
2025-10-09T21:48:00.018687Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1089
2025-10-09T21:48:00.018401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.018408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.018410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.018414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:00.018493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:00.018496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.018512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.018525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.018542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:00.018567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342067994577105:2432], now have 1 active actors on pipe
2025-10-09T21:48:00.018571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342072289544559:9667], now have 1 active actors on pipe
2025-10-09T21:48:00.018579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342071927723177:8637]
2025-10-09T21:48:00.018582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342071927723177:8637]
2025-10-09T21:48:00.018584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342071927723177:8637]
2025-10-09T21:48:00.018597Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:00.018613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:00.018625Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:00.018628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:00.018629Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:00.018632Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:00.018632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:00.018686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342071927723177:8637]
2025-10-09T21:48:00.018690Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:00.018707Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:00.018724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.018731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.018772Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1581dc65-54ce7cd7-fe6b8bb4-7276cbb4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:00.018780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:00.018784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:00.018787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:00.018798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:00.018810Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:00.018832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:00.018997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.019009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.019044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:00.019140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1090 partition: 0 MaxSeqNo: 25 sessionId: src_id|1581dc65-54ce7cd7-fe6b8bb4-7276cbb4_0
2025-10-09T21:48:00.019528Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b1aaa27-f3942547-6be2538-4ad6426_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1581dc65-54ce7cd7-fe6b8bb4-7276cbb4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:00.022264Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1581dc65-54ce7cd7-fe6b8bb4-7276cbb4_0] Write session: try to update token
2025-10-09T21:48:00.022282Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1581dc65-54ce7cd7-fe6b8bb4-7276cbb4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:00.033601Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:00.033625Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:00.033628Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:00.033630Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:00.033641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342071927723177:8637]
2025-10-09T21:48:00.033647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342071927723177:8637]
2025-10-09T21:48:00.033650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342071927723177:8637]
2025-10-09T21:48:00.033689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:00.033696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.033699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:00.033705Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:00.033713Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:00.033717Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:00.033861Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:00.036775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1090 sessionId: src_id|1581dc65-54ce7cd7-fe6b8bb4-7276cbb4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:00.039292Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:00.093176Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342072289544571:9144]
2025-10-09T21:48:00.093397Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 1089, pipe: [1:7559342072289544571:9144]
2025-10-09T21:48:00.198311Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:00.198620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.198638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.198688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342072289544571:9144], now have 1 active actors on pipe
2025-10-09T21:48:00.198930Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:00.198973Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:00.198994Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:00.198998Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:00.199049Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342072289544571:9144]
2025-10-09T21:48:00.199067Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1090 reason: error at write: Disconnected sessionId: src_id|1581dc65-54ce7cd7-fe6b8bb4-7276cbb4_0
2025-10-09T21:48:00.199267Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1090 sessionId: src_id|1581dc65-54ce7cd7-fe6b8bb4-7276cbb4_0 is DEAD
2025-10-09T21:48:00.199423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1581dc65-54ce7cd7-fe6b8bb4-7276cbb4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:00.199435Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1581dc65-54ce7cd7-fe6b8bb4-7276cbb4_0] Write session will restart in 0.009642s
2025-10-09T21:48:00.199463Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1581dc65-54ce7cd7-fe6b8bb4-7276cbb4_0] Write session: Do CDS request
2025-10-09T21:48:00.199470Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1581dc65-54ce7cd7-fe6b8bb4-7276cbb4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:00.210561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1581dc65-54ce7cd7-fe6b8bb4-7276cbb4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:00.210642Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:00.210661Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1091
2025-10-09T21:48:00.210820Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1091 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:00.210864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1091 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:00.210874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1091 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:00.210878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:00.211316Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1091 sessionId:  describe result for acl check
2025-10-09T21:48:00.211369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:00.211375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:00.211377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:00.211383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342072289544711:9672] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:00.211940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342072289544711:9672] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:00.212925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:00.213895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342072289544711:9672] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:00.215353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:00.215575Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:00.215587Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:00.215599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:00.215656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:00.215698Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342071927723377:8649]
2025-10-09T21:48:00.216336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:00.216641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:00.216685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:00.216817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342072289544711:9672] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:00.216834Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342072289544711:9672] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:00.216837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342072289544711:9672] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:00.216847Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1091 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:00.217008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:00.217058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.217063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.217064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.217065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.217065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.217066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.217067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.217067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.217068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.217072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.217079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.217081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.217084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.217086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.217089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.217091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.217093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.217095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.217102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.217109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.217115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.217120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.217125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.217130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.217132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.217135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.217138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.217141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.217144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.217147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.217150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.217153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.217156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.217159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.217162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.217165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.217182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:00.224135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:00.225460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:00.225476Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:00.225479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:00.225484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:00.225492Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1090 [2:7559342071927723377:8649]
2025-10-09T21:48:00.225508Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:00.225529Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:00.225538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:00.225542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:00.225544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:00.225563Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:00.225576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:00.225585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:00.225648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.225655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.225657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.225660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:00.225738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:00.225746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.225760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.225799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.225819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:00.225844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342072289544747:9672], now have 1 active actors on pipe
2025-10-09T21:48:00.225853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342072289544580:2432], now have 1 active actors on pipe
2025-10-09T21:48:00.225862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342071927723372:8649]
2025-10-09T21:48:00.225865Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342071927723372:8649]
2025-10-09T21:48:00.225867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342071927723372:8649]
2025-10-09T21:48:00.225880Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:00.225895Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:00.225896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:00.225904Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:00.225906Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:00.225913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:00.225917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:00.225964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342071927723372:8649]
2025-10-09T21:48:00.225977Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:00.225992Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:00.225968Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1090
2025-10-09T21:48:00.225967Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1090
2025-10-09T21:48:00.226022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.226053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.226083Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8739b03-b7e027b7-b3339cac-e4fc0cd2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:00.226098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:00.226102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:00.226106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:00.226118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:00.226129Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:00.226145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:00.226334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.226345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.226370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:00.226468Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1091 partition: 0 MaxSeqNo: 25 sessionId: src_id|8739b03-b7e027b7-b3339cac-e4fc0cd2_0
2025-10-09T21:48:00.228110Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1581dc65-54ce7cd7-fe6b8bb4-7276cbb4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8739b03-b7e027b7-b3339cac-e4fc0cd2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:00.230638Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8739b03-b7e027b7-b3339cac-e4fc0cd2_0] Write session: try to update token
2025-10-09T21:48:00.230659Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8739b03-b7e027b7-b3339cac-e4fc0cd2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:00.238344Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:00.238361Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:00.238373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342071927723372:8649]
2025-10-09T21:48:00.238421Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:00.238425Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:00.238623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:00.238767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:00.238903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:00.239043Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:00.239260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:00.239308Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:00.245068Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1091 sessionId: src_id|8739b03-b7e027b7-b3339cac-e4fc0cd2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:00.247633Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:00.247845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.247873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.247915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:00.248007Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:00.252087Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:00.252103Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:00.252106Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:00.252108Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:00.252118Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342071927723372:8649]
2025-10-09T21:48:00.252122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342071927723372:8649]
2025-10-09T21:48:00.252125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342071927723372:8649]
2025-10-09T21:48:00.252178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:00.252182Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:00.252188Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:00.252190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.252193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:00.252200Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:00.252359Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:00.298965Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342072289544782:9144]
2025-10-09T21:48:00.299203Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 1090, pipe: [1:7559342072289544782:9144]
2025-10-09T21:48:00.414315Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:00.421003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.421027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.421194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.421253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.421314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.421372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.421415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.421427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.421486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.421541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.421596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.421656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.421701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.421708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.421760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.421832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.421900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.421961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.422004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.422012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.422148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.422218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.422301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.422361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.422401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.422406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.422465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.422523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.422587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.422649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.422688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.422694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.422758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.422817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.422897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.422948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.422990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.422995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.423055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.423109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.423165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.423226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.423274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.423281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.423345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.423400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.423456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.423517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.423580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.423588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.423646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.423705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.423763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.423824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.423887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.423895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.423955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.424013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.424071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.424130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.424170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.424176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.424224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342072289544782:9144], now have 1 active actors on pipe
2025-10-09T21:48:00.424499Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342072289544782:9144]
2025-10-09T21:48:00.424518Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:00.424523Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:00.424530Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:00.424564Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1091 reason: error at write: Disconnected sessionId: src_id|8739b03-b7e027b7-b3339cac-e4fc0cd2_0
2025-10-09T21:48:00.424766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1091 sessionId: src_id|8739b03-b7e027b7-b3339cac-e4fc0cd2_0 is DEAD
2025-10-09T21:48:00.424911Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8739b03-b7e027b7-b3339cac-e4fc0cd2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:00.424921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8739b03-b7e027b7-b3339cac-e4fc0cd2_0] Write session will restart in 0.007149s
2025-10-09T21:48:00.424940Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8739b03-b7e027b7-b3339cac-e4fc0cd2_0] Write session: Do CDS request
2025-10-09T21:48:00.424944Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8739b03-b7e027b7-b3339cac-e4fc0cd2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:00.425065Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:00.433575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8739b03-b7e027b7-b3339cac-e4fc0cd2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:00.433602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:00.433616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1092
2025-10-09T21:48:00.433771Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1092 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:00.433820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1092 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:00.433827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1092 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:00.433831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:00.434336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1092 sessionId:  describe result for acl check
2025-10-09T21:48:00.434373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:00.434377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:00.434379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:00.434385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342072289544869:9683] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:00.434980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342072289544869:9683] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:00.437070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:00.437294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342072289544869:9683] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:00.439335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:00.439553Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:00.439570Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:00.439584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:00.439650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:00.439697Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342071927723553:8653]
2025-10-09T21:48:00.439793Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342072289544869:9683] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:00.439808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342072289544869:9683] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:00.439812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342072289544869:9683] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:00.439821Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1092 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:00.440316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:00.440659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:00.440728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:00.441044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:00.441116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.441124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.441125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.441126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.441127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.441129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.441130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.441131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.441132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.441138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.441149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.441153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.441157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.441161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.441165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.441169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.441173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.441177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.441186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.441201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.441207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.441212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.441216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.441220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.441223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.441228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.441231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.441235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.441239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.441243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.441247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.441251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.441255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.441259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.441263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.441273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.441298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:00.447604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:00.448810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:00.448829Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:00.448833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:00.448837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:00.448846Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1091 [2:7559342071927723553:8653]
2025-10-09T21:48:00.448858Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:00.448875Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:00.448884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:00.448887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:00.448889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:00.448905Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:00.448916Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:00.448923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:00.448983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.448989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.448991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.448994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:00.449063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:00.449072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.449083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.449094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.449108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:00.449128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342072289544948:9683], now have 1 active actors on pipe
2025-10-09T21:48:00.449142Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342071927723548:8653]
2025-10-09T21:48:00.449148Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342071927723548:8653]
2025-10-09T21:48:00.449150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342071927723548:8653]
2025-10-09T21:48:00.449163Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:00.449172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342072289544783:2432], now have 1 active actors on pipe
2025-10-09T21:48:00.449181Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:00.449182Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:00.449186Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:00.449188Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:00.449196Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:00.449199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:00.449228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342071927723548:8653]
2025-10-09T21:48:00.449238Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:00.449244Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:00.449227Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1091
2025-10-09T21:48:00.449231Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1091
2025-10-09T21:48:00.449322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.449334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.449365Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|72491935-2c651185-6ae486e7-c0999e89_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:00.449377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:00.449381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:00.449385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:00.449398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:00.449412Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:00.449431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:00.449573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.449584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.449599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:00.449738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1092 partition: 0 MaxSeqNo: 25 sessionId: src_id|72491935-2c651185-6ae486e7-c0999e89_0
2025-10-09T21:48:00.450148Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8739b03-b7e027b7-b3339cac-e4fc0cd2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|72491935-2c651185-6ae486e7-c0999e89_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:00.452588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72491935-2c651185-6ae486e7-c0999e89_0] Write session: try to update token
2025-10-09T21:48:00.452605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72491935-2c651185-6ae486e7-c0999e89_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:00.460885Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:00.460901Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:00.460911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342071927723548:8653]
2025-10-09T21:48:00.460963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:00.460963Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:00.461146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:00.461287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:00.461444Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:00.461599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:00.461825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:00.461868Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:00.465899Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1092 sessionId: src_id|72491935-2c651185-6ae486e7-c0999e89_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:00.468360Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:00.468524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.468544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.468588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:00.468656Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:00.474335Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:00.474349Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:00.474352Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:00.474354Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:00.474365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342071927723548:8653]
2025-10-09T21:48:00.474370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342071927723548:8653]
2025-10-09T21:48:00.474374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342071927723548:8653]
2025-10-09T21:48:00.474415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:00.474422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.474425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:00.474427Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:00.474434Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:00.474439Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:00.474559Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:00.524799Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342072289544963:9144]
2025-10-09T21:48:00.525154Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 1091, pipe: [1:7559342072289544963:9144]
2025-10-09T21:48:00.625872Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:00.633581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.633606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.633768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.633836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.633899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.633965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.634009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.634023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.634111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.634177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.634240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.634303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.634349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.634356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.634415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.634475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.634535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.634596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.634638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.634646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.634713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.634774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.634840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.634903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.634950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.634958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.635024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.635086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.635151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.635214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.635257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.635265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.635327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.635389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.635456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.635517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.635583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.635590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.635651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.635708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.635773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.635835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.635882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.635889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.635959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.636017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.636079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.636138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.636205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.636213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.636276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.636334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.636396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.636460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.636509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.636516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.636577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:00.636635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:00.636696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:00.636759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:00.636821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:00.636828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:00.636885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342072289544963:9144], now have 1 active actors on pipe
2025-10-09T21:48:00.637189Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:00.637203Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:00.637236Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1092 reason: error at write: Disconnected sessionId: src_id|72491935-2c651185-6ae486e7-c0999e89_0
2025-10-09T21:48:00.637235Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342072289544963:9144]
2025-10-09T21:48:00.637264Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:00.637430Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1092 sessionId: src_id|72491935-2c651185-6ae486e7-c0999e89_0 is DEAD
2025-10-09T21:48:00.637592Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72491935-2c651185-6ae486e7-c0999e89_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:00.637606Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72491935-2c651185-6ae486e7-c0999e89_0] Write session will restart in 0.009707s
2025-10-09T21:48:00.637632Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72491935-2c651185-6ae486e7-c0999e89_0] Write session: Do CDS request
2025-10-09T21:48:00.637639Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72491935-2c651185-6ae486e7-c0999e89_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:00.637916Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:00.648505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72491935-2c651185-6ae486e7-c0999e89_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:00.648654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:00.648668Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1093
2025-10-09T21:48:00.648829Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1093 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:00.648873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1093 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:00.648893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1093 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:00.648898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:00.649356Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1093 sessionId:  describe result for acl check
2025-10-09T21:48:00.649395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:00.649397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:00.649399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:00.649404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342072289545096:9687] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:00.649967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342072289545096:9687] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:00.650091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:00.652097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:00.652309Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:00.652318Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:00.652331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:00.652376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342072289545096:9687] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:00.652391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:00.652428Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342071927723725:8657]
2025-10-09T21:48:00.653023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:00.653294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:00.653326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:00.653566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:00.653602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.653603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.653605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.653606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.653607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.653608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.653609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.653610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.653611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.653616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.653627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.653631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.653635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.653639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.653643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.653647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.653651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.653655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.653660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.653668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.653675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.653680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.653685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.653690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.653695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.653700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.653704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.653709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.653713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.653717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.653721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.653726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.653730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.653734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.653739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.653756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.653797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:00.658231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342072289545096:9687] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:00.658247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342072289545096:9687] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:00.658251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342072289545096:9687] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:00.658260Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1093 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:00.660195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:00.661220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:00.661233Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:00.661234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:00.661238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:00.661244Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1092 [2:7559342071927723725:8657]
2025-10-09T21:48:00.661258Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:00.661275Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:00.661279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:00.661282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:00.661284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:00.661300Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:00.661311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:00.661319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:00.661389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.661396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.661399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.661402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:00.661467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:00.661476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.661488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.661500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.661515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:00.661542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342072289545136:9687], now have 1 active actors on pipe
2025-10-09T21:48:00.661554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342071927723720:8657]
2025-10-09T21:48:00.661556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342071927723720:8657]
2025-10-09T21:48:00.661559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342071927723720:8657]
2025-10-09T21:48:00.661577Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:00.661587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342072289544967:2432], now have 1 active actors on pipe
2025-10-09T21:48:00.661596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:00.661604Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:00.661609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:00.661609Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:00.661613Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:00.661617Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1092
2025-10-09T21:48:00.661613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:00.661641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342071927723720:8657]
2025-10-09T21:48:00.661620Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1092
2025-10-09T21:48:00.661644Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:00.661654Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:00.661676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.661680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.661710Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|91dbfd3c-971bd071-32519a8c-a8bd4293_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:00.661723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:00.661727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:00.661734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:00.661744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:00.661759Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:00.661787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:00.661944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.661955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.661971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:00.662076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1093 partition: 0 MaxSeqNo: 25 sessionId: src_id|91dbfd3c-971bd071-32519a8c-a8bd4293_0
2025-10-09T21:48:00.662431Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72491935-2c651185-6ae486e7-c0999e89_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|91dbfd3c-971bd071-32519a8c-a8bd4293_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:00.664994Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91dbfd3c-971bd071-32519a8c-a8bd4293_0] Write session: try to update token
2025-10-09T21:48:00.665015Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91dbfd3c-971bd071-32519a8c-a8bd4293_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:00.675685Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:00.675700Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:00.675702Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:00.675704Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:00.675714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342071927723720:8657]
2025-10-09T21:48:00.675719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342071927723720:8657]
2025-10-09T21:48:00.675723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342071927723720:8657]
2025-10-09T21:48:00.675767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:00.675770Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:00.675774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.675776Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:00.675777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:00.675781Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:00.675913Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:00.678092Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1093 sessionId: src_id|91dbfd3c-971bd071-32519a8c-a8bd4293_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:00.680358Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:00.737247Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342072289545143:9144]
2025-10-09T21:48:00.737547Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 1092, pipe: [1:7559342072289545143:9144]
2025-10-09T21:48:00.825903Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:00.826453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.826472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.826524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342072289545143:9144], now have 1 active actors on pipe
2025-10-09T21:48:00.826760Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:00.826782Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342072289545143:9144]
2025-10-09T21:48:00.826818Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:00.826822Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:00.826890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1093 reason: error at write: Disconnected sessionId: src_id|91dbfd3c-971bd071-32519a8c-a8bd4293_0
2025-10-09T21:48:00.826955Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:00.827083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1093 sessionId: src_id|91dbfd3c-971bd071-32519a8c-a8bd4293_0 is DEAD
2025-10-09T21:48:00.827273Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91dbfd3c-971bd071-32519a8c-a8bd4293_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:00.827285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91dbfd3c-971bd071-32519a8c-a8bd4293_0] Write session will restart in 0.008025s
2025-10-09T21:48:00.827314Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91dbfd3c-971bd071-32519a8c-a8bd4293_0] Write session: Do CDS request
2025-10-09T21:48:00.827321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91dbfd3c-971bd071-32519a8c-a8bd4293_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:00.836558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91dbfd3c-971bd071-32519a8c-a8bd4293_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:00.836755Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:00.836770Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1094
2025-10-09T21:48:00.836874Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1094 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:00.836907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1094 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:00.836919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1094 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:00.836923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:00.837383Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1094 sessionId:  describe result for acl check
2025-10-09T21:48:00.837419Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:00.837425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:00.837427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:00.837433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342072289545267:9692] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:00.838016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342072289545267:9692] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:00.839117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:00.840252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342072289545267:9692] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:00.841038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:00.841217Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:00.841225Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:00.841234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:00.841276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:00.841310Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342071927723893:8659]
2025-10-09T21:48:00.841728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:00.842024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:00.842075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:00.842307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:00.842344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.842349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.842350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.842350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.842351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.842352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.842352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.842353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.842354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.842357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.842375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.842379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.842383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.842386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.842390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.842394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.842398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.842403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.842409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.842416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:00.842422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.842426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:00.842436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.842440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:00.842444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.842450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:00.842456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.842461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:00.842466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.842476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:00.842481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.842491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.842494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.842497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.842501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.842504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.842520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:00.842933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342072289545267:9692] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:00.842948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342072289545267:9692] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:00.842953Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342072289545267:9692] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:00.842962Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1094 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:00.848957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:00.850062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:00.850074Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:00.850077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:00.850082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:00.850090Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1093 [2:7559342071927723893:8659]
2025-10-09T21:48:00.850102Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:00.850119Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:00.850121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:00.850123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:00.850125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:00.850139Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:00.850143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:00.850149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:00.850193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.850200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.850201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.850203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:00.850264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:00.850273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.850284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:00.850295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:00.850311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:00.850333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342071927723888:8659]
2025-10-09T21:48:00.850339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342071927723888:8659]
2025-10-09T21:48:00.850342Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342071927723888:8659]
2025-10-09T21:48:00.850352Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:00.850363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342072289545148:2432], now have 1 active actors on pipe
2025-10-09T21:48:00.850367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342072289545318:9692], now have 1 active actors on pipe
2025-10-09T21:48:00.850367Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:00.850371Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:00.850373Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:00.850374Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:00.850382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:00.850385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:00.850421Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342071927723888:8659]
2025-10-09T21:48:00.850437Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:00.850459Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:00.850462Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1093
2025-10-09T21:48:00.850516Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1093
2025-10-09T21:48:00.850611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.850624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.850654Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|45f9bf99-4d8ab6-8f1e67d0-51a5a51e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:00.850667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:00.850672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:00.850675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:00.850687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:00.850705Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:00.850732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:00.850901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.850910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.850924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:00.851023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1094 partition: 0 MaxSeqNo: 25 sessionId: src_id|45f9bf99-4d8ab6-8f1e67d0-51a5a51e_0
2025-10-09T21:48:00.851398Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91dbfd3c-971bd071-32519a8c-a8bd4293_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|45f9bf99-4d8ab6-8f1e67d0-51a5a51e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:00.853759Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45f9bf99-4d8ab6-8f1e67d0-51a5a51e_0] Write session: try to update token
2025-10-09T21:48:00.853790Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45f9bf99-4d8ab6-8f1e67d0-51a5a51e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:00.862430Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:00.862446Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:00.862457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342071927723888:8659]
2025-10-09T21:48:00.862509Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:00.862530Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:00.862683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:00.862822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:00.862956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:00.863091Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:00.863273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:00.863315Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:00.866884Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1094 sessionId: src_id|45f9bf99-4d8ab6-8f1e67d0-51a5a51e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:00.869212Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:00.869411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:00.869431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:00.869464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:00.869588Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:00.875614Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:00.875627Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:00.875630Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:00.875632Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:00.875642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342071927723888:8659]
2025-10-09T21:48:00.875647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342071927723888:8659]
2025-10-09T21:48:00.875650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342071927723888:8659]
2025-10-09T21:48:00.875694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:00.875701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:00.875703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:00.875706Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:00.875712Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:00.875717Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:00.875829Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:00.926727Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342072289545325:9144]
2025-10-09T21:48:00.930318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 1093, pipe: [1:7559342072289545325:9144]
2025-10-09T21:48:01.034545Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:01.042525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:01.042557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:01.042726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.042794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.042859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.042932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.042980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.042991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.043060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.043129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.043204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.043274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.043316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.043325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.043389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.043450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.043520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.043581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.043625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.043633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.043701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.043762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.043823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.043887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.043930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.043937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.044004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.044069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.044128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.044209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.044253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.044261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.044326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.044387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.044451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.044512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.044554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.044563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.044621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.044674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.044727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.044778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.044819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.044826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.044889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.044947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.045013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.045076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.045137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.045144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.045203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.045258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.045319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.045381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.045430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.045436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.045496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.045554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.045616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.045677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.045824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.045833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.045901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342072289545325:9144], now have 1 active actors on pipe
2025-10-09T21:48:01.046165Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:01.046180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:01.046203Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:01.046213Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1094 reason: error at write: Disconnected sessionId: src_id|45f9bf99-4d8ab6-8f1e67d0-51a5a51e_0
2025-10-09T21:48:01.046268Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342072289545325:9144]
2025-10-09T21:48:01.046405Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1094 sessionId: src_id|45f9bf99-4d8ab6-8f1e67d0-51a5a51e_0 is DEAD
2025-10-09T21:48:01.046551Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45f9bf99-4d8ab6-8f1e67d0-51a5a51e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:01.046567Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45f9bf99-4d8ab6-8f1e67d0-51a5a51e_0] Write session will restart in 0.007162s
2025-10-09T21:48:01.046593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45f9bf99-4d8ab6-8f1e67d0-51a5a51e_0] Write session: Do CDS request
2025-10-09T21:48:01.046601Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45f9bf99-4d8ab6-8f1e67d0-51a5a51e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:01.047071Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:01.054521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45f9bf99-4d8ab6-8f1e67d0-51a5a51e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:01.054643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:01.054660Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1095
2025-10-09T21:48:01.054765Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1095 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:01.054798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1095 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:01.054804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1095 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:01.054808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:01.055367Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1095 sessionId:  describe result for acl check
2025-10-09T21:48:01.055408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:01.055416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:01.055417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:01.055424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342076584512670:9696] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:01.056011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342076584512670:9696] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:01.059725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342076584512670:9696] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:01.061471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:01.063950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:01.064166Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:01.064178Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:01.064191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:01.064251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:01.064292Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342076222691358:8661]
2025-10-09T21:48:01.064371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342076584512670:9696] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:01.064388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342076584512670:9696] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:01.064391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342076584512670:9696] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:01.064401Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1095 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:01.064915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:01.065202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:01.065253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:01.065570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:01.065629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.065630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.065632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.065633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.065634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.065635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.065637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.065637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.065639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.065644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.065654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.065658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.065662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.065665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.065669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.065673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.065677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.065681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.065688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.065697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.065703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.065708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.065712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.065717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.065721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.065725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.065730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.065735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.065739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.065744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.065749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.065754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.065758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.065762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.065766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.065770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.065809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:01.072921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:01.073995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:01.074011Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:01.074015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:01.074020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:01.074026Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1094 [2:7559342076222691358:8661]
2025-10-09T21:48:01.074059Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:01.074081Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:01.074084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:01.074088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:01.074091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:01.074109Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:01.074116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:01.074123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:01.074186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.074196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.074198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.074202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:01.074281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:01.074288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.074300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.074310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.074331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:01.074362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342076584512794:9696], now have 1 active actors on pipe
2025-10-09T21:48:01.074370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342076584512625:2432], now have 1 active actors on pipe
2025-10-09T21:48:01.074378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342076222691353:8661]
2025-10-09T21:48:01.074385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342076222691353:8661]
2025-10-09T21:48:01.074388Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342076222691353:8661]
2025-10-09T21:48:01.074402Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:01.074419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:01.074418Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:01.074421Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:01.074423Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:01.074434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:01.074438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:01.074477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342076222691353:8661]
2025-10-09T21:48:01.074489Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:01.074496Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:01.074506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:01.074516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:01.074569Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5f1cd11-c1da82af-34fcb29d-4c3f5c9c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:01.074582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:01.074586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:01.074588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:01.074600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:01.074615Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:01.074642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:01.075088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:01.075108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:01.075126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:01.075680Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45f9bf99-4d8ab6-8f1e67d0-51a5a51e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5f1cd11-c1da82af-34fcb29d-4c3f5c9c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:01.074427Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1094
2025-10-09T21:48:01.074428Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1094
2025-10-09T21:48:01.075280Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1095 partition: 0 MaxSeqNo: 25 sessionId: src_id|5f1cd11-c1da82af-34fcb29d-4c3f5c9c_0
2025-10-09T21:48:01.078272Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f1cd11-c1da82af-34fcb29d-4c3f5c9c_0] Write session: try to update token
2025-10-09T21:48:01.078293Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f1cd11-c1da82af-34fcb29d-4c3f5c9c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:01.089842Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:01.089859Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:01.089862Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:01.089864Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:01.089875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342076222691353:8661]
2025-10-09T21:48:01.089880Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342076222691353:8661]
2025-10-09T21:48:01.089884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342076222691353:8661]
2025-10-09T21:48:01.089930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:01.089935Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:01.089938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.089941Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:01.089941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:01.089946Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:01.090098Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:01.093197Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1095 sessionId: src_id|5f1cd11-c1da82af-34fcb29d-4c3f5c9c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:01.095775Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:01.146322Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342076584512809:9144]
2025-10-09T21:48:01.150289Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 1094, pipe: [1:7559342076584512809:9144]
2025-10-09T21:48:01.254735Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:01.255186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:01.255201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:01.255251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342076584512809:9144], now have 1 active actors on pipe
2025-10-09T21:48:01.255448Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:01.255515Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:01.255523Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342076584512809:9144]
2025-10-09T21:48:01.255527Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:01.255543Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:01.255593Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1095 reason: error at write: Disconnected sessionId: src_id|5f1cd11-c1da82af-34fcb29d-4c3f5c9c_0
2025-10-09T21:48:01.255976Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f1cd11-c1da82af-34fcb29d-4c3f5c9c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:01.255988Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f1cd11-c1da82af-34fcb29d-4c3f5c9c_0] Write session will restart in 0.008176s
2025-10-09T21:48:01.256013Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f1cd11-c1da82af-34fcb29d-4c3f5c9c_0] Write session: Do CDS request
2025-10-09T21:48:01.256023Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f1cd11-c1da82af-34fcb29d-4c3f5c9c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:01.255867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1095 sessionId: src_id|5f1cd11-c1da82af-34fcb29d-4c3f5c9c_0 is DEAD
2025-10-09T21:48:01.265821Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f1cd11-c1da82af-34fcb29d-4c3f5c9c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:01.266258Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:01.266274Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1096
2025-10-09T21:48:01.266386Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1096 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:01.266415Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1096 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:01.266421Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1096 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:01.266427Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:01.266930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1096 sessionId:  describe result for acl check
2025-10-09T21:48:01.266964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:01.266966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:01.266968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:01.266975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342076584512834:9701] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:01.267551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342076584512834:9701] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:01.269956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342076584512834:9701] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:01.273397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342076584512834:9701] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:01.273413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342076584512834:9701] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:01.273418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342076584512834:9701] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:01.273444Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1096 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:01.275966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:01.278583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:01.278807Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:01.278818Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:01.278830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:01.278913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:01.278958Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342076222691560:8673]
2025-10-09T21:48:01.279440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:01.279797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:01.279848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:01.280170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:01.280245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.280252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.280253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.280254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.280255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.280257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.280258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.280259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.280261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.280266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.280278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.280282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.280285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.280289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.280293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.280297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.280301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.280304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.280313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.280321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.280333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.280338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.280347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.280352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.280357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.280366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.280370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.280375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.280380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.280385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.280394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.280399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.280403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.280408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.280419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.280424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.280454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:01.288380Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1095
2025-10-09T21:48:01.288433Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1095
2025-10-09T21:48:01.287023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:01.288035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:01.288039Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:01.288042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:01.288045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:01.288051Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1095 [2:7559342076222691560:8673]
2025-10-09T21:48:01.288060Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:01.288075Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:01.288078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:01.288081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:01.288084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:01.288101Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:01.288107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:01.288131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:01.288184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.288186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.288187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.288189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:01.288247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:01.288249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.288257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.288264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.288274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:01.288289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342076222691555:8673]
2025-10-09T21:48:01.288291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342076222691555:8673]
2025-10-09T21:48:01.288293Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342076222691555:8673]
2025-10-09T21:48:01.288299Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:01.288311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:01.288315Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:01.288320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:01.288322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:01.288327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:01.288330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:01.288333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342076584512818:2432], now have 1 active actors on pipe
2025-10-09T21:48:01.288337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342076584512976:9701], now have 1 active actors on pipe
2025-10-09T21:48:01.288368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342076222691555:8673]
2025-10-09T21:48:01.288377Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:01.288390Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:01.288680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:01.288687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:01.288710Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|aa14c15f-bd0a47b-a556ae62-d8ccdbdb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:01.288716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:01.288720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:01.288723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:01.288741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:01.288754Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:01.288769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:01.288927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:01.288931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:01.288941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:01.289033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1096 partition: 0 MaxSeqNo: 25 sessionId: src_id|aa14c15f-bd0a47b-a556ae62-d8ccdbdb_0
2025-10-09T21:48:01.289411Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f1cd11-c1da82af-34fcb29d-4c3f5c9c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|aa14c15f-bd0a47b-a556ae62-d8ccdbdb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:01.292177Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aa14c15f-bd0a47b-a556ae62-d8ccdbdb_0] Write session: try to update token
2025-10-09T21:48:01.292198Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aa14c15f-bd0a47b-a556ae62-d8ccdbdb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:01.303387Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:01.303411Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:01.303413Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:01.303416Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:01.303425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342076222691555:8673]
2025-10-09T21:48:01.303430Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342076222691555:8673]
2025-10-09T21:48:01.303434Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342076222691555:8673]
2025-10-09T21:48:01.303485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:01.303493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.303495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:01.303501Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:01.303509Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:01.303513Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:01.303634Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:01.307521Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1096 sessionId: src_id|aa14c15f-bd0a47b-a556ae62-d8ccdbdb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:01.309982Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:01.355753Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342076584513020:9144]
2025-10-09T21:48:01.357269Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 1095, pipe: [1:7559342076584513020:9144]
2025-10-09T21:48:01.466148Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:01.466713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:01.466731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:01.466780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342076584513020:9144], now have 1 active actors on pipe
2025-10-09T21:48:01.467041Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:01.467094Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342076584513020:9144]
2025-10-09T21:48:01.467107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:01.467110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:01.467048Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:01.467187Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1096 reason: error at write: Disconnected sessionId: src_id|aa14c15f-bd0a47b-a556ae62-d8ccdbdb_0
2025-10-09T21:48:01.467383Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1096 sessionId: src_id|aa14c15f-bd0a47b-a556ae62-d8ccdbdb_0 is DEAD
2025-10-09T21:48:01.467567Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa14c15f-bd0a47b-a556ae62-d8ccdbdb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:01.467579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa14c15f-bd0a47b-a556ae62-d8ccdbdb_0] Write session will restart in 0.008958s
2025-10-09T21:48:01.467623Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa14c15f-bd0a47b-a556ae62-d8ccdbdb_0] Write session: Do CDS request
2025-10-09T21:48:01.467630Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa14c15f-bd0a47b-a556ae62-d8ccdbdb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:01.477588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aa14c15f-bd0a47b-a556ae62-d8ccdbdb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:01.477710Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:01.477725Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1097
2025-10-09T21:48:01.478332Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1097 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:01.478381Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1097 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:01.478393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1097 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:01.478398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:01.478852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1097 sessionId:  describe result for acl check
2025-10-09T21:48:01.478886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:01.478893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:01.478895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:01.478901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342076584513097:9712] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:01.479480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342076584513097:9712] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:01.482062Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342076584513097:9712] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:01.485195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:01.485597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342076584513097:9712] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:01.485613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342076584513097:9712] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:01.485617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342076584513097:9712] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:01.485626Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1097 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:01.485750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:01.485938Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:01.485947Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:01.485958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:01.486006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:01.486062Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342076222691739:8677]
2025-10-09T21:48:01.486574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:01.487014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:01.487083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:01.487366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:01.487405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.487411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.487412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.487413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.487415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.487416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.487417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.487418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.487419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.487425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.487437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.487440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.487443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.487445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.487447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.487449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.487452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.487454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.487460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.487466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.487470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.487474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.487477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.487480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.487483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.487486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.487489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.487521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.487547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.487561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.487566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.487571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.487575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.487581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.487586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.487591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.487614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:01.494446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:01.495508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:01.495524Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:01.495527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:01.495532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:01.495540Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1096 [2:7559342076222691739:8677]
2025-10-09T21:48:01.495556Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:01.495574Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:01.495587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:01.495590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:01.495593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:01.495611Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:01.495617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:01.495624Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:01.495686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.495694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.495696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.495699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:01.495773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:01.495782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.495794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.495805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.495825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:01.495851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342076584513198:9712], now have 1 active actors on pipe
2025-10-09T21:48:01.495859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342076584513021:2432], now have 1 active actors on pipe
2025-10-09T21:48:01.495868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342076222691733:8677]
2025-10-09T21:48:01.495875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342076222691733:8677]
2025-10-09T21:48:01.495877Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342076222691733:8677]
2025-10-09T21:48:01.495890Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:01.495909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:01.495916Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:01.495920Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:01.495923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:01.495928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:01.495932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:01.495934Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1096
2025-10-09T21:48:01.495935Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1096
2025-10-09T21:48:01.495970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342076222691733:8677]
2025-10-09T21:48:01.495978Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:01.495981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:01.495994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:01.495998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:01.496049Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|659a1480-90bc7920-9cc335a9-1d5042db_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:01.496062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:01.496068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:01.496074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:01.496084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:01.496099Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:01.496124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:01.496316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:01.496324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:01.496340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:01.496547Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1097 partition: 0 MaxSeqNo: 25 sessionId: src_id|659a1480-90bc7920-9cc335a9-1d5042db_0
2025-10-09T21:48:01.496947Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa14c15f-bd0a47b-a556ae62-d8ccdbdb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|659a1480-90bc7920-9cc335a9-1d5042db_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:01.499435Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|659a1480-90bc7920-9cc335a9-1d5042db_0] Write session: try to update token
2025-10-09T21:48:01.499456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|659a1480-90bc7920-9cc335a9-1d5042db_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:01.510682Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:01.510698Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:01.510700Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:01.510701Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:01.510710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342076222691733:8677]
2025-10-09T21:48:01.510713Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342076222691733:8677]
2025-10-09T21:48:01.510718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342076222691733:8677]
2025-10-09T21:48:01.510766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:01.510774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.510777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:01.510785Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:01.510810Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:01.510818Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:01.510919Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:01.513111Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1097 sessionId: src_id|659a1480-90bc7920-9cc335a9-1d5042db_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:01.515503Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:01.567238Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342076584513209:9144]
2025-10-09T21:48:01.567517Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 1096, pipe: [1:7559342076584513209:9144]
2025-10-09T21:48:01.626432Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: CLIENT_DEADLINE_EXCEEDED. Description: [ { <main>: Error: GRpc error: (4): Deadline Exceeded } { <main>: Error: Grpc error response on endpoint localhost:14704 } ]
2025-10-09T21:48:01.669373Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:01.669930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:01.669944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:01.669982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342076584513209:9144], now have 1 active actors on pipe
2025-10-09T21:48:01.670189Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:01.670222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:01.670235Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:01.670244Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:01.670297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1097 reason: error at write: Disconnected sessionId: src_id|659a1480-90bc7920-9cc335a9-1d5042db_0
2025-10-09T21:48:01.670297Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342076584513209:9144]
2025-10-09T21:48:01.670448Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1097 sessionId: src_id|659a1480-90bc7920-9cc335a9-1d5042db_0 is DEAD
2025-10-09T21:48:01.670593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|659a1480-90bc7920-9cc335a9-1d5042db_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:01.670605Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|659a1480-90bc7920-9cc335a9-1d5042db_0] Write session will restart in 0.008859s
2025-10-09T21:48:01.670623Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|659a1480-90bc7920-9cc335a9-1d5042db_0] Write session: Do CDS request
2025-10-09T21:48:01.670628Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|659a1480-90bc7920-9cc335a9-1d5042db_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:01.680614Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|659a1480-90bc7920-9cc335a9-1d5042db_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:01.680702Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:01.680720Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1098
2025-10-09T21:48:01.680842Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1098 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:01.680875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1098 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:01.680884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1098 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:01.680892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:01.681321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1098 sessionId:  describe result for acl check
2025-10-09T21:48:01.681359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:01.681365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:01.681367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:01.681373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342076584513344:9716] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:01.681891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342076584513344:9716] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:01.682679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:01.683682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342076584513344:9716] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:01.684896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:01.685099Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:01.685110Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:01.685122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:01.685176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:01.685217Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342076222691915:8681]
2025-10-09T21:48:01.685856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:01.686160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342076584513344:9716] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:01.686176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342076584513344:9716] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:01.686180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342076584513344:9716] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:01.686189Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1098 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:01.686339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:01.686390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:01.686739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:01.686789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.686795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.686797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.686798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.686799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.686800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.686802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.686803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.686804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.686810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.686821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.686825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.686830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.686834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.686838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.686842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.686846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.686851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.686857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.686865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.686871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.686882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.686886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.686892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.686901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.686906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.686916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.686921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.686925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.686931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.686940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.686945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.686949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.686959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.686964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.686974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.687003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:01.693456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:01.694544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:01.694555Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:01.694556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:01.694560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:01.694566Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1097 [2:7559342076222691915:8681]
2025-10-09T21:48:01.694577Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:01.694595Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:01.694598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:01.694601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:01.694604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:01.694621Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:01.694632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:01.694639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:01.694689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.694696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.694698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.694701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:01.694936Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1097
2025-10-09T21:48:01.694937Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1097
2025-10-09T21:48:01.694987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:01.694998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.695010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.695027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.695049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:01.695075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342076584513384:9716], now have 1 active actors on pipe
2025-10-09T21:48:01.695083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342076584513212:2432], now have 1 active actors on pipe
2025-10-09T21:48:01.695099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:01.695104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:01.695127Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342076222691910:8681]
2025-10-09T21:48:01.695135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342076222691910:8681]
2025-10-09T21:48:01.695137Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342076222691910:8681]
2025-10-09T21:48:01.695152Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:01.695171Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:01.695172Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:01.695176Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:01.695178Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:01.695189Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:01.695193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:01.695212Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7d71afe-2b6d4a5f-b1bb95eb-83da6dc2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:01.695223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:01.695228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:01.695230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:01.695240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:01.695251Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:01.695296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342076222691910:8681]
2025-10-09T21:48:01.695308Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:01.695316Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:01.695320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:01.695454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:01.695474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:01.695488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:01.695585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1098 partition: 0 MaxSeqNo: 25 sessionId: src_id|7d71afe-2b6d4a5f-b1bb95eb-83da6dc2_0
2025-10-09T21:48:01.695947Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|659a1480-90bc7920-9cc335a9-1d5042db_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7d71afe-2b6d4a5f-b1bb95eb-83da6dc2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:01.698539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7d71afe-2b6d4a5f-b1bb95eb-83da6dc2_0] Write session: try to update token
2025-10-09T21:48:01.698559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7d71afe-2b6d4a5f-b1bb95eb-83da6dc2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:01.711929Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1098 sessionId: src_id|7d71afe-2b6d4a5f-b1bb95eb-83da6dc2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:01.712107Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:01.712123Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:01.712131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342076222691910:8681]
2025-10-09T21:48:01.712184Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:01.712196Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:01.712350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:01.712529Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:01.712658Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:01.712797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:01.712973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:01.713009Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:01.714323Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:01.714977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:01.715004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:01.715041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:01.715122Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:01.719888Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:01.719899Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:01.719901Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:01.719902Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:01.719909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342076222691910:8681]
2025-10-09T21:48:01.719912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342076222691910:8681]
2025-10-09T21:48:01.719915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342076222691910:8681]
2025-10-09T21:48:01.719945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:01.719951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.719953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:01.719956Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:01.719961Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:01.719965Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:01.720081Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:01.770518Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342076584513396:9144]
2025-10-09T21:48:01.774357Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 1097, pipe: [1:7559342076584513396:9144]
2025-10-09T21:48:01.876464Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:01.884130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:01.884158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:01.884333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.884398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.884467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.884544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.884599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.884612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.884685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.884751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.884819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.884897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.884940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.884948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.885010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.885069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.885131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.885192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.885232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.885241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.885308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.885371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.885435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.885498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.885542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.885549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.885621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.885683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.885751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.885831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.885877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.885884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.885951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.886011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.886098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.886166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.886211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.886217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.886281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.886340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.886400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.886467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.886513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.886520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.886584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.886643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.886708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.886769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.886808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.886813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.886871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.886922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.886985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.887052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.887099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.887105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.887170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:01.887229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:01.887287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:01.887350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:01.887395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:01.887401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:01.887465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342076584513396:9144], now have 1 active actors on pipe
2025-10-09T21:48:01.887698Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:01.887712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:01.887717Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342076584513396:9144]
2025-10-09T21:48:01.887737Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:01.887747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1098 reason: error at write: Disconnected sessionId: src_id|7d71afe-2b6d4a5f-b1bb95eb-83da6dc2_0
2025-10-09T21:48:01.887969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1098 sessionId: src_id|7d71afe-2b6d4a5f-b1bb95eb-83da6dc2_0 is DEAD
2025-10-09T21:48:01.888122Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d71afe-2b6d4a5f-b1bb95eb-83da6dc2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:01.888141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d71afe-2b6d4a5f-b1bb95eb-83da6dc2_0] Write session will restart in 0.008544s
2025-10-09T21:48:01.888164Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d71afe-2b6d4a5f-b1bb95eb-83da6dc2_0] Write session: Do CDS request
2025-10-09T21:48:01.888170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d71afe-2b6d4a5f-b1bb95eb-83da6dc2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:01.888479Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:01.897542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7d71afe-2b6d4a5f-b1bb95eb-83da6dc2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:01.897675Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:01.897692Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1099
2025-10-09T21:48:01.897837Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1099 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:01.897873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1099 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:01.897884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1099 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:01.897888Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:01.898406Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1099 sessionId:  describe result for acl check
2025-10-09T21:48:01.898444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:01.898450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:01.898452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:01.898457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342076584513477:9723] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:01.898998Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342076584513477:9723] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:01.901567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342076584513477:9723] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:01.902062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:01.904022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:01.904225Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:01.904234Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:01.904246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:01.904316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:01.904355Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342076222692088:8683]
2025-10-09T21:48:01.904691Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342076584513477:9723] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:01.904705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342076584513477:9723] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:01.904709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342076584513477:9723] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:01.904721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1099 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:01.904914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:01.905266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:01.905322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:01.905645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:01.905694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.905699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.905700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.905701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.905702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.905704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.905705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.905706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.905707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.905713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.905724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.905729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.905733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.905737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.905742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.905746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.905753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.905757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.905764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.905785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:01.905795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.905799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:01.905803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.905808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:01.905813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.905819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:01.905823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.905839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:01.905847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.905852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:01.905861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.905867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.905875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.905880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.905889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.905896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.905928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:01.912441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:01.913794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:01.913809Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:01.913812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:01.913817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:01.913825Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1098 [2:7559342076222692088:8683]
2025-10-09T21:48:01.913841Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:01.913863Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:01.913866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:01.913869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:01.913871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:01.913891Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:01.913906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:01.913913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:01.913976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.913983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.913985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.913988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:01.914122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:01.914135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.914148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:01.914160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:01.914174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:01.914202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342076584513401:2432], now have 1 active actors on pipe
2025-10-09T21:48:01.914210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342076584513572:9723], now have 1 active actors on pipe
2025-10-09T21:48:01.914218Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342076222692083:8683]
2025-10-09T21:48:01.914224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342076222692083:8683]
2025-10-09T21:48:01.914226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342076222692083:8683]
2025-10-09T21:48:01.914240Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:01.914258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:01.914231Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1098
2025-10-09T21:48:01.914267Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1098
2025-10-09T21:48:01.914272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:01.914276Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:01.914280Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:01.914281Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:01.914284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:01.914338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:01.914347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:01.914359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342076222692083:8683]
2025-10-09T21:48:01.914373Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:01.914397Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:01.914413Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|43a86c09-328664a4-630010d1-b6ce4c43_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:01.914420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:01.914425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:01.914428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:01.914439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:01.914449Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:01.914472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:01.914670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:01.914681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:01.914714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:01.914816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1099 partition: 0 MaxSeqNo: 25 sessionId: src_id|43a86c09-328664a4-630010d1-b6ce4c43_0
2025-10-09T21:48:01.915277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d71afe-2b6d4a5f-b1bb95eb-83da6dc2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|43a86c09-328664a4-630010d1-b6ce4c43_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:01.917759Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|43a86c09-328664a4-630010d1-b6ce4c43_0] Write session: try to update token
2025-10-09T21:48:01.917790Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|43a86c09-328664a4-630010d1-b6ce4c43_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:01.928370Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:01.928387Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:01.928390Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:01.928392Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:01.928401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342076222692083:8683]
2025-10-09T21:48:01.928406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342076222692083:8683]
2025-10-09T21:48:01.928410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342076222692083:8683]
2025-10-09T21:48:01.928454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:01.928462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:01.928465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:01.928466Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:01.928472Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:01.928475Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:01.928595Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:01.931681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1099 sessionId: src_id|43a86c09-328664a4-630010d1-b6ce4c43_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:01.934212Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:01.988091Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342076584513584:9144]
2025-10-09T21:48:01.988444Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 1098, pipe: [1:7559342076584513584:9144]
2025-10-09T21:48:02.086985Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:02.087462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.087482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.087531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342076584513584:9144], now have 1 active actors on pipe
2025-10-09T21:48:02.087809Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:02.087864Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:02.087876Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:02.087887Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342076584513584:9144]
2025-10-09T21:48:02.087946Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1099 reason: error at write: Disconnected sessionId: src_id|43a86c09-328664a4-630010d1-b6ce4c43_0
2025-10-09T21:48:02.087975Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:02.088178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1099 sessionId: src_id|43a86c09-328664a4-630010d1-b6ce4c43_0 is DEAD
2025-10-09T21:48:02.088309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43a86c09-328664a4-630010d1-b6ce4c43_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:02.088321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43a86c09-328664a4-630010d1-b6ce4c43_0] Write session will restart in 0.006369s
2025-10-09T21:48:02.088350Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43a86c09-328664a4-630010d1-b6ce4c43_0] Write session: Do CDS request
2025-10-09T21:48:02.088357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43a86c09-328664a4-630010d1-b6ce4c43_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:02.095595Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|43a86c09-328664a4-630010d1-b6ce4c43_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:02.095715Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:02.095733Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1100
2025-10-09T21:48:02.095866Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1100 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:02.095911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1100 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:02.095922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1100 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:02.095926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:02.096522Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1100 sessionId:  describe result for acl check
2025-10-09T21:48:02.096562Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:02.096569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:02.096574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:02.096580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342080879480899:9727] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:02.097256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342080879480899:9727] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:02.102225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342080879480899:9727] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:02.105933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342080879480899:9727] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:02.105952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342080879480899:9727] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:02.105957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342080879480899:9727] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:02.105970Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1100 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:02.106786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:02.108883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:02.109115Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:02.109128Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:02.109140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:02.109201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:02.109245Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342080517659560:8685]
2025-10-09T21:48:02.109880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:02.110269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:02.110330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:02.110710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:02.110792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.110798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.110799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.110801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.110802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.110804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.110805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.110806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.110808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.110815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.110828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.110832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.110836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.110839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.110844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.110848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.110853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.110857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.110865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.110876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.110888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.110894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.110904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.110908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.110912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.110923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.110929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.110941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.110953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.110960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.110976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.110992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.110999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.111006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.111014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.111020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.111055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:02.118719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:02.119763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:02.119779Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:02.119783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:02.119788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:02.119797Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1099 [2:7559342080517659560:8685]
2025-10-09T21:48:02.119815Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:02.119839Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:02.119847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:02.119851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:02.119854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:02.119874Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:02.119885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:02.119898Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:02.119967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.119974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.119977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.119980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:02.120060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:02.120067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.120080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.120097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.120119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:02.120148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342080879480884:2432], now have 1 active actors on pipe
2025-10-09T21:48:02.120156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342080879481055:9727], now have 1 active actors on pipe
2025-10-09T21:48:02.120165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342080517659555:8685]
2025-10-09T21:48:02.120168Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342080517659555:8685]
2025-10-09T21:48:02.120170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342080517659555:8685]
2025-10-09T21:48:02.120185Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:02.120201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:02.120200Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:02.120204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:02.120206Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:02.120221Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:02.120225Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:02.120267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342080517659555:8685]
2025-10-09T21:48:02.120227Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1099
2025-10-09T21:48:02.120284Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:02.120291Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1099
2025-10-09T21:48:02.120304Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:02.120306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.120312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.120361Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a4163c8b-eb3f5b7a-48997f80-2d5a8ad5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:02.120373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:02.120378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:02.120385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:02.120397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:02.120412Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:02.120440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:02.120629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.120637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.120650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:02.120763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1100 partition: 0 MaxSeqNo: 25 sessionId: src_id|a4163c8b-eb3f5b7a-48997f80-2d5a8ad5_0
2025-10-09T21:48:02.121162Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43a86c09-328664a4-630010d1-b6ce4c43_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a4163c8b-eb3f5b7a-48997f80-2d5a8ad5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:02.123854Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a4163c8b-eb3f5b7a-48997f80-2d5a8ad5_0] Write session: try to update token
2025-10-09T21:48:02.123877Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a4163c8b-eb3f5b7a-48997f80-2d5a8ad5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:02.135371Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:02.135384Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:02.135386Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:02.135388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:02.135398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342080517659555:8685]
2025-10-09T21:48:02.135403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342080517659555:8685]
2025-10-09T21:48:02.135408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342080517659555:8685]
2025-10-09T21:48:02.135461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:02.135462Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:02.135468Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:02.135472Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:02.135473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.135476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:02.135621Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:02.140453Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1100 sessionId: src_id|a4163c8b-eb3f5b7a-48997f80-2d5a8ad5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:02.143039Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:02.187733Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342080879481071:9144]
2025-10-09T21:48:02.188101Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 1099, pipe: [1:7559342080879481071:9144]
2025-10-09T21:48:02.300837Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:02.301438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.301456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.301507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342080879481071:9144], now have 1 active actors on pipe
2025-10-09T21:48:02.301750Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:02.301796Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:02.301800Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:02.301822Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342080879481071:9144]
2025-10-09T21:48:02.301859Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:02.301891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1100 reason: error at write: Disconnected sessionId: src_id|a4163c8b-eb3f5b7a-48997f80-2d5a8ad5_0
2025-10-09T21:48:02.302116Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1100 sessionId: src_id|a4163c8b-eb3f5b7a-48997f80-2d5a8ad5_0 is DEAD
2025-10-09T21:48:02.302303Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4163c8b-eb3f5b7a-48997f80-2d5a8ad5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:02.302317Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4163c8b-eb3f5b7a-48997f80-2d5a8ad5_0] Write session will restart in 0.007624s
2025-10-09T21:48:02.302346Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4163c8b-eb3f5b7a-48997f80-2d5a8ad5_0] Write session: Do CDS request
2025-10-09T21:48:02.302353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4163c8b-eb3f5b7a-48997f80-2d5a8ad5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:02.311730Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a4163c8b-eb3f5b7a-48997f80-2d5a8ad5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:02.311796Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:02.311813Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1101
2025-10-09T21:48:02.311973Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1101 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:02.312009Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1101 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:02.312016Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1101 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:02.312020Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:02.312546Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1101 sessionId:  describe result for acl check
2025-10-09T21:48:02.312583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:02.312589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:02.312591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:02.312598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342080879481155:9732] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:02.313157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342080879481155:9732] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:02.315361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342080879481155:9732] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:02.316778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:02.319279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:02.319415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342080879481155:9732] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:02.319433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342080879481155:9732] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:02.319437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342080879481155:9732] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:02.319451Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1101 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:02.319505Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:02.319516Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:02.319528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:02.319591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:02.319629Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342080517659764:8697]
2025-10-09T21:48:02.320249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:02.320637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:02.320701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:02.321015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:02.321086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.321092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.321093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.321094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.321096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.321097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.321098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.321099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.321100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.321106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.321117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.321122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.321125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.321129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.321133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.321137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.321140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.321144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.321152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.321159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.321166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.321171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.321176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.321180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.321184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.321189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.321194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.321199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.321203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.321208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.321214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.321219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.321223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.321227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.321232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.321236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.321264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:02.328007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:02.329464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:02.329481Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:02.329485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:02.329489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:02.329497Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1100 [2:7559342080517659764:8697]
2025-10-09T21:48:02.329511Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:02.329532Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:02.329535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:02.329539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:02.329541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:02.329560Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:02.329566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:02.329574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:02.329641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.329652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.329654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.329657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:02.329735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:02.329744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.329758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.329781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.329798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:02.329822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342080879481081:2432], now have 1 active actors on pipe
2025-10-09T21:48:02.329826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342080879481251:9732], now have 1 active actors on pipe
2025-10-09T21:48:02.329834Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342080517659759:8697]
2025-10-09T21:48:02.329838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342080517659759:8697]
2025-10-09T21:48:02.329840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342080517659759:8697]
2025-10-09T21:48:02.329856Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:02.329876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:02.329877Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:02.329881Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:02.329886Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:02.329891Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:02.329894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:02.329940Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342080517659759:8697]
2025-10-09T21:48:02.329956Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:02.329940Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1100
2025-10-09T21:48:02.329977Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:02.329957Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1100
2025-10-09T21:48:02.330006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.330012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.330091Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5b758d6d-478327ea-9aef3c77-1869e570_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:02.330108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:02.330113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:02.330117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:02.330129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:02.330143Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:02.330164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:02.330391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.330402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.330421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:02.330528Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1101 partition: 0 MaxSeqNo: 25 sessionId: src_id|5b758d6d-478327ea-9aef3c77-1869e570_0
2025-10-09T21:48:02.330928Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4163c8b-eb3f5b7a-48997f80-2d5a8ad5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5b758d6d-478327ea-9aef3c77-1869e570_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:02.333429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b758d6d-478327ea-9aef3c77-1869e570_0] Write session: try to update token
2025-10-09T21:48:02.333450Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b758d6d-478327ea-9aef3c77-1869e570_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:02.345108Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:02.345125Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:02.345128Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:02.345130Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:02.345140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342080517659759:8697]
2025-10-09T21:48:02.345145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342080517659759:8697]
2025-10-09T21:48:02.345150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342080517659759:8697]
2025-10-09T21:48:02.345209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:02.345221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.345221Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:02.345228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:02.345238Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:02.345242Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:02.345373Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:02.348460Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1101 sessionId: src_id|5b758d6d-478327ea-9aef3c77-1869e570_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:02.350927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:02.401756Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342080879481287:9144]
2025-10-09T21:48:02.402109Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 1100, pipe: [1:7559342080879481287:9144]
2025-10-09T21:48:02.503681Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:02.504122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.504140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.504194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342080879481287:9144], now have 1 active actors on pipe
2025-10-09T21:48:02.504448Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:02.504546Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342080879481287:9144]
2025-10-09T21:48:02.504590Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:02.504626Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:02.504713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1101 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5b758d6d-478327ea-9aef3c77-1869e570_0
2025-10-09T21:48:02.504856Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:48:02.504890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1101 sessionId: src_id|5b758d6d-478327ea-9aef3c77-1869e570_0 is DEAD
2025-10-09T21:48:02.505049Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b758d6d-478327ea-9aef3c77-1869e570_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:02.505028Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 11 consumer shared/user session shared/user_1_11_8138003183284567198_v1 is DEAD
2025-10-09T21:48:02.505071Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b758d6d-478327ea-9aef3c77-1869e570_0] Write session will restart in 0.006038s
2025-10-09T21:48:02.505106Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b758d6d-478327ea-9aef3c77-1869e570_0] Write session: Do CDS request
2025-10-09T21:48:02.505112Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b758d6d-478327ea-9aef3c77-1869e570_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:02.505130Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:48:02.505177Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:48:02.505194Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:48:02.505204Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.009339s
2025-10-09T21:48:02.505329Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341994980120875:9141] disconnected.
2025-10-09T21:48:02.505341Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341994980120875:9141] disconnected; active server actors: 1
2025-10-09T21:48:02.505344Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559341994980120875:9141] client user disconnected session shared/user_1_11_8138003183284567198_v1
2025-10-09T21:48:02.512574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b758d6d-478327ea-9aef3c77-1869e570_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:02.512717Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:02.512734Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1102
2025-10-09T21:48:02.512883Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1102 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:02.512918Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1102 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:02.512928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1102 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:02.512932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:02.513477Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1102 sessionId:  describe result for acl check
2025-10-09T21:48:02.513514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:02.513516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:02.513518Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:02.513523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342080879481306:9743] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:02.514143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342080879481306:9743] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:02.515371Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:48:02.515560Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:48:02.515569Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 12
2025-10-09T21:48:02.515743Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 12 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:48:02.515805Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:48:02.515890Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 auth for : user
2025-10-09T21:48:02.516320Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 Handle describe topics response
2025-10-09T21:48:02.516342Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 auth is DEAD
2025-10-09T21:48:02.516355Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:48:02.516616Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:48:02.516720Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_12_13548706377113208994_v1
2025-10-09T21:48:02.516819Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342080879481361:9747] connected; active server actors: 1
2025-10-09T21:48:02.516850Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559342080879481361:9747] session shared/user_1_12_13548706377113208994_v1
2025-10-09T21:48:02.516861Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:48:02.516877Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:48:02.516888Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_12_13548706377113208994_v1" (Sender=[1:7559342080879481338:9747], Pipe=[1:7559342080879481361:9747], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:48:02.516897Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:48:02.516910Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:48:02.516924Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_12_13548706377113208994_v1" (Sender=[1:7559342080879481338:9747], Pipe=[1:7559342080879481361:9747], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:48:02.516938Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_12_13548706377113208994_v1" sender [1:7559342080879481338:9747] lock partition 0 for ReadingSession "shared/user_1_12_13548706377113208994_v1" (Sender=[1:7559342080879481338:9747], Pipe=[1:7559342080879481361:9747], Partitions=[], ActiveFamilyCount=1) generation 1 step 12
2025-10-09T21:48:02.516951Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:48:02.516961Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000042s
2025-10-09T21:48:02.517206Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 12 Session: "shared/user_1_12_13548706377113208994_v1" ClientId: "user" PipeClient { RawX1: 7559342080879481361 RawX2: 4503603922347539 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:48:02.517242Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:48:02.517721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342080879481306:9743] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:02.521262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342080879481306:9743] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:02.521285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342080879481306:9743] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:02.521289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342080879481306:9743] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:02.521297Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1102 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:02.523134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:02.525926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:02.526169Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:02.526178Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:02.526190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:02.526247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:02.526294Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342080517659950:8701]
2025-10-09T21:48:02.526930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:02.527288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:02.527348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:02.527708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:02.527758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.527764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.527765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.527767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.527768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.527769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.527770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.527772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.527773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.527778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.527789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.527794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.527797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.527801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.527805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.527809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.527812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.527815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.527823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.527831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.527837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.527847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.527851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.527856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.527862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.527868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.527872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.527877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.527882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.527887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.527891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.527897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.527901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.527906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.527913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.527918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.527943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:02.534513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:02.535584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:02.535600Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:02.535603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:02.535608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:02.535616Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1101 [2:7559342080517659950:8701]
2025-10-09T21:48:02.535630Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:02.535651Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:02.535654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:02.535657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:02.535660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:02.535679Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:02.535690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:02.535698Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:02.535762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.535773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.535775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.535779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:02.535850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:02.535857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.535869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.535879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.535893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:02.535915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342080879481467:9743], now have 1 active actors on pipe
2025-10-09T21:48:02.535918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342080879481289:2432], now have 1 active actors on pipe
2025-10-09T21:48:02.535926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342080517659945:8701]
2025-10-09T21:48:02.535929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342080517659945:8701]
2025-10-09T21:48:02.535930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342080517659945:8701]
2025-10-09T21:48:02.535944Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:02.535954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342080879481372:9750], now have 1 active actors on pipe
2025-10-09T21:48:02.535963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:02.535976Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:02.535982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:02.536019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342080517659945:8701]
2025-10-09T21:48:02.536004Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1101
2025-10-09T21:48:02.536004Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1101
2025-10-09T21:48:02.536041Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1101, pipe: [1:7559342080879481372:9750]
2025-10-09T21:48:02.536036Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:02.536038Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:02.536044Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:02.536046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:02.536056Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:02.536297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.536308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.536331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.536337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.536345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_12_13548706377113208994_v1 on pipe: [1:7559342080879481372:9750]
2025-10-09T21:48:02.536376Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5497589d-b10f008a-9cfd79d0-1839b1a9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:02.536385Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_12_13548706377113208994_v1:1 with generation 1101
2025-10-09T21:48:02.536390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:02.536394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:02.536396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:02.536409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:02.536421Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:02.536435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:02.536440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:02.536441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:02.536459Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_12_13548706377113208994_v1
2025-10-09T21:48:02.536463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:02.536490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:02.536516Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:48:02.536698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.536708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.536726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:02.536842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1102 partition: 0 MaxSeqNo: 25 sessionId: src_id|5497589d-b10f008a-9cfd79d0-1839b1a9_0
2025-10-09T21:48:02.537228Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b758d6d-478327ea-9aef3c77-1869e570_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5497589d-b10f008a-9cfd79d0-1839b1a9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:02.539768Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5497589d-b10f008a-9cfd79d0-1839b1a9_0] Write session: try to update token
2025-10-09T21:48:02.539787Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5497589d-b10f008a-9cfd79d0-1839b1a9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:02.541250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:48:02.541295Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:48:02.541311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:02.541319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:02.541322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:02.541331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:48:02.541352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:48:02.541498Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046482536 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:48:02.541520Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:48:02.541559Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 sending to client partition status
2025-10-09T21:48:02.547968Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:02.547984Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:02.547995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342080517659945:8701]
2025-10-09T21:48:02.548036Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:02.548039Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:02.548223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:02.548363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:02.548498Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:02.548620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:02.548818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:02.548867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:02.555027Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1102 sessionId: src_id|5497589d-b10f008a-9cfd79d0-1839b1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:02.557454Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:02.557847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.557868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.557922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:02.558048Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:02.561044Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:02.561057Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:02.561059Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:02.561061Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:02.561072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342080517659945:8701]
2025-10-09T21:48:02.561076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342080517659945:8701]
2025-10-09T21:48:02.561079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342080517659945:8701]
2025-10-09T21:48:02.561126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:02.561136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.561144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:02.561155Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:02.561161Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:02.561165Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:02.561276Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:02.720108Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:02.728624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.728654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.728827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:02.728892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:02.728963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:02.729034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:02.729084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:02.729100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:02.729172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:02.729232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:02.729303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:02.729371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:02.729414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:02.729424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:02.729481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:02.729540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:02.729604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:02.729662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:02.729704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:02.729710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:02.729785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:02.729854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:02.729915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:02.730045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:02.730096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:02.730107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:02.730187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:02.730284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:02.730349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:02.730425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:02.730472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:02.730480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:02.730550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:02.730619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:02.730681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:02.730747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:02.730794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:02.730803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:02.730867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:02.730927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:02.731013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:02.731090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:02.731149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:02.731158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:02.731229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:02.731297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:02.731364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:02.731440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:02.731488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:02.731496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:02.731566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:02.731635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:02.731705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:02.731771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:02.731820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:02.731827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:02.731897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:02.731968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:02.732038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:02.732108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:02.732160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:02.732170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:02.732486Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:02.732499Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:02.732504Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:02.732538Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1102 reason: error at write: Disconnected sessionId: src_id|5497589d-b10f008a-9cfd79d0-1839b1a9_0
2025-10-09T21:48:02.732538Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342080879481372:9750]
2025-10-09T21:48:02.732732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1102 sessionId: src_id|5497589d-b10f008a-9cfd79d0-1839b1a9_0 is DEAD
2025-10-09T21:48:02.732876Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5497589d-b10f008a-9cfd79d0-1839b1a9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:02.732887Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5497589d-b10f008a-9cfd79d0-1839b1a9_0] Write session will restart in 0.006229s
2025-10-09T21:48:02.732914Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5497589d-b10f008a-9cfd79d0-1839b1a9_0] Write session: Do CDS request
2025-10-09T21:48:02.732920Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5497589d-b10f008a-9cfd79d0-1839b1a9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:02.733442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_12_13548706377113208994_v1
2025-10-09T21:48:02.733457Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:02.733486Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_12_13548706377113208994_v1
2025-10-09T21:48:02.740521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5497589d-b10f008a-9cfd79d0-1839b1a9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:02.740586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:02.740601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1103
2025-10-09T21:48:02.740787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1103 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:02.740823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1103 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:02.740834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1103 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:02.740838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:02.741307Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1103 sessionId:  describe result for acl check
2025-10-09T21:48:02.741349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:02.741355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:02.741357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:02.741364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342080879481499:9751] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:02.741961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342080879481499:9751] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:02.744523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342080879481499:9751] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:02.748580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342080879481499:9751] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:02.748597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342080879481499:9751] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:02.748601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342080879481499:9751] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:02.748610Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1103 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:02.749222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:02.751226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:02.751450Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:02.751459Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:02.751471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:02.751527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:02.751570Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342080517660133:8705]
2025-10-09T21:48:02.752173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:02.752416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:02.752456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:02.752790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:02.752837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.752843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.752844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.752845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.752846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.752848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.752849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.752850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.752851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.752856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.752868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.752872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.752876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.752880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.752884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.752888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.752892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.752896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.752904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.752912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.752919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.752925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.752930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.752936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.752940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.752946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.752951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.752956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.752960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.752965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.752971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.752975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.752979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.752984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.752987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.752991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.753019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:02.759604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:02.760876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:02.760890Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:02.760892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:02.760897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:02.760904Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1102 [2:7559342080517660133:8705]
2025-10-09T21:48:02.760918Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:02.760941Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:02.760949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:02.760953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:02.760956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:02.760975Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:02.760988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:02.761000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:02.761068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.761075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.761077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.761081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:02.761149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:02.761156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.761169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.761184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.761203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:02.761229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342080879481484:2432], now have 1 active actors on pipe
2025-10-09T21:48:02.761237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342080879481658:9751], now have 1 active actors on pipe
2025-10-09T21:48:02.761245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342080517660128:8705]
2025-10-09T21:48:02.761252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342080517660128:8705]
2025-10-09T21:48:02.761254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342080517660128:8705]
2025-10-09T21:48:02.761268Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:02.761286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:02.761304Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:02.761302Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:02.761307Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:02.761309Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:02.761312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:02.761298Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1102
2025-10-09T21:48:02.761346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342080517660128:8705]
2025-10-09T21:48:02.761319Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1102
2025-10-09T21:48:02.761357Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:02.761368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:02.761382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.761394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.761444Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a595c97f-fb57a5b-aa2b4283-87f25643_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:02.761456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:02.761466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:02.761469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:02.761480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:02.761490Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:02.761512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:02.761820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.761832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.761854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:02.761994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1103 partition: 0 MaxSeqNo: 25 sessionId: src_id|a595c97f-fb57a5b-aa2b4283-87f25643_0
2025-10-09T21:48:02.762381Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5497589d-b10f008a-9cfd79d0-1839b1a9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a595c97f-fb57a5b-aa2b4283-87f25643_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:02.764885Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a595c97f-fb57a5b-aa2b4283-87f25643_0] Write session: try to update token
2025-10-09T21:48:02.764903Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a595c97f-fb57a5b-aa2b4283-87f25643_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:02.775744Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:02.775760Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:02.775763Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:02.775765Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:02.775775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342080517660128:8705]
2025-10-09T21:48:02.775780Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342080517660128:8705]
2025-10-09T21:48:02.775784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342080517660128:8705]
2025-10-09T21:48:02.775829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:02.775842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.775841Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:02.775845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:02.775848Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:02.775852Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:02.775982Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:02.778303Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1103 sessionId: src_id|a595c97f-fb57a5b-aa2b4283-87f25643_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:02.780624Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:02.832903Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342080879481673:9750]
2025-10-09T21:48:02.833225Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 1102, pipe: [1:7559342080879481673:9750]
2025-10-09T21:48:02.936130Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:02.936551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.936566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.936607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342080879481673:9750], now have 1 active actors on pipe
2025-10-09T21:48:02.936765Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:02.936819Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:02.936868Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:02.936874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:02.936877Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342080879481673:9750]
2025-10-09T21:48:02.936936Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1103 reason: error at write: Disconnected sessionId: src_id|a595c97f-fb57a5b-aa2b4283-87f25643_0
2025-10-09T21:48:02.937140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1103 sessionId: src_id|a595c97f-fb57a5b-aa2b4283-87f25643_0 is DEAD
2025-10-09T21:48:02.937278Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a595c97f-fb57a5b-aa2b4283-87f25643_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:02.937290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a595c97f-fb57a5b-aa2b4283-87f25643_0] Write session will restart in 0.009433s
2025-10-09T21:48:02.937317Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a595c97f-fb57a5b-aa2b4283-87f25643_0] Write session: Do CDS request
2025-10-09T21:48:02.937324Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a595c97f-fb57a5b-aa2b4283-87f25643_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:02.947626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a595c97f-fb57a5b-aa2b4283-87f25643_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:02.947751Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:02.947771Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1104
2025-10-09T21:48:02.947888Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1104 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:02.947915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1104 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:02.947919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1104 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:02.947922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:02.948504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1104 sessionId:  describe result for acl check
2025-10-09T21:48:02.948539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:02.948545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:02.948547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:02.948553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342080879481784:9756] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:02.949103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342080879481784:9756] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:02.951719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:02.951186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342080879481784:9756] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:02.953739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:02.953955Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:02.953965Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:02.953977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:02.954042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:02.954089Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342080517660311:8707]
2025-10-09T21:48:02.954490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342080879481784:9756] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:02.954503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342080879481784:9756] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:02.954507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342080879481784:9756] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:02.954517Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1104 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:02.954680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:02.954973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:02.955010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:02.955309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:02.955358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.955364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.955366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.955367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.955368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.955370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.955371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.955372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.955373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.955378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.955388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.955393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.955397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.955401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.955405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.955410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.955414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.955417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.955425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.955433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:02.955438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.955450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:02.955455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.955465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:02.955472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.955481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:02.955486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.955491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:02.955499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.955504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:02.955510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.955515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.955525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.955530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.955539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.955545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.955572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:02.962127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:02.963173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:02.963186Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:02.963189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:02.963194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:02.963201Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1103 [2:7559342080517660311:8707]
2025-10-09T21:48:02.963214Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:02.963232Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:02.963240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:02.963243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:02.963245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:02.963263Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:02.963269Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:02.963276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:02.963340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.963348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.963350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.963353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:02.963420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:02.963428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.963440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:02.963452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:02.963471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:02.963491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342080879481851:9756], now have 1 active actors on pipe
2025-10-09T21:48:02.963500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342080879481675:2432], now have 1 active actors on pipe
2025-10-09T21:48:02.963509Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342080517660306:8707]
2025-10-09T21:48:02.963511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342080517660306:8707]
2025-10-09T21:48:02.963514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342080517660306:8707]
2025-10-09T21:48:02.963528Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:02.963548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:02.963551Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:02.963555Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:02.963557Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:02.963567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:02.963571Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:02.963602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342080517660306:8707]
2025-10-09T21:48:02.963569Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1103
2025-10-09T21:48:02.963611Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:02.963615Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:02.963627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.963632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.963574Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1103
2025-10-09T21:48:02.963684Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e69231bd-1f3dd31f-84fe4889-18a77ea8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:02.963699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:02.963704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:02.963706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:02.963717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:02.963727Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:02.963746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:02.963901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:02.963911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:02.963924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:02.964027Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1104 partition: 0 MaxSeqNo: 25 sessionId: src_id|e69231bd-1f3dd31f-84fe4889-18a77ea8_0
2025-10-09T21:48:02.964365Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a595c97f-fb57a5b-aa2b4283-87f25643_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e69231bd-1f3dd31f-84fe4889-18a77ea8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:02.966898Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e69231bd-1f3dd31f-84fe4889-18a77ea8_0] Write session: try to update token
2025-10-09T21:48:02.966915Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e69231bd-1f3dd31f-84fe4889-18a77ea8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:02.978141Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:02.978158Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:02.978160Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:02.978162Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:02.978173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342080517660306:8707]
2025-10-09T21:48:02.978179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342080517660306:8707]
2025-10-09T21:48:02.978183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342080517660306:8707]
2025-10-09T21:48:02.978229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:02.978231Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:02.978238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:02.978238Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:02.978240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:02.978257Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:02.978373Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:02.980250Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1104 sessionId: src_id|e69231bd-1f3dd31f-84fe4889-18a77ea8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:02.982677Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:03.037157Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342085174449159:9750]
2025-10-09T21:48:03.037415Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 1103, pipe: [1:7559342085174449159:9750]
2025-10-09T21:48:03.135726Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:03.136238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:03.136250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:03.136289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342085174449159:9750], now have 1 active actors on pipe
2025-10-09T21:48:03.136432Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:03.136477Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342085174449159:9750]
2025-10-09T21:48:03.136509Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:03.136524Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:03.136583Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1104 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e69231bd-1f3dd31f-84fe4889-18a77ea8_0
2025-10-09T21:48:03.136769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1104 sessionId: src_id|e69231bd-1f3dd31f-84fe4889-18a77ea8_0 is DEAD
2025-10-09T21:48:03.136968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e69231bd-1f3dd31f-84fe4889-18a77ea8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:03.136978Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e69231bd-1f3dd31f-84fe4889-18a77ea8_0] Write session will restart in 0.009913s
2025-10-09T21:48:03.137008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e69231bd-1f3dd31f-84fe4889-18a77ea8_0] Write session: Do CDS request
2025-10-09T21:48:03.137016Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e69231bd-1f3dd31f-84fe4889-18a77ea8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:03.148609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e69231bd-1f3dd31f-84fe4889-18a77ea8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:03.148677Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:03.148691Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1105
2025-10-09T21:48:03.148843Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1105 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:03.148869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1105 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:03.148877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1105 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:03.148880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:03.149352Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1105 sessionId:  describe result for acl check
2025-10-09T21:48:03.149401Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:03.149404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:03.149405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:03.149411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342085174449307:9760] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:03.149570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:03.149975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342085174449307:9760] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:03.151780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:03.151991Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:03.152000Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:03.152012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:03.152074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:03.152110Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342084812627786:8709]
2025-10-09T21:48:03.152180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342085174449307:9760] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:03.152675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:03.152962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:03.153008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:03.153327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:03.153382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.153387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.153388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.153389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.153391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.153392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.153393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.153394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.153396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.153401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.153411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.153413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.153415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.153417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.153419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.153421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.153423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.153425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.153430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.153435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.153439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.153445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.153448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.153451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.153454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.153460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.153463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.153466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.153468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.153472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.153476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.153479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.153481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.153484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.153487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.153490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.153508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:03.158616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342085174449307:9760] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:03.158634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342085174449307:9760] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:03.158638Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342085174449307:9760] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:03.158647Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1105 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:03.159940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:03.161187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:03.161201Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:03.161204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:03.161209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:03.161216Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1104 [2:7559342084812627786:8709]
2025-10-09T21:48:03.161230Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:03.161248Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:03.161251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:03.161254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:03.161257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:03.161274Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:03.161284Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:03.161292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:03.161351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.161359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.161361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.161364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:03.161433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:03.161441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.161454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.161471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.161493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:03.161521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342085174449165:2432], now have 1 active actors on pipe
2025-10-09T21:48:03.161524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342085174449346:9760], now have 1 active actors on pipe
2025-10-09T21:48:03.161532Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342084812627781:8709]
2025-10-09T21:48:03.161539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342084812627781:8709]
2025-10-09T21:48:03.161542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342084812627781:8709]
2025-10-09T21:48:03.161557Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:03.161573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:03.161552Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1104
2025-10-09T21:48:03.161592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:03.161600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:03.161576Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1104
2025-10-09T21:48:03.161605Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:03.161609Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:03.161611Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:03.161636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342084812627781:8709]
2025-10-09T21:48:03.161647Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:03.161650Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:03.161667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:03.161672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:03.161891Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8fa713fe-eb68bbde-e212304e-538a8a6a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:03.161905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:03.161909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:03.161912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:03.161922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:03.161937Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:03.161959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:03.164862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:03.164878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:03.164909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:03.165020Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1105 partition: 0 MaxSeqNo: 25 sessionId: src_id|8fa713fe-eb68bbde-e212304e-538a8a6a_0
2025-10-09T21:48:03.165422Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e69231bd-1f3dd31f-84fe4889-18a77ea8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8fa713fe-eb68bbde-e212304e-538a8a6a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:03.167965Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8fa713fe-eb68bbde-e212304e-538a8a6a_0] Write session: try to update token
2025-10-09T21:48:03.167984Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8fa713fe-eb68bbde-e212304e-538a8a6a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:03.176468Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:03.176487Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:03.176489Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:03.176492Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:03.176502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342084812627781:8709]
2025-10-09T21:48:03.176510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342084812627781:8709]
2025-10-09T21:48:03.176516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342084812627781:8709]
2025-10-09T21:48:03.176565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:03.176573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.176576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:03.176591Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:03.176598Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:03.176602Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:03.176725Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:03.181461Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1105 sessionId: src_id|8fa713fe-eb68bbde-e212304e-538a8a6a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:03.183947Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:03.236981Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342085174449358:9750]
2025-10-09T21:48:03.237328Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 1104, pipe: [1:7559342085174449358:9750]
2025-10-09T21:48:03.330283Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:03.330587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:03.330604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:03.330652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342085174449358:9750], now have 1 active actors on pipe
2025-10-09T21:48:03.330870Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:03.330933Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:03.330970Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:03.330989Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342085174449358:9750]
2025-10-09T21:48:03.331108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1105 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8fa713fe-eb68bbde-e212304e-538a8a6a_0
2025-10-09T21:48:03.331352Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1105 sessionId: src_id|8fa713fe-eb68bbde-e212304e-538a8a6a_0 is DEAD
2025-10-09T21:48:03.331485Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fa713fe-eb68bbde-e212304e-538a8a6a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:03.331495Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fa713fe-eb68bbde-e212304e-538a8a6a_0] Write session will restart in 0.008947s
2025-10-09T21:48:03.331515Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fa713fe-eb68bbde-e212304e-538a8a6a_0] Write session: Do CDS request
2025-10-09T21:48:03.331519Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fa713fe-eb68bbde-e212304e-538a8a6a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:03.341633Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8fa713fe-eb68bbde-e212304e-538a8a6a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:03.341700Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:03.341718Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1106
2025-10-09T21:48:03.341923Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1106 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:03.341964Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1106 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:03.341970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1106 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:03.341974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:03.342511Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1106 sessionId:  describe result for acl check
2025-10-09T21:48:03.342556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:03.342558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:03.342560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:03.342566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342085174449456:9765] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:03.343160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342085174449456:9765] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:03.345252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342085174449456:9765] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:03.345369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:03.347812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:03.348001Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:03.348012Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:03.348025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:03.348077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:03.348097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342085174449456:9765] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:03.348111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342085174449456:9765] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:03.348135Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342084812627996:8721]
2025-10-09T21:48:03.348115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342085174449456:9765] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:03.348169Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1106 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:03.348702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:03.349010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:03.349061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:03.349369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:03.349412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.349417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.349418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.349418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.349419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.349420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.349420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.349421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.349421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.349424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.349434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.349437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.349441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.349444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.349448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.349452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.349455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.349459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.349464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.349469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.349474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.349477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.349479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.349484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.349486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.349489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.349492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.349495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.349497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.349500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.349503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.349506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.349509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.349512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.349514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.349517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.349532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:03.355709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:03.356783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:03.356798Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:03.356800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:03.356804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:03.356810Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1105 [2:7559342084812627996:8721]
2025-10-09T21:48:03.356824Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:03.356843Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:03.356851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:03.356855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:03.356857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:03.356874Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:03.356878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:03.356884Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:03.356947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.356957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.356959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.356961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:03.357013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:03.357020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.357028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.357035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.357044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:03.357064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342084812627991:8721]
2025-10-09T21:48:03.357070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342084812627991:8721]
2025-10-09T21:48:03.357072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342084812627991:8721]
2025-10-09T21:48:03.357078Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:03.357095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:03.357101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:03.357107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:03.357110Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:03.357110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:03.357113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:03.357123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342085174449363:2432], now have 1 active actors on pipe
2025-10-09T21:48:03.357128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342085174449538:9765], now have 1 active actors on pipe
2025-10-09T21:48:03.357156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342084812627991:8721]
2025-10-09T21:48:03.357168Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:03.357174Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:03.357203Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1105
2025-10-09T21:48:03.357207Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1105
2025-10-09T21:48:03.357288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:03.357298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:03.357327Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5a5cf516-9e1a4634-b8d65d7e-40492d74_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:03.357340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:03.357344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:03.357347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:03.357357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:03.357367Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:03.357378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:03.357568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:03.357584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:03.357609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:03.357828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1106 partition: 0 MaxSeqNo: 25 sessionId: src_id|5a5cf516-9e1a4634-b8d65d7e-40492d74_0
2025-10-09T21:48:03.358248Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fa713fe-eb68bbde-e212304e-538a8a6a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5a5cf516-9e1a4634-b8d65d7e-40492d74_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:03.360788Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a5cf516-9e1a4634-b8d65d7e-40492d74_0] Write session: try to update token
2025-10-09T21:48:03.360806Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a5cf516-9e1a4634-b8d65d7e-40492d74_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:03.371136Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:03.371152Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:03.371154Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:03.371157Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:03.371166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342084812627991:8721]
2025-10-09T21:48:03.371171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342084812627991:8721]
2025-10-09T21:48:03.371174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342084812627991:8721]
2025-10-09T21:48:03.371217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:03.371227Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:03.371230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.371233Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:03.371233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:03.371236Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:03.371366Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:03.374084Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1106 sessionId: src_id|5a5cf516-9e1a4634-b8d65d7e-40492d74_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:03.376511Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:03.430895Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342085174449574:9750]
2025-10-09T21:48:03.431064Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 1105, pipe: [1:7559342085174449574:9750]
2025-10-09T21:48:03.523228Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:03.523496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:03.523510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:03.523559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342085174449574:9750], now have 1 active actors on pipe
2025-10-09T21:48:03.523812Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:03.523824Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:03.523870Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:03.523873Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:03.523877Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342085174449574:9750]
2025-10-09T21:48:03.523944Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1106 reason: error at write: Disconnected sessionId: src_id|5a5cf516-9e1a4634-b8d65d7e-40492d74_0
2025-10-09T21:48:03.524142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1106 sessionId: src_id|5a5cf516-9e1a4634-b8d65d7e-40492d74_0 is DEAD
2025-10-09T21:48:03.524316Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a5cf516-9e1a4634-b8d65d7e-40492d74_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:03.524327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a5cf516-9e1a4634-b8d65d7e-40492d74_0] Write session will restart in 0.008712s
2025-10-09T21:48:03.524357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a5cf516-9e1a4634-b8d65d7e-40492d74_0] Write session: Do CDS request
2025-10-09T21:48:03.524364Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a5cf516-9e1a4634-b8d65d7e-40492d74_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:03.534570Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a5cf516-9e1a4634-b8d65d7e-40492d74_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:03.534797Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:03.534816Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1107
2025-10-09T21:48:03.534947Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1107 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:03.534982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1107 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:03.534989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1107 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:03.534994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:03.535615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1107 sessionId:  describe result for acl check
2025-10-09T21:48:03.535654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:03.535663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:03.535665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:03.535671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342085174449667:9776] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:03.536275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342085174449667:9776] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:03.538630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342085174449667:9776] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:03.539683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:03.541900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342085174449667:9776] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:03.541915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342085174449667:9776] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:03.541919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342085174449667:9776] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:03.541928Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1107 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:03.541707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:03.541930Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:03.541935Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:03.541949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:03.542011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:03.542065Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342084812628185:8725]
2025-10-09T21:48:03.542635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:03.542985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:03.543037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:03.543354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:03.543423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.543428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.543429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.543430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.543432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.543433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.543434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.543435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.543436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.543442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.543454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.543458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.543462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.543466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.543470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.543474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.543478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.543482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.543489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.543497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.543504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.543510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.543514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.543519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.543523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.543528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.543533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.543538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.543543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.543548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.543553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.543558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.543563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.543568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.543572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.543585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.543616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:03.550288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:03.551445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:03.551462Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:03.551465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:03.551470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:03.551477Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1106 [2:7559342084812628185:8725]
2025-10-09T21:48:03.551493Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:03.551513Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:03.551521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:03.551525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:03.551528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:03.551546Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:03.551561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:03.551569Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:03.551837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.551850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.551853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.551855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:03.551932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:03.551944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.551957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.551969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.551986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:03.552010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342085174449752:9776], now have 1 active actors on pipe
2025-10-09T21:48:03.552021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342085174449576:2432], now have 1 active actors on pipe
2025-10-09T21:48:03.552029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342084812628180:8725]
2025-10-09T21:48:03.552032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342084812628180:8725]
2025-10-09T21:48:03.552034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342084812628180:8725]
2025-10-09T21:48:03.552053Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:03.552071Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:03.552074Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:03.552075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:03.552085Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:03.552091Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:03.552094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:03.552205Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1106
2025-10-09T21:48:03.552240Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1106
2025-10-09T21:48:03.552129Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342084812628180:8725]
2025-10-09T21:48:03.552141Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:03.552149Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:03.552374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:03.552381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:03.552415Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bb486b12-73670137-a547612c-9a9a52d2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:03.552431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:03.552436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:03.552438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:03.552450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:03.552463Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:03.552486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:03.552722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:03.552733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:03.552749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:03.552863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1107 partition: 0 MaxSeqNo: 25 sessionId: src_id|bb486b12-73670137-a547612c-9a9a52d2_0
2025-10-09T21:48:03.553227Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a5cf516-9e1a4634-b8d65d7e-40492d74_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bb486b12-73670137-a547612c-9a9a52d2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:03.555744Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb486b12-73670137-a547612c-9a9a52d2_0] Write session: try to update token
2025-10-09T21:48:03.555765Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb486b12-73670137-a547612c-9a9a52d2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:03.566936Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:03.566956Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:03.566959Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:03.566961Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:03.566972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342084812628180:8725]
2025-10-09T21:48:03.566976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342084812628180:8725]
2025-10-09T21:48:03.566981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342084812628180:8725]
2025-10-09T21:48:03.567028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:03.567030Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:03.567034Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:03.567038Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:03.567039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.567042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:03.567181Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:03.569853Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1107 sessionId: src_id|bb486b12-73670137-a547612c-9a9a52d2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:03.572396Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:03.623899Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342085174449764:9750]
2025-10-09T21:48:03.624231Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 1106, pipe: [1:7559342085174449764:9750]
2025-10-09T21:48:03.724823Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:03.725215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:03.725231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:03.725279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342085174449764:9750], now have 1 active actors on pipe
2025-10-09T21:48:03.725514Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:03.725601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:03.725679Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:03.725698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1107 reason: status is not ok: tablet will be restarted right now sessionId: src_id|bb486b12-73670137-a547612c-9a9a52d2_0
2025-10-09T21:48:03.725735Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342085174449764:9750]
2025-10-09T21:48:03.725910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1107 sessionId: src_id|bb486b12-73670137-a547612c-9a9a52d2_0 is DEAD
2025-10-09T21:48:03.726075Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb486b12-73670137-a547612c-9a9a52d2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:03.726087Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb486b12-73670137-a547612c-9a9a52d2_0] Write session will restart in 0.009957s
2025-10-09T21:48:03.726116Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb486b12-73670137-a547612c-9a9a52d2_0] Write session: Do CDS request
2025-10-09T21:48:03.726122Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb486b12-73670137-a547612c-9a9a52d2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:03.737553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb486b12-73670137-a547612c-9a9a52d2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:03.737653Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:03.737668Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1108
2025-10-09T21:48:03.737884Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1108 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:03.737918Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1108 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:03.737928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1108 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:03.737932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:03.738375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1108 sessionId:  describe result for acl check
2025-10-09T21:48:03.738414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:03.738416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:03.738418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:03.738425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342085174449911:9780] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:03.738558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:03.738973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342085174449911:9780] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:03.740680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:03.740871Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:03.740883Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:03.740894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:03.740947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:03.740976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342085174449911:9780] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:03.740989Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342084812628371:8729]
2025-10-09T21:48:03.741597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:03.741916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:03.741959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:03.742289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:03.742340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.742348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.742350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.742351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.742352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.742353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.742354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.742355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.742356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.742361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.742371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.742375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.742380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.742384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.742388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.742392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.742396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.742400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.742407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.742416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.742422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.742427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.742431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.742436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.742440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.742445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.742449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.742455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.742459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.742464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.742469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.742476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.742480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.742485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.742491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.742496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.742526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:03.743391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342085174449911:9780] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:03.743405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342085174449911:9780] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:03.743409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342085174449911:9780] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:03.743418Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1108 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:03.749106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:03.750324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:03.750340Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:03.750343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:03.750347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:03.750355Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1107 [2:7559342084812628371:8729]
2025-10-09T21:48:03.750370Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:03.750389Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:03.750397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:03.750401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:03.750403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:03.750421Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:03.750432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:03.750439Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:03.750503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.750510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.750512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.750515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:03.750587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:03.750595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.750608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.750624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.750644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:03.750671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342085174449766:2432], now have 1 active actors on pipe
2025-10-09T21:48:03.750679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342085174449950:9780], now have 1 active actors on pipe
2025-10-09T21:48:03.750688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342084812628366:8729]
2025-10-09T21:48:03.750694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342084812628366:8729]
2025-10-09T21:48:03.750697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342084812628366:8729]
2025-10-09T21:48:03.750719Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:03.750738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:03.750758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:03.750766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:03.750771Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:03.750775Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:03.750777Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:03.750769Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1107
2025-10-09T21:48:03.750797Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1107
2025-10-09T21:48:03.750805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342084812628366:8729]
2025-10-09T21:48:03.750816Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:03.750819Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:03.750848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:03.750854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:03.750890Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6697a4f5-7841ca2-5fb410a3-db2b0877_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:03.750902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:03.750906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:03.750909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:03.750919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:03.750930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:03.750952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:03.751088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:03.751098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:03.751111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:03.751188Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1108 partition: 0 MaxSeqNo: 25 sessionId: src_id|6697a4f5-7841ca2-5fb410a3-db2b0877_0
2025-10-09T21:48:03.751592Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb486b12-73670137-a547612c-9a9a52d2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6697a4f5-7841ca2-5fb410a3-db2b0877_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:03.754126Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6697a4f5-7841ca2-5fb410a3-db2b0877_0] Write session: try to update token
2025-10-09T21:48:03.754141Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6697a4f5-7841ca2-5fb410a3-db2b0877_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:03.766887Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1108 sessionId: src_id|6697a4f5-7841ca2-5fb410a3-db2b0877_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:03.767103Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:03.767122Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:03.767124Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:03.767127Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:03.767136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342084812628366:8729]
2025-10-09T21:48:03.767141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342084812628366:8729]
2025-10-09T21:48:03.767146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342084812628366:8729]
2025-10-09T21:48:03.767190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:03.767197Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:03.767202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.767202Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:03.767205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:03.767205Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:03.767344Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:03.769248Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:03.825683Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342085174449960:9750]
2025-10-09T21:48:03.826005Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 1107, pipe: [1:7559342085174449960:9750]
2025-10-09T21:48:03.924997Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:03.925455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:03.925471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:03.925517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342085174449960:9750], now have 1 active actors on pipe
2025-10-09T21:48:03.925728Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:03.925804Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342085174449960:9750]
2025-10-09T21:48:03.925816Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:03.925843Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:03.925929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1108 reason: status is not ok: tablet will be restarted right now sessionId: src_id|6697a4f5-7841ca2-5fb410a3-db2b0877_0
2025-10-09T21:48:03.926147Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1108 sessionId: src_id|6697a4f5-7841ca2-5fb410a3-db2b0877_0 is DEAD
2025-10-09T21:48:03.926272Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6697a4f5-7841ca2-5fb410a3-db2b0877_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:03.926283Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6697a4f5-7841ca2-5fb410a3-db2b0877_0] Write session will restart in 0.008028s
2025-10-09T21:48:03.926308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6697a4f5-7841ca2-5fb410a3-db2b0877_0] Write session: Do CDS request
2025-10-09T21:48:03.926314Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6697a4f5-7841ca2-5fb410a3-db2b0877_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:03.935749Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6697a4f5-7841ca2-5fb410a3-db2b0877_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:03.935776Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:03.935798Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1109
2025-10-09T21:48:03.935962Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1109 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:03.935986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1109 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:03.935995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1109 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:03.935999Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:03.936470Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1109 sessionId:  describe result for acl check
2025-10-09T21:48:03.936511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:03.936517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:03.936519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:03.936525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342085174449996:9785] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:03.937072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342085174449996:9785] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:03.939521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342085174449996:9785] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:03.941485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:03.943539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:03.943684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342085174449996:9785] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:03.943696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342085174449996:9785] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:03.943700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342085174449996:9785] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:03.943709Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1109 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:03.943730Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:03.943739Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:03.943751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:03.943807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:03.943845Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342084812628555:8731]
2025-10-09T21:48:03.944425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:03.944690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:03.944727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:03.945021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:03.945072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.945077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.945078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.945079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.945080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.945081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.945082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.945084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.945085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.945089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.945100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.945104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.945107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.945111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.945115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.945118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.945122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.945125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.945132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.945140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:03.945150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.945160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:03.945164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.945169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:03.945177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.945182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:03.945191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.945200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:03.945208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.945217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:03.945225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.945234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.945242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.945252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.945256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.945265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.945289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:03.951640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:03.952938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:03.952955Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:03.952957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:03.952961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:03.952968Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1108 [2:7559342084812628555:8731]
2025-10-09T21:48:03.952981Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:03.952999Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:03.953007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:03.953010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:03.953013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:03.953031Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:03.953041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:03.953048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:03.953109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.953116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.953118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.953121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:03.953192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:03.953203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.953214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:03.953225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:03.953240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:03.953265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342085174450142:9785], now have 1 active actors on pipe
2025-10-09T21:48:03.953275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342085174449962:2432], now have 1 active actors on pipe
2025-10-09T21:48:03.953283Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342084812628550:8731]
2025-10-09T21:48:03.953285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342084812628550:8731]
2025-10-09T21:48:03.953287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342084812628550:8731]
2025-10-09T21:48:03.953299Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:03.953318Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:03.953321Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:03.953324Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:03.953326Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:03.953335Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:03.953338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:03.953342Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1108
2025-10-09T21:48:03.953343Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1108
2025-10-09T21:48:03.953371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342084812628550:8731]
2025-10-09T21:48:03.953378Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:03.953380Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:03.953398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:03.953402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:03.953457Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d501f790-931f101b-31f30425-d22338d4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:03.953471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:03.953475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:03.953478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:03.953489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:03.953499Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:03.953528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:03.953707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:03.953715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:03.953729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:03.953890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1109 partition: 0 MaxSeqNo: 25 sessionId: src_id|d501f790-931f101b-31f30425-d22338d4_0
2025-10-09T21:48:03.954278Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6697a4f5-7841ca2-5fb410a3-db2b0877_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d501f790-931f101b-31f30425-d22338d4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:03.956688Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d501f790-931f101b-31f30425-d22338d4_0] Write session: try to update token
2025-10-09T21:48:03.956704Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d501f790-931f101b-31f30425-d22338d4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:03.968286Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:03.968304Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:03.968307Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:03.968309Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:03.968318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342084812628550:8731]
2025-10-09T21:48:03.968323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342084812628550:8731]
2025-10-09T21:48:03.968328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342084812628550:8731]
2025-10-09T21:48:03.968376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:03.968379Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:03.968390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:03.968393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:03.968397Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:03.968402Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:03.968523Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:03.969737Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1109 sessionId: src_id|d501f790-931f101b-31f30425-d22338d4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:03.972342Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:04.025877Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342089469417451:9750]
2025-10-09T21:48:04.026219Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 1108, pipe: [1:7559342089469417451:9750]
2025-10-09T21:48:04.131365Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:04.131377Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:04.131385Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342089469417451:9750]
2025-10-09T21:48:04.131417Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:04.131449Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1109 reason: error at write: Disconnected sessionId: src_id|d501f790-931f101b-31f30425-d22338d4_0
2025-10-09T21:48:04.131626Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1109 sessionId: src_id|d501f790-931f101b-31f30425-d22338d4_0 is DEAD
2025-10-09T21:48:04.130778Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:04.131061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:04.131072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:04.131117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342089469417451:9750], now have 1 active actors on pipe
2025-10-09T21:48:04.131528Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:04.131831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d501f790-931f101b-31f30425-d22338d4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:04.131843Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d501f790-931f101b-31f30425-d22338d4_0] Write session will restart in 0.007019s
2025-10-09T21:48:04.131869Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d501f790-931f101b-31f30425-d22338d4_0] Write session: Do CDS request
2025-10-09T21:48:04.131875Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d501f790-931f101b-31f30425-d22338d4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:04.140648Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d501f790-931f101b-31f30425-d22338d4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:04.140724Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:04.140740Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1110
2025-10-09T21:48:04.140949Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1110 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:04.140988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1110 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:04.140999Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1110 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:04.141003Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:04.141507Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1110 sessionId:  describe result for acl check
2025-10-09T21:48:04.141542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:04.141549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:04.141550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:04.141556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342089469417499:9789] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:04.142155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342089469417499:9789] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:04.145643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342089469417499:9789] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:04.146563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:04.148892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:04.149126Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:04.149137Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:04.149150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:04.149206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:04.149249Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342089107596035:8733]
2025-10-09T21:48:04.149347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342089469417499:9789] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:04.149364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342089469417499:9789] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:04.149367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342089469417499:9789] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:04.149377Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1110 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:04.149876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:04.150209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:04.150259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:04.150598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:04.150654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:04.150661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:04.150662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:04.150664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:04.150665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:04.150666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:04.150667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.150669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.150670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.150675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:04.150685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:04.150689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:04.150692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:04.150695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:04.150699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:04.150703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.150707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.150711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.150718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:04.150726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:04.150733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:04.150738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:04.150748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:04.150753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:04.150757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:04.150762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:04.150766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:04.150771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:04.150777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:04.150783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:04.150789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.150795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.150799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.150804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.150809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.150815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.150844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:04.157441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:04.158885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:04.158900Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:04.158903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:04.158907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:04.158914Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1109 [2:7559342089107596035:8733]
2025-10-09T21:48:04.158933Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:04.158951Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:04.158954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:04.158958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:04.158960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:04.158978Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:04.158990Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:04.158997Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:04.159061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.159069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.159071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.159073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:04.159147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:04.159156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.159169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.159181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.159198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:04.159225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342089107596030:8733]
2025-10-09T21:48:04.159231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342089107596030:8733]
2025-10-09T21:48:04.159233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342089107596030:8733]
2025-10-09T21:48:04.159242Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:04.159251Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:04.159254Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:04.159256Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:04.159264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342089469417639:9789], now have 1 active actors on pipe
2025-10-09T21:48:04.159268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342089469417456:2432], now have 1 active actors on pipe
2025-10-09T21:48:04.159273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:04.159287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:04.159291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:04.159329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342089107596030:8733]
2025-10-09T21:48:04.159341Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:04.159354Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:04.159329Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1109
2025-10-09T21:48:04.159361Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1109
2025-10-09T21:48:04.159389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:04.159400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:04.159427Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|70bf3600-85441a32-6089d616-3f44533f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:04.159441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:04.159444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:04.159447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:04.159458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:04.159468Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:04.159483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:04.159777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:04.159784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:04.159802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:04.159909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1110 partition: 0 MaxSeqNo: 25 sessionId: src_id|70bf3600-85441a32-6089d616-3f44533f_0
2025-10-09T21:48:04.160309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d501f790-931f101b-31f30425-d22338d4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|70bf3600-85441a32-6089d616-3f44533f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:04.162814Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70bf3600-85441a32-6089d616-3f44533f_0] Write session: try to update token
2025-10-09T21:48:04.162833Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70bf3600-85441a32-6089d616-3f44533f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:04.176399Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1110 sessionId: src_id|70bf3600-85441a32-6089d616-3f44533f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:04.177050Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:04.177072Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:04.177082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342089107596030:8733]
2025-10-09T21:48:04.177138Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:04.177148Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:04.177310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:04.177451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:04.177590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:04.177728Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:04.177932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:04.177984Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:04.178945Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:04.179145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:04.179161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:04.179200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:04.179302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:04.184708Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:04.184722Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:04.184725Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:04.184727Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:04.184737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342089107596030:8733]
2025-10-09T21:48:04.184741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342089107596030:8733]
2025-10-09T21:48:04.184747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342089107596030:8733]
2025-10-09T21:48:04.184791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:04.184795Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:04.184801Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:04.184803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.184805Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:04.184805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:04.184940Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:04.231691Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342089469417655:9750]
2025-10-09T21:48:04.231959Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 1109, pipe: [1:7559342089469417655:9750]
2025-10-09T21:48:04.339325Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:04.346229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:04.346249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:04.346399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.346456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.346508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.346562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.346606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.346756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.346825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.346874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.346928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.346990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.347032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.347038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.347098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.347154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.347212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.347269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.347312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.347319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.347384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.347437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.347489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.347543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.347581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.347586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.347641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.347692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.347743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.347796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.347840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.347851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.347911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.347965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.348019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.348077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.348119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.348126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.348188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.348243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.348301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.348357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.348489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.348497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.348556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.348609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.348668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.348724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.348765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.348772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.348829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.348885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.348944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.349006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.349051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.349060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.349116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.349175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.349231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.349286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.349326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.349332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.349371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342089469417655:9750], now have 1 active actors on pipe
2025-10-09T21:48:04.349604Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:04.349662Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:04.349669Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:04.349711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1110 reason: error at write: Disconnected sessionId: src_id|70bf3600-85441a32-6089d616-3f44533f_0
2025-10-09T21:48:04.349920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1110 sessionId: src_id|70bf3600-85441a32-6089d616-3f44533f_0 is DEAD
2025-10-09T21:48:04.349966Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342089469417655:9750]
2025-10-09T21:48:04.350067Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70bf3600-85441a32-6089d616-3f44533f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:04.350078Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70bf3600-85441a32-6089d616-3f44533f_0] Write session will restart in 0.005678s
2025-10-09T21:48:04.350107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70bf3600-85441a32-6089d616-3f44533f_0] Write session: Do CDS request
2025-10-09T21:48:04.350113Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70bf3600-85441a32-6089d616-3f44533f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:04.350229Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:04.356452Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70bf3600-85441a32-6089d616-3f44533f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:04.356512Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:04.356532Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1111
2025-10-09T21:48:04.356692Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1111 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:04.356728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1111 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:04.356735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1111 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:04.356738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:04.357145Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1111 sessionId:  describe result for acl check
2025-10-09T21:48:04.357179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:04.357186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:04.357188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:04.357194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342089469417680:9794] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:04.357763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342089469417680:9794] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:04.361624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342089469417680:9794] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:04.363858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:04.364296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342089469417680:9794] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:04.364307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342089469417680:9794] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:04.364310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342089469417680:9794] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:04.364321Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1111 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:04.366098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:04.366283Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:04.366295Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:04.366307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:04.366359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:04.366414Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342089107596251:8745]
2025-10-09T21:48:04.366880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:04.367171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:04.367214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:04.367498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:04.367558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:04.367566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:04.367567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:04.367569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:04.367570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:04.367571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:04.367572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.367573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.367574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.367579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:04.367589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:04.367593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:04.367596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:04.367600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:04.367603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:04.367606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.367610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.367613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.367619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:04.367626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:04.367631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:04.367636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:04.367640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:04.367645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:04.367648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:04.367653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:04.367657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:04.367661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:04.367665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:04.367670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:04.367678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.367688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.367692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.367701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.367705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.367714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.367745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:04.373921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:04.374987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:04.375002Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:04.375004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:04.375008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:04.375016Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1110 [2:7559342089107596251:8745]
2025-10-09T21:48:04.375028Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:04.375046Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:04.375055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:04.375058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:04.375061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:04.375075Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:04.375079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:04.375085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:04.375140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.375149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.375151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.375154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:04.375243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:04.375250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.375261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.375272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.375300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:04.375332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342089469417841:9794], now have 1 active actors on pipe
2025-10-09T21:48:04.375335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342089469417661:2432], now have 1 active actors on pipe
2025-10-09T21:48:04.375343Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342089107596245:8745]
2025-10-09T21:48:04.375345Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342089107596245:8745]
2025-10-09T21:48:04.375347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342089107596245:8745]
2025-10-09T21:48:04.375359Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:04.375346Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1110
2025-10-09T21:48:04.375374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:04.375386Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:04.375350Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1110
2025-10-09T21:48:04.375390Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:04.375392Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:04.375393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:04.375396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:04.375443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:04.375456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:04.375469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342089107596245:8745]
2025-10-09T21:48:04.375476Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:04.375486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:04.375517Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d167105-95506274-ca69f90-3d641897_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:04.375530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:04.375534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:04.375536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:04.375546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:04.375560Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:04.375587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:04.375726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:04.375734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:04.375749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:04.375827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1111 partition: 0 MaxSeqNo: 25 sessionId: src_id|d167105-95506274-ca69f90-3d641897_0
2025-10-09T21:48:04.376205Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70bf3600-85441a32-6089d616-3f44533f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d167105-95506274-ca69f90-3d641897_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:04.378551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d167105-95506274-ca69f90-3d641897_0] Write session: try to update token
2025-10-09T21:48:04.378568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d167105-95506274-ca69f90-3d641897_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:04.389236Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:04.389256Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:04.389258Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:04.389260Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:04.389270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342089107596245:8745]
2025-10-09T21:48:04.389274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342089107596245:8745]
2025-10-09T21:48:04.389278Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342089107596245:8745]
2025-10-09T21:48:04.389323Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:04.389324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:04.389329Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:04.389332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.389334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:04.389335Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:04.389466Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:04.390949Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1111 sessionId: src_id|d167105-95506274-ca69f90-3d641897_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:04.393191Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:04.450308Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342089469417865:9750]
2025-10-09T21:48:04.450638Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 1110, pipe: [1:7559342089469417865:9750]
2025-10-09T21:48:04.551389Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:04.551850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:04.551868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:04.551918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342089469417865:9750], now have 1 active actors on pipe
2025-10-09T21:48:04.552190Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:04.552204Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:04.552226Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:04.552286Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342089469417865:9750]
2025-10-09T21:48:04.552306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1111 reason: error at write: Disconnected sessionId: src_id|d167105-95506274-ca69f90-3d641897_0
2025-10-09T21:48:04.552514Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1111 sessionId: src_id|d167105-95506274-ca69f90-3d641897_0 is DEAD
2025-10-09T21:48:04.552698Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d167105-95506274-ca69f90-3d641897_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:04.552710Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d167105-95506274-ca69f90-3d641897_0] Write session will restart in 0.008153s
2025-10-09T21:48:04.552734Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d167105-95506274-ca69f90-3d641897_0] Write session: Do CDS request
2025-10-09T21:48:04.552743Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d167105-95506274-ca69f90-3d641897_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:04.553454Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:04.562604Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d167105-95506274-ca69f90-3d641897_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:04.562653Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:04.562668Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1112
2025-10-09T21:48:04.562843Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1112 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:04.562893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1112 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:04.562906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1112 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:04.562910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:04.563395Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1112 sessionId:  describe result for acl check
2025-10-09T21:48:04.563429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:04.563432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:04.563434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:04.563440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342089469417941:9805] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:04.563997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342089469417941:9805] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:04.566705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342089469417941:9805] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:04.567952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:04.570622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:04.570866Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:04.570876Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:04.570891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:04.570952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:04.570993Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342089107596444:8749]
2025-10-09T21:48:04.571096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342089469417941:9805] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:04.571112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342089469417941:9805] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:04.571116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342089469417941:9805] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:04.571125Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1112 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:04.571604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:04.571987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:04.572073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:04.572520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:04.572648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:04.572655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:04.572656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:04.572657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:04.572659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:04.572660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:04.572661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.572662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.572663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.572669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:04.572679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:04.572684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:04.572688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:04.572691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:04.572695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:04.572699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.572703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.572707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.572715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:04.572723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:04.572731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:04.572738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:04.572744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:04.572751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:04.572757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:04.572763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:04.572779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:04.572792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:04.572804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:04.572811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:04.572825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.572839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.572853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.572866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.572872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.572878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.572915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:04.579539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:04.580860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:04.580877Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:04.580879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:04.580883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:04.580892Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1111 [2:7559342089107596444:8749]
2025-10-09T21:48:04.580906Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:04.580921Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:04.580924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:04.580927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:04.580928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:04.580944Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:04.580952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:04.580958Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:04.581001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.581008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.581010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.581011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:04.581070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:04.581075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.581083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.581089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.581111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:04.581141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342089107596439:8749]
2025-10-09T21:48:04.581144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342089107596439:8749]
2025-10-09T21:48:04.581146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342089107596439:8749]
2025-10-09T21:48:04.581158Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:04.581172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342089469418060:9805], now have 1 active actors on pipe
2025-10-09T21:48:04.581173Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:04.581175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342089469417879:2432], now have 1 active actors on pipe
2025-10-09T21:48:04.581176Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:04.581179Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:04.581184Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:04.581199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:04.581203Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:04.581239Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342089107596439:8749]
2025-10-09T21:48:04.581253Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:04.581265Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:04.581256Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1111
2025-10-09T21:48:04.581257Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1111
2025-10-09T21:48:04.581310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:04.581321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:04.581347Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c9b6fe39-524b93ad-d09a6fa3-dfedfe19_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:04.581358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:04.581362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:04.581365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:04.581373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:04.581385Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:04.581402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:04.581607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:04.581615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:04.581632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:04.581762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1112 partition: 0 MaxSeqNo: 25 sessionId: src_id|c9b6fe39-524b93ad-d09a6fa3-dfedfe19_0
2025-10-09T21:48:04.582219Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d167105-95506274-ca69f90-3d641897_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c9b6fe39-524b93ad-d09a6fa3-dfedfe19_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:04.585644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c9b6fe39-524b93ad-d09a6fa3-dfedfe19_0] Write session: try to update token
2025-10-09T21:48:04.585665Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c9b6fe39-524b93ad-d09a6fa3-dfedfe19_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:04.593178Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:04.593196Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:04.593205Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342089107596439:8749]
2025-10-09T21:48:04.593255Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:04.593257Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:04.593426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:04.593573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:04.593725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:04.593878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:04.594075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:04.594120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:04.601058Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1112 sessionId: src_id|c9b6fe39-524b93ad-d09a6fa3-dfedfe19_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:04.603448Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:04.603686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:04.603718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:04.603761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:04.603879Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:04.606983Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:04.607001Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:04.607004Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:04.607006Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:04.607017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342089107596439:8749]
2025-10-09T21:48:04.607021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342089107596439:8749]
2025-10-09T21:48:04.607025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342089107596439:8749]
2025-10-09T21:48:04.607070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:04.607077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.607080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:04.607085Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:04.607092Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:04.607096Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:04.607222Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:04.652544Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342089469418073:9750]
2025-10-09T21:48:04.652875Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 1111, pipe: [1:7559342089469418073:9750]
2025-10-09T21:48:04.774289Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:04.782647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:04.782673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:04.782837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.782904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.782977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.783044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.783092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.783103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.783176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.783245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.783308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.783383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.783432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.783435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.783498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.783565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.783631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.783707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.783758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.783766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.783837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.783906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.783971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.784038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.784082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.784090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.784161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.784231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.784300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.784367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.784414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.784422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.784489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.784555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.784622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.784687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.784730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.784737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.784803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.784865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.784934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.784998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.785047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.785056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.785121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.785183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.785255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.785322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.785375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.785384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.785446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.785508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.785569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.785638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.785690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.785699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.785778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:04.785841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:04.785910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:04.785973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:04.786023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:04.786049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:04.786118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342089469418073:9750], now have 1 active actors on pipe
2025-10-09T21:48:04.786332Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:04.786362Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342089469418073:9750]
2025-10-09T21:48:04.786388Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:04.786392Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:04.786423Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1112 reason: error at write: Disconnected sessionId: src_id|c9b6fe39-524b93ad-d09a6fa3-dfedfe19_0
2025-10-09T21:48:04.786699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1112 sessionId: src_id|c9b6fe39-524b93ad-d09a6fa3-dfedfe19_0 is DEAD
2025-10-09T21:48:04.786770Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9b6fe39-524b93ad-d09a6fa3-dfedfe19_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:04.786782Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9b6fe39-524b93ad-d09a6fa3-dfedfe19_0] Write session will restart in 0.005982s
2025-10-09T21:48:04.786804Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9b6fe39-524b93ad-d09a6fa3-dfedfe19_0] Write session: Do CDS request
2025-10-09T21:48:04.786810Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9b6fe39-524b93ad-d09a6fa3-dfedfe19_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:04.787055Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:04.793576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c9b6fe39-524b93ad-d09a6fa3-dfedfe19_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:04.793707Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:04.793722Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1113
2025-10-09T21:48:04.793878Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1113 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:04.793912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1113 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:04.793918Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1113 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:04.793921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:04.794378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1113 sessionId:  describe result for acl check
2025-10-09T21:48:04.794413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:04.794421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:04.794423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:04.794430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342089469418092:9809] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:04.795026Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342089469418092:9809] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:04.799918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342089469418092:9809] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:04.804034Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342089469418092:9809] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:04.804049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342089469418092:9809] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:04.804053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342089469418092:9809] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:04.804063Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1113 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:04.804506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:04.806843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:04.807058Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:04.807071Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:04.807084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:04.807140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:04.807184Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342089107596638:8753]
2025-10-09T21:48:04.807805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:04.808130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:04.808177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:04.808515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:04.808569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:04.808577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:04.808578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:04.808579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:04.808581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:04.808582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:04.808583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.808584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.808585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.808589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:04.808600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:04.808604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:04.808607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:04.808611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:04.808614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:04.808618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.808621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.808624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.808632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:04.808639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:04.808645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:04.808651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:04.808655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:04.808660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:04.808664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:04.808668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:04.808673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:04.808677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:04.808682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:04.808687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:04.808691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.808695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.808699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.808704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.808708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.808713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.808738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:04.815822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:04.816907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:04.816923Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:04.816926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:04.816931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:04.816938Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1112 [2:7559342089107596638:8753]
2025-10-09T21:48:04.816954Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:04.816975Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:04.816978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:04.816982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:04.816984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:04.817003Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:04.817010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:04.817019Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:04.817085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.817087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.817092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.817095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:04.817172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:04.817175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.817188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:04.817199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:04.817216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:04.817237Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342089107596632:8753]
2025-10-09T21:48:04.817240Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342089107596632:8753]
2025-10-09T21:48:04.817242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342089107596632:8753]
2025-10-09T21:48:04.817259Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:04.817276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:04.817293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:04.817297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:04.817321Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:04.817324Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:04.817331Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:04.817337Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342089107596632:8753]
2025-10-09T21:48:04.817341Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:04.817359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:04.817370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342089469418077:2432], now have 1 active actors on pipe
2025-10-09T21:48:04.817374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342089469418264:9809], now have 1 active actors on pipe
2025-10-09T21:48:04.817612Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1112
2025-10-09T21:48:04.817657Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1112
2025-10-09T21:48:04.817846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:04.817860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:04.817894Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c54f726f-56e3c96e-4e637154-1e8df114_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:04.817921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:04.817926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:04.817929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:04.817941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:04.817953Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:04.817977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:04.818239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:04.818250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:04.818280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:04.818411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1113 partition: 0 MaxSeqNo: 25 sessionId: src_id|c54f726f-56e3c96e-4e637154-1e8df114_0
2025-10-09T21:48:04.818798Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9b6fe39-524b93ad-d09a6fa3-dfedfe19_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c54f726f-56e3c96e-4e637154-1e8df114_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:04.821311Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c54f726f-56e3c96e-4e637154-1e8df114_0] Write session: try to update token
2025-10-09T21:48:04.821331Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c54f726f-56e3c96e-4e637154-1e8df114_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:04.832797Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:04.832817Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:04.832820Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:04.832823Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:04.832833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342089107596632:8753]
2025-10-09T21:48:04.832839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342089107596632:8753]
2025-10-09T21:48:04.832844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342089107596632:8753]
2025-10-09T21:48:04.832890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:04.832894Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:04.832899Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:04.832899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:04.832902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:04.832904Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:04.833030Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:04.835759Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1113 sessionId: src_id|c54f726f-56e3c96e-4e637154-1e8df114_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:04.838245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:04.886534Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342089469418278:9750]
2025-10-09T21:48:04.886881Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 1112, pipe: [1:7559342089469418278:9750]
2025-10-09T21:48:04.996122Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:04.996556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:04.996571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:04.996617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342089469418278:9750], now have 1 active actors on pipe
2025-10-09T21:48:04.996835Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:04.996877Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:04.996913Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342089469418278:9750]
2025-10-09T21:48:04.996928Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:04.997004Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1113 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c54f726f-56e3c96e-4e637154-1e8df114_0
2025-10-09T21:48:04.997183Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1113 sessionId: src_id|c54f726f-56e3c96e-4e637154-1e8df114_0 is DEAD
2025-10-09T21:48:04.997350Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c54f726f-56e3c96e-4e637154-1e8df114_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:04.997363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c54f726f-56e3c96e-4e637154-1e8df114_0] Write session will restart in 0.006133s
2025-10-09T21:48:04.997392Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c54f726f-56e3c96e-4e637154-1e8df114_0] Write session: Do CDS request
2025-10-09T21:48:04.997398Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c54f726f-56e3c96e-4e637154-1e8df114_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:05.004569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c54f726f-56e3c96e-4e637154-1e8df114_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:05.004666Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:05.004682Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1114
2025-10-09T21:48:05.004841Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1114 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:05.004882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1114 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:05.004892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1114 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:05.004896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:05.005392Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1114 sessionId:  describe result for acl check
2025-10-09T21:48:05.005432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:05.005435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:05.005437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:05.005443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342093764385599:9814] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:05.006002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342093764385599:9814] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:05.008716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342093764385599:9814] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:05.011744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:05.011812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342093764385599:9814] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:05.011828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342093764385599:9814] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:05.011832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342093764385599:9814] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:05.011842Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1114 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:05.014577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:05.014813Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:05.014821Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:05.014834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:05.014895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:05.014942Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342093402564122:8755]
2025-10-09T21:48:05.015551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:05.015853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:05.015897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:05.016224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:05.016277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.016285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.016286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.016288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.016289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.016290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.016291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.016292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.016294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.016299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.016311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.016316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.016320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.016324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.016328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.016332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.016335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.016339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.016347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.016356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.016362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.016368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.016375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.016380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.016386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.016391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.016396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.016402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.016407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.016413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.016418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.016424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.016428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.016434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.016439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.016445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.016478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:05.023325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:05.024470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:05.024493Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:05.024496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:05.024501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:05.024508Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1113 [2:7559342093402564122:8755]
2025-10-09T21:48:05.024524Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:05.024546Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:05.024550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:05.024554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:05.024556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:05.024579Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:05.024591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:05.024599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:05.024670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.024680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.024682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.024686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:05.024762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:05.024773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.024787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.024798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.024818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:05.024844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342093402564116:8755]
2025-10-09T21:48:05.024852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342093402564116:8755]
2025-10-09T21:48:05.024854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342093402564116:8755]
2025-10-09T21:48:05.024868Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:05.024891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342093764385762:9814], now have 1 active actors on pipe
2025-10-09T21:48:05.024892Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:05.024895Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:05.024897Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:05.024900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342089469418284:2432], now have 1 active actors on pipe
2025-10-09T21:48:05.024909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:05.024925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:05.024929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:05.024976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342093402564116:8755]
2025-10-09T21:48:05.024965Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1113
2025-10-09T21:48:05.024992Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:05.024972Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1113
2025-10-09T21:48:05.025026Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:05.025052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:05.025061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:05.025091Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|63abe7d3-37fb2eb0-97675716-53f7e57c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:05.025105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:05.025114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:05.025117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:05.025129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:05.025140Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:05.025160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:05.025276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:05.025283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:05.025298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:05.025399Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1114 partition: 0 MaxSeqNo: 25 sessionId: src_id|63abe7d3-37fb2eb0-97675716-53f7e57c_0
2025-10-09T21:48:05.025788Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c54f726f-56e3c96e-4e637154-1e8df114_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|63abe7d3-37fb2eb0-97675716-53f7e57c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:05.028412Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63abe7d3-37fb2eb0-97675716-53f7e57c_0] Write session: try to update token
2025-10-09T21:48:05.028431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63abe7d3-37fb2eb0-97675716-53f7e57c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:05.042190Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1114 sessionId: src_id|63abe7d3-37fb2eb0-97675716-53f7e57c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:05.043328Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:05.043347Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:05.043358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342093402564116:8755]
2025-10-09T21:48:05.043415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:05.043428Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:05.043602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:05.043746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:05.043890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:05.044032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:05.044232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:05.044280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:05.044599Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:05.044786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:05.044803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:05.044839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:05.044919Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:05.050148Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:05.050161Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:05.050164Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:05.050166Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:05.050176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342093402564116:8755]
2025-10-09T21:48:05.050180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342093402564116:8755]
2025-10-09T21:48:05.050184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342093402564116:8755]
2025-10-09T21:48:05.050225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:05.050228Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:05.050233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.050234Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:05.050236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:05.050239Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:05.050370Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:05.096962Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342093764385774:9750]
2025-10-09T21:48:05.097257Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 1113, pipe: [1:7559342093764385774:9750]
2025-10-09T21:48:05.212534Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:05.220605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:05.220639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:05.220814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:05.220879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:05.220948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:05.221019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:05.221071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:05.221076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:05.221147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:05.221205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:05.221267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:05.221331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:05.221371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:05.221373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:05.221439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:05.221496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:05.221552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:05.221620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:05.221662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:05.221666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:05.221740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:05.221814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:05.221875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:05.221933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:05.221977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:05.221979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:05.222184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:05.222251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:05.222316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:05.222384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:05.222428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:05.222436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:05.222499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:05.222563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:05.222628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:05.222697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:05.222750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:05.222761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:05.222828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:05.222887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:05.222950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:05.223014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:05.223060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:05.223069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:05.223128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:05.223189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:05.223251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:05.223310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:05.223356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:05.223365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:05.223426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:05.223486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:05.223549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:05.223608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:05.223652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:05.223659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:05.223722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:05.223782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:05.223846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:05.223907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:05.223954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:05.223962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:05.224030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342093764385774:9750], now have 1 active actors on pipe
2025-10-09T21:48:05.224378Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342093764385774:9750]
2025-10-09T21:48:05.224413Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:05.224417Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:05.224449Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1114 reason: error at write: Disconnected sessionId: src_id|63abe7d3-37fb2eb0-97675716-53f7e57c_0
2025-10-09T21:48:05.224614Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:05.224653Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1114 sessionId: src_id|63abe7d3-37fb2eb0-97675716-53f7e57c_0 is DEAD
2025-10-09T21:48:05.224806Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63abe7d3-37fb2eb0-97675716-53f7e57c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:05.224818Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63abe7d3-37fb2eb0-97675716-53f7e57c_0] Write session will restart in 0.006787s
2025-10-09T21:48:05.224843Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63abe7d3-37fb2eb0-97675716-53f7e57c_0] Write session: Do CDS request
2025-10-09T21:48:05.224850Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63abe7d3-37fb2eb0-97675716-53f7e57c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:05.225056Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:05.232511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63abe7d3-37fb2eb0-97675716-53f7e57c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:05.232616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:05.232637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1115
2025-10-09T21:48:05.232795Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1115 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:05.232834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1115 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:05.232845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1115 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:05.232850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:05.233335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1115 sessionId:  describe result for acl check
2025-10-09T21:48:05.233371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:05.233378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:05.233380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:05.233386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342093764385805:9819] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:05.234022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342093764385805:9819] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:05.237372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342093764385805:9819] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:05.240819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342093764385805:9819] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:05.240833Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342093764385805:9819] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:05.240837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342093764385805:9819] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:05.240846Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1115 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:05.241679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:05.244069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:05.244396Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:05.244406Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:05.244420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:05.244484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:05.244528Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342093402564311:8757]
2025-10-09T21:48:05.245181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:05.245532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:05.245581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:05.245960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:05.246013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.246019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.246020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.246021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.246023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.246024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.246025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.246026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.246041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.246048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.246060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.246064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.246069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.246073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.246077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.246081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.246085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.246089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.246096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.246105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.246111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.246124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.246129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.246134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.246144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.246150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.246154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.246160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.246164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.246174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.246179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.246184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.246188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.246199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.246203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.246210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.246244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:05.253005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:05.254126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:05.254141Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:05.254145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:05.254150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:05.254158Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1114 [2:7559342093402564311:8757]
2025-10-09T21:48:05.254174Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:05.254193Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:05.254196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:05.254200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:05.254203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:05.254225Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:05.254232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:05.254240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:05.254301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.254308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.254312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.254315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:05.254391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:05.254403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.254417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.254430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.254449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:05.254471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342093764385969:9819], now have 1 active actors on pipe
2025-10-09T21:48:05.254479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342093764385785:2432], now have 1 active actors on pipe
2025-10-09T21:48:05.254487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342093402564306:8757]
2025-10-09T21:48:05.254490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342093402564306:8757]
2025-10-09T21:48:05.254492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342093402564306:8757]
2025-10-09T21:48:05.254505Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:05.254518Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:05.254527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:05.254528Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:05.254530Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:05.254542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:05.254546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:05.254564Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1114
2025-10-09T21:48:05.254570Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1114
2025-10-09T21:48:05.254587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342093402564306:8757]
2025-10-09T21:48:05.254597Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:05.254605Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:05.254628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:05.254633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:05.254671Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f8bf4eff-bcf841aa-c1dfaafa-bdd74618_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:05.254685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:05.254689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:05.254692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:05.254703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:05.254719Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:05.254750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:05.254909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:05.254918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:05.254933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:05.255056Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1115 partition: 0 MaxSeqNo: 25 sessionId: src_id|f8bf4eff-bcf841aa-c1dfaafa-bdd74618_0
2025-10-09T21:48:05.255455Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63abe7d3-37fb2eb0-97675716-53f7e57c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f8bf4eff-bcf841aa-c1dfaafa-bdd74618_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:05.258102Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f8bf4eff-bcf841aa-c1dfaafa-bdd74618_0] Write session: try to update token
2025-10-09T21:48:05.258121Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f8bf4eff-bcf841aa-c1dfaafa-bdd74618_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:05.268823Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:05.268840Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:05.268842Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:05.268844Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:05.268854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342093402564306:8757]
2025-10-09T21:48:05.268859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342093402564306:8757]
2025-10-09T21:48:05.268863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342093402564306:8757]
2025-10-09T21:48:05.268912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:05.268916Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:05.268919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.268922Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:05.268922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:05.268926Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:05.269061Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:05.274317Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1115 sessionId: src_id|f8bf4eff-bcf841aa-c1dfaafa-bdd74618_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:05.276797Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:05.324823Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342093764385984:9750]
2025-10-09T21:48:05.325122Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 1114, pipe: [1:7559342093764385984:9750]
2025-10-09T21:48:05.422682Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:05.423333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:05.423350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:05.423399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342093764385984:9750], now have 1 active actors on pipe
2025-10-09T21:48:05.423600Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:05.423671Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:05.423683Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:05.423689Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:05.423730Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342093764385984:9750]
2025-10-09T21:48:05.423753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1115 reason: error at write: Disconnected sessionId: src_id|f8bf4eff-bcf841aa-c1dfaafa-bdd74618_0
2025-10-09T21:48:05.423951Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1115 sessionId: src_id|f8bf4eff-bcf841aa-c1dfaafa-bdd74618_0 is DEAD
2025-10-09T21:48:05.424119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f8bf4eff-bcf841aa-c1dfaafa-bdd74618_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:05.424129Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f8bf4eff-bcf841aa-c1dfaafa-bdd74618_0] Write session will restart in 0.007396s
2025-10-09T21:48:05.424158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f8bf4eff-bcf841aa-c1dfaafa-bdd74618_0] Write session: Do CDS request
2025-10-09T21:48:05.424176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f8bf4eff-bcf841aa-c1dfaafa-bdd74618_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:05.432555Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f8bf4eff-bcf841aa-c1dfaafa-bdd74618_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:05.432807Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:05.432823Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1116
2025-10-09T21:48:05.432980Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1116 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:05.433033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1116 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:05.433043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1116 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:05.433047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:05.433537Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1116 sessionId:  describe result for acl check
2025-10-09T21:48:05.433572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:05.433574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:05.433576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:05.433582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342093764386024:9823] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:05.434254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342093764386024:9823] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:05.436857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342093764386024:9823] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:05.439265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:05.439567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342093764386024:9823] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:05.439580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342093764386024:9823] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:05.439583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342093764386024:9823] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:05.439591Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1116 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:05.441879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:05.442101Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:05.442111Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:05.442124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:05.442173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:05.442210Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342093402564532:8769]
2025-10-09T21:48:05.442943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:05.443249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:05.443290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:05.443604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:05.443651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.443658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.443658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.443659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.443661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.443661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.443662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.443663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.443665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.443669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.443679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.443682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.443685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.443688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.443691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.443694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.443697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.443700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.443707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.443714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.443721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.443725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.443729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.443733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.443737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.443741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.443745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.443749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.443753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.443757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.443761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.443765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.443769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.443775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.443779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.443783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.443806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:05.450261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:05.451328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:05.451344Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:05.451347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:05.451351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:05.451358Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1115 [2:7559342093402564532:8769]
2025-10-09T21:48:05.451374Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:05.451395Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:05.451406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:05.451409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:05.451411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:05.451429Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:05.451435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:05.451442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:05.451501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.451508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.451510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.451512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:05.451583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:05.451591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.451602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.451613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.451626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:05.451651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342093402564526:8769]
2025-10-09T21:48:05.451656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342093402564526:8769]
2025-10-09T21:48:05.451658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342093402564526:8769]
2025-10-09T21:48:05.451673Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:05.451689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342093764385988:2432], now have 1 active actors on pipe
2025-10-09T21:48:05.451695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342093764386172:9823], now have 1 active actors on pipe
2025-10-09T21:48:05.451703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:05.451702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:05.451707Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:05.451710Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:05.451715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:05.451718Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:05.451755Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342093402564526:8769]
2025-10-09T21:48:05.451767Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:05.451773Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:05.451793Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1115
2025-10-09T21:48:05.451797Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1115
2025-10-09T21:48:05.451887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:05.451911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:05.451949Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b15d4206-1e7b5736-b1663114-663cd430_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:05.451965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:05.451970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:05.451973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:05.451986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:05.451998Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:05.452017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:05.452210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:05.452220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:05.452235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:05.452334Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1116 partition: 0 MaxSeqNo: 25 sessionId: src_id|b15d4206-1e7b5736-b1663114-663cd430_0
2025-10-09T21:48:05.452703Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f8bf4eff-bcf841aa-c1dfaafa-bdd74618_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b15d4206-1e7b5736-b1663114-663cd430_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:05.455260Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b15d4206-1e7b5736-b1663114-663cd430_0] Write session: try to update token
2025-10-09T21:48:05.455282Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b15d4206-1e7b5736-b1663114-663cd430_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:05.468569Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:05.468591Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:05.468593Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:05.468596Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:05.468605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342093402564526:8769]
2025-10-09T21:48:05.468610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342093402564526:8769]
2025-10-09T21:48:05.468613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342093402564526:8769]
2025-10-09T21:48:05.468659Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:05.468661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:05.468665Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:05.468668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.468669Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:05.468671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:05.468810Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:05.471825Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1116 sessionId: src_id|b15d4206-1e7b5736-b1663114-663cd430_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:05.474295Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:05.523964Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342093764386210:9750]
2025-10-09T21:48:05.524256Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 1115, pipe: [1:7559342093764386210:9750]
2025-10-09T21:48:05.624689Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:05.625323Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342093764386210:9750]
2025-10-09T21:48:05.625354Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:05.625355Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:05.625358Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:05.625421Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1116 reason: error at write: Disconnected sessionId: src_id|b15d4206-1e7b5736-b1663114-663cd430_0
2025-10-09T21:48:05.625646Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1116 sessionId: src_id|b15d4206-1e7b5736-b1663114-663cd430_0 is DEAD
2025-10-09T21:48:05.625854Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b15d4206-1e7b5736-b1663114-663cd430_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:05.625872Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b15d4206-1e7b5736-b1663114-663cd430_0] Write session will restart in 0.008167s
2025-10-09T21:48:05.625896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b15d4206-1e7b5736-b1663114-663cd430_0] Write session: Do CDS request
2025-10-09T21:48:05.625904Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b15d4206-1e7b5736-b1663114-663cd430_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:05.624988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:05.625005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:05.625050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342093764386210:9750], now have 1 active actors on pipe
2025-10-09T21:48:05.625691Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:05.635659Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b15d4206-1e7b5736-b1663114-663cd430_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:05.635766Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:05.635784Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1117
2025-10-09T21:48:05.635930Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1117 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:05.635976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1117 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:05.635986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1117 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:05.635990Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:05.636474Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1117 sessionId:  describe result for acl check
2025-10-09T21:48:05.636514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:05.636517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:05.636519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:05.636526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342093764386235:9834] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:05.637096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342093764386235:9834] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:05.640274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342093764386235:9834] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:05.642600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:05.642911Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342093764386235:9834] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:05.642926Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342093764386235:9834] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:05.642929Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342093764386235:9834] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:05.642939Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1117 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:05.645090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:05.645288Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:05.645298Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:05.645311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:05.645364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:05.645405Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342093402564735:8774]
2025-10-09T21:48:05.646021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:05.646383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:05.646456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:05.646812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:05.646869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.646875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.646876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.646877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.646878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.646879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.646880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.646882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.646883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.646889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.646900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.646903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.646906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.646910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.646913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.646917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.646920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.646923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.646933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.646941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.646949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.646954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.646958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.646963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.646968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.646972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.646977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.646982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.646986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.646993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.646997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.647002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.647006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.647011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.647014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.647018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.647047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:05.653538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:05.654611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:05.654626Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:05.654629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:05.654633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:05.654641Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1116 [2:7559342093402564735:8774]
2025-10-09T21:48:05.654655Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:05.654674Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:05.654681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:05.654684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:05.654686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:05.654705Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:05.654714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:05.654722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:05.654780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.654788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.654790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.654793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:05.654864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:05.654872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.654882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.654895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.654911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:05.654938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342093764386211:2432], now have 1 active actors on pipe
2025-10-09T21:48:05.654945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342093764386400:9834], now have 1 active actors on pipe
2025-10-09T21:48:05.654953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342093402564729:8774]
2025-10-09T21:48:05.654959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342093402564729:8774]
2025-10-09T21:48:05.654961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342093402564729:8774]
2025-10-09T21:48:05.654975Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:05.654988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:05.655001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:05.655010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:05.655009Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:05.655014Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:05.655016Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:05.655046Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342093402564729:8774]
2025-10-09T21:48:05.655054Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:05.655062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:05.655043Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1116
2025-10-09T21:48:05.655092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:05.655043Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1116
2025-10-09T21:48:05.655101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:05.655133Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|247575ce-1d660648-39afddff-3ce6d42d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:05.655144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:05.655148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:05.655151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:05.655162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:05.655172Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:05.655187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:05.655325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:05.655333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:05.655345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:05.655428Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1117 partition: 0 MaxSeqNo: 25 sessionId: src_id|247575ce-1d660648-39afddff-3ce6d42d_0
2025-10-09T21:48:05.655803Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b15d4206-1e7b5736-b1663114-663cd430_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|247575ce-1d660648-39afddff-3ce6d42d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:05.658458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|247575ce-1d660648-39afddff-3ce6d42d_0] Write session: try to update token
2025-10-09T21:48:05.658476Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|247575ce-1d660648-39afddff-3ce6d42d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:05.669240Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:05.669258Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:05.669261Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:05.669263Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:05.669272Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342093402564729:8774]
2025-10-09T21:48:05.669276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342093402564729:8774]
2025-10-09T21:48:05.669280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342093402564729:8774]
2025-10-09T21:48:05.669327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:05.669330Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:05.669336Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:05.669336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.669339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:05.669342Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:05.669476Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:05.672365Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1117 sessionId: src_id|247575ce-1d660648-39afddff-3ce6d42d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:05.674975Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:05.725671Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342093764386412:9750]
2025-10-09T21:48:05.726063Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 1116, pipe: [1:7559342093764386412:9750]
2025-10-09T21:48:05.823425Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:05.823996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:05.824015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:05.824063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342093764386412:9750], now have 1 active actors on pipe
2025-10-09T21:48:05.824381Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:05.824447Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:05.824468Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:05.824502Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342093764386412:9750]
2025-10-09T21:48:05.824528Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1117 reason: error at write: Disconnected sessionId: src_id|247575ce-1d660648-39afddff-3ce6d42d_0
2025-10-09T21:48:05.824706Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:05.824900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|247575ce-1d660648-39afddff-3ce6d42d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:05.824921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|247575ce-1d660648-39afddff-3ce6d42d_0] Write session will restart in 0.005920s
2025-10-09T21:48:05.824945Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|247575ce-1d660648-39afddff-3ce6d42d_0] Write session: Do CDS request
2025-10-09T21:48:05.824954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|247575ce-1d660648-39afddff-3ce6d42d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:05.825221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1117 sessionId: src_id|247575ce-1d660648-39afddff-3ce6d42d_0 is DEAD
2025-10-09T21:48:05.832542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|247575ce-1d660648-39afddff-3ce6d42d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:05.832674Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:05.832688Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1118
2025-10-09T21:48:05.832849Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1118 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:05.832896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1118 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:05.832907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1118 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:05.832911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:05.833349Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1118 sessionId:  describe result for acl check
2025-10-09T21:48:05.833383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:05.833389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:05.833391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:05.833397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342093764386449:9839] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:05.833968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342093764386449:9839] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:05.837096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342093764386449:9839] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:05.840070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342093764386449:9839] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:05.840085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342093764386449:9839] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:05.840088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342093764386449:9839] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:05.840098Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1118 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:05.840489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:05.842857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:05.843080Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:05.843090Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:05.843103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:05.843158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:05.843201Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342093402564927:8777]
2025-10-09T21:48:05.843796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:05.844075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:05.844114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:05.844416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:05.844468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.844474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.844475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.844476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.844477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.844479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.844480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.844481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.844482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.844487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.844497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.844501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.844505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.844509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.844514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.844517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.844521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.844524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.844531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.844539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:05.844545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.844551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:05.844555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.844560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:05.844564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.844569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:05.844580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.844585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:05.844589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.844594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:05.844599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.844604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.844608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.844613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.844618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.844622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.844652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:05.851115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:05.852449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:05.852465Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:05.852468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:05.852472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:05.852478Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1117 [2:7559342093402564927:8777]
2025-10-09T21:48:05.852494Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:05.852513Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:05.852516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:05.852520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:05.852522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:05.852540Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:05.852552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:05.852559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:05.852621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.852626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.852628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.852631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:05.852703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:05.852709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.852722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:05.852737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:05.852753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:05.852776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342093764386417:2432], now have 1 active actors on pipe
2025-10-09T21:48:05.852784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342093764386613:9839], now have 1 active actors on pipe
2025-10-09T21:48:05.852792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342093402564922:8777]
2025-10-09T21:48:05.852798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342093402564922:8777]
2025-10-09T21:48:05.852800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342093402564922:8777]
2025-10-09T21:48:05.852813Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:05.852829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:05.852831Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:05.852834Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:05.852845Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:05.852846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:05.852850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:05.852832Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1117
2025-10-09T21:48:05.852887Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1117
2025-10-09T21:48:05.852890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342093402564922:8777]
2025-10-09T21:48:05.852902Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:05.852913Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:05.853375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1118 partition: 0 MaxSeqNo: 25 sessionId: src_id|fb16a640-26b2f521-e653cccd-eca34266_0
2025-10-09T21:48:05.852931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:05.852944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:05.852998Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fb16a640-26b2f521-e653cccd-eca34266_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:05.853007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:05.853011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:05.853013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:05.853024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:05.853035Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:05.853054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:05.853264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:05.853270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:05.853291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:05.853753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|247575ce-1d660648-39afddff-3ce6d42d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fb16a640-26b2f521-e653cccd-eca34266_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:05.856209Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fb16a640-26b2f521-e653cccd-eca34266_0] Write session: try to update token
2025-10-09T21:48:05.856227Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fb16a640-26b2f521-e653cccd-eca34266_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:05.867983Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:05.868001Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:05.868003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:05.868006Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:05.868016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342093402564922:8777]
2025-10-09T21:48:05.868020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342093402564922:8777]
2025-10-09T21:48:05.868024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342093402564922:8777]
2025-10-09T21:48:05.868069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:05.868073Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:05.868076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:05.868079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:05.868080Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:05.868084Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:05.868208Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:05.869473Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1118 sessionId: src_id|fb16a640-26b2f521-e653cccd-eca34266_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:05.871908Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:05.924857Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342093764386626:9750]
2025-10-09T21:48:05.925316Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 1117, pipe: [1:7559342093764386626:9750]
2025-10-09T21:48:06.027950Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:06.028236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:06.028263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:06.028329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342093764386626:9750], now have 1 active actors on pipe
2025-10-09T21:48:06.028602Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342093764386626:9750]
2025-10-09T21:48:06.028609Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:06.028614Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:06.028648Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:06.028675Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1118 reason: error at write: Disconnected sessionId: src_id|fb16a640-26b2f521-e653cccd-eca34266_0
2025-10-09T21:48:06.028774Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:06.028865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1118 sessionId: src_id|fb16a640-26b2f521-e653cccd-eca34266_0 is DEAD
2025-10-09T21:48:06.029021Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb16a640-26b2f521-e653cccd-eca34266_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:06.029033Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb16a640-26b2f521-e653cccd-eca34266_0] Write session will restart in 0.007006s
2025-10-09T21:48:06.029063Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb16a640-26b2f521-e653cccd-eca34266_0] Write session: Do CDS request
2025-10-09T21:48:06.029069Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb16a640-26b2f521-e653cccd-eca34266_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:06.037593Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fb16a640-26b2f521-e653cccd-eca34266_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:06.037669Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:06.037684Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1119
2025-10-09T21:48:06.038210Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1119 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:06.038267Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1119 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:06.038280Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1119 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:06.038285Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:06.039143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1119 sessionId:  describe result for acl check
2025-10-09T21:48:06.039184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:06.039186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:06.039188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:06.039195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342098059353952:9843] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:06.039746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342098059353952:9843] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:06.044605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342098059353952:9843] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:06.048630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342098059353952:9843] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:06.048647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342098059353952:9843] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:06.048651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342098059353952:9843] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:06.048661Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1119 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:06.050136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:06.052543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:06.052765Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:06.052778Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:06.052791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:06.052852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:06.052893Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342097697532416:8779]
2025-10-09T21:48:06.053514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:06.053890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:06.053948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:06.054293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:06.054364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.054374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.054375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.054376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.054378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.054379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.054380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.054381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.054382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.054388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.054399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.054402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.054406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.054410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.054414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.054417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.054420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.054423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.054431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.054439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.054445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.054450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.054455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.054459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.054464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.054469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.054474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.054480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.054485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.054489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.054493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.054498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.054501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.054506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.054510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.054515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.054542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:06.061240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:06.062481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:06.062496Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:06.062499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:06.062503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:06.062511Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1118 [2:7559342097697532416:8779]
2025-10-09T21:48:06.062527Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:06.062548Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:06.062551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:06.062554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:06.062557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:06.062576Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:06.062587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:06.062595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:06.062660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.062670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.062672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.062674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:06.062745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:06.062753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.062766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.062780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.062800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:06.062823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342098059354114:9843], now have 1 active actors on pipe
2025-10-09T21:48:06.062832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342098059353925:2432], now have 1 active actors on pipe
2025-10-09T21:48:06.062840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342097697532411:8779]
2025-10-09T21:48:06.062844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342097697532411:8779]
2025-10-09T21:48:06.062845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342097697532411:8779]
2025-10-09T21:48:06.062858Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:06.062877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:06.062892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:06.062894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:06.062895Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:06.062897Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:06.062900Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:06.062922Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1118
2025-10-09T21:48:06.062966Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1118
2025-10-09T21:48:06.062937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342097697532411:8779]
2025-10-09T21:48:06.062948Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:06.062969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:06.062985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:06.062990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:06.063019Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d80c3de3-90bee238-69b39e2-d349a4ec_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:06.063028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:06.063032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:06.063034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:06.063048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:06.063059Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:06.063077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:06.063275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:06.063283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:06.063297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:06.063379Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1119 partition: 0 MaxSeqNo: 25 sessionId: src_id|d80c3de3-90bee238-69b39e2-d349a4ec_0
2025-10-09T21:48:06.063777Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb16a640-26b2f521-e653cccd-eca34266_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d80c3de3-90bee238-69b39e2-d349a4ec_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:06.066813Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d80c3de3-90bee238-69b39e2-d349a4ec_0] Write session: try to update token
2025-10-09T21:48:06.066834Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d80c3de3-90bee238-69b39e2-d349a4ec_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:06.078117Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:06.078138Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:06.078141Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:06.078144Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:06.078154Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342097697532411:8779]
2025-10-09T21:48:06.078159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342097697532411:8779]
2025-10-09T21:48:06.078163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342097697532411:8779]
2025-10-09T21:48:06.078213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:06.078219Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:06.078225Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:06.078226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.078229Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:06.078229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:06.078367Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:06.082497Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1119 sessionId: src_id|d80c3de3-90bee238-69b39e2-d349a4ec_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:06.085029Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:06.128641Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342098059354132:9750]
2025-10-09T21:48:06.128936Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 1118, pipe: [1:7559342098059354132:9750]
2025-10-09T21:48:06.241432Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:06.241931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:06.241950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:06.241996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342098059354132:9750], now have 1 active actors on pipe
2025-10-09T21:48:06.242248Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342098059354132:9750]
2025-10-09T21:48:06.242284Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:06.242287Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:06.242345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1119 reason: error at write: Disconnected sessionId: src_id|d80c3de3-90bee238-69b39e2-d349a4ec_0
2025-10-09T21:48:06.242396Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:06.242529Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1119 sessionId: src_id|d80c3de3-90bee238-69b39e2-d349a4ec_0 is DEAD
2025-10-09T21:48:06.242678Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:06.242729Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d80c3de3-90bee238-69b39e2-d349a4ec_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:06.242740Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d80c3de3-90bee238-69b39e2-d349a4ec_0] Write session will restart in 0.006891s
2025-10-09T21:48:06.242768Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d80c3de3-90bee238-69b39e2-d349a4ec_0] Write session: Do CDS request
2025-10-09T21:48:06.242775Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d80c3de3-90bee238-69b39e2-d349a4ec_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:06.250518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d80c3de3-90bee238-69b39e2-d349a4ec_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:06.250587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:06.250603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1120
2025-10-09T21:48:06.250802Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1120 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:06.250846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1120 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:06.250852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1120 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:06.250856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:06.251281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1120 sessionId:  describe result for acl check
2025-10-09T21:48:06.251320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:06.251323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:06.251325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:06.251331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342098059354169:9848] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:06.251887Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342098059354169:9848] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:06.255245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342098059354169:9848] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:06.258754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342098059354169:9848] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:06.258769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342098059354169:9848] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:06.258773Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342098059354169:9848] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:06.258784Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1120 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:06.260043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:06.262795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:06.263048Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:06.263059Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:06.263072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:06.263129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:06.263165Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342097697532612:8781]
2025-10-09T21:48:06.263815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:06.264153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:06.264199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:06.264565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:06.264618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.264625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.264627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.264628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.264630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.264631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.264632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.264633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.264635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.264640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.264653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.264658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.264662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.264666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.264670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.264675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.264679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.264683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.264691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.264700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.264707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.264713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.264718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.264724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.264729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.264735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.264740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.264746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.264750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.264755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.264761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.264767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.264772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.264778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.264783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.264789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.264821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:06.271746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:06.272918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:06.272933Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:06.272935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:06.272940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:06.272949Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1119 [2:7559342097697532612:8781]
2025-10-09T21:48:06.272963Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:06.272984Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:06.272992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:06.272995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:06.272997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:06.273016Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:06.273026Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:06.273033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:06.273343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.273354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.273356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.273359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:06.273436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:06.273444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.273457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.273469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.273488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:06.273515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342098059354328:9848], now have 1 active actors on pipe
2025-10-09T21:48:06.273524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342098059354140:2432], now have 1 active actors on pipe
2025-10-09T21:48:06.273533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342097697532607:8781]
2025-10-09T21:48:06.273536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342097697532607:8781]
2025-10-09T21:48:06.273538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342097697532607:8781]
2025-10-09T21:48:06.273550Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:06.273565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:06.273567Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:06.273570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:06.273572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:06.273586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:06.273589Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:06.273627Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342097697532607:8781]
2025-10-09T21:48:06.273642Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:06.273641Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1119
2025-10-09T21:48:06.273686Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1119
2025-10-09T21:48:06.273726Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:06.273854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:06.273869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:06.273915Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|15485a06-7be364b9-e6b7a97f-79abc05e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:06.273930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:06.273939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:06.273942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:06.273955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:06.273967Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:06.273989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:06.274141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:06.274152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:06.274167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:06.274324Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1120 partition: 0 MaxSeqNo: 25 sessionId: src_id|15485a06-7be364b9-e6b7a97f-79abc05e_0
2025-10-09T21:48:06.274733Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d80c3de3-90bee238-69b39e2-d349a4ec_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|15485a06-7be364b9-e6b7a97f-79abc05e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:06.277192Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15485a06-7be364b9-e6b7a97f-79abc05e_0] Write session: try to update token
2025-10-09T21:48:06.277212Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15485a06-7be364b9-e6b7a97f-79abc05e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:06.289412Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:06.289435Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:06.289438Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:06.289440Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:06.289451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342097697532607:8781]
2025-10-09T21:48:06.289456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342097697532607:8781]
2025-10-09T21:48:06.289461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342097697532607:8781]
2025-10-09T21:48:06.289510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:06.289517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.289520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:06.289527Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:06.289533Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:06.289540Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:06.289655Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:06.293326Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1120 sessionId: src_id|15485a06-7be364b9-e6b7a97f-79abc05e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:06.295911Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:06.342673Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342098059354344:9750]
2025-10-09T21:48:06.343057Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 1119, pipe: [1:7559342098059354344:9750]
2025-10-09T21:48:06.449263Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:06.449663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:06.449679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:06.449723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342098059354344:9750], now have 1 active actors on pipe
2025-10-09T21:48:06.449953Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:06.449937Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:06.449968Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:06.449978Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342098059354344:9750]
2025-10-09T21:48:06.450014Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:06.450053Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1120 reason: error at write: Disconnected sessionId: src_id|15485a06-7be364b9-e6b7a97f-79abc05e_0
2025-10-09T21:48:06.450218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1120 sessionId: src_id|15485a06-7be364b9-e6b7a97f-79abc05e_0 is DEAD
2025-10-09T21:48:06.450334Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15485a06-7be364b9-e6b7a97f-79abc05e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:06.450343Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15485a06-7be364b9-e6b7a97f-79abc05e_0] Write session will restart in 0.006052s
2025-10-09T21:48:06.450362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15485a06-7be364b9-e6b7a97f-79abc05e_0] Write session: Do CDS request
2025-10-09T21:48:06.450367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15485a06-7be364b9-e6b7a97f-79abc05e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:06.457587Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15485a06-7be364b9-e6b7a97f-79abc05e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:06.457715Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:06.457731Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1121
2025-10-09T21:48:06.457873Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1121 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:06.457908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1121 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:06.457918Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1121 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:06.457922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:06.458410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1121 sessionId:  describe result for acl check
2025-10-09T21:48:06.458451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:06.458458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:06.458460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:06.458466Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342098059354364:9852] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:06.459058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342098059354364:9852] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:06.462110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342098059354364:9852] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:06.464893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342098059354364:9852] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:06.464909Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342098059354364:9852] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:06.464913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342098059354364:9852] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:06.464922Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1121 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:06.465925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:06.468266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:06.468464Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:06.468474Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:06.468485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:06.468542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:06.468586Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342097697532837:8793]
2025-10-09T21:48:06.469164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:06.469465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:06.469506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:06.469833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:06.469883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.469890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.469891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.469892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.469894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.469895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.469895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.469897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.469898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.469903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.469913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.469917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.469921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.469925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.469929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.469932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.469935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.469939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.469946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.469958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.469964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.469969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.469973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.469978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.469983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.469988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.469994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.469999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.470003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.470012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.470018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.470023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.470027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.470065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.470069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.470074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.470100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:06.476423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:06.477621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:06.477640Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:06.477644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:06.477648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:06.477655Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1120 [2:7559342097697532837:8793]
2025-10-09T21:48:06.477940Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:06.477967Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:06.477970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:06.477974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:06.477976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:06.477994Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:06.478001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:06.478008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:06.478086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.478095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.478097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.478100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:06.478173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:06.478181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.478194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.478212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.478232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:06.478259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342098059354536:9852], now have 1 active actors on pipe
2025-10-09T21:48:06.478268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342098059354349:2432], now have 1 active actors on pipe
2025-10-09T21:48:06.478277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342097697532832:8793]
2025-10-09T21:48:06.478282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342097697532832:8793]
2025-10-09T21:48:06.478286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342097697532832:8793]
2025-10-09T21:48:06.478302Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:06.478322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:06.478327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:06.478331Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:06.478333Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:06.478340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:06.478344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:06.478327Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1120
2025-10-09T21:48:06.478331Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1120
2025-10-09T21:48:06.478381Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342097697532832:8793]
2025-10-09T21:48:06.478395Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:06.478406Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:06.478414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:06.478419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:06.478472Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|44324337-7df55975-f943d5e1-7fd0529b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:06.478485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:06.478493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:06.478497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:06.478507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:06.478522Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:06.478550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:06.478761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:06.478782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:06.478804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:06.478894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1121 partition: 0 MaxSeqNo: 25 sessionId: src_id|44324337-7df55975-f943d5e1-7fd0529b_0
2025-10-09T21:48:06.479256Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15485a06-7be364b9-e6b7a97f-79abc05e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|44324337-7df55975-f943d5e1-7fd0529b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:06.481790Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|44324337-7df55975-f943d5e1-7fd0529b_0] Write session: try to update token
2025-10-09T21:48:06.481809Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|44324337-7df55975-f943d5e1-7fd0529b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:06.493426Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:06.493445Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:06.493448Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:06.493450Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:06.493459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342097697532832:8793]
2025-10-09T21:48:06.493464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342097697532832:8793]
2025-10-09T21:48:06.493468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342097697532832:8793]
2025-10-09T21:48:06.493512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:06.493511Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:06.493517Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:06.493519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.493520Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:06.493522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:06.493654Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:06.495077Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1121 sessionId: src_id|44324337-7df55975-f943d5e1-7fd0529b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:06.497410Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:06.550070Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342098059354564:9750]
2025-10-09T21:48:06.550625Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 1120, pipe: [1:7559342098059354564:9750]
2025-10-09T21:48:06.650279Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:06.650679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:06.650692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:06.650733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342098059354564:9750], now have 1 active actors on pipe
2025-10-09T21:48:06.650977Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:06.650980Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:06.650993Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:06.651008Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342098059354564:9750]
2025-10-09T21:48:06.651032Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:06.651049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1121 reason: error at write: Disconnected sessionId: src_id|44324337-7df55975-f943d5e1-7fd0529b_0
2025-10-09T21:48:06.651238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1121 sessionId: src_id|44324337-7df55975-f943d5e1-7fd0529b_0 is DEAD
2025-10-09T21:48:06.651489Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44324337-7df55975-f943d5e1-7fd0529b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:06.651501Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44324337-7df55975-f943d5e1-7fd0529b_0] Write session will restart in 0.009851s
2025-10-09T21:48:06.651540Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44324337-7df55975-f943d5e1-7fd0529b_0] Write session: Do CDS request
2025-10-09T21:48:06.651548Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44324337-7df55975-f943d5e1-7fd0529b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:06.662511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|44324337-7df55975-f943d5e1-7fd0529b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:06.662607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:06.662623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1122
2025-10-09T21:48:06.662773Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1122 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:06.662812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1122 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:06.662824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1122 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:06.662828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:06.663361Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1122 sessionId:  describe result for acl check
2025-10-09T21:48:06.663404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:06.663411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:06.663413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:06.663420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342098059354714:9863] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:06.663994Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342098059354714:9863] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:06.665447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:06.666425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342098059354714:9863] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:06.668143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:06.668373Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:06.668383Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:06.668395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:06.668469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:06.668530Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342097697533044:8799]
2025-10-09T21:48:06.669006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342098059354714:9863] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:06.669020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342098059354714:9863] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:06.669024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342098059354714:9863] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:06.669032Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1122 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:06.669099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:06.669373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:06.669414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:06.669724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:06.669783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.669789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.669790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.669791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.669792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.669793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.669794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.669796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.669797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.669802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.669812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.669817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.669821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.669824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.669828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.669832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.669836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.669839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.669845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.669852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.669858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.669863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.669867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.669872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.669875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.669885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.669889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.669894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.669904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.669909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.669918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.669923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.669931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.669936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.669940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.669952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.669981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:06.676163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:06.677410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:06.677427Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:06.677429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:06.677434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:06.677441Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1121 [2:7559342097697533044:8799]
2025-10-09T21:48:06.677454Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:06.677472Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:06.677476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:06.677479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:06.677487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:06.677506Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:06.677517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:06.677531Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:06.677586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.677589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.677591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.677594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:06.677667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:06.677675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.677688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.677705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.677740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:06.677777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342098059354577:2432], now have 1 active actors on pipe
2025-10-09T21:48:06.677787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342098059354769:9863], now have 1 active actors on pipe
2025-10-09T21:48:06.677795Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342097697533039:8799]
2025-10-09T21:48:06.677797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342097697533039:8799]
2025-10-09T21:48:06.677799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342097697533039:8799]
2025-10-09T21:48:06.677813Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:06.677829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:06.677829Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:06.677832Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:06.677834Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:06.677845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:06.677848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:06.677813Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1121
2025-10-09T21:48:06.677858Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1121
2025-10-09T21:48:06.677888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342097697533039:8799]
2025-10-09T21:48:06.677896Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:06.677898Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:06.677924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:06.677933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:06.677969Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|562d0770-e42da569-a9e1d2ca-3b79777c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:06.677983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:06.677988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:06.677990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:06.678000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:06.678009Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:06.678048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:06.678202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:06.678210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:06.678225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:06.678313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1122 partition: 0 MaxSeqNo: 25 sessionId: src_id|562d0770-e42da569-a9e1d2ca-3b79777c_0
2025-10-09T21:48:06.678679Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44324337-7df55975-f943d5e1-7fd0529b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|562d0770-e42da569-a9e1d2ca-3b79777c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:06.681040Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|562d0770-e42da569-a9e1d2ca-3b79777c_0] Write session: try to update token
2025-10-09T21:48:06.681058Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|562d0770-e42da569-a9e1d2ca-3b79777c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:06.691601Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:06.691619Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:06.691621Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:06.691623Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:06.691633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342097697533039:8799]
2025-10-09T21:48:06.691637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342097697533039:8799]
2025-10-09T21:48:06.691641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342097697533039:8799]
2025-10-09T21:48:06.691684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:06.691686Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:06.691691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.691692Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:06.691694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:06.691696Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:06.691857Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:06.693986Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1122 sessionId: src_id|562d0770-e42da569-a9e1d2ca-3b79777c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:06.696406Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:06.751299Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342098059354784:9750]
2025-10-09T21:48:06.751675Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 1121, pipe: [1:7559342098059354784:9750]
2025-10-09T21:48:06.849405Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:06.849914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:06.849925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:06.849977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342098059354784:9750], now have 1 active actors on pipe
2025-10-09T21:48:06.850240Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:06.850241Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:06.850261Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342098059354784:9750]
2025-10-09T21:48:06.850290Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:06.850384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1122 reason: status is not ok: tablet will be restarted right now sessionId: src_id|562d0770-e42da569-a9e1d2ca-3b79777c_0
2025-10-09T21:48:06.850613Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1122 sessionId: src_id|562d0770-e42da569-a9e1d2ca-3b79777c_0 is DEAD
2025-10-09T21:48:06.850768Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|562d0770-e42da569-a9e1d2ca-3b79777c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:06.850780Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|562d0770-e42da569-a9e1d2ca-3b79777c_0] Write session will restart in 0.008149s
2025-10-09T21:48:06.850810Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|562d0770-e42da569-a9e1d2ca-3b79777c_0] Write session: Do CDS request
2025-10-09T21:48:06.850817Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|562d0770-e42da569-a9e1d2ca-3b79777c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:06.860610Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|562d0770-e42da569-a9e1d2ca-3b79777c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:06.860708Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:06.860728Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1123
2025-10-09T21:48:06.860896Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1123 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:06.860941Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1123 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:06.860952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1123 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:06.860956Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:06.861528Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1123 sessionId:  describe result for acl check
2025-10-09T21:48:06.861566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:06.861574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:06.861576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:06.861583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342098059354840:9870] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:06.862192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342098059354840:9870] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:06.864970Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342098059354840:9870] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:06.868193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342098059354840:9870] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:06.868208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342098059354840:9870] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:06.868212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342098059354840:9870] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:06.868309Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1123 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:06.868402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:06.870789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:06.871000Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:06.871014Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:06.871028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:06.871092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:06.871138Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342097697533242:8801]
2025-10-09T21:48:06.871713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:06.872085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:06.872139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:06.872497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:06.872550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.872552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.872553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.872554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.872555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.872556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.872567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.872569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.872570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.872576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.872588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.872593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.872597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.872601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.872605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.872609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.872613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.872617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.872625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.872633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:06.872640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.872645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:06.872650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.872655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:06.872660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.872665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:06.872669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.872674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:06.872679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.872684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:06.872689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.872695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.872699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.872703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.872709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.872715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.872746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:06.879496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:06.880863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:06.880879Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:06.880881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:06.880886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:06.880893Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1122 [2:7559342097697533242:8801]
2025-10-09T21:48:06.880905Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:06.880924Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:06.880929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:06.880932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:06.880935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:06.880953Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:06.880966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:06.880974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:06.881041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.881049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.881051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.881054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:06.881139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:06.881147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.881159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:06.881171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:06.881194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:06.881223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342098059354788:2432], now have 1 active actors on pipe
2025-10-09T21:48:06.881232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342098059354981:9870], now have 1 active actors on pipe
2025-10-09T21:48:06.881292Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1122
2025-10-09T21:48:06.881240Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342097697533236:8801]
2025-10-09T21:48:06.881299Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1122
2025-10-09T21:48:06.881248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342097697533236:8801]
2025-10-09T21:48:06.881250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342097697533236:8801]
2025-10-09T21:48:06.881261Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:06.881278Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:06.881288Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:06.881290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:06.881291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:06.881293Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:06.881293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:06.881334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342097697533236:8801]
2025-10-09T21:48:06.881337Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:06.881339Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:06.881358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:06.881363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:06.881417Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1d4058be-4444d529-7d178e27-8c9049e2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:06.881430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:06.881434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:06.881437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:06.881449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:06.881460Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:06.881485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:06.881688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:06.881698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:06.881713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:06.881827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1123 partition: 0 MaxSeqNo: 25 sessionId: src_id|1d4058be-4444d529-7d178e27-8c9049e2_0
2025-10-09T21:48:06.882236Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|562d0770-e42da569-a9e1d2ca-3b79777c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1d4058be-4444d529-7d178e27-8c9049e2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:06.884761Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1d4058be-4444d529-7d178e27-8c9049e2_0] Write session: try to update token
2025-10-09T21:48:06.884783Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1d4058be-4444d529-7d178e27-8c9049e2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:06.900176Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1123 sessionId: src_id|1d4058be-4444d529-7d178e27-8c9049e2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:06.901800Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:06.901819Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:06.901822Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:06.901825Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:06.901834Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342097697533236:8801]
2025-10-09T21:48:06.901839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342097697533236:8801]
2025-10-09T21:48:06.901843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342097697533236:8801]
2025-10-09T21:48:06.901899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:06.901902Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:06.901907Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:06.901912Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:06.901913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:06.901916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:06.902110Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:06.902792Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:06.950400Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342098059354994:9750]
2025-10-09T21:48:06.950863Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 1122, pipe: [1:7559342098059354994:9750]
2025-10-09T21:48:07.061586Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:07.062148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:07.062164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:07.062213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342098059354994:9750], now have 1 active actors on pipe
2025-10-09T21:48:07.062478Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:07.062499Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:07.062537Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342098059354994:9750]
2025-10-09T21:48:07.062425Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:07.062601Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1123 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1d4058be-4444d529-7d178e27-8c9049e2_0
2025-10-09T21:48:07.062801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1123 sessionId: src_id|1d4058be-4444d529-7d178e27-8c9049e2_0 is DEAD
2025-10-09T21:48:07.062956Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d4058be-4444d529-7d178e27-8c9049e2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:07.062977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d4058be-4444d529-7d178e27-8c9049e2_0] Write session will restart in 0.005641s
2025-10-09T21:48:07.063004Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d4058be-4444d529-7d178e27-8c9049e2_0] Write session: Do CDS request
2025-10-09T21:48:07.063011Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d4058be-4444d529-7d178e27-8c9049e2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:07.069601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1d4058be-4444d529-7d178e27-8c9049e2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:07.069710Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:07.069727Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1124
2025-10-09T21:48:07.069871Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1124 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:07.069906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1124 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:07.069919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1124 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:07.069922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:07.070370Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1124 sessionId:  describe result for acl check
2025-10-09T21:48:07.070408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:07.070417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:07.070419Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:07.070425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342102354322310:9874] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:07.071004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342102354322310:9874] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:07.075558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342102354322310:9874] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:07.079065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342102354322310:9874] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:07.079084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342102354322310:9874] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:07.079088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342102354322310:9874] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:07.079099Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1124 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:07.080337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:07.082822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:07.083049Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:07.083063Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:07.083075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:07.083137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:07.083180Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342101992500736:8803]
2025-10-09T21:48:07.083774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:07.084130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:07.084191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:07.084539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:07.084604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.084611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.084612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.084613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.084614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.084616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.084617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.084618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.084619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.084625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.084635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.084639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.084643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.084646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.084650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.084654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.084657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.084661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.084669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.084676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.084683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.084689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.084693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.084699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.084711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.084716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.084725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.084731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.084736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.084742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.084747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.084752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.084763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.084769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.084774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.084784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.084817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:07.091706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:07.093029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:07.093047Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:07.093049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:07.093054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:07.093064Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1123 [2:7559342101992500736:8803]
2025-10-09T21:48:07.093081Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:07.093102Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:07.093111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:07.093114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:07.093117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:07.093136Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:07.093142Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:07.093149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:07.093217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.093226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.093227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.093230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:07.093310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:07.093316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.093330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.093343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.093361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:07.093383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342101992500731:8803]
2025-10-09T21:48:07.093391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342101992500731:8803]
2025-10-09T21:48:07.093393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342101992500731:8803]
2025-10-09T21:48:07.093408Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:07.093422Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:07.093428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342102354322488:9874], now have 1 active actors on pipe
2025-10-09T21:48:07.093432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342102354322295:2432], now have 1 active actors on pipe
2025-10-09T21:48:07.093432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:07.093434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:07.093441Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:07.093456Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:07.093459Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:07.093489Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1123
2025-10-09T21:48:07.093500Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1123
2025-10-09T21:48:07.093499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342101992500731:8803]
2025-10-09T21:48:07.093513Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:07.093529Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:07.093562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:07.093567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:07.093600Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|87308570-3efd0302-9db51b83-2c4ef327_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:07.093618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:07.093622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:07.093625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:07.093637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:07.093649Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:07.093673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:07.093871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:07.093881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:07.093899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:07.094022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1124 partition: 0 MaxSeqNo: 25 sessionId: src_id|87308570-3efd0302-9db51b83-2c4ef327_0
2025-10-09T21:48:07.094459Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d4058be-4444d529-7d178e27-8c9049e2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|87308570-3efd0302-9db51b83-2c4ef327_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:07.097076Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87308570-3efd0302-9db51b83-2c4ef327_0] Write session: try to update token
2025-10-09T21:48:07.097095Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87308570-3efd0302-9db51b83-2c4ef327_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:07.108505Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:07.108524Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:07.108527Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:07.108530Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:07.108540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342101992500731:8803]
2025-10-09T21:48:07.108548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342101992500731:8803]
2025-10-09T21:48:07.108553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342101992500731:8803]
2025-10-09T21:48:07.108600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:07.108605Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:07.108611Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:07.108612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.108615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:07.108615Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:07.108746Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:07.112489Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1124 sessionId: src_id|87308570-3efd0302-9db51b83-2c4ef327_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:07.115125Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:07.162866Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342102354322506:9750]
2025-10-09T21:48:07.163177Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 1123, pipe: [1:7559342102354322506:9750]
2025-10-09T21:48:07.269091Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:07.269473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:07.269492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:07.269540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342102354322506:9750], now have 1 active actors on pipe
2025-10-09T21:48:07.269826Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:07.269835Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342102354322506:9750]
2025-10-09T21:48:07.269840Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:07.269859Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:07.269903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1124 reason: error at write: Disconnected sessionId: src_id|87308570-3efd0302-9db51b83-2c4ef327_0
2025-10-09T21:48:07.270134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1124 sessionId: src_id|87308570-3efd0302-9db51b83-2c4ef327_0 is DEAD
2025-10-09T21:48:07.270284Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87308570-3efd0302-9db51b83-2c4ef327_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:07.270307Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87308570-3efd0302-9db51b83-2c4ef327_0] Write session will restart in 0.008122s
2025-10-09T21:48:07.270286Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:07.270333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87308570-3efd0302-9db51b83-2c4ef327_0] Write session: Do CDS request
2025-10-09T21:48:07.270339Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87308570-3efd0302-9db51b83-2c4ef327_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:07.279588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87308570-3efd0302-9db51b83-2c4ef327_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:07.279677Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:07.279694Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1125
2025-10-09T21:48:07.279810Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1125 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:07.279848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1125 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:07.279859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1125 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:07.279863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:07.280323Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1125 sessionId:  describe result for acl check
2025-10-09T21:48:07.280361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:07.280367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:07.280369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:07.280375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342102354322528:9879] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:07.280961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342102354322528:9879] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:07.284810Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342102354322528:9879] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:07.288999Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342102354322528:9879] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:07.289016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342102354322528:9879] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:07.289020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342102354322528:9879] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:07.289030Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1125 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:07.291666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:07.293531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:07.293756Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:07.293777Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:07.293790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:07.293848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:07.293895Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342101992500936:8805]
2025-10-09T21:48:07.294485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:07.294810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:07.294860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:07.295178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:07.295235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.295241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.295242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.295243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.295244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.295246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.295247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.295248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.295249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.295255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.295266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.295270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.295275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.295279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.295283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.295287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.295291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.295294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.295302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.295310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.295316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.295326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.295331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.295340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.295346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.295355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.295360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.295367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.295376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.295381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.295386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.295391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.295400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.295405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.295409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.295414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.295443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:07.301990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:07.303324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:07.303343Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:07.303345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:07.303350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:07.303358Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1124 [2:7559342101992500936:8805]
2025-10-09T21:48:07.303375Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:07.303394Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:07.303397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:07.303401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:07.303404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:07.303422Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:07.303431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:07.303447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:07.303518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.303529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.303531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.303533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:07.303777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:07.303789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.303803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.303815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.303833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:07.303858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342102354322706:9879], now have 1 active actors on pipe
2025-10-09T21:48:07.303868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342102354322513:2432], now have 1 active actors on pipe
2025-10-09T21:48:07.303784Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1124
2025-10-09T21:48:07.303876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342101992500931:8805]
2025-10-09T21:48:07.303793Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1124
2025-10-09T21:48:07.303878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342101992500931:8805]
2025-10-09T21:48:07.303881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342101992500931:8805]
2025-10-09T21:48:07.303897Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:07.303911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:07.303929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:07.303932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:07.303970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342101992500931:8805]
2025-10-09T21:48:07.303973Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:07.303977Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:07.303979Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:07.303980Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:07.303987Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:07.304013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:07.304024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:07.304058Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|db15c29e-e3fd2760-85e28202-4904ef9a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:07.304071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:07.304077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:07.304080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:07.304091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:07.304106Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:07.304122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:07.304384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:07.304392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:07.304409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:07.304517Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1125 partition: 0 MaxSeqNo: 25 sessionId: src_id|db15c29e-e3fd2760-85e28202-4904ef9a_0
2025-10-09T21:48:07.304900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87308570-3efd0302-9db51b83-2c4ef327_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|db15c29e-e3fd2760-85e28202-4904ef9a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:07.307406Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db15c29e-e3fd2760-85e28202-4904ef9a_0] Write session: try to update token
2025-10-09T21:48:07.307429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db15c29e-e3fd2760-85e28202-4904ef9a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:07.316288Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:07.316310Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:07.316320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342101992500931:8805]
2025-10-09T21:48:07.316366Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:07.316370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:07.316559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:07.316729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:07.316879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:07.317032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:07.317240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:07.317291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:07.325333Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1125 sessionId: src_id|db15c29e-e3fd2760-85e28202-4904ef9a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:07.328133Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:07.328537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:07.328565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:07.328620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:07.328746Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:07.332137Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:07.332153Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:07.332156Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:07.332158Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:07.332180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342101992500931:8805]
2025-10-09T21:48:07.332185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342101992500931:8805]
2025-10-09T21:48:07.332188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342101992500931:8805]
2025-10-09T21:48:07.332235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:07.332247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.332254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:07.332254Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:07.332260Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:07.332264Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:07.332396Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:07.370269Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342102354322721:9750]
2025-10-09T21:48:07.370598Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 1124, pipe: [1:7559342102354322721:9750]
2025-10-09T21:48:07.490622Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:07.498418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:07.498444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:07.498616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:07.498682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:07.498747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:07.498815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:07.498955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:07.498967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:07.499228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:07.499484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:07.499541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:07.499603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:07.499738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:07.499741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:07.499802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:07.499909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:07.499963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:07.500172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:07.500212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:07.500215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:07.500718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:07.500780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:07.500835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:07.500896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:07.501394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:07.501399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:07.501468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:07.501522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:07.501577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:07.501824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:07.501948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:07.501951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:07.502306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:07.502372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:07.502438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:07.502678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:07.502723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:07.502730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:07.502981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:07.503049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:07.503108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:07.503165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:07.503205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:07.503215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:07.503281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:07.503343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:07.503403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:07.503463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:07.503508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:07.503515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:07.503579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:07.503830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:07.504090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:07.504339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:07.504467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:07.504475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:07.504730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:07.504786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:07.504845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:07.504905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:07.504949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:07.504959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:07.505027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342102354322721:9750], now have 1 active actors on pipe
2025-10-09T21:48:07.505230Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342102354322721:9750]
2025-10-09T21:48:07.505244Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:07.505270Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:07.505273Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:07.505301Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1125 reason: error at write: Disconnected sessionId: src_id|db15c29e-e3fd2760-85e28202-4904ef9a_0
2025-10-09T21:48:07.505502Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1125 sessionId: src_id|db15c29e-e3fd2760-85e28202-4904ef9a_0 is DEAD
2025-10-09T21:48:07.505628Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:07.505675Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db15c29e-e3fd2760-85e28202-4904ef9a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:07.505683Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db15c29e-e3fd2760-85e28202-4904ef9a_0] Write session will restart in 0.007526s
2025-10-09T21:48:07.505716Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db15c29e-e3fd2760-85e28202-4904ef9a_0] Write session: Do CDS request
2025-10-09T21:48:07.505725Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db15c29e-e3fd2760-85e28202-4904ef9a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:07.514541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db15c29e-e3fd2760-85e28202-4904ef9a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:07.514657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:07.514676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1126
2025-10-09T21:48:07.514823Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1126 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:07.514866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1126 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:07.514876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1126 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:07.514880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:07.515476Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1126 sessionId:  describe result for acl check
2025-10-09T21:48:07.515521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:07.515528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:07.515530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:07.515536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342102354322759:9883] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:07.516090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342102354322759:9883] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:07.518265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342102354322759:9883] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:07.521597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342102354322759:9883] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:07.521615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342102354322759:9883] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:07.521618Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342102354322759:9883] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:07.521627Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1126 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:07.522986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:07.525105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:07.525413Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:07.525424Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:07.525440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:07.525508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:07.525562Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342101992501166:8817]
2025-10-09T21:48:07.526192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:07.526587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:07.526652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:07.527026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:07.527091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.527096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.527098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.527099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.527100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.527101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.527103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.527104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.527105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.527111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.527122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.527127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.527131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.527135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.527139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.527142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.527146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.527150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.527158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.527166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.527173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.527178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.527183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.527189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.527193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.527198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.527203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.527208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.527219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.527225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.527230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.527235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.527239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.527244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.527249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.527254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.527283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:07.533755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:07.534839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:07.534857Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:07.534860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:07.534864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:07.534871Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1125 [2:7559342101992501166:8817]
2025-10-09T21:48:07.534885Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:07.534907Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:07.534915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:07.534919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:07.534922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:07.534939Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:07.534953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:07.534961Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:07.535026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.535033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.535035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.535038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:07.535285Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1125
2025-10-09T21:48:07.535288Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1125
2025-10-09T21:48:07.535346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:07.535358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.535371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.535384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.535403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:07.535438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342102354322920:9883], now have 1 active actors on pipe
2025-10-09T21:48:07.535442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342102354322727:2432], now have 1 active actors on pipe
2025-10-09T21:48:07.535467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:07.535478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:07.535498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342101992501161:8817]
2025-10-09T21:48:07.535506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342101992501161:8817]
2025-10-09T21:48:07.535508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342101992501161:8817]
2025-10-09T21:48:07.535523Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:07.535539Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:07.535542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:07.535549Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:07.535552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:07.535558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:07.535561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:07.535587Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ebb4aacb-eb152ef-700ff9c6-af090a3c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:07.535600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:07.535611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:07.535613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:07.535625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:07.535637Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:07.535676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342101992501161:8817]
2025-10-09T21:48:07.535691Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:07.535699Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:07.535705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:07.535856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:07.535866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:07.535881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:07.535977Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1126 partition: 0 MaxSeqNo: 25 sessionId: src_id|ebb4aacb-eb152ef-700ff9c6-af090a3c_0
2025-10-09T21:48:07.536385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db15c29e-e3fd2760-85e28202-4904ef9a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ebb4aacb-eb152ef-700ff9c6-af090a3c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:07.544496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb4aacb-eb152ef-700ff9c6-af090a3c_0] Write session: try to update token
2025-10-09T21:48:07.544523Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb4aacb-eb152ef-700ff9c6-af090a3c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:07.548117Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:07.548150Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:07.548162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342101992501161:8817]
2025-10-09T21:48:07.548208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:07.548208Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:07.548384Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:07.548524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:07.548654Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:07.548785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:07.548966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:07.549011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:07.560601Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1126 sessionId: src_id|ebb4aacb-eb152ef-700ff9c6-af090a3c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:07.561394Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:07.561415Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:07.561418Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:07.561420Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:07.561431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342101992501161:8817]
2025-10-09T21:48:07.561436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342101992501161:8817]
2025-10-09T21:48:07.561440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342101992501161:8817]
2025-10-09T21:48:07.561499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:07.561507Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:07.561512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.561513Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:07.561515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:07.561517Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:07.561670Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:07.563115Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:07.605244Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342102354322946:9750]
2025-10-09T21:48:07.605527Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 1125, pipe: [1:7559342102354322946:9750]
2025-10-09T21:48:07.719700Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:07.720086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:07.720106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:07.720131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342102354322946:9750], now have 1 active actors on pipe
2025-10-09T21:48:07.720477Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:07.720797Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:07.720814Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:07.720821Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:07.720877Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342102354322946:9750]
2025-10-09T21:48:07.720881Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1126 reason: error at write: Disconnected sessionId: src_id|ebb4aacb-eb152ef-700ff9c6-af090a3c_0
2025-10-09T21:48:07.721068Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1126 sessionId: src_id|ebb4aacb-eb152ef-700ff9c6-af090a3c_0 is DEAD
2025-10-09T21:48:07.721497Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb4aacb-eb152ef-700ff9c6-af090a3c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:07.721520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb4aacb-eb152ef-700ff9c6-af090a3c_0] Write session will restart in 0.009759s
2025-10-09T21:48:07.721551Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb4aacb-eb152ef-700ff9c6-af090a3c_0] Write session: Do CDS request
2025-10-09T21:48:07.721558Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb4aacb-eb152ef-700ff9c6-af090a3c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:07.732632Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb4aacb-eb152ef-700ff9c6-af090a3c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:07.732744Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:07.732760Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1127
2025-10-09T21:48:07.738087Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1127 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:07.738139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1127 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:07.738160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1127 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:07.738171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:07.738937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:07.739082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1127 sessionId:  describe result for acl check
2025-10-09T21:48:07.739123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:07.739130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:07.739132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:07.739139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342102354323124:9894] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:07.739676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342102354323124:9894] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:07.741528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:07.741736Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:07.741746Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:07.741768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:07.741828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:07.741873Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342101992501378:8823]
2025-10-09T21:48:07.742125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342102354323124:9894] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:07.742497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:07.742805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:07.742850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:07.743173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:07.743219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.743225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.743226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.743227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.743229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.743230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.743231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.743232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.743233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.743238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.743249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.743253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.743258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.743263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.743266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.743270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.743273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.743277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.743284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.743292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.743298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.743304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.743309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.743314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.743319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.743324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.743328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.743334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.743338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.743344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.743349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.743361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.743365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.743370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.743374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.743384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.743412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:07.749067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342102354323124:9894] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:07.749085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342102354323124:9894] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:07.749089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342102354323124:9894] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:07.749099Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1127 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:07.750239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:07.751333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:07.751350Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:07.751352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:07.751366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:07.751373Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1126 [2:7559342101992501378:8823]
2025-10-09T21:48:07.751390Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:07.751411Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:07.751420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:07.751423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:07.751426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:07.751445Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:07.751451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:07.751459Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:07.751518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.751526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.751528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.751531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:07.751602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:07.751610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.751625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.751642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.751664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:07.751692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342102354322960:2432], now have 1 active actors on pipe
2025-10-09T21:48:07.751696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342102354323163:9894], now have 1 active actors on pipe
2025-10-09T21:48:07.751704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342101992501373:8823]
2025-10-09T21:48:07.751709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342101992501373:8823]
2025-10-09T21:48:07.751712Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342101992501373:8823]
2025-10-09T21:48:07.751725Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:07.751746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:07.751756Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:07.751760Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:07.751762Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:07.751765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:07.751769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:07.751781Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1126
2025-10-09T21:48:07.751806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342101992501373:8823]
2025-10-09T21:48:07.751799Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1126
2025-10-09T21:48:07.751814Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:07.751816Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:07.751845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:07.751850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:07.751891Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b42b2e5b-dc0125eb-abc99fcd-f22afb31_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:07.751906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:07.751910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:07.751912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:07.751923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:07.751933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:07.751954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:07.752410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:07.752423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:07.752444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:07.752572Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1127 partition: 0 MaxSeqNo: 25 sessionId: src_id|b42b2e5b-dc0125eb-abc99fcd-f22afb31_0
2025-10-09T21:48:07.752935Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb4aacb-eb152ef-700ff9c6-af090a3c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b42b2e5b-dc0125eb-abc99fcd-f22afb31_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:07.755413Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b42b2e5b-dc0125eb-abc99fcd-f22afb31_0] Write session: try to update token
2025-10-09T21:48:07.755434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b42b2e5b-dc0125eb-abc99fcd-f22afb31_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:07.766651Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:07.766669Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:07.766672Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:07.766674Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:07.766685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342101992501373:8823]
2025-10-09T21:48:07.766689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342101992501373:8823]
2025-10-09T21:48:07.766696Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342101992501373:8823]
2025-10-09T21:48:07.766745Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:07.766749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:07.766751Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:07.766756Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:07.766756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.766759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:07.766929Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:07.769287Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1127 sessionId: src_id|b42b2e5b-dc0125eb-abc99fcd-f22afb31_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:07.771716Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:07.820819Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342102354323172:9750]
2025-10-09T21:48:07.821079Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 1126, pipe: [1:7559342102354323172:9750]
2025-10-09T21:48:07.926709Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:07.927208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:07.927230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:07.927289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342102354323172:9750], now have 1 active actors on pipe
2025-10-09T21:48:07.927499Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:07.927534Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:07.927538Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:07.927560Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342102354323172:9750]
2025-10-09T21:48:07.927531Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:07.927607Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1127 reason: error at write: Disconnected sessionId: src_id|b42b2e5b-dc0125eb-abc99fcd-f22afb31_0
2025-10-09T21:48:07.927810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1127 sessionId: src_id|b42b2e5b-dc0125eb-abc99fcd-f22afb31_0 is DEAD
2025-10-09T21:48:07.928013Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b42b2e5b-dc0125eb-abc99fcd-f22afb31_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:07.928025Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b42b2e5b-dc0125eb-abc99fcd-f22afb31_0] Write session will restart in 0.006267s
2025-10-09T21:48:07.928062Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b42b2e5b-dc0125eb-abc99fcd-f22afb31_0] Write session: Do CDS request
2025-10-09T21:48:07.928070Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b42b2e5b-dc0125eb-abc99fcd-f22afb31_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:07.935552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b42b2e5b-dc0125eb-abc99fcd-f22afb31_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:07.935609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:07.935626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1128
2025-10-09T21:48:07.935783Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1128 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:07.935819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1128 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:07.935830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1128 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:07.935834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:07.936327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1128 sessionId:  describe result for acl check
2025-10-09T21:48:07.936364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:07.936380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:07.936381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:07.936388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342102354323190:9899] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:07.937109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342102354323190:9899] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:07.940314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342102354323190:9899] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:07.943840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342102354323190:9899] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:07.943856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342102354323190:9899] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:07.943860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342102354323190:9899] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:07.943869Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1128 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:07.945866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:07.948011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:07.948215Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:07.948223Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:07.948236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:07.948293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:07.948392Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342101992501582:8825]
2025-10-09T21:48:07.948932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:07.949252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:07.949324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:07.949670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:07.949724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.949730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.949731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.949732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.949733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.949735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.949736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.949737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.949738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.949744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.949754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.949770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.949774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.949778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.949781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.949785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.949788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.949792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.949800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.949808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:07.949816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.949821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:07.949826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.949831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:07.949835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.949841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:07.949845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.949850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:07.949854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.949860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:07.949865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.949871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.949876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.949881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.949886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.949897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.949929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:07.957342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:07.958755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:07.958777Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:07.958780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:07.958786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:07.958792Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1127 [2:7559342101992501582:8825]
2025-10-09T21:48:07.958807Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:07.958827Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:07.958838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:07.958842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:07.958844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:07.958862Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:07.958868Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:07.958876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:07.958939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.958948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.958950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.958953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:07.959033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:07.959043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.959057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:07.959075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:07.959094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:07.959115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342102354323175:2432], now have 1 active actors on pipe
2025-10-09T21:48:07.959118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342102354323372:9899], now have 1 active actors on pipe
2025-10-09T21:48:07.959126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342101992501577:8825]
2025-10-09T21:48:07.959128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342101992501577:8825]
2025-10-09T21:48:07.959129Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342101992501577:8825]
2025-10-09T21:48:07.959143Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:07.959162Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:07.959171Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:07.959175Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:07.959177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:07.959181Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:07.959185Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:07.959205Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1127
2025-10-09T21:48:07.959209Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1127
2025-10-09T21:48:07.959225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342101992501577:8825]
2025-10-09T21:48:07.959236Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:07.959242Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:07.959266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:07.959272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:07.959311Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e990a0af-9461ed64-4e208692-3423bf54_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:07.959327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:07.959331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:07.959334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:07.959344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:07.959355Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:07.959375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:07.959557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:07.959567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:07.959583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:07.959685Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1128 partition: 0 MaxSeqNo: 25 sessionId: src_id|e990a0af-9461ed64-4e208692-3423bf54_0
2025-10-09T21:48:07.960081Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b42b2e5b-dc0125eb-abc99fcd-f22afb31_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e990a0af-9461ed64-4e208692-3423bf54_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:07.962630Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e990a0af-9461ed64-4e208692-3423bf54_0] Write session: try to update token
2025-10-09T21:48:07.962650Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e990a0af-9461ed64-4e208692-3423bf54_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:07.974017Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:07.974052Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:07.974054Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:07.974056Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:07.974066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342101992501577:8825]
2025-10-09T21:48:07.974072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342101992501577:8825]
2025-10-09T21:48:07.974075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342101992501577:8825]
2025-10-09T21:48:07.974120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:07.974127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:07.974127Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:07.974130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:07.974133Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:07.974137Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:07.974258Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:07.976512Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1128 sessionId: src_id|e990a0af-9461ed64-4e208692-3423bf54_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:07.979307Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:08.027658Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342106649290683:9750]
2025-10-09T21:48:08.028364Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 1127, pipe: [1:7559342106649290683:9750]
2025-10-09T21:48:08.137251Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:08.137722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:08.137744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:08.137812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342106649290683:9750], now have 1 active actors on pipe
2025-10-09T21:48:08.138119Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:08.138120Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:08.138144Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342106649290683:9750]
2025-10-09T21:48:08.138189Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:08.138192Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:08.138258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1128 reason: error at write: Disconnected sessionId: src_id|e990a0af-9461ed64-4e208692-3423bf54_0
2025-10-09T21:48:08.138472Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1128 sessionId: src_id|e990a0af-9461ed64-4e208692-3423bf54_0 is DEAD
2025-10-09T21:48:08.138646Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e990a0af-9461ed64-4e208692-3423bf54_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:08.138656Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e990a0af-9461ed64-4e208692-3423bf54_0] Write session will restart in 0.009942s
2025-10-09T21:48:08.138684Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e990a0af-9461ed64-4e208692-3423bf54_0] Write session: Do CDS request
2025-10-09T21:48:08.138690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e990a0af-9461ed64-4e208692-3423bf54_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:08.149640Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e990a0af-9461ed64-4e208692-3423bf54_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:08.149751Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:08.149781Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1129
2025-10-09T21:48:08.149925Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1129 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:08.149976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1129 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:08.149990Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1129 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:08.149995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:08.150531Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1129 sessionId:  describe result for acl check
2025-10-09T21:48:08.150574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:08.150577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:08.150579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:08.150586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342106649290781:9903] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:08.151207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342106649290781:9903] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:08.155910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:08.156732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342106649290781:9903] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:08.159000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:08.159246Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:08.159258Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:08.159272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:08.159332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:08.159381Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342106287469080:8827]
2025-10-09T21:48:08.159548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342106649290781:9903] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:08.159567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342106649290781:9903] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:08.159571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342106649290781:9903] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:08.159581Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1129 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:08.159960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:08.160380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:08.160443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:08.160819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:08.160876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:08.160882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:08.160883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:08.160885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:08.160886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:08.160887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:08.160888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.160890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.160892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.160898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:08.160910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:08.160915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:08.160919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:08.160923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:08.160927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:08.160932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.160936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.160941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.160951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:08.160960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:08.160967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:08.160972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:08.160977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:08.160982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:08.160987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:08.160992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:08.160996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:08.161001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:08.161007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:08.161013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:08.161018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.161029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.161034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.161039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.161044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.161050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.161083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:08.168105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:08.169220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:08.169237Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:08.169241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:08.169245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:08.169256Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1128 [2:7559342106287469080:8827]
2025-10-09T21:48:08.169272Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:08.169292Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:08.169300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:08.169304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:08.169306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:08.169327Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:08.169337Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:08.169345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:08.169413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.169423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.169425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.169428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:08.169510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:08.169520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.169532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.169541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.169558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:08.169583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342106649290690:2432], now have 1 active actors on pipe
2025-10-09T21:48:08.169592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342106649290891:9903], now have 1 active actors on pipe
2025-10-09T21:48:08.169601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342106287469075:8827]
2025-10-09T21:48:08.169607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342106287469075:8827]
2025-10-09T21:48:08.169609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342106287469075:8827]
2025-10-09T21:48:08.169620Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:08.169636Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:08.169645Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:08.169649Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:08.169651Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:08.169636Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1128
2025-10-09T21:48:08.169656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:08.169645Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1128
2025-10-09T21:48:08.169659Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:08.169705Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342106287469075:8827]
2025-10-09T21:48:08.169714Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:08.169721Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:08.169739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:08.169744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:08.169811Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|28cc1618-99e39a0e-88674609-92dc867_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:08.169824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:08.169828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:08.169830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:08.169842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:08.169853Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:08.169887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:08.170129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:08.170141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:08.170157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:08.170282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1129 partition: 0 MaxSeqNo: 25 sessionId: src_id|28cc1618-99e39a0e-88674609-92dc867_0
2025-10-09T21:48:08.170669Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e990a0af-9461ed64-4e208692-3423bf54_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|28cc1618-99e39a0e-88674609-92dc867_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:08.173381Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28cc1618-99e39a0e-88674609-92dc867_0] Write session: try to update token
2025-10-09T21:48:08.173401Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28cc1618-99e39a0e-88674609-92dc867_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:08.186300Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:08.186323Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:08.186325Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:08.186327Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:08.186339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342106287469075:8827]
2025-10-09T21:48:08.186343Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342106287469075:8827]
2025-10-09T21:48:08.186348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342106287469075:8827]
2025-10-09T21:48:08.186400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:08.186412Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:08.186415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.186418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:08.186420Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:08.186424Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:08.186575Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:08.187839Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1129 sessionId: src_id|28cc1618-99e39a0e-88674609-92dc867_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:08.190524Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:08.238089Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342106649290908:9750]
2025-10-09T21:48:08.238448Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 1128, pipe: [1:7559342106649290908:9750]
2025-10-09T21:48:08.351637Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:08.352044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:08.352065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:08.352117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342106649290908:9750], now have 1 active actors on pipe
2025-10-09T21:48:08.352393Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:08.352404Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:08.352415Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:08.352451Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342106649290908:9750]
2025-10-09T21:48:08.352479Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1129 reason: error at write: Disconnected sessionId: src_id|28cc1618-99e39a0e-88674609-92dc867_0
2025-10-09T21:48:08.352564Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:08.352687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1129 sessionId: src_id|28cc1618-99e39a0e-88674609-92dc867_0 is DEAD
2025-10-09T21:48:08.352854Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28cc1618-99e39a0e-88674609-92dc867_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:08.352865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28cc1618-99e39a0e-88674609-92dc867_0] Write session will restart in 0.009332s
2025-10-09T21:48:08.352894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28cc1618-99e39a0e-88674609-92dc867_0] Write session: Do CDS request
2025-10-09T21:48:08.352900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28cc1618-99e39a0e-88674609-92dc867_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:08.363579Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28cc1618-99e39a0e-88674609-92dc867_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:08.363670Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:08.363689Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1130
2025-10-09T21:48:08.363822Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1130 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:08.363875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1130 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:08.363882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1130 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:08.363886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:08.364384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1130 sessionId:  describe result for acl check
2025-10-09T21:48:08.364428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:08.364435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:08.364437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:08.364444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342106649290998:9908] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:08.365016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342106649290998:9908] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:08.367796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342106649290998:9908] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:08.369849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:08.372506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:08.372673Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342106649290998:9908] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:08.372691Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342106649290998:9908] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:08.372694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342106649290998:9908] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:08.372703Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1130 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:08.372812Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:08.372826Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:08.372841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:08.372911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:08.372970Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342106287469284:8829]
2025-10-09T21:48:08.373553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:08.373885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:08.373933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:08.374282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:08.374342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:08.374349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:08.374350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:08.374351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:08.374352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:08.374353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:08.374354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.374355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.374356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.374361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:08.374372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:08.374377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:08.374381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:08.374384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:08.374387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:08.374390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.374393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.374396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.374404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:08.374412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:08.374419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:08.374423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:08.374428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:08.374434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:08.374439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:08.374445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:08.374451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:08.374457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:08.374462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:08.374467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:08.374481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.374487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.374497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.374503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.374513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.374520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.374551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:08.381413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:08.382493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:08.382509Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:08.382512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:08.382516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:08.382524Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1129 [2:7559342106287469284:8829]
2025-10-09T21:48:08.382538Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:08.382557Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:08.382565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:08.382568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:08.382571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:08.382589Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:08.382599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:08.382608Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:08.382669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.382679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.382681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.382685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:08.382763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:08.382773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.382786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.382797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.382813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:08.382836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342106649291113:9908], now have 1 active actors on pipe
2025-10-09T21:48:08.382840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342106649290911:2432], now have 1 active actors on pipe
2025-10-09T21:48:08.382848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342106287469279:8829]
2025-10-09T21:48:08.382857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342106287469279:8829]
2025-10-09T21:48:08.382860Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342106287469279:8829]
2025-10-09T21:48:08.382873Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:08.382888Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:08.382908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:08.382917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:08.382946Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:08.382951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342106287469279:8829]
2025-10-09T21:48:08.382954Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:08.382958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:08.382960Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:08.382940Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1129
2025-10-09T21:48:08.382969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:08.382948Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1129
2025-10-09T21:48:08.382997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:08.383002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:08.383033Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cb1ee0c-b1edf565-564102db-e44596f4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:08.383047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:08.383053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:08.383056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:08.383068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:08.383079Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:08.383098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:08.383285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:08.383295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:08.383311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:08.383408Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1130 partition: 0 MaxSeqNo: 25 sessionId: src_id|cb1ee0c-b1edf565-564102db-e44596f4_0
2025-10-09T21:48:08.383796Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28cc1618-99e39a0e-88674609-92dc867_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cb1ee0c-b1edf565-564102db-e44596f4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:08.386303Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cb1ee0c-b1edf565-564102db-e44596f4_0] Write session: try to update token
2025-10-09T21:48:08.386322Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cb1ee0c-b1edf565-564102db-e44596f4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:08.394985Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:08.395007Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:08.395017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342106287469279:8829]
2025-10-09T21:48:08.395070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:08.395180Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:08.395249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:08.395390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:08.395523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:08.395660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:08.395922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:08.395968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:08.399574Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1130 sessionId: src_id|cb1ee0c-b1edf565-564102db-e44596f4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:08.402073Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:08.402501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:08.402527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:08.402571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:08.402661Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:08.408497Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:08.408520Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:08.408523Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:08.408525Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:08.408536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342106287469279:8829]
2025-10-09T21:48:08.408540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342106287469279:8829]
2025-10-09T21:48:08.408544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342106287469279:8829]
2025-10-09T21:48:08.408589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:08.408589Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:08.408595Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:08.408599Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:08.408601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.408603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:08.408743Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:08.452607Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342106649291124:9750]
2025-10-09T21:48:08.452879Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 1129, pipe: [1:7559342106649291124:9750]
2025-10-09T21:48:08.566278Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:08.574159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:08.574183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:08.574330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.574398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.574467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.574538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.574817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.574833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.574902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.574967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.575030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.575098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.575138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.575146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.575207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.575270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.575333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.575397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.575443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.575450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.575523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.575629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.575688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.575774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.575818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.575826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.575896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.575988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.576043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.576098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.576146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.576162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.576223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.576276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.576335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.576405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.576454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.576464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.576535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.576636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.576727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.576785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.576826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.576834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.576896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.576947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.577002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.577065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.577136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.577144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.577206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.577265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.577327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.577393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.577457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.577468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.577536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.577598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.577660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.577722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.577777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.577784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.577851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342106649291124:9750], now have 1 active actors on pipe
2025-10-09T21:48:08.578111Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:08.578125Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:08.578128Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:08.578157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1130 reason: error at write: Disconnected sessionId: src_id|cb1ee0c-b1edf565-564102db-e44596f4_0
2025-10-09T21:48:08.578177Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342106649291124:9750]
2025-10-09T21:48:08.578386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1130 sessionId: src_id|cb1ee0c-b1edf565-564102db-e44596f4_0 is DEAD
2025-10-09T21:48:08.578536Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb1ee0c-b1edf565-564102db-e44596f4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:08.578548Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb1ee0c-b1edf565-564102db-e44596f4_0] Write session will restart in 0.006481s
2025-10-09T21:48:08.578573Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb1ee0c-b1edf565-564102db-e44596f4_0] Write session: Do CDS request
2025-10-09T21:48:08.578593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb1ee0c-b1edf565-564102db-e44596f4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:08.578865Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:08.586503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cb1ee0c-b1edf565-564102db-e44596f4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:08.586610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:08.586625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1131
2025-10-09T21:48:08.586748Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1131 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:08.586793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1131 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:08.586805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1131 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:08.586809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:08.587364Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1131 sessionId:  describe result for acl check
2025-10-09T21:48:08.587405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:08.587413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:08.587415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:08.587421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342106649291156:9912] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:08.587982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342106649291156:9912] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:08.591113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342106649291156:9912] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:08.594235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342106649291156:9912] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:08.594258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342106649291156:9912] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:08.594262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342106649291156:9912] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:08.594270Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1131 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:08.594891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:08.597150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:08.597347Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:08.597358Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:08.597370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:08.597424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:08.597465Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342106287469521:8841]
2025-10-09T21:48:08.598096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:08.598426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:08.598489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:08.598846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:08.598904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:08.598911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:08.598912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:08.598913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:08.598915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:08.598916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:08.598917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.598918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.598919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.598925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:08.598936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:08.598939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:08.598944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:08.598948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:08.598952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:08.598955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.598959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.598962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.598970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:08.598991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:08.598997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:08.599003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:08.599008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:08.599012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:08.599017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:08.599022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:08.599029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:08.599034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:08.599039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:08.599043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:08.599048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.599053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.599058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.599063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.599067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.599072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.599101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:08.605944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:08.607198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:08.607216Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:08.607219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:08.607223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:08.607231Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1130 [2:7559342106287469521:8841]
2025-10-09T21:48:08.607248Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:08.607268Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:08.607278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:08.607282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:08.607285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:08.607304Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:08.607311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:08.607318Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:08.607622Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1130
2025-10-09T21:48:08.607646Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1130
2025-10-09T21:48:08.607377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.607389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.607391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.607395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:08.607463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:08.607465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.607478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.607490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.607508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:08.607532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342106649291129:2432], now have 1 active actors on pipe
2025-10-09T21:48:08.607540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342106287469516:8841]
2025-10-09T21:48:08.607543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342106287469516:8841]
2025-10-09T21:48:08.607546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342106287469516:8841]
2025-10-09T21:48:08.607560Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:08.607570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342106649291327:9912], now have 1 active actors on pipe
2025-10-09T21:48:08.607572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:08.607575Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:08.607577Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:08.607578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:08.607593Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:08.607596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:08.607631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342106287469516:8841]
2025-10-09T21:48:08.607635Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:08.607646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:08.607677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:08.607681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:08.607711Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7393e896-58d875d5-d20fede6-7546fbb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:08.607719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:08.607724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:08.607726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:08.607737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:08.607747Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:08.607761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:08.607928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:08.607935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:08.607957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:08.608074Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1131 partition: 0 MaxSeqNo: 25 sessionId: src_id|7393e896-58d875d5-d20fede6-7546fbb_0
2025-10-09T21:48:08.608491Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb1ee0c-b1edf565-564102db-e44596f4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7393e896-58d875d5-d20fede6-7546fbb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:08.611078Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7393e896-58d875d5-d20fede6-7546fbb_0] Write session: try to update token
2025-10-09T21:48:08.611100Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7393e896-58d875d5-d20fede6-7546fbb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:08.619621Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:08.619644Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:08.619654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342106287469516:8841]
2025-10-09T21:48:08.619709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:08.619712Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:08.619909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:08.620064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:08.620201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:08.620346Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:08.620529Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:08.620575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:08.625455Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1131 sessionId: src_id|7393e896-58d875d5-d20fede6-7546fbb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:08.627887Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:08.628097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:08.628128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:08.628169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:08.628281Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:08.633374Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:08.633393Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:08.633396Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:08.633398Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:08.633408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342106287469516:8841]
2025-10-09T21:48:08.633413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342106287469516:8841]
2025-10-09T21:48:08.633416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342106287469516:8841]
2025-10-09T21:48:08.633459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:08.633461Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:08.633468Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:08.633469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.633472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:08.633472Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:08.633602Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:08.678600Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342106649291366:9750]
2025-10-09T21:48:08.678914Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 1130, pipe: [1:7559342106649291366:9750]
2025-10-09T21:48:08.796129Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:08.803956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:08.803986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:08.804147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.804208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.804269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.804338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.804386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.804396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.804465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.804523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.804585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.804646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.804693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.804701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.804762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.804822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.804884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.804942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.804982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.804991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.805056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.805124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.805181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.805249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.805309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.805320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.805385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.805449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.805511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.805574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.805616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.805624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.805683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.805748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.805819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.805883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.805929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.805941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.806002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.806089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.806151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.806209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.806255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.806262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.806324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.806380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.806434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.806494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.806537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.806547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.806608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.806667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.806725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.806786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.806841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.806848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.806908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:08.806965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:08.807022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:08.807086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:08.807180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:08.807183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:08.807247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342106649291366:9750], now have 1 active actors on pipe
2025-10-09T21:48:08.807475Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:08.807493Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342106649291366:9750]
2025-10-09T21:48:08.807520Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:08.807522Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:08.807554Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1131 reason: error at write: Disconnected sessionId: src_id|7393e896-58d875d5-d20fede6-7546fbb_0
2025-10-09T21:48:08.807755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1131 sessionId: src_id|7393e896-58d875d5-d20fede6-7546fbb_0 is DEAD
2025-10-09T21:48:08.807924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7393e896-58d875d5-d20fede6-7546fbb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:08.807936Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7393e896-58d875d5-d20fede6-7546fbb_0] Write session will restart in 0.005377s
2025-10-09T21:48:08.807960Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7393e896-58d875d5-d20fede6-7546fbb_0] Write session: Do CDS request
2025-10-09T21:48:08.807966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7393e896-58d875d5-d20fede6-7546fbb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:08.808275Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:08.814932Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7393e896-58d875d5-d20fede6-7546fbb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:08.815075Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:08.815090Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1132
2025-10-09T21:48:08.815215Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1132 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:08.815249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1132 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:08.815260Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1132 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:08.815264Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:08.815813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1132 sessionId:  describe result for acl check
2025-10-09T21:48:08.815849Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:08.815853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:08.815855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:08.815860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342106649291384:9924] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:08.816408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342106649291384:9924] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:08.819815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342106649291384:9924] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:08.825036Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342106649291384:9924] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:08.825052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342106649291384:9924] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:08.825056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342106649291384:9924] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:08.825066Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1132 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:08.825170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:08.828083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:08.828315Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:08.828326Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:08.828339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:08.828430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:08.828472Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342106287469739:8847]
2025-10-09T21:48:08.829109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:08.829547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:08.829602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:08.829936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:08.830007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:08.830013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:08.830014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:08.830016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:08.830017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:08.830018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:08.830019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.830021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.830022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.830045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:08.830058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:08.830062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:08.830067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:08.830071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:08.830075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:08.830079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.830083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.830087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.830095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:08.830104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:08.830110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:08.830116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:08.830120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:08.830126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:08.830130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:08.830135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:08.830140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:08.830145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:08.830149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:08.830153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:08.830158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.830164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.830169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.830174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.830178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.830184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.830211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:08.836891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:08.838279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:08.838295Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:08.838298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:08.838302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:08.838309Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1131 [2:7559342106287469739:8847]
2025-10-09T21:48:08.838328Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:08.838348Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:08.838358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:08.838361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:08.838363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:08.838382Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:08.838393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:08.838402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:08.838469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.838477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.838480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.838483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:08.838566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:08.838574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.838587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:08.838598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:08.838615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:08.838642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342106649291369:2432], now have 1 active actors on pipe
2025-10-09T21:48:08.838650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342106649291571:9924], now have 1 active actors on pipe
2025-10-09T21:48:08.838658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342106287469733:8847]
2025-10-09T21:48:08.838660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342106287469733:8847]
2025-10-09T21:48:08.838663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342106287469733:8847]
2025-10-09T21:48:08.838675Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:08.838691Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:08.838696Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:08.838703Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:08.838705Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:08.838712Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:08.838715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:08.838752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342106287469733:8847]
2025-10-09T21:48:08.838760Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:08.838767Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:08.839033Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1131
2025-10-09T21:48:08.839081Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1131
2025-10-09T21:48:08.839266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:08.839283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:08.839318Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8fe69fdd-dd39bd5c-aa9c3e2f-478cb289_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:08.839333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:08.839338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:08.839341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:08.839354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:08.839371Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:08.839388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:08.839646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:08.839658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:08.839676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:08.839773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1132 partition: 0 MaxSeqNo: 25 sessionId: src_id|8fe69fdd-dd39bd5c-aa9c3e2f-478cb289_0
2025-10-09T21:48:08.840187Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7393e896-58d875d5-d20fede6-7546fbb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8fe69fdd-dd39bd5c-aa9c3e2f-478cb289_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:08.842744Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8fe69fdd-dd39bd5c-aa9c3e2f-478cb289_0] Write session: try to update token
2025-10-09T21:48:08.842766Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8fe69fdd-dd39bd5c-aa9c3e2f-478cb289_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:08.851223Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:08.851252Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:08.851267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342106287469733:8847]
2025-10-09T21:48:08.851328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:08.851425Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:08.851570Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:08.851787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:08.851988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:08.852184Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:08.852506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:08.852548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:08.857148Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1132 sessionId: src_id|8fe69fdd-dd39bd5c-aa9c3e2f-478cb289_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:08.859938Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:08.860155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:08.860176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:08.860234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:08.860335Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:08.864984Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:08.865003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:08.865006Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:08.865008Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:08.865018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342106287469733:8847]
2025-10-09T21:48:08.865022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342106287469733:8847]
2025-10-09T21:48:08.865026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342106287469733:8847]
2025-10-09T21:48:08.865069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:08.865068Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:08.865075Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:08.865080Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:08.865082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:08.865084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:08.865214Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:08.907774Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342106649291585:9750]
2025-10-09T21:48:08.908041Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 1131, pipe: [1:7559342106649291585:9750]
2025-10-09T21:48:09.022086Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:09.030285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.030311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.030458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.030518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.030588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.030652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.030703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.030711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.030777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.030837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.030906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.030972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.031021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.031029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.031090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.031155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.031220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.031277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.031318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.031327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.031397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.031464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.031528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.031594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.031649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.031659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.031727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.031791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.031861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.031926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.031970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.031974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.032038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.032107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.032172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.032232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.032277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.032285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.032344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.032405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.032469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.032534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.032577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.032585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.032646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.032706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.032769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.032831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.032874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.032882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.032945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.033004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.033069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.033143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.033190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.033197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.033270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.033335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.033398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.033458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.033499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.033506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.033570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342106649291585:9750], now have 1 active actors on pipe
2025-10-09T21:48:09.033777Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:09.033805Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342106649291585:9750]
2025-10-09T21:48:09.033835Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:09.033838Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:09.033869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1132 reason: error at write: Disconnected sessionId: src_id|8fe69fdd-dd39bd5c-aa9c3e2f-478cb289_0
2025-10-09T21:48:09.034122Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1132 sessionId: src_id|8fe69fdd-dd39bd5c-aa9c3e2f-478cb289_0 is DEAD
2025-10-09T21:48:09.034254Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fe69fdd-dd39bd5c-aa9c3e2f-478cb289_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:09.034266Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fe69fdd-dd39bd5c-aa9c3e2f-478cb289_0] Write session will restart in 0.007020s
2025-10-09T21:48:09.034292Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fe69fdd-dd39bd5c-aa9c3e2f-478cb289_0] Write session: Do CDS request
2025-10-09T21:48:09.034299Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fe69fdd-dd39bd5c-aa9c3e2f-478cb289_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:09.034680Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:09.042625Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8fe69fdd-dd39bd5c-aa9c3e2f-478cb289_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:09.042694Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:09.042710Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1133
2025-10-09T21:48:09.042860Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1133 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:09.042904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1133 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:09.042909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1133 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:09.042913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:09.043380Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1133 sessionId:  describe result for acl check
2025-10-09T21:48:09.043453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:09.043459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:09.043461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:09.043466Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342110944258906:9928] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:09.044005Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342110944258906:9928] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:09.047417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342110944258906:9928] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:09.050634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342110944258906:9928] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:09.050650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342110944258906:9928] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:09.050654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342110944258906:9928] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:09.050664Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1133 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:09.052729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:09.055283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:09.055505Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:09.055517Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:09.055531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:09.055592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:09.055635Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342110582437242:8849]
2025-10-09T21:48:09.056666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:09.057025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:09.057093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:09.057532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:09.057602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.057609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.057610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.057611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.057613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.057614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.057615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.057616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.057618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.057624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.057636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.057640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.057644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.057648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.057652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.057656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.057660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.057668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.057676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.057689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.057700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.057706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.057715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.057721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.057726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.057732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.057741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.057747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.057752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.057768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.057773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.057779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.057788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.057793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.057802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.057807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.057837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:09.065061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:09.066460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:09.066490Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:09.066493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:09.066499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:09.066510Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1132 [2:7559342110582437242:8849]
2025-10-09T21:48:09.066525Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:09.066546Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:09.066549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:09.066553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:09.066556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:09.066575Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:09.066589Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:09.066597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:09.066668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.066676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.066679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.066683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:09.066762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:09.066771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.066786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.066796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.066816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:09.066846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342110944258886:2432], now have 1 active actors on pipe
2025-10-09T21:48:09.066857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342110944259082:9928], now have 1 active actors on pipe
2025-10-09T21:48:09.066865Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342110582437237:8849]
2025-10-09T21:48:09.066868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342110582437237:8849]
2025-10-09T21:48:09.066870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342110582437237:8849]
2025-10-09T21:48:09.066885Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:09.066901Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:09.066910Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:09.066914Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:09.066915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:09.066916Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:09.066918Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:09.066902Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1132
2025-10-09T21:48:09.066958Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1132
2025-10-09T21:48:09.066960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342110582437237:8849]
2025-10-09T21:48:09.066972Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:09.066974Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:09.067197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.067212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.067245Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3a208067-c25c4dce-5bfebc16-43fbc565_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:09.067255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:09.067259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:09.067262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:09.067275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:09.067288Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:09.067306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:09.067529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.067545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.067567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:09.067652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1133 partition: 0 MaxSeqNo: 25 sessionId: src_id|3a208067-c25c4dce-5bfebc16-43fbc565_0
2025-10-09T21:48:09.068060Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fe69fdd-dd39bd5c-aa9c3e2f-478cb289_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3a208067-c25c4dce-5bfebc16-43fbc565_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:09.070576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a208067-c25c4dce-5bfebc16-43fbc565_0] Write session: try to update token
2025-10-09T21:48:09.070595Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a208067-c25c4dce-5bfebc16-43fbc565_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:09.081893Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:09.081911Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:09.081914Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:09.081916Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:09.081925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342110582437237:8849]
2025-10-09T21:48:09.081930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342110582437237:8849]
2025-10-09T21:48:09.081934Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342110582437237:8849]
2025-10-09T21:48:09.081985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:09.081983Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:09.081990Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:09.081993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.081994Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:09.081996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:09.083695Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1133 sessionId: src_id|3a208067-c25c4dce-5bfebc16-43fbc565_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:09.082143Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:09.086181Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:09.134154Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342110944259103:9750]
2025-10-09T21:48:09.136388Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 1132, pipe: [1:7559342110944259103:9750]
2025-10-09T21:48:09.247094Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:09.247508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.247527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.247575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342110944259103:9750], now have 1 active actors on pipe
2025-10-09T21:48:09.247810Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:09.247915Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:09.247932Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:09.247956Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342110944259103:9750]
2025-10-09T21:48:09.247994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1133 reason: error at write: Disconnected sessionId: src_id|3a208067-c25c4dce-5bfebc16-43fbc565_0
2025-10-09T21:48:09.247994Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:09.248181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1133 sessionId: src_id|3a208067-c25c4dce-5bfebc16-43fbc565_0 is DEAD
2025-10-09T21:48:09.248368Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a208067-c25c4dce-5bfebc16-43fbc565_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:09.248376Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a208067-c25c4dce-5bfebc16-43fbc565_0] Write session will restart in 0.008408s
2025-10-09T21:48:09.248420Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a208067-c25c4dce-5bfebc16-43fbc565_0] Write session: Do CDS request
2025-10-09T21:48:09.248427Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a208067-c25c4dce-5bfebc16-43fbc565_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:09.257958Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a208067-c25c4dce-5bfebc16-43fbc565_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:09.258042Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:09.258061Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1134
2025-10-09T21:48:09.258245Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1134 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:09.258275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1134 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:09.258280Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1134 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:09.258284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:09.259332Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1134 sessionId:  describe result for acl check
2025-10-09T21:48:09.259384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:09.259387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:09.259388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:09.259396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342110944259147:9933] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:09.260057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342110944259147:9933] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:09.263243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342110944259147:9933] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:09.266869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342110944259147:9933] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:09.266883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342110944259147:9933] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:09.266887Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342110944259147:9933] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:09.266901Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1134 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:09.268343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:09.270843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:09.271051Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:09.271062Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:09.271074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:09.271130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:09.271172Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342110582437451:8851]
2025-10-09T21:48:09.271794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:09.272156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:09.272215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:09.272543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:09.272593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.272601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.272602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.272603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.272605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.272607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.272608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.272609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.272610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.272616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.272628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.272632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.272636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.272640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.272644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.272649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.272652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.272656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.272664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.272685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.272691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.272703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.272708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.272718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.272722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.272728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.272733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.272738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.272743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.272748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.272753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.272758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.272762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.272770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.272775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.272780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.272809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:09.280002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:09.281067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:09.281083Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:09.281086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:09.281090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:09.281098Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1133 [2:7559342110582437451:8851]
2025-10-09T21:48:09.281113Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:09.281133Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:09.281136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:09.281140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:09.281142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:09.281162Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:09.281172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:09.281181Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:09.281245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.281252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.281254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.281257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:09.281316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:09.281323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.281332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.281338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.281351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:09.281369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342110582437446:8851]
2025-10-09T21:48:09.281374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342110582437446:8851]
2025-10-09T21:48:09.281376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342110582437446:8851]
2025-10-09T21:48:09.281386Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:09.281401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:09.281410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:09.281412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:09.281445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342110582437446:8851]
2025-10-09T21:48:09.281447Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:09.281980Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:09.281987Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:09.281988Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:09.281993Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:09.282248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342110944259110:2432], now have 1 active actors on pipe
2025-10-09T21:48:09.282256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342110944259313:9933], now have 1 active actors on pipe
2025-10-09T21:48:09.282289Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1133
2025-10-09T21:48:09.282301Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1133
2025-10-09T21:48:09.282549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.282563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.282600Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|54cecedd-cbc33850-8447217e-d678f37_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:09.282615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:09.282622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:09.282624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:09.282637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:09.282656Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:09.282680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:09.283379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.283389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.283407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:09.283535Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1134 partition: 0 MaxSeqNo: 25 sessionId: src_id|54cecedd-cbc33850-8447217e-d678f37_0
2025-10-09T21:48:09.283878Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a208067-c25c4dce-5bfebc16-43fbc565_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|54cecedd-cbc33850-8447217e-d678f37_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:09.286308Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54cecedd-cbc33850-8447217e-d678f37_0] Write session: try to update token
2025-10-09T21:48:09.286325Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54cecedd-cbc33850-8447217e-d678f37_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:09.296700Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:09.296719Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:09.296722Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:09.296724Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:09.296734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342110582437446:8851]
2025-10-09T21:48:09.296739Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342110582437446:8851]
2025-10-09T21:48:09.296743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342110582437446:8851]
2025-10-09T21:48:09.296784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:09.296793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.296795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:09.296847Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:09.296853Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:09.296858Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:09.296932Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:09.299649Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1134 sessionId: src_id|54cecedd-cbc33850-8447217e-d678f37_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:09.302185Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:09.348189Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342110944259328:9750]
2025-10-09T21:48:09.348531Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 1133, pipe: [1:7559342110944259328:9750]
2025-10-09T21:48:09.458116Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:09.458666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.458683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.458730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342110944259328:9750], now have 1 active actors on pipe
2025-10-09T21:48:09.459007Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:09.459428Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54cecedd-cbc33850-8447217e-d678f37_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:09.459439Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54cecedd-cbc33850-8447217e-d678f37_0] Write session will restart in 0.005742s
2025-10-09T21:48:09.459462Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54cecedd-cbc33850-8447217e-d678f37_0] Write session: Do CDS request
2025-10-09T21:48:09.459469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54cecedd-cbc33850-8447217e-d678f37_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:09.459020Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342110944259328:9750]
2025-10-09T21:48:09.459052Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:09.459054Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:09.459057Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:09.459112Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1134 reason: error at write: Disconnected sessionId: src_id|54cecedd-cbc33850-8447217e-d678f37_0
2025-10-09T21:48:09.459304Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1134 sessionId: src_id|54cecedd-cbc33850-8447217e-d678f37_0 is DEAD
2025-10-09T21:48:09.466512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54cecedd-cbc33850-8447217e-d678f37_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:09.466549Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:09.466572Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1135
2025-10-09T21:48:09.466974Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1135 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:09.467017Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1135 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:09.467028Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1135 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:09.467032Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:09.467497Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1135 sessionId:  describe result for acl check
2025-10-09T21:48:09.467534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:09.467536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:09.467537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:09.467543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342110944259348:9937] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:09.468096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342110944259348:9937] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:09.470845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342110944259348:9937] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:09.473828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342110944259348:9937] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:09.473844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342110944259348:9937] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:09.473848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342110944259348:9937] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:09.473856Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1135 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:09.475209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:09.477419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:09.477627Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:09.477639Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:09.477653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:09.477710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:09.477775Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342110582437660:8853]
2025-10-09T21:48:09.478361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:09.478691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:09.478741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:09.479055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:09.479106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.479112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.479113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.479114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.479115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.479116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.479117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.479118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.479119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.479124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.479135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.479139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.479143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.479147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.479150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.479154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.479158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.479161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.479168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.479176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.479181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.479186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.479190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.479195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.479211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.479220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.479224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.479229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.479233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.479239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.479244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.479249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.479253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.479262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.479267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.479272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.479300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:09.485496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:09.486679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:09.486694Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:09.486697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:09.486701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:09.486707Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1134 [2:7559342110582437660:8853]
2025-10-09T21:48:09.486720Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:09.486737Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:09.486744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:09.486748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:09.486750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:09.486765Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:09.486781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:09.486788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:09.486843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.486851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.486853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.486856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:09.486918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:09.486926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.486937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.486954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.486971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:09.486992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342110944259534:9937], now have 1 active actors on pipe
2025-10-09T21:48:09.487000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342110944259330:2432], now have 1 active actors on pipe
2025-10-09T21:48:09.487008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342110582437655:8853]
2025-10-09T21:48:09.487011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342110582437655:8853]
2025-10-09T21:48:09.487013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342110582437655:8853]
2025-10-09T21:48:09.487027Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:09.487041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:09.487040Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:09.487044Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:09.487046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:09.487059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:09.487063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:09.487071Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1134
2025-10-09T21:48:09.487077Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1134
2025-10-09T21:48:09.487090Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342110582437655:8853]
2025-10-09T21:48:09.487104Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:09.487115Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:09.487137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.487146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.487176Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e211d7f2-2ba9ef51-40f1648e-8bbd6f39_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:09.487187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:09.487192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:09.487198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:09.487210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:09.487220Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:09.487239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:09.487691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.487706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.487729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:09.487848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1135 partition: 0 MaxSeqNo: 25 sessionId: src_id|e211d7f2-2ba9ef51-40f1648e-8bbd6f39_0
2025-10-09T21:48:09.488210Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54cecedd-cbc33850-8447217e-d678f37_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e211d7f2-2ba9ef51-40f1648e-8bbd6f39_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:09.490740Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e211d7f2-2ba9ef51-40f1648e-8bbd6f39_0] Write session: try to update token
2025-10-09T21:48:09.490759Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e211d7f2-2ba9ef51-40f1648e-8bbd6f39_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:09.498322Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:09.498352Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:09.498363Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342110582437655:8853]
2025-10-09T21:48:09.498405Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:09.498407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:09.498915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:09.499052Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:09.499177Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:09.499301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:09.499488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:09.499533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:09.504094Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1135 sessionId: src_id|e211d7f2-2ba9ef51-40f1648e-8bbd6f39_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:09.506490Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:09.506705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.506725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.506767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:09.507481Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:09.510805Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:09.510822Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:09.510824Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:09.510826Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:09.510836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342110582437655:8853]
2025-10-09T21:48:09.510840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342110582437655:8853]
2025-10-09T21:48:09.510847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342110582437655:8853]
2025-10-09T21:48:09.510890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:09.510896Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:09.510901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.510902Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:09.510904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:09.510906Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:09.511036Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:09.559015Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342110944259549:9750]
2025-10-09T21:48:09.559340Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 1134, pipe: [1:7559342110944259549:9750]
2025-10-09T21:48:09.666313Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:09.674215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.674238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.674395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.674464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.674534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.677866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:09.677880Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:09.677920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1135 reason: error at write: Disconnected sessionId: src_id|e211d7f2-2ba9ef51-40f1648e-8bbd6f39_0
2025-10-09T21:48:09.674606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.674659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.674665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.674734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.674786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.674842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.674902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.674941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.674944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.675002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.675057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.675114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.675168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.675335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.675339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.675403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.675457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.675516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.675571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.675612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.675616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.675683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.675738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.678128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1135 sessionId: src_id|e211d7f2-2ba9ef51-40f1648e-8bbd6f39_0 is DEAD
2025-10-09T21:48:09.675797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.675853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.675984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.675988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.676054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.676110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.676171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.676226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.676266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.676269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.676331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.676385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.676444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.676504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.676543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.676549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.676611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.676666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.676724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.676781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.676821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.676824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.676892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.676949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.677010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.677067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.677106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.677109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.677179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.677236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.677295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.677356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.677399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.677402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.677471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342110944259549:9750], now have 1 active actors on pipe
2025-10-09T21:48:09.678328Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:09.678376Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342110944259549:9750]
2025-10-09T21:48:09.678513Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:09.682263Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e211d7f2-2ba9ef51-40f1648e-8bbd6f39_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:09.682274Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e211d7f2-2ba9ef51-40f1648e-8bbd6f39_0] Write session will restart in 0.007131s
2025-10-09T21:48:09.682303Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e211d7f2-2ba9ef51-40f1648e-8bbd6f39_0] Write session: Do CDS request
2025-10-09T21:48:09.682310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e211d7f2-2ba9ef51-40f1648e-8bbd6f39_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:09.690554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e211d7f2-2ba9ef51-40f1648e-8bbd6f39_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:09.690607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:09.690620Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1136
2025-10-09T21:48:09.690804Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1136 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:09.690849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1136 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:09.690861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1136 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:09.690865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:09.691237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1136 sessionId:  describe result for acl check
2025-10-09T21:48:09.691278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:09.691280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:09.691282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:09.691288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342110944259688:9947] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:09.692130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342110944259688:9947] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:09.694664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342110944259688:9947] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:09.694889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:09.697156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:09.697374Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:09.697384Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:09.697395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:09.697447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:09.697484Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342110582437903:8867]
2025-10-09T21:48:09.697549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342110944259688:9947] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:09.697564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342110944259688:9947] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:09.697569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342110944259688:9947] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:09.697577Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1136 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:09.698092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:09.698484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:09.698536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:09.698863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:09.698929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.698935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.698936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.698937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.698938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.698939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.698941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.698942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.698943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.698948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.698959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.698962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.698967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.698970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.698973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.698976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.698979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.698982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.698988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.698996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.699001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.699006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.699010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.699015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.699018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.699023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.699027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.699031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.699034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.699038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.699043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.699047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.699051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.699056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.699060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.699065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.699103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:09.705405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:09.706889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:09.706906Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:09.706908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:09.706912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:09.706919Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1135 [2:7559342110582437903:8867]
2025-10-09T21:48:09.706933Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:09.706952Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:09.706955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:09.706958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:09.706960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:09.706978Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:09.706988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:09.706996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:09.707053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.707059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.707061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.707064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:09.707134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:09.707142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.707154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.707165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.707182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:09.707211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342110944259772:9947], now have 1 active actors on pipe
2025-10-09T21:48:09.707214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342110944259565:2432], now have 1 active actors on pipe
2025-10-09T21:48:09.707221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342110582437898:8867]
2025-10-09T21:48:09.707223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342110582437898:8867]
2025-10-09T21:48:09.707224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342110582437898:8867]
2025-10-09T21:48:09.707236Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:09.707251Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:09.707253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:09.707254Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:09.707257Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:09.707268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:09.707270Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:09.707306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342110582437898:8867]
2025-10-09T21:48:09.707322Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:09.707333Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1135
2025-10-09T21:48:09.707364Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1135
2025-10-09T21:48:09.707345Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:09.707404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.707409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.707441Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a5110343-6586467b-85a4ce3d-984ba8dd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:09.707449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:09.707453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:09.707455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:09.707466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:09.707477Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:09.707493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:09.707603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.707620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.707634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:09.707723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1136 partition: 0 MaxSeqNo: 25 sessionId: src_id|a5110343-6586467b-85a4ce3d-984ba8dd_0
2025-10-09T21:48:09.708084Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e211d7f2-2ba9ef51-40f1648e-8bbd6f39_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a5110343-6586467b-85a4ce3d-984ba8dd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:09.710519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a5110343-6586467b-85a4ce3d-984ba8dd_0] Write session: try to update token
2025-10-09T21:48:09.710538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a5110343-6586467b-85a4ce3d-984ba8dd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:09.718675Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:09.718693Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:09.718703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342110582437898:8867]
2025-10-09T21:48:09.718748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:09.718755Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:09.718914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:09.719137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:09.719332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:09.719560Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:09.719747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:09.719796Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:09.724140Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1136 sessionId: src_id|a5110343-6586467b-85a4ce3d-984ba8dd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:09.726562Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:09.726814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.726843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.726883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:09.726987Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:09.736696Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:09.736714Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:09.736717Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:09.736719Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:09.736730Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342110582437898:8867]
2025-10-09T21:48:09.736737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342110582437898:8867]
2025-10-09T21:48:09.736740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342110582437898:8867]
2025-10-09T21:48:09.736787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:09.736799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.736801Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:09.736803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:09.736809Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:09.736812Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:09.736937Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:09.778700Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342110944259793:9750]
2025-10-09T21:48:09.779010Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 1135, pipe: [1:7559342110944259793:9750]
2025-10-09T21:48:09.897458Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:09.905571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.905597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.905771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.905829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.905897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.905962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.906008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.906019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.906098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.906159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.906224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.906296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.906337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.906348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.906411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.906469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.906531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.906603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.906650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.906653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.906721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.906777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.906851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.906918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.906961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.906968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.907036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.907092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.907150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.907208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.907269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.907277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.907343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.907406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.907471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.907532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.907579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.907585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.907645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.907704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.907764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.907820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.909420Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:09.909444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:09.909448Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342110944259793:9750]
2025-10-09T21:48:09.909472Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:09.909476Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1136 reason: error at write: Disconnected sessionId: src_id|a5110343-6586467b-85a4ce3d-984ba8dd_0
2025-10-09T21:48:09.909817Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a5110343-6586467b-85a4ce3d-984ba8dd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:09.909828Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a5110343-6586467b-85a4ce3d-984ba8dd_0] Write session will restart in 0.009204s
2025-10-09T21:48:09.909852Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a5110343-6586467b-85a4ce3d-984ba8dd_0] Write session: Do CDS request
2025-10-09T21:48:09.909858Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a5110343-6586467b-85a4ce3d-984ba8dd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:09.909661Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1136 sessionId: src_id|a5110343-6586467b-85a4ce3d-984ba8dd_0 is DEAD
2025-10-09T21:48:09.908219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.908231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.908304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.908358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.908409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.908463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.908501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.908504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.908563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.908615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.908670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.908728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.908770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.908773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.908833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:09.908884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:09.908937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:09.908987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:09.909106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:09.909109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:09.909180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342110944259793:9750], now have 1 active actors on pipe
2025-10-09T21:48:09.909735Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:09.920534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a5110343-6586467b-85a4ce3d-984ba8dd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:09.920641Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:09.920659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1137
2025-10-09T21:48:09.920778Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1137 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:09.920814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1137 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:09.920823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1137 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:09.920826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:09.921694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1137 sessionId:  describe result for acl check
2025-10-09T21:48:09.921741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:09.921742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:09.921744Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:09.921764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342110944259856:9953] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:09.922361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342110944259856:9953] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:09.926157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342110944259856:9953] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:09.930141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342110944259856:9953] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:09.930157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342110944259856:9953] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:09.930161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342110944259856:9953] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:09.930172Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1137 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:09.931083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:09.933649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:09.933867Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:09.933877Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:09.933890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:09.933943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:09.933985Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342110582438124:8871]
2025-10-09T21:48:09.934582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:09.934918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:09.934974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:09.935328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:09.935381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.935387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.935388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.935389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.935391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.935392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.935393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.935394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.935395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.935400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.935410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.935414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.935417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.935424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.935428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.935432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.935435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.935438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.935446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.935454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:09.935460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.935466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:09.935472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.935477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:09.935482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.935487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:09.935491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.935496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:09.935500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.935505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:09.935511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.935517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.935521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.935525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.935529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.935534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.935562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:09.942453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:09.943616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:09.943639Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:09.943641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:09.943645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:09.943652Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1136 [2:7559342110582438124:8871]
2025-10-09T21:48:09.943667Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:09.943687Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:09.943690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:09.943693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:09.943696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:09.943714Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:09.943726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:09.943734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:09.943804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.943813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.943814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.943817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:09.943887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:09.943894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.943906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:09.943920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:09.943938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:09.944096Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1136
2025-10-09T21:48:09.944147Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1136
2025-10-09T21:48:09.943962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342110944260004:9953], now have 1 active actors on pipe
2025-10-09T21:48:09.943974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342110944259798:2432], now have 1 active actors on pipe
2025-10-09T21:48:09.943981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342110582438119:8871]
2025-10-09T21:48:09.943984Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342110582438119:8871]
2025-10-09T21:48:09.943986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342110582438119:8871]
2025-10-09T21:48:09.943998Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:09.944012Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:09.944012Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:09.944016Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:09.944019Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:09.944028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:09.944048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:09.944086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342110582438119:8871]
2025-10-09T21:48:09.944094Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:09.944121Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:09.944249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.944260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.944289Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|aed33d44-5cbfdeb-4acb6ad-d4ffb700_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:09.944302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:09.944306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:09.944308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:09.944321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:09.944332Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:09.944347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:09.944542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.944557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.944581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:09.944743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1137 partition: 0 MaxSeqNo: 25 sessionId: src_id|aed33d44-5cbfdeb-4acb6ad-d4ffb700_0
2025-10-09T21:48:09.945123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a5110343-6586467b-85a4ce3d-984ba8dd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|aed33d44-5cbfdeb-4acb6ad-d4ffb700_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:09.947623Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aed33d44-5cbfdeb-4acb6ad-d4ffb700_0] Write session: try to update token
2025-10-09T21:48:09.947643Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aed33d44-5cbfdeb-4acb6ad-d4ffb700_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:09.959608Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:09.959628Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:09.959638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342110582438119:8871]
2025-10-09T21:48:09.959687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:09.959697Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:09.959860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:09.960006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:09.960145Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:09.960342Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:09.960594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:09.960637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:09.961791Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1137 sessionId: src_id|aed33d44-5cbfdeb-4acb6ad-d4ffb700_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:09.964172Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:09.964398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:09.964434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:09.964472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:09.964555Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:09.972357Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:09.972376Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:09.972379Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:09.972381Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:09.972391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342110582438119:8871]
2025-10-09T21:48:09.972395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342110582438119:8871]
2025-10-09T21:48:09.972399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342110582438119:8871]
2025-10-09T21:48:09.972443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:09.972450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:09.972458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:09.972584Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:09.972700Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:09.972714Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:09.972718Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:10.009607Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342115239227315:9750]
2025-10-09T21:48:10.010013Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 1136, pipe: [1:7559342115239227315:9750]
2025-10-09T21:48:10.135285Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:10.143797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:10.143835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:10.144028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.144097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.144164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.144235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.144283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.144289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.144361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.144416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.144477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.144544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.144582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.144586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.144650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.144706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.144766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.144829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.144874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.144877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.144948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.145006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.145068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.145128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.145165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.145167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.145236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.145302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.145363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.145425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.145467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.145469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.145537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.145593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.145647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.145713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.145771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.145775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.145843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.145902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.145965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.146281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.146336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.146339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.146406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.146463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.146524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.146587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.146632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.146636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.146706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.146772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.146833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.146897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.146938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.146941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.147011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.147070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.147134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.147200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.147245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.147248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.147340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342115239227315:9750], now have 1 active actors on pipe
2025-10-09T21:48:10.148826Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:10.148821Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342115239227315:9750]
2025-10-09T21:48:10.148843Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:10.148866Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:10.148891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1137 reason: error at write: Disconnected sessionId: src_id|aed33d44-5cbfdeb-4acb6ad-d4ffb700_0
2025-10-09T21:48:10.149104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1137 sessionId: src_id|aed33d44-5cbfdeb-4acb6ad-d4ffb700_0 is DEAD
2025-10-09T21:48:10.149251Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aed33d44-5cbfdeb-4acb6ad-d4ffb700_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:10.149262Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aed33d44-5cbfdeb-4acb6ad-d4ffb700_0] Write session will restart in 0.007062s
2025-10-09T21:48:10.148984Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:10.149286Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aed33d44-5cbfdeb-4acb6ad-d4ffb700_0] Write session: Do CDS request
2025-10-09T21:48:10.149293Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aed33d44-5cbfdeb-4acb6ad-d4ffb700_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:10.157622Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aed33d44-5cbfdeb-4acb6ad-d4ffb700_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:10.158144Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:10.158164Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1138
2025-10-09T21:48:10.158343Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1138 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:10.158400Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1138 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:10.158406Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1138 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:10.158412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:10.158926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1138 sessionId:  describe result for acl check
2025-10-09T21:48:10.158964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:10.158966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:10.158968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:10.158975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342115239227335:9957] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:10.159585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342115239227335:9957] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:10.162998Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342115239227335:9957] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:10.168197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342115239227335:9957] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:10.168216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342115239227335:9957] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:10.168220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342115239227335:9957] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:10.168231Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1138 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:10.171594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:10.174932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:10.175203Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:10.175212Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:10.175227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:10.175295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:10.175346Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342114877405632:8873]
2025-10-09T21:48:10.175978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:10.176360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:10.176420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:10.176731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:10.176784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:10.176789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:10.176790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:10.176792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:10.176793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:10.176794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:10.176795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.176796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.176797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.176803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:10.176815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:10.176819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:10.176823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:10.176826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:10.176830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:10.176834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.176838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.176841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.176849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:10.176858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:10.176864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:10.176870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:10.176874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:10.176880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:10.176886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:10.176891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:10.176896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:10.176901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:10.176906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:10.176914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:10.176925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.176931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.176935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.176940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.176945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.176950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.176980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:10.184103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:10.185456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:10.185471Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:10.185474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:10.185478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:10.185485Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1137 [2:7559342114877405632:8873]
2025-10-09T21:48:10.185501Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:10.185523Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:10.185528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:10.185532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:10.185536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:10.185560Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:10.185575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:10.185584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:10.185649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.185657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.185660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.185663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:10.185741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:10.185749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.185772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.185784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.185806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:10.185835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342115239227320:2432], now have 1 active actors on pipe
2025-10-09T21:48:10.185847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342115239227485:9957], now have 1 active actors on pipe
2025-10-09T21:48:10.185857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342114877405627:8873]
2025-10-09T21:48:10.185861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342114877405627:8873]
2025-10-09T21:48:10.185863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342114877405627:8873]
2025-10-09T21:48:10.185878Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:10.185899Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:10.185905Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:10.185915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:10.185917Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:10.185918Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:10.185919Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:10.185960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342114877405627:8873]
2025-10-09T21:48:10.185947Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1137
2025-10-09T21:48:10.185954Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1137
2025-10-09T21:48:10.185975Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:10.186048Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:10.186142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:10.186153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:10.186185Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ac7cfc7b-e00ee778-2bec05d4-e3aa1462_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:10.186198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:10.186203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:10.186206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:10.186217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:10.186229Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:10.186247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:10.186485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:10.186500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:10.186524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:10.186988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1138 partition: 0 MaxSeqNo: 25 sessionId: src_id|ac7cfc7b-e00ee778-2bec05d4-e3aa1462_0
2025-10-09T21:48:10.187450Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aed33d44-5cbfdeb-4acb6ad-d4ffb700_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ac7cfc7b-e00ee778-2bec05d4-e3aa1462_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:10.190106Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac7cfc7b-e00ee778-2bec05d4-e3aa1462_0] Write session: try to update token
2025-10-09T21:48:10.190125Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac7cfc7b-e00ee778-2bec05d4-e3aa1462_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:10.201821Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:10.201842Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:10.201844Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:10.201846Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:10.201857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342114877405627:8873]
2025-10-09T21:48:10.201862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342114877405627:8873]
2025-10-09T21:48:10.201867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342114877405627:8873]
2025-10-09T21:48:10.201919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:10.201923Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:10.201928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.201930Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:10.201931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:10.201935Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:10.202138Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:10.205519Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1138 sessionId: src_id|ac7cfc7b-e00ee778-2bec05d4-e3aa1462_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:10.208351Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:10.248690Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342115239227547:9750]
2025-10-09T21:48:10.249092Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 1137, pipe: [1:7559342115239227547:9750]
2025-10-09T21:48:10.392154Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:10.392689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:10.392701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:10.392751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342115239227547:9750], now have 1 active actors on pipe
2025-10-09T21:48:10.393917Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac7cfc7b-e00ee778-2bec05d4-e3aa1462_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:10.393929Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac7cfc7b-e00ee778-2bec05d4-e3aa1462_0] Write session will restart in 0.007823s
2025-10-09T21:48:10.393955Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac7cfc7b-e00ee778-2bec05d4-e3aa1462_0] Write session: Do CDS request
2025-10-09T21:48:10.393964Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac7cfc7b-e00ee778-2bec05d4-e3aa1462_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:10.393397Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:10.393468Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342115239227547:9750]
2025-10-09T21:48:10.393488Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:10.393491Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:10.393551Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1138 reason: error at write: Disconnected sessionId: src_id|ac7cfc7b-e00ee778-2bec05d4-e3aa1462_0
2025-10-09T21:48:10.393748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1138 sessionId: src_id|ac7cfc7b-e00ee778-2bec05d4-e3aa1462_0 is DEAD
2025-10-09T21:48:10.393448Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:10.402547Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac7cfc7b-e00ee778-2bec05d4-e3aa1462_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:10.402644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:10.402661Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1139
2025-10-09T21:48:10.402885Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1139 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:10.402928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1139 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:10.402939Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1139 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:10.402943Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:10.403389Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1139 sessionId:  describe result for acl check
2025-10-09T21:48:10.403427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:10.403433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:10.403435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:10.403441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342115239227565:9962] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:10.404041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342115239227565:9962] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:10.407710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342115239227565:9962] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:10.411515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342115239227565:9962] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:10.411532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342115239227565:9962] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:10.411536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342115239227565:9962] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:10.411544Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1139 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:10.417107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:10.419566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:10.419791Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:10.419801Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:10.419812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:10.419876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:10.419919Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342114877405845:8875]
2025-10-09T21:48:10.420469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:10.420795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:10.420850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:10.421197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:10.421264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:10.421272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:10.421274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:10.421275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:10.421277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:10.421278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:10.421279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.421280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.421282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.421287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:10.421299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:10.421302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:10.421306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:10.421311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:10.421315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:10.421319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.421323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.421327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.421334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:10.421342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:10.421348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:10.421353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:10.421357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:10.421362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:10.421366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:10.421371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:10.421376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:10.421381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:10.421385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:10.421390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:10.421396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.421402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.421413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.421419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.421424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.421431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.421465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:10.429450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:10.430555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:10.430575Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:10.430579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:10.430584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:10.430591Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1138 [2:7559342114877405845:8875]
2025-10-09T21:48:10.430605Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:10.430625Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:10.430629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:10.430632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:10.430634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:10.430654Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:10.430661Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:10.430669Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:10.430730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.430743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.430745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.430748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:10.430819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:10.430826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.430839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.430854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.430868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:10.430892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342114877405840:8875]
2025-10-09T21:48:10.430898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342114877405840:8875]
2025-10-09T21:48:10.430901Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342114877405840:8875]
2025-10-09T21:48:10.430914Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:10.430934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342115239227550:2432], now have 1 active actors on pipe
2025-10-09T21:48:10.430942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342115239227686:9962], now have 1 active actors on pipe
2025-10-09T21:48:10.430952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:10.430973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:10.430980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:10.431021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342114877405840:8875]
2025-10-09T21:48:10.431029Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:10.430994Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1138
2025-10-09T21:48:10.431020Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1138
2025-10-09T21:48:10.431075Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:10.431083Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:10.431086Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:10.431092Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:10.431196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:10.431211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:10.431246Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4f1c5f3e-f1d6a913-a295a596-c33a7e99_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:10.431264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:10.431269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:10.431272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:10.431285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:10.431297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:10.431315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:10.431626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:10.431638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:10.431658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:10.431752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1139 partition: 0 MaxSeqNo: 25 sessionId: src_id|4f1c5f3e-f1d6a913-a295a596-c33a7e99_0
2025-10-09T21:48:10.432150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac7cfc7b-e00ee778-2bec05d4-e3aa1462_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4f1c5f3e-f1d6a913-a295a596-c33a7e99_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:10.434829Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4f1c5f3e-f1d6a913-a295a596-c33a7e99_0] Write session: try to update token
2025-10-09T21:48:10.434850Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4f1c5f3e-f1d6a913-a295a596-c33a7e99_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:10.446348Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:10.446369Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:10.446373Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:10.446375Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:10.446386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342114877405840:8875]
2025-10-09T21:48:10.446391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342114877405840:8875]
2025-10-09T21:48:10.446395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342114877405840:8875]
2025-10-09T21:48:10.446443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:10.446446Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:10.446451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.446452Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:10.446454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:10.446457Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:10.446596Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:10.453822Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1139 sessionId: src_id|4f1c5f3e-f1d6a913-a295a596-c33a7e99_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:10.456267Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:10.493447Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342115239227772:9750]
2025-10-09T21:48:10.493805Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 1138, pipe: [1:7559342115239227772:9750]
2025-10-09T21:48:10.608730Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:10.609318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:10.609339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:10.609386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342115239227772:9750], now have 1 active actors on pipe
2025-10-09T21:48:10.609588Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:10.609671Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342115239227772:9750]
2025-10-09T21:48:10.609689Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:10.609703Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:10.609705Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:10.609821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1139 reason: error at write: Disconnected sessionId: src_id|4f1c5f3e-f1d6a913-a295a596-c33a7e99_0
2025-10-09T21:48:10.610045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1139 sessionId: src_id|4f1c5f3e-f1d6a913-a295a596-c33a7e99_0 is DEAD
2025-10-09T21:48:10.610216Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f1c5f3e-f1d6a913-a295a596-c33a7e99_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:10.610228Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f1c5f3e-f1d6a913-a295a596-c33a7e99_0] Write session will restart in 0.005615s
2025-10-09T21:48:10.610257Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f1c5f3e-f1d6a913-a295a596-c33a7e99_0] Write session: Do CDS request
2025-10-09T21:48:10.610264Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f1c5f3e-f1d6a913-a295a596-c33a7e99_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:10.617624Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4f1c5f3e-f1d6a913-a295a596-c33a7e99_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:10.617702Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:10.617719Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1140
2025-10-09T21:48:10.617874Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1140 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:10.617904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1140 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:10.617909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1140 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:10.617915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:10.618466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1140 sessionId:  describe result for acl check
2025-10-09T21:48:10.618510Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:10.618527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:10.618529Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:10.618534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342115239227787:9966] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:10.619174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342115239227787:9966] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:10.622113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342115239227787:9966] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:10.626516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342115239227787:9966] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:10.626533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342115239227787:9966] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:10.626537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342115239227787:9966] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:10.626547Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1140 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:10.632731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:10.636389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:10.636881Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:10.636901Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:10.636918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:10.637001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:10.637057Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342114877406092:8883]
2025-10-09T21:48:10.637686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:10.638078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:10.638149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:10.638513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:10.638580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:10.638586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:10.638588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:10.638589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:10.638590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:10.638592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:10.638593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.638594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.638595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.638601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:10.638614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:10.638618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:10.638622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:10.638627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:10.638631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:10.638635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.638639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.638643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.638651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:10.638659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:10.638666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:10.638671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:10.638676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:10.638680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:10.638685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:10.638691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:10.638696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:10.638701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:10.638706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:10.638714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:10.638719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.638724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.638728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.638733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.638738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.638742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.638774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:10.645521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:10.646585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:10.646602Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:10.646604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:10.646609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:10.646615Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1139 [2:7559342114877406092:8883]
2025-10-09T21:48:10.646630Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:10.646648Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:10.646651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:10.646655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:10.646657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:10.646676Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:10.646682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:10.646689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:10.646768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.646775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.646778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.646781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:10.646853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:10.646860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.646872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.646888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.646908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:10.646934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342115239227775:2432], now have 1 active actors on pipe
2025-10-09T21:48:10.646941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342115239227896:9966], now have 1 active actors on pipe
2025-10-09T21:48:10.646949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342114877406086:8883]
2025-10-09T21:48:10.646951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342114877406086:8883]
2025-10-09T21:48:10.646954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342114877406086:8883]
2025-10-09T21:48:10.646968Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:10.646986Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:10.647001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:10.647005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:10.647040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342114877406086:8883]
2025-10-09T21:48:10.647024Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1139
2025-10-09T21:48:10.647044Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:10.647045Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1139
2025-10-09T21:48:10.647047Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:10.647048Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:10.647051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:10.647058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:10.647172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:10.647184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:10.647213Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fd10f76f-cdd13fd3-843dbd49-3fe66db6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:10.647227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:10.647237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:10.647240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:10.647251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:10.647267Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:10.647291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:10.647583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:10.647602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:10.647620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:10.647732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1140 partition: 0 MaxSeqNo: 25 sessionId: src_id|fd10f76f-cdd13fd3-843dbd49-3fe66db6_0
2025-10-09T21:48:10.648149Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f1c5f3e-f1d6a913-a295a596-c33a7e99_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fd10f76f-cdd13fd3-843dbd49-3fe66db6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:10.650881Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fd10f76f-cdd13fd3-843dbd49-3fe66db6_0] Write session: try to update token
2025-10-09T21:48:10.650900Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fd10f76f-cdd13fd3-843dbd49-3fe66db6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:10.661427Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:10.661447Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:10.661457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342114877406086:8883]
2025-10-09T21:48:10.661519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:10.661522Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:10.661698Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:10.661855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:10.661994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:10.662157Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:10.662339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:10.662386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:10.664795Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1140 sessionId: src_id|fd10f76f-cdd13fd3-843dbd49-3fe66db6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:10.667356Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:10.667570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:10.667594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:10.667645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:10.667773Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:10.674688Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:10.674709Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:10.674712Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:10.674714Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:10.674724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342114877406086:8883]
2025-10-09T21:48:10.674729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342114877406086:8883]
2025-10-09T21:48:10.674732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342114877406086:8883]
2025-10-09T21:48:10.674777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:10.674787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.674787Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:10.674790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:10.674801Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:10.674804Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:10.674923Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:10.710056Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342115239227999:9750]
2025-10-09T21:48:10.710360Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 1139, pipe: [1:7559342115239227999:9750]
2025-10-09T21:48:10.834561Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:10.842785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:10.842821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:10.843001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.843066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.843131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.843197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.843243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.843254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.843316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.843379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.843436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.843499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.843544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.843552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.843616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.843674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.843739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.843805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.843850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.843859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.843926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.843993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.844053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.844111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.844160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.844168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.844232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.844295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.844358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.844418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.844467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.844476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.844536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.844596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.844653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.844715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.844762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.844765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.844825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.844876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.844931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.844988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.845026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.845030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.845090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.845142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.845196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.845253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.845291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.845296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.845361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.845416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.845470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.845525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.845562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.845565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.845625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:10.845679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:10.845733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:10.845808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:10.845845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:10.845848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:10.845918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342115239227999:9750], now have 1 active actors on pipe
2025-10-09T21:48:10.846277Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342115239227999:9750]
2025-10-09T21:48:10.846281Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:10.846286Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:10.846313Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:10.846319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1140 reason: error at write: Disconnected sessionId: src_id|fd10f76f-cdd13fd3-843dbd49-3fe66db6_0
2025-10-09T21:48:10.846531Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1140 sessionId: src_id|fd10f76f-cdd13fd3-843dbd49-3fe66db6_0 is DEAD
2025-10-09T21:48:10.846707Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd10f76f-cdd13fd3-843dbd49-3fe66db6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:10.846719Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd10f76f-cdd13fd3-843dbd49-3fe66db6_0] Write session will restart in 0.007069s
2025-10-09T21:48:10.846744Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd10f76f-cdd13fd3-843dbd49-3fe66db6_0] Write session: Do CDS request
2025-10-09T21:48:10.846750Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd10f76f-cdd13fd3-843dbd49-3fe66db6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:10.847247Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:10.854592Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fd10f76f-cdd13fd3-843dbd49-3fe66db6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:10.854641Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:10.854657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1141
2025-10-09T21:48:10.854844Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1141 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:10.854879Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1141 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:10.854892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1141 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:10.854904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:10.855364Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1141 sessionId:  describe result for acl check
2025-10-09T21:48:10.855400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:10.855406Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:10.855408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:10.855414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342115239228053:9978] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:10.855993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342115239228053:9978] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:10.859094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342115239228053:9978] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:10.862969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:10.865575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:10.865827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342115239228053:9978] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:10.865841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342115239228053:9978] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:10.865845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342115239228053:9978] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:10.865855Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1141 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:10.865918Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:10.865927Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:10.865941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:10.866005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:10.866065Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342114877406316:8893]
2025-10-09T21:48:10.866644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:10.866973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:10.867020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:10.867365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:10.867413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:10.867420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:10.867421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:10.867423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:10.867424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:10.867425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:10.867426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.867427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.867428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.867433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:10.867444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:10.867448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:10.867452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:10.867456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:10.867460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:10.867464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.867467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.867471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.867478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:10.867486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:10.867492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:10.867497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:10.867502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:10.867507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:10.867511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:10.867516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:10.867521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:10.867526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:10.867530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:10.867535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:10.867540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.867545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.867550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.867555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.867559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.867564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.867593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:10.874326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:10.875425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:10.875440Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:10.875443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:10.875447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:10.875454Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1140 [2:7559342114877406316:8893]
2025-10-09T21:48:10.875469Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:10.875487Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:10.875496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:10.875500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:10.875502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:10.875520Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:10.875526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:10.875534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:10.875597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.875607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.875609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.875612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:10.875690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:10.875700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.875714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:10.875730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:10.875753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:10.875780Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342114877406311:8893]
2025-10-09T21:48:10.875788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342114877406311:8893]
2025-10-09T21:48:10.875790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342114877406311:8893]
2025-10-09T21:48:10.875805Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:10.875826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342115239228238:9978], now have 1 active actors on pipe
2025-10-09T21:48:10.875831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342115239228028:2432], now have 1 active actors on pipe
2025-10-09T21:48:10.875841Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:10.875842Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:10.875846Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:10.875849Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:10.875858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:10.875862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:10.875898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342114877406311:8893]
2025-10-09T21:48:10.875917Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:10.875877Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1140
2025-10-09T21:48:10.875938Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1140
2025-10-09T21:48:10.875968Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:10.876049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:10.876062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:10.876095Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|267d673c-f12cdfc3-6c1514f9-1b3fad1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:10.876103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:10.876108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:10.876110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:10.876123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:10.876141Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:10.876163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:10.876332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:10.876346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:10.876369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:10.876453Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1141 partition: 0 MaxSeqNo: 25 sessionId: src_id|267d673c-f12cdfc3-6c1514f9-1b3fad1_0
2025-10-09T21:48:10.876788Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd10f76f-cdd13fd3-843dbd49-3fe66db6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|267d673c-f12cdfc3-6c1514f9-1b3fad1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:10.879394Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|267d673c-f12cdfc3-6c1514f9-1b3fad1_0] Write session: try to update token
2025-10-09T21:48:10.879417Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|267d673c-f12cdfc3-6c1514f9-1b3fad1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:10.890713Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:10.890768Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:10.890771Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:10.890773Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:10.890783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342114877406311:8893]
2025-10-09T21:48:10.890788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342114877406311:8893]
2025-10-09T21:48:10.890791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342114877406311:8893]
2025-10-09T21:48:10.890841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:10.890854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:10.890857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:10.890942Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:10.890959Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:10.890963Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:10.890988Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:10.892680Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1141 sessionId: src_id|267d673c-f12cdfc3-6c1514f9-1b3fad1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:10.895074Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:10.946521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342115239228250:9750]
2025-10-09T21:48:10.946869Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 1140, pipe: [1:7559342115239228250:9750]
2025-10-09T21:48:11.052392Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:11.052909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.052927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.052977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342115239228250:9750], now have 1 active actors on pipe
2025-10-09T21:48:11.053245Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:11.053247Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:11.053262Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:11.053274Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342115239228250:9750]
2025-10-09T21:48:11.053296Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:11.053327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1141 reason: error at write: Disconnected sessionId: src_id|267d673c-f12cdfc3-6c1514f9-1b3fad1_0
2025-10-09T21:48:11.053529Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1141 sessionId: src_id|267d673c-f12cdfc3-6c1514f9-1b3fad1_0 is DEAD
2025-10-09T21:48:11.053687Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|267d673c-f12cdfc3-6c1514f9-1b3fad1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:11.053699Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|267d673c-f12cdfc3-6c1514f9-1b3fad1_0] Write session will restart in 0.007252s
2025-10-09T21:48:11.053726Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|267d673c-f12cdfc3-6c1514f9-1b3fad1_0] Write session: Do CDS request
2025-10-09T21:48:11.053733Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|267d673c-f12cdfc3-6c1514f9-1b3fad1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:11.062547Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|267d673c-f12cdfc3-6c1514f9-1b3fad1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:11.062702Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:11.062717Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1142
2025-10-09T21:48:11.062865Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1142 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:11.062898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1142 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:11.062908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1142 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:11.062912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:11.063338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1142 sessionId:  describe result for acl check
2025-10-09T21:48:11.063374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:11.063376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:11.063378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:11.063384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342119534195593:9982] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:11.063946Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342119534195593:9982] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:11.068065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342119534195593:9982] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:11.071126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342119534195593:9982] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:11.071141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342119534195593:9982] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:11.071146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342119534195593:9982] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:11.071154Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1142 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:11.072419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:11.074843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:11.075041Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:11.075053Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:11.075066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:11.075120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:11.075167Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342119172373829:8895]
2025-10-09T21:48:11.075835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:11.076129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:11.076172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:11.076453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:11.076490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.076495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.076496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.076496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.076497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.076498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.076498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.076499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.076499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.076503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.076510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.076512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.076514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.076517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.076519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.076521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.076523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.076525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.076538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.076543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.076547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.076550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.076553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.076556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.076559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.076562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.076564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.076568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.076571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.076574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.076577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.076580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.076583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.076586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.076589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.076592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.076608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:11.083132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:11.084173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:11.084193Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:11.084195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:11.084200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:11.084207Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1141 [2:7559342119172373829:8895]
2025-10-09T21:48:11.084220Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:11.084241Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:11.084249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:11.084253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:11.084255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:11.084273Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:11.084285Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:11.084294Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:11.084347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.084354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.084356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.084359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:11.084427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:11.084434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.084448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.084460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.084479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:11.084504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342119534195552:2432], now have 1 active actors on pipe
2025-10-09T21:48:11.084512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342119534195760:9982], now have 1 active actors on pipe
2025-10-09T21:48:11.084520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342119172373824:8895]
2025-10-09T21:48:11.084522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342119172373824:8895]
2025-10-09T21:48:11.084524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342119172373824:8895]
2025-10-09T21:48:11.084538Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:11.084559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:11.084563Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:11.084568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:11.084570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:11.084579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:11.084582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:11.084575Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1141
2025-10-09T21:48:11.084601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1141
2025-10-09T21:48:11.084616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342119172373824:8895]
2025-10-09T21:48:11.084631Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:11.084638Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:11.084654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.084659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.084713Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8e5f96d7-4d59d1c8-a6c16650-ac39961d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:11.084728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:11.084738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:11.084740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:11.084751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:11.084770Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:11.084790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:11.085038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.085051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.085070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:11.085201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1142 partition: 0 MaxSeqNo: 25 sessionId: src_id|8e5f96d7-4d59d1c8-a6c16650-ac39961d_0
2025-10-09T21:48:11.085560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|267d673c-f12cdfc3-6c1514f9-1b3fad1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8e5f96d7-4d59d1c8-a6c16650-ac39961d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:11.088009Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e5f96d7-4d59d1c8-a6c16650-ac39961d_0] Write session: try to update token
2025-10-09T21:48:11.088030Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e5f96d7-4d59d1c8-a6c16650-ac39961d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:11.099362Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:11.099386Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:11.099389Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:11.099391Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:11.099402Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342119172373824:8895]
2025-10-09T21:48:11.099407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342119172373824:8895]
2025-10-09T21:48:11.099411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342119172373824:8895]
2025-10-09T21:48:11.099459Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:11.099461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:11.099465Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:11.099469Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:11.099469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.099472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:11.099597Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:11.101248Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1142 sessionId: src_id|8e5f96d7-4d59d1c8-a6c16650-ac39961d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:11.103839Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:11.153361Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342119534195778:9750]
2025-10-09T21:48:11.153674Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 1141, pipe: [1:7559342119534195778:9750]
2025-10-09T21:48:11.258053Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:11.258319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.258334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.258377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342119534195778:9750], now have 1 active actors on pipe
2025-10-09T21:48:11.258596Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:11.258591Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:11.258625Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:11.258629Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:11.258649Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342119534195778:9750]
2025-10-09T21:48:11.258705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1142 reason: error at write: Disconnected sessionId: src_id|8e5f96d7-4d59d1c8-a6c16650-ac39961d_0
2025-10-09T21:48:11.258885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1142 sessionId: src_id|8e5f96d7-4d59d1c8-a6c16650-ac39961d_0 is DEAD
2025-10-09T21:48:11.259039Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e5f96d7-4d59d1c8-a6c16650-ac39961d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:11.259051Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e5f96d7-4d59d1c8-a6c16650-ac39961d_0] Write session will restart in 0.006680s
2025-10-09T21:48:11.259089Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e5f96d7-4d59d1c8-a6c16650-ac39961d_0] Write session: Do CDS request
2025-10-09T21:48:11.259096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e5f96d7-4d59d1c8-a6c16650-ac39961d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:11.266539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e5f96d7-4d59d1c8-a6c16650-ac39961d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:11.266623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:11.266640Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1143
2025-10-09T21:48:11.266804Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1143 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:11.266845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1143 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:11.266850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1143 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:11.266854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:11.267520Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1143 sessionId:  describe result for acl check
2025-10-09T21:48:11.267555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:11.267563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:11.267564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:11.267571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342119534195821:9987] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:11.268101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342119534195821:9987] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:11.270684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342119534195821:9987] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:11.273460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342119534195821:9987] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:11.273473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342119534195821:9987] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:11.273477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342119534195821:9987] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:11.273485Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1143 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:11.275286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:11.277377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:11.277565Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:11.277577Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:11.277589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:11.277639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:11.277680Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342119172374047:8897]
2025-10-09T21:48:11.278300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:11.278626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:11.278671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:11.278980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:11.279034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.279042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.279043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.279045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.279046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.279047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.279048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.279050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.279051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.279056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.279068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.279072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.279076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.279081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.279084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.279088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.279092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.279096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.279102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.279110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.279116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.279122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.279127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.279132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.279137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.279142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.279147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.279152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.279156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.279161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.279165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.279171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.279175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.279180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.279185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.279190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.279218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:11.285842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:11.286966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:11.286984Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:11.286987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:11.286991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:11.286997Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1142 [2:7559342119172374047:8897]
2025-10-09T21:48:11.287011Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:11.287030Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:11.287033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:11.287036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:11.287038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:11.287056Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:11.287069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:11.287077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:11.287135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.287142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.287144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.287147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:11.287215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:11.287223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.287235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.287248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.287270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:11.287297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342119534195785:2432], now have 1 active actors on pipe
2025-10-09T21:48:11.287305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342119534195997:9987], now have 1 active actors on pipe
2025-10-09T21:48:11.287313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342119172374041:8897]
2025-10-09T21:48:11.287316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342119172374041:8897]
2025-10-09T21:48:11.287318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342119172374041:8897]
2025-10-09T21:48:11.287333Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:11.287350Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:11.287351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:11.287360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:11.287362Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:11.287366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:11.287369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:11.287405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342119172374041:8897]
2025-10-09T21:48:11.287413Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:11.287415Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:11.287387Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1142
2025-10-09T21:48:11.287397Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1142
2025-10-09T21:48:11.287438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.287449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.287743Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cf1625bf-1e472bdf-ba9f5e98-c9e6791c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:11.287757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:11.287762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:11.287765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:11.287776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:11.287787Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:11.287817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:11.288042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.288052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.288082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:11.288190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1143 partition: 0 MaxSeqNo: 25 sessionId: src_id|cf1625bf-1e472bdf-ba9f5e98-c9e6791c_0
2025-10-09T21:48:11.288581Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e5f96d7-4d59d1c8-a6c16650-ac39961d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cf1625bf-1e472bdf-ba9f5e98-c9e6791c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:11.291088Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf1625bf-1e472bdf-ba9f5e98-c9e6791c_0] Write session: try to update token
2025-10-09T21:48:11.291107Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf1625bf-1e472bdf-ba9f5e98-c9e6791c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:11.301275Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:11.301291Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:11.301293Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:11.301295Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:11.301305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342119172374041:8897]
2025-10-09T21:48:11.301309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342119172374041:8897]
2025-10-09T21:48:11.301313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342119172374041:8897]
2025-10-09T21:48:11.301359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:11.301359Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:11.301365Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:11.301369Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:11.301373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.301376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:11.301508Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:11.304420Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1143 sessionId: src_id|cf1625bf-1e472bdf-ba9f5e98-c9e6791c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:11.306795Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:11.358705Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342119534196012:9750]
2025-10-09T21:48:11.359071Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 1142, pipe: [1:7559342119534196012:9750]
2025-10-09T21:48:11.461002Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:11.461302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.461319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.461367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342119534196012:9750], now have 1 active actors on pipe
2025-10-09T21:48:11.461571Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:11.461618Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:11.461622Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:11.461623Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342119534196012:9750]
2025-10-09T21:48:11.461686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1143 reason: error at write: Disconnected sessionId: src_id|cf1625bf-1e472bdf-ba9f5e98-c9e6791c_0
2025-10-09T21:48:11.461873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1143 sessionId: src_id|cf1625bf-1e472bdf-ba9f5e98-c9e6791c_0 is DEAD
2025-10-09T21:48:11.461582Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:11.462056Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf1625bf-1e472bdf-ba9f5e98-c9e6791c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:11.462065Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf1625bf-1e472bdf-ba9f5e98-c9e6791c_0] Write session will restart in 0.008176s
2025-10-09T21:48:11.462101Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf1625bf-1e472bdf-ba9f5e98-c9e6791c_0] Write session: Do CDS request
2025-10-09T21:48:11.462107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf1625bf-1e472bdf-ba9f5e98-c9e6791c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:11.471546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf1625bf-1e472bdf-ba9f5e98-c9e6791c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:11.471631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:11.471649Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1144
2025-10-09T21:48:11.471884Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1144 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:11.471926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1144 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:11.471936Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1144 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:11.471940Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:11.472382Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1144 sessionId:  describe result for acl check
2025-10-09T21:48:11.472417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:11.472424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:11.472426Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:11.472432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342119534196088:9991] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:11.472952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342119534196088:9991] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:11.475455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342119534196088:9991] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:11.477763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:11.478258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342119534196088:9991] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:11.478275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342119534196088:9991] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:11.478279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342119534196088:9991] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:11.478287Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1144 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:11.480480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:11.480684Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:11.480695Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:11.480707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:11.480773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:11.480821Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342119172374265:8899]
2025-10-09T21:48:11.481387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:11.481784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:11.481844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:11.482183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:11.482247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.482253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.482255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.482256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.482257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.482258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.482259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.482260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.482261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.482266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.482277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.482280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.482284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.482288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.482292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.482295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.482299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.482302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.482310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.482317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.482323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.482329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.482339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.482344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.482348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.482354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.482359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.482365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.482370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.482375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.482385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.482390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.482399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.482403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.482407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.482412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.482438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:11.488608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:11.489617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:11.489634Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:11.489637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:11.489641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:11.489648Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1143 [2:7559342119172374265:8899]
2025-10-09T21:48:11.489661Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:11.489679Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:11.489686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:11.489690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:11.489692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:11.489709Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:11.489720Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:11.489727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:11.489790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.489797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.489799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.489802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:11.490015Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1143
2025-10-09T21:48:11.490027Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1143
2025-10-09T21:48:11.490153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:11.490163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.490176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.490188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.490210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:11.490241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342119534196230:9991], now have 1 active actors on pipe
2025-10-09T21:48:11.490250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342119534196014:2432], now have 1 active actors on pipe
2025-10-09T21:48:11.490268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.490272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.490290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342119172374259:8899]
2025-10-09T21:48:11.490297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342119172374259:8899]
2025-10-09T21:48:11.490299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342119172374259:8899]
2025-10-09T21:48:11.490314Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:11.490335Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:11.490334Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:11.490338Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:11.490340Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:11.490354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:11.490358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:11.490377Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|599b2411-63b1a797-89d017c2-80772944_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:11.490389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:11.490393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:11.490400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:11.490410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:11.490425Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:11.490460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342119172374259:8899]
2025-10-09T21:48:11.490477Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:11.490488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:11.490504Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:11.490710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.490721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.490737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:11.490834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1144 partition: 0 MaxSeqNo: 25 sessionId: src_id|599b2411-63b1a797-89d017c2-80772944_0
2025-10-09T21:48:11.491224Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf1625bf-1e472bdf-ba9f5e98-c9e6791c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|599b2411-63b1a797-89d017c2-80772944_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:11.493856Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|599b2411-63b1a797-89d017c2-80772944_0] Write session: try to update token
2025-10-09T21:48:11.493874Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|599b2411-63b1a797-89d017c2-80772944_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:11.507007Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1144 sessionId: src_id|599b2411-63b1a797-89d017c2-80772944_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:11.507900Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:11.507921Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:11.507932Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342119172374259:8899]
2025-10-09T21:48:11.507987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:11.507998Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:11.508171Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:11.508322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:11.508464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:11.508605Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:11.508797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:11.508849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:11.509521Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:11.509727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.509744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.509789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:11.509884Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:11.514541Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:11.514556Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:11.514559Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:11.514561Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:11.514570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342119172374259:8899]
2025-10-09T21:48:11.514575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342119172374259:8899]
2025-10-09T21:48:11.514578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342119172374259:8899]
2025-10-09T21:48:11.514617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:11.514624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.514625Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:11.514626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:11.514631Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:11.514636Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:11.514752Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:11.562205Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342119534196242:9750]
2025-10-09T21:48:11.562473Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 1143, pipe: [1:7559342119534196242:9750]
2025-10-09T21:48:11.671347Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:11.679085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.679119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.679264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:11.679321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:11.679374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:11.679432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:11.679470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:11.679474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:11.679541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:11.679599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:11.679656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:11.679712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:11.679749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:11.679752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:11.679818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:11.679869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:11.679928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:11.679984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:11.680032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:11.680035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:11.680102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:11.680158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:11.680215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:11.680272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:11.680313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:11.680316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:11.680384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:11.680441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:11.680500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:11.680551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:11.680584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:11.680587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:11.680654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:11.680705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:11.680763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:11.680823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:11.680864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:11.680866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:11.680925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:11.682149Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:11.680974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:11.681027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:11.682158Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342119534196242:9750]
2025-10-09T21:48:11.681082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:11.681120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:11.681122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:11.681179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:11.681228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:11.681282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:11.681333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:11.681370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:11.682163Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:11.681373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:11.681431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:11.682196Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:11.681478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:11.681531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:11.681591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:11.681630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:11.681632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:11.681690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:11.681741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:11.681810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:11.681868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:11.682201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1144 reason: error at write: Disconnected sessionId: src_id|599b2411-63b1a797-89d017c2-80772944_0
2025-10-09T21:48:11.681904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:11.681907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:11.681962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342119534196242:9750], now have 1 active actors on pipe
2025-10-09T21:48:11.682391Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1144 sessionId: src_id|599b2411-63b1a797-89d017c2-80772944_0 is DEAD
2025-10-09T21:48:11.682544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|599b2411-63b1a797-89d017c2-80772944_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:11.682554Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|599b2411-63b1a797-89d017c2-80772944_0] Write session will restart in 0.007553s
2025-10-09T21:48:11.682579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|599b2411-63b1a797-89d017c2-80772944_0] Write session: Do CDS request
2025-10-09T21:48:11.682594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|599b2411-63b1a797-89d017c2-80772944_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:11.682948Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:11.691539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|599b2411-63b1a797-89d017c2-80772944_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:11.691645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:11.691669Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1145
2025-10-09T21:48:11.691794Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1145 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:11.691827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1145 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:11.691833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1145 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:11.691837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:11.692440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1145 sessionId:  describe result for acl check
2025-10-09T21:48:11.692487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:11.692493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:11.692495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:11.692501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342119534196276:9995] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:11.693025Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342119534196276:9995] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:11.696970Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342119534196276:9995] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:11.700031Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342119534196276:9995] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:11.700059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342119534196276:9995] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:11.700063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342119534196276:9995] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:11.700070Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1145 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:11.700674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:11.702998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:11.703203Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:11.703218Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:11.703230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:11.703295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:11.703338Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342119172374519:8913]
2025-10-09T21:48:11.703912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:11.704241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:11.704289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:11.704590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:11.704640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.704645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.704646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.704648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.704649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.704650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.704651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.704652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.704653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.704658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.704668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.704672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.704675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.704679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.704683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.704687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.704690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.704694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.704700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.704708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.704718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.704723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.704732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.704737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.704741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.704745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.704750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.704754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.704757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.704762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.704766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.704771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.704774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.704779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.704783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.704787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.704811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:11.711123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:11.712263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:11.712284Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:11.712286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:11.712290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:11.712297Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1144 [2:7559342119172374519:8913]
2025-10-09T21:48:11.712310Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:11.712329Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:11.712338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:11.712342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:11.712344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:11.712360Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:11.712366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:11.712372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:11.712427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.712436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.712438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.712441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:11.712508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:11.712516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.712528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.712543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.712563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:11.712590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342119534196248:2432], now have 1 active actors on pipe
2025-10-09T21:48:11.712599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342119534196460:9995], now have 1 active actors on pipe
2025-10-09T21:48:11.712607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342119172374514:8913]
2025-10-09T21:48:11.712609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342119172374514:8913]
2025-10-09T21:48:11.712612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342119172374514:8913]
2025-10-09T21:48:11.712624Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:11.712639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:11.712647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:11.712655Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:11.712657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:11.712659Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:11.712661Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:11.712661Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1144
2025-10-09T21:48:11.712687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342119172374514:8913]
2025-10-09T21:48:11.712670Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1144
2025-10-09T21:48:11.712695Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:11.712706Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:11.712731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.712741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.712775Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d9332728-a730981a-237e3f67-81978865_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:11.712786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:11.712790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:11.712793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:11.712803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:11.712813Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:11.712826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:11.712973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.712981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.712994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:11.713091Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1145 partition: 0 MaxSeqNo: 25 sessionId: src_id|d9332728-a730981a-237e3f67-81978865_0
2025-10-09T21:48:11.713500Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|599b2411-63b1a797-89d017c2-80772944_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d9332728-a730981a-237e3f67-81978865_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:11.715930Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d9332728-a730981a-237e3f67-81978865_0] Write session: try to update token
2025-10-09T21:48:11.715949Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d9332728-a730981a-237e3f67-81978865_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:11.726860Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:11.726881Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:11.726884Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:11.726886Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:11.726895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342119172374514:8913]
2025-10-09T21:48:11.726900Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342119172374514:8913]
2025-10-09T21:48:11.726903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342119172374514:8913]
2025-10-09T21:48:11.726951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:11.726952Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:11.726958Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:11.726962Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:11.726965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.726969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:11.727100Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:11.730659Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1145 sessionId: src_id|d9332728-a730981a-237e3f67-81978865_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:11.733245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:11.782251Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342119534196488:9750]
2025-10-09T21:48:11.782537Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 1144, pipe: [1:7559342119534196488:9750]
2025-10-09T21:48:11.890737Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:11.891032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.891049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.891099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342119534196488:9750], now have 1 active actors on pipe
2025-10-09T21:48:11.891315Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:11.891467Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:11.891527Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342119534196488:9750]
2025-10-09T21:48:11.891548Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:11.891988Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9332728-a730981a-237e3f67-81978865_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:11.892000Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9332728-a730981a-237e3f67-81978865_0] Write session will restart in 0.008889s
2025-10-09T21:48:11.891636Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1145 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d9332728-a730981a-237e3f67-81978865_0
2025-10-09T21:48:11.892025Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9332728-a730981a-237e3f67-81978865_0] Write session: Do CDS request
2025-10-09T21:48:11.892034Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9332728-a730981a-237e3f67-81978865_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:11.891827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1145 sessionId: src_id|d9332728-a730981a-237e3f67-81978865_0 is DEAD
2025-10-09T21:48:11.902524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d9332728-a730981a-237e3f67-81978865_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:11.902662Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:11.902680Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1146
2025-10-09T21:48:11.902802Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1146 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:11.902838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1146 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:11.902848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1146 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:11.902853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:11.903582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1146 sessionId:  describe result for acl check
2025-10-09T21:48:11.903625Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:11.903634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:11.903636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:11.903643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342119534196547:10009] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:11.904220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342119534196547:10009] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:11.906986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342119534196547:10009] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:11.910506Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342119534196547:10009] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:11.910524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342119534196547:10009] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:11.910527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342119534196547:10009] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:11.910548Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1146 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:11.911884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:11.914588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:11.914807Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:11.914819Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:11.914832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:11.914891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:11.914938Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342119172374748:8917]
2025-10-09T21:48:11.915518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:11.915867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:11.915915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:11.916252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:11.916301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.916307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.916308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.916309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.916310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.916312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.916313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.916314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.916316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.916321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.916333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.916337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.916342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.916345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.916350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.916354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.916357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.916361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.916369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.916377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:11.916385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.916391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:11.916395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.916400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:11.916405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.916410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:11.916415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.916420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:11.916424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.916429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:11.916435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.916440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.916444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.916449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.916453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.916458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.916488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:11.924064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:11.925228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:11.925250Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:11.925253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:11.925258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:11.925267Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1145 [2:7559342119172374748:8917]
2025-10-09T21:48:11.925283Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:11.925306Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:11.925310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:11.925314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:11.925316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:11.925337Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:11.925352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:11.925364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:11.925769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.925783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.925785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.925788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:11.925872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:11.925880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.925895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:11.925907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:11.925925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:11.925961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342119534196504:2432], now have 1 active actors on pipe
2025-10-09T21:48:11.925966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342119534196716:10009], now have 1 active actors on pipe
2025-10-09T21:48:11.925974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342119172374743:8917]
2025-10-09T21:48:11.925977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342119172374743:8917]
2025-10-09T21:48:11.925979Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342119172374743:8917]
2025-10-09T21:48:11.925995Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:11.926019Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:11.926058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:11.926063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:11.926066Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:11.926067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:11.926068Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:11.926071Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1145
2025-10-09T21:48:11.926111Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342119172374743:8917]
2025-10-09T21:48:11.926123Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:11.926129Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:11.926094Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1145
2025-10-09T21:48:11.926248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.926265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.926298Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d6a4dde7-3e9bb0f4-1c91eac0-12c70b9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:11.926311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:11.926317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:11.926320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:11.926332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:11.926350Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:11.926370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:11.926628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.926640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.926660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:11.926797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1146 partition: 0 MaxSeqNo: 25 sessionId: src_id|d6a4dde7-3e9bb0f4-1c91eac0-12c70b9_0
2025-10-09T21:48:11.927203Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9332728-a730981a-237e3f67-81978865_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d6a4dde7-3e9bb0f4-1c91eac0-12c70b9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:11.929698Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d6a4dde7-3e9bb0f4-1c91eac0-12c70b9_0] Write session: try to update token
2025-10-09T21:48:11.929717Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d6a4dde7-3e9bb0f4-1c91eac0-12c70b9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:11.938310Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:11.938329Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:11.938339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342119172374743:8917]
2025-10-09T21:48:11.938382Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:11.938393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:11.938579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:11.938745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:11.938923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:11.939072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:11.939277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:11.939326Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:11.946439Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1146 sessionId: src_id|d6a4dde7-3e9bb0f4-1c91eac0-12c70b9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:11.948990Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:11.949289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:11.949315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:11.949358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:11.949456Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:11.951981Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:11.952001Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:11.952004Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:11.952006Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:11.952017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342119172374743:8917]
2025-10-09T21:48:11.952020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342119172374743:8917]
2025-10-09T21:48:11.952024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342119172374743:8917]
2025-10-09T21:48:11.952071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:11.952083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:11.952086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:11.952110Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:11.952124Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:11.952127Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:11.952231Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:11.991689Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342119534196733:9750]
2025-10-09T21:48:11.992112Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 1145, pipe: [1:7559342119534196733:9750]
2025-10-09T21:48:12.117224Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:12.125529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:12.125567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:12.125761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.125824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.125887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.125957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.126009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.126021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.126109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.126173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.126238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.126310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.126358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.126364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.126438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.126502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.126557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.126627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.126674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.126677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.126747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.126817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.126887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.126955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.127002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.127010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.127078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.127141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.127210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.127278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.127322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.127331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.127395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.127460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.127518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.127582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.127692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.127700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.127764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.127821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.127887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.127950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.127990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.127999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.128060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.128117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.128184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.128249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.128299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.128309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.128380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.128441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.128513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.128577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.128627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.128635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.128703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.128765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.128832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.128898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.128948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.128955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.129026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342119534196733:9750], now have 1 active actors on pipe
2025-10-09T21:48:12.129276Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:12.129275Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342119534196733:9750]
2025-10-09T21:48:12.129320Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:12.129323Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:12.129358Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1146 reason: error at write: Disconnected sessionId: src_id|d6a4dde7-3e9bb0f4-1c91eac0-12c70b9_0
2025-10-09T21:48:12.129580Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1146 sessionId: src_id|d6a4dde7-3e9bb0f4-1c91eac0-12c70b9_0 is DEAD
2025-10-09T21:48:12.129745Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6a4dde7-3e9bb0f4-1c91eac0-12c70b9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:12.129767Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6a4dde7-3e9bb0f4-1c91eac0-12c70b9_0] Write session will restart in 0.006374s
2025-10-09T21:48:12.129803Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6a4dde7-3e9bb0f4-1c91eac0-12c70b9_0] Write session: Do CDS request
2025-10-09T21:48:12.129811Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6a4dde7-3e9bb0f4-1c91eac0-12c70b9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:12.130290Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:12.137610Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d6a4dde7-3e9bb0f4-1c91eac0-12c70b9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:12.137757Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:12.137779Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1147
2025-10-09T21:48:12.137908Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1147 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:12.137952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1147 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:12.137964Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1147 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:12.137970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:12.138430Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1147 sessionId:  describe result for acl check
2025-10-09T21:48:12.138471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:12.138473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:12.138475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:12.138481Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342123829164050:10013] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:12.139088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342123829164050:10013] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:12.142747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342123829164050:10013] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:12.147322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342123829164050:10013] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:12.147340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342123829164050:10013] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:12.147345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342123829164050:10013] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:12.147354Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1147 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:12.151563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:12.154745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:12.155009Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:12.155021Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:12.155035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:12.155097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:12.155142Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342123467342266:8919]
2025-10-09T21:48:12.155773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:12.156186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:12.156262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:12.156611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:12.156688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:12.156698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:12.156699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:12.156700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:12.156701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:12.156702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:12.156703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.156704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.156705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.156711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:12.156722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:12.156726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:12.156730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:12.156734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:12.156738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:12.156742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.156746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.156749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.156757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:12.156764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:12.156770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:12.156775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:12.156779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:12.156783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:12.156787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:12.156792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:12.156795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:12.156800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:12.156804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:12.156808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:12.156812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.156816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.156820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.156825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.156829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.156833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.156859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:12.164486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:12.165949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:12.165968Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:12.165972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:12.165976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:12.165984Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1146 [2:7559342123467342266:8919]
2025-10-09T21:48:12.166002Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:12.166023Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:12.166046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:12.166051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:12.166059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:12.166081Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:12.166093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:12.166102Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:12.166177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.166186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.166188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.166191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:12.166272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:12.166282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.166297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.166309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.166326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:12.166349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342123829164032:2432], now have 1 active actors on pipe
2025-10-09T21:48:12.166358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342123829164215:10013], now have 1 active actors on pipe
2025-10-09T21:48:12.166367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342123467342261:8919]
2025-10-09T21:48:12.166369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342123467342261:8919]
2025-10-09T21:48:12.166371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342123467342261:8919]
2025-10-09T21:48:12.166387Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:12.166402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:12.166417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:12.166425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:12.166453Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1146
2025-10-09T21:48:12.166474Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1146
2025-10-09T21:48:12.166471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342123467342261:8919]
2025-10-09T21:48:12.166480Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:12.166484Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:12.166488Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:12.166491Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:12.166499Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:12.166536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:12.166541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:12.166571Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1628fcba-390a0987-43d87f9a-41296d3e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:12.166587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:12.166591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:12.166594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:12.166607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:12.166617Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:12.166635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:12.167146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:12.167162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:12.167186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:12.167313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1147 partition: 0 MaxSeqNo: 25 sessionId: src_id|1628fcba-390a0987-43d87f9a-41296d3e_0
2025-10-09T21:48:12.167721Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d6a4dde7-3e9bb0f4-1c91eac0-12c70b9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1628fcba-390a0987-43d87f9a-41296d3e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:12.170562Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1628fcba-390a0987-43d87f9a-41296d3e_0] Write session: try to update token
2025-10-09T21:48:12.170585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1628fcba-390a0987-43d87f9a-41296d3e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:12.178927Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:12.178950Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:12.178962Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342123467342261:8919]
2025-10-09T21:48:12.179018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:12.179206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:12.179363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:12.179513Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:12.179667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:12.179698Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:12.179880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:12.179933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:12.187068Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1147 sessionId: src_id|1628fcba-390a0987-43d87f9a-41296d3e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:12.189729Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:12.189961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:12.189984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:12.190027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:12.190127Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:12.193584Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:12.193609Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:12.193612Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:12.193615Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:12.193628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342123467342261:8919]
2025-10-09T21:48:12.193633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342123467342261:8919]
2025-10-09T21:48:12.193639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342123467342261:8919]
2025-10-09T21:48:12.193691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:12.193695Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:12.193706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.193709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:12.193715Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:12.193720Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:12.193919Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:12.229615Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342123829164271:9750]
2025-10-09T21:48:12.229978Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 1146, pipe: [1:7559342123829164271:9750]
2025-10-09T21:48:12.355802Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:12.363973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:12.364005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:12.364172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.364229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.364288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.364347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.364394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.364407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.364468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.364520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.364580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.364645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.364686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.364693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.364754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.364808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.364868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.364939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.364982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.364989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.365052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.365119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.365181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.365242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.365287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.365295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.365358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.365421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.365486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.365555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.365602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.365610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.365681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.365742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.365814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.365871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.365911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.365923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.365986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.366066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.366131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.366192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.366241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.366250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.366311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.366369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.366430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.366491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.366536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.366545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.366606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.366667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.366729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.366793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.366836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.366844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.366908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:12.366971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:12.367036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:12.367458Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:12.367471Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342123829164271:9750]
2025-10-09T21:48:12.367510Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:12.367512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:12.367546Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1147 reason: error at write: Disconnected sessionId: src_id|1628fcba-390a0987-43d87f9a-41296d3e_0
2025-10-09T21:48:12.367748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1147 sessionId: src_id|1628fcba-390a0987-43d87f9a-41296d3e_0 is DEAD
2025-10-09T21:48:12.367943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1628fcba-390a0987-43d87f9a-41296d3e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:12.367954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1628fcba-390a0987-43d87f9a-41296d3e_0] Write session will restart in 0.007788s
2025-10-09T21:48:12.367981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1628fcba-390a0987-43d87f9a-41296d3e_0] Write session: Do CDS request
2025-10-09T21:48:12.367988Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1628fcba-390a0987-43d87f9a-41296d3e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:12.367095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:12.367147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:12.367150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:12.367220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342123829164271:9750], now have 1 active actors on pipe
2025-10-09T21:48:12.368226Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:12.376666Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1628fcba-390a0987-43d87f9a-41296d3e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:12.376795Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:12.376815Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1148
2025-10-09T21:48:12.376961Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1148 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:12.377017Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1148 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:12.377027Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1148 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:12.377034Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:12.379485Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1148 sessionId:  describe result for acl check
2025-10-09T21:48:12.379530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:12.379537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:12.379539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:12.379546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342123829164298:10018] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:12.380139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342123829164298:10018] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:12.387736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342123829164298:10018] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:12.392694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342123829164298:10018] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:12.392711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342123829164298:10018] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:12.392714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342123829164298:10018] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:12.392725Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1148 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:12.392828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:12.396430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:12.396648Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:12.396664Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:12.396677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:12.396736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:12.396779Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342123467342488:8921]
2025-10-09T21:48:12.397376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:12.398001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:12.398086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:12.398419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:12.398483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:12.398489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:12.398491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:12.398492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:12.398494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:12.398495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:12.398496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.398498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.398499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.398505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:12.398517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:12.398521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:12.398526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:12.398530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:12.398534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:12.398538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.398542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.398546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.398554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:12.398562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:12.398569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:12.398574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:12.398579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:12.398585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:12.398595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:12.398600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:12.398609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:12.398615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:12.398625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:12.398630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:12.398635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.398644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.398648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.398653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.398663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.398669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.398703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:12.405660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:12.406994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:12.407014Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:12.407016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:12.407021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:12.407030Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1147 [2:7559342123467342488:8921]
2025-10-09T21:48:12.407045Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:12.407064Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:12.407076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:12.407079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:12.407081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:12.407101Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:12.407108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:12.407115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:12.407174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.407182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.407184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.407187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:12.407263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:12.407275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.407287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.407299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.407317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:12.407341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342123829164274:2432], now have 1 active actors on pipe
2025-10-09T21:48:12.407351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342123829164492:10018], now have 1 active actors on pipe
2025-10-09T21:48:12.407359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342123467342482:8921]
2025-10-09T21:48:12.407361Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342123467342482:8921]
2025-10-09T21:48:12.407363Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342123467342482:8921]
2025-10-09T21:48:12.407379Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:12.407401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:12.407458Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1147
2025-10-09T21:48:12.407404Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:12.407408Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:12.407410Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:12.407505Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1147
2025-10-09T21:48:12.407416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:12.407419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:12.407459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342123467342482:8921]
2025-10-09T21:48:12.407471Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:12.407509Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:12.407660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:12.407672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:12.407702Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|951e8d2-2e9da2d2-9ba591e1-d71a4e66_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:12.407717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:12.407722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:12.407724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:12.407736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:12.407748Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:12.407763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:12.408357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:12.408371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:12.408392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:12.408512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1148 partition: 0 MaxSeqNo: 25 sessionId: src_id|951e8d2-2e9da2d2-9ba591e1-d71a4e66_0
2025-10-09T21:48:12.408896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1628fcba-390a0987-43d87f9a-41296d3e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|951e8d2-2e9da2d2-9ba591e1-d71a4e66_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:12.411399Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|951e8d2-2e9da2d2-9ba591e1-d71a4e66_0] Write session: try to update token
2025-10-09T21:48:12.411420Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|951e8d2-2e9da2d2-9ba591e1-d71a4e66_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:12.422417Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:12.422453Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:12.422457Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:12.422458Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:12.422470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342123467342482:8921]
2025-10-09T21:48:12.422478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342123467342482:8921]
2025-10-09T21:48:12.422484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342123467342482:8921]
2025-10-09T21:48:12.422547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:12.422551Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:12.422557Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:12.422561Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:12.422568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.422572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:12.422736Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:12.426391Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1148 sessionId: src_id|951e8d2-2e9da2d2-9ba591e1-d71a4e66_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:12.428818Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:12.467742Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342123829164505:9750]
2025-10-09T21:48:12.468017Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 1147, pipe: [1:7559342123829164505:9750]
2025-10-09T21:48:12.593410Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:12.593939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:12.593961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:12.594009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342123829164505:9750], now have 1 active actors on pipe
2025-10-09T21:48:12.594367Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:12.594454Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342123829164505:9750]
2025-10-09T21:48:12.594492Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:12.594562Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1148 reason: status is not ok: tablet will be restarted right now sessionId: src_id|951e8d2-2e9da2d2-9ba591e1-d71a4e66_0
2025-10-09T21:48:12.594641Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:12.594761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1148 sessionId: src_id|951e8d2-2e9da2d2-9ba591e1-d71a4e66_0 is DEAD
2025-10-09T21:48:12.594989Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|951e8d2-2e9da2d2-9ba591e1-d71a4e66_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:12.595002Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|951e8d2-2e9da2d2-9ba591e1-d71a4e66_0] Write session will restart in 0.006111s
2025-10-09T21:48:12.595030Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|951e8d2-2e9da2d2-9ba591e1-d71a4e66_0] Write session: Do CDS request
2025-10-09T21:48:12.595037Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|951e8d2-2e9da2d2-9ba591e1-d71a4e66_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:12.602545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|951e8d2-2e9da2d2-9ba591e1-d71a4e66_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:12.602611Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:12.602627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1149
2025-10-09T21:48:12.604257Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1149 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:12.604316Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1149 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:12.604322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1149 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:12.604326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:12.604903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1149 sessionId:  describe result for acl check
2025-10-09T21:48:12.604952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:12.604956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:12.604958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:12.604964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342123829164535:10022] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:12.605554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342123829164535:10022] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:12.608319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342123829164535:10022] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:12.612403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342123829164535:10022] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:12.612423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342123829164535:10022] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:12.612427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342123829164535:10022] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:12.612437Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1149 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:12.616251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:12.619542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:12.619781Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:12.619795Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:12.619810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:12.619885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:12.619930Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342123467342712:8923]
2025-10-09T21:48:12.620524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:12.620879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:12.620947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:12.621265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:12.621307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:12.621308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:12.621309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:12.621309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:12.621310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:12.621311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:12.621311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.621312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.621313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.621316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:12.621323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:12.621326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:12.621328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:12.621330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:12.621332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:12.621334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.621336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.621338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.621344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:12.621349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:12.621354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:12.621358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:12.621361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:12.621364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:12.621366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:12.621370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:12.621373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:12.621378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:12.621382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:12.621387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:12.621393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.621398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.621403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.621411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.621414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.621417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.621443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:12.628346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:12.629423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:12.629440Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:12.629443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:12.629447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:12.629455Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1148 [2:7559342123467342712:8923]
2025-10-09T21:48:12.629470Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:12.629491Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:12.629494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:12.629498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:12.629500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:12.629519Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:12.629530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:12.629543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:12.629609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.629618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.629621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.629624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:12.629698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:12.629707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.629720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.629737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.629769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:12.630143Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1148
2025-10-09T21:48:12.629794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342123467342706:8923]
2025-10-09T21:48:12.629803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342123467342706:8923]
2025-10-09T21:48:12.629806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342123467342706:8923]
2025-10-09T21:48:12.630146Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1148
2025-10-09T21:48:12.629822Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:12.629840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:12.629856Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:12.629860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:12.629880Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:12.629883Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:12.629887Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:12.629899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342123467342706:8923]
2025-10-09T21:48:12.629902Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:12.629981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:12.630069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342123829164508:2432], now have 1 active actors on pipe
2025-10-09T21:48:12.630078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342123829164709:10022], now have 1 active actors on pipe
2025-10-09T21:48:12.630316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:12.630331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:12.630367Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|da3d615b-d2b43be1-73a49219-813bbc31_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:12.630382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:12.630387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:12.630390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:12.630405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:12.630420Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:12.630438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:12.630737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:12.630746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:12.630761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:12.630921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1149 partition: 0 MaxSeqNo: 25 sessionId: src_id|da3d615b-d2b43be1-73a49219-813bbc31_0
2025-10-09T21:48:12.631301Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|951e8d2-2e9da2d2-9ba591e1-d71a4e66_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|da3d615b-d2b43be1-73a49219-813bbc31_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:12.633774Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da3d615b-d2b43be1-73a49219-813bbc31_0] Write session: try to update token
2025-10-09T21:48:12.633792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da3d615b-d2b43be1-73a49219-813bbc31_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:12.647554Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:12.647574Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:12.647577Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:12.647579Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:12.647590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342123467342706:8923]
2025-10-09T21:48:12.647596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342123467342706:8923]
2025-10-09T21:48:12.647600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342123467342706:8923]
2025-10-09T21:48:12.647651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:12.647664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.647667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:12.647819Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:12.648056Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:12.648065Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:12.648072Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:12.651877Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1149 sessionId: src_id|da3d615b-d2b43be1-73a49219-813bbc31_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:12.654518Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:12.695677Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342123829164741:9750]
2025-10-09T21:48:12.696021Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 1148, pipe: [1:7559342123829164741:9750]
2025-10-09T21:48:12.813550Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:12.814298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:12.814318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:12.814365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342123829164741:9750], now have 1 active actors on pipe
2025-10-09T21:48:12.814605Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:12.814653Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342123829164741:9750]
2025-10-09T21:48:12.814684Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:12.814726Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:12.814729Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:12.814794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1149 reason: error at write: Disconnected sessionId: src_id|da3d615b-d2b43be1-73a49219-813bbc31_0
2025-10-09T21:48:12.814964Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1149 sessionId: src_id|da3d615b-d2b43be1-73a49219-813bbc31_0 is DEAD
2025-10-09T21:48:12.815147Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da3d615b-d2b43be1-73a49219-813bbc31_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:12.815160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da3d615b-d2b43be1-73a49219-813bbc31_0] Write session will restart in 0.007465s
2025-10-09T21:48:12.815188Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da3d615b-d2b43be1-73a49219-813bbc31_0] Write session: Do CDS request
2025-10-09T21:48:12.815195Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da3d615b-d2b43be1-73a49219-813bbc31_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:12.823778Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da3d615b-d2b43be1-73a49219-813bbc31_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:12.823792Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:12.823810Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1150
2025-10-09T21:48:12.824029Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1150 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:12.824062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1150 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:12.824068Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1150 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:12.824072Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:12.825736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1150 sessionId:  describe result for acl check
2025-10-09T21:48:12.825794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:12.825796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:12.825798Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:12.825804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342123829164793:10033] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:12.826440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342123829164793:10033] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:12.841093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:12.842993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342123829164793:10033] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:12.846601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:12.846830Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:12.846844Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:12.846857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:12.846911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:12.846952Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342123467342969:8937]
2025-10-09T21:48:12.847539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:12.847958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:12.848006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:12.848335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:12.848386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:12.848392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:12.848393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:12.848395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:12.848396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:12.848397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:12.848398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.848399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.848400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.848405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:12.848414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:12.848418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:12.848422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:12.848425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:12.848429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:12.848432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.848436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.848439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.848446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:12.848454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:12.848460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:12.848465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:12.848469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:12.848474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:12.848478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:12.848482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:12.848487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:12.848491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:12.848497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:12.848501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:12.848508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.848512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.848516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.848520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.848524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.848529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.848557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:12.854614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342123829164793:10033] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:12.854632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342123829164793:10033] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:12.854636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342123829164793:10033] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:12.854647Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1150 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:12.856191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:12.857326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:12.857341Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:12.857344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:12.857349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:12.857355Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1149 [2:7559342123467342969:8937]
2025-10-09T21:48:12.857369Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:12.857390Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:12.857398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:12.857401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:12.857404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:12.857421Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:12.857431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:12.857439Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:12.857499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.857506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.857508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.857511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:12.857580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:12.857590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.857603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:12.857616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:12.857633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:12.857656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342123829164747:2432], now have 1 active actors on pipe
2025-10-09T21:48:12.857666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342123829164983:10033], now have 1 active actors on pipe
2025-10-09T21:48:12.857674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342123467342964:8937]
2025-10-09T21:48:12.857676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342123467342964:8937]
2025-10-09T21:48:12.857678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342123467342964:8937]
2025-10-09T21:48:12.857691Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:12.857707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:12.857722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:12.857730Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:12.857759Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:12.857774Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:12.857777Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:12.857784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342123467342964:8937]
2025-10-09T21:48:12.857788Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:12.857791Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:12.857852Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1149
2025-10-09T21:48:12.857893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1149
2025-10-09T21:48:12.858136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:12.858149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:12.858188Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a7e3314-3338ab10-9dccca89-ca01ba64_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:12.858201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:12.858205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:12.858207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:12.858217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:12.858228Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:12.858244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:12.858434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:12.858447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:12.858469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:12.858574Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1150 partition: 0 MaxSeqNo: 25 sessionId: src_id|a7e3314-3338ab10-9dccca89-ca01ba64_0
2025-10-09T21:48:12.858967Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da3d615b-d2b43be1-73a49219-813bbc31_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a7e3314-3338ab10-9dccca89-ca01ba64_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:12.861590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7e3314-3338ab10-9dccca89-ca01ba64_0] Write session: try to update token
2025-10-09T21:48:12.861608Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7e3314-3338ab10-9dccca89-ca01ba64_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:12.873428Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:12.873447Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:12.873450Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:12.873452Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:12.873461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342123467342964:8937]
2025-10-09T21:48:12.873469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342123467342964:8937]
2025-10-09T21:48:12.873473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342123467342964:8937]
2025-10-09T21:48:12.873518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:12.873526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:12.873534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:12.873533Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:12.873539Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:12.873543Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:12.873657Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:12.876950Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1150 sessionId: src_id|a7e3314-3338ab10-9dccca89-ca01ba64_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:12.879545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:12.914712Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342123829165003:9750]
2025-10-09T21:48:12.915041Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 1149, pipe: [1:7559342123829165003:9750]
2025-10-09T21:48:13.035511Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:13.036000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.036011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.036059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342123829165003:9750], now have 1 active actors on pipe
2025-10-09T21:48:13.036400Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:13.036411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:13.036410Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:13.036426Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:13.036477Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342123829165003:9750]
2025-10-09T21:48:13.036501Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1150 reason: error at write: Disconnected sessionId: src_id|a7e3314-3338ab10-9dccca89-ca01ba64_0
2025-10-09T21:48:13.036705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1150 sessionId: src_id|a7e3314-3338ab10-9dccca89-ca01ba64_0 is DEAD
2025-10-09T21:48:13.036855Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7e3314-3338ab10-9dccca89-ca01ba64_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:13.036868Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7e3314-3338ab10-9dccca89-ca01ba64_0] Write session will restart in 0.009950s
2025-10-09T21:48:13.036897Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7e3314-3338ab10-9dccca89-ca01ba64_0] Write session: Do CDS request
2025-10-09T21:48:13.036913Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7e3314-3338ab10-9dccca89-ca01ba64_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:13.047676Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7e3314-3338ab10-9dccca89-ca01ba64_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:13.047793Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:13.047812Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1151
2025-10-09T21:48:13.048063Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1151 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:13.048109Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1151 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:13.048122Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1151 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:13.048126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:13.049144Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1151 sessionId:  describe result for acl check
2025-10-09T21:48:13.049192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:13.049197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:13.049199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:13.049205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342128124132361:10038] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:13.049758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342128124132361:10038] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:13.052605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342128124132361:10038] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:13.056373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:13.060585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342128124132361:10038] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:13.060604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342128124132361:10038] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:13.060608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342128124132361:10038] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:13.060619Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1151 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:13.059888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:13.060181Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:13.060186Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:13.060198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:13.060260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:13.060298Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342127762310498:8941]
2025-10-09T21:48:13.060905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:13.061357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:13.061433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:13.061817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:13.061887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.061889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.061890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.061892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.061893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.061894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.061896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.061897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.061898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.061904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.061916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.061920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.061925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.061928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.061932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.061936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.061940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.061943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.061950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.061958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.061965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.061970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.061974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.061982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.061990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.061996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.062001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.062006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.062011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.062017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.062022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.062047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.062052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.062059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.062071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.062077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.062111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:13.070619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:13.071784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:13.071805Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:13.071808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:13.071813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:13.071821Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1150 [2:7559342127762310498:8941]
2025-10-09T21:48:13.071839Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:13.071860Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:13.071863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:13.071867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:13.071870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:13.071889Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:13.071905Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:13.071914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:13.071984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.071991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.071995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.071999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:13.072078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:13.072089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.072107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.072118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.072136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:13.072159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342128124132302:2432], now have 1 active actors on pipe
2025-10-09T21:48:13.072163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342128124132526:10038], now have 1 active actors on pipe
2025-10-09T21:48:13.072171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342127762310493:8941]
2025-10-09T21:48:13.072174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342127762310493:8941]
2025-10-09T21:48:13.072176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342127762310493:8941]
2025-10-09T21:48:13.072191Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:13.072208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:13.072212Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:13.072224Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:13.072217Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1150
2025-10-09T21:48:13.072226Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:13.072229Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:13.072236Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:13.072274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342127762310493:8941]
2025-10-09T21:48:13.072272Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1150
2025-10-09T21:48:13.072287Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:13.072304Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:13.072334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.072346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.072376Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|28f2ddbf-5bcb4319-6fb57467-c2534948_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:13.072385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:13.072390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:13.072393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:13.072404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:13.072415Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:13.072440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:13.072627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.072637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.072654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:13.072792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1151 partition: 0 MaxSeqNo: 25 sessionId: src_id|28f2ddbf-5bcb4319-6fb57467-c2534948_0
2025-10-09T21:48:13.074355Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7e3314-3338ab10-9dccca89-ca01ba64_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|28f2ddbf-5bcb4319-6fb57467-c2534948_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:13.076861Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28f2ddbf-5bcb4319-6fb57467-c2534948_0] Write session: try to update token
2025-10-09T21:48:13.076880Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28f2ddbf-5bcb4319-6fb57467-c2534948_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:13.085743Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:13.085780Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:13.085791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342127762310493:8941]
2025-10-09T21:48:13.085843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:13.086022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:13.086201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:13.086336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:13.086466Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:13.086647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:13.086682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:13.086866Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:13.095918Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1151 sessionId: src_id|28f2ddbf-5bcb4319-6fb57467-c2534948_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:13.098415Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:13.098966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.098990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.100348Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:13.100367Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:13.100369Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:13.100371Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:13.100381Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342127762310493:8941]
2025-10-09T21:48:13.100385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342127762310493:8941]
2025-10-09T21:48:13.100388Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342127762310493:8941]
2025-10-09T21:48:13.100562Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:13.100431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:13.100434Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:13.100442Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:13.100446Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:13.100455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:13.100463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.100466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:13.100595Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:13.136905Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342128124132541:9750]
2025-10-09T21:48:13.137281Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 1150, pipe: [1:7559342128124132541:9750]
2025-10-09T21:48:13.264216Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:13.272292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.272325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.272496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.272552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.272616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.272681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.272733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.272739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.272805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.272859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.272916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.272976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.273012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.273018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.273077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.273126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.273179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.273235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.273271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.273274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.273344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.273401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.273455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.273510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.273549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.273552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.273614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.273670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.273728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.273797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.273856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.273859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.273918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.273968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.274020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.274252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.276206Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28f2ddbf-5bcb4319-6fb57467-c2534948_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:13.276218Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28f2ddbf-5bcb4319-6fb57467-c2534948_0] Write session will restart in 0.008851s
2025-10-09T21:48:13.276244Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28f2ddbf-5bcb4319-6fb57467-c2534948_0] Write session: Do CDS request
2025-10-09T21:48:13.276250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28f2ddbf-5bcb4319-6fb57467-c2534948_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:13.275766Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:13.275773Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:13.275777Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:13.275820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1151 reason: error at write: Disconnected sessionId: src_id|28f2ddbf-5bcb4319-6fb57467-c2534948_0
2025-10-09T21:48:13.275858Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342128124132541:9750]
2025-10-09T21:48:13.276038Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1151 sessionId: src_id|28f2ddbf-5bcb4319-6fb57467-c2534948_0 is DEAD
2025-10-09T21:48:13.274305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.274316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.274379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.274435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.274489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.274544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.274585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.274588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.274645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.274699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.274751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.274804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.274843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.274846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.274904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.274956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.275012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.275073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.275115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.275118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.275187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.275243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.275299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.275356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.275400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.275403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.275473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342128124132541:9750], now have 1 active actors on pipe
2025-10-09T21:48:13.276568Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:13.286607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28f2ddbf-5bcb4319-6fb57467-c2534948_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:13.286699Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:13.286717Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1152
2025-10-09T21:48:13.286961Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1152 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:13.287004Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1152 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:13.287010Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1152 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:13.287014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:13.287455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1152 sessionId:  describe result for acl check
2025-10-09T21:48:13.287500Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:13.287507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:13.287509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:13.287515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342128124132565:10043] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:13.288072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342128124132565:10043] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:13.293475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342128124132565:10043] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:13.297732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342128124132565:10043] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:13.297760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342128124132565:10043] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:13.297764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342128124132565:10043] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:13.297773Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1152 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:13.302213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:13.304501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:13.304829Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:13.304844Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:13.304858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:13.304922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:13.304967Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342127762310726:8943]
2025-10-09T21:48:13.305519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:13.305962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:13.306016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:13.306368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:13.306423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.306429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.306430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.306431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.306433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.306434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.306435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.306436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.306437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.306442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.306454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.306457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.306460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.306464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.306468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.306471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.306475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.306478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.306485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.306494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.306500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.306505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.306509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.306513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.306517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.306521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.306526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.306530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.306533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.306538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.306550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.306555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.306559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.306563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.306567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.306572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.306599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:13.313396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:13.314540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:13.314559Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:13.314561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:13.314565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:13.314573Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1151 [2:7559342127762310726:8943]
2025-10-09T21:48:13.314588Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:13.314609Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:13.314612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:13.314615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:13.314618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:13.314637Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:13.314655Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:13.314663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:13.314722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.314730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.314732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.314735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:13.314810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:13.314821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.314831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.314841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.314855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:13.314877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342128124132549:2432], now have 1 active actors on pipe
2025-10-09T21:48:13.314881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342128124132730:10043], now have 1 active actors on pipe
2025-10-09T21:48:13.314888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342127762310721:8943]
2025-10-09T21:48:13.314890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342127762310721:8943]
2025-10-09T21:48:13.314893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342127762310721:8943]
2025-10-09T21:48:13.314908Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:13.314923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:13.314927Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1151
2025-10-09T21:48:13.314967Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1151
2025-10-09T21:48:13.314937Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:13.314947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:13.314983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342127762310721:8943]
2025-10-09T21:48:13.314985Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:13.315032Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:13.315045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:13.315047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:13.315054Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:13.315230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.315243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.315285Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d590f7bc-78a2ac32-809eda9f-6059ada5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:13.315301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:13.315306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:13.315309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:13.315321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:13.315350Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:13.315372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:13.315549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.315561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.315582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:13.315693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1152 partition: 0 MaxSeqNo: 25 sessionId: src_id|d590f7bc-78a2ac32-809eda9f-6059ada5_0
2025-10-09T21:48:13.316085Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28f2ddbf-5bcb4319-6fb57467-c2534948_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d590f7bc-78a2ac32-809eda9f-6059ada5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:13.318589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d590f7bc-78a2ac32-809eda9f-6059ada5_0] Write session: try to update token
2025-10-09T21:48:13.318607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d590f7bc-78a2ac32-809eda9f-6059ada5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:13.330978Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:13.331001Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:13.331003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:13.331005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:13.331015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342127762310721:8943]
2025-10-09T21:48:13.331019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342127762310721:8943]
2025-10-09T21:48:13.331023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342127762310721:8943]
2025-10-09T21:48:13.331071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:13.331077Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:13.331079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.331081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:13.331083Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:13.331087Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:13.331230Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:13.333277Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1152 sessionId: src_id|d590f7bc-78a2ac32-809eda9f-6059ada5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:13.335813Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:13.376207Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342128124132785:9750]
2025-10-09T21:48:13.376545Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 1151, pipe: [1:7559342128124132785:9750]
2025-10-09T21:48:13.492911Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:13.493514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.493535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.493585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342128124132785:9750], now have 1 active actors on pipe
2025-10-09T21:48:13.493822Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:13.493844Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:13.493846Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342128124132785:9750]
2025-10-09T21:48:13.493859Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:13.493868Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:13.493935Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1152 reason: error at write: Disconnected sessionId: src_id|d590f7bc-78a2ac32-809eda9f-6059ada5_0
2025-10-09T21:48:13.494140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1152 sessionId: src_id|d590f7bc-78a2ac32-809eda9f-6059ada5_0 is DEAD
2025-10-09T21:48:13.494412Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d590f7bc-78a2ac32-809eda9f-6059ada5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:13.494425Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d590f7bc-78a2ac32-809eda9f-6059ada5_0] Write session will restart in 0.009099s
2025-10-09T21:48:13.494465Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d590f7bc-78a2ac32-809eda9f-6059ada5_0] Write session: Do CDS request
2025-10-09T21:48:13.494473Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d590f7bc-78a2ac32-809eda9f-6059ada5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:13.505895Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d590f7bc-78a2ac32-809eda9f-6059ada5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:13.506242Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:13.506260Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1153
2025-10-09T21:48:13.506380Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1153 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:13.506415Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1153 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:13.506428Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1153 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:13.506432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:13.506862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1153 sessionId:  describe result for acl check
2025-10-09T21:48:13.506893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:13.506899Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:13.506900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:13.506907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342128124132894:10047] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:13.507475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342128124132894:10047] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:13.509703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342128124132894:10047] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:13.511523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:13.513863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:13.514063Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:13.514072Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:13.514083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:13.514095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342128124132894:10047] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:13.514110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342128124132894:10047] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:13.514114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342128124132894:10047] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:13.514125Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1153 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:13.514150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:13.514190Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342127762310952:8945]
2025-10-09T21:48:13.514770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:13.515097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:13.515151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:13.515478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:13.515524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.515531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.515533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.515534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.515535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.515536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.515537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.515538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.515539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.515545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.515555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.515558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.515561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.515565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.515569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.515572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.515576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.515580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.515587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.515596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.515602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.515607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.515611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.515617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.515621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.515626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.515631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.515636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.515640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.515646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.515651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.515656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.515661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.515666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.515678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.515685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.515714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:13.522383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:13.523465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:13.523481Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:13.523484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:13.523488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:13.523495Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1152 [2:7559342127762310952:8945]
2025-10-09T21:48:13.523512Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:13.523532Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:13.523540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:13.523544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:13.523546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:13.523564Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:13.523570Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:13.523578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:13.523639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.523649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.523652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.523655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:13.523730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:13.523738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.523750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.523766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.523789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:13.523811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342128124133013:10047], now have 1 active actors on pipe
2025-10-09T21:48:13.523820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342128124132787:2432], now have 1 active actors on pipe
2025-10-09T21:48:13.523828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342127762310947:8945]
2025-10-09T21:48:13.523835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342127762310947:8945]
2025-10-09T21:48:13.523837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342127762310947:8945]
2025-10-09T21:48:13.523851Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:13.523866Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:13.523868Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:13.523877Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:13.523879Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:13.523884Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:13.523887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:13.523923Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342127762310947:8945]
2025-10-09T21:48:13.523933Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:13.523912Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1152
2025-10-09T21:48:13.523934Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:13.523916Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1152
2025-10-09T21:48:13.523959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.523964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.524000Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dc3ca6df-29ebe28f-6f90d55a-7e16a70e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:13.524008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:13.524013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:13.524015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:13.524027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:13.524037Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:13.524055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:13.524438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.524458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.524484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:13.524631Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1153 partition: 0 MaxSeqNo: 25 sessionId: src_id|dc3ca6df-29ebe28f-6f90d55a-7e16a70e_0
2025-10-09T21:48:13.525005Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d590f7bc-78a2ac32-809eda9f-6059ada5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dc3ca6df-29ebe28f-6f90d55a-7e16a70e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:13.527464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dc3ca6df-29ebe28f-6f90d55a-7e16a70e_0] Write session: try to update token
2025-10-09T21:48:13.527481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dc3ca6df-29ebe28f-6f90d55a-7e16a70e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:13.538327Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:13.538345Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:13.538348Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:13.538350Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:13.538360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342127762310947:8945]
2025-10-09T21:48:13.538365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342127762310947:8945]
2025-10-09T21:48:13.538368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342127762310947:8945]
2025-10-09T21:48:13.538415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:13.538419Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:13.538424Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:13.538428Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:13.538429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.538431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:13.541031Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1153 sessionId: src_id|dc3ca6df-29ebe28f-6f90d55a-7e16a70e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:13.538566Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:13.543608Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:13.594108Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342128124133024:9750]
2025-10-09T21:48:13.594444Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 1152, pipe: [1:7559342128124133024:9750]
2025-10-09T21:48:13.694938Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:13.695555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.695576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.695956Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:13.695979Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:13.696380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc3ca6df-29ebe28f-6f90d55a-7e16a70e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:13.696392Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc3ca6df-29ebe28f-6f90d55a-7e16a70e_0] Write session will restart in 0.005800s
2025-10-09T21:48:13.696043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1153 reason: error at write: Disconnected sessionId: src_id|dc3ca6df-29ebe28f-6f90d55a-7e16a70e_0
2025-10-09T21:48:13.696111Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:13.695623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342128124133024:9750], now have 1 active actors on pipe
2025-10-09T21:48:13.696153Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342128124133024:9750]
2025-10-09T21:48:13.696220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1153 sessionId: src_id|dc3ca6df-29ebe28f-6f90d55a-7e16a70e_0 is DEAD
2025-10-09T21:48:13.696418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc3ca6df-29ebe28f-6f90d55a-7e16a70e_0] Write session: Do CDS request
2025-10-09T21:48:13.696429Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc3ca6df-29ebe28f-6f90d55a-7e16a70e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:13.696570Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:13.703563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dc3ca6df-29ebe28f-6f90d55a-7e16a70e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:13.703635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:13.703653Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1154
2025-10-09T21:48:13.703816Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1154 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:13.703859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1154 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:13.703881Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1154 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:13.703886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:13.704358Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1154 sessionId:  describe result for acl check
2025-10-09T21:48:13.704407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:13.704416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:13.704418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:13.704424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342128124133042:10051] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:13.704991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342128124133042:10051] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:13.707890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342128124133042:10051] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:13.711327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342128124133042:10051] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:13.711342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342128124133042:10051] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:13.711346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342128124133042:10051] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:13.711355Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1154 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:13.714316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:13.717050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:13.717270Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:13.717281Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:13.717295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:13.717351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:13.717393Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342127762311185:8949]
2025-10-09T21:48:13.718021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:13.718454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:13.718502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:13.718832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:13.718887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.718893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.718895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.718896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.718897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.718899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.718900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.718901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.718902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.718908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.718921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.718925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.718929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.718932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.718936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.718941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.718945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.718949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.718956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.718969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.718991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.719000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.719005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.719010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.719014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.719019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.719024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.719029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.719033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.719040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.719047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.719052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.719061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.719065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.719071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.719076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.719102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:13.726066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:13.727188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:13.727209Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:13.727212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:13.727216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:13.727224Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1153 [2:7559342127762311185:8949]
2025-10-09T21:48:13.727239Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:13.727260Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:13.727264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:13.727267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:13.727270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:13.727288Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:13.727294Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:13.727302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:13.727365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.727375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.727377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.727379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:13.727454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:13.727461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.727469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.727476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.727486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:13.727504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342128124133027:2432], now have 1 active actors on pipe
2025-10-09T21:48:13.727507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342128124133200:10051], now have 1 active actors on pipe
2025-10-09T21:48:13.727513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342127762311180:8949]
2025-10-09T21:48:13.727515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342127762311180:8949]
2025-10-09T21:48:13.727516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342127762311180:8949]
2025-10-09T21:48:13.727523Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:13.727534Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:13.727535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:13.727537Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:13.727539Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:13.727544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:13.727548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:13.727586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342127762311180:8949]
2025-10-09T21:48:13.727594Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:13.727655Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:13.727811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.727829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.727860Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|55945461-57b683b4-23a44278-25b908d4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:13.727875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:13.727883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:13.727886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:13.727900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:13.727919Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:13.728067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:13.728463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.728482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.728498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:13.728996Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc3ca6df-29ebe28f-6f90d55a-7e16a70e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|55945461-57b683b4-23a44278-25b908d4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:13.727636Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1153
2025-10-09T21:48:13.727683Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1153
2025-10-09T21:48:13.728608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1154 partition: 0 MaxSeqNo: 25 sessionId: src_id|55945461-57b683b4-23a44278-25b908d4_0
2025-10-09T21:48:13.731509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55945461-57b683b4-23a44278-25b908d4_0] Write session: try to update token
2025-10-09T21:48:13.731527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55945461-57b683b4-23a44278-25b908d4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:13.739527Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:13.739548Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:13.739559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342127762311180:8949]
2025-10-09T21:48:13.739602Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:13.739608Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:13.739801Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:13.739944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:13.740086Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:13.740220Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:13.740400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:13.740439Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:13.746194Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1154 sessionId: src_id|55945461-57b683b4-23a44278-25b908d4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:13.748706Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:13.748934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.748958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.749009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:13.749105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:13.753658Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:13.753677Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:13.753680Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:13.753682Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:13.753692Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342127762311180:8949]
2025-10-09T21:48:13.753695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342127762311180:8949]
2025-10-09T21:48:13.753699Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342127762311180:8949]
2025-10-09T21:48:13.753756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:13.753763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.753766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:13.753766Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:13.753775Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:13.753779Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:13.753896Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:13.793528Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342128124133264:9750]
2025-10-09T21:48:13.793838Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 1153, pipe: [1:7559342128124133264:9750]
2025-10-09T21:48:13.913131Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:13.921438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.921467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.921643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.921702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.921765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.921834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.922134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.922145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.922207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.922267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.922323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.922384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.922428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.922436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.922493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.922548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.922607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.922664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.922705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.922715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.922776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.922833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.922896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.922959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.923005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.923012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.923076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.923139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.923200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.923267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.923330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.923337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.923406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.923466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.923525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.923586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.923632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.923640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.923700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.923753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.923821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.923882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.923925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.923932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.923995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.924054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.924117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.924179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.924222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.924229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.924292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.924353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.924416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.924475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.924518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.924523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.924585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:13.924652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:13.924713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:13.924770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:13.924811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:13.924818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:13.924888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342128124133264:9750], now have 1 active actors on pipe
2025-10-09T21:48:13.925131Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342128124133264:9750]
2025-10-09T21:48:13.925146Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:13.925183Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:13.925186Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:13.925218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1154 reason: error at write: Disconnected sessionId: src_id|55945461-57b683b4-23a44278-25b908d4_0
2025-10-09T21:48:13.925466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1154 sessionId: src_id|55945461-57b683b4-23a44278-25b908d4_0 is DEAD
2025-10-09T21:48:13.925512Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:13.925649Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55945461-57b683b4-23a44278-25b908d4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:13.925681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55945461-57b683b4-23a44278-25b908d4_0] Write session will restart in 0.008949s
2025-10-09T21:48:13.925711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55945461-57b683b4-23a44278-25b908d4_0] Write session: Do CDS request
2025-10-09T21:48:13.925730Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55945461-57b683b4-23a44278-25b908d4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:13.935602Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55945461-57b683b4-23a44278-25b908d4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:13.935686Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:13.935702Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1155
2025-10-09T21:48:13.935907Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1155 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:13.935954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1155 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:13.935959Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1155 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:13.935964Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:13.936640Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1155 sessionId:  describe result for acl check
2025-10-09T21:48:13.936678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:13.936680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:13.936682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:13.936688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342128124133350:10063] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:13.937225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342128124133350:10063] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:13.939863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342128124133350:10063] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:13.943071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342128124133350:10063] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:13.943084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342128124133350:10063] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:13.943088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342128124133350:10063] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:13.943106Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1155 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:13.943680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:13.946172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:13.946380Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:13.946390Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:13.946402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:13.946471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:13.946514Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342127762311452:8963]
2025-10-09T21:48:13.947139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:13.947417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:13.947459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:13.947759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:13.947821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.947829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.947830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.947832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.947833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.947834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.947835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.947836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.947837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.947843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.947854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.947858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.947862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.947865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.947869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.947872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.947876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.947879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.947887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.947894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:13.947906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.947915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:13.947919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.947930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:13.947935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.947940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:13.947947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.947953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:13.947962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.947968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:13.947972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.947977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.947981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.947987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.947996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.948001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.948031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:13.955019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:13.956334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:13.956351Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:13.956353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:13.956358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:13.956365Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1154 [2:7559342127762311452:8963]
2025-10-09T21:48:13.956383Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:13.956402Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:13.956405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:13.956409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:13.956411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:13.956430Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:13.956443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:13.956450Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:13.956513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.956520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.956522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.956526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:13.956594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:13.956602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.956614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:13.956628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:13.956648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:13.956674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342128124133517:10063], now have 1 active actors on pipe
2025-10-09T21:48:13.956682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342128124133294:2432], now have 1 active actors on pipe
2025-10-09T21:48:13.956690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342127762311447:8963]
2025-10-09T21:48:13.956700Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342127762311447:8963]
2025-10-09T21:48:13.956702Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342127762311447:8963]
2025-10-09T21:48:13.956713Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:13.956723Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:13.956726Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:13.956728Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:13.956728Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:13.956744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:13.956747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:13.956733Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1154
2025-10-09T21:48:13.956761Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1154
2025-10-09T21:48:13.956783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342127762311447:8963]
2025-10-09T21:48:13.956793Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:13.956798Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:13.956808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.956819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.956870Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b4994d0-836395a0-d973b1fd-463da23d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:13.956885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:13.956895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:13.956897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:13.956908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:13.956922Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:13.956947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:13.957139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:13.957149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:13.957162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:13.957237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1155 partition: 0 MaxSeqNo: 25 sessionId: src_id|b4994d0-836395a0-d973b1fd-463da23d_0
2025-10-09T21:48:13.957585Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55945461-57b683b4-23a44278-25b908d4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b4994d0-836395a0-d973b1fd-463da23d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:13.960390Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4994d0-836395a0-d973b1fd-463da23d_0] Write session: try to update token
2025-10-09T21:48:13.960407Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4994d0-836395a0-d973b1fd-463da23d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:13.972186Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:13.972212Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:13.972214Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:13.972217Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:13.972228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342127762311447:8963]
2025-10-09T21:48:13.972234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342127762311447:8963]
2025-10-09T21:48:13.972239Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342127762311447:8963]
2025-10-09T21:48:13.972289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:13.972296Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:13.972303Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:13.972304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:13.972308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:13.972308Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:13.972453Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:13.975598Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1155 sessionId: src_id|b4994d0-836395a0-d973b1fd-463da23d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:13.978198Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:14.025216Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342132419100835:9750]
2025-10-09T21:48:14.025530Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 1154, pipe: [1:7559342132419100835:9750]
2025-10-09T21:48:14.135366Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:14.135981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:14.135998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:14.136046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342132419100835:9750], now have 1 active actors on pipe
2025-10-09T21:48:14.136605Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342132419100835:9750]
2025-10-09T21:48:14.136612Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:14.136616Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:14.136637Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:14.136267Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:14.136676Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1155 reason: error at write: Disconnected sessionId: src_id|b4994d0-836395a0-d973b1fd-463da23d_0
2025-10-09T21:48:14.136876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1155 sessionId: src_id|b4994d0-836395a0-d973b1fd-463da23d_0 is DEAD
2025-10-09T21:48:14.137095Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4994d0-836395a0-d973b1fd-463da23d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:14.137106Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4994d0-836395a0-d973b1fd-463da23d_0] Write session will restart in 0.008515s
2025-10-09T21:48:14.137134Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4994d0-836395a0-d973b1fd-463da23d_0] Write session: Do CDS request
2025-10-09T21:48:14.137141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4994d0-836395a0-d973b1fd-463da23d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:14.146588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4994d0-836395a0-d973b1fd-463da23d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:14.146845Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:14.146862Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1156
2025-10-09T21:48:14.147007Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1156 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:14.147053Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1156 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:14.147065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1156 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:14.147070Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:14.147565Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1156 sessionId:  describe result for acl check
2025-10-09T21:48:14.147601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:14.147608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:14.147609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:14.147615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342132419100883:10067] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:14.148144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342132419100883:10067] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:14.150670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342132419100883:10067] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:14.153642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342132419100883:10067] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:14.153655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342132419100883:10067] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:14.153659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342132419100883:10067] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:14.153667Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1156 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:14.155522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:14.157992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:14.158205Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:14.158217Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:14.158230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:14.158299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:14.158339Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342132057278979:8965]
2025-10-09T21:48:14.158957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:14.159341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:14.159408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:14.159775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:14.159846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:14.159856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:14.159858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:14.159859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:14.159860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:14.159861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:14.159862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.159863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.159864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.159870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:14.159881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:14.159885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:14.159889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:14.159893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:14.159897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:14.159902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.159905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.159909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.159916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:14.159924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:14.159931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:14.159936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:14.159939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:14.159943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:14.159946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:14.159950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:14.159954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:14.159959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:14.159963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:14.159968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:14.159981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.159994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.160005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.160011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.160015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.160020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.160050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:14.166791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:14.168080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:14.168100Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:14.168102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:14.168107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:14.168114Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1155 [2:7559342132057278979:8965]
2025-10-09T21:48:14.168128Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:14.168148Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:14.168151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:14.168155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:14.168158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:14.168177Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:14.168182Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:14.168191Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:14.168248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.168256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.168258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.168261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:14.168340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:14.168352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.168366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.168378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.168397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:14.168422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342132419100842:2432], now have 1 active actors on pipe
2025-10-09T21:48:14.168432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342132419101067:10067], now have 1 active actors on pipe
2025-10-09T21:48:14.168441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342132057278974:8965]
2025-10-09T21:48:14.168444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342132057278974:8965]
2025-10-09T21:48:14.168446Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342132057278974:8965]
2025-10-09T21:48:14.168460Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:14.168474Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:14.168458Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1155
2025-10-09T21:48:14.168489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:14.168497Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:14.168511Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1155
2025-10-09T21:48:14.168531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342132057278974:8965]
2025-10-09T21:48:14.168539Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:14.168539Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:14.168543Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:14.168545Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:14.168552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:14.168652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:14.168662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:14.168689Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fa5abcf1-7f38e143-219921d7-dc27b047_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:14.168702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:14.168707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:14.168709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:14.168730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:14.168746Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:14.168763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:14.168939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:14.168958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:14.168973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:14.169115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1156 partition: 0 MaxSeqNo: 25 sessionId: src_id|fa5abcf1-7f38e143-219921d7-dc27b047_0
2025-10-09T21:48:14.169502Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4994d0-836395a0-d973b1fd-463da23d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fa5abcf1-7f38e143-219921d7-dc27b047_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:14.172100Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fa5abcf1-7f38e143-219921d7-dc27b047_0] Write session: try to update token
2025-10-09T21:48:14.172121Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fa5abcf1-7f38e143-219921d7-dc27b047_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:14.180349Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:14.180362Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:14.180372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342132057278974:8965]
2025-10-09T21:48:14.180423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:14.180597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:14.180747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:14.180878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:14.181014Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:14.181200Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:14.181249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:14.181381Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:14.186100Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1156 sessionId: src_id|fa5abcf1-7f38e143-219921d7-dc27b047_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:14.188465Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:14.188876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:14.188904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:14.188944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:14.189047Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:14.192977Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:14.192992Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:14.192995Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:14.192997Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:14.193007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342132057278974:8965]
2025-10-09T21:48:14.193011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342132057278974:8965]
2025-10-09T21:48:14.193015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342132057278974:8965]
2025-10-09T21:48:14.193057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:14.193067Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:14.193069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.193072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:14.193072Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:14.193075Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:14.193209Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:14.236697Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342132419101087:9750]
2025-10-09T21:48:14.237020Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 1155, pipe: [1:7559342132419101087:9750]
2025-10-09T21:48:14.358256Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:14.366338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:14.366362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:14.366535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.366597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.366665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.366729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.366780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.366790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.366856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.366918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.366981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.367045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.367095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.367098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.369635Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342132419101087:9750]
2025-10-09T21:48:14.369651Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:14.369655Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:14.369669Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:14.369686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1156 reason: error at write: Disconnected sessionId: src_id|fa5abcf1-7f38e143-219921d7-dc27b047_0
2025-10-09T21:48:14.369950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1156 sessionId: src_id|fa5abcf1-7f38e143-219921d7-dc27b047_0 is DEAD
2025-10-09T21:48:14.367162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.367224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.367281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.367334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.367373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.367376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.367442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.367497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.367553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.367612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.367652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.367656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.367722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.367777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.367835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.367889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.370123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa5abcf1-7f38e143-219921d7-dc27b047_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:14.367927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.367931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.370136Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa5abcf1-7f38e143-219921d7-dc27b047_0] Write session will restart in 0.008444s
2025-10-09T21:48:14.367992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.368046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.368099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.368159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.368197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.368201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.368260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.368308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.368362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.368418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.368523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.368528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.368588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.370175Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa5abcf1-7f38e143-219921d7-dc27b047_0] Write session: Do CDS request
2025-10-09T21:48:14.368638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.368696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.370183Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa5abcf1-7f38e143-219921d7-dc27b047_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:14.368749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.368786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.368789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.368849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.368901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.368955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.369012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.369052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.369055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.369115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.369168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.369222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.369276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.369317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.369320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.369394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342132419101087:9750], now have 1 active actors on pipe
2025-10-09T21:48:14.370657Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:14.379651Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fa5abcf1-7f38e143-219921d7-dc27b047_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:14.379847Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:14.379865Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1157
2025-10-09T21:48:14.380017Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1157 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:14.380066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1157 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:14.380077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1157 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:14.380081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:14.380506Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1157 sessionId:  describe result for acl check
2025-10-09T21:48:14.380557Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:14.380564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:14.380566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:14.380572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342132419101116:10072] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:14.381181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342132419101116:10072] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:14.383945Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342132419101116:10072] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:14.387462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342132419101116:10072] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:14.387475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342132419101116:10072] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:14.387479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342132419101116:10072] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:14.387488Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1157 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:14.389858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:14.392283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:14.392516Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:14.392530Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:14.392543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:14.392602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:14.392645Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342132057279210:8967]
2025-10-09T21:48:14.393286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:14.393621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:14.393698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:14.394063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:14.394132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:14.394140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:14.394141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:14.394143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:14.394144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:14.394145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:14.394147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.394148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.394149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.394155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:14.394166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:14.394170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:14.394174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:14.394178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:14.394182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:14.394185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.394190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.394193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.394201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:14.394209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:14.394215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:14.394220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:14.394225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:14.394231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:14.394235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:14.394240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:14.394244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:14.394248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:14.394252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:14.394256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:14.394263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.394273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.394277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.394283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.394292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.394297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.394325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:14.400633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:14.401669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:14.401685Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:14.401688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:14.401693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:14.401700Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1156 [2:7559342132057279210:8967]
2025-10-09T21:48:14.401714Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:14.402150Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1156
2025-10-09T21:48:14.402191Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1156
2025-10-09T21:48:14.401733Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:14.401755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:14.401758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:14.401761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:14.401778Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:14.401792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:14.401799Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:14.401852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.401854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.401856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.401859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:14.401932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:14.401935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.401947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.401960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.401977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:14.401998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342132419101313:10072], now have 1 active actors on pipe
2025-10-09T21:48:14.402002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342132419101090:2432], now have 1 active actors on pipe
2025-10-09T21:48:14.402010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342132057279205:8967]
2025-10-09T21:48:14.402012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342132057279205:8967]
2025-10-09T21:48:14.402014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342132057279205:8967]
2025-10-09T21:48:14.402045Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:14.402060Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:14.402064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:14.402068Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:14.402070Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:14.402076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:14.402079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:14.402113Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342132057279205:8967]
2025-10-09T21:48:14.402122Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:14.402161Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:14.402272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:14.402280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:14.402323Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cf58c1c9-97af28ea-617480a4-2dd78491_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:14.402332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:14.402337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:14.402340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:14.402354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:14.402372Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:14.402396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:14.402844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:14.402868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:14.402889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:14.403049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1157 partition: 0 MaxSeqNo: 25 sessionId: src_id|cf58c1c9-97af28ea-617480a4-2dd78491_0
2025-10-09T21:48:14.403424Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa5abcf1-7f38e143-219921d7-dc27b047_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cf58c1c9-97af28ea-617480a4-2dd78491_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:14.406009Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf58c1c9-97af28ea-617480a4-2dd78491_0] Write session: try to update token
2025-10-09T21:48:14.406025Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf58c1c9-97af28ea-617480a4-2dd78491_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:14.413386Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:14.413401Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:14.413422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342132057279205:8967]
2025-10-09T21:48:14.413479Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:14.413487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:14.413651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:14.413789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:14.413914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:14.414078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:14.414268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:14.414306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:14.419772Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1157 sessionId: src_id|cf58c1c9-97af28ea-617480a4-2dd78491_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:14.422149Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:14.422374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:14.422396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:14.422439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:14.422537Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:14.427607Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:14.427624Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:14.427626Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:14.427628Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:14.427640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342132057279205:8967]
2025-10-09T21:48:14.427644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342132057279205:8967]
2025-10-09T21:48:14.427648Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342132057279205:8967]
2025-10-09T21:48:14.427692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:14.427697Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:14.427701Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:14.427704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.427705Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:14.427707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:14.427838Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:14.469682Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342132419101330:9750]
2025-10-09T21:48:14.469973Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 1156, pipe: [1:7559342132419101330:9750]
2025-10-09T21:48:14.583136Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:14.593664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:14.593693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:14.593942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.594270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.594336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.594399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.594905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.594918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.595140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.595377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.595600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.595842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.595885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.595899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.595956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.596185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.596246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.596306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.596349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.596357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.596626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.596689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.596747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.596807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.596851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.596859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.596924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.596985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.597043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.597102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.597148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.597155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.597213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.597270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.597328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.597393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.597438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.597445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.597504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.597558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.597627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.597683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.597725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.597732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.597803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.598026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.598275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.598336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.598384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.598395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.598451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.598504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.598829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.599092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.599136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.599143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.599372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.599432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.599498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.599561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.599600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.599608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.599681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342132419101330:9750], now have 1 active actors on pipe
2025-10-09T21:48:14.599935Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:14.599949Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:14.599965Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342132419101330:9750]
2025-10-09T21:48:14.599995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1157 reason: error at write: Disconnected sessionId: src_id|cf58c1c9-97af28ea-617480a4-2dd78491_0
2025-10-09T21:48:14.600002Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:14.600252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1157 sessionId: src_id|cf58c1c9-97af28ea-617480a4-2dd78491_0 is DEAD
2025-10-09T21:48:14.600344Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:14.600403Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf58c1c9-97af28ea-617480a4-2dd78491_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:14.600418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf58c1c9-97af28ea-617480a4-2dd78491_0] Write session will restart in 0.005122s
2025-10-09T21:48:14.600451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf58c1c9-97af28ea-617480a4-2dd78491_0] Write session: Do CDS request
2025-10-09T21:48:14.600457Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf58c1c9-97af28ea-617480a4-2dd78491_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:14.606579Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf58c1c9-97af28ea-617480a4-2dd78491_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:14.606690Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:14.606707Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1158
2025-10-09T21:48:14.606884Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1158 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:14.606928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1158 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:14.606947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1158 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:14.606957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:14.607608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1158 sessionId:  describe result for acl check
2025-10-09T21:48:14.607653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:14.607661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:14.607665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:14.607672Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342132419101346:10076] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:14.608366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342132419101346:10076] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:14.611589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342132419101346:10076] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:14.615550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342132419101346:10076] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:14.615564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342132419101346:10076] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:14.615569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342132419101346:10076] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:14.615578Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1158 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:14.623110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:14.625643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:14.625926Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:14.625936Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:14.625949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:14.626009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:14.626075Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342132057279443:8969]
2025-10-09T21:48:14.626748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:14.627125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:14.627181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:14.627542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:14.627598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:14.627605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:14.627606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:14.627607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:14.627608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:14.627609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:14.627610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.627611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.627612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.627617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:14.627627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:14.627631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:14.627634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:14.627638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:14.627641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:14.627644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.627648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.627651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.627662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:14.627669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:14.627679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:14.627683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:14.627688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:14.627692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:14.627699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:14.627704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:14.627708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:14.627712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:14.627716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:14.627720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:14.627727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.627732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.627736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.627740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.627744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.627748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.627779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:14.635164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:14.636519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:14.636535Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:14.636538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:14.636542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:14.636549Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1157 [2:7559342132057279443:8969]
2025-10-09T21:48:14.636562Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:14.636580Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:14.636588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:14.636591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:14.636594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:14.636611Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:14.636617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:14.636626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:14.636681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.636689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.636691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.636694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:14.636762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:14.636769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.636781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.636798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.636816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:14.636842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342132419101475:10076], now have 1 active actors on pipe
2025-10-09T21:48:14.636851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342132419101333:2432], now have 1 active actors on pipe
2025-10-09T21:48:14.636859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342132057279438:8969]
2025-10-09T21:48:14.636862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342132057279438:8969]
2025-10-09T21:48:14.636864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342132057279438:8969]
2025-10-09T21:48:14.636880Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:14.636898Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:14.636916Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:14.636924Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:14.636917Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1157
2025-10-09T21:48:14.636955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342132057279438:8969]
2025-10-09T21:48:14.636962Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:14.636936Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1157
2025-10-09T21:48:14.636974Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:14.636978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:14.636981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:14.636987Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:14.637103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:14.637116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:14.637167Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a53e33a3-6b863280-aef8bc12-24fbce16_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:14.637182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:14.637188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:14.637191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:14.637202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:14.637219Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:14.637247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:14.637461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:14.637483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:14.637504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:14.637616Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1158 partition: 0 MaxSeqNo: 25 sessionId: src_id|a53e33a3-6b863280-aef8bc12-24fbce16_0
2025-10-09T21:48:14.638078Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf58c1c9-97af28ea-617480a4-2dd78491_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a53e33a3-6b863280-aef8bc12-24fbce16_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:14.642917Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a53e33a3-6b863280-aef8bc12-24fbce16_0] Write session: try to update token
2025-10-09T21:48:14.642937Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a53e33a3-6b863280-aef8bc12-24fbce16_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:14.649638Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:14.649656Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:14.649667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342132057279438:8969]
2025-10-09T21:48:14.649717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:14.649793Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:14.649921Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:14.650091Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:14.650236Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:14.650376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:14.650565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:14.650614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:14.658313Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1158 sessionId: src_id|a53e33a3-6b863280-aef8bc12-24fbce16_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:14.660683Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:14.660891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:14.660913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:14.660957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:14.661070Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:14.663801Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:14.663815Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:14.663818Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:14.663820Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:14.663831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342132057279438:8969]
2025-10-09T21:48:14.663835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342132057279438:8969]
2025-10-09T21:48:14.663839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342132057279438:8969]
2025-10-09T21:48:14.663879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:14.663886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.663889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:14.663926Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:14.663938Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:14.663943Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:14.664031Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:14.700038Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342132419101575:9750]
2025-10-09T21:48:14.700304Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 1157, pipe: [1:7559342132419101575:9750]
2025-10-09T21:48:14.819797Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:14.828031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:14.828054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:14.828212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.828268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.828328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.828393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.828439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.828449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.828519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.828578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.828642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.828709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.828752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.828762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.828822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.828884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.828944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.829003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.829046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.829053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.829117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.829176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.829236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.829298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.829340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.829352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.829423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.829520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.829587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.829643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.829692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.829694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.829770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.829825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.829884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.829941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.829981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.829983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.830127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.830185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.830249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.830319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.831413Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342132419101575:9750]
2025-10-09T21:48:14.831451Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:14.831456Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:14.831485Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1158 reason: error at write: Disconnected sessionId: src_id|a53e33a3-6b863280-aef8bc12-24fbce16_0
2025-10-09T21:48:14.830367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.830375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.830437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.830490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.830548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.830606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.830646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.830649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.830709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.830764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.830823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.830884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.830925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.830928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.830989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:14.831038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:14.831092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:14.831150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:14.831189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:14.831191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:14.831244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342132419101575:9750], now have 1 active actors on pipe
2025-10-09T21:48:14.831845Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a53e33a3-6b863280-aef8bc12-24fbce16_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:14.831872Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a53e33a3-6b863280-aef8bc12-24fbce16_0] Write session will restart in 0.007775s
2025-10-09T21:48:14.831698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1158 sessionId: src_id|a53e33a3-6b863280-aef8bc12-24fbce16_0 is DEAD
2025-10-09T21:48:14.831719Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:14.831896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a53e33a3-6b863280-aef8bc12-24fbce16_0] Write session: Do CDS request
2025-10-09T21:48:14.831903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a53e33a3-6b863280-aef8bc12-24fbce16_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:14.832277Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:14.840932Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a53e33a3-6b863280-aef8bc12-24fbce16_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:14.840952Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:14.840970Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1159
2025-10-09T21:48:14.841157Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1159 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:14.841202Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1159 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:14.841215Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1159 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:14.841221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:14.841709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1159 sessionId:  describe result for acl check
2025-10-09T21:48:14.841757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:14.841763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:14.841766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:14.841771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342132419101617:10081] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:14.842300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342132419101617:10081] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:14.845522Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342132419101617:10081] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:14.849455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342132419101617:10081] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:14.849471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342132419101617:10081] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:14.849475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342132419101617:10081] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:14.849486Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1159 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:14.850245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:14.852789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:14.853039Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:14.853048Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:14.853060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:14.853120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:14.853162Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342132057279711:8983]
2025-10-09T21:48:14.853739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:14.854133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:14.854184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:14.854533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:14.854589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:14.854595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:14.854597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:14.854598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:14.854599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:14.854600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:14.854602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.854603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.854604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.854610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:14.854622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:14.854626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:14.854630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:14.854633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:14.854638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:14.854641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.854645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.854649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.854655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:14.854663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:14.854669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:14.854675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:14.854679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:14.854683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:14.854687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:14.854691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:14.854696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:14.854701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:14.854705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:14.854711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:14.854715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.854720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.854725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.854730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.854734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.854739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.854768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:14.861425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:14.862572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:14.862590Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:14.862593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:14.862598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:14.862605Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1158 [2:7559342132057279711:8983]
2025-10-09T21:48:14.862621Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:14.862644Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:14.862647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:14.862651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:14.862654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:14.862674Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:14.862688Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:14.862700Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:14.862765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.862773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.862776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.862779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:14.862866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:14.862874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.862887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:14.862902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:14.862923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:14.862950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342132419101586:2432], now have 1 active actors on pipe
2025-10-09T21:48:14.862958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342132419101813:10081], now have 1 active actors on pipe
2025-10-09T21:48:14.862966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342132057279706:8983]
2025-10-09T21:48:14.862973Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342132057279706:8983]
2025-10-09T21:48:14.862975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342132057279706:8983]
2025-10-09T21:48:14.862989Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:14.863001Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:14.863009Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:14.863009Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:14.863011Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:14.863024Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:14.863028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:14.862994Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1158
2025-10-09T21:48:14.863025Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1158
2025-10-09T21:48:14.863066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342132057279706:8983]
2025-10-09T21:48:14.863078Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:14.863084Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:14.863104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:14.863113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:14.863164Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f2d9ec8d-d4d1765a-f59bfdc3-8b729172_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:14.863177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:14.863181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:14.863184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:14.863195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:14.863206Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:14.863230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:14.863408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:14.863418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:14.863432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:14.863518Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1159 partition: 0 MaxSeqNo: 25 sessionId: src_id|f2d9ec8d-d4d1765a-f59bfdc3-8b729172_0
2025-10-09T21:48:14.863908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a53e33a3-6b863280-aef8bc12-24fbce16_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f2d9ec8d-d4d1765a-f59bfdc3-8b729172_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:14.866394Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2d9ec8d-d4d1765a-f59bfdc3-8b729172_0] Write session: try to update token
2025-10-09T21:48:14.866415Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2d9ec8d-d4d1765a-f59bfdc3-8b729172_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:14.875605Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:14.875635Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:14.875645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342132057279706:8983]
2025-10-09T21:48:14.875700Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:14.875710Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:14.875873Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:14.876019Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:14.876158Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:14.876291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:14.876477Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:14.876523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:14.879527Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1159 sessionId: src_id|f2d9ec8d-d4d1765a-f59bfdc3-8b729172_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:14.881917Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:14.882361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:14.882385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:14.882425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:14.882532Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:14.889703Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:14.889717Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:14.889720Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:14.889722Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:14.889732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342132057279706:8983]
2025-10-09T21:48:14.889736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342132057279706:8983]
2025-10-09T21:48:14.889739Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342132057279706:8983]
2025-10-09T21:48:14.889794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:14.889808Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:14.889810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:14.889812Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:14.889813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:14.889816Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:14.889969Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:14.931450Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342132419101842:9750]
2025-10-09T21:48:14.931809Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 1158, pipe: [1:7559342132419101842:9750]
2025-10-09T21:48:15.047321Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:15.055178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.055204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.055365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.055431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.055506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.055572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.055615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.055628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.055701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.055767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.055834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.055903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.055947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.055954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.056017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.056074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.056131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.056195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.056238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.056244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.056312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.056376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.056433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.056499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.056550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.056558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.056623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.056682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.056751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.056813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.056862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.056869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.056939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.057004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.057068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.057129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.057170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.057177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.057239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.057296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.057354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.057422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.057464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.057473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.057532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.057590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.057650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.057716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.057772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.057783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.057847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.057909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.057973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.058053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.058101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.058112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.058173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.058680Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:15.058706Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342132419101842:9750]
2025-10-09T21:48:15.058745Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:15.058757Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:15.058787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1159 reason: error at write: Disconnected sessionId: src_id|f2d9ec8d-d4d1765a-f59bfdc3-8b729172_0
2025-10-09T21:48:15.058231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.058292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.058352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.058394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.058397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.058466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342132419101842:9750], now have 1 active actors on pipe
2025-10-09T21:48:15.059007Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1159 sessionId: src_id|f2d9ec8d-d4d1765a-f59bfdc3-8b729172_0 is DEAD
2025-10-09T21:48:15.059147Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2d9ec8d-d4d1765a-f59bfdc3-8b729172_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:15.059159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2d9ec8d-d4d1765a-f59bfdc3-8b729172_0] Write session will restart in 0.006176s
2025-10-09T21:48:15.059181Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2d9ec8d-d4d1765a-f59bfdc3-8b729172_0] Write session: Do CDS request
2025-10-09T21:48:15.059187Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2d9ec8d-d4d1765a-f59bfdc3-8b729172_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:15.059521Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:15.066503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2d9ec8d-d4d1765a-f59bfdc3-8b729172_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:15.066592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:15.066609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1160
2025-10-09T21:48:15.066743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1160 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:15.066775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1160 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:15.066781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1160 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:15.066785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:15.067344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1160 sessionId:  describe result for acl check
2025-10-09T21:48:15.067391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:15.067401Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:15.067403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:15.067409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342136714069167:10092] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:15.067955Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342136714069167:10092] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:15.071391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342136714069167:10092] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:15.074387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342136714069167:10092] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:15.074401Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342136714069167:10092] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:15.074404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342136714069167:10092] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:15.074413Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1160 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:15.078066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:15.080197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:15.080384Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:15.080395Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:15.080406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:15.080454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:15.080489Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342136352247249:8987]
2025-10-09T21:48:15.081078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:15.081365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:15.081404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:15.081694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:15.081749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.081755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.081756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.081757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.081758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.081760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.081761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.081762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.081764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.081768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.081778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.081783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.081787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.081790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.081794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.081797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.081801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.081804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.081811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.081819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.081825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.081830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.081834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.081839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.081844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.081848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.081852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.081859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.081866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.081872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.081878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.081883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.081888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.081892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.081903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.081908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.081941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:15.088926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:15.090014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:15.090044Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:15.090046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:15.090051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:15.090058Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1159 [2:7559342136352247249:8987]
2025-10-09T21:48:15.090073Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:15.090091Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:15.090099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:15.090102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:15.090105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:15.090124Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:15.090134Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:15.090141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:15.090198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.090206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.090208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.090211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:15.090282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:15.090290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.090302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.090319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.090339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:15.090370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342136714069327:10092], now have 1 active actors on pipe
2025-10-09T21:48:15.090374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342136714069152:2432], now have 1 active actors on pipe
2025-10-09T21:48:15.090382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342136352247244:8987]
2025-10-09T21:48:15.090384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342136352247244:8987]
2025-10-09T21:48:15.090386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342136352247244:8987]
2025-10-09T21:48:15.090399Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:15.090414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:15.090415Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:15.090418Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:15.090421Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:15.090433Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:15.090436Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:15.090426Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1159
2025-10-09T21:48:15.090428Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1159
2025-10-09T21:48:15.090483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.090492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.090505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342136352247244:8987]
2025-10-09T21:48:15.090517Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:15.090523Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:15.090555Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c328884d-86cfb0ec-2236d744-7d286a99_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:15.090568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:15.090572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:15.090574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:15.090585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:15.090596Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:15.090622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:15.090763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.090770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.090785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:15.090905Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1160 partition: 0 MaxSeqNo: 25 sessionId: src_id|c328884d-86cfb0ec-2236d744-7d286a99_0
2025-10-09T21:48:15.091298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2d9ec8d-d4d1765a-f59bfdc3-8b729172_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c328884d-86cfb0ec-2236d744-7d286a99_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:15.093754Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c328884d-86cfb0ec-2236d744-7d286a99_0] Write session: try to update token
2025-10-09T21:48:15.093774Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c328884d-86cfb0ec-2236d744-7d286a99_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:15.105529Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:15.105546Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:15.105549Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:15.105551Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:15.105562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342136352247244:8987]
2025-10-09T21:48:15.105567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342136352247244:8987]
2025-10-09T21:48:15.105571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342136352247244:8987]
2025-10-09T21:48:15.105616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:15.105624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.105626Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:15.105628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:15.105632Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:15.105636Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:15.105782Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:15.107185Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1160 sessionId: src_id|c328884d-86cfb0ec-2236d744-7d286a99_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:15.109783Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:15.158824Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342136714069400:9750]
2025-10-09T21:48:15.159108Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 1159, pipe: [1:7559342136714069400:9750]
2025-10-09T21:48:15.266759Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:15.267110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.267122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.267178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342136714069400:9750], now have 1 active actors on pipe
2025-10-09T21:48:15.267432Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:15.267577Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:15.267594Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:15.267616Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342136714069400:9750]
2025-10-09T21:48:15.267659Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:15.267669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1160 reason: error at write: Disconnected sessionId: src_id|c328884d-86cfb0ec-2236d744-7d286a99_0
2025-10-09T21:48:15.268010Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c328884d-86cfb0ec-2236d744-7d286a99_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:15.268021Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c328884d-86cfb0ec-2236d744-7d286a99_0] Write session will restart in 0.009439s
2025-10-09T21:48:15.268050Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c328884d-86cfb0ec-2236d744-7d286a99_0] Write session: Do CDS request
2025-10-09T21:48:15.267840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1160 sessionId: src_id|c328884d-86cfb0ec-2236d744-7d286a99_0 is DEAD
2025-10-09T21:48:15.268059Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c328884d-86cfb0ec-2236d744-7d286a99_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:15.278589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c328884d-86cfb0ec-2236d744-7d286a99_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:15.278678Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:15.278697Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1161
2025-10-09T21:48:15.278846Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1161 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:15.278924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1161 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:15.278936Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1161 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:15.278940Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:15.279456Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1161 sessionId:  describe result for acl check
2025-10-09T21:48:15.279501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:15.279511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:15.279513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:15.279519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342136714069454:10097] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:15.280120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342136714069454:10097] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:15.282603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342136714069454:10097] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:15.286381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342136714069454:10097] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:15.286400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342136714069454:10097] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:15.286404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342136714069454:10097] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:15.286414Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1161 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:15.287779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:15.290834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:15.291063Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:15.291072Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:15.291085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:15.291147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:15.291190Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342136352247485:8989]
2025-10-09T21:48:15.291765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:15.292093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:15.292140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:15.292456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:15.292506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.292521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.292527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.292528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.292529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.292530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.292531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.292533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.292534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.292539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.292550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.292553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.292556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.292559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.292562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.292566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.292569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.292572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.292579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.292587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.292593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.292598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.292603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.292607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.292611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.292616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.292620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.292624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.292628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.292633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.292638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.292642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.292648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.292653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.292657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.292662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.292691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:15.299431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:15.300548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:15.300568Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:15.300570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:15.300575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:15.300582Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1160 [2:7559342136352247485:8989]
2025-10-09T21:48:15.300596Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:15.300616Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:15.300624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:15.300628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:15.300630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:15.300661Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:15.300674Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:15.300684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:15.300744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.300751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.300754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.300757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:15.300830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:15.300837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.300850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.300865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.300885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:15.300912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342136714069638:10097], now have 1 active actors on pipe
2025-10-09T21:48:15.300919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342136714069407:2432], now have 1 active actors on pipe
2025-10-09T21:48:15.300927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342136352247480:8989]
2025-10-09T21:48:15.300930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342136352247480:8989]
2025-10-09T21:48:15.300932Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342136352247480:8989]
2025-10-09T21:48:15.300945Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:15.300959Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:15.300962Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:15.300964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:15.300964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:15.300979Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:15.300987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:15.301027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342136352247480:8989]
2025-10-09T21:48:15.301038Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:15.301046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:15.301037Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1160
2025-10-09T21:48:15.301044Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1160
2025-10-09T21:48:15.301094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.301104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.301132Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3d6d2af1-92df961e-6aa9e5d3-f96d22ac_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:15.301144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:15.301147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:15.301150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:15.301160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:15.301171Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:15.301190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:15.301342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.301355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.301378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:15.301529Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1161 partition: 0 MaxSeqNo: 25 sessionId: src_id|3d6d2af1-92df961e-6aa9e5d3-f96d22ac_0
2025-10-09T21:48:15.301961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c328884d-86cfb0ec-2236d744-7d286a99_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3d6d2af1-92df961e-6aa9e5d3-f96d22ac_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:15.304504Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d6d2af1-92df961e-6aa9e5d3-f96d22ac_0] Write session: try to update token
2025-10-09T21:48:15.304524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d6d2af1-92df961e-6aa9e5d3-f96d22ac_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:15.312611Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:15.312628Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:15.312638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342136352247480:8989]
2025-10-09T21:48:15.312691Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:15.312701Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:15.312865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:15.313010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:15.313153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:15.313300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:15.313498Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:15.313596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:15.318165Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1161 sessionId: src_id|3d6d2af1-92df961e-6aa9e5d3-f96d22ac_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:15.320754Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:15.321030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.321052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.321095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:15.321193Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:15.326578Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:15.326592Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:15.326594Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:15.326596Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:15.326604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342136352247480:8989]
2025-10-09T21:48:15.326607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342136352247480:8989]
2025-10-09T21:48:15.326610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342136352247480:8989]
2025-10-09T21:48:15.326653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:15.326655Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:15.326662Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:15.326662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.326665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:15.326666Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:15.326796Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:15.367691Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342136714069652:9750]
2025-10-09T21:48:15.367990Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 1160, pipe: [1:7559342136714069652:9750]
2025-10-09T21:48:15.488274Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:15.495542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.495570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.495731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.495790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.495855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.495916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.495956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.495968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.496028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.496083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.496142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.496198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.496240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.496248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.496310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.496363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.496415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.496475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.496519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.496528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.496592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.496650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.496715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.496775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.496820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.496828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.496895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.496956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.497018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.497081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.497128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.497137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.497199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.497259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.497323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.497381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.497421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.497430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.497488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.497540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.497590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.497645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.497713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.497720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.497787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.497836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.497887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.497942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.497984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.497989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.498058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.498109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.498159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.498215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.498261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.498266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.498320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.498371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.498422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.498483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.498522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.498527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.498579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342136714069652:9750], now have 1 active actors on pipe
2025-10-09T21:48:15.498860Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:15.498874Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:15.498875Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:15.498908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1161 reason: error at write: Disconnected sessionId: src_id|3d6d2af1-92df961e-6aa9e5d3-f96d22ac_0
2025-10-09T21:48:15.498947Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342136714069652:9750]
2025-10-09T21:48:15.499077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1161 sessionId: src_id|3d6d2af1-92df961e-6aa9e5d3-f96d22ac_0 is DEAD
2025-10-09T21:48:15.499240Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d6d2af1-92df961e-6aa9e5d3-f96d22ac_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:15.499252Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d6d2af1-92df961e-6aa9e5d3-f96d22ac_0] Write session will restart in 0.009614s
2025-10-09T21:48:15.499277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d6d2af1-92df961e-6aa9e5d3-f96d22ac_0] Write session: Do CDS request
2025-10-09T21:48:15.499283Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d6d2af1-92df961e-6aa9e5d3-f96d22ac_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:15.499625Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:15.510531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d6d2af1-92df961e-6aa9e5d3-f96d22ac_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:15.510602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:15.510617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1162
2025-10-09T21:48:15.510736Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1162 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:15.510768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1162 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:15.510774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1162 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:15.510778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:15.511177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1162 sessionId:  describe result for acl check
2025-10-09T21:48:15.511221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:15.511227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:15.511229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:15.511235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342136714069739:10101] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:15.511792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342136714069739:10101] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:15.514806Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342136714069739:10101] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:15.516732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:15.519155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:15.519358Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:15.519367Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:15.519378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:15.519429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:15.519415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342136714069739:10101] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:15.519428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342136714069739:10101] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:15.519434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342136714069739:10101] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:15.519443Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1162 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:15.519469Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342136352247723:8991]
2025-10-09T21:48:15.520042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:15.521293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:15.521340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:15.521667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:15.521726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.521732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.521733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.521734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.521735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.521736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.521738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.521748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.521749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.521755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.521767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.521771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.521775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.521779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.521783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.521787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.521791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.521794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.521801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.521809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.521816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.521821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.521826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.521831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.521835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.521851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.521861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.521866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.521871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.521876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.521880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.521887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.521892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.521897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.521902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.521907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.521936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:15.528326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:15.529613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:15.529630Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:15.529632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:15.529636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:15.529644Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1161 [2:7559342136352247723:8991]
2025-10-09T21:48:15.529658Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:15.529688Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:15.529697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:15.529700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:15.529703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:15.529723Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:15.529729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:15.529736Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:15.529808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.529815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.529817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.529820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:15.529884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:15.529893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.529904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.529915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.529928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:15.529950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342136714069655:2432], now have 1 active actors on pipe
2025-10-09T21:48:15.529960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342136714069889:10101], now have 1 active actors on pipe
2025-10-09T21:48:15.529968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342136352247718:8991]
2025-10-09T21:48:15.529971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342136352247718:8991]
2025-10-09T21:48:15.529973Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342136352247718:8991]
2025-10-09T21:48:15.529984Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:15.529994Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:15.529996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:15.529999Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:15.530000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:15.530015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:15.530018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:15.530080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342136352247718:8991]
2025-10-09T21:48:15.530093Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:15.530012Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1161
2025-10-09T21:48:15.530067Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1161
2025-10-09T21:48:15.530109Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:15.530147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.530157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.530181Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e5c18874-aa64a4e8-173587b9-bd1a9bda_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:15.530194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:15.530198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:15.530200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:15.530210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:15.530220Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:15.530234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:15.530374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.530383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.530396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:15.530493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1162 partition: 0 MaxSeqNo: 25 sessionId: src_id|e5c18874-aa64a4e8-173587b9-bd1a9bda_0
2025-10-09T21:48:15.530870Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d6d2af1-92df961e-6aa9e5d3-f96d22ac_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e5c18874-aa64a4e8-173587b9-bd1a9bda_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:15.533323Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5c18874-aa64a4e8-173587b9-bd1a9bda_0] Write session: try to update token
2025-10-09T21:48:15.533341Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5c18874-aa64a4e8-173587b9-bd1a9bda_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:15.541381Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:15.541393Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:15.541403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342136352247718:8991]
2025-10-09T21:48:15.541459Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:15.541465Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:15.541634Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:15.541785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:15.541931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:15.542090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:15.542281Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:15.542323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:15.546777Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1162 sessionId: src_id|e5c18874-aa64a4e8-173587b9-bd1a9bda_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:15.549254Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:15.549461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.549480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.549521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:15.549621Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:15.554747Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:15.554761Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:15.554763Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:15.554765Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:15.554776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342136352247718:8991]
2025-10-09T21:48:15.554781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342136352247718:8991]
2025-10-09T21:48:15.554784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342136352247718:8991]
2025-10-09T21:48:15.554829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:15.554842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.554844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:15.554847Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:15.554855Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:15.554859Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:15.554978Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:15.599070Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342136714069901:9750]
2025-10-09T21:48:15.599393Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 1161, pipe: [1:7559342136714069901:9750]
2025-10-09T21:48:15.708877Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:15.716560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.716587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.716771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.716835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.716899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.716978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.717481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.717498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.717564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.717620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.717684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.717767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.717811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.717822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.717881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.717940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.717999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.718105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.718152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.718161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.718231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.718296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.718354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.718420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.718469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.718476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.718544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.718618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.718687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.718755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.718799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.718812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.718881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.718946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.719012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.719075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.719125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.719133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.719198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.719256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.719421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.719490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.719534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.719542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.719608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.719672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.719734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.719799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.719842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.719848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.719906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.719965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.720024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.720083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.720122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.720129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.720195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.720262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.720324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.720390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.720435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.720441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.720511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342136714069901:9750], now have 1 active actors on pipe
2025-10-09T21:48:15.720779Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342136714069901:9750]
2025-10-09T21:48:15.720789Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:15.720793Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:15.720824Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:15.720824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1162 reason: error at write: Disconnected sessionId: src_id|e5c18874-aa64a4e8-173587b9-bd1a9bda_0
2025-10-09T21:48:15.720876Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:15.721051Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1162 sessionId: src_id|e5c18874-aa64a4e8-173587b9-bd1a9bda_0 is DEAD
2025-10-09T21:48:15.721254Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5c18874-aa64a4e8-173587b9-bd1a9bda_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:15.721266Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5c18874-aa64a4e8-173587b9-bd1a9bda_0] Write session will restart in 0.006331s
2025-10-09T21:48:15.721292Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5c18874-aa64a4e8-173587b9-bd1a9bda_0] Write session: Do CDS request
2025-10-09T21:48:15.721300Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5c18874-aa64a4e8-173587b9-bd1a9bda_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:15.728600Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5c18874-aa64a4e8-173587b9-bd1a9bda_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:15.728620Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:15.728649Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1163
2025-10-09T21:48:15.728913Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1163 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:15.728943Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1163 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:15.728951Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1163 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:15.728954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:15.729464Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1163 sessionId:  describe result for acl check
2025-10-09T21:48:15.729501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:15.729504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:15.729506Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:15.729513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342136714069918:10105] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:15.730114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342136714069918:10105] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:15.733993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342136714069918:10105] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:15.737656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342136714069918:10105] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:15.737671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342136714069918:10105] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:15.737678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342136714069918:10105] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:15.737688Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1163 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:15.740968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:15.743632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:15.743862Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:15.743874Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:15.743888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:15.743955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:15.744003Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342136352247965:8997]
2025-10-09T21:48:15.744619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:15.744976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:15.745050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:15.745388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:15.745450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.745457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.745458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.745459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.745461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.745462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.745463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.745465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.745466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.745472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.745484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.745487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.745491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.745495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.745498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.745502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.745506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.745509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.745518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.745527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.745535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.745542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.745548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.745554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.745558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.745563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.745568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.745574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.745579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.745585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.745591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.745596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.745601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.745606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.745611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.745616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.745653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:15.752575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:15.753663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:15.753678Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:15.753681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:15.753686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:15.753694Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1162 [2:7559342136352247965:8997]
2025-10-09T21:48:15.753725Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:15.753756Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:15.753766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:15.753769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:15.753772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:15.753791Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:15.753797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:15.753807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:15.753864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.753875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.753877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.753881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:15.753949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:15.753958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.753969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.753981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.753998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:15.754018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342136352247960:8997]
2025-10-09T21:48:15.754025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342136352247960:8997]
2025-10-09T21:48:15.754043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342136352247960:8997]
2025-10-09T21:48:15.754057Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:15.754068Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:15.754071Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:15.754073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:15.754079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342136714070098:10105], now have 1 active actors on pipe
2025-10-09T21:48:15.754083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342136714069903:2432], now have 1 active actors on pipe
2025-10-09T21:48:15.754092Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:15.754110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:15.754118Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:15.754117Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1162
2025-10-09T21:48:15.754154Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342136352247960:8997]
2025-10-09T21:48:15.754119Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1162
2025-10-09T21:48:15.754170Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:15.754188Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:15.754210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.754215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.754251Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|909bc903-8ad37ae-ff46ba64-c25810bb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:15.754266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:15.754270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:15.754273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:15.754285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:15.754308Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:15.754326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:15.754510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.754529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.754550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:15.754673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1163 partition: 0 MaxSeqNo: 25 sessionId: src_id|909bc903-8ad37ae-ff46ba64-c25810bb_0
2025-10-09T21:48:15.755056Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5c18874-aa64a4e8-173587b9-bd1a9bda_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|909bc903-8ad37ae-ff46ba64-c25810bb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:15.757561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|909bc903-8ad37ae-ff46ba64-c25810bb_0] Write session: try to update token
2025-10-09T21:48:15.757579Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|909bc903-8ad37ae-ff46ba64-c25810bb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:15.766271Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:15.766286Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:15.766297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342136352247960:8997]
2025-10-09T21:48:15.766342Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:15.766347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:15.766522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:15.766671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:15.766807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:15.766951Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:15.767160Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:15.767213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:15.771763Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1163 sessionId: src_id|909bc903-8ad37ae-ff46ba64-c25810bb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:15.774289Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:15.774563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.774585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.774627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:15.774713Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:15.780014Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:15.780033Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:15.780036Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:15.780038Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:15.780050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342136352247960:8997]
2025-10-09T21:48:15.780054Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342136352247960:8997]
2025-10-09T21:48:15.780059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342136352247960:8997]
2025-10-09T21:48:15.780107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:15.780115Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:15.780117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.780120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:15.780122Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:15.780126Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:15.780266Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:15.820814Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342136714070152:9750]
2025-10-09T21:48:15.821111Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 1162, pipe: [1:7559342136714070152:9750]
2025-10-09T21:48:15.941193Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:15.949863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.949896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.950085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.950152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.950230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.950304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.950352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.950358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.950436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.950514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.950590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.950666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.950718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.950721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.950795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.950867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.950938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.951011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.951058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.951061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.951143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.951211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.951283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.951357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.951400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.951404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.951482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.951544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.951604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.951665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.951709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.951712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.951779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.951843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.951915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.951982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.952022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.952025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.952091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.952149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.952213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.952276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.952319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.952325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.952391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.952451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.952509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.952566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.952607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.952611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.952677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.952736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.952800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.952863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.952907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.952913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.952983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:15.953040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:15.953102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:15.953161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:15.953205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:15.953208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:15.953296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342136714070152:9750], now have 1 active actors on pipe
2025-10-09T21:48:15.953533Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342136714070152:9750]
2025-10-09T21:48:15.953537Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:15.953582Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:15.953585Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:15.953615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1163 reason: error at write: Disconnected sessionId: src_id|909bc903-8ad37ae-ff46ba64-c25810bb_0
2025-10-09T21:48:15.953857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1163 sessionId: src_id|909bc903-8ad37ae-ff46ba64-c25810bb_0 is DEAD
2025-10-09T21:48:15.954010Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|909bc903-8ad37ae-ff46ba64-c25810bb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:15.954021Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|909bc903-8ad37ae-ff46ba64-c25810bb_0] Write session will restart in 0.008890s
2025-10-09T21:48:15.954020Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:15.954061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|909bc903-8ad37ae-ff46ba64-c25810bb_0] Write session: Do CDS request
2025-10-09T21:48:15.954068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|909bc903-8ad37ae-ff46ba64-c25810bb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:15.964555Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|909bc903-8ad37ae-ff46ba64-c25810bb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:15.964648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:15.964666Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1164
2025-10-09T21:48:15.964864Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1164 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:15.964911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1164 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:15.964921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1164 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:15.964925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:15.965360Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1164 sessionId:  describe result for acl check
2025-10-09T21:48:15.965400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:15.965402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:15.965403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:15.965409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342136714070213:10112] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:15.965974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342136714070213:10112] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:15.968696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342136714070213:10112] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:15.972081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342136714070213:10112] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:15.972098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342136714070213:10112] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:15.972102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342136714070213:10112] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:15.972112Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1164 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:15.974349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:15.977199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:15.977426Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:15.977436Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:15.977448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:15.977501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:15.977542Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342136352248234:9009]
2025-10-09T21:48:15.978171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:15.978632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:15.978685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:15.979008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:15.979075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.979082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.979083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.979084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.979085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.979086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.979088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.979089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.979090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.979096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.979107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.979111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.979115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.979119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.979124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.979128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.979132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.979136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.979144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.979152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:15.979158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.979164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:15.979173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.979179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:15.979188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.979194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:15.979198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.979204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:15.979210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.979215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:15.979225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.979231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.979235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.979245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.979250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.979255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.979290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:15.986805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:15.988205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:15.988219Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:15.988221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:15.988225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:15.988231Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1163 [2:7559342136352248234:9009]
2025-10-09T21:48:15.988243Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:15.988257Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:15.988259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:15.988261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:15.988262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:15.988277Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:15.988280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:15.988286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:15.988332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.988339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.988341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.988342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:15.988401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:15.988403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:15.988415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:15.988428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:15.988450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:15.988473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342136714070391:10112], now have 1 active actors on pipe
2025-10-09T21:48:15.988476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342136714070157:2432], now have 1 active actors on pipe
2025-10-09T21:48:15.988483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342136352248229:9009]
2025-10-09T21:48:15.988486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342136352248229:9009]
2025-10-09T21:48:15.988489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342136352248229:9009]
2025-10-09T21:48:15.988503Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:15.988518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:15.988526Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:15.988531Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:15.988533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:15.988533Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:15.988535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:15.988574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342136352248229:9009]
2025-10-09T21:48:15.988589Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:15.988596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:15.988578Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1163
2025-10-09T21:48:15.988615Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1163
2025-10-09T21:48:15.988757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.988769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.988803Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d0f81a36-a63e4e5a-1f00b583-86a3c7c7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:15.988821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:15.988825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:15.988829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:15.988843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:15.988854Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:15.988878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:15.989088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:15.989099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:15.989117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:15.989248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1164 partition: 0 MaxSeqNo: 25 sessionId: src_id|d0f81a36-a63e4e5a-1f00b583-86a3c7c7_0
2025-10-09T21:48:15.989633Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|909bc903-8ad37ae-ff46ba64-c25810bb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d0f81a36-a63e4e5a-1f00b583-86a3c7c7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:15.992232Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d0f81a36-a63e4e5a-1f00b583-86a3c7c7_0] Write session: try to update token
2025-10-09T21:48:15.992254Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d0f81a36-a63e4e5a-1f00b583-86a3c7c7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:16.003859Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:16.003874Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:16.003877Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:16.003878Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:16.003888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342136352248229:9009]
2025-10-09T21:48:16.003893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342136352248229:9009]
2025-10-09T21:48:16.003898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342136352248229:9009]
2025-10-09T21:48:16.003943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:16.003944Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:16.003951Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:16.003951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.003954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:16.003954Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:16.004100Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:16.007059Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1164 sessionId: src_id|d0f81a36-a63e4e5a-1f00b583-86a3c7c7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:16.009716Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:16.053781Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342141009037722:9750]
2025-10-09T21:48:16.054054Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 1163, pipe: [1:7559342141009037722:9750]
2025-10-09T21:48:16.159228Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:16.159692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:16.159708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:16.159753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342141009037722:9750], now have 1 active actors on pipe
2025-10-09T21:48:16.159939Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:16.159946Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:16.159951Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:16.159961Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:16.160006Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342141009037722:9750]
2025-10-09T21:48:16.160023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1164 reason: error at write: Disconnected sessionId: src_id|d0f81a36-a63e4e5a-1f00b583-86a3c7c7_0
2025-10-09T21:48:16.160256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1164 sessionId: src_id|d0f81a36-a63e4e5a-1f00b583-86a3c7c7_0 is DEAD
2025-10-09T21:48:16.160389Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d0f81a36-a63e4e5a-1f00b583-86a3c7c7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:16.160403Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d0f81a36-a63e4e5a-1f00b583-86a3c7c7_0] Write session will restart in 0.006935s
2025-10-09T21:48:16.160431Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d0f81a36-a63e4e5a-1f00b583-86a3c7c7_0] Write session: Do CDS request
2025-10-09T21:48:16.160438Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d0f81a36-a63e4e5a-1f00b583-86a3c7c7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:16.168558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d0f81a36-a63e4e5a-1f00b583-86a3c7c7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:16.168661Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:16.168676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1165
2025-10-09T21:48:16.168830Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1165 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:16.168863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1165 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:16.168873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1165 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:16.168877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:16.169388Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1165 sessionId:  describe result for acl check
2025-10-09T21:48:16.169433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:16.169442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:16.169444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:16.169449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342141009037770:10121] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:16.169985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342141009037770:10121] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:16.172828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342141009037770:10121] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:16.176076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342141009037770:10121] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:16.176091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342141009037770:10121] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:16.176094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342141009037770:10121] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:16.176103Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1165 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:16.177129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:16.179601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:16.179809Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:16.179818Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:16.179831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:16.179898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:16.179939Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342140647215777:9013]
2025-10-09T21:48:16.180546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:16.180888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:16.180947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:16.181283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:16.181344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:16.181352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:16.181353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:16.181354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:16.181356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:16.181357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:16.181358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.181359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.181360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.181365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:16.181376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:16.181379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:16.181383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:16.181386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:16.181389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:16.181392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.181395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.181398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.181405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:16.181414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:16.181419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:16.181423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:16.181427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:16.181432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:16.181436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:16.181440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:16.181445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:16.181449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:16.181454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:16.181459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:16.181464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.181469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.181475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.181485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.181490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.181500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.181530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:16.188220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:16.189283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:16.189298Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:16.189300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:16.189305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:16.189312Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1164 [2:7559342140647215777:9013]
2025-10-09T21:48:16.189326Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:16.189346Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:16.189354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:16.189357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:16.189360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:16.189380Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:16.189391Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:16.189403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:16.189471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.189481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.189482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.189485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:16.189558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:16.189568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.189579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.189591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.189610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:16.189636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342141009037727:2432], now have 1 active actors on pipe
2025-10-09T21:48:16.189645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342141009037962:10121], now have 1 active actors on pipe
2025-10-09T21:48:16.189652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342140647215772:9013]
2025-10-09T21:48:16.189655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342140647215772:9013]
2025-10-09T21:48:16.189657Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342140647215772:9013]
2025-10-09T21:48:16.189670Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:16.189688Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:16.189688Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:16.189692Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:16.189694Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:16.189707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:16.189710Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:16.189712Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1164
2025-10-09T21:48:16.189722Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1164
2025-10-09T21:48:16.189762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342140647215772:9013]
2025-10-09T21:48:16.189773Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:16.189780Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:16.189790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:16.189796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:16.189850Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4558946f-17fa34c3-f83369df-24cb6e97_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:16.189863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:16.189867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:16.189870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:16.189881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:16.189895Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:16.189914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:16.190090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:16.190099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:16.190115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:16.190228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1165 partition: 0 MaxSeqNo: 25 sessionId: src_id|4558946f-17fa34c3-f83369df-24cb6e97_0
2025-10-09T21:48:16.190594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d0f81a36-a63e4e5a-1f00b583-86a3c7c7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4558946f-17fa34c3-f83369df-24cb6e97_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:16.193078Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4558946f-17fa34c3-f83369df-24cb6e97_0] Write session: try to update token
2025-10-09T21:48:16.193095Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4558946f-17fa34c3-f83369df-24cb6e97_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:16.204583Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:16.204600Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:16.204602Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:16.204605Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:16.204614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342140647215772:9013]
2025-10-09T21:48:16.204618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342140647215772:9013]
2025-10-09T21:48:16.204623Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342140647215772:9013]
2025-10-09T21:48:16.204667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:16.204676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.204679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:16.204687Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:16.204700Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:16.204704Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:16.204808Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:16.206380Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1165 sessionId: src_id|4558946f-17fa34c3-f83369df-24cb6e97_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:16.208688Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:16.260106Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342141009037981:9750]
2025-10-09T21:48:16.260375Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 1164, pipe: [1:7559342141009037981:9750]
2025-10-09T21:48:16.359007Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:16.359736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:16.359753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:16.359802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342141009037981:9750], now have 1 active actors on pipe
2025-10-09T21:48:16.360028Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:16.360076Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:16.360094Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:16.360106Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:16.360164Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342141009037981:9750]
2025-10-09T21:48:16.360170Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1165 reason: error at write: Disconnected sessionId: src_id|4558946f-17fa34c3-f83369df-24cb6e97_0
2025-10-09T21:48:16.360362Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1165 sessionId: src_id|4558946f-17fa34c3-f83369df-24cb6e97_0 is DEAD
2025-10-09T21:48:16.360507Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4558946f-17fa34c3-f83369df-24cb6e97_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:16.360516Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4558946f-17fa34c3-f83369df-24cb6e97_0] Write session will restart in 0.007875s
2025-10-09T21:48:16.360539Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4558946f-17fa34c3-f83369df-24cb6e97_0] Write session: Do CDS request
2025-10-09T21:48:16.360544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4558946f-17fa34c3-f83369df-24cb6e97_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:16.369569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4558946f-17fa34c3-f83369df-24cb6e97_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:16.369673Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:16.369688Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1166
2025-10-09T21:48:16.369862Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1166 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:16.369918Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1166 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:16.369931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1166 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:16.369935Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:16.370484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1166 sessionId:  describe result for acl check
2025-10-09T21:48:16.370522Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:16.370529Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:16.370530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:16.370536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342141009038018:10126] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:16.371092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342141009038018:10126] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:16.374290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342141009038018:10126] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:16.377400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342141009038018:10126] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:16.377419Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342141009038018:10126] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:16.377423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342141009038018:10126] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:16.377434Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1166 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:16.379329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:16.381833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:16.382059Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:16.382070Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:16.382084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:16.382139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:16.382186Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342140647216017:9015]
2025-10-09T21:48:16.382792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:16.383131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:16.383201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:16.383567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:16.383631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:16.383637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:16.383638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:16.383639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:16.383640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:16.383642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:16.383643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.383645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.383646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.383652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:16.383663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:16.383667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:16.383672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:16.383676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:16.383680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:16.383684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.383688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.383691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.383699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:16.383707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:16.383714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:16.383725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:16.383730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:16.383735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:16.383742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:16.383747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:16.383757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:16.383762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:16.383767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:16.383773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:16.383780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.383786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.383796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.383801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.383806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.383811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.383843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:16.390306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:16.391679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:16.391694Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:16.391697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:16.391701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:16.391708Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1165 [2:7559342140647216017:9015]
2025-10-09T21:48:16.391721Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:16.391743Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:16.391750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:16.391754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:16.391757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:16.391773Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:16.391784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:16.391791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:16.391851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.391857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.391859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.391862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:16.391936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:16.391944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.391957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.391973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.391995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:16.392021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342141009038218:10126], now have 1 active actors on pipe
2025-10-09T21:48:16.392024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342141009037983:2432], now have 1 active actors on pipe
2025-10-09T21:48:16.392033Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342140647216012:9015]
2025-10-09T21:48:16.392035Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342140647216012:9015]
2025-10-09T21:48:16.392038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342140647216012:9015]
2025-10-09T21:48:16.392052Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:16.392070Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:16.392071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:16.392073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:16.392075Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:16.392085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:16.392088Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:16.392118Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342140647216012:9015]
2025-10-09T21:48:16.392129Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:16.392115Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1165
2025-10-09T21:48:16.392116Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1165
2025-10-09T21:48:16.392153Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:16.392160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:16.392165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:16.392198Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|eba76fa4-7b85cfb9-95c60cc2-6b57cbd2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:16.392213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:16.392222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:16.392224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:16.392236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:16.392251Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:16.392271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:16.392403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:16.392413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:16.392429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:16.392508Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1166 partition: 0 MaxSeqNo: 25 sessionId: src_id|eba76fa4-7b85cfb9-95c60cc2-6b57cbd2_0
2025-10-09T21:48:16.392891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4558946f-17fa34c3-f83369df-24cb6e97_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|eba76fa4-7b85cfb9-95c60cc2-6b57cbd2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:16.395410Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eba76fa4-7b85cfb9-95c60cc2-6b57cbd2_0] Write session: try to update token
2025-10-09T21:48:16.395431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eba76fa4-7b85cfb9-95c60cc2-6b57cbd2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:16.404113Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:16.404131Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:16.404141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342140647216012:9015]
2025-10-09T21:48:16.404187Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:16.404199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:16.404354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:16.404505Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:16.404636Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:16.404761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:16.404952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:16.404994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:16.408466Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1166 sessionId: src_id|eba76fa4-7b85cfb9-95c60cc2-6b57cbd2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:16.410858Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:16.411061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:16.411081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:16.411120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:16.411203Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:16.417367Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:16.417385Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:16.417388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:16.417390Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:16.417403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342140647216012:9015]
2025-10-09T21:48:16.417407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342140647216012:9015]
2025-10-09T21:48:16.417411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342140647216012:9015]
2025-10-09T21:48:16.417456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:16.417458Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:16.417465Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:16.417467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.417470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:16.417480Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:16.417606Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:16.460287Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342141009038233:9750]
2025-10-09T21:48:16.460561Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 1165, pipe: [1:7559342141009038233:9750]
2025-10-09T21:48:16.579050Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:16.587020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:16.587045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:16.587206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:16.587269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:16.587337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:16.587405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:16.587454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:16.587466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:16.587539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:16.587605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:16.587669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:16.587737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:16.587782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:16.587791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:16.587853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:16.587916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:16.587979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:16.588043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:16.588088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:16.588091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:16.588159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:16.588233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:16.588296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:16.588354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:16.588397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:16.588405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:16.588475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:16.588547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:16.588611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:16.588675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:16.588718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:16.588722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:16.588786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:16.588857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:16.588923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:16.588986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:16.589035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:16.589044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:16.589105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:16.589171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:16.589232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:16.589291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:16.589333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:16.589341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:16.589400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:16.589458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:16.589518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:16.589582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:16.589624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:16.589628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:16.589686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:16.589761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:16.589830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:16.589890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:16.589938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:16.589946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:16.590008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:16.590083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:16.590142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:16.590206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:16.590259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:16.590268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:16.590336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342141009038233:9750], now have 1 active actors on pipe
2025-10-09T21:48:16.590618Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342141009038233:9750]
2025-10-09T21:48:16.590656Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:16.590870Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:16.590885Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:16.590915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1166 reason: error at write: Disconnected sessionId: src_id|eba76fa4-7b85cfb9-95c60cc2-6b57cbd2_0
2025-10-09T21:48:16.591111Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1166 sessionId: src_id|eba76fa4-7b85cfb9-95c60cc2-6b57cbd2_0 is DEAD
2025-10-09T21:48:16.591255Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eba76fa4-7b85cfb9-95c60cc2-6b57cbd2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:16.591264Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eba76fa4-7b85cfb9-95c60cc2-6b57cbd2_0] Write session will restart in 0.007964s
2025-10-09T21:48:16.591291Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eba76fa4-7b85cfb9-95c60cc2-6b57cbd2_0] Write session: Do CDS request
2025-10-09T21:48:16.591298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eba76fa4-7b85cfb9-95c60cc2-6b57cbd2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:16.591400Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:16.600527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eba76fa4-7b85cfb9-95c60cc2-6b57cbd2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:16.600572Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:16.600586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1167
2025-10-09T21:48:16.600763Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1167 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:16.600808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1167 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:16.600819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1167 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:16.600823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:16.601256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1167 sessionId:  describe result for acl check
2025-10-09T21:48:16.601295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:16.601297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:16.601299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:16.601305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342141009038282:10130] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:16.601875Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342141009038282:10130] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:16.605285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342141009038282:10130] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:16.608075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342141009038282:10130] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:16.608089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342141009038282:10130] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:16.608093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342141009038282:10130] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:16.608101Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1167 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:16.609352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:16.611725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:16.611951Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:16.611962Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:16.611978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:16.612048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:16.612094Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342140647216259:9017]
2025-10-09T21:48:16.612698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:16.613081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:16.613125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:16.613449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:16.613494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:16.613501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:16.613502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:16.613504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:16.613505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:16.613506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:16.613507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.613509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.613510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.613515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:16.613527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:16.613531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:16.613535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:16.613539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:16.613544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:16.613548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.613552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.613556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.613562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:16.613570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:16.613576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:16.613582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:16.613586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:16.613592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:16.613596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:16.613601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:16.613606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:16.613610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:16.613613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:16.613618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:16.613623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.613634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.613639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.613644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.613649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.613655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.613688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:16.620074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:16.621142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:16.621156Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:16.621159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:16.621164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:16.621171Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1166 [2:7559342140647216259:9017]
2025-10-09T21:48:16.621184Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:16.621203Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:16.621206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:16.621210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:16.621212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:16.621231Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:16.621244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:16.621256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:16.621312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.621321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.621323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.621326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:16.621394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:16.621397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.621408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.621425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.621449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:16.621473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342140647216254:9017]
2025-10-09T21:48:16.621480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342140647216254:9017]
2025-10-09T21:48:16.621482Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342140647216254:9017]
2025-10-09T21:48:16.621492Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:16.621501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:16.621506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:16.621519Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:16.621521Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:16.621521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:16.621525Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:16.621537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342141009038472:10130], now have 1 active actors on pipe
2025-10-09T21:48:16.621546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342141009038236:2432], now have 1 active actors on pipe
2025-10-09T21:48:16.621576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342140647216254:9017]
2025-10-09T21:48:16.621588Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:16.621595Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:16.621603Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1166
2025-10-09T21:48:16.621603Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1166
2025-10-09T21:48:16.621785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:16.621798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:16.621828Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|53287c05-cbe33f52-14d20d8b-e7c8f17c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:16.621836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:16.621840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:16.621843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:16.621854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:16.621869Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:16.621888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:16.622122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:16.622138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:16.622161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:16.622290Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1167 partition: 0 MaxSeqNo: 25 sessionId: src_id|53287c05-cbe33f52-14d20d8b-e7c8f17c_0
2025-10-09T21:48:16.622660Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eba76fa4-7b85cfb9-95c60cc2-6b57cbd2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|53287c05-cbe33f52-14d20d8b-e7c8f17c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:16.625106Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53287c05-cbe33f52-14d20d8b-e7c8f17c_0] Write session: try to update token
2025-10-09T21:48:16.625126Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53287c05-cbe33f52-14d20d8b-e7c8f17c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:16.635981Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:16.635998Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:16.636000Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:16.636002Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:16.636013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342140647216254:9017]
2025-10-09T21:48:16.636018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342140647216254:9017]
2025-10-09T21:48:16.636023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342140647216254:9017]
2025-10-09T21:48:16.636069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:16.636077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.636079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:16.636078Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:16.636085Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:16.636090Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:16.636218Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:16.638441Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1167 sessionId: src_id|53287c05-cbe33f52-14d20d8b-e7c8f17c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:16.640862Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:16.690704Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342141009038486:9750]
2025-10-09T21:48:16.691039Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 1166, pipe: [1:7559342141009038486:9750]
2025-10-09T21:48:16.798120Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:16.798921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:16.798938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:16.798983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342141009038486:9750], now have 1 active actors on pipe
2025-10-09T21:48:16.799175Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:16.799220Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342141009038486:9750]
2025-10-09T21:48:16.799226Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:16.799231Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:16.799254Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:16.799294Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1167 reason: error at write: Disconnected sessionId: src_id|53287c05-cbe33f52-14d20d8b-e7c8f17c_0
2025-10-09T21:48:16.799466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1167 sessionId: src_id|53287c05-cbe33f52-14d20d8b-e7c8f17c_0 is DEAD
2025-10-09T21:48:16.799614Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53287c05-cbe33f52-14d20d8b-e7c8f17c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:16.799625Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53287c05-cbe33f52-14d20d8b-e7c8f17c_0] Write session will restart in 0.005893s
2025-10-09T21:48:16.799652Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53287c05-cbe33f52-14d20d8b-e7c8f17c_0] Write session: Do CDS request
2025-10-09T21:48:16.799659Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53287c05-cbe33f52-14d20d8b-e7c8f17c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:16.806539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53287c05-cbe33f52-14d20d8b-e7c8f17c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:16.806620Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:16.806637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1168
2025-10-09T21:48:16.806782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1168 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:16.806829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1168 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:16.806840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1168 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:16.806844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:16.807344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1168 sessionId:  describe result for acl check
2025-10-09T21:48:16.807385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:16.807388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:16.807390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:16.807396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342141009038509:10136] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:16.807982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342141009038509:10136] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:16.810452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342141009038509:10136] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:16.813583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342141009038509:10136] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:16.813599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342141009038509:10136] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:16.813603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342141009038509:10136] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:16.813612Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1168 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:16.816359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:16.819266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:16.819503Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:16.819514Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:16.819526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:16.819589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:16.819640Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342140647216506:9021]
2025-10-09T21:48:16.820226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:16.820591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:16.820643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:16.820989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:16.821046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:16.821051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:16.821052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:16.821053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:16.821054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:16.821055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:16.821056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.821057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.821059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.821064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:16.821075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:16.821078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:16.821082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:16.821085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:16.821089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:16.821093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.821096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.821099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.821106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:16.821114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:16.821120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:16.821126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:16.821133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:16.821138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:16.821147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:16.821153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:16.821157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:16.821163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:16.821171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:16.821176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:16.821182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.821192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.821197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.821203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.821208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.821217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.821247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:16.827919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:16.828953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:16.828974Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:16.828977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:16.828982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:16.828990Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1167 [2:7559342140647216506:9021]
2025-10-09T21:48:16.829005Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:16.829025Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:16.829033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:16.829037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:16.829040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:16.829059Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:16.829066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:16.829073Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:16.829130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.829138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.829141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.829144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:16.829218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:16.829226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.829239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:16.829254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:16.829276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:16.829303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342141009038492:2432], now have 1 active actors on pipe
2025-10-09T21:48:16.829311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342141009038699:10136], now have 1 active actors on pipe
2025-10-09T21:48:16.829319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342140647216501:9021]
2025-10-09T21:48:16.829321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342140647216501:9021]
2025-10-09T21:48:16.829324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342140647216501:9021]
2025-10-09T21:48:16.829338Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:16.829357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:16.829358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:16.829361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:16.829363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:16.829377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:16.829380Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:16.829377Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1167
2025-10-09T21:48:16.829390Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1167
2025-10-09T21:48:16.829419Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342140647216501:9021]
2025-10-09T21:48:16.829429Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:16.829437Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:16.829469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:16.829479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:16.829527Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2df3d666-db0f4dc5-e2671ee5-730e3b53_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:16.829541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:16.829545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:16.829548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:16.829559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:16.829573Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:16.829593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:16.829776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:16.829785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:16.829800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:16.829909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1168 partition: 0 MaxSeqNo: 25 sessionId: src_id|2df3d666-db0f4dc5-e2671ee5-730e3b53_0
2025-10-09T21:48:16.830276Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53287c05-cbe33f52-14d20d8b-e7c8f17c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2df3d666-db0f4dc5-e2671ee5-730e3b53_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:16.832647Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2df3d666-db0f4dc5-e2671ee5-730e3b53_0] Write session: try to update token
2025-10-09T21:48:16.832667Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2df3d666-db0f4dc5-e2671ee5-730e3b53_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:16.843648Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:16.843665Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:16.843668Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:16.843670Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:16.843680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342140647216501:9021]
2025-10-09T21:48:16.843685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342140647216501:9021]
2025-10-09T21:48:16.843690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342140647216501:9021]
2025-10-09T21:48:16.843735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:16.843743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:16.843746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:16.843745Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:16.843752Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:16.843756Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:16.843882Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:16.845877Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1168 sessionId: src_id|2df3d666-db0f4dc5-e2671ee5-730e3b53_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:16.848222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:16.899588Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342141009038745:9750]
2025-10-09T21:48:16.899831Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 1167, pipe: [1:7559342141009038745:9750]
2025-10-09T21:48:17.001750Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:17.002519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.002539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.002591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342141009038745:9750], now have 1 active actors on pipe
2025-10-09T21:48:17.002869Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:17.002889Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342141009038745:9750]
2025-10-09T21:48:17.002890Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:17.002921Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:17.002995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1168 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2df3d666-db0f4dc5-e2671ee5-730e3b53_0
2025-10-09T21:48:17.003184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1168 sessionId: src_id|2df3d666-db0f4dc5-e2671ee5-730e3b53_0 is DEAD
2025-10-09T21:48:17.003335Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2df3d666-db0f4dc5-e2671ee5-730e3b53_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:17.003346Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2df3d666-db0f4dc5-e2671ee5-730e3b53_0] Write session will restart in 0.007896s
2025-10-09T21:48:17.003387Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2df3d666-db0f4dc5-e2671ee5-730e3b53_0] Write session: Do CDS request
2025-10-09T21:48:17.003394Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2df3d666-db0f4dc5-e2671ee5-730e3b53_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:17.012597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2df3d666-db0f4dc5-e2671ee5-730e3b53_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:17.012661Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:17.012677Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1169
2025-10-09T21:48:17.012863Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1169 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:17.012907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1169 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:17.012918Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1169 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:17.012925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:17.013327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1169 sessionId:  describe result for acl check
2025-10-09T21:48:17.013363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:17.013371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:17.013372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:17.013379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342145304006085:10143] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:17.013825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342145304006085:10143] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:17.016174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342145304006085:10143] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:17.019494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342145304006085:10143] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:17.019509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342145304006085:10143] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:17.019513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342145304006085:10143] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:17.019521Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1169 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:17.022519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:17.025050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:17.025306Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:17.025318Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:17.025332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:17.025393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:17.025436Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342144942184075:9033]
2025-10-09T21:48:17.026061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:17.026428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:17.026489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:17.026837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:17.026900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.026907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.026908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.026909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.026911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.026912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.026913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.026914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.026915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.026922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.026932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.026936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.026941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.026945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.026949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.026953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.026957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.026960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.026968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.026976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.026982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.026987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.026997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.027003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.027007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.027012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.027017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.027022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.027027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.027032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.027037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.027042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.027046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.027052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.027061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.027067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.027108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:17.034048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:17.035112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:17.035130Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:17.035132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:17.035136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:17.035143Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1168 [2:7559342144942184075:9033]
2025-10-09T21:48:17.035160Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:17.035180Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:17.035188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:17.035191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:17.035193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:17.035211Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:17.035225Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:17.035233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:17.035291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.035301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.035303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.035306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:17.035546Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1168
2025-10-09T21:48:17.035583Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1168
2025-10-09T21:48:17.035611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:17.035621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.035636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.035649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.035671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:17.035697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342145304006271:10143], now have 1 active actors on pipe
2025-10-09T21:48:17.035705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342145304006047:2432], now have 1 active actors on pipe
2025-10-09T21:48:17.035723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.035731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.035753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342144942184070:9033]
2025-10-09T21:48:17.035761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342144942184070:9033]
2025-10-09T21:48:17.035763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342144942184070:9033]
2025-10-09T21:48:17.035776Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:17.035791Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:17.035795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:17.035800Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:17.035802Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:17.035809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:17.035812Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:17.035829Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|17f221bf-20040c7b-2fd8eb45-ca7df668_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:17.035837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:17.035841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:17.035844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:17.035854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:17.035871Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:17.035913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342144942184070:9033]
2025-10-09T21:48:17.035923Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:17.035929Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:17.035939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:17.036099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.036107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.036121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:17.036205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1169 partition: 0 MaxSeqNo: 25 sessionId: src_id|17f221bf-20040c7b-2fd8eb45-ca7df668_0
2025-10-09T21:48:17.036565Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2df3d666-db0f4dc5-e2671ee5-730e3b53_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|17f221bf-20040c7b-2fd8eb45-ca7df668_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:17.039098Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|17f221bf-20040c7b-2fd8eb45-ca7df668_0] Write session: try to update token
2025-10-09T21:48:17.039116Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|17f221bf-20040c7b-2fd8eb45-ca7df668_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:17.047743Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:17.047758Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:17.047767Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342144942184070:9033]
2025-10-09T21:48:17.047810Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:17.047812Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:17.047990Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:17.048125Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:17.048258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:17.048393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:17.048680Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:17.048728Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:17.052403Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1169 sessionId: src_id|17f221bf-20040c7b-2fd8eb45-ca7df668_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:17.054877Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:17.055084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.055114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.055155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:17.055259Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:17.061084Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:17.061098Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:17.061100Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:17.061102Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:17.061112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342144942184070:9033]
2025-10-09T21:48:17.061115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342144942184070:9033]
2025-10-09T21:48:17.061118Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342144942184070:9033]
2025-10-09T21:48:17.061174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:17.061181Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:17.061185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.061187Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:17.061188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:17.061190Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:17.061317Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:17.102847Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342145304006319:9750]
2025-10-09T21:48:17.103288Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 1168, pipe: [1:7559342145304006319:9750]
2025-10-09T21:48:17.222935Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:17.231607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.231630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.231828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:17.231897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:17.231965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:17.232034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:17.232082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:17.232094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:17.232161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:17.232257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:17.232319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:17.232382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:17.232423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:17.232430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:17.232491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:17.232550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:17.232609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:17.232666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:17.232707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:17.232721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:17.232791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:17.232865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:17.232932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:17.232992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:17.233038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:17.233046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:17.233111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:17.233179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:17.233241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:17.233316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:17.233356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:17.233364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:17.233422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:17.233481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:17.233541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:17.233603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:17.233652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:17.233659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:17.233718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:17.233785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:17.233845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:17.233906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:17.233944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:17.233949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:17.234012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:17.234084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:17.234145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:17.234202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:17.234241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:17.234248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:17.234306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:17.234361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:17.234420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:17.234481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:17.234523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:17.234530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:17.234591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:17.234653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:17.234711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:17.234768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:17.234817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:17.234826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:17.234887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342145304006319:9750], now have 1 active actors on pipe
2025-10-09T21:48:17.235119Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:17.235130Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:17.235139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:17.235171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1169 reason: error at write: Disconnected sessionId: src_id|17f221bf-20040c7b-2fd8eb45-ca7df668_0
2025-10-09T21:48:17.235193Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342145304006319:9750]
2025-10-09T21:48:17.235366Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1169 sessionId: src_id|17f221bf-20040c7b-2fd8eb45-ca7df668_0 is DEAD
2025-10-09T21:48:17.235539Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|17f221bf-20040c7b-2fd8eb45-ca7df668_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:17.235549Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|17f221bf-20040c7b-2fd8eb45-ca7df668_0] Write session will restart in 0.005456s
2025-10-09T21:48:17.235578Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|17f221bf-20040c7b-2fd8eb45-ca7df668_0] Write session: Do CDS request
2025-10-09T21:48:17.235595Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|17f221bf-20040c7b-2fd8eb45-ca7df668_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:17.235870Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:17.242523Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|17f221bf-20040c7b-2fd8eb45-ca7df668_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:17.242612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:17.242625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1170
2025-10-09T21:48:17.242740Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1170 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:17.242773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1170 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:17.242779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1170 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:17.242782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:17.243258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1170 sessionId:  describe result for acl check
2025-10-09T21:48:17.243296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:17.243298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:17.243299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:17.243305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342145304006344:10153] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:17.243931Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342145304006344:10153] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:17.247721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342145304006344:10153] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:17.251340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342145304006344:10153] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:17.251356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342145304006344:10153] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:17.251359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342145304006344:10153] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:17.251367Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1170 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:17.253769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:17.256265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:17.256467Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:17.256477Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:17.256488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:17.256540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:17.256579Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342144942184327:9037]
2025-10-09T21:48:17.257179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:17.257513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:17.257567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:17.257922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:17.257984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.257990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.257991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.257993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.257994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.257995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.257996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.257998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.257999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.258004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.258014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.258023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.258045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.258049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.258053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.258057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.258061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.258065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.258072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.258085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.258096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.258101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.258105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.258115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.258119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.258124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.258129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.258133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.258137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.258142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.258147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.258151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.258155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.258160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.258176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.258187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.258216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:17.266323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:17.267601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:17.267616Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:17.267619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:17.267624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:17.267630Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1169 [2:7559342144942184327:9037]
2025-10-09T21:48:17.267645Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:17.267665Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:17.267668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:17.267671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:17.267674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:17.267691Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:17.267702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:17.267709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:17.267772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.267779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.267781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.267784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:17.267854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:17.267862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.267873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.267888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.267909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:17.267935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342145304006537:10153], now have 1 active actors on pipe
2025-10-09T21:48:17.267944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342145304006329:2432], now have 1 active actors on pipe
2025-10-09T21:48:17.267951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342144942184322:9037]
2025-10-09T21:48:17.267954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342144942184322:9037]
2025-10-09T21:48:17.267956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342144942184322:9037]
2025-10-09T21:48:17.267975Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:17.267994Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:17.267996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:17.267997Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:17.267984Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1169
2025-10-09T21:48:17.267995Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1169
2025-10-09T21:48:17.267999Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:17.268011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:17.268014Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:17.268047Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342144942184322:9037]
2025-10-09T21:48:17.268050Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:17.268057Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:17.268068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.268073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.268133Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1bb36c5e-4fce5f4a-f9664226-91e46acc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:17.268146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:17.268155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:17.268157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:17.268172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:17.268182Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:17.268206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:17.268371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.268386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.268422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:17.268549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1170 partition: 0 MaxSeqNo: 25 sessionId: src_id|1bb36c5e-4fce5f4a-f9664226-91e46acc_0
2025-10-09T21:48:17.268906Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|17f221bf-20040c7b-2fd8eb45-ca7df668_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1bb36c5e-4fce5f4a-f9664226-91e46acc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:17.271410Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1bb36c5e-4fce5f4a-f9664226-91e46acc_0] Write session: try to update token
2025-10-09T21:48:17.271425Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1bb36c5e-4fce5f4a-f9664226-91e46acc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:17.285753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1170 sessionId: src_id|1bb36c5e-4fce5f4a-f9664226-91e46acc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:17.287340Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:17.287355Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:17.287357Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:17.287358Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:17.287366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342144942184322:9037]
2025-10-09T21:48:17.287370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342144942184322:9037]
2025-10-09T21:48:17.287373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342144942184322:9037]
2025-10-09T21:48:17.287407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:17.287412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.287412Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:17.287415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:17.287418Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:17.287422Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:17.287553Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:17.288136Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:17.335291Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342145304006583:9750]
2025-10-09T21:48:17.335551Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 1169, pipe: [1:7559342145304006583:9750]
2025-10-09T21:48:17.446437Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:17.446960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.446979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.447026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342145304006583:9750], now have 1 active actors on pipe
2025-10-09T21:48:17.447222Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:17.447276Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342145304006583:9750]
2025-10-09T21:48:17.447293Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:17.447297Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:17.447309Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:17.447367Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1170 reason: error at write: Disconnected sessionId: src_id|1bb36c5e-4fce5f4a-f9664226-91e46acc_0
2025-10-09T21:48:17.447552Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1170 sessionId: src_id|1bb36c5e-4fce5f4a-f9664226-91e46acc_0 is DEAD
2025-10-09T21:48:17.447705Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bb36c5e-4fce5f4a-f9664226-91e46acc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:17.447716Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bb36c5e-4fce5f4a-f9664226-91e46acc_0] Write session will restart in 0.009527s
2025-10-09T21:48:17.447743Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bb36c5e-4fce5f4a-f9664226-91e46acc_0] Write session: Do CDS request
2025-10-09T21:48:17.447750Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bb36c5e-4fce5f4a-f9664226-91e46acc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:17.458523Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1bb36c5e-4fce5f4a-f9664226-91e46acc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:17.458565Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:17.458577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1171
2025-10-09T21:48:17.458733Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1171 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:17.458774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1171 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:17.458784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1171 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:17.458787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:17.459178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1171 sessionId:  describe result for acl check
2025-10-09T21:48:17.459211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:17.459215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:17.459218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:17.459222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342145304006686:10157] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:17.459702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342145304006686:10157] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:17.462198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342145304006686:10157] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:17.464457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:17.466738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:17.466908Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:17.466918Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:17.466931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:17.466948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342145304006686:10157] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:17.466960Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342145304006686:10157] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:17.466964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342145304006686:10157] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:17.466971Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1171 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:17.466986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:17.467026Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342144942184572:9039]
2025-10-09T21:48:17.467568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:17.467846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:17.467886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:17.468186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:17.468244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.468252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.468253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.468254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.468255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.468257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.468258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.468259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.468260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.468264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.468275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.468279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.468283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.468286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.468291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.468294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.468298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.468302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.468308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.468315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.468321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.468327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.468332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.468337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.468341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.468346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.468350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.468355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.468361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.468365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.468370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.468375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.468379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.468384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.468389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.468393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.468421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:17.475819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:17.476857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:17.476871Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:17.476874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:17.476878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:17.476883Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1170 [2:7559342144942184572:9039]
2025-10-09T21:48:17.476898Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:17.476915Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:17.476918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:17.476921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:17.476923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:17.476939Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:17.476944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:17.476951Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:17.477001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.477009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.477011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.477014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:17.477077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:17.477083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.477096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.477112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.477135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:17.477159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342145304006828:10157], now have 1 active actors on pipe
2025-10-09T21:48:17.477167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342145304006586:2432], now have 1 active actors on pipe
2025-10-09T21:48:17.477174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342144942184567:9039]
2025-10-09T21:48:17.477177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342144942184567:9039]
2025-10-09T21:48:17.477180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342144942184567:9039]
2025-10-09T21:48:17.477192Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:17.477206Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:17.477212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:17.477215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:17.477217Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:17.477226Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:17.477202Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1170
2025-10-09T21:48:17.477230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:17.477226Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1170
2025-10-09T21:48:17.477250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342144942184567:9039]
2025-10-09T21:48:17.477253Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:17.477255Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:17.477269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.477274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.477322Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bc2c1c93-1bb631ce-d001b9f1-bc95d6f2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:17.477338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:17.477342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:17.477344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:17.477353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:17.477364Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:17.477388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:17.477563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.477576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.477598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:17.477715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1171 partition: 0 MaxSeqNo: 25 sessionId: src_id|bc2c1c93-1bb631ce-d001b9f1-bc95d6f2_0
2025-10-09T21:48:17.478075Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bb36c5e-4fce5f4a-f9664226-91e46acc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bc2c1c93-1bb631ce-d001b9f1-bc95d6f2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:17.480553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2c1c93-1bb631ce-d001b9f1-bc95d6f2_0] Write session: try to update token
2025-10-09T21:48:17.480568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2c1c93-1bb631ce-d001b9f1-bc95d6f2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:17.493693Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1171 sessionId: src_id|bc2c1c93-1bb631ce-d001b9f1-bc95d6f2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:17.496143Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:17.496169Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:17.496183Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:17.496186Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:17.496188Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:17.496197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342144942184567:9039]
2025-10-09T21:48:17.496201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342144942184567:9039]
2025-10-09T21:48:17.496207Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342144942184567:9039]
2025-10-09T21:48:17.496249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:17.496251Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:17.496256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.496258Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:17.496259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:17.496262Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:17.496405Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:17.547258Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342145304006840:9750]
2025-10-09T21:48:17.547495Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 1170, pipe: [1:7559342145304006840:9750]
2025-10-09T21:48:17.651796Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:17.652433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.652451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.652496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342145304006840:9750], now have 1 active actors on pipe
2025-10-09T21:48:17.652710Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:17.652742Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:17.652753Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:17.652772Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342145304006840:9750]
2025-10-09T21:48:17.652794Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:17.652807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1171 reason: error at write: Disconnected sessionId: src_id|bc2c1c93-1bb631ce-d001b9f1-bc95d6f2_0
2025-10-09T21:48:17.652996Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1171 sessionId: src_id|bc2c1c93-1bb631ce-d001b9f1-bc95d6f2_0 is DEAD
2025-10-09T21:48:17.653139Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2c1c93-1bb631ce-d001b9f1-bc95d6f2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:17.653151Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2c1c93-1bb631ce-d001b9f1-bc95d6f2_0] Write session will restart in 0.005330s
2025-10-09T21:48:17.653177Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2c1c93-1bb631ce-d001b9f1-bc95d6f2_0] Write session: Do CDS request
2025-10-09T21:48:17.653191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2c1c93-1bb631ce-d001b9f1-bc95d6f2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:17.659471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2c1c93-1bb631ce-d001b9f1-bc95d6f2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:17.659531Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:17.659553Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1172
2025-10-09T21:48:17.659678Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1172 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:17.659709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1172 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:17.659719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1172 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:17.659723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:17.660116Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1172 sessionId:  describe result for acl check
2025-10-09T21:48:17.660143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:17.660144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:17.660145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:17.660149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342145304006858:10161] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:17.660673Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342145304006858:10161] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:17.663186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342145304006858:10161] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:17.666442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342145304006858:10161] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:17.666455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342145304006858:10161] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:17.666459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342145304006858:10161] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:17.666466Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1172 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:17.670384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:17.672577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:17.672744Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:17.672753Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:17.672765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:17.672825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:17.672863Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342144942184823:9043]
2025-10-09T21:48:17.673420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:17.673820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:17.673876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:17.674222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:17.674274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.674280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.674281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.674283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.674284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.674285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.674286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.674287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.674288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.674294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.674305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.674309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.674313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.674317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.674321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.674324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.674328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.674332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.674340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.674347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.674359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.674364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.674370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.674375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.674379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.674385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.674393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.674397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.674402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.674407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.674417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.674422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.674426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.674432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.674436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.674441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.674473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:17.682175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:17.683386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:17.683400Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:17.683403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:17.683407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:17.683413Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1171 [2:7559342144942184823:9043]
2025-10-09T21:48:17.683427Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:17.683445Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:17.683453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:17.683456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:17.683458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:17.683474Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:17.683485Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:17.683492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:17.683547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.683554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.683556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.683558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:17.683623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:17.683631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.683643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.683658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.683680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:17.683706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342145304007010:10161], now have 1 active actors on pipe
2025-10-09T21:48:17.683715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342145304006843:2432], now have 1 active actors on pipe
2025-10-09T21:48:17.683723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342144942184818:9043]
2025-10-09T21:48:17.683726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342144942184818:9043]
2025-10-09T21:48:17.683728Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342144942184818:9043]
2025-10-09T21:48:17.683741Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:17.683757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:17.683763Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:17.683766Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:17.683769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:17.683778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:17.683781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:17.683759Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1171
2025-10-09T21:48:17.683761Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1171
2025-10-09T21:48:17.683824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.683834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.683852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342144942184818:9043]
2025-10-09T21:48:17.683864Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:17.683876Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:17.683897Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8bc99218-e0171d5b-1e9bc6bf-600c6842_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:17.683910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:17.683915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:17.683919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:17.683930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:17.683944Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:17.683973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:17.684127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.684139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.684161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:17.684398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1172 partition: 0 MaxSeqNo: 25 sessionId: src_id|8bc99218-e0171d5b-1e9bc6bf-600c6842_0
2025-10-09T21:48:17.684744Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2c1c93-1bb631ce-d001b9f1-bc95d6f2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8bc99218-e0171d5b-1e9bc6bf-600c6842_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:17.687337Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8bc99218-e0171d5b-1e9bc6bf-600c6842_0] Write session: try to update token
2025-10-09T21:48:17.687358Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8bc99218-e0171d5b-1e9bc6bf-600c6842_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:17.700513Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:17.700531Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:17.700534Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:17.700535Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:17.700546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342144942184818:9043]
2025-10-09T21:48:17.700550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342144942184818:9043]
2025-10-09T21:48:17.700554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342144942184818:9043]
2025-10-09T21:48:17.700599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:17.700607Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:17.700611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.700613Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:17.700614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:17.700617Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:17.700630Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1172 sessionId: src_id|8bc99218-e0171d5b-1e9bc6bf-600c6842_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:17.700763Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:17.703136Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:17.753083Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342145304007099:9750]
2025-10-09T21:48:17.753357Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 1171, pipe: [1:7559342145304007099:9750]
2025-10-09T21:48:17.862244Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:17.862531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.862543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.862584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342145304007099:9750], now have 1 active actors on pipe
2025-10-09T21:48:17.862812Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:17.862836Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:17.862864Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342145304007099:9750]
2025-10-09T21:48:17.862890Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:17.862968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1172 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8bc99218-e0171d5b-1e9bc6bf-600c6842_0
2025-10-09T21:48:17.863149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1172 sessionId: src_id|8bc99218-e0171d5b-1e9bc6bf-600c6842_0 is DEAD
2025-10-09T21:48:17.863337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8bc99218-e0171d5b-1e9bc6bf-600c6842_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:17.863349Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8bc99218-e0171d5b-1e9bc6bf-600c6842_0] Write session will restart in 0.006778s
2025-10-09T21:48:17.863375Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8bc99218-e0171d5b-1e9bc6bf-600c6842_0] Write session: Do CDS request
2025-10-09T21:48:17.863382Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8bc99218-e0171d5b-1e9bc6bf-600c6842_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:17.871570Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8bc99218-e0171d5b-1e9bc6bf-600c6842_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:17.871660Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:17.871680Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1173
2025-10-09T21:48:17.871846Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1173 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:17.871896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1173 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:17.871906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1173 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:17.871910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:17.872351Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1173 sessionId:  describe result for acl check
2025-10-09T21:48:17.872390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:17.872396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:17.872398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:17.872404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342145304007127:10166] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:17.872964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342145304007127:10166] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:17.876126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342145304007127:10166] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:17.879458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342145304007127:10166] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:17.879473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342145304007127:10166] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:17.879477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342145304007127:10166] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:17.879485Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1173 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:17.882284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:17.884912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:17.885103Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:17.885118Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:17.885131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:17.885188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:17.885230Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342144942185070:9045]
2025-10-09T21:48:17.885808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:17.886183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:17.886237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:17.886591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:17.886658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.886668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.886670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.886671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.886672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.886673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.886674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.886676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.886677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.886682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.886694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.886699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.886703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.886707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.886711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.886715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.886719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.886723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.886731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.886739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:17.886745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.886756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:17.886763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.886775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:17.886779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.886784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:17.886789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.886795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:17.886800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.886806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:17.886811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.886817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.886821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.886827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.886832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.886837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.886867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:17.895021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:17.896059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:17.896077Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:17.896080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:17.896085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:17.896092Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1172 [2:7559342144942185070:9045]
2025-10-09T21:48:17.896106Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:17.896125Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:17.896133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:17.896136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:17.896137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:17.896151Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:17.896163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:17.896170Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:17.896230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.896237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.896239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.896242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:17.896399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:17.896408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.896422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:17.896433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:17.896452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:17.896478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342145304007301:10166], now have 1 active actors on pipe
2025-10-09T21:48:17.896485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342145304007103:2432], now have 1 active actors on pipe
2025-10-09T21:48:17.896493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342144942185065:9045]
2025-10-09T21:48:17.896495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342144942185065:9045]
2025-10-09T21:48:17.896500Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1172
2025-10-09T21:48:17.896505Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1172
2025-10-09T21:48:17.896498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342144942185065:9045]
2025-10-09T21:48:17.896515Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:17.896529Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:17.896538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:17.896542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:17.896546Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:17.896551Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:17.896554Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:17.896590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.896601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.896618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342144942185065:9045]
2025-10-09T21:48:17.896627Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:17.896640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:17.896663Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1ba78987-ee9bf7f7-e34d7c3e-bee64e8d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:17.896676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:17.896680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:17.896686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:17.896699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:17.896715Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:17.896743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:17.896937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.896952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.896976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:17.897088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1173 partition: 0 MaxSeqNo: 25 sessionId: src_id|1ba78987-ee9bf7f7-e34d7c3e-bee64e8d_0
2025-10-09T21:48:17.897449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8bc99218-e0171d5b-1e9bc6bf-600c6842_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1ba78987-ee9bf7f7-e34d7c3e-bee64e8d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:17.900013Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ba78987-ee9bf7f7-e34d7c3e-bee64e8d_0] Write session: try to update token
2025-10-09T21:48:17.900033Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ba78987-ee9bf7f7-e34d7c3e-bee64e8d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:17.913483Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1173 sessionId: src_id|1ba78987-ee9bf7f7-e34d7c3e-bee64e8d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:17.916009Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:17.916741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:17.916766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:17.916792Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:17.916795Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:17.916797Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:17.916799Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:17.916808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342144942185065:9045]
2025-10-09T21:48:17.916813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342144942185065:9045]
2025-10-09T21:48:17.916817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342144942185065:9045]
2025-10-09T21:48:17.916865Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:17.916878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:17.916885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:17.916888Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:17.916889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:17.916892Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:17.917023Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:17.962994Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342145304007360:9750]
2025-10-09T21:48:17.963262Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 1172, pipe: [1:7559342145304007360:9750]
2025-10-09T21:48:18.081788Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:18.082082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:18.082152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342145304007360:9750], now have 1 active actors on pipe
2025-10-09T21:48:18.082234Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:18.082346Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:18.082355Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:18.082442Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1173 reason: error at write: Disconnected sessionId: src_id|1ba78987-ee9bf7f7-e34d7c3e-bee64e8d_0
2025-10-09T21:48:18.082589Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1173 sessionId: src_id|1ba78987-ee9bf7f7-e34d7c3e-bee64e8d_0 is DEAD
2025-10-09T21:48:18.082759Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ba78987-ee9bf7f7-e34d7c3e-bee64e8d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:18.082769Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ba78987-ee9bf7f7-e34d7c3e-bee64e8d_0] Write session will restart in 0.005014s
2025-10-09T21:48:18.082793Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ba78987-ee9bf7f7-e34d7c3e-bee64e8d_0] Write session: Do CDS request
2025-10-09T21:48:18.082799Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ba78987-ee9bf7f7-e34d7c3e-bee64e8d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:18.082757Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:18.082807Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342145304007360:9750]
2025-10-09T21:48:18.088492Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ba78987-ee9bf7f7-e34d7c3e-bee64e8d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:18.088609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:18.088623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1174
2025-10-09T21:48:18.088755Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1174 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:18.088791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1174 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:18.088803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1174 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:18.088806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:18.089252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1174 sessionId:  describe result for acl check
2025-10-09T21:48:18.089290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:18.089297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:18.089299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:18.089304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342149598974687:10176] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:18.089890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342149598974687:10176] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:18.091966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342149598974687:10176] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:18.095787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342149598974687:10176] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:18.095804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342149598974687:10176] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:18.095808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342149598974687:10176] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:18.095817Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1174 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:18.102765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:18.105442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:18.105806Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:18.105817Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:18.105833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:18.105907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:18.105954Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342149237152644:9057]
2025-10-09T21:48:18.106562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:18.106950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:18.107008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:18.107353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:18.107418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.107424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.107425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.107426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.107428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.107429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.107430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.107431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.107432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.107437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.107446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.107450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.107453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.107457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.107461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.107465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.107468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.107471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.107480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.107488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.107494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.107500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.107504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.107509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.107514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.107519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.107524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.107529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.107533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.107538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.107544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.107549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.107554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.107558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.107566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.107571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.107599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:18.115390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:18.116721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:18.116737Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:18.116740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:18.116744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:18.116751Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1173 [2:7559342149237152644:9057]
2025-10-09T21:48:18.116766Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:18.116787Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:18.116790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:18.116794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:18.116796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:18.116816Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:18.116829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:18.116838Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:18.116899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.116908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.116910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.116913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:18.116986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:18.116994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.117007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.117023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.117044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:18.117072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342149598974798:10176], now have 1 active actors on pipe
2025-10-09T21:48:18.117081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342149598974672:2432], now have 1 active actors on pipe
2025-10-09T21:48:18.117088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342149237152639:9057]
2025-10-09T21:48:18.117091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342149237152639:9057]
2025-10-09T21:48:18.117093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342149237152639:9057]
2025-10-09T21:48:18.117106Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:18.117124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:18.117124Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:18.117127Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:18.117131Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:18.117144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:18.117149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:18.117110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1173
2025-10-09T21:48:18.117197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:18.117206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:18.117219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342149237152639:9057]
2025-10-09T21:48:18.117231Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:18.117255Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:18.117263Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|aa618e00-b4525c43-28a20fd1-da875535_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:18.117271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:18.117276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:18.117284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:18.117296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:18.117307Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:18.117335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:18.117492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:18.117502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:18.117494Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1173
2025-10-09T21:48:18.117524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:18.117635Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1174 partition: 0 MaxSeqNo: 25 sessionId: src_id|aa618e00-b4525c43-28a20fd1-da875535_0
2025-10-09T21:48:18.118096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ba78987-ee9bf7f7-e34d7c3e-bee64e8d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|aa618e00-b4525c43-28a20fd1-da875535_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:18.120565Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aa618e00-b4525c43-28a20fd1-da875535_0] Write session: try to update token
2025-10-09T21:48:18.120585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aa618e00-b4525c43-28a20fd1-da875535_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:18.133344Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:18.133361Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:18.133363Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:18.133369Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:18.133380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342149237152639:9057]
2025-10-09T21:48:18.133384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342149237152639:9057]
2025-10-09T21:48:18.133387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342149237152639:9057]
2025-10-09T21:48:18.133435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:18.133439Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:18.133446Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:18.133447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.133449Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:18.133449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:18.133588Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:18.134105Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1174 sessionId: src_id|aa618e00-b4525c43-28a20fd1-da875535_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:18.136619Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:18.183174Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342149598974941:9750]
2025-10-09T21:48:18.183561Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 1173, pipe: [1:7559342149598974941:9750]
2025-10-09T21:48:18.281323Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:18.281686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:18.281702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:18.281755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342149598974941:9750], now have 1 active actors on pipe
2025-10-09T21:48:18.282048Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:18.282050Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:18.282064Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:18.282095Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342149598974941:9750]
2025-10-09T21:48:18.282128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1174 reason: error at write: Disconnected sessionId: src_id|aa618e00-b4525c43-28a20fd1-da875535_0
2025-10-09T21:48:18.282303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1174 sessionId: src_id|aa618e00-b4525c43-28a20fd1-da875535_0 is DEAD
2025-10-09T21:48:18.282478Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa618e00-b4525c43-28a20fd1-da875535_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:18.282496Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa618e00-b4525c43-28a20fd1-da875535_0] Write session will restart in 0.007758s
2025-10-09T21:48:18.282521Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa618e00-b4525c43-28a20fd1-da875535_0] Write session: Do CDS request
2025-10-09T21:48:18.282530Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa618e00-b4525c43-28a20fd1-da875535_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:18.281962Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:18.291544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aa618e00-b4525c43-28a20fd1-da875535_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:18.291672Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:18.291689Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1175
2025-10-09T21:48:18.291835Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1175 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:18.291877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1175 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:18.291884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1175 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:18.291887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:18.292364Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1175 sessionId:  describe result for acl check
2025-10-09T21:48:18.292399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:18.292400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:18.292402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:18.292407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342149598974989:10182] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:18.292941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342149598974989:10182] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:18.296086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342149598974989:10182] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:18.299255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342149598974989:10182] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:18.299267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342149598974989:10182] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:18.299271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342149598974989:10182] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:18.299278Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1175 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:18.300770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:18.303231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:18.303560Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:18.303579Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:18.303592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:18.303674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:18.303712Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342149237152901:9061]
2025-10-09T21:48:18.304314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:18.304694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:18.304755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:18.305043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:18.305085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.305090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.305092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.305093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.305094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.305095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.305096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.305097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.305098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.305103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.305113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.305133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.305137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.305140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.305143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.305147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.305150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.305154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.305161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.305169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.305180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.305185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.305194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.305199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.305204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.305209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.305214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.305218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.305223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.305228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.305234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.305243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.305247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.305252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.305258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.305262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.305288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:18.311564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:18.312625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:18.312637Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:18.312640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:18.312644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:18.312651Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1174 [2:7559342149237152901:9061]
2025-10-09T21:48:18.312664Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:18.312683Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:18.312691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:18.312694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:18.312696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:18.312713Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:18.312723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:18.312730Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:18.312790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.312796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.312798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.312801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:18.312866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:18.312876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.312888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.312900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.312916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:18.312939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342149598974949:2432], now have 1 active actors on pipe
2025-10-09T21:48:18.312946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342149598975192:10182], now have 1 active actors on pipe
2025-10-09T21:48:18.312954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342149237152896:9061]
2025-10-09T21:48:18.312958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342149237152896:9061]
2025-10-09T21:48:18.312961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342149237152896:9061]
2025-10-09T21:48:18.312973Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:18.312984Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:18.312987Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:18.312990Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:18.312991Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:18.313005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:18.313013Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:18.313006Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1174
2025-10-09T21:48:18.313015Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1174
2025-10-09T21:48:18.313044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342149237152896:9061]
2025-10-09T21:48:18.313061Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:18.313081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:18.313080Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:18.313092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:18.313348Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|da2c2934-99ceca7d-906e93da-25d63fa3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:18.313364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:18.313368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:18.313370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:18.313381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:18.313391Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:18.313414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:18.313578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:18.313591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:18.313613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:18.313767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1175 partition: 0 MaxSeqNo: 25 sessionId: src_id|da2c2934-99ceca7d-906e93da-25d63fa3_0
2025-10-09T21:48:18.314152Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa618e00-b4525c43-28a20fd1-da875535_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|da2c2934-99ceca7d-906e93da-25d63fa3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:18.316513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da2c2934-99ceca7d-906e93da-25d63fa3_0] Write session: try to update token
2025-10-09T21:48:18.316528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da2c2934-99ceca7d-906e93da-25d63fa3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:18.328017Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:18.328034Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:18.328037Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:18.328039Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:18.328049Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342149237152896:9061]
2025-10-09T21:48:18.328053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342149237152896:9061]
2025-10-09T21:48:18.328057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342149237152896:9061]
2025-10-09T21:48:18.328103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:18.328110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.328111Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:18.328113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:18.328118Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:18.328122Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:18.328254Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:18.329626Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1175 sessionId: src_id|da2c2934-99ceca7d-906e93da-25d63fa3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:18.332011Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:18.382351Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342149598975207:9750]
2025-10-09T21:48:18.382675Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 1174, pipe: [1:7559342149598975207:9750]
2025-10-09T21:48:18.483875Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:18.484326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:18.484342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:18.484393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342149598975207:9750], now have 1 active actors on pipe
2025-10-09T21:48:18.484668Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:18.484679Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:18.484697Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:18.485132Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da2c2934-99ceca7d-906e93da-25d63fa3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:18.485143Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da2c2934-99ceca7d-906e93da-25d63fa3_0] Write session will restart in 0.007385s
2025-10-09T21:48:18.485167Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da2c2934-99ceca7d-906e93da-25d63fa3_0] Write session: Do CDS request
2025-10-09T21:48:18.485183Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da2c2934-99ceca7d-906e93da-25d63fa3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:18.484760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1175 reason: error at write: Disconnected sessionId: src_id|da2c2934-99ceca7d-906e93da-25d63fa3_0
2025-10-09T21:48:18.484833Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342149598975207:9750]
2025-10-09T21:48:18.484862Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:18.484983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1175 sessionId: src_id|da2c2934-99ceca7d-906e93da-25d63fa3_0 is DEAD
2025-10-09T21:48:18.493558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da2c2934-99ceca7d-906e93da-25d63fa3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:18.493645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:18.493661Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1176
2025-10-09T21:48:18.493818Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1176 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:18.493861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1176 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:18.493869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1176 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:18.493873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:18.494310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1176 sessionId:  describe result for acl check
2025-10-09T21:48:18.494348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:18.494352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:18.494353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:18.494359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342149598975240:10186] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:18.494908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342149598975240:10186] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:18.497329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342149598975240:10186] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:18.500392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342149598975240:10186] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:18.500407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342149598975240:10186] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:18.500412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342149598975240:10186] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:18.500419Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1176 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:18.503529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:18.506047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:18.506236Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:18.506246Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:18.506258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:18.506313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:18.506361Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342149237153152:9063]
2025-10-09T21:48:18.506895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:18.507175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:18.507222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:18.507568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:18.507634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.507641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.507642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.507644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.507645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.507646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.507647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.507648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.507649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.507655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.507666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.507670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.507674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.507678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.507683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.507687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.507690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.507694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.507702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.507713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.507721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.507726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.507731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.507736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.507740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.507747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.507751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.507760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.507764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.507772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.507781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.507786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.507792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.507798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.507807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.507817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.507866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:18.514600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:18.515902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:18.515915Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:18.515917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:18.515921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:18.515927Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1175 [2:7559342149237153152:9063]
2025-10-09T21:48:18.515943Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:18.515964Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:18.515970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:18.515973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:18.515975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:18.515992Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:18.516003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:18.516010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:18.516063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.516070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.516072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.516075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:18.516140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:18.516147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.516158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.516173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.516194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:18.516219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342149598975405:10186], now have 1 active actors on pipe
2025-10-09T21:48:18.516223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342149598975210:2432], now have 1 active actors on pipe
2025-10-09T21:48:18.516232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342149237153147:9063]
2025-10-09T21:48:18.516234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342149237153147:9063]
2025-10-09T21:48:18.516236Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342149237153147:9063]
2025-10-09T21:48:18.516246Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:18.516264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:18.516265Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:18.516268Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:18.516270Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:18.516276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:18.516279Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:18.516304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342149237153147:9063]
2025-10-09T21:48:18.516307Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:18.516309Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:18.516274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1175
2025-10-09T21:48:18.516343Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1175
2025-10-09T21:48:18.516374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:18.516385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:18.516413Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e2ddac79-7b8138c6-6adbd2b-c284d70a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:18.516425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:18.516431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:18.516433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:18.516446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:18.516460Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:18.516476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:18.516652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:18.516660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:18.516677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:18.516806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1176 partition: 0 MaxSeqNo: 25 sessionId: src_id|e2ddac79-7b8138c6-6adbd2b-c284d70a_0
2025-10-09T21:48:18.517165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da2c2934-99ceca7d-906e93da-25d63fa3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e2ddac79-7b8138c6-6adbd2b-c284d70a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:18.519635Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2ddac79-7b8138c6-6adbd2b-c284d70a_0] Write session: try to update token
2025-10-09T21:48:18.519657Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2ddac79-7b8138c6-6adbd2b-c284d70a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:18.532870Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1176 sessionId: src_id|e2ddac79-7b8138c6-6adbd2b-c284d70a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:18.534122Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:18.534136Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:18.534146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342149237153147:9063]
2025-10-09T21:48:18.534197Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:18.534204Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:18.534365Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:18.534513Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:18.534658Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:18.534800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:18.534997Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:18.535039Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:18.535292Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:18.535467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:18.535484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:18.535678Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:18.535516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:18.541437Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:18.541449Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:18.541452Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:18.541454Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:18.541463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342149237153147:9063]
2025-10-09T21:48:18.541467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342149237153147:9063]
2025-10-09T21:48:18.541471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342149237153147:9063]
2025-10-09T21:48:18.541512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:18.541520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.541527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:18.541527Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:18.541535Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:18.541540Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:18.541648Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:18.585100Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342149598975470:9750]
2025-10-09T21:48:18.585370Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 1175, pipe: [1:7559342149598975470:9750]
2025-10-09T21:48:18.698748Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:18.705674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:18.705700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:18.705854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.705908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.705966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.706046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.706096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.706105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.706168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.706220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.706279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.706354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.706396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.706402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.706455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.706515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.706576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.706635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.706688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.706696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.706767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.707037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.707094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.707148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.707187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.707193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.707265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.707333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.707396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.707463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.707511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.707519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.707589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.707652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.707712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.707775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.707822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.707829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.707889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.707944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.708004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.708068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.708112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.708121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.708182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.708240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.708298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.708358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.708405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.708413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.708482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.708538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.708597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.708655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.708722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.708730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.708793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.708852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.708913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.708973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.709032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.709039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.709103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342149598975470:9750], now have 1 active actors on pipe
2025-10-09T21:48:18.709344Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:18.709345Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:18.709350Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:18.709386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1176 reason: error at write: Disconnected sessionId: src_id|e2ddac79-7b8138c6-6adbd2b-c284d70a_0
2025-10-09T21:48:18.709403Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342149598975470:9750]
2025-10-09T21:48:18.709569Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1176 sessionId: src_id|e2ddac79-7b8138c6-6adbd2b-c284d70a_0 is DEAD
2025-10-09T21:48:18.709753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2ddac79-7b8138c6-6adbd2b-c284d70a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:18.709762Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2ddac79-7b8138c6-6adbd2b-c284d70a_0] Write session will restart in 0.008993s
2025-10-09T21:48:18.709798Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2ddac79-7b8138c6-6adbd2b-c284d70a_0] Write session: Do CDS request
2025-10-09T21:48:18.709805Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2ddac79-7b8138c6-6adbd2b-c284d70a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:18.709849Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:18.719606Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2ddac79-7b8138c6-6adbd2b-c284d70a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:18.719715Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:18.719733Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1177
2025-10-09T21:48:18.719863Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1177 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:18.719906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1177 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:18.719916Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1177 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:18.719919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:18.720398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1177 sessionId:  describe result for acl check
2025-10-09T21:48:18.720435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:18.720445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:18.720449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:18.720455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342149598975512:10190] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:18.721008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342149598975512:10190] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:18.724307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342149598975512:10190] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:18.727554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342149598975512:10190] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:18.727567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342149598975512:10190] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:18.727571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342149598975512:10190] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:18.727581Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1177 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:18.728359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:18.731053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:18.731357Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:18.731369Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:18.731383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:18.731462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:18.731506Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342149237153407:9067]
2025-10-09T21:48:18.732114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:18.732447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:18.732492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:18.732831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:18.732892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.732899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.732901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.732902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.732903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.732904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.732906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.732907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.732908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.732914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.732926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.732930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.732934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.732938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.732942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.732946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.732950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.732954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.732961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.732970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.732976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.732983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.732987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.732992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.732998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.733003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.733009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.733014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.733018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.733024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.733028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.733034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.733039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.733044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.733049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.733054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.733084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:18.740014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:18.741373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:18.741390Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:18.741392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:18.741396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:18.741405Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1176 [2:7559342149237153407:9067]
2025-10-09T21:48:18.741419Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:18.741439Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:18.741448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:18.741451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:18.741454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:18.741472Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:18.741478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:18.741484Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:18.741532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.741537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.741538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.741540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:18.741601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:18.741608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.741620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.741636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.741654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:18.741680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342149598975719:10190], now have 1 active actors on pipe
2025-10-09T21:48:18.741688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342149598975472:2432], now have 1 active actors on pipe
2025-10-09T21:48:18.741695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342149237153402:9067]
2025-10-09T21:48:18.741703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342149237153402:9067]
2025-10-09T21:48:18.741705Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342149237153402:9067]
2025-10-09T21:48:18.741714Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:18.741727Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:18.741729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:18.741742Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:18.741745Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:18.741754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:18.741757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:18.741754Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1176
2025-10-09T21:48:18.741789Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342149237153402:9067]
2025-10-09T21:48:18.741807Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:18.741836Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:18.741860Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1176
2025-10-09T21:48:18.741893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:18.741903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:18.741932Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|65286328-309e66f6-a65ed64d-8b248ffe_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:18.741946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:18.741950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:18.741953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:18.741965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:18.741980Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:18.742001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:18.742180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:18.742193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:18.742217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:18.742314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1177 partition: 0 MaxSeqNo: 25 sessionId: src_id|65286328-309e66f6-a65ed64d-8b248ffe_0
2025-10-09T21:48:18.742683Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2ddac79-7b8138c6-6adbd2b-c284d70a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|65286328-309e66f6-a65ed64d-8b248ffe_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:18.745166Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|65286328-309e66f6-a65ed64d-8b248ffe_0] Write session: try to update token
2025-10-09T21:48:18.745183Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|65286328-309e66f6-a65ed64d-8b248ffe_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:18.754095Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:18.754111Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:18.754120Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342149237153402:9067]
2025-10-09T21:48:18.754165Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:18.754167Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:18.754324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:18.754454Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:18.754585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:18.754717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:18.755020Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:18.755070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:18.758233Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1177 sessionId: src_id|65286328-309e66f6-a65ed64d-8b248ffe_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:18.760445Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:18.760654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:18.760672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:18.760710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:18.760802Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:18.767537Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:18.767559Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:18.767561Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:18.767563Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:18.767574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342149237153402:9067]
2025-10-09T21:48:18.767579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342149237153402:9067]
2025-10-09T21:48:18.767582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342149237153402:9067]
2025-10-09T21:48:18.767629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:18.767640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.767643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:18.767655Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:18.767671Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:18.767675Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:18.767786Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:18.809351Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342149598975733:9750]
2025-10-09T21:48:18.809663Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 1176, pipe: [1:7559342149598975733:9750]
2025-10-09T21:48:18.924760Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:18.931906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:18.931931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:18.932095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.932155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.932221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.932279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.932320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.932331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.932397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.932458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.932515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.932570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.932615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.932618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.932674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.932733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.932793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.932851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.932890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.932896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.932961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.933021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.933071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.933131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.933176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.933184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.933248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.933304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.933362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.933417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.933464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.933475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.933534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.933596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.933655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.933711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.933763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.933770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.933823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.933875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.933929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.933994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.934059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.934067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.934137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.934194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.934257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.934315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.934353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.934358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.934412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.934465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.934522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.934578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.934616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.934621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.934675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:18.934726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:18.934786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:18.934841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:18.934879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:18.934884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:18.934926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342149598975733:9750], now have 1 active actors on pipe
2025-10-09T21:48:18.935217Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:18.935229Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:18.935229Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:18.935261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1177 reason: error at write: Disconnected sessionId: src_id|65286328-309e66f6-a65ed64d-8b248ffe_0
2025-10-09T21:48:18.935296Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342149598975733:9750]
2025-10-09T21:48:18.935451Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1177 sessionId: src_id|65286328-309e66f6-a65ed64d-8b248ffe_0 is DEAD
2025-10-09T21:48:18.935588Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|65286328-309e66f6-a65ed64d-8b248ffe_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:18.935598Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|65286328-309e66f6-a65ed64d-8b248ffe_0] Write session will restart in 0.009169s
2025-10-09T21:48:18.935622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|65286328-309e66f6-a65ed64d-8b248ffe_0] Write session: Do CDS request
2025-10-09T21:48:18.935629Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|65286328-309e66f6-a65ed64d-8b248ffe_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:18.935841Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:18.945545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|65286328-309e66f6-a65ed64d-8b248ffe_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:18.945614Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:18.945629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1178
2025-10-09T21:48:18.945786Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1178 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:18.945820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1178 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:18.945827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1178 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:18.945831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:18.946271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1178 sessionId:  describe result for acl check
2025-10-09T21:48:18.946303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:18.946309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:18.946312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:18.946318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342149598975786:10195] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:18.946853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342149598975786:10195] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:18.949969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342149598975786:10195] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:18.953606Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342149598975786:10195] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:18.953621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342149598975786:10195] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:18.953625Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342149598975786:10195] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:18.953633Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1178 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:18.955651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:18.958162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:18.958376Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:18.958388Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:18.958403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:18.958469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:18.958514Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342149237153659:9069]
2025-10-09T21:48:18.959108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:18.959537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:18.959585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:18.959909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:18.959960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.959968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.959970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.959971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.959972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.959973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.959974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.959976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.959977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.959982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.959993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.959996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.960000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.960003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.960007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.960010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.960014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.960017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.960024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.960032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:18.960038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.960043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:18.960047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.960052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:18.960056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.960062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:18.960066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.960071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:18.960076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.960080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:18.960085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.960091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.960095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.960100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.960104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.960109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.960140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:18.966685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:18.967695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:18.967710Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:18.967712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:18.967717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:18.967723Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1177 [2:7559342149237153659:9069]
2025-10-09T21:48:18.967736Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:18.967754Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:18.967761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:18.967764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:18.967767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:18.967783Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:18.967794Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:18.967804Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:18.967861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.967872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.967874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.967876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:18.967943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:18.967949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.967961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:18.967975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:18.967997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:18.968021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342149598975738:2432], now have 1 active actors on pipe
2025-10-09T21:48:18.968029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342149598975985:10195], now have 1 active actors on pipe
2025-10-09T21:48:18.968037Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342149237153654:9069]
2025-10-09T21:48:18.968041Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342149237153654:9069]
2025-10-09T21:48:18.968043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342149237153654:9069]
2025-10-09T21:48:18.968055Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:18.968069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:18.968072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:18.968073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:18.968075Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:18.968085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:18.968088Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:18.968080Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1177
2025-10-09T21:48:18.968112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342149237153654:9069]
2025-10-09T21:48:18.968117Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:18.968120Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:18.968089Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1177
2025-10-09T21:48:18.968144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:18.968153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:18.968187Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|62c8f2c2-523b8bd8-4b900b35-984204e7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:18.968199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:18.968203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:18.968206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:18.968217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:18.968231Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:18.968249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:18.968439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:18.968451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:18.968474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:18.968607Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1178 partition: 0 MaxSeqNo: 25 sessionId: src_id|62c8f2c2-523b8bd8-4b900b35-984204e7_0
2025-10-09T21:48:18.968973Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|65286328-309e66f6-a65ed64d-8b248ffe_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|62c8f2c2-523b8bd8-4b900b35-984204e7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:18.971378Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62c8f2c2-523b8bd8-4b900b35-984204e7_0] Write session: try to update token
2025-10-09T21:48:18.971396Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62c8f2c2-523b8bd8-4b900b35-984204e7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:18.984035Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:18.984052Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:18.984055Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:18.984056Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:18.984067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342149237153654:9069]
2025-10-09T21:48:18.984072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342149237153654:9069]
2025-10-09T21:48:18.984075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342149237153654:9069]
2025-10-09T21:48:18.984122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:18.984124Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:18.984129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:18.984130Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:18.984132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:18.984133Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:18.984266Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:18.984634Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1178 sessionId: src_id|62c8f2c2-523b8bd8-4b900b35-984204e7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:18.987179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:19.035615Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342153893943295:9750]
2025-10-09T21:48:19.035904Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 1177, pipe: [1:7559342153893943295:9750]
2025-10-09T21:48:19.146504Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:19.146835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:19.146850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:19.146898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342153893943295:9750], now have 1 active actors on pipe
2025-10-09T21:48:19.147145Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:19.147504Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:19.147567Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342153893943295:9750]
2025-10-09T21:48:19.147592Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:19.147661Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1178 reason: status is not ok: tablet will be restarted right now sessionId: src_id|62c8f2c2-523b8bd8-4b900b35-984204e7_0
2025-10-09T21:48:19.147857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1178 sessionId: src_id|62c8f2c2-523b8bd8-4b900b35-984204e7_0 is DEAD
2025-10-09T21:48:19.148007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62c8f2c2-523b8bd8-4b900b35-984204e7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:19.148018Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62c8f2c2-523b8bd8-4b900b35-984204e7_0] Write session will restart in 0.008844s
2025-10-09T21:48:19.148044Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62c8f2c2-523b8bd8-4b900b35-984204e7_0] Write session: Do CDS request
2025-10-09T21:48:19.148059Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62c8f2c2-523b8bd8-4b900b35-984204e7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:19.158500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62c8f2c2-523b8bd8-4b900b35-984204e7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:19.158582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:19.158595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1179
2025-10-09T21:48:19.158750Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1179 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:19.158780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1179 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:19.158792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1179 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:19.158796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:19.159229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1179 sessionId:  describe result for acl check
2025-10-09T21:48:19.159263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:19.159266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:19.159268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:19.159273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342153893943391:10206] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:19.159764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342153893943391:10206] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:19.162573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342153893943391:10206] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:19.165297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342153893943391:10206] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:19.165311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342153893943391:10206] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:19.165314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342153893943391:10206] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:19.165322Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1179 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:19.166366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:19.168779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:19.168980Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:19.168992Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:19.169004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:19.169057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:19.169099Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342153532121247:9081]
2025-10-09T21:48:19.169701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:19.170061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:19.170107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:19.170422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:19.170536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:19.170543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:19.170544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:19.170545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:19.170547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:19.170548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:19.170549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.170550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.170551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.170557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:19.170568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:19.170572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:19.170576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:19.170580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:19.170585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:19.170589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.170592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.170596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.170603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:19.170611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:19.170617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:19.170622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:19.170627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:19.170632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:19.170637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:19.170642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:19.170649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:19.170659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:19.170663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:19.170668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:19.170679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.170684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.170688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.170693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.170697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.170702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.170731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:19.177323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:19.178486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:19.178501Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:19.178504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:19.178508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:19.178516Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1178 [2:7559342153532121247:9081]
2025-10-09T21:48:19.178529Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:19.178546Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:19.178550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:19.178553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:19.178555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:19.178572Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:19.178579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:19.178586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:19.178651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.178657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.178659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.178663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:19.178737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:19.178744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.178757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.178773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.178796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:19.178824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342153893943316:2432], now have 1 active actors on pipe
2025-10-09T21:48:19.178839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342153893943573:10206], now have 1 active actors on pipe
2025-10-09T21:48:19.178852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342153532121242:9081]
2025-10-09T21:48:19.178858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342153532121242:9081]
2025-10-09T21:48:19.178861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342153532121242:9081]
2025-10-09T21:48:19.178875Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:19.178893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:19.178894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:19.178898Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:19.178900Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:19.178911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:19.178914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:19.178901Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1178
2025-10-09T21:48:19.178906Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1178
2025-10-09T21:48:19.178947Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342153532121242:9081]
2025-10-09T21:48:19.178954Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:19.178956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:19.178976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:19.178982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:19.179035Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|23e65868-e788145a-ec7bbc37-804df600_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:19.179046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:19.179051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:19.179055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:19.179065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:19.179079Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:19.179099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:19.179222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:19.179230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:19.179244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:19.179320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1179 partition: 0 MaxSeqNo: 25 sessionId: src_id|23e65868-e788145a-ec7bbc37-804df600_0
2025-10-09T21:48:19.179679Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62c8f2c2-523b8bd8-4b900b35-984204e7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|23e65868-e788145a-ec7bbc37-804df600_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:19.182177Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23e65868-e788145a-ec7bbc37-804df600_0] Write session: try to update token
2025-10-09T21:48:19.182197Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23e65868-e788145a-ec7bbc37-804df600_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:19.193489Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:19.193503Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:19.193506Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:19.193508Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:19.193521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342153532121242:9081]
2025-10-09T21:48:19.193526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342153532121242:9081]
2025-10-09T21:48:19.193529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342153532121242:9081]
2025-10-09T21:48:19.193575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:19.193576Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:19.193583Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:19.193586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.193587Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:19.193589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:19.193724Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:19.195784Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1179 sessionId: src_id|23e65868-e788145a-ec7bbc37-804df600_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:19.198240Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:19.247615Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342153893943593:9750]
2025-10-09T21:48:19.247871Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 1178, pipe: [1:7559342153893943593:9750]
2025-10-09T21:48:19.354802Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:19.355084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:19.355098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:19.355145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342153893943593:9750], now have 1 active actors on pipe
2025-10-09T21:48:19.355376Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:19.355376Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342153893943593:9750]
2025-10-09T21:48:19.355392Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:19.355416Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:19.355420Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:19.355484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1179 reason: error at write: Disconnected sessionId: src_id|23e65868-e788145a-ec7bbc37-804df600_0
2025-10-09T21:48:19.355690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1179 sessionId: src_id|23e65868-e788145a-ec7bbc37-804df600_0 is DEAD
2025-10-09T21:48:19.355844Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23e65868-e788145a-ec7bbc37-804df600_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:19.355856Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23e65868-e788145a-ec7bbc37-804df600_0] Write session will restart in 0.009291s
2025-10-09T21:48:19.355885Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23e65868-e788145a-ec7bbc37-804df600_0] Write session: Do CDS request
2025-10-09T21:48:19.355893Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23e65868-e788145a-ec7bbc37-804df600_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:19.366572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23e65868-e788145a-ec7bbc37-804df600_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:19.366638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:19.366655Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1180
2025-10-09T21:48:19.366798Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1180 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:19.366833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1180 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:19.366847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1180 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:19.366852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:19.367417Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1180 sessionId:  describe result for acl check
2025-10-09T21:48:19.367456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:19.367458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:19.367460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:19.367467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342153893943652:10211] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:19.368051Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342153893943652:10211] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:19.371112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342153893943652:10211] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:19.373925Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342153893943652:10211] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:19.373941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342153893943652:10211] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:19.373945Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342153893943652:10211] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:19.373954Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1180 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:19.375566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:19.378456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:19.378695Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:19.378709Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:19.378723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:19.378782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:19.378827Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342153532121501:9085]
2025-10-09T21:48:19.379464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:19.379806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:19.379867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:19.380225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:19.380286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:19.380291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:19.380293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:19.380294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:19.380296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:19.380297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:19.380298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.380300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.380301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.380307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:19.380318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:19.380322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:19.380327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:19.380331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:19.380335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:19.380339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.380343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.380347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.380355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:19.380363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:19.380370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:19.380376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:19.380381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:19.380386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:19.380397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:19.380403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:19.380408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:19.380413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:19.380435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:19.380440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:19.380446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.380451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.380455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.380460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.380464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.380469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.380509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:19.387563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:19.388700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:19.388714Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:19.388716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:19.388720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:19.388726Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1179 [2:7559342153532121501:9085]
2025-10-09T21:48:19.388737Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:19.388754Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:19.388756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:19.388758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:19.388760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:19.388775Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:19.388784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:19.388793Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:19.388857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.388864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.388866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.388869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:19.388946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:19.388955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.388970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.388983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.389000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:19.389027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342153893943844:10211], now have 1 active actors on pipe
2025-10-09T21:48:19.389035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342153893943595:2432], now have 1 active actors on pipe
2025-10-09T21:48:19.389045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342153532121496:9085]
2025-10-09T21:48:19.389052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342153532121496:9085]
2025-10-09T21:48:19.389054Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342153532121496:9085]
2025-10-09T21:48:19.389069Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:19.389084Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:19.389089Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:19.389091Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:19.389097Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:19.389113Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:19.389117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:19.389118Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1179
2025-10-09T21:48:19.389169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342153532121496:9085]
2025-10-09T21:48:19.389177Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:19.389180Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:19.389193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:19.389201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:19.389241Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|14bed8dd-172146e4-56cb55ee-a7f1d6fd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:19.389251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:19.389255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:19.389257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:19.389270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:19.389281Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:19.389312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:19.389488Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1179
2025-10-09T21:48:19.389487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:19.389497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:19.389519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:19.389612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1180 partition: 0 MaxSeqNo: 25 sessionId: src_id|14bed8dd-172146e4-56cb55ee-a7f1d6fd_0
2025-10-09T21:48:19.390050Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23e65868-e788145a-ec7bbc37-804df600_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|14bed8dd-172146e4-56cb55ee-a7f1d6fd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:19.392687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14bed8dd-172146e4-56cb55ee-a7f1d6fd_0] Write session: try to update token
2025-10-09T21:48:19.392708Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14bed8dd-172146e4-56cb55ee-a7f1d6fd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:19.404033Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:19.404050Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:19.404053Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:19.404055Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:19.404066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342153532121496:9085]
2025-10-09T21:48:19.404070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342153532121496:9085]
2025-10-09T21:48:19.404075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342153532121496:9085]
2025-10-09T21:48:19.404121Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:19.404126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:19.404127Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:19.404131Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:19.404134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.404138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:19.404281Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:19.407396Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1180 sessionId: src_id|14bed8dd-172146e4-56cb55ee-a7f1d6fd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:19.409950Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:19.455504Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342153893943859:9750]
2025-10-09T21:48:19.455754Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 1179, pipe: [1:7559342153893943859:9750]
2025-10-09T21:48:19.565400Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:19.565625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:19.565638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:19.565677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342153893943859:9750], now have 1 active actors on pipe
2025-10-09T21:48:19.565897Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:19.565935Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342153893943859:9750]
2025-10-09T21:48:19.565942Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:19.565946Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:19.565968Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:19.566017Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1180 reason: error at write: Disconnected sessionId: src_id|14bed8dd-172146e4-56cb55ee-a7f1d6fd_0
2025-10-09T21:48:19.566212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1180 sessionId: src_id|14bed8dd-172146e4-56cb55ee-a7f1d6fd_0 is DEAD
2025-10-09T21:48:19.566353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14bed8dd-172146e4-56cb55ee-a7f1d6fd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:19.566362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14bed8dd-172146e4-56cb55ee-a7f1d6fd_0] Write session will restart in 0.005535s
2025-10-09T21:48:19.566381Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14bed8dd-172146e4-56cb55ee-a7f1d6fd_0] Write session: Do CDS request
2025-10-09T21:48:19.566385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14bed8dd-172146e4-56cb55ee-a7f1d6fd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:19.573536Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14bed8dd-172146e4-56cb55ee-a7f1d6fd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:19.573609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:19.573622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1181
2025-10-09T21:48:19.573750Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1181 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:19.573792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1181 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:19.573804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1181 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:19.573807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:19.574314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1181 sessionId:  describe result for acl check
2025-10-09T21:48:19.574347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:19.574354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:19.574356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:19.574362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342153893943876:10215] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:19.574908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342153893943876:10215] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:19.577313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342153893943876:10215] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:19.580450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342153893943876:10215] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:19.580474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342153893943876:10215] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:19.580477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342153893943876:10215] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:19.580485Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1181 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:19.585148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:19.587796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:19.587997Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:19.588006Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:19.588019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:19.588071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:19.588113Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342153532121756:9087]
2025-10-09T21:48:19.588692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:19.588997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:19.589036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:19.589334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:19.589390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:19.589396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:19.589397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:19.589398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:19.589399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:19.589401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:19.589402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.589403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.589404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.589409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:19.589419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:19.589423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:19.589427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:19.589430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:19.589434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:19.589437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.589439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.589443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.589449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:19.589456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:19.589463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:19.589468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:19.589472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:19.589477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:19.589480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:19.589485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:19.589490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:19.589494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:19.589498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:19.589503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:19.589507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.589512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.589516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.589521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.589525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.589530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.589557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:19.595919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:19.596929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:19.596943Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:19.596945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:19.596949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:19.596956Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1180 [2:7559342153532121756:9087]
2025-10-09T21:48:19.596968Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:19.596987Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:19.596989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:19.596992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:19.596995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:19.597009Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:19.597016Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:19.597024Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:19.597093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.597100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.597102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.597105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:19.597168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:19.597175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.597187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.597207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.597223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:19.597244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342153893943861:2432], now have 1 active actors on pipe
2025-10-09T21:48:19.597247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342153893944024:10215], now have 1 active actors on pipe
2025-10-09T21:48:19.597254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342153532121751:9087]
2025-10-09T21:48:19.597261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342153532121751:9087]
2025-10-09T21:48:19.597263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342153532121751:9087]
2025-10-09T21:48:19.597276Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:19.597289Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:19.597291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:19.597304Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:19.597305Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:19.597307Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:19.597308Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:19.597331Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342153532121751:9087]
2025-10-09T21:48:19.597336Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:19.597308Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1180
2025-10-09T21:48:19.597338Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:19.597474Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1180
2025-10-09T21:48:19.597517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:19.597529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:19.597564Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a9ac557f-7c8ae98b-fbc0ee41-b85400db_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:19.597576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:19.597580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:19.597582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:19.597593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:19.597607Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:19.597625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:19.597813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:19.597820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:19.597833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:19.597925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1181 partition: 0 MaxSeqNo: 25 sessionId: src_id|a9ac557f-7c8ae98b-fbc0ee41-b85400db_0
2025-10-09T21:48:19.598286Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14bed8dd-172146e4-56cb55ee-a7f1d6fd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a9ac557f-7c8ae98b-fbc0ee41-b85400db_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:19.600861Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9ac557f-7c8ae98b-fbc0ee41-b85400db_0] Write session: try to update token
2025-10-09T21:48:19.600880Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9ac557f-7c8ae98b-fbc0ee41-b85400db_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:19.608854Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:19.608881Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:19.608891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342153532121751:9087]
2025-10-09T21:48:19.608938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:19.608937Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:19.609110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:19.609245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:19.609375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:19.609510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:19.609693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:19.609750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:19.617918Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1181 sessionId: src_id|a9ac557f-7c8ae98b-fbc0ee41-b85400db_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:19.620337Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:19.620521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:19.620541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:19.620593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:19.620663Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:19.621790Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:19.621801Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:19.621804Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:19.621806Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:19.621815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342153532121751:9087]
2025-10-09T21:48:19.621819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342153532121751:9087]
2025-10-09T21:48:19.621823Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342153532121751:9087]
2025-10-09T21:48:19.621863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:19.621866Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:19.621870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.621872Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:19.621873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:19.621876Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:19.622007Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:19.665919Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342153893944126:9750]
2025-10-09T21:48:19.666203Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 1180, pipe: [1:7559342153893944126:9750]
2025-10-09T21:48:19.783764Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:19.792129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:19.792153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:19.792329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:19.792388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:19.792449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:19.792507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:19.792551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:19.792560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:19.792623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:19.792683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:19.792741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:19.792809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:19.792854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:19.792863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:19.792921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:19.792979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:19.793042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:19.793100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:19.793552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:19.793564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:19.793630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:19.793691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:19.793767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:19.793825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:19.793871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:19.793879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:19.793940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:19.794000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:19.794079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:19.794141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:19.794179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:19.794188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:19.794256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:19.794308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:19.794364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:19.794418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:19.794455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:19.794460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:19.794512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:19.794570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:19.794635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:19.794691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:19.794732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:19.794739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:19.794797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:19.794852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:19.794911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:19.794970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:19.795015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:19.795022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:19.795085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:19.795142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:19.795203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:19.795261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:19.795304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:19.795311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:19.795374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:19.795432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:19.795491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:19.795549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:19.795591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:19.795598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:19.795657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342153893944126:9750], now have 1 active actors on pipe
2025-10-09T21:48:19.795882Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342153893944126:9750]
2025-10-09T21:48:19.795924Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:19.795927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:19.795957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1181 reason: error at write: Disconnected sessionId: src_id|a9ac557f-7c8ae98b-fbc0ee41-b85400db_0
2025-10-09T21:48:19.796164Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:19.796280Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:19.796296Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9ac557f-7c8ae98b-fbc0ee41-b85400db_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:19.796320Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9ac557f-7c8ae98b-fbc0ee41-b85400db_0] Write session will restart in 0.008466s
2025-10-09T21:48:19.796345Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9ac557f-7c8ae98b-fbc0ee41-b85400db_0] Write session: Do CDS request
2025-10-09T21:48:19.796351Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9ac557f-7c8ae98b-fbc0ee41-b85400db_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:19.796352Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1181 sessionId: src_id|a9ac557f-7c8ae98b-fbc0ee41-b85400db_0 is DEAD
2025-10-09T21:48:19.805506Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9ac557f-7c8ae98b-fbc0ee41-b85400db_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:19.805569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:19.805583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1182
2025-10-09T21:48:19.805726Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1182 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:19.805784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1182 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:19.805794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1182 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:19.805798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:19.806287Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1182 sessionId:  describe result for acl check
2025-10-09T21:48:19.806326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:19.806330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:19.806332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:19.806338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342153893944162:10219] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:19.806885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342153893944162:10219] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:19.810339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342153893944162:10219] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:19.813205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342153893944162:10219] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:19.813218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342153893944162:10219] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:19.813222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342153893944162:10219] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:19.813229Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1182 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:19.815395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:19.817910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:19.818119Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:19.818130Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:19.818143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:19.818205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:19.818247Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342153532122017:9091]
2025-10-09T21:48:19.818829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:19.819244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:19.819299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:19.819643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:19.819699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:19.819705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:19.819706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:19.819707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:19.819708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:19.819709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:19.819711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.819712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.819713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.819718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:19.819729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:19.819733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:19.819737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:19.819741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:19.819744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:19.819748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.819752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.819756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.819763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:19.819771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:19.819780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:19.819785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:19.819790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:19.819795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:19.819800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:19.819805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:19.819810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:19.819815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:19.819819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:19.819825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:19.819829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.819835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.819840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.819845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.819850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.819860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.819887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:19.826648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:19.827729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:19.827745Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:19.827749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:19.827753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:19.827763Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1181 [2:7559342153532122017:9091]
2025-10-09T21:48:19.827778Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:19.827798Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:19.827804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:19.827807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:19.827809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:19.827829Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:19.827835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:19.827843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:19.827901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.827912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.827915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.827918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:19.828000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:19.828007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.828019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:19.828035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:19.828052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:19.828070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342153893944372:10219], now have 1 active actors on pipe
2025-10-09T21:48:19.828073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342153893944129:2432], now have 1 active actors on pipe
2025-10-09T21:48:19.828079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342153532122012:9091]
2025-10-09T21:48:19.828080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342153532122012:9091]
2025-10-09T21:48:19.828081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342153532122012:9091]
2025-10-09T21:48:19.828091Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:19.828103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:19.828107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:19.828111Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:19.828113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:19.828117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:19.828120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:19.828160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342153532122012:9091]
2025-10-09T21:48:19.828173Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:19.828157Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1181
2025-10-09T21:48:19.828162Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1181
2025-10-09T21:48:19.828209Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:19.828298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:19.828307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:19.828328Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|51dfb084-901c3932-8908ef4c-c51bbb97_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:19.828334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:19.828338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:19.828340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:19.828349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:19.828357Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:19.828376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:19.828529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:19.828546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:19.828563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:19.828702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1182 partition: 0 MaxSeqNo: 25 sessionId: src_id|51dfb084-901c3932-8908ef4c-c51bbb97_0
2025-10-09T21:48:19.829077Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9ac557f-7c8ae98b-fbc0ee41-b85400db_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|51dfb084-901c3932-8908ef4c-c51bbb97_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:19.831636Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51dfb084-901c3932-8908ef4c-c51bbb97_0] Write session: try to update token
2025-10-09T21:48:19.831656Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51dfb084-901c3932-8908ef4c-c51bbb97_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:19.839842Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:19.839859Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:19.839868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342153532122012:9091]
2025-10-09T21:48:19.839914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:19.839937Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:19.840097Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:19.840243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:19.840381Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:19.840521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:19.840710Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:19.840758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:19.845121Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1182 sessionId: src_id|51dfb084-901c3932-8908ef4c-c51bbb97_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:19.847533Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:19.847744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:19.847762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:19.847812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:19.847895Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:19.853130Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:19.853147Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:19.853150Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:19.853152Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:19.853163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342153532122012:9091]
2025-10-09T21:48:19.853168Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342153532122012:9091]
2025-10-09T21:48:19.853171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342153532122012:9091]
2025-10-09T21:48:19.853209Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:19.853215Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:19.853216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:19.853222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:19.853225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:19.853228Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:19.853359Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:19.895818Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342153893944396:9750]
2025-10-09T21:48:19.896082Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 1181, pipe: [1:7559342153893944396:9750]
2025-10-09T21:48:20.017034Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:20.034409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:20.034433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:20.034573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:20.034633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:20.034696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:20.034759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:20.034809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:20.034823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:20.034897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:20.034963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:20.035023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:20.035081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:20.035127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:20.035135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:20.035199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:20.035258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:20.035313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:20.035376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:20.035418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:20.035426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:20.035493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:20.035555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:20.035614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:20.035673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:20.035720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:20.035729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:20.035799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:20.035858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:20.035924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:20.035990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:20.036036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:20.036046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:20.036107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:20.036171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:20.036238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:20.036299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:20.036340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:20.036346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:20.036403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:20.036458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:20.036525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:20.036580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:20.036622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:20.036630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:20.036699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:20.036765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:20.036830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:20.036897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:20.036948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:20.036958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:20.037023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:20.037085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:20.037147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:20.037215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:20.037353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:20.037362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:20.037427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:20.037485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:20.037554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:20.037618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:20.037660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:20.037667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:20.037724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342153893944396:9750], now have 1 active actors on pipe
2025-10-09T21:48:20.037920Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:20.037925Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:20.037943Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:20.037972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1182 reason: error at write: Disconnected sessionId: src_id|51dfb084-901c3932-8908ef4c-c51bbb97_0
2025-10-09T21:48:20.037979Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342153893944396:9750]
2025-10-09T21:48:20.038186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1182 sessionId: src_id|51dfb084-901c3932-8908ef4c-c51bbb97_0 is DEAD
2025-10-09T21:48:20.038377Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51dfb084-901c3932-8908ef4c-c51bbb97_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:20.038404Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51dfb084-901c3932-8908ef4c-c51bbb97_0] Write session will restart in 0.005973s
2025-10-09T21:48:20.038432Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51dfb084-901c3932-8908ef4c-c51bbb97_0] Write session: Do CDS request
2025-10-09T21:48:20.038438Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51dfb084-901c3932-8908ef4c-c51bbb97_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:20.038911Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:20.045546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51dfb084-901c3932-8908ef4c-c51bbb97_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:20.045668Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:20.045683Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1183
2025-10-09T21:48:20.045801Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1183 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:20.045826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1183 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:20.045836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1183 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:20.045839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:20.046348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1183 sessionId:  describe result for acl check
2025-10-09T21:48:20.046385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:20.046391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:20.046393Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:20.046399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342158188911714:10224] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:20.046970Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342158188911714:10224] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:20.050574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342158188911714:10224] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:20.054432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342158188911714:10224] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:20.054445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342158188911714:10224] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:20.054449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342158188911714:10224] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:20.054457Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1183 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:20.057469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:20.060070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:20.060281Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:20.060292Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:20.060303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:20.060358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:20.060400Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342157827089583:9099]
2025-10-09T21:48:20.060999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:20.061338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:20.061387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:20.061710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:20.061785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.061792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.061793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.061794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.061795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.061796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.061797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.061798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.061799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.061805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.061814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.061818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.061822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.061825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.061829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.061832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.061835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.061839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.061846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.061854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.061860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.061864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.061870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.061874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.061878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.061882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.061887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.061891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.061895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.061899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.061904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.061909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.061913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.061918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.061922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.061927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.061955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:20.070192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:20.071488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:20.071504Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:20.071506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:20.071510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:20.071517Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1182 [2:7559342157827089583:9099]
2025-10-09T21:48:20.071530Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:20.071549Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:20.071552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:20.071555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:20.071557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:20.071575Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:20.071586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:20.071596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:20.071655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.071663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.071665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.071668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:20.071742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:20.071750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.071762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.071778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.071798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:20.071826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342158188911699:2432], now have 1 active actors on pipe
2025-10-09T21:48:20.071829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342158188911908:10224], now have 1 active actors on pipe
2025-10-09T21:48:20.071838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342157827089578:9099]
2025-10-09T21:48:20.071846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342157827089578:9099]
2025-10-09T21:48:20.071847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342157827089578:9099]
2025-10-09T21:48:20.071859Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:20.071874Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:20.071876Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:20.071879Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:20.071881Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:20.071888Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:20.071890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:20.071853Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1182
2025-10-09T21:48:20.071884Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1182
2025-10-09T21:48:20.071944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:20.071953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:20.071966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342157827089578:9099]
2025-10-09T21:48:20.071978Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:20.071996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:20.072018Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|579524bc-36bc20e2-a9a767e3-453d810_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:20.072031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:20.072041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:20.072044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:20.072054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:20.072070Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:20.072099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:20.072271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:20.072284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:20.072298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:20.072389Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1183 partition: 0 MaxSeqNo: 25 sessionId: src_id|579524bc-36bc20e2-a9a767e3-453d810_0
2025-10-09T21:48:20.072753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51dfb084-901c3932-8908ef4c-c51bbb97_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|579524bc-36bc20e2-a9a767e3-453d810_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:20.079853Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|579524bc-36bc20e2-a9a767e3-453d810_0] Write session: try to update token
2025-10-09T21:48:20.079874Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|579524bc-36bc20e2-a9a767e3-453d810_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:20.086532Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:20.086549Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:20.086552Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:20.086554Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:20.086564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342157827089578:9099]
2025-10-09T21:48:20.086568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342157827089578:9099]
2025-10-09T21:48:20.086571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342157827089578:9099]
2025-10-09T21:48:20.086618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:20.086622Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:20.086629Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:20.086630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.086633Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:20.086633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:20.086773Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:20.093291Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1183 sessionId: src_id|579524bc-36bc20e2-a9a767e3-453d810_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:20.099581Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:20.137988Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342158188911967:9750]
2025-10-09T21:48:20.138289Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 1182, pipe: [1:7559342158188911967:9750]
2025-10-09T21:48:20.248264Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:20.248544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:20.248559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:20.248584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342158188911967:9750], now have 1 active actors on pipe
2025-10-09T21:48:20.248782Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:20.248761Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342158188911967:9750]
2025-10-09T21:48:20.248782Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:20.248786Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:20.248792Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:20.248846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1183 reason: error at write: Disconnected sessionId: src_id|579524bc-36bc20e2-a9a767e3-453d810_0
2025-10-09T21:48:20.249029Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1183 sessionId: src_id|579524bc-36bc20e2-a9a767e3-453d810_0 is DEAD
2025-10-09T21:48:20.249190Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|579524bc-36bc20e2-a9a767e3-453d810_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:20.249203Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|579524bc-36bc20e2-a9a767e3-453d810_0] Write session will restart in 0.005630s
2025-10-09T21:48:20.249243Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|579524bc-36bc20e2-a9a767e3-453d810_0] Write session: Do CDS request
2025-10-09T21:48:20.249250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|579524bc-36bc20e2-a9a767e3-453d810_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:20.256558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|579524bc-36bc20e2-a9a767e3-453d810_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:20.256615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:20.256637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1184
2025-10-09T21:48:20.256796Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1184 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:20.256849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1184 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:20.256860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1184 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:20.256863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:20.257368Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1184 sessionId:  describe result for acl check
2025-10-09T21:48:20.257402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:20.257408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:20.257410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:20.257415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342158188912023:10236] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:20.258238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342158188912023:10236] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:20.261680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342158188912023:10236] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:20.264857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342158188912023:10236] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:20.264873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342158188912023:10236] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:20.264878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342158188912023:10236] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:20.264947Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1184 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:20.267331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:20.270143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:20.270339Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:20.270349Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:20.270360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:20.270412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:20.270465Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342157827089866:9107]
2025-10-09T21:48:20.271031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:20.271335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:20.271379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:20.271700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:20.271743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.271748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.271749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.271750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.271751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.271753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.271754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.271755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.271756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.271761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.271772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.271775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.271779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.271783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.271787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.271791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.271795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.271799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.271806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.271813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.271819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.271824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.271829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.271833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.271839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.271848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.271853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.271858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.271868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.271873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.271879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.271884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.271893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.271898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.271902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.271907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.271939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:20.278238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:20.279298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:20.279316Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:20.279318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:20.279323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:20.279330Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1183 [2:7559342157827089866:9107]
2025-10-09T21:48:20.279343Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:20.279361Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:20.279364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:20.279368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:20.279371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:20.279388Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:20.279399Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:20.279406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:20.279465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.279473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.279475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.279478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:20.279543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:20.279551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.279563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.279579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.279599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:20.279620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342158188911999:2432], now have 1 active actors on pipe
2025-10-09T21:48:20.279628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342158188912216:10236], now have 1 active actors on pipe
2025-10-09T21:48:20.279636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342157827089861:9107]
2025-10-09T21:48:20.279642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342157827089861:9107]
2025-10-09T21:48:20.279644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342157827089861:9107]
2025-10-09T21:48:20.279658Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:20.279677Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:20.279685Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:20.279690Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:20.279692Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:20.279695Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:20.279698Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:20.279669Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1183
2025-10-09T21:48:20.279685Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1183
2025-10-09T21:48:20.279729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342157827089861:9107]
2025-10-09T21:48:20.279742Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:20.279747Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:20.279762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:20.279771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:20.279818Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|16a4fed-564ee6e2-9ad4221f-2648bb9d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:20.279829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:20.279833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:20.279835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:20.279845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:20.279859Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:20.279878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:20.280040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:20.280046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:20.280058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:20.280147Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1184 partition: 0 MaxSeqNo: 25 sessionId: src_id|16a4fed-564ee6e2-9ad4221f-2648bb9d_0
2025-10-09T21:48:20.280520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|579524bc-36bc20e2-a9a767e3-453d810_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|16a4fed-564ee6e2-9ad4221f-2648bb9d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:20.282953Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16a4fed-564ee6e2-9ad4221f-2648bb9d_0] Write session: try to update token
2025-10-09T21:48:20.282971Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16a4fed-564ee6e2-9ad4221f-2648bb9d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:20.294190Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:20.294205Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:20.294207Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:20.294209Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:20.294219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342157827089861:9107]
2025-10-09T21:48:20.294224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342157827089861:9107]
2025-10-09T21:48:20.294227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342157827089861:9107]
2025-10-09T21:48:20.294271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:20.294274Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:20.294281Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:20.294281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.294284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:20.294284Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:20.294412Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:20.296234Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1184 sessionId: src_id|16a4fed-564ee6e2-9ad4221f-2648bb9d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:20.299110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:20.348937Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342158188912263:9750]
2025-10-09T21:48:20.349242Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 1183, pipe: [1:7559342158188912263:9750]
2025-10-09T21:48:20.454336Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:20.454603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:20.454618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:20.454664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342158188912263:9750], now have 1 active actors on pipe
2025-10-09T21:48:20.454872Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:20.454897Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:20.454924Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342158188912263:9750]
2025-10-09T21:48:20.454950Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:20.455021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1184 reason: status is not ok: tablet will be restarted right now sessionId: src_id|16a4fed-564ee6e2-9ad4221f-2648bb9d_0
2025-10-09T21:48:20.455216Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1184 sessionId: src_id|16a4fed-564ee6e2-9ad4221f-2648bb9d_0 is DEAD
2025-10-09T21:48:20.455408Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16a4fed-564ee6e2-9ad4221f-2648bb9d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:20.455418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16a4fed-564ee6e2-9ad4221f-2648bb9d_0] Write session will restart in 0.009652s
2025-10-09T21:48:20.455449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16a4fed-564ee6e2-9ad4221f-2648bb9d_0] Write session: Do CDS request
2025-10-09T21:48:20.455454Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16a4fed-564ee6e2-9ad4221f-2648bb9d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:20.466576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16a4fed-564ee6e2-9ad4221f-2648bb9d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:20.466667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:20.466683Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1185
2025-10-09T21:48:20.466801Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1185 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:20.466836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1185 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:20.466842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1185 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:20.466852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:20.467282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1185 sessionId:  describe result for acl check
2025-10-09T21:48:20.467318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:20.467323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:20.467326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:20.467332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342158188912348:5335] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:20.467869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342158188912348:5335] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:20.470739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342158188912348:5335] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:20.473386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342158188912348:5335] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:20.473400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342158188912348:5335] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:20.473404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342158188912348:5335] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:20.473412Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1185 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:20.473645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:20.475802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:20.476001Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:20.476009Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:20.476021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:20.476078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:20.476115Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342157827090126:9109]
2025-10-09T21:48:20.476543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:20.476827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:20.476869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:20.477177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:20.477225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.477231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.477232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.477233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.477234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.477235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.477236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.477237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.477239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.477244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.477254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.477258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.477262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.477266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.477270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.477274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.477277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.477282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.477287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.477295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.477301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.477307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.477311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.477316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.477320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.477325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.477330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.477335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.477339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.477344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.477349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.477354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.477358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.477363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.477368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.477377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.477404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:20.483597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:20.484675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:20.484690Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:20.484693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:20.484697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:20.484704Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1184 [2:7559342157827090126:9109]
2025-10-09T21:48:20.484718Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:20.484736Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:20.484744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:20.484748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:20.484751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:20.484766Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:20.484776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:20.484783Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:20.484841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.484850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.484851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.484854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:20.485052Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1184
2025-10-09T21:48:20.485099Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1184
2025-10-09T21:48:20.484939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:20.484947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.484960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.484972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.484990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:20.485012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342158188912266:2432], now have 1 active actors on pipe
2025-10-09T21:48:20.485016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342158188912522:5335], now have 1 active actors on pipe
2025-10-09T21:48:20.485023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342157827090121:9109]
2025-10-09T21:48:20.485025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342157827090121:9109]
2025-10-09T21:48:20.485028Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342157827090121:9109]
2025-10-09T21:48:20.485041Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:20.485055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:20.485055Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:20.485058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:20.485060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:20.485072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:20.485075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:20.485109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342157827090121:9109]
2025-10-09T21:48:20.485119Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:20.485134Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:20.485147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:20.485152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:20.485182Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6ea67f4c-cf687c84-2369607f-45923b91_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:20.485189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:20.485193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:20.485195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:20.485205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:20.485216Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:20.485229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:20.485372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:20.485388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:20.485410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:20.485518Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1185 partition: 0 MaxSeqNo: 25 sessionId: src_id|6ea67f4c-cf687c84-2369607f-45923b91_0
2025-10-09T21:48:20.485882Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16a4fed-564ee6e2-9ad4221f-2648bb9d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6ea67f4c-cf687c84-2369607f-45923b91_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:20.488417Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ea67f4c-cf687c84-2369607f-45923b91_0] Write session: try to update token
2025-10-09T21:48:20.488434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ea67f4c-cf687c84-2369607f-45923b91_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:20.499299Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:20.499321Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:20.499324Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:20.499326Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:20.499336Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342157827090121:9109]
2025-10-09T21:48:20.499341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342157827090121:9109]
2025-10-09T21:48:20.499345Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342157827090121:9109]
2025-10-09T21:48:20.499392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:20.499399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.499399Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:20.499402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:20.499406Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:20.499409Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:20.499552Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:20.501805Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1185 sessionId: src_id|6ea67f4c-cf687c84-2369607f-45923b91_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:20.504329Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:20.555266Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342158188912534:9750]
2025-10-09T21:48:20.555593Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 1184, pipe: [1:7559342158188912534:9750]
2025-10-09T21:48:20.655482Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:20.656209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:20.656227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:20.656275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342158188912534:9750], now have 1 active actors on pipe
2025-10-09T21:48:20.656486Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:20.656504Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342158188912534:9750]
2025-10-09T21:48:20.656533Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:20.656536Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:20.656596Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1185 reason: error at write: Disconnected sessionId: src_id|6ea67f4c-cf687c84-2369607f-45923b91_0
2025-10-09T21:48:20.656761Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:20.656801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1185 sessionId: src_id|6ea67f4c-cf687c84-2369607f-45923b91_0 is DEAD
2025-10-09T21:48:20.656974Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ea67f4c-cf687c84-2369607f-45923b91_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:20.656997Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ea67f4c-cf687c84-2369607f-45923b91_0] Write session will restart in 0.008369s
2025-10-09T21:48:20.657024Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ea67f4c-cf687c84-2369607f-45923b91_0] Write session: Do CDS request
2025-10-09T21:48:20.657030Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ea67f4c-cf687c84-2369607f-45923b91_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:20.666608Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ea67f4c-cf687c84-2369607f-45923b91_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:20.666635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:20.666651Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1186
2025-10-09T21:48:20.666847Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1186 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:20.666880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1186 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:20.666890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1186 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:20.666895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:20.667289Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1186 sessionId:  describe result for acl check
2025-10-09T21:48:20.667323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:20.667326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:20.667327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:20.667332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342158188912584:5319] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:20.667879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342158188912584:5319] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:20.671371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342158188912584:5319] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:20.674474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342158188912584:5319] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:20.674489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342158188912584:5319] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:20.674493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342158188912584:5319] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:20.674511Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1186 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:20.676033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:20.678604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:20.678791Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:20.678800Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:20.678812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:20.678871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:20.678913Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342157827090390:9113]
2025-10-09T21:48:20.679471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:20.679781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:20.679829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:20.680118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:20.680160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.680165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.680166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.680167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.680168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.680169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.680170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.680171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.680172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.680176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.680185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.680188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.680192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.680195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.680198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.680202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.680205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.680207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.680214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.680221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.680226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.680230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.680233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.680237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.680241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.680244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.680248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.680252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.680255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.680260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.680268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.680273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.680276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.680280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.680288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.680293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.680316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:20.686336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:20.687623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:20.687635Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:20.687637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:20.687641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:20.687647Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1185 [2:7559342157827090390:9113]
2025-10-09T21:48:20.687660Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:20.687675Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:20.687678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:20.687681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:20.687684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:20.687700Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:20.687706Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:20.687721Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:20.687780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.687787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.687788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.687790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:20.687855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:20.687862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.687874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.687885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.687905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:20.687929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342158188912792:5319], now have 1 active actors on pipe
2025-10-09T21:48:20.687936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342158188912537:2432], now have 1 active actors on pipe
2025-10-09T21:48:20.687943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342157827090385:9113]
2025-10-09T21:48:20.687945Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342157827090385:9113]
2025-10-09T21:48:20.687947Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342157827090385:9113]
2025-10-09T21:48:20.687961Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:20.687979Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:20.687984Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:20.687989Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:20.687991Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:20.687997Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:20.688000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:20.688004Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1185
2025-10-09T21:48:20.688030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342157827090385:9113]
2025-10-09T21:48:20.688037Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:20.688005Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1185
2025-10-09T21:48:20.688039Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:20.688064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:20.688072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:20.688105Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2e3b0b27-bc3d66b4-61c85c51-33bc4414_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:20.688117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:20.688125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:20.688128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:20.688138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:20.688148Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:20.688164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:20.688327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:20.688339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:20.688360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:20.688474Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1186 partition: 0 MaxSeqNo: 25 sessionId: src_id|2e3b0b27-bc3d66b4-61c85c51-33bc4414_0
2025-10-09T21:48:20.688839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ea67f4c-cf687c84-2369607f-45923b91_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2e3b0b27-bc3d66b4-61c85c51-33bc4414_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:20.691252Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e3b0b27-bc3d66b4-61c85c51-33bc4414_0] Write session: try to update token
2025-10-09T21:48:20.691269Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e3b0b27-bc3d66b4-61c85c51-33bc4414_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:20.701695Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:20.701708Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:20.701710Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:20.701712Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:20.701723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342157827090385:9113]
2025-10-09T21:48:20.701736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342157827090385:9113]
2025-10-09T21:48:20.701740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342157827090385:9113]
2025-10-09T21:48:20.701787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:20.701793Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:20.701796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.701799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:20.701803Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:20.701808Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:20.702311Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:20.705231Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1186 sessionId: src_id|2e3b0b27-bc3d66b4-61c85c51-33bc4414_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:20.707592Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:20.756467Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342158188912807:9750]
2025-10-09T21:48:20.756769Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 1185, pipe: [1:7559342158188912807:9750]
2025-10-09T21:48:20.857501Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:20.858019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:20.858050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:20.858102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342158188912807:9750], now have 1 active actors on pipe
2025-10-09T21:48:20.858326Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:20.858355Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342158188912807:9750]
2025-10-09T21:48:20.858380Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:20.858457Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1186 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2e3b0b27-bc3d66b4-61c85c51-33bc4414_0
2025-10-09T21:48:20.858646Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1186 sessionId: src_id|2e3b0b27-bc3d66b4-61c85c51-33bc4414_0 is DEAD
2025-10-09T21:48:20.858309Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:20.858764Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e3b0b27-bc3d66b4-61c85c51-33bc4414_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:20.858776Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e3b0b27-bc3d66b4-61c85c51-33bc4414_0] Write session will restart in 0.006592s
2025-10-09T21:48:20.858813Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e3b0b27-bc3d66b4-61c85c51-33bc4414_0] Write session: Do CDS request
2025-10-09T21:48:20.858819Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e3b0b27-bc3d66b4-61c85c51-33bc4414_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:20.866494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e3b0b27-bc3d66b4-61c85c51-33bc4414_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:20.866633Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:20.866649Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1187
2025-10-09T21:48:20.866784Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1187 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:20.866824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1187 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:20.866834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1187 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:20.866839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:20.867281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1187 sessionId:  describe result for acl check
2025-10-09T21:48:20.867325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:20.867327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:20.867328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:20.867337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342158188912838:5321] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:20.867875Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342158188912838:5321] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:20.870957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342158188912838:5321] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:20.873754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342158188912838:5321] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:20.873766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342158188912838:5321] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:20.873770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342158188912838:5321] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:20.873777Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1187 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:20.877483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:20.879766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:20.879974Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:20.879984Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:20.879996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:20.880068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:20.880109Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342157827090651:9115]
2025-10-09T21:48:20.880686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:20.881004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:20.881064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:20.881382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:20.881433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.881439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.881441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.881442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.881443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.881444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.881445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.881446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.881447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.881451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.881463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.881467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.881471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.881475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.881479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.881483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.881487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.881490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.881497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.881505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:20.881512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.881517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:20.881522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.881527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:20.881531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.881536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:20.881540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.881545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:20.881549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.881554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:20.881559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.881566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.881571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.881576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.881584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.881590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.881621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:20.882905Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:48:20.882927Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 11
2025-10-09T21:48:20.888019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:20.889074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:20.889088Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:20.889092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:20.889095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:20.889103Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1186 [2:7559342157827090651:9115]
2025-10-09T21:48:20.889117Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:20.889135Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:20.889146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:20.889149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:20.889152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:20.889169Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:20.889175Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:20.889182Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:20.889235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.889241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.889243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.889246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:20.889312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:20.889320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.889333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:20.889345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:20.889361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:20.889386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342158188912814:2432], now have 1 active actors on pipe
2025-10-09T21:48:20.889390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342158188913001:5321], now have 1 active actors on pipe
2025-10-09T21:48:20.889397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342157827090646:9115]
2025-10-09T21:48:20.889400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342157827090646:9115]
2025-10-09T21:48:20.889402Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342157827090646:9115]
2025-10-09T21:48:20.889416Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:20.889459Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1186
2025-10-09T21:48:20.889430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:20.889430Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:20.889432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:20.889437Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:20.889445Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:20.889461Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1186
2025-10-09T21:48:20.889448Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:20.889472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342157827090646:9115]
2025-10-09T21:48:20.889476Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:20.889485Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:20.889520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:20.889524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:20.889596Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:48:20.889635Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c7d1e564-9712bf11-1e8e83c2-754784ef_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:20.889647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:20.889652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:20.889659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:20.889669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:20.889684Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:20.889700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:20.889834Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 14 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:48:20.889887Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:48:20.889899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:20.889912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:20.889936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:20.890066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1187 partition: 0 MaxSeqNo: 25 sessionId: src_id|c7d1e564-9712bf11-1e8e83c2-754784ef_0
2025-10-09T21:48:20.890395Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e3b0b27-bc3d66b4-61c85c51-33bc4414_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c7d1e564-9712bf11-1e8e83c2-754784ef_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:20.892854Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c7d1e564-9712bf11-1e8e83c2-754784ef_0] Write session: try to update token
2025-10-09T21:48:20.892870Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c7d1e564-9712bf11-1e8e83c2-754784ef_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:20.903703Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:20.903723Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:20.903726Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:20.903728Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:20.903738Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342157827090646:9115]
2025-10-09T21:48:20.903743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342157827090646:9115]
2025-10-09T21:48:20.903747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342157827090646:9115]
2025-10-09T21:48:20.903793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:20.903800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:20.903802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:20.903809Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:20.903816Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:20.903821Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:20.903946Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:20.910191Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1187 sessionId: src_id|c7d1e564-9712bf11-1e8e83c2-754784ef_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:20.912538Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:20.958732Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342158188913087:9750]
2025-10-09T21:48:20.958971Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 1186, pipe: [1:7559342158188913087:9750]
2025-10-09T21:48:21.062005Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:21.062607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.062623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.062646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342158188913087:9750], now have 1 active actors on pipe
2025-10-09T21:48:21.062845Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:21.062957Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:21.062964Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342158188913087:9750]
2025-10-09T21:48:21.062987Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:21.062994Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 0
2025-10-09T21:48:21.063060Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1187 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c7d1e564-9712bf11-1e8e83c2-754784ef_0
2025-10-09T21:48:21.063234Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1187 sessionId: src_id|c7d1e564-9712bf11-1e8e83c2-754784ef_0 is DEAD
2025-10-09T21:48:21.063412Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7d1e564-9712bf11-1e8e83c2-754784ef_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:21.063436Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7d1e564-9712bf11-1e8e83c2-754784ef_0] Write session will restart in 0.007146s
2025-10-09T21:48:21.063462Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7d1e564-9712bf11-1e8e83c2-754784ef_0] Write session: Do CDS request
2025-10-09T21:48:21.063470Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7d1e564-9712bf11-1e8e83c2-754784ef_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:21.071543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c7d1e564-9712bf11-1e8e83c2-754784ef_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:21.071610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:21.071626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1188
2025-10-09T21:48:21.071779Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1188 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:21.071815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1188 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:21.071826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1188 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:21.071830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:21.072337Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1188 sessionId:  describe result for acl check
2025-10-09T21:48:21.072378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:21.072380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:21.072382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:21.072389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342162483880413:5317] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:21.072945Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342162483880413:5317] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:21.075596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342162483880413:5317] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:21.079121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342162483880413:5317] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:21.079138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342162483880413:5317] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:21.079141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342162483880413:5317] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:21.079150Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1188 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:21.082988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:21.085381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:21.085683Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:21.085695Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:21.085710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:21.085798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:21.085839Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342162122058223:9118]
2025-10-09T21:48:21.086460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:21.086778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:21.086832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:21.087167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:21.087222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.087227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.087227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.087228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.087229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.087229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.087230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.087230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.087231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.087234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.087241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.087243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.087247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.087250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.087254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.087258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.087261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.087265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.087273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.087281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.087287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.087292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.087296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.087303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.087306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.087312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.087316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.087320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.087329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.087333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.087339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.087349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.087353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.087358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.087362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.087368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.087394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:21.093849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:21.094992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:21.095004Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:21.095007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:21.095011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:21.095016Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1187 [2:7559342162122058223:9118]
2025-10-09T21:48:21.095029Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:21.095045Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:21.095048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:21.095051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:21.095054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:21.095072Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:21.095077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:21.095089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:21.095142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.095149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.095150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.095153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:21.095219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:21.095228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.095239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.095248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.095264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:21.095288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342162483880389:2432], now have 1 active actors on pipe
2025-10-09T21:48:21.095291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342162483880588:5317], now have 1 active actors on pipe
2025-10-09T21:48:21.095298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342162122058218:9118]
2025-10-09T21:48:21.095300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342162122058218:9118]
2025-10-09T21:48:21.095302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342162122058218:9118]
2025-10-09T21:48:21.095318Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:21.095332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:21.095334Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:21.095337Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:21.095339Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:21.095344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:21.095347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:21.095376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342162122058218:9118]
2025-10-09T21:48:21.095384Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:21.095365Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1187
2025-10-09T21:48:21.095398Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1187
2025-10-09T21:48:21.095404Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:21.095460Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:48:21.095498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.095506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.095528Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b2382a70-e9c40464-574f48d8-fb783609_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:21.095540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:21.095544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:21.095546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:21.095555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:21.095569Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:21.095584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:21.095743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.095753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.095778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:21.095770Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 15 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:48:21.095803Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:48:21.095854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1188 partition: 0 MaxSeqNo: 25 sessionId: src_id|b2382a70-e9c40464-574f48d8-fb783609_0
2025-10-09T21:48:21.096167Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7d1e564-9712bf11-1e8e83c2-754784ef_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b2382a70-e9c40464-574f48d8-fb783609_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:21.098607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b2382a70-e9c40464-574f48d8-fb783609_0] Write session: try to update token
2025-10-09T21:48:21.098625Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b2382a70-e9c40464-574f48d8-fb783609_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:21.112259Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1188 sessionId: src_id|b2382a70-e9c40464-574f48d8-fb783609_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:21.114179Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:21.114199Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:21.114210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342162122058218:9118]
2025-10-09T21:48:21.114262Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:21.114265Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:21.114442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:21.114595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:21.114733Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:21.114814Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:21.114870Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:21.115057Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:21.115119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.115135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.115157Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:21.115199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:21.115294Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:21.123042Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:21.123063Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:21.123065Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:21.123070Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:21.123081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342162122058218:9118]
2025-10-09T21:48:21.123084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342162122058218:9118]
2025-10-09T21:48:21.123087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342162122058218:9118]
2025-10-09T21:48:21.123133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:21.123135Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:21.123141Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:21.123144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.123145Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:21.123146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:21.123295Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:21.163013Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342162483880668:9750]
2025-10-09T21:48:21.163334Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 1187, pipe: [1:7559342162483880668:9750]
2025-10-09T21:48:21.286547Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:21.294581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.294611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.294779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.294840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.294907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.294977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.295433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.295449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.295524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.295586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.295649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.295716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.295760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.295768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.295832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.295890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.295951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.296010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.296054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.296057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.296121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.296188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.296252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.296322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.296370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.296381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.296447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.296506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.296568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.296632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.296694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.296702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.296770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.296835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.296988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.297052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.297095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.297102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.297256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.297311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.297375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.297435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.297488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.297495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.297556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.297615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.297675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.297746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.297786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.297793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.297855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.297913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.297980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.298061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.298103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.298112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.298175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.298236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.298294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.298350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.298391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.298397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.298453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342162483880668:9750], now have 1 active actors on pipe
2025-10-09T21:48:21.298734Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342162483880668:9750]
2025-10-09T21:48:21.298757Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:21.298762Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:21.298764Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:21.298793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1188 reason: error at write: Disconnected sessionId: src_id|b2382a70-e9c40464-574f48d8-fb783609_0
2025-10-09T21:48:21.298986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1188 sessionId: src_id|b2382a70-e9c40464-574f48d8-fb783609_0 is DEAD
2025-10-09T21:48:21.299142Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b2382a70-e9c40464-574f48d8-fb783609_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:21.299152Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b2382a70-e9c40464-574f48d8-fb783609_0] Write session will restart in 0.005706s
2025-10-09T21:48:21.299176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b2382a70-e9c40464-574f48d8-fb783609_0] Write session: Do CDS request
2025-10-09T21:48:21.299182Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b2382a70-e9c40464-574f48d8-fb783609_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:21.299235Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:21.306581Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b2382a70-e9c40464-574f48d8-fb783609_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:21.306653Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:21.306669Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1189
2025-10-09T21:48:21.306806Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1189 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:21.306860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1189 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:21.306871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1189 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:21.306875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:21.307340Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1189 sessionId:  describe result for acl check
2025-10-09T21:48:21.307379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:21.307385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:21.307387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:21.307394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342162483880724:5298] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:21.307950Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342162483880724:5298] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:21.310834Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342162483880724:5298] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:21.313848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342162483880724:5298] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:21.313860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342162483880724:5298] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:21.313865Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342162483880724:5298] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:21.313873Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1189 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:21.318673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:21.321397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:21.321606Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:21.321616Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:21.321628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:21.321679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:21.321722Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342162122058514:9133]
2025-10-09T21:48:21.322350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:21.322699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:21.322741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:21.323050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:21.323093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.323099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.323100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.323101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.323102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.323103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.323105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.323106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.323107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.323112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.323123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.323127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.323131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.323135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.323138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.323142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.323146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.323149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.323156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.323164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.323170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.323175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.323179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.323184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.323188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.323193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.323198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.323202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.323207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.323218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.323224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.323229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.323239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.323244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.323249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.323258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.323284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:21.329832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:21.330874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:21.330890Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:21.330892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:21.330897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:21.330904Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1188 [2:7559342162122058514:9133]
2025-10-09T21:48:21.330917Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:21.330937Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:21.330944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:21.330948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:21.330951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:21.330967Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:21.330977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:21.330989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:21.331046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.331053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.331055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.331058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:21.331295Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1188
2025-10-09T21:48:21.331298Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1188
2025-10-09T21:48:21.331337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:21.331349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.331362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.331373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.331392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:21.331422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342162483880861:5298], now have 1 active actors on pipe
2025-10-09T21:48:21.331430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342162483880703:2432], now have 1 active actors on pipe
2025-10-09T21:48:21.331447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.331456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.331477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342162122058509:9133]
2025-10-09T21:48:21.331484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342162122058509:9133]
2025-10-09T21:48:21.331486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342162122058509:9133]
2025-10-09T21:48:21.331499Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:21.331514Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:21.331515Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:21.331518Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:21.331520Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:21.331534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:21.331537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:21.331557Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a28af870-f3b9c3f9-d9805fd7-3e387551_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:21.331570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:21.331578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:21.331581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:21.331591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:21.331601Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:21.331634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342162122058509:9133]
2025-10-09T21:48:21.331646Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:21.331653Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:21.331656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:21.331827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.331834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.331849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:21.331960Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1189 partition: 0 MaxSeqNo: 25 sessionId: src_id|a28af870-f3b9c3f9-d9805fd7-3e387551_0
2025-10-09T21:48:21.332306Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b2382a70-e9c40464-574f48d8-fb783609_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a28af870-f3b9c3f9-d9805fd7-3e387551_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:21.334796Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a28af870-f3b9c3f9-d9805fd7-3e387551_0] Write session: try to update token
2025-10-09T21:48:21.334816Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a28af870-f3b9c3f9-d9805fd7-3e387551_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:21.343396Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:21.343415Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:21.343426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342162122058509:9133]
2025-10-09T21:48:21.343475Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:21.343475Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:21.343664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:21.343811Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:21.343962Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:21.344098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:21.344288Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:21.344335Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:21.348109Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1189 sessionId: src_id|a28af870-f3b9c3f9-d9805fd7-3e387551_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:21.350676Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:21.350879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.350899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.350939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:21.351041Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:21.357425Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:21.357441Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:21.357444Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:21.357445Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:21.357456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342162122058509:9133]
2025-10-09T21:48:21.357460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342162122058509:9133]
2025-10-09T21:48:21.357463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342162122058509:9133]
2025-10-09T21:48:21.357505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:21.357513Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:21.357516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.357519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:21.357520Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:21.357525Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:21.357648Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:21.399016Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342162483880973:9750]
2025-10-09T21:48:21.399320Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 1188, pipe: [1:7559342162483880973:9750]
2025-10-09T21:48:21.516528Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:21.523973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.524001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.524160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.524215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.524276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.524339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.524388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.524396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.524457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.524516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.524581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.524648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.524700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.524707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.524772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.524821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.524877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.524933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.524972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.524977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.525037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.525092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.525158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.525217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.525255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.525262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.525324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.525386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.525452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.525518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.525565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.525574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.525635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.525693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.525763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.525828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.525874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.525881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.525943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.526006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.526087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.526152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.526193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.526200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.526267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.526324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.526381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.526442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.526484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.526491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.526552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.526612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.526677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.526736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.526778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.526788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.526850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.526915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.526982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.527045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.527088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.527096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.527162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342162483880973:9750], now have 1 active actors on pipe
2025-10-09T21:48:21.527373Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:21.527387Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:21.527418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1189 reason: error at write: Disconnected sessionId: src_id|a28af870-f3b9c3f9-d9805fd7-3e387551_0
2025-10-09T21:48:21.527608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1189 sessionId: src_id|a28af870-f3b9c3f9-d9805fd7-3e387551_0 is DEAD
2025-10-09T21:48:21.527739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a28af870-f3b9c3f9-d9805fd7-3e387551_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:21.527760Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a28af870-f3b9c3f9-d9805fd7-3e387551_0] Write session will restart in 0.007202s
2025-10-09T21:48:21.527785Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a28af870-f3b9c3f9-d9805fd7-3e387551_0] Write session: Do CDS request
2025-10-09T21:48:21.527792Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a28af870-f3b9c3f9-d9805fd7-3e387551_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:21.527801Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342162483880973:9750]
2025-10-09T21:48:21.527914Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:21.528234Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:21.536585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a28af870-f3b9c3f9-d9805fd7-3e387551_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:21.536652Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:21.536668Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1190
2025-10-09T21:48:21.536801Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1190 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:21.536845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1190 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:21.536854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1190 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:21.536858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:21.537346Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1190 sessionId:  describe result for acl check
2025-10-09T21:48:21.537386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:21.537388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:21.537389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:21.537396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342162483881008:5290] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:21.537986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342162483881008:5290] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:21.542284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342162483881008:5290] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:21.545634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342162483881008:5290] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:21.545649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342162483881008:5290] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:21.545653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342162483881008:5290] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:21.545662Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1190 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:21.547381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:21.550399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:21.550618Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:21.550630Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:21.550643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:21.550701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:21.550747Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342162122058778:9135]
2025-10-09T21:48:21.551371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:21.551687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:21.551731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:21.552036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:21.552096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.552103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.552104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.552105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.552107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.552108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.552109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.552110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.552111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.552117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.552127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.552131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.552135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.552138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.552142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.552146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.552150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.552154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.552161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.552175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.552186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.552192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.552201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.552206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.552211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.552216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.552220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.552225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.552234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.552239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.552249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.552254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.552259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.552264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.552269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.552274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.552303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:21.558881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:21.559934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:21.559951Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:21.559953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:21.559958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:21.559964Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1189 [2:7559342162122058778:9135]
2025-10-09T21:48:21.559977Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:21.559997Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:21.560005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:21.560009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:21.560011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:21.560027Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:21.560033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:21.560040Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:21.560098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.560105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.560107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.560109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:21.560179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:21.560189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.560201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.560212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.560229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:21.560257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342162483880976:2432], now have 1 active actors on pipe
2025-10-09T21:48:21.560267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342162483881234:5290], now have 1 active actors on pipe
2025-10-09T21:48:21.560274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342162122058773:9135]
2025-10-09T21:48:21.560277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342162122058773:9135]
2025-10-09T21:48:21.560279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342162122058773:9135]
2025-10-09T21:48:21.560291Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:21.560305Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:21.560306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:21.560308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:21.560310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:21.560320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:21.560323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:21.560352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342162122058773:9135]
2025-10-09T21:48:21.560359Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:21.560382Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:21.560390Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1189
2025-10-09T21:48:21.560500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.560510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.560536Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9ee962d3-c9a3c673-e876aee0-e04ec1e8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:21.560549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:21.560552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:21.560555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:21.560566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:21.560557Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1189
2025-10-09T21:48:21.560576Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:21.560596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:21.560744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.560758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.560781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:21.560859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1190 partition: 0 MaxSeqNo: 25 sessionId: src_id|9ee962d3-c9a3c673-e876aee0-e04ec1e8_0
2025-10-09T21:48:21.561217Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a28af870-f3b9c3f9-d9805fd7-3e387551_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9ee962d3-c9a3c673-e876aee0-e04ec1e8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:21.563785Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ee962d3-c9a3c673-e876aee0-e04ec1e8_0] Write session: try to update token
2025-10-09T21:48:21.563804Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ee962d3-c9a3c673-e876aee0-e04ec1e8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:21.571900Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:21.571919Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:21.571929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342162122058773:9135]
2025-10-09T21:48:21.571977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:21.571977Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:21.572149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:21.572292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:21.572422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:21.572565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:21.572757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:21.572803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:21.576838Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1190 sessionId: src_id|9ee962d3-c9a3c673-e876aee0-e04ec1e8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:21.579378Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:21.579591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.579612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.579657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:21.579756Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:21.585160Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:21.585176Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:21.585178Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:21.585180Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:21.585192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342162122058773:9135]
2025-10-09T21:48:21.585196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342162122058773:9135]
2025-10-09T21:48:21.585199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342162122058773:9135]
2025-10-09T21:48:21.585249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:21.585251Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:21.585258Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:21.585261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.585262Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:21.585265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:21.585398Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:21.627913Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342162483881249:9750]
2025-10-09T21:48:21.628520Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 1189, pipe: [1:7559342162483881249:9750]
2025-10-09T21:48:21.746672Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:21.754737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.754763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.754925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.754988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.755048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.755112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.755156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.755166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.755238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.755305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.755370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.755432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.755474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.755481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.755542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.755600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.755661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.755725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.755765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.755772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.755835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.755895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.755959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.756022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.756070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.756078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.756145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.756207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.756264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.756326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.756370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.756377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.756441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.756506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.756570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.756629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.756672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.756679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.756743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.756805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.756863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.756921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.756966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.756973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.757031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.757086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.757143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.757206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.757251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.757259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.757318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.757375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.757434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.757495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.757539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.757547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.757608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:21.757665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:21.757738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:21.757799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:21.757843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:21.757850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:21.757908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342162483881249:9750], now have 1 active actors on pipe
2025-10-09T21:48:21.758139Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:21.758150Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342162483881249:9750]
2025-10-09T21:48:21.758179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:21.758182Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:21.758228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1190 reason: error at write: Disconnected sessionId: src_id|9ee962d3-c9a3c673-e876aee0-e04ec1e8_0
2025-10-09T21:48:21.758426Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1190 sessionId: src_id|9ee962d3-c9a3c673-e876aee0-e04ec1e8_0 is DEAD
2025-10-09T21:48:21.758576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ee962d3-c9a3c673-e876aee0-e04ec1e8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:21.758588Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ee962d3-c9a3c673-e876aee0-e04ec1e8_0] Write session will restart in 0.009749s
2025-10-09T21:48:21.758617Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ee962d3-c9a3c673-e876aee0-e04ec1e8_0] Write session: Do CDS request
2025-10-09T21:48:21.758625Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ee962d3-c9a3c673-e876aee0-e04ec1e8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:21.758896Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:21.769500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ee962d3-c9a3c673-e876aee0-e04ec1e8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:21.769634Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:21.769651Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1191
2025-10-09T21:48:21.769807Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1191 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:21.769855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1191 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:21.769860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1191 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:21.769865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:21.770289Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1191 sessionId:  describe result for acl check
2025-10-09T21:48:21.770328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:21.770338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:21.770340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:21.770347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342162483881317:5277] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:21.770902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342162483881317:5277] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:21.774363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342162483881317:5277] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:21.777152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342162483881317:5277] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:21.777166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342162483881317:5277] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:21.777186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342162483881317:5277] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:21.777194Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1191 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:21.779343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:21.781802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:21.781994Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:21.782004Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:21.782016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:21.782085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:21.782132Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342162122059047:9139]
2025-10-09T21:48:21.782707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:21.783041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:21.783084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:21.783375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:21.783423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.783424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.783425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.783426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.783427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.783428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.783429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.783430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.783431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.783435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.783446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.783449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.783452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.783455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.783458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.783461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.783464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.783467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.783474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.783481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.783486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.783490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.783494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.783499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.783502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.783506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.783510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.783514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.783518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.783522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.783526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.783530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.783533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.783537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.783541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.783545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.783569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:21.789780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:21.790979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:21.790993Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:21.790994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:21.790998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:21.791004Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1190 [2:7559342162122059047:9139]
2025-10-09T21:48:21.791018Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:21.791034Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:21.791037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:21.791040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:21.791041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:21.791055Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:21.791061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:21.791067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:21.791121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.791130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.791131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.791134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:21.791197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:21.791206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.791216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.791226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.791241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:21.791265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342162483881255:2432], now have 1 active actors on pipe
2025-10-09T21:48:21.791275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342162483881502:5277], now have 1 active actors on pipe
2025-10-09T21:48:21.791282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342162122059042:9139]
2025-10-09T21:48:21.791284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342162122059042:9139]
2025-10-09T21:48:21.791286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342162122059042:9139]
2025-10-09T21:48:21.791300Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:21.791314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:21.791317Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:21.791320Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:21.791322Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:21.791327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:21.791330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:21.791354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342162122059042:9139]
2025-10-09T21:48:21.791361Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:21.791341Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1190
2025-10-09T21:48:21.791368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:21.791349Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1190
2025-10-09T21:48:21.791390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.791399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.791436Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a878f18a-16aad439-80fc8a68-3d45a49f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:21.791448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:21.791451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:21.791453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:21.791463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:21.791473Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:21.791491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:21.791687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.791702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.791725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:21.791824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1191 partition: 0 MaxSeqNo: 25 sessionId: src_id|a878f18a-16aad439-80fc8a68-3d45a49f_0
2025-10-09T21:48:21.792174Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ee962d3-c9a3c673-e876aee0-e04ec1e8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a878f18a-16aad439-80fc8a68-3d45a49f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:21.794582Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a878f18a-16aad439-80fc8a68-3d45a49f_0] Write session: try to update token
2025-10-09T21:48:21.794603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a878f18a-16aad439-80fc8a68-3d45a49f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:21.805589Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:21.805607Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:21.805609Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:21.805611Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:21.805621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342162122059042:9139]
2025-10-09T21:48:21.805626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342162122059042:9139]
2025-10-09T21:48:21.805631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342162122059042:9139]
2025-10-09T21:48:21.805676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:21.805679Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:21.805685Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:21.805687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.805690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:21.805692Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:21.805841Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:21.808060Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1191 sessionId: src_id|a878f18a-16aad439-80fc8a68-3d45a49f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:21.810381Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:21.858261Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342162483881532:9750]
2025-10-09T21:48:21.858581Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 1190, pipe: [1:7559342162483881532:9750]
2025-10-09T21:48:21.965817Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:21.966483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.966504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.966550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342162483881532:9750], now have 1 active actors on pipe
2025-10-09T21:48:21.966755Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:21.966779Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342162483881532:9750]
2025-10-09T21:48:21.966795Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:21.966799Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:21.966809Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:21.966860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1191 reason: error at write: Disconnected sessionId: src_id|a878f18a-16aad439-80fc8a68-3d45a49f_0
2025-10-09T21:48:21.967033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1191 sessionId: src_id|a878f18a-16aad439-80fc8a68-3d45a49f_0 is DEAD
2025-10-09T21:48:21.967177Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a878f18a-16aad439-80fc8a68-3d45a49f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:21.967185Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a878f18a-16aad439-80fc8a68-3d45a49f_0] Write session will restart in 0.006737s
2025-10-09T21:48:21.967211Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a878f18a-16aad439-80fc8a68-3d45a49f_0] Write session: Do CDS request
2025-10-09T21:48:21.967218Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a878f18a-16aad439-80fc8a68-3d45a49f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:21.975538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a878f18a-16aad439-80fc8a68-3d45a49f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:21.975619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:21.975632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1192
2025-10-09T21:48:21.975781Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1192 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:21.975810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1192 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:21.975823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1192 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:21.975826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:21.976244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1192 sessionId:  describe result for acl check
2025-10-09T21:48:21.976280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:21.976281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:21.976284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:21.976289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342162483881561:5275] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:21.976830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342162483881561:5275] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:21.979972Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342162483881561:5275] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:21.983276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342162483881561:5275] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:21.983292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342162483881561:5275] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:21.983296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342162483881561:5275] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:21.983305Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1192 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:21.986457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:21.989196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:21.989414Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:21.989424Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:21.989437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:21.989490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:21.989531Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342162122059314:9141]
2025-10-09T21:48:21.990134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:21.990466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:21.990512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:21.990841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:21.990896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.990902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.990904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.990905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.990906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.990907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.990908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.990909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.990911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.990915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.990926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.990930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.990934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.990938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.990942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.990946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.990950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.990953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.990960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.990968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:21.990974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.990978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:21.990983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.990988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:21.990992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.990998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:21.991002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.991008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:21.991012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.991017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:21.991022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.991028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.991032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.991042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.991047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.991052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.991083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:21.997619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:21.998737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:21.998753Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:21.998755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:21.998760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:21.998766Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1191 [2:7559342162122059314:9141]
2025-10-09T21:48:21.998792Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:21.998814Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:21.998822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:21.998825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:21.998828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:21.998844Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:21.998851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:21.998859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:21.998920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.998930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.998932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.998935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:21.999014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:21.999022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:21.999035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:21.999045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:21.999067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:21.999104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342162122059309:9141]
2025-10-09T21:48:21.999112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342162122059309:9141]
2025-10-09T21:48:21.999114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342162122059309:9141]
2025-10-09T21:48:21.999128Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:21.999144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:21.999145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342162483881742:5275], now have 1 active actors on pipe
2025-10-09T21:48:21.999147Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:21.999149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342162483881537:2432], now have 1 active actors on pipe
2025-10-09T21:48:21.999149Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:21.999158Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:21.999173Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:21.999176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:21.999203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342162122059309:9141]
2025-10-09T21:48:21.999211Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:21.999207Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1191
2025-10-09T21:48:21.999232Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:21.999223Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1191
2025-10-09T21:48:21.999296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.999319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.999360Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2bd30e2b-40186ceb-db0d21eb-60047a54_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:21.999375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:21.999392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:21.999399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:21.999413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:21.999428Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:21.999450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:21.999606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:21.999631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:21.999654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:21.999776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1192 partition: 0 MaxSeqNo: 25 sessionId: src_id|2bd30e2b-40186ceb-db0d21eb-60047a54_0
2025-10-09T21:48:22.000138Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a878f18a-16aad439-80fc8a68-3d45a49f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2bd30e2b-40186ceb-db0d21eb-60047a54_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:22.002723Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2bd30e2b-40186ceb-db0d21eb-60047a54_0] Write session: try to update token
2025-10-09T21:48:22.002741Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2bd30e2b-40186ceb-db0d21eb-60047a54_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:22.013692Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:22.013712Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:22.013715Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:22.013717Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:22.013740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342162122059309:9141]
2025-10-09T21:48:22.013745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342162122059309:9141]
2025-10-09T21:48:22.013750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342162122059309:9141]
2025-10-09T21:48:22.013799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:22.013801Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:22.013808Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:22.013810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.013811Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:22.013814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:22.013952Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:22.016156Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1192 sessionId: src_id|2bd30e2b-40186ceb-db0d21eb-60047a54_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:22.018619Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:22.067165Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342166778849106:9750]
2025-10-09T21:48:22.067495Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 1191, pipe: [1:7559342166778849106:9750]
2025-10-09T21:48:22.175603Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:22.176281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:22.176296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:22.176338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342166778849106:9750], now have 1 active actors on pipe
2025-10-09T21:48:22.176535Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342166778849106:9750]
2025-10-09T21:48:22.176537Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:22.176541Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:22.176562Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:22.176596Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1192 reason: error at write: Disconnected sessionId: src_id|2bd30e2b-40186ceb-db0d21eb-60047a54_0
2025-10-09T21:48:22.176778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1192 sessionId: src_id|2bd30e2b-40186ceb-db0d21eb-60047a54_0 is DEAD
2025-10-09T21:48:22.176955Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2bd30e2b-40186ceb-db0d21eb-60047a54_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:22.176963Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2bd30e2b-40186ceb-db0d21eb-60047a54_0] Write session will restart in 0.005830s
2025-10-09T21:48:22.177000Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2bd30e2b-40186ceb-db0d21eb-60047a54_0] Write session: Do CDS request
2025-10-09T21:48:22.177010Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2bd30e2b-40186ceb-db0d21eb-60047a54_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:22.177249Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:22.183481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2bd30e2b-40186ceb-db0d21eb-60047a54_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:22.183526Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:22.183538Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1193
2025-10-09T21:48:22.183666Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1193 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:22.183701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1193 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:22.183706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1193 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:22.183709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:22.184147Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1193 sessionId:  describe result for acl check
2025-10-09T21:48:22.184190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:22.184197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:22.184199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:22.184204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342166778849132:5269] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:22.184733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342166778849132:5269] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:22.187176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342166778849132:5269] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:22.190679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342166778849132:5269] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:22.190692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342166778849132:5269] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:22.190696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342166778849132:5269] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:22.190702Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1193 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:22.195620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:22.198053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:22.198229Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:22.198240Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:22.198251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:22.198298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:22.198335Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342166417026907:9153]
2025-10-09T21:48:22.198924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:22.199271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:22.199325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:22.199646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:22.199696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:22.199701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:22.199703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:22.199704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:22.199705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:22.199706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:22.199707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.199708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.199709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.199714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:22.199725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:22.199728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:22.199733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:22.199736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:22.199740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:22.199744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.199747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.199750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.199756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:22.199764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:22.199769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:22.199774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:22.199777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:22.199782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:22.199786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:22.199792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:22.199796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:22.199801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:22.199805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:22.199810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:22.199821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.199826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.199830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.199834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.199839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.199843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.199869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:22.206283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:22.207483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:22.207500Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:22.207502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:22.207507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:22.207514Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1192 [2:7559342166417026907:9153]
2025-10-09T21:48:22.207527Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:22.207544Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:22.207547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:22.207550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:22.207552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:22.207569Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:22.207575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:22.207582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:22.207643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.207651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.207653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.207656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:22.207721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:22.207730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.207742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.207753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.207769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:22.207790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342166778849277:5269], now have 1 active actors on pipe
2025-10-09T21:48:22.207799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342166778849115:2432], now have 1 active actors on pipe
2025-10-09T21:48:22.207806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342166417026902:9153]
2025-10-09T21:48:22.207809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342166417026902:9153]
2025-10-09T21:48:22.207811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342166417026902:9153]
2025-10-09T21:48:22.207824Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:22.207837Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:22.207843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:22.207845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:22.207846Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:22.207859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:22.207868Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:22.207836Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1192
2025-10-09T21:48:22.207840Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1192
2025-10-09T21:48:22.207900Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342166417026902:9153]
2025-10-09T21:48:22.207917Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:22.207926Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:22.207935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:22.207940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:22.207996Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|68a91829-7e7018b6-a42a9c20-c2d37a2e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:22.208010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:22.208018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:22.208021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:22.208032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:22.208046Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:22.208074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:22.208202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:22.208210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:22.208225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:22.208304Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1193 partition: 0 MaxSeqNo: 25 sessionId: src_id|68a91829-7e7018b6-a42a9c20-c2d37a2e_0
2025-10-09T21:48:22.208674Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2bd30e2b-40186ceb-db0d21eb-60047a54_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|68a91829-7e7018b6-a42a9c20-c2d37a2e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:22.211240Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|68a91829-7e7018b6-a42a9c20-c2d37a2e_0] Write session: try to update token
2025-10-09T21:48:22.211260Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|68a91829-7e7018b6-a42a9c20-c2d37a2e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:22.222215Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:22.222233Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:22.222235Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:22.222237Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:22.222247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342166417026902:9153]
2025-10-09T21:48:22.222251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342166417026902:9153]
2025-10-09T21:48:22.222255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342166417026902:9153]
2025-10-09T21:48:22.222300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:22.222306Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:22.222311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.222312Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:22.222314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:22.222319Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:22.222534Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:22.224477Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1193 sessionId: src_id|68a91829-7e7018b6-a42a9c20-c2d37a2e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:22.226809Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:22.276607Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342166778849406:9750]
2025-10-09T21:48:22.276871Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 1192, pipe: [1:7559342166778849406:9750]
2025-10-09T21:48:22.380128Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:22.380403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:22.380422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:22.380472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342166778849406:9750], now have 1 active actors on pipe
2025-10-09T21:48:22.380699Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:22.380719Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342166778849406:9750]
2025-10-09T21:48:22.380721Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:22.380725Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:22.380750Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:22.380872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1193 reason: error at write: Disconnected sessionId: src_id|68a91829-7e7018b6-a42a9c20-c2d37a2e_0
2025-10-09T21:48:22.381065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1193 sessionId: src_id|68a91829-7e7018b6-a42a9c20-c2d37a2e_0 is DEAD
2025-10-09T21:48:22.381223Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|68a91829-7e7018b6-a42a9c20-c2d37a2e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:22.381233Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|68a91829-7e7018b6-a42a9c20-c2d37a2e_0] Write session will restart in 0.006890s
2025-10-09T21:48:22.381259Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|68a91829-7e7018b6-a42a9c20-c2d37a2e_0] Write session: Do CDS request
2025-10-09T21:48:22.381265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|68a91829-7e7018b6-a42a9c20-c2d37a2e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:22.389540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|68a91829-7e7018b6-a42a9c20-c2d37a2e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:22.389621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:22.389635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1194
2025-10-09T21:48:22.390205Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1194 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:22.390240Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1194 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:22.390246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1194 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:22.390249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:22.390656Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1194 sessionId:  describe result for acl check
2025-10-09T21:48:22.390687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:22.390692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:22.390693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:22.390697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342166778849453:5257] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:22.391179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342166778849453:5257] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:22.393814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342166778849453:5257] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:22.397010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342166778849453:5257] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:22.397024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342166778849453:5257] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:22.397027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342166778849453:5257] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:22.397036Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1194 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:22.400607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:22.403235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:22.403423Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:22.403433Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:22.403445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:22.403508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:22.403550Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342166417027183:9157]
2025-10-09T21:48:22.404104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:22.404427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:22.404482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:22.404803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:22.404859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:22.404864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:22.404866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:22.404867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:22.404868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:22.404869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:22.404870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.404871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.404872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.404877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:22.404887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:22.404891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:22.404895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:22.404899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:22.404903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:22.404906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.404910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.404913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.404920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:22.404927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:22.404938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:22.404942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:22.404946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:22.404950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:22.404954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:22.404962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:22.404968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:22.404977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:22.404981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:22.404986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:22.404994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.404999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.405004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.405008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.405013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.405017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.405044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:22.411174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:22.412198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:22.412213Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:22.412215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:22.412218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:22.412223Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1193 [2:7559342166417027183:9157]
2025-10-09T21:48:22.412232Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:22.412246Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:22.412247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:22.412250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:22.412251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:22.412266Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:22.412272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:22.412280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:22.412334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.412339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.412341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.412343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:22.412405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:22.412413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.412423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.412432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.412453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:22.412478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342166778849420:2432], now have 1 active actors on pipe
2025-10-09T21:48:22.412486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342166778849621:5257], now have 1 active actors on pipe
2025-10-09T21:48:22.412494Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342166417027178:9157]
2025-10-09T21:48:22.412496Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342166417027178:9157]
2025-10-09T21:48:22.412498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342166417027178:9157]
2025-10-09T21:48:22.412511Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:22.412524Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:22.412525Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:22.412527Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:22.412530Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:22.412534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:22.412537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:22.412543Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1193
2025-10-09T21:48:22.412564Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1193
2025-10-09T21:48:22.412563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342166417027178:9157]
2025-10-09T21:48:22.412571Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:22.412583Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:22.412599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:22.412605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:22.412640Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|91b8b598-33118511-c978719f-5b8baacc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:22.412652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:22.412656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:22.412658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:22.412668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:22.412682Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:22.412698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:22.412833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:22.412840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:22.412854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:22.412963Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1194 partition: 0 MaxSeqNo: 25 sessionId: src_id|91b8b598-33118511-c978719f-5b8baacc_0
2025-10-09T21:48:22.413325Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|68a91829-7e7018b6-a42a9c20-c2d37a2e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|91b8b598-33118511-c978719f-5b8baacc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:22.415845Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91b8b598-33118511-c978719f-5b8baacc_0] Write session: try to update token
2025-10-09T21:48:22.415866Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91b8b598-33118511-c978719f-5b8baacc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:22.426951Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:22.426970Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:22.426973Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:22.426975Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:22.426988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342166417027178:9157]
2025-10-09T21:48:22.426993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342166417027178:9157]
2025-10-09T21:48:22.426997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342166417027178:9157]
2025-10-09T21:48:22.427043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:22.427050Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:22.427056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.427056Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:22.427058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:22.427060Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:22.427221Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:22.431323Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1194 sessionId: src_id|91b8b598-33118511-c978719f-5b8baacc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:22.433546Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:22.480908Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342166778849695:9750]
2025-10-09T21:48:22.481171Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 1193, pipe: [1:7559342166778849695:9750]
2025-10-09T21:48:22.579087Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:22.579315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:22.579330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:22.579374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342166778849695:9750], now have 1 active actors on pipe
2025-10-09T21:48:22.579580Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:22.579658Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:22.579659Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:22.579673Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:22.579702Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342166778849695:9750]
2025-10-09T21:48:22.579734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1194 reason: error at write: Disconnected sessionId: src_id|91b8b598-33118511-c978719f-5b8baacc_0
2025-10-09T21:48:22.579900Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1194 sessionId: src_id|91b8b598-33118511-c978719f-5b8baacc_0 is DEAD
2025-10-09T21:48:22.580040Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91b8b598-33118511-c978719f-5b8baacc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:22.580050Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91b8b598-33118511-c978719f-5b8baacc_0] Write session will restart in 0.006270s
2025-10-09T21:48:22.580076Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91b8b598-33118511-c978719f-5b8baacc_0] Write session: Do CDS request
2025-10-09T21:48:22.580082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91b8b598-33118511-c978719f-5b8baacc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:22.587546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91b8b598-33118511-c978719f-5b8baacc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:22.587617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:22.587637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1195
2025-10-09T21:48:22.587774Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1195 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:22.587817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1195 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:22.587839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1195 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:22.587854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:22.588356Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1195 sessionId:  describe result for acl check
2025-10-09T21:48:22.588397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:22.588399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:22.588402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:22.588408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342166778849713:5248] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:22.589030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342166778849713:5248] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:22.592436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342166778849713:5248] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:22.595368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342166778849713:5248] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:22.595382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342166778849713:5248] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:22.595385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342166778849713:5248] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:22.595393Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1195 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:22.598982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:22.601357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:22.601567Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:22.601580Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:22.601592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:22.601654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:22.601695Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342166417027452:9159]
2025-10-09T21:48:22.602300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:22.602628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:22.602687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:22.603030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:22.603089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:22.603095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:22.603095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:22.603096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:22.603097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:22.603097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:22.603098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.603099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.603100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.603113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:22.603123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:22.603128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:22.603132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:22.603135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:22.603140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:22.603143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.603147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.603151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.603159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:22.603167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:22.603173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:22.603178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:22.603182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:22.603186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:22.603190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:22.603194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:22.603198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:22.603209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:22.603213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:22.603218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:22.603223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.603232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.603237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.603243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.603247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.603257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.603284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:22.609464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:22.610939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:22.610954Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:22.610956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:22.610960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:22.610967Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1194 [2:7559342166417027452:9159]
2025-10-09T21:48:22.610981Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:22.611000Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:22.611003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:22.611006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:22.611009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:22.611025Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:22.611031Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:22.611038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:22.611097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.611105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.611107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.611110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:22.611179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:22.611187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.611199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.611210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.611225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:22.611247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342166778849884:5248], now have 1 active actors on pipe
2025-10-09T21:48:22.611255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342166778849697:2432], now have 1 active actors on pipe
2025-10-09T21:48:22.611263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342166417027447:9159]
2025-10-09T21:48:22.611265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342166417027447:9159]
2025-10-09T21:48:22.611268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342166417027447:9159]
2025-10-09T21:48:22.611282Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:22.611301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:22.611303Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:22.611307Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:22.611312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:22.611319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:22.611321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:22.611336Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1194
2025-10-09T21:48:22.611353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342166417027447:9159]
2025-10-09T21:48:22.611340Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1194
2025-10-09T21:48:22.611367Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:22.611373Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:22.611406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:22.611411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:22.611446Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|10642eab-8e8df3c9-79dfc206-7888f127_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:22.611457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:22.611461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:22.611463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:22.611476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:22.611490Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:22.611509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:22.611660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:22.611691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:22.611711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:22.611822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1195 partition: 0 MaxSeqNo: 25 sessionId: src_id|10642eab-8e8df3c9-79dfc206-7888f127_0
2025-10-09T21:48:22.612237Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91b8b598-33118511-c978719f-5b8baacc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|10642eab-8e8df3c9-79dfc206-7888f127_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:22.614781Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10642eab-8e8df3c9-79dfc206-7888f127_0] Write session: try to update token
2025-10-09T21:48:22.614799Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10642eab-8e8df3c9-79dfc206-7888f127_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:22.622683Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:22.622701Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:22.622711Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342166417027447:9159]
2025-10-09T21:48:22.622759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:22.622763Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:22.622929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:22.623070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:22.623203Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:22.623340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:22.623534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:22.623578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:22.628148Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1195 sessionId: src_id|10642eab-8e8df3c9-79dfc206-7888f127_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:22.630487Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:22.630682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:22.630702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:22.630743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:22.630832Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:22.635178Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:22.635195Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:22.635197Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:22.635199Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:22.635210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342166417027447:9159]
2025-10-09T21:48:22.635213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342166417027447:9159]
2025-10-09T21:48:22.635217Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342166417027447:9159]
2025-10-09T21:48:22.635257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:22.635259Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:22.635269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.635272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:22.635274Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:22.635278Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:22.635402Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:22.680079Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342166778849976:9750]
2025-10-09T21:48:22.680409Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 1194, pipe: [1:7559342166778849976:9750]
2025-10-09T21:48:22.795191Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:22.810098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:22.810130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:22.810459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:22.810525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:22.810592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:22.810650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:22.810701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:22.810716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:22.810783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:22.810842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:22.810903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:22.810967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:22.811012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:22.811021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:22.811078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:22.811132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:22.811387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:22.811451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:22.811499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:22.811506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:22.811576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:22.811655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:22.811722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:22.811784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:22.811827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:22.811835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:22.811900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:22.811960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:22.812184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:22.812392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:22.812774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:22.812783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:22.812842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:22.813559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:22.814192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:22.814256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:22.814374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:22.814382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:22.814444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:22.814500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:22.814566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:22.814624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:22.814664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:22.814671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:22.814727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:22.814783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:22.814843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:22.815090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:22.815139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:22.815147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:22.815389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:22.815639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:22.815866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:22.816091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:22.816133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:22.816139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:22.816332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:22.816392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:22.816447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:22.816585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:22.816631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:22.816639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:22.816705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342166778849976:9750], now have 1 active actors on pipe
2025-10-09T21:48:22.816929Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:22.816938Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:22.816942Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:22.816977Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1195 reason: error at write: Disconnected sessionId: src_id|10642eab-8e8df3c9-79dfc206-7888f127_0
2025-10-09T21:48:22.816992Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342166778849976:9750]
2025-10-09T21:48:22.817184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1195 sessionId: src_id|10642eab-8e8df3c9-79dfc206-7888f127_0 is DEAD
2025-10-09T21:48:22.817324Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10642eab-8e8df3c9-79dfc206-7888f127_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:22.817336Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10642eab-8e8df3c9-79dfc206-7888f127_0] Write session will restart in 0.005648s
2025-10-09T21:48:22.817368Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10642eab-8e8df3c9-79dfc206-7888f127_0] Write session: Do CDS request
2025-10-09T21:48:22.817374Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10642eab-8e8df3c9-79dfc206-7888f127_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:22.817859Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:22.824536Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10642eab-8e8df3c9-79dfc206-7888f127_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:22.824604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:22.824619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1196
2025-10-09T21:48:22.824762Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1196 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:22.824798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1196 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:22.824808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1196 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:22.824812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:22.825235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1196 sessionId:  describe result for acl check
2025-10-09T21:48:22.825286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:22.825292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:22.825294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:22.825300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342166778849995:5240] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:22.825863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342166778849995:5240] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:22.828455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342166778849995:5240] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:22.833108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342166778849995:5240] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:22.833121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342166778849995:5240] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:22.833125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342166778849995:5240] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:22.833134Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1196 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:22.838641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:22.843865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:22.844081Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:22.844087Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:22.844096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:22.844153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:22.844221Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342166417027727:9163]
2025-10-09T21:48:22.844670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:22.844984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:22.845018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:22.845227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:22.845258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:22.845263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:22.845263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:22.845264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:22.845265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:22.845265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:22.845266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.845266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.845267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.845271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:22.845277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:22.845279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:22.845281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:22.845283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:22.845286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:22.845288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.845290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.845292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.845297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:22.845302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:22.845306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:22.845309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:22.845313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:22.845316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:22.845318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:22.845321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:22.845324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:22.845326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:22.845329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:22.845332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:22.845334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.845337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.845340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.845342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.845345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.845348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.845365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:22.851697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:22.852878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:22.852897Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:22.852899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:22.852903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:22.852910Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1195 [2:7559342166417027727:9163]
2025-10-09T21:48:22.852925Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:22.852943Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:22.852946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:22.852949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:22.852951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:22.852967Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:22.852978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:22.852985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:22.853040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.853047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.853049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.853052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:22.853286Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1195
2025-10-09T21:48:22.853292Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1195
2025-10-09T21:48:22.853401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:22.853412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.853425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:22.853438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:22.853459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:22.853485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342166778849980:2432], now have 1 active actors on pipe
2025-10-09T21:48:22.853494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342166778850142:5240], now have 1 active actors on pipe
2025-10-09T21:48:22.853511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:22.853520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:22.853542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342166417027722:9163]
2025-10-09T21:48:22.853548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342166417027722:9163]
2025-10-09T21:48:22.853550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342166417027722:9163]
2025-10-09T21:48:22.853563Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:22.853577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:22.853578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:22.853582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:22.853584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:22.853596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:22.853599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:22.853615Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b7a75f83-74647a49-6c7a7d5f-6f07b64b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:22.853625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:22.853628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:22.853630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:22.853637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:22.853645Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:22.853687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342166417027722:9163]
2025-10-09T21:48:22.853697Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:22.853706Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:22.853706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:22.853866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:22.853874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:22.853886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:22.853963Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1196 partition: 0 MaxSeqNo: 25 sessionId: src_id|b7a75f83-74647a49-6c7a7d5f-6f07b64b_0
2025-10-09T21:48:22.854322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10642eab-8e8df3c9-79dfc206-7888f127_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b7a75f83-74647a49-6c7a7d5f-6f07b64b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:22.856853Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7a75f83-74647a49-6c7a7d5f-6f07b64b_0] Write session: try to update token
2025-10-09T21:48:22.856869Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7a75f83-74647a49-6c7a7d5f-6f07b64b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:22.868069Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:22.868089Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:22.868100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342166417027722:9163]
2025-10-09T21:48:22.868143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:22.868135Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:22.868319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:22.868459Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:22.868595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:22.868734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:22.868940Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:22.868993Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:22.877302Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1196 sessionId: src_id|b7a75f83-74647a49-6c7a7d5f-6f07b64b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:22.878588Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:22.878605Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:22.878607Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:22.878610Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:22.878619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342166417027722:9163]
2025-10-09T21:48:22.878624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342166417027722:9163]
2025-10-09T21:48:22.878628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342166417027722:9163]
2025-10-09T21:48:22.878670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:22.878681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:22.878685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:22.878803Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:22.878817Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:22.878821Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:22.878826Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:22.879668Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:22.917094Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342166778850260:9750]
2025-10-09T21:48:22.917325Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 1195, pipe: [1:7559342166778850260:9750]
2025-10-09T21:48:23.037842Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:23.038130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:23.038147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:23.038172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342166778850260:9750], now have 1 active actors on pipe
2025-10-09T21:48:23.038374Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:23.038421Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:23.038423Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342166778850260:9750]
2025-10-09T21:48:23.038442Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:23.038516Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1196 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b7a75f83-74647a49-6c7a7d5f-6f07b64b_0
2025-10-09T21:48:23.038707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1196 sessionId: src_id|b7a75f83-74647a49-6c7a7d5f-6f07b64b_0 is DEAD
2025-10-09T21:48:23.038837Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7a75f83-74647a49-6c7a7d5f-6f07b64b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:23.038848Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7a75f83-74647a49-6c7a7d5f-6f07b64b_0] Write session will restart in 0.005035s
2025-10-09T21:48:23.038884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7a75f83-74647a49-6c7a7d5f-6f07b64b_0] Write session: Do CDS request
2025-10-09T21:48:23.038891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7a75f83-74647a49-6c7a7d5f-6f07b64b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:23.045494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7a75f83-74647a49-6c7a7d5f-6f07b64b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:23.045610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:23.045624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1197
2025-10-09T21:48:23.045790Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1197 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:23.045832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1197 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:23.045841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1197 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:23.045846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:23.046313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1197 sessionId:  describe result for acl check
2025-10-09T21:48:23.046347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:23.046353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:23.046354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:23.046364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342171073817576:5235] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:23.046916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342171073817576:5235] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:23.049310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342171073817576:5235] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:23.053213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342171073817576:5235] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:23.053229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342171073817576:5235] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:23.053233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342171073817576:5235] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:23.053242Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1197 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:23.059514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:23.062395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:23.062598Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:23.062613Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:23.062626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:23.062687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:23.062732Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342170711995294:9165]
2025-10-09T21:48:23.063330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:23.063600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:23.063642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:23.063957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:23.064003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.064009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.064010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.064011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.064013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.064014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.064015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.064016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.064017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.064022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.064034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.064039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.064043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.064047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.064051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.064055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.064059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.064063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.064071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.064080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.064089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.064094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.064099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.064104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.064108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.064113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.064122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.064126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.064130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.064134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.064140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.064150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.064155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.064160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.064164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.064174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.064203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:23.070902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:23.072010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:23.072026Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:23.072029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:23.072034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:23.072041Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1196 [2:7559342170711995294:9165]
2025-10-09T21:48:23.072056Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:23.072074Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:23.072081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:23.072085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:23.072087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:23.072104Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:23.072116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:23.072124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:23.072204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.072209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.072211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.072213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:23.072287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:23.072295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.072307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.072319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.072335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:23.072363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342171073817561:2432], now have 1 active actors on pipe
2025-10-09T21:48:23.072375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342171073817694:5235], now have 1 active actors on pipe
2025-10-09T21:48:23.072383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342170711995289:9165]
2025-10-09T21:48:23.072385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342170711995289:9165]
2025-10-09T21:48:23.072387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342170711995289:9165]
2025-10-09T21:48:23.072400Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:23.072415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:23.072415Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:23.072419Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:23.072422Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:23.072433Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:23.072436Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:23.072435Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1196
2025-10-09T21:48:23.072469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342170711995289:9165]
2025-10-09T21:48:23.072441Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1196
2025-10-09T21:48:23.072477Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:23.072480Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:23.072499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:23.072504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:23.072552Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ffb1bd0d-65527c6-2458d0d0-8e5ad525_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:23.072565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:23.072569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:23.072572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:23.072584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:23.072598Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:23.072619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:23.072774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:23.072785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:23.072797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:23.072878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1197 partition: 0 MaxSeqNo: 25 sessionId: src_id|ffb1bd0d-65527c6-2458d0d0-8e5ad525_0
2025-10-09T21:48:23.073242Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7a75f83-74647a49-6c7a7d5f-6f07b64b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ffb1bd0d-65527c6-2458d0d0-8e5ad525_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:23.075654Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ffb1bd0d-65527c6-2458d0d0-8e5ad525_0] Write session: try to update token
2025-10-09T21:48:23.075672Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ffb1bd0d-65527c6-2458d0d0-8e5ad525_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:23.086939Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:23.086975Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:23.086978Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:23.086980Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:23.086990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342170711995289:9165]
2025-10-09T21:48:23.086995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342170711995289:9165]
2025-10-09T21:48:23.086999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342170711995289:9165]
2025-10-09T21:48:23.087043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:23.087043Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:23.087053Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:23.087055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.087056Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:23.087058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:23.087195Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:23.090669Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1197 sessionId: src_id|ffb1bd0d-65527c6-2458d0d0-8e5ad525_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:23.093021Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:23.138815Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342171073817842:9750]
2025-10-09T21:48:23.139047Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 1196, pipe: [1:7559342171073817842:9750]
2025-10-09T21:48:23.247424Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:23.248298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:23.248313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:23.248363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342171073817842:9750], now have 1 active actors on pipe
2025-10-09T21:48:23.248576Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:23.248592Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:23.248609Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342171073817842:9750]
2025-10-09T21:48:23.248637Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:23.248639Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:23.248707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1197 reason: error at write: Disconnected sessionId: src_id|ffb1bd0d-65527c6-2458d0d0-8e5ad525_0
2025-10-09T21:48:23.248917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1197 sessionId: src_id|ffb1bd0d-65527c6-2458d0d0-8e5ad525_0 is DEAD
2025-10-09T21:48:23.249049Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffb1bd0d-65527c6-2458d0d0-8e5ad525_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:23.249061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffb1bd0d-65527c6-2458d0d0-8e5ad525_0] Write session will restart in 0.009569s
2025-10-09T21:48:23.249089Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffb1bd0d-65527c6-2458d0d0-8e5ad525_0] Write session: Do CDS request
2025-10-09T21:48:23.249096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffb1bd0d-65527c6-2458d0d0-8e5ad525_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:23.259620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ffb1bd0d-65527c6-2458d0d0-8e5ad525_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:23.259676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:23.259693Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1198
2025-10-09T21:48:23.259865Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1198 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:23.259913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1198 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:23.259925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1198 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:23.259929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:23.260503Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1198 sessionId:  describe result for acl check
2025-10-09T21:48:23.260543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:23.260545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:23.260547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:23.260554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342171073817917:5224] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:23.261120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342171073817917:5224] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:23.263853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342171073817917:5224] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:23.267418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342171073817917:5224] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:23.267436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342171073817917:5224] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:23.267440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342171073817917:5224] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:23.267450Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1198 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:23.270785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:23.273934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:23.274189Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:23.274202Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:23.274216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:23.274282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:23.274328Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342170711995596:9177]
2025-10-09T21:48:23.274983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:23.275378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:23.275439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:23.275799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:23.275855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.275861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.275862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.275864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.275865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.275867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.275868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.275869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.275870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.275876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.275888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.275893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.275898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.275901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.275905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.275909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.275913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.275916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.275924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.275933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.275938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.275944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.275950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.275955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.275961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.275967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.275976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.275982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.275991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.275997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.276002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.276008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.276018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.276024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.276030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.276036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.276083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:23.283176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:23.284445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:23.284463Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:23.284466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:23.284470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:23.284478Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1197 [2:7559342170711995596:9177]
2025-10-09T21:48:23.284494Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:23.284516Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:23.284519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:23.284522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:23.284525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:23.284545Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:23.284557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:23.284565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:23.284633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.284641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.284643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.284646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:23.284725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:23.284733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.284746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.284759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.284778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:23.284799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342171073818078:5224], now have 1 active actors on pipe
2025-10-09T21:48:23.284807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342171073817852:2432], now have 1 active actors on pipe
2025-10-09T21:48:23.284819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342170711995591:9177]
2025-10-09T21:48:23.284825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342170711995591:9177]
2025-10-09T21:48:23.284827Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342170711995591:9177]
2025-10-09T21:48:23.284841Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:23.284855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:23.284859Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:23.284861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:23.284861Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:23.284877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:23.284884Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:23.284924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342170711995591:9177]
2025-10-09T21:48:23.284932Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:23.284916Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1197
2025-10-09T21:48:23.284961Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:23.284975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:23.284918Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1197
2025-10-09T21:48:23.284980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:23.285019Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9ca71963-8f335646-dcb625d-8f951279_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:23.285032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:23.285036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:23.285038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:23.285049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:23.285060Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:23.285081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:23.285238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:23.285245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:23.285260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:23.285380Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1198 partition: 0 MaxSeqNo: 25 sessionId: src_id|9ca71963-8f335646-dcb625d-8f951279_0
2025-10-09T21:48:23.285804Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffb1bd0d-65527c6-2458d0d0-8e5ad525_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9ca71963-8f335646-dcb625d-8f951279_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:23.288397Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ca71963-8f335646-dcb625d-8f951279_0] Write session: try to update token
2025-10-09T21:48:23.288420Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ca71963-8f335646-dcb625d-8f951279_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:23.302013Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1198 sessionId: src_id|9ca71963-8f335646-dcb625d-8f951279_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:23.302548Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:23.302568Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:23.302578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342170711995591:9177]
2025-10-09T21:48:23.302627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:23.302627Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:23.302857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:23.303007Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:23.303146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:23.303297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:23.303492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:23.303539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:23.304425Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:23.304619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:23.304634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:23.304669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:23.304738Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:23.310303Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:23.310315Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:23.310318Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:23.310320Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:23.310330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342170711995591:9177]
2025-10-09T21:48:23.310333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342170711995591:9177]
2025-10-09T21:48:23.310336Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342170711995591:9177]
2025-10-09T21:48:23.310373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:23.310378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.310380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:23.310380Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:23.310386Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:23.310390Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:23.310502Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:23.348714Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342171073818156:9750]
2025-10-09T21:48:23.348924Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 1197, pipe: [1:7559342171073818156:9750]
2025-10-09T21:48:23.468444Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:23.476629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:23.476656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:23.476813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:23.476869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:23.476928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:23.476992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:23.477037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:23.477045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:23.477107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:23.477164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:23.477234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:23.477302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:23.477345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:23.477354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:23.477421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:23.477484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:23.477548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:23.477611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:23.477652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:23.477658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:23.477735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:23.477803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:23.477869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:23.477929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:23.477975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:23.477979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:23.478059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:23.478126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:23.478192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:23.478255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:23.478296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:23.478306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:23.478372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:23.478434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:23.478498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:23.478558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:23.478605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:23.478613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:23.478676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:23.478739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:23.478802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:23.478864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:23.478903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:23.478910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:23.478968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:23.479025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:23.479087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:23.479153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:23.479199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:23.479206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:23.479306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:23.479368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:23.479453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:23.479585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:23.479639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:23.479647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:23.479746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:23.479822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:23.479886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:23.479944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:23.480003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:23.480012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:23.480073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342171073818156:9750], now have 1 active actors on pipe
2025-10-09T21:48:23.480297Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:23.480309Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:23.480313Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:23.480345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1198 reason: error at write: Disconnected sessionId: src_id|9ca71963-8f335646-dcb625d-8f951279_0
2025-10-09T21:48:23.480367Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342171073818156:9750]
2025-10-09T21:48:23.480549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1198 sessionId: src_id|9ca71963-8f335646-dcb625d-8f951279_0 is DEAD
2025-10-09T21:48:23.480704Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ca71963-8f335646-dcb625d-8f951279_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:23.480711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ca71963-8f335646-dcb625d-8f951279_0] Write session will restart in 0.008687s
2025-10-09T21:48:23.480736Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ca71963-8f335646-dcb625d-8f951279_0] Write session: Do CDS request
2025-10-09T21:48:23.480742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ca71963-8f335646-dcb625d-8f951279_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:23.481109Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:23.490446Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ca71963-8f335646-dcb625d-8f951279_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:23.490586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:23.490602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1199
2025-10-09T21:48:23.490738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1199 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:23.490772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1199 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:23.490781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1199 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:23.490785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:23.491653Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1199 sessionId:  describe result for acl check
2025-10-09T21:48:23.491688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:23.491690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:23.491692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:23.491698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342171073818218:4549] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:23.492259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342171073818218:4549] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:23.495381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342171073818218:4549] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:23.498428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342171073818218:4549] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:23.498442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342171073818218:4549] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:23.498446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342171073818218:4549] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:23.498453Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1199 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:23.500388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:23.503260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:23.503462Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:23.503473Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:23.503486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:23.503538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:23.503577Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342170711995878:9181]
2025-10-09T21:48:23.504209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:23.504510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:23.504575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:23.505064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:23.505129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.505136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.505137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.505139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.505140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.505141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.505142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.505143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.505145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.505150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.505162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.505166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.505171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.505175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.505179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.505183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.505187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.505190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.505198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.505206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.505212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.505219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.505223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.505229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.505234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.505239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.505244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.505249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.505258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.505263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.505272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.505278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.505287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.505293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.505302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.505307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.505336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:23.511806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:23.512856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:23.512873Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:23.512875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:23.512879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:23.512885Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1198 [2:7559342170711995878:9181]
2025-10-09T21:48:23.512898Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:23.512916Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:23.512919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:23.512923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:23.512925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:23.512943Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:23.512956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:23.512963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:23.513021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.513030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.513032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.513035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:23.513102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:23.513112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.513124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.513137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.513155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:23.513177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342171073818157:2432], now have 1 active actors on pipe
2025-10-09T21:48:23.513186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342171073818409:4549], now have 1 active actors on pipe
2025-10-09T21:48:23.513194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342170711995873:9181]
2025-10-09T21:48:23.513196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342170711995873:9181]
2025-10-09T21:48:23.513198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342170711995873:9181]
2025-10-09T21:48:23.513210Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:23.513229Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:23.513232Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:23.513235Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:23.513237Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:23.513247Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:23.513251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:23.513253Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1198
2025-10-09T21:48:23.513274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342170711995873:9181]
2025-10-09T21:48:23.513268Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1198
2025-10-09T21:48:23.513284Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:23.513290Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:23.513316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:23.513327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:23.513363Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|30b570ce-5fc9ad3e-6d639a62-5d2e7e25_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:23.513378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:23.513382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:23.513384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:23.513394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:23.513403Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:23.513414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:23.513557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:23.513565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:23.513588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:23.513763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1199 partition: 0 MaxSeqNo: 25 sessionId: src_id|30b570ce-5fc9ad3e-6d639a62-5d2e7e25_0
2025-10-09T21:48:23.514169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ca71963-8f335646-dcb625d-8f951279_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|30b570ce-5fc9ad3e-6d639a62-5d2e7e25_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:23.516647Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|30b570ce-5fc9ad3e-6d639a62-5d2e7e25_0] Write session: try to update token
2025-10-09T21:48:23.516663Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|30b570ce-5fc9ad3e-6d639a62-5d2e7e25_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:23.527427Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:23.527447Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:23.527450Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:23.527452Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:23.527462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342170711995873:9181]
2025-10-09T21:48:23.527467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342170711995873:9181]
2025-10-09T21:48:23.527471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342170711995873:9181]
2025-10-09T21:48:23.527514Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:23.527516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:23.527522Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:23.527523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.527526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:23.527526Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:23.527659Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:23.530000Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1199 sessionId: src_id|30b570ce-5fc9ad3e-6d639a62-5d2e7e25_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:23.532344Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:23.580683Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342171073818440:9750]
2025-10-09T21:48:23.581011Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 1198, pipe: [1:7559342171073818440:9750]
2025-10-09T21:48:23.681403Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:23.682021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:23.682054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:23.682101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342171073818440:9750], now have 1 active actors on pipe
2025-10-09T21:48:23.682298Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:23.682369Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:23.682381Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:23.682386Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:23.682422Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342171073818440:9750]
2025-10-09T21:48:23.682443Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1199 reason: error at write: Disconnected sessionId: src_id|30b570ce-5fc9ad3e-6d639a62-5d2e7e25_0
2025-10-09T21:48:23.682617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1199 sessionId: src_id|30b570ce-5fc9ad3e-6d639a62-5d2e7e25_0 is DEAD
2025-10-09T21:48:23.682756Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30b570ce-5fc9ad3e-6d639a62-5d2e7e25_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:23.682764Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30b570ce-5fc9ad3e-6d639a62-5d2e7e25_0] Write session will restart in 0.008523s
2025-10-09T21:48:23.682784Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30b570ce-5fc9ad3e-6d639a62-5d2e7e25_0] Write session: Do CDS request
2025-10-09T21:48:23.682789Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30b570ce-5fc9ad3e-6d639a62-5d2e7e25_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:23.692536Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|30b570ce-5fc9ad3e-6d639a62-5d2e7e25_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:23.692965Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:23.692988Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1200
2025-10-09T21:48:23.693165Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1200 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:23.693195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1200 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:23.693204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1200 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:23.693209Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:23.693710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1200 sessionId:  describe result for acl check
2025-10-09T21:48:23.693759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:23.693761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:23.693762Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:23.693768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342171073818509:5201] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:23.694295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342171073818509:5201] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:23.696681Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342171073818509:5201] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:23.699485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342171073818509:5201] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:23.699498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342171073818509:5201] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:23.699501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342171073818509:5201] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:23.699509Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1200 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:23.702139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:23.704857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:23.705056Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:23.705069Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:23.705081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:23.705135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:23.705174Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342170711996156:9185]
2025-10-09T21:48:23.705743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:23.706125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:23.706179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:23.706528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:23.706582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.706593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.706594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.706595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.706596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.706598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.706599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.706600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.706601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.706607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.706617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.706620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.706624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.706627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.706631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.706635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.706638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.706642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.706649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.706657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.706663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.706668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.706674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.706679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.706682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.706687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.706691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.706696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.706701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.706706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.706711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.706716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.706721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.706726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.706731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.706736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.706761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:23.713229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:23.714319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:23.714334Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:23.714336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:23.714341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:23.714348Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1199 [2:7559342170711996156:9185]
2025-10-09T21:48:23.714365Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:23.714385Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:23.714394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:23.714397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:23.714400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:23.714417Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:23.714423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:23.714432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:23.714488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.714498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.714500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.714503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:23.714577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:23.714584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.714596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.714613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.714635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:23.714663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342171073818691:5201], now have 1 active actors on pipe
2025-10-09T21:48:23.714667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342171073818443:2432], now have 1 active actors on pipe
2025-10-09T21:48:23.714676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342170711996151:9185]
2025-10-09T21:48:23.714679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342170711996151:9185]
2025-10-09T21:48:23.714681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342170711996151:9185]
2025-10-09T21:48:23.714694Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:23.714712Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:23.714713Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:23.714715Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:23.714717Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:23.714728Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:23.714732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:23.714739Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1199
2025-10-09T21:48:23.714732Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1199
2025-10-09T21:48:23.714761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342170711996151:9185]
2025-10-09T21:48:23.714772Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:23.714774Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:23.714789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:23.714795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:23.714849Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e63e1653-edfe561b-318de991-ea4bb4e1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:23.714865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:23.714869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:23.714872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:23.714883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:23.714894Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:23.714920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:23.715055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:23.715065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:23.715078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:23.715154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1200 partition: 0 MaxSeqNo: 25 sessionId: src_id|e63e1653-edfe561b-318de991-ea4bb4e1_0
2025-10-09T21:48:23.715525Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30b570ce-5fc9ad3e-6d639a62-5d2e7e25_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e63e1653-edfe561b-318de991-ea4bb4e1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:23.717995Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e63e1653-edfe561b-318de991-ea4bb4e1_0] Write session: try to update token
2025-10-09T21:48:23.718014Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e63e1653-edfe561b-318de991-ea4bb4e1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:23.729847Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:23.729863Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:23.729866Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:23.729868Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:23.729879Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342170711996151:9185]
2025-10-09T21:48:23.729884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342170711996151:9185]
2025-10-09T21:48:23.729889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342170711996151:9185]
2025-10-09T21:48:23.729934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:23.729941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.729944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:23.729949Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:23.729956Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:23.729960Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:23.730087Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:23.732641Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1200 sessionId: src_id|e63e1653-edfe561b-318de991-ea4bb4e1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:23.735053Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:23.782410Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342171073818726:9750]
2025-10-09T21:48:23.782856Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 1199, pipe: [1:7559342171073818726:9750]
2025-10-09T21:48:23.883893Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:23.884159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:23.884174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:23.884220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342171073818726:9750], now have 1 active actors on pipe
2025-10-09T21:48:23.884416Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:23.884433Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:23.884447Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:23.884451Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342171073818726:9750]
2025-10-09T21:48:23.884472Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:23.884519Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1200 reason: error at write: Disconnected sessionId: src_id|e63e1653-edfe561b-318de991-ea4bb4e1_0
2025-10-09T21:48:23.884697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1200 sessionId: src_id|e63e1653-edfe561b-318de991-ea4bb4e1_0 is DEAD
2025-10-09T21:48:23.884858Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e63e1653-edfe561b-318de991-ea4bb4e1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:23.884867Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e63e1653-edfe561b-318de991-ea4bb4e1_0] Write session will restart in 0.007960s
2025-10-09T21:48:23.884895Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e63e1653-edfe561b-318de991-ea4bb4e1_0] Write session: Do CDS request
2025-10-09T21:48:23.884901Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e63e1653-edfe561b-318de991-ea4bb4e1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:23.893522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e63e1653-edfe561b-318de991-ea4bb4e1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:23.893600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:23.893613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1201
2025-10-09T21:48:23.893812Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1201 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:23.893856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1201 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:23.893867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1201 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:23.893870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:23.894301Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1201 sessionId:  describe result for acl check
2025-10-09T21:48:23.894334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:23.894338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:23.894340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:23.894345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342171073818757:5188] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:23.894860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342171073818757:5188] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:23.897797Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342171073818757:5188] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:23.901038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342171073818757:5188] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:23.901051Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342171073818757:5188] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:23.901054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342171073818757:5188] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:23.901061Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1201 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:23.904712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:23.907409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:23.907612Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:23.907622Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:23.907634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:23.907685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:23.907726Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342170711996431:9187]
2025-10-09T21:48:23.908319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:23.908630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:23.908680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:23.909009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:23.909058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.909064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.909065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.909066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.909068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.909069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.909070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.909071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.909072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.909077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.909088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.909092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.909096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.909100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.909104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.909108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.909112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.909116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.909124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.909131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:23.909138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.909144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:23.909148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.909153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:23.909158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.909163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:23.909167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.909178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:23.909182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.909188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:23.909199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.909204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.909209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.909214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.909221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.909226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.909255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:23.915743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:23.916853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:23.916870Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:23.916872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:23.916876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:23.916883Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1200 [2:7559342170711996431:9187]
2025-10-09T21:48:23.916896Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:23.916916Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:23.916924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:23.916926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:23.916929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:23.916946Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:23.916956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:23.916963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:23.917023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.917034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.917035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.917038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:23.917114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:23.917120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.917130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:23.917140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:23.917157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:23.917183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342171073818731:2432], now have 1 active actors on pipe
2025-10-09T21:48:23.917191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342171073818940:5188], now have 1 active actors on pipe
2025-10-09T21:48:23.917200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342170711996426:9187]
2025-10-09T21:48:23.917206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342170711996426:9187]
2025-10-09T21:48:23.917207Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342170711996426:9187]
2025-10-09T21:48:23.917223Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:23.917239Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:23.917251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:23.917252Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:23.917256Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:23.917258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:23.917258Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:23.917314Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1200
2025-10-09T21:48:23.917340Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1200
2025-10-09T21:48:23.917289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342170711996426:9187]
2025-10-09T21:48:23.917303Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:23.917324Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:23.917397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:23.917405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:23.917440Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7fb05cfa-d4db9a74-dde6529d-3f745302_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:23.917458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:23.917463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:23.917466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:23.917478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:23.917490Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:23.917507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:23.917680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:23.917689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:23.917703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:23.917803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1201 partition: 0 MaxSeqNo: 25 sessionId: src_id|7fb05cfa-d4db9a74-dde6529d-3f745302_0
2025-10-09T21:48:23.918187Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e63e1653-edfe561b-318de991-ea4bb4e1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7fb05cfa-d4db9a74-dde6529d-3f745302_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:23.920641Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7fb05cfa-d4db9a74-dde6529d-3f745302_0] Write session: try to update token
2025-10-09T21:48:23.920656Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7fb05cfa-d4db9a74-dde6529d-3f745302_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:23.931283Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:23.931305Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:23.931307Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:23.931309Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:23.931319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342170711996426:9187]
2025-10-09T21:48:23.931324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342170711996426:9187]
2025-10-09T21:48:23.931328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342170711996426:9187]
2025-10-09T21:48:23.931373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:23.931376Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:23.931381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:23.931382Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:23.931384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:23.931386Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:23.931525Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:23.934086Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1201 sessionId: src_id|7fb05cfa-d4db9a74-dde6529d-3f745302_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:23.936504Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:23.984597Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342171073819015:9750]
2025-10-09T21:48:23.984865Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 1200, pipe: [1:7559342171073819015:9750]
2025-10-09T21:48:24.091027Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:24.091773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.091795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.091843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342171073819015:9750], now have 1 active actors on pipe
2025-10-09T21:48:24.092060Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:24.092056Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:24.092095Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:24.092099Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:24.092113Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342171073819015:9750]
2025-10-09T21:48:24.092174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1201 reason: error at write: Disconnected sessionId: src_id|7fb05cfa-d4db9a74-dde6529d-3f745302_0
2025-10-09T21:48:24.092238Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:48:24.092353Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 12 consumer shared/user session shared/user_1_12_13548706377113208994_v1 is DEAD
2025-10-09T21:48:24.092361Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1201 sessionId: src_id|7fb05cfa-d4db9a74-dde6529d-3f745302_0 is DEAD
2025-10-09T21:48:24.092487Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342080879481361:9747] disconnected.
2025-10-09T21:48:24.092513Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fb05cfa-d4db9a74-dde6529d-3f745302_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:24.092498Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342080879481361:9747] disconnected; active server actors: 1
2025-10-09T21:48:24.092501Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342080879481361:9747] client user disconnected session shared/user_1_12_13548706377113208994_v1
2025-10-09T21:48:24.092545Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fb05cfa-d4db9a74-dde6529d-3f745302_0] Write session will restart in 0.007160s
2025-10-09T21:48:24.092573Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fb05cfa-d4db9a74-dde6529d-3f745302_0] Write session: Do CDS request
2025-10-09T21:48:24.092581Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fb05cfa-d4db9a74-dde6529d-3f745302_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:24.092605Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:48:24.092688Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:48:24.092693Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:48:24.092702Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.008976s
2025-10-09T21:48:24.100543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7fb05cfa-d4db9a74-dde6529d-3f745302_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:24.100619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:24.100637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1202
2025-10-09T21:48:24.100789Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1202 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:24.100834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1202 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:24.100845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1202 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:24.100849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:24.101611Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1202 sessionId:  describe result for acl check
2025-10-09T21:48:24.101647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:24.101655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:24.101657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:24.101664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342175368786338:5184] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:24.102250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342175368786338:5184] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:24.102385Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:48:24.102518Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:48:24.102527Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 13
2025-10-09T21:48:24.102690Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 13 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:48:24.102738Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:48:24.102821Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 auth for : user
2025-10-09T21:48:24.102918Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 Handle describe topics response
2025-10-09T21:48:24.102946Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 auth is DEAD
2025-10-09T21:48:24.102953Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:48:24.103233Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:48:24.103329Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_13_13907029840570446113_v1
2025-10-09T21:48:24.103716Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342175368786361:5182] connected; active server actors: 1
2025-10-09T21:48:24.103738Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559342175368786361:5182] session shared/user_1_13_13907029840570446113_v1
2025-10-09T21:48:24.103750Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:48:24.103761Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:48:24.103768Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_13_13907029840570446113_v1" (Sender=[1:7559342175368786349:5182], Pipe=[1:7559342175368786361:5182], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:48:24.103772Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:48:24.103783Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:48:24.103792Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_13_13907029840570446113_v1" (Sender=[1:7559342175368786349:5182], Pipe=[1:7559342175368786361:5182], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:48:24.103802Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_13_13907029840570446113_v1" sender [1:7559342175368786349:5182] lock partition 0 for ReadingSession "shared/user_1_13_13907029840570446113_v1" (Sender=[1:7559342175368786349:5182], Pipe=[1:7559342175368786361:5182], Partitions=[], ActiveFamilyCount=1) generation 1 step 13
2025-10-09T21:48:24.103812Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:48:24.103816Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000030s
2025-10-09T21:48:24.104040Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 13 Session: "shared/user_1_13_13907029840570446113_v1" ClientId: "user" PipeClient { RawX1: 7559342175368786361 RawX2: 4503603922342974 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:48:24.104067Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:48:24.105662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342175368786338:5184] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:24.109342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342175368786338:5184] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:24.109354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342175368786338:5184] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:24.109357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342175368786338:5184] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:24.109365Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1202 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:24.114573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:24.117325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:24.117535Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:24.117548Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:24.117561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:24.117621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:24.117663Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342175006964004:9189]
2025-10-09T21:48:24.118311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:24.118620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:24.118671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:24.118989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:24.119045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.119046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.119048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.119049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.119050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.119052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.119053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.119054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.119055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.119060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.119072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.119075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.119080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.119083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.119088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.119092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.119096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.119100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.119109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.119117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.119123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.119129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.119134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.119145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.119149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.119155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.119159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.119164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.119169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.119180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.119185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.119190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.119195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.119200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.119206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.119211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.119246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:24.125820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:24.126923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:24.126941Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:24.126944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:24.126949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:24.126957Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1201 [2:7559342175006964004:9189]
2025-10-09T21:48:24.126972Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:24.126992Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:24.127000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:24.127004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:24.127006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:24.127025Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:24.127036Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:24.127043Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:24.127110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.127118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.127120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.127124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:24.127197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:24.127206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.127219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.127232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.127250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:24.127279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342175368786314:2432], now have 1 active actors on pipe
2025-10-09T21:48:24.127288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342175368786501:5184], now have 1 active actors on pipe
2025-10-09T21:48:24.127292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342175368786382:5175], now have 1 active actors on pipe
2025-10-09T21:48:24.127300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342175006963999:9189]
2025-10-09T21:48:24.127302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342175006963999:9189]
2025-10-09T21:48:24.127305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342175006963999:9189]
2025-10-09T21:48:24.127319Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:24.127338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:24.127353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:24.127354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:24.127357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:24.127327Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1201
2025-10-09T21:48:24.127363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:24.127335Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1201
2025-10-09T21:48:24.127366Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:24.127371Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1201, pipe: [1:7559342175368786382:5175]
2025-10-09T21:48:24.127397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342175006963999:9189]
2025-10-09T21:48:24.127408Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:24.127421Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:24.127423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.127428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.127445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.127452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.127459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_13_13907029840570446113_v1 on pipe: [1:7559342175368786382:5175]
2025-10-09T21:48:24.127498Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_13_13907029840570446113_v1:1 with generation 1201
2025-10-09T21:48:24.127515Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a014f93d-4f791a94-3d8226d5-3454a4f4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:24.127530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:24.127539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:24.127542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:24.127553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:24.127565Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:24.127581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:24.127590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:24.127592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:24.127608Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_13_13907029840570446113_v1
2025-10-09T21:48:24.127616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:24.127646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:24.127662Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:48:24.127808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.127815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.127830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:24.127944Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1202 partition: 0 MaxSeqNo: 25 sessionId: src_id|a014f93d-4f791a94-3d8226d5-3454a4f4_0
2025-10-09T21:48:24.128305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fb05cfa-d4db9a74-dde6529d-3f745302_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a014f93d-4f791a94-3d8226d5-3454a4f4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:24.130860Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a014f93d-4f791a94-3d8226d5-3454a4f4_0] Write session: try to update token
2025-10-09T21:48:24.130881Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a014f93d-4f791a94-3d8226d5-3454a4f4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:24.132159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:48:24.132207Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:48:24.132222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:24.132235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:24.132237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:24.132247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:48:24.132262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:48:24.132440Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046504127 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:48:24.132462Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:48:24.132500Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 sending to client partition status
2025-10-09T21:48:24.144771Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1202 sessionId: src_id|a014f93d-4f791a94-3d8226d5-3454a4f4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:24.142114Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:24.142137Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:24.142139Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:24.142142Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:24.142152Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342175006963999:9189]
2025-10-09T21:48:24.142170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342175006963999:9189]
2025-10-09T21:48:24.142175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342175006963999:9189]
2025-10-09T21:48:24.142228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:24.142232Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:24.142239Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:24.142241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.142244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:24.142246Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:24.142383Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:24.147310Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:24.303340Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:24.303888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.303906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.304152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_13_13907029840570446113_v1
2025-10-09T21:48:24.304174Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:24.304206Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_13_13907029840570446113_v1
2025-10-09T21:48:24.304232Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342175368786382:5175]
2025-10-09T21:48:24.304262Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:24.304273Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:24.304344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1202 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a014f93d-4f791a94-3d8226d5-3454a4f4_0
2025-10-09T21:48:24.304549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1202 sessionId: src_id|a014f93d-4f791a94-3d8226d5-3454a4f4_0 is DEAD
2025-10-09T21:48:24.304690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a014f93d-4f791a94-3d8226d5-3454a4f4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:24.304702Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a014f93d-4f791a94-3d8226d5-3454a4f4_0] Write session will restart in 0.005288s
2025-10-09T21:48:24.304727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a014f93d-4f791a94-3d8226d5-3454a4f4_0] Write session: Do CDS request
2025-10-09T21:48:24.304733Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a014f93d-4f791a94-3d8226d5-3454a4f4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:24.311549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a014f93d-4f791a94-3d8226d5-3454a4f4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:24.311599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:24.311615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1203
2025-10-09T21:48:24.311797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1203 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:24.311840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1203 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:24.311849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1203 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:24.311854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:24.312307Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1203 sessionId:  describe result for acl check
2025-10-09T21:48:24.312346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:24.312347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:24.312349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:24.312356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342175368786641:5168] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:24.312935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342175368786641:5168] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:24.316135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342175368786641:5168] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:24.319715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342175368786641:5168] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:24.319733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342175368786641:5168] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:24.319737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342175368786641:5168] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:24.319745Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1203 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:24.323435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:24.326050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:24.326243Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:24.326253Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:24.326264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:24.326309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:24.326356Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342175006964314:9201]
2025-10-09T21:48:24.326826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:24.327119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:24.327152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:24.327427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:24.327484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.327490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.327491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.327492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.327493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.327494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.327496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.327497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.327498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.327503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.327513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.327516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.327520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.327523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.327527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.327531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.327535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.327538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.327545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.327552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.327558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.327563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.327567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.327571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.327575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.327580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.327585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.327590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.327595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.327606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.327611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.327616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.327624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.327629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.327632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.327635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.327661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:24.333772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:24.334979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:24.334995Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:24.334998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:24.335002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:24.335008Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1202 [2:7559342175006964314:9201]
2025-10-09T21:48:24.335021Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:24.335040Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:24.335043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:24.335046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:24.335049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:24.335066Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:24.335072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:24.335080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:24.335133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.335143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.335145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.335148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:24.335234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:24.335241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.335253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.335267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.335289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:24.335318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342175368786819:5168], now have 1 active actors on pipe
2025-10-09T21:48:24.335322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342175368786619:2432], now have 1 active actors on pipe
2025-10-09T21:48:24.335329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342175006964309:9201]
2025-10-09T21:48:24.335332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342175006964309:9201]
2025-10-09T21:48:24.335334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342175006964309:9201]
2025-10-09T21:48:24.335344Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:24.335357Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:24.335359Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:24.335367Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:24.335369Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:24.335372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:24.335375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:24.335408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342175006964309:9201]
2025-10-09T21:48:24.335382Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1202
2025-10-09T21:48:24.335417Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:24.335418Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:24.335433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.335438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.335383Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1202
2025-10-09T21:48:24.335495Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a846e28f-e8c1aba0-60a5631a-ed8c715a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:24.335507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:24.335511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:24.335514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:24.335524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:24.335539Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:24.335567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:24.335782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.335795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.335817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:24.335917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1203 partition: 0 MaxSeqNo: 25 sessionId: src_id|a846e28f-e8c1aba0-60a5631a-ed8c715a_0
2025-10-09T21:48:24.336286Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a014f93d-4f791a94-3d8226d5-3454a4f4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a846e28f-e8c1aba0-60a5631a-ed8c715a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:24.338819Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a846e28f-e8c1aba0-60a5631a-ed8c715a_0] Write session: try to update token
2025-10-09T21:48:24.338840Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a846e28f-e8c1aba0-60a5631a-ed8c715a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:24.349527Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:24.349547Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:24.349549Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:24.349551Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:24.349561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342175006964309:9201]
2025-10-09T21:48:24.349566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342175006964309:9201]
2025-10-09T21:48:24.349570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342175006964309:9201]
2025-10-09T21:48:24.349615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:24.349616Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:24.349622Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:24.349623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.349625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:24.349626Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:24.349764Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:24.352295Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1203 sessionId: src_id|a846e28f-e8c1aba0-60a5631a-ed8c715a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:24.354726Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:24.404478Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342175368786925:5175]
2025-10-09T21:48:24.404759Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 1202, pipe: [1:7559342175368786925:5175]
2025-10-09T21:48:24.505943Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:24.506212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.506227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.506273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342175368786925:5175], now have 1 active actors on pipe
2025-10-09T21:48:24.506472Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:24.506486Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:24.506491Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342175368786925:5175]
2025-10-09T21:48:24.506502Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:24.506509Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:24.506565Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1203 reason: error at write: Disconnected sessionId: src_id|a846e28f-e8c1aba0-60a5631a-ed8c715a_0
2025-10-09T21:48:24.506738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1203 sessionId: src_id|a846e28f-e8c1aba0-60a5631a-ed8c715a_0 is DEAD
2025-10-09T21:48:24.506872Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a846e28f-e8c1aba0-60a5631a-ed8c715a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:24.506883Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a846e28f-e8c1aba0-60a5631a-ed8c715a_0] Write session will restart in 0.006546s
2025-10-09T21:48:24.506911Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a846e28f-e8c1aba0-60a5631a-ed8c715a_0] Write session: Do CDS request
2025-10-09T21:48:24.506918Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a846e28f-e8c1aba0-60a5631a-ed8c715a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:24.514483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a846e28f-e8c1aba0-60a5631a-ed8c715a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:24.514902Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:24.514918Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1204
2025-10-09T21:48:24.515533Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1204 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:24.515570Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1204 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:24.515582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1204 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:24.515586Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:24.516036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1204 sessionId:  describe result for acl check
2025-10-09T21:48:24.516077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:24.516085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:24.516087Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:24.516092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342175368786955:5154] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:24.516579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342175368786955:5154] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:24.520217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342175368786955:5154] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:24.524115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342175368786955:5154] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:24.524132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342175368786955:5154] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:24.524136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342175368786955:5154] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:24.524145Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1204 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:24.526964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:24.530127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:24.530407Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:24.530420Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:24.530436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:24.530511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:24.530561Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342175006964600:9205]
2025-10-09T21:48:24.531055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:24.531352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:24.531392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:24.531664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:24.531723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.531730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.531731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.531732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.531734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.531735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.531736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.531737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.531738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.531744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.531755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.531760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.531764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.531768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.531771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.531775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.531778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.531781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.531789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.531797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.531803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.531807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.531811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.531816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.531820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.531825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.531830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.531835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.531839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.531845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.531852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.531857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.531862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.531866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.531871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.531877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.531914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:24.538338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:24.539552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:24.539568Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:24.539571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:24.539575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:24.539582Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1203 [2:7559342175006964600:9205]
2025-10-09T21:48:24.539597Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:24.539613Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:24.539619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:24.539622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:24.539624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:24.539639Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:24.539646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:24.539652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:24.539700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.539705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.539706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.539708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:24.539765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:24.539771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.539783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.539795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.539815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:24.539841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342175368786928:2432], now have 1 active actors on pipe
2025-10-09T21:48:24.539852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342175006964595:9205]
2025-10-09T21:48:24.539853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342175006964595:9205]
2025-10-09T21:48:24.539855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342175006964595:9205]
2025-10-09T21:48:24.539867Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:24.539883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342175368787168:5154], now have 1 active actors on pipe
2025-10-09T21:48:24.539883Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:24.539887Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:24.539888Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:24.539895Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:24.539907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:24.539909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:24.539942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342175006964595:9205]
2025-10-09T21:48:24.539954Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:24.539939Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1203
2025-10-09T21:48:24.539975Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:24.539947Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1203
2025-10-09T21:48:24.540371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.540386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.540417Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8fb7e2ac-8dbea336-98c552de-29eb4c15_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:24.540431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:24.540436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:24.540438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:24.540450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:24.540462Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:24.540483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:24.540655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.540664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.540678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:24.540767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1204 partition: 0 MaxSeqNo: 25 sessionId: src_id|8fb7e2ac-8dbea336-98c552de-29eb4c15_0
2025-10-09T21:48:24.541160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a846e28f-e8c1aba0-60a5631a-ed8c715a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8fb7e2ac-8dbea336-98c552de-29eb4c15_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:24.543710Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8fb7e2ac-8dbea336-98c552de-29eb4c15_0] Write session: try to update token
2025-10-09T21:48:24.543729Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8fb7e2ac-8dbea336-98c552de-29eb4c15_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:24.557482Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1204 sessionId: src_id|8fb7e2ac-8dbea336-98c552de-29eb4c15_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:24.557838Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:24.557858Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:24.557867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342175006964595:9205]
2025-10-09T21:48:24.557918Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:24.557923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:24.558111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:24.558259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:24.558402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:24.558544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:24.558735Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:24.558783Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:24.559936Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:24.560137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.560155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.560191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:24.560264Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:24.564337Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:24.564354Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:24.564356Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:24.564358Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:24.564367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342175006964595:9205]
2025-10-09T21:48:24.564371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342175006964595:9205]
2025-10-09T21:48:24.564375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342175006964595:9205]
2025-10-09T21:48:24.564411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:24.564421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.564422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:24.564461Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:24.564477Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:24.564481Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:24.564548Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:24.606719Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342175368787215:5175]
2025-10-09T21:48:24.606998Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 1203, pipe: [1:7559342175368787215:5175]
2025-10-09T21:48:24.722968Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:24.731034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.731065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.731232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:24.731298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:24.731368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:24.731432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:24.731474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:24.731484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:24.731550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:24.731611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:24.731674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:24.731740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:24.731785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:24.731794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:24.731856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:24.731915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:24.731977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:24.732040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:24.732088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:24.732097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:24.732187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:24.732257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:24.732318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:24.732380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:24.732419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:24.732426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:24.732488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:24.732547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:24.732608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:24.732669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:24.732717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:24.732725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:24.732811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:24.732878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:24.732944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:24.733008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:24.733050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:24.733058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:24.733120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:24.733176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:24.733231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:24.733293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:24.733337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:24.733345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:24.733405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:24.733457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:24.733511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:24.733568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:24.733731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:24.733741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:24.733839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:24.733930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:24.734025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:24.734128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:24.734183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:24.734192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:24.734251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:24.734306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:24.734369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:24.734432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:24.734474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:24.734481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:24.734542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342175368787215:5175], now have 1 active actors on pipe
2025-10-09T21:48:24.734795Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:24.734836Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342175368787215:5175]
2025-10-09T21:48:24.734851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:24.734855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:24.734887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1204 reason: error at write: Disconnected sessionId: src_id|8fb7e2ac-8dbea336-98c552de-29eb4c15_0
2025-10-09T21:48:24.735082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1204 sessionId: src_id|8fb7e2ac-8dbea336-98c552de-29eb4c15_0 is DEAD
2025-10-09T21:48:24.735230Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fb7e2ac-8dbea336-98c552de-29eb4c15_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:24.735256Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fb7e2ac-8dbea336-98c552de-29eb4c15_0] Write session will restart in 0.007828s
2025-10-09T21:48:24.735285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fb7e2ac-8dbea336-98c552de-29eb4c15_0] Write session: Do CDS request
2025-10-09T21:48:24.735292Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fb7e2ac-8dbea336-98c552de-29eb4c15_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:24.735528Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:24.744524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8fb7e2ac-8dbea336-98c552de-29eb4c15_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:24.744593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:24.744609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1205
2025-10-09T21:48:24.744727Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1205 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:24.744761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1205 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:24.744768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1205 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:24.744771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:24.745261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1205 sessionId:  describe result for acl check
2025-10-09T21:48:24.745304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:24.745306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:24.745307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:24.745313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342175368787253:5149] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:24.745888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342175368787253:5149] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:24.748929Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342175368787253:5149] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:24.753003Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342175368787253:5149] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:24.753017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342175368787253:5149] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:24.753021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342175368787253:5149] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:24.753030Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1205 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:24.757247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:24.759954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:24.760121Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:24.760129Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:24.760138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:24.760178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:24.760212Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342175006964884:9209]
2025-10-09T21:48:24.760611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:24.760894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:24.760932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:24.761185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:24.761243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.761249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.761250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.761252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.761253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.761254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.761255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.761256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.761258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.761263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.761275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.761279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.761283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.761287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.761291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.761295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.761299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.761303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.761310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.761319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.761325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.761331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.761341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.761346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.761351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.761360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.761364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.761369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.761373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.761379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.761384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.761389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.761394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.761404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.761409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.761413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.761446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:24.767890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:24.768940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:24.768961Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:24.768963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:24.768967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:24.768974Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1204 [2:7559342175006964884:9209]
2025-10-09T21:48:24.768988Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:24.769006Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:24.769010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:24.769013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:24.769015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:24.769033Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:24.769043Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:24.769051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:24.769115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.769126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.769128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.769131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:24.769199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:24.769210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.769222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.769234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.769250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:24.769287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342175368787433:5149], now have 1 active actors on pipe
2025-10-09T21:48:24.769298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342175368787219:2432], now have 1 active actors on pipe
2025-10-09T21:48:24.769305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342175006964879:9209]
2025-10-09T21:48:24.769308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342175006964879:9209]
2025-10-09T21:48:24.769310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342175006964879:9209]
2025-10-09T21:48:24.769323Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:24.769338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:24.769352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:24.769360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:24.769366Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:24.769370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:24.769372Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:24.769391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342175006964879:9209]
2025-10-09T21:48:24.769399Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:24.769405Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:24.769354Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1204
2025-10-09T21:48:24.769382Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1204
2025-10-09T21:48:24.769436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.769448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.769492Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|982e4841-b3fe9e33-147c4082-594d7c8f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:24.769503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:24.769507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:24.769510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:24.769521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:24.769530Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:24.769544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:24.769747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.769766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.769784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:24.769890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1205 partition: 0 MaxSeqNo: 25 sessionId: src_id|982e4841-b3fe9e33-147c4082-594d7c8f_0
2025-10-09T21:48:24.770258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8fb7e2ac-8dbea336-98c552de-29eb4c15_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|982e4841-b3fe9e33-147c4082-594d7c8f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:24.772622Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|982e4841-b3fe9e33-147c4082-594d7c8f_0] Write session: try to update token
2025-10-09T21:48:24.772639Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|982e4841-b3fe9e33-147c4082-594d7c8f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:24.783653Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:24.783676Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:24.783678Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:24.783680Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:24.783689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342175006964879:9209]
2025-10-09T21:48:24.783694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342175006964879:9209]
2025-10-09T21:48:24.783698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342175006964879:9209]
2025-10-09T21:48:24.783749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:24.783755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.783758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:24.783753Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:24.783769Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:24.783773Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:24.783894Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:24.785637Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1205 sessionId: src_id|982e4841-b3fe9e33-147c4082-594d7c8f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:24.788047Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:24.835081Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342175368787513:5175]
2025-10-09T21:48:24.835388Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 1204, pipe: [1:7559342175368787513:5175]
2025-10-09T21:48:24.940744Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:24.941652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.941670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.941728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342175368787513:5175], now have 1 active actors on pipe
2025-10-09T21:48:24.941939Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:24.941954Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342175368787513:5175]
2025-10-09T21:48:24.941961Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:24.941992Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:24.942092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1205 reason: status is not ok: tablet will be restarted right now sessionId: src_id|982e4841-b3fe9e33-147c4082-594d7c8f_0
2025-10-09T21:48:24.942250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1205 sessionId: src_id|982e4841-b3fe9e33-147c4082-594d7c8f_0 is DEAD
2025-10-09T21:48:24.942414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|982e4841-b3fe9e33-147c4082-594d7c8f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:24.942425Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|982e4841-b3fe9e33-147c4082-594d7c8f_0] Write session will restart in 0.008522s
2025-10-09T21:48:24.942453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|982e4841-b3fe9e33-147c4082-594d7c8f_0] Write session: Do CDS request
2025-10-09T21:48:24.942460Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|982e4841-b3fe9e33-147c4082-594d7c8f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:24.952594Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|982e4841-b3fe9e33-147c4082-594d7c8f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:24.952669Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:24.952687Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1206
2025-10-09T21:48:24.952843Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1206 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:24.952892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1206 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:24.952898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1206 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:24.952902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:24.953466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1206 sessionId:  describe result for acl check
2025-10-09T21:48:24.953504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:24.953513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:24.953515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:24.953521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342175368787558:5142] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:24.954156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342175368787558:5142] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:24.956927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342175368787558:5142] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:24.960607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342175368787558:5142] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:24.960622Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342175368787558:5142] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:24.960626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342175368787558:5142] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:24.960635Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1206 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:24.964869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:24.967823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:24.968052Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:24.968064Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:24.968079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:24.968147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:24.968194Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342175006965164:9211]
2025-10-09T21:48:24.968808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:24.969163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:24.969228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:24.969583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:24.969638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.969639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.969641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.969642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.969643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.969645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.969646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.969647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.969649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.969654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.969665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.969669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.969673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.969677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.969682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.969686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.969690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.969694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.969702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.969711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:24.969732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.969737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:24.969742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.969748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:24.969752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.969758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:24.969763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.969769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:24.969775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.969781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:24.969786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.969791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.969795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.969801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.969806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.969810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.969841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:24.976541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:24.977584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:24.977601Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:24.977604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:24.977609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:24.977616Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1205 [2:7559342175006965164:9211]
2025-10-09T21:48:24.977630Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:24.977648Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:24.977651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:24.977654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:24.977657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:24.977675Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:24.977682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:24.977690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:24.977756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.977765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.977767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.977770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:24.977854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:24.977862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.977876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:24.977883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:24.977896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:24.977914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342175368787516:2432], now have 1 active actors on pipe
2025-10-09T21:48:24.977921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342175368787723:5142], now have 1 active actors on pipe
2025-10-09T21:48:24.977927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342175006965159:9211]
2025-10-09T21:48:24.977929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342175006965159:9211]
2025-10-09T21:48:24.977931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342175006965159:9211]
2025-10-09T21:48:24.977940Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:24.977952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:24.977959Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:24.977962Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:24.977965Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:24.977964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:24.977967Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:24.977996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342175006965159:9211]
2025-10-09T21:48:24.978002Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:24.978019Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:24.977992Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1205
2025-10-09T21:48:24.978051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.978058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.977998Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1205
2025-10-09T21:48:24.978083Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bcf3e03d-539f13db-c13986b-85f78628_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:24.978093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:24.978096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:24.978098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:24.978106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:24.978118Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:24.978131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:24.978299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:24.978325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:24.978353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:24.978495Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1206 partition: 0 MaxSeqNo: 25 sessionId: src_id|bcf3e03d-539f13db-c13986b-85f78628_0
2025-10-09T21:48:24.978846Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|982e4841-b3fe9e33-147c4082-594d7c8f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bcf3e03d-539f13db-c13986b-85f78628_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:24.981306Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bcf3e03d-539f13db-c13986b-85f78628_0] Write session: try to update token
2025-10-09T21:48:24.981325Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bcf3e03d-539f13db-c13986b-85f78628_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:24.992331Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:24.992353Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:24.992356Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:24.992358Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:24.992368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342175006965159:9211]
2025-10-09T21:48:24.992373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342175006965159:9211]
2025-10-09T21:48:24.992377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342175006965159:9211]
2025-10-09T21:48:24.992424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:24.992428Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:24.992433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:24.992435Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:24.992435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:24.992441Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:24.992567Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:24.995352Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1206 sessionId: src_id|bcf3e03d-539f13db-c13986b-85f78628_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:24.997664Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:25.041923Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342179663755102:5175]
2025-10-09T21:48:25.042247Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 1205, pipe: [1:7559342179663755102:5175]
2025-10-09T21:48:25.154383Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:25.155121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:25.155142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:25.155190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342179663755102:5175], now have 1 active actors on pipe
2025-10-09T21:48:25.155391Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:25.155466Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342179663755102:5175]
2025-10-09T21:48:25.155477Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:25.155502Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:25.155585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1206 reason: status is not ok: tablet will be restarted right now sessionId: src_id|bcf3e03d-539f13db-c13986b-85f78628_0
2025-10-09T21:48:25.155774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1206 sessionId: src_id|bcf3e03d-539f13db-c13986b-85f78628_0 is DEAD
2025-10-09T21:48:25.155938Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bcf3e03d-539f13db-c13986b-85f78628_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:25.155949Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bcf3e03d-539f13db-c13986b-85f78628_0] Write session will restart in 0.005007s
2025-10-09T21:48:25.155976Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bcf3e03d-539f13db-c13986b-85f78628_0] Write session: Do CDS request
2025-10-09T21:48:25.155983Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bcf3e03d-539f13db-c13986b-85f78628_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:25.162517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bcf3e03d-539f13db-c13986b-85f78628_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:25.162586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:25.162601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1207
2025-10-09T21:48:25.162775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1207 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:25.162809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1207 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:25.162820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1207 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:25.162824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:25.163261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1207 sessionId:  describe result for acl check
2025-10-09T21:48:25.163303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:25.163305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:25.163307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:25.163313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342179663755124:5136] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:25.163903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342179663755124:5136] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:25.167030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342179663755124:5136] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:25.171171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342179663755124:5136] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:25.171191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342179663755124:5136] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:25.171195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342179663755124:5136] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:25.171205Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1207 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:25.177452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:25.180516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:25.180733Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:25.180744Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:25.180756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:25.180812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:25.180863Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342179301932742:9213]
2025-10-09T21:48:25.181378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:25.181735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:25.181778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:25.182118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:25.182164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:25.182170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:25.182171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:25.182173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:25.182174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:25.182176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:25.182177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.182178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.182180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.182185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:25.182196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:25.182201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:25.182205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:25.182209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:25.182213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:25.182217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.182220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.182224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.182232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:25.182240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:25.182248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:25.182253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:25.182259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:25.182264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:25.182268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:25.182273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:25.182278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:25.182283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:25.182287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:25.182292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:25.182303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.182309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.182313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.182319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.182328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.182333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.182363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:25.189164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:25.190361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:25.190379Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:25.190382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:25.190386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:25.190393Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1206 [2:7559342179301932742:9213]
2025-10-09T21:48:25.190409Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:25.190431Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:25.190434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:25.190437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:25.190439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:25.190457Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:25.190464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:25.190471Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:25.190533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.190535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.190537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.190540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:25.190613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:25.190623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.190638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.190656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.190674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:25.190706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342179663755279:5136], now have 1 active actors on pipe
2025-10-09T21:48:25.190710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342179663755109:2432], now have 1 active actors on pipe
2025-10-09T21:48:25.190717Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342179301932737:9213]
2025-10-09T21:48:25.190720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342179301932737:9213]
2025-10-09T21:48:25.190723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342179301932737:9213]
2025-10-09T21:48:25.190735Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:25.190753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:25.190757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:25.190763Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:25.190766Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:25.190773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:25.190776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:25.190772Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1206
2025-10-09T21:48:25.190782Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1206
2025-10-09T21:48:25.190814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342179301932737:9213]
2025-10-09T21:48:25.190827Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:25.190836Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:25.190848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:25.190856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:25.190906Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9dfda970-b74723f-30fa6396-5d32a17_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:25.190923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:25.190927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:25.190930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:25.190940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:25.190951Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:25.190970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:25.191166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:25.191194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:25.191222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:25.191304Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1207 partition: 0 MaxSeqNo: 25 sessionId: src_id|9dfda970-b74723f-30fa6396-5d32a17_0
2025-10-09T21:48:25.191701Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bcf3e03d-539f13db-c13986b-85f78628_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9dfda970-b74723f-30fa6396-5d32a17_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:25.194344Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9dfda970-b74723f-30fa6396-5d32a17_0] Write session: try to update token
2025-10-09T21:48:25.194366Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9dfda970-b74723f-30fa6396-5d32a17_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:25.205500Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:25.205517Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:25.205519Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:25.205521Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:25.205531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342179301932737:9213]
2025-10-09T21:48:25.205535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342179301932737:9213]
2025-10-09T21:48:25.205539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342179301932737:9213]
2025-10-09T21:48:25.205588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:25.205588Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:25.205594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.205594Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:25.205596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:25.205599Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:25.205753Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:25.207961Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1207 sessionId: src_id|9dfda970-b74723f-30fa6396-5d32a17_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:25.210398Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:25.255436Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342179663755405:5175]
2025-10-09T21:48:25.255733Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 1206, pipe: [1:7559342179663755405:5175]
2025-10-09T21:48:25.368498Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:25.368799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:25.368820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:25.368869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342179663755405:5175], now have 1 active actors on pipe
2025-10-09T21:48:25.369099Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:25.369166Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:25.369186Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:25.369190Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:25.369248Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342179663755405:5175]
2025-10-09T21:48:25.369252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1207 reason: error at write: Disconnected sessionId: src_id|9dfda970-b74723f-30fa6396-5d32a17_0
2025-10-09T21:48:25.369459Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1207 sessionId: src_id|9dfda970-b74723f-30fa6396-5d32a17_0 is DEAD
2025-10-09T21:48:25.369644Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9dfda970-b74723f-30fa6396-5d32a17_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:25.369657Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9dfda970-b74723f-30fa6396-5d32a17_0] Write session will restart in 0.008529s
2025-10-09T21:48:25.369689Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9dfda970-b74723f-30fa6396-5d32a17_0] Write session: Do CDS request
2025-10-09T21:48:25.369697Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9dfda970-b74723f-30fa6396-5d32a17_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:25.379653Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9dfda970-b74723f-30fa6396-5d32a17_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:25.379760Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:25.379779Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1208
2025-10-09T21:48:25.379958Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1208 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:25.379997Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1208 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:25.380009Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1208 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:25.380013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:25.380662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1208 sessionId:  describe result for acl check
2025-10-09T21:48:25.380704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:25.380712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:25.380714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:25.380722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342179663755447:5132] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:25.381347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342179663755447:5132] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:25.384316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342179663755447:5132] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:25.388469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342179663755447:5132] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:25.388487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342179663755447:5132] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:25.388491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342179663755447:5132] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:25.388501Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1208 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:25.391751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:25.394949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:25.395157Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:25.395165Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:25.395175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:25.395216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:25.395248Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342179301933054:9225]
2025-10-09T21:48:25.395667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:25.395946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:25.395981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:25.396195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:25.396239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:25.396244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:25.396245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:25.396247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:25.396247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:25.396248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:25.396249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.396249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.396250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.396253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:25.396260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:25.396262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:25.396264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:25.396266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:25.396269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:25.396271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.396273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.396275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.396279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:25.396284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:25.396289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:25.396292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:25.396295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:25.396298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:25.396301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:25.396304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:25.396307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:25.396310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:25.396312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:25.396315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:25.396319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.396322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.396325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.396327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.396332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.396335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.396351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:25.403017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:25.404193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:25.404211Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:25.404214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:25.404219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:25.404225Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1207 [2:7559342179301933054:9225]
2025-10-09T21:48:25.404240Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:25.404258Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:25.404261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:25.404264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:25.404266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:25.404287Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:25.404293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:25.404300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:25.404361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.404370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.404372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.404375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:25.404444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:25.404453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.404465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.404482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.404499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:25.404650Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1207
2025-10-09T21:48:25.404684Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1207
2025-10-09T21:48:25.404527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342179663755641:5132], now have 1 active actors on pipe
2025-10-09T21:48:25.404538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342179663755410:2432], now have 1 active actors on pipe
2025-10-09T21:48:25.404546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342179301933049:9225]
2025-10-09T21:48:25.404548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342179301933049:9225]
2025-10-09T21:48:25.404551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342179301933049:9225]
2025-10-09T21:48:25.404564Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:25.404578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:25.404579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:25.404581Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:25.404584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:25.404595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:25.404599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:25.404633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342179301933049:9225]
2025-10-09T21:48:25.404640Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:25.404646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:25.404787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:25.404793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:25.404823Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fade4ef9-1bb78b23-893cfc20-d6499384_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:25.404832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:25.404836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:25.404839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:25.404863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:25.404877Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:25.404894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:25.405028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:25.405036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:25.405049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:25.405141Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1208 partition: 0 MaxSeqNo: 25 sessionId: src_id|fade4ef9-1bb78b23-893cfc20-d6499384_0
2025-10-09T21:48:25.405520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9dfda970-b74723f-30fa6396-5d32a17_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fade4ef9-1bb78b23-893cfc20-d6499384_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:25.408227Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fade4ef9-1bb78b23-893cfc20-d6499384_0] Write session: try to update token
2025-10-09T21:48:25.408250Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fade4ef9-1bb78b23-893cfc20-d6499384_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:25.419325Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:25.419350Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:25.419352Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:25.419354Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:25.419362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342179301933049:9225]
2025-10-09T21:48:25.419367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342179301933049:9225]
2025-10-09T21:48:25.419371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342179301933049:9225]
2025-10-09T21:48:25.419412Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:25.419417Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:25.419417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:25.419420Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:25.419423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.419426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:25.419557Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:25.423019Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1208 sessionId: src_id|fade4ef9-1bb78b23-893cfc20-d6499384_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:25.425668Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:25.469431Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342179663755720:5175]
2025-10-09T21:48:25.469627Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 1207, pipe: [1:7559342179663755720:5175]
2025-10-09T21:48:25.577339Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:25.578027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:25.578056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:25.578106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342179663755720:5175], now have 1 active actors on pipe
2025-10-09T21:48:25.578320Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:25.578327Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:25.578339Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:25.578358Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342179663755720:5175]
2025-10-09T21:48:25.578379Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:25.578399Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1208 reason: error at write: Disconnected sessionId: src_id|fade4ef9-1bb78b23-893cfc20-d6499384_0
2025-10-09T21:48:25.578575Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1208 sessionId: src_id|fade4ef9-1bb78b23-893cfc20-d6499384_0 is DEAD
2025-10-09T21:48:25.578725Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fade4ef9-1bb78b23-893cfc20-d6499384_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:25.578737Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fade4ef9-1bb78b23-893cfc20-d6499384_0] Write session will restart in 0.007334s
2025-10-09T21:48:25.578776Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fade4ef9-1bb78b23-893cfc20-d6499384_0] Write session: Do CDS request
2025-10-09T21:48:25.578782Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fade4ef9-1bb78b23-893cfc20-d6499384_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:25.587522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fade4ef9-1bb78b23-893cfc20-d6499384_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:25.587628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:25.587645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1209
2025-10-09T21:48:25.587788Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1209 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:25.587821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1209 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:25.587831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1209 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:25.587836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:25.588267Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1209 sessionId:  describe result for acl check
2025-10-09T21:48:25.588305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:25.588313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:25.588315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:25.588321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342179663755748:5114] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:25.588907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342179663755748:5114] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:25.592814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342179663755748:5114] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:25.595838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342179663755748:5114] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:25.595853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342179663755748:5114] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:25.595856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342179663755748:5114] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:25.595863Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1209 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:25.599875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:25.602536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:25.602745Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:25.602754Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:25.602764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:25.602815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:25.602849Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342179301933345:9229]
2025-10-09T21:48:25.603224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:25.603517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:25.603560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:25.603948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:25.604010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:25.604016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:25.604017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:25.604018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:25.604020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:25.604021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:25.604022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.604023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.604024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.604029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:25.604039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:25.604043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:25.604048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:25.604051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:25.604055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:25.604059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.604062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.604065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.604072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:25.604079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:25.604086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:25.604091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:25.604095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:25.604100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:25.604110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:25.604115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:25.604124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:25.604129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:25.604133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:25.604138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:25.604143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.604148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.604152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.604161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.604166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.604169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.604194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:25.610564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:25.612110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:25.612127Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:25.612130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:25.612134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:25.612141Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1208 [2:7559342179301933345:9229]
2025-10-09T21:48:25.612154Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:25.612173Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:25.612176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:25.612180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:25.612183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:25.612200Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:25.612206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:25.612213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:25.612276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.612287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.612289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.612292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:25.612371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:25.612379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.612391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.612404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.612421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:25.612452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342179663755723:2432], now have 1 active actors on pipe
2025-10-09T21:48:25.612456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342179663755926:5114], now have 1 active actors on pipe
2025-10-09T21:48:25.612464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342179301933340:9229]
2025-10-09T21:48:25.612472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342179301933340:9229]
2025-10-09T21:48:25.612474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342179301933340:9229]
2025-10-09T21:48:25.612487Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:25.612502Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:25.612506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:25.612513Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:25.612516Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:25.612522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:25.612525Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:25.612524Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1208
2025-10-09T21:48:25.612535Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1208
2025-10-09T21:48:25.612558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342179301933340:9229]
2025-10-09T21:48:25.612570Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:25.612572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:25.612599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:25.612605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:25.612649Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|52d94ef3-88d2b9d-e9443311-8084c942_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:25.612667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:25.612671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:25.612674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:25.612685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:25.612697Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:25.612717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:25.612904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:25.612921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:25.612944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:25.613041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1209 partition: 0 MaxSeqNo: 25 sessionId: src_id|52d94ef3-88d2b9d-e9443311-8084c942_0
2025-10-09T21:48:25.613397Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fade4ef9-1bb78b23-893cfc20-d6499384_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|52d94ef3-88d2b9d-e9443311-8084c942_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:25.616004Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52d94ef3-88d2b9d-e9443311-8084c942_0] Write session: try to update token
2025-10-09T21:48:25.616025Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52d94ef3-88d2b9d-e9443311-8084c942_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:25.627058Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:25.627080Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:25.627083Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:25.627085Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:25.627095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342179301933340:9229]
2025-10-09T21:48:25.627100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342179301933340:9229]
2025-10-09T21:48:25.627104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342179301933340:9229]
2025-10-09T21:48:25.627148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:25.627159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.627162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:25.627173Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:25.627188Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:25.627192Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:25.627291Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:25.630007Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1209 sessionId: src_id|52d94ef3-88d2b9d-e9443311-8084c942_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:25.632360Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:25.678339Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342179663756016:5175]
2025-10-09T21:48:25.678628Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 1208, pipe: [1:7559342179663756016:5175]
2025-10-09T21:48:25.791471Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:25.791754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:25.791774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:25.791820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342179663756016:5175], now have 1 active actors on pipe
2025-10-09T21:48:25.792030Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:25.792046Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:25.792108Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:25.792185Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1209 reason: status is not ok: tablet will be restarted right now sessionId: src_id|52d94ef3-88d2b9d-e9443311-8084c942_0
2025-10-09T21:48:25.792372Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1209 sessionId: src_id|52d94ef3-88d2b9d-e9443311-8084c942_0 is DEAD
2025-10-09T21:48:25.792489Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342179663756016:5175]
2025-10-09T21:48:25.792522Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52d94ef3-88d2b9d-e9443311-8084c942_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:25.792533Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52d94ef3-88d2b9d-e9443311-8084c942_0] Write session will restart in 0.007021s
2025-10-09T21:48:25.792571Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52d94ef3-88d2b9d-e9443311-8084c942_0] Write session: Do CDS request
2025-10-09T21:48:25.792578Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52d94ef3-88d2b9d-e9443311-8084c942_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:25.800566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52d94ef3-88d2b9d-e9443311-8084c942_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:25.800679Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:25.800693Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1210
2025-10-09T21:48:25.800820Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1210 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:25.800852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1210 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:25.800863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1210 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:25.800866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:25.801333Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1210 sessionId:  describe result for acl check
2025-10-09T21:48:25.801373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:25.801381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:25.801383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:25.801388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342179663756039:5104] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:25.801959Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342179663756039:5104] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:25.804743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342179663756039:5104] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:25.808046Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342179663756039:5104] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:25.808060Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342179663756039:5104] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:25.808063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342179663756039:5104] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:25.808072Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1210 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:25.813379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:25.816101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:25.816260Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:25.816268Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:25.816277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:25.816315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:25.816352Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342179301933634:9233]
2025-10-09T21:48:25.816754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:25.817075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:25.817133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:25.817483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:25.817552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:25.817557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:25.817559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:25.817560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:25.817561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:25.817562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:25.817563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.817565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.817566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.817572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:25.817583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:25.817587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:25.817591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:25.817595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:25.817599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:25.817603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.817607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.817611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.817619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:25.817629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:25.817635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:25.817641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:25.817645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:25.817650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:25.817655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:25.817660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:25.817670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:25.817675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:25.817679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:25.817689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:25.817694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.817699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.817708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.817714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.817737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.817744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.817787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:25.824224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:25.825523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:25.825541Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:25.825544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:25.825549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:25.825556Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1209 [2:7559342179301933634:9233]
2025-10-09T21:48:25.825573Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:25.825592Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:25.825601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:25.825604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:25.825607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:25.825626Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:25.825637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:25.825644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:25.825707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.825714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.825724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.825728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:25.825811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:25.825821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.825835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:25.825848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:25.825868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:25.825895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342179663756183:5104], now have 1 active actors on pipe
2025-10-09T21:48:25.825903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342179663756018:2432], now have 1 active actors on pipe
2025-10-09T21:48:25.825912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342179301933629:9233]
2025-10-09T21:48:25.825914Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342179301933629:9233]
2025-10-09T21:48:25.825917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342179301933629:9233]
2025-10-09T21:48:25.825930Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:25.825949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:25.825956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:25.825960Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:25.825963Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:25.825971Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:25.825974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:25.825977Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1209
2025-10-09T21:48:25.825984Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1209
2025-10-09T21:48:25.826012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342179301933629:9233]
2025-10-09T21:48:25.826021Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:25.826024Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:25.826069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:25.826079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:25.826116Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e604e8ae-f96831f1-49ed1a93-beb91136_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:25.826127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:25.826131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:25.826134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:25.826145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:25.826160Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:25.826187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:25.826310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:25.826319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:25.826334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:25.826427Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1210 partition: 0 MaxSeqNo: 25 sessionId: src_id|e604e8ae-f96831f1-49ed1a93-beb91136_0
2025-10-09T21:48:25.826814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52d94ef3-88d2b9d-e9443311-8084c942_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e604e8ae-f96831f1-49ed1a93-beb91136_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:25.829646Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e604e8ae-f96831f1-49ed1a93-beb91136_0] Write session: try to update token
2025-10-09T21:48:25.829671Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e604e8ae-f96831f1-49ed1a93-beb91136_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:25.840760Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:25.840783Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:25.840786Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:25.840788Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:25.840799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342179301933629:9233]
2025-10-09T21:48:25.840804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342179301933629:9233]
2025-10-09T21:48:25.840808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342179301933629:9233]
2025-10-09T21:48:25.840854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:25.840856Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:25.840863Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:25.840867Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:25.840868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:25.840871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:25.841008Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:25.843663Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1210 sessionId: src_id|e604e8ae-f96831f1-49ed1a93-beb91136_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:25.845939Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:25.892865Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342179663756314:5175]
2025-10-09T21:48:25.893105Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 1209, pipe: [1:7559342179663756314:5175]
2025-10-09T21:48:25.991578Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:25.991823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:25.991838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:25.991882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342179663756314:5175], now have 1 active actors on pipe
2025-10-09T21:48:25.992058Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:25.992072Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:25.992073Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:25.992120Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342179663756314:5175]
2025-10-09T21:48:25.992135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1210 reason: error at write: Disconnected sessionId: src_id|e604e8ae-f96831f1-49ed1a93-beb91136_0
2025-10-09T21:48:25.992292Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:25.992300Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1210 sessionId: src_id|e604e8ae-f96831f1-49ed1a93-beb91136_0 is DEAD
2025-10-09T21:48:25.992463Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e604e8ae-f96831f1-49ed1a93-beb91136_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:25.992475Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e604e8ae-f96831f1-49ed1a93-beb91136_0] Write session will restart in 0.006288s
2025-10-09T21:48:25.992501Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e604e8ae-f96831f1-49ed1a93-beb91136_0] Write session: Do CDS request
2025-10-09T21:48:25.992508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e604e8ae-f96831f1-49ed1a93-beb91136_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:25.999541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e604e8ae-f96831f1-49ed1a93-beb91136_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:25.999601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:25.999616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1211
2025-10-09T21:48:25.999784Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1211 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:25.999836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1211 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:25.999850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1211 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:25.999854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:26.000337Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1211 sessionId:  describe result for acl check
2025-10-09T21:48:26.000371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:26.000378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:26.000379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:26.000385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342183958723636:5102] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:26.000979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342183958723636:5102] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:26.004496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342183958723636:5102] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:26.007726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342183958723636:5102] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:26.007742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342183958723636:5102] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:26.007746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342183958723636:5102] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:26.007756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1211 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:26.012692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:26.015822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:26.016263Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:26.016278Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:26.016297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:26.016377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:26.016429Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342183596901215:9235]
2025-10-09T21:48:26.017033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:26.017361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:26.017408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:26.017748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:26.017805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.017813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.017814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.017815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.017816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.017818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.017819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.017820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.017821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.017827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.017839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.017843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.017847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.017850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.017855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.017859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.017863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.017867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.017875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.017883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.017890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.017895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.017901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.017907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.017911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.017917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.017921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.017927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.017933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.017938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.017942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.017948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.017953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.017958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.017962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.017966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.017996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:26.024559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:26.025995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:26.026011Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:26.026014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:26.026017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:26.026025Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1210 [2:7559342183596901215:9235]
2025-10-09T21:48:26.026055Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:26.026077Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:26.026080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:26.026082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:26.026084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:26.026099Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:26.026103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:26.026109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:26.026173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.026182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.026184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.026187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:26.026253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:26.026260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.026269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.026275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.026285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:26.026308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342183596901210:9235]
2025-10-09T21:48:26.026310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342183596901210:9235]
2025-10-09T21:48:26.026311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342183596901210:9235]
2025-10-09T21:48:26.026318Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:26.026329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:26.026331Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:26.026331Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:26.026334Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:26.026340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:26.026341Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:26.026352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342183958723796:5102], now have 1 active actors on pipe
2025-10-09T21:48:26.026354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342179663756318:2432], now have 1 active actors on pipe
2025-10-09T21:48:26.026385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342183596901210:9235]
2025-10-09T21:48:26.026395Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:26.026403Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:26.026541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:26.026555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:26.026584Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a9e970f0-388ceac2-41cd5b2a-aebf7a0c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:26.026592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:26.026596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:26.026599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:26.026612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:26.026453Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1210
2025-10-09T21:48:26.026546Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1210
2025-10-09T21:48:26.026628Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:26.026649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:26.026820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:26.026829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:26.026844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:26.026954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1211 partition: 0 MaxSeqNo: 25 sessionId: src_id|a9e970f0-388ceac2-41cd5b2a-aebf7a0c_0
2025-10-09T21:48:26.027335Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e604e8ae-f96831f1-49ed1a93-beb91136_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a9e970f0-388ceac2-41cd5b2a-aebf7a0c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:26.029773Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9e970f0-388ceac2-41cd5b2a-aebf7a0c_0] Write session: try to update token
2025-10-09T21:48:26.029790Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9e970f0-388ceac2-41cd5b2a-aebf7a0c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:26.038200Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:26.038223Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:26.038238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342183596901210:9235]
2025-10-09T21:48:26.038280Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:26.038287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:26.038466Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:26.038612Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:26.038758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:26.038897Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:26.039087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:26.039130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:26.043437Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1211 sessionId: src_id|a9e970f0-388ceac2-41cd5b2a-aebf7a0c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:26.045773Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:26.045996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:26.046019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:26.046077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:26.046176Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:26.052357Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:26.052375Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:26.052376Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:26.052378Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:26.052387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342183596901210:9235]
2025-10-09T21:48:26.052391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342183596901210:9235]
2025-10-09T21:48:26.052394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342183596901210:9235]
2025-10-09T21:48:26.052436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:26.052442Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:26.052445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.052448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:26.052451Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:26.052455Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:26.052622Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:26.092557Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342183958723908:5175]
2025-10-09T21:48:26.092902Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 1210, pipe: [1:7559342183958723908:5175]
2025-10-09T21:48:26.207016Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:26.215507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:26.215539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:26.215712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:26.215779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:26.215850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:26.215932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:26.215980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:26.215989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:26.216058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:26.216119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:26.216184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:26.216246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:26.216287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:26.216292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:26.216351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:26.216412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:26.216471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:26.216529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:26.216573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:26.216579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:26.216644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:26.216915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:26.217023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:26.217114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:26.217161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:26.217167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:26.217250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:26.217309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:26.217367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:26.217426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:26.217468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:26.217475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:26.217536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:26.217595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:26.217654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:26.217712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:26.217770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:26.217778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:26.217840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:26.217909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:26.217970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:26.218051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:26.218104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:26.218115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:26.218176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:26.218232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:26.218293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:26.218350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:26.218396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:26.218401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:26.218457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:26.218511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:26.218569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:26.218629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:26.218671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:26.218677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:26.218736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:26.218792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:26.218860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:26.218927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:26.218995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:26.219001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:26.219053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342183958723908:5175], now have 1 active actors on pipe
2025-10-09T21:48:26.219272Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:26.219289Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:26.219297Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:26.219322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1211 reason: error at write: Disconnected sessionId: src_id|a9e970f0-388ceac2-41cd5b2a-aebf7a0c_0
2025-10-09T21:48:26.219350Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342183958723908:5175]
2025-10-09T21:48:26.219535Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1211 sessionId: src_id|a9e970f0-388ceac2-41cd5b2a-aebf7a0c_0 is DEAD
2025-10-09T21:48:26.219671Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9e970f0-388ceac2-41cd5b2a-aebf7a0c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:26.219682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9e970f0-388ceac2-41cd5b2a-aebf7a0c_0] Write session will restart in 0.007588s
2025-10-09T21:48:26.219677Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:26.219706Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9e970f0-388ceac2-41cd5b2a-aebf7a0c_0] Write session: Do CDS request
2025-10-09T21:48:26.219713Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9e970f0-388ceac2-41cd5b2a-aebf7a0c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:26.228560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9e970f0-388ceac2-41cd5b2a-aebf7a0c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:26.228644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:26.228660Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1212
2025-10-09T21:48:26.228817Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1212 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:26.228851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1212 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:26.228861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1212 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:26.228865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:26.229310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1212 sessionId:  describe result for acl check
2025-10-09T21:48:26.229352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:26.229353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:26.229355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:26.229362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342183958723951:5084] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:26.229957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342183958723951:5084] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:26.233186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342183958723951:5084] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:26.236685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342183958723951:5084] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:26.236702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342183958723951:5084] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:26.236707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342183958723951:5084] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:26.236717Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1212 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:26.241674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:26.244494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:26.244886Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:26.244899Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:26.244914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:26.244974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:26.245009Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342183596901501:9237]
2025-10-09T21:48:26.245515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:26.245856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:26.245912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:26.246263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:26.246316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.246322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.246324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.246325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.246326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.246328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.246329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.246330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.246331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.246336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.246347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.246351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.246356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.246360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.246365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.246374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.246378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.246385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.246393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.246408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.246415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.246420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.246425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.246431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.246435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.246440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.246445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.246450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.246460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.246470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.246480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.246485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.246494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.246499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.246504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.246510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.246546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:26.252961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:26.254428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:26.254443Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:26.254445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:26.254448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:26.254455Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1211 [2:7559342183596901501:9237]
2025-10-09T21:48:26.254473Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:26.254492Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:26.254498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:26.254501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:26.254502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:26.254517Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:26.254528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:26.254536Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:26.254596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.254604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.254606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.254609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:26.254681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:26.254689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.254701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.254717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.254740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:26.254770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342183958723919:2432], now have 1 active actors on pipe
2025-10-09T21:48:26.254773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342183958724113:5084], now have 1 active actors on pipe
2025-10-09T21:48:26.254782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342183596901496:9237]
2025-10-09T21:48:26.254785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342183596901496:9237]
2025-10-09T21:48:26.254787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342183596901496:9237]
2025-10-09T21:48:26.254800Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:26.254815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:26.254823Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:26.254828Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:26.254829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:26.254831Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:26.254832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:26.254835Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1211
2025-10-09T21:48:26.254872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342183596901496:9237]
2025-10-09T21:48:26.254883Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:26.254884Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:26.254856Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1211
2025-10-09T21:48:26.254916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:26.254921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:26.254960Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a40d7df3-526c066f-1e1e4791-b7ebb12c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:26.254974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:26.254983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:26.254987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:26.254998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:26.255014Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:26.255034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:26.255199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:26.255207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:26.255223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:26.255322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1212 partition: 0 MaxSeqNo: 25 sessionId: src_id|a40d7df3-526c066f-1e1e4791-b7ebb12c_0
2025-10-09T21:48:26.255692Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9e970f0-388ceac2-41cd5b2a-aebf7a0c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a40d7df3-526c066f-1e1e4791-b7ebb12c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:26.258319Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a40d7df3-526c066f-1e1e4791-b7ebb12c_0] Write session: try to update token
2025-10-09T21:48:26.258339Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a40d7df3-526c066f-1e1e4791-b7ebb12c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:26.269601Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:26.269622Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:26.269625Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:26.269627Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:26.269636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342183596901496:9237]
2025-10-09T21:48:26.269642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342183596901496:9237]
2025-10-09T21:48:26.269647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342183596901496:9237]
2025-10-09T21:48:26.269694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:26.269696Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:26.269703Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:26.269707Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:26.269707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.269710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:26.269861Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:26.272012Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1212 sessionId: src_id|a40d7df3-526c066f-1e1e4791-b7ebb12c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:26.274501Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:26.319365Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342183958724214:5175]
2025-10-09T21:48:26.319654Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 1211, pipe: [1:7559342183958724214:5175]
2025-10-09T21:48:26.426337Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:26.426776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:26.426793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:26.426838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342183958724214:5175], now have 1 active actors on pipe
2025-10-09T21:48:26.427059Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:26.427073Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:26.427081Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342183958724214:5175]
2025-10-09T21:48:26.427091Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:26.427103Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:26.427155Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1212 reason: error at write: Disconnected sessionId: src_id|a40d7df3-526c066f-1e1e4791-b7ebb12c_0
2025-10-09T21:48:26.427334Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1212 sessionId: src_id|a40d7df3-526c066f-1e1e4791-b7ebb12c_0 is DEAD
2025-10-09T21:48:26.427485Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a40d7df3-526c066f-1e1e4791-b7ebb12c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:26.427497Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a40d7df3-526c066f-1e1e4791-b7ebb12c_0] Write session will restart in 0.009279s
2025-10-09T21:48:26.427526Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a40d7df3-526c066f-1e1e4791-b7ebb12c_0] Write session: Do CDS request
2025-10-09T21:48:26.427532Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a40d7df3-526c066f-1e1e4791-b7ebb12c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:26.437530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a40d7df3-526c066f-1e1e4791-b7ebb12c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:26.437595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:26.437613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1213
2025-10-09T21:48:26.437756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1213 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:26.437793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1213 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:26.437804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1213 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:26.437813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:26.438313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1213 sessionId:  describe result for acl check
2025-10-09T21:48:26.438347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:26.438349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:26.438351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:26.438356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342183958724274:5087] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:26.438769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342183958724274:5087] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:26.442009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342183958724274:5087] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:26.445083Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342183958724274:5087] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:26.445102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342183958724274:5087] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:26.445107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342183958724274:5087] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:26.445114Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1213 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:26.448422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:26.450954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:26.451257Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:26.451269Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:26.451283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:26.451350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:26.451394Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342183596901818:9249]
2025-10-09T21:48:26.452030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:26.452360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:26.452407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:26.452765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:26.452815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.452821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.452823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.452824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.452825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.452827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.452828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.452829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.452831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.452836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.452846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.452851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.452855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.452859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.452864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.452868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.452872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.452875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.452882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.452894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.452906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.452916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.452925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.452930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.452936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.452941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.452950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.452956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.452966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.452972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.452977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.452983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.452988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.452994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.452998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.453004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.453039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:26.459456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:26.460670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:26.460684Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:26.460686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:26.460691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:26.460697Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1212 [2:7559342183596901818:9249]
2025-10-09T21:48:26.460711Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:26.460730Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:26.460733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:26.460736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:26.460739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:26.460754Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:26.460765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:26.460772Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:26.460814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.460822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.460824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.460827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:26.460894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:26.460901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.460914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.460925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.460935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:26.460955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342183958724461:5087], now have 1 active actors on pipe
2025-10-09T21:48:26.460967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342183596901813:9249]
2025-10-09T21:48:26.460973Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342183596901813:9249]
2025-10-09T21:48:26.460975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342183596901813:9249]
2025-10-09T21:48:26.460987Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:26.461000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342183958724219:2432], now have 1 active actors on pipe
2025-10-09T21:48:26.461002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:26.461005Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:26.461007Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:26.461009Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:26.461024Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:26.461032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:26.461060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342183596901813:9249]
2025-10-09T21:48:26.461063Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:26.461070Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:26.461048Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1212
2025-10-09T21:48:26.461087Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1212
2025-10-09T21:48:26.461092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:26.461101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:26.461134Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dadf745b-f9d3d5b8-d12b2972-30a8583b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:26.461146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:26.461150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:26.461157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:26.461169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:26.461183Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:26.461202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:26.461356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:26.461366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:26.461383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:26.461594Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1213 partition: 0 MaxSeqNo: 25 sessionId: src_id|dadf745b-f9d3d5b8-d12b2972-30a8583b_0
2025-10-09T21:48:26.461982Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a40d7df3-526c066f-1e1e4791-b7ebb12c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dadf745b-f9d3d5b8-d12b2972-30a8583b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:26.464418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dadf745b-f9d3d5b8-d12b2972-30a8583b_0] Write session: try to update token
2025-10-09T21:48:26.464433Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dadf745b-f9d3d5b8-d12b2972-30a8583b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:26.475532Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:26.475552Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:26.475554Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:26.475556Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:26.475565Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342183596901813:9249]
2025-10-09T21:48:26.475569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342183596901813:9249]
2025-10-09T21:48:26.475573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342183596901813:9249]
2025-10-09T21:48:26.475621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:26.475630Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:26.475632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.475635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:26.475636Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:26.475640Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:26.475764Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:26.477510Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1213 sessionId: src_id|dadf745b-f9d3d5b8-d12b2972-30a8583b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:26.479733Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:26.527232Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342183958724535:5175]
2025-10-09T21:48:26.527507Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 1212, pipe: [1:7559342183958724535:5175]
2025-10-09T21:48:26.632015Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:26.632449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:26.632470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:26.632540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342183958724535:5175], now have 1 active actors on pipe
2025-10-09T21:48:26.632761Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342183958724535:5175]
2025-10-09T21:48:26.632764Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:26.632789Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:26.632792Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:26.632847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1213 reason: error at write: Disconnected sessionId: src_id|dadf745b-f9d3d5b8-d12b2972-30a8583b_0
2025-10-09T21:48:26.633002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1213 sessionId: src_id|dadf745b-f9d3d5b8-d12b2972-30a8583b_0 is DEAD
2025-10-09T21:48:26.633210Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dadf745b-f9d3d5b8-d12b2972-30a8583b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:26.633222Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dadf745b-f9d3d5b8-d12b2972-30a8583b_0] Write session will restart in 0.008909s
2025-10-09T21:48:26.633247Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dadf745b-f9d3d5b8-d12b2972-30a8583b_0] Write session: Do CDS request
2025-10-09T21:48:26.633256Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dadf745b-f9d3d5b8-d12b2972-30a8583b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:26.634511Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:26.643558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dadf745b-f9d3d5b8-d12b2972-30a8583b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:26.643634Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:26.643650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1214
2025-10-09T21:48:26.643807Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1214 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:26.643852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1214 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:26.643863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1214 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:26.643867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:26.644379Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1214 sessionId:  describe result for acl check
2025-10-09T21:48:26.644417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:26.644424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:26.644426Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:26.644432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342183958724588:5073] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:26.644888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342183958724588:5073] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:26.648232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342183958724588:5073] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:26.651861Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342183958724588:5073] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:26.651873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342183958724588:5073] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:26.651878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342183958724588:5073] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:26.651886Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1214 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:26.656592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:26.659435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:26.659659Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:26.659671Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:26.659683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:26.659735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:26.659777Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342183596902119:9254]
2025-10-09T21:48:26.660371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:26.660681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:26.660725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:26.661041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:26.661090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.661096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.661098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.661099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.661100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.661102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.661102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.661104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.661105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.661110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.661121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.661126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.661130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.661134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.661139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.661143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.661146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.661150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.661158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.661165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.661176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.661182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.661186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.661192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.661196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.661202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.661207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.661212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.661218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.661228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.661233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.661239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.661244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.661249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.661255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.661260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.661288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:26.667729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:26.669133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:26.669153Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:26.669155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:26.669160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:26.669166Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1213 [2:7559342183596902119:9254]
2025-10-09T21:48:26.669180Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:26.669199Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:26.669202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:26.669206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:26.669208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:26.669225Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:26.669238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:26.669245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:26.669303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.669311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.669313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.669316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:26.669387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:26.669395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.669406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.669422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.669439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:26.669475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342183958724537:2432], now have 1 active actors on pipe
2025-10-09T21:48:26.669483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342183958724742:5073], now have 1 active actors on pipe
2025-10-09T21:48:26.669491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342183596902114:9254]
2025-10-09T21:48:26.669493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342183596902114:9254]
2025-10-09T21:48:26.669495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342183596902114:9254]
2025-10-09T21:48:26.669508Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:26.669523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:26.669528Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:26.669532Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:26.669535Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:26.669541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:26.669544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:26.669517Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1213
2025-10-09T21:48:26.669546Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1213
2025-10-09T21:48:26.669572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342183596902114:9254]
2025-10-09T21:48:26.669580Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:26.669582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:26.669601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:26.669610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:26.669656Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|29ea3c91-89f4f716-cee99eff-48f62aea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:26.669669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:26.669673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:26.669676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:26.669686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:26.669697Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:26.669734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:26.669894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:26.669903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:26.669920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:26.670010Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1214 partition: 0 MaxSeqNo: 25 sessionId: src_id|29ea3c91-89f4f716-cee99eff-48f62aea_0
2025-10-09T21:48:26.670379Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dadf745b-f9d3d5b8-d12b2972-30a8583b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|29ea3c91-89f4f716-cee99eff-48f62aea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:26.672887Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29ea3c91-89f4f716-cee99eff-48f62aea_0] Write session: try to update token
2025-10-09T21:48:26.672906Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29ea3c91-89f4f716-cee99eff-48f62aea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:26.685380Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:26.685412Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:26.685415Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:26.685418Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:26.685432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342183596902114:9254]
2025-10-09T21:48:26.685439Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342183596902114:9254]
2025-10-09T21:48:26.685447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342183596902114:9254]
2025-10-09T21:48:26.685500Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:26.685507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:26.685515Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:26.685517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.685521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:26.685522Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:26.685797Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:26.686752Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1214 sessionId: src_id|29ea3c91-89f4f716-cee99eff-48f62aea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:26.689085Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:26.733072Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342183958724838:5175]
2025-10-09T21:48:26.733294Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 1213, pipe: [1:7559342183958724838:5175]
2025-10-09T21:48:26.841542Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:26.842063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:26.842079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:26.842126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342183958724838:5175], now have 1 active actors on pipe
2025-10-09T21:48:26.842337Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:26.842367Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:26.842404Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:26.842423Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342183958724838:5175]
2025-10-09T21:48:26.842493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1214 reason: status is not ok: tablet will be restarted right now sessionId: src_id|29ea3c91-89f4f716-cee99eff-48f62aea_0
2025-10-09T21:48:26.842681Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1214 sessionId: src_id|29ea3c91-89f4f716-cee99eff-48f62aea_0 is DEAD
2025-10-09T21:48:26.842840Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29ea3c91-89f4f716-cee99eff-48f62aea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:26.842851Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29ea3c91-89f4f716-cee99eff-48f62aea_0] Write session will restart in 0.006014s
2025-10-09T21:48:26.842877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29ea3c91-89f4f716-cee99eff-48f62aea_0] Write session: Do CDS request
2025-10-09T21:48:26.842883Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29ea3c91-89f4f716-cee99eff-48f62aea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:26.850493Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29ea3c91-89f4f716-cee99eff-48f62aea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:26.850754Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:26.850767Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1215
2025-10-09T21:48:26.850896Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1215 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:26.850928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1215 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:26.850940Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1215 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:26.850943Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:26.851714Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1215 sessionId:  describe result for acl check
2025-10-09T21:48:26.851760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:26.851767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:26.851771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:26.851777Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342183958724867:5060] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:26.852301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342183958724867:5060] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:26.855205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342183958724867:5060] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:26.857962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342183958724867:5060] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:26.857978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342183958724867:5060] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:26.857982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342183958724867:5060] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:26.857996Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1215 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:26.864301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:26.867195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:26.867402Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:26.867412Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:26.867423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:26.867477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:26.867516Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342183596902409:9257]
2025-10-09T21:48:26.868169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:26.868622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:26.868685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:26.869030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:26.869090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.869097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.869098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.869099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.869100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.869102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.869103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.869105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.869106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.869112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.869123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.869127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.869132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.869136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.869140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.869144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.869148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.869151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.869161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.869169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:26.869175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.869180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:26.869185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.869190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:26.869194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.869199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:26.869204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.869209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:26.869219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.869225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:26.869229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.869234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.869239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.869250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.869255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.869260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.869289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:26.875742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:26.877122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:26.877137Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:26.877139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:26.877144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:26.877150Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1214 [2:7559342183596902409:9257]
2025-10-09T21:48:26.877173Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:26.877191Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:26.877194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:26.877197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:26.877199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:26.877215Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:26.877227Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:26.877234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:26.877292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.877301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.877302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.877305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:26.877370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:26.877379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.877391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:26.877406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:26.877426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:26.877453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342183958724993:5060], now have 1 active actors on pipe
2025-10-09T21:48:26.877462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342183958724843:2432], now have 1 active actors on pipe
2025-10-09T21:48:26.877469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342183596902404:9257]
2025-10-09T21:48:26.877477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342183596902404:9257]
2025-10-09T21:48:26.877479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342183596902404:9257]
2025-10-09T21:48:26.877494Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:26.877512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:26.877511Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:26.877515Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:26.877517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:26.877533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:26.877537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:26.877506Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1214
2025-10-09T21:48:26.877533Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1214
2025-10-09T21:48:26.877568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342183596902404:9257]
2025-10-09T21:48:26.877576Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:26.877578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:26.877597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:26.877606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:26.877655Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7dbc978-7326cf7f-152e2cbe-96a22084_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:26.877669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:26.877679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:26.877682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:26.877693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:26.877708Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:26.877744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:26.877929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:26.877937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:26.877952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:26.878058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1215 partition: 0 MaxSeqNo: 25 sessionId: src_id|7dbc978-7326cf7f-152e2cbe-96a22084_0
2025-10-09T21:48:26.878431Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29ea3c91-89f4f716-cee99eff-48f62aea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7dbc978-7326cf7f-152e2cbe-96a22084_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:26.880862Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7dbc978-7326cf7f-152e2cbe-96a22084_0] Write session: try to update token
2025-10-09T21:48:26.880881Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7dbc978-7326cf7f-152e2cbe-96a22084_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:26.891980Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:26.892002Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:26.892005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:26.892007Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:26.892017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342183596902404:9257]
2025-10-09T21:48:26.892022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342183596902404:9257]
2025-10-09T21:48:26.892026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342183596902404:9257]
2025-10-09T21:48:26.892074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:26.892075Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:26.892081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:26.892084Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:26.892084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:26.892087Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:26.892214Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:26.893693Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1215 sessionId: src_id|7dbc978-7326cf7f-152e2cbe-96a22084_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:26.896222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:26.942446Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342183958725141:5175]
2025-10-09T21:48:26.942805Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 1214, pipe: [1:7559342183958725141:5175]
2025-10-09T21:48:27.051557Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:27.052031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.052050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.052097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342183958725141:5175], now have 1 active actors on pipe
2025-10-09T21:48:27.052297Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:27.052307Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:27.052358Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:27.052361Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342183958725141:5175]
2025-10-09T21:48:27.052430Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1215 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7dbc978-7326cf7f-152e2cbe-96a22084_0
2025-10-09T21:48:27.052598Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1215 sessionId: src_id|7dbc978-7326cf7f-152e2cbe-96a22084_0 is DEAD
2025-10-09T21:48:27.052736Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7dbc978-7326cf7f-152e2cbe-96a22084_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:27.052748Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7dbc978-7326cf7f-152e2cbe-96a22084_0] Write session will restart in 0.009051s
2025-10-09T21:48:27.052774Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7dbc978-7326cf7f-152e2cbe-96a22084_0] Write session: Do CDS request
2025-10-09T21:48:27.052782Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7dbc978-7326cf7f-152e2cbe-96a22084_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:27.062613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7dbc978-7326cf7f-152e2cbe-96a22084_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:27.062714Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:27.062744Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1216
2025-10-09T21:48:27.062856Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1216 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:27.062889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1216 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:27.062909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1216 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:27.062913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:27.063454Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1216 sessionId:  describe result for acl check
2025-10-09T21:48:27.063494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:27.063496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:27.063497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:27.063503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342188253692480:5049] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:27.064026Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342188253692480:5049] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:27.067113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342188253692480:5049] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:27.070465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342188253692480:5049] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:27.070481Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342188253692480:5049] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:27.070484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342188253692480:5049] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:27.070493Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1216 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:27.074388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:27.077362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:27.077571Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:27.077581Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:27.077593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:27.077647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:27.077686Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342187891869995:9259]
2025-10-09T21:48:27.078158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:27.078439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:27.078481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:27.078781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:27.078832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.078838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.078839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.078840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.078842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.078843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.078844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.078845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.078846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.078852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.078863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.078867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.078871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.078875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.078879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.078884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.078888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.078891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.078899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.078907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.078914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.078920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.078924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.078929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.078934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.078947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.078952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.078961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.078966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.078972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.078981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.078988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.078996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.079001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.079006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.079010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.079040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:27.085476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:27.086632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:27.086652Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:27.086654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:27.086659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:27.086668Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1215 [2:7559342187891869995:9259]
2025-10-09T21:48:27.086683Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:27.086704Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:27.086709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:27.086712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:27.086715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:27.086732Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:27.086743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:27.086751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:27.086814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.086821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.086823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.086826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:27.087017Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1215
2025-10-09T21:48:27.087026Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1215
2025-10-09T21:48:27.087222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:27.087230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.087239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.087251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.087262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:27.087282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342188253692666:5049], now have 1 active actors on pipe
2025-10-09T21:48:27.087292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342188253692441:2432], now have 1 active actors on pipe
2025-10-09T21:48:27.087307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.087311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.087330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342187891869990:9259]
2025-10-09T21:48:27.087331Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342187891869990:9259]
2025-10-09T21:48:27.087332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342187891869990:9259]
2025-10-09T21:48:27.087343Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:27.087354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:27.087360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:27.087363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:27.087365Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:27.087367Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:27.087369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:27.087384Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|26f56336-aa7181b4-76e172f3-c3abf712_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:27.087390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:27.087397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:27.087399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:27.087406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:27.087414Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:27.087444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342187891869990:9259]
2025-10-09T21:48:27.087450Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:27.087459Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:27.087464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:27.087621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.087629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.087643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:27.087724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1216 partition: 0 MaxSeqNo: 25 sessionId: src_id|26f56336-aa7181b4-76e172f3-c3abf712_0
2025-10-09T21:48:27.088094Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7dbc978-7326cf7f-152e2cbe-96a22084_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|26f56336-aa7181b4-76e172f3-c3abf712_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:27.090546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26f56336-aa7181b4-76e172f3-c3abf712_0] Write session: try to update token
2025-10-09T21:48:27.090564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26f56336-aa7181b4-76e172f3-c3abf712_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:27.103888Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1216 sessionId: src_id|26f56336-aa7181b4-76e172f3-c3abf712_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:27.105099Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:27.105119Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:27.105133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342187891869990:9259]
2025-10-09T21:48:27.105189Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:27.105190Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:27.105406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:27.105559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:27.105722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:27.105872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:27.106096Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:27.106141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:27.106273Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:27.106471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.106494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.106542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:27.106658Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:27.112775Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:27.112793Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:27.112796Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:27.112798Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:27.112808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342187891869990:9259]
2025-10-09T21:48:27.112812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342187891869990:9259]
2025-10-09T21:48:27.112816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342187891869990:9259]
2025-10-09T21:48:27.112856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:27.112862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.112865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:27.112865Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:27.112871Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:27.112875Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:27.113005Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:27.152386Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342188253692743:5175]
2025-10-09T21:48:27.152645Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 1215, pipe: [1:7559342188253692743:5175]
2025-10-09T21:48:27.274332Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:27.282429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.282456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.282622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.282691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.282761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.282826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.282872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.282882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.282950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.283011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.283076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.283143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.283189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.283196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.283252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.283313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.283377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.283465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.283507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.283515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.283594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.283662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.283727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.283807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.283855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.283863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.283935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.283997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.284064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.284136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.284181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.284189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.284254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.284313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.284380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.284445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.284559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.284569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.284635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.284704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.284780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.284845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.284972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.284983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.285045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.285101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.285169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.285234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.285294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.285302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.285362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.285421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.285483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.285548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.285592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.285599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.285666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.285737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.285798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.285866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.285914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.285921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.285986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342188253692743:5175], now have 1 active actors on pipe
2025-10-09T21:48:27.286251Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342188253692743:5175]
2025-10-09T21:48:27.286279Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:27.286283Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:27.286285Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:27.286325Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1216 reason: error at write: Disconnected sessionId: src_id|26f56336-aa7181b4-76e172f3-c3abf712_0
2025-10-09T21:48:27.286538Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1216 sessionId: src_id|26f56336-aa7181b4-76e172f3-c3abf712_0 is DEAD
2025-10-09T21:48:27.286660Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26f56336-aa7181b4-76e172f3-c3abf712_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:27.286670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26f56336-aa7181b4-76e172f3-c3abf712_0] Write session will restart in 0.008052s
2025-10-09T21:48:27.286696Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26f56336-aa7181b4-76e172f3-c3abf712_0] Write session: Do CDS request
2025-10-09T21:48:27.286703Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26f56336-aa7181b4-76e172f3-c3abf712_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:27.286965Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:27.295535Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26f56336-aa7181b4-76e172f3-c3abf712_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:27.295628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:27.295643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1217
2025-10-09T21:48:27.295788Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1217 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:27.295833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1217 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:27.295844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1217 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:27.295848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:27.296271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1217 sessionId:  describe result for acl check
2025-10-09T21:48:27.296314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:27.296320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:27.296322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:27.296327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342188253692789:5047] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:27.296940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342188253692789:5047] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:27.300291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342188253692789:5047] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:27.303753Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342188253692789:5047] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:27.303768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342188253692789:5047] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:27.303771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342188253692789:5047] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:27.303779Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1217 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:27.308548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:27.311169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:27.311357Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:27.311367Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:27.311378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:27.311428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:27.311469Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342187891870286:9261]
2025-10-09T21:48:27.312084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:27.312395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:27.312439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:27.312775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:27.312834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.312840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.312842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.312843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.312844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.312845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.312846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.312848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.312849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.312855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.312865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.312870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.312873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.312877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.312882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.312886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.312890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.312892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.312899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.312906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.312914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.312919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.312923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.312927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.312931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.312936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.312940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.312945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.312949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.312954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.312960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.312965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.312969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.312974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.312978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.312992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.313019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:27.319537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:27.320589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:27.320604Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:27.320605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:27.320609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:27.320615Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1216 [2:7559342187891870286:9261]
2025-10-09T21:48:27.320626Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:27.320642Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:27.320644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:27.320646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:27.320648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:27.320661Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:27.320665Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:27.320675Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:27.320716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.320723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.320725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.320728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:27.320792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:27.320798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.320805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.320812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.320822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:27.320840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342187891870281:9261]
2025-10-09T21:48:27.320845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342187891870281:9261]
2025-10-09T21:48:27.320846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342187891870281:9261]
2025-10-09T21:48:27.320854Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:27.320863Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:27.320865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:27.320874Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:27.320874Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:27.320876Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:27.320876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:27.320885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342188253692948:5047], now have 1 active actors on pipe
2025-10-09T21:48:27.320888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342188253692751:2432], now have 1 active actors on pipe
2025-10-09T21:48:27.320909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342187891870281:9261]
2025-10-09T21:48:27.320918Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:27.320924Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:27.320963Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1216
2025-10-09T21:48:27.320967Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1216
2025-10-09T21:48:27.321027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.321041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.321069Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cf49e804-d04f5006-4601a663-6d07f2f5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:27.321080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:27.321083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:27.321085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:27.321092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:27.321100Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:27.321111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:27.321266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.321275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.321289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:27.321395Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1217 partition: 0 MaxSeqNo: 25 sessionId: src_id|cf49e804-d04f5006-4601a663-6d07f2f5_0
2025-10-09T21:48:27.321760Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26f56336-aa7181b4-76e172f3-c3abf712_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cf49e804-d04f5006-4601a663-6d07f2f5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:27.324272Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf49e804-d04f5006-4601a663-6d07f2f5_0] Write session: try to update token
2025-10-09T21:48:27.324289Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf49e804-d04f5006-4601a663-6d07f2f5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:27.335104Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:27.335126Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:27.335128Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:27.335131Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:27.335141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342187891870281:9261]
2025-10-09T21:48:27.335146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342187891870281:9261]
2025-10-09T21:48:27.335150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342187891870281:9261]
2025-10-09T21:48:27.335194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:27.335201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.335204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:27.335208Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:27.335215Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:27.335219Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:27.335332Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:27.337787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1217 sessionId: src_id|cf49e804-d04f5006-4601a663-6d07f2f5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:27.340107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:27.386372Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342188253693052:5175]
2025-10-09T21:48:27.386657Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 1216, pipe: [1:7559342188253693052:5175]
2025-10-09T21:48:27.492842Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:27.493308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.493326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.493372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342188253693052:5175], now have 1 active actors on pipe
2025-10-09T21:48:27.493566Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:27.493622Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:27.493639Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:27.493694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1217 reason: error at write: Disconnected sessionId: src_id|cf49e804-d04f5006-4601a663-6d07f2f5_0
2025-10-09T21:48:27.493880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1217 sessionId: src_id|cf49e804-d04f5006-4601a663-6d07f2f5_0 is DEAD
2025-10-09T21:48:27.494047Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf49e804-d04f5006-4601a663-6d07f2f5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:27.494058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf49e804-d04f5006-4601a663-6d07f2f5_0] Write session will restart in 0.007366s
2025-10-09T21:48:27.494056Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:27.494084Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf49e804-d04f5006-4601a663-6d07f2f5_0] Write session: Do CDS request
2025-10-09T21:48:27.494091Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf49e804-d04f5006-4601a663-6d07f2f5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:27.494095Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342188253693052:5175]
2025-10-09T21:48:27.502524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf49e804-d04f5006-4601a663-6d07f2f5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:27.502647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:27.502664Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1218
2025-10-09T21:48:27.502780Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1218 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:27.502815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1218 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:27.502829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1218 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:27.502832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:27.503220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1218 sessionId:  describe result for acl check
2025-10-09T21:48:27.503253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:27.503259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:27.503261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:27.503267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342188253693098:5035] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:27.503827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342188253693098:5035] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:27.506623Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342188253693098:5035] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:27.509723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342188253693098:5035] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:27.509738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342188253693098:5035] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:27.509741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342188253693098:5035] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:27.509749Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1218 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:27.515170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:27.517548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:27.517770Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:27.517782Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:27.517795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:27.517854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:27.517913Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342187891870609:9273]
2025-10-09T21:48:27.518553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:27.518861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:27.518908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:27.519215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:27.519265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.519272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.519273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.519274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.519275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.519276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.519277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.519278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.519279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.519284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.519295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.519299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.519303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.519307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.519311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.519315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.519318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.519322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.519329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.519337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.519343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.519349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.519353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.519359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.519363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.519368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.519372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.519377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.519383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.519388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.519394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.519399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.519401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.519404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.519407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.519410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.519435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:27.525835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:27.527307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:27.527323Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:27.527325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:27.527329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:27.527336Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1217 [2:7559342187891870609:9273]
2025-10-09T21:48:27.527351Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:27.527372Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:27.527375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:27.527379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:27.527381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:27.527399Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:27.527406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:27.527412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:27.527468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.527482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.527484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.527487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:27.527553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:27.527561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.527575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.527587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.527607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:27.527632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342188253693255:5035], now have 1 active actors on pipe
2025-10-09T21:48:27.527635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342188253693067:2432], now have 1 active actors on pipe
2025-10-09T21:48:27.527643Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342187891870604:9273]
2025-10-09T21:48:27.527650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342187891870604:9273]
2025-10-09T21:48:27.527652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342187891870604:9273]
2025-10-09T21:48:27.527665Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:27.527679Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:27.527681Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:27.527683Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:27.527692Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:27.527708Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:27.527711Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:27.527722Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1217
2025-10-09T21:48:27.527741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342187891870604:9273]
2025-10-09T21:48:27.527725Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1217
2025-10-09T21:48:27.527754Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:27.527765Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:27.527892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.527910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.527956Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1e9c22f4-c0da6b24-9a5abfbc-fa34e6e2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:27.527975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:27.527979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:27.527982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:27.527994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:27.528012Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:27.528034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:27.528245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.528259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.528280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:27.528371Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1218 partition: 0 MaxSeqNo: 25 sessionId: src_id|1e9c22f4-c0da6b24-9a5abfbc-fa34e6e2_0
2025-10-09T21:48:27.528760Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf49e804-d04f5006-4601a663-6d07f2f5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1e9c22f4-c0da6b24-9a5abfbc-fa34e6e2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:27.531290Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1e9c22f4-c0da6b24-9a5abfbc-fa34e6e2_0] Write session: try to update token
2025-10-09T21:48:27.531306Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1e9c22f4-c0da6b24-9a5abfbc-fa34e6e2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:27.539415Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:27.539438Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:27.539449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342187891870604:9273]
2025-10-09T21:48:27.539496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:27.539497Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:27.539678Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:27.539824Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:27.539951Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:27.540077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:27.540273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:27.540322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:27.544369Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1218 sessionId: src_id|1e9c22f4-c0da6b24-9a5abfbc-fa34e6e2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:27.546932Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:27.547137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.547158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.547209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:27.547288Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:27.552770Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:27.552789Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:27.552792Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:27.552794Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:27.552804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342187891870604:9273]
2025-10-09T21:48:27.552807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342187891870604:9273]
2025-10-09T21:48:27.552810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342187891870604:9273]
2025-10-09T21:48:27.552851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:27.552862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.552863Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:27.552865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:27.552869Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:27.552873Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:27.552991Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:27.594242Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342188253693378:5175]
2025-10-09T21:48:27.594510Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 1217, pipe: [1:7559342188253693378:5175]
2025-10-09T21:48:27.712537Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:27.720046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.720076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.720229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.720292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.720351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.720408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.720452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.720462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.720530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.720593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.720661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.720734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.720780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.720788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.720851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.720913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.720969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.721066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.721110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.721115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.721183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.721246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.721311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.721372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.721415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.721422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.721482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.721541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.721600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.721659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.721699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.721706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.721777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.721843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.721906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.721961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.722009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.722018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.722098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.722164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.722227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.722294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.722343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.722350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.722412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.722470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.722540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.722602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.722644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.722649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.722703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.722757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.722817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.722875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.722922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.722930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.722986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:27.723044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:27.723109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:27.723179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:27.723223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:27.723229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:27.723288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342188253693378:5175], now have 1 active actors on pipe
2025-10-09T21:48:27.723581Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:27.723591Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342188253693378:5175]
2025-10-09T21:48:27.723637Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:27.723651Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:27.723681Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1218 reason: error at write: Disconnected sessionId: src_id|1e9c22f4-c0da6b24-9a5abfbc-fa34e6e2_0
2025-10-09T21:48:27.723868Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1218 sessionId: src_id|1e9c22f4-c0da6b24-9a5abfbc-fa34e6e2_0 is DEAD
2025-10-09T21:48:27.724028Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1e9c22f4-c0da6b24-9a5abfbc-fa34e6e2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:27.724040Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1e9c22f4-c0da6b24-9a5abfbc-fa34e6e2_0] Write session will restart in 0.006317s
2025-10-09T21:48:27.724067Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1e9c22f4-c0da6b24-9a5abfbc-fa34e6e2_0] Write session: Do CDS request
2025-10-09T21:48:27.724074Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1e9c22f4-c0da6b24-9a5abfbc-fa34e6e2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:27.724276Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:27.731510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1e9c22f4-c0da6b24-9a5abfbc-fa34e6e2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:27.731834Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:27.731848Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1219
2025-10-09T21:48:27.731986Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1219 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:27.732039Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1219 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:27.732049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1219 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:27.732054Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:27.732483Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1219 sessionId:  describe result for acl check
2025-10-09T21:48:27.732524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:27.732526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:27.732528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:27.732534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342188253693397:5028] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:27.733085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342188253693397:5028] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:27.736177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342188253693397:5028] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:27.740038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342188253693397:5028] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:27.740055Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342188253693397:5028] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:27.740060Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342188253693397:5028] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:27.740069Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1219 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:27.746343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:27.749378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:27.749576Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:27.749586Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:27.749599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:27.749652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:27.749693Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342187891870914:9279]
2025-10-09T21:48:27.750356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:27.750692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:27.750748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:27.751042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:27.751095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.751100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.751101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.751101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.751102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.751103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.751104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.751104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.751105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.751109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.751116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.751119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.751121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.751123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.751125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.751127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.751130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.751132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.751137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.751143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.751147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.751151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.751153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.751156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.751159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.751162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.751164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.751167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.751170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.751173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.751176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.751179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.751181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.751184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.751187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.751190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.751207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:27.757642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:27.758735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:27.758751Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:27.758754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:27.758759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:27.758766Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1218 [2:7559342187891870914:9279]
2025-10-09T21:48:27.758784Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:27.758804Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:27.758813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:27.758817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:27.758820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:27.758838Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:27.758848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:27.758856Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:27.758909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.758915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.758917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.758920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:27.758987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:27.758994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.759006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.759024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.759047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:27.759147Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1218
2025-10-09T21:48:27.759074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342188253693545:5028], now have 1 active actors on pipe
2025-10-09T21:48:27.759083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342188253693381:2432], now have 1 active actors on pipe
2025-10-09T21:48:27.759148Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1218
2025-10-09T21:48:27.759091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342187891870909:9279]
2025-10-09T21:48:27.759094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342187891870909:9279]
2025-10-09T21:48:27.759096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342187891870909:9279]
2025-10-09T21:48:27.759111Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:27.759126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:27.759141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:27.759144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:27.759148Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:27.759153Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:27.759156Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:27.759185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342187891870909:9279]
2025-10-09T21:48:27.759188Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:27.759191Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:27.759205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.759210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.759264Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|726a3b3b-59488f9d-8afd6032-a1bb7fb5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:27.759272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:27.759282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:27.759284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:27.759295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:27.759309Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:27.759331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:27.759521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.759550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.759578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:27.759655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1219 partition: 0 MaxSeqNo: 25 sessionId: src_id|726a3b3b-59488f9d-8afd6032-a1bb7fb5_0
2025-10-09T21:48:27.760044Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1e9c22f4-c0da6b24-9a5abfbc-fa34e6e2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|726a3b3b-59488f9d-8afd6032-a1bb7fb5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:27.762508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|726a3b3b-59488f9d-8afd6032-a1bb7fb5_0] Write session: try to update token
2025-10-09T21:48:27.762527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|726a3b3b-59488f9d-8afd6032-a1bb7fb5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:27.774007Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:27.774026Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:27.774044Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:27.774046Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:27.774056Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342187891870909:9279]
2025-10-09T21:48:27.774060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342187891870909:9279]
2025-10-09T21:48:27.774064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342187891870909:9279]
2025-10-09T21:48:27.774116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:27.774120Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:27.774123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.774126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:27.774126Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:27.774130Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:27.774256Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:27.776075Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1219 sessionId: src_id|726a3b3b-59488f9d-8afd6032-a1bb7fb5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:27.778454Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:27.823610Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342188253693685:5175]
2025-10-09T21:48:27.823883Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 1218, pipe: [1:7559342188253693685:5175]
2025-10-09T21:48:27.933394Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:27.934213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.934234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.934283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342188253693685:5175], now have 1 active actors on pipe
2025-10-09T21:48:27.934484Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:27.934490Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:27.934499Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342188253693685:5175]
2025-10-09T21:48:27.934508Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:27.934523Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:27.934568Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1219 reason: error at write: Disconnected sessionId: src_id|726a3b3b-59488f9d-8afd6032-a1bb7fb5_0
2025-10-09T21:48:27.934759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1219 sessionId: src_id|726a3b3b-59488f9d-8afd6032-a1bb7fb5_0 is DEAD
2025-10-09T21:48:27.934930Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|726a3b3b-59488f9d-8afd6032-a1bb7fb5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:27.934941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|726a3b3b-59488f9d-8afd6032-a1bb7fb5_0] Write session will restart in 0.008874s
2025-10-09T21:48:27.934969Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|726a3b3b-59488f9d-8afd6032-a1bb7fb5_0] Write session: Do CDS request
2025-10-09T21:48:27.934977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|726a3b3b-59488f9d-8afd6032-a1bb7fb5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:27.944510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|726a3b3b-59488f9d-8afd6032-a1bb7fb5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:27.944646Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:27.944661Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1220
2025-10-09T21:48:27.944811Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1220 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:27.944838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1220 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:27.944845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1220 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:27.944849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:27.945448Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1220 sessionId:  describe result for acl check
2025-10-09T21:48:27.945488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:27.945496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:27.945498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:27.945504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342188253693729:5020] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:27.946078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342188253693729:5020] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:27.949352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342188253693729:5020] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:27.952776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342188253693729:5020] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:27.952792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342188253693729:5020] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:27.952796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342188253693729:5020] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:27.952804Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1220 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:27.958308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:27.961146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:27.961349Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:27.961360Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:27.961372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:27.961424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:27.961467Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342187891871208:9281]
2025-10-09T21:48:27.962105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:27.962474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:27.962521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:27.962887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:27.962937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.962943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.962945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.962946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.962947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.962949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.962950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.962951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.962953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.962958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.962969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.962973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.962978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.962981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.962985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.962989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.962992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.962996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.963003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.963012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:27.963018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.963023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:27.963033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.963038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:27.963047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.963052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:27.963057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.963066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:27.963070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.963076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:27.963085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.963092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.963101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.963105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.963110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.963119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.963148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:27.969603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:27.970725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:27.970746Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:27.970749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:27.970765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:27.970772Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1219 [2:7559342187891871208:9281]
2025-10-09T21:48:27.970785Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:27.970804Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:27.970810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:27.970813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:27.970816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:27.970833Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:27.970848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:27.970856Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:27.970914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.970921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.970924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.970926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:27.971011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:27.971019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.971030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:27.971039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:27.971054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:27.971079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342188253693688:2432], now have 1 active actors on pipe
2025-10-09T21:48:27.971089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342188253693879:5020], now have 1 active actors on pipe
2025-10-09T21:48:27.971096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342187891871203:9281]
2025-10-09T21:48:27.971098Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342187891871203:9281]
2025-10-09T21:48:27.971101Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342187891871203:9281]
2025-10-09T21:48:27.971115Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:27.971130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:27.971141Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:27.971145Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:27.971148Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:27.971150Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:27.971154Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:27.971183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342187891871203:9281]
2025-10-09T21:48:27.971196Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:27.971203Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:27.971216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.971226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.971145Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1219
2025-10-09T21:48:27.971164Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1219
2025-10-09T21:48:27.971277Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|90c85491-eda869a-d17aaae4-71851aee_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:27.971291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:27.971295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:27.971297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:27.971309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:27.971320Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:27.971337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:27.971568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:27.971577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:27.971593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:27.971802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1220 partition: 0 MaxSeqNo: 25 sessionId: src_id|90c85491-eda869a-d17aaae4-71851aee_0
2025-10-09T21:48:27.972185Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|726a3b3b-59488f9d-8afd6032-a1bb7fb5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|90c85491-eda869a-d17aaae4-71851aee_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:27.974646Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90c85491-eda869a-d17aaae4-71851aee_0] Write session: try to update token
2025-10-09T21:48:27.974664Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90c85491-eda869a-d17aaae4-71851aee_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:27.985633Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:27.985657Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:27.985660Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:27.985662Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:27.985672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342187891871203:9281]
2025-10-09T21:48:27.985677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342187891871203:9281]
2025-10-09T21:48:27.985681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342187891871203:9281]
2025-10-09T21:48:27.985736Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:27.985739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:27.985742Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:27.985745Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:27.985746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:27.985749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:27.985886Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:27.988304Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1220 sessionId: src_id|90c85491-eda869a-d17aaae4-71851aee_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:27.990714Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:28.034550Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342192548661288:5175]
2025-10-09T21:48:28.034888Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 1219, pipe: [1:7559342192548661288:5175]
2025-10-09T21:48:28.145685Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:28.145967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:28.145982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:28.146045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342192548661288:5175], now have 1 active actors on pipe
2025-10-09T21:48:28.146271Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:28.146286Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342192548661288:5175]
2025-10-09T21:48:28.146289Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:28.146325Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:28.146327Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:28.146386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1220 reason: error at write: Disconnected sessionId: src_id|90c85491-eda869a-d17aaae4-71851aee_0
2025-10-09T21:48:28.146571Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1220 sessionId: src_id|90c85491-eda869a-d17aaae4-71851aee_0 is DEAD
2025-10-09T21:48:28.146729Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90c85491-eda869a-d17aaae4-71851aee_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:28.146741Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90c85491-eda869a-d17aaae4-71851aee_0] Write session will restart in 0.009683s
2025-10-09T21:48:28.146771Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90c85491-eda869a-d17aaae4-71851aee_0] Write session: Do CDS request
2025-10-09T21:48:28.146778Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90c85491-eda869a-d17aaae4-71851aee_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:28.157587Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90c85491-eda869a-d17aaae4-71851aee_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:28.157644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:28.157661Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1221
2025-10-09T21:48:28.157857Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1221 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:28.157904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1221 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:28.157915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1221 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:28.157919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:28.158529Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1221 sessionId:  describe result for acl check
2025-10-09T21:48:28.158569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:28.158577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:28.158579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:28.158586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342192548661353:5017] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:28.159178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342192548661353:5017] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:28.162100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342192548661353:5017] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:28.165172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342192548661353:5017] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:28.165185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342192548661353:5017] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:28.165189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342192548661353:5017] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:28.165197Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1221 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:28.168595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:28.171381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:28.171564Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:28.171576Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:28.171587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:28.171638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:28.171679Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342192186838800:9283]
2025-10-09T21:48:28.172304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:28.172625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:28.172668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:28.172988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:28.173024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.173031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.173032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.173032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.173033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.173033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.173034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.173035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.173035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.173039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.173050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.173053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.173058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.173061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.173065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.173068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.173072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.173075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.173082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.173090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.173096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.173103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.173114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.173124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.173133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.173139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.173148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.173158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.173163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.173173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.173183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.173193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.173197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.173203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.173212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.173217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.173248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:28.179787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:28.180830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:28.180847Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:28.180849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:28.180853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:28.180859Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1220 [2:7559342192186838800:9283]
2025-10-09T21:48:28.180870Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:28.180889Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:28.180892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:28.180895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:28.180898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:28.180917Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:28.180923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:28.180932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:28.181003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.181012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.181014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.181017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:28.181088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:28.181095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.181103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.181109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.181119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:28.181151Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342192186838795:9283]
2025-10-09T21:48:28.181158Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342192186838795:9283]
2025-10-09T21:48:28.181160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342192186838795:9283]
2025-10-09T21:48:28.181173Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:28.181184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:28.181187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:28.181188Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:28.181196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342192548661293:2432], now have 1 active actors on pipe
2025-10-09T21:48:28.181205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342192548661538:5017], now have 1 active actors on pipe
2025-10-09T21:48:28.181214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:28.181234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:28.181242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:28.181222Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1220
2025-10-09T21:48:28.181267Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1220
2025-10-09T21:48:28.181281Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342192186838795:9283]
2025-10-09T21:48:28.181293Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:28.181295Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:28.181335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:28.181344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:28.181385Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|907f5927-e8b10684-23554cce-55811c9b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:28.181396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:28.181406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:28.181409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:28.181419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:28.181433Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:28.181451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:28.181923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:28.181934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:28.181951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:28.182059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1221 partition: 0 MaxSeqNo: 25 sessionId: src_id|907f5927-e8b10684-23554cce-55811c9b_0
2025-10-09T21:48:28.182419Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90c85491-eda869a-d17aaae4-71851aee_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|907f5927-e8b10684-23554cce-55811c9b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:28.184837Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|907f5927-e8b10684-23554cce-55811c9b_0] Write session: try to update token
2025-10-09T21:48:28.184854Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|907f5927-e8b10684-23554cce-55811c9b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:28.195749Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:28.195770Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:28.195773Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:28.195775Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:28.195785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342192186838795:9283]
2025-10-09T21:48:28.195791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342192186838795:9283]
2025-10-09T21:48:28.195794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342192186838795:9283]
2025-10-09T21:48:28.195841Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:28.195843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:28.195850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.195853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:28.195855Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:28.195859Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:28.195983Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:28.198242Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1221 sessionId: src_id|907f5927-e8b10684-23554cce-55811c9b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:28.200493Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:28.246522Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342192548661603:5175]
2025-10-09T21:48:28.246769Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 1220, pipe: [1:7559342192548661603:5175]
2025-10-09T21:48:28.355435Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:28.355971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:28.355990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:28.356034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342192548661603:5175], now have 1 active actors on pipe
2025-10-09T21:48:28.356250Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:28.356258Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342192548661603:5175]
2025-10-09T21:48:28.356260Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:28.356297Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:28.356300Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:28.356355Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1221 reason: error at write: Disconnected sessionId: src_id|907f5927-e8b10684-23554cce-55811c9b_0
2025-10-09T21:48:28.356537Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1221 sessionId: src_id|907f5927-e8b10684-23554cce-55811c9b_0 is DEAD
2025-10-09T21:48:28.356702Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|907f5927-e8b10684-23554cce-55811c9b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:28.356713Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|907f5927-e8b10684-23554cce-55811c9b_0] Write session will restart in 0.008763s
2025-10-09T21:48:28.356753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|907f5927-e8b10684-23554cce-55811c9b_0] Write session: Do CDS request
2025-10-09T21:48:28.356760Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|907f5927-e8b10684-23554cce-55811c9b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:28.366557Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|907f5927-e8b10684-23554cce-55811c9b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:28.366624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:28.366642Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1222
2025-10-09T21:48:28.366797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1222 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:28.366837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1222 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:28.366844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1222 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:28.366862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:28.367409Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1222 sessionId:  describe result for acl check
2025-10-09T21:48:28.367447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:28.367453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:28.367455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:28.367461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342192548661656:5009] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:28.368033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342192548661656:5009] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:28.371039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342192548661656:5009] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:28.373908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342192548661656:5009] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:28.373920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342192548661656:5009] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:28.373923Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342192548661656:5009] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:28.373929Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1222 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:28.378193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:28.381618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:28.381877Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:28.381887Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:28.381900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:28.381965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:28.382007Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342192186839096:9285]
2025-10-09T21:48:28.382561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:28.382862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:28.382901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:28.383172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:28.383221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.383229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.383230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.383231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.383232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.383233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.383234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.383235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.383236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.383240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.383250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.383254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.383257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.383260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.383263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.383266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.383269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.383272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.383279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.383286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.383291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.383296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.383299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.383303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.383307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.383311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.383315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.383319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.383323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.383327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.383332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.383336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.383339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.383344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.383347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.383352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.383377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:28.389954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:28.391024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:28.391042Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:28.391044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:28.391049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:28.391058Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1221 [2:7559342192186839096:9285]
2025-10-09T21:48:28.391072Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:28.391092Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:28.391095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:28.391098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:28.391100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:28.391116Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:28.391122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:28.391130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:28.391187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.391198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.391200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.391202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:28.391265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:28.391275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.391285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.391295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.391308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:28.391331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342192548661606:2432], now have 1 active actors on pipe
2025-10-09T21:48:28.391339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342192548661813:5009], now have 1 active actors on pipe
2025-10-09T21:48:28.391347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342192186839091:9285]
2025-10-09T21:48:28.391354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342192186839091:9285]
2025-10-09T21:48:28.391356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342192186839091:9285]
2025-10-09T21:48:28.391368Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:28.391382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:28.391394Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:28.391398Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:28.391401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:28.391409Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:28.391412Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:28.391392Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1221
2025-10-09T21:48:28.391430Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342192186839091:9285]
2025-10-09T21:48:28.391417Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1221
2025-10-09T21:48:28.391433Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:28.391439Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:28.391469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:28.391479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:28.391514Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|21f2c9da-11ff693c-fa8c0b1f-b4c26354_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:28.391526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:28.391531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:28.391534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:28.391544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:28.391559Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:28.391579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:28.391753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:28.391760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:28.391775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:28.391892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1222 partition: 0 MaxSeqNo: 25 sessionId: src_id|21f2c9da-11ff693c-fa8c0b1f-b4c26354_0
2025-10-09T21:48:28.392278Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|907f5927-e8b10684-23554cce-55811c9b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|21f2c9da-11ff693c-fa8c0b1f-b4c26354_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:28.394754Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21f2c9da-11ff693c-fa8c0b1f-b4c26354_0] Write session: try to update token
2025-10-09T21:48:28.394773Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21f2c9da-11ff693c-fa8c0b1f-b4c26354_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:28.405813Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:28.405847Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:28.405849Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:28.405851Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:28.405860Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342192186839091:9285]
2025-10-09T21:48:28.405863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342192186839091:9285]
2025-10-09T21:48:28.405867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342192186839091:9285]
2025-10-09T21:48:28.405912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:28.405920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.405923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:28.405942Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:28.405957Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:28.405962Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:28.406077Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:28.407735Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1222 sessionId: src_id|21f2c9da-11ff693c-fa8c0b1f-b4c26354_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:28.410143Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:28.456382Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342192548661911:5175]
2025-10-09T21:48:28.456574Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 1221, pipe: [1:7559342192548661911:5175]
2025-10-09T21:48:28.555728Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:28.556214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:28.556228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:28.556271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342192548661911:5175], now have 1 active actors on pipe
2025-10-09T21:48:28.556459Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:28.556566Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:28.556619Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342192548661911:5175]
2025-10-09T21:48:28.556642Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:28.556715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1222 reason: status is not ok: tablet will be restarted right now sessionId: src_id|21f2c9da-11ff693c-fa8c0b1f-b4c26354_0
2025-10-09T21:48:28.556893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1222 sessionId: src_id|21f2c9da-11ff693c-fa8c0b1f-b4c26354_0 is DEAD
2025-10-09T21:48:28.557023Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21f2c9da-11ff693c-fa8c0b1f-b4c26354_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:28.557034Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21f2c9da-11ff693c-fa8c0b1f-b4c26354_0] Write session will restart in 0.007979s
2025-10-09T21:48:28.557061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21f2c9da-11ff693c-fa8c0b1f-b4c26354_0] Write session: Do CDS request
2025-10-09T21:48:28.557066Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21f2c9da-11ff693c-fa8c0b1f-b4c26354_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:28.566785Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21f2c9da-11ff693c-fa8c0b1f-b4c26354_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:28.566851Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:28.566868Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1223
2025-10-09T21:48:28.567026Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1223 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:28.567058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1223 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:28.567067Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1223 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:28.567071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:28.567500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1223 sessionId:  describe result for acl check
2025-10-09T21:48:28.567537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:28.567544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:28.567545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:28.567551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342192548661991:4992] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:28.568021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342192548661991:4992] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:28.570562Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342192548661991:4992] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:28.573733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342192548661991:4992] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:28.573745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342192548661991:4992] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:28.573748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342192548661991:4992] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:28.573757Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1223 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:28.577352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:28.580018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:28.580202Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:28.580212Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:28.580223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:28.580268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:28.580305Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342192186839431:9297]
2025-10-09T21:48:28.580799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:28.581107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:28.581145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:28.581419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:28.581466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.581471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.581472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.581473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.581474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.581475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.581476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.581477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.581478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.581482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.581491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.581494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.581497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.581500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.581503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.581506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.581509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.581512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.581518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.581524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.581530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.581533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.581538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.581542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.581546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.581549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.581553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.581557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.581560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.581565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.581569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.581573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.581576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.581580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.581583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.581587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.581612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:28.587783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:28.588982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:28.588998Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:28.589000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:28.589004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:28.589009Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1222 [2:7559342192186839431:9297]
2025-10-09T21:48:28.589023Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:28.589039Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:28.589042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:28.589045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:28.589047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:28.589062Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:28.589071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:28.589078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:28.589131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.589137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.589138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.589140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:28.589200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:28.589208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.589218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.589230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.589246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:28.589271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342192548661927:2432], now have 1 active actors on pipe
2025-10-09T21:48:28.589282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342192186839426:9297]
2025-10-09T21:48:28.589288Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342192186839426:9297]
2025-10-09T21:48:28.589290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342192186839426:9297]
2025-10-09T21:48:28.589302Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:28.589313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342192548662161:4992], now have 1 active actors on pipe
2025-10-09T21:48:28.589316Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:28.589321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:28.589327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:28.589329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:28.589332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:28.589334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:28.589357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342192186839426:9297]
2025-10-09T21:48:28.589364Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:28.589366Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:28.589349Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1222
2025-10-09T21:48:28.589358Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1222
2025-10-09T21:48:28.589404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:28.589413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:28.589438Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c85b9048-a03e16bc-c329ac7c-bc8fecae_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:28.589450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:28.589457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:28.589460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:28.589470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:28.589483Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:28.589500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:28.589646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:28.589657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:28.589671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:28.589758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1223 partition: 0 MaxSeqNo: 25 sessionId: src_id|c85b9048-a03e16bc-c329ac7c-bc8fecae_0
2025-10-09T21:48:28.590174Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21f2c9da-11ff693c-fa8c0b1f-b4c26354_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c85b9048-a03e16bc-c329ac7c-bc8fecae_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:28.592692Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c85b9048-a03e16bc-c329ac7c-bc8fecae_0] Write session: try to update token
2025-10-09T21:48:28.592718Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c85b9048-a03e16bc-c329ac7c-bc8fecae_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:28.605658Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1223 sessionId: src_id|c85b9048-a03e16bc-c329ac7c-bc8fecae_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:28.605975Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:28.605996Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:28.606005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342192186839426:9297]
2025-10-09T21:48:28.606068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:28.606073Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:28.606266Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:28.606414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:28.606559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:28.606705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:28.606903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:28.606954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:28.607911Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:28.608095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:28.608111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:28.608143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:28.608218Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:28.612980Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:28.612996Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:28.612998Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:28.613000Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:28.613009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342192186839426:9297]
2025-10-09T21:48:28.613013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342192186839426:9297]
2025-10-09T21:48:28.613016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342192186839426:9297]
2025-10-09T21:48:28.613051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:28.613063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.613065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:28.613067Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:28.613074Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:28.613092Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:28.613188Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:28.656635Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342192548662242:5175]
2025-10-09T21:48:28.656866Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 1222, pipe: [1:7559342192548662242:5175]
2025-10-09T21:48:28.757428Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:28.765470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:28.765504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:28.765663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:28.765733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:28.765791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:28.765848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:28.766198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:28.766212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:28.766271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:28.766324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:28.766383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:28.766446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:28.766483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:28.766489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:28.766541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:28.766592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:28.766653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:28.766707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:28.766749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:28.766757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:28.766829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:28.766893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:28.766950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:28.767006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:28.767044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:28.767049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:28.767117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:28.767176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:28.767237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:28.767299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:28.767344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:28.767354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:28.767429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:28.767487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:28.767547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:28.767603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:28.767653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:28.767660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:28.767721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:28.767776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:28.767832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:28.767887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:28.767954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:28.767961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:28.768016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:28.768068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:28.768121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:28.768179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:28.768218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:28.768227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:28.768280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:28.768338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:28.768400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:28.768461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:28.768501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:28.768508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:28.768563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:28.768618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:28.768676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:28.768731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:28.768769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:28.768776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:28.768834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342192548662242:5175], now have 1 active actors on pipe
2025-10-09T21:48:28.769076Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342192548662242:5175]
2025-10-09T21:48:28.769109Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:28.769401Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:28.769506Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:28.769554Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1223 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c85b9048-a03e16bc-c329ac7c-bc8fecae_0
2025-10-09T21:48:28.769781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1223 sessionId: src_id|c85b9048-a03e16bc-c329ac7c-bc8fecae_0 is DEAD
2025-10-09T21:48:28.769880Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c85b9048-a03e16bc-c329ac7c-bc8fecae_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:28.769889Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c85b9048-a03e16bc-c329ac7c-bc8fecae_0] Write session will restart in 0.007909s
2025-10-09T21:48:28.769908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c85b9048-a03e16bc-c329ac7c-bc8fecae_0] Write session: Do CDS request
2025-10-09T21:48:28.769913Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c85b9048-a03e16bc-c329ac7c-bc8fecae_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:28.778458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c85b9048-a03e16bc-c329ac7c-bc8fecae_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:28.778543Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:28.778557Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1224
2025-10-09T21:48:28.778681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1224 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:28.778715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1224 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:28.778726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1224 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:28.778730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:28.779402Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1224 sessionId:  describe result for acl check
2025-10-09T21:48:28.779437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:28.779455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:28.779462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:28.779468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342192548662299:4979] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:28.780197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342192548662299:4979] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:28.783717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342192548662299:4979] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:28.786546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342192548662299:4979] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:28.786557Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342192548662299:4979] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:28.786560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342192548662299:4979] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:28.786568Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1224 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:28.789421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:28.792257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:28.792446Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:28.792457Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:28.792469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:28.792528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:28.792567Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342192186839733:9303]
2025-10-09T21:48:28.793154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:28.793438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:28.793481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:28.793808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:28.793856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.793863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.793864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.793865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.793866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.793867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.793868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.793869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.793870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.793875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.793886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.793891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.793895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.793899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.793903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.793907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.793911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.793915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.793922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.793930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.793935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.793940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.793944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.793950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.793954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.793959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.793969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.793974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.793978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.793983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.793990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.793995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.793998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.794003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.794007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.794012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.794057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:28.800406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:28.801887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:28.801901Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:28.801903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:28.801906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:28.801913Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1223 [2:7559342192186839733:9303]
2025-10-09T21:48:28.801925Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:28.801942Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:28.801945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:28.801948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:28.801950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:28.801966Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:28.801977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:28.801987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:28.802059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.802067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.802068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.802071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:28.802135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:28.802144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.802156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.802166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.802181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:28.802206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342192548662245:2432], now have 1 active actors on pipe
2025-10-09T21:48:28.802214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342192548662501:4979], now have 1 active actors on pipe
2025-10-09T21:48:28.802221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342192186839728:9303]
2025-10-09T21:48:28.802223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342192186839728:9303]
2025-10-09T21:48:28.802225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342192186839728:9303]
2025-10-09T21:48:28.802237Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:28.802251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:28.802252Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:28.802255Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:28.802258Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:28.802266Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:28.802268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:28.802296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342192186839728:9303]
2025-10-09T21:48:28.802308Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:28.802314Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:28.802293Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1223
2025-10-09T21:48:28.802295Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1223
2025-10-09T21:48:28.802361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:28.802371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:28.802401Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5030dc64-e479c284-10409fa1-d0d5d677_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:28.802414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:28.802424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:28.802427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:28.802437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:28.802452Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:28.802472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:28.802623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:28.802632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:28.802646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:28.802750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1224 partition: 0 MaxSeqNo: 25 sessionId: src_id|5030dc64-e479c284-10409fa1-d0d5d677_0
2025-10-09T21:48:28.803126Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c85b9048-a03e16bc-c329ac7c-bc8fecae_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5030dc64-e479c284-10409fa1-d0d5d677_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:28.805698Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5030dc64-e479c284-10409fa1-d0d5d677_0] Write session: try to update token
2025-10-09T21:48:28.805728Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5030dc64-e479c284-10409fa1-d0d5d677_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:28.817136Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:28.817150Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:28.817151Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:28.817153Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:28.817161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342192186839728:9303]
2025-10-09T21:48:28.817165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342192186839728:9303]
2025-10-09T21:48:28.817168Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342192186839728:9303]
2025-10-09T21:48:28.817204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:28.817210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.817211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:28.817223Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:28.817230Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:28.817234Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:28.817331Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:28.818687Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1224 sessionId: src_id|5030dc64-e479c284-10409fa1-d0d5d677_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:28.820968Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:28.869131Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342192548662555:5175]
2025-10-09T21:48:28.869392Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 1223, pipe: [1:7559342192548662555:5175]
2025-10-09T21:48:28.972067Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:28.972328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:28.972343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:28.972388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342192548662555:5175], now have 1 active actors on pipe
2025-10-09T21:48:28.972589Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:28.972603Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:28.972619Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:28.972627Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:28.972679Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342192548662555:5175]
2025-10-09T21:48:28.972679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1224 reason: error at write: Disconnected sessionId: src_id|5030dc64-e479c284-10409fa1-d0d5d677_0
2025-10-09T21:48:28.972852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1224 sessionId: src_id|5030dc64-e479c284-10409fa1-d0d5d677_0 is DEAD
2025-10-09T21:48:28.973007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5030dc64-e479c284-10409fa1-d0d5d677_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:28.973019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5030dc64-e479c284-10409fa1-d0d5d677_0] Write session will restart in 0.009402s
2025-10-09T21:48:28.973048Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5030dc64-e479c284-10409fa1-d0d5d677_0] Write session: Do CDS request
2025-10-09T21:48:28.973054Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5030dc64-e479c284-10409fa1-d0d5d677_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:28.983545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5030dc64-e479c284-10409fa1-d0d5d677_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:28.983631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:28.983646Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1225
2025-10-09T21:48:28.983796Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1225 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:28.983827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1225 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:28.983838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1225 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:28.983842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:28.984332Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1225 sessionId:  describe result for acl check
2025-10-09T21:48:28.984371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:28.984372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:28.984374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:28.984380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342192548662624:4978] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:28.984910Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342192548662624:4978] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:28.987388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342192548662624:4978] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:28.990574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342192548662624:4978] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:28.990590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342192548662624:4978] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:28.990594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342192548662624:4978] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:28.990607Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1225 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:28.993629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:28.996330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:28.996538Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:28.996548Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:28.996561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:28.996616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:28.996653Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342192186840032:9305]
2025-10-09T21:48:28.997242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:28.997554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:28.997625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:28.997957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:28.998013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.998019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.998020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.998021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.998022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.998024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.998025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.998026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.998027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.998046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.998057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.998061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.998065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.998074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.998078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.998086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.998089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.998093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.998100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.998112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:28.998122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.998132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:28.998138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.998143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:28.998147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.998152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:28.998157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.998162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:28.998170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.998175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:28.998183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.998193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:28.998201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.998206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:28.998214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.998220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:28.998252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:29.004396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:29.005448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:29.005462Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:29.005466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:29.005470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:29.005476Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1224 [2:7559342192186840032:9305]
2025-10-09T21:48:29.005493Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:29.005513Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:29.005517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:29.005520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:29.005523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:29.005542Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:29.005555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:29.005561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:29.005618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.005625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.005627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.005630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:29.005697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:29.005704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.005726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.005743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.005760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:29.005788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342192548662560:2432], now have 1 active actors on pipe
2025-10-09T21:48:29.005796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342192548662806:4978], now have 1 active actors on pipe
2025-10-09T21:48:29.005804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342192186840027:9305]
2025-10-09T21:48:29.005806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342192186840027:9305]
2025-10-09T21:48:29.005808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342192186840027:9305]
2025-10-09T21:48:29.005821Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:29.005835Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:29.005839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:29.005844Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:29.005846Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:29.005854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:29.005848Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1224
2025-10-09T21:48:29.005857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:29.005851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1224
2025-10-09T21:48:29.005894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342192186840027:9305]
2025-10-09T21:48:29.005897Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:29.005900Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:29.005915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.005925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.005979Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|26ce84bc-45a7a213-b470bf85-c7b17d6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:29.005992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:29.005996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:29.005999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:29.006011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:29.006020Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:29.006055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:29.006256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.006270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.006290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:29.006412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1225 partition: 0 MaxSeqNo: 25 sessionId: src_id|26ce84bc-45a7a213-b470bf85-c7b17d6_0
2025-10-09T21:48:29.006813Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5030dc64-e479c284-10409fa1-d0d5d677_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|26ce84bc-45a7a213-b470bf85-c7b17d6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:29.009352Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26ce84bc-45a7a213-b470bf85-c7b17d6_0] Write session: try to update token
2025-10-09T21:48:29.009368Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26ce84bc-45a7a213-b470bf85-c7b17d6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:29.020985Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:29.021003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:29.021005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:29.021007Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:29.021018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342192186840027:9305]
2025-10-09T21:48:29.021023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342192186840027:9305]
2025-10-09T21:48:29.021027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342192186840027:9305]
2025-10-09T21:48:29.021072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:29.021072Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:29.021079Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:29.021079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.021082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:29.021083Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:29.021221Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:29.022801Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1225 sessionId: src_id|26ce84bc-45a7a213-b470bf85-c7b17d6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:29.025071Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:29.072901Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342196843630162:5175]
2025-10-09T21:48:29.073180Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 1224, pipe: [1:7559342196843630162:5175]
2025-10-09T21:48:29.172665Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:29.173304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.173320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.173356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342196843630162:5175], now have 1 active actors on pipe
2025-10-09T21:48:29.173549Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:29.173554Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342196843630162:5175]
2025-10-09T21:48:29.173558Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:29.173582Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:29.173584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:29.173643Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1225 reason: error at write: Disconnected sessionId: src_id|26ce84bc-45a7a213-b470bf85-c7b17d6_0
2025-10-09T21:48:29.173854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1225 sessionId: src_id|26ce84bc-45a7a213-b470bf85-c7b17d6_0 is DEAD
2025-10-09T21:48:29.174019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26ce84bc-45a7a213-b470bf85-c7b17d6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:29.174047Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26ce84bc-45a7a213-b470bf85-c7b17d6_0] Write session will restart in 0.009254s
2025-10-09T21:48:29.174076Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26ce84bc-45a7a213-b470bf85-c7b17d6_0] Write session: Do CDS request
2025-10-09T21:48:29.174083Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26ce84bc-45a7a213-b470bf85-c7b17d6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:29.184580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26ce84bc-45a7a213-b470bf85-c7b17d6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:29.184674Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:29.184692Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1226
2025-10-09T21:48:29.184834Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1226 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:29.184867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1226 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:29.184877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1226 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:29.184885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:29.185335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1226 sessionId:  describe result for acl check
2025-10-09T21:48:29.185365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:29.185371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:29.185373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:29.185379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342196843630229:4972] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:29.185873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342196843630229:4972] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:29.188847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342196843630229:4972] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:29.192198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342196843630229:4972] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:29.192211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342196843630229:4972] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:29.192214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342196843630229:4972] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:29.192221Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1226 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:29.195502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:29.198242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:29.198433Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:29.198443Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:29.198455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:29.198506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:29.198545Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342196481807629:9307]
2025-10-09T21:48:29.199142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:29.199447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:29.199494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:29.199795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:29.199848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:29.199854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:29.199856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:29.199857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:29.199858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:29.199859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:29.199860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.199862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.199863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.199868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:29.199879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:29.199884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:29.199888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:29.199892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:29.199896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:29.199900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.199904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.199908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.199916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:29.199924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:29.199930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:29.199935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:29.199941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:29.199951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:29.199955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:29.199961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:29.199971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:29.199975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:29.199980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:29.199990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:29.199995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.200005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.200009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.200014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.200023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.200029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.200064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:29.206264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:29.207320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:29.207333Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:29.207336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:29.207340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:29.207347Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1225 [2:7559342196481807629:9307]
2025-10-09T21:48:29.207360Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:29.207383Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:29.207390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:29.207394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:29.207396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:29.207413Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:29.207419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:29.207427Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:29.207481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.207489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.207491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.207493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:29.207561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:29.207569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.207581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.207594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.207615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:29.207640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342196843630168:2432], now have 1 active actors on pipe
2025-10-09T21:48:29.207649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342196843630427:4972], now have 1 active actors on pipe
2025-10-09T21:48:29.207658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342196481807624:9307]
2025-10-09T21:48:29.207661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342196481807624:9307]
2025-10-09T21:48:29.207663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342196481807624:9307]
2025-10-09T21:48:29.207678Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:29.207693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:29.207700Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:29.207704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:29.207706Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:29.207713Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:29.207720Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:29.207713Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1225
2025-10-09T21:48:29.207745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342196481807624:9307]
2025-10-09T21:48:29.207729Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1225
2025-10-09T21:48:29.207752Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:29.207755Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:29.207774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.207783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.207942Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e26c02b7-e83db672-7644756e-6d2b1370_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:29.207957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:29.207961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:29.207965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:29.207975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:29.207985Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:29.208009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:29.208162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.208171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.208188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:29.208278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1226 partition: 0 MaxSeqNo: 25 sessionId: src_id|e26c02b7-e83db672-7644756e-6d2b1370_0
2025-10-09T21:48:29.208666Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26ce84bc-45a7a213-b470bf85-c7b17d6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e26c02b7-e83db672-7644756e-6d2b1370_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:29.211136Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e26c02b7-e83db672-7644756e-6d2b1370_0] Write session: try to update token
2025-10-09T21:48:29.211152Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e26c02b7-e83db672-7644756e-6d2b1370_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:29.221775Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:29.221790Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:29.221792Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:29.221795Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:29.221805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342196481807624:9307]
2025-10-09T21:48:29.221810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342196481807624:9307]
2025-10-09T21:48:29.221814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342196481807624:9307]
2025-10-09T21:48:29.221859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:29.221860Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:29.221867Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:29.221867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.221870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:29.221871Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:29.221999Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:29.224105Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1226 sessionId: src_id|e26c02b7-e83db672-7644756e-6d2b1370_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:29.226600Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:29.273604Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342196843630482:5175]
2025-10-09T21:48:29.273916Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 1225, pipe: [1:7559342196843630482:5175]
2025-10-09T21:48:29.374093Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:29.374315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.374330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.374372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342196843630482:5175], now have 1 active actors on pipe
2025-10-09T21:48:29.374579Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:29.374680Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:29.374697Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:29.374701Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:29.374739Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342196843630482:5175]
2025-10-09T21:48:29.374771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1226 reason: error at write: Disconnected sessionId: src_id|e26c02b7-e83db672-7644756e-6d2b1370_0
2025-10-09T21:48:29.374962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1226 sessionId: src_id|e26c02b7-e83db672-7644756e-6d2b1370_0 is DEAD
2025-10-09T21:48:29.375109Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e26c02b7-e83db672-7644756e-6d2b1370_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:29.375129Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e26c02b7-e83db672-7644756e-6d2b1370_0] Write session will restart in 0.006295s
2025-10-09T21:48:29.375149Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e26c02b7-e83db672-7644756e-6d2b1370_0] Write session: Do CDS request
2025-10-09T21:48:29.375154Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e26c02b7-e83db672-7644756e-6d2b1370_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:29.382510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e26c02b7-e83db672-7644756e-6d2b1370_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:29.382639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:29.382654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1227
2025-10-09T21:48:29.382762Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1227 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:29.382787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1227 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:29.382794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1227 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:29.382798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:29.383179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1227 sessionId:  describe result for acl check
2025-10-09T21:48:29.383208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:29.383212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:29.383215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:29.383220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342196843630509:4961] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:29.383649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342196843630509:4961] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:29.386553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342196843630509:4961] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:29.390118Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342196843630509:4961] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:29.390137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342196843630509:4961] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:29.390140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342196843630509:4961] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:29.390148Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1227 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:29.395532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:29.397654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:29.397857Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:29.397868Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:29.397880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:29.397931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:29.397972Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342196481807930:9309]
2025-10-09T21:48:29.398494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:29.398764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:29.398794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:29.399033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:29.399082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:29.399087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:29.399088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:29.399089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:29.399091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:29.399091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:29.399092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.399094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.399095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.399100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:29.399111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:29.399116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:29.399120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:29.399124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:29.399127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:29.399129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.399131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.399133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.399138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:29.399144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:29.399148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:29.399151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:29.399154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:29.399157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:29.399159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:29.399162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:29.399165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:29.399168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:29.399170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:29.399173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:29.399176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.399179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.399181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.399184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.399188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.399191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.399206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:29.405353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:29.406449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:29.406467Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:29.406469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:29.406473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:29.406479Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1226 [2:7559342196481807930:9309]
2025-10-09T21:48:29.406492Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:29.406510Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:29.406513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:29.406516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:29.406518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:29.406535Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:29.406546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:29.406553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:29.406611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.406619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.406621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.406624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:29.406689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:29.406696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.406708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.406719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.406742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:29.406767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342196843630485:2432], now have 1 active actors on pipe
2025-10-09T21:48:29.406774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342196843630665:4961], now have 1 active actors on pipe
2025-10-09T21:48:29.406781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342196481807925:9309]
2025-10-09T21:48:29.406786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342196481807925:9309]
2025-10-09T21:48:29.406788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342196481807925:9309]
2025-10-09T21:48:29.406801Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:29.406820Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:29.406832Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:29.406803Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1226
2025-10-09T21:48:29.406844Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1226
2025-10-09T21:48:29.406840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:29.406843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:29.406843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:29.406847Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:29.406867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342196481807925:9309]
2025-10-09T21:48:29.406875Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:29.406894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:29.406984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.406995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.407023Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1d9691-35cc4494-3ab60b0c-7d9552c2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:29.407035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:29.407039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:29.407041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:29.407051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:29.407061Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:29.407078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:29.407274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.407281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.407297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:29.407401Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1227 partition: 0 MaxSeqNo: 25 sessionId: src_id|1d9691-35cc4494-3ab60b0c-7d9552c2_0
2025-10-09T21:48:29.407784Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e26c02b7-e83db672-7644756e-6d2b1370_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1d9691-35cc4494-3ab60b0c-7d9552c2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:29.410314Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1d9691-35cc4494-3ab60b0c-7d9552c2_0] Write session: try to update token
2025-10-09T21:48:29.410330Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1d9691-35cc4494-3ab60b0c-7d9552c2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:29.423486Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1227 sessionId: src_id|1d9691-35cc4494-3ab60b0c-7d9552c2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:29.423880Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:29.423896Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:29.423907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342196481807925:9309]
2025-10-09T21:48:29.423957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:29.423957Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:29.424146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:29.424287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:29.424417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:29.424550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:29.424745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:29.424787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:29.425780Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:29.425967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.425984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.426018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:29.426129Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:29.430854Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:29.430869Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:29.430871Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:29.430873Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:29.430882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342196481807925:9309]
2025-10-09T21:48:29.430887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342196481807925:9309]
2025-10-09T21:48:29.430891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342196481807925:9309]
2025-10-09T21:48:29.430927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:29.430930Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:29.430934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.430937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:29.430939Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:29.430943Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:29.431067Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:29.474809Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342196843630795:5175]
2025-10-09T21:48:29.475104Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 1226, pipe: [1:7559342196843630795:5175]
2025-10-09T21:48:29.578428Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:29.586173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.586194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.586352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.586412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.586474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.586540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.586591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.586600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.586667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.586727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.586789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.586855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.586897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.586906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.586967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.587024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.587087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.587146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.587187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.587193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.587262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.587322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.587383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.587444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.587486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.587494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.587555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.587616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.587677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.587735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.587779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.587786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.587847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.587906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.587967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.588026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.588067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.588076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.588132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.588186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.588253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.588309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.588351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.588358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.588416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.588475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.588533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.588591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.588630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.588637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.588694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.588747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.588802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.588860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.588902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.588910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.588971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.589027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.589092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.589152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.589193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.589199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.589259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342196843630795:5175], now have 1 active actors on pipe
2025-10-09T21:48:29.589482Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:29.589553Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:29.589559Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:29.589588Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1227 reason: error at write: Disconnected sessionId: src_id|1d9691-35cc4494-3ab60b0c-7d9552c2_0
2025-10-09T21:48:29.589865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1227 sessionId: src_id|1d9691-35cc4494-3ab60b0c-7d9552c2_0 is DEAD
2025-10-09T21:48:29.590001Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d9691-35cc4494-3ab60b0c-7d9552c2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:29.590010Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d9691-35cc4494-3ab60b0c-7d9552c2_0] Write session will restart in 0.007755s
2025-10-09T21:48:29.590050Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d9691-35cc4494-3ab60b0c-7d9552c2_0] Write session: Do CDS request
2025-10-09T21:48:29.590058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d9691-35cc4494-3ab60b0c-7d9552c2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:29.590128Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342196843630795:5175]
2025-10-09T21:48:29.590268Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:29.598504Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1d9691-35cc4494-3ab60b0c-7d9552c2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:29.598631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:29.598646Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1228
2025-10-09T21:48:29.598786Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1228 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:29.598820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1228 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:29.598837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1228 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:29.598845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:29.599208Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1228 sessionId:  describe result for acl check
2025-10-09T21:48:29.599239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:29.599241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:29.599243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:29.599249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342196843630840:4952] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:29.599776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342196843630840:4952] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:29.603380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342196843630840:4952] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:29.606872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342196843630840:4952] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:29.606888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342196843630840:4952] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:29.606892Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342196843630840:4952] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:29.606898Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1228 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:29.611397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:29.613837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:29.614020Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:29.614059Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:29.614076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:29.614137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:29.614181Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342196481808270:9321]
2025-10-09T21:48:29.614755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:29.615047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:29.615108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:29.615332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:29.615364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:29.615368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:29.615369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:29.615369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:29.615370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:29.615371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:29.615371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.615372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.615372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.615376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:29.615383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:29.615385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:29.615387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:29.615389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:29.615391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:29.615393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.615395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.615397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.615402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:29.615407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:29.615412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:29.615415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:29.615417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:29.615421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:29.615423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:29.615426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:29.615428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:29.615431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:29.615434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:29.615437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:29.615439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.615442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.615445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.615447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.615449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.615452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.615467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:29.621409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:29.622531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:29.622542Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:29.622543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:29.622546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:29.622551Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1227 [2:7559342196481808270:9321]
2025-10-09T21:48:29.622561Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:29.622575Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:29.622576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:29.622578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:29.622580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:29.622592Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:29.622595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:29.622601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:29.622648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.622653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.622655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.622657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:29.622707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:29.622712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.622719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.622727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.622739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:29.622757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342196843631021:4952], now have 1 active actors on pipe
2025-10-09T21:48:29.622767Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342196481808265:9321]
2025-10-09T21:48:29.622769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342196481808265:9321]
2025-10-09T21:48:29.622770Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342196481808265:9321]
2025-10-09T21:48:29.622778Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:29.622787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342196843630811:2432], now have 1 active actors on pipe
2025-10-09T21:48:29.622795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:29.622795Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:29.622798Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:29.622800Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:29.622807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:29.622808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:29.622828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342196481808265:9321]
2025-10-09T21:48:29.622836Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:29.622847Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:29.622851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1227
2025-10-09T21:48:29.622899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.622906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.622852Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1227
2025-10-09T21:48:29.622929Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|776300fc-250b07fb-abb3f525-86fd97e7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:29.622937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:29.622940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:29.622943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:29.622951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:29.622962Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:29.622975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:29.623109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.623118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.623136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:29.623242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1228 partition: 0 MaxSeqNo: 25 sessionId: src_id|776300fc-250b07fb-abb3f525-86fd97e7_0
2025-10-09T21:48:29.623612Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d9691-35cc4494-3ab60b0c-7d9552c2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|776300fc-250b07fb-abb3f525-86fd97e7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:29.625997Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|776300fc-250b07fb-abb3f525-86fd97e7_0] Write session: try to update token
2025-10-09T21:48:29.626012Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|776300fc-250b07fb-abb3f525-86fd97e7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:29.636574Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:29.636589Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:29.636592Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:29.636594Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:29.636604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342196481808265:9321]
2025-10-09T21:48:29.636608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342196481808265:9321]
2025-10-09T21:48:29.636612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342196481808265:9321]
2025-10-09T21:48:29.636656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:29.636656Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:29.636663Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:29.636667Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:29.636679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.636683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:29.636821Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:29.638625Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1228 sessionId: src_id|776300fc-250b07fb-abb3f525-86fd97e7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:29.640978Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:29.690365Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342196843631132:5175]
2025-10-09T21:48:29.690630Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 1227, pipe: [1:7559342196843631132:5175]
2025-10-09T21:48:29.781874Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:29.782318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.782331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.782373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342196843631132:5175], now have 1 active actors on pipe
2025-10-09T21:48:29.782537Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:29.782578Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:29.782585Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342196843631132:5175]
2025-10-09T21:48:29.782592Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:29.782602Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:29.782668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1228 reason: error at write: Disconnected sessionId: src_id|776300fc-250b07fb-abb3f525-86fd97e7_0
2025-10-09T21:48:29.782847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1228 sessionId: src_id|776300fc-250b07fb-abb3f525-86fd97e7_0 is DEAD
2025-10-09T21:48:29.782975Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|776300fc-250b07fb-abb3f525-86fd97e7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:29.782994Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|776300fc-250b07fb-abb3f525-86fd97e7_0] Write session will restart in 0.005446s
2025-10-09T21:48:29.783014Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|776300fc-250b07fb-abb3f525-86fd97e7_0] Write session: Do CDS request
2025-10-09T21:48:29.783019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|776300fc-250b07fb-abb3f525-86fd97e7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:29.789417Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|776300fc-250b07fb-abb3f525-86fd97e7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:29.789504Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:29.789520Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1229
2025-10-09T21:48:29.789622Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1229 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:29.789652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1229 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:29.789664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1229 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:29.789678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:29.790255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1229 sessionId:  describe result for acl check
2025-10-09T21:48:29.790282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:29.790283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:29.790284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:29.790288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342196843631158:4948] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:29.790726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342196843631158:4948] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:29.794121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342196843631158:4948] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:29.797187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342196843631158:4948] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:29.797200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342196843631158:4948] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:29.797202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342196843631158:4948] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:29.797208Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1229 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:29.800982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:29.803144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:29.803277Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:29.803284Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:29.803292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:29.803326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:29.803351Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342196481808577:9327]
2025-10-09T21:48:29.803686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:29.803919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:29.803945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:29.804136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:29.804164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:29.804168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:29.804168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:29.804169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:29.804170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:29.804170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:29.804171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.804171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.804172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.804175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:29.804181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:29.804183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:29.804185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:29.804187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:29.804189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:29.804191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.804192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.804194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.804198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:29.804203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:29.804207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:29.804210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:29.804212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:29.804215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:29.804217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:29.804220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:29.804223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:29.804226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:29.804228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:29.804232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:29.804235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.804237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.804240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.804242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.804244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.804247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.804274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:29.809988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:29.810911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:29.810921Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:29.810922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:29.810925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:29.810929Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1228 [2:7559342196481808577:9327]
2025-10-09T21:48:29.810937Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:29.810949Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:29.810951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:29.810952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:29.810954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:29.810965Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:29.810968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:29.810975Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:29.811016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.811022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.811023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.811025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:29.811067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:29.811072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.811079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:29.811085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:29.811097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:29.811119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342196481808572:9327]
2025-10-09T21:48:29.811125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342196481808572:9327]
2025-10-09T21:48:29.811127Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342196481808572:9327]
2025-10-09T21:48:29.811138Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:29.811152Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:29.811155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342196843631341:4948], now have 1 active actors on pipe
2025-10-09T21:48:29.811158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342196843631134:2432], now have 1 active actors on pipe
2025-10-09T21:48:29.811159Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:29.811161Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:29.811166Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:29.811179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:29.811182Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:29.811200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342196481808572:9327]
2025-10-09T21:48:29.811213Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:29.811233Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:29.811223Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1228
2025-10-09T21:48:29.811235Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1228
2025-10-09T21:48:29.811272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.811279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.811300Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|92582926-739d6105-ac9430f3-285936fb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:29.811309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:29.811315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:29.811316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:29.811323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:29.811335Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:29.811351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:29.811508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.811518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.811536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:29.811639Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1229 partition: 0 MaxSeqNo: 25 sessionId: src_id|92582926-739d6105-ac9430f3-285936fb_0
2025-10-09T21:48:29.811954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|776300fc-250b07fb-abb3f525-86fd97e7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|92582926-739d6105-ac9430f3-285936fb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:29.814270Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92582926-739d6105-ac9430f3-285936fb_0] Write session: try to update token
2025-10-09T21:48:29.814286Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92582926-739d6105-ac9430f3-285936fb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:29.821968Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:29.821985Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:29.821995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342196481808572:9327]
2025-10-09T21:48:29.822040Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:29.822056Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:29.822230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:29.822350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:29.822471Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:29.822625Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:29.822812Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:29.822843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:29.825824Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1229 sessionId: src_id|92582926-739d6105-ac9430f3-285936fb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:29.828061Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:29.828216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.828235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.828263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:29.828354Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:29.834231Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:29.834243Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:29.834245Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:29.834246Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:29.834253Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342196481808572:9327]
2025-10-09T21:48:29.834256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342196481808572:9327]
2025-10-09T21:48:29.834258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342196481808572:9327]
2025-10-09T21:48:29.834289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:29.834297Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:29.834299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:29.834302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:29.834306Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:29.834311Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:29.834418Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:29.882873Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342196843631449:5175]
2025-10-09T21:48:29.883113Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 1228, pipe: [1:7559342196843631449:5175]
2025-10-09T21:48:29.982831Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:29.990284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:29.990309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:29.990500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.990561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.990627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.990693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.990740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.990755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.990823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.990889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.990954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.991019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.991060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.991067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.991127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.991187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.991248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.991317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.991358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.991366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.991437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.991499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.991562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.991628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.991671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.991677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.991742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.991802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.991866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.991924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.991967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.991975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.992033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.992088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.992149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.992211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.992256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.992264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.992323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.992375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.992430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.992486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.992525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.992533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.992587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.992641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.992700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.992764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.992808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.992816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.992874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.992924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.992983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.993044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.993155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.993163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.993226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:29.993280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:29.993338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:29.993395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:29.993433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:29.993438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:29.993497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342196843631449:5175], now have 1 active actors on pipe
2025-10-09T21:48:29.993698Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:29.993701Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:29.993718Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:29.993745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1229 reason: error at write: Disconnected sessionId: src_id|92582926-739d6105-ac9430f3-285936fb_0
2025-10-09T21:48:29.993757Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342196843631449:5175]
2025-10-09T21:48:29.993892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1229 sessionId: src_id|92582926-739d6105-ac9430f3-285936fb_0 is DEAD
2025-10-09T21:48:29.994091Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92582926-739d6105-ac9430f3-285936fb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:29.994111Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92582926-739d6105-ac9430f3-285936fb_0] Write session will restart in 0.008562s
2025-10-09T21:48:29.994139Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92582926-739d6105-ac9430f3-285936fb_0] Write session: Do CDS request
2025-10-09T21:48:29.994145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92582926-739d6105-ac9430f3-285936fb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:29.994386Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:30.003611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92582926-739d6105-ac9430f3-285936fb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:30.003722Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:30.003740Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1230
2025-10-09T21:48:30.003879Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1230 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:30.003926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1230 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:30.003941Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1230 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:30.003945Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:30.004559Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1230 sessionId:  describe result for acl check
2025-10-09T21:48:30.004598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:30.004600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:30.004602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:30.004608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342201138598788:4929] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:30.005110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342201138598788:4929] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:30.008795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342201138598788:4929] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:30.012420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342201138598788:4929] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:30.012436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342201138598788:4929] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:30.012440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342201138598788:4929] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:30.012452Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1230 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:30.017071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:30.020567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:30.020792Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:30.020803Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:30.020814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:30.020867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:30.020900Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342200776776187:9329]
2025-10-09T21:48:30.021433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:30.021739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:30.021781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:30.022017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:30.022079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.022086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.022087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.022089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.022090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.022091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.022092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.022093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.022094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.022100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.022111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.022115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.022119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.022122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.022126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.022129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.022133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.022137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.022145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.022153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.022159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.022164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.022169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.022174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.022179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.022184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.022190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.022195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.022200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.022205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.022211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.022216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.022220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.022223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.022226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.022230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.022246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:30.028573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:30.030012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:30.030026Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:30.030046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:30.030050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:30.030058Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1229 [2:7559342200776776187:9329]
2025-10-09T21:48:30.030072Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:30.030089Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:30.030095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:30.030098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:30.030100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:30.030114Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:30.030121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:30.030129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:30.030178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.030183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.030184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.030186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:30.030254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:30.030261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.030274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.030290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.030313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:30.030335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342200776776178:9329]
2025-10-09T21:48:30.030339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342200776776178:9329]
2025-10-09T21:48:30.030341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342200776776178:9329]
2025-10-09T21:48:30.030348Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:30.030359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342201138598967:4929], now have 1 active actors on pipe
2025-10-09T21:48:30.030365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342196843631455:2432], now have 1 active actors on pipe
2025-10-09T21:48:30.030365Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:30.030368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:30.030369Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:30.030372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:30.030383Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:30.030386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:30.030422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342200776776178:9329]
2025-10-09T21:48:30.030437Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:30.030457Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:30.030440Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1229
2025-10-09T21:48:30.030445Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1229
2025-10-09T21:48:30.030514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.030524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.030552Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d95622c4-da621a1-328cc0ec-47361342_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:30.030564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:30.030568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:30.030575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:30.030583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:30.030592Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:30.030612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:30.030780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.030790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.030810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:30.030889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1230 partition: 0 MaxSeqNo: 25 sessionId: src_id|d95622c4-da621a1-328cc0ec-47361342_0
2025-10-09T21:48:30.031263Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92582926-739d6105-ac9430f3-285936fb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d95622c4-da621a1-328cc0ec-47361342_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:30.033854Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d95622c4-da621a1-328cc0ec-47361342_0] Write session: try to update token
2025-10-09T21:48:30.033872Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d95622c4-da621a1-328cc0ec-47361342_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:30.044549Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:30.044565Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:30.044566Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:30.044568Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:30.044578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342200776776178:9329]
2025-10-09T21:48:30.044582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342200776776178:9329]
2025-10-09T21:48:30.044586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342200776776178:9329]
2025-10-09T21:48:30.044634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:30.044635Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:30.044640Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:30.044642Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:30.044656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.044664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:30.044791Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:30.046874Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1230 sessionId: src_id|d95622c4-da621a1-328cc0ec-47361342_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:30.049148Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:30.093824Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342201138599065:5175]
2025-10-09T21:48:30.094147Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 1229, pipe: [1:7559342201138599065:5175]
2025-10-09T21:48:30.194456Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:30.195046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.195061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.195105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342201138599065:5175], now have 1 active actors on pipe
2025-10-09T21:48:30.195291Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:30.195372Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342201138599065:5175]
2025-10-09T21:48:30.195390Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:30.195409Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:30.195466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1230 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d95622c4-da621a1-328cc0ec-47361342_0
2025-10-09T21:48:30.195655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1230 sessionId: src_id|d95622c4-da621a1-328cc0ec-47361342_0 is DEAD
2025-10-09T21:48:30.195839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d95622c4-da621a1-328cc0ec-47361342_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:30.195850Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d95622c4-da621a1-328cc0ec-47361342_0] Write session will restart in 0.006403s
2025-10-09T21:48:30.195891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d95622c4-da621a1-328cc0ec-47361342_0] Write session: Do CDS request
2025-10-09T21:48:30.195898Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d95622c4-da621a1-328cc0ec-47361342_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:30.203355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:30.203587Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d95622c4-da621a1-328cc0ec-47361342_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:30.203652Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:30.203667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1231
2025-10-09T21:48:30.203797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1231 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:30.203840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1231 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:30.203859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1231 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:30.203864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:30.204336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1231 sessionId:  describe result for acl check
2025-10-09T21:48:30.204370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:30.204408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:30.204415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:30.204422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342201138599091:4932] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:30.204882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:30.204919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342201138599091:4932] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:30.205087Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:30.205097Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:30.205109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:30.205165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:30.205203Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342200776776243:9331]
2025-10-09T21:48:30.205831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:30.206133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:30.206192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:30.206469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:30.206516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.206523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.206524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.206525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.206527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.206528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.206529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.206530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.206532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.206538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.206549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.206554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.206558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.206562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.206566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.206571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.206575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.206579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.206586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.206595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.206602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.206610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.206616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.206622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.206626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.206633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.206641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.206647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.206652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.206660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.206665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.206671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.206680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.206685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.206690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.206693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.206711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:30.206746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342201138599091:4932] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:30.212740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342201138599091:4932] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:30.212755Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342201138599091:4932] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:30.212759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342201138599091:4932] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:30.212772Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1231 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:30.213091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:30.214226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:30.214239Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:30.214242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:30.214247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:30.214253Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1230 [2:7559342200776776243:9331]
2025-10-09T21:48:30.214268Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:30.214289Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:30.214297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:30.214300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:30.214303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:30.214322Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:30.214332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:30.214339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:30.214403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.214410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.214412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.214415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:30.214488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:30.214494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.214507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.214525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.214557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:30.214583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342201138599071:2432], now have 1 active actors on pipe
2025-10-09T21:48:30.214590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342201138599130:4932], now have 1 active actors on pipe
2025-10-09T21:48:30.214576Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1230
2025-10-09T21:48:30.214615Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1230
2025-10-09T21:48:30.214599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342200776776238:9331]
2025-10-09T21:48:30.214604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342200776776238:9331]
2025-10-09T21:48:30.214607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342200776776238:9331]
2025-10-09T21:48:30.214617Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:30.214630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:30.214632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:30.214634Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:30.214636Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:30.214648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:30.214652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:30.214709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.214719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.214733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342200776776238:9331]
2025-10-09T21:48:30.214742Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:30.214766Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:30.214775Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|23326cf2-9b990ee6-95ef121a-c443ce3c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:30.214783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:30.214787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:30.214791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:30.214801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:30.214815Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:30.214841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:30.214953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.214959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.214972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:30.215062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1231 partition: 0 MaxSeqNo: 25 sessionId: src_id|23326cf2-9b990ee6-95ef121a-c443ce3c_0
2025-10-09T21:48:30.215374Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d95622c4-da621a1-328cc0ec-47361342_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|23326cf2-9b990ee6-95ef121a-c443ce3c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:30.217871Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23326cf2-9b990ee6-95ef121a-c443ce3c_0] Write session: try to update token
2025-10-09T21:48:30.217891Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23326cf2-9b990ee6-95ef121a-c443ce3c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:30.229051Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:30.229070Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:30.229072Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:30.229074Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:30.229085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342200776776238:9331]
2025-10-09T21:48:30.229089Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342200776776238:9331]
2025-10-09T21:48:30.229093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342200776776238:9331]
2025-10-09T21:48:30.229142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:30.229151Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:30.229156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.229157Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:30.229159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:30.229162Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:30.229683Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:30.232352Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1231 sessionId: src_id|23326cf2-9b990ee6-95ef121a-c443ce3c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:30.234827Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:30.295526Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342201138599142:5175]
2025-10-09T21:48:30.295852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 1230, pipe: [1:7559342201138599142:5175]
2025-10-09T21:48:30.384981Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:30.385524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.385548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.385601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342201138599142:5175], now have 1 active actors on pipe
2025-10-09T21:48:30.385816Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:30.385839Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:30.385853Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:30.385891Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:30.385922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1231 reason: error at write: Disconnected sessionId: src_id|23326cf2-9b990ee6-95ef121a-c443ce3c_0
2025-10-09T21:48:30.385938Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342201138599142:5175]
2025-10-09T21:48:30.386112Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1231 sessionId: src_id|23326cf2-9b990ee6-95ef121a-c443ce3c_0 is DEAD
2025-10-09T21:48:30.386324Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23326cf2-9b990ee6-95ef121a-c443ce3c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:30.386334Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23326cf2-9b990ee6-95ef121a-c443ce3c_0] Write session will restart in 0.007548s
2025-10-09T21:48:30.386360Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23326cf2-9b990ee6-95ef121a-c443ce3c_0] Write session: Do CDS request
2025-10-09T21:48:30.386366Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23326cf2-9b990ee6-95ef121a-c443ce3c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:30.393977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:30.395511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23326cf2-9b990ee6-95ef121a-c443ce3c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:30.395559Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:30.395575Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1232
2025-10-09T21:48:30.395598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:30.395754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1232 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:30.395790Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:30.395797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1232 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:30.395803Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:30.395807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1232 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:30.395818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:30.395811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:30.395881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:30.395926Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342200776776300:9333]
2025-10-09T21:48:30.396181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1232 sessionId:  describe result for acl check
2025-10-09T21:48:30.396213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:30.396220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:30.396221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:30.396226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342201138599165:4927] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:30.396389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:30.396659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:30.396659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342201138599165:4927] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:30.396692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:30.396990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:30.397040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.397047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.397050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.397051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.397053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.397054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.397056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.397057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.397058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.397061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.397067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.397071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.397073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.397075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.397077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.397079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.397081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.397083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.397088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.397094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.397099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.397102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.397105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.397108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.397110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.397113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.397115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.397118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.397121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.397125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.397128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.397131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.397134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.397137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.397140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.397143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.397160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:30.398728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342201138599165:4927] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:30.403064Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342201138599165:4927] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:30.403083Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342201138599165:4927] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:30.403089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342201138599165:4927] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:30.403099Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1232 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:30.403375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:30.404372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:30.404385Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:30.404387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:30.404393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:30.404398Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1231 [2:7559342200776776300:9333]
2025-10-09T21:48:30.404407Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:30.404422Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:30.404428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:30.404430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:30.404432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:30.404446Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:30.404450Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:30.404456Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:30.404497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.404503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.404504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.404507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:30.404564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:30.404570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.404577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.404583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.404593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:30.404611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342200776776295:9333]
2025-10-09T21:48:30.404616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342200776776295:9333]
2025-10-09T21:48:30.404617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342200776776295:9333]
2025-10-09T21:48:30.404623Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:30.404634Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:30.404634Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:30.404637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:30.404639Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:30.404646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:30.404649Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:30.404658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342201138599204:4927], now have 1 active actors on pipe
2025-10-09T21:48:30.404660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342201138599145:2432], now have 1 active actors on pipe
2025-10-09T21:48:30.404684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342200776776295:9333]
2025-10-09T21:48:30.404696Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:30.404717Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:30.404719Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1231
2025-10-09T21:48:30.404721Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1231
2025-10-09T21:48:30.404809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.404821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.404850Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e67de7e7-c09be708-4abcc2d3-e9b2229_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:30.404871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:30.404880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:30.404883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:30.404897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:30.404906Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:30.404938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:30.405090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.405097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.405109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:30.405187Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1232 partition: 0 MaxSeqNo: 25 sessionId: src_id|e67de7e7-c09be708-4abcc2d3-e9b2229_0
2025-10-09T21:48:30.405512Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23326cf2-9b990ee6-95ef121a-c443ce3c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e67de7e7-c09be708-4abcc2d3-e9b2229_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:30.408036Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e67de7e7-c09be708-4abcc2d3-e9b2229_0] Write session: try to update token
2025-10-09T21:48:30.408056Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e67de7e7-c09be708-4abcc2d3-e9b2229_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:30.421947Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1232 sessionId: src_id|e67de7e7-c09be708-4abcc2d3-e9b2229_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:30.422388Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:30.422403Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:30.422414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342200776776295:9333]
2025-10-09T21:48:30.422467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:30.422480Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:30.422643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:30.422792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:30.422929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:30.423069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:30.423258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:30.423300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:30.424259Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:30.424441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.424455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.424490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:30.424553Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:30.429248Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:30.429264Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:30.429266Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:30.429268Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:30.429277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342200776776295:9333]
2025-10-09T21:48:30.429281Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342200776776295:9333]
2025-10-09T21:48:30.429285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342200776776295:9333]
2025-10-09T21:48:30.429321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:30.429328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.429331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:30.429332Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:30.429337Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:30.429341Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:30.429461Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:30.486530Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342201138599211:5175]
2025-10-09T21:48:30.486803Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 1231, pipe: [1:7559342201138599211:5175]
2025-10-09T21:48:30.584478Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:30.591531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.591554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.591706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:30.591768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:30.591828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:30.591898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:30.591940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:30.591950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:30.592019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:30.592077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:30.592141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:30.592206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:30.592251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:30.592259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:30.592313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:30.592365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:30.592423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:30.592484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:30.592521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:30.592528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:30.592589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:30.592644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:30.592705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:30.592769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:30.592809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:30.592812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:30.592871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:30.592932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:30.592992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:30.593048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:30.593091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:30.593098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:30.593159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:30.593212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:30.593270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:30.593324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:30.593361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:30.593367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:30.593421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:30.593476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:30.593536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:30.593593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:30.593634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:30.593641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:30.593698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:30.593763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:30.593819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:30.593878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:30.593938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:30.593946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:30.593999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:30.594060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:30.594112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:30.594162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:30.594201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:30.594209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:30.594263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:30.594311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:30.594362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:30.594411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:30.594447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:30.594452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:30.594489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342201138599211:5175], now have 1 active actors on pipe
2025-10-09T21:48:30.594686Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342201138599211:5175]
2025-10-09T21:48:30.594714Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:30.594718Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:30.594723Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:30.594760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1232 reason: error at write: Disconnected sessionId: src_id|e67de7e7-c09be708-4abcc2d3-e9b2229_0
2025-10-09T21:48:30.594958Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1232 sessionId: src_id|e67de7e7-c09be708-4abcc2d3-e9b2229_0 is DEAD
2025-10-09T21:48:30.595082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e67de7e7-c09be708-4abcc2d3-e9b2229_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:30.595093Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e67de7e7-c09be708-4abcc2d3-e9b2229_0] Write session will restart in 0.008131s
2025-10-09T21:48:30.595119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e67de7e7-c09be708-4abcc2d3-e9b2229_0] Write session: Do CDS request
2025-10-09T21:48:30.595133Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e67de7e7-c09be708-4abcc2d3-e9b2229_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:30.595322Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:30.602344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:30.603761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:30.603937Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:30.603948Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:30.603958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:30.604007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:30.604046Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342200776776388:9345]
2025-10-09T21:48:30.604519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e67de7e7-c09be708-4abcc2d3-e9b2229_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:30.604583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:30.604597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1233
2025-10-09T21:48:30.604644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:30.604765Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1233 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:30.604797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1233 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:30.604802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1233 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:30.604805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:30.604965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:30.605008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:30.605204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1233 sessionId:  describe result for acl check
2025-10-09T21:48:30.605235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:30.605243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:30.605244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:30.605250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342201138599239:4921] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:30.605318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:30.605367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.605373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.605374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.605375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.605376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.605377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.605378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.605379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.605380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.605385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.605395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.605399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.605402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.605405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.605409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.605412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.605416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.605420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.605427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.605434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.605440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.605446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.605450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.605455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.605460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.605465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.605469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.605475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.605480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.605485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.605489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.605495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.605503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.605508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.605512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.605517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.605546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:30.605794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342201138599239:4921] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:30.607842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342201138599239:4921] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:30.611068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342201138599239:4921] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:30.611081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342201138599239:4921] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:30.611086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342201138599239:4921] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:30.611097Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1233 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:30.611429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:30.612466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:30.612481Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:30.612484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:30.612488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:30.612495Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1232 [2:7559342200776776388:9345]
2025-10-09T21:48:30.612507Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:30.612533Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:30.612540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:30.612544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:30.612547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:30.612567Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:30.612578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:30.612595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:30.612647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.612654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.612656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.612658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:30.612725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:30.612737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.612751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.612763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.612785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:30.612810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342201138599276:4921], now have 1 active actors on pipe
2025-10-09T21:48:30.612817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342201138599216:2432], now have 1 active actors on pipe
2025-10-09T21:48:30.612824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342200776776383:9345]
2025-10-09T21:48:30.612830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342200776776383:9345]
2025-10-09T21:48:30.612832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342200776776383:9345]
2025-10-09T21:48:30.612844Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:30.612839Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1232
2025-10-09T21:48:30.612858Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:30.612863Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:30.612875Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:30.612876Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1232
2025-10-09T21:48:30.612878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:30.612878Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:30.612881Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:30.612913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342200776776383:9345]
2025-10-09T21:48:30.612923Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:30.612935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.612939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.612943Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:30.612978Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|79fc161-960aca2c-f7285d5-5b106bed_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:30.612990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:30.612993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:30.612995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:30.613005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:30.613015Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:30.613033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:30.613188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.613200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.613217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:30.613280Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1233 partition: 0 MaxSeqNo: 25 sessionId: src_id|79fc161-960aca2c-f7285d5-5b106bed_0
2025-10-09T21:48:30.613629Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e67de7e7-c09be708-4abcc2d3-e9b2229_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|79fc161-960aca2c-f7285d5-5b106bed_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:30.616182Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|79fc161-960aca2c-f7285d5-5b106bed_0] Write session: try to update token
2025-10-09T21:48:30.616199Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|79fc161-960aca2c-f7285d5-5b106bed_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:30.628203Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:30.628221Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:30.628223Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:30.628225Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:30.628234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342200776776383:9345]
2025-10-09T21:48:30.628238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342200776776383:9345]
2025-10-09T21:48:30.628242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342200776776383:9345]
2025-10-09T21:48:30.628289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:30.628296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.628299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:30.628414Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:30.628427Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:30.628434Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:30.628440Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:30.629695Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1233 sessionId: src_id|79fc161-960aca2c-f7285d5-5b106bed_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:30.632066Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:30.694888Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342201138599303:5175]
2025-10-09T21:48:30.695149Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 1232, pipe: [1:7559342201138599303:5175]
2025-10-09T21:48:30.783660Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:30.783965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.783981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.784030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342201138599303:5175], now have 1 active actors on pipe
2025-10-09T21:48:30.784250Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342201138599303:5175]
2025-10-09T21:48:30.784258Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:30.784281Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:30.784296Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:30.784300Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:30.784359Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1233 reason: error at write: Disconnected sessionId: src_id|79fc161-960aca2c-f7285d5-5b106bed_0
2025-10-09T21:48:30.784572Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1233 sessionId: src_id|79fc161-960aca2c-f7285d5-5b106bed_0 is DEAD
2025-10-09T21:48:30.784702Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79fc161-960aca2c-f7285d5-5b106bed_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:30.784724Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79fc161-960aca2c-f7285d5-5b106bed_0] Write session will restart in 0.009564s
2025-10-09T21:48:30.784752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79fc161-960aca2c-f7285d5-5b106bed_0] Write session: Do CDS request
2025-10-09T21:48:30.784758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79fc161-960aca2c-f7285d5-5b106bed_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:30.791835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:30.793428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:30.793590Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:30.793602Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:30.793614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:30.793666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:30.793721Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342200776776459:9351]
2025-10-09T21:48:30.794316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:30.794591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:30.794629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:30.794922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:30.794961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.794967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.794971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.794972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.794973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.794974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.794975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.794976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.794978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.794983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.794994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.794997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.795001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.795005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.795008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.795012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.795017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.795021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.795028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.795041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.795048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.795058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.795062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.795067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.795074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.795079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.795088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.795093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.795097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.795106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.795110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.795116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.795121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.795127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.795134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.795139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.795170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:30.795475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|79fc161-960aca2c-f7285d5-5b106bed_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:30.795545Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:30.795560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1234
2025-10-09T21:48:30.795689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1234 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:30.795717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1234 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:30.795728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1234 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:30.795731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:30.796034Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1234 sessionId:  describe result for acl check
2025-10-09T21:48:30.796065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:30.796073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:30.796077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:30.796083Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342201138599332:4909] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:30.796536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342201138599332:4909] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:30.798608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342201138599332:4909] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:30.801280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:30.801506Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342201138599332:4909] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:30.801521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342201138599332:4909] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:30.801525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342201138599332:4909] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:30.801536Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1234 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:30.802294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:30.802306Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:30.802310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:30.802313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:30.802318Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1233 [2:7559342200776776459:9351]
2025-10-09T21:48:30.802327Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:30.802341Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:30.802342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:30.802345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:30.802347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:30.802359Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:30.802363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:30.802369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:30.802410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.802415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.802416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.802418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:30.802471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:30.802478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.802489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.802502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.802511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:30.802530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342201138599366:4909], now have 1 active actors on pipe
2025-10-09T21:48:30.802536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342201138599305:2432], now have 1 active actors on pipe
2025-10-09T21:48:30.802542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342200776776454:9351]
2025-10-09T21:48:30.802543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342200776776454:9351]
2025-10-09T21:48:30.802545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342200776776454:9351]
2025-10-09T21:48:30.802553Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:30.802567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:30.802567Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:30.802570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:30.802573Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:30.802586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:30.802590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:30.802584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1233
2025-10-09T21:48:30.802603Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1233
2025-10-09T21:48:30.802613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342200776776454:9351]
2025-10-09T21:48:30.802619Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:30.802622Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:30.802632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.802638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.802676Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a0fc873c-5e35ff24-257d177b-22fea3ea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:30.802685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:30.802688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:30.802689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:30.802696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:30.802703Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:30.802719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:30.802864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.802875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.802896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:30.803002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1234 partition: 0 MaxSeqNo: 25 sessionId: src_id|a0fc873c-5e35ff24-257d177b-22fea3ea_0
2025-10-09T21:48:30.803342Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79fc161-960aca2c-f7285d5-5b106bed_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a0fc873c-5e35ff24-257d177b-22fea3ea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:30.805770Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a0fc873c-5e35ff24-257d177b-22fea3ea_0] Write session: try to update token
2025-10-09T21:48:30.805787Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a0fc873c-5e35ff24-257d177b-22fea3ea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:30.816636Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:30.816654Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:30.816657Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:30.816659Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:30.816670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342200776776454:9351]
2025-10-09T21:48:30.816679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342200776776454:9351]
2025-10-09T21:48:30.816682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342200776776454:9351]
2025-10-09T21:48:30.816727Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:30.816729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:30.816733Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:30.816735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.816738Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:30.816738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:30.816873Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:30.818857Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1234 sessionId: src_id|a0fc873c-5e35ff24-257d177b-22fea3ea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:30.821104Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:30.884368Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342201138599376:5175]
2025-10-09T21:48:30.884613Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 1233, pipe: [1:7559342201138599376:5175]
2025-10-09T21:48:30.971504Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:30.971779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.971794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.971840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342201138599376:5175], now have 1 active actors on pipe
2025-10-09T21:48:30.972059Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342201138599376:5175]
2025-10-09T21:48:30.972092Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:30.972199Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:30.972208Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:30.972272Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1234 reason: error at write: Disconnected sessionId: src_id|a0fc873c-5e35ff24-257d177b-22fea3ea_0
2025-10-09T21:48:30.972420Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1234 sessionId: src_id|a0fc873c-5e35ff24-257d177b-22fea3ea_0 is DEAD
2025-10-09T21:48:30.972517Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:30.972583Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0fc873c-5e35ff24-257d177b-22fea3ea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:30.972594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0fc873c-5e35ff24-257d177b-22fea3ea_0] Write session will restart in 0.008290s
2025-10-09T21:48:30.972621Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0fc873c-5e35ff24-257d177b-22fea3ea_0] Write session: Do CDS request
2025-10-09T21:48:30.972629Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0fc873c-5e35ff24-257d177b-22fea3ea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:30.979685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:30.981123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:30.981317Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:30.981326Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:30.981338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:30.981392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:30.981431Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342200776776519:9353]
2025-10-09T21:48:30.981861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:30.982173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:30.982235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:30.982527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a0fc873c-5e35ff24-257d177b-22fea3ea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:30.982564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:30.982618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.982624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.982625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.982626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.982627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.982629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.982630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.982631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.982632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.982621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:30.982637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1235
2025-10-09T21:48:30.982637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.982649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.982653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.982656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.982660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.982664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.982668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.982671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.982675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.982684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.982692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:30.982698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.982702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:30.982706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.982710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:30.982714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.982718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:30.982727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.982732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:30.982736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.982741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:30.982747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.982752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.982755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.982760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.982764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.982769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.982800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:30.982773Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1235 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:30.982803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1235 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:30.982809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1235 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:30.982812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:30.983212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1235 sessionId:  describe result for acl check
2025-10-09T21:48:30.983244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:30.983251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:30.983252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:30.983258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342201138599407:4896] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:30.983786Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342201138599407:4896] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:30.986065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342201138599407:4896] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:30.988472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342201138599407:4896] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:30.988488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342201138599407:4896] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:30.988493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342201138599407:4896] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:30.988506Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1235 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:30.988822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:30.989877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:30.989891Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:30.989893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:30.989898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:30.989905Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1234 [2:7559342200776776519:9353]
2025-10-09T21:48:30.989918Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:30.989940Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:30.989944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:30.989947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:30.989950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:30.989969Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:30.989978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:30.989985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:30.990067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.990076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.990078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.990081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:30.990157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:30.990165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:30.990179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:30.990191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:30.990208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:30.990235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342201138599380:2432], now have 1 active actors on pipe
2025-10-09T21:48:30.990244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342201138599439:4896], now have 1 active actors on pipe
2025-10-09T21:48:30.990251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342200776776514:9353]
2025-10-09T21:48:30.990254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342200776776514:9353]
2025-10-09T21:48:30.990255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342200776776514:9353]
2025-10-09T21:48:30.990254Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1234
2025-10-09T21:48:30.990262Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1234
2025-10-09T21:48:30.990270Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:30.990291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:30.990307Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:30.990307Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:30.990310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:30.990311Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:30.990312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:30.990353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.990371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.990390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342200776776514:9353]
2025-10-09T21:48:30.990397Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:30.990411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:30.990427Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2cd4b93c-82dd5fbd-5c7790c9-b3ab747c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:30.990439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:30.990449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:30.990452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:30.990462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:30.990477Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:30.990501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:30.990654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:30.990663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:30.990676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:30.990787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1235 partition: 0 MaxSeqNo: 25 sessionId: src_id|2cd4b93c-82dd5fbd-5c7790c9-b3ab747c_0
2025-10-09T21:48:30.991112Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0fc873c-5e35ff24-257d177b-22fea3ea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2cd4b93c-82dd5fbd-5c7790c9-b3ab747c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:30.993472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2cd4b93c-82dd5fbd-5c7790c9-b3ab747c_0] Write session: try to update token
2025-10-09T21:48:30.993489Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2cd4b93c-82dd5fbd-5c7790c9-b3ab747c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:31.004818Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:31.004839Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:31.004842Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:31.004844Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:31.004854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342200776776514:9353]
2025-10-09T21:48:31.004859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342200776776514:9353]
2025-10-09T21:48:31.004866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342200776776514:9353]
2025-10-09T21:48:31.004916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:31.004917Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:31.004924Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:31.004928Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:31.004929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.004931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:31.005072Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:31.007160Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1235 sessionId: src_id|2cd4b93c-82dd5fbd-5c7790c9-b3ab747c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:31.009579Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:31.072252Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342205433566745:5175]
2025-10-09T21:48:31.072514Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 1234, pipe: [1:7559342205433566745:5175]
2025-10-09T21:48:31.160163Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:31.160532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:31.160546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:31.160593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342205433566745:5175], now have 1 active actors on pipe
2025-10-09T21:48:31.160812Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:31.160842Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:31.160847Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:31.160869Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342205433566745:5175]
2025-10-09T21:48:31.160939Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1235 reason: error at write: Disconnected sessionId: src_id|2cd4b93c-82dd5fbd-5c7790c9-b3ab747c_0
2025-10-09T21:48:31.161142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1235 sessionId: src_id|2cd4b93c-82dd5fbd-5c7790c9-b3ab747c_0 is DEAD
2025-10-09T21:48:31.161300Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2cd4b93c-82dd5fbd-5c7790c9-b3ab747c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:31.161310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2cd4b93c-82dd5fbd-5c7790c9-b3ab747c_0] Write session will restart in 0.005692s
2025-10-09T21:48:31.161328Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2cd4b93c-82dd5fbd-5c7790c9-b3ab747c_0] Write session: Do CDS request
2025-10-09T21:48:31.161345Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2cd4b93c-82dd5fbd-5c7790c9-b3ab747c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:31.161593Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:31.168475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2cd4b93c-82dd5fbd-5c7790c9-b3ab747c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:31.168606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:31.168621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1236
2025-10-09T21:48:31.168763Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1236 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:31.168824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1236 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:31.168834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1236 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:31.168838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:31.169288Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1236 sessionId:  describe result for acl check
2025-10-09T21:48:31.169323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:31.169330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:31.169332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:31.169337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342205433566773:4892] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:31.169547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:31.169919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342205433566773:4892] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:31.171048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:31.171220Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:31.171231Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:31.171242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:31.171293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:31.171330Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342205071743877:9355]
2025-10-09T21:48:31.171918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:31.171979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342205433566773:4892] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:31.172214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:31.172257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:31.172578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:31.172623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.172629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.172630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.172631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.172633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.172634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.172635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.172638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.172639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.172644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.172655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.172659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.172663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.172668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.172672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.172676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.172679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.172682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.172689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.172697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.172703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.172709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.172713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.172718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.172728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.172733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.172737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.172742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.172748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.172753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.172762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.172768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.172773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.172783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.172788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.172797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.172827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:31.178003Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342205433566773:4892] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:31.178023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342205433566773:4892] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:31.178027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342205433566773:4892] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:31.178057Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1236 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:31.179401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:31.180402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:31.180419Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:31.180421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:31.180425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:31.180435Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1235 [2:7559342205071743877:9355]
2025-10-09T21:48:31.180449Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:31.180471Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:31.180474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:31.180477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:31.180480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:31.180503Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:31.180507Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:31.180514Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:31.180569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.180577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.180579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.180581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:31.180655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:31.180662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.180676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.180694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.180712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:31.180734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342205433566814:4892], now have 1 active actors on pipe
2025-10-09T21:48:31.180740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342205433566750:2432], now have 1 active actors on pipe
2025-10-09T21:48:31.180749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342205071743872:9355]
2025-10-09T21:48:31.180751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342205071743872:9355]
2025-10-09T21:48:31.180753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342205071743872:9355]
2025-10-09T21:48:31.180765Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:31.180778Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:31.180780Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:31.180790Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:31.180793Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:31.180793Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:31.180798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:31.180808Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1235
2025-10-09T21:48:31.180808Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1235
2025-10-09T21:48:31.180838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342205071743872:9355]
2025-10-09T21:48:31.180848Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:31.180851Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:31.180870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:31.180874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:31.180925Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a36e6102-7b1866bc-1501f33e-7fd67dc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:31.180938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:31.180941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:31.180944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:31.180955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:31.180965Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:31.180989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:31.181152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:31.181161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:31.181178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:31.181283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1236 partition: 0 MaxSeqNo: 25 sessionId: src_id|a36e6102-7b1866bc-1501f33e-7fd67dc_0
2025-10-09T21:48:31.181646Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2cd4b93c-82dd5fbd-5c7790c9-b3ab747c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a36e6102-7b1866bc-1501f33e-7fd67dc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:31.184114Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a36e6102-7b1866bc-1501f33e-7fd67dc_0] Write session: try to update token
2025-10-09T21:48:31.184130Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a36e6102-7b1866bc-1501f33e-7fd67dc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:31.195378Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:31.195394Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:31.195396Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:31.195400Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:31.195409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342205071743872:9355]
2025-10-09T21:48:31.195413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342205071743872:9355]
2025-10-09T21:48:31.195415Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342205071743872:9355]
2025-10-09T21:48:31.195450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:31.195460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.195460Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:31.195462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:31.195466Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:31.195471Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:31.195578Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:31.196995Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1236 sessionId: src_id|a36e6102-7b1866bc-1501f33e-7fd67dc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:31.199369Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:31.261186Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342205433566827:5175]
2025-10-09T21:48:31.261452Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 1235, pipe: [1:7559342205433566827:5175]
2025-10-09T21:48:31.352013Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:31.352413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:31.352429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:31.352477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342205433566827:5175], now have 1 active actors on pipe
2025-10-09T21:48:31.352690Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:31.352696Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:31.352712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:31.352718Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342205433566827:5175]
2025-10-09T21:48:31.352742Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:31.352774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1236 reason: error at write: Disconnected sessionId: src_id|a36e6102-7b1866bc-1501f33e-7fd67dc_0
2025-10-09T21:48:31.352954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1236 sessionId: src_id|a36e6102-7b1866bc-1501f33e-7fd67dc_0 is DEAD
2025-10-09T21:48:31.353107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a36e6102-7b1866bc-1501f33e-7fd67dc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:31.353118Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a36e6102-7b1866bc-1501f33e-7fd67dc_0] Write session will restart in 0.005023s
2025-10-09T21:48:31.353146Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a36e6102-7b1866bc-1501f33e-7fd67dc_0] Write session: Do CDS request
2025-10-09T21:48:31.353154Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a36e6102-7b1866bc-1501f33e-7fd67dc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:31.359504Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a36e6102-7b1866bc-1501f33e-7fd67dc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:31.359555Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:31.359567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1237
2025-10-09T21:48:31.359857Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1237 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:31.359893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1237 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:31.359899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1237 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:31.359903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:31.360375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1237 sessionId:  describe result for acl check
2025-10-09T21:48:31.360421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:31.360428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:31.360430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:31.360437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342205433566852:4887] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:31.361043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342205433566852:4887] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:31.361631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:31.363008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342205433566852:4887] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:31.363058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:31.363254Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:31.363263Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:31.363275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:31.363328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:31.363366Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342205071743939:9357]
2025-10-09T21:48:31.363956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:31.364269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:31.364321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:31.364639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:31.364701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.364709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.364710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.364711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.364713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.364714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.364716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.364717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.364720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.364725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.364735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.364739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.364742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.364745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.364749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.364753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.364756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.364760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.364767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.364775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.364781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.364786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.364791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.364797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.364801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.364807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.364811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.364817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.364826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.364832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.364840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.364850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.364858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.364864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.364873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.364878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.364911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:31.365219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342205433566852:4887] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:31.365232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342205433566852:4887] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:31.365235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342205433566852:4887] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:31.365240Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1237 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:31.371285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:31.372298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:31.372315Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:31.372321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:31.372326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:31.372332Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1236 [2:7559342205071743939:9357]
2025-10-09T21:48:31.372345Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:31.372362Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:31.372369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:31.372372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:31.372374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:31.372390Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:31.372403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:31.372410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:31.372466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.372473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.372475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.372478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:31.372552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:31.372559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.372570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.372581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.372602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:31.372628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342205433566830:2432], now have 1 active actors on pipe
2025-10-09T21:48:31.372636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342205433566894:4887], now have 1 active actors on pipe
2025-10-09T21:48:31.372644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342205071743934:9357]
2025-10-09T21:48:31.372648Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342205071743934:9357]
2025-10-09T21:48:31.372651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342205071743934:9357]
2025-10-09T21:48:31.372666Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:31.372684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:31.372703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:31.372705Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:31.372709Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:31.372710Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:31.372711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:31.372703Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1236
2025-10-09T21:48:31.372740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342205071743934:9357]
2025-10-09T21:48:31.372747Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:31.372748Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:31.372810Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1236
2025-10-09T21:48:31.372856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:31.372869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:31.372897Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2ed92c3b-f96bc401-be7d21c8-8131bbf3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:31.372909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:31.372913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:31.372916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:31.372926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:31.372939Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:31.372956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:31.373099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:31.373114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:31.373136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:31.373234Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1237 partition: 0 MaxSeqNo: 25 sessionId: src_id|2ed92c3b-f96bc401-be7d21c8-8131bbf3_0
2025-10-09T21:48:31.373566Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a36e6102-7b1866bc-1501f33e-7fd67dc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2ed92c3b-f96bc401-be7d21c8-8131bbf3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:31.375917Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed92c3b-f96bc401-be7d21c8-8131bbf3_0] Write session: try to update token
2025-10-09T21:48:31.375934Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed92c3b-f96bc401-be7d21c8-8131bbf3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:31.386903Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:31.386924Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:31.386927Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:31.386929Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:31.386942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342205071743934:9357]
2025-10-09T21:48:31.386946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342205071743934:9357]
2025-10-09T21:48:31.386950Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342205071743934:9357]
2025-10-09T21:48:31.386995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:31.386996Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:31.387002Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:31.387005Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:31.387007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.387013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:31.387158Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:31.389352Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1237 sessionId: src_id|2ed92c3b-f96bc401-be7d21c8-8131bbf3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:31.391792Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:31.452657Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342205433566901:5175]
2025-10-09T21:48:31.452979Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 1236, pipe: [1:7559342205433566901:5175]
2025-10-09T21:48:31.535624Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:31.535994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:31.536010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:31.536054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342205433566901:5175], now have 1 active actors on pipe
2025-10-09T21:48:31.536254Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:31.536244Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:31.536300Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342205433566901:5175]
2025-10-09T21:48:31.536325Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:31.536410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1237 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2ed92c3b-f96bc401-be7d21c8-8131bbf3_0
2025-10-09T21:48:31.536595Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1237 sessionId: src_id|2ed92c3b-f96bc401-be7d21c8-8131bbf3_0 is DEAD
2025-10-09T21:48:31.536787Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed92c3b-f96bc401-be7d21c8-8131bbf3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:31.536799Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed92c3b-f96bc401-be7d21c8-8131bbf3_0] Write session will restart in 0.007366s
2025-10-09T21:48:31.536828Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed92c3b-f96bc401-be7d21c8-8131bbf3_0] Write session: Do CDS request
2025-10-09T21:48:31.536835Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed92c3b-f96bc401-be7d21c8-8131bbf3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:31.544512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:31.545512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed92c3b-f96bc401-be7d21c8-8131bbf3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:31.545590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:31.545604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1238
2025-10-09T21:48:31.545720Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1238 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:31.545751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1238 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:31.545763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1238 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:31.545771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:31.546049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:31.546188Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1238 sessionId:  describe result for acl check
2025-10-09T21:48:31.546225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:31.546232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:31.546234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:31.546242Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:31.546239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342205433566930:4882] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:31.546252Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:31.546264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:31.546317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:31.546358Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342205071744011:9359]
2025-10-09T21:48:31.546765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342205433566930:4882] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:31.546989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:31.547282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:31.547329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:31.547656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:31.547707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.547713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.547715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.547716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.547717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.547719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.547720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.547721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.547722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.547729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.547739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.547743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.547747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.547751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.547755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.547758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.547763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.547767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.547773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.547783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.547789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.547793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.547799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.547804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.547808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.547818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.547822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.547827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.547831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.547842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.547846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.547853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.547857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.547862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.547869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.547874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.547898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:31.548692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342205433566930:4882] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:31.553600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342205433566930:4882] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:31.553614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342205433566930:4882] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:31.553618Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342205433566930:4882] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:31.553626Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1238 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:31.554085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:31.555125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:31.555142Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:31.555144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:31.555153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:31.555159Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1237 [2:7559342205071744011:9359]
2025-10-09T21:48:31.555171Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:31.555187Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:31.555192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:31.555194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:31.555195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:31.555207Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:31.555210Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:31.555214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:31.555267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.555277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.555278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.555282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:31.555345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:31.555355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.555367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.555380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.555391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:31.555407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342205433566969:4882], now have 1 active actors on pipe
2025-10-09T21:48:31.555415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342205433566904:2432], now have 1 active actors on pipe
2025-10-09T21:48:31.555424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342205071744002:9359]
2025-10-09T21:48:31.555426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342205071744002:9359]
2025-10-09T21:48:31.555428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342205071744002:9359]
2025-10-09T21:48:31.555444Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:31.555462Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:31.555464Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:31.555468Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:31.555471Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:31.555482Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:31.555486Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:31.555464Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1237
2025-10-09T21:48:31.555466Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1237
2025-10-09T21:48:31.555514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342205071744002:9359]
2025-10-09T21:48:31.555524Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:31.555531Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:31.555540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:31.555545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:31.555591Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f75cfff6-63d3e254-ae519d80-9dbdbb14_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:31.555600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:31.555604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:31.555605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:31.555615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:31.555627Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:31.555645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:31.555794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:31.555801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:31.555814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:31.555926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1238 partition: 0 MaxSeqNo: 25 sessionId: src_id|f75cfff6-63d3e254-ae519d80-9dbdbb14_0
2025-10-09T21:48:31.556258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed92c3b-f96bc401-be7d21c8-8131bbf3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f75cfff6-63d3e254-ae519d80-9dbdbb14_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:31.558631Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f75cfff6-63d3e254-ae519d80-9dbdbb14_0] Write session: try to update token
2025-10-09T21:48:31.558647Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f75cfff6-63d3e254-ae519d80-9dbdbb14_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:31.570463Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:31.570480Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:31.570482Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:31.570484Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:31.570493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342205071744002:9359]
2025-10-09T21:48:31.570497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342205071744002:9359]
2025-10-09T21:48:31.570500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342205071744002:9359]
2025-10-09T21:48:31.570541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:31.570549Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:31.570552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.570555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:31.570561Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:31.570564Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:31.570681Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:31.571642Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1238 sessionId: src_id|f75cfff6-63d3e254-ae519d80-9dbdbb14_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:31.574170Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:31.636484Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342205433566976:5175]
2025-10-09T21:48:31.636782Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 1237, pipe: [1:7559342205433566976:5175]
2025-10-09T21:48:31.722801Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:31.723147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:31.723161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:31.723201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342205433566976:5175], now have 1 active actors on pipe
2025-10-09T21:48:31.723400Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:31.723413Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:31.723479Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:31.723482Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342205433566976:5175]
2025-10-09T21:48:31.723556Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1238 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f75cfff6-63d3e254-ae519d80-9dbdbb14_0
2025-10-09T21:48:31.723735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1238 sessionId: src_id|f75cfff6-63d3e254-ae519d80-9dbdbb14_0 is DEAD
2025-10-09T21:48:31.723901Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f75cfff6-63d3e254-ae519d80-9dbdbb14_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:31.723912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f75cfff6-63d3e254-ae519d80-9dbdbb14_0] Write session will restart in 0.007341s
2025-10-09T21:48:31.723939Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f75cfff6-63d3e254-ae519d80-9dbdbb14_0] Write session: Do CDS request
2025-10-09T21:48:31.723947Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f75cfff6-63d3e254-ae519d80-9dbdbb14_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:31.731881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:31.732528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f75cfff6-63d3e254-ae519d80-9dbdbb14_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:31.732602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:31.732618Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1239
2025-10-09T21:48:31.732732Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1239 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:31.732765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1239 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:31.732775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1239 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:31.732783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:31.733222Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1239 sessionId:  describe result for acl check
2025-10-09T21:48:31.733263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:31.733265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:31.733267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:31.733273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342205433567033:4852] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:31.733435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:31.733610Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:31.733620Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:31.733631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:31.733684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:31.733736Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342205071744109:9375]
2025-10-09T21:48:31.733832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342205433567033:4852] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:31.734323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:31.734625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:31.734683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:31.735013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:31.735064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.735074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.735075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.735077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.735078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.735079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.735082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.735083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.735084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.735090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.735099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.735103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.735106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.735110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.735115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.735119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.735123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.735127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.735134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.735142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.735148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.735153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.735157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.735163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.735166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.735171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.735175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.735179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.735183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.735187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.735194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.735199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.735203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.735208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.735212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.735217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.735250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:31.735995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342205433567033:4852] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:31.741306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342205433567033:4852] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:31.741324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342205433567033:4852] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:31.741328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342205433567033:4852] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:31.741343Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1239 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:31.741590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:31.742662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:31.742680Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:31.742683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:31.742687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:31.742694Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1238 [2:7559342205071744109:9375]
2025-10-09T21:48:31.742707Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:31.742723Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:31.742726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:31.742731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:31.742733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:31.742749Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:31.742755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:31.742765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:31.742824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.742830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.742833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.742835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:31.742902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:31.742909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.742921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.742932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.742953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:31.742977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342205433567003:2432], now have 1 active actors on pipe
2025-10-09T21:48:31.742985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342205433567069:4852], now have 1 active actors on pipe
2025-10-09T21:48:31.742993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342205071744104:9375]
2025-10-09T21:48:31.742996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342205071744104:9375]
2025-10-09T21:48:31.742999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342205071744104:9375]
2025-10-09T21:48:31.743012Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:31.743023Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:31.743026Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:31.743028Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:31.743030Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:31.743045Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:31.743052Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:31.743039Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1238
2025-10-09T21:48:31.743049Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1238
2025-10-09T21:48:31.743082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342205071744104:9375]
2025-10-09T21:48:31.743092Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:31.743099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:31.743111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:31.743116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:31.743162Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|eb384825-22368c96-26e1d00b-f383c044_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:31.743170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:31.743174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:31.743175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:31.743185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:31.743200Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:31.743223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:31.743369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:31.743378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:31.743392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:31.743488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1239 partition: 0 MaxSeqNo: 25 sessionId: src_id|eb384825-22368c96-26e1d00b-f383c044_0
2025-10-09T21:48:31.743862Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f75cfff6-63d3e254-ae519d80-9dbdbb14_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|eb384825-22368c96-26e1d00b-f383c044_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:31.746369Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eb384825-22368c96-26e1d00b-f383c044_0] Write session: try to update token
2025-10-09T21:48:31.746390Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eb384825-22368c96-26e1d00b-f383c044_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:31.757480Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:31.757496Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:31.757498Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:31.757501Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:31.757511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342205071744104:9375]
2025-10-09T21:48:31.757516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342205071744104:9375]
2025-10-09T21:48:31.757523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342205071744104:9375]
2025-10-09T21:48:31.757569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:31.757569Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:31.757576Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:31.757580Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:31.757583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.757586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:31.757727Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:31.759412Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1239 sessionId: src_id|eb384825-22368c96-26e1d00b-f383c044_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:31.761691Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:31.823868Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342205433567078:5175]
2025-10-09T21:48:31.824163Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 1238, pipe: [1:7559342205433567078:5175]
2025-10-09T21:48:31.906507Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:31.906825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:31.906840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:31.906887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342205433567078:5175], now have 1 active actors on pipe
2025-10-09T21:48:31.907087Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:31.907118Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:31.907142Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:31.907148Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:31.907192Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342205433567078:5175]
2025-10-09T21:48:31.907211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1239 reason: error at write: Disconnected sessionId: src_id|eb384825-22368c96-26e1d00b-f383c044_0
2025-10-09T21:48:31.907388Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1239 sessionId: src_id|eb384825-22368c96-26e1d00b-f383c044_0 is DEAD
2025-10-09T21:48:31.907555Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb384825-22368c96-26e1d00b-f383c044_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:31.907566Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb384825-22368c96-26e1d00b-f383c044_0] Write session will restart in 0.007471s
2025-10-09T21:48:31.907594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb384825-22368c96-26e1d00b-f383c044_0] Write session: Do CDS request
2025-10-09T21:48:31.907600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb384825-22368c96-26e1d00b-f383c044_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:31.915796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:31.916505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eb384825-22368c96-26e1d00b-f383c044_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:31.916567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:31.916582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1240
2025-10-09T21:48:31.916700Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1240 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:31.916731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1240 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:31.916742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1240 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:31.916749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:31.917166Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1240 sessionId:  describe result for acl check
2025-10-09T21:48:31.917155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:31.917202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:31.917209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:31.917211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:31.917217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342205433567109:4850] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:31.917325Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:31.917338Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:31.917350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:31.917403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:31.917441Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342205071744174:9377]
2025-10-09T21:48:31.917761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342205433567109:4850] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:31.918009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:31.918287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:31.918322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:31.918550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:31.918592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.918598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.918599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.918602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.918604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.918605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.918606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.918607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.918608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.918612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.918618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.918620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.918623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.918625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.918627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.918629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.918631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.918633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.918637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.918642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:31.918646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.918649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:31.918652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.918655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:31.918657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.918660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:31.918664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.918667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:31.918669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.918673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:31.918676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.918678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.918681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.918683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.918686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.918689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.918705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:31.919927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342205433567109:4850] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:31.924190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342205433567109:4850] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:31.924204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342205433567109:4850] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:31.924208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342205433567109:4850] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:31.924217Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1240 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:31.924556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:31.925517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:31.925532Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:31.925534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:31.925538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:31.925543Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1239 [2:7559342205071744174:9377]
2025-10-09T21:48:31.925552Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:31.925567Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:31.925568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:31.925570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:31.925572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:31.925584Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:31.925588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:31.925594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:31.925635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.925642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.925643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.925645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:31.925695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:31.925700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.925719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:31.925724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:31.925734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:31.925749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342205071744169:9377]
2025-10-09T21:48:31.925754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342205071744169:9377]
2025-10-09T21:48:31.925755Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342205071744169:9377]
2025-10-09T21:48:31.925761Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:31.925771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342205433567081:2432], now have 1 active actors on pipe
2025-10-09T21:48:31.925773Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:31.925776Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:31.925778Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:31.925780Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:31.925788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:31.925790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:31.925793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342205433567148:4850], now have 1 active actors on pipe
2025-10-09T21:48:31.925813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342205071744169:9377]
2025-10-09T21:48:31.925821Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:31.925829Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:31.925868Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1239
2025-10-09T21:48:31.925878Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1239
2025-10-09T21:48:31.925920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:31.925927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:31.925949Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1548ceeb-490ac20b-5494264a-1c8df840_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:31.925957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:31.925962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:31.925964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:31.925970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:31.925978Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:31.925993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:31.926205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:31.926210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:31.926222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:31.926312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1240 partition: 0 MaxSeqNo: 25 sessionId: src_id|1548ceeb-490ac20b-5494264a-1c8df840_0
2025-10-09T21:48:31.926655Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb384825-22368c96-26e1d00b-f383c044_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1548ceeb-490ac20b-5494264a-1c8df840_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:31.928989Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1548ceeb-490ac20b-5494264a-1c8df840_0] Write session: try to update token
2025-10-09T21:48:31.929005Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1548ceeb-490ac20b-5494264a-1c8df840_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:31.936856Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:31.936871Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:31.936881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342205071744169:9377]
2025-10-09T21:48:31.936919Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:31.936925Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:31.937069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:31.937201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:31.937321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:31.937452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:31.937624Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:31.937656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:31.941445Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1240 sessionId: src_id|1548ceeb-490ac20b-5494264a-1c8df840_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:31.943755Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:31.943965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:31.943989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:31.944033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:31.944117Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:31.949152Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:31.949169Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:31.949171Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:31.949173Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:31.949184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342205071744169:9377]
2025-10-09T21:48:31.949188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342205071744169:9377]
2025-10-09T21:48:31.949192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342205071744169:9377]
2025-10-09T21:48:31.949228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:31.949235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:31.949237Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:31.949239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:31.949244Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:31.949248Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:31.949388Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:32.007142Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342209728534455:5175]
2025-10-09T21:48:32.007423Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 1239, pipe: [1:7559342209728534455:5175]
2025-10-09T21:48:32.107715Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:32.115672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:32.115701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:32.115885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:32.115953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:32.116026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:32.116094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:32.116138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:32.116148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:32.116218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:32.116281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:32.116344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:32.116414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:32.116459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:32.116467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:32.116535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:32.116595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:32.116662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:32.116726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:32.116776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:32.116784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:32.116853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:32.116919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:32.116986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:32.117052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:32.117099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:32.117107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:32.117175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:32.117235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:32.117298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:32.117357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:32.117400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:32.117409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:32.117475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:32.117544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:32.117609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:32.117674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:32.117731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:32.117742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:32.117806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:32.117865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:32.117934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:32.117998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:32.118060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:32.118069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:32.118129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:32.118191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:32.118256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:32.118319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:32.118365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:32.118373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:32.118439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:32.118497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:32.118556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:32.118622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:32.118666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:32.118673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:32.118734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:32.118793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:32.118860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:32.118926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:32.118973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:32.118981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:32.119048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342209728534455:5175], now have 1 active actors on pipe
2025-10-09T21:48:32.119300Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:32.119333Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342209728534455:5175]
2025-10-09T21:48:32.119357Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:32.119363Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:32.119393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1240 reason: error at write: Disconnected sessionId: src_id|1548ceeb-490ac20b-5494264a-1c8df840_0
2025-10-09T21:48:32.119592Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1240 sessionId: src_id|1548ceeb-490ac20b-5494264a-1c8df840_0 is DEAD
2025-10-09T21:48:32.119719Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1548ceeb-490ac20b-5494264a-1c8df840_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:32.119742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1548ceeb-490ac20b-5494264a-1c8df840_0] Write session will restart in 0.005692s
2025-10-09T21:48:32.119768Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1548ceeb-490ac20b-5494264a-1c8df840_0] Write session: Do CDS request
2025-10-09T21:48:32.119774Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1548ceeb-490ac20b-5494264a-1c8df840_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:32.120173Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:32.126460Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1548ceeb-490ac20b-5494264a-1c8df840_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:32.126569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:32.126583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1241
2025-10-09T21:48:32.126689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1241 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:32.126715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1241 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:32.126723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1241 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:32.126726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:32.127076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1241 sessionId:  describe result for acl check
2025-10-09T21:48:32.127108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:32.127112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:32.127114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:32.127119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342209728534485:4840] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:32.127534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342209728534485:4840] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:32.128430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:32.130050Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342209728534485:4840] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:32.130077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:32.130263Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:32.130275Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:32.130287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:32.130346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:32.130386Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342209366711536:9379]
2025-10-09T21:48:32.131010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:32.131263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:32.131307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:32.131616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:32.131664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.131670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.131671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.131672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.131673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.131675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.131676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.131677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.131678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.131683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.131694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.131698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.131704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.131708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.131712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.131716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.131719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.131724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.131730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.131738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.131745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.131753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.131757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.131762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.131766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.131771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.131777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.131783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.131787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.131793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.131803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.131807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.131811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.131816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.131821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.131825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.131857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:32.133926Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342209728534485:4840] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:32.133940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342209728534485:4840] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:32.133944Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342209728534485:4840] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:32.133956Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1241 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:32.138194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:32.139228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:32.139240Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:32.139243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:32.139249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:32.139256Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1240 [2:7559342209366711536:9379]
2025-10-09T21:48:32.139270Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:32.139288Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:32.139298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:32.139301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:32.139303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:32.139319Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:32.139331Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:32.139338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:32.139404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.139411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.139413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.139416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:32.139505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:32.139512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.139524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.139536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.139559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:32.139582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342209728534461:2432], now have 1 active actors on pipe
2025-10-09T21:48:32.139592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342209728534529:4840], now have 1 active actors on pipe
2025-10-09T21:48:32.139599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342209366711531:9379]
2025-10-09T21:48:32.139602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342209366711531:9379]
2025-10-09T21:48:32.139604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342209366711531:9379]
2025-10-09T21:48:32.139617Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:32.139604Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1240
2025-10-09T21:48:32.139627Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:32.139632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:32.139637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:32.139639Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:32.139647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:32.139646Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1240
2025-10-09T21:48:32.139650Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:32.139689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342209366711531:9379]
2025-10-09T21:48:32.139700Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:32.139707Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:32.139715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:32.139720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:32.139767Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|94c1a7d6-a8469bd7-29f3eb34-9d46cbf7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:32.139778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:32.139783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:32.139789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:32.139797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:32.139805Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:32.139823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:32.139945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:32.139951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:32.139961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:32.140045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1241 partition: 0 MaxSeqNo: 25 sessionId: src_id|94c1a7d6-a8469bd7-29f3eb34-9d46cbf7_0
2025-10-09T21:48:32.140410Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1548ceeb-490ac20b-5494264a-1c8df840_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|94c1a7d6-a8469bd7-29f3eb34-9d46cbf7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:32.142682Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94c1a7d6-a8469bd7-29f3eb34-9d46cbf7_0] Write session: try to update token
2025-10-09T21:48:32.142698Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94c1a7d6-a8469bd7-29f3eb34-9d46cbf7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:32.153945Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:32.153964Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:32.153968Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:32.153969Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:32.153977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342209366711531:9379]
2025-10-09T21:48:32.153980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342209366711531:9379]
2025-10-09T21:48:32.153983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342209366711531:9379]
2025-10-09T21:48:32.154043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:32.154050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.154053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:32.154054Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:32.154061Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:32.154064Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:32.154200Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:32.155309Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1241 sessionId: src_id|94c1a7d6-a8469bd7-29f3eb34-9d46cbf7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:32.157644Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:32.219654Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342209728534545:5175]
2025-10-09T21:48:32.219907Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 1240, pipe: [1:7559342209728534545:5175]
2025-10-09T21:48:32.305174Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:32.305537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:32.305552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:32.305598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342209728534545:5175], now have 1 active actors on pipe
2025-10-09T21:48:32.305833Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342209728534545:5175]
2025-10-09T21:48:32.305840Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:32.305847Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:32.305844Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:32.305879Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:32.305920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1241 reason: error at write: Disconnected sessionId: src_id|94c1a7d6-a8469bd7-29f3eb34-9d46cbf7_0
2025-10-09T21:48:32.306120Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1241 sessionId: src_id|94c1a7d6-a8469bd7-29f3eb34-9d46cbf7_0 is DEAD
2025-10-09T21:48:32.306303Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94c1a7d6-a8469bd7-29f3eb34-9d46cbf7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:32.306312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94c1a7d6-a8469bd7-29f3eb34-9d46cbf7_0] Write session will restart in 0.006554s
2025-10-09T21:48:32.306339Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94c1a7d6-a8469bd7-29f3eb34-9d46cbf7_0] Write session: Do CDS request
2025-10-09T21:48:32.306346Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94c1a7d6-a8469bd7-29f3eb34-9d46cbf7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:32.314487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94c1a7d6-a8469bd7-29f3eb34-9d46cbf7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:32.314498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:32.314558Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:32.314571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1242
2025-10-09T21:48:32.314670Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1242 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:32.314701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1242 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:32.314712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1242 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:32.314716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:32.315083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1242 sessionId:  describe result for acl check
2025-10-09T21:48:32.315115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:32.315121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:32.315123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:32.315129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342209728534577:4837] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:32.315595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342209728534577:4837] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:32.315876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:32.316050Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:32.316060Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:32.316082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:32.316141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:32.316178Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342209366711603:9381]
2025-10-09T21:48:32.316722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:32.317002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:32.317047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:32.317363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:32.317422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.317427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.317428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.317429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.317430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.317431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.317434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.317435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.317437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.317443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.317452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.317455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.317459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.317463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.317467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.317470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.317474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.317478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.317484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.317491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.317497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.317503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.317509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.317514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.317518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.317523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.317529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.317534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.317538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.317543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.317548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.317553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.317557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.317563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.317568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.317573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.317602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:32.317669Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342209728534577:4837] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:32.323449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342209728534577:4837] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:32.323468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342209728534577:4837] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:32.323472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342209728534577:4837] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:32.323483Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1242 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:32.323746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:32.324694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:32.324710Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:32.324712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:32.324715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:32.324721Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1241 [2:7559342209366711603:9381]
2025-10-09T21:48:32.324732Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:32.324746Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:32.324748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:32.324750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:32.324752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:32.324765Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:32.324769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:32.324775Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:32.324826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.324835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.324838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.324840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:32.324896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:32.324904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.324914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.324920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.324931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:32.324953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342209728534616:4837], now have 1 active actors on pipe
2025-10-09T21:48:32.324965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342209366711598:9381]
2025-10-09T21:48:32.324968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342209366711598:9381]
2025-10-09T21:48:32.324970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342209366711598:9381]
2025-10-09T21:48:32.324985Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:32.324998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342209728534547:2432], now have 1 active actors on pipe
2025-10-09T21:48:32.325007Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:32.325010Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:32.325010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:32.325012Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:32.325023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:32.325027Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:32.325050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342209366711598:9381]
2025-10-09T21:48:32.325062Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:32.325052Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1241
2025-10-09T21:48:32.325072Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:32.325093Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1241
2025-10-09T21:48:32.325102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:32.325111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:32.325138Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9514563f-4b712b28-b7bcdd49-fe72f131_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:32.325149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:32.325153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:32.325155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:32.325164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:32.325174Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:32.325192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:32.325311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:32.325318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:32.325329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:32.325410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1242 partition: 0 MaxSeqNo: 25 sessionId: src_id|9514563f-4b712b28-b7bcdd49-fe72f131_0
2025-10-09T21:48:32.325694Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94c1a7d6-a8469bd7-29f3eb34-9d46cbf7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9514563f-4b712b28-b7bcdd49-fe72f131_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:32.327926Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9514563f-4b712b28-b7bcdd49-fe72f131_0] Write session: try to update token
2025-10-09T21:48:32.327940Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9514563f-4b712b28-b7bcdd49-fe72f131_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:32.338984Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:32.339001Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:32.339003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:32.339005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:32.339014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342209366711598:9381]
2025-10-09T21:48:32.339019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342209366711598:9381]
2025-10-09T21:48:32.339023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342209366711598:9381]
2025-10-09T21:48:32.339065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:32.339067Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:32.339074Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:32.339078Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:32.339080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.339083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:32.339208Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:32.340308Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1242 sessionId: src_id|9514563f-4b712b28-b7bcdd49-fe72f131_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:32.342653Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:32.406038Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342209728534624:5175]
2025-10-09T21:48:32.406303Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 1241, pipe: [1:7559342209728534624:5175]
2025-10-09T21:48:32.492854Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:32.493235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:32.493253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:32.493299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342209728534624:5175], now have 1 active actors on pipe
2025-10-09T21:48:32.493533Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:32.493585Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:32.493598Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:32.493603Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:32.493637Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342209728534624:5175]
2025-10-09T21:48:32.493675Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1242 reason: error at write: Disconnected sessionId: src_id|9514563f-4b712b28-b7bcdd49-fe72f131_0
2025-10-09T21:48:32.493861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1242 sessionId: src_id|9514563f-4b712b28-b7bcdd49-fe72f131_0 is DEAD
2025-10-09T21:48:32.494078Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9514563f-4b712b28-b7bcdd49-fe72f131_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:32.494086Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9514563f-4b712b28-b7bcdd49-fe72f131_0] Write session will restart in 0.009063s
2025-10-09T21:48:32.494109Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9514563f-4b712b28-b7bcdd49-fe72f131_0] Write session: Do CDS request
2025-10-09T21:48:32.494117Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9514563f-4b712b28-b7bcdd49-fe72f131_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:32.502936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:32.504404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:32.504572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9514563f-4b712b28-b7bcdd49-fe72f131_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:32.504613Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:32.504627Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:32.504613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:32.504640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:32.504629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1243
2025-10-09T21:48:32.504696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:32.504738Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342209366711672:9383]
2025-10-09T21:48:32.504773Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1243 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:32.504816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1243 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:32.504826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1243 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:32.504829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:32.505257Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1243 sessionId:  describe result for acl check
2025-10-09T21:48:32.505293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:32.505299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:32.505301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:32.505306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342209728534657:4832] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:32.505322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:32.505620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:32.505661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:32.505886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342209728534657:4832] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:32.505998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:32.506077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.506083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.506084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.506085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.506086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.506087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.506088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.506089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.506090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.506096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.506105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.506108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.506112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.506116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.506120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.506123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.506127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.506131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.506139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.506146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.506152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.506158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.506167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.506172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.506178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.506187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.506192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.506197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.506206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.506210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.506215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.506220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.506224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.506227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.506231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.506235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.506265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:32.507728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342209728534657:4832] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:32.511961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342209728534657:4832] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:32.511976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342209728534657:4832] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:32.511980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342209728534657:4832] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:32.511988Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1243 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:32.512317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:32.513348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:32.513364Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:32.513366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:32.513370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:32.513377Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1242 [2:7559342209366711672:9383]
2025-10-09T21:48:32.513389Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:32.513406Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:32.513415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:32.513418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:32.513421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:32.513438Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:32.513448Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:32.513454Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:32.513512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.513518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.513521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.513523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:32.513592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:32.513600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.513611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.513621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.513641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:32.513667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342209728534626:2432], now have 1 active actors on pipe
2025-10-09T21:48:32.513674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342209728534697:4832], now have 1 active actors on pipe
2025-10-09T21:48:32.513682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342209366711667:9383]
2025-10-09T21:48:32.513688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342209366711667:9383]
2025-10-09T21:48:32.513690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342209366711667:9383]
2025-10-09T21:48:32.513681Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1242
2025-10-09T21:48:32.513709Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1242
2025-10-09T21:48:32.513721Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:32.513725Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:32.513728Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:32.513730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:32.513741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:32.513755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:32.513762Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:32.513801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:32.513810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:32.513825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342209366711667:9383]
2025-10-09T21:48:32.513837Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:32.513850Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:32.513871Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f42ac53b-a0429bb6-90728249-181dddfa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:32.513882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:32.513886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:32.513889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:32.513899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:32.513908Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:32.513930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:32.514131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:32.514142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:32.514158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:32.514238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1243 partition: 0 MaxSeqNo: 25 sessionId: src_id|f42ac53b-a0429bb6-90728249-181dddfa_0
2025-10-09T21:48:32.514550Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9514563f-4b712b28-b7bcdd49-fe72f131_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f42ac53b-a0429bb6-90728249-181dddfa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:32.516855Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f42ac53b-a0429bb6-90728249-181dddfa_0] Write session: try to update token
2025-10-09T21:48:32.516872Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f42ac53b-a0429bb6-90728249-181dddfa_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:32.527256Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:32.527273Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:32.527276Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:32.527278Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:32.527287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342209366711667:9383]
2025-10-09T21:48:32.527292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342209366711667:9383]
2025-10-09T21:48:32.527300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342209366711667:9383]
2025-10-09T21:48:32.527341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:32.527344Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:32.527349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.527350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:32.527351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:32.527353Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:32.527475Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:32.529410Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1243 sessionId: src_id|f42ac53b-a0429bb6-90728249-181dddfa_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:32.531682Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:32.593927Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342209728534704:5175]
2025-10-09T21:48:32.594205Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 1242, pipe: [1:7559342209728534704:5175]
2025-10-09T21:48:32.686592Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:32.686895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:32.686908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:32.686947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342209728534704:5175], now have 1 active actors on pipe
2025-10-09T21:48:32.687143Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:32.687162Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:32.687175Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:32.687233Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342209728534704:5175]
2025-10-09T21:48:32.687321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1243 reason: error at write: Disconnected sessionId: src_id|f42ac53b-a0429bb6-90728249-181dddfa_0
2025-10-09T21:48:32.687141Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:32.687542Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1243 sessionId: src_id|f42ac53b-a0429bb6-90728249-181dddfa_0 is DEAD
2025-10-09T21:48:32.687699Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f42ac53b-a0429bb6-90728249-181dddfa_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:32.687711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f42ac53b-a0429bb6-90728249-181dddfa_0] Write session will restart in 0.007171s
2025-10-09T21:48:32.687738Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f42ac53b-a0429bb6-90728249-181dddfa_0] Write session: Do CDS request
2025-10-09T21:48:32.687755Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f42ac53b-a0429bb6-90728249-181dddfa_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:32.695896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:32.696521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f42ac53b-a0429bb6-90728249-181dddfa_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:32.696544Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:32.696557Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1244
2025-10-09T21:48:32.696751Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1244 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:32.696798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1244 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:32.696810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1244 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:32.696814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:32.697202Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1244 sessionId:  describe result for acl check
2025-10-09T21:48:32.697243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:32.697245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:32.697247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:32.697253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342209728534745:4825] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:32.697538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:32.697675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342209728534745:4825] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:32.697797Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:32.697822Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:32.697836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:32.697904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:32.697954Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342209366711775:9397]
2025-10-09T21:48:32.698544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:32.698823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:32.698868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:32.699196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:32.699254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.699262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.699264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.699265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.699266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.699267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.699268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.699270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.699271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.699276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.699286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.699290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.699295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.699299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.699303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.699307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.699311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.699315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.699321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.699330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.699341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.699347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.699352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.699357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.699361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.699366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.699370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.699374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.699378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.699383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.699387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.699392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.699396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.699400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.699405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.699409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.699433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:32.699827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342209728534745:4825] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:32.705673Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342209728534745:4825] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:32.705695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342209728534745:4825] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:32.705710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342209728534745:4825] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:32.705725Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1244 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:32.706001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:32.707616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:32.707632Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:32.707647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:32.707652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:32.707659Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1243 [2:7559342209366711775:9397]
2025-10-09T21:48:32.707673Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:32.707697Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:32.707708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:32.707712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:32.707714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:32.707733Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:32.707740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:32.707751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:32.707814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.707821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.707824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.707827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:32.707899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:32.707907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.707920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.707936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.707959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:32.707984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342209728534709:2432], now have 1 active actors on pipe
2025-10-09T21:48:32.707992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342209728534789:4825], now have 1 active actors on pipe
2025-10-09T21:48:32.708000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342209366711770:9397]
2025-10-09T21:48:32.708007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342209366711770:9397]
2025-10-09T21:48:32.708009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342209366711770:9397]
2025-10-09T21:48:32.708021Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:32.708035Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:32.708028Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1243
2025-10-09T21:48:32.708038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:32.708047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:32.708081Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1243
2025-10-09T21:48:32.708049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:32.708053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:32.708057Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:32.708093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342209366711770:9397]
2025-10-09T21:48:32.708102Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:32.708116Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:32.708150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:32.708162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:32.708191Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|94ec6c47-7889ab64-54d610ff-1a32d3e2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:32.708203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:32.708207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:32.708211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:32.708220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:32.708231Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:32.708249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:32.708423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:32.708435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:32.708460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:32.708543Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1244 partition: 0 MaxSeqNo: 25 sessionId: src_id|94ec6c47-7889ab64-54d610ff-1a32d3e2_0
2025-10-09T21:48:32.708908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f42ac53b-a0429bb6-90728249-181dddfa_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|94ec6c47-7889ab64-54d610ff-1a32d3e2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:32.716623Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94ec6c47-7889ab64-54d610ff-1a32d3e2_0] Write session: try to update token
2025-10-09T21:48:32.716646Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94ec6c47-7889ab64-54d610ff-1a32d3e2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:32.726138Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:32.726156Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:32.726166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342209366711770:9397]
2025-10-09T21:48:32.726223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:32.726225Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:32.726402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:32.726551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:32.726694Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:32.726834Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:32.727147Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:32.727205Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:32.733176Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1244 sessionId: src_id|94ec6c47-7889ab64-54d610ff-1a32d3e2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:32.733649Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:32.733664Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:32.733666Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:32.733668Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:32.733678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342209366711770:9397]
2025-10-09T21:48:32.733682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342209366711770:9397]
2025-10-09T21:48:32.733685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342209366711770:9397]
2025-10-09T21:48:32.733736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:32.733744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.733746Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:32.733752Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:32.733753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:32.733761Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:32.733938Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:32.735523Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:32.787434Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342209728534806:5175]
2025-10-09T21:48:32.787710Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 1243, pipe: [1:7559342209728534806:5175]
2025-10-09T21:48:32.893792Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:32.894152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:32.894168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:32.894193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342209728534806:5175], now have 1 active actors on pipe
2025-10-09T21:48:32.894409Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:32.894418Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:32.894424Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342209728534806:5175]
2025-10-09T21:48:32.894433Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:32.894444Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:32.894500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1244 reason: error at write: Disconnected sessionId: src_id|94ec6c47-7889ab64-54d610ff-1a32d3e2_0
2025-10-09T21:48:32.894687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1244 sessionId: src_id|94ec6c47-7889ab64-54d610ff-1a32d3e2_0 is DEAD
2025-10-09T21:48:32.894866Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94ec6c47-7889ab64-54d610ff-1a32d3e2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:32.894877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94ec6c47-7889ab64-54d610ff-1a32d3e2_0] Write session will restart in 0.009999s
2025-10-09T21:48:32.894905Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94ec6c47-7889ab64-54d610ff-1a32d3e2_0] Write session: Do CDS request
2025-10-09T21:48:32.894912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94ec6c47-7889ab64-54d610ff-1a32d3e2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:32.903423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:32.905100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:32.905289Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:32.905300Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:32.905311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:32.905378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:32.905416Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342209366711853:9401]
2025-10-09T21:48:32.905946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:32.906280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:32.906325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:32.906565Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|94ec6c47-7889ab64-54d610ff-1a32d3e2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:32.906653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:32.906713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.906681Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:32.906699Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1245
2025-10-09T21:48:32.906719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.906721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.906722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.906723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.906725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.906726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.906727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.906729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.906734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.906745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.906749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.906753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.906759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.906763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.906767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.906770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.906774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.906782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.906790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:32.906797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.906803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:32.906808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.906815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:32.906820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.906827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:32.906823Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1245 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:32.906833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.906839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:32.906845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.906853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:32.906857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.906852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1245 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:32.906862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1245 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:32.906866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:32.906863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.906867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.906871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.906876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.906881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.906912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:32.907242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1245 sessionId:  describe result for acl check
2025-10-09T21:48:32.907269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:32.907274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:32.907275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:32.907281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342209728534849:4808] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:32.907770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342209728534849:4808] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:32.910286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342209728534849:4808] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:32.912973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342209728534849:4808] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:32.912985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342209728534849:4808] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:32.912988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342209728534849:4808] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:32.913001Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1245 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:32.913322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:32.914424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:32.914440Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:32.914443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:32.914447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:32.914456Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1244 [2:7559342209366711853:9401]
2025-10-09T21:48:32.914470Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:32.914486Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:32.914492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:32.914495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:32.914496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:32.914509Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:32.914518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:32.914523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:32.914563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.914569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.914570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.914572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:32.914635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:32.914643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.914654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:32.914667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:32.914689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:32.914711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342209728534811:2432], now have 1 active actors on pipe
2025-10-09T21:48:32.914717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342209728534883:4808], now have 1 active actors on pipe
2025-10-09T21:48:32.914725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342209366711848:9401]
2025-10-09T21:48:32.914731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342209366711848:9401]
2025-10-09T21:48:32.914733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342209366711848:9401]
2025-10-09T21:48:32.914748Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:32.914765Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1244
2025-10-09T21:48:32.914767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:32.914765Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1244
2025-10-09T21:48:32.914770Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:32.914774Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:32.914776Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:32.914785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:32.914788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:32.914825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342209366711848:9401]
2025-10-09T21:48:32.914837Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:32.914849Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:32.914863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:32.914872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:32.914919Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|73085843-14e66f68-a972b4a9-f82798a6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:32.914928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:32.914931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:32.914932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:32.914940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:32.914950Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:32.914967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:32.915126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:32.915138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:32.915160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:32.915257Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1245 partition: 0 MaxSeqNo: 25 sessionId: src_id|73085843-14e66f68-a972b4a9-f82798a6_0
2025-10-09T21:48:32.915569Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|94ec6c47-7889ab64-54d610ff-1a32d3e2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|73085843-14e66f68-a972b4a9-f82798a6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:32.917954Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73085843-14e66f68-a972b4a9-f82798a6_0] Write session: try to update token
2025-10-09T21:48:32.917972Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73085843-14e66f68-a972b4a9-f82798a6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:32.929011Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:32.929026Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:32.929028Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:32.929029Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:32.929036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342209366711848:9401]
2025-10-09T21:48:32.929039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342209366711848:9401]
2025-10-09T21:48:32.929044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342209366711848:9401]
2025-10-09T21:48:32.929079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:32.929084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:32.929088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:32.929098Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:32.929105Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:32.929109Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:32.929264Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:32.930939Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1245 sessionId: src_id|73085843-14e66f68-a972b4a9-f82798a6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:32.933218Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:32.994773Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342209728534890:5175]
2025-10-09T21:48:32.995042Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 1244, pipe: [1:7559342209728534890:5175]
2025-10-09T21:48:33.087196Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:33.087497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:33.087513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:33.087559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342209728534890:5175], now have 1 active actors on pipe
2025-10-09T21:48:33.087833Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:33.087852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342209728534890:5175]
2025-10-09T21:48:33.087886Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:33.087889Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:33.087865Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:33.087950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1245 reason: error at write: Disconnected sessionId: src_id|73085843-14e66f68-a972b4a9-f82798a6_0
2025-10-09T21:48:33.088381Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1245 sessionId: src_id|73085843-14e66f68-a972b4a9-f82798a6_0 is DEAD
2025-10-09T21:48:33.088575Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73085843-14e66f68-a972b4a9-f82798a6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:33.088587Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73085843-14e66f68-a972b4a9-f82798a6_0] Write session will restart in 0.007459s
2025-10-09T21:48:33.088614Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73085843-14e66f68-a972b4a9-f82798a6_0] Write session: Do CDS request
2025-10-09T21:48:33.088622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73085843-14e66f68-a972b4a9-f82798a6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:33.096827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:33.097538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73085843-14e66f68-a972b4a9-f82798a6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:33.097631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:33.097645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1246
2025-10-09T21:48:33.097804Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1246 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:33.097845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1246 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:33.097856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1246 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:33.097860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:33.098332Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1246 sessionId:  describe result for acl check
2025-10-09T21:48:33.098371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:33.098378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:33.098380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:33.098388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342214023502223:4806] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:33.098523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:33.098716Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:33.098728Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:33.098742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:33.098810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:33.098854Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342213661679220:9403]
2025-10-09T21:48:33.098951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342214023502223:4806] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:33.099447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:33.099750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:33.099802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:33.100123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:33.100173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.100179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.100180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.100181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.100182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.100183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.100185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.100186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.100187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.100193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.100203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.100207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.100210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.100215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.100219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.100222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.100226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.100230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.100236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.100244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.100250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.100255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.100260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.100265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.100269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.100274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.100279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.100284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.100289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.100294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.100299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.100305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.100309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.100314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.100318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.100324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.100360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:33.101134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342214023502223:4806] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:33.106479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342214023502223:4806] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:33.106498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342214023502223:4806] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:33.106502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342214023502223:4806] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:33.106511Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1246 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:33.106748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:33.107810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:33.107830Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:33.107833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:33.107837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:33.107845Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1245 [2:7559342213661679220:9403]
2025-10-09T21:48:33.107860Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:33.107880Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:33.107883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:33.107890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:33.107893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:33.107910Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:33.107921Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:33.107931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:33.107994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.108002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.108004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.108007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:33.108133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:33.108142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.108153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.108166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.108184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:33.108212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342214023502262:4806], now have 1 active actors on pipe
2025-10-09T21:48:33.108216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342214023502189:2432], now have 1 active actors on pipe
2025-10-09T21:48:33.108223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342213661679215:9403]
2025-10-09T21:48:33.108230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342213661679215:9403]
2025-10-09T21:48:33.108216Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1245
2025-10-09T21:48:33.108217Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1245
2025-10-09T21:48:33.108231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342213661679215:9403]
2025-10-09T21:48:33.108244Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:33.108258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:33.108258Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:33.108261Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:33.108263Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:33.108272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:33.108275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:33.108321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342213661679215:9403]
2025-10-09T21:48:33.108335Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:33.108349Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:33.108350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:33.108355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:33.108406Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9e7e2270-890f77b8-5dbb78c5-e4caf9e7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:33.108418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:33.108422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:33.108424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:33.108435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:33.108449Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:33.108474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:33.108660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:33.108672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:33.108688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:33.108780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1246 partition: 0 MaxSeqNo: 25 sessionId: src_id|9e7e2270-890f77b8-5dbb78c5-e4caf9e7_0
2025-10-09T21:48:33.109149Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73085843-14e66f68-a972b4a9-f82798a6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9e7e2270-890f77b8-5dbb78c5-e4caf9e7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:33.111587Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7e2270-890f77b8-5dbb78c5-e4caf9e7_0] Write session: try to update token
2025-10-09T21:48:33.111608Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7e2270-890f77b8-5dbb78c5-e4caf9e7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:33.122987Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:33.123005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:33.123007Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:33.123009Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:33.123021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342213661679215:9403]
2025-10-09T21:48:33.123026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342213661679215:9403]
2025-10-09T21:48:33.123030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342213661679215:9403]
2025-10-09T21:48:33.123074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:33.123080Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:33.123086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.123087Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:33.123088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:33.123091Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:33.123263Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:33.124582Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1246 sessionId: src_id|9e7e2270-890f77b8-5dbb78c5-e4caf9e7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:33.126929Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:33.187790Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342214023502273:5175]
2025-10-09T21:48:33.188051Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 1245, pipe: [1:7559342214023502273:5175]
2025-10-09T21:48:33.280789Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:33.281107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:33.281123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:33.281170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342214023502273:5175], now have 1 active actors on pipe
2025-10-09T21:48:33.281394Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:33.281406Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:33.281410Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342214023502273:5175]
2025-10-09T21:48:33.281421Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:33.281434Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:33.281498Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1246 reason: error at write: Disconnected sessionId: src_id|9e7e2270-890f77b8-5dbb78c5-e4caf9e7_0
2025-10-09T21:48:33.281668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1246 sessionId: src_id|9e7e2270-890f77b8-5dbb78c5-e4caf9e7_0 is DEAD
2025-10-09T21:48:33.281874Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7e2270-890f77b8-5dbb78c5-e4caf9e7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:33.281885Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7e2270-890f77b8-5dbb78c5-e4caf9e7_0] Write session will restart in 0.007899s
2025-10-09T21:48:33.281909Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7e2270-890f77b8-5dbb78c5-e4caf9e7_0] Write session: Do CDS request
2025-10-09T21:48:33.281914Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7e2270-890f77b8-5dbb78c5-e4caf9e7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:33.290468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:33.290514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7e2270-890f77b8-5dbb78c5-e4caf9e7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:33.290593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:33.290605Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1247
2025-10-09T21:48:33.290743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1247 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:33.290778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1247 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:33.290789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1247 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:33.290793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:33.291191Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1247 sessionId:  describe result for acl check
2025-10-09T21:48:33.291224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:33.291230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:33.291232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:33.291236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342214023502315:4792] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:33.291698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342214023502315:4792] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:33.291994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:33.292175Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:33.292187Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:33.292199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:33.292269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:33.292320Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342213661679292:9405]
2025-10-09T21:48:33.292898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:33.293184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:33.293248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:33.293601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:33.293653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.293659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.293661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.293662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.293663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.293664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.293666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.293667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.293669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.293674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.293685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.293690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.293694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.293711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.293715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.293719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.293723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.293728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.293734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.293742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.293753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.293758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.293761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.293767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.293772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.293777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.293784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.293789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.293793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.293774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342214023502315:4792] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:33.293800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.293806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.293812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.293816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.293825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.293830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.293835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.293864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:33.300004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342214023502315:4792] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:33.300022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342214023502315:4792] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:33.300026Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342214023502315:4792] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:33.300037Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1247 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:33.300289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:33.301319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:33.301336Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:33.301338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:33.301342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:33.301349Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1246 [2:7559342213661679292:9405]
2025-10-09T21:48:33.301363Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:33.301380Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:33.301388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:33.301391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:33.301394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:33.301413Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:33.301422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:33.301429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:33.301487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.301493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.301494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.301496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:33.301547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:33.301552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.301559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.301565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.301574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:33.301593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342214023502354:4792], now have 1 active actors on pipe
2025-10-09T21:48:33.301603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342213661679287:9405]
2025-10-09T21:48:33.301605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342213661679287:9405]
2025-10-09T21:48:33.301606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342213661679287:9405]
2025-10-09T21:48:33.301615Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:33.301622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342214023502280:2432], now have 1 active actors on pipe
2025-10-09T21:48:33.301630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:33.301637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:33.301640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:33.301641Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:33.301649Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:33.301652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:33.301681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342213661679287:9405]
2025-10-09T21:48:33.301690Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:33.301692Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1246
2025-10-09T21:48:33.301711Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1246
2025-10-09T21:48:33.301721Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:33.301748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:33.301751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:33.301776Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ff0ea3a9-18405c7d-f78908d-523efd48_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:33.301782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:33.301785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:33.301786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:33.301794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:33.301802Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:33.301817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:33.302014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:33.302041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:33.302060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:33.302156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1247 partition: 0 MaxSeqNo: 25 sessionId: src_id|ff0ea3a9-18405c7d-f78908d-523efd48_0
2025-10-09T21:48:33.302547Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e7e2270-890f77b8-5dbb78c5-e4caf9e7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ff0ea3a9-18405c7d-f78908d-523efd48_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:33.304998Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff0ea3a9-18405c7d-f78908d-523efd48_0] Write session: try to update token
2025-10-09T21:48:33.305014Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff0ea3a9-18405c7d-f78908d-523efd48_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:33.316107Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:33.316130Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:33.316132Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:33.316135Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:33.316145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342213661679287:9405]
2025-10-09T21:48:33.316150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342213661679287:9405]
2025-10-09T21:48:33.316154Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342213661679287:9405]
2025-10-09T21:48:33.316198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:33.316206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.316207Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:33.316209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:33.316214Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:33.316218Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:33.316417Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:33.318587Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1247 sessionId: src_id|ff0ea3a9-18405c7d-f78908d-523efd48_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:33.321006Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:33.381369Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342214023502362:5175]
2025-10-09T21:48:33.381691Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 1246, pipe: [1:7559342214023502362:5175]
2025-10-09T21:48:33.472014Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:33.472379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:33.472394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:33.472441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342214023502362:5175], now have 1 active actors on pipe
2025-10-09T21:48:33.472651Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:33.472697Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:33.472711Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:33.472712Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:33.472769Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342214023502362:5175]
2025-10-09T21:48:33.472772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1247 reason: error at write: Disconnected sessionId: src_id|ff0ea3a9-18405c7d-f78908d-523efd48_0
2025-10-09T21:48:33.472945Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1247 sessionId: src_id|ff0ea3a9-18405c7d-f78908d-523efd48_0 is DEAD
2025-10-09T21:48:33.473114Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff0ea3a9-18405c7d-f78908d-523efd48_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:33.473126Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff0ea3a9-18405c7d-f78908d-523efd48_0] Write session will restart in 0.005976s
2025-10-09T21:48:33.473153Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff0ea3a9-18405c7d-f78908d-523efd48_0] Write session: Do CDS request
2025-10-09T21:48:33.473159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff0ea3a9-18405c7d-f78908d-523efd48_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:33.480483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff0ea3a9-18405c7d-f78908d-523efd48_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:33.480590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:33.480605Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1248
2025-10-09T21:48:33.480749Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1248 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:33.480781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1248 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:33.480790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1248 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:33.480793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:33.481192Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1248 sessionId:  describe result for acl check
2025-10-09T21:48:33.481229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:33.481236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:33.481238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:33.481243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342214023502399:4793] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:33.481326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:33.481748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342214023502399:4793] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:33.482921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:33.483090Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:33.483100Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:33.483111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:33.483161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:33.483215Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342213661679365:9407]
2025-10-09T21:48:33.483734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:33.483718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342214023502399:4793] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:33.484036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:33.484087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:33.484402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:33.484453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.484459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.484460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.484461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.484462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.484464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.484465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.484466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.484468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.484473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.484484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.484488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.484492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.484496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.484500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.484504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.484508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.484512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.484519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.484528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.484534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.484539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.484549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.484554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.484559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.484564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.484570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.484576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.484580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.484585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.484593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.484597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.484601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.484606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.484610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.484620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.484650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:33.489504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342214023502399:4793] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:33.489516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342214023502399:4793] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:33.489518Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342214023502399:4793] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:33.489526Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1248 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:33.490883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:33.491960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:33.491978Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:33.491981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:33.491985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:33.491992Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1247 [2:7559342213661679365:9407]
2025-10-09T21:48:33.492010Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:33.492027Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:33.492030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:33.492033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:33.492035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:33.492052Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:33.492063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:33.492070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:33.492128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.492137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.492140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.492143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:33.492222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:33.492231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.492244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.492256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.492273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:33.492301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342214023502438:4793], now have 1 active actors on pipe
2025-10-09T21:48:33.492309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342214023502364:2432], now have 1 active actors on pipe
2025-10-09T21:48:33.492316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342213661679360:9407]
2025-10-09T21:48:33.492319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342213661679360:9407]
2025-10-09T21:48:33.492320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342213661679360:9407]
2025-10-09T21:48:33.492333Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:33.492344Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:33.492347Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:33.492349Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:33.492352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:33.492332Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1247
2025-10-09T21:48:33.492352Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1247
2025-10-09T21:48:33.492367Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:33.492372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:33.492409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:33.492418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:33.492435Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342213661679360:9407]
2025-10-09T21:48:33.492448Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:33.492479Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:33.492479Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|72c60512-2af6e14e-51a9a199-b47aaede_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:33.492488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:33.492492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:33.492495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:33.492505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:33.492515Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:33.492593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:33.492749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:33.492758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:33.492775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:33.492886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1248 partition: 0 MaxSeqNo: 25 sessionId: src_id|72c60512-2af6e14e-51a9a199-b47aaede_0
2025-10-09T21:48:33.493204Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff0ea3a9-18405c7d-f78908d-523efd48_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|72c60512-2af6e14e-51a9a199-b47aaede_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:33.495635Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72c60512-2af6e14e-51a9a199-b47aaede_0] Write session: try to update token
2025-10-09T21:48:33.495653Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72c60512-2af6e14e-51a9a199-b47aaede_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:33.506973Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:33.506992Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:33.506995Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:33.506998Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:33.507009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342213661679360:9407]
2025-10-09T21:48:33.507014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342213661679360:9407]
2025-10-09T21:48:33.507017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342213661679360:9407]
2025-10-09T21:48:33.507062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:33.507063Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:33.507069Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:33.507073Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:33.507075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.507078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:33.507251Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:33.508992Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1248 sessionId: src_id|72c60512-2af6e14e-51a9a199-b47aaede_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:33.511467Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:33.572874Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342214023502447:5175]
2025-10-09T21:48:33.573175Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 1247, pipe: [1:7559342214023502447:5175]
2025-10-09T21:48:33.660151Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:33.660415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:33.660431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:33.660474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342214023502447:5175], now have 1 active actors on pipe
2025-10-09T21:48:33.660728Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:33.660744Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342214023502447:5175]
2025-10-09T21:48:33.660763Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:33.660765Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:33.660818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1248 reason: error at write: Disconnected sessionId: src_id|72c60512-2af6e14e-51a9a199-b47aaede_0
2025-10-09T21:48:33.660986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1248 sessionId: src_id|72c60512-2af6e14e-51a9a199-b47aaede_0 is DEAD
2025-10-09T21:48:33.661036Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:33.661162Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72c60512-2af6e14e-51a9a199-b47aaede_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:33.661172Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72c60512-2af6e14e-51a9a199-b47aaede_0] Write session will restart in 0.009229s
2025-10-09T21:48:33.661205Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72c60512-2af6e14e-51a9a199-b47aaede_0] Write session: Do CDS request
2025-10-09T21:48:33.661211Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72c60512-2af6e14e-51a9a199-b47aaede_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:33.669874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:33.671567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72c60512-2af6e14e-51a9a199-b47aaede_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:33.671576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:33.671623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:33.671643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1249
2025-10-09T21:48:33.671781Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:33.671792Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:33.671807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:33.671813Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1249 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:33.671845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1249 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:33.671854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1249 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:33.671858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:33.671860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:33.671903Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342213661679456:9417]
2025-10-09T21:48:33.672247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1249 sessionId:  describe result for acl check
2025-10-09T21:48:33.672280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:33.672287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:33.672288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:33.672295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342214023502487:4783] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:33.672491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:33.672833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:33.672828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342214023502487:4783] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:33.672889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:33.673240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:33.673300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.673307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.673308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.673309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.673311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.673312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.673313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.673314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.673315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.673322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.673333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.673337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.673340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.673344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.673348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.673351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.673355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.673359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.673366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.673374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.673380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.673391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.673395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.673401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.673408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.673418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.673423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.673428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.673432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.673442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.673447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.673453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.673462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.673467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.673472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.673482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.673511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:33.674597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342214023502487:4783] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:33.679557Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342214023502487:4783] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:33.679573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342214023502487:4783] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:33.679577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342214023502487:4783] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:33.679586Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1249 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:33.679960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:33.680950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:33.680964Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:33.680966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:33.680971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:33.680977Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1248 [2:7559342213661679456:9417]
2025-10-09T21:48:33.680986Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:33.681004Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:33.681013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:33.681017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:33.681019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:33.681040Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:33.681052Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:33.681059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:33.681117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.681125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.681126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.681128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:33.681193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:33.681201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.681214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.681229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.681249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:33.681276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342214023502526:4783], now have 1 active actors on pipe
2025-10-09T21:48:33.681284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342214023502449:2432], now have 1 active actors on pipe
2025-10-09T21:48:33.681292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342213661679451:9417]
2025-10-09T21:48:33.681298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342213661679451:9417]
2025-10-09T21:48:33.681300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342213661679451:9417]
2025-10-09T21:48:33.681313Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:33.681330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:33.681330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:33.681334Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:33.681336Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:33.681349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:33.681352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:33.681371Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1248
2025-10-09T21:48:33.681379Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1248
2025-10-09T21:48:33.681391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342213661679451:9417]
2025-10-09T21:48:33.681400Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:33.681434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:33.681442Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:33.681444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:33.681475Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5925354e-c29bb2d-140ce81c-adc1569e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:33.681487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:33.681491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:33.681494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:33.681504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:33.681518Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:33.681541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:33.681754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:33.681765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:33.681780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:33.681898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1249 partition: 0 MaxSeqNo: 25 sessionId: src_id|5925354e-c29bb2d-140ce81c-adc1569e_0
2025-10-09T21:48:33.682280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72c60512-2af6e14e-51a9a199-b47aaede_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5925354e-c29bb2d-140ce81c-adc1569e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:33.684791Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5925354e-c29bb2d-140ce81c-adc1569e_0] Write session: try to update token
2025-10-09T21:48:33.684810Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5925354e-c29bb2d-140ce81c-adc1569e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:33.697834Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1249 sessionId: src_id|5925354e-c29bb2d-140ce81c-adc1569e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:33.698732Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:33.698752Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:33.698762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342213661679451:9417]
2025-10-09T21:48:33.698812Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:33.698822Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:33.698979Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:33.699116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:33.699253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:33.699396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:33.699736Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:33.699784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:33.700379Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:33.700570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:33.700582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:33.700609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:33.700706Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:33.706136Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:33.706153Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:33.706157Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:33.706159Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:33.706172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342213661679451:9417]
2025-10-09T21:48:33.706175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342213661679451:9417]
2025-10-09T21:48:33.706179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342213661679451:9417]
2025-10-09T21:48:33.706221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:33.706226Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:33.706232Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:33.706233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.706235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:33.706238Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:33.706365Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:33.760746Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342214023502546:5175]
2025-10-09T21:48:33.761024Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 1248, pipe: [1:7559342214023502546:5175]
2025-10-09T21:48:33.863738Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:33.871218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:33.871245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:33.871413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:33.871477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:33.871531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:33.871585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:33.871624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:33.871634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:33.871694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:33.871745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:33.871801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:33.871863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:33.871948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:33.871960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:33.872026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:33.872076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:33.872129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:33.872181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:33.872219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:33.872224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:33.872288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:33.872358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:33.872426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:33.872488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:33.872614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:33.872624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:33.872693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:33.872756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:33.872821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:33.872883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:33.872924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:33.872929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:33.872983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:33.873033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:33.873086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:33.873139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:33.873177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:33.873182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:33.873234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:33.873283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:33.873336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:33.873389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:33.873428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:33.873433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:33.873484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:33.873536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:33.873588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:33.873640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:33.873757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:33.873762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:33.873814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:33.873861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:33.873911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:33.873966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:33.874391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:33.874403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:33.874469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:33.874526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:33.874587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:33.874653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:33.874774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:33.874780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:33.874838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342214023502546:5175], now have 1 active actors on pipe
2025-10-09T21:48:33.875111Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342214023502546:5175]
2025-10-09T21:48:33.875115Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:33.875119Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:33.875149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1249 reason: error at write: Disconnected sessionId: src_id|5925354e-c29bb2d-140ce81c-adc1569e_0
2025-10-09T21:48:33.875157Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:33.875292Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:33.875342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1249 sessionId: src_id|5925354e-c29bb2d-140ce81c-adc1569e_0 is DEAD
2025-10-09T21:48:33.875465Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5925354e-c29bb2d-140ce81c-adc1569e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:33.875480Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5925354e-c29bb2d-140ce81c-adc1569e_0] Write session will restart in 0.009891s
2025-10-09T21:48:33.875506Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5925354e-c29bb2d-140ce81c-adc1569e_0] Write session: Do CDS request
2025-10-09T21:48:33.875513Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5925354e-c29bb2d-140ce81c-adc1569e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:33.883343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:33.884847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:33.885027Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:33.885037Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:33.885050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:33.885104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:33.885144Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342213661679557:9425]
2025-10-09T21:48:33.885737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:33.886043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:33.886089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:33.886400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:33.886429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5925354e-c29bb2d-140ce81c-adc1569e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:33.886455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.886461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.886462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.886463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.886465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.886466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.886467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.886468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.886469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.886474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.886486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.886490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.886494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.886497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.886502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.886506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.886510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.886514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.886522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.886530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:33.886537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.886545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:33.886549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.886555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:33.886561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.886566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:33.886570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.886575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:33.886580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.886554Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:33.886570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1250
2025-10-09T21:48:33.886586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:33.886592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.886598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.886602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.886607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.886611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.886616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.886647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:33.886957Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1250 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:33.886990Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1250 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:33.887000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1250 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:33.887003Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:33.887417Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1250 sessionId:  describe result for acl check
2025-10-09T21:48:33.887450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:33.887457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:33.887458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:33.887465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342214023502603:4760] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:33.887988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342214023502603:4760] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:33.890437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342214023502603:4760] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:33.892893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:33.893195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342214023502603:4760] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:33.893210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342214023502603:4760] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:33.893214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342214023502603:4760] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:33.893223Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1250 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:33.894009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:33.894025Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:33.894048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:33.894053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:33.894060Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1249 [2:7559342213661679557:9425]
2025-10-09T21:48:33.894074Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:33.894096Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:33.894107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:33.894111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:33.894113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:33.894133Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:33.894140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:33.894147Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:33.894203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.894212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.894214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.894216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:33.894284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:33.894293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.894305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:33.894317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:33.894338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:33.894364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342214023502637:4760], now have 1 active actors on pipe
2025-10-09T21:48:33.894372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342214023502560:2432], now have 1 active actors on pipe
2025-10-09T21:48:33.894380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342213661679552:9425]
2025-10-09T21:48:33.894385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342213661679552:9425]
2025-10-09T21:48:33.894388Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342213661679552:9425]
2025-10-09T21:48:33.894403Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:33.894400Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1249
2025-10-09T21:48:33.894414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1249
2025-10-09T21:48:33.894422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:33.894439Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:33.894442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:33.894444Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:33.894447Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:33.894450Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:33.894493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:33.894503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:33.894516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342213661679552:9425]
2025-10-09T21:48:33.894523Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:33.894536Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:33.894554Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3b15550e-56d588d-3e1992d2-8243503f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:33.894561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:33.894566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:33.894567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:33.894575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:33.894583Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:33.894599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:33.894797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:33.894811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:33.894834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:33.894919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1250 partition: 0 MaxSeqNo: 25 sessionId: src_id|3b15550e-56d588d-3e1992d2-8243503f_0
2025-10-09T21:48:33.895259Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5925354e-c29bb2d-140ce81c-adc1569e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3b15550e-56d588d-3e1992d2-8243503f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:33.897627Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3b15550e-56d588d-3e1992d2-8243503f_0] Write session: try to update token
2025-10-09T21:48:33.897642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3b15550e-56d588d-3e1992d2-8243503f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:33.909145Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:33.909163Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:33.909164Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:33.909166Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:33.909174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342213661679552:9425]
2025-10-09T21:48:33.909178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342213661679552:9425]
2025-10-09T21:48:33.909180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342213661679552:9425]
2025-10-09T21:48:33.909216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:33.909222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:33.909224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:33.909229Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:33.909236Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:33.909239Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:33.909363Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:33.911022Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1250 sessionId: src_id|3b15550e-56d588d-3e1992d2-8243503f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:33.913507Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:33.975291Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342214023502644:5175]
2025-10-09T21:48:33.975562Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 1249, pipe: [1:7559342214023502644:5175]
2025-10-09T21:48:34.064530Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:34.064835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:34.064850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:34.064897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342214023502644:5175], now have 1 active actors on pipe
2025-10-09T21:48:34.065097Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:34.065114Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342214023502644:5175]
2025-10-09T21:48:34.065153Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:34.065158Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:34.065238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1250 reason: error at write: Disconnected sessionId: src_id|3b15550e-56d588d-3e1992d2-8243503f_0
2025-10-09T21:48:34.065274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:34.065416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1250 sessionId: src_id|3b15550e-56d588d-3e1992d2-8243503f_0 is DEAD
2025-10-09T21:48:34.065569Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b15550e-56d588d-3e1992d2-8243503f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:34.065580Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b15550e-56d588d-3e1992d2-8243503f_0] Write session will restart in 0.005153s
2025-10-09T21:48:34.065597Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b15550e-56d588d-3e1992d2-8243503f_0] Write session: Do CDS request
2025-10-09T21:48:34.065612Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b15550e-56d588d-3e1992d2-8243503f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:34.071486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3b15550e-56d588d-3e1992d2-8243503f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:34.071580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:34.071594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1251
2025-10-09T21:48:34.071713Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1251 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:34.071746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1251 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:34.071756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1251 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:34.071760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:34.072258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1251 sessionId:  describe result for acl check
2025-10-09T21:48:34.072294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:34.072300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:34.072302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:34.072307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342218318469979:4761] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:34.072838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342218318469979:4761] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:34.073813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:34.075115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342218318469979:4761] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:34.075385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:34.075561Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:34.075573Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:34.075586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:34.075652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:34.075697Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342217956646929:9427]
2025-10-09T21:48:34.076288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:34.076584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:34.076676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:34.077007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:34.077055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.077061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.077062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.077063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.077065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.077066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.077067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.077068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.077069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.077075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.077086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.077090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.077094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.077098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.077102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.077106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.077110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.077114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.077121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.077129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.077135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.077141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.077145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.077150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.077153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.077159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.077169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.077174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.077181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.077186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.077203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.077208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.077212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.077218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.077204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342218318469979:4761] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:34.077217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342218318469979:4761] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:34.077221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342218318469979:4761] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:34.077228Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1251 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:34.077231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.077241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.077269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:34.083446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:34.084495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:34.084506Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:34.084509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:34.084512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:34.084519Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1250 [2:7559342217956646929:9427]
2025-10-09T21:48:34.084531Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:34.084547Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:34.084550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:34.084553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:34.084555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:34.084570Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:34.084576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:34.084582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:34.084633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.084639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.084641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.084644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:34.084706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:34.084713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.084723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.084738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.084753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:34.084773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342218318469943:2432], now have 1 active actors on pipe
2025-10-09T21:48:34.084781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342218318470019:4761], now have 1 active actors on pipe
2025-10-09T21:48:34.084788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342217956646924:9427]
2025-10-09T21:48:34.084792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342217956646924:9427]
2025-10-09T21:48:34.084794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342217956646924:9427]
2025-10-09T21:48:34.084806Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:34.084820Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:34.084820Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:34.084823Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:34.084824Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:34.084834Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:34.084845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:34.084828Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1250
2025-10-09T21:48:34.084838Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1250
2025-10-09T21:48:34.084866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342217956646924:9427]
2025-10-09T21:48:34.084874Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:34.084877Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:34.084888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:34.084892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:34.084936Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fce9a969-98f6265f-b100ae8a-c580f4f5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:34.084949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:34.084953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:34.084956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:34.084965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:34.084979Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:34.085002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:34.085127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:34.085135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:34.085149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:34.085232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1251 partition: 0 MaxSeqNo: 25 sessionId: src_id|fce9a969-98f6265f-b100ae8a-c580f4f5_0
2025-10-09T21:48:34.085562Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b15550e-56d588d-3e1992d2-8243503f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fce9a969-98f6265f-b100ae8a-c580f4f5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:34.088019Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fce9a969-98f6265f-b100ae8a-c580f4f5_0] Write session: try to update token
2025-10-09T21:48:34.088035Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fce9a969-98f6265f-b100ae8a-c580f4f5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:34.099106Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:34.099123Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:34.099126Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:34.099130Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:34.099141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342217956646924:9427]
2025-10-09T21:48:34.099146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342217956646924:9427]
2025-10-09T21:48:34.099150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342217956646924:9427]
2025-10-09T21:48:34.099197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:34.099205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.099210Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:34.099215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:34.099219Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:34.099223Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:34.099355Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:34.100914Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1251 sessionId: src_id|fce9a969-98f6265f-b100ae8a-c580f4f5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:34.103523Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:34.165270Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342218318470031:5175]
2025-10-09T21:48:34.165557Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 1250, pipe: [1:7559342218318470031:5175]
2025-10-09T21:48:34.254574Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:34.254880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:34.254894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:34.254933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342218318470031:5175], now have 1 active actors on pipe
2025-10-09T21:48:34.255129Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:34.255221Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:34.255234Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342218318470031:5175]
2025-10-09T21:48:34.255253Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:34.255321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1251 reason: status is not ok: tablet will be restarted right now sessionId: src_id|fce9a969-98f6265f-b100ae8a-c580f4f5_0
2025-10-09T21:48:34.255491Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1251 sessionId: src_id|fce9a969-98f6265f-b100ae8a-c580f4f5_0 is DEAD
2025-10-09T21:48:34.255669Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fce9a969-98f6265f-b100ae8a-c580f4f5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:34.255681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fce9a969-98f6265f-b100ae8a-c580f4f5_0] Write session will restart in 0.008926s
2025-10-09T21:48:34.255710Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fce9a969-98f6265f-b100ae8a-c580f4f5_0] Write session: Do CDS request
2025-10-09T21:48:34.255717Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fce9a969-98f6265f-b100ae8a-c580f4f5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:34.264252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:34.265598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fce9a969-98f6265f-b100ae8a-c580f4f5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:34.265647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:34.265665Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1252
2025-10-09T21:48:34.265836Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1252 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:34.265869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1252 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:34.265880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1252 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:34.265884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:34.266063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:34.266283Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:34.266295Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:34.266307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:34.266354Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1252 sessionId:  describe result for acl check
2025-10-09T21:48:34.266361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:34.266389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:34.266406Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342217956647006:9429]
2025-10-09T21:48:34.266397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:34.266398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:34.266405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342218318470079:4750] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:34.266928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:34.266941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342218318470079:4750] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:34.267332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:34.267390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:34.267715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:34.267789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.267795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.267797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.267798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.267799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.267799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.267800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.267801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.267801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.267805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.267811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.267813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.267816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.267818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.267820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.267822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.267824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.267826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.267831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.267837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.267841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.267844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.267847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.267850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.267853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.267856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.267858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.267862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.267866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.267871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.267876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.267880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.267884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.267892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.267898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.267903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.267929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:34.269051Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342218318470079:4750] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:34.274202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342218318470079:4750] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:34.274226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342218318470079:4750] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:34.274230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342218318470079:4750] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:34.274241Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1252 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:34.274571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:34.275608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:34.275626Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:34.275628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:34.275633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:34.275639Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1251 [2:7559342217956647006:9429]
2025-10-09T21:48:34.275650Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:34.275665Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:34.275667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:34.275669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:34.275670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:34.275685Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:34.275696Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:34.275705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:34.275764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.275768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.275769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.275771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:34.275830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:34.275837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.275850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.275869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.275888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:34.275914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342218318470118:4750], now have 1 active actors on pipe
2025-10-09T21:48:34.275922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342218318470039:2432], now have 1 active actors on pipe
2025-10-09T21:48:34.275930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342217956647001:9429]
2025-10-09T21:48:34.275932Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342217956647001:9429]
2025-10-09T21:48:34.275934Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342217956647001:9429]
2025-10-09T21:48:34.275945Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:34.275956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:34.275960Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:34.275966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:34.275969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:34.275969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:34.275971Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:34.276009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342217956647001:9429]
2025-10-09T21:48:34.275999Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1251
2025-10-09T21:48:34.276018Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:34.275998Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1251
2025-10-09T21:48:34.276026Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:34.276044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:34.276047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:34.276175Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|28d9e9f2-d71432d1-6cfb42a5-ece41432_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:34.276192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:34.276196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:34.276198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:34.276209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:34.276224Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:34.276249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:34.276404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:34.276414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:34.276433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:34.276521Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1252 partition: 0 MaxSeqNo: 25 sessionId: src_id|28d9e9f2-d71432d1-6cfb42a5-ece41432_0
2025-10-09T21:48:34.276900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fce9a969-98f6265f-b100ae8a-c580f4f5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|28d9e9f2-d71432d1-6cfb42a5-ece41432_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:34.279759Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28d9e9f2-d71432d1-6cfb42a5-ece41432_0] Write session: try to update token
2025-10-09T21:48:34.279781Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28d9e9f2-d71432d1-6cfb42a5-ece41432_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:34.290835Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:34.290855Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:34.290860Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:34.290863Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:34.290875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342217956647001:9429]
2025-10-09T21:48:34.290879Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342217956647001:9429]
2025-10-09T21:48:34.290883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342217956647001:9429]
2025-10-09T21:48:34.290929Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:34.290934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:34.290941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.290944Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:34.290944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:34.290947Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:34.291099Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:34.295688Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1252 sessionId: src_id|28d9e9f2-d71432d1-6cfb42a5-ece41432_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:34.298374Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:34.355202Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342218318470126:5175]
2025-10-09T21:48:34.355466Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 1251, pipe: [1:7559342218318470126:5175]
2025-10-09T21:48:34.448411Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:34.448703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:34.448717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:34.448764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342218318470126:5175], now have 1 active actors on pipe
2025-10-09T21:48:34.449007Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:34.449058Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:34.449069Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342218318470126:5175]
2025-10-09T21:48:34.449107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:34.449181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1252 reason: status is not ok: tablet will be restarted right now sessionId: src_id|28d9e9f2-d71432d1-6cfb42a5-ece41432_0
2025-10-09T21:48:34.449379Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1252 sessionId: src_id|28d9e9f2-d71432d1-6cfb42a5-ece41432_0 is DEAD
2025-10-09T21:48:34.449495Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28d9e9f2-d71432d1-6cfb42a5-ece41432_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:34.449504Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28d9e9f2-d71432d1-6cfb42a5-ece41432_0] Write session will restart in 0.009821s
2025-10-09T21:48:34.449531Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28d9e9f2-d71432d1-6cfb42a5-ece41432_0] Write session: Do CDS request
2025-10-09T21:48:34.449537Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28d9e9f2-d71432d1-6cfb42a5-ece41432_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:34.457753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:34.459263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:34.459458Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:34.459472Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:34.459484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:34.459535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:34.459567Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342217956647084:9431]
2025-10-09T21:48:34.460141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:34.460469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:34.460522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:34.460527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28d9e9f2-d71432d1-6cfb42a5-ece41432_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:34.460623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:34.460637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1253
2025-10-09T21:48:34.460755Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1253 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:34.460792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1253 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:34.460805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1253 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:34.460808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:34.460884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:34.460949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.460955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.460956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.460957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.460958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.460959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.460960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.460961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.460962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.460968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.460978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.460983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.460987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.460991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.460995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.461000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.461004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.461008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.461015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.461025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.461035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.461040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.461045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.461049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.461053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.461063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.461068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.461073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.461082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.461087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.461093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.461098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.461102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.461106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.461112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.461116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.461147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:34.461233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1253 sessionId:  describe result for acl check
2025-10-09T21:48:34.461273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:34.461280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:34.461282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:34.461288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342218318470172:4751] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:34.461874Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342218318470172:4751] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:34.464080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342218318470172:4751] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:34.467098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342218318470172:4751] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:34.467114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342218318470172:4751] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:34.467120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342218318470172:4751] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:34.467128Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1253 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:34.467453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:34.468461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:34.468474Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:34.468477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:34.468481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:34.468487Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1252 [2:7559342217956647084:9431]
2025-10-09T21:48:34.468500Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:34.468515Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:34.468519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:34.468522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:34.468524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:34.468540Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:34.468552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:34.468559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:34.468609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.468615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.468617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.468620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:34.468684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:34.468692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.468703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.468720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.468741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:34.468765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342218318470128:2432], now have 1 active actors on pipe
2025-10-09T21:48:34.468773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342218318470208:4751], now have 1 active actors on pipe
2025-10-09T21:48:34.468780Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342217956647079:9431]
2025-10-09T21:48:34.468783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342217956647079:9431]
2025-10-09T21:48:34.468785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342217956647079:9431]
2025-10-09T21:48:34.468798Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:34.468816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:34.468818Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:34.468821Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:34.468823Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:34.468836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:34.468839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:34.468813Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1252
2025-10-09T21:48:34.468824Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1252
2025-10-09T21:48:34.468861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342217956647079:9431]
2025-10-09T21:48:34.468869Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:34.468870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:34.468885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:34.468894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:34.468939Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2e050888-f43d2b0c-ccb3aaf4-b96e7b9c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:34.468950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:34.468955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:34.468957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:34.468968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:34.468981Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:34.469003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:34.469178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:34.469192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:34.469213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:34.469325Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1253 partition: 0 MaxSeqNo: 25 sessionId: src_id|2e050888-f43d2b0c-ccb3aaf4-b96e7b9c_0
2025-10-09T21:48:34.469608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28d9e9f2-d71432d1-6cfb42a5-ece41432_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2e050888-f43d2b0c-ccb3aaf4-b96e7b9c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:34.472013Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e050888-f43d2b0c-ccb3aaf4-b96e7b9c_0] Write session: try to update token
2025-10-09T21:48:34.472031Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e050888-f43d2b0c-ccb3aaf4-b96e7b9c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:34.483064Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:34.483083Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:34.483086Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:34.483088Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:34.483099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342217956647079:9431]
2025-10-09T21:48:34.483104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342217956647079:9431]
2025-10-09T21:48:34.483112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342217956647079:9431]
2025-10-09T21:48:34.483154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:34.483153Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:34.483159Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:34.483163Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:34.483166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.483168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:34.483291Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:34.484845Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1253 sessionId: src_id|2e050888-f43d2b0c-ccb3aaf4-b96e7b9c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:34.487167Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:34.549253Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342218318470216:5175]
2025-10-09T21:48:34.549498Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 1252, pipe: [1:7559342218318470216:5175]
2025-10-09T21:48:34.637882Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:34.638209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:34.638222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:34.638260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342218318470216:5175], now have 1 active actors on pipe
2025-10-09T21:48:34.638515Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:34.638556Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342218318470216:5175]
2025-10-09T21:48:34.638564Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:34.638567Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:34.638627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1253 reason: error at write: Disconnected sessionId: src_id|2e050888-f43d2b0c-ccb3aaf4-b96e7b9c_0
2025-10-09T21:48:34.638693Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:34.638806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1253 sessionId: src_id|2e050888-f43d2b0c-ccb3aaf4-b96e7b9c_0 is DEAD
2025-10-09T21:48:34.639002Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e050888-f43d2b0c-ccb3aaf4-b96e7b9c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:34.639014Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e050888-f43d2b0c-ccb3aaf4-b96e7b9c_0] Write session will restart in 0.005435s
2025-10-09T21:48:34.639042Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e050888-f43d2b0c-ccb3aaf4-b96e7b9c_0] Write session: Do CDS request
2025-10-09T21:48:34.639049Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e050888-f43d2b0c-ccb3aaf4-b96e7b9c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:34.645489Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e050888-f43d2b0c-ccb3aaf4-b96e7b9c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:34.645556Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:34.645568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1254
2025-10-09T21:48:34.645691Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1254 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:34.645753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1254 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:34.645764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1254 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:34.645768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:34.646244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1254 sessionId:  describe result for acl check
2025-10-09T21:48:34.646280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:34.646286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:34.646288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:34.646294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342218318470259:4740] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:34.646497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:34.646841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342218318470259:4740] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:34.648253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:34.648399Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:34.648408Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:34.648419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:34.648466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:34.648506Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342217956647165:9433]
2025-10-09T21:48:34.648877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:34.648933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342218318470259:4740] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:34.649152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:34.649212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:34.649540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:34.649600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.649607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.649608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.649609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.649610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.649611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.649612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.649613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.649615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.649620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.649630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.649636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.649640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.649645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.649649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.649653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.649657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.649661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.649667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.649675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.649681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.649686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.649692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.649710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.649715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.649726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.649731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.649741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.649750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.649755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.649760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.649770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.649774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.649779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.649788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.649793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.649823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:34.654770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342218318470259:4740] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:34.654785Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342218318470259:4740] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:34.654789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342218318470259:4740] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:34.654798Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1254 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:34.656052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:34.657178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:34.657193Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:34.657195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:34.657201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:34.657206Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1253 [2:7559342217956647165:9433]
2025-10-09T21:48:34.657219Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:34.657235Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:34.657243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:34.657246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:34.657248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:34.657264Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:34.657274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:34.657280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:34.657333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.657340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.657342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.657345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:34.657410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:34.657417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.657428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.657439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.657457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:34.657482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342218318470218:2432], now have 1 active actors on pipe
2025-10-09T21:48:34.657490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342218318470299:4740], now have 1 active actors on pipe
2025-10-09T21:48:34.657497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342217956647160:9433]
2025-10-09T21:48:34.657500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342217956647160:9433]
2025-10-09T21:48:34.657502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342217956647160:9433]
2025-10-09T21:48:34.657516Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:34.657531Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:34.657518Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1253
2025-10-09T21:48:34.657546Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:34.657529Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1253
2025-10-09T21:48:34.657551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:34.657554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:34.657555Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:34.657557Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:34.657593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:34.657602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:34.657614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342217956647160:9433]
2025-10-09T21:48:34.657625Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:34.657629Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:34.657656Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|baae8d32-54611d15-ac373e2c-74c1e158_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:34.657667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:34.657673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:34.657675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:34.657684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:34.657709Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:34.657736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:34.657843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:34.657849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:34.657862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:34.657956Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1254 partition: 0 MaxSeqNo: 25 sessionId: src_id|baae8d32-54611d15-ac373e2c-74c1e158_0
2025-10-09T21:48:34.658293Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e050888-f43d2b0c-ccb3aaf4-b96e7b9c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|baae8d32-54611d15-ac373e2c-74c1e158_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:34.660726Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|baae8d32-54611d15-ac373e2c-74c1e158_0] Write session: try to update token
2025-10-09T21:48:34.660741Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|baae8d32-54611d15-ac373e2c-74c1e158_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:34.671854Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:34.671874Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:34.671876Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:34.671878Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:34.671889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342217956647160:9433]
2025-10-09T21:48:34.671893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342217956647160:9433]
2025-10-09T21:48:34.671897Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342217956647160:9433]
2025-10-09T21:48:34.671942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:34.671942Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:34.671948Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:34.671950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.671952Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:34.671952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:34.672083Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:34.673408Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1254 sessionId: src_id|baae8d32-54611d15-ac373e2c-74c1e158_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:34.675839Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:34.738637Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342218318470308:5175]
2025-10-09T21:48:34.738858Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 1253, pipe: [1:7559342218318470308:5175]
2025-10-09T21:48:34.829685Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:34.829913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:34.829925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:34.829962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342218318470308:5175], now have 1 active actors on pipe
2025-10-09T21:48:34.830212Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342218318470308:5175]
2025-10-09T21:48:34.830223Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:34.830239Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:34.830315Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:34.830326Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:34.830385Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1254 reason: error at write: Disconnected sessionId: src_id|baae8d32-54611d15-ac373e2c-74c1e158_0
2025-10-09T21:48:34.830586Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1254 sessionId: src_id|baae8d32-54611d15-ac373e2c-74c1e158_0 is DEAD
2025-10-09T21:48:34.830718Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baae8d32-54611d15-ac373e2c-74c1e158_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:34.830729Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baae8d32-54611d15-ac373e2c-74c1e158_0] Write session will restart in 0.007776s
2025-10-09T21:48:34.830766Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baae8d32-54611d15-ac373e2c-74c1e158_0] Write session: Do CDS request
2025-10-09T21:48:34.830774Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baae8d32-54611d15-ac373e2c-74c1e158_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:34.838892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:34.839517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|baae8d32-54611d15-ac373e2c-74c1e158_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:34.839587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:34.839612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1255
2025-10-09T21:48:34.839730Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1255 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:34.839762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1255 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:34.839772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1255 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:34.839776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:34.840188Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1255 sessionId:  describe result for acl check
2025-10-09T21:48:34.840236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:34.840243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:34.840273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:34.840245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:34.840251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342218318470383:4719] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:34.840444Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:34.840455Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:34.840466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:34.840530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:34.840569Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342217956647287:9449]
2025-10-09T21:48:34.840806Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342218318470383:4719] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:34.841056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:34.841349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:34.841395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:34.841691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:34.841763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.841769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.841771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.841772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.841773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.841773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.841774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.841774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.841775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.841778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.841784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.841786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.841789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.841791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.841793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.841795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.841797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.841800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.841804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.841810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:34.841817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.841823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:34.841827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.841831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:34.841835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.841841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:34.841846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.841851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:34.841855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.841860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:34.841865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.841870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.841875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.841881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.841885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.841889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.841918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:34.843290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342218318470383:4719] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:34.847785Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342218318470383:4719] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:34.847799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342218318470383:4719] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:34.847804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342218318470383:4719] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:34.847814Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1255 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:34.848137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:34.849271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:34.849292Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:34.849296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:34.849300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:34.849306Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1254 [2:7559342217956647287:9449]
2025-10-09T21:48:34.849319Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:34.849336Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:34.849344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:34.849347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:34.849350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:34.849366Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:34.849378Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:34.849390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:34.849447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.849453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.849455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.849458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:34.849521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:34.849528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.849539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:34.849554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:34.849576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:34.849596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342218318470420:4719], now have 1 active actors on pipe
2025-10-09T21:48:34.849604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342218318470340:2432], now have 1 active actors on pipe
2025-10-09T21:48:34.849612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342217956647282:9449]
2025-10-09T21:48:34.849614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342217956647282:9449]
2025-10-09T21:48:34.849617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342217956647282:9449]
2025-10-09T21:48:34.849624Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:34.849602Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1254
2025-10-09T21:48:34.849637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:34.849629Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1254
2025-10-09T21:48:34.849645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:34.849645Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:34.849647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:34.849659Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:34.849663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:34.849678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:34.849687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:34.849735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342217956647282:9449]
2025-10-09T21:48:34.849753Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:34.849771Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e93b5-49168f0b-3868dabf-af64c444_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:34.849775Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:34.849779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:34.849785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:34.849787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:34.849798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:34.849807Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:34.849840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:34.850010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:34.850017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:34.850048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:34.850135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1255 partition: 0 MaxSeqNo: 25 sessionId: src_id|e93b5-49168f0b-3868dabf-af64c444_0
2025-10-09T21:48:34.850454Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baae8d32-54611d15-ac373e2c-74c1e158_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e93b5-49168f0b-3868dabf-af64c444_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:34.852762Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e93b5-49168f0b-3868dabf-af64c444_0] Write session: try to update token
2025-10-09T21:48:34.852779Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e93b5-49168f0b-3868dabf-af64c444_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:34.864160Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:34.864178Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:34.864180Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:34.864183Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:34.864193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342217956647282:9449]
2025-10-09T21:48:34.864198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342217956647282:9449]
2025-10-09T21:48:34.864202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342217956647282:9449]
2025-10-09T21:48:34.864248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:34.864258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:34.864261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:34.864271Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:34.864281Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:34.864287Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:34.864397Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:34.865767Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1255 sessionId: src_id|e93b5-49168f0b-3868dabf-af64c444_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:34.868219Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:34.930184Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342218318470428:5175]
2025-10-09T21:48:34.930453Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 1254, pipe: [1:7559342218318470428:5175]
2025-10-09T21:48:35.014198Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:35.014461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.014474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.014512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342218318470428:5175], now have 1 active actors on pipe
2025-10-09T21:48:35.014687Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:35.014754Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:35.014822Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342218318470428:5175]
2025-10-09T21:48:35.014921Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:35.015002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1255 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e93b5-49168f0b-3868dabf-af64c444_0
2025-10-09T21:48:35.015195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1255 sessionId: src_id|e93b5-49168f0b-3868dabf-af64c444_0 is DEAD
2025-10-09T21:48:35.015329Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e93b5-49168f0b-3868dabf-af64c444_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:35.015337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e93b5-49168f0b-3868dabf-af64c444_0] Write session will restart in 0.009557s
2025-10-09T21:48:35.015355Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e93b5-49168f0b-3868dabf-af64c444_0] Write session: Do CDS request
2025-10-09T21:48:35.015364Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e93b5-49168f0b-3868dabf-af64c444_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:35.023973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:35.025774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:35.025953Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:35.025963Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:35.025973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:35.026019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:35.026080Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342222251614664:9451]
2025-10-09T21:48:35.026509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:35.026528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e93b5-49168f0b-3868dabf-af64c444_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:35.026595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:35.026613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1256
2025-10-09T21:48:35.026765Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1256 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:35.026799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1256 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:35.026810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1256 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:35.026814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:35.026814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:35.026871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:35.027199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:35.027244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.027253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.027254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.027255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.027256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.027258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.027259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1256 sessionId:  describe result for acl check
2025-10-09T21:48:35.027259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.027297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:35.027260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.027299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:35.027261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.027301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:35.027307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342222613437776:4718] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:35.027266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.027277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.027281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.027285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.027289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.027294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.027299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.027303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.027307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.027314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.027322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.027328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.027333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.027337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.027341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.027345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.027351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.027356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.027362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.027366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.027371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.027376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.027381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.027386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.027390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.027397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.027403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.027432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:35.027856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342222613437776:4718] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:35.029827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342222613437776:4718] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:35.033503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342222613437776:4718] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:35.033516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342222613437776:4718] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:35.033519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342222613437776:4718] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:35.033526Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1256 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:35.033898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:35.035034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:35.035054Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:35.035056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:35.035061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:35.035067Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1255 [2:7559342222251614664:9451]
2025-10-09T21:48:35.035081Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:35.035098Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:35.035101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:35.035103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:35.035107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:35.035125Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:35.035136Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:35.035152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:35.035208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.035216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.035217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.035220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:35.035358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:35.035367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.035377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.035388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.035404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:35.035436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613437809:4718], now have 1 active actors on pipe
2025-10-09T21:48:35.035444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613437729:2432], now have 1 active actors on pipe
2025-10-09T21:48:35.035439Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1255
2025-10-09T21:48:35.035451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342222251614659:9451]
2025-10-09T21:48:35.035445Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1255
2025-10-09T21:48:35.035458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342222251614659:9451]
2025-10-09T21:48:35.035460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342222251614659:9451]
2025-10-09T21:48:35.035469Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:35.035483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:35.035486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:35.035500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:35.035501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:35.035503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:35.035503Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:35.035551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.035560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.035577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342222251614659:9451]
2025-10-09T21:48:35.035591Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:35.035603Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:35.035625Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|abc1d11-fcad3429-97469d9c-b6fa4f7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:35.035635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:35.035641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:35.035644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:35.035653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:35.035662Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:35.035684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:35.035864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.035873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.035887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:35.035970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1256 partition: 0 MaxSeqNo: 25 sessionId: src_id|abc1d11-fcad3429-97469d9c-b6fa4f7_0
2025-10-09T21:48:35.036333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e93b5-49168f0b-3868dabf-af64c444_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|abc1d11-fcad3429-97469d9c-b6fa4f7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:35.038731Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|abc1d11-fcad3429-97469d9c-b6fa4f7_0] Write session: try to update token
2025-10-09T21:48:35.038749Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|abc1d11-fcad3429-97469d9c-b6fa4f7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:35.049966Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:35.049991Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:35.049994Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:35.049995Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:35.050005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342222251614659:9451]
2025-10-09T21:48:35.050009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342222251614659:9451]
2025-10-09T21:48:35.050012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342222251614659:9451]
2025-10-09T21:48:35.050068Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:35.050072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:35.050079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.050082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:35.050083Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:35.050096Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:35.050213Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:35.051726Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1256 sessionId: src_id|abc1d11-fcad3429-97469d9c-b6fa4f7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:35.054104Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:35.115034Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342222613437816:5175]
2025-10-09T21:48:35.115290Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 1255, pipe: [1:7559342222613437816:5175]
2025-10-09T21:48:35.208173Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:35.208577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.208600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.208649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613437816:5175], now have 1 active actors on pipe
2025-10-09T21:48:35.208891Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:35.208883Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342222613437816:5175]
2025-10-09T21:48:35.208890Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:35.208924Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:35.208927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:35.208988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1256 reason: error at write: Disconnected sessionId: src_id|abc1d11-fcad3429-97469d9c-b6fa4f7_0
2025-10-09T21:48:35.209174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1256 sessionId: src_id|abc1d11-fcad3429-97469d9c-b6fa4f7_0 is DEAD
2025-10-09T21:48:35.209342Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|abc1d11-fcad3429-97469d9c-b6fa4f7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:35.209355Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|abc1d11-fcad3429-97469d9c-b6fa4f7_0] Write session will restart in 0.005746s
2025-10-09T21:48:35.209385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|abc1d11-fcad3429-97469d9c-b6fa4f7_0] Write session: Do CDS request
2025-10-09T21:48:35.209392Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|abc1d11-fcad3429-97469d9c-b6fa4f7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:35.216589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|abc1d11-fcad3429-97469d9c-b6fa4f7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:35.216679Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:35.216695Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1257
2025-10-09T21:48:35.216814Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1257 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:35.216864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1257 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:35.216875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1257 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:35.216879Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:35.217288Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1257 sessionId:  describe result for acl check
2025-10-09T21:48:35.217325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:35.217330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:35.217332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:35.217339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342222613437864:4698] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:35.217883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342222613437864:4698] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:35.218017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:35.219854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342222613437864:4698] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:35.219930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:35.220123Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:35.220134Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:35.220145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:35.220198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:35.220239Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342222251614746:9453]
2025-10-09T21:48:35.220777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:35.221065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:35.221124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:35.221477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:35.221528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.221536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.221537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.221539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.221540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.221542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.221544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.221545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.221547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.221562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.221573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.221584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.221587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.221592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.221596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.221600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.221604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.221607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.221615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.221623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.221635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.221641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.221646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.221651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.221660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.221664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.221668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.221673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.221677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.221681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.221688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.221702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.221718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.221724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.221729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.221734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.221778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:35.222230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342222613437864:4698] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:35.222245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342222613437864:4698] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:35.222249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342222613437864:4698] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:35.222259Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1257 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:35.228094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:35.229239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:35.229256Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:35.229259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:35.229263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:35.229270Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1256 [2:7559342222251614746:9453]
2025-10-09T21:48:35.229286Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:35.229308Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:35.229316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:35.229319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:35.229321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:35.229337Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:35.229348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:35.229358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:35.229422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.229430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.229432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.229434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:35.229504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:35.229511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.229522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.229537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.229556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:35.229585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613437911:4698], now have 1 active actors on pipe
2025-10-09T21:48:35.229590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613437822:2432], now have 1 active actors on pipe
2025-10-09T21:48:35.229598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342222251614741:9453]
2025-10-09T21:48:35.229600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342222251614741:9453]
2025-10-09T21:48:35.229602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342222251614741:9453]
2025-10-09T21:48:35.229615Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:35.229630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:35.229636Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:35.229639Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:35.229642Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:35.229648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:35.229653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:35.229687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342222251614741:9453]
2025-10-09T21:48:35.229687Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1256
2025-10-09T21:48:35.229714Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:35.229719Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:35.229708Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1256
2025-10-09T21:48:35.229763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.229771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.229797Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e6bc53f6-b437ad9f-25745900-472d772a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:35.229809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:35.229815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:35.229822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:35.229834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:35.229851Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:35.229865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:35.230047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.230056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.230069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:35.230255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1257 partition: 0 MaxSeqNo: 25 sessionId: src_id|e6bc53f6-b437ad9f-25745900-472d772a_0
2025-10-09T21:48:35.230599Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|abc1d11-fcad3429-97469d9c-b6fa4f7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e6bc53f6-b437ad9f-25745900-472d772a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:35.233095Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e6bc53f6-b437ad9f-25745900-472d772a_0] Write session: try to update token
2025-10-09T21:48:35.233113Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e6bc53f6-b437ad9f-25745900-472d772a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:35.246283Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1257 sessionId: src_id|e6bc53f6-b437ad9f-25745900-472d772a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:35.247153Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:35.247173Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:35.247184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342222251614741:9453]
2025-10-09T21:48:35.247238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:35.247249Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:35.247414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:35.247562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:35.247702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:35.247837Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:35.248025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:35.248071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:35.248670Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:35.248866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.248881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.248912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:35.248994Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:35.254773Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:35.254797Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:35.254803Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:35.254805Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:35.254815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342222251614741:9453]
2025-10-09T21:48:35.254819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342222251614741:9453]
2025-10-09T21:48:35.254822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342222251614741:9453]
2025-10-09T21:48:35.254864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:35.254868Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:35.254875Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:35.254877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.254879Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:35.254880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:35.255006Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:35.309004Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342222613437918:5175]
2025-10-09T21:48:35.309261Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 1256, pipe: [1:7559342222613437918:5175]
2025-10-09T21:48:35.411144Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:35.417917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.417940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.418094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:35.418146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:35.418202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:35.418257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:35.418302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:35.418312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:35.418377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:35.418432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:35.418489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:35.418543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:35.418581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:35.418588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:35.418640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:35.418689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:35.418740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:35.418792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:35.419191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:35.419207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:35.419269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:35.419321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:35.419373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:35.419430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:35.419470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:35.419472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:35.419528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:35.419581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:35.419636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:35.419688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:35.419728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:35.419734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:35.419787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:35.419839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:35.419891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:35.419945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:35.419984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:35.419989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:35.420041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:35.420103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:35.420159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:35.420211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:35.420248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:35.420253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:35.420305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:35.420357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:35.420408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:35.420460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:35.420512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:35.420517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:35.420568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:35.420616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:35.420665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:35.420719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:35.420756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:35.420761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:35.420813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:35.420863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:35.420916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:35.420968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:35.421008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:35.421010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:35.421048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613437918:5175], now have 1 active actors on pipe
2025-10-09T21:48:35.421306Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:35.421311Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:35.421323Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:35.421354Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1257 reason: error at write: Disconnected sessionId: src_id|e6bc53f6-b437ad9f-25745900-472d772a_0
2025-10-09T21:48:35.421355Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342222613437918:5175]
2025-10-09T21:48:35.421471Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:35.421550Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1257 sessionId: src_id|e6bc53f6-b437ad9f-25745900-472d772a_0 is DEAD
2025-10-09T21:48:35.421675Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6bc53f6-b437ad9f-25745900-472d772a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:35.421686Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6bc53f6-b437ad9f-25745900-472d772a_0] Write session will restart in 0.007721s
2025-10-09T21:48:35.421733Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6bc53f6-b437ad9f-25745900-472d772a_0] Write session: Do CDS request
2025-10-09T21:48:35.421741Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6bc53f6-b437ad9f-25745900-472d772a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:35.430517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e6bc53f6-b437ad9f-25745900-472d772a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:35.430612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:35.430629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1258
2025-10-09T21:48:35.430771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:35.430887Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1258 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:35.430922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1258 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:35.430930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1258 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:35.430934Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:35.431375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1258 sessionId:  describe result for acl check
2025-10-09T21:48:35.431412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:35.431418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:35.431420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:35.431427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342222613437967:4707] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:35.431960Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342222613437967:4707] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:35.432387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:35.432571Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:35.432585Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:35.432596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:35.432652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:35.432697Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342222251614830:9455]
2025-10-09T21:48:35.433281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:35.433619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:35.433672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:35.434022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:35.434089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.434098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.434099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.434100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.434101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.434102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.434103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.434104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.434105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.434112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.434123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.434127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.434131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.434134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.434138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.434142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.434146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.434150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.434157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.434165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.434171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.434177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.434181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.434186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.434191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.434196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.434199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.434204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.434209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.434214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.434219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.434224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.434228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.434233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.434237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.434248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.434276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:35.434303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342222613437967:4707] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:35.440191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342222613437967:4707] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:35.440209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342222613437967:4707] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:35.440214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342222613437967:4707] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:35.440222Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1258 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:35.440583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:35.441648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:35.441666Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:35.441668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:35.441675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:35.441683Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1257 [2:7559342222251614830:9455]
2025-10-09T21:48:35.441706Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:35.441725Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:35.441728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:35.441731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:35.441734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:35.441751Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:35.441769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:35.441781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:35.441837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.441843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.441844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.441847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:35.441914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:35.441921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.441934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.441947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.441964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:35.441991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613438005:4707], now have 1 active actors on pipe
2025-10-09T21:48:35.441998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613437921:2432], now have 1 active actors on pipe
2025-10-09T21:48:35.442006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342222251614824:9455]
2025-10-09T21:48:35.442009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342222251614824:9455]
2025-10-09T21:48:35.442011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342222251614824:9455]
2025-10-09T21:48:35.442040Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:35.442061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:35.442080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:35.442087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:35.442067Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1257
2025-10-09T21:48:35.442095Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:35.442105Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:35.442068Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1257
2025-10-09T21:48:35.442108Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:35.442121Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342222251614824:9455]
2025-10-09T21:48:35.442129Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:35.442139Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:35.442148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.442153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.442200Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|28a9a8e0-d54b5250-25a5310-dc67905a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:35.442213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:35.442218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:35.442220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:35.442231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:35.442244Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:35.442266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:35.442404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.442410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.442423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:35.442540Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1258 partition: 0 MaxSeqNo: 25 sessionId: src_id|28a9a8e0-d54b5250-25a5310-dc67905a_0
2025-10-09T21:48:35.442939Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6bc53f6-b437ad9f-25745900-472d772a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|28a9a8e0-d54b5250-25a5310-dc67905a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:35.445363Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28a9a8e0-d54b5250-25a5310-dc67905a_0] Write session: try to update token
2025-10-09T21:48:35.445382Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28a9a8e0-d54b5250-25a5310-dc67905a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:35.455988Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:35.456008Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:35.456010Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:35.456012Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:35.456022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342222251614824:9455]
2025-10-09T21:48:35.456026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342222251614824:9455]
2025-10-09T21:48:35.456029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342222251614824:9455]
2025-10-09T21:48:35.456072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:35.456072Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:35.456079Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:35.456083Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:35.456086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.456091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:35.456223Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:35.457961Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1258 sessionId: src_id|28a9a8e0-d54b5250-25a5310-dc67905a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:35.460171Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:35.521525Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342222613438014:5175]
2025-10-09T21:48:35.521794Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 1257, pipe: [1:7559342222613438014:5175]
2025-10-09T21:48:35.597767Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:35.598050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.598066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.598113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613438014:5175], now have 1 active actors on pipe
2025-10-09T21:48:35.598314Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342222613438014:5175]
2025-10-09T21:48:35.598327Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:35.598332Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:35.598368Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:35.598397Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1258 reason: error at write: Disconnected sessionId: src_id|28a9a8e0-d54b5250-25a5310-dc67905a_0
2025-10-09T21:48:35.598579Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1258 sessionId: src_id|28a9a8e0-d54b5250-25a5310-dc67905a_0 is DEAD
2025-10-09T21:48:35.598742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28a9a8e0-d54b5250-25a5310-dc67905a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:35.598753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28a9a8e0-d54b5250-25a5310-dc67905a_0] Write session will restart in 0.007766s
2025-10-09T21:48:35.598781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28a9a8e0-d54b5250-25a5310-dc67905a_0] Write session: Do CDS request
2025-10-09T21:48:35.598795Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28a9a8e0-d54b5250-25a5310-dc67905a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:35.598993Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:35.607576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28a9a8e0-d54b5250-25a5310-dc67905a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:35.607610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:35.607636Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:35.607653Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1259
2025-10-09T21:48:35.607809Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1259 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:35.607843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1259 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:35.607854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1259 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:35.607857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:35.609377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:35.609563Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:35.609559Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1259 sessionId:  describe result for acl check
2025-10-09T21:48:35.609573Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:35.609583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:35.609600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:35.609608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:35.609610Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:35.609616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342222613438063:4699] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:35.609643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:35.609679Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342222251614913:9457]
2025-10-09T21:48:35.610177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:35.610212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342222613438063:4699] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:35.610515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:35.610565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:35.610854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:35.610910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.610919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.610920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.610921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.610923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.610924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.610925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.610926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.610927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.610932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.610941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.610945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.610950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.610953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.610957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.610961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.610964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.610968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.610974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.610984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.610989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.610994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.610997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.611001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.611004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.611008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.611012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.611016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.611020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.611025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.611029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.611036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.611046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.611051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.611056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.611061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.611106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:35.611954Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342222613438063:4699] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:35.616959Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342222613438063:4699] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:35.616976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342222613438063:4699] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:35.616980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342222613438063:4699] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:35.616989Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1259 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:35.617278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:35.618315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:35.618331Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:35.618334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:35.618339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:35.618345Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1258 [2:7559342222251614913:9457]
2025-10-09T21:48:35.618358Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:35.618375Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:35.618382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:35.618385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:35.618388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:35.618405Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:35.618416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:35.618428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:35.618490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.618497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.618499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.618501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:35.618565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:35.618573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.618585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.618600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.618621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:35.618647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613438102:4699], now have 1 active actors on pipe
2025-10-09T21:48:35.618655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613438015:2432], now have 1 active actors on pipe
2025-10-09T21:48:35.618663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342222251614908:9457]
2025-10-09T21:48:35.618669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342222251614908:9457]
2025-10-09T21:48:35.618671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342222251614908:9457]
2025-10-09T21:48:35.618684Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:35.618702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:35.618672Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1258
2025-10-09T21:48:35.618708Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:35.618680Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1258
2025-10-09T21:48:35.618714Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:35.618716Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:35.618720Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:35.618723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:35.618768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.618776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.618793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342222251614908:9457]
2025-10-09T21:48:35.618803Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:35.618816Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:35.618835Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d7463de9-fdeb0c22-3ffcd7d5-9340f715_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:35.618846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:35.618854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:35.618858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:35.618867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:35.618884Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:35.618907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:35.619033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.619040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.619052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:35.619132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1259 partition: 0 MaxSeqNo: 25 sessionId: src_id|d7463de9-fdeb0c22-3ffcd7d5-9340f715_0
2025-10-09T21:48:35.619467Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28a9a8e0-d54b5250-25a5310-dc67905a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d7463de9-fdeb0c22-3ffcd7d5-9340f715_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:35.621843Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7463de9-fdeb0c22-3ffcd7d5-9340f715_0] Write session: try to update token
2025-10-09T21:48:35.621859Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7463de9-fdeb0c22-3ffcd7d5-9340f715_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:35.632905Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:35.632923Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:35.632925Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:35.632927Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:35.632937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342222251614908:9457]
2025-10-09T21:48:35.632942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342222251614908:9457]
2025-10-09T21:48:35.632945Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342222251614908:9457]
2025-10-09T21:48:35.632990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:35.632994Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:35.632999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.633001Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:35.633002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:35.633005Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:35.633133Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:35.634545Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1259 sessionId: src_id|d7463de9-fdeb0c22-3ffcd7d5-9340f715_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:35.636989Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:35.698702Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342222613438110:5175]
2025-10-09T21:48:35.698941Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 1258, pipe: [1:7559342222613438110:5175]
2025-10-09T21:48:35.783630Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:35.783957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.783976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.784020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613438110:5175], now have 1 active actors on pipe
2025-10-09T21:48:35.784251Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:35.784258Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342222613438110:5175]
2025-10-09T21:48:35.784283Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:35.784286Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:35.784344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1259 reason: error at write: Disconnected sessionId: src_id|d7463de9-fdeb0c22-3ffcd7d5-9340f715_0
2025-10-09T21:48:35.784479Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:35.784532Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1259 sessionId: src_id|d7463de9-fdeb0c22-3ffcd7d5-9340f715_0 is DEAD
2025-10-09T21:48:35.784701Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7463de9-fdeb0c22-3ffcd7d5-9340f715_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:35.784712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7463de9-fdeb0c22-3ffcd7d5-9340f715_0] Write session will restart in 0.008950s
2025-10-09T21:48:35.784740Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7463de9-fdeb0c22-3ffcd7d5-9340f715_0] Write session: Do CDS request
2025-10-09T21:48:35.784747Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7463de9-fdeb0c22-3ffcd7d5-9340f715_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:35.793520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:35.794544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7463de9-fdeb0c22-3ffcd7d5-9340f715_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:35.794641Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:35.794658Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1260
2025-10-09T21:48:35.794774Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1260 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:35.794804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1260 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:35.794815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1260 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:35.794818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:35.795080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:35.795250Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:35.795260Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:35.795271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:35.795263Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1260 sessionId:  describe result for acl check
2025-10-09T21:48:35.795305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:35.795307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:35.795309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:35.795317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342222613438162:4696] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:35.795319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:35.795352Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342222251615032:9471]
2025-10-09T21:48:35.795820Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342222613438162:4696] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:35.795883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:35.796171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:35.796226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:35.796553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:35.796597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.796601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.796603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.796604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.796605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.796606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.796607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.796608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.796609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.796614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.796624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.796629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.796633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.796637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.796642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.796645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.796649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.796654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.796661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.796669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.796675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.796680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.796685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.796690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.796694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.796699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.796704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.796709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.796713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.796718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.796722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.796727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.796733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.796738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.796742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.796747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.796778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:35.797684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342222613438162:4696] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:35.802659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342222613438162:4696] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:35.802675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342222613438162:4696] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:35.802679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342222613438162:4696] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:35.802689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1260 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:35.802976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:35.804031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:35.804049Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:35.804052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:35.804057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:35.804063Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1259 [2:7559342222251615032:9471]
2025-10-09T21:48:35.804078Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:35.804099Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:35.804102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:35.804106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:35.804108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:35.804126Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:35.804140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:35.804147Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:35.804211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.804219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.804221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.804224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:35.804295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:35.804304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.804318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.804332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.804351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:35.804379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613438198:4696], now have 1 active actors on pipe
2025-10-09T21:48:35.804408Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1259
2025-10-09T21:48:35.804394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613438114:2432], now have 1 active actors on pipe
2025-10-09T21:48:35.804437Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1259
2025-10-09T21:48:35.804403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342222251615027:9471]
2025-10-09T21:48:35.804404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342222251615027:9471]
2025-10-09T21:48:35.804408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342222251615027:9471]
2025-10-09T21:48:35.804420Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:35.804435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:35.804451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:35.804455Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:35.804475Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:35.804478Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:35.804480Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:35.804507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.804519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.804533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342222251615027:9471]
2025-10-09T21:48:35.804541Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:35.804578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:35.804582Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|16117c44-7b957283-49b867a2-8e7dd04b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:35.804590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:35.804597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:35.804601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:35.804612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:35.804626Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:35.804657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:35.804834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.804849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.804871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:35.805028Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1260 partition: 0 MaxSeqNo: 25 sessionId: src_id|16117c44-7b957283-49b867a2-8e7dd04b_0
2025-10-09T21:48:35.805393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7463de9-fdeb0c22-3ffcd7d5-9340f715_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|16117c44-7b957283-49b867a2-8e7dd04b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:35.807916Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16117c44-7b957283-49b867a2-8e7dd04b_0] Write session: try to update token
2025-10-09T21:48:35.807941Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16117c44-7b957283-49b867a2-8e7dd04b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:35.819140Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:35.819165Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:35.819169Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:35.819171Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:35.819181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342222251615027:9471]
2025-10-09T21:48:35.819185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342222251615027:9471]
2025-10-09T21:48:35.819190Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342222251615027:9471]
2025-10-09T21:48:35.819239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:35.819240Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:35.819247Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:35.819248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.819251Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:35.819251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:35.819396Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:35.821038Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1260 sessionId: src_id|16117c44-7b957283-49b867a2-8e7dd04b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:35.823467Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:35.884533Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342222613438232:5175]
2025-10-09T21:48:35.884803Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 1259, pipe: [1:7559342222613438232:5175]
2025-10-09T21:48:35.974248Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:35.974654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.974675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.974722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613438232:5175], now have 1 active actors on pipe
2025-10-09T21:48:35.974957Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:35.974973Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:35.974973Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342222613438232:5175]
2025-10-09T21:48:35.975002Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:35.975072Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1260 reason: status is not ok: tablet will be restarted right now sessionId: src_id|16117c44-7b957283-49b867a2-8e7dd04b_0
2025-10-09T21:48:35.975239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1260 sessionId: src_id|16117c44-7b957283-49b867a2-8e7dd04b_0 is DEAD
2025-10-09T21:48:35.975391Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16117c44-7b957283-49b867a2-8e7dd04b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:35.975403Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16117c44-7b957283-49b867a2-8e7dd04b_0] Write session will restart in 0.009575s
2025-10-09T21:48:35.975431Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16117c44-7b957283-49b867a2-8e7dd04b_0] Write session: Do CDS request
2025-10-09T21:48:35.975438Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16117c44-7b957283-49b867a2-8e7dd04b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:35.983469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:35.984916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:35.985074Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:35.985083Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:35.985095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:35.985134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:35.985166Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342222251615126:9475]
2025-10-09T21:48:35.985514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:35.985760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:35.985788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:35.985978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:35.986009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.986013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.986014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.986015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.986015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.986016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.986016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.986017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.986018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.986020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.986040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.986043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.986045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.986047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.986049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.986051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.986052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.986056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.986061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.986065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:35.986069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.986072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:35.986075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.986078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:35.986080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.986083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:35.986085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.986088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:35.986090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.986093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:35.986096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.986099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.986101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.986104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.986106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.986110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.986126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:35.986501Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16117c44-7b957283-49b867a2-8e7dd04b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:35.986574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:35.986590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1261
2025-10-09T21:48:35.986741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1261 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:35.986773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1261 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:35.986783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1261 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:35.986793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:35.987180Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1261 sessionId:  describe result for acl check
2025-10-09T21:48:35.987213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:35.987222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:35.987226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:35.987232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342222613438291:4673] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:35.987759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342222613438291:4673] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:35.990394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342222613438291:4673] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:35.992270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:35.992662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342222613438291:4673] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:35.992678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342222613438291:4673] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:35.992682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342222613438291:4673] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:35.992690Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1261 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:35.993512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:35.993527Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:35.993529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:35.993532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:35.993537Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1260 [2:7559342222251615126:9475]
2025-10-09T21:48:35.993546Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:35.993561Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:35.993562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:35.993564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:35.993566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:35.993579Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:35.993583Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:35.993589Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:35.993630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.993636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.993637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.993639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:35.993687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:35.993703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:35.993712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:35.993718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:35.993728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:35.993745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342222251615121:9475]
2025-10-09T21:48:35.993748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342222251615121:9475]
2025-10-09T21:48:35.993750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342222251615121:9475]
2025-10-09T21:48:35.993756Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:35.993768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613438237:2432], now have 1 active actors on pipe
2025-10-09T21:48:35.993768Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:35.993771Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:35.993773Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:35.993778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:35.993787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:35.993789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:35.993793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342222613438322:4673], now have 1 active actors on pipe
2025-10-09T21:48:35.993814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342222251615121:9475]
2025-10-09T21:48:35.993821Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:35.993857Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:35.993851Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1260
2025-10-09T21:48:35.993880Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1260
2025-10-09T21:48:35.994017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.994024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.994058Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|98db07e5-668e0d55-a2bbbe1d-64c8adcb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:35.994065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:35.994068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:35.994069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:35.994076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:35.994088Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:35.994099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:35.994235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:35.994241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:35.994252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:35.994335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1261 partition: 0 MaxSeqNo: 25 sessionId: src_id|98db07e5-668e0d55-a2bbbe1d-64c8adcb_0
2025-10-09T21:48:35.994619Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16117c44-7b957283-49b867a2-8e7dd04b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|98db07e5-668e0d55-a2bbbe1d-64c8adcb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:35.996977Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98db07e5-668e0d55-a2bbbe1d-64c8adcb_0] Write session: try to update token
2025-10-09T21:48:35.996991Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98db07e5-668e0d55-a2bbbe1d-64c8adcb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:36.007570Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:36.007592Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:36.007595Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:36.007598Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:36.007609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342222251615121:9475]
2025-10-09T21:48:36.007615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342222251615121:9475]
2025-10-09T21:48:36.007619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342222251615121:9475]
2025-10-09T21:48:36.007669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:36.007670Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:36.007675Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:36.007677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.007679Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:36.007679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:36.007838Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:36.010375Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1261 sessionId: src_id|98db07e5-668e0d55-a2bbbe1d-64c8adcb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:36.013095Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:36.075004Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342226908405626:5175]
2025-10-09T21:48:36.075327Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 1260, pipe: [1:7559342226908405626:5175]
2025-10-09T21:48:36.173611Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:36.173956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:36.173971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:36.174043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342226908405626:5175], now have 1 active actors on pipe
2025-10-09T21:48:36.174218Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:36.174228Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:36.174240Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342226908405626:5175]
2025-10-09T21:48:36.174263Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:36.174266Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:36.174325Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1261 reason: error at write: Disconnected sessionId: src_id|98db07e5-668e0d55-a2bbbe1d-64c8adcb_0
2025-10-09T21:48:36.174508Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1261 sessionId: src_id|98db07e5-668e0d55-a2bbbe1d-64c8adcb_0 is DEAD
2025-10-09T21:48:36.174693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98db07e5-668e0d55-a2bbbe1d-64c8adcb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:36.174700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98db07e5-668e0d55-a2bbbe1d-64c8adcb_0] Write session will restart in 0.007094s
2025-10-09T21:48:36.174733Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98db07e5-668e0d55-a2bbbe1d-64c8adcb_0] Write session: Do CDS request
2025-10-09T21:48:36.174740Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98db07e5-668e0d55-a2bbbe1d-64c8adcb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:36.182548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98db07e5-668e0d55-a2bbbe1d-64c8adcb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:36.182627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:36.182642Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1262
2025-10-09T21:48:36.182772Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1262 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:36.182817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1262 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:36.182829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1262 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:36.182833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:36.183259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1262 sessionId:  describe result for acl check
2025-10-09T21:48:36.183296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:36.183298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:36.183300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:36.183306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342226908405679:4670] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:36.183859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342226908405679:4670] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:36.184188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:36.185705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:36.185931Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:36.185942Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:36.185953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:36.186011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:36.186073Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342226546582510:9477]
2025-10-09T21:48:36.186156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342226908405679:4670] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:36.186707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:36.186980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:36.187022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:36.187323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:36.187369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.187375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.187376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.187377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.187378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.187379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.187381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.187382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.187383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.187388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.187397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.187401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.187405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.187409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.187413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.187417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.187421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.187424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.187430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.187438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.187444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.187449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.187454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.187459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.187463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.187468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.187473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.187478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.187482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.187493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.187499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.187504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.187512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.187517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.187523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.187532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.187560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:36.192333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342226908405679:4670] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:36.192349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342226908405679:4670] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:36.192353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342226908405679:4670] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:36.192362Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1262 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:36.193604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:36.194837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:36.194849Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:36.194851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:36.194854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:36.194859Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1261 [2:7559342226546582510:9477]
2025-10-09T21:48:36.194870Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:36.194884Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:36.194886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:36.194888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:36.194889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:36.194906Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:36.194912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:36.194921Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:36.194973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.194981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.194983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.194985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:36.195046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:36.195053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.195065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.195083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.195105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:36.195129Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342226546582505:9477]
2025-10-09T21:48:36.195136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342226546582505:9477]
2025-10-09T21:48:36.195138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342226546582505:9477]
2025-10-09T21:48:36.195151Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:36.195164Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:36.195172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342226908405632:2432], now have 1 active actors on pipe
2025-10-09T21:48:36.195176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342226908405721:4670], now have 1 active actors on pipe
2025-10-09T21:48:36.195176Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:36.195179Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:36.195185Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:36.195199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:36.195207Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:36.195189Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1261
2025-10-09T21:48:36.195210Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1261
2025-10-09T21:48:36.195236Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342226546582505:9477]
2025-10-09T21:48:36.195248Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:36.195255Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:36.195262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:36.195267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:36.195317Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c43b5fc6-8b7cb977-c4af75db-b168e339_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:36.195331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:36.195340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:36.195343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:36.195352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:36.195370Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:36.195388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:36.195517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:36.195524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:36.195537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:36.195617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1262 partition: 0 MaxSeqNo: 25 sessionId: src_id|c43b5fc6-8b7cb977-c4af75db-b168e339_0
2025-10-09T21:48:36.195992Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98db07e5-668e0d55-a2bbbe1d-64c8adcb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c43b5fc6-8b7cb977-c4af75db-b168e339_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:36.198422Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c43b5fc6-8b7cb977-c4af75db-b168e339_0] Write session: try to update token
2025-10-09T21:48:36.198440Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c43b5fc6-8b7cb977-c4af75db-b168e339_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:36.209638Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:36.209661Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:36.209664Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:36.209666Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:36.209675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342226546582505:9477]
2025-10-09T21:48:36.209680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342226546582505:9477]
2025-10-09T21:48:36.209684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342226546582505:9477]
2025-10-09T21:48:36.209747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:36.209755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.209756Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:36.209761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:36.209762Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:36.209766Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:36.209907Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:36.211043Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1262 sessionId: src_id|c43b5fc6-8b7cb977-c4af75db-b168e339_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:36.213550Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:36.274176Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342226908405734:5175]
2025-10-09T21:48:36.274428Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 1261, pipe: [1:7559342226908405734:5175]
2025-10-09T21:48:36.361365Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:36.361613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:36.361627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:36.361666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342226908405734:5175], now have 1 active actors on pipe
2025-10-09T21:48:36.361903Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:36.361918Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:36.361924Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:36.361972Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342226908405734:5175]
2025-10-09T21:48:36.362019Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1262 reason: error at write: Disconnected sessionId: src_id|c43b5fc6-8b7cb977-c4af75db-b168e339_0
2025-10-09T21:48:36.362220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1262 sessionId: src_id|c43b5fc6-8b7cb977-c4af75db-b168e339_0 is DEAD
2025-10-09T21:48:36.362393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c43b5fc6-8b7cb977-c4af75db-b168e339_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:36.362404Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c43b5fc6-8b7cb977-c4af75db-b168e339_0] Write session will restart in 0.009436s
2025-10-09T21:48:36.362430Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c43b5fc6-8b7cb977-c4af75db-b168e339_0] Write session: Do CDS request
2025-10-09T21:48:36.362438Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c43b5fc6-8b7cb977-c4af75db-b168e339_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:36.362476Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:36.371552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:36.372534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c43b5fc6-8b7cb977-c4af75db-b168e339_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:36.372602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:36.372619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1263
2025-10-09T21:48:36.372759Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1263 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:36.372795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1263 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:36.372805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1263 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:36.372808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:36.373176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:36.373221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1263 sessionId:  describe result for acl check
2025-10-09T21:48:36.373256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:36.373263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:36.373265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:36.373271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342226908405788:4663] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:36.373347Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:36.373358Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:36.373370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:36.373425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:36.373467Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342226546582599:9479]
2025-10-09T21:48:36.373974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342226908405788:4663] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:36.374098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:36.374428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:36.374475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:36.374741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:36.374774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.374778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.374779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.374781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.374782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.374783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.374784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.374787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.374788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.374793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.374801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.374805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.374808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.374811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.374814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.374817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.374821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.374825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.374830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.374837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.374841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.374844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.374847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.374849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.374852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.374855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.374857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.374860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.374863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.374866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.374868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.374871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.374874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.374876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.374879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.374882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.374899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:36.375756Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342226908405788:4663] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:36.380681Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342226908405788:4663] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:36.380695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342226908405788:4663] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:36.380698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342226908405788:4663] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:36.380713Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1263 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:36.381178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:36.382371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:36.382387Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:36.382389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:36.382394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:36.382400Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1262 [2:7559342226546582599:9479]
2025-10-09T21:48:36.382412Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:36.382428Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:36.382430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:36.382433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:36.382436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:36.382451Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:36.382461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:36.382468Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:36.382520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.382525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.382526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.382527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:36.382577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:36.382584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.382591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.382597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.382609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:36.382628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342226908405737:2432], now have 1 active actors on pipe
2025-10-09T21:48:36.382634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342226908405827:4663], now have 1 active actors on pipe
2025-10-09T21:48:36.382639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342226546582594:9479]
2025-10-09T21:48:36.382641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342226546582594:9479]
2025-10-09T21:48:36.382642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342226546582594:9479]
2025-10-09T21:48:36.382653Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:36.382668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:36.382669Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:36.382672Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:36.382674Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:36.382681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:36.382683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:36.382665Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1262
2025-10-09T21:48:36.382667Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1262
2025-10-09T21:48:36.382707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342226546582594:9479]
2025-10-09T21:48:36.382717Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:36.382718Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:36.382743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:36.382747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:36.382784Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d5c7d776-a97ce520-4fe6d72-2c900a81_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:36.382797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:36.382800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:36.382807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:36.382817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:36.382831Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:36.382857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:36.383039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:36.383053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:36.383072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:36.383185Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1263 partition: 0 MaxSeqNo: 25 sessionId: src_id|d5c7d776-a97ce520-4fe6d72-2c900a81_0
2025-10-09T21:48:36.383530Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c43b5fc6-8b7cb977-c4af75db-b168e339_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d5c7d776-a97ce520-4fe6d72-2c900a81_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:36.385896Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d5c7d776-a97ce520-4fe6d72-2c900a81_0] Write session: try to update token
2025-10-09T21:48:36.385915Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d5c7d776-a97ce520-4fe6d72-2c900a81_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:36.396906Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:36.396931Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:36.396934Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:36.396936Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:36.396946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342226546582594:9479]
2025-10-09T21:48:36.396951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342226546582594:9479]
2025-10-09T21:48:36.396955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342226546582594:9479]
2025-10-09T21:48:36.397002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:36.397011Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:36.397014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.397017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:36.397018Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:36.397024Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:36.397149Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:36.398081Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1263 sessionId: src_id|d5c7d776-a97ce520-4fe6d72-2c900a81_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:36.400372Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:36.462102Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342226908405834:5175]
2025-10-09T21:48:36.462346Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 1262, pipe: [1:7559342226908405834:5175]
2025-10-09T21:48:36.544806Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:36.545068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:36.545086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:36.545132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342226908405834:5175], now have 1 active actors on pipe
2025-10-09T21:48:36.545308Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:36.545363Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:36.545425Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342226908405834:5175]
2025-10-09T21:48:36.545484Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:36.545586Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1263 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d5c7d776-a97ce520-4fe6d72-2c900a81_0
2025-10-09T21:48:36.545769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1263 sessionId: src_id|d5c7d776-a97ce520-4fe6d72-2c900a81_0 is DEAD
2025-10-09T21:48:36.545912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d5c7d776-a97ce520-4fe6d72-2c900a81_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:36.545921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d5c7d776-a97ce520-4fe6d72-2c900a81_0] Write session will restart in 0.005361s
2025-10-09T21:48:36.545941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d5c7d776-a97ce520-4fe6d72-2c900a81_0] Write session: Do CDS request
2025-10-09T21:48:36.545946Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d5c7d776-a97ce520-4fe6d72-2c900a81_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:36.552484Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d5c7d776-a97ce520-4fe6d72-2c900a81_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:36.552531Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:36.552543Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1264
2025-10-09T21:48:36.552699Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1264 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:36.552734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1264 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:36.552749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1264 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:36.552760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:36.553217Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1264 sessionId:  describe result for acl check
2025-10-09T21:48:36.553254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:36.553263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:36.553265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:36.553271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342226908405868:4654] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:36.553839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342226908405868:4654] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:36.555318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:36.556197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342226908405868:4654] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:36.557122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:36.557309Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:36.557325Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:36.557337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:36.557390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:36.557434Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342226546582688:9481]
2025-10-09T21:48:36.558091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:36.558381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:36.558427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:36.558498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342226908405868:4654] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:36.558511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342226908405868:4654] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:36.558514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342226908405868:4654] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:36.558523Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1264 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:36.558770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:36.558835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.558843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.558844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.558846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.558847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.558848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.558849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.558851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.558852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.558857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.558867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.558871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.558875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.558879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.558885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.558888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.558892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.558896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.558903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.558911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.558917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.558923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.558928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.558933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.558937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.558942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.558947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.558952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.558958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.558968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.558973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.558979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.558987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.558992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.558996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.559001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.559028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:36.565753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:36.567063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:36.567079Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:36.567082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:36.567086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:36.567094Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1263 [2:7559342226546582688:9481]
2025-10-09T21:48:36.567108Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:36.567125Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:36.567128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:36.567131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:36.567134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:36.567149Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:36.567162Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:36.567170Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:36.567231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.567233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.567235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.567238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:36.567310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:36.567318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.567332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.567350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.567374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:36.567404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342226908405837:2432], now have 1 active actors on pipe
2025-10-09T21:48:36.567412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342226908405926:4654], now have 1 active actors on pipe
2025-10-09T21:48:36.567421Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342226546582683:9481]
2025-10-09T21:48:36.567428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342226546582683:9481]
2025-10-09T21:48:36.567430Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342226546582683:9481]
2025-10-09T21:48:36.567444Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:36.567460Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:36.567461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:36.567470Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:36.567472Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:36.567476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:36.567480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:36.567456Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1263
2025-10-09T21:48:36.567459Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1263
2025-10-09T21:48:36.567506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342226546582683:9481]
2025-10-09T21:48:36.567516Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:36.567518Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:36.567546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:36.567556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:36.567590Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|38508013-d40ad37b-1d1331d7-a8d890c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:36.567602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:36.567606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:36.567609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:36.567619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:36.567629Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:36.567647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:36.567820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:36.567831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:36.567847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:36.567947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1264 partition: 0 MaxSeqNo: 25 sessionId: src_id|38508013-d40ad37b-1d1331d7-a8d890c_0
2025-10-09T21:48:36.568336Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d5c7d776-a97ce520-4fe6d72-2c900a81_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|38508013-d40ad37b-1d1331d7-a8d890c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:36.571076Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38508013-d40ad37b-1d1331d7-a8d890c_0] Write session: try to update token
2025-10-09T21:48:36.571094Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38508013-d40ad37b-1d1331d7-a8d890c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:36.582621Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:36.582645Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:36.582648Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:36.582653Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:36.582664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342226546582683:9481]
2025-10-09T21:48:36.582668Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342226546582683:9481]
2025-10-09T21:48:36.582673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342226546582683:9481]
2025-10-09T21:48:36.582725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:36.582733Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:36.582736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.582743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:36.582750Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:36.582754Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:36.582892Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:36.584879Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1264 sessionId: src_id|38508013-d40ad37b-1d1331d7-a8d890c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:36.587505Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:36.645369Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342226908405935:5175]
2025-10-09T21:48:36.645629Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 1263, pipe: [1:7559342226908405935:5175]
2025-10-09T21:48:36.749687Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:36.749981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:36.749997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:36.750062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342226908405935:5175], now have 1 active actors on pipe
2025-10-09T21:48:36.750286Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:36.750278Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342226908405935:5175]
2025-10-09T21:48:36.750295Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:36.750314Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:36.750317Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:36.750397Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1264 reason: error at write: Disconnected sessionId: src_id|38508013-d40ad37b-1d1331d7-a8d890c_0
2025-10-09T21:48:36.750576Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1264 sessionId: src_id|38508013-d40ad37b-1d1331d7-a8d890c_0 is DEAD
2025-10-09T21:48:36.750729Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38508013-d40ad37b-1d1331d7-a8d890c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:36.750739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38508013-d40ad37b-1d1331d7-a8d890c_0] Write session will restart in 0.006593s
2025-10-09T21:48:36.750776Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38508013-d40ad37b-1d1331d7-a8d890c_0] Write session: Do CDS request
2025-10-09T21:48:36.750783Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38508013-d40ad37b-1d1331d7-a8d890c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:36.758440Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38508013-d40ad37b-1d1331d7-a8d890c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:36.758521Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:36.758535Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1265
2025-10-09T21:48:36.758687Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1265 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:36.758730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1265 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:36.758740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1265 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:36.758744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:36.759242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1265 sessionId:  describe result for acl check
2025-10-09T21:48:36.759290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:36.759297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:36.759299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:36.759305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342226908405992:4650] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:36.759827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342226908405992:4650] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:36.759912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:36.761550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:36.761746Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:36.761759Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:36.761771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:36.761827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:36.761867Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342226546582782:9485]
2025-10-09T21:48:36.762050Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342226908405992:4650] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:36.762329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:36.762657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:36.762703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:36.763000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:36.763043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.763052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.763053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.763054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.763056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.763057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.763058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.763059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.763060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.763065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.763075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.763079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.763084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.763089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.763093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.763097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.763100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.763104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.763111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.763124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.763134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.763144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.763148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.763154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.763162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.763167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.763177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.763183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.763187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.763196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.763203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.763212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.763221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.763230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.763239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.763249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.763281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:36.768270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342226908405992:4650] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:36.768285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342226908405992:4650] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:36.768289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342226908405992:4650] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:36.768297Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1265 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:36.769577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:36.770651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:36.770666Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:36.770669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:36.770673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:36.770681Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1264 [2:7559342226546582782:9485]
2025-10-09T21:48:36.770696Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:36.770713Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:36.770723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:36.770726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:36.770733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:36.770748Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:36.770758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:36.770769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:36.770822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.770830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.770832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.770834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:36.770898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:36.770909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.770919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.770932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.770955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:36.770981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342226908406033:4650], now have 1 active actors on pipe
2025-10-09T21:48:36.770989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342226908405941:2432], now have 1 active actors on pipe
2025-10-09T21:48:36.770996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342226546582777:9485]
2025-10-09T21:48:36.770998Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342226546582777:9485]
2025-10-09T21:48:36.771000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342226546582777:9485]
2025-10-09T21:48:36.771006Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1264
2025-10-09T21:48:36.771014Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:36.771007Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1264
2025-10-09T21:48:36.771035Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:36.771046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:36.771049Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:36.771051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:36.771052Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:36.771053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:36.771091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:36.771101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:36.771114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342226546582777:9485]
2025-10-09T21:48:36.771126Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:36.771146Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:36.771157Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|749da26b-43b20d9c-27deceeb-eb72ef4e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:36.771164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:36.771168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:36.771170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:36.771181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:36.771191Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:36.771271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:36.771437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:36.771446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:36.771459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:36.771539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1265 partition: 0 MaxSeqNo: 25 sessionId: src_id|749da26b-43b20d9c-27deceeb-eb72ef4e_0
2025-10-09T21:48:36.771844Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38508013-d40ad37b-1d1331d7-a8d890c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|749da26b-43b20d9c-27deceeb-eb72ef4e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:36.774371Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|749da26b-43b20d9c-27deceeb-eb72ef4e_0] Write session: try to update token
2025-10-09T21:48:36.774390Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|749da26b-43b20d9c-27deceeb-eb72ef4e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:36.785290Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:36.785307Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:36.785310Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:36.785311Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:36.785321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342226546582777:9485]
2025-10-09T21:48:36.785325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342226546582777:9485]
2025-10-09T21:48:36.785328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342226546582777:9485]
2025-10-09T21:48:36.785372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:36.785378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.785378Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:36.785381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:36.785382Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:36.785386Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:36.785506Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:36.787505Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1265 sessionId: src_id|749da26b-43b20d9c-27deceeb-eb72ef4e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:36.789979Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:36.850677Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342226908406043:5175]
2025-10-09T21:48:36.850917Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 1264, pipe: [1:7559342226908406043:5175]
2025-10-09T21:48:36.942399Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:36.942944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:36.942964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:36.943010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342226908406043:5175], now have 1 active actors on pipe
2025-10-09T21:48:36.943253Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:36.943266Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:36.943339Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1265 reason: error at write: Disconnected sessionId: src_id|749da26b-43b20d9c-27deceeb-eb72ef4e_0
2025-10-09T21:48:36.943373Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:36.943518Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1265 sessionId: src_id|749da26b-43b20d9c-27deceeb-eb72ef4e_0 is DEAD
2025-10-09T21:48:36.943693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|749da26b-43b20d9c-27deceeb-eb72ef4e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:36.943706Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|749da26b-43b20d9c-27deceeb-eb72ef4e_0] Write session will restart in 0.008528s
2025-10-09T21:48:36.943687Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:36.943746Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|749da26b-43b20d9c-27deceeb-eb72ef4e_0] Write session: Do CDS request
2025-10-09T21:48:36.943753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|749da26b-43b20d9c-27deceeb-eb72ef4e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:36.943797Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342226908406043:5175]
2025-10-09T21:48:36.953503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|749da26b-43b20d9c-27deceeb-eb72ef4e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:36.953592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:36.953609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1266
2025-10-09T21:48:36.953742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1266 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:36.953775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1266 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:36.953785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1266 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:36.953788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:36.953959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:36.954223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1266 sessionId:  describe result for acl check
2025-10-09T21:48:36.954262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:36.954270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:36.954272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:36.954279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342226908406122:4628] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:36.954808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342226908406122:4628] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:36.955381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:36.955557Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:36.955568Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:36.955580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:36.955636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:36.955675Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342226546582912:9499]
2025-10-09T21:48:36.956251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:36.956628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:36.956687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:36.956973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:36.957022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.957029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.957031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.957033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.957033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.957034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.957035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.957035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.957036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.957040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.957046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.957048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.957050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.957052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.957054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.957056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.957058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.957062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.957067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.957072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:36.957078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.957084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:36.957087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.957092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:36.957095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.957100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:36.957104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.957109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:36.957114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.957118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:36.957123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.957128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.957132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.957137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.957142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.957147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.957136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342226908406122:4628] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:36.957176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:36.963288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342226908406122:4628] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:36.963300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342226908406122:4628] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:36.963303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342226908406122:4628] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:36.963310Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1266 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:36.963341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:36.964427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:36.964441Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:36.964443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:36.964447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:36.964453Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1265 [2:7559342226546582912:9499]
2025-10-09T21:48:36.964465Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:36.964481Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:36.964484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:36.964487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:36.964490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:36.964506Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:36.964513Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:36.964520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:36.964569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.964579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.964581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.964583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:36.964648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:36.964656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.964667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:36.964683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:36.964703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:36.964729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342226908406159:4628], now have 1 active actors on pipe
2025-10-09T21:48:36.964738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342226908406069:2432], now have 1 active actors on pipe
2025-10-09T21:48:36.964744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342226546582906:9499]
2025-10-09T21:48:36.964746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342226546582906:9499]
2025-10-09T21:48:36.964748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342226546582906:9499]
2025-10-09T21:48:36.964762Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:36.964774Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:36.964777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:36.964766Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1265
2025-10-09T21:48:36.964767Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1265
2025-10-09T21:48:36.964784Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:36.964786Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:36.964792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:36.964795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:36.964818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342226546582906:9499]
2025-10-09T21:48:36.964821Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:36.964823Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:36.964841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:36.964852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:36.964896Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e5f6cbad-b35e5b1-393fa1bf-b7d28ab8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:36.964909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:36.964913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:36.964916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:36.964925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:36.964935Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:36.964956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:36.965099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:36.965108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:36.965122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:36.965197Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1266 partition: 0 MaxSeqNo: 25 sessionId: src_id|e5f6cbad-b35e5b1-393fa1bf-b7d28ab8_0
2025-10-09T21:48:36.965512Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|749da26b-43b20d9c-27deceeb-eb72ef4e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e5f6cbad-b35e5b1-393fa1bf-b7d28ab8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:36.968113Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5f6cbad-b35e5b1-393fa1bf-b7d28ab8_0] Write session: try to update token
2025-10-09T21:48:36.968130Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5f6cbad-b35e5b1-393fa1bf-b7d28ab8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:36.979274Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:36.979296Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:36.979298Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:36.979300Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:36.979310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342226546582906:9499]
2025-10-09T21:48:36.979315Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342226546582906:9499]
2025-10-09T21:48:36.979320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342226546582906:9499]
2025-10-09T21:48:36.979367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:36.979371Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:36.979377Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:36.979380Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:36.979380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:36.979385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:36.979523Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:36.981051Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1266 sessionId: src_id|e5f6cbad-b35e5b1-393fa1bf-b7d28ab8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:36.983459Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:37.044185Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342231203373463:5175]
2025-10-09T21:48:37.044431Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 1265, pipe: [1:7559342231203373463:5175]
2025-10-09T21:48:37.138377Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:37.138736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:37.138754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:37.138798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342231203373463:5175], now have 1 active actors on pipe
2025-10-09T21:48:37.139006Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:37.139015Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:37.139026Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342231203373463:5175]
2025-10-09T21:48:37.139051Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:37.139053Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:37.139104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1266 reason: error at write: Disconnected sessionId: src_id|e5f6cbad-b35e5b1-393fa1bf-b7d28ab8_0
2025-10-09T21:48:37.139288Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1266 sessionId: src_id|e5f6cbad-b35e5b1-393fa1bf-b7d28ab8_0 is DEAD
2025-10-09T21:48:37.139466Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5f6cbad-b35e5b1-393fa1bf-b7d28ab8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:37.139477Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5f6cbad-b35e5b1-393fa1bf-b7d28ab8_0] Write session will restart in 0.007535s
2025-10-09T21:48:37.139518Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5f6cbad-b35e5b1-393fa1bf-b7d28ab8_0] Write session: Do CDS request
2025-10-09T21:48:37.139526Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5f6cbad-b35e5b1-393fa1bf-b7d28ab8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:37.148517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5f6cbad-b35e5b1-393fa1bf-b7d28ab8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:37.148632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:37.148648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1267
2025-10-09T21:48:37.148780Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1267 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:37.148820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1267 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:37.148830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1267 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:37.148834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:37.148881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:37.149271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1267 sessionId:  describe result for acl check
2025-10-09T21:48:37.149306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:37.149312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:37.149314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:37.149322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342231203373525:4620] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:37.149862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342231203373525:4620] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:37.150637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:37.150813Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:37.150823Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:37.150836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:37.150890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:37.150931Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342230841550299:9501]
2025-10-09T21:48:37.151527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:37.151798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:37.151846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:37.152139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342231203373525:4620] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:37.152180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:37.152238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.152245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.152247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.152248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.152249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.152250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.152251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.152252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.152254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.152259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.152268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.152272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.152277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.152283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.152288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.152296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.152301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.152309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.152316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.152328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.152339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.152349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.152354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.152361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.152369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.152375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.152379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.152386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.152394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.152399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.152404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.152409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.152413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.152423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.152428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.152432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.152462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:37.158392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342231203373525:4620] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:37.158411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342231203373525:4620] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:37.158416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342231203373525:4620] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:37.158425Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1267 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:37.158702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:37.159812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:37.159827Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:37.159830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:37.159834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:37.159840Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1266 [2:7559342230841550299:9501]
2025-10-09T21:48:37.159854Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:37.159870Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:37.159877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:37.159881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:37.159883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:37.159898Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:37.159904Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:37.159912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:37.159971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.159979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.159981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.159984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:37.160048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:37.160056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.160068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.160081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.160099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:37.160125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342231203373564:4620], now have 1 active actors on pipe
2025-10-09T21:48:37.160129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342231203373467:2432], now have 1 active actors on pipe
2025-10-09T21:48:37.160136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342230841550294:9501]
2025-10-09T21:48:37.160139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342230841550294:9501]
2025-10-09T21:48:37.160141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342230841550294:9501]
2025-10-09T21:48:37.160156Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:37.160170Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:37.160180Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:37.160184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:37.160187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:37.160192Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:37.160197Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:37.160222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342230841550294:9501]
2025-10-09T21:48:37.160199Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1266
2025-10-09T21:48:37.160229Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:37.160232Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:37.160253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:37.160260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:37.160290Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fa684fe3-9b58d811-7e069c0e-3270fd10_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:37.160301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:37.160305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:37.160307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:37.160316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:37.160331Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:37.160356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:37.160394Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1266
2025-10-09T21:48:37.160503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:37.160515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:37.160531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:37.160613Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1267 partition: 0 MaxSeqNo: 25 sessionId: src_id|fa684fe3-9b58d811-7e069c0e-3270fd10_0
2025-10-09T21:48:37.160943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5f6cbad-b35e5b1-393fa1bf-b7d28ab8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fa684fe3-9b58d811-7e069c0e-3270fd10_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:37.163381Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fa684fe3-9b58d811-7e069c0e-3270fd10_0] Write session: try to update token
2025-10-09T21:48:37.163399Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fa684fe3-9b58d811-7e069c0e-3270fd10_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:37.175432Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:37.175454Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:37.175456Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:37.175458Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:37.175468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342230841550294:9501]
2025-10-09T21:48:37.175473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342230841550294:9501]
2025-10-09T21:48:37.175477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342230841550294:9501]
2025-10-09T21:48:37.175522Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:37.175528Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:37.175529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:37.175531Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:37.175537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.175540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:37.175679Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:37.176485Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1267 sessionId: src_id|fa684fe3-9b58d811-7e069c0e-3270fd10_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:37.178964Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:37.239313Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342231203373577:5175]
2025-10-09T21:48:37.239580Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 1266, pipe: [1:7559342231203373577:5175]
2025-10-09T21:48:37.332754Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:37.333126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:37.333141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:37.333183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342231203373577:5175], now have 1 active actors on pipe
2025-10-09T21:48:37.333414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:37.333415Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342231203373577:5175]
2025-10-09T21:48:37.333437Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:37.333440Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:37.333485Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:37.333519Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1267 reason: error at write: Disconnected sessionId: src_id|fa684fe3-9b58d811-7e069c0e-3270fd10_0
2025-10-09T21:48:37.333763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1267 sessionId: src_id|fa684fe3-9b58d811-7e069c0e-3270fd10_0 is DEAD
2025-10-09T21:48:37.333937Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa684fe3-9b58d811-7e069c0e-3270fd10_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:37.333952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa684fe3-9b58d811-7e069c0e-3270fd10_0] Write session will restart in 0.006108s
2025-10-09T21:48:37.333990Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa684fe3-9b58d811-7e069c0e-3270fd10_0] Write session: Do CDS request
2025-10-09T21:48:37.333997Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa684fe3-9b58d811-7e069c0e-3270fd10_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:37.341511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fa684fe3-9b58d811-7e069c0e-3270fd10_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:37.341636Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:37.341651Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1268
2025-10-09T21:48:37.341778Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1268 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:37.341810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1268 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:37.341818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1268 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:37.341822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:37.342457Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1268 sessionId:  describe result for acl check
2025-10-09T21:48:37.342508Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:37.342517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:37.342521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:37.342526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342231203373604:4618] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:37.343108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342231203373604:4618] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:37.344751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:37.345304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342231203373604:4618] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:37.346411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:37.346673Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:37.346684Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:37.346697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:37.346758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:37.346806Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342230841550394:9503]
2025-10-09T21:48:37.347502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:37.347885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:37.347944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:37.347997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342231203373604:4618] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:37.348009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342231203373604:4618] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:37.348014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342231203373604:4618] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:37.348023Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1268 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:37.348297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:37.348346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.348351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.348352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.348352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.348353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.348354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.348354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.348355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.348355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.348359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.348365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.348367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.348370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.348372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.348374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.348376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.348378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.348380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.348385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.348390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.348395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.348398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.348400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.348405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.348407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.348410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.348413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.348416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.348419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.348422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.348424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.348427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.348430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.348433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.348435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.348438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.348455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:37.355204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:37.357246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:37.357260Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:37.357262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:37.357265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:37.357270Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1267 [2:7559342230841550394:9503]
2025-10-09T21:48:37.357280Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:37.357294Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:37.357296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:37.357298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:37.357299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:37.357311Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:37.357315Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:37.357321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:37.357369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.357377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.357379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.357382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:37.357436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:37.357443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.357454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.357465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.357484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:37.357507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342231203373580:2432], now have 1 active actors on pipe
2025-10-09T21:48:37.357515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342231203373673:4618], now have 1 active actors on pipe
2025-10-09T21:48:37.357523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342230841550388:9503]
2025-10-09T21:48:37.357525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342230841550388:9503]
2025-10-09T21:48:37.357528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342230841550388:9503]
2025-10-09T21:48:37.357541Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:37.357558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:37.357572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:37.357575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:37.357576Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:37.357578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:37.357579Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:37.357608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342230841550388:9503]
2025-10-09T21:48:37.357616Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:37.357601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1267
2025-10-09T21:48:37.357612Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1267
2025-10-09T21:48:37.357632Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:37.357662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:37.357673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:37.357713Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|791ca1a9-8fb221a6-86fafdf-c2ae2ee5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:37.357725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:37.357729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:37.357732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:37.357741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:37.357757Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:37.357774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:37.357917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:37.357925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:37.357935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:37.358018Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1268 partition: 0 MaxSeqNo: 25 sessionId: src_id|791ca1a9-8fb221a6-86fafdf-c2ae2ee5_0
2025-10-09T21:48:37.358322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa684fe3-9b58d811-7e069c0e-3270fd10_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|791ca1a9-8fb221a6-86fafdf-c2ae2ee5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:37.362888Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|791ca1a9-8fb221a6-86fafdf-c2ae2ee5_0] Write session: try to update token
2025-10-09T21:48:37.362909Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|791ca1a9-8fb221a6-86fafdf-c2ae2ee5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:37.371909Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:37.371928Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:37.371931Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:37.371932Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:37.371941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342230841550388:9503]
2025-10-09T21:48:37.371946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342230841550388:9503]
2025-10-09T21:48:37.371951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342230841550388:9503]
2025-10-09T21:48:37.371994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:37.371998Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:37.372007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.372010Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:37.372010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:37.372014Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:37.372162Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:37.376907Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1268 sessionId: src_id|791ca1a9-8fb221a6-86fafdf-c2ae2ee5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:37.379250Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:37.433351Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342231203373682:5175]
2025-10-09T21:48:37.433632Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 1267, pipe: [1:7559342231203373682:5175]
2025-10-09T21:48:37.528849Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:37.529211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:37.529228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:37.529274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342231203373682:5175], now have 1 active actors on pipe
2025-10-09T21:48:37.529499Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:37.529520Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342231203373682:5175]
2025-10-09T21:48:37.529532Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:37.529562Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:37.529639Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1268 reason: status is not ok: tablet will be restarted right now sessionId: src_id|791ca1a9-8fb221a6-86fafdf-c2ae2ee5_0
2025-10-09T21:48:37.529864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1268 sessionId: src_id|791ca1a9-8fb221a6-86fafdf-c2ae2ee5_0 is DEAD
2025-10-09T21:48:37.530042Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|791ca1a9-8fb221a6-86fafdf-c2ae2ee5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:37.530055Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|791ca1a9-8fb221a6-86fafdf-c2ae2ee5_0] Write session will restart in 0.009314s
2025-10-09T21:48:37.530098Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|791ca1a9-8fb221a6-86fafdf-c2ae2ee5_0] Write session: Do CDS request
2025-10-09T21:48:37.530104Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|791ca1a9-8fb221a6-86fafdf-c2ae2ee5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:37.539236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:37.540537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|791ca1a9-8fb221a6-86fafdf-c2ae2ee5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:37.540585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:37.540597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1269
2025-10-09T21:48:37.540758Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1269 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:37.540788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1269 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:37.540795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1269 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:37.540804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:37.540878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:37.541079Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:37.541090Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:37.541101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:37.541149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:37.541150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1269 sessionId:  describe result for acl check
2025-10-09T21:48:37.541184Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342230841550487:9505]
2025-10-09T21:48:37.541182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:37.541188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:37.541190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:37.541196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342231203373742:4611] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:37.541734Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342231203373742:4611] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:37.541751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:37.542051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:37.542096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:37.542313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:37.542344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.542348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.542349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.542350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.542350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.542351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.542351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.542352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.542353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.542355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.542361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.542363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.542365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.542368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.542370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.542372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.542374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.542375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.542380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.542384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.542389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.542392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.542396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.542399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.542401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.542404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.542407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.542410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.542412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.542415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.542418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.542421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.542423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.542426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.542430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.542433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.542449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:37.543687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342231203373742:4611] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:37.548432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342231203373742:4611] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:37.548446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342231203373742:4611] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:37.548450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342231203373742:4611] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:37.548459Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1269 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:37.548829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:37.549913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:37.549929Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:37.549932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:37.549936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:37.549942Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1268 [2:7559342230841550487:9505]
2025-10-09T21:48:37.549955Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:37.549971Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:37.549974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:37.549977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:37.549979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:37.549995Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:37.550006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:37.550013Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:37.550081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.550088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.550090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.550093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:37.550229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:37.550240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.550253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.550266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.550288Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1268
2025-10-09T21:48:37.550297Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1268
2025-10-09T21:48:37.550287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:37.550313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342231203373781:4611], now have 1 active actors on pipe
2025-10-09T21:48:37.550317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342231203373685:2432], now have 1 active actors on pipe
2025-10-09T21:48:37.550324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342230841550482:9505]
2025-10-09T21:48:37.550327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342230841550482:9505]
2025-10-09T21:48:37.550329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342230841550482:9505]
2025-10-09T21:48:37.550338Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:37.550349Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:37.550352Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:37.550352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:37.550354Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:37.550370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:37.550373Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:37.550394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:37.550399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:37.550432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342230841550482:9505]
2025-10-09T21:48:37.550448Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:37.550464Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:37.550473Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ca55b28d-872c64b8-6e1701ad-628b44d9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:37.550481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:37.550485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:37.550488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:37.550499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:37.550514Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:37.550559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:37.550740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:37.550747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:37.550762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:37.550869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1269 partition: 0 MaxSeqNo: 25 sessionId: src_id|ca55b28d-872c64b8-6e1701ad-628b44d9_0
2025-10-09T21:48:37.551202Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|791ca1a9-8fb221a6-86fafdf-c2ae2ee5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ca55b28d-872c64b8-6e1701ad-628b44d9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:37.553580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca55b28d-872c64b8-6e1701ad-628b44d9_0] Write session: try to update token
2025-10-09T21:48:37.553596Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca55b28d-872c64b8-6e1701ad-628b44d9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:37.564791Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:37.564815Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:37.564818Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:37.564820Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:37.564829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342230841550482:9505]
2025-10-09T21:48:37.564837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342230841550482:9505]
2025-10-09T21:48:37.564842Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342230841550482:9505]
2025-10-09T21:48:37.564882Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:37.564884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:37.564886Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:37.564889Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:37.564891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.564893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:37.565029Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:37.568110Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1269 sessionId: src_id|ca55b28d-872c64b8-6e1701ad-628b44d9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:37.570749Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:37.629444Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342231203373788:5175]
2025-10-09T21:48:37.629752Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 1268, pipe: [1:7559342231203373788:5175]
2025-10-09T21:48:37.710687Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:37.710917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:37.710932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:37.710970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342231203373788:5175], now have 1 active actors on pipe
2025-10-09T21:48:37.711145Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:37.711252Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342231203373788:5175]
2025-10-09T21:48:37.711276Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:37.711300Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:37.711357Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1269 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ca55b28d-872c64b8-6e1701ad-628b44d9_0
2025-10-09T21:48:37.711528Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1269 sessionId: src_id|ca55b28d-872c64b8-6e1701ad-628b44d9_0 is DEAD
2025-10-09T21:48:37.711682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca55b28d-872c64b8-6e1701ad-628b44d9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:37.711692Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca55b28d-872c64b8-6e1701ad-628b44d9_0] Write session will restart in 0.008865s
2025-10-09T21:48:37.711720Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca55b28d-872c64b8-6e1701ad-628b44d9_0] Write session: Do CDS request
2025-10-09T21:48:37.711726Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca55b28d-872c64b8-6e1701ad-628b44d9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:37.720825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:37.721550Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca55b28d-872c64b8-6e1701ad-628b44d9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:37.721643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:37.721659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1270
2025-10-09T21:48:37.721828Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1270 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:37.721856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1270 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:37.721862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1270 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:37.721870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:37.722367Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1270 sessionId:  describe result for acl check
2025-10-09T21:48:37.722385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:37.722408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:37.722415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:37.722417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:37.722433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342231203373849:4605] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:37.722585Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:37.722599Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:37.722612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:37.722674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:37.722712Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342230841550586:9509]
2025-10-09T21:48:37.723006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342231203373849:4605] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:37.723315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:37.723555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:37.723599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:37.723910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:37.723964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.723971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.723972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.723973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.723974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.723976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.723977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.723978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.723979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.723984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.723994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.723999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.724003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.724007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.724012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.724016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.724020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.724024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.724032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.724045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.724056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.724062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.724067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.724072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.724082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.724088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.724093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.724103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.724107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.724113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.724119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.724129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.724134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.724143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.724150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.724160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.724189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:37.725016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342231203373849:4605] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:37.730417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342231203373849:4605] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:37.730433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342231203373849:4605] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:37.730437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342231203373849:4605] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:37.730446Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1270 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:37.730725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:37.731724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:37.731742Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:37.731744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:37.731749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:37.731755Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1269 [2:7559342230841550586:9509]
2025-10-09T21:48:37.731791Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:37.731816Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:37.731819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:37.731823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:37.731825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:37.731843Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:37.731850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:37.731864Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:37.731919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.731928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.731930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.731933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:37.731998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:37.732005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.732018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.732030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.732051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:37.732077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342231203373888:4605], now have 1 active actors on pipe
2025-10-09T21:48:37.732086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342231203373791:2432], now have 1 active actors on pipe
2025-10-09T21:48:37.732093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342230841550581:9509]
2025-10-09T21:48:37.732100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342230841550581:9509]
2025-10-09T21:48:37.732103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342230841550581:9509]
2025-10-09T21:48:37.732118Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:37.732134Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:37.732136Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:37.732146Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:37.732149Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:37.732134Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1269
2025-10-09T21:48:37.732152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:37.732154Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1269
2025-10-09T21:48:37.732155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:37.732184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342230841550581:9509]
2025-10-09T21:48:37.732187Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:37.732189Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:37.732210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:37.732220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:37.732255Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|39fc8184-3bf72cee-e8c1e31a-b7ddb538_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:37.732272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:37.732281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:37.732283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:37.732293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:37.732306Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:37.732330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:37.732479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:37.732487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:37.732500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:37.732619Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1270 partition: 0 MaxSeqNo: 25 sessionId: src_id|39fc8184-3bf72cee-e8c1e31a-b7ddb538_0
2025-10-09T21:48:37.732988Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca55b28d-872c64b8-6e1701ad-628b44d9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|39fc8184-3bf72cee-e8c1e31a-b7ddb538_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:37.735429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|39fc8184-3bf72cee-e8c1e31a-b7ddb538_0] Write session: try to update token
2025-10-09T21:48:37.735449Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|39fc8184-3bf72cee-e8c1e31a-b7ddb538_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:37.747455Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:37.747476Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:37.747479Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:37.747481Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:37.747491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342230841550581:9509]
2025-10-09T21:48:37.747498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342230841550581:9509]
2025-10-09T21:48:37.747501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342230841550581:9509]
2025-10-09T21:48:37.747547Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:37.747548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:37.747553Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:37.747555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.747556Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:37.747557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:37.747689Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:37.748696Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1270 sessionId: src_id|39fc8184-3bf72cee-e8c1e31a-b7ddb538_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:37.751073Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:37.811197Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342231203373895:5175]
2025-10-09T21:48:37.811467Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 1269, pipe: [1:7559342231203373895:5175]
2025-10-09T21:48:37.890496Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:37.890699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:37.890711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:37.890746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342231203373895:5175], now have 1 active actors on pipe
2025-10-09T21:48:37.890940Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:37.890941Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:37.890955Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:37.890978Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342231203373895:5175]
2025-10-09T21:48:37.890900Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:37.891017Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1270 reason: error at write: Disconnected sessionId: src_id|39fc8184-3bf72cee-e8c1e31a-b7ddb538_0
2025-10-09T21:48:37.891209Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1270 sessionId: src_id|39fc8184-3bf72cee-e8c1e31a-b7ddb538_0 is DEAD
2025-10-09T21:48:37.891351Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39fc8184-3bf72cee-e8c1e31a-b7ddb538_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:37.891362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39fc8184-3bf72cee-e8c1e31a-b7ddb538_0] Write session will restart in 0.007185s
2025-10-09T21:48:37.891389Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39fc8184-3bf72cee-e8c1e31a-b7ddb538_0] Write session: Do CDS request
2025-10-09T21:48:37.891396Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39fc8184-3bf72cee-e8c1e31a-b7ddb538_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:37.899514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|39fc8184-3bf72cee-e8c1e31a-b7ddb538_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:37.899591Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:37.899607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1271
2025-10-09T21:48:37.899688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:37.899739Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1271 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:37.899763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1271 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:37.899768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1271 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:37.899771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:37.900113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1271 sessionId:  describe result for acl check
2025-10-09T21:48:37.900138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:37.900142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:37.900143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:37.900147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342231203373961:4598] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:37.900563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342231203373961:4598] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:37.901485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:37.901678Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:37.901687Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:37.901707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:37.901761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:37.901796Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342230841550712:9521]
2025-10-09T21:48:37.902377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:37.902420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342231203373961:4598] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:37.902641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:37.902680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:37.902983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:37.903031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.903038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.903039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.903040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.903041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.903042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.903043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.903044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.903045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.903050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.903060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.903064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.903068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.903071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.903075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.903078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.903082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.903085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.903092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.903099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:37.903105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.903110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:37.903114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.903119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:37.903123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.903129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:37.903133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.903138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:37.903142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.903147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:37.903152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.903157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.903161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.903166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.903171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.903176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.903203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:37.908726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342231203373961:4598] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:37.908742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342231203373961:4598] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:37.908746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342231203373961:4598] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:37.908755Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1271 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:37.910192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:37.911371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:37.911386Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:37.911388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:37.911393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:37.911399Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1270 [2:7559342230841550712:9521]
2025-10-09T21:48:37.911411Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:37.911430Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:37.911438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:37.911440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:37.911443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:37.911457Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:37.911467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:37.911478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:37.911531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.911537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.911539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.911542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:37.911604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:37.911614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.911625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:37.911637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:37.911655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:37.911680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342231203373997:4598], now have 1 active actors on pipe
2025-10-09T21:48:37.911684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342231203373902:2432], now have 1 active actors on pipe
2025-10-09T21:48:37.911691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342230841550707:9521]
2025-10-09T21:48:37.911696Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342230841550707:9521]
2025-10-09T21:48:37.911699Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342230841550707:9521]
2025-10-09T21:48:37.911713Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:37.911725Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:37.911711Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1270
2025-10-09T21:48:37.911723Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1270
2025-10-09T21:48:37.911728Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:37.911730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:37.911731Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:37.911746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:37.911750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:37.911776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342230841550707:9521]
2025-10-09T21:48:37.911785Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:37.911789Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:37.911804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:37.911812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:37.911858Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1b428737-3e827a4b-3c134dfb-bdcc60d9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:37.911873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:37.911876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:37.911878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:37.911888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:37.911898Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:37.911919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:37.912077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:37.912084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:37.912098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:37.912162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1271 partition: 0 MaxSeqNo: 25 sessionId: src_id|1b428737-3e827a4b-3c134dfb-bdcc60d9_0
2025-10-09T21:48:37.912496Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39fc8184-3bf72cee-e8c1e31a-b7ddb538_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1b428737-3e827a4b-3c134dfb-bdcc60d9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:37.914858Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b428737-3e827a4b-3c134dfb-bdcc60d9_0] Write session: try to update token
2025-10-09T21:48:37.914875Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b428737-3e827a4b-3c134dfb-bdcc60d9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:37.926044Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:37.926064Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:37.926066Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:37.926068Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:37.926078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342230841550707:9521]
2025-10-09T21:48:37.926083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342230841550707:9521]
2025-10-09T21:48:37.926086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342230841550707:9521]
2025-10-09T21:48:37.926130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:37.926142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:37.926148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:37.926151Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:37.926160Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:37.926166Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:37.926281Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:37.927324Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1271 sessionId: src_id|1b428737-3e827a4b-3c134dfb-bdcc60d9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:37.929545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:37.990907Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342231203374026:5175]
2025-10-09T21:48:37.991163Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 1270, pipe: [1:7559342231203374026:5175]
2025-10-09T21:48:38.077166Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:38.077486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:38.077502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:38.077548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342231203374026:5175], now have 1 active actors on pipe
2025-10-09T21:48:38.077803Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:38.077827Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342231203374026:5175]
2025-10-09T21:48:38.077838Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:38.077842Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:38.077866Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:38.077907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1271 reason: error at write: Disconnected sessionId: src_id|1b428737-3e827a4b-3c134dfb-bdcc60d9_0
2025-10-09T21:48:38.078100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1271 sessionId: src_id|1b428737-3e827a4b-3c134dfb-bdcc60d9_0 is DEAD
2025-10-09T21:48:38.078251Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b428737-3e827a4b-3c134dfb-bdcc60d9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:38.078261Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b428737-3e827a4b-3c134dfb-bdcc60d9_0] Write session will restart in 0.008262s
2025-10-09T21:48:38.078290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b428737-3e827a4b-3c134dfb-bdcc60d9_0] Write session: Do CDS request
2025-10-09T21:48:38.078297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b428737-3e827a4b-3c134dfb-bdcc60d9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:38.087281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:38.087572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b428737-3e827a4b-3c134dfb-bdcc60d9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:38.087635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:38.087651Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1272
2025-10-09T21:48:38.087782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1272 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:38.087815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1272 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:38.087826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1272 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:38.087830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:38.088267Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1272 sessionId:  describe result for acl check
2025-10-09T21:48:38.088303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:38.088308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:38.088310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:38.088317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342235498341387:4581] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:38.088857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342235498341387:4581] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:38.088885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:38.089034Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:38.089041Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:38.089050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:38.089088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:38.089117Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342235136518113:9525]
2025-10-09T21:48:38.089475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:38.089753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:38.089810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:38.090173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:38.090231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.090238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.090239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.090240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.090241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.090242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.090243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.090245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.090246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.090250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.090260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.090264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.090268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.090271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.090275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.090279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.090283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.090288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.090296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.090309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.090320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.090329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.090338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.090342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.090351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.090356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.090365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.090375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.090383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.090389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.090398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.090405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.090413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.090418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.090424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.090429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.090463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:38.091494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342235498341387:4581] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:38.096515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342235498341387:4581] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:38.096532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342235498341387:4581] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:38.096536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342235498341387:4581] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:38.096546Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1272 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:38.096793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:38.097939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:38.097956Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:38.097959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:38.097963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:38.097969Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1271 [2:7559342235136518113:9525]
2025-10-09T21:48:38.097982Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:38.097998Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:38.098007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:38.098010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:38.098013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:38.098047Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:38.098053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:38.098061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:38.098113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.098125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.098127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.098130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:38.098196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:38.098205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.098217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.098231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.098255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:38.098280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342235498341426:4581], now have 1 active actors on pipe
2025-10-09T21:48:38.098290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342235498341327:2432], now have 1 active actors on pipe
2025-10-09T21:48:38.098298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342235136518108:9525]
2025-10-09T21:48:38.098301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342235136518108:9525]
2025-10-09T21:48:38.098303Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342235136518108:9525]
2025-10-09T21:48:38.098315Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:38.098329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:38.098330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:38.098334Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:38.098336Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:38.098348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:38.098352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:38.098341Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1271
2025-10-09T21:48:38.098356Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1271
2025-10-09T21:48:38.098383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342235136518108:9525]
2025-10-09T21:48:38.098391Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:38.098394Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:38.098420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:38.098430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:38.098462Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bde62347-b0a479e-c258870f-8390c795_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:38.098475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:38.098480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:38.098483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:38.098492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:38.098502Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:38.098516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:38.098656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:38.098673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:38.098694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:38.098818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1272 partition: 0 MaxSeqNo: 25 sessionId: src_id|bde62347-b0a479e-c258870f-8390c795_0
2025-10-09T21:48:38.099166Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b428737-3e827a4b-3c134dfb-bdcc60d9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bde62347-b0a479e-c258870f-8390c795_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:38.101520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bde62347-b0a479e-c258870f-8390c795_0] Write session: try to update token
2025-10-09T21:48:38.101537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bde62347-b0a479e-c258870f-8390c795_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:38.113030Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:38.113051Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:38.113053Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:38.113055Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:38.113064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342235136518108:9525]
2025-10-09T21:48:38.113070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342235136518108:9525]
2025-10-09T21:48:38.113074Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342235136518108:9525]
2025-10-09T21:48:38.113123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:38.113126Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:38.113133Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:38.113135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.113137Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:38.113138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:38.113278Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:38.114631Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1272 sessionId: src_id|bde62347-b0a479e-c258870f-8390c795_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:38.116949Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:38.177778Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342235498341437:5175]
2025-10-09T21:48:38.178054Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 1271, pipe: [1:7559342235498341437:5175]
2025-10-09T21:48:38.276768Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:38.276979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:38.276991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:38.277029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342235498341437:5175], now have 1 active actors on pipe
2025-10-09T21:48:38.277212Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:38.277289Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:38.277301Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342235498341437:5175]
2025-10-09T21:48:38.277303Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:38.277322Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:38.277374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1272 reason: error at write: Disconnected sessionId: src_id|bde62347-b0a479e-c258870f-8390c795_0
2025-10-09T21:48:38.277551Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1272 sessionId: src_id|bde62347-b0a479e-c258870f-8390c795_0 is DEAD
2025-10-09T21:48:38.277730Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bde62347-b0a479e-c258870f-8390c795_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:38.277738Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bde62347-b0a479e-c258870f-8390c795_0] Write session will restart in 0.007126s
2025-10-09T21:48:38.277775Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bde62347-b0a479e-c258870f-8390c795_0] Write session: Do CDS request
2025-10-09T21:48:38.277782Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bde62347-b0a479e-c258870f-8390c795_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:38.286581Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bde62347-b0a479e-c258870f-8390c795_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:38.286747Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:38.286763Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1273
2025-10-09T21:48:38.286925Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1273 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:38.286976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1273 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:38.286987Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1273 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:38.286992Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:38.287045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:38.287430Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1273 sessionId:  describe result for acl check
2025-10-09T21:48:38.287468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:38.287476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:38.287478Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:38.287484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342235498341505:4575] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:38.288026Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342235498341505:4575] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:38.288633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:38.288809Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:38.288817Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:38.288830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:38.288901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:38.288942Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342235136518211:9527]
2025-10-09T21:48:38.289344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:38.289627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:38.289672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:38.289997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:38.290072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.290079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.290080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.290081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.290082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.290083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.290085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.290086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.290087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.290093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.290103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.290106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.290108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.290110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.290112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.290114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.290116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.290118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.290123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.290129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.290135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.290140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.290145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.290151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.290155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.290160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.290165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.290170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.290174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.290180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.290186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.290196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.290201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.290206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.290213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.290218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.290247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:38.290268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342235498341505:4575] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:38.296330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342235498341505:4575] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:38.296347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342235498341505:4575] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:38.296351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342235498341505:4575] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:38.296361Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1273 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:38.296646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:38.297866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:38.297883Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:38.297885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:38.297890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:38.297898Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1272 [2:7559342235136518211:9527]
2025-10-09T21:48:38.297913Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:38.297936Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:38.297943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:38.297946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:38.297949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:38.297966Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:38.297977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:38.297989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:38.298067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.298074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.298077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.298080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:38.298150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:38.298158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.298171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.298186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.298206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:38.298235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342235498341544:4575], now have 1 active actors on pipe
2025-10-09T21:48:38.298239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342235498341444:2432], now have 1 active actors on pipe
2025-10-09T21:48:38.298246Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342235136518206:9527]
2025-10-09T21:48:38.298249Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342235136518206:9527]
2025-10-09T21:48:38.298251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342235136518206:9527]
2025-10-09T21:48:38.298300Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1272
2025-10-09T21:48:38.298269Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:38.298293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:38.298309Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:38.298300Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1272
2025-10-09T21:48:38.298312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:38.298319Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:38.298323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:38.298327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:38.298353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342235136518206:9527]
2025-10-09T21:48:38.298356Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:38.298367Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:38.298373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:38.298378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:38.298430Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cbb73f21-da953201-a87b3f5a-b0853bed_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:38.298445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:38.298449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:38.298452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:38.298462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:38.298474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:38.298500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:38.298704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:38.298718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:38.298741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:38.298844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1273 partition: 0 MaxSeqNo: 25 sessionId: src_id|cbb73f21-da953201-a87b3f5a-b0853bed_0
2025-10-09T21:48:38.299238Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bde62347-b0a479e-c258870f-8390c795_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cbb73f21-da953201-a87b3f5a-b0853bed_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:38.301863Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cbb73f21-da953201-a87b3f5a-b0853bed_0] Write session: try to update token
2025-10-09T21:48:38.301883Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cbb73f21-da953201-a87b3f5a-b0853bed_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:38.314261Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:38.314281Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:38.314283Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:38.314286Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:38.314297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342235136518206:9527]
2025-10-09T21:48:38.314302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342235136518206:9527]
2025-10-09T21:48:38.314306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342235136518206:9527]
2025-10-09T21:48:38.314356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:38.314364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.314365Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:38.314367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:38.314371Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:38.314375Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:38.314512Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:38.316371Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1273 sessionId: src_id|cbb73f21-da953201-a87b3f5a-b0853bed_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:38.318909Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:38.377454Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342235498341552:5175]
2025-10-09T21:48:38.377742Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 1272, pipe: [1:7559342235498341552:5175]
2025-10-09T21:48:38.474429Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:38.474730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:38.474746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:38.474783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342235498341552:5175], now have 1 active actors on pipe
2025-10-09T21:48:38.474948Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:38.475023Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:38.475040Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:38.475047Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342235498341552:5175]
2025-10-09T21:48:38.475079Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:38.475102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1273 reason: error at write: Disconnected sessionId: src_id|cbb73f21-da953201-a87b3f5a-b0853bed_0
2025-10-09T21:48:38.475287Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1273 sessionId: src_id|cbb73f21-da953201-a87b3f5a-b0853bed_0 is DEAD
2025-10-09T21:48:38.475436Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cbb73f21-da953201-a87b3f5a-b0853bed_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:38.475446Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cbb73f21-da953201-a87b3f5a-b0853bed_0] Write session will restart in 0.005826s
2025-10-09T21:48:38.475471Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cbb73f21-da953201-a87b3f5a-b0853bed_0] Write session: Do CDS request
2025-10-09T21:48:38.475479Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cbb73f21-da953201-a87b3f5a-b0853bed_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:38.482511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cbb73f21-da953201-a87b3f5a-b0853bed_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:38.482593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:38.482607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1274
2025-10-09T21:48:38.482751Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1274 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:38.482782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1274 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:38.482792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1274 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:38.482796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:38.483142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1274 sessionId:  describe result for acl check
2025-10-09T21:48:38.483167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:38.483168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:38.483169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:38.483173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342235498341612:4572] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:38.483643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342235498341612:4572] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:38.484433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:38.485799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342235498341612:4572] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:38.486113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:38.486306Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:38.486319Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:38.486333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:38.486404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:38.486446Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342235136518311:9529]
2025-10-09T21:48:38.487006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:38.487289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:38.487323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:38.487568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:38.487626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.487632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.487633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.487634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.487636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.487637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.487638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.487639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.487640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.487645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.487653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.487655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.487657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.487659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.487661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.487663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.487665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.487667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.487673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.487678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.487683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.487686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.487688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.487691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.487694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.487697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.487699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.487702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.487705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.487708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.487711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.487714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.487717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.487720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.487723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.487727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.487744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:38.488078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342235498341612:4572] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:38.488094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342235498341612:4572] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:38.488108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342235498341612:4572] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:38.488119Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1274 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:38.493789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:38.494876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:38.494893Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:38.494895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:38.494899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:38.494904Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1273 [2:7559342235136518311:9529]
2025-10-09T21:48:38.494916Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:38.494932Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:38.494935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:38.494938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:38.494940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:38.494967Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:38.494978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:38.494985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:38.495041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.495048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.495050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.495053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:38.495116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:38.495126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.495138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.495153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.495174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:38.495199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342235498341655:4572], now have 1 active actors on pipe
2025-10-09T21:48:38.495207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342235498341554:2432], now have 1 active actors on pipe
2025-10-09T21:48:38.495214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342235136518305:9529]
2025-10-09T21:48:38.495218Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342235136518305:9529]
2025-10-09T21:48:38.495220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342235136518305:9529]
2025-10-09T21:48:38.495237Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:38.495254Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:38.495262Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:38.495265Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:38.495267Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:38.495273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:38.495276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:38.495258Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1273
2025-10-09T21:48:38.495262Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1273
2025-10-09T21:48:38.495297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342235136518305:9529]
2025-10-09T21:48:38.495304Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:38.495307Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:38.495326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:38.495337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:38.495379Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e029ff96-6e4e6a2e-e1cfde45-ce424d65_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:38.495394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:38.495397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:38.495400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:38.495410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:38.495421Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:38.495441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:38.495630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:38.495646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:38.495666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:38.495785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1274 partition: 0 MaxSeqNo: 25 sessionId: src_id|e029ff96-6e4e6a2e-e1cfde45-ce424d65_0
2025-10-09T21:48:38.496079Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cbb73f21-da953201-a87b3f5a-b0853bed_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e029ff96-6e4e6a2e-e1cfde45-ce424d65_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:38.498445Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e029ff96-6e4e6a2e-e1cfde45-ce424d65_0] Write session: try to update token
2025-10-09T21:48:38.498461Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e029ff96-6e4e6a2e-e1cfde45-ce424d65_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:38.509236Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:38.509255Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:38.509258Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:38.509260Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:38.509270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342235136518305:9529]
2025-10-09T21:48:38.509274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342235136518305:9529]
2025-10-09T21:48:38.509277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342235136518305:9529]
2025-10-09T21:48:38.509322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:38.509321Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:38.509326Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:38.509329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.509330Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:38.509332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:38.509495Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:38.510880Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1274 sessionId: src_id|e029ff96-6e4e6a2e-e1cfde45-ce424d65_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:38.513281Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:38.575104Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342235498341663:5175]
2025-10-09T21:48:38.575452Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 1273, pipe: [1:7559342235498341663:5175]
2025-10-09T21:48:38.660182Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:38.660453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:38.660469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:38.660508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342235498341663:5175], now have 1 active actors on pipe
2025-10-09T21:48:38.660708Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:38.660724Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:38.660730Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:38.660760Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:38.660785Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342235498341663:5175]
2025-10-09T21:48:38.660788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1274 reason: error at write: Disconnected sessionId: src_id|e029ff96-6e4e6a2e-e1cfde45-ce424d65_0
2025-10-09T21:48:38.660968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1274 sessionId: src_id|e029ff96-6e4e6a2e-e1cfde45-ce424d65_0 is DEAD
2025-10-09T21:48:38.661115Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e029ff96-6e4e6a2e-e1cfde45-ce424d65_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:38.661125Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e029ff96-6e4e6a2e-e1cfde45-ce424d65_0] Write session will restart in 0.007070s
2025-10-09T21:48:38.661163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e029ff96-6e4e6a2e-e1cfde45-ce424d65_0] Write session: Do CDS request
2025-10-09T21:48:38.661170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e029ff96-6e4e6a2e-e1cfde45-ce424d65_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:38.669518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e029ff96-6e4e6a2e-e1cfde45-ce424d65_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:38.669567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:38.669582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1275
2025-10-09T21:48:38.669772Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1275 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:38.669782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:38.669819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1275 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:38.669831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1275 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:38.669835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:38.670294Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1275 sessionId:  describe result for acl check
2025-10-09T21:48:38.670326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:38.670333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:38.670335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:38.670341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342235498341729:4559] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:38.670899Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342235498341729:4559] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:38.671765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:38.671968Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:38.671978Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:38.671988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:38.672029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:38.672058Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342235136518414:9532]
2025-10-09T21:48:38.672545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:38.672650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342235498341729:4559] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:38.672852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:38.672907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:38.673241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:38.673292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.673299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.673300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.673301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.673302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.673303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.673304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.673305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.673307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.673312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.673320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.673324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.673327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.673330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.673334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.673337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.673340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.673343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.673350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.673358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.673364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.673369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.673373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.673378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.673382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.673387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.673392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.673397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.673401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.673412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.673417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.673422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.673431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.673436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.673440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.673450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.673481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:38.678524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342235498341729:4559] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:38.678539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342235498341729:4559] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:38.678543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342235498341729:4559] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:38.678553Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1275 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:38.680091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:38.681245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:38.681261Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:38.681264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:38.681268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:38.681275Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1274 [2:7559342235136518414:9532]
2025-10-09T21:48:38.681289Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:38.681302Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:38.681308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:38.681311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:38.681313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:38.681326Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:38.681330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:38.681335Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:38.681391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.681399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.681401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.681404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:38.681475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:38.681483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.681496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.681513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.681536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:38.681562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342235498341768:4559], now have 1 active actors on pipe
2025-10-09T21:48:38.681572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342235498341665:2432], now have 1 active actors on pipe
2025-10-09T21:48:38.681580Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342235136518409:9532]
2025-10-09T21:48:38.681582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342235136518409:9532]
2025-10-09T21:48:38.681584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342235136518409:9532]
2025-10-09T21:48:38.681596Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:38.681611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:38.681616Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:38.681620Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:38.681623Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:38.681626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:38.681629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:38.681616Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1274
2025-10-09T21:48:38.681660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342235136518409:9532]
2025-10-09T21:48:38.681636Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1274
2025-10-09T21:48:38.681668Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:38.681669Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:38.681708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:38.681718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:38.681757Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f214526e-f081bb21-d479e316-64940ed2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:38.681769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:38.681773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:38.681776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:38.681786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:38.681800Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:38.681823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:38.681966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:38.681974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:38.681988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:38.682088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1275 partition: 0 MaxSeqNo: 25 sessionId: src_id|f214526e-f081bb21-d479e316-64940ed2_0
2025-10-09T21:48:38.682437Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e029ff96-6e4e6a2e-e1cfde45-ce424d65_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f214526e-f081bb21-d479e316-64940ed2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:38.684873Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f214526e-f081bb21-d479e316-64940ed2_0] Write session: try to update token
2025-10-09T21:48:38.684891Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f214526e-f081bb21-d479e316-64940ed2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:38.696299Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:38.696320Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:38.696321Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:38.696323Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:38.696331Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342235136518409:9532]
2025-10-09T21:48:38.696337Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342235136518409:9532]
2025-10-09T21:48:38.696340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342235136518409:9532]
2025-10-09T21:48:38.696377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:38.696382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.696384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:38.696394Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:38.696410Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:38.696414Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:38.696510Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:38.697670Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1275 sessionId: src_id|f214526e-f081bb21-d479e316-64940ed2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:38.700109Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:38.760977Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342235498341775:5175]
2025-10-09T21:48:38.761274Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 1274, pipe: [1:7559342235498341775:5175]
2025-10-09T21:48:38.856560Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:38.856831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:38.856844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:38.856883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342235498341775:5175], now have 1 active actors on pipe
2025-10-09T21:48:38.857052Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:38.857104Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:38.857119Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:38.857138Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:38.857193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1275 reason: error at write: Disconnected sessionId: src_id|f214526e-f081bb21-d479e316-64940ed2_0
2025-10-09T21:48:38.857202Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342235498341775:5175]
2025-10-09T21:48:38.857387Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1275 sessionId: src_id|f214526e-f081bb21-d479e316-64940ed2_0 is DEAD
2025-10-09T21:48:38.857544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f214526e-f081bb21-d479e316-64940ed2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:38.857567Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f214526e-f081bb21-d479e316-64940ed2_0] Write session will restart in 0.005522s
2025-10-09T21:48:38.857594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f214526e-f081bb21-d479e316-64940ed2_0] Write session: Do CDS request
2025-10-09T21:48:38.857601Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f214526e-f081bb21-d479e316-64940ed2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:38.864515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f214526e-f081bb21-d479e316-64940ed2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:38.864607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:38.864620Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1276
2025-10-09T21:48:38.864753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1276 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:38.864794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1276 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:38.864804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1276 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:38.864807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:38.865261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1276 sessionId:  describe result for acl check
2025-10-09T21:48:38.865299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:38.865306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:38.865308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:38.865325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342235498341816:4561] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:38.865965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342235498341816:4561] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:38.867466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:38.867861Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342235498341816:4561] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:38.869089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:38.869323Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:38.869335Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:38.869349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:38.869404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:38.869444Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342235136518517:9535]
2025-10-09T21:48:38.870083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:38.870124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342235498341816:4561] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:38.870143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342235498341816:4561] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:38.870147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342235498341816:4561] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:38.870155Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1276 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:38.870395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:38.870453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:38.870827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:38.870881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.870887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.870888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.870889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.870890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.870891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.870893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.870894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.870895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.870900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.870911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.870919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.870924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.870928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.870936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.870940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.870948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.870952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.870963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.870971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:38.870981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.870990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:38.870999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.871007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:38.871017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.871023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:38.871027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.871032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:38.871037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.871042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:38.871048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.871053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.871062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.871067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.871072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.871077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.871107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:38.877136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:38.878202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:38.878215Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:38.878217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:38.878221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:38.878229Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1275 [2:7559342235136518517:9535]
2025-10-09T21:48:38.878242Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:38.878258Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:38.878266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:38.878269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:38.878272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:38.878290Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:38.878301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:38.878312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:38.878364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.878371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.878373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.878375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:38.878503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:38.878511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.878524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:38.878541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:38.878565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:38.878567Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1275
2025-10-09T21:48:38.878592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342235498341780:2432], now have 1 active actors on pipe
2025-10-09T21:48:38.878600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342235498341879:4561], now have 1 active actors on pipe
2025-10-09T21:48:38.878585Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1275
2025-10-09T21:48:38.878608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342235136518511:9535]
2025-10-09T21:48:38.878611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342235136518511:9535]
2025-10-09T21:48:38.878614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342235136518511:9535]
2025-10-09T21:48:38.878622Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:38.878636Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:38.878640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:38.878643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:38.878645Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:38.878655Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:38.878658Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:38.878675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:38.878685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:38.878729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342235136518511:9535]
2025-10-09T21:48:38.878741Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:38.878748Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:38.878762Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8a6b1ccd-509b377d-932efb4a-f3a59bba_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:38.878774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:38.878782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:38.878784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:38.878796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:38.878809Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:38.878863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:38.878991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:38.879000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:38.879014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:38.879083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1276 partition: 0 MaxSeqNo: 25 sessionId: src_id|8a6b1ccd-509b377d-932efb4a-f3a59bba_0
2025-10-09T21:48:38.879421Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f214526e-f081bb21-d479e316-64940ed2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8a6b1ccd-509b377d-932efb4a-f3a59bba_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:38.881774Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a6b1ccd-509b377d-932efb4a-f3a59bba_0] Write session: try to update token
2025-10-09T21:48:38.881793Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a6b1ccd-509b377d-932efb4a-f3a59bba_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:38.892905Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:38.892920Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:38.892921Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:38.892922Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:38.892931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342235136518511:9535]
2025-10-09T21:48:38.892935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342235136518511:9535]
2025-10-09T21:48:38.892941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342235136518511:9535]
2025-10-09T21:48:38.892984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:38.892989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:38.892993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:38.892992Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:38.892998Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:38.893002Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:38.893116Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:38.895425Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1276 sessionId: src_id|8a6b1ccd-509b377d-932efb4a-f3a59bba_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:38.897776Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:38.957551Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342235498341903:5175]
2025-10-09T21:48:38.957852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 1275, pipe: [1:7559342235498341903:5175]
2025-10-09T21:48:39.051698Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:39.051960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:39.051972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:39.052011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342235498341903:5175], now have 1 active actors on pipe
2025-10-09T21:48:39.052265Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:39.052243Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:39.052249Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342235498341903:5175]
2025-10-09T21:48:39.052292Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:39.052295Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:39.052368Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1276 reason: error at write: Disconnected sessionId: src_id|8a6b1ccd-509b377d-932efb4a-f3a59bba_0
2025-10-09T21:48:39.052572Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1276 sessionId: src_id|8a6b1ccd-509b377d-932efb4a-f3a59bba_0 is DEAD
2025-10-09T21:48:39.052693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a6b1ccd-509b377d-932efb4a-f3a59bba_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:39.052705Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a6b1ccd-509b377d-932efb4a-f3a59bba_0] Write session will restart in 0.007288s
2025-10-09T21:48:39.052732Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a6b1ccd-509b377d-932efb4a-f3a59bba_0] Write session: Do CDS request
2025-10-09T21:48:39.052737Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a6b1ccd-509b377d-932efb4a-f3a59bba_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:39.061548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a6b1ccd-509b377d-932efb4a-f3a59bba_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:39.061625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:39.061643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1277
2025-10-09T21:48:39.061778Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1277 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:39.061813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1277 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:39.061820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1277 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:39.061824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:39.062335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1277 sessionId:  describe result for acl check
2025-10-09T21:48:39.062372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:39.062379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:39.062381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:39.062387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342239793309275:4530] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:39.062377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:39.062933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342239793309275:4530] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:39.064284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:39.064476Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:39.064489Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:39.064502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:39.064564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:39.064609Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342239431485953:9549]
2025-10-09T21:48:39.065138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:39.065248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342239793309275:4530] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:39.065422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:39.065465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:39.065759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:39.065802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.065807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.065807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.065808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.065809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.065809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.065810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.065810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.065811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.065815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.065822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.065824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.065826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.065828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.065830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.065832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.065834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.065836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.065842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.065850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.065856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.065861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.065866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.065872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.065876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.065881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.065886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.065891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.065894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.065897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.065900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.065903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.065906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.065909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.065912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.065915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.065930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:39.071239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342239793309275:4530] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:39.071256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342239793309275:4530] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:39.071260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342239793309275:4530] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:39.071270Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1277 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:39.072245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:39.073361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:39.073371Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:39.073374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:39.073378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:39.073384Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1276 [2:7559342239431485953:9549]
2025-10-09T21:48:39.073396Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:39.073415Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:39.073423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:39.073426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:39.073428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:39.073445Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:39.073453Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:39.073461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:39.073515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.073522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.073524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.073527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:39.073591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:39.073599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.073611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.073621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.073637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:39.073662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342239793309211:2432], now have 1 active actors on pipe
2025-10-09T21:48:39.073670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342239793309315:4530], now have 1 active actors on pipe
2025-10-09T21:48:39.073678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342239431485947:9549]
2025-10-09T21:48:39.073699Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342239431485947:9549]
2025-10-09T21:48:39.073702Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342239431485947:9549]
2025-10-09T21:48:39.073717Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:39.073731Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:39.073745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:39.073722Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1276
2025-10-09T21:48:39.073752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:39.073733Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1276
2025-10-09T21:48:39.073753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:39.073758Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:39.073760Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:39.073784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342239431485947:9549]
2025-10-09T21:48:39.073787Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:39.073812Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:39.073851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:39.073862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:39.073891Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7cf33780-faf36eff-5cb5d666-e1634345_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:39.073904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:39.073908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:39.073910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:39.073920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:39.073931Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:39.073947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:39.074097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:39.074105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:39.074118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:39.074200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1277 partition: 0 MaxSeqNo: 25 sessionId: src_id|7cf33780-faf36eff-5cb5d666-e1634345_0
2025-10-09T21:48:39.074524Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a6b1ccd-509b377d-932efb4a-f3a59bba_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7cf33780-faf36eff-5cb5d666-e1634345_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:39.077036Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7cf33780-faf36eff-5cb5d666-e1634345_0] Write session: try to update token
2025-10-09T21:48:39.077053Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7cf33780-faf36eff-5cb5d666-e1634345_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:39.085270Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:39.085289Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:39.085299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342239431485947:9549]
2025-10-09T21:48:39.085343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:39.085354Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:39.085558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:39.085713Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:39.085865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:39.086007Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:39.086205Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:39.086247Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:39.089960Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1277 sessionId: src_id|7cf33780-faf36eff-5cb5d666-e1634345_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:39.092226Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:39.092413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:39.092434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:39.092484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:39.092576Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:39.098234Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:39.098247Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:39.098249Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:39.098250Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:39.098259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342239431485947:9549]
2025-10-09T21:48:39.098262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342239431485947:9549]
2025-10-09T21:48:39.098264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342239431485947:9549]
2025-10-09T21:48:39.098306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:39.098313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.098314Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:39.098316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:39.098320Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:39.098325Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:39.098447Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:39.152636Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342239793309325:5175]
2025-10-09T21:48:39.152962Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 1276, pipe: [1:7559342239793309325:5175]
2025-10-09T21:48:39.257885Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:39.265440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:39.265466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:39.265641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:39.265715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:39.265781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:39.265845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:39.265893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:39.265898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:39.265967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:39.266054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:39.266118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:39.266183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:39.266226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:39.266229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:39.266290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:39.266345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:39.266406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:39.266464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:39.266505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:39.266512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:39.266580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:39.266643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:39.266750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:39.266815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:39.266862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:39.266869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:39.266931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:39.266990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:39.267051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:39.267139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:39.267184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:39.267192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:39.267253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:39.267313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:39.267373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:39.267435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:39.267476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:39.267481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:39.267537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:39.267617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:39.267675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:39.267739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:39.267783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:39.267790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:39.267846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:39.267900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:39.267957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:39.268021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:39.268070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:39.268079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:39.268134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:39.268187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:39.268247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:39.268311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:39.268370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:39.268377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:39.268434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:39.268488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:39.268547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:39.268605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:39.268644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:39.268652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:39.268707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342239793309325:5175], now have 1 active actors on pipe
2025-10-09T21:48:39.268999Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342239793309325:5175]
2025-10-09T21:48:39.269014Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:39.269018Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:39.269033Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:39.269050Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1277 reason: error at write: Disconnected sessionId: src_id|7cf33780-faf36eff-5cb5d666-e1634345_0
2025-10-09T21:48:39.269258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1277 sessionId: src_id|7cf33780-faf36eff-5cb5d666-e1634345_0 is DEAD
2025-10-09T21:48:39.269401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cf33780-faf36eff-5cb5d666-e1634345_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:39.269411Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cf33780-faf36eff-5cb5d666-e1634345_0] Write session will restart in 0.009568s
2025-10-09T21:48:39.269430Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cf33780-faf36eff-5cb5d666-e1634345_0] Write session: Do CDS request
2025-10-09T21:48:39.269435Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cf33780-faf36eff-5cb5d666-e1634345_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:39.269575Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:39.280377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:39.280542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7cf33780-faf36eff-5cb5d666-e1634345_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:39.280610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:39.280623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1278
2025-10-09T21:48:39.280734Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1278 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:39.280766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1278 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:39.280785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1278 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:39.280789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:39.281190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1278 sessionId:  describe result for acl check
2025-10-09T21:48:39.281220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:39.281225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:39.281226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:39.281230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342239793309400:4527] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:39.281666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342239793309400:4527] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:39.281962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:39.282158Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:39.282168Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:39.282179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:39.282234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:39.282275Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342239431486056:9551]
2025-10-09T21:48:39.282849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:39.283117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:39.283171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:39.283318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342239793309400:4527] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:39.283522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:39.283570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.283576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.283577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.283578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.283579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.283580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.283581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.283582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.283584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.283589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.283599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.283603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.283607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.283610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.283614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.283618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.283622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.283626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.283631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.283639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.283649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.283659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.283663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.283672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.283675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.283685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.283690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.283695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.283703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.283708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.283717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.283726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.283730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.283735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.283745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.283750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.283778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:39.288692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342239793309400:4527] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:39.288710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342239793309400:4527] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:39.288714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342239793309400:4527] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:39.288722Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1278 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:39.290063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:39.291079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:39.291089Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:39.291091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:39.291095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:39.291100Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1277 [2:7559342239431486056:9551]
2025-10-09T21:48:39.291112Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:39.291126Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:39.291128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:39.291130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:39.291132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:39.291145Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:39.291150Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:39.291155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:39.291197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.291202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.291204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.291205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:39.291256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:39.291261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.291268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.291275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.291288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:39.291305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342239431486050:9551]
2025-10-09T21:48:39.291310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342239431486050:9551]
2025-10-09T21:48:39.291311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342239431486050:9551]
2025-10-09T21:48:39.291319Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:39.291331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342239793309333:2432], now have 1 active actors on pipe
2025-10-09T21:48:39.291333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342239793309437:4527], now have 1 active actors on pipe
2025-10-09T21:48:39.291340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:39.291340Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:39.291343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:39.291346Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:39.291348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:39.291350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:39.291374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342239431486050:9551]
2025-10-09T21:48:39.291382Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:39.291389Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:39.291392Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1277
2025-10-09T21:48:39.291426Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1277
2025-10-09T21:48:39.291442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:39.291451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:39.291471Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e9cb536d-bf3b68f3-41543eea-ae71b498_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:39.291476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:39.291480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:39.291481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:39.291489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:39.291497Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:39.291511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:39.291636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:39.291644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:39.291658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:39.291749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1278 partition: 0 MaxSeqNo: 25 sessionId: src_id|e9cb536d-bf3b68f3-41543eea-ae71b498_0
2025-10-09T21:48:39.292081Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cf33780-faf36eff-5cb5d666-e1634345_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e9cb536d-bf3b68f3-41543eea-ae71b498_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:39.294496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9cb536d-bf3b68f3-41543eea-ae71b498_0] Write session: try to update token
2025-10-09T21:48:39.294515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9cb536d-bf3b68f3-41543eea-ae71b498_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:39.305281Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:39.305297Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:39.305300Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:39.305302Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:39.305311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342239431486050:9551]
2025-10-09T21:48:39.305316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342239431486050:9551]
2025-10-09T21:48:39.305319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342239431486050:9551]
2025-10-09T21:48:39.305364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:39.305368Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:39.305375Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:39.305376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.305378Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:39.305378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:39.305503Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:39.307285Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1278 sessionId: src_id|e9cb536d-bf3b68f3-41543eea-ae71b498_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:39.310162Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:39.369227Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342239793309446:5175]
2025-10-09T21:48:39.369553Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 1277, pipe: [1:7559342239793309446:5175]
2025-10-09T21:48:39.459592Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:39.459838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:39.459852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:39.459893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342239793309446:5175], now have 1 active actors on pipe
2025-10-09T21:48:39.460079Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:39.460152Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342239793309446:5175]
2025-10-09T21:48:39.460168Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:39.460189Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:39.460266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1278 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e9cb536d-bf3b68f3-41543eea-ae71b498_0
2025-10-09T21:48:39.460456Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1278 sessionId: src_id|e9cb536d-bf3b68f3-41543eea-ae71b498_0 is DEAD
2025-10-09T21:48:39.460608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9cb536d-bf3b68f3-41543eea-ae71b498_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:39.460620Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9cb536d-bf3b68f3-41543eea-ae71b498_0] Write session will restart in 0.006920s
2025-10-09T21:48:39.460644Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9cb536d-bf3b68f3-41543eea-ae71b498_0] Write session: Do CDS request
2025-10-09T21:48:39.460650Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9cb536d-bf3b68f3-41543eea-ae71b498_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:39.468532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9cb536d-bf3b68f3-41543eea-ae71b498_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:39.468574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:39.468600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1279
2025-10-09T21:48:39.468774Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1279 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:39.468816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1279 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:39.468824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1279 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:39.468827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:39.469342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1279 sessionId:  describe result for acl check
2025-10-09T21:48:39.469386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:39.469392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:39.469394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:39.469400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342239793309506:4524] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:39.470155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342239793309506:4524] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:39.471189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:39.472075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342239793309506:4524] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:39.473130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:39.473321Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:39.473330Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:39.473342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:39.473394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:39.473430Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342239431486160:9553]
2025-10-09T21:48:39.473894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:39.474359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:39.474416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:39.474678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:39.474730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.474737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.474738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.474739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.474740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.474741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.474742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.474743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.474744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.474749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.474759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.474763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.474768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.474772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.474777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.474781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.474785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.474790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.474797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.474805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.474788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342239793309506:4524] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:39.474811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.474802Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342239793309506:4524] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:39.474817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.474822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.474828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.474833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.474838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.474806Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342239793309506:4524] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:39.474813Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1279 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:39.474848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.474855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.474859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.474865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.474881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.474889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.474894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.474899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.474904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.474910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.474940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:39.482766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:39.483794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:39.483807Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:39.483809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:39.483811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:39.483819Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1278 [2:7559342239431486160:9553]
2025-10-09T21:48:39.483828Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:39.483842Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:39.483849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:39.483852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:39.483854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:39.483873Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:39.483883Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:39.483895Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:39.483949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.483955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.483956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.483958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:39.484010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:39.484018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.484029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.484039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.484056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:39.484079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342239793309552:4524], now have 1 active actors on pipe
2025-10-09T21:48:39.484087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342239793309448:2432], now have 1 active actors on pipe
2025-10-09T21:48:39.484095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342239431486154:9553]
2025-10-09T21:48:39.484100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342239431486154:9553]
2025-10-09T21:48:39.484102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342239431486154:9553]
2025-10-09T21:48:39.484121Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:39.484145Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:39.484145Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:39.484149Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:39.484152Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:39.484164Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:39.484167Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:39.484145Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1278
2025-10-09T21:48:39.484153Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1278
2025-10-09T21:48:39.484197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342239431486154:9553]
2025-10-09T21:48:39.484209Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:39.484216Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:39.484226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:39.484235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:39.484287Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a6737453-5891243e-45ab150-33106666_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:39.484299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:39.484303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:39.484305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:39.484315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:39.484329Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:39.484352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:39.484485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:39.484492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:39.484506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:39.484585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1279 partition: 0 MaxSeqNo: 25 sessionId: src_id|a6737453-5891243e-45ab150-33106666_0
2025-10-09T21:48:39.484913Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9cb536d-bf3b68f3-41543eea-ae71b498_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a6737453-5891243e-45ab150-33106666_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:39.487200Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6737453-5891243e-45ab150-33106666_0] Write session: try to update token
2025-10-09T21:48:39.487216Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6737453-5891243e-45ab150-33106666_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:39.499946Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1279 sessionId: src_id|a6737453-5891243e-45ab150-33106666_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:39.500418Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:39.500435Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:39.500438Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:39.500441Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:39.500449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342239431486154:9553]
2025-10-09T21:48:39.500452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342239431486154:9553]
2025-10-09T21:48:39.500454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342239431486154:9553]
2025-10-09T21:48:39.500497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:39.500503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.500508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:39.500507Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:39.500514Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:39.500517Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:39.500647Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:39.502165Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:39.560201Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342239793309562:5175]
2025-10-09T21:48:39.560495Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 1278, pipe: [1:7559342239793309562:5175]
2025-10-09T21:48:39.658680Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:39.659003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:39.659020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:39.659069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342239793309562:5175], now have 1 active actors on pipe
2025-10-09T21:48:39.659284Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:39.659322Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:39.659335Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:39.659389Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1279 reason: error at write: Disconnected sessionId: src_id|a6737453-5891243e-45ab150-33106666_0
2025-10-09T21:48:39.659396Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342239793309562:5175]
2025-10-09T21:48:39.659428Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:39.659561Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1279 sessionId: src_id|a6737453-5891243e-45ab150-33106666_0 is DEAD
2025-10-09T21:48:39.659727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6737453-5891243e-45ab150-33106666_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:39.659739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6737453-5891243e-45ab150-33106666_0] Write session will restart in 0.006732s
2025-10-09T21:48:39.659769Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6737453-5891243e-45ab150-33106666_0] Write session: Do CDS request
2025-10-09T21:48:39.659775Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6737453-5891243e-45ab150-33106666_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:39.667500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6737453-5891243e-45ab150-33106666_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:39.667609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:39.667625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1280
2025-10-09T21:48:39.667799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1280 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:39.667844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1280 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:39.667856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1280 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:39.667861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:39.668381Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1280 sessionId:  describe result for acl check
2025-10-09T21:48:39.668423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:39.668429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:39.668430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:39.668436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342239793309630:4519] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:39.669018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342239793309630:4519] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:39.669164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:39.671031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:39.671184Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:39.671193Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:39.671201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:39.671250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:39.671279Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342239431486268:9556]
2025-10-09T21:48:39.671312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342239793309630:4519] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:39.671657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:39.671907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:39.671944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:39.672212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:39.672259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.672267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.672268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.672269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.672271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.672272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.672273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.672274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.672275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.672280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.672289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.672293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.672297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.672301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.672305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.672308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.672312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.672315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.672321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.672330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.672335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.672340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.672344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.672349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.672353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.672358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.672362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.672367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.672371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.672376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.672382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.672387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.672391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.672396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.672402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.672411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.672438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:39.677874Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342239793309630:4519] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:39.677890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342239793309630:4519] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:39.677894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342239793309630:4519] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:39.677901Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1280 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:39.679405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:39.680475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:39.680490Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:39.680492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:39.680497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:39.680503Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1279 [2:7559342239431486268:9556]
2025-10-09T21:48:39.680515Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:39.680530Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:39.680536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:39.680539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:39.680541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:39.680557Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:39.680565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:39.680576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:39.680626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.680632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.680634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.680636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:39.680697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:39.680703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.680714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.680727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.680746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:39.680765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342239793309564:2432], now have 1 active actors on pipe
2025-10-09T21:48:39.680772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342239793309672:4519], now have 1 active actors on pipe
2025-10-09T21:48:39.680778Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342239431486263:9556]
2025-10-09T21:48:39.680780Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342239431486263:9556]
2025-10-09T21:48:39.680782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342239431486263:9556]
2025-10-09T21:48:39.680794Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:39.680808Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:39.680809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:39.680810Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:39.680812Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:39.680822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:39.680824Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:39.680810Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1279
2025-10-09T21:48:39.680848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342239431486263:9556]
2025-10-09T21:48:39.680832Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1279
2025-10-09T21:48:39.680857Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:39.680861Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:39.680874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:39.680882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:39.680923Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4803ce9d-b0b5f9be-668b296f-7106662f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:39.680934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:39.680938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:39.680940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:39.680948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:39.680957Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:39.680976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:39.681120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:39.681128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:39.681142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:39.681212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1280 partition: 0 MaxSeqNo: 25 sessionId: src_id|4803ce9d-b0b5f9be-668b296f-7106662f_0
2025-10-09T21:48:39.681502Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6737453-5891243e-45ab150-33106666_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4803ce9d-b0b5f9be-668b296f-7106662f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:39.683819Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4803ce9d-b0b5f9be-668b296f-7106662f_0] Write session: try to update token
2025-10-09T21:48:39.683837Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4803ce9d-b0b5f9be-668b296f-7106662f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:39.696270Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1280 sessionId: src_id|4803ce9d-b0b5f9be-668b296f-7106662f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:39.698115Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:39.698129Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:39.698130Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:39.698132Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:39.698140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342239431486263:9556]
2025-10-09T21:48:39.698145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342239431486263:9556]
2025-10-09T21:48:39.698149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342239431486263:9556]
2025-10-09T21:48:39.698195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:39.698201Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:39.698205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.698208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:39.698209Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:39.698214Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:39.698331Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:39.698528Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:39.759377Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342239793309679:5175]
2025-10-09T21:48:39.759672Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 1279, pipe: [1:7559342239793309679:5175]
2025-10-09T21:48:39.852880Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:39.853262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:39.853277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:39.853326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342239793309679:5175], now have 1 active actors on pipe
2025-10-09T21:48:39.853541Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:39.853637Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:39.853648Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342239793309679:5175]
2025-10-09T21:48:39.853672Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:39.853739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1280 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4803ce9d-b0b5f9be-668b296f-7106662f_0
2025-10-09T21:48:39.853933Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1280 sessionId: src_id|4803ce9d-b0b5f9be-668b296f-7106662f_0 is DEAD
2025-10-09T21:48:39.854093Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4803ce9d-b0b5f9be-668b296f-7106662f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:39.854103Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4803ce9d-b0b5f9be-668b296f-7106662f_0] Write session will restart in 0.005988s
2025-10-09T21:48:39.854141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4803ce9d-b0b5f9be-668b296f-7106662f_0] Write session: Do CDS request
2025-10-09T21:48:39.854147Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4803ce9d-b0b5f9be-668b296f-7106662f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:39.861527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4803ce9d-b0b5f9be-668b296f-7106662f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:39.861619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:39.861647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1281
2025-10-09T21:48:39.861829Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1281 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:39.861860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1281 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:39.861870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1281 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:39.861875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:39.862319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1281 sessionId:  describe result for acl check
2025-10-09T21:48:39.862360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:39.862362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:39.862365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:39.862370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342239793309708:4513] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:39.862922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342239793309708:4513] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:39.865440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342239793309708:4513] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:39.865716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:39.867407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:39.867599Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:39.867609Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:39.867620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:39.867667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:39.867708Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342239431486375:9559]
2025-10-09T21:48:39.868060Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342239793309708:4513] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:39.868072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342239793309708:4513] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:39.868075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342239793309708:4513] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:39.868083Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1281 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:39.868138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:39.868399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:39.868434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:39.868643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:39.868675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.868679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.868680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.868681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.868681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.868682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.868683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.868683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.868684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.868687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.868693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.868696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.868698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.868700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.868702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.868704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.868706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.868710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.868716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.868724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:39.868730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.868735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:39.868738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.868743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:39.868747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.868752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:39.868756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.868761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:39.868765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.868769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:39.868776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.868782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.868788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.868793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.868798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.868804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.868820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:39.876837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:39.877856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:39.877867Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:39.877869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:39.877874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:39.877880Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1280 [2:7559342239431486375:9559]
2025-10-09T21:48:39.877999Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:39.878019Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:39.878045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:39.878049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:39.878052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:39.878072Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:39.878083Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:39.878090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:39.878148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.878157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.878159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.878161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:39.878232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:39.878242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.878254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:39.878265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:39.878283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:39.878305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342239793309684:2432], now have 1 active actors on pipe
2025-10-09T21:48:39.878312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342239793309788:4513], now have 1 active actors on pipe
2025-10-09T21:48:39.878321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342239431486370:9559]
2025-10-09T21:48:39.878328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342239431486370:9559]
2025-10-09T21:48:39.878331Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342239431486370:9559]
2025-10-09T21:48:39.878344Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:39.878359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:39.878363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:39.878368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:39.878370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:39.878377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:39.878380Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:39.878408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342239431486370:9559]
2025-10-09T21:48:39.878415Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:39.878396Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1280
2025-10-09T21:48:39.878439Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1280
2025-10-09T21:48:39.878430Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:39.878535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:39.878545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:39.878575Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|af172057-d757f97-12f132df-9b89ec9e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:39.878588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:39.878595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:39.878598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:39.878608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:39.878620Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:39.878633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:39.878818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:39.878825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:39.878839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:39.878918Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1281 partition: 0 MaxSeqNo: 25 sessionId: src_id|af172057-d757f97-12f132df-9b89ec9e_0
2025-10-09T21:48:39.879279Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4803ce9d-b0b5f9be-668b296f-7106662f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|af172057-d757f97-12f132df-9b89ec9e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:39.882304Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af172057-d757f97-12f132df-9b89ec9e_0] Write session: try to update token
2025-10-09T21:48:39.882326Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af172057-d757f97-12f132df-9b89ec9e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:39.895888Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:39.895901Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:39.895903Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:39.895905Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:39.895916Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342239431486370:9559]
2025-10-09T21:48:39.895921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342239431486370:9559]
2025-10-09T21:48:39.895924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342239431486370:9559]
2025-10-09T21:48:39.895970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:39.895969Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:39.895977Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:39.895981Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:39.895983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:39.895988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:39.896129Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:39.898698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1281 sessionId: src_id|af172057-d757f97-12f132df-9b89ec9e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:39.901177Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:39.953932Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342239793309799:5175]
2025-10-09T21:48:39.954311Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 1280, pipe: [1:7559342239793309799:5175]
2025-10-09T21:48:40.051839Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:40.052140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:40.052156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:40.052202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342239793309799:5175], now have 1 active actors on pipe
2025-10-09T21:48:40.052411Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:40.052407Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342239793309799:5175]
2025-10-09T21:48:40.052419Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:40.052423Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:40.052433Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:40.052477Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1281 reason: error at write: Disconnected sessionId: src_id|af172057-d757f97-12f132df-9b89ec9e_0
2025-10-09T21:48:40.052638Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1281 sessionId: src_id|af172057-d757f97-12f132df-9b89ec9e_0 is DEAD
2025-10-09T21:48:40.052801Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af172057-d757f97-12f132df-9b89ec9e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:40.052809Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af172057-d757f97-12f132df-9b89ec9e_0] Write session will restart in 0.008067s
2025-10-09T21:48:40.052830Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af172057-d757f97-12f132df-9b89ec9e_0] Write session: Do CDS request
2025-10-09T21:48:40.052835Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af172057-d757f97-12f132df-9b89ec9e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:40.062508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af172057-d757f97-12f132df-9b89ec9e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:40.062510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:40.062606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:40.062619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1282
2025-10-09T21:48:40.062754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1282 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:40.062794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1282 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:40.062802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1282 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:40.062805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:40.063229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1282 sessionId:  describe result for acl check
2025-10-09T21:48:40.063268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:40.063274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:40.063276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:40.063282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342244088277193:4494] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:40.063741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342244088277193:4494] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:40.064599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:40.064804Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:40.064813Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:40.064826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:40.064876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:40.064919Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342243726453817:9573]
2025-10-09T21:48:40.065455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:40.065737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:40.065775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:40.066095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:40.066146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.066151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.066152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.066153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.066154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.066155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.066156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.066157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.066158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.066163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.066172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.066176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.066179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.066182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.066190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.066169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342244088277193:4494] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:40.066193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.066200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.066203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.066209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.066221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.066227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.066231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.066239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.066243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.066247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.066251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.066262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.066266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.066270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.066275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.066279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.066283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.066288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.066293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.066297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.066301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.066326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:40.073583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342244088277193:4494] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:40.073600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342244088277193:4494] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:40.073603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342244088277193:4494] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:40.073611Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1282 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:40.073957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:40.075115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:40.075131Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:40.075134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:40.075138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:40.075144Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1281 [2:7559342243726453817:9573]
2025-10-09T21:48:40.075159Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:40.075181Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:40.075189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:40.075193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:40.075195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:40.075214Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:40.075224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:40.075232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:40.075291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.075298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.075300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.075303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:40.075376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:40.075385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.075399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.075411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.075429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:40.075451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342244088277229:4494], now have 1 active actors on pipe
2025-10-09T21:48:40.075459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342244088277123:2432], now have 1 active actors on pipe
2025-10-09T21:48:40.075466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342243726453811:9573]
2025-10-09T21:48:40.075469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342243726453811:9573]
2025-10-09T21:48:40.075471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342243726453811:9573]
2025-10-09T21:48:40.075483Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:40.075474Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1281
2025-10-09T21:48:40.075476Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1281
2025-10-09T21:48:40.075500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:40.075500Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:40.075503Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:40.075505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:40.075520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:40.075527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:40.075584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:40.075594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:40.075606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342243726453811:9573]
2025-10-09T21:48:40.075616Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:40.075630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:40.075658Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9ad25348-d8728d71-8855663f-3b0695b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:40.075672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:40.075676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:40.075678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:40.075688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:40.075699Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:40.075722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:40.075933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:40.075947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:40.075970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:40.076046Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1282 partition: 0 MaxSeqNo: 25 sessionId: src_id|9ad25348-d8728d71-8855663f-3b0695b_0
2025-10-09T21:48:40.076377Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af172057-d757f97-12f132df-9b89ec9e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9ad25348-d8728d71-8855663f-3b0695b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:40.078944Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ad25348-d8728d71-8855663f-3b0695b_0] Write session: try to update token
2025-10-09T21:48:40.078959Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ad25348-d8728d71-8855663f-3b0695b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:40.091153Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1282 sessionId: src_id|9ad25348-d8728d71-8855663f-3b0695b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:40.092408Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:40.092428Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:40.092432Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:40.092433Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:40.092441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342243726453811:9573]
2025-10-09T21:48:40.092445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342243726453811:9573]
2025-10-09T21:48:40.092448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342243726453811:9573]
2025-10-09T21:48:40.092488Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:40.092493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:40.092494Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:40.092497Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:40.092500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.092503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:40.092630Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:40.093545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:40.152575Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342244088277239:5175]
2025-10-09T21:48:40.152855Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 1281, pipe: [1:7559342244088277239:5175]
2025-10-09T21:48:40.254068Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:40.254376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:40.254393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:40.254439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342244088277239:5175], now have 1 active actors on pipe
2025-10-09T21:48:40.254667Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:40.254759Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342244088277239:5175]
2025-10-09T21:48:40.254795Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:40.254807Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:40.254868Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1282 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9ad25348-d8728d71-8855663f-3b0695b_0
2025-10-09T21:48:40.255065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1282 sessionId: src_id|9ad25348-d8728d71-8855663f-3b0695b_0 is DEAD
2025-10-09T21:48:40.255213Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ad25348-d8728d71-8855663f-3b0695b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:40.255221Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ad25348-d8728d71-8855663f-3b0695b_0] Write session will restart in 0.008435s
2025-10-09T21:48:40.255261Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ad25348-d8728d71-8855663f-3b0695b_0] Write session: Do CDS request
2025-10-09T21:48:40.255268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ad25348-d8728d71-8855663f-3b0695b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:40.264480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ad25348-d8728d71-8855663f-3b0695b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:40.264528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:40.264645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:40.264659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1283
2025-10-09T21:48:40.264819Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1283 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:40.264845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1283 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:40.264857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1283 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:40.264861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:40.265233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1283 sessionId:  describe result for acl check
2025-10-09T21:48:40.265265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:40.265271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:40.265276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:40.265281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342244088277318:4480] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:40.265751Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342244088277318:4480] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:40.266272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:40.266456Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:40.266467Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:40.266478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:40.266533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:40.266574Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342243726453924:9575]
2025-10-09T21:48:40.267039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:40.267257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:40.267287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:40.267574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:40.267592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342244088277318:4480] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:40.267628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.267636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.267637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.267638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.267639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.267640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.267642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.267643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.267644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.267649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.267659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.267663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.267667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.267671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.267675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.267679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.267683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.267687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.267694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.267701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.267708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.267713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.267718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.267723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.267732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.267737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.267746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.267751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.267756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.267760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.267765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.267770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.267774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.267780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.267784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.267789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.267819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:40.273866Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342244088277318:4480] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:40.273881Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342244088277318:4480] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:40.273885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342244088277318:4480] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:40.273894Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1283 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:40.275418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:40.276456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:40.276468Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:40.276469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:40.276472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:40.276477Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1282 [2:7559342243726453924:9575]
2025-10-09T21:48:40.276486Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:40.276503Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:40.276505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:40.276508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:40.276509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:40.276521Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:40.276526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:40.276534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:40.276593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.276601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.276602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.276604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:40.276654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:40.276659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.276667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.276674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.276683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:40.276701Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342243726453919:9575]
2025-10-09T21:48:40.276702Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342243726453919:9575]
2025-10-09T21:48:40.276704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342243726453919:9575]
2025-10-09T21:48:40.276711Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:40.276720Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:40.276723Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:40.276725Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:40.276728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342244088277357:4480], now have 1 active actors on pipe
2025-10-09T21:48:40.276734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:40.276748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:40.276751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:40.276757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342244088277247:2432], now have 1 active actors on pipe
2025-10-09T21:48:40.276785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342243726453919:9575]
2025-10-09T21:48:40.276799Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:40.276833Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:40.276825Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1282
2025-10-09T21:48:40.276824Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1282
2025-10-09T21:48:40.276895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:40.276904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:40.276933Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|69ce56b0-6bf9c89b-faaf32a0-c8ae9377_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:40.276946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:40.276950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:40.276951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:40.276959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:40.276968Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:40.276981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:40.277147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:40.277155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:40.277168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:40.277298Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1283 partition: 0 MaxSeqNo: 25 sessionId: src_id|69ce56b0-6bf9c89b-faaf32a0-c8ae9377_0
2025-10-09T21:48:40.277642Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ad25348-d8728d71-8855663f-3b0695b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|69ce56b0-6bf9c89b-faaf32a0-c8ae9377_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:40.280163Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|69ce56b0-6bf9c89b-faaf32a0-c8ae9377_0] Write session: try to update token
2025-10-09T21:48:40.280182Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|69ce56b0-6bf9c89b-faaf32a0-c8ae9377_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:40.292352Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:40.292372Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:40.292374Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:40.292377Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:40.292387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342243726453919:9575]
2025-10-09T21:48:40.292392Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342243726453919:9575]
2025-10-09T21:48:40.292396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342243726453919:9575]
2025-10-09T21:48:40.292441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:40.292441Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:40.292447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.292447Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:40.292450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:40.292451Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:40.292585Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:40.293658Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1283 sessionId: src_id|69ce56b0-6bf9c89b-faaf32a0-c8ae9377_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:40.295926Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:40.354777Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342244088277365:5175]
2025-10-09T21:48:40.355052Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 1282, pipe: [1:7559342244088277365:5175]
2025-10-09T21:48:40.444789Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:40.445141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:40.445157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:40.445203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342244088277365:5175], now have 1 active actors on pipe
2025-10-09T21:48:40.445415Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:40.445429Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:40.445406Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:40.445446Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342244088277365:5175]
2025-10-09T21:48:40.445466Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:40.445497Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1283 reason: error at write: Disconnected sessionId: src_id|69ce56b0-6bf9c89b-faaf32a0-c8ae9377_0
2025-10-09T21:48:40.445658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1283 sessionId: src_id|69ce56b0-6bf9c89b-faaf32a0-c8ae9377_0 is DEAD
2025-10-09T21:48:40.445808Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69ce56b0-6bf9c89b-faaf32a0-c8ae9377_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:40.445818Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69ce56b0-6bf9c89b-faaf32a0-c8ae9377_0] Write session will restart in 0.007411s
2025-10-09T21:48:40.445839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69ce56b0-6bf9c89b-faaf32a0-c8ae9377_0] Write session: Do CDS request
2025-10-09T21:48:40.445844Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69ce56b0-6bf9c89b-faaf32a0-c8ae9377_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:40.454437Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|69ce56b0-6bf9c89b-faaf32a0-c8ae9377_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:40.454526Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:40.454539Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1284
2025-10-09T21:48:40.454686Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1284 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:40.454728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1284 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:40.454739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1284 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:40.454743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:40.455153Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1284 sessionId:  describe result for acl check
2025-10-09T21:48:40.455190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:40.455196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:40.455198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:40.455203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342244088277438:4470] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:40.455378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:40.455719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342244088277438:4470] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:40.457157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:40.457332Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:40.457342Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:40.457354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:40.457409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:40.457450Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342243726454034:9577]
2025-10-09T21:48:40.458091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:40.458196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342244088277438:4470] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:40.458366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:40.458407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:40.458728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:40.458780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.458786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.458787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.458788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.458789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.458791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.458792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.458793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.458794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.458800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.458810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.458814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.458818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.458822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.458826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.458830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.458834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.458837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.458843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.458852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.458858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.458868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.458873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.458879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.458887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.458892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.458897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.458902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.458905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.458909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.458918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.458923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.458927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.458937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.458941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.458945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.458975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:40.463882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342244088277438:4470] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:40.463894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342244088277438:4470] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:40.463897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342244088277438:4470] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:40.463904Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1284 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:40.465436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:40.466518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:40.466532Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:40.466534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:40.466537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:40.466541Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1283 [2:7559342243726454034:9577]
2025-10-09T21:48:40.466550Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:40.466563Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:40.466565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:40.466567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:40.466568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:40.466580Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:40.466584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:40.466590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:40.466630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.466636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.466637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.466640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:40.466705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:40.466711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.466719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.466725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.466734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:40.466754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342244088277367:2432], now have 1 active actors on pipe
2025-10-09T21:48:40.466759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342244088277478:4470], now have 1 active actors on pipe
2025-10-09T21:48:40.466765Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342243726454028:9577]
2025-10-09T21:48:40.466767Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342243726454028:9577]
2025-10-09T21:48:40.466768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342243726454028:9577]
2025-10-09T21:48:40.466778Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:40.466791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:40.466792Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:40.466794Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:40.466796Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:40.466800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:40.466802Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:40.466821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342243726454028:9577]
2025-10-09T21:48:40.466829Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:40.466835Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:40.466838Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1283
2025-10-09T21:48:40.466839Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1283
2025-10-09T21:48:40.466954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:40.466964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:40.466984Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1a885fe-4f18dbb5-51de5c91-fdec1c24_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:40.466996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:40.466998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:40.467000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:40.467008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:40.467016Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:40.467029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:40.467186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:40.467195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:40.467213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:40.467293Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1284 partition: 0 MaxSeqNo: 25 sessionId: src_id|1a885fe-4f18dbb5-51de5c91-fdec1c24_0
2025-10-09T21:48:40.467572Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69ce56b0-6bf9c89b-faaf32a0-c8ae9377_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1a885fe-4f18dbb5-51de5c91-fdec1c24_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:40.470011Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a885fe-4f18dbb5-51de5c91-fdec1c24_0] Write session: try to update token
2025-10-09T21:48:40.470044Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a885fe-4f18dbb5-51de5c91-fdec1c24_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:40.481604Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:40.481620Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:40.481623Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:40.481626Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:40.481636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342243726454028:9577]
2025-10-09T21:48:40.481640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342243726454028:9577]
2025-10-09T21:48:40.481644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342243726454028:9577]
2025-10-09T21:48:40.481702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:40.481707Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:40.481709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.481711Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:40.481712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:40.481714Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:40.481844Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:40.482998Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1284 sessionId: src_id|1a885fe-4f18dbb5-51de5c91-fdec1c24_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:40.485279Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:40.545757Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342244088277486:5175]
2025-10-09T21:48:40.546081Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 1283, pipe: [1:7559342244088277486:5175]
2025-10-09T21:48:40.644617Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:40.644949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:40.644967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:40.645013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342244088277486:5175], now have 1 active actors on pipe
2025-10-09T21:48:40.645232Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:40.645251Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:40.645245Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342244088277486:5175]
2025-10-09T21:48:40.645265Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:40.645278Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:40.645322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1284 reason: error at write: Disconnected sessionId: src_id|1a885fe-4f18dbb5-51de5c91-fdec1c24_0
2025-10-09T21:48:40.645506Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1284 sessionId: src_id|1a885fe-4f18dbb5-51de5c91-fdec1c24_0 is DEAD
2025-10-09T21:48:40.645655Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a885fe-4f18dbb5-51de5c91-fdec1c24_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:40.645667Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a885fe-4f18dbb5-51de5c91-fdec1c24_0] Write session will restart in 0.005760s
2025-10-09T21:48:40.645706Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a885fe-4f18dbb5-51de5c91-fdec1c24_0] Write session: Do CDS request
2025-10-09T21:48:40.645713Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a885fe-4f18dbb5-51de5c91-fdec1c24_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:40.652519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a885fe-4f18dbb5-51de5c91-fdec1c24_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:40.652600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:40.652612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1285
2025-10-09T21:48:40.652721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1285 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:40.652753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1285 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:40.652769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1285 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:40.652774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:40.653226Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1285 sessionId:  describe result for acl check
2025-10-09T21:48:40.653264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:40.653266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:40.653268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:40.653274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342244088277516:4465] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:40.653848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342244088277516:4465] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:40.656154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:40.656370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342244088277516:4465] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:40.657983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:40.658175Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:40.658184Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:40.658195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:40.658248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:40.658286Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342243726454146:9580]
2025-10-09T21:48:40.658854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:40.659182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:40.659224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342244088277516:4465] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:40.659235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:40.659241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342244088277516:4465] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:40.659245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342244088277516:4465] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:40.659253Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1285 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:40.659596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:40.659649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.659656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.659657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.659659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.659660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.659661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.659663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.659664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.659665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.659670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.659680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.659685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.659689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.659693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.659697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.659701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.659704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.659708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.659714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.659722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.659729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.659734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.659738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.659744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.659748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.659759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.659765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.659770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.659775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.659780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.659786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.659791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.659810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.659819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.659824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.659829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.659857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:40.666984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:40.668103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:40.668118Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:40.668120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:40.668125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:40.668132Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1284 [2:7559342243726454146:9580]
2025-10-09T21:48:40.668145Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:40.668165Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:40.668175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:40.668179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:40.668181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:40.668198Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:40.668205Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:40.668216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:40.668271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.668277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.668279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.668282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:40.668349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:40.668356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.668368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.668384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.668406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:40.668430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342244088277598:4465], now have 1 active actors on pipe
2025-10-09T21:48:40.668442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342244088277488:2432], now have 1 active actors on pipe
2025-10-09T21:48:40.668450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342243726454140:9580]
2025-10-09T21:48:40.668452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342243726454140:9580]
2025-10-09T21:48:40.668454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342243726454140:9580]
2025-10-09T21:48:40.668468Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:40.668482Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:40.668492Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:40.668496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:40.668499Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:40.668501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:40.668503Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:40.668512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1284
2025-10-09T21:48:40.668523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342243726454140:9580]
2025-10-09T21:48:40.668529Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:40.668519Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1284
2025-10-09T21:48:40.668547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:40.668561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:40.668565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:40.668604Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6779d0e6-187e7f36-8ee99319-4483f6a2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:40.668616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:40.668620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:40.668627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:40.668637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:40.668651Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:40.668669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:40.668834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:40.668844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:40.668875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:40.669002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1285 partition: 0 MaxSeqNo: 25 sessionId: src_id|6779d0e6-187e7f36-8ee99319-4483f6a2_0
2025-10-09T21:48:40.669321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a885fe-4f18dbb5-51de5c91-fdec1c24_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6779d0e6-187e7f36-8ee99319-4483f6a2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:40.671793Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6779d0e6-187e7f36-8ee99319-4483f6a2_0] Write session: try to update token
2025-10-09T21:48:40.671813Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6779d0e6-187e7f36-8ee99319-4483f6a2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:40.683640Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:40.683658Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:40.683660Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:40.683663Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:40.683674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342243726454140:9580]
2025-10-09T21:48:40.683679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342243726454140:9580]
2025-10-09T21:48:40.683682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342243726454140:9580]
2025-10-09T21:48:40.683728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:40.683736Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:40.683743Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:40.683744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.683747Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:40.683747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:40.683883Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:40.684245Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1285 sessionId: src_id|6779d0e6-187e7f36-8ee99319-4483f6a2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:40.686601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:40.745476Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342244088277608:5175]
2025-10-09T21:48:40.745813Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 1284, pipe: [1:7559342244088277608:5175]
2025-10-09T21:48:40.837615Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:40.837934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:40.837949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:40.837994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342244088277608:5175], now have 1 active actors on pipe
2025-10-09T21:48:40.838217Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:40.838226Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:40.838241Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:40.838271Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342244088277608:5175]
2025-10-09T21:48:40.838293Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:40.838303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1285 reason: error at write: Disconnected sessionId: src_id|6779d0e6-187e7f36-8ee99319-4483f6a2_0
2025-10-09T21:48:40.838481Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1285 sessionId: src_id|6779d0e6-187e7f36-8ee99319-4483f6a2_0 is DEAD
2025-10-09T21:48:40.838637Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6779d0e6-187e7f36-8ee99319-4483f6a2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:40.838647Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6779d0e6-187e7f36-8ee99319-4483f6a2_0] Write session will restart in 0.009440s
2025-10-09T21:48:40.838674Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6779d0e6-187e7f36-8ee99319-4483f6a2_0] Write session: Do CDS request
2025-10-09T21:48:40.838681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6779d0e6-187e7f36-8ee99319-4483f6a2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:40.849574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6779d0e6-187e7f36-8ee99319-4483f6a2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:40.849632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:40.849647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1286
2025-10-09T21:48:40.849836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:40.849845Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1286 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:40.849895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1286 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:40.849906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1286 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:40.849910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:40.850384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1286 sessionId:  describe result for acl check
2025-10-09T21:48:40.850439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:40.850446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:40.850448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:40.850455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342244088277688:4456] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:40.851127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342244088277688:4456] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:40.851703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:40.851902Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:40.851921Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:40.851942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:40.852053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:40.852103Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342243726454259:9583]
2025-10-09T21:48:40.852753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:40.853066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:40.853113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:40.853416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342244088277688:4456] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:40.853429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:40.853500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.853510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.853511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.853512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.853513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.853514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.853515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.853516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.853517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.853523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.853532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.853536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.853541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.853544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.853549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.853553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.853557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.853561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.853570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.853580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:40.853588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.853594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:40.853605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.853611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:40.853628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.853634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:40.853640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.853654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:40.853659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.853672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:40.853678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.853705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.853710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.853716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.853722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.853728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.853765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:40.860723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342244088277688:4456] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:40.860741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342244088277688:4456] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:40.860746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342244088277688:4456] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:40.860756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1286 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:40.861098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:40.862248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:40.862264Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:40.862267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:40.862271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:40.862278Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1285 [2:7559342243726454259:9583]
2025-10-09T21:48:40.862293Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:40.862313Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:40.862316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:40.862319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:40.862322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:40.862340Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:40.862356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:40.862364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:40.862420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.862427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.862429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.862433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:40.862503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:40.862510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.862524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:40.862542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:40.862566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:40.862592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342244088277726:4456], now have 1 active actors on pipe
2025-10-09T21:48:40.862600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342244088277613:2432], now have 1 active actors on pipe
2025-10-09T21:48:40.862608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342243726454253:9583]
2025-10-09T21:48:40.862615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342243726454253:9583]
2025-10-09T21:48:40.862618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342243726454253:9583]
2025-10-09T21:48:40.862631Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:40.862644Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:40.862648Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:40.862650Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:40.862651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:40.862666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:40.862687Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1285
2025-10-09T21:48:40.862676Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:40.862695Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1285
2025-10-09T21:48:40.862715Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342243726454253:9583]
2025-10-09T21:48:40.862719Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:40.862721Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:40.862749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:40.862760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:40.862894Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|44724093-84bceda1-d805b20a-989a3df5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:40.862908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:40.862912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:40.862915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:40.862925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:40.862941Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:40.862969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:40.863112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:40.863120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:40.863134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:40.863251Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1286 partition: 0 MaxSeqNo: 25 sessionId: src_id|44724093-84bceda1-d805b20a-989a3df5_0
2025-10-09T21:48:40.863631Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6779d0e6-187e7f36-8ee99319-4483f6a2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|44724093-84bceda1-d805b20a-989a3df5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:40.866261Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|44724093-84bceda1-d805b20a-989a3df5_0] Write session: try to update token
2025-10-09T21:48:40.866282Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|44724093-84bceda1-d805b20a-989a3df5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:40.878536Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:40.878553Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:40.878556Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:40.878559Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:40.878572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342243726454253:9583]
2025-10-09T21:48:40.878576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342243726454253:9583]
2025-10-09T21:48:40.878580Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342243726454253:9583]
2025-10-09T21:48:40.878632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:40.878631Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:40.878638Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:40.878639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:40.878641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:40.878654Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:40.878773Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:40.879755Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1286 sessionId: src_id|44724093-84bceda1-d805b20a-989a3df5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:40.882119Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:40.938531Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342244088277733:5175]
2025-10-09T21:48:40.938919Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 1285, pipe: [1:7559342244088277733:5175]
2025-10-09T21:48:41.039023Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:41.039403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:41.039418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:41.039462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342244088277733:5175], now have 1 active actors on pipe
2025-10-09T21:48:41.039686Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:41.039700Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342244088277733:5175]
2025-10-09T21:48:41.039703Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:41.039710Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:41.039748Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:41.039784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1286 reason: error at write: Disconnected sessionId: src_id|44724093-84bceda1-d805b20a-989a3df5_0
2025-10-09T21:48:41.039987Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1286 sessionId: src_id|44724093-84bceda1-d805b20a-989a3df5_0 is DEAD
2025-10-09T21:48:41.040165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44724093-84bceda1-d805b20a-989a3df5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:41.040176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44724093-84bceda1-d805b20a-989a3df5_0] Write session will restart in 0.007431s
2025-10-09T21:48:41.040207Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44724093-84bceda1-d805b20a-989a3df5_0] Write session: Do CDS request
2025-10-09T21:48:41.040215Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44724093-84bceda1-d805b20a-989a3df5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:41.048602Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|44724093-84bceda1-d805b20a-989a3df5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:41.048657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:41.048675Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1287
2025-10-09T21:48:41.048841Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1287 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:41.048877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1287 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:41.048882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1287 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:41.048886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:41.049344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1287 sessionId:  describe result for acl check
2025-10-09T21:48:41.049375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:41.049377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:41.049380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:41.049386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342248383245092:4444] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:41.049948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342248383245092:4444] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:41.051838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:41.051987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342248383245092:4444] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:41.053670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:41.053893Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:41.053906Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:41.053917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:41.053990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:41.054045Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342248021421699:9595]
2025-10-09T21:48:41.054615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:41.054920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:41.054969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:41.054991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342248383245092:4444] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:41.055010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342248383245092:4444] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:41.055013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342248383245092:4444] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:41.055022Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1287 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:41.055288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:41.055363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.055373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.055374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.055375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.055377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.055378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.055379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.055380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.055381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.055386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.055396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.055401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.055405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.055410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.055414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.055418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.055422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.055426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.055433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.055442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.055450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.055456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.055462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.055467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.055472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.055478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.055483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.055492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.055497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.055502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.055512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.055518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.055527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.055533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.055539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.055544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.055582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:41.064829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:41.066170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:41.066185Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:41.066188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:41.066192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:41.066200Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1286 [2:7559342248021421699:9595]
2025-10-09T21:48:41.066214Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:41.066233Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:41.066241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:41.066244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:41.066247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:41.066266Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:41.066278Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:41.066286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:41.066347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.066354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.066357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.066360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:41.066435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:41.066443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.066458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.066477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.066500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:41.066527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342248383245041:2432], now have 1 active actors on pipe
2025-10-09T21:48:41.066540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342248021421693:9595]
2025-10-09T21:48:41.066542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342248021421693:9595]
2025-10-09T21:48:41.066545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342248021421693:9595]
2025-10-09T21:48:41.066558Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:41.066573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342248383245156:4444], now have 1 active actors on pipe
2025-10-09T21:48:41.066577Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:41.066580Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:41.066582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:41.066587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:41.066602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:41.066610Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:41.066622Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1286
2025-10-09T21:48:41.066648Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342248021421693:9595]
2025-10-09T21:48:41.066682Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1286
2025-10-09T21:48:41.066662Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:41.066666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:41.066738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:41.066749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:41.066777Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e75e65b-b503af2a-bb0f3cf9-9fc6f8b7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:41.066792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:41.066796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:41.066798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:41.066808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:41.066820Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:41.066841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:41.066995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:41.067003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:41.067017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:41.067097Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1287 partition: 0 MaxSeqNo: 25 sessionId: src_id|e75e65b-b503af2a-bb0f3cf9-9fc6f8b7_0
2025-10-09T21:48:41.067471Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|44724093-84bceda1-d805b20a-989a3df5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e75e65b-b503af2a-bb0f3cf9-9fc6f8b7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:41.077633Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e75e65b-b503af2a-bb0f3cf9-9fc6f8b7_0] Write session: try to update token
2025-10-09T21:48:41.077658Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e75e65b-b503af2a-bb0f3cf9-9fc6f8b7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:41.084636Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:41.084657Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:41.084659Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:41.084661Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:41.084671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342248021421693:9595]
2025-10-09T21:48:41.084675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342248021421693:9595]
2025-10-09T21:48:41.084679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342248021421693:9595]
2025-10-09T21:48:41.084723Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:41.084726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:41.084729Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:41.084733Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:41.084733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.084737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:41.084875Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:41.090974Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1287 sessionId: src_id|e75e65b-b503af2a-bb0f3cf9-9fc6f8b7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:41.099010Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:41.139709Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342248383245175:5175]
2025-10-09T21:48:41.140109Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 1286, pipe: [1:7559342248383245175:5175]
2025-10-09T21:48:41.241837Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:41.242190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:41.242207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:41.242233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342248383245175:5175], now have 1 active actors on pipe
2025-10-09T21:48:41.242520Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:41.242546Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:41.242551Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:41.242589Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342248383245175:5175]
2025-10-09T21:48:41.242615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1287 reason: error at write: Disconnected sessionId: src_id|e75e65b-b503af2a-bb0f3cf9-9fc6f8b7_0
2025-10-09T21:48:41.242661Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:41.242807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1287 sessionId: src_id|e75e65b-b503af2a-bb0f3cf9-9fc6f8b7_0 is DEAD
2025-10-09T21:48:41.242940Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e75e65b-b503af2a-bb0f3cf9-9fc6f8b7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:41.242951Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e75e65b-b503af2a-bb0f3cf9-9fc6f8b7_0] Write session will restart in 0.008583s
2025-10-09T21:48:41.242978Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e75e65b-b503af2a-bb0f3cf9-9fc6f8b7_0] Write session: Do CDS request
2025-10-09T21:48:41.242984Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e75e65b-b503af2a-bb0f3cf9-9fc6f8b7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:41.252573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e75e65b-b503af2a-bb0f3cf9-9fc6f8b7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:41.252625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:41.252643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1288
2025-10-09T21:48:41.252772Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1288 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:41.252808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1288 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:41.252821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1288 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:41.252825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:41.253374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1288 sessionId:  describe result for acl check
2025-10-09T21:48:41.253416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:41.253426Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:41.253428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:41.253434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342248383245259:4437] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:41.253978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342248383245259:4437] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:41.254556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:41.256609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:41.256792Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:41.256803Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:41.256815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:41.256881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:41.256894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342248383245259:4437] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:41.256917Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342248021421820:9599]
2025-10-09T21:48:41.257504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:41.257883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:41.257954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:41.258310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:41.258372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.258378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.258379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.258380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.258382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.258383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.258384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.258385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.258386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.258391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.258402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.258406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.258410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.258414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.258418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.258422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.258426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.258430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.258437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.258445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.258451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.258456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.258462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.258468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.258472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.258477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.258482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.258489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.258494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.258500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.258505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.258525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.258529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.258534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.258539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.258544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.258574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:41.264020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342248383245259:4437] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:41.264034Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342248383245259:4437] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:41.264038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342248383245259:4437] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:41.264047Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1288 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:41.265335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:41.266470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:41.266487Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:41.266490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:41.266494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:41.266501Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1287 [2:7559342248021421820:9599]
2025-10-09T21:48:41.266516Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:41.266533Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:41.266543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:41.266546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:41.266551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:41.266568Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:41.266579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:41.266586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:41.266644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.266654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.266657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.266660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:41.266726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:41.266734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.266747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.266760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.266779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:41.266801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342248383245186:2432], now have 1 active actors on pipe
2025-10-09T21:48:41.266811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342248383245301:4437], now have 1 active actors on pipe
2025-10-09T21:48:41.266818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342248021421814:9599]
2025-10-09T21:48:41.266825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342248021421814:9599]
2025-10-09T21:48:41.266828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342248021421814:9599]
2025-10-09T21:48:41.266844Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:41.266857Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1287
2025-10-09T21:48:41.266862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:41.266866Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:41.266870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:41.266872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:41.266886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:41.266889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:41.266928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342248021421814:9599]
2025-10-09T21:48:41.266936Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:41.266942Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:41.267067Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1287
2025-10-09T21:48:41.267133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:41.267147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:41.267176Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d47414fa-44772dc1-54dae191-15b75592_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:41.267190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:41.267199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:41.267202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:41.267216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:41.267231Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:41.267248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:41.267420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:41.267430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:41.267452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:41.267545Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1288 partition: 0 MaxSeqNo: 25 sessionId: src_id|d47414fa-44772dc1-54dae191-15b75592_0
2025-10-09T21:48:41.268017Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e75e65b-b503af2a-bb0f3cf9-9fc6f8b7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d47414fa-44772dc1-54dae191-15b75592_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:41.270612Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d47414fa-44772dc1-54dae191-15b75592_0] Write session: try to update token
2025-10-09T21:48:41.270633Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d47414fa-44772dc1-54dae191-15b75592_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:41.278449Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:41.278465Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:41.278474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342248021421814:9599]
2025-10-09T21:48:41.278519Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:41.278525Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:41.278687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:41.278824Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:41.278947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:41.279077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:41.279295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:41.279332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:41.285131Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1288 sessionId: src_id|d47414fa-44772dc1-54dae191-15b75592_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:41.288023Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:41.288228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:41.288247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:41.288286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:41.288374Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:41.293674Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:41.293701Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:41.293703Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:41.293705Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:41.293715Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342248021421814:9599]
2025-10-09T21:48:41.293719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342248021421814:9599]
2025-10-09T21:48:41.293726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342248021421814:9599]
2025-10-09T21:48:41.293769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:41.293774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.293777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:41.293782Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:41.293789Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:41.293803Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:41.293905Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:41.342982Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342248383245309:5175]
2025-10-09T21:48:41.343244Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 1287, pipe: [1:7559342248383245309:5175]
2025-10-09T21:48:41.446290Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:41.453609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:41.453634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:41.453809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:41.453874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:41.453932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:41.453998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:41.454170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:41.454183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:41.454256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:41.454324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:41.454390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:41.454461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:41.454506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:41.454514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:41.454575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:41.454635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:41.454692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:41.454754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:41.454800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:41.454803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:41.454865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:41.454929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:41.454990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:41.455049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:41.455091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:41.455100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:41.455165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:41.455224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:41.455287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:41.455346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:41.455386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:41.455393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:41.455453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:41.455516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:41.455590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:41.455647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:41.455689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:41.455697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:41.455756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:41.455808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:41.455865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:41.455928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:41.455978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:41.455985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:41.456037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:41.456088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:41.456146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:41.456204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:41.456257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:41.456261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:41.456313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:41.456362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:41.456421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:41.456477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:41.456514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:41.456519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:41.456571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:41.456620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:41.456679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:41.456729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:41.456770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:41.456777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:41.456843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342248383245309:5175], now have 1 active actors on pipe
2025-10-09T21:48:41.457116Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342248383245309:5175]
2025-10-09T21:48:41.457132Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:41.457161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:41.457164Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:41.457195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1288 reason: error at write: Disconnected sessionId: src_id|d47414fa-44772dc1-54dae191-15b75592_0
2025-10-09T21:48:41.457432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1288 sessionId: src_id|d47414fa-44772dc1-54dae191-15b75592_0 is DEAD
2025-10-09T21:48:41.457501Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:41.457543Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d47414fa-44772dc1-54dae191-15b75592_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:41.457553Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d47414fa-44772dc1-54dae191-15b75592_0] Write session will restart in 0.008434s
2025-10-09T21:48:41.457578Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d47414fa-44772dc1-54dae191-15b75592_0] Write session: Do CDS request
2025-10-09T21:48:41.457585Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d47414fa-44772dc1-54dae191-15b75592_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:41.467512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d47414fa-44772dc1-54dae191-15b75592_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:41.467584Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:41.467602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1289
2025-10-09T21:48:41.467718Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1289 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:41.467760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1289 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:41.467771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1289 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:41.467775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:41.468218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1289 sessionId:  describe result for acl check
2025-10-09T21:48:41.468254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:41.468263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:41.468265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:41.468271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342248383245385:4426] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:41.468818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342248383245385:4426] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:41.469194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:41.470855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:41.471037Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:41.471048Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:41.471060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:41.471111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:41.471151Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342248021421934:9601]
2025-10-09T21:48:41.471354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342248383245385:4426] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:41.471682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:41.472005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:41.472044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:41.472353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:41.472402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.472407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.472408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.472409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.472410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.472411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.472412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.472414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.472415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.472420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.472429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.472433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.472437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.472440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.472444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.472447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.472451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.472454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.472461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.472468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.472474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.472479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.472483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.472487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.472491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.472496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.472505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.472509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.472513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.472518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.472526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.472531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.472540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.472544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.472549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.472553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.472581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:41.477256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342248383245385:4426] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:41.477270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342248383245385:4426] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:41.477273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342248383245385:4426] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:41.477282Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1289 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:41.478650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:41.479766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:41.479786Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:41.479789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:41.479793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:41.479810Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1288 [2:7559342248021421934:9601]
2025-10-09T21:48:41.479824Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:41.479840Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:41.479847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:41.479850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:41.479852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:41.479869Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:41.479881Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:41.479888Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:41.479944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.479951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.479953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.479957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:41.480028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:41.480035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.480049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.480065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.480085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:41.480110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342248383245427:4426], now have 1 active actors on pipe
2025-10-09T21:48:41.480115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342248383245311:2432], now have 1 active actors on pipe
2025-10-09T21:48:41.480123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342248021421929:9601]
2025-10-09T21:48:41.480125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342248021421929:9601]
2025-10-09T21:48:41.480127Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342248021421929:9601]
2025-10-09T21:48:41.480141Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:41.480157Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:41.480163Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:41.480169Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:41.480171Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:41.480176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:41.480183Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:41.480168Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1288
2025-10-09T21:48:41.480171Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1288
2025-10-09T21:48:41.480210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342248021421929:9601]
2025-10-09T21:48:41.480217Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:41.480229Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:41.480234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:41.480238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:41.480292Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3cd1ed-c3718298-d348cd42-769feec7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:41.480306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:41.480311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:41.480313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:41.480323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:41.480337Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:41.480359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:41.480537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:41.480550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:41.480569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:41.480672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1289 partition: 0 MaxSeqNo: 25 sessionId: src_id|3cd1ed-c3718298-d348cd42-769feec7_0
2025-10-09T21:48:41.481001Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d47414fa-44772dc1-54dae191-15b75592_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3cd1ed-c3718298-d348cd42-769feec7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:41.483412Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3cd1ed-c3718298-d348cd42-769feec7_0] Write session: try to update token
2025-10-09T21:48:41.483432Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3cd1ed-c3718298-d348cd42-769feec7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:41.494193Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:41.494210Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:41.494212Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:41.494214Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:41.494225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342248021421929:9601]
2025-10-09T21:48:41.494229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342248021421929:9601]
2025-10-09T21:48:41.494233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342248021421929:9601]
2025-10-09T21:48:41.494275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:41.494275Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:41.494280Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:41.494282Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:41.494287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.494290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:41.494413Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: CLIENT_DEADLINE_EXCEEDED. Description: [ { <main>: Error: GRpc error: (4): Deadline Exceeded } { <main>: Error: Grpc error response on endpoint localhost:14704 } ]
2025-10-09T21:48:41.494422Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:41.496412Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1289 sessionId: src_id|3cd1ed-c3718298-d348cd42-769feec7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:41.498701Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:41.557530Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342248383245435:5175]
2025-10-09T21:48:41.557870Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 1288, pipe: [1:7559342248383245435:5175]
2025-10-09T21:48:41.649358Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:41.649639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:41.649657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:41.649713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342248383245435:5175], now have 1 active actors on pipe
2025-10-09T21:48:41.649929Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:41.649960Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:41.649971Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:41.649976Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:41.650044Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342248383245435:5175]
2025-10-09T21:48:41.650054Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1289 reason: error at write: Disconnected sessionId: src_id|3cd1ed-c3718298-d348cd42-769feec7_0
2025-10-09T21:48:41.650240Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1289 sessionId: src_id|3cd1ed-c3718298-d348cd42-769feec7_0 is DEAD
2025-10-09T21:48:41.650397Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cd1ed-c3718298-d348cd42-769feec7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:41.650407Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cd1ed-c3718298-d348cd42-769feec7_0] Write session will restart in 0.009485s
2025-10-09T21:48:41.650437Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cd1ed-c3718298-d348cd42-769feec7_0] Write session: Do CDS request
2025-10-09T21:48:41.650443Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cd1ed-c3718298-d348cd42-769feec7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:41.661172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:41.661518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3cd1ed-c3718298-d348cd42-769feec7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:41.661621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:41.661642Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1290
2025-10-09T21:48:41.661823Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1290 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:41.661862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1290 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:41.661881Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1290 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:41.661886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:41.662451Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1290 sessionId:  describe result for acl check
2025-10-09T21:48:41.662494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:41.662501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:41.662503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:41.662509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342248383245516:4423] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:41.663110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342248383245516:4423] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:41.663365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:41.663531Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:41.663539Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:41.663548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:41.663590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:41.663619Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342248021422051:9604]
2025-10-09T21:48:41.664043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:41.664399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:41.664468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:41.664773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:41.664832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.664848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.664849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.664850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.664851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.664852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.664854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.664855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.664856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.664862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.664871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.664875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.664880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.664883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.664886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.664890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.664893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.664907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.664914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.664922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.664928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.664940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.664944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.664950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.664954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.664959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.664964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.664969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.664973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.664984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.664989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.664999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.665003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.665017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.665029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.665034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.665062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:41.665408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342248383245516:4423] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:41.671203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342248383245516:4423] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:41.671218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342248383245516:4423] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:41.671222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342248383245516:4423] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:41.671230Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1290 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:41.671529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:41.672489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:41.672505Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:41.672508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:41.672512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:41.672518Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1289 [2:7559342248021422051:9604]
2025-10-09T21:48:41.672532Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:41.672548Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:41.672556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:41.672559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:41.672561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:41.672578Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:41.672588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:41.672596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:41.672653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.672660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.672661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.672663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:41.672721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:41.672726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.672734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.672744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.672757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:41.672778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342248383245555:4423], now have 1 active actors on pipe
2025-10-09T21:48:41.672786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342248383245437:2432], now have 1 active actors on pipe
2025-10-09T21:48:41.672793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342248021422045:9604]
2025-10-09T21:48:41.672801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342248021422045:9604]
2025-10-09T21:48:41.672804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342248021422045:9604]
2025-10-09T21:48:41.672817Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:41.672832Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:41.672836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:41.672841Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:41.672844Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:41.672849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:41.672852Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:41.672848Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1289
2025-10-09T21:48:41.672858Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1289
2025-10-09T21:48:41.672877Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342248021422045:9604]
2025-10-09T21:48:41.672886Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:41.672888Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:41.672908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:41.672915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:41.672940Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a7e12381-587c17b2-6b374ca9-1eddaf9d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:41.672950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:41.672953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:41.672954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:41.672961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:41.672972Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:41.672988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:41.673163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:41.673171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:41.673183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:41.673255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1290 partition: 0 MaxSeqNo: 25 sessionId: src_id|a7e12381-587c17b2-6b374ca9-1eddaf9d_0
2025-10-09T21:48:41.673603Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cd1ed-c3718298-d348cd42-769feec7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a7e12381-587c17b2-6b374ca9-1eddaf9d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:41.675984Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7e12381-587c17b2-6b374ca9-1eddaf9d_0] Write session: try to update token
2025-10-09T21:48:41.676001Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7e12381-587c17b2-6b374ca9-1eddaf9d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:41.687321Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:41.687339Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:41.687342Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:41.687344Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:41.687355Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342248021422045:9604]
2025-10-09T21:48:41.687359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342248021422045:9604]
2025-10-09T21:48:41.687363Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342248021422045:9604]
2025-10-09T21:48:41.687405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:41.687404Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:41.687412Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:41.687413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.687416Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:41.687416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:41.687558Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:41.688802Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1290 sessionId: src_id|a7e12381-587c17b2-6b374ca9-1eddaf9d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:41.691071Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:41.750040Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342248383245565:5175]
2025-10-09T21:48:41.750379Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 1289, pipe: [1:7559342248383245565:5175]
2025-10-09T21:48:41.848001Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:41.848401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:41.848414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:41.848453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342248383245565:5175], now have 1 active actors on pipe
2025-10-09T21:48:41.848654Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:41.848736Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342248383245565:5175]
2025-10-09T21:48:41.848750Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:41.848778Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:41.848851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1290 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a7e12381-587c17b2-6b374ca9-1eddaf9d_0
2025-10-09T21:48:41.849040Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1290 sessionId: src_id|a7e12381-587c17b2-6b374ca9-1eddaf9d_0 is DEAD
2025-10-09T21:48:41.849189Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7e12381-587c17b2-6b374ca9-1eddaf9d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:41.849200Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7e12381-587c17b2-6b374ca9-1eddaf9d_0] Write session will restart in 0.005300s
2025-10-09T21:48:41.849227Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7e12381-587c17b2-6b374ca9-1eddaf9d_0] Write session: Do CDS request
2025-10-09T21:48:41.849235Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7e12381-587c17b2-6b374ca9-1eddaf9d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:41.855548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7e12381-587c17b2-6b374ca9-1eddaf9d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:41.855645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:41.855662Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1291
2025-10-09T21:48:41.855767Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1291 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:41.855814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1291 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:41.855824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1291 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:41.855828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:41.856309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1291 sessionId:  describe result for acl check
2025-10-09T21:48:41.856353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:41.856360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:41.856362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:41.856368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342248383245585:4401] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:41.856927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342248383245585:4401] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:41.859619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342248383245585:4401] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:41.860613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:41.862651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:41.862862Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:41.862871Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:41.862882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:41.862930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:41.862957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342248383245585:4401] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:41.862972Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342248383245585:4401] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:41.862976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342248383245585:4401] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:41.862985Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1291 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:41.862983Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342248021422169:9607]
2025-10-09T21:48:41.863587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:41.863940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:41.863996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:41.864391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:41.864456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.864463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.864465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.864466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.864467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.864468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.864470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.864471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.864472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.864478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.864489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.864494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.864499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.864504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.864508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.864511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.864515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.864518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.864525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.864533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:41.864539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.864545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:41.864550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.864555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:41.864559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.864568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:41.864573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.864583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:41.864587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.864594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:41.864603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.864608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.864617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.864636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.864646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.864651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.864681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:41.871389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:41.872443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:41.872461Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:41.872464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:41.872469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:41.872476Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1290 [2:7559342248021422169:9607]
2025-10-09T21:48:41.872491Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:41.872510Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:41.872513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:41.872516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:41.872519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:41.872538Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:41.872549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:41.872557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:41.872618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.872625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.872627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.872630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:41.872703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:41.872711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.872724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:41.872741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:41.872763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:41.872788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342248383245570:2432], now have 1 active actors on pipe
2025-10-09T21:48:41.872797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342248383245684:4401], now have 1 active actors on pipe
2025-10-09T21:48:41.872805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342248021422163:9607]
2025-10-09T21:48:41.872807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342248021422163:9607]
2025-10-09T21:48:41.872810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342248021422163:9607]
2025-10-09T21:48:41.872821Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:41.872836Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:41.872839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:41.872841Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:41.872843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:41.872855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:41.872859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:41.872842Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1290
2025-10-09T21:48:41.872853Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1290
2025-10-09T21:48:41.872896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342248021422163:9607]
2025-10-09T21:48:41.872908Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:41.872913Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:41.872928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:41.872933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:41.873002Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1db8cd58-a581d60f-b49a93cf-ece5ba8a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:41.873014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:41.873018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:41.873020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:41.873032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:41.873043Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:41.873066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:41.873201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:41.873209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:41.873223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:41.873303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1291 partition: 0 MaxSeqNo: 25 sessionId: src_id|1db8cd58-a581d60f-b49a93cf-ece5ba8a_0
2025-10-09T21:48:41.873732Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7e12381-587c17b2-6b374ca9-1eddaf9d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1db8cd58-a581d60f-b49a93cf-ece5ba8a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:41.876208Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1db8cd58-a581d60f-b49a93cf-ece5ba8a_0] Write session: try to update token
2025-10-09T21:48:41.876229Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1db8cd58-a581d60f-b49a93cf-ece5ba8a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:41.887612Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:41.887632Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:41.887635Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:41.887637Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:41.887647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342248021422163:9607]
2025-10-09T21:48:41.887651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342248021422163:9607]
2025-10-09T21:48:41.887658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342248021422163:9607]
2025-10-09T21:48:41.887705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:41.887708Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:41.887714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:41.887714Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:41.887716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:41.887718Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:41.887856Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:41.889441Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1291 sessionId: src_id|1db8cd58-a581d60f-b49a93cf-ece5ba8a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:41.891838Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:41.948744Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342248383245695:5175]
2025-10-09T21:48:41.949128Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 1290, pipe: [1:7559342248383245695:5175]
2025-10-09T21:48:42.047596Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:42.048331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:42.048351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:42.048392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342248383245695:5175], now have 1 active actors on pipe
2025-10-09T21:48:42.048566Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:42.048567Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:42.048575Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:42.048584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:42.048632Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342248383245695:5175]
2025-10-09T21:48:42.048645Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1291 reason: error at write: Disconnected sessionId: src_id|1db8cd58-a581d60f-b49a93cf-ece5ba8a_0
2025-10-09T21:48:42.048832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1291 sessionId: src_id|1db8cd58-a581d60f-b49a93cf-ece5ba8a_0 is DEAD
2025-10-09T21:48:42.048998Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1db8cd58-a581d60f-b49a93cf-ece5ba8a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:42.049005Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1db8cd58-a581d60f-b49a93cf-ece5ba8a_0] Write session will restart in 0.008960s
2025-10-09T21:48:42.049032Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1db8cd58-a581d60f-b49a93cf-ece5ba8a_0] Write session: Do CDS request
2025-10-09T21:48:42.049039Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1db8cd58-a581d60f-b49a93cf-ece5ba8a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:42.059590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1db8cd58-a581d60f-b49a93cf-ece5ba8a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:42.059987Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:42.060013Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1292
2025-10-09T21:48:42.060213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:42.060231Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1292 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:42.060272Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1292 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:42.060286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1292 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:42.060291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:42.060833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1292 sessionId:  describe result for acl check
2025-10-09T21:48:42.060877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:42.060885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:42.060888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:42.060896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342252678213077:4400] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:42.061524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342252678213077:4400] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:42.061901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:42.062104Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:42.062113Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:42.062124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:42.062182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:42.062224Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342252316389613:9619]
2025-10-09T21:48:42.062803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:42.063071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:42.063113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:42.063425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:42.063473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.063479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.063480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.063481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.063482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.063483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.063484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.063485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.063487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.063491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.063502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.063506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.063510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.063514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.063518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.063522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.063525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.063529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.063535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.063543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.063549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.063555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.063558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.063563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.063566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.063570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.063577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.063581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.063585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.063591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.063596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.063601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.063606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.063611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.063615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.063625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.063652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:42.063670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342252678213077:4400] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:42.074121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342252678213077:4400] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:42.074145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342252678213077:4400] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:42.074149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342252678213077:4400] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:42.074159Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1292 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:42.074344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:42.075959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:42.075973Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:42.075976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:42.075981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:42.075989Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1291 [2:7559342252316389613:9619]
2025-10-09T21:48:42.076002Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:42.076024Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:42.076027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:42.076031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:42.076033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:42.076051Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:42.076060Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:42.076067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:42.076199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.076211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.076214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.076216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:42.076312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:42.076322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.076335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.076347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.076368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:42.076392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342252678212994:2432], now have 1 active actors on pipe
2025-10-09T21:48:42.076401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342252678213113:4400], now have 1 active actors on pipe
2025-10-09T21:48:42.076408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342252316389607:9619]
2025-10-09T21:48:42.076415Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342252316389607:9619]
2025-10-09T21:48:42.076417Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342252316389607:9619]
2025-10-09T21:48:42.076433Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:42.076453Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:42.076469Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:42.076472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:42.076483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:42.076494Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:42.076506Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:42.076508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342252316389607:9619]
2025-10-09T21:48:42.076512Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:42.076519Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:42.076458Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1291
2025-10-09T21:48:42.076471Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1291
2025-10-09T21:48:42.076668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:42.076683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:42.076714Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b11135bc-e0f99d99-8b485c41-4fbb3ef1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:42.076729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:42.076733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:42.076736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:42.076748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:42.076762Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:42.076785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:42.077003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:42.077012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:42.077028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:42.077136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1292 partition: 0 MaxSeqNo: 25 sessionId: src_id|b11135bc-e0f99d99-8b485c41-4fbb3ef1_0
2025-10-09T21:48:42.077511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1db8cd58-a581d60f-b49a93cf-ece5ba8a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b11135bc-e0f99d99-8b485c41-4fbb3ef1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:42.080034Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b11135bc-e0f99d99-8b485c41-4fbb3ef1_0] Write session: try to update token
2025-10-09T21:48:42.080056Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b11135bc-e0f99d99-8b485c41-4fbb3ef1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:42.091561Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:42.091580Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:42.091590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342252316389607:9619]
2025-10-09T21:48:42.091639Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:42.091640Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:42.092189Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:42.092521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:42.092835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:42.093150Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:42.093624Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:42.093671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:42.099291Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1292 sessionId: src_id|b11135bc-e0f99d99-8b485c41-4fbb3ef1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:42.101815Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:42.102220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:42.102244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:42.102267Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:42.102274Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:42.102277Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:42.102279Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:42.102288Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342252316389607:9619]
2025-10-09T21:48:42.102291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342252316389607:9619]
2025-10-09T21:48:42.102295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342252316389607:9619]
2025-10-09T21:48:42.102331Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:42.102343Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:42.102347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:42.102347Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:42.102353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.102356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:42.102479Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:42.148878Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342252678213145:5175]
2025-10-09T21:48:42.149213Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 1291, pipe: [1:7559342252678213145:5175]
2025-10-09T21:48:42.264020Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:42.264410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:42.264450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342252678213145:5175], now have 1 active actors on pipe
2025-10-09T21:48:42.264574Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:42.264613Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342252678213145:5175]
2025-10-09T21:48:42.264633Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:42.264672Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:42.264682Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:42.264762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1292 reason: error at write: Disconnected sessionId: src_id|b11135bc-e0f99d99-8b485c41-4fbb3ef1_0
2025-10-09T21:48:42.264929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1292 sessionId: src_id|b11135bc-e0f99d99-8b485c41-4fbb3ef1_0 is DEAD
2025-10-09T21:48:42.265092Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b11135bc-e0f99d99-8b485c41-4fbb3ef1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:42.265104Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b11135bc-e0f99d99-8b485c41-4fbb3ef1_0] Write session will restart in 0.007487s
2025-10-09T21:48:42.265135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b11135bc-e0f99d99-8b485c41-4fbb3ef1_0] Write session: Do CDS request
2025-10-09T21:48:42.265146Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b11135bc-e0f99d99-8b485c41-4fbb3ef1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:42.273543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b11135bc-e0f99d99-8b485c41-4fbb3ef1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:42.273890Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:42.273905Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1293
2025-10-09T21:48:42.274051Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1293 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:42.274084Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1293 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:42.274094Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1293 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:42.274097Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:42.274547Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1293 sessionId:  describe result for acl check
2025-10-09T21:48:42.274589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:42.274592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:42.274596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:42.274602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342252678213197:4381] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:42.275138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342252678213197:4381] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:42.276899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:42.277297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342252678213197:4381] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:42.280628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:42.280822Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:42.280832Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:42.280843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:42.280894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:42.280899Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342252678213197:4381] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:42.280931Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342252316389739:9623]
2025-10-09T21:48:42.280914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342252678213197:4381] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:42.280917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342252678213197:4381] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:42.280936Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1293 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:42.281480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:42.281790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:42.281839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:42.282109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:42.282164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.282173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.282174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.282175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.282177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.282178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.282179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.282180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.282181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.282186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.282197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.282201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.282205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.282208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.282213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.282216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.282221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.282225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.282231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.282240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.282246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.282251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.282255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.282260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.282265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.282270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.282275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.282280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.282284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.282289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.282294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.282299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.282304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.282309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.282313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.282319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.282348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:42.288904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:42.289997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:42.290015Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:42.290018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:42.290022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:42.290045Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1292 [2:7559342252316389739:9623]
2025-10-09T21:48:42.290061Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:42.290079Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:42.290087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:42.290090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:42.290093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:42.290110Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:42.290121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:42.290129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:42.290190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.290197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.290200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.290203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:42.290275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:42.290286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.290299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.290311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.290327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:42.290355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342252678213269:4381], now have 1 active actors on pipe
2025-10-09T21:48:42.290363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342252678213153:2432], now have 1 active actors on pipe
2025-10-09T21:48:42.290371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342252316389734:9623]
2025-10-09T21:48:42.290379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342252316389734:9623]
2025-10-09T21:48:42.290382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342252316389734:9623]
2025-10-09T21:48:42.290395Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:42.290406Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:42.290409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:42.290410Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:42.290412Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:42.290424Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:42.290428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:42.290459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342252316389734:9623]
2025-10-09T21:48:42.290468Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:42.290441Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1292
2025-10-09T21:48:42.290483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:42.290441Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1292
2025-10-09T21:48:42.290499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:42.290508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:42.290544Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|824c424d-4dbd2437-cbad181a-38732c15_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:42.290556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:42.290560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:42.290562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:42.290575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:42.290586Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:42.290608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:42.290795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:42.290803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:42.290818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:42.290920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1293 partition: 0 MaxSeqNo: 25 sessionId: src_id|824c424d-4dbd2437-cbad181a-38732c15_0
2025-10-09T21:48:42.291319Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b11135bc-e0f99d99-8b485c41-4fbb3ef1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|824c424d-4dbd2437-cbad181a-38732c15_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:42.293781Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|824c424d-4dbd2437-cbad181a-38732c15_0] Write session: try to update token
2025-10-09T21:48:42.293800Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|824c424d-4dbd2437-cbad181a-38732c15_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:42.304968Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:42.304987Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:42.304989Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:42.304992Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:42.305003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342252316389734:9623]
2025-10-09T21:48:42.305008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342252316389734:9623]
2025-10-09T21:48:42.305013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342252316389734:9623]
2025-10-09T21:48:42.305062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:42.305069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.305072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:42.305075Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:42.305083Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:42.305098Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:42.305226Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:42.307713Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1293 sessionId: src_id|824c424d-4dbd2437-cbad181a-38732c15_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:42.310013Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:42.364960Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342252678213281:5175]
2025-10-09T21:48:42.365247Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 1292, pipe: [1:7559342252678213281:5175]
2025-10-09T21:48:42.467755Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:42.468186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:42.468201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:42.468250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342252678213281:5175], now have 1 active actors on pipe
2025-10-09T21:48:42.468465Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:42.468513Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:42.468528Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342252678213281:5175]
2025-10-09T21:48:42.468529Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:42.468549Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:42.468599Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1293 reason: error at write: Disconnected sessionId: src_id|824c424d-4dbd2437-cbad181a-38732c15_0
2025-10-09T21:48:42.468774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1293 sessionId: src_id|824c424d-4dbd2437-cbad181a-38732c15_0 is DEAD
2025-10-09T21:48:42.468921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|824c424d-4dbd2437-cbad181a-38732c15_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:42.468932Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|824c424d-4dbd2437-cbad181a-38732c15_0] Write session will restart in 0.009764s
2025-10-09T21:48:42.468958Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|824c424d-4dbd2437-cbad181a-38732c15_0] Write session: Do CDS request
2025-10-09T21:48:42.468965Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|824c424d-4dbd2437-cbad181a-38732c15_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:42.479542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|824c424d-4dbd2437-cbad181a-38732c15_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:42.479673Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:42.479705Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1294
2025-10-09T21:48:42.479869Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1294 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:42.479923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1294 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:42.479934Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1294 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:42.479938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:42.479947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:42.480376Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1294 sessionId:  describe result for acl check
2025-10-09T21:48:42.480413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:42.480420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:42.480422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:42.480429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342252678213366:4378] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:42.480984Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342252678213366:4378] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:42.481761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:42.481955Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:42.481970Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:42.481983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:42.482056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:42.482096Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342252316389858:9625]
2025-10-09T21:48:42.482627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:42.482922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:42.482956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:42.483014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342252678213366:4378] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:42.483313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:42.483382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.483388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.483389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.483390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.483392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.483393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.483394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.483395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.483396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.483401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.483411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.483415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.483418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.483421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.483425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.483428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.483431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.483434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.483440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.483448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.483455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.483460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.483464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.483469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.483472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.483478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.483482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.483487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.483491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.483496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.483501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.483506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.483510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.483515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.483519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.483524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.483558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:42.488617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342252678213366:4378] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:42.488634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342252678213366:4378] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:42.488638Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342252678213366:4378] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:42.488647Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1294 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:42.489979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:42.491031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:42.491042Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:42.491044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:42.491047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:42.491052Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1293 [2:7559342252316389858:9625]
2025-10-09T21:48:42.491061Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:42.491076Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:42.491078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:42.491080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:42.491081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:42.491097Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:42.491108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:42.491115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:42.491157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.491163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.491164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.491166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:42.491216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:42.491222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.491229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.491235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.491245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:42.491260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342252316389852:9625]
2025-10-09T21:48:42.491265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342252316389852:9625]
2025-10-09T21:48:42.491266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342252316389852:9625]
2025-10-09T21:48:42.491272Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:42.491282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:42.491285Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:42.491286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342252678213283:2432], now have 1 active actors on pipe
2025-10-09T21:48:42.491287Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:42.491290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342252678213404:4378], now have 1 active actors on pipe
2025-10-09T21:48:42.491298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:42.491313Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:42.491320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:42.491342Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342252316389852:9625]
2025-10-09T21:48:42.491357Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:42.491350Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1293
2025-10-09T21:48:42.491371Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:42.491421Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1293
2025-10-09T21:48:42.491475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:42.491500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:42.491537Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|74b79e82-9a69f33b-e3fefc34-94ae943_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:42.491553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:42.491558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:42.491560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:42.491571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:42.491583Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:42.491598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:42.491790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:42.491800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:42.491819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:42.491928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1294 partition: 0 MaxSeqNo: 25 sessionId: src_id|74b79e82-9a69f33b-e3fefc34-94ae943_0
2025-10-09T21:48:42.492255Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|824c424d-4dbd2437-cbad181a-38732c15_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|74b79e82-9a69f33b-e3fefc34-94ae943_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:42.494750Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74b79e82-9a69f33b-e3fefc34-94ae943_0] Write session: try to update token
2025-10-09T21:48:42.494768Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74b79e82-9a69f33b-e3fefc34-94ae943_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:42.502908Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:42.502921Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:42.502934Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342252316389852:9625]
2025-10-09T21:48:42.502989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:42.503328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:42.503483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:42.503630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:42.503736Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:42.503779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:42.504461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:42.504513Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:42.508844Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1294 sessionId: src_id|74b79e82-9a69f33b-e3fefc34-94ae943_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:42.511282Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:42.511502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:42.511523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:42.511566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:42.511704Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:42.517436Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:42.517449Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:42.517452Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:42.517454Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:42.517466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342252316389852:9625]
2025-10-09T21:48:42.517470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342252316389852:9625]
2025-10-09T21:48:42.517473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342252316389852:9625]
2025-10-09T21:48:42.517521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:42.517533Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:42.517535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.517537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:42.517539Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:42.517544Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:42.517725Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:42.568737Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342252678213412:5175]
2025-10-09T21:48:42.569132Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 1293, pipe: [1:7559342252678213412:5175]
2025-10-09T21:48:42.680389Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:42.688314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:42.688339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:42.688504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:42.688569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:42.688635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:42.688705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:42.688766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:42.688779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:42.688847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:42.688911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:42.688987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:42.689048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:42.689088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:42.689097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:42.689152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:42.689209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:42.689265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:42.689323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:42.689433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:42.689440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:42.689500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:42.689559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:42.689617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:42.689675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:42.689731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:42.689737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:42.689796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:42.689853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:42.689910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:42.689978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:42.690023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:42.690042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:42.690105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:42.690162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:42.690219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:42.690277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:42.690318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:42.690323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:42.690380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:42.690434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:42.690499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:42.690557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:42.690610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:42.690615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:42.690673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:42.690730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:42.690786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:42.690849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:42.690912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:42.690922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:42.690981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:42.691037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:42.691104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:42.691171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:42.691217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:42.691226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:42.691287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:42.691348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:42.691410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:42.691474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:42.691607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:42.691615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:42.691674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342252678213412:5175], now have 1 active actors on pipe
2025-10-09T21:48:42.691958Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:42.691963Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342252678213412:5175]
2025-10-09T21:48:42.691996Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:42.692000Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:42.692031Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1294 reason: error at write: Disconnected sessionId: src_id|74b79e82-9a69f33b-e3fefc34-94ae943_0
2025-10-09T21:48:42.692237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1294 sessionId: src_id|74b79e82-9a69f33b-e3fefc34-94ae943_0 is DEAD
2025-10-09T21:48:42.692393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74b79e82-9a69f33b-e3fefc34-94ae943_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:42.692404Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74b79e82-9a69f33b-e3fefc34-94ae943_0] Write session will restart in 0.008152s
2025-10-09T21:48:42.692430Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74b79e82-9a69f33b-e3fefc34-94ae943_0] Write session: Do CDS request
2025-10-09T21:48:42.692453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74b79e82-9a69f33b-e3fefc34-94ae943_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:42.692642Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:42.701567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74b79e82-9a69f33b-e3fefc34-94ae943_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:42.701804Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:42.701827Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1295
2025-10-09T21:48:42.702057Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1295 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:42.702099Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1295 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:42.702105Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1295 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:42.702109Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:42.702673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1295 sessionId:  describe result for acl check
2025-10-09T21:48:42.702721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:42.702728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:42.702731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:42.702737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342252678213486:4369] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:42.703381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342252678213486:4369] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:42.704962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:42.706403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342252678213486:4369] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:42.707261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:42.707535Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:42.707549Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:42.707564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:42.707632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:42.707680Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342252316389982:9629]
2025-10-09T21:48:42.708295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:42.708662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:42.708718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:42.709060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:42.709118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.709126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.709128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.709129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.709131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.709132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.709134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.709135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.709136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.709159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342252678213486:4369] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:42.709176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342252678213486:4369] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:42.709181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342252678213486:4369] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:42.709142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.709191Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1295 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:42.709153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.709158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.709162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.709166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.709171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.709175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.709178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.709182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.709190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.709199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.709206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.709212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.709216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.709222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.709227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.709233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.709237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.709243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.709248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.709253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.709259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.709264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.709270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.709276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.709281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.709287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.709317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:42.716249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:42.717515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:42.717534Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:42.717537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:42.717542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:42.717549Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1294 [2:7559342252316389982:9629]
2025-10-09T21:48:42.717565Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:42.717585Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:42.717595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:42.717599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:42.717602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:42.717619Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:42.717626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:42.717635Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:42.717711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.717720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.717722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.717725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:42.717801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:42.717808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.717821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.717839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.717859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:42.717889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342252678213415:2432], now have 1 active actors on pipe
2025-10-09T21:48:42.717897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342252678213535:4369], now have 1 active actors on pipe
2025-10-09T21:48:42.717905Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342252316389977:9629]
2025-10-09T21:48:42.717908Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342252316389977:9629]
2025-10-09T21:48:42.717910Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342252316389977:9629]
2025-10-09T21:48:42.717924Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:42.717945Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:42.717945Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:42.717965Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:42.717966Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:42.717969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:42.717973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:42.718006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342252316389977:9629]
2025-10-09T21:48:42.718014Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:42.718046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:42.718069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:42.717992Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1294
2025-10-09T21:48:42.718077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:42.718003Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1294
2025-10-09T21:48:42.718117Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f7eeab0a-cfb7bc38-87b5df9c-3e23ae81_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:42.718130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:42.718134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:42.718141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:42.718154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:42.718168Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:42.718189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:42.718394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:42.718403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:42.718420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:42.718522Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1295 partition: 0 MaxSeqNo: 25 sessionId: src_id|f7eeab0a-cfb7bc38-87b5df9c-3e23ae81_0
2025-10-09T21:48:42.719029Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74b79e82-9a69f33b-e3fefc34-94ae943_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f7eeab0a-cfb7bc38-87b5df9c-3e23ae81_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:42.721513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7eeab0a-cfb7bc38-87b5df9c-3e23ae81_0] Write session: try to update token
2025-10-09T21:48:42.721533Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7eeab0a-cfb7bc38-87b5df9c-3e23ae81_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:42.732834Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:42.732853Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:42.732856Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:42.732858Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:42.732869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342252316389977:9629]
2025-10-09T21:48:42.732874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342252316389977:9629]
2025-10-09T21:48:42.732877Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342252316389977:9629]
2025-10-09T21:48:42.732921Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:42.732923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:42.732926Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:42.732930Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:42.732930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.732934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:42.733060Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:42.737698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1295 sessionId: src_id|f7eeab0a-cfb7bc38-87b5df9c-3e23ae81_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:42.740535Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:42.791969Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342252678213544:5175]
2025-10-09T21:48:42.792287Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 1294, pipe: [1:7559342252678213544:5175]
2025-10-09T21:48:42.890102Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:42.890453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:42.890474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:42.890523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342252678213544:5175], now have 1 active actors on pipe
2025-10-09T21:48:42.890776Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:42.890855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:42.890862Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:42.890876Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:42.890919Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342252678213544:5175]
2025-10-09T21:48:42.890963Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1295 reason: error at write: Disconnected sessionId: src_id|f7eeab0a-cfb7bc38-87b5df9c-3e23ae81_0
2025-10-09T21:48:42.891153Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1295 sessionId: src_id|f7eeab0a-cfb7bc38-87b5df9c-3e23ae81_0 is DEAD
2025-10-09T21:48:42.891292Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7eeab0a-cfb7bc38-87b5df9c-3e23ae81_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:42.891303Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7eeab0a-cfb7bc38-87b5df9c-3e23ae81_0] Write session will restart in 0.005406s
2025-10-09T21:48:42.891385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7eeab0a-cfb7bc38-87b5df9c-3e23ae81_0] Write session: Do CDS request
2025-10-09T21:48:42.891399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7eeab0a-cfb7bc38-87b5df9c-3e23ae81_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:42.897488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7eeab0a-cfb7bc38-87b5df9c-3e23ae81_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:42.897590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:42.897617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1296
2025-10-09T21:48:42.897821Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1296 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:42.897872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1296 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:42.897884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1296 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:42.897889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:42.898421Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1296 sessionId:  describe result for acl check
2025-10-09T21:48:42.898468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:42.898476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:42.898479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:42.898486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342252678213564:4361] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:42.899144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342252678213564:4361] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:42.901563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342252678213564:4361] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:42.901897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:42.903565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:42.903747Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:42.903757Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:42.903768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:42.903819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:42.903860Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342252316390104:9631]
2025-10-09T21:48:42.904243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342252678213564:4361] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:42.904265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342252678213564:4361] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:42.904269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342252678213564:4361] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:42.904278Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1296 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:42.904474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:42.904829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:42.904872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:42.905180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:42.905226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.905232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.905233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.905234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.905235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.905236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.905237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.905238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.905239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.905244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.905255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.905259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.905263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.905267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.905271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.905275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.905278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.905282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.905289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.905296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:42.905302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.905307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:42.905312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.905325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:42.905330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.905335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:42.905340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.905345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:42.905353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.905358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:42.905363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.905368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.905377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.905382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.905387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.905392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.905423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:42.911966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:42.913007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:42.913025Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:42.913028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:42.913031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:42.913039Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1295 [2:7559342252316390104:9631]
2025-10-09T21:48:42.913051Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:42.913067Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:42.913077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:42.913080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:42.913082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:42.913097Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:42.913103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:42.913111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:42.913159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.913166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.913168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.913171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:42.913232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:42.913238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.913250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:42.913265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:42.913284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:42.913312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342252678213549:2432], now have 1 active actors on pipe
2025-10-09T21:48:42.913315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342252678213668:4361], now have 1 active actors on pipe
2025-10-09T21:48:42.913322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342252316390098:9631]
2025-10-09T21:48:42.913324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342252316390098:9631]
2025-10-09T21:48:42.913327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342252316390098:9631]
2025-10-09T21:48:42.913338Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:42.913351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:42.913370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:42.913377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:42.913358Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1295
2025-10-09T21:48:42.913397Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:42.913413Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1295
2025-10-09T21:48:42.913399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342252316390098:9631]
2025-10-09T21:48:42.913401Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:42.913407Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:42.913409Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:42.913417Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:42.913477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:42.913487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:42.913515Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2659d239-c8a95b46-bce8e521-96a28ef2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:42.913527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:42.913532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:42.913534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:42.913543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:42.913554Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:42.913567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:42.913788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:42.913801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:42.913823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:42.913931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1296 partition: 0 MaxSeqNo: 25 sessionId: src_id|2659d239-c8a95b46-bce8e521-96a28ef2_0
2025-10-09T21:48:42.914236Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7eeab0a-cfb7bc38-87b5df9c-3e23ae81_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2659d239-c8a95b46-bce8e521-96a28ef2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:42.916640Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2659d239-c8a95b46-bce8e521-96a28ef2_0] Write session: try to update token
2025-10-09T21:48:42.916660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2659d239-c8a95b46-bce8e521-96a28ef2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:42.927999Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:42.928020Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:42.928023Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:42.928027Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:42.928037Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342252316390098:9631]
2025-10-09T21:48:42.928041Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342252316390098:9631]
2025-10-09T21:48:42.928045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342252316390098:9631]
2025-10-09T21:48:42.928089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:42.928090Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:42.928097Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:42.928101Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:42.928101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:42.928104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:42.928238Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:42.929769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1296 sessionId: src_id|2659d239-c8a95b46-bce8e521-96a28ef2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:42.932177Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:42.991135Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342252678213679:5175]
2025-10-09T21:48:42.991497Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 1295, pipe: [1:7559342252678213679:5175]
2025-10-09T21:48:43.087009Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:43.087316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:43.087333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:43.087378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342252678213679:5175], now have 1 active actors on pipe
2025-10-09T21:48:43.087592Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:43.087702Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:43.087716Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342252678213679:5175]
2025-10-09T21:48:43.087722Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:43.087741Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:43.087822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1296 reason: error at write: Disconnected sessionId: src_id|2659d239-c8a95b46-bce8e521-96a28ef2_0
2025-10-09T21:48:43.088034Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1296 sessionId: src_id|2659d239-c8a95b46-bce8e521-96a28ef2_0 is DEAD
2025-10-09T21:48:43.088209Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2659d239-c8a95b46-bce8e521-96a28ef2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:43.088238Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2659d239-c8a95b46-bce8e521-96a28ef2_0] Write session will restart in 0.005794s
2025-10-09T21:48:43.088322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2659d239-c8a95b46-bce8e521-96a28ef2_0] Write session: Do CDS request
2025-10-09T21:48:43.088334Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2659d239-c8a95b46-bce8e521-96a28ef2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:43.095522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2659d239-c8a95b46-bce8e521-96a28ef2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:43.095634Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:43.095659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1297
2025-10-09T21:48:43.095845Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1297 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:43.095887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1297 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:43.095910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1297 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:43.095921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:43.096480Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1297 sessionId:  describe result for acl check
2025-10-09T21:48:43.096526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:43.096537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:43.096540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:43.096547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342256973181003:4356] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:43.097180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342256973181003:4356] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:43.099218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342256973181003:4356] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:43.099958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:43.101721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:43.101901Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:43.101915Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:43.101927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:43.101981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:43.101975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342256973181003:4356] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:43.101987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342256973181003:4356] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:43.101989Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342256973181003:4356] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:43.101996Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1297 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:43.102042Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342256611357552:9639]
2025-10-09T21:48:43.102663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:43.102970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:43.103014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:43.103337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:43.103383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.103389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.103391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.103392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.103393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.103394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.103396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.103397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.103398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.103403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.103414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.103418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.103422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.103426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.103431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.103434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.103438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.103441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.103448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.103455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.103461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.103468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.103474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.103479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.103482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.103487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.103494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.103498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.103502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.103512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.103517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.103521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.103525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.103530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.103535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.103544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.103573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:43.109922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:43.111096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:43.111115Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:43.111118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:43.111122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:43.111128Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1296 [2:7559342256611357552:9639]
2025-10-09T21:48:43.111142Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:43.111160Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:43.111168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:43.111171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:43.111173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:43.111189Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:43.111199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:43.111208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:43.111275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.111282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.111284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.111286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:43.111355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:43.111361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.111372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.111384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.111406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:43.111433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342256973180980:2432], now have 1 active actors on pipe
2025-10-09T21:48:43.111441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342256973181100:4356], now have 1 active actors on pipe
2025-10-09T21:48:43.111449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342256611357547:9639]
2025-10-09T21:48:43.111452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342256611357547:9639]
2025-10-09T21:48:43.111455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342256611357547:9639]
2025-10-09T21:48:43.111467Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:43.111487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:43.111498Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:43.111502Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:43.111465Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1296
2025-10-09T21:48:43.111502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:43.111504Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:43.111505Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:43.111484Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1296
2025-10-09T21:48:43.111543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:43.111553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:43.111571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342256611357547:9639]
2025-10-09T21:48:43.111583Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:43.111596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:43.111615Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ab772538-8d5624d2-f950dbdb-3ad7293_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:43.111627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:43.111631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:43.111634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:43.111649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:43.111663Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:43.111683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:43.111906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:43.111916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:43.111932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:43.112016Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1297 partition: 0 MaxSeqNo: 25 sessionId: src_id|ab772538-8d5624d2-f950dbdb-3ad7293_0
2025-10-09T21:48:43.112445Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2659d239-c8a95b46-bce8e521-96a28ef2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ab772538-8d5624d2-f950dbdb-3ad7293_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:43.114970Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ab772538-8d5624d2-f950dbdb-3ad7293_0] Write session: try to update token
2025-10-09T21:48:43.114988Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ab772538-8d5624d2-f950dbdb-3ad7293_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:43.126009Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:43.126046Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:43.126048Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:43.126050Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:43.126060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342256611357547:9639]
2025-10-09T21:48:43.126067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342256611357547:9639]
2025-10-09T21:48:43.126071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342256611357547:9639]
2025-10-09T21:48:43.126115Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:43.126117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:43.126124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.126128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:43.126129Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:43.126132Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:43.126260Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:43.128334Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1297 sessionId: src_id|ab772538-8d5624d2-f950dbdb-3ad7293_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:43.130767Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:43.187722Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342256973181137:5175]
2025-10-09T21:48:43.188012Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 1296, pipe: [1:7559342256973181137:5175]
2025-10-09T21:48:43.286140Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:43.286426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:43.286444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:43.286487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342256973181137:5175], now have 1 active actors on pipe
2025-10-09T21:48:43.286698Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:43.286709Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:43.286772Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:43.286859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1297 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ab772538-8d5624d2-f950dbdb-3ad7293_0
2025-10-09T21:48:43.287039Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1297 sessionId: src_id|ab772538-8d5624d2-f950dbdb-3ad7293_0 is DEAD
2025-10-09T21:48:43.287084Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342256973181137:5175]
2025-10-09T21:48:43.287183Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab772538-8d5624d2-f950dbdb-3ad7293_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:43.287194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab772538-8d5624d2-f950dbdb-3ad7293_0] Write session will restart in 0.007961s
2025-10-09T21:48:43.287222Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab772538-8d5624d2-f950dbdb-3ad7293_0] Write session: Do CDS request
2025-10-09T21:48:43.287229Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab772538-8d5624d2-f950dbdb-3ad7293_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:43.296510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ab772538-8d5624d2-f950dbdb-3ad7293_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:43.296603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:43.296620Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1298
2025-10-09T21:48:43.296741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1298 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:43.296786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1298 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:43.296798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1298 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:43.296801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:43.297229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1298 sessionId:  describe result for acl check
2025-10-09T21:48:43.297263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:43.297271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:43.297273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:43.297279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342256973181215:4334] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:43.297835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342256973181215:4334] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:43.299093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:43.300420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342256973181215:4334] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:43.301192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:43.301378Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:43.301388Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:43.301403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:43.301459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:43.301500Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342256611357683:9647]
2025-10-09T21:48:43.302088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:43.302453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:43.302504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:43.302833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:43.302898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.302904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.302905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.302906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.302907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.302909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.302909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.302910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.302911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.302917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.302927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.302930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.302934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.302937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.302940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.302944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.302947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.302951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.302958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.302966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.302973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.302978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.302982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.302987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.302991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.302995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.302999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.303003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.303008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.303017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.303022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.303027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.303033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.303037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.303047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.303052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.303081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:43.303085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342256973181215:4334] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:43.303102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342256973181215:4334] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:43.303106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342256973181215:4334] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:43.303115Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1298 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:43.309319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:43.310338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:43.310352Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:43.310354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:43.310358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:43.310365Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1297 [2:7559342256611357683:9647]
2025-10-09T21:48:43.310377Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:43.310393Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:43.310396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:43.310400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:43.310401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:43.310418Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:43.310425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:43.310434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:43.310489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.310498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.310500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.310503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:43.310565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:43.310575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.310588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.310600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.310618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:43.310640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342256973181144:2432], now have 1 active actors on pipe
2025-10-09T21:48:43.310643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342256973181267:4334], now have 1 active actors on pipe
2025-10-09T21:48:43.310650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342256611357678:9647]
2025-10-09T21:48:43.310653Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342256611357678:9647]
2025-10-09T21:48:43.310655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342256611357678:9647]
2025-10-09T21:48:43.310669Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:43.310684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:43.310684Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:43.310686Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:43.310689Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:43.310699Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:43.310703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:43.310729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342256611357678:9647]
2025-10-09T21:48:43.310740Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:43.310746Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:43.310742Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1297
2025-10-09T21:48:43.310749Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1297
2025-10-09T21:48:43.310848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:43.310862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:43.310889Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f1c13b07-992098b1-3caf65ea-2681d692_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:43.310904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:43.310908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:43.310910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:43.310922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:43.310934Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:43.310950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:43.311145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:43.311164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:43.311176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:43.311250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1298 partition: 0 MaxSeqNo: 25 sessionId: src_id|f1c13b07-992098b1-3caf65ea-2681d692_0
2025-10-09T21:48:43.311632Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab772538-8d5624d2-f950dbdb-3ad7293_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f1c13b07-992098b1-3caf65ea-2681d692_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:43.314013Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f1c13b07-992098b1-3caf65ea-2681d692_0] Write session: try to update token
2025-10-09T21:48:43.314045Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f1c13b07-992098b1-3caf65ea-2681d692_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:43.324947Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:43.324966Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:43.324968Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:43.324970Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:43.324981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342256611357678:9647]
2025-10-09T21:48:43.324985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342256611357678:9647]
2025-10-09T21:48:43.324988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342256611357678:9647]
2025-10-09T21:48:43.325031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:43.325032Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:43.325037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.325039Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:43.325040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:43.325043Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:43.325180Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:43.327104Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1298 sessionId: src_id|f1c13b07-992098b1-3caf65ea-2681d692_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:43.329443Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:43.387410Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342256973181277:5175]
2025-10-09T21:48:43.387656Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 1297, pipe: [1:7559342256973181277:5175]
2025-10-09T21:48:43.476803Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:43.477243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:43.477261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:43.477308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342256973181277:5175], now have 1 active actors on pipe
2025-10-09T21:48:43.477498Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:43.477542Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:43.477557Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:43.477603Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342256973181277:5175]
2025-10-09T21:48:43.477628Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:43.477630Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1298 reason: error at write: Disconnected sessionId: src_id|f1c13b07-992098b1-3caf65ea-2681d692_0
2025-10-09T21:48:43.477849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1298 sessionId: src_id|f1c13b07-992098b1-3caf65ea-2681d692_0 is DEAD
2025-10-09T21:48:43.478008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f1c13b07-992098b1-3caf65ea-2681d692_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:43.478020Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f1c13b07-992098b1-3caf65ea-2681d692_0] Write session will restart in 0.008541s
2025-10-09T21:48:43.478066Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f1c13b07-992098b1-3caf65ea-2681d692_0] Write session: Do CDS request
2025-10-09T21:48:43.478073Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f1c13b07-992098b1-3caf65ea-2681d692_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:43.487536Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f1c13b07-992098b1-3caf65ea-2681d692_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:43.487581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:43.487599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1299
2025-10-09T21:48:43.487735Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1299 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:43.487771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1299 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:43.487782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1299 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:43.487786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:43.488229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1299 sessionId:  describe result for acl check
2025-10-09T21:48:43.488264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:43.488272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:43.488274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:43.488280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342256973181365:4330] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:43.488524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:43.488803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342256973181365:4330] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:43.490237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:43.490417Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:43.490427Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:43.490441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:43.490503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:43.490542Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342256611357807:9649]
2025-10-09T21:48:43.490655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342256973181365:4330] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:43.491139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:43.491542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:43.491597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:43.491957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:43.492025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.492032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.492033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.492034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.492036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.492037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.492038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.492039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.492041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.492046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.492056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.492060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.492064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.492068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.492072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.492076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.492079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.492083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.492089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.492097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.492105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.492115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.492124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.492129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.492133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.492138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.492143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.492148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.492153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.492159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.492163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.492168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.492171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.492176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.492180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.492184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.492213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:43.493049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342256973181365:4330] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:43.493065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342256973181365:4330] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:43.493069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342256973181365:4330] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:43.493077Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1299 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:43.498233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:43.499255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:43.499271Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:43.499273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:43.499277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:43.499283Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1298 [2:7559342256611357807:9649]
2025-10-09T21:48:43.499295Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:43.499313Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:43.499315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:43.499318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:43.499321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:43.499336Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:43.499341Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:43.499349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:43.499405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.499411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.499413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.499415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:43.499478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:43.499488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.499498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.499507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.499521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:43.499543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342256973181279:2432], now have 1 active actors on pipe
2025-10-09T21:48:43.499554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342256973181405:4330], now have 1 active actors on pipe
2025-10-09T21:48:43.499563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342256611357801:9649]
2025-10-09T21:48:43.499565Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342256611357801:9649]
2025-10-09T21:48:43.499567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342256611357801:9649]
2025-10-09T21:48:43.499582Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:43.499602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:43.499603Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:43.499606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:43.499608Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:43.499619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:43.499621Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:43.499647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342256611357801:9649]
2025-10-09T21:48:43.499650Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:43.499657Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:43.499634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1298
2025-10-09T21:48:43.499634Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1298
2025-10-09T21:48:43.499806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:43.499818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:43.499852Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ffd49521-31c195b0-faf7dadd-f2065cea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:43.499868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:43.499872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:43.499874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:43.499884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:43.499893Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:43.499907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:43.500091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:43.500100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:43.500113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:43.500236Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1299 partition: 0 MaxSeqNo: 25 sessionId: src_id|ffd49521-31c195b0-faf7dadd-f2065cea_0
2025-10-09T21:48:43.500551Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f1c13b07-992098b1-3caf65ea-2681d692_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ffd49521-31c195b0-faf7dadd-f2065cea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:43.502974Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ffd49521-31c195b0-faf7dadd-f2065cea_0] Write session: try to update token
2025-10-09T21:48:43.502992Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ffd49521-31c195b0-faf7dadd-f2065cea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:43.513193Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:43.513208Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:43.513211Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:43.513213Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:43.513222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342256611357801:9649]
2025-10-09T21:48:43.513230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342256611357801:9649]
2025-10-09T21:48:43.513233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342256611357801:9649]
2025-10-09T21:48:43.513278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:43.513285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.513283Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:43.513290Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:43.513291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:43.513293Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:43.513467Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:43.516912Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1299 sessionId: src_id|ffd49521-31c195b0-faf7dadd-f2065cea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:43.519423Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:43.577885Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342256973181413:5175]
2025-10-09T21:48:43.578208Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 1298, pipe: [1:7559342256973181413:5175]
2025-10-09T21:48:43.664413Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:43.664690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:43.664705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:43.664752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342256973181413:5175], now have 1 active actors on pipe
2025-10-09T21:48:43.664949Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:43.664963Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:43.664966Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342256973181413:5175]
2025-10-09T21:48:43.664989Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:43.665027Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1299 reason: error at write: Disconnected sessionId: src_id|ffd49521-31c195b0-faf7dadd-f2065cea_0
2025-10-09T21:48:43.665148Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:43.665225Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1299 sessionId: src_id|ffd49521-31c195b0-faf7dadd-f2065cea_0 is DEAD
2025-10-09T21:48:43.665380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffd49521-31c195b0-faf7dadd-f2065cea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:43.665388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffd49521-31c195b0-faf7dadd-f2065cea_0] Write session will restart in 0.009915s
2025-10-09T21:48:43.665415Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffd49521-31c195b0-faf7dadd-f2065cea_0] Write session: Do CDS request
2025-10-09T21:48:43.665421Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffd49521-31c195b0-faf7dadd-f2065cea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:43.676140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:43.676520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ffd49521-31c195b0-faf7dadd-f2065cea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:43.676594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:43.676622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1300
2025-10-09T21:48:43.676742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1300 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:43.676777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1300 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:43.676787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1300 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:43.676791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:43.677182Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1300 sessionId:  describe result for acl check
2025-10-09T21:48:43.677226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:43.677233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:43.677235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:43.677240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342256973181505:4326] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:43.677854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342256973181505:4326] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:43.677856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:43.678041Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:43.678050Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:43.678062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:43.678114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:43.678153Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342256611357936:9653]
2025-10-09T21:48:43.678766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:43.679157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:43.679206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:43.679513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:43.679557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.679564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.679566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.679566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.679568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.679569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.679570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.679571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.679571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.679576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.679584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.679588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.679592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.679595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.679599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.679603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.679607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.679611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.679617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.679624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.679630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.679635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.679640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.679646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.679650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.679655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.679659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.679664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.679668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.679673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.679677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.679682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.679686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.679691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.679695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.679700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.679726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:43.680066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342256973181505:4326] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:43.685150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342256973181505:4326] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:43.685164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342256973181505:4326] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:43.685168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342256973181505:4326] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:43.685175Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1300 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:43.685443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:43.686725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:43.686737Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:43.686738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:43.686741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:43.686745Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1299 [2:7559342256611357936:9653]
2025-10-09T21:48:43.686754Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:43.686767Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:43.686769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:43.686771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:43.686772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:43.686787Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:43.686797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:43.686804Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:43.686854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.686862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.686864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.686866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:43.686963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:43.686969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.686978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.686984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.686994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:43.687052Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1299
2025-10-09T21:48:43.687059Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1299
2025-10-09T21:48:43.687009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342256973181543:4326], now have 1 active actors on pipe
2025-10-09T21:48:43.687016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342256973181415:2432], now have 1 active actors on pipe
2025-10-09T21:48:43.687021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342256611357930:9653]
2025-10-09T21:48:43.687023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342256611357930:9653]
2025-10-09T21:48:43.687024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342256611357930:9653]
2025-10-09T21:48:43.687033Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:43.687043Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:43.687048Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:43.687051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:43.687051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:43.687053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:43.687054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:43.687070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342256611357930:9653]
2025-10-09T21:48:43.687072Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:43.687078Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:43.687104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:43.687107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:43.687127Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cab829ac-8cce7728-df2def16-b422602f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:43.687133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:43.687136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:43.687137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:43.687144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:43.687152Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:43.687164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:43.687255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:43.687261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:43.687271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:43.687341Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1300 partition: 0 MaxSeqNo: 25 sessionId: src_id|cab829ac-8cce7728-df2def16-b422602f_0
2025-10-09T21:48:43.687646Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffd49521-31c195b0-faf7dadd-f2065cea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cab829ac-8cce7728-df2def16-b422602f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:43.690081Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cab829ac-8cce7728-df2def16-b422602f_0] Write session: try to update token
2025-10-09T21:48:43.690098Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cab829ac-8cce7728-df2def16-b422602f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:43.702878Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1300 sessionId: src_id|cab829ac-8cce7728-df2def16-b422602f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:43.703512Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:43.703530Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:43.703540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342256611357930:9653]
2025-10-09T21:48:43.703590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:43.703597Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:43.703756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:43.703897Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:43.704029Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:43.704159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:43.704399Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:43.704443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:43.705176Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:43.705353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:43.705368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:43.705400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:43.705523Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:43.711070Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:43.711095Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:43.711098Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:43.711100Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:43.711111Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342256611357930:9653]
2025-10-09T21:48:43.711115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342256611357930:9653]
2025-10-09T21:48:43.711118Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342256611357930:9653]
2025-10-09T21:48:43.711156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:43.711160Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:43.711167Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:43.711169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.711171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:43.711173Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:43.711303Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:43.765265Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342256973181550:5175]
2025-10-09T21:48:43.765536Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 1299, pipe: [1:7559342256973181550:5175]
2025-10-09T21:48:43.864364Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:43.871716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:43.871743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:43.871900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:43.871960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:43.872020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:43.872082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:43.872124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:43.872133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:43.872201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:43.872262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:43.872330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:43.872393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:43.872434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:43.872443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:43.872498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:43.872555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:43.872615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:43.872671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:43.872718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:43.872726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:43.872784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:43.872843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:43.872895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:43.872992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:43.873034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:43.873040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:43.873105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:43.873177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:43.873267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:43.873326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:43.873367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:43.873376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:43.873453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:43.873515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:43.873577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:43.873633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:43.873699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:43.873706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:43.873767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:43.873828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:43.873892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:43.873953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:43.873992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:43.874003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:43.874079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:43.874133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:43.874191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:43.874248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:43.874292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:43.874299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:43.874356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:43.874412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:43.874473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:43.874530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:43.874586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:43.874593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:43.874652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:43.874706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:43.874765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:43.874823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:43.874863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:43.874869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:43.874925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342256973181550:5175], now have 1 active actors on pipe
2025-10-09T21:48:43.875136Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:43.875139Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:43.875152Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:43.875184Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342256973181550:5175]
2025-10-09T21:48:43.875190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1300 reason: error at write: Disconnected sessionId: src_id|cab829ac-8cce7728-df2def16-b422602f_0
2025-10-09T21:48:43.875381Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1300 sessionId: src_id|cab829ac-8cce7728-df2def16-b422602f_0 is DEAD
2025-10-09T21:48:43.875560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cab829ac-8cce7728-df2def16-b422602f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:43.875572Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cab829ac-8cce7728-df2def16-b422602f_0] Write session will restart in 0.008206s
2025-10-09T21:48:43.875598Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cab829ac-8cce7728-df2def16-b422602f_0] Write session: Do CDS request
2025-10-09T21:48:43.875604Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cab829ac-8cce7728-df2def16-b422602f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:43.875857Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:43.884582Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cab829ac-8cce7728-df2def16-b422602f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:43.884683Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:43.884698Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1301
2025-10-09T21:48:43.884922Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1301 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:43.884959Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1301 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:43.884969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1301 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:43.884973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:43.885753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1301 sessionId:  describe result for acl check
2025-10-09T21:48:43.885797Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:43.885804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:43.885807Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:43.885813Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342256973181641:2334] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:43.886380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342256973181641:2334] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:43.887693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:43.889147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342256973181641:2334] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:43.889602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:43.889794Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:43.889806Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:43.889818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:43.889876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:43.889917Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342256611358063:9655]
2025-10-09T21:48:43.890513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:43.890801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:43.890845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:43.891154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:43.891207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.891214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.891216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.891217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.891218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.891219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.891220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.891221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.891222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.891226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.891237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.891242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.891246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.891251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.891255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.891259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.891263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.891267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.891272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.891280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:43.891288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.891295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:43.891298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.891303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:43.891307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.891311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:43.891315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.891321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:43.891325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.891331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:43.891336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.891341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.891346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.891350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.891361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.891367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.891397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:43.891928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342256973181641:2334] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:43.891941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342256973181641:2334] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:43.891945Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342256973181641:2334] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:43.891953Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1301 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:43.897763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:43.898944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:43.898961Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:43.898963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:43.898967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:43.898973Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1300 [2:7559342256611358063:9655]
2025-10-09T21:48:43.898986Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:43.899004Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:43.899012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:43.899015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:43.899017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:43.899034Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:43.899045Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:43.899052Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:43.899107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.899115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.899117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.899120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:43.899187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:43.899196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.899209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:43.899227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:43.899250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:43.899277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342256973181555:2432], now have 1 active actors on pipe
2025-10-09T21:48:43.899285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342256973181683:2334], now have 1 active actors on pipe
2025-10-09T21:48:43.899293Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342256611358058:9655]
2025-10-09T21:48:43.899296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342256611358058:9655]
2025-10-09T21:48:43.899298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342256611358058:9655]
2025-10-09T21:48:43.899312Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:43.899326Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:43.899331Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:43.899333Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:43.899335Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:43.899348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:43.899352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:43.899357Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1300
2025-10-09T21:48:43.899358Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1300
2025-10-09T21:48:43.899386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342256611358058:9655]
2025-10-09T21:48:43.899394Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:43.899410Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:43.899426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:43.899431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:43.899470Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8b79d42d-c2fa4060-2c837c8-eba99ebe_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:43.899483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:43.899487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:43.899490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:43.899500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:43.899517Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:43.899533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:43.899659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:43.899667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:43.899681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:43.899758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1301 partition: 0 MaxSeqNo: 25 sessionId: src_id|8b79d42d-c2fa4060-2c837c8-eba99ebe_0
2025-10-09T21:48:43.900090Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cab829ac-8cce7728-df2def16-b422602f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8b79d42d-c2fa4060-2c837c8-eba99ebe_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:43.902502Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b79d42d-c2fa4060-2c837c8-eba99ebe_0] Write session: try to update token
2025-10-09T21:48:43.902521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b79d42d-c2fa4060-2c837c8-eba99ebe_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:43.913585Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:43.913604Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:43.913606Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:43.913609Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:43.913619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342256611358058:9655]
2025-10-09T21:48:43.913624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342256611358058:9655]
2025-10-09T21:48:43.913627Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342256611358058:9655]
2025-10-09T21:48:43.913670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:43.913672Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:43.913689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:43.913691Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:43.913692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:43.913695Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:43.913869Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:43.915250Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1301 sessionId: src_id|8b79d42d-c2fa4060-2c837c8-eba99ebe_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:43.917492Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:43.975186Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342256973181690:5175]
2025-10-09T21:48:43.975563Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 1300, pipe: [1:7559342256973181690:5175]
2025-10-09T21:48:44.074791Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:44.075177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.075196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.075245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342256973181690:5175], now have 1 active actors on pipe
2025-10-09T21:48:44.075510Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:44.075533Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:44.075543Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:44.075548Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:44.075610Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342256973181690:5175]
2025-10-09T21:48:44.075616Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1301 reason: error at write: Disconnected sessionId: src_id|8b79d42d-c2fa4060-2c837c8-eba99ebe_0
2025-10-09T21:48:44.075626Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:48:44.075767Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 13 consumer shared/user session shared/user_1_13_13907029840570446113_v1 is DEAD
2025-10-09T21:48:44.075800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1301 sessionId: src_id|8b79d42d-c2fa4060-2c837c8-eba99ebe_0 is DEAD
2025-10-09T21:48:44.075938Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b79d42d-c2fa4060-2c837c8-eba99ebe_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:44.075950Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b79d42d-c2fa4060-2c837c8-eba99ebe_0] Write session will restart in 0.007913s
2025-10-09T21:48:44.075974Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b79d42d-c2fa4060-2c837c8-eba99ebe_0] Write session: Do CDS request
2025-10-09T21:48:44.075981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b79d42d-c2fa4060-2c837c8-eba99ebe_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:44.075993Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342175368786361:5182] disconnected.
2025-10-09T21:48:44.076006Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342175368786361:5182] disconnected; active server actors: 1
2025-10-09T21:48:44.076010Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342175368786361:5182] client user disconnected session shared/user_1_13_13907029840570446113_v1
2025-10-09T21:48:44.076041Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:48:44.076182Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:48:44.076188Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:48:44.076195Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.009193s
2025-10-09T21:48:44.085624Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b79d42d-c2fa4060-2c837c8-eba99ebe_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:44.085743Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:44.085773Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1302
2025-10-09T21:48:44.085915Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1302 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:44.085950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1302 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:44.085961Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1302 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:44.085971Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:44.086279Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:48:44.086429Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:48:44.086445Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 14
2025-10-09T21:48:44.086555Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1302 sessionId:  describe result for acl check
2025-10-09T21:48:44.086584Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 14 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:48:44.086595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:44.086598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:44.086599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:44.086605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342261268149046:4304] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:44.086637Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:48:44.086737Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 auth for : user
2025-10-09T21:48:44.086880Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 Handle describe topics response
2025-10-09T21:48:44.086909Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 auth is DEAD
2025-10-09T21:48:44.086932Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:48:44.087147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342261268149046:4304] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:44.087189Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:48:44.087279Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_14_17138514154924300407_v1
2025-10-09T21:48:44.087358Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342261268149068:4306] connected; active server actors: 1
2025-10-09T21:48:44.087377Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559342261268149068:4306] session shared/user_1_14_17138514154924300407_v1
2025-10-09T21:48:44.087384Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:48:44.087393Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:48:44.087399Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_14_17138514154924300407_v1" (Sender=[1:7559342261268149042:4306], Pipe=[1:7559342261268149068:4306], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:48:44.087409Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:48:44.087423Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:48:44.087437Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_14_17138514154924300407_v1" (Sender=[1:7559342261268149042:4306], Pipe=[1:7559342261268149068:4306], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:48:44.087454Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_14_17138514154924300407_v1" sender [1:7559342261268149042:4306] lock partition 0 for ReadingSession "shared/user_1_14_17138514154924300407_v1" (Sender=[1:7559342261268149042:4306], Pipe=[1:7559342261268149068:4306], Partitions=[], ActiveFamilyCount=1) generation 1 step 14
2025-10-09T21:48:44.087467Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:48:44.087476Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000045s
2025-10-09T21:48:44.087711Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 14 Session: "shared/user_1_14_17138514154924300407_v1" ClientId: "user" PipeClient { RawX1: 7559342261268149068 RawX2: 4503603922342098 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:48:44.087742Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:48:44.089256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342261268149046:4304] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:44.089512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:44.092215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:44.092441Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:44.092451Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:44.092464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:44.092527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:44.092577Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342260906325487:9657]
2025-10-09T21:48:44.092636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342261268149046:4304] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:44.092651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342261268149046:4304] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:44.092655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342261268149046:4304] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:44.092663Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1302 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:44.093175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:44.093581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:44.093632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:44.093962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:44.094005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.094010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.094010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.094011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.094012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.094013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.094014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.094014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.094015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.094019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.094026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.094040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.094042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.094045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.094047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.094050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.094051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.094053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.094059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.094065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.094069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.094073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.094076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.094079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.094081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.094085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.094088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.094091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.094093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.094096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.094100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.094103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.094106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.094109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.094112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.094115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.094132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:44.101310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:44.102432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:44.102445Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:44.102448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:44.102451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:44.102456Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1301 [2:7559342260906325487:9657]
2025-10-09T21:48:44.102467Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:44.102480Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:44.102483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:44.102485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:44.102486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:44.102499Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:44.102503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:44.102508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:44.102554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.102561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.102564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.102567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:44.102629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:44.102635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.102643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.102650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.102667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:44.102691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342260906325482:9657]
2025-10-09T21:48:44.102697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342260906325482:9657]
2025-10-09T21:48:44.102699Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342260906325482:9657]
2025-10-09T21:48:44.102706Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:44.102721Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:44.102722Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:44.102727Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:44.102730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:44.102739Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:44.102743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:44.102755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342261268148991:2432], now have 1 active actors on pipe
2025-10-09T21:48:44.102764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342261268149122:4304], now have 1 active actors on pipe
2025-10-09T21:48:44.102767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342261268149091:4303], now have 1 active actors on pipe
2025-10-09T21:48:44.102801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342260906325482:9657]
2025-10-09T21:48:44.102816Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:44.102835Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:44.102806Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1301
2025-10-09T21:48:44.102852Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1301
2025-10-09T21:48:44.102879Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1301, pipe: [1:7559342261268149091:4303]
2025-10-09T21:48:44.102939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.102953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.102975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.102983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.103000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_14_17138514154924300407_v1 on pipe: [1:7559342261268149091:4303]
2025-10-09T21:48:44.103030Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b10dc9b1-dbce246d-9cd4bd26-c074d1bc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:44.103053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:44.103062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:44.103065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:44.103077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:44.103094Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:44.103107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:44.103112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:44.103113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:44.103129Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_14_17138514154924300407_v1
2025-10-09T21:48:44.103133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:44.103162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:44.103187Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:48:44.103336Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_14_17138514154924300407_v1:1 with generation 1301
2025-10-09T21:48:44.103375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.103383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.103400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:44.103526Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1302 partition: 0 MaxSeqNo: 25 sessionId: src_id|b10dc9b1-dbce246d-9cd4bd26-c074d1bc_0
2025-10-09T21:48:44.103906Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b79d42d-c2fa4060-2c837c8-eba99ebe_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b10dc9b1-dbce246d-9cd4bd26-c074d1bc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:44.107016Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b10dc9b1-dbce246d-9cd4bd26-c074d1bc_0] Write session: try to update token
2025-10-09T21:48:44.107041Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b10dc9b1-dbce246d-9cd4bd26-c074d1bc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:44.107840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:48:44.107871Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:48:44.107880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:44.107883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:44.107885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:44.107894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:48:44.107908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:48:44.108090Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046524103 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:48:44.108124Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:48:44.108160Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 sending to client partition status
2025-10-09T21:48:44.117736Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:44.117758Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:44.117761Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:44.117762Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:44.117772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342260906325482:9657]
2025-10-09T21:48:44.117777Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342260906325482:9657]
2025-10-09T21:48:44.117783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342260906325482:9657]
2025-10-09T21:48:44.117832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:44.117844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.117848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:44.118010Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:44.121512Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1302 sessionId: src_id|b10dc9b1-dbce246d-9cd4bd26-c074d1bc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:44.122090Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:44.122119Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:44.122123Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:44.124541Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:44.276224Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:44.276764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.276781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.277052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_14_17138514154924300407_v1
2025-10-09T21:48:44.277075Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:44.277117Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_14_17138514154924300407_v1
2025-10-09T21:48:44.277088Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:44.277095Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:44.277107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:44.277161Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342261268149091:4303]
2025-10-09T21:48:44.277217Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1302 reason: error at write: Disconnected sessionId: src_id|b10dc9b1-dbce246d-9cd4bd26-c074d1bc_0
2025-10-09T21:48:44.277407Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1302 sessionId: src_id|b10dc9b1-dbce246d-9cd4bd26-c074d1bc_0 is DEAD
2025-10-09T21:48:44.277564Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b10dc9b1-dbce246d-9cd4bd26-c074d1bc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:44.277579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b10dc9b1-dbce246d-9cd4bd26-c074d1bc_0] Write session will restart in 0.007002s
2025-10-09T21:48:44.277606Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b10dc9b1-dbce246d-9cd4bd26-c074d1bc_0] Write session: Do CDS request
2025-10-09T21:48:44.277612Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b10dc9b1-dbce246d-9cd4bd26-c074d1bc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:44.285564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b10dc9b1-dbce246d-9cd4bd26-c074d1bc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:44.285605Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:44.285621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1303
2025-10-09T21:48:44.285864Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1303 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:44.285908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1303 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:44.285921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1303 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:44.285926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:44.286432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1303 sessionId:  describe result for acl check
2025-10-09T21:48:44.286472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:44.286479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:44.286483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:44.286489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342261268149209:4289] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:44.287027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342261268149209:4289] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:44.289137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342261268149209:4289] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:44.289437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:44.291491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:44.291667Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:44.291676Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:44.291688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:44.291742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:44.291780Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342260906325655:9671]
2025-10-09T21:48:44.291823Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342261268149209:4289] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:44.291837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342261268149209:4289] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:44.291842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342261268149209:4289] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:44.291850Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1303 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:44.292364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:44.292703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:44.292759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:44.293090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:44.293142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.293148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.293149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.293150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.293151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.293153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.293153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.293154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.293155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.293160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.293171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.293175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.293179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.293183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.293187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.293191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.293194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.293197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.293204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.293210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.293216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.293221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.293225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.293230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.293235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.293240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.293244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.293249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.293253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.293258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.293263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.293268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.293273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.293278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.293294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.293304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.293336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:44.299661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:44.300847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:44.300865Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:44.300868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:44.300872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:44.300879Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1302 [2:7559342260906325655:9671]
2025-10-09T21:48:44.300895Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:44.300917Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:44.300924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:44.300927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:44.300930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:44.300946Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:44.300957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:44.300964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:44.301016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.301022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.301024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.301027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:44.301099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:44.301107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.301119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.301132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.301148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:44.301166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342261268149169:2432], now have 1 active actors on pipe
2025-10-09T21:48:44.301175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342261268149292:4289], now have 1 active actors on pipe
2025-10-09T21:48:44.301182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342260906325650:9671]
2025-10-09T21:48:44.301185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342260906325650:9671]
2025-10-09T21:48:44.301187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342260906325650:9671]
2025-10-09T21:48:44.301210Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1302
2025-10-09T21:48:44.301217Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1302
2025-10-09T21:48:44.301212Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:44.301227Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:44.301230Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:44.301232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:44.301233Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:44.301246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:44.301249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:44.301282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342260906325650:9671]
2025-10-09T21:48:44.301294Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:44.301302Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:44.301313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.301317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.301362Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|954876b9-83b8c0e1-2264b758-e58cc5f5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:44.301373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:44.301378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:44.301385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:44.301395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:44.301409Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:44.301426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:44.301586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.301600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.301624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:44.301704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1303 partition: 0 MaxSeqNo: 25 sessionId: src_id|954876b9-83b8c0e1-2264b758-e58cc5f5_0
2025-10-09T21:48:44.302063Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b10dc9b1-dbce246d-9cd4bd26-c074d1bc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|954876b9-83b8c0e1-2264b758-e58cc5f5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:44.304591Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|954876b9-83b8c0e1-2264b758-e58cc5f5_0] Write session: try to update token
2025-10-09T21:48:44.304607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|954876b9-83b8c0e1-2264b758-e58cc5f5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:44.315583Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:44.315604Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:44.315606Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:44.315608Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:44.315617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342260906325650:9671]
2025-10-09T21:48:44.315622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342260906325650:9671]
2025-10-09T21:48:44.315625Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342260906325650:9671]
2025-10-09T21:48:44.315673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:44.315680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.315682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:44.315693Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:44.315699Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:44.315702Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:44.315815Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:44.317562Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1303 sessionId: src_id|954876b9-83b8c0e1-2264b758-e58cc5f5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:44.319810Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:44.377095Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342261268149304:4303]
2025-10-09T21:48:44.377389Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 1302, pipe: [1:7559342261268149304:4303]
2025-10-09T21:48:44.472860Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:44.473089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.473106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.473156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342261268149304:4303], now have 1 active actors on pipe
2025-10-09T21:48:44.473403Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:44.473404Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342261268149304:4303]
2025-10-09T21:48:44.473431Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:44.473434Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:44.473520Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1303 reason: error at write: Disconnected sessionId: src_id|954876b9-83b8c0e1-2264b758-e58cc5f5_0
2025-10-09T21:48:44.473590Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:44.473713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1303 sessionId: src_id|954876b9-83b8c0e1-2264b758-e58cc5f5_0 is DEAD
2025-10-09T21:48:44.473859Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|954876b9-83b8c0e1-2264b758-e58cc5f5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:44.473869Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|954876b9-83b8c0e1-2264b758-e58cc5f5_0] Write session will restart in 0.009941s
2025-10-09T21:48:44.473902Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|954876b9-83b8c0e1-2264b758-e58cc5f5_0] Write session: Do CDS request
2025-10-09T21:48:44.473907Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|954876b9-83b8c0e1-2264b758-e58cc5f5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:44.484542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|954876b9-83b8c0e1-2264b758-e58cc5f5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:44.484593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:44.484609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1304
2025-10-09T21:48:44.484731Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1304 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:44.484771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1304 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:44.484782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1304 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:44.484785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:44.485276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1304 sessionId:  describe result for acl check
2025-10-09T21:48:44.485317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:44.485320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:44.485322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:44.485328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342261268149398:4282] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:44.485840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:44.485980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342261268149398:4282] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:44.487771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:44.487966Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:44.487983Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:44.487993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:44.488043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:44.488081Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342260906325784:9673]
2025-10-09T21:48:44.488187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342261268149398:4282] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:44.488594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:44.488904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:44.488953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:44.489288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:44.489338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.489347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.489348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.489349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.489351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.489352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.489353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.489354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.489355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.489360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.489370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.489374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.489378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.489382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.489387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.489391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.489395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.489399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.489407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.489415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.489420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.489426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.489436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.489442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.489446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.489452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.489457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.489462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.489467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.489473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.489478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.489483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.489488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.489494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.489499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.489504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.489534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:44.494829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342261268149398:4282] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:44.494847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342261268149398:4282] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:44.494851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342261268149398:4282] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:44.494859Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1304 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:44.496226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:44.497257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:44.497270Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:44.497272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:44.497276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:44.497281Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1303 [2:7559342260906325784:9673]
2025-10-09T21:48:44.497293Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:44.497308Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:44.497310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:44.497312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:44.497313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:44.497328Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:44.497332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:44.497338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:44.497386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.497394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.497395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.497397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:44.497458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:44.497466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.497478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.497491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.497502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:44.497522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342261268149437:4282], now have 1 active actors on pipe
2025-10-09T21:48:44.497528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342261268149306:2432], now have 1 active actors on pipe
2025-10-09T21:48:44.497535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342260906325779:9673]
2025-10-09T21:48:44.497537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342260906325779:9673]
2025-10-09T21:48:44.497538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342260906325779:9673]
2025-10-09T21:48:44.497551Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:44.497567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:44.497574Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:44.497579Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:44.497584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:44.497588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:44.497562Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1303
2025-10-09T21:48:44.497591Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:44.497576Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1303
2025-10-09T21:48:44.497626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342260906325779:9673]
2025-10-09T21:48:44.497635Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:44.497638Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:44.497663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.497683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.497721Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ca55c437-c4cf9dff-35229a11-35af899d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:44.497734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:44.497738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:44.497742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:44.497751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:44.497766Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:44.497789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:44.497929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.497938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.497952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:44.498059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1304 partition: 0 MaxSeqNo: 25 sessionId: src_id|ca55c437-c4cf9dff-35229a11-35af899d_0
2025-10-09T21:48:44.498380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|954876b9-83b8c0e1-2264b758-e58cc5f5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ca55c437-c4cf9dff-35229a11-35af899d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:44.500828Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca55c437-c4cf9dff-35229a11-35af899d_0] Write session: try to update token
2025-10-09T21:48:44.500846Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca55c437-c4cf9dff-35229a11-35af899d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:44.513100Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:44.513114Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:44.513116Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:44.513118Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:44.513125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342260906325779:9673]
2025-10-09T21:48:44.513130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342260906325779:9673]
2025-10-09T21:48:44.513134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342260906325779:9673]
2025-10-09T21:48:44.513181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:44.513189Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:44.513194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.513197Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:44.513197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:44.513201Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:44.513333Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:44.514416Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1304 sessionId: src_id|ca55c437-c4cf9dff-35229a11-35af899d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:44.516691Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:44.573724Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342261268149445:4303]
2025-10-09T21:48:44.574007Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 1303, pipe: [1:7559342261268149445:4303]
2025-10-09T21:48:44.669366Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:44.669741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.669757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.669804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342261268149445:4303], now have 1 active actors on pipe
2025-10-09T21:48:44.670002Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:44.670165Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342261268149445:4303]
2025-10-09T21:48:44.670185Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:44.670190Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:44.670200Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:44.670268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1304 reason: error at write: Disconnected sessionId: src_id|ca55c437-c4cf9dff-35229a11-35af899d_0
2025-10-09T21:48:44.670494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1304 sessionId: src_id|ca55c437-c4cf9dff-35229a11-35af899d_0 is DEAD
2025-10-09T21:48:44.670714Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca55c437-c4cf9dff-35229a11-35af899d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:44.670727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca55c437-c4cf9dff-35229a11-35af899d_0] Write session will restart in 0.009055s
2025-10-09T21:48:44.670764Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca55c437-c4cf9dff-35229a11-35af899d_0] Write session: Do CDS request
2025-10-09T21:48:44.670770Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca55c437-c4cf9dff-35229a11-35af899d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:44.680561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca55c437-c4cf9dff-35229a11-35af899d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:44.680584Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:44.680611Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1305
2025-10-09T21:48:44.680794Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1305 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:44.680831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1305 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:44.680842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1305 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:44.680846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:44.681294Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1305 sessionId:  describe result for acl check
2025-10-09T21:48:44.681332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:44.681340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:44.681342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:44.681348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342261268149521:4276] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:44.681949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342261268149521:4276] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:44.683387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:44.684331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342261268149521:4276] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:44.685638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:44.685850Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:44.685860Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:44.685872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:44.685926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:44.685969Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342260906325918:9677]
2025-10-09T21:48:44.686595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:44.686920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:44.686974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:44.687170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342261268149521:4276] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:44.687188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342261268149521:4276] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:44.687192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342261268149521:4276] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:44.687204Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1305 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:44.687311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:44.687385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.687391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.687392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.687393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.687394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.687395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.687396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.687397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.687398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.687404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.687415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.687419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.687424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.687428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.687432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.687436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.687440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.687444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.687452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.687471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.687482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.687488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.687493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.687498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.687502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.687508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.687512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.687518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.687522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.687532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.687537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.687542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.687548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.687555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.687559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.687564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.687594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:44.694324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:44.695358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:44.695371Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:44.695373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:44.695377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:44.695383Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1304 [2:7559342260906325918:9677]
2025-10-09T21:48:44.695412Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:44.695433Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:44.695440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:44.695443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:44.695446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:44.695462Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:44.695470Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:44.695476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:44.695528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.695535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.695537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.695540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:44.695617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:44.695625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.695636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.695649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.695667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:44.695692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342261268149447:2432], now have 1 active actors on pipe
2025-10-09T21:48:44.695700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342261268149577:4276], now have 1 active actors on pipe
2025-10-09T21:48:44.695707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342260906325913:9677]
2025-10-09T21:48:44.695709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342260906325913:9677]
2025-10-09T21:48:44.695710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342260906325913:9677]
2025-10-09T21:48:44.695723Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:44.695737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:44.695747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:44.695750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:44.695753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:44.695757Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:44.695760Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:44.695770Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342260906325913:9677]
2025-10-09T21:48:44.695787Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:44.695798Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:44.695763Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1304
2025-10-09T21:48:44.695813Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1304
2025-10-09T21:48:44.695892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.695901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.695926Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e07ad82b-f0adcc2c-732b59b6-11925753_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:44.695938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:44.695942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:44.695946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:44.695956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:44.695968Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:44.695986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:44.696145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.696153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.696167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:44.696273Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1305 partition: 0 MaxSeqNo: 25 sessionId: src_id|e07ad82b-f0adcc2c-732b59b6-11925753_0
2025-10-09T21:48:44.696585Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca55c437-c4cf9dff-35229a11-35af899d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e07ad82b-f0adcc2c-732b59b6-11925753_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:44.698873Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e07ad82b-f0adcc2c-732b59b6-11925753_0] Write session: try to update token
2025-10-09T21:48:44.698891Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e07ad82b-f0adcc2c-732b59b6-11925753_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:44.708430Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:44.708443Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:44.708451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342260906325913:9677]
2025-10-09T21:48:44.708489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:44.708495Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:44.708649Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:44.708777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:44.708900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:44.709035Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:44.709269Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:44.709309Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:44.711642Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1305 sessionId: src_id|e07ad82b-f0adcc2c-732b59b6-11925753_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:44.713960Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:44.714384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.714404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.714453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:44.714522Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:44.722096Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:44.722113Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:44.722116Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:44.722119Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:44.722129Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342260906325913:9677]
2025-10-09T21:48:44.722133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342260906325913:9677]
2025-10-09T21:48:44.722137Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342260906325913:9677]
2025-10-09T21:48:44.722179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:44.722190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.722190Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:44.722193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:44.722196Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:44.722203Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:44.722313Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:44.770312Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342261268149588:4303]
2025-10-09T21:48:44.770597Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 1304, pipe: [1:7559342261268149588:4303]
2025-10-09T21:48:44.883089Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:44.889859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.889887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.890060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:44.890116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:44.890176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:44.890244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:44.890286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:44.890295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:44.890356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:44.890410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:44.890471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:44.890532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:44.890573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:44.890582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:44.890649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:44.890710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:44.890762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:44.890820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:44.890861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:44.890870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:44.890934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:44.890992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:44.891048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:44.891107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:44.891149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:44.891156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:44.891216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:44.891269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:44.891323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:44.891385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:44.891427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:44.891436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:44.891495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:44.891554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:44.891613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:44.891670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:44.891716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:44.891723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:44.891779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:44.891833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:44.891904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:44.891976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:44.892029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:44.892036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:44.892092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:44.892167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:44.892223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:44.892280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:44.892326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:44.892332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:44.892390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:44.892448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:44.892506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:44.892561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:44.892604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:44.892612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:44.892671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:44.892728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:44.892784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:44.892847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:44.892907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:44.892914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:44.892960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342261268149588:4303], now have 1 active actors on pipe
2025-10-09T21:48:44.893221Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342261268149588:4303]
2025-10-09T21:48:44.893271Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:44.893472Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:44.893483Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:44.893510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1305 reason: error at write: Disconnected sessionId: src_id|e07ad82b-f0adcc2c-732b59b6-11925753_0
2025-10-09T21:48:44.893690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1305 sessionId: src_id|e07ad82b-f0adcc2c-732b59b6-11925753_0 is DEAD
2025-10-09T21:48:44.893845Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e07ad82b-f0adcc2c-732b59b6-11925753_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:44.893853Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e07ad82b-f0adcc2c-732b59b6-11925753_0] Write session will restart in 0.007497s
2025-10-09T21:48:44.893878Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e07ad82b-f0adcc2c-732b59b6-11925753_0] Write session: Do CDS request
2025-10-09T21:48:44.893884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e07ad82b-f0adcc2c-732b59b6-11925753_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:44.893955Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:44.902528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e07ad82b-f0adcc2c-732b59b6-11925753_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:44.902629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:44.902648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1306
2025-10-09T21:48:44.902808Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1306 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:44.902854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1306 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:44.902867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1306 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:44.902871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:44.903410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1306 sessionId:  describe result for acl check
2025-10-09T21:48:44.903449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:44.903457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:44.903459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:44.903466Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342261268149656:4264] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:44.904031Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342261268149656:4264] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:44.905978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:44.906434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342261268149656:4264] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:44.907829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:44.908028Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:44.908039Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:44.908049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:44.908106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:44.908147Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342260906326051:9679]
2025-10-09T21:48:44.908673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:44.908947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:44.909006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:44.909211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342261268149656:4264] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:44.909225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342261268149656:4264] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:44.909228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342261268149656:4264] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:44.909237Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1306 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:44.909314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:44.909362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.909370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.909371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.909372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.909373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.909375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.909376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.909378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.909379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.909384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.909395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.909399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.909403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.909407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.909410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.909415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.909419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.909423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.909429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.909437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:44.909443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.909449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:44.909453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.909458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:44.909463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.909468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:44.909473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.909478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:44.909483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.909488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:44.909493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.909498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.909502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.909507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.909512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.909516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.909544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:44.916113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:44.917158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:44.917177Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:44.917180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:44.917185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:44.917192Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1305 [2:7559342260906326051:9679]
2025-10-09T21:48:44.917207Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:44.917226Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:44.917234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:44.917238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:44.917241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:44.917257Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:44.917271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:44.917279Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:44.917340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.917348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.917350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.917353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:44.917421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:44.917428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.917440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:44.917457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:44.917479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:44.917568Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1305
2025-10-09T21:48:44.917588Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1305
2025-10-09T21:48:44.917510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342261268149730:4264], now have 1 active actors on pipe
2025-10-09T21:48:44.917517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342261268149597:2432], now have 1 active actors on pipe
2025-10-09T21:48:44.917525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342260906326046:9679]
2025-10-09T21:48:44.917528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342260906326046:9679]
2025-10-09T21:48:44.917530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342260906326046:9679]
2025-10-09T21:48:44.917544Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:44.917558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:44.917566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:44.917570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:44.917574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:44.917574Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:44.917577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:44.917613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342260906326046:9679]
2025-10-09T21:48:44.917616Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:44.917626Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:44.917651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.917656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.917702Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ea5a8f23-baf00dae-8cb978b5-a18448ea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:44.917718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:44.917722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:44.917724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:44.917736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:44.917746Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:44.917761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:44.917941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.917949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.917963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:44.918167Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1306 partition: 0 MaxSeqNo: 25 sessionId: src_id|ea5a8f23-baf00dae-8cb978b5-a18448ea_0
2025-10-09T21:48:44.918534Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e07ad82b-f0adcc2c-732b59b6-11925753_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ea5a8f23-baf00dae-8cb978b5-a18448ea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:44.920979Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea5a8f23-baf00dae-8cb978b5-a18448ea_0] Write session: try to update token
2025-10-09T21:48:44.920996Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea5a8f23-baf00dae-8cb978b5-a18448ea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:44.929465Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:44.929483Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:44.929492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342260906326046:9679]
2025-10-09T21:48:44.929537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:44.929538Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:44.930116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:44.930617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:44.930754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:44.930887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:44.931078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:44.931124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:44.934474Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1306 sessionId: src_id|ea5a8f23-baf00dae-8cb978b5-a18448ea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:44.936809Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:44.937017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:44.937038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:44.937076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:44.937179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:44.942751Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:44.942766Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:44.942769Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:44.942771Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:44.942782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342260906326046:9679]
2025-10-09T21:48:44.942786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342260906326046:9679]
2025-10-09T21:48:44.942790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342260906326046:9679]
2025-10-09T21:48:44.942832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:44.942843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:44.942844Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:44.942846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:44.942852Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:44.942856Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:44.942981Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:44.993543Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342261268149739:4303]
2025-10-09T21:48:44.993863Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 1305, pipe: [1:7559342261268149739:4303]
2025-10-09T21:48:45.103063Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:45.111374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:45.111399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:45.111558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:45.111609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:45.111666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:45.111730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:45.112207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:45.112221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:45.112292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:45.112350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:45.112409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:45.112477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:45.112540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:45.112548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:45.112608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:45.112660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:45.112721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:45.112788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:45.112833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:45.112844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:45.112913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:45.112975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:45.113039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:45.113103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:45.113149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:45.113157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:45.113220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:45.113284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:45.113348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:45.113413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:45.113457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:45.113465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:45.113529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:45.113588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:45.113651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:45.113725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:45.113769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:45.113777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:45.113837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:45.113891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:45.113955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:45.114019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:45.114209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:45.114217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:45.114278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:45.114333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:45.114389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:45.114456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:45.114499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:45.114507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:45.114569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:45.114626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:45.114686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:45.114743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:45.114784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:45.114791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:45.114854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:45.114912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:45.114980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:45.115041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:45.115173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:45.115185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:45.115252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342261268149739:4303], now have 1 active actors on pipe
2025-10-09T21:48:45.115467Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:45.115483Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:45.115486Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342261268149739:4303]
2025-10-09T21:48:45.115505Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:45.115515Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1306 reason: error at write: Disconnected sessionId: src_id|ea5a8f23-baf00dae-8cb978b5-a18448ea_0
2025-10-09T21:48:45.115656Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:45.115723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1306 sessionId: src_id|ea5a8f23-baf00dae-8cb978b5-a18448ea_0 is DEAD
2025-10-09T21:48:45.115879Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea5a8f23-baf00dae-8cb978b5-a18448ea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:45.115889Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea5a8f23-baf00dae-8cb978b5-a18448ea_0] Write session will restart in 0.007023s
2025-10-09T21:48:45.115913Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea5a8f23-baf00dae-8cb978b5-a18448ea_0] Write session: Do CDS request
2025-10-09T21:48:45.115920Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea5a8f23-baf00dae-8cb978b5-a18448ea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:45.124577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea5a8f23-baf00dae-8cb978b5-a18448ea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:45.124686Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:45.124705Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1307
2025-10-09T21:48:45.124818Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1307 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:45.124863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1307 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:45.124874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1307 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:45.124878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:45.125316Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1307 sessionId:  describe result for acl check
2025-10-09T21:48:45.125354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:45.125361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:45.125365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:45.125377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342265563117090:4259] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:45.125980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342265563117090:4259] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:45.128463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:45.128577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342265563117090:4259] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:45.130458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:45.130661Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:45.130673Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:45.130687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:45.130749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:45.130791Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342265201293479:9681]
2025-10-09T21:48:45.131376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:45.131498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342265563117090:4259] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:45.131517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342265563117090:4259] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:45.131522Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342265563117090:4259] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:45.131531Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1307 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:45.131674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:45.131724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:45.132053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:45.132109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.132115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.132116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.132118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.132119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.132120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.132121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.132123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.132124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.132129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.132140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.132144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.132149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.132153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.132157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.132161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.132165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.132168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.132175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.132183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.132189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.132193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.132197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.132202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.132207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.132212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.132217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.132222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.132226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.132231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.132237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.132242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.132246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.132251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.132261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.132267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.132296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:45.138676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:45.139728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:45.139746Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:45.139748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:45.139753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:45.139760Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1306 [2:7559342265201293479:9681]
2025-10-09T21:48:45.139775Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:45.139795Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:45.139801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:45.139805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:45.139807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:45.139824Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:45.139836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:45.139843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:45.139906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.139914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.139917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.139920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:45.139986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:45.139997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.140013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.140027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.140047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:45.140072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342265563117170:4259], now have 1 active actors on pipe
2025-10-09T21:48:45.140081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342265563117038:2432], now have 1 active actors on pipe
2025-10-09T21:48:45.140090Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342265201293474:9681]
2025-10-09T21:48:45.140092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342265201293474:9681]
2025-10-09T21:48:45.140094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342265201293474:9681]
2025-10-09T21:48:45.140107Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:45.140124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:45.140125Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:45.140129Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:45.140131Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:45.140145Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:45.140149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:45.140137Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1306
2025-10-09T21:48:45.140145Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1306
2025-10-09T21:48:45.140182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342265201293474:9681]
2025-10-09T21:48:45.140191Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:45.140193Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:45.140213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:45.140224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:45.140273Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6b136274-a4465fe7-b98d8524-a5e92d1e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:45.140286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:45.140290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:45.140297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:45.140308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:45.140322Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:45.140346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:45.140505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:45.140520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:45.140544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:45.140634Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1307 partition: 0 MaxSeqNo: 25 sessionId: src_id|6b136274-a4465fe7-b98d8524-a5e92d1e_0
2025-10-09T21:48:45.140987Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea5a8f23-baf00dae-8cb978b5-a18448ea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6b136274-a4465fe7-b98d8524-a5e92d1e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:45.143526Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b136274-a4465fe7-b98d8524-a5e92d1e_0] Write session: try to update token
2025-10-09T21:48:45.143548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b136274-a4465fe7-b98d8524-a5e92d1e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:45.154772Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:45.154790Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:45.154793Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:45.154795Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:45.154805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342265201293474:9681]
2025-10-09T21:48:45.154811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342265201293474:9681]
2025-10-09T21:48:45.154815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342265201293474:9681]
2025-10-09T21:48:45.154857Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:45.154860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:45.154868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.154870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:45.154879Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:45.154884Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:45.155016Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:45.159635Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1307 sessionId: src_id|6b136274-a4465fe7-b98d8524-a5e92d1e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:45.162012Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:45.215709Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342265563117183:4303]
2025-10-09T21:48:45.215949Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 1306, pipe: [1:7559342265563117183:4303]
2025-10-09T21:48:45.310098Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:45.310526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:45.310542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:45.310593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342265563117183:4303], now have 1 active actors on pipe
2025-10-09T21:48:45.310801Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:45.310808Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:45.310811Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:45.310859Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342265563117183:4303]
2025-10-09T21:48:45.310862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1307 reason: error at write: Disconnected sessionId: src_id|6b136274-a4465fe7-b98d8524-a5e92d1e_0
2025-10-09T21:48:45.311044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1307 sessionId: src_id|6b136274-a4465fe7-b98d8524-a5e92d1e_0 is DEAD
2025-10-09T21:48:45.311175Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b136274-a4465fe7-b98d8524-a5e92d1e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:45.311184Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b136274-a4465fe7-b98d8524-a5e92d1e_0] Write session will restart in 0.008585s
2025-10-09T21:48:45.311166Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:45.311208Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b136274-a4465fe7-b98d8524-a5e92d1e_0] Write session: Do CDS request
2025-10-09T21:48:45.311214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b136274-a4465fe7-b98d8524-a5e92d1e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:45.320578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b136274-a4465fe7-b98d8524-a5e92d1e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:45.320685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:45.320703Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1308
2025-10-09T21:48:45.320827Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1308 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:45.320860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1308 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:45.320873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1308 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:45.320877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:45.321349Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1308 sessionId:  describe result for acl check
2025-10-09T21:48:45.321388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:45.321390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:45.321392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:45.321398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342265563117309:4244] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:45.321963Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342265563117309:4244] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:45.322777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:45.324538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342265563117309:4244] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:45.324647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:45.324858Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:45.324869Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:45.324882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:45.324935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:45.325117Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342265201293650:9695]
2025-10-09T21:48:45.325734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:45.326078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:45.326125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:45.326451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:45.326515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.326521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.326522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.326523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.326525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.326526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.326527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.326528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.326529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.326534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.326544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.326547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.326551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.326554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.326558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.326561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.326564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.326567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.326575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.326583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.326589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.326593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.326597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.326602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.326607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.326612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.326617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.326622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.326632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.326638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.326642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.326654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.326658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.326663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.326667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.326672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.326706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:45.327504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342265563117309:4244] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:45.327521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342265563117309:4244] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:45.327525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342265563117309:4244] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:45.327533Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1308 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:45.333103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:45.334204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:45.334221Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:45.334224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:45.334228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:45.334235Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1307 [2:7559342265201293650:9695]
2025-10-09T21:48:45.334248Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:45.334266Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:45.334273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:45.334277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:45.334280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:45.334294Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:45.334302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:45.334308Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:45.334355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.334363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.334365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.334367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:45.334499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:45.334506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.334514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.334520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.334530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:45.334546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342265563117348:4244], now have 1 active actors on pipe
2025-10-09T21:48:45.334555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342265563117216:2432], now have 1 active actors on pipe
2025-10-09T21:48:45.334548Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1307
2025-10-09T21:48:45.334560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342265201293645:9695]
2025-10-09T21:48:45.334569Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1307
2025-10-09T21:48:45.334562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342265201293645:9695]
2025-10-09T21:48:45.334563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342265201293645:9695]
2025-10-09T21:48:45.334572Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:45.334585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:45.334592Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:45.334596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:45.334598Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:45.334599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:45.334602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:45.334636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342265201293645:9695]
2025-10-09T21:48:45.334649Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:45.334664Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:45.334667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:45.334672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:45.334714Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6144fb4a-6722bbc0-7c86b465-c309a365_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:45.334727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:45.334731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:45.334739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:45.334749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:45.334765Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:45.334783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:45.334931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:45.334938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:45.334954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:45.335061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1308 partition: 0 MaxSeqNo: 25 sessionId: src_id|6144fb4a-6722bbc0-7c86b465-c309a365_0
2025-10-09T21:48:45.335440Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b136274-a4465fe7-b98d8524-a5e92d1e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6144fb4a-6722bbc0-7c86b465-c309a365_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:45.337910Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6144fb4a-6722bbc0-7c86b465-c309a365_0] Write session: try to update token
2025-10-09T21:48:45.337930Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6144fb4a-6722bbc0-7c86b465-c309a365_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:45.349116Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:45.349139Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:45.349142Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:45.349147Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:45.349157Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342265201293645:9695]
2025-10-09T21:48:45.349161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342265201293645:9695]
2025-10-09T21:48:45.349165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342265201293645:9695]
2025-10-09T21:48:45.349211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:45.349215Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:45.349220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.349222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:45.349223Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:45.349227Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:45.349358Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:45.351584Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1308 sessionId: src_id|6144fb4a-6722bbc0-7c86b465-c309a365_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:45.354048Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:45.410823Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342265563117356:4303]
2025-10-09T21:48:45.411102Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 1307, pipe: [1:7559342265563117356:4303]
2025-10-09T21:48:45.504043Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:45.504278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:45.504292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:45.504335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342265563117356:4303], now have 1 active actors on pipe
2025-10-09T21:48:45.504537Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:45.504570Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342265563117356:4303]
2025-10-09T21:48:45.504584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:45.504587Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:45.504644Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1308 reason: error at write: Disconnected sessionId: src_id|6144fb4a-6722bbc0-7c86b465-c309a365_0
2025-10-09T21:48:45.504683Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:45.504843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1308 sessionId: src_id|6144fb4a-6722bbc0-7c86b465-c309a365_0 is DEAD
2025-10-09T21:48:45.505001Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6144fb4a-6722bbc0-7c86b465-c309a365_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:45.505010Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6144fb4a-6722bbc0-7c86b465-c309a365_0] Write session will restart in 0.005080s
2025-10-09T21:48:45.505035Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6144fb4a-6722bbc0-7c86b465-c309a365_0] Write session: Do CDS request
2025-10-09T21:48:45.505041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6144fb4a-6722bbc0-7c86b465-c309a365_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:45.511517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6144fb4a-6722bbc0-7c86b465-c309a365_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:45.511587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:45.511604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1309
2025-10-09T21:48:45.511754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1309 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:45.511786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1309 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:45.511797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1309 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:45.511801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:45.512197Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1309 sessionId:  describe result for acl check
2025-10-09T21:48:45.512224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:45.512225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:45.512227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:45.512230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342265563117374:4237] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:45.512722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342265563117374:4237] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:45.515244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342265563117374:4237] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:45.516542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:45.518561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:45.518761Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:45.518781Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:45.518797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:45.518853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:45.518906Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342265201293786:9699]
2025-10-09T21:48:45.518928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342265563117374:4237] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:45.518943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342265563117374:4237] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:45.518948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342265563117374:4237] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:45.518962Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1309 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:45.519480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:45.519789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:45.519830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:45.520130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:45.520177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.520182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.520183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.520184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.520185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.520187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.520188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.520189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.520190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.520195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.520207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.520211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.520215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.520219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.520223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.520226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.520230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.520233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.520241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.520254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.520264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.520270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.520274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.520280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.520289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.520293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.520298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.520304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.520313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.520318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.520322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.520327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.520331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.520336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.520342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.520347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.520382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:45.526572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:45.527650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:45.527666Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:45.527668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:45.527673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:45.527680Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1308 [2:7559342265201293786:9699]
2025-10-09T21:48:45.527696Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:45.527714Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:45.527724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:45.527727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:45.527731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:45.527747Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:45.527754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:45.527761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:45.527817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.527825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.527829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.527832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:45.527920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:45.527928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.527941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.527957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.527975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:45.528001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342265563117358:2432], now have 1 active actors on pipe
2025-10-09T21:48:45.528009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342265563117491:4237], now have 1 active actors on pipe
2025-10-09T21:48:45.528016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342265201293781:9699]
2025-10-09T21:48:45.528023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342265201293781:9699]
2025-10-09T21:48:45.528025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342265201293781:9699]
2025-10-09T21:48:45.528038Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:45.528044Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1308
2025-10-09T21:48:45.528057Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1308
2025-10-09T21:48:45.528055Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:45.528057Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:45.528066Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:45.528068Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:45.528071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:45.528074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:45.528104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342265201293781:9699]
2025-10-09T21:48:45.528107Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:45.528110Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:45.528129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:45.528136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:45.528184Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2e1522bc-3ea1370c-a5cc5441-e5962056_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:45.528201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:45.528206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:45.528209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:45.528220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:45.528230Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:45.528247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:45.528469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:45.528479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:45.528494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:45.528603Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1309 partition: 0 MaxSeqNo: 25 sessionId: src_id|2e1522bc-3ea1370c-a5cc5441-e5962056_0
2025-10-09T21:48:45.528950Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6144fb4a-6722bbc0-7c86b465-c309a365_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2e1522bc-3ea1370c-a5cc5441-e5962056_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:45.531334Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e1522bc-3ea1370c-a5cc5441-e5962056_0] Write session: try to update token
2025-10-09T21:48:45.531354Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e1522bc-3ea1370c-a5cc5441-e5962056_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:45.543956Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:45.543985Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:45.543989Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:45.543993Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:45.544005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342265201293781:9699]
2025-10-09T21:48:45.544012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342265201293781:9699]
2025-10-09T21:48:45.544019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342265201293781:9699]
2025-10-09T21:48:45.544063Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:45.544075Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:45.544078Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:45.544081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:45.544094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.544098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:45.544365Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:45.545158Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1309 sessionId: src_id|2e1522bc-3ea1370c-a5cc5441-e5962056_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:45.547536Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:45.604807Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342265563117502:4303]
2025-10-09T21:48:45.605100Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 1308, pipe: [1:7559342265563117502:4303]
2025-10-09T21:48:45.698766Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:45.699041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:45.699061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:45.699107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342265563117502:4303], now have 1 active actors on pipe
2025-10-09T21:48:45.699312Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:45.699340Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342265563117502:4303]
2025-10-09T21:48:45.699351Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:45.699377Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:45.699465Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1309 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2e1522bc-3ea1370c-a5cc5441-e5962056_0
2025-10-09T21:48:45.699655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1309 sessionId: src_id|2e1522bc-3ea1370c-a5cc5441-e5962056_0 is DEAD
2025-10-09T21:48:45.699764Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e1522bc-3ea1370c-a5cc5441-e5962056_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:45.699772Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e1522bc-3ea1370c-a5cc5441-e5962056_0] Write session will restart in 0.006747s
2025-10-09T21:48:45.699798Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e1522bc-3ea1370c-a5cc5441-e5962056_0] Write session: Do CDS request
2025-10-09T21:48:45.699804Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e1522bc-3ea1370c-a5cc5441-e5962056_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:45.707575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e1522bc-3ea1370c-a5cc5441-e5962056_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:45.707679Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:45.707697Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1310
2025-10-09T21:48:45.707836Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1310 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:45.707870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1310 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:45.707883Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1310 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:45.707886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:45.708336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1310 sessionId:  describe result for acl check
2025-10-09T21:48:45.708371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:45.708380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:45.708381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:45.708389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342265563117540:4233] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:45.708969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342265563117540:4233] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:45.711279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342265563117540:4233] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:45.711643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:45.713523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:45.713761Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:45.713774Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:45.713788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:45.713861Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342265563117540:4233] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:45.713873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342265563117540:4233] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:45.713877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342265563117540:4233] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:45.713887Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1310 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:45.713866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:45.713905Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342265201293925:9703]
2025-10-09T21:48:45.714356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:45.714649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:45.714703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:45.715051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:45.715111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.715118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.715119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.715120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.715121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.715123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.715124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.715125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.715127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.715132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.715143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.715147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.715151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.715156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.715160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.715164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.715168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.715171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.715178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.715186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.715200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.715206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.715210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.715216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.715220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.715226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.715231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.715236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.715240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.715246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.715251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.715256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.715260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.715265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.715270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.715276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.715306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:45.721804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:45.723095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:45.723113Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:45.723117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:45.723121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:45.723128Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1309 [2:7559342265201293925:9703]
2025-10-09T21:48:45.723145Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:45.723165Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:45.723173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:45.723177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:45.723180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:45.723203Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:45.723215Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:45.723227Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:45.723298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.723305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.723307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.723310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:45.723380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:45.723388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.723401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.723414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.723431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:45.723454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342265563117638:4233], now have 1 active actors on pipe
2025-10-09T21:48:45.723463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342265563117505:2432], now have 1 active actors on pipe
2025-10-09T21:48:45.723471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342265201293920:9703]
2025-10-09T21:48:45.723475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342265201293920:9703]
2025-10-09T21:48:45.723477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342265201293920:9703]
2025-10-09T21:48:45.723491Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:45.723509Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:45.723520Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:45.723524Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:45.723527Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:45.723530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:45.723533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:45.723532Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1309
2025-10-09T21:48:45.723535Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1309
2025-10-09T21:48:45.723571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342265201293920:9703]
2025-10-09T21:48:45.723581Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:45.723583Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:45.723604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:45.723609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:45.723663Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2281e579-bee0918e-fa96ee68-5904e076_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:45.723674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:45.723679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:45.723682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:45.723694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:45.723709Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:45.723730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:45.723848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:45.723856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:45.723869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:45.723945Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1310 partition: 0 MaxSeqNo: 25 sessionId: src_id|2281e579-bee0918e-fa96ee68-5904e076_0
2025-10-09T21:48:45.724322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e1522bc-3ea1370c-a5cc5441-e5962056_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2281e579-bee0918e-fa96ee68-5904e076_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:45.726764Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2281e579-bee0918e-fa96ee68-5904e076_0] Write session: try to update token
2025-10-09T21:48:45.726785Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2281e579-bee0918e-fa96ee68-5904e076_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:45.738025Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:45.738060Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:45.738062Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:45.738064Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:45.738074Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342265201293920:9703]
2025-10-09T21:48:45.738079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342265201293920:9703]
2025-10-09T21:48:45.738085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342265201293920:9703]
2025-10-09T21:48:45.738129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:45.738142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.738145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:45.738146Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:45.738157Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:45.738161Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:45.738279Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:45.740000Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1310 sessionId: src_id|2281e579-bee0918e-fa96ee68-5904e076_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:45.742430Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:45.799422Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342265563117649:4303]
2025-10-09T21:48:45.799717Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 1309, pipe: [1:7559342265563117649:4303]
2025-10-09T21:48:45.893876Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:45.894192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:45.894210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:45.894257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342265563117649:4303], now have 1 active actors on pipe
2025-10-09T21:48:45.894478Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:45.894549Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:45.894564Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:45.894640Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1310 reason: error at write: Disconnected sessionId: src_id|2281e579-bee0918e-fa96ee68-5904e076_0
2025-10-09T21:48:45.894835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1310 sessionId: src_id|2281e579-bee0918e-fa96ee68-5904e076_0 is DEAD
2025-10-09T21:48:45.894896Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:45.894947Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342265563117649:4303]
2025-10-09T21:48:45.895028Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2281e579-bee0918e-fa96ee68-5904e076_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:45.895039Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2281e579-bee0918e-fa96ee68-5904e076_0] Write session will restart in 0.006959s
2025-10-09T21:48:45.895067Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2281e579-bee0918e-fa96ee68-5904e076_0] Write session: Do CDS request
2025-10-09T21:48:45.895074Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2281e579-bee0918e-fa96ee68-5904e076_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:45.903545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2281e579-bee0918e-fa96ee68-5904e076_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:45.903641Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:45.903661Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1311
2025-10-09T21:48:45.903783Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1311 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:45.903819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1311 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:45.903831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1311 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:45.903835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:45.904309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1311 sessionId:  describe result for acl check
2025-10-09T21:48:45.904352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:45.904358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:45.904360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:45.904366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342265563117704:4225] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:45.904863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342265563117704:4225] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:45.906828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342265563117704:4225] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:45.907124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:45.909016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:45.909216Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:45.909225Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:45.909237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:45.909290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:45.909338Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342265201294062:9705]
2025-10-09T21:48:45.909401Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342265563117704:4225] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:45.909418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342265563117704:4225] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:45.909422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342265563117704:4225] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:45.909431Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1311 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:45.909923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:45.910234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:45.910279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:45.910598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:45.910647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.910652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.910654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.910655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.910657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.910658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.910659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.910660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.910661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.910666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.910677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.910681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.910685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.910689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.910693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.910697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.910701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.910705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.910711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.910719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:45.910725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.910730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:45.910735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.910740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:45.910745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.910750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:45.910754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.910759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:45.910764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.910768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:45.910774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.910779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.910783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.910788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.910793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.910799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.910830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:45.917228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:45.918289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:45.918304Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:45.918307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:45.918311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:45.918318Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1310 [2:7559342265201294062:9705]
2025-10-09T21:48:45.918330Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:45.918350Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:45.918353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:45.918356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:45.918358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:45.918377Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:45.918387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:45.918399Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:45.918458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.918467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.918469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.918472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:45.918539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:45.918546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.918557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:45.918568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:45.918586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:45.918611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342265563117788:4225], now have 1 active actors on pipe
2025-10-09T21:48:45.918621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342265563117654:2432], now have 1 active actors on pipe
2025-10-09T21:48:45.918629Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342265201294057:9705]
2025-10-09T21:48:45.918631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342265201294057:9705]
2025-10-09T21:48:45.918633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342265201294057:9705]
2025-10-09T21:48:45.918650Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:45.918664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:45.918669Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:45.918679Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:45.918682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:45.918682Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:45.918684Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:45.918690Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1310
2025-10-09T21:48:45.918710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342265201294057:9705]
2025-10-09T21:48:45.918710Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1310
2025-10-09T21:48:45.918720Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:45.918728Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:45.918755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:45.918760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:45.918798Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a60d9816-8fa2c00f-5a91c956-972ece56_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:45.918811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:45.918815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:45.918817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:45.918826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:45.918836Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:45.918848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:45.918986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:45.918994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:45.919008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:45.919090Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1311 partition: 0 MaxSeqNo: 25 sessionId: src_id|a60d9816-8fa2c00f-5a91c956-972ece56_0
2025-10-09T21:48:45.919497Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2281e579-bee0918e-fa96ee68-5904e076_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a60d9816-8fa2c00f-5a91c956-972ece56_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:45.921904Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a60d9816-8fa2c00f-5a91c956-972ece56_0] Write session: try to update token
2025-10-09T21:48:45.921922Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a60d9816-8fa2c00f-5a91c956-972ece56_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:45.933109Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:45.933129Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:45.933131Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:45.933133Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:45.933143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342265201294057:9705]
2025-10-09T21:48:45.933147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342265201294057:9705]
2025-10-09T21:48:45.933152Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342265201294057:9705]
2025-10-09T21:48:45.933194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:45.933202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:45.933204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:45.933210Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:45.933216Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:45.933220Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:45.933334Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:45.934251Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1311 sessionId: src_id|a60d9816-8fa2c00f-5a91c956-972ece56_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:45.936600Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:45.995019Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342265563117799:4303]
2025-10-09T21:48:45.995322Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 1310, pipe: [1:7559342265563117799:4303]
2025-10-09T21:48:46.092202Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:46.092583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:46.092594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:46.092633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342265563117799:4303], now have 1 active actors on pipe
2025-10-09T21:48:46.092816Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:46.092882Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:46.092900Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:46.092982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1311 reason: error at write: Disconnected sessionId: src_id|a60d9816-8fa2c00f-5a91c956-972ece56_0
2025-10-09T21:48:46.092987Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342265563117799:4303]
2025-10-09T21:48:46.093008Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:46.093176Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1311 sessionId: src_id|a60d9816-8fa2c00f-5a91c956-972ece56_0 is DEAD
2025-10-09T21:48:46.093332Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a60d9816-8fa2c00f-5a91c956-972ece56_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:46.093342Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a60d9816-8fa2c00f-5a91c956-972ece56_0] Write session will restart in 0.006913s
2025-10-09T21:48:46.093378Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a60d9816-8fa2c00f-5a91c956-972ece56_0] Write session: Do CDS request
2025-10-09T21:48:46.093384Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a60d9816-8fa2c00f-5a91c956-972ece56_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:46.101577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a60d9816-8fa2c00f-5a91c956-972ece56_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:46.101667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:46.101710Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1312
2025-10-09T21:48:46.101850Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1312 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:46.101897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1312 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:46.101908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1312 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:46.101912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:46.102385Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1312 sessionId:  describe result for acl check
2025-10-09T21:48:46.102424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:46.102430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:46.102432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:46.102439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342269858085138:4220] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:46.103012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342269858085138:4220] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:46.105235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342269858085138:4220] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:46.106058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:46.108073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:46.108171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342269858085138:4220] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:46.108186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342269858085138:4220] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:46.108192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342269858085138:4220] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:46.108201Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1312 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:46.108265Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:46.108277Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:46.108288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:46.108343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:46.108382Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342269496261495:9707]
2025-10-09T21:48:46.108993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:46.109311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:46.109370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:46.109722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:46.109776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.109782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.109784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.109785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.109786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.109788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.109789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.109790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.109792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.109797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.109809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.109814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.109819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.109824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.109828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.109831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.109835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.109838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.109850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.109860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.109871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.109881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.109886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.109892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.109896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.109901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.109906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.109914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.109918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.109925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.109932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.109937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.109943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.109949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.109958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.109963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.109992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:46.116632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:46.117900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:46.117914Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:46.117916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:46.117921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:46.117928Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1311 [2:7559342269496261495:9707]
2025-10-09T21:48:46.117951Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:46.117973Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:46.117981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:46.117984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:46.117987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:46.118005Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:46.118017Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:46.118025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:46.118121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.118128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.118130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.118134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:46.118209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:46.118217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.118231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.118248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.118269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:46.118296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342269858085233:4220], now have 1 active actors on pipe
2025-10-09T21:48:46.118305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342269858085098:2432], now have 1 active actors on pipe
2025-10-09T21:48:46.118316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342269496261490:9707]
2025-10-09T21:48:46.118318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342269496261490:9707]
2025-10-09T21:48:46.118321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342269496261490:9707]
2025-10-09T21:48:46.118333Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:46.118348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:46.118361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:46.118362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:46.118366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:46.118374Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:46.118377Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:46.118380Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1311
2025-10-09T21:48:46.118406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342269496261490:9707]
2025-10-09T21:48:46.118415Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:46.118420Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:46.118379Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1311
2025-10-09T21:48:46.118458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:46.118467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:46.118498Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c2386beb-94b95afb-76323229-2f58ac4e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:46.118510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:46.118516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:46.118523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:46.118534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:46.118547Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:46.118564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:46.118724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:46.118731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:46.118743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:46.118842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1312 partition: 0 MaxSeqNo: 25 sessionId: src_id|c2386beb-94b95afb-76323229-2f58ac4e_0
2025-10-09T21:48:46.119207Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a60d9816-8fa2c00f-5a91c956-972ece56_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c2386beb-94b95afb-76323229-2f58ac4e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:46.121609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c2386beb-94b95afb-76323229-2f58ac4e_0] Write session: try to update token
2025-10-09T21:48:46.121626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c2386beb-94b95afb-76323229-2f58ac4e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:46.132911Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:46.132929Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:46.132932Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:46.132934Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:46.132944Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342269496261490:9707]
2025-10-09T21:48:46.132949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342269496261490:9707]
2025-10-09T21:48:46.132954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342269496261490:9707]
2025-10-09T21:48:46.132998Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:46.133002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:46.133009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.133010Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:46.133012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:46.133013Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:46.133136Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:46.134874Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1312 sessionId: src_id|c2386beb-94b95afb-76323229-2f58ac4e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:46.137216Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:46.193161Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342269858085248:4303]
2025-10-09T21:48:46.193424Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 1311, pipe: [1:7559342269858085248:4303]
2025-10-09T21:48:46.291587Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:46.291841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:46.291856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:46.291901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342269858085248:4303], now have 1 active actors on pipe
2025-10-09T21:48:46.292132Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:46.292187Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:46.292241Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342269858085248:4303]
2025-10-09T21:48:46.292268Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:46.292348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1312 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c2386beb-94b95afb-76323229-2f58ac4e_0
2025-10-09T21:48:46.292519Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1312 sessionId: src_id|c2386beb-94b95afb-76323229-2f58ac4e_0 is DEAD
2025-10-09T21:48:46.292652Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2386beb-94b95afb-76323229-2f58ac4e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:46.292673Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2386beb-94b95afb-76323229-2f58ac4e_0] Write session will restart in 0.006080s
2025-10-09T21:48:46.292701Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2386beb-94b95afb-76323229-2f58ac4e_0] Write session: Do CDS request
2025-10-09T21:48:46.292708Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2386beb-94b95afb-76323229-2f58ac4e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:46.299540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c2386beb-94b95afb-76323229-2f58ac4e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:46.299581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:46.299609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1313
2025-10-09T21:48:46.299777Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1313 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:46.299821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1313 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:46.299829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1313 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:46.299833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:46.300251Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1313 sessionId:  describe result for acl check
2025-10-09T21:48:46.300286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:46.300293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:46.300297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:46.300303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342269858085318:4202] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:46.300852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342269858085318:4202] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:46.303182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342269858085318:4202] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:46.303640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:46.305658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:46.305885Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:46.305895Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:46.305909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:46.305927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342269858085318:4202] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:46.305941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342269858085318:4202] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:46.305944Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342269858085318:4202] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:46.305952Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1313 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:46.306015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:46.306071Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342269496261669:9719]
2025-10-09T21:48:46.306714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:46.306994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:46.307045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:46.307366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:46.307411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.307417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.307418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.307419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.307420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.307422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.307423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.307424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.307425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.307430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.307440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.307444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.307448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.307452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.307457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.307460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.307464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.307468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.307474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.307482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.307492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.307496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.307500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.307525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.307533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.307538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.307541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.307547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.307551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.307556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.307561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.307572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.307576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.307581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.307587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.307596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.307627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:46.313980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:46.315148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:46.315161Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:46.315164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:46.315168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:46.315175Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1312 [2:7559342269496261669:9719]
2025-10-09T21:48:46.315190Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:46.315209Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:46.315217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:46.315221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:46.315223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:46.315241Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:46.315252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:46.315260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:46.315321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.315328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.315331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.315335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:46.315403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:46.315415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.315427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.315438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.315461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:46.315486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342269858085283:2432], now have 1 active actors on pipe
2025-10-09T21:48:46.315494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342269858085423:4202], now have 1 active actors on pipe
2025-10-09T21:48:46.315502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342269496261657:9719]
2025-10-09T21:48:46.315505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342269496261657:9719]
2025-10-09T21:48:46.315507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342269496261657:9719]
2025-10-09T21:48:46.315519Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:46.315534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:46.315542Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:46.315545Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:46.315547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:46.315554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:46.315557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:46.315543Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1312
2025-10-09T21:48:46.315564Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1312
2025-10-09T21:48:46.315588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342269496261657:9719]
2025-10-09T21:48:46.315595Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:46.315598Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:46.315617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:46.315626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:46.315671Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|70a4c373-60dc42a0-2f2a3339-d0447163_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:46.315683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:46.315687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:46.315691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:46.315701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:46.315714Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:46.315735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:46.315948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:46.315955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:46.315971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:46.316082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1313 partition: 0 MaxSeqNo: 25 sessionId: src_id|70a4c373-60dc42a0-2f2a3339-d0447163_0
2025-10-09T21:48:46.316445Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2386beb-94b95afb-76323229-2f58ac4e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|70a4c373-60dc42a0-2f2a3339-d0447163_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:46.318877Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70a4c373-60dc42a0-2f2a3339-d0447163_0] Write session: try to update token
2025-10-09T21:48:46.318895Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70a4c373-60dc42a0-2f2a3339-d0447163_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:46.330021Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:46.330055Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:46.330058Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:46.330060Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:46.330072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342269496261657:9719]
2025-10-09T21:48:46.330077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342269496261657:9719]
2025-10-09T21:48:46.330080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342269496261657:9719]
2025-10-09T21:48:46.330126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:46.330133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.330136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:46.330138Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:46.330144Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:46.330147Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:46.330273Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:46.331884Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1313 sessionId: src_id|70a4c373-60dc42a0-2f2a3339-d0447163_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:46.334379Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:46.392322Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342269858085437:4303]
2025-10-09T21:48:46.392583Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 1312, pipe: [1:7559342269858085437:4303]
2025-10-09T21:48:46.493397Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:46.493685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:46.493702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:46.493749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342269858085437:4303], now have 1 active actors on pipe
2025-10-09T21:48:46.493972Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:46.493988Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342269858085437:4303]
2025-10-09T21:48:46.493999Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:46.494024Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:46.494113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1313 reason: status is not ok: tablet will be restarted right now sessionId: src_id|70a4c373-60dc42a0-2f2a3339-d0447163_0
2025-10-09T21:48:46.494252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1313 sessionId: src_id|70a4c373-60dc42a0-2f2a3339-d0447163_0 is DEAD
2025-10-09T21:48:46.494391Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70a4c373-60dc42a0-2f2a3339-d0447163_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:46.494401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70a4c373-60dc42a0-2f2a3339-d0447163_0] Write session will restart in 0.009795s
2025-10-09T21:48:46.494430Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70a4c373-60dc42a0-2f2a3339-d0447163_0] Write session: Do CDS request
2025-10-09T21:48:46.494436Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70a4c373-60dc42a0-2f2a3339-d0447163_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:46.505560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70a4c373-60dc42a0-2f2a3339-d0447163_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:46.505650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:46.505680Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1314
2025-10-09T21:48:46.505839Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1314 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:46.505872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1314 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:46.505883Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1314 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:46.505887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:46.506333Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1314 sessionId:  describe result for acl check
2025-10-09T21:48:46.506377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:46.506385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:46.506387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:46.506394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342269858085541:4198] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:46.506493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:46.506931Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342269858085541:4198] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:46.508568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:46.508794Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:46.508806Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:46.508819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:46.508878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:46.508924Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342269496261810:9723]
2025-10-09T21:48:46.509373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342269858085541:4198] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:46.509452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:46.509733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:46.509768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:46.509983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:46.510017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.510022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.510022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.510023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.510024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.510024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.510025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.510026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.510026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.510043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.510050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.510052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.510055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.510057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.510059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.510062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.510064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.510066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.510072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.510077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.510082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.510086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.510088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.510093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.510095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.510098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.510101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.510103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.510106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.510109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.510112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.510115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.510117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.510120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.510122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.510125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.510141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:46.515109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342269858085541:4198] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:46.515125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342269858085541:4198] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:46.515127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342269858085541:4198] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:46.515135Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1314 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:46.516384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:46.517566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:46.517581Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:46.517584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:46.517588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:46.517594Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1313 [2:7559342269496261810:9723]
2025-10-09T21:48:46.517608Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:46.517625Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:46.517630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:46.517633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:46.517636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:46.517652Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:46.517663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:46.517683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:46.517742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.517749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.517751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.517754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:46.517820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:46.517828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.517840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.517852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.517869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:46.517895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342269858085439:2432], now have 1 active actors on pipe
2025-10-09T21:48:46.517903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342269858085581:4198], now have 1 active actors on pipe
2025-10-09T21:48:46.517911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342269496261805:9723]
2025-10-09T21:48:46.517914Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342269496261805:9723]
2025-10-09T21:48:46.517917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342269496261805:9723]
2025-10-09T21:48:46.517929Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:46.517943Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:46.517947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:46.517961Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:46.517963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:46.517973Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1313
2025-10-09T21:48:46.517978Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1313
2025-10-09T21:48:46.517964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:46.517966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:46.517998Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342269496261805:9723]
2025-10-09T21:48:46.518000Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:46.518019Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:46.518057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:46.518067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:46.518104Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6e98ba1d-af4cebd1-1ca394aa-b155b2ed_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:46.518116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:46.518120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:46.518127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:46.518139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:46.518153Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:46.518171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:46.518348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:46.518363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:46.518386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:46.518483Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1314 partition: 0 MaxSeqNo: 25 sessionId: src_id|6e98ba1d-af4cebd1-1ca394aa-b155b2ed_0
2025-10-09T21:48:46.518831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70a4c373-60dc42a0-2f2a3339-d0447163_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6e98ba1d-af4cebd1-1ca394aa-b155b2ed_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:46.521276Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e98ba1d-af4cebd1-1ca394aa-b155b2ed_0] Write session: try to update token
2025-10-09T21:48:46.521293Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e98ba1d-af4cebd1-1ca394aa-b155b2ed_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:46.532428Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:46.532450Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:46.532453Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:46.532455Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:46.532465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342269496261805:9723]
2025-10-09T21:48:46.532473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342269496261805:9723]
2025-10-09T21:48:46.532478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342269496261805:9723]
2025-10-09T21:48:46.532526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:46.532535Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:46.532537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.532540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:46.532541Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:46.532545Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:46.532671Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:46.534268Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1314 sessionId: src_id|6e98ba1d-af4cebd1-1ca394aa-b155b2ed_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:46.536797Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:46.594065Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342269858085589:4303]
2025-10-09T21:48:46.594346Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 1313, pipe: [1:7559342269858085589:4303]
2025-10-09T21:48:46.689069Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:46.689361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:46.689374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:46.689415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342269858085589:4303], now have 1 active actors on pipe
2025-10-09T21:48:46.689587Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:46.689663Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:46.689692Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:46.689749Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342269858085589:4303]
2025-10-09T21:48:46.689770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1314 reason: status is not ok: tablet will be restarted right now sessionId: src_id|6e98ba1d-af4cebd1-1ca394aa-b155b2ed_0
2025-10-09T21:48:46.689949Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1314 sessionId: src_id|6e98ba1d-af4cebd1-1ca394aa-b155b2ed_0 is DEAD
2025-10-09T21:48:46.690102Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e98ba1d-af4cebd1-1ca394aa-b155b2ed_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:46.690114Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e98ba1d-af4cebd1-1ca394aa-b155b2ed_0] Write session will restart in 0.009240s
2025-10-09T21:48:46.690140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e98ba1d-af4cebd1-1ca394aa-b155b2ed_0] Write session: Do CDS request
2025-10-09T21:48:46.690146Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e98ba1d-af4cebd1-1ca394aa-b155b2ed_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:46.700541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e98ba1d-af4cebd1-1ca394aa-b155b2ed_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:46.700628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:46.700646Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1315
2025-10-09T21:48:46.700781Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1315 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:46.700815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1315 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:46.700828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1315 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:46.700832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:46.701221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:46.701248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1315 sessionId:  describe result for acl check
2025-10-09T21:48:46.701299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:46.701308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:46.701310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:46.701316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342269858085694:4184] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:46.701881Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342269858085694:4184] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:46.703049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:46.703259Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:46.703271Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:46.703283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:46.703340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:46.703383Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342269496261955:9727]
2025-10-09T21:48:46.703914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:46.703964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342269858085694:4184] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:46.704175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:46.704206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:46.704492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:46.704553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.704560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.704561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.704562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.704564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.704565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.704566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.704568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.704569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.704574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.704585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.704589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.704593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.704597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.704601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.704604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.704607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.704609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.704614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.704619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.704623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.704628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.704632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.704637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.704641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.704646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.704650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.704655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.704659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.704670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.704675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.704680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.704684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.704689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.704694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.704698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.704727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:46.709558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342269858085694:4184] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:46.709576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342269858085694:4184] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:46.709580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342269858085694:4184] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:46.709590Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1315 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:46.710916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:46.711916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:46.711930Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:46.711932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:46.711937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:46.711943Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1314 [2:7559342269496261955:9727]
2025-10-09T21:48:46.711961Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:46.711979Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:46.711986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:46.711989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:46.711992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:46.712008Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:46.712017Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:46.712029Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:46.712162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.712173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.712175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.712178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:46.712245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:46.712254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.712265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.712278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.712299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:46.712329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342269858085592:2432], now have 1 active actors on pipe
2025-10-09T21:48:46.712337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342269858085734:4184], now have 1 active actors on pipe
2025-10-09T21:48:46.712344Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342269496261949:9727]
2025-10-09T21:48:46.712347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342269496261949:9727]
2025-10-09T21:48:46.712349Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342269496261949:9727]
2025-10-09T21:48:46.712363Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:46.712381Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:46.712388Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:46.712392Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:46.712395Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:46.712395Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:46.712398Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:46.712379Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1314
2025-10-09T21:48:46.712387Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1314
2025-10-09T21:48:46.712432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342269496261949:9727]
2025-10-09T21:48:46.712443Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:46.712448Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:46.712466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:46.712470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:46.712517Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fde318e2-cd15e082-1a0e3efe-f0a0fac_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:46.712525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:46.712529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:46.712531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:46.712542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:46.712552Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:46.712576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:46.712742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:46.712749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:46.712764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:46.712869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1315 partition: 0 MaxSeqNo: 25 sessionId: src_id|fde318e2-cd15e082-1a0e3efe-f0a0fac_0
2025-10-09T21:48:46.713194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e98ba1d-af4cebd1-1ca394aa-b155b2ed_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fde318e2-cd15e082-1a0e3efe-f0a0fac_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:46.715520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fde318e2-cd15e082-1a0e3efe-f0a0fac_0] Write session: try to update token
2025-10-09T21:48:46.715538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fde318e2-cd15e082-1a0e3efe-f0a0fac_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:46.727104Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:46.727124Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:46.727127Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:46.727129Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:46.727139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342269496261949:9727]
2025-10-09T21:48:46.727144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342269496261949:9727]
2025-10-09T21:48:46.727149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342269496261949:9727]
2025-10-09T21:48:46.727196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:46.727203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.727206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:46.727245Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:46.727259Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:46.727263Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:46.727337Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:46.728775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1315 sessionId: src_id|fde318e2-cd15e082-1a0e3efe-f0a0fac_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:46.731272Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:46.790158Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342269858085744:4303]
2025-10-09T21:48:46.790471Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 1314, pipe: [1:7559342269858085744:4303]
2025-10-09T21:48:46.883564Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:46.883948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:46.883960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:46.884001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342269858085744:4303], now have 1 active actors on pipe
2025-10-09T21:48:46.884203Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:46.884223Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:46.884256Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342269858085744:4303]
2025-10-09T21:48:46.884275Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:46.884352Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1315 reason: status is not ok: tablet will be restarted right now sessionId: src_id|fde318e2-cd15e082-1a0e3efe-f0a0fac_0
2025-10-09T21:48:46.884548Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1315 sessionId: src_id|fde318e2-cd15e082-1a0e3efe-f0a0fac_0 is DEAD
2025-10-09T21:48:46.884676Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fde318e2-cd15e082-1a0e3efe-f0a0fac_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:46.884684Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fde318e2-cd15e082-1a0e3efe-f0a0fac_0] Write session will restart in 0.006981s
2025-10-09T21:48:46.884709Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fde318e2-cd15e082-1a0e3efe-f0a0fac_0] Write session: Do CDS request
2025-10-09T21:48:46.884716Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fde318e2-cd15e082-1a0e3efe-f0a0fac_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:46.892574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fde318e2-cd15e082-1a0e3efe-f0a0fac_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:46.892622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:46.892643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1316
2025-10-09T21:48:46.892796Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1316 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:46.892832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1316 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:46.892843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1316 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:46.892846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:46.893305Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1316 sessionId:  describe result for acl check
2025-10-09T21:48:46.893343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:46.893349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:46.893351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:46.893357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342269858085781:4170] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:46.893955Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342269858085781:4170] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:46.896501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342269858085781:4170] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:46.897117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:46.899121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:46.899351Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:46.899363Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:46.899379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:46.899447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:46.899492Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342269496262096:9729]
2025-10-09T21:48:46.899548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342269858085781:4170] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:46.899565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342269858085781:4170] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:46.899569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342269858085781:4170] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:46.899578Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1316 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:46.900115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:46.900460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:46.900512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:46.900863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:46.900923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.900932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.900934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.900935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.900937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.900938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.900939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.900941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.900942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.900948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.900959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.900963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.900967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.900970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.900974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.900978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.900982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.900986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.900993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.901002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:46.901013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.901024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:46.901028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.901039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:46.901044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.901053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:46.901063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.901073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:46.901085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.901095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:46.901105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.901110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.901114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.901119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.901124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.901129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.901167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:46.907560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:46.908668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:46.908686Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:46.908689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:46.908694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:46.908700Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1315 [2:7559342269496262096:9729]
2025-10-09T21:48:46.908713Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:46.908732Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:46.908735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:46.908739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:46.908741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:46.908759Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:46.908772Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:46.908780Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:46.908846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.908855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.908857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.908860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:46.908932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:46.908940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.908953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:46.908965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:46.908989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:46.909017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342269858085887:4170], now have 1 active actors on pipe
2025-10-09T21:48:46.909023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342269858085749:2432], now have 1 active actors on pipe
2025-10-09T21:48:46.909030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342269496262091:9729]
2025-10-09T21:48:46.909037Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342269496262091:9729]
2025-10-09T21:48:46.909039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342269496262091:9729]
2025-10-09T21:48:46.909052Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:46.909073Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:46.909055Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1315
2025-10-09T21:48:46.909078Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:46.909066Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1315
2025-10-09T21:48:46.909082Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:46.909084Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:46.909095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:46.909098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:46.909143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:46.909153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:46.909171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342269496262091:9729]
2025-10-09T21:48:46.909184Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:46.909212Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:46.909216Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|45ab8dc0-ce7ee64d-f7a4c0e-adadd396_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:46.909225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:46.909230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:46.909232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:46.909243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:46.909257Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:46.909288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:46.909447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:46.909455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:46.909471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:46.909565Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1316 partition: 0 MaxSeqNo: 25 sessionId: src_id|45ab8dc0-ce7ee64d-f7a4c0e-adadd396_0
2025-10-09T21:48:46.909970Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fde318e2-cd15e082-1a0e3efe-f0a0fac_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|45ab8dc0-ce7ee64d-f7a4c0e-adadd396_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:46.912560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45ab8dc0-ce7ee64d-f7a4c0e-adadd396_0] Write session: try to update token
2025-10-09T21:48:46.912578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45ab8dc0-ce7ee64d-f7a4c0e-adadd396_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:46.923746Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:46.923764Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:46.923767Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:46.923768Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:46.923780Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342269496262091:9729]
2025-10-09T21:48:46.923785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342269496262091:9729]
2025-10-09T21:48:46.923788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342269496262091:9729]
2025-10-09T21:48:46.923836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:46.923842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:46.923845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:46.923851Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:46.923861Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:46.923865Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:46.923979Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:46.925989Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1316 sessionId: src_id|45ab8dc0-ce7ee64d-f7a4c0e-adadd396_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:46.928310Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:46.984210Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342269858085900:4303]
2025-10-09T21:48:46.984586Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 1315, pipe: [1:7559342269858085900:4303]
2025-10-09T21:48:47.085600Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:47.085900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.085915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.085963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342269858085900:4303], now have 1 active actors on pipe
2025-10-09T21:48:47.086167Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:47.086206Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:47.086223Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:47.086234Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342269858085900:4303]
2025-10-09T21:48:47.086254Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:47.086290Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1316 reason: error at write: Disconnected sessionId: src_id|45ab8dc0-ce7ee64d-f7a4c0e-adadd396_0
2025-10-09T21:48:47.086479Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1316 sessionId: src_id|45ab8dc0-ce7ee64d-f7a4c0e-adadd396_0 is DEAD
2025-10-09T21:48:47.086673Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45ab8dc0-ce7ee64d-f7a4c0e-adadd396_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:47.086684Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45ab8dc0-ce7ee64d-f7a4c0e-adadd396_0] Write session will restart in 0.007631s
2025-10-09T21:48:47.086713Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45ab8dc0-ce7ee64d-f7a4c0e-adadd396_0] Write session: Do CDS request
2025-10-09T21:48:47.086720Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45ab8dc0-ce7ee64d-f7a4c0e-adadd396_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:47.095559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45ab8dc0-ce7ee64d-f7a4c0e-adadd396_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:47.095659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:47.095676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1317
2025-10-09T21:48:47.095841Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1317 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:47.095877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1317 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:47.095885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1317 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:47.095892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:47.096330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1317 sessionId:  describe result for acl check
2025-10-09T21:48:47.096376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:47.096383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:47.096385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:47.096391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342274153053284:4164] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:47.096929Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342274153053284:4164] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:47.098646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:47.098741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342274153053284:4164] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:47.100796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:47.100990Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:47.100999Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:47.101011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:47.101067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:47.101111Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342273791229535:9731]
2025-10-09T21:48:47.101595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342274153053284:4164] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:47.101609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342274153053284:4164] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:47.101613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342274153053284:4164] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:47.101622Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1317 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:47.101744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:47.102072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:47.102133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:47.102474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:47.102537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.102544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.102545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.102546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.102547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.102548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.102549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.102550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.102551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.102557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.102567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.102572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.102575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.102578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.102583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.102587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.102591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.102595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.102602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.102610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.102616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.102623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.102627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.102632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.102636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.102641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.102648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.102653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.102658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.102663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.102671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.102675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.102680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.102685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.102691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.102696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.102727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:47.109096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:47.110196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:47.110215Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:47.110218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:47.110223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:47.110231Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1316 [2:7559342273791229535:9731]
2025-10-09T21:48:47.110246Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:47.110265Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:47.110274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:47.110277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:47.110279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:47.110297Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:47.110307Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:47.110316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:47.110386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.110394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.110396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.110400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:47.110473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:47.110482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.110495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.110508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.110532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:47.110557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342274153053339:4164], now have 1 active actors on pipe
2025-10-09T21:48:47.110565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342274153053199:2432], now have 1 active actors on pipe
2025-10-09T21:48:47.110573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342273791229530:9731]
2025-10-09T21:48:47.110575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342273791229530:9731]
2025-10-09T21:48:47.110578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342273791229530:9731]
2025-10-09T21:48:47.110591Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:47.110605Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:47.110618Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:47.110626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:47.110629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:47.110630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:47.110632Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:47.110627Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1316
2025-10-09T21:48:47.110638Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1316
2025-10-09T21:48:47.110662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342273791229530:9731]
2025-10-09T21:48:47.110669Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:47.110672Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:47.110685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.110690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.110743Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9c31fee2-61508eb7-a8d2d1e1-7bd86b49_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:47.110755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:47.110759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:47.110762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:47.110773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:47.110785Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:47.110808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:47.110948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.110957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.110973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:47.111111Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1317 partition: 0 MaxSeqNo: 25 sessionId: src_id|9c31fee2-61508eb7-a8d2d1e1-7bd86b49_0
2025-10-09T21:48:47.111473Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45ab8dc0-ce7ee64d-f7a4c0e-adadd396_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9c31fee2-61508eb7-a8d2d1e1-7bd86b49_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:47.113945Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c31fee2-61508eb7-a8d2d1e1-7bd86b49_0] Write session: try to update token
2025-10-09T21:48:47.113963Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c31fee2-61508eb7-a8d2d1e1-7bd86b49_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:47.125543Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:47.125560Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:47.125563Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:47.125565Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:47.125576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342273791229530:9731]
2025-10-09T21:48:47.125579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342273791229530:9731]
2025-10-09T21:48:47.125583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342273791229530:9731]
2025-10-09T21:48:47.125627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:47.125630Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:47.125637Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:47.125639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.125641Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:47.125642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:47.125786Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:47.127529Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1317 sessionId: src_id|9c31fee2-61508eb7-a8d2d1e1-7bd86b49_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:47.129873Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:47.186398Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342274153053354:4303]
2025-10-09T21:48:47.186722Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 1316, pipe: [1:7559342274153053354:4303]
2025-10-09T21:48:47.284485Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:47.285580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.285600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.285643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342274153053354:4303], now have 1 active actors on pipe
2025-10-09T21:48:47.285829Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:47.285913Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:47.285929Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:47.285939Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:47.285982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1317 reason: error at write: Disconnected sessionId: src_id|9c31fee2-61508eb7-a8d2d1e1-7bd86b49_0
2025-10-09T21:48:47.285993Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342274153053354:4303]
2025-10-09T21:48:47.286162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1317 sessionId: src_id|9c31fee2-61508eb7-a8d2d1e1-7bd86b49_0 is DEAD
2025-10-09T21:48:47.286319Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c31fee2-61508eb7-a8d2d1e1-7bd86b49_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:47.286329Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c31fee2-61508eb7-a8d2d1e1-7bd86b49_0] Write session will restart in 0.005660s
2025-10-09T21:48:47.286356Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c31fee2-61508eb7-a8d2d1e1-7bd86b49_0] Write session: Do CDS request
2025-10-09T21:48:47.286362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c31fee2-61508eb7-a8d2d1e1-7bd86b49_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:47.293529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c31fee2-61508eb7-a8d2d1e1-7bd86b49_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:47.293586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:47.293600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1318
2025-10-09T21:48:47.293756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1318 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:47.293798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1318 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:47.293807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1318 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:47.293811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:47.294218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1318 sessionId:  describe result for acl check
2025-10-09T21:48:47.294252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:47.294259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:47.294261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:47.294266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342274153053413:4159] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:47.294752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342274153053413:4159] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:47.296565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342274153053413:4159] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:47.296883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:47.298524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:47.298694Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:47.298705Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:47.298715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:47.298761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:47.298798Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342273791229708:9739]
2025-10-09T21:48:47.298797Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342274153053413:4159] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:47.298822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342274153053413:4159] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:47.298826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342274153053413:4159] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:47.298833Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1318 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:47.299346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:47.299600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:47.299638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:47.299927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:47.299968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.299974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.299975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.299976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.299977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.299978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.299979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.299980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.299981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.299986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.299995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.299999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.300002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.300006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.300010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.300014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.300017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.300021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.300026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.300034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.300039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.300045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.300049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.300054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.300058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.300063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.300066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.300071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.300075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.300080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.300084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.300089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.300093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.300097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.300102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.300106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.300132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:47.306087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:47.307258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:47.307273Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:47.307275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:47.307279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:47.307284Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1317 [2:7559342273791229708:9739]
2025-10-09T21:48:47.307294Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:47.307307Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:47.307309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:47.307312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:47.307313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:47.307325Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:47.307329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:47.307334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:47.307375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.307380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.307381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.307383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:47.307448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:47.307455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.307461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.307467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.307476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:47.307491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342273791229702:9739]
2025-10-09T21:48:47.307493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342273791229702:9739]
2025-10-09T21:48:47.307494Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342273791229702:9739]
2025-10-09T21:48:47.307500Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:47.307511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342274153053502:4159], now have 1 active actors on pipe
2025-10-09T21:48:47.307512Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:47.307515Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:47.307517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342274153053361:2432], now have 1 active actors on pipe
2025-10-09T21:48:47.307517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:47.307523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:47.307530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:47.307532Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:47.307555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342273791229702:9739]
2025-10-09T21:48:47.307564Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:47.307592Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:47.307578Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1317
2025-10-09T21:48:47.307610Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1317
2025-10-09T21:48:47.307631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.307638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.307658Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|731899f0-d872cdd0-fdebc421-2b0ceaf0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:47.307668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:47.307671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:47.307672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:47.307679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:47.307686Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:47.307703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:47.307857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.307868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.307888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:47.307973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1318 partition: 0 MaxSeqNo: 25 sessionId: src_id|731899f0-d872cdd0-fdebc421-2b0ceaf0_0
2025-10-09T21:48:47.308312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c31fee2-61508eb7-a8d2d1e1-7bd86b49_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|731899f0-d872cdd0-fdebc421-2b0ceaf0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:47.310716Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|731899f0-d872cdd0-fdebc421-2b0ceaf0_0] Write session: try to update token
2025-10-09T21:48:47.310733Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|731899f0-d872cdd0-fdebc421-2b0ceaf0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:47.323116Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1318 sessionId: src_id|731899f0-d872cdd0-fdebc421-2b0ceaf0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:47.324386Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:47.324404Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:47.324413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342273791229702:9739]
2025-10-09T21:48:47.324462Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:47.324469Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:47.324629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:47.324784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:47.324929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:47.325071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:47.325273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:47.325323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:47.325462Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:47.325651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.325666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.325708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:47.325799Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:47.331787Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:47.331801Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:47.331803Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:47.331805Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:47.331814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342273791229702:9739]
2025-10-09T21:48:47.331819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342273791229702:9739]
2025-10-09T21:48:47.331822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342273791229702:9739]
2025-10-09T21:48:47.331860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:47.331865Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:47.331871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.331871Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:47.331873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:47.331875Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:47.332002Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:47.386119Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342274153053536:4303]
2025-10-09T21:48:47.386388Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 1317, pipe: [1:7559342274153053536:4303]
2025-10-09T21:48:47.484415Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:47.491353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.491381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.491545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.491605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.491665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.491720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.491769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.491783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.491860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.491925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.491983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.492043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.492084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.492096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.492159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.492216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.492274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.492332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.492372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.492379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.492441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.492507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.492572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.492628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.492664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.492670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.492798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.492854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.492907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.492960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.492999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.493004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.493059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.493110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.493164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.493231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.493278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.493286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.493341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.493392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.493451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.493516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.493578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.493593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.493650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.493733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.493791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.493845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.493883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.493889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.493941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.493990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.494057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.494111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.494158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.494164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.494226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.494276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.494328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.494382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.494428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.494435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.494483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342274153053536:4303], now have 1 active actors on pipe
2025-10-09T21:48:47.494712Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342274153053536:4303]
2025-10-09T21:48:47.494718Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:47.494723Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:47.494753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1318 reason: error at write: Disconnected sessionId: src_id|731899f0-d872cdd0-fdebc421-2b0ceaf0_0
2025-10-09T21:48:47.494759Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:47.494979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1318 sessionId: src_id|731899f0-d872cdd0-fdebc421-2b0ceaf0_0 is DEAD
2025-10-09T21:48:47.495077Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|731899f0-d872cdd0-fdebc421-2b0ceaf0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:47.495088Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|731899f0-d872cdd0-fdebc421-2b0ceaf0_0] Write session will restart in 0.006229s
2025-10-09T21:48:47.495112Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|731899f0-d872cdd0-fdebc421-2b0ceaf0_0] Write session: Do CDS request
2025-10-09T21:48:47.495118Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|731899f0-d872cdd0-fdebc421-2b0ceaf0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:47.495391Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:47.502561Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:47.502592Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|731899f0-d872cdd0-fdebc421-2b0ceaf0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:47.502574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1319
2025-10-09T21:48:47.502803Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1319 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:47.502838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1319 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:47.502847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1319 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:47.502851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:47.503251Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1319 sessionId:  describe result for acl check
2025-10-09T21:48:47.503292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:47.503298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:47.503300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:47.503306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342274153053558:4126] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:47.503842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342274153053558:4126] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:47.506514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342274153053558:4126] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:47.508152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:47.510073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:47.510264Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:47.510276Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:47.510287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:47.510338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:47.510375Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342273791229860:9747]
2025-10-09T21:48:47.510387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342274153053558:4126] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:47.510402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342274153053558:4126] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:47.510406Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342274153053558:4126] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:47.510414Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1319 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:47.510911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:47.511207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:47.511247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:47.511551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:47.511599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.511604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.511606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.511607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.511608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.511609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.511610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.511612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.511613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.511617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.511627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.511632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.511635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.511639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.511643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.511647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.511650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.511654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.511660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.511668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.511674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.511680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.511684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.511688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.511692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.511697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.511702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.511707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.511711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.511716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.511721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.511726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.511730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.511735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.511740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.511744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.511773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:47.518120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:47.519145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:47.519163Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:47.519166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:47.519171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:47.519177Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1318 [2:7559342273791229860:9747]
2025-10-09T21:48:47.519191Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:47.519208Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:47.519218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:47.519221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:47.519223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:47.519240Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:47.519246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:47.519253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:47.519311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.519319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.519322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.519325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:47.519396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:47.519404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.519416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.519426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.519450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:47.519471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342274153053538:2432], now have 1 active actors on pipe
2025-10-09T21:48:47.519477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342274153053680:4126], now have 1 active actors on pipe
2025-10-09T21:48:47.519484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342273791229855:9747]
2025-10-09T21:48:47.519485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342273791229855:9747]
2025-10-09T21:48:47.519486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342273791229855:9747]
2025-10-09T21:48:47.519500Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:47.519511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:47.519524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:47.519525Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:47.519529Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:47.519529Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:47.519532Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:47.519559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342273791229855:9747]
2025-10-09T21:48:47.519570Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:47.519555Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1318
2025-10-09T21:48:47.519591Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:47.519582Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1318
2025-10-09T21:48:47.519615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.519625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.519652Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9bd701bc-1178f25f-ee412bb3-ea39bbe1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:47.519664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:47.519668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:47.519676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:47.519685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:47.519695Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:47.519714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:47.519852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.519860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.519874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:47.519977Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1319 partition: 0 MaxSeqNo: 25 sessionId: src_id|9bd701bc-1178f25f-ee412bb3-ea39bbe1_0
2025-10-09T21:48:47.520342Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|731899f0-d872cdd0-fdebc421-2b0ceaf0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9bd701bc-1178f25f-ee412bb3-ea39bbe1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:47.522961Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9bd701bc-1178f25f-ee412bb3-ea39bbe1_0] Write session: try to update token
2025-10-09T21:48:47.522979Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9bd701bc-1178f25f-ee412bb3-ea39bbe1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:47.531015Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:47.531034Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:47.531045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342273791229855:9747]
2025-10-09T21:48:47.531095Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:47.531108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:47.531285Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:47.531474Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:47.531628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:47.531857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:47.532087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:47.532138Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:47.535866Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1319 sessionId: src_id|9bd701bc-1178f25f-ee412bb3-ea39bbe1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:47.538170Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:47.538398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.538418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.538468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:47.538576Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:47.543689Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:47.543703Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:47.543706Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:47.543708Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:47.543717Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342273791229855:9747]
2025-10-09T21:48:47.543721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342273791229855:9747]
2025-10-09T21:48:47.543725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342273791229855:9747]
2025-10-09T21:48:47.543764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:47.543777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.543776Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:47.543779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:47.543782Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:47.543786Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:47.543903Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:47.595001Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342274153053692:4303]
2025-10-09T21:48:47.595251Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 1318, pipe: [1:7559342274153053692:4303]
2025-10-09T21:48:47.693400Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:47.703432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.703465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.703634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.703694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.703940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.704008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.704139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.704155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.704368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.704426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.704658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.704725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.704768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.704775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.704834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.705079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.705146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.705204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.705258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.705262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.705329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.705392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.705453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.705515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.705641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.705652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.705724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.705784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.705845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.705906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.705958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.705965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.706042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.706100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.706157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.706219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.706260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.706268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.706325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.706379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.706434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.706488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.706528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.706535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.706592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.706647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.706706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.706774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.706814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.706821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.706880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.706933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.706992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.707048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.707090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.707098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.707155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.707210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.707265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.707319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.707360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.707367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.707432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342274153053692:4303], now have 1 active actors on pipe
2025-10-09T21:48:47.707820Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342274153053692:4303]
2025-10-09T21:48:47.707863Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:47.708160Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:47.708173Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:47.708200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1319 reason: error at write: Disconnected sessionId: src_id|9bd701bc-1178f25f-ee412bb3-ea39bbe1_0
2025-10-09T21:48:47.708410Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:47.708410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1319 sessionId: src_id|9bd701bc-1178f25f-ee412bb3-ea39bbe1_0 is DEAD
2025-10-09T21:48:47.708554Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9bd701bc-1178f25f-ee412bb3-ea39bbe1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:47.708567Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9bd701bc-1178f25f-ee412bb3-ea39bbe1_0] Write session will restart in 0.006554s
2025-10-09T21:48:47.708596Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9bd701bc-1178f25f-ee412bb3-ea39bbe1_0] Write session: Do CDS request
2025-10-09T21:48:47.708603Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9bd701bc-1178f25f-ee412bb3-ea39bbe1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:47.716534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9bd701bc-1178f25f-ee412bb3-ea39bbe1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:47.716629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:47.716647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1320
2025-10-09T21:48:47.716767Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1320 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:47.716798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1320 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:47.716831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1320 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:47.716835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:47.717259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1320 sessionId:  describe result for acl check
2025-10-09T21:48:47.717302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:47.717309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:47.717310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:47.717315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342274153053752:4128] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:47.717867Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342274153053752:4128] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:47.720299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342274153053752:4128] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:47.720512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:47.722311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:47.722503Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:47.722513Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:47.722524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:47.722576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:47.722623Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342273791230009:9751]
2025-10-09T21:48:47.723055Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342274153053752:4128] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:47.723070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342274153053752:4128] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:47.723074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342274153053752:4128] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:47.723081Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1320 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:47.723110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:47.723376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:47.723438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:47.723780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:47.723823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.723829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.723830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.723831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.723832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.723833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.723834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.723835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.723837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.723842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.723851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.723855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.723859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.723862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.723868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.723876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.723879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.723887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.723893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.723904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.723915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.723920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.723925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.723929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.723933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.723940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.723948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.723953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.723961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.723966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.723975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.723980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.723990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.723995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.723999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.724013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.724039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:47.730233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:47.731242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:47.731256Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:47.731258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:47.731261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:47.731266Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1319 [2:7559342273791230009:9751]
2025-10-09T21:48:47.731275Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:47.731288Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:47.731290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:47.731292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:47.731293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:47.731306Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:47.731309Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:47.731315Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:47.731359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.731367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.731369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.731372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:47.731424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:47.731430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.731437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.731443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.731453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:47.731466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342273791230004:9751]
2025-10-09T21:48:47.731472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342273791230004:9751]
2025-10-09T21:48:47.731473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342273791230004:9751]
2025-10-09T21:48:47.731479Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:47.731489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342274153053837:4128], now have 1 active actors on pipe
2025-10-09T21:48:47.731490Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:47.731493Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:47.731495Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:47.731496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:47.731504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:47.731506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:47.731510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342274153053695:2432], now have 1 active actors on pipe
2025-10-09T21:48:47.731528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342273791230004:9751]
2025-10-09T21:48:47.731538Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:47.731575Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:47.731574Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1319
2025-10-09T21:48:47.731577Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1319
2025-10-09T21:48:47.731690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.731705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.731734Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a72e1495-59921251-a951a2e7-e710eb9a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:47.731747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:47.731756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:47.731758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:47.731768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:47.731780Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:47.731794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:47.731960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.731969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.731985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:47.732080Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1320 partition: 0 MaxSeqNo: 25 sessionId: src_id|a72e1495-59921251-a951a2e7-e710eb9a_0
2025-10-09T21:48:47.732406Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9bd701bc-1178f25f-ee412bb3-ea39bbe1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a72e1495-59921251-a951a2e7-e710eb9a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:47.734844Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a72e1495-59921251-a951a2e7-e710eb9a_0] Write session: try to update token
2025-10-09T21:48:47.734863Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a72e1495-59921251-a951a2e7-e710eb9a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:47.742751Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:47.742781Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:47.742791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342273791230004:9751]
2025-10-09T21:48:47.742835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:47.742845Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:47.743003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:47.743137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:47.743266Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:47.743395Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:47.743572Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:47.743609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:47.748915Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1320 sessionId: src_id|a72e1495-59921251-a951a2e7-e710eb9a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:47.751271Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:47.751674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.751703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.751741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:47.751841Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:47.755194Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:47.755208Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:47.755211Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:47.755213Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:47.755224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342273791230004:9751]
2025-10-09T21:48:47.755228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342273791230004:9751]
2025-10-09T21:48:47.755232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342273791230004:9751]
2025-10-09T21:48:47.755272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:47.755283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.755282Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:47.755286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:47.755289Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:47.755302Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:47.755452Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:47.808010Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342274153053849:4303]
2025-10-09T21:48:47.808276Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 1319, pipe: [1:7559342274153053849:4303]
2025-10-09T21:48:47.901991Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:47.908458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.908480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.908619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.908678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.908731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.908784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.908822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.908829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.908887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.908953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.909013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.909079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.909118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.909123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.909181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.909243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.909302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.909361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.909398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.909406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.909491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.909549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.909605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.909656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.909709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.909716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.909773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.909829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.909890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.909949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.909987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.909994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.910065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.910120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.910176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.910239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.910278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.910285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.910339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.910391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.910447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.910506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.910545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.910551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.910607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.910662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.910715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.910773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.910809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.910814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.910861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.910909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.910961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.911017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.911057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.911062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.911114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:47.911161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:47.911220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:47.911285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:47.911385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:47.911393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:47.911431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342274153053849:4303], now have 1 active actors on pipe
2025-10-09T21:48:47.911613Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342274153053849:4303]
2025-10-09T21:48:47.911648Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:47.911720Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:47.911732Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:47.911761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1320 reason: error at write: Disconnected sessionId: src_id|a72e1495-59921251-a951a2e7-e710eb9a_0
2025-10-09T21:48:47.911968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1320 sessionId: src_id|a72e1495-59921251-a951a2e7-e710eb9a_0 is DEAD
2025-10-09T21:48:47.912023Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:47.912085Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a72e1495-59921251-a951a2e7-e710eb9a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:47.912095Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a72e1495-59921251-a951a2e7-e710eb9a_0] Write session will restart in 0.007667s
2025-10-09T21:48:47.912119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a72e1495-59921251-a951a2e7-e710eb9a_0] Write session: Do CDS request
2025-10-09T21:48:47.912126Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a72e1495-59921251-a951a2e7-e710eb9a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:47.920519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a72e1495-59921251-a951a2e7-e710eb9a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:47.920592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:47.920608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1321
2025-10-09T21:48:47.921222Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1321 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:47.921283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1321 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:47.921289Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1321 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:47.921292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:47.921703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1321 sessionId:  describe result for acl check
2025-10-09T21:48:47.921743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:47.921749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:47.921751Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:47.921756Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342274153053934:4115] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:47.922324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342274153053934:4115] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:47.923804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:47.924959Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342274153053934:4115] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:47.925677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:47.925894Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:47.925907Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:47.925919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:47.925970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:47.926009Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342273791230156:9753]
2025-10-09T21:48:47.926604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:47.926927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:47.926968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:47.927284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:47.927340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.927346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.927359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342274153053934:4115] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:47.927348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.927374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342274153053934:4115] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:47.927348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.927377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342274153053934:4115] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:47.927349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.927384Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1321 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:47.927350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.927351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.927352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.927353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.927357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.927367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.927371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.927375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.927379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.927383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.927387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.927391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.927395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.927401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.927409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:47.927415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.927420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:47.927426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.927431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:47.927435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.927440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:47.927445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.927450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:47.927454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.927460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:47.927465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.927470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.927474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.927479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.927483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.927488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.927519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:47.933546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:47.934591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:47.934603Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:47.934605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:47.934609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:47.934615Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1320 [2:7559342273791230156:9753]
2025-10-09T21:48:47.934627Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:47.934642Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:47.934645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:47.934648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:47.934650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:47.934665Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:47.934671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:47.934680Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:47.934733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.934741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.934743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.934746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:47.934817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:47.934826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.934837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:47.934855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:47.934870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:47.934905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342274153054000:4115], now have 1 active actors on pipe
2025-10-09T21:48:47.934914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342274153053854:2432], now have 1 active actors on pipe
2025-10-09T21:48:47.934922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342273791230151:9753]
2025-10-09T21:48:47.934924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342273791230151:9753]
2025-10-09T21:48:47.934926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342273791230151:9753]
2025-10-09T21:48:47.934939Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:47.934954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:47.934959Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:47.934968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:47.934958Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1320
2025-10-09T21:48:47.934971Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:47.934978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:47.934980Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:47.934992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342273791230151:9753]
2025-10-09T21:48:47.934996Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:47.934998Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:47.935016Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1320
2025-10-09T21:48:47.935017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.935028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.935062Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1de721cf-bba8a809-3a8146b6-424759e2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:47.935073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:47.935077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:47.935080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:47.935089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:47.935103Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:47.935126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:47.935255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:47.935263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:47.935277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:47.935361Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1321 partition: 0 MaxSeqNo: 25 sessionId: src_id|1de721cf-bba8a809-3a8146b6-424759e2_0
2025-10-09T21:48:47.935692Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a72e1495-59921251-a951a2e7-e710eb9a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1de721cf-bba8a809-3a8146b6-424759e2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:47.937980Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1de721cf-bba8a809-3a8146b6-424759e2_0] Write session: try to update token
2025-10-09T21:48:47.937997Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1de721cf-bba8a809-3a8146b6-424759e2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:47.948512Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:47.948533Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:47.948535Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:47.948537Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:47.948546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342273791230151:9753]
2025-10-09T21:48:47.948550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342273791230151:9753]
2025-10-09T21:48:47.948553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342273791230151:9753]
2025-10-09T21:48:47.948597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:47.948608Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:47.948610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:47.948613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:47.948614Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:47.948618Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:47.948737Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:47.953504Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1321 sessionId: src_id|1de721cf-bba8a809-3a8146b6-424759e2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:47.956137Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:48.011756Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342278448021306:4303]
2025-10-09T21:48:48.012042Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 1320, pipe: [1:7559342278448021306:4303]
2025-10-09T21:48:48.102713Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:48.103139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.103154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.103195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342278448021306:4303], now have 1 active actors on pipe
2025-10-09T21:48:48.103409Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:48.103456Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342278448021306:4303]
2025-10-09T21:48:48.103465Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:48.103487Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:48.103552Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1321 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1de721cf-bba8a809-3a8146b6-424759e2_0
2025-10-09T21:48:48.103743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1321 sessionId: src_id|1de721cf-bba8a809-3a8146b6-424759e2_0 is DEAD
2025-10-09T21:48:48.103881Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1de721cf-bba8a809-3a8146b6-424759e2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:48.103893Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1de721cf-bba8a809-3a8146b6-424759e2_0] Write session will restart in 0.007545s
2025-10-09T21:48:48.103921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1de721cf-bba8a809-3a8146b6-424759e2_0] Write session: Do CDS request
2025-10-09T21:48:48.103928Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1de721cf-bba8a809-3a8146b6-424759e2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:48.112480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1de721cf-bba8a809-3a8146b6-424759e2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:48.112590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:48.112606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1322
2025-10-09T21:48:48.112727Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1322 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:48.112761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1322 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:48.112772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1322 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:48.112776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:48.113257Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1322 sessionId:  describe result for acl check
2025-10-09T21:48:48.113301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:48.113303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:48.113305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:48.113311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342278448021407:4117] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:48.113899Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342278448021407:4117] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:48.115151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:48.116140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342278448021407:4117] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:48.117021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:48.117196Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:48.117210Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:48.117222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:48.117282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:48.117316Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342278086197599:9755]
2025-10-09T21:48:48.117736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:48.118050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:48.118083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:48.118367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342278448021407:4117] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:48.118372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:48.118382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342278448021407:4117] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:48.118388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342278448021407:4117] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:48.118396Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1322 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:48.118436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.118443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.118444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.118445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.118446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.118447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.118448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.118449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.118450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.118455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.118463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.118467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.118470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.118474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.118477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.118480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.118484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.118487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.118494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.118501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.118507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.118511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.118516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.118521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.118525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.118530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.118535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.118539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.118543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.118549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.118554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.118558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.118562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.118568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.118573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.118578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.118597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:48.124863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:48.125881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:48.125896Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:48.125899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:48.125903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:48.125909Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1321 [2:7559342278086197599:9755]
2025-10-09T21:48:48.125920Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:48.125935Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:48.125941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:48.125943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:48.125945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:48.125957Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:48.125961Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:48.125970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:48.126011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.126016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.126017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.126019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:48.126093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:48.126103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.126113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.126124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.126140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:48.126168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342278448021308:2432], now have 1 active actors on pipe
2025-10-09T21:48:48.126178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342278448021464:4117], now have 1 active actors on pipe
2025-10-09T21:48:48.126184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342278086197593:9755]
2025-10-09T21:48:48.126186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342278086197593:9755]
2025-10-09T21:48:48.126187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342278086197593:9755]
2025-10-09T21:48:48.126199Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:48.126210Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:48.126218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:48.126221Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:48.126222Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:48.126225Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:48.126228Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:48.126245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342278086197593:9755]
2025-10-09T21:48:48.126251Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:48.126256Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:48.126235Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1321
2025-10-09T21:48:48.126243Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1321
2025-10-09T21:48:48.126281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.126290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.126315Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5fea66a6-bc72716-9b3159be-b760781b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:48.126329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:48.126333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:48.126340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:48.126351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:48.126365Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:48.126383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:48.126520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.126529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.126541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:48.126639Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1322 partition: 0 MaxSeqNo: 25 sessionId: src_id|5fea66a6-bc72716-9b3159be-b760781b_0
2025-10-09T21:48:48.126945Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1de721cf-bba8a809-3a8146b6-424759e2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5fea66a6-bc72716-9b3159be-b760781b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:48.129429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5fea66a6-bc72716-9b3159be-b760781b_0] Write session: try to update token
2025-10-09T21:48:48.129449Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5fea66a6-bc72716-9b3159be-b760781b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:48.143482Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1322 sessionId: src_id|5fea66a6-bc72716-9b3159be-b760781b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:48.143941Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:48.143960Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:48.143970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342278086197593:9755]
2025-10-09T21:48:48.144021Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:48.144024Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:48.144214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:48.144368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:48.144509Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:48.144650Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:48.144836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:48.144876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:48.146087Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:48.146276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.146300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.146334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:48.146422Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:48.151204Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:48.151220Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:48.151223Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:48.151225Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:48.151235Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342278086197593:9755]
2025-10-09T21:48:48.151238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342278086197593:9755]
2025-10-09T21:48:48.151242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342278086197593:9755]
2025-10-09T21:48:48.151280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:48.151285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.151287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:48.151289Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:48.151298Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:48.151302Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:48.151407Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:48.203792Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342278448021477:4303]
2025-10-09T21:48:48.204111Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 1321, pipe: [1:7559342278448021477:4303]
2025-10-09T21:48:48.310911Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:48.318361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.318386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.318523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:48.318578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:48.318645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:48.318714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:48.318761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:48.318777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:48.318843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:48.318902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:48.318967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:48.319030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:48.319071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:48.319078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:48.319136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:48.319191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:48.319250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:48.319310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:48.319350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:48.319358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:48.319419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:48.319476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:48.319536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:48.319594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:48.319636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:48.319644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:48.319704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:48.319762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:48.319819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:48.319876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:48.319917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:48.319925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:48.319984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:48.320043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:48.320105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:48.320163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:48.320226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:48.320231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:48.320289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:48.320360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:48.320419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:48.320476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:48.320516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:48.320524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:48.320580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:48.320636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:48.320694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:48.320754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:48.320794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:48.320803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:48.320860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:48.320913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:48.320968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:48.321025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:48.321063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:48.321071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:48.321142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:48.321195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:48.321252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:48.321315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:48.321357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:48.321364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:48.321422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342278448021477:4303], now have 1 active actors on pipe
2025-10-09T21:48:48.321700Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:48.321727Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:48.321731Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:48.321756Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342278448021477:4303]
2025-10-09T21:48:48.321761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1322 reason: error at write: Disconnected sessionId: src_id|5fea66a6-bc72716-9b3159be-b760781b_0
2025-10-09T21:48:48.321942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1322 sessionId: src_id|5fea66a6-bc72716-9b3159be-b760781b_0 is DEAD
2025-10-09T21:48:48.322102Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fea66a6-bc72716-9b3159be-b760781b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:48.322113Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fea66a6-bc72716-9b3159be-b760781b_0] Write session will restart in 0.009209s
2025-10-09T21:48:48.322138Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fea66a6-bc72716-9b3159be-b760781b_0] Write session: Do CDS request
2025-10-09T21:48:48.322145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fea66a6-bc72716-9b3159be-b760781b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:48.322481Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:48.332593Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5fea66a6-bc72716-9b3159be-b760781b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:48.332734Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:48.332763Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1323
2025-10-09T21:48:48.332909Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1323 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:48.332944Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1323 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:48.332956Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1323 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:48.332960Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:48.333466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1323 sessionId:  describe result for acl check
2025-10-09T21:48:48.333506Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:48.333515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:48.333518Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:48.333523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342278448021589:4103] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:48.334149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342278448021589:4103] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:48.335356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:48.336414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342278448021589:4103] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:48.337434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:48.337636Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:48.337646Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:48.337658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:48.337718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:48.337756Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342278086197777:9763]
2025-10-09T21:48:48.338341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:48.338629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:48.338670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:48.338970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:48.339014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.339015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.339016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.339016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.339017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.339018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.339018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.339019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.339013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342278448021589:4103] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:48.339019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.339029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342278448021589:4103] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:48.339024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.339032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342278448021589:4103] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:48.339033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.339040Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1323 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:48.339036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.339040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.339043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.339047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.339050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.339053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.339057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.339063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.339072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.339078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.339083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.339087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.339092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.339096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.339100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.339104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.339109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.339113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.339117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.339123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.339127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.339131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.339135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.339140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.339144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.339176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:48.345546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:48.346670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:48.346686Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:48.346689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:48.346694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:48.346701Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1322 [2:7559342278086197777:9763]
2025-10-09T21:48:48.346717Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:48.346739Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:48.346749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:48.346752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:48.346755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:48.346773Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:48.346779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:48.346787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:48.346845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.346853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.346855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.346857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:48.346927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:48.346936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.346950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.346967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.346989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:48.347013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342278448021484:2432], now have 1 active actors on pipe
2025-10-09T21:48:48.347022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342278448021631:4103], now have 1 active actors on pipe
2025-10-09T21:48:48.347030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342278086197759:9763]
2025-10-09T21:48:48.347032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342278086197759:9763]
2025-10-09T21:48:48.347034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342278086197759:9763]
2025-10-09T21:48:48.347048Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:48.347058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:48.347060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:48.347061Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:48.347063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:48.347077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:48.347080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:48.347074Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1322
2025-10-09T21:48:48.347101Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1322
2025-10-09T21:48:48.347111Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342278086197759:9763]
2025-10-09T21:48:48.347120Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:48.347122Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:48.347141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.347160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.347209Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e327bbce-c6aa2ac6-53da13d3-bd116a03_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:48.347221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:48.347225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:48.347234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:48.347245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:48.347255Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:48.347277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:48.347441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.347448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.347462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:48.347558Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1323 partition: 0 MaxSeqNo: 25 sessionId: src_id|e327bbce-c6aa2ac6-53da13d3-bd116a03_0
2025-10-09T21:48:48.347898Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fea66a6-bc72716-9b3159be-b760781b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e327bbce-c6aa2ac6-53da13d3-bd116a03_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:48.350442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e327bbce-c6aa2ac6-53da13d3-bd116a03_0] Write session: try to update token
2025-10-09T21:48:48.350457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e327bbce-c6aa2ac6-53da13d3-bd116a03_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:48.361346Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:48.361370Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:48.361372Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:48.361374Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:48.361383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342278086197759:9763]
2025-10-09T21:48:48.361387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342278086197759:9763]
2025-10-09T21:48:48.361393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342278086197759:9763]
2025-10-09T21:48:48.361440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:48.361439Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:48.361446Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:48.361448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.361449Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:48.361451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:48.361586Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:48.364447Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1323 sessionId: src_id|e327bbce-c6aa2ac6-53da13d3-bd116a03_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:48.366878Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:48.421884Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342278448021664:4303]
2025-10-09T21:48:48.422194Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 1322, pipe: [1:7559342278448021664:4303]
2025-10-09T21:48:48.511597Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:48.511959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.511976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.512023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342278448021664:4303], now have 1 active actors on pipe
2025-10-09T21:48:48.512202Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:48.512239Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342278448021664:4303]
2025-10-09T21:48:48.512264Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:48.512271Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:48.512355Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1323 reason: error at write: Disconnected sessionId: src_id|e327bbce-c6aa2ac6-53da13d3-bd116a03_0
2025-10-09T21:48:48.512532Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1323 sessionId: src_id|e327bbce-c6aa2ac6-53da13d3-bd116a03_0 is DEAD
2025-10-09T21:48:48.512676Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e327bbce-c6aa2ac6-53da13d3-bd116a03_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:48.512699Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e327bbce-c6aa2ac6-53da13d3-bd116a03_0] Write session will restart in 0.005541s
2025-10-09T21:48:48.512724Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e327bbce-c6aa2ac6-53da13d3-bd116a03_0] Write session: Do CDS request
2025-10-09T21:48:48.512734Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e327bbce-c6aa2ac6-53da13d3-bd116a03_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:48.513368Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:48.519485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e327bbce-c6aa2ac6-53da13d3-bd116a03_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:48.519576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:48.519605Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1324
2025-10-09T21:48:48.519736Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1324 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:48.519767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1324 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:48.519776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1324 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:48.519782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:48.520144Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1324 sessionId:  describe result for acl check
2025-10-09T21:48:48.520185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:48.520192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:48.520194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:48.520201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342278448021693:4083] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:48.520640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342278448021693:4083] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:48.523113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342278448021693:4083] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:48.523751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:48.525730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:48.525927Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:48.525936Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:48.525948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:48.526003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:48.526057Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342278086197934:9771]
2025-10-09T21:48:48.526192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342278448021693:4083] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:48.526206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342278448021693:4083] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:48.526210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342278448021693:4083] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:48.526218Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1324 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:48.526617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:48.526884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:48.526921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:48.527204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:48.527245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.527251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.527253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.527254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.527255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.527256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.527257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.527258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.527259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.527263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.527273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.527277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.527281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.527284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.527288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.527292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.527295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.527299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.527304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.527311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.527317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.527323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.527327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.527332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.527336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.527340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.527345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.527350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.527354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.527359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.527363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.527368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.527372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.527376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.527379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.527383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.527412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:48.533576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:48.534641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:48.534658Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:48.534661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:48.534665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:48.534672Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1323 [2:7559342278086197934:9771]
2025-10-09T21:48:48.534687Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:48.534707Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:48.534717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:48.534720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:48.534723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:48.534741Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:48.534752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:48.534759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:48.534824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.534832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.534834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.534837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:48.534914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:48.534925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.534939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.534952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.534971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:48.535001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342278448021814:4083], now have 1 active actors on pipe
2025-10-09T21:48:48.535005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342278448021667:2432], now have 1 active actors on pipe
2025-10-09T21:48:48.535013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342278086197928:9771]
2025-10-09T21:48:48.535016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342278086197928:9771]
2025-10-09T21:48:48.535018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342278086197928:9771]
2025-10-09T21:48:48.535033Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:48.535057Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:48.535070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:48.535068Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:48.535073Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:48.535074Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:48.535077Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:48.535061Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1323
2025-10-09T21:48:48.535073Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1323
2025-10-09T21:48:48.535109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342278086197928:9771]
2025-10-09T21:48:48.535121Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:48.535125Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:48.535148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.535154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.535208Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|41627619-6530ee1e-75fc4f7c-2650d6cb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:48.535217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:48.535222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:48.535224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:48.535235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:48.535248Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:48.535274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:48.535448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.535457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.535474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:48.535565Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1324 partition: 0 MaxSeqNo: 25 sessionId: src_id|41627619-6530ee1e-75fc4f7c-2650d6cb_0
2025-10-09T21:48:48.535896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e327bbce-c6aa2ac6-53da13d3-bd116a03_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|41627619-6530ee1e-75fc4f7c-2650d6cb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:48.538248Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|41627619-6530ee1e-75fc4f7c-2650d6cb_0] Write session: try to update token
2025-10-09T21:48:48.538265Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|41627619-6530ee1e-75fc4f7c-2650d6cb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:48.549281Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:48.549299Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:48.549302Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:48.549304Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:48.549315Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342278086197928:9771]
2025-10-09T21:48:48.549319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342278086197928:9771]
2025-10-09T21:48:48.549323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342278086197928:9771]
2025-10-09T21:48:48.549365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:48.549372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.549374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:48.549373Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:48.549380Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:48.549384Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:48.549509Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:48.551002Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1324 sessionId: src_id|41627619-6530ee1e-75fc4f7c-2650d6cb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:48.553188Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:48.612367Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342278448021825:4303]
2025-10-09T21:48:48.612657Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 1323, pipe: [1:7559342278448021825:4303]
2025-10-09T21:48:48.696066Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:48.696501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.696515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.696556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342278448021825:4303], now have 1 active actors on pipe
2025-10-09T21:48:48.696787Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:48.696862Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342278448021825:4303]
2025-10-09T21:48:48.696892Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:48.696979Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:48.697055Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1324 reason: status is not ok: tablet will be restarted right now sessionId: src_id|41627619-6530ee1e-75fc4f7c-2650d6cb_0
2025-10-09T21:48:48.697224Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1324 sessionId: src_id|41627619-6530ee1e-75fc4f7c-2650d6cb_0 is DEAD
2025-10-09T21:48:48.697376Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41627619-6530ee1e-75fc4f7c-2650d6cb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:48.697388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41627619-6530ee1e-75fc4f7c-2650d6cb_0] Write session will restart in 0.005507s
2025-10-09T21:48:48.697413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41627619-6530ee1e-75fc4f7c-2650d6cb_0] Write session: Do CDS request
2025-10-09T21:48:48.697420Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41627619-6530ee1e-75fc4f7c-2650d6cb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:48.704497Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|41627619-6530ee1e-75fc4f7c-2650d6cb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:48.704588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:48.704606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1325
2025-10-09T21:48:48.704718Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1325 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:48.704750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1325 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:48.704760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1325 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:48.704769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:48.705255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1325 sessionId:  describe result for acl check
2025-10-09T21:48:48.705291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:48.705292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:48.705294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:48.705298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342278448021886:4078] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:48.705835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342278448021886:4078] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:48.708020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342278448021886:4078] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:48.708288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:48.710088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:48.710289Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:48.710297Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:48.710308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:48.710365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:48.710405Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342278086198088:9775]
2025-10-09T21:48:48.710414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342278448021886:4078] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:48.710427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342278448021886:4078] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:48.710430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342278448021886:4078] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:48.710440Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1325 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:48.710977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:48.711283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:48.711328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:48.711642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:48.711689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.711696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.711697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.711698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.711700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.711701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.711702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.711703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.711704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.711709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.711720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.711724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.711728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.711732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.711736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.711740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.711743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.711747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.711754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.711763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.711769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.711775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.711779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.711784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.711789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.711794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.711798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.711804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.711812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.711818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.711822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.711831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.711837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.711842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.711850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.711854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.711881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:48.718234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:48.719304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:48.719321Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:48.719324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:48.719329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:48.719336Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1324 [2:7559342278086198088:9775]
2025-10-09T21:48:48.719352Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:48.719371Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:48.719379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:48.719382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:48.719384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:48.719405Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:48.719416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:48.719424Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:48.719500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.719508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.719511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.719514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:48.719590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:48.719598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.719610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.719620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.719641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:48.719671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342278448021976:4078], now have 1 active actors on pipe
2025-10-09T21:48:48.719675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342278448021828:2432], now have 1 active actors on pipe
2025-10-09T21:48:48.719683Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342278086198082:9775]
2025-10-09T21:48:48.719686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342278086198082:9775]
2025-10-09T21:48:48.719688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342278086198082:9775]
2025-10-09T21:48:48.719701Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:48.719713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:48.719718Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:48.719721Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:48.719722Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:48.719734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:48.719737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:48.719727Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1324
2025-10-09T21:48:48.719735Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1324
2025-10-09T21:48:48.719778Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342278086198082:9775]
2025-10-09T21:48:48.719790Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:48.719797Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:48.719807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.719812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.719866Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2771442d-cade0432-247aa315-ebc8c80f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:48.719881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:48.719885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:48.719888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:48.719899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:48.719911Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:48.719935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:48.720072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.720082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.720096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:48.720164Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1325 partition: 0 MaxSeqNo: 25 sessionId: src_id|2771442d-cade0432-247aa315-ebc8c80f_0
2025-10-09T21:48:48.720508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41627619-6530ee1e-75fc4f7c-2650d6cb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2771442d-cade0432-247aa315-ebc8c80f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:48.722927Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2771442d-cade0432-247aa315-ebc8c80f_0] Write session: try to update token
2025-10-09T21:48:48.722945Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2771442d-cade0432-247aa315-ebc8c80f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:48.734191Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:48.734208Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:48.734211Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:48.734213Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:48.734223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342278086198082:9775]
2025-10-09T21:48:48.734228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342278086198082:9775]
2025-10-09T21:48:48.734232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342278086198082:9775]
2025-10-09T21:48:48.734278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:48.734283Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:48.734289Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:48.734290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.734293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:48.734294Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:48.734452Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:48.735657Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1325 sessionId: src_id|2771442d-cade0432-247aa315-ebc8c80f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:48.738004Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:48.797196Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342278448021987:4303]
2025-10-09T21:48:48.797491Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 1324, pipe: [1:7559342278448021987:4303]
2025-10-09T21:48:48.882590Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:48.883002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.883018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.883060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342278448021987:4303], now have 1 active actors on pipe
2025-10-09T21:48:48.883239Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:48.883308Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342278448021987:4303]
2025-10-09T21:48:48.883334Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:48.883393Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:48.883483Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1325 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2771442d-cade0432-247aa315-ebc8c80f_0
2025-10-09T21:48:48.883665Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1325 sessionId: src_id|2771442d-cade0432-247aa315-ebc8c80f_0 is DEAD
2025-10-09T21:48:48.883808Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2771442d-cade0432-247aa315-ebc8c80f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:48.883820Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2771442d-cade0432-247aa315-ebc8c80f_0] Write session will restart in 0.006220s
2025-10-09T21:48:48.883856Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2771442d-cade0432-247aa315-ebc8c80f_0] Write session: Do CDS request
2025-10-09T21:48:48.883863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2771442d-cade0432-247aa315-ebc8c80f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:48.891437Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2771442d-cade0432-247aa315-ebc8c80f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:48.891510Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:48.891537Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1326
2025-10-09T21:48:48.891653Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1326 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:48.891684Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1326 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:48.891693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1326 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:48.891697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:48.892275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1326 sessionId:  describe result for acl check
2025-10-09T21:48:48.892312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:48.892314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:48.892315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:48.892322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342278448022072:4075] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:48.892848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342278448022072:4075] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:48.894417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:48.894868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342278448022072:4075] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:48.896380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:48.896540Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:48.896548Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:48.896557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:48.896599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:48.896631Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342278086198239:9777]
2025-10-09T21:48:48.897048Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342278448022072:4075] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:48.897058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:48.897061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342278448022072:4075] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:48.897066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342278448022072:4075] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:48.897084Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1326 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:48.897291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:48.897318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:48.897616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:48.897654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.897655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.897656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.897657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.897658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.897659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.897660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.897661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.897662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.897674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.897684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.897688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.897691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.897695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.897699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.897703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.897707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.897709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.897714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.897720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:48.897725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.897728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:48.897731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.897734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:48.897736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.897739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:48.897743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.897746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:48.897749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.897752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:48.897754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.897757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.897760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.897763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.897765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.897768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.897784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:48.904177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:48.905252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:48.905268Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:48.905270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:48.905274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:48.905279Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1325 [2:7559342278086198239:9777]
2025-10-09T21:48:48.905289Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:48.905306Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:48.905308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:48.905311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:48.905315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:48.905333Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:48.905344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:48.905356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:48.905424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.905433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.905435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.905438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:48.905677Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1325
2025-10-09T21:48:48.905680Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1325
2025-10-09T21:48:48.905729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:48.905737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.905751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:48.905762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:48.905773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:48.905792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342278448022141:4075], now have 1 active actors on pipe
2025-10-09T21:48:48.905801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342278448021992:2432], now have 1 active actors on pipe
2025-10-09T21:48:48.905817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.905820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.905834Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342278086198233:9777]
2025-10-09T21:48:48.905839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342278086198233:9777]
2025-10-09T21:48:48.905841Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342278086198233:9777]
2025-10-09T21:48:48.905851Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:48.905867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:48.905867Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:48.905870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:48.905872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:48.905876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:48.905880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:48.905896Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|113b911c-87995b4d-e6a51989-7569be04_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:48.905905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:48.905908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:48.905910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:48.905917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:48.905927Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:48.905963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342278086198233:9777]
2025-10-09T21:48:48.905971Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:48.905980Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:48.905982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:48.906117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.906123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.906134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:48.906215Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1326 partition: 0 MaxSeqNo: 25 sessionId: src_id|113b911c-87995b4d-e6a51989-7569be04_0
2025-10-09T21:48:48.906617Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2771442d-cade0432-247aa315-ebc8c80f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|113b911c-87995b4d-e6a51989-7569be04_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:48.909264Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|113b911c-87995b4d-e6a51989-7569be04_0] Write session: try to update token
2025-10-09T21:48:48.909282Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|113b911c-87995b4d-e6a51989-7569be04_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:48.916948Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:48.916966Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:48.916976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342278086198233:9777]
2025-10-09T21:48:48.917017Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:48.917022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:48.917230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:48.917760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:48.917904Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:48.918072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:48.918336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:48.918380Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:48.922684Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1326 sessionId: src_id|113b911c-87995b4d-e6a51989-7569be04_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:48.924939Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:48.925149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:48.925171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:48.925212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:48.925292Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:48.930108Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:48.930122Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:48.930124Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:48.930126Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:48.930136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342278086198233:9777]
2025-10-09T21:48:48.930140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342278086198233:9777]
2025-10-09T21:48:48.930143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342278086198233:9777]
2025-10-09T21:48:48.930183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:48.930189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:48.930194Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:48.930196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:48.930200Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:48.930205Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:48.930318Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:48.983553Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342278448022152:4303]
2025-10-09T21:48:48.983799Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 1325, pipe: [1:7559342278448022152:4303]
2025-10-09T21:48:49.082808Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:49.090368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.090391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.090538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.090593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.090662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.090725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.090767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.090779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.090854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.090911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.090968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.091029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.091070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.091076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.091132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.091216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.091272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.091328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.091368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.091380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.091447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.091511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.091574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.091652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.091699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.091705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.091769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.091823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.091878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.091939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.091990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.091996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.092056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.092120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.092180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.092238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.092278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.092284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.092341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.092397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.092454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.092514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.092560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.092568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.092626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.092754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.092824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.092888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.092930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.092938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.092994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.093042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.093098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.093156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.093203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.093210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.093272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.093337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.093398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.093455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.093493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.093499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.093558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342278448022152:4303], now have 1 active actors on pipe
2025-10-09T21:48:49.093818Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:49.093848Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342278448022152:4303]
2025-10-09T21:48:49.093884Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:49.093893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:49.093922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1326 reason: error at write: Disconnected sessionId: src_id|113b911c-87995b4d-e6a51989-7569be04_0
2025-10-09T21:48:49.094185Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1326 sessionId: src_id|113b911c-87995b4d-e6a51989-7569be04_0 is DEAD
2025-10-09T21:48:49.094300Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|113b911c-87995b4d-e6a51989-7569be04_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:49.094311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|113b911c-87995b4d-e6a51989-7569be04_0] Write session will restart in 0.009954s
2025-10-09T21:48:49.094336Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|113b911c-87995b4d-e6a51989-7569be04_0] Write session: Do CDS request
2025-10-09T21:48:49.094342Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|113b911c-87995b4d-e6a51989-7569be04_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:49.094546Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:49.105565Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|113b911c-87995b4d-e6a51989-7569be04_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:49.105639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:49.105660Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1327
2025-10-09T21:48:49.105862Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1327 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:49.105903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1327 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:49.105909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1327 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:49.105913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:49.106365Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1327 sessionId:  describe result for acl check
2025-10-09T21:48:49.106406Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:49.106413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:49.106415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:49.106421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342282742989566:4060] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:49.106982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342282742989566:4060] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:49.107729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:49.109957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342282742989566:4060] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:49.110081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:49.110268Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:49.110279Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:49.110289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:49.110339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:49.110374Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342282381165688:9779]
2025-10-09T21:48:49.110914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:49.111271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:49.111317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:49.111636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:49.111692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.111694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.111695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.111696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.111698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.111699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.111701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.111702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.111703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.111709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.111721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.111725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.111729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.111732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.111737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.111747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.111751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.111756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.111768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.111782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.111793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.111804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.111813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.111819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.111828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.111838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.111847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.111857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.111865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.111875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.111884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.111894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.111902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.111908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.111917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.111927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.111962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:49.113080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342282742989566:4060] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:49.113098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342282742989566:4060] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:49.113103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342282742989566:4060] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:49.113115Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1327 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:49.118986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:49.120347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:49.120366Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:49.120369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:49.120375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:49.120383Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1326 [2:7559342282381165688:9779]
2025-10-09T21:48:49.120399Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:49.120423Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:49.120431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:49.120435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:49.120438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:49.120459Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:49.120476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:49.120482Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:49.120541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.120546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.120547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.120551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:49.120623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:49.120629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.120640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.120648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.120657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:49.120677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342282381165683:9779]
2025-10-09T21:48:49.120683Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342282381165683:9779]
2025-10-09T21:48:49.120684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342282381165683:9779]
2025-10-09T21:48:49.120694Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:49.120710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342282742989607:4060], now have 1 active actors on pipe
2025-10-09T21:48:49.120711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:49.120714Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:49.120716Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:49.120720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342282742989453:2432], now have 1 active actors on pipe
2025-10-09T21:48:49.120729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:49.120744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:49.120752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:49.120800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342282381165683:9779]
2025-10-09T21:48:49.120815Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:49.120806Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1326
2025-10-09T21:48:49.120838Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:49.120806Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1326
2025-10-09T21:48:49.120881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.120893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.120928Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|aaaf5a66-5da091f2-87e2c36-2398d1bd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:49.120942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:49.120949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:49.120952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:49.120965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:49.120981Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:49.121005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:49.121162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.121170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.121184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:49.121268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1327 partition: 0 MaxSeqNo: 25 sessionId: src_id|aaaf5a66-5da091f2-87e2c36-2398d1bd_0
2025-10-09T21:48:49.121649Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|113b911c-87995b4d-e6a51989-7569be04_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|aaaf5a66-5da091f2-87e2c36-2398d1bd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:49.124318Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aaaf5a66-5da091f2-87e2c36-2398d1bd_0] Write session: try to update token
2025-10-09T21:48:49.124341Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aaaf5a66-5da091f2-87e2c36-2398d1bd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:49.139242Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:49.139262Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:49.139274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342282381165683:9779]
2025-10-09T21:48:49.139332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:49.139353Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:49.139520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:49.139671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:49.139826Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:49.139975Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:49.140173Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:49.140223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:49.140930Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1327 sessionId: src_id|aaaf5a66-5da091f2-87e2c36-2398d1bd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:49.143492Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:49.143714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.143736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.143782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:49.143864Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:49.146420Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:49.146437Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:49.146439Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:49.146442Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:49.146452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342282381165683:9779]
2025-10-09T21:48:49.146457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342282381165683:9779]
2025-10-09T21:48:49.146462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342282381165683:9779]
2025-10-09T21:48:49.146507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:49.146520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.146523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:49.146606Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:49.146624Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:49.146629Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:49.146674Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:49.193995Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342282742989618:4303]
2025-10-09T21:48:49.194322Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 1326, pipe: [1:7559342282742989618:4303]
2025-10-09T21:48:49.306407Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:49.314353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.314386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.314558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.314619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.314688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.314748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.314797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.314807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.314875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.314931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.314993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.315057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.315100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.315109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.315166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.315222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.315287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.315351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.315393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.315403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.315462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.315523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.315584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.315644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.315685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.315692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.315759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.315820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.315881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.315948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.315990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.315997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.316063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.316122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.316180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.316235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.316279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.316290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.316353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.316410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.316473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.316536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.316581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.316589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.316654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.316718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.316781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.316841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.316886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.316903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.316967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.317028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.317093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.317157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.317206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.317214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.317278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.317342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.317407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.317479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.317585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.317593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.317660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342282742989618:4303], now have 1 active actors on pipe
2025-10-09T21:48:49.317928Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:49.317987Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342282742989618:4303]
2025-10-09T21:48:49.318076Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:49.318091Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:49.318133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1327 reason: error at write: Disconnected sessionId: src_id|aaaf5a66-5da091f2-87e2c36-2398d1bd_0
2025-10-09T21:48:49.318335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1327 sessionId: src_id|aaaf5a66-5da091f2-87e2c36-2398d1bd_0 is DEAD
2025-10-09T21:48:49.318449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aaaf5a66-5da091f2-87e2c36-2398d1bd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:49.318460Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aaaf5a66-5da091f2-87e2c36-2398d1bd_0] Write session will restart in 0.007579s
2025-10-09T21:48:49.318487Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aaaf5a66-5da091f2-87e2c36-2398d1bd_0] Write session: Do CDS request
2025-10-09T21:48:49.318492Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aaaf5a66-5da091f2-87e2c36-2398d1bd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:49.318739Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:49.327555Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aaaf5a66-5da091f2-87e2c36-2398d1bd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:49.327649Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:49.327668Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1328
2025-10-09T21:48:49.327802Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1328 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:49.327836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1328 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:49.327846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1328 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:49.327850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:49.328268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1328 sessionId:  describe result for acl check
2025-10-09T21:48:49.328305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:49.328312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:49.328314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:49.328320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342282742989688:4055] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:49.328882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342282742989688:4055] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:49.331053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:49.331511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342282742989688:4055] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:49.332907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:49.333097Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:49.333108Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:49.333120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:49.333178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:49.333219Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342282381165841:9781]
2025-10-09T21:48:49.333798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:49.334059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:49.334106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:49.334202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342282742989688:4055] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:49.334217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342282742989688:4055] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:49.334221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342282742989688:4055] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:49.334230Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1328 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:49.334411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:49.334459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.334464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.334466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.334467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.334468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.334469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.334470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.334472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.334473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.334478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.334489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.334493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.334496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.334500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.334504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.334508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.334513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.334516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.334523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.334531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.334538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.334543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.334548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.334558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.334563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.334573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.334578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.334583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.334591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.334597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.334602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.334607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.334616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.334621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.334625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.334630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.334660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:49.340822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:49.341871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:49.341888Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:49.341891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:49.341895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:49.341902Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1327 [2:7559342282381165841:9781]
2025-10-09T21:48:49.341915Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:49.341930Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:49.341938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:49.341941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:49.341944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:49.341959Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:49.341965Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:49.341975Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:49.342050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.342057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.342059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.342062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:49.342123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:49.342134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.342146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.342155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.342168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:49.342191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342282742989778:4055], now have 1 active actors on pipe
2025-10-09T21:48:49.342201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342282742989625:2432], now have 1 active actors on pipe
2025-10-09T21:48:49.342209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342282381165836:9781]
2025-10-09T21:48:49.342212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342282381165836:9781]
2025-10-09T21:48:49.342214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342282381165836:9781]
2025-10-09T21:48:49.342222Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:49.342246Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1327
2025-10-09T21:48:49.342250Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1327
2025-10-09T21:48:49.342237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:49.342238Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:49.342240Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:49.342242Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:49.342255Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:49.342259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:49.342288Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342282381165836:9781]
2025-10-09T21:48:49.342291Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:49.342298Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:49.342307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.342312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.342365Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9916b344-9382ef40-1b45519f-94480e7a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:49.342379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:49.342386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:49.342389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:49.342400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:49.342410Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:49.342433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:49.342620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.342635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.342658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:49.342754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1328 partition: 0 MaxSeqNo: 25 sessionId: src_id|9916b344-9382ef40-1b45519f-94480e7a_0
2025-10-09T21:48:49.343140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aaaf5a66-5da091f2-87e2c36-2398d1bd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9916b344-9382ef40-1b45519f-94480e7a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:49.345856Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9916b344-9382ef40-1b45519f-94480e7a_0] Write session: try to update token
2025-10-09T21:48:49.345873Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9916b344-9382ef40-1b45519f-94480e7a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:49.356767Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:49.356786Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:49.356788Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:49.356790Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:49.356802Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342282381165836:9781]
2025-10-09T21:48:49.356807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342282381165836:9781]
2025-10-09T21:48:49.356810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342282381165836:9781]
2025-10-09T21:48:49.356849Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:49.356854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:49.356856Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:49.356860Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:49.356861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.356864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:49.357118Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:49.359643Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1328 sessionId: src_id|9916b344-9382ef40-1b45519f-94480e7a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:49.362010Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:49.418200Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342282742989801:4303]
2025-10-09T21:48:49.418587Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 1327, pipe: [1:7559342282742989801:4303]
2025-10-09T21:48:49.510686Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:49.511253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.511271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.511317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342282742989801:4303], now have 1 active actors on pipe
2025-10-09T21:48:49.511554Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:49.511540Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342282742989801:4303]
2025-10-09T21:48:49.511561Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:49.511565Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:49.511572Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:49.511643Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1328 reason: error at write: Disconnected sessionId: src_id|9916b344-9382ef40-1b45519f-94480e7a_0
2025-10-09T21:48:49.511856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1328 sessionId: src_id|9916b344-9382ef40-1b45519f-94480e7a_0 is DEAD
2025-10-09T21:48:49.512008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9916b344-9382ef40-1b45519f-94480e7a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:49.512019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9916b344-9382ef40-1b45519f-94480e7a_0] Write session will restart in 0.006569s
2025-10-09T21:48:49.512062Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9916b344-9382ef40-1b45519f-94480e7a_0] Write session: Do CDS request
2025-10-09T21:48:49.512071Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9916b344-9382ef40-1b45519f-94480e7a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:49.519588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9916b344-9382ef40-1b45519f-94480e7a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:49.519657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:49.519678Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1329
2025-10-09T21:48:49.519895Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1329 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:49.519935Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1329 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:49.519946Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1329 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:49.519951Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:49.520408Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1329 sessionId:  describe result for acl check
2025-10-09T21:48:49.520442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:49.520447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:49.520448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:49.520453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342282742989837:4038] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:49.520958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342282742989837:4038] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:49.523418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342282742989837:4038] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:49.525097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:49.527059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:49.527260Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:49.527270Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:49.527282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:49.527350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:49.527392Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342282381166033:9795]
2025-10-09T21:48:49.527646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342282742989837:4038] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:49.527662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342282742989837:4038] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:49.527666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342282742989837:4038] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:49.527675Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1329 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:49.528013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:49.528325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:49.528378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:49.528710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:49.528755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.528761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.528762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.528763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.528764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.528765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.528767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.528768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.528769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.528774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.528786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.528790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.528794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.528798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.528802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.528806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.528810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.528814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.528821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.528828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.528833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.528836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.528838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.528841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.528844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.528847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.528849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.528852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.528854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.528858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.528861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.528864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.528866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.528870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.528873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.528876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.528892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:49.535487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:49.537222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:49.537239Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:49.537242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:49.537248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:49.537255Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1328 [2:7559342282381166033:9795]
2025-10-09T21:48:49.537269Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:49.537289Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:49.537298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:49.537301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:49.537304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:49.537324Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:49.537334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:49.537343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:49.537413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.537420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.537423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.537426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:49.537499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:49.537510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.537524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.537536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.537554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:49.537582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342282742989965:4038], now have 1 active actors on pipe
2025-10-09T21:48:49.537591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342282742989812:2432], now have 1 active actors on pipe
2025-10-09T21:48:49.537599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342282381166028:9795]
2025-10-09T21:48:49.537601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342282381166028:9795]
2025-10-09T21:48:49.537604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342282381166028:9795]
2025-10-09T21:48:49.537616Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:49.537630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:49.537630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:49.537633Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:49.537636Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:49.537649Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:49.537653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:49.537661Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1328
2025-10-09T21:48:49.537695Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1328
2025-10-09T21:48:49.537709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342282381166028:9795]
2025-10-09T21:48:49.537718Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:49.537721Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:49.537735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.537740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.537797Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2c6ad281-5fd5a31e-d8f1dd1b-722eb01f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:49.537811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:49.537815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:49.537820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:49.537832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:49.537847Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:49.537873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:49.538053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.538068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.538102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:49.538192Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1329 partition: 0 MaxSeqNo: 25 sessionId: src_id|2c6ad281-5fd5a31e-d8f1dd1b-722eb01f_0
2025-10-09T21:48:49.538554Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9916b344-9382ef40-1b45519f-94480e7a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2c6ad281-5fd5a31e-d8f1dd1b-722eb01f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:49.541191Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c6ad281-5fd5a31e-d8f1dd1b-722eb01f_0] Write session: try to update token
2025-10-09T21:48:49.541211Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c6ad281-5fd5a31e-d8f1dd1b-722eb01f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:49.553303Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:49.553325Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:49.553328Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:49.553331Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:49.553340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342282381166028:9795]
2025-10-09T21:48:49.553344Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342282381166028:9795]
2025-10-09T21:48:49.553348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342282381166028:9795]
2025-10-09T21:48:49.553392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:49.553399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.553402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:49.553408Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:49.553414Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:49.553418Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:49.553552Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:49.555688Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1329 sessionId: src_id|2c6ad281-5fd5a31e-d8f1dd1b-722eb01f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:49.558189Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:49.611759Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342282742989976:4303]
2025-10-09T21:48:49.612024Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 1328, pipe: [1:7559342282742989976:4303]
2025-10-09T21:48:49.706705Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:49.707401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.707419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.707467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342282742989976:4303], now have 1 active actors on pipe
2025-10-09T21:48:49.707683Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:49.707689Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:49.707706Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:49.707726Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342282742989976:4303]
2025-10-09T21:48:49.707748Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:49.707770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1329 reason: error at write: Disconnected sessionId: src_id|2c6ad281-5fd5a31e-d8f1dd1b-722eb01f_0
2025-10-09T21:48:49.707937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1329 sessionId: src_id|2c6ad281-5fd5a31e-d8f1dd1b-722eb01f_0 is DEAD
2025-10-09T21:48:49.708105Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c6ad281-5fd5a31e-d8f1dd1b-722eb01f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:49.708118Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c6ad281-5fd5a31e-d8f1dd1b-722eb01f_0] Write session will restart in 0.008436s
2025-10-09T21:48:49.708148Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c6ad281-5fd5a31e-d8f1dd1b-722eb01f_0] Write session: Do CDS request
2025-10-09T21:48:49.708155Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c6ad281-5fd5a31e-d8f1dd1b-722eb01f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:49.717614Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c6ad281-5fd5a31e-d8f1dd1b-722eb01f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:49.717712Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:49.717729Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1330
2025-10-09T21:48:49.717851Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1330 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:49.717876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1330 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:49.717884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1330 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:49.717886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:49.718371Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1330 sessionId:  describe result for acl check
2025-10-09T21:48:49.718414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:49.718421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:49.718423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:49.718429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342282742990047:4033] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:49.719183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342282742990047:4033] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:49.721262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:49.721300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342282742990047:4033] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:49.723561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:49.723809Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:49.723821Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:49.723841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:49.723900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:49.723947Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342282381166192:9799]
2025-10-09T21:48:49.723990Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342282742990047:4033] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:49.724007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342282742990047:4033] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:49.724011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342282742990047:4033] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:49.724022Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1330 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:49.724545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:49.724864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:49.724925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:49.725176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:49.725222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.725228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.725230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.725231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.725232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.725233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.725234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.725236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.725237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.725242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.725252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.725257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.725261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.725264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.725268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.725272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.725275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.725279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.725288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.725297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.725304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.725308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.725312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.725315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.725317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.725320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.725324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.725327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.725336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.725339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.725343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.725346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.725348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.725351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.725354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.725357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.725374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:49.732274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:49.733354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:49.733374Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:49.733377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:49.733381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:49.733387Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1329 [2:7559342282381166192:9799]
2025-10-09T21:48:49.733399Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:49.733416Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:49.733418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:49.733421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:49.733424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:49.733446Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:49.733453Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:49.733462Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:49.733527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.733537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.733539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.733541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:49.733602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:49.733609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.733622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.733640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.733659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:49.733694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342282381166187:9799]
2025-10-09T21:48:49.733702Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342282381166187:9799]
2025-10-09T21:48:49.733704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342282381166187:9799]
2025-10-09T21:48:49.733718Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:49.733729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:49.733732Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:49.733734Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:49.733738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342282742989979:2432], now have 1 active actors on pipe
2025-10-09T21:48:49.733741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342282742990132:4033], now have 1 active actors on pipe
2025-10-09T21:48:49.733748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:49.733757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:49.733760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:49.733793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342282381166187:9799]
2025-10-09T21:48:49.733803Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:49.733796Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1329
2025-10-09T21:48:49.733805Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1329
2025-10-09T21:48:49.733820Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:49.733836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.733839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.733865Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|29fb4bf7-f3c11639-e31cda36-ebf6b16f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:49.733875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:49.733879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:49.733881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:49.733889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:49.733898Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:49.733911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:49.734055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.734063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.734076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:49.734169Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1330 partition: 0 MaxSeqNo: 25 sessionId: src_id|29fb4bf7-f3c11639-e31cda36-ebf6b16f_0
2025-10-09T21:48:49.734583Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c6ad281-5fd5a31e-d8f1dd1b-722eb01f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|29fb4bf7-f3c11639-e31cda36-ebf6b16f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:49.737385Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29fb4bf7-f3c11639-e31cda36-ebf6b16f_0] Write session: try to update token
2025-10-09T21:48:49.737414Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29fb4bf7-f3c11639-e31cda36-ebf6b16f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:49.745824Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:49.745844Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:49.745855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342282381166187:9799]
2025-10-09T21:48:49.745908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:49.745907Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:49.746113Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:49.746271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:49.746423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:49.746573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:49.746779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:49.746822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:49.751394Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1330 sessionId: src_id|29fb4bf7-f3c11639-e31cda36-ebf6b16f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:49.753958Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:49.754190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.754210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.754254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:49.754351Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:49.759051Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:49.759069Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:49.759071Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:49.759074Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:49.759086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342282381166187:9799]
2025-10-09T21:48:49.759089Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342282381166187:9799]
2025-10-09T21:48:49.759092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342282381166187:9799]
2025-10-09T21:48:49.759131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:49.759141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.759147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:49.759153Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:49.759159Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:49.759164Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:49.759282Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:49.807893Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342282742990143:4303]
2025-10-09T21:48:49.808157Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 1329, pipe: [1:7559342282742990143:4303]
2025-10-09T21:48:49.917468Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:49.925280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.925310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.925485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.925553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.925616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.925693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.925801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.925811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.925878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.925942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.926013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.926096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.926139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.926147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.926210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.926268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.926331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.926392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.926434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.926441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.926509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.926578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.926641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.926705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.926748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.926751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.926819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.926895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.926955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.927085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.927128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.927136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.927260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.927322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.927388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.927449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.927491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.927499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.927561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.927624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.927686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.927751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.927800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.927808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.927868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.927926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.927985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.928048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.928091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.928098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.928157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.928293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.928414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.928475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.928520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.928527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.928586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:49.928645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:49.928709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:49.928775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:49.928813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:49.928819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:49.928877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342282742990143:4303], now have 1 active actors on pipe
2025-10-09T21:48:49.929092Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:49.929100Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:49.929104Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:49.929133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1330 reason: error at write: Disconnected sessionId: src_id|29fb4bf7-f3c11639-e31cda36-ebf6b16f_0
2025-10-09T21:48:49.929152Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342282742990143:4303]
2025-10-09T21:48:49.929310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1330 sessionId: src_id|29fb4bf7-f3c11639-e31cda36-ebf6b16f_0 is DEAD
2025-10-09T21:48:49.929497Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29fb4bf7-f3c11639-e31cda36-ebf6b16f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:49.929508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29fb4bf7-f3c11639-e31cda36-ebf6b16f_0] Write session will restart in 0.006039s
2025-10-09T21:48:49.929532Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29fb4bf7-f3c11639-e31cda36-ebf6b16f_0] Write session: Do CDS request
2025-10-09T21:48:49.929539Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29fb4bf7-f3c11639-e31cda36-ebf6b16f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:49.929797Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:49.936480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29fb4bf7-f3c11639-e31cda36-ebf6b16f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:49.936549Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:49.936563Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1331
2025-10-09T21:48:49.936702Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1331 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:49.936734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1331 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:49.936744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1331 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:49.936748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:49.937236Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1331 sessionId:  describe result for acl check
2025-10-09T21:48:49.937273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:49.937279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:49.937281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:49.937287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342282742990189:4021] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:49.937843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342282742990189:4021] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:49.940442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342282742990189:4021] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:49.940639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:49.942695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:49.942858Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:49.942866Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:49.942875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:49.942910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:49.942939Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342282381166349:9801]
2025-10-09T21:48:49.943062Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342282742990189:4021] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:49.943075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342282742990189:4021] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:49.943078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342282742990189:4021] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:49.943086Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1331 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:49.943335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:49.943600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:49.943639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:49.943885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:49.943925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.943930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.943931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.943933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.943934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.943935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.943936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.943937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.943938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.943943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.943953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.943957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.943961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.943965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.943969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.943973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.943977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.943980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.943986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.943994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:49.944004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.944010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:49.944013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.944024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:49.944028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.944033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:49.944038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.944043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:49.944051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.944057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:49.944061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.944070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.944075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.944080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.944084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.944089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.944117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:49.950378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:49.951486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:49.951511Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:49.951514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:49.951519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:49.951525Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1330 [2:7559342282381166349:9801]
2025-10-09T21:48:49.951540Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:49.951560Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:49.951563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:49.951566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:49.951569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:49.951586Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:49.951595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:49.951602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:49.951664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.951671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.951675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.951678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:49.951753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:49.951756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.951768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:49.951779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:49.951794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:49.951818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342282742990302:4021], now have 1 active actors on pipe
2025-10-09T21:48:49.951830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342282742990148:2432], now have 1 active actors on pipe
2025-10-09T21:48:49.951837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342282381166344:9801]
2025-10-09T21:48:49.951839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342282381166344:9801]
2025-10-09T21:48:49.951841Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342282381166344:9801]
2025-10-09T21:48:49.951857Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:49.951909Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1330
2025-10-09T21:48:49.951952Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1330
2025-10-09T21:48:49.951876Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:49.951879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:49.951888Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:49.951891Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:49.951893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:49.951896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:49.951932Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342282381166344:9801]
2025-10-09T21:48:49.951935Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:49.951945Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:49.951966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.951971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.952005Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6176cb1e-fc944cbd-b26d3328-3e9bfbdc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:49.952012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:49.952017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:49.952020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:49.952030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:49.952041Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:49.952069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:49.952246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:49.952256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:49.952270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:49.952342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1331 partition: 0 MaxSeqNo: 25 sessionId: src_id|6176cb1e-fc944cbd-b26d3328-3e9bfbdc_0
2025-10-09T21:48:49.952724Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29fb4bf7-f3c11639-e31cda36-ebf6b16f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6176cb1e-fc944cbd-b26d3328-3e9bfbdc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:49.955212Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6176cb1e-fc944cbd-b26d3328-3e9bfbdc_0] Write session: try to update token
2025-10-09T21:48:49.955230Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6176cb1e-fc944cbd-b26d3328-3e9bfbdc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:49.967303Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:49.967329Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:49.967332Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:49.967335Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:49.967347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342282381166344:9801]
2025-10-09T21:48:49.967354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342282381166344:9801]
2025-10-09T21:48:49.967361Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342282381166344:9801]
2025-10-09T21:48:49.967415Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:49.967419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:49.967428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:49.967429Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:49.967432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:49.967433Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:49.967679Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:49.968753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1331 sessionId: src_id|6176cb1e-fc944cbd-b26d3328-3e9bfbdc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:49.971151Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:50.029567Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342287037957610:4303]
2025-10-09T21:48:50.029906Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 1330, pipe: [1:7559342287037957610:4303]
2025-10-09T21:48:50.118516Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:50.118914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:50.118932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:50.118990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342287037957610:4303], now have 1 active actors on pipe
2025-10-09T21:48:50.119227Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:50.119242Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:50.119251Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342287037957610:4303]
2025-10-09T21:48:50.119278Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:50.119313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1331 reason: error at write: Disconnected sessionId: src_id|6176cb1e-fc944cbd-b26d3328-3e9bfbdc_0
2025-10-09T21:48:50.119379Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:50.119522Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1331 sessionId: src_id|6176cb1e-fc944cbd-b26d3328-3e9bfbdc_0 is DEAD
2025-10-09T21:48:50.119665Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6176cb1e-fc944cbd-b26d3328-3e9bfbdc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:50.119676Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6176cb1e-fc944cbd-b26d3328-3e9bfbdc_0] Write session will restart in 0.005780s
2025-10-09T21:48:50.119706Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6176cb1e-fc944cbd-b26d3328-3e9bfbdc_0] Write session: Do CDS request
2025-10-09T21:48:50.119711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6176cb1e-fc944cbd-b26d3328-3e9bfbdc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:50.126503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6176cb1e-fc944cbd-b26d3328-3e9bfbdc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:50.126629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:50.126647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1332
2025-10-09T21:48:50.126787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1332 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:50.126832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1332 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:50.126843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1332 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:50.126847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:50.127340Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1332 sessionId:  describe result for acl check
2025-10-09T21:48:50.127378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:50.127384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:50.127386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:50.127392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342287037957630:4025] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:50.127950Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342287037957630:4025] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:50.130976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342287037957630:4025] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:50.132911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:50.134803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:50.134952Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:50.134960Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:50.134969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:50.135010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:50.135040Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342286676133802:9803]
2025-10-09T21:48:50.135037Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342287037957630:4025] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:50.135066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342287037957630:4025] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:50.135070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342287037957630:4025] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:50.135080Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1332 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:50.135462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:50.135732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:50.135762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:50.135976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:50.136008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.136009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.136010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.136010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.136011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.136012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.136012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.136013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.136013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.136017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.136024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.136026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.136028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.136030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.136032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.136034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.136036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.136038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.136043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.136048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.136052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.136055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.136058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.136061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.136063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.136066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.136069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.136072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.136075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.136078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.136081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.136084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.136086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.136089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.136092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.136095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.136112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:50.142533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:50.143584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:50.143599Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:50.143601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:50.143604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:50.143610Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1331 [2:7559342286676133802:9803]
2025-10-09T21:48:50.143620Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:50.143636Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:50.143638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:50.143640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:50.143641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:50.143656Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:50.143660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:50.143666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:50.143732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.143742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.143744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.143747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:50.143820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:50.143827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.143835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.143845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.143859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:50.143883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342287037957769:4025], now have 1 active actors on pipe
2025-10-09T21:48:50.143890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342287037957614:2432], now have 1 active actors on pipe
2025-10-09T21:48:50.143897Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342286676133797:9803]
2025-10-09T21:48:50.143902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342286676133797:9803]
2025-10-09T21:48:50.143903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342286676133797:9803]
2025-10-09T21:48:50.143913Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:50.143929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:50.143938Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:50.143941Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:50.143945Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:50.143949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:50.143953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:50.143981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342286676133797:9803]
2025-10-09T21:48:50.143993Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:50.143992Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1331
2025-10-09T21:48:50.144023Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:50.143995Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1331
2025-10-09T21:48:50.144108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:50.144118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:50.144153Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|772c5df9-9e7f1373-a6e693bc-1cefdf5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:50.144164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:50.144168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:50.144171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:50.144182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:50.144207Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:50.144227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:50.144375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:50.144385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:50.144405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:50.144544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1332 partition: 0 MaxSeqNo: 25 sessionId: src_id|772c5df9-9e7f1373-a6e693bc-1cefdf5_0
2025-10-09T21:48:50.144877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6176cb1e-fc944cbd-b26d3328-3e9bfbdc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|772c5df9-9e7f1373-a6e693bc-1cefdf5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:50.147444Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|772c5df9-9e7f1373-a6e693bc-1cefdf5_0] Write session: try to update token
2025-10-09T21:48:50.147464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|772c5df9-9e7f1373-a6e693bc-1cefdf5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:50.158490Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:50.158508Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:50.158511Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:50.158513Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:50.158523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342286676133797:9803]
2025-10-09T21:48:50.158527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342286676133797:9803]
2025-10-09T21:48:50.158531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342286676133797:9803]
2025-10-09T21:48:50.158578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:50.158582Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:50.158587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.158588Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:50.158590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:50.158592Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:50.158723Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:50.161564Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1332 sessionId: src_id|772c5df9-9e7f1373-a6e693bc-1cefdf5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:50.163983Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:50.219521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342287037957784:4303]
2025-10-09T21:48:50.219763Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 1331, pipe: [1:7559342287037957784:4303]
2025-10-09T21:48:50.311680Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:50.311910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:50.311923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:50.311960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342287037957784:4303], now have 1 active actors on pipe
2025-10-09T21:48:50.312172Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:50.312225Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:50.312236Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342287037957784:4303]
2025-10-09T21:48:50.312240Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:50.312259Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:50.312390Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1332 reason: error at write: Disconnected sessionId: src_id|772c5df9-9e7f1373-a6e693bc-1cefdf5_0
2025-10-09T21:48:50.312579Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1332 sessionId: src_id|772c5df9-9e7f1373-a6e693bc-1cefdf5_0 is DEAD
2025-10-09T21:48:50.312743Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|772c5df9-9e7f1373-a6e693bc-1cefdf5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:50.312757Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|772c5df9-9e7f1373-a6e693bc-1cefdf5_0] Write session will restart in 0.009945s
2025-10-09T21:48:50.312785Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|772c5df9-9e7f1373-a6e693bc-1cefdf5_0] Write session: Do CDS request
2025-10-09T21:48:50.312791Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|772c5df9-9e7f1373-a6e693bc-1cefdf5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:50.323547Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|772c5df9-9e7f1373-a6e693bc-1cefdf5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:50.323619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:50.323637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1333
2025-10-09T21:48:50.323781Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1333 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:50.323815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1333 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:50.323825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1333 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:50.323829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:50.324225Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1333 sessionId:  describe result for acl check
2025-10-09T21:48:50.324261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:50.324270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:50.324272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:50.324277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342287037957912:4012] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:50.324411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:50.324827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342287037957912:4012] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:50.326786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:50.326931Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342287037957912:4012] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:50.327001Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:50.327009Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:50.327022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:50.327086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:50.327127Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342286676133960:9805]
2025-10-09T21:48:50.327746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:50.327999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:50.328043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:50.328364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:50.328414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.328421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.328422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.328423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.328425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.328426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.328427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.328428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.328429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.328433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.328444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.328449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.328454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.328458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.328462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.328466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.328470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.328474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.328481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.328490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.328497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.328503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.328508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.328515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.328524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.328530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.328539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.328550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.328554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.328559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.328565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.328570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.328575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.328581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.328586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.328596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.328626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:50.333894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342287037957912:4012] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:50.333912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342287037957912:4012] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:50.333915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342287037957912:4012] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:50.333923Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1333 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:50.335194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:50.336227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:50.336241Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:50.336244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:50.336248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:50.336253Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1332 [2:7559342286676133960:9805]
2025-10-09T21:48:50.336264Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:50.336280Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:50.336282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:50.336284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:50.336285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:50.336299Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:50.336303Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:50.336310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:50.336368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.336376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.336378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.336381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:50.336442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:50.336449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.336457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.336464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.336474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:50.336489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342286676133955:9805]
2025-10-09T21:48:50.336492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342286676133955:9805]
2025-10-09T21:48:50.336493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342286676133955:9805]
2025-10-09T21:48:50.336500Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:50.336515Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:50.336521Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:50.336524Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:50.336527Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:50.336528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:50.336530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:50.336568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342287037957791:2432], now have 1 active actors on pipe
2025-10-09T21:48:50.336574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342287037957950:4012], now have 1 active actors on pipe
2025-10-09T21:48:50.336576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342286676133955:9805]
2025-10-09T21:48:50.336581Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:50.336600Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:50.336634Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1332
2025-10-09T21:48:50.336636Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1332
2025-10-09T21:48:50.336721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:50.336729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:50.336755Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5b22a732-46db5702-76a62d91-7c4dac4a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:50.336765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:50.336768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:50.336770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:50.336779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:50.336788Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:50.336805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:50.336981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:50.336992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:50.337021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:50.337129Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1333 partition: 0 MaxSeqNo: 25 sessionId: src_id|5b22a732-46db5702-76a62d91-7c4dac4a_0
2025-10-09T21:48:50.337484Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|772c5df9-9e7f1373-a6e693bc-1cefdf5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5b22a732-46db5702-76a62d91-7c4dac4a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:50.339845Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b22a732-46db5702-76a62d91-7c4dac4a_0] Write session: try to update token
2025-10-09T21:48:50.339864Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b22a732-46db5702-76a62d91-7c4dac4a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:50.351158Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:50.351177Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:50.351180Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:50.351183Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:50.351192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342286676133955:9805]
2025-10-09T21:48:50.351197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342286676133955:9805]
2025-10-09T21:48:50.351201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342286676133955:9805]
2025-10-09T21:48:50.351248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:50.351264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.351267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:50.351269Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:50.351275Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:50.351279Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:50.351428Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:50.353632Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1333 sessionId: src_id|5b22a732-46db5702-76a62d91-7c4dac4a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:50.356189Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:50.412538Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342287037957959:4303]
2025-10-09T21:48:50.412820Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 1332, pipe: [1:7559342287037957959:4303]
2025-10-09T21:48:50.501937Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:50.502428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:50.502453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:50.502509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342287037957959:4303], now have 1 active actors on pipe
2025-10-09T21:48:50.502765Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:50.502764Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:50.502768Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:50.502772Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:50.502808Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342287037957959:4303]
2025-10-09T21:48:50.502841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1333 reason: error at write: Disconnected sessionId: src_id|5b22a732-46db5702-76a62d91-7c4dac4a_0
2025-10-09T21:48:50.502996Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1333 sessionId: src_id|5b22a732-46db5702-76a62d91-7c4dac4a_0 is DEAD
2025-10-09T21:48:50.503150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b22a732-46db5702-76a62d91-7c4dac4a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:50.503160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b22a732-46db5702-76a62d91-7c4dac4a_0] Write session will restart in 0.009872s
2025-10-09T21:48:50.503187Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b22a732-46db5702-76a62d91-7c4dac4a_0] Write session: Do CDS request
2025-10-09T21:48:50.503193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b22a732-46db5702-76a62d91-7c4dac4a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:50.514572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b22a732-46db5702-76a62d91-7c4dac4a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:50.514637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:50.514653Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1334
2025-10-09T21:48:50.514812Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1334 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:50.514862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1334 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:50.514873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1334 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:50.514877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:50.515210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:50.515289Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1334 sessionId:  describe result for acl check
2025-10-09T21:48:50.515326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:50.515334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:50.515335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:50.515342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342287037958108:3997] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:50.515800Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342287037958108:3997] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:50.517222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:50.517408Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:50.517419Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:50.517429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:50.517479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:50.517519Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342286676134157:9819]
2025-10-09T21:48:50.517694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342287037958108:3997] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:50.518134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:50.518491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:50.518527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:50.518786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:50.518835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.518841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.518842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.518843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.518843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.518844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.518845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.518845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.518846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.518849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.518856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.518859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.518861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.518863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.518865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.518867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.518869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.518871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.518877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.518882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.518887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.518893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.518897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.518903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.518907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.518912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.518917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.518922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.518926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.518931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.518935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.518941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.518946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.518951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.518959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.518962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.518979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:50.523190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342287037958108:3997] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:50.523203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342287037958108:3997] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:50.523206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342287037958108:3997] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:50.523214Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1334 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:50.525093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:50.526382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:50.526402Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:50.526404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:50.526409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:50.526415Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1333 [2:7559342286676134157:9819]
2025-10-09T21:48:50.526428Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:50.526444Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:50.526455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:50.526458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:50.526460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:50.526477Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:50.526483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:50.526492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:50.526548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.526554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.526555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.526557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:50.526609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:50.526617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.526628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.526641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.526661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:50.526687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342287037958144:3997], now have 1 active actors on pipe
2025-10-09T21:48:50.526697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342287037957985:2432], now have 1 active actors on pipe
2025-10-09T21:48:50.526704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342286676134152:9819]
2025-10-09T21:48:50.526708Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342286676134152:9819]
2025-10-09T21:48:50.526710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342286676134152:9819]
2025-10-09T21:48:50.526722Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:50.526736Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:50.526741Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:50.526751Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:50.526751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:50.526753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:50.526724Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1333
2025-10-09T21:48:50.526754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:50.526734Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1333
2025-10-09T21:48:50.526800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:50.526809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:50.526822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342286676134152:9819]
2025-10-09T21:48:50.526833Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:50.526852Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:50.526864Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|14020d07-c6c92091-10ad7cc0-dc7ea069_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:50.526871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:50.526881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:50.526883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:50.526893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:50.526907Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:50.526928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:50.527077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:50.527084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:50.527097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:50.527172Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1334 partition: 0 MaxSeqNo: 25 sessionId: src_id|14020d07-c6c92091-10ad7cc0-dc7ea069_0
2025-10-09T21:48:50.527545Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b22a732-46db5702-76a62d91-7c4dac4a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|14020d07-c6c92091-10ad7cc0-dc7ea069_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:50.529936Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14020d07-c6c92091-10ad7cc0-dc7ea069_0] Write session: try to update token
2025-10-09T21:48:50.529953Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14020d07-c6c92091-10ad7cc0-dc7ea069_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:50.541252Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:50.541272Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:50.541275Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:50.541277Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:50.541287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342286676134152:9819]
2025-10-09T21:48:50.541292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342286676134152:9819]
2025-10-09T21:48:50.541296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342286676134152:9819]
2025-10-09T21:48:50.541341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:50.541343Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:50.541348Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:50.541349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.541352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:50.541352Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:50.541487Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:50.542800Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1334 sessionId: src_id|14020d07-c6c92091-10ad7cc0-dc7ea069_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:50.545148Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:50.603015Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342287037958151:4303]
2025-10-09T21:48:50.603281Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 1333, pipe: [1:7559342287037958151:4303]
2025-10-09T21:48:50.698024Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:50.698275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:50.698292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:50.698347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342287037958151:4303], now have 1 active actors on pipe
2025-10-09T21:48:50.698529Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:50.698545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:50.698561Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:50.698571Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:50.698615Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342287037958151:4303]
2025-10-09T21:48:50.698639Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1334 reason: error at write: Disconnected sessionId: src_id|14020d07-c6c92091-10ad7cc0-dc7ea069_0
2025-10-09T21:48:50.698791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1334 sessionId: src_id|14020d07-c6c92091-10ad7cc0-dc7ea069_0 is DEAD
2025-10-09T21:48:50.698966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14020d07-c6c92091-10ad7cc0-dc7ea069_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:50.698977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14020d07-c6c92091-10ad7cc0-dc7ea069_0] Write session will restart in 0.006415s
2025-10-09T21:48:50.699012Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14020d07-c6c92091-10ad7cc0-dc7ea069_0] Write session: Do CDS request
2025-10-09T21:48:50.699035Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14020d07-c6c92091-10ad7cc0-dc7ea069_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:50.706584Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14020d07-c6c92091-10ad7cc0-dc7ea069_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:50.706638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:50.706651Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1335
2025-10-09T21:48:50.706790Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1335 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:50.706825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1335 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:50.706833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1335 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:50.706836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:50.707210Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1335 sessionId:  describe result for acl check
2025-10-09T21:48:50.707238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:50.707239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:50.707241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:50.707244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342287037958178:3983] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:50.707735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342287037958178:3983] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:50.710259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342287037958178:3983] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:50.712023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:50.714047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:50.714221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342287037958178:3983] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:50.714235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342287037958178:3983] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:50.714238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342287037958178:3983] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:50.714244Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1335 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:50.714264Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:50.714275Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:50.714288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:50.714344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:50.714389Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342286676134321:9823]
2025-10-09T21:48:50.714984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:50.715288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:50.715334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:50.715654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:50.715699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.715705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.715706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.715707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.715709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.715710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.715711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.715712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.715713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.715718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.715729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.715734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.715738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.715742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.715746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.715750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.715754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.715758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.715765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.715773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.715779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.715791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.715796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.715801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.715811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.715816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.715826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.715831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.715835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.715840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.715846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.715856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.715861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.715871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.715880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.715885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.715915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:50.722226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:50.723247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:50.723262Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:50.723265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:50.723269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:50.723276Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1334 [2:7559342286676134321:9823]
2025-10-09T21:48:50.723291Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:50.723314Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:50.723321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:50.723325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:50.723327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:50.723344Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:50.723355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:50.723362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:50.723424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.723430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.723433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.723435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:50.723503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:50.723510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.723524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.723538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.723647Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1334
2025-10-09T21:48:50.723660Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1334
2025-10-09T21:48:50.723560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:50.723587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342287037958154:2432], now have 1 active actors on pipe
2025-10-09T21:48:50.723591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342287037958312:3983], now have 1 active actors on pipe
2025-10-09T21:48:50.723598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342286676134316:9823]
2025-10-09T21:48:50.723600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342286676134316:9823]
2025-10-09T21:48:50.723602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342286676134316:9823]
2025-10-09T21:48:50.723614Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:50.723629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:50.723642Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:50.723645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:50.723645Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:50.723647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:50.723647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:50.723679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342286676134316:9823]
2025-10-09T21:48:50.723682Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:50.723684Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:50.723713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:50.723717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:50.723750Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e7128ebd-587bbfc8-760f879b-b19d973_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:50.723757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:50.723761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:50.723764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:50.723774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:50.723784Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:50.723799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:50.723979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:50.723992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:50.724011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:50.724109Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1335 partition: 0 MaxSeqNo: 25 sessionId: src_id|e7128ebd-587bbfc8-760f879b-b19d973_0
2025-10-09T21:48:50.724481Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14020d07-c6c92091-10ad7cc0-dc7ea069_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e7128ebd-587bbfc8-760f879b-b19d973_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:50.726886Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e7128ebd-587bbfc8-760f879b-b19d973_0] Write session: try to update token
2025-10-09T21:48:50.726906Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e7128ebd-587bbfc8-760f879b-b19d973_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:50.737641Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:50.737659Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:50.737680Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:50.737682Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:50.737690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342286676134316:9823]
2025-10-09T21:48:50.737694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342286676134316:9823]
2025-10-09T21:48:50.737696Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342286676134316:9823]
2025-10-09T21:48:50.737735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:50.737747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.737747Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:50.737750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:50.737753Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:50.737760Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:50.737878Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:50.740026Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1335 sessionId: src_id|e7128ebd-587bbfc8-760f879b-b19d973_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:50.742319Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:50.798582Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342287037958323:4303]
2025-10-09T21:48:50.798864Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 1334, pipe: [1:7559342287037958323:4303]
2025-10-09T21:48:50.883066Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:48:50.883095Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 12
2025-10-09T21:48:50.889351Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:50.889790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:50.889802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:50.889842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342287037958323:4303], now have 1 active actors on pipe
2025-10-09T21:48:50.890072Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:50.890113Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342287037958323:4303]
2025-10-09T21:48:50.890114Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:50.890129Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 12
2025-10-09T21:48:50.890161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:50.890238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1335 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e7128ebd-587bbfc8-760f879b-b19d973_0
2025-10-09T21:48:50.890427Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1335 sessionId: src_id|e7128ebd-587bbfc8-760f879b-b19d973_0 is DEAD
2025-10-09T21:48:50.890572Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e7128ebd-587bbfc8-760f879b-b19d973_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:50.890588Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e7128ebd-587bbfc8-760f879b-b19d973_0] Write session will restart in 0.009497s
2025-10-09T21:48:50.890614Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e7128ebd-587bbfc8-760f879b-b19d973_0] Write session: Do CDS request
2025-10-09T21:48:50.890621Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e7128ebd-587bbfc8-760f879b-b19d973_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:50.901559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e7128ebd-587bbfc8-760f879b-b19d973_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:50.901596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:50.901614Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1336
2025-10-09T21:48:50.901784Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1336 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:50.901827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1336 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:50.901839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1336 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:50.901843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:50.902291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1336 sessionId:  describe result for acl check
2025-10-09T21:48:50.902331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:50.902340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:50.902343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:50.902347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342287037958452:3979] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:50.902907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342287037958452:3979] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:50.903303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:50.905030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342287037958452:3979] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:50.905425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:50.905635Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:50.905645Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:50.905656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:50.905721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:50.905753Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342286676134483:9826]
2025-10-09T21:48:50.906312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:50.906616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:50.906676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:50.906979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:50.907026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.907032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.907033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.907034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.907034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.907035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.907036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.907037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.907037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.907041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.907049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.907051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.907053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.907062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.907064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.907067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.907069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.907073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.907079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.907088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:50.907095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.907100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:50.907107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.907112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:50.907116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.907122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:50.907126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.907132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:50.907136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.907142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:50.907147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.907152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.907157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.907162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.907166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.907172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.907202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:50.907239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342287037958452:3979] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:50.907263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342287037958452:3979] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:50.907267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342287037958452:3979] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:50.907276Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1336 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:50.913775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:50.914853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:50.914870Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:50.914872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:50.914876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:50.914882Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1335 [2:7559342286676134483:9826]
2025-10-09T21:48:50.914893Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:50.914908Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:50.914910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:50.914913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:50.914915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:50.914930Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:50.914933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:50.914941Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:50.914991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.914997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.915001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.915003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:50.915070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:50.915076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.915084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:50.915090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:50.915100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:50.915120Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342286676134478:9826]
2025-10-09T21:48:50.915127Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342286676134478:9826]
2025-10-09T21:48:50.915129Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342286676134478:9826]
2025-10-09T21:48:50.915139Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:50.915149Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:50.915153Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:50.915155Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:50.915160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342287037958330:2432], now have 1 active actors on pipe
2025-10-09T21:48:50.915165Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:50.915178Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:50.915184Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:50.915188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342287037958491:3979], now have 1 active actors on pipe
2025-10-09T21:48:50.915221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342286676134478:9826]
2025-10-09T21:48:50.915233Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:50.915255Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:50.915247Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1335
2025-10-09T21:48:50.915267Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1335
2025-10-09T21:48:50.915327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:50.915335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:50.915378Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4e4c98b2-299748bc-1310d960-5f2842b9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:50.915392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:50.915397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:50.915400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:50.915411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:50.915421Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:50.915471Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:48:50.915499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:50.915639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:50.915649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:50.915675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:50.915741Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 16 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:48:50.915774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1336 partition: 0 MaxSeqNo: 25 sessionId: src_id|4e4c98b2-299748bc-1310d960-5f2842b9_0
2025-10-09T21:48:50.915787Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:48:50.916137Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e7128ebd-587bbfc8-760f879b-b19d973_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4e4c98b2-299748bc-1310d960-5f2842b9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:50.918812Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e4c98b2-299748bc-1310d960-5f2842b9_0] Write session: try to update token
2025-10-09T21:48:50.918832Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e4c98b2-299748bc-1310d960-5f2842b9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:50.929506Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:50.929524Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:50.929527Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:50.929529Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:50.929539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342286676134478:9826]
2025-10-09T21:48:50.929543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342286676134478:9826]
2025-10-09T21:48:50.929548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342286676134478:9826]
2025-10-09T21:48:50.929594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:50.929595Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:50.929601Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:50.929601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:50.929603Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:50.929605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:50.929753Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:50.932745Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1336 sessionId: src_id|4e4c98b2-299748bc-1310d960-5f2842b9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:50.935249Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:50.990059Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342287037958503:4303]
2025-10-09T21:48:50.990318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 1335, pipe: [1:7559342287037958503:4303]
2025-10-09T21:48:51.082698Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:51.083260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:51.083277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:51.083319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342287037958503:4303], now have 1 active actors on pipe
2025-10-09T21:48:51.083483Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:51.083568Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:51.083613Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:51.083659Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1336 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4e4c98b2-299748bc-1310d960-5f2842b9_0
2025-10-09T21:48:51.083676Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342287037958503:4303]
2025-10-09T21:48:51.083867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1336 sessionId: src_id|4e4c98b2-299748bc-1310d960-5f2842b9_0 is DEAD
2025-10-09T21:48:51.084048Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e4c98b2-299748bc-1310d960-5f2842b9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:51.084060Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e4c98b2-299748bc-1310d960-5f2842b9_0] Write session will restart in 0.008420s
2025-10-09T21:48:51.084088Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e4c98b2-299748bc-1310d960-5f2842b9_0] Write session: Do CDS request
2025-10-09T21:48:51.084095Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e4c98b2-299748bc-1310d960-5f2842b9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:51.093507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e4c98b2-299748bc-1310d960-5f2842b9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:51.093607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:51.093623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1337
2025-10-09T21:48:51.093795Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1337 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:51.093828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1337 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:51.093835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1337 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:51.093841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:51.094324Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1337 sessionId:  describe result for acl check
2025-10-09T21:48:51.094373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:51.094380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:51.094381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:51.094388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342291332925927:3976] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:51.094947Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342291332925927:3976] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:51.095629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:51.097102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342291332925927:3976] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:51.097821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:51.098010Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:51.098020Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:51.098047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:51.098106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:51.098147Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342290971101943:9829]
2025-10-09T21:48:51.098759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:51.099064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:51.099105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:51.099421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:51.099475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.099481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.099482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.099483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.099484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.099485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.099486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.099488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.099489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.099493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.099503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.099506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.099509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.099513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.099516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.099521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.099524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.099528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.099534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.099541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.099548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.099552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.099558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.099563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.099567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.099575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.099580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.099584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.099592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.099598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.099602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.099607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.099611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.099618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.099622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.099626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.099658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:51.099815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342291332925927:3976] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:51.099828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342291332925927:3976] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:51.099832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342291332925927:3976] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:51.099842Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1337 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:51.106133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:51.107343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:51.107363Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:51.107366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:51.107370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:51.107377Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1336 [2:7559342290971101943:9829]
2025-10-09T21:48:51.107393Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:51.107413Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:51.107416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:51.107419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:51.107421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:51.107439Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:51.107449Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:51.107457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:51.107515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.107523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.107526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.107528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:51.107607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:51.107615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.107626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.107637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.107652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:51.107678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342291332925805:2432], now have 1 active actors on pipe
2025-10-09T21:48:51.107687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342291332925969:3976], now have 1 active actors on pipe
2025-10-09T21:48:51.107695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342290971101938:9829]
2025-10-09T21:48:51.107702Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342290971101938:9829]
2025-10-09T21:48:51.107704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342290971101938:9829]
2025-10-09T21:48:51.107717Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:51.107729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:51.107732Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:51.107734Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:51.107738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:51.107751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:51.107759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:51.107745Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1336
2025-10-09T21:48:51.107748Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1336
2025-10-09T21:48:51.107809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:51.107819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:51.107832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342290971101938:9829]
2025-10-09T21:48:51.107847Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:51.107859Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:51.107884Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8c631715-4363a92a-e3e7e455-8f7e282e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:51.107896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:51.107900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:51.107902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:51.107915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:51.107930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:51.107959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:51.108144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:51.108152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:51.108167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:51.108260Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1337 partition: 0 MaxSeqNo: 25 sessionId: src_id|8c631715-4363a92a-e3e7e455-8f7e282e_0
2025-10-09T21:48:51.108623Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e4c98b2-299748bc-1310d960-5f2842b9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8c631715-4363a92a-e3e7e455-8f7e282e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:51.111292Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c631715-4363a92a-e3e7e455-8f7e282e_0] Write session: try to update token
2025-10-09T21:48:51.111312Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c631715-4363a92a-e3e7e455-8f7e282e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:51.121954Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:51.121975Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:51.121977Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:51.121978Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:51.121987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342290971101938:9829]
2025-10-09T21:48:51.121990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342290971101938:9829]
2025-10-09T21:48:51.121993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342290971101938:9829]
2025-10-09T21:48:51.122047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:51.122055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.122058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:51.122058Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:51.122064Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:51.122068Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:51.122192Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:51.125686Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1337 sessionId: src_id|8c631715-4363a92a-e3e7e455-8f7e282e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:51.128009Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:51.183640Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342291332925981:4303]
2025-10-09T21:48:51.183926Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 1336, pipe: [1:7559342291332925981:4303]
2025-10-09T21:48:51.270361Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:51.270606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:51.270620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:51.270659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342291332925981:4303], now have 1 active actors on pipe
2025-10-09T21:48:51.270887Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:51.270904Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:51.270915Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:51.270963Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342291332925981:4303]
2025-10-09T21:48:51.271020Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1337 reason: error at write: Disconnected sessionId: src_id|8c631715-4363a92a-e3e7e455-8f7e282e_0
2025-10-09T21:48:51.271231Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1337 sessionId: src_id|8c631715-4363a92a-e3e7e455-8f7e282e_0 is DEAD
2025-10-09T21:48:51.271387Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c631715-4363a92a-e3e7e455-8f7e282e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:51.271397Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c631715-4363a92a-e3e7e455-8f7e282e_0] Write session will restart in 0.007532s
2025-10-09T21:48:51.271415Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c631715-4363a92a-e3e7e455-8f7e282e_0] Write session: Do CDS request
2025-10-09T21:48:51.271422Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c631715-4363a92a-e3e7e455-8f7e282e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:51.272429Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:51.280529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c631715-4363a92a-e3e7e455-8f7e282e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:51.280609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:51.280626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1338
2025-10-09T21:48:51.280742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1338 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:51.280771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1338 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:51.280779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1338 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:51.280781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:51.281205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1338 sessionId:  describe result for acl check
2025-10-09T21:48:51.281241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:51.281251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:51.281253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:51.281259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342291332926080:3971] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:51.281810Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342291332926080:3971] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:51.283081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:51.283737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342291332926080:3971] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:51.284874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:51.285057Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:51.285067Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:51.285089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:51.285145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:51.285187Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342290971102106:9831]
2025-10-09T21:48:51.285644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:51.285953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:51.285999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:51.286151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342291332926080:3971] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:51.286168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342291332926080:3971] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:51.286174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342291332926080:3971] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:51.286182Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1338 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:51.286261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:51.286301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.286307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.286308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.286309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.286311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.286312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.286313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.286314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.286315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.286320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.286331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.286335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.286339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.286342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.286346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.286349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.286352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.286354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.286360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.286367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.286374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.286380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.286384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.286389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.286393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.286398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.286404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.286414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.286419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.286428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.286433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.286438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.286447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.286451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.286456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.286461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.286488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:51.292665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:51.293909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:51.293924Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:51.293927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:51.293931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:51.293938Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1337 [2:7559342290971102106:9831]
2025-10-09T21:48:51.293951Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:51.293969Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:51.293976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:51.293979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:51.293982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:51.293997Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:51.294003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:51.294009Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:51.294065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.294070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.294072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.294073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:51.294127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:51.294133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.294140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.294147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.294157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:51.294176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342290971102101:9831]
2025-10-09T21:48:51.294179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342290971102101:9831]
2025-10-09T21:48:51.294180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342290971102101:9831]
2025-10-09T21:48:51.294185Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:51.294197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342291332926151:3971], now have 1 active actors on pipe
2025-10-09T21:48:51.294197Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:51.294199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342291332925988:2432], now have 1 active actors on pipe
2025-10-09T21:48:51.294200Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:51.294202Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:51.294206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:51.294214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:51.294216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:51.294239Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342290971102101:9831]
2025-10-09T21:48:51.294245Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:51.294250Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:51.294283Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1337
2025-10-09T21:48:51.294292Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1337
2025-10-09T21:48:51.294322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:51.294331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:51.294356Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2fec95f0-ee21dfd-a4d4a48a-14dc0ed0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:51.294364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:51.294367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:51.294369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:51.294376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:51.294384Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:51.294396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:51.294542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:51.294548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:51.294558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:51.294629Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1338 partition: 0 MaxSeqNo: 25 sessionId: src_id|2fec95f0-ee21dfd-a4d4a48a-14dc0ed0_0
2025-10-09T21:48:51.294971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c631715-4363a92a-e3e7e455-8f7e282e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2fec95f0-ee21dfd-a4d4a48a-14dc0ed0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:51.297364Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2fec95f0-ee21dfd-a4d4a48a-14dc0ed0_0] Write session: try to update token
2025-10-09T21:48:51.297381Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2fec95f0-ee21dfd-a4d4a48a-14dc0ed0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:51.308066Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:51.308084Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:51.308086Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:51.308087Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:51.308097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342290971102101:9831]
2025-10-09T21:48:51.308100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342290971102101:9831]
2025-10-09T21:48:51.308103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342290971102101:9831]
2025-10-09T21:48:51.308147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:51.308151Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:51.308154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.308157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:51.308159Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:51.308163Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:51.308284Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:51.310832Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1338 sessionId: src_id|2fec95f0-ee21dfd-a4d4a48a-14dc0ed0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:51.313191Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:51.371015Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342291332926161:4303]
2025-10-09T21:48:51.371289Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 1337, pipe: [1:7559342291332926161:4303]
2025-10-09T21:48:51.461930Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:51.462205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:51.462221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:51.462261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342291332926161:4303], now have 1 active actors on pipe
2025-10-09T21:48:51.462446Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:51.462555Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342291332926161:4303]
2025-10-09T21:48:51.462564Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:51.462568Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:51.462591Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:51.462626Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1338 reason: error at write: Disconnected sessionId: src_id|2fec95f0-ee21dfd-a4d4a48a-14dc0ed0_0
2025-10-09T21:48:51.462816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1338 sessionId: src_id|2fec95f0-ee21dfd-a4d4a48a-14dc0ed0_0 is DEAD
2025-10-09T21:48:51.462978Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2fec95f0-ee21dfd-a4d4a48a-14dc0ed0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:51.462988Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2fec95f0-ee21dfd-a4d4a48a-14dc0ed0_0] Write session will restart in 0.006065s
2025-10-09T21:48:51.463008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2fec95f0-ee21dfd-a4d4a48a-14dc0ed0_0] Write session: Do CDS request
2025-10-09T21:48:51.463013Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2fec95f0-ee21dfd-a4d4a48a-14dc0ed0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:51.470569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2fec95f0-ee21dfd-a4d4a48a-14dc0ed0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:51.470641Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:51.470658Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1339
2025-10-09T21:48:51.470809Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1339 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:51.470851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1339 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:51.470862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1339 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:51.470866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:51.471358Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1339 sessionId:  describe result for acl check
2025-10-09T21:48:51.471400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:51.471408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:51.471410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:51.471418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342291332926199:3962] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:51.471972Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342291332926199:3962] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:51.474255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342291332926199:3962] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:51.475919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:51.478054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:51.478249Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:51.478263Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:51.478275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:51.478333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:51.478374Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342290971102300:9839]
2025-10-09T21:48:51.478550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342291332926199:3962] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:51.478567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342291332926199:3962] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:51.478571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342291332926199:3962] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:51.478583Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1339 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:51.478935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:51.479224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:51.479269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:51.479588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:51.479650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.479657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.479658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.479659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.479660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.479661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.479662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.479663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.479664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.479669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.479679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.479683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.479691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.479696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.479700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.479709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.479712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.479716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.479722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.479729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.479734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.479745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.479755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.479764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.479772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.479777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.479784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.479789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.479796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.479801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.479810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.479820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.479829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.479835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.479840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.479848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.479884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:51.486329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:51.487396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:51.487412Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:51.487415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:51.487421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:51.487427Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1338 [2:7559342290971102300:9839]
2025-10-09T21:48:51.487441Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:51.487457Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:51.487460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:51.487463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:51.487465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:51.487482Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:51.487493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:51.487502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:51.487556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.487563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.487565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.487568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:51.487638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:51.487645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.487657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.487673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.487695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:51.487721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342291332926163:2432], now have 1 active actors on pipe
2025-10-09T21:48:51.487728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342291332926326:3962], now have 1 active actors on pipe
2025-10-09T21:48:51.487736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342290971102295:9839]
2025-10-09T21:48:51.487739Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342290971102295:9839]
2025-10-09T21:48:51.487741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342290971102295:9839]
2025-10-09T21:48:51.487753Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:51.487770Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:51.487771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:51.487783Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:51.487786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:51.487786Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:51.487789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:51.487780Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1338
2025-10-09T21:48:51.487823Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342290971102295:9839]
2025-10-09T21:48:51.487835Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:51.487839Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1338
2025-10-09T21:48:51.487841Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:51.487937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:51.487947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:51.487975Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|19498a9b-dab2963c-2e217268-8e3d7044_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:51.487988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:51.488012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:51.488015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:51.488026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:51.488039Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:51.488061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:51.488255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:51.488266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:51.488283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:51.488363Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1339 partition: 0 MaxSeqNo: 25 sessionId: src_id|19498a9b-dab2963c-2e217268-8e3d7044_0
2025-10-09T21:48:51.488721Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2fec95f0-ee21dfd-a4d4a48a-14dc0ed0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|19498a9b-dab2963c-2e217268-8e3d7044_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:51.491204Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19498a9b-dab2963c-2e217268-8e3d7044_0] Write session: try to update token
2025-10-09T21:48:51.491220Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19498a9b-dab2963c-2e217268-8e3d7044_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:51.502390Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:51.502409Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:51.502411Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:51.502413Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:51.502423Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342290971102295:9839]
2025-10-09T21:48:51.502427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342290971102295:9839]
2025-10-09T21:48:51.502431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342290971102295:9839]
2025-10-09T21:48:51.502475Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:51.502479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:51.502485Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:51.502486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.502488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:51.502489Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:51.502635Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:51.505045Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1339 sessionId: src_id|19498a9b-dab2963c-2e217268-8e3d7044_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:51.507677Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:51.562537Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342291332926359:4303]
2025-10-09T21:48:51.562870Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 1338, pipe: [1:7559342291332926359:4303]
2025-10-09T21:48:51.654647Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:51.654917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:51.654931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:51.654972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342291332926359:4303], now have 1 active actors on pipe
2025-10-09T21:48:51.655157Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:51.655189Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:51.655205Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:51.655209Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:51.655246Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342291332926359:4303]
2025-10-09T21:48:51.655266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1339 reason: error at write: Disconnected sessionId: src_id|19498a9b-dab2963c-2e217268-8e3d7044_0
2025-10-09T21:48:51.655463Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1339 sessionId: src_id|19498a9b-dab2963c-2e217268-8e3d7044_0 is DEAD
2025-10-09T21:48:51.655575Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19498a9b-dab2963c-2e217268-8e3d7044_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:51.655587Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19498a9b-dab2963c-2e217268-8e3d7044_0] Write session will restart in 0.008517s
2025-10-09T21:48:51.655614Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19498a9b-dab2963c-2e217268-8e3d7044_0] Write session: Do CDS request
2025-10-09T21:48:51.655621Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19498a9b-dab2963c-2e217268-8e3d7044_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:51.665468Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19498a9b-dab2963c-2e217268-8e3d7044_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:51.665496Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:51.665515Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1340
2025-10-09T21:48:51.665951Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1340 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:51.665988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1340 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:51.665999Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1340 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:51.666003Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:51.666490Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1340 sessionId:  describe result for acl check
2025-10-09T21:48:51.666530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:51.666537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:51.666539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:51.666545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342291332926470:3948] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:51.667097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342291332926470:3948] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:51.668720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:51.669559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342291332926470:3948] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:51.671129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:51.671340Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:51.671348Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:51.671358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:51.671402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:51.671439Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342290971102477:9848]
2025-10-09T21:48:51.671909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:51.672159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:51.672218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:51.672228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342291332926470:3948] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:51.672246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342291332926470:3948] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:51.672250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342291332926470:3948] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:51.672261Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1340 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:51.672479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:51.672535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.672541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.672542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.672544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.672545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.672546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.672548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.672549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.672550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.672556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.672567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.672572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.672576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.672580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.672584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.672588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.672590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.672592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.672597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.672603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.672607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.672611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.672614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.672617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.672620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.672623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.672626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.672629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.672632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.672635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.672638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.672641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.672644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.672647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.672650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.672653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.672671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:51.679018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:51.680376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:51.680387Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:51.680390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:51.680395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:51.680402Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1339 [2:7559342290971102477:9848]
2025-10-09T21:48:51.680418Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:51.680439Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:51.680445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:51.680447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:51.680449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:51.680464Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:51.680472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:51.680478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:51.680529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.680538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.680539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.680541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:51.680600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:51.680605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.680613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.680624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.680634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:51.680651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342290971102470:9848]
2025-10-09T21:48:51.680658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342290971102470:9848]
2025-10-09T21:48:51.680661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342290971102470:9848]
2025-10-09T21:48:51.680670Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:51.680681Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:51.680689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:51.680691Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:51.680693Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:51.680703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:51.680706Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:51.680719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342291332926533:3948], now have 1 active actors on pipe
2025-10-09T21:48:51.680727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342291332926361:2432], now have 1 active actors on pipe
2025-10-09T21:48:51.680763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342290971102470:9848]
2025-10-09T21:48:51.680785Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:51.680789Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:51.680786Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1339
2025-10-09T21:48:51.680792Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1339
2025-10-09T21:48:51.680845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:51.680855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:51.680886Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6d896981-93fc541a-fc8b28ce-fa4caeb1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:51.680897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:51.680902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:51.680904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:51.680916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:51.680933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:51.680951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:51.681104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:51.681116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:51.681129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:51.681218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1340 partition: 0 MaxSeqNo: 25 sessionId: src_id|6d896981-93fc541a-fc8b28ce-fa4caeb1_0
2025-10-09T21:48:51.681566Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19498a9b-dab2963c-2e217268-8e3d7044_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6d896981-93fc541a-fc8b28ce-fa4caeb1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:51.684211Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6d896981-93fc541a-fc8b28ce-fa4caeb1_0] Write session: try to update token
2025-10-09T21:48:51.684229Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6d896981-93fc541a-fc8b28ce-fa4caeb1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:51.692335Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:51.692354Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:51.692365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342290971102470:9848]
2025-10-09T21:48:51.692408Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:51.692411Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:51.692564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:51.692709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:51.692849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:51.692980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:51.693170Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:51.693211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:51.698114Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1340 sessionId: src_id|6d896981-93fc541a-fc8b28ce-fa4caeb1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:51.700702Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:51.700972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:51.700994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:51.701043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:51.701162Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:51.706581Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:51.706605Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:51.706608Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:51.706611Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:51.706624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342290971102470:9848]
2025-10-09T21:48:51.706630Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342290971102470:9848]
2025-10-09T21:48:51.706636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342290971102470:9848]
2025-10-09T21:48:51.706692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:51.706697Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:51.706705Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:51.706706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.706709Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:51.706710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:51.706953Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:51.755496Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342291332926545:4303]
2025-10-09T21:48:51.755808Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 1339, pipe: [1:7559342291332926545:4303]
2025-10-09T21:48:51.854482Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:51.861723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:51.861749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:51.861904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:51.861960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:51.862019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:51.862105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:51.862155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:51.862159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:51.862224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:51.862281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:51.862338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:51.862396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:51.862444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:51.862452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:51.862530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:51.862583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:51.862649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:51.862722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:51.862767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:51.862775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:51.862847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:51.862913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:51.862982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:51.863047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:51.863094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:51.863101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:51.863177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:51.863242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:51.863312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:51.863377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:51.863422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:51.863432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:51.863501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:51.863569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:51.863628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:51.863684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:51.863728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:51.863735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:51.863807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:51.863867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:51.863930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:51.863986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:51.864053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:51.864059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:51.864115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:51.864167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:51.864229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:51.864300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:51.864384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:51.864392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:51.864453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:51.864515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:51.864580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:51.864644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:51.864707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:51.864715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:51.864779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:51.864840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:51.864905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:51.864967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:51.865013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:51.865021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:51.865087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342291332926545:4303], now have 1 active actors on pipe
2025-10-09T21:48:51.865362Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:51.865364Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:51.865368Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:51.865404Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1340 reason: error at write: Disconnected sessionId: src_id|6d896981-93fc541a-fc8b28ce-fa4caeb1_0
2025-10-09T21:48:51.865421Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342291332926545:4303]
2025-10-09T21:48:51.865618Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1340 sessionId: src_id|6d896981-93fc541a-fc8b28ce-fa4caeb1_0 is DEAD
2025-10-09T21:48:51.865749Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d896981-93fc541a-fc8b28ce-fa4caeb1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:51.865760Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d896981-93fc541a-fc8b28ce-fa4caeb1_0] Write session will restart in 0.006859s
2025-10-09T21:48:51.865786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d896981-93fc541a-fc8b28ce-fa4caeb1_0] Write session: Do CDS request
2025-10-09T21:48:51.865792Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d896981-93fc541a-fc8b28ce-fa4caeb1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:51.866004Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:51.873584Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6d896981-93fc541a-fc8b28ce-fa4caeb1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:51.873741Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:51.873759Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1341
2025-10-09T21:48:51.873892Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1341 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:51.873947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1341 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:51.873957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1341 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:51.873960Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:51.874411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1341 sessionId:  describe result for acl check
2025-10-09T21:48:51.874442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:51.874448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:51.874451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:51.874456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342291332926601:3930] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:51.874917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342291332926601:3930] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:51.877761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:51.877983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342291332926601:3930] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:51.879957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:51.880179Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:51.880189Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:51.880201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:51.880260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:51.880299Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342290971102643:9851]
2025-10-09T21:48:51.880414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342291332926601:3930] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:51.880427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342291332926601:3930] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:51.880431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342291332926601:3930] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:51.880442Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1341 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:51.880786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:51.881064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:51.881095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:51.881335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:51.881370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.881375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.881376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.881377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.881377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.881378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.881378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.881379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.881380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.881383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.881390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.881392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.881394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.881396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.881398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.881400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.881403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.881404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.881409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.881414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:51.881418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.881421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:51.881424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.881427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:51.881429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.881432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:51.881435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.881438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:51.881440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.881443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:51.881446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.881449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.881452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.881456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.881461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.881466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.881491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:51.887825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:51.888902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:51.888921Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:51.888923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:51.888927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:51.888934Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1340 [2:7559342290971102643:9851]
2025-10-09T21:48:51.888947Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:51.888963Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:51.888971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:51.888974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:51.888977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:51.888993Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:51.889005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:51.889012Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:51.889064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.889070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.889072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.889075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:51.889142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:51.889150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.889160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:51.889175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:51.889197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:51.889223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342291332926717:3930], now have 1 active actors on pipe
2025-10-09T21:48:51.889231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342291332926553:2432], now have 1 active actors on pipe
2025-10-09T21:48:51.889238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342290971102638:9851]
2025-10-09T21:48:51.889240Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342290971102638:9851]
2025-10-09T21:48:51.889242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342290971102638:9851]
2025-10-09T21:48:51.889256Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:51.889275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:51.889283Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:51.889287Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:51.889289Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:51.889295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:51.889298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:51.889319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342290971102638:9851]
2025-10-09T21:48:51.889327Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:51.889338Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:51.889314Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1340
2025-10-09T21:48:51.889321Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1340
2025-10-09T21:48:51.889361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:51.889371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:51.889402Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|87df3a82-f44edd0d-cb65c1b9-da2ca480_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:51.889416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:51.889420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:51.889423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:51.889434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:51.889449Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:51.889469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:51.889638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:51.889649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:51.889676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:51.889771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1341 partition: 0 MaxSeqNo: 25 sessionId: src_id|87df3a82-f44edd0d-cb65c1b9-da2ca480_0
2025-10-09T21:48:51.890107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d896981-93fc541a-fc8b28ce-fa4caeb1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|87df3a82-f44edd0d-cb65c1b9-da2ca480_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:51.892652Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87df3a82-f44edd0d-cb65c1b9-da2ca480_0] Write session: try to update token
2025-10-09T21:48:51.892670Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87df3a82-f44edd0d-cb65c1b9-da2ca480_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:51.903317Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:51.903339Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:51.903342Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:51.903344Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:51.903354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342290971102638:9851]
2025-10-09T21:48:51.903359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342290971102638:9851]
2025-10-09T21:48:51.903364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342290971102638:9851]
2025-10-09T21:48:51.903410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:51.903419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:51.903418Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:51.903422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:51.903424Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:51.903428Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:51.903557Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:51.906285Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1341 sessionId: src_id|87df3a82-f44edd0d-cb65c1b9-da2ca480_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:51.908544Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:51.965450Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342291332926728:4303]
2025-10-09T21:48:51.965827Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 1340, pipe: [1:7559342291332926728:4303]
2025-10-09T21:48:52.057243Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:52.057624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.057640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.057696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342291332926728:4303], now have 1 active actors on pipe
2025-10-09T21:48:52.057940Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:52.057945Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:52.057953Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:52.057991Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342291332926728:4303]
2025-10-09T21:48:52.058006Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1341 reason: error at write: Disconnected sessionId: src_id|87df3a82-f44edd0d-cb65c1b9-da2ca480_0
2025-10-09T21:48:52.058021Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:52.058189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1341 sessionId: src_id|87df3a82-f44edd0d-cb65c1b9-da2ca480_0 is DEAD
2025-10-09T21:48:52.058327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87df3a82-f44edd0d-cb65c1b9-da2ca480_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:52.058337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87df3a82-f44edd0d-cb65c1b9-da2ca480_0] Write session will restart in 0.005487s
2025-10-09T21:48:52.058365Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87df3a82-f44edd0d-cb65c1b9-da2ca480_0] Write session: Do CDS request
2025-10-09T21:48:52.058371Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87df3a82-f44edd0d-cb65c1b9-da2ca480_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:52.064580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87df3a82-f44edd0d-cb65c1b9-da2ca480_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:52.064606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:52.064632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1342
2025-10-09T21:48:52.064829Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1342 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:52.064887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1342 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:52.064899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1342 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:52.064902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:52.065397Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1342 sessionId:  describe result for acl check
2025-10-09T21:48:52.065435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:52.065441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:52.065443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:52.065449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342295627894043:3917] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:52.066061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342295627894043:3917] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:52.068726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342295627894043:3917] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:52.070961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342295627894043:3917] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:52.070976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342295627894043:3917] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:52.070981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342295627894043:3917] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:52.070989Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1342 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:52.071788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:52.073974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:52.074183Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:52.074192Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:52.074202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:52.074251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:52.074284Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342295266070108:9853]
2025-10-09T21:48:52.074765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:52.075089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:52.075142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:52.075467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:52.075524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.075532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.075533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.075535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.075536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.075537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.075538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.075539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.075541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.075546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.075558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.075562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.075566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.075570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.075574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.075578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.075582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.075586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.075596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.075605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.075611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.075617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.075621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.075626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.075629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.075634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.075639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.075644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.075648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.075652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.075657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.075662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.075667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.075671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.075675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.075680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.075707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:52.082313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:52.083439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:52.083458Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:52.083462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:52.083466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:52.083475Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1341 [2:7559342295266070108:9853]
2025-10-09T21:48:52.083489Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:52.083510Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:52.083513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:52.083516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:52.083519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:52.083542Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:52.083554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:52.083566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:52.083626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.083634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.083637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.083639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:52.083720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:52.083732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.083743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.083754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.083772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:52.083802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342295627894027:2432], now have 1 active actors on pipe
2025-10-09T21:48:52.083811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342295627894190:3917], now have 1 active actors on pipe
2025-10-09T21:48:52.083819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342295266070103:9853]
2025-10-09T21:48:52.083827Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342295266070103:9853]
2025-10-09T21:48:52.083830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342295266070103:9853]
2025-10-09T21:48:52.083844Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:52.083858Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:52.083867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:52.083868Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:52.083870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:52.083884Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:52.083892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:52.083863Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1341
2025-10-09T21:48:52.083882Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1341
2025-10-09T21:48:52.083941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342295266070103:9853]
2025-10-09T21:48:52.083955Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:52.083960Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:52.083971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.083977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.084026Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9b6d8a2e-49130598-97d2bf50-88dae639_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:52.084037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:52.084040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:52.084043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:52.084051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:52.084061Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:52.084079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:52.084279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.084290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.084308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:52.084397Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1342 partition: 0 MaxSeqNo: 25 sessionId: src_id|9b6d8a2e-49130598-97d2bf50-88dae639_0
2025-10-09T21:48:52.084734Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87df3a82-f44edd0d-cb65c1b9-da2ca480_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9b6d8a2e-49130598-97d2bf50-88dae639_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:52.087205Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9b6d8a2e-49130598-97d2bf50-88dae639_0] Write session: try to update token
2025-10-09T21:48:52.087222Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9b6d8a2e-49130598-97d2bf50-88dae639_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:52.098336Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:52.098355Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:52.098358Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:52.098360Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:52.098370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342295266070103:9853]
2025-10-09T21:48:52.098375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342295266070103:9853]
2025-10-09T21:48:52.098380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342295266070103:9853]
2025-10-09T21:48:52.098426Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:52.098430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:52.098431Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:52.098438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.098441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:52.098442Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:52.098583Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:52.100281Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1342 sessionId: src_id|9b6d8a2e-49130598-97d2bf50-88dae639_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:52.102892Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:52.157936Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342295627894206:4303]
2025-10-09T21:48:52.158248Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 1341, pipe: [1:7559342295627894206:4303]
2025-10-09T21:48:52.250300Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:52.250537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.250551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.250589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342295627894206:4303], now have 1 active actors on pipe
2025-10-09T21:48:52.250776Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:52.250790Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:52.250810Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342295627894206:4303]
2025-10-09T21:48:52.250835Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:52.250907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1342 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9b6d8a2e-49130598-97d2bf50-88dae639_0
2025-10-09T21:48:52.251064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1342 sessionId: src_id|9b6d8a2e-49130598-97d2bf50-88dae639_0 is DEAD
2025-10-09T21:48:52.251256Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b6d8a2e-49130598-97d2bf50-88dae639_0] Counters: { Errors: 0 CurrentSessionLifetimeMs: 166 BytesWritten: 56320040 MessagesWritten: 25 BytesWrittenCompressed: 0 BytesInflightUncompressed: 22528020 BytesInflightCompressed: 0 BytesInflightTotal: 18022416 MessagesInflight: 10 }
2025-10-09T21:48:52.251287Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b6d8a2e-49130598-97d2bf50-88dae639_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:52.251293Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b6d8a2e-49130598-97d2bf50-88dae639_0] Write session will restart in 0.006322s
2025-10-09T21:48:52.251319Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b6d8a2e-49130598-97d2bf50-88dae639_0] Write session: Do CDS request
2025-10-09T21:48:52.251324Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b6d8a2e-49130598-97d2bf50-88dae639_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:52.258501Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9b6d8a2e-49130598-97d2bf50-88dae639_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:52.258624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:52.258643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1343
2025-10-09T21:48:52.258807Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1343 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:52.258856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1343 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:52.258867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1343 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:52.258871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:52.259286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1343 sessionId:  describe result for acl check
2025-10-09T21:48:52.259319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:52.259326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:52.259328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:52.259334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342295627894257:3925] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:52.259763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342295627894257:3925] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:52.261926Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342295627894257:3925] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:52.263335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:52.265284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:52.265474Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:52.265485Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:52.265498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:52.265568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:52.265670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342295627894257:3925] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:52.265688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342295627894257:3925] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:52.265611Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342295266070276:9855]
2025-10-09T21:48:52.265692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342295627894257:3925] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:52.265702Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1343 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:52.266226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:52.266529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:52.266583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:52.266928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:52.266989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.266996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.266997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.266999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.267000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.267002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.267003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.267004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.267005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.267010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.267021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.267025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.267029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.267033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.267038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.267042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.267046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.267050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.267056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.267064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.267071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.267077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.267081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.267087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.267091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.267096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.267106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.267111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.267116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.267121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.267126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.267131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.267136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.267141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.267151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.267157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.267188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:52.273627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:52.274927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:52.274943Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:52.274946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:52.274950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:52.274957Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1342 [2:7559342295266070276:9855]
2025-10-09T21:48:52.274970Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:52.274989Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:52.274996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:52.274999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:52.275002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:52.275019Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:52.275030Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:52.275041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:52.275100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.275110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.275112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.275115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:52.275186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:52.275192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.275204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.275221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.275246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:52.275273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342295627894214:2432], now have 1 active actors on pipe
2025-10-09T21:48:52.275282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342295627894380:3925], now have 1 active actors on pipe
2025-10-09T21:48:52.275290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342295266070271:9855]
2025-10-09T21:48:52.275295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342295266070271:9855]
2025-10-09T21:48:52.275297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342295266070271:9855]
2025-10-09T21:48:52.275312Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:52.275332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:52.275341Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:52.275345Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:52.275347Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:52.275350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:52.275354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:52.275377Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1342
2025-10-09T21:48:52.275388Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342295266070271:9855]
2025-10-09T21:48:52.275379Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1342
2025-10-09T21:48:52.275400Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:52.275412Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:52.275440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.275447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.275484Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b4218f71-9d7737c7-c272d484-d02950ce_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:52.275497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:52.275501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:52.275504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:52.275514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:52.275524Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:52.275542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:52.275685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.275693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.275706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:52.275789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1343 partition: 0 MaxSeqNo: 25 sessionId: src_id|b4218f71-9d7737c7-c272d484-d02950ce_0
2025-10-09T21:48:52.276132Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b6d8a2e-49130598-97d2bf50-88dae639_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b4218f71-9d7737c7-c272d484-d02950ce_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:52.278763Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4218f71-9d7737c7-c272d484-d02950ce_0] Write session: try to update token
2025-10-09T21:48:52.278780Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4218f71-9d7737c7-c272d484-d02950ce_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:52.289692Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:52.289709Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:52.289711Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:52.289713Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:52.289722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342295266070271:9855]
2025-10-09T21:48:52.289727Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342295266070271:9855]
2025-10-09T21:48:52.289732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342295266070271:9855]
2025-10-09T21:48:52.289774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:52.289775Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:52.289782Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:52.289783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.289786Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:52.289786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:52.289912Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:52.291753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1343 sessionId: src_id|b4218f71-9d7737c7-c272d484-d02950ce_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:52.294087Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:52.350956Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342295627894392:4303]
2025-10-09T21:48:52.351279Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 1342, pipe: [1:7559342295627894392:4303]
2025-10-09T21:48:52.438813Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:52.439126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.439139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.439182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342295627894392:4303], now have 1 active actors on pipe
2025-10-09T21:48:52.439344Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:52.439393Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342295627894392:4303]
2025-10-09T21:48:52.439404Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:52.439432Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:52.439503Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1343 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b4218f71-9d7737c7-c272d484-d02950ce_0
2025-10-09T21:48:52.439684Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1343 sessionId: src_id|b4218f71-9d7737c7-c272d484-d02950ce_0 is DEAD
2025-10-09T21:48:52.439811Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4218f71-9d7737c7-c272d484-d02950ce_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:52.439823Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4218f71-9d7737c7-c272d484-d02950ce_0] Write session will restart in 0.006659s
2025-10-09T21:48:52.439850Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4218f71-9d7737c7-c272d484-d02950ce_0] Write session: Do CDS request
2025-10-09T21:48:52.439857Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4218f71-9d7737c7-c272d484-d02950ce_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:52.447475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4218f71-9d7737c7-c272d484-d02950ce_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:52.447598Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:52.447612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1344
2025-10-09T21:48:52.447731Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1344 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:52.447765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1344 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:52.447771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1344 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:52.447774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:52.448249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1344 sessionId:  describe result for acl check
2025-10-09T21:48:52.448284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:52.448290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:52.448292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:52.448299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342295627894438:3913] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:52.448877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342295627894438:3913] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:52.450918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342295627894438:3913] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:52.452277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:52.454081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:52.454257Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:52.454268Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:52.454279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:52.454326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:52.454364Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342295266070445:9857]
2025-10-09T21:48:52.454398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342295627894438:3913] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:52.454410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342295627894438:3913] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:52.454414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342295627894438:3913] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:52.454427Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1344 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:52.454926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:52.455167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:52.455208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:52.455504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:52.455549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.455555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.455556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.455558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.455559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.455560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.455561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.455562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.455563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.455568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.455579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.455583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.455586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.455590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.455594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.455598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.455602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.455606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.455611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.455619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.455625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.455631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.455637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.455642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.455651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.455658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.455667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.455673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.455681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.455686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.455695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.455704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.455712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.455719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.455727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.455733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.455765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:52.461747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:52.462907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:52.462920Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:52.462921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:52.462925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:52.462930Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1343 [2:7559342295266070445:9857]
2025-10-09T21:48:52.462940Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:52.462953Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:52.462958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:52.462961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:52.462963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:52.462980Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:52.462990Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:52.463002Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:52.463061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.463069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.463072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.463074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:52.463137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:52.463146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.463156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.463162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.463172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:52.463188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342295627894561:3913], now have 1 active actors on pipe
2025-10-09T21:48:52.463191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342295627894394:2432], now have 1 active actors on pipe
2025-10-09T21:48:52.463199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342295266070440:9857]
2025-10-09T21:48:52.463201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342295266070440:9857]
2025-10-09T21:48:52.463203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342295266070440:9857]
2025-10-09T21:48:52.463217Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:52.463231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:52.463249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:52.463249Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:52.463252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:52.463253Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:52.463255Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:52.463277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342295266070440:9857]
2025-10-09T21:48:52.463290Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:52.463283Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1343
2025-10-09T21:48:52.463284Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1343
2025-10-09T21:48:52.463298Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:52.463335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.463339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.463369Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3c15f2ae-d43e8b4c-7df0b81f-9fcba0dd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:52.463382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:52.463386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:52.463391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:52.463401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:52.463410Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:52.463427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:52.463605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.463616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.463633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:52.463755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1344 partition: 0 MaxSeqNo: 25 sessionId: src_id|3c15f2ae-d43e8b4c-7df0b81f-9fcba0dd_0
2025-10-09T21:48:52.464078Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4218f71-9d7737c7-c272d484-d02950ce_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3c15f2ae-d43e8b4c-7df0b81f-9fcba0dd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:52.466460Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c15f2ae-d43e8b4c-7df0b81f-9fcba0dd_0] Write session: try to update token
2025-10-09T21:48:52.466475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c15f2ae-d43e8b4c-7df0b81f-9fcba0dd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:52.474214Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:52.474227Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:52.474234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342295266070440:9857]
2025-10-09T21:48:52.474274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:52.474273Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:52.474418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:52.474552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:52.474666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:52.474781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:52.475042Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:52.475074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:52.478847Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1344 sessionId: src_id|3c15f2ae-d43e8b4c-7df0b81f-9fcba0dd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:52.481246Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:52.481503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.481534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.481567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:52.481647Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:52.487159Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:52.487178Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:52.487181Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:52.487182Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:52.487193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342295266070440:9857]
2025-10-09T21:48:52.487197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342295266070440:9857]
2025-10-09T21:48:52.487202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342295266070440:9857]
2025-10-09T21:48:52.487248Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:52.487252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:52.487258Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:52.487262Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:52.487262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.487265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:52.487393Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:52.539389Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342295627894586:4303]
2025-10-09T21:48:52.539697Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 1343, pipe: [1:7559342295627894586:4303]
2025-10-09T21:48:52.635211Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:52.642157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.642181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.642322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:52.642378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:52.642433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:52.642492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:52.642844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:52.642862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:52.642934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:52.642990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:52.643044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:52.643113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:52.643155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:52.643163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:52.643230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:52.643280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:52.643332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:52.643383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:52.643423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:52.643431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:52.643494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:52.643550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:52.643604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:52.643659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:52.643698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:52.643703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:52.643762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:52.643828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:52.643898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:52.643961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:52.644008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:52.644016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:52.644077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:52.644140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:52.644200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:52.644258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:52.644296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:52.644304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:52.644358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:52.644437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:52.644493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:52.644549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:52.644587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:52.644593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:52.644647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:52.644697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:52.644749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:52.644804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:52.644849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:52.644856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:52.644915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:52.644973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:52.645037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:52.645093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:52.645132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:52.645137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:52.645194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:52.645255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:52.645318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:52.645376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:52.645415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:52.645420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:52.645461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342295627894586:4303], now have 1 active actors on pipe
2025-10-09T21:48:52.645722Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:52.645737Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:52.645769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1344 reason: error at write: Disconnected sessionId: src_id|3c15f2ae-d43e8b4c-7df0b81f-9fcba0dd_0
2025-10-09T21:48:52.645894Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342295627894586:4303]
2025-10-09T21:48:52.645928Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:52.645940Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:52.645962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1344 sessionId: src_id|3c15f2ae-d43e8b4c-7df0b81f-9fcba0dd_0 is DEAD
2025-10-09T21:48:52.646098Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c15f2ae-d43e8b4c-7df0b81f-9fcba0dd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:52.646113Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c15f2ae-d43e8b4c-7df0b81f-9fcba0dd_0] Write session will restart in 0.006799s
2025-10-09T21:48:52.646138Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c15f2ae-d43e8b4c-7df0b81f-9fcba0dd_0] Write session: Do CDS request
2025-10-09T21:48:52.646144Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c15f2ae-d43e8b4c-7df0b81f-9fcba0dd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:52.654600Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c15f2ae-d43e8b4c-7df0b81f-9fcba0dd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:52.654669Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:52.654683Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1345
2025-10-09T21:48:52.654824Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1345 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:52.654855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1345 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:52.654866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1345 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:52.654869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:52.655343Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1345 sessionId:  describe result for acl check
2025-10-09T21:48:52.655375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:52.655382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:52.655383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:52.655390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342295627894671:3905] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:52.655929Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342295627894671:3905] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:52.658111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342295627894671:3905] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:52.658546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:52.660481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:52.660720Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:52.660731Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:52.660743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:52.660804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:52.660841Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342295266070655:9871]
2025-10-09T21:48:52.660877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342295627894671:3905] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:52.660890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342295627894671:3905] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:52.660895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342295627894671:3905] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:52.660903Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1345 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:52.661425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:52.661715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:52.661761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:52.662100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:52.662147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.662153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.662155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.662156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.662157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.662158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.662160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.662161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.662162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.662167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.662177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.662181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.662185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.662188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.662192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.662196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.662200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.662204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.662210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.662217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.662223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.662228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.662235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.662246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.662250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.662255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.662260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.662265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.662274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.662279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.662284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.662289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.662298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.662304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.662309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.662314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.662342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:52.668795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:52.669842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:52.669858Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:52.669861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:52.669866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:52.669874Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1344 [2:7559342295266070655:9871]
2025-10-09T21:48:52.669888Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:52.669914Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:52.669921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:52.669925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:52.669927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:52.669946Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:52.669957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:52.669964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:52.670026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.670053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.670055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.670058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:52.670129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:52.670137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.670149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.670163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.670186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:52.670211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342295627894597:2432], now have 1 active actors on pipe
2025-10-09T21:48:52.670219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342295627894765:3905], now have 1 active actors on pipe
2025-10-09T21:48:52.670227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342295266070650:9871]
2025-10-09T21:48:52.670229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342295266070650:9871]
2025-10-09T21:48:52.670231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342295266070650:9871]
2025-10-09T21:48:52.670246Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:52.670215Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1344
2025-10-09T21:48:52.670221Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1344
2025-10-09T21:48:52.670259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:52.670261Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:52.670267Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:52.670269Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:52.670271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:52.670273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:52.670304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342295266070650:9871]
2025-10-09T21:48:52.670314Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:52.670319Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:52.670327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.670333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.670600Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d42c5e96-4f2eea88-6378d3b0-4997a003_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:52.670617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:52.670621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:52.670624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:52.670633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:52.670648Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:52.670671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:52.670863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.670873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.670887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:52.670972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1345 partition: 0 MaxSeqNo: 25 sessionId: src_id|d42c5e96-4f2eea88-6378d3b0-4997a003_0
2025-10-09T21:48:52.671344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c15f2ae-d43e8b4c-7df0b81f-9fcba0dd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d42c5e96-4f2eea88-6378d3b0-4997a003_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:52.673630Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d42c5e96-4f2eea88-6378d3b0-4997a003_0] Write session: try to update token
2025-10-09T21:48:52.673644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d42c5e96-4f2eea88-6378d3b0-4997a003_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:52.684183Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:52.684201Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:52.684203Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:52.684205Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:52.684214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342295266070650:9871]
2025-10-09T21:48:52.684217Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342295266070650:9871]
2025-10-09T21:48:52.684220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342295266070650:9871]
2025-10-09T21:48:52.684255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:52.684259Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:52.684261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.684263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:52.684264Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:52.684266Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:52.684384Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:52.685751Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1345 sessionId: src_id|d42c5e96-4f2eea88-6378d3b0-4997a003_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:52.688024Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:52.746240Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342295627894777:4303]
2025-10-09T21:48:52.746534Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 1344, pipe: [1:7559342295627894777:4303]
2025-10-09T21:48:52.833145Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:52.833484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.833496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.833536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342295627894777:4303], now have 1 active actors on pipe
2025-10-09T21:48:52.833716Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:52.833805Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:52.833806Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342295627894777:4303]
2025-10-09T21:48:52.833828Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:52.833901Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1345 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d42c5e96-4f2eea88-6378d3b0-4997a003_0
2025-10-09T21:48:52.834101Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1345 sessionId: src_id|d42c5e96-4f2eea88-6378d3b0-4997a003_0 is DEAD
2025-10-09T21:48:52.834284Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d42c5e96-4f2eea88-6378d3b0-4997a003_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:52.834298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d42c5e96-4f2eea88-6378d3b0-4997a003_0] Write session will restart in 0.007853s
2025-10-09T21:48:52.834317Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d42c5e96-4f2eea88-6378d3b0-4997a003_0] Write session: Do CDS request
2025-10-09T21:48:52.834323Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d42c5e96-4f2eea88-6378d3b0-4997a003_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:52.843532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d42c5e96-4f2eea88-6378d3b0-4997a003_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:52.843651Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:52.843671Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1346
2025-10-09T21:48:52.843779Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1346 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:52.843812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1346 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:52.843818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1346 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:52.843822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:52.844296Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1346 sessionId:  describe result for acl check
2025-10-09T21:48:52.844338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:52.844345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:52.844346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:52.844352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342295627894890:3897] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:52.844893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342295627894890:3897] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:52.846362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:52.846943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342295627894890:3897] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:52.848435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:52.848621Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:52.848633Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:52.848646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:52.848700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:52.848737Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342295266070828:9875]
2025-10-09T21:48:52.849167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342295627894890:3897] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:52.849180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342295627894890:3897] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:52.849184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342295627894890:3897] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:52.849193Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1346 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:52.849315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:52.849600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:52.849642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:52.849955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:52.850017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.850023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.850024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.850025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.850042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.850043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.850045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.850046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.850047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.850053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.850063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.850068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.850072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.850076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.850080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.850084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.850088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.850092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.850098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.850106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:52.850115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.850121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:52.850128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.850133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:52.850137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.850143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:52.850149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.850154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:52.850159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.850166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:52.850172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.850176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.850182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.850187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.850190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.850195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.850225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:52.856291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:52.857338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:52.857358Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:52.857360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:52.857365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:52.857372Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1345 [2:7559342295266070828:9875]
2025-10-09T21:48:52.857385Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:52.857402Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:52.857405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:52.857408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:52.857411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:52.857428Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:52.857440Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:52.857447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:52.857503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.857510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.857512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.857515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:52.857574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:52.857580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.857587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:52.857594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:52.857604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:52.857621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342295266070823:9875]
2025-10-09T21:48:52.857626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342295266070823:9875]
2025-10-09T21:48:52.857628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342295266070823:9875]
2025-10-09T21:48:52.857634Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:52.857645Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:52.857648Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:52.857648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:52.857650Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:52.857673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:52.857681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:52.857693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342295627894951:3897], now have 1 active actors on pipe
2025-10-09T21:48:52.857701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342295627894782:2432], now have 1 active actors on pipe
2025-10-09T21:48:52.857726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342295266070823:9875]
2025-10-09T21:48:52.857718Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1345
2025-10-09T21:48:52.857739Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:52.857758Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:52.857718Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1345
2025-10-09T21:48:52.857782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.857791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.857823Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|41236a05-a2b9ef0a-5b651457-3de20a2b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:52.857835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:52.857839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:52.857844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:52.857854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:52.857864Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:52.857884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:52.858051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.858061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.858077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:52.858181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1346 partition: 0 MaxSeqNo: 25 sessionId: src_id|41236a05-a2b9ef0a-5b651457-3de20a2b_0
2025-10-09T21:48:52.858486Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d42c5e96-4f2eea88-6378d3b0-4997a003_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|41236a05-a2b9ef0a-5b651457-3de20a2b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:52.860934Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|41236a05-a2b9ef0a-5b651457-3de20a2b_0] Write session: try to update token
2025-10-09T21:48:52.860953Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|41236a05-a2b9ef0a-5b651457-3de20a2b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:52.873650Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1346 sessionId: src_id|41236a05-a2b9ef0a-5b651457-3de20a2b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:52.874717Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:52.874734Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:52.874742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342295266070823:9875]
2025-10-09T21:48:52.874790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:52.874789Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:52.874953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:52.875095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:52.875224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:52.875370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:52.875561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:52.875599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:52.876016Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:52.876196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:52.876212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:52.876244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:52.876340Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:52.881550Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:52.881568Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:52.881571Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:52.881573Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:52.881583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342295266070823:9875]
2025-10-09T21:48:52.881587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342295266070823:9875]
2025-10-09T21:48:52.881591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342295266070823:9875]
2025-10-09T21:48:52.881631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:52.881635Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:52.881641Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:52.881643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:52.881645Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:52.881646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:52.881784Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:52.934121Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342295627894962:4303]
2025-10-09T21:48:52.934418Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 1345, pipe: [1:7559342295627894962:4303]
2025-10-09T21:48:53.036375Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:53.044238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:53.044270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:53.044439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:53.044502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:53.044570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:53.044635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:53.044679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:53.044689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:53.044759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:53.044824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:53.044889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:53.044959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:53.045002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:53.045011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:53.045076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:53.045136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:53.045201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:53.045259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:53.045304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:53.045307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:53.045373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:53.045436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:53.045497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:53.045558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:53.045601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:53.045610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:53.045687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:53.045750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:53.045813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:53.045878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:53.045923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:53.045931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:53.045995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:53.046073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:53.046138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:53.046200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:53.046264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:53.046272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:53.046331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:53.046393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:53.046457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:53.046520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:53.046564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:53.046572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:53.046633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:53.046688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:53.046750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:53.046810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:53.046855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:53.046863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:53.046927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:53.046987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:53.047051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:53.047117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:53.047161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:53.047169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:53.047229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:53.047286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:53.047345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:53.047406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:53.047454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:53.047461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:53.047528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342295627894962:4303], now have 1 active actors on pipe
2025-10-09T21:48:53.047782Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:53.047799Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:53.047802Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342295627894962:4303]
2025-10-09T21:48:53.047827Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:53.047834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1346 reason: error at write: Disconnected sessionId: src_id|41236a05-a2b9ef0a-5b651457-3de20a2b_0
2025-10-09T21:48:53.048050Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1346 sessionId: src_id|41236a05-a2b9ef0a-5b651457-3de20a2b_0 is DEAD
2025-10-09T21:48:53.048213Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41236a05-a2b9ef0a-5b651457-3de20a2b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:53.048226Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41236a05-a2b9ef0a-5b651457-3de20a2b_0] Write session will restart in 0.008168s
2025-10-09T21:48:53.048256Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41236a05-a2b9ef0a-5b651457-3de20a2b_0] Write session: Do CDS request
2025-10-09T21:48:53.048262Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41236a05-a2b9ef0a-5b651457-3de20a2b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:53.048687Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:53.057578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|41236a05-a2b9ef0a-5b651457-3de20a2b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:53.057712Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:53.057733Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1347
2025-10-09T21:48:53.057876Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1347 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:53.057913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1347 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:53.057926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1347 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:53.057929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:53.058438Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1347 sessionId:  describe result for acl check
2025-10-09T21:48:53.058479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:53.058486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:53.058488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:53.058494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342299922862309:3890] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:53.059047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342299922862309:3890] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:53.061452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342299922862309:3890] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:53.063110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:53.065125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:53.065337Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:53.065347Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:53.065360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:53.065411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:53.065451Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342299561038296:9877]
2025-10-09T21:48:53.065537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342299922862309:3890] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:53.065556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342299922862309:3890] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:53.065560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342299922862309:3890] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:53.065569Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1347 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:53.066097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:53.066444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:53.066489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:53.066722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:53.066756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.066760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.066761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.066762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.066762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.066763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.066764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.066765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.066766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.066769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.066776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.066778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.066781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.066783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.066785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.066787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.066789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.066791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.066797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.066805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.066810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.066815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.066818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.066823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.066827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.066831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.066836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.066841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.066845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.066850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.066854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.066859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.066863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.066868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.066872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.066877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.066898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:53.073338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:53.074682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:53.074700Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:53.074703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:53.074707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:53.074714Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1346 [2:7559342299561038296:9877]
2025-10-09T21:48:53.074729Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:53.074749Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:53.074752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:53.074756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:53.074758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:53.074778Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:53.074790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:53.074798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:53.074862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.074870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.074872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.074875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:53.074949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:53.074956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.074969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.074987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.075009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:53.075039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342299922862261:2432], now have 1 active actors on pipe
2025-10-09T21:48:53.075047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342299922862431:3890], now have 1 active actors on pipe
2025-10-09T21:48:53.075056Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342299561038291:9877]
2025-10-09T21:48:53.075063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342299561038291:9877]
2025-10-09T21:48:53.075066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342299561038291:9877]
2025-10-09T21:48:53.075079Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:53.075093Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:53.075098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:53.075104Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:53.075106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:53.075113Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:53.075116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:53.075154Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342299561038291:9877]
2025-10-09T21:48:53.075167Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:53.075177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:53.075164Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1346
2025-10-09T21:48:53.075221Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1346
2025-10-09T21:48:53.075480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:53.075494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:53.075523Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dfe935e0-51fdc555-616be6a-5cd55104_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:53.075532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:53.075537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:53.075540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:53.075550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:53.075561Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:53.075577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:53.075766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:53.075774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:53.075789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:53.075922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1347 partition: 0 MaxSeqNo: 25 sessionId: src_id|dfe935e0-51fdc555-616be6a-5cd55104_0
2025-10-09T21:48:53.076296Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41236a05-a2b9ef0a-5b651457-3de20a2b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dfe935e0-51fdc555-616be6a-5cd55104_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:53.078992Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfe935e0-51fdc555-616be6a-5cd55104_0] Write session: try to update token
2025-10-09T21:48:53.079026Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfe935e0-51fdc555-616be6a-5cd55104_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:53.089990Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:53.090011Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:53.090013Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:53.090015Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:53.090025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342299561038291:9877]
2025-10-09T21:48:53.090048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342299561038291:9877]
2025-10-09T21:48:53.090053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342299561038291:9877]
2025-10-09T21:48:53.090098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:53.090106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.090109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:53.090120Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:53.090135Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:53.090139Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:53.090254Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:53.092950Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1347 sessionId: src_id|dfe935e0-51fdc555-616be6a-5cd55104_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:53.095387Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:53.142175Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] Counters: { Errors: 13 CurrentSessionLifetimeMs: 361593 BytesRead: 24780812 MessagesRead: 11 BytesReadCompressed: 24780812 BytesInflightUncompressed: 2252802 BytesInflightCompressed: 0 BytesInflightTotal: 2252802 MessagesInflight: 1 }
2025-10-09T21:48:53.148183Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342299922862442:4303]
2025-10-09T21:48:53.148440Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 1346, pipe: [1:7559342299922862442:4303]
2025-10-09T21:48:53.249487Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:53.249856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:53.249870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:53.249909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342299922862442:4303], now have 1 active actors on pipe
2025-10-09T21:48:53.250115Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:53.250164Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:53.250180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:53.250212Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:53.250244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1347 reason: error at write: Disconnected sessionId: src_id|dfe935e0-51fdc555-616be6a-5cd55104_0
2025-10-09T21:48:53.250269Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342299922862442:4303]
2025-10-09T21:48:53.250422Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1347 sessionId: src_id|dfe935e0-51fdc555-616be6a-5cd55104_0 is DEAD
2025-10-09T21:48:53.250561Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfe935e0-51fdc555-616be6a-5cd55104_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:53.250569Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfe935e0-51fdc555-616be6a-5cd55104_0] Write session will restart in 0.005104s
2025-10-09T21:48:53.250606Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfe935e0-51fdc555-616be6a-5cd55104_0] Write session: Do CDS request
2025-10-09T21:48:53.250613Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfe935e0-51fdc555-616be6a-5cd55104_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:53.256541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfe935e0-51fdc555-616be6a-5cd55104_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:53.256622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:53.256639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1348
2025-10-09T21:48:53.256767Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1348 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:53.256801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1348 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:53.256811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1348 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:53.256815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:53.257251Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1348 sessionId:  describe result for acl check
2025-10-09T21:48:53.257285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:53.257292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:53.257294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:53.257300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342299922862468:3866] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:53.257858Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342299922862468:3866] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:53.260469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342299922862468:3866] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:53.263101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:53.263425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342299922862468:3866] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:53.263441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342299922862468:3866] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:53.263445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342299922862468:3866] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:53.263455Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1348 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:53.265285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:53.265482Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:53.265495Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:53.265506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:53.265565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:53.265609Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342299561038471:9879]
2025-10-09T21:48:53.266224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:53.266532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:53.266583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:53.266870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:53.266931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.266938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.266939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.266940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.266941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.266942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.266943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.266945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.266946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.266952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.266963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.266967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.266971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.266975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.266979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.266983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.266986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.266989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.266997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.267005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.267011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.267016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.267020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.267025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.267030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.267042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.267046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.267051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.267062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.267069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.267078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.267084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.267089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.267094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.267099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.267105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.267135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:53.273689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:53.274807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:53.274826Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:53.274828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:53.274833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:53.274841Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1347 [2:7559342299561038471:9879]
2025-10-09T21:48:53.274854Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:53.274873Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:53.274882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:53.274885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:53.274888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:53.274906Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:53.274917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:53.274926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:53.274988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.274995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.274998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.275001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:53.275070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:53.275077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.275086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.275098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.275115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:53.275137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342299922862453:2432], now have 1 active actors on pipe
2025-10-09T21:48:53.275144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342299922862624:3866], now have 1 active actors on pipe
2025-10-09T21:48:53.275151Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342299561038465:9879]
2025-10-09T21:48:53.275153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342299561038465:9879]
2025-10-09T21:48:53.275155Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342299561038465:9879]
2025-10-09T21:48:53.275165Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:53.275176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:53.275185Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:53.275188Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:53.275190Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:53.275194Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:53.275198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:53.275232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342299561038465:9879]
2025-10-09T21:48:53.275239Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:53.275213Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1347
2025-10-09T21:48:53.275243Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:53.275220Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1347
2025-10-09T21:48:53.275261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:53.275268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:53.275305Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fb8c930d-80f97231-20ae5579-508246fe_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:53.275318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:53.275327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:53.275330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:53.275340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:53.275349Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:53.275379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:53.275530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:53.275535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:53.275546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:53.275609Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1348 partition: 0 MaxSeqNo: 25 sessionId: src_id|fb8c930d-80f97231-20ae5579-508246fe_0
2025-10-09T21:48:53.275961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfe935e0-51fdc555-616be6a-5cd55104_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fb8c930d-80f97231-20ae5579-508246fe_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:53.278469Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fb8c930d-80f97231-20ae5579-508246fe_0] Write session: try to update token
2025-10-09T21:48:53.278485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fb8c930d-80f97231-20ae5579-508246fe_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:53.289582Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:53.289604Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:53.289606Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:53.289608Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:53.289618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342299561038465:9879]
2025-10-09T21:48:53.289622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342299561038465:9879]
2025-10-09T21:48:53.289628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342299561038465:9879]
2025-10-09T21:48:53.289684Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:53.289688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:53.289696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.289697Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:53.289699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:53.289700Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:53.289839Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:53.291340Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1348 sessionId: src_id|fb8c930d-80f97231-20ae5579-508246fe_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:53.293800Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:53.350371Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342299922862636:4303]
2025-10-09T21:48:53.350682Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 1347, pipe: [1:7559342299922862636:4303]
2025-10-09T21:48:53.438231Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:53.438501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:53.438514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:53.438560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342299922862636:4303], now have 1 active actors on pipe
2025-10-09T21:48:53.438755Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:53.438784Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342299922862636:4303]
2025-10-09T21:48:53.438825Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:53.438828Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:53.438850Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:53.438888Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1348 reason: error at write: Disconnected sessionId: src_id|fb8c930d-80f97231-20ae5579-508246fe_0
2025-10-09T21:48:53.439062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1348 sessionId: src_id|fb8c930d-80f97231-20ae5579-508246fe_0 is DEAD
2025-10-09T21:48:53.439182Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb8c930d-80f97231-20ae5579-508246fe_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:53.439192Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb8c930d-80f97231-20ae5579-508246fe_0] Write session will restart in 0.005356s
2025-10-09T21:48:53.439213Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb8c930d-80f97231-20ae5579-508246fe_0] Write session: Do CDS request
2025-10-09T21:48:53.439218Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb8c930d-80f97231-20ae5579-508246fe_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:53.445469Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fb8c930d-80f97231-20ae5579-508246fe_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:53.445572Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:53.445588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1349
2025-10-09T21:48:53.445738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1349 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:53.445779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1349 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:53.445789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1349 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:53.445793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:53.446186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1349 sessionId:  describe result for acl check
2025-10-09T21:48:53.446217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:53.446218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:53.446219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:53.446230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342299922862662:3868] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:53.446636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342299922862662:3868] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:53.448633Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342299922862662:3868] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:53.450665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342299922862662:3868] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:53.450678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342299922862662:3868] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:53.450683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342299922862662:3868] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:53.450690Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1349 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:53.450699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:53.452573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:53.452741Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:53.452752Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:53.452764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:53.452815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:53.452851Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342299561038645:9881]
2025-10-09T21:48:53.453275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:53.453546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:53.453584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:53.453796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:53.453859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.453865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.453866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.453867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.453867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.453868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.453868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.453869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.453869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.453873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.453879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.453881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.453883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.453885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.453887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.453889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.453891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.453893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.453897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.453902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.453906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.453909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.453912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.453914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.453916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.453919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.453922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.453924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.453927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.453929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.453932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.453935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.453938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.453940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.453943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.453946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.453961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:53.459668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:53.460663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:53.460679Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:53.460681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:53.460683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:53.460688Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1348 [2:7559342299561038645:9881]
2025-10-09T21:48:53.460696Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:53.460711Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:53.460713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:53.460715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:53.460716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:53.460727Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:53.460735Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:53.460741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:53.460778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.460783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.460784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.460786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:53.460846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:53.460852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.460859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.460865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.460874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:53.460890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342299561038638:9881]
2025-10-09T21:48:53.460896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342299561038638:9881]
2025-10-09T21:48:53.460898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342299561038638:9881]
2025-10-09T21:48:53.460910Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:53.460924Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:53.460928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342299922862638:2432], now have 1 active actors on pipe
2025-10-09T21:48:53.460932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342299922862808:3868], now have 1 active actors on pipe
2025-10-09T21:48:53.460935Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:53.460937Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:53.460940Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:53.460953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:53.460961Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:53.460944Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1348
2025-10-09T21:48:53.460977Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1348
2025-10-09T21:48:53.460983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342299561038638:9881]
2025-10-09T21:48:53.460999Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:53.461006Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:53.461019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:53.461024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:53.461068Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|34c32f97-f2d5e6ab-558e53ac-f579a641_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:53.461085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:53.461089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:53.461091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:53.461100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:53.461110Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:53.461132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:53.461346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:53.461358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:53.461375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:53.461458Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1349 partition: 0 MaxSeqNo: 25 sessionId: src_id|34c32f97-f2d5e6ab-558e53ac-f579a641_0
2025-10-09T21:48:53.461764Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fb8c930d-80f97231-20ae5579-508246fe_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|34c32f97-f2d5e6ab-558e53ac-f579a641_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:53.464054Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|34c32f97-f2d5e6ab-558e53ac-f579a641_0] Write session: try to update token
2025-10-09T21:48:53.464073Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|34c32f97-f2d5e6ab-558e53ac-f579a641_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:53.475607Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:53.475635Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:53.475639Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:53.475642Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:53.475655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342299561038638:9881]
2025-10-09T21:48:53.475662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342299561038638:9881]
2025-10-09T21:48:53.475668Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342299561038638:9881]
2025-10-09T21:48:53.475734Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:53.475738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:53.475747Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:53.475747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.475751Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:53.475751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:53.476011Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:53.477345Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1349 sessionId: src_id|34c32f97-f2d5e6ab-558e53ac-f579a641_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:53.479613Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:53.538826Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342299922862822:4303]
2025-10-09T21:48:53.539083Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 1348, pipe: [1:7559342299922862822:4303]
2025-10-09T21:48:53.623562Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:53.623930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:53.623951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:53.624002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342299922862822:4303], now have 1 active actors on pipe
2025-10-09T21:48:53.624195Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:53.624231Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:53.624235Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342299922862822:4303]
2025-10-09T21:48:53.624258Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:53.624326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1349 reason: status is not ok: tablet will be restarted right now sessionId: src_id|34c32f97-f2d5e6ab-558e53ac-f579a641_0
2025-10-09T21:48:53.624511Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1349 sessionId: src_id|34c32f97-f2d5e6ab-558e53ac-f579a641_0 is DEAD
2025-10-09T21:48:53.624678Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34c32f97-f2d5e6ab-558e53ac-f579a641_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:53.624690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34c32f97-f2d5e6ab-558e53ac-f579a641_0] Write session will restart in 0.005373s
2025-10-09T21:48:53.624717Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34c32f97-f2d5e6ab-558e53ac-f579a641_0] Write session: Do CDS request
2025-10-09T21:48:53.624725Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34c32f97-f2d5e6ab-558e53ac-f579a641_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:53.631529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|34c32f97-f2d5e6ab-558e53ac-f579a641_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:53.631626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:53.631644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1350
2025-10-09T21:48:53.631791Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1350 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:53.631826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1350 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:53.631832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1350 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:53.631835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:53.632274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1350 sessionId:  describe result for acl check
2025-10-09T21:48:53.632329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:53.632337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:53.632338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:53.632345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342299922862863:3858] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:53.632962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342299922862863:3858] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:53.635692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342299922862863:3858] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:53.638356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342299922862863:3858] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:53.638368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342299922862863:3858] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:53.638370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342299922862863:3858] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:53.638377Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1350 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:53.638555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:53.640567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:53.640905Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:53.640918Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:53.640931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:53.640994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:53.641035Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342299561038858:9895]
2025-10-09T21:48:53.641620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:53.641904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:53.641941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:53.642250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:53.642301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.642308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.642309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.642310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.642311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.642313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.642314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.642315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.642316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.642321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.642332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.642336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.642340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.642344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.642348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.642352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.642356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.642359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.642367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.642375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.642386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.642395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.642404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.642414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.642422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.642427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.642435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.642442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.642450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.642457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.642465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.642472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.642481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.642490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.642495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.642500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.642528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:53.648790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:53.649925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:53.649939Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:53.649941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:53.649944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:53.649950Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1349 [2:7559342299561038858:9895]
2025-10-09T21:48:53.649958Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:53.649974Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:53.649975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:53.649978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:53.649979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:53.649993Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:53.649997Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:53.650004Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:53.650063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.650068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.650070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.650072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:53.650140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:53.650149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.650161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.650175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.650195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:53.650215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342299922862848:2432], now have 1 active actors on pipe
2025-10-09T21:48:53.650221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342299922863017:3858], now have 1 active actors on pipe
2025-10-09T21:48:53.650227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342299561038852:9895]
2025-10-09T21:48:53.650229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342299561038852:9895]
2025-10-09T21:48:53.650230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342299561038852:9895]
2025-10-09T21:48:53.650240Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:53.650255Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:53.650256Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:53.650259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:53.650262Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:53.650263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:53.650265Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:53.650270Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1349
2025-10-09T21:48:53.650293Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342299561038852:9895]
2025-10-09T21:48:53.650291Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1349
2025-10-09T21:48:53.650299Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:53.650310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:53.650330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:53.650340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:53.650374Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2e60bf57-eb1e5861-8c94d00c-9ec765b9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:53.650383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:53.650389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:53.650394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:53.650402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:53.650415Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:53.650432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:53.650585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:53.650592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:53.650606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:53.650691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1350 partition: 0 MaxSeqNo: 25 sessionId: src_id|2e60bf57-eb1e5861-8c94d00c-9ec765b9_0
2025-10-09T21:48:53.651021Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34c32f97-f2d5e6ab-558e53ac-f579a641_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2e60bf57-eb1e5861-8c94d00c-9ec765b9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:53.653438Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e60bf57-eb1e5861-8c94d00c-9ec765b9_0] Write session: try to update token
2025-10-09T21:48:53.653453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e60bf57-eb1e5861-8c94d00c-9ec765b9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:53.663923Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:53.663942Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:53.663944Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:53.663945Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:53.663953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342299561038852:9895]
2025-10-09T21:48:53.663957Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342299561038852:9895]
2025-10-09T21:48:53.663960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342299561038852:9895]
2025-10-09T21:48:53.663995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:53.664000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.664002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:53.664014Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:53.664029Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:53.664034Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:53.664138Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:53.666264Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1350 sessionId: src_id|2e60bf57-eb1e5861-8c94d00c-9ec765b9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:53.668526Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:53.724170Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342299922863030:4303]
2025-10-09T21:48:53.724432Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 1349, pipe: [1:7559342299922863030:4303]
2025-10-09T21:48:53.807839Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:53.808143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:53.808159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:53.808205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342299922863030:4303], now have 1 active actors on pipe
2025-10-09T21:48:53.808398Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:53.808419Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:53.808465Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342299922863030:4303]
2025-10-09T21:48:53.808479Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1350 reason: error at write: Disconnected sessionId: src_id|2e60bf57-eb1e5861-8c94d00c-9ec765b9_0
2025-10-09T21:48:53.808493Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:53.808654Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1350 sessionId: src_id|2e60bf57-eb1e5861-8c94d00c-9ec765b9_0 is DEAD
2025-10-09T21:48:53.808647Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:53.808808Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e60bf57-eb1e5861-8c94d00c-9ec765b9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:53.808815Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e60bf57-eb1e5861-8c94d00c-9ec765b9_0] Write session will restart in 0.009066s
2025-10-09T21:48:53.808834Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e60bf57-eb1e5861-8c94d00c-9ec765b9_0] Write session: Do CDS request
2025-10-09T21:48:53.808840Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e60bf57-eb1e5861-8c94d00c-9ec765b9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:53.819528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e60bf57-eb1e5861-8c94d00c-9ec765b9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:53.819596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:53.819614Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1351
2025-10-09T21:48:53.819730Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1351 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:53.819762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1351 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:53.819773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1351 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:53.819776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:53.820313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1351 sessionId:  describe result for acl check
2025-10-09T21:48:53.820357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:53.820363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:53.820365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:53.820371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342299922863169:3851] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:53.820922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342299922863169:3851] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:53.821561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:53.822889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342299922863169:3851] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:53.823716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:53.823919Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:53.823929Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:53.823941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:53.823998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:53.824037Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342299561039037:9899]
2025-10-09T21:48:53.824593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:53.824872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:53.824913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:53.825147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342299922863169:3851] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:53.825161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342299922863169:3851] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:53.825164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342299922863169:3851] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:53.825173Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1351 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:53.825211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:53.825342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.825349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.825351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.825352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.825353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.825354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.825356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.825357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.825358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.825363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.825374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.825378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.825382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.825386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.825390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.825393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.825397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.825401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.825407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.825414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:53.825420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.825430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:53.825436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.825445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:53.825454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.825460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:53.825464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.825474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:53.825478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.825483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:53.825488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.825493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.825497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.825502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.825506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.825511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.825537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:53.831788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:53.833034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:53.833051Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:53.833053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:53.833058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:53.833064Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1350 [2:7559342299561039037:9899]
2025-10-09T21:48:53.833077Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:53.833097Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:53.833106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:53.833110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:53.833112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:53.833129Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:53.833135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:53.833143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:53.833199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.833209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.833211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.833214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:53.833279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:53.833290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.833302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:53.833314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:53.833329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:53.833353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342299922863032:2432], now have 1 active actors on pipe
2025-10-09T21:48:53.833362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342299922863210:3851], now have 1 active actors on pipe
2025-10-09T21:48:53.833371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342299561039032:9899]
2025-10-09T21:48:53.833374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342299561039032:9899]
2025-10-09T21:48:53.833376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342299561039032:9899]
2025-10-09T21:48:53.833389Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:53.833407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:53.833411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:53.833415Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:53.833417Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:53.833423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:53.833426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:53.833455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342299561039032:9899]
2025-10-09T21:48:53.833474Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:53.833487Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1350
2025-10-09T21:48:53.833502Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1350
2025-10-09T21:48:53.833498Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:53.833608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:53.833621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:53.833665Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6be59c87-8523f4db-b2620bf0-c35dd584_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:53.833680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:53.833684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:53.833687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:53.833699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:53.833715Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:53.833736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:53.833928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:53.833937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:53.833952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:53.834062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1351 partition: 0 MaxSeqNo: 25 sessionId: src_id|6be59c87-8523f4db-b2620bf0-c35dd584_0
2025-10-09T21:48:53.834385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e60bf57-eb1e5861-8c94d00c-9ec765b9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6be59c87-8523f4db-b2620bf0-c35dd584_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:53.836765Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6be59c87-8523f4db-b2620bf0-c35dd584_0] Write session: try to update token
2025-10-09T21:48:53.836780Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6be59c87-8523f4db-b2620bf0-c35dd584_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:53.844425Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:53.844446Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:53.844456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342299561039032:9899]
2025-10-09T21:48:53.844504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:53.844519Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:53.844683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:53.844835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:53.844973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:53.845120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:53.845315Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:53.845357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:53.849096Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1351 sessionId: src_id|6be59c87-8523f4db-b2620bf0-c35dd584_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:53.851406Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:53.851602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:53.851624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:53.851666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:53.851763Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:53.856884Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:53.856904Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:53.856907Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:53.856908Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:53.856918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342299561039032:9899]
2025-10-09T21:48:53.856922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342299561039032:9899]
2025-10-09T21:48:53.856925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342299561039032:9899]
2025-10-09T21:48:53.856969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:53.856970Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:53.856976Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:53.856979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:53.856980Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:53.856982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:53.857119Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:53.908502Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342299922863220:4303]
2025-10-09T21:48:53.908779Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 1350, pipe: [1:7559342299922863220:4303]
2025-10-09T21:48:54.006127Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:54.014203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.014232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.014406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.014472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.014537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.014604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.014662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.014674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.014749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.014810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.014876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.014947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.014993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.015001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.015066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.015124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.015183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.015243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.015635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.015652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.015724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.015946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.016013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.016072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.016115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.016124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.016190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.016246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.016302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.016368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.016414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.016419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.016476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.016530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.016593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.016654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.016695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.016701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.016756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.016809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.016877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.016935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.016976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.016981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.017036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.017086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.017137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.017190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.017230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.017235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.017292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.017342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.017398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.017455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.017505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.017518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.017579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.017634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.017706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.017780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.017831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.017838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.017902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342299922863220:4303], now have 1 active actors on pipe
2025-10-09T21:48:54.018168Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:54.018202Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342299922863220:4303]
2025-10-09T21:48:54.018222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:54.018229Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:54.018259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1351 reason: error at write: Disconnected sessionId: src_id|6be59c87-8523f4db-b2620bf0-c35dd584_0
2025-10-09T21:48:54.018327Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:54.018449Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1351 sessionId: src_id|6be59c87-8523f4db-b2620bf0-c35dd584_0 is DEAD
2025-10-09T21:48:54.018593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6be59c87-8523f4db-b2620bf0-c35dd584_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:54.018604Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6be59c87-8523f4db-b2620bf0-c35dd584_0] Write session will restart in 0.007329s
2025-10-09T21:48:54.018628Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6be59c87-8523f4db-b2620bf0-c35dd584_0] Write session: Do CDS request
2025-10-09T21:48:54.018636Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6be59c87-8523f4db-b2620bf0-c35dd584_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:54.027509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6be59c87-8523f4db-b2620bf0-c35dd584_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:54.027623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:54.027640Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1352
2025-10-09T21:48:54.028040Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1352 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:54.028077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1352 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:54.028088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1352 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:54.028092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:54.028558Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1352 sessionId:  describe result for acl check
2025-10-09T21:48:54.028600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:54.028606Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:54.028608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:54.028614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342304217830587:3848] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:54.029121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342304217830587:3848] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:54.031663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342304217830587:3848] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:54.032270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:54.034446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:54.034646Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:54.034658Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:54.034670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:54.034723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:54.034764Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342303856006510:9901]
2025-10-09T21:48:54.034807Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342304217830587:3848] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:54.034822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342304217830587:3848] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:54.034825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342304217830587:3848] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:54.034834Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1352 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:54.035355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:54.035630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:54.035672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:54.035993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:54.036042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.036048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.036049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.036051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.036052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.036053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.036054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.036055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.036056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.036061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.036072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.036077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.036081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.036085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.036093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.036097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.036101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.036110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.036117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.036124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.036135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.036140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.036149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.036154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.036159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.036164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.036168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.036173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.036177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.036182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.036187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.036192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.036197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.036202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.036206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.036212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.036245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:54.042815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:54.044100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:54.044119Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:54.044123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:54.044127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:54.044139Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1351 [2:7559342303856006510:9901]
2025-10-09T21:48:54.044154Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:54.044172Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:54.044175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:54.044179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:54.044181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:54.044207Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:54.044219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:54.044231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:54.044299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.044307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.044309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.044312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:54.044386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:54.044395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.044408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.044421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.044439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:54.044465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342303856006505:9901]
2025-10-09T21:48:54.044472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342303856006505:9901]
2025-10-09T21:48:54.044474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342303856006505:9901]
2025-10-09T21:48:54.044487Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:54.044501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:54.044505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217830521:2432], now have 1 active actors on pipe
2025-10-09T21:48:54.044509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217830693:3848], now have 1 active actors on pipe
2025-10-09T21:48:54.044512Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:54.044514Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:54.044518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:54.044534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:54.044537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:54.044572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342303856006505:9901]
2025-10-09T21:48:54.044583Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:54.044590Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:54.044586Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1351
2025-10-09T21:48:54.044640Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1351
2025-10-09T21:48:54.044671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.044683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.044712Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b6b399d4-6992f364-7766904b-ed89a371_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:54.044720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:54.044725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:54.044728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:54.044741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:54.044756Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:54.044772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:54.044950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.044959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.044974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:54.045100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1352 partition: 0 MaxSeqNo: 25 sessionId: src_id|b6b399d4-6992f364-7766904b-ed89a371_0
2025-10-09T21:48:54.045466Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6be59c87-8523f4db-b2620bf0-c35dd584_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b6b399d4-6992f364-7766904b-ed89a371_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:54.047989Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6b399d4-6992f364-7766904b-ed89a371_0] Write session: try to update token
2025-10-09T21:48:54.048010Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6b399d4-6992f364-7766904b-ed89a371_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:54.059015Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:54.059035Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:54.059037Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:54.059039Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:54.059050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342303856006505:9901]
2025-10-09T21:48:54.059055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342303856006505:9901]
2025-10-09T21:48:54.059059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342303856006505:9901]
2025-10-09T21:48:54.059103Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:54.059104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:54.059107Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:54.059110Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:54.059113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.059116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:54.059257Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:54.061124Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1352 sessionId: src_id|b6b399d4-6992f364-7766904b-ed89a371_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:54.063542Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:54.118369Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342304217830705:4303]
2025-10-09T21:48:54.118610Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 1351, pipe: [1:7559342304217830705:4303]
2025-10-09T21:48:54.213535Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:54.213949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.213964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.214007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217830705:4303], now have 1 active actors on pipe
2025-10-09T21:48:54.214215Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:54.214302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:54.214312Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342304217830705:4303]
2025-10-09T21:48:54.214317Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:54.214334Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:54.214374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1352 reason: error at write: Disconnected sessionId: src_id|b6b399d4-6992f364-7766904b-ed89a371_0
2025-10-09T21:48:54.214544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1352 sessionId: src_id|b6b399d4-6992f364-7766904b-ed89a371_0 is DEAD
2025-10-09T21:48:54.214701Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6b399d4-6992f364-7766904b-ed89a371_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:54.214711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6b399d4-6992f364-7766904b-ed89a371_0] Write session will restart in 0.009764s
2025-10-09T21:48:54.214739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6b399d4-6992f364-7766904b-ed89a371_0] Write session: Do CDS request
2025-10-09T21:48:54.214745Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6b399d4-6992f364-7766904b-ed89a371_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:54.225548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6b399d4-6992f364-7766904b-ed89a371_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:54.225756Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:54.225774Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1353
2025-10-09T21:48:54.225895Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1353 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:54.225929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1353 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:54.225939Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1353 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:54.225943Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:54.226472Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1353 sessionId:  describe result for acl check
2025-10-09T21:48:54.226510Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:54.226519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:54.226521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:54.226527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342304217830833:3842] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:54.227097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342304217830833:3842] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:54.228230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:54.229295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342304217830833:3842] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:54.230255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:54.230409Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:54.230417Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:54.230425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:54.230477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:54.230511Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342303856006688:9903]
2025-10-09T21:48:54.231021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:54.231336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:54.231392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:54.231641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342304217830833:3842] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:54.231651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342304217830833:3842] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:54.231654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342304217830833:3842] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:54.231661Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1353 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:54.231827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:54.231887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.231893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.231895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.231896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.231897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.231899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.231900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.231901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.231902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.231908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.231919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.231923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.231928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.231932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.231936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.231940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.231944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.231947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.231954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.231962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.231973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.231983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.231988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.231993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.231998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.232010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.232015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.232020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.232025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.232030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.232035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.232040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.232045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.232051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.232056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.232061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.232091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:54.238607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:54.239632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:54.239649Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:54.239652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:54.239657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:54.239665Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1352 [2:7559342303856006688:9903]
2025-10-09T21:48:54.239680Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:54.239697Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:54.239701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:54.239704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:54.239707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:54.239725Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:54.239737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:54.239745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:54.239814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.239821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.239823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.239826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:54.239891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:54.239899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.239912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.239923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.239943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:54.239970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217830710:2432], now have 1 active actors on pipe
2025-10-09T21:48:54.239979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217830893:3842], now have 1 active actors on pipe
2025-10-09T21:48:54.239986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342303856006683:9903]
2025-10-09T21:48:54.239993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342303856006683:9903]
2025-10-09T21:48:54.239995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342303856006683:9903]
2025-10-09T21:48:54.240010Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:54.240030Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:54.240031Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:54.240035Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:54.240038Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:54.240050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:54.240059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:54.240038Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1352
2025-10-09T21:48:54.240093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342303856006683:9903]
2025-10-09T21:48:54.240046Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1352
2025-10-09T21:48:54.240096Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:54.240099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:54.240117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.240122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.240175Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7bfbff65-6157f3c2-e2990efa-24ba2561_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:54.240189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:54.240192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:54.240194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:54.240205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:54.240216Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:54.240240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:54.240458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.240473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.240493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:54.240595Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1353 partition: 0 MaxSeqNo: 25 sessionId: src_id|7bfbff65-6157f3c2-e2990efa-24ba2561_0
2025-10-09T21:48:54.240941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6b399d4-6992f364-7766904b-ed89a371_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7bfbff65-6157f3c2-e2990efa-24ba2561_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:54.243426Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7bfbff65-6157f3c2-e2990efa-24ba2561_0] Write session: try to update token
2025-10-09T21:48:54.243443Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7bfbff65-6157f3c2-e2990efa-24ba2561_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:54.254563Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:54.254585Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:54.254588Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:54.254591Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:54.254600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342303856006683:9903]
2025-10-09T21:48:54.254605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342303856006683:9903]
2025-10-09T21:48:54.254608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342303856006683:9903]
2025-10-09T21:48:54.254645Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:54.254653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:54.254658Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:54.254660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.254662Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:54.254663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:54.254810Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:54.256441Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1353 sessionId: src_id|7bfbff65-6157f3c2-e2990efa-24ba2561_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:54.258769Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:54.314498Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342304217830902:4303]
2025-10-09T21:48:54.314767Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 1352, pipe: [1:7559342304217830902:4303]
2025-10-09T21:48:54.406212Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:54.406483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.406495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.406533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217830902:4303], now have 1 active actors on pipe
2025-10-09T21:48:54.406698Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:54.406748Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:54.406752Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:54.406807Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342304217830902:4303]
2025-10-09T21:48:54.406822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1353 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7bfbff65-6157f3c2-e2990efa-24ba2561_0
2025-10-09T21:48:54.407014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1353 sessionId: src_id|7bfbff65-6157f3c2-e2990efa-24ba2561_0 is DEAD
2025-10-09T21:48:54.407167Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bfbff65-6157f3c2-e2990efa-24ba2561_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:54.407188Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bfbff65-6157f3c2-e2990efa-24ba2561_0] Write session will restart in 0.006432s
2025-10-09T21:48:54.407214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bfbff65-6157f3c2-e2990efa-24ba2561_0] Write session: Do CDS request
2025-10-09T21:48:54.407221Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bfbff65-6157f3c2-e2990efa-24ba2561_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:54.414491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7bfbff65-6157f3c2-e2990efa-24ba2561_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:54.414563Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:54.414580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1354
2025-10-09T21:48:54.414671Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1354 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:54.414705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1354 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:54.414710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1354 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:54.414714Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:54.415158Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1354 sessionId:  describe result for acl check
2025-10-09T21:48:54.415195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:54.415200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:54.415202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:54.415207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342304217830948:3839] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:54.415738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342304217830948:3839] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:54.417884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342304217830948:3839] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:54.419650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:54.421510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:54.421717Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:54.421714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342304217830948:3839] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:54.421728Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:54.421730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342304217830948:3839] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:54.421740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:54.421734Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342304217830948:3839] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:54.421742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1354 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:54.421791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:54.421828Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342303856006867:9905]
2025-10-09T21:48:54.422456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:54.422761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:54.422810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:54.423121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:54.423169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.423175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.423176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.423177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.423179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.423180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.423181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.423182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.423184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.423188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.423200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.423208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.423213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.423217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.423221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.423229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.423234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.423237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.423247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.423258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.423268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.423278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.423287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.423292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.423300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.423306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.423315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.423320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.423329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.423336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.423341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.423351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.423360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.423367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.423375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.423381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.423413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:54.429485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:54.430543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:54.430560Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:54.430563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:54.430567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:54.430573Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1353 [2:7559342303856006867:9905]
2025-10-09T21:48:54.430597Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:54.430620Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:54.430627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:54.430630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:54.430632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:54.430647Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:54.430660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:54.430668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:54.430719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.430726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.430728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.430731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:54.430797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:54.430804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.430816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.430829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.430847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:54.430872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217831082:3839], now have 1 active actors on pipe
2025-10-09T21:48:54.430882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217830905:2432], now have 1 active actors on pipe
2025-10-09T21:48:54.430889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342303856006862:9905]
2025-10-09T21:48:54.430891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342303856006862:9905]
2025-10-09T21:48:54.430893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342303856006862:9905]
2025-10-09T21:48:54.430907Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:54.430921Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:54.430925Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:54.430918Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1353
2025-10-09T21:48:54.430928Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:54.430921Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1353
2025-10-09T21:48:54.430930Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:54.430939Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:54.430941Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:54.430967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342303856006862:9905]
2025-10-09T21:48:54.430971Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:54.430972Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:54.430990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.431000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.431044Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2a7c19b-abd83ba4-59623a22-c8d28984_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:54.431056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:54.431062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:54.431065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:54.431075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:54.431090Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:54.431112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:54.431233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.431241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.431254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:54.431394Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1354 partition: 0 MaxSeqNo: 25 sessionId: src_id|2a7c19b-abd83ba4-59623a22-c8d28984_0
2025-10-09T21:48:54.431758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bfbff65-6157f3c2-e2990efa-24ba2561_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2a7c19b-abd83ba4-59623a22-c8d28984_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:54.434055Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a7c19b-abd83ba4-59623a22-c8d28984_0] Write session: try to update token
2025-10-09T21:48:54.434071Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a7c19b-abd83ba4-59623a22-c8d28984_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:54.444267Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:54.444292Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:54.444295Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:54.444297Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:54.444306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342303856006862:9905]
2025-10-09T21:48:54.444311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342303856006862:9905]
2025-10-09T21:48:54.444314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342303856006862:9905]
2025-10-09T21:48:54.444357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:54.444361Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:54.444366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.444369Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:54.444369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:54.444373Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:54.444493Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:54.446282Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1354 sessionId: src_id|2a7c19b-abd83ba4-59623a22-c8d28984_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:54.448425Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:54.507009Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342304217831094:4303]
2025-10-09T21:48:54.507297Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 1353, pipe: [1:7559342304217831094:4303]
2025-10-09T21:48:54.587401Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:54.587605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.587622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.587663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217831094:4303], now have 1 active actors on pipe
2025-10-09T21:48:54.587822Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:54.587895Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342304217831094:4303]
2025-10-09T21:48:54.587907Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:54.587922Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:54.587983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1354 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2a7c19b-abd83ba4-59623a22-c8d28984_0
2025-10-09T21:48:54.588166Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1354 sessionId: src_id|2a7c19b-abd83ba4-59623a22-c8d28984_0 is DEAD
2025-10-09T21:48:54.588331Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a7c19b-abd83ba4-59623a22-c8d28984_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:54.588338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a7c19b-abd83ba4-59623a22-c8d28984_0] Write session will restart in 0.006322s
2025-10-09T21:48:54.588357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a7c19b-abd83ba4-59623a22-c8d28984_0] Write session: Do CDS request
2025-10-09T21:48:54.588361Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a7c19b-abd83ba4-59623a22-c8d28984_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:54.595517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a7c19b-abd83ba4-59623a22-c8d28984_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:54.595596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:54.595610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1355
2025-10-09T21:48:54.595745Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1355 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:54.595788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1355 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:54.595796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1355 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:54.595799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:54.596091Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1355 sessionId:  describe result for acl check
2025-10-09T21:48:54.596116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:54.596122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:54.596123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:54.596128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342304217831123:3826] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:54.596593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342304217831123:3826] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:54.598652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342304217831123:3826] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:54.601065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:54.601271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342304217831123:3826] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:54.601285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342304217831123:3826] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:54.601289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342304217831123:3826] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:54.601297Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1355 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:54.603334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:54.603557Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:54.603568Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:54.603580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:54.603655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:54.603693Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342303856007079:9913]
2025-10-09T21:48:54.604238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:54.604524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:54.604578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:54.604915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:54.604964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.604970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.604971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.604972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.604973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.604974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.604976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.604977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.604978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.604983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.604992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.604996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.605000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.605004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.605008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.605011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.605015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.605018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.605024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.605032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.605039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.605044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.605049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.605055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.605058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.605068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.605076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.605082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.605090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.605109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.605119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.605131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.605139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.605144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.605148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.605153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.605184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:54.611245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:54.612364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:54.612380Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:54.612382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:54.612387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:54.612392Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1354 [2:7559342303856007079:9913]
2025-10-09T21:48:54.612401Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:54.612414Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:54.612415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:54.612418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:54.612419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:54.612433Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:54.612436Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:54.612443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:54.612492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.612500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.612501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.612503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:54.612555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:54.612560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.612567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.612573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.612588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:54.612608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342303856007073:9913]
2025-10-09T21:48:54.612615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342303856007073:9913]
2025-10-09T21:48:54.612617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342303856007073:9913]
2025-10-09T21:48:54.612629Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:54.612644Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:54.612647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:54.612649Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:54.612649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217831274:3826], now have 1 active actors on pipe
2025-10-09T21:48:54.612653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217831096:2432], now have 1 active actors on pipe
2025-10-09T21:48:54.612660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:54.612669Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:54.612671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:54.612696Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342303856007073:9913]
2025-10-09T21:48:54.612705Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:54.612710Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:54.612701Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1354
2025-10-09T21:48:54.612737Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1354
2025-10-09T21:48:54.612742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.612750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.612770Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|26480f45-7098d068-b2ba0d35-5c54132c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:54.612776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:54.612778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:54.612780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:54.612786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:54.612793Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:54.612811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:54.612951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.612958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.612971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:54.613047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1355 partition: 0 MaxSeqNo: 25 sessionId: src_id|26480f45-7098d068-b2ba0d35-5c54132c_0
2025-10-09T21:48:54.613370Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a7c19b-abd83ba4-59623a22-c8d28984_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|26480f45-7098d068-b2ba0d35-5c54132c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:54.615751Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26480f45-7098d068-b2ba0d35-5c54132c_0] Write session: try to update token
2025-10-09T21:48:54.615765Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26480f45-7098d068-b2ba0d35-5c54132c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:54.628331Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1355 sessionId: src_id|26480f45-7098d068-b2ba0d35-5c54132c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:54.628834Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:54.628852Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:54.628861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342303856007073:9913]
2025-10-09T21:48:54.628909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:54.628919Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:54.629091Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:54.629241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:54.629385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:54.629528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:54.629725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:54.629763Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:54.630723Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:54.630888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.630905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.630937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:54.631010Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:54.636058Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:54.636074Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:54.636077Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:54.636079Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:54.636087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342303856007073:9913]
2025-10-09T21:48:54.636091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342303856007073:9913]
2025-10-09T21:48:54.636094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342303856007073:9913]
2025-10-09T21:48:54.636129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:54.636135Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:54.636142Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:54.636142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.636144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:54.636145Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:54.636266Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:54.688252Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342304217831308:4303]
2025-10-09T21:48:54.688585Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 1354, pipe: [1:7559342304217831308:4303]
2025-10-09T21:48:54.783393Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:54.790119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.790147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.790308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.790365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.790419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.790475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.790513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.790521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.790582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.790633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.790685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.790757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.791207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.791219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.791274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.791324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.791375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.791434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.791480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.791488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.791546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.791599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.791650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.791702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.791738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.791743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.791800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.791852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.791904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.791958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.791994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.792001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.792054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.792106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.792160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.792217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.792257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.792268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.792331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.792382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.792433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.792482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.792519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.792524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.792576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.792627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.792679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.792731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.792791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.792798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.792859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.792909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.792958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.793009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.793048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.793053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.793105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:54.793156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:54.793208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:54.793260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:54.793303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:54.793310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:54.793355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217831308:4303], now have 1 active actors on pipe
2025-10-09T21:48:54.793553Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:54.793568Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:54.793589Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342304217831308:4303]
2025-10-09T21:48:54.793598Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1355 reason: error at write: Disconnected sessionId: src_id|26480f45-7098d068-b2ba0d35-5c54132c_0
2025-10-09T21:48:54.793611Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:54.793785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1355 sessionId: src_id|26480f45-7098d068-b2ba0d35-5c54132c_0 is DEAD
2025-10-09T21:48:54.793793Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:54.793933Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26480f45-7098d068-b2ba0d35-5c54132c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:54.793943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26480f45-7098d068-b2ba0d35-5c54132c_0] Write session will restart in 0.006059s
2025-10-09T21:48:54.793961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26480f45-7098d068-b2ba0d35-5c54132c_0] Write session: Do CDS request
2025-10-09T21:48:54.793965Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26480f45-7098d068-b2ba0d35-5c54132c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:54.801467Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26480f45-7098d068-b2ba0d35-5c54132c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:54.801552Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:54.801566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1356
2025-10-09T21:48:54.801698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1356 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:54.801722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1356 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:54.801725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1356 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:54.801728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:54.802091Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1356 sessionId:  describe result for acl check
2025-10-09T21:48:54.802123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:54.802131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:54.802133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:54.802138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342304217831347:3813] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:54.802591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342304217831347:3813] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:54.805537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342304217831347:3813] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:54.807003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:54.808755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:54.808942Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:54.808951Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:54.808962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:54.809013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:54.809008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342304217831347:3813] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:54.809022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342304217831347:3813] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:54.809026Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342304217831347:3813] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:54.809034Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1356 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:54.809050Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342303856007272:9923]
2025-10-09T21:48:54.809588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:54.809846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:54.809881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:54.810182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:54.810228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.810236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.810237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.810238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.810239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.810240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.810241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.810242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.810244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.810249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.810259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.810263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.810267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.810271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.810275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.810278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.810282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.810286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.810293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.810300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:54.810308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.810318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:54.810327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.810336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:54.810340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.810345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:54.810353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.810357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:54.810365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.810369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:54.810374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.810378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.810386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.810391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.810398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.810402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.810428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:54.816129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:54.817136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:54.817153Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:54.817155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:54.817159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:54.817165Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1355 [2:7559342303856007272:9923]
2025-10-09T21:48:54.817176Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:54.817192Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:54.817195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:54.817198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:54.817200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:54.817215Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:54.817225Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:54.817232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:54.817284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.817292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.817294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.817297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:54.817356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:54.817364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.817377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:54.817389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:54.817406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:54.817431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217831313:2432], now have 1 active actors on pipe
2025-10-09T21:48:54.817436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217831493:3813], now have 1 active actors on pipe
2025-10-09T21:48:54.817442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342303856007267:9923]
2025-10-09T21:48:54.817444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342303856007267:9923]
2025-10-09T21:48:54.817447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342303856007267:9923]
2025-10-09T21:48:54.817439Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1355
2025-10-09T21:48:54.817459Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:54.817459Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1355
2025-10-09T21:48:54.817475Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:54.817478Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:54.817480Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:54.817482Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:54.817491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:54.817493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:54.817521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.817529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.817539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342303856007267:9923]
2025-10-09T21:48:54.817546Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:54.817567Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c4e8479-a840110c-dcd8f221-428f7a62_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:54.817569Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:54.817576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:54.817579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:54.817580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:54.817587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:54.817595Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:54.817612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:54.817934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.817947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.817964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:54.818062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1356 partition: 0 MaxSeqNo: 25 sessionId: src_id|c4e8479-a840110c-dcd8f221-428f7a62_0
2025-10-09T21:48:54.818413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26480f45-7098d068-b2ba0d35-5c54132c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c4e8479-a840110c-dcd8f221-428f7a62_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:54.820773Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4e8479-a840110c-dcd8f221-428f7a62_0] Write session: try to update token
2025-10-09T21:48:54.820792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4e8479-a840110c-dcd8f221-428f7a62_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:54.830819Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:54.830838Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:54.830841Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:54.830842Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:54.830853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342303856007267:9923]
2025-10-09T21:48:54.830857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342303856007267:9923]
2025-10-09T21:48:54.830861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342303856007267:9923]
2025-10-09T21:48:54.830905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:54.830915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:54.830917Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:54.830920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:54.830924Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:54.830928Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:54.831044Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:54.833385Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1356 sessionId: src_id|c4e8479-a840110c-dcd8f221-428f7a62_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:54.835682Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:54.893563Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342304217831508:4303]
2025-10-09T21:48:54.893821Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 1355, pipe: [1:7559342304217831508:4303]
2025-10-09T21:48:54.983038Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:54.983497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:54.983511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:54.983554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217831508:4303], now have 1 active actors on pipe
2025-10-09T21:48:54.983785Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342304217831508:4303]
2025-10-09T21:48:54.983904Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:54.983953Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:54.983981Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:54.984047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1356 reason: error at write: Disconnected sessionId: src_id|c4e8479-a840110c-dcd8f221-428f7a62_0
2025-10-09T21:48:54.984240Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1356 sessionId: src_id|c4e8479-a840110c-dcd8f221-428f7a62_0 is DEAD
2025-10-09T21:48:54.984367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4e8479-a840110c-dcd8f221-428f7a62_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:54.984377Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4e8479-a840110c-dcd8f221-428f7a62_0] Write session will restart in 0.008203s
2025-10-09T21:48:54.984399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4e8479-a840110c-dcd8f221-428f7a62_0] Write session: Do CDS request
2025-10-09T21:48:54.984406Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4e8479-a840110c-dcd8f221-428f7a62_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:54.984511Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:54.993513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4e8479-a840110c-dcd8f221-428f7a62_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:54.993640Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:54.993665Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1357
2025-10-09T21:48:54.993774Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1357 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:54.993799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1357 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:54.993807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1357 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:54.993811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:54.994276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1357 sessionId:  describe result for acl check
2025-10-09T21:48:54.994321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:54.994329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:54.994331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:54.994339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342304217831611:3802] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:54.994916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342304217831611:3802] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:54.996502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342304217831611:3802] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:54.996546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:54.998751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:54.998947Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:54.998956Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:54.998967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:54.999019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:54.999060Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342303856007454:9925]
2025-10-09T21:48:54.999097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342304217831611:3802] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:54.999111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342304217831611:3802] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:54.999114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342304217831611:3802] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:54.999122Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1357 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:54.999654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:54.999917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:54.999962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:55.000212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:55.000261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.000267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.000268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.000269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.000270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.000271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.000273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.000274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.000275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.000281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.000287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.000289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.000291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.000293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.000296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.000298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.000300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.000302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.000308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.000316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.000335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.000347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.000351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.000356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.000360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.000365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.000369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.000372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.000375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.000378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.000382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.000385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.000387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.000390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.000393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.000395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.000411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:55.006616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:55.007927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:55.007945Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:55.007948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:55.007952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:55.007960Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1356 [2:7559342303856007454:9925]
2025-10-09T21:48:55.007977Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:55.007996Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:55.008006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:55.008010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:55.008013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:55.008030Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:55.008046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:55.008054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:55.008116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.008126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.008128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.008131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:55.008208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:55.008218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.008230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.008240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.008259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:55.008287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217831689:3802], now have 1 active actors on pipe
2025-10-09T21:48:55.008297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342304217831512:2432], now have 1 active actors on pipe
2025-10-09T21:48:55.008305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342303856007449:9925]
2025-10-09T21:48:55.008319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342303856007449:9925]
2025-10-09T21:48:55.008322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342303856007449:9925]
2025-10-09T21:48:55.008338Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:55.008359Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:55.008366Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:55.008370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:55.008372Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:55.008377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:55.008381Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:55.008354Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1356
2025-10-09T21:48:55.008373Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1356
2025-10-09T21:48:55.008415Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342303856007449:9925]
2025-10-09T21:48:55.008421Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:55.008424Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:55.008437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.008447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.008496Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|50e606f-475b2a31-d364edd7-f51fee6b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:55.008510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:55.008521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:55.008523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:55.008534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:55.008545Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:55.008562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:55.008685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.008692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.008703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:55.008781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1357 partition: 0 MaxSeqNo: 25 sessionId: src_id|50e606f-475b2a31-d364edd7-f51fee6b_0
2025-10-09T21:48:55.009119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4e8479-a840110c-dcd8f221-428f7a62_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|50e606f-475b2a31-d364edd7-f51fee6b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:55.011581Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|50e606f-475b2a31-d364edd7-f51fee6b_0] Write session: try to update token
2025-10-09T21:48:55.011603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|50e606f-475b2a31-d364edd7-f51fee6b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:55.022635Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:55.022668Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:55.022671Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:55.022673Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:55.022683Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342303856007449:9925]
2025-10-09T21:48:55.022687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342303856007449:9925]
2025-10-09T21:48:55.022691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342303856007449:9925]
2025-10-09T21:48:55.022738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:55.022752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.022755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:55.022761Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:55.022792Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:55.022798Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:55.022892Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:55.024334Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1357 sessionId: src_id|50e606f-475b2a31-d364edd7-f51fee6b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:55.026657Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:55.084038Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342308512798999:4303]
2025-10-09T21:48:55.084325Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 1356, pipe: [1:7559342308512798999:4303]
2025-10-09T21:48:55.174095Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:55.174620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.174639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.174684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342308512798999:4303], now have 1 active actors on pipe
2025-10-09T21:48:55.174894Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:55.174926Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:55.174930Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:55.174967Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342308512798999:4303]
2025-10-09T21:48:55.174991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1357 reason: error at write: Disconnected sessionId: src_id|50e606f-475b2a31-d364edd7-f51fee6b_0
2025-10-09T21:48:55.175177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1357 sessionId: src_id|50e606f-475b2a31-d364edd7-f51fee6b_0 is DEAD
2025-10-09T21:48:55.175334Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50e606f-475b2a31-d364edd7-f51fee6b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:55.175356Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50e606f-475b2a31-d364edd7-f51fee6b_0] Write session will restart in 0.007806s
2025-10-09T21:48:55.175380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50e606f-475b2a31-d364edd7-f51fee6b_0] Write session: Do CDS request
2025-10-09T21:48:55.175390Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50e606f-475b2a31-d364edd7-f51fee6b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:55.175444Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:55.184530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|50e606f-475b2a31-d364edd7-f51fee6b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:55.184682Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:55.184699Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1358
2025-10-09T21:48:55.184907Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1358 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:55.184958Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1358 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:55.184971Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1358 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:55.184974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:55.185428Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1358 sessionId:  describe result for acl check
2025-10-09T21:48:55.185467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:55.185478Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:55.185480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:55.185485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342308512799061:3792] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:55.186071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342308512799061:3792] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:55.188764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342308512799061:3792] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:55.189589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:55.191656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:55.191839Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:55.191850Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:55.191862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:55.191917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:55.191959Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342308150974933:9927]
2025-10-09T21:48:55.192094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342308512799061:3792] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:55.192111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342308512799061:3792] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:55.192115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342308512799061:3792] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:55.192123Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1358 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:55.192538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:55.192904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:55.192952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:55.193267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:55.193320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.193326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.193328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.193329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.193330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.193331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.193332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.193333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.193335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.193340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.193351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.193356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.193360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.193364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.193368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.193372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.193376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.193380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.193386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.193394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.193405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.193415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.193424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.193429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.193438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.193443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.193452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.193465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.193473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.193478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.193488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.193495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.193503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.193508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.193513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.193523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.193553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:55.199990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:55.201108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:55.201126Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:55.201127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:55.201131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:55.201137Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1357 [2:7559342308150974933:9927]
2025-10-09T21:48:55.201147Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:55.201160Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:55.201162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:55.201164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:55.201166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:55.201178Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:55.201182Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:55.201188Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:55.201231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.201237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.201238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.201240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:55.201303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:55.201311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.201320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.201327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.201342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:55.201358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342308512799005:2432], now have 1 active actors on pipe
2025-10-09T21:48:55.201365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342308150974928:9927]
2025-10-09T21:48:55.201366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342308150974928:9927]
2025-10-09T21:48:55.201368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342308150974928:9927]
2025-10-09T21:48:55.201377Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:55.201388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342308512799186:3792], now have 1 active actors on pipe
2025-10-09T21:48:55.201393Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:55.201396Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:55.201398Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:55.201399Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:55.201407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:55.201410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:55.201430Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342308150974928:9927]
2025-10-09T21:48:55.201439Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:55.201446Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1357
2025-10-09T21:48:55.201474Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:55.201484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.201460Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1357
2025-10-09T21:48:55.201487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.201518Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4c4b9939-31a7e848-8f9fb1d4-777fd003_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:55.201530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:55.201535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:55.201537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:55.201547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:55.201560Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:55.201572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:55.201715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.201722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.201736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:55.201846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1358 partition: 0 MaxSeqNo: 25 sessionId: src_id|4c4b9939-31a7e848-8f9fb1d4-777fd003_0
2025-10-09T21:48:55.202221Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50e606f-475b2a31-d364edd7-f51fee6b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4c4b9939-31a7e848-8f9fb1d4-777fd003_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:55.204716Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4c4b9939-31a7e848-8f9fb1d4-777fd003_0] Write session: try to update token
2025-10-09T21:48:55.204734Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4c4b9939-31a7e848-8f9fb1d4-777fd003_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:55.212531Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:55.212547Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:55.212557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342308150974928:9927]
2025-10-09T21:48:55.212600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:55.212727Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:55.212774Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:55.212920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:55.213053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:55.213195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:55.213375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:55.213415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:55.217439Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1358 sessionId: src_id|4c4b9939-31a7e848-8f9fb1d4-777fd003_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:55.219772Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:55.220165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.220185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.220225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:55.220415Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:55.225302Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:55.225313Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:55.225315Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:55.225316Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:55.225325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342308150974928:9927]
2025-10-09T21:48:55.225327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342308150974928:9927]
2025-10-09T21:48:55.225330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342308150974928:9927]
2025-10-09T21:48:55.225360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:55.225365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.225371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:55.225376Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:55.225382Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:55.225384Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:55.225500Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:55.275009Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342308512799202:4303]
2025-10-09T21:48:55.275273Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 1357, pipe: [1:7559342308512799202:4303]
2025-10-09T21:48:55.375437Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:55.382213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.382236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.382378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.382432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.382488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.382543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.382588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.382599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.382670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.382725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.382785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.382844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.382879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.382887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.382946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.383001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.383057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.383113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.383157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.383164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.383223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.383285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.383346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.383402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.383441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.383446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.383504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.383556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.383613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.383670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.383709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.383715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.383772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.383840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.383907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.383969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.384010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.384015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.384070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.384126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.384192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.384247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.384285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.384290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.384345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.384394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.384459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.384515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.384554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.384561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.384618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.384674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.384731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.384786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.384824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.384830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.384882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.384935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.384989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.385044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.385084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.385089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.385138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342308512799202:4303], now have 1 active actors on pipe
2025-10-09T21:48:55.385470Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342308512799202:4303]
2025-10-09T21:48:55.385482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:55.385486Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:55.385515Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:55.385516Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1358 reason: error at write: Disconnected sessionId: src_id|4c4b9939-31a7e848-8f9fb1d4-777fd003_0
2025-10-09T21:48:55.385782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1358 sessionId: src_id|4c4b9939-31a7e848-8f9fb1d4-777fd003_0 is DEAD
2025-10-09T21:48:55.385901Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c4b9939-31a7e848-8f9fb1d4-777fd003_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:55.385912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c4b9939-31a7e848-8f9fb1d4-777fd003_0] Write session will restart in 0.005845s
2025-10-09T21:48:55.385949Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c4b9939-31a7e848-8f9fb1d4-777fd003_0] Write session: Do CDS request
2025-10-09T21:48:55.385955Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c4b9939-31a7e848-8f9fb1d4-777fd003_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:55.385990Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:55.392568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4c4b9939-31a7e848-8f9fb1d4-777fd003_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:55.392623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:55.392651Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1359
2025-10-09T21:48:55.392786Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1359 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:55.392823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1359 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:55.392834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1359 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:55.392839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:55.393214Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1359 sessionId:  describe result for acl check
2025-10-09T21:48:55.393246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:55.393251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:55.393253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:55.393257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342308512799229:3787] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:55.393766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342308512799229:3787] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:55.397280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342308512799229:3787] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:55.399394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:55.401452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:55.401537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342308512799229:3787] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:55.401554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342308512799229:3787] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:55.401557Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342308512799229:3787] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:55.401571Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1359 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:55.401636Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:55.401660Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:55.401673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:55.401736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:55.401779Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342308150975117:9929]
2025-10-09T21:48:55.402359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:55.402645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:55.402690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:55.403019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:55.403073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.403079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.403080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.403081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.403082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.403084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.403085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.403086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.403087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.403092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.403103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.403108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.403112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.403116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.403120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.403124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.403128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.403131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.403137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.403147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.403157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.403163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.403169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.403174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.403179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.403189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.403193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.403199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.403203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.403208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.403212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.403217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.403221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.403225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.403227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.403230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.403246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:55.409372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:55.410452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:55.410471Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:55.410473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:55.410477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:55.410484Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1358 [2:7559342308150975117:9929]
2025-10-09T21:48:55.410496Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:55.410517Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:55.410524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:55.410528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:55.410530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:55.410547Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:55.410558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:55.410566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:55.410625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.410632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.410634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.410637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:55.410710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:55.410719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.410730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.410747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.410766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:55.410794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342308512799396:3787], now have 1 active actors on pipe
2025-10-09T21:48:55.410798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342308512799205:2432], now have 1 active actors on pipe
2025-10-09T21:48:55.410805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342308150975112:9929]
2025-10-09T21:48:55.410807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342308150975112:9929]
2025-10-09T21:48:55.410809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342308150975112:9929]
2025-10-09T21:48:55.410822Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:55.410840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:55.410840Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:55.410843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:55.410845Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:55.410831Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1358
2025-10-09T21:48:55.410855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:55.410845Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1358
2025-10-09T21:48:55.410858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:55.410884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342308150975112:9929]
2025-10-09T21:48:55.410887Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:55.410889Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:55.410903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.410908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.410958Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dccf6da-9340ca0d-d7c7d932-3a50068c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:55.410969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:55.410973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:55.410976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:55.410987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:55.411002Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:55.411023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:55.411196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.411207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.411223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:55.411333Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1359 partition: 0 MaxSeqNo: 25 sessionId: src_id|dccf6da-9340ca0d-d7c7d932-3a50068c_0
2025-10-09T21:48:55.411676Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c4b9939-31a7e848-8f9fb1d4-777fd003_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dccf6da-9340ca0d-d7c7d932-3a50068c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:55.414094Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dccf6da-9340ca0d-d7c7d932-3a50068c_0] Write session: try to update token
2025-10-09T21:48:55.414111Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dccf6da-9340ca0d-d7c7d932-3a50068c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:55.424690Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:55.424707Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:55.424710Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:55.424712Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:55.424721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342308150975112:9929]
2025-10-09T21:48:55.424725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342308150975112:9929]
2025-10-09T21:48:55.424730Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342308150975112:9929]
2025-10-09T21:48:55.424771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:55.424776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.424779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:55.424783Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:55.424789Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:55.424794Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:55.424912Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:55.426398Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1359 sessionId: src_id|dccf6da-9340ca0d-d7c7d932-3a50068c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:55.428657Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:55.485420Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342308512799407:4303]
2025-10-09T21:48:55.485643Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 1358, pipe: [1:7559342308512799407:4303]
2025-10-09T21:48:55.565830Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:55.566154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.566166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.566200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342308512799407:4303], now have 1 active actors on pipe
2025-10-09T21:48:55.566350Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:55.566369Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:55.566380Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342308512799407:4303]
2025-10-09T21:48:55.566405Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:55.566475Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1359 reason: status is not ok: tablet will be restarted right now sessionId: src_id|dccf6da-9340ca0d-d7c7d932-3a50068c_0
2025-10-09T21:48:55.566629Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1359 sessionId: src_id|dccf6da-9340ca0d-d7c7d932-3a50068c_0 is DEAD
2025-10-09T21:48:55.566797Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dccf6da-9340ca0d-d7c7d932-3a50068c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:55.566807Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dccf6da-9340ca0d-d7c7d932-3a50068c_0] Write session will restart in 0.009616s
2025-10-09T21:48:55.566828Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dccf6da-9340ca0d-d7c7d932-3a50068c_0] Write session: Do CDS request
2025-10-09T21:48:55.566833Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dccf6da-9340ca0d-d7c7d932-3a50068c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:55.577509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dccf6da-9340ca0d-d7c7d932-3a50068c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:55.577576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:55.577591Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1360
2025-10-09T21:48:55.577743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1360 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:55.577783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1360 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:55.577793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1360 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:55.577798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:55.578244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1360 sessionId:  describe result for acl check
2025-10-09T21:48:55.578279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:55.578285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:55.578287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:55.578293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342308512799556:3783] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:55.578829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342308512799556:3783] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:55.579482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:55.580951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342308512799556:3783] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:55.581375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:55.581549Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:55.581558Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:55.581569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:55.581621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:55.581667Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342308150975302:9931]
2025-10-09T21:48:55.582227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:55.582556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:55.582603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:55.582869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:55.582902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.582907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.582907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.582908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.582908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.582909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.582909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.582910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.582911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.582914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.582921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.582923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.582925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.582927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.582929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.582931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.582933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.582934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.582939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.582944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.582948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.582951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.582954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.582957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.582959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.582962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.582964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.582967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.582970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.582973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.582975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.582978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.582981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.582983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.582986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.582989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.583005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:55.583233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342308512799556:3783] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:55.583249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342308512799556:3783] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:55.583252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342308512799556:3783] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:55.583259Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1360 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:55.588887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:55.589901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:55.589914Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:55.589916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:55.589919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:55.589923Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1359 [2:7559342308150975302:9931]
2025-10-09T21:48:55.589934Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:55.589948Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:55.589953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:55.589955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:55.589957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:55.589969Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:55.589973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:55.589978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:55.590023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.590043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.590045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.590048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:55.590098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:55.590103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.590110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.590118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.590127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:55.590143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342308150975297:9931]
2025-10-09T21:48:55.590148Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342308150975297:9931]
2025-10-09T21:48:55.590150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342308150975297:9931]
2025-10-09T21:48:55.590156Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:55.590166Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:55.590167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:55.590170Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:55.590173Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:55.590175Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:55.590177Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:55.590186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342308512799410:2432], now have 1 active actors on pipe
2025-10-09T21:48:55.590192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342308512799597:3783], now have 1 active actors on pipe
2025-10-09T21:48:55.590208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342308150975297:9931]
2025-10-09T21:48:55.590213Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:55.590237Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:55.590230Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1359
2025-10-09T21:48:55.590252Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1359
2025-10-09T21:48:55.590329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.590341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.590372Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3d33a340-df3fc7d6-bc525a0a-a84664c2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:55.590386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:55.590391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:55.590395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:55.590408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:55.590424Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:55.590451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:55.590632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.590640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.590654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:55.590756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1360 partition: 0 MaxSeqNo: 25 sessionId: src_id|3d33a340-df3fc7d6-bc525a0a-a84664c2_0
2025-10-09T21:48:55.591093Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dccf6da-9340ca0d-d7c7d932-3a50068c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3d33a340-df3fc7d6-bc525a0a-a84664c2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:55.593615Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d33a340-df3fc7d6-bc525a0a-a84664c2_0] Write session: try to update token
2025-10-09T21:48:55.593629Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d33a340-df3fc7d6-bc525a0a-a84664c2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:55.601043Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:55.601058Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:55.601068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342308150975297:9931]
2025-10-09T21:48:55.601113Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:55.601116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:55.601291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:55.601432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:55.601568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:55.601752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:55.601941Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:55.601975Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:55.606205Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1360 sessionId: src_id|3d33a340-df3fc7d6-bc525a0a-a84664c2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:55.608368Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:55.608551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.608568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.608603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:55.608692Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:55.613453Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:55.613465Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:55.613467Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:55.613468Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:55.613475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342308150975297:9931]
2025-10-09T21:48:55.613478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342308150975297:9931]
2025-10-09T21:48:55.613480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342308150975297:9931]
2025-10-09T21:48:55.613511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:55.613516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.613519Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:55.613521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:55.613525Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:55.613529Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:55.613639Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:55.666671Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342308512799617:4303]
2025-10-09T21:48:55.666933Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 1359, pipe: [1:7559342308512799617:4303]
2025-10-09T21:48:55.755279Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:55.762494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.762516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.762661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.762721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.762782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.762850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.762896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.762907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.762975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.763033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.763096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.763162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.763205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.763214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.763276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.763340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.763404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.763464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.763507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.763515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.763580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.763643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.763703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.763761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.763809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.763817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.763880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.763941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.764004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.764069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.764111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.764119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.764177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.764239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.764300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.764362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.764403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.764413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.764472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.764529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.764594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.764665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.764710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.764719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.764777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.764841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.764898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.764956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.765001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.765008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.765070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.765127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.765184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.765241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.765282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.765292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.765353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:55.765406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:55.765466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:55.765521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:55.765568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:55.765574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:55.765613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342308512799617:4303], now have 1 active actors on pipe
2025-10-09T21:48:55.765787Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342308512799617:4303]
2025-10-09T21:48:55.765792Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:55.765827Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:55.765830Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:55.765861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1360 reason: error at write: Disconnected sessionId: src_id|3d33a340-df3fc7d6-bc525a0a-a84664c2_0
2025-10-09T21:48:55.766064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1360 sessionId: src_id|3d33a340-df3fc7d6-bc525a0a-a84664c2_0 is DEAD
2025-10-09T21:48:55.766208Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d33a340-df3fc7d6-bc525a0a-a84664c2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:55.766221Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d33a340-df3fc7d6-bc525a0a-a84664c2_0] Write session will restart in 0.008897s
2025-10-09T21:48:55.766246Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d33a340-df3fc7d6-bc525a0a-a84664c2_0] Write session: Do CDS request
2025-10-09T21:48:55.766252Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d33a340-df3fc7d6-bc525a0a-a84664c2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:55.766500Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:55.776510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d33a340-df3fc7d6-bc525a0a-a84664c2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:55.776590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:55.776608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1361
2025-10-09T21:48:55.776721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1361 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:55.776754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1361 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:55.776765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1361 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:55.776769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:55.777210Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1361 sessionId:  describe result for acl check
2025-10-09T21:48:55.777243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:55.777246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:55.777247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:55.777254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342308512799714:3764] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:55.778099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342308512799714:3764] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:55.779930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:55.780121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342308512799714:3764] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:55.781830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:55.782004Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:55.782013Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:55.782024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:55.782086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:55.782116Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342308150975531:9947]
2025-10-09T21:48:55.782498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342308512799714:3764] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:55.782511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342308512799714:3764] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:55.782513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342308512799714:3764] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:55.782520Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1361 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:55.782570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:55.782800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:55.782831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:55.783048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:55.783084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.783088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.783089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.783089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.783090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.783091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.783091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.783092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.783093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.783096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.783102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.783105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.783107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.783109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.783111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.783113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.783115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.783117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.783121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.783126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.783130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.783137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.783140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.783143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.783145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.783148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.783151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.783153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.783156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.783159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.783162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.783165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.783168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.783171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.783174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.783177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.783205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:55.789461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:55.790662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:55.790677Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:55.790679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:55.790683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:55.790692Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1360 [2:7559342308150975531:9947]
2025-10-09T21:48:55.790705Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:55.790722Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:55.790730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:55.790733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:55.790735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:55.790753Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:55.790764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:55.790772Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:55.790832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.790838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.790840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.790843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:55.790909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:55.790916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.790928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.790944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.790965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:55.790990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342308512799813:3764], now have 1 active actors on pipe
2025-10-09T21:48:55.791000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342308512799629:2432], now have 1 active actors on pipe
2025-10-09T21:48:55.791008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342308150975526:9947]
2025-10-09T21:48:55.791010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342308150975526:9947]
2025-10-09T21:48:55.791012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342308150975526:9947]
2025-10-09T21:48:55.791024Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:55.791039Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:55.791041Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:55.791044Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:55.791045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:55.791054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:55.791025Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1360
2025-10-09T21:48:55.791057Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:55.791032Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1360
2025-10-09T21:48:55.791101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.791110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.791123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342308150975526:9947]
2025-10-09T21:48:55.791130Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:55.791145Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:55.791169Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c549d1c-f68e4605-6b5e87e4-335ac030_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:55.791180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:55.791184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:55.791187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:55.791197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:55.791210Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:55.791237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:55.791364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.791371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.791383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:55.791451Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1361 partition: 0 MaxSeqNo: 25 sessionId: src_id|c549d1c-f68e4605-6b5e87e4-335ac030_0
2025-10-09T21:48:55.791814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d33a340-df3fc7d6-bc525a0a-a84664c2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c549d1c-f68e4605-6b5e87e4-335ac030_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:55.794227Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c549d1c-f68e4605-6b5e87e4-335ac030_0] Write session: try to update token
2025-10-09T21:48:55.794244Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c549d1c-f68e4605-6b5e87e4-335ac030_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:55.804602Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:55.804615Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:55.804617Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:55.804618Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:55.804626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342308150975526:9947]
2025-10-09T21:48:55.804629Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342308150975526:9947]
2025-10-09T21:48:55.804632Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342308150975526:9947]
2025-10-09T21:48:55.804666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:55.804672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.804674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:55.804704Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:55.804713Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:55.804718Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:55.804793Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:55.806544Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1361 sessionId: src_id|c549d1c-f68e4605-6b5e87e4-335ac030_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:55.808721Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:55.865848Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342308512799827:4303]
2025-10-09T21:48:55.866194Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 1360, pipe: [1:7559342308512799827:4303]
2025-10-09T21:48:55.946667Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:55.947060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.947070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.947106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342308512799827:4303], now have 1 active actors on pipe
2025-10-09T21:48:55.947286Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:55.947349Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:55.947352Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342308512799827:4303]
2025-10-09T21:48:55.947365Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:55.947371Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:55.947436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1361 reason: error at write: Disconnected sessionId: src_id|c549d1c-f68e4605-6b5e87e4-335ac030_0
2025-10-09T21:48:55.947622Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1361 sessionId: src_id|c549d1c-f68e4605-6b5e87e4-335ac030_0 is DEAD
2025-10-09T21:48:55.947758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c549d1c-f68e4605-6b5e87e4-335ac030_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:55.947767Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c549d1c-f68e4605-6b5e87e4-335ac030_0] Write session will restart in 0.008053s
2025-10-09T21:48:55.947806Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c549d1c-f68e4605-6b5e87e4-335ac030_0] Write session: Do CDS request
2025-10-09T21:48:55.947813Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c549d1c-f68e4605-6b5e87e4-335ac030_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:55.956576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c549d1c-f68e4605-6b5e87e4-335ac030_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:55.956668Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:55.956688Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1362
2025-10-09T21:48:55.956810Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1362 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:55.956843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1362 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:55.956853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1362 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:55.956856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:55.957427Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1362 sessionId:  describe result for acl check
2025-10-09T21:48:55.957469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:55.957476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:55.957478Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:55.957484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342308512799883:3760] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:55.958099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342308512799883:3760] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:55.960242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342308512799883:3760] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:55.962099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:55.963909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:55.964065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342308512799883:3760] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:55.964081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342308512799883:3760] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:55.964084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342308512799883:3760] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:55.964092Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1362 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:55.964088Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:55.964101Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:55.964112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:55.964163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:55.964202Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342308150975718:9949]
2025-10-09T21:48:55.964791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:55.965050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:55.965090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:55.965399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:55.965452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.965457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.965458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.965459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.965460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.965461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.965462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.965463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.965464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.965469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.965478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.965482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.965486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.965489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.965493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.965496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.965500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.965503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.965510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.965518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:55.965524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.965530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:55.965534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.965540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:55.965544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.965549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:55.965554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.965559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:55.965563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.965568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:55.965574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.965579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.965583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.965589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.965594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.965599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.965629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:55.971961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:55.972990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:55.973002Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:55.973003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:55.973006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:55.973011Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1361 [2:7559342308150975718:9949]
2025-10-09T21:48:55.973030Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:55.973045Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:55.973050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:55.973052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:55.973053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:55.973066Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:55.973070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:55.973076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:55.973121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.973127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.973128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.973130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:55.973191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:55.973200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.973206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:55.973213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:55.973223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:55.973238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342308150975713:9949]
2025-10-09T21:48:55.973240Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342308150975713:9949]
2025-10-09T21:48:55.973241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342308150975713:9949]
2025-10-09T21:48:55.973248Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:55.973259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:55.973260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:55.973267Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:55.973268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:55.973269Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:55.973270Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:55.973279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342308512799829:2432], now have 1 active actors on pipe
2025-10-09T21:48:55.973282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342308512800014:3760], now have 1 active actors on pipe
2025-10-09T21:48:55.973309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342308150975713:9949]
2025-10-09T21:48:55.973322Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:55.973345Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:55.973344Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1361
2025-10-09T21:48:55.973353Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1361
2025-10-09T21:48:55.973424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.973434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.973466Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|24646807-5b3ef10b-29af0c96-2ad468c7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:55.973476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:55.973479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:55.973481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:55.973489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:55.973499Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:55.973516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:55.973784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:55.973793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:55.973809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:55.973901Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1362 partition: 0 MaxSeqNo: 25 sessionId: src_id|24646807-5b3ef10b-29af0c96-2ad468c7_0
2025-10-09T21:48:55.974258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c549d1c-f68e4605-6b5e87e4-335ac030_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|24646807-5b3ef10b-29af0c96-2ad468c7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:55.976541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|24646807-5b3ef10b-29af0c96-2ad468c7_0] Write session: try to update token
2025-10-09T21:48:55.976559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|24646807-5b3ef10b-29af0c96-2ad468c7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:55.987723Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:55.987739Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:55.987742Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:55.987744Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:55.987754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342308150975713:9949]
2025-10-09T21:48:55.987759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342308150975713:9949]
2025-10-09T21:48:55.987764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342308150975713:9949]
2025-10-09T21:48:55.987807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:55.987812Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:55.987815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:55.987817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:55.987818Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:55.987822Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:55.987946Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:55.989476Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1362 sessionId: src_id|24646807-5b3ef10b-29af0c96-2ad468c7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:55.991881Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:56.047610Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342312807767322:4303]
2025-10-09T21:48:56.047854Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 1361, pipe: [1:7559342312807767322:4303]
2025-10-09T21:48:56.139502Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:56.140002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.140015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.140056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342312807767322:4303], now have 1 active actors on pipe
2025-10-09T21:48:56.140215Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:56.140274Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342312807767322:4303]
2025-10-09T21:48:56.140273Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:56.140281Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:56.140319Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:56.140381Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1362 reason: error at write: Disconnected sessionId: src_id|24646807-5b3ef10b-29af0c96-2ad468c7_0
2025-10-09T21:48:56.140586Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1362 sessionId: src_id|24646807-5b3ef10b-29af0c96-2ad468c7_0 is DEAD
2025-10-09T21:48:56.140746Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24646807-5b3ef10b-29af0c96-2ad468c7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:56.140761Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24646807-5b3ef10b-29af0c96-2ad468c7_0] Write session will restart in 0.009779s
2025-10-09T21:48:56.140782Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24646807-5b3ef10b-29af0c96-2ad468c7_0] Write session: Do CDS request
2025-10-09T21:48:56.140787Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24646807-5b3ef10b-29af0c96-2ad468c7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:56.151609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|24646807-5b3ef10b-29af0c96-2ad468c7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:56.151675Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:56.151691Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1363
2025-10-09T21:48:56.151821Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1363 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:56.151857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1363 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:56.151868Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1363 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:56.151873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:56.152291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1363 sessionId:  describe result for acl check
2025-10-09T21:48:56.152333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:56.152336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:56.152338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:56.152347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342312807767462:3753] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:56.152858Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342312807767462:3753] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:56.154498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:56.155110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342312807767462:3753] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:56.156950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:56.157196Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:56.157206Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:56.157220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:56.157282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:56.157322Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342312445943203:9951]
2025-10-09T21:48:56.157879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342312807767462:3753] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:56.157895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342312807767462:3753] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:56.157899Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342312807767462:3753] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:56.157909Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1363 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:56.157948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:56.158225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:56.158270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:56.158622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:56.158689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.158695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.158697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.158698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.158699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.158700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.158702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.158703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.158704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.158710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.158721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.158725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.158728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.158732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.158736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.158740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.158743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.158747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.158755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.158763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.158770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.158776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.158780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.158784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.158788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.158792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.158798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.158803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.158807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.158812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.158816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.158827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.158834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.158839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.158843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.158848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.158877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:56.165404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:56.166428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:56.166444Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:56.166447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:56.166453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:56.166460Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1362 [2:7559342312445943203:9951]
2025-10-09T21:48:56.166474Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:56.166489Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:56.166491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:56.166493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:56.166495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:56.166510Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:56.166514Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:56.166520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:56.166568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.166575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.166576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.166578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:56.166631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:56.166638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.166646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.166651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.166667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:56.166682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342312445943198:9951]
2025-10-09T21:48:56.166684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342312445943198:9951]
2025-10-09T21:48:56.166685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342312445943198:9951]
2025-10-09T21:48:56.166693Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:56.166704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:56.166707Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:56.166711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:56.166713Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:56.166713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:56.166715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:56.166724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342312807767515:3753], now have 1 active actors on pipe
2025-10-09T21:48:56.166726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342312807767326:2432], now have 1 active actors on pipe
2025-10-09T21:48:56.166757Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342312445943198:9951]
2025-10-09T21:48:56.166769Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:56.166826Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:56.166810Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1362
2025-10-09T21:48:56.166811Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1362
2025-10-09T21:48:56.166877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.166881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.166907Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|37adf0d3-1860890f-5b206188-2cf65a9e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:56.166923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:56.166928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:56.166930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:56.166941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:56.166956Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:56.166978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:56.167096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.167103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.167115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:56.167175Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1363 partition: 0 MaxSeqNo: 25 sessionId: src_id|37adf0d3-1860890f-5b206188-2cf65a9e_0
2025-10-09T21:48:56.167566Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24646807-5b3ef10b-29af0c96-2ad468c7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|37adf0d3-1860890f-5b206188-2cf65a9e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:56.170112Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|37adf0d3-1860890f-5b206188-2cf65a9e_0] Write session: try to update token
2025-10-09T21:48:56.170130Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|37adf0d3-1860890f-5b206188-2cf65a9e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:56.178234Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:56.178249Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:56.178262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342312445943198:9951]
2025-10-09T21:48:56.178311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:56.178310Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:56.179030Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:56.179178Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:56.179314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:56.179454Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:56.179752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:56.179804Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:56.183404Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1363 sessionId: src_id|37adf0d3-1860890f-5b206188-2cf65a9e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:56.186019Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:56.186500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.186518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.186559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:56.186763Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:56.191153Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:56.191169Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:56.191172Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:56.191174Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:56.191185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342312445943198:9951]
2025-10-09T21:48:56.191189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342312445943198:9951]
2025-10-09T21:48:56.191194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342312445943198:9951]
2025-10-09T21:48:56.191238Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:56.191242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:56.191243Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:56.191248Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:56.191250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.191253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:56.191388Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:56.240592Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342312807767532:4303]
2025-10-09T21:48:56.240841Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 1362, pipe: [1:7559342312807767532:4303]
2025-10-09T21:48:56.340193Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:56.346834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.346857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.346994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.347048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.347107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.347162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.347202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.347209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.347266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.347318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.347380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.347434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.347473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.347480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.347535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.347586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.347641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.347693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.347730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.347736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.347792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.347844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.347902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.347961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.348008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.348015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.348077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.348134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.348191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.348242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.348278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.348283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.348336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.348390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.348447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.348498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.348535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.348540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.348591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.348639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.348696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.348747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.348785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.348791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.348848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.348899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.348955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.349005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.349040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.349045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.349095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.349146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.349201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.349250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.349286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.349291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.349339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.349388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.349443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.349493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.349527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.349531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.349569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342312807767532:4303], now have 1 active actors on pipe
2025-10-09T21:48:56.349797Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:56.349822Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:56.349846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1363 reason: error at write: Disconnected sessionId: src_id|37adf0d3-1860890f-5b206188-2cf65a9e_0
2025-10-09T21:48:56.349855Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:56.349903Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342312807767532:4303]
2025-10-09T21:48:56.350017Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1363 sessionId: src_id|37adf0d3-1860890f-5b206188-2cf65a9e_0 is DEAD
2025-10-09T21:48:56.350185Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37adf0d3-1860890f-5b206188-2cf65a9e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:56.350197Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37adf0d3-1860890f-5b206188-2cf65a9e_0] Write session will restart in 0.008700s
2025-10-09T21:48:56.350222Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37adf0d3-1860890f-5b206188-2cf65a9e_0] Write session: Do CDS request
2025-10-09T21:48:56.350228Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37adf0d3-1860890f-5b206188-2cf65a9e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:56.350238Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:56.360535Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|37adf0d3-1860890f-5b206188-2cf65a9e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:56.360607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:56.360624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1364
2025-10-09T21:48:56.360792Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1364 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:56.360825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1364 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:56.360834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1364 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:56.360840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:56.361278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1364 sessionId:  describe result for acl check
2025-10-09T21:48:56.361317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:56.361319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:56.361320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:56.361326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342312807767674:3748] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:56.361911Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342312807767674:3748] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:56.363445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:56.364194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342312807767674:3748] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:56.365380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:56.365532Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:56.365541Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:56.365549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:56.365585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:56.365685Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342312445943392:9953]
2025-10-09T21:48:56.366135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:56.366368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:56.366399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:56.366604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:56.366631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.366635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.366636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.366637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.366638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.366638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.366639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.366640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.366640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.366644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.366650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.366652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.366655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.366657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.366659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.366661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342312807767674:3748] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:56.366661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.366675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342312807767674:3748] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:56.366678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342312807767674:3748] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:56.366663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.366685Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1364 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:56.366665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.366669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.366674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.366678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.366682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.366686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.366691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.366696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.366700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.366705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.366710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.366715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.366719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.366722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.366725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.366727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.366730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.366732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.366735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.366751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:56.372803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:56.373823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:56.373839Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:56.373861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:56.373865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:56.373871Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1363 [2:7559342312445943392:9953]
2025-10-09T21:48:56.373884Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:56.373900Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:56.373906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:56.373908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:56.373909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:56.373921Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:56.373925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:56.373931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:56.373970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.373975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.373976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.373978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:56.374051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:56.374058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.374071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.374084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.374094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:56.374115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342312807767535:2432], now have 1 active actors on pipe
2025-10-09T21:48:56.374124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342312807767723:3748], now have 1 active actors on pipe
2025-10-09T21:48:56.374131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342312445943387:9953]
2025-10-09T21:48:56.374134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342312445943387:9953]
2025-10-09T21:48:56.374136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342312445943387:9953]
2025-10-09T21:48:56.374151Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:56.374169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:56.374185Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:56.374190Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:56.374191Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:56.374194Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:56.374196Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:56.374210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342312445943387:9953]
2025-10-09T21:48:56.374219Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:56.374212Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1363
2025-10-09T21:48:56.374220Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1363
2025-10-09T21:48:56.374245Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:56.374328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.374341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.374372Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5eace1b-b97e7d93-96a05235-3ad1aaaa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:56.374384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:56.374388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:56.374391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:56.374402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:56.374418Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:56.374433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:56.374619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.374628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.374644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:56.374793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1364 partition: 0 MaxSeqNo: 25 sessionId: src_id|5eace1b-b97e7d93-96a05235-3ad1aaaa_0
2025-10-09T21:48:56.375137Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37adf0d3-1860890f-5b206188-2cf65a9e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5eace1b-b97e7d93-96a05235-3ad1aaaa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:56.377531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5eace1b-b97e7d93-96a05235-3ad1aaaa_0] Write session: try to update token
2025-10-09T21:48:56.377548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5eace1b-b97e7d93-96a05235-3ad1aaaa_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:56.388505Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:56.388521Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:56.388522Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:56.388523Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:56.388531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342312445943387:9953]
2025-10-09T21:48:56.388534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342312445943387:9953]
2025-10-09T21:48:56.388537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342312445943387:9953]
2025-10-09T21:48:56.388572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:56.388578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.388579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:56.388585Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:56.388591Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:56.388593Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:56.388699Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:56.391128Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1364 sessionId: src_id|5eace1b-b97e7d93-96a05235-3ad1aaaa_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:56.393508Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:56.450127Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342312807767733:4303]
2025-10-09T21:48:56.450396Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 1363, pipe: [1:7559342312807767733:4303]
2025-10-09T21:48:56.538430Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:56.538857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.538870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.538910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342312807767733:4303], now have 1 active actors on pipe
2025-10-09T21:48:56.539085Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:56.539145Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342312807767733:4303]
2025-10-09T21:48:56.539176Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:56.539184Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:56.539259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1364 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5eace1b-b97e7d93-96a05235-3ad1aaaa_0
2025-10-09T21:48:56.539448Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1364 sessionId: src_id|5eace1b-b97e7d93-96a05235-3ad1aaaa_0 is DEAD
2025-10-09T21:48:56.539582Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5eace1b-b97e7d93-96a05235-3ad1aaaa_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:56.539593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5eace1b-b97e7d93-96a05235-3ad1aaaa_0] Write session will restart in 0.006165s
2025-10-09T21:48:56.539621Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5eace1b-b97e7d93-96a05235-3ad1aaaa_0] Write session: Do CDS request
2025-10-09T21:48:56.539626Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5eace1b-b97e7d93-96a05235-3ad1aaaa_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:56.546512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5eace1b-b97e7d93-96a05235-3ad1aaaa_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:56.546587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:56.546604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1365
2025-10-09T21:48:56.546714Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1365 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:56.546747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1365 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:56.546759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1365 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:56.546763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:56.547175Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1365 sessionId:  describe result for acl check
2025-10-09T21:48:56.547211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:56.547219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:56.547221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:56.547227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342312807767756:3744] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:56.547766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342312807767756:3744] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:56.549932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342312807767756:3744] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:56.552515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342312807767756:3744] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:56.552531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342312807767756:3744] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:56.552535Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342312807767756:3744] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:56.552542Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1365 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:56.552955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:56.555081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:56.555260Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:56.555273Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:56.555285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:56.555342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:56.555382Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342312445943583:9955]
2025-10-09T21:48:56.555945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:56.556210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:56.556258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:56.556564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:56.556605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.556606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.556607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.556608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.556609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.556610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.556611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.556612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.556613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.556617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.556628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.556632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.556636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.556640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.556644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.556648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.556652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.556655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.556661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.556669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.556675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.556680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.556685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.556690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.556695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.556700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.556704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.556709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.556714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.556719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.556725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.556735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.556739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.556744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.556747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.556752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.556782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:56.562864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:56.563797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:56.563809Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:56.563810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:56.563813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:56.563818Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1364 [2:7559342312445943583:9955]
2025-10-09T21:48:56.563827Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:56.563840Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:56.563842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:56.563844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:56.563845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:56.563857Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:56.563860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:56.563866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:56.563905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.563911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.563912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.563914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:56.563966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:56.563974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.563985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.563995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.564011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:56.564026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342312445943578:9955]
2025-10-09T21:48:56.564031Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342312445943578:9955]
2025-10-09T21:48:56.564032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342312445943578:9955]
2025-10-09T21:48:56.564040Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:56.564049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:56.564052Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:56.564054Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:56.564055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342312807767736:2432], now have 1 active actors on pipe
2025-10-09T21:48:56.564058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342312807767923:3744], now have 1 active actors on pipe
2025-10-09T21:48:56.564067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:56.564081Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:56.564088Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:56.564115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342312445943578:9955]
2025-10-09T21:48:56.564128Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:56.564151Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:56.564140Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1364
2025-10-09T21:48:56.564143Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1364
2025-10-09T21:48:56.564272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.564287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.564330Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|49ebfe78-d4d7a64d-fdd055d9-45d2d8c3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:56.564343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:56.564350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:56.564353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:56.564363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:56.564379Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:56.564399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:56.564604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.564614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.564633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:56.564724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1365 partition: 0 MaxSeqNo: 25 sessionId: src_id|49ebfe78-d4d7a64d-fdd055d9-45d2d8c3_0
2025-10-09T21:48:56.565053Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5eace1b-b97e7d93-96a05235-3ad1aaaa_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|49ebfe78-d4d7a64d-fdd055d9-45d2d8c3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:56.567332Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|49ebfe78-d4d7a64d-fdd055d9-45d2d8c3_0] Write session: try to update token
2025-10-09T21:48:56.567346Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|49ebfe78-d4d7a64d-fdd055d9-45d2d8c3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:56.575085Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:56.575096Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:56.575103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342312445943578:9955]
2025-10-09T21:48:56.575138Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:56.575145Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:56.575275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:56.575389Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:56.575506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:56.575629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:56.575799Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:56.575842Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:56.579885Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1365 sessionId: src_id|49ebfe78-d4d7a64d-fdd055d9-45d2d8c3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:56.582077Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:56.582274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.582288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.582320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:56.582398Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:56.587646Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:56.587660Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:56.587662Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:56.587664Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:56.587686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342312445943578:9955]
2025-10-09T21:48:56.587690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342312445943578:9955]
2025-10-09T21:48:56.587694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342312445943578:9955]
2025-10-09T21:48:56.587734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:56.587742Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:56.587747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.587748Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:56.587749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:56.587752Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:56.587893Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:56.639482Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342312807767935:4303]
2025-10-09T21:48:56.639683Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 1364, pipe: [1:7559342312807767935:4303]
2025-10-09T21:48:56.736213Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:56.742894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.742914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.743051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.743105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.743159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.743216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.743257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.743264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.743321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.743387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.743442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.743500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.743542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.743547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.743601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.743657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.743714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.743769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.743806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.743814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.743871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.743925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.743980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.744037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.744078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.744084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.744142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.744197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.744258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.744313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.744352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.744357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.744416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.744481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.744538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.744603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.744642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.744648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.744700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.744750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.744818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.744875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.744914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.744919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.744973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.745025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.745075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.745130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.745171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.745177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.745228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.745280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.745332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.745389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.745428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.745434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.745488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:56.745549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:56.745609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:56.745673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:56.745711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:56.745717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:56.745757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342312807767935:4303], now have 1 active actors on pipe
2025-10-09T21:48:56.745956Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342312807767935:4303]
2025-10-09T21:48:56.746011Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:56.746014Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:56.746016Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:56.746058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1365 reason: error at write: Disconnected sessionId: src_id|49ebfe78-d4d7a64d-fdd055d9-45d2d8c3_0
2025-10-09T21:48:56.746233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1365 sessionId: src_id|49ebfe78-d4d7a64d-fdd055d9-45d2d8c3_0 is DEAD
2025-10-09T21:48:56.746366Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49ebfe78-d4d7a64d-fdd055d9-45d2d8c3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:56.746377Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49ebfe78-d4d7a64d-fdd055d9-45d2d8c3_0] Write session will restart in 0.005220s
2025-10-09T21:48:56.746396Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49ebfe78-d4d7a64d-fdd055d9-45d2d8c3_0] Write session: Do CDS request
2025-10-09T21:48:56.746401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49ebfe78-d4d7a64d-fdd055d9-45d2d8c3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:56.746597Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:56.752438Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|49ebfe78-d4d7a64d-fdd055d9-45d2d8c3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:56.752525Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:56.752539Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1366
2025-10-09T21:48:56.752640Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1366 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:56.752672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1366 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:56.752678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1366 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:56.752682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:56.753120Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1366 sessionId:  describe result for acl check
2025-10-09T21:48:56.753150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:56.753158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:56.753160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:56.753165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342312807767981:3726] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:56.753663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342312807767981:3726] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:56.755553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342312807767981:3726] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:56.758428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342312807767981:3726] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:56.758440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342312807767981:3726] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:56.758443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342312807767981:3726] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:56.758453Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1366 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:56.760065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:56.762317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:56.762466Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:56.762473Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:56.762482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:56.762518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:56.762546Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342312445943816:9971]
2025-10-09T21:48:56.763117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:56.763423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:56.763477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:56.763805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:56.763875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.763881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.763883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.763884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.763885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.763886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.763888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.763889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.763890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.763895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.763906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.763911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.763915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.763919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.763923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.763926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.763930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.763933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.763940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.763948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.763960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.763966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.763973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.763979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.763983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.763989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.763994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.763999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.764004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.764014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.764019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.764030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.764034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.764039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.764048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.764053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.764081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:56.770156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:56.771185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:56.771198Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:56.771201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:56.771204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:56.771209Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1365 [2:7559342312445943816:9971]
2025-10-09T21:48:56.771219Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:56.771234Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:56.771239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:56.771241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:56.771243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:56.771254Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:56.771262Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:56.771269Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:56.771308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.771313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.771314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.771317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:56.771376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:56.771381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.771389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.771399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.771412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:56.771430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342312807768139:3726], now have 1 active actors on pipe
2025-10-09T21:48:56.771436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342312807767966:2432], now have 1 active actors on pipe
2025-10-09T21:48:56.771441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342312445943811:9971]
2025-10-09T21:48:56.771443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342312445943811:9971]
2025-10-09T21:48:56.771444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342312445943811:9971]
2025-10-09T21:48:56.771454Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:56.771466Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:56.771478Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:56.771485Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:56.771486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:56.771488Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:56.771489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:56.771513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342312445943811:9971]
2025-10-09T21:48:56.771516Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:56.771489Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1365
2025-10-09T21:48:56.771520Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:56.771501Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1365
2025-10-09T21:48:56.771540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.771550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.771586Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e1ca1f21-f9d84b1f-8ccd4228-d4ed13dd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:56.771603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:56.771607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:56.771610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:56.771619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:56.771628Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:56.771651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:56.771775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.771782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.771795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:56.771896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1366 partition: 0 MaxSeqNo: 25 sessionId: src_id|e1ca1f21-f9d84b1f-8ccd4228-d4ed13dd_0
2025-10-09T21:48:56.772228Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49ebfe78-d4d7a64d-fdd055d9-45d2d8c3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e1ca1f21-f9d84b1f-8ccd4228-d4ed13dd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:56.774661Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ca1f21-f9d84b1f-8ccd4228-d4ed13dd_0] Write session: try to update token
2025-10-09T21:48:56.774680Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ca1f21-f9d84b1f-8ccd4228-d4ed13dd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:56.785509Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:56.785523Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:56.785526Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:56.785527Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:56.785538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342312445943811:9971]
2025-10-09T21:48:56.785542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342312445943811:9971]
2025-10-09T21:48:56.785546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342312445943811:9971]
2025-10-09T21:48:56.785590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:56.785591Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:56.785597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.785600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:56.785616Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:56.785621Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:56.785742Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:56.787652Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1366 sessionId: src_id|e1ca1f21-f9d84b1f-8ccd4228-d4ed13dd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:56.790195Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:56.846273Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342312807768167:4303]
2025-10-09T21:48:56.846524Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 1365, pipe: [1:7559342312807768167:4303]
2025-10-09T21:48:56.950355Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:56.950932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.950950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.950994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342312807768167:4303], now have 1 active actors on pipe
2025-10-09T21:48:56.951189Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:56.951310Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:56.951313Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342312807768167:4303]
2025-10-09T21:48:56.951362Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:56.951440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1366 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e1ca1f21-f9d84b1f-8ccd4228-d4ed13dd_0
2025-10-09T21:48:56.951621Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1366 sessionId: src_id|e1ca1f21-f9d84b1f-8ccd4228-d4ed13dd_0 is DEAD
2025-10-09T21:48:56.951991Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ca1f21-f9d84b1f-8ccd4228-d4ed13dd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:56.952002Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ca1f21-f9d84b1f-8ccd4228-d4ed13dd_0] Write session will restart in 0.005298s
2025-10-09T21:48:56.952030Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ca1f21-f9d84b1f-8ccd4228-d4ed13dd_0] Write session: Do CDS request
2025-10-09T21:48:56.952036Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ca1f21-f9d84b1f-8ccd4228-d4ed13dd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:56.958471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ca1f21-f9d84b1f-8ccd4228-d4ed13dd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:56.958560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:56.958579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1367
2025-10-09T21:48:56.958736Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1367 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:56.958770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1367 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:56.958781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1367 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:56.958785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:56.959241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1367 sessionId:  describe result for acl check
2025-10-09T21:48:56.959285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:56.959293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:56.959295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:56.959301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342312807768186:3718] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:56.959857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342312807768186:3718] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:56.962418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342312807768186:3718] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:56.965447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342312807768186:3718] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:56.965463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342312807768186:3718] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:56.965467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342312807768186:3718] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:56.965476Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1367 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:56.965928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:56.968119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:56.968316Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:56.968325Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:56.968338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:56.968389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:56.968428Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342312445944009:9973]
2025-10-09T21:48:56.969050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:56.969392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:56.969432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:56.969752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:56.969802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.969809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.969811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.969812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.969814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.969815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.969816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.969817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.969818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.969822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.969835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.969840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.969844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.969848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.969852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.969855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.969859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.969863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.969871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.969880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:56.969886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.969892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:56.969897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.969902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:56.969906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.969912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:56.969916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.969923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:56.969928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.969933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:56.969938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.969943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.969948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.969954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.969958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.969963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.969995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:56.976625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:56.977710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:56.977725Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:56.977727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:56.977731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:56.977736Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1366 [2:7559342312445944009:9973]
2025-10-09T21:48:56.977757Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:56.977778Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:56.977781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:56.977785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:56.977787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:56.977807Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:56.977816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:56.977829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:56.977910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.977918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.977920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.977922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:56.977983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:56.977989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.977999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:56.978015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:56.978051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:56.978070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342312807768171:2432], now have 1 active actors on pipe
2025-10-09T21:48:56.978076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342312807768357:3718], now have 1 active actors on pipe
2025-10-09T21:48:56.978082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342312445944004:9973]
2025-10-09T21:48:56.978084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342312445944004:9973]
2025-10-09T21:48:56.978085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342312445944004:9973]
2025-10-09T21:48:56.978101Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:56.978119Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:56.978130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:56.978134Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:56.978137Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:56.978139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:56.978143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:56.978145Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1366
2025-10-09T21:48:56.978178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342312445944004:9973]
2025-10-09T21:48:56.978152Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1366
2025-10-09T21:48:56.978185Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:56.978187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:56.978202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.978205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.978247Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8e58d2ab-aa93ec52-747aec59-cc5539dd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:56.978260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:56.978265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:56.978272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:56.978284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:56.978299Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:56.978324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:56.978548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:56.978558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:56.978575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:56.978681Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1367 partition: 0 MaxSeqNo: 25 sessionId: src_id|8e58d2ab-aa93ec52-747aec59-cc5539dd_0
2025-10-09T21:48:56.979048Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ca1f21-f9d84b1f-8ccd4228-d4ed13dd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8e58d2ab-aa93ec52-747aec59-cc5539dd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:56.981446Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e58d2ab-aa93ec52-747aec59-cc5539dd_0] Write session: try to update token
2025-10-09T21:48:56.981464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e58d2ab-aa93ec52-747aec59-cc5539dd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:56.992685Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:56.992705Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:56.992707Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:56.992708Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:56.992716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342312445944004:9973]
2025-10-09T21:48:56.992720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342312445944004:9973]
2025-10-09T21:48:56.992722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342312445944004:9973]
2025-10-09T21:48:56.992758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:56.992764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:56.992766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:56.992768Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:56.992776Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:56.992779Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:56.992883Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:56.994502Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1367 sessionId: src_id|8e58d2ab-aa93ec52-747aec59-cc5539dd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:56.996990Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:57.051590Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342317102735666:4303]
2025-10-09T21:48:57.051829Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 1366, pipe: [1:7559342317102735666:4303]
2025-10-09T21:48:57.141692Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:57.141926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.141940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.141980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342317102735666:4303], now have 1 active actors on pipe
2025-10-09T21:48:57.142157Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:57.142219Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:57.142235Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:57.142246Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342317102735666:4303]
2025-10-09T21:48:57.142263Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:57.142298Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1367 reason: error at write: Disconnected sessionId: src_id|8e58d2ab-aa93ec52-747aec59-cc5539dd_0
2025-10-09T21:48:57.142489Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1367 sessionId: src_id|8e58d2ab-aa93ec52-747aec59-cc5539dd_0 is DEAD
2025-10-09T21:48:57.142634Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e58d2ab-aa93ec52-747aec59-cc5539dd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:57.142647Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e58d2ab-aa93ec52-747aec59-cc5539dd_0] Write session will restart in 0.005662s
2025-10-09T21:48:57.142674Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e58d2ab-aa93ec52-747aec59-cc5539dd_0] Write session: Do CDS request
2025-10-09T21:48:57.142680Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e58d2ab-aa93ec52-747aec59-cc5539dd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:57.149529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e58d2ab-aa93ec52-747aec59-cc5539dd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:57.149601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:57.149617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1368
2025-10-09T21:48:57.149741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1368 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:57.149779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1368 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:57.149785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1368 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:57.149790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:57.150270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1368 sessionId:  describe result for acl check
2025-10-09T21:48:57.150311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:57.150320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:57.150322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:57.150329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342317102735688:3713] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:57.150881Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342317102735688:3713] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:57.153666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342317102735688:3713] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:57.156327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342317102735688:3713] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:57.156344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342317102735688:3713] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:57.156348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342317102735688:3713] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:57.156356Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1368 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:57.157206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:57.159504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:57.159691Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:57.159699Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:57.159717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:57.159785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:57.159818Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342316740911498:9975]
2025-10-09T21:48:57.160225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:57.160518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:57.160550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:57.160772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:57.160808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.160812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.160813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.160814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.160814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.160815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.160816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.160816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.160817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.160820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.160827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.160829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.160832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.160834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.160836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.160838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.160840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.160842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.160847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.160852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.160856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.160860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.160862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.160865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.160868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.160871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.160873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.160876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.160879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.160882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.160885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.160888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.160890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.160893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.160896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.160898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.160915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:57.167327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:57.168322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:57.168335Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:57.168337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:57.168340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:57.168346Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1367 [2:7559342316740911498:9975]
2025-10-09T21:48:57.168355Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:57.168370Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:57.168372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:57.168375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:57.168376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:57.168390Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:57.168398Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:57.168404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:57.168447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.168452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.168453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.168455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:57.168509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:57.168515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.168522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.168528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.168542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:57.168559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342316740911493:9975]
2025-10-09T21:48:57.168564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342316740911493:9975]
2025-10-09T21:48:57.168565Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342316740911493:9975]
2025-10-09T21:48:57.168573Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:57.168585Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:57.168589Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:57.168595Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:57.168597Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:57.168601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:57.168604Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:57.168616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342317102735668:2432], now have 1 active actors on pipe
2025-10-09T21:48:57.168625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342317102735860:3713], now have 1 active actors on pipe
2025-10-09T21:48:57.168652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342316740911493:9975]
2025-10-09T21:48:57.168663Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:57.168678Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:57.168686Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1367
2025-10-09T21:48:57.168705Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1367
2025-10-09T21:48:57.168814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.168838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.168874Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5f8b1ced-80cfb63d-12cc17dd-d4878830_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:57.168887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:57.168896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:57.168899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:57.168912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:57.168930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:57.168952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:57.169153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.169163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.169180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:57.169288Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1368 partition: 0 MaxSeqNo: 25 sessionId: src_id|5f8b1ced-80cfb63d-12cc17dd-d4878830_0
2025-10-09T21:48:57.169655Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e58d2ab-aa93ec52-747aec59-cc5539dd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5f8b1ced-80cfb63d-12cc17dd-d4878830_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:57.171982Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f8b1ced-80cfb63d-12cc17dd-d4878830_0] Write session: try to update token
2025-10-09T21:48:57.171996Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f8b1ced-80cfb63d-12cc17dd-d4878830_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:57.180300Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:57.180319Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:57.180328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342316740911493:9975]
2025-10-09T21:48:57.180375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:57.180374Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:57.180596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:57.180737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:57.180875Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:57.181011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:57.181198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:57.181243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:57.184446Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1368 sessionId: src_id|5f8b1ced-80cfb63d-12cc17dd-d4878830_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:57.186886Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:57.187092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.187115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.187156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:57.187230Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:57.193421Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:57.193438Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:57.193440Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:57.193442Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:57.193452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342316740911493:9975]
2025-10-09T21:48:57.193455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342316740911493:9975]
2025-10-09T21:48:57.193460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342316740911493:9975]
2025-10-09T21:48:57.193500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:57.193502Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:57.193509Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:57.193512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.193513Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:57.193515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:57.193645Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:57.242599Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342317102735879:4303]
2025-10-09T21:48:57.242876Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 1367, pipe: [1:7559342317102735879:4303]
2025-10-09T21:48:57.338648Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:57.345489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.345511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.345663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.345716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.345773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.345829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.345877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.345886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.345944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.345998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.346073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.346127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.346165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.346171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.346228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.346282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.346335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.346386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.346425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.346430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.346486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.346541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.346596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.346649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.346689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.346694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.346749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.346801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.346859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.346921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.346970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.346979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.347044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.347097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.347151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.347205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.347243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.347253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.347318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.347376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.347430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.347481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.347519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.347525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.347579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.347629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.347684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.347738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.347776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.347781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.347840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.347899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.347959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.348026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.348068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.348074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.348127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.348177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.348233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.348302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.348345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.348351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.348388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342317102735879:4303], now have 1 active actors on pipe
2025-10-09T21:48:57.348645Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:57.348649Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342317102735879:4303]
2025-10-09T21:48:57.348661Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:57.348667Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:57.348691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1368 reason: error at write: Disconnected sessionId: src_id|5f8b1ced-80cfb63d-12cc17dd-d4878830_0
2025-10-09T21:48:57.348870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1368 sessionId: src_id|5f8b1ced-80cfb63d-12cc17dd-d4878830_0 is DEAD
2025-10-09T21:48:57.348992Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f8b1ced-80cfb63d-12cc17dd-d4878830_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:57.349011Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f8b1ced-80cfb63d-12cc17dd-d4878830_0] Write session will restart in 0.006940s
2025-10-09T21:48:57.349046Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f8b1ced-80cfb63d-12cc17dd-d4878830_0] Write session: Do CDS request
2025-10-09T21:48:57.349052Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f8b1ced-80cfb63d-12cc17dd-d4878830_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:57.349256Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:57.357485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f8b1ced-80cfb63d-12cc17dd-d4878830_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:57.357686Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:57.357701Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1369
2025-10-09T21:48:57.357815Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1369 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:57.357849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1369 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:57.357861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1369 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:57.357869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:57.358304Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1369 sessionId:  describe result for acl check
2025-10-09T21:48:57.358333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:57.358334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:57.358335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:57.358339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342317102735932:3708] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:57.358754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342317102735932:3708] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:57.361247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342317102735932:3708] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:57.362423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:57.364442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:57.364614Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:57.364623Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:57.364635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:57.364684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:57.364714Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342316740911691:9977]
2025-10-09T21:48:57.364741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342317102735932:3708] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:57.364754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342317102735932:3708] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:57.364759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342317102735932:3708] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:57.364769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1369 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:57.365121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:57.365348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:57.365374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:57.365595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:57.365630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.365634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.365635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.365636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.365636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.365637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.365638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.365638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.365639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.365642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.365661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.365663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.365665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.365667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.365670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.365672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.365674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.365676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.365680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.365685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.365689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.365696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.365698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.365701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.365704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.365707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.365713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.365716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.365718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.365721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.365724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.365727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.365730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.365736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.365739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.365742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.365757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:57.371996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:57.373025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:57.373038Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:57.373041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:57.373045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:57.373051Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1368 [2:7559342316740911691:9977]
2025-10-09T21:48:57.373063Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:57.373087Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:57.373097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:57.373100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:57.373103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:57.373119Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:57.373124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:57.373131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:57.373184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.373191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.373193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.373196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:57.373259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:57.373268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.373281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.373298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.373320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:57.373346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342317102735882:2432], now have 1 active actors on pipe
2025-10-09T21:48:57.373354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342317102736074:3708], now have 1 active actors on pipe
2025-10-09T21:48:57.373361Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342316740911686:9977]
2025-10-09T21:48:57.373367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342316740911686:9977]
2025-10-09T21:48:57.373370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342316740911686:9977]
2025-10-09T21:48:57.373382Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:57.373402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:57.373409Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:57.373413Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:57.373415Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:57.373393Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1368
2025-10-09T21:48:57.373394Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1368
2025-10-09T21:48:57.373421Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:57.373428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:57.373463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.373472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.373493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342316740911686:9977]
2025-10-09T21:48:57.373505Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:57.373517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:57.373538Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e3a4b863-a5c6ade7-77493490-5a8653d7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:57.373549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:57.373557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:57.373559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:57.373569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:57.373582Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:57.373606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:57.373770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.373776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.373790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:57.373917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1369 partition: 0 MaxSeqNo: 25 sessionId: src_id|e3a4b863-a5c6ade7-77493490-5a8653d7_0
2025-10-09T21:48:57.374248Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f8b1ced-80cfb63d-12cc17dd-d4878830_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e3a4b863-a5c6ade7-77493490-5a8653d7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:57.376524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3a4b863-a5c6ade7-77493490-5a8653d7_0] Write session: try to update token
2025-10-09T21:48:57.376538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3a4b863-a5c6ade7-77493490-5a8653d7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:57.388983Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:57.389000Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:57.389003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:57.389005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:57.389014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342316740911686:9977]
2025-10-09T21:48:57.389019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342316740911686:9977]
2025-10-09T21:48:57.389023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342316740911686:9977]
2025-10-09T21:48:57.389078Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:57.389091Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:57.389096Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:57.389304Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1369 sessionId: src_id|e3a4b863-a5c6ade7-77493490-5a8653d7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:57.389352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:57.389364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.389366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:57.389487Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:57.391656Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:57.448963Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342317102736085:4303]
2025-10-09T21:48:57.449218Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 1368, pipe: [1:7559342317102736085:4303]
2025-10-09T21:48:57.533765Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:57.534137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.534150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.534187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342317102736085:4303], now have 1 active actors on pipe
2025-10-09T21:48:57.534431Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:57.534436Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:57.534445Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:57.534485Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342317102736085:4303]
2025-10-09T21:48:57.534509Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1369 reason: error at write: Disconnected sessionId: src_id|e3a4b863-a5c6ade7-77493490-5a8653d7_0
2025-10-09T21:48:57.534683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1369 sessionId: src_id|e3a4b863-a5c6ade7-77493490-5a8653d7_0 is DEAD
2025-10-09T21:48:57.534846Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3a4b863-a5c6ade7-77493490-5a8653d7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:57.534858Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3a4b863-a5c6ade7-77493490-5a8653d7_0] Write session will restart in 0.009234s
2025-10-09T21:48:57.534363Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:57.534877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3a4b863-a5c6ade7-77493490-5a8653d7_0] Write session: Do CDS request
2025-10-09T21:48:57.534883Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3a4b863-a5c6ade7-77493490-5a8653d7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:57.545578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3a4b863-a5c6ade7-77493490-5a8653d7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:57.545667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:57.545695Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1370
2025-10-09T21:48:57.545842Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1370 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:57.545869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1370 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:57.545877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1370 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:57.545880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:57.546420Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1370 sessionId:  describe result for acl check
2025-10-09T21:48:57.546460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:57.546462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:57.546464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:57.546470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342317102736231:3703] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:57.547020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342317102736231:3703] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:57.548102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:57.549182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342317102736231:3703] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:57.550084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:57.550260Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:57.550269Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:57.550280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:57.550327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:57.550364Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342316740911886:9979]
2025-10-09T21:48:57.550948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:57.551239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:57.551278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:57.551413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342317102736231:3703] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:57.551427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342317102736231:3703] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:57.551430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342317102736231:3703] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:57.551438Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1370 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:57.551580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:57.551625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.551631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.551633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.551634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.551636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.551637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.551638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.551639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.551640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.551645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.551655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.551659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.551663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.551667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.551671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.551675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.551679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.551683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.551689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.551697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.551702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.551708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.551714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.551718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.551727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.551733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.551737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.551746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.551751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.551760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.551766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.551775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.551780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.551785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.551791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.551796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.551829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:57.558300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:57.559310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:57.559324Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:57.559325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:57.559329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:57.559335Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1369 [2:7559342316740911886:9979]
2025-10-09T21:48:57.559348Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:57.559365Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:57.559367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:57.559371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:57.559373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:57.559389Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:57.559400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:57.559407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:57.559460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.559467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.559469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.559472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:57.559534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:57.559541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.559552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.559569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.559589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:57.559612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342317102736283:3703], now have 1 active actors on pipe
2025-10-09T21:48:57.559620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342317102736088:2432], now have 1 active actors on pipe
2025-10-09T21:48:57.559627Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342316740911881:9979]
2025-10-09T21:48:57.559634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342316740911881:9979]
2025-10-09T21:48:57.559636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342316740911881:9979]
2025-10-09T21:48:57.559651Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:57.559665Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:57.559647Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1369
2025-10-09T21:48:57.559675Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:57.559679Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:57.559673Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1369
2025-10-09T21:48:57.559682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:57.559683Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:57.559685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:57.559723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.559732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.559745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342316740911881:9979]
2025-10-09T21:48:57.559758Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:57.559768Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:57.559796Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|89396021-56248e56-a02b386-f42a90a6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:57.559808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:57.559817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:57.559819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:57.559834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:57.559847Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:57.559875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:57.560004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.560011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.560024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:57.560099Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1370 partition: 0 MaxSeqNo: 25 sessionId: src_id|89396021-56248e56-a02b386-f42a90a6_0
2025-10-09T21:48:57.560425Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3a4b863-a5c6ade7-77493490-5a8653d7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|89396021-56248e56-a02b386-f42a90a6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:57.562785Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|89396021-56248e56-a02b386-f42a90a6_0] Write session: try to update token
2025-10-09T21:48:57.562804Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|89396021-56248e56-a02b386-f42a90a6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:57.573327Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:57.573345Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:57.573347Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:57.573349Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:57.573359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342316740911881:9979]
2025-10-09T21:48:57.573364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342316740911881:9979]
2025-10-09T21:48:57.573367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342316740911881:9979]
2025-10-09T21:48:57.573410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:57.573413Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:57.573419Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:57.573421Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:57.573424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.573427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:57.573555Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:57.575271Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1370 sessionId: src_id|89396021-56248e56-a02b386-f42a90a6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:57.577558Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:57.634769Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342317102736292:4303]
2025-10-09T21:48:57.635075Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 1369, pipe: [1:7559342317102736292:4303]
2025-10-09T21:48:57.727270Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:57.727469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.727488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.727532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342317102736292:4303], now have 1 active actors on pipe
2025-10-09T21:48:57.727691Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:57.727716Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:57.727729Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:57.727774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1370 reason: error at write: Disconnected sessionId: src_id|89396021-56248e56-a02b386-f42a90a6_0
2025-10-09T21:48:57.727930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1370 sessionId: src_id|89396021-56248e56-a02b386-f42a90a6_0 is DEAD
2025-10-09T21:48:57.728063Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89396021-56248e56-a02b386-f42a90a6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:57.728090Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89396021-56248e56-a02b386-f42a90a6_0] Write session will restart in 0.008978s
2025-10-09T21:48:57.728084Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:57.728115Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89396021-56248e56-a02b386-f42a90a6_0] Write session: Do CDS request
2025-10-09T21:48:57.728122Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89396021-56248e56-a02b386-f42a90a6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:57.728122Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342317102736292:4303]
2025-10-09T21:48:57.738463Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|89396021-56248e56-a02b386-f42a90a6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:57.738514Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:57.738530Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1371
2025-10-09T21:48:57.738675Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1371 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:57.738707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1371 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:57.738713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1371 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:57.738717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:57.739229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1371 sessionId:  describe result for acl check
2025-10-09T21:48:57.739260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:57.739267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:57.739269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:57.739274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342317102736422:3684] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:57.739805Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342317102736422:3684] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:57.741549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342317102736422:3684] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:57.741729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:57.743847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:57.744039Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:57.744049Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:57.744062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:57.744130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:57.744166Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342316740912116:9993]
2025-10-09T21:48:57.744157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342317102736422:3684] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:57.744171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342317102736422:3684] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:57.744175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342317102736422:3684] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:57.744182Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1371 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:57.744577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:57.744819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:57.744857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:57.745157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:57.745222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.745228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.745229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.745230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.745232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.745233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.745234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.745235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.745236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.745241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.745251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.745255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.745259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.745262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.745266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.745269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.745273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.745277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.745283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.745290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.745297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.745303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.745307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.745312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.745316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.745322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.745326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.745332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.745336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.745342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.745346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.745353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.745357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.745362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.745368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.745373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.745406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:57.751829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:57.752821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:57.752836Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:57.752838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:57.752842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:57.752848Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1370 [2:7559342316740912116:9993]
2025-10-09T21:48:57.752861Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:57.752878Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:57.752881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:57.752884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:57.752887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:57.752903Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:57.752908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:57.752915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:57.752966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.752974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.752976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.752979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:57.753041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:57.753048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.753060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.753077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.753095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:57.753120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342317102736500:3684], now have 1 active actors on pipe
2025-10-09T21:48:57.753127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342317102736302:2432], now have 1 active actors on pipe
2025-10-09T21:48:57.753135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342316740912111:9993]
2025-10-09T21:48:57.753142Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342316740912111:9993]
2025-10-09T21:48:57.753144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342316740912111:9993]
2025-10-09T21:48:57.753158Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:57.753174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:57.753196Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:57.753204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:57.753204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:57.753208Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:57.753210Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:57.753231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342316740912111:9993]
2025-10-09T21:48:57.753234Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:57.753236Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:57.753237Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1370
2025-10-09T21:48:57.753274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1370
2025-10-09T21:48:57.753279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.753292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.753323Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|13933075-968b43c-ae6854dd-6e08394_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:57.753330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:57.753334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:57.753336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:57.753347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:57.753363Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:57.753378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:57.753548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.753556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.753568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:57.753673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1371 partition: 0 MaxSeqNo: 25 sessionId: src_id|13933075-968b43c-ae6854dd-6e08394_0
2025-10-09T21:48:57.754042Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89396021-56248e56-a02b386-f42a90a6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|13933075-968b43c-ae6854dd-6e08394_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:57.756671Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|13933075-968b43c-ae6854dd-6e08394_0] Write session: try to update token
2025-10-09T21:48:57.756690Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|13933075-968b43c-ae6854dd-6e08394_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:57.769604Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1371 sessionId: src_id|13933075-968b43c-ae6854dd-6e08394_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:57.771743Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:57.771756Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:57.771764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342316740912111:9993]
2025-10-09T21:48:57.771802Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:57.771804Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:57.771821Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:57.771950Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:57.772066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:57.772183Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:57.772298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:57.772472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:57.772512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.772523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.772539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:57.772572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:57.772642Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:57.777638Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:57.777665Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:57.777668Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:57.777671Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:57.777679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342316740912111:9993]
2025-10-09T21:48:57.777683Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342316740912111:9993]
2025-10-09T21:48:57.777686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342316740912111:9993]
2025-10-09T21:48:57.777727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:57.777731Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:57.777736Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:57.777737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.777740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:57.777740Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:57.777860Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:57.828297Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342317102736523:4303]
2025-10-09T21:48:57.828537Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 1370, pipe: [1:7559342317102736523:4303]
2025-10-09T21:48:57.923464Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:57.930992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.931019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.931181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.931248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.931319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.931388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.931430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.931439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.931505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.931561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.931630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.931691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.931732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.931741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.931797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.931852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.931914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.931965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.932004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.932009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.932065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.932117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.932173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.932226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.932264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.932269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.932325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.932379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.932431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.932485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.932522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.932527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.932579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.932630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.932683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.932736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.932775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.932781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.932831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.932881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.932936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.932988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.933026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.933031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.933080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.933141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.933196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.933245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.933307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.933312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.933360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.933410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.933460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.933517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.933571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.933577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.933628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:57.933689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:57.933747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:57.933807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:57.933844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:57.933850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:57.933888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342317102736523:4303], now have 1 active actors on pipe
2025-10-09T21:48:57.934118Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342317102736523:4303]
2025-10-09T21:48:57.934149Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:57.934151Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:57.934155Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:57.934184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1371 reason: error at write: Disconnected sessionId: src_id|13933075-968b43c-ae6854dd-6e08394_0
2025-10-09T21:48:57.934370Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1371 sessionId: src_id|13933075-968b43c-ae6854dd-6e08394_0 is DEAD
2025-10-09T21:48:57.934500Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13933075-968b43c-ae6854dd-6e08394_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:57.934512Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13933075-968b43c-ae6854dd-6e08394_0] Write session will restart in 0.009243s
2025-10-09T21:48:57.934537Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13933075-968b43c-ae6854dd-6e08394_0] Write session: Do CDS request
2025-10-09T21:48:57.934544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13933075-968b43c-ae6854dd-6e08394_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:57.934750Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:57.944516Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|13933075-968b43c-ae6854dd-6e08394_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:57.944643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:57.944661Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1372
2025-10-09T21:48:57.944821Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1372 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:57.944855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1372 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:57.944866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1372 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:57.944874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:57.945327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1372 sessionId:  describe result for acl check
2025-10-09T21:48:57.945358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:57.945361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:57.945363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:57.945379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342317102736625:3678] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:57.945930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342317102736625:3678] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:57.948247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:57.948750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342317102736625:3678] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:57.950266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:57.950447Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:57.950461Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:57.950474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:57.950533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:57.950577Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342316740912322:9997]
2025-10-09T21:48:57.951152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:57.951506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:57.951560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:57.951908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:57.951982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342317102736625:3678] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:57.952002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.951996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342317102736625:3678] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:57.952009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.952000Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342317102736625:3678] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:57.952011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.952008Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1372 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:57.952012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.952013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.952014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.952015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.952017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.952018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.952024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.952035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.952040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.952044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.952048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.952052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.952056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.952060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.952064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.952072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.952080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:57.952087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.952093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:57.952098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.952108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:57.952112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.952117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:57.952122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.952131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:57.952135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.952140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:57.952145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.952150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.952154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.952159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.952168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.952173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.952203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:57.958540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:57.959786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:57.959800Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:57.959801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:57.959804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:57.959810Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1371 [2:7559342316740912322:9997]
2025-10-09T21:48:57.959818Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:57.959832Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:57.959834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:57.959836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:57.959837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:57.959850Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:57.959854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:57.959861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:57.959908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.959914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.959915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.959917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:57.959975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:57.959983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.959994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:57.960006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:57.960028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:57.960051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342317102736526:2432], now have 1 active actors on pipe
2025-10-09T21:48:57.960060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342316740912317:9997]
2025-10-09T21:48:57.960062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342316740912317:9997]
2025-10-09T21:48:57.960063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342316740912317:9997]
2025-10-09T21:48:57.960073Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:57.960080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342317102736722:3678], now have 1 active actors on pipe
2025-10-09T21:48:57.960089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:57.960091Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:57.960094Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:57.960095Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:57.960101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:57.960104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:57.960106Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1371
2025-10-09T21:48:57.960128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342316740912317:9997]
2025-10-09T21:48:57.960113Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1371
2025-10-09T21:48:57.960136Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:57.960137Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:57.960152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.960161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.960213Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bcd6891f-5cf7bbfe-b82402af-e4193ae_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:57.960225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:57.960229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:57.960231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:57.960242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:57.960256Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:57.960278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:57.960414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:57.960421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:57.960434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:57.960515Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1372 partition: 0 MaxSeqNo: 25 sessionId: src_id|bcd6891f-5cf7bbfe-b82402af-e4193ae_0
2025-10-09T21:48:57.960850Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13933075-968b43c-ae6854dd-6e08394_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bcd6891f-5cf7bbfe-b82402af-e4193ae_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:57.963177Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bcd6891f-5cf7bbfe-b82402af-e4193ae_0] Write session: try to update token
2025-10-09T21:48:57.963192Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bcd6891f-5cf7bbfe-b82402af-e4193ae_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:57.973976Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:57.973991Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:57.973994Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:57.973997Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:57.974006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342316740912317:9997]
2025-10-09T21:48:57.974010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342316740912317:9997]
2025-10-09T21:48:57.974014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342316740912317:9997]
2025-10-09T21:48:57.974065Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:57.974070Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:57.974073Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:57.974073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:57.974081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:57.974084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:57.974216Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:57.975663Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1372 sessionId: src_id|bcd6891f-5cf7bbfe-b82402af-e4193ae_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:57.977967Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:58.034200Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342321397704029:4303]
2025-10-09T21:48:58.034498Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 1371, pipe: [1:7559342321397704029:4303]
2025-10-09T21:48:58.126070Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:58.126456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.126472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.126519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342321397704029:4303], now have 1 active actors on pipe
2025-10-09T21:48:58.126730Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:58.126782Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:58.126811Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342321397704029:4303]
2025-10-09T21:48:58.126833Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:58.126910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1372 reason: status is not ok: tablet will be restarted right now sessionId: src_id|bcd6891f-5cf7bbfe-b82402af-e4193ae_0
2025-10-09T21:48:58.127104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1372 sessionId: src_id|bcd6891f-5cf7bbfe-b82402af-e4193ae_0 is DEAD
2025-10-09T21:48:58.127255Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bcd6891f-5cf7bbfe-b82402af-e4193ae_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:58.127265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bcd6891f-5cf7bbfe-b82402af-e4193ae_0] Write session will restart in 0.009285s
2025-10-09T21:48:58.127285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bcd6891f-5cf7bbfe-b82402af-e4193ae_0] Write session: Do CDS request
2025-10-09T21:48:58.127290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bcd6891f-5cf7bbfe-b82402af-e4193ae_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:58.137545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bcd6891f-5cf7bbfe-b82402af-e4193ae_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:58.137674Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:58.137694Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1373
2025-10-09T21:48:58.137826Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1373 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:58.137861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1373 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:58.137872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1373 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:58.137875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:58.138311Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1373 sessionId:  describe result for acl check
2025-10-09T21:48:58.138344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:58.138349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:58.138351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:58.138356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342321397704111:3675] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:58.138838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342321397704111:3675] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:58.141390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342321397704111:3675] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:58.142661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:58.144783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:58.144978Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:58.144989Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:58.145000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:58.145047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:58.145087Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342321035879816:9999]
2025-10-09T21:48:58.145134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342321397704111:3675] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:58.145152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342321397704111:3675] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:58.145156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342321397704111:3675] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:58.145166Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1373 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:58.145639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:58.145951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:58.146026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:58.146390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:58.146469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.146476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.146477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.146478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.146479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.146480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.146481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.146482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.146484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.146490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.146501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.146505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.146509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.146512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.146516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.146519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.146524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.146528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.146536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.146544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.146552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.146560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.146565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.146575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.146579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.146585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.146589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.146595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.146602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.146608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.146614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.146624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.146629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.146634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.146644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.146649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.146679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:58.153482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:58.154587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:58.154601Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:58.154604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:58.154609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:58.154617Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1372 [2:7559342321035879816:9999]
2025-10-09T21:48:58.154636Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:58.154657Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:58.154660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:58.154663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:58.154666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:58.154686Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:58.154694Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:58.154701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:58.154748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.154753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.154755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.154758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:58.154838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:58.154847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.154860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.154879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.154896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:58.154919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342321397704227:3675], now have 1 active actors on pipe
2025-10-09T21:48:58.154930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342321397704031:2432], now have 1 active actors on pipe
2025-10-09T21:48:58.154943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342321035879811:9999]
2025-10-09T21:48:58.154950Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342321035879811:9999]
2025-10-09T21:48:58.154952Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342321035879811:9999]
2025-10-09T21:48:58.154966Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:58.154981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:58.154989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:58.154990Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:58.155004Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:58.155005Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:58.155008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:58.154994Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1372
2025-10-09T21:48:58.155025Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1372
2025-10-09T21:48:58.155052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342321035879811:9999]
2025-10-09T21:48:58.155066Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:58.155086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.155088Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:58.155097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.155153Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1a7a5c5b-15f11a0b-3e6c9066-eb08f9af_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:58.155166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:58.155170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:58.155175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:58.155186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:58.155201Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:58.155230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:58.155371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.155382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.155396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:58.155471Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1373 partition: 0 MaxSeqNo: 25 sessionId: src_id|1a7a5c5b-15f11a0b-3e6c9066-eb08f9af_0
2025-10-09T21:48:58.155853Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bcd6891f-5cf7bbfe-b82402af-e4193ae_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1a7a5c5b-15f11a0b-3e6c9066-eb08f9af_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:58.158566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a7a5c5b-15f11a0b-3e6c9066-eb08f9af_0] Write session: try to update token
2025-10-09T21:48:58.158586Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a7a5c5b-15f11a0b-3e6c9066-eb08f9af_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:58.169824Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:58.169839Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:58.169841Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:58.169843Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:58.169852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342321035879811:9999]
2025-10-09T21:48:58.169856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342321035879811:9999]
2025-10-09T21:48:58.169858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342321035879811:9999]
2025-10-09T21:48:58.169897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:58.169902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.169902Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:58.169904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:58.169908Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:58.169915Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:58.170053Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:58.173539Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1373 sessionId: src_id|1a7a5c5b-15f11a0b-3e6c9066-eb08f9af_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:58.176264Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:58.227193Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342321397704242:4303]
2025-10-09T21:48:58.227463Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 1372, pipe: [1:7559342321397704242:4303]
2025-10-09T21:48:58.320731Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:58.320962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.320973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.321008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342321397704242:4303], now have 1 active actors on pipe
2025-10-09T21:48:58.321181Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:58.321249Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342321397704242:4303]
2025-10-09T21:48:58.321274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:58.321282Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:58.321348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1373 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1a7a5c5b-15f11a0b-3e6c9066-eb08f9af_0
2025-10-09T21:48:58.321497Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1373 sessionId: src_id|1a7a5c5b-15f11a0b-3e6c9066-eb08f9af_0 is DEAD
2025-10-09T21:48:58.321635Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a7a5c5b-15f11a0b-3e6c9066-eb08f9af_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:58.321657Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a7a5c5b-15f11a0b-3e6c9066-eb08f9af_0] Write session will restart in 0.006915s
2025-10-09T21:48:58.321683Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a7a5c5b-15f11a0b-3e6c9066-eb08f9af_0] Write session: Do CDS request
2025-10-09T21:48:58.321690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a7a5c5b-15f11a0b-3e6c9066-eb08f9af_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:58.329472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a7a5c5b-15f11a0b-3e6c9066-eb08f9af_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:58.329551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:58.329564Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1374
2025-10-09T21:48:58.329690Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1374 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:58.329715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1374 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:58.329719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1374 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:58.329722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:58.330140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1374 sessionId:  describe result for acl check
2025-10-09T21:48:58.330176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:58.330180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:58.330181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:58.330187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342321397704300:3665] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:58.330741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342321397704300:3665] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:58.333219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342321397704300:3665] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:58.335041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:58.336773Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342321397704300:3665] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:58.336787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342321397704300:3665] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:58.336790Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342321397704300:3665] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:58.336797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1374 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:58.336833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:58.336990Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:58.336996Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:58.337007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:58.337050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:58.337086Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342321035880016:10001]
2025-10-09T21:48:58.337678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:58.337916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:58.337954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:58.338270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:58.338314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.338319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.338320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.338321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.338322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.338324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.338325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.338326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.338327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.338332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.338342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.338346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.338350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.338354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.338358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.338362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.338371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.338375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.338385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.338397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.338407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.338414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.338419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.338424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.338432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.338438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.338446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.338451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.338461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.338468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.338477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.338486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.338490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.338499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.338504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.338509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.338539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:58.344716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:58.345915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:58.345927Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:58.345930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:58.345933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:58.345940Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1373 [2:7559342321035880016:10001]
2025-10-09T21:48:58.345952Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:58.345967Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:58.345975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:58.345978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:58.345980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:58.345996Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:58.346007Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:58.346018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:58.346086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.346092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.346094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.346097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:58.346161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:58.346168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.346180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.346197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.346219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:58.346243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342321397704249:2432], now have 1 active actors on pipe
2025-10-09T21:48:58.346251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342321397704446:3665], now have 1 active actors on pipe
2025-10-09T21:48:58.346258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342321035880010:10001]
2025-10-09T21:48:58.346264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342321035880010:10001]
2025-10-09T21:48:58.346266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342321035880010:10001]
2025-10-09T21:48:58.346283Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:58.346275Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1373
2025-10-09T21:48:58.346300Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:58.346276Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1373
2025-10-09T21:48:58.346303Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:58.346304Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:58.346305Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:58.346320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:58.346324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:58.346355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.346359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.346375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342321035880010:10001]
2025-10-09T21:48:58.346389Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:58.346400Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:58.346416Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f2838ba-6b453cb3-b1145923-65af6405_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:58.346428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:58.346432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:58.346436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:58.346449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:58.346462Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:58.346486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:58.346599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.346607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.346623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:58.346717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1374 partition: 0 MaxSeqNo: 25 sessionId: src_id|f2838ba-6b453cb3-b1145923-65af6405_0
2025-10-09T21:48:58.346994Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a7a5c5b-15f11a0b-3e6c9066-eb08f9af_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f2838ba-6b453cb3-b1145923-65af6405_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:58.349181Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2838ba-6b453cb3-b1145923-65af6405_0] Write session: try to update token
2025-10-09T21:48:58.349194Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2838ba-6b453cb3-b1145923-65af6405_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:58.360349Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:58.360364Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:58.360366Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:58.360368Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:58.360378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342321035880010:10001]
2025-10-09T21:48:58.360382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342321035880010:10001]
2025-10-09T21:48:58.360386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342321035880010:10001]
2025-10-09T21:48:58.360426Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:58.360429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:58.360436Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:58.360436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.360439Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:58.360439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:58.360566Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:58.360754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1374 sessionId: src_id|f2838ba-6b453cb3-b1145923-65af6405_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:58.363046Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:58.421211Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342321397704458:4303]
2025-10-09T21:48:58.421479Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 1373, pipe: [1:7559342321397704458:4303]
2025-10-09T21:48:58.509531Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:58.509759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.509776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.509816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342321397704458:4303], now have 1 active actors on pipe
2025-10-09T21:48:58.509972Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:58.510060Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342321397704458:4303]
2025-10-09T21:48:58.510062Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:58.510092Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:58.510158Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1374 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f2838ba-6b453cb3-b1145923-65af6405_0
2025-10-09T21:48:58.510316Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1374 sessionId: src_id|f2838ba-6b453cb3-b1145923-65af6405_0 is DEAD
2025-10-09T21:48:58.510470Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2838ba-6b453cb3-b1145923-65af6405_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:58.510480Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2838ba-6b453cb3-b1145923-65af6405_0] Write session will restart in 0.009906s
2025-10-09T21:48:58.510503Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2838ba-6b453cb3-b1145923-65af6405_0] Write session: Do CDS request
2025-10-09T21:48:58.510509Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2838ba-6b453cb3-b1145923-65af6405_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:58.521439Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2838ba-6b453cb3-b1145923-65af6405_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:58.521534Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:58.521548Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1375
2025-10-09T21:48:58.521696Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1375 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:58.521730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1375 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:58.521736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1375 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:58.521744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:58.522123Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1375 sessionId:  describe result for acl check
2025-10-09T21:48:58.522153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:58.522159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:58.522161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:58.522167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342321397704608:3661] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:58.522592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342321397704608:3661] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:58.523976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:58.524275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342321397704608:3661] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:58.526015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:58.526181Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:58.526190Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:58.526201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:58.526250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:58.526286Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342321035880215:10003]
2025-10-09T21:48:58.526855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:58.527128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:58.527169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:58.527422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342321397704608:3661] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:58.527434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342321397704608:3661] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:58.527438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342321397704608:3661] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:58.527445Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1375 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:58.527453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:58.527491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.527497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.527498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.527500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.527501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.527502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.527503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.527504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.527505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.527509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.527516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.527518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.527520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.527522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.527525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.527529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.527533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.527536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.527542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.527550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.527555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.527562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.527568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.527573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.527577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.527581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.527586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.527590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.527599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.527607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.527615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.527618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.527624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.527627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.527630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.527634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.527650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:58.533503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:58.534512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:58.534526Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:58.534529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:58.534533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:58.534539Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1374 [2:7559342321035880215:10003]
2025-10-09T21:48:58.534551Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:58.534566Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:58.534569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:58.534572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:58.534574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:58.534589Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:58.534597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:58.534604Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:58.534687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.534694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.534696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.534699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:58.534760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:58.534767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.534778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.534794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.534817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:58.534842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342321397704461:2432], now have 1 active actors on pipe
2025-10-09T21:48:58.534852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342321397704661:3661], now have 1 active actors on pipe
2025-10-09T21:48:58.534859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342321035880210:10003]
2025-10-09T21:48:58.534861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342321035880210:10003]
2025-10-09T21:48:58.534863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342321035880210:10003]
2025-10-09T21:48:58.534875Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:58.534889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:58.534897Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:58.534899Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:58.534901Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:58.534909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:58.534912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:58.534898Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1374
2025-10-09T21:48:58.534911Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1374
2025-10-09T21:48:58.534934Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342321035880210:10003]
2025-10-09T21:48:58.534945Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:58.534947Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:58.534961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.534969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.535015Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7ce2bc34-7deed736-4ccb95cc-68120b0f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:58.535029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:58.535033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:58.535035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:58.535045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:58.535054Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:58.535076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:58.535224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.535232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.535245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:58.535320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1375 partition: 0 MaxSeqNo: 25 sessionId: src_id|7ce2bc34-7deed736-4ccb95cc-68120b0f_0
2025-10-09T21:48:58.535619Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2838ba-6b453cb3-b1145923-65af6405_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7ce2bc34-7deed736-4ccb95cc-68120b0f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:58.537939Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ce2bc34-7deed736-4ccb95cc-68120b0f_0] Write session: try to update token
2025-10-09T21:48:58.537956Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ce2bc34-7deed736-4ccb95cc-68120b0f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:58.548255Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:58.548274Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:58.548277Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:58.548279Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:58.548289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342321035880210:10003]
2025-10-09T21:48:58.548294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342321035880210:10003]
2025-10-09T21:48:58.548297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342321035880210:10003]
2025-10-09T21:48:58.548341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:58.548341Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:58.548347Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:58.548352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.548354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:58.548352Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:58.548482Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:58.549791Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1375 sessionId: src_id|7ce2bc34-7deed736-4ccb95cc-68120b0f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:58.552057Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:58.610193Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342321397704670:4303]
2025-10-09T21:48:58.610423Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 1374, pipe: [1:7559342321397704670:4303]
2025-10-09T21:48:58.688662Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:58.688869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.688882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.688920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342321397704670:4303], now have 1 active actors on pipe
2025-10-09T21:48:58.689070Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:58.689155Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342321397704670:4303]
2025-10-09T21:48:58.689158Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:58.689192Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:58.689262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1375 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7ce2bc34-7deed736-4ccb95cc-68120b0f_0
2025-10-09T21:48:58.689433Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1375 sessionId: src_id|7ce2bc34-7deed736-4ccb95cc-68120b0f_0 is DEAD
2025-10-09T21:48:58.689603Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ce2bc34-7deed736-4ccb95cc-68120b0f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:48:58.689615Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ce2bc34-7deed736-4ccb95cc-68120b0f_0] Write session will restart in 0.008903s
2025-10-09T21:48:58.689642Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ce2bc34-7deed736-4ccb95cc-68120b0f_0] Write session: Do CDS request
2025-10-09T21:48:58.689660Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ce2bc34-7deed736-4ccb95cc-68120b0f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:58.699484Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ce2bc34-7deed736-4ccb95cc-68120b0f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:58.699571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:58.699587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1376
2025-10-09T21:48:58.699695Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1376 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:58.699729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1376 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:58.699740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1376 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:58.699743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:58.700137Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1376 sessionId:  describe result for acl check
2025-10-09T21:48:58.700172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:58.700179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:58.700181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:58.700188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342321397704792:3658] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:58.700749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342321397704792:3658] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:58.702604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342321397704792:3658] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:58.702643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:58.704518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:58.704690Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:58.704701Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:58.704713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:58.704772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:58.704811Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342321035880420:10007]
2025-10-09T21:48:58.704801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342321397704792:3658] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:58.704813Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342321397704792:3658] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:58.704815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342321397704792:3658] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:58.704827Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1376 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:58.705354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:58.705633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:58.705677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:58.705917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:58.705955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.705959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.705960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.705960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.705961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.705962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.705962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.705963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.705963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.705966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.705973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.705975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.705977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.705979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.705981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.705983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.705985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.705987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.705991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.705996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.706000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.706003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.706006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.706009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.706011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.706014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.706017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.706020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.706022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.706041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.706050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.706055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.706058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.706063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.706072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.706076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.706100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:58.712001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:58.713261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:58.713276Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:58.713277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:58.713281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:58.713286Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1375 [2:7559342321035880420:10007]
2025-10-09T21:48:58.713300Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:58.713315Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:58.713319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:58.713322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:58.713324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:58.713339Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:58.713346Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:58.713357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:58.713408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.713412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.713414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.713417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:58.713470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:58.713475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.713487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.713499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.713515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:58.713533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342321035880415:10007]
2025-10-09T21:48:58.713538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342321035880415:10007]
2025-10-09T21:48:58.713540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342321035880415:10007]
2025-10-09T21:48:58.713546Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:58.713557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:58.713559Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:58.713562Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:58.713564Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:58.713565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:58.713567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:58.713575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342321397704872:3658], now have 1 active actors on pipe
2025-10-09T21:48:58.713578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342321397704672:2432], now have 1 active actors on pipe
2025-10-09T21:48:58.713594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342321035880415:10007]
2025-10-09T21:48:58.713602Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:58.713612Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:58.713637Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1375
2025-10-09T21:48:58.713696Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1375
2025-10-09T21:48:58.713730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.713741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.713767Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|53ea8eb8-c06913db-3f36ff65-14b0fc98_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:58.713776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:58.713779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:58.713781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:58.713790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:58.713803Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:58.713815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:58.714011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.714020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.714050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:58.714139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1376 partition: 0 MaxSeqNo: 25 sessionId: src_id|53ea8eb8-c06913db-3f36ff65-14b0fc98_0
2025-10-09T21:48:58.714444Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ce2bc34-7deed736-4ccb95cc-68120b0f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|53ea8eb8-c06913db-3f36ff65-14b0fc98_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:58.716759Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53ea8eb8-c06913db-3f36ff65-14b0fc98_0] Write session: try to update token
2025-10-09T21:48:58.716773Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53ea8eb8-c06913db-3f36ff65-14b0fc98_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:58.728585Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1376 sessionId: src_id|53ea8eb8-c06913db-3f36ff65-14b0fc98_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:58.730620Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:58.730639Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:58.730653Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342321035880415:10007]
2025-10-09T21:48:58.730698Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:58.730714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:58.730892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:58.730978Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:58.731071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:58.731283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:58.731490Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:58.731706Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:58.731751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.731763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.731780Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:58.731823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:58.731927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:58.737402Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:58.737418Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:58.737420Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:58.737422Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:58.737430Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342321035880415:10007]
2025-10-09T21:48:58.737433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342321035880415:10007]
2025-10-09T21:48:58.737440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342321035880415:10007]
2025-10-09T21:48:58.737480Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:58.737482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:58.737489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.737490Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:58.737491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:58.737493Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:58.737618Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:58.789366Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342321397704896:4303]
2025-10-09T21:48:58.789626Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 1375, pipe: [1:7559342321397704896:4303]
2025-10-09T21:48:58.884493Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:58.891057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.891081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.891222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:58.891279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:58.891337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:58.891395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:58.891593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:58.891602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:58.891662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:58.891717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:58.891772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:58.891832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:58.891870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:58.891875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:58.891928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:58.891980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:58.892040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:58.892097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:58.892141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:58.892149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:58.892206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:58.892365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:58.892429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:58.892487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:58.892525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:58.892531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:58.892598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:58.892661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:58.892730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:58.892798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:58.892837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:58.892845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:58.892903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:58.892963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:58.893018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:58.893077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:58.893118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:58.893124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:58.893174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:58.893224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:58.893280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:58.893334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:58.893372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:58.893376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:58.893430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:58.893482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:58.893536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:58.893591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:58.893629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:58.893634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:58.893700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:58.893752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:58.893805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:58.893861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:58.893897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:58.893903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:58.893957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:58.894009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:58.894080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:58.894136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:58.894182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:58.894192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:58.894236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342321397704896:4303], now have 1 active actors on pipe
2025-10-09T21:48:58.894481Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:58.894550Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342321397704896:4303]
2025-10-09T21:48:58.894555Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:58.894941Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:58.894954Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:58.894984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1376 reason: error at write: Disconnected sessionId: src_id|53ea8eb8-c06913db-3f36ff65-14b0fc98_0
2025-10-09T21:48:58.895152Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1376 sessionId: src_id|53ea8eb8-c06913db-3f36ff65-14b0fc98_0 is DEAD
2025-10-09T21:48:58.895271Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53ea8eb8-c06913db-3f36ff65-14b0fc98_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:58.895281Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53ea8eb8-c06913db-3f36ff65-14b0fc98_0] Write session will restart in 0.006194s
2025-10-09T21:48:58.895298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53ea8eb8-c06913db-3f36ff65-14b0fc98_0] Write session: Do CDS request
2025-10-09T21:48:58.895305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53ea8eb8-c06913db-3f36ff65-14b0fc98_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:58.902447Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53ea8eb8-c06913db-3f36ff65-14b0fc98_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:58.902532Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:58.902546Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1377
2025-10-09T21:48:58.902660Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1377 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:58.902692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1377 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:58.902699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1377 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:58.902707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:58.903161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1377 sessionId:  describe result for acl check
2025-10-09T21:48:58.903196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:58.903202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:58.903204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:58.903212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342321397704954:3633] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:58.903723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342321397704954:3633] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:58.906264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342321397704954:3633] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:58.908201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:58.910335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:58.910357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342321397704954:3633] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:58.910372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342321397704954:3633] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:58.910376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342321397704954:3633] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:58.910384Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1377 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:58.910519Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:58.910528Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:58.910540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:58.910588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:58.910624Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342321035880660:10021]
2025-10-09T21:48:58.911232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:58.911531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:58.911571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:58.911868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:58.911910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.911915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.911916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.911918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.911919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.911920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.911921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.911922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.911923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.911928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.911938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.911942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.911945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.911949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.911952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.911955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.911958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.911962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.911973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.911984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:58.911994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.912000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:58.912005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.912014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:58.912018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.912023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:58.912027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.912032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:58.912040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.912045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:58.912049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.912058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.912066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.912070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.912076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.912082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.912113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:58.918010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:58.919249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:58.919261Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:58.919263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:58.919267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:58.919273Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1376 [2:7559342321035880660:10021]
2025-10-09T21:48:58.919285Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:58.919299Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:58.919302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:58.919304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:58.919306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:58.919318Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:58.919321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:58.919329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:58.919379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.919386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.919388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.919392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:58.919441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:58.919446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.919453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:58.919459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:58.919469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:58.919483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342321035880655:10021]
2025-10-09T21:48:58.919488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342321035880655:10021]
2025-10-09T21:48:58.919490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342321035880655:10021]
2025-10-09T21:48:58.919495Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:58.919504Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:58.919506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:58.919506Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:58.919508Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:58.919519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:58.919522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:58.919534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342321397704910:2432], now have 1 active actors on pipe
2025-10-09T21:48:58.919542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342321397705110:3633], now have 1 active actors on pipe
2025-10-09T21:48:58.919561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342321035880655:10021]
2025-10-09T21:48:58.919571Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:58.919575Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:58.919599Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1376
2025-10-09T21:48:58.919636Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1376
2025-10-09T21:48:58.919666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.919675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.919700Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6509baa0-99d8bb43-2d074e04-f566901f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:58.919709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:58.919711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:58.919713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:58.919720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:58.919728Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:58.919742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:58.919852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.919858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.919870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:58.919962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1377 partition: 0 MaxSeqNo: 25 sessionId: src_id|6509baa0-99d8bb43-2d074e04-f566901f_0
2025-10-09T21:48:58.920262Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53ea8eb8-c06913db-3f36ff65-14b0fc98_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6509baa0-99d8bb43-2d074e04-f566901f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:58.922513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6509baa0-99d8bb43-2d074e04-f566901f_0] Write session: try to update token
2025-10-09T21:48:58.922530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6509baa0-99d8bb43-2d074e04-f566901f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:58.930799Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:48:58.930809Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:58.930816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342321035880655:10021]
2025-10-09T21:48:58.930854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:48:58.930869Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:48:58.931002Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:48:58.931138Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:58.931255Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:58.931369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:48:58.931528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:48:58.931558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:48:58.934898Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1377 sessionId: src_id|6509baa0-99d8bb43-2d074e04-f566901f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:58.937222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:58.937405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:58.937421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:58.937460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:48:58.937544Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:48:58.943693Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:58.943706Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:58.943709Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:58.943710Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:58.943722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342321035880655:10021]
2025-10-09T21:48:58.943725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342321035880655:10021]
2025-10-09T21:48:58.943728Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342321035880655:10021]
2025-10-09T21:48:58.943761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:58.943767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:58.943769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:58.943779Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:58.943786Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:58.943789Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:58.943883Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:58.994690Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342321397705121:4303]
2025-10-09T21:48:58.994996Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 1376, pipe: [1:7559342321397705121:4303]
2025-10-09T21:48:59.089406Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:59.096229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:59.096253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:59.096402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:59.096455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:59.096509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:59.096564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:59.096658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:59.096666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:59.096726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:59.096780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:59.096837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:59.096892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:59.096930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:59.096935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:59.096993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:59.097049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:59.097099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:59.097151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:59.097240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:59.097246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:59.097301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:59.097353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:59.097404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:59.097455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:59.097493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:59.097498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:59.097554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:59.097606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:59.097670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:59.097723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:59.097761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:59.097766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:59.097822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:59.097876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:59.097931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:59.097985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:59.098022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:59.098041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:59.098105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:59.098159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:59.098211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:59.098263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:59.098384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:59.098393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:59.098448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:59.098495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:59.098547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:59.098596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:59.098632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:59.098639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:59.098691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:59.098740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:59.098790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:59.098841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:59.098880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:59.098885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:59.098938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:48:59.098988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:48:59.099040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:48:59.099092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:48:59.099130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:48:59.099135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:48:59.099186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342321397705121:4303], now have 1 active actors on pipe
2025-10-09T21:48:59.099442Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342321397705121:4303]
2025-10-09T21:48:59.099456Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:59.099485Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:59.099488Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:59.099679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1377 reason: error at write: Disconnected sessionId: src_id|6509baa0-99d8bb43-2d074e04-f566901f_0
2025-10-09T21:48:59.099859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1377 sessionId: src_id|6509baa0-99d8bb43-2d074e04-f566901f_0 is DEAD
2025-10-09T21:48:59.099985Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:59.100029Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6509baa0-99d8bb43-2d074e04-f566901f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:59.100038Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6509baa0-99d8bb43-2d074e04-f566901f_0] Write session will restart in 0.007961s
2025-10-09T21:48:59.100055Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6509baa0-99d8bb43-2d074e04-f566901f_0] Write session: Do CDS request
2025-10-09T21:48:59.100061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6509baa0-99d8bb43-2d074e04-f566901f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:59.109501Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6509baa0-99d8bb43-2d074e04-f566901f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:59.109590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:59.109606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1378
2025-10-09T21:48:59.109752Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1378 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:59.109784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1378 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:59.109789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1378 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:59.109796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:59.110293Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1378 sessionId:  describe result for acl check
2025-10-09T21:48:59.110325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:59.110327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:59.110328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:59.110333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342325692672484:3628] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:59.110814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342325692672484:3628] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:59.113281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342325692672484:3628] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:59.115222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:59.117463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:59.117651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342325692672484:3628] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:59.117738Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:59.117752Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:59.117768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:59.117666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342325692672484:3628] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:59.117670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342325692672484:3628] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:59.117679Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1378 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:59.117840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:59.117885Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342325330848160:10023]
2025-10-09T21:48:59.118492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:59.118795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:59.118838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:59.119158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:59.119216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.119224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.119225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.119226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.119228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.119229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.119230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.119231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.119232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.119237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.119249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.119253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.119258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.119262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.119266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.119270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.119274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.119278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.119284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.119292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.119303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.119314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.119322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.119332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.119336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.119342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.119351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.119356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.119365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.119375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.119385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.119394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.119402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.119412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.119420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.119427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.119459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:59.125855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:59.127160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:59.127177Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:59.127180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:59.127185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:59.127192Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1377 [2:7559342325330848160:10023]
2025-10-09T21:48:59.127208Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:59.127228Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:59.127231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:59.127235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:59.127237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:59.127255Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:59.127269Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:59.127277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:59.127336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.127346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.127348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.127350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:59.127423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:59.127432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.127446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.127458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.127485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:59.127513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342325692672621:3628], now have 1 active actors on pipe
2025-10-09T21:48:59.127521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342325692672420:2432], now have 1 active actors on pipe
2025-10-09T21:48:59.127530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342325330848155:10023]
2025-10-09T21:48:59.127532Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342325330848155:10023]
2025-10-09T21:48:59.127534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342325330848155:10023]
2025-10-09T21:48:59.127548Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:59.127543Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1377
2025-10-09T21:48:59.127570Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:59.127549Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1377
2025-10-09T21:48:59.127579Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:59.127582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:59.127584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:59.127591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:59.127595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:59.127615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:59.127625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:59.127675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342325330848155:10023]
2025-10-09T21:48:59.127689Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:59.127697Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:59.127709Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b27eede6-f88bda13-88328bd4-63cd6cbf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:59.127718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:59.127728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:59.127732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:59.127743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:59.127760Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:59.127806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:59.127969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:59.127975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:59.127990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:59.128082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1378 partition: 0 MaxSeqNo: 25 sessionId: src_id|b27eede6-f88bda13-88328bd4-63cd6cbf_0
2025-10-09T21:48:59.128419Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6509baa0-99d8bb43-2d074e04-f566901f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b27eede6-f88bda13-88328bd4-63cd6cbf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:59.130891Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b27eede6-f88bda13-88328bd4-63cd6cbf_0] Write session: try to update token
2025-10-09T21:48:59.130911Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b27eede6-f88bda13-88328bd4-63cd6cbf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:59.141981Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:59.142001Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:59.142003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:59.142005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:59.142016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342325330848155:10023]
2025-10-09T21:48:59.142022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342325330848155:10023]
2025-10-09T21:48:59.142026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342325330848155:10023]
2025-10-09T21:48:59.142090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:59.142097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.142100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:59.142116Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:59.142124Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:59.142128Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:59.142232Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:59.143227Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1378 sessionId: src_id|b27eede6-f88bda13-88328bd4-63cd6cbf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:59.145524Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:59.199437Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342325692672636:4303]
2025-10-09T21:48:59.199686Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 1377, pipe: [1:7559342325692672636:4303]
2025-10-09T21:48:59.290714Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:59.290979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:59.290994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:59.291039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342325692672636:4303], now have 1 active actors on pipe
2025-10-09T21:48:59.291306Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:59.291290Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:59.291292Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:59.291306Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:59.291353Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342325692672636:4303]
2025-10-09T21:48:59.291367Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1378 reason: error at write: Disconnected sessionId: src_id|b27eede6-f88bda13-88328bd4-63cd6cbf_0
2025-10-09T21:48:59.291558Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1378 sessionId: src_id|b27eede6-f88bda13-88328bd4-63cd6cbf_0 is DEAD
2025-10-09T21:48:59.291690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b27eede6-f88bda13-88328bd4-63cd6cbf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:59.291702Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b27eede6-f88bda13-88328bd4-63cd6cbf_0] Write session will restart in 0.009777s
2025-10-09T21:48:59.291729Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b27eede6-f88bda13-88328bd4-63cd6cbf_0] Write session: Do CDS request
2025-10-09T21:48:59.291735Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b27eede6-f88bda13-88328bd4-63cd6cbf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:59.302496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b27eede6-f88bda13-88328bd4-63cd6cbf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:59.302578Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:59.302594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1379
2025-10-09T21:48:59.302704Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1379 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:59.302747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1379 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:59.302756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1379 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:59.302760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:59.303177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1379 sessionId:  describe result for acl check
2025-10-09T21:48:59.303218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:59.303226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:59.303228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:59.303234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342325692672761:3617] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:59.303836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342325692672761:3617] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:59.305700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342325692672761:3617] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:59.305927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:59.307825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:59.307985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342325692672761:3617] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:59.307994Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:59.307995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342325692672761:3617] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:59.308004Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:59.308016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:59.307998Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342325692672761:3617] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:59.308005Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1379 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:59.308066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:59.308106Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342325330848364:10025]
2025-10-09T21:48:59.308677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:59.308991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:59.309033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:59.309345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:59.309388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.309394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.309396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.309397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.309398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.309399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.309400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.309401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.309402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.309407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.309418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.309422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.309426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.309430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.309434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.309438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.309442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.309446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.309451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.309459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.309465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.309471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.309475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.309480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.309484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.309490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.309494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.309499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.309505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.309513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.309518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.309528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.309532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.309537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.309546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.309551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.309580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:59.315668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:59.316684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:59.316702Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:59.316704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:59.316709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:59.316715Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1378 [2:7559342325330848364:10025]
2025-10-09T21:48:59.316727Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:59.316743Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:59.316746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:59.316749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:59.316751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:59.316767Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:59.316775Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:59.316787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:59.316837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.316844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.316847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.316849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:59.316914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:59.316922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.316934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.316947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.316969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:59.316992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342325692672643:2432], now have 1 active actors on pipe
2025-10-09T21:48:59.317000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342325692672845:3617], now have 1 active actors on pipe
2025-10-09T21:48:59.317007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342325330848359:10025]
2025-10-09T21:48:59.317013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342325330848359:10025]
2025-10-09T21:48:59.317017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342325330848359:10025]
2025-10-09T21:48:59.317030Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:59.317039Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:59.317041Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:59.317043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:59.317049Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:59.317063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:59.317070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:59.317054Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1378
2025-10-09T21:48:59.317076Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1378
2025-10-09T21:48:59.317103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342325330848359:10025]
2025-10-09T21:48:59.317111Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:59.317114Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:59.317136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:59.317145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:59.317178Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|420842ad-9445fa9c-efcf5078-9c7f0057_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:59.317189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:59.317193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:59.317196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:59.317206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:59.317219Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:59.317244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:59.317409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:59.317418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:59.317434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:59.317528Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1379 partition: 0 MaxSeqNo: 25 sessionId: src_id|420842ad-9445fa9c-efcf5078-9c7f0057_0
2025-10-09T21:48:59.317887Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b27eede6-f88bda13-88328bd4-63cd6cbf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|420842ad-9445fa9c-efcf5078-9c7f0057_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:59.320255Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|420842ad-9445fa9c-efcf5078-9c7f0057_0] Write session: try to update token
2025-10-09T21:48:59.320274Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|420842ad-9445fa9c-efcf5078-9c7f0057_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:59.330836Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:59.330850Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:59.330852Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:59.330853Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:59.330860Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342325330848359:10025]
2025-10-09T21:48:59.330864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342325330848359:10025]
2025-10-09T21:48:59.330866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342325330848359:10025]
2025-10-09T21:48:59.330900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:59.330906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.330908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:59.330917Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:59.330925Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:59.330929Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:59.331025Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:59.332785Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1379 sessionId: src_id|420842ad-9445fa9c-efcf5078-9c7f0057_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:59.335112Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:59.391501Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342325692672857:4303]
2025-10-09T21:48:59.391784Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 1378, pipe: [1:7559342325692672857:4303]
2025-10-09T21:48:59.475328Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:59.475566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:59.475578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:59.475614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342325692672857:4303], now have 1 active actors on pipe
2025-10-09T21:48:59.475821Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342325692672857:4303]
2025-10-09T21:48:59.475834Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:59.475866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:59.475869Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:59.475949Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1379 reason: error at write: Disconnected sessionId: src_id|420842ad-9445fa9c-efcf5078-9c7f0057_0
2025-10-09T21:48:59.476003Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:59.476106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1379 sessionId: src_id|420842ad-9445fa9c-efcf5078-9c7f0057_0 is DEAD
2025-10-09T21:48:59.476230Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|420842ad-9445fa9c-efcf5078-9c7f0057_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:59.476242Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|420842ad-9445fa9c-efcf5078-9c7f0057_0] Write session will restart in 0.006931s
2025-10-09T21:48:59.476262Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|420842ad-9445fa9c-efcf5078-9c7f0057_0] Write session: Do CDS request
2025-10-09T21:48:59.476267Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|420842ad-9445fa9c-efcf5078-9c7f0057_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:59.484492Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|420842ad-9445fa9c-efcf5078-9c7f0057_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:59.484586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:59.484609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1380
2025-10-09T21:48:59.484739Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1380 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:59.484765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1380 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:59.484768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1380 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:59.484771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:59.485248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1380 sessionId:  describe result for acl check
2025-10-09T21:48:59.485285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:59.485292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:59.485294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:59.485300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342325692672920:3614] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:59.485827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342325692672920:3614] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:59.488196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342325692672920:3614] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:59.489652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:59.491651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:59.491795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342325692672920:3614] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:59.491808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342325692672920:3614] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:59.491812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342325692672920:3614] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:59.491817Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:59.491825Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1380 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:59.491828Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:59.491839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:59.491885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:59.491923Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342325330848569:10027]
2025-10-09T21:48:59.492394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:59.492638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:59.492681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:59.492935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:59.492979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.492988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.492989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.492990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.492992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.492993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.492994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.492995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.492996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.493001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.493010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.493015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.493019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.493023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.493027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.493030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.493034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.493038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.493044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.493052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.493058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.493068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.493077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.493083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.493093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.493098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.493102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.493107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.493111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.493116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.493121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.493131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.493135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.493140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.493147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.493152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.493184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:59.499057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:59.500213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:59.500227Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:59.500229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:59.500233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:59.500239Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1379 [2:7559342325330848569:10027]
2025-10-09T21:48:59.500250Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:59.500262Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:59.500264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:59.500266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:59.500267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:59.500279Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:59.500283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:59.500289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:59.500341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.500352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.500353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.500356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:59.500408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:59.500413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.500420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.500430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.500443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:59.500457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342325330848564:10027]
2025-10-09T21:48:59.500462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342325330848564:10027]
2025-10-09T21:48:59.500463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342325330848564:10027]
2025-10-09T21:48:59.500469Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:59.500479Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:59.500480Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:59.500484Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:59.500486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:59.500496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:59.500499Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:59.500527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342325692672859:2432], now have 1 active actors on pipe
2025-10-09T21:48:59.500535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342325692673062:3614], now have 1 active actors on pipe
2025-10-09T21:48:59.500538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342325330848564:10027]
2025-10-09T21:48:59.500545Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:59.500566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:59.500565Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1379
2025-10-09T21:48:59.500579Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1379
2025-10-09T21:48:59.500642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:59.500650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:59.500672Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c3cdb52c-e001aaaf-350babaa-710976b9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:59.500681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:59.500688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:59.500689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:59.500697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:59.500707Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:59.500723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:59.500856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:59.500863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:59.500876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:59.500957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1380 partition: 0 MaxSeqNo: 25 sessionId: src_id|c3cdb52c-e001aaaf-350babaa-710976b9_0
2025-10-09T21:48:59.501276Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|420842ad-9445fa9c-efcf5078-9c7f0057_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c3cdb52c-e001aaaf-350babaa-710976b9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:59.503654Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3cdb52c-e001aaaf-350babaa-710976b9_0] Write session: try to update token
2025-10-09T21:48:59.503672Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3cdb52c-e001aaaf-350babaa-710976b9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:59.514327Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:59.514344Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:59.514346Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:59.514348Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:59.514356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342325330848564:10027]
2025-10-09T21:48:59.514360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342325330848564:10027]
2025-10-09T21:48:59.514365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342325330848564:10027]
2025-10-09T21:48:59.514397Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:59.514402Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:59.514405Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:59.514407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:59.514415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.514417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:59.514546Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:59.516340Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1380 sessionId: src_id|c3cdb52c-e001aaaf-350babaa-710976b9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:59.518528Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:59.575819Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342325692673074:4303]
2025-10-09T21:48:59.576048Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 1379, pipe: [1:7559342325692673074:4303]
2025-10-09T21:48:59.656102Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:59.656330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:59.656344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:59.656381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342325692673074:4303], now have 1 active actors on pipe
2025-10-09T21:48:59.656537Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:59.656620Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:59.656634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:59.656639Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:59.656673Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342325692673074:4303]
2025-10-09T21:48:59.656708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1380 reason: error at write: Disconnected sessionId: src_id|c3cdb52c-e001aaaf-350babaa-710976b9_0
2025-10-09T21:48:59.656859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1380 sessionId: src_id|c3cdb52c-e001aaaf-350babaa-710976b9_0 is DEAD
2025-10-09T21:48:59.657008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3cdb52c-e001aaaf-350babaa-710976b9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:59.657019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3cdb52c-e001aaaf-350babaa-710976b9_0] Write session will restart in 0.006809s
2025-10-09T21:48:59.657046Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3cdb52c-e001aaaf-350babaa-710976b9_0] Write session: Do CDS request
2025-10-09T21:48:59.657052Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3cdb52c-e001aaaf-350babaa-710976b9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:59.664482Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3cdb52c-e001aaaf-350babaa-710976b9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:59.664588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:59.664601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1381
2025-10-09T21:48:59.664695Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1381 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:59.664728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1381 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:59.664735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1381 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:59.664738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:59.665163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1381 sessionId:  describe result for acl check
2025-10-09T21:48:59.665201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:59.665207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:59.665209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:59.665215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342325692673116:3611] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:59.665722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342325692673116:3611] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:59.668305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342325692673116:3611] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:59.670542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342325692673116:3611] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:59.670532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:59.670552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342325692673116:3611] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:59.670555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342325692673116:3611] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:59.670560Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1381 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:59.672458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:59.672623Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:59.672633Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:59.672643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:59.672693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:59.672729Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342325330848776:10029]
2025-10-09T21:48:59.673267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:59.673533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:59.673572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:59.673860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:59.673909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.673914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.673916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.673917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.673918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.673919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.673920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.673921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.673922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.673927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.673936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.673940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.673944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.673948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.673952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.673955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.673959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.673962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.673968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.673976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.673986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.673991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.673995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.674000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.674004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.674008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.674014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.674019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.674023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.674043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.674055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.674062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.674066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.674071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.674079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.674084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.674116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:59.679907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:59.680884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:59.680897Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:59.680899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:59.680903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:59.680910Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1380 [2:7559342325330848776:10029]
2025-10-09T21:48:59.680924Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:59.680937Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:59.680945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:59.680948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:59.680950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:59.680965Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:59.680975Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:59.680986Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:59.681038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.681044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.681046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.681049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:59.681108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:59.681115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.681126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.681141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.681161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:59.681184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342325692673279:3611], now have 1 active actors on pipe
2025-10-09T21:48:59.681191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342325692673076:2432], now have 1 active actors on pipe
2025-10-09T21:48:59.681198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342325330848770:10029]
2025-10-09T21:48:59.681203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342325330848770:10029]
2025-10-09T21:48:59.681205Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342325330848770:10029]
2025-10-09T21:48:59.681217Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:59.681230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:59.681232Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:59.681235Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:59.681223Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1380
2025-10-09T21:48:59.681237Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:59.681224Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1380
2025-10-09T21:48:59.681248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:59.681250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:59.681269Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342325330848770:10029]
2025-10-09T21:48:59.681276Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:59.681285Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:59.681294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:59.681299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:59.681349Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|99c225d5-5cfd779-68accd6-23c8fcb1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:59.681360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:59.681368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:59.681370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:59.681380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:59.681392Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:59.681411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:59.681554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:59.681562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:59.681576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:59.681678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1381 partition: 0 MaxSeqNo: 25 sessionId: src_id|99c225d5-5cfd779-68accd6-23c8fcb1_0
2025-10-09T21:48:59.681966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3cdb52c-e001aaaf-350babaa-710976b9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|99c225d5-5cfd779-68accd6-23c8fcb1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:59.684271Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99c225d5-5cfd779-68accd6-23c8fcb1_0] Write session: try to update token
2025-10-09T21:48:59.684287Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99c225d5-5cfd779-68accd6-23c8fcb1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:59.694602Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:59.694619Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:59.694622Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:59.694624Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:59.694633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342325330848770:10029]
2025-10-09T21:48:59.694638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342325330848770:10029]
2025-10-09T21:48:59.694642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342325330848770:10029]
2025-10-09T21:48:59.694684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:59.694691Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:59.694695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.694697Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:59.694698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:59.694701Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:59.694818Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:59.696493Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1381 sessionId: src_id|99c225d5-5cfd779-68accd6-23c8fcb1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:59.698854Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:59.757021Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342325692673292:4303]
2025-10-09T21:48:59.757263Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 1380, pipe: [1:7559342325692673292:4303]
2025-10-09T21:48:59.837059Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:48:59.837422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:59.837435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:59.837472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342325692673292:4303], now have 1 active actors on pipe
2025-10-09T21:48:59.837786Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342325692673292:4303]
2025-10-09T21:48:59.837813Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:48:59.837815Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:48:59.837855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1381 reason: error at write: Disconnected sessionId: src_id|99c225d5-5cfd779-68accd6-23c8fcb1_0
2025-10-09T21:48:59.837876Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:48:59.838002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1381 sessionId: src_id|99c225d5-5cfd779-68accd6-23c8fcb1_0 is DEAD
2025-10-09T21:48:59.838197Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99c225d5-5cfd779-68accd6-23c8fcb1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:48:59.838206Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99c225d5-5cfd779-68accd6-23c8fcb1_0] Write session will restart in 0.009837s
2025-10-09T21:48:59.838191Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:48:59.838228Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99c225d5-5cfd779-68accd6-23c8fcb1_0] Write session: Do CDS request
2025-10-09T21:48:59.838234Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99c225d5-5cfd779-68accd6-23c8fcb1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:48:59.849518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99c225d5-5cfd779-68accd6-23c8fcb1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:59.849588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:48:59.849606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1382
2025-10-09T21:48:59.849751Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1382 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:48:59.849789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1382 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:48:59.849794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1382 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:48:59.849803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:48:59.850275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1382 sessionId:  describe result for acl check
2025-10-09T21:48:59.850316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:48:59.850322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:48:59.850324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:48:59.850330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342325692673466:3594] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:48:59.850851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342325692673466:3594] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:48:59.852148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:48:59.852473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342325692673466:3594] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:48:59.854217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:48:59.854386Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:48:59.854393Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:48:59.854403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:48:59.854455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:48:59.854504Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342325330849024:10045]
2025-10-09T21:48:59.855073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342325692673466:3594] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:48:59.855085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342325692673466:3594] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:48:59.855107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:48:59.855087Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342325692673466:3594] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:48:59.855093Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1382 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:48:59.855351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:48:59.855400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:48:59.855669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:48:59.855708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.855710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.855711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.855712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.855713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.855715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.855716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.855717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.855718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.855723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.855733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.855737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.855741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.855745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.855749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.855753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.855761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.855764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.855769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.855778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:48:59.855785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.855795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:48:59.855802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.855807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:48:59.855811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.855820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:48:59.855824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.855827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:48:59.855829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.855832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:48:59.855836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.855840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.855843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.855846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.855853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.855857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.855872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:48:59.862174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:48:59.863535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:48:59.863548Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:48:59.863550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:48:59.863552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:48:59.863557Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1381 [2:7559342325330849024:10045]
2025-10-09T21:48:59.863566Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:48:59.863580Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:48:59.863582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:59.863584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:59.863585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:59.863598Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:48:59.863606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:48:59.863613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:48:59.863666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.863671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.863672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.863674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:48:59.863734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:48:59.863743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.863754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:48:59.863767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:48:59.863785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:48:59.863805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342325692673311:2432], now have 1 active actors on pipe
2025-10-09T21:48:59.863811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342325692673523:3594], now have 1 active actors on pipe
2025-10-09T21:48:59.863817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342325330849019:10045]
2025-10-09T21:48:59.863819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342325330849019:10045]
2025-10-09T21:48:59.863820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342325330849019:10045]
2025-10-09T21:48:59.863830Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:48:59.863841Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:48:59.863846Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:48:59.863849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:48:59.863852Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:48:59.863855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:48:59.863857Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:48:59.863877Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342325330849019:10045]
2025-10-09T21:48:59.863879Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:48:59.863881Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:48:59.863882Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1381
2025-10-09T21:48:59.863887Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1381
2025-10-09T21:48:59.864013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:59.864021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:59.864043Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4377d3a3-dfadaa8f-5c756a33-848630e8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:48:59.864051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:48:59.864054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:48:59.864055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:48:59.864063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:48:59.864071Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:48:59.864095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:59.864270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:48:59.864278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:48:59.864293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:48:59.864397Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1382 partition: 0 MaxSeqNo: 25 sessionId: src_id|4377d3a3-dfadaa8f-5c756a33-848630e8_0
2025-10-09T21:48:59.864734Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99c225d5-5cfd779-68accd6-23c8fcb1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4377d3a3-dfadaa8f-5c756a33-848630e8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:48:59.867066Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4377d3a3-dfadaa8f-5c756a33-848630e8_0] Write session: try to update token
2025-10-09T21:48:59.867084Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4377d3a3-dfadaa8f-5c756a33-848630e8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:48:59.877753Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:48:59.877766Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:48:59.877767Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:48:59.877769Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:48:59.877778Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342325330849019:10045]
2025-10-09T21:48:59.877783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342325330849019:10045]
2025-10-09T21:48:59.877786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342325330849019:10045]
2025-10-09T21:48:59.877825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:48:59.877836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:48:59.877837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:48:59.877865Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:48:59.877874Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:48:59.877879Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:48:59.877956Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:48:59.879564Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1382 sessionId: src_id|4377d3a3-dfadaa8f-5c756a33-848630e8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:48:59.881743Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:48:59.937752Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342325692673536:4303]
2025-10-09T21:48:59.938049Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 1381, pipe: [1:7559342325692673536:4303]
2025-10-09T21:49:00.022101Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:00.022349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.022360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.022402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342325692673536:4303], now have 1 active actors on pipe
2025-10-09T21:49:00.022581Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:00.022616Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342325692673536:4303]
2025-10-09T21:49:00.022618Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:00.022650Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:00.022715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1382 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4377d3a3-dfadaa8f-5c756a33-848630e8_0
2025-10-09T21:49:00.022882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1382 sessionId: src_id|4377d3a3-dfadaa8f-5c756a33-848630e8_0 is DEAD
2025-10-09T21:49:00.023053Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4377d3a3-dfadaa8f-5c756a33-848630e8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:00.023063Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4377d3a3-dfadaa8f-5c756a33-848630e8_0] Write session will restart in 0.006828s
2025-10-09T21:49:00.023107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4377d3a3-dfadaa8f-5c756a33-848630e8_0] Write session: Do CDS request
2025-10-09T21:49:00.023114Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4377d3a3-dfadaa8f-5c756a33-848630e8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:00.031552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4377d3a3-dfadaa8f-5c756a33-848630e8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:00.031603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:00.031621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1383
2025-10-09T21:49:00.031812Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1383 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:00.031846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1383 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:00.031856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1383 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:00.031860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:00.032307Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1383 sessionId:  describe result for acl check
2025-10-09T21:49:00.032349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:00.032356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:00.032358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:00.032366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342329987640866:3588] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:00.032917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342329987640866:3588] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:00.035627Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342329987640866:3588] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:00.038482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342329987640866:3588] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:00.038498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342329987640866:3588] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:00.038502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342329987640866:3588] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:00.038512Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1383 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:00.038942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:00.041069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:00.041239Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:00.041247Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:00.041257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:00.041307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:00.041338Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342329625816530:10047]
2025-10-09T21:49:00.041792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:00.042115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:00.042155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:00.042405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:00.042451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.042457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.042458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.042459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.042461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.042462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.042463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.042464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.042465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.042470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.042480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.042484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.042488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.042491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.042495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.042499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.042507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.042511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.042520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.042529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.042535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.042541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.042545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.042553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.042556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.042559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.042561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.042564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.042567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.042570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.042573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.042576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.042578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.042581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.042584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.042587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.042603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:00.049003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:00.049995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:00.050009Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:00.050013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:00.050017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:00.050025Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1382 [2:7559342329625816530:10047]
2025-10-09T21:49:00.050054Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:00.050071Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:00.050077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:00.050079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:00.050081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:00.050094Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:00.050102Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:00.050111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:00.050167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.050172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.050173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.050175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:00.050233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:00.050243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.050254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.050265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.050282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:00.050303Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342329625816525:10047]
2025-10-09T21:49:00.050311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342329625816525:10047]
2025-10-09T21:49:00.050314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342329625816525:10047]
2025-10-09T21:49:00.050323Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:00.050333Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:00.050335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342329987640837:2432], now have 1 active actors on pipe
2025-10-09T21:49:00.050336Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:00.050338Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:00.050340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342329987641038:3588], now have 1 active actors on pipe
2025-10-09T21:49:00.050344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:00.050353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:00.050356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:00.050387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342329625816525:10047]
2025-10-09T21:49:00.050399Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:00.050426Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:00.050404Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1382
2025-10-09T21:49:00.050404Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1382
2025-10-09T21:49:00.050459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.050470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.050497Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|faf45fa9-ddd8b55e-faa7af2b-3f0e7310_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:00.050505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:00.050511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:00.050513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:00.050524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:00.050533Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:00.050551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:00.050683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.050690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.050703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:00.050801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1383 partition: 0 MaxSeqNo: 25 sessionId: src_id|faf45fa9-ddd8b55e-faa7af2b-3f0e7310_0
2025-10-09T21:49:00.051191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4377d3a3-dfadaa8f-5c756a33-848630e8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|faf45fa9-ddd8b55e-faa7af2b-3f0e7310_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:00.053743Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|faf45fa9-ddd8b55e-faa7af2b-3f0e7310_0] Write session: try to update token
2025-10-09T21:49:00.053761Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|faf45fa9-ddd8b55e-faa7af2b-3f0e7310_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:00.062679Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:00.062700Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:00.062711Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342329625816525:10047]
2025-10-09T21:49:00.062761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:00.062765Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:00.062946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:00.063085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:00.063213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:00.063337Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:00.063511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:00.063547Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:00.067329Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1383 sessionId: src_id|faf45fa9-ddd8b55e-faa7af2b-3f0e7310_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:00.069547Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:00.069741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.069769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.069813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:00.069888Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:00.075357Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:00.075373Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:00.075375Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:00.075377Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:00.075389Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342329625816525:10047]
2025-10-09T21:49:00.075393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342329625816525:10047]
2025-10-09T21:49:00.075397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342329625816525:10047]
2025-10-09T21:49:00.075445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:00.075445Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:00.075454Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:00.075456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.075457Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:00.075459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:00.075598Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:00.122635Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342329987641051:4303]
2025-10-09T21:49:00.122918Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 1382, pipe: [1:7559342329987641051:4303]
2025-10-09T21:49:00.227037Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:00.233795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.233816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.233963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.234040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.234101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.234159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.234212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.234223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.234291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.234346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.234404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.234472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.234515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.234521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.234579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.234645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.234705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.234764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.234805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.234812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.234881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.234941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.235008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.235073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.235115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.235121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.235182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.235245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.235304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.235360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.235399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.235405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.235463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.235526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.235585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.235641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.235683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.235689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.235748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.235813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.235873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.235931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.235969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.235975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.236039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.236094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.236145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.236198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.236236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.236242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.236304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.236358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.236424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.236490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.236550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.236559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.236635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.236701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.236768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.236828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.236867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.236873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.236924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342329987641051:4303], now have 1 active actors on pipe
2025-10-09T21:49:00.237116Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:00.237119Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:00.237131Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:00.237152Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342329987641051:4303]
2025-10-09T21:49:00.237156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1383 reason: error at write: Disconnected sessionId: src_id|faf45fa9-ddd8b55e-faa7af2b-3f0e7310_0
2025-10-09T21:49:00.237347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1383 sessionId: src_id|faf45fa9-ddd8b55e-faa7af2b-3f0e7310_0 is DEAD
2025-10-09T21:49:00.237488Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|faf45fa9-ddd8b55e-faa7af2b-3f0e7310_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:00.237500Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|faf45fa9-ddd8b55e-faa7af2b-3f0e7310_0] Write session will restart in 0.005145s
2025-10-09T21:49:00.237525Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|faf45fa9-ddd8b55e-faa7af2b-3f0e7310_0] Write session: Do CDS request
2025-10-09T21:49:00.237531Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|faf45fa9-ddd8b55e-faa7af2b-3f0e7310_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:00.237777Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:00.243504Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|faf45fa9-ddd8b55e-faa7af2b-3f0e7310_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:00.243505Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:00.243518Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1384
2025-10-09T21:49:00.243744Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1384 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:00.243788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1384 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:00.243794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1384 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:00.243797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:00.244185Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1384 sessionId:  describe result for acl check
2025-10-09T21:49:00.244237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:00.244244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:00.244246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:00.244252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342329987641081:3580] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:00.244671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342329987641081:3580] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:00.247241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342329987641081:3580] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:00.250388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342329987641081:3580] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:00.250404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342329987641081:3580] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:00.250408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342329987641081:3580] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:00.250416Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1384 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:00.253124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:00.255222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:00.255409Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:00.255419Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:00.255431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:00.255492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:00.255533Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342329625816739:10049]
2025-10-09T21:49:00.256124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:00.256427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:00.256466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:00.256779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:00.256825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.256831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.256832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.256834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.256835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.256836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.256838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.256839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.256840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.256845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.256857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.256861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.256865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.256869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.256874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.256877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.256882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.256886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.256892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.256902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.256908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.256917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.256922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.256926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.256930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.256934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.256939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.256944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.256949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.256954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.256959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.256965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.256974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.256979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.256984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.256989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.257018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:00.263397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:00.264578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:00.264591Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:00.264593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:00.264597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:00.264604Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1383 [2:7559342329625816739:10049]
2025-10-09T21:49:00.264619Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:00.264636Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:00.264644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:00.264648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:00.264650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:00.264666Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:00.264672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:00.264679Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:00.264731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.264738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.264741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.264744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:00.264811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:00.264818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.264825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.264831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.264843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:00.264865Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342329625816734:10049]
2025-10-09T21:49:00.264871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342329625816734:10049]
2025-10-09T21:49:00.264873Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342329625816734:10049]
2025-10-09T21:49:00.264882Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:00.264892Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:00.264895Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:00.264897Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:00.264898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342329987641224:3580], now have 1 active actors on pipe
2025-10-09T21:49:00.264904Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:00.264912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:00.264915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:00.264919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342329987641061:2432], now have 1 active actors on pipe
2025-10-09T21:49:00.264941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342329625816734:10049]
2025-10-09T21:49:00.264956Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:00.264966Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:00.264951Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1383
2025-10-09T21:49:00.265038Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1383
2025-10-09T21:49:00.265100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.265111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.265137Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7fb409d4-506c5f02-a13b0044-a6d92f85_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:00.265150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:00.265155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:00.265162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:00.265172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:00.265183Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:00.265202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:00.265376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.265384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.265397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:00.265472Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1384 partition: 0 MaxSeqNo: 25 sessionId: src_id|7fb409d4-506c5f02-a13b0044-a6d92f85_0
2025-10-09T21:49:00.265852Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|faf45fa9-ddd8b55e-faa7af2b-3f0e7310_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7fb409d4-506c5f02-a13b0044-a6d92f85_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:00.268251Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7fb409d4-506c5f02-a13b0044-a6d92f85_0] Write session: try to update token
2025-10-09T21:49:00.268270Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7fb409d4-506c5f02-a13b0044-a6d92f85_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:00.276130Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:00.276144Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:00.276153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342329625816734:10049]
2025-10-09T21:49:00.276193Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:00.276196Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:00.276345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:00.276485Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:00.276602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:00.276722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:00.276890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:00.276923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:00.280839Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1384 sessionId: src_id|7fb409d4-506c5f02-a13b0044-a6d92f85_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:00.283243Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:00.283465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.283484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.283524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:00.283603Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:00.289089Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:00.289102Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:00.289104Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:00.289106Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:00.289116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342329625816734:10049]
2025-10-09T21:49:00.289120Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342329625816734:10049]
2025-10-09T21:49:00.289123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342329625816734:10049]
2025-10-09T21:49:00.289156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:00.289161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.289163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:00.289166Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:00.289171Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:00.289176Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:00.289298Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:00.337147Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342329987641286:4303]
2025-10-09T21:49:00.337416Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 1383, pipe: [1:7559342329987641286:4303]
2025-10-09T21:49:00.433483Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:00.440299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.440323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.440456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.440507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.440568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.440624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.440662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.440671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.440728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.440780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.440841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.440894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.440930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.440936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.440991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.441038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.441094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.441143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.441180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.441188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.441253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.441314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.441368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.441423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.441461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.441466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.441587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.441704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.441800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.441857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.441895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.441899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.441951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.442009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.442081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.442133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.442169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.442175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.442224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.442272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.442326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.442381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.442417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.442423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.442478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.442525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.442571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.442628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.442665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.442671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.442719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.442762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.442811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.442867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.442905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.442910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.442959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:00.443006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:00.443051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:00.443104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:00.443144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:00.443149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:00.443185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342329987641286:4303], now have 1 active actors on pipe
2025-10-09T21:49:00.443378Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342329987641286:4303]
2025-10-09T21:49:00.443377Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:00.443393Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:00.443411Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:00.443418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1384 reason: error at write: Disconnected sessionId: src_id|7fb409d4-506c5f02-a13b0044-a6d92f85_0
2025-10-09T21:49:00.443572Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1384 sessionId: src_id|7fb409d4-506c5f02-a13b0044-a6d92f85_0 is DEAD
2025-10-09T21:49:00.443692Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fb409d4-506c5f02-a13b0044-a6d92f85_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:00.443700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fb409d4-506c5f02-a13b0044-a6d92f85_0] Write session will restart in 0.006108s
2025-10-09T21:49:00.443718Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fb409d4-506c5f02-a13b0044-a6d92f85_0] Write session: Do CDS request
2025-10-09T21:49:00.443722Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fb409d4-506c5f02-a13b0044-a6d92f85_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:00.443926Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:00.450452Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7fb409d4-506c5f02-a13b0044-a6d92f85_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:00.450546Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:00.450560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1385
2025-10-09T21:49:00.450887Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1385 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:00.450920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1385 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:00.450930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1385 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:00.450937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:00.451266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1385 sessionId:  describe result for acl check
2025-10-09T21:49:00.451296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:00.451301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:00.451303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:00.451318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342329987641313:3582] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:00.451838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342329987641313:3582] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:00.454541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342329987641313:3582] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:00.456967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342329987641313:3582] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:00.456983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342329987641313:3582] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:00.456986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342329987641313:3582] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:00.456995Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1385 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:00.457882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:00.460131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:00.460306Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:00.460317Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:00.460328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:00.460374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:00.460415Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342329625816950:10051]
2025-10-09T21:49:00.460989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:00.461251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:00.461293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:00.461599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:00.461653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.461659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.461661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.461662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.461663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.461664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.461665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.461666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.461668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.461673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.461683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.461688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.461692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.461696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.461701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.461705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.461708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.461712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.461719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.461727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.461733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.461739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.461743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.461749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.461753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.461758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.461762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.461767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.461771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.461777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.461783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.461790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.461795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.461800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.461804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.461810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.461836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:00.468063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:00.469179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:00.469193Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:00.469195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:00.469199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:00.469205Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1384 [2:7559342329625816950:10051]
2025-10-09T21:49:00.469217Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:00.469232Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:00.469239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:00.469242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:00.469245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:00.469261Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:00.469272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:00.469282Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:00.469339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.469348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.469350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.469353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:00.469407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:00.469411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.469423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.469446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.469464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:00.469485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342329987641494:3582], now have 1 active actors on pipe
2025-10-09T21:49:00.469493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342329987641289:2432], now have 1 active actors on pipe
2025-10-09T21:49:00.469500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342329625816945:10051]
2025-10-09T21:49:00.469503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342329625816945:10051]
2025-10-09T21:49:00.469505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342329625816945:10051]
2025-10-09T21:49:00.469519Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:00.469535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:00.469543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:00.469544Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:00.469545Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:00.469547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:00.469549Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:00.469563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342329625816945:10051]
2025-10-09T21:49:00.469566Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:00.469545Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1384
2025-10-09T21:49:00.469574Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:00.469547Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1384
2025-10-09T21:49:00.469583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.469586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.469614Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e1ca7429-c396a640-1a0a5b47-b259a563_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:00.469619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:00.469623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:00.469625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:00.469635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:00.469660Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:00.469683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:00.469811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.469821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.469838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:00.469937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1385 partition: 0 MaxSeqNo: 25 sessionId: src_id|e1ca7429-c396a640-1a0a5b47-b259a563_0
2025-10-09T21:49:00.470284Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fb409d4-506c5f02-a13b0044-a6d92f85_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e1ca7429-c396a640-1a0a5b47-b259a563_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:00.472690Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ca7429-c396a640-1a0a5b47-b259a563_0] Write session: try to update token
2025-10-09T21:49:00.472708Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ca7429-c396a640-1a0a5b47-b259a563_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:00.483434Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:00.483451Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:00.483454Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:00.483456Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:00.483465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342329625816945:10051]
2025-10-09T21:49:00.483469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342329625816945:10051]
2025-10-09T21:49:00.483474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342329625816945:10051]
2025-10-09T21:49:00.483514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:00.483514Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:00.483518Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:00.483522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.483523Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:00.483525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:00.483652Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:00.485090Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1385 sessionId: src_id|e1ca7429-c396a640-1a0a5b47-b259a563_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:00.487315Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:00.543500Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342329987641509:4303]
2025-10-09T21:49:00.543823Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 1384, pipe: [1:7559342329987641509:4303]
2025-10-09T21:49:00.627984Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:00.628451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.628462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.628502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342329987641509:4303], now have 1 active actors on pipe
2025-10-09T21:49:00.628704Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:00.628821Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342329987641509:4303]
2025-10-09T21:49:00.628839Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:00.628843Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:00.628862Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:00.628934Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1385 reason: error at write: Disconnected sessionId: src_id|e1ca7429-c396a640-1a0a5b47-b259a563_0
2025-10-09T21:49:00.629102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1385 sessionId: src_id|e1ca7429-c396a640-1a0a5b47-b259a563_0 is DEAD
2025-10-09T21:49:00.629271Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ca7429-c396a640-1a0a5b47-b259a563_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:00.629285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ca7429-c396a640-1a0a5b47-b259a563_0] Write session will restart in 0.007259s
2025-10-09T21:49:00.629312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ca7429-c396a640-1a0a5b47-b259a563_0] Write session: Do CDS request
2025-10-09T21:49:00.629319Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ca7429-c396a640-1a0a5b47-b259a563_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:00.637450Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ca7429-c396a640-1a0a5b47-b259a563_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:00.637552Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:00.637568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1386
2025-10-09T21:49:00.637716Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1386 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:00.637749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1386 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:00.637753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1386 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:00.637757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:00.638229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1386 sessionId:  describe result for acl check
2025-10-09T21:49:00.638274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:00.638285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:00.638286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:00.638292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342329987641536:3573] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:00.638828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342329987641536:3573] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:00.641422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342329987641536:3573] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:00.644064Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342329987641536:3573] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:00.644077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342329987641536:3573] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:00.644080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342329987641536:3573] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:00.644086Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1386 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:00.646070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:00.648201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:00.648388Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:00.648396Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:00.648404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:00.648443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:00.648477Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342329625817161:10053]
2025-10-09T21:49:00.648906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:00.649212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:00.649257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:00.649558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:00.649624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.649630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.649631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.649633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.649634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.649635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.649636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.649638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.649648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.649654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.649664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.649668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.649672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.649676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.649680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.649685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.649688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.649692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.649699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.649707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.649713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.649718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.649723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.649728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.649733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.649738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.649746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.649751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.649756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.649761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.649766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.649771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.649779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.649784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.649789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.649795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.649826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:00.656086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:00.657445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:00.657460Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:00.657462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:00.657467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:00.657473Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1385 [2:7559342329625817161:10053]
2025-10-09T21:49:00.657486Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:00.657504Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:00.657512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:00.657515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:00.657518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:00.657534Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:00.657545Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:00.657555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:00.657613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.657620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.657622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.657625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:00.657702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:00.657710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.657721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.657734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.657751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:00.657789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342329987641701:3573], now have 1 active actors on pipe
2025-10-09T21:49:00.657799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342329987641511:2432], now have 1 active actors on pipe
2025-10-09T21:49:00.657807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342329625817156:10053]
2025-10-09T21:49:00.657810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342329625817156:10053]
2025-10-09T21:49:00.657812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342329625817156:10053]
2025-10-09T21:49:00.657825Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:00.657842Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:00.657852Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:00.657855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:00.657857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:00.657857Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:00.657861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:00.657888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342329625817156:10053]
2025-10-09T21:49:00.657896Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:00.657853Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1385
2025-10-09T21:49:00.657899Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:00.657914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.657918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.657921Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1385
2025-10-09T21:49:00.657964Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7846c941-13f4acfe-86494493-438d1462_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:00.657976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:00.657980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:00.657983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:00.657993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:00.658003Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:00.658026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:00.658197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.658205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.658218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:00.658286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1386 partition: 0 MaxSeqNo: 25 sessionId: src_id|7846c941-13f4acfe-86494493-438d1462_0
2025-10-09T21:49:00.658642Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1ca7429-c396a640-1a0a5b47-b259a563_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7846c941-13f4acfe-86494493-438d1462_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:00.660987Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7846c941-13f4acfe-86494493-438d1462_0] Write session: try to update token
2025-10-09T21:49:00.661005Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7846c941-13f4acfe-86494493-438d1462_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:00.671978Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:00.671997Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:00.671999Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:00.672001Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:00.672012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342329625817156:10053]
2025-10-09T21:49:00.672017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342329625817156:10053]
2025-10-09T21:49:00.672021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342329625817156:10053]
2025-10-09T21:49:00.672066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:00.672066Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:00.672073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.672073Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:00.672076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:00.672077Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:00.672200Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:00.673302Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1386 sessionId: src_id|7846c941-13f4acfe-86494493-438d1462_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:00.675698Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:00.728822Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342329987641732:4303]
2025-10-09T21:49:00.729098Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 1385, pipe: [1:7559342329987641732:4303]
2025-10-09T21:49:00.817613Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:00.817849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.817859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.817895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342329987641732:4303], now have 1 active actors on pipe
2025-10-09T21:49:00.818053Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:00.818119Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342329987641732:4303]
2025-10-09T21:49:00.818149Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:00.818174Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:00.818315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1386 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7846c941-13f4acfe-86494493-438d1462_0
2025-10-09T21:49:00.818510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1386 sessionId: src_id|7846c941-13f4acfe-86494493-438d1462_0 is DEAD
2025-10-09T21:49:00.818676Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7846c941-13f4acfe-86494493-438d1462_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:00.818689Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7846c941-13f4acfe-86494493-438d1462_0] Write session will restart in 0.005031s
2025-10-09T21:49:00.818717Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7846c941-13f4acfe-86494493-438d1462_0] Write session: Do CDS request
2025-10-09T21:49:00.818735Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7846c941-13f4acfe-86494493-438d1462_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:00.824496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7846c941-13f4acfe-86494493-438d1462_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:00.824608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:00.824625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1387
2025-10-09T21:49:00.824790Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1387 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:00.824833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1387 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:00.824838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1387 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:00.824843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:00.825375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1387 sessionId:  describe result for acl check
2025-10-09T21:49:00.825422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:00.825428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:00.825430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:00.825437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342329987641749:3570] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:00.826083Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342329987641749:3570] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:00.828312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342329987641749:3570] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:00.831276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342329987641749:3570] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:00.831291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342329987641749:3570] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:00.831293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342329987641749:3570] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:00.831300Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1387 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:00.833309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:00.835804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:00.836030Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:00.836042Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:00.836055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:00.836113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:00.836144Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342329625817390:10063]
2025-10-09T21:49:00.836536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:00.836879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:00.836938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:00.837217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:00.837263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.837270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.837271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.837272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.837273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.837274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.837275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.837276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.837277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.837282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.837293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.837296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.837300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.837304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.837308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.837312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.837315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.837319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.837326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.837335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:00.837341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.837346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:00.837351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.837355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:00.837360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.837365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:00.837369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.837375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:00.837379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.837385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:00.837390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.837402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.837411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.837421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.837430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.837438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.837462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:00.843857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:00.844849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:00.844863Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:00.844865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:00.844868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:00.844872Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1386 [2:7559342329625817390:10063]
2025-10-09T21:49:00.844882Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:00.844897Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:00.844898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:00.844901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:00.844902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:00.844916Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:00.844919Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:00.844925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:00.844987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.844995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.844997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.844999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:00.845171Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1386
2025-10-09T21:49:00.845204Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1386
2025-10-09T21:49:00.845241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:00.845248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.845257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:00.845264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:00.845275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:00.845292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342329987641925:3570], now have 1 active actors on pipe
2025-10-09T21:49:00.845298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342329987641734:2432], now have 1 active actors on pipe
2025-10-09T21:49:00.845311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.845317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.845335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342329625817385:10063]
2025-10-09T21:49:00.845339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342329625817385:10063]
2025-10-09T21:49:00.845341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342329625817385:10063]
2025-10-09T21:49:00.845350Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:00.845361Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:00.845370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:00.845373Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:00.845375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:00.845380Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:00.845382Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:00.845388Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|78813bc7-c077836d-dd0afa64-3d63e9a0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:00.845394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:00.845397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:00.845399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:00.845405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:00.845413Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:00.845444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342329625817385:10063]
2025-10-09T21:49:00.845454Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:00.845463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:00.845478Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:00.845677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.845690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.845710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:00.845815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1387 partition: 0 MaxSeqNo: 25 sessionId: src_id|78813bc7-c077836d-dd0afa64-3d63e9a0_0
2025-10-09T21:49:00.846193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7846c941-13f4acfe-86494493-438d1462_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|78813bc7-c077836d-dd0afa64-3d63e9a0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:00.848565Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78813bc7-c077836d-dd0afa64-3d63e9a0_0] Write session: try to update token
2025-10-09T21:49:00.848586Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78813bc7-c077836d-dd0afa64-3d63e9a0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:00.862754Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:00.862772Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:00.862783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342329625817385:10063]
2025-10-09T21:49:00.862832Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:00.862835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:00.863021Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:00.863172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:00.863312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:00.863451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:00.864081Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:00.864124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:00.864115Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1387 sessionId: src_id|78813bc7-c077836d-dd0afa64-3d63e9a0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:00.866343Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:00.866675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:00.866688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:00.866717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:00.866789Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:00.870085Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:00.870096Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:00.870097Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:00.870099Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:00.870106Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342329625817385:10063]
2025-10-09T21:49:00.870123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342329625817385:10063]
2025-10-09T21:49:00.870126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342329625817385:10063]
2025-10-09T21:49:00.870164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:00.870170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:00.870173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:00.870179Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:00.870184Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:00.870188Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:00.870300Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:00.918278Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342329987641980:4303]
2025-10-09T21:49:00.918573Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 1386, pipe: [1:7559342329987641980:4303]
2025-10-09T21:49:01.018989Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:01.025598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.025622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.025775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.025828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.025884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.025940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.025979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.025987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.026068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.026122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.026176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.026233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.026272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.026277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.026331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.026382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.026434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.026485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.026525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.026531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.026588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.026640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.026693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.026748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.026789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.026791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.026844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.026898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.026950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.027004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.027061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.027067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.027121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.027172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.027225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.027276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.027315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.027320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.027371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.027420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.027470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.027518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.027554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.027561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.027610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.027660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.027712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.027763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.027799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.027805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.027857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.027904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.027955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.028006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.028042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.028047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.028095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.028143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.028194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.028245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.028280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.028286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.028322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342329987641980:4303], now have 1 active actors on pipe
2025-10-09T21:49:01.028591Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:01.028625Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:01.028629Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:01.028641Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342329987641980:4303]
2025-10-09T21:49:01.028659Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1387 reason: error at write: Disconnected sessionId: src_id|78813bc7-c077836d-dd0afa64-3d63e9a0_0
2025-10-09T21:49:01.028831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1387 sessionId: src_id|78813bc7-c077836d-dd0afa64-3d63e9a0_0 is DEAD
2025-10-09T21:49:01.028988Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78813bc7-c077836d-dd0afa64-3d63e9a0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:01.029000Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78813bc7-c077836d-dd0afa64-3d63e9a0_0] Write session will restart in 0.006927s
2025-10-09T21:49:01.029039Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78813bc7-c077836d-dd0afa64-3d63e9a0_0] Write session: Do CDS request
2025-10-09T21:49:01.029045Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78813bc7-c077836d-dd0afa64-3d63e9a0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:01.029090Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:01.037527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78813bc7-c077836d-dd0afa64-3d63e9a0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:01.037565Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:01.037578Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1388
2025-10-09T21:49:01.037741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1388 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:01.037769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1388 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:01.037776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1388 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:01.037779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:01.038153Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1388 sessionId:  describe result for acl check
2025-10-09T21:49:01.038184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:01.038187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:01.038189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:01.038194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342334282609331:3543] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:01.038630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342334282609331:3543] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:01.041111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342334282609331:3543] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:01.044216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342334282609331:3543] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:01.044231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342334282609331:3543] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:01.044234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342334282609331:3543] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:01.044242Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1388 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:01.044400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:01.046556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:01.046737Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:01.046749Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:01.046761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:01.046816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:01.046857Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342333920784924:10071]
2025-10-09T21:49:01.047436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:01.047746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:01.047789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:01.048108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:01.048159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.048165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.048167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.048168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.048169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.048170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.048172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.048173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.048175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.048179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.048191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.048195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.048200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.048204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.048212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.048217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.048225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.048229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.048239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.048251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.048262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.048268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.048273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.048283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.048287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.048296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.048301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.048306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.048310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.048315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.048320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.048325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.048329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.048333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.048338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.048343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.048369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:01.054732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:01.055739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:01.055754Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:01.055757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:01.055761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:01.055767Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1387 [2:7559342333920784924:10071]
2025-10-09T21:49:01.055779Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:01.055796Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:01.055804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:01.055807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:01.055809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:01.055827Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:01.055838Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:01.055846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:01.055904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.055911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.055913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.055916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:01.055979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:01.055987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.055998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.056014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.056032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:01.056056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342334282609488:3543], now have 1 active actors on pipe
2025-10-09T21:49:01.056064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342334282609281:2432], now have 1 active actors on pipe
2025-10-09T21:49:01.056071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342333920784918:10071]
2025-10-09T21:49:01.056073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342333920784918:10071]
2025-10-09T21:49:01.056076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342333920784918:10071]
2025-10-09T21:49:01.056088Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:01.056107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:01.056118Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:01.056126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:01.056128Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:01.056129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:01.056131Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:01.056126Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1387
2025-10-09T21:49:01.056154Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342333920784918:10071]
2025-10-09T21:49:01.056128Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1387
2025-10-09T21:49:01.056162Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:01.056164Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:01.056189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.056198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.056231Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ee988a9f-328124d9-1e393246-7510e3d3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:01.056242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:01.056263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:01.056270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:01.056281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:01.056302Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:01.056326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:01.056564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.056577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.056608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:01.056700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1388 partition: 0 MaxSeqNo: 25 sessionId: src_id|ee988a9f-328124d9-1e393246-7510e3d3_0
2025-10-09T21:49:01.057010Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78813bc7-c077836d-dd0afa64-3d63e9a0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ee988a9f-328124d9-1e393246-7510e3d3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:01.059413Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee988a9f-328124d9-1e393246-7510e3d3_0] Write session: try to update token
2025-10-09T21:49:01.059430Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee988a9f-328124d9-1e393246-7510e3d3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:01.070330Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:01.070346Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:01.070348Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:01.070349Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:01.070356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342333920784918:10071]
2025-10-09T21:49:01.070360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342333920784918:10071]
2025-10-09T21:49:01.070362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342333920784918:10071]
2025-10-09T21:49:01.070399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:01.070405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.070412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:01.070414Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:01.070419Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:01.070421Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:01.070533Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:01.072461Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1388 sessionId: src_id|ee988a9f-328124d9-1e393246-7510e3d3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:01.074780Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:01.128692Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342334282609503:4303]
2025-10-09T21:49:01.128940Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 1387, pipe: [1:7559342334282609503:4303]
2025-10-09T21:49:01.217479Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:01.217969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.217982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.218021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342334282609503:4303], now have 1 active actors on pipe
2025-10-09T21:49:01.218229Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:01.218283Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:01.218298Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:01.218300Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342334282609503:4303]
2025-10-09T21:49:01.218321Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:01.218348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1388 reason: error at write: Disconnected sessionId: src_id|ee988a9f-328124d9-1e393246-7510e3d3_0
2025-10-09T21:49:01.218509Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1388 sessionId: src_id|ee988a9f-328124d9-1e393246-7510e3d3_0 is DEAD
2025-10-09T21:49:01.218654Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee988a9f-328124d9-1e393246-7510e3d3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:01.218663Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee988a9f-328124d9-1e393246-7510e3d3_0] Write session will restart in 0.007156s
2025-10-09T21:49:01.218684Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee988a9f-328124d9-1e393246-7510e3d3_0] Write session: Do CDS request
2025-10-09T21:49:01.218689Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee988a9f-328124d9-1e393246-7510e3d3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:01.226444Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee988a9f-328124d9-1e393246-7510e3d3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:01.226549Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:01.226566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1389
2025-10-09T21:49:01.226671Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1389 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:01.226703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1389 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:01.226714Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1389 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:01.226717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:01.227133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1389 sessionId:  describe result for acl check
2025-10-09T21:49:01.227166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:01.227169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:01.227170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:01.227174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342334282609545:3544] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:01.227612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342334282609545:3544] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:01.229919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342334282609545:3544] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:01.232209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342334282609545:3544] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:01.232220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342334282609545:3544] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:01.232222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342334282609545:3544] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:01.232228Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1389 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:01.233132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:01.235111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:01.235273Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:01.235282Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:01.235292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:01.235332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:01.235366Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342333920785138:10073]
2025-10-09T21:49:01.235805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:01.236091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:01.236127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:01.236417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:01.236471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.236477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.236478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.236479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.236480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.236481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.236483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.236485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.236486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.236491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.236502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.236506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.236510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.236514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.236520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.236524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.236528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.236532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.236538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.236551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.236556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.236565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.236570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.236580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.236588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.236593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.236602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.236607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.236617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.236624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.236633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.236645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.236654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.236661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.236666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.236671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.236700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:01.243124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:01.244163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:01.244178Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:01.244181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:01.244186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:01.244193Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1388 [2:7559342333920785138:10073]
2025-10-09T21:49:01.244208Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:01.244226Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:01.244234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:01.244237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:01.244239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:01.244257Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:01.244268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:01.244281Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:01.244338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.244345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.244347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.244350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:01.244415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:01.244423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.244435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.244451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.244471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:01.244496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342334282609508:2432], now have 1 active actors on pipe
2025-10-09T21:49:01.244504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342334282609720:3544], now have 1 active actors on pipe
2025-10-09T21:49:01.244512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342333920785133:10073]
2025-10-09T21:49:01.244520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342333920785133:10073]
2025-10-09T21:49:01.244522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342333920785133:10073]
2025-10-09T21:49:01.244538Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:01.244557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:01.244568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:01.244573Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:01.244543Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1388
2025-10-09T21:49:01.244575Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:01.244577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:01.244580Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:01.244610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342333920785133:10073]
2025-10-09T21:49:01.244613Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:01.244621Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:01.244552Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1388
2025-10-09T21:49:01.244646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.244656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.244685Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bb204fea-8894fb7a-86094f73-dbc91993_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:01.244696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:01.244700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:01.244703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:01.244715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:01.244730Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:01.244749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:01.244897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.244906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.244921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:01.245021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1389 partition: 0 MaxSeqNo: 25 sessionId: src_id|bb204fea-8894fb7a-86094f73-dbc91993_0
2025-10-09T21:49:01.245355Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee988a9f-328124d9-1e393246-7510e3d3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bb204fea-8894fb7a-86094f73-dbc91993_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:01.247660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb204fea-8894fb7a-86094f73-dbc91993_0] Write session: try to update token
2025-10-09T21:49:01.247678Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb204fea-8894fb7a-86094f73-dbc91993_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:01.258742Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:01.258763Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:01.258765Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:01.258767Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:01.258777Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342333920785133:10073]
2025-10-09T21:49:01.258782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342333920785133:10073]
2025-10-09T21:49:01.258786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342333920785133:10073]
2025-10-09T21:49:01.258832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:01.258831Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:01.258845Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:01.258846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.258849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:01.258849Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:01.258977Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:01.259964Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1389 sessionId: src_id|bb204fea-8894fb7a-86094f73-dbc91993_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:01.262149Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:01.318654Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342334282609736:4303]
2025-10-09T21:49:01.318920Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 1388, pipe: [1:7559342334282609736:4303]
2025-10-09T21:49:01.405620Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:01.406231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.406245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.406289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342334282609736:4303], now have 1 active actors on pipe
2025-10-09T21:49:01.406540Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342334282609736:4303]
2025-10-09T21:49:01.406542Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:01.406578Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:01.406580Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:01.406647Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1389 reason: error at write: Disconnected sessionId: src_id|bb204fea-8894fb7a-86094f73-dbc91993_0
2025-10-09T21:49:01.406818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1389 sessionId: src_id|bb204fea-8894fb7a-86094f73-dbc91993_0 is DEAD
2025-10-09T21:49:01.406967Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb204fea-8894fb7a-86094f73-dbc91993_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:01.406978Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb204fea-8894fb7a-86094f73-dbc91993_0] Write session will restart in 0.009492s
2025-10-09T21:49:01.406961Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:01.407002Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb204fea-8894fb7a-86094f73-dbc91993_0] Write session: Do CDS request
2025-10-09T21:49:01.407021Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb204fea-8894fb7a-86094f73-dbc91993_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:01.417507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb204fea-8894fb7a-86094f73-dbc91993_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:01.417527Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:01.417540Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1390
2025-10-09T21:49:01.417746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1390 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:01.417800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1390 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:01.417810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1390 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:01.417814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:01.418261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1390 sessionId:  describe result for acl check
2025-10-09T21:49:01.418297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:01.418298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:01.418301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:01.418306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342334282609856:3540] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:01.418854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342334282609856:3540] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:01.421130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342334282609856:3540] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:01.421849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:01.424230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:01.424401Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:01.424412Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:01.424424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:01.424482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:01.424523Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342333920785353:10075]
2025-10-09T21:49:01.424601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342334282609856:3540] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:01.424615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342334282609856:3540] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:01.424619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342334282609856:3540] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:01.424627Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1390 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:01.425037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:01.425341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:01.425374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:01.425680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:01.425734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.425740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.425741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.425742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.425743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.425744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.425746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.425747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.425748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.425753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.425763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.425766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.425770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.425773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.425777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.425781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.425784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.425788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.425794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.425802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.425813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.425818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.425828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.425833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.425836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.425843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.425847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.425852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.425856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.425861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.425866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.425870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.425876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.425881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.425888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.425894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.425925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:01.431917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:01.432932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:01.432947Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:01.432949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:01.432953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:01.432959Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1389 [2:7559342333920785353:10075]
2025-10-09T21:49:01.432970Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:01.432992Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:01.432996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:01.432999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:01.433001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:01.433016Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:01.433022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:01.433029Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:01.433081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.433090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.433091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.433094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:01.433165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:01.433174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.433187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.433203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.433228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:01.433252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342334282609739:2432], now have 1 active actors on pipe
2025-10-09T21:49:01.433260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342334282609952:3540], now have 1 active actors on pipe
2025-10-09T21:49:01.433267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342333920785348:10075]
2025-10-09T21:49:01.433270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342333920785348:10075]
2025-10-09T21:49:01.433272Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342333920785348:10075]
2025-10-09T21:49:01.433283Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:01.433301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:01.433306Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:01.433309Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:01.433311Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:01.433319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:01.433322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:01.433312Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1389
2025-10-09T21:49:01.433341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342333920785348:10075]
2025-10-09T21:49:01.433350Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:01.433318Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1389
2025-10-09T21:49:01.433359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:01.433373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.433377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.433414Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d977f58a-aff3ee00-dd11302b-50840f0a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:01.433426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:01.433430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:01.433432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:01.433442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:01.433455Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:01.433481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:01.433663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.433677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.433710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:01.433821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1390 partition: 0 MaxSeqNo: 25 sessionId: src_id|d977f58a-aff3ee00-dd11302b-50840f0a_0
2025-10-09T21:49:01.434168Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb204fea-8894fb7a-86094f73-dbc91993_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d977f58a-aff3ee00-dd11302b-50840f0a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:01.436579Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d977f58a-aff3ee00-dd11302b-50840f0a_0] Write session: try to update token
2025-10-09T21:49:01.436596Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d977f58a-aff3ee00-dd11302b-50840f0a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:01.446977Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:01.446995Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:01.446997Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:01.446999Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:01.447009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342333920785348:10075]
2025-10-09T21:49:01.447014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342333920785348:10075]
2025-10-09T21:49:01.447017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342333920785348:10075]
2025-10-09T21:49:01.447059Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:01.447061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:01.447068Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:01.447068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.447071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:01.447073Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:01.447207Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:01.448754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1390 sessionId: src_id|d977f58a-aff3ee00-dd11302b-50840f0a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:01.451027Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:01.506556Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342334282609963:4303]
2025-10-09T21:49:01.506832Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 1389, pipe: [1:7559342334282609963:4303]
2025-10-09T21:49:01.589774Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:01.589997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.590008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.590060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342334282609963:4303], now have 1 active actors on pipe
2025-10-09T21:49:01.590280Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:01.590272Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:01.590292Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342334282609963:4303]
2025-10-09T21:49:01.590314Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:01.590317Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:01.590369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1390 reason: error at write: Disconnected sessionId: src_id|d977f58a-aff3ee00-dd11302b-50840f0a_0
2025-10-09T21:49:01.590575Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1390 sessionId: src_id|d977f58a-aff3ee00-dd11302b-50840f0a_0 is DEAD
2025-10-09T21:49:01.590716Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d977f58a-aff3ee00-dd11302b-50840f0a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:01.590729Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d977f58a-aff3ee00-dd11302b-50840f0a_0] Write session will restart in 0.006241s
2025-10-09T21:49:01.590767Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d977f58a-aff3ee00-dd11302b-50840f0a_0] Write session: Do CDS request
2025-10-09T21:49:01.590773Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d977f58a-aff3ee00-dd11302b-50840f0a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:01.598501Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d977f58a-aff3ee00-dd11302b-50840f0a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:01.598575Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:01.598593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1391
2025-10-09T21:49:01.598725Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1391 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:01.598761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1391 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:01.598773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1391 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:01.598778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:01.599171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1391 sessionId:  describe result for acl check
2025-10-09T21:49:01.599208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:01.599210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:01.599212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:01.599219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342334282609999:3534] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:01.599740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342334282609999:3534] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:01.602166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342334282609999:3534] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:01.604982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342334282609999:3534] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:01.604996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342334282609999:3534] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:01.605000Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342334282609999:3534] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:01.605009Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1391 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:01.606097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:01.608520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:01.608727Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:01.608739Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:01.608753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:01.608823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:01.608860Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342333920785571:10077]
2025-10-09T21:49:01.609450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:01.609794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:01.609841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:01.610178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:01.610227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.610233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.610234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.610235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.610236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.610237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.610238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.610239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.610240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.610246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.610257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.610262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.610267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.610271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.610275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.610279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.610284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.610287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.610295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.610303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.610309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.610314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.610320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.610325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.610330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.610337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.610346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.610352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.610356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.610364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.610369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.610378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.610383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.610388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.610396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.610401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.610435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:01.616725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:01.618022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:01.618044Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:01.618046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:01.618049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:01.618054Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1390 [2:7559342333920785571:10077]
2025-10-09T21:49:01.618064Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:01.618079Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:01.618081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:01.618084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:01.618087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:01.618104Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:01.618114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:01.618121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:01.618179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.618184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.618185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.618187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:01.618249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:01.618258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.618270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.618289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.618311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:01.618331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342334282610177:3534], now have 1 active actors on pipe
2025-10-09T21:49:01.618339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342334282609966:2432], now have 1 active actors on pipe
2025-10-09T21:49:01.618347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342333920785566:10077]
2025-10-09T21:49:01.618349Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342333920785566:10077]
2025-10-09T21:49:01.618351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342333920785566:10077]
2025-10-09T21:49:01.618366Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:01.618384Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:01.618395Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:01.618403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:01.618405Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:01.618406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:01.618408Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:01.618396Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1390
2025-10-09T21:49:01.618437Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342333920785566:10077]
2025-10-09T21:49:01.618399Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1390
2025-10-09T21:49:01.618448Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:01.618453Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:01.618465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.618470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.618523Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f3c4eb11-2c5ccd4b-197f32d4-8b397441_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:01.618537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:01.618543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:01.618546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:01.618556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:01.618569Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:01.618591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:01.618725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.618733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.618745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:01.618812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1391 partition: 0 MaxSeqNo: 25 sessionId: src_id|f3c4eb11-2c5ccd4b-197f32d4-8b397441_0
2025-10-09T21:49:01.619188Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d977f58a-aff3ee00-dd11302b-50840f0a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f3c4eb11-2c5ccd4b-197f32d4-8b397441_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:01.621622Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3c4eb11-2c5ccd4b-197f32d4-8b397441_0] Write session: try to update token
2025-10-09T21:49:01.621656Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3c4eb11-2c5ccd4b-197f32d4-8b397441_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:01.632574Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:01.632592Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:01.632595Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:01.632597Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:01.632606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342333920785566:10077]
2025-10-09T21:49:01.632611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342333920785566:10077]
2025-10-09T21:49:01.632615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342333920785566:10077]
2025-10-09T21:49:01.632661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:01.632665Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:01.632668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.632671Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:01.632671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:01.632675Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:01.632797Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:01.634618Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1391 sessionId: src_id|f3c4eb11-2c5ccd4b-197f32d4-8b397441_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:01.637110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:01.690359Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342334282610192:4303]
2025-10-09T21:49:01.690636Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 1390, pipe: [1:7559342334282610192:4303]
2025-10-09T21:49:01.785159Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:01.785397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.785432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.785485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342334282610192:4303], now have 1 active actors on pipe
2025-10-09T21:49:01.785698Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:01.785700Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342334282610192:4303]
2025-10-09T21:49:01.785719Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:01.785723Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:01.785788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1391 reason: error at write: Disconnected sessionId: src_id|f3c4eb11-2c5ccd4b-197f32d4-8b397441_0
2025-10-09T21:49:01.785909Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:01.785952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1391 sessionId: src_id|f3c4eb11-2c5ccd4b-197f32d4-8b397441_0 is DEAD
2025-10-09T21:49:01.786094Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3c4eb11-2c5ccd4b-197f32d4-8b397441_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:01.786110Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3c4eb11-2c5ccd4b-197f32d4-8b397441_0] Write session will restart in 0.008038s
2025-10-09T21:49:01.786130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3c4eb11-2c5ccd4b-197f32d4-8b397441_0] Write session: Do CDS request
2025-10-09T21:49:01.786135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3c4eb11-2c5ccd4b-197f32d4-8b397441_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:01.795495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3c4eb11-2c5ccd4b-197f32d4-8b397441_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:01.795571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:01.795585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1392
2025-10-09T21:49:01.795735Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1392 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:01.795772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1392 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:01.795780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1392 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:01.795783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:01.796267Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1392 sessionId:  describe result for acl check
2025-10-09T21:49:01.796302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:01.796309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:01.796311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:01.796317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342334282610274:3524] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:01.796831Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342334282610274:3524] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:01.799060Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342334282610274:3524] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:01.801020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:01.803389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:01.803599Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:01.803611Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:01.803623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:01.803623Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342334282610274:3524] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:01.803640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342334282610274:3524] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:01.803644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342334282610274:3524] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:01.803653Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1392 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:01.803682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:01.803725Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342333920785791:10081]
2025-10-09T21:49:01.804293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:01.804583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:01.804630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:01.804944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:01.805008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.805015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.805017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.805018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.805020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.805022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.805023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.805025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.805026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.805031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.805044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.805053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.805056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.805060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.805064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.805067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.805075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.805078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.805085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.805098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:01.805109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.805115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:01.805124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.805129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:01.805132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.805137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:01.805148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.805153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:01.805161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.805166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:01.805171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.805176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.805180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.805184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.805189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.805193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.805219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:01.811670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:01.812680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:01.812695Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:01.812698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:01.812703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:01.812710Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1391 [2:7559342333920785791:10081]
2025-10-09T21:49:01.812728Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:01.812751Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:01.812759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:01.812762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:01.812765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:01.812783Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:01.812794Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:01.812801Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:01.812865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.812874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.812876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.812879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:01.812950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:01.812958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.812966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:01.812978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:01.812987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:01.813005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342334282610198:2432], now have 1 active actors on pipe
2025-10-09T21:49:01.813008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342334282610413:3524], now have 1 active actors on pipe
2025-10-09T21:49:01.813014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342333920785786:10081]
2025-10-09T21:49:01.813016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342333920785786:10081]
2025-10-09T21:49:01.813017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342333920785786:10081]
2025-10-09T21:49:01.813028Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:01.813040Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:01.813042Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:01.813043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:01.813045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:01.813058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:01.813060Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:01.813089Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342333920785786:10081]
2025-10-09T21:49:01.813099Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:01.813080Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1391
2025-10-09T21:49:01.813114Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:01.813083Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1391
2025-10-09T21:49:01.813146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.813156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.813189Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|50f3b7e9-d598a5b3-28b0ee51-56ff4673_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:01.813201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:01.813207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:01.813209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:01.813220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:01.813234Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:01.813253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:01.813414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.813424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.813440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:01.813540Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1392 partition: 0 MaxSeqNo: 25 sessionId: src_id|50f3b7e9-d598a5b3-28b0ee51-56ff4673_0
2025-10-09T21:49:01.813927Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3c4eb11-2c5ccd4b-197f32d4-8b397441_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|50f3b7e9-d598a5b3-28b0ee51-56ff4673_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:01.816257Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|50f3b7e9-d598a5b3-28b0ee51-56ff4673_0] Write session: try to update token
2025-10-09T21:49:01.816275Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|50f3b7e9-d598a5b3-28b0ee51-56ff4673_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:01.824387Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:01.824406Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:01.824416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342333920785786:10081]
2025-10-09T21:49:01.824459Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:01.824464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:01.824630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:01.824772Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:01.824907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:01.825039Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:01.825239Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:01.825288Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:01.828443Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1392 sessionId: src_id|50f3b7e9-d598a5b3-28b0ee51-56ff4673_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:01.830737Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:01.830912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.830933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.830983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:01.831051Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:01.836632Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:01.836649Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:01.836651Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:01.836653Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:01.836664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342333920785786:10081]
2025-10-09T21:49:01.836667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342333920785786:10081]
2025-10-09T21:49:01.836670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342333920785786:10081]
2025-10-09T21:49:01.836702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:01.836702Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:01.836707Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:01.836708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:01.836711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:01.836721Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:01.836826Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:01.885923Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342334282610440:4303]
2025-10-09T21:49:01.886164Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 1391, pipe: [1:7559342334282610440:4303]
2025-10-09T21:49:01.986134Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:01.993303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:01.993329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:01.993479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.993533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.993602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.993687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.993731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.993742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.993804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.993858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.993914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.993983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.994044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.994050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.994106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.994159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.994216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.994275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.994314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.994320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.994376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.994433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.994496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.994550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.994592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.994597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.994656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.994714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.994769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.994828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.994866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.994872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.994927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.994990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.995045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.995104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.995148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.995157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.995214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.995273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.995326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.995379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.995418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.995422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.995480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.995536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.995586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.995637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.995675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.995680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.995744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.995798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.995854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.995908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.995949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.995954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.996015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:01.996066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:01.996125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:01.996182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:01.996225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:01.996233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:01.996283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342334282610440:4303], now have 1 active actors on pipe
2025-10-09T21:49:01.996561Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:01.996578Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:01.996612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1392 reason: error at write: Disconnected sessionId: src_id|50f3b7e9-d598a5b3-28b0ee51-56ff4673_0
2025-10-09T21:49:01.996667Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342334282610440:4303]
2025-10-09T21:49:01.996699Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:01.996822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1392 sessionId: src_id|50f3b7e9-d598a5b3-28b0ee51-56ff4673_0 is DEAD
2025-10-09T21:49:01.996959Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50f3b7e9-d598a5b3-28b0ee51-56ff4673_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:01.996969Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50f3b7e9-d598a5b3-28b0ee51-56ff4673_0] Write session will restart in 0.007206s
2025-10-09T21:49:01.996989Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50f3b7e9-d598a5b3-28b0ee51-56ff4673_0] Write session: Do CDS request
2025-10-09T21:49:01.996994Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50f3b7e9-d598a5b3-28b0ee51-56ff4673_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:01.997107Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:02.005566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|50f3b7e9-d598a5b3-28b0ee51-56ff4673_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:02.005631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:02.005663Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1393
2025-10-09T21:49:02.005819Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1393 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:02.005854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1393 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:02.005863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1393 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:02.005867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:02.006277Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1393 sessionId:  describe result for acl check
2025-10-09T21:49:02.006313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:02.006321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:02.006325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:02.006331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342338577577788:3507] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:02.006842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342338577577788:3507] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:02.009307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342338577577788:3507] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:02.012509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:02.012655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342338577577788:3507] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:02.012680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342338577577788:3507] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:02.012687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342338577577788:3507] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:02.012696Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1393 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:02.015097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:02.015325Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:02.015336Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:02.015349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:02.015401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:02.015442Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342338215753344:10095]
2025-10-09T21:49:02.016051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:02.016418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:02.016471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:02.016735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:02.016780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.016784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.016785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.016786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.016787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.016787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.016788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.016789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.016790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.016794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.016801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.016803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.016806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.016808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.016810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.016812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.016814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.016816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.016822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.016827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.016832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.016835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.016838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.016841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.016844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.016847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.016849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.016853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.016855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.016858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.016861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.016864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.016866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.016869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.016872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.016875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.016899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:02.023465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:02.024466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:02.024481Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:02.024482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:02.024487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:02.024492Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1392 [2:7559342338215753344:10095]
2025-10-09T21:49:02.024505Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:02.024522Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:02.024524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:02.024526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:02.024528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:02.024542Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:02.024546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:02.024553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:02.024598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.024604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.024606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.024608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:02.024788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:02.024795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.024803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.024810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.024821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:02.024811Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1392
2025-10-09T21:49:02.024840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342338577577961:3507], now have 1 active actors on pipe
2025-10-09T21:49:02.024812Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1392
2025-10-09T21:49:02.024846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342334282610453:2432], now have 1 active actors on pipe
2025-10-09T21:49:02.024853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342338215753338:10095]
2025-10-09T21:49:02.024855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342338215753338:10095]
2025-10-09T21:49:02.024856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342338215753338:10095]
2025-10-09T21:49:02.024864Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:02.024872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:02.024875Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:02.024877Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:02.024878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:02.024895Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:02.024898Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:02.024919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.024923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.024962Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342338215753338:10095]
2025-10-09T21:49:02.024973Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:02.024988Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4ea5caab-d9b090e9-37b004b4-df6f6e43_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:02.024993Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:02.024995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:02.024998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:02.025000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:02.025009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:02.025017Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:02.025043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:02.025228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.025241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.025265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:02.025388Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1393 partition: 0 MaxSeqNo: 25 sessionId: src_id|4ea5caab-d9b090e9-37b004b4-df6f6e43_0
2025-10-09T21:49:02.025770Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50f3b7e9-d598a5b3-28b0ee51-56ff4673_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4ea5caab-d9b090e9-37b004b4-df6f6e43_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:02.028065Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ea5caab-d9b090e9-37b004b4-df6f6e43_0] Write session: try to update token
2025-10-09T21:49:02.028082Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ea5caab-d9b090e9-37b004b4-df6f6e43_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:02.036746Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:02.036762Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:02.036771Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342338215753338:10095]
2025-10-09T21:49:02.036814Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:02.036820Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:02.036992Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:02.037127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:02.037268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:02.037396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:02.037569Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:02.037607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:02.040329Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1393 sessionId: src_id|4ea5caab-d9b090e9-37b004b4-df6f6e43_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:02.042991Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:02.043184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.043204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.043239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:02.043324Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:02.049622Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:02.049652Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:02.049654Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:02.049655Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:02.049664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342338215753338:10095]
2025-10-09T21:49:02.049667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342338215753338:10095]
2025-10-09T21:49:02.049670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342338215753338:10095]
2025-10-09T21:49:02.049705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:02.049710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.049712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:02.049720Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:02.049728Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:02.049732Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:02.049835Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:02.096887Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342338577577977:4303]
2025-10-09T21:49:02.097150Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 1392, pipe: [1:7559342338577577977:4303]
2025-10-09T21:49:02.199759Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:02.206725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.206750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.206899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.206953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.207010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.207076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.207117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.207125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.207182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.207234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.207296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.207354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.207402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.207410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.207471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.207525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.207586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.207642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.207689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.207698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.207771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.207836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.207903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.207970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.208016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.208024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.208089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.208149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.208206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.208265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.208313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.208321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.208376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.208426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.208477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.208532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.208578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.208587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.208645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.208697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.208753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.208810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.208854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.208860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.208916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.208978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.209045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.209104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.209146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.209151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.209207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.209260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.209314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.209370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.209406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.209412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.209468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.209519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.209581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.209647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.209686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.209691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.209736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342338577577977:4303], now have 1 active actors on pipe
2025-10-09T21:49:02.210090Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:02.210099Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:02.210104Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:02.210138Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342338577577977:4303]
2025-10-09T21:49:02.210139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1393 reason: error at write: Disconnected sessionId: src_id|4ea5caab-d9b090e9-37b004b4-df6f6e43_0
2025-10-09T21:49:02.210303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1393 sessionId: src_id|4ea5caab-d9b090e9-37b004b4-df6f6e43_0 is DEAD
2025-10-09T21:49:02.210465Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ea5caab-d9b090e9-37b004b4-df6f6e43_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:02.210478Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ea5caab-d9b090e9-37b004b4-df6f6e43_0] Write session will restart in 0.009775s
2025-10-09T21:49:02.210516Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ea5caab-d9b090e9-37b004b4-df6f6e43_0] Write session: Do CDS request
2025-10-09T21:49:02.210503Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:02.210522Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ea5caab-d9b090e9-37b004b4-df6f6e43_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:02.221515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ea5caab-d9b090e9-37b004b4-df6f6e43_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:02.221598Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:02.221613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1394
2025-10-09T21:49:02.221763Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1394 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:02.221807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1394 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:02.221813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1394 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:02.221816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:02.222327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1394 sessionId:  describe result for acl check
2025-10-09T21:49:02.222364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:02.222372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:02.222374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:02.222379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342338577578080:3503] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:02.222939Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342338577578080:3503] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:02.225287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342338577578080:3503] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:02.226318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:02.228437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:02.228621Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:02.228630Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:02.228641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:02.228683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:02.228716Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342338215753562:10097]
2025-10-09T21:49:02.228762Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342338577578080:3503] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:02.228776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342338577578080:3503] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:02.228779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342338577578080:3503] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:02.228786Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1394 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:02.229132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:02.229445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:02.229517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:02.229879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:02.229935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.229940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.229941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.229942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.229943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.229943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.229944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.229945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.229945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.229949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.229957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.229959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.229961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.229963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.229965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.229967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.229969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.229971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.229976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.229982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.229986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.229989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.229992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.229995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.229997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.230000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.230003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.230006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.230008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.230011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.230014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.230017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.230019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.230022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.230025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.230043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.230072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:02.236389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:02.237564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:02.237577Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:02.237579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:02.237583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:02.237590Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1393 [2:7559342338215753562:10097]
2025-10-09T21:49:02.237605Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:02.237624Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:02.237632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:02.237648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:02.237650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:02.237668Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:02.237682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:02.237689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:02.237750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.237759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.237761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.237764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:02.237836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:02.237844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.237858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.237870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.237892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:02.237919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342338577577982:2432], now have 1 active actors on pipe
2025-10-09T21:49:02.237927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342338577578198:3503], now have 1 active actors on pipe
2025-10-09T21:49:02.237935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342338215753557:10097]
2025-10-09T21:49:02.237938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342338215753557:10097]
2025-10-09T21:49:02.237940Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342338215753557:10097]
2025-10-09T21:49:02.237953Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:02.237971Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:02.237972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:02.237982Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:02.237980Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1393
2025-10-09T21:49:02.237983Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:02.237985Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:02.237989Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1393
2025-10-09T21:49:02.237985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:02.238015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342338215753557:10097]
2025-10-09T21:49:02.238019Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:02.238022Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:02.238058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.238063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.238110Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f442282a-715ad66e-3a03ea1e-d3b57520_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:02.238122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:02.238125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:02.238127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:02.238135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:02.238143Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:02.238160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:02.238311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.238317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.238328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:02.238437Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1394 partition: 0 MaxSeqNo: 25 sessionId: src_id|f442282a-715ad66e-3a03ea1e-d3b57520_0
2025-10-09T21:49:02.238793Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ea5caab-d9b090e9-37b004b4-df6f6e43_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f442282a-715ad66e-3a03ea1e-d3b57520_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:02.241098Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f442282a-715ad66e-3a03ea1e-d3b57520_0] Write session: try to update token
2025-10-09T21:49:02.241118Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f442282a-715ad66e-3a03ea1e-d3b57520_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:02.252182Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:02.252196Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:02.252197Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:02.252199Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:02.252208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342338215753557:10097]
2025-10-09T21:49:02.252211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342338215753557:10097]
2025-10-09T21:49:02.252214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342338215753557:10097]
2025-10-09T21:49:02.252252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:02.252257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.252257Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:02.252259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:02.252262Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:02.252265Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:02.252374Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:02.253884Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1394 sessionId: src_id|f442282a-715ad66e-3a03ea1e-d3b57520_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:02.256213Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:02.310388Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342338577578215:4303]
2025-10-09T21:49:02.310663Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 1393, pipe: [1:7559342338577578215:4303]
2025-10-09T21:49:02.399779Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:02.400202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.400215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.400252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342338577578215:4303], now have 1 active actors on pipe
2025-10-09T21:49:02.400413Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:02.400455Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:02.400500Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:02.400502Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342338577578215:4303]
2025-10-09T21:49:02.400589Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1394 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f442282a-715ad66e-3a03ea1e-d3b57520_0
2025-10-09T21:49:02.400743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1394 sessionId: src_id|f442282a-715ad66e-3a03ea1e-d3b57520_0 is DEAD
2025-10-09T21:49:02.400892Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f442282a-715ad66e-3a03ea1e-d3b57520_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:02.400902Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f442282a-715ad66e-3a03ea1e-d3b57520_0] Write session will restart in 0.006608s
2025-10-09T21:49:02.400924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f442282a-715ad66e-3a03ea1e-d3b57520_0] Write session: Do CDS request
2025-10-09T21:49:02.400929Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f442282a-715ad66e-3a03ea1e-d3b57520_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:02.408453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f442282a-715ad66e-3a03ea1e-d3b57520_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:02.408587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:02.408606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1395
2025-10-09T21:49:02.408730Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1395 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:02.408763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1395 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:02.408781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1395 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:02.408789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:02.409299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1395 sessionId:  describe result for acl check
2025-10-09T21:49:02.409340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:02.409343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:02.409344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:02.409353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342338577578255:3499] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:02.409956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342338577578255:3499] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:02.413328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342338577578255:3499] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:02.416678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342338577578255:3499] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:02.416700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342338577578255:3499] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:02.416707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342338577578255:3499] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:02.416716Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1395 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:02.418055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:02.421353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:02.421523Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:02.421541Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:02.421553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:02.421609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:02.421659Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342338215753783:10099]
2025-10-09T21:49:02.422095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:02.422409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:02.422452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:02.422768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:02.422822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.422828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.422829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.422830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.422832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.422833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.422834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.422835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.422836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.422842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.422853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.422857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.422861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.422865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.422869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.422874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.422877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.422881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.422888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.422892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.422897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.422901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.422904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.422909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.422914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.422919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.422923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.422928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.422939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.422944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.422949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.422959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.422964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.422974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.422978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.422982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.422998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:02.429354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:02.430465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:02.430477Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:02.430479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:02.430482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:02.430487Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1394 [2:7559342338215753783:10099]
2025-10-09T21:49:02.430497Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:02.430511Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:02.430514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:02.430516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:02.430517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:02.430530Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:02.430538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:02.430543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:02.430585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.430590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.430591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.430593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:02.430645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:02.430650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.430657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.430666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.430687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:02.430707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342338215753778:10099]
2025-10-09T21:49:02.430712Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342338215753778:10099]
2025-10-09T21:49:02.430714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342338215753778:10099]
2025-10-09T21:49:02.430721Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:02.430734Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:02.430736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342338577578218:2432], now have 1 active actors on pipe
2025-10-09T21:49:02.430741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:02.430748Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:02.430750Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:02.430752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:02.430755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:02.430761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342338577578433:3499], now have 1 active actors on pipe
2025-10-09T21:49:02.430784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342338215753778:10099]
2025-10-09T21:49:02.430802Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:02.430822Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:02.430878Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1394
2025-10-09T21:49:02.430889Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1394
2025-10-09T21:49:02.430967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.430980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.431009Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f0cc925-4ce7011b-51066e94-ef5c1cf0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:02.431029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:02.431039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:02.431042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:02.431052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:02.431066Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:02.431084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:02.431257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.431266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.431280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:02.431381Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1395 partition: 0 MaxSeqNo: 25 sessionId: src_id|f0cc925-4ce7011b-51066e94-ef5c1cf0_0
2025-10-09T21:49:02.431774Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f442282a-715ad66e-3a03ea1e-d3b57520_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f0cc925-4ce7011b-51066e94-ef5c1cf0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:02.434201Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0cc925-4ce7011b-51066e94-ef5c1cf0_0] Write session: try to update token
2025-10-09T21:49:02.434218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0cc925-4ce7011b-51066e94-ef5c1cf0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:02.445156Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:02.445175Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:02.445177Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:02.445180Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:02.445190Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342338215753778:10099]
2025-10-09T21:49:02.445194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342338215753778:10099]
2025-10-09T21:49:02.445199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342338215753778:10099]
2025-10-09T21:49:02.445245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:02.445254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.445258Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:02.445264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:02.445266Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:02.445271Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:02.445391Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:02.446282Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1395 sessionId: src_id|f0cc925-4ce7011b-51066e94-ef5c1cf0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:02.448634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:02.500855Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342338577578447:4303]
2025-10-09T21:49:02.501217Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 1394, pipe: [1:7559342338577578447:4303]
2025-10-09T21:49:02.592332Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:02.592889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.592902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.592942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342338577578447:4303], now have 1 active actors on pipe
2025-10-09T21:49:02.593128Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:02.593202Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:02.593216Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:02.593244Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:02.593276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1395 reason: error at write: Disconnected sessionId: src_id|f0cc925-4ce7011b-51066e94-ef5c1cf0_0
2025-10-09T21:49:02.593325Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342338577578447:4303]
2025-10-09T21:49:02.593424Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1395 sessionId: src_id|f0cc925-4ce7011b-51066e94-ef5c1cf0_0 is DEAD
2025-10-09T21:49:02.593553Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0cc925-4ce7011b-51066e94-ef5c1cf0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:02.593562Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0cc925-4ce7011b-51066e94-ef5c1cf0_0] Write session will restart in 0.005617s
2025-10-09T21:49:02.593584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0cc925-4ce7011b-51066e94-ef5c1cf0_0] Write session: Do CDS request
2025-10-09T21:49:02.593591Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0cc925-4ce7011b-51066e94-ef5c1cf0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:02.600562Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0cc925-4ce7011b-51066e94-ef5c1cf0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:02.600644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:02.600673Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1396
2025-10-09T21:49:02.600833Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1396 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:02.600882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1396 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:02.600894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1396 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:02.600898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:02.601449Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1396 sessionId:  describe result for acl check
2025-10-09T21:49:02.601485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:02.601491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:02.601493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:02.601499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342338577578474:3491] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:02.602111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342338577578474:3491] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:02.604866Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342338577578474:3491] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:02.607482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342338577578474:3491] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:02.607497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342338577578474:3491] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:02.607501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342338577578474:3491] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:02.607510Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1396 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:02.608556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:02.610745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:02.610945Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:02.610957Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:02.610971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:02.611040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:02.611082Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342338215754005:10101]
2025-10-09T21:49:02.611673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:02.611984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:02.612037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:02.612371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:02.612427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.612433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.612434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.612435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.612436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.612437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.612439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.612440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.612441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.612447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.612458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.612462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.612467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.612470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.612475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.612479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.612487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.612490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.612500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.612512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.612523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.612531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.612537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.612542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.612547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.612552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.612557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.612563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.612567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.612573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.612578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.612582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.612585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.612590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.612594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.612600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.612628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:02.619143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:02.620644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:02.620661Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:02.620664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:02.620669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:02.620676Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1395 [2:7559342338215754005:10101]
2025-10-09T21:49:02.620690Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:02.620709Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:02.620712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:02.620715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:02.620717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:02.620737Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:02.620743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:02.620750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:02.620820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.620827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.620830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.620833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:02.620902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:02.620910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.620921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.620932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.620948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:02.620979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342338577578450:2432], now have 1 active actors on pipe
2025-10-09T21:49:02.620992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342338577578666:3491], now have 1 active actors on pipe
2025-10-09T21:49:02.621000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342338215754000:10101]
2025-10-09T21:49:02.621002Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342338215754000:10101]
2025-10-09T21:49:02.621005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342338215754000:10101]
2025-10-09T21:49:02.621019Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:02.621034Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:02.621043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:02.621051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:02.621053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:02.621055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:02.621056Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:02.621053Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1395
2025-10-09T21:49:02.621103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1395
2025-10-09T21:49:02.621097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342338215754000:10101]
2025-10-09T21:49:02.621105Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:02.621107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:02.621145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.621150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.621179Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a96d6cc3-9f647503-ad73533-c99d3270_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:02.621191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:02.621197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:02.621199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:02.621210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:02.621224Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:02.621241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:02.621374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.621383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.621397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:02.621471Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1396 partition: 0 MaxSeqNo: 25 sessionId: src_id|a96d6cc3-9f647503-ad73533-c99d3270_0
2025-10-09T21:49:02.621865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0cc925-4ce7011b-51066e94-ef5c1cf0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a96d6cc3-9f647503-ad73533-c99d3270_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:02.624412Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a96d6cc3-9f647503-ad73533-c99d3270_0] Write session: try to update token
2025-10-09T21:49:02.624431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a96d6cc3-9f647503-ad73533-c99d3270_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:02.632824Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:02.632842Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:02.632852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342338215754000:10101]
2025-10-09T21:49:02.632898Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:02.632900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:02.633074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:02.633217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:02.633343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:02.633466Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:02.633658Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:02.633705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:02.637501Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1396 sessionId: src_id|a96d6cc3-9f647503-ad73533-c99d3270_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:02.639855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:02.640207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.640223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.640266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:02.640351Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:02.645849Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:02.645872Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:02.645876Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:02.645878Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:02.645889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342338215754000:10101]
2025-10-09T21:49:02.645894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342338215754000:10101]
2025-10-09T21:49:02.645899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342338215754000:10101]
2025-10-09T21:49:02.645943Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:02.645950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:02.645953Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:02.645955Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:02.645958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.645961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:02.646177Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:02.693366Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342338577578681:4303]
2025-10-09T21:49:02.693613Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 1395, pipe: [1:7559342338577578681:4303]
2025-10-09T21:49:02.792503Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:02.799495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.799521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.799688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.799763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.799841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.799904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.799951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.799968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.800038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.800094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.800151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.800214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.800257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.800262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.800329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.800389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.800449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.800519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.800564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.800573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.800636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.800702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.800761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.800819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.800859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.800864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.800925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.800992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.801064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.801124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.801168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.801176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.801247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.801305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.801363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.801430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.801476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.801482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.801539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.801590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.801664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.801719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.801760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.801765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.801827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.801884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.801949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.802005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.802064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.802069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.802125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.802180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.802235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.802295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.802340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.802349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.802424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:02.802489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:02.802556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:02.802619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:02.802661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:02.802667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:02.802717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342338577578681:4303], now have 1 active actors on pipe
2025-10-09T21:49:02.802978Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:02.802997Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:02.802996Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342338577578681:4303]
2025-10-09T21:49:02.803024Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:02.803030Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1396 reason: error at write: Disconnected sessionId: src_id|a96d6cc3-9f647503-ad73533-c99d3270_0
2025-10-09T21:49:02.803239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1396 sessionId: src_id|a96d6cc3-9f647503-ad73533-c99d3270_0 is DEAD
2025-10-09T21:49:02.803376Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a96d6cc3-9f647503-ad73533-c99d3270_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:02.803388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a96d6cc3-9f647503-ad73533-c99d3270_0] Write session will restart in 0.008746s
2025-10-09T21:49:02.803413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a96d6cc3-9f647503-ad73533-c99d3270_0] Write session: Do CDS request
2025-10-09T21:49:02.803420Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a96d6cc3-9f647503-ad73533-c99d3270_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:02.803713Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:02.813520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a96d6cc3-9f647503-ad73533-c99d3270_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:02.813615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:02.813646Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1397
2025-10-09T21:49:02.813784Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1397 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:02.813839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1397 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:02.813848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1397 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:02.813854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:02.814296Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1397 sessionId:  describe result for acl check
2025-10-09T21:49:02.814331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:02.814336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:02.814337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:02.814342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342338577578734:3480] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:02.814877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342338577578734:3480] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:02.817600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342338577578734:3480] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:02.820294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:02.820385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342338577578734:3480] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:02.820400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342338577578734:3480] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:02.820403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342338577578734:3480] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:02.820411Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1397 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:02.822678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:02.822848Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:02.822857Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:02.822868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:02.822919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:02.822959Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342338215754231:10105]
2025-10-09T21:49:02.823523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:02.823815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:02.823860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:02.824171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:02.824224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.824230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.824231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.824232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.824233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.824234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.824235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.824236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.824238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.824242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.824253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.824257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.824261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.824265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.824269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.824273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.824276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.824280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.824286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.824294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:02.824300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.824305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:02.824313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.824319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:02.824323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.824328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:02.824332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.824337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:02.824341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.824346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:02.824350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.824355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.824360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.824368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.824373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.824377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.824404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:02.830791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:02.831999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:02.832016Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:02.832018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:02.832023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:02.832029Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1396 [2:7559342338215754231:10105]
2025-10-09T21:49:02.832044Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:02.832063Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:02.832067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:02.832070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:02.832072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:02.832090Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:02.832104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:02.832111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:02.832170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.832178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.832180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.832182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:02.832249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:02.832257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.832269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:02.832282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:02.832299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:02.832325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342338577578902:3480], now have 1 active actors on pipe
2025-10-09T21:49:02.832333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342338577578683:2432], now have 1 active actors on pipe
2025-10-09T21:49:02.832340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342338215754225:10105]
2025-10-09T21:49:02.832343Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342338215754225:10105]
2025-10-09T21:49:02.832345Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342338215754225:10105]
2025-10-09T21:49:02.832359Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:02.832378Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:02.832378Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:02.832381Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:02.832383Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:02.832394Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:02.832398Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:02.832433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342338215754225:10105]
2025-10-09T21:49:02.832441Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:02.832413Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1396
2025-10-09T21:49:02.832448Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1396
2025-10-09T21:49:02.832452Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:02.832468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.832473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.832510Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8993d64f-2e6875ac-7c503271-7effa1f7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:02.832522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:02.832526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:02.832529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:02.832540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:02.832553Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:02.832573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:02.832741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.832766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.832786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:02.832899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1397 partition: 0 MaxSeqNo: 25 sessionId: src_id|8993d64f-2e6875ac-7c503271-7effa1f7_0
2025-10-09T21:49:02.833236Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a96d6cc3-9f647503-ad73533-c99d3270_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8993d64f-2e6875ac-7c503271-7effa1f7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:02.835566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8993d64f-2e6875ac-7c503271-7effa1f7_0] Write session: try to update token
2025-10-09T21:49:02.835581Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8993d64f-2e6875ac-7c503271-7effa1f7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:02.847187Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:02.847210Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:02.847212Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:02.847215Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:02.847225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342338215754225:10105]
2025-10-09T21:49:02.847230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342338215754225:10105]
2025-10-09T21:49:02.847234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342338215754225:10105]
2025-10-09T21:49:02.847277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:02.847290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:02.847293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:02.847292Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:02.847298Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:02.847302Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:02.847435Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:02.848711Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1397 sessionId: src_id|8993d64f-2e6875ac-7c503271-7effa1f7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:02.851059Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:02.903260Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342338577578917:4303]
2025-10-09T21:49:02.903552Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 1396, pipe: [1:7559342338577578917:4303]
2025-10-09T21:49:02.991335Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:02.991701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:02.991717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:02.991759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342338577578917:4303], now have 1 active actors on pipe
2025-10-09T21:49:02.991934Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:02.992013Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342338577578917:4303]
2025-10-09T21:49:02.992043Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:02.992531Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:02.992605Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1397 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8993d64f-2e6875ac-7c503271-7effa1f7_0
2025-10-09T21:49:02.992764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1397 sessionId: src_id|8993d64f-2e6875ac-7c503271-7effa1f7_0 is DEAD
2025-10-09T21:49:02.992909Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8993d64f-2e6875ac-7c503271-7effa1f7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:02.992921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8993d64f-2e6875ac-7c503271-7effa1f7_0] Write session will restart in 0.007643s
2025-10-09T21:49:02.992948Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8993d64f-2e6875ac-7c503271-7effa1f7_0] Write session: Do CDS request
2025-10-09T21:49:02.992954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8993d64f-2e6875ac-7c503271-7effa1f7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:03.001550Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8993d64f-2e6875ac-7c503271-7effa1f7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:03.001664Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:03.001682Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1398
2025-10-09T21:49:03.001799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1398 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:03.001846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1398 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:03.001857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1398 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:03.001866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:03.002346Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1398 sessionId:  describe result for acl check
2025-10-09T21:49:03.002382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:03.002389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:03.002391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:03.002397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342342872546313:3469] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:03.002942Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342342872546313:3469] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:03.005489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342342872546313:3469] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:03.007695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:03.010274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:03.010473Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:03.010482Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:03.010492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:03.010545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:03.010581Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342342510721787:10119]
2025-10-09T21:49:03.010612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342342872546313:3469] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:03.010629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342342872546313:3469] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:03.010633Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342342872546313:3469] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:03.010642Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1398 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:03.011099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:03.011376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:03.011415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:03.011679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:03.011723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.011727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.011728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.011729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.011730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.011731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.011732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.011733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.011735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.011740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.011751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.011755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.011759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.011763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.011767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.011771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.011775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.011779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.011786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.011794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.011800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.011807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.011812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.011817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.011822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.011827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.011831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.011840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.011843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.011846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.011849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.011853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.011856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.011858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.011861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.011864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.011880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:03.018429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:03.019424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:03.019440Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:03.019443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:03.019448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:03.019453Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1397 [2:7559342342510721787:10119]
2025-10-09T21:49:03.019464Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:03.019481Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:03.019486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:03.019489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:03.019490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:03.019506Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:03.019514Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:03.019520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:03.019570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.019574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.019576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.019578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:03.019637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:03.019642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.019651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.019658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.019667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:03.019686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342342510721782:10119]
2025-10-09T21:49:03.019691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342342510721782:10119]
2025-10-09T21:49:03.019693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342342510721782:10119]
2025-10-09T21:49:03.019699Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:03.019711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342342872546457:3469], now have 1 active actors on pipe
2025-10-09T21:49:03.019714Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:03.019718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342338577578943:2432], now have 1 active actors on pipe
2025-10-09T21:49:03.019725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:03.019726Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:03.019728Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:03.019738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:03.019741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:03.019781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342342510721782:10119]
2025-10-09T21:49:03.019797Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:03.019799Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1397
2025-10-09T21:49:03.019822Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:03.019806Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1397
2025-10-09T21:49:03.019888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.019900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.019923Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ee0a9bcf-c16c059-91a1facc-f861a4fc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:03.019929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:03.019933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:03.019935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:03.019942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:03.019955Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:03.019970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:03.020106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.020113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.020126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:03.020233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1398 partition: 0 MaxSeqNo: 25 sessionId: src_id|ee0a9bcf-c16c059-91a1facc-f861a4fc_0
2025-10-09T21:49:03.020570Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8993d64f-2e6875ac-7c503271-7effa1f7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ee0a9bcf-c16c059-91a1facc-f861a4fc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:03.022974Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee0a9bcf-c16c059-91a1facc-f861a4fc_0] Write session: try to update token
2025-10-09T21:49:03.022990Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee0a9bcf-c16c059-91a1facc-f861a4fc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:03.031046Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:03.031064Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:03.031075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342342510721782:10119]
2025-10-09T21:49:03.031122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:03.031124Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:03.031297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:03.031419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:03.031537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:03.031657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:03.031835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:03.031872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:03.035612Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1398 sessionId: src_id|ee0a9bcf-c16c059-91a1facc-f861a4fc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:03.037938Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:03.038465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.038483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.038525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:03.038713Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:03.044662Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:03.044678Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:03.044681Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:03.044683Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:03.044693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342342510721782:10119]
2025-10-09T21:49:03.044697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342342510721782:10119]
2025-10-09T21:49:03.044700Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342342510721782:10119]
2025-10-09T21:49:03.044743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:03.044754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.044757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:03.044757Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:03.044762Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:03.044765Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:03.044896Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:03.092242Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342342872546469:4303]
2025-10-09T21:49:03.092497Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 1397, pipe: [1:7559342342872546469:4303]
2025-10-09T21:49:03.192198Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:03.198610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.198631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.198773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.198827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.198886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.198951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.198996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.199004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.199065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.199119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.199174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.199232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.199270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.199276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.199335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.199393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.199452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.199512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.199552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.199557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.199616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.199671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.199738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.199795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.199834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.199839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.199898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.199950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.200008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.200072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.200110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.200115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.200175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.200228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.200283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.200337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.200381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.200384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.200447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.200500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.200553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.200611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.200654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.200663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.200721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.200773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.200825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.200879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.200962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.200968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.201021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.201078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.201140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.201195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.201232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.201237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.201293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.201344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.201398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.201454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.201518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.201530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.201571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342342872546469:4303], now have 1 active actors on pipe
2025-10-09T21:49:03.201819Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342342872546469:4303]
2025-10-09T21:49:03.201831Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:03.201863Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:03.201866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:03.201899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1398 reason: error at write: Disconnected sessionId: src_id|ee0a9bcf-c16c059-91a1facc-f861a4fc_0
2025-10-09T21:49:03.202113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1398 sessionId: src_id|ee0a9bcf-c16c059-91a1facc-f861a4fc_0 is DEAD
2025-10-09T21:49:03.202263Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee0a9bcf-c16c059-91a1facc-f861a4fc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:03.202276Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee0a9bcf-c16c059-91a1facc-f861a4fc_0] Write session will restart in 0.008015s
2025-10-09T21:49:03.202301Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee0a9bcf-c16c059-91a1facc-f861a4fc_0] Write session: Do CDS request
2025-10-09T21:49:03.202307Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee0a9bcf-c16c059-91a1facc-f861a4fc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:03.202331Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:03.211518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee0a9bcf-c16c059-91a1facc-f861a4fc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:03.211612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:03.211632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1399
2025-10-09T21:49:03.211749Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1399 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:03.211781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1399 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:03.211792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1399 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:03.211795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:03.212283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1399 sessionId:  describe result for acl check
2025-10-09T21:49:03.212326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:03.212331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:03.212333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:03.212340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342342872546525:3461] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:03.212804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342342872546525:3461] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:03.215698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342342872546525:3461] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:03.218758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342342872546525:3461] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:03.218772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342342872546525:3461] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:03.218776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342342872546525:3461] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:03.218784Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1399 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:03.218874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:03.221180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:03.221441Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:03.221452Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:03.221464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:03.221523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:03.221567Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342342510722012:10121]
2025-10-09T21:49:03.222157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:03.222476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:03.222517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:03.222827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:03.222875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.222880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.222882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.222883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.222884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.222885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.222886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.222887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.222889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.222894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.222904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.222908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.222912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.222916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.222921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.222924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.222928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.222934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.222944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.222952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.222958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.222964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.222967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.222972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.222975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.222980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.222984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.222989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.222994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.223000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.223007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.223012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.223018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.223024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.223033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.223038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.223069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:03.229414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:03.230473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:03.230489Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:03.230491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:03.230495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:03.230502Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1398 [2:7559342342510722012:10121]
2025-10-09T21:49:03.230517Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:03.230533Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:03.230536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:03.230539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:03.230541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:03.230556Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:03.230562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:03.230574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:03.230630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.230638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.230640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.230643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:03.230706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:03.230713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.230725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.230736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.230756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:03.230781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342342872546475:2432], now have 1 active actors on pipe
2025-10-09T21:49:03.230789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342342872546696:3461], now have 1 active actors on pipe
2025-10-09T21:49:03.230796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342342510722006:10121]
2025-10-09T21:49:03.230799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342342510722006:10121]
2025-10-09T21:49:03.230801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342342510722006:10121]
2025-10-09T21:49:03.230816Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:03.230832Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:03.230815Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1398
2025-10-09T21:49:03.230834Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:03.230839Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1398
2025-10-09T21:49:03.230843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:03.230846Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:03.230850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:03.230853Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:03.230882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342342510722006:10121]
2025-10-09T21:49:03.230897Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:03.230912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.230915Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:03.230917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.230956Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d85d954f-d36336bb-1e8fbd54-73f0d7f7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:03.230968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:03.230972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:03.230979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:03.230989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:03.231003Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:03.231025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:03.231165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.231172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.231186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:03.231268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1399 partition: 0 MaxSeqNo: 25 sessionId: src_id|d85d954f-d36336bb-1e8fbd54-73f0d7f7_0
2025-10-09T21:49:03.231602Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee0a9bcf-c16c059-91a1facc-f861a4fc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d85d954f-d36336bb-1e8fbd54-73f0d7f7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:03.233997Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d85d954f-d36336bb-1e8fbd54-73f0d7f7_0] Write session: try to update token
2025-10-09T21:49:03.234016Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d85d954f-d36336bb-1e8fbd54-73f0d7f7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:03.245239Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:03.245261Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:03.245263Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:03.245265Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:03.245275Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342342510722006:10121]
2025-10-09T21:49:03.245279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342342510722006:10121]
2025-10-09T21:49:03.245284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342342510722006:10121]
2025-10-09T21:49:03.245325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:03.245333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.245336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:03.245340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:03.245346Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:03.245350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:03.245460Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:03.247131Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1399 sessionId: src_id|d85d954f-d36336bb-1e8fbd54-73f0d7f7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:03.249512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:03.302144Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342342872546713:4303]
2025-10-09T21:49:03.302399Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 1398, pipe: [1:7559342342872546713:4303]
2025-10-09T21:49:03.394237Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:03.394631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.394643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.394680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342342872546713:4303], now have 1 active actors on pipe
2025-10-09T21:49:03.394859Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:03.394934Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:03.394964Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342342872546713:4303]
2025-10-09T21:49:03.394983Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:03.395071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1399 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d85d954f-d36336bb-1e8fbd54-73f0d7f7_0
2025-10-09T21:49:03.395241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1399 sessionId: src_id|d85d954f-d36336bb-1e8fbd54-73f0d7f7_0 is DEAD
2025-10-09T21:49:03.395414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d85d954f-d36336bb-1e8fbd54-73f0d7f7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:03.395469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d85d954f-d36336bb-1e8fbd54-73f0d7f7_0] Write session will restart in 0.005722s
2025-10-09T21:49:03.395511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d85d954f-d36336bb-1e8fbd54-73f0d7f7_0] Write session: Do CDS request
2025-10-09T21:49:03.395518Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d85d954f-d36336bb-1e8fbd54-73f0d7f7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:03.402482Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d85d954f-d36336bb-1e8fbd54-73f0d7f7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:03.402558Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:03.402577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1400
2025-10-09T21:49:03.402741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1400 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:03.402777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1400 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:03.402787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1400 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:03.402791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:03.403205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1400 sessionId:  describe result for acl check
2025-10-09T21:49:03.403246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:03.403253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:03.403254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:03.403258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342342872546742:3454] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:03.403709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342342872546742:3454] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:03.406805Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342342872546742:3454] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:03.409384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342342872546742:3454] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:03.409396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342342872546742:3454] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:03.409399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342342872546742:3454] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:03.409406Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1400 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:03.411203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:03.413305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:03.413472Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:03.413481Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:03.413493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:03.413541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:03.413575Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342342510722237:10123]
2025-10-09T21:49:03.413967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:03.414225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:03.414263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:03.414475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:03.414518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.414526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.414527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.414529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.414530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.414531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.414532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.414533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.414534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.414537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.414544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.414546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.414549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.414551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.414553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.414555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.414557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.414559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.414564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.414568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.414572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.414576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.414579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.414582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.414584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.414587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.414590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.414593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.414595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.414598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.414603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.414607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.414609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.414612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.414614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.414618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.414633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:03.420519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:03.421457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:03.421470Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:03.421472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:03.421475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:03.421480Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1399 [2:7559342342510722237:10123]
2025-10-09T21:49:03.421489Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:03.421504Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:03.421506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:03.421508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:03.421510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:03.421523Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:03.421527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:03.421532Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:03.421576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.421583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.421584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.421586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:03.421650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:03.421656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.421663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.421669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.421679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:03.421697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342342510722232:10123]
2025-10-09T21:49:03.421701Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342342510722232:10123]
2025-10-09T21:49:03.421703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342342510722232:10123]
2025-10-09T21:49:03.421709Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:03.421719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342342872546936:3454], now have 1 active actors on pipe
2025-10-09T21:49:03.421725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342342872546716:2432], now have 1 active actors on pipe
2025-10-09T21:49:03.421728Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:03.421731Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:03.421732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:03.421733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:03.421740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:03.421742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:03.421767Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342342510722232:10123]
2025-10-09T21:49:03.421777Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:03.421792Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:03.421770Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1399
2025-10-09T21:49:03.421773Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1399
2025-10-09T21:49:03.421877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.421884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.421905Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d93fc8ae-d724aed1-9d54c0e3-b6c1d823_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:03.421914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:03.421917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:03.421919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:03.421925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:03.421933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:03.421943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:03.422084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.422099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.422123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:03.422226Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1400 partition: 0 MaxSeqNo: 25 sessionId: src_id|d93fc8ae-d724aed1-9d54c0e3-b6c1d823_0
2025-10-09T21:49:03.422558Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d85d954f-d36336bb-1e8fbd54-73f0d7f7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d93fc8ae-d724aed1-9d54c0e3-b6c1d823_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:03.424937Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d93fc8ae-d724aed1-9d54c0e3-b6c1d823_0] Write session: try to update token
2025-10-09T21:49:03.424958Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d93fc8ae-d724aed1-9d54c0e3-b6c1d823_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:03.435759Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:03.435777Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:03.435780Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:03.435782Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:03.435793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342342510722232:10123]
2025-10-09T21:49:03.435798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342342510722232:10123]
2025-10-09T21:49:03.435801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342342510722232:10123]
2025-10-09T21:49:03.435846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:03.435855Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:03.435858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.435860Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:03.435860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:03.435863Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:03.436009Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:03.437411Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1400 sessionId: src_id|d93fc8ae-d724aed1-9d54c0e3-b6c1d823_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:03.439604Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:03.495187Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342342872546950:4303]
2025-10-09T21:49:03.495448Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 1399, pipe: [1:7559342342872546950:4303]
2025-10-09T21:49:03.577983Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:03.578656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.578671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.578714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342342872546950:4303], now have 1 active actors on pipe
2025-10-09T21:49:03.578946Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:03.578957Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:03.578959Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342342872546950:4303]
2025-10-09T21:49:03.578972Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:03.578979Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:03.579031Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1400 reason: error at write: Disconnected sessionId: src_id|d93fc8ae-d724aed1-9d54c0e3-b6c1d823_0
2025-10-09T21:49:03.579203Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1400 sessionId: src_id|d93fc8ae-d724aed1-9d54c0e3-b6c1d823_0 is DEAD
2025-10-09T21:49:03.579346Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d93fc8ae-d724aed1-9d54c0e3-b6c1d823_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:03.579357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d93fc8ae-d724aed1-9d54c0e3-b6c1d823_0] Write session will restart in 0.009278s
2025-10-09T21:49:03.579384Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d93fc8ae-d724aed1-9d54c0e3-b6c1d823_0] Write session: Do CDS request
2025-10-09T21:49:03.579391Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d93fc8ae-d724aed1-9d54c0e3-b6c1d823_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:03.589520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d93fc8ae-d724aed1-9d54c0e3-b6c1d823_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:03.589574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:03.589592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1401
2025-10-09T21:49:03.589763Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1401 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:03.589795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1401 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:03.589805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1401 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:03.589817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:03.590325Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1401 sessionId:  describe result for acl check
2025-10-09T21:49:03.590364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:03.590370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:03.590371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:03.590377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342342872547032:3449] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:03.590903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342342872547032:3449] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:03.593354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342342872547032:3449] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:03.595514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:03.597701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:03.597900Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:03.597909Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:03.597922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:03.597929Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342342872547032:3449] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:03.597944Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342342872547032:3449] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:03.597948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342342872547032:3449] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:03.597957Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1401 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:03.597975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:03.598026Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342342510722462:10125]
2025-10-09T21:49:03.598629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:03.598878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:03.598916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:03.599219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:03.599265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.599270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.599271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.599273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.599274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.599275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.599276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.599277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.599278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.599283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.599293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.599297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.599301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.599304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.599313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.599322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.599326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.599333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.599340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.599352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.599362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.599371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.599379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.599384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.599392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.599397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.599405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.599411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.599415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.599420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.599425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.599434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.599442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.599446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.599455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.599459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.599491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:03.605179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:03.606306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:03.606318Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:03.606320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:03.606324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:03.606331Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1400 [2:7559342342510722462:10125]
2025-10-09T21:49:03.606344Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:03.606361Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:03.606369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:03.606373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:03.606374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:03.606390Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:03.606402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:03.606409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:03.606458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.606465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.606466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.606468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:03.606519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:03.606524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.606532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.606543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.606552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:03.606566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342342510722457:10125]
2025-10-09T21:49:03.606568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342342510722457:10125]
2025-10-09T21:49:03.606569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342342510722457:10125]
2025-10-09T21:49:03.606575Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:03.606582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:03.606586Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:03.606588Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:03.606592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342342872546953:2432], now have 1 active actors on pipe
2025-10-09T21:49:03.606599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342342872547177:3449], now have 1 active actors on pipe
2025-10-09T21:49:03.606605Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:03.606619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:03.606622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:03.606642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342342510722457:10125]
2025-10-09T21:49:03.606649Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:03.606665Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:03.606672Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1400
2025-10-09T21:49:03.606692Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1400
2025-10-09T21:49:03.606756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.606763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.606788Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|be4b4c3f-5852d4-fe11b797-232dfe34_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:03.606796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:03.606799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:03.606801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:03.606809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:03.606817Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:03.606832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:03.606952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.606958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.606967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:03.607054Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1401 partition: 0 MaxSeqNo: 25 sessionId: src_id|be4b4c3f-5852d4-fe11b797-232dfe34_0
2025-10-09T21:49:03.607386Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d93fc8ae-d724aed1-9d54c0e3-b6c1d823_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|be4b4c3f-5852d4-fe11b797-232dfe34_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:03.609706Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be4b4c3f-5852d4-fe11b797-232dfe34_0] Write session: try to update token
2025-10-09T21:49:03.609722Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be4b4c3f-5852d4-fe11b797-232dfe34_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:03.616884Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:03.616904Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:03.616914Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342342510722457:10125]
2025-10-09T21:49:03.616955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:03.616961Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:03.617107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:03.617238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:03.617358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:03.617486Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:03.617666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:03.617704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:03.622009Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1401 sessionId: src_id|be4b4c3f-5852d4-fe11b797-232dfe34_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:03.624309Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:03.624751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.624768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.624799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:03.624887Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:03.629893Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:03.629910Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:03.629913Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:03.629915Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:03.629924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342342510722457:10125]
2025-10-09T21:49:03.629928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342342510722457:10125]
2025-10-09T21:49:03.629931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342342510722457:10125]
2025-10-09T21:49:03.629973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:03.629979Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:03.629984Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:03.629985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.629987Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:03.629988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:03.630134Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:03.678958Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342342872547188:4303]
2025-10-09T21:49:03.679187Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 1400, pipe: [1:7559342342872547188:4303]
2025-10-09T21:49:03.775289Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:03.781848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.781871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.782016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.782088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.782149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.782207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.782248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.782257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.782328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.782382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.782441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.782506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.782663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.782672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.782729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.782781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.782841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.782896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.782935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.782941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.783000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.783067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.783124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.783180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.783220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.783227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.783286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.783348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.783404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.783459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.783498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.783505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.783566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.783622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.783675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.783729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.783770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.783775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.783841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.783894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.783950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.784004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.784044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.784052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.784120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.784172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.784226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.784279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.784324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.784332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.784393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.784442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.784494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.784550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.784597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.784605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.784662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:03.784714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:03.784771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:03.784827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:03.784871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:03.784877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:03.784921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342342872547188:4303], now have 1 active actors on pipe
2025-10-09T21:49:03.785135Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342342872547188:4303]
2025-10-09T21:49:03.785158Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:03.785163Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:03.785170Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:03.785193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1401 reason: error at write: Disconnected sessionId: src_id|be4b4c3f-5852d4-fe11b797-232dfe34_0
2025-10-09T21:49:03.785196Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:49:03.785282Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 14 consumer shared/user session shared/user_1_14_17138514154924300407_v1 is DEAD
2025-10-09T21:49:03.785299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1401 sessionId: src_id|be4b4c3f-5852d4-fe11b797-232dfe34_0 is DEAD
2025-10-09T21:49:03.785388Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342261268149068:4306] disconnected.
2025-10-09T21:49:03.785408Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342261268149068:4306] disconnected; active server actors: 1
2025-10-09T21:49:03.785416Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342261268149068:4306] client user disconnected session shared/user_1_14_17138514154924300407_v1
2025-10-09T21:49:03.785544Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:49:03.785544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be4b4c3f-5852d4-fe11b797-232dfe34_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:03.785569Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be4b4c3f-5852d4-fe11b797-232dfe34_0] Write session will restart in 0.009611s
2025-10-09T21:49:03.785593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be4b4c3f-5852d4-fe11b797-232dfe34_0] Write session: Do CDS request
2025-10-09T21:49:03.785599Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be4b4c3f-5852d4-fe11b797-232dfe34_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:03.785701Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:49:03.785706Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:49:03.785712Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.007901s
2025-10-09T21:49:03.785729Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:03.794393Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:49:03.794581Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:49:03.794596Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 15
2025-10-09T21:49:03.794757Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 15 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:49:03.794802Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:49:03.794877Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 auth for : user
2025-10-09T21:49:03.795057Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 Handle describe topics response
2025-10-09T21:49:03.795076Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 auth is DEAD
2025-10-09T21:49:03.795081Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:49:03.795326Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:49:03.795425Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_15_12240953663583783860_v1
2025-10-09T21:49:03.795425Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342342872547263:3432] connected; active server actors: 1
2025-10-09T21:49:03.795444Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559342342872547263:3432] session shared/user_1_15_12240953663583783860_v1
2025-10-09T21:49:03.795455Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:49:03.795470Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:49:03.795481Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_15_12240953663583783860_v1" (Sender=[1:7559342342872547245:3432], Pipe=[1:7559342342872547263:3432], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:49:03.795485Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:49:03.795496Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:49:03.795504Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_15_12240953663583783860_v1" (Sender=[1:7559342342872547245:3432], Pipe=[1:7559342342872547263:3432], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:49:03.795527Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_15_12240953663583783860_v1" sender [1:7559342342872547245:3432] lock partition 0 for ReadingSession "shared/user_1_15_12240953663583783860_v1" (Sender=[1:7559342342872547245:3432], Pipe=[1:7559342342872547263:3432], Partitions=[], ActiveFamilyCount=1) generation 1 step 15
2025-10-09T21:49:03.795541Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:49:03.795551Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000052s
2025-10-09T21:49:03.795734Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 15 Session: "shared/user_1_15_12240953663583783860_v1" ClientId: "user" PipeClient { RawX1: 7559342342872547263 RawX2: 4503603922341224 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:49:03.795759Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:49:03.796382Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be4b4c3f-5852d4-fe11b797-232dfe34_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:03.796444Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:03.796454Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1402
2025-10-09T21:49:03.796564Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1402 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:03.796595Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1402 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:03.796604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1402 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:03.796607Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:03.796914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1402 sessionId:  describe result for acl check
2025-10-09T21:49:03.796949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:03.796956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:03.796957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:03.796961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342342872547317:3442] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:03.797402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342342872547317:3442] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:03.799456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342342872547317:3442] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:03.800309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:03.802535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:03.802753Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:03.802761Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:03.802770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:03.802777Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342342872547317:3442] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:03.802790Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342342872547317:3442] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:03.802794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342342872547317:3442] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:03.802801Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1402 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:03.802820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:03.802868Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342342510722695:10129]
2025-10-09T21:49:03.803255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:03.803572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:03.803637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:03.803901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:03.803949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.803955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.803956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.803957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.803958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.803959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.803959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.803960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.803961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.803966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.803977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.803982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.803985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.803988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.803991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.803995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.803998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.804001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.804008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.804020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.804034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.804039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.804043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.804047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.804051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.804056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.804061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.804065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.804069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.804074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.804079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.804085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.804089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.804095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.804113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.804125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.804150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:03.810347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:03.811351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:03.811368Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:03.811370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:03.811375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:03.811381Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1401 [2:7559342342510722695:10129]
2025-10-09T21:49:03.811394Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:03.811412Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:03.811416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:03.811419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:03.811421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:03.811437Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:03.811446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:03.811458Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:03.811517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.811525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.811527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.811530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:03.811593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:03.811600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.811610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.811627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.811652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:03.811675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342342872547275:3441], now have 1 active actors on pipe
2025-10-09T21:49:03.811683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342342872547422:3442], now have 1 active actors on pipe
2025-10-09T21:49:03.811686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342342872547191:2432], now have 1 active actors on pipe
2025-10-09T21:49:03.811693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342342510722690:10129]
2025-10-09T21:49:03.811695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342342510722690:10129]
2025-10-09T21:49:03.811698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342342510722690:10129]
2025-10-09T21:49:03.811711Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:03.811725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:03.811733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:03.811736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:03.811739Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:03.811746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:03.811749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:03.811728Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1401, pipe: [1:7559342342872547275:3441]
2025-10-09T21:49:03.811739Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1401
2025-10-09T21:49:03.811766Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1401
2025-10-09T21:49:03.811778Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342342510722690:10129]
2025-10-09T21:49:03.811788Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:03.811798Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:03.811807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.811811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.811824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_15_12240953663583783860_v1 on pipe: [1:7559342342872547275:3441]
2025-10-09T21:49:03.811846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.811850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.811873Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_15_12240953663583783860_v1:1 with generation 1401
2025-10-09T21:49:03.811882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:03.811889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:03.811891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:03.811906Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_15_12240953663583783860_v1
2025-10-09T21:49:03.811913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:03.811947Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9033826e-77d9d5ad-573b73cb-9e424644_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:03.811958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:49:03.811970Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:49:03.814366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:49:03.814391Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:49:03.814399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:03.814403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:03.814406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:03.814416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:03.814427Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:03.814435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:49:03.814446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:49:03.814465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:03.814570Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046543810 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:49:03.814589Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:49:03.814619Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 sending to client partition status
2025-10-09T21:49:03.814651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.814661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.814678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:03.814756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1402 partition: 0 MaxSeqNo: 25 sessionId: src_id|9033826e-77d9d5ad-573b73cb-9e424644_0
2025-10-09T21:49:03.815041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be4b4c3f-5852d4-fe11b797-232dfe34_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9033826e-77d9d5ad-573b73cb-9e424644_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:03.817299Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9033826e-77d9d5ad-573b73cb-9e424644_0] Write session: try to update token
2025-10-09T21:49:03.817313Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9033826e-77d9d5ad-573b73cb-9e424644_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:03.826064Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:03.826083Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:03.826085Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:03.826087Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:03.826097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342342510722690:10129]
2025-10-09T21:49:03.826101Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342342510722690:10129]
2025-10-09T21:49:03.826105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342342510722690:10129]
2025-10-09T21:49:03.826147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:03.826159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.826166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:03.826183Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:03.826197Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:03.826201Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:03.826296Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:03.829509Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1402 sessionId: src_id|9033826e-77d9d5ad-573b73cb-9e424644_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:03.831746Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:03.974207Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:03.974677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:03.974692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:03.974867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_15_12240953663583783860_v1
2025-10-09T21:49:03.974882Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:03.974911Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_15_12240953663583783860_v1
2025-10-09T21:49:03.974967Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342342872547275:3441]
2025-10-09T21:49:03.975002Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:03.975066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1402 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9033826e-77d9d5ad-573b73cb-9e424644_0
2025-10-09T21:49:03.975241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1402 sessionId: src_id|9033826e-77d9d5ad-573b73cb-9e424644_0 is DEAD
2025-10-09T21:49:03.975421Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9033826e-77d9d5ad-573b73cb-9e424644_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:03.975432Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9033826e-77d9d5ad-573b73cb-9e424644_0] Write session will restart in 0.006531s
2025-10-09T21:49:03.975424Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:03.975450Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9033826e-77d9d5ad-573b73cb-9e424644_0] Write session: Do CDS request
2025-10-09T21:49:03.975498Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9033826e-77d9d5ad-573b73cb-9e424644_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:03.983523Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9033826e-77d9d5ad-573b73cb-9e424644_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:03.983570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:03.983591Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1403
2025-10-09T21:49:03.983782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1403 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:03.983816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1403 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:03.983827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1403 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:03.983831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:03.984196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1403 sessionId:  describe result for acl check
2025-10-09T21:49:03.984230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:03.984237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:03.984239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:03.984244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342342872547485:3421] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:03.984759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342342872547485:3421] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:03.987197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342342872547485:3421] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:03.990094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342342872547485:3421] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:03.990108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342342872547485:3421] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:03.990112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342342872547485:3421] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:03.990119Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1403 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:03.991317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:03.993471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:03.993677Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:03.993688Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:03.993700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:03.993749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:03.993787Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342342510722957:10141]
2025-10-09T21:49:03.994378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:03.994706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:03.994746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:03.995060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:03.995104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.995110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.995112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.995113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.995114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.995115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.995116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.995117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.995119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.995124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.995134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.995138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.995142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.995146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.995151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.995155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.995159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.995163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.995169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.995177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:03.995183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.995189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:03.995194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.995199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:03.995203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.995209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:03.995213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.995218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:03.995223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.995230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:03.995234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.995239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:03.995243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.995254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:03.995259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.995264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:03.995294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:04.001820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:04.002974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:04.002988Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:04.002991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:04.002995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:04.003001Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1402 [2:7559342342510722957:10141]
2025-10-09T21:49:04.003014Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:04.003032Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:04.003035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:04.003038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:04.003040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:04.003056Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:04.003066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:04.003077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:04.003138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.003147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.003149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.003152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:04.003220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:04.003228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.003240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.003258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.003277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:04.003306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342342872547672:3421], now have 1 active actors on pipe
2025-10-09T21:49:04.003309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342342872547453:2432], now have 1 active actors on pipe
2025-10-09T21:49:04.003318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342342510722952:10141]
2025-10-09T21:49:04.003320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342342510722952:10141]
2025-10-09T21:49:04.003322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342342510722952:10141]
2025-10-09T21:49:04.003336Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:04.003354Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:04.003357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:04.003365Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:04.003367Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:04.003373Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:04.003376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:04.003371Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1402
2025-10-09T21:49:04.003387Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1402
2025-10-09T21:49:04.003410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342342510722952:10141]
2025-10-09T21:49:04.003420Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:04.003426Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:04.003437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.003442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.003490Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e913a646-3a08a703-1b163070-ed1d6850_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:04.003503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:04.003506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:04.003509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:04.003519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:04.003530Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:04.003554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:04.003706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.003714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.003730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:04.003830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1403 partition: 0 MaxSeqNo: 25 sessionId: src_id|e913a646-3a08a703-1b163070-ed1d6850_0
2025-10-09T21:49:04.004212Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9033826e-77d9d5ad-573b73cb-9e424644_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e913a646-3a08a703-1b163070-ed1d6850_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:04.006898Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e913a646-3a08a703-1b163070-ed1d6850_0] Write session: try to update token
2025-10-09T21:49:04.006921Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e913a646-3a08a703-1b163070-ed1d6850_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:04.018823Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:04.018842Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:04.018845Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:04.018847Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:04.018857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342342510722952:10141]
2025-10-09T21:49:04.018862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342342510722952:10141]
2025-10-09T21:49:04.018865Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342342510722952:10141]
2025-10-09T21:49:04.018910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:04.018917Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:04.018922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.018924Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:04.018925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:04.018928Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:04.019051Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:04.020583Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1403 sessionId: src_id|e913a646-3a08a703-1b163070-ed1d6850_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:04.023222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:04.075266Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342347167514993:3441]
2025-10-09T21:49:04.075537Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 1402, pipe: [1:7559342347167514993:3441]
2025-10-09T21:49:04.172635Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:04.172883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.172895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.172936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342347167514993:3441], now have 1 active actors on pipe
2025-10-09T21:49:04.173168Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342347167514993:3441]
2025-10-09T21:49:04.173197Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:04.173203Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:04.173207Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:04.173270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1403 reason: error at write: Disconnected sessionId: src_id|e913a646-3a08a703-1b163070-ed1d6850_0
2025-10-09T21:49:04.173323Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:04.173453Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1403 sessionId: src_id|e913a646-3a08a703-1b163070-ed1d6850_0 is DEAD
2025-10-09T21:49:04.173627Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e913a646-3a08a703-1b163070-ed1d6850_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:04.173649Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e913a646-3a08a703-1b163070-ed1d6850_0] Write session will restart in 0.009305s
2025-10-09T21:49:04.173685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e913a646-3a08a703-1b163070-ed1d6850_0] Write session: Do CDS request
2025-10-09T21:49:04.173692Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e913a646-3a08a703-1b163070-ed1d6850_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:04.184626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e913a646-3a08a703-1b163070-ed1d6850_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:04.184714Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:04.184740Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1404
2025-10-09T21:49:04.184883Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1404 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:04.184911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1404 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:04.184919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1404 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:04.184922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:04.185320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1404 sessionId:  describe result for acl check
2025-10-09T21:49:04.185354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:04.185357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:04.185358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:04.185363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342347167515106:3409] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:04.185851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342347167515106:3409] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:04.189079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342347167515106:3409] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:04.189729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:04.192393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:04.192597Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:04.192609Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:04.192621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:04.192672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:04.192713Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342346805690489:10145]
2025-10-09T21:49:04.192706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342347167515106:3409] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:04.192725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342347167515106:3409] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:04.192729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342347167515106:3409] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:04.192739Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1404 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:04.193302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:04.193605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:04.193669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:04.193988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:04.194055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.194061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.194062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.194063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.194064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.194065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.194067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.194068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.194069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.194074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.194084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.194089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.194093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.194096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.194100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.194103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.194107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.194111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.194117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.194133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.194141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.194146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.194152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.194157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.194161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.194166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.194171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.194177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.194181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.194186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.194191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.194196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.194201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.194205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.194210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.194215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.194247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:04.200762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:04.202054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:04.202066Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:04.202069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:04.202074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:04.202081Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1403 [2:7559342346805690489:10145]
2025-10-09T21:49:04.202096Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:04.202117Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:04.202127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:04.202131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:04.202133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:04.202153Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:04.202159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:04.202167Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:04.202233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.202241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.202244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.202246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:04.202311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:04.202318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.202326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.202333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.202342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:04.202363Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342346805690484:10145]
2025-10-09T21:49:04.202370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342346805690484:10145]
2025-10-09T21:49:04.202372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342346805690484:10145]
2025-10-09T21:49:04.202382Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:04.202392Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:04.202400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342347167515225:3409], now have 1 active actors on pipe
2025-10-09T21:49:04.202401Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:04.202404Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:04.202407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:04.202422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:04.202431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:04.202437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342347167514998:2432], now have 1 active actors on pipe
2025-10-09T21:49:04.202475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342346805690484:10145]
2025-10-09T21:49:04.202486Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:04.202492Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:04.202500Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1403
2025-10-09T21:49:04.202503Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1403
2025-10-09T21:49:04.202539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.202552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.202583Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e560d6c5-6c57d9ae-eb69176-650524df_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:04.202598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:04.202602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:04.202609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:04.202617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:04.202631Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:04.202653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:04.202796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.202807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.202823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:04.202910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1404 partition: 0 MaxSeqNo: 25 sessionId: src_id|e560d6c5-6c57d9ae-eb69176-650524df_0
2025-10-09T21:49:04.203270Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e913a646-3a08a703-1b163070-ed1d6850_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e560d6c5-6c57d9ae-eb69176-650524df_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:04.205646Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e560d6c5-6c57d9ae-eb69176-650524df_0] Write session: try to update token
2025-10-09T21:49:04.205662Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e560d6c5-6c57d9ae-eb69176-650524df_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:04.214216Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:04.214234Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:04.214243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342346805690484:10145]
2025-10-09T21:49:04.214291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:04.214297Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:04.214467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:04.214618Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:04.214760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:04.214899Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:04.215093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:04.215142Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:04.218348Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1404 sessionId: src_id|e560d6c5-6c57d9ae-eb69176-650524df_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:04.220808Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:04.221216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.221238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.221279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:04.221364Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:04.227217Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:04.227238Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:04.227241Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:04.227243Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:04.227254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342346805690484:10145]
2025-10-09T21:49:04.227258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342346805690484:10145]
2025-10-09T21:49:04.227262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342346805690484:10145]
2025-10-09T21:49:04.227302Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:04.227303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:04.227311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.227314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:04.227316Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:04.227320Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:04.227447Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:04.273412Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342347167515242:3441]
2025-10-09T21:49:04.273713Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 1403, pipe: [1:7559342347167515242:3441]
2025-10-09T21:49:04.374388Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:04.380855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.380882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.381034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.381106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.381168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.381234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.381754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.381768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.381839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.381901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.381962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.382023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.382082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.382092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.382150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.382204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.382265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.382328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.382371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.382378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.382440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.382502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.382559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.382619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.382659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.382666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.382729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.382788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.382847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.382907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.382948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.382955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.383013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.383070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.383130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.383193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.383235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.383242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.383303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.383358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.383418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.383474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.383515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.383521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.383578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.383631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.383686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.383742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.383782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.383789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.383843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.383896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.383951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.384012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.384053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.384059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.384117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.384171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.384237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.384300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.384344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.384351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.384395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342347167515242:3441], now have 1 active actors on pipe
2025-10-09T21:49:04.384586Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:04.384591Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342347167515242:3441]
2025-10-09T21:49:04.384602Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:04.384613Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:04.384634Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1404 reason: error at write: Disconnected sessionId: src_id|e560d6c5-6c57d9ae-eb69176-650524df_0
2025-10-09T21:49:04.384813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1404 sessionId: src_id|e560d6c5-6c57d9ae-eb69176-650524df_0 is DEAD
2025-10-09T21:49:04.384946Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e560d6c5-6c57d9ae-eb69176-650524df_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:04.384958Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e560d6c5-6c57d9ae-eb69176-650524df_0] Write session will restart in 0.006668s
2025-10-09T21:49:04.384925Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:04.384981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e560d6c5-6c57d9ae-eb69176-650524df_0] Write session: Do CDS request
2025-10-09T21:49:04.384988Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e560d6c5-6c57d9ae-eb69176-650524df_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:04.392548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e560d6c5-6c57d9ae-eb69176-650524df_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:04.392623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:04.392654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1405
2025-10-09T21:49:04.392789Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1405 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:04.392826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1405 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:04.392838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1405 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:04.392846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:04.393312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1405 sessionId:  describe result for acl check
2025-10-09T21:49:04.393347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:04.393355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:04.393357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:04.393362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342347167515270:3398] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:04.393955Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342347167515270:3398] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:04.396957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342347167515270:3398] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:04.400321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342347167515270:3398] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:04.400336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342347167515270:3398] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:04.400342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342347167515270:3398] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:04.400351Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1405 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:04.401453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:04.403986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:04.404158Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:04.404167Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:04.404179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:04.404229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:04.404269Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342346805690720:10147]
2025-10-09T21:49:04.404659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:04.404921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:04.404962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:04.405185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:04.405227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.405233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.405234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.405236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.405237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.405238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.405239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.405240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.405242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.405247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.405259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.405263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.405267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.405271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.405276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.405280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.405284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.405289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.405296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.405308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.405318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.405325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.405333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.405339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.405345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.405351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.405356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.405365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.405369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.405374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.405379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.405389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.405394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.405403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.405408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.405415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.405447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:04.411899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:04.412909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:04.412924Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:04.412927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:04.412931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:04.412938Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1404 [2:7559342346805690720:10147]
2025-10-09T21:49:04.412953Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:04.412974Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:04.412981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:04.412985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:04.412987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:04.413005Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:04.413016Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:04.413028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:04.413091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.413098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.413100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.413103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:04.413174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:04.413181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.413192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.413206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.413228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:04.413254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342347167515470:3398], now have 1 active actors on pipe
2025-10-09T21:49:04.413262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342347167515245:2432], now have 1 active actors on pipe
2025-10-09T21:49:04.413270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342346805690715:10147]
2025-10-09T21:49:04.413276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342346805690715:10147]
2025-10-09T21:49:04.413278Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342346805690715:10147]
2025-10-09T21:49:04.413292Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:04.413310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:04.413312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:04.413316Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:04.413318Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:04.413329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:04.413332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:04.413364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342346805690715:10147]
2025-10-09T21:49:04.413333Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1404
2025-10-09T21:49:04.413372Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:04.413352Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1404
2025-10-09T21:49:04.413374Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:04.413389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.413394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.413445Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|800738f1-eb7ef985-a2dd9e1a-892e3c80_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:04.413457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:04.413465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:04.413468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:04.413479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:04.413493Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:04.413513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:04.413690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.413700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.413717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:04.413800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1405 partition: 0 MaxSeqNo: 25 sessionId: src_id|800738f1-eb7ef985-a2dd9e1a-892e3c80_0
2025-10-09T21:49:04.414130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e560d6c5-6c57d9ae-eb69176-650524df_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|800738f1-eb7ef985-a2dd9e1a-892e3c80_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:04.416298Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|800738f1-eb7ef985-a2dd9e1a-892e3c80_0] Write session: try to update token
2025-10-09T21:49:04.416315Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|800738f1-eb7ef985-a2dd9e1a-892e3c80_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:04.427572Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:04.427589Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:04.427591Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:04.427593Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:04.427603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342346805690715:10147]
2025-10-09T21:49:04.427607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342346805690715:10147]
2025-10-09T21:49:04.427611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342346805690715:10147]
2025-10-09T21:49:04.427653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:04.427660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.427663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:04.427667Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:04.427673Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:04.427678Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:04.427788Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:04.428674Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1405 sessionId: src_id|800738f1-eb7ef985-a2dd9e1a-892e3c80_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:04.430981Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:04.484858Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342347167515484:3441]
2025-10-09T21:49:04.485112Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 1404, pipe: [1:7559342347167515484:3441]
2025-10-09T21:49:04.574745Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:04.575202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.575223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.575272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342347167515484:3441], now have 1 active actors on pipe
2025-10-09T21:49:04.575507Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342347167515484:3441]
2025-10-09T21:49:04.575508Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:04.575511Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:04.575541Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:04.575563Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1405 reason: error at write: Disconnected sessionId: src_id|800738f1-eb7ef985-a2dd9e1a-892e3c80_0
2025-10-09T21:49:04.575715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1405 sessionId: src_id|800738f1-eb7ef985-a2dd9e1a-892e3c80_0 is DEAD
2025-10-09T21:49:04.575851Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|800738f1-eb7ef985-a2dd9e1a-892e3c80_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:04.575831Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:04.575867Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|800738f1-eb7ef985-a2dd9e1a-892e3c80_0] Write session will restart in 0.008879s
2025-10-09T21:49:04.575900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|800738f1-eb7ef985-a2dd9e1a-892e3c80_0] Write session: Do CDS request
2025-10-09T21:49:04.575906Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|800738f1-eb7ef985-a2dd9e1a-892e3c80_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:04.585431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|800738f1-eb7ef985-a2dd9e1a-892e3c80_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:04.585547Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:04.585561Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1406
2025-10-09T21:49:04.585681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1406 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:04.585712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1406 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:04.585718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1406 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:04.585723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:04.586188Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1406 sessionId:  describe result for acl check
2025-10-09T21:49:04.586231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:04.586238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:04.586239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:04.586246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342347167515557:3393] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:04.586787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342347167515557:3393] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:04.589094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342347167515557:3393] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:04.592200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:04.592271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342347167515557:3393] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:04.592282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342347167515557:3393] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:04.592285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342347167515557:3393] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:04.592292Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1406 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:04.594282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:04.594482Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:04.594490Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:04.594501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:04.594552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:04.594591Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342346805690951:10149]
2025-10-09T21:49:04.595160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:04.595504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:04.595549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:04.595864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:04.595913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.595922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.595923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.595924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.595926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.595927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.595928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.595929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.595930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.595935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.595946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.595950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.595954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.595958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.595962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.595966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.595970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.595974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.595981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.595990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.595997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.596003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.596007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.596012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.596017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.596022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.596027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.596032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.596040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.596046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.596051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.596058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.596062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.596067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.596076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.596082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.596113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:04.602382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:04.603665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:04.603681Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:04.603684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:04.603688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:04.603694Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1405 [2:7559342346805690951:10149]
2025-10-09T21:49:04.603707Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:04.603724Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:04.603726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:04.603730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:04.603733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:04.603748Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:04.603770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:04.603781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:04.603834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.603842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.603844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.603847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:04.603910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:04.603917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.603928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.603938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.603960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:04.603987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342347167515715:3393], now have 1 active actors on pipe
2025-10-09T21:49:04.603994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342347167515487:2432], now have 1 active actors on pipe
2025-10-09T21:49:04.604003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342346805690945:10149]
2025-10-09T21:49:04.604009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342346805690945:10149]
2025-10-09T21:49:04.604012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342346805690945:10149]
2025-10-09T21:49:04.604024Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:04.604038Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:04.604042Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:04.604048Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:04.604051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:04.604057Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:04.604060Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:04.604090Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342346805690945:10149]
2025-10-09T21:49:04.604073Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1405
2025-10-09T21:49:04.604098Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:04.604074Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1405
2025-10-09T21:49:04.604102Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:04.604134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.604143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.604184Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|34927a3d-47f0796e-b5d6f87b-31019c2b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:04.604196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:04.604206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:04.604209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:04.604220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:04.604231Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:04.604251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:04.604412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.604423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.604439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:04.604539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1406 partition: 0 MaxSeqNo: 25 sessionId: src_id|34927a3d-47f0796e-b5d6f87b-31019c2b_0
2025-10-09T21:49:04.604886Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|800738f1-eb7ef985-a2dd9e1a-892e3c80_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|34927a3d-47f0796e-b5d6f87b-31019c2b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:04.607158Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|34927a3d-47f0796e-b5d6f87b-31019c2b_0] Write session: try to update token
2025-10-09T21:49:04.607175Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|34927a3d-47f0796e-b5d6f87b-31019c2b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:04.618696Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:04.618710Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:04.618713Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:04.618715Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:04.618725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342346805690945:10149]
2025-10-09T21:49:04.618730Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342346805690945:10149]
2025-10-09T21:49:04.618733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342346805690945:10149]
2025-10-09T21:49:04.618775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:04.618777Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:04.618783Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:04.618786Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:04.618788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.618790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:04.618923Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:04.619495Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1406 sessionId: src_id|34927a3d-47f0796e-b5d6f87b-31019c2b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:04.621970Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:04.675869Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342347167515727:3441]
2025-10-09T21:49:04.676218Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 1405, pipe: [1:7559342347167515727:3441]
2025-10-09T21:49:04.766386Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:04.766768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.766793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.766844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342347167515727:3441], now have 1 active actors on pipe
2025-10-09T21:49:04.767094Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:04.767168Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:04.767184Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342347167515727:3441]
2025-10-09T21:49:04.767186Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:04.767210Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:04.767250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1406 reason: error at write: Disconnected sessionId: src_id|34927a3d-47f0796e-b5d6f87b-31019c2b_0
2025-10-09T21:49:04.767394Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1406 sessionId: src_id|34927a3d-47f0796e-b5d6f87b-31019c2b_0 is DEAD
2025-10-09T21:49:04.767551Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34927a3d-47f0796e-b5d6f87b-31019c2b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:04.767559Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34927a3d-47f0796e-b5d6f87b-31019c2b_0] Write session will restart in 0.006149s
2025-10-09T21:49:04.767584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34927a3d-47f0796e-b5d6f87b-31019c2b_0] Write session: Do CDS request
2025-10-09T21:49:04.767590Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34927a3d-47f0796e-b5d6f87b-31019c2b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:04.774464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|34927a3d-47f0796e-b5d6f87b-31019c2b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:04.774523Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:04.774539Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1407
2025-10-09T21:49:04.774659Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1407 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:04.774701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1407 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:04.774712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1407 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:04.774717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:04.775062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1407 sessionId:  describe result for acl check
2025-10-09T21:49:04.775097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:04.775104Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:04.775106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:04.775112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342347167515746:3394] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:04.775563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342347167515746:3394] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:04.777924Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342347167515746:3394] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:04.782199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342347167515746:3394] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:04.782214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342347167515746:3394] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:04.782217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342347167515746:3394] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:04.782226Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1407 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:04.787994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:04.790627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:04.790809Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:04.790817Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:04.790825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:04.790865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:04.790896Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342346805691189:10153]
2025-10-09T21:49:04.791373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:04.791707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:04.791765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:04.792027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:04.792067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.792072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.792073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.792074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.792075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.792076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.792077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.792077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.792078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.792082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.792089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.792092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.792094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.792096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.792099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.792101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.792103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.792105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.792112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.792120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.792126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.792132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.792136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.792142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.792147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.792151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.792154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.792157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.792159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.792162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.792165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.792168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.792171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.792174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.792176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.792180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.792197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:04.798538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:04.799641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:04.799652Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:04.799654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:04.799658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:04.799663Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1406 [2:7559342346805691189:10153]
2025-10-09T21:49:04.799674Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:04.799689Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:04.799694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:04.799697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:04.799699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:04.799712Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:04.799716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:04.799722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:04.799769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.799777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.799779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.799782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:04.799846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:04.799852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.799859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.799866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.799876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:04.799894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342346805691184:10153]
2025-10-09T21:49:04.799899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342346805691184:10153]
2025-10-09T21:49:04.799900Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342346805691184:10153]
2025-10-09T21:49:04.799909Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:04.799923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342347167515731:2432], now have 1 active actors on pipe
2025-10-09T21:49:04.799925Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:04.799932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:04.799934Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:04.799936Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:04.799941Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:04.799943Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:04.799947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342347167515869:3394], now have 1 active actors on pipe
2025-10-09T21:49:04.799973Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342346805691184:10153]
2025-10-09T21:49:04.799983Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:04.799994Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:04.800011Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1406
2025-10-09T21:49:04.800028Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1406
2025-10-09T21:49:04.800115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.800123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.800154Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|356c31ea-e25ad236-fe6f10d8-c5596860_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:04.800165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:04.800168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:04.800170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:04.800178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:04.800185Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:04.800197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:04.800376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.800382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.800392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:04.800484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1407 partition: 0 MaxSeqNo: 25 sessionId: src_id|356c31ea-e25ad236-fe6f10d8-c5596860_0
2025-10-09T21:49:04.800905Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34927a3d-47f0796e-b5d6f87b-31019c2b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|356c31ea-e25ad236-fe6f10d8-c5596860_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:04.803355Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|356c31ea-e25ad236-fe6f10d8-c5596860_0] Write session: try to update token
2025-10-09T21:49:04.803373Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|356c31ea-e25ad236-fe6f10d8-c5596860_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:04.816476Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:04.816492Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:04.816502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342346805691184:10153]
2025-10-09T21:49:04.816552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:04.816585Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:04.817032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:04.817491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:04.817616Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:04.817763Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:04.817944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:04.817982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:04.818146Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1407 sessionId: src_id|356c31ea-e25ad236-fe6f10d8-c5596860_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:04.821521Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:04.821753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.821770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.821819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:04.821934Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:04.823972Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:04.823984Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:04.823985Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:04.823986Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:04.823994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342346805691184:10153]
2025-10-09T21:49:04.823997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342346805691184:10153]
2025-10-09T21:49:04.824000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342346805691184:10153]
2025-10-09T21:49:04.824035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:04.824046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.824054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:04.824054Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:04.824062Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:04.824074Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:04.824182Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:04.867319Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342347167515978:3441]
2025-10-09T21:49:04.867593Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 1406, pipe: [1:7559342347167515978:3441]
2025-10-09T21:49:04.967098Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:04.973695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:04.973716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:04.973859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.973913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.973968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.974025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.974079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.974087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.974146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.974197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.974250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.974307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.974345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.974350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.974403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.974464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.974517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.974568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.974606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.974612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.974668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.974721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.974775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.974829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.974867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.974872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.974928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.974978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.975031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.975089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.975136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.975143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.975199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.975251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.975303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.975432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.975478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.975486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.975542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.975591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.975645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.975697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.975736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.975742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.975792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.975839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.975890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.975944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.975983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.975986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.976034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.976081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.976131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.976181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.976218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.976223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.976274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:04.976322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:04.976373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:04.976425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:04.976461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:04.976467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:04.976504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342347167515978:3441], now have 1 active actors on pipe
2025-10-09T21:49:04.976731Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342347167515978:3441]
2025-10-09T21:49:04.976735Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:04.976769Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:04.976771Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:04.976798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1407 reason: error at write: Disconnected sessionId: src_id|356c31ea-e25ad236-fe6f10d8-c5596860_0
2025-10-09T21:49:04.976955Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1407 sessionId: src_id|356c31ea-e25ad236-fe6f10d8-c5596860_0 is DEAD
2025-10-09T21:49:04.977117Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|356c31ea-e25ad236-fe6f10d8-c5596860_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:04.977127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|356c31ea-e25ad236-fe6f10d8-c5596860_0] Write session will restart in 0.005168s
2025-10-09T21:49:04.977145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|356c31ea-e25ad236-fe6f10d8-c5596860_0] Write session: Do CDS request
2025-10-09T21:49:04.977150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|356c31ea-e25ad236-fe6f10d8-c5596860_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:04.977262Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:04.983454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|356c31ea-e25ad236-fe6f10d8-c5596860_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:04.983527Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:04.983542Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1408
2025-10-09T21:49:04.983679Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1408 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:04.983712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1408 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:04.983720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1408 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:04.983724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:04.984063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1408 sessionId:  describe result for acl check
2025-10-09T21:49:04.984091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:04.984092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:04.984094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:04.984099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342347167515999:3378] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:04.984560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342347167515999:3378] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:04.987017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342347167515999:3378] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:04.990371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342347167515999:3378] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:04.990386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342347167515999:3378] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:04.990390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342347167515999:3378] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:04.990399Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1408 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:04.994329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:04.996639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:04.996842Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:04.996852Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:04.996863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:04.996915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:04.996952Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342346805691438:10161]
2025-10-09T21:49:04.997533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:04.997901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:04.997942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:04.998263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:04.998307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.998311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.998312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.998313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.998314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.998314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.998315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.998316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.998317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.998320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.998326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.998329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.998331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.998333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.998335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.998337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.998340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.998341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.998347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.998355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:04.998362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.998367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:04.998370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.998375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:04.998379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.998384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:04.998388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.998393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:04.998396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.998401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:04.998407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.998411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:04.998416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.998420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:04.998424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.998429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:04.998457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:05.004888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:05.006095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:05.006107Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:05.006108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:05.006112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:05.006117Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1407 [2:7559342346805691438:10161]
2025-10-09T21:49:05.006128Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:05.006144Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:05.006146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:05.006148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:05.006150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:05.006170Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:05.006181Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:05.006190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:05.006247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.006257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.006259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.006262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:05.006339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:05.006348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.006361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.006381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.006394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:05.006419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342347167515984:2432], now have 1 active actors on pipe
2025-10-09T21:49:05.006428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342347167516128:3378], now have 1 active actors on pipe
2025-10-09T21:49:05.006435Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342346805691430:10161]
2025-10-09T21:49:05.006440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342346805691430:10161]
2025-10-09T21:49:05.006442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342346805691430:10161]
2025-10-09T21:49:05.006453Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:05.006452Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1407
2025-10-09T21:49:05.006493Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1407
2025-10-09T21:49:05.006465Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:05.006471Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:05.006473Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:05.006475Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:05.006479Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:05.006482Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:05.006519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342346805691430:10161]
2025-10-09T21:49:05.006521Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:05.006523Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:05.006541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.006544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.006577Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|554ac743-a39c7b63-99187c1-c3608328_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:05.006591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:05.006597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:05.006600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:05.006610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:05.006621Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:05.006646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:05.006797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.006803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.006814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:05.006903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1408 partition: 0 MaxSeqNo: 25 sessionId: src_id|554ac743-a39c7b63-99187c1-c3608328_0
2025-10-09T21:49:05.007270Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|356c31ea-e25ad236-fe6f10d8-c5596860_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|554ac743-a39c7b63-99187c1-c3608328_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:05.009581Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|554ac743-a39c7b63-99187c1-c3608328_0] Write session: try to update token
2025-10-09T21:49:05.009598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|554ac743-a39c7b63-99187c1-c3608328_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:05.020792Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:05.020813Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:05.020816Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:05.020819Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:05.020829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342346805691430:10161]
2025-10-09T21:49:05.020833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342346805691430:10161]
2025-10-09T21:49:05.020836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342346805691430:10161]
2025-10-09T21:49:05.020879Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:05.020885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:05.020890Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:05.020892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.020893Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:05.020895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:05.021035Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:05.022633Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1408 sessionId: src_id|554ac743-a39c7b63-99187c1-c3608328_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:05.024955Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:05.076732Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342351462483542:3441]
2025-10-09T21:49:05.076999Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 1407, pipe: [1:7559342351462483542:3441]
2025-10-09T21:49:05.168515Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:05.168949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.168964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.169012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342351462483542:3441], now have 1 active actors on pipe
2025-10-09T21:49:05.169258Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:05.169273Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:05.169277Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:05.169325Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342351462483542:3441]
2025-10-09T21:49:05.169336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1408 reason: error at write: Disconnected sessionId: src_id|554ac743-a39c7b63-99187c1-c3608328_0
2025-10-09T21:49:05.169446Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:05.169510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1408 sessionId: src_id|554ac743-a39c7b63-99187c1-c3608328_0 is DEAD
2025-10-09T21:49:05.169676Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|554ac743-a39c7b63-99187c1-c3608328_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:05.169688Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|554ac743-a39c7b63-99187c1-c3608328_0] Write session will restart in 0.007903s
2025-10-09T21:49:05.169731Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|554ac743-a39c7b63-99187c1-c3608328_0] Write session: Do CDS request
2025-10-09T21:49:05.169738Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|554ac743-a39c7b63-99187c1-c3608328_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:05.178505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|554ac743-a39c7b63-99187c1-c3608328_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:05.178556Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:05.178583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1409
2025-10-09T21:49:05.178754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1409 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:05.178790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1409 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:05.178798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1409 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:05.178802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:05.179161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1409 sessionId:  describe result for acl check
2025-10-09T21:49:05.179198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:05.179205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:05.179207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:05.179214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342351462483583:3372] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:05.179650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342351462483583:3372] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:05.182070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342351462483583:3372] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:05.185005Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342351462483583:3372] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:05.185020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342351462483583:3372] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:05.185024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342351462483583:3372] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:05.185032Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1409 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:05.186397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:05.188729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:05.188925Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:05.188937Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:05.188949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:05.189003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:05.189042Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342351100658990:10169]
2025-10-09T21:49:05.189656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:05.189954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:05.189998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:05.190267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:05.190313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.190318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.190319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.190320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.190321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.190323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.190324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.190325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.190326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.190331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.190342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.190346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.190351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.190355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.190359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.190363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.190366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.190368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.190373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.190378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.190383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.190386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.190389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.190392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.190395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.190398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.190400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.190403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.190405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.190408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.190411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.190414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.190417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.190420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.190422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.190425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.190452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:05.196558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:05.197585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:05.197599Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:05.197601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:05.197606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:05.197612Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1408 [2:7559342351100658990:10169]
2025-10-09T21:49:05.197625Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:05.197657Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:05.197665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:05.197668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:05.197670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:05.197686Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:05.197697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:05.197704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:05.197763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.197772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.197774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.197776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:05.197847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:05.197856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.197866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.197881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.197897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:05.197919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342351462483777:3372], now have 1 active actors on pipe
2025-10-09T21:49:05.197926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342351462483547:2432], now have 1 active actors on pipe
2025-10-09T21:49:05.197935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342351100658985:10169]
2025-10-09T21:49:05.197941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342351100658985:10169]
2025-10-09T21:49:05.197943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342351100658985:10169]
2025-10-09T21:49:05.197957Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:05.197970Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:05.197951Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1408
2025-10-09T21:49:05.197966Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1408
2025-10-09T21:49:05.197977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:05.197978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:05.197981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:05.197991Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:05.197994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:05.198047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.198058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.198075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342351100658985:10169]
2025-10-09T21:49:05.198086Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:05.198093Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:05.198116Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e7bd0415-bb3e4807-7f0314bb-e0799448_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:05.198130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:05.198134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:05.198137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:05.198148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:05.198158Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:05.198181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:05.198315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.198325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.198341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:05.198427Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1409 partition: 0 MaxSeqNo: 25 sessionId: src_id|e7bd0415-bb3e4807-7f0314bb-e0799448_0
2025-10-09T21:49:05.198772Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|554ac743-a39c7b63-99187c1-c3608328_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e7bd0415-bb3e4807-7f0314bb-e0799448_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:05.201016Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e7bd0415-bb3e4807-7f0314bb-e0799448_0] Write session: try to update token
2025-10-09T21:49:05.201030Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e7bd0415-bb3e4807-7f0314bb-e0799448_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:05.212645Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:05.212667Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:05.212669Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:05.212672Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:05.212682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342351100658985:10169]
2025-10-09T21:49:05.212687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342351100658985:10169]
2025-10-09T21:49:05.212690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342351100658985:10169]
2025-10-09T21:49:05.212732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:05.212735Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:05.212738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.212740Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:05.212741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:05.212743Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:05.212886Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:05.213422Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1409 sessionId: src_id|e7bd0415-bb3e4807-7f0314bb-e0799448_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:05.215733Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:05.269194Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342351462483796:3441]
2025-10-09T21:49:05.269447Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 1408, pipe: [1:7559342351462483796:3441]
2025-10-09T21:49:05.367782Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:05.368472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.368491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.368538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342351462483796:3441], now have 1 active actors on pipe
2025-10-09T21:49:05.368739Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:05.368830Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:05.368891Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342351462483796:3441]
2025-10-09T21:49:05.368995Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:05.369058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1409 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e7bd0415-bb3e4807-7f0314bb-e0799448_0
2025-10-09T21:49:05.369236Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1409 sessionId: src_id|e7bd0415-bb3e4807-7f0314bb-e0799448_0 is DEAD
2025-10-09T21:49:05.369390Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e7bd0415-bb3e4807-7f0314bb-e0799448_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:05.369399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e7bd0415-bb3e4807-7f0314bb-e0799448_0] Write session will restart in 0.007197s
2025-10-09T21:49:05.369419Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e7bd0415-bb3e4807-7f0314bb-e0799448_0] Write session: Do CDS request
2025-10-09T21:49:05.369424Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e7bd0415-bb3e4807-7f0314bb-e0799448_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:05.377472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e7bd0415-bb3e4807-7f0314bb-e0799448_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:05.377527Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:05.377540Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1410
2025-10-09T21:49:05.377687Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1410 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:05.377728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1410 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:05.377736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1410 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:05.377739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:05.378106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1410 sessionId:  describe result for acl check
2025-10-09T21:49:05.378136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:05.378137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:05.378139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:05.378144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342351462483840:3361] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:05.378659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342351462483840:3361] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:05.381372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342351462483840:3361] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:05.383777Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342351462483840:3361] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:05.383791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342351462483840:3361] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:05.383796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342351462483840:3361] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:05.383804Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1410 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:05.384989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:05.387313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:05.387461Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:05.387471Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:05.387482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:05.387527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:05.387560Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342351100659225:10171]
2025-10-09T21:49:05.387933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:05.388195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:05.388233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:05.388455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:05.388490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.388496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.388498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.388498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.388500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.388501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.388502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.388503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.388505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.388510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.388518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.388520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.388522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.388524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.388526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.388529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.388530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.388533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.388537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.388543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.388546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.388550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.388552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.388555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.388558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.388561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.388564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.388567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.388574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.388577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.388580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.388583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.388585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.388589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.388592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.388595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.388612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:05.394531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:05.395648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:05.395662Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:05.395665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:05.395669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:05.395675Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1409 [2:7559342351100659225:10171]
2025-10-09T21:49:05.395687Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:05.395706Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:05.395713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:05.395716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:05.395719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:05.395735Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:05.395745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:05.395756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:05.395812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.395819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.395821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.395824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:05.395888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:05.395896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.395908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.395918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.395935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:05.395957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342351462483799:2432], now have 1 active actors on pipe
2025-10-09T21:49:05.395965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342351462484029:3361], now have 1 active actors on pipe
2025-10-09T21:49:05.395973Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342351100659220:10171]
2025-10-09T21:49:05.395977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342351100659220:10171]
2025-10-09T21:49:05.395979Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342351100659220:10171]
2025-10-09T21:49:05.395990Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:05.396008Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:05.396010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:05.396018Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:05.396020Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:05.396023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:05.396026Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:05.396048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342351100659220:10171]
2025-10-09T21:49:05.396051Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:05.396055Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:05.396039Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1409
2025-10-09T21:49:05.396048Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1409
2025-10-09T21:49:05.396083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.396096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.396130Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|101e9f77-98e2a52c-9f9a4fd6-a36956a5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:05.396142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:05.396147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:05.396153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:05.396164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:05.396175Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:05.396196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:05.396340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.396348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.396362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:05.396460Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1410 partition: 0 MaxSeqNo: 25 sessionId: src_id|101e9f77-98e2a52c-9f9a4fd6-a36956a5_0
2025-10-09T21:49:05.396754Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e7bd0415-bb3e4807-7f0314bb-e0799448_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|101e9f77-98e2a52c-9f9a4fd6-a36956a5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:05.399051Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|101e9f77-98e2a52c-9f9a4fd6-a36956a5_0] Write session: try to update token
2025-10-09T21:49:05.399068Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|101e9f77-98e2a52c-9f9a4fd6-a36956a5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:05.409794Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:05.409814Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:05.409816Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:05.409818Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:05.409828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342351100659220:10171]
2025-10-09T21:49:05.409833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342351100659220:10171]
2025-10-09T21:49:05.409837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342351100659220:10171]
2025-10-09T21:49:05.409879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:05.409886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.409887Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:05.409889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:05.409891Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:05.409894Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:05.410054Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:05.411590Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1410 sessionId: src_id|101e9f77-98e2a52c-9f9a4fd6-a36956a5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:05.413967Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:05.468909Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342351462484043:3441]
2025-10-09T21:49:05.469122Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 1409, pipe: [1:7559342351462484043:3441]
2025-10-09T21:49:05.553662Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:05.553911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.553923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.553959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342351462484043:3441], now have 1 active actors on pipe
2025-10-09T21:49:05.554129Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:05.554149Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:05.554182Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342351462484043:3441]
2025-10-09T21:49:05.554194Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:05.554256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1410 reason: status is not ok: tablet will be restarted right now sessionId: src_id|101e9f77-98e2a52c-9f9a4fd6-a36956a5_0
2025-10-09T21:49:05.554414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1410 sessionId: src_id|101e9f77-98e2a52c-9f9a4fd6-a36956a5_0 is DEAD
2025-10-09T21:49:05.554585Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|101e9f77-98e2a52c-9f9a4fd6-a36956a5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:05.554595Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|101e9f77-98e2a52c-9f9a4fd6-a36956a5_0] Write session will restart in 0.006553s
2025-10-09T21:49:05.554615Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|101e9f77-98e2a52c-9f9a4fd6-a36956a5_0] Write session: Do CDS request
2025-10-09T21:49:05.554620Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|101e9f77-98e2a52c-9f9a4fd6-a36956a5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:05.562514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|101e9f77-98e2a52c-9f9a4fd6-a36956a5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:05.562561Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:05.562583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1411
2025-10-09T21:49:05.562733Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1411 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:05.562767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1411 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:05.562772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1411 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:05.562775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:05.563210Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1411 sessionId:  describe result for acl check
2025-10-09T21:49:05.563245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:05.563252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:05.563254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:05.563260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342351462484094:3356] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:05.563767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342351462484094:3356] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:05.566355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342351462484094:3356] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:05.568677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342351462484094:3356] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:05.568689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342351462484094:3356] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:05.568692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342351462484094:3356] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:05.568696Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1411 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:05.569728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:05.571870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:05.572034Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:05.572044Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:05.572056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:05.572100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:05.572136Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342351100659461:10173]
2025-10-09T21:49:05.572679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:05.572937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:05.572974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:05.573263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:05.573303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.573308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.573309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.573310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.573312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.573313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.573314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.573315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.573316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.573320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.573329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.573333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.573337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.573343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.573346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.573354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.573360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.573364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.573370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.573381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.573390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.573395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.573402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.573407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.573410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.573415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.573419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.573426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.573433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.573442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.573450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.573456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.573464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.573469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.573473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.573478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.573510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:05.579409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:05.580657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:05.580673Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:05.580675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:05.580679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:05.580684Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1410 [2:7559342351100659461:10173]
2025-10-09T21:49:05.580695Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:05.580710Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:05.580719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:05.580722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:05.580724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:05.580740Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:05.580746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:05.580752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:05.580803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.580809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.580811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.580814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:05.580873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:05.580882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.580893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.580903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.580919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:05.580940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342351462484277:3356], now have 1 active actors on pipe
2025-10-09T21:49:05.580947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342351462484046:2432], now have 1 active actors on pipe
2025-10-09T21:49:05.580955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342351100659456:10173]
2025-10-09T21:49:05.580961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342351100659456:10173]
2025-10-09T21:49:05.580963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342351100659456:10173]
2025-10-09T21:49:05.580975Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:05.580994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:05.580988Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1410
2025-10-09T21:49:05.581002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:05.580994Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1410
2025-10-09T21:49:05.581008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:05.581011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:05.581016Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:05.581018Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:05.581031Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342351100659456:10173]
2025-10-09T21:49:05.581034Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:05.581039Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:05.581065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.581069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.581102Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b6ef2a39-ece217f9-e67459b8-f36ce274_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:05.581115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:05.581118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:05.581121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:05.581132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:05.581140Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:05.581157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:05.581322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.581331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.581345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:05.581434Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1411 partition: 0 MaxSeqNo: 25 sessionId: src_id|b6ef2a39-ece217f9-e67459b8-f36ce274_0
2025-10-09T21:49:05.581739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|101e9f77-98e2a52c-9f9a4fd6-a36956a5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b6ef2a39-ece217f9-e67459b8-f36ce274_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:05.584073Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6ef2a39-ece217f9-e67459b8-f36ce274_0] Write session: try to update token
2025-10-09T21:49:05.584087Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6ef2a39-ece217f9-e67459b8-f36ce274_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:05.595190Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:05.595211Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:05.595213Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:05.595215Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:05.595225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342351100659456:10173]
2025-10-09T21:49:05.595229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342351100659456:10173]
2025-10-09T21:49:05.595234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342351100659456:10173]
2025-10-09T21:49:05.595278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:05.595282Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:05.595285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.595287Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:05.595288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:05.595290Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:05.595422Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:05.596485Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1411 sessionId: src_id|b6ef2a39-ece217f9-e67459b8-f36ce274_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:05.598688Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:05.654258Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342351462484291:3441]
2025-10-09T21:49:05.654482Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 1410, pipe: [1:7559342351462484291:3441]
2025-10-09T21:49:05.739764Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:05.740076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.740096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.740144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342351462484291:3441], now have 1 active actors on pipe
2025-10-09T21:49:05.740362Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:05.740376Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342351462484291:3441]
2025-10-09T21:49:05.740377Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:05.740395Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:05.740405Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:05.740427Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1411 reason: error at write: Disconnected sessionId: src_id|b6ef2a39-ece217f9-e67459b8-f36ce274_0
2025-10-09T21:49:05.740582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1411 sessionId: src_id|b6ef2a39-ece217f9-e67459b8-f36ce274_0 is DEAD
2025-10-09T21:49:05.740734Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6ef2a39-ece217f9-e67459b8-f36ce274_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:05.740758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6ef2a39-ece217f9-e67459b8-f36ce274_0] Write session will restart in 0.009754s
2025-10-09T21:49:05.740778Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6ef2a39-ece217f9-e67459b8-f36ce274_0] Write session: Do CDS request
2025-10-09T21:49:05.740783Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6ef2a39-ece217f9-e67459b8-f36ce274_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:05.751595Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6ef2a39-ece217f9-e67459b8-f36ce274_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:05.751667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:05.751687Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1412
2025-10-09T21:49:05.751852Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1412 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:05.751905Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1412 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:05.751912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1412 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:05.751916Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:05.752286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1412 sessionId:  describe result for acl check
2025-10-09T21:49:05.752313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:05.752314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:05.752316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:05.752320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342351462484410:3348] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:05.752747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342351462484410:3348] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:05.755083Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342351462484410:3348] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:05.756300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:05.758519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:05.758642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342351462484410:3348] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:05.758656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342351462484410:3348] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:05.758659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342351462484410:3348] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:05.758667Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1412 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:05.758690Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:05.758703Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:05.758715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:05.758770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:05.758809Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342351100659701:10177]
2025-10-09T21:49:05.759366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:05.759647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:05.759691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:05.759992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:05.760036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.760043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.760044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.760045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.760046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.760047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.760048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.760049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.760050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.760055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.760064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.760068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.760072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.760075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.760078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.760082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.760085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.760089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.760095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.760102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.760108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.760113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.760118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.760123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.760127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.760132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.760136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.760146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.760150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.760156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.760165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.760170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.760176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.760181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.760186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.760193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.760223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:05.766285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:05.767560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:05.767576Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:05.767578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:05.767582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:05.767588Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1411 [2:7559342351100659701:10177]
2025-10-09T21:49:05.767602Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:05.767619Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:05.767622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:05.767625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:05.767627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:05.767644Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:05.767654Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:05.767661Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:05.767717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.767724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.767726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.767729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:05.767791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:05.767799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.767810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.767826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.767846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:05.767872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342351462484294:2432], now have 1 active actors on pipe
2025-10-09T21:49:05.767880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342351462484529:3348], now have 1 active actors on pipe
2025-10-09T21:49:05.767888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342351100659696:10177]
2025-10-09T21:49:05.767895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342351100659696:10177]
2025-10-09T21:49:05.767898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342351100659696:10177]
2025-10-09T21:49:05.767910Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:05.767924Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:05.767894Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1411
2025-10-09T21:49:05.767928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:05.767935Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:05.767937Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:05.767942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:05.767946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:05.767973Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342351100659696:10177]
2025-10-09T21:49:05.767986Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:05.767982Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1411
2025-10-09T21:49:05.767996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:05.768023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.768035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.768065Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b1d93cfb-ae025db5-a495debf-4219c054_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:05.768073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:05.768081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:05.768084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:05.768095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:05.768112Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:05.768130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:05.768285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.768295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.768310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:05.768377Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1412 partition: 0 MaxSeqNo: 25 sessionId: src_id|b1d93cfb-ae025db5-a495debf-4219c054_0
2025-10-09T21:49:05.768685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6ef2a39-ece217f9-e67459b8-f36ce274_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b1d93cfb-ae025db5-a495debf-4219c054_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:05.771056Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1d93cfb-ae025db5-a495debf-4219c054_0] Write session: try to update token
2025-10-09T21:49:05.771072Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1d93cfb-ae025db5-a495debf-4219c054_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:05.782726Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:05.782746Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:05.782749Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:05.782751Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:05.782761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342351100659696:10177]
2025-10-09T21:49:05.782766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342351100659696:10177]
2025-10-09T21:49:05.782770Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342351100659696:10177]
2025-10-09T21:49:05.782814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:05.782820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.782823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:05.782825Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:05.782832Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:05.782836Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:05.782951Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:05.784074Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1412 sessionId: src_id|b1d93cfb-ae025db5-a495debf-4219c054_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:05.786570Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:05.840638Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342351462484562:3441]
2025-10-09T21:49:05.840911Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 1411, pipe: [1:7559342351462484562:3441]
2025-10-09T21:49:05.925642Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:05.926006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.926019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.926079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342351462484562:3441], now have 1 active actors on pipe
2025-10-09T21:49:05.926247Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:05.926281Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:05.926295Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:05.926307Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:05.926345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1412 reason: error at write: Disconnected sessionId: src_id|b1d93cfb-ae025db5-a495debf-4219c054_0
2025-10-09T21:49:05.926349Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342351462484562:3441]
2025-10-09T21:49:05.926554Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1412 sessionId: src_id|b1d93cfb-ae025db5-a495debf-4219c054_0 is DEAD
2025-10-09T21:49:05.926673Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1d93cfb-ae025db5-a495debf-4219c054_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:05.926685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1d93cfb-ae025db5-a495debf-4219c054_0] Write session will restart in 0.009738s
2025-10-09T21:49:05.926712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1d93cfb-ae025db5-a495debf-4219c054_0] Write session: Do CDS request
2025-10-09T21:49:05.926719Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1d93cfb-ae025db5-a495debf-4219c054_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:05.937536Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1d93cfb-ae025db5-a495debf-4219c054_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:05.937643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:05.937669Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1413
2025-10-09T21:49:05.937788Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1413 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:05.937823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1413 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:05.937828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1413 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:05.937833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:05.938313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1413 sessionId:  describe result for acl check
2025-10-09T21:49:05.938340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:05.938345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:05.938346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:05.938352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342351462484707:3344] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:05.938796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342351462484707:3344] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:05.940771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342351462484707:3344] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:05.941272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:05.943598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:05.943704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342351462484707:3344] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:05.943716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342351462484707:3344] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:05.943719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342351462484707:3344] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:05.943725Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1413 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:05.943789Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:05.943800Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:05.943812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:05.943870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:05.943907Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342351100659939:10179]
2025-10-09T21:49:05.944439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:05.944717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:05.944762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:05.945033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:05.945088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.945093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.945094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.945095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.945096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.945096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.945097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.945097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.945098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.945102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.945108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.945111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.945113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.945115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.945117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.945119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.945121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.945123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.945128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.945133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:05.945136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.945140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:05.945142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.945145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:05.945148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.945151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:05.945153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.945156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:05.945159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.945162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:05.945165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.945168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.945170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.945173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.945176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.945179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.945194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:05.951193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:05.952239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:05.952257Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:05.952260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:05.952264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:05.952274Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1412 [2:7559342351100659939:10179]
2025-10-09T21:49:05.952287Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:05.952307Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:05.952315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:05.952318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:05.952321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:05.952338Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:05.952348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:05.952356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:05.952413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.952419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.952421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.952424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:05.952612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:05.952621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.952633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:05.952649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:05.952671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:05.952653Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1412
2025-10-09T21:49:05.952656Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1412
2025-10-09T21:49:05.952697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342351462484801:3344], now have 1 active actors on pipe
2025-10-09T21:49:05.952705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342351462484565:2432], now have 1 active actors on pipe
2025-10-09T21:49:05.952713Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342351100659934:10179]
2025-10-09T21:49:05.952717Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342351100659934:10179]
2025-10-09T21:49:05.952720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342351100659934:10179]
2025-10-09T21:49:05.952735Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:05.952748Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:05.952753Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:05.952758Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:05.952767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:05.952769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:05.952770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:05.952788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.952796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.952847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342351100659934:10179]
2025-10-09T21:49:05.952858Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:05.952874Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:05.952876Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4b019c1b-66e1deca-ff10282f-3590ef0c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:05.952883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:05.952887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:05.952889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:05.952899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:05.952910Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:05.952942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:05.953064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:05.953071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:05.953083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:05.953138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1413 partition: 0 MaxSeqNo: 25 sessionId: src_id|4b019c1b-66e1deca-ff10282f-3590ef0c_0
2025-10-09T21:49:05.953463Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1d93cfb-ae025db5-a495debf-4219c054_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4b019c1b-66e1deca-ff10282f-3590ef0c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:05.955765Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b019c1b-66e1deca-ff10282f-3590ef0c_0] Write session: try to update token
2025-10-09T21:49:05.955780Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b019c1b-66e1deca-ff10282f-3590ef0c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:05.966502Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:05.966521Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:05.966524Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:05.966527Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:05.966536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342351100659934:10179]
2025-10-09T21:49:05.966540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342351100659934:10179]
2025-10-09T21:49:05.966545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342351100659934:10179]
2025-10-09T21:49:05.966582Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:05.966587Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:05.966588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:05.966589Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:05.966594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:05.966597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:05.966720Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:05.967600Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1413 sessionId: src_id|4b019c1b-66e1deca-ff10282f-3590ef0c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:05.969776Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:06.026490Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342355757452109:3441]
2025-10-09T21:49:06.026808Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 1412, pipe: [1:7559342355757452109:3441]
2025-10-09T21:49:06.115154Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:06.115683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:06.115700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:06.115743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342355757452109:3441], now have 1 active actors on pipe
2025-10-09T21:49:06.115984Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:06.115985Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342355757452109:3441]
2025-10-09T21:49:06.115999Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:06.116005Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:06.116066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1413 reason: error at write: Disconnected sessionId: src_id|4b019c1b-66e1deca-ff10282f-3590ef0c_0
2025-10-09T21:49:06.116088Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:06.116266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1413 sessionId: src_id|4b019c1b-66e1deca-ff10282f-3590ef0c_0 is DEAD
2025-10-09T21:49:06.116414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b019c1b-66e1deca-ff10282f-3590ef0c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:06.116428Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b019c1b-66e1deca-ff10282f-3590ef0c_0] Write session will restart in 0.009969s
2025-10-09T21:49:06.116453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b019c1b-66e1deca-ff10282f-3590ef0c_0] Write session: Do CDS request
2025-10-09T21:49:06.116460Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b019c1b-66e1deca-ff10282f-3590ef0c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:06.127504Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b019c1b-66e1deca-ff10282f-3590ef0c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:06.127570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:06.127587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1414
2025-10-09T21:49:06.127721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1414 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:06.127766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1414 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:06.127775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1414 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:06.127779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:06.128379Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1414 sessionId:  describe result for acl check
2025-10-09T21:49:06.128411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:06.128417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:06.128418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:06.128424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342355757452246:3313] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:06.128968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342355757452246:3313] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:06.131975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342355757452246:3313] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:06.132746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:06.135182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:06.135382Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:06.135391Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:06.135403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:06.135454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:06.135470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342355757452246:3313] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:06.135485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342355757452246:3313] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:06.135489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342355757452246:3313] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:06.135493Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342355395627513:10193]
2025-10-09T21:49:06.135498Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1414 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:06.136090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:06.136356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:06.136393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:06.136699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:06.136747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.136753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.136754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.136755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.136757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.136758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.136759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.136760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.136762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.136766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.136778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.136782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.136787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.136791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.136796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.136800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.136804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.136808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.136814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.136822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.136828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.136834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.136838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.136843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.136848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.136856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.136861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.136869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.136873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.136878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.136885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.136890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.136895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.136905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.136910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.136915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.136944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:06.143370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:06.144869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:06.144882Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:06.144885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:06.144889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:06.144896Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1413 [2:7559342355395627513:10193]
2025-10-09T21:49:06.144909Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:06.144928Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:06.144936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:06.144939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:06.144941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:06.144959Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:06.144969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:06.144976Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:06.145030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.145039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.145041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.145044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:06.145111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:06.145118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.145131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.145148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.145169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:06.145196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342355757452369:3313], now have 1 active actors on pipe
2025-10-09T21:49:06.145203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342355757452135:2432], now have 1 active actors on pipe
2025-10-09T21:49:06.145211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342355395627508:10193]
2025-10-09T21:49:06.145213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342355395627508:10193]
2025-10-09T21:49:06.145215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342355395627508:10193]
2025-10-09T21:49:06.145228Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:06.145242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:06.145228Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1413
2025-10-09T21:49:06.145261Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:06.145247Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1413
2025-10-09T21:49:06.145260Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:06.145265Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:06.145267Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:06.145268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:06.145311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:06.145320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:06.145336Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342355395627508:10193]
2025-10-09T21:49:06.145348Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:06.145359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:06.145382Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1ee58cad-cffb9085-8c70128f-ec405eed_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:06.145392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:06.145400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:06.145403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:06.145413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:06.145423Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:06.145444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:06.145587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:06.145594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:06.145608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:06.145718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1414 partition: 0 MaxSeqNo: 25 sessionId: src_id|1ee58cad-cffb9085-8c70128f-ec405eed_0
2025-10-09T21:49:06.146062Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b019c1b-66e1deca-ff10282f-3590ef0c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1ee58cad-cffb9085-8c70128f-ec405eed_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:06.148561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ee58cad-cffb9085-8c70128f-ec405eed_0] Write session: try to update token
2025-10-09T21:49:06.148578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ee58cad-cffb9085-8c70128f-ec405eed_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:06.159739Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:06.159758Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:06.159760Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:06.159761Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:06.159770Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342355395627508:10193]
2025-10-09T21:49:06.159773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342355395627508:10193]
2025-10-09T21:49:06.159776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342355395627508:10193]
2025-10-09T21:49:06.159818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:06.159823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.159825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:06.159834Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:06.159841Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:06.159845Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:06.159953Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:06.162002Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1414 sessionId: src_id|1ee58cad-cffb9085-8c70128f-ec405eed_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:06.164379Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:06.215902Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342355757452384:3441]
2025-10-09T21:49:06.216187Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 1413, pipe: [1:7559342355757452384:3441]
2025-10-09T21:49:06.306217Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:06.306454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:06.306469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:06.306512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342355757452384:3441], now have 1 active actors on pipe
2025-10-09T21:49:06.306686Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:06.306693Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342355757452384:3441]
2025-10-09T21:49:06.306710Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:06.306714Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:06.306730Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:06.306774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1414 reason: error at write: Disconnected sessionId: src_id|1ee58cad-cffb9085-8c70128f-ec405eed_0
2025-10-09T21:49:06.306912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1414 sessionId: src_id|1ee58cad-cffb9085-8c70128f-ec405eed_0 is DEAD
2025-10-09T21:49:06.307071Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ee58cad-cffb9085-8c70128f-ec405eed_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:06.307081Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ee58cad-cffb9085-8c70128f-ec405eed_0] Write session will restart in 0.005292s
2025-10-09T21:49:06.307099Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ee58cad-cffb9085-8c70128f-ec405eed_0] Write session: Do CDS request
2025-10-09T21:49:06.307113Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ee58cad-cffb9085-8c70128f-ec405eed_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:06.313360Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ee58cad-cffb9085-8c70128f-ec405eed_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:06.313456Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:06.313468Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1415
2025-10-09T21:49:06.313574Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1415 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:06.313599Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1415 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:06.313606Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1415 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:06.313616Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:06.313974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1415 sessionId:  describe result for acl check
2025-10-09T21:49:06.314006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:06.314011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:06.314013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:06.314017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342355757452407:3322] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:06.314522Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342355757452407:3322] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:06.316933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342355757452407:3322] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:06.319902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342355757452407:3322] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:06.319917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342355757452407:3322] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:06.319922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342355757452407:3322] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:06.319932Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1415 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:06.324869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:06.327011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:06.327182Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:06.327191Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:06.327201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:06.327242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:06.327282Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342355395627754:10195]
2025-10-09T21:49:06.327837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:06.328091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:06.328145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:06.328390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:06.328425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.328429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.328430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.328431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.328431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.328432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.328433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.328433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.328434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.328438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.328445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.328447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.328449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.328451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.328453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.328455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.328457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.328459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.328464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.328469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.328474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.328477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.328479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.328483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.328485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.328488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.328491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.328493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.328502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.328506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.328508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.328512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.328517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.328522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.328526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.328532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.328559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:06.334642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:06.336651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:06.336663Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:06.336665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:06.336668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:06.336673Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1414 [2:7559342355395627754:10195]
2025-10-09T21:49:06.336682Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:06.336694Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:06.336696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:06.336698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:06.336699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:06.336713Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:06.336716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:06.336721Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:06.336959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.336967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.336969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.336971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:06.337020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:06.337026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.337034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.337040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.337058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:06.337081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342355395627749:10195]
2025-10-09T21:49:06.337087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342355395627749:10195]
2025-10-09T21:49:06.337088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342355395627749:10195]
2025-10-09T21:49:06.337097Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:06.337111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342355757452526:3322], now have 1 active actors on pipe
2025-10-09T21:49:06.337110Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:06.337113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:06.337115Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:06.337116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342355757452392:2432], now have 1 active actors on pipe
2025-10-09T21:49:06.337124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:06.337134Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:06.337139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:06.337163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342355395627749:10195]
2025-10-09T21:49:06.337172Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:06.337180Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:06.337178Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1414
2025-10-09T21:49:06.337185Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1414
2025-10-09T21:49:06.337267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:06.337274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:06.337294Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|201fa117-c62ba86f-bd40e05-88ed41ea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:06.337304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:06.337308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:06.337310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:06.337320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:06.337330Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:06.337343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:06.337461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:06.337466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:06.337475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:06.337570Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1415 partition: 0 MaxSeqNo: 25 sessionId: src_id|201fa117-c62ba86f-bd40e05-88ed41ea_0
2025-10-09T21:49:06.337900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ee58cad-cffb9085-8c70128f-ec405eed_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|201fa117-c62ba86f-bd40e05-88ed41ea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:06.345558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|201fa117-c62ba86f-bd40e05-88ed41ea_0] Write session: try to update token
2025-10-09T21:49:06.345579Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|201fa117-c62ba86f-bd40e05-88ed41ea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:06.347867Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:06.347882Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:06.347891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342355395627749:10195]
2025-10-09T21:49:06.347925Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:06.347945Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:06.348115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:06.348258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:06.348391Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:06.348518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:06.348696Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:06.348738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:06.359960Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:06.359979Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:06.359981Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:06.359983Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:06.359994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342355395627749:10195]
2025-10-09T21:49:06.359997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342355395627749:10195]
2025-10-09T21:49:06.360001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342355395627749:10195]
2025-10-09T21:49:06.360044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:06.360046Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:06.360050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.360052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:06.360145Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:06.360155Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:06.360178Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:06.360577Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1415 sessionId: src_id|201fa117-c62ba86f-bd40e05-88ed41ea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:06.362727Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:06.406915Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342355757452642:3441]
2025-10-09T21:49:06.407157Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 1414, pipe: [1:7559342355757452642:3441]
2025-10-09T21:49:06.507185Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:06.507733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:06.507749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:06.507773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342355757452642:3441], now have 1 active actors on pipe
2025-10-09T21:49:06.507967Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:06.507960Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:06.507987Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:06.507996Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342355757452642:3441]
2025-10-09T21:49:06.508023Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:06.508046Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1415 reason: error at write: Disconnected sessionId: src_id|201fa117-c62ba86f-bd40e05-88ed41ea_0
2025-10-09T21:49:06.508223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1415 sessionId: src_id|201fa117-c62ba86f-bd40e05-88ed41ea_0 is DEAD
2025-10-09T21:49:06.508373Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|201fa117-c62ba86f-bd40e05-88ed41ea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:06.508384Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|201fa117-c62ba86f-bd40e05-88ed41ea_0] Write session will restart in 0.006595s
2025-10-09T21:49:06.508410Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|201fa117-c62ba86f-bd40e05-88ed41ea_0] Write session: Do CDS request
2025-10-09T21:49:06.508416Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|201fa117-c62ba86f-bd40e05-88ed41ea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:06.516549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|201fa117-c62ba86f-bd40e05-88ed41ea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:06.516592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:06.516610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1416
2025-10-09T21:49:06.516760Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1416 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:06.516805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1416 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:06.516815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1416 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:06.516819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:06.517340Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1416 sessionId:  describe result for acl check
2025-10-09T21:49:06.517373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:06.517378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:06.517380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:06.517384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342355757452669:3311] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:06.517915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342355757452669:3311] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:06.520337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342355757452669:3311] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:06.522867Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342355757452669:3311] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:06.522880Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342355757452669:3311] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:06.522884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342355757452669:3311] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:06.522891Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1416 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:06.526478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:06.528698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:06.528957Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:06.528970Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:06.528984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:06.529049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:06.529090Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342355395627995:10197]
2025-10-09T21:49:06.529688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:06.529998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:06.530051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:06.530312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:06.530371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.530377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.530378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.530379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.530379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.530380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.530381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.530381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.530382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.530386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.530394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.530396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.530398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.530400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.530402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.530404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.530407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.530411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.530418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.530426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.530432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.530438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.530442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.530447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.530451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.530455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.530457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.530460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.530463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.530466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.530468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.530471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.530474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.530477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.530482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.530487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.530515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:06.536749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:06.537939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:06.537951Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:06.537952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:06.537955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:06.537961Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1415 [2:7559342355395627995:10197]
2025-10-09T21:49:06.537972Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:06.537990Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:06.537993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:06.537996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:06.537998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:06.538010Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:06.538014Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:06.538020Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:06.538073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.538079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.538080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.538082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:06.538135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:06.538141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.538148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.538154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.538163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:06.538188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342355395627990:10197]
2025-10-09T21:49:06.538193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342355395627990:10197]
2025-10-09T21:49:06.538194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342355395627990:10197]
2025-10-09T21:49:06.538201Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:06.538211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:06.538212Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:06.538215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:06.538217Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:06.538223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:06.538226Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:06.538237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342355757452808:3311], now have 1 active actors on pipe
2025-10-09T21:49:06.538245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342355757452645:2432], now have 1 active actors on pipe
2025-10-09T21:49:06.538266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342355395627990:10197]
2025-10-09T21:49:06.538276Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:06.538282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:06.538312Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1415
2025-10-09T21:49:06.538314Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1415
2025-10-09T21:49:06.538395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:06.538407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:06.538433Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cad5b78a-35c375e5-2f96a9f8-2d20ccfe_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:06.538440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:06.538447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:06.538449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:06.538457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:06.538467Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:06.538482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:06.538642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:06.538655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:06.538674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:06.538760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1416 partition: 0 MaxSeqNo: 25 sessionId: src_id|cad5b78a-35c375e5-2f96a9f8-2d20ccfe_0
2025-10-09T21:49:06.539112Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|201fa117-c62ba86f-bd40e05-88ed41ea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cad5b78a-35c375e5-2f96a9f8-2d20ccfe_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:06.541436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cad5b78a-35c375e5-2f96a9f8-2d20ccfe_0] Write session: try to update token
2025-10-09T21:49:06.541453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cad5b78a-35c375e5-2f96a9f8-2d20ccfe_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:06.552156Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:06.552176Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:06.552178Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:06.552181Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:06.552190Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342355395627990:10197]
2025-10-09T21:49:06.552193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342355395627990:10197]
2025-10-09T21:49:06.552197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342355395627990:10197]
2025-10-09T21:49:06.552242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:06.552245Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:06.552249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.552251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:06.552253Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:06.552255Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:06.552384Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:06.554112Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1416 sessionId: src_id|cad5b78a-35c375e5-2f96a9f8-2d20ccfe_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:06.556231Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:06.608118Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342355757452895:3441]
2025-10-09T21:49:06.608359Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 1415, pipe: [1:7559342355757452895:3441]
2025-10-09T21:49:06.699542Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:06.699974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:06.699992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:06.700032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342355757452895:3441], now have 1 active actors on pipe
2025-10-09T21:49:06.700245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:06.700266Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:06.700263Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:06.700264Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:06.700318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342355757452895:3441]
2025-10-09T21:49:06.700334Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1416 reason: error at write: Disconnected sessionId: src_id|cad5b78a-35c375e5-2f96a9f8-2d20ccfe_0
2025-10-09T21:49:06.700540Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1416 sessionId: src_id|cad5b78a-35c375e5-2f96a9f8-2d20ccfe_0 is DEAD
2025-10-09T21:49:06.700671Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cad5b78a-35c375e5-2f96a9f8-2d20ccfe_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:06.700679Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cad5b78a-35c375e5-2f96a9f8-2d20ccfe_0] Write session will restart in 0.006535s
2025-10-09T21:49:06.700712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cad5b78a-35c375e5-2f96a9f8-2d20ccfe_0] Write session: Do CDS request
2025-10-09T21:49:06.700719Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cad5b78a-35c375e5-2f96a9f8-2d20ccfe_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:06.708547Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cad5b78a-35c375e5-2f96a9f8-2d20ccfe_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:06.708661Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:06.708675Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1417
2025-10-09T21:49:06.708832Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1417 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:06.708867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1417 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:06.708874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1417 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:06.708878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:06.709337Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1417 sessionId:  describe result for acl check
2025-10-09T21:49:06.709374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:06.709378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:06.709379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:06.709386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342355757452922:3306] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:06.709961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342355757452922:3306] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:06.713110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342355757452922:3306] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:06.716506Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342355757452922:3306] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:06.716526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342355757452922:3306] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:06.716530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342355757452922:3306] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:06.716538Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1417 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:06.718466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:06.720807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:06.721010Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:06.721021Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:06.721033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:06.721088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:06.721120Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342355395628241:10201]
2025-10-09T21:49:06.721502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:06.721794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:06.721838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:06.722083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:06.722120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.722125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.722125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.722126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.722127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.722127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.722128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.722129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.722130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.722133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.722140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.722142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.722145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.722147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.722149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.722151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.722153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.722155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.722160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.722165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.722169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.722173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.722175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.722178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.722181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.722184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.722187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.722190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.722193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.722196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.722199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.722202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.722205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.722208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.722210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.722214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.722230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:06.728646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:06.729913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:06.729927Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:06.729929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:06.729935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:06.729941Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1416 [2:7559342355395628241:10201]
2025-10-09T21:49:06.729954Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:06.729970Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:06.729973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:06.729976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:06.729978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:06.729994Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:06.730002Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:06.730009Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:06.730067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.730072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.730073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.730075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:06.730126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:06.730131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.730139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.730145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.730155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:06.730177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342355395628236:10201]
2025-10-09T21:49:06.730187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342355395628236:10201]
2025-10-09T21:49:06.730189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342355395628236:10201]
2025-10-09T21:49:06.730196Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:06.730206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:06.730207Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:06.730210Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:06.730212Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:06.730216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:06.730218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:06.730228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342355757453133:3306], now have 1 active actors on pipe
2025-10-09T21:49:06.730231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342355757452898:2432], now have 1 active actors on pipe
2025-10-09T21:49:06.730256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342355395628236:10201]
2025-10-09T21:49:06.730264Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:06.730270Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:06.730274Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1416
2025-10-09T21:49:06.730284Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1416
2025-10-09T21:49:06.730305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:06.730318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:06.730346Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f49a09b3-90bf5c78-1fbb2036-2358f11e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:06.730351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:06.730354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:06.730356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:06.730363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:06.730375Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:06.730393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:06.730714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:06.730730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:06.730769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:06.730883Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1417 partition: 0 MaxSeqNo: 25 sessionId: src_id|f49a09b3-90bf5c78-1fbb2036-2358f11e_0
2025-10-09T21:49:06.731196Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cad5b78a-35c375e5-2f96a9f8-2d20ccfe_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f49a09b3-90bf5c78-1fbb2036-2358f11e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:06.733492Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f49a09b3-90bf5c78-1fbb2036-2358f11e_0] Write session: try to update token
2025-10-09T21:49:06.733509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f49a09b3-90bf5c78-1fbb2036-2358f11e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:06.743974Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:06.743993Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:06.743995Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:06.743996Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:06.744004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342355395628236:10201]
2025-10-09T21:49:06.744007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342355395628236:10201]
2025-10-09T21:49:06.744010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342355395628236:10201]
2025-10-09T21:49:06.744044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:06.744051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.744053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:06.744076Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:06.744081Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:06.744084Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:06.744168Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:06.745676Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1417 sessionId: src_id|f49a09b3-90bf5c78-1fbb2036-2358f11e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:06.747927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:06.800592Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342355757453152:3441]
2025-10-09T21:49:06.800842Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 1416, pipe: [1:7559342355757453152:3441]
2025-10-09T21:49:06.895224Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:06.895493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:06.895510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:06.895553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342355757453152:3441], now have 1 active actors on pipe
2025-10-09T21:49:06.895733Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:06.895775Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342355757453152:3441]
2025-10-09T21:49:06.895781Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:06.895785Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:06.895806Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:06.895886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1417 reason: error at write: Disconnected sessionId: src_id|f49a09b3-90bf5c78-1fbb2036-2358f11e_0
2025-10-09T21:49:06.896045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1417 sessionId: src_id|f49a09b3-90bf5c78-1fbb2036-2358f11e_0 is DEAD
2025-10-09T21:49:06.896211Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f49a09b3-90bf5c78-1fbb2036-2358f11e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:06.896221Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f49a09b3-90bf5c78-1fbb2036-2358f11e_0] Write session will restart in 0.008864s
2025-10-09T21:49:06.896248Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f49a09b3-90bf5c78-1fbb2036-2358f11e_0] Write session: Do CDS request
2025-10-09T21:49:06.896254Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f49a09b3-90bf5c78-1fbb2036-2358f11e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:06.906512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f49a09b3-90bf5c78-1fbb2036-2358f11e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:06.906561Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:06.906587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1418
2025-10-09T21:49:06.906748Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1418 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:06.906784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1418 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:06.906794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1418 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:06.906798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:06.907275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1418 sessionId:  describe result for acl check
2025-10-09T21:49:06.907316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:06.907322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:06.907324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:06.907329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342355757453229:3300] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:06.907882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342355757453229:3300] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:06.910343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342355757453229:3300] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:06.912830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342355757453229:3300] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:06.912843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342355757453229:3300] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:06.912847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342355757453229:3300] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:06.912854Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1418 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:06.913777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:06.916049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:06.916227Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:06.916235Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:06.916246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:06.916301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:06.916345Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342355395628485:10203]
2025-10-09T21:49:06.916912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:06.917196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:06.917240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:06.917545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:06.917590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.917596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.917597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.917598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.917600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.917601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.917602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.917603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.917604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.917609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.917619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.917623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.917635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.917638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.917642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.917646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.917650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.917653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.917660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.917668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:06.917674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.917679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:06.917683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.917688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:06.917693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.917697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:06.917702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.917707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:06.917711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.917716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:06.917720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.917725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.917730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.917734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.917740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.917744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.917769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:06.923636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:06.924896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:06.924910Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:06.924911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:06.924914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:06.924919Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1417 [2:7559342355395628485:10203]
2025-10-09T21:49:06.924929Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:06.924943Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:06.924948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:06.924951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:06.924952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:06.924964Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:06.924972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:06.924977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:06.925017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.925023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.925024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.925025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:06.925074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:06.925079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.925086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:06.925092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:06.925102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:06.925116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342355395628480:10203]
2025-10-09T21:49:06.925117Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342355395628480:10203]
2025-10-09T21:49:06.925119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342355395628480:10203]
2025-10-09T21:49:06.925124Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:06.925134Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:06.925135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:06.925142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:06.925144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:06.925144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:06.925146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:06.925155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342355757453157:2432], now have 1 active actors on pipe
2025-10-09T21:49:06.925157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342355757453396:3300], now have 1 active actors on pipe
2025-10-09T21:49:06.925174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342355395628480:10203]
2025-10-09T21:49:06.925180Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:06.925185Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:06.925231Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1417
2025-10-09T21:49:06.925232Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1417
2025-10-09T21:49:06.925285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:06.925297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:06.925329Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|54247e44-49df5974-67d74c87-e03221d5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:06.925342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:06.925351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:06.925354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:06.925365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:06.925381Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:06.925405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:06.925526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:06.925533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:06.925546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:06.925647Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1418 partition: 0 MaxSeqNo: 25 sessionId: src_id|54247e44-49df5974-67d74c87-e03221d5_0
2025-10-09T21:49:06.925977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f49a09b3-90bf5c78-1fbb2036-2358f11e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|54247e44-49df5974-67d74c87-e03221d5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:06.928561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54247e44-49df5974-67d74c87-e03221d5_0] Write session: try to update token
2025-10-09T21:49:06.928581Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54247e44-49df5974-67d74c87-e03221d5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:06.941307Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1418 sessionId: src_id|54247e44-49df5974-67d74c87-e03221d5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:06.942366Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:06.942385Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:06.942395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342355395628480:10203]
2025-10-09T21:49:06.942442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:06.942455Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:06.942627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:06.942777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:06.942926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:06.943075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:06.943275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:06.943328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:06.943787Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:06.943976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:06.943993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:06.944028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:06.944107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:06.950205Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:06.950222Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:06.950225Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:06.950227Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:06.950238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342355395628480:10203]
2025-10-09T21:49:06.950242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342355395628480:10203]
2025-10-09T21:49:06.950246Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342355395628480:10203]
2025-10-09T21:49:06.950281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:06.950288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:06.950291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:06.950295Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:06.950310Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:06.950314Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:06.950426Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:06.995705Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342355757453409:3441]
2025-10-09T21:49:06.995974Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 1417, pipe: [1:7559342355757453409:3441]
2025-10-09T21:49:07.104528Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:07.112078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.112103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.112258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.112321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.112392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.112463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.112513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.112518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.112587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.112661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.112732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.112804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.112847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.112855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.112919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.112984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.113046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.113104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.113405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.113415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.113480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.113815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.113874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.113937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.113984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.113992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.114083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.114195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.114252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.114308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.114346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.114352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.114412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.114481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.114546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.114602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.114641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.114647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.114707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.114767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.114826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.114895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.114941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.114949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.115012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.115076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.115138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.115191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.115229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.115234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.115295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.115349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.115402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.115461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.115600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.115610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.115672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.115727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.115788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.115853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.115897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.115905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.115964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342355757453409:3441], now have 1 active actors on pipe
2025-10-09T21:49:07.116172Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:07.116190Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:07.116221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1418 reason: error at write: Disconnected sessionId: src_id|54247e44-49df5974-67d74c87-e03221d5_0
2025-10-09T21:49:07.116408Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1418 sessionId: src_id|54247e44-49df5974-67d74c87-e03221d5_0 is DEAD
2025-10-09T21:49:07.116444Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:07.116555Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54247e44-49df5974-67d74c87-e03221d5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:07.116563Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54247e44-49df5974-67d74c87-e03221d5_0] Write session will restart in 0.008572s
2025-10-09T21:49:07.116588Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54247e44-49df5974-67d74c87-e03221d5_0] Write session: Do CDS request
2025-10-09T21:49:07.116594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54247e44-49df5974-67d74c87-e03221d5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:07.116580Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342355757453409:3441]
2025-10-09T21:49:07.116604Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:07.126463Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54247e44-49df5974-67d74c87-e03221d5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:07.126522Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:07.126538Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1419
2025-10-09T21:49:07.126708Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1419 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:07.126743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1419 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:07.126748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1419 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:07.126751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:07.127311Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1419 sessionId:  describe result for acl check
2025-10-09T21:49:07.127365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:07.127372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:07.127374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:07.127379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342360052420788:3285] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:07.127901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342360052420788:3285] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:07.130859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342360052420788:3285] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:07.133613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342360052420788:3285] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:07.133636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342360052420788:3285] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:07.133640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342360052420788:3285] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:07.133648Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1419 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:07.134403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:07.136551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:07.136730Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:07.136740Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:07.136749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:07.136790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:07.136820Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342359690596064:10215]
2025-10-09T21:49:07.137237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:07.137589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:07.137662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:07.138023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:07.138086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.138093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.138094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.138095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.138096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.138097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.138099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.138100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.138101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.138107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.138117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.138120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.138124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.138127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.138141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.138144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.138148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.138150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.138158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.138165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.138171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.138176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.138186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.138191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.138196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.138200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.138210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.138215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.138219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.138229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.138233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.138239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.138248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.138253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.138257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.138263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.138293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:07.144576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:07.145659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:07.145673Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:07.145674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:07.145678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:07.145682Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1418 [2:7559342359690596064:10215]
2025-10-09T21:49:07.145692Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:07.145707Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:07.145709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:07.145711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:07.145712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:07.145725Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:07.145729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:07.145735Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:07.145792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.145799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.145804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.145806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:07.145863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:07.145869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.145876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.145882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.145892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:07.145907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342359690596059:10215]
2025-10-09T21:49:07.145913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342359690596059:10215]
2025-10-09T21:49:07.145915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342359690596059:10215]
2025-10-09T21:49:07.145921Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:07.145932Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:07.145936Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:07.145938Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:07.145944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342360052420972:3285], now have 1 active actors on pipe
2025-10-09T21:49:07.145947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342360052420726:2432], now have 1 active actors on pipe
2025-10-09T21:49:07.145952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:07.145965Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:07.145968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:07.145995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342359690596059:10215]
2025-10-09T21:49:07.146007Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:07.146025Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1418
2025-10-09T21:49:07.146015Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:07.146085Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1418
2025-10-09T21:49:07.146122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.146135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.146180Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|843858d8-6b85a420-b03e26ff-2b6f4210_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:07.146195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:07.146200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:07.146203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:07.146214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:07.146227Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:07.146243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:07.146371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.146376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.146387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:07.146473Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1419 partition: 0 MaxSeqNo: 25 sessionId: src_id|843858d8-6b85a420-b03e26ff-2b6f4210_0
2025-10-09T21:49:07.146787Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54247e44-49df5974-67d74c87-e03221d5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|843858d8-6b85a420-b03e26ff-2b6f4210_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:07.149182Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|843858d8-6b85a420-b03e26ff-2b6f4210_0] Write session: try to update token
2025-10-09T21:49:07.149198Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|843858d8-6b85a420-b03e26ff-2b6f4210_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:07.160366Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:07.160384Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:07.160387Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:07.160389Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:07.160400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342359690596059:10215]
2025-10-09T21:49:07.160405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342359690596059:10215]
2025-10-09T21:49:07.160410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342359690596059:10215]
2025-10-09T21:49:07.160454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:07.160461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.160461Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:07.160464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:07.160468Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:07.160472Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:07.160604Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:07.162517Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1419 sessionId: src_id|843858d8-6b85a420-b03e26ff-2b6f4210_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:07.164780Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:07.216912Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342360052420989:3441]
2025-10-09T21:49:07.217227Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 1418, pipe: [1:7559342360052420989:3441]
2025-10-09T21:49:07.311736Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:07.312326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.312345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.312405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342360052420989:3441], now have 1 active actors on pipe
2025-10-09T21:49:07.312611Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:07.312623Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:07.312623Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342360052420989:3441]
2025-10-09T21:49:07.312640Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:07.312645Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:07.312698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1419 reason: error at write: Disconnected sessionId: src_id|843858d8-6b85a420-b03e26ff-2b6f4210_0
2025-10-09T21:49:07.312867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1419 sessionId: src_id|843858d8-6b85a420-b03e26ff-2b6f4210_0 is DEAD
2025-10-09T21:49:07.313020Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|843858d8-6b85a420-b03e26ff-2b6f4210_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:07.313032Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|843858d8-6b85a420-b03e26ff-2b6f4210_0] Write session will restart in 0.006517s
2025-10-09T21:49:07.313058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|843858d8-6b85a420-b03e26ff-2b6f4210_0] Write session: Do CDS request
2025-10-09T21:49:07.313064Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|843858d8-6b85a420-b03e26ff-2b6f4210_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:07.320469Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|843858d8-6b85a420-b03e26ff-2b6f4210_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:07.320510Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:07.320526Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1420
2025-10-09T21:49:07.320661Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1420 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:07.320694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1420 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:07.320699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1420 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:07.320703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:07.321114Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1420 sessionId:  describe result for acl check
2025-10-09T21:49:07.321150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:07.321155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:07.321156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:07.321160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342360052421011:3275] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:07.321607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342360052421011:3275] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:07.324381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342360052421011:3275] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:07.327930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342360052421011:3275] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:07.327941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342360052421011:3275] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:07.327944Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342360052421011:3275] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:07.327950Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1420 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:07.330559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:07.332791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:07.332975Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:07.332986Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:07.332997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:07.333048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:07.333085Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342359690596309:10219]
2025-10-09T21:49:07.333667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:07.333977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:07.334041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:07.334361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:07.334422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.334431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.334433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.334434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.334435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.334436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.334437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.334439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.334440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.334445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.334457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.334461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.334465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.334469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.334473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.334477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.334481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.334485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.334492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.334499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.334506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.334511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.334516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.334522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.334526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.334531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.334535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.334540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.334545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.334550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.334555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.334561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.334565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.334570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.334581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.334587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.334621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:07.340868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:07.342337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:07.342354Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:07.342356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:07.342360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:07.342366Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1419 [2:7559342359690596309:10219]
2025-10-09T21:49:07.342380Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:07.342399Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:07.342410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:07.342413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:07.342415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:07.342431Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:07.342438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:07.342445Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:07.342498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.342503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.342505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.342507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:07.342573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:07.342581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.342593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.342605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.342622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:07.342647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342360052420996:2432], now have 1 active actors on pipe
2025-10-09T21:49:07.342657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342360052421202:3275], now have 1 active actors on pipe
2025-10-09T21:49:07.342664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342359690596304:10219]
2025-10-09T21:49:07.342667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342359690596304:10219]
2025-10-09T21:49:07.342669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342359690596304:10219]
2025-10-09T21:49:07.342682Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:07.342697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:07.342709Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:07.342716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:07.342718Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:07.342720Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:07.342722Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:07.342719Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1419
2025-10-09T21:49:07.342736Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1419
2025-10-09T21:49:07.342743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342359690596304:10219]
2025-10-09T21:49:07.342751Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:07.342767Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:07.342788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.342796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.342828Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|98a6a32-81c3cb66-41dcf63a-5012f952_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:07.342840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:07.342844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:07.342846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:07.342856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:07.342870Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:07.342882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:07.343025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.343035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.343052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:07.343158Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1420 partition: 0 MaxSeqNo: 25 sessionId: src_id|98a6a32-81c3cb66-41dcf63a-5012f952_0
2025-10-09T21:49:07.343499Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|843858d8-6b85a420-b03e26ff-2b6f4210_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|98a6a32-81c3cb66-41dcf63a-5012f952_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:07.345865Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98a6a32-81c3cb66-41dcf63a-5012f952_0] Write session: try to update token
2025-10-09T21:49:07.345882Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98a6a32-81c3cb66-41dcf63a-5012f952_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:07.356929Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:07.356950Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:07.356953Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:07.356955Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:07.356965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342359690596304:10219]
2025-10-09T21:49:07.356971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342359690596304:10219]
2025-10-09T21:49:07.356975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342359690596304:10219]
2025-10-09T21:49:07.357019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:07.357027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.357030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:07.357033Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:07.357040Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:07.357044Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:07.357163Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:07.358229Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1420 sessionId: src_id|98a6a32-81c3cb66-41dcf63a-5012f952_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:07.360554Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:07.413002Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342360052421251:3441]
2025-10-09T21:49:07.413294Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 1419, pipe: [1:7559342360052421251:3441]
2025-10-09T21:49:07.504192Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:07.504460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.504476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.504526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342360052421251:3441], now have 1 active actors on pipe
2025-10-09T21:49:07.504782Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:07.504772Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:07.504773Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:07.504777Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:07.504838Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342360052421251:3441]
2025-10-09T21:49:07.504841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1420 reason: error at write: Disconnected sessionId: src_id|98a6a32-81c3cb66-41dcf63a-5012f952_0
2025-10-09T21:49:07.505043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1420 sessionId: src_id|98a6a32-81c3cb66-41dcf63a-5012f952_0 is DEAD
2025-10-09T21:49:07.505187Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98a6a32-81c3cb66-41dcf63a-5012f952_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:07.505200Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98a6a32-81c3cb66-41dcf63a-5012f952_0] Write session will restart in 0.008402s
2025-10-09T21:49:07.505229Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98a6a32-81c3cb66-41dcf63a-5012f952_0] Write session: Do CDS request
2025-10-09T21:49:07.505235Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98a6a32-81c3cb66-41dcf63a-5012f952_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:07.514521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98a6a32-81c3cb66-41dcf63a-5012f952_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:07.514607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:07.514638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1421
2025-10-09T21:49:07.514754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1421 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:07.514788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1421 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:07.514800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1421 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:07.514803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:07.515304Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1421 sessionId:  describe result for acl check
2025-10-09T21:49:07.515345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:07.515352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:07.515354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:07.515360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342360052421305:3268] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:07.515921Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342360052421305:3268] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:07.518397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342360052421305:3268] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:07.521362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342360052421305:3268] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:07.521380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342360052421305:3268] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:07.521384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342360052421305:3268] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:07.521393Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1421 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:07.524075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:07.526685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:07.526903Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:07.526915Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:07.526927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:07.526979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:07.527022Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342359690596555:10221]
2025-10-09T21:49:07.527632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:07.527982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:07.528035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:07.528355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:07.528410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.528415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.528416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.528417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.528417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.528418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.528419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.528420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.528421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.528424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.528431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.528434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.528436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.528438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.528440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.528442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.528444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.528446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.528452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.528457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.528461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.528465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.528467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.528470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.528475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.528480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.528485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.528490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.528494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.528499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.528504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.528508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.528512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.528517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.528522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.528527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.528559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:07.535189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:07.536338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:07.536357Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:07.536360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:07.536365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:07.536372Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1420 [2:7559342359690596555:10221]
2025-10-09T21:49:07.536387Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:07.536408Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:07.536411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:07.536415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:07.536417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:07.536435Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:07.536447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:07.536455Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:07.536523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.536533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.536536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.536539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:07.536612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:07.536620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.536633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.536645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.536669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:07.536697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342360052421254:2432], now have 1 active actors on pipe
2025-10-09T21:49:07.536707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342360052421475:3268], now have 1 active actors on pipe
2025-10-09T21:49:07.536714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342359690596550:10221]
2025-10-09T21:49:07.536717Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342359690596550:10221]
2025-10-09T21:49:07.536720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342359690596550:10221]
2025-10-09T21:49:07.536732Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:07.536753Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:07.536756Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:07.536759Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:07.536761Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:07.536738Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1420
2025-10-09T21:49:07.536744Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1420
2025-10-09T21:49:07.536768Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:07.536776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:07.536831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.536840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.536855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342359690596550:10221]
2025-10-09T21:49:07.536868Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:07.536887Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:07.536909Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b038b30-da6325f3-e9d48b4a-4aac6776_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:07.536921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:07.536926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:07.536928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:07.536940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:07.536954Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:07.536979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:07.537154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.537163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.537177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:07.537275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1421 partition: 0 MaxSeqNo: 25 sessionId: src_id|b038b30-da6325f3-e9d48b4a-4aac6776_0
2025-10-09T21:49:07.537662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98a6a32-81c3cb66-41dcf63a-5012f952_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b038b30-da6325f3-e9d48b4a-4aac6776_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:07.540060Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b038b30-da6325f3-e9d48b4a-4aac6776_0] Write session: try to update token
2025-10-09T21:49:07.540077Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b038b30-da6325f3-e9d48b4a-4aac6776_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:07.551746Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:07.551771Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:07.551773Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:07.551775Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:07.551787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342359690596550:10221]
2025-10-09T21:49:07.551791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342359690596550:10221]
2025-10-09T21:49:07.551796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342359690596550:10221]
2025-10-09T21:49:07.551846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:07.551846Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:07.551853Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:07.551854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.551857Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:07.551858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:07.552001Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:07.553483Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1421 sessionId: src_id|b038b30-da6325f3-e9d48b4a-4aac6776_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:07.556010Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:07.604989Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342360052421509:3441]
2025-10-09T21:49:07.605280Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 1420, pipe: [1:7559342360052421509:3441]
2025-10-09T21:49:07.704785Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:07.705294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.705311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.705351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342360052421509:3441], now have 1 active actors on pipe
2025-10-09T21:49:07.705541Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:07.705580Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:07.705601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:07.705606Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342360052421509:3441]
2025-10-09T21:49:07.705651Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:07.705678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1421 reason: error at write: Disconnected sessionId: src_id|b038b30-da6325f3-e9d48b4a-4aac6776_0
2025-10-09T21:49:07.705823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1421 sessionId: src_id|b038b30-da6325f3-e9d48b4a-4aac6776_0 is DEAD
2025-10-09T21:49:07.706003Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b038b30-da6325f3-e9d48b4a-4aac6776_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:07.706014Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b038b30-da6325f3-e9d48b4a-4aac6776_0] Write session will restart in 0.009280s
2025-10-09T21:49:07.706046Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b038b30-da6325f3-e9d48b4a-4aac6776_0] Write session: Do CDS request
2025-10-09T21:49:07.706052Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b038b30-da6325f3-e9d48b4a-4aac6776_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:07.716561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b038b30-da6325f3-e9d48b4a-4aac6776_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:07.716632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:07.716648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1422
2025-10-09T21:49:07.716779Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1422 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:07.716828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1422 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:07.716839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1422 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:07.716843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:07.717271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1422 sessionId:  describe result for acl check
2025-10-09T21:49:07.717305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:07.717307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:07.717308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:07.717313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342360052421587:3262] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:07.717779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342360052421587:3262] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:07.720572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342360052421587:3262] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:07.722748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342360052421587:3262] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:07.722761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342360052421587:3262] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:07.722765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342360052421587:3262] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:07.722771Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1422 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:07.723310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:07.725636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:07.725817Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:07.725827Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:07.725839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:07.725889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:07.725930Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342359690596806:10225]
2025-10-09T21:49:07.726533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:07.726862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:07.726905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:07.727215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:07.727263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.727269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.727271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.727272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.727273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.727274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.727276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.727277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.727278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.727283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.727293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.727298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.727302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.727306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.727309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.727314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.727318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.727321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.727328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.727335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.727341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.727347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.727352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.727358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.727362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.727367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.727371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.727377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.727381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.727391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.727396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.727401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.727405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.727413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.727417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.727423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.727450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:07.733434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:07.734637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:07.734651Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:07.734653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:07.734655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:07.734660Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1421 [2:7559342359690596806:10225]
2025-10-09T21:49:07.734668Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:07.734682Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:07.734683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:07.734686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:07.734687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:07.734699Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:07.734703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:07.734709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:07.734748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.734754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.734755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.734757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:07.734808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:07.734814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.734821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.734827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.734841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:07.734858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342359690596801:10225]
2025-10-09T21:49:07.734862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342359690596801:10225]
2025-10-09T21:49:07.734864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342359690596801:10225]
2025-10-09T21:49:07.734871Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:07.734883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342360052421512:2432], now have 1 active actors on pipe
2025-10-09T21:49:07.734883Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:07.734885Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:07.734886Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:07.734892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342360052421755:3262], now have 1 active actors on pipe
2025-10-09T21:49:07.734900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:07.734912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:07.734920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:07.734941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342359690596801:10225]
2025-10-09T21:49:07.734953Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:07.734958Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1421
2025-10-09T21:49:07.734966Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1421
2025-10-09T21:49:07.734967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:07.735007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.735012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.735048Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8b133b1-2aa9c931-6521cc65-4b5dbe5b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:07.735061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:07.735065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:07.735067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:07.735077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:07.735092Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:07.735105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:07.735256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.735267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.735284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:07.735395Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1422 partition: 0 MaxSeqNo: 25 sessionId: src_id|8b133b1-2aa9c931-6521cc65-4b5dbe5b_0
2025-10-09T21:49:07.735708Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b038b30-da6325f3-e9d48b4a-4aac6776_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8b133b1-2aa9c931-6521cc65-4b5dbe5b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:07.738146Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b133b1-2aa9c931-6521cc65-4b5dbe5b_0] Write session: try to update token
2025-10-09T21:49:07.738165Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b133b1-2aa9c931-6521cc65-4b5dbe5b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:07.746243Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:07.746262Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:07.746272Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342359690596801:10225]
2025-10-09T21:49:07.746324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:07.746328Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:07.746508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:07.746655Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:07.746800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:07.746956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:07.747152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:07.747195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:07.750577Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1422 sessionId: src_id|8b133b1-2aa9c931-6521cc65-4b5dbe5b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:07.752838Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:07.753045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.753065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.753104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:07.753195Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:07.758940Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:07.758957Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:07.758960Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:07.758961Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:07.758972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342359690596801:10225]
2025-10-09T21:49:07.758976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342359690596801:10225]
2025-10-09T21:49:07.758980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342359690596801:10225]
2025-10-09T21:49:07.759015Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:07.759018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:07.759026Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:07.759026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.759029Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:07.759029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:07.759164Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:07.805709Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342360052421768:3441]
2025-10-09T21:49:07.806020Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 1421, pipe: [1:7559342360052421768:3441]
2025-10-09T21:49:07.907528Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:07.915097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.915125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.915471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.915669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.915731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.915788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.916006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.916026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.916238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.916303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.916368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.916427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.916463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.916471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.916523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.916573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.916627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.916678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.916716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.916721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.916782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.916846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.916918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.917148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.917187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.917192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.917250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.917303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.917357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.917412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.917451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.917457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.917510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.917562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.917617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.917688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.917735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.917743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.917826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.918046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.918241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.918414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.918458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.918467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.918664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.918717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.918770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.918821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.918871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.918877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.918928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.918978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.919028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.919080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.919132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.919139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.919197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:07.919261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:07.919316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:07.919370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:07.919495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:07.919502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:07.919548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342360052421768:3441], now have 1 active actors on pipe
2025-10-09T21:49:07.919827Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:07.919845Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:07.919850Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342360052421768:3441]
2025-10-09T21:49:07.919880Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:07.919883Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1422 reason: error at write: Disconnected sessionId: src_id|8b133b1-2aa9c931-6521cc65-4b5dbe5b_0
2025-10-09T21:49:07.920121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1422 sessionId: src_id|8b133b1-2aa9c931-6521cc65-4b5dbe5b_0 is DEAD
2025-10-09T21:49:07.920228Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b133b1-2aa9c931-6521cc65-4b5dbe5b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:07.920242Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b133b1-2aa9c931-6521cc65-4b5dbe5b_0] Write session will restart in 0.007508s
2025-10-09T21:49:07.920260Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:07.920285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b133b1-2aa9c931-6521cc65-4b5dbe5b_0] Write session: Do CDS request
2025-10-09T21:49:07.920294Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b133b1-2aa9c931-6521cc65-4b5dbe5b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:07.928500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b133b1-2aa9c931-6521cc65-4b5dbe5b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:07.928610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:07.928634Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1423
2025-10-09T21:49:07.928737Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1423 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:07.928771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1423 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:07.928776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1423 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:07.928779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:07.929394Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1423 sessionId:  describe result for acl check
2025-10-09T21:49:07.929434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:07.929440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:07.929443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:07.929448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342360052421800:3257] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:07.930097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342360052421800:3257] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:07.933660Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342360052421800:3257] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:07.936441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342360052421800:3257] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:07.936463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342360052421800:3257] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:07.936467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342360052421800:3257] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:07.936474Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1423 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:07.938005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:07.940146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:07.940434Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:07.940446Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:07.940459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:07.940516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:07.940549Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342359690597055:10227]
2025-10-09T21:49:07.941173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:07.941530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:07.941583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:07.941952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:07.942013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.942020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.942021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.942023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.942024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.942025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.942041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.942043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.942044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.942049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.942060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.942064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.942068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.942072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.942076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.942080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.942084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.942088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.942094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.942102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:07.942108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.942114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:07.942120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.942125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:07.942130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.942135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:07.942139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.942144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:07.942148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.942153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:07.942158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.942163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.942172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.942177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.942181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.942186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.942212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:07.948542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:07.949876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:07.949888Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:07.949890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:07.949893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:07.949900Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1422 [2:7559342359690597055:10227]
2025-10-09T21:49:07.949912Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:07.949927Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:07.949930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:07.949933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:07.949936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:07.949952Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:07.949959Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:07.949966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:07.950041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.950049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.950051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.950054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:07.950127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:07.950136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.950149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:07.950162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:07.950180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:07.950202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342360052422016:3257], now have 1 active actors on pipe
2025-10-09T21:49:07.950205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342360052421773:2432], now have 1 active actors on pipe
2025-10-09T21:49:07.950212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342359690597050:10227]
2025-10-09T21:49:07.950214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342359690597050:10227]
2025-10-09T21:49:07.950216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342359690597050:10227]
2025-10-09T21:49:07.950233Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:07.950247Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:07.950260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:07.950263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:07.950270Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1422
2025-10-09T21:49:07.950294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342359690597050:10227]
2025-10-09T21:49:07.950306Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:07.950319Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:07.950322Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1422
2025-10-09T21:49:07.950322Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:07.950324Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:07.950334Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:07.950414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.950425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.950460Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|48c36cf0-964cd77c-be607d81-f4ae4dc7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:07.950478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:07.950483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:07.950485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:07.950496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:07.950508Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:07.950525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:07.951640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.951656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.951679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:07.951837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1423 partition: 0 MaxSeqNo: 25 sessionId: src_id|48c36cf0-964cd77c-be607d81-f4ae4dc7_0
2025-10-09T21:49:07.952210Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b133b1-2aa9c931-6521cc65-4b5dbe5b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|48c36cf0-964cd77c-be607d81-f4ae4dc7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:07.956905Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48c36cf0-964cd77c-be607d81-f4ae4dc7_0] Write session: try to update token
2025-10-09T21:49:07.956922Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48c36cf0-964cd77c-be607d81-f4ae4dc7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:07.962735Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:07.962750Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:07.962758Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342359690597050:10227]
2025-10-09T21:49:07.962799Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:07.962803Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:07.962974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:07.963117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:07.963260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:07.963399Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:07.963608Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:07.963651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:07.970984Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1423 sessionId: src_id|48c36cf0-964cd77c-be607d81-f4ae4dc7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:07.973363Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:07.973565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:07.973587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:07.973650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:07.973751Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:07.977198Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:07.977218Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:07.977221Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:07.977222Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:07.977234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342359690597050:10227]
2025-10-09T21:49:07.977238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342359690597050:10227]
2025-10-09T21:49:07.977242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342359690597050:10227]
2025-10-09T21:49:07.977285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:07.977293Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:07.977297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:07.977300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:07.977302Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:07.977308Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:07.977439Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:08.020193Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342364347389326:3441]
2025-10-09T21:49:08.020479Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 1422, pipe: [1:7559342364347389326:3441]
2025-10-09T21:49:08.126309Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:08.133675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:08.133702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:08.133855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.133922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.133990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.134074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.134315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.134324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.134390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.134450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.134516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.134587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.134628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.134634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.134689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.134740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.134800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.134856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.134893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.134899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.134956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.135161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.135216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.135271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.135309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.135314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.135371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.135429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.135484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.135540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.135578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.135584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.135637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.135695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.135753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.135807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.135846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.135851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.135907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.135956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.136045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.136128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.136182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.136189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.136250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.136300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.136353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.136405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.136444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.136449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.136501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.136550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.136601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.136651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.136690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.136695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.136747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.136797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.136848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.136899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.136943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.136953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.137013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342364347389326:3441], now have 1 active actors on pipe
2025-10-09T21:49:08.137207Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:08.137222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:08.137248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1423 reason: error at write: Disconnected sessionId: src_id|48c36cf0-964cd77c-be607d81-f4ae4dc7_0
2025-10-09T21:49:08.137438Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1423 sessionId: src_id|48c36cf0-964cd77c-be607d81-f4ae4dc7_0 is DEAD
2025-10-09T21:49:08.137485Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:08.137537Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48c36cf0-964cd77c-be607d81-f4ae4dc7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:08.137546Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48c36cf0-964cd77c-be607d81-f4ae4dc7_0] Write session will restart in 0.008505s
2025-10-09T21:49:08.137567Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48c36cf0-964cd77c-be607d81-f4ae4dc7_0] Write session: Do CDS request
2025-10-09T21:49:08.137572Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48c36cf0-964cd77c-be607d81-f4ae4dc7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:08.137765Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342364347389326:3441]
2025-10-09T21:49:08.137805Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:08.147539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48c36cf0-964cd77c-be607d81-f4ae4dc7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:08.147623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:08.147643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1424
2025-10-09T21:49:08.147761Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1424 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:08.147807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1424 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:08.147818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1424 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:08.147821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:08.148345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1424 sessionId:  describe result for acl check
2025-10-09T21:49:08.148389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:08.148395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:08.148397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:08.148406Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342364347389385:3243] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:08.148962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342364347389385:3243] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:08.151514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342364347389385:3243] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:08.154908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342364347389385:3243] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:08.154926Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342364347389385:3243] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:08.154930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342364347389385:3243] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:08.154939Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1424 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:08.156952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:08.159621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:08.159797Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:08.159809Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:08.159822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:08.159864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:08.159897Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342363985564639:10239]
2025-10-09T21:49:08.160422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:08.160708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:08.160744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:08.160980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:08.161021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.161026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.161026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.161027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.161028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.161028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.161029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.161030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.161030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.161034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.161040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.161042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.161044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.161046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.161048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.161051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.161053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.161055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.161059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.161064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.161070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.161076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.161080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.161086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.161090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.161096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.161100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.161103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.161105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.161108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.161111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.161115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.161117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.161120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.161123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.161126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.161141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:08.167842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:08.169367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:08.169382Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:08.169384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:08.169389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:08.169395Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1423 [2:7559342363985564639:10239]
2025-10-09T21:49:08.169409Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:08.169428Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:08.169440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:08.169443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:08.169446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:08.169464Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:08.169470Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:08.169478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:08.169526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.169533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.169534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.169536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:08.169599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:08.169608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.169619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.169640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.169656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:08.169685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342364347389577:3243], now have 1 active actors on pipe
2025-10-09T21:49:08.169694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342364347389346:2432], now have 1 active actors on pipe
2025-10-09T21:49:08.169702Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342363985564626:10239]
2025-10-09T21:49:08.169704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342363985564626:10239]
2025-10-09T21:49:08.169706Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342363985564626:10239]
2025-10-09T21:49:08.169718Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:08.169734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:08.169736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:08.169739Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:08.169741Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:08.169752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:08.169756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:08.169734Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1423
2025-10-09T21:49:08.169787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342363985564626:10239]
2025-10-09T21:49:08.169795Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:08.169797Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:08.169822Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1423
2025-10-09T21:49:08.169821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:08.169832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:08.169861Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8b2c6041-bb490475-37b65205-3c3d9be6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:08.169867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:08.169871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:08.169873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:08.169879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:08.169888Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:08.169905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:08.170047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:08.170052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:08.170063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:08.170138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1424 partition: 0 MaxSeqNo: 25 sessionId: src_id|8b2c6041-bb490475-37b65205-3c3d9be6_0
2025-10-09T21:49:08.170483Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48c36cf0-964cd77c-be607d81-f4ae4dc7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8b2c6041-bb490475-37b65205-3c3d9be6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:08.173038Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2c6041-bb490475-37b65205-3c3d9be6_0] Write session: try to update token
2025-10-09T21:49:08.173053Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2c6041-bb490475-37b65205-3c3d9be6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:08.185272Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:08.185291Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:08.185294Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:08.185296Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:08.185309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342363985564626:10239]
2025-10-09T21:49:08.185313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342363985564626:10239]
2025-10-09T21:49:08.185317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342363985564626:10239]
2025-10-09T21:49:08.185361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:08.185362Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:08.185368Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:08.185371Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:08.185373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.185377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:08.185510Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:08.186151Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1424 sessionId: src_id|8b2c6041-bb490475-37b65205-3c3d9be6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:08.188594Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:08.237811Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342364347389619:3441]
2025-10-09T21:49:08.238080Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 1423, pipe: [1:7559342364347389619:3441]
2025-10-09T21:49:08.339799Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:08.340020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:08.340033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:08.340071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342364347389619:3441], now have 1 active actors on pipe
2025-10-09T21:49:08.340247Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:08.340291Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:08.340336Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342364347389619:3441]
2025-10-09T21:49:08.340422Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:08.340735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1424 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8b2c6041-bb490475-37b65205-3c3d9be6_0
2025-10-09T21:49:08.340915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1424 sessionId: src_id|8b2c6041-bb490475-37b65205-3c3d9be6_0 is DEAD
2025-10-09T21:49:08.341053Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2c6041-bb490475-37b65205-3c3d9be6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:08.341066Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2c6041-bb490475-37b65205-3c3d9be6_0] Write session will restart in 0.007848s
2025-10-09T21:49:08.341098Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2c6041-bb490475-37b65205-3c3d9be6_0] Write session: Do CDS request
2025-10-09T21:49:08.341104Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2c6041-bb490475-37b65205-3c3d9be6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:08.350551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2c6041-bb490475-37b65205-3c3d9be6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:08.350645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:08.350672Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1425
2025-10-09T21:49:08.350795Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1425 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:08.350827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1425 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:08.350849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1425 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:08.350860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:08.351351Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1425 sessionId:  describe result for acl check
2025-10-09T21:49:08.351394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:08.351403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:08.351405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:08.351412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342364347389697:3239] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:08.351965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342364347389697:3239] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:08.354601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342364347389697:3239] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:08.357664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:08.357718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342364347389697:3239] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:08.357736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342364347389697:3239] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:08.357740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342364347389697:3239] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:08.357749Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1425 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:08.360340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:08.360514Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:08.360522Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:08.360532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:08.360570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:08.360602Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342363985564889:6073]
2025-10-09T21:49:08.361005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:08.361311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:08.361342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:08.361565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:08.361602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.361610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.361610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.361611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.361612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.361612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.361613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.361614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.361614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.361618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.361636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.361639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.361641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.361643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.361645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.361647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.361649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.361651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.361656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.361662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.361665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.361669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.361673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.361676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.361678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.361681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.361683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.361686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.361689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.361691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.361694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.361697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.361700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.361703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.361706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.361709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.361724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:08.368376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:08.369877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:08.369891Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:08.369893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:08.369897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:08.369904Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1424 [2:7559342363985564889:6073]
2025-10-09T21:49:08.369916Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:08.369932Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:08.369940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:08.369943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:08.369946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:08.369961Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:08.369972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:08.369979Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:08.370050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.370059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.370061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.370063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:08.370126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:08.370133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.370146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.370159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.370178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:08.370203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342364347389869:3239], now have 1 active actors on pipe
2025-10-09T21:49:08.370211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342364347389622:2432], now have 1 active actors on pipe
2025-10-09T21:49:08.370219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342363985564883:6073]
2025-10-09T21:49:08.370221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342363985564883:6073]
2025-10-09T21:49:08.370225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342363985564883:6073]
2025-10-09T21:49:08.370237Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:08.370251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:08.370259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:08.370262Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:08.370264Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:08.370269Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:08.370271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:08.370301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342363985564883:6073]
2025-10-09T21:49:08.370278Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1424
2025-10-09T21:49:08.370308Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:08.370291Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1424
2025-10-09T21:49:08.370310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:08.370332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:08.370337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:08.370376Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d31251b0-df647f89-f0717336-ad8844c5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:08.370388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:08.370393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:08.370395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:08.370406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:08.370421Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:08.370443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:08.370602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:08.370613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:08.370629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:08.370710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1425 partition: 0 MaxSeqNo: 25 sessionId: src_id|d31251b0-df647f89-f0717336-ad8844c5_0
2025-10-09T21:49:08.371024Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2c6041-bb490475-37b65205-3c3d9be6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d31251b0-df647f89-f0717336-ad8844c5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:08.373410Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d31251b0-df647f89-f0717336-ad8844c5_0] Write session: try to update token
2025-10-09T21:49:08.373425Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d31251b0-df647f89-f0717336-ad8844c5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:08.385364Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:08.385389Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:08.385392Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:08.385394Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:08.385404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342363985564883:6073]
2025-10-09T21:49:08.385409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342363985564883:6073]
2025-10-09T21:49:08.385413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342363985564883:6073]
2025-10-09T21:49:08.385460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:08.385461Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:08.385467Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:08.385467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.385469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:08.385470Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:08.385615Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:08.386463Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1425 sessionId: src_id|d31251b0-df647f89-f0717336-ad8844c5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:08.388808Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:08.440586Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342364347389881:3441]
2025-10-09T21:49:08.440876Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 1424, pipe: [1:7559342364347389881:3441]
2025-10-09T21:49:08.531500Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:08.531727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:08.531740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:08.531778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342364347389881:3441], now have 1 active actors on pipe
2025-10-09T21:49:08.532022Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:08.532050Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:08.532095Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342364347389881:3441]
2025-10-09T21:49:08.532099Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:08.532178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1425 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d31251b0-df647f89-f0717336-ad8844c5_0
2025-10-09T21:49:08.532342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1425 sessionId: src_id|d31251b0-df647f89-f0717336-ad8844c5_0 is DEAD
2025-10-09T21:49:08.532528Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d31251b0-df647f89-f0717336-ad8844c5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:08.532540Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d31251b0-df647f89-f0717336-ad8844c5_0] Write session will restart in 0.009942s
2025-10-09T21:49:08.532573Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d31251b0-df647f89-f0717336-ad8844c5_0] Write session: Do CDS request
2025-10-09T21:49:08.532579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d31251b0-df647f89-f0717336-ad8844c5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:08.543563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d31251b0-df647f89-f0717336-ad8844c5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:08.543642Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:08.543662Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1426
2025-10-09T21:49:08.543819Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1426 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:08.543868Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1426 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:08.543879Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1426 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:08.543882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:08.544399Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1426 sessionId:  describe result for acl check
2025-10-09T21:49:08.544432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:08.544437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:08.544439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:08.544443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342364347389976:3236] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:08.544863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342364347389976:3236] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:08.547425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342364347389976:3236] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:08.550301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:08.550376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342364347389976:3236] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:08.550391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342364347389976:3236] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:08.550395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342364347389976:3236] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:08.550404Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1426 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:08.552827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:08.553031Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:08.553041Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:08.553054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:08.553103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:08.553216Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342363985565140:6064]
2025-10-09T21:49:08.553783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:08.554106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:08.554148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:08.554448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:08.554503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.554510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.554511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.554512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.554514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.554515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.554516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.554517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.554519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.554523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.554535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.554539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.554542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.554546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.554551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.554554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.554558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.554562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.554569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.554577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.554583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.554589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.554595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.554600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.554604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.554609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.554614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.554619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.554624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.554634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.554642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.554651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.554660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.554665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.554674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.554679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.554712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:08.560999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:08.562099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:08.562112Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:08.562115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:08.562119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:08.562125Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1425 [2:7559342363985565140:6064]
2025-10-09T21:49:08.562144Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:08.562161Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:08.562164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:08.562168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:08.562170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:08.562186Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:08.562193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:08.562208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:08.562267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.562275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.562277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.562280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:08.562451Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1425
2025-10-09T21:49:08.562453Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1425
2025-10-09T21:49:08.562575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:08.562588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.562600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.562613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.562636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:08.562663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342364347390132:3236], now have 1 active actors on pipe
2025-10-09T21:49:08.562671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342364347389884:2432], now have 1 active actors on pipe
2025-10-09T21:49:08.562685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:08.562694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:08.562716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342363985565134:6064]
2025-10-09T21:49:08.562723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342363985565134:6064]
2025-10-09T21:49:08.562725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342363985565134:6064]
2025-10-09T21:49:08.562738Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:08.562749Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:08.562752Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:08.562753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:08.562756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:08.562771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:08.562779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:08.562799Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cacf78bc-7e3db934-2b3be1b-51a11a0d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:08.562810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:08.562815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:08.562817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:08.562826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:08.562842Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:08.562874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342363985565134:6064]
2025-10-09T21:49:08.562886Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:08.562889Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:08.562896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:08.563047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:08.563056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:08.563071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:08.563155Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1426 partition: 0 MaxSeqNo: 25 sessionId: src_id|cacf78bc-7e3db934-2b3be1b-51a11a0d_0
2025-10-09T21:49:08.563487Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d31251b0-df647f89-f0717336-ad8844c5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cacf78bc-7e3db934-2b3be1b-51a11a0d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:08.566051Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cacf78bc-7e3db934-2b3be1b-51a11a0d_0] Write session: try to update token
2025-10-09T21:49:08.566069Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cacf78bc-7e3db934-2b3be1b-51a11a0d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:08.574263Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:08.574279Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:08.574289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342363985565134:6064]
2025-10-09T21:49:08.574328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:08.574336Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:08.574477Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:08.574609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:08.574732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:08.574869Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:08.575037Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:08.575071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:08.578913Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1426 sessionId: src_id|cacf78bc-7e3db934-2b3be1b-51a11a0d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:08.581212Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:08.581396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:08.581424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:08.581463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:08.581532Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:08.586879Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:08.586896Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:08.586898Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:08.586900Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:08.586911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342363985565134:6064]
2025-10-09T21:49:08.586914Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342363985565134:6064]
2025-10-09T21:49:08.586918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342363985565134:6064]
2025-10-09T21:49:08.586957Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:08.586959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:08.586966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.586969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:08.586969Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:08.586972Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:08.587099Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:08.632042Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342364347390144:3441]
2025-10-09T21:49:08.632323Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 1425, pipe: [1:7559342364347390144:3441]
2025-10-09T21:49:08.735484Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:08.742688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:08.742716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:08.742866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.742925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.742991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.743050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.743097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.743104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.743169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.743232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.743295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.743356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.743397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.743404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.743469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.743535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.743598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.743650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.743695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.743703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.743768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.743826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.743884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.743946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.743989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.743997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.744058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.744111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.744166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.744230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.744271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.744276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.744334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.744387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.744453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.744515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.744620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.744628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.744688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.744743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.744799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.744854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.744894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.744900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.744958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.745015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.745074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.745132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.745170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.745178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.745239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.745298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.745357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.745415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.745455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.745462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.745525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:08.745584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:08.745653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:08.745716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:08.745758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:08.745768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:08.745814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342364347390144:3441], now have 1 active actors on pipe
2025-10-09T21:49:08.746076Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:08.746111Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:08.746115Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:08.746147Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1426 reason: error at write: Disconnected sessionId: src_id|cacf78bc-7e3db934-2b3be1b-51a11a0d_0
2025-10-09T21:49:08.746149Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342364347390144:3441]
2025-10-09T21:49:08.746348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1426 sessionId: src_id|cacf78bc-7e3db934-2b3be1b-51a11a0d_0 is DEAD
2025-10-09T21:49:08.746474Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cacf78bc-7e3db934-2b3be1b-51a11a0d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:08.746486Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cacf78bc-7e3db934-2b3be1b-51a11a0d_0] Write session will restart in 0.009309s
2025-10-09T21:49:08.746511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cacf78bc-7e3db934-2b3be1b-51a11a0d_0] Write session: Do CDS request
2025-10-09T21:49:08.746517Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cacf78bc-7e3db934-2b3be1b-51a11a0d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:08.746685Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:08.756475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cacf78bc-7e3db934-2b3be1b-51a11a0d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:08.756590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:08.756606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1427
2025-10-09T21:49:08.756722Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1427 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:08.756755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1427 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:08.756766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1427 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:08.756770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:08.757197Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1427 sessionId:  describe result for acl check
2025-10-09T21:49:08.757238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:08.757245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:08.757247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:08.757253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342364347390207:3226] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:08.757820Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342364347390207:3226] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:08.760661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342364347390207:3226] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:08.763235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342364347390207:3226] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:08.763247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342364347390207:3226] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:08.763250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342364347390207:3226] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:08.763255Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1427 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:08.764527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:08.766898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:08.767052Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:08.767060Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:08.767069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:08.767109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:08.767138Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342363985565396:6065]
2025-10-09T21:49:08.767511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:08.767803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:08.767855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:08.768099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:08.768136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.768140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.768141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.768142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.768142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.768143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.768143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.768144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.768145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.768148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.768154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.768157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.768159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.768161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.768163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.768165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.768167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.768169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.768174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.768178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.768183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.768186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.768190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.768193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.768195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.768198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.768201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.768203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.768206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.768208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.768211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.768214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.768216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.768219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.768222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.768225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.768240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:08.774810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:08.775967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:08.775978Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:08.775980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:08.775983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:08.775991Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1426 [2:7559342363985565396:6065]
2025-10-09T21:49:08.776006Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:08.776024Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:08.776027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:08.776030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:08.776032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:08.776048Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:08.776054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:08.776061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:08.776130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.776137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.776139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.776143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:08.776210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:08.776217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.776230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.776247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.776262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:08.776289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342364347390393:3226], now have 1 active actors on pipe
2025-10-09T21:49:08.776296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342364347390147:2432], now have 1 active actors on pipe
2025-10-09T21:49:08.776304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342363985565391:6065]
2025-10-09T21:49:08.776306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342363985565391:6065]
2025-10-09T21:49:08.776308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342363985565391:6065]
2025-10-09T21:49:08.776320Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:08.776334Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:08.776340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:08.776347Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:08.776349Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:08.776356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:08.776359Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:08.776385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342363985565391:6065]
2025-10-09T21:49:08.776387Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:08.776397Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:08.776375Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1426
2025-10-09T21:49:08.776421Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1426
2025-10-09T21:49:08.776528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:08.776544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:08.776575Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|99a140e5-14fd002a-6a7b3b19-f6b5fe5b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:08.776597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:08.776604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:08.776606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:08.776617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:08.776631Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:08.776659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:08.776843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:08.776853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:08.776872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:08.776986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1427 partition: 0 MaxSeqNo: 25 sessionId: src_id|99a140e5-14fd002a-6a7b3b19-f6b5fe5b_0
2025-10-09T21:49:08.777344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cacf78bc-7e3db934-2b3be1b-51a11a0d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|99a140e5-14fd002a-6a7b3b19-f6b5fe5b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:08.779778Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99a140e5-14fd002a-6a7b3b19-f6b5fe5b_0] Write session: try to update token
2025-10-09T21:49:08.779800Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99a140e5-14fd002a-6a7b3b19-f6b5fe5b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:08.790331Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:08.790350Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:08.790351Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:08.790353Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:08.790361Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342363985565391:6065]
2025-10-09T21:49:08.790364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342363985565391:6065]
2025-10-09T21:49:08.790367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342363985565391:6065]
2025-10-09T21:49:08.790402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:08.790408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.790410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:08.790416Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:08.790422Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:08.790429Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:08.790525Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:08.792416Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1427 sessionId: src_id|99a140e5-14fd002a-6a7b3b19-f6b5fe5b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:08.794688Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:08.846094Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342364347390411:3441]
2025-10-09T21:49:08.846347Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 1426, pipe: [1:7559342364347390411:3441]
2025-10-09T21:49:08.935923Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:08.936412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:08.936426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:08.936465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342364347390411:3441], now have 1 active actors on pipe
2025-10-09T21:49:08.936623Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:08.936701Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342364347390411:3441]
2025-10-09T21:49:08.936704Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:08.936708Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:08.936745Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:08.936778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1427 reason: error at write: Disconnected sessionId: src_id|99a140e5-14fd002a-6a7b3b19-f6b5fe5b_0
2025-10-09T21:49:08.936953Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1427 sessionId: src_id|99a140e5-14fd002a-6a7b3b19-f6b5fe5b_0 is DEAD
2025-10-09T21:49:08.937099Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99a140e5-14fd002a-6a7b3b19-f6b5fe5b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:08.937112Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99a140e5-14fd002a-6a7b3b19-f6b5fe5b_0] Write session will restart in 0.005561s
2025-10-09T21:49:08.937141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99a140e5-14fd002a-6a7b3b19-f6b5fe5b_0] Write session: Do CDS request
2025-10-09T21:49:08.937147Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99a140e5-14fd002a-6a7b3b19-f6b5fe5b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:08.943479Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99a140e5-14fd002a-6a7b3b19-f6b5fe5b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:08.943641Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:08.943659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1428
2025-10-09T21:49:08.943775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1428 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:08.943805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1428 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:08.943816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1428 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:08.943820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:08.944234Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1428 sessionId:  describe result for acl check
2025-10-09T21:49:08.944264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:08.944265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:08.944266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:08.944272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342364347390429:3224] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:08.944707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342364347390429:3224] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:08.946781Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342364347390429:3224] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:08.950024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342364347390429:3224] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:08.950058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342364347390429:3224] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:08.950061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342364347390429:3224] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:08.950069Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1428 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:08.955364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:08.957734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:08.957937Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:08.957947Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:08.957959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:08.958012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:08.958074Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342363985565650:6056]
2025-10-09T21:49:08.958685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:08.959005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:08.959053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:08.959383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:08.959456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.959463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.959464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.959465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.959466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.959468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.959469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.959470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.959472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.959477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.959488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.959492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.959497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.959501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.959505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.959509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.959513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.959517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.959524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.959532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:08.959538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.959544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:08.959548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.959553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:08.959557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.959562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:08.959566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.959571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:08.959581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.959587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:08.959592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.959599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.959603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.959614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.959619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.959624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.959656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:08.966020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:08.967313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:08.967329Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:08.967332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:08.967336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:08.967343Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1427 [2:7559342363985565650:6056]
2025-10-09T21:49:08.967355Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:08.967373Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:08.967376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:08.967379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:08.967382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:08.967398Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:08.967409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:08.967417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:08.967485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.967496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.967498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.967501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:08.967568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:08.967576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.967589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:08.967602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:08.967623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:08.967650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342364347390413:2432], now have 1 active actors on pipe
2025-10-09T21:49:08.967658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342364347390561:3224], now have 1 active actors on pipe
2025-10-09T21:49:08.967666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342363985565644:6056]
2025-10-09T21:49:08.967673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342363985565644:6056]
2025-10-09T21:49:08.967676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342363985565644:6056]
2025-10-09T21:49:08.967689Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:08.967708Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:08.967710Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:08.967722Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1427
2025-10-09T21:49:08.967725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:08.967742Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1427
2025-10-09T21:49:08.967729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:08.967736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:08.967739Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:08.967761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342363985565644:6056]
2025-10-09T21:49:08.967764Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:08.967766Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:08.967785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:08.967790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:08.967841Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bef1867f-6da57642-171c573-90c4aa99_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:08.967853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:08.967857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:08.967864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:08.967877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:08.967886Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:08.967912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:08.968093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:08.968105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:08.968125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:08.968237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1428 partition: 0 MaxSeqNo: 25 sessionId: src_id|bef1867f-6da57642-171c573-90c4aa99_0
2025-10-09T21:49:08.968600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99a140e5-14fd002a-6a7b3b19-f6b5fe5b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bef1867f-6da57642-171c573-90c4aa99_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:08.971059Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bef1867f-6da57642-171c573-90c4aa99_0] Write session: try to update token
2025-10-09T21:49:08.971077Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bef1867f-6da57642-171c573-90c4aa99_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:08.982154Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:08.982170Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:08.982173Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:08.982175Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:08.982184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342363985565644:6056]
2025-10-09T21:49:08.982189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342363985565644:6056]
2025-10-09T21:49:08.982192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342363985565644:6056]
2025-10-09T21:49:08.982237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:08.982244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:08.982244Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:08.982247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:08.982250Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:08.982255Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:08.982387Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:08.983757Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1428 sessionId: src_id|bef1867f-6da57642-171c573-90c4aa99_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:08.986149Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:09.037054Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342368642357972:3441]
2025-10-09T21:49:09.037349Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 1427, pipe: [1:7559342368642357972:3441]
2025-10-09T21:49:09.132688Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:09.133178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.133195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.133233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342368642357972:3441], now have 1 active actors on pipe
2025-10-09T21:49:09.133491Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:09.133505Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:09.133548Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:09.133552Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342368642357972:3441]
2025-10-09T21:49:09.133640Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1428 reason: status is not ok: tablet will be restarted right now sessionId: src_id|bef1867f-6da57642-171c573-90c4aa99_0
2025-10-09T21:49:09.133837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1428 sessionId: src_id|bef1867f-6da57642-171c573-90c4aa99_0 is DEAD
2025-10-09T21:49:09.133977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bef1867f-6da57642-171c573-90c4aa99_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:09.133986Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bef1867f-6da57642-171c573-90c4aa99_0] Write session will restart in 0.005830s
2025-10-09T21:49:09.134006Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bef1867f-6da57642-171c573-90c4aa99_0] Write session: Do CDS request
2025-10-09T21:49:09.134012Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bef1867f-6da57642-171c573-90c4aa99_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:09.140491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bef1867f-6da57642-171c573-90c4aa99_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:09.140573Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:09.140589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1429
2025-10-09T21:49:09.140710Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1429 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:09.140753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1429 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:09.140765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1429 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:09.140769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:09.141204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1429 sessionId:  describe result for acl check
2025-10-09T21:49:09.141241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:09.141244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:09.141246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:09.141251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342368642357989:3219] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:09.141820Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342368642357989:3219] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:09.143936Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342368642357989:3219] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:09.147308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342368642357989:3219] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:09.147324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342368642357989:3219] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:09.147328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342368642357989:3219] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:09.147336Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1429 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:09.151452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:09.152001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:09.152182Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:09.152192Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:09.152202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:09.152240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:09.152278Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342368280533213:6054]
2025-10-09T21:49:09.152853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:09.154512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:09.154565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:09.154900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:09.154959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.154965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.154967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.154968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.154969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.154970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.154972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.154973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.154974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.154980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.154990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.154994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.154998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.155001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.155005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.155009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.155013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.155016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.155023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.155031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.155038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.155044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.155049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.155055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.155059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.155064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.155077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.155082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.155086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.155092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.155097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.155104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.155108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.155114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.155123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.155128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.155161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:09.161984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:09.163321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:09.163338Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:09.163340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:09.163345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:09.163351Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1428 [2:7559342368280533213:6054]
2025-10-09T21:49:09.163364Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:09.163381Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:09.163386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:09.163390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:09.163393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:09.163409Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:09.163419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:09.163427Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:09.163488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.163495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.163498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.163501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:09.163572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:09.163581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.163594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.163612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.163636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:09.163663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342368642357974:2432], now have 1 active actors on pipe
2025-10-09T21:49:09.163672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342368642358133:3219], now have 1 active actors on pipe
2025-10-09T21:49:09.163681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342368280533208:6054]
2025-10-09T21:49:09.163688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342368280533208:6054]
2025-10-09T21:49:09.163690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342368280533208:6054]
2025-10-09T21:49:09.163703Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:09.163714Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:09.163717Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:09.163718Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:09.163724Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:09.163739Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:09.163745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:09.163714Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1428
2025-10-09T21:49:09.163760Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1428
2025-10-09T21:49:09.163779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342368280533208:6054]
2025-10-09T21:49:09.163788Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:09.163790Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:09.163810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.163820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.163881Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5e32cf1b-7b6f686b-c5e1814e-2f24e96_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:09.163894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:09.163898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:09.163901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:09.163912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:09.163927Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:09.163946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:09.164063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.164071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.164087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:09.164177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1429 partition: 0 MaxSeqNo: 25 sessionId: src_id|5e32cf1b-7b6f686b-c5e1814e-2f24e96_0
2025-10-09T21:49:09.164520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bef1867f-6da57642-171c573-90c4aa99_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5e32cf1b-7b6f686b-c5e1814e-2f24e96_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:09.166986Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5e32cf1b-7b6f686b-c5e1814e-2f24e96_0] Write session: try to update token
2025-10-09T21:49:09.167002Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5e32cf1b-7b6f686b-c5e1814e-2f24e96_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:09.178790Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:09.178807Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:09.178808Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:09.178810Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:09.178817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342368280533208:6054]
2025-10-09T21:49:09.178821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342368280533208:6054]
2025-10-09T21:49:09.178823Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342368280533208:6054]
2025-10-09T21:49:09.178865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:09.178872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.178875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:09.178880Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:09.178887Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:09.178891Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:09.179019Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:09.181167Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1429 sessionId: src_id|5e32cf1b-7b6f686b-c5e1814e-2f24e96_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:09.183766Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:09.233669Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342368642358263:3441]
2025-10-09T21:49:09.233983Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 1428, pipe: [1:7559342368642358263:3441]
2025-10-09T21:49:09.332036Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:09.332477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.332498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.332548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342368642358263:3441], now have 1 active actors on pipe
2025-10-09T21:49:09.332774Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:09.332779Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:09.332783Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:09.332787Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:09.332835Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342368642358263:3441]
2025-10-09T21:49:09.332850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1429 reason: error at write: Disconnected sessionId: src_id|5e32cf1b-7b6f686b-c5e1814e-2f24e96_0
2025-10-09T21:49:09.333032Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1429 sessionId: src_id|5e32cf1b-7b6f686b-c5e1814e-2f24e96_0 is DEAD
2025-10-09T21:49:09.333198Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e32cf1b-7b6f686b-c5e1814e-2f24e96_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:09.333209Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e32cf1b-7b6f686b-c5e1814e-2f24e96_0] Write session will restart in 0.009269s
2025-10-09T21:49:09.333237Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e32cf1b-7b6f686b-c5e1814e-2f24e96_0] Write session: Do CDS request
2025-10-09T21:49:09.333244Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e32cf1b-7b6f686b-c5e1814e-2f24e96_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:09.343519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5e32cf1b-7b6f686b-c5e1814e-2f24e96_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:09.343622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:09.343640Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1430
2025-10-09T21:49:09.343791Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1430 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:09.343825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1430 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:09.343835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1430 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:09.343840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:09.344320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1430 sessionId:  describe result for acl check
2025-10-09T21:49:09.344362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:09.344368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:09.344370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:09.344376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342368642358326:3197] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:09.344919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342368642358326:3197] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:09.347571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342368642358326:3197] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:09.350115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342368642358326:3197] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:09.350130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342368642358326:3197] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:09.350134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342368642358326:3197] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:09.350140Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1430 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:09.351760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:09.354292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:09.354504Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:09.354516Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:09.354529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:09.354585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:09.354627Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342368280533493:6035]
2025-10-09T21:49:09.355182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:09.355552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:09.355605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:09.355935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:09.355979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.355984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.355985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.355986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.355987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.355988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.355989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.355990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.355991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.355996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.356008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.356013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.356017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.356021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.356026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.356030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.356034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.356038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.356045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.356054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.356061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.356066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.356071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.356076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.356080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.356091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.356096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.356101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.356106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.356111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.356120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.356126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.356130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.356135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.356141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.356149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.356177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:09.362534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:09.363803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:09.363819Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:09.363821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:09.363826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:09.363832Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1429 [2:7559342368280533493:6035]
2025-10-09T21:49:09.363847Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:09.363865Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:09.363873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:09.363876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:09.363878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:09.363894Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:09.363904Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:09.363917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:09.363983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.363991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.363993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.363996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:09.364063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:09.364071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.364084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.364104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.364121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:09.364146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342368642358521:3197], now have 1 active actors on pipe
2025-10-09T21:49:09.364154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342368642358270:2432], now have 1 active actors on pipe
2025-10-09T21:49:09.364163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342368280533488:6035]
2025-10-09T21:49:09.364169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342368280533488:6035]
2025-10-09T21:49:09.364172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342368280533488:6035]
2025-10-09T21:49:09.364184Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:09.364199Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:09.364202Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:09.364204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:09.364204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:09.364219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:09.364189Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1429
2025-10-09T21:49:09.364222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:09.364195Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1429
2025-10-09T21:49:09.364256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342368280533488:6035]
2025-10-09T21:49:09.364267Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:09.364282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.364282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:09.364291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.364340Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9451ffc0-be57e030-7c3728e8-9c1d1ec0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:09.364353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:09.364357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:09.364359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:09.364370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:09.364379Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:09.364405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:09.364575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.364581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.364595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:09.364685Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1430 partition: 0 MaxSeqNo: 25 sessionId: src_id|9451ffc0-be57e030-7c3728e8-9c1d1ec0_0
2025-10-09T21:49:09.365016Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e32cf1b-7b6f686b-c5e1814e-2f24e96_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9451ffc0-be57e030-7c3728e8-9c1d1ec0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:09.367495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9451ffc0-be57e030-7c3728e8-9c1d1ec0_0] Write session: try to update token
2025-10-09T21:49:09.367515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9451ffc0-be57e030-7c3728e8-9c1d1ec0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:09.378215Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:09.378234Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:09.378236Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:09.378238Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:09.378245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342368280533488:6035]
2025-10-09T21:49:09.378248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342368280533488:6035]
2025-10-09T21:49:09.378251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342368280533488:6035]
2025-10-09T21:49:09.378287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:09.378292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.378294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:09.378296Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:09.378301Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:09.378305Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:09.378422Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:09.380555Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1430 sessionId: src_id|9451ffc0-be57e030-7c3728e8-9c1d1ec0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:09.382857Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:09.432781Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342368642358535:3441]
2025-10-09T21:49:09.433077Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 1429, pipe: [1:7559342368642358535:3441]
2025-10-09T21:49:09.526020Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:09.526268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.526283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.526321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342368642358535:3441], now have 1 active actors on pipe
2025-10-09T21:49:09.526509Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:09.526595Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342368642358535:3441]
2025-10-09T21:49:09.526627Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:09.527091Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:09.527168Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1430 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9451ffc0-be57e030-7c3728e8-9c1d1ec0_0
2025-10-09T21:49:09.527346Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1430 sessionId: src_id|9451ffc0-be57e030-7c3728e8-9c1d1ec0_0 is DEAD
2025-10-09T21:49:09.527490Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9451ffc0-be57e030-7c3728e8-9c1d1ec0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:09.527507Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9451ffc0-be57e030-7c3728e8-9c1d1ec0_0] Write session will restart in 0.005530s
2025-10-09T21:49:09.527529Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9451ffc0-be57e030-7c3728e8-9c1d1ec0_0] Write session: Do CDS request
2025-10-09T21:49:09.527534Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9451ffc0-be57e030-7c3728e8-9c1d1ec0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:09.534532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9451ffc0-be57e030-7c3728e8-9c1d1ec0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:09.534587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:09.534611Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1431
2025-10-09T21:49:09.534747Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1431 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:09.534782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1431 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:09.534792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1431 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:09.534798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:09.535189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1431 sessionId:  describe result for acl check
2025-10-09T21:49:09.535220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:09.535225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:09.535226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:09.535230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342368642358563:3196] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:09.535649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342368642358563:3196] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:09.538515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342368642358563:3196] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:09.541334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342368642358563:3196] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:09.541347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342368642358563:3196] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:09.541349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342368642358563:3196] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:09.541354Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1431 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:09.544822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:09.547570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:09.547757Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:09.547768Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:09.547779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:09.547828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:09.547870Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342368280533749:6032]
2025-10-09T21:49:09.548438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:09.548747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:09.548797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:09.549110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:09.549160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.549165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.549166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.549167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.549169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.549170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.549172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.549173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.549174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.549178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.549188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.549192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.549195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.549198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.549202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.549206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.549209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.549212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.549218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.549226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.549232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.549238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.549242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.549248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.549252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.549256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.549261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.549266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.549276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.549282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.549287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.549294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.549298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.549303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.549307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.549312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.549339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:09.555477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:09.556566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:09.556584Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:09.556586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:09.556590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:09.556596Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1430 [2:7559342368280533749:6032]
2025-10-09T21:49:09.556610Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:09.556626Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:09.556634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:09.556637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:09.556639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:09.556655Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:09.556665Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:09.556673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:09.556731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.556739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.556742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.556744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:09.556806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:09.556814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.557007Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1430
2025-10-09T21:49:09.557016Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1430
2025-10-09T21:49:09.557062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.557083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.557104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:09.557130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342368642358728:3196], now have 1 active actors on pipe
2025-10-09T21:49:09.557138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342368642358538:2432], now have 1 active actors on pipe
2025-10-09T21:49:09.557153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.557162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.557183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342368280533744:6032]
2025-10-09T21:49:09.557189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342368280533744:6032]
2025-10-09T21:49:09.557191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342368280533744:6032]
2025-10-09T21:49:09.557205Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:09.557217Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:09.557220Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:09.557221Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:09.557223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:09.557237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:09.557240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:09.557258Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a83924a3-53459ea7-67ba2858-1ed8aac3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:09.557270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:09.557275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:09.557277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:09.557288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:09.557302Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:09.557329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342368280533744:6032]
2025-10-09T21:49:09.557340Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:09.557351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:09.557376Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:09.557522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.557532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.557546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:09.557649Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1431 partition: 0 MaxSeqNo: 25 sessionId: src_id|a83924a3-53459ea7-67ba2858-1ed8aac3_0
2025-10-09T21:49:09.557994Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9451ffc0-be57e030-7c3728e8-9c1d1ec0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a83924a3-53459ea7-67ba2858-1ed8aac3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:09.560388Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a83924a3-53459ea7-67ba2858-1ed8aac3_0] Write session: try to update token
2025-10-09T21:49:09.560405Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a83924a3-53459ea7-67ba2858-1ed8aac3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:09.572980Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1431 sessionId: src_id|a83924a3-53459ea7-67ba2858-1ed8aac3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:09.575079Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:09.575094Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:09.575104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342368280533744:6032]
2025-10-09T21:49:09.575145Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:09.575155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:09.575161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:09.575345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:09.575526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:09.575682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:09.575832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:09.576022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:09.576077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.576092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.576114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:09.576154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:09.576262Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:09.581850Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:09.581864Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:09.581867Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:09.581869Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:09.581878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342368280533744:6032]
2025-10-09T21:49:09.581882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342368280533744:6032]
2025-10-09T21:49:09.581886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342368280533744:6032]
2025-10-09T21:49:09.581929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:09.581933Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:09.581938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.581938Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:09.581940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:09.581942Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:09.582092Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:09.626878Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342368642358803:3441]
2025-10-09T21:49:09.627108Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 1430, pipe: [1:7559342368642358803:3441]
2025-10-09T21:49:09.726299Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:09.732639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.732663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.732803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.732855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.732917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.732993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.733039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.733046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.733106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.733158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.733211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.733266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.733425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.733431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.733482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.733530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.733591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.733657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.733694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.733699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.733754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.733815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.733878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.733958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.733998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.734005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.734108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.734158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.734208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.734258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.734294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.734302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.734357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.734407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.734459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.734515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.734554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.734556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.734610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.734659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.734710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.734758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.734793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.734798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.734849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.734896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.734944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.734992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.735028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.735033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.735086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.735144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.735192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.735243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.735291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.735296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.735345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.735395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.735447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.735499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.735535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.735540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.735578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342368642358803:3441], now have 1 active actors on pipe
2025-10-09T21:49:09.735809Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:09.735867Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342368642358803:3441]
2025-10-09T21:49:09.735948Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:09.736041Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:09.736082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1431 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a83924a3-53459ea7-67ba2858-1ed8aac3_0
2025-10-09T21:49:09.736245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1431 sessionId: src_id|a83924a3-53459ea7-67ba2858-1ed8aac3_0 is DEAD
2025-10-09T21:49:09.736390Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a83924a3-53459ea7-67ba2858-1ed8aac3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:09.736403Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a83924a3-53459ea7-67ba2858-1ed8aac3_0] Write session will restart in 0.005537s
2025-10-09T21:49:09.736429Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a83924a3-53459ea7-67ba2858-1ed8aac3_0] Write session: Do CDS request
2025-10-09T21:49:09.736436Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a83924a3-53459ea7-67ba2858-1ed8aac3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:09.743444Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a83924a3-53459ea7-67ba2858-1ed8aac3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:09.743533Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:09.743557Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1432
2025-10-09T21:49:09.743673Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1432 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:09.743708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1432 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:09.743718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1432 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:09.743725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:09.744107Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1432 sessionId:  describe result for acl check
2025-10-09T21:49:09.744135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:09.744140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:09.744141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:09.744146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342368642358822:3188] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:09.744560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342368642358822:3188] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:09.747664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342368642358822:3188] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:09.750459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342368642358822:3188] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:09.750480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342368642358822:3188] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:09.750483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342368642358822:3188] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:09.750488Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1432 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:09.754354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:09.756603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:09.756792Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:09.756805Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:09.756818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:09.756869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:09.756906Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342368280534010:6028]
2025-10-09T21:49:09.757495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:09.757822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:09.757872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:09.758189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:09.758254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.758263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.758265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.758266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.758267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.758268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.758269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.758271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.758272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.758277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.758289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.758293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.758297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.758301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.758305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.758309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.758313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.758317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.758325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.758333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.758344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.758355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.758363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.758368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.758372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.758377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.758381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.758386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.758396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.758401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.758405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.758411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.758415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.758420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.758430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.758440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.758471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:09.764531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:09.765578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:09.765595Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:09.765597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:09.765601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:09.765608Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1431 [2:7559342368280534010:6028]
2025-10-09T21:49:09.765629Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:09.765646Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:09.765649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:09.765652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:09.765654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:09.765670Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:09.765679Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:09.765686Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:09.765733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.765740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.765741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.765743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:09.765790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:09.765794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.765802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.765808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.765818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:09.765832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342368280534005:6028]
2025-10-09T21:49:09.765837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342368280534005:6028]
2025-10-09T21:49:09.765838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342368280534005:6028]
2025-10-09T21:49:09.765844Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:09.765854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:09.765864Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:09.765869Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:09.765869Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:09.765872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:09.765882Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:09.765887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342368280534005:6028]
2025-10-09T21:49:09.765889Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:09.765893Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:09.765896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342368642358972:3188], now have 1 active actors on pipe
2025-10-09T21:49:09.765905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342368642358806:2432], now have 1 active actors on pipe
2025-10-09T21:49:09.765955Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1431
2025-10-09T21:49:09.765956Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1431
2025-10-09T21:49:09.765995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.766002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.766026Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|76020315-4acdae71-2105c4cf-6acd42ab_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:09.766045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:09.766048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:09.766049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:09.766057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:09.766065Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:09.766076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:09.766260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.766275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.766298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:09.766419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1432 partition: 0 MaxSeqNo: 25 sessionId: src_id|76020315-4acdae71-2105c4cf-6acd42ab_0
2025-10-09T21:49:09.766745Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a83924a3-53459ea7-67ba2858-1ed8aac3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|76020315-4acdae71-2105c4cf-6acd42ab_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:09.769094Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76020315-4acdae71-2105c4cf-6acd42ab_0] Write session: try to update token
2025-10-09T21:49:09.769109Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76020315-4acdae71-2105c4cf-6acd42ab_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:09.782678Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1432 sessionId: src_id|76020315-4acdae71-2105c4cf-6acd42ab_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:09.783414Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:09.783434Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:09.783445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342368280534005:6028]
2025-10-09T21:49:09.783491Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:09.783498Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:09.783674Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:09.783841Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:09.783995Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:09.784137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:09.784334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:09.784377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:09.784867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:09.785031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.785057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.785092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:09.785162Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:09.789852Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:09.789868Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:09.789869Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:09.789871Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:09.789878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342368280534005:6028]
2025-10-09T21:49:09.789881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342368280534005:6028]
2025-10-09T21:49:09.789889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342368280534005:6028]
2025-10-09T21:49:09.789919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:09.789924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.789926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:09.789952Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:09.789967Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:09.789970Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:09.790064Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:09.836274Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342368642359074:3441]
2025-10-09T21:49:09.836535Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 1431, pipe: [1:7559342368642359074:3441]
2025-10-09T21:49:09.931165Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:09.937485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.937508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.937659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.937713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.937771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.937827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.937867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.937874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.937934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.937985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.938058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.938131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.938173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.938179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.938233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.938282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.938337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.938401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.938440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.938445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.938503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.938558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.938613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.938664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.938701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.938708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.938765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.938821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.938881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.938934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.938973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.938978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.939033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.939084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.939136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.939187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.939226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.939231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.939283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.939330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.939386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.939446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.939493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.939500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.939553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.939601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.939651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.939718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.939761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.939771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.939827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.939873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.939923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.939976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.940012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.940017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.940068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:09.940118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:09.940169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:09.940224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:09.940260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:09.940270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:09.940310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342368642359074:3441], now have 1 active actors on pipe
2025-10-09T21:49:09.940723Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:09.940785Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342368642359074:3441]
2025-10-09T21:49:09.940789Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:09.940792Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:09.940819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1432 reason: error at write: Disconnected sessionId: src_id|76020315-4acdae71-2105c4cf-6acd42ab_0
2025-10-09T21:49:09.940983Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:09.940997Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1432 sessionId: src_id|76020315-4acdae71-2105c4cf-6acd42ab_0 is DEAD
2025-10-09T21:49:09.941124Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76020315-4acdae71-2105c4cf-6acd42ab_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:09.941132Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76020315-4acdae71-2105c4cf-6acd42ab_0] Write session will restart in 0.008053s
2025-10-09T21:49:09.941157Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76020315-4acdae71-2105c4cf-6acd42ab_0] Write session: Do CDS request
2025-10-09T21:49:09.941162Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76020315-4acdae71-2105c4cf-6acd42ab_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:09.950466Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76020315-4acdae71-2105c4cf-6acd42ab_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:09.950548Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:09.950566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1433
2025-10-09T21:49:09.950698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1433 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:09.950736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1433 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:09.950743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1433 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:09.950747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:09.951246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1433 sessionId:  describe result for acl check
2025-10-09T21:49:09.951283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:09.951285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:09.951287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:09.951292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342368642359117:3184] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:09.951844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342368642359117:3184] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:09.954465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342368642359117:3184] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:09.957442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342368642359117:3184] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:09.957459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342368642359117:3184] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:09.957463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342368642359117:3184] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:09.957490Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1433 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:09.959591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:09.961955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:09.962166Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:09.962179Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:09.962194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:09.962256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:09.962308Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342368280534269:6022]
2025-10-09T21:49:09.962839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:09.963138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:09.963185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:09.963495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:09.963558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.963567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.963568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.963569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.963570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.963571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.963572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.963573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.963574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.963580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.963590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.963594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.963598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.963602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.963606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.963610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.963614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.963618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.963636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.963644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:09.963651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.963657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:09.963661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.963667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:09.963671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.963676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:09.963680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.963685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:09.963690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.963695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:09.963699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.963708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.963712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.963721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.963725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.963730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.963754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:09.969742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:09.970786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:09.970803Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:09.970805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:09.970809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:09.970815Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1432 [2:7559342368280534269:6022]
2025-10-09T21:49:09.970828Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:09.970844Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:09.970852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:09.970855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:09.970858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:09.970873Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:09.970883Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:09.970890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:09.970947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.970954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.970956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.970958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:09.971010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:09.971016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.971023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:09.971035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:09.971052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:09.971075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342368642359077:2432], now have 1 active actors on pipe
2025-10-09T21:49:09.971082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342368642359325:3184], now have 1 active actors on pipe
2025-10-09T21:49:09.971090Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342368280534264:6022]
2025-10-09T21:49:09.971093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342368280534264:6022]
2025-10-09T21:49:09.971095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342368280534264:6022]
2025-10-09T21:49:09.971107Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:09.971126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:09.971133Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:09.971135Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:09.971137Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:09.971145Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:09.971149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:09.971113Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1432
2025-10-09T21:49:09.971133Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1432
2025-10-09T21:49:09.971178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.971185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.971194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342368280534264:6022]
2025-10-09T21:49:09.971199Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:09.971210Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:09.971221Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c3ffa174-b49679fe-e516bbb2-2614ed4e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:09.971230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:09.971232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:09.971234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:09.971241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:09.971248Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:09.971264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:09.971439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:09.971461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:09.971480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:09.971579Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1433 partition: 0 MaxSeqNo: 25 sessionId: src_id|c3ffa174-b49679fe-e516bbb2-2614ed4e_0
2025-10-09T21:49:09.971919Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76020315-4acdae71-2105c4cf-6acd42ab_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c3ffa174-b49679fe-e516bbb2-2614ed4e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:09.974187Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3ffa174-b49679fe-e516bbb2-2614ed4e_0] Write session: try to update token
2025-10-09T21:49:09.974205Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3ffa174-b49679fe-e516bbb2-2614ed4e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:09.984906Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:09.984924Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:09.984926Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:09.984928Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:09.984938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342368280534264:6022]
2025-10-09T21:49:09.984943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342368280534264:6022]
2025-10-09T21:49:09.984947Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342368280534264:6022]
2025-10-09T21:49:09.984991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:09.984994Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:09.984997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:09.984999Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:09.985000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:09.985003Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:09.985127Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:09.985860Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1433 sessionId: src_id|c3ffa174-b49679fe-e516bbb2-2614ed4e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:09.988091Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:10.041086Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342372937326641:3441]
2025-10-09T21:49:10.041365Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 1432, pipe: [1:7559342372937326641:3441]
2025-10-09T21:49:10.139950Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:10.140693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.140715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.140762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342372937326641:3441], now have 1 active actors on pipe
2025-10-09T21:49:10.141073Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342372937326641:3441]
2025-10-09T21:49:10.141084Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:10.141117Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:10.141119Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:10.141183Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1433 reason: error at write: Disconnected sessionId: src_id|c3ffa174-b49679fe-e516bbb2-2614ed4e_0
2025-10-09T21:49:10.141384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1433 sessionId: src_id|c3ffa174-b49679fe-e516bbb2-2614ed4e_0 is DEAD
2025-10-09T21:49:10.141559Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3ffa174-b49679fe-e516bbb2-2614ed4e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:10.141571Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3ffa174-b49679fe-e516bbb2-2614ed4e_0] Write session will restart in 0.006311s
2025-10-09T21:49:10.141596Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3ffa174-b49679fe-e516bbb2-2614ed4e_0] Write session: Do CDS request
2025-10-09T21:49:10.141606Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3ffa174-b49679fe-e516bbb2-2614ed4e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:10.141909Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:10.149560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3ffa174-b49679fe-e516bbb2-2614ed4e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:10.149650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:10.149668Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1434
2025-10-09T21:49:10.149816Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1434 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:10.149852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1434 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:10.149863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1434 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:10.149866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:10.150340Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1434 sessionId:  describe result for acl check
2025-10-09T21:49:10.150373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:10.150381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:10.150382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:10.150387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342372937326673:3179] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:10.150918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342372937326673:3179] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:10.154009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342372937326673:3179] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:10.157453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342372937326673:3179] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:10.157473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342372937326673:3179] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:10.157477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342372937326673:3179] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:10.157486Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1434 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:10.159939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:10.162912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:10.163128Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:10.163137Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:10.163149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:10.163203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:10.163252Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342372575501824:6025]
2025-10-09T21:49:10.163758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:10.164025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:10.164065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:10.164298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:10.164346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.164352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.164353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.164354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.164356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.164357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.164359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.164360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.164361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.164367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.164378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.164383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.164387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.164391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.164394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.164398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.164402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.164406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.164414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.164422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.164429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.164435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.164440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.164446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.164450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.164461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.164467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.164473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.164477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.164487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.164494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.164500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.164507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.164512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.164517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.164523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.164556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:10.171201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:10.172692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:10.172708Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:10.172710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:10.172713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:10.172720Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1433 [2:7559342372575501824:6025]
2025-10-09T21:49:10.172734Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:10.172753Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:10.172755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:10.172757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:10.172758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:10.172773Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:10.172777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:10.172783Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:10.172847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.172853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.172854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.172856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:10.172911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:10.172917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.172930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.172941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.172964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:10.172992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342372937326643:2432], now have 1 active actors on pipe
2025-10-09T21:49:10.173002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342372937326869:3179], now have 1 active actors on pipe
2025-10-09T21:49:10.173011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342372575501819:6025]
2025-10-09T21:49:10.173014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342372575501819:6025]
2025-10-09T21:49:10.173016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342372575501819:6025]
2025-10-09T21:49:10.173031Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:10.173045Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:10.173054Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:10.173057Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1433
2025-10-09T21:49:10.173058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:10.173059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:10.173058Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1433
2025-10-09T21:49:10.173060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:10.173063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:10.173103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342372575501819:6025]
2025-10-09T21:49:10.173107Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:10.173129Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:10.173140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.173154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.173185Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|540fa49e-75530c35-b8233d86-bf57c788_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:10.173196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:10.173199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:10.173201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:10.173208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:10.173220Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:10.173240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:10.173372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.173380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.173393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:10.173495Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1434 partition: 0 MaxSeqNo: 25 sessionId: src_id|540fa49e-75530c35-b8233d86-bf57c788_0
2025-10-09T21:49:10.173901Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3ffa174-b49679fe-e516bbb2-2614ed4e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|540fa49e-75530c35-b8233d86-bf57c788_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:10.176436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|540fa49e-75530c35-b8233d86-bf57c788_0] Write session: try to update token
2025-10-09T21:49:10.176455Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|540fa49e-75530c35-b8233d86-bf57c788_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:10.188001Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:10.188022Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:10.188025Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:10.188027Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:10.188037Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342372575501819:6025]
2025-10-09T21:49:10.188042Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342372575501819:6025]
2025-10-09T21:49:10.188045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342372575501819:6025]
2025-10-09T21:49:10.188091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:10.188097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.188097Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:10.188100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:10.188103Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:10.188107Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:10.188241Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:10.190666Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1434 sessionId: src_id|540fa49e-75530c35-b8233d86-bf57c788_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:10.193174Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:10.241265Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342372937326941:3441]
2025-10-09T21:49:10.241414Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 1433, pipe: [1:7559342372937326941:3441]
2025-10-09T21:49:10.331503Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:10.331738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.331750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.331791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342372937326941:3441], now have 1 active actors on pipe
2025-10-09T21:49:10.331969Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:10.331980Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:10.331986Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342372937326941:3441]
2025-10-09T21:49:10.332004Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:10.332072Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1434 reason: status is not ok: tablet will be restarted right now sessionId: src_id|540fa49e-75530c35-b8233d86-bf57c788_0
2025-10-09T21:49:10.332226Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1434 sessionId: src_id|540fa49e-75530c35-b8233d86-bf57c788_0 is DEAD
2025-10-09T21:49:10.332360Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|540fa49e-75530c35-b8233d86-bf57c788_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:10.332369Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|540fa49e-75530c35-b8233d86-bf57c788_0] Write session will restart in 0.007133s
2025-10-09T21:49:10.332389Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|540fa49e-75530c35-b8233d86-bf57c788_0] Write session: Do CDS request
2025-10-09T21:49:10.332394Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|540fa49e-75530c35-b8233d86-bf57c788_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:10.340354Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|540fa49e-75530c35-b8233d86-bf57c788_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:10.340470Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:10.340483Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1435
2025-10-09T21:49:10.340956Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1435 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:10.340989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1435 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:10.340999Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1435 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:10.341002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:10.341363Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1435 sessionId:  describe result for acl check
2025-10-09T21:49:10.341395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:10.341402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:10.341403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:10.341408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342372937326974:3162] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:10.341883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342372937326974:3162] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:10.344771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342372937326974:3162] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:10.347447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342372937326974:3162] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:10.347463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342372937326974:3162] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:10.347468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342372937326974:3162] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:10.347474Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1435 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:10.350201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:10.352382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:10.352582Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:10.352592Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:10.352604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:10.352668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:10.352707Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342372575502122:5978]
2025-10-09T21:49:10.353171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:10.353460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:10.353510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:10.353765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:10.353805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.353810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.353811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.353811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.353812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.353813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.353813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.353814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.353815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.353818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.353825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.353827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.353829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.353831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.353833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.353835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.353837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.353839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.353843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.353848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.353852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.353855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.353857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.353861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.353863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.353866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.353869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.353872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.353875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.353878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.353881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.353884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.353887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.353889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.353893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.353896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.353911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:10.359708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:10.360722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:10.360735Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:10.360736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:10.360739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:10.360743Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1434 [2:7559342372575502122:5978]
2025-10-09T21:49:10.360754Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:10.360767Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:10.360768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:10.360770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:10.360771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:10.360783Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:10.360786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:10.360791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:10.360832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.360838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.360839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.360841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:10.360898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:10.360903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.360911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.360916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.360926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:10.360944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342372937326944:2432], now have 1 active actors on pipe
2025-10-09T21:49:10.360954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342372575502117:5978]
2025-10-09T21:49:10.360955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342372575502117:5978]
2025-10-09T21:49:10.360956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342372575502117:5978]
2025-10-09T21:49:10.360964Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:10.360971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342372937327168:3162], now have 1 active actors on pipe
2025-10-09T21:49:10.360978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:10.360987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:10.360989Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:10.360992Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:10.360993Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:10.360995Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:10.361009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342372575502117:5978]
2025-10-09T21:49:10.361011Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:10.361013Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:10.361038Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1434
2025-10-09T21:49:10.361054Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1434
2025-10-09T21:49:10.361142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.361153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.361182Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|36ca0377-32ec14a8-24f6bf-6c232c4f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:10.361194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:10.361202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:10.361205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:10.361215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:10.361229Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:10.361247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:10.361429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.361438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.361452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:10.361539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1435 partition: 0 MaxSeqNo: 25 sessionId: src_id|36ca0377-32ec14a8-24f6bf-6c232c4f_0
2025-10-09T21:49:10.361872Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|540fa49e-75530c35-b8233d86-bf57c788_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|36ca0377-32ec14a8-24f6bf-6c232c4f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:10.364203Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36ca0377-32ec14a8-24f6bf-6c232c4f_0] Write session: try to update token
2025-10-09T21:49:10.364218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36ca0377-32ec14a8-24f6bf-6c232c4f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:10.371244Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:10.371261Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:10.371270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342372575502117:5978]
2025-10-09T21:49:10.371310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:10.371312Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:10.371539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:10.371687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:10.371834Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:10.371968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:10.372142Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:10.372183Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:10.376298Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1435 sessionId: src_id|36ca0377-32ec14a8-24f6bf-6c232c4f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:10.378597Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:10.378777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.378796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.378835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:10.378906Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:10.383492Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:10.383507Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:10.383509Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:10.383510Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:10.383517Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342372575502117:5978]
2025-10-09T21:49:10.383520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342372575502117:5978]
2025-10-09T21:49:10.383524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342372575502117:5978]
2025-10-09T21:49:10.383555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:10.383560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.383561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:10.383566Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:10.383573Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:10.383577Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:10.383681Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:10.432250Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342372937327214:3441]
2025-10-09T21:49:10.432488Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 1434, pipe: [1:7559342372937327214:3441]
2025-10-09T21:49:10.528242Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:10.535667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.535695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.535850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.535910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.535971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.536032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.536078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.536091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.536156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.536215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.536279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.536340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.536382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.536390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.536449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.536543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.536597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.536656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.536694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.536704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.536765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.536825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.536885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.536942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.537037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.537045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.537104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.537161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.537215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.537275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.537315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.537323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.537378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.537455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.537512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.537571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.537610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.537627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.537685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.537742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.537799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.537856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.537897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.537905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.537959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.538013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.538080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.538137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.538233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.538241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.538295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.538348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.538403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.538458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.538498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.538508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.538564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.538621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.538673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.538726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.538782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.538790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.538844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342372937327214:3441], now have 1 active actors on pipe
2025-10-09T21:49:10.539084Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342372937327214:3441]
2025-10-09T21:49:10.539114Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:10.539228Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:10.539240Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:10.539263Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1435 reason: error at write: Disconnected sessionId: src_id|36ca0377-32ec14a8-24f6bf-6c232c4f_0
2025-10-09T21:49:10.539423Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1435 sessionId: src_id|36ca0377-32ec14a8-24f6bf-6c232c4f_0 is DEAD
2025-10-09T21:49:10.539550Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36ca0377-32ec14a8-24f6bf-6c232c4f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:10.539559Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36ca0377-32ec14a8-24f6bf-6c232c4f_0] Write session will restart in 0.008618s
2025-10-09T21:49:10.539577Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36ca0377-32ec14a8-24f6bf-6c232c4f_0] Write session: Do CDS request
2025-10-09T21:49:10.539582Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36ca0377-32ec14a8-24f6bf-6c232c4f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:10.539657Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:10.549431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36ca0377-32ec14a8-24f6bf-6c232c4f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:10.549508Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:10.549523Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1436
2025-10-09T21:49:10.549671Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1436 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:10.549704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1436 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:10.549715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1436 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:10.549721Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:10.550158Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1436 sessionId:  describe result for acl check
2025-10-09T21:49:10.550191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:10.550196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:10.550197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:10.550201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342372937327283:3159] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:10.550690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342372937327283:3159] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:10.553235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342372937327283:3159] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:10.556085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342372937327283:3159] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:10.556096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342372937327283:3159] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:10.556098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342372937327283:3159] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:10.556103Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1436 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:10.557165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:10.559456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:10.559622Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:10.559632Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:10.559643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:10.559700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:10.559741Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342372575502383:5972]
2025-10-09T21:49:10.560306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:10.560592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:10.560633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:10.560933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:10.560977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.560983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.560984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.560985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.560986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.560987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.560988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.560989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.560990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.560994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.561005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.561009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.561013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.561017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.561021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.561025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.561029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.561033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.561039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.561045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.561053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.561059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.561068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.561073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.561076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.561085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.561092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.561101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.561106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.561115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.561120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.561125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.561129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.561134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.561139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.561144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.561173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:10.567213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:10.568144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:10.568161Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:10.568164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:10.568168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:10.568173Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1435 [2:7559342372575502383:5972]
2025-10-09T21:49:10.568189Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:10.568210Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:10.568218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:10.568220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:10.568221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:10.568233Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:10.568240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:10.568246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:10.568283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.568289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.568290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.568292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:10.568335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:10.568340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.568347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.568353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.568362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:10.568378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342372575502378:5972]
2025-10-09T21:49:10.568383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342372575502378:5972]
2025-10-09T21:49:10.568384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342372575502378:5972]
2025-10-09T21:49:10.568390Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:10.568400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342372937327217:2432], now have 1 active actors on pipe
2025-10-09T21:49:10.568401Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:10.568404Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:10.568406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342372937327473:3159], now have 1 active actors on pipe
2025-10-09T21:49:10.568406Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:10.568412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:10.568422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:10.568429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:10.568449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342372575502378:5972]
2025-10-09T21:49:10.568463Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:10.568485Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:10.568466Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1435
2025-10-09T21:49:10.568481Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1435
2025-10-09T21:49:10.568550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.568558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.568577Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4e63be97-a83afdf-b1561c59-6335e5c7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:10.568583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:10.568588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:10.568589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:10.568596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:10.568603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:10.568613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:10.568718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.568724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.568732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:10.568796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1436 partition: 0 MaxSeqNo: 25 sessionId: src_id|4e63be97-a83afdf-b1561c59-6335e5c7_0
2025-10-09T21:49:10.569106Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36ca0377-32ec14a8-24f6bf-6c232c4f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4e63be97-a83afdf-b1561c59-6335e5c7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:10.571490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e63be97-a83afdf-b1561c59-6335e5c7_0] Write session: try to update token
2025-10-09T21:49:10.571507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e63be97-a83afdf-b1561c59-6335e5c7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:10.579637Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:10.579655Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:10.579664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342372575502378:5972]
2025-10-09T21:49:10.579705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:10.579714Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:10.579872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:10.580006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:10.580136Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:10.580262Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:10.580446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:10.580486Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:10.583852Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1436 sessionId: src_id|4e63be97-a83afdf-b1561c59-6335e5c7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:10.585986Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:10.586455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.586476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.586510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:10.586606Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:10.592879Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:10.592903Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:10.592906Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:10.592908Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:10.592919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342372575502378:5972]
2025-10-09T21:49:10.592924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342372575502378:5972]
2025-10-09T21:49:10.592928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342372575502378:5972]
2025-10-09T21:49:10.592971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:10.592983Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:10.592987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.592989Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:10.592990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:10.592994Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:10.593124Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:10.639094Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342372937327487:3441]
2025-10-09T21:49:10.639345Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 1435, pipe: [1:7559342372937327487:3441]
2025-10-09T21:49:10.732206Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:10.738779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.738802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.738942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.738994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.739050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.739109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.739149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.739157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.739224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.739277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.739332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.739385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.739525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.739532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.739586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.739634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.739684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.739741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.739777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.739784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.739840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.739891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.739944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.740003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.740041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.740046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.740107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.740158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.740216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.740271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.740307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.740313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.740364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.740412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.740465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.740520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.740556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.740561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.740611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.740655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.740708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.740761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.740813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.740818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.740869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.740916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.740966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.741022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.741058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.741064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.741121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.741173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.741217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.741281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.741319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.741328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.741392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:10.741447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:10.741502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:10.741557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:10.741601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:10.741608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:10.741661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342372937327487:3441], now have 1 active actors on pipe
2025-10-09T21:49:10.741921Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342372937327487:3441]
2025-10-09T21:49:10.741948Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:10.741960Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:10.741963Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:10.742000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1436 reason: error at write: Disconnected sessionId: src_id|4e63be97-a83afdf-b1561c59-6335e5c7_0
2025-10-09T21:49:10.742192Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1436 sessionId: src_id|4e63be97-a83afdf-b1561c59-6335e5c7_0 is DEAD
2025-10-09T21:49:10.742337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e63be97-a83afdf-b1561c59-6335e5c7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:10.742348Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e63be97-a83afdf-b1561c59-6335e5c7_0] Write session will restart in 0.005318s
2025-10-09T21:49:10.742367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e63be97-a83afdf-b1561c59-6335e5c7_0] Write session: Do CDS request
2025-10-09T21:49:10.742372Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e63be97-a83afdf-b1561c59-6335e5c7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:10.742533Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:10.748443Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e63be97-a83afdf-b1561c59-6335e5c7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:10.748515Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:10.748528Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1437
2025-10-09T21:49:10.748647Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1437 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:10.748672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1437 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:10.748675Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1437 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:10.748680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:10.749066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1437 sessionId:  describe result for acl check
2025-10-09T21:49:10.749102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:10.749104Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:10.749106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:10.749111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342372937327503:3152] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:10.749670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342372937327503:3152] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:10.751781Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342372937327503:3152] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:10.755109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342372937327503:3152] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:10.755123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342372937327503:3152] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:10.755127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342372937327503:3152] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:10.755135Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1437 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:10.760759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:10.762979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:10.763158Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:10.763167Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:10.763178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:10.763228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:10.763266Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342372575502649:5954]
2025-10-09T21:49:10.763812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:10.764105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:10.764146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:10.764434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:10.764476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.764482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.764483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.764484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.764485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.764486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.764487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.764489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.764490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.764494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.764504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.764508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.764512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.764516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.764519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.764523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.764527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.764530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.764536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.764543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.764549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.764554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.764558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.764566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.764570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.764575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.764579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.764583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.764587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.764592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.764597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.764603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.764607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.764612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.764616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.764623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.764650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:10.770272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:10.771360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:10.771375Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:10.771377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:10.771381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:10.771387Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1436 [2:7559342372575502649:5954]
2025-10-09T21:49:10.771401Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:10.771416Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:10.771419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:10.771422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:10.771424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:10.771440Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:10.771451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:10.771458Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:10.771511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.771518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.771520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.771523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:10.771592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:10.771600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.771612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.771629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.771651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:10.771674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342372937327490:2432], now have 1 active actors on pipe
2025-10-09T21:49:10.771682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342372937327613:3152], now have 1 active actors on pipe
2025-10-09T21:49:10.771689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342372575502644:5954]
2025-10-09T21:49:10.771692Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342372575502644:5954]
2025-10-09T21:49:10.771694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342372575502644:5954]
2025-10-09T21:49:10.771707Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:10.771724Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:10.771724Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:10.771727Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:10.771729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:10.771741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:10.771743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:10.771730Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1436
2025-10-09T21:49:10.771741Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1436
2025-10-09T21:49:10.771763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342372575502644:5954]
2025-10-09T21:49:10.771771Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:10.771773Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:10.771797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.771805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.771836Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2e98753e-f02d1888-235189f-f1d7efb8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:10.771848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:10.771852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:10.771854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:10.771864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:10.771878Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:10.771896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:10.772034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.772048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.772070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:10.772194Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1437 partition: 0 MaxSeqNo: 25 sessionId: src_id|2e98753e-f02d1888-235189f-f1d7efb8_0
2025-10-09T21:49:10.772554Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e63be97-a83afdf-b1561c59-6335e5c7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2e98753e-f02d1888-235189f-f1d7efb8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:10.774994Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e98753e-f02d1888-235189f-f1d7efb8_0] Write session: try to update token
2025-10-09T21:49:10.775011Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e98753e-f02d1888-235189f-f1d7efb8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:10.785088Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:10.785108Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:10.785111Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:10.785113Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:10.785122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342372575502644:5954]
2025-10-09T21:49:10.785127Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342372575502644:5954]
2025-10-09T21:49:10.785131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342372575502644:5954]
2025-10-09T21:49:10.785175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:10.785179Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:10.785181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.785184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:10.785185Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:10.785189Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:10.785310Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:10.787534Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1437 sessionId: src_id|2e98753e-f02d1888-235189f-f1d7efb8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:10.789879Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:10.841840Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342372937327764:3441]
2025-10-09T21:49:10.842141Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 1436, pipe: [1:7559342372937327764:3441]
2025-10-09T21:49:10.931399Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:10.931962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.931978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.932016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342372937327764:3441], now have 1 active actors on pipe
2025-10-09T21:49:10.932288Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342372937327764:3441]
2025-10-09T21:49:10.932298Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:10.932318Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:10.932321Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:10.932376Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1437 reason: error at write: Disconnected sessionId: src_id|2e98753e-f02d1888-235189f-f1d7efb8_0
2025-10-09T21:49:10.932418Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:10.932573Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1437 sessionId: src_id|2e98753e-f02d1888-235189f-f1d7efb8_0 is DEAD
2025-10-09T21:49:10.932698Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e98753e-f02d1888-235189f-f1d7efb8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:10.932708Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e98753e-f02d1888-235189f-f1d7efb8_0] Write session will restart in 0.008460s
2025-10-09T21:49:10.932728Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e98753e-f02d1888-235189f-f1d7efb8_0] Write session: Do CDS request
2025-10-09T21:49:10.932733Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e98753e-f02d1888-235189f-f1d7efb8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:10.942503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e98753e-f02d1888-235189f-f1d7efb8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:10.942560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:10.942576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1438
2025-10-09T21:49:10.942714Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1438 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:10.942746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1438 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:10.942756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1438 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:10.942760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:10.943271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1438 sessionId:  describe result for acl check
2025-10-09T21:49:10.943313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:10.943315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:10.943316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:10.943322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342372937327823:3145] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:10.943856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342372937327823:3145] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:10.946472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342372937327823:3145] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:10.949016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342372937327823:3145] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:10.949029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342372937327823:3145] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:10.949033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342372937327823:3145] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:10.949041Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1438 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:10.950509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:10.953680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:10.953856Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:10.953865Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:10.953873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:10.953922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:10.953954Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342372575502912:5965]
2025-10-09T21:49:10.954483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:10.954768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:10.954798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:10.954998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:10.955029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.955034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.955034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.955035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.955036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.955036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.955037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.955037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.955038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.955041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.955048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.955050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.955053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.955055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.955059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.955063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.955066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.955070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.955076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.955083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:10.955090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.955094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:10.955098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.955103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:10.955107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.955113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:10.955117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.955122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:10.955126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.955129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:10.955132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.955137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.955142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.955146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.955153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.955159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.955179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:10.961371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:10.962486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:10.962500Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:10.962502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:10.962507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:10.962513Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1437 [2:7559342372575502912:5965]
2025-10-09T21:49:10.962525Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:10.962541Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:10.962544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:10.962547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:10.962549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:10.962566Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:10.962574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:10.962582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:10.962635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.962642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.962644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.962646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:10.962710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:10.962717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.962729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:10.962741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:10.962758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:10.962784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342372937327766:2432], now have 1 active actors on pipe
2025-10-09T21:49:10.962791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342372937328024:3145], now have 1 active actors on pipe
2025-10-09T21:49:10.962798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342372575502907:5965]
2025-10-09T21:49:10.962801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342372575502907:5965]
2025-10-09T21:49:10.962803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342372575502907:5965]
2025-10-09T21:49:10.962815Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:10.962833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:10.962839Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:10.962843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:10.962845Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:10.962852Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:10.962856Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:10.962878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342372575502907:5965]
2025-10-09T21:49:10.962859Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1437
2025-10-09T21:49:10.962886Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:10.962860Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1437
2025-10-09T21:49:10.962887Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:10.962914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.962923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.962955Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|79e93b9d-4c7c090-b7f39668-b6f58596_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:10.962970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:10.962977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:10.962980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:10.962990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:10.963003Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:10.963022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:10.963160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:10.963167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:10.963180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:10.963315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1438 partition: 0 MaxSeqNo: 25 sessionId: src_id|79e93b9d-4c7c090-b7f39668-b6f58596_0
2025-10-09T21:49:10.963667Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e98753e-f02d1888-235189f-f1d7efb8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|79e93b9d-4c7c090-b7f39668-b6f58596_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:10.965988Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|79e93b9d-4c7c090-b7f39668-b6f58596_0] Write session: try to update token
2025-10-09T21:49:10.966003Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|79e93b9d-4c7c090-b7f39668-b6f58596_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:10.976487Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:10.976507Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:10.976509Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:10.976510Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:10.976519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342372575502907:5965]
2025-10-09T21:49:10.976522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342372575502907:5965]
2025-10-09T21:49:10.976524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342372575502907:5965]
2025-10-09T21:49:10.976566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:10.976571Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:10.976574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:10.976577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:10.976578Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:10.976582Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:10.976700Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:10.977985Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1438 sessionId: src_id|79e93b9d-4c7c090-b7f39668-b6f58596_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:10.980242Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:11.032350Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342377232295335:3441]
2025-10-09T21:49:11.032642Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 1437, pipe: [1:7559342377232295335:3441]
2025-10-09T21:49:11.130512Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:11.131082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:11.131101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:11.131149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342377232295335:3441], now have 1 active actors on pipe
2025-10-09T21:49:11.131332Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:11.131384Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342377232295335:3441]
2025-10-09T21:49:11.131386Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:11.131390Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:11.131420Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:11.131441Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1438 reason: error at write: Disconnected sessionId: src_id|79e93b9d-4c7c090-b7f39668-b6f58596_0
2025-10-09T21:49:11.131620Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1438 sessionId: src_id|79e93b9d-4c7c090-b7f39668-b6f58596_0 is DEAD
2025-10-09T21:49:11.131762Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79e93b9d-4c7c090-b7f39668-b6f58596_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:11.131775Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79e93b9d-4c7c090-b7f39668-b6f58596_0] Write session will restart in 0.005531s
2025-10-09T21:49:11.131802Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79e93b9d-4c7c090-b7f39668-b6f58596_0] Write session: Do CDS request
2025-10-09T21:49:11.131819Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79e93b9d-4c7c090-b7f39668-b6f58596_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:11.138487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|79e93b9d-4c7c090-b7f39668-b6f58596_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:11.138562Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:11.138577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1439
2025-10-09T21:49:11.138677Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1439 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:11.138707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1439 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:11.138716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1439 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:11.138719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:11.139178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1439 sessionId:  describe result for acl check
2025-10-09T21:49:11.139228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:11.139232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:11.139234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:11.139241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342377232295353:2476] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:11.139796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342377232295353:2476] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:11.143027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342377232295353:2476] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:11.145818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342377232295353:2476] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:11.145830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342377232295353:2476] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:11.145834Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342377232295353:2476] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:11.145841Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1439 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:11.150446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:11.153029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:11.153238Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:11.153254Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:11.153267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:11.153320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:11.153362Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342376870470473:5963]
2025-10-09T21:49:11.153983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:11.154308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:11.154354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:11.154669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:11.154715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.154721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.154722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.154723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.154724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.154726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.154727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.154728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.154729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.154734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.154746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.154750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.154754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.154757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.154761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.154764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.154767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.154769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.154775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.154780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.154784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.154788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.154790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.154793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.154796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.154799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.154802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.154804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.154807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.154810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.154813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.154816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.154819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.154823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.154828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.154833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.154861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:11.161280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:11.162447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:11.162463Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:11.162465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:11.162470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:11.162477Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1438 [2:7559342376870470473:5963]
2025-10-09T21:49:11.162491Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:11.162509Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:11.162512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:11.162515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:11.162518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:11.162534Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:11.162540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:11.162551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:11.162609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.162616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.162619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.162622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:11.162680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:11.162687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.162700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.162711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.162727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:11.162748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342377232295338:2432], now have 1 active actors on pipe
2025-10-09T21:49:11.162756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342377232295503:2476], now have 1 active actors on pipe
2025-10-09T21:49:11.162763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342376870470468:5963]
2025-10-09T21:49:11.162765Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342376870470468:5963]
2025-10-09T21:49:11.162767Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342376870470468:5963]
2025-10-09T21:49:11.162781Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:11.162800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:11.162804Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:11.162809Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:11.162811Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:11.162814Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:11.162817Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:11.162848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342376870470468:5963]
2025-10-09T21:49:11.162862Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:11.162842Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1438
2025-10-09T21:49:11.162845Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1438
2025-10-09T21:49:11.162891Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:11.162932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:11.162942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:11.162970Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f74a5853-78401c17-15026106-89f42222_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:11.162983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:11.162988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:11.162991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:11.163001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:11.163013Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:11.163028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:11.163173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:11.163185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:11.163200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:11.163287Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1439 partition: 0 MaxSeqNo: 25 sessionId: src_id|f74a5853-78401c17-15026106-89f42222_0
2025-10-09T21:49:11.163622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79e93b9d-4c7c090-b7f39668-b6f58596_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f74a5853-78401c17-15026106-89f42222_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:11.166125Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f74a5853-78401c17-15026106-89f42222_0] Write session: try to update token
2025-10-09T21:49:11.166141Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f74a5853-78401c17-15026106-89f42222_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:11.179581Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1439 sessionId: src_id|f74a5853-78401c17-15026106-89f42222_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:11.181854Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:11.181839Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:11.181857Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:11.181868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342376870470468:5963]
2025-10-09T21:49:11.181917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:11.181919Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:11.182107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:11.182253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:11.182414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:11.182548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:11.182726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:11.182766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:11.182780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:11.182800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:11.182840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:11.182955Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:11.187955Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:11.187972Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:11.187974Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:11.187976Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:11.187985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342376870470468:5963]
2025-10-09T21:49:11.187989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342376870470468:5963]
2025-10-09T21:49:11.187992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342376870470468:5963]
2025-10-09T21:49:11.188028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:11.188035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.188038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:11.188060Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:11.188078Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:11.188083Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:11.188170Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:11.231485Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342377232295615:3441]
2025-10-09T21:49:11.231733Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 1438, pipe: [1:7559342377232295615:3441]
2025-10-09T21:49:11.337376Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:11.344502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:11.344531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:11.344701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:11.344769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:11.344837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:11.344905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:11.345068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:11.345076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:11.345136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:11.345189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:11.345244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:11.345306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:11.345347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:11.345353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:11.345403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:11.345452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:11.345502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:11.345562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:11.345606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:11.345614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:11.345688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:11.345753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:11.345816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:11.345884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:11.345930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:11.345939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:11.346004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:11.346086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:11.346154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:11.346212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:11.346250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:11.346256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:11.346310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:11.346362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:11.346417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:11.346470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:11.346507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:11.346511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:11.346572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:11.346628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:11.346691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:11.346758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:11.346801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:11.346808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:11.346869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:11.346927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:11.346988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:11.347045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:11.347215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:11.347222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:11.347277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:11.347330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:11.347384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:11.347447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:11.347483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:11.347488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:11.347546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:11.347605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:11.347664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:11.347731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:11.347773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:11.347780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:11.347837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342377232295615:3441], now have 1 active actors on pipe
2025-10-09T21:49:11.348064Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342377232295615:3441]
2025-10-09T21:49:11.348087Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:11.348100Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:11.348103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:11.348132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1439 reason: error at write: Disconnected sessionId: src_id|f74a5853-78401c17-15026106-89f42222_0
2025-10-09T21:49:11.348220Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:11.348349Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1439 sessionId: src_id|f74a5853-78401c17-15026106-89f42222_0 is DEAD
2025-10-09T21:49:11.348468Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f74a5853-78401c17-15026106-89f42222_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:11.348476Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f74a5853-78401c17-15026106-89f42222_0] Write session will restart in 0.006207s
2025-10-09T21:49:11.348493Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f74a5853-78401c17-15026106-89f42222_0] Write session: Do CDS request
2025-10-09T21:49:11.348499Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f74a5853-78401c17-15026106-89f42222_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:11.355495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f74a5853-78401c17-15026106-89f42222_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:11.355554Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:11.355572Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1440
2025-10-09T21:49:11.355744Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1440 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:11.355789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1440 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:11.355799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1440 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:11.355803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:11.356360Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1440 sessionId:  describe result for acl check
2025-10-09T21:49:11.356399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:11.356401Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:11.356402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:11.356408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342377232295670:3122] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:11.356951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342377232295670:3122] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:11.360249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342377232295670:3122] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:11.363360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342377232295670:3122] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:11.363373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342377232295670:3122] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:11.363387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342377232295670:3122] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:11.363395Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1440 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:11.366536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:11.369100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:11.369292Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:11.369304Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:11.369315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:11.369364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:11.369401Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342376870470776:5937]
2025-10-09T21:49:11.369957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:11.370303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:11.370353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:11.370658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:11.370725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.370731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.370732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.370734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.370735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.370736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.370737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.370738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.370739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.370744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.370755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.370759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.370763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.370766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.370770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.370778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.370782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.370789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.370796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.370809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.370819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.370829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.370837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.370847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.370851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.370860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.370868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.370877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.370885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.370889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.370893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.370902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.370910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.370915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.370923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.370933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.370965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:11.377085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:11.378462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:11.378476Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:11.378478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:11.378482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:11.378488Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1439 [2:7559342376870470776:5937]
2025-10-09T21:49:11.378501Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:11.378517Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:11.378519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:11.378523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:11.378525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:11.378541Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:11.378553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:11.378560Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:11.378612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.378619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.378621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.378623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:11.378684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:11.378694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.378705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.378717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.378734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:11.378758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342377232295838:3122], now have 1 active actors on pipe
2025-10-09T21:49:11.378766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342377232295647:2432], now have 1 active actors on pipe
2025-10-09T21:49:11.378773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342376870470771:5937]
2025-10-09T21:49:11.378775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342376870470771:5937]
2025-10-09T21:49:11.378777Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342376870470771:5937]
2025-10-09T21:49:11.378793Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:11.378807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:11.378811Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:11.378814Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:11.378816Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:11.378825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:11.378828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:11.378854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342376870470771:5937]
2025-10-09T21:49:11.378865Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:11.378870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:11.378855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1439
2025-10-09T21:49:11.378899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:11.378879Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1439
2025-10-09T21:49:11.378908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:11.378945Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7bd6b6d-3257e030-2bbd1a59-a926f206_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:11.378956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:11.378961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:11.378968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:11.378978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:11.378988Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:11.379006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:11.379146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:11.379154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:11.379166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:11.379261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1440 partition: 0 MaxSeqNo: 25 sessionId: src_id|7bd6b6d-3257e030-2bbd1a59-a926f206_0
2025-10-09T21:49:11.379624Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f74a5853-78401c17-15026106-89f42222_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7bd6b6d-3257e030-2bbd1a59-a926f206_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:11.382094Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7bd6b6d-3257e030-2bbd1a59-a926f206_0] Write session: try to update token
2025-10-09T21:49:11.382109Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7bd6b6d-3257e030-2bbd1a59-a926f206_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:11.393398Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:11.393416Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:11.393418Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:11.393419Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:11.393427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342376870470771:5937]
2025-10-09T21:49:11.393430Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342376870470771:5937]
2025-10-09T21:49:11.393433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342376870470771:5937]
2025-10-09T21:49:11.393474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:11.393487Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:11.393490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.393494Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:11.393494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:11.393497Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:11.393626Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:11.394940Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1440 sessionId: src_id|7bd6b6d-3257e030-2bbd1a59-a926f206_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:11.397172Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:11.448048Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342377232295919:3441]
2025-10-09T21:49:11.448358Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 1439, pipe: [1:7559342377232295919:3441]
2025-10-09T21:49:11.534579Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:11.534970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:11.534988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:11.535030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342377232295919:3441], now have 1 active actors on pipe
2025-10-09T21:49:11.535294Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:11.535297Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342377232295919:3441]
2025-10-09T21:49:11.535314Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:11.535321Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:11.535358Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:11.535375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1440 reason: error at write: Disconnected sessionId: src_id|7bd6b6d-3257e030-2bbd1a59-a926f206_0
2025-10-09T21:49:11.535570Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1440 sessionId: src_id|7bd6b6d-3257e030-2bbd1a59-a926f206_0 is DEAD
2025-10-09T21:49:11.535712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bd6b6d-3257e030-2bbd1a59-a926f206_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:11.535733Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bd6b6d-3257e030-2bbd1a59-a926f206_0] Write session will restart in 0.007804s
2025-10-09T21:49:11.535753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bd6b6d-3257e030-2bbd1a59-a926f206_0] Write session: Do CDS request
2025-10-09T21:49:11.535758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bd6b6d-3257e030-2bbd1a59-a926f206_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:11.544534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7bd6b6d-3257e030-2bbd1a59-a926f206_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:11.544622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:11.544641Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1441
2025-10-09T21:49:11.544790Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1441 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:11.544827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1441 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:11.544831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1441 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:11.544834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:11.545220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1441 sessionId:  describe result for acl check
2025-10-09T21:49:11.545257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:11.545262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:11.545263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:11.545267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342377232295962:3121] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:11.545858Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342377232295962:3121] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:11.548478Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342377232295962:3121] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:11.551159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342377232295962:3121] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:11.551175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342377232295962:3121] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:11.551178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342377232295962:3121] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:11.551186Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1441 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:11.553508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:11.555795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:11.555980Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:11.555989Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:11.556000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:11.556052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:11.556104Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342376870471042:5935]
2025-10-09T21:49:11.556697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:11.556947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:11.556986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:11.557286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:11.557333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.557340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.557342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.557343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.557344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.557345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.557346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.557347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.557348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.557354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.557364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.557376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.557380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.557384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.557388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.557392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.557396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.557399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.557407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.557419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.557425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.557430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.557440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.557444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.557452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.557457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.557462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.557471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.557479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.557484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.557489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.557494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.557503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.557508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.557516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.557525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.557564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:11.563393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:11.564567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:11.564578Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:11.564579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:11.564582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:11.564586Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1440 [2:7559342376870471042:5935]
2025-10-09T21:49:11.564594Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:11.564606Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:11.564608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:11.564610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:11.564611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:11.564623Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:11.564626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:11.564632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:11.564668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.564673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.564675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.564676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:11.564721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:11.564729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.564736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.564742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.564751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:11.564766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342376870471037:5935]
2025-10-09T21:49:11.564767Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342376870471037:5935]
2025-10-09T21:49:11.564768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342376870471037:5935]
2025-10-09T21:49:11.564774Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:11.564782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:11.564783Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:11.564791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:11.564791Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:11.564793Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:11.564793Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:11.564801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342377232296149:3121], now have 1 active actors on pipe
2025-10-09T21:49:11.564804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342377232295921:2432], now have 1 active actors on pipe
2025-10-09T21:49:11.564822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342376870471037:5935]
2025-10-09T21:49:11.564834Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:11.564837Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:11.564864Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1440
2025-10-09T21:49:11.564866Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1440
2025-10-09T21:49:11.564937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:11.564946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:11.564967Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4023ec8c-1ebdd98b-a4a3de0f-d75bb8fc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:11.564976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:11.564979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:11.564981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:11.564987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:11.564995Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:11.565005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:11.565153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:11.565167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:11.565189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:11.565298Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1441 partition: 0 MaxSeqNo: 25 sessionId: src_id|4023ec8c-1ebdd98b-a4a3de0f-d75bb8fc_0
2025-10-09T21:49:11.565629Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bd6b6d-3257e030-2bbd1a59-a926f206_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4023ec8c-1ebdd98b-a4a3de0f-d75bb8fc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:11.567923Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4023ec8c-1ebdd98b-a4a3de0f-d75bb8fc_0] Write session: try to update token
2025-10-09T21:49:11.567939Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4023ec8c-1ebdd98b-a4a3de0f-d75bb8fc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:11.578735Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:11.578753Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:11.578755Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:11.578757Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:11.578767Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342376870471037:5935]
2025-10-09T21:49:11.578772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342376870471037:5935]
2025-10-09T21:49:11.578775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342376870471037:5935]
2025-10-09T21:49:11.578817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:11.578817Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:11.578824Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:11.578824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.578826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:11.578828Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:11.578959Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:11.580115Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1441 sessionId: src_id|4023ec8c-1ebdd98b-a4a3de0f-d75bb8fc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:11.582407Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:11.635568Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342377232296196:3441]
2025-10-09T21:49:11.635779Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 1440, pipe: [1:7559342377232296196:3441]
2025-10-09T21:49:11.719478Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:11.719718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:11.719731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:11.719773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342377232296196:3441], now have 1 active actors on pipe
2025-10-09T21:49:11.719939Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:11.720021Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:11.720042Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342377232296196:3441]
2025-10-09T21:49:11.720067Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:11.720141Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1441 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4023ec8c-1ebdd98b-a4a3de0f-d75bb8fc_0
2025-10-09T21:49:11.720312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1441 sessionId: src_id|4023ec8c-1ebdd98b-a4a3de0f-d75bb8fc_0 is DEAD
2025-10-09T21:49:11.720468Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4023ec8c-1ebdd98b-a4a3de0f-d75bb8fc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:11.720481Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4023ec8c-1ebdd98b-a4a3de0f-d75bb8fc_0] Write session will restart in 0.008533s
2025-10-09T21:49:11.720507Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4023ec8c-1ebdd98b-a4a3de0f-d75bb8fc_0] Write session: Do CDS request
2025-10-09T21:49:11.720514Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4023ec8c-1ebdd98b-a4a3de0f-d75bb8fc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:11.730535Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4023ec8c-1ebdd98b-a4a3de0f-d75bb8fc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:11.730606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:11.730625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1442
2025-10-09T21:49:11.730761Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1442 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:11.730795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1442 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:11.730805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1442 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:11.730809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:11.731329Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1442 sessionId:  describe result for acl check
2025-10-09T21:49:11.731367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:11.731374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:11.731376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:11.731382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342377232296289:3113] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:11.731918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342377232296289:3113] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:11.734423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342377232296289:3113] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:11.736939Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342377232296289:3113] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:11.736953Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342377232296289:3113] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:11.736957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342377232296289:3113] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:11.736963Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1442 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:11.737598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:11.739868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:11.740044Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:11.740054Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:11.740065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:11.740118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:11.740157Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342376870471314:5926]
2025-10-09T21:49:11.740738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:11.741035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:11.741084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:11.741392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:11.741437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.741443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.741444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.741445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.741446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.741447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.741448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.741449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.741450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.741455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.741464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.741468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.741472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.741476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.741480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.741484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.741488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.741492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.741498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.741506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.741512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.741518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.741522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.741527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.741531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.741540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.741544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.741548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.741557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.741563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.741568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.741578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.741582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.741587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.741596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.741601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.741640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:11.747753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:11.748997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:11.749013Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:11.749016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:11.749020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:11.749026Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1441 [2:7559342376870471314:5926]
2025-10-09T21:49:11.749040Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:11.749055Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:11.749058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:11.749062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:11.749064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:11.749081Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:11.749094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:11.749102Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:11.749152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.749158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.749160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.749163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:11.749231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:11.749238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.749249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.749268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.749285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:11.749308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342377232296465:3113], now have 1 active actors on pipe
2025-10-09T21:49:11.749316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342377232296199:2432], now have 1 active actors on pipe
2025-10-09T21:49:11.749323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342376870471309:5926]
2025-10-09T21:49:11.749326Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342376870471309:5926]
2025-10-09T21:49:11.749328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342376870471309:5926]
2025-10-09T21:49:11.749342Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:11.749355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:11.749337Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1441
2025-10-09T21:49:11.749350Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1441
2025-10-09T21:49:11.749370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:11.749377Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:11.749378Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:11.749380Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:11.749382Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:11.749407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342376870471309:5926]
2025-10-09T21:49:11.749421Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:11.749434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:11.749435Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:11.749441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:11.749481Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6dd466f-70e53e40-32a210d2-f36189b0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:11.749493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:11.749496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:11.749499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:11.749508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:11.749517Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:11.749540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:11.749718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:11.749730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:11.749745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:11.749835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1442 partition: 0 MaxSeqNo: 25 sessionId: src_id|6dd466f-70e53e40-32a210d2-f36189b0_0
2025-10-09T21:49:11.750176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4023ec8c-1ebdd98b-a4a3de0f-d75bb8fc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6dd466f-70e53e40-32a210d2-f36189b0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:11.752480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6dd466f-70e53e40-32a210d2-f36189b0_0] Write session: try to update token
2025-10-09T21:49:11.752496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6dd466f-70e53e40-32a210d2-f36189b0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:11.763585Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:11.763606Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:11.763608Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:11.763610Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:11.763620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342376870471309:5926]
2025-10-09T21:49:11.763624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342376870471309:5926]
2025-10-09T21:49:11.763627Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342376870471309:5926]
2025-10-09T21:49:11.763669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:11.763677Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:11.763681Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:11.763681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.763684Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:11.763684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:11.763814Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:11.764990Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1442 sessionId: src_id|6dd466f-70e53e40-32a210d2-f36189b0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:11.767265Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:11.820287Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342377232296478:3441]
2025-10-09T21:49:11.820574Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 1441, pipe: [1:7559342377232296478:3441]
2025-10-09T21:49:11.912506Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:11.912727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:11.912740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:11.912777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342377232296478:3441], now have 1 active actors on pipe
2025-10-09T21:49:11.912941Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:11.912978Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342377232296478:3441]
2025-10-09T21:49:11.913007Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:11.913011Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:11.913092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1442 reason: status is not ok: tablet will be restarted right now sessionId: src_id|6dd466f-70e53e40-32a210d2-f36189b0_0
2025-10-09T21:49:11.913283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1442 sessionId: src_id|6dd466f-70e53e40-32a210d2-f36189b0_0 is DEAD
2025-10-09T21:49:11.913439Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6dd466f-70e53e40-32a210d2-f36189b0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:11.913450Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6dd466f-70e53e40-32a210d2-f36189b0_0] Write session will restart in 0.008689s
2025-10-09T21:49:11.913470Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6dd466f-70e53e40-32a210d2-f36189b0_0] Write session: Do CDS request
2025-10-09T21:49:11.913475Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6dd466f-70e53e40-32a210d2-f36189b0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:11.923500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6dd466f-70e53e40-32a210d2-f36189b0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:11.923567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:11.923582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1443
2025-10-09T21:49:11.923707Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1443 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:11.923731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1443 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:11.923752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1443 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:11.923760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:11.924388Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1443 sessionId:  describe result for acl check
2025-10-09T21:49:11.924423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:11.924430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:11.924431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:11.924437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342377232296568:3110] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:11.924974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342377232296568:3110] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:11.927239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342377232296568:3110] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:11.929866Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342377232296568:3110] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:11.929879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342377232296568:3110] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:11.929881Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342377232296568:3110] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:11.929896Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1443 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:11.930773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:11.932923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:11.933157Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:11.933166Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:11.933178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:11.933225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:11.933257Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342376870471582:5927]
2025-10-09T21:49:11.933683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:11.933902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:11.933942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:11.934174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:11.934212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.934216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.934217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.934217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.934218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.934219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.934219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.934220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.934221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.934224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.934230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.934232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.934234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.934236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.934238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.934240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.934242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.934244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.934249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.934254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:11.934258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.934262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:11.934265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.934268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:11.934270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.934273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:11.934276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.934278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:11.934281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.934284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:11.934287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.934290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.934292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.934295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.934298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.934301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.934316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:11.940627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:11.941900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:11.941915Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:11.941918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:11.941922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:11.941929Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1442 [2:7559342376870471582:5927]
2025-10-09T21:49:11.941944Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:11.941961Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:11.941964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:11.941967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:11.941970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:11.941986Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:11.941998Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:11.942005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:11.942104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.942112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.942115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.942118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:11.942181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:11.942188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.942201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:11.942219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:11.942240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:11.942262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342377232296483:2432], now have 1 active actors on pipe
2025-10-09T21:49:11.942269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342377232296747:3110], now have 1 active actors on pipe
2025-10-09T21:49:11.942274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342376870471577:5927]
2025-10-09T21:49:11.942276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342376870471577:5927]
2025-10-09T21:49:11.942277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342376870471577:5927]
2025-10-09T21:49:11.942288Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:11.942285Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1442
2025-10-09T21:49:11.942301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:11.942300Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1442
2025-10-09T21:49:11.942305Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:11.942308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:11.942310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:11.942313Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:11.942316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:11.942348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:11.942355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:11.942365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342376870471577:5927]
2025-10-09T21:49:11.942373Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:11.942384Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:11.942396Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d7c6b0fd-41ef589a-342d406a-e1c764f2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:11.942402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:11.942405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:11.942406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:11.942413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:11.942420Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:11.942437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:11.942565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:11.942575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:11.942590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:11.942662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1443 partition: 0 MaxSeqNo: 25 sessionId: src_id|d7c6b0fd-41ef589a-342d406a-e1c764f2_0
2025-10-09T21:49:11.942972Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6dd466f-70e53e40-32a210d2-f36189b0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d7c6b0fd-41ef589a-342d406a-e1c764f2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:11.945278Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7c6b0fd-41ef589a-342d406a-e1c764f2_0] Write session: try to update token
2025-10-09T21:49:11.945293Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7c6b0fd-41ef589a-342d406a-e1c764f2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:11.956293Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:11.956313Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:11.956315Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:11.956318Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:11.956328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342376870471577:5927]
2025-10-09T21:49:11.956333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342376870471577:5927]
2025-10-09T21:49:11.956337Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342376870471577:5927]
2025-10-09T21:49:11.956383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:11.956393Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:11.956397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:11.956400Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:11.956401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:11.956404Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:11.956524Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:11.958528Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1443 sessionId: src_id|d7c6b0fd-41ef589a-342d406a-e1c764f2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:11.960865Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:12.013305Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342381527264057:3441]
2025-10-09T21:49:12.013542Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 1442, pipe: [1:7559342381527264057:3441]
2025-10-09T21:49:12.110827Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:12.111226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:12.111240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:12.111280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342381527264057:3441], now have 1 active actors on pipe
2025-10-09T21:49:12.111431Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:12.111479Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:12.111481Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342381527264057:3441]
2025-10-09T21:49:12.111506Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:12.111583Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1443 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d7c6b0fd-41ef589a-342d406a-e1c764f2_0
2025-10-09T21:49:12.111780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1443 sessionId: src_id|d7c6b0fd-41ef589a-342d406a-e1c764f2_0 is DEAD
2025-10-09T21:49:12.111916Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7c6b0fd-41ef589a-342d406a-e1c764f2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:12.111928Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7c6b0fd-41ef589a-342d406a-e1c764f2_0] Write session will restart in 0.009851s
2025-10-09T21:49:12.111967Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7c6b0fd-41ef589a-342d406a-e1c764f2_0] Write session: Do CDS request
2025-10-09T21:49:12.111974Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7c6b0fd-41ef589a-342d406a-e1c764f2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:12.122446Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7c6b0fd-41ef589a-342d406a-e1c764f2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:12.122551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:12.122566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1444
2025-10-09T21:49:12.122666Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1444 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:12.122692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1444 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:12.122696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1444 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:12.122704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:12.123162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1444 sessionId:  describe result for acl check
2025-10-09T21:49:12.123191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:12.123192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:12.123194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:12.123199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342381527264113:3103] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:12.123638Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342381527264113:3103] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:12.126712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342381527264113:3103] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:12.129242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342381527264113:3103] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:12.129258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342381527264113:3103] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:12.129261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342381527264113:3103] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:12.129269Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1444 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:12.131108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:12.133974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:12.134276Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:12.134290Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:12.134306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:12.134371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:12.134407Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342381165439148:5920]
2025-10-09T21:49:12.134901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:12.135186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:12.135228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:12.135530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:12.135582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.135587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.135588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.135589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.135589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.135590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.135591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.135591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.135593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.135596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.135603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.135605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.135608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.135610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.135612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.135614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.135616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.135618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.135624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.135630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.135634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.135637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.135640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.135643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.135646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.135649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.135652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.135655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.135658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.135661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.135664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.135667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.135674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.135677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.135680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.135683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.135703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:12.141783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:12.143024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:12.143042Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:12.143044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:12.143048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:12.143054Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1443 [2:7559342381165439148:5920]
2025-10-09T21:49:12.143067Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:12.143085Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:12.143092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:12.143096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:12.143098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:12.143112Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:12.143122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:12.143128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:12.143169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.143175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.143176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.143178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:12.143232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:12.143234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.143241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.143247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.143261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:12.143280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342381165439142:5920]
2025-10-09T21:49:12.143281Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342381165439142:5920]
2025-10-09T21:49:12.143283Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342381165439142:5920]
2025-10-09T21:49:12.143289Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:12.143300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342381527264312:3103], now have 1 active actors on pipe
2025-10-09T21:49:12.143303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342381527264059:2432], now have 1 active actors on pipe
2025-10-09T21:49:12.143309Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:12.143314Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:12.143317Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:12.143317Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:12.143319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:12.143320Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:12.143340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342381165439142:5920]
2025-10-09T21:49:12.143346Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:12.143348Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:12.143361Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1443
2025-10-09T21:49:12.143371Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1443
2025-10-09T21:49:12.143460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:12.143467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:12.143488Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|84db3bb0-72a0d99d-92b7dfdb-3c4266fd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:12.143498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:12.143501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:12.143503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:12.143510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:12.143518Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:12.143532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:12.143676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:12.143683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:12.143693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:12.143790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1444 partition: 0 MaxSeqNo: 25 sessionId: src_id|84db3bb0-72a0d99d-92b7dfdb-3c4266fd_0
2025-10-09T21:49:12.144110Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7c6b0fd-41ef589a-342d406a-e1c764f2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|84db3bb0-72a0d99d-92b7dfdb-3c4266fd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:12.146512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|84db3bb0-72a0d99d-92b7dfdb-3c4266fd_0] Write session: try to update token
2025-10-09T21:49:12.146527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|84db3bb0-72a0d99d-92b7dfdb-3c4266fd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:12.158770Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1444 sessionId: src_id|84db3bb0-72a0d99d-92b7dfdb-3c4266fd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:12.159700Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:12.159719Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:12.159726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342381165439142:5920]
2025-10-09T21:49:12.159769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:12.159785Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:12.159925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:12.160055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:12.160193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:12.160318Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:12.160488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:12.160523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:12.161008Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:12.161190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:12.161207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:12.161235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:12.161298Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:12.167289Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:12.167302Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:12.167304Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:12.167305Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:12.167314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342381165439142:5920]
2025-10-09T21:49:12.167317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342381165439142:5920]
2025-10-09T21:49:12.167319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342381165439142:5920]
2025-10-09T21:49:12.167358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:12.167359Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:12.167365Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:12.167368Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:12.167370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.167373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:12.167492Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:12.211396Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342381527264341:3441]
2025-10-09T21:49:12.211660Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 1443, pipe: [1:7559342381527264341:3441]
2025-10-09T21:49:12.319090Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:12.325881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:12.325902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:12.326052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:12.326110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:12.326165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:12.326222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:12.326369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:12.326378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:12.326437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:12.326491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:12.326548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:12.326759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:12.326802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:12.326810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:12.326866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:12.326919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:12.326975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:12.327028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:12.327152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:12.327158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:12.327214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:12.327272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:12.327332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:12.327388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:12.327432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:12.327439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:12.327497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:12.327558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:12.327625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:12.327682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:12.327722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:12.327729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:12.327792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:12.327853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:12.327913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:12.327971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:12.328018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:12.328026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:12.328083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:12.328139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:12.328199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:12.328259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:12.328302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:12.328310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:12.328369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:12.328421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:12.328478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:12.328536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:12.328579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:12.328587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:12.328649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:12.328707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:12.328765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:12.328830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:12.328876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:12.328884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:12.328945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:12.329000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:12.329061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:12.329121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:12.329162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:12.329172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:12.329216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342381527264341:3441], now have 1 active actors on pipe
2025-10-09T21:49:12.329346Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342381527264341:3441]
2025-10-09T21:49:12.329379Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:12.329385Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:12.329389Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:12.329425Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1444 reason: error at write: Disconnected sessionId: src_id|84db3bb0-72a0d99d-92b7dfdb-3c4266fd_0
2025-10-09T21:49:12.329585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1444 sessionId: src_id|84db3bb0-72a0d99d-92b7dfdb-3c4266fd_0 is DEAD
2025-10-09T21:49:12.329740Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84db3bb0-72a0d99d-92b7dfdb-3c4266fd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:12.329752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84db3bb0-72a0d99d-92b7dfdb-3c4266fd_0] Write session will restart in 0.006778s
2025-10-09T21:49:12.329761Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:12.329778Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84db3bb0-72a0d99d-92b7dfdb-3c4266fd_0] Write session: Do CDS request
2025-10-09T21:49:12.329784Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84db3bb0-72a0d99d-92b7dfdb-3c4266fd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:12.337515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|84db3bb0-72a0d99d-92b7dfdb-3c4266fd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:12.337635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:12.337653Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1445
2025-10-09T21:49:12.337757Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1445 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:12.337781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1445 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:12.337785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1445 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:12.337787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:12.338201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1445 sessionId:  describe result for acl check
2025-10-09T21:49:12.338231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:12.338237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:12.338238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:12.338243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342381527264399:3079] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:12.338653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342381527264399:3079] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:12.341040Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342381527264399:3079] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:12.344022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342381527264399:3079] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:12.344041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342381527264399:3079] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:12.344045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342381527264399:3079] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:12.344053Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1445 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:12.348501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:12.351022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:12.351220Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:12.351232Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:12.351245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:12.351316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:12.351356Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342381165439456:5905]
2025-10-09T21:49:12.351943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:12.352243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:12.352309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:12.352655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:12.352702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.352708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.352709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.352710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.352711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.352712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.352713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.352714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.352715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.352721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.352731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.352735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.352739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.352744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.352747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.352751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.352754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.352758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.352766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.352774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.352781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.352786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.352797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.352802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.352807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.352812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.352817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.352822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.352826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.352832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.352836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.352841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.352844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.352849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.352854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.352859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.352888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:12.359078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:12.360234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:12.360247Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:12.360250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:12.360254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:12.360260Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1444 [2:7559342381165439456:5905]
2025-10-09T21:49:12.360274Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:12.360292Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:12.360300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:12.360303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:12.360306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:12.360323Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:12.360336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:12.360343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:12.360395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.360403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.360405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.360408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:12.360473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:12.360480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.360491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.360508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.360531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:12.360556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342381527264547:3079], now have 1 active actors on pipe
2025-10-09T21:49:12.360564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342381527264373:2432], now have 1 active actors on pipe
2025-10-09T21:49:12.360571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342381165439451:5905]
2025-10-09T21:49:12.360577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342381165439451:5905]
2025-10-09T21:49:12.360579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342381165439451:5905]
2025-10-09T21:49:12.360591Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:12.360604Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:12.360608Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:12.360610Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:12.360612Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:12.360626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:12.360629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:12.360618Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1444
2025-10-09T21:49:12.360620Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1444
2025-10-09T21:49:12.360659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342381165439451:5905]
2025-10-09T21:49:12.360671Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:12.360676Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:12.360688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:12.360697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:12.360745Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5cd1ff82-60d63c90-e2b3490f-160c9427_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:12.360758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:12.360771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:12.360774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:12.360785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:12.360794Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:12.360811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:12.360941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:12.360948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:12.360961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:12.361062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1445 partition: 0 MaxSeqNo: 25 sessionId: src_id|5cd1ff82-60d63c90-e2b3490f-160c9427_0
2025-10-09T21:49:12.361425Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84db3bb0-72a0d99d-92b7dfdb-3c4266fd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5cd1ff82-60d63c90-e2b3490f-160c9427_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:12.363852Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5cd1ff82-60d63c90-e2b3490f-160c9427_0] Write session: try to update token
2025-10-09T21:49:12.363870Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5cd1ff82-60d63c90-e2b3490f-160c9427_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:12.374549Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:12.374569Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:12.374571Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:12.374573Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:12.374583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342381165439451:5905]
2025-10-09T21:49:12.374589Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342381165439451:5905]
2025-10-09T21:49:12.374592Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342381165439451:5905]
2025-10-09T21:49:12.374630Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:12.374635Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:12.374635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:12.374637Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:12.374642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.374645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:12.374784Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:12.376963Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1445 sessionId: src_id|5cd1ff82-60d63c90-e2b3490f-160c9427_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:12.379335Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:12.429520Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342381527264650:3441]
2025-10-09T21:49:12.429820Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 1444, pipe: [1:7559342381527264650:3441]
2025-10-09T21:49:12.524197Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:12.524640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:12.524654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:12.524692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342381527264650:3441], now have 1 active actors on pipe
2025-10-09T21:49:12.524858Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:12.524985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:12.525001Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:12.525064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1445 reason: error at write: Disconnected sessionId: src_id|5cd1ff82-60d63c90-e2b3490f-160c9427_0
2025-10-09T21:49:12.525086Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:12.525135Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342381527264650:3441]
2025-10-09T21:49:12.525233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1445 sessionId: src_id|5cd1ff82-60d63c90-e2b3490f-160c9427_0 is DEAD
2025-10-09T21:49:12.525374Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5cd1ff82-60d63c90-e2b3490f-160c9427_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:12.525385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5cd1ff82-60d63c90-e2b3490f-160c9427_0] Write session will restart in 0.006744s
2025-10-09T21:49:12.525414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5cd1ff82-60d63c90-e2b3490f-160c9427_0] Write session: Do CDS request
2025-10-09T21:49:12.525420Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5cd1ff82-60d63c90-e2b3490f-160c9427_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:12.533478Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5cd1ff82-60d63c90-e2b3490f-160c9427_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:12.533546Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:12.533560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1446
2025-10-09T21:49:12.533712Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1446 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:12.533746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1446 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:12.533755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1446 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:12.533759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:12.534220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1446 sessionId:  describe result for acl check
2025-10-09T21:49:12.534254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:12.534260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:12.534262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:12.534268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342381527264678:3080] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:12.534801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342381527264678:3080] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:12.537858Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342381527264678:3080] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:12.540464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342381527264678:3080] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:12.540476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342381527264678:3080] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:12.540480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342381527264678:3080] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:12.540487Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1446 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:12.543640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:12.546081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:12.546251Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:12.546259Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:12.546268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:12.546309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:12.546339Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342381165439727:5900]
2025-10-09T21:49:12.546806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:12.547097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:12.547128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:12.547331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:12.547361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.547366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.547366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.547367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.547368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.547368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.547369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.547370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.547370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.547374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.547388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.547390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.547392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.547395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.547397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.547399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.547401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.547402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.547407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.547412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.547416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.547420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.547422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.547425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.547428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.547431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.547434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.547436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.547439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.547442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.547445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.547448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.547450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.547454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.547456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.547459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.547477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:12.553495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:12.554819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:12.554830Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:12.554831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:12.554835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:12.554840Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1445 [2:7559342381165439727:5900]
2025-10-09T21:49:12.554851Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:12.554864Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:12.554866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:12.554869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:12.554870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:12.554882Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:12.554886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:12.554892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:12.554942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.554947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.554948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.554950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:12.555001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:12.555006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.555013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.555020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.555033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:12.555047Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342381165439722:5900]
2025-10-09T21:49:12.555051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342381165439722:5900]
2025-10-09T21:49:12.555053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342381165439722:5900]
2025-10-09T21:49:12.555059Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:12.555070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:12.555071Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:12.555074Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:12.555076Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:12.555082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:12.555084Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:12.555092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342381527264653:2432], now have 1 active actors on pipe
2025-10-09T21:49:12.555098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342381527264853:3080], now have 1 active actors on pipe
2025-10-09T21:49:12.555115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342381165439722:5900]
2025-10-09T21:49:12.555124Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:12.555144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:12.555150Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1445
2025-10-09T21:49:12.555161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1445
2025-10-09T21:49:12.555211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:12.555217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:12.555239Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|62b24d67-90940125-7ba2785d-3219e5dd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:12.555249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:12.555255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:12.555257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:12.555263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:12.555274Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:12.555285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:12.555427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:12.555432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:12.555442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:12.555545Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1446 partition: 0 MaxSeqNo: 25 sessionId: src_id|62b24d67-90940125-7ba2785d-3219e5dd_0
2025-10-09T21:49:12.555932Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5cd1ff82-60d63c90-e2b3490f-160c9427_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|62b24d67-90940125-7ba2785d-3219e5dd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:12.558316Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62b24d67-90940125-7ba2785d-3219e5dd_0] Write session: try to update token
2025-10-09T21:49:12.558334Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62b24d67-90940125-7ba2785d-3219e5dd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:12.568819Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:12.568836Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:12.568839Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:12.568841Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:12.568852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342381165439722:5900]
2025-10-09T21:49:12.568856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342381165439722:5900]
2025-10-09T21:49:12.568859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342381165439722:5900]
2025-10-09T21:49:12.568899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:12.568900Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:12.568906Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:12.568906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.568908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:12.568910Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:12.569044Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:12.570689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1446 sessionId: src_id|62b24d67-90940125-7ba2785d-3219e5dd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:12.572926Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:12.625159Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342381527264933:3441]
2025-10-09T21:49:12.625416Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 1445, pipe: [1:7559342381527264933:3441]
2025-10-09T21:49:12.717344Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:12.717786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:12.717801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:12.717846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342381527264933:3441], now have 1 active actors on pipe
2025-10-09T21:49:12.718009Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:12.718107Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342381527264933:3441]
2025-10-09T21:49:12.718114Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:12.718150Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:12.718225Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1446 reason: status is not ok: tablet will be restarted right now sessionId: src_id|62b24d67-90940125-7ba2785d-3219e5dd_0
2025-10-09T21:49:12.718377Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1446 sessionId: src_id|62b24d67-90940125-7ba2785d-3219e5dd_0 is DEAD
2025-10-09T21:49:12.718542Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62b24d67-90940125-7ba2785d-3219e5dd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:12.718554Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62b24d67-90940125-7ba2785d-3219e5dd_0] Write session will restart in 0.007013s
2025-10-09T21:49:12.718582Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62b24d67-90940125-7ba2785d-3219e5dd_0] Write session: Do CDS request
2025-10-09T21:49:12.718589Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62b24d67-90940125-7ba2785d-3219e5dd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:12.726424Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62b24d67-90940125-7ba2785d-3219e5dd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:12.726442Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:12.726458Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1447
2025-10-09T21:49:12.726621Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1447 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:12.726656Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1447 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:12.726666Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1447 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:12.726670Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:12.727277Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1447 sessionId:  describe result for acl check
2025-10-09T21:49:12.727311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:12.727318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:12.727322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:12.727329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342381527264961:3074] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:12.727797Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342381527264961:3074] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:12.730873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342381527264961:3074] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:12.734024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342381527264961:3074] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:12.734052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342381527264961:3074] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:12.734057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342381527264961:3074] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:12.734065Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1447 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:12.736834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:12.739281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:12.739454Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:12.739463Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:12.739474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:12.739525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:12.739561Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342381165440003:5897]
2025-10-09T21:49:12.740138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:12.740447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:12.740491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:12.740803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:12.740852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.740858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.740859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.740860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.740861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.740863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.740864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.740865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.740867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.740872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.740882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.740887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.740892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.740895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.740899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.740903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.740907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.740912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.740918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.740927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.740937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.740942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.740946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.740952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.740956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.740962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.740967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.740973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.740978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.740988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.740993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.741002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.741007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.741013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.741019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.741028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.741059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:12.747443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:12.748758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:12.748771Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:12.748774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:12.748778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:12.748785Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1446 [2:7559342381165440003:5897]
2025-10-09T21:49:12.748802Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:12.748825Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:12.748835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:12.748839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:12.748841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:12.748858Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:12.748864Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:12.748871Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:12.748925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.748932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.748934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.748937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:12.749002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:12.749009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.749020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.749034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.749055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:12.749083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342381527265152:3074], now have 1 active actors on pipe
2025-10-09T21:49:12.749086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342381527264936:2432], now have 1 active actors on pipe
2025-10-09T21:49:12.749093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342381165439998:5897]
2025-10-09T21:49:12.749096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342381165439998:5897]
2025-10-09T21:49:12.749098Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342381165439998:5897]
2025-10-09T21:49:12.749113Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:12.749127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:12.749133Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:12.749136Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:12.749138Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:12.749143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:12.749146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:12.749176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342381165439998:5897]
2025-10-09T21:49:12.749184Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:12.749198Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:12.749171Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1446
2025-10-09T21:49:12.749180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1446
2025-10-09T21:49:12.749220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:12.749231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:12.749267Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6b93f2fb-4fa60bf0-84c95fbb-6dc9a5b3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:12.749280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:12.749289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:12.749292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:12.749303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:12.749318Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:12.749331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:12.749520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:12.749530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:12.749547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:12.749667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1447 partition: 0 MaxSeqNo: 25 sessionId: src_id|6b93f2fb-4fa60bf0-84c95fbb-6dc9a5b3_0
2025-10-09T21:49:12.750025Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62b24d67-90940125-7ba2785d-3219e5dd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6b93f2fb-4fa60bf0-84c95fbb-6dc9a5b3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:12.752474Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b93f2fb-4fa60bf0-84c95fbb-6dc9a5b3_0] Write session: try to update token
2025-10-09T21:49:12.752495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b93f2fb-4fa60bf0-84c95fbb-6dc9a5b3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:12.763166Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:12.763181Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:12.763184Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:12.763186Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:12.763196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342381165439998:5897]
2025-10-09T21:49:12.763201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342381165439998:5897]
2025-10-09T21:49:12.763205Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342381165439998:5897]
2025-10-09T21:49:12.763249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:12.763254Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:12.763256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.763259Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:12.763259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:12.763265Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:12.763393Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:12.764687Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1447 sessionId: src_id|6b93f2fb-4fa60bf0-84c95fbb-6dc9a5b3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:12.767263Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:12.818163Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342381527265217:3441]
2025-10-09T21:49:12.818480Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 1446, pipe: [1:7559342381527265217:3441]
2025-10-09T21:49:12.916484Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:12.916926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:12.916940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:12.916990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342381527265217:3441], now have 1 active actors on pipe
2025-10-09T21:49:12.917187Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:12.917213Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342381527265217:3441]
2025-10-09T21:49:12.917215Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:12.917219Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:12.917248Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:12.917277Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1447 reason: error at write: Disconnected sessionId: src_id|6b93f2fb-4fa60bf0-84c95fbb-6dc9a5b3_0
2025-10-09T21:49:12.917432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1447 sessionId: src_id|6b93f2fb-4fa60bf0-84c95fbb-6dc9a5b3_0 is DEAD
2025-10-09T21:49:12.917586Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b93f2fb-4fa60bf0-84c95fbb-6dc9a5b3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:12.917598Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b93f2fb-4fa60bf0-84c95fbb-6dc9a5b3_0] Write session will restart in 0.007338s
2025-10-09T21:49:12.917635Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b93f2fb-4fa60bf0-84c95fbb-6dc9a5b3_0] Write session: Do CDS request
2025-10-09T21:49:12.917642Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b93f2fb-4fa60bf0-84c95fbb-6dc9a5b3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:12.926523Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b93f2fb-4fa60bf0-84c95fbb-6dc9a5b3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:12.926606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:12.926622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1448
2025-10-09T21:49:12.926767Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1448 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:12.926800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1448 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:12.926811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1448 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:12.926814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:12.927248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1448 sessionId:  describe result for acl check
2025-10-09T21:49:12.927281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:12.927282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:12.927284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:12.927290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342381527265258:3066] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:12.927834Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342381527265258:3066] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:12.930876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342381527265258:3066] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:12.933528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342381527265258:3066] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:12.933540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342381527265258:3066] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:12.933543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342381527265258:3066] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:12.933550Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1448 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:12.935902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:12.938229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:12.938409Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:12.938419Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:12.938432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:12.938481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:12.938523Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342381165440276:5895]
2025-10-09T21:49:12.939102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:12.939394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:12.939435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:12.939766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:12.939813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.939818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.939820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.939821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.939822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.939824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.939825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.939826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.939827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.939832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.939843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.939848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.939852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.939856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.939861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.939865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.939869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.939873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.939880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.939889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:12.939899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.939905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:12.939909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.939914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:12.939919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.939925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:12.939931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.939942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:12.939946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.939952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:12.939963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.939969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.939977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.939983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.939988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.939998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.940028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:12.946267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:12.947482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:12.947492Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:12.947494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:12.947498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:12.947504Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1447 [2:7559342381165440276:5895]
2025-10-09T21:49:12.947519Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:12.947536Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:12.947541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:12.947544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:12.947547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:12.947563Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:12.947574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:12.947584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:12.947637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.947645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.947647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.947650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:12.947714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:12.947721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.947732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:12.947744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:12.947765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:12.947789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342381527265222:2432], now have 1 active actors on pipe
2025-10-09T21:49:12.947796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342381527265459:3066], now have 1 active actors on pipe
2025-10-09T21:49:12.947803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342381165440271:5895]
2025-10-09T21:49:12.947807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342381165440271:5895]
2025-10-09T21:49:12.947809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342381165440271:5895]
2025-10-09T21:49:12.947822Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:12.947836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:12.947846Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:12.947847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:12.947849Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:12.947849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:12.947851Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:12.947869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342381165440271:5895]
2025-10-09T21:49:12.947844Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1447
2025-10-09T21:49:12.947872Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:12.947863Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1447
2025-10-09T21:49:12.947875Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:12.947900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:12.947909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:12.948134Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cc46e0bf-adae999e-c315013e-dd0235dc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:12.948148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:12.948152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:12.948155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:12.948166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:12.948177Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:12.948204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:12.948391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:12.948398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:12.948414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:12.948494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1448 partition: 0 MaxSeqNo: 25 sessionId: src_id|cc46e0bf-adae999e-c315013e-dd0235dc_0
2025-10-09T21:49:12.948823Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b93f2fb-4fa60bf0-84c95fbb-6dc9a5b3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cc46e0bf-adae999e-c315013e-dd0235dc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:12.951319Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc46e0bf-adae999e-c315013e-dd0235dc_0] Write session: try to update token
2025-10-09T21:49:12.951339Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc46e0bf-adae999e-c315013e-dd0235dc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:12.961783Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:12.961801Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:12.961804Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:12.961806Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:12.961816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342381165440271:5895]
2025-10-09T21:49:12.961821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342381165440271:5895]
2025-10-09T21:49:12.961825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342381165440271:5895]
2025-10-09T21:49:12.961871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:12.961878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:12.961881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:12.961880Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:12.961886Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:12.961893Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:12.962018Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:12.964642Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1448 sessionId: src_id|cc46e0bf-adae999e-c315013e-dd0235dc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:12.966869Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:13.017347Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342385822232801:3441]
2025-10-09T21:49:13.017573Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 1447, pipe: [1:7559342385822232801:3441]
2025-10-09T21:49:13.120164Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:13.120630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.120646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.120694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342385822232801:3441], now have 1 active actors on pipe
2025-10-09T21:49:13.120902Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:13.120987Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342385822232801:3441]
2025-10-09T21:49:13.120988Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:13.121025Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:13.121027Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:13.121088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1448 reason: error at write: Disconnected sessionId: src_id|cc46e0bf-adae999e-c315013e-dd0235dc_0
2025-10-09T21:49:13.121264Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1448 sessionId: src_id|cc46e0bf-adae999e-c315013e-dd0235dc_0 is DEAD
2025-10-09T21:49:13.121412Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc46e0bf-adae999e-c315013e-dd0235dc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:13.121423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc46e0bf-adae999e-c315013e-dd0235dc_0] Write session will restart in 0.008943s
2025-10-09T21:49:13.121446Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc46e0bf-adae999e-c315013e-dd0235dc_0] Write session: Do CDS request
2025-10-09T21:49:13.121451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc46e0bf-adae999e-c315013e-dd0235dc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:13.131498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc46e0bf-adae999e-c315013e-dd0235dc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:13.131618Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:13.131635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1449
2025-10-09T21:49:13.131775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1449 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:13.131808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1449 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:13.131818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1449 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:13.131823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:13.132202Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1449 sessionId:  describe result for acl check
2025-10-09T21:49:13.132243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:13.132249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:13.132251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:13.132257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342385822232878:3062] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:13.132729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342385822232878:3062] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:13.135358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342385822232878:3062] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:13.138443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342385822232878:3062] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:13.138455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342385822232878:3062] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:13.138457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342385822232878:3062] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:13.138463Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1449 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:13.140191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:13.142651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:13.142855Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:13.142865Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:13.142877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:13.142929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:13.142969Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342385460407847:5893]
2025-10-09T21:49:13.143598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:13.143844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:13.143886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:13.144235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:13.144288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.144294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.144295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.144296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.144297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.144298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.144299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.144300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.144302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.144306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.144316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.144320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.144323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.144327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.144330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.144333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.144336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.144340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.144347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.144355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.144361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.144368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.144374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.144378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.144386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.144390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.144399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.144403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.144411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.144416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.144425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.144430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.144437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.144442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.144450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.144454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.144480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:13.150614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:13.152024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:13.152041Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:13.152044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:13.152048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:13.152055Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1448 [2:7559342385460407847:5893]
2025-10-09T21:49:13.152069Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:13.152087Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:13.152095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:13.152098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:13.152101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:13.152117Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:13.152127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:13.152141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:13.152197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.152208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.152212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.152215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:13.152388Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1448
2025-10-09T21:49:13.152430Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1448
2025-10-09T21:49:13.152430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:13.152439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.152452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.152472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.152503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:13.152530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342385822232805:2432], now have 1 active actors on pipe
2025-10-09T21:49:13.152538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342385822233074:3062], now have 1 active actors on pipe
2025-10-09T21:49:13.152558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.152567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.152586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342385460407842:5893]
2025-10-09T21:49:13.152595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342385460407842:5893]
2025-10-09T21:49:13.152597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342385460407842:5893]
2025-10-09T21:49:13.152611Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:13.152626Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:13.152627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:13.152634Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:13.152637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:13.152643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:13.152646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:13.152668Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9d970040-afb60d68-90f06ab9-2ed4a6f6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:13.152682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:13.152686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:13.152689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:13.152699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:13.152714Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:13.152750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342385460407842:5893]
2025-10-09T21:49:13.152762Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:13.152769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:13.152778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:13.152923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.152933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.152948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:13.153059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1449 partition: 0 MaxSeqNo: 25 sessionId: src_id|9d970040-afb60d68-90f06ab9-2ed4a6f6_0
2025-10-09T21:49:13.153398Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc46e0bf-adae999e-c315013e-dd0235dc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9d970040-afb60d68-90f06ab9-2ed4a6f6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:13.156121Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d970040-afb60d68-90f06ab9-2ed4a6f6_0] Write session: try to update token
2025-10-09T21:49:13.156141Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d970040-afb60d68-90f06ab9-2ed4a6f6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:13.164138Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:13.164150Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:13.164161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342385460407842:5893]
2025-10-09T21:49:13.164213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:13.164218Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:13.164398Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:13.164548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:13.164690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:13.164826Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:13.165007Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:13.165053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:13.169533Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1449 sessionId: src_id|9d970040-afb60d68-90f06ab9-2ed4a6f6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:13.171888Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:13.172082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.172102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.172141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:13.172234Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:13.176815Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:13.176830Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:13.176832Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:13.176833Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:13.176843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342385460407842:5893]
2025-10-09T21:49:13.176847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342385460407842:5893]
2025-10-09T21:49:13.176851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342385460407842:5893]
2025-10-09T21:49:13.176888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:13.176898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.176898Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:13.176901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:13.176903Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:13.176905Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:13.177030Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:13.221081Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342385822233092:3441]
2025-10-09T21:49:13.221328Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 1448, pipe: [1:7559342385822233092:3441]
2025-10-09T21:49:13.321517Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:13.328532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.328553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.328689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:13.328747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:13.328812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:13.328879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:13.329057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:13.329070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:13.329137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:13.329202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:13.329262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:13.329324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:13.329368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:13.329377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:13.329438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:13.329501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:13.329556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:13.329609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:13.329657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:13.329664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:13.329724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:13.330009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:13.330087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:13.330140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:13.330177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:13.330182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:13.330241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:13.330294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:13.330357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:13.330418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:13.330462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:13.330468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:13.330521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:13.330573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:13.330626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:13.330681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:13.330726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:13.330734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:13.330790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:13.330841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:13.330894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:13.330946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:13.330984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:13.330990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:13.331041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:13.331093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:13.331141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:13.331196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:13.331232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:13.331237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:13.331286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:13.331336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:13.331385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:13.331437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:13.331473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:13.331478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:13.331530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:13.331579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:13.331630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:13.331683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:13.331736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:13.331741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:13.331782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342385822233092:3441], now have 1 active actors on pipe
2025-10-09T21:49:13.331993Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:13.332035Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342385822233092:3441]
2025-10-09T21:49:13.332057Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:13.332065Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:13.332093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1449 reason: error at write: Disconnected sessionId: src_id|9d970040-afb60d68-90f06ab9-2ed4a6f6_0
2025-10-09T21:49:13.332131Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:13.332248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1449 sessionId: src_id|9d970040-afb60d68-90f06ab9-2ed4a6f6_0 is DEAD
2025-10-09T21:49:13.332401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d970040-afb60d68-90f06ab9-2ed4a6f6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:13.332410Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d970040-afb60d68-90f06ab9-2ed4a6f6_0] Write session will restart in 0.008374s
2025-10-09T21:49:13.332428Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d970040-afb60d68-90f06ab9-2ed4a6f6_0] Write session: Do CDS request
2025-10-09T21:49:13.332432Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d970040-afb60d68-90f06ab9-2ed4a6f6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:13.341479Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d970040-afb60d68-90f06ab9-2ed4a6f6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:13.341585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:13.341599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1450
2025-10-09T21:49:13.341777Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1450 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:13.341801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1450 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:13.341807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1450 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:13.341811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:13.342148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1450 sessionId:  describe result for acl check
2025-10-09T21:49:13.342172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:13.342179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:13.342180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:13.342183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342385822233160:3039] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:13.342628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342385822233160:3039] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:13.345296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342385822233160:3039] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:13.348096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342385822233160:3039] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:13.348110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342385822233160:3039] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:13.348113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342385822233160:3039] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:13.348120Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1450 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:13.351127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:13.353884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:13.354108Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:13.354118Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:13.354130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:13.354184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:13.354220Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342385460408152:5864]
2025-10-09T21:49:13.354789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:13.355130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:13.355175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:13.355478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:13.355521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.355526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.355528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.355529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.355530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.355531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.355532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.355533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.355534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.355539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.355549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.355553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.355557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.355561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.355565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.355569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.355572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.355575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.355582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.355589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.355594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.355600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.355604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.355609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.355612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.355618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.355627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.355632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.355635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.355640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.355652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.355657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.355665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.355670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.355675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.355684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.355711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:13.361979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:13.363352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:13.363366Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:13.363368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:13.363372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:13.363378Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1449 [2:7559342385460408152:5864]
2025-10-09T21:49:13.363391Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:13.363407Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:13.363410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:13.363413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:13.363415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:13.363432Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:13.363443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:13.363450Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:13.363506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.363512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.363514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.363516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:13.363581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:13.363589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.363600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.363611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.363634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:13.363662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342385822233103:2432], now have 1 active actors on pipe
2025-10-09T21:49:13.363670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342385822233324:3039], now have 1 active actors on pipe
2025-10-09T21:49:13.363677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342385460408134:5864]
2025-10-09T21:49:13.363684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342385460408134:5864]
2025-10-09T21:49:13.363686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342385460408134:5864]
2025-10-09T21:49:13.363699Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:13.363717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:13.363723Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:13.363728Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:13.363730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:13.363737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:13.363740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:13.363687Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1449
2025-10-09T21:49:13.363691Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1449
2025-10-09T21:49:13.363782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.363791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.363807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342385460408134:5864]
2025-10-09T21:49:13.363818Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:13.363839Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:13.363853Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1af1af7e-43ab97a4-e3ba9b57-9b0a88c3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:13.363865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:13.363868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:13.363871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:13.363880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:13.363894Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:13.363921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:13.364077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.364085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.364097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:13.364198Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1450 partition: 0 MaxSeqNo: 25 sessionId: src_id|1af1af7e-43ab97a4-e3ba9b57-9b0a88c3_0
2025-10-09T21:49:13.364556Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d970040-afb60d68-90f06ab9-2ed4a6f6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1af1af7e-43ab97a4-e3ba9b57-9b0a88c3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:13.366989Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1af1af7e-43ab97a4-e3ba9b57-9b0a88c3_0] Write session: try to update token
2025-10-09T21:49:13.367006Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1af1af7e-43ab97a4-e3ba9b57-9b0a88c3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:13.377945Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:13.377961Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:13.377964Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:13.377966Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:13.377976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342385460408134:5864]
2025-10-09T21:49:13.377980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342385460408134:5864]
2025-10-09T21:49:13.377983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342385460408134:5864]
2025-10-09T21:49:13.378045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:13.378047Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:13.378055Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:13.378058Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:13.378059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.378062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:13.378190Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:13.379347Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1450 sessionId: src_id|1af1af7e-43ab97a4-e3ba9b57-9b0a88c3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:13.381609Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:13.432081Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342385822233405:3441]
2025-10-09T21:49:13.432356Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 1449, pipe: [1:7559342385822233405:3441]
2025-10-09T21:49:13.533006Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:13.533594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.533624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.533671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342385822233405:3441], now have 1 active actors on pipe
2025-10-09T21:49:13.533889Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:13.533867Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342385822233405:3441]
2025-10-09T21:49:13.533890Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:13.533894Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:13.533897Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:13.533962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1450 reason: error at write: Disconnected sessionId: src_id|1af1af7e-43ab97a4-e3ba9b57-9b0a88c3_0
2025-10-09T21:49:13.534181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1450 sessionId: src_id|1af1af7e-43ab97a4-e3ba9b57-9b0a88c3_0 is DEAD
2025-10-09T21:49:13.534385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1af1af7e-43ab97a4-e3ba9b57-9b0a88c3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:13.534395Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1af1af7e-43ab97a4-e3ba9b57-9b0a88c3_0] Write session will restart in 0.005567s
2025-10-09T21:49:13.534435Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1af1af7e-43ab97a4-e3ba9b57-9b0a88c3_0] Write session: Do CDS request
2025-10-09T21:49:13.534441Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1af1af7e-43ab97a4-e3ba9b57-9b0a88c3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:13.541469Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1af1af7e-43ab97a4-e3ba9b57-9b0a88c3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:13.541632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:13.541661Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1451
2025-10-09T21:49:13.541775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1451 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:13.541810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1451 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:13.541816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1451 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:13.541822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:13.542281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1451 sessionId:  describe result for acl check
2025-10-09T21:49:13.542315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:13.542321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:13.542326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:13.542332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342385822233429:3034] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:13.542890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342385822233429:3034] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:13.546078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342385822233429:3034] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:13.548663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342385822233429:3034] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:13.548678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342385822233429:3034] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:13.548682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342385822233429:3034] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:13.548689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1451 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:13.553300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:13.555940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:13.556208Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:13.556218Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:13.556229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:13.556279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:13.556328Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342385460408436:5851]
2025-10-09T21:49:13.556908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:13.557172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:13.557210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:13.557512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:13.557557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.557562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.557563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.557564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.557566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.557567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.557568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.557569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.557570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.557575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.557586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.557590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.557595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.557599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.557603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.557607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.557622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.557626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.557632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.557641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.557651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.557661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.557669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.557679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.557683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.557688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.557693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.557702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.557712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.557721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.557730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.557740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.557749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.557754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.557759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.557764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.557793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:13.563889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:13.565166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:13.565179Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:13.565181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:13.565184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:13.565190Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1450 [2:7559342385460408436:5851]
2025-10-09T21:49:13.565203Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:13.565221Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:13.565224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:13.565227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:13.565229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:13.565244Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:13.565249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:13.565258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:13.565305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.565313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.565315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.565317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:13.565380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:13.565388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.565398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.565407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.565427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:13.565446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342385822233581:3034], now have 1 active actors on pipe
2025-10-09T21:49:13.565454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342385822233408:2432], now have 1 active actors on pipe
2025-10-09T21:49:13.565462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342385460408431:5851]
2025-10-09T21:49:13.565468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342385460408431:5851]
2025-10-09T21:49:13.565470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342385460408431:5851]
2025-10-09T21:49:13.565486Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:13.565504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:13.565505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:13.565508Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:13.565491Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1450
2025-10-09T21:49:13.565509Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:13.565502Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1450
2025-10-09T21:49:13.565523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:13.565526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:13.565547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342385460408431:5851]
2025-10-09T21:49:13.565554Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:13.565555Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:13.565571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.565579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.565642Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|456ac5fc-ade4167f-387503ff-f01a42dd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:13.565658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:13.565662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:13.565665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:13.565675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:13.565684Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:13.565700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:13.565864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.565872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.565886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:13.565970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1451 partition: 0 MaxSeqNo: 25 sessionId: src_id|456ac5fc-ade4167f-387503ff-f01a42dd_0
2025-10-09T21:49:13.566312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1af1af7e-43ab97a4-e3ba9b57-9b0a88c3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|456ac5fc-ade4167f-387503ff-f01a42dd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:13.568687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|456ac5fc-ade4167f-387503ff-f01a42dd_0] Write session: try to update token
2025-10-09T21:49:13.568705Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|456ac5fc-ade4167f-387503ff-f01a42dd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:13.579262Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:13.579280Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:13.579282Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:13.579284Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:13.579294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342385460408431:5851]
2025-10-09T21:49:13.579299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342385460408431:5851]
2025-10-09T21:49:13.579304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342385460408431:5851]
2025-10-09T21:49:13.579339Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:13.579343Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:13.579347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:13.579348Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:13.579354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.579357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:13.579484Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:13.581158Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1451 sessionId: src_id|456ac5fc-ade4167f-387503ff-f01a42dd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:13.583584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:13.633795Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342385822233697:3441]
2025-10-09T21:49:13.634060Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 1450, pipe: [1:7559342385822233697:3441]
2025-10-09T21:49:13.732618Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:13.733090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.733104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.733152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342385822233697:3441], now have 1 active actors on pipe
2025-10-09T21:49:13.733347Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:13.733367Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:13.733373Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342385822233697:3441]
2025-10-09T21:49:13.733392Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:13.733395Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:13.733467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1451 reason: error at write: Disconnected sessionId: src_id|456ac5fc-ade4167f-387503ff-f01a42dd_0
2025-10-09T21:49:13.733657Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1451 sessionId: src_id|456ac5fc-ade4167f-387503ff-f01a42dd_0 is DEAD
2025-10-09T21:49:13.733820Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|456ac5fc-ade4167f-387503ff-f01a42dd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:13.733830Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|456ac5fc-ade4167f-387503ff-f01a42dd_0] Write session will restart in 0.007643s
2025-10-09T21:49:13.733859Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|456ac5fc-ade4167f-387503ff-f01a42dd_0] Write session: Do CDS request
2025-10-09T21:49:13.733865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|456ac5fc-ade4167f-387503ff-f01a42dd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:13.742516Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|456ac5fc-ade4167f-387503ff-f01a42dd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:13.742624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:13.742647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1452
2025-10-09T21:49:13.742801Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1452 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:13.742846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1452 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:13.742859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1452 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:13.742862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:13.743516Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1452 sessionId:  describe result for acl check
2025-10-09T21:49:13.743552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:13.743559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:13.743560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:13.743565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342385822233751:3028] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:13.744007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342385822233751:3028] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:13.747599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342385822233751:3028] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:13.750346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342385822233751:3028] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:13.750359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342385822233751:3028] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:13.750363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342385822233751:3028] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:13.750370Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1452 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:13.754309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:13.754664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:13.754839Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:13.754847Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:13.754860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:13.754907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:13.754944Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342385460408717:5847]
2025-10-09T21:49:13.755494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:13.755712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:13.755740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:13.755962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:13.755998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.756003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.756004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.756004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.756005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.756006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.756006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.756007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.756008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.756011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.756022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.756026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.756030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.756033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.756037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.756041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.756045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.756049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.756055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.756060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.756064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.756067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.756071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.756074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.756077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.756080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.756083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.756085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.756088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.756091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.756094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.756097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.756099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.756102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.756104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.756107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.756123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:13.762433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:13.763885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:13.763898Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:13.763900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:13.763905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:13.763912Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1451 [2:7559342385460408717:5847]
2025-10-09T21:49:13.763925Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:13.763942Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:13.763949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:13.763953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:13.763956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:13.763974Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:13.763985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:13.763996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:13.764051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.764057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.764059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.764062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:13.764125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:13.764132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.764144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.764161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.764178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:13.764205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342385822233700:2432], now have 1 active actors on pipe
2025-10-09T21:49:13.764208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342385822233973:3028], now have 1 active actors on pipe
2025-10-09T21:49:13.764215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342385460408712:5847]
2025-10-09T21:49:13.764218Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342385460408712:5847]
2025-10-09T21:49:13.764220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342385460408712:5847]
2025-10-09T21:49:13.764233Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:13.764247Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:13.764250Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:13.764251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:13.764252Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:13.764265Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:13.764251Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1451
2025-10-09T21:49:13.764268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:13.764298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342385460408712:5847]
2025-10-09T21:49:13.764310Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:13.764316Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:13.764326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.764331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.764362Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1451
2025-10-09T21:49:13.764385Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3cc9aecf-9f7274cd-a8817247-5d57f318_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:13.764398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:13.764403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:13.764405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:13.764415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:13.764426Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:13.764449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:13.764607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.764617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.764632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:13.764745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1452 partition: 0 MaxSeqNo: 25 sessionId: src_id|3cc9aecf-9f7274cd-a8817247-5d57f318_0
2025-10-09T21:49:13.765135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|456ac5fc-ade4167f-387503ff-f01a42dd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3cc9aecf-9f7274cd-a8817247-5d57f318_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:13.767614Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3cc9aecf-9f7274cd-a8817247-5d57f318_0] Write session: try to update token
2025-10-09T21:49:13.767632Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3cc9aecf-9f7274cd-a8817247-5d57f318_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:13.778639Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:13.778652Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:13.778654Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:13.778656Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:13.778664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342385460408712:5847]
2025-10-09T21:49:13.778667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342385460408712:5847]
2025-10-09T21:49:13.778670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342385460408712:5847]
2025-10-09T21:49:13.778706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:13.778714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.778716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:13.778723Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:13.778732Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:13.778738Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:13.778845Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:13.781638Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1452 sessionId: src_id|3cc9aecf-9f7274cd-a8817247-5d57f318_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:13.783890Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:13.833481Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342385822233987:3441]
2025-10-09T21:49:13.833731Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 1451, pipe: [1:7559342385822233987:3441]
2025-10-09T21:49:13.927231Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:13.927500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.927515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.927560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342385822233987:3441], now have 1 active actors on pipe
2025-10-09T21:49:13.927788Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:13.927814Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:13.927819Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:13.927860Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342385822233987:3441]
2025-10-09T21:49:13.927880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1452 reason: error at write: Disconnected sessionId: src_id|3cc9aecf-9f7274cd-a8817247-5d57f318_0
2025-10-09T21:49:13.928051Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:13.928092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1452 sessionId: src_id|3cc9aecf-9f7274cd-a8817247-5d57f318_0 is DEAD
2025-10-09T21:49:13.928240Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cc9aecf-9f7274cd-a8817247-5d57f318_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:13.928252Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cc9aecf-9f7274cd-a8817247-5d57f318_0] Write session will restart in 0.005294s
2025-10-09T21:49:13.928280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cc9aecf-9f7274cd-a8817247-5d57f318_0] Write session: Do CDS request
2025-10-09T21:49:13.928286Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cc9aecf-9f7274cd-a8817247-5d57f318_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:13.934455Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3cc9aecf-9f7274cd-a8817247-5d57f318_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:13.934550Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:13.934567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1453
2025-10-09T21:49:13.934698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1453 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:13.934732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1453 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:13.934742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1453 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:13.934749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:13.935210Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1453 sessionId:  describe result for acl check
2025-10-09T21:49:13.935251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:13.935258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:13.935260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:13.935266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342385822234007:3018] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:13.935830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342385822234007:3018] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:13.938974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342385822234007:3018] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:13.942215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342385822234007:3018] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:13.942233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342385822234007:3018] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:13.942237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342385822234007:3018] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:13.942245Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1453 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:13.947736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:13.950326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:13.950526Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:13.950535Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:13.950548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:13.950611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:13.950649Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342385460408995:5846]
2025-10-09T21:49:13.951224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:13.951670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:13.951730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:13.952086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:13.952142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.952148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.952149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.952150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.952151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.952153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.952154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.952155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.952156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.952161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.952173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.952177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.952181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.952185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.952189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.952193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.952197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.952201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.952208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.952216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:13.952222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.952227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:13.952237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.952242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:13.952247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.952253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:13.952257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.952262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:13.952267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.952272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:13.952277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.952282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.952291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.952296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.952302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.952309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.952344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:13.959254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:13.960412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:13.960426Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:13.960428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:13.960431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:13.960436Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1452 [2:7559342385460408995:5846]
2025-10-09T21:49:13.960446Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:13.960461Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:13.960463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:13.960465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:13.960466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:13.960478Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:13.960488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:13.960495Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:13.960559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.960567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.960568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.960570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:13.960622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:13.960628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.960635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:13.960641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:13.960651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:13.960665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342385460408990:5846]
2025-10-09T21:49:13.960667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342385460408990:5846]
2025-10-09T21:49:13.960668Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342385460408990:5846]
2025-10-09T21:49:13.960675Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:13.960687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:13.960687Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:13.960690Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:13.960692Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:13.960695Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:13.960697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:13.960706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342385822233992:2432], now have 1 active actors on pipe
2025-10-09T21:49:13.960709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342385822234156:3018], now have 1 active actors on pipe
2025-10-09T21:49:13.960744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342385460408990:5846]
2025-10-09T21:49:13.960753Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:13.960773Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:13.960770Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1452
2025-10-09T21:49:13.960792Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1452
2025-10-09T21:49:13.960836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.960845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.960876Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|26f1aa5e-e0062f41-bfc9d142-e95a2999_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:13.960885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:13.960888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:13.960890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:13.960899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:13.960906Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:13.960917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:13.961024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.961031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.961041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:13.961126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1453 partition: 0 MaxSeqNo: 25 sessionId: src_id|26f1aa5e-e0062f41-bfc9d142-e95a2999_0
2025-10-09T21:49:13.961512Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cc9aecf-9f7274cd-a8817247-5d57f318_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|26f1aa5e-e0062f41-bfc9d142-e95a2999_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:13.964108Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26f1aa5e-e0062f41-bfc9d142-e95a2999_0] Write session: try to update token
2025-10-09T21:49:13.964128Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26f1aa5e-e0062f41-bfc9d142-e95a2999_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:13.971927Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:13.971941Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:13.971951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342385460408990:5846]
2025-10-09T21:49:13.971996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:13.972001Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:13.972163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:13.972302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:13.972440Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:13.972576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:13.972759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:13.972802Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:13.976462Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1453 sessionId: src_id|26f1aa5e-e0062f41-bfc9d142-e95a2999_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:13.978692Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:13.978854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:13.978873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:13.978910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:13.978994Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:13.984828Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:13.984842Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:13.984844Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:13.984846Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:13.984858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342385460408990:5846]
2025-10-09T21:49:13.984862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342385460408990:5846]
2025-10-09T21:49:13.984866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342385460408990:5846]
2025-10-09T21:49:13.984907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:13.984921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:13.984923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:13.984922Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:13.984926Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:13.984939Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:13.985056Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:14.028047Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342390117201575:3441]
2025-10-09T21:49:14.028368Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 1452, pipe: [1:7559342390117201575:3441]
2025-10-09T21:49:14.131498Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:14.139118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:14.139140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:14.139286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:14.139344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:14.139402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:14.139460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:14.139503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:14.139511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:14.139589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:14.139656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:14.139726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:14.139800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:14.139843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:14.139851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:14.139910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:14.139972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:14.140034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:14.140098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:14.140142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:14.140148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:14.140206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:14.140266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:14.140323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:14.140380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:14.140420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:14.140425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:14.140484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:14.140539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:14.140596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:14.140661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:14.140710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:14.140716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:14.140772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:14.140824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:14.140884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:14.140940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:14.140980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:14.140985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:14.141041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:14.141093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:14.141160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:14.141220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:14.141259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:14.141264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:14.141330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:14.141393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:14.141456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:14.141509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:14.141551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:14.141558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:14.141631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:14.141701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:14.141768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:14.141829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:14.141875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:14.141885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:14.141950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:14.142014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:14.142086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:14.142141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:14.142183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:14.142189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:14.142238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342390117201575:3441], now have 1 active actors on pipe
2025-10-09T21:49:14.142456Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:14.142503Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342390117201575:3441]
2025-10-09T21:49:14.142505Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:14.142509Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:14.142541Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1453 reason: error at write: Disconnected sessionId: src_id|26f1aa5e-e0062f41-bfc9d142-e95a2999_0
2025-10-09T21:49:14.142748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1453 sessionId: src_id|26f1aa5e-e0062f41-bfc9d142-e95a2999_0 is DEAD
2025-10-09T21:49:14.142902Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26f1aa5e-e0062f41-bfc9d142-e95a2999_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:14.142916Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26f1aa5e-e0062f41-bfc9d142-e95a2999_0] Write session will restart in 0.006625s
2025-10-09T21:49:14.142941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26f1aa5e-e0062f41-bfc9d142-e95a2999_0] Write session: Do CDS request
2025-10-09T21:49:14.142947Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26f1aa5e-e0062f41-bfc9d142-e95a2999_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:14.142975Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:14.150427Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26f1aa5e-e0062f41-bfc9d142-e95a2999_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:14.150499Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:14.150522Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1454
2025-10-09T21:49:14.150680Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1454 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:14.150725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1454 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:14.150736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1454 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:14.150742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:14.151236Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1454 sessionId:  describe result for acl check
2025-10-09T21:49:14.151272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:14.151279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:14.151281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:14.151287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342390117201606:3007] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:14.151854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342390117201606:3007] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:14.155172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342390117201606:3007] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:14.158717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342390117201606:3007] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:14.158734Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342390117201606:3007] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:14.158738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342390117201606:3007] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:14.158748Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1454 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:14.163001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:14.165989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:14.166219Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:14.166230Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:14.166243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:14.166298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:14.166346Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342389755376571:5840]
2025-10-09T21:49:14.166944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:14.167286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:14.167343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:14.167677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:14.167730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.167736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.167738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.167739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.167740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.167742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.167743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.167744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.167745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.167750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.167763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.167767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.167771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.167775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.167779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.167783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.167787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.167791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.167799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.167807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.167813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.167819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.167829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.167834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.167838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.167844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.167848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.167859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.167863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.167868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.167873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.167882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.167886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.167892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.167896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.167902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.167940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:14.174630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:14.175910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:14.175928Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:14.175930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:14.175935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:14.175943Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1453 [2:7559342389755376571:5840]
2025-10-09T21:49:14.175962Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:14.175982Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:14.175990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:14.175993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:14.175996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:14.176016Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:14.176027Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:14.176035Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:14.176098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.176105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.176107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.176110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:14.176189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:14.176197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.176210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.176227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.176250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:14.176276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342390117201777:3007], now have 1 active actors on pipe
2025-10-09T21:49:14.176284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342390117201580:2432], now have 1 active actors on pipe
2025-10-09T21:49:14.176293Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342389755376566:5840]
2025-10-09T21:49:14.176296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342389755376566:5840]
2025-10-09T21:49:14.176298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342389755376566:5840]
2025-10-09T21:49:14.176310Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:14.176322Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:14.176329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:14.176329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:14.176331Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:14.176345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:14.176353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:14.176343Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1453
2025-10-09T21:49:14.176354Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1453
2025-10-09T21:49:14.176390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342389755376566:5840]
2025-10-09T21:49:14.176398Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:14.176399Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:14.176430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:14.176440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:14.176476Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|50f40dec-a337c74f-99ff4876-191510e1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:14.176488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:14.176496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:14.176499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:14.176509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:14.176523Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:14.176543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:14.176710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:14.176719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:14.176736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:14.176806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1454 partition: 0 MaxSeqNo: 25 sessionId: src_id|50f40dec-a337c74f-99ff4876-191510e1_0
2025-10-09T21:49:14.177173Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26f1aa5e-e0062f41-bfc9d142-e95a2999_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|50f40dec-a337c74f-99ff4876-191510e1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:14.179746Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|50f40dec-a337c74f-99ff4876-191510e1_0] Write session: try to update token
2025-10-09T21:49:14.179772Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|50f40dec-a337c74f-99ff4876-191510e1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:14.191241Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:14.191260Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:14.191263Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:14.191265Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:14.191275Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342389755376566:5840]
2025-10-09T21:49:14.191279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342389755376566:5840]
2025-10-09T21:49:14.191284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342389755376566:5840]
2025-10-09T21:49:14.191332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:14.191341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.191342Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:14.191344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:14.191348Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:14.191352Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:14.191488Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:14.192740Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1454 sessionId: src_id|50f40dec-a337c74f-99ff4876-191510e1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:14.195235Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:14.242548Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342390117201877:3441]
2025-10-09T21:49:14.242789Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 1453, pipe: [1:7559342390117201877:3441]
2025-10-09T21:49:14.340882Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:14.341114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:14.341128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:14.341168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342390117201877:3441], now have 1 active actors on pipe
2025-10-09T21:49:14.341324Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:14.341415Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:14.341433Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342390117201877:3441]
2025-10-09T21:49:14.341444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:14.341453Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:14.341506Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1454 reason: error at write: Disconnected sessionId: src_id|50f40dec-a337c74f-99ff4876-191510e1_0
2025-10-09T21:49:14.341696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1454 sessionId: src_id|50f40dec-a337c74f-99ff4876-191510e1_0 is DEAD
2025-10-09T21:49:14.341853Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50f40dec-a337c74f-99ff4876-191510e1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:14.341864Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50f40dec-a337c74f-99ff4876-191510e1_0] Write session will restart in 0.006994s
2025-10-09T21:49:14.341891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50f40dec-a337c74f-99ff4876-191510e1_0] Write session: Do CDS request
2025-10-09T21:49:14.341896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50f40dec-a337c74f-99ff4876-191510e1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:14.350434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|50f40dec-a337c74f-99ff4876-191510e1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:14.350515Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:14.350534Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1455
2025-10-09T21:49:14.350646Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1455 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:14.350680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1455 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:14.350692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1455 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:14.350700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:14.351136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1455 sessionId:  describe result for acl check
2025-10-09T21:49:14.351173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:14.351181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:14.351183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:14.351189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342390117201922:2996] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:14.351760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342390117201922:2996] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:14.354551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342390117201922:2996] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:14.357050Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342390117201922:2996] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:14.357065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342390117201922:2996] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:14.357068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342390117201922:2996] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:14.357076Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1455 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:14.359921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:14.362310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:14.362493Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:14.362502Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:14.362513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:14.362562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:14.362607Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342389755376864:5822]
2025-10-09T21:49:14.363191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:14.363464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:14.363524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:14.363852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:14.363902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.363907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.363909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.363910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.363911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.363914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.363915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.363917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.363918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.363923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.363932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.363937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.363941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.363944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.363948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.363951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.363954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.363957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.363963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.363970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.363977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.363983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.363991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.363997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.364005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.364015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.364025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.364030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.364034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.364039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.364043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.364048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.364052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.364057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.364063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.364072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.364099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:14.370206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:14.371395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:14.371407Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:14.371409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:14.371414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:14.371419Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1454 [2:7559342389755376864:5822]
2025-10-09T21:49:14.371430Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:14.371446Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:14.371449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:14.371452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:14.371454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:14.371470Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:14.371480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:14.371487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:14.371545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.371554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.371555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.371558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:14.371620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:14.371628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.371640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.371653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.371674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:14.371697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342390117202112:2996], now have 1 active actors on pipe
2025-10-09T21:49:14.371705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342390117201879:2432], now have 1 active actors on pipe
2025-10-09T21:49:14.371712Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342389755376859:5822]
2025-10-09T21:49:14.371715Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342389755376859:5822]
2025-10-09T21:49:14.371716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342389755376859:5822]
2025-10-09T21:49:14.371727Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:14.371738Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:14.371731Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1454
2025-10-09T21:49:14.371745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:14.371755Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:14.371757Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:14.371758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:14.371761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:14.371768Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1454
2025-10-09T21:49:14.371793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342389755376859:5822]
2025-10-09T21:49:14.371800Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:14.371802Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:14.371817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:14.371825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:14.371871Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|192358b0-97af7770-b367a379-152fcf1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:14.371881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:14.371886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:14.371892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:14.371903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:14.371916Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:14.371936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:14.372067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:14.372076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:14.372089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:14.372161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1455 partition: 0 MaxSeqNo: 25 sessionId: src_id|192358b0-97af7770-b367a379-152fcf1_0
2025-10-09T21:49:14.372508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50f40dec-a337c74f-99ff4876-191510e1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|192358b0-97af7770-b367a379-152fcf1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:14.375087Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|192358b0-97af7770-b367a379-152fcf1_0] Write session: try to update token
2025-10-09T21:49:14.375103Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|192358b0-97af7770-b367a379-152fcf1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:14.386762Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:14.386777Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:14.386779Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:14.386782Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:14.386791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342389755376859:5822]
2025-10-09T21:49:14.386795Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342389755376859:5822]
2025-10-09T21:49:14.386798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342389755376859:5822]
2025-10-09T21:49:14.386840Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:14.386841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:14.386847Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:14.386849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.386851Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:14.386852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:14.387000Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:14.388944Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1455 sessionId: src_id|192358b0-97af7770-b367a379-152fcf1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:14.391432Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:14.441676Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342390117202191:3441]
2025-10-09T21:49:14.441912Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 1454, pipe: [1:7559342390117202191:3441]
2025-10-09T21:49:14.529201Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:14.529474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:14.529492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:14.529532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342390117202191:3441], now have 1 active actors on pipe
2025-10-09T21:49:14.529747Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:14.529796Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:14.529799Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342390117202191:3441]
2025-10-09T21:49:14.529815Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:14.529818Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:14.529874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1455 reason: error at write: Disconnected sessionId: src_id|192358b0-97af7770-b367a379-152fcf1_0
2025-10-09T21:49:14.530069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1455 sessionId: src_id|192358b0-97af7770-b367a379-152fcf1_0 is DEAD
2025-10-09T21:49:14.530191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|192358b0-97af7770-b367a379-152fcf1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:14.530202Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|192358b0-97af7770-b367a379-152fcf1_0] Write session will restart in 0.006463s
2025-10-09T21:49:14.530230Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|192358b0-97af7770-b367a379-152fcf1_0] Write session: Do CDS request
2025-10-09T21:49:14.530236Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|192358b0-97af7770-b367a379-152fcf1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:14.537464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|192358b0-97af7770-b367a379-152fcf1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:14.537618Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:14.537636Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1456
2025-10-09T21:49:14.537767Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1456 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:14.537804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1456 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:14.537812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1456 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:14.537814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:14.538295Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1456 sessionId:  describe result for acl check
2025-10-09T21:49:14.538331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:14.538337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:14.538340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:14.538345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342390117202213:2981] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:14.538859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342390117202213:2981] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:14.542349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342390117202213:2981] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:14.545655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342390117202213:2981] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:14.545671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342390117202213:2981] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:14.545677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342390117202213:2981] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:14.545686Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1456 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:14.549238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:14.551810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:14.551995Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:14.552003Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:14.552013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:14.552069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:14.552105Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342389755377170:5798]
2025-10-09T21:49:14.552686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:14.552943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:14.552976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:14.553270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:14.553310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.553315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.553316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.553318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.553319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.553320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.553321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.553323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.553324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.553329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.553340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.553348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.553352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.553355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.553364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.553369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.553376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.553380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.553391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.553402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.553407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.553414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.553422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.553426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.553434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.553438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.553445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.553450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.553457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.553462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.553471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.553480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.553484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.553492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.553496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.553502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.553529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:14.559611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:14.560819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:14.560834Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:14.560837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:14.560841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:14.560847Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1455 [2:7559342389755377170:5798]
2025-10-09T21:49:14.560859Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:14.560876Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:14.560880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:14.560883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:14.560885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:14.560900Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:14.560910Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:14.560917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:14.560976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.560982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.560984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.560986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:14.561052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:14.561060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.561071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.561082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.561104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:14.561124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342390117202193:2432], now have 1 active actors on pipe
2025-10-09T21:49:14.561130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342390117202403:2981], now have 1 active actors on pipe
2025-10-09T21:49:14.561138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342389755377165:5798]
2025-10-09T21:49:14.561141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342389755377165:5798]
2025-10-09T21:49:14.561144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342389755377165:5798]
2025-10-09T21:49:14.561157Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:14.561166Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:14.561168Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:14.561170Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:14.561178Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:14.561195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:14.561204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:14.561186Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1455
2025-10-09T21:49:14.561233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342389755377165:5798]
2025-10-09T21:49:14.561220Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1455
2025-10-09T21:49:14.561237Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:14.561241Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:14.561266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:14.561271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:14.561304Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d87c8dde-67a6fe88-2ab5bfb0-a9494acd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:14.561316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:14.561320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:14.561322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:14.561332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:14.561342Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:14.561361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:14.561523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:14.561532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:14.561547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:14.561669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1456 partition: 0 MaxSeqNo: 25 sessionId: src_id|d87c8dde-67a6fe88-2ab5bfb0-a9494acd_0
2025-10-09T21:49:14.562051Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|192358b0-97af7770-b367a379-152fcf1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d87c8dde-67a6fe88-2ab5bfb0-a9494acd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:14.564471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d87c8dde-67a6fe88-2ab5bfb0-a9494acd_0] Write session: try to update token
2025-10-09T21:49:14.564490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d87c8dde-67a6fe88-2ab5bfb0-a9494acd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:14.575001Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:14.575019Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:14.575021Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:14.575023Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:14.575034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342389755377165:5798]
2025-10-09T21:49:14.575038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342389755377165:5798]
2025-10-09T21:49:14.575042Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342389755377165:5798]
2025-10-09T21:49:14.575088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:14.575088Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:14.575094Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:14.575096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.575099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:14.575111Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:14.575241Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:14.577034Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1456 sessionId: src_id|d87c8dde-67a6fe88-2ab5bfb0-a9494acd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:14.579472Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:14.630117Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342390117202484:3441]
2025-10-09T21:49:14.630456Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 1455, pipe: [1:7559342390117202484:3441]
2025-10-09T21:49:14.735320Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:14.735870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:14.735888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:14.735933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342390117202484:3441], now have 1 active actors on pipe
2025-10-09T21:49:14.736144Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:14.736179Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:14.736188Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342390117202484:3441]
2025-10-09T21:49:14.736214Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:14.736291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1456 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d87c8dde-67a6fe88-2ab5bfb0-a9494acd_0
2025-10-09T21:49:14.736480Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1456 sessionId: src_id|d87c8dde-67a6fe88-2ab5bfb0-a9494acd_0 is DEAD
2025-10-09T21:49:14.736659Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d87c8dde-67a6fe88-2ab5bfb0-a9494acd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:14.736672Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d87c8dde-67a6fe88-2ab5bfb0-a9494acd_0] Write session will restart in 0.006134s
2025-10-09T21:49:14.736701Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d87c8dde-67a6fe88-2ab5bfb0-a9494acd_0] Write session: Do CDS request
2025-10-09T21:49:14.736707Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d87c8dde-67a6fe88-2ab5bfb0-a9494acd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:14.743571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d87c8dde-67a6fe88-2ab5bfb0-a9494acd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:14.743627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:14.743647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1457
2025-10-09T21:49:14.743813Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1457 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:14.743845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1457 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:14.743855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1457 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:14.743858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:14.744283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1457 sessionId:  describe result for acl check
2025-10-09T21:49:14.744317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:14.744319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:14.744320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:14.744325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342390117202503:2980] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:14.744870Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342390117202503:2980] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:14.746997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342390117202503:2980] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:14.750564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342390117202503:2980] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:14.750581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342390117202503:2980] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:14.750585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342390117202503:2980] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:14.750593Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1457 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:14.756390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:14.758837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:14.759015Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:14.759025Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:14.759036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:14.759088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:14.759127Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342389755377457:5783]
2025-10-09T21:49:14.759723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:14.760001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:14.760037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:14.760371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:14.760420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.760427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.760428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.760429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.760431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.760432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.760433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.760434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.760435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.760440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.760451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.760455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.760460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.760464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.760468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.760472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.760476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.760480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.760486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.760494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.760500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.760505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.760510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.760515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.760520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.760525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.760529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.760534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.760539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.760544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.760549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.760554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.760558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.760563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.760568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.760573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.760602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:14.766664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:14.767843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:14.767857Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:14.767859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:14.767863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:14.767870Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1456 [2:7559342389755377457:5783]
2025-10-09T21:49:14.767884Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:14.767906Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:14.767909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:14.767917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:14.767919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:14.767935Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:14.767944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:14.767955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:14.768009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.768015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.768017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.768020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:14.768085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:14.768093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.768105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.768117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.768135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:14.768159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342390117202634:2980], now have 1 active actors on pipe
2025-10-09T21:49:14.768167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342390117202488:2432], now have 1 active actors on pipe
2025-10-09T21:49:14.768174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342389755377452:5783]
2025-10-09T21:49:14.768176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342389755377452:5783]
2025-10-09T21:49:14.768179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342389755377452:5783]
2025-10-09T21:49:14.768191Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:14.768208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:14.768210Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:14.768213Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:14.768215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:14.768227Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:14.768234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:14.768213Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1456
2025-10-09T21:49:14.768254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342389755377452:5783]
2025-10-09T21:49:14.768264Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:14.768232Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1456
2025-10-09T21:49:14.768271Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:14.768280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:14.768284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:14.768330Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2a59d3cb-6c6342d9-61a4bfac-e45c9ab8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:14.768339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:14.768347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:14.768350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:14.768360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:14.768373Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:14.768393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:14.768521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:14.768528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:14.768540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:14.768646Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1457 partition: 0 MaxSeqNo: 25 sessionId: src_id|2a59d3cb-6c6342d9-61a4bfac-e45c9ab8_0
2025-10-09T21:49:14.768996Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d87c8dde-67a6fe88-2ab5bfb0-a9494acd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2a59d3cb-6c6342d9-61a4bfac-e45c9ab8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:14.771474Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a59d3cb-6c6342d9-61a4bfac-e45c9ab8_0] Write session: try to update token
2025-10-09T21:49:14.771492Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a59d3cb-6c6342d9-61a4bfac-e45c9ab8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:14.782266Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:14.782283Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:14.782286Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:14.782288Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:14.782297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342389755377452:5783]
2025-10-09T21:49:14.782302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342389755377452:5783]
2025-10-09T21:49:14.782306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342389755377452:5783]
2025-10-09T21:49:14.782347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:14.782350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:14.782357Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:14.782357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.782360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:14.782361Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:14.782488Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:14.785052Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1457 sessionId: src_id|2a59d3cb-6c6342d9-61a4bfac-e45c9ab8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:14.787515Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:14.836469Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342390117202784:3441]
2025-10-09T21:49:14.836711Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 1456, pipe: [1:7559342390117202784:3441]
2025-10-09T21:49:14.926515Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:14.926911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:14.926926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:14.926964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342390117202784:3441], now have 1 active actors on pipe
2025-10-09T21:49:14.927113Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:14.927180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:14.927193Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342390117202784:3441]
2025-10-09T21:49:14.927198Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:14.927214Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:14.927260Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1457 reason: error at write: Disconnected sessionId: src_id|2a59d3cb-6c6342d9-61a4bfac-e45c9ab8_0
2025-10-09T21:49:14.927429Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1457 sessionId: src_id|2a59d3cb-6c6342d9-61a4bfac-e45c9ab8_0 is DEAD
2025-10-09T21:49:14.927570Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a59d3cb-6c6342d9-61a4bfac-e45c9ab8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:14.927579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a59d3cb-6c6342d9-61a4bfac-e45c9ab8_0] Write session will restart in 0.008709s
2025-10-09T21:49:14.927609Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a59d3cb-6c6342d9-61a4bfac-e45c9ab8_0] Write session: Do CDS request
2025-10-09T21:49:14.927615Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a59d3cb-6c6342d9-61a4bfac-e45c9ab8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:14.937549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a59d3cb-6c6342d9-61a4bfac-e45c9ab8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:14.937632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:14.937657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1458
2025-10-09T21:49:14.937769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1458 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:14.937794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1458 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:14.937799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1458 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:14.937804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:14.938410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1458 sessionId:  describe result for acl check
2025-10-09T21:49:14.938442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:14.938444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:14.938445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:14.938449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342390117202858:2974] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:14.938944Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342390117202858:2974] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:14.941503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342390117202858:2974] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:14.944494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342390117202858:2974] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:14.944508Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342390117202858:2974] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:14.944511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342390117202858:2974] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:14.944518Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1458 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:14.946681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:14.949090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:14.949265Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:14.949276Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:14.949288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:14.949338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:14.949374Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342389755377740:5791]
2025-10-09T21:49:14.949949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:14.950274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:14.950313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:14.950607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:14.950674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.950676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.950677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.950678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.950679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.950680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.950681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.950682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.950683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.950688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.950699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.950702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.950706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.950709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.950713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.950717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.950720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.950724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.950731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.950739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:14.950745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.950749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:14.950754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.950759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:14.950763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.950768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:14.950773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.950778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:14.950782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.950787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:14.950793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.950797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.950802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.950806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.950810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.950814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.950839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:14.957318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:14.958496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:14.958513Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:14.958516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:14.958520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:14.958527Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1457 [2:7559342389755377740:5791]
2025-10-09T21:49:14.958542Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:14.958561Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:14.958564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:14.958567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:14.958570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:14.958587Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:14.958598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:14.958605Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:14.958661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.958668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.958670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.958673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:14.958742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:14.958750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.958762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:14.958779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:14.958799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:14.958824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342390117202790:2432], now have 1 active actors on pipe
2025-10-09T21:49:14.958831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342390117203050:2974], now have 1 active actors on pipe
2025-10-09T21:49:14.958839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342389755377735:5791]
2025-10-09T21:49:14.958841Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342389755377735:5791]
2025-10-09T21:49:14.958844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342389755377735:5791]
2025-10-09T21:49:14.958860Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:14.958875Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:14.958887Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:14.958890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:14.958890Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:14.958892Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:14.958893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:14.958866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1457
2025-10-09T21:49:14.958875Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1457
2025-10-09T21:49:14.958923Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342389755377735:5791]
2025-10-09T21:49:14.958932Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:14.958939Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:14.958949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:14.958953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:14.959009Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d1cbde96-5ca263b3-663bdcea-45a0792e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:14.959022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:14.959027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:14.959032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:14.959042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:14.959050Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:14.959066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:14.959201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:14.959208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:14.959220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:14.959299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1458 partition: 0 MaxSeqNo: 25 sessionId: src_id|d1cbde96-5ca263b3-663bdcea-45a0792e_0
2025-10-09T21:49:14.959693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a59d3cb-6c6342d9-61a4bfac-e45c9ab8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d1cbde96-5ca263b3-663bdcea-45a0792e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:14.962009Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1cbde96-5ca263b3-663bdcea-45a0792e_0] Write session: try to update token
2025-10-09T21:49:14.962025Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1cbde96-5ca263b3-663bdcea-45a0792e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:14.973470Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:14.973487Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:14.973490Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:14.973492Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:14.973502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342389755377735:5791]
2025-10-09T21:49:14.973506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342389755377735:5791]
2025-10-09T21:49:14.973510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342389755377735:5791]
2025-10-09T21:49:14.973551Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:14.973554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:14.973556Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:14.973558Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:14.973562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:14.973564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:14.973719Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:14.974871Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1458 sessionId: src_id|d1cbde96-5ca263b3-663bdcea-45a0792e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:14.977042Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:15.027454Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342394412170378:3441]
2025-10-09T21:49:15.027750Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 1457, pipe: [1:7559342394412170378:3441]
2025-10-09T21:49:15.132205Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:15.132485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:15.132500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:15.132545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342394412170378:3441], now have 1 active actors on pipe
2025-10-09T21:49:15.132762Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:15.132776Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342394412170378:3441]
2025-10-09T21:49:15.132853Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:15.132878Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:15.132958Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1458 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d1cbde96-5ca263b3-663bdcea-45a0792e_0
2025-10-09T21:49:15.133170Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1458 sessionId: src_id|d1cbde96-5ca263b3-663bdcea-45a0792e_0 is DEAD
2025-10-09T21:49:15.133344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1cbde96-5ca263b3-663bdcea-45a0792e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:15.133354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1cbde96-5ca263b3-663bdcea-45a0792e_0] Write session will restart in 0.009628s
2025-10-09T21:49:15.133376Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1cbde96-5ca263b3-663bdcea-45a0792e_0] Write session: Do CDS request
2025-10-09T21:49:15.133381Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1cbde96-5ca263b3-663bdcea-45a0792e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:15.144506Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1cbde96-5ca263b3-663bdcea-45a0792e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:15.144572Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:15.144595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1459
2025-10-09T21:49:15.144714Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1459 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:15.144739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1459 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:15.144755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1459 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:15.144762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:15.145194Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1459 sessionId:  describe result for acl check
2025-10-09T21:49:15.145243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:15.145249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:15.145250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:15.145256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342394412170464:2971] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:15.145715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342394412170464:2971] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:15.148389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342394412170464:2971] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:15.151022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342394412170464:2971] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:15.151037Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342394412170464:2971] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:15.151039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342394412170464:2971] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:15.151048Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1459 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:15.152515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:15.155275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:15.155561Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:15.155571Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:15.155584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:15.155644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:15.155683Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342394050345321:5789]
2025-10-09T21:49:15.156283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:15.156596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:15.156667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:15.157031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:15.157099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.157105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.157107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.157108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.157109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.157110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.157111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.157112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.157114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.157119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.157130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.157134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.157137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.157140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.157143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.157146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.157149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.157153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.157161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.157169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.157176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.157182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.157186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.157192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.157196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.157201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.157206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.157211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.157215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.157221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.157226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.157231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.157236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.157241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.157245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.157251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.157282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:15.163507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:15.164838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:15.164854Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:15.164856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:15.164861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:15.164867Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1458 [2:7559342394050345321:5789]
2025-10-09T21:49:15.164881Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:15.164900Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:15.164906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:15.164909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:15.164910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:15.164927Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:15.164939Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:15.164947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:15.165004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.165011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.165013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.165016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:15.165085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:15.165092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.165104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.165120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.165140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:15.165160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342394412170660:2971], now have 1 active actors on pipe
2025-10-09T21:49:15.165165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342394412170381:2432], now have 1 active actors on pipe
2025-10-09T21:49:15.165172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342394050345316:5789]
2025-10-09T21:49:15.165173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342394050345316:5789]
2025-10-09T21:49:15.165175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342394050345316:5789]
2025-10-09T21:49:15.165185Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:15.165198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:15.165205Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:15.165206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:15.165208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:15.165215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:15.165217Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:15.165237Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342394050345316:5789]
2025-10-09T21:49:15.165243Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:15.165250Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:15.165224Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1458
2025-10-09T21:49:15.165227Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1458
2025-10-09T21:49:15.165273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:15.165280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:15.165309Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7abc8477-483ee92b-4c426fec-fc564b6d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:15.165318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:15.165321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:15.165323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:15.165331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:15.165342Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:15.165358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:15.165455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:15.165460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:15.165470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:15.165564Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1459 partition: 0 MaxSeqNo: 25 sessionId: src_id|7abc8477-483ee92b-4c426fec-fc564b6d_0
2025-10-09T21:49:15.165903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1cbde96-5ca263b3-663bdcea-45a0792e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7abc8477-483ee92b-4c426fec-fc564b6d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:15.168212Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7abc8477-483ee92b-4c426fec-fc564b6d_0] Write session: try to update token
2025-10-09T21:49:15.168227Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7abc8477-483ee92b-4c426fec-fc564b6d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:15.179755Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:15.179774Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:15.179777Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:15.179779Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:15.179789Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342394050345316:5789]
2025-10-09T21:49:15.179794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342394050345316:5789]
2025-10-09T21:49:15.179798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342394050345316:5789]
2025-10-09T21:49:15.179845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:15.179852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.179855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:15.179856Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:15.179862Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:15.179867Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:15.180000Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:15.182851Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1459 sessionId: src_id|7abc8477-483ee92b-4c426fec-fc564b6d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:15.185545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:15.232765Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342394412170678:3441]
2025-10-09T21:49:15.233111Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 1458, pipe: [1:7559342394412170678:3441]
2025-10-09T21:49:15.338305Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:15.338772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:15.338800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:15.338848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342394412170678:3441], now have 1 active actors on pipe
2025-10-09T21:49:15.339065Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:15.339099Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:15.339128Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:15.339132Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:15.339156Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342394412170678:3441]
2025-10-09T21:49:15.339204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1459 reason: error at write: Disconnected sessionId: src_id|7abc8477-483ee92b-4c426fec-fc564b6d_0
2025-10-09T21:49:15.339415Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1459 sessionId: src_id|7abc8477-483ee92b-4c426fec-fc564b6d_0 is DEAD
2025-10-09T21:49:15.339564Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7abc8477-483ee92b-4c426fec-fc564b6d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:15.339577Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7abc8477-483ee92b-4c426fec-fc564b6d_0] Write session will restart in 0.005139s
2025-10-09T21:49:15.339606Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7abc8477-483ee92b-4c426fec-fc564b6d_0] Write session: Do CDS request
2025-10-09T21:49:15.339614Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7abc8477-483ee92b-4c426fec-fc564b6d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:15.345491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7abc8477-483ee92b-4c426fec-fc564b6d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:15.345602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:15.345630Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1460
2025-10-09T21:49:15.345810Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1460 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:15.345843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1460 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:15.345851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1460 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:15.345854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:15.346378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1460 sessionId:  describe result for acl check
2025-10-09T21:49:15.346418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:15.346420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:15.346421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:15.346427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342394412170701:2953] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:15.346993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342394412170701:2953] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:15.348819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342394412170701:2953] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:15.352666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342394412170701:2953] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:15.352683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342394412170701:2953] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:15.352686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342394412170701:2953] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:15.352695Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1460 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:15.362878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:15.365910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:15.366154Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:15.366166Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:15.366180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:15.366244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:15.366292Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342394050345606:5787]
2025-10-09T21:49:15.366918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:15.367302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:15.367364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:15.367713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:15.367780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.367786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.367787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.367789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.367790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.367791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.367792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.367793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.367795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.367800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.367812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.367817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.367821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.367825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.367829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.367833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.367838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.367842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.367849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.367857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.367863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.367868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.367872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.367877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.367886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.367891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.367900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.367906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.367910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.367916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.367928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.367943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.367952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.367958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.367967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.367971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.367997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:15.374784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:15.376217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:15.376231Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:15.376232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:15.376236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:15.376250Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1459 [2:7559342394050345606:5787]
2025-10-09T21:49:15.376261Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:15.376281Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:15.376284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:15.376287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:15.376289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:15.376310Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:15.376321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:15.376329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:15.376394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.376405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.376407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.376410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:15.376481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:15.376489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.376502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.376519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.376540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:15.376567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342394412170688:2432], now have 1 active actors on pipe
2025-10-09T21:49:15.376575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342394412170794:2953], now have 1 active actors on pipe
2025-10-09T21:49:15.376583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342394050345601:5787]
2025-10-09T21:49:15.376586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342394050345601:5787]
2025-10-09T21:49:15.376589Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342394050345601:5787]
2025-10-09T21:49:15.376604Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:15.376623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:15.376639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:15.376641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:15.376660Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1459
2025-10-09T21:49:15.376663Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1459
2025-10-09T21:49:15.376679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342394050345601:5787]
2025-10-09T21:49:15.376686Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:15.376690Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:15.376690Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:15.376693Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:15.376701Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:15.376759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:15.376764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:15.376795Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|369d126c-6ccea437-31e24764-ceb7014_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:15.376812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:15.376816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:15.376819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:15.376829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:15.376840Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:15.376855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:15.377063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:15.377076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:15.377104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:15.377207Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1460 partition: 0 MaxSeqNo: 25 sessionId: src_id|369d126c-6ccea437-31e24764-ceb7014_0
2025-10-09T21:49:15.377595Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7abc8477-483ee92b-4c426fec-fc564b6d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|369d126c-6ccea437-31e24764-ceb7014_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:15.380147Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|369d126c-6ccea437-31e24764-ceb7014_0] Write session: try to update token
2025-10-09T21:49:15.380168Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|369d126c-6ccea437-31e24764-ceb7014_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:15.388719Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:15.388739Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:15.388750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342394050345601:5787]
2025-10-09T21:49:15.388794Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:15.388800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:15.388977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:15.389127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:15.389272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:15.389407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:15.389593Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:15.389654Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:15.393517Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1460 sessionId: src_id|369d126c-6ccea437-31e24764-ceb7014_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:15.396054Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:15.396220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:15.396238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:15.396276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:15.396374Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:15.402130Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:15.402142Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:15.402143Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:15.402145Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:15.402154Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342394050345601:5787]
2025-10-09T21:49:15.402157Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342394050345601:5787]
2025-10-09T21:49:15.402159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342394050345601:5787]
2025-10-09T21:49:15.402199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:15.402205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.402205Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:15.402208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:15.402212Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:15.402217Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:15.402384Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:15.439083Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342394412170996:3441]
2025-10-09T21:49:15.439273Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 1459, pipe: [1:7559342394412170996:3441]
2025-10-09T21:49:15.565138Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:15.571929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:15.571950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:15.572091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:15.572145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:15.572206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:15.572267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:15.572583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:15.572595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:15.572665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:15.572717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:15.572774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:15.572858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:15.572898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:15.572904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:15.572955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:15.573008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:15.573065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:15.573115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:15.573152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:15.573157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:15.573221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:15.573379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:15.573432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:15.573484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:15.573523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:15.573529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:15.573584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:15.573649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:15.573703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:15.573757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:15.573794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:15.573799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:15.573849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:15.573901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:15.573953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:15.574006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:15.574064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:15.574071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:15.574123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:15.574172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:15.574224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:15.574274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:15.574310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:15.574315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:15.574367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:15.574434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:15.574484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:15.574534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:15.574586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:15.574591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:15.574642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:15.574689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:15.574738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:15.574790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:15.574828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:15.574833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:15.574883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:15.574930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:15.574979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:15.575034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:15.575085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:15.575093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:15.575130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342394412170996:3441], now have 1 active actors on pipe
2025-10-09T21:49:15.575368Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342394412170996:3441]
2025-10-09T21:49:15.575383Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:15.575408Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:15.575411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:15.575435Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:15.575440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1460 reason: error at write: Disconnected sessionId: src_id|369d126c-6ccea437-31e24764-ceb7014_0
2025-10-09T21:49:15.575647Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1460 sessionId: src_id|369d126c-6ccea437-31e24764-ceb7014_0 is DEAD
2025-10-09T21:49:15.575777Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|369d126c-6ccea437-31e24764-ceb7014_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:15.575787Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|369d126c-6ccea437-31e24764-ceb7014_0] Write session will restart in 0.007250s
2025-10-09T21:49:15.575804Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|369d126c-6ccea437-31e24764-ceb7014_0] Write session: Do CDS request
2025-10-09T21:49:15.575809Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|369d126c-6ccea437-31e24764-ceb7014_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:15.584514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|369d126c-6ccea437-31e24764-ceb7014_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:15.584626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:15.584646Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1461
2025-10-09T21:49:15.584772Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1461 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:15.584806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1461 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:15.584816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1461 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:15.584820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:15.585360Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1461 sessionId:  describe result for acl check
2025-10-09T21:49:15.585399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:15.585405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:15.585407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:15.585412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342394412171044:2934] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:15.585994Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342394412171044:2934] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:15.589378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342394412171044:2934] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:15.592789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342394412171044:2934] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:15.592805Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342394412171044:2934] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:15.592809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342394412171044:2934] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:15.592817Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1461 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:15.596063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:15.598601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:15.598930Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:15.598939Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:15.598953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:15.599015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:15.599059Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342394050345933:5766]
2025-10-09T21:49:15.599640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:15.599959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:15.600002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:15.600317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:15.600372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.600377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.600379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.600380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.600381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.600382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.600384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.600385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.600386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.600391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.600402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.600406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.600410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.600414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.600419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.600423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.600427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.600431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.600438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.600446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.600466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.600472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.600477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.600482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.600487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.600493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.600502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.600507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.600516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.600522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.600532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.600541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.600545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.600549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.600552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.600557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.600586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:15.606680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:15.607708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:15.607722Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:15.607726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:15.607730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:15.607736Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1460 [2:7559342394050345933:5766]
2025-10-09T21:49:15.607747Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:15.607765Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:15.607767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:15.607770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:15.607772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:15.607788Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:15.607798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:15.607809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:15.607864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.607873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.607875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.607878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:15.607932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:15.607938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.607945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.607951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.607965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:15.607981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342394050345928:5766]
2025-10-09T21:49:15.607983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342394050345928:5766]
2025-10-09T21:49:15.607985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342394050345928:5766]
2025-10-09T21:49:15.607992Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:15.608002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342394412171008:2432], now have 1 active actors on pipe
2025-10-09T21:49:15.608005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342394412171237:2934], now have 1 active actors on pipe
2025-10-09T21:49:15.608009Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:15.608011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:15.608012Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:15.608014Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:15.608019Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:15.608022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:15.608040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342394050345928:5766]
2025-10-09T21:49:15.608044Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:15.608048Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:15.608067Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1460
2025-10-09T21:49:15.608085Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1460
2025-10-09T21:49:15.608117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:15.608125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:15.608147Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6a94fa72-cad65c3d-7cc4b95a-30e4cb3b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:15.608153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:15.608159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:15.608161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:15.608168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:15.608175Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:15.608188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:15.608340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:15.608350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:15.608368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:15.608472Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1461 partition: 0 MaxSeqNo: 25 sessionId: src_id|6a94fa72-cad65c3d-7cc4b95a-30e4cb3b_0
2025-10-09T21:49:15.608861Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|369d126c-6ccea437-31e24764-ceb7014_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6a94fa72-cad65c3d-7cc4b95a-30e4cb3b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:15.611373Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6a94fa72-cad65c3d-7cc4b95a-30e4cb3b_0] Write session: try to update token
2025-10-09T21:49:15.611391Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6a94fa72-cad65c3d-7cc4b95a-30e4cb3b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:15.622053Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:15.622071Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:15.622073Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:15.622075Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:15.622085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342394050345928:5766]
2025-10-09T21:49:15.622089Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342394050345928:5766]
2025-10-09T21:49:15.622093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342394050345928:5766]
2025-10-09T21:49:15.622137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:15.622135Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:15.622141Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:15.622144Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:15.622145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.622148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:15.622265Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:15.624089Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1461 sessionId: src_id|6a94fa72-cad65c3d-7cc4b95a-30e4cb3b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:15.626496Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:15.675641Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342394412171303:3441]
2025-10-09T21:49:15.675932Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 1460, pipe: [1:7559342394412171303:3441]
2025-10-09T21:49:15.768648Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:15.769342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:15.769362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:15.769411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342394412171303:3441], now have 1 active actors on pipe
2025-10-09T21:49:15.769604Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:15.769645Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342394412171303:3441]
2025-10-09T21:49:15.769646Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:15.769651Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:15.769680Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:15.769714Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1461 reason: error at write: Disconnected sessionId: src_id|6a94fa72-cad65c3d-7cc4b95a-30e4cb3b_0
2025-10-09T21:49:15.769887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1461 sessionId: src_id|6a94fa72-cad65c3d-7cc4b95a-30e4cb3b_0 is DEAD
2025-10-09T21:49:15.770065Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6a94fa72-cad65c3d-7cc4b95a-30e4cb3b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:15.770078Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6a94fa72-cad65c3d-7cc4b95a-30e4cb3b_0] Write session will restart in 0.008033s
2025-10-09T21:49:15.770107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6a94fa72-cad65c3d-7cc4b95a-30e4cb3b_0] Write session: Do CDS request
2025-10-09T21:49:15.770113Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6a94fa72-cad65c3d-7cc4b95a-30e4cb3b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:15.779500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6a94fa72-cad65c3d-7cc4b95a-30e4cb3b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:15.779617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:15.779648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1462
2025-10-09T21:49:15.779788Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1462 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:15.779820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1462 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:15.779830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1462 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:15.779834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:15.780380Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1462 sessionId:  describe result for acl check
2025-10-09T21:49:15.780416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:15.780423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:15.780426Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:15.780432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342394412171358:2928] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:15.780969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342394412171358:2928] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:15.783724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342394412171358:2928] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:15.786675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342394412171358:2928] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:15.786690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342394412171358:2928] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:15.786693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342394412171358:2928] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:15.786701Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1462 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:15.789707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:15.792229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:15.792430Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:15.792441Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:15.792453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:15.792508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:15.792547Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342394050346224:5758]
2025-10-09T21:49:15.793137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:15.793441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:15.793481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:15.793791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:15.793851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.793857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.793859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.793860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.793861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.793862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.793863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.793864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.793865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.793871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.793881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.793885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.793889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.793892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.793896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.793899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.793903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.793907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.793914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.793923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.793929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.793935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.793939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.793944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.793948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.793959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.793965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.793970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.793975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.793980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.793986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.793993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.793997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.794002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.794007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.794012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.794085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:15.800210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:15.801273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:15.801286Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:15.801288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:15.801293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:15.801299Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1461 [2:7559342394050346224:5758]
2025-10-09T21:49:15.801311Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:15.801331Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:15.801338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:15.801341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:15.801343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:15.801359Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:15.801370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:15.801380Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:15.801443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.801450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.801451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.801454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:15.801690Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1461
2025-10-09T21:49:15.801708Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1461
2025-10-09T21:49:15.801794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:15.801804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.801816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.801829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.801848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:15.801875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342394412171306:2432], now have 1 active actors on pipe
2025-10-09T21:49:15.801883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342394412171536:2928], now have 1 active actors on pipe
2025-10-09T21:49:15.801897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:15.801906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:15.801927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342394050346219:5758]
2025-10-09T21:49:15.801934Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342394050346219:5758]
2025-10-09T21:49:15.801936Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342394050346219:5758]
2025-10-09T21:49:15.801950Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:15.801967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:15.801970Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:15.801971Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:15.801971Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:15.801987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:15.801990Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:15.802008Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|37009a30-bc08ba1a-c673a2b7-e95d72ad_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:15.802017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:15.802024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:15.802044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:15.802059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:15.802074Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:15.802107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342394050346219:5758]
2025-10-09T21:49:15.802119Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:15.802127Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:15.802130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:15.802328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:15.802337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:15.802351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:15.802441Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1462 partition: 0 MaxSeqNo: 25 sessionId: src_id|37009a30-bc08ba1a-c673a2b7-e95d72ad_0
2025-10-09T21:49:15.802767Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6a94fa72-cad65c3d-7cc4b95a-30e4cb3b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|37009a30-bc08ba1a-c673a2b7-e95d72ad_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:15.805214Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|37009a30-bc08ba1a-c673a2b7-e95d72ad_0] Write session: try to update token
2025-10-09T21:49:15.805231Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|37009a30-bc08ba1a-c673a2b7-e95d72ad_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:15.816184Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:15.816204Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:15.816206Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:15.816208Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:15.816218Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342394050346219:5758]
2025-10-09T21:49:15.816222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342394050346219:5758]
2025-10-09T21:49:15.816226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342394050346219:5758]
2025-10-09T21:49:15.816268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:15.816270Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:15.816277Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:15.816280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.816283Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:15.816283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:15.816412Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:15.818111Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1462 sessionId: src_id|37009a30-bc08ba1a-c673a2b7-e95d72ad_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:15.820353Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:15.869715Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342394412171604:3441]
2025-10-09T21:49:15.870022Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 1461, pipe: [1:7559342394412171604:3441]
2025-10-09T21:49:15.961074Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:15.961301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:15.961314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:15.961351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342394412171604:3441], now have 1 active actors on pipe
2025-10-09T21:49:15.961514Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:15.961559Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:15.961591Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342394412171604:3441]
2025-10-09T21:49:15.961635Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:15.961709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1462 reason: status is not ok: tablet will be restarted right now sessionId: src_id|37009a30-bc08ba1a-c673a2b7-e95d72ad_0
2025-10-09T21:49:15.961884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1462 sessionId: src_id|37009a30-bc08ba1a-c673a2b7-e95d72ad_0 is DEAD
2025-10-09T21:49:15.962042Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37009a30-bc08ba1a-c673a2b7-e95d72ad_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:15.962055Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37009a30-bc08ba1a-c673a2b7-e95d72ad_0] Write session will restart in 0.007250s
2025-10-09T21:49:15.962082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37009a30-bc08ba1a-c673a2b7-e95d72ad_0] Write session: Do CDS request
2025-10-09T21:49:15.962089Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37009a30-bc08ba1a-c673a2b7-e95d72ad_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:15.970559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|37009a30-bc08ba1a-c673a2b7-e95d72ad_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:15.970594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:15.970611Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1463
2025-10-09T21:49:15.970797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1463 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:15.970839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1463 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:15.970850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1463 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:15.970854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:15.971374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1463 sessionId:  describe result for acl check
2025-10-09T21:49:15.971409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:15.971410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:15.971411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:15.971415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342394412171656:2913] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:15.971948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342394412171656:2913] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:15.974576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342394412171656:2913] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:15.977881Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342394412171656:2913] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:15.977897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342394412171656:2913] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:15.977900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342394412171656:2913] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:15.977909Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1463 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:15.980599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:15.983347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:15.983549Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:15.983559Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:15.983572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:15.983627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:15.983667Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342394050346513:5752]
2025-10-09T21:49:15.984199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:15.984523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:15.984566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:15.984873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:15.984925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.984935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.984937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.984938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.984938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.984940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.984940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.984941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.984942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.984947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.984958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.984963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.984967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.984971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.984975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.984979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.984983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.984987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.984995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.985003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:15.985009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.985015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:15.985020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.985033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:15.985037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.985042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:15.985047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.985052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:15.985056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.985062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:15.985066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.985071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.985075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.985080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.985086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.985091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.985121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:15.991465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:15.992622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:15.992639Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:15.992641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:15.992646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:15.992653Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1462 [2:7559342394050346513:5752]
2025-10-09T21:49:15.992667Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:15.992686Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:15.992693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:15.992697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:15.992699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:15.992715Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:15.992719Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:15.992725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:15.992782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.992790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.992793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.992795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:15.992865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:15.992871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:15.992878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:15.992887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:15.992902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:15.992929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342394412171610:2432], now have 1 active actors on pipe
2025-10-09T21:49:15.992941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342394050346508:5752]
2025-10-09T21:49:15.992943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342394050346508:5752]
2025-10-09T21:49:15.992946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342394050346508:5752]
2025-10-09T21:49:15.992958Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:15.992967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342394412171842:2913], now have 1 active actors on pipe
2025-10-09T21:49:15.992976Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:15.992979Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:15.992981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:15.992982Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:15.992996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:15.993003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:15.993036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342394050346508:5752]
2025-10-09T21:49:15.993034Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1462
2025-10-09T21:49:15.993048Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:15.993040Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1462
2025-10-09T21:49:15.993063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:15.993081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:15.993085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:15.993111Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9af60966-e40de987-a05f3de9-dc0dc011_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:15.993123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:15.993125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:15.993128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:15.993135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:15.993143Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:15.993153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:15.993282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:15.993290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:15.993304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:15.993393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1463 partition: 0 MaxSeqNo: 25 sessionId: src_id|9af60966-e40de987-a05f3de9-dc0dc011_0
2025-10-09T21:49:15.993752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37009a30-bc08ba1a-c673a2b7-e95d72ad_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9af60966-e40de987-a05f3de9-dc0dc011_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:15.996175Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9af60966-e40de987-a05f3de9-dc0dc011_0] Write session: try to update token
2025-10-09T21:49:15.996195Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9af60966-e40de987-a05f3de9-dc0dc011_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:16.007167Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:16.007186Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:16.007190Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:16.007191Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:16.007203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342394050346508:5752]
2025-10-09T21:49:16.007207Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342394050346508:5752]
2025-10-09T21:49:16.007210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342394050346508:5752]
2025-10-09T21:49:16.007255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:16.007258Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:16.007262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.007264Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:16.007264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:16.007280Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:16.007404Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:16.009014Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1463 sessionId: src_id|9af60966-e40de987-a05f3de9-dc0dc011_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:16.011388Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:16.061696Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342398707139201:3441]
2025-10-09T21:49:16.061962Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 1462, pipe: [1:7559342398707139201:3441]
2025-10-09T21:49:16.159446Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:16.159713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:16.159726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:16.159765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342398707139201:3441], now have 1 active actors on pipe
2025-10-09T21:49:16.160034Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:16.160042Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:16.160085Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:16.160089Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:16.160115Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342398707139201:3441]
2025-10-09T21:49:16.160165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1463 reason: error at write: Disconnected sessionId: src_id|9af60966-e40de987-a05f3de9-dc0dc011_0
2025-10-09T21:49:16.160360Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1463 sessionId: src_id|9af60966-e40de987-a05f3de9-dc0dc011_0 is DEAD
2025-10-09T21:49:16.160506Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9af60966-e40de987-a05f3de9-dc0dc011_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:16.160515Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9af60966-e40de987-a05f3de9-dc0dc011_0] Write session will restart in 0.007016s
2025-10-09T21:49:16.160543Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9af60966-e40de987-a05f3de9-dc0dc011_0] Write session: Do CDS request
2025-10-09T21:49:16.160550Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9af60966-e40de987-a05f3de9-dc0dc011_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:16.168486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9af60966-e40de987-a05f3de9-dc0dc011_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:16.168601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:16.168622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1464
2025-10-09T21:49:16.168774Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1464 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:16.168816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1464 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:16.168827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1464 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:16.168831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:16.169284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1464 sessionId:  describe result for acl check
2025-10-09T21:49:16.169331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:16.169339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:16.169341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:16.169346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342398707139223:2916] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:16.169946Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342398707139223:2916] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:16.173711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342398707139223:2916] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:16.176967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342398707139223:2916] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:16.176985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342398707139223:2916] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:16.176988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342398707139223:2916] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:16.176998Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1464 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:16.182214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:16.184956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:16.185170Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:16.185178Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:16.185187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:16.185237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:16.185277Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342398345314098:5741]
2025-10-09T21:49:16.185894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:16.186199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:16.186242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:16.186552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:16.186604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:16.186609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:16.186611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:16.186612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:16.186613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:16.186614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:16.186616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.186617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.186618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.186623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:16.186635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:16.186639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:16.186644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:16.186648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:16.186652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:16.186656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.186660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.186663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.186671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:16.186678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:16.186689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:16.186698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:16.186703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:16.186713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:16.186722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:16.186727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:16.186731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:16.186741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:16.186750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:16.186760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:16.186770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.186780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.186788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.186793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.186802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.186812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.186844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:16.193301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:16.194876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:16.194890Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:16.194892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:16.194897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:16.194904Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1463 [2:7559342398345314098:5741]
2025-10-09T21:49:16.194918Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:16.194935Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:16.194939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:16.194942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:16.194944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:16.194962Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:16.194973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:16.194981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:16.195042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.195050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.195052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.195055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:16.195124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:16.195133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.195146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.195158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.195175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:16.195196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342398707139206:2432], now have 1 active actors on pipe
2025-10-09T21:49:16.195202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342398707139383:2916], now have 1 active actors on pipe
2025-10-09T21:49:16.195210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342398345314093:5741]
2025-10-09T21:49:16.195219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342398345314093:5741]
2025-10-09T21:49:16.195222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342398345314093:5741]
2025-10-09T21:49:16.195234Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:16.195252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:16.195252Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:16.195255Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:16.195257Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:16.195271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:16.195274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:16.195307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342398345314093:5741]
2025-10-09T21:49:16.195314Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:16.195296Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1463
2025-10-09T21:49:16.195318Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:16.195320Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1463
2025-10-09T21:49:16.195349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:16.195353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:16.195392Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a2baa2d2-4e5de025-fa253f78-caf3423e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:16.195404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:16.195408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:16.195413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:16.195424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:16.195437Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:16.195457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:16.195609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:16.195616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:16.195630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:16.195713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1464 partition: 0 MaxSeqNo: 25 sessionId: src_id|a2baa2d2-4e5de025-fa253f78-caf3423e_0
2025-10-09T21:49:16.196079Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9af60966-e40de987-a05f3de9-dc0dc011_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a2baa2d2-4e5de025-fa253f78-caf3423e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:16.198573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a2baa2d2-4e5de025-fa253f78-caf3423e_0] Write session: try to update token
2025-10-09T21:49:16.198594Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a2baa2d2-4e5de025-fa253f78-caf3423e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:16.209548Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:16.209567Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:16.209569Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:16.209571Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:16.209582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342398345314093:5741]
2025-10-09T21:49:16.209586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342398345314093:5741]
2025-10-09T21:49:16.209590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342398345314093:5741]
2025-10-09T21:49:16.209647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:16.209648Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:16.209654Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:16.209654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.209657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:16.209659Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:16.209793Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:16.211385Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1464 sessionId: src_id|a2baa2d2-4e5de025-fa253f78-caf3423e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:16.213948Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:16.260357Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342398707139510:3441]
2025-10-09T21:49:16.260614Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 1463, pipe: [1:7559342398707139510:3441]
2025-10-09T21:49:16.358758Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:16.358974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:16.358986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:16.359024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342398707139510:3441], now have 1 active actors on pipe
2025-10-09T21:49:16.359225Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:16.359243Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:16.359263Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342398707139510:3441]
2025-10-09T21:49:16.359286Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:16.359378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1464 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a2baa2d2-4e5de025-fa253f78-caf3423e_0
2025-10-09T21:49:16.359567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1464 sessionId: src_id|a2baa2d2-4e5de025-fa253f78-caf3423e_0 is DEAD
2025-10-09T21:49:16.359718Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2baa2d2-4e5de025-fa253f78-caf3423e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:16.359728Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2baa2d2-4e5de025-fa253f78-caf3423e_0] Write session will restart in 0.008065s
2025-10-09T21:49:16.359755Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2baa2d2-4e5de025-fa253f78-caf3423e_0] Write session: Do CDS request
2025-10-09T21:49:16.359761Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2baa2d2-4e5de025-fa253f78-caf3423e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:16.368527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a2baa2d2-4e5de025-fa253f78-caf3423e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:16.368613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:16.368634Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1465
2025-10-09T21:49:16.368754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1465 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:16.368778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1465 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:16.368785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1465 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:16.368789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:16.369237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1465 sessionId:  describe result for acl check
2025-10-09T21:49:16.369275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:16.369281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:16.369283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:16.369290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342398707139537:2907] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:16.369873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342398707139537:2907] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:16.372241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342398707139537:2907] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:16.375778Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342398707139537:2907] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:16.375792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342398707139537:2907] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:16.375796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342398707139537:2907] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:16.375805Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1465 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:16.380489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:16.382914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:16.383114Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:16.383127Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:16.383139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:16.383195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:16.383241Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342398345314388:5749]
2025-10-09T21:49:16.383840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:16.384174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:16.384221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:16.384680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:16.384759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:16.384768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:16.384769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:16.384772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:16.384773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:16.384774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:16.384776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.384777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.384778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.384784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:16.384796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:16.384800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:16.384805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:16.384809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:16.384813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:16.384817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.384821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.384824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.384833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:16.384842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:16.384848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:16.384853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:16.384858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:16.384863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:16.384868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:16.384873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:16.384877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:16.384881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:16.384885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:16.384889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:16.384894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.384899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.384903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.384907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.384913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.384918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.384946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:16.391627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:16.392994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:16.393009Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:16.393011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:16.393016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:16.393023Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1464 [2:7559342398345314388:5749]
2025-10-09T21:49:16.393037Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:16.393057Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:16.393064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:16.393068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:16.393070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:16.393087Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:16.393098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:16.393105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:16.393165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.393172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.393174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.393177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:16.393246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:16.393254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.393267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.393280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.393297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:16.393319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342398345314383:5749]
2025-10-09T21:49:16.393326Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342398345314383:5749]
2025-10-09T21:49:16.393328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342398345314383:5749]
2025-10-09T21:49:16.393337Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:16.393346Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:16.393352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:16.393352Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:16.393354Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:16.393368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:16.393376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:16.393388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342398707139697:2907], now have 1 active actors on pipe
2025-10-09T21:49:16.393392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342398707139512:2432], now have 1 active actors on pipe
2025-10-09T21:49:16.393424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342398345314383:5749]
2025-10-09T21:49:16.393435Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:16.393446Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:16.393524Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1464
2025-10-09T21:49:16.393527Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1464
2025-10-09T21:49:16.393590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:16.393600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:16.393644Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c4f7dc02-d9f7f08f-cac79c04-c2764a8b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:16.393656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:16.393660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:16.393666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:16.393678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:16.393695Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:16.393710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:16.393933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:16.393946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:16.393968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:16.394126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1465 partition: 0 MaxSeqNo: 25 sessionId: src_id|c4f7dc02-d9f7f08f-cac79c04-c2764a8b_0
2025-10-09T21:49:16.394463Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2baa2d2-4e5de025-fa253f78-caf3423e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c4f7dc02-d9f7f08f-cac79c04-c2764a8b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:16.396902Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4f7dc02-d9f7f08f-cac79c04-c2764a8b_0] Write session: try to update token
2025-10-09T21:49:16.396920Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4f7dc02-d9f7f08f-cac79c04-c2764a8b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:16.409407Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1465 sessionId: src_id|c4f7dc02-d9f7f08f-cac79c04-c2764a8b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:16.410556Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:16.410577Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:16.410586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342398345314383:5749]
2025-10-09T21:49:16.410633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:16.410724Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:16.410800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:16.410933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:16.411071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:16.411235Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:16.411412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:16.411447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:16.411669Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:16.411839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:16.411853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:16.411884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:16.411977Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:16.418056Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:16.418070Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:16.418073Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:16.418075Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:16.418084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342398345314383:5749]
2025-10-09T21:49:16.418088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342398345314383:5749]
2025-10-09T21:49:16.418092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342398345314383:5749]
2025-10-09T21:49:16.418128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:16.418128Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:16.418133Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:16.418136Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:16.418139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.418142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:16.418267Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:16.459495Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342398707139826:3441]
2025-10-09T21:49:16.459698Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 1464, pipe: [1:7559342398707139826:3441]
2025-10-09T21:49:16.565327Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:16.572209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:16.572233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:16.572390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:16.572448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:16.572512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:16.572577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:16.572620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:16.572624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:16.572683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:16.572733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:16.572786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:16.572851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:16.573300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:16.573311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:16.573368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:16.573420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:16.573473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:16.573527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:16.573565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:16.573572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:16.573648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:16.573711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:16.573766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:16.573822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:16.573864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:16.573873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:16.573930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:16.573982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:16.574054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:16.574109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:16.574148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:16.574154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:16.574210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:16.574264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:16.574332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:16.574392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:16.574505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:16.574515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:16.574566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:16.574617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:16.574669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:16.574731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:16.574779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:16.574787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:16.574838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:16.574889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:16.574954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:16.575012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:16.575057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:16.575063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:16.575116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:16.575166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:16.575220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:16.575276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:16.575315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:16.575322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:16.575376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:16.575428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:16.575490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:16.575549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:16.575589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:16.575596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:16.575634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342398707139826:3441], now have 1 active actors on pipe
2025-10-09T21:49:16.575842Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:16.575860Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:16.575873Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:16.575892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1465 reason: error at write: Disconnected sessionId: src_id|c4f7dc02-d9f7f08f-cac79c04-c2764a8b_0
2025-10-09T21:49:16.575923Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342398707139826:3441]
2025-10-09T21:49:16.576069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1465 sessionId: src_id|c4f7dc02-d9f7f08f-cac79c04-c2764a8b_0 is DEAD
2025-10-09T21:49:16.576153Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:16.576212Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4f7dc02-d9f7f08f-cac79c04-c2764a8b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:16.576225Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4f7dc02-d9f7f08f-cac79c04-c2764a8b_0] Write session will restart in 0.005435s
2025-10-09T21:49:16.576255Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4f7dc02-d9f7f08f-cac79c04-c2764a8b_0] Write session: Do CDS request
2025-10-09T21:49:16.576262Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4f7dc02-d9f7f08f-cac79c04-c2764a8b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:16.582488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4f7dc02-d9f7f08f-cac79c04-c2764a8b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:16.582555Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:16.582587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1466
2025-10-09T21:49:16.582727Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1466 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:16.582752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1466 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:16.582758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1466 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:16.582761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:16.583206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1466 sessionId:  describe result for acl check
2025-10-09T21:49:16.583243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:16.583252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:16.583253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:16.583259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342398707139853:2895] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:16.583800Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342398707139853:2895] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:16.586608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342398707139853:2895] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:16.590195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342398707139853:2895] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:16.590211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342398707139853:2895] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:16.590215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342398707139853:2895] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:16.590223Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1466 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:16.596997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:16.599633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:16.599827Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:16.599840Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:16.599852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:16.599907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:16.599956Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342398345314718:5721]
2025-10-09T21:49:16.600555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:16.600837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:16.600884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:16.601301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:16.601375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:16.601382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:16.601383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:16.601385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:16.601386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:16.601387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:16.601388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.601389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.601390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.601395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:16.601407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:16.601411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:16.601415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:16.601419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:16.601422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:16.601426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.601430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.601435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.601441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:16.601449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:16.601456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:16.601462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:16.601466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:16.601472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:16.601476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:16.601482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:16.601486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:16.601491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:16.601495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:16.601500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:16.601504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.601510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.601514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.601519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.601523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.601530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.601559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:16.607989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:16.609081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:16.609096Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:16.609099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:16.609103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:16.609109Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1465 [2:7559342398345314718:5721]
2025-10-09T21:49:16.609122Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:16.609142Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:16.609146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:16.609149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:16.609151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:16.609168Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:16.609180Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:16.609188Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:16.609244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.609256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.609258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.609261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:16.609330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:16.609340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.609353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.609370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.609384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:16.609413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342398707139838:2432], now have 1 active actors on pipe
2025-10-09T21:49:16.609423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342398707139979:2895], now have 1 active actors on pipe
2025-10-09T21:49:16.609431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342398345314713:5721]
2025-10-09T21:49:16.609434Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342398345314713:5721]
2025-10-09T21:49:16.609436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342398345314713:5721]
2025-10-09T21:49:16.609451Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:16.609466Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:16.609484Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:16.609486Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:16.609488Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:16.609466Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1465
2025-10-09T21:49:16.609485Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1465
2025-10-09T21:49:16.609489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:16.609490Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:16.609514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342398345314713:5721]
2025-10-09T21:49:16.609517Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:16.609530Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:16.609539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:16.609544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:16.609580Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4c2ef26-3b56fea8-ce5ee187-5600d17c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:16.609594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:16.609599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:16.609602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:16.609624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:16.609640Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:16.609659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:16.609804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:16.609813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:16.609828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:16.609932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1466 partition: 0 MaxSeqNo: 25 sessionId: src_id|4c2ef26-3b56fea8-ce5ee187-5600d17c_0
2025-10-09T21:49:16.610182Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4f7dc02-d9f7f08f-cac79c04-c2764a8b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4c2ef26-3b56fea8-ce5ee187-5600d17c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:16.612570Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4c2ef26-3b56fea8-ce5ee187-5600d17c_0] Write session: try to update token
2025-10-09T21:49:16.612594Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4c2ef26-3b56fea8-ce5ee187-5600d17c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:16.623740Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:16.623756Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:16.623758Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:16.623760Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:16.623768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342398345314713:5721]
2025-10-09T21:49:16.623772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342398345314713:5721]
2025-10-09T21:49:16.623775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342398345314713:5721]
2025-10-09T21:49:16.623808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:16.623815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.623817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:16.623827Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:16.623832Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:16.623835Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:16.623945Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:16.625358Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1466 sessionId: src_id|4c2ef26-3b56fea8-ce5ee187-5600d17c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:16.627735Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:16.676079Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342398707140138:3441]
2025-10-09T21:49:16.676402Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 1465, pipe: [1:7559342398707140138:3441]
2025-10-09T21:49:16.773819Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:16.774216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:16.774231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:16.774270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342398707140138:3441], now have 1 active actors on pipe
2025-10-09T21:49:16.774483Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:16.774511Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342398707140138:3441]
2025-10-09T21:49:16.774536Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:16.774538Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:16.774635Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1466 reason: error at write: Disconnected sessionId: src_id|4c2ef26-3b56fea8-ce5ee187-5600d17c_0
2025-10-09T21:49:16.774665Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:16.774794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1466 sessionId: src_id|4c2ef26-3b56fea8-ce5ee187-5600d17c_0 is DEAD
2025-10-09T21:49:16.774941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c2ef26-3b56fea8-ce5ee187-5600d17c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:16.774953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c2ef26-3b56fea8-ce5ee187-5600d17c_0] Write session will restart in 0.008882s
2025-10-09T21:49:16.774981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c2ef26-3b56fea8-ce5ee187-5600d17c_0] Write session: Do CDS request
2025-10-09T21:49:16.774987Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c2ef26-3b56fea8-ce5ee187-5600d17c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:16.785460Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4c2ef26-3b56fea8-ce5ee187-5600d17c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:16.785538Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:16.785564Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1467
2025-10-09T21:49:16.785720Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1467 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:16.785764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1467 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:16.785773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1467 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:16.785777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:16.786266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1467 sessionId:  describe result for acl check
2025-10-09T21:49:16.786303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:16.786310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:16.786312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:16.786318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342398707140196:2877] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:16.786852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342398707140196:2877] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:16.790162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342398707140196:2877] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:16.793233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342398707140196:2877] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:16.793246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342398707140196:2877] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:16.793250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342398707140196:2877] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:16.793258Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1467 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:16.795741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:16.798320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:16.798500Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:16.798510Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:16.798521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:16.798570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:16.798607Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342398345315014:5727]
2025-10-09T21:49:16.799199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:16.799503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:16.799543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:16.799815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:16.799873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:16.799878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:16.799880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:16.799881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:16.799882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:16.799884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:16.799885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.799886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.799887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.799892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:16.799903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:16.799907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:16.799911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:16.799913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:16.799915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:16.799917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.799919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.799921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.799927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:16.799932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:16.799936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:16.799939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:16.799941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:16.799944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:16.799947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:16.799950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:16.799953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:16.799959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:16.799963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:16.799970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:16.799974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.799980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.799984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.799989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.799993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.799998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.800028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:16.806411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:16.807479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:16.807493Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:16.807494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:16.807497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:16.807503Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1466 [2:7559342398345315014:5727]
2025-10-09T21:49:16.807512Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:16.807527Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:16.807529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:16.807531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:16.807532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:16.807545Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:16.807548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:16.807555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:16.807597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.807603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.807604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.807606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:16.807660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:16.807666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.807673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:16.807680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:16.807689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:16.807705Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342398345315009:5727]
2025-10-09T21:49:16.807710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342398345315009:5727]
2025-10-09T21:49:16.807711Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342398345315009:5727]
2025-10-09T21:49:16.807717Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:16.807729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:16.807729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:16.807732Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:16.807734Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:16.807741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:16.807743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:16.807752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342398707140144:2432], now have 1 active actors on pipe
2025-10-09T21:49:16.807754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342398707140398:2877], now have 1 active actors on pipe
2025-10-09T21:49:16.807777Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342398345315009:5727]
2025-10-09T21:49:16.807782Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:16.807809Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:16.807814Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1466
2025-10-09T21:49:16.807829Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1466
2025-10-09T21:49:16.808435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:16.808448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:16.808479Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8b33d3cf-f2dc0f3a-90c660eb-1a8ba142_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:16.808490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:16.808494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:16.808497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:16.808506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:16.808517Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:16.808533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:16.808759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:16.808780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:16.808802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:16.808901Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1467 partition: 0 MaxSeqNo: 25 sessionId: src_id|8b33d3cf-f2dc0f3a-90c660eb-1a8ba142_0
2025-10-09T21:49:16.809229Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c2ef26-3b56fea8-ce5ee187-5600d17c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8b33d3cf-f2dc0f3a-90c660eb-1a8ba142_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:16.811608Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b33d3cf-f2dc0f3a-90c660eb-1a8ba142_0] Write session: try to update token
2025-10-09T21:49:16.811625Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b33d3cf-f2dc0f3a-90c660eb-1a8ba142_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:16.822302Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:16.822321Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:16.822323Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:16.822325Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:16.822334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342398345315009:5727]
2025-10-09T21:49:16.822340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342398345315009:5727]
2025-10-09T21:49:16.822344Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342398345315009:5727]
2025-10-09T21:49:16.822390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:16.822390Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:16.822397Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:16.822400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:16.822401Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:16.822403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:16.822534Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:16.824811Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1467 sessionId: src_id|8b33d3cf-f2dc0f3a-90c660eb-1a8ba142_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:16.827327Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:16.874779Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342398707140447:3441]
2025-10-09T21:49:16.875115Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 1466, pipe: [1:7559342398707140447:3441]
2025-10-09T21:49:16.976809Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:16.977044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:16.977062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:16.977105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342398707140447:3441], now have 1 active actors on pipe
2025-10-09T21:49:16.977346Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:16.977366Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:16.977371Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:16.977412Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342398707140447:3441]
2025-10-09T21:49:16.977460Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1467 reason: error at write: Disconnected sessionId: src_id|8b33d3cf-f2dc0f3a-90c660eb-1a8ba142_0
2025-10-09T21:49:16.977514Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:16.977631Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1467 sessionId: src_id|8b33d3cf-f2dc0f3a-90c660eb-1a8ba142_0 is DEAD
2025-10-09T21:49:16.977801Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b33d3cf-f2dc0f3a-90c660eb-1a8ba142_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:16.977814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b33d3cf-f2dc0f3a-90c660eb-1a8ba142_0] Write session will restart in 0.009964s
2025-10-09T21:49:16.977840Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b33d3cf-f2dc0f3a-90c660eb-1a8ba142_0] Write session: Do CDS request
2025-10-09T21:49:16.977846Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b33d3cf-f2dc0f3a-90c660eb-1a8ba142_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:16.988529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b33d3cf-f2dc0f3a-90c660eb-1a8ba142_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:16.988556Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:16.988574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1468
2025-10-09T21:49:16.988762Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1468 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:16.988795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1468 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:16.988802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1468 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:16.988808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:16.989250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1468 sessionId:  describe result for acl check
2025-10-09T21:49:16.989294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:16.989300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:16.989302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:16.989309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342398707140524:2875] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:16.989943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342398707140524:2875] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:16.992742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342398707140524:2875] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:16.995544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342398707140524:2875] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:16.995561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342398707140524:2875] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:16.995566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342398707140524:2875] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:16.995575Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1468 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:16.997540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:17.000106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:17.000297Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:17.000308Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:17.000320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:17.000373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:17.000417Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342402640282604:5725]
2025-10-09T21:49:17.000993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:17.001263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:17.001305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:17.001629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:17.001681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.001687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.001688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.001689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.001690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.001691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.001693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.001695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.001696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.001701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.001712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.001716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.001720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.001724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.001728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.001732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.001736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.001740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.001746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.001755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.001761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.001771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.001776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.001785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.001789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.001795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.001803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.001808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.001817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.001823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.001832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.001837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.001841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.001846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.001851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.001856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.001885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:17.008359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:17.009784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:17.009799Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:17.009801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:17.009805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:17.009813Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1467 [2:7559342402640282604:5725]
2025-10-09T21:49:17.009826Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:17.009845Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:17.009852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:17.009856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:17.009859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:17.009874Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:17.009886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:17.009897Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:17.009958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.009968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.009970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.009972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:17.010054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:17.010064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.010077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.010090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.010107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:17.010133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342398707140723:2875], now have 1 active actors on pipe
2025-10-09T21:49:17.010142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342398707140452:2432], now have 1 active actors on pipe
2025-10-09T21:49:17.010150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342398345315303:5725]
2025-10-09T21:49:17.010157Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342398345315303:5725]
2025-10-09T21:49:17.010159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342398345315303:5725]
2025-10-09T21:49:17.010176Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:17.010190Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:17.010193Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:17.010194Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:17.010200Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:17.010216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:17.010219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:17.010251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342398345315303:5725]
2025-10-09T21:49:17.010259Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:17.010230Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1467
2025-10-09T21:49:17.010269Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:17.010230Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1467
2025-10-09T21:49:17.010290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.010302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.010336Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f7b6db7d-3653db56-e1ddce21-4c1b8f4f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:17.010348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:17.010357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:17.010360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:17.010370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:17.010385Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:17.010406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:17.010555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.010564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.010579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:17.010765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1468 partition: 0 MaxSeqNo: 25 sessionId: src_id|f7b6db7d-3653db56-e1ddce21-4c1b8f4f_0
2025-10-09T21:49:17.011136Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b33d3cf-f2dc0f3a-90c660eb-1a8ba142_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f7b6db7d-3653db56-e1ddce21-4c1b8f4f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:17.013623Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7b6db7d-3653db56-e1ddce21-4c1b8f4f_0] Write session: try to update token
2025-10-09T21:49:17.013643Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7b6db7d-3653db56-e1ddce21-4c1b8f4f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:17.024999Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:17.025018Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:17.025021Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:17.025023Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:17.025032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342398345315303:5725]
2025-10-09T21:49:17.025037Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342398345315303:5725]
2025-10-09T21:49:17.025040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342398345315303:5725]
2025-10-09T21:49:17.025086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:17.025093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.025096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:17.025099Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:17.025108Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:17.025112Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:17.025230Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:17.027577Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1468 sessionId: src_id|f7b6db7d-3653db56-e1ddce21-4c1b8f4f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:17.029966Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:17.077504Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342403002108048:3441]
2025-10-09T21:49:17.077814Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 1467, pipe: [1:7559342403002108048:3441]
2025-10-09T21:49:17.181287Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:17.181778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.181797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.181844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342403002108048:3441], now have 1 active actors on pipe
2025-10-09T21:49:17.182072Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342403002108048:3441]
2025-10-09T21:49:17.182090Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:17.182094Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:17.182107Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:17.182192Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1468 reason: error at write: Disconnected sessionId: src_id|f7b6db7d-3653db56-e1ddce21-4c1b8f4f_0
2025-10-09T21:49:17.182393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1468 sessionId: src_id|f7b6db7d-3653db56-e1ddce21-4c1b8f4f_0 is DEAD
2025-10-09T21:49:17.182547Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7b6db7d-3653db56-e1ddce21-4c1b8f4f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:17.182558Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7b6db7d-3653db56-e1ddce21-4c1b8f4f_0] Write session will restart in 0.008013s
2025-10-09T21:49:17.182579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7b6db7d-3653db56-e1ddce21-4c1b8f4f_0] Write session: Do CDS request
2025-10-09T21:49:17.182588Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7b6db7d-3653db56-e1ddce21-4c1b8f4f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:17.182900Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:17.191546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7b6db7d-3653db56-e1ddce21-4c1b8f4f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:17.191604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:17.191622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1469
2025-10-09T21:49:17.191782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1469 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:17.191827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1469 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:17.191837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1469 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:17.191841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:17.192322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1469 sessionId:  describe result for acl check
2025-10-09T21:49:17.192352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:17.192354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:17.192356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:17.192361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342403002108083:2872] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:17.192920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342403002108083:2872] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:17.195909Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342403002108083:2872] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:17.198717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342403002108083:2872] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:17.198733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342403002108083:2872] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:17.198737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342403002108083:2872] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:17.198745Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1469 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:17.203849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:17.206213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:17.206412Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:17.206424Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:17.206437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:17.206496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:17.206538Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342402640282898:5723]
2025-10-09T21:49:17.207136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:17.207454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:17.207497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:17.207808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:17.207862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.207869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.207870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.207872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.207873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.207874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.207875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.207876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.207878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.207883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.207894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.207898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.207902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.207906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.207910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.207914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.207918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.207922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.207929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.207942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.207949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.207958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.207963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.207967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.207971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.207976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.207981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.207986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.207990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.207995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.208000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.208005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.208009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.208015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.208024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.208030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.208060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:17.214348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:17.215310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:17.215322Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:17.215324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:17.215327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:17.215333Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1468 [2:7559342402640282898:5723]
2025-10-09T21:49:17.215342Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:17.215356Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:17.215361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:17.215363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:17.215365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:17.215376Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:17.215383Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:17.215389Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:17.215429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.215435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.215436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.215438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:17.215503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:17.215511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.215522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.215533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.215550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:17.215575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342403002108246:2872], now have 1 active actors on pipe
2025-10-09T21:49:17.215583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342403002108053:2432], now have 1 active actors on pipe
2025-10-09T21:49:17.215589Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342402640282893:5723]
2025-10-09T21:49:17.215595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342402640282893:5723]
2025-10-09T21:49:17.215597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342402640282893:5723]
2025-10-09T21:49:17.215609Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:17.215629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:17.215643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:17.215644Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:17.215647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:17.215646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:17.215648Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:17.215672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342402640282893:5723]
2025-10-09T21:49:17.215681Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:17.215683Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:17.215650Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1468
2025-10-09T21:49:17.215653Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1468
2025-10-09T21:49:17.215708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.215718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.215753Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|53a879f0-505c9a1d-67ca86c5-4d969048_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:17.215766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:17.215769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:17.215771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:17.215781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:17.215791Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:17.215807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:17.215956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.215964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.215977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:17.216062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1469 partition: 0 MaxSeqNo: 25 sessionId: src_id|53a879f0-505c9a1d-67ca86c5-4d969048_0
2025-10-09T21:49:17.216410Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7b6db7d-3653db56-e1ddce21-4c1b8f4f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|53a879f0-505c9a1d-67ca86c5-4d969048_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:17.218856Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53a879f0-505c9a1d-67ca86c5-4d969048_0] Write session: try to update token
2025-10-09T21:49:17.218877Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53a879f0-505c9a1d-67ca86c5-4d969048_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:17.229933Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:17.229957Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:17.229961Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:17.229962Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:17.229970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342402640282893:5723]
2025-10-09T21:49:17.229973Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342402640282893:5723]
2025-10-09T21:49:17.229977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342402640282893:5723]
2025-10-09T21:49:17.230023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:17.230044Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:17.230051Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:17.230055Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:17.230063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.230067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:17.230201Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:17.231232Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1469 sessionId: src_id|53a879f0-505c9a1d-67ca86c5-4d969048_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:17.233555Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:17.282297Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342403002108370:3441]
2025-10-09T21:49:17.282592Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 1468, pipe: [1:7559342403002108370:3441]
2025-10-09T21:49:17.326398Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: CLIENT_DEADLINE_EXCEEDED. Description: [ { <main>: Error: GRpc error: (4): Deadline Exceeded } { <main>: Error: Grpc error response on endpoint localhost:14704 } ]
2025-10-09T21:49:17.381458Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:17.381729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.381740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.381784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342403002108370:3441], now have 1 active actors on pipe
2025-10-09T21:49:17.382050Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:17.382055Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342403002108370:3441]
2025-10-09T21:49:17.382081Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:17.382084Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:17.382165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1469 reason: error at write: Disconnected sessionId: src_id|53a879f0-505c9a1d-67ca86c5-4d969048_0
2025-10-09T21:49:17.382360Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1469 sessionId: src_id|53a879f0-505c9a1d-67ca86c5-4d969048_0 is DEAD
2025-10-09T21:49:17.382470Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:17.382513Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53a879f0-505c9a1d-67ca86c5-4d969048_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:17.382520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53a879f0-505c9a1d-67ca86c5-4d969048_0] Write session will restart in 0.006990s
2025-10-09T21:49:17.382547Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53a879f0-505c9a1d-67ca86c5-4d969048_0] Write session: Do CDS request
2025-10-09T21:49:17.382556Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53a879f0-505c9a1d-67ca86c5-4d969048_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:17.390510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53a879f0-505c9a1d-67ca86c5-4d969048_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:17.390594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:17.390613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1470
2025-10-09T21:49:17.390734Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1470 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:17.390767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1470 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:17.390782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1470 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:17.390786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:17.391241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1470 sessionId:  describe result for acl check
2025-10-09T21:49:17.391278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:17.391285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:17.391287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:17.391294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342403002108389:2867] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:17.391866Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342403002108389:2867] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:17.395459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342403002108389:2867] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:17.399360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342403002108389:2867] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:17.399374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342403002108389:2867] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:17.399378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342403002108389:2867] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:17.399387Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1470 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:17.404529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:17.407491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:17.407704Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:17.407714Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:17.407727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:17.407782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:17.407821Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342402640283193:5716]
2025-10-09T21:49:17.408345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:17.408641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:17.408677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:17.408940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:17.408988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.408994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.408995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.408996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.408998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.408999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.409000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.409001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.409002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.409007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.409018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.409021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.409025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.409029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.409033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.409037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.409040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.409043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.409050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.409057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.409063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.409068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.409074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.409079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.409083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.409088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.409093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.409098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.409102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.409107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.409113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.409124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.409133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.409150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.409160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.409166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.409193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:17.415297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:17.416503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:17.416514Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:17.416516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:17.416519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:17.416525Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1469 [2:7559342402640283193:5716]
2025-10-09T21:49:17.416538Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:17.416555Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:17.416563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:17.416566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:17.416568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:17.416584Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:17.416596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:17.416602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:17.416654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.416661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.416663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.416666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:17.416728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:17.416734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.416743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.416749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.416758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:17.416776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342402640283188:5716]
2025-10-09T21:49:17.416781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342402640283188:5716]
2025-10-09T21:49:17.416783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342402640283188:5716]
2025-10-09T21:49:17.416789Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:17.416800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342403002108553:2867], now have 1 active actors on pipe
2025-10-09T21:49:17.416802Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:17.416806Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:17.416806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342403002108373:2432], now have 1 active actors on pipe
2025-10-09T21:49:17.416808Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:17.416814Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:17.416822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:17.416824Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:17.416840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342402640283188:5716]
2025-10-09T21:49:17.416845Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:17.416850Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:17.416865Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1469
2025-10-09T21:49:17.416878Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1469
2025-10-09T21:49:17.416951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.416961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.416986Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|48d1f285-b6661a6e-3c190ec3-5ee52d9d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:17.416995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:17.416998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:17.417000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:17.417008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:17.417017Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:17.417026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:17.417208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.417215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.417225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:17.417315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1470 partition: 0 MaxSeqNo: 25 sessionId: src_id|48d1f285-b6661a6e-3c190ec3-5ee52d9d_0
2025-10-09T21:49:17.417680Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53a879f0-505c9a1d-67ca86c5-4d969048_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|48d1f285-b6661a6e-3c190ec3-5ee52d9d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:17.420154Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48d1f285-b6661a6e-3c190ec3-5ee52d9d_0] Write session: try to update token
2025-10-09T21:49:17.420173Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48d1f285-b6661a6e-3c190ec3-5ee52d9d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:17.430679Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:17.430696Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:17.430699Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:17.430700Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:17.430710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342402640283188:5716]
2025-10-09T21:49:17.430715Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342402640283188:5716]
2025-10-09T21:49:17.430719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342402640283188:5716]
2025-10-09T21:49:17.430764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:17.430763Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:17.430770Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:17.430773Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:17.430778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.430781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:17.430905Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:17.432797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1470 sessionId: src_id|48d1f285-b6661a6e-3c190ec3-5ee52d9d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:17.434988Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:17.481974Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342403002108677:3441]
2025-10-09T21:49:17.482181Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 1469, pipe: [1:7559342403002108677:3441]
2025-10-09T21:49:17.580381Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:17.580839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.580851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.580889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342403002108677:3441], now have 1 active actors on pipe
2025-10-09T21:49:17.581024Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:17.581086Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:17.581089Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342403002108677:3441]
2025-10-09T21:49:17.581101Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:17.581107Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:17.581159Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1470 reason: error at write: Disconnected sessionId: src_id|48d1f285-b6661a6e-3c190ec3-5ee52d9d_0
2025-10-09T21:49:17.581324Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1470 sessionId: src_id|48d1f285-b6661a6e-3c190ec3-5ee52d9d_0 is DEAD
2025-10-09T21:49:17.581481Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48d1f285-b6661a6e-3c190ec3-5ee52d9d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:17.581492Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48d1f285-b6661a6e-3c190ec3-5ee52d9d_0] Write session will restart in 0.006343s
2025-10-09T21:49:17.581520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48d1f285-b6661a6e-3c190ec3-5ee52d9d_0] Write session: Do CDS request
2025-10-09T21:49:17.581525Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48d1f285-b6661a6e-3c190ec3-5ee52d9d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:17.589459Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48d1f285-b6661a6e-3c190ec3-5ee52d9d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:17.589535Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:17.589549Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1471
2025-10-09T21:49:17.589679Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1471 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:17.589723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1471 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:17.589733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1471 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:17.589737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:17.590180Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1471 sessionId:  describe result for acl check
2025-10-09T21:49:17.590217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:17.590224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:17.590226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:17.590232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342403002108719:2845] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:17.590750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342403002108719:2845] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:17.593814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342403002108719:2845] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:17.598197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342403002108719:2845] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:17.598212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342403002108719:2845] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:17.598216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342403002108719:2845] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:17.598224Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1471 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:17.602818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:17.605094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:17.605256Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:17.605265Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:17.605277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:17.605327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:17.605355Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342402640283527:5704]
2025-10-09T21:49:17.605918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:17.606171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:17.606211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:17.606501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:17.606554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.606560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.606561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.606563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.606564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.606565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.606566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.606567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.606568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.606572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.606581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.606585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.606589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.606593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.606597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.606600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.606604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.606608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.606614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.606622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.606628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.606632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.606637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.606641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.606645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.606650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.606654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.606658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.606662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.606667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.606671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.606681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.606685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.606690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.606696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.606704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.606728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:17.612666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:17.614009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:17.614022Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:17.614024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:17.614043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:17.614050Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1470 [2:7559342402640283527:5704]
2025-10-09T21:49:17.614060Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:17.614075Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:17.614084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:17.614087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:17.614089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:17.614105Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:17.614111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:17.614117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:17.614173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.614180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.614182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.614184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:17.614246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:17.614254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.614265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.614280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.614300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:17.614323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342403002108704:2432], now have 1 active actors on pipe
2025-10-09T21:49:17.614327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342403002108894:2845], now have 1 active actors on pipe
2025-10-09T21:49:17.614333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342402640283522:5704]
2025-10-09T21:49:17.614335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342402640283522:5704]
2025-10-09T21:49:17.614337Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342402640283522:5704]
2025-10-09T21:49:17.614349Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:17.614367Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:17.614379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:17.614382Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:17.614383Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:17.614384Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:17.614386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:17.614407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342402640283522:5704]
2025-10-09T21:49:17.614418Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:17.614423Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:17.614408Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1470
2025-10-09T21:49:17.614409Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1470
2025-10-09T21:49:17.614453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.614462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.614492Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|efcab636-94fb9df2-64570a42-55720ffc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:17.614503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:17.614506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:17.614508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:17.614518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:17.614530Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:17.614542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:17.614716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.614728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.614747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:17.614855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1471 partition: 0 MaxSeqNo: 25 sessionId: src_id|efcab636-94fb9df2-64570a42-55720ffc_0
2025-10-09T21:49:17.615183Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48d1f285-b6661a6e-3c190ec3-5ee52d9d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|efcab636-94fb9df2-64570a42-55720ffc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:17.617644Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|efcab636-94fb9df2-64570a42-55720ffc_0] Write session: try to update token
2025-10-09T21:49:17.617660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|efcab636-94fb9df2-64570a42-55720ffc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:17.627722Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:17.627738Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:17.627741Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:17.627743Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:17.627751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342402640283522:5704]
2025-10-09T21:49:17.627756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342402640283522:5704]
2025-10-09T21:49:17.627760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342402640283522:5704]
2025-10-09T21:49:17.627801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:17.627807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.627810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:17.627811Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:17.627817Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:17.627820Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:17.627933Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:17.630364Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1471 sessionId: src_id|efcab636-94fb9df2-64570a42-55720ffc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:17.632405Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:17.681147Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342403002109006:3441]
2025-10-09T21:49:17.681468Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 1470, pipe: [1:7559342403002109006:3441]
2025-10-09T21:49:17.776824Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:17.777297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.777311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.777355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342403002109006:3441], now have 1 active actors on pipe
2025-10-09T21:49:17.777570Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:17.777579Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:17.777597Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342403002109006:3441]
2025-10-09T21:49:17.777634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:17.777713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1471 reason: status is not ok: tablet will be restarted right now sessionId: src_id|efcab636-94fb9df2-64570a42-55720ffc_0
2025-10-09T21:49:17.777889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1471 sessionId: src_id|efcab636-94fb9df2-64570a42-55720ffc_0 is DEAD
2025-10-09T21:49:17.778070Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efcab636-94fb9df2-64570a42-55720ffc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:17.778089Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efcab636-94fb9df2-64570a42-55720ffc_0] Write session will restart in 0.006792s
2025-10-09T21:49:17.778128Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efcab636-94fb9df2-64570a42-55720ffc_0] Write session: Do CDS request
2025-10-09T21:49:17.778136Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efcab636-94fb9df2-64570a42-55720ffc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:17.786505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|efcab636-94fb9df2-64570a42-55720ffc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:17.786532Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:17.786550Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1472
2025-10-09T21:49:17.786721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1472 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:17.786775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1472 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:17.786786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1472 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:17.786790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:17.787452Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1472 sessionId:  describe result for acl check
2025-10-09T21:49:17.787487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:17.787493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:17.787495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:17.787501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342403002109033:2840] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:17.788007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342403002109033:2840] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:17.790712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342403002109033:2840] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:17.793973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342403002109033:2840] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:17.794000Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342403002109033:2840] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:17.794003Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342403002109033:2840] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:17.794009Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1472 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:17.798913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:17.801302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:17.801488Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:17.801501Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:17.801515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:17.801576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:17.801629Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342402640283828:5692]
2025-10-09T21:49:17.802237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:17.802585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:17.802642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:17.802997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:17.803062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.803070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.803072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.803073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.803074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.803075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.803076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.803078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.803079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.803084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.803095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.803100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.803104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.803108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.803112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.803116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.803120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.803124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.803130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.803139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.803145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.803151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.803161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.803166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.803171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.803176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.803180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.803197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.803206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.803211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.803218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.803227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.803231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.803236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.803241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.803251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.803281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:17.809537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:17.810675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:17.810688Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:17.810691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:17.810695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:17.810701Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1471 [2:7559342402640283828:5692]
2025-10-09T21:49:17.810711Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:17.810729Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:17.810734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:17.810737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:17.810740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:17.810753Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:17.810761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:17.810766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:17.810807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.810811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.810813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.810814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:17.810863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:17.810868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.810875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.810881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.810891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:17.810905Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342402640283823:5692]
2025-10-09T21:49:17.810914Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342402640283823:5692]
2025-10-09T21:49:17.810915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342402640283823:5692]
2025-10-09T21:49:17.810922Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:17.810932Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:17.810934Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:17.810936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:17.810936Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:17.810950Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:17.810953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:17.810963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342403002109009:2432], now have 1 active actors on pipe
2025-10-09T21:49:17.810967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342403002109197:2840], now have 1 active actors on pipe
2025-10-09T21:49:17.810986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342402640283823:5692]
2025-10-09T21:49:17.810993Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:17.811004Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:17.811016Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1471
2025-10-09T21:49:17.811119Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1471
2025-10-09T21:49:17.811158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.811165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.811189Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a4cb6e98-3a0fe2b9-70662f5f-53c5bc08_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:17.811200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:17.811204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:17.811209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:17.811216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:17.811224Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:17.811245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:17.811399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.811406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.811419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:17.811499Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1472 partition: 0 MaxSeqNo: 25 sessionId: src_id|a4cb6e98-3a0fe2b9-70662f5f-53c5bc08_0
2025-10-09T21:49:17.811814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efcab636-94fb9df2-64570a42-55720ffc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a4cb6e98-3a0fe2b9-70662f5f-53c5bc08_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:17.814140Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a4cb6e98-3a0fe2b9-70662f5f-53c5bc08_0] Write session: try to update token
2025-10-09T21:49:17.814156Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a4cb6e98-3a0fe2b9-70662f5f-53c5bc08_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:17.825234Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:17.825247Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:17.825249Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:17.825250Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:17.825261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342402640283823:5692]
2025-10-09T21:49:17.825264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342402640283823:5692]
2025-10-09T21:49:17.825266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342402640283823:5692]
2025-10-09T21:49:17.825298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:17.825304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.825306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:17.825314Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:17.825320Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:17.825324Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:17.825419Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:17.826089Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1472 sessionId: src_id|a4cb6e98-3a0fe2b9-70662f5f-53c5bc08_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:17.828385Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:17.877758Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342403002109318:3441]
2025-10-09T21:49:17.878060Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 1471, pipe: [1:7559342403002109318:3441]
2025-10-09T21:49:17.961889Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:17.962337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.962350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.962387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342403002109318:3441], now have 1 active actors on pipe
2025-10-09T21:49:17.962604Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:17.962632Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:17.962639Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:17.962643Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:17.962692Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342403002109318:3441]
2025-10-09T21:49:17.962717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1472 reason: error at write: Disconnected sessionId: src_id|a4cb6e98-3a0fe2b9-70662f5f-53c5bc08_0
2025-10-09T21:49:17.962908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1472 sessionId: src_id|a4cb6e98-3a0fe2b9-70662f5f-53c5bc08_0 is DEAD
2025-10-09T21:49:17.963050Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4cb6e98-3a0fe2b9-70662f5f-53c5bc08_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:17.963058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4cb6e98-3a0fe2b9-70662f5f-53c5bc08_0] Write session will restart in 0.008742s
2025-10-09T21:49:17.963082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4cb6e98-3a0fe2b9-70662f5f-53c5bc08_0] Write session: Do CDS request
2025-10-09T21:49:17.963087Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4cb6e98-3a0fe2b9-70662f5f-53c5bc08_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:17.972447Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a4cb6e98-3a0fe2b9-70662f5f-53c5bc08_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:17.972531Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:17.972547Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1473
2025-10-09T21:49:17.972673Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1473 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:17.972706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1473 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:17.972717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1473 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:17.972721Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:17.973323Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1473 sessionId:  describe result for acl check
2025-10-09T21:49:17.973359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:17.973365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:17.973367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:17.973374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342403002109364:2833] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:17.973920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342403002109364:2833] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:17.976375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342403002109364:2833] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:17.979600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342403002109364:2833] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:17.979623Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342403002109364:2833] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:17.979625Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342403002109364:2833] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:17.979632Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1473 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:17.983852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:17.986222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:17.986404Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:17.986415Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:17.986428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:17.986493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:17.986531Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342402640284127:5681]
2025-10-09T21:49:17.987072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:17.987416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:17.987469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:17.987786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:17.987829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.987834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.987836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.987837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.987838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.987839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.987840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.987841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.987842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.987847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.987858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.987862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.987866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.987870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.987874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.987877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.987881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.987885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.987891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.987899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:17.987905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.987910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:17.987916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.987921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:17.987925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.987930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:17.987935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.987940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:17.987944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.987949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:17.987953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.987959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.987962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.987968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.987973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.987978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.988003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:17.993930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:17.994972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:17.994988Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:17.994990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:17.994995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:17.995000Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1472 [2:7559342402640284127:5681]
2025-10-09T21:49:17.995012Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:17.995030Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:17.995033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:17.995037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:17.995039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:17.995055Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:17.995062Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:17.995072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:17.995123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.995129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.995131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.995134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:17.995193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:17.995203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:17.995214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:17.995226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:17.995276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:17.995304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342403002109322:2432], now have 1 active actors on pipe
2025-10-09T21:49:17.995312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342403002109533:2833], now have 1 active actors on pipe
2025-10-09T21:49:17.995319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342402640284122:5681]
2025-10-09T21:49:17.995321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342402640284122:5681]
2025-10-09T21:49:17.995323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342402640284122:5681]
2025-10-09T21:49:17.995292Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1472
2025-10-09T21:49:17.995331Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:17.995293Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1472
2025-10-09T21:49:17.995347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:17.995349Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:17.995352Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:17.995354Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:17.995364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:17.995368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:17.995383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.995390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.995425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342402640284122:5681]
2025-10-09T21:49:17.995437Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:17.995443Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:17.995456Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|633ba980-7996e989-b90c30d3-7c85cfea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:17.995467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:17.995469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:17.995471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:17.995478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:17.995487Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:17.995512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:17.995711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:17.995721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:17.995734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:17.995809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1473 partition: 0 MaxSeqNo: 25 sessionId: src_id|633ba980-7996e989-b90c30d3-7c85cfea_0
2025-10-09T21:49:17.996118Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4cb6e98-3a0fe2b9-70662f5f-53c5bc08_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|633ba980-7996e989-b90c30d3-7c85cfea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:17.998505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|633ba980-7996e989-b90c30d3-7c85cfea_0] Write session: try to update token
2025-10-09T21:49:17.998522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|633ba980-7996e989-b90c30d3-7c85cfea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:18.009531Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:18.009551Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:18.009553Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:18.009555Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:18.009565Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342402640284122:5681]
2025-10-09T21:49:18.009571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342402640284122:5681]
2025-10-09T21:49:18.009578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342402640284122:5681]
2025-10-09T21:49:18.009642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:18.009649Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:18.009655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.009658Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:18.009658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:18.009662Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:18.009796Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:18.010988Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1473 sessionId: src_id|633ba980-7996e989-b90c30d3-7c85cfea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:18.013845Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:18.063097Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342407297076923:3441]
2025-10-09T21:49:18.063360Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 1472, pipe: [1:7559342407297076923:3441]
2025-10-09T21:49:18.159352Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:18.159596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:18.159613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:18.159658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342407297076923:3441], now have 1 active actors on pipe
2025-10-09T21:49:18.159874Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:18.159888Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:18.159891Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:18.159895Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:18.159942Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342407297076923:3441]
2025-10-09T21:49:18.159962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1473 reason: error at write: Disconnected sessionId: src_id|633ba980-7996e989-b90c30d3-7c85cfea_0
2025-10-09T21:49:18.160143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1473 sessionId: src_id|633ba980-7996e989-b90c30d3-7c85cfea_0 is DEAD
2025-10-09T21:49:18.160294Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|633ba980-7996e989-b90c30d3-7c85cfea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:18.160305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|633ba980-7996e989-b90c30d3-7c85cfea_0] Write session will restart in 0.008775s
2025-10-09T21:49:18.160332Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|633ba980-7996e989-b90c30d3-7c85cfea_0] Write session: Do CDS request
2025-10-09T21:49:18.160338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|633ba980-7996e989-b90c30d3-7c85cfea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:18.170566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|633ba980-7996e989-b90c30d3-7c85cfea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:18.170676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:18.170699Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1474
2025-10-09T21:49:18.170888Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1474 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:18.170926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1474 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:18.170932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1474 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:18.170937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:18.171530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1474 sessionId:  describe result for acl check
2025-10-09T21:49:18.171565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:18.171567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:18.171569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:18.171575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342407297076991:2826] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:18.172126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342407297076991:2826] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:18.175583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342407297076991:2826] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:18.178525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342407297076991:2826] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:18.178541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342407297076991:2826] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:18.178545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342407297076991:2826] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:18.178553Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1474 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:18.181000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:18.183801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:18.184032Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:18.184042Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:18.184054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:18.184117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:18.184162Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342406935251722:5689]
2025-10-09T21:49:18.184770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:18.185058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:18.185103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:18.185434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:18.185490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:18.185498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:18.185499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:18.185500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:18.185502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:18.185503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:18.185504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.185505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.185507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.185512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:18.185523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:18.185527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:18.185531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:18.185535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:18.185539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:18.185543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.185547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.185551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.185559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:18.185567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:18.185574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:18.185580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:18.185584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:18.185590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:18.185594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:18.185599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:18.185618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:18.185622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:18.185626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:18.185631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:18.185636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.185646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.185650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.185655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.185664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.185669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.185706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:18.192103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:18.193589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:18.193616Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:18.193619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:18.193624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:18.193631Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1473 [2:7559342406935251722:5689]
2025-10-09T21:49:18.193648Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:18.193669Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:18.193680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:18.193683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:18.193686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:18.193705Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:18.193711Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:18.193719Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:18.193790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.193798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.193800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.193803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:18.193883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:18.193891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.193904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.193919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.193942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:18.193971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342407297077190:2826], now have 1 active actors on pipe
2025-10-09T21:49:18.193979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342407297076928:2432], now have 1 active actors on pipe
2025-10-09T21:49:18.193987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342406935251717:5689]
2025-10-09T21:49:18.193994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342406935251717:5689]
2025-10-09T21:49:18.193996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342406935251717:5689]
2025-10-09T21:49:18.194010Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:18.194045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:18.194049Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:18.194056Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:18.194058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:18.194064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:18.194056Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1473
2025-10-09T21:49:18.194068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:18.194107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342406935251717:5689]
2025-10-09T21:49:18.194112Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:18.194063Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1473
2025-10-09T21:49:18.194117Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:18.194128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:18.194134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:18.194182Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b7e54f1a-f100e8b8-8b5f3d7a-fd217ce3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:18.194194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:18.194198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:18.194201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:18.194211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:18.194222Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:18.194251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:18.194413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:18.194424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:18.194441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:18.194515Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1474 partition: 0 MaxSeqNo: 25 sessionId: src_id|b7e54f1a-f100e8b8-8b5f3d7a-fd217ce3_0
2025-10-09T21:49:18.194865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|633ba980-7996e989-b90c30d3-7c85cfea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b7e54f1a-f100e8b8-8b5f3d7a-fd217ce3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:18.197591Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7e54f1a-f100e8b8-8b5f3d7a-fd217ce3_0] Write session: try to update token
2025-10-09T21:49:18.197620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7e54f1a-f100e8b8-8b5f3d7a-fd217ce3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:18.208315Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:18.208336Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:18.208338Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:18.208340Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:18.208351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342406935251717:5689]
2025-10-09T21:49:18.208356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342406935251717:5689]
2025-10-09T21:49:18.208360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342406935251717:5689]
2025-10-09T21:49:18.208403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:18.208411Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:18.208415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.208418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:18.208420Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:18.208424Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:18.208555Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:18.211446Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1474 sessionId: src_id|b7e54f1a-f100e8b8-8b5f3d7a-fd217ce3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:18.213916Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:18.260228Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342407297077242:3441]
2025-10-09T21:49:18.260504Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 1473, pipe: [1:7559342407297077242:3441]
2025-10-09T21:49:18.368478Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:18.368737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:18.368805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:18.368854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342407297077242:3441], now have 1 active actors on pipe
2025-10-09T21:49:18.369064Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:18.369081Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342407297077242:3441]
2025-10-09T21:49:18.369127Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:18.369141Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:18.369244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1474 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b7e54f1a-f100e8b8-8b5f3d7a-fd217ce3_0
2025-10-09T21:49:18.369416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1474 sessionId: src_id|b7e54f1a-f100e8b8-8b5f3d7a-fd217ce3_0 is DEAD
2025-10-09T21:49:18.369572Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7e54f1a-f100e8b8-8b5f3d7a-fd217ce3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:18.369642Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7e54f1a-f100e8b8-8b5f3d7a-fd217ce3_0] Write session will restart in 0.007604s
2025-10-09T21:49:18.369681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7e54f1a-f100e8b8-8b5f3d7a-fd217ce3_0] Write session: Do CDS request
2025-10-09T21:49:18.369688Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7e54f1a-f100e8b8-8b5f3d7a-fd217ce3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:18.378380Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7e54f1a-f100e8b8-8b5f3d7a-fd217ce3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:18.378512Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:18.378550Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1475
2025-10-09T21:49:18.378676Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1475 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:18.378719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1475 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:18.378732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1475 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:18.378736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:18.379240Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1475 sessionId:  describe result for acl check
2025-10-09T21:49:18.379275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:18.379283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:18.379285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:18.379291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342407297077269:2820] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:18.379919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342407297077269:2820] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:18.382252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342407297077269:2820] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:18.385867Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342407297077269:2820] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:18.385893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342407297077269:2820] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:18.385899Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342407297077269:2820] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:18.385908Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1475 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:18.391750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:18.394511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:18.394748Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:18.394760Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:18.394773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:18.394835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:18.394881Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342406935252022:5687]
2025-10-09T21:49:18.395484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:18.395821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:18.395864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:18.396166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:18.396230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:18.396236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:18.396238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:18.396239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:18.396240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:18.396242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:18.396243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.396244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.396245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.396250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:18.396262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:18.396266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:18.396270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:18.396274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:18.396278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:18.396282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.396286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.396289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.396296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:18.396304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:18.396311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:18.396316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:18.396321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:18.396326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:18.396336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:18.396341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:18.396346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:18.396351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:18.396355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:18.396360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:18.396367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.396374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.396379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.396385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.396389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.396395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.396422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:18.402879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:18.404200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:18.404214Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:18.404216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:18.404219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:18.404224Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1474 [2:7559342406935252022:5687]
2025-10-09T21:49:18.404233Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:18.404247Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:18.404249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:18.404252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:18.404253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:18.404266Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:18.404270Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:18.404276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:18.404320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.404326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.404327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.404329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:18.404382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:18.404388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.404395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.404404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.404420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:18.404442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342406935252017:5687]
2025-10-09T21:49:18.404449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342406935252017:5687]
2025-10-09T21:49:18.404451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342406935252017:5687]
2025-10-09T21:49:18.404460Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:18.404468Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:18.404470Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:18.404472Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:18.404481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342407297077245:2432], now have 1 active actors on pipe
2025-10-09T21:49:18.404490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342407297077421:2820], now have 1 active actors on pipe
2025-10-09T21:49:18.404496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:18.404515Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:18.404517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:18.404544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342406935252017:5687]
2025-10-09T21:49:18.404558Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:18.404557Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1474
2025-10-09T21:49:18.404583Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:18.404560Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1474
2025-10-09T21:49:18.404656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:18.404668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:18.404695Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6f91f01a-f3854f77-1178cbd9-738f2209_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:18.404709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:18.404713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:18.404716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:18.404725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:18.404735Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:18.404753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:18.404890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:18.404897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:18.404907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:18.404972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1475 partition: 0 MaxSeqNo: 25 sessionId: src_id|6f91f01a-f3854f77-1178cbd9-738f2209_0
2025-10-09T21:49:18.405331Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7e54f1a-f100e8b8-8b5f3d7a-fd217ce3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6f91f01a-f3854f77-1178cbd9-738f2209_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:18.407742Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6f91f01a-f3854f77-1178cbd9-738f2209_0] Write session: try to update token
2025-10-09T21:49:18.407761Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6f91f01a-f3854f77-1178cbd9-738f2209_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:18.418429Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:18.418449Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:18.418452Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:18.418454Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:18.418464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342406935252017:5687]
2025-10-09T21:49:18.418469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342406935252017:5687]
2025-10-09T21:49:18.418473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342406935252017:5687]
2025-10-09T21:49:18.418517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:18.418524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.418533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:18.418534Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:18.418539Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:18.418543Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:18.418663Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:18.420226Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1475 sessionId: src_id|6f91f01a-f3854f77-1178cbd9-738f2209_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:18.422613Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:18.469145Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342407297077554:3441]
2025-10-09T21:49:18.469422Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 1474, pipe: [1:7559342407297077554:3441]
2025-10-09T21:49:18.567160Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:18.567520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:18.567534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:18.567567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342407297077554:3441], now have 1 active actors on pipe
2025-10-09T21:49:18.567777Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342407297077554:3441]
2025-10-09T21:49:18.567786Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:18.567790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:18.567807Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:18.567842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1475 reason: error at write: Disconnected sessionId: src_id|6f91f01a-f3854f77-1178cbd9-738f2209_0
2025-10-09T21:49:18.568014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1475 sessionId: src_id|6f91f01a-f3854f77-1178cbd9-738f2209_0 is DEAD
2025-10-09T21:49:18.568124Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:18.568168Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6f91f01a-f3854f77-1178cbd9-738f2209_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:18.568180Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6f91f01a-f3854f77-1178cbd9-738f2209_0] Write session will restart in 0.005077s
2025-10-09T21:49:18.568221Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6f91f01a-f3854f77-1178cbd9-738f2209_0] Write session: Do CDS request
2025-10-09T21:49:18.568227Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6f91f01a-f3854f77-1178cbd9-738f2209_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:18.574411Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6f91f01a-f3854f77-1178cbd9-738f2209_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:18.574540Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:18.574555Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1476
2025-10-09T21:49:18.574782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1476 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:18.574816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1476 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:18.574820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1476 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:18.574824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:18.575143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1476 sessionId:  describe result for acl check
2025-10-09T21:49:18.575166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:18.575171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:18.575172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:18.575176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342407297077597:2802] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:18.575602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342407297077597:2802] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:18.577594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342407297077597:2802] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:18.581031Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342407297077597:2802] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:18.581047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342407297077597:2802] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:18.581050Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342407297077597:2802] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:18.581057Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1476 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:18.589219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:18.592114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:18.592324Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:18.592345Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:18.592358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:18.592418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:18.592458Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342406935252361:5669]
2025-10-09T21:49:18.593059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:18.594254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:18.594312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:18.594595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:18.594640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:18.594646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:18.594647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:18.594648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:18.594649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:18.594650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:18.594651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.594652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.594653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.594658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:18.594669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:18.594672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:18.594677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:18.594680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:18.594683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:18.594687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.594690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.594693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.594702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:18.594710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:18.594717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:18.594723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:18.594728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:18.594733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:18.594736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:18.594741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:18.594746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:18.594750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:18.594754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:18.594758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:18.594763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.594768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.594772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.594776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.594780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.594786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.594816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:18.601056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:18.602519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:18.602533Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:18.602536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:18.602539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:18.602544Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1475 [2:7559342406935252361:5669]
2025-10-09T21:49:18.602554Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:18.602569Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:18.602571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:18.602573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:18.602574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:18.602586Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:18.602589Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:18.602595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:18.602639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.602644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.602645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.602647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:18.602702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:18.602708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.602716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.602722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.602731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:18.602745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342406935252356:5669]
2025-10-09T21:49:18.602750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342406935252356:5669]
2025-10-09T21:49:18.602751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342406935252356:5669]
2025-10-09T21:49:18.602760Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:18.602770Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:18.602773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342407297077693:2802], now have 1 active actors on pipe
2025-10-09T21:49:18.602778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:18.602780Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:18.602782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:18.602785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:18.602787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:18.602791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342407297077582:2432], now have 1 active actors on pipe
2025-10-09T21:49:18.602813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342406935252356:5669]
2025-10-09T21:49:18.602821Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:18.602836Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:18.602850Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1475
2025-10-09T21:49:18.602864Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1475
2025-10-09T21:49:18.602944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:18.602955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:18.602986Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e6fcbee8-2d89d1bd-6c30d3da-7dc670b1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:18.602996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:18.602999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:18.603000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:18.603008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:18.603016Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:18.603031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:18.603197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:18.603211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:18.603233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:18.603345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1476 partition: 0 MaxSeqNo: 25 sessionId: src_id|e6fcbee8-2d89d1bd-6c30d3da-7dc670b1_0
2025-10-09T21:49:18.603701Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6f91f01a-f3854f77-1178cbd9-738f2209_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e6fcbee8-2d89d1bd-6c30d3da-7dc670b1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:18.606095Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e6fcbee8-2d89d1bd-6c30d3da-7dc670b1_0] Write session: try to update token
2025-10-09T21:49:18.606110Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e6fcbee8-2d89d1bd-6c30d3da-7dc670b1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:18.614322Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:18.614340Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:18.614350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342406935252356:5669]
2025-10-09T21:49:18.614393Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:18.614396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:18.614560Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:18.614698Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:18.614843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:18.614982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:18.615167Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:18.615214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:18.619054Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1476 sessionId: src_id|e6fcbee8-2d89d1bd-6c30d3da-7dc670b1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:18.621300Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:18.621759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:18.621780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:18.621821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:18.621927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:18.627114Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:18.627134Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:18.627136Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:18.627138Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:18.627148Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342406935252356:5669]
2025-10-09T21:49:18.627152Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342406935252356:5669]
2025-10-09T21:49:18.627156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342406935252356:5669]
2025-10-09T21:49:18.627200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:18.627208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.627208Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:18.627211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:18.627214Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:18.627217Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:18.627349Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:18.667781Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342407297077889:3441]
2025-10-09T21:49:18.668122Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 1475, pipe: [1:7559342407297077889:3441]
2025-10-09T21:49:18.777214Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:18.784114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:18.784141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:18.784299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:18.784357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:18.784417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:18.784484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:18.784535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:18.784545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:18.784610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:18.784674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:18.784739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:18.784807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:18.784850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:18.784858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:18.784914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:18.784967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:18.785019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:18.785082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:18.785125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:18.785133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:18.785196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:18.785262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:18.785323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:18.785387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:18.785433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:18.785441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:18.785503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:18.785563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:18.785640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:18.785702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:18.785743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:18.785751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:18.785808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:18.785863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:18.785926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:18.785985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:18.786025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:18.786045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:18.786106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:18.786162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:18.786222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:18.786285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:18.786326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:18.786334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:18.786389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:18.786446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:18.786505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:18.786561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:18.786606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:18.786615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:18.786670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:18.786720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:18.786772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:18.786826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:18.786866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:18.786873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:18.786930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:18.786985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:18.787044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:18.787103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:18.787150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:18.787159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:18.787207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342407297077889:3441], now have 1 active actors on pipe
2025-10-09T21:49:18.787425Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:18.787429Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:18.787442Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:18.787478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1476 reason: error at write: Disconnected sessionId: src_id|e6fcbee8-2d89d1bd-6c30d3da-7dc670b1_0
2025-10-09T21:49:18.787580Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342407297077889:3441]
2025-10-09T21:49:18.787680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1476 sessionId: src_id|e6fcbee8-2d89d1bd-6c30d3da-7dc670b1_0 is DEAD
2025-10-09T21:49:18.787827Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6fcbee8-2d89d1bd-6c30d3da-7dc670b1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:18.787840Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6fcbee8-2d89d1bd-6c30d3da-7dc670b1_0] Write session will restart in 0.005199s
2025-10-09T21:49:18.787869Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6fcbee8-2d89d1bd-6c30d3da-7dc670b1_0] Write session: Do CDS request
2025-10-09T21:49:18.787874Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6fcbee8-2d89d1bd-6c30d3da-7dc670b1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:18.787978Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:18.794367Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e6fcbee8-2d89d1bd-6c30d3da-7dc670b1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:18.794453Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:18.794468Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1477
2025-10-09T21:49:18.794572Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1477 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:18.794605Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1477 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:18.794617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1477 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:18.794620Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:18.794976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1477 sessionId:  describe result for acl check
2025-10-09T21:49:18.795009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:18.795015Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:18.795016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:18.795020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342407297077907:2795] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:18.795496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342407297077907:2795] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:18.798330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342407297077907:2795] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:18.802008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342407297077907:2795] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:18.802021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342407297077907:2795] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:18.802023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342407297077907:2795] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:18.802042Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1477 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:18.808241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:18.810681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:18.810870Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:18.810880Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:18.810892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:18.810949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:18.810991Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342406935252667:5664]
2025-10-09T21:49:18.811583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:18.811895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:18.811971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:18.812331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:18.812388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:18.812394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:18.812396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:18.812397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:18.812399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:18.812400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:18.812401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.812402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.812403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.812408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:18.812419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:18.812424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:18.812428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:18.812432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:18.812436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:18.812439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.812443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.812447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.812453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:18.812461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:18.812468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:18.812473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:18.812479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:18.812499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:18.812507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:18.812513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:18.812517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:18.812527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:18.812531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:18.812536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:18.812542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.812547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.812554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.812559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.812564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.812568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.812600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:18.818777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:18.819982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:18.819998Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:18.820001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:18.820004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:18.820010Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1476 [2:7559342406935252667:5664]
2025-10-09T21:49:18.820024Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:18.820039Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:18.820048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:18.820051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:18.820053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:18.820069Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:18.820080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:18.820087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:18.820138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.820148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.820150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.820152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:18.820215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:18.820222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.820234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:18.820251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:18.820273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:18.820297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342407297077892:2432], now have 1 active actors on pipe
2025-10-09T21:49:18.820305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342407297078038:2795], now have 1 active actors on pipe
2025-10-09T21:49:18.820312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342406935252662:5664]
2025-10-09T21:49:18.820320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342406935252662:5664]
2025-10-09T21:49:18.820322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342406935252662:5664]
2025-10-09T21:49:18.820336Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:18.820353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:18.820356Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:18.820338Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1476
2025-10-09T21:49:18.820360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:18.820357Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1476
2025-10-09T21:49:18.820362Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:18.820371Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:18.820374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:18.820401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342406935252662:5664]
2025-10-09T21:49:18.820404Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:18.820411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:18.820423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:18.820432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:18.820478Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2948331e-d0225fe3-2b888f2b-d6d27a88_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:18.820490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:18.820498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:18.820501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:18.820510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:18.820524Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:18.820544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:18.820715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:18.820725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:18.820741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:18.820876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1477 partition: 0 MaxSeqNo: 25 sessionId: src_id|2948331e-d0225fe3-2b888f2b-d6d27a88_0
2025-10-09T21:49:18.821213Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6fcbee8-2d89d1bd-6c30d3da-7dc670b1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2948331e-d0225fe3-2b888f2b-d6d27a88_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:18.823603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2948331e-d0225fe3-2b888f2b-d6d27a88_0] Write session: try to update token
2025-10-09T21:49:18.823618Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2948331e-d0225fe3-2b888f2b-d6d27a88_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:18.834624Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:18.834642Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:18.834645Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:18.834647Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:18.834656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342406935252662:5664]
2025-10-09T21:49:18.834661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342406935252662:5664]
2025-10-09T21:49:18.834664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342406935252662:5664]
2025-10-09T21:49:18.834704Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:18.834709Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:18.834710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:18.834712Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:18.834716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:18.834719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:18.834855Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:18.836208Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1477 sessionId: src_id|2948331e-d0225fe3-2b888f2b-d6d27a88_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:18.838502Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:18.887941Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342407297078206:3441]
2025-10-09T21:49:18.888239Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 1476, pipe: [1:7559342407297078206:3441]
2025-10-09T21:49:18.980322Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:18.980817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:18.980830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:18.980867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342407297078206:3441], now have 1 active actors on pipe
2025-10-09T21:49:18.981052Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:18.981091Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:18.981145Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:18.981206Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342407297078206:3441]
2025-10-09T21:49:18.981236Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1477 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2948331e-d0225fe3-2b888f2b-d6d27a88_0
2025-10-09T21:49:18.981399Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1477 sessionId: src_id|2948331e-d0225fe3-2b888f2b-d6d27a88_0 is DEAD
2025-10-09T21:49:18.981531Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2948331e-d0225fe3-2b888f2b-d6d27a88_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:18.981548Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2948331e-d0225fe3-2b888f2b-d6d27a88_0] Write session will restart in 0.009363s
2025-10-09T21:49:18.981567Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2948331e-d0225fe3-2b888f2b-d6d27a88_0] Write session: Do CDS request
2025-10-09T21:49:18.981573Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2948331e-d0225fe3-2b888f2b-d6d27a88_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:18.992464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2948331e-d0225fe3-2b888f2b-d6d27a88_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:18.992549Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:18.992568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1478
2025-10-09T21:49:18.992680Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1478 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:18.992716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1478 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:18.992728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1478 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:18.992731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:18.993441Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1478 sessionId:  describe result for acl check
2025-10-09T21:49:18.993476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:18.993483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:18.993485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:18.993491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342407297078299:2790] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:18.994064Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342407297078299:2790] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:18.996823Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342407297078299:2790] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:18.999816Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342407297078299:2790] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:18.999832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342407297078299:2790] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:18.999836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342407297078299:2790] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:18.999844Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1478 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:19.002201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:19.004933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:19.005116Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:19.005125Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:19.005137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:19.005189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:19.005224Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342411230220267:5663]
2025-10-09T21:49:19.005783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:19.006106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:19.006147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:19.006448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:19.006496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.006504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.006506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.006507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.006508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.006509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.006510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.006511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.006512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.006516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.006527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.006530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.006535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.006538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.006543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.006546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.006549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.006553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.006560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.006568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.006574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.006579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.006583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.006588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.006593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.006597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.006602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.006607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.006611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.006618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.006623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.006628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.006632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.006637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.006641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.006647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.006676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:19.013045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:19.014128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:19.014142Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:19.014145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:19.014149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:19.014157Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1477 [2:7559342411230220267:5663]
2025-10-09T21:49:19.014171Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:19.014190Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:19.014197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:19.014201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:19.014203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:19.014221Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:19.014232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:19.014238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:19.014302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.014311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.014313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.014316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:19.014381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:19.014390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.014399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.014409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.014421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:19.014437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342407297078210:2432], now have 1 active actors on pipe
2025-10-09T21:49:19.014447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342411230220262:5663]
2025-10-09T21:49:19.014449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342411230220262:5663]
2025-10-09T21:49:19.014451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342411230220262:5663]
2025-10-09T21:49:19.014462Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:19.014472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342407297078492:2790], now have 1 active actors on pipe
2025-10-09T21:49:19.014480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:19.014481Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:19.014485Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:19.014487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:19.014497Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:19.014504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:19.014535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342411230220262:5663]
2025-10-09T21:49:19.014543Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:19.014529Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1477
2025-10-09T21:49:19.014541Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1477
2025-10-09T21:49:19.014570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:19.014595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.014604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.014630Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7047e29-bc57e19c-181412c0-ec7355b3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:19.014644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:19.014648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:19.014650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:19.014661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:19.014671Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:19.014683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:19.014857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.014867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.014882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:19.014976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1478 partition: 0 MaxSeqNo: 25 sessionId: src_id|7047e29-bc57e19c-181412c0-ec7355b3_0
2025-10-09T21:49:19.015305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2948331e-d0225fe3-2b888f2b-d6d27a88_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7047e29-bc57e19c-181412c0-ec7355b3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:19.017816Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7047e29-bc57e19c-181412c0-ec7355b3_0] Write session: try to update token
2025-10-09T21:49:19.017833Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7047e29-bc57e19c-181412c0-ec7355b3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:19.026053Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:19.026070Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:19.026080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342411230220262:5663]
2025-10-09T21:49:19.026117Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:19.026128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:19.026284Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:19.026426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:19.026557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:19.026689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:19.026887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:19.026936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:19.030899Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1478 sessionId: src_id|7047e29-bc57e19c-181412c0-ec7355b3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:19.033258Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:19.033440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.033456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.033490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:19.033576Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:19.039111Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:19.039129Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:19.039131Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:19.039133Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:19.039143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342411230220262:5663]
2025-10-09T21:49:19.039147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342411230220262:5663]
2025-10-09T21:49:19.039151Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342411230220262:5663]
2025-10-09T21:49:19.039190Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:19.039190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:19.039200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.039201Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:19.039202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:19.039204Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:19.039329Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:19.081482Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342411592045819:3441]
2025-10-09T21:49:19.081776Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 1477, pipe: [1:7559342411592045819:3441]
2025-10-09T21:49:19.186650Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:19.194327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.194350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.194502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.194559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.194617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.194681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.195018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.195030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.195090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.195144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.195202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.195413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.195454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.195463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.195517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.195570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.195625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.195685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.195731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.195734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.195800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.195872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.195936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.196002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.196045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.196052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.196113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.196173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.196241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.196307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.196349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.196356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.196421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.196479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.196542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.196601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.196641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.196648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.196714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.196769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.196833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.196891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.196938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.196945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.197000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.197055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.197111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.197162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.197207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.197219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.197280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.197340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.197405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.197471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.197512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.197519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.197575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.197639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.197696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.197755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.197794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.197799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.197856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342411592045819:3441], now have 1 active actors on pipe
2025-10-09T21:49:19.198084Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:19.198094Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342411592045819:3441]
2025-10-09T21:49:19.198124Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:19.198127Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:19.198159Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1478 reason: error at write: Disconnected sessionId: src_id|7047e29-bc57e19c-181412c0-ec7355b3_0
2025-10-09T21:49:19.198232Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:19.198347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1478 sessionId: src_id|7047e29-bc57e19c-181412c0-ec7355b3_0 is DEAD
2025-10-09T21:49:19.198498Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7047e29-bc57e19c-181412c0-ec7355b3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:19.198510Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7047e29-bc57e19c-181412c0-ec7355b3_0] Write session will restart in 0.008741s
2025-10-09T21:49:19.198529Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7047e29-bc57e19c-181412c0-ec7355b3_0] Write session: Do CDS request
2025-10-09T21:49:19.198533Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7047e29-bc57e19c-181412c0-ec7355b3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:19.208499Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7047e29-bc57e19c-181412c0-ec7355b3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:19.208577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:19.208595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1479
2025-10-09T21:49:19.208747Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1479 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:19.208789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1479 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:19.208799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1479 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:19.208802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:19.209348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1479 sessionId:  describe result for acl check
2025-10-09T21:49:19.209383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:19.209390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:19.209392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:19.209397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342411592045870:2783] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:19.210315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342411592045870:2783] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:19.213897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342411592045870:2783] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:19.217221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342411592045870:2783] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:19.217236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342411592045870:2783] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:19.217239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342411592045870:2783] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:19.217247Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1479 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:19.220520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:19.223151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:19.223356Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:19.223368Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:19.223411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:19.223465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:19.223496Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342411230220581:5658]
2025-10-09T21:49:19.224015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:19.224419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:19.224485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:19.224783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:19.224832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.224837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.224837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.224838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.224839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.224839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.224840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.224841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.224841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.224845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.224852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.224854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.224856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.224858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.224861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.224863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.224865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.224867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.224874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.224882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.224887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.224892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.224897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.224902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.224911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.224917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.224927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.224930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.224933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.224936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.224938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.224941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.224944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.224947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.224949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.224952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.224968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:19.231345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:19.232442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:19.232454Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:19.232455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:19.232458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:19.232463Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1478 [2:7559342411230220581:5658]
2025-10-09T21:49:19.232471Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:19.232484Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:19.232486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:19.232488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:19.232489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:19.232501Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:19.232505Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:19.232510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:19.232548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.232555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.232557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.232560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:19.232612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:19.232618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.232625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.232635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.232648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:19.232666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342411230220571:5658]
2025-10-09T21:49:19.232670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342411230220571:5658]
2025-10-09T21:49:19.232672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342411230220571:5658]
2025-10-09T21:49:19.232682Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:19.232695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342411592046064:2783], now have 1 active actors on pipe
2025-10-09T21:49:19.232701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342411592045826:2432], now have 1 active actors on pipe
2025-10-09T21:49:19.232707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:19.232711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:19.232714Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:19.232716Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:19.232719Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:19.232721Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:19.232744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342411230220571:5658]
2025-10-09T21:49:19.232759Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:19.232783Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:19.232771Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1478
2025-10-09T21:49:19.232791Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1478
2025-10-09T21:49:19.232864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.232879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.232912Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b7e78745-f2118a91-9befafba-a18e5d84_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:19.232925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:19.232934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:19.232937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:19.232948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:19.232959Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:19.232978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:19.233105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.233113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.233127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:19.233234Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1479 partition: 0 MaxSeqNo: 25 sessionId: src_id|b7e78745-f2118a91-9befafba-a18e5d84_0
2025-10-09T21:49:19.233616Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7047e29-bc57e19c-181412c0-ec7355b3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b7e78745-f2118a91-9befafba-a18e5d84_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:19.236088Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7e78745-f2118a91-9befafba-a18e5d84_0] Write session: try to update token
2025-10-09T21:49:19.236106Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7e78745-f2118a91-9befafba-a18e5d84_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:19.243763Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:19.243776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:19.243783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342411230220571:5658]
2025-10-09T21:49:19.243819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:19.243825Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:19.243974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:19.244109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:19.244237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:19.244376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:19.244569Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:19.244615Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:19.249010Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1479 sessionId: src_id|b7e78745-f2118a91-9befafba-a18e5d84_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:19.251208Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:19.251410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.251432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.251469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:19.251568Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:19.256558Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:19.256576Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:19.256579Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:19.256581Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:19.256590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342411230220571:5658]
2025-10-09T21:49:19.256594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342411230220571:5658]
2025-10-09T21:49:19.256598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342411230220571:5658]
2025-10-09T21:49:19.256639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:19.256642Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:19.256648Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:19.256651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.256654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:19.256661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:19.256786Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:19.298073Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342411592046148:3441]
2025-10-09T21:49:19.298361Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 1478, pipe: [1:7559342411592046148:3441]
2025-10-09T21:49:19.407101Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:19.414531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.414558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.414709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.414768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.414825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.414886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.414933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.414942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.415000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.415056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.415124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.415193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.415238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.415246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.415300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.415350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.415407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.415467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.415509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.415519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.415586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.415637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.415687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.415740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.415778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.415784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.415840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.415893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.415954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.416005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.416048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.416055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.416112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.416173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.416231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.416297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.416340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.416348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.416404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.416459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.416519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.416574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.416614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.416619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.416670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.416727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.416781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.416837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.416880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.416889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.416944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.416997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.417051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.417107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.417150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.417158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.417223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:19.417281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:19.417338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:19.417393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:19.417435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:19.417444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:19.417491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342411592046148:3441], now have 1 active actors on pipe
2025-10-09T21:49:19.417763Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:19.417778Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342411592046148:3441]
2025-10-09T21:49:19.417791Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:19.417813Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:19.417823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1479 reason: error at write: Disconnected sessionId: src_id|b7e78745-f2118a91-9befafba-a18e5d84_0
2025-10-09T21:49:19.418010Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1479 sessionId: src_id|b7e78745-f2118a91-9befafba-a18e5d84_0 is DEAD
2025-10-09T21:49:19.418163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7e78745-f2118a91-9befafba-a18e5d84_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:19.418174Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7e78745-f2118a91-9befafba-a18e5d84_0] Write session will restart in 0.005061s
2025-10-09T21:49:19.418210Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7e78745-f2118a91-9befafba-a18e5d84_0] Write session: Do CDS request
2025-10-09T21:49:19.418217Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7e78745-f2118a91-9befafba-a18e5d84_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:19.418458Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:19.424440Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7e78745-f2118a91-9befafba-a18e5d84_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:19.424516Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:19.424530Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1480
2025-10-09T21:49:19.424633Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1480 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:19.424674Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1480 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:19.424683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1480 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:19.424688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:19.425100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1480 sessionId:  describe result for acl check
2025-10-09T21:49:19.425137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:19.425142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:19.425143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:19.425150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342411592046165:2768] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:19.425675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342411592046165:2768] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:19.428011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:19.428094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342411592046165:2768] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:19.430933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:19.431142Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:19.431152Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:19.431164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:19.431204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342411592046165:2768] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:19.431218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342411592046165:2768] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:19.431219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:19.431221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342411592046165:2768] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:19.431228Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1480 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:19.431259Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342411230220637:5654]
2025-10-09T21:49:19.431786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:19.432095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:19.432134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:19.432419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:19.432463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.432468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.432469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.432470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.432472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.432473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.432474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.432475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.432476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.432481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.432490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.432495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.432503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.432507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.432515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.432518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.432526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.432530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.432539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.432546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.432555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.432564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.432572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.432577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.432584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.432590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.432600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.432604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.432609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.432614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.432618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.432623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.432628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.432633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.432637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.432642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.432668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:19.438715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:19.439704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:19.439720Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:19.439722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:19.439725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:19.439731Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1479 [2:7559342411230220637:5654]
2025-10-09T21:49:19.439743Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:19.439760Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:19.439763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:19.439766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:19.439768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:19.439782Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:19.439792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:19.439799Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:19.439848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.439854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.439856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.439859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:19.439916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:19.439924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.439935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.439944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.439959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:19.439982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342411592046151:2432], now have 1 active actors on pipe
2025-10-09T21:49:19.439990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342411592046205:2768], now have 1 active actors on pipe
2025-10-09T21:49:19.439997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342411230220631:5654]
2025-10-09T21:49:19.439999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342411230220631:5654]
2025-10-09T21:49:19.440001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342411230220631:5654]
2025-10-09T21:49:19.440013Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:19.440026Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:19.440030Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:19.440035Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:19.440038Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:19.440044Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:19.440055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:19.440031Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1479
2025-10-09T21:49:19.440081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342411230220631:5654]
2025-10-09T21:49:19.440092Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:19.440098Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:19.440093Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1479
2025-10-09T21:49:19.440163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.440171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.440198Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c3c182fe-d964c10c-f1fe9206-43ff2ab9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:19.440208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:19.440212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:19.440215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:19.440225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:19.440238Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:19.440251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:19.440398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.440416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.440437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:19.440571Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1480 partition: 0 MaxSeqNo: 25 sessionId: src_id|c3c182fe-d964c10c-f1fe9206-43ff2ab9_0
2025-10-09T21:49:19.440858Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7e78745-f2118a91-9befafba-a18e5d84_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c3c182fe-d964c10c-f1fe9206-43ff2ab9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:19.443205Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3c182fe-d964c10c-f1fe9206-43ff2ab9_0] Write session: try to update token
2025-10-09T21:49:19.443222Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3c182fe-d964c10c-f1fe9206-43ff2ab9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:19.453719Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:19.453738Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:19.453741Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:19.453742Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:19.453753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342411230220631:5654]
2025-10-09T21:49:19.453757Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342411230220631:5654]
2025-10-09T21:49:19.453761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342411230220631:5654]
2025-10-09T21:49:19.453801Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:19.453807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:19.453811Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:19.453814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.453814Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:19.453816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:19.453942Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:19.455487Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1480 sessionId: src_id|c3c182fe-d964c10c-f1fe9206-43ff2ab9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:19.457691Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:19.517702Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342411592046217:3441]
2025-10-09T21:49:19.517951Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 1479, pipe: [1:7559342411592046217:3441]
2025-10-09T21:49:19.602773Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:19.603170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.603182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.603212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342411592046217:3441], now have 1 active actors on pipe
2025-10-09T21:49:19.603323Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:19.603371Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342411592046217:3441]
2025-10-09T21:49:19.603383Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:19.603396Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:19.603447Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1480 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c3c182fe-d964c10c-f1fe9206-43ff2ab9_0
2025-10-09T21:49:19.603607Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1480 sessionId: src_id|c3c182fe-d964c10c-f1fe9206-43ff2ab9_0 is DEAD
2025-10-09T21:49:19.603749Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3c182fe-d964c10c-f1fe9206-43ff2ab9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:19.603760Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3c182fe-d964c10c-f1fe9206-43ff2ab9_0] Write session will restart in 0.008149s
2025-10-09T21:49:19.603785Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3c182fe-d964c10c-f1fe9206-43ff2ab9_0] Write session: Do CDS request
2025-10-09T21:49:19.603792Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3c182fe-d964c10c-f1fe9206-43ff2ab9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:19.611931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:19.613465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3c182fe-d964c10c-f1fe9206-43ff2ab9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:19.613496Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:19.613509Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1481
2025-10-09T21:49:19.613677Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1481 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:19.613723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1481 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:19.613733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1481 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:19.613736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:19.614076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1481 sessionId:  describe result for acl check
2025-10-09T21:49:19.614112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:19.614117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:19.614118Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:19.614122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342411592046265:2755] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:19.614509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:19.614539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342411592046265:2755] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:19.614680Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:19.614689Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:19.614700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:19.614748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:19.614783Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342411230220731:5636]
2025-10-09T21:49:19.615285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:19.615593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:19.615639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:19.615921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:19.615972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.615980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.615981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.615982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.615983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.615984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.615985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.615986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.615987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.615992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.616001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.616005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.616009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.616012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.616016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.616019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.616023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.616026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.616032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.616039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.616047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.616052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.616057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.616061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.616065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.616069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.616073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.616078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.616082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.616087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.616090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.616095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.616099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.616104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.616114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.616094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342411592046265:2755] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:19.616119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.616147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:19.622128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342411592046265:2755] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:19.622146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342411592046265:2755] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:19.622149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342411592046265:2755] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:19.622157Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1481 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:19.622194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:19.623188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:19.623203Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:19.623205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:19.623208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:19.623214Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1480 [2:7559342411230220731:5636]
2025-10-09T21:49:19.623239Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:19.623255Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:19.623260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:19.623263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:19.623264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:19.623275Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:19.623281Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:19.623286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:19.623327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.623331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.623332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.623334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:19.623384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:19.623391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.623400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.623415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.623433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:19.623459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342411592046301:2755], now have 1 active actors on pipe
2025-10-09T21:49:19.623462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342411592046245:2432], now have 1 active actors on pipe
2025-10-09T21:49:19.623469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342411230220726:5636]
2025-10-09T21:49:19.623471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342411230220726:5636]
2025-10-09T21:49:19.623472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342411230220726:5636]
2025-10-09T21:49:19.623486Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:19.623498Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:19.623505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:19.623508Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:19.623510Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:19.623493Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1480
2025-10-09T21:49:19.623515Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:19.623522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:19.623514Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1480
2025-10-09T21:49:19.623544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342411230220726:5636]
2025-10-09T21:49:19.623552Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:19.623561Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:19.623565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.623570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.623610Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ef0e29e5-b086c385-44be89c8-f3eaac5c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:19.623621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:19.623629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:19.623631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:19.623641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:19.623654Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:19.623674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:19.623798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.623804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.623818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:19.623906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1481 partition: 0 MaxSeqNo: 25 sessionId: src_id|ef0e29e5-b086c385-44be89c8-f3eaac5c_0
2025-10-09T21:49:19.624174Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3c182fe-d964c10c-f1fe9206-43ff2ab9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ef0e29e5-b086c385-44be89c8-f3eaac5c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:19.626536Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef0e29e5-b086c385-44be89c8-f3eaac5c_0] Write session: try to update token
2025-10-09T21:49:19.626554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef0e29e5-b086c385-44be89c8-f3eaac5c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:19.637815Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:19.637835Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:19.637837Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:19.637839Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:19.637850Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342411230220726:5636]
2025-10-09T21:49:19.637854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342411230220726:5636]
2025-10-09T21:49:19.637858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342411230220726:5636]
2025-10-09T21:49:19.637904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:19.637911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.637911Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:19.637913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:19.637917Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:19.637921Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:19.638072Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:19.639164Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1481 sessionId: src_id|ef0e29e5-b086c385-44be89c8-f3eaac5c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:19.641558Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:19.703531Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342411592046309:3441]
2025-10-09T21:49:19.703838Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 1480, pipe: [1:7559342411592046309:3441]
2025-10-09T21:49:19.784812Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:19.785319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.785333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.785379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342411592046309:3441], now have 1 active actors on pipe
2025-10-09T21:49:19.785594Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:19.785657Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342411592046309:3441]
2025-10-09T21:49:19.785668Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:19.785672Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:19.785706Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:19.785736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1481 reason: error at write: Disconnected sessionId: src_id|ef0e29e5-b086c385-44be89c8-f3eaac5c_0
2025-10-09T21:49:19.785919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1481 sessionId: src_id|ef0e29e5-b086c385-44be89c8-f3eaac5c_0 is DEAD
2025-10-09T21:49:19.786092Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef0e29e5-b086c385-44be89c8-f3eaac5c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:19.786105Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef0e29e5-b086c385-44be89c8-f3eaac5c_0] Write session will restart in 0.005863s
2025-10-09T21:49:19.786130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef0e29e5-b086c385-44be89c8-f3eaac5c_0] Write session: Do CDS request
2025-10-09T21:49:19.786135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef0e29e5-b086c385-44be89c8-f3eaac5c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:19.793510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef0e29e5-b086c385-44be89c8-f3eaac5c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:19.793589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:19.793616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1482
2025-10-09T21:49:19.793754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1482 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:19.793788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1482 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:19.793798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1482 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:19.793803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:19.794281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1482 sessionId:  describe result for acl check
2025-10-09T21:49:19.794320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:19.794329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:19.794331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:19.794337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342411592046330:2743] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:19.794886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342411592046330:2743] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:19.795104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:19.797155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342411592046330:2743] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:19.798093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:19.798281Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:19.798290Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:19.798301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:19.798353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:19.798389Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342411230220794:5629]
2025-10-09T21:49:19.798970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:19.799189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:19.799241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:19.799281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342411592046330:2743] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:19.799294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342411592046330:2743] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:19.799298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342411592046330:2743] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:19.799306Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1482 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:19.799559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:19.799613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.799620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.799621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.799622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.799623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.799625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.799626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.799629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.799630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.799635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.799645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.799649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.799653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.799657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.799661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.799664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.799667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.799671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.799681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.799693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.799704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.799713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.799722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.799727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.799736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.799741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.799746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.799751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.799760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.799765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.799774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.799780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.799789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.799793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.799802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.799806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.799840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:19.805831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:19.806870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:19.806884Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:19.806887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:19.806891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:19.806897Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1481 [2:7559342411230220794:5629]
2025-10-09T21:49:19.806909Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:19.806926Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:19.806929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:19.806932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:19.806935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:19.806950Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:19.806955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:19.806962Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:19.807014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.807022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.807024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.807026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:19.807092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:19.807100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.807112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.807124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.807146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:19.807171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342411592046311:2432], now have 1 active actors on pipe
2025-10-09T21:49:19.807178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342411592046369:2743], now have 1 active actors on pipe
2025-10-09T21:49:19.807186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342411230220788:5629]
2025-10-09T21:49:19.807188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342411230220788:5629]
2025-10-09T21:49:19.807190Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342411230220788:5629]
2025-10-09T21:49:19.807202Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:19.807220Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1481
2025-10-09T21:49:19.807216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:19.807225Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1481
2025-10-09T21:49:19.807225Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:19.807227Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:19.807230Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:19.807235Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:19.807238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:19.807281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.807285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.807297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342411230220788:5629]
2025-10-09T21:49:19.807308Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:19.807318Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:19.807340Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d662714f-31dd4633-f7f05933-ab3620db_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:19.807352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:19.807356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:19.807363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:19.807373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:19.807387Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:19.807616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:19.807783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.807795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.807811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:19.807895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1482 partition: 0 MaxSeqNo: 25 sessionId: src_id|d662714f-31dd4633-f7f05933-ab3620db_0
2025-10-09T21:49:19.808224Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef0e29e5-b086c385-44be89c8-f3eaac5c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d662714f-31dd4633-f7f05933-ab3620db_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:19.810651Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d662714f-31dd4633-f7f05933-ab3620db_0] Write session: try to update token
2025-10-09T21:49:19.810668Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d662714f-31dd4633-f7f05933-ab3620db_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:19.821280Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:19.821298Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:19.821301Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:19.821303Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:19.821313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342411230220788:5629]
2025-10-09T21:49:19.821318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342411230220788:5629]
2025-10-09T21:49:19.821321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342411230220788:5629]
2025-10-09T21:49:19.821364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:19.821372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.821371Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:19.821378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:19.821378Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:19.821382Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:19.821511Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:19.823240Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1482 sessionId: src_id|d662714f-31dd4633-f7f05933-ab3620db_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:19.825505Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:19.885817Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342411592046381:3441]
2025-10-09T21:49:19.886132Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 1481, pipe: [1:7559342411592046381:3441]
2025-10-09T21:49:19.976792Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:19.977071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.977087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.977138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342411592046381:3441], now have 1 active actors on pipe
2025-10-09T21:49:19.977420Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:19.977437Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:19.977412Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:19.977500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1482 reason: error at write: Disconnected sessionId: src_id|d662714f-31dd4633-f7f05933-ab3620db_0
2025-10-09T21:49:19.977689Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1482 sessionId: src_id|d662714f-31dd4633-f7f05933-ab3620db_0 is DEAD
2025-10-09T21:49:19.977690Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342411592046381:3441]
2025-10-09T21:49:19.977719Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:19.977834Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d662714f-31dd4633-f7f05933-ab3620db_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:19.977845Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d662714f-31dd4633-f7f05933-ab3620db_0] Write session will restart in 0.009031s
2025-10-09T21:49:19.977882Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d662714f-31dd4633-f7f05933-ab3620db_0] Write session: Do CDS request
2025-10-09T21:49:19.977889Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d662714f-31dd4633-f7f05933-ab3620db_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:19.987017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:19.988497Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d662714f-31dd4633-f7f05933-ab3620db_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:19.988607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:19.988624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1483
2025-10-09T21:49:19.988755Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1483 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:19.988782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1483 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:19.988786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1483 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:19.988789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:19.989247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1483 sessionId:  describe result for acl check
2025-10-09T21:49:19.989287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:19.989294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:19.989296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:19.989302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342411592046407:2733] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:19.989801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342411592046407:2733] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:19.989839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:19.990012Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:19.990042Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:19.990055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:19.990111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:19.990196Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342411230220854:5608]
2025-10-09T21:49:19.990763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:19.991093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:19.991138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:19.991457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:19.991506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.991516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.991517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.991519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.991520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.991521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.991522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.991523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.991524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.991530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.991540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.991545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.991549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.991553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.991557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.991561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.991565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.991569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.991575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.991583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:19.991590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.991595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:19.991601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.991607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:19.991611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.991617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:19.991621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.991627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:19.991609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342411592046407:2733] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:19.991631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.991636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:19.991640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.991646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.991650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.991655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.991659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.991664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.991695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:19.997724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342411592046407:2733] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:19.997742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342411592046407:2733] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:19.997746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342411592046407:2733] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:19.997757Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1483 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:19.998074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:19.999084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:19.999101Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:19.999103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:19.999107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:19.999114Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1482 [2:7559342411230220854:5608]
2025-10-09T21:49:19.999126Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:19.999144Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:19.999151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:19.999154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:19.999156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:19.999171Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:19.999177Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:19.999183Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:19.999238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.999244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.999246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.999249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:19.999315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:19.999321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:19.999328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:19.999334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:19.999352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:19.999376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342411592046385:2432], now have 1 active actors on pipe
2025-10-09T21:49:19.999388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342411592046443:2733], now have 1 active actors on pipe
2025-10-09T21:49:19.999395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342411230220848:5608]
2025-10-09T21:49:19.999397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342411230220848:5608]
2025-10-09T21:49:19.999399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342411230220848:5608]
2025-10-09T21:49:19.999413Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:19.999429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:19.999431Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:19.999434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:19.999436Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:19.999449Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:19.999452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:19.999455Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1482
2025-10-09T21:49:19.999462Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1482
2025-10-09T21:49:19.999479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342411230220848:5608]
2025-10-09T21:49:19.999489Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:19.999492Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:19.999516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.999521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.999562Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|660d0894-10721bcf-6445cc8a-25033bc6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:19.999575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:19.999579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:19.999582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:19.999592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:19.999608Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:19.999625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:19.999793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:19.999803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:19.999821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:19.999991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1483 partition: 0 MaxSeqNo: 25 sessionId: src_id|660d0894-10721bcf-6445cc8a-25033bc6_0
2025-10-09T21:49:20.000338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d662714f-31dd4633-f7f05933-ab3620db_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|660d0894-10721bcf-6445cc8a-25033bc6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:20.003023Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|660d0894-10721bcf-6445cc8a-25033bc6_0] Write session: try to update token
2025-10-09T21:49:20.003044Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|660d0894-10721bcf-6445cc8a-25033bc6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:20.013675Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:20.013693Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:20.013695Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:20.013698Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:20.013706Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342411230220848:5608]
2025-10-09T21:49:20.013711Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342411230220848:5608]
2025-10-09T21:49:20.013715Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342411230220848:5608]
2025-10-09T21:49:20.013771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:20.013771Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:20.013777Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:20.013778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.013780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:20.013781Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:20.013901Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:20.016577Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1483 sessionId: src_id|660d0894-10721bcf-6445cc8a-25033bc6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:20.019115Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:20.077849Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342415887013749:3441]
2025-10-09T21:49:20.078140Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 1482, pipe: [1:7559342415887013749:3441]
2025-10-09T21:49:20.165437Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:20.166048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:20.166064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:20.166109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342415887013749:3441], now have 1 active actors on pipe
2025-10-09T21:49:20.166321Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:20.166329Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342415887013749:3441]
2025-10-09T21:49:20.166340Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:20.166373Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:20.166449Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1483 reason: status is not ok: tablet will be restarted right now sessionId: src_id|660d0894-10721bcf-6445cc8a-25033bc6_0
2025-10-09T21:49:20.166627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1483 sessionId: src_id|660d0894-10721bcf-6445cc8a-25033bc6_0 is DEAD
2025-10-09T21:49:20.166799Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|660d0894-10721bcf-6445cc8a-25033bc6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:20.166811Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|660d0894-10721bcf-6445cc8a-25033bc6_0] Write session will restart in 0.008458s
2025-10-09T21:49:20.166839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|660d0894-10721bcf-6445cc8a-25033bc6_0] Write session: Do CDS request
2025-10-09T21:49:20.166846Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|660d0894-10721bcf-6445cc8a-25033bc6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:20.176516Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|660d0894-10721bcf-6445cc8a-25033bc6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:20.176632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:20.176649Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1484
2025-10-09T21:49:20.176608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:20.176762Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1484 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:20.176809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1484 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:20.176820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1484 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:20.176824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:20.177410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1484 sessionId:  describe result for acl check
2025-10-09T21:49:20.177454Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:20.177462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:20.177463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:20.177469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342415887013777:2728] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:20.178010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342415887013777:2728] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:20.179692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:20.179909Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:20.179923Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:20.179936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:20.179997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:20.180037Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342415525188210:5590]
2025-10-09T21:49:20.180177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342415887013777:2728] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:20.180600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:20.180886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:20.180927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:20.181143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:20.181175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.181180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.181181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.181182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.181182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.181185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.181185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.181186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.181186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.181189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.181196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.181198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.181200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.181202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.181204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.181206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.181208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.181210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.181215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.181220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.181224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.181227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.181230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.181233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.181235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.181238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.181243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.181246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.181248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.181252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.181254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.181257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.181260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.181263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.181266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.181268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.181284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:20.186434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342415887013777:2728] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:20.186452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342415887013777:2728] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:20.186456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342415887013777:2728] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:20.186466Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1484 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:20.187726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:20.188738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:20.188755Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:20.188758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:20.188762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:20.188769Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1483 [2:7559342415525188210:5590]
2025-10-09T21:49:20.188786Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:20.188804Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:20.188815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:20.188818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:20.188821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:20.188839Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:20.188846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:20.188853Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:20.188910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.188918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.188922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.188924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:20.188991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:20.188999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.189011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.189022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.189037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:20.189064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342415887013754:2432], now have 1 active actors on pipe
2025-10-09T21:49:20.189068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342415887013816:2728], now have 1 active actors on pipe
2025-10-09T21:49:20.189075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342415525188204:5590]
2025-10-09T21:49:20.189077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342415525188204:5590]
2025-10-09T21:49:20.189079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342415525188204:5590]
2025-10-09T21:49:20.189093Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:20.189096Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1483
2025-10-09T21:49:20.189107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:20.189100Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1483
2025-10-09T21:49:20.189111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:20.189119Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:20.189126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:20.189129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:20.189131Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:20.189177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:20.189182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:20.189195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342415525188204:5590]
2025-10-09T21:49:20.189207Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:20.189217Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:20.189240Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b555ba1b-984c8fe2-ec9d9c95-3493b05a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:20.189252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:20.189257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:20.189261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:20.189272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:20.189281Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:20.189301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:20.189461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:20.189470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:20.189484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:20.189587Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1484 partition: 0 MaxSeqNo: 25 sessionId: src_id|b555ba1b-984c8fe2-ec9d9c95-3493b05a_0
2025-10-09T21:49:20.189978Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|660d0894-10721bcf-6445cc8a-25033bc6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b555ba1b-984c8fe2-ec9d9c95-3493b05a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:20.192468Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b555ba1b-984c8fe2-ec9d9c95-3493b05a_0] Write session: try to update token
2025-10-09T21:49:20.192490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b555ba1b-984c8fe2-ec9d9c95-3493b05a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:20.203536Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:20.203555Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:20.203557Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:20.203559Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:20.203570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342415525188204:5590]
2025-10-09T21:49:20.203575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342415525188204:5590]
2025-10-09T21:49:20.203579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342415525188204:5590]
2025-10-09T21:49:20.203625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:20.203632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.203632Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:20.203636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:20.203639Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:20.203643Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:20.203768Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:20.204828Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1484 sessionId: src_id|b555ba1b-984c8fe2-ec9d9c95-3493b05a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:20.207218Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:20.266722Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342415887013829:3441]
2025-10-09T21:49:20.266986Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 1483, pipe: [1:7559342415887013829:3441]
2025-10-09T21:49:20.352735Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:20.353296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:20.353310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:20.353348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342415887013829:3441], now have 1 active actors on pipe
2025-10-09T21:49:20.353567Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:20.353628Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:20.353662Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342415887013829:3441]
2025-10-09T21:49:20.353690Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:20.353693Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:20.353758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1484 reason: error at write: Disconnected sessionId: src_id|b555ba1b-984c8fe2-ec9d9c95-3493b05a_0
2025-10-09T21:49:20.353976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1484 sessionId: src_id|b555ba1b-984c8fe2-ec9d9c95-3493b05a_0 is DEAD
2025-10-09T21:49:20.354152Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b555ba1b-984c8fe2-ec9d9c95-3493b05a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:20.354170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b555ba1b-984c8fe2-ec9d9c95-3493b05a_0] Write session will restart in 0.008956s
2025-10-09T21:49:20.354190Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b555ba1b-984c8fe2-ec9d9c95-3493b05a_0] Write session: Do CDS request
2025-10-09T21:49:20.354197Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b555ba1b-984c8fe2-ec9d9c95-3493b05a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:20.363131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:20.364522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b555ba1b-984c8fe2-ec9d9c95-3493b05a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:20.364608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:20.364626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1485
2025-10-09T21:49:20.364769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1485 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:20.364809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1485 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:20.364822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1485 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:20.364826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:20.365310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1485 sessionId:  describe result for acl check
2025-10-09T21:49:20.365343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:20.365345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:20.365346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:20.365349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342415887013855:2724] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:20.365737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:20.365871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342415887013855:2724] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:20.365901Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:20.365911Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:20.365925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:20.365980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:20.366051Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342415525188271:5567]
2025-10-09T21:49:20.366617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:20.366925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:20.366970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:20.367283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:20.367337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.367343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.367344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.367346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.367347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.367348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.367349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.367350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.367352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.367358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.367368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.367372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.367377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.367380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.367385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.367389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.367392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.367396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.367402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.367408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.367414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.367420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.367425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.367430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.367435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.367440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.367445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.367450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.367454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.367460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.367464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.367470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.367474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.367480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.367484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.367488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.367521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:20.367930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342415887013855:2724] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:20.374213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342415887013855:2724] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:20.374229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342415887013855:2724] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:20.374233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342415887013855:2724] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:20.374242Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1485 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:20.374609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:20.375684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:20.375700Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:20.375702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:20.375709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:20.375715Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1484 [2:7559342415525188271:5567]
2025-10-09T21:49:20.375728Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:20.375750Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:20.375759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:20.375762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:20.375764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:20.375777Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:20.375781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:20.375786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:20.375844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.375851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.375853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.375856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:20.375919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:20.375926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.375938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.375950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.375971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:20.375990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342415887013831:2432], now have 1 active actors on pipe
2025-10-09T21:49:20.375997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342415887013895:2724], now have 1 active actors on pipe
2025-10-09T21:49:20.376005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342415525188265:5567]
2025-10-09T21:49:20.376007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342415525188265:5567]
2025-10-09T21:49:20.376009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342415525188265:5567]
2025-10-09T21:49:20.376029Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:20.376015Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1484
2025-10-09T21:49:20.376026Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1484
2025-10-09T21:49:20.376048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:20.376068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:20.376075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:20.376087Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:20.376096Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:20.376099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:20.376110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342415525188265:5567]
2025-10-09T21:49:20.376114Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:20.376122Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:20.376132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:20.376136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:20.376191Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3c79c84a-861ff1e1-d302d9e-5b31a971_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:20.376202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:20.376206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:20.376209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:20.376219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:20.376232Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:20.376260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:20.376427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:20.376441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:20.376463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:20.376574Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1485 partition: 0 MaxSeqNo: 25 sessionId: src_id|3c79c84a-861ff1e1-d302d9e-5b31a971_0
2025-10-09T21:49:20.376885Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b555ba1b-984c8fe2-ec9d9c95-3493b05a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3c79c84a-861ff1e1-d302d9e-5b31a971_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:20.379160Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c79c84a-861ff1e1-d302d9e-5b31a971_0] Write session: try to update token
2025-10-09T21:49:20.379179Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c79c84a-861ff1e1-d302d9e-5b31a971_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:20.391548Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1485 sessionId: src_id|3c79c84a-861ff1e1-d302d9e-5b31a971_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:20.393618Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:20.393635Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:20.393638Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:20.393640Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:20.393650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342415525188265:5567]
2025-10-09T21:49:20.393654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342415525188265:5567]
2025-10-09T21:49:20.393658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342415525188265:5567]
2025-10-09T21:49:20.393703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:20.393704Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:20.393710Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:20.393714Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:20.393715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.393718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:20.393849Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:20.393955Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:20.453635Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342415887013902:3441]
2025-10-09T21:49:20.453918Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 1484, pipe: [1:7559342415887013902:3441]
2025-10-09T21:49:20.535942Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:20.536426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:20.536440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:20.536481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342415887013902:3441], now have 1 active actors on pipe
2025-10-09T21:49:20.536658Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:20.536703Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342415887013902:3441]
2025-10-09T21:49:20.536718Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:20.536735Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:20.536797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1485 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3c79c84a-861ff1e1-d302d9e-5b31a971_0
2025-10-09T21:49:20.536974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1485 sessionId: src_id|3c79c84a-861ff1e1-d302d9e-5b31a971_0 is DEAD
2025-10-09T21:49:20.537139Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c79c84a-861ff1e1-d302d9e-5b31a971_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:20.537151Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c79c84a-861ff1e1-d302d9e-5b31a971_0] Write session will restart in 0.006755s
2025-10-09T21:49:20.537171Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c79c84a-861ff1e1-d302d9e-5b31a971_0] Write session: Do CDS request
2025-10-09T21:49:20.537176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c79c84a-861ff1e1-d302d9e-5b31a971_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:20.545436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c79c84a-861ff1e1-d302d9e-5b31a971_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:20.545542Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:20.545558Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1486
2025-10-09T21:49:20.545699Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1486 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:20.545733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1486 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:20.545744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1486 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:20.545748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:20.546154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1486 sessionId:  describe result for acl check
2025-10-09T21:49:20.546189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:20.546195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:20.546197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:20.546202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342415887013929:2714] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:20.546250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:20.546738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342415887013929:2714] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:20.548575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342415887013929:2714] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:20.549128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:20.549310Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:20.549321Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:20.549333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:20.549391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:20.549436Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342415525188333:5562]
2025-10-09T21:49:20.550001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:20.550316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:20.550358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:20.550660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:20.550708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.550714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.550715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.550716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.550718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.550719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.550720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.550722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.550723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.550729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.550739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.550744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.550747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.550751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.550755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.550759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.550762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.550766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.550773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.550780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.550787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.550793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.550797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.550806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.550812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.550819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.550827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.550832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.550847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.550856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.550861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.550865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.550871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.550876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.550884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.550891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.550924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:20.550966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342415887013929:2714] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:20.550980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342415887013929:2714] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:20.550983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342415887013929:2714] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:20.550991Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1486 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:20.558184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:20.559149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:20.559164Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:20.559167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:20.559173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:20.559179Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1485 [2:7559342415525188333:5562]
2025-10-09T21:49:20.559190Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:20.559204Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:20.559211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:20.559214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:20.559216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:20.559231Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:20.559241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:20.559247Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:20.559297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.559304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.559306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.559308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:20.559369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:20.559377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.559388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.559403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.559424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:20.559449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342415887013904:2432], now have 1 active actors on pipe
2025-10-09T21:49:20.559452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342415887013967:2714], now have 1 active actors on pipe
2025-10-09T21:49:20.559460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342415525188327:5562]
2025-10-09T21:49:20.559462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342415525188327:5562]
2025-10-09T21:49:20.559464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342415525188327:5562]
2025-10-09T21:49:20.559477Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:20.559468Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1485
2025-10-09T21:49:20.559487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:20.559489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:20.559490Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:20.559494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:20.559508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:20.559475Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1485
2025-10-09T21:49:20.559510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:20.559545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:20.559552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:20.559565Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342415525188327:5562]
2025-10-09T21:49:20.559575Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:20.559582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:20.559604Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|19aed3d8-37bcca1d-705e5edd-ba5993ef_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:20.559615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:20.559619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:20.559626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:20.559635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:20.559648Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:20.559666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:20.559804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:20.559813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:20.559827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:20.559908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1486 partition: 0 MaxSeqNo: 25 sessionId: src_id|19aed3d8-37bcca1d-705e5edd-ba5993ef_0
2025-10-09T21:49:20.560204Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c79c84a-861ff1e1-d302d9e-5b31a971_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|19aed3d8-37bcca1d-705e5edd-ba5993ef_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:20.562566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19aed3d8-37bcca1d-705e5edd-ba5993ef_0] Write session: try to update token
2025-10-09T21:49:20.562582Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19aed3d8-37bcca1d-705e5edd-ba5993ef_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:20.574390Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1486 sessionId: src_id|19aed3d8-37bcca1d-705e5edd-ba5993ef_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:20.575933Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:20.575947Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:20.575948Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:20.575949Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:20.575957Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342415525188327:5562]
2025-10-09T21:49:20.575960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342415525188327:5562]
2025-10-09T21:49:20.575964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342415525188327:5562]
2025-10-09T21:49:20.575996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:20.576001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.576003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:20.576011Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:20.576024Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:20.576028Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:20.576132Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:20.576609Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:20.636929Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342415887013989:3441]
2025-10-09T21:49:20.637172Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 1485, pipe: [1:7559342415887013989:3441]
2025-10-09T21:49:20.726147Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:20.726765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:20.726783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:20.726828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342415887013989:3441], now have 1 active actors on pipe
2025-10-09T21:49:20.727040Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:20.727083Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:20.727094Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:20.727105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:20.727141Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342415887013989:3441]
2025-10-09T21:49:20.727180Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1486 reason: error at write: Disconnected sessionId: src_id|19aed3d8-37bcca1d-705e5edd-ba5993ef_0
2025-10-09T21:49:20.727362Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1486 sessionId: src_id|19aed3d8-37bcca1d-705e5edd-ba5993ef_0 is DEAD
2025-10-09T21:49:20.727521Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19aed3d8-37bcca1d-705e5edd-ba5993ef_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:20.727532Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19aed3d8-37bcca1d-705e5edd-ba5993ef_0] Write session will restart in 0.005174s
2025-10-09T21:49:20.727552Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19aed3d8-37bcca1d-705e5edd-ba5993ef_0] Write session: Do CDS request
2025-10-09T21:49:20.727571Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19aed3d8-37bcca1d-705e5edd-ba5993ef_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:20.733401Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19aed3d8-37bcca1d-705e5edd-ba5993ef_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:20.733516Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:20.733530Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1487
2025-10-09T21:49:20.733660Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1487 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:20.733693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1487 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:20.733699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1487 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:20.733703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:20.734159Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1487 sessionId:  describe result for acl check
2025-10-09T21:49:20.734196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:20.734203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:20.734205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:20.734210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342415887014014:2699] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:20.734747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342415887014014:2699] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:20.737207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342415887014014:2699] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:20.737342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:20.741748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:20.741952Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:20.741965Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:20.741977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:20.742056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:20.742102Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342415525188438:5546]
2025-10-09T21:49:20.742195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342415887014014:2699] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:20.742209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342415887014014:2699] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:20.742212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342415887014014:2699] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:20.742221Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1487 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:20.742637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:20.742924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:20.742970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:20.743265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:20.743317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.743323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.743325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.743326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.743327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.743328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.743329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.743330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.743331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.743336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.743346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.743351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.743355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.743359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.743363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.743367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.743371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.743374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.743381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.743390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.743396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.743401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.743405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.743411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.743415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.743425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.743429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.743434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.743438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.743444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.743452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.743456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.743460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.743465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.743474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.743479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.743510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:20.750541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:20.751468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:20.751485Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:20.751488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:20.751492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:20.751498Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1486 [2:7559342415525188438:5546]
2025-10-09T21:49:20.751511Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:20.751527Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:20.751529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:20.751531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:20.751532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:20.751546Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:20.751554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:20.751559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:20.751596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.751601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.751602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.751604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:20.751656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:20.751660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.751672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.751682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.751704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:20.751730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342415887014062:2699], now have 1 active actors on pipe
2025-10-09T21:49:20.751733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342415887014001:2432], now have 1 active actors on pipe
2025-10-09T21:49:20.751741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342415525188432:5546]
2025-10-09T21:49:20.751743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342415525188432:5546]
2025-10-09T21:49:20.751745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342415525188432:5546]
2025-10-09T21:49:20.751759Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:20.751772Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:20.751790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:20.751795Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:20.751798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:20.751799Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:20.751801Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:20.751820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342415525188432:5546]
2025-10-09T21:49:20.751828Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:20.751816Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1486
2025-10-09T21:49:20.751820Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1486
2025-10-09T21:49:20.751839Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:20.752074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:20.752090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:20.752119Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|575e806d-fd9c922d-ec5fb445-a392b365_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:20.752128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:20.752137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:20.752139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:20.752149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:20.752163Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:20.752184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:20.752354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:20.752361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:20.752374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:20.752462Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1487 partition: 0 MaxSeqNo: 25 sessionId: src_id|575e806d-fd9c922d-ec5fb445-a392b365_0
2025-10-09T21:49:20.752801Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19aed3d8-37bcca1d-705e5edd-ba5993ef_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|575e806d-fd9c922d-ec5fb445-a392b365_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:20.755148Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|575e806d-fd9c922d-ec5fb445-a392b365_0] Write session: try to update token
2025-10-09T21:49:20.755167Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|575e806d-fd9c922d-ec5fb445-a392b365_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:20.767234Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1487 sessionId: src_id|575e806d-fd9c922d-ec5fb445-a392b365_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:20.768261Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:20.768279Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:20.768281Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:20.768283Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:20.768291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342415525188432:5546]
2025-10-09T21:49:20.768297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342415525188432:5546]
2025-10-09T21:49:20.768301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342415525188432:5546]
2025-10-09T21:49:20.768346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:20.768357Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:20.768360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.768362Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:20.768362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:20.768365Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:20.768497Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:20.769631Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:20.827393Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342415887014073:3441]
2025-10-09T21:49:20.827628Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 1486, pipe: [1:7559342415887014073:3441]
2025-10-09T21:49:20.883161Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:49:20.883193Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 13
2025-10-09T21:49:20.913213Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:20.913699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:20.913717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:20.913762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342415887014073:3441], now have 1 active actors on pipe
2025-10-09T21:49:20.914011Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:20.914098Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:20.914122Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342415887014073:3441]
2025-10-09T21:49:20.914126Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 13
2025-10-09T21:49:20.914158Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:20.914244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1487 reason: status is not ok: tablet will be restarted right now sessionId: src_id|575e806d-fd9c922d-ec5fb445-a392b365_0
2025-10-09T21:49:20.914408Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1487 sessionId: src_id|575e806d-fd9c922d-ec5fb445-a392b365_0 is DEAD
2025-10-09T21:49:20.914584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|575e806d-fd9c922d-ec5fb445-a392b365_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:20.914597Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|575e806d-fd9c922d-ec5fb445-a392b365_0] Write session will restart in 0.009984s
2025-10-09T21:49:20.914622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|575e806d-fd9c922d-ec5fb445-a392b365_0] Write session: Do CDS request
2025-10-09T21:49:20.914629Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|575e806d-fd9c922d-ec5fb445-a392b365_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:20.923297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:20.925557Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|575e806d-fd9c922d-ec5fb445-a392b365_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:20.925629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:20.925645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1488
2025-10-09T21:49:20.925792Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1488 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:20.925838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1488 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:20.925846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1488 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:20.925850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:20.926263Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1488 sessionId:  describe result for acl check
2025-10-09T21:49:20.926299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:20.926300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:20.926302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:20.926308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342415887014120:2695] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:20.926398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:20.926596Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:20.926606Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:20.926618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:20.926670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:20.926712Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342415525188503:5542]
2025-10-09T21:49:20.926834Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342415887014120:2695] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:20.927272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:20.927620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:20.927669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:20.927974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:20.928031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.928038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.928039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.928040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.928041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.928042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.928043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.928044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.928046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.928051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.928061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.928065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.928068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.928071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.928076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.928080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.928083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.928087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.928094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.928101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:20.928107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.928112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:20.928116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.928123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:20.928127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.928132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:20.928142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.928147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:20.928151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.928155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:20.928160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.928164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.928168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.928178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.928182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.928186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.928216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:20.929257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342415887014120:2695] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:20.935412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342415887014120:2695] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:20.935429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342415887014120:2695] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:20.935433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342415887014120:2695] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:20.935443Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1488 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:20.935703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:20.936711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:20.936727Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:20.936729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:20.936733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:20.936740Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1487 [2:7559342415525188503:5542]
2025-10-09T21:49:20.937000Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:20.937023Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:20.937026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:20.937029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:20.937031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:20.937049Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:20.937055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:20.937062Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:20.937121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.937130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.937131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.937134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:20.937195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:20.937207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.937218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:20.937230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:20.937246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:20.937267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342415887014157:2695], now have 1 active actors on pipe
2025-10-09T21:49:20.937273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342415887014080:2432], now have 1 active actors on pipe
2025-10-09T21:49:20.937280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342415525188497:5542]
2025-10-09T21:49:20.937282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342415525188497:5542]
2025-10-09T21:49:20.937284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342415525188497:5542]
2025-10-09T21:49:20.937299Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:20.937312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:20.937319Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:20.937323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:20.937325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:20.937325Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:20.937329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:20.937356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342415525188497:5542]
2025-10-09T21:49:20.937341Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1487
2025-10-09T21:49:20.937358Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:20.937345Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1487
2025-10-09T21:49:20.937372Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:20.937401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:20.937412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:20.937479Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:49:20.937512Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cc83e51b-21d0b941-846d636c-27e44c02_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:20.937525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:20.937534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:20.937537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:20.937546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:20.937555Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:20.937579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:20.937737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:20.937746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:20.937760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:20.937743Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 17 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:49:20.937797Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:49:20.937854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1488 partition: 0 MaxSeqNo: 25 sessionId: src_id|cc83e51b-21d0b941-846d636c-27e44c02_0
2025-10-09T21:49:20.938217Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|575e806d-fd9c922d-ec5fb445-a392b365_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cc83e51b-21d0b941-846d636c-27e44c02_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:20.941202Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc83e51b-21d0b941-846d636c-27e44c02_0] Write session: try to update token
2025-10-09T21:49:20.941222Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc83e51b-21d0b941-846d636c-27e44c02_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:20.954795Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:20.954815Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:20.954817Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:20.954819Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:20.954828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342415525188497:5542]
2025-10-09T21:49:20.954833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342415525188497:5542]
2025-10-09T21:49:20.954837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342415525188497:5542]
2025-10-09T21:49:20.954881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:20.954883Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:20.954888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:20.954888Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:20.954891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:20.954891Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:20.955026Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:20.956456Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1488 sessionId: src_id|cc83e51b-21d0b941-846d636c-27e44c02_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:20.967725Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:21.014273Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342420181981466:3441]
2025-10-09T21:49:21.014557Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 1487, pipe: [1:7559342420181981466:3441]
2025-10-09T21:49:21.107583Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:21.107863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.107878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.107899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342420181981466:3441], now have 1 active actors on pipe
2025-10-09T21:49:21.108210Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:21.108229Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:21.108233Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:21.108279Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342420181981466:3441]
2025-10-09T21:49:21.108305Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1488 reason: error at write: Disconnected sessionId: src_id|cc83e51b-21d0b941-846d636c-27e44c02_0
2025-10-09T21:49:21.108477Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1488 sessionId: src_id|cc83e51b-21d0b941-846d636c-27e44c02_0 is DEAD
2025-10-09T21:49:21.108655Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc83e51b-21d0b941-846d636c-27e44c02_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:21.108665Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc83e51b-21d0b941-846d636c-27e44c02_0] Write session will restart in 0.007830s
2025-10-09T21:49:21.108687Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc83e51b-21d0b941-846d636c-27e44c02_0] Write session: Do CDS request
2025-10-09T21:49:21.108697Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc83e51b-21d0b941-846d636c-27e44c02_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:21.108891Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:21.116680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:21.117539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc83e51b-21d0b941-846d636c-27e44c02_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:21.117543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:21.117618Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:21.117654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1489
2025-10-09T21:49:21.117746Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:21.117758Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:21.117770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:21.117782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1489 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:21.117811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1489 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:21.117817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1489 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:21.117821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:21.117830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:21.117877Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342419820155865:5540]
2025-10-09T21:49:21.118291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1489 sessionId:  describe result for acl check
2025-10-09T21:49:21.118335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:21.118337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:21.118339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:21.118348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342420181981513:2686] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:21.118493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:21.118810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:21.118859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:21.118902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342420181981513:2686] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:21.119162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:21.119219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.119220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.119221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.119222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.119223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.119225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.119226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.119227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.119228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.119233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.119243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.119246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.119250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.119254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.119257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.119261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.119264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.119267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.119273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.119281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.119288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.119293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.119297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.119303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.119307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.119312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.119315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.119319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.119324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.119328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.119333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.119338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.119342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.119347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.119351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.119357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.119395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:21.121053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342420181981513:2686] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:21.125241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342420181981513:2686] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:21.125256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342420181981513:2686] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:21.125259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342420181981513:2686] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:21.125269Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1489 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:21.125687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:21.126902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:21.126917Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:21.126919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:21.126924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:21.126931Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1488 [2:7559342419820155865:5540]
2025-10-09T21:49:21.126945Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:21.126962Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:21.126970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:21.126973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:21.126976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:21.126993Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:21.127001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:21.127008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:21.127067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.127074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.127076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.127078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:21.127308Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1488
2025-10-09T21:49:21.127310Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1488
2025-10-09T21:49:21.127372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:21.127381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.127393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.127407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.127429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:21.127457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342420181981552:2686], now have 1 active actors on pipe
2025-10-09T21:49:21.127466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342420181981473:2432], now have 1 active actors on pipe
2025-10-09T21:49:21.127485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.127494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.127513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342419820155860:5540]
2025-10-09T21:49:21.127522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342419820155860:5540]
2025-10-09T21:49:21.127524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342419820155860:5540]
2025-10-09T21:49:21.127537Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:21.127551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:21.127563Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:21.127570Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:21.127574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:21.127574Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:21.127576Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:21.127594Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9a1f66e-1e03d9eb-5be386f3-90d860fd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:21.127602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:21.127611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:21.127614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:21.127623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:21.127634Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:21.127674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342419820155860:5540]
2025-10-09T21:49:21.127686Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:21.127694Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:21.127699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:21.127853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.127862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.127877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:21.128008Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1489 partition: 0 MaxSeqNo: 25 sessionId: src_id|9a1f66e-1e03d9eb-5be386f3-90d860fd_0
2025-10-09T21:49:21.128356Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc83e51b-21d0b941-846d636c-27e44c02_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9a1f66e-1e03d9eb-5be386f3-90d860fd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:21.131174Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9a1f66e-1e03d9eb-5be386f3-90d860fd_0] Write session: try to update token
2025-10-09T21:49:21.131192Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9a1f66e-1e03d9eb-5be386f3-90d860fd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:21.139820Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:21.139839Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:21.139849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342419820155860:5540]
2025-10-09T21:49:21.139905Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:21.139931Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:21.140077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:21.140212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:21.140347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:21.140486Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:21.140676Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:21.140721Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:21.143980Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1489 sessionId: src_id|9a1f66e-1e03d9eb-5be386f3-90d860fd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:21.146516Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:21.146747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.146767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.146805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:21.146933Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:21.153923Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:21.153938Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:21.153941Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:21.153943Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:21.153952Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342419820155860:5540]
2025-10-09T21:49:21.153956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342419820155860:5540]
2025-10-09T21:49:21.153960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342419820155860:5540]
2025-10-09T21:49:21.154005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:21.154012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.154014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:21.154013Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:21.154040Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:21.154045Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:21.154173Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:21.208332Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342420181981563:3441]
2025-10-09T21:49:21.208687Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 1488, pipe: [1:7559342420181981563:3441]
2025-10-09T21:49:21.303830Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:21.310498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.310520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.310647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:21.310701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:21.310757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:21.310823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:21.310864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:21.310872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:21.310932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:21.310983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:21.311051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:21.311122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:21.311167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:21.311177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:21.311238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:21.311287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:21.311347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:21.311412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:21.311453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:21.311460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:21.311527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:21.311590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:21.311642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:21.311692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:21.311730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:21.311736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:21.311805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:21.311864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:21.311920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:21.311974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:21.312012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:21.312017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:21.312076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:21.312140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:21.312197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:21.312250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:21.312288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:21.312294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:21.312345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:21.312391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:21.312442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:21.312493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:21.312576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:21.312582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:21.312632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:21.312688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:21.312737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:21.312788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:21.312825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:21.312830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:21.312889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:21.312945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:21.313000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:21.313051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:21.313092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:21.313099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:21.313160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:21.313216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:21.313269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:21.313329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:21.313372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:21.313379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:21.313427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342420181981563:3441], now have 1 active actors on pipe
2025-10-09T21:49:21.313839Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:21.313853Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:21.313867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:21.313899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1489 reason: error at write: Disconnected sessionId: src_id|9a1f66e-1e03d9eb-5be386f3-90d860fd_0
2025-10-09T21:49:21.313905Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342420181981563:3441]
2025-10-09T21:49:21.314106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1489 sessionId: src_id|9a1f66e-1e03d9eb-5be386f3-90d860fd_0 is DEAD
2025-10-09T21:49:21.314280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a1f66e-1e03d9eb-5be386f3-90d860fd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:21.314292Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a1f66e-1e03d9eb-5be386f3-90d860fd_0] Write session will restart in 0.006050s
2025-10-09T21:49:21.314317Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a1f66e-1e03d9eb-5be386f3-90d860fd_0] Write session: Do CDS request
2025-10-09T21:49:21.314302Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:21.314336Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a1f66e-1e03d9eb-5be386f3-90d860fd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:21.321505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9a1f66e-1e03d9eb-5be386f3-90d860fd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:21.321606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:21.321628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1490
2025-10-09T21:49:21.321787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1490 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:21.321826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1490 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:21.321836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1490 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:21.321840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:21.322335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1490 sessionId:  describe result for acl check
2025-10-09T21:49:21.322376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:21.322383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:21.322385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:21.322391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342420181981606:2676] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:21.322993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342420181981606:2676] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:21.323665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:21.325401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:21.325607Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:21.325618Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:21.325629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:21.325678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:21.325719Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342419820155932:5538]
2025-10-09T21:49:21.325928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342420181981606:2676] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:21.326368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:21.326669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:21.326710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:21.327027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:21.327081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.327088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.327089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.327090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.327092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.327093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.327094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.327096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.327097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.327102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.327113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.327117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.327122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.327126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.327130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.327134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.327138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.327141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.327150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.327158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.327164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.327170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.327175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.327180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.327184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.327192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.327197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.327203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.327207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.327213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.327219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.327224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.327229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.327234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.327240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.327246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.327275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:21.332796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342420181981606:2676] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:21.332821Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342420181981606:2676] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:21.332825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342420181981606:2676] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:21.332833Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1490 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:21.334133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:21.335713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:21.335734Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:21.335737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:21.335741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:21.335749Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1489 [2:7559342419820155932:5538]
2025-10-09T21:49:21.335768Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:21.335784Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:21.335787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:21.335790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:21.335793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:21.335809Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:21.335815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:21.335822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:21.335890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.335902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.335904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.335907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:21.335977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:21.335988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.336000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.336010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.336028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:21.336047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342420181981570:2432], now have 1 active actors on pipe
2025-10-09T21:49:21.336057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342420181981649:2676], now have 1 active actors on pipe
2025-10-09T21:49:21.336063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342419820155927:5538]
2025-10-09T21:49:21.336066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342419820155927:5538]
2025-10-09T21:49:21.336069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342419820155927:5538]
2025-10-09T21:49:21.336082Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:21.336095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:21.336096Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:21.336100Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:21.336104Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:21.336111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:21.336115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:21.336104Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1489
2025-10-09T21:49:21.336146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342419820155927:5538]
2025-10-09T21:49:21.336107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1489
2025-10-09T21:49:21.336156Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:21.336158Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:21.336180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.336184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.336216Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b112e2-da366abc-450239d8-670e1766_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:21.336228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:21.336236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:21.336238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:21.336247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:21.336256Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:21.336271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:21.336494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.336506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.336520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:21.336632Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1490 partition: 0 MaxSeqNo: 25 sessionId: src_id|b112e2-da366abc-450239d8-670e1766_0
2025-10-09T21:49:21.337043Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a1f66e-1e03d9eb-5be386f3-90d860fd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b112e2-da366abc-450239d8-670e1766_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:21.339592Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b112e2-da366abc-450239d8-670e1766_0] Write session: try to update token
2025-10-09T21:49:21.339612Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b112e2-da366abc-450239d8-670e1766_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:21.350938Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:21.350963Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:21.350966Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:21.350969Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:21.350978Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342419820155927:5538]
2025-10-09T21:49:21.350983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342419820155927:5538]
2025-10-09T21:49:21.350988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342419820155927:5538]
2025-10-09T21:49:21.351043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:21.351047Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:21.351054Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:21.351057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.351059Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:21.351060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:21.351218Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:21.354285Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1490 sessionId: src_id|b112e2-da366abc-450239d8-670e1766_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:21.357049Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:21.414139Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342420181981656:3441]
2025-10-09T21:49:21.414457Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 1489, pipe: [1:7559342420181981656:3441]
2025-10-09T21:49:21.504250Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:21.504694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.504714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.504763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342420181981656:3441], now have 1 active actors on pipe
2025-10-09T21:49:21.504952Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:21.505042Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342420181981656:3441]
2025-10-09T21:49:21.505090Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:21.505118Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:21.505171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1490 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b112e2-da366abc-450239d8-670e1766_0
2025-10-09T21:49:21.505363Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1490 sessionId: src_id|b112e2-da366abc-450239d8-670e1766_0 is DEAD
2025-10-09T21:49:21.505524Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b112e2-da366abc-450239d8-670e1766_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:21.505536Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b112e2-da366abc-450239d8-670e1766_0] Write session will restart in 0.008338s
2025-10-09T21:49:21.505584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b112e2-da366abc-450239d8-670e1766_0] Write session: Do CDS request
2025-10-09T21:49:21.505590Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b112e2-da366abc-450239d8-670e1766_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:21.515453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:21.515514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b112e2-da366abc-450239d8-670e1766_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:21.515636Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:21.515653Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1491
2025-10-09T21:49:21.515783Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1491 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:21.515821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1491 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:21.515827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1491 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:21.515831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:21.516285Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1491 sessionId:  describe result for acl check
2025-10-09T21:49:21.516325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:21.516332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:21.516334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:21.516340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342420181981699:2672] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:21.516563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:21.516742Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:21.516754Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:21.516764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:21.516811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:21.516854Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342419820155999:5532]
2025-10-09T21:49:21.517000Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342420181981699:2672] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:21.517523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:21.517949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:21.518007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:21.518392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:21.518472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.518483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.518485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.518486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.518487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.518488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.518490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.518491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.518492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.518498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.518509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.518512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.518516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.518520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.518524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.518527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.518531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.518534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.518542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.518549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.518555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.518560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.518565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.518570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.518575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.518580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.518586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.518590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.518596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.518601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.518607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.518619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.518626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.518631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.518635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.518659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.518695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:21.519105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342420181981699:2672] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:21.524292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342420181981699:2672] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:21.524308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342420181981699:2672] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:21.524320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342420181981699:2672] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:21.524329Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1491 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:21.528428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:21.529682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:21.529697Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:21.529700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:21.529704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:21.529711Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1490 [2:7559342419820155999:5532]
2025-10-09T21:49:21.529728Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:21.529745Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:21.529748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:21.529751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:21.529754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:21.529769Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:21.529781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:21.529787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:21.529855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.529865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.529868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.529871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:21.529954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:21.529963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.529975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.529994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.530011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:21.530051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342420181981659:2432], now have 1 active actors on pipe
2025-10-09T21:49:21.530066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342420181981738:2672], now have 1 active actors on pipe
2025-10-09T21:49:21.530109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342419820155994:5532]
2025-10-09T21:49:21.530100Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1490
2025-10-09T21:49:21.530125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342419820155994:5532]
2025-10-09T21:49:21.530133Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1490
2025-10-09T21:49:21.530132Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342419820155994:5532]
2025-10-09T21:49:21.530155Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:21.530162Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:21.530165Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:21.530167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:21.530187Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:21.530210Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:21.530229Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:21.530255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.530266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.530330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342419820155994:5532]
2025-10-09T21:49:21.530344Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:21.530349Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:21.530362Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|56770ada-a2729662-13cc593d-27dc9db4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:21.530377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:21.530386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:21.530388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:21.530398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:21.530413Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:21.530450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:21.530605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.530612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.530622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:21.530726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1491 partition: 0 MaxSeqNo: 25 sessionId: src_id|56770ada-a2729662-13cc593d-27dc9db4_0
2025-10-09T21:49:21.531058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b112e2-da366abc-450239d8-670e1766_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|56770ada-a2729662-13cc593d-27dc9db4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:21.540593Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56770ada-a2729662-13cc593d-27dc9db4_0] Write session: try to update token
2025-10-09T21:49:21.540617Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56770ada-a2729662-13cc593d-27dc9db4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:21.547125Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:21.547144Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:21.547147Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:21.547149Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:21.547162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342419820155994:5532]
2025-10-09T21:49:21.547167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342419820155994:5532]
2025-10-09T21:49:21.547172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342419820155994:5532]
2025-10-09T21:49:21.547217Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:21.547219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:21.547223Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:21.547226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.547228Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:21.547229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:21.547357Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:21.553915Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1491 sessionId: src_id|56770ada-a2729662-13cc593d-27dc9db4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:21.560558Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:21.605108Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342420181981745:3441]
2025-10-09T21:49:21.605363Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 1490, pipe: [1:7559342420181981745:3441]
2025-10-09T21:49:21.696940Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:21.697514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.697530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.697552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342420181981745:3441], now have 1 active actors on pipe
2025-10-09T21:49:21.697860Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:21.697893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:21.698000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1491 reason: error at write: Disconnected sessionId: src_id|56770ada-a2729662-13cc593d-27dc9db4_0
2025-10-09T21:49:21.698062Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342420181981745:3441]
2025-10-09T21:49:21.698109Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:21.698246Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:21.698260Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1491 sessionId: src_id|56770ada-a2729662-13cc593d-27dc9db4_0 is DEAD
2025-10-09T21:49:21.698394Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56770ada-a2729662-13cc593d-27dc9db4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:21.698405Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56770ada-a2729662-13cc593d-27dc9db4_0] Write session will restart in 0.005957s
2025-10-09T21:49:21.698436Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56770ada-a2729662-13cc593d-27dc9db4_0] Write session: Do CDS request
2025-10-09T21:49:21.698442Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56770ada-a2729662-13cc593d-27dc9db4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:21.705481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56770ada-a2729662-13cc593d-27dc9db4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:21.705588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:21.705616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1492
2025-10-09T21:49:21.705797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1492 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:21.705824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1492 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:21.705828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1492 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:21.705831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:21.706073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:21.706298Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1492 sessionId:  describe result for acl check
2025-10-09T21:49:21.706338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:21.706345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:21.706347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:21.706354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342420181981812:2656] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:21.706974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342420181981812:2656] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:21.707333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:21.707507Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:21.707516Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:21.707528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:21.707577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:21.707612Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342419820156109:5513]
2025-10-09T21:49:21.708206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:21.708495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:21.708539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:21.708667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342420181981812:2656] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:21.708813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:21.708859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.708865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.708866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.708868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.708869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.708870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.708871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.708872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.708873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.708877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.708887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.708891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.708895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.708900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.708904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.708907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.708910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.708914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.708920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.708928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.708933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.708938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.708943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.708948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.708952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.708957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.708967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.708972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.708976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.708981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.708987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.708992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.708999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.709003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.709008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.709013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.709044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:21.715374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:21.715626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342420181981812:2656] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:21.715643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342420181981812:2656] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:21.715646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342420181981812:2656] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:21.715653Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1492 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:21.716499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:21.716514Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:21.716516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:21.716520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:21.716526Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1491 [2:7559342419820156109:5513]
2025-10-09T21:49:21.716539Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:21.716557Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:21.716563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:21.716566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:21.716569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:21.716586Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:21.716596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:21.716603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:21.716660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.716667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.716669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.716672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:21.716749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:21.716757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.716769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.716782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.716804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:21.716828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342420181981772:2432], now have 1 active actors on pipe
2025-10-09T21:49:21.716836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342420181981848:2656], now have 1 active actors on pipe
2025-10-09T21:49:21.716843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342419820156104:5513]
2025-10-09T21:49:21.716849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342419820156104:5513]
2025-10-09T21:49:21.716819Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1491
2025-10-09T21:49:21.716852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342419820156104:5513]
2025-10-09T21:49:21.716861Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:21.716848Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1491
2025-10-09T21:49:21.716868Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:21.716870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:21.716872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:21.716874Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:21.716889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:21.716893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:21.716934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.716944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.716960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342419820156104:5513]
2025-10-09T21:49:21.716973Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:21.716979Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:21.717009Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8d2f1a87-bca2fb65-c9feb902-d441318d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:21.717021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:21.717025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:21.717031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:21.717041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:21.717051Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:21.717072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:21.717228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.717242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.717263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:21.717348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1492 partition: 0 MaxSeqNo: 25 sessionId: src_id|8d2f1a87-bca2fb65-c9feb902-d441318d_0
2025-10-09T21:49:21.717697Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56770ada-a2729662-13cc593d-27dc9db4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8d2f1a87-bca2fb65-c9feb902-d441318d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:21.720150Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8d2f1a87-bca2fb65-c9feb902-d441318d_0] Write session: try to update token
2025-10-09T21:49:21.720169Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8d2f1a87-bca2fb65-c9feb902-d441318d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:21.732308Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:21.732326Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:21.732329Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:21.732331Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:21.732340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342419820156104:5513]
2025-10-09T21:49:21.732345Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342419820156104:5513]
2025-10-09T21:49:21.732350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342419820156104:5513]
2025-10-09T21:49:21.732394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:21.732402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.732403Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:21.732405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:21.732409Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:21.732414Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:21.732539Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:21.733120Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1492 sessionId: src_id|8d2f1a87-bca2fb65-c9feb902-d441318d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:21.735644Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:21.798136Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342420181981859:3441]
2025-10-09T21:49:21.798436Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 1491, pipe: [1:7559342420181981859:3441]
2025-10-09T21:49:21.881943Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:21.882232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.882249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.882293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342420181981859:3441], now have 1 active actors on pipe
2025-10-09T21:49:21.882543Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:21.882559Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:21.882575Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:21.882577Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:21.882637Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342420181981859:3441]
2025-10-09T21:49:21.882642Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1492 reason: error at write: Disconnected sessionId: src_id|8d2f1a87-bca2fb65-c9feb902-d441318d_0
2025-10-09T21:49:21.882811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1492 sessionId: src_id|8d2f1a87-bca2fb65-c9feb902-d441318d_0 is DEAD
2025-10-09T21:49:21.882976Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d2f1a87-bca2fb65-c9feb902-d441318d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:21.882986Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d2f1a87-bca2fb65-c9feb902-d441318d_0] Write session will restart in 0.007282s
2025-10-09T21:49:21.883007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d2f1a87-bca2fb65-c9feb902-d441318d_0] Write session: Do CDS request
2025-10-09T21:49:21.883013Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d2f1a87-bca2fb65-c9feb902-d441318d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:21.891494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8d2f1a87-bca2fb65-c9feb902-d441318d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:21.891571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:21.891587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1493
2025-10-09T21:49:21.891713Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1493 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:21.891746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1493 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:21.891752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1493 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:21.891757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:21.892060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:21.892154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1493 sessionId:  describe result for acl check
2025-10-09T21:49:21.892185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:21.892189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:21.892191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:21.892195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342420181981905:2649] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:21.892834Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342420181981905:2649] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:21.893728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:21.893878Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:21.893885Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:21.893893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:21.893937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:21.893963Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342419820156178:5512]
2025-10-09T21:49:21.894432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:21.894679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342420181981905:2649] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:21.894746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:21.894803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:21.895082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:21.895131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.895135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.895135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.895136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.895137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.895137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.895138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.895138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.895139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.895142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.895149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.895151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.895153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.895155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.895165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.895167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.895169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.895171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.895175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.895180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:21.895186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.895189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:21.895192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.895195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:21.895197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.895200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:21.895202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.895205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:21.895209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.895212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:21.895214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.895217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.895219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.895222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.895225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.895228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.895243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:21.900222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342420181981905:2649] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:21.900236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342420181981905:2649] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:21.900239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342420181981905:2649] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:21.900248Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1493 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:21.901524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:21.902648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:21.902664Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:21.902667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:21.902670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:21.902676Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1492 [2:7559342419820156178:5512]
2025-10-09T21:49:21.902691Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:21.902706Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:21.902709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:21.902712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:21.902714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:21.902730Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:21.902740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:21.902748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:21.902800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.902808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.902809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.902812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:21.903019Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1492
2025-10-09T21:49:21.903048Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1492
2025-10-09T21:49:21.903076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:21.903084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.903092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:21.903098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:21.903109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:21.903130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342420181981864:2432], now have 1 active actors on pipe
2025-10-09T21:49:21.903137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342420181981945:2649], now have 1 active actors on pipe
2025-10-09T21:49:21.903156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.903165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.903183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342419820156173:5512]
2025-10-09T21:49:21.903190Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342419820156173:5512]
2025-10-09T21:49:21.903192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342419820156173:5512]
2025-10-09T21:49:21.903208Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:21.903223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:21.903234Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:21.903240Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:21.903242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:21.903243Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:21.903245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:21.903265Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|86a5e034-7fd6cc2b-6f424d06-dc905e6c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:21.903280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:21.903288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:21.903291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:21.903301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:21.903308Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:21.903338Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342419820156173:5512]
2025-10-09T21:49:21.903349Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:21.903358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:21.903362Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:21.903570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.903581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.903596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:21.903709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1493 partition: 0 MaxSeqNo: 25 sessionId: src_id|86a5e034-7fd6cc2b-6f424d06-dc905e6c_0
2025-10-09T21:49:21.903984Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8d2f1a87-bca2fb65-c9feb902-d441318d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|86a5e034-7fd6cc2b-6f424d06-dc905e6c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:21.906329Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86a5e034-7fd6cc2b-6f424d06-dc905e6c_0] Write session: try to update token
2025-10-09T21:49:21.906343Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86a5e034-7fd6cc2b-6f424d06-dc905e6c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:21.914821Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:21.914837Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:21.914849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342419820156173:5512]
2025-10-09T21:49:21.914896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:21.914900Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:21.915060Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:21.915198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:21.915332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:21.915457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:21.915687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:21.915725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:21.918555Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1493 sessionId: src_id|86a5e034-7fd6cc2b-6f424d06-dc905e6c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:21.921105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:21.921316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:21.921335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:21.921376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:21.921477Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:21.930988Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:21.931000Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:21.931003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:21.931005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:21.931015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342419820156173:5512]
2025-10-09T21:49:21.931018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342419820156173:5512]
2025-10-09T21:49:21.931022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342419820156173:5512]
2025-10-09T21:49:21.931062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:21.931068Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:21.931075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:21.931076Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:21.931078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:21.931080Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:21.931208Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:21.982997Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342420181981952:3441]
2025-10-09T21:49:21.983341Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 1492, pipe: [1:7559342420181981952:3441]
2025-10-09T21:49:22.079842Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:22.086389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:22.086414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:22.086561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:22.086618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:22.086675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:22.086732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:22.086775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:22.086783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:22.086854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:22.086912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:22.086966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:22.087023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:22.087065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:22.087072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:22.087133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:22.087183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:22.087234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:22.087284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:22.087329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:22.087337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:22.087397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:22.087450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:22.087503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:22.087557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:22.087603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:22.087611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:22.087670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:22.087725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:22.087780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:22.087833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:22.087880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:22.087886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:22.087941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:22.087994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:22.088048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:22.088113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:22.088153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:22.088159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:22.088211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:22.088261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:22.088315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:22.088377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:22.088416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:22.088422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:22.088475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:22.088524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:22.088578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:22.088638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:22.088677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:22.088682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:22.088734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:22.088782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:22.088835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:22.088893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:22.088933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:22.088938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:22.088992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:22.089042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:22.089097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:22.089161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:22.089199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:22.089205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:22.089245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342420181981952:3441], now have 1 active actors on pipe
2025-10-09T21:49:22.089520Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:22.089528Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342420181981952:3441]
2025-10-09T21:49:22.089561Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:22.089572Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:22.089614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1493 reason: error at write: Disconnected sessionId: src_id|86a5e034-7fd6cc2b-6f424d06-dc905e6c_0
2025-10-09T21:49:22.089803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1493 sessionId: src_id|86a5e034-7fd6cc2b-6f424d06-dc905e6c_0 is DEAD
2025-10-09T21:49:22.089943Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:22.089985Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86a5e034-7fd6cc2b-6f424d06-dc905e6c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:22.089996Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86a5e034-7fd6cc2b-6f424d06-dc905e6c_0] Write session will restart in 0.006393s
2025-10-09T21:49:22.090014Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86a5e034-7fd6cc2b-6f424d06-dc905e6c_0] Write session: Do CDS request
2025-10-09T21:49:22.090019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86a5e034-7fd6cc2b-6f424d06-dc905e6c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:22.097447Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86a5e034-7fd6cc2b-6f424d06-dc905e6c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:22.097514Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:22.097529Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1494
2025-10-09T21:49:22.097658Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1494 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:22.097684Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1494 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:22.097691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1494 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:22.097694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:22.098144Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1494 sessionId:  describe result for acl check
2025-10-09T21:49:22.098186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:22.098194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:22.098196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:22.098201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342424476949291:2633] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:22.098698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342424476949291:2633] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:22.099354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:22.101114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:22.101166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342424476949291:2633] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:22.101315Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:22.101328Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:22.101339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:22.101388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:22.101433Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342424115123544:5506]
2025-10-09T21:49:22.102114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:22.102406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:22.102451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:22.102805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:22.102858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.102866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.102868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.102869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.102871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.102872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.102873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.102875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.102876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.102881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.102892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.102897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.102901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.102905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.102909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.102914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.102918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.102922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.102932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.102940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.102946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.102952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.102957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.102962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.102967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.102974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.102985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.102990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.102992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.102996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.103000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.103004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.103006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.103009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.103012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.103016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.103033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:22.105619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342424476949291:2633] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:22.105646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342424476949291:2633] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:22.105650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342424476949291:2633] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:22.105657Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1494 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:22.109663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:22.110818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:22.110834Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:22.110837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:22.110841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:22.110848Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1493 [2:7559342424115123544:5506]
2025-10-09T21:49:22.110862Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:22.110880Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:22.110883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:22.110887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:22.110889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:22.110908Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:22.110920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:22.110928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:22.110983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.110990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.110992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.110995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:22.111065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:22.111072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.111084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.111103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.111121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:22.111144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342424476949333:2633], now have 1 active actors on pipe
2025-10-09T21:49:22.111153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342424476949251:2432], now have 1 active actors on pipe
2025-10-09T21:49:22.111161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342424115123539:5506]
2025-10-09T21:49:22.111163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342424115123539:5506]
2025-10-09T21:49:22.111165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342424115123539:5506]
2025-10-09T21:49:22.111178Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:22.111191Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:22.111194Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:22.111196Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:22.111197Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:22.111212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:22.111219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:22.111211Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1493
2025-10-09T21:49:22.111216Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1493
2025-10-09T21:49:22.111255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342424115123539:5506]
2025-10-09T21:49:22.111263Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:22.111265Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:22.111281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:22.111290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:22.111342Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7d36f10-278426cf-2a91d877-2499244f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:22.111355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:22.111360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:22.111364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:22.111380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:22.111394Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:22.111415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:22.111631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:22.111649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:22.111684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:22.111797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1494 partition: 0 MaxSeqNo: 25 sessionId: src_id|7d36f10-278426cf-2a91d877-2499244f_0
2025-10-09T21:49:22.112142Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86a5e034-7fd6cc2b-6f424d06-dc905e6c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7d36f10-278426cf-2a91d877-2499244f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:22.114586Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7d36f10-278426cf-2a91d877-2499244f_0] Write session: try to update token
2025-10-09T21:49:22.114607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7d36f10-278426cf-2a91d877-2499244f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:22.126002Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:22.126044Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:22.126049Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:22.126051Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:22.126061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342424115123539:5506]
2025-10-09T21:49:22.126066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342424115123539:5506]
2025-10-09T21:49:22.126072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342424115123539:5506]
2025-10-09T21:49:22.126120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:22.126127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.126125Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:22.126130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:22.126131Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:22.126134Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:22.126294Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:22.127588Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1494 sessionId: src_id|7d36f10-278426cf-2a91d877-2499244f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:22.130015Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:22.189809Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342424476949344:3441]
2025-10-09T21:49:22.190153Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 1493, pipe: [1:7559342424476949344:3441]
2025-10-09T21:49:22.282870Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:22.283121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:22.283137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:22.283180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342424476949344:3441], now have 1 active actors on pipe
2025-10-09T21:49:22.283415Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:22.283477Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:22.283524Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342424476949344:3441]
2025-10-09T21:49:22.283549Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:22.283603Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1494 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7d36f10-278426cf-2a91d877-2499244f_0
2025-10-09T21:49:22.283846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1494 sessionId: src_id|7d36f10-278426cf-2a91d877-2499244f_0 is DEAD
2025-10-09T21:49:22.283961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d36f10-278426cf-2a91d877-2499244f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:22.283971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d36f10-278426cf-2a91d877-2499244f_0] Write session will restart in 0.007772s
2025-10-09T21:49:22.283991Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d36f10-278426cf-2a91d877-2499244f_0] Write session: Do CDS request
2025-10-09T21:49:22.283997Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d36f10-278426cf-2a91d877-2499244f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:22.292239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:22.292437Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7d36f10-278426cf-2a91d877-2499244f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:22.292537Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:22.292557Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1495
2025-10-09T21:49:22.292681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1495 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:22.292716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1495 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:22.292724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1495 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:22.292730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:22.293108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1495 sessionId:  describe result for acl check
2025-10-09T21:49:22.293134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:22.293139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:22.293140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:22.293144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342424476949395:2643] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:22.293607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342424476949395:2643] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:22.294157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:22.294298Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:22.294320Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:22.294328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:22.294368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:22.294400Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342424115123615:5500]
2025-10-09T21:49:22.294801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:22.295040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:22.295076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:22.295292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:22.295323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.295327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.295328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.295329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.295330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.295331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.295331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.295332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.295333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.295336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.295343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.295345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.295347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.295349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.295351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.295354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.295356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.295358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.295362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.295368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.295372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.295376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.295379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.295382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.295385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.295388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.295390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.295393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.295396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.295399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.295401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.295405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.295407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.295410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.295413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.295417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.295433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:22.295545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342424476949395:2643] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:22.301570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342424476949395:2643] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:22.301588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342424476949395:2643] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:22.301591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342424476949395:2643] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:22.301612Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1495 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:22.301783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:22.302846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:22.302863Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:22.302865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:22.302870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:22.302876Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1494 [2:7559342424115123615:5500]
2025-10-09T21:49:22.302888Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:22.302905Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:22.302908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:22.302911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:22.302913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:22.302929Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:22.302940Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:22.302947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:22.303003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.303011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.303012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.303015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:22.303087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:22.303094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.303106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.303121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.303141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:22.303167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342424476949434:2643], now have 1 active actors on pipe
2025-10-09T21:49:22.303176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342424476949351:2432], now have 1 active actors on pipe
2025-10-09T21:49:22.303183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342424115123610:5500]
2025-10-09T21:49:22.303190Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342424115123610:5500]
2025-10-09T21:49:22.303192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342424115123610:5500]
2025-10-09T21:49:22.303208Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:22.303226Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:22.303207Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1494
2025-10-09T21:49:22.303208Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1494
2025-10-09T21:49:22.303241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:22.303239Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:22.303244Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:22.303247Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:22.303248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:22.303276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342424115123610:5500]
2025-10-09T21:49:22.303283Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:22.303299Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:22.303304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:22.303308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:22.303343Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5440fee-d5ee87a4-fcc926c5-9c5c80d0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:22.303356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:22.303359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:22.303361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:22.303373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:22.303387Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:22.303410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:22.303543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:22.303549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:22.303562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:22.303720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1495 partition: 0 MaxSeqNo: 25 sessionId: src_id|5440fee-d5ee87a4-fcc926c5-9c5c80d0_0
2025-10-09T21:49:22.304040Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d36f10-278426cf-2a91d877-2499244f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5440fee-d5ee87a4-fcc926c5-9c5c80d0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:22.306419Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5440fee-d5ee87a4-fcc926c5-9c5c80d0_0] Write session: try to update token
2025-10-09T21:49:22.306439Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5440fee-d5ee87a4-fcc926c5-9c5c80d0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:22.317984Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:22.318001Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:22.318004Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:22.318006Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:22.318016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342424115123610:5500]
2025-10-09T21:49:22.318021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342424115123610:5500]
2025-10-09T21:49:22.318025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342424115123610:5500]
2025-10-09T21:49:22.318084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:22.318091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.318094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:22.318119Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:22.318140Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:22.318144Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:22.318219Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:22.318655Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1495 sessionId: src_id|5440fee-d5ee87a4-fcc926c5-9c5c80d0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:22.320936Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:22.383829Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342424476949442:3441]
2025-10-09T21:49:22.384193Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 1494, pipe: [1:7559342424476949442:3441]
2025-10-09T21:49:22.468730Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:22.469183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:22.469196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:22.469233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342424476949442:3441], now have 1 active actors on pipe
2025-10-09T21:49:22.469545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:22.469547Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342424476949442:3441]
2025-10-09T21:49:22.469563Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:22.469575Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:22.469571Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:22.469646Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1495 reason: error at write: Disconnected sessionId: src_id|5440fee-d5ee87a4-fcc926c5-9c5c80d0_0
2025-10-09T21:49:22.469840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1495 sessionId: src_id|5440fee-d5ee87a4-fcc926c5-9c5c80d0_0 is DEAD
2025-10-09T21:49:22.469995Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5440fee-d5ee87a4-fcc926c5-9c5c80d0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:22.470006Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5440fee-d5ee87a4-fcc926c5-9c5c80d0_0] Write session will restart in 0.005876s
2025-10-09T21:49:22.470051Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5440fee-d5ee87a4-fcc926c5-9c5c80d0_0] Write session: Do CDS request
2025-10-09T21:49:22.470058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5440fee-d5ee87a4-fcc926c5-9c5c80d0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:22.477227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:22.477409Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5440fee-d5ee87a4-fcc926c5-9c5c80d0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:22.477517Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:22.477533Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1496
2025-10-09T21:49:22.477661Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1496 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:22.477685Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1496 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:22.477693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1496 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:22.477696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:22.478125Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1496 sessionId:  describe result for acl check
2025-10-09T21:49:22.478164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:22.478165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:22.478167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:22.478174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342424476949489:2636] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:22.478224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:22.478389Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:22.478399Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:22.478410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:22.478460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:22.478495Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342424115123687:5494]
2025-10-09T21:49:22.478655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342424476949489:2636] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:22.478954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:22.479443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:22.479481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:22.479810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:22.479866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.479867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.479867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.479868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.479870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.479871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.479872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.479873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.479874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.479880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.479892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.479896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.479900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.479903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.479907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.479909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.479911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.479914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.479919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.479929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.479934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.479937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.479941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.479944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.479948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.479955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.479961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.479967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.479975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.479982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.479989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.479997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.480001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.480006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.480011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.480016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.480038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:22.480617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342424476949489:2636] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:22.485787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342424476949489:2636] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:22.485803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342424476949489:2636] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:22.485810Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342424476949489:2636] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:22.485818Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1496 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:22.486152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:22.487739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:22.487751Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:22.487752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:22.487757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:22.487762Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1495 [2:7559342424115123687:5494]
2025-10-09T21:49:22.487771Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:22.487785Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:22.487791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:22.487794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:22.487796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:22.487811Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:22.487823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:22.487835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:22.487915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.487919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.487920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.487923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:22.487988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:22.487993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.487999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.488005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.488015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:22.488050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342424476949528:2636], now have 1 active actors on pipe
2025-10-09T21:49:22.488058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342424476949444:2432], now have 1 active actors on pipe
2025-10-09T21:49:22.488065Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342424115123682:5494]
2025-10-09T21:49:22.488071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342424115123682:5494]
2025-10-09T21:49:22.488072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342424115123682:5494]
2025-10-09T21:49:22.488084Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:22.488099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:22.488090Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1495
2025-10-09T21:49:22.488103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:22.488096Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1495
2025-10-09T21:49:22.488109Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:22.488111Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:22.488117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:22.488122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:22.488153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:22.488161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:22.488173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342424115123682:5494]
2025-10-09T21:49:22.488183Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:22.488189Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:22.488207Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8085318f-f6355b9c-19cafdd5-8a73d58d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:22.488213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:22.488217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:22.488219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:22.488225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:22.488234Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:22.488252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:22.488385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:22.488390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:22.488399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:22.488480Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1496 partition: 0 MaxSeqNo: 25 sessionId: src_id|8085318f-f6355b9c-19cafdd5-8a73d58d_0
2025-10-09T21:49:22.488777Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5440fee-d5ee87a4-fcc926c5-9c5c80d0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8085318f-f6355b9c-19cafdd5-8a73d58d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:22.491136Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8085318f-f6355b9c-19cafdd5-8a73d58d_0] Write session: try to update token
2025-10-09T21:49:22.491154Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8085318f-f6355b9c-19cafdd5-8a73d58d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:22.502003Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:22.502025Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:22.502071Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:22.502074Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:22.502086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342424115123682:5494]
2025-10-09T21:49:22.502096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342424115123682:5494]
2025-10-09T21:49:22.502100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342424115123682:5494]
2025-10-09T21:49:22.502150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:22.502153Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:22.502159Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:22.502163Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:22.502179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.502182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:22.502517Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:22.512035Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1496 sessionId: src_id|8085318f-f6355b9c-19cafdd5-8a73d58d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:22.514387Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:22.569760Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342424476949536:3441]
2025-10-09T21:49:22.570158Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 1495, pipe: [1:7559342424476949536:3441]
2025-10-09T21:49:22.651340Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:22.652022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:22.652036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:22.652054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342424476949536:3441], now have 1 active actors on pipe
2025-10-09T21:49:22.652262Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:22.652335Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:22.652362Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342424476949536:3441]
2025-10-09T21:49:22.652387Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:22.652474Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1496 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8085318f-f6355b9c-19cafdd5-8a73d58d_0
2025-10-09T21:49:22.652665Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1496 sessionId: src_id|8085318f-f6355b9c-19cafdd5-8a73d58d_0 is DEAD
2025-10-09T21:49:22.652816Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8085318f-f6355b9c-19cafdd5-8a73d58d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:22.652829Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8085318f-f6355b9c-19cafdd5-8a73d58d_0] Write session will restart in 0.006609s
2025-10-09T21:49:22.652853Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8085318f-f6355b9c-19cafdd5-8a73d58d_0] Write session: Do CDS request
2025-10-09T21:49:22.652859Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8085318f-f6355b9c-19cafdd5-8a73d58d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:22.660453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8085318f-f6355b9c-19cafdd5-8a73d58d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:22.660569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:22.660588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1497
2025-10-09T21:49:22.660746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1497 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:22.660779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1497 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:22.660787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1497 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:22.660800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:22.660909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:22.661230Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1497 sessionId:  describe result for acl check
2025-10-09T21:49:22.661270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:22.661277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:22.661278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:22.661283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342424476949583:2630] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:22.661752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342424476949583:2630] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:22.662461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:22.662645Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:22.662655Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:22.662667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:22.662721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:22.662762Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342424115123760:5498]
2025-10-09T21:49:22.663343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:22.663602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:22.663645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:22.663793Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342424476949583:2630] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:22.663969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:22.664014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.664020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.664021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.664022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.664023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.664024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.664025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.664026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.664027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.664032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.664041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.664044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.664053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.664057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.664061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.664064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.664068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.664072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.664083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.664096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.664109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.664115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.664119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.664124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.664129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.664134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.664139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.664144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.664148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.664154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.664158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.664164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.664168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.664173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.664178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.664183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.664212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:22.669233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342424476949583:2630] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:22.669247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342424476949583:2630] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:22.669251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342424476949583:2630] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:22.669259Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1497 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:22.670726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:22.671928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:22.671944Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:22.671947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:22.671951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:22.671957Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1496 [2:7559342424115123760:5498]
2025-10-09T21:49:22.671970Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:22.671989Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:22.671996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:22.672000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:22.672002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:22.672021Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:22.672031Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:22.672042Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:22.672101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.672109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.672111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.672114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:22.672201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:22.672211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.672222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.672234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.672253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:22.672275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342424476949623:2630], now have 1 active actors on pipe
2025-10-09T21:49:22.672283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342424476949538:2432], now have 1 active actors on pipe
2025-10-09T21:49:22.672290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342424115123755:5498]
2025-10-09T21:49:22.672296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342424115123755:5498]
2025-10-09T21:49:22.672298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342424115123755:5498]
2025-10-09T21:49:22.672310Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:22.672325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:22.672328Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:22.672329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:22.672329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:22.672315Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1496
2025-10-09T21:49:22.672315Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1496
2025-10-09T21:49:22.672344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:22.672352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:22.672395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:22.672404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:22.672420Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342424115123755:5498]
2025-10-09T21:49:22.672434Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:22.672454Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:22.672464Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f2da9607-14172e35-10af3fc0-6a13ade5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:22.672471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:22.672475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:22.672477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:22.672487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:22.672497Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:22.672528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:22.672672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:22.672683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:22.672701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:22.672815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1497 partition: 0 MaxSeqNo: 25 sessionId: src_id|f2da9607-14172e35-10af3fc0-6a13ade5_0
2025-10-09T21:49:22.673164Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8085318f-f6355b9c-19cafdd5-8a73d58d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f2da9607-14172e35-10af3fc0-6a13ade5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:22.675710Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2da9607-14172e35-10af3fc0-6a13ade5_0] Write session: try to update token
2025-10-09T21:49:22.675729Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2da9607-14172e35-10af3fc0-6a13ade5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:22.686696Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:22.686712Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:22.686714Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:22.686717Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:22.686726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342424115123755:5498]
2025-10-09T21:49:22.686730Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342424115123755:5498]
2025-10-09T21:49:22.686734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342424115123755:5498]
2025-10-09T21:49:22.686779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:22.686781Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:22.686786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.686787Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:22.686788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:22.686791Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:22.686917Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:22.689761Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1497 sessionId: src_id|f2da9607-14172e35-10af3fc0-6a13ade5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:22.692563Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:22.752525Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342424476949653:3441]
2025-10-09T21:49:22.752922Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 1496, pipe: [1:7559342424476949653:3441]
2025-10-09T21:49:22.837130Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:22.837421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:22.837439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:22.837501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342424476949653:3441], now have 1 active actors on pipe
2025-10-09T21:49:22.837762Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:22.837785Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:22.837792Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:22.837798Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:22.837846Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342424476949653:3441]
2025-10-09T21:49:22.837890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1497 reason: error at write: Disconnected sessionId: src_id|f2da9607-14172e35-10af3fc0-6a13ade5_0
2025-10-09T21:49:22.838199Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1497 sessionId: src_id|f2da9607-14172e35-10af3fc0-6a13ade5_0 is DEAD
2025-10-09T21:49:22.838367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2da9607-14172e35-10af3fc0-6a13ade5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:22.838383Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2da9607-14172e35-10af3fc0-6a13ade5_0] Write session will restart in 0.006869s
2025-10-09T21:49:22.838411Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2da9607-14172e35-10af3fc0-6a13ade5_0] Write session: Do CDS request
2025-10-09T21:49:22.838418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2da9607-14172e35-10af3fc0-6a13ade5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:22.846544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2da9607-14172e35-10af3fc0-6a13ade5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:22.846589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:22.846611Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1498
2025-10-09T21:49:22.846799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1498 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:22.846831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1498 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:22.846842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1498 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:22.846845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:22.846988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:22.847315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1498 sessionId:  describe result for acl check
2025-10-09T21:49:22.847357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:22.847359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:22.847361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:22.847368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342424476949704:2616] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:22.847927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342424476949704:2616] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:22.848163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:22.848359Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:22.848368Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:22.848380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:22.848439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:22.848484Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342424115123877:5468]
2025-10-09T21:49:22.849061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:22.849387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:22.849436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:22.849769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:22.849839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.849857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.849858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.849859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.849861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.849862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.849863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.849864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.849865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.849871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.849880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.849884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.849888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.849892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.849895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.849899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.849905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.849908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.849915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.849923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:22.849929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.849934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:22.849939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.849944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:22.849950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.849957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:22.849961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.849971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:22.849975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.849980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:22.849989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.849997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.850006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.850011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.850020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.850025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.850070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:22.850192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342424476949704:2616] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:22.856159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342424476949704:2616] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:22.856174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342424476949704:2616] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:22.856177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342424476949704:2616] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:22.856186Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1498 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:22.856411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:22.857566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:22.857581Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:22.857584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:22.857589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:22.857605Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1497 [2:7559342424115123877:5468]
2025-10-09T21:49:22.857617Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:22.857634Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:22.857642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:22.857645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:22.857649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:22.857664Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:22.857671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:22.857677Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:22.857717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.857722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.857724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.857725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:22.857804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:22.857811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.857823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:22.857839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:22.857860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:22.857879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342424476949743:2616], now have 1 active actors on pipe
2025-10-09T21:49:22.857885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342424476949656:2432], now have 1 active actors on pipe
2025-10-09T21:49:22.857892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342424115123872:5468]
2025-10-09T21:49:22.857893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342424115123872:5468]
2025-10-09T21:49:22.857894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342424115123872:5468]
2025-10-09T21:49:22.857905Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:22.857918Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:22.857924Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:22.857926Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:22.857929Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:22.857930Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:22.857933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:22.857956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342424115123872:5468]
2025-10-09T21:49:22.857958Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:22.857942Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1497
2025-10-09T21:49:22.857967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:22.857970Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1497
2025-10-09T21:49:22.857984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:22.857990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:22.858017Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5b67b406-7a95effa-14fc1a83-e80a97e8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:22.858025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:22.858066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:22.858072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:22.858082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:22.858105Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:22.858125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:22.858286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:22.858294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:22.858307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:22.858400Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1498 partition: 0 MaxSeqNo: 25 sessionId: src_id|5b67b406-7a95effa-14fc1a83-e80a97e8_0
2025-10-09T21:49:22.858811Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2da9607-14172e35-10af3fc0-6a13ade5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5b67b406-7a95effa-14fc1a83-e80a97e8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:22.861253Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b67b406-7a95effa-14fc1a83-e80a97e8_0] Write session: try to update token
2025-10-09T21:49:22.861272Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b67b406-7a95effa-14fc1a83-e80a97e8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:22.871937Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:22.871954Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:22.871956Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:22.871958Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:22.871967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342424115123872:5468]
2025-10-09T21:49:22.871972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342424115123872:5468]
2025-10-09T21:49:22.871975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342424115123872:5468]
2025-10-09T21:49:22.872019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:22.872027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:22.872027Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:22.872030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:22.872034Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:22.872038Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:22.872206Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:22.876961Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1498 sessionId: src_id|5b67b406-7a95effa-14fc1a83-e80a97e8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:22.879443Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:22.937864Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342424476949751:3441]
2025-10-09T21:49:22.938207Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 1497, pipe: [1:7559342424476949751:3441]
2025-10-09T21:49:23.031951Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:23.032232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.032250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.032296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342424476949751:3441], now have 1 active actors on pipe
2025-10-09T21:49:23.032577Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:23.032572Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342424476949751:3441]
2025-10-09T21:49:23.032586Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:23.032590Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:23.032609Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:23.032678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1498 reason: error at write: Disconnected sessionId: src_id|5b67b406-7a95effa-14fc1a83-e80a97e8_0
2025-10-09T21:49:23.032866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1498 sessionId: src_id|5b67b406-7a95effa-14fc1a83-e80a97e8_0 is DEAD
2025-10-09T21:49:23.033069Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b67b406-7a95effa-14fc1a83-e80a97e8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:23.033082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b67b406-7a95effa-14fc1a83-e80a97e8_0] Write session will restart in 0.007132s
2025-10-09T21:49:23.033103Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b67b406-7a95effa-14fc1a83-e80a97e8_0] Write session: Do CDS request
2025-10-09T21:49:23.033107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b67b406-7a95effa-14fc1a83-e80a97e8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:23.041534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b67b406-7a95effa-14fc1a83-e80a97e8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:23.041547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:23.041800Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:23.041818Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1499
2025-10-09T21:49:23.041951Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1499 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:23.041985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1499 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:23.041995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1499 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:23.042000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:23.042434Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1499 sessionId:  describe result for acl check
2025-10-09T21:49:23.042471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:23.042476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:23.042477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:23.042483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342428771917098:2603] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:23.042964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342428771917098:2603] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:23.043067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:23.043224Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:23.043235Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:23.043245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:23.043285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:23.043314Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342428410091248:5464]
2025-10-09T21:49:23.043722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:23.044134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:23.044182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:23.044491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:23.044552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.044558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.044559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.044561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.044562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.044564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.044565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.044566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.044568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.044573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.044583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.044587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.044591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.044595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.044599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.044604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.044608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.044611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.044618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.044626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.044632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.044638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.044642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.044648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.044652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.044657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.044663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.044668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.044672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.044676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.044681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.044685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.044692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.044696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.044700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.044705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.044734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:23.045293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342428771917098:2603] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:23.051313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342428771917098:2603] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:23.051332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342428771917098:2603] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:23.051335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342428771917098:2603] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:23.051346Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1499 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:23.051565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:23.052654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:23.052667Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:23.052669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:23.052672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:23.052678Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1498 [2:7559342428410091248:5464]
2025-10-09T21:49:23.052688Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:23.052702Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:23.052703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:23.052708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:23.052710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:23.052722Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:23.052726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:23.052731Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:23.052773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.052778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.052780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.052781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:23.052846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:23.052853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.052860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.052866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.052876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:23.052892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342428410091243:5464]
2025-10-09T21:49:23.052893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342428410091243:5464]
2025-10-09T21:49:23.052895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342428410091243:5464]
2025-10-09T21:49:23.052901Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:23.052913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:23.052913Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:23.052916Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:23.052918Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:23.052921Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:23.052923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:23.052933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342428771917050:2432], now have 1 active actors on pipe
2025-10-09T21:49:23.052936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342428771917137:2603], now have 1 active actors on pipe
2025-10-09T21:49:23.052957Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342428410091243:5464]
2025-10-09T21:49:23.052967Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:23.052993Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:23.053012Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1498
2025-10-09T21:49:23.053013Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1498
2025-10-09T21:49:23.053104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.053114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.053140Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|46b69730-2ecd6206-637fcdbf-1a8e29d8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:23.053150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:23.053153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:23.053155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:23.053162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:23.053171Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:23.053188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:23.053459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.053467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.053485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:23.053584Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1499 partition: 0 MaxSeqNo: 25 sessionId: src_id|46b69730-2ecd6206-637fcdbf-1a8e29d8_0
2025-10-09T21:49:23.053926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b67b406-7a95effa-14fc1a83-e80a97e8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|46b69730-2ecd6206-637fcdbf-1a8e29d8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:23.056480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46b69730-2ecd6206-637fcdbf-1a8e29d8_0] Write session: try to update token
2025-10-09T21:49:23.056499Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46b69730-2ecd6206-637fcdbf-1a8e29d8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:23.070329Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1499 sessionId: src_id|46b69730-2ecd6206-637fcdbf-1a8e29d8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:23.070800Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:23.070819Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:23.070831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342428410091243:5464]
2025-10-09T21:49:23.070885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:23.070886Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:23.071078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:23.071280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:23.071471Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:23.071657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:23.071863Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:23.071908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:23.072938Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:23.073160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.073175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.073211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:23.073389Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:23.078528Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:23.078545Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:23.078548Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:23.078550Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:23.078564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342428410091243:5464]
2025-10-09T21:49:23.078569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342428410091243:5464]
2025-10-09T21:49:23.078572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342428410091243:5464]
2025-10-09T21:49:23.078618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:23.078620Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:23.078627Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:23.078631Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:23.078632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.078634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:23.078776Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:23.132943Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342428771917144:3441]
2025-10-09T21:49:23.133269Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 1498, pipe: [1:7559342428771917144:3441]
2025-10-09T21:49:23.233279Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:23.240618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.240645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.240790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:23.240844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:23.240903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:23.240959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:23.241004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:23.241015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:23.241078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:23.241132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:23.241188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:23.241251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:23.241296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:23.241302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:23.241358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:23.241408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:23.241467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:23.241530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:23.241570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:23.241575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:23.241647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:23.241702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:23.241757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:23.241815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:23.241860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:23.241865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:23.241923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:23.241973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:23.242145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:23.242199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:23.242298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:23.242304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:23.242359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:23.242410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:23.242461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:23.242514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:23.242552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:23.242557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:23.242609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:23.242661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:23.242714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:23.242767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:23.242818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:23.242824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:23.242875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:23.242926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:23.242977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:23.243027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:23.243139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:23.243146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:23.243200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:23.243253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:23.243311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:23.243365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:23.243468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:23.243475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:23.243529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:23.243579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:23.243633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:23.243689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:23.243730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:23.243736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:23.243775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342428771917144:3441], now have 1 active actors on pipe
2025-10-09T21:49:23.243961Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342428771917144:3441]
2025-10-09T21:49:23.243982Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:23.243995Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:23.243997Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:23.244022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1499 reason: error at write: Disconnected sessionId: src_id|46b69730-2ecd6206-637fcdbf-1a8e29d8_0
2025-10-09T21:49:23.244210Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1499 sessionId: src_id|46b69730-2ecd6206-637fcdbf-1a8e29d8_0 is DEAD
2025-10-09T21:49:23.244347Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46b69730-2ecd6206-637fcdbf-1a8e29d8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:23.244355Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46b69730-2ecd6206-637fcdbf-1a8e29d8_0] Write session will restart in 0.005649s
2025-10-09T21:49:23.244373Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46b69730-2ecd6206-637fcdbf-1a8e29d8_0] Write session: Do CDS request
2025-10-09T21:49:23.244378Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46b69730-2ecd6206-637fcdbf-1a8e29d8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:23.244657Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:23.251467Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46b69730-2ecd6206-637fcdbf-1a8e29d8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:23.251940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:23.252047Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:23.252065Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1500
2025-10-09T21:49:23.252187Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1500 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:23.252219Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1500 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:23.252225Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1500 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:23.252232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:23.252624Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1500 sessionId:  describe result for acl check
2025-10-09T21:49:23.252655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:23.252656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:23.252657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:23.252661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342428771917204:2601] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:23.253081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342428771917204:2601] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:23.253252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:23.253411Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:23.253422Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:23.253432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:23.253475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:23.253505Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342428410091324:5460]
2025-10-09T21:49:23.253871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:23.254188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:23.254261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:23.254518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:23.254558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.254562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.254563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.254564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.254565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.254565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.254566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.254566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.254567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.254571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.254577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.254579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.254581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.254583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.254585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.254587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.254589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.254591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.254596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.254601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.254605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.254608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.254611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.254614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.254616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.254619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.254622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.254625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.254627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.254631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.254634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.254637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.254639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.254643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.254645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.254648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.254664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:23.255288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342428771917204:2601] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:23.260321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342428771917204:2601] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:23.260342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342428771917204:2601] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:23.260345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342428771917204:2601] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:23.260352Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1500 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:23.260795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:23.262304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:23.262321Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:23.262323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:23.262328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:23.262334Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1499 [2:7559342428410091324:5460]
2025-10-09T21:49:23.262348Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:23.262365Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:23.262374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:23.262377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:23.262380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:23.262398Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:23.262409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:23.262420Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:23.262478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.262484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.262486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.262489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:23.262560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:23.262569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.262581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.262593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.262610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:23.262632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342428771917155:2432], now have 1 active actors on pipe
2025-10-09T21:49:23.262640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342428771917243:2601], now have 1 active actors on pipe
2025-10-09T21:49:23.262647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342428410091319:5460]
2025-10-09T21:49:23.262650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342428410091319:5460]
2025-10-09T21:49:23.262652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342428410091319:5460]
2025-10-09T21:49:23.262665Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:23.262670Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1499
2025-10-09T21:49:23.262681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:23.262667Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1499
2025-10-09T21:49:23.262683Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:23.262685Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:23.262687Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:23.262700Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:23.262703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:23.262742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342428410091319:5460]
2025-10-09T21:49:23.262753Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:23.262764Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:23.262769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.262773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.262821Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b663ba1c-cebcbcc-72aeb5b5-92182cad_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:23.262833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:23.262838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:23.262840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:23.262850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:23.262860Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:23.262880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:23.263040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.263048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.263061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:23.263133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1500 partition: 0 MaxSeqNo: 25 sessionId: src_id|b663ba1c-cebcbcc-72aeb5b5-92182cad_0
2025-10-09T21:49:23.263455Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46b69730-2ecd6206-637fcdbf-1a8e29d8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b663ba1c-cebcbcc-72aeb5b5-92182cad_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:23.265871Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b663ba1c-cebcbcc-72aeb5b5-92182cad_0] Write session: try to update token
2025-10-09T21:49:23.265888Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b663ba1c-cebcbcc-72aeb5b5-92182cad_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:23.277299Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:23.277320Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:23.277322Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:23.277324Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:23.277335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342428410091319:5460]
2025-10-09T21:49:23.277340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342428410091319:5460]
2025-10-09T21:49:23.277343Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342428410091319:5460]
2025-10-09T21:49:23.277390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:23.277390Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:23.277396Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:23.277399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.277401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:23.277411Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:23.277540Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:23.279532Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1500 sessionId: src_id|b663ba1c-cebcbcc-72aeb5b5-92182cad_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:23.281985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:23.344394Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342428771917251:3441]
2025-10-09T21:49:23.344734Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 1499, pipe: [1:7559342428771917251:3441]
2025-10-09T21:49:23.427190Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:23.427430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.427443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.427485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342428771917251:3441], now have 1 active actors on pipe
2025-10-09T21:49:23.427803Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342428771917251:3441]
2025-10-09T21:49:23.427833Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:23.427815Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:23.427844Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:23.427847Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:23.428109Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1500 reason: error at write: Disconnected sessionId: src_id|b663ba1c-cebcbcc-72aeb5b5-92182cad_0
2025-10-09T21:49:23.428286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1500 sessionId: src_id|b663ba1c-cebcbcc-72aeb5b5-92182cad_0 is DEAD
2025-10-09T21:49:23.428444Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b663ba1c-cebcbcc-72aeb5b5-92182cad_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:23.428452Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b663ba1c-cebcbcc-72aeb5b5-92182cad_0] Write session will restart in 0.005203s
2025-10-09T21:49:23.428478Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b663ba1c-cebcbcc-72aeb5b5-92182cad_0] Write session: Do CDS request
2025-10-09T21:49:23.428483Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b663ba1c-cebcbcc-72aeb5b5-92182cad_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:23.434505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b663ba1c-cebcbcc-72aeb5b5-92182cad_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:23.434560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:23.434576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1501
2025-10-09T21:49:23.434732Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1501 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:23.434782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1501 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:23.434792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1501 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:23.434798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:23.435212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1501 sessionId:  describe result for acl check
2025-10-09T21:49:23.435250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:23.435251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:23.435253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:23.435257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342428771917301:2597] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:23.435778Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342428771917301:2597] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:23.436083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:23.437112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:23.437273Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:23.437284Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:23.437295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:23.437342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:23.437380Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342428410091401:5458]
2025-10-09T21:49:23.437911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:23.438023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342428771917301:2597] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:23.438208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:23.438253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:23.438487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:23.438531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.438536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.438537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.438538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.438539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.438539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.438540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.438541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.438542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.438545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.438552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.438554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.438556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.438558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.438560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.438563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.438566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.438568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.438573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.438578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.438581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.438585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.438588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.438594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.438598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.438603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.438606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.438610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.438615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.438620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.438625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.438631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.438635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.438638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.438641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.438644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.438660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:23.443539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342428771917301:2597] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:23.443554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342428771917301:2597] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:23.443558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342428771917301:2597] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:23.443567Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1501 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:23.444863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:23.446009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:23.446026Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:23.446045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:23.446049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:23.446057Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1500 [2:7559342428410091401:5458]
2025-10-09T21:49:23.446069Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:23.446085Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:23.446093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:23.446096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:23.446100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:23.446116Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:23.446127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:23.446134Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:23.446191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.446198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.446200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.446203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:23.446271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:23.446279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.446289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.446301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.446321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:23.446345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342428771917342:2597], now have 1 active actors on pipe
2025-10-09T21:49:23.446353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342428771917253:2432], now have 1 active actors on pipe
2025-10-09T21:49:23.446360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342428410091396:5458]
2025-10-09T21:49:23.446362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342428410091396:5458]
2025-10-09T21:49:23.446364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342428410091396:5458]
2025-10-09T21:49:23.446377Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:23.446396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:23.446415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:23.446393Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1500
2025-10-09T21:49:23.446423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:23.446433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:23.446397Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1500
2025-10-09T21:49:23.446444Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:23.446446Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:23.446446Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342428410091396:5458]
2025-10-09T21:49:23.446449Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:23.446454Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:23.446489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.446498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.446524Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ecbfbc2a-454ec890-e5086619-eeb56b9f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:23.446536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:23.446539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:23.446541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:23.446552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:23.446566Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:23.446583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:23.446750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.446761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.446777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:23.446898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1501 partition: 0 MaxSeqNo: 25 sessionId: src_id|ecbfbc2a-454ec890-e5086619-eeb56b9f_0
2025-10-09T21:49:23.447195Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b663ba1c-cebcbcc-72aeb5b5-92182cad_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ecbfbc2a-454ec890-e5086619-eeb56b9f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:23.449544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecbfbc2a-454ec890-e5086619-eeb56b9f_0] Write session: try to update token
2025-10-09T21:49:23.449560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecbfbc2a-454ec890-e5086619-eeb56b9f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:23.461160Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:23.461183Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:23.461185Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:23.461188Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:23.461201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342428410091396:5458]
2025-10-09T21:49:23.461210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342428410091396:5458]
2025-10-09T21:49:23.461216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342428410091396:5458]
2025-10-09T21:49:23.461263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:23.461269Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:23.461275Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:23.461276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.461278Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:23.461279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:23.461418Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:23.464410Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1501 sessionId: src_id|ecbfbc2a-454ec890-e5086619-eeb56b9f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:23.466729Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:23.528199Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342428771917350:3441]
2025-10-09T21:49:23.528486Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 1500, pipe: [1:7559342428771917350:3441]
2025-10-09T21:49:23.607006Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:23.607781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.607804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.607843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342428771917350:3441], now have 1 active actors on pipe
2025-10-09T21:49:23.608075Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:23.608102Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342428771917350:3441]
2025-10-09T21:49:23.608121Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:23.608160Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:49:23.608285Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:23.608354Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1501 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ecbfbc2a-454ec890-e5086619-eeb56b9f_0
2025-10-09T21:49:23.608378Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 15 consumer shared/user session shared/user_1_15_12240953663583783860_v1 is DEAD
2025-10-09T21:49:23.608460Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1501 sessionId: src_id|ecbfbc2a-454ec890-e5086619-eeb56b9f_0 is DEAD
2025-10-09T21:49:23.608567Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:49:23.608568Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342342872547263:3432] disconnected.
2025-10-09T21:49:23.608582Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342342872547263:3432] disconnected; active server actors: 1
2025-10-09T21:49:23.608585Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342342872547263:3432] client user disconnected session shared/user_1_15_12240953663583783860_v1
2025-10-09T21:49:23.608643Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecbfbc2a-454ec890-e5086619-eeb56b9f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:23.608666Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecbfbc2a-454ec890-e5086619-eeb56b9f_0] Write session will restart in 0.006467s
2025-10-09T21:49:23.608700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecbfbc2a-454ec890-e5086619-eeb56b9f_0] Write session: Do CDS request
2025-10-09T21:49:23.608706Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecbfbc2a-454ec890-e5086619-eeb56b9f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:23.608732Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:49:23.608738Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:49:23.608746Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.006122s
2025-10-09T21:49:23.615385Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:49:23.615575Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:49:23.615592Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 16
2025-10-09T21:49:23.615757Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 16 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:49:23.615809Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:49:23.615891Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 auth for : user
2025-10-09T21:49:23.616075Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 Handle describe topics response
2025-10-09T21:49:23.616114Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 auth is DEAD
2025-10-09T21:49:23.616138Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:49:23.616370Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecbfbc2a-454ec890-e5086619-eeb56b9f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:23.616402Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:49:23.616405Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:23.616410Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1502
2025-10-09T21:49:23.616399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:23.616469Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342428771917403:2586] connected; active server actors: 1
2025-10-09T21:49:23.616479Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_16_18381273765076421244_v1
2025-10-09T21:49:23.616481Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559342428771917403:2586] session shared/user_1_16_18381273765076421244_v1
2025-10-09T21:49:23.616490Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:49:23.616500Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:49:23.616507Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_16_18381273765076421244_v1" (Sender=[1:7559342428771917399:2586], Pipe=[1:7559342428771917403:2586], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:49:23.616516Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:49:23.616528Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:49:23.616541Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_16_18381273765076421244_v1" (Sender=[1:7559342428771917399:2586], Pipe=[1:7559342428771917403:2586], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:49:23.616555Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_16_18381273765076421244_v1" sender [1:7559342428771917399:2586] lock partition 0 for ReadingSession "shared/user_1_16_18381273765076421244_v1" (Sender=[1:7559342428771917399:2586], Pipe=[1:7559342428771917403:2586], Partitions=[], ActiveFamilyCount=1) generation 1 step 16
2025-10-09T21:49:23.616566Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1502 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:23.616567Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:49:23.616572Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000036s
2025-10-09T21:49:23.616591Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1502 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:23.616603Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1502 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:23.616607Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:23.616765Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 16 Session: "shared/user_1_16_18381273765076421244_v1" ClientId: "user" PipeClient { RawX1: 7559342428771917403 RawX2: 4503603922340378 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:49:23.616783Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:49:23.616913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1502 sessionId:  describe result for acl check
2025-10-09T21:49:23.616938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:23.616944Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:23.616946Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:23.616951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342428771917411:2589] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:23.617550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342428771917411:2589] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:23.617800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:23.617934Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:23.617942Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:23.617951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:23.617994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:23.618021Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342428410091480:5441]
2025-10-09T21:49:23.618395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:23.618649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:23.618674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:23.618865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:23.618897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.618901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.618902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.618902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.618903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.618904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.618904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.618905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.618906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.618908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.618914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.618918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.618920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.618922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.618924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.618926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.618928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.618930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.618934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.618938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.618943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.618946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.618948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.618951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.618954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.618957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.618959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.618962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.618964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.618967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.618970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.618973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.618976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.618978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.618981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.618984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.618999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:23.619563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342428771917411:2589] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:23.624725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342428771917411:2589] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:23.624741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342428771917411:2589] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:23.624745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342428771917411:2589] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:23.624753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1502 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:23.625019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:23.626644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:23.626656Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:23.626657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:23.626660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:23.626664Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1501 [2:7559342428410091480:5441]
2025-10-09T21:49:23.626677Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:23.626688Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:23.626690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:23.626692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:23.626694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:23.626706Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:23.626710Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:23.626715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:23.626758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.626766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.626767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.626769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:23.626831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:23.626838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.626845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.626851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.626860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:23.626874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342428771917409:2585], now have 1 active actors on pipe
2025-10-09T21:49:23.626877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342428771917448:2589], now have 1 active actors on pipe
2025-10-09T21:49:23.626882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342428410091474:5441]
2025-10-09T21:49:23.626884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342428410091474:5441]
2025-10-09T21:49:23.626886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342428410091474:5441]
2025-10-09T21:49:23.626892Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:23.626902Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:23.626903Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:23.626907Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:23.626909Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:23.626916Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:23.626920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:23.626924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342428771917352:2432], now have 1 active actors on pipe
2025-10-09T21:49:23.626945Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342428410091474:5441]
2025-10-09T21:49:23.626952Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:23.626953Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:23.626946Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1501, pipe: [1:7559342428771917409:2585]
2025-10-09T21:49:23.626963Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1501
2025-10-09T21:49:23.626977Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1501
2025-10-09T21:49:23.626989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.626999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.627009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_16_18381273765076421244_v1 on pipe: [1:7559342428771917409:2585]
2025-10-09T21:49:23.627028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.627033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.627043Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_16_18381273765076421244_v1:1 with generation 1501
2025-10-09T21:49:23.627045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:23.627050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:23.627052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:23.627068Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_16_18381273765076421244_v1
2025-10-09T21:49:23.627071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:23.627113Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|25a4df0e-e27276f8-c883a61c-6ff3c11e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:23.627126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:49:23.627131Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:49:23.631277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:49:23.631305Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:49:23.631315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:23.631321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:23.631323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:23.631344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:23.631363Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:23.631374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:49:23.631388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:49:23.631409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:23.631532Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046563626 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:49:23.631551Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:49:23.631584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.631598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.631595Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 sending to client partition status
2025-10-09T21:49:23.631619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:23.631743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1502 partition: 0 MaxSeqNo: 25 sessionId: src_id|25a4df0e-e27276f8-c883a61c-6ff3c11e_0
2025-10-09T21:49:23.632082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecbfbc2a-454ec890-e5086619-eeb56b9f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|25a4df0e-e27276f8-c883a61c-6ff3c11e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:23.634404Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25a4df0e-e27276f8-c883a61c-6ff3c11e_0] Write session: try to update token
2025-10-09T21:49:23.634422Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25a4df0e-e27276f8-c883a61c-6ff3c11e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:23.641073Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:23.641089Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:23.641092Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:23.641094Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:23.641104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342428410091474:5441]
2025-10-09T21:49:23.641109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342428410091474:5441]
2025-10-09T21:49:23.641113Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342428410091474:5441]
2025-10-09T21:49:23.641156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:23.641158Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:23.641162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.641163Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:23.641165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:23.641167Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:23.641287Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:23.652777Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1502 sessionId: src_id|25a4df0e-e27276f8-c883a61c-6ff3c11e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:23.655228Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:23.794895Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:23.795157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.795173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.795381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_16_18381273765076421244_v1
2025-10-09T21:49:23.795405Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:23.795376Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:23.795423Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_16_18381273765076421244_v1
2025-10-09T21:49:23.795430Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342428771917409:2585]
2025-10-09T21:49:23.795443Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:23.795445Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:23.795499Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1502 reason: error at write: Disconnected sessionId: src_id|25a4df0e-e27276f8-c883a61c-6ff3c11e_0
2025-10-09T21:49:23.795695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1502 sessionId: src_id|25a4df0e-e27276f8-c883a61c-6ff3c11e_0 is DEAD
2025-10-09T21:49:23.795835Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25a4df0e-e27276f8-c883a61c-6ff3c11e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:23.795847Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25a4df0e-e27276f8-c883a61c-6ff3c11e_0] Write session will restart in 0.009916s
2025-10-09T21:49:23.795869Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25a4df0e-e27276f8-c883a61c-6ff3c11e_0] Write session: Do CDS request
2025-10-09T21:49:23.795874Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25a4df0e-e27276f8-c883a61c-6ff3c11e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:23.803928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:23.805211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:23.805455Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:23.805466Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:23.805479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:23.805544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:23.805584Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342428410091603:5414]
2025-10-09T21:49:23.806218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:23.806490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25a4df0e-e27276f8-c883a61c-6ff3c11e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:23.806537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:23.806590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:23.806919Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:23.806941Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1503
2025-10-09T21:49:23.806922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:23.806973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.806978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.806979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.806980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.806980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.806981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.806982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.806982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.806983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.806986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.806993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.806995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.806997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.806999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.807001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.807003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.807005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.807008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.807042Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1503 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:23.807014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.807019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.807023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.807027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.807029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.807032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.807034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.807037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.807039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.807042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.807045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.807074Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1503 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:23.807084Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1503 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:23.807088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:23.807050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.807055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.807060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.807065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.807069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.807074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.807078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.807107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:23.807445Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1503 sessionId:  describe result for acl check
2025-10-09T21:49:23.807487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:23.807493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:23.807495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:23.807502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342428771917540:2570] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:23.807982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342428771917540:2570] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:23.810049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342428771917540:2570] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:23.812900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342428771917540:2570] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:23.812913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342428771917540:2570] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:23.812916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342428771917540:2570] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:23.812922Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1503 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:23.813252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:23.814383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:23.814398Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:23.814400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:23.814404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:23.814410Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1502 [2:7559342428410091603:5414]
2025-10-09T21:49:23.814423Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:23.814439Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:23.814442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:23.814445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:23.814447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:23.814465Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:23.814473Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:23.814483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:23.814540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.814547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.814549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.814552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:23.814615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:23.814623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.814634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.814644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.814667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:23.814691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342428771917483:2432], now have 1 active actors on pipe
2025-10-09T21:49:23.814700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342428771917570:2570], now have 1 active actors on pipe
2025-10-09T21:49:23.814707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342428410091598:5414]
2025-10-09T21:49:23.814709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342428410091598:5414]
2025-10-09T21:49:23.814711Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342428410091598:5414]
2025-10-09T21:49:23.814724Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:23.814744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:23.814746Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:23.814749Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:23.814751Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:23.814762Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:23.814765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:23.814756Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1502
2025-10-09T21:49:23.814791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342428410091598:5414]
2025-10-09T21:49:23.814799Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:23.814802Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:23.814823Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1502
2025-10-09T21:49:23.814834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.814844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.814875Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dfa23b2e-dcdee2be-a540e9ba-1dcbf7fb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:23.814888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:23.814892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:23.814899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:23.814908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:23.814917Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:23.814934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:23.815081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.815091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.815104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:23.815223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1503 partition: 0 MaxSeqNo: 25 sessionId: src_id|dfa23b2e-dcdee2be-a540e9ba-1dcbf7fb_0
2025-10-09T21:49:23.815503Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25a4df0e-e27276f8-c883a61c-6ff3c11e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dfa23b2e-dcdee2be-a540e9ba-1dcbf7fb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:23.817972Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfa23b2e-dcdee2be-a540e9ba-1dcbf7fb_0] Write session: try to update token
2025-10-09T21:49:23.817988Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfa23b2e-dcdee2be-a540e9ba-1dcbf7fb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:23.828563Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:23.828579Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:23.828581Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:23.828584Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:23.828592Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342428410091598:5414]
2025-10-09T21:49:23.828597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342428410091598:5414]
2025-10-09T21:49:23.828601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342428410091598:5414]
2025-10-09T21:49:23.828645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:23.828651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.828654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:23.828659Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:23.828666Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:23.828670Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:23.828782Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:23.833141Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1503 sessionId: src_id|dfa23b2e-dcdee2be-a540e9ba-1dcbf7fb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:23.835515Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:23.895811Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342428771917581:2585]
2025-10-09T21:49:23.896124Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 1502, pipe: [1:7559342428771917581:2585]
2025-10-09T21:49:23.984269Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:23.984851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:23.984868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:23.984912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342428771917581:2585], now have 1 active actors on pipe
2025-10-09T21:49:23.985109Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:23.985169Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:23.985187Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:23.985219Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342428771917581:2585]
2025-10-09T21:49:23.985237Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:23.985244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1503 reason: error at write: Disconnected sessionId: src_id|dfa23b2e-dcdee2be-a540e9ba-1dcbf7fb_0
2025-10-09T21:49:23.985387Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1503 sessionId: src_id|dfa23b2e-dcdee2be-a540e9ba-1dcbf7fb_0 is DEAD
2025-10-09T21:49:23.985504Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfa23b2e-dcdee2be-a540e9ba-1dcbf7fb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:23.985515Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfa23b2e-dcdee2be-a540e9ba-1dcbf7fb_0] Write session will restart in 0.008043s
2025-10-09T21:49:23.985536Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfa23b2e-dcdee2be-a540e9ba-1dcbf7fb_0] Write session: Do CDS request
2025-10-09T21:49:23.985540Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfa23b2e-dcdee2be-a540e9ba-1dcbf7fb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:23.994085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:23.994479Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfa23b2e-dcdee2be-a540e9ba-1dcbf7fb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:23.994553Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:23.994570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1504
2025-10-09T21:49:23.994703Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1504 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:23.994746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1504 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:23.994758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1504 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:23.994762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:23.995156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1504 sessionId:  describe result for acl check
2025-10-09T21:49:23.995189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:23.995196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:23.995198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:23.995204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342428771917639:2561] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:23.995646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:23.995740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342428771917639:2561] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:23.995823Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:23.995836Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:23.995848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:23.995902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:23.995951Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342428410091683:5412]
2025-10-09T21:49:23.996524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:23.996887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:23.996937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:23.997238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:23.997299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.997305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.997306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.997307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.997308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.997309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.997310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.997311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.997313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.997318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.997330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.997334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.997338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.997342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.997346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.997350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.997353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.997357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.997363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.997371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:23.997381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.997391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:23.997398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.997384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342428771917639:2561] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:23.997407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:23.997416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.997420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:23.997425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.997429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:23.997434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.997439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:23.997444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.997450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:23.997453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.997458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:23.997462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.997465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:23.997494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:24.003477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342428771917639:2561] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:24.003497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342428771917639:2561] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:24.003501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342428771917639:2561] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:24.003510Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1504 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:24.003889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:24.005002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:24.005018Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:24.005020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:24.005024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:24.005031Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1503 [2:7559342428410091683:5412]
2025-10-09T21:49:24.005047Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:24.005064Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:24.005074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:24.005077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:24.005079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:24.005096Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:24.005102Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:24.005109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:24.005164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.005171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.005173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.005175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:24.005256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:24.005262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.005274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.005287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.005310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:24.005335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342433066884972:2561], now have 1 active actors on pipe
2025-10-09T21:49:24.005344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342428771917586:2432], now have 1 active actors on pipe
2025-10-09T21:49:24.005351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342428410091678:5412]
2025-10-09T21:49:24.005358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342428410091678:5412]
2025-10-09T21:49:24.005360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342428410091678:5412]
2025-10-09T21:49:24.005374Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:24.005389Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:24.005381Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1503
2025-10-09T21:49:24.005400Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:24.005404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:24.005388Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1503
2025-10-09T21:49:24.005410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:24.005410Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:24.005412Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:24.005441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342428410091678:5412]
2025-10-09T21:49:24.005452Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:24.005454Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:24.005472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.005477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.005526Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a1bf9e2-13dbbe7e-72bbf27d-840fd221_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:24.005538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:24.005542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:24.005553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:24.005563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:24.005573Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:24.005605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:24.005790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.005803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.005819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:24.005946Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1504 partition: 0 MaxSeqNo: 25 sessionId: src_id|a1bf9e2-13dbbe7e-72bbf27d-840fd221_0
2025-10-09T21:49:24.006296Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfa23b2e-dcdee2be-a540e9ba-1dcbf7fb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a1bf9e2-13dbbe7e-72bbf27d-840fd221_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:24.008688Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a1bf9e2-13dbbe7e-72bbf27d-840fd221_0] Write session: try to update token
2025-10-09T21:49:24.008705Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a1bf9e2-13dbbe7e-72bbf27d-840fd221_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:24.019886Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:24.019905Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:24.019907Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:24.019908Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:24.019915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342428410091678:5412]
2025-10-09T21:49:24.019919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342428410091678:5412]
2025-10-09T21:49:24.019921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342428410091678:5412]
2025-10-09T21:49:24.019961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:24.019966Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:24.019970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.019972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:24.019973Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:24.019976Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:24.020111Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:24.023351Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1504 sessionId: src_id|a1bf9e2-13dbbe7e-72bbf27d-840fd221_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:24.025723Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:24.085263Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342433066884979:2585]
2025-10-09T21:49:24.085492Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 1503, pipe: [1:7559342433066884979:2585]
2025-10-09T21:49:24.167393Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:24.167924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.167939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.167977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342433066884979:2585], now have 1 active actors on pipe
2025-10-09T21:49:24.168134Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:24.168190Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:24.168192Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342433066884979:2585]
2025-10-09T21:49:24.168206Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:24.168208Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:24.168283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1504 reason: error at write: Disconnected sessionId: src_id|a1bf9e2-13dbbe7e-72bbf27d-840fd221_0
2025-10-09T21:49:24.168455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1504 sessionId: src_id|a1bf9e2-13dbbe7e-72bbf27d-840fd221_0 is DEAD
2025-10-09T21:49:24.168626Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1bf9e2-13dbbe7e-72bbf27d-840fd221_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:24.168636Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1bf9e2-13dbbe7e-72bbf27d-840fd221_0] Write session will restart in 0.009470s
2025-10-09T21:49:24.168664Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1bf9e2-13dbbe7e-72bbf27d-840fd221_0] Write session: Do CDS request
2025-10-09T21:49:24.168671Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1bf9e2-13dbbe7e-72bbf27d-840fd221_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:24.177060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:24.178154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:24.178342Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:24.178352Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:24.178363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:24.178419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:24.178463Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342432705059061:5399]
2025-10-09T21:49:24.179094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:24.179540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:24.179564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a1bf9e2-13dbbe7e-72bbf27d-840fd221_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:24.179597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:24.179676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:24.179694Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1505
2025-10-09T21:49:24.179815Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1505 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:24.179841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1505 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:24.179849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1505 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:24.179852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:24.179937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:24.179988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.179994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.179995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.179996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.179998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.180001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.180002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.180003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.180004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.180010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.180020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.180024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.180028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.180032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.180037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.180040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.180044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.180048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.180058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.180067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.180074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.180084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.180089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.180094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.180098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.180103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.180114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.180120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.180129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.180139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.180148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.180154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.180158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.180164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.180175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.180182Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1505 sessionId:  describe result for acl check
2025-10-09T21:49:24.180179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.180207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:24.180217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:24.180225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:24.180230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:24.180234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342433066885042:2560] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:24.180801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342433066885042:2560] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:24.182871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342433066885042:2560] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:24.186265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342433066885042:2560] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:24.186295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342433066885042:2560] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:24.186300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342433066885042:2560] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:24.186311Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1505 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:24.186730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:24.187948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:24.187975Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:24.187978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:24.187983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:24.187991Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1504 [2:7559342432705059061:5399]
2025-10-09T21:49:24.188006Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:24.188027Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:24.188035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:24.188039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:24.188041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:24.188061Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:24.188072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:24.188080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:24.188150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.188157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.188160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.188163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:24.188249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:24.188258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.188271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.188284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.188303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:24.188325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342433066885078:2560], now have 1 active actors on pipe
2025-10-09T21:49:24.188329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342433066884985:2432], now have 1 active actors on pipe
2025-10-09T21:49:24.188337Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342432705059056:5399]
2025-10-09T21:49:24.188341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342432705059056:5399]
2025-10-09T21:49:24.188343Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342432705059056:5399]
2025-10-09T21:49:24.188356Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:24.188370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:24.188372Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:24.188376Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:24.188378Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:24.188392Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:24.188400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:24.188393Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1504
2025-10-09T21:49:24.188417Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1504
2025-10-09T21:49:24.188454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.188464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.188477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342432705059056:5399]
2025-10-09T21:49:24.188490Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:24.188509Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:24.188533Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|853b2c66-7055331a-d5c033a7-5f8250ed_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:24.188545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:24.188554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:24.188557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:24.188572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:24.188587Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:24.188620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:24.188864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.188880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.188905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:24.189023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1505 partition: 0 MaxSeqNo: 25 sessionId: src_id|853b2c66-7055331a-d5c033a7-5f8250ed_0
2025-10-09T21:49:24.189380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1bf9e2-13dbbe7e-72bbf27d-840fd221_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|853b2c66-7055331a-d5c033a7-5f8250ed_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:24.191895Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|853b2c66-7055331a-d5c033a7-5f8250ed_0] Write session: try to update token
2025-10-09T21:49:24.191919Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|853b2c66-7055331a-d5c033a7-5f8250ed_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:24.203150Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:24.203172Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:24.203174Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:24.203176Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:24.203187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342432705059056:5399]
2025-10-09T21:49:24.203192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342432705059056:5399]
2025-10-09T21:49:24.203198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342432705059056:5399]
2025-10-09T21:49:24.203239Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:24.203241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:24.203244Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:24.203247Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:24.203249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.203252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:24.203389Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:24.205484Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1505 sessionId: src_id|853b2c66-7055331a-d5c033a7-5f8250ed_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:24.207805Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:24.268569Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342433066885090:2585]
2025-10-09T21:49:24.268888Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 1504, pipe: [1:7559342433066885090:2585]
2025-10-09T21:49:24.355715Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:24.355973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.355987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.356023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342433066885090:2585], now have 1 active actors on pipe
2025-10-09T21:49:24.356214Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:24.356243Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:24.356248Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342433066885090:2585]
2025-10-09T21:49:24.356351Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:24.356359Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:24.356436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1505 reason: error at write: Disconnected sessionId: src_id|853b2c66-7055331a-d5c033a7-5f8250ed_0
2025-10-09T21:49:24.356622Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1505 sessionId: src_id|853b2c66-7055331a-d5c033a7-5f8250ed_0 is DEAD
2025-10-09T21:49:24.356785Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|853b2c66-7055331a-d5c033a7-5f8250ed_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:24.356795Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|853b2c66-7055331a-d5c033a7-5f8250ed_0] Write session will restart in 0.007010s
2025-10-09T21:49:24.356823Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|853b2c66-7055331a-d5c033a7-5f8250ed_0] Write session: Do CDS request
2025-10-09T21:49:24.356829Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|853b2c66-7055331a-d5c033a7-5f8250ed_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:24.364464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|853b2c66-7055331a-d5c033a7-5f8250ed_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:24.364473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:24.364550Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:24.364565Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1506
2025-10-09T21:49:24.364762Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1506 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:24.364792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1506 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:24.364796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1506 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:24.364799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:24.365225Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1506 sessionId:  describe result for acl check
2025-10-09T21:49:24.365265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:24.365271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:24.365273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:24.365279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342433066885148:2550] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:24.365744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:24.365900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342433066885148:2550] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:24.365912Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:24.365922Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:24.365934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:24.365985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:24.366021Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342432705059143:5408]
2025-10-09T21:49:24.366582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:24.366888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:24.366934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:24.367268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:24.367319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.367325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.367326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.367327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.367328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.367329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.367330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.367332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.367333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.367338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.367349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.367353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.367357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.367361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.367365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.367369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.367373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.367377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.367383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.367391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.367398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.367404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.367409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.367415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.367424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.367429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.367434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.367439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.367444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.367450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.367457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.367462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.367468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.367472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.367476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.367484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.367513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:24.367546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342433066885148:2550] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:24.373223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342433066885148:2550] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:24.373239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342433066885148:2550] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:24.373243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342433066885148:2550] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:24.373251Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1506 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:24.373645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:24.374840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:24.374857Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:24.374860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:24.374864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:24.374870Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1505 [2:7559342432705059143:5408]
2025-10-09T21:49:24.374882Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:24.374898Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:24.374901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:24.374904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:24.374907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:24.374923Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:24.374936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:24.374944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:24.374999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.375006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.375008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.375011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:24.375078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:24.375087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.375098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.375111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.375129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:24.375153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342433066885187:2550], now have 1 active actors on pipe
2025-10-09T21:49:24.375162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342433066885093:2432], now have 1 active actors on pipe
2025-10-09T21:49:24.375169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342432705059138:5408]
2025-10-09T21:49:24.375171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342432705059138:5408]
2025-10-09T21:49:24.375173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342432705059138:5408]
2025-10-09T21:49:24.375185Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:24.375197Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:24.375200Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:24.375208Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:24.375211Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1505
2025-10-09T21:49:24.375210Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:24.375218Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1505
2025-10-09T21:49:24.375214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:24.375217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:24.375245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342432705059138:5408]
2025-10-09T21:49:24.375248Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:24.375250Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:24.375267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.375277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.375322Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6f49b930-4ecaff8c-bb29dc27-b8269ff9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:24.375335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:24.375339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:24.375341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:24.375350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:24.375359Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:24.375375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:24.375517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.375527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.375542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:24.375654Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1506 partition: 0 MaxSeqNo: 25 sessionId: src_id|6f49b930-4ecaff8c-bb29dc27-b8269ff9_0
2025-10-09T21:49:24.375962Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|853b2c66-7055331a-d5c033a7-5f8250ed_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6f49b930-4ecaff8c-bb29dc27-b8269ff9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:24.378387Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6f49b930-4ecaff8c-bb29dc27-b8269ff9_0] Write session: try to update token
2025-10-09T21:49:24.378408Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6f49b930-4ecaff8c-bb29dc27-b8269ff9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:24.389662Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:24.389680Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:24.389683Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:24.389685Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:24.389695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342432705059138:5408]
2025-10-09T21:49:24.389700Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342432705059138:5408]
2025-10-09T21:49:24.389704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342432705059138:5408]
2025-10-09T21:49:24.389749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:24.389751Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:24.389756Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:24.389759Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:24.389759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.389762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:24.389894Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:24.391607Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1506 sessionId: src_id|6f49b930-4ecaff8c-bb29dc27-b8269ff9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:24.393964Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:24.456443Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342433066885194:2585]
2025-10-09T21:49:24.456665Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 1505, pipe: [1:7559342433066885194:2585]
2025-10-09T21:49:24.538522Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:24.539007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.539024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.539070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342433066885194:2585], now have 1 active actors on pipe
2025-10-09T21:49:24.539272Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:24.539349Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:24.539368Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:24.539368Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342433066885194:2585]
2025-10-09T21:49:24.539387Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:24.539443Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1506 reason: error at write: Disconnected sessionId: src_id|6f49b930-4ecaff8c-bb29dc27-b8269ff9_0
2025-10-09T21:49:24.539641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1506 sessionId: src_id|6f49b930-4ecaff8c-bb29dc27-b8269ff9_0 is DEAD
2025-10-09T21:49:24.539787Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6f49b930-4ecaff8c-bb29dc27-b8269ff9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:24.539797Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6f49b930-4ecaff8c-bb29dc27-b8269ff9_0] Write session will restart in 0.009794s
2025-10-09T21:49:24.539825Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6f49b930-4ecaff8c-bb29dc27-b8269ff9_0] Write session: Do CDS request
2025-10-09T21:49:24.539831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6f49b930-4ecaff8c-bb29dc27-b8269ff9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:24.548018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:24.549298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:24.549481Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:24.549494Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:24.549507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:24.549561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:24.549617Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342432705059226:5406]
2025-10-09T21:49:24.550199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:24.550529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:24.550555Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6f49b930-4ecaff8c-bb29dc27-b8269ff9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:24.550573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:24.550601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:24.550619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1507
2025-10-09T21:49:24.550727Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1507 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:24.550751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1507 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:24.550754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1507 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:24.550758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:24.550882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:24.550961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.550968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.550969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.550970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.550972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.550973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.550974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.550975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.550976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.550982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.550992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.550997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.551001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.551009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.551014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.551018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.551022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.551025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.551033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.551043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.551054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.551064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.551073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.551078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.551086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.551092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.551100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1507 sessionId:  describe result for acl check
2025-10-09T21:49:24.551102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.551107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.551111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.551116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.551127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:24.551121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.551126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.551132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.551131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:24.551138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.551132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:24.551142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.551148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.551136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342433066885255:2548] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:24.551178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:24.551736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342433066885255:2548] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:24.553616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342433066885255:2548] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:24.556892Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342433066885255:2548] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:24.556907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342433066885255:2548] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:24.556911Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342433066885255:2548] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:24.556919Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1507 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:24.557305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:24.558740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:24.558760Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:24.558762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:24.558766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:24.558772Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1506 [2:7559342432705059226:5406]
2025-10-09T21:49:24.558784Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:24.558801Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:24.558804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:24.558807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:24.558810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:24.558828Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:24.558839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:24.558846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:24.558903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.558911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.558913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.558915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:24.558975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:24.558982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.558990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.558996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.559008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:24.559033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342433066885197:2432], now have 1 active actors on pipe
2025-10-09T21:49:24.559042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342433066885292:2548], now have 1 active actors on pipe
2025-10-09T21:49:24.559049Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342432705059221:5406]
2025-10-09T21:49:24.559052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342432705059221:5406]
2025-10-09T21:49:24.559054Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342432705059221:5406]
2025-10-09T21:49:24.559066Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:24.559087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:24.559079Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1506
2025-10-09T21:49:24.559097Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1506
2025-10-09T21:49:24.559101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:24.559106Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:24.559109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:24.559111Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:24.559112Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:24.559138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342432705059221:5406]
2025-10-09T21:49:24.559152Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:24.559170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.559174Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:24.559177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.559213Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dcce4950-29c51d91-da7d8e8f-de3fa069_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:24.559224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:24.559228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:24.559230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:24.559237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:24.559245Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:24.559260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:24.559476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.559485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.559498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:24.559616Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1507 partition: 0 MaxSeqNo: 25 sessionId: src_id|dcce4950-29c51d91-da7d8e8f-de3fa069_0
2025-10-09T21:49:24.559921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6f49b930-4ecaff8c-bb29dc27-b8269ff9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dcce4950-29c51d91-da7d8e8f-de3fa069_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:24.562286Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dcce4950-29c51d91-da7d8e8f-de3fa069_0] Write session: try to update token
2025-10-09T21:49:24.562301Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dcce4950-29c51d91-da7d8e8f-de3fa069_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:24.573338Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:24.573358Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:24.573360Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:24.573363Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:24.573372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342432705059221:5406]
2025-10-09T21:49:24.573377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342432705059221:5406]
2025-10-09T21:49:24.573381Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342432705059221:5406]
2025-10-09T21:49:24.573424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:24.573429Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:24.573432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.573435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:24.573435Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:24.573439Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:24.573619Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:24.574404Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1507 sessionId: src_id|dcce4950-29c51d91-da7d8e8f-de3fa069_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:24.576600Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:24.639201Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342433066885299:2585]
2025-10-09T21:49:24.639484Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 1506, pipe: [1:7559342433066885299:2585]
2025-10-09T21:49:24.723988Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:24.724204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.724215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.724252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342433066885299:2585], now have 1 active actors on pipe
2025-10-09T21:49:24.724436Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:24.724468Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342433066885299:2585]
2025-10-09T21:49:24.724474Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:24.724477Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:24.724547Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1507 reason: error at write: Disconnected sessionId: src_id|dcce4950-29c51d91-da7d8e8f-de3fa069_0
2025-10-09T21:49:24.724735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1507 sessionId: src_id|dcce4950-29c51d91-da7d8e8f-de3fa069_0 is DEAD
2025-10-09T21:49:24.724887Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dcce4950-29c51d91-da7d8e8f-de3fa069_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:24.724900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dcce4950-29c51d91-da7d8e8f-de3fa069_0] Write session will restart in 0.005517s
2025-10-09T21:49:24.724924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dcce4950-29c51d91-da7d8e8f-de3fa069_0] Write session: Do CDS request
2025-10-09T21:49:24.724948Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dcce4950-29c51d91-da7d8e8f-de3fa069_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:24.725010Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:24.731513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dcce4950-29c51d91-da7d8e8f-de3fa069_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:24.731611Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:24.731625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1508
2025-10-09T21:49:24.731762Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1508 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:24.731797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1508 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:24.731807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1508 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:24.731811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:24.732237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1508 sessionId:  describe result for acl check
2025-10-09T21:49:24.732271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:24.732278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:24.732282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:24.732287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342433066885358:2535] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:24.732828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342433066885358:2535] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:24.732981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:24.734536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:24.734699Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:24.734707Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:24.734715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:24.734751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:24.734780Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342432705059315:5404]
2025-10-09T21:49:24.734952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342433066885358:2535] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:24.735149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:24.735456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:24.735525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:24.735779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:24.735830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.735834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.735835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.735838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.735838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.735839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.735840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.735840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.735841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.735845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.735852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.735854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.735856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.735859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.735861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.735863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.735865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.735867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.735871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.735877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.735883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.735889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.735893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.735898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.735903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.735908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.735910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.735913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.735916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.735919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.735922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.735925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.735928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.735930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.735933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.735936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.735954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:24.740769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342433066885358:2535] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:24.740781Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342433066885358:2535] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:24.740785Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342433066885358:2535] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:24.740793Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1508 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:24.742251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:24.743366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:24.743382Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:24.743385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:24.743389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:24.743396Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1507 [2:7559342432705059315:5404]
2025-10-09T21:49:24.743408Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:24.743424Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:24.743427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:24.743431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:24.743433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:24.743449Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:24.743460Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:24.743467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:24.743520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.743526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.743527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.743529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:24.743577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:24.743582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.743589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.743595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.743605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:24.743623Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342432705059310:5404]
2025-10-09T21:49:24.743629Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342432705059310:5404]
2025-10-09T21:49:24.743630Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342432705059310:5404]
2025-10-09T21:49:24.743637Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:24.743647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:24.743651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342433066885399:2535], now have 1 active actors on pipe
2025-10-09T21:49:24.743655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342433066885303:2432], now have 1 active actors on pipe
2025-10-09T21:49:24.743657Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:24.743662Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:24.743664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:24.743679Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:24.743682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:24.743704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342432705059310:5404]
2025-10-09T21:49:24.743679Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1507
2025-10-09T21:49:24.743711Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:24.743681Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1507
2025-10-09T21:49:24.743715Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:24.743732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.743735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.743761Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8bf4f432-98c9aaa5-d7dceb8a-9d3e3b79_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:24.743769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:24.743772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:24.743773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:24.743780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:24.743788Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:24.743804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:24.743961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.743972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.743991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:24.744096Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1508 partition: 0 MaxSeqNo: 25 sessionId: src_id|8bf4f432-98c9aaa5-d7dceb8a-9d3e3b79_0
2025-10-09T21:49:24.744405Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dcce4950-29c51d91-da7d8e8f-de3fa069_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8bf4f432-98c9aaa5-d7dceb8a-9d3e3b79_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:24.746812Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8bf4f432-98c9aaa5-d7dceb8a-9d3e3b79_0] Write session: try to update token
2025-10-09T21:49:24.746835Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8bf4f432-98c9aaa5-d7dceb8a-9d3e3b79_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:24.758075Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:24.758092Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:24.758094Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:24.758095Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:24.758103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342432705059310:5404]
2025-10-09T21:49:24.758107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342432705059310:5404]
2025-10-09T21:49:24.758112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342432705059310:5404]
2025-10-09T21:49:24.758147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:24.758152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.758154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:24.758161Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:24.758167Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:24.758174Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:24.758286Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:24.759852Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1508 sessionId: src_id|8bf4f432-98c9aaa5-d7dceb8a-9d3e3b79_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:24.762163Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:24.824547Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342433066885431:2585]
2025-10-09T21:49:24.824741Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 1507, pipe: [1:7559342433066885431:2585]
2025-10-09T21:49:24.910584Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:24.910993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.911006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.911044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342433066885431:2585], now have 1 active actors on pipe
2025-10-09T21:49:24.911263Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:24.911273Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342433066885431:2585]
2025-10-09T21:49:24.911277Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:24.911296Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:24.911372Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1508 reason: error at write: Disconnected sessionId: src_id|8bf4f432-98c9aaa5-d7dceb8a-9d3e3b79_0
2025-10-09T21:49:24.911400Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:24.911546Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1508 sessionId: src_id|8bf4f432-98c9aaa5-d7dceb8a-9d3e3b79_0 is DEAD
2025-10-09T21:49:24.911977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8bf4f432-98c9aaa5-d7dceb8a-9d3e3b79_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:24.911990Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8bf4f432-98c9aaa5-d7dceb8a-9d3e3b79_0] Write session will restart in 0.008571s
2025-10-09T21:49:24.912028Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8bf4f432-98c9aaa5-d7dceb8a-9d3e3b79_0] Write session: Do CDS request
2025-10-09T21:49:24.912034Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8bf4f432-98c9aaa5-d7dceb8a-9d3e3b79_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:24.919983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:24.921154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:24.921356Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:24.921365Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:24.921377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:24.921434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:24.921472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8bf4f432-98c9aaa5-d7dceb8a-9d3e3b79_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:24.921475Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342432705059438:5365]
2025-10-09T21:49:24.921954Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:24.921965Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1509
2025-10-09T21:49:24.922114Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1509 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:24.922132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:24.922146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1509 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:24.922156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1509 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:24.922160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:24.922407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:24.922449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:24.922577Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1509 sessionId:  describe result for acl check
2025-10-09T21:49:24.922615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:24.922622Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:24.922624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:24.922630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342433066885498:2528] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:24.922756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:24.922801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.922807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.922808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.922809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.922810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.922811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.922812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.922813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.922815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.922819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.922830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.922834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.922838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.922842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.922846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.922851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.922854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.922858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.922864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.922877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:24.922883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.922888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:24.922892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.922902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:24.922910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.922917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:24.922925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.922930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:24.922938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.922944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:24.922952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.922958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.922965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.922970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.922978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.922984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.923017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:24.923162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342433066885498:2528] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:24.925183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342433066885498:2528] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:24.928948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342433066885498:2528] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:24.928962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342433066885498:2528] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:24.928966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342433066885498:2528] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:24.928975Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1509 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:24.929343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:24.930733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:24.930745Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:24.930746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:24.930749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:24.930754Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1508 [2:7559342432705059438:5365]
2025-10-09T21:49:24.930764Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:24.930779Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:24.930786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:24.930789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:24.930791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:24.930807Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:24.930816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:24.930822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:24.930878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.930886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.930887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.930890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:24.930954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:24.930961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.930974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:24.930986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:24.931000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:24.931019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342433066885437:2432], now have 1 active actors on pipe
2025-10-09T21:49:24.931027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342433066885533:2528], now have 1 active actors on pipe
2025-10-09T21:49:24.931034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342432705059433:5365]
2025-10-09T21:49:24.931036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342432705059433:5365]
2025-10-09T21:49:24.931038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342432705059433:5365]
2025-10-09T21:49:24.931050Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:24.931063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:24.931065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:24.931065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:24.931068Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:24.931079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:24.931083Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:24.931093Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1508
2025-10-09T21:49:24.931112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342432705059433:5365]
2025-10-09T21:49:24.931096Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1508
2025-10-09T21:49:24.931120Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:24.931146Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:24.931171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.931180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.931208Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8773ff7e-79fa1fcd-be9a3a60-2ff9dec5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:24.931220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:24.931224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:24.931226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:24.931235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:24.931249Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:24.931266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:24.931414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:24.931421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:24.931435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:24.931501Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1509 partition: 0 MaxSeqNo: 25 sessionId: src_id|8773ff7e-79fa1fcd-be9a3a60-2ff9dec5_0
2025-10-09T21:49:24.931826Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8bf4f432-98c9aaa5-d7dceb8a-9d3e3b79_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8773ff7e-79fa1fcd-be9a3a60-2ff9dec5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:24.934232Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8773ff7e-79fa1fcd-be9a3a60-2ff9dec5_0] Write session: try to update token
2025-10-09T21:49:24.934250Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8773ff7e-79fa1fcd-be9a3a60-2ff9dec5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:24.945321Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:24.945335Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:24.945337Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:24.945339Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:24.945349Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342432705059433:5365]
2025-10-09T21:49:24.945354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342432705059433:5365]
2025-10-09T21:49:24.945358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342432705059433:5365]
2025-10-09T21:49:24.945401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:24.945410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:24.945410Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:24.945412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:24.945416Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:24.945421Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:24.945541Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:24.946944Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1509 sessionId: src_id|8773ff7e-79fa1fcd-be9a3a60-2ff9dec5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:24.949260Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:25.011463Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342437361852838:2585]
2025-10-09T21:49:25.011724Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 1508, pipe: [1:7559342437361852838:2585]
2025-10-09T21:49:25.090109Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:25.090323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.090337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.090375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342437361852838:2585], now have 1 active actors on pipe
2025-10-09T21:49:25.090627Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342437361852838:2585]
2025-10-09T21:49:25.090636Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:25.090658Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:25.090661Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:25.090720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1509 reason: error at write: Disconnected sessionId: src_id|8773ff7e-79fa1fcd-be9a3a60-2ff9dec5_0
2025-10-09T21:49:25.090783Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:25.090902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1509 sessionId: src_id|8773ff7e-79fa1fcd-be9a3a60-2ff9dec5_0 is DEAD
2025-10-09T21:49:25.091053Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8773ff7e-79fa1fcd-be9a3a60-2ff9dec5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:25.091062Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8773ff7e-79fa1fcd-be9a3a60-2ff9dec5_0] Write session will restart in 0.009961s
2025-10-09T21:49:25.091079Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8773ff7e-79fa1fcd-be9a3a60-2ff9dec5_0] Write session: Do CDS request
2025-10-09T21:49:25.091084Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8773ff7e-79fa1fcd-be9a3a60-2ff9dec5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:25.098621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:25.100130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:25.100311Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:25.100318Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:25.100329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:25.100369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:25.100400Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342437000026820:5363]
2025-10-09T21:49:25.100796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:25.101051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:25.101079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:25.101302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:25.101334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.101338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.101339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.101340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.101340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.101341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.101342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.101342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.101343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.101346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.101353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.101355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.101359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.101361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.101363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.101365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.101367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.101369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.101374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.101378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.101383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.101386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.101389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.101391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.101394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.101397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.101399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.101402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.101404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.101407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.101410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.101413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.101415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.101418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.101422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.101424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.101439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:25.102549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8773ff7e-79fa1fcd-be9a3a60-2ff9dec5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:25.102627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:25.102648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1510
2025-10-09T21:49:25.102795Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1510 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:25.102825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1510 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:25.102832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1510 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:25.102836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:25.103201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1510 sessionId:  describe result for acl check
2025-10-09T21:49:25.103251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:25.103257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:25.103259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:25.103264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342437361852904:2516] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:25.103760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342437361852904:2516] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:25.105999Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342437361852904:2516] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:25.107637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:25.108325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342437361852904:2516] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:25.108337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342437361852904:2516] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:25.108342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342437361852904:2516] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:25.108372Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1510 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:25.108841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:25.108851Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:25.108853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:25.108857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:25.108862Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1509 [2:7559342437000026820:5363]
2025-10-09T21:49:25.108872Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:25.108885Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:25.108887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:25.108890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:25.108891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:25.108903Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:25.108906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:25.108912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:25.108958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.108965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.108966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.108969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:25.109019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:25.109024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.109031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.109037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.109046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:25.109062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342437361852938:2516], now have 1 active actors on pipe
2025-10-09T21:49:25.109067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342437361852839:2432], now have 1 active actors on pipe
2025-10-09T21:49:25.109073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342437000026815:5363]
2025-10-09T21:49:25.109074Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342437000026815:5363]
2025-10-09T21:49:25.109075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342437000026815:5363]
2025-10-09T21:49:25.109087Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:25.109098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:25.109109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:25.109111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:25.109117Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:25.109126Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:25.109129Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:25.109133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342437000026815:5363]
2025-10-09T21:49:25.109137Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:25.109141Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:25.109151Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1509
2025-10-09T21:49:25.109201Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1509
2025-10-09T21:49:25.109282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.109294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.109320Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1470dacb-1f087d67-fcea6235-169d5835_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:25.109331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:25.109335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:25.109337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:25.109345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:25.109359Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:25.109373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:25.109518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.109523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.109533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:25.109610Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1510 partition: 0 MaxSeqNo: 25 sessionId: src_id|1470dacb-1f087d67-fcea6235-169d5835_0
2025-10-09T21:49:25.109897Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8773ff7e-79fa1fcd-be9a3a60-2ff9dec5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1470dacb-1f087d67-fcea6235-169d5835_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:25.112378Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1470dacb-1f087d67-fcea6235-169d5835_0] Write session: try to update token
2025-10-09T21:49:25.112397Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1470dacb-1f087d67-fcea6235-169d5835_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:25.122973Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:25.122990Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:25.122992Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:25.122994Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:25.123005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342437000026815:5363]
2025-10-09T21:49:25.123009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342437000026815:5363]
2025-10-09T21:49:25.123012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342437000026815:5363]
2025-10-09T21:49:25.123055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:25.123064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.123067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:25.123069Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:25.123076Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:25.123080Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:25.123238Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:25.124664Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1510 sessionId: src_id|1470dacb-1f087d67-fcea6235-169d5835_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:25.127094Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:25.190654Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342437361852949:2585]
2025-10-09T21:49:25.190948Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 1509, pipe: [1:7559342437361852949:2585]
2025-10-09T21:49:25.273373Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:25.273622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.273636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.273677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342437361852949:2585], now have 1 active actors on pipe
2025-10-09T21:49:25.273847Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:25.273905Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:25.273917Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:25.273932Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342437361852949:2585]
2025-10-09T21:49:25.273952Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:25.273991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1510 reason: error at write: Disconnected sessionId: src_id|1470dacb-1f087d67-fcea6235-169d5835_0
2025-10-09T21:49:25.274178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1510 sessionId: src_id|1470dacb-1f087d67-fcea6235-169d5835_0 is DEAD
2025-10-09T21:49:25.274318Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1470dacb-1f087d67-fcea6235-169d5835_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:25.274343Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1470dacb-1f087d67-fcea6235-169d5835_0] Write session will restart in 0.008325s
2025-10-09T21:49:25.274370Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1470dacb-1f087d67-fcea6235-169d5835_0] Write session: Do CDS request
2025-10-09T21:49:25.274376Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1470dacb-1f087d67-fcea6235-169d5835_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:25.283072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:25.283507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1470dacb-1f087d67-fcea6235-169d5835_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:25.283595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:25.283609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1511
2025-10-09T21:49:25.283747Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1511 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:25.283787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1511 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:25.283797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1511 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:25.283801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:25.284255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1511 sessionId:  describe result for acl check
2025-10-09T21:49:25.284296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:25.284304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:25.284305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:25.284311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342437361853016:2510] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:25.284679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:25.284871Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:25.284881Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:25.284857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342437361853016:2510] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:25.284893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:25.284945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:25.284976Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342437000026907:5359]
2025-10-09T21:49:25.285360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:25.285699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:25.285730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:25.285959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:25.286002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.286006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.286007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.286008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.286009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.286009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.286010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.286011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.286011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.286014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.286024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.286062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.286067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.286070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.286072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.286074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.286076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.286079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.286083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.286089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.286093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.286097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.286100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.286103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.286105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.286110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.286113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.286117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.286120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.286123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.286126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.286129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.286131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.286134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.286136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.286139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.286156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:25.286698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342437361853016:2510] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:25.292077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342437361853016:2510] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:25.292092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342437361853016:2510] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:25.292098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342437361853016:2510] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:25.292117Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1511 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:25.292463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:25.293844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:25.293859Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:25.293862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:25.293866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:25.293874Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1510 [2:7559342437000026907:5359]
2025-10-09T21:49:25.293887Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:25.293905Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:25.293909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:25.293912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:25.293915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:25.293931Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:25.293942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:25.293949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:25.294012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.294017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.294019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.294022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:25.294107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:25.294114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.294127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.294143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.294162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:25.294190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342437361852956:2432], now have 1 active actors on pipe
2025-10-09T21:49:25.294201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342437361853054:2510], now have 1 active actors on pipe
2025-10-09T21:49:25.294208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342437000026902:5359]
2025-10-09T21:49:25.294211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342437000026902:5359]
2025-10-09T21:49:25.294213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342437000026902:5359]
2025-10-09T21:49:25.294226Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:25.294240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:25.294241Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:25.294245Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:25.294247Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:25.294256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:25.294259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:25.294292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342437000026902:5359]
2025-10-09T21:49:25.294304Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:25.294295Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1510
2025-10-09T21:49:25.294304Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1510
2025-10-09T21:49:25.294326Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:25.294371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.294381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.294410Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bc2d2b77-9056f56-bd1b1fb-9b1810bb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:25.294422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:25.294426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:25.294429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:25.294438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:25.294453Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:25.294470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:25.294689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.294701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.294724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:25.294838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1511 partition: 0 MaxSeqNo: 25 sessionId: src_id|bc2d2b77-9056f56-bd1b1fb-9b1810bb_0
2025-10-09T21:49:25.295171Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1470dacb-1f087d67-fcea6235-169d5835_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bc2d2b77-9056f56-bd1b1fb-9b1810bb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:25.297531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2d2b77-9056f56-bd1b1fb-9b1810bb_0] Write session: try to update token
2025-10-09T21:49:25.297552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2d2b77-9056f56-bd1b1fb-9b1810bb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:25.308880Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:25.308898Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:25.308901Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:25.308902Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:25.308911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342437000026902:5359]
2025-10-09T21:49:25.308916Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342437000026902:5359]
2025-10-09T21:49:25.308920Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342437000026902:5359]
2025-10-09T21:49:25.308966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:25.308973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.308975Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:25.308980Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:25.308982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:25.308984Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:25.309130Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:25.310930Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1511 sessionId: src_id|bc2d2b77-9056f56-bd1b1fb-9b1810bb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:25.313324Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:25.373950Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342437361853063:2585]
2025-10-09T21:49:25.374248Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 1510, pipe: [1:7559342437361853063:2585]
2025-10-09T21:49:25.457000Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:25.457363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.457376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.457413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342437361853063:2585], now have 1 active actors on pipe
2025-10-09T21:49:25.457555Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:25.457580Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:25.457611Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342437361853063:2585]
2025-10-09T21:49:25.457635Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:25.457699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1511 reason: status is not ok: tablet will be restarted right now sessionId: src_id|bc2d2b77-9056f56-bd1b1fb-9b1810bb_0
2025-10-09T21:49:25.457857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1511 sessionId: src_id|bc2d2b77-9056f56-bd1b1fb-9b1810bb_0 is DEAD
2025-10-09T21:49:25.457993Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2d2b77-9056f56-bd1b1fb-9b1810bb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:25.458005Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2d2b77-9056f56-bd1b1fb-9b1810bb_0] Write session will restart in 0.009222s
2025-10-09T21:49:25.458051Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2d2b77-9056f56-bd1b1fb-9b1810bb_0] Write session: Do CDS request
2025-10-09T21:49:25.458058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2d2b77-9056f56-bd1b1fb-9b1810bb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:25.465896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:25.467420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:25.467608Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:25.467618Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:25.467627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:25.467672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:25.467704Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342437000026995:5357]
2025-10-09T21:49:25.468096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:25.468392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:25.468437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:25.468471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2d2b77-9056f56-bd1b1fb-9b1810bb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:25.468554Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:25.468567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1512
2025-10-09T21:49:25.468678Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1512 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:25.468709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1512 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:25.468714Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1512 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:25.468711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:25.468722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:25.468769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.468775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.468777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.468778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.468780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.468781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.468782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.468783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.468784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.468790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.468799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.468803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.468807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.468812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.468816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.468819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.468822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.468826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.468833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.468840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.468851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.468861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.468869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.468874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.468884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.468889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.468893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.468898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.468904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.468909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.468911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.468914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.468917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.468919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.468922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.468925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.468941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:25.469125Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1512 sessionId:  describe result for acl check
2025-10-09T21:49:25.469163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:25.469171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:25.469172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:25.469178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342437361853130:2505] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:25.469766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342437361853130:2505] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:25.471625Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342437361853130:2505] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:25.474671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342437361853130:2505] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:25.474686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342437361853130:2505] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:25.474690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342437361853130:2505] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:25.474699Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1512 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:25.475117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:25.476368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:25.476380Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:25.476382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:25.476385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:25.476389Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1511 [2:7559342437000026995:5357]
2025-10-09T21:49:25.476398Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:25.476414Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:25.476416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:25.476418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:25.476419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:25.476434Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:25.476439Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:25.476447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:25.476509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.476516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.476518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.476521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:25.476598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:25.476608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.476619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.476631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.476648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:25.476667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342437361853165:2505], now have 1 active actors on pipe
2025-10-09T21:49:25.476670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342437361853065:2432], now have 1 active actors on pipe
2025-10-09T21:49:25.476676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342437000026990:5357]
2025-10-09T21:49:25.476677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342437000026990:5357]
2025-10-09T21:49:25.476678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342437000026990:5357]
2025-10-09T21:49:25.476688Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:25.476698Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:25.476705Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:25.476707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:25.476709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:25.476714Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:25.476716Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:25.476721Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1511
2025-10-09T21:49:25.476727Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1511
2025-10-09T21:49:25.476734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342437000026990:5357]
2025-10-09T21:49:25.476741Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:25.476756Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:25.476790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.476794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.476823Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|14531589-135ab77c-d874acef-ca5c226_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:25.476837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:25.476840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:25.476843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:25.476851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:25.476861Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:25.476877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:25.477028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.477038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.477053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:25.477154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1512 partition: 0 MaxSeqNo: 25 sessionId: src_id|14531589-135ab77c-d874acef-ca5c226_0
2025-10-09T21:49:25.477449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2d2b77-9056f56-bd1b1fb-9b1810bb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|14531589-135ab77c-d874acef-ca5c226_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:25.479973Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14531589-135ab77c-d874acef-ca5c226_0] Write session: try to update token
2025-10-09T21:49:25.479994Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14531589-135ab77c-d874acef-ca5c226_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:25.487857Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:25.487871Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:25.487892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342437000026990:5357]
2025-10-09T21:49:25.487940Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:25.487951Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:25.488177Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:25.488349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:25.488486Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:25.488622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:25.488813Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:25.488859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:25.492904Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1512 sessionId: src_id|14531589-135ab77c-d874acef-ca5c226_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:25.495219Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:25.495389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.495403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.495436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:25.495543Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:25.501207Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:25.501222Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:25.501225Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:25.501230Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:25.501239Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342437000026990:5357]
2025-10-09T21:49:25.501243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342437000026990:5357]
2025-10-09T21:49:25.501245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342437000026990:5357]
2025-10-09T21:49:25.501285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:25.501288Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:25.501296Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:25.501297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.501298Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:25.501300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:25.501456Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:25.557772Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342437361853173:2585]
2025-10-09T21:49:25.558047Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 1511, pipe: [1:7559342437361853173:2585]
2025-10-09T21:49:25.654335Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:25.661250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.661273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.661421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.661482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.661543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.661610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.661650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.661658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.661722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.661784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.661839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.661897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.661937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.661942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.661999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.662065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.662129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.662187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.662225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.662228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.662289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.662356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.662409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.662468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.662510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.662522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.662618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.662700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.662754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.662808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.662853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.662860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.662943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.663001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.663055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.663115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.663176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.663183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.663242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.663355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.663487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.663590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.663640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.663645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.663698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.663747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.663801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.663854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.663912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.663917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.663978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.664030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.664084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.664145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.664185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.664189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.664244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.664297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.664350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.664404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.664452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.664460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.664512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342437361853173:2585], now have 1 active actors on pipe
2025-10-09T21:49:25.664759Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:25.664781Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342437361853173:2585]
2025-10-09T21:49:25.664804Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:25.664806Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:25.664834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1512 reason: error at write: Disconnected sessionId: src_id|14531589-135ab77c-d874acef-ca5c226_0
2025-10-09T21:49:25.665015Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1512 sessionId: src_id|14531589-135ab77c-d874acef-ca5c226_0 is DEAD
2025-10-09T21:49:25.665193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14531589-135ab77c-d874acef-ca5c226_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:25.665202Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14531589-135ab77c-d874acef-ca5c226_0] Write session will restart in 0.006042s
2025-10-09T21:49:25.665219Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14531589-135ab77c-d874acef-ca5c226_0] Write session: Do CDS request
2025-10-09T21:49:25.665223Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14531589-135ab77c-d874acef-ca5c226_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:25.665349Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:25.672489Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14531589-135ab77c-d874acef-ca5c226_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:25.672567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:25.672579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1513
2025-10-09T21:49:25.672686Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1513 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:25.672731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1513 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:25.672741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1513 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:25.672745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:25.673139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1513 sessionId:  describe result for acl check
2025-10-09T21:49:25.673177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:25.673183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:25.673187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:25.673193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342437361853235:2495] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:25.673757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342437361853235:2495] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:25.674659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:25.676243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342437361853235:2495] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:25.676392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:25.676551Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:25.676560Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:25.676571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:25.676620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:25.676666Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342437000027084:5352]
2025-10-09T21:49:25.677263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:25.677569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:25.677652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:25.678019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:25.678101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.678107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.678108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.678110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.678111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.678112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.678114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.678115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.678116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.678122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.678133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.678137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.678141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.678145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.678149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.678153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.678157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.678165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.678175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.678187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.678197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.678208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.678212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.678217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.678221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.678226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.678235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.678240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.678246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.678255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.678260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.678269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.678275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.678280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.678284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.678293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.678322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:25.678686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342437361853235:2495] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:25.678699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342437361853235:2495] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:25.678703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342437361853235:2495] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:25.678710Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1513 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:25.684546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:25.685994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:25.686006Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:25.686007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:25.686011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:25.686015Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1512 [2:7559342437000027084:5352]
2025-10-09T21:49:25.686024Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:25.686055Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:25.686060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:25.686063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:25.686064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:25.686078Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:25.686084Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:25.686090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:25.686134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.686139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.686140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.686142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:25.686195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:25.686204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.686216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.686224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.686233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:25.686248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342437000027079:5352]
2025-10-09T21:49:25.686250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342437000027079:5352]
2025-10-09T21:49:25.686251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342437000027079:5352]
2025-10-09T21:49:25.686257Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:25.686268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:25.686272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:25.686275Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:25.686277Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:25.686288Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:25.686291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:25.686303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342437361853276:2495], now have 1 active actors on pipe
2025-10-09T21:49:25.686328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342437361853175:2432], now have 1 active actors on pipe
2025-10-09T21:49:25.686333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342437000027079:5352]
2025-10-09T21:49:25.686339Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:25.686353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:25.686394Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1512
2025-10-09T21:49:25.686404Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1512
2025-10-09T21:49:25.686467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.686479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.686509Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8e9f68a8-5011b1c7-1eaf842c-c5021333_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:25.686518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:25.686522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:25.686524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:25.686534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:25.686544Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:25.686563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:25.686691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.686697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.686711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:25.686819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1513 partition: 0 MaxSeqNo: 25 sessionId: src_id|8e9f68a8-5011b1c7-1eaf842c-c5021333_0
2025-10-09T21:49:25.687132Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14531589-135ab77c-d874acef-ca5c226_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8e9f68a8-5011b1c7-1eaf842c-c5021333_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:25.689419Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e9f68a8-5011b1c7-1eaf842c-c5021333_0] Write session: try to update token
2025-10-09T21:49:25.689442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e9f68a8-5011b1c7-1eaf842c-c5021333_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:25.701957Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1513 sessionId: src_id|8e9f68a8-5011b1c7-1eaf842c-c5021333_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:25.702387Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:25.702401Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:25.702408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342437000027079:5352]
2025-10-09T21:49:25.702456Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:25.702457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:25.702619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:25.702745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:25.702869Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:25.703023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:25.703208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:25.703245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:25.704375Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:25.704559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.704576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.704611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:25.704687Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:25.710509Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:25.710518Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:25.710523Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:25.710524Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:25.710531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342437000027079:5352]
2025-10-09T21:49:25.710534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342437000027079:5352]
2025-10-09T21:49:25.710536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342437000027079:5352]
2025-10-09T21:49:25.710564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:25.710573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.710573Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:25.710574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:25.710577Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:25.710580Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:25.710697Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:25.765132Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342437361853284:2585]
2025-10-09T21:49:25.765411Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 1512, pipe: [1:7559342437361853284:2585]
2025-10-09T21:49:25.861644Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:25.868745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.868770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.868934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.868988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.869053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.869120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.869167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.869178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.869248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.869311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.869373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.869448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.869493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.869502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.869569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.869643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.869704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.869764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.870121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.870133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.870203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.870270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.870337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.870425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.870466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.870473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.870541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.870599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.870703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.870764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.870809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.870816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.870876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.870935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.870992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.871050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.871092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.871098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.871156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.871215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.871272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.871326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.871365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.871372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.871427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.871491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.871577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.871649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.871693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.871698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.871766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.871859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.871926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.871992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.872039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.872047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.872121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:25.872192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:25.872259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:25.872326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:25.872389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:25.872395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:25.872465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342437361853284:2585], now have 1 active actors on pipe
2025-10-09T21:49:25.872712Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:25.872757Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342437361853284:2585]
2025-10-09T21:49:25.872782Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:25.872791Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:25.872818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1513 reason: error at write: Disconnected sessionId: src_id|8e9f68a8-5011b1c7-1eaf842c-c5021333_0
2025-10-09T21:49:25.873016Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1513 sessionId: src_id|8e9f68a8-5011b1c7-1eaf842c-c5021333_0 is DEAD
2025-10-09T21:49:25.873168Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e9f68a8-5011b1c7-1eaf842c-c5021333_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:25.873175Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e9f68a8-5011b1c7-1eaf842c-c5021333_0] Write session will restart in 0.009731s
2025-10-09T21:49:25.873162Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:25.873201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e9f68a8-5011b1c7-1eaf842c-c5021333_0] Write session: Do CDS request
2025-10-09T21:49:25.873208Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e9f68a8-5011b1c7-1eaf842c-c5021333_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:25.881983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:25.883645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:25.883801Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:25.883809Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:25.883818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:25.883852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:25.883892Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342437000027217:5330]
2025-10-09T21:49:25.884281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:25.884545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e9f68a8-5011b1c7-1eaf842c-c5021333_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:25.884602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:25.884595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:25.884612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1514
2025-10-09T21:49:25.884636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:25.884759Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1514 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:25.884792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1514 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:25.884796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1514 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:25.884799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:25.884872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:25.884906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.884911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.884911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.884913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.884914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.884914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.884915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.884915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.884916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.884919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.884926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.884928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.884931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.884933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.884935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.884937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.884939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.884941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.884947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.884952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:25.884957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.884960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:25.884962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.884965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:25.884967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.884970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:25.884973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.884975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:25.884978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.884981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:25.884993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.884996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.884999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.885002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.885007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.885010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.885028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:25.885155Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1514 sessionId:  describe result for acl check
2025-10-09T21:49:25.885181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:25.885186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:25.885187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:25.885191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342437361853380:2475] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:25.885666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342437361853380:2475] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:25.888347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342437361853380:2475] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:25.890897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342437361853380:2475] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:25.890909Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342437361853380:2475] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:25.890912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342437361853380:2475] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:25.890924Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1514 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:25.891354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:25.892467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:25.892480Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:25.892482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:25.892487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:25.892493Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1513 [2:7559342437000027217:5330]
2025-10-09T21:49:25.892507Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:25.892525Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:25.892529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:25.892532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:25.892534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:25.892552Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:25.892562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:25.892570Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:25.892625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.892632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.892634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.892637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:25.892703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:25.892710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.892722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:25.892738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:25.892762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:25.892787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342437361853314:2432], now have 1 active actors on pipe
2025-10-09T21:49:25.892795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342437361853413:2475], now have 1 active actors on pipe
2025-10-09T21:49:25.892802Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342437000027212:5330]
2025-10-09T21:49:25.892809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342437000027212:5330]
2025-10-09T21:49:25.892811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342437000027212:5330]
2025-10-09T21:49:25.892824Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:25.892818Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1513
2025-10-09T21:49:25.892837Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:25.892844Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:25.892848Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:25.892839Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1513
2025-10-09T21:49:25.892857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:25.892860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:25.892862Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:25.892887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342437000027212:5330]
2025-10-09T21:49:25.892897Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:25.892903Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:25.892916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.892925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.892970Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|82ad6854-2535f129-21c42d5c-90d9084e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:25.892984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:25.892987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:25.892990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:25.892999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:25.893009Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:25.893033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:25.893163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:25.893172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:25.893191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:25.893305Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1514 partition: 0 MaxSeqNo: 25 sessionId: src_id|82ad6854-2535f129-21c42d5c-90d9084e_0
2025-10-09T21:49:25.893624Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e9f68a8-5011b1c7-1eaf842c-c5021333_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|82ad6854-2535f129-21c42d5c-90d9084e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:25.896000Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|82ad6854-2535f129-21c42d5c-90d9084e_0] Write session: try to update token
2025-10-09T21:49:25.896015Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|82ad6854-2535f129-21c42d5c-90d9084e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:25.907502Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:25.907520Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:25.907523Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:25.907525Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:25.907534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342437000027212:5330]
2025-10-09T21:49:25.907542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342437000027212:5330]
2025-10-09T21:49:25.907547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342437000027212:5330]
2025-10-09T21:49:25.907588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:25.907590Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:25.907594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:25.907595Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:25.907596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:25.907601Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:25.907724Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:25.909382Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1514 sessionId: src_id|82ad6854-2535f129-21c42d5c-90d9084e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:25.911845Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:25.972972Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342437361853420:2585]
2025-10-09T21:49:25.973299Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 1513, pipe: [1:7559342437361853420:2585]
2025-10-09T21:49:26.064648Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:26.064915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:26.064929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:26.064975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342437361853420:2585], now have 1 active actors on pipe
2025-10-09T21:49:26.065195Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342437361853420:2585]
2025-10-09T21:49:26.065237Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:26.065239Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:26.065305Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1514 reason: error at write: Disconnected sessionId: src_id|82ad6854-2535f129-21c42d5c-90d9084e_0
2025-10-09T21:49:26.065441Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:26.065482Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1514 sessionId: src_id|82ad6854-2535f129-21c42d5c-90d9084e_0 is DEAD
2025-10-09T21:49:26.065643Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82ad6854-2535f129-21c42d5c-90d9084e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:26.065652Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82ad6854-2535f129-21c42d5c-90d9084e_0] Write session will restart in 0.007350s
2025-10-09T21:49:26.065632Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:26.065691Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82ad6854-2535f129-21c42d5c-90d9084e_0] Write session: Do CDS request
2025-10-09T21:49:26.065697Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82ad6854-2535f129-21c42d5c-90d9084e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:26.074473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|82ad6854-2535f129-21c42d5c-90d9084e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:26.074579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:26.074594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1515
2025-10-09T21:49:26.074610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:26.074740Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1515 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:26.074786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1515 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:26.074796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1515 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:26.074800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:26.075247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1515 sessionId:  describe result for acl check
2025-10-09T21:49:26.075286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:26.075292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:26.075293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:26.075300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342441656820782:2466] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:26.075873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342441656820782:2466] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:26.076151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:26.076331Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:26.076340Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:26.076351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:26.076422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:26.076461Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342441294994605:5325]
2025-10-09T21:49:26.076993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:26.077277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:26.077321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:26.077658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:26.077708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.077713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.077714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.077715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.077716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.077717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.077718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.077719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.077720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.077725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.077733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.077737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.077740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.077744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.077748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.077751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.077754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.077757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.077766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.077774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.077780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.077786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.077790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.077794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.077798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.077803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.077808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.077812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342441656820782:2466] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:26.077812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.077819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.077824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.077829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.077833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.077837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.077841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.077846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.077850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.077876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:26.083822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342441656820782:2466] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:26.083835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342441656820782:2466] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:26.083839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342441656820782:2466] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:26.083849Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1515 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:26.084220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:26.085515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:26.085527Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:26.085529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:26.085532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:26.085537Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1514 [2:7559342441294994605:5325]
2025-10-09T21:49:26.085547Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:26.085561Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:26.085563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:26.085566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:26.085567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:26.085583Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:26.085597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:26.085604Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:26.085650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.085658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.085660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.085662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:26.085728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:26.085734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.085741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.085747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.085761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:26.085780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342441656820719:2432], now have 1 active actors on pipe
2025-10-09T21:49:26.085783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342441656820821:2466], now have 1 active actors on pipe
2025-10-09T21:49:26.085791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342441294994600:5325]
2025-10-09T21:49:26.085793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342441294994600:5325]
2025-10-09T21:49:26.085795Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342441294994600:5325]
2025-10-09T21:49:26.085808Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:26.085826Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:26.085843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:26.085852Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:26.085861Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:26.085867Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:26.085872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:26.085881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342441294994600:5325]
2025-10-09T21:49:26.085885Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:26.085898Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:26.085942Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1514
2025-10-09T21:49:26.085957Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1514
2025-10-09T21:49:26.086045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:26.086059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:26.086090Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|136c7854-e2c1dd31-6ac10f41-b835427a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:26.086104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:26.086108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:26.086110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:26.086118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:26.086129Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:26.086141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:26.086320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:26.086329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:26.086357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:26.086443Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1515 partition: 0 MaxSeqNo: 25 sessionId: src_id|136c7854-e2c1dd31-6ac10f41-b835427a_0
2025-10-09T21:49:26.086814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82ad6854-2535f129-21c42d5c-90d9084e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|136c7854-e2c1dd31-6ac10f41-b835427a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:26.089195Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|136c7854-e2c1dd31-6ac10f41-b835427a_0] Write session: try to update token
2025-10-09T21:49:26.089211Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|136c7854-e2c1dd31-6ac10f41-b835427a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:26.097490Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:26.097507Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:26.097516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342441294994600:5325]
2025-10-09T21:49:26.097562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:26.097561Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:26.097738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:26.097871Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:26.098012Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:26.098156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:26.098338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:26.098378Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:26.101431Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1515 sessionId: src_id|136c7854-e2c1dd31-6ac10f41-b835427a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:26.103960Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:26.104136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:26.104154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:26.104195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:26.104279Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:26.110704Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:26.110720Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:26.110723Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:26.110725Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:26.110736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342441294994600:5325]
2025-10-09T21:49:26.110742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342441294994600:5325]
2025-10-09T21:49:26.110748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342441294994600:5325]
2025-10-09T21:49:26.110789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:26.110792Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:26.110798Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:26.110800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.110802Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:26.110803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:26.110970Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:26.165509Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342441656820831:2585]
2025-10-09T21:49:26.165797Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 1514, pipe: [1:7559342441656820831:2585]
2025-10-09T21:49:26.266514Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:26.274341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:26.274363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:26.274523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:26.274585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:26.274647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:26.274710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:26.274756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:26.274767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:26.274831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:26.274886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:26.274944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:26.275007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:26.275047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:26.275054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:26.275111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:26.275168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:26.275227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:26.275289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:26.275329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:26.275339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:26.275398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:26.275462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:26.275523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:26.275579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:26.275617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:26.275625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:26.275686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:26.275748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:26.275847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:26.275907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:26.275962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:26.275969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:26.276031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:26.276094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:26.276156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:26.276213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:26.276272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:26.276279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:26.276337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:26.276392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:26.276451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:26.276510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:26.276557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:26.276564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:26.276625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:26.276680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:26.276738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:26.276795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:26.276838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:26.276849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:26.276909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:26.276966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:26.277022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:26.277083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:26.277216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:26.277224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:26.277283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:26.277339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:26.277398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:26.277452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:26.277593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:26.277603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:26.277662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342441656820831:2585], now have 1 active actors on pipe
2025-10-09T21:49:26.277882Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:26.277914Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:26.277918Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:26.277947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1515 reason: error at write: Disconnected sessionId: src_id|136c7854-e2c1dd31-6ac10f41-b835427a_0
2025-10-09T21:49:26.277954Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342441656820831:2585]
2025-10-09T21:49:26.278137Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1515 sessionId: src_id|136c7854-e2c1dd31-6ac10f41-b835427a_0 is DEAD
2025-10-09T21:49:26.278293Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|136c7854-e2c1dd31-6ac10f41-b835427a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:26.278305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|136c7854-e2c1dd31-6ac10f41-b835427a_0] Write session will restart in 0.005388s
2025-10-09T21:49:26.278331Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|136c7854-e2c1dd31-6ac10f41-b835427a_0] Write session: Do CDS request
2025-10-09T21:49:26.278336Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|136c7854-e2c1dd31-6ac10f41-b835427a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:26.278715Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:26.284507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|136c7854-e2c1dd31-6ac10f41-b835427a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:26.284614Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:26.284626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1516
2025-10-09T21:49:26.284793Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1516 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:26.284825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1516 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:26.284831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1516 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:26.284835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:26.285284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1516 sessionId:  describe result for acl check
2025-10-09T21:49:26.285324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:26.285331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:26.285336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:26.285342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342441656820892:2457] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:26.285928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342441656820892:2457] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:26.288097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342441656820892:2457] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:26.289560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:26.290021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:26.290253Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:26.290270Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:26.290284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:26.290349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:26.290408Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342441294994698:5316]
2025-10-09T21:49:26.290634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342441656820892:2457] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:26.290647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342441656820892:2457] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:26.290651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342441656820892:2457] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:26.290661Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1516 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:26.291068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:26.291381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:26.291439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:26.291800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:26.291879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.291888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.291890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.291891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.291892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.291894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.291896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.291897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.291898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.291904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.291915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.291919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.291923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.291928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.291931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.291935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.291939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.291943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.291950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.291958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.291965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.291971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.291975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.291981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.291988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.291993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.292000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.292005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.292010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.292015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.292021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.292029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.292033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.292038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.292042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.292049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.292087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:26.298671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:26.299785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:26.299798Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:26.299800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:26.299803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:26.299809Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1515 [2:7559342441294994698:5316]
2025-10-09T21:49:26.299826Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:26.299846Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:26.299849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:26.299853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:26.299855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:26.299875Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:26.299879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:26.299885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:26.299934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.299946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.299949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.299954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:26.300021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:26.300031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.300044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.300061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.300083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:26.300112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342441656820939:2457], now have 1 active actors on pipe
2025-10-09T21:49:26.300122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342441656820839:2432], now have 1 active actors on pipe
2025-10-09T21:49:26.300130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342441294994692:5316]
2025-10-09T21:49:26.300133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342441294994692:5316]
2025-10-09T21:49:26.300135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342441294994692:5316]
2025-10-09T21:49:26.300149Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:26.300165Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:26.300168Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:26.300172Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:26.300174Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:26.300187Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:26.300191Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:26.300178Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1515
2025-10-09T21:49:26.300185Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1515
2025-10-09T21:49:26.300230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342441294994692:5316]
2025-10-09T21:49:26.300239Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:26.300242Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:26.300256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:26.300265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:26.300307Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|90232f87-7e655cf9-1ea8d935-492e9202_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:26.300321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:26.300325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:26.300328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:26.300339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:26.300356Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:26.300380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:26.300521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:26.300536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:26.300563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:26.300693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1516 partition: 0 MaxSeqNo: 25 sessionId: src_id|90232f87-7e655cf9-1ea8d935-492e9202_0
2025-10-09T21:49:26.301038Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|136c7854-e2c1dd31-6ac10f41-b835427a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|90232f87-7e655cf9-1ea8d935-492e9202_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:26.303502Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90232f87-7e655cf9-1ea8d935-492e9202_0] Write session: try to update token
2025-10-09T21:49:26.303523Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90232f87-7e655cf9-1ea8d935-492e9202_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:26.314429Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:26.314446Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:26.314448Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:26.314450Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:26.314460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342441294994692:5316]
2025-10-09T21:49:26.314465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342441294994692:5316]
2025-10-09T21:49:26.314469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342441294994692:5316]
2025-10-09T21:49:26.314515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:26.314523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.314527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:26.314531Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:26.314538Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:26.314542Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:26.314661Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:26.316897Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1516 sessionId: src_id|90232f87-7e655cf9-1ea8d935-492e9202_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:26.319130Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:26.377987Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342441656820951:2585]
2025-10-09T21:49:26.378255Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 1515, pipe: [1:7559342441656820951:2585]
2025-10-09T21:49:26.466761Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:26.466999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:26.467012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:26.467053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342441656820951:2585], now have 1 active actors on pipe
2025-10-09T21:49:26.467221Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:26.467276Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342441656820951:2585]
2025-10-09T21:49:26.467290Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:26.467293Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:26.467307Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:26.467368Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1516 reason: error at write: Disconnected sessionId: src_id|90232f87-7e655cf9-1ea8d935-492e9202_0
2025-10-09T21:49:26.467552Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1516 sessionId: src_id|90232f87-7e655cf9-1ea8d935-492e9202_0 is DEAD
2025-10-09T21:49:26.467698Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90232f87-7e655cf9-1ea8d935-492e9202_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:26.467708Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90232f87-7e655cf9-1ea8d935-492e9202_0] Write session will restart in 0.009454s
2025-10-09T21:49:26.467735Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90232f87-7e655cf9-1ea8d935-492e9202_0] Write session: Do CDS request
2025-10-09T21:49:26.467742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90232f87-7e655cf9-1ea8d935-492e9202_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:26.476371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:26.478138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:26.478308Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:26.478318Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:26.478339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:26.478385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:26.478411Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90232f87-7e655cf9-1ea8d935-492e9202_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:26.478425Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342441294994790:5322]
2025-10-09T21:49:26.478536Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:26.478545Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1517
2025-10-09T21:49:26.478699Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1517 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:26.478731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1517 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:26.478738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1517 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:26.478743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:26.479003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:26.479062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1517 sessionId:  describe result for acl check
2025-10-09T21:49:26.479091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:26.479092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:26.479094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:26.479100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342441656821020:2453] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:26.479254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:26.479289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:26.479591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342441656821020:2453] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:26.479601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:26.479643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.479648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.479650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.479651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.479652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.479653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.479654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.479655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.479656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.479661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.479671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.479675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.479679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.479685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.479689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.479693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.479697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.479701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.479707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.479715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.479721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.479727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.479731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.479736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.479742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.479747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.479751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.479756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.479761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.479767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.479777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.479782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.479790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.479795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.479803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.479809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.479838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:26.481581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342441656821020:2453] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:26.485405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342441656821020:2453] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:26.485416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342441656821020:2453] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:26.485418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342441656821020:2453] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:26.485424Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1517 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:26.485826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:26.486837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:26.486848Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:26.486849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:26.486852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:26.486857Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1516 [2:7559342441294994790:5322]
2025-10-09T21:49:26.486865Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:26.486880Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:26.486882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:26.486884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:26.486885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:26.486897Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:26.486901Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:26.486906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:26.486959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.486966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.486968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.486970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:26.487023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:26.487026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.487034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.487040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.487054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:26.487066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342441294994785:5322]
2025-10-09T21:49:26.487072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342441294994785:5322]
2025-10-09T21:49:26.487074Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342441294994785:5322]
2025-10-09T21:49:26.487080Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:26.487091Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:26.487092Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:26.487093Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:26.487095Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:26.487101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:26.487102Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:26.487111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342441656820953:2432], now have 1 active actors on pipe
2025-10-09T21:49:26.487114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342441656821058:2453], now have 1 active actors on pipe
2025-10-09T21:49:26.487131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342441294994785:5322]
2025-10-09T21:49:26.487138Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:26.487147Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1516
2025-10-09T21:49:26.487168Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1516
2025-10-09T21:49:26.487170Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:26.487206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:26.487211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:26.487238Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a64edbbe-410660bb-41df564b-946565c4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:26.487248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:26.487255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:26.487256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:26.487263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:26.487270Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:26.487284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:26.487430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:26.487435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:26.487446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:26.487536Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1517 partition: 0 MaxSeqNo: 25 sessionId: src_id|a64edbbe-410660bb-41df564b-946565c4_0
2025-10-09T21:49:26.487828Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90232f87-7e655cf9-1ea8d935-492e9202_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a64edbbe-410660bb-41df564b-946565c4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:26.489958Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a64edbbe-410660bb-41df564b-946565c4_0] Write session: try to update token
2025-10-09T21:49:26.489970Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a64edbbe-410660bb-41df564b-946565c4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:26.500847Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:26.500863Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:26.500865Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:26.500867Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:26.500875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342441294994785:5322]
2025-10-09T21:49:26.500879Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342441294994785:5322]
2025-10-09T21:49:26.500883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342441294994785:5322]
2025-10-09T21:49:26.500927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:26.500934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.500936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:26.500945Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:26.500953Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:26.500957Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:26.501055Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:26.501582Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1517 sessionId: src_id|a64edbbe-410660bb-41df564b-946565c4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:26.503901Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:26.567376Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342441656821066:2585]
2025-10-09T21:49:26.567690Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 1516, pipe: [1:7559342441656821066:2585]
2025-10-09T21:49:26.643863Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:26.644382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:26.644398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:26.644435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342441656821066:2585], now have 1 active actors on pipe
2025-10-09T21:49:26.644614Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:26.644670Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:26.644675Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342441656821066:2585]
2025-10-09T21:49:26.644685Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:26.644695Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:26.644736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1517 reason: error at write: Disconnected sessionId: src_id|a64edbbe-410660bb-41df564b-946565c4_0
2025-10-09T21:49:26.644902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1517 sessionId: src_id|a64edbbe-410660bb-41df564b-946565c4_0 is DEAD
2025-10-09T21:49:26.645111Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a64edbbe-410660bb-41df564b-946565c4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:26.645123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a64edbbe-410660bb-41df564b-946565c4_0] Write session will restart in 0.005008s
2025-10-09T21:49:26.645150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a64edbbe-410660bb-41df564b-946565c4_0] Write session: Do CDS request
2025-10-09T21:49:26.645157Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a64edbbe-410660bb-41df564b-946565c4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:26.651474Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a64edbbe-410660bb-41df564b-946565c4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:26.651575Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:26.651587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1518
2025-10-09T21:49:26.651758Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1518 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:26.651804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1518 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:26.651814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1518 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:26.651818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:26.652367Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1518 sessionId:  describe result for acl check
2025-10-09T21:49:26.652414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:26.652424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:26.652431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:26.652438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342441656821132:2447] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:26.653038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342441656821132:2447] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:26.653992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:26.655081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342441656821132:2447] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:26.655244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:26.655417Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:26.655426Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:26.655437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:26.655484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:26.655523Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342441294994884:5309]
2025-10-09T21:49:26.656121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:26.656422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:26.656471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:26.656785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:26.656840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.656846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.656847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.656849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.656850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.656851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.656853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.656854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.656855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.656860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.656870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.656875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.656879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.656883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.656888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.656892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.656895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.656899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.656906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.656915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.656921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.656926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.656931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.656937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.656943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.656948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.656953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.656958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.656962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.656969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.656978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.656983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.656990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.656993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.656995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.656998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.657026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:26.657593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342441656821132:2447] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:26.657605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342441656821132:2447] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:26.657608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342441656821132:2447] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:26.657614Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1518 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:26.663483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:26.664596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:26.664611Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:26.664613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:26.664618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:26.664625Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1517 [2:7559342441294994884:5309]
2025-10-09T21:49:26.664640Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:26.664658Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:26.664666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:26.664669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:26.664672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:26.664688Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:26.664699Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:26.664709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:26.664776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.664785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.664787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.664790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:26.664859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:26.664867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.664881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.664898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.664917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:26.664943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342441656821174:2447], now have 1 active actors on pipe
2025-10-09T21:49:26.664950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342441656821067:2432], now have 1 active actors on pipe
2025-10-09T21:49:26.664958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342441294994879:5309]
2025-10-09T21:49:26.664965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342441294994879:5309]
2025-10-09T21:49:26.664967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342441294994879:5309]
2025-10-09T21:49:26.664957Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1517
2025-10-09T21:49:26.664985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1517
2025-10-09T21:49:26.664984Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:26.665005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:26.665008Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:26.665012Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:26.665015Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:26.665019Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:26.665023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:26.665072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:26.665082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:26.665095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342441294994879:5309]
2025-10-09T21:49:26.665107Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:26.665119Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:26.665145Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3a8fb04c-526f7485-95f75664-efe64d1e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:26.665157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:26.665161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:26.665166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:26.665177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:26.665191Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:26.665222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:26.665406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:26.665416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:26.665431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:26.665531Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1518 partition: 0 MaxSeqNo: 25 sessionId: src_id|3a8fb04c-526f7485-95f75664-efe64d1e_0
2025-10-09T21:49:26.665966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a64edbbe-410660bb-41df564b-946565c4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3a8fb04c-526f7485-95f75664-efe64d1e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:26.668318Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a8fb04c-526f7485-95f75664-efe64d1e_0] Write session: try to update token
2025-10-09T21:49:26.668352Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a8fb04c-526f7485-95f75664-efe64d1e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:26.679352Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:26.679367Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:26.679370Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:26.679372Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:26.679382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342441294994879:5309]
2025-10-09T21:49:26.679386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342441294994879:5309]
2025-10-09T21:49:26.679391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342441294994879:5309]
2025-10-09T21:49:26.679434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:26.679441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.679442Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:26.679444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:26.679449Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:26.679453Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:26.679579Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:26.680831Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1518 sessionId: src_id|3a8fb04c-526f7485-95f75664-efe64d1e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:26.683518Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:26.745104Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342441656821185:2585]
2025-10-09T21:49:26.745397Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 1517, pipe: [1:7559342441656821185:2585]
2025-10-09T21:49:26.833311Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:26.833800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:26.833813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:26.833853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342441656821185:2585], now have 1 active actors on pipe
2025-10-09T21:49:26.834046Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:26.834158Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:26.834169Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:26.834206Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342441656821185:2585]
2025-10-09T21:49:26.834250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1518 reason: error at write: Disconnected sessionId: src_id|3a8fb04c-526f7485-95f75664-efe64d1e_0
2025-10-09T21:49:26.834264Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:26.834409Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1518 sessionId: src_id|3a8fb04c-526f7485-95f75664-efe64d1e_0 is DEAD
2025-10-09T21:49:26.834543Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a8fb04c-526f7485-95f75664-efe64d1e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:26.834551Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a8fb04c-526f7485-95f75664-efe64d1e_0] Write session will restart in 0.006927s
2025-10-09T21:49:26.834589Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a8fb04c-526f7485-95f75664-efe64d1e_0] Write session: Do CDS request
2025-10-09T21:49:26.834595Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a8fb04c-526f7485-95f75664-efe64d1e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:26.842481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a8fb04c-526f7485-95f75664-efe64d1e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:26.842585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:26.842600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1519
2025-10-09T21:49:26.842741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1519 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:26.842773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1519 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:26.842779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1519 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:26.842783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:26.843171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1519 sessionId:  describe result for acl check
2025-10-09T21:49:26.843203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:26.843210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:26.843212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:26.843218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342441656821257:2441] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:26.843365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:26.843769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342441656821257:2441] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:26.844830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:26.844992Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:26.845002Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:26.845014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:26.845068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:26.845110Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342441294994983:5317]
2025-10-09T21:49:26.845678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:26.845711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342441656821257:2441] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:26.846001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:26.846056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:26.846358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:26.846413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.846420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.846421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.846422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.846423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.846424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.846425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.846426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.846427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.846434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.846444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.846449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.846453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.846456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.846461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.846465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.846469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.846473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.846480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.846488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:26.846494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.846499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:26.846504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.846509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:26.846514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.846519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:26.846524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.846529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:26.846533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.846539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:26.846545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.846550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.846555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.846564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.846569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.846575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.846606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:26.851559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342441656821257:2441] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:26.851573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342441656821257:2441] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:26.851578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342441656821257:2441] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:26.851588Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1519 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:26.852931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:26.854287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:26.854299Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:26.854301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:26.854305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:26.854311Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1518 [2:7559342441294994983:5317]
2025-10-09T21:49:26.854324Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:26.854339Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:26.854346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:26.854352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:26.854359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:26.854375Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:26.854384Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:26.854391Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:26.854451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.854458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.854460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.854463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:26.854522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:26.854529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.854541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:26.854555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:26.854573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:26.854598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342441656821187:2432], now have 1 active actors on pipe
2025-10-09T21:49:26.854611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342441294994978:5317]
2025-10-09T21:49:26.854614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342441294994978:5317]
2025-10-09T21:49:26.854615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342441294994978:5317]
2025-10-09T21:49:26.854629Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:26.854642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342441656821298:2441], now have 1 active actors on pipe
2025-10-09T21:49:26.854643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:26.854646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:26.854648Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:26.854651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:26.854666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:26.854673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:26.854684Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1518
2025-10-09T21:49:26.854701Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342441294994978:5317]
2025-10-09T21:49:26.854709Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:26.854697Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1518
2025-10-09T21:49:26.854720Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:26.854735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:26.854739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:26.854770Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|be85b09-6385f589-97abf542-5d29f944_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:26.854781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:26.854787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:26.854794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:26.854803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:26.854812Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:26.854831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:26.854948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:26.854955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:26.854969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:26.855052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1519 partition: 0 MaxSeqNo: 25 sessionId: src_id|be85b09-6385f589-97abf542-5d29f944_0
2025-10-09T21:49:26.855374Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a8fb04c-526f7485-95f75664-efe64d1e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|be85b09-6385f589-97abf542-5d29f944_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:26.857668Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be85b09-6385f589-97abf542-5d29f944_0] Write session: try to update token
2025-10-09T21:49:26.857683Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be85b09-6385f589-97abf542-5d29f944_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:26.869036Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:26.869054Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:26.869057Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:26.869059Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:26.869069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342441294994978:5317]
2025-10-09T21:49:26.869073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342441294994978:5317]
2025-10-09T21:49:26.869076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342441294994978:5317]
2025-10-09T21:49:26.869125Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:26.869135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:26.869136Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:26.869139Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:26.869143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:26.869146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:26.869275Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:26.870338Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1519 sessionId: src_id|be85b09-6385f589-97abf542-5d29f944_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:26.872793Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:26.934527Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342441656821327:2585]
2025-10-09T21:49:26.934786Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 1518, pipe: [1:7559342441656821327:2585]
2025-10-09T21:49:27.017426Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:27.017883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.017898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.017941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342441656821327:2585], now have 1 active actors on pipe
2025-10-09T21:49:27.018135Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:27.018177Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:27.018177Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:27.018188Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:27.018266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1519 reason: error at write: Disconnected sessionId: src_id|be85b09-6385f589-97abf542-5d29f944_0
2025-10-09T21:49:27.018465Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1519 sessionId: src_id|be85b09-6385f589-97abf542-5d29f944_0 is DEAD
2025-10-09T21:49:27.018613Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be85b09-6385f589-97abf542-5d29f944_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:27.018622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be85b09-6385f589-97abf542-5d29f944_0] Write session will restart in 0.008189s
2025-10-09T21:49:27.018640Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be85b09-6385f589-97abf542-5d29f944_0] Write session: Do CDS request
2025-10-09T21:49:27.018645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be85b09-6385f589-97abf542-5d29f944_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:27.018676Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342441656821327:2585]
2025-10-09T21:49:27.027479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:27.027526Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be85b09-6385f589-97abf542-5d29f944_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:27.027624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:27.027636Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1520
2025-10-09T21:49:27.027740Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1520 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:27.027772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1520 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:27.027778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1520 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:27.027782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:27.028128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1520 sessionId:  describe result for acl check
2025-10-09T21:49:27.028154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:27.028158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:27.028159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:27.028171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342445951788695:2419] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:27.028619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342445951788695:2419] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:27.029013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:27.029186Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:27.029197Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:27.029210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:27.029264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:27.029317Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342445589962414:5294]
2025-10-09T21:49:27.029893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:27.030184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:27.030232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:27.030507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342445951788695:2419] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:27.030544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:27.030611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.030619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.030620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.030621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.030622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.030623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.030624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.030625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.030626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.030632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.030641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.030645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.030649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.030653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.030657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.030661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.030664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.030667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.030675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.030683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.030689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.030694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.030699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.030705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.030711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.030716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.030723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.030727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.030732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.030736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.030741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.030750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.030755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.030760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.030768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.030773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.030804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:27.036937Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342445951788695:2419] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:27.036952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342445951788695:2419] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:27.036956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:27.036956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342445951788695:2419] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:27.036964Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1520 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:27.038343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:27.038355Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:27.038358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:27.038362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:27.038368Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1519 [2:7559342445589962414:5294]
2025-10-09T21:49:27.038380Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:27.038395Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:27.038401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:27.038403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:27.038405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:27.038417Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:27.038420Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:27.038426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:27.038472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.038477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.038478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.038480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:27.038535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:27.038543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.038556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.038568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.038577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:27.038597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951788734:2419], now have 1 active actors on pipe
2025-10-09T21:49:27.038607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951788626:2432], now have 1 active actors on pipe
2025-10-09T21:49:27.038614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342445589962409:5294]
2025-10-09T21:49:27.038617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342445589962409:5294]
2025-10-09T21:49:27.038619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342445589962409:5294]
2025-10-09T21:49:27.038632Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:27.038646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:27.038652Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:27.038656Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:27.038659Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:27.038645Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1519
2025-10-09T21:49:27.038665Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:27.038673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:27.038656Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1519
2025-10-09T21:49:27.038698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342445589962409:5294]
2025-10-09T21:49:27.038704Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:27.038707Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:27.038717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.038726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.038759Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|38a6a446-ee9f5995-ccdb3701-949399f3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:27.038771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:27.038775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:27.038777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:27.038788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:27.038797Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:27.038817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:27.039010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.039023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.039045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:27.039190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1520 partition: 0 MaxSeqNo: 25 sessionId: src_id|38a6a446-ee9f5995-ccdb3701-949399f3_0
2025-10-09T21:49:27.039496Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be85b09-6385f589-97abf542-5d29f944_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|38a6a446-ee9f5995-ccdb3701-949399f3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:27.041873Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38a6a446-ee9f5995-ccdb3701-949399f3_0] Write session: try to update token
2025-10-09T21:49:27.041890Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38a6a446-ee9f5995-ccdb3701-949399f3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:27.053021Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:27.053039Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:27.053041Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:27.053044Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:27.053052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342445589962409:5294]
2025-10-09T21:49:27.053057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342445589962409:5294]
2025-10-09T21:49:27.053061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342445589962409:5294]
2025-10-09T21:49:27.053105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:27.053109Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:27.053116Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:27.053117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.053119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:27.053120Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:27.053252Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:27.054873Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1520 sessionId: src_id|38a6a446-ee9f5995-ccdb3701-949399f3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:27.057164Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:27.118841Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342445951788741:2585]
2025-10-09T21:49:27.119097Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 1519, pipe: [1:7559342445951788741:2585]
2025-10-09T21:49:27.203755Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:27.204138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.204153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.204199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951788741:2585], now have 1 active actors on pipe
2025-10-09T21:49:27.204365Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:27.204480Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:27.204504Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:27.204505Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342445951788741:2585]
2025-10-09T21:49:27.204521Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:27.204576Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1520 reason: error at write: Disconnected sessionId: src_id|38a6a446-ee9f5995-ccdb3701-949399f3_0
2025-10-09T21:49:27.204752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1520 sessionId: src_id|38a6a446-ee9f5995-ccdb3701-949399f3_0 is DEAD
2025-10-09T21:49:27.204882Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38a6a446-ee9f5995-ccdb3701-949399f3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:27.204893Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38a6a446-ee9f5995-ccdb3701-949399f3_0] Write session will restart in 0.007405s
2025-10-09T21:49:27.204920Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38a6a446-ee9f5995-ccdb3701-949399f3_0] Write session: Do CDS request
2025-10-09T21:49:27.204926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38a6a446-ee9f5995-ccdb3701-949399f3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:27.213482Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38a6a446-ee9f5995-ccdb3701-949399f3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:27.213568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:27.213582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1521
2025-10-09T21:49:27.213915Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1521 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:27.213956Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1521 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:27.213979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1521 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:27.213988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:27.213982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:27.214501Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1521 sessionId:  describe result for acl check
2025-10-09T21:49:27.214533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:27.214539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:27.214540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:27.214546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342445951788816:2413] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:27.215097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342445951788816:2413] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:27.215717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:27.215889Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:27.215898Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:27.215910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:27.215963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:27.216019Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342445589962511:5288]
2025-10-09T21:49:27.216586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:27.216887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:27.216910Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342445951788816:2413] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:27.216940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:27.217267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:27.217328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.217334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.217336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.217337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.217338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.217339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.217340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.217341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.217343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.217348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.217359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.217363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.217367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.217370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.217374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.217377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.217380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.217383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.217390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.217399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.217406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.217411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.217415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.217420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.217425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.217430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.217435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.217440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.217444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.217449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.217454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.217461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.217465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.217471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.217476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.217486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.217511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:27.223036Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342445951788816:2413] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:27.223054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342445951788816:2413] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:27.223058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342445951788816:2413] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:27.223073Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1521 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:27.224332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:27.225902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:27.225916Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:27.225919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:27.225923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:27.225930Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1520 [2:7559342445589962511:5288]
2025-10-09T21:49:27.225946Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:27.225963Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:27.225966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:27.225970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:27.225972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:27.225990Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:27.226003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:27.226010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:27.226158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.226166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.226168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.226171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:27.226237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:27.226245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.226256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.226274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.226297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:27.226320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342445589962506:5288]
2025-10-09T21:49:27.226327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342445589962506:5288]
2025-10-09T21:49:27.226329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342445589962506:5288]
2025-10-09T21:49:27.226338Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:27.226349Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:27.226355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:27.226358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:27.226360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:27.226370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:27.226373Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:27.226385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951788856:2413], now have 1 active actors on pipe
2025-10-09T21:49:27.226393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951788747:2432], now have 1 active actors on pipe
2025-10-09T21:49:27.226431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342445589962506:5288]
2025-10-09T21:49:27.226443Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:27.226449Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:27.226434Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1520
2025-10-09T21:49:27.226437Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1520
2025-10-09T21:49:27.226481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.226491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.226519Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1a1986f2-5eeba8a3-88951c02-f304e14d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:27.226531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:27.226541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:27.226544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:27.226555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:27.226570Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:27.226588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:27.226747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.226754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.226767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:27.226861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1521 partition: 0 MaxSeqNo: 25 sessionId: src_id|1a1986f2-5eeba8a3-88951c02-f304e14d_0
2025-10-09T21:49:27.227203Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38a6a446-ee9f5995-ccdb3701-949399f3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1a1986f2-5eeba8a3-88951c02-f304e14d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:27.229630Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a1986f2-5eeba8a3-88951c02-f304e14d_0] Write session: try to update token
2025-10-09T21:49:27.229645Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a1986f2-5eeba8a3-88951c02-f304e14d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:27.238381Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:27.238398Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:27.238408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342445589962506:5288]
2025-10-09T21:49:27.238448Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:27.238460Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:27.238614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:27.238750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:27.238884Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:27.239016Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:27.239197Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:27.239240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:27.243307Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1521 sessionId: src_id|1a1986f2-5eeba8a3-88951c02-f304e14d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:27.245825Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:27.246059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.246079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.246122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:27.246202Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:27.251711Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:27.251725Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:27.251728Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:27.251730Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:27.251740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342445589962506:5288]
2025-10-09T21:49:27.251745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342445589962506:5288]
2025-10-09T21:49:27.251752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342445589962506:5288]
2025-10-09T21:49:27.251800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:27.251803Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:27.251807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.251810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:27.251811Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:27.251815Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:27.251944Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:27.304701Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342445951788868:2585]
2025-10-09T21:49:27.304940Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 1520, pipe: [1:7559342445951788868:2585]
2025-10-09T21:49:27.408520Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:27.415443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.415471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.415643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:27.415705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:27.415763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:27.415825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:27.415877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:27.415887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:27.415948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:27.416009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:27.416067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:27.416128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:27.416167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:27.416173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:27.416227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:27.416286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:27.416342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:27.416402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:27.416447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:27.416456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:27.416524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:27.416577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:27.416633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:27.416693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:27.416730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:27.416736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:27.416793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:27.416855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:27.416934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:27.416998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:27.417057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:27.417064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:27.417131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:27.417189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:27.417269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:27.417336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:27.417394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:27.417402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:27.417463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:27.417519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:27.417578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:27.417647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:27.417692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:27.417701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:27.417763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:27.417821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:27.417880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:27.417940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:27.417983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:27.417992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:27.418067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:27.418156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:27.418213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:27.418276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:27.418320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:27.418324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:27.418382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:27.418440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:27.418495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:27.418553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:27.418604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:27.418611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:27.418654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951788868:2585], now have 1 active actors on pipe
2025-10-09T21:49:27.418894Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:27.418895Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342445951788868:2585]
2025-10-09T21:49:27.418906Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:27.418928Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:27.418938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1521 reason: error at write: Disconnected sessionId: src_id|1a1986f2-5eeba8a3-88951c02-f304e14d_0
2025-10-09T21:49:27.419119Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1521 sessionId: src_id|1a1986f2-5eeba8a3-88951c02-f304e14d_0 is DEAD
2025-10-09T21:49:27.419267Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a1986f2-5eeba8a3-88951c02-f304e14d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:27.419284Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a1986f2-5eeba8a3-88951c02-f304e14d_0] Write session will restart in 0.007433s
2025-10-09T21:49:27.419309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a1986f2-5eeba8a3-88951c02-f304e14d_0] Write session: Do CDS request
2025-10-09T21:49:27.419315Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a1986f2-5eeba8a3-88951c02-f304e14d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:27.419573Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:27.427519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a1986f2-5eeba8a3-88951c02-f304e14d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:27.427594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:27.427608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1522
2025-10-09T21:49:27.427716Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1522 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:27.427753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1522 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:27.427760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1522 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:27.427770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:27.428191Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1522 sessionId:  describe result for acl check
2025-10-09T21:49:27.428225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:27.428231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:27.428233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:27.428238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342445951788939:2402] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:27.428770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342445951788939:2402] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:27.429908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:27.431390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:27.431570Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:27.431581Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:27.431592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:27.431643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:27.431685Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342445589962610:5292]
2025-10-09T21:49:27.431769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342445951788939:2402] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:27.432267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:27.432591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:27.432636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:27.432906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:27.432960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.432966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.432967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.432968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.432969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.432969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.432970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.432970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.432971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.432975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.432982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.432984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.432986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.432988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.432990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.432992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.432995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.432999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.433005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.433012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.433018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.433023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.433029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.433034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.433038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.433044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.433048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.433053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.433058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.433062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.433068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.433073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.433078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.433083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.433088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.433093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.433133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:27.438214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342445951788939:2402] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:27.438229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342445951788939:2402] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:27.438232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342445951788939:2402] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:27.438244Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1522 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:27.439537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:27.440843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:27.440856Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:27.440858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:27.440862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:27.440868Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1521 [2:7559342445589962610:5292]
2025-10-09T21:49:27.440883Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:27.440899Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:27.440902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:27.440905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:27.440907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:27.440923Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:27.440929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:27.440936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:27.440993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.441001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.441003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.441006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:27.441070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:27.441078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.441089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.441102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.441125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:27.441149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951788981:2402], now have 1 active actors on pipe
2025-10-09T21:49:27.441156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951788871:2432], now have 1 active actors on pipe
2025-10-09T21:49:27.441164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342445589962604:5292]
2025-10-09T21:49:27.441166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342445589962604:5292]
2025-10-09T21:49:27.441168Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342445589962604:5292]
2025-10-09T21:49:27.441181Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:27.441195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:27.441198Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:27.441201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:27.441203Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:27.441213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:27.441216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:27.441206Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1521
2025-10-09T21:49:27.441208Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1521
2025-10-09T21:49:27.441246Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342445589962604:5292]
2025-10-09T21:49:27.441254Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:27.441256Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:27.441268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.441272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.441321Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1491395f-8c20c460-6ac04897-f9da8a39_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:27.441334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:27.441337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:27.441340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:27.441350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:27.441360Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:27.441382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:27.441527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.441534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.441549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:27.441645Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1522 partition: 0 MaxSeqNo: 25 sessionId: src_id|1491395f-8c20c460-6ac04897-f9da8a39_0
2025-10-09T21:49:27.441988Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a1986f2-5eeba8a3-88951c02-f304e14d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1491395f-8c20c460-6ac04897-f9da8a39_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:27.444419Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1491395f-8c20c460-6ac04897-f9da8a39_0] Write session: try to update token
2025-10-09T21:49:27.444441Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1491395f-8c20c460-6ac04897-f9da8a39_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:27.455542Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:27.455560Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:27.455563Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:27.455564Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:27.455574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342445589962604:5292]
2025-10-09T21:49:27.455579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342445589962604:5292]
2025-10-09T21:49:27.455583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342445589962604:5292]
2025-10-09T21:49:27.455628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:27.455635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.455640Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:27.455643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:27.455646Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:27.455650Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:27.455784Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:27.456981Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1522 sessionId: src_id|1491395f-8c20c460-6ac04897-f9da8a39_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:27.459413Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:27.519239Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342445951788989:2585]
2025-10-09T21:49:27.519493Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 1521, pipe: [1:7559342445951788989:2585]
2025-10-09T21:49:27.610496Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:27.611010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.611031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.611075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951788989:2585], now have 1 active actors on pipe
2025-10-09T21:49:27.611286Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:27.611338Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:27.611352Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:27.611415Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1522 reason: error at write: Disconnected sessionId: src_id|1491395f-8c20c460-6ac04897-f9da8a39_0
2025-10-09T21:49:27.611534Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342445951788989:2585]
2025-10-09T21:49:27.611557Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:27.611600Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1522 sessionId: src_id|1491395f-8c20c460-6ac04897-f9da8a39_0 is DEAD
2025-10-09T21:49:27.611774Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1491395f-8c20c460-6ac04897-f9da8a39_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:27.611786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1491395f-8c20c460-6ac04897-f9da8a39_0] Write session will restart in 0.006698s
2025-10-09T21:49:27.611813Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1491395f-8c20c460-6ac04897-f9da8a39_0] Write session: Do CDS request
2025-10-09T21:49:27.611819Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1491395f-8c20c460-6ac04897-f9da8a39_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:27.619493Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1491395f-8c20c460-6ac04897-f9da8a39_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:27.619583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:27.619597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1523
2025-10-09T21:49:27.619710Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1523 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:27.619752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1523 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:27.619762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1523 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:27.619767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:27.620161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1523 sessionId:  describe result for acl check
2025-10-09T21:49:27.620190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:27.620196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:27.620197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:27.620204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342445951789062:2398] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:27.620591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:27.620758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342445951789062:2398] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:27.622481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:27.622661Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:27.622671Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:27.622681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:27.622731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:27.622767Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342445589962708:5289]
2025-10-09T21:49:27.622988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342445951789062:2398] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:27.623187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:27.623455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:27.623510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:27.623834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:27.623892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.623898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.623899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.623900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.623901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.623902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.623903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.623904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.623905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.623910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.623920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.623924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.623928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.623932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.623936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.623939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.623943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.623947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.623953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.623960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.623967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.623972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.623976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.623981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.623985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.623990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.623997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.624002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.624006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.624013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.624020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.624025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.624029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.624033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.624038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.624047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.624075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:27.628693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342445951789062:2398] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:27.628705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342445951789062:2398] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:27.628710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342445951789062:2398] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:27.628720Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1523 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:27.629992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:27.630984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:27.630994Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:27.630996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:27.630998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:27.631003Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1522 [2:7559342445589962708:5289]
2025-10-09T21:49:27.631013Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:27.631025Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:27.631026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:27.631028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:27.631030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:27.631041Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:27.631046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:27.631050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:27.631090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.631101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.631102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.631104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:27.631161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:27.631168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.631179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.631189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.631204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:27.631228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951788991:2432], now have 1 active actors on pipe
2025-10-09T21:49:27.631235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951789102:2398], now have 1 active actors on pipe
2025-10-09T21:49:27.631240Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342445589962703:5289]
2025-10-09T21:49:27.631242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342445589962703:5289]
2025-10-09T21:49:27.631243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342445589962703:5289]
2025-10-09T21:49:27.631252Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:27.631263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:27.631271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:27.631272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:27.631274Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:27.631278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:27.631299Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1522
2025-10-09T21:49:27.631314Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1522
2025-10-09T21:49:27.631291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342445589962703:5289]
2025-10-09T21:49:27.631291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:27.631298Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:27.631300Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:27.631401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.631409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.631433Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ca7046e-22e44176-f464e281-7d95d953_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:27.631440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:27.631444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:27.631446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:27.631455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:27.631462Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:27.631477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:27.631647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.631668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.631690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:27.631780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1523 partition: 0 MaxSeqNo: 25 sessionId: src_id|ca7046e-22e44176-f464e281-7d95d953_0
2025-10-09T21:49:27.632044Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1491395f-8c20c460-6ac04897-f9da8a39_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ca7046e-22e44176-f464e281-7d95d953_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:27.634350Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca7046e-22e44176-f464e281-7d95d953_0] Write session: try to update token
2025-10-09T21:49:27.634365Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca7046e-22e44176-f464e281-7d95d953_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:27.645074Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:27.645088Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:27.645094Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:27.645096Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:27.645105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342445589962703:5289]
2025-10-09T21:49:27.645109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342445589962703:5289]
2025-10-09T21:49:27.645112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342445589962703:5289]
2025-10-09T21:49:27.645155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:27.645160Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:27.645164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.645165Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:27.645167Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:27.645167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:27.645293Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:27.646375Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1523 sessionId: src_id|ca7046e-22e44176-f464e281-7d95d953_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:27.648720Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:27.711762Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342445951789110:2585]
2025-10-09T21:49:27.712017Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 1522, pipe: [1:7559342445951789110:2585]
2025-10-09T21:49:27.791699Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:27.792219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.792232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.792273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951789110:2585], now have 1 active actors on pipe
2025-10-09T21:49:27.792485Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:27.792491Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:27.792492Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342445951789110:2585]
2025-10-09T21:49:27.792515Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:27.792518Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:27.792585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1523 reason: error at write: Disconnected sessionId: src_id|ca7046e-22e44176-f464e281-7d95d953_0
2025-10-09T21:49:27.792763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1523 sessionId: src_id|ca7046e-22e44176-f464e281-7d95d953_0 is DEAD
2025-10-09T21:49:27.792903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca7046e-22e44176-f464e281-7d95d953_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:27.792925Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca7046e-22e44176-f464e281-7d95d953_0] Write session will restart in 0.008302s
2025-10-09T21:49:27.792952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca7046e-22e44176-f464e281-7d95d953_0] Write session: Do CDS request
2025-10-09T21:49:27.792959Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca7046e-22e44176-f464e281-7d95d953_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:27.801089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:27.802545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca7046e-22e44176-f464e281-7d95d953_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:27.802643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:27.802659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1524
2025-10-09T21:49:27.802793Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1524 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:27.802832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1524 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:27.802843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1524 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:27.802847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:27.803218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:27.803289Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1524 sessionId:  describe result for acl check
2025-10-09T21:49:27.803325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:27.803332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:27.803334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:27.803340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342445951789185:2387] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:27.803385Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:27.803392Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:27.803400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:27.803440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:27.803475Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342445589962812:5270]
2025-10-09T21:49:27.803826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342445951789185:2387] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:27.803884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:27.804193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:27.804232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:27.804487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:27.804534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.804541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.804542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.804543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.804544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.804545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.804546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.804546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.804547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.804552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.804562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.804566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.804570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.804574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.804578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.804582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.804586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.804590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.804595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.804603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.804609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.804614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.804618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.804623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.804627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.804633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.804637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.804641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.804645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.804650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.804655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.804659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.804663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.804668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.804672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.804678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.804706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:27.805688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342445951789185:2387] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:27.810635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342445951789185:2387] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:27.810648Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342445951789185:2387] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:27.810652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342445951789185:2387] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:27.810666Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1524 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:27.810929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:27.812116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:27.812128Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:27.812129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:27.812133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:27.812137Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1523 [2:7559342445589962812:5270]
2025-10-09T21:49:27.812148Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:27.812160Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:27.812162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:27.812164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:27.812166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:27.812178Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:27.812186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:27.812192Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:27.812234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.812241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.812243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.812244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:27.812300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:27.812306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.812317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.812327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.812342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:27.812361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951789112:2432], now have 1 active actors on pipe
2025-10-09T21:49:27.812367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951789224:2387], now have 1 active actors on pipe
2025-10-09T21:49:27.812373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342445589962807:5270]
2025-10-09T21:49:27.812374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342445589962807:5270]
2025-10-09T21:49:27.812375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342445589962807:5270]
2025-10-09T21:49:27.812384Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:27.812395Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:27.812400Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:27.812403Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:27.812403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:27.812404Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:27.812406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:27.812424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342445589962807:5270]
2025-10-09T21:49:27.812426Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:27.812428Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:27.812457Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1523
2025-10-09T21:49:27.812471Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1523
2025-10-09T21:49:27.812539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.812555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.812591Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9cfb3842-c64cbd45-570ececf-de2fa4ff_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:27.812607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:27.812631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:27.812638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:27.812648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:27.812665Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:27.812683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:27.812853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.812860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.812873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:27.812969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1524 partition: 0 MaxSeqNo: 25 sessionId: src_id|9cfb3842-c64cbd45-570ececf-de2fa4ff_0
2025-10-09T21:49:27.813276Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca7046e-22e44176-f464e281-7d95d953_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9cfb3842-c64cbd45-570ececf-de2fa4ff_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:27.815664Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9cfb3842-c64cbd45-570ececf-de2fa4ff_0] Write session: try to update token
2025-10-09T21:49:27.815681Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9cfb3842-c64cbd45-570ececf-de2fa4ff_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:27.826509Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:27.826527Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:27.826528Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:27.826529Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:27.826537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342445589962807:5270]
2025-10-09T21:49:27.826541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342445589962807:5270]
2025-10-09T21:49:27.826543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342445589962807:5270]
2025-10-09T21:49:27.826576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:27.826582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.826583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:27.826586Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:27.826591Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:27.826594Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:27.826712Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:27.828251Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1524 sessionId: src_id|9cfb3842-c64cbd45-570ececf-de2fa4ff_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:27.830552Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:27.892478Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342445951789234:2585]
2025-10-09T21:49:27.892730Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 1523, pipe: [1:7559342445951789234:2585]
2025-10-09T21:49:27.975829Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:27.976084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.976098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.976137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951789234:2585], now have 1 active actors on pipe
2025-10-09T21:49:27.976310Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:27.976374Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342445951789234:2585]
2025-10-09T21:49:27.976383Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:27.976418Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:27.976503Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1524 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9cfb3842-c64cbd45-570ececf-de2fa4ff_0
2025-10-09T21:49:27.976679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1524 sessionId: src_id|9cfb3842-c64cbd45-570ececf-de2fa4ff_0 is DEAD
2025-10-09T21:49:27.976821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9cfb3842-c64cbd45-570ececf-de2fa4ff_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:27.976831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9cfb3842-c64cbd45-570ececf-de2fa4ff_0] Write session will restart in 0.009749s
2025-10-09T21:49:27.976857Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9cfb3842-c64cbd45-570ececf-de2fa4ff_0] Write session: Do CDS request
2025-10-09T21:49:27.976872Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9cfb3842-c64cbd45-570ececf-de2fa4ff_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:27.985436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:27.987116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:27.987313Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:27.987325Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:27.987337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:27.987391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:27.987431Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342445589962951:5245]
2025-10-09T21:49:27.987509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9cfb3842-c64cbd45-570ececf-de2fa4ff_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:27.987586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:27.987600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1525
2025-10-09T21:49:27.987739Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1525 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:27.987769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1525 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:27.987779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1525 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:27.987783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:27.987977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:27.988267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:27.988265Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1525 sessionId:  describe result for acl check
2025-10-09T21:49:27.988319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:27.988315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:27.988324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:27.988325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:27.988331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342445951789337:2375] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:27.988615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:27.988662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.988669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.988670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.988671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.988672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.988673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.988674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.988675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.988676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.988681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.988691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.988695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.988699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.988702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.988706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.988710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.988713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.988717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.988723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.988733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:27.988739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.988744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:27.988747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.988752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:27.988757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.988762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:27.988767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.988771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:27.988775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.988780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:27.988786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.988798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.988802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.988807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.988811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.988816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.988839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342445951789337:2375] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:27.988843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:27.991143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342445951789337:2375] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:27.994280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342445951789337:2375] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:27.994293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342445951789337:2375] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:27.994299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342445951789337:2375] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:27.994308Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1525 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:27.994661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:27.996076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:27.996090Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:27.996094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:27.996099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:27.996104Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1524 [2:7559342445589962951:5245]
2025-10-09T21:49:27.996118Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:27.996133Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:27.996136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:27.996139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:27.996142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:27.996158Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:27.996164Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:27.996177Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:27.996226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.996232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.996233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.996234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:27.996299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:27.996304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:27.996311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:27.996317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:27.996328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:27.996352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951789370:2375], now have 1 active actors on pipe
2025-10-09T21:49:27.996355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342445951789260:2432], now have 1 active actors on pipe
2025-10-09T21:49:27.996362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342445589962946:5245]
2025-10-09T21:49:27.996365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342445589962946:5245]
2025-10-09T21:49:27.996367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342445589962946:5245]
2025-10-09T21:49:27.996377Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:27.996387Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:27.996390Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:27.996392Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:27.996392Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:27.996406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:27.996409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:27.996404Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1524
2025-10-09T21:49:27.996404Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1524
2025-10-09T21:49:27.996435Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342445589962946:5245]
2025-10-09T21:49:27.996445Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:27.996447Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:27.996474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.996483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.996518Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bbd5cf53-a9b5a797-da77196f-1e3a6cf9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:27.996529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:27.996533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:27.996537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:27.996547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:27.996562Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:27.996582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:27.996727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:27.996735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:27.996749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:27.996833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1525 partition: 0 MaxSeqNo: 25 sessionId: src_id|bbd5cf53-a9b5a797-da77196f-1e3a6cf9_0
2025-10-09T21:49:27.997089Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9cfb3842-c64cbd45-570ececf-de2fa4ff_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bbd5cf53-a9b5a797-da77196f-1e3a6cf9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:27.999435Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bbd5cf53-a9b5a797-da77196f-1e3a6cf9_0] Write session: try to update token
2025-10-09T21:49:27.999455Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bbd5cf53-a9b5a797-da77196f-1e3a6cf9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:28.010370Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:28.010386Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:28.010389Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:28.010392Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:28.010401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342445589962946:5245]
2025-10-09T21:49:28.010405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342445589962946:5245]
2025-10-09T21:49:28.010409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342445589962946:5245]
2025-10-09T21:49:28.010450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:28.010450Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:28.010456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.010459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:28.010463Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:28.010466Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:28.010595Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:28.012505Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1525 sessionId: src_id|bbd5cf53-a9b5a797-da77196f-1e3a6cf9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:28.015002Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:28.076767Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342450246756674:2585]
2025-10-09T21:49:28.077052Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 1524, pipe: [1:7559342450246756674:2585]
2025-10-09T21:49:28.162394Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:28.162627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:28.162640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:28.162677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342450246756674:2585], now have 1 active actors on pipe
2025-10-09T21:49:28.162907Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:28.162938Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342450246756674:2585]
2025-10-09T21:49:28.162950Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:28.162975Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:28.163062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1525 reason: status is not ok: tablet will be restarted right now sessionId: src_id|bbd5cf53-a9b5a797-da77196f-1e3a6cf9_0
2025-10-09T21:49:28.163233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1525 sessionId: src_id|bbd5cf53-a9b5a797-da77196f-1e3a6cf9_0 is DEAD
2025-10-09T21:49:28.163390Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbd5cf53-a9b5a797-da77196f-1e3a6cf9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:28.163402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbd5cf53-a9b5a797-da77196f-1e3a6cf9_0] Write session will restart in 0.005220s
2025-10-09T21:49:28.163425Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbd5cf53-a9b5a797-da77196f-1e3a6cf9_0] Write session: Do CDS request
2025-10-09T21:49:28.163432Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbd5cf53-a9b5a797-da77196f-1e3a6cf9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:28.169486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bbd5cf53-a9b5a797-da77196f-1e3a6cf9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:28.169501Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:28.169514Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1526
2025-10-09T21:49:28.169691Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1526 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:28.169731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1526 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:28.169742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1526 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:28.169749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:28.170156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1526 sessionId:  describe result for acl check
2025-10-09T21:49:28.170187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:28.170188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:28.170189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:28.170194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342450246756731:2373] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:28.170721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342450246756731:2373] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:28.172895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342450246756731:2373] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:28.175315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:28.175527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342450246756731:2373] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:28.175538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342450246756731:2373] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:28.175541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342450246756731:2373] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:28.175549Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1526 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:28.175863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:28.176064Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:28.176080Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:28.176094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:28.176163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:28.176218Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342449884930351:5244]
2025-10-09T21:49:28.176901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:28.177192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:28.177250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:28.177601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:28.177684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.177693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.177694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.177696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.177697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.177698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.177700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.177701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.177702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.177708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.177717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.177722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.177726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.177731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.177735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.177740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.177744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.177748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.177755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.177764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.177771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.177777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.177782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.177788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.177793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.177799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.177805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.177810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.177816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.177822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.177828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.177833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.177838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.177843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.177850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.177855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.177894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:28.184398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:28.185661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:28.185681Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:28.185685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:28.185689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:28.185698Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1525 [2:7559342449884930351:5244]
2025-10-09T21:49:28.185715Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:28.185736Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:28.185744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:28.185748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:28.185751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:28.185767Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:28.185781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:28.185788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:28.185860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.185870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.185873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.185876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:28.185956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:28.185967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.185980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.185993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.186016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:28.186059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342450246756676:2432], now have 1 active actors on pipe
2025-10-09T21:49:28.186073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342450246756790:2373], now have 1 active actors on pipe
2025-10-09T21:49:28.186081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342449884930345:5244]
2025-10-09T21:49:28.186083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342449884930345:5244]
2025-10-09T21:49:28.186086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342449884930345:5244]
2025-10-09T21:49:28.186102Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:28.186093Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1525
2025-10-09T21:49:28.186111Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1525
2025-10-09T21:49:28.186122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:28.186125Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:28.186130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:28.186132Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:28.186148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:28.186152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:28.186206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:28.186220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:28.186236Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342449884930345:5244]
2025-10-09T21:49:28.186247Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:28.186261Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:28.186281Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8acef1ef-9893286e-e31f34e6-4727eaf7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:28.186296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:28.186300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:28.186303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:28.186314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:28.186327Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:28.186352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:28.186608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:28.186621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:28.186638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:28.186762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1526 partition: 0 MaxSeqNo: 25 sessionId: src_id|8acef1ef-9893286e-e31f34e6-4727eaf7_0
2025-10-09T21:49:28.187128Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbd5cf53-a9b5a797-da77196f-1e3a6cf9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8acef1ef-9893286e-e31f34e6-4727eaf7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:28.189532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8acef1ef-9893286e-e31f34e6-4727eaf7_0] Write session: try to update token
2025-10-09T21:49:28.189551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8acef1ef-9893286e-e31f34e6-4727eaf7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:28.200582Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:28.200600Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:28.200602Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:28.200604Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:28.200616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342449884930345:5244]
2025-10-09T21:49:28.200621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342449884930345:5244]
2025-10-09T21:49:28.200626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342449884930345:5244]
2025-10-09T21:49:28.200662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:28.200666Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:28.200668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.200670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:28.200671Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:28.200673Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:28.200801Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:28.202622Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1526 sessionId: src_id|8acef1ef-9893286e-e31f34e6-4727eaf7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:28.204899Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:28.263169Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342450246756806:2585]
2025-10-09T21:49:28.263446Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 1525, pipe: [1:7559342450246756806:2585]
2025-10-09T21:49:28.350471Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:28.351097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:28.351115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:28.351157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342450246756806:2585], now have 1 active actors on pipe
2025-10-09T21:49:28.351336Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:28.351364Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:28.351369Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:28.351394Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342450246756806:2585]
2025-10-09T21:49:28.351432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1526 reason: error at write: Disconnected sessionId: src_id|8acef1ef-9893286e-e31f34e6-4727eaf7_0
2025-10-09T21:49:28.351624Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1526 sessionId: src_id|8acef1ef-9893286e-e31f34e6-4727eaf7_0 is DEAD
2025-10-09T21:49:28.351806Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8acef1ef-9893286e-e31f34e6-4727eaf7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:28.351818Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8acef1ef-9893286e-e31f34e6-4727eaf7_0] Write session will restart in 0.007811s
2025-10-09T21:49:28.351842Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8acef1ef-9893286e-e31f34e6-4727eaf7_0] Write session: Do CDS request
2025-10-09T21:49:28.351851Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8acef1ef-9893286e-e31f34e6-4727eaf7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:28.351976Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:28.360269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:28.360522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8acef1ef-9893286e-e31f34e6-4727eaf7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:28.360623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:28.360639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1527
2025-10-09T21:49:28.360796Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1527 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:28.360831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1527 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:28.360838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1527 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:28.360841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:28.361206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1527 sessionId:  describe result for acl check
2025-10-09T21:49:28.361234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:28.361241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:28.361249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:28.361253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342450246756885:2360] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:28.361776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342450246756885:2360] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:28.362496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:28.362681Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:28.362691Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:28.362704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:28.362760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:28.362788Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342449884930454:5239]
2025-10-09T21:49:28.363375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:28.363622Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342450246756885:2360] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:28.363662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:28.363710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:28.364015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:28.364080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.364087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.364088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.364089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.364090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.364091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.364092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.364093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.364094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.364099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.364108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.364112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.364115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.364119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.364122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.364125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.364129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.364132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.364139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.364146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.364151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.364157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.364160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.364165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.364168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.364173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.364178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.364182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.364186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.364190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.364195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.364199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.364209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.364213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.364218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.364223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.364254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:28.369058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342450246756885:2360] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:28.369070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342450246756885:2360] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:28.369074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342450246756885:2360] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:28.369082Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1527 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:28.370418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:28.371700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:28.371713Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:28.371716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:28.371719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:28.371725Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1526 [2:7559342449884930454:5239]
2025-10-09T21:49:28.371737Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:28.371751Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:28.371755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:28.371760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:28.371762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:28.371778Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:28.371788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:28.371795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:28.371851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.371857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.371859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.371862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:28.371926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:28.371934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.371944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.371961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.371985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:28.372005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342450246756809:2432], now have 1 active actors on pipe
2025-10-09T21:49:28.372008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342450246756923:2360], now have 1 active actors on pipe
2025-10-09T21:49:28.372015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342449884930448:5239]
2025-10-09T21:49:28.372017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342449884930448:5239]
2025-10-09T21:49:28.372019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342449884930448:5239]
2025-10-09T21:49:28.372030Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:28.372045Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:28.372054Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:28.372056Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:28.372057Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:28.372063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:28.372067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:28.372041Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1526
2025-10-09T21:49:28.372053Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1526
2025-10-09T21:49:28.372093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342449884930448:5239]
2025-10-09T21:49:28.372107Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:28.372119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:28.372120Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:28.372123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:28.372161Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|33c7d795-a0a19d3a-5ae34b02-9b98a17c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:28.372173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:28.372177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:28.372184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:28.372194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:28.372207Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:28.372226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:28.372371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:28.372380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:28.372394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:28.372497Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1527 partition: 0 MaxSeqNo: 25 sessionId: src_id|33c7d795-a0a19d3a-5ae34b02-9b98a17c_0
2025-10-09T21:49:28.372781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8acef1ef-9893286e-e31f34e6-4727eaf7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|33c7d795-a0a19d3a-5ae34b02-9b98a17c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:28.375251Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33c7d795-a0a19d3a-5ae34b02-9b98a17c_0] Write session: try to update token
2025-10-09T21:49:28.375268Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33c7d795-a0a19d3a-5ae34b02-9b98a17c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:28.386230Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:28.386245Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:28.386247Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:28.386248Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:28.386257Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342449884930448:5239]
2025-10-09T21:49:28.386260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342449884930448:5239]
2025-10-09T21:49:28.386263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342449884930448:5239]
2025-10-09T21:49:28.386298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:28.386304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.386304Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:28.386306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:28.386309Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:28.386312Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:28.386438Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:28.388144Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1527 sessionId: src_id|33c7d795-a0a19d3a-5ae34b02-9b98a17c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:28.390401Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:28.451550Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342450246756931:2585]
2025-10-09T21:49:28.451834Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 1526, pipe: [1:7559342450246756931:2585]
2025-10-09T21:49:28.532716Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:28.532972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:28.532991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:28.533045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342450246756931:2585], now have 1 active actors on pipe
2025-10-09T21:49:28.533244Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:28.533251Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:28.533256Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:28.533310Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342450246756931:2585]
2025-10-09T21:49:28.533321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1527 reason: error at write: Disconnected sessionId: src_id|33c7d795-a0a19d3a-5ae34b02-9b98a17c_0
2025-10-09T21:49:28.533302Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:28.533525Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1527 sessionId: src_id|33c7d795-a0a19d3a-5ae34b02-9b98a17c_0 is DEAD
2025-10-09T21:49:28.533681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33c7d795-a0a19d3a-5ae34b02-9b98a17c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:28.533691Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33c7d795-a0a19d3a-5ae34b02-9b98a17c_0] Write session will restart in 0.005550s
2025-10-09T21:49:28.533715Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33c7d795-a0a19d3a-5ae34b02-9b98a17c_0] Write session: Do CDS request
2025-10-09T21:49:28.533722Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33c7d795-a0a19d3a-5ae34b02-9b98a17c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:28.540487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33c7d795-a0a19d3a-5ae34b02-9b98a17c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:28.540565Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:28.540577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1528
2025-10-09T21:49:28.540718Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1528 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:28.540747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1528 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:28.540752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1528 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:28.540760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:28.541258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1528 sessionId:  describe result for acl check
2025-10-09T21:49:28.541297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:28.541305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:28.541307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:28.541316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342450246756983:2357] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:28.541894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342450246756983:2357] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:28.543695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:28.544130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342450246756983:2357] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:28.545927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:28.546138Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:28.546148Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:28.546160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:28.546216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:28.546256Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342449884930558:5234]
2025-10-09T21:49:28.546366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342450246756983:2357] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:28.546380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342450246756983:2357] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:28.546384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342450246756983:2357] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:28.546394Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1528 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:28.546844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:28.547116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:28.547156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:28.547387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:28.547422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.547427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.547428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.547429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.547429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.547430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.547430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.547431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.547431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.547435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.547442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.547444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.547446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.547448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.547451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.547453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.547455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.547456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.547462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.547467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.547471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.547474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.547477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.547480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.547483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.547486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.547489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.547492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.547494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.547497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.547501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.547504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.547507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.547510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.547513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.547516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.547532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:28.554042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:28.555234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:28.555250Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:28.555253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:28.555257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:28.555263Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1527 [2:7559342449884930558:5234]
2025-10-09T21:49:28.555277Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:28.555294Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:28.555299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:28.555302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:28.555304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:28.555320Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:28.555332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:28.555340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:28.555396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.555404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.555406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.555409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:28.555475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:28.555482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.555496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.555513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.555535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:28.555560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342450246757053:2357], now have 1 active actors on pipe
2025-10-09T21:49:28.555568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342450246756934:2432], now have 1 active actors on pipe
2025-10-09T21:49:28.555576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342449884930552:5234]
2025-10-09T21:49:28.555583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342449884930552:5234]
2025-10-09T21:49:28.555585Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342449884930552:5234]
2025-10-09T21:49:28.555598Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:28.555616Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:28.555618Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:28.555621Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:28.555604Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1527
2025-10-09T21:49:28.555622Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:28.555634Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:28.555637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:28.555627Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1527
2025-10-09T21:49:28.555670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:28.555679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:28.555690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342449884930552:5234]
2025-10-09T21:49:28.555702Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:28.555729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:28.555740Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|97e0d42c-5641a86a-6d43f942-f06dcf39_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:28.555748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:28.555756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:28.555759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:28.555768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:28.555781Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:28.555804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:28.555958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:28.555966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:28.555980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:28.556041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1528 partition: 0 MaxSeqNo: 25 sessionId: src_id|97e0d42c-5641a86a-6d43f942-f06dcf39_0
2025-10-09T21:49:28.556363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33c7d795-a0a19d3a-5ae34b02-9b98a17c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|97e0d42c-5641a86a-6d43f942-f06dcf39_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:28.558775Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97e0d42c-5641a86a-6d43f942-f06dcf39_0] Write session: try to update token
2025-10-09T21:49:28.558789Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97e0d42c-5641a86a-6d43f942-f06dcf39_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:28.570402Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:28.570416Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:28.570418Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:28.570419Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:28.570435Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342449884930552:5234]
2025-10-09T21:49:28.570439Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342449884930552:5234]
2025-10-09T21:49:28.570443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342449884930552:5234]
2025-10-09T21:49:28.570484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:28.570487Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:28.570490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.570492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:28.570493Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:28.570498Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:28.570616Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:28.572116Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1528 sessionId: src_id|97e0d42c-5641a86a-6d43f942-f06dcf39_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:28.574737Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:28.633369Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342450246757064:2585]
2025-10-09T21:49:28.633708Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 1527, pipe: [1:7559342450246757064:2585]
2025-10-09T21:49:28.720322Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:28.720576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:28.720592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:28.720637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342450246757064:2585], now have 1 active actors on pipe
2025-10-09T21:49:28.720870Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:28.720868Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:28.720909Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342450246757064:2585]
2025-10-09T21:49:28.720914Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:28.720916Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:28.720972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1528 reason: error at write: Disconnected sessionId: src_id|97e0d42c-5641a86a-6d43f942-f06dcf39_0
2025-10-09T21:49:28.721154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1528 sessionId: src_id|97e0d42c-5641a86a-6d43f942-f06dcf39_0 is DEAD
2025-10-09T21:49:28.721264Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97e0d42c-5641a86a-6d43f942-f06dcf39_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:28.721274Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97e0d42c-5641a86a-6d43f942-f06dcf39_0] Write session will restart in 0.009608s
2025-10-09T21:49:28.721294Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97e0d42c-5641a86a-6d43f942-f06dcf39_0] Write session: Do CDS request
2025-10-09T21:49:28.721299Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97e0d42c-5641a86a-6d43f942-f06dcf39_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:28.731132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:28.732519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97e0d42c-5641a86a-6d43f942-f06dcf39_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:28.732584Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:28.732595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1529
2025-10-09T21:49:28.732732Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1529 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:28.732776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1529 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:28.732786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1529 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:28.732790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:28.732887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:28.733059Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:28.733069Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:28.733080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:28.733136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:28.733175Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342449884930667:5223]
2025-10-09T21:49:28.733215Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1529 sessionId:  describe result for acl check
2025-10-09T21:49:28.733249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:28.733256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:28.733258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:28.733264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342450246757146:2332] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:28.733752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:28.733826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342450246757146:2332] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:28.734065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:28.734110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:28.734417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:28.734467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.734472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.734473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.734474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.734475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.734476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.734477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.734478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.734479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.734484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.734492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.734496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.734499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.734503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.734507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.734511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.734515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.734519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.734525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.734532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.734538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.734543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.734547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.734551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.734556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.734561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.734565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.734570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.734574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.734580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.734585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.734590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.734594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.734599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.734603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.734608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.734642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:28.736167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342450246757146:2332] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:28.740387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342450246757146:2332] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:28.740402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342450246757146:2332] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:28.740405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342450246757146:2332] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:28.740414Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1529 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:28.740763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:28.741877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:28.741890Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:28.741892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:28.741896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:28.741903Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1528 [2:7559342449884930667:5223]
2025-10-09T21:49:28.741916Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:28.741932Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:28.741939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:28.741941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:28.741942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:28.741955Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:28.741958Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:28.741963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:28.742002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.742008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.742009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.742011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:28.742077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:28.742082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.742089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.742095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.742108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:28.742127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342450246757184:2332], now have 1 active actors on pipe
2025-10-09T21:49:28.742133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342450246757067:2432], now have 1 active actors on pipe
2025-10-09T21:49:28.742139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342449884930661:5223]
2025-10-09T21:49:28.742140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342449884930661:5223]
2025-10-09T21:49:28.742141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342449884930661:5223]
2025-10-09T21:49:28.742150Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:28.742163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:28.742174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:28.742174Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:28.742178Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:28.742180Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:28.742180Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:28.742199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342449884930661:5223]
2025-10-09T21:49:28.742201Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:28.742204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:28.742185Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1528
2025-10-09T21:49:28.742221Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1528
2025-10-09T21:49:28.742254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:28.742260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:28.742279Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e2d10650-25b523c9-7cad2b99-d3844e71_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:28.742287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:28.742290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:28.742295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:28.742301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:28.742312Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:28.742325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:28.742458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:28.742463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:28.742474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:28.742549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1529 partition: 0 MaxSeqNo: 25 sessionId: src_id|e2d10650-25b523c9-7cad2b99-d3844e71_0
2025-10-09T21:49:28.742794Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97e0d42c-5641a86a-6d43f942-f06dcf39_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e2d10650-25b523c9-7cad2b99-d3844e71_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:28.744984Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2d10650-25b523c9-7cad2b99-d3844e71_0] Write session: try to update token
2025-10-09T21:49:28.744996Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2d10650-25b523c9-7cad2b99-d3844e71_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:28.756845Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:28.756862Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:28.756866Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:28.756868Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:28.756878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342449884930661:5223]
2025-10-09T21:49:28.756882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342449884930661:5223]
2025-10-09T21:49:28.756887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342449884930661:5223]
2025-10-09T21:49:28.756931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:28.756939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.756942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:28.756954Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:28.756960Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:28.756965Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:28.757087Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:28.757425Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1529 sessionId: src_id|e2d10650-25b523c9-7cad2b99-d3844e71_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:28.759828Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:28.821241Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342450246757191:2585]
2025-10-09T21:49:28.821452Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 1528, pipe: [1:7559342450246757191:2585]
2025-10-09T21:49:28.909148Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:28.909401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:28.909413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:28.909451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342450246757191:2585], now have 1 active actors on pipe
2025-10-09T21:49:28.909631Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:28.909661Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:28.909684Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342450246757191:2585]
2025-10-09T21:49:28.909706Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:28.909790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1529 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e2d10650-25b523c9-7cad2b99-d3844e71_0
2025-10-09T21:49:28.909968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1529 sessionId: src_id|e2d10650-25b523c9-7cad2b99-d3844e71_0 is DEAD
2025-10-09T21:49:28.910108Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2d10650-25b523c9-7cad2b99-d3844e71_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:28.910120Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2d10650-25b523c9-7cad2b99-d3844e71_0] Write session will restart in 0.008689s
2025-10-09T21:49:28.910149Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2d10650-25b523c9-7cad2b99-d3844e71_0] Write session: Do CDS request
2025-10-09T21:49:28.910155Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2d10650-25b523c9-7cad2b99-d3844e71_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:28.919474Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2d10650-25b523c9-7cad2b99-d3844e71_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:28.919479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:28.919582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:28.919595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1530
2025-10-09T21:49:28.919716Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1530 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:28.919746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1530 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:28.919756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1530 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:28.919760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:28.920154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1530 sessionId:  describe result for acl check
2025-10-09T21:49:28.920185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:28.920192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:28.920194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:28.920200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342450246757275:2338] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:28.920757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342450246757275:2338] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:28.921836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:28.922012Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:28.922023Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:28.922062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:28.922142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:28.922197Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342449884930773:5220]
2025-10-09T21:49:28.922734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:28.923087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:28.923120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342450246757275:2338] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:28.923155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:28.923497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:28.923539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.923543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.923544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.923545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.923545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.923547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.923547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.923548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.923548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.923552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.923558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.923560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.923563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.923565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.923567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.923569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.923571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.923573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.923578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.923583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:28.923587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.923590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:28.923592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.923596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:28.923598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.923601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:28.923604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.923608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:28.923610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.923614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:28.923616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.923619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.923622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.923626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.923629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.923632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.923649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:28.928577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342450246757275:2338] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:28.928587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342450246757275:2338] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:28.928590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342450246757275:2338] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:28.928604Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1530 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:28.929898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:28.931019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:28.931036Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:28.931038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:28.931042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:28.931048Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1529 [2:7559342449884930773:5220]
2025-10-09T21:49:28.931062Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:28.931080Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:28.931087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:28.931091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:28.931093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:28.931109Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:28.931117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:28.931123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:28.931167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.931173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.931175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.931177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:28.931240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:28.931248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.931259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:28.931277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:28.931299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:28.931324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342450246757314:2338], now have 1 active actors on pipe
2025-10-09T21:49:28.931332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342450246757196:2432], now have 1 active actors on pipe
2025-10-09T21:49:28.931340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342449884930767:5220]
2025-10-09T21:49:28.931346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342449884930767:5220]
2025-10-09T21:49:28.931348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342449884930767:5220]
2025-10-09T21:49:28.931361Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:28.931380Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:28.931382Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:28.931367Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1529
2025-10-09T21:49:28.931386Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:28.931370Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1529
2025-10-09T21:49:28.931388Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:28.931399Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:28.931402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:28.931454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:28.931463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:28.931476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342449884930767:5220]
2025-10-09T21:49:28.931487Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:28.931495Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:28.931522Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|83acb0c1-e998a091-19c60ffa-dfeae553_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:28.931533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:28.931541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:28.931544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:28.931555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:28.931567Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:28.931589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:28.931751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:28.931762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:28.931778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:28.931895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1530 partition: 0 MaxSeqNo: 25 sessionId: src_id|83acb0c1-e998a091-19c60ffa-dfeae553_0
2025-10-09T21:49:28.932219Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2d10650-25b523c9-7cad2b99-d3844e71_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|83acb0c1-e998a091-19c60ffa-dfeae553_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:28.934756Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|83acb0c1-e998a091-19c60ffa-dfeae553_0] Write session: try to update token
2025-10-09T21:49:28.934773Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|83acb0c1-e998a091-19c60ffa-dfeae553_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:28.945846Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:28.945867Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:28.945869Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:28.945871Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:28.945881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342449884930767:5220]
2025-10-09T21:49:28.945885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342449884930767:5220]
2025-10-09T21:49:28.945888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342449884930767:5220]
2025-10-09T21:49:28.945932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:28.945938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:28.945942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:28.945946Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:28.945953Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:28.945957Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:28.946103Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:28.947966Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1530 sessionId: src_id|83acb0c1-e998a091-19c60ffa-dfeae553_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:28.950268Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:29.009626Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342454541724640:2585]
2025-10-09T21:49:29.009855Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 1529, pipe: [1:7559342454541724640:2585]
2025-10-09T21:49:29.103687Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:29.104208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:29.104226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:29.104272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342454541724640:2585], now have 1 active actors on pipe
2025-10-09T21:49:29.104488Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342454541724640:2585]
2025-10-09T21:49:29.104512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:29.104516Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:29.104512Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:29.104520Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:29.104580Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1530 reason: error at write: Disconnected sessionId: src_id|83acb0c1-e998a091-19c60ffa-dfeae553_0
2025-10-09T21:49:29.104751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1530 sessionId: src_id|83acb0c1-e998a091-19c60ffa-dfeae553_0 is DEAD
2025-10-09T21:49:29.104898Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83acb0c1-e998a091-19c60ffa-dfeae553_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:29.104907Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83acb0c1-e998a091-19c60ffa-dfeae553_0] Write session will restart in 0.007680s
2025-10-09T21:49:29.104929Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83acb0c1-e998a091-19c60ffa-dfeae553_0] Write session: Do CDS request
2025-10-09T21:49:29.104953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83acb0c1-e998a091-19c60ffa-dfeae553_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:29.113492Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|83acb0c1-e998a091-19c60ffa-dfeae553_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:29.113576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:29.113596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1531
2025-10-09T21:49:29.113791Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1531 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:29.113832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1531 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:29.113844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1531 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:29.113848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:29.114239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:29.114251Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1531 sessionId:  describe result for acl check
2025-10-09T21:49:29.114283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:29.114285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:29.114286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:29.114290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342454541724721:2333] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:29.114745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342454541724721:2333] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:29.116160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:29.116333Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:29.116353Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:29.116367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:29.116417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:29.116450Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342454179898214:5194]
2025-10-09T21:49:29.116895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342454541724721:2333] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:29.116903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:29.117229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:29.117281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:29.117636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:29.117691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.117697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.117698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.117700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.117701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.117702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.117703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.117704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.117705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.117711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.117721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.117725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.117729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.117733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.117737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.117741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.117745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.117757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.117764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.117773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.117779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.117785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.117789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.117795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.117799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.117804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.117809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.117814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.117819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.117824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.117829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.117834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.117839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.117847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.117852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.117858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.117884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:29.122999Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342454541724721:2333] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:29.123013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342454541724721:2333] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:29.123017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342454541724721:2333] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:29.123027Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1531 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:29.123967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:29.124997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:29.125011Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:29.125013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:29.125017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:29.125023Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1530 [2:7559342454179898214:5194]
2025-10-09T21:49:29.125032Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:29.125049Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:29.125056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:29.125059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:29.125060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:29.125073Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:29.125077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:29.125083Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:29.125127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.125132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.125133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.125135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:29.125189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:29.125194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.125201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.125209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.125221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:29.125241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342454179898208:5194]
2025-10-09T21:49:29.125248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342454179898208:5194]
2025-10-09T21:49:29.125250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342454179898208:5194]
2025-10-09T21:49:29.125259Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:29.125310Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1530
2025-10-09T21:49:29.125265Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:29.125272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:29.125317Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1530
2025-10-09T21:49:29.125274Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:29.125276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342454541724642:2432], now have 1 active actors on pipe
2025-10-09T21:49:29.125279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342454541724761:2333], now have 1 active actors on pipe
2025-10-09T21:49:29.125283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:29.125298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:29.125300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:29.125334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342454179898208:5194]
2025-10-09T21:49:29.125337Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:29.125342Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:29.125375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:29.125380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:29.125419Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5861ed4b-b8108a32-2af677ee-8b5b1814_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:29.125434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:29.125439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:29.125441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:29.125452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:29.125468Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:29.125488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:29.125691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:29.125707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:29.125730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:29.125844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1531 partition: 0 MaxSeqNo: 25 sessionId: src_id|5861ed4b-b8108a32-2af677ee-8b5b1814_0
2025-10-09T21:49:29.126184Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83acb0c1-e998a091-19c60ffa-dfeae553_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5861ed4b-b8108a32-2af677ee-8b5b1814_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:29.128590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5861ed4b-b8108a32-2af677ee-8b5b1814_0] Write session: try to update token
2025-10-09T21:49:29.128609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5861ed4b-b8108a32-2af677ee-8b5b1814_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:29.139762Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:29.139779Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:29.139782Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:29.139784Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:29.139793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342454179898208:5194]
2025-10-09T21:49:29.139798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342454179898208:5194]
2025-10-09T21:49:29.139802Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342454179898208:5194]
2025-10-09T21:49:29.139847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:29.139854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.139857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:29.139857Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:29.139863Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:29.139867Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:29.139986Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:29.141152Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1531 sessionId: src_id|5861ed4b-b8108a32-2af677ee-8b5b1814_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:29.143464Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:29.204694Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342454541724772:2585]
2025-10-09T21:49:29.204977Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 1530, pipe: [1:7559342454541724772:2585]
2025-10-09T21:49:29.287668Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:29.288349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:29.288363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:29.288405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342454541724772:2585], now have 1 active actors on pipe
2025-10-09T21:49:29.288586Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:29.288616Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:29.288619Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342454541724772:2585]
2025-10-09T21:49:29.288644Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:29.288725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1531 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5861ed4b-b8108a32-2af677ee-8b5b1814_0
2025-10-09T21:49:29.288923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1531 sessionId: src_id|5861ed4b-b8108a32-2af677ee-8b5b1814_0 is DEAD
2025-10-09T21:49:29.289066Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5861ed4b-b8108a32-2af677ee-8b5b1814_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:29.289078Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5861ed4b-b8108a32-2af677ee-8b5b1814_0] Write session will restart in 0.006322s
2025-10-09T21:49:29.289101Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5861ed4b-b8108a32-2af677ee-8b5b1814_0] Write session: Do CDS request
2025-10-09T21:49:29.289107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5861ed4b-b8108a32-2af677ee-8b5b1814_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:29.296540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5861ed4b-b8108a32-2af677ee-8b5b1814_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:29.296676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:29.296690Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1532
2025-10-09T21:49:29.296834Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1532 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:29.296870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1532 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:29.296880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1532 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:29.296884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:29.297304Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1532 sessionId:  describe result for acl check
2025-10-09T21:49:29.297349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:29.297356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:29.297358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:29.297364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342454541724857:2307] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:29.297932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342454541724857:2307] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:29.298563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:29.299833Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342454541724857:2307] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:29.300228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:29.300409Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:29.300420Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:29.300433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:29.300482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:29.300521Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342454179898321:5191]
2025-10-09T21:49:29.301089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:29.301330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:29.301372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:29.301679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:29.301728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.301735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.301736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.301737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.301739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.301740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.301741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.301743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.301744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.301749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.301760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.301764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.301768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.301772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.301776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.301779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.301783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.301787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.301794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.301802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.301807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.301813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.301817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.301822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.301826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.301830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.301834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.301839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.301843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.301847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.301858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.301863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.301868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.301873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.301878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.301883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.301913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:29.302094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342454541724857:2307] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:29.302117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342454541724857:2307] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:29.302121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342454541724857:2307] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:29.302129Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1532 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:29.308262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:29.309354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:29.309370Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:29.309373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:29.309377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:29.309386Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1531 [2:7559342454179898321:5191]
2025-10-09T21:49:29.309401Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:29.309420Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:29.309425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:29.309429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:29.309431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:29.309448Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:29.309456Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:29.309464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:29.309532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.309539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.309541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.309544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:29.309624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:29.309632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.309644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.309656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.309671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:29.309702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342454541724899:2307], now have 1 active actors on pipe
2025-10-09T21:49:29.309710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342454541724779:2432], now have 1 active actors on pipe
2025-10-09T21:49:29.309718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342454179898316:5191]
2025-10-09T21:49:29.309720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342454179898316:5191]
2025-10-09T21:49:29.309722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342454179898316:5191]
2025-10-09T21:49:29.309734Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:29.309752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:29.309754Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:29.309757Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:29.309759Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:29.309769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:29.309773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:29.309765Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1531
2025-10-09T21:49:29.309768Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1531
2025-10-09T21:49:29.309800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342454179898316:5191]
2025-10-09T21:49:29.309810Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:29.309819Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:29.309830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:29.309835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:29.309883Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dabd9adc-c6ef2e5d-1ebb0f2-af819472_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:29.309895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:29.309899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:29.309901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:29.309911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:29.309922Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:29.309940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:29.310093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:29.310104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:29.310121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:29.310199Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1532 partition: 0 MaxSeqNo: 25 sessionId: src_id|dabd9adc-c6ef2e5d-1ebb0f2-af819472_0
2025-10-09T21:49:29.310525Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5861ed4b-b8108a32-2af677ee-8b5b1814_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dabd9adc-c6ef2e5d-1ebb0f2-af819472_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:29.312889Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dabd9adc-c6ef2e5d-1ebb0f2-af819472_0] Write session: try to update token
2025-10-09T21:49:29.312905Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dabd9adc-c6ef2e5d-1ebb0f2-af819472_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:29.324275Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:29.324295Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:29.324297Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:29.324299Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:29.324308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342454179898316:5191]
2025-10-09T21:49:29.324312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342454179898316:5191]
2025-10-09T21:49:29.324316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342454179898316:5191]
2025-10-09T21:49:29.324361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:29.324369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.324372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:29.324371Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:29.324377Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:29.324380Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:29.324510Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:29.326052Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1532 sessionId: src_id|dabd9adc-c6ef2e5d-1ebb0f2-af819472_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:29.328337Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:29.388540Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342454541724907:2585]
2025-10-09T21:49:29.388814Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 1531, pipe: [1:7559342454541724907:2585]
2025-10-09T21:49:29.474936Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:29.475223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:29.475237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:29.475279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342454541724907:2585], now have 1 active actors on pipe
2025-10-09T21:49:29.475503Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:29.475527Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:29.475540Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:29.475546Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:29.475590Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342454541724907:2585]
2025-10-09T21:49:29.475612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1532 reason: error at write: Disconnected sessionId: src_id|dabd9adc-c6ef2e5d-1ebb0f2-af819472_0
2025-10-09T21:49:29.475805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1532 sessionId: src_id|dabd9adc-c6ef2e5d-1ebb0f2-af819472_0 is DEAD
2025-10-09T21:49:29.475999Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dabd9adc-c6ef2e5d-1ebb0f2-af819472_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:29.476008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dabd9adc-c6ef2e5d-1ebb0f2-af819472_0] Write session will restart in 0.008475s
2025-10-09T21:49:29.476030Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dabd9adc-c6ef2e5d-1ebb0f2-af819472_0] Write session: Do CDS request
2025-10-09T21:49:29.476034Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dabd9adc-c6ef2e5d-1ebb0f2-af819472_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:29.485071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:29.485474Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dabd9adc-c6ef2e5d-1ebb0f2-af819472_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:29.485554Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:29.485565Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1533
2025-10-09T21:49:29.485705Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1533 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:29.485729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1533 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:29.485736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1533 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:29.485739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:29.486186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1533 sessionId:  describe result for acl check
2025-10-09T21:49:29.486223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:29.486229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:29.486230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:29.486235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342454541724991:2301] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:29.486772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342454541724991:2301] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:29.487037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:29.487207Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:29.487219Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:29.487232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:29.487294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:29.487329Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342454179898430:5174]
2025-10-09T21:49:29.487889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:29.488192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:29.488237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:29.488460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:29.488495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.488501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.488502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.488503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.488503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.488504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.488504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.488505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.488505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.488509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.488515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.488517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.488519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.488521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.488523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.488525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.488527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.488529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.488533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.488539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.488543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.488546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.488550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.488554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.488557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.488559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.488562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.488565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.488568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.488571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.488574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.488577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.488579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.488582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.488585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.488590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.488617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:29.488679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342454541724991:2301] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:29.494190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342454541724991:2301] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:29.494204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342454541724991:2301] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:29.494207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342454541724991:2301] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:29.494215Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1533 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:29.494632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:29.495763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:29.495779Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:29.495781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:29.495785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:29.495791Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1532 [2:7559342454179898430:5174]
2025-10-09T21:49:29.495805Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:29.495826Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:29.495835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:29.495838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:29.495842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:29.495857Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:29.495863Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:29.495875Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:29.495930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.495938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.495940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.495942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:29.496004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:29.496012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.496023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.496039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.496056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:29.496080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342454541725029:2301], now have 1 active actors on pipe
2025-10-09T21:49:29.496088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342454541724909:2432], now have 1 active actors on pipe
2025-10-09T21:49:29.496097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342454179898425:5174]
2025-10-09T21:49:29.496104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342454179898425:5174]
2025-10-09T21:49:29.496106Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342454179898425:5174]
2025-10-09T21:49:29.496119Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:29.496137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:29.496125Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1532
2025-10-09T21:49:29.496138Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:29.496124Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1532
2025-10-09T21:49:29.496141Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:29.496143Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:29.496156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:29.496159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:29.496194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:29.496198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:29.496215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342454179898425:5174]
2025-10-09T21:49:29.496228Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:29.496238Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:29.496254Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3880c5b8-a605596f-838f3d8a-92dc3379_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:29.496265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:29.496269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:29.496275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:29.496285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:29.496299Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:29.496327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:29.496442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:29.496451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:29.496466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:29.496565Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1533 partition: 0 MaxSeqNo: 25 sessionId: src_id|3880c5b8-a605596f-838f3d8a-92dc3379_0
2025-10-09T21:49:29.496868Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dabd9adc-c6ef2e5d-1ebb0f2-af819472_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3880c5b8-a605596f-838f3d8a-92dc3379_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:29.499363Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3880c5b8-a605596f-838f3d8a-92dc3379_0] Write session: try to update token
2025-10-09T21:49:29.499382Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3880c5b8-a605596f-838f3d8a-92dc3379_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:29.509571Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:29.509602Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:29.509605Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:29.509607Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:29.509617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342454179898425:5174]
2025-10-09T21:49:29.509621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342454179898425:5174]
2025-10-09T21:49:29.509624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342454179898425:5174]
2025-10-09T21:49:29.509658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:29.509666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.509668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:29.509671Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:29.509685Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:29.509689Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:29.509785Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:29.511655Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1533 sessionId: src_id|3880c5b8-a605596f-838f3d8a-92dc3379_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:29.513878Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:29.575930Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342454541725038:2585]
2025-10-09T21:49:29.576192Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 1532, pipe: [1:7559342454541725038:2585]
2025-10-09T21:49:29.656389Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:29.656608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:29.656621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:29.656657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342454541725038:2585], now have 1 active actors on pipe
2025-10-09T21:49:29.656869Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:29.656870Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342454541725038:2585]
2025-10-09T21:49:29.656882Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:29.656892Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:29.656942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1533 reason: error at write: Disconnected sessionId: src_id|3880c5b8-a605596f-838f3d8a-92dc3379_0
2025-10-09T21:49:29.657088Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1533 sessionId: src_id|3880c5b8-a605596f-838f3d8a-92dc3379_0 is DEAD
2025-10-09T21:49:29.657250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3880c5b8-a605596f-838f3d8a-92dc3379_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:29.657258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3880c5b8-a605596f-838f3d8a-92dc3379_0] Write session will restart in 0.005377s
2025-10-09T21:49:29.657281Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3880c5b8-a605596f-838f3d8a-92dc3379_0] Write session: Do CDS request
2025-10-09T21:49:29.657290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3880c5b8-a605596f-838f3d8a-92dc3379_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:29.657280Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:29.663498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3880c5b8-a605596f-838f3d8a-92dc3379_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:29.663551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:29.663563Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1534
2025-10-09T21:49:29.663713Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1534 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:29.663746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1534 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:29.663757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1534 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:29.663767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:29.664169Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1534 sessionId:  describe result for acl check
2025-10-09T21:49:29.664199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:29.664206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:29.664208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:29.664218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342454541725094:8354] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:29.664752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342454541725094:8354] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:29.666823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:29.666997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342454541725094:8354] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:29.668744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:29.668943Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:29.668954Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:29.668966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:29.669020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:29.669059Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342454179898541:5185]
2025-10-09T21:49:29.669171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342454541725094:8354] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:29.669183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342454541725094:8354] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:29.669186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342454541725094:8354] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:29.669194Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1534 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:29.669644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:29.669980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:29.670061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:29.670329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:29.670368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.670373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.670374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.670374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.670375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.670376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.670376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.670377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.670378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.670381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.670390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.670395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.670398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.670403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.670407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.670411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.670415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.670419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.670428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.670437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.670447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.670456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.670460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.670466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.670474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.670478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.670487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.670496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.670506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.670511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.670516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.670522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.670527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.670533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.670539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.670549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.670579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:29.677011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:29.678378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:29.678396Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:29.678398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:29.678402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:29.678410Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1533 [2:7559342454179898541:5185]
2025-10-09T21:49:29.678423Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:29.678440Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:29.678448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:29.678451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:29.678454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:29.678470Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:29.678483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:29.678490Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:29.678547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.678554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.678556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.678559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:29.678624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:29.678632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.678645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.678661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.678681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:29.678707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342454541725158:8354], now have 1 active actors on pipe
2025-10-09T21:49:29.678717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342454541725040:2432], now have 1 active actors on pipe
2025-10-09T21:49:29.678724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342454179898535:5185]
2025-10-09T21:49:29.678727Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342454179898535:5185]
2025-10-09T21:49:29.678729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342454179898535:5185]
2025-10-09T21:49:29.678742Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:29.678756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:29.678750Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1533
2025-10-09T21:49:29.678766Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:29.678762Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1533
2025-10-09T21:49:29.678775Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:29.678775Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:29.678777Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:29.678777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:29.678806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342454179898535:5185]
2025-10-09T21:49:29.678814Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:29.678827Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:29.678831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:29.678836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:29.678885Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|76ec5122-2be3acdf-6df0523e-9bd0334_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:29.678898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:29.678902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:29.678909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:29.678919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:29.678929Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:29.678951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:29.679117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:29.679130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:29.679147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:29.679256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1534 partition: 0 MaxSeqNo: 25 sessionId: src_id|76ec5122-2be3acdf-6df0523e-9bd0334_0
2025-10-09T21:49:29.679568Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3880c5b8-a605596f-838f3d8a-92dc3379_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|76ec5122-2be3acdf-6df0523e-9bd0334_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:29.681940Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76ec5122-2be3acdf-6df0523e-9bd0334_0] Write session: try to update token
2025-10-09T21:49:29.681957Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76ec5122-2be3acdf-6df0523e-9bd0334_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:29.693139Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:29.693154Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:29.693156Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:29.693157Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:29.693164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342454179898535:5185]
2025-10-09T21:49:29.693168Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342454179898535:5185]
2025-10-09T21:49:29.693170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342454179898535:5185]
2025-10-09T21:49:29.693205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:29.693211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.693213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:29.693215Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:29.693221Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:29.693225Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:29.693340Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:29.694776Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1534 sessionId: src_id|76ec5122-2be3acdf-6df0523e-9bd0334_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:29.697034Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:29.757156Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342454541725170:2585]
2025-10-09T21:49:29.757375Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 1533, pipe: [1:7559342454541725170:2585]
2025-10-09T21:49:29.838544Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:29.839016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:29.839031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:29.839079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342454541725170:2585], now have 1 active actors on pipe
2025-10-09T21:49:29.839271Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:29.839282Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:29.839312Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:29.839294Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342454541725170:2585]
2025-10-09T21:49:29.839315Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:29.839367Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1534 reason: error at write: Disconnected sessionId: src_id|76ec5122-2be3acdf-6df0523e-9bd0334_0
2025-10-09T21:49:29.839559Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1534 sessionId: src_id|76ec5122-2be3acdf-6df0523e-9bd0334_0 is DEAD
2025-10-09T21:49:29.839718Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76ec5122-2be3acdf-6df0523e-9bd0334_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:29.839726Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76ec5122-2be3acdf-6df0523e-9bd0334_0] Write session will restart in 0.009357s
2025-10-09T21:49:29.839753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76ec5122-2be3acdf-6df0523e-9bd0334_0] Write session: Do CDS request
2025-10-09T21:49:29.839759Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76ec5122-2be3acdf-6df0523e-9bd0334_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:29.849401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:29.850501Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|76ec5122-2be3acdf-6df0523e-9bd0334_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:29.850559Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:29.850571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1535
2025-10-09T21:49:29.850723Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1535 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:29.850766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1535 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:29.850778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1535 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:29.850782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:29.851189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1535 sessionId:  describe result for acl check
2025-10-09T21:49:29.851224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:29.851232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:29.851234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:29.851240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342454541725260:8347] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:29.851256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:29.851437Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:29.851446Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:29.851456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:29.851499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:29.851534Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342454179898657:5172]
2025-10-09T21:49:29.851778Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342454541725260:8347] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:29.851981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:29.852245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:29.852285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:29.852502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:29.852534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.852538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.852539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.852539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.852540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.852540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.852541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.852542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.852542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.852546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.852552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.852554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.852556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.852560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.852564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.852568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.852572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.852575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.852582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.852590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:29.852594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.852598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:29.852600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.852604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:29.852607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.852610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:29.852612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.852615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:29.852618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.852621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:29.852624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.852627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.852629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.852632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.852635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.852638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.852653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:29.853660Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342454541725260:8347] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:29.858385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342454541725260:8347] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:29.858400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342454541725260:8347] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:29.858403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342454541725260:8347] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:29.858411Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1535 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:29.858747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:29.859833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:29.859845Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:29.859846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:29.859849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:29.859853Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1534 [2:7559342454179898657:5172]
2025-10-09T21:49:29.859865Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:29.859881Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:29.859888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:29.859891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:29.859894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:29.859909Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:29.859918Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:29.859925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:29.859982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.859990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.859992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.859995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:29.860164Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1534
2025-10-09T21:49:29.860177Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1534
2025-10-09T21:49:29.860261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:29.860268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.860276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:29.860282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:29.860297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:29.860319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342454541725298:8347], now have 1 active actors on pipe
2025-10-09T21:49:29.860325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342454541725172:2432], now have 1 active actors on pipe
2025-10-09T21:49:29.860344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:29.860353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:29.860371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342454179898650:5172]
2025-10-09T21:49:29.860377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342454179898650:5172]
2025-10-09T21:49:29.860379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342454179898650:5172]
2025-10-09T21:49:29.860392Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:29.860406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:29.860408Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:29.860410Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:29.860412Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:29.860425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:29.860428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:29.860448Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|55457eb1-82e0bc8e-f700d7c0-a86ed9b9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:29.860459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:29.860463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:29.860469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:29.860479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:29.860489Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:29.860521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342454179898650:5172]
2025-10-09T21:49:29.860532Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:29.860542Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:29.860544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:29.860692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:29.860702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:29.860718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:29.860806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1535 partition: 0 MaxSeqNo: 25 sessionId: src_id|55457eb1-82e0bc8e-f700d7c0-a86ed9b9_0
2025-10-09T21:49:29.861101Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|76ec5122-2be3acdf-6df0523e-9bd0334_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|55457eb1-82e0bc8e-f700d7c0-a86ed9b9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:29.863589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55457eb1-82e0bc8e-f700d7c0-a86ed9b9_0] Write session: try to update token
2025-10-09T21:49:29.863607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55457eb1-82e0bc8e-f700d7c0-a86ed9b9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:29.871692Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:29.871711Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:29.871725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342454179898650:5172]
2025-10-09T21:49:29.871768Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:29.871767Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:29.871935Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:29.872083Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:29.872224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:29.872357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:29.872548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:29.872595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:29.876656Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1535 sessionId: src_id|55457eb1-82e0bc8e-f700d7c0-a86ed9b9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:29.879076Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:29.879278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:29.879300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:29.879338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:29.879444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:29.885237Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:29.885252Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:29.885255Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:29.885257Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:29.885268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342454179898650:5172]
2025-10-09T21:49:29.885271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342454179898650:5172]
2025-10-09T21:49:29.885274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342454179898650:5172]
2025-10-09T21:49:29.885311Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:29.885313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:29.885316Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:29.885321Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:29.885321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:29.885325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:29.885451Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:29.939649Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342454541725307:2585]
2025-10-09T21:49:29.939909Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 1534, pipe: [1:7559342454541725307:2585]
2025-10-09T21:49:30.038829Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:30.046198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.046228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.046377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.046436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.046496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.046562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.046609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.046616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.046676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.046732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.046795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.046856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.046895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.046900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.046957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.047014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.047075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.047132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.047180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.047188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.047248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.047311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.047367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.047422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.047463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.047465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.047527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.047581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.047640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.047732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.047778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.047785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.047848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.047904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.047959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.048015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.048061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.048069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.048127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.048177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.048231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.048284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.048328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.048335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.048389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.048438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.048490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.048542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.048593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.048600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.048655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.048705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.048760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.048814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.048857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.048863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.048919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.048973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.049028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.049089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.049130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.049135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.049178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342454541725307:2585], now have 1 active actors on pipe
2025-10-09T21:49:30.049441Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342454541725307:2585]
2025-10-09T21:49:30.049479Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:30.049482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:30.049494Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:30.049525Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1535 reason: error at write: Disconnected sessionId: src_id|55457eb1-82e0bc8e-f700d7c0-a86ed9b9_0
2025-10-09T21:49:30.049738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1535 sessionId: src_id|55457eb1-82e0bc8e-f700d7c0-a86ed9b9_0 is DEAD
2025-10-09T21:49:30.049869Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55457eb1-82e0bc8e-f700d7c0-a86ed9b9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:30.049874Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55457eb1-82e0bc8e-f700d7c0-a86ed9b9_0] Write session will restart in 0.005710s
2025-10-09T21:49:30.049892Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55457eb1-82e0bc8e-f700d7c0-a86ed9b9_0] Write session: Do CDS request
2025-10-09T21:49:30.049896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55457eb1-82e0bc8e-f700d7c0-a86ed9b9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:30.050074Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:30.056524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55457eb1-82e0bc8e-f700d7c0-a86ed9b9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:30.056633Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:30.056654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1536
2025-10-09T21:49:30.056788Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1536 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:30.056827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1536 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:30.056839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1536 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:30.056843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:30.057337Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1536 sessionId:  describe result for acl check
2025-10-09T21:49:30.057382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:30.057385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:30.057386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:30.057392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342458836692670:8335] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:30.057965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342458836692670:8335] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:30.060024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342458836692670:8335] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:30.060696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:30.062941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:30.063129Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:30.063138Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:30.063150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:30.063204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:30.063247Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342458474866103:5156]
2025-10-09T21:49:30.063506Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342458836692670:8335] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:30.063520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342458836692670:8335] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:30.063525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342458836692670:8335] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:30.063536Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1536 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:30.063739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:30.063984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:30.064021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:30.064261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:30.064319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.064327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.064328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.064330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.064331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.064332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.064333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.064334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.064335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.064340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.064351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.064355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.064359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.064362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.064364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.064366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.064368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.064370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.064377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.064383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.064387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.064392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.064395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.064398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.064401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.064404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.064407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.064410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.064412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.064415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.064418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.064421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.064423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.064426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.064429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.064433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.064448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:30.071237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:30.072393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:30.072406Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:30.072408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:30.072411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:30.072417Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1535 [2:7559342458474866103:5156]
2025-10-09T21:49:30.072427Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:30.072443Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:30.072445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:30.072447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:30.072448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:30.072463Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:30.072469Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:30.072475Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:30.072523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.072529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.072530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.072532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:30.072590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:30.072597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.072604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.072611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.072620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:30.072636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342458474866097:5156]
2025-10-09T21:49:30.072638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342458474866097:5156]
2025-10-09T21:49:30.072639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342458474866097:5156]
2025-10-09T21:49:30.072646Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:30.072658Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:30.072658Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:30.072661Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:30.072664Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:30.072668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:30.072671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:30.072682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342458836692748:8335], now have 1 active actors on pipe
2025-10-09T21:49:30.072686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342458836692631:2432], now have 1 active actors on pipe
2025-10-09T21:49:30.072719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342458474866097:5156]
2025-10-09T21:49:30.072731Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:30.072742Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:30.072716Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1535
2025-10-09T21:49:30.072734Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1535
2025-10-09T21:49:30.072771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.072786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.072812Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e2823870-14283bb4-403eb4f5-59e3c603_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:30.072825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:30.072828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:30.072830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:30.072838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:30.072846Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:30.072857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:30.073028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.073036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.073049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:30.073132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1536 partition: 0 MaxSeqNo: 25 sessionId: src_id|e2823870-14283bb4-403eb4f5-59e3c603_0
2025-10-09T21:49:30.073494Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55457eb1-82e0bc8e-f700d7c0-a86ed9b9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e2823870-14283bb4-403eb4f5-59e3c603_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:30.076071Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2823870-14283bb4-403eb4f5-59e3c603_0] Write session: try to update token
2025-10-09T21:49:30.076088Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2823870-14283bb4-403eb4f5-59e3c603_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:30.084660Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:30.084680Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:30.084691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342458474866097:5156]
2025-10-09T21:49:30.084742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:30.084746Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:30.084927Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:30.085088Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:30.085255Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:30.085389Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:30.085572Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:30.085639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:30.089982Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1536 sessionId: src_id|e2823870-14283bb4-403eb4f5-59e3c603_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:30.092429Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:30.092601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.092621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.092658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:30.092727Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:30.098200Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:30.098215Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:30.098217Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:30.098219Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:30.098233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342458474866097:5156]
2025-10-09T21:49:30.098236Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342458474866097:5156]
2025-10-09T21:49:30.098243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342458474866097:5156]
2025-10-09T21:49:30.098281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:30.098287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.098288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:30.098293Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:30.098299Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:30.098303Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:30.098419Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:30.149843Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342458836692759:2585]
2025-10-09T21:49:30.150126Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 1535, pipe: [1:7559342458836692759:2585]
2025-10-09T21:49:30.247244Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:30.254372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.254393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.254553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.254619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.254681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.254739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.254783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.254787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.254863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.254931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.254995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.255055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.255093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.255099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.255154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.255205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.255255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.255334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.255383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.255389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.255448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.255508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.255579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.255642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.255682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.255687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.255745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.255805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.255871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.255930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.256042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.256048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.256102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.256154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.256207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.256261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.256315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.256321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.256373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.256430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.256495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.256562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.256604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.256609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.256663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.256713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.256773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.256838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.256881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.256887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.256940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.256987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.257039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.257091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.257129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.257134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.257191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.257242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.257296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.257349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.257388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.257393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.257436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342458836692759:2585], now have 1 active actors on pipe
2025-10-09T21:49:30.257674Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342458836692759:2585]
2025-10-09T21:49:30.257692Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:30.257713Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:30.257715Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:30.257742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1536 reason: error at write: Disconnected sessionId: src_id|e2823870-14283bb4-403eb4f5-59e3c603_0
2025-10-09T21:49:30.257925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1536 sessionId: src_id|e2823870-14283bb4-403eb4f5-59e3c603_0 is DEAD
2025-10-09T21:49:30.258078Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2823870-14283bb4-403eb4f5-59e3c603_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:30.258091Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2823870-14283bb4-403eb4f5-59e3c603_0] Write session will restart in 0.005116s
2025-10-09T21:49:30.258116Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2823870-14283bb4-403eb4f5-59e3c603_0] Write session: Do CDS request
2025-10-09T21:49:30.258123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2823870-14283bb4-403eb4f5-59e3c603_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:30.258433Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:30.264483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2823870-14283bb4-403eb4f5-59e3c603_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:30.264573Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:30.264585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1537
2025-10-09T21:49:30.264681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1537 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:30.264715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1537 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:30.264720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1537 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:30.264724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:30.265113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1537 sessionId:  describe result for acl check
2025-10-09T21:49:30.265149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:30.265153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:30.265155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:30.265161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342458836692801:8325] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:30.265652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342458836692801:8325] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:30.268016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342458836692801:8325] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:30.268589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:30.270334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:30.270505Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:30.270515Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:30.270526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:30.270541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342458836692801:8325] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:30.270556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342458836692801:8325] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:30.270559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342458836692801:8325] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:30.270566Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1537 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:30.270574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:30.270614Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342458474866216:5149]
2025-10-09T21:49:30.271192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:30.271447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:30.271482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:30.271703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:30.271733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.271737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.271738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.271738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.271739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.271740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.271740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.271741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.271741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.271744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.271751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.271753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.271755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.271757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.271760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.271761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.271764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.271766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.271770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.271775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.271779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.271782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.271786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.271789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.271791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.271795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.271797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.271800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.271802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.271806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.271808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.271811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.271814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.271817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.271820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.271825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.271840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:30.277924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:30.279025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:30.279041Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:30.279044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:30.279048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:30.279066Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1536 [2:7559342458474866216:5149]
2025-10-09T21:49:30.279080Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:30.279096Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:30.279108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:30.279111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:30.279114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:30.279133Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:30.279141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:30.279149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:30.279215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.279222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.279224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.279227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:30.279296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:30.279306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.279318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.279331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.279349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:30.279377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342458836692891:8325], now have 1 active actors on pipe
2025-10-09T21:49:30.279386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342458836692770:2432], now have 1 active actors on pipe
2025-10-09T21:49:30.279394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342458474866210:5149]
2025-10-09T21:49:30.279396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342458474866210:5149]
2025-10-09T21:49:30.279398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342458474866210:5149]
2025-10-09T21:49:30.279411Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:30.279426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:30.279435Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:30.279438Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:30.279440Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:30.279445Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:30.279448Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:30.279430Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1536
2025-10-09T21:49:30.279433Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1536
2025-10-09T21:49:30.279475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342458474866210:5149]
2025-10-09T21:49:30.279487Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:30.279490Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:30.279509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.279518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.279568Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|631908d0-fb7a045f-aca1d35c-b487260f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:30.279581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:30.279585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:30.279587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:30.279598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:30.279613Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:30.279637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:30.279816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.279824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.279839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:30.279957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1537 partition: 0 MaxSeqNo: 25 sessionId: src_id|631908d0-fb7a045f-aca1d35c-b487260f_0
2025-10-09T21:49:30.280325Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2823870-14283bb4-403eb4f5-59e3c603_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|631908d0-fb7a045f-aca1d35c-b487260f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:30.282728Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|631908d0-fb7a045f-aca1d35c-b487260f_0] Write session: try to update token
2025-10-09T21:49:30.282744Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|631908d0-fb7a045f-aca1d35c-b487260f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:30.293548Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:30.293567Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:30.293570Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:30.293572Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:30.293593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342458474866210:5149]
2025-10-09T21:49:30.293598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342458474866210:5149]
2025-10-09T21:49:30.293605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342458474866210:5149]
2025-10-09T21:49:30.293651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:30.293661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.293664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:30.293663Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:30.293669Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:30.293673Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:30.293795Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:30.295505Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1537 sessionId: src_id|631908d0-fb7a045f-aca1d35c-b487260f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:30.298070Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:30.357744Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342458836692903:2585]
2025-10-09T21:49:30.358079Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 1536, pipe: [1:7559342458836692903:2585]
2025-10-09T21:49:30.446636Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:30.447180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.447195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.447238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342458836692903:2585], now have 1 active actors on pipe
2025-10-09T21:49:30.447413Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:30.447461Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:30.447464Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:30.447483Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:30.447509Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342458836692903:2585]
2025-10-09T21:49:30.447547Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1537 reason: error at write: Disconnected sessionId: src_id|631908d0-fb7a045f-aca1d35c-b487260f_0
2025-10-09T21:49:30.447716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1537 sessionId: src_id|631908d0-fb7a045f-aca1d35c-b487260f_0 is DEAD
2025-10-09T21:49:30.447853Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|631908d0-fb7a045f-aca1d35c-b487260f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:30.447864Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|631908d0-fb7a045f-aca1d35c-b487260f_0] Write session will restart in 0.009173s
2025-10-09T21:49:30.447886Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|631908d0-fb7a045f-aca1d35c-b487260f_0] Write session: Do CDS request
2025-10-09T21:49:30.447891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|631908d0-fb7a045f-aca1d35c-b487260f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:30.456554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:30.458553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|631908d0-fb7a045f-aca1d35c-b487260f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:30.458607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:30.458621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1538
2025-10-09T21:49:30.458662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:30.458759Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1538 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:30.458783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1538 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:30.458790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1538 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:30.458793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:30.458811Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:30.458820Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:30.458828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:30.458863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:30.458889Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342458474866330:5144]
2025-10-09T21:49:30.459186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1538 sessionId:  describe result for acl check
2025-10-09T21:49:30.459213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:30.459218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:30.459220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:30.459224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342458836692992:8320] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:30.459361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:30.459632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:30.459669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:30.459695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342458836692992:8320] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:30.459885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:30.459916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.459920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.459921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.459922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.459922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.459923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.459923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.459924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.459925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.459930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.459939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.459943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.459947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.459952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.459956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.459959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.459961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.459963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.459967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.459972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.459976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.459979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.459981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.459984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.459987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.459990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.459992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.459995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.459998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.460001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.460003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.460006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.460009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.460012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.460014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.460019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.460034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:30.461344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342458836692992:8320] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:30.465706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342458836692992:8320] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:30.465721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342458836692992:8320] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:30.465726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342458836692992:8320] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:30.465736Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1538 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:30.466153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:30.467242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:30.467257Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:30.467260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:30.467264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:30.467271Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1537 [2:7559342458474866330:5144]
2025-10-09T21:49:30.467283Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:30.467300Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:30.467303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:30.467306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:30.467308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:30.467324Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:30.467331Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:30.467340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:30.467398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.467405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.467408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.467410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:30.467478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:30.467487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.467498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.467511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.467532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:30.467557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342458836693030:8320], now have 1 active actors on pipe
2025-10-09T21:49:30.467565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342458836692905:2432], now have 1 active actors on pipe
2025-10-09T21:49:30.467574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342458474866324:5144]
2025-10-09T21:49:30.467578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342458474866324:5144]
2025-10-09T21:49:30.467581Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342458474866324:5144]
2025-10-09T21:49:30.467568Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1537
2025-10-09T21:49:30.467588Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1537
2025-10-09T21:49:30.467596Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:30.467617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:30.467632Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:30.467638Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:30.467640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:30.467645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:30.467648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:30.467665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.467675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.467715Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342458474866324:5144]
2025-10-09T21:49:30.467726Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:30.467739Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:30.467746Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|aaa05b97-ff2a3e50-2dddee22-b409f037_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:30.467753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:30.467757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:30.467760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:30.467770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:30.467783Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:30.467818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:30.467956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.467964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.467979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:30.468057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1538 partition: 0 MaxSeqNo: 25 sessionId: src_id|aaa05b97-ff2a3e50-2dddee22-b409f037_0
2025-10-09T21:49:30.468386Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|631908d0-fb7a045f-aca1d35c-b487260f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|aaa05b97-ff2a3e50-2dddee22-b409f037_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:30.470794Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aaa05b97-ff2a3e50-2dddee22-b409f037_0] Write session: try to update token
2025-10-09T21:49:30.470811Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aaa05b97-ff2a3e50-2dddee22-b409f037_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:30.478992Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:30.479011Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:30.479021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342458474866324:5144]
2025-10-09T21:49:30.479064Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:30.479067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:30.479258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:30.479404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:30.479546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:30.479691Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:30.479886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:30.479931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:30.483738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1538 sessionId: src_id|aaa05b97-ff2a3e50-2dddee22-b409f037_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:30.485946Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:30.486147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.486168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.486204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:30.486274Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:30.491709Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:30.491723Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:30.491724Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:30.491726Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:30.491734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342458474866324:5144]
2025-10-09T21:49:30.491737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342458474866324:5144]
2025-10-09T21:49:30.491741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342458474866324:5144]
2025-10-09T21:49:30.491774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:30.491780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.491782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:30.491775Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:30.491790Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:30.491793Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:30.491897Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:30.547699Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342458836693038:2585]
2025-10-09T21:49:30.547981Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 1537, pipe: [1:7559342458836693038:2585]
2025-10-09T21:49:30.644962Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:30.652745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.652772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.652942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.653005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.653068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.653130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.653174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.653190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.653260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.653317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.653382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.653445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.653489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.653496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.653552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.653622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.653679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.653735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.653773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.653780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.653842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.653902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.653964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.654022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.654077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.654084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.654152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.654212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.654271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.654329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.654375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.654383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.654441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.654497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.654555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.654615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.654657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.654664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.654720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.654776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.654833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.654891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.654933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.654940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.655000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.655055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.655111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.655170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.655216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.655223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.655281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.655332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.655390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.655450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.655490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.655497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.655556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.655608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.655668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.655729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.655769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.655776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.655832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342458836693038:2585], now have 1 active actors on pipe
2025-10-09T21:49:30.656028Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:30.656069Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:30.656074Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342458836693038:2585]
2025-10-09T21:49:30.656074Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:30.656106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1538 reason: error at write: Disconnected sessionId: src_id|aaa05b97-ff2a3e50-2dddee22-b409f037_0
2025-10-09T21:49:30.656299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1538 sessionId: src_id|aaa05b97-ff2a3e50-2dddee22-b409f037_0 is DEAD
2025-10-09T21:49:30.656445Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aaa05b97-ff2a3e50-2dddee22-b409f037_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:30.656466Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aaa05b97-ff2a3e50-2dddee22-b409f037_0] Write session will restart in 0.005722s
2025-10-09T21:49:30.656491Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aaa05b97-ff2a3e50-2dddee22-b409f037_0] Write session: Do CDS request
2025-10-09T21:49:30.656497Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aaa05b97-ff2a3e50-2dddee22-b409f037_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:30.656992Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:30.663498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aaa05b97-ff2a3e50-2dddee22-b409f037_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:30.663585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:30.663599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1539
2025-10-09T21:49:30.663703Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1539 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:30.663746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1539 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:30.663756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1539 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:30.663760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:30.664215Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1539 sessionId:  describe result for acl check
2025-10-09T21:49:30.664254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:30.664260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:30.664261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:30.664268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342458836693102:8315] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:30.664847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342458836693102:8315] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:30.666601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:30.667592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342458836693102:8315] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:30.668866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:30.669038Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:30.669050Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:30.669059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:30.669101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:30.669140Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342458474866446:5131]
2025-10-09T21:49:30.669575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:30.669930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:30.669973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:30.670275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:30.670334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.670316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342458836693102:8315] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:30.670341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.670329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342458836693102:8315] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:30.670342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.670333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342458836693102:8315] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:30.670343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.670341Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1539 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:30.670344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.670346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.670347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.670348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.670349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.670355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.670362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.670364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.670366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.670369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.670371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.670373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.670375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.670377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.670382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.670388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.670392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.670395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.670398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.670401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.670403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.670406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.670409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.670412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.670414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.670417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.670421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.670425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.670427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.670430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.670433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.670436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.670451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:30.676708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:30.678146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:30.678157Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:30.678159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:30.678162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:30.678168Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1538 [2:7559342458474866446:5131]
2025-10-09T21:49:30.678178Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:30.678194Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:30.678203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:30.678206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:30.678208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:30.678225Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:30.678232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:30.678242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:30.678284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.678290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.678291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.678293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:30.678347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:30.678352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.678359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.678365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.678375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:30.678393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342458474866440:5131]
2025-10-09T21:49:30.678397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342458474866440:5131]
2025-10-09T21:49:30.678399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342458474866440:5131]
2025-10-09T21:49:30.678404Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:30.678415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:30.678415Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:30.678418Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:30.678420Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:30.678423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:30.678425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:30.678434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342458836693165:8315], now have 1 active actors on pipe
2025-10-09T21:49:30.678436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342458836693040:2432], now have 1 active actors on pipe
2025-10-09T21:49:30.678457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342458474866440:5131]
2025-10-09T21:49:30.678466Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:30.678489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:30.678521Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1538
2025-10-09T21:49:30.678522Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1538
2025-10-09T21:49:30.678574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.678583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.678607Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7cd922d-3840df4b-ded0e991-8d37bea6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:30.678613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:30.678616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:30.678618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:30.678625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:30.678633Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:30.678645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:30.678797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.678803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.678812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:30.678893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1539 partition: 0 MaxSeqNo: 25 sessionId: src_id|7cd922d-3840df4b-ded0e991-8d37bea6_0
2025-10-09T21:49:30.679253Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aaa05b97-ff2a3e50-2dddee22-b409f037_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7cd922d-3840df4b-ded0e991-8d37bea6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:30.681766Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7cd922d-3840df4b-ded0e991-8d37bea6_0] Write session: try to update token
2025-10-09T21:49:30.681784Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7cd922d-3840df4b-ded0e991-8d37bea6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:30.690011Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:30.690027Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:30.690050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342458474866440:5131]
2025-10-09T21:49:30.690093Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:30.690098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:30.690616Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:30.690755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:30.690902Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:30.691032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:30.691215Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:30.691255Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:30.695427Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1539 sessionId: src_id|7cd922d-3840df4b-ded0e991-8d37bea6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:30.697821Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:30.698053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.698074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.698116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:30.698203Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:30.703451Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:30.703464Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:30.703465Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:30.703467Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:30.703477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342458474866440:5131]
2025-10-09T21:49:30.703480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342458474866440:5131]
2025-10-09T21:49:30.703486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342458474866440:5131]
2025-10-09T21:49:30.703523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:30.703528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.703530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:30.703533Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:30.703539Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:30.703544Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:30.703657Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:30.756101Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342458836693175:2585]
2025-10-09T21:49:30.756401Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 1538, pipe: [1:7559342458836693175:2585]
2025-10-09T21:49:30.853865Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:30.861199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.861224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.861384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.861446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.861508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.861593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.861640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.861652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.861718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.861781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.861855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.861934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.861984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.861991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.862077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.862139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.862201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.862262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.862309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.862317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.862386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.862447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.862506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.862573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.862619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.862626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.862718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.862779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.862849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.862916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.862960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.862966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.863028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.863096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.863159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.863222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.863268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.863274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.863339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.863398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.863458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.863517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.863558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.863564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.863633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.863691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.863753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.863820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.863863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.863869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.863926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.863980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.864040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.864104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.864161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.864167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.864229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:30.864320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:30.864382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:30.864448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:30.864499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:30.864507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:30.864579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342458836693175:2585], now have 1 active actors on pipe
2025-10-09T21:49:30.864835Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342458836693175:2585]
2025-10-09T21:49:30.864860Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:30.864874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:30.864877Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:30.864910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1539 reason: error at write: Disconnected sessionId: src_id|7cd922d-3840df4b-ded0e991-8d37bea6_0
2025-10-09T21:49:30.865134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1539 sessionId: src_id|7cd922d-3840df4b-ded0e991-8d37bea6_0 is DEAD
2025-10-09T21:49:30.865287Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cd922d-3840df4b-ded0e991-8d37bea6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:30.865297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cd922d-3840df4b-ded0e991-8d37bea6_0] Write session will restart in 0.008482s
2025-10-09T21:49:30.865317Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cd922d-3840df4b-ded0e991-8d37bea6_0] Write session: Do CDS request
2025-10-09T21:49:30.865323Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cd922d-3840df4b-ded0e991-8d37bea6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:30.865446Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:30.874251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:30.874647Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7cd922d-3840df4b-ded0e991-8d37bea6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:30.874651Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:30.874667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1540
2025-10-09T21:49:30.874867Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1540 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:30.874903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1540 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:30.874913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1540 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:30.874923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:30.875414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1540 sessionId:  describe result for acl check
2025-10-09T21:49:30.875462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:30.875474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:30.875476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:30.875484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342458836693269:8302] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:30.876032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342458836693269:8302] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:30.876462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:30.876656Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:30.876667Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:30.876681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:30.876746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:30.876777Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342458474866567:5137]
2025-10-09T21:49:30.877305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:30.877590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:30.877657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:30.877930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:30.877979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.877983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.877984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.877985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.877985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.877986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.877987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.877987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.877988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.877992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.877999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.878009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.878012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.878014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.878016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.878018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.878020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.878022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.878037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.878042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:30.878047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.878050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:30.878055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.878058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:30.878061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.878064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:30.878067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.878070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:30.878072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.878076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:30.878079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.878082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.878084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.878087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.878090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.878093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.878114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:30.878716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342458836693269:8302] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:30.884209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342458836693269:8302] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:30.884226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342458836693269:8302] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:30.884231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342458836693269:8302] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:30.884241Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1540 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:30.884557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:30.885606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:30.885620Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:30.885622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:30.885626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:30.885633Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1539 [2:7559342458474866567:5137]
2025-10-09T21:49:30.885644Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:30.885660Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:30.885662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:30.885664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:30.885666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:30.885681Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:30.885687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:30.885694Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:30.885740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.885746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.885747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.885749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:30.885818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:30.885821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.885829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:30.885840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:30.885851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:30.885869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342458836693180:2432], now have 1 active actors on pipe
2025-10-09T21:49:30.885875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342458836693308:8302], now have 1 active actors on pipe
2025-10-09T21:49:30.885881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342458474866561:5137]
2025-10-09T21:49:30.885883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342458474866561:5137]
2025-10-09T21:49:30.885885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342458474866561:5137]
2025-10-09T21:49:30.885894Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:30.885906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:30.885910Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:30.885914Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:30.885916Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:30.885920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:30.885923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:30.885942Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1539
2025-10-09T21:49:30.885961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342458474866561:5137]
2025-10-09T21:49:30.885942Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1539
2025-10-09T21:49:30.885969Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:30.885993Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:30.886005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.886010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.886055Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fbe4fbe3-16c2621b-ba4e820a-38b366ea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:30.886068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:30.886072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:30.886074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:30.886084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:30.886095Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:30.886110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:30.886293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:30.886303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:30.886323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:30.886401Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1540 partition: 0 MaxSeqNo: 25 sessionId: src_id|fbe4fbe3-16c2621b-ba4e820a-38b366ea_0
2025-10-09T21:49:30.886754Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cd922d-3840df4b-ded0e991-8d37bea6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fbe4fbe3-16c2621b-ba4e820a-38b366ea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:30.889179Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fbe4fbe3-16c2621b-ba4e820a-38b366ea_0] Write session: try to update token
2025-10-09T21:49:30.889198Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fbe4fbe3-16c2621b-ba4e820a-38b366ea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:30.900961Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:30.900980Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:30.900983Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:30.900985Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:30.900995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342458474866561:5137]
2025-10-09T21:49:30.901001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342458474866561:5137]
2025-10-09T21:49:30.901005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342458474866561:5137]
2025-10-09T21:49:30.901051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:30.901061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:30.901064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:30.901065Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:30.901073Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:30.901085Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:30.901204Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:30.903304Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1540 sessionId: src_id|fbe4fbe3-16c2621b-ba4e820a-38b366ea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:30.905672Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:30.965009Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342458836693315:2585]
2025-10-09T21:49:30.965282Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 1539, pipe: [1:7559342458836693315:2585]
2025-10-09T21:49:31.058691Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:31.059219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.059235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.059280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342458836693315:2585], now have 1 active actors on pipe
2025-10-09T21:49:31.059466Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:31.059485Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342458836693315:2585]
2025-10-09T21:49:31.059521Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:31.059547Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:31.059606Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1540 reason: status is not ok: tablet will be restarted right now sessionId: src_id|fbe4fbe3-16c2621b-ba4e820a-38b366ea_0
2025-10-09T21:49:31.059765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1540 sessionId: src_id|fbe4fbe3-16c2621b-ba4e820a-38b366ea_0 is DEAD
2025-10-09T21:49:31.059925Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fbe4fbe3-16c2621b-ba4e820a-38b366ea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:31.059937Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fbe4fbe3-16c2621b-ba4e820a-38b366ea_0] Write session will restart in 0.008189s
2025-10-09T21:49:31.059973Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fbe4fbe3-16c2621b-ba4e820a-38b366ea_0] Write session: Do CDS request
2025-10-09T21:49:31.059980Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fbe4fbe3-16c2621b-ba4e820a-38b366ea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:31.069598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fbe4fbe3-16c2621b-ba4e820a-38b366ea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:31.069700Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:31.069714Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1541
2025-10-09T21:49:31.069854Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1541 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:31.069886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1541 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:31.069895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1541 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:31.069900Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:31.070354Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1541 sessionId:  describe result for acl check
2025-10-09T21:49:31.070391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:31.070398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:31.070400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:31.070406Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342463131660726:8276] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:31.070941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342463131660726:8276] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:31.072544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342463131660726:8276] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:31.072803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:31.073311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:31.073481Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:31.073494Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:31.073507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:31.073561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:31.073612Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342462769834018:5127]
2025-10-09T21:49:31.074244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:31.074590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:31.074638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:31.074975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:31.075029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.075036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.075037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.075039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.075040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.075041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.075042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.075044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.075045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.075051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.075062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.075066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.075071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.075075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.075080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.075084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.075088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.075092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.075098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.075107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.075113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.075119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.075123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.075128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.075133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.075138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.075142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.075148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.075153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.075158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.075166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.075176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.075181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.075186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.075196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.075201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.075232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:31.075248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342463131660726:8276] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:31.075261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342463131660726:8276] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:31.075264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342463131660726:8276] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:31.075273Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1541 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:31.081746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:31.083435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:31.083454Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:31.083455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:31.083459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:31.083466Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1540 [2:7559342462769834018:5127]
2025-10-09T21:49:31.083475Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:31.083492Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:31.083494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:31.083497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:31.083499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:31.083513Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:31.083522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:31.083528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:31.083589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.083597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.083598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.083600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:31.083665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:31.083672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.083680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.083687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.083698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:31.083720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342462769834012:5127]
2025-10-09T21:49:31.083722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342462769834012:5127]
2025-10-09T21:49:31.083724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342462769834012:5127]
2025-10-09T21:49:31.083737Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:31.083748Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:31.083750Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:31.083752Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:31.083754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342463131660627:2432], now have 1 active actors on pipe
2025-10-09T21:49:31.083758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342463131660763:8276], now have 1 active actors on pipe
2025-10-09T21:49:31.083767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:31.083780Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:31.083783Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:31.083827Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342462769834012:5127]
2025-10-09T21:49:31.083839Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:31.083845Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:31.083852Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1540
2025-10-09T21:49:31.083852Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1540
2025-10-09T21:49:31.083891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.083902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.083935Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4e7a586c-e2780692-50b8a6fe-e0b3a7fa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:31.083943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:31.083948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:31.083950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:31.083961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:31.083971Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:31.083988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:31.084177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.084187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.084203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:31.084289Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1541 partition: 0 MaxSeqNo: 25 sessionId: src_id|4e7a586c-e2780692-50b8a6fe-e0b3a7fa_0
2025-10-09T21:49:31.084645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fbe4fbe3-16c2621b-ba4e820a-38b366ea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4e7a586c-e2780692-50b8a6fe-e0b3a7fa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:31.087237Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e7a586c-e2780692-50b8a6fe-e0b3a7fa_0] Write session: try to update token
2025-10-09T21:49:31.087254Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e7a586c-e2780692-50b8a6fe-e0b3a7fa_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:31.098455Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:31.098479Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:31.098482Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:31.098484Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:31.098495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342462769834012:5127]
2025-10-09T21:49:31.098503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342462769834012:5127]
2025-10-09T21:49:31.098508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342462769834012:5127]
2025-10-09T21:49:31.098556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:31.098566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.098566Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:31.098569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:31.098572Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:31.098577Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:31.098712Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:31.101452Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1541 sessionId: src_id|4e7a586c-e2780692-50b8a6fe-e0b3a7fa_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:31.103959Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:31.159532Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342463131660774:2585]
2025-10-09T21:49:31.159798Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 1540, pipe: [1:7559342463131660774:2585]
2025-10-09T21:49:31.253953Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:31.254482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.254501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.254549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342463131660774:2585], now have 1 active actors on pipe
2025-10-09T21:49:31.254765Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:31.254777Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:31.254779Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:31.254785Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342463131660774:2585]
2025-10-09T21:49:31.254804Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:31.254851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1541 reason: error at write: Disconnected sessionId: src_id|4e7a586c-e2780692-50b8a6fe-e0b3a7fa_0
2025-10-09T21:49:31.255043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1541 sessionId: src_id|4e7a586c-e2780692-50b8a6fe-e0b3a7fa_0 is DEAD
2025-10-09T21:49:31.255193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e7a586c-e2780692-50b8a6fe-e0b3a7fa_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:31.255205Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e7a586c-e2780692-50b8a6fe-e0b3a7fa_0] Write session will restart in 0.009134s
2025-10-09T21:49:31.255235Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e7a586c-e2780692-50b8a6fe-e0b3a7fa_0] Write session: Do CDS request
2025-10-09T21:49:31.255242Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e7a586c-e2780692-50b8a6fe-e0b3a7fa_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:31.265095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:31.265570Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e7a586c-e2780692-50b8a6fe-e0b3a7fa_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:31.265689Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:31.265706Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1542
2025-10-09T21:49:31.265844Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1542 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:31.265876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1542 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:31.265887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1542 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:31.265892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:31.266329Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1542 sessionId:  describe result for acl check
2025-10-09T21:49:31.266365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:31.266366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:31.266367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:31.266371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342463131660876:8278] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:31.266808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342463131660876:8278] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:31.267387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:31.267579Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:31.267593Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:31.267606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:31.267664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:31.267706Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342462769834136:5086]
2025-10-09T21:49:31.268300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:31.268615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:31.268668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:31.268907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342463131660876:8278] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:31.268995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:31.269055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.269061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.269062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.269063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.269064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.269066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.269067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.269068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.269069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.269074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.269085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.269088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.269093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.269096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.269100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.269104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.269107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.269111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.269118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.269127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.269138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.269149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.269158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.269168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.269177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.269183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.269192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.269198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.269206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.269213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.269222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.269227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.269233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.269243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.269252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.269262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.269296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:31.274266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342463131660876:8278] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:31.274283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342463131660876:8278] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:31.274287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342463131660876:8278] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:31.274297Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1542 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:31.275697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:31.276852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:31.276874Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:31.276877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:31.276882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:31.276889Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1541 [2:7559342462769834136:5086]
2025-10-09T21:49:31.276903Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:31.276919Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:31.276921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:31.276923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:31.276925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:31.276939Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:31.276947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:31.276953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:31.277019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.277024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.277025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.277027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:31.277264Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1541
2025-10-09T21:49:31.277266Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1541
2025-10-09T21:49:31.277479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:31.277499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.277518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.277538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.277564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:31.277610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342463131660785:2432], now have 1 active actors on pipe
2025-10-09T21:49:31.277622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342463131660914:8278], now have 1 active actors on pipe
2025-10-09T21:49:31.277640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.277655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.277688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342462769834130:5086]
2025-10-09T21:49:31.277696Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342462769834130:5086]
2025-10-09T21:49:31.277698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342462769834130:5086]
2025-10-09T21:49:31.277716Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:31.277735Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:31.277740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:31.277744Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:31.277745Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:31.277757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:31.277760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:31.277782Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|458c19b9-eb87b786-37442272-423e0abd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:31.277796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:31.277807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:31.277814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:31.277826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:31.277843Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:31.277900Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342462769834130:5086]
2025-10-09T21:49:31.277916Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:31.277935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:31.277941Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:31.278182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.278198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.278217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:31.278318Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1542 partition: 0 MaxSeqNo: 25 sessionId: src_id|458c19b9-eb87b786-37442272-423e0abd_0
2025-10-09T21:49:31.278675Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e7a586c-e2780692-50b8a6fe-e0b3a7fa_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|458c19b9-eb87b786-37442272-423e0abd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:31.281161Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|458c19b9-eb87b786-37442272-423e0abd_0] Write session: try to update token
2025-10-09T21:49:31.281184Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|458c19b9-eb87b786-37442272-423e0abd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:31.292640Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:31.292659Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:31.292661Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:31.292664Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:31.292674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342462769834130:5086]
2025-10-09T21:49:31.292679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342462769834130:5086]
2025-10-09T21:49:31.292684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342462769834130:5086]
2025-10-09T21:49:31.292727Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:31.292731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:31.292732Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:31.292735Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:31.292741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.292743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:31.292896Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:31.294312Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1542 sessionId: src_id|458c19b9-eb87b786-37442272-423e0abd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:31.296694Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:31.355135Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342463131660923:2585]
2025-10-09T21:49:31.355484Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 1541, pipe: [1:7559342463131660923:2585]
2025-10-09T21:49:31.444380Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:31.444626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.444639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.444686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342463131660923:2585], now have 1 active actors on pipe
2025-10-09T21:49:31.444891Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:31.444897Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:31.444906Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:31.444910Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:31.444955Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342463131660923:2585]
2025-10-09T21:49:31.444981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1542 reason: error at write: Disconnected sessionId: src_id|458c19b9-eb87b786-37442272-423e0abd_0
2025-10-09T21:49:31.445177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1542 sessionId: src_id|458c19b9-eb87b786-37442272-423e0abd_0 is DEAD
2025-10-09T21:49:31.445334Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|458c19b9-eb87b786-37442272-423e0abd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:31.445346Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|458c19b9-eb87b786-37442272-423e0abd_0] Write session will restart in 0.009104s
2025-10-09T21:49:31.445374Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|458c19b9-eb87b786-37442272-423e0abd_0] Write session: Do CDS request
2025-10-09T21:49:31.445380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|458c19b9-eb87b786-37442272-423e0abd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:31.455345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:31.455549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|458c19b9-eb87b786-37442272-423e0abd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:31.455615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:31.455631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1543
2025-10-09T21:49:31.455771Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1543 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:31.455804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1543 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:31.455810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1543 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:31.455818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:31.456292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1543 sessionId:  describe result for acl check
2025-10-09T21:49:31.456332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:31.456340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:31.456341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:31.456348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342463131661018:8273] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:31.456913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342463131661018:8273] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:31.458072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:31.458276Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:31.458293Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:31.458303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:31.458354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:31.458401Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342462769834256:5095]
2025-10-09T21:49:31.458848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:31.459007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342463131661018:8273] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:31.459126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:31.459167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:31.459396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:31.459434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.459438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.459439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.459440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.459441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.459441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.459442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.459443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.459443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.459447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.459453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.459455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.459457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.459459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.459461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.459463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.459465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.459468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.459473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.459478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.459482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.459486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.459488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.459491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.459494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.459497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.459501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.459504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.459507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.459510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.459513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.459517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.459519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.459522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.459525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.459528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.459544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:31.464686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342463131661018:8273] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:31.464705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342463131661018:8273] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:31.464709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342463131661018:8273] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:31.464720Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1543 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:31.465965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:31.467433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:31.467447Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:31.467448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:31.467451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:31.467456Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1542 [2:7559342462769834256:5095]
2025-10-09T21:49:31.467465Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:31.467479Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:31.467482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:31.467484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:31.467485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:31.467498Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:31.467502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:31.467509Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:31.467566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.467573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.467575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.467576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:31.467632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:31.467633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.467640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.467656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.467679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:31.467703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342463131661057:8273], now have 1 active actors on pipe
2025-10-09T21:49:31.467709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342463131660926:2432], now have 1 active actors on pipe
2025-10-09T21:49:31.467717Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342462769834250:5095]
2025-10-09T21:49:31.467718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342462769834250:5095]
2025-10-09T21:49:31.467719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342462769834250:5095]
2025-10-09T21:49:31.467730Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:31.467746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:31.467752Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:31.467754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:31.467756Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:31.467756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:31.467758Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:31.467784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342462769834250:5095]
2025-10-09T21:49:31.467792Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:31.467802Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:31.467782Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1542
2025-10-09T21:49:31.467783Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1542
2025-10-09T21:49:31.467842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.467850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.467874Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8e061a02-5e559964-3dcc314-9a10c0e9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:31.467880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:31.467886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:31.467888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:31.467895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:31.467903Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:31.467919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:31.468099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.468112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.468144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:31.468241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1543 partition: 0 MaxSeqNo: 25 sessionId: src_id|8e061a02-5e559964-3dcc314-9a10c0e9_0
2025-10-09T21:49:31.468587Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|458c19b9-eb87b786-37442272-423e0abd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8e061a02-5e559964-3dcc314-9a10c0e9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:31.471128Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e061a02-5e559964-3dcc314-9a10c0e9_0] Write session: try to update token
2025-10-09T21:49:31.471149Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e061a02-5e559964-3dcc314-9a10c0e9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:31.484223Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1543 sessionId: src_id|8e061a02-5e559964-3dcc314-9a10c0e9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:31.485249Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:31.485268Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:31.485279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342462769834250:5095]
2025-10-09T21:49:31.485331Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:31.485340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:31.485521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:31.485727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:31.485869Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:31.486062Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:31.486287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:31.486336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:31.486722Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:31.486900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.486914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.486943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:31.487026Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:31.493605Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:31.493624Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:31.493627Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:31.493629Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:31.493643Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342462769834250:5095]
2025-10-09T21:49:31.493648Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342462769834250:5095]
2025-10-09T21:49:31.493653Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342462769834250:5095]
2025-10-09T21:49:31.493701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:31.493701Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:31.493706Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:31.493709Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:31.493710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.493713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:31.493900Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:31.545094Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342463131661065:2585]
2025-10-09T21:49:31.545357Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 1542, pipe: [1:7559342463131661065:2585]
2025-10-09T21:49:31.641007Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:31.648221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.648246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.648397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.648450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.648508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.648568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.648610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.648618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.648678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.648730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.648784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.648847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.648887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.648892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.648944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.648997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.649054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.649111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.649149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.649155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.649212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.649266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.649333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.649390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.649429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.649437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.649493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.649546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.649616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.649673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.649724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.649730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.649783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.649841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.649898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.650010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.650082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.650090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.650150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.650207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.650263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.650319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.650367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.650375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.650430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.650488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.650539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.650597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.650641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.650648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.650701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.650751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.650804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.650862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.650905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.650911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.650966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.651015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.651065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.651125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.651166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.651172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.651212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342463131661065:2585], now have 1 active actors on pipe
2025-10-09T21:49:31.651504Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342463131661065:2585]
2025-10-09T21:49:31.651521Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:31.651538Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:31.651543Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:31.651574Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1543 reason: error at write: Disconnected sessionId: src_id|8e061a02-5e559964-3dcc314-9a10c0e9_0
2025-10-09T21:49:31.651767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1543 sessionId: src_id|8e061a02-5e559964-3dcc314-9a10c0e9_0 is DEAD
2025-10-09T21:49:31.651915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e061a02-5e559964-3dcc314-9a10c0e9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:31.651926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e061a02-5e559964-3dcc314-9a10c0e9_0] Write session will restart in 0.005901s
2025-10-09T21:49:31.651951Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e061a02-5e559964-3dcc314-9a10c0e9_0] Write session: Do CDS request
2025-10-09T21:49:31.651957Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e061a02-5e559964-3dcc314-9a10c0e9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:31.651977Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:31.658530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e061a02-5e559964-3dcc314-9a10c0e9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:31.658635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:31.658647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1544
2025-10-09T21:49:31.658778Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1544 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:31.658811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1544 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:31.658823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1544 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:31.658830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:31.659274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1544 sessionId:  describe result for acl check
2025-10-09T21:49:31.659310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:31.659317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:31.659319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:31.659325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342463131661105:8270] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:31.659795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342463131661105:8270] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:31.662406Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342463131661105:8270] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:31.662773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:31.663330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:31.663512Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:31.663522Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:31.663534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:31.663584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:31.663624Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342462769834376:5090]
2025-10-09T21:49:31.664206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:31.665253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:31.665301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:31.665584Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342463131661105:8270] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:31.665599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342463131661105:8270] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:31.665603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342463131661105:8270] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:31.665612Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1544 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:31.665676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:31.665727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.665733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.665735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.665736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.665737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.665738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.665739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.665741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.665742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.665747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.665758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.665762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.665766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.665770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.665774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.665778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.665781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.665785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.665792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.665800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.665806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.665812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.665822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.665827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.665831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.665835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.665840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.665845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.665849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.665854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.665859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.665866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.665870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.665876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.665887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.665894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.665924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:31.672235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:31.673613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:31.673627Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:31.673630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:31.673635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:31.673642Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1543 [2:7559342462769834376:5090]
2025-10-09T21:49:31.673655Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:31.673676Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:31.673679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:31.673682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:31.673685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:31.673703Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:31.673715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:31.673722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:31.673780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.673787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.673790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.673792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:31.673869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:31.673877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.673889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.673900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.673920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:31.673948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342463131661197:8270], now have 1 active actors on pipe
2025-10-09T21:49:31.673956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342463131661067:2432], now have 1 active actors on pipe
2025-10-09T21:49:31.673965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342462769834370:5090]
2025-10-09T21:49:31.673967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342462769834370:5090]
2025-10-09T21:49:31.673969Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342462769834370:5090]
2025-10-09T21:49:31.673982Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:31.674000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:31.674002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:31.674006Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:31.674008Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:31.674019Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:31.674022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:31.674072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342462769834370:5090]
2025-10-09T21:49:31.674086Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:31.674067Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1543
2025-10-09T21:49:31.674066Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1543
2025-10-09T21:49:31.674102Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:31.674127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.674136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.674165Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|db4a7bed-dace37f9-860af054-f5709bbc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:31.674177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:31.674181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:31.674184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:31.674194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:31.674203Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:31.674220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:31.674397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.674407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.674422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:31.674497Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1544 partition: 0 MaxSeqNo: 25 sessionId: src_id|db4a7bed-dace37f9-860af054-f5709bbc_0
2025-10-09T21:49:31.674835Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e061a02-5e559964-3dcc314-9a10c0e9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|db4a7bed-dace37f9-860af054-f5709bbc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:31.677212Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db4a7bed-dace37f9-860af054-f5709bbc_0] Write session: try to update token
2025-10-09T21:49:31.677230Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db4a7bed-dace37f9-860af054-f5709bbc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:31.684935Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:31.684955Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:31.684965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342462769834370:5090]
2025-10-09T21:49:31.685013Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:31.685020Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:31.685212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:31.685356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:31.685496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:31.685647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:31.685837Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:31.685884Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:31.689554Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1544 sessionId: src_id|db4a7bed-dace37f9-860af054-f5709bbc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:31.691867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:31.692075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.692098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.692139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:31.692221Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:31.697555Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:31.697572Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:31.697591Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:31.697593Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:31.697604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342462769834370:5090]
2025-10-09T21:49:31.697608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342462769834370:5090]
2025-10-09T21:49:31.697611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342462769834370:5090]
2025-10-09T21:49:31.697656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:31.697657Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:31.697663Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:31.697664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.697667Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:31.697667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:31.697831Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:31.751952Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342463131661210:2585]
2025-10-09T21:49:31.752266Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 1543, pipe: [1:7559342463131661210:2585]
2025-10-09T21:49:31.848757Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:31.855772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.855798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.855954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.856012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.856071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.856135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.856178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.856188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.856248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.856305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.856362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.856423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.856463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.856469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.856525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.856583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.856637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.856692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.856732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.856738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.856798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.856857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.856914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.856974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.857016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.857021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.857084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.857138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.857195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.857250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.857292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.857297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.857353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.857408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.857463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.857519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.857559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.857567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.857637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.857688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.857742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.857797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.857838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.857844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.857963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.858104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.858169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.858223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.858262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.858268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.858324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.858377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.858433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.858501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.858550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.858556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.858617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:31.858670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:31.858729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:31.858787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:31.858827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:31.858833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:31.858879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342463131661210:2585], now have 1 active actors on pipe
2025-10-09T21:49:31.859127Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:31.859190Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:31.859197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:31.859230Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1544 reason: error at write: Disconnected sessionId: src_id|db4a7bed-dace37f9-860af054-f5709bbc_0
2025-10-09T21:49:31.859413Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342463131661210:2585]
2025-10-09T21:49:31.859446Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1544 sessionId: src_id|db4a7bed-dace37f9-860af054-f5709bbc_0 is DEAD
2025-10-09T21:49:31.859571Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db4a7bed-dace37f9-860af054-f5709bbc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:31.859583Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db4a7bed-dace37f9-860af054-f5709bbc_0] Write session will restart in 0.007884s
2025-10-09T21:49:31.859608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db4a7bed-dace37f9-860af054-f5709bbc_0] Write session: Do CDS request
2025-10-09T21:49:31.859615Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db4a7bed-dace37f9-860af054-f5709bbc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:31.859922Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:31.868598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db4a7bed-dace37f9-860af054-f5709bbc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:31.868667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:31.868681Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1545
2025-10-09T21:49:31.868857Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1545 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:31.868885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1545 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:31.868893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1545 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:31.868896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:31.869371Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1545 sessionId:  describe result for acl check
2025-10-09T21:49:31.869411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:31.869420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:31.869422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:31.869428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342463131661307:8261] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:31.870042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342463131661307:8261] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:31.870088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:31.872250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:31.872455Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:31.872464Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:31.872476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:31.872532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:31.872576Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342462769834502:5085]
2025-10-09T21:49:31.872800Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342463131661307:8261] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:31.873169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:31.873512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:31.873601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:31.873954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:31.873997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.874002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.874002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.874003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.874004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.874005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.874006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.874006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.874007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.874011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.874018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.874021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.874023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.874025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.874041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.874044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.874046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.874048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.874053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.874058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:31.874063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.874066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:31.874069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.874072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:31.874075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.874078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:31.874081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.874084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:31.874087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.874090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:31.874093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.874096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.874100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.874103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.874106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.874110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.874127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:31.879414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342463131661307:8261] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:31.879435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342463131661307:8261] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:31.879439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342463131661307:8261] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:31.879451Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1545 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:31.880683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:31.881851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:31.881868Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:31.881871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:31.881875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:31.881883Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1544 [2:7559342462769834502:5085]
2025-10-09T21:49:31.881898Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:31.881919Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:31.881922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:31.881926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:31.881928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:31.881947Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:31.881954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:31.881961Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:31.882027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.882052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.882054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.882057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:31.882133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:31.882143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.882156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:31.882170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:31.882188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:31.882211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342463131661215:2432], now have 1 active actors on pipe
2025-10-09T21:49:31.882220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342463131661348:8261], now have 1 active actors on pipe
2025-10-09T21:49:31.882229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342462769834496:5085]
2025-10-09T21:49:31.882232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342462769834496:5085]
2025-10-09T21:49:31.882234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342462769834496:5085]
2025-10-09T21:49:31.882246Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:31.882258Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:31.882264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:31.882266Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:31.882267Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:31.882279Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:31.882283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:31.882330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342462769834496:5085]
2025-10-09T21:49:31.882322Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1544
2025-10-09T21:49:31.882341Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:31.882334Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1544
2025-10-09T21:49:31.882344Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:31.882374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.882379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.882414Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e229d56d-72a18e6a-813804ba-2f8f207c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:31.882426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:31.882435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:31.882438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:31.882450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:31.882466Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:31.882489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:31.882615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:31.882623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:31.882637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:31.882722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1545 partition: 0 MaxSeqNo: 25 sessionId: src_id|e229d56d-72a18e6a-813804ba-2f8f207c_0
2025-10-09T21:49:31.883097Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db4a7bed-dace37f9-860af054-f5709bbc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e229d56d-72a18e6a-813804ba-2f8f207c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:31.885540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e229d56d-72a18e6a-813804ba-2f8f207c_0] Write session: try to update token
2025-10-09T21:49:31.885560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e229d56d-72a18e6a-813804ba-2f8f207c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:31.896864Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:31.896886Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:31.896888Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:31.896891Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:31.896900Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342462769834496:5085]
2025-10-09T21:49:31.896905Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342462769834496:5085]
2025-10-09T21:49:31.896912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342462769834496:5085]
2025-10-09T21:49:31.896959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:31.896966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:31.896969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:31.896975Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:31.896982Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:31.896988Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:31.897106Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:31.898906Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1545 sessionId: src_id|e229d56d-72a18e6a-813804ba-2f8f207c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:31.901325Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:31.959773Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342463131661355:2585]
2025-10-09T21:49:31.960063Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 1544, pipe: [1:7559342463131661355:2585]
2025-10-09T21:49:32.051527Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:32.052554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.052569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.052612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342463131661355:2585], now have 1 active actors on pipe
2025-10-09T21:49:32.052832Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342463131661355:2585]
2025-10-09T21:49:32.052851Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:32.052870Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:32.052874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:32.052938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1545 reason: error at write: Disconnected sessionId: src_id|e229d56d-72a18e6a-813804ba-2f8f207c_0
2025-10-09T21:49:32.053125Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1545 sessionId: src_id|e229d56d-72a18e6a-813804ba-2f8f207c_0 is DEAD
2025-10-09T21:49:32.053271Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e229d56d-72a18e6a-813804ba-2f8f207c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:32.053282Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e229d56d-72a18e6a-813804ba-2f8f207c_0] Write session will restart in 0.006966s
2025-10-09T21:49:32.053306Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e229d56d-72a18e6a-813804ba-2f8f207c_0] Write session: Do CDS request
2025-10-09T21:49:32.053329Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e229d56d-72a18e6a-813804ba-2f8f207c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:32.053820Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:32.061547Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e229d56d-72a18e6a-813804ba-2f8f207c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:32.061647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:32.061662Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1546
2025-10-09T21:49:32.061775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1546 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:32.061808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1546 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:32.061818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1546 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:32.061826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:32.062288Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1546 sessionId:  describe result for acl check
2025-10-09T21:49:32.062338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:32.062344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:32.062346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:32.062353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342467426628746:8253] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:32.062904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342467426628746:8253] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:32.063204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:32.065182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:32.065233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342467426628746:8253] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:32.065371Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:32.065380Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:32.065389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:32.065438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:32.065481Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342467064801920:5071]
2025-10-09T21:49:32.066020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:32.066309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:32.066344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:32.066575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:32.066625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.066631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.066632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.066634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.066635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.066636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.066637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.066638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.066640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.066645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.066656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.066661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.066664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.066666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.066668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.066670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.066673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.066675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.066680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.066687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.066692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.066699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.066706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.066716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.066720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.066725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.066729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.066733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.066742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.066747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.066757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.066763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.066772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.066778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.066782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.066785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.066802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:32.072493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342467426628746:8253] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:32.072508Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342467426628746:8253] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:32.072512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342467426628746:8253] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:32.072524Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1546 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:32.073279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:32.074756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:32.074776Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:32.074779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:32.074784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:32.074790Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1545 [2:7559342467064801920:5071]
2025-10-09T21:49:32.074803Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:32.074822Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:32.074830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:32.074833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:32.074836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:32.074853Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:32.074864Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:32.074871Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:32.074932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.074941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.074943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.074945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:32.075018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:32.075026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.075038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.075050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.075072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:32.075097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342467064801914:5071]
2025-10-09T21:49:32.075105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342467064801914:5071]
2025-10-09T21:49:32.075107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342467064801914:5071]
2025-10-09T21:49:32.075116Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:32.075125Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:32.075128Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:32.075130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:32.075130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:32.075145Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:32.075148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:32.075161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342467426628791:8253], now have 1 active actors on pipe
2025-10-09T21:49:32.075165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342467426628654:2432], now have 1 active actors on pipe
2025-10-09T21:49:32.075194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342467064801914:5071]
2025-10-09T21:49:32.075204Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:32.075214Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:32.075218Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1545
2025-10-09T21:49:32.075253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.075250Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1545
2025-10-09T21:49:32.075257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.075295Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c0eb97e2-a1ec1591-49d061f8-774ec44b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:32.075307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:32.075315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:32.075317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:32.075327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:32.075340Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:32.075364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:32.075514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.075524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.075540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:32.075617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1546 partition: 0 MaxSeqNo: 25 sessionId: src_id|c0eb97e2-a1ec1591-49d061f8-774ec44b_0
2025-10-09T21:49:32.075964Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e229d56d-72a18e6a-813804ba-2f8f207c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c0eb97e2-a1ec1591-49d061f8-774ec44b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:32.078395Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c0eb97e2-a1ec1591-49d061f8-774ec44b_0] Write session: try to update token
2025-10-09T21:49:32.078414Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c0eb97e2-a1ec1591-49d061f8-774ec44b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:32.087349Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:32.087375Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:32.087385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342467064801914:5071]
2025-10-09T21:49:32.087437Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:32.087441Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:32.087611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:32.087781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:32.087930Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:32.088066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:32.088397Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:32.088433Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:32.092221Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1546 sessionId: src_id|c0eb97e2-a1ec1591-49d061f8-774ec44b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:32.094763Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:32.095044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.095061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.095096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:32.095178Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:32.099708Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:32.099726Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:32.099728Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:32.099730Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:32.099739Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342467064801914:5071]
2025-10-09T21:49:32.099742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342467064801914:5071]
2025-10-09T21:49:32.099745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342467064801914:5071]
2025-10-09T21:49:32.099794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:32.099795Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:32.099800Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:32.099804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.099805Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:32.099806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:32.099957Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:32.152839Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342467426628819:2585]
2025-10-09T21:49:32.153125Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 1545, pipe: [1:7559342467426628819:2585]
2025-10-09T21:49:32.248978Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:32.255696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.255723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.255865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.255920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.255987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.256054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.256377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.256388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.256450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.256503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.256558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.256615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.256774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.256781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.256834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.256892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.256944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.257010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.257063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.257065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.257147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.257235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.257293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.257355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.257401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.257408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.257476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.257529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.257597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.257649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.257689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.257694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.257751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.257809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.257878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.257933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.257971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.257977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.258044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.258093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.258151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.258220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.258258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.258266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.258317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.258364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.258417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.258470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.258520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.258527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.258580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.258629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.258687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.258791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.258831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.258836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.258902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.258951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.259008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.259066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.259106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.259111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.259153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342467426628819:2585], now have 1 active actors on pipe
2025-10-09T21:49:32.259442Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:32.259455Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:32.259488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1546 reason: error at write: Disconnected sessionId: src_id|c0eb97e2-a1ec1591-49d061f8-774ec44b_0
2025-10-09T21:49:32.259532Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:32.259544Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:32.259584Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342467426628819:2585]
2025-10-09T21:49:32.259690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1546 sessionId: src_id|c0eb97e2-a1ec1591-49d061f8-774ec44b_0 is DEAD
2025-10-09T21:49:32.259846Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0eb97e2-a1ec1591-49d061f8-774ec44b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:32.259857Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0eb97e2-a1ec1591-49d061f8-774ec44b_0] Write session will restart in 0.009361s
2025-10-09T21:49:32.259875Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0eb97e2-a1ec1591-49d061f8-774ec44b_0] Write session: Do CDS request
2025-10-09T21:49:32.259880Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0eb97e2-a1ec1591-49d061f8-774ec44b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:32.269964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:32.270481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c0eb97e2-a1ec1591-49d061f8-774ec44b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:32.270581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:32.270598Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1547
2025-10-09T21:49:32.270750Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1547 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:32.270783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1547 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:32.270795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1547 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:32.270799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:32.271247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1547 sessionId:  describe result for acl check
2025-10-09T21:49:32.271285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:32.271292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:32.271294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:32.271301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342467426628927:8238] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:32.271842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342467426628927:8238] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:32.272039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:32.272234Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:32.272247Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:32.272261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:32.272327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:32.272367Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342467064802081:5048]
2025-10-09T21:49:32.272903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:32.273172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:32.273220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:32.273534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:32.273613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.273619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.273620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.273621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.273622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.273624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.273625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.273626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.273627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.273633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.273643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.273647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.273650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.273653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.273657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.273661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.273664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.273668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.273675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.273683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.273690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.273695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.273699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.273705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.273709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.273713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.273718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.273727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.273732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.273737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.273746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.273751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.273756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.273762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.273767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.273774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.273803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:32.273995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342467426628927:8238] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:32.280059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342467426628927:8238] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:32.280071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342467426628927:8238] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:32.280074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342467426628927:8238] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:32.280082Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1547 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:32.280234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:32.281696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:32.281709Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:32.281711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:32.281714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:32.281720Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1546 [2:7559342467064802081:5048]
2025-10-09T21:49:32.281729Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:32.281744Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:32.281745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:32.281747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:32.281749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:32.281762Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:32.281765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:32.281771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:32.281818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.281825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.281826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.281827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:32.281881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:32.281887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.281895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.281905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.281923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:32.281947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342467426628830:2432], now have 1 active actors on pipe
2025-10-09T21:49:32.281955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342467426628964:8238], now have 1 active actors on pipe
2025-10-09T21:49:32.281962Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342467064802075:5048]
2025-10-09T21:49:32.281963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342467064802075:5048]
2025-10-09T21:49:32.281964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342467064802075:5048]
2025-10-09T21:49:32.281974Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:32.281989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:32.281997Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:32.282001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:32.282002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:32.282006Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:32.282009Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:32.281989Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1546
2025-10-09T21:49:32.282007Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1546
2025-10-09T21:49:32.282044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342467064802075:5048]
2025-10-09T21:49:32.282052Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:32.282060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:32.282068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.282077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.282136Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e1e55fbd-8e4d058d-ac099711-d0e248ca_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:32.282148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:32.282152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:32.282154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:32.282164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:32.282179Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:32.282203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:32.282325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.282333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.282345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:32.282429Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1547 partition: 0 MaxSeqNo: 25 sessionId: src_id|e1e55fbd-8e4d058d-ac099711-d0e248ca_0
2025-10-09T21:49:32.282794Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0eb97e2-a1ec1591-49d061f8-774ec44b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e1e55fbd-8e4d058d-ac099711-d0e248ca_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:32.285103Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1e55fbd-8e4d058d-ac099711-d0e248ca_0] Write session: try to update token
2025-10-09T21:49:32.285118Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1e55fbd-8e4d058d-ac099711-d0e248ca_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:32.296487Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:32.296507Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:32.296510Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:32.296512Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:32.296522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342467064802075:5048]
2025-10-09T21:49:32.296530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342467064802075:5048]
2025-10-09T21:49:32.296534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342467064802075:5048]
2025-10-09T21:49:32.296576Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:32.296580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:32.296587Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:32.296587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.296590Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:32.296590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:32.296730Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:32.297491Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1547 sessionId: src_id|e1e55fbd-8e4d058d-ac099711-d0e248ca_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:32.300046Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:32.359685Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342467426628973:2585]
2025-10-09T21:49:32.359985Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 1546, pipe: [1:7559342467426628973:2585]
2025-10-09T21:49:32.446376Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:32.446741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.446754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.446791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342467426628973:2585], now have 1 active actors on pipe
2025-10-09T21:49:32.446938Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:32.447009Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342467426628973:2585]
2025-10-09T21:49:32.447016Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:32.447039Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:32.447116Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1547 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e1e55fbd-8e4d058d-ac099711-d0e248ca_0
2025-10-09T21:49:32.447288Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1547 sessionId: src_id|e1e55fbd-8e4d058d-ac099711-d0e248ca_0 is DEAD
2025-10-09T21:49:32.447403Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1e55fbd-8e4d058d-ac099711-d0e248ca_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:32.447413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1e55fbd-8e4d058d-ac099711-d0e248ca_0] Write session will restart in 0.007941s
2025-10-09T21:49:32.447434Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1e55fbd-8e4d058d-ac099711-d0e248ca_0] Write session: Do CDS request
2025-10-09T21:49:32.447439Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1e55fbd-8e4d058d-ac099711-d0e248ca_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:32.456566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1e55fbd-8e4d058d-ac099711-d0e248ca_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:32.456650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:32.456667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1548
2025-10-09T21:49:32.456692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:32.456814Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1548 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:32.456849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1548 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:32.456855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1548 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:32.456864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:32.457305Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1548 sessionId:  describe result for acl check
2025-10-09T21:49:32.457347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:32.457353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:32.457355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:32.457361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342467426629072:8233] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:32.457958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342467426629072:8233] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:32.458827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:32.459017Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:32.459026Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:32.459039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:32.459098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:32.459140Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342467064802205:5046]
2025-10-09T21:49:32.459721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:32.459844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342467426629072:8233] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:32.460071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:32.460139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:32.460456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:32.460521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.460528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.460529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.460530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.460531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.460531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.460532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.460533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.460534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.460539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.460550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.460554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.460558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.460562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.460566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.460569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.460573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.460577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.460583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.460591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.460597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.460603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.460608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.460613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.460617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.460622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.460627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.460632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.460636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.460641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.460648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.460654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.460660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.460667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.460672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.460681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.460711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:32.465476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342467426629072:8233] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:32.465490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342467426629072:8233] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:32.465494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342467426629072:8233] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:32.465503Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1548 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:32.466798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:32.467819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:32.467834Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:32.467837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:32.467840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:32.467850Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1547 [2:7559342467064802205:5046]
2025-10-09T21:49:32.467869Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:32.467890Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:32.467895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:32.467897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:32.467899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:32.467912Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:32.467919Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:32.467924Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:32.467971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.467976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.467978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.467979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:32.468052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:32.468059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.468065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.468072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.468081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:32.468104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342467426628975:2432], now have 1 active actors on pipe
2025-10-09T21:49:32.468112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342467426629111:8233], now have 1 active actors on pipe
2025-10-09T21:49:32.468119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342467064802199:5046]
2025-10-09T21:49:32.468126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342467064802199:5046]
2025-10-09T21:49:32.468128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342467064802199:5046]
2025-10-09T21:49:32.468141Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:32.468161Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:32.468173Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:32.468178Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:32.468182Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:32.468185Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:32.468187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:32.468179Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1547
2025-10-09T21:49:32.468213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342467064802199:5046]
2025-10-09T21:49:32.468222Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:32.468184Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1547
2025-10-09T21:49:32.468224Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:32.468248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.468257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.468291Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|62558d05-626438ff-2ca4c3cc-97f47580_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:32.468302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:32.468306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:32.468308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:32.468319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:32.468333Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:32.468351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:32.468494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.468503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.468518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:32.468636Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1548 partition: 0 MaxSeqNo: 25 sessionId: src_id|62558d05-626438ff-2ca4c3cc-97f47580_0
2025-10-09T21:49:32.468956Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1e55fbd-8e4d058d-ac099711-d0e248ca_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|62558d05-626438ff-2ca4c3cc-97f47580_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:32.471182Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62558d05-626438ff-2ca4c3cc-97f47580_0] Write session: try to update token
2025-10-09T21:49:32.471198Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62558d05-626438ff-2ca4c3cc-97f47580_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:32.481670Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:32.481691Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:32.481694Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:32.481695Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:32.481704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342467064802199:5046]
2025-10-09T21:49:32.481709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342467064802199:5046]
2025-10-09T21:49:32.481713Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342467064802199:5046]
2025-10-09T21:49:32.481758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:32.481763Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:32.481769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.481770Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:32.481772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:32.481774Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:32.481908Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:32.483410Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1548 sessionId: src_id|62558d05-626438ff-2ca4c3cc-97f47580_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:32.485719Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:32.547067Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342467426629119:2585]
2025-10-09T21:49:32.547366Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 1547, pipe: [1:7559342467426629119:2585]
2025-10-09T21:49:32.629694Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:32.630300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.630313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.630351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342467426629119:2585], now have 1 active actors on pipe
2025-10-09T21:49:32.630542Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:32.630554Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:32.630563Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:32.630567Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:32.630604Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342467426629119:2585]
2025-10-09T21:49:32.630641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1548 reason: error at write: Disconnected sessionId: src_id|62558d05-626438ff-2ca4c3cc-97f47580_0
2025-10-09T21:49:32.630815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1548 sessionId: src_id|62558d05-626438ff-2ca4c3cc-97f47580_0 is DEAD
2025-10-09T21:49:32.630953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62558d05-626438ff-2ca4c3cc-97f47580_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:32.630962Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62558d05-626438ff-2ca4c3cc-97f47580_0] Write session will restart in 0.006666s
2025-10-09T21:49:32.630982Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62558d05-626438ff-2ca4c3cc-97f47580_0] Write session: Do CDS request
2025-10-09T21:49:32.630986Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62558d05-626438ff-2ca4c3cc-97f47580_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:32.638500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62558d05-626438ff-2ca4c3cc-97f47580_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:32.638588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:32.638601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1549
2025-10-09T21:49:32.638742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1549 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:32.638777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1549 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:32.638783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1549 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:32.638787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:32.639241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1549 sessionId:  describe result for acl check
2025-10-09T21:49:32.639277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:32.639283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:32.639285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:32.639291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342467426629216:8223] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:32.639723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342467426629216:8223] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:32.640701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:32.641746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342467426629216:8223] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:32.642709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:32.642872Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:32.642882Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:32.642893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:32.642944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:32.642980Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342467064802330:5045]
2025-10-09T21:49:32.643567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:32.643798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:32.643834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:32.643897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342467426629216:8223] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:32.643912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342467426629216:8223] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:32.643915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342467426629216:8223] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:32.643923Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1549 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:32.644143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:32.644192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.644198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.644199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.644200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.644202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.644203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.644204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.644205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.644206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.644211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.644222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.644226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.644230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.644234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.644238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.644242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.644246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.644250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.644255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.644263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.644269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.644275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.644280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.644285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.644290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.644295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.644300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.644305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.644309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.644314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.644320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.644325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.644329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.644334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.644338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.644343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.644372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:32.650592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:32.651865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:32.651878Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:32.651880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:32.651883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:32.651887Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1548 [2:7559342467064802330:5045]
2025-10-09T21:49:32.651896Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:32.651910Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:32.651912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:32.651914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:32.651915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:32.651928Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:32.651932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:32.651938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:32.651980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.651987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.651989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.651990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:32.652039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:32.652045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.652052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.652058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.652072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:32.652095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342467064802324:5045]
2025-10-09T21:49:32.652102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342467064802324:5045]
2025-10-09T21:49:32.652103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342467064802324:5045]
2025-10-09T21:49:32.652115Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:32.652131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342467426629121:2432], now have 1 active actors on pipe
2025-10-09T21:49:32.652135Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:32.652138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342467426629256:8223], now have 1 active actors on pipe
2025-10-09T21:49:32.652139Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:32.652142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:32.652145Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:32.652154Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:32.652157Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:32.652179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342467064802324:5045]
2025-10-09T21:49:32.652184Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:32.652198Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:32.652182Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1548
2025-10-09T21:49:32.652208Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1548
2025-10-09T21:49:32.652232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.652242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.652266Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dba2ba75-e8e1048b-d43c8807-ee18b35f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:32.652279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:32.652281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:32.652283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:32.652290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:32.652297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:32.652309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:32.652446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.652452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.652463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:32.652563Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1549 partition: 0 MaxSeqNo: 25 sessionId: src_id|dba2ba75-e8e1048b-d43c8807-ee18b35f_0
2025-10-09T21:49:32.652899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62558d05-626438ff-2ca4c3cc-97f47580_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dba2ba75-e8e1048b-d43c8807-ee18b35f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:32.655334Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dba2ba75-e8e1048b-d43c8807-ee18b35f_0] Write session: try to update token
2025-10-09T21:49:32.655351Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dba2ba75-e8e1048b-d43c8807-ee18b35f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:32.663724Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:32.663740Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:32.663750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342467064802324:5045]
2025-10-09T21:49:32.663798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:32.663809Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:32.663994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:32.664131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:32.664272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:32.664423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:32.664606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:32.664655Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:32.668018Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1549 sessionId: src_id|dba2ba75-e8e1048b-d43c8807-ee18b35f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:32.670514Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:32.670704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.670722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.670755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:32.670834Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:32.676394Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:32.676411Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:32.676413Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:32.676415Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:32.676423Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342467064802324:5045]
2025-10-09T21:49:32.676429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342467064802324:5045]
2025-10-09T21:49:32.676432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342467064802324:5045]
2025-10-09T21:49:32.676467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:32.676475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.676475Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:32.676478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:32.676481Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:32.676484Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:32.676645Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:32.730893Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342467426629266:2585]
2025-10-09T21:49:32.731167Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 1548, pipe: [1:7559342467426629266:2585]
2025-10-09T21:49:32.836375Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:32.844280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.844307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.844468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.844531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.844595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.844660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.844705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.844714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.844783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.844845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.844911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.844976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.845021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.845029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.845088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.845143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.845203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.845264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.845312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.845319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.845382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.845447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.845515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.845589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.845636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.845643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.845712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.845773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.845834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.845896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.845941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.845949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.846013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.846090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.846155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.846218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.846261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.846269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.846338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.846398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.846460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.846521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.846561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.846570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.846630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.846688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.846748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.846809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.846850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.846857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.846916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.846977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.847038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.847099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.847146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.847158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.847227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:32.847285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:32.847349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:32.847405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:32.847455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:32.847463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:32.847559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342467426629266:2585], now have 1 active actors on pipe
2025-10-09T21:49:32.847795Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:32.847813Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342467426629266:2585]
2025-10-09T21:49:32.847842Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:32.847845Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:32.847874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1549 reason: error at write: Disconnected sessionId: src_id|dba2ba75-e8e1048b-d43c8807-ee18b35f_0
2025-10-09T21:49:32.847948Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:32.848066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1549 sessionId: src_id|dba2ba75-e8e1048b-d43c8807-ee18b35f_0 is DEAD
2025-10-09T21:49:32.848224Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dba2ba75-e8e1048b-d43c8807-ee18b35f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:32.848232Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dba2ba75-e8e1048b-d43c8807-ee18b35f_0] Write session will restart in 0.007191s
2025-10-09T21:49:32.848258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dba2ba75-e8e1048b-d43c8807-ee18b35f_0] Write session: Do CDS request
2025-10-09T21:49:32.848265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dba2ba75-e8e1048b-d43c8807-ee18b35f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:32.856472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dba2ba75-e8e1048b-d43c8807-ee18b35f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:32.856587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:32.856603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1550
2025-10-09T21:49:32.856734Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1550 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:32.856767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1550 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:32.856790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1550 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:32.856795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:32.857231Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1550 sessionId:  describe result for acl check
2025-10-09T21:49:32.857266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:32.857273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:32.857275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:32.857281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342467426629355:8216] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:32.857862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342467426629355:8216] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:32.859257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:32.860202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342467426629355:8216] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:32.861838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:32.861995Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:32.862002Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:32.862011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:32.862068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:32.862098Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342467064802461:5036]
2025-10-09T21:49:32.862499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342467426629355:8216] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:32.862510Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342467426629355:8216] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:32.862513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342467426629355:8216] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:32.862521Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1550 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:32.862562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:32.862821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:32.862855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:32.863085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:32.863118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.863119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.863120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.863121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.863121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.863122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.863123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.863123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.863124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.863127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.863134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.863136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.863138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.863140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.863142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.863144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.863146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.863148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.863153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.863158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:32.863162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.863166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:32.863169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.863172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:32.863175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.863178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:32.863180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.863183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:32.863186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.863189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:32.863191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.863194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.863198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.863201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.863203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.863206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.863221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:32.869326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:32.870715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:32.870728Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:32.870729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:32.870732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:32.870736Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1549 [2:7559342467064802461:5036]
2025-10-09T21:49:32.870745Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:32.870759Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:32.870762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:32.870764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:32.870768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:32.870784Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:32.870790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:32.870797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:32.870856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.870865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.870867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.870869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:32.870928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:32.870933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.870942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:32.870953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:32.870967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:32.870993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342467426629405:8216], now have 1 active actors on pipe
2025-10-09T21:49:32.871000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342467426629271:2432], now have 1 active actors on pipe
2025-10-09T21:49:32.871008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342467064802455:5036]
2025-10-09T21:49:32.871010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342467064802455:5036]
2025-10-09T21:49:32.871012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342467064802455:5036]
2025-10-09T21:49:32.871025Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:32.871044Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:32.871047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:32.871051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:32.871053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:32.871061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:32.871034Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1549
2025-10-09T21:49:32.871064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:32.871038Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1549
2025-10-09T21:49:32.871089Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342467064802455:5036]
2025-10-09T21:49:32.871100Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:32.871118Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:32.871118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.871125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.871164Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a20c5eda-38dd51c9-6465aff3-7c18dab6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:32.871176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:32.871180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:32.871182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:32.871191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:32.871200Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:32.871219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:32.871350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:32.871355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:32.871367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:32.871469Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1550 partition: 0 MaxSeqNo: 25 sessionId: src_id|a20c5eda-38dd51c9-6465aff3-7c18dab6_0
2025-10-09T21:49:32.871766Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dba2ba75-e8e1048b-d43c8807-ee18b35f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a20c5eda-38dd51c9-6465aff3-7c18dab6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:32.874135Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a20c5eda-38dd51c9-6465aff3-7c18dab6_0] Write session: try to update token
2025-10-09T21:49:32.874153Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a20c5eda-38dd51c9-6465aff3-7c18dab6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:32.884543Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:32.884558Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:32.884560Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:32.884563Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:32.884572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342467064802455:5036]
2025-10-09T21:49:32.884576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342467064802455:5036]
2025-10-09T21:49:32.884580Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342467064802455:5036]
2025-10-09T21:49:32.884620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:32.884630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:32.884633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:32.884634Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:32.884640Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:32.884644Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:32.884757Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:32.886793Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1550 sessionId: src_id|a20c5eda-38dd51c9-6465aff3-7c18dab6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:32.889187Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:32.947965Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342467426629416:2585]
2025-10-09T21:49:32.948254Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 1549, pipe: [1:7559342467426629416:2585]
2025-10-09T21:49:33.038638Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:33.039144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:33.039159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:33.039196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342467426629416:2585], now have 1 active actors on pipe
2025-10-09T21:49:33.039406Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:33.039436Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342467426629416:2585]
2025-10-09T21:49:33.039438Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:33.039441Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:33.039462Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:33.039494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1550 reason: error at write: Disconnected sessionId: src_id|a20c5eda-38dd51c9-6465aff3-7c18dab6_0
2025-10-09T21:49:33.039690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1550 sessionId: src_id|a20c5eda-38dd51c9-6465aff3-7c18dab6_0 is DEAD
2025-10-09T21:49:33.039850Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a20c5eda-38dd51c9-6465aff3-7c18dab6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:33.039860Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a20c5eda-38dd51c9-6465aff3-7c18dab6_0] Write session will restart in 0.009799s
2025-10-09T21:49:33.039888Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a20c5eda-38dd51c9-6465aff3-7c18dab6_0] Write session: Do CDS request
2025-10-09T21:49:33.039894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a20c5eda-38dd51c9-6465aff3-7c18dab6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:33.049798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:33.050428Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a20c5eda-38dd51c9-6465aff3-7c18dab6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:33.050525Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:33.050535Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1551
2025-10-09T21:49:33.050628Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1551 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:33.050656Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1551 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:33.050665Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1551 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:33.050668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:33.051095Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1551 sessionId:  describe result for acl check
2025-10-09T21:49:33.051132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:33.051134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:33.051136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:33.051141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342471721596815:8199] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:33.051699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342471721596815:8199] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:33.052012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:33.052199Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:33.052209Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:33.052220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:33.052267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:33.052308Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342471359769884:5034]
2025-10-09T21:49:33.052886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:33.053116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:33.053156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:33.053456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:33.053503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.053509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.053510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.053511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.053512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.053513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.053514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.053516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.053516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.053521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.053531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.053535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.053539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.053543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.053547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.053551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.053554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.053558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.053564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.053580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.053587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.053592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.053597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.053603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.053607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.053612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.053616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.053621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.053630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.053635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.053640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.053646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.053650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.053655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.053660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.053670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.053700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:33.053796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342471721596815:8199] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:33.059491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342471721596815:8199] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:33.059504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342471721596815:8199] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:33.059508Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342471721596815:8199] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:33.059516Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1551 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:33.059866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:33.060873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:33.060887Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:33.060889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:33.060892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:33.060898Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1550 [2:7559342471359769884:5034]
2025-10-09T21:49:33.060913Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:33.060930Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:33.060936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:33.060938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:33.060939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:33.060952Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:33.060959Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:33.060964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:33.061005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.061011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.061012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.061013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:33.061227Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1550
2025-10-09T21:49:33.061241Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1550
2025-10-09T21:49:33.061326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:33.061336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.061344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.061351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.061368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:33.061389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721596854:8199], now have 1 active actors on pipe
2025-10-09T21:49:33.061398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721596715:2432], now have 1 active actors on pipe
2025-10-09T21:49:33.061413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:33.061423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:33.061444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342471359769878:5034]
2025-10-09T21:49:33.061452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342471359769878:5034]
2025-10-09T21:49:33.061455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342471359769878:5034]
2025-10-09T21:49:33.061472Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:33.061491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:33.061491Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:33.061495Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:33.061497Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:33.061510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:33.061513Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:33.061529Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|403a7d1f-102c39fb-4575604d-8e8270c2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:33.061535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:33.061542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:33.061543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:33.061551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:33.061559Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:33.061598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342471359769878:5034]
2025-10-09T21:49:33.061607Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:33.061618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:33.061619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:33.061778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:33.061784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:33.061795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:33.061876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1551 partition: 0 MaxSeqNo: 25 sessionId: src_id|403a7d1f-102c39fb-4575604d-8e8270c2_0
2025-10-09T21:49:33.062244Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a20c5eda-38dd51c9-6465aff3-7c18dab6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|403a7d1f-102c39fb-4575604d-8e8270c2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:33.064560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|403a7d1f-102c39fb-4575604d-8e8270c2_0] Write session: try to update token
2025-10-09T21:49:33.064576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|403a7d1f-102c39fb-4575604d-8e8270c2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:33.075674Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:33.075690Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:33.075692Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:33.075695Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:33.075703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342471359769878:5034]
2025-10-09T21:49:33.075706Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342471359769878:5034]
2025-10-09T21:49:33.075709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342471359769878:5034]
2025-10-09T21:49:33.075742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:33.075747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.075750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:33.075753Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:33.075767Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:33.075771Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:33.075878Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:33.077020Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1551 sessionId: src_id|403a7d1f-102c39fb-4575604d-8e8270c2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:33.079354Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:33.139435Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342471721596874:2585]
2025-10-09T21:49:33.139709Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 1550, pipe: [1:7559342471721596874:2585]
2025-10-09T21:49:33.223256Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:33.223723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:33.223738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:33.223782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721596874:2585], now have 1 active actors on pipe
2025-10-09T21:49:33.223960Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:33.223991Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:33.223999Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342471721596874:2585]
2025-10-09T21:49:33.224023Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:33.224098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1551 reason: status is not ok: tablet will be restarted right now sessionId: src_id|403a7d1f-102c39fb-4575604d-8e8270c2_0
2025-10-09T21:49:33.224284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1551 sessionId: src_id|403a7d1f-102c39fb-4575604d-8e8270c2_0 is DEAD
2025-10-09T21:49:33.224429Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|403a7d1f-102c39fb-4575604d-8e8270c2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:33.224438Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|403a7d1f-102c39fb-4575604d-8e8270c2_0] Write session will restart in 0.008952s
2025-10-09T21:49:33.224465Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|403a7d1f-102c39fb-4575604d-8e8270c2_0] Write session: Do CDS request
2025-10-09T21:49:33.224471Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|403a7d1f-102c39fb-4575604d-8e8270c2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:33.234579Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|403a7d1f-102c39fb-4575604d-8e8270c2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:33.234668Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:33.234685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1552
2025-10-09T21:49:33.234808Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1552 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:33.234849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1552 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:33.234855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1552 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:33.234860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:33.234847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:33.235297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1552 sessionId:  describe result for acl check
2025-10-09T21:49:33.235336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:33.235343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:33.235345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:33.235351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342471721596990:8189] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:33.235896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342471721596990:8189] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:33.237239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:33.237447Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:33.237462Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:33.237473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:33.237528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:33.237568Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342471359770051:5007]
2025-10-09T21:49:33.238167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342471721596990:8189] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:33.238184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:33.238516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:33.238573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:33.238896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:33.238950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.238957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.238958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.238959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.238960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.238962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.238963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.238964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.238965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.238971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.238983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.238987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.238991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.238995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.238999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.239003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.239007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.239011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.239018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.239027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.239033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.239038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.239043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.239048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.239053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.239064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.239069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.239074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.239081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.239086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.239095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.239101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.239110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.239117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.239126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.239131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.239164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:33.244696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342471721596990:8189] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:33.244712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342471721596990:8189] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:33.244716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342471721596990:8189] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:33.244727Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1552 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:33.245544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:33.246969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:33.246986Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:33.246988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:33.246993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:33.247001Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1551 [2:7559342471359770051:5007]
2025-10-09T21:49:33.247019Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:33.247041Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:33.247048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:33.247052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:33.247055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:33.247075Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:33.247082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:33.247096Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:33.247166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.247176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.247178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.247181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:33.247256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:33.247268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.247282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.247295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.247313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:33.247336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721597028:8189], now have 1 active actors on pipe
2025-10-09T21:49:33.247348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721596889:2432], now have 1 active actors on pipe
2025-10-09T21:49:33.247356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342471359770045:5007]
2025-10-09T21:49:33.247358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342471359770045:5007]
2025-10-09T21:49:33.247360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342471359770045:5007]
2025-10-09T21:49:33.247375Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:33.247391Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:33.247392Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:33.247396Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:33.247398Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:33.247408Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:33.247416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:33.247417Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1551
2025-10-09T21:49:33.247418Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1551
2025-10-09T21:49:33.247458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342471359770045:5007]
2025-10-09T21:49:33.247467Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:33.247469Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:33.247499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:33.247509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:33.247548Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|db4e4893-1adc2237-41774317-a3151af1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:33.247563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:33.247567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:33.247574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:33.247586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:33.247600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:33.247626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:33.247754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:33.247764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:33.247778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:33.247868Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1552 partition: 0 MaxSeqNo: 25 sessionId: src_id|db4e4893-1adc2237-41774317-a3151af1_0
2025-10-09T21:49:33.248240Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|403a7d1f-102c39fb-4575604d-8e8270c2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|db4e4893-1adc2237-41774317-a3151af1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:33.250714Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db4e4893-1adc2237-41774317-a3151af1_0] Write session: try to update token
2025-10-09T21:49:33.250731Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db4e4893-1adc2237-41774317-a3151af1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:33.261938Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:33.261957Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:33.261961Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:33.261963Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:33.261972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342471359770045:5007]
2025-10-09T21:49:33.261977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342471359770045:5007]
2025-10-09T21:49:33.261981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342471359770045:5007]
2025-10-09T21:49:33.262046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:33.262050Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:33.262056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.262057Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:33.262059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:33.262062Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:33.262209Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:33.263831Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1552 sessionId: src_id|db4e4893-1adc2237-41774317-a3151af1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:33.266322Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:33.324294Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342471721597041:2585]
2025-10-09T21:49:33.324589Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 1551, pipe: [1:7559342471721597041:2585]
2025-10-09T21:49:33.415610Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:33.416101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:33.416117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:33.416158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721597041:2585], now have 1 active actors on pipe
2025-10-09T21:49:33.416364Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:33.416380Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342471721597041:2585]
2025-10-09T21:49:33.416395Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:33.416399Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:33.416409Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:33.416460Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1552 reason: error at write: Disconnected sessionId: src_id|db4e4893-1adc2237-41774317-a3151af1_0
2025-10-09T21:49:33.416648Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1552 sessionId: src_id|db4e4893-1adc2237-41774317-a3151af1_0 is DEAD
2025-10-09T21:49:33.416812Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db4e4893-1adc2237-41774317-a3151af1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:33.416822Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db4e4893-1adc2237-41774317-a3151af1_0] Write session will restart in 0.006702s
2025-10-09T21:49:33.416842Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db4e4893-1adc2237-41774317-a3151af1_0] Write session: Do CDS request
2025-10-09T21:49:33.416847Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db4e4893-1adc2237-41774317-a3151af1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:33.424518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db4e4893-1adc2237-41774317-a3151af1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:33.424613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:33.424627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1553
2025-10-09T21:49:33.424749Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1553 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:33.424782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1553 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:33.424793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1553 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:33.424797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:33.425242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1553 sessionId:  describe result for acl check
2025-10-09T21:49:33.425280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:33.425287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:33.425289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:33.425295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342471721597114:8183] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:33.425882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342471721597114:8183] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:33.428006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:33.428112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342471721597114:8183] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:33.430529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:33.430716Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:33.430729Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:33.430741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:33.430794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:33.430794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342471721597114:8183] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:33.430805Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342471721597114:8183] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:33.430837Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342471359770180:5006]
2025-10-09T21:49:33.430809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342471721597114:8183] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:33.430817Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1553 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:33.431420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:33.431670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:33.431702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:33.431941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:33.431995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.432001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.432002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.432003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.432004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.432006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.432007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.432008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.432010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.432014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.432025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.432029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.432034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.432038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.432042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.432046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.432050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.432053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.432060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.432068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.432078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.432082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.432090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.432096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.432099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.432102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.432105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.432108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.432111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.432114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.432116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.432119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.432122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.432125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.432128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.432131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.432146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:33.438540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:33.439598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:33.439615Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:33.439618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:33.439622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:33.439628Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1552 [2:7559342471359770180:5006]
2025-10-09T21:49:33.439642Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:33.439660Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:33.439668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:33.439671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:33.439674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:33.439692Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:33.439702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:33.439713Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:33.439767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.439774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.439776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.439778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:33.439843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:33.439851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.439864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.439876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.439892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:33.439917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721597181:8183], now have 1 active actors on pipe
2025-10-09T21:49:33.439925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721597044:2432], now have 1 active actors on pipe
2025-10-09T21:49:33.439935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342471359770174:5006]
2025-10-09T21:49:33.439937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342471359770174:5006]
2025-10-09T21:49:33.439939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342471359770174:5006]
2025-10-09T21:49:33.439964Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1552
2025-10-09T21:49:33.439989Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1552
2025-10-09T21:49:33.440415Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1553 partition: 0 MaxSeqNo: 25 sessionId: src_id|e3965cbf-87fec710-198fe12c-18a070fd_0
2025-10-09T21:49:33.439952Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:33.439964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:33.439967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:33.439969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:33.439969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:33.439984Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:33.439987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:33.440021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342471359770174:5006]
2025-10-09T21:49:33.440028Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:33.440032Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:33.440042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:33.440047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:33.440134Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e3965cbf-87fec710-198fe12c-18a070fd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:33.440140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:33.440144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:33.440147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:33.440157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:33.440167Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:33.440188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:33.440310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:33.440314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:33.440329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:33.440770Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db4e4893-1adc2237-41774317-a3151af1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e3965cbf-87fec710-198fe12c-18a070fd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:33.443152Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3965cbf-87fec710-198fe12c-18a070fd_0] Write session: try to update token
2025-10-09T21:49:33.443174Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3965cbf-87fec710-198fe12c-18a070fd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:33.454377Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:33.454394Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:33.454396Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:33.454398Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:33.454407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342471359770174:5006]
2025-10-09T21:49:33.454414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342471359770174:5006]
2025-10-09T21:49:33.454418Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342471359770174:5006]
2025-10-09T21:49:33.454461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:33.454468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.454471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:33.454477Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:33.454485Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:33.454502Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:33.454605Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:33.456214Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1553 sessionId: src_id|e3965cbf-87fec710-198fe12c-18a070fd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:33.458584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:33.516785Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342471721597193:2585]
2025-10-09T21:49:33.517018Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 1552, pipe: [1:7559342471721597193:2585]
2025-10-09T21:49:33.606654Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:33.606898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:33.606914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:33.606955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721597193:2585], now have 1 active actors on pipe
2025-10-09T21:49:33.607124Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:33.607161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:33.607176Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:33.607241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1553 reason: error at write: Disconnected sessionId: src_id|e3965cbf-87fec710-198fe12c-18a070fd_0
2025-10-09T21:49:33.607429Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1553 sessionId: src_id|e3965cbf-87fec710-198fe12c-18a070fd_0 is DEAD
2025-10-09T21:49:33.607493Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342471721597193:2585]
2025-10-09T21:49:33.607521Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:33.607577Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3965cbf-87fec710-198fe12c-18a070fd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:33.607588Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3965cbf-87fec710-198fe12c-18a070fd_0] Write session will restart in 0.009599s
2025-10-09T21:49:33.607612Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3965cbf-87fec710-198fe12c-18a070fd_0] Write session: Do CDS request
2025-10-09T21:49:33.607618Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3965cbf-87fec710-198fe12c-18a070fd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:33.617700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:33.618570Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3965cbf-87fec710-198fe12c-18a070fd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:33.618644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:33.618668Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1554
2025-10-09T21:49:33.618794Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1554 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:33.618826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1554 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:33.618835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1554 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:33.618839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:33.619397Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1554 sessionId:  describe result for acl check
2025-10-09T21:49:33.619434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:33.619436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:33.619437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:33.619442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342471721597298:8169] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:33.619894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:33.619968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342471721597298:8169] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:33.620063Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:33.620072Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:33.620082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:33.620129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:33.620167Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342471359770310:5004]
2025-10-09T21:49:33.620730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:33.621055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:33.621097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:33.621398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:33.621438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.621444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.621445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.621446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.621448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.621449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.621451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.621452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.621453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.621458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.621468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.621472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.621476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.621482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.621486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.621490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.621494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.621497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.621504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.621511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.621517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.621527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.621535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.621540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.621545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.621554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.621563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.621582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.621590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.621597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.621606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.621614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.621622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.621627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.621636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.621641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.621669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:33.621986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342471721597298:8169] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:33.627272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342471721597298:8169] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:33.627284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342471721597298:8169] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:33.627287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342471721597298:8169] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:33.627295Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1554 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:33.627735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:33.628718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:33.628731Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:33.628733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:33.628736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:33.628740Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1553 [2:7559342471359770310:5004]
2025-10-09T21:49:33.628748Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:33.628761Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:33.628763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:33.628764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:33.628766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:33.628778Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:33.628782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:33.628787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:33.628826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.628831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.628833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.628834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:33.628893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:33.628899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.628909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.628919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.628934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:33.628959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721597336:8169], now have 1 active actors on pipe
2025-10-09T21:49:33.628966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721597195:2432], now have 1 active actors on pipe
2025-10-09T21:49:33.628974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342471359770305:5004]
2025-10-09T21:49:33.628976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342471359770305:5004]
2025-10-09T21:49:33.628978Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342471359770305:5004]
2025-10-09T21:49:33.628993Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:33.629009Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:33.629017Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:33.629019Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:33.629026Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:33.629031Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:33.629034Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:33.629040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342471359770305:5004]
2025-10-09T21:49:33.629044Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:33.629041Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1553
2025-10-09T21:49:33.629043Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1553
2025-10-09T21:49:33.629055Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:33.629098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:33.629101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:33.629120Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fcd26725-33db6f3e-6c7901a8-fbb7d856_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:33.629125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:33.629127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:33.629129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:33.629136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:33.629142Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:33.629152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:33.629332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:33.629354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:33.629382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:33.629504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1554 partition: 0 MaxSeqNo: 25 sessionId: src_id|fcd26725-33db6f3e-6c7901a8-fbb7d856_0
2025-10-09T21:49:33.629827Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3965cbf-87fec710-198fe12c-18a070fd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fcd26725-33db6f3e-6c7901a8-fbb7d856_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:33.632222Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fcd26725-33db6f3e-6c7901a8-fbb7d856_0] Write session: try to update token
2025-10-09T21:49:33.632241Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fcd26725-33db6f3e-6c7901a8-fbb7d856_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:33.643740Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:33.643768Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:33.643771Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:33.643774Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:33.643784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342471359770305:5004]
2025-10-09T21:49:33.643790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342471359770305:5004]
2025-10-09T21:49:33.643795Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342471359770305:5004]
2025-10-09T21:49:33.643840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:33.643845Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:33.643846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.643850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:33.643851Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:33.643855Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:33.644010Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:33.645139Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1554 sessionId: src_id|fcd26725-33db6f3e-6c7901a8-fbb7d856_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:33.647458Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:33.707848Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342471721597344:2585]
2025-10-09T21:49:33.708185Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 1553, pipe: [1:7559342471721597344:2585]
2025-10-09T21:49:33.792476Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:33.793157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:33.793175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:33.793212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721597344:2585], now have 1 active actors on pipe
2025-10-09T21:49:33.793407Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:33.793445Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342471721597344:2585]
2025-10-09T21:49:33.793453Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:33.793457Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:33.793491Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:33.793519Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1554 reason: error at write: Disconnected sessionId: src_id|fcd26725-33db6f3e-6c7901a8-fbb7d856_0
2025-10-09T21:49:33.793719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1554 sessionId: src_id|fcd26725-33db6f3e-6c7901a8-fbb7d856_0 is DEAD
2025-10-09T21:49:33.793862Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fcd26725-33db6f3e-6c7901a8-fbb7d856_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:33.793873Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fcd26725-33db6f3e-6c7901a8-fbb7d856_0] Write session will restart in 0.006239s
2025-10-09T21:49:33.793898Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fcd26725-33db6f3e-6c7901a8-fbb7d856_0] Write session: Do CDS request
2025-10-09T21:49:33.793904Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fcd26725-33db6f3e-6c7901a8-fbb7d856_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:33.801462Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fcd26725-33db6f3e-6c7901a8-fbb7d856_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:33.801527Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:33.801539Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1555
2025-10-09T21:49:33.801679Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1555 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:33.801715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1555 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:33.801727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1555 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:33.801731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:33.802283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1555 sessionId:  describe result for acl check
2025-10-09T21:49:33.802323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:33.802328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:33.802329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:33.802333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342471721597416:8172] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:33.802845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342471721597416:8172] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:33.804940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342471721597416:8172] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:33.808204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:33.808418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342471721597416:8172] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:33.808431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342471721597416:8172] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:33.808435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342471721597416:8172] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:33.808443Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1555 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:33.808586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:33.808756Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:33.808766Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:33.808776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:33.808823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:33.808866Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342471359770446:5002]
2025-10-09T21:49:33.809536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:33.809872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:33.809913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:33.810164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:33.810199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.810200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.810201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.810201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.810202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.810203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.810204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.810204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.810205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.810210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.810216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.810219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.810221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.810223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.810225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.810227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.810229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.810231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.810237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.810242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.810246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.810249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.810252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.810255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.810258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.810261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.810263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.810266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.810270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.810273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.810276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.810279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.810282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.810285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.810287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.810290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.810306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:33.816494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:33.817515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:33.817526Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:33.817528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:33.817532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:33.817538Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1554 [2:7559342471359770446:5002]
2025-10-09T21:49:33.817548Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:33.817564Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:33.817566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:33.817581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:33.817583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:33.817597Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:33.817601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:33.817608Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:33.817653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.817660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.817662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.817664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:33.817722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:33.817729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.817738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.817744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.817754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:33.817769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342471359770440:5002]
2025-10-09T21:49:33.817775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342471359770440:5002]
2025-10-09T21:49:33.817777Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342471359770440:5002]
2025-10-09T21:49:33.817784Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:33.817795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:33.817798Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:33.817802Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:33.817804Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:33.817805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:33.817807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:33.817817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721597485:8172], now have 1 active actors on pipe
2025-10-09T21:49:33.817820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721597346:2432], now have 1 active actors on pipe
2025-10-09T21:49:33.817848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342471359770440:5002]
2025-10-09T21:49:33.817866Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:33.817881Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:33.817899Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1554
2025-10-09T21:49:33.817908Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1554
2025-10-09T21:49:33.818154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:33.818166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:33.818199Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|96fb6fd5-43070b4c-ce3664d8-d06e4441_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:33.818212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:33.818216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:33.818218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:33.818226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:33.818235Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:33.818252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:33.818440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:33.818449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:33.818463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:33.818596Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1555 partition: 0 MaxSeqNo: 25 sessionId: src_id|96fb6fd5-43070b4c-ce3664d8-d06e4441_0
2025-10-09T21:49:33.818932Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fcd26725-33db6f3e-6c7901a8-fbb7d856_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|96fb6fd5-43070b4c-ce3664d8-d06e4441_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:33.821271Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96fb6fd5-43070b4c-ce3664d8-d06e4441_0] Write session: try to update token
2025-10-09T21:49:33.821290Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96fb6fd5-43070b4c-ce3664d8-d06e4441_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:33.832075Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:33.832094Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:33.832097Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:33.832099Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:33.832109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342471359770440:5002]
2025-10-09T21:49:33.832114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342471359770440:5002]
2025-10-09T21:49:33.832121Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342471359770440:5002]
2025-10-09T21:49:33.832167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:33.832174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.832176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:33.832187Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:33.832194Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:33.832198Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:33.832303Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:33.833682Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1555 sessionId: src_id|96fb6fd5-43070b4c-ce3664d8-d06e4441_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:33.835852Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:33.893641Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342471721597499:2585]
2025-10-09T21:49:33.893852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 1554, pipe: [1:7559342471721597499:2585]
2025-10-09T21:49:33.980537Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:33.980984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:33.981001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:33.981042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721597499:2585], now have 1 active actors on pipe
2025-10-09T21:49:33.981222Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:33.981284Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:33.981298Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342471721597499:2585]
2025-10-09T21:49:33.981324Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:33.981403Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1555 reason: status is not ok: tablet will be restarted right now sessionId: src_id|96fb6fd5-43070b4c-ce3664d8-d06e4441_0
2025-10-09T21:49:33.981609Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1555 sessionId: src_id|96fb6fd5-43070b4c-ce3664d8-d06e4441_0 is DEAD
2025-10-09T21:49:33.981754Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96fb6fd5-43070b4c-ce3664d8-d06e4441_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:33.981768Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96fb6fd5-43070b4c-ce3664d8-d06e4441_0] Write session will restart in 0.006892s
2025-10-09T21:49:33.981795Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96fb6fd5-43070b4c-ce3664d8-d06e4441_0] Write session: Do CDS request
2025-10-09T21:49:33.981802Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96fb6fd5-43070b4c-ce3664d8-d06e4441_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:33.989461Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96fb6fd5-43070b4c-ce3664d8-d06e4441_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:33.989554Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:33.989566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1556
2025-10-09T21:49:33.989682Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1556 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:33.989713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1556 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:33.989722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1556 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:33.989726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:33.990165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1556 sessionId:  describe result for acl check
2025-10-09T21:49:33.990197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:33.990204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:33.990206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:33.990211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342471721597605:8164] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:33.990776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342471721597605:8164] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:33.991336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:33.992851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342471721597605:8164] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:33.993694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:33.993849Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:33.993861Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:33.993872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:33.993916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:33.993952Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342471359770577:4999]
2025-10-09T21:49:33.994397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:33.994669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:33.994719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:33.995002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342471721597605:8164] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:33.995017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:33.995014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342471721597605:8164] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:33.995018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342471721597605:8164] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:33.995026Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1556 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:33.995065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.995074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.995075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.995076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.995077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.995078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.995079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.995081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.995082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.995087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.995097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.995101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.995104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.995108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.995112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.995116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.995119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.995122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.995129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.995136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:33.995142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.995146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:33.995150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.995155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:33.995159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.995165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:33.995169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.995174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:33.995179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.995184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:33.995189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.995194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:33.995198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.995203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:33.995208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.995213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:33.995244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:34.001515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:34.002574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:34.002594Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:34.002598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:34.002603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:34.002609Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1555 [2:7559342471359770577:4999]
2025-10-09T21:49:34.002619Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:34.002636Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:34.002638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:34.002640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:34.002642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:34.002657Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:34.002662Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:34.002667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:34.002724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.002733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.002734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.002737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:34.002974Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1555
2025-10-09T21:49:34.002977Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1555
2025-10-09T21:49:34.003109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:34.003120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.003135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.003144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.003160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:34.003179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721597642:8164], now have 1 active actors on pipe
2025-10-09T21:49:34.003187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342471721597503:2432], now have 1 active actors on pipe
2025-10-09T21:49:34.003201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.003209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.003227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342471359770571:4999]
2025-10-09T21:49:34.003236Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342471359770571:4999]
2025-10-09T21:49:34.003237Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342471359770571:4999]
2025-10-09T21:49:34.003246Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:34.003258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:34.003267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:34.003268Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:34.003269Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:34.003272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:34.003273Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:34.003284Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5c9396f8-417e4cb1-4fc0eb17-b724301f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:34.003289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:34.003292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:34.003294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:34.003301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:34.003309Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:34.003340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342471359770571:4999]
2025-10-09T21:49:34.003348Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:34.003361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:34.003361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:34.003510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.003518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.003531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:34.003618Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1556 partition: 0 MaxSeqNo: 25 sessionId: src_id|5c9396f8-417e4cb1-4fc0eb17-b724301f_0
2025-10-09T21:49:34.003985Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96fb6fd5-43070b4c-ce3664d8-d06e4441_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5c9396f8-417e4cb1-4fc0eb17-b724301f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:34.006501Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9396f8-417e4cb1-4fc0eb17-b724301f_0] Write session: try to update token
2025-10-09T21:49:34.006519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9396f8-417e4cb1-4fc0eb17-b724301f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:34.020584Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1556 sessionId: src_id|5c9396f8-417e4cb1-4fc0eb17-b724301f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:34.022049Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:34.022071Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:34.022085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342471359770571:4999]
2025-10-09T21:49:34.022134Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:34.022142Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:34.022313Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:34.022471Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:34.022614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:34.022761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:34.022960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:34.023007Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:34.023010Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:34.023171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.023184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.023216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:34.023289Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:34.028752Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:34.028770Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:34.028772Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:34.028774Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:34.028784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342471359770571:4999]
2025-10-09T21:49:34.028788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342471359770571:4999]
2025-10-09T21:49:34.028792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342471359770571:4999]
2025-10-09T21:49:34.028833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:34.028837Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:34.028840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.028843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:34.028844Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:34.028848Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:34.028975Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:34.081491Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342476016564948:2585]
2025-10-09T21:49:34.081761Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 1555, pipe: [1:7559342476016564948:2585]
2025-10-09T21:49:34.180744Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:34.187681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.187707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.187850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.187914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.187980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.188047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.188093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.188104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.188164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.188220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.188286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.188351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.188391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.188398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.188455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.188521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.188586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.188649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.188692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.188699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.188762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.188825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.188882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.188938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.188977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.188983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.189051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.189106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.189163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.189219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.189313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.189323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.189381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.189439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.189506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.189581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.189620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.189628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.189685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.189738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.189799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.189854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.189894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.189903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.189958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.190012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.190094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.190154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.190194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.190201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.190255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.190314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.190369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.190424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.190464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.190470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.190534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.190591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.190645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.190701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.190754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.190764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.190812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342476016564948:2585], now have 1 active actors on pipe
2025-10-09T21:49:34.191032Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342476016564948:2585]
2025-10-09T21:49:34.191061Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:34.191063Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:34.191087Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1556 reason: error at write: Disconnected sessionId: src_id|5c9396f8-417e4cb1-4fc0eb17-b724301f_0
2025-10-09T21:49:34.191265Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1556 sessionId: src_id|5c9396f8-417e4cb1-4fc0eb17-b724301f_0 is DEAD
2025-10-09T21:49:34.191398Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9396f8-417e4cb1-4fc0eb17-b724301f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:34.191409Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9396f8-417e4cb1-4fc0eb17-b724301f_0] Write session will restart in 0.005435s
2025-10-09T21:49:34.191427Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9396f8-417e4cb1-4fc0eb17-b724301f_0] Write session: Do CDS request
2025-10-09T21:49:34.191432Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9396f8-417e4cb1-4fc0eb17-b724301f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:34.191700Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:34.191815Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:34.198485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9396f8-417e4cb1-4fc0eb17-b724301f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:34.198603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:34.198617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1557
2025-10-09T21:49:34.198764Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1557 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:34.198798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1557 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:34.198822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1557 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:34.198826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:34.199254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1557 sessionId:  describe result for acl check
2025-10-09T21:49:34.199289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:34.199296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:34.199298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:34.199304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342476016565047:8148] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:34.199857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342476016565047:8148] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:34.202372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342476016565047:8148] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:34.204455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:34.204976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:34.205166Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:34.205179Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:34.205191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:34.205240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:34.205255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342476016565047:8148] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:34.205271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342476016565047:8148] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:34.205275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342476016565047:8148] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:34.205285Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1557 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:34.205384Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342475654738045:4983]
2025-10-09T21:49:34.205916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:34.206249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:34.206298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:34.206556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:34.206593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.206599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.206600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.206600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.206601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.206602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.206602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.206603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.206604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.206608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.206614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.206617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.206619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.206621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.206623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.206625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.206628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.206630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.206635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.206640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.206644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.206647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.206650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.206654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.206656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.206659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.206662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.206665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.206668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.206671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.206674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.206677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.206680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.206683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.206685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.206689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.206704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:34.213111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:34.214164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:34.214180Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:34.214182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:34.214186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:34.214193Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1556 [2:7559342475654738045:4983]
2025-10-09T21:49:34.214206Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:34.214228Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:34.214230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:34.214233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:34.214235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:34.214250Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:34.214257Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:34.214265Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:34.214313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.214325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.214327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.214330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:34.214403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:34.214410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.214418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.214425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.214434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:34.214456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342476016565115:8148], now have 1 active actors on pipe
2025-10-09T21:49:34.214469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342476016564967:2432], now have 1 active actors on pipe
2025-10-09T21:49:34.214477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342475654738038:4983]
2025-10-09T21:49:34.214479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342475654738038:4983]
2025-10-09T21:49:34.214481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342475654738038:4983]
2025-10-09T21:49:34.214494Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:34.214508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:34.214514Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:34.214517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:34.214517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:34.214519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:34.214520Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:34.214545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342475654738038:4983]
2025-10-09T21:49:34.214557Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:34.214556Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1556
2025-10-09T21:49:34.214578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:34.214568Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1556
2025-10-09T21:49:34.214585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.214589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.214611Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|482ece57-fffdc0a9-705adfa8-bc574b84_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:34.214616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:34.214619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:34.214621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:34.214630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:34.214637Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:34.214647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:34.214764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.214771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.214781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:34.214866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1557 partition: 0 MaxSeqNo: 25 sessionId: src_id|482ece57-fffdc0a9-705adfa8-bc574b84_0
2025-10-09T21:49:34.215221Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9396f8-417e4cb1-4fc0eb17-b724301f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|482ece57-fffdc0a9-705adfa8-bc574b84_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:34.217530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|482ece57-fffdc0a9-705adfa8-bc574b84_0] Write session: try to update token
2025-10-09T21:49:34.217547Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|482ece57-fffdc0a9-705adfa8-bc574b84_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:34.225509Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:34.225529Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:34.225538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342475654738038:4983]
2025-10-09T21:49:34.225589Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:34.225590Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:34.225759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:34.225906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:34.226066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:34.226207Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:34.226397Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:34.226441Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:34.230218Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1557 sessionId: src_id|482ece57-fffdc0a9-705adfa8-bc574b84_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:34.232723Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:34.232907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.232929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.232967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:34.233048Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:34.238192Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:34.238209Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:34.238212Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:34.238214Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:34.238224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342475654738038:4983]
2025-10-09T21:49:34.238227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342475654738038:4983]
2025-10-09T21:49:34.238232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342475654738038:4983]
2025-10-09T21:49:34.238275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:34.238280Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:34.238282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.238285Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:34.238285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:34.238291Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:34.238418Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:34.291459Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342476016565132:2585]
2025-10-09T21:49:34.291678Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 1556, pipe: [1:7559342476016565132:2585]
2025-10-09T21:49:34.381393Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:34.387943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.387971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.388113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.388168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.388225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.388290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.388337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.388352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.388415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.388468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.388528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.388589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.388629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.388638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.388691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.388742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.388806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.388858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.388897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.388904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.388960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.389018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.389077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.389132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.389171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.389184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.389239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.389298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.389356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.389410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.389449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.389455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.389508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.389563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.389630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.389686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.389732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.389739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.389792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.389849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.389902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.389954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.389997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.390000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.390069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.390130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.390184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.390247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.390286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.390292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.390352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.390403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.390455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.390508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.390566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.390574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.390632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:34.390683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:34.390735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:34.390791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:34.390837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:34.390847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:34.390887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342476016565132:2585], now have 1 active actors on pipe
2025-10-09T21:49:34.391098Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:34.391099Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:34.391104Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:34.391147Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342476016565132:2585]
2025-10-09T21:49:34.391148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1557 reason: error at write: Disconnected sessionId: src_id|482ece57-fffdc0a9-705adfa8-bc574b84_0
2025-10-09T21:49:34.391345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1557 sessionId: src_id|482ece57-fffdc0a9-705adfa8-bc574b84_0 is DEAD
2025-10-09T21:49:34.391465Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|482ece57-fffdc0a9-705adfa8-bc574b84_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:34.391477Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|482ece57-fffdc0a9-705adfa8-bc574b84_0] Write session will restart in 0.007902s
2025-10-09T21:49:34.391517Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|482ece57-fffdc0a9-705adfa8-bc574b84_0] Write session: Do CDS request
2025-10-09T21:49:34.391524Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|482ece57-fffdc0a9-705adfa8-bc574b84_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:34.391667Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:34.400598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|482ece57-fffdc0a9-705adfa8-bc574b84_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:34.401064Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:34.401075Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1558
2025-10-09T21:49:34.401245Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1558 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:34.401289Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1558 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:34.401295Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1558 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:34.401298Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:34.401726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1558 sessionId:  describe result for acl check
2025-10-09T21:49:34.401759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:34.401760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:34.401761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:34.401766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342476016565241:8139] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:34.401825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:34.402227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342476016565241:8139] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:34.404191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:34.404371Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:34.404382Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:34.404391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:34.404432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:34.404459Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342475654738179:4972]
2025-10-09T21:49:34.404826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342476016565241:8139] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:34.404861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:34.405148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:34.405182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:34.405387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:34.405421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.405427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.405428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.405429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.405430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.405430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.405431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.405432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.405433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.405437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.405444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.405447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.405449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.405451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.405453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.405455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.405457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.405460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.405464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.405469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.405474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.405477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.405480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.405483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.405486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.405489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.405492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.405495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.405498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.405501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.405504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.405507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.405510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.405513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.405516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.405519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.405535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:34.410397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342476016565241:8139] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:34.410409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342476016565241:8139] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:34.410412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342476016565241:8139] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:34.410420Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1558 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:34.411750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:34.413077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:34.413093Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:34.413095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:34.413099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:34.413106Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1557 [2:7559342475654738179:4972]
2025-10-09T21:49:34.413118Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:34.413135Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:34.413138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:34.413141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:34.413143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:34.413159Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:34.413165Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:34.413171Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:34.413224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.413235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.413237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.413240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:34.413311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:34.413322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.413333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.413345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.413363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:34.413389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342476016565135:2432], now have 1 active actors on pipe
2025-10-09T21:49:34.413398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342476016565281:8139], now have 1 active actors on pipe
2025-10-09T21:49:34.413406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342475654738173:4972]
2025-10-09T21:49:34.413408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342475654738173:4972]
2025-10-09T21:49:34.413410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342475654738173:4972]
2025-10-09T21:49:34.413422Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:34.413432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:34.413435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:34.413443Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:34.413445Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:34.413448Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:34.413451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:34.413436Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1557
2025-10-09T21:49:34.413485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342475654738173:4972]
2025-10-09T21:49:34.413516Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1557
2025-10-09T21:49:34.413503Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:34.413507Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:34.413519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.413539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.413597Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e4f8683d-5d79424b-b0332cb7-d227e67d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:34.413613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:34.413617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:34.413620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:34.413631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:34.413641Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:34.413659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:34.413873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.413888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.413911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:34.414014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1558 partition: 0 MaxSeqNo: 25 sessionId: src_id|e4f8683d-5d79424b-b0332cb7-d227e67d_0
2025-10-09T21:49:34.414358Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|482ece57-fffdc0a9-705adfa8-bc574b84_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e4f8683d-5d79424b-b0332cb7-d227e67d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:34.416665Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4f8683d-5d79424b-b0332cb7-d227e67d_0] Write session: try to update token
2025-10-09T21:49:34.416681Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4f8683d-5d79424b-b0332cb7-d227e67d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:34.427265Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:34.427289Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:34.427291Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:34.427293Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:34.427304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342475654738173:4972]
2025-10-09T21:49:34.427312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342475654738173:4972]
2025-10-09T21:49:34.427315Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342475654738173:4972]
2025-10-09T21:49:34.427364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:34.427371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.427374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:34.427390Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:34.427397Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:34.427402Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:34.427508Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:34.429438Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1558 sessionId: src_id|e4f8683d-5d79424b-b0332cb7-d227e67d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:34.431818Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:34.491137Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342476016565288:2585]
2025-10-09T21:49:34.491344Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 1557, pipe: [1:7559342476016565288:2585]
2025-10-09T21:49:34.572668Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:34.573098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.573111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.573146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342476016565288:2585], now have 1 active actors on pipe
2025-10-09T21:49:34.573371Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:34.573412Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:34.573415Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:34.573479Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1558 reason: error at write: Disconnected sessionId: src_id|e4f8683d-5d79424b-b0332cb7-d227e67d_0
2025-10-09T21:49:34.573473Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:34.573611Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342476016565288:2585]
2025-10-09T21:49:34.573666Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1558 sessionId: src_id|e4f8683d-5d79424b-b0332cb7-d227e67d_0 is DEAD
2025-10-09T21:49:34.573807Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4f8683d-5d79424b-b0332cb7-d227e67d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:34.573817Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4f8683d-5d79424b-b0332cb7-d227e67d_0] Write session will restart in 0.009953s
2025-10-09T21:49:34.573837Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4f8683d-5d79424b-b0332cb7-d227e67d_0] Write session: Do CDS request
2025-10-09T21:49:34.573841Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4f8683d-5d79424b-b0332cb7-d227e67d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:34.584537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4f8683d-5d79424b-b0332cb7-d227e67d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:34.584652Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:34.584668Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1559
2025-10-09T21:49:34.584809Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1559 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:34.584856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1559 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:34.584862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1559 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:34.584866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:34.585247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1559 sessionId:  describe result for acl check
2025-10-09T21:49:34.585281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:34.585290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:34.585292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:34.585298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342476016565397:8131] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:34.585856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342476016565397:8131] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:34.587555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:34.587719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342476016565397:8131] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:34.588065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:34.588236Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:34.588245Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:34.588253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:34.588289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:34.588325Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342475654738316:4966]
2025-10-09T21:49:34.588688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:34.588941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:34.588971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:34.589250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:34.589323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.589337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.589339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.589340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.589341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.589342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.589344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.589345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.589346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.589351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.589358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.589360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.589362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.589364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.589367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.589369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.589371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.589373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.589378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.589383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.589387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.589390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.589393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.589396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.589399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.589402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.589407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.589412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.589416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.589421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.589426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.589432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.589436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.589443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.589447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.589453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.589482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:34.589997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342476016565397:8131] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:34.590009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342476016565397:8131] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:34.590013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342476016565397:8131] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:34.590021Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1559 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:34.595398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:34.596396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:34.596410Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:34.596411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:34.596415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:34.596419Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1558 [2:7559342475654738316:4966]
2025-10-09T21:49:34.596429Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:34.596443Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:34.596445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:34.596447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:34.596448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:34.596462Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:34.596467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:34.596474Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:34.596529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.596536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.596538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.596540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:34.596587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:34.596593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.596600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.596606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.596616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:34.596631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342475654738310:4966]
2025-10-09T21:49:34.596633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342475654738310:4966]
2025-10-09T21:49:34.596634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342475654738310:4966]
2025-10-09T21:49:34.596640Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:34.596655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342476016565291:2432], now have 1 active actors on pipe
2025-10-09T21:49:34.596659Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:34.596660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:34.596662Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:34.596664Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:34.596669Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:34.596671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:34.596675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342476016565436:8131], now have 1 active actors on pipe
2025-10-09T21:49:34.596694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342475654738310:4966]
2025-10-09T21:49:34.596701Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:34.596710Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:34.596748Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1558
2025-10-09T21:49:34.596763Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1558
2025-10-09T21:49:34.596844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.596861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.596891Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3682e8dd-c1fa5e02-6ec7cf5d-edca43bf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:34.596900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:34.596904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:34.596906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:34.596917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:34.596930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:34.596964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:34.597117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.597127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.597142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:34.597221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1559 partition: 0 MaxSeqNo: 25 sessionId: src_id|3682e8dd-c1fa5e02-6ec7cf5d-edca43bf_0
2025-10-09T21:49:34.597510Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4f8683d-5d79424b-b0332cb7-d227e67d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3682e8dd-c1fa5e02-6ec7cf5d-edca43bf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:34.599984Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3682e8dd-c1fa5e02-6ec7cf5d-edca43bf_0] Write session: try to update token
2025-10-09T21:49:34.600001Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3682e8dd-c1fa5e02-6ec7cf5d-edca43bf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:34.610879Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:34.610901Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:34.610903Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:34.610905Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:34.610916Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342475654738310:4966]
2025-10-09T21:49:34.610921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342475654738310:4966]
2025-10-09T21:49:34.610925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342475654738310:4966]
2025-10-09T21:49:34.610968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:34.610976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.610979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:34.610970Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:34.610988Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:34.610992Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:34.611110Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:34.612549Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1559 sessionId: src_id|3682e8dd-c1fa5e02-6ec7cf5d-edca43bf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:34.614835Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:34.673920Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342476016565443:2585]
2025-10-09T21:49:34.674202Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 1558, pipe: [1:7559342476016565443:2585]
2025-10-09T21:49:34.758355Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:34.758928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.758944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.758981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342476016565443:2585], now have 1 active actors on pipe
2025-10-09T21:49:34.759141Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:34.759173Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:34.759185Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342476016565443:2585]
2025-10-09T21:49:34.759206Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:34.759403Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1559 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3682e8dd-c1fa5e02-6ec7cf5d-edca43bf_0
2025-10-09T21:49:34.759540Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1559 sessionId: src_id|3682e8dd-c1fa5e02-6ec7cf5d-edca43bf_0 is DEAD
2025-10-09T21:49:34.759680Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3682e8dd-c1fa5e02-6ec7cf5d-edca43bf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:34.759690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3682e8dd-c1fa5e02-6ec7cf5d-edca43bf_0] Write session will restart in 0.005416s
2025-10-09T21:49:34.759714Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3682e8dd-c1fa5e02-6ec7cf5d-edca43bf_0] Write session: Do CDS request
2025-10-09T21:49:34.759721Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3682e8dd-c1fa5e02-6ec7cf5d-edca43bf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:34.766460Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3682e8dd-c1fa5e02-6ec7cf5d-edca43bf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:34.766560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:34.766574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1560
2025-10-09T21:49:34.766703Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1560 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:34.766737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1560 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:34.766743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1560 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:34.766746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:34.767178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1560 sessionId:  describe result for acl check
2025-10-09T21:49:34.767215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:34.767221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:34.767223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:34.767228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342476016565524:8120] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:34.767778Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342476016565524:8120] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:34.769736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:34.769956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342476016565524:8120] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:34.771980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:34.772215Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:34.772224Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:34.772237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:34.772305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:34.772293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342476016565524:8120] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:34.772305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342476016565524:8120] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:34.772308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342476016565524:8120] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:34.772317Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1560 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:34.772343Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342475654738457:4953]
2025-10-09T21:49:34.772942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:34.773245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:34.773288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:34.773521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:34.773582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.773589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.773590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.773591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.773592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.773593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.773594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.773596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.773597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.773602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.773611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.773614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.773616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.773618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.773620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.773622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.773624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.773626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.773631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.773636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.773641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.773644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.773647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.773650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.773652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.773655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.773659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.773662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.773665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.773668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.773671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.773674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.773676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.773678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.773681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.773684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.773701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:34.780006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:34.781108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:34.781125Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:34.781127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:34.781131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:34.781138Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1559 [2:7559342475654738457:4953]
2025-10-09T21:49:34.781151Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:34.781170Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:34.781178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:34.781181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:34.781184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:34.781200Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:34.781206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:34.781219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:34.781272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.781279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.781281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.781284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:34.781348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:34.781357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.781368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.781380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.781395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:34.781418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342476016565447:2432], now have 1 active actors on pipe
2025-10-09T21:49:34.781427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342476016565591:8120], now have 1 active actors on pipe
2025-10-09T21:49:34.781435Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342475654738451:4953]
2025-10-09T21:49:34.781441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342475654738451:4953]
2025-10-09T21:49:34.781443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342475654738451:4953]
2025-10-09T21:49:34.781456Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:34.781467Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1559
2025-10-09T21:49:34.781488Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1559
2025-10-09T21:49:34.781474Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:34.781480Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:34.781484Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:34.781493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:34.781495Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:34.781500Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:34.781521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342475654738451:4953]
2025-10-09T21:49:34.781525Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:34.781527Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:34.781544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.781555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.781879Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2894c88c-1286d821-942ff3a5-79630591_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:34.781897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:34.781901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:34.781904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:34.781915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:34.781930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:34.781962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:34.782182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.782192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.782208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:34.782301Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1560 partition: 0 MaxSeqNo: 25 sessionId: src_id|2894c88c-1286d821-942ff3a5-79630591_0
2025-10-09T21:49:34.782615Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3682e8dd-c1fa5e02-6ec7cf5d-edca43bf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2894c88c-1286d821-942ff3a5-79630591_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:34.784989Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2894c88c-1286d821-942ff3a5-79630591_0] Write session: try to update token
2025-10-09T21:49:34.785008Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2894c88c-1286d821-942ff3a5-79630591_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:34.796040Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:34.796062Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:34.796065Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:34.796067Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:34.796079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342475654738451:4953]
2025-10-09T21:49:34.796083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342475654738451:4953]
2025-10-09T21:49:34.796087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342475654738451:4953]
2025-10-09T21:49:34.796132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:34.796131Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:34.796137Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:34.796139Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:34.796144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.796147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:34.796271Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:34.797456Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1560 sessionId: src_id|2894c88c-1286d821-942ff3a5-79630591_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:34.799895Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:34.859206Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342476016565608:2585]
2025-10-09T21:49:34.859445Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 1559, pipe: [1:7559342476016565608:2585]
2025-10-09T21:49:34.944771Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:34.945190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.945203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.945242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342476016565608:2585], now have 1 active actors on pipe
2025-10-09T21:49:34.945468Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342476016565608:2585]
2025-10-09T21:49:34.945494Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:34.945497Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:34.945501Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:34.945602Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:34.945586Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1560 reason: error at write: Disconnected sessionId: src_id|2894c88c-1286d821-942ff3a5-79630591_0
2025-10-09T21:49:34.945752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1560 sessionId: src_id|2894c88c-1286d821-942ff3a5-79630591_0 is DEAD
2025-10-09T21:49:34.945889Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2894c88c-1286d821-942ff3a5-79630591_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:34.945900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2894c88c-1286d821-942ff3a5-79630591_0] Write session will restart in 0.009431s
2025-10-09T21:49:34.945921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2894c88c-1286d821-942ff3a5-79630591_0] Write session: Do CDS request
2025-10-09T21:49:34.945926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2894c88c-1286d821-942ff3a5-79630591_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:34.956519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2894c88c-1286d821-942ff3a5-79630591_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:34.956620Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:34.956642Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1561
2025-10-09T21:49:34.956769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1561 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:34.956798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1561 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:34.956807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1561 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:34.956811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:34.957175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:34.957211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1561 sessionId:  describe result for acl check
2025-10-09T21:49:34.957249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:34.957255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:34.957257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:34.957263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342476016565724:8118] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:34.957789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342476016565724:8118] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:34.959466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:34.959639Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:34.959652Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:34.959664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:34.959717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:34.959690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342476016565724:8118] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:34.959755Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342475654738594:4961]
2025-10-09T21:49:34.960260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:34.960549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:34.960588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:34.960795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:34.960825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.960829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.960830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.960831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.960832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.960832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.960833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.960833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.960834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.960837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.960843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.960845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.960847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.960849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.960851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.960853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.960855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.960856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.960861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.960865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:34.960869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.960873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:34.960875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.960878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:34.960881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.960883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:34.960886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.960889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:34.960892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.960894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:34.960897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.960900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.960903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.960905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.960908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.960911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.960926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:34.963890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342476016565724:8118] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:34.963906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342476016565724:8118] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:34.963910Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342476016565724:8118] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:34.963918Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1561 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:34.966837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:34.967857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:34.967870Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:34.967872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:34.967875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:34.967879Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1560 [2:7559342475654738594:4961]
2025-10-09T21:49:34.967888Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:34.967901Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:34.967903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:34.967905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:34.967906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:34.967918Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:34.967926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:34.967932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:34.967970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.967976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.967977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.967979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:34.968032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:34.968039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.968047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:34.968053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:34.968062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:34.968075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342475654738588:4961]
2025-10-09T21:49:34.968077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342475654738588:4961]
2025-10-09T21:49:34.968078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342475654738588:4961]
2025-10-09T21:49:34.968083Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:34.968093Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:34.968095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342476016565760:8118], now have 1 active actors on pipe
2025-10-09T21:49:34.968096Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:34.968098Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:34.968101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342476016565611:2432], now have 1 active actors on pipe
2025-10-09T21:49:34.968106Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:34.968120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:34.968128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:34.968149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342475654738588:4961]
2025-10-09T21:49:34.968161Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:34.968184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:34.968183Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1560
2025-10-09T21:49:34.968202Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1560
2025-10-09T21:49:34.968283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.968292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.968313Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|984410eb-cc6fbc1c-b4ce9698-869d8d43_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:34.968325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:34.968328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:34.968330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:34.968337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:34.968345Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:34.968355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:34.968538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.968548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.968565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:34.968820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1561 partition: 0 MaxSeqNo: 25 sessionId: src_id|984410eb-cc6fbc1c-b4ce9698-869d8d43_0
2025-10-09T21:49:34.969143Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2894c88c-1286d821-942ff3a5-79630591_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|984410eb-cc6fbc1c-b4ce9698-869d8d43_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:34.971665Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|984410eb-cc6fbc1c-b4ce9698-869d8d43_0] Write session: try to update token
2025-10-09T21:49:34.971687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|984410eb-cc6fbc1c-b4ce9698-869d8d43_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:34.978993Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:34.979012Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:34.979387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342475654738588:4961]
2025-10-09T21:49:34.979437Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:34.979438Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:34.979601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:34.979737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:34.979866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:34.979996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:34.980190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:34.980233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:34.984473Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1561 sessionId: src_id|984410eb-cc6fbc1c-b4ce9698-869d8d43_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:34.986773Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:34.986955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:34.986975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:34.987014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:34.987097Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:34.991455Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:34.991471Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:34.991473Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:34.991475Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:34.991485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342475654738588:4961]
2025-10-09T21:49:34.991489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342475654738588:4961]
2025-10-09T21:49:34.991492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342475654738588:4961]
2025-10-09T21:49:34.991530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:34.991535Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:34.991541Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:34.991545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:34.991546Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:34.991548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:34.991675Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:35.045614Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342480311533064:2585]
2025-10-09T21:49:35.045918Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 1560, pipe: [1:7559342480311533064:2585]
2025-10-09T21:49:35.141824Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:35.148602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.148629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.148786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.148843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.148895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.148954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.148997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.149004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.149061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.149117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.149170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.149225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.149446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.149455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.149515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.149587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.149653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.149709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.149745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.149752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.149807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.149861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.149914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.149968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.150008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.150016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.150100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.150155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.150216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.150268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.150306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.150308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.150370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.150433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.150490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.150558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.150600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.150609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.150675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.150733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.150791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.150849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.150894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.150903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.150963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.151022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.151079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.151136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.151178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.151186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.151245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.151304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.151358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.151409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.151447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.151452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.151506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.151555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.151605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.151656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.151713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.151718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.151758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342480311533064:2585], now have 1 active actors on pipe
2025-10-09T21:49:35.151946Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:35.151999Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:35.152006Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:35.152012Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342480311533064:2585]
2025-10-09T21:49:35.152036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1561 reason: error at write: Disconnected sessionId: src_id|984410eb-cc6fbc1c-b4ce9698-869d8d43_0
2025-10-09T21:49:35.152223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1561 sessionId: src_id|984410eb-cc6fbc1c-b4ce9698-869d8d43_0 is DEAD
2025-10-09T21:49:35.152280Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:35.152376Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|984410eb-cc6fbc1c-b4ce9698-869d8d43_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:35.152391Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|984410eb-cc6fbc1c-b4ce9698-869d8d43_0] Write session will restart in 0.008172s
2025-10-09T21:49:35.152416Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|984410eb-cc6fbc1c-b4ce9698-869d8d43_0] Write session: Do CDS request
2025-10-09T21:49:35.152423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|984410eb-cc6fbc1c-b4ce9698-869d8d43_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:35.161528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|984410eb-cc6fbc1c-b4ce9698-869d8d43_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:35.161650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:35.161665Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1562
2025-10-09T21:49:35.161789Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1562 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:35.161823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1562 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:35.161829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1562 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:35.161835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:35.162290Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1562 sessionId:  describe result for acl check
2025-10-09T21:49:35.162321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:35.162323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:35.162325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:35.162332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342480311533174:8110] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:35.162881Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342480311533174:8110] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:35.163479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:35.165397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342480311533174:8110] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:35.165738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:35.165940Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:35.165952Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:35.165963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:35.166012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:35.166066Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342479949706028:4959]
2025-10-09T21:49:35.166510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:35.166807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:35.166852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:35.167116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:35.167165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.167170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.167170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.167171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.167172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.167172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.167173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.167174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.167175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.167178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.167185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.167187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.167189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.167191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.167194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.167196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.167198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.167200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.167205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.167210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.167216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.167221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.167226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.167234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.167239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.167248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.167257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.167264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.167270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.167274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.167284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.167290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.167297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.167301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.167307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.167311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.167328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:35.168146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342480311533174:8110] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:35.168157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342480311533174:8110] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:35.168160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342480311533174:8110] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:35.168174Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1562 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:35.173487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:35.174783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:35.174800Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:35.174802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:35.174805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:35.174810Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1561 [2:7559342479949706028:4959]
2025-10-09T21:49:35.174821Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:35.174835Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:35.174837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:35.174839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:35.174840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:35.174853Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:35.174862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:35.174867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:35.174911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.174916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.174917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.174919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:35.174972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:35.174974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.174981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.174992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.175007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:35.175026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342479949706022:4959]
2025-10-09T21:49:35.175034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342479949706022:4959]
2025-10-09T21:49:35.175036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342479949706022:4959]
2025-10-09T21:49:35.175050Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:35.175063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:35.175066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342480311533218:8110], now have 1 active actors on pipe
2025-10-09T21:49:35.175069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342480311533066:2432], now have 1 active actors on pipe
2025-10-09T21:49:35.175072Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:35.175073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:35.175078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:35.175092Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:35.175095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:35.175096Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1561
2025-10-09T21:49:35.175127Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342479949706022:4959]
2025-10-09T21:49:35.175142Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:35.175146Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:35.175097Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1561
2025-10-09T21:49:35.175157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.175160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.175202Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|32843f99-75227d23-42b13d2e-71d2a1ac_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:35.175210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:35.175213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:35.175215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:35.175222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:35.175235Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:35.175249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:35.175376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.175383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.175393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:35.175477Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1562 partition: 0 MaxSeqNo: 25 sessionId: src_id|32843f99-75227d23-42b13d2e-71d2a1ac_0
2025-10-09T21:49:35.175814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|984410eb-cc6fbc1c-b4ce9698-869d8d43_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|32843f99-75227d23-42b13d2e-71d2a1ac_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:35.178231Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32843f99-75227d23-42b13d2e-71d2a1ac_0] Write session: try to update token
2025-10-09T21:49:35.178247Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32843f99-75227d23-42b13d2e-71d2a1ac_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:35.189362Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:35.189380Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:35.189382Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:35.189384Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:35.189395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342479949706022:4959]
2025-10-09T21:49:35.189398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342479949706022:4959]
2025-10-09T21:49:35.189400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342479949706022:4959]
2025-10-09T21:49:35.189436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:35.189442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.189444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:35.189444Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:35.189450Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:35.189453Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:35.189566Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:35.191260Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1562 sessionId: src_id|32843f99-75227d23-42b13d2e-71d2a1ac_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:35.194158Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:35.251947Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342480311533254:2585]
2025-10-09T21:49:35.252170Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 1561, pipe: [1:7559342480311533254:2585]
2025-10-09T21:49:35.333970Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:35.334439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.334456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.334507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342480311533254:2585], now have 1 active actors on pipe
2025-10-09T21:49:35.334671Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:35.334694Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342480311533254:2585]
2025-10-09T21:49:35.334699Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:35.334703Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:35.334725Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:35.334756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1562 reason: error at write: Disconnected sessionId: src_id|32843f99-75227d23-42b13d2e-71d2a1ac_0
2025-10-09T21:49:35.334909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1562 sessionId: src_id|32843f99-75227d23-42b13d2e-71d2a1ac_0 is DEAD
2025-10-09T21:49:35.335094Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32843f99-75227d23-42b13d2e-71d2a1ac_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:35.335106Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32843f99-75227d23-42b13d2e-71d2a1ac_0] Write session will restart in 0.007826s
2025-10-09T21:49:35.335131Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32843f99-75227d23-42b13d2e-71d2a1ac_0] Write session: Do CDS request
2025-10-09T21:49:35.335136Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32843f99-75227d23-42b13d2e-71d2a1ac_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:35.344025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:35.344527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32843f99-75227d23-42b13d2e-71d2a1ac_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:35.344579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:35.344590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1563
2025-10-09T21:49:35.344723Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1563 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:35.344762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1563 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:35.344773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1563 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:35.344782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:35.345578Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1563 sessionId:  describe result for acl check
2025-10-09T21:49:35.345614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:35.345616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:35.345617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:35.345623Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342480311533368:8093] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:35.346170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342480311533368:8093] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:35.346410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:35.346570Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:35.346577Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:35.346587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:35.346645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:35.346688Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342479949706205:4920]
2025-10-09T21:49:35.347084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:35.347296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:35.347325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:35.347540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:35.347575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.347579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.347580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.347581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.347581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.347582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.347583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.347583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.347584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.347587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.347594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.347596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.347598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.347600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.347602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.347604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.347606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.347608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.347612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.347617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.347621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.347625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.347627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.347630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.347633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.347636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.347638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.347641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.347644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.347647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.347651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.347654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.347656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.347659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.347662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.347665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.347681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:35.347925Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342480311533368:8093] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:35.353174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342480311533368:8093] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:35.353185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342480311533368:8093] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:35.353188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342480311533368:8093] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:35.353195Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1563 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:35.353544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:35.354864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:35.354879Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:35.354881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:35.354884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:35.354888Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1562 [2:7559342479949706205:4920]
2025-10-09T21:49:35.354897Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:35.354911Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:35.354912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:35.354915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:35.354916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:35.354929Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:35.354933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:35.354939Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:35.354978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.354983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.354985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.354986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:35.355038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:35.355044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.355051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.355057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.355066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:35.355081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342480311533256:2432], now have 1 active actors on pipe
2025-10-09T21:49:35.355087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342480311533408:8093], now have 1 active actors on pipe
2025-10-09T21:49:35.355093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342479949706199:4920]
2025-10-09T21:49:35.355095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342479949706199:4920]
2025-10-09T21:49:35.355096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342479949706199:4920]
2025-10-09T21:49:35.355105Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:35.355115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:35.355115Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:35.355117Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:35.355118Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:35.355128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:35.355131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:35.355156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342479949706199:4920]
2025-10-09T21:49:35.355170Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:35.355179Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:35.355154Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1562
2025-10-09T21:49:35.355161Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1562
2025-10-09T21:49:35.355203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.355209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.355229Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|62383afd-3c12b158-fa68aa6a-7ce02a4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:35.355239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:35.355242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:35.355244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:35.355252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:35.355259Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:35.355269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:35.355389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.355394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.355403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:35.355484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1563 partition: 0 MaxSeqNo: 25 sessionId: src_id|62383afd-3c12b158-fa68aa6a-7ce02a4_0
2025-10-09T21:49:35.355789Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32843f99-75227d23-42b13d2e-71d2a1ac_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|62383afd-3c12b158-fa68aa6a-7ce02a4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:35.358255Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62383afd-3c12b158-fa68aa6a-7ce02a4_0] Write session: try to update token
2025-10-09T21:49:35.358272Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62383afd-3c12b158-fa68aa6a-7ce02a4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:35.370778Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:35.370791Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:35.370799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342479949706199:4920]
2025-10-09T21:49:35.370830Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:35.370837Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:35.370981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:35.371092Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:35.371068Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1563 sessionId: src_id|62383afd-3c12b158-fa68aa6a-7ce02a4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:35.371226Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:35.371349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:35.371520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:35.371551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:35.373256Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:35.373400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.373411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.373437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:35.373517Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:35.378381Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:35.378393Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:35.378395Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:35.378396Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:35.378402Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342479949706199:4920]
2025-10-09T21:49:35.378405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342479949706199:4920]
2025-10-09T21:49:35.378408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342479949706199:4920]
2025-10-09T21:49:35.378436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:35.378445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.378447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:35.378449Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:35.378456Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:35.378459Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:35.378562Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:35.434740Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342480311533415:2585]
2025-10-09T21:49:35.435044Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 1562, pipe: [1:7559342480311533415:2585]
2025-10-09T21:49:35.521388Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:35.528695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.528720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.528870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.528936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.528995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.529052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.529093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.529106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.529177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.529233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.529291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.529349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.529392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.529401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.529458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.529510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.529587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.529642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.529682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.529690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.529748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.529802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.529860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.529920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.529964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.529972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.530045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.530103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.530167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.530227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.530330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.530337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.530396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.530452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.530508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.530565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.530603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.530611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.530673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.530730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.530789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.530846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.530885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.530889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.530946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.531003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.531058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.531115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.531148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.531154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.531210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.531263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.531318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.531370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.531410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.531420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.531476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:35.531529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:35.531587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:35.531644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:35.531687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:35.531695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:35.531747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342480311533415:2585], now have 1 active actors on pipe
2025-10-09T21:49:35.531977Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:35.531988Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:35.532012Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1563 reason: error at write: Disconnected sessionId: src_id|62383afd-3c12b158-fa68aa6a-7ce02a4_0
2025-10-09T21:49:35.532199Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1563 sessionId: src_id|62383afd-3c12b158-fa68aa6a-7ce02a4_0 is DEAD
2025-10-09T21:49:35.532345Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62383afd-3c12b158-fa68aa6a-7ce02a4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:35.532357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62383afd-3c12b158-fa68aa6a-7ce02a4_0] Write session will restart in 0.008014s
2025-10-09T21:49:35.532353Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:35.532381Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62383afd-3c12b158-fa68aa6a-7ce02a4_0] Write session: Do CDS request
2025-10-09T21:49:35.532387Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62383afd-3c12b158-fa68aa6a-7ce02a4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:35.532414Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342480311533415:2585]
2025-10-09T21:49:35.532719Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:35.541444Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62383afd-3c12b158-fa68aa6a-7ce02a4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:35.541551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:35.541577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1564
2025-10-09T21:49:35.541704Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1564 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:35.541727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1564 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:35.541744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1564 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:35.541750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:35.542064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1564 sessionId:  describe result for acl check
2025-10-09T21:49:35.542085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:35.542091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:35.542093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:35.542097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342480311533529:8090] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:35.542510Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342480311533529:8090] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:35.544552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:35.544818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342480311533529:8090] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:35.544913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:35.545057Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:35.545067Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:35.545078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:35.545127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:35.545164Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342479949706346:4915]
2025-10-09T21:49:35.545737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:35.545937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:35.545971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:35.546282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:35.546325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.546331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.546332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.546333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.546334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.546335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.546336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.546337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.546338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.546343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.546353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.546357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.546361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.546365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.546369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.546372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.546380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.546384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.546390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.546397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.546407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.546416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.546420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.546429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.546433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.546438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.546443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.546452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.546456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.546461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.546466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.546471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.546479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.546483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.546488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.546494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.546525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:35.551072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342480311533529:8090] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:35.551082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342480311533529:8090] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:35.551085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342480311533529:8090] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:35.551091Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1564 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:35.552339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:35.553533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:35.553545Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:35.553547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:35.553550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:35.553556Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1563 [2:7559342479949706346:4915]
2025-10-09T21:49:35.553574Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:35.553593Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:35.553599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:35.553602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:35.553605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:35.553619Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:35.553629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:35.553639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:35.553691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.553697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.553699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.553702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:35.553757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:35.553764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.553774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.553789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.553812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:35.553830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342480311533569:8090], now have 1 active actors on pipe
2025-10-09T21:49:35.553837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342480311533418:2432], now have 1 active actors on pipe
2025-10-09T21:49:35.553844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342479949706341:4915]
2025-10-09T21:49:35.553846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342479949706341:4915]
2025-10-09T21:49:35.553848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342479949706341:4915]
2025-10-09T21:49:35.553860Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:35.553852Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1563
2025-10-09T21:49:35.553862Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1563
2025-10-09T21:49:35.553877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:35.553893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:35.553894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:35.553897Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:35.553899Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:35.553900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:35.553932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.553942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.553953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342479949706341:4915]
2025-10-09T21:49:35.553963Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:35.553969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:35.553989Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d151572-2bc29912-130fc9e2-26348605_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:35.554000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:35.554004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:35.554005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:35.554015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:35.554023Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:35.554060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:35.554157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.554164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.554176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:35.554284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1564 partition: 0 MaxSeqNo: 25 sessionId: src_id|d151572-2bc29912-130fc9e2-26348605_0
2025-10-09T21:49:35.554583Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62383afd-3c12b158-fa68aa6a-7ce02a4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d151572-2bc29912-130fc9e2-26348605_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:35.556849Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d151572-2bc29912-130fc9e2-26348605_0] Write session: try to update token
2025-10-09T21:49:35.556863Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d151572-2bc29912-130fc9e2-26348605_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:35.567674Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:35.567690Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:35.567693Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:35.567695Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:35.567703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342479949706341:4915]
2025-10-09T21:49:35.567707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342479949706341:4915]
2025-10-09T21:49:35.567711Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342479949706341:4915]
2025-10-09T21:49:35.567748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:35.567747Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:35.567753Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:35.567753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.567756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:35.567764Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:35.567884Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:35.568529Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1564 sessionId: src_id|d151572-2bc29912-130fc9e2-26348605_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:35.570745Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:35.632372Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342480311533577:2585]
2025-10-09T21:49:35.632622Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 1563, pipe: [1:7559342480311533577:2585]
2025-10-09T21:49:35.724677Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:35.725157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.725170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.725212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342480311533577:2585], now have 1 active actors on pipe
2025-10-09T21:49:35.725395Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:35.725463Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:35.725481Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:35.725489Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342480311533577:2585]
2025-10-09T21:49:35.725522Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:35.725534Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1564 reason: error at write: Disconnected sessionId: src_id|d151572-2bc29912-130fc9e2-26348605_0
2025-10-09T21:49:35.725715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1564 sessionId: src_id|d151572-2bc29912-130fc9e2-26348605_0 is DEAD
2025-10-09T21:49:35.725933Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d151572-2bc29912-130fc9e2-26348605_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:35.725947Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d151572-2bc29912-130fc9e2-26348605_0] Write session will restart in 0.007781s
2025-10-09T21:49:35.725975Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d151572-2bc29912-130fc9e2-26348605_0] Write session: Do CDS request
2025-10-09T21:49:35.725982Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d151572-2bc29912-130fc9e2-26348605_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:35.734494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d151572-2bc29912-130fc9e2-26348605_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:35.734647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:35.734658Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1565
2025-10-09T21:49:35.734767Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1565 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:35.734791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1565 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:35.734795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1565 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:35.734798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:35.735135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:35.735261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1565 sessionId:  describe result for acl check
2025-10-09T21:49:35.735308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:35.735314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:35.735316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:35.735322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342480311533693:8087] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:35.735830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342480311533693:8087] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:35.737453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:35.737605Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:35.737613Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:35.737622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:35.737601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342480311533693:8087] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:35.737659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:35.737698Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342479949706491:4906]
2025-10-09T21:49:35.738123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:35.738416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:35.738449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:35.738704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:35.738742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.738746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.738747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.738748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.738748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.738749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.738749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.738750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.738751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.738754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.738760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.738765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.738769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.738772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.738776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.738779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.738783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.738787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.738793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.738811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.738821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.738831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.738839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.738846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.738855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.738862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.738871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.738878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.738881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.738885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.738890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.738894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.738896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.738899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.738905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.738908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.738924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:35.743644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342480311533693:8087] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:35.743662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342480311533693:8087] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:35.743664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342480311533693:8087] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:35.743671Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1565 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:35.744969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:35.745977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:35.745991Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:35.745992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:35.745996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:35.746001Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1564 [2:7559342479949706491:4906]
2025-10-09T21:49:35.746010Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:35.746024Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:35.746026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:35.746041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:35.746042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:35.746054Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:35.746062Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:35.746067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:35.746112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.746118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.746119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.746121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:35.746172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:35.746177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.746185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.746196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.746217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:35.746237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342480311533580:2432], now have 1 active actors on pipe
2025-10-09T21:49:35.746243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342480311533733:8087], now have 1 active actors on pipe
2025-10-09T21:49:35.746248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342479949706485:4906]
2025-10-09T21:49:35.746250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342479949706485:4906]
2025-10-09T21:49:35.746251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342479949706485:4906]
2025-10-09T21:49:35.746260Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:35.746272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:35.746273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:35.746280Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:35.746281Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:35.746282Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:35.746283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:35.746302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342479949706485:4906]
2025-10-09T21:49:35.746308Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:35.746292Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1564
2025-10-09T21:49:35.746310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:35.746299Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1564
2025-10-09T21:49:35.746325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.746328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.746355Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c5cdb2dd-d35a92f8-6dab2445-64e1b4b2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:35.746364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:35.746367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:35.746368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:35.746375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:35.746383Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:35.746395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:35.746532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.746537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.746547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:35.746612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1565 partition: 0 MaxSeqNo: 25 sessionId: src_id|c5cdb2dd-d35a92f8-6dab2445-64e1b4b2_0
2025-10-09T21:49:35.746935Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d151572-2bc29912-130fc9e2-26348605_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c5cdb2dd-d35a92f8-6dab2445-64e1b4b2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:35.749228Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c5cdb2dd-d35a92f8-6dab2445-64e1b4b2_0] Write session: try to update token
2025-10-09T21:49:35.749242Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c5cdb2dd-d35a92f8-6dab2445-64e1b4b2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:35.760116Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:35.760133Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:35.760135Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:35.760137Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:35.760147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342479949706485:4906]
2025-10-09T21:49:35.760151Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342479949706485:4906]
2025-10-09T21:49:35.760155Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342479949706485:4906]
2025-10-09T21:49:35.760196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:35.760211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.760213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:35.760220Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:35.760229Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:35.760253Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:35.760335Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:35.761507Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1565 sessionId: src_id|c5cdb2dd-d35a92f8-6dab2445-64e1b4b2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:35.763704Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:35.825382Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342480311533740:2585]
2025-10-09T21:49:35.825682Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 1564, pipe: [1:7559342480311533740:2585]
2025-10-09T21:49:35.900607Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:35.901256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.901278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.901332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342480311533740:2585], now have 1 active actors on pipe
2025-10-09T21:49:35.901494Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:35.901522Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:35.901546Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342480311533740:2585]
2025-10-09T21:49:35.901563Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:35.901661Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1565 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c5cdb2dd-d35a92f8-6dab2445-64e1b4b2_0
2025-10-09T21:49:35.901844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1565 sessionId: src_id|c5cdb2dd-d35a92f8-6dab2445-64e1b4b2_0 is DEAD
2025-10-09T21:49:35.901985Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5cdb2dd-d35a92f8-6dab2445-64e1b4b2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:35.901997Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5cdb2dd-d35a92f8-6dab2445-64e1b4b2_0] Write session will restart in 0.006190s
2025-10-09T21:49:35.902026Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5cdb2dd-d35a92f8-6dab2445-64e1b4b2_0] Write session: Do CDS request
2025-10-09T21:49:35.902049Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5cdb2dd-d35a92f8-6dab2445-64e1b4b2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:35.909529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c5cdb2dd-d35a92f8-6dab2445-64e1b4b2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:35.909679Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:35.909691Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1566
2025-10-09T21:49:35.909801Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1566 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:35.909834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1566 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:35.909841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1566 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:35.909844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:35.910287Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1566 sessionId:  describe result for acl check
2025-10-09T21:49:35.910325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:35.910330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:35.910332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:35.910338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342480311533857:8080] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:35.910897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342480311533857:8080] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:35.912240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:35.912654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342480311533857:8080] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:35.914899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:35.915082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342480311533857:8080] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:35.915102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342480311533857:8080] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:35.915106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342480311533857:8080] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:35.915113Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1566 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:35.915099Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:35.915109Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:35.915121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:35.915170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:35.915211Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342479949706633:4900]
2025-10-09T21:49:35.915805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:35.916109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:35.916167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:35.916509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:35.916557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.916563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.916564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.916565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.916566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.916567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.916569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.916570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.916572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.916577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.916587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.916596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.916600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.916604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.916608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.916612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.916621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.916625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.916636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.916649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:35.916659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.916668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:35.916678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.916687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:35.916696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.916705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:35.916714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.916723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:35.916732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.916737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:35.916747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.916756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.916765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.916772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.916775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.916779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.916797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:35.923089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:35.924395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:35.924408Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:35.924410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:35.924413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:35.924419Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1565 [2:7559342479949706633:4900]
2025-10-09T21:49:35.924432Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:35.924449Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:35.924454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:35.924457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:35.924458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:35.924471Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:35.924480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:35.924489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:35.924531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.924536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.924537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.924539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:35.924597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:35.924604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.924618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:35.924627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:35.924639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:35.924657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342480311533897:8080], now have 1 active actors on pipe
2025-10-09T21:49:35.924663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342480311533746:2432], now have 1 active actors on pipe
2025-10-09T21:49:35.924668Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342479949706627:4900]
2025-10-09T21:49:35.924674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342479949706627:4900]
2025-10-09T21:49:35.924676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342479949706627:4900]
2025-10-09T21:49:35.924689Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:35.924707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:35.924713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:35.924716Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:35.924718Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:35.924722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:35.924724Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:35.924747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342479949706627:4900]
2025-10-09T21:49:35.924757Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:35.924766Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:35.924751Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1565
2025-10-09T21:49:35.924757Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1565
2025-10-09T21:49:35.924805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.924812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.924832Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|419ce40c-c23e5c49-27816491-97899c05_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:35.924841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:35.924844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:35.924848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:35.924855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:35.924866Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:35.924879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:35.925014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.925027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.925047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:35.925144Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1566 partition: 0 MaxSeqNo: 25 sessionId: src_id|419ce40c-c23e5c49-27816491-97899c05_0
2025-10-09T21:49:35.925469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5cdb2dd-d35a92f8-6dab2445-64e1b4b2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|419ce40c-c23e5c49-27816491-97899c05_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:35.927827Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|419ce40c-c23e5c49-27816491-97899c05_0] Write session: try to update token
2025-10-09T21:49:35.927840Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|419ce40c-c23e5c49-27816491-97899c05_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:35.935643Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:35.935656Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:35.935664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342479949706627:4900]
2025-10-09T21:49:35.935705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:35.935713Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:35.935863Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:35.935992Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:35.936114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:35.936241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:35.936406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:35.936439Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:35.939545Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1566 sessionId: src_id|419ce40c-c23e5c49-27816491-97899c05_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:35.941760Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:35.941930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:35.941946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:35.941981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:35.942062Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:35.948445Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:35.948464Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:35.948466Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:35.948471Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:35.948480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342479949706627:4900]
2025-10-09T21:49:35.948484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342479949706627:4900]
2025-10-09T21:49:35.948487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342479949706627:4900]
2025-10-09T21:49:35.948526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:35.948538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:35.948540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:35.948595Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:35.948608Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:35.948614Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:35.948670Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:36.001560Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342484606501205:2585]
2025-10-09T21:49:36.001808Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 1565, pipe: [1:7559342484606501205:2585]
2025-10-09T21:49:36.091093Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:36.097594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:36.097619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:36.097761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.097818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.097878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.097936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.097978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.097993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.098068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.098126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.098185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.098243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.098283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.098290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.098345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.098400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.098460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.098529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.098574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.098586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.098647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.098706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.098761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.098821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.098861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.098866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.098924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.098979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.099036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.099092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.099132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.099137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.099193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.099246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.099302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.099357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.099398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.099404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.099460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.099514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.099569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.099624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.099663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.099672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.099739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.099806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.099867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.099923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.099963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.099968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.100024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.100077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.100135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.100198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.100238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.100247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.100303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.100356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.100412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.100471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.100515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.100521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.100561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342484606501205:2585], now have 1 active actors on pipe
2025-10-09T21:49:36.100813Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:36.100816Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342484606501205:2585]
2025-10-09T21:49:36.100825Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:36.100837Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:36.100860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1566 reason: error at write: Disconnected sessionId: src_id|419ce40c-c23e5c49-27816491-97899c05_0
2025-10-09T21:49:36.101015Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1566 sessionId: src_id|419ce40c-c23e5c49-27816491-97899c05_0 is DEAD
2025-10-09T21:49:36.101211Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|419ce40c-c23e5c49-27816491-97899c05_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:36.101219Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|419ce40c-c23e5c49-27816491-97899c05_0] Write session will restart in 0.005882s
2025-10-09T21:49:36.101237Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|419ce40c-c23e5c49-27816491-97899c05_0] Write session: Do CDS request
2025-10-09T21:49:36.101242Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|419ce40c-c23e5c49-27816491-97899c05_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:36.101338Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:36.108513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|419ce40c-c23e5c49-27816491-97899c05_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:36.108555Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:36.108580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1567
2025-10-09T21:49:36.108798Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1567 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:36.108834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1567 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:36.108856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1567 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:36.108866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:36.109241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1567 sessionId:  describe result for acl check
2025-10-09T21:49:36.109272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:36.109274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:36.109275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:36.109280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342484606501307:8075] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:36.109710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342484606501307:8075] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:36.111120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:36.111705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342484606501307:8075] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:36.113365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:36.113531Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:36.113538Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:36.113546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:36.113594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:36.113628Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342484244674072:4889]
2025-10-09T21:49:36.114010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:36.114106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342484606501307:8075] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:36.114116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342484606501307:8075] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:36.114119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342484606501307:8075] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:36.114125Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1567 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:36.114252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:36.114280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:36.114496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:36.114538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.114544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.114545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.114546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.114547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.114548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.114549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.114549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.114550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.114555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.114565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.114569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.114574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.114577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.114585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.114588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.114597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.114601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.114612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.114619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.114626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.114630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.114633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.114636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.114638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.114643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.114647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.114652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.114662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.114667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.114676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.114681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.114690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.114693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.114695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.114698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.114715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:36.120741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:36.121775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:36.121787Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:36.121789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:36.121793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:36.121798Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1566 [2:7559342484244674072:4889]
2025-10-09T21:49:36.121807Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:36.121821Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:36.121823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:36.121825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:36.121826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:36.121839Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:36.121843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:36.121848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:36.121890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.121896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.121898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.121900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:36.121960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:36.121970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.121981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.121991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.122007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:36.122027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342484606501207:2432], now have 1 active actors on pipe
2025-10-09T21:49:36.122047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342484606501360:8075], now have 1 active actors on pipe
2025-10-09T21:49:36.122059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342484244674066:4889]
2025-10-09T21:49:36.122061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342484244674066:4889]
2025-10-09T21:49:36.122063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342484244674066:4889]
2025-10-09T21:49:36.122075Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:36.122087Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:36.122090Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:36.122092Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:36.122094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:36.122108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:36.122115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:36.122115Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1566
2025-10-09T21:49:36.122142Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342484244674066:4889]
2025-10-09T21:49:36.122121Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1566
2025-10-09T21:49:36.122149Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:36.122150Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:36.122164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:36.122168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:36.122213Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|73cb1062-42c14e27-5ce91cf1-2133c999_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:36.122227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:36.122235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:36.122237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:36.122248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:36.122262Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:36.122307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:36.122453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:36.122460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:36.122476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:36.122561Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1567 partition: 0 MaxSeqNo: 25 sessionId: src_id|73cb1062-42c14e27-5ce91cf1-2133c999_0
2025-10-09T21:49:36.122901Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|419ce40c-c23e5c49-27816491-97899c05_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|73cb1062-42c14e27-5ce91cf1-2133c999_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:36.125268Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73cb1062-42c14e27-5ce91cf1-2133c999_0] Write session: try to update token
2025-10-09T21:49:36.125284Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73cb1062-42c14e27-5ce91cf1-2133c999_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:36.136800Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:36.136816Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:36.136818Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:36.136819Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:36.136827Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342484244674066:4889]
2025-10-09T21:49:36.136831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342484244674066:4889]
2025-10-09T21:49:36.136834Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342484244674066:4889]
2025-10-09T21:49:36.136879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:36.136887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.136890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:36.136891Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:36.136900Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:36.136904Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:36.137026Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:36.137788Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1567 sessionId: src_id|73cb1062-42c14e27-5ce91cf1-2133c999_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:36.140008Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:36.201212Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342484606501373:2585]
2025-10-09T21:49:36.201445Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 1566, pipe: [1:7559342484606501373:2585]
2025-10-09T21:49:36.285626Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:36.285788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:36.285800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:36.285833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342484606501373:2585], now have 1 active actors on pipe
2025-10-09T21:49:36.285976Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:36.286046Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:36.286049Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:36.286058Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:36.286088Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342484606501373:2585]
2025-10-09T21:49:36.286124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1567 reason: error at write: Disconnected sessionId: src_id|73cb1062-42c14e27-5ce91cf1-2133c999_0
2025-10-09T21:49:36.286302Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1567 sessionId: src_id|73cb1062-42c14e27-5ce91cf1-2133c999_0 is DEAD
2025-10-09T21:49:36.286471Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73cb1062-42c14e27-5ce91cf1-2133c999_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:36.286482Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73cb1062-42c14e27-5ce91cf1-2133c999_0] Write session will restart in 0.005025s
2025-10-09T21:49:36.286507Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73cb1062-42c14e27-5ce91cf1-2133c999_0] Write session: Do CDS request
2025-10-09T21:49:36.286514Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73cb1062-42c14e27-5ce91cf1-2133c999_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:36.292398Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73cb1062-42c14e27-5ce91cf1-2133c999_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:36.292495Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:36.292507Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1568
2025-10-09T21:49:36.292640Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1568 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:36.292679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1568 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:36.292688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1568 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:36.292692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:36.293043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1568 sessionId:  describe result for acl check
2025-10-09T21:49:36.293073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:36.293080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:36.293081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:36.293087Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342484606501465:8054] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:36.293475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342484606501465:8054] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:36.295049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342484606501465:8054] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:36.295884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:36.298325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:36.298509Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:36.298518Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:36.298529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:36.298579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:36.298616Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342484244674254:4867]
2025-10-09T21:49:36.298711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342484606501465:8054] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:36.298725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342484606501465:8054] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:36.298729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342484606501465:8054] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:36.298735Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1568 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:36.299229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:36.299493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:36.299540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:36.299814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:36.299868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.299873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.299873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.299874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.299875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.299875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.299876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.299877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.299877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.299881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.299887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.299889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.299891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.299893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.299895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.299898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.299899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.299902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.299906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.299911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.299915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.299918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.299921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.299924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.299927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.299929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.299940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.299943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.299945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.299948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.299951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.299954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.299957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.299960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.299962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.299965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.299981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:36.306147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:36.307193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:36.307206Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:36.307208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:36.307211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:36.307217Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1567 [2:7559342484244674254:4867]
2025-10-09T21:49:36.307230Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:36.307246Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:36.307259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:36.307262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:36.307263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:36.307277Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:36.307285Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:36.307290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:36.307337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.307342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.307344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.307345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:36.307401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:36.307406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.307414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.307423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.307437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:36.307456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342484606501404:2432], now have 1 active actors on pipe
2025-10-09T21:49:36.307462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342484606501553:8054], now have 1 active actors on pipe
2025-10-09T21:49:36.307468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342484244674248:4867]
2025-10-09T21:49:36.307470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342484244674248:4867]
2025-10-09T21:49:36.307471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342484244674248:4867]
2025-10-09T21:49:36.307481Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:36.307493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:36.307497Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:36.307500Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:36.307502Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:36.307505Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:36.307508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:36.307535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342484244674248:4867]
2025-10-09T21:49:36.307540Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:36.307543Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:36.307563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:36.307566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:36.307523Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1567
2025-10-09T21:49:36.307592Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|74502f22-c206f5bd-7184226b-4913cd8f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:36.307536Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1567
2025-10-09T21:49:36.307604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:36.307608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:36.307611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:36.307622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:36.307635Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:36.307653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:36.307798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:36.307805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:36.307816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:36.307889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1568 partition: 0 MaxSeqNo: 25 sessionId: src_id|74502f22-c206f5bd-7184226b-4913cd8f_0
2025-10-09T21:49:36.308209Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73cb1062-42c14e27-5ce91cf1-2133c999_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|74502f22-c206f5bd-7184226b-4913cd8f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:36.310625Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74502f22-c206f5bd-7184226b-4913cd8f_0] Write session: try to update token
2025-10-09T21:49:36.310643Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74502f22-c206f5bd-7184226b-4913cd8f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:36.322699Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:36.322715Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:36.322717Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:36.322718Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:36.322726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342484244674248:4867]
2025-10-09T21:49:36.322729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342484244674248:4867]
2025-10-09T21:49:36.322732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342484244674248:4867]
2025-10-09T21:49:36.322768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:36.322774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.322776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:36.322781Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:36.322786Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:36.322792Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:36.322898Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:36.323936Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1568 sessionId: src_id|74502f22-c206f5bd-7184226b-4913cd8f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:36.326261Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:36.386049Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342484606501565:2585]
2025-10-09T21:49:36.386338Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 1567, pipe: [1:7559342484606501565:2585]
2025-10-09T21:49:36.471154Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:36.471452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:36.471468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:36.471515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342484606501565:2585], now have 1 active actors on pipe
2025-10-09T21:49:36.471803Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:36.471801Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:36.471835Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342484606501565:2585]
2025-10-09T21:49:36.471861Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:36.471871Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:36.471935Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1568 reason: error at write: Disconnected sessionId: src_id|74502f22-c206f5bd-7184226b-4913cd8f_0
2025-10-09T21:49:36.472111Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1568 sessionId: src_id|74502f22-c206f5bd-7184226b-4913cd8f_0 is DEAD
2025-10-09T21:49:36.472256Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74502f22-c206f5bd-7184226b-4913cd8f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:36.472268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74502f22-c206f5bd-7184226b-4913cd8f_0] Write session will restart in 0.006556s
2025-10-09T21:49:36.472295Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74502f22-c206f5bd-7184226b-4913cd8f_0] Write session: Do CDS request
2025-10-09T21:49:36.472301Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74502f22-c206f5bd-7184226b-4913cd8f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:36.479500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74502f22-c206f5bd-7184226b-4913cd8f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:36.479559Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:36.479569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1569
2025-10-09T21:49:36.479716Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1569 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:36.479767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1569 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:36.479777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1569 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:36.479781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:36.480146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1569 sessionId:  describe result for acl check
2025-10-09T21:49:36.480172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:36.480178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:36.480180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:36.480184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342484606501622:8053] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:36.480659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342484606501622:8053] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:36.482903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342484606501622:8053] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:36.486193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:36.486442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342484606501622:8053] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:36.486452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342484606501622:8053] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:36.486455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342484606501622:8053] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:36.486462Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1569 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:36.486601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:36.486745Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:36.486753Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:36.486761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:36.486804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:36.486836Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342484244674400:4875]
2025-10-09T21:49:36.487210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:36.487456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:36.487493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:36.487726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:36.487764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.487769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.487770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.487770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.487771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.487772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.487772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.487773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.487773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.487777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.487783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.487786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.487788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.487790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.487792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.487794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.487796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.487798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.487803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.487808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.487812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.487815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.487818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.487821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.487823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.487826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.487829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.487834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.487838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.487843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.487848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.487852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.487867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.487870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.487878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.487881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.487898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:36.494130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:36.495216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:36.495230Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:36.495232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:36.495235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:36.495239Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1568 [2:7559342484244674400:4875]
2025-10-09T21:49:36.495249Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:36.495262Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:36.495263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:36.495265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:36.495267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:36.495279Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:36.495287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:36.495292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:36.495331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.495336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.495338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.495340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:36.495388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:36.495394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.495401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.495407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.495427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:36.495451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342484606501567:2432], now have 1 active actors on pipe
2025-10-09T21:49:36.495463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342484244674394:4875]
2025-10-09T21:49:36.495465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342484244674394:4875]
2025-10-09T21:49:36.495467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342484244674394:4875]
2025-10-09T21:49:36.495476Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:36.495485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342484606501720:8053], now have 1 active actors on pipe
2025-10-09T21:49:36.495492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:36.495492Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:36.495495Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:36.495497Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:36.495504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:36.495506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:36.495526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342484244674394:4875]
2025-10-09T21:49:36.495532Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:36.495543Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:36.495537Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1568
2025-10-09T21:49:36.495554Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1568
2025-10-09T21:49:36.495618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:36.495625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:36.495645Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f96d6046-9013cecc-4e7aa40a-224be946_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:36.495654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:36.495661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:36.495662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:36.495670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:36.495681Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:36.495696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:36.495839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:36.495845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:36.495857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:36.495948Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1569 partition: 0 MaxSeqNo: 25 sessionId: src_id|f96d6046-9013cecc-4e7aa40a-224be946_0
2025-10-09T21:49:36.496231Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74502f22-c206f5bd-7184226b-4913cd8f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f96d6046-9013cecc-4e7aa40a-224be946_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:36.498895Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f96d6046-9013cecc-4e7aa40a-224be946_0] Write session: try to update token
2025-10-09T21:49:36.498909Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f96d6046-9013cecc-4e7aa40a-224be946_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:36.507003Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:36.507021Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:36.507030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342484244674394:4875]
2025-10-09T21:49:36.507065Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:36.507072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:36.507246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:36.507387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:36.507524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:36.507653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:36.507853Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:36.507895Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:36.512491Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1569 sessionId: src_id|f96d6046-9013cecc-4e7aa40a-224be946_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:36.514994Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:36.515159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:36.515179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:36.515229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:36.515302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:36.519810Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:36.519822Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:36.519824Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:36.519825Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:36.519833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342484244674394:4875]
2025-10-09T21:49:36.519837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342484244674394:4875]
2025-10-09T21:49:36.519839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342484244674394:4875]
2025-10-09T21:49:36.519875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:36.519880Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:36.519883Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:36.519886Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:36.519888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.519891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:36.520026Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:36.571894Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342484606501732:2585]
2025-10-09T21:49:36.572194Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 1568, pipe: [1:7559342484606501732:2585]
2025-10-09T21:49:36.671213Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:36.679010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:36.679034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:36.679192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.679253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.679312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.679380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.679517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.679527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.679591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.679649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.679713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.679779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.679819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.679827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.679889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.679947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.680006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.680061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.680104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.680112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.680175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.680235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.680296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.680359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.680400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.680407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.680475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.680538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.680596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.680652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.680694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.680705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.680774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.680834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.680895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.680958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.680998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.681006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.681071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.681127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.681184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.681243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.681283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.681290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.681349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.681404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.681465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.681519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.681643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.681648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.681706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.681761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.681813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.681865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.681910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.681918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.681979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:36.682052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:36.682110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:36.682163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:36.682200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:36.682205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:36.682263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342484606501732:2585], now have 1 active actors on pipe
2025-10-09T21:49:36.682494Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:36.682550Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:36.682554Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:36.682566Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342484606501732:2585]
2025-10-09T21:49:36.682582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1569 reason: error at write: Disconnected sessionId: src_id|f96d6046-9013cecc-4e7aa40a-224be946_0
2025-10-09T21:49:36.682784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1569 sessionId: src_id|f96d6046-9013cecc-4e7aa40a-224be946_0 is DEAD
2025-10-09T21:49:36.682925Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f96d6046-9013cecc-4e7aa40a-224be946_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:36.682949Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f96d6046-9013cecc-4e7aa40a-224be946_0] Write session will restart in 0.007639s
2025-10-09T21:49:36.682969Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f96d6046-9013cecc-4e7aa40a-224be946_0] Write session: Do CDS request
2025-10-09T21:49:36.682974Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f96d6046-9013cecc-4e7aa40a-224be946_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:36.683324Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:36.691485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f96d6046-9013cecc-4e7aa40a-224be946_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:36.691599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:36.691612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1570
2025-10-09T21:49:36.691758Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1570 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:36.691792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1570 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:36.691798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1570 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:36.691803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:36.692211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1570 sessionId:  describe result for acl check
2025-10-09T21:49:36.692245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:36.692251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:36.692253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:36.692259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342484606501845:8042] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:36.692817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342484606501845:8042] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:36.694151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:36.694757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342484606501845:8042] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:36.696939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:36.697190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342484606501845:8042] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:36.697197Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:36.697208Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:36.697220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:36.697202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342484606501845:8042] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:36.697205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342484606501845:8042] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:36.697213Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1570 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:36.697289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:36.697329Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342484244674546:4873]
2025-10-09T21:49:36.697932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:36.698200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:36.698240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:36.698555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:36.698609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.698615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.698616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.698617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.698619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.698621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.698622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.698623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.698624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.698629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.698640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.698644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.698647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.698651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.698655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.698659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.698663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.698667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.698674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.698682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.698694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.698699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.698709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.698715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.698722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.698728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.698736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.698742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.698746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.698751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.698756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.698766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.698770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.698776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.698786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.698792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.698825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:36.705166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:36.706208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:36.706222Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:36.706223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:36.706226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:36.706231Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1569 [2:7559342484244674546:4873]
2025-10-09T21:49:36.706241Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:36.706255Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:36.706257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:36.706259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:36.706260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:36.706273Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:36.706284Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:36.706296Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:36.706345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.706352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.706353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.706355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:36.706416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:36.706425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.706436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.706449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.706465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:36.706486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342484606501735:2432], now have 1 active actors on pipe
2025-10-09T21:49:36.706490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342484606501894:8042], now have 1 active actors on pipe
2025-10-09T21:49:36.706497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342484244674540:4873]
2025-10-09T21:49:36.706500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342484244674540:4873]
2025-10-09T21:49:36.706502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342484244674540:4873]
2025-10-09T21:49:36.706514Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:36.706528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:36.706539Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:36.706542Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:36.706544Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:36.706546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:36.706550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:36.706576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342484244674540:4873]
2025-10-09T21:49:36.706584Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:36.706546Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1569
2025-10-09T21:49:36.706585Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:36.706550Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1569
2025-10-09T21:49:36.706604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:36.706608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:36.706658Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6213e5b5-6f79843e-bc98f4c7-76e241ac_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:36.706668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:36.706672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:36.706681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:36.706691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:36.706701Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:36.706723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:36.706901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:36.706908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:36.706922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:36.707005Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1570 partition: 0 MaxSeqNo: 25 sessionId: src_id|6213e5b5-6f79843e-bc98f4c7-76e241ac_0
2025-10-09T21:49:36.707379Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f96d6046-9013cecc-4e7aa40a-224be946_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6213e5b5-6f79843e-bc98f4c7-76e241ac_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:36.709726Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6213e5b5-6f79843e-bc98f4c7-76e241ac_0] Write session: try to update token
2025-10-09T21:49:36.709740Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6213e5b5-6f79843e-bc98f4c7-76e241ac_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:36.720559Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:36.720577Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:36.720580Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:36.720582Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:36.720592Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342484244674540:4873]
2025-10-09T21:49:36.720597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342484244674540:4873]
2025-10-09T21:49:36.720601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342484244674540:4873]
2025-10-09T21:49:36.720644Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:36.720648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:36.720649Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:36.720654Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:36.720654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.720657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:36.720790Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:36.722599Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1570 sessionId: src_id|6213e5b5-6f79843e-bc98f4c7-76e241ac_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:36.724990Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:36.782809Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342484606501913:2585]
2025-10-09T21:49:36.783076Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 1569, pipe: [1:7559342484606501913:2585]
2025-10-09T21:49:36.877283Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:36.877910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:36.877924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:36.877962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342484606501913:2585], now have 1 active actors on pipe
2025-10-09T21:49:36.878202Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:36.878289Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342484606501913:2585]
2025-10-09T21:49:36.878296Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:36.878322Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:36.878396Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1570 reason: status is not ok: tablet will be restarted right now sessionId: src_id|6213e5b5-6f79843e-bc98f4c7-76e241ac_0
2025-10-09T21:49:36.878579Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1570 sessionId: src_id|6213e5b5-6f79843e-bc98f4c7-76e241ac_0 is DEAD
2025-10-09T21:49:36.878726Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6213e5b5-6f79843e-bc98f4c7-76e241ac_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:36.878738Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6213e5b5-6f79843e-bc98f4c7-76e241ac_0] Write session will restart in 0.006409s
2025-10-09T21:49:36.878761Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6213e5b5-6f79843e-bc98f4c7-76e241ac_0] Write session: Do CDS request
2025-10-09T21:49:36.878768Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6213e5b5-6f79843e-bc98f4c7-76e241ac_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:36.886502Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6213e5b5-6f79843e-bc98f4c7-76e241ac_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:36.886588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:36.886603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1571
2025-10-09T21:49:36.886742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1571 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:36.886774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1571 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:36.886785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1571 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:36.886788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:36.887211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1571 sessionId:  describe result for acl check
2025-10-09T21:49:36.887242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:36.887248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:36.887250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:36.887256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342484606502006:8039] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:36.887783Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342484606502006:8039] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:36.889675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342484606502006:8039] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:36.892355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:36.892599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342484606502006:8039] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:36.892611Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342484606502006:8039] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:36.892615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342484606502006:8039] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:36.892622Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1571 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:36.892755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:36.892967Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:36.892977Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:36.892989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:36.893036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:36.893073Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342484244674699:4869]
2025-10-09T21:49:36.893660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:36.893977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:36.894050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:36.894385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:36.894438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.894444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.894445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.894446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.894447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.894448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.894450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.894451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.894452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.894457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.894467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.894471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.894474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.894478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.894482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.894486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.894494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.894498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.894508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.894516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:36.894526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.894545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:36.894554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.894563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:36.894568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.894576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:36.894581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.894590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:36.894599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.894608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:36.894617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.894625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.894634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.894643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.894652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.894661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.894694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:36.901032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:36.902125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:36.902138Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:36.902140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:36.902143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:36.902148Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1570 [2:7559342484244674699:4869]
2025-10-09T21:49:36.902160Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:36.902174Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:36.902176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:36.902179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:36.902180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:36.902194Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:36.902198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:36.902204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:36.902247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.902253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.902254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.902256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:36.902320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:36.902328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.902336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:36.902342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:36.902352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:36.902369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342484606501918:2432], now have 1 active actors on pipe
2025-10-09T21:49:36.902376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342484606502073:8039], now have 1 active actors on pipe
2025-10-09T21:49:36.902383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342484244674693:4869]
2025-10-09T21:49:36.902386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342484244674693:4869]
2025-10-09T21:49:36.902388Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342484244674693:4869]
2025-10-09T21:49:36.902402Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:36.902417Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:36.902419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:36.902427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:36.902429Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:36.902431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:36.902435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:36.902444Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1570
2025-10-09T21:49:36.902468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342484244674693:4869]
2025-10-09T21:49:36.902476Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:36.902478Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:36.902455Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1570
2025-10-09T21:49:36.902503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:36.902512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:36.902538Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|129fc03f-cf9ca3c1-f233e96f-93bc1874_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:36.902548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:36.902552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:36.902554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:36.902561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:36.902568Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:36.902584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:36.902713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:36.902719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:36.902728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:36.902809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1571 partition: 0 MaxSeqNo: 25 sessionId: src_id|129fc03f-cf9ca3c1-f233e96f-93bc1874_0
2025-10-09T21:49:36.903158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6213e5b5-6f79843e-bc98f4c7-76e241ac_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|129fc03f-cf9ca3c1-f233e96f-93bc1874_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:36.905538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|129fc03f-cf9ca3c1-f233e96f-93bc1874_0] Write session: try to update token
2025-10-09T21:49:36.905554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|129fc03f-cf9ca3c1-f233e96f-93bc1874_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:36.916457Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:36.916475Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:36.916477Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:36.916479Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:36.916490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342484244674693:4869]
2025-10-09T21:49:36.916495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342484244674693:4869]
2025-10-09T21:49:36.916498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342484244674693:4869]
2025-10-09T21:49:36.916537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:36.916544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:36.916545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:36.916545Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:36.916554Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:36.916558Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:36.916667Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:36.918856Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1571 sessionId: src_id|129fc03f-cf9ca3c1-f233e96f-93bc1874_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:36.921117Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:36.978419Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342484606502084:2585]
2025-10-09T21:49:36.978698Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 1570, pipe: [1:7559342484606502084:2585]
2025-10-09T21:49:37.059051Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:37.059637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.059649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.059686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342484606502084:2585], now have 1 active actors on pipe
2025-10-09T21:49:37.059886Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:37.059920Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342484606502084:2585]
2025-10-09T21:49:37.059920Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:37.059955Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:37.060029Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1571 reason: status is not ok: tablet will be restarted right now sessionId: src_id|129fc03f-cf9ca3c1-f233e96f-93bc1874_0
2025-10-09T21:49:37.060212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1571 sessionId: src_id|129fc03f-cf9ca3c1-f233e96f-93bc1874_0 is DEAD
2025-10-09T21:49:37.060343Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|129fc03f-cf9ca3c1-f233e96f-93bc1874_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:37.060353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|129fc03f-cf9ca3c1-f233e96f-93bc1874_0] Write session will restart in 0.006200s
2025-10-09T21:49:37.060375Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|129fc03f-cf9ca3c1-f233e96f-93bc1874_0] Write session: Do CDS request
2025-10-09T21:49:37.060379Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|129fc03f-cf9ca3c1-f233e96f-93bc1874_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:37.067494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|129fc03f-cf9ca3c1-f233e96f-93bc1874_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:37.067539Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:37.067554Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1572
2025-10-09T21:49:37.067713Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1572 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:37.067745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1572 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:37.067756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1572 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:37.067760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:37.068184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1572 sessionId:  describe result for acl check
2025-10-09T21:49:37.068218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:37.068223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:37.068226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:37.068231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342488901469501:8031] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:37.068784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342488901469501:8031] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:37.069753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:37.070902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342488901469501:8031] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:37.072160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:37.072310Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:37.072317Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:37.072326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:37.072365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:37.072395Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342488539642142:4862]
2025-10-09T21:49:37.072757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:37.073010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:37.073046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:37.073078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342488901469501:8031] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:37.073091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342488901469501:8031] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:37.073094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342488901469501:8031] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:37.073101Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1572 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:37.073283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:37.073322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.073327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.073328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.073329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.073329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.073330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.073331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.073331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.073332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.073335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.073341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.073344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.073346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.073348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.073350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.073352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.073354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.073356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.073360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.073365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.073369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.073372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.073375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.073378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.073380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.073383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.073386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.073388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.073391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.073394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.073397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.073400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.073403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.073406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.073409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.073412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.073429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:37.079591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:37.080838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:37.080850Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:37.080851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:37.080854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:37.080858Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1571 [2:7559342488539642142:4862]
2025-10-09T21:49:37.080866Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:37.080878Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:37.080885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:37.080888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:37.080890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:37.080904Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:37.080914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:37.080921Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:37.080963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.080971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.080972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.080974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:37.081033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:37.081041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.081051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.081077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.081098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:37.081124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901469383:2432], now have 1 active actors on pipe
2025-10-09T21:49:37.081130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901469541:8031], now have 1 active actors on pipe
2025-10-09T21:49:37.081135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342488539642136:4862]
2025-10-09T21:49:37.081136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342488539642136:4862]
2025-10-09T21:49:37.081138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342488539642136:4862]
2025-10-09T21:49:37.081148Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:37.081161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1571
2025-10-09T21:49:37.081166Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1571
2025-10-09T21:49:37.081164Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:37.081167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:37.081169Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:37.081169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:37.081182Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:37.081185Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:37.081203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342488539642136:4862]
2025-10-09T21:49:37.081205Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:37.081207Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:37.081222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.081226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.081271Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|738f34bd-4b8b20de-d0255ce-839359e5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:37.081284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:37.081288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:37.081290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:37.081299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:37.081312Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:37.081331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:37.081466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.081473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.081487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:37.081608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1572 partition: 0 MaxSeqNo: 25 sessionId: src_id|738f34bd-4b8b20de-d0255ce-839359e5_0
2025-10-09T21:49:37.081922Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|129fc03f-cf9ca3c1-f233e96f-93bc1874_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|738f34bd-4b8b20de-d0255ce-839359e5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:37.084437Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|738f34bd-4b8b20de-d0255ce-839359e5_0] Write session: try to update token
2025-10-09T21:49:37.084452Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|738f34bd-4b8b20de-d0255ce-839359e5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:37.095497Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:37.095514Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:37.095516Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:37.095517Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:37.095525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342488539642136:4862]
2025-10-09T21:49:37.095528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342488539642136:4862]
2025-10-09T21:49:37.095531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342488539642136:4862]
2025-10-09T21:49:37.095566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:37.095572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.095575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:37.095577Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:37.095633Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:37.095646Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:37.095703Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:37.097186Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1572 sessionId: src_id|738f34bd-4b8b20de-d0255ce-839359e5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:37.099661Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:37.160184Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342488901469552:2585]
2025-10-09T21:49:37.160470Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 1571, pipe: [1:7559342488901469552:2585]
2025-10-09T21:49:37.240485Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:37.240720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.240734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.240775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901469552:2585], now have 1 active actors on pipe
2025-10-09T21:49:37.240935Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:37.241029Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:37.241059Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342488901469552:2585]
2025-10-09T21:49:37.241099Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:37.241179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1572 reason: status is not ok: tablet will be restarted right now sessionId: src_id|738f34bd-4b8b20de-d0255ce-839359e5_0
2025-10-09T21:49:37.241356Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1572 sessionId: src_id|738f34bd-4b8b20de-d0255ce-839359e5_0 is DEAD
2025-10-09T21:49:37.241498Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|738f34bd-4b8b20de-d0255ce-839359e5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:37.241511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|738f34bd-4b8b20de-d0255ce-839359e5_0] Write session will restart in 0.009989s
2025-10-09T21:49:37.241538Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|738f34bd-4b8b20de-d0255ce-839359e5_0] Write session: Do CDS request
2025-10-09T21:49:37.241544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|738f34bd-4b8b20de-d0255ce-839359e5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:37.251901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:37.252431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|738f34bd-4b8b20de-d0255ce-839359e5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:37.252505Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:37.252516Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1573
2025-10-09T21:49:37.252621Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1573 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:37.252646Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1573 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:37.252653Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1573 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:37.252657Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:37.253080Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1573 sessionId:  describe result for acl check
2025-10-09T21:49:37.253120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:37.253128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:37.253130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:37.253135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342488901469686:8027] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:37.253696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342488901469686:8027] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:37.254396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:37.254634Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:37.254649Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:37.254663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:37.254725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:37.254783Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342488539642291:4863]
2025-10-09T21:49:37.255332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:37.255566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:37.255598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:37.255581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342488901469686:8027] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:37.255815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:37.255846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.255850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.255851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.255852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.255852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.255853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.255854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.255854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.255855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.255858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.255863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.255866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.255868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.255870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.255872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.255874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.255876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.255878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.255884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.255888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.255893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.255896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.255898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.255901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.255904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.255907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.255909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.255912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.255915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.255918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.255921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.255924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.255926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.255929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.255933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.255936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.255952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:37.261719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342488901469686:8027] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:37.261732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342488901469686:8027] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:37.261735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342488901469686:8027] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:37.261744Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1573 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:37.262057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:37.263132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:37.263146Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:37.263148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:37.263150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:37.263155Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1572 [2:7559342488539642291:4863]
2025-10-09T21:49:37.263163Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:37.263175Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:37.263180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:37.263183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:37.263184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:37.263196Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:37.263200Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:37.263205Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:37.263242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.263247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.263249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.263251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:37.263299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:37.263305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.263312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.263318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.263329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:37.263346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901469723:8027], now have 1 active actors on pipe
2025-10-09T21:49:37.263352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901469558:2432], now have 1 active actors on pipe
2025-10-09T21:49:37.263357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342488539642285:4863]
2025-10-09T21:49:37.263359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342488539642285:4863]
2025-10-09T21:49:37.263360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342488539642285:4863]
2025-10-09T21:49:37.263371Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:37.263386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:37.263399Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:37.263387Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1572
2025-10-09T21:49:37.263402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:37.263390Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1572
2025-10-09T21:49:37.263405Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:37.263410Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:37.263412Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:37.263427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342488539642285:4863]
2025-10-09T21:49:37.263430Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:37.263433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:37.263448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.263455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.263490Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|47dd464f-f143f211-4106e0df-f9f91411_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:37.263499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:37.263502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:37.263503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:37.263510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:37.263521Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:37.263536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:37.263674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.263680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.263690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:37.263756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1573 partition: 0 MaxSeqNo: 25 sessionId: src_id|47dd464f-f143f211-4106e0df-f9f91411_0
2025-10-09T21:49:37.264070Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|738f34bd-4b8b20de-d0255ce-839359e5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|47dd464f-f143f211-4106e0df-f9f91411_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:37.266578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|47dd464f-f143f211-4106e0df-f9f91411_0] Write session: try to update token
2025-10-09T21:49:37.266603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|47dd464f-f143f211-4106e0df-f9f91411_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:37.277606Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:37.277622Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:37.277624Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:37.277625Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:37.277632Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342488539642285:4863]
2025-10-09T21:49:37.277635Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342488539642285:4863]
2025-10-09T21:49:37.277638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342488539642285:4863]
2025-10-09T21:49:37.277671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:37.277677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.277679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:37.277680Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:37.277685Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:37.277692Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:37.278312Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:37.280242Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1573 sessionId: src_id|47dd464f-f143f211-4106e0df-f9f91411_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:37.282686Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:37.341416Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342488901469753:2585]
2025-10-09T21:49:37.341616Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 1572, pipe: [1:7559342488901469753:2585]
2025-10-09T21:49:37.420327Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:37.420965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.420980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.421018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901469753:2585], now have 1 active actors on pipe
2025-10-09T21:49:37.421181Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:37.421254Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:37.421256Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:37.421260Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:37.421315Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342488901469753:2585]
2025-10-09T21:49:37.421321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1573 reason: error at write: Disconnected sessionId: src_id|47dd464f-f143f211-4106e0df-f9f91411_0
2025-10-09T21:49:37.421493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1573 sessionId: src_id|47dd464f-f143f211-4106e0df-f9f91411_0 is DEAD
2025-10-09T21:49:37.421635Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|47dd464f-f143f211-4106e0df-f9f91411_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:37.421645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|47dd464f-f143f211-4106e0df-f9f91411_0] Write session will restart in 0.009738s
2025-10-09T21:49:37.421665Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|47dd464f-f143f211-4106e0df-f9f91411_0] Write session: Do CDS request
2025-10-09T21:49:37.421670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|47dd464f-f143f211-4106e0df-f9f91411_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:37.430872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:37.432430Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|47dd464f-f143f211-4106e0df-f9f91411_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:37.432816Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:37.432827Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1574
2025-10-09T21:49:37.432922Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1574 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:37.432953Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1574 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:37.432963Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1574 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:37.432967Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:37.433228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:37.433324Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1574 sessionId:  describe result for acl check
2025-10-09T21:49:37.433351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:37.433356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:37.433357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:37.433361Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:37.433361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342488901469880:8009] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:37.433369Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:37.433377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:37.433411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:37.433437Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342488539642479:4813]
2025-10-09T21:49:37.433820Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342488901469880:8009] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:37.433839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:37.434162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:37.434206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:37.434455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:37.434490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.434495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.434495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.434496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.434497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.434497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.434498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.434498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.434499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.434502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.434508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.434510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.434512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.434514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.434516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.434518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.434520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.434522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.434526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.434531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.434535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.434538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.434541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.434544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.434546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.434549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.434551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.434554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.434556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.434559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.434562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.434565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.434568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.434571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.434575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.434578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.434594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:37.435747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342488901469880:8009] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:37.440321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342488901469880:8009] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:37.440332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342488901469880:8009] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:37.440336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342488901469880:8009] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:37.440343Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1574 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:37.440734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:37.441859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:37.441876Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:37.441878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:37.441882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:37.441888Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1573 [2:7559342488539642479:4813]
2025-10-09T21:49:37.441900Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:37.441918Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:37.441925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:37.441927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:37.441929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:37.441944Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:37.441954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:37.441965Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:37.442015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.442021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.442023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.442026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:37.442100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:37.442108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.442119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.442131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.442151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:37.442173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901469755:2432], now have 1 active actors on pipe
2025-10-09T21:49:37.442181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901469918:8009], now have 1 active actors on pipe
2025-10-09T21:49:37.442187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342488539642473:4813]
2025-10-09T21:49:37.442189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342488539642473:4813]
2025-10-09T21:49:37.442190Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342488539642473:4813]
2025-10-09T21:49:37.442172Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1573
2025-10-09T21:49:37.442199Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:37.442180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1573
2025-10-09T21:49:37.442206Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:37.442208Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:37.442209Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:37.442216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:37.442231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:37.442238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:37.442259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.442267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.442300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342488539642473:4813]
2025-10-09T21:49:37.442306Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:37.442310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:37.442333Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5de90dec-e1704473-937cd124-987baa3a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:37.442345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:37.442349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:37.442355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:37.442364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:37.442376Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:37.442409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:37.442521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.442527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.442539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:37.442600Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1574 partition: 0 MaxSeqNo: 25 sessionId: src_id|5de90dec-e1704473-937cd124-987baa3a_0
2025-10-09T21:49:37.442846Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|47dd464f-f143f211-4106e0df-f9f91411_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5de90dec-e1704473-937cd124-987baa3a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:37.445124Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5de90dec-e1704473-937cd124-987baa3a_0] Write session: try to update token
2025-10-09T21:49:37.445137Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5de90dec-e1704473-937cd124-987baa3a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:37.455926Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:37.455944Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:37.455945Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:37.455949Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:37.455957Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342488539642473:4813]
2025-10-09T21:49:37.455960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342488539642473:4813]
2025-10-09T21:49:37.455963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342488539642473:4813]
2025-10-09T21:49:37.455996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:37.456001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.456002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:37.456006Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:37.456017Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:37.456021Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:37.456121Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:37.457527Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1574 sessionId: src_id|5de90dec-e1704473-937cd124-987baa3a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:37.459954Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:37.521646Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342488901469925:2585]
2025-10-09T21:49:37.521883Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 1573, pipe: [1:7559342488901469925:2585]
2025-10-09T21:49:37.592386Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:37.593135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.593151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.593188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901469925:2585], now have 1 active actors on pipe
2025-10-09T21:49:37.593374Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:37.593402Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:37.593414Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342488901469925:2585]
2025-10-09T21:49:37.593438Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:37.593507Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1574 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5de90dec-e1704473-937cd124-987baa3a_0
2025-10-09T21:49:37.593689Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1574 sessionId: src_id|5de90dec-e1704473-937cd124-987baa3a_0 is DEAD
2025-10-09T21:49:37.593826Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5de90dec-e1704473-937cd124-987baa3a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:37.593836Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5de90dec-e1704473-937cd124-987baa3a_0] Write session will restart in 0.009271s
2025-10-09T21:49:37.593862Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5de90dec-e1704473-937cd124-987baa3a_0] Write session: Do CDS request
2025-10-09T21:49:37.593868Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5de90dec-e1704473-937cd124-987baa3a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:37.602225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:37.604429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5de90dec-e1704473-937cd124-987baa3a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:37.604496Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:37.604507Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1575
2025-10-09T21:49:37.604565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:37.604625Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1575 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:37.604652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1575 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:37.604656Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1575 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:37.604658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:37.604744Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:37.604755Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:37.604765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:37.604810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:37.604849Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342488539642630:4811]
2025-10-09T21:49:37.605023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1575 sessionId:  describe result for acl check
2025-10-09T21:49:37.605055Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:37.605061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:37.605063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:37.605069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342488901470051:8001] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:37.605427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:37.605594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342488901470051:8001] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:37.605676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:37.605716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:37.606012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:37.606072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.606077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.606079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.606080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.606081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.606082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.606083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.606084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.606085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.606090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.606100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.606104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.606107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.606111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.606115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.606119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.606123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.606127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.606133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.606140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.606146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.606151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.606156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.606160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.606166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.606175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.606180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.606185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.606189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.606199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.606207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.606212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.606221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.606226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.606231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.606239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.606267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:37.607214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342488901470051:8001] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:37.611586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342488901470051:8001] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:37.611601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342488901470051:8001] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:37.611605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342488901470051:8001] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:37.611613Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1575 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:37.611950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:37.613241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:37.613253Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:37.613255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:37.613258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:37.613262Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1574 [2:7559342488539642630:4811]
2025-10-09T21:49:37.613272Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:37.613285Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:37.613287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:37.613289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:37.613290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:37.613303Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:37.613306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:37.613312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:37.613357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.613365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.613367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.613369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:37.613433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:37.613441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.613453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.613464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.613474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:37.613493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901470090:8001], now have 1 active actors on pipe
2025-10-09T21:49:37.613496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901469927:2432], now have 1 active actors on pipe
2025-10-09T21:49:37.613501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342488539642624:4811]
2025-10-09T21:49:37.613503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342488539642624:4811]
2025-10-09T21:49:37.613504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342488539642624:4811]
2025-10-09T21:49:37.613514Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:37.613527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:37.613533Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:37.613536Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:37.613538Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:37.613538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:37.613541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:37.613539Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1574
2025-10-09T21:49:37.613570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342488539642624:4811]
2025-10-09T21:49:37.613579Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:37.613542Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1574
2025-10-09T21:49:37.613581Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:37.613599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.613605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.613634Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f8c2fb46-bd1483c-6468c19e-20e3369_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:37.613644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:37.613647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:37.613649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:37.613655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:37.613662Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:37.613679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:37.613775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.613780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.613789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:37.613845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1575 partition: 0 MaxSeqNo: 25 sessionId: src_id|f8c2fb46-bd1483c-6468c19e-20e3369_0
2025-10-09T21:49:37.614142Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5de90dec-e1704473-937cd124-987baa3a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f8c2fb46-bd1483c-6468c19e-20e3369_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:37.616391Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f8c2fb46-bd1483c-6468c19e-20e3369_0] Write session: try to update token
2025-10-09T21:49:37.616404Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f8c2fb46-bd1483c-6468c19e-20e3369_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:37.627171Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:37.627186Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:37.627187Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:37.627189Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:37.627196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342488539642624:4811]
2025-10-09T21:49:37.627199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342488539642624:4811]
2025-10-09T21:49:37.627201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342488539642624:4811]
2025-10-09T21:49:37.627233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:37.627238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.627240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:37.627248Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:37.627262Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:37.627266Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:37.627359Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:37.628592Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1575 sessionId: src_id|f8c2fb46-bd1483c-6468c19e-20e3369_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:37.630906Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:37.693694Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342488901470097:2585]
2025-10-09T21:49:37.693937Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 1574, pipe: [1:7559342488901470097:2585]
2025-10-09T21:49:37.767544Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:37.767964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.767982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.768032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901470097:2585], now have 1 active actors on pipe
2025-10-09T21:49:37.768270Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:37.768310Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342488901470097:2585]
2025-10-09T21:49:37.768311Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:37.768316Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:37.768342Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:37.768378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1575 reason: error at write: Disconnected sessionId: src_id|f8c2fb46-bd1483c-6468c19e-20e3369_0
2025-10-09T21:49:37.768583Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1575 sessionId: src_id|f8c2fb46-bd1483c-6468c19e-20e3369_0 is DEAD
2025-10-09T21:49:37.768743Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f8c2fb46-bd1483c-6468c19e-20e3369_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:37.768754Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f8c2fb46-bd1483c-6468c19e-20e3369_0] Write session will restart in 0.008237s
2025-10-09T21:49:37.768777Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f8c2fb46-bd1483c-6468c19e-20e3369_0] Write session: Do CDS request
2025-10-09T21:49:37.768782Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f8c2fb46-bd1483c-6468c19e-20e3369_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:37.777840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:37.778390Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f8c2fb46-bd1483c-6468c19e-20e3369_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:37.778511Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:37.778524Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1576
2025-10-09T21:49:37.778629Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1576 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:37.778659Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1576 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:37.778669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1576 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:37.778674Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:37.779061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1576 sessionId:  describe result for acl check
2025-10-09T21:49:37.779094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:37.779096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:37.779098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:37.779103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342488901470225:7998] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:37.779617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342488901470225:7998] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:37.780066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:37.780195Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:37.780203Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:37.780211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:37.780245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:37.780272Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342488539642786:4806]
2025-10-09T21:49:37.780620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:37.780851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:37.780878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:37.781071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:37.781115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.781121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.781122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.781123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.781124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.781125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.781126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.781126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.781127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.781130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.781136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.781138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.781140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.781142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.781144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.781146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.781148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.781150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.781155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.781160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.781164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.781167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.781169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.781172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.781175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.781178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.781180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.781183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.781186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.781188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.781191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.781194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.781197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.781200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.781203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.781205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.781223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:37.781424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342488901470225:7998] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:37.786687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342488901470225:7998] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:37.786700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342488901470225:7998] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:37.786704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342488901470225:7998] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:37.786712Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1576 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:37.787008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:37.788056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:37.788072Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:37.788074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:37.788078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:37.788083Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1575 [2:7559342488539642786:4806]
2025-10-09T21:49:37.788095Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:37.788112Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:37.788119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:37.788122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:37.788124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:37.788139Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:37.788148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:37.788155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:37.788212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.788218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.788221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.788223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:37.788282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:37.788288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.788301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.788317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.788338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:37.788363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901470264:7998], now have 1 active actors on pipe
2025-10-09T21:49:37.788371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901470099:2432], now have 1 active actors on pipe
2025-10-09T21:49:37.788378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342488539642781:4806]
2025-10-09T21:49:37.788380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342488539642781:4806]
2025-10-09T21:49:37.788382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342488539642781:4806]
2025-10-09T21:49:37.788393Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:37.788402Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:37.788410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:37.788401Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1575
2025-10-09T21:49:37.788411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:37.788412Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:37.788413Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1575
2025-10-09T21:49:37.788425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:37.788428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:37.788461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.788469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.788480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342488539642781:4806]
2025-10-09T21:49:37.788491Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:37.788497Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:37.788520Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6db461fc-65dfdea1-69de5d6e-ebe317b9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:37.788532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:37.788535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:37.788537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:37.788548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:37.788557Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:37.788582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:37.788709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.788716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.788729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:37.788811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1576 partition: 0 MaxSeqNo: 25 sessionId: src_id|6db461fc-65dfdea1-69de5d6e-ebe317b9_0
2025-10-09T21:49:37.789104Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f8c2fb46-bd1483c-6468c19e-20e3369_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6db461fc-65dfdea1-69de5d6e-ebe317b9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:37.791404Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6db461fc-65dfdea1-69de5d6e-ebe317b9_0] Write session: try to update token
2025-10-09T21:49:37.791418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6db461fc-65dfdea1-69de5d6e-ebe317b9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:37.803067Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:37.803095Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:37.803098Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:37.803101Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:37.803111Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342488539642781:4806]
2025-10-09T21:49:37.803118Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342488539642781:4806]
2025-10-09T21:49:37.803124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342488539642781:4806]
2025-10-09T21:49:37.803163Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:37.803174Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:37.803177Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:37.803177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:37.803187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.803190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:37.803447Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:37.803985Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1576 sessionId: src_id|6db461fc-65dfdea1-69de5d6e-ebe317b9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:37.806240Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:37.868254Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342488901470274:2585]
2025-10-09T21:49:37.868449Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 1575, pipe: [1:7559342488901470274:2585]
2025-10-09T21:49:37.937796Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:37.938416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.938433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.938474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901470274:2585], now have 1 active actors on pipe
2025-10-09T21:49:37.938758Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:37.938770Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:37.938777Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:37.938823Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:37.938821Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342488901470274:2585]
2025-10-09T21:49:37.938826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1576 reason: error at write: Disconnected sessionId: src_id|6db461fc-65dfdea1-69de5d6e-ebe317b9_0
2025-10-09T21:49:37.938989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1576 sessionId: src_id|6db461fc-65dfdea1-69de5d6e-ebe317b9_0 is DEAD
2025-10-09T21:49:37.939170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6db461fc-65dfdea1-69de5d6e-ebe317b9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:37.939181Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6db461fc-65dfdea1-69de5d6e-ebe317b9_0] Write session will restart in 0.007686s
2025-10-09T21:49:37.939208Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6db461fc-65dfdea1-69de5d6e-ebe317b9_0] Write session: Do CDS request
2025-10-09T21:49:37.939214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6db461fc-65dfdea1-69de5d6e-ebe317b9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:37.948409Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6db461fc-65dfdea1-69de5d6e-ebe317b9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:37.948461Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:37.948470Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1577
2025-10-09T21:49:37.948605Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1577 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:37.948631Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1577 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:37.948641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1577 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:37.948649Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:37.948976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1577 sessionId:  describe result for acl check
2025-10-09T21:49:37.948999Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:37.949007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:37.949008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:37.949013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342488901470400:7986] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:37.949405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342488901470400:7986] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:37.949666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:37.952001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342488901470400:7986] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:37.952252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:37.952422Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:37.952431Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:37.952439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:37.952477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:37.952517Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342488539642940:4803]
2025-10-09T21:49:37.952873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:37.953086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:37.953112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:37.953310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:37.953338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.953342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.953343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.953343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.953344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.953345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.953345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.953346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.953346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.953349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.953356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.953358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.953360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.953362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.953364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.953366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.953367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.953370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.953374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.953379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:37.953383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.953386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:37.953389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.953392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:37.953396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.953399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:37.953403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.953405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:37.953410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.953413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:37.953416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.953419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.953425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.953428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.953430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.953433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.953449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:37.957730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342488901470400:7986] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:37.957741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342488901470400:7986] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:37.957745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342488901470400:7986] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:37.957752Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1577 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:37.958968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:37.959927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:37.959939Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:37.959940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:37.959943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:37.959947Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1576 [2:7559342488539642940:4803]
2025-10-09T21:49:37.959956Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:37.959967Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:37.959973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:37.959975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:37.959976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:37.959987Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:37.959992Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:37.959997Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:37.960037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.960042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.960043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.960045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:37.960098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:37.960105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.960115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:37.960125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:37.960143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:37.960164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901470441:7986], now have 1 active actors on pipe
2025-10-09T21:49:37.960171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342488901470275:2432], now have 1 active actors on pipe
2025-10-09T21:49:37.960177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342488539642934:4803]
2025-10-09T21:49:37.960179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342488539642934:4803]
2025-10-09T21:49:37.960181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342488539642934:4803]
2025-10-09T21:49:37.960169Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1576
2025-10-09T21:49:37.960174Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1576
2025-10-09T21:49:37.960194Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:37.960206Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:37.960209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:37.960213Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:37.960217Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:37.960217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:37.960219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:37.960243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.960249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.960262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342488539642934:4803]
2025-10-09T21:49:37.960267Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:37.960278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:37.960287Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9b97dba4-bb43be8a-16f6b389-97a3801a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:37.960296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:37.960298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:37.960300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:37.960306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:37.960313Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:37.960335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:37.960466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:37.960474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:37.960486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:37.960559Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1577 partition: 0 MaxSeqNo: 25 sessionId: src_id|9b97dba4-bb43be8a-16f6b389-97a3801a_0
2025-10-09T21:49:37.960830Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6db461fc-65dfdea1-69de5d6e-ebe317b9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9b97dba4-bb43be8a-16f6b389-97a3801a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:37.963141Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9b97dba4-bb43be8a-16f6b389-97a3801a_0] Write session: try to update token
2025-10-09T21:49:37.963155Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9b97dba4-bb43be8a-16f6b389-97a3801a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:37.973443Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:37.973460Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:37.973463Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:37.973464Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:37.973477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342488539642934:4803]
2025-10-09T21:49:37.973481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342488539642934:4803]
2025-10-09T21:49:37.973484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342488539642934:4803]
2025-10-09T21:49:37.973515Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:37.973520Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:37.973523Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:37.973523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:37.973529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:37.973532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:37.973673Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:37.974606Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1577 sessionId: src_id|9b97dba4-bb43be8a-16f6b389-97a3801a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:37.976734Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:38.038768Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342493196437745:2585]
2025-10-09T21:49:38.038964Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 1576, pipe: [1:7559342493196437745:2585]
2025-10-09T21:49:38.119232Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:38.119753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:38.119767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:38.119806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342493196437745:2585], now have 1 active actors on pipe
2025-10-09T21:49:38.119986Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:38.120112Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:38.120131Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:38.120166Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342493196437745:2585]
2025-10-09T21:49:38.120199Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1577 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9b97dba4-bb43be8a-16f6b389-97a3801a_0
2025-10-09T21:49:38.120389Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1577 sessionId: src_id|9b97dba4-bb43be8a-16f6b389-97a3801a_0 is DEAD
2025-10-09T21:49:38.120527Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b97dba4-bb43be8a-16f6b389-97a3801a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:38.120538Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b97dba4-bb43be8a-16f6b389-97a3801a_0] Write session will restart in 0.008955s
2025-10-09T21:49:38.120562Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b97dba4-bb43be8a-16f6b389-97a3801a_0] Write session: Do CDS request
2025-10-09T21:49:38.120568Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b97dba4-bb43be8a-16f6b389-97a3801a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:38.130460Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9b97dba4-bb43be8a-16f6b389-97a3801a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:38.130547Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:38.130563Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1578
2025-10-09T21:49:38.130693Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1578 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:38.130726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1578 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:38.130736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1578 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:38.130740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:38.131189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1578 sessionId:  describe result for acl check
2025-10-09T21:49:38.131236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:38.131242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:38.131244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:38.131250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342493196437874:7979] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:38.131652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:38.131796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342493196437874:7979] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:38.133851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342493196437874:7979] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:38.134192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:38.134365Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:38.134374Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:38.134383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:38.134428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:38.134459Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342492834610390:4800]
2025-10-09T21:49:38.134898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:38.135168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:38.135202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:38.135419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:38.135457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.135467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.135468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.135469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.135470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.135471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.135472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.135473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.135474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.135479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.135490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.135494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.135498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.135500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.135502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.135504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.135506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.135508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.135513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.135518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.135523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.135526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.135529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.135532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.135534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.135537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.135539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.135542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.135545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.135548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.135550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.135553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.135556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.135560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.135562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.135565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.135589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:38.136022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342493196437874:7979] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:38.136033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342493196437874:7979] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:38.136035Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342493196437874:7979] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:38.136041Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1578 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:38.141785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:38.142911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:38.142927Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:38.142930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:38.142934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:38.142940Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1577 [2:7559342492834610390:4800]
2025-10-09T21:49:38.142955Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:38.142978Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:38.142988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:38.142991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:38.142994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:38.143011Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:38.143017Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:38.143032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:38.143090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.143097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.143098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.143101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:38.143219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:38.143227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.143239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.143256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.143282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:38.143308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342493196437915:7979], now have 1 active actors on pipe
2025-10-09T21:49:38.143317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342493196437749:2432], now have 1 active actors on pipe
2025-10-09T21:49:38.143324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342492834610384:4800]
2025-10-09T21:49:38.143327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342492834610384:4800]
2025-10-09T21:49:38.143328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342492834610384:4800]
2025-10-09T21:49:38.143341Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:38.143352Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:38.143360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:38.143345Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1577
2025-10-09T21:49:38.143362Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:38.143362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:38.143377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:38.143380Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:38.143373Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1577
2025-10-09T21:49:38.143427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:38.143437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:38.143450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342492834610384:4800]
2025-10-09T21:49:38.143463Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:38.143470Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:38.143497Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fd33a724-a1bb3d75-87c5921d-63842079_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:38.143511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:38.143515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:38.143518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:38.143529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:38.143540Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:38.143564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:38.143709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:38.143716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:38.143730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:38.143822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1578 partition: 0 MaxSeqNo: 25 sessionId: src_id|fd33a724-a1bb3d75-87c5921d-63842079_0
2025-10-09T21:49:38.144126Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b97dba4-bb43be8a-16f6b389-97a3801a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fd33a724-a1bb3d75-87c5921d-63842079_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:38.146409Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fd33a724-a1bb3d75-87c5921d-63842079_0] Write session: try to update token
2025-10-09T21:49:38.146424Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fd33a724-a1bb3d75-87c5921d-63842079_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:38.157766Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:38.157784Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:38.157787Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:38.157789Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:38.157798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342492834610384:4800]
2025-10-09T21:49:38.157803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342492834610384:4800]
2025-10-09T21:49:38.157808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342492834610384:4800]
2025-10-09T21:49:38.157849Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:38.157850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:38.157858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.157859Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:38.157860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:38.157862Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:38.158149Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1578 sessionId: src_id|fd33a724-a1bb3d75-87c5921d-63842079_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:38.158445Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:38.160468Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:38.220560Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342493196437926:2585]
2025-10-09T21:49:38.220823Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 1577, pipe: [1:7559342493196437926:2585]
2025-10-09T21:49:38.305230Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:38.305489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:38.305502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:38.305540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342493196437926:2585], now have 1 active actors on pipe
2025-10-09T21:49:38.305723Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:38.305807Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342493196437926:2585]
2025-10-09T21:49:38.305807Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:38.305812Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:38.305837Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:38.305876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1578 reason: error at write: Disconnected sessionId: src_id|fd33a724-a1bb3d75-87c5921d-63842079_0
2025-10-09T21:49:38.306027Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1578 sessionId: src_id|fd33a724-a1bb3d75-87c5921d-63842079_0 is DEAD
2025-10-09T21:49:38.306194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd33a724-a1bb3d75-87c5921d-63842079_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:38.306203Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd33a724-a1bb3d75-87c5921d-63842079_0] Write session will restart in 0.006921s
2025-10-09T21:49:38.306232Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd33a724-a1bb3d75-87c5921d-63842079_0] Write session: Do CDS request
2025-10-09T21:49:38.306237Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd33a724-a1bb3d75-87c5921d-63842079_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:38.314491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fd33a724-a1bb3d75-87c5921d-63842079_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:38.314583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:38.314597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1579
2025-10-09T21:49:38.314744Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1579 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:38.314777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1579 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:38.314787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1579 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:38.314791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:38.315132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1579 sessionId:  describe result for acl check
2025-10-09T21:49:38.315161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:38.315166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:38.315167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:38.315171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342493196438058:7975] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:38.315582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342493196438058:7975] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:38.317548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342493196438058:7975] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:38.318451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:38.318938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:38.318953Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342493196438058:7975] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:38.318961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342493196438058:7975] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:38.318963Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342493196438058:7975] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:38.318969Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1579 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:38.319105Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:38.319118Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:38.319131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:38.319181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:38.319224Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342492834610558:4792]
2025-10-09T21:49:38.319775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:38.320031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:38.320077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:38.320386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:38.320441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.320448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.320450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.320452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.320453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.320454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.320455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.320456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.320458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.320463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.320474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.320478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.320487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.320491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.320495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.320499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.320503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.320507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.320514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.320522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.320533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.320544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.320552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.320559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.320568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.320576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.320587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.320597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.320605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.320613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.320623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.320633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.320642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.320652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.320660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.320666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.320702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:38.326938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:38.328079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:38.328094Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:38.328096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:38.328100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:38.328106Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1578 [2:7559342492834610558:4792]
2025-10-09T21:49:38.328122Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:38.328142Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:38.328150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:38.328153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:38.328155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:38.328170Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:38.328180Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:38.328187Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:38.328240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.328247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.328249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.328251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:38.328313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:38.328323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.328335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.328347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.328367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:38.328393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342493196437933:2432], now have 1 active actors on pipe
2025-10-09T21:49:38.328401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342493196438096:7975], now have 1 active actors on pipe
2025-10-09T21:49:38.328384Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1578
2025-10-09T21:49:38.328408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342492834610553:4792]
2025-10-09T21:49:38.328415Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342492834610553:4792]
2025-10-09T21:49:38.328417Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342492834610553:4792]
2025-10-09T21:49:38.328426Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:38.328433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:38.328425Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1578
2025-10-09T21:49:38.328435Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:38.328437Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:38.328443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:38.328459Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:38.328467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:38.328508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:38.328518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:38.328531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342492834610553:4792]
2025-10-09T21:49:38.328544Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:38.328553Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:38.328575Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|176733a8-d5918b57-95ee956a-e29164ab_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:38.328586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:38.328595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:38.328597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:38.328607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:38.328621Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:38.328699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:38.328837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:38.328845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:38.328861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:38.328943Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1579 partition: 0 MaxSeqNo: 25 sessionId: src_id|176733a8-d5918b57-95ee956a-e29164ab_0
2025-10-09T21:49:38.329245Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd33a724-a1bb3d75-87c5921d-63842079_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|176733a8-d5918b57-95ee956a-e29164ab_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:38.331500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|176733a8-d5918b57-95ee956a-e29164ab_0] Write session: try to update token
2025-10-09T21:49:38.331513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|176733a8-d5918b57-95ee956a-e29164ab_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:38.342736Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:38.342758Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:38.342760Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:38.342762Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:38.342771Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342492834610553:4792]
2025-10-09T21:49:38.342775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342492834610553:4792]
2025-10-09T21:49:38.342778Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342492834610553:4792]
2025-10-09T21:49:38.342815Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:38.342820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:38.342826Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:38.342827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.342829Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:38.342829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:38.342965Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:38.344318Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1579 sessionId: src_id|176733a8-d5918b57-95ee956a-e29164ab_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:38.346601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:38.406459Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342493196438130:2585]
2025-10-09T21:49:38.406694Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 1578, pipe: [1:7559342493196438130:2585]
2025-10-09T21:49:38.490748Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:38.491074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:38.491088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:38.491125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342493196438130:2585], now have 1 active actors on pipe
2025-10-09T21:49:38.491329Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:38.491369Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:38.491373Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342493196438130:2585]
2025-10-09T21:49:38.491381Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:38.491396Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:38.491446Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1579 reason: error at write: Disconnected sessionId: src_id|176733a8-d5918b57-95ee956a-e29164ab_0
2025-10-09T21:49:38.491620Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1579 sessionId: src_id|176733a8-d5918b57-95ee956a-e29164ab_0 is DEAD
2025-10-09T21:49:38.491755Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|176733a8-d5918b57-95ee956a-e29164ab_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:38.491765Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|176733a8-d5918b57-95ee956a-e29164ab_0] Write session will restart in 0.009565s
2025-10-09T21:49:38.491783Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|176733a8-d5918b57-95ee956a-e29164ab_0] Write session: Do CDS request
2025-10-09T21:49:38.491788Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|176733a8-d5918b57-95ee956a-e29164ab_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:38.502122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:38.502477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|176733a8-d5918b57-95ee956a-e29164ab_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:38.502579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:38.502592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1580
2025-10-09T21:49:38.502697Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1580 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:38.502728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1580 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:38.502735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1580 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:38.502739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:38.503066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1580 sessionId:  describe result for acl check
2025-10-09T21:49:38.503095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:38.503097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:38.503098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:38.503102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342493196438261:7957] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:38.503635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342493196438261:7957] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:38.504691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:38.504927Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:38.504937Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:38.504949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:38.505005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:38.505045Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342492834610739:4757]
2025-10-09T21:49:38.505552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:38.505752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342493196438261:7957] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:38.505848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:38.505896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:38.506229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:38.506269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.506273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.506274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.506274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.506275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.506276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.506276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.506277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.506277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.506281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.506288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.506290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.506293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.506295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.506297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.506299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.506301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.506303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.506307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.506312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.506316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.506320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.506322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.506325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.506327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.506331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.506333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.506336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.506339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.506342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.506345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.506348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.506351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.506353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.506356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.506358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.506374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:38.511959Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342493196438261:7957] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:38.511977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342493196438261:7957] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:38.511981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342493196438261:7957] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:38.511992Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1580 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:38.512262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:38.513674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:38.513686Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:38.513688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:38.513691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:38.513696Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1579 [2:7559342492834610739:4757]
2025-10-09T21:49:38.513705Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:38.513719Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:38.513722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:38.513725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:38.513727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:38.513744Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:38.513752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:38.513758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:38.513799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.513805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.513806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.513808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:38.513858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:38.513863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.513869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.513875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.513884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:38.513903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342492834610733:4757]
2025-10-09T21:49:38.513907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342492834610733:4757]
2025-10-09T21:49:38.513909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342492834610733:4757]
2025-10-09T21:49:38.513915Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:38.513924Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:38.513925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:38.513927Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:38.513929Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:38.513933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:38.513935Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:38.513945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342493196438132:2432], now have 1 active actors on pipe
2025-10-09T21:49:38.513949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342493196438300:7957], now have 1 active actors on pipe
2025-10-09T21:49:38.513968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342492834610733:4757]
2025-10-09T21:49:38.513982Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:38.513972Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1579
2025-10-09T21:49:38.513999Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1579
2025-10-09T21:49:38.514006Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:38.514107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:38.514116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:38.514137Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ad790fec-3e9445d4-fe9fc311-5e78710c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:38.514147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:38.514150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:38.514151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:38.514158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:38.514169Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:38.514183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:38.514325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:38.514332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:38.514343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:38.514407Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1580 partition: 0 MaxSeqNo: 25 sessionId: src_id|ad790fec-3e9445d4-fe9fc311-5e78710c_0
2025-10-09T21:49:38.514690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|176733a8-d5918b57-95ee956a-e29164ab_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ad790fec-3e9445d4-fe9fc311-5e78710c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:38.516918Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad790fec-3e9445d4-fe9fc311-5e78710c_0] Write session: try to update token
2025-10-09T21:49:38.516931Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad790fec-3e9445d4-fe9fc311-5e78710c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:38.528707Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1580 sessionId: src_id|ad790fec-3e9445d4-fe9fc311-5e78710c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:38.530518Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:38.530541Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:38.530552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342492834610733:4757]
2025-10-09T21:49:38.530603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:38.530615Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:38.530788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:38.530951Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:38.531020Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:38.531090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:38.531223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:38.531404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:38.531452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:38.531466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:38.531490Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:38.531529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:38.531599Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:38.537158Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:38.537178Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:38.537180Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:38.537182Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:38.537192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342492834610733:4757]
2025-10-09T21:49:38.537194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342492834610733:4757]
2025-10-09T21:49:38.537197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342492834610733:4757]
2025-10-09T21:49:38.537227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:38.537233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.537234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:38.537236Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:38.537241Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:38.537243Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:38.537361Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:38.591729Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342493196438308:2585]
2025-10-09T21:49:38.591993Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 1579, pipe: [1:7559342493196438308:2585]
2025-10-09T21:49:38.682367Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:38.688822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:38.688846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:38.688989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:38.689044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:38.689111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:38.689178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:38.689221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:38.689229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:38.689289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:38.689340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:38.689402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:38.689458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:38.689497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:38.689502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:38.689555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:38.689620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:38.689675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:38.689727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:38.689765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:38.689770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:38.689827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:38.689883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:38.689936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:38.689989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:38.690041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:38.690047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:38.690104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:38.690155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:38.690208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:38.690262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:38.690308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:38.690317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:38.690379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:38.690430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:38.690485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:38.690539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:38.690577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:38.690582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:38.690636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:38.690685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:38.690740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:38.690883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:38.690930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:38.690936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:38.690990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:38.691090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:38.691141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:38.691191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:38.691231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:38.691236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:38.691296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:38.691350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:38.691410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:38.691464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:38.691502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:38.691507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:38.691560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:38.691608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:38.691659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:38.691716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:38.691762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:38.691768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:38.691806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342493196438308:2585], now have 1 active actors on pipe
2025-10-09T21:49:38.692093Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342493196438308:2585]
2025-10-09T21:49:38.692094Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:38.692098Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:38.692127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1580 reason: error at write: Disconnected sessionId: src_id|ad790fec-3e9445d4-fe9fc311-5e78710c_0
2025-10-09T21:49:38.692317Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1580 sessionId: src_id|ad790fec-3e9445d4-fe9fc311-5e78710c_0 is DEAD
2025-10-09T21:49:38.692341Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:38.692426Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad790fec-3e9445d4-fe9fc311-5e78710c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:38.692436Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad790fec-3e9445d4-fe9fc311-5e78710c_0] Write session will restart in 0.009974s
2025-10-09T21:49:38.692456Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad790fec-3e9445d4-fe9fc311-5e78710c_0] Write session: Do CDS request
2025-10-09T21:49:38.692461Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad790fec-3e9445d4-fe9fc311-5e78710c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:38.692867Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:38.703472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:38.703521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad790fec-3e9445d4-fe9fc311-5e78710c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:38.703594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:38.703606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1581
2025-10-09T21:49:38.703732Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1581 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:38.703768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1581 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:38.703776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1581 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:38.703779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:38.704092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1581 sessionId:  describe result for acl check
2025-10-09T21:49:38.704122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:38.704127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:38.704128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:38.704134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342493196438442:7951] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:38.704542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342493196438442:7951] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:38.705658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:38.705864Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:38.705874Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:38.705886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:38.705946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:38.705987Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342492834610901:4765]
2025-10-09T21:49:38.706558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:38.706811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:38.706857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:38.706870Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342493196438442:7951] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:38.707174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:38.707209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.707215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.707216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.707216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.707217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.707217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.707218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.707219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.707219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.707223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.707229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.707231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.707234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.707236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.707238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.707240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.707242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.707244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.707248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.707253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.707257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.707260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.707263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.707266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.707269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.707272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.707275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.707278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.707280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.707283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.707286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.707289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.707292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.707295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.707298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.707300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.707317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:38.712056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342493196438442:7951] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:38.712071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342493196438442:7951] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:38.712074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342493196438442:7951] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:38.712081Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1581 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:38.713375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:38.714474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:38.714491Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:38.714493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:38.714498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:38.714506Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1580 [2:7559342492834610901:4765]
2025-10-09T21:49:38.714519Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:38.714536Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:38.714539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:38.714543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:38.714545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:38.714562Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:38.714568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:38.714575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:38.714627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.714634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.714635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.714637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:38.714697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:38.714705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.714714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.714720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.714729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:38.714745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342493196438310:2432], now have 1 active actors on pipe
2025-10-09T21:49:38.714756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342492834610894:4765]
2025-10-09T21:49:38.714758Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342492834610894:4765]
2025-10-09T21:49:38.714760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342492834610894:4765]
2025-10-09T21:49:38.714770Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:38.714780Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:38.714783Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:38.714784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342493196438480:7951], now have 1 active actors on pipe
2025-10-09T21:49:38.714785Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:38.714792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:38.714805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:38.714790Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1580
2025-10-09T21:49:38.714806Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1580
2025-10-09T21:49:38.714808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:38.714843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342492834610894:4765]
2025-10-09T21:49:38.714855Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:38.714865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:38.714867Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:38.714872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:38.714904Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|55bb73d5-cc828ffa-37e100b8-c1fe2ae4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:38.714916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:38.714920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:38.714923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:38.714933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:38.714948Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:38.714967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:38.715082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:38.715090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:38.715103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:38.715184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1581 partition: 0 MaxSeqNo: 25 sessionId: src_id|55bb73d5-cc828ffa-37e100b8-c1fe2ae4_0
2025-10-09T21:49:38.715527Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad790fec-3e9445d4-fe9fc311-5e78710c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|55bb73d5-cc828ffa-37e100b8-c1fe2ae4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:38.717967Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55bb73d5-cc828ffa-37e100b8-c1fe2ae4_0] Write session: try to update token
2025-10-09T21:49:38.717980Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55bb73d5-cc828ffa-37e100b8-c1fe2ae4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:38.728679Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:38.728699Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:38.728701Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:38.728702Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:38.728710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342492834610894:4765]
2025-10-09T21:49:38.728714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342492834610894:4765]
2025-10-09T21:49:38.728717Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342492834610894:4765]
2025-10-09T21:49:38.728751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:38.728757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.728759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:38.728773Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:38.728778Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:38.728786Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:38.728886Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:38.730431Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1581 sessionId: src_id|55bb73d5-cc828ffa-37e100b8-c1fe2ae4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:38.732656Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:38.792436Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342493196438487:2585]
2025-10-09T21:49:38.792806Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 1580, pipe: [1:7559342493196438487:2585]
2025-10-09T21:49:38.878546Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:38.879220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:38.879236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:38.879277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342493196438487:2585], now have 1 active actors on pipe
2025-10-09T21:49:38.879483Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:38.879491Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342493196438487:2585]
2025-10-09T21:49:38.879512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:38.879516Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:38.879523Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:38.879570Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1581 reason: error at write: Disconnected sessionId: src_id|55bb73d5-cc828ffa-37e100b8-c1fe2ae4_0
2025-10-09T21:49:38.879731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1581 sessionId: src_id|55bb73d5-cc828ffa-37e100b8-c1fe2ae4_0 is DEAD
2025-10-09T21:49:38.879888Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55bb73d5-cc828ffa-37e100b8-c1fe2ae4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:38.879899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55bb73d5-cc828ffa-37e100b8-c1fe2ae4_0] Write session will restart in 0.007036s
2025-10-09T21:49:38.879919Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55bb73d5-cc828ffa-37e100b8-c1fe2ae4_0] Write session: Do CDS request
2025-10-09T21:49:38.879924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55bb73d5-cc828ffa-37e100b8-c1fe2ae4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:38.888416Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55bb73d5-cc828ffa-37e100b8-c1fe2ae4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:38.888488Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:38.888499Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1582
2025-10-09T21:49:38.888595Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1582 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:38.888618Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1582 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:38.888622Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1582 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:38.888629Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:38.888970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1582 sessionId:  describe result for acl check
2025-10-09T21:49:38.888999Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:38.889006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:38.889007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:38.889011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342493196438622:7946] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:38.889009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:38.889403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342493196438622:7946] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:38.891084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:38.891230Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:38.891239Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:38.891250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:38.891301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:38.891339Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342492834611058:4761]
2025-10-09T21:49:38.891353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342493196438622:7946] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:38.891699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:38.891915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:38.891951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:38.892161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:38.892204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.892210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.892211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.892212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.892214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.892215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.892216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.892217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.892218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.892223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.892233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.892237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.892241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.892246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.892250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.892254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.892258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.892263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.892269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.892275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:38.892279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.892284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:38.892289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.892294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:38.892300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.892305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:38.892315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.892320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:38.892324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.892330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:38.892336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.892343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.892347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.892354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.892360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.892365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.892397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:38.896917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342493196438622:7946] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:38.896934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342493196438622:7946] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:38.896937Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342493196438622:7946] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:38.896942Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1582 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:38.898971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:38.899947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:38.899962Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:38.899964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:38.899967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:38.899973Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1581 [2:7559342492834611058:4761]
2025-10-09T21:49:38.899985Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:38.900000Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:38.900004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:38.900008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:38.900010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:38.900026Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:38.900038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:38.900045Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:38.900089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.900095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.900097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.900099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:38.900159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:38.900166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.900177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:38.900194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:38.900214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:38.900237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342493196438492:2432], now have 1 active actors on pipe
2025-10-09T21:49:38.900244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342493196438662:7946], now have 1 active actors on pipe
2025-10-09T21:49:38.900251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342492834611053:4761]
2025-10-09T21:49:38.900257Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342492834611053:4761]
2025-10-09T21:49:38.900259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342492834611053:4761]
2025-10-09T21:49:38.900251Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1581
2025-10-09T21:49:38.900273Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:38.900268Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1581
2025-10-09T21:49:38.900290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:38.900293Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:38.900296Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:38.900297Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:38.900307Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:38.900310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:38.900344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:38.900353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:38.900366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342492834611053:4761]
2025-10-09T21:49:38.900376Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:38.900395Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:38.900404Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bfcd6d47-eb82ee72-94dce12b-62479996_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:38.900410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:38.900414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:38.900416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:38.900425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:38.900437Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:38.900457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:38.900717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1582 partition: 0 MaxSeqNo: 25 sessionId: src_id|bfcd6d47-eb82ee72-94dce12b-62479996_0
2025-10-09T21:49:38.900616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:38.900627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:38.900642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:38.900967Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55bb73d5-cc828ffa-37e100b8-c1fe2ae4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bfcd6d47-eb82ee72-94dce12b-62479996_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:38.903222Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bfcd6d47-eb82ee72-94dce12b-62479996_0] Write session: try to update token
2025-10-09T21:49:38.903236Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bfcd6d47-eb82ee72-94dce12b-62479996_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:38.914965Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1582 sessionId: src_id|bfcd6d47-eb82ee72-94dce12b-62479996_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:38.916165Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:38.916186Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:38.916189Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:38.916191Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:38.916200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342492834611053:4761]
2025-10-09T21:49:38.916204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342492834611053:4761]
2025-10-09T21:49:38.916207Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342492834611053:4761]
2025-10-09T21:49:38.916245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:38.916254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:38.916256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:38.916258Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:38.916264Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:38.916269Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:38.916380Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:38.917104Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:38.979846Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342493196438669:2585]
2025-10-09T21:49:38.980096Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 1581, pipe: [1:7559342493196438669:2585]
2025-10-09T21:49:39.058629Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:39.058856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:39.058869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:39.058905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342493196438669:2585], now have 1 active actors on pipe
2025-10-09T21:49:39.059087Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:39.059163Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342493196438669:2585]
2025-10-09T21:49:39.059184Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:39.059188Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:39.059189Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:39.059264Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1582 reason: error at write: Disconnected sessionId: src_id|bfcd6d47-eb82ee72-94dce12b-62479996_0
2025-10-09T21:49:39.059431Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1582 sessionId: src_id|bfcd6d47-eb82ee72-94dce12b-62479996_0 is DEAD
2025-10-09T21:49:39.059589Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfcd6d47-eb82ee72-94dce12b-62479996_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:39.059600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfcd6d47-eb82ee72-94dce12b-62479996_0] Write session will restart in 0.007232s
2025-10-09T21:49:39.059632Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfcd6d47-eb82ee72-94dce12b-62479996_0] Write session: Do CDS request
2025-10-09T21:49:39.059637Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfcd6d47-eb82ee72-94dce12b-62479996_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:39.067485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bfcd6d47-eb82ee72-94dce12b-62479996_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:39.067544Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:39.067558Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1583
2025-10-09T21:49:39.067692Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1583 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:39.067726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1583 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:39.067745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1583 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:39.067753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:39.068146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1583 sessionId:  describe result for acl check
2025-10-09T21:49:39.068178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:39.068184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:39.068186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:39.068190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342497491406046:7941] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:39.068658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342497491406046:7941] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:39.070635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342497491406046:7941] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:39.073241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:39.073425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342497491406046:7941] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:39.073436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342497491406046:7941] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:39.073441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342497491406046:7941] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:39.073446Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1583 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:39.073579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:39.073731Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:39.073741Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:39.073749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:39.073786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:39.073817Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342497129578514:4760]
2025-10-09T21:49:39.074194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:39.074747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:39.074779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:39.074990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:39.075022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.075026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.075027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.075028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.075028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.075029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.075030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.075030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.075031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.075034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.075040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.075042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.075044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.075046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.075049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.075050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.075053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.075055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.075059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.075064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.075069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.075072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.075076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.075079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.075081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.075084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.075086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.075089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.075092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.075095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.075098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.075100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.075103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.075106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.075108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.075111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.075127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:39.081042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:39.082417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:39.082429Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:39.082431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:39.082434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:39.082439Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1582 [2:7559342497129578514:4760]
2025-10-09T21:49:39.082447Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:39.082459Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:39.082461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:39.082463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:39.082465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:39.082477Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:39.082481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:39.082486Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:39.082526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.082532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.082533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.082535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:39.082585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:39.082590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.082597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.082603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.082613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:39.082631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342497129578509:4760]
2025-10-09T21:49:39.082637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342497129578509:4760]
2025-10-09T21:49:39.082638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342497129578509:4760]
2025-10-09T21:49:39.082645Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:39.082653Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:39.082655Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:39.082657Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:39.082658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342497491405968:2432], now have 1 active actors on pipe
2025-10-09T21:49:39.082665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342497491406135:7941], now have 1 active actors on pipe
2025-10-09T21:49:39.082670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:39.082685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:39.082691Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:39.082714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342497129578509:4760]
2025-10-09T21:49:39.082723Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:39.082725Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1582
2025-10-09T21:49:39.082732Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1582
2025-10-09T21:49:39.082753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:39.082796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:39.082803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:39.082824Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|980a2569-7082ac97-e235e5a0-1ff4d0d9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:39.082832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:39.082835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:39.082837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:39.082844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:39.082851Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:39.082868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:39.083059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:39.083074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:39.083105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:39.083212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1583 partition: 0 MaxSeqNo: 25 sessionId: src_id|980a2569-7082ac97-e235e5a0-1ff4d0d9_0
2025-10-09T21:49:39.083534Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfcd6d47-eb82ee72-94dce12b-62479996_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|980a2569-7082ac97-e235e5a0-1ff4d0d9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:39.085893Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|980a2569-7082ac97-e235e5a0-1ff4d0d9_0] Write session: try to update token
2025-10-09T21:49:39.085910Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|980a2569-7082ac97-e235e5a0-1ff4d0d9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:39.096380Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:39.096399Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:39.096402Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:39.096404Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:39.096414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342497129578509:4760]
2025-10-09T21:49:39.096419Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342497129578509:4760]
2025-10-09T21:49:39.096424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342497129578509:4760]
2025-10-09T21:49:39.096468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:39.096471Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:39.096476Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:39.096477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.096480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:39.096480Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:39.096610Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:39.098551Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1583 sessionId: src_id|980a2569-7082ac97-e235e5a0-1ff4d0d9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:39.100777Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:39.159542Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342497491406146:2585]
2025-10-09T21:49:39.159838Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 1582, pipe: [1:7559342497491406146:2585]
2025-10-09T21:49:39.242927Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:39.243167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:39.243180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:39.243219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342497491406146:2585], now have 1 active actors on pipe
2025-10-09T21:49:39.243426Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342497491406146:2585]
2025-10-09T21:49:39.243445Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:39.243462Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:39.243464Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:39.243461Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:39.243512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1583 reason: error at write: Disconnected sessionId: src_id|980a2569-7082ac97-e235e5a0-1ff4d0d9_0
2025-10-09T21:49:39.243664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1583 sessionId: src_id|980a2569-7082ac97-e235e5a0-1ff4d0d9_0 is DEAD
2025-10-09T21:49:39.243805Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|980a2569-7082ac97-e235e5a0-1ff4d0d9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:39.243816Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|980a2569-7082ac97-e235e5a0-1ff4d0d9_0] Write session will restart in 0.007817s
2025-10-09T21:49:39.243859Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|980a2569-7082ac97-e235e5a0-1ff4d0d9_0] Write session: Do CDS request
2025-10-09T21:49:39.243863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|980a2569-7082ac97-e235e5a0-1ff4d0d9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:39.252527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|980a2569-7082ac97-e235e5a0-1ff4d0d9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:39.252614Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:39.252628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1584
2025-10-09T21:49:39.252746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1584 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:39.252778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1584 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:39.252788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1584 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:39.252792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:39.253177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1584 sessionId:  describe result for acl check
2025-10-09T21:49:39.253205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:39.253212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:39.253214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:39.253220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342497491406265:7926] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:39.253698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342497491406265:7926] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:39.255273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:39.255479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342497491406265:7926] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:39.258271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342497491406265:7926] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:39.258282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342497491406265:7926] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:39.258284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342497491406265:7926] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:39.258291Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1584 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:39.258869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:39.259060Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:39.259069Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:39.259078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:39.259123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:39.259150Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342497129578675:4756]
2025-10-09T21:49:39.259570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:39.259820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:39.259847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:39.260058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:39.260094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.260095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.260096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.260097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.260097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.260098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.260099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.260099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.260100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.260103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.260110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.260112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.260115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.260117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.260119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.260121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.260123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.260125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.260130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.260135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.260140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.260143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.260146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.260149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.260151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.260154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.260157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.260160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.260163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.260166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.260169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.260182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.260185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.260187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.260190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.260193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.260222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:39.267205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:39.268369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:39.268382Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:39.268383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:39.268386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:39.268392Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1583 [2:7559342497129578675:4756]
2025-10-09T21:49:39.268401Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:39.268416Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:39.268421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:39.268424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:39.268426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:39.268439Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:39.268448Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:39.268453Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:39.268491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.268496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.268497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.268499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:39.268552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:39.268557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.268564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.268571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.268580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:39.268597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342497129578668:4756]
2025-10-09T21:49:39.268602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342497129578668:4756]
2025-10-09T21:49:39.268604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342497129578668:4756]
2025-10-09T21:49:39.268610Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:39.268620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:39.268620Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:39.268624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:39.268627Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:39.268629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:39.268631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:39.268640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342497491406325:7926], now have 1 active actors on pipe
2025-10-09T21:49:39.268643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342497491406152:2432], now have 1 active actors on pipe
2025-10-09T21:49:39.268665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342497129578668:4756]
2025-10-09T21:49:39.268674Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:39.268688Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:39.268695Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1583
2025-10-09T21:49:39.268695Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1583
2025-10-09T21:49:39.268753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:39.268760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:39.268781Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|274a76f9-c20fcd8-4eec5a92-bd7bfa91_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:39.268791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:39.268795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:39.268797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:39.268804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:39.268811Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:39.268825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:39.268972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:39.268978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:39.268988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:39.269070Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1584 partition: 0 MaxSeqNo: 25 sessionId: src_id|274a76f9-c20fcd8-4eec5a92-bd7bfa91_0
2025-10-09T21:49:39.269375Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|980a2569-7082ac97-e235e5a0-1ff4d0d9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|274a76f9-c20fcd8-4eec5a92-bd7bfa91_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:39.271749Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|274a76f9-c20fcd8-4eec5a92-bd7bfa91_0] Write session: try to update token
2025-10-09T21:49:39.271766Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|274a76f9-c20fcd8-4eec5a92-bd7bfa91_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:39.282814Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:39.282832Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:39.282834Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:39.282835Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:39.282844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342497129578668:4756]
2025-10-09T21:49:39.282848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342497129578668:4756]
2025-10-09T21:49:39.282851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342497129578668:4756]
2025-10-09T21:49:39.282888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:39.282893Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:39.282895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.282898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:39.282899Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:39.282902Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:39.283069Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:39.283939Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1584 sessionId: src_id|274a76f9-c20fcd8-4eec5a92-bd7bfa91_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:39.286201Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:39.343347Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342497491406336:2585]
2025-10-09T21:49:39.343646Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 1583, pipe: [1:7559342497491406336:2585]
2025-10-09T21:49:39.428218Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:39.428779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:39.428793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:39.428832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342497491406336:2585], now have 1 active actors on pipe
2025-10-09T21:49:39.429018Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:39.429105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:39.429110Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:39.429118Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:39.429145Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342497491406336:2585]
2025-10-09T21:49:39.429197Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1584 reason: error at write: Disconnected sessionId: src_id|274a76f9-c20fcd8-4eec5a92-bd7bfa91_0
2025-10-09T21:49:39.429387Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1584 sessionId: src_id|274a76f9-c20fcd8-4eec5a92-bd7bfa91_0 is DEAD
2025-10-09T21:49:39.429566Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|274a76f9-c20fcd8-4eec5a92-bd7bfa91_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:39.429580Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|274a76f9-c20fcd8-4eec5a92-bd7bfa91_0] Write session will restart in 0.008068s
2025-10-09T21:49:39.429618Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|274a76f9-c20fcd8-4eec5a92-bd7bfa91_0] Write session: Do CDS request
2025-10-09T21:49:39.429625Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|274a76f9-c20fcd8-4eec5a92-bd7bfa91_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:39.438543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|274a76f9-c20fcd8-4eec5a92-bd7bfa91_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:39.438655Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:39.438670Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1585
2025-10-09T21:49:39.438785Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1585 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:39.438824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1585 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:39.438835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1585 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:39.438843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:39.439228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1585 sessionId:  describe result for acl check
2025-10-09T21:49:39.439260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:39.439265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:39.439266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:39.439270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342497491406494:7916] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:39.439730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342497491406494:7916] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:39.440452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:39.441370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342497491406494:7916] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:39.442947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:39.443147Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:39.443158Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:39.443171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:39.443230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:39.443282Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342497129578873:4720]
2025-10-09T21:49:39.443879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:39.444012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342497491406494:7916] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:39.444025Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342497491406494:7916] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:39.444028Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342497491406494:7916] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:39.444035Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1585 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:39.444185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:39.444226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:39.444540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:39.444591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.444597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.444598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.444599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.444600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.444602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.444603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.444604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.444605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.444610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.444620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.444625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.444629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.444633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.444637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.444641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.444644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.444648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.444656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.444664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.444670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.444676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.444680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.444686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.444692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.444698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.444708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.444713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.444720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.444726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.444736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.444741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.444746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.444751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.444756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.444762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.444791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:39.451185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:39.452366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:39.452383Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:39.452387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:39.452393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:39.452399Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1584 [2:7559342497129578873:4720]
2025-10-09T21:49:39.452417Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:39.452442Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:39.452447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:39.452451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:39.452454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:39.452475Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:39.452486Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:39.452493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:39.452563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.452572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.452574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.452578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:39.452650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:39.452658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.452672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.452689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.452712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:39.452741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342497491406531:7916], now have 1 active actors on pipe
2025-10-09T21:49:39.452749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342497491406363:2432], now have 1 active actors on pipe
2025-10-09T21:49:39.452756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342497129578867:4720]
2025-10-09T21:49:39.452759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342497129578867:4720]
2025-10-09T21:49:39.452762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342497129578867:4720]
2025-10-09T21:49:39.452778Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:39.452796Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:39.452804Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:39.452807Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:39.452810Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:39.452816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:39.452820Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:39.452810Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1584
2025-10-09T21:49:39.452859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342497129578867:4720]
2025-10-09T21:49:39.452871Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:39.452880Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:39.452824Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1584
2025-10-09T21:49:39.452889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:39.452899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:39.452951Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8379e42f-545f214d-8d491b07-e45ecae2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:39.452964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:39.452968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:39.452975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:39.452986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:39.452995Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:39.453019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:39.453165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:39.453173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:39.453188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:39.453271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1585 partition: 0 MaxSeqNo: 25 sessionId: src_id|8379e42f-545f214d-8d491b07-e45ecae2_0
2025-10-09T21:49:39.453634Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|274a76f9-c20fcd8-4eec5a92-bd7bfa91_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8379e42f-545f214d-8d491b07-e45ecae2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:39.456056Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8379e42f-545f214d-8d491b07-e45ecae2_0] Write session: try to update token
2025-10-09T21:49:39.456074Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8379e42f-545f214d-8d491b07-e45ecae2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:39.467412Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:39.467430Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:39.467432Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:39.467433Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:39.467441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342497129578867:4720]
2025-10-09T21:49:39.467445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342497129578867:4720]
2025-10-09T21:49:39.467448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342497129578867:4720]
2025-10-09T21:49:39.467484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:39.467491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.467493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:39.467501Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:39.467507Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:39.467511Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:39.467617Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:39.469221Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1585 sessionId: src_id|8379e42f-545f214d-8d491b07-e45ecae2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:39.471543Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:39.529156Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342497491406540:2585]
2025-10-09T21:49:39.529378Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 1584, pipe: [1:7559342497491406540:2585]
2025-10-09T21:49:39.613914Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:39.614560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:39.614574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:39.614613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342497491406540:2585], now have 1 active actors on pipe
2025-10-09T21:49:39.614802Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:39.614844Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:39.614846Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:39.614858Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:39.614901Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342497491406540:2585]
2025-10-09T21:49:39.614930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1585 reason: error at write: Disconnected sessionId: src_id|8379e42f-545f214d-8d491b07-e45ecae2_0
2025-10-09T21:49:39.615132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1585 sessionId: src_id|8379e42f-545f214d-8d491b07-e45ecae2_0 is DEAD
2025-10-09T21:49:39.615306Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8379e42f-545f214d-8d491b07-e45ecae2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:39.615315Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8379e42f-545f214d-8d491b07-e45ecae2_0] Write session will restart in 0.005028s
2025-10-09T21:49:39.615341Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8379e42f-545f214d-8d491b07-e45ecae2_0] Write session: Do CDS request
2025-10-09T21:49:39.615348Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8379e42f-545f214d-8d491b07-e45ecae2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:39.621510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8379e42f-545f214d-8d491b07-e45ecae2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:39.621625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:39.621639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1586
2025-10-09T21:49:39.621788Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1586 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:39.621822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1586 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:39.621827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1586 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:39.621831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:39.622251Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1586 sessionId:  describe result for acl check
2025-10-09T21:49:39.622280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:39.622296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:39.622298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:39.622305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342497491406558:7907] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:39.622780Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342497491406558:7907] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:39.625277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342497491406558:7907] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:39.627518Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342497491406558:7907] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:39.627532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342497491406558:7907] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:39.627536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342497491406558:7907] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:39.627543Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1586 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:39.628325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:39.631126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:39.631311Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:39.631320Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:39.631332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:39.631382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:39.631421Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342497129579036:4718]
2025-10-09T21:49:39.632016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:39.632305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:39.632345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:39.632581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:39.632618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.632623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.632623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.632624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.632625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.632625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.632626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.632627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.632627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.632630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.632637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.632639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.632643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.632647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.632651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.632655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.632659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.632662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.632669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.632677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.632683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.632686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.632689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.632693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.632695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.632698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.632701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.632704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.632706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.632709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.632712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.632715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.632717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.632720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.632723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.632726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.632742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:39.639198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:39.640312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:39.640329Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:39.640332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:39.640336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:39.640343Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1585 [2:7559342497129579036:4718]
2025-10-09T21:49:39.640356Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:39.640375Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:39.640383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:39.640386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:39.640389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:39.640404Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:39.640414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:39.640426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:39.640482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.640489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.640491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.640494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:39.640568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:39.640577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.640589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.640607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.640629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:39.640650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342497491406711:7907], now have 1 active actors on pipe
2025-10-09T21:49:39.640658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342497491406543:2432], now have 1 active actors on pipe
2025-10-09T21:49:39.640666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342497129579031:4718]
2025-10-09T21:49:39.640670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342497129579031:4718]
2025-10-09T21:49:39.640673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342497129579031:4718]
2025-10-09T21:49:39.640687Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:39.640705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:39.640708Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:39.640711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:39.640713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:39.640726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:39.640729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:39.640759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342497129579031:4718]
2025-10-09T21:49:39.640767Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:39.640769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:39.640752Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1585
2025-10-09T21:49:39.640791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:39.640801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:39.640837Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9f0ec258-7c240eef-fa0834d1-5d3c0350_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:39.640849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:39.640854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:39.640856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:39.640866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:39.640880Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:39.640902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:39.640956Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1585
2025-10-09T21:49:39.641036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:39.641044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:39.641064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:39.641166Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1586 partition: 0 MaxSeqNo: 25 sessionId: src_id|9f0ec258-7c240eef-fa0834d1-5d3c0350_0
2025-10-09T21:49:39.641527Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8379e42f-545f214d-8d491b07-e45ecae2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9f0ec258-7c240eef-fa0834d1-5d3c0350_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:39.643818Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f0ec258-7c240eef-fa0834d1-5d3c0350_0] Write session: try to update token
2025-10-09T21:49:39.643833Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f0ec258-7c240eef-fa0834d1-5d3c0350_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:39.655374Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:39.655392Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:39.655393Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:39.655395Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:39.655404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342497129579031:4718]
2025-10-09T21:49:39.655407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342497129579031:4718]
2025-10-09T21:49:39.655411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342497129579031:4718]
2025-10-09T21:49:39.655452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:39.655459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.655462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:39.655464Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:39.655468Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:39.655474Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:39.655596Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:39.655815Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1586 sessionId: src_id|9f0ec258-7c240eef-fa0834d1-5d3c0350_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:39.658166Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:39.715024Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342497491406725:2585]
2025-10-09T21:49:39.715288Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 1585, pipe: [1:7559342497491406725:2585]
2025-10-09T21:49:39.801686Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:39.802225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:39.802243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:39.802291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342497491406725:2585], now have 1 active actors on pipe
2025-10-09T21:49:39.802532Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:39.802559Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342497491406725:2585]
2025-10-09T21:49:39.802561Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:39.802564Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:39.802586Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:39.802631Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1586 reason: error at write: Disconnected sessionId: src_id|9f0ec258-7c240eef-fa0834d1-5d3c0350_0
2025-10-09T21:49:39.802792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1586 sessionId: src_id|9f0ec258-7c240eef-fa0834d1-5d3c0350_0 is DEAD
2025-10-09T21:49:39.802972Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f0ec258-7c240eef-fa0834d1-5d3c0350_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:39.802989Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f0ec258-7c240eef-fa0834d1-5d3c0350_0] Write session will restart in 0.005603s
2025-10-09T21:49:39.803010Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f0ec258-7c240eef-fa0834d1-5d3c0350_0] Write session: Do CDS request
2025-10-09T21:49:39.803015Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f0ec258-7c240eef-fa0834d1-5d3c0350_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:39.809442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f0ec258-7c240eef-fa0834d1-5d3c0350_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:39.809570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:39.809583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1587
2025-10-09T21:49:39.809698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1587 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:39.809738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1587 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:39.809748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1587 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:39.809752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:39.810327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1587 sessionId:  describe result for acl check
2025-10-09T21:49:39.810368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:39.810376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:39.810378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:39.810384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342497491406777:7905] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:39.810914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342497491406777:7905] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:39.812807Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342497491406777:7905] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:39.816721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:39.816806Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342497491406777:7905] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:39.816818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342497491406777:7905] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:39.816822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342497491406777:7905] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:39.816829Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1587 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:39.817122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:39.817293Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:39.817303Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:39.817314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:39.817359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:39.817396Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342497129579203:4715]
2025-10-09T21:49:39.817836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:39.818120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:39.818163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:39.818457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:39.818503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.818509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.818511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.818512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.818513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.818514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.818515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.818517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.818518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.818523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.818536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.818541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.818545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.818549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.818553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.818557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.818561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.818565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.818571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.818579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:39.818585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.818591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:39.818595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.818600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:39.818604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.818610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:39.818619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.818625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:39.818629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.818634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:39.818639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.818648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.818653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.818658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.818663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.818668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.818697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:39.824994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:39.826145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:39.826159Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:39.826161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:39.826165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:39.826171Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1586 [2:7559342497129579203:4715]
2025-10-09T21:49:39.826186Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:39.826202Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:39.826205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:39.826208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:39.826210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:39.826228Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:39.826239Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:39.826245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:39.826304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.826311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.826313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.826315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:39.826383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:39.826391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.826402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:39.826419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:39.826441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:39.826465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342497491406898:7905], now have 1 active actors on pipe
2025-10-09T21:49:39.826470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342497491406727:2432], now have 1 active actors on pipe
2025-10-09T21:49:39.826479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342497129579197:4715]
2025-10-09T21:49:39.826481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342497129579197:4715]
2025-10-09T21:49:39.826483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342497129579197:4715]
2025-10-09T21:49:39.826496Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:39.826516Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:39.826516Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:39.826521Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:39.826525Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:39.826530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:39.826533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:39.826554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342497129579197:4715]
2025-10-09T21:49:39.826533Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1586
2025-10-09T21:49:39.826562Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:39.826538Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1586
2025-10-09T21:49:39.826563Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:39.826586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:39.826591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:39.826629Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4f8ce2cd-da3c26f5-672ba961-5ef89e00_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:39.826641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:39.826649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:39.826652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:39.826661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:39.826677Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:39.826698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:39.826861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:39.826871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:39.826885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:39.826997Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1587 partition: 0 MaxSeqNo: 25 sessionId: src_id|4f8ce2cd-da3c26f5-672ba961-5ef89e00_0
2025-10-09T21:49:39.827299Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f0ec258-7c240eef-fa0834d1-5d3c0350_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4f8ce2cd-da3c26f5-672ba961-5ef89e00_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:39.829619Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4f8ce2cd-da3c26f5-672ba961-5ef89e00_0] Write session: try to update token
2025-10-09T21:49:39.829633Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4f8ce2cd-da3c26f5-672ba961-5ef89e00_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:39.840901Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:39.840924Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:39.840926Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:39.840929Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:39.840939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342497129579197:4715]
2025-10-09T21:49:39.840946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342497129579197:4715]
2025-10-09T21:49:39.840950Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342497129579197:4715]
2025-10-09T21:49:39.840990Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:39.840993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:39.840995Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:39.840998Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:39.841000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:39.841003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:39.841144Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:39.842026Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1587 sessionId: src_id|4f8ce2cd-da3c26f5-672ba961-5ef89e00_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:39.844306Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:39.902979Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342497491406912:2585]
2025-10-09T21:49:39.903303Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 1586, pipe: [1:7559342497491406912:2585]
2025-10-09T21:49:39.993743Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:39.993998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:39.994011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:39.994065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342497491406912:2585], now have 1 active actors on pipe
2025-10-09T21:49:39.994262Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:39.994361Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:39.994373Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:39.994379Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:39.994419Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342497491406912:2585]
2025-10-09T21:49:39.994462Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1587 reason: error at write: Disconnected sessionId: src_id|4f8ce2cd-da3c26f5-672ba961-5ef89e00_0
2025-10-09T21:49:39.994662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1587 sessionId: src_id|4f8ce2cd-da3c26f5-672ba961-5ef89e00_0 is DEAD
2025-10-09T21:49:39.994839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f8ce2cd-da3c26f5-672ba961-5ef89e00_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:39.994850Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f8ce2cd-da3c26f5-672ba961-5ef89e00_0] Write session will restart in 0.007973s
2025-10-09T21:49:39.994877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f8ce2cd-da3c26f5-672ba961-5ef89e00_0] Write session: Do CDS request
2025-10-09T21:49:39.994884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f8ce2cd-da3c26f5-672ba961-5ef89e00_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:40.003558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4f8ce2cd-da3c26f5-672ba961-5ef89e00_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:40.003910Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:40.003929Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1588
2025-10-09T21:49:40.004072Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1588 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:40.004109Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1588 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:40.004121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1588 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:40.004125Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:40.004607Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1588 sessionId:  describe result for acl check
2025-10-09T21:49:40.004649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:40.004656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:40.004658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:40.004665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342501786374346:7236] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:40.005283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342501786374346:7236] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:40.005923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:40.007322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342501786374346:7236] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:40.008685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:40.008925Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:40.008935Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:40.008946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:40.009002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:40.009042Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342501424546662:4706]
2025-10-09T21:49:40.009612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342501786374346:7236] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:40.009623Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342501786374346:7236] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:40.009627Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342501786374346:7236] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:40.009635Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1588 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:40.009657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:40.009923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:40.009956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:40.010226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:40.010264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.010269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.010270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.010271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.010271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.010272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.010273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.010273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.010274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.010277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.010284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.010286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.010288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.010291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.010293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.010295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.010297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.010299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.010304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.010309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.010313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.010316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.010320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.010324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.010326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.010329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.010332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.010335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.010337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.010340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.010343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.010345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.010348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.010351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.010354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.010357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.010373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:40.016838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:40.018400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:40.018416Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:40.018418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:40.018423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:40.018431Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1587 [2:7559342501424546662:4706]
2025-10-09T21:49:40.018448Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:40.018468Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:40.018472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:40.018475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:40.018479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:40.018497Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:40.018508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:40.018516Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:40.018573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.018581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.018583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.018586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:40.018671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:40.018680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.018691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.018706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.018730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:40.018758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342501786374381:7236], now have 1 active actors on pipe
2025-10-09T21:49:40.018768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342497491406916:2432], now have 1 active actors on pipe
2025-10-09T21:49:40.018777Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342501424546656:4706]
2025-10-09T21:49:40.018784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342501424546656:4706]
2025-10-09T21:49:40.018786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342501424546656:4706]
2025-10-09T21:49:40.018800Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:40.018789Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1587
2025-10-09T21:49:40.018792Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1587
2025-10-09T21:49:40.018818Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:40.018819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:40.018827Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:40.018829Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:40.018833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:40.018836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:40.018883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.018892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.018904Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342501424546656:4706]
2025-10-09T21:49:40.018915Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:40.018922Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:40.018949Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a193ae7a-e1fc8fc6-d678fe21-ac9e1693_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:40.018962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:40.018966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:40.018973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:40.018983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:40.018994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:40.019024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:40.019213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.019223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.019239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:40.019325Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1588 partition: 0 MaxSeqNo: 25 sessionId: src_id|a193ae7a-e1fc8fc6-d678fe21-ac9e1693_0
2025-10-09T21:49:40.019658Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4f8ce2cd-da3c26f5-672ba961-5ef89e00_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a193ae7a-e1fc8fc6-d678fe21-ac9e1693_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:40.022394Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a193ae7a-e1fc8fc6-d678fe21-ac9e1693_0] Write session: try to update token
2025-10-09T21:49:40.022409Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a193ae7a-e1fc8fc6-d678fe21-ac9e1693_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:40.033448Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:40.033472Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:40.033475Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:40.033478Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:40.033488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342501424546656:4706]
2025-10-09T21:49:40.033492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342501424546656:4706]
2025-10-09T21:49:40.033497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342501424546656:4706]
2025-10-09T21:49:40.033539Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:40.033545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:40.033546Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:40.033548Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:40.033553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.033570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:40.033714Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:40.035056Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1588 sessionId: src_id|a193ae7a-e1fc8fc6-d678fe21-ac9e1693_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:40.037661Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:40.094463Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342501786374393:2585]
2025-10-09T21:49:40.094763Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 1587, pipe: [1:7559342501786374393:2585]
2025-10-09T21:49:40.181459Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:40.181936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.181961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.182006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342501786374393:2585], now have 1 active actors on pipe
2025-10-09T21:49:40.182254Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:40.182301Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:40.182317Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:40.182334Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:40.182381Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1588 reason: error at write: Disconnected sessionId: src_id|a193ae7a-e1fc8fc6-d678fe21-ac9e1693_0
2025-10-09T21:49:40.182381Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342501786374393:2585]
2025-10-09T21:49:40.182582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1588 sessionId: src_id|a193ae7a-e1fc8fc6-d678fe21-ac9e1693_0 is DEAD
2025-10-09T21:49:40.182736Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a193ae7a-e1fc8fc6-d678fe21-ac9e1693_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:40.182743Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a193ae7a-e1fc8fc6-d678fe21-ac9e1693_0] Write session will restart in 0.009925s
2025-10-09T21:49:40.182763Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a193ae7a-e1fc8fc6-d678fe21-ac9e1693_0] Write session: Do CDS request
2025-10-09T21:49:40.182768Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a193ae7a-e1fc8fc6-d678fe21-ac9e1693_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:40.192542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:40.193483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a193ae7a-e1fc8fc6-d678fe21-ac9e1693_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:40.193631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:40.193647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1589
2025-10-09T21:49:40.193776Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1589 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:40.193809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1589 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:40.193821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1589 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:40.193825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:40.194294Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1589 sessionId:  describe result for acl check
2025-10-09T21:49:40.194334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:40.194340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:40.194341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:40.194347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342501786374537:7884] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:40.194947Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342501786374537:7884] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:40.195405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:40.195605Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:40.195615Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:40.195626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:40.195686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:40.195728Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342501424546827:4701]
2025-10-09T21:49:40.196161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:40.196394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:40.196438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:40.196677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:40.196726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.196732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.196734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.196735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.196736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.196738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.196739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.196740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.196741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.196746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.196755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.196761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.196764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.196766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.196768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.196770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.196772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.196774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.196779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.196784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.196789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.196792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.196795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.196802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.196805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.196808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.196810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.196820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.196824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.196829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.196837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.196842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.196850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.196856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.196866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.196876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.196909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:40.197141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342501786374537:7884] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:40.202797Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342501786374537:7884] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:40.202812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342501786374537:7884] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:40.202817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342501786374537:7884] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:40.202829Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1589 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:40.203173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:40.204692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:40.204708Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:40.204710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:40.204715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:40.204721Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1588 [2:7559342501424546827:4701]
2025-10-09T21:49:40.204736Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:40.204756Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:40.204764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:40.204767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:40.204770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:40.204787Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:40.204797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:40.204809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:40.204868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.204875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.204877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.204880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:40.204950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:40.204958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.204970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.204980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.205001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:40.205028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342501786374575:7884], now have 1 active actors on pipe
2025-10-09T21:49:40.205036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342501786374398:2432], now have 1 active actors on pipe
2025-10-09T21:49:40.205044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342501424546821:4701]
2025-10-09T21:49:40.205050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342501424546821:4701]
2025-10-09T21:49:40.205053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342501424546821:4701]
2025-10-09T21:49:40.205070Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:40.205089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:40.205103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:40.205089Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1588
2025-10-09T21:49:40.205108Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:40.205110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:40.205112Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:40.205114Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:40.205139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342501424546821:4701]
2025-10-09T21:49:40.205147Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:40.205148Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:40.205163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.205172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.205176Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1588
2025-10-09T21:49:40.205211Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|80d37c8d-e83dac2b-48b7cc31-3a8b4e66_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:40.205224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:40.205232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:40.205234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:40.205244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:40.205259Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:40.205284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:40.205402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.205410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.205423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:40.205497Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1589 partition: 0 MaxSeqNo: 25 sessionId: src_id|80d37c8d-e83dac2b-48b7cc31-3a8b4e66_0
2025-10-09T21:49:40.205855Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a193ae7a-e1fc8fc6-d678fe21-ac9e1693_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|80d37c8d-e83dac2b-48b7cc31-3a8b4e66_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:40.208227Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80d37c8d-e83dac2b-48b7cc31-3a8b4e66_0] Write session: try to update token
2025-10-09T21:49:40.208241Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80d37c8d-e83dac2b-48b7cc31-3a8b4e66_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:40.219442Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:40.219464Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:40.219466Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:40.219468Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:40.219479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342501424546821:4701]
2025-10-09T21:49:40.219484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342501424546821:4701]
2025-10-09T21:49:40.219488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342501424546821:4701]
2025-10-09T21:49:40.219533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:40.219540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.219548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:40.219562Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:40.219575Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:40.219578Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:40.219687Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:40.220560Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1589 sessionId: src_id|80d37c8d-e83dac2b-48b7cc31-3a8b4e66_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:40.222917Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:40.282383Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342501786374588:2585]
2025-10-09T21:49:40.282637Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 1588, pipe: [1:7559342501786374588:2585]
2025-10-09T21:49:40.374652Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:40.375141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.375155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.375194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342501786374588:2585], now have 1 active actors on pipe
2025-10-09T21:49:40.375365Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:40.375466Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:40.375480Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:40.375482Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:40.375533Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1589 reason: error at write: Disconnected sessionId: src_id|80d37c8d-e83dac2b-48b7cc31-3a8b4e66_0
2025-10-09T21:49:40.375535Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342501786374588:2585]
2025-10-09T21:49:40.375722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1589 sessionId: src_id|80d37c8d-e83dac2b-48b7cc31-3a8b4e66_0 is DEAD
2025-10-09T21:49:40.375875Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80d37c8d-e83dac2b-48b7cc31-3a8b4e66_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:40.375887Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80d37c8d-e83dac2b-48b7cc31-3a8b4e66_0] Write session will restart in 0.006767s
2025-10-09T21:49:40.375915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80d37c8d-e83dac2b-48b7cc31-3a8b4e66_0] Write session: Do CDS request
2025-10-09T21:49:40.375921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80d37c8d-e83dac2b-48b7cc31-3a8b4e66_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:40.383546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80d37c8d-e83dac2b-48b7cc31-3a8b4e66_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:40.383578Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:40.383593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1590
2025-10-09T21:49:40.383773Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1590 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:40.383819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1590 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:40.383829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1590 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:40.383834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:40.384278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1590 sessionId:  describe result for acl check
2025-10-09T21:49:40.384316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:40.384322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:40.384324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:40.384330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342501786374668:7873] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:40.384898Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342501786374668:7873] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:40.386729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342501786374668:7873] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:40.390861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:40.390885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342501786374668:7873] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:40.390896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342501786374668:7873] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:40.390899Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342501786374668:7873] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:40.390905Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1590 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:40.391325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:40.391527Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:40.391537Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:40.391549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:40.391603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:40.391642Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342501424546995:4690]
2025-10-09T21:49:40.392242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:40.392530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:40.392574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:40.393140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:40.393206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.393213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.393215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.393216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.393217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.393218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.393220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.393221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.393222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.393228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.393239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.393243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.393247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.393251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.393255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.393259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.393264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.393268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.393275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.393283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.393289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.393295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.393306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.393311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.393315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.393321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.393326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.393331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.393336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.393341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.393351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.393356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.393361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.393372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.393376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.393381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.393408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:40.399909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:40.401072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:40.401085Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:40.401087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:40.401090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:40.401096Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1589 [2:7559342501424546995:4690]
2025-10-09T21:49:40.401105Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:40.401119Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:40.401125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:40.401127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:40.401128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:40.401141Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:40.401148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:40.401156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:40.401230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.401240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.401242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.401244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:40.401304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:40.401310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.401318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.401324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.401337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:40.401352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342501424546989:4690]
2025-10-09T21:49:40.401358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342501424546989:4690]
2025-10-09T21:49:40.401359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342501424546989:4690]
2025-10-09T21:49:40.401365Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:40.401374Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:40.401377Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:40.401377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342501786374591:2432], now have 1 active actors on pipe
2025-10-09T21:49:40.401379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:40.401381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342501786374764:7873], now have 1 active actors on pipe
2025-10-09T21:49:40.401388Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:40.401401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:40.401409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:40.401442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342501424546989:4690]
2025-10-09T21:49:40.401456Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:40.401465Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:40.401450Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1589
2025-10-09T21:49:40.401500Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1589
2025-10-09T21:49:40.401602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.401612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.401638Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5403ce59-1e65f97d-6f7a06b3-15ccf6c9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:40.401651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:40.401659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:40.401661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:40.401671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:40.401686Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:40.401705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:40.401933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.401940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.401954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:40.402047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1590 partition: 0 MaxSeqNo: 25 sessionId: src_id|5403ce59-1e65f97d-6f7a06b3-15ccf6c9_0
2025-10-09T21:49:40.402411Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80d37c8d-e83dac2b-48b7cc31-3a8b4e66_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5403ce59-1e65f97d-6f7a06b3-15ccf6c9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:40.404953Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5403ce59-1e65f97d-6f7a06b3-15ccf6c9_0] Write session: try to update token
2025-10-09T21:49:40.404971Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5403ce59-1e65f97d-6f7a06b3-15ccf6c9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:40.416177Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:40.416194Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:40.416196Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:40.416197Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:40.416204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342501424546989:4690]
2025-10-09T21:49:40.416207Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342501424546989:4690]
2025-10-09T21:49:40.416209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342501424546989:4690]
2025-10-09T21:49:40.416242Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:40.416245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:40.416247Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:40.416251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.416252Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:40.416253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:40.416384Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:40.418812Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1590 sessionId: src_id|5403ce59-1e65f97d-6f7a06b3-15ccf6c9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:40.421429Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:40.475846Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342501786374798:2585]
2025-10-09T21:49:40.476133Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 1589, pipe: [1:7559342501786374798:2585]
2025-10-09T21:49:40.564925Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:40.565166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.565178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.565223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342501786374798:2585], now have 1 active actors on pipe
2025-10-09T21:49:40.565413Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:40.565429Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:40.565467Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:40.565496Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1590 reason: error at write: Disconnected sessionId: src_id|5403ce59-1e65f97d-6f7a06b3-15ccf6c9_0
2025-10-09T21:49:40.565533Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342501786374798:2585]
2025-10-09T21:49:40.565566Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:40.565669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1590 sessionId: src_id|5403ce59-1e65f97d-6f7a06b3-15ccf6c9_0 is DEAD
2025-10-09T21:49:40.565843Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5403ce59-1e65f97d-6f7a06b3-15ccf6c9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:40.565855Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5403ce59-1e65f97d-6f7a06b3-15ccf6c9_0] Write session will restart in 0.005945s
2025-10-09T21:49:40.565882Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5403ce59-1e65f97d-6f7a06b3-15ccf6c9_0] Write session: Do CDS request
2025-10-09T21:49:40.565888Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5403ce59-1e65f97d-6f7a06b3-15ccf6c9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:40.572506Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5403ce59-1e65f97d-6f7a06b3-15ccf6c9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:40.572564Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:40.572575Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1591
2025-10-09T21:49:40.572729Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1591 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:40.572766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1591 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:40.572772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1591 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:40.572775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:40.573243Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1591 sessionId:  describe result for acl check
2025-10-09T21:49:40.573273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:40.573275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:40.573276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:40.573281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342501786374867:7870] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:40.573854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342501786374867:7870] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:40.576394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342501786374867:7870] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:40.579658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:40.579849Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342501786374867:7870] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:40.579859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342501786374867:7870] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:40.579862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342501786374867:7870] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:40.579868Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1591 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:40.580004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:40.580341Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:40.580358Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:40.580372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:40.580435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:40.580478Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342501424547200:4682]
2025-10-09T21:49:40.581059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:40.581329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:40.581370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:40.581693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:40.581747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.581754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.581754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.581756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.581757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.581758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.581759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.581760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.581761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.581765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.581775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.581778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.581783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.581786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.581790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.581793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.581797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.581800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.581807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.581814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.581820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.581826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.581830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.581836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.581847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.581852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.581858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.581868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.581872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.581877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.581886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.581891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.581897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.581905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.581909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.581914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.581944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:40.588234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:40.589411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:40.589429Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:40.589432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:40.589437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:40.589444Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1590 [2:7559342501424547200:4682]
2025-10-09T21:49:40.589461Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:40.589484Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:40.589492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:40.589496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:40.589498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:40.589517Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:40.589528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:40.589542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:40.589628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.589635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.589638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.589640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:40.589718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:40.589726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.589739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.589756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.589779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:40.589806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342501786374801:2432], now have 1 active actors on pipe
2025-10-09T21:49:40.589815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342501786374976:7870], now have 1 active actors on pipe
2025-10-09T21:49:40.589823Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342501424547194:4682]
2025-10-09T21:49:40.589831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342501424547194:4682]
2025-10-09T21:49:40.589833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342501424547194:4682]
2025-10-09T21:49:40.589847Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:40.589863Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:40.589867Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:40.589869Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:40.589872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:40.589854Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1590
2025-10-09T21:49:40.589885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:40.589854Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1590
2025-10-09T21:49:40.589889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:40.589944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.589954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.589968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342501424547194:4682]
2025-10-09T21:49:40.589975Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:40.589982Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:40.590010Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9ec4c13-e112d9f0-3d450c60-aa74bddf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:40.590024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:40.590043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:40.590046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:40.590056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:40.590071Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:40.590099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:40.590239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.590247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.590263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:40.590349Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1591 partition: 0 MaxSeqNo: 25 sessionId: src_id|9ec4c13-e112d9f0-3d450c60-aa74bddf_0
2025-10-09T21:49:40.590707Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5403ce59-1e65f97d-6f7a06b3-15ccf6c9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9ec4c13-e112d9f0-3d450c60-aa74bddf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:40.593429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ec4c13-e112d9f0-3d450c60-aa74bddf_0] Write session: try to update token
2025-10-09T21:49:40.593444Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ec4c13-e112d9f0-3d450c60-aa74bddf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:40.604146Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:40.604169Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:40.604171Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:40.604173Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:40.604186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342501424547194:4682]
2025-10-09T21:49:40.604191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342501424547194:4682]
2025-10-09T21:49:40.604195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342501424547194:4682]
2025-10-09T21:49:40.604234Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:40.604240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:40.604246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.604248Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:40.604249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:40.604252Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:40.604405Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:40.606454Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1591 sessionId: src_id|9ec4c13-e112d9f0-3d450c60-aa74bddf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:40.608801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:40.665852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342501786374987:2585]
2025-10-09T21:49:40.666126Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 1590, pipe: [1:7559342501786374987:2585]
2025-10-09T21:49:40.758850Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:40.759547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.759564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.759610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342501786374987:2585], now have 1 active actors on pipe
2025-10-09T21:49:40.759817Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:40.759870Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342501786374987:2585]
2025-10-09T21:49:40.759906Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:40.759979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1591 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9ec4c13-e112d9f0-3d450c60-aa74bddf_0
2025-10-09T21:49:40.760156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1591 sessionId: src_id|9ec4c13-e112d9f0-3d450c60-aa74bddf_0 is DEAD
2025-10-09T21:49:40.760224Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:40.760313Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ec4c13-e112d9f0-3d450c60-aa74bddf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:40.760326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ec4c13-e112d9f0-3d450c60-aa74bddf_0] Write session will restart in 0.006457s
2025-10-09T21:49:40.760356Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ec4c13-e112d9f0-3d450c60-aa74bddf_0] Write session: Do CDS request
2025-10-09T21:49:40.760363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ec4c13-e112d9f0-3d450c60-aa74bddf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:40.767383Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ec4c13-e112d9f0-3d450c60-aa74bddf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:40.767517Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:40.767531Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1592
2025-10-09T21:49:40.767650Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1592 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:40.767685Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1592 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:40.767697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1592 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:40.767701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:40.768149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1592 sessionId:  describe result for acl check
2025-10-09T21:49:40.768177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:40.768183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:40.768185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:40.768189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342501786375063:7866] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:40.768614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342501786375063:7866] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:40.770824Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342501786375063:7866] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:40.774374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:40.774432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342501786375063:7866] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:40.774444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342501786375063:7866] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:40.774448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342501786375063:7866] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:40.774455Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1592 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:40.774769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:40.774946Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:40.774958Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:40.774969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:40.775019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:40.775059Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342501424547372:4677]
2025-10-09T21:49:40.775627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:40.775908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:40.775957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:40.776255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:40.776312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.776318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.776320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.776321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.776322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.776323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.776325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.776326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.776327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.776332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.776343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.776347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.776351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.776355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.776360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.776364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.776367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.776371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.776377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.776384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.776390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.776394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.776399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.776403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.776409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.776415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.776424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.776449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.776454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.776458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.776463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.776468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.776477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.776483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.776489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.776499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.776529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:40.782754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:40.784161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:40.784175Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:40.784177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:40.784180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:40.784185Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1591 [2:7559342501424547372:4677]
2025-10-09T21:49:40.784198Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:40.784215Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:40.784218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:40.784222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:40.784224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:40.784239Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:40.784251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:40.784258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:40.784305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.784311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.784312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.784314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:40.784366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:40.784371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.784379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.784385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.784394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:40.784417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342501786375166:7866], now have 1 active actors on pipe
2025-10-09T21:49:40.784425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342501786374990:2432], now have 1 active actors on pipe
2025-10-09T21:49:40.784433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342501424547366:4677]
2025-10-09T21:49:40.784440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342501424547366:4677]
2025-10-09T21:49:40.784441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342501424547366:4677]
2025-10-09T21:49:40.784454Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:40.784466Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:40.784472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:40.784475Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:40.784477Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:40.784487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:40.784488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:40.784507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342501424547366:4677]
2025-10-09T21:49:40.784514Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:40.784492Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1591
2025-10-09T21:49:40.784520Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:40.784494Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1591
2025-10-09T21:49:40.784534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.784537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.784568Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|724eb707-fdc37aae-abe6db68-e7b1e536_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:40.784577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:40.784580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:40.784581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:40.784589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:40.784596Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:40.784618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:40.784747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.784753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.784763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:40.784857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1592 partition: 0 MaxSeqNo: 25 sessionId: src_id|724eb707-fdc37aae-abe6db68-e7b1e536_0
2025-10-09T21:49:40.785183Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ec4c13-e112d9f0-3d450c60-aa74bddf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|724eb707-fdc37aae-abe6db68-e7b1e536_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:40.787642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|724eb707-fdc37aae-abe6db68-e7b1e536_0] Write session: try to update token
2025-10-09T21:49:40.787660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|724eb707-fdc37aae-abe6db68-e7b1e536_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:40.798580Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:40.798596Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:40.798598Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:40.798599Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:40.798608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342501424547366:4677]
2025-10-09T21:49:40.798611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342501424547366:4677]
2025-10-09T21:49:40.798614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342501424547366:4677]
2025-10-09T21:49:40.798646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:40.798651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.798653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:40.798655Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:40.798659Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:40.798665Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:40.798768Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:40.801049Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1592 sessionId: src_id|724eb707-fdc37aae-abe6db68-e7b1e536_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:40.803361Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:40.859884Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342501786375179:2585]
2025-10-09T21:49:40.860174Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 1591, pipe: [1:7559342501786375179:2585]
2025-10-09T21:49:40.951575Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:40.952045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.952062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.952102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342501786375179:2585], now have 1 active actors on pipe
2025-10-09T21:49:40.952277Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:40.952297Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:40.952300Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:40.952300Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:40.952347Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342501786375179:2585]
2025-10-09T21:49:40.952362Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1592 reason: error at write: Disconnected sessionId: src_id|724eb707-fdc37aae-abe6db68-e7b1e536_0
2025-10-09T21:49:40.952526Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1592 sessionId: src_id|724eb707-fdc37aae-abe6db68-e7b1e536_0 is DEAD
2025-10-09T21:49:40.952727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|724eb707-fdc37aae-abe6db68-e7b1e536_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:40.952734Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|724eb707-fdc37aae-abe6db68-e7b1e536_0] Write session will restart in 0.008027s
2025-10-09T21:49:40.952757Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|724eb707-fdc37aae-abe6db68-e7b1e536_0] Write session: Do CDS request
2025-10-09T21:49:40.952763Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|724eb707-fdc37aae-abe6db68-e7b1e536_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:40.961709Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|724eb707-fdc37aae-abe6db68-e7b1e536_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:40.961774Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:40.961790Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1593
2025-10-09T21:49:40.961983Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1593 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:40.962016Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1593 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:40.962026Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1593 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:40.962070Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:40.962558Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1593 sessionId:  describe result for acl check
2025-10-09T21:49:40.962595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:40.962597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:40.962599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:40.962605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342501786375298:7862] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:40.963230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342501786375298:7862] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:40.965090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:40.965132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342501786375298:7862] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:40.967950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:40.968225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342501786375298:7862] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:40.968238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342501786375298:7862] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:40.968241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342501786375298:7862] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:40.968250Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1593 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:40.968387Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:40.968404Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:40.968422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:40.968503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:40.968547Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342501424547540:4675]
2025-10-09T21:49:40.969174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:40.969496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:40.969569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:40.969945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:40.970026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.970052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.970053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.970055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.970056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.970058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.970059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.970060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.970062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.970067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.970078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.970087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.970091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.970099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.970106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.970115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.970119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.970123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.970131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.970139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:40.970152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.970162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:40.970172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.970182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:40.970185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.970190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:40.970194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.970199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:40.970207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.970212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:40.970221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.970231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.970236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.970246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.970255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.970264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.970295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:40.976795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:40.978008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:40.978048Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:40.978051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:40.978056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:40.978062Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1592 [2:7559342501424547540:4675]
2025-10-09T21:49:40.978077Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:40.978100Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:40.978113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:40.978117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:40.978120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:40.978136Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:40.978146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:40.978155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:40.978218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.978225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.978228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.978231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:40.978314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:40.978324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.978337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:40.978354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:40.978376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:40.978404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342501786375359:7862], now have 1 active actors on pipe
2025-10-09T21:49:40.978412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342501786375185:2432], now have 1 active actors on pipe
2025-10-09T21:49:40.978420Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342501424547535:4675]
2025-10-09T21:49:40.978426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342501424547535:4675]
2025-10-09T21:49:40.978428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342501424547535:4675]
2025-10-09T21:49:40.978441Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:40.978455Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:40.978466Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:40.978469Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:40.978471Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:40.978474Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:40.978477Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:40.978460Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1592
2025-10-09T21:49:40.978471Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1592
2025-10-09T21:49:40.978511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342501424547535:4675]
2025-10-09T21:49:40.978523Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:40.978528Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:40.978544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.978552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.978602Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b0d270b5-fe8019a2-1be90675-e12c7590_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:40.978615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:40.978620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:40.978622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:40.978633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:40.978648Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:40.978670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:40.978846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:40.978856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:40.978871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:40.978983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1593 partition: 0 MaxSeqNo: 25 sessionId: src_id|b0d270b5-fe8019a2-1be90675-e12c7590_0
2025-10-09T21:49:40.979330Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|724eb707-fdc37aae-abe6db68-e7b1e536_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b0d270b5-fe8019a2-1be90675-e12c7590_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:40.981763Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0d270b5-fe8019a2-1be90675-e12c7590_0] Write session: try to update token
2025-10-09T21:49:40.981779Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0d270b5-fe8019a2-1be90675-e12c7590_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:40.992564Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:40.992588Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:40.992590Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:40.992593Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:40.992606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342501424547535:4675]
2025-10-09T21:49:40.992610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342501424547535:4675]
2025-10-09T21:49:40.992615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342501424547535:4675]
2025-10-09T21:49:40.992665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:40.992668Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:40.992672Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:40.992675Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:40.992677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:40.992681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:40.992824Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:40.994657Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1593 sessionId: src_id|b0d270b5-fe8019a2-1be90675-e12c7590_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:40.997053Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:41.052368Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342506081342667:2585]
2025-10-09T21:49:41.052642Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 1592, pipe: [1:7559342506081342667:2585]
2025-10-09T21:49:41.146556Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:41.147012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.147038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.147080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342506081342667:2585], now have 1 active actors on pipe
2025-10-09T21:49:41.147316Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:41.147314Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:41.147316Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342506081342667:2585]
2025-10-09T21:49:41.147327Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:41.147335Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:41.147391Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1593 reason: error at write: Disconnected sessionId: src_id|b0d270b5-fe8019a2-1be90675-e12c7590_0
2025-10-09T21:49:41.147578Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1593 sessionId: src_id|b0d270b5-fe8019a2-1be90675-e12c7590_0 is DEAD
2025-10-09T21:49:41.147712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0d270b5-fe8019a2-1be90675-e12c7590_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:41.147724Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0d270b5-fe8019a2-1be90675-e12c7590_0] Write session will restart in 0.006193s
2025-10-09T21:49:41.147751Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0d270b5-fe8019a2-1be90675-e12c7590_0] Write session: Do CDS request
2025-10-09T21:49:41.147758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0d270b5-fe8019a2-1be90675-e12c7590_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:41.155434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0d270b5-fe8019a2-1be90675-e12c7590_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:41.155545Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:41.155561Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1594
2025-10-09T21:49:41.155699Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1594 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:41.155731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1594 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:41.155741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1594 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:41.155744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:41.156096Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1594 sessionId:  describe result for acl check
2025-10-09T21:49:41.156127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:41.156128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:41.156129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:41.156135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342506081342765:7852] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:41.156596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342506081342765:7852] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:41.158613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342506081342765:7852] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:41.162107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:41.162276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342506081342765:7852] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:41.162289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342506081342765:7852] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:41.162292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342506081342765:7852] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:41.162301Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1594 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:41.162571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:41.162737Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:41.162747Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:41.162757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:41.162808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:41.162843Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342505719515007:4673]
2025-10-09T21:49:41.163439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:41.163735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:41.163776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:41.164091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:41.164141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.164146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.164148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.164149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.164150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.164151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.164153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.164154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.164155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.164160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.164171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.164175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.164180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.164184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.164188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.164196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.164200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.164208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.164214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.164227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.164238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.164248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.164256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.164262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.164271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.164280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.164289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.164298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.164305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.164311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.164317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.164326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.164335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.164340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.164348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.164357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.164389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:41.170726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:41.172214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:41.172229Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:41.172232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:41.172236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:41.172244Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1593 [2:7559342505719515007:4673]
2025-10-09T21:49:41.172259Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:41.172276Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:41.172279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:41.172282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:41.172285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:41.172304Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:41.172317Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:41.172332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:41.172388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.172396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.172401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.172404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:41.172475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:41.172483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.172494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.172509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.172532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:41.172559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342506081342669:2432], now have 1 active actors on pipe
2025-10-09T21:49:41.172567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342506081342847:7852], now have 1 active actors on pipe
2025-10-09T21:49:41.172575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342505719515001:4673]
2025-10-09T21:49:41.172577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342505719515001:4673]
2025-10-09T21:49:41.172579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342505719515001:4673]
2025-10-09T21:49:41.172592Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:41.172611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:41.172611Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:41.172615Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:41.172617Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:41.172631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:41.172639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:41.172611Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1593
2025-10-09T21:49:41.172629Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1593
2025-10-09T21:49:41.172679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342505719515001:4673]
2025-10-09T21:49:41.172690Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:41.172702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:41.172710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.172714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.172759Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|465a85b8-f7c0345b-1e7e74a1-3ea7e16d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:41.172771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:41.172775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:41.172783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:41.172793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:41.172807Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:41.172827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:41.172981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.172991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.173005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:41.173076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1594 partition: 0 MaxSeqNo: 25 sessionId: src_id|465a85b8-f7c0345b-1e7e74a1-3ea7e16d_0
2025-10-09T21:49:41.173403Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0d270b5-fe8019a2-1be90675-e12c7590_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|465a85b8-f7c0345b-1e7e74a1-3ea7e16d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:41.175831Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|465a85b8-f7c0345b-1e7e74a1-3ea7e16d_0] Write session: try to update token
2025-10-09T21:49:41.175849Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|465a85b8-f7c0345b-1e7e74a1-3ea7e16d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:41.187177Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:41.187202Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:41.187204Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:41.187211Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:41.187221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342505719515001:4673]
2025-10-09T21:49:41.187225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342505719515001:4673]
2025-10-09T21:49:41.187228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342505719515001:4673]
2025-10-09T21:49:41.187276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:41.187288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.187289Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:41.187291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:41.187294Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:41.187299Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:41.187428Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:41.188748Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1594 sessionId: src_id|465a85b8-f7c0345b-1e7e74a1-3ea7e16d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:41.191210Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:41.247479Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342506081342862:2585]
2025-10-09T21:49:41.247724Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 1593, pipe: [1:7559342506081342862:2585]
2025-10-09T21:49:41.337444Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:41.337899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.337912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.337953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342506081342862:2585], now have 1 active actors on pipe
2025-10-09T21:49:41.338219Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:41.338231Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:41.338253Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:41.338302Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1594 reason: error at write: Disconnected sessionId: src_id|465a85b8-f7c0345b-1e7e74a1-3ea7e16d_0
2025-10-09T21:49:41.338308Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342506081342862:2585]
2025-10-09T21:49:41.338320Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:41.338482Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1594 sessionId: src_id|465a85b8-f7c0345b-1e7e74a1-3ea7e16d_0 is DEAD
2025-10-09T21:49:41.338630Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|465a85b8-f7c0345b-1e7e74a1-3ea7e16d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:41.338642Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|465a85b8-f7c0345b-1e7e74a1-3ea7e16d_0] Write session will restart in 0.005413s
2025-10-09T21:49:41.338671Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|465a85b8-f7c0345b-1e7e74a1-3ea7e16d_0] Write session: Do CDS request
2025-10-09T21:49:41.338690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|465a85b8-f7c0345b-1e7e74a1-3ea7e16d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:41.345486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|465a85b8-f7c0345b-1e7e74a1-3ea7e16d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:41.345625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:41.345641Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1595
2025-10-09T21:49:41.345788Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1595 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:41.345823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1595 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:41.345833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1595 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:41.345838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:41.346391Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1595 sessionId:  describe result for acl check
2025-10-09T21:49:41.346428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:41.346434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:41.346446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:41.346453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342506081342911:7842] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:41.346953Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342506081342911:7842] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:41.349612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342506081342911:7842] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:41.351409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:41.354019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:41.354197Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:41.354205Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:41.354215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:41.354255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:41.354263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342506081342911:7842] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:41.354294Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342505719515177:4671]
2025-10-09T21:49:41.354279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342506081342911:7842] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:41.354283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342506081342911:7842] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:41.354292Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1595 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:41.354935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:41.355265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:41.355307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:41.355625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:41.355679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.355685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.355686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.355687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.355689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.355690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.355692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.355693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.355694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.355700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.355710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.355715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.355719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.355723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.355728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.355732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.355736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.355740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.355747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.355757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.355768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.355774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.355779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.355784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.355788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.355793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.355803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.355809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.355819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.355824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.355829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.355835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.355844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.355849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.355855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.355866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.355895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:41.362321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:41.363435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:41.363448Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:41.363451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:41.363456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:41.363462Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1594 [2:7559342505719515177:4671]
2025-10-09T21:49:41.363476Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:41.363496Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:41.363503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:41.363507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:41.363509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:41.363528Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:41.363539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:41.363550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:41.363613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.363620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.363622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.363625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:41.363696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:41.363704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.363716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.363728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.363750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:41.363777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342506081342869:2432], now have 1 active actors on pipe
2025-10-09T21:49:41.363781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342506081343048:7842], now have 1 active actors on pipe
2025-10-09T21:49:41.363789Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342505719515172:4671]
2025-10-09T21:49:41.363792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342505719515172:4671]
2025-10-09T21:49:41.363794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342505719515172:4671]
2025-10-09T21:49:41.363810Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:41.363825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:41.363833Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:41.363836Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:41.363822Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1594
2025-10-09T21:49:41.363838Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:41.363825Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1594
2025-10-09T21:49:41.363847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:41.363851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:41.363887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342505719515172:4671]
2025-10-09T21:49:41.363899Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:41.363904Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:41.363922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.363931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.363980Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|686b2e76-64cd1a05-cf029445-b4178b67_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:41.363991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:41.363995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:41.364005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:41.364015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:41.364026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:41.364047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:41.364187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.364194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.364207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:41.364304Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1595 partition: 0 MaxSeqNo: 25 sessionId: src_id|686b2e76-64cd1a05-cf029445-b4178b67_0
2025-10-09T21:49:41.364662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|465a85b8-f7c0345b-1e7e74a1-3ea7e16d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|686b2e76-64cd1a05-cf029445-b4178b67_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:41.367142Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|686b2e76-64cd1a05-cf029445-b4178b67_0] Write session: try to update token
2025-10-09T21:49:41.367157Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|686b2e76-64cd1a05-cf029445-b4178b67_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:41.378580Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:41.378592Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:41.378593Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:41.378595Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:41.378606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342505719515172:4671]
2025-10-09T21:49:41.378611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342505719515172:4671]
2025-10-09T21:49:41.378615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342505719515172:4671]
2025-10-09T21:49:41.378660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:41.378661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:41.378668Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:41.378670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.378671Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:41.378673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:41.378803Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:41.380329Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1595 sessionId: src_id|686b2e76-64cd1a05-cf029445-b4178b67_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:41.382864Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:41.438423Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342506081343073:2585]
2025-10-09T21:49:41.438616Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 1594, pipe: [1:7559342506081343073:2585]
2025-10-09T21:49:41.523339Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:41.523571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.523585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.523628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342506081343073:2585], now have 1 active actors on pipe
2025-10-09T21:49:41.523832Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:41.523874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:41.523877Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:41.523878Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342506081343073:2585]
2025-10-09T21:49:41.523933Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1595 reason: error at write: Disconnected sessionId: src_id|686b2e76-64cd1a05-cf029445-b4178b67_0
2025-10-09T21:49:41.524098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1595 sessionId: src_id|686b2e76-64cd1a05-cf029445-b4178b67_0 is DEAD
2025-10-09T21:49:41.524265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|686b2e76-64cd1a05-cf029445-b4178b67_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:41.524278Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|686b2e76-64cd1a05-cf029445-b4178b67_0] Write session will restart in 0.009280s
2025-10-09T21:49:41.524302Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|686b2e76-64cd1a05-cf029445-b4178b67_0] Write session: Do CDS request
2025-10-09T21:49:41.524325Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|686b2e76-64cd1a05-cf029445-b4178b67_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:41.524548Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:41.534516Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|686b2e76-64cd1a05-cf029445-b4178b67_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:41.534593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:41.534613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1596
2025-10-09T21:49:41.534763Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1596 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:41.534794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1596 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:41.534806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1596 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:41.534810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:41.535514Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1596 sessionId:  describe result for acl check
2025-10-09T21:49:41.535542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:41.535544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:41.535546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:41.535551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342506081343229:7829] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:41.536056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:41.536068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342506081343229:7829] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:41.538131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342506081343229:7829] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:41.538859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:41.539034Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:41.539045Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:41.539056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:41.539119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:41.539154Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342505719515388:4651]
2025-10-09T21:49:41.539716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:41.539963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:41.540000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:41.540306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:41.540349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.540354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.540355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.540356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.540357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.540358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.540359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.540361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.540362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.540366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.540376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.540380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.540384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.540388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.540396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.540400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.540407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.540411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.540420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.540428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.540438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.540446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.540452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.540462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.540471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.540476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.540481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.540485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.540490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.540499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.540509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.540502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342506081343229:7829] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:41.540514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342506081343229:7829] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:41.540518Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342506081343229:7829] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:41.540518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.540525Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1596 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:41.540526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.540531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.540537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.540543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.540570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:41.546722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:41.547942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:41.547952Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:41.547954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:41.547957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:41.547962Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1595 [2:7559342505719515388:4651]
2025-10-09T21:49:41.547972Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:41.547987Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:41.547989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:41.547991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:41.547992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:41.548005Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:41.548009Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:41.548015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:41.548058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.548063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.548065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.548068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:41.548133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:41.548140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.548153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.548169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.548183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:41.548200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342506081343086:2432], now have 1 active actors on pipe
2025-10-09T21:49:41.548206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342506081343265:7829], now have 1 active actors on pipe
2025-10-09T21:49:41.548211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342505719515382:4651]
2025-10-09T21:49:41.548213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342505719515382:4651]
2025-10-09T21:49:41.548214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342505719515382:4651]
2025-10-09T21:49:41.548223Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:41.548234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:41.548235Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:41.548245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:41.548247Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:41.548251Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:41.548253Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:41.548274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342505719515382:4651]
2025-10-09T21:49:41.548279Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:41.548261Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1595
2025-10-09T21:49:41.548278Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1595
2025-10-09T21:49:41.548299Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:41.548308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.548311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.548334Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c12e56f1-72f3ea6a-cd6a9bca-20e873ea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:41.548344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:41.548351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:41.548352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:41.548360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:41.548371Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:41.548385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:41.548517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.548522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.548533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:41.548621Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1596 partition: 0 MaxSeqNo: 25 sessionId: src_id|c12e56f1-72f3ea6a-cd6a9bca-20e873ea_0
2025-10-09T21:49:41.548946Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|686b2e76-64cd1a05-cf029445-b4178b67_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c12e56f1-72f3ea6a-cd6a9bca-20e873ea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:41.551446Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c12e56f1-72f3ea6a-cd6a9bca-20e873ea_0] Write session: try to update token
2025-10-09T21:49:41.551462Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c12e56f1-72f3ea6a-cd6a9bca-20e873ea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:41.558862Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:41.558874Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:41.558884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342505719515382:4651]
2025-10-09T21:49:41.558923Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:41.558931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:41.559104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:41.559244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:41.559380Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:41.559503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:41.559679Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:41.559716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:41.564540Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1596 sessionId: src_id|c12e56f1-72f3ea6a-cd6a9bca-20e873ea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:41.566934Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:41.567097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.567116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.567164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:41.567247Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:41.571078Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:41.571089Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:41.571091Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:41.571092Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:41.571099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342505719515382:4651]
2025-10-09T21:49:41.571103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342505719515382:4651]
2025-10-09T21:49:41.571105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342505719515382:4651]
2025-10-09T21:49:41.571135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:41.571137Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:41.571140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.571142Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:41.571142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:41.571256Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:41.571261Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:41.623800Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342506081343275:2585]
2025-10-09T21:49:41.624062Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 1595, pipe: [1:7559342506081343275:2585]
2025-10-09T21:49:41.714977Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:41.721449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.721471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.721624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:41.721677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:41.721735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:41.721788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:41.721828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:41.721838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:41.721895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:41.721945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:41.722001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:41.722078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:41.722116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:41.722124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:41.722181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:41.722229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:41.722278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:41.722341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:41.722384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:41.722390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:41.722446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:41.722498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:41.722550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:41.722606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:41.722643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:41.722649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:41.722706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:41.722761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:41.722815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:41.722867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:41.722907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:41.722913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:41.722967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:41.723019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:41.723071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:41.723130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:41.723170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:41.723175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:41.723228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:41.723283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:41.723335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:41.723386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:41.723424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:41.723429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:41.723480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:41.723530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:41.723584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:41.723634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:41.723672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:41.723677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:41.723728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:41.723777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:41.723829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:41.723886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:41.723933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:41.723941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:41.723997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:41.724047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:41.724101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:41.724151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:41.724188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:41.724193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:41.724235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342506081343275:2585], now have 1 active actors on pipe
2025-10-09T21:49:41.724487Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342506081343275:2585]
2025-10-09T21:49:41.724488Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:41.724492Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:41.724527Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1596 reason: error at write: Disconnected sessionId: src_id|c12e56f1-72f3ea6a-cd6a9bca-20e873ea_0
2025-10-09T21:49:41.724659Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:41.724693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1596 sessionId: src_id|c12e56f1-72f3ea6a-cd6a9bca-20e873ea_0 is DEAD
2025-10-09T21:49:41.724851Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c12e56f1-72f3ea6a-cd6a9bca-20e873ea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:41.724863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c12e56f1-72f3ea6a-cd6a9bca-20e873ea_0] Write session will restart in 0.005718s
2025-10-09T21:49:41.724888Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c12e56f1-72f3ea6a-cd6a9bca-20e873ea_0] Write session: Do CDS request
2025-10-09T21:49:41.724894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c12e56f1-72f3ea6a-cd6a9bca-20e873ea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:41.725061Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:41.731446Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c12e56f1-72f3ea6a-cd6a9bca-20e873ea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:41.731542Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:41.731556Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1597
2025-10-09T21:49:41.731679Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1597 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:41.731712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1597 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:41.731723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1597 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:41.731726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:41.732192Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1597 sessionId:  describe result for acl check
2025-10-09T21:49:41.732226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:41.732233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:41.732235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:41.732241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342506081343327:7820] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:41.732780Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342506081343327:7820] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:41.735411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342506081343327:7820] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:41.737099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:41.739752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:41.739952Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:41.739961Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:41.739972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:41.740024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:41.740039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342506081343327:7820] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:41.740053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342506081343327:7820] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:41.740057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342506081343327:7820] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:41.740066Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1597 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:41.740120Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342505719515566:4648]
2025-10-09T21:49:41.740698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:41.740970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:41.741018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:41.741339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:41.741392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.741398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.741399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.741401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.741402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.741403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.741404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.741405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.741407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.741411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.741423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.741427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.741431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.741435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.741444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.741448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.741452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.741456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.741468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.741480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.741490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.741497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.741500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.741503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.741506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.741509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.741511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.741514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.741516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.741520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.741523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.741526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.741535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.741542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.741552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.741567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.741585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:41.747961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:41.749469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:41.749482Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:41.749485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:41.749489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:41.749495Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1596 [2:7559342505719515566:4648]
2025-10-09T21:49:41.749509Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:41.749527Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:41.749535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:41.749538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:41.749541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:41.749572Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:41.749585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:41.749593Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:41.749647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.749654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.749656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.749659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:41.749725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:41.749734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.749744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.749756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.749772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:41.749798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342506081343278:2432], now have 1 active actors on pipe
2025-10-09T21:49:41.749806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342506081343461:7820], now have 1 active actors on pipe
2025-10-09T21:49:41.749813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342505719515560:4648]
2025-10-09T21:49:41.749816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342505719515560:4648]
2025-10-09T21:49:41.749818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342505719515560:4648]
2025-10-09T21:49:41.749831Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:41.749841Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:41.749848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:41.749849Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:41.749851Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:41.749862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:41.749865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:41.749891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342505719515560:4648]
2025-10-09T21:49:41.749875Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1596
2025-10-09T21:49:41.749900Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:41.749874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1596
2025-10-09T21:49:41.749901Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:41.749918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.749922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.749969Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|31c93e64-5f4a6064-bfc1d6a9-986dfd5d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:41.749981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:41.749984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:41.749987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:41.749999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:41.750012Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:41.750055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:41.750199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.750208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.750223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:41.750318Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1597 partition: 0 MaxSeqNo: 25 sessionId: src_id|31c93e64-5f4a6064-bfc1d6a9-986dfd5d_0
2025-10-09T21:49:41.750641Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c12e56f1-72f3ea6a-cd6a9bca-20e873ea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|31c93e64-5f4a6064-bfc1d6a9-986dfd5d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:41.753126Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|31c93e64-5f4a6064-bfc1d6a9-986dfd5d_0] Write session: try to update token
2025-10-09T21:49:41.753141Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|31c93e64-5f4a6064-bfc1d6a9-986dfd5d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:41.765602Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:41.765626Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:41.765629Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:41.765632Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:41.765646Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342505719515560:4648]
2025-10-09T21:49:41.765652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342505719515560:4648]
2025-10-09T21:49:41.765657Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342505719515560:4648]
2025-10-09T21:49:41.765704Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:41.765711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:41.765717Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:41.765721Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:41.765722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.765725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:41.765974Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:41.766956Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1597 sessionId: src_id|31c93e64-5f4a6064-bfc1d6a9-986dfd5d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:41.769301Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:41.824496Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342506081343472:2585]
2025-10-09T21:49:41.824729Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 1596, pipe: [1:7559342506081343472:2585]
2025-10-09T21:49:41.917246Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:41.917681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.917695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.917736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342506081343472:2585], now have 1 active actors on pipe
2025-10-09T21:49:41.917913Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:41.917964Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:41.917964Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342506081343472:2585]
2025-10-09T21:49:41.917987Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:41.917996Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:41.918071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1597 reason: error at write: Disconnected sessionId: src_id|31c93e64-5f4a6064-bfc1d6a9-986dfd5d_0
2025-10-09T21:49:41.918241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1597 sessionId: src_id|31c93e64-5f4a6064-bfc1d6a9-986dfd5d_0 is DEAD
2025-10-09T21:49:41.918373Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31c93e64-5f4a6064-bfc1d6a9-986dfd5d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:41.918381Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31c93e64-5f4a6064-bfc1d6a9-986dfd5d_0] Write session will restart in 0.008125s
2025-10-09T21:49:41.918409Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31c93e64-5f4a6064-bfc1d6a9-986dfd5d_0] Write session: Do CDS request
2025-10-09T21:49:41.918415Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31c93e64-5f4a6064-bfc1d6a9-986dfd5d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:41.927455Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|31c93e64-5f4a6064-bfc1d6a9-986dfd5d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:41.927593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:41.927607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1598
2025-10-09T21:49:41.927722Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1598 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:41.927753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1598 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:41.927764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1598 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:41.927769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:41.928239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1598 sessionId:  describe result for acl check
2025-10-09T21:49:41.928273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:41.928278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:41.928280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:41.928287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342506081343603:7814] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:41.928835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342506081343603:7814] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:41.930774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342506081343603:7814] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:41.933185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:41.933337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342506081343603:7814] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:41.933351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342506081343603:7814] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:41.933354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342506081343603:7814] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:41.933361Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1598 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:41.933583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:41.933750Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:41.933761Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:41.933771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:41.933823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:41.933860Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342505719515741:4638]
2025-10-09T21:49:41.934465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:41.934722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:41.934764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:41.935066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:41.935111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.935117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.935118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.935119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.935120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.935121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.935122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.935124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.935125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.935130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.935140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.935148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.935152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.935156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.935161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.935164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.935167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.935170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.935176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.935183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:41.935194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.935203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:41.935208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.935214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:41.935222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.935228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:41.935236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.935241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:41.935252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.935262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:41.935271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.935280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.935284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.935288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.935292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.935301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.935335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:41.941417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:41.942537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:41.942551Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:41.942554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:41.942559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:41.942566Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1597 [2:7559342505719515741:4638]
2025-10-09T21:49:41.942959Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:41.942983Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:41.942991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:41.942995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:41.942997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:41.943013Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:41.943023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:41.943030Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:41.943090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.943097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.943099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.943102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:41.943170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:41.943179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.943191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:41.943203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:41.943221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:41.943243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342506081343478:2432], now have 1 active actors on pipe
2025-10-09T21:49:41.943251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342506081343660:7814], now have 1 active actors on pipe
2025-10-09T21:49:41.943259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342505719515735:4638]
2025-10-09T21:49:41.943263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342505719515735:4638]
2025-10-09T21:49:41.943265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342505719515735:4638]
2025-10-09T21:49:41.943278Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:41.943292Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:41.943274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1597
2025-10-09T21:49:41.943296Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:41.943301Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:41.943303Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:41.943309Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:41.943312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:41.943351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342505719515735:4638]
2025-10-09T21:49:41.943364Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:41.943347Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1597
2025-10-09T21:49:41.943386Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:41.943438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.943448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.943476Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ee9d0663-f26a25b8-b2e80850-667254ef_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:41.943488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:41.943496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:41.943498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:41.943509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:41.943519Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:41.943532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:41.943662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.943667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.943680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:41.943757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1598 partition: 0 MaxSeqNo: 25 sessionId: src_id|ee9d0663-f26a25b8-b2e80850-667254ef_0
2025-10-09T21:49:41.944131Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31c93e64-5f4a6064-bfc1d6a9-986dfd5d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ee9d0663-f26a25b8-b2e80850-667254ef_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:41.946648Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee9d0663-f26a25b8-b2e80850-667254ef_0] Write session: try to update token
2025-10-09T21:49:41.946665Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee9d0663-f26a25b8-b2e80850-667254ef_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:41.954714Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:41.954730Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:41.954740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342505719515735:4638]
2025-10-09T21:49:41.954785Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:41.954787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:41.954960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:41.955097Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:41.955237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:41.955382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:41.955567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:41.955606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:41.959884Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1598 sessionId: src_id|ee9d0663-f26a25b8-b2e80850-667254ef_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:41.962440Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:41.962616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:41.962645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:41.962683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:41.962768Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:41.968546Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:41.968562Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:41.968565Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:41.968567Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:41.968577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342505719515735:4638]
2025-10-09T21:49:41.968580Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342505719515735:4638]
2025-10-09T21:49:41.968585Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342505719515735:4638]
2025-10-09T21:49:41.968626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:41.968629Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:41.968633Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:41.968634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:41.968636Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:41.968637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:41.968766Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:42.018071Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342510376310967:2585]
2025-10-09T21:49:42.018347Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 1597, pipe: [1:7559342510376310967:2585]
2025-10-09T21:49:42.119841Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:42.127161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.127193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.127370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.127432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.127502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.127575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.127617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.127627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.127697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.127764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.127830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.127895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.127940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.127947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.128007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.128076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.128134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.128189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.128228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.128235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.128296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.128352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.128411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.128481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.128530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.128540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.128615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.128683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.128747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.128808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.128853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.128861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.128924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.128984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.129047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.129109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.129151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.129157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.129219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.129282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.129344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.129399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.129447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.129455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.129515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.129583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.129638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.129694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.129736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.129740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.129794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.129845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.129897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.129961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.130002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.130011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.130091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.130142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.130201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.130260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.130299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.130304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.130359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342510376310967:2585], now have 1 active actors on pipe
2025-10-09T21:49:42.130631Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342510376310967:2585]
2025-10-09T21:49:42.130641Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:42.130646Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:42.130670Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:42.130676Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1598 reason: error at write: Disconnected sessionId: src_id|ee9d0663-f26a25b8-b2e80850-667254ef_0
2025-10-09T21:49:42.130897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1598 sessionId: src_id|ee9d0663-f26a25b8-b2e80850-667254ef_0 is DEAD
2025-10-09T21:49:42.131040Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee9d0663-f26a25b8-b2e80850-667254ef_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:42.131052Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee9d0663-f26a25b8-b2e80850-667254ef_0] Write session will restart in 0.006833s
2025-10-09T21:49:42.131076Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee9d0663-f26a25b8-b2e80850-667254ef_0] Write session: Do CDS request
2025-10-09T21:49:42.131082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee9d0663-f26a25b8-b2e80850-667254ef_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:42.131389Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:42.139627Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee9d0663-f26a25b8-b2e80850-667254ef_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:42.139648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:42.139666Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1599
2025-10-09T21:49:42.139865Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1599 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:42.139909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1599 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:42.139923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1599 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:42.139928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:42.140434Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1599 sessionId:  describe result for acl check
2025-10-09T21:49:42.140469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:42.140475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:42.140477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:42.140483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342510376311052:7809] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:42.141229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342510376311052:7809] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:42.144010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342510376311052:7809] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:42.144502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:42.148013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342510376311052:7809] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:42.148028Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342510376311052:7809] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:42.148032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342510376311052:7809] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:42.148041Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1599 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:42.147639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:42.147856Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:42.147861Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:42.147873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:42.147940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:42.147978Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342510014483212:4621]
2025-10-09T21:49:42.148573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:42.148861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:42.148908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:42.149239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:42.149293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.149300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.149301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.149302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.149303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.149304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.149306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.149307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.149308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.149314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.149325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.149330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.149334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.149338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.149343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.149346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.149350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.149354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.149361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.149369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.149375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.149381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.149385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.149391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.149395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.149400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.149405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.149410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.149413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.149418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.149424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.149429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.149434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.149439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.149451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.149456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.149486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:42.156310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:42.157637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:42.157651Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:42.157654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:42.157659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:42.157668Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1598 [2:7559342510014483212:4621]
2025-10-09T21:49:42.157686Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:42.157710Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:42.157718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:42.157722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:42.157724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:42.157743Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:42.157749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:42.157757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:42.157818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.157825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.157827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.157830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:42.157907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:42.157916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.157928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.157943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.157962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:42.157991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342510376311153:7809], now have 1 active actors on pipe
2025-10-09T21:49:42.157995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342510376310970:2432], now have 1 active actors on pipe
2025-10-09T21:49:42.158003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342510014483207:4621]
2025-10-09T21:49:42.158006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342510014483207:4621]
2025-10-09T21:49:42.158008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342510014483207:4621]
2025-10-09T21:49:42.158025Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:42.158059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:42.158078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:42.158086Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:42.158120Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1598
2025-10-09T21:49:42.158128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342510014483207:4621]
2025-10-09T21:49:42.158131Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1598
2025-10-09T21:49:42.158138Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:42.158159Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:42.158162Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:42.158164Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:42.158171Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:42.158228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.158240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.158272Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ddfcce13-c6bfe5c9-3e81fde7-1a812375_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:42.158285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:42.158291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:42.158299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:42.158311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:42.158328Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:42.158355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:42.158566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.158579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.158603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:42.158727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1599 partition: 0 MaxSeqNo: 25 sessionId: src_id|ddfcce13-c6bfe5c9-3e81fde7-1a812375_0
2025-10-09T21:49:42.159124Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee9d0663-f26a25b8-b2e80850-667254ef_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ddfcce13-c6bfe5c9-3e81fde7-1a812375_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:42.161785Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ddfcce13-c6bfe5c9-3e81fde7-1a812375_0] Write session: try to update token
2025-10-09T21:49:42.161805Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ddfcce13-c6bfe5c9-3e81fde7-1a812375_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:42.172954Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:42.172972Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:42.172975Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:42.172978Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:42.172988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342510014483207:4621]
2025-10-09T21:49:42.172993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342510014483207:4621]
2025-10-09T21:49:42.172997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342510014483207:4621]
2025-10-09T21:49:42.173041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:42.173049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.173052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:42.173052Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:42.173058Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:42.173063Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:42.173216Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:42.175425Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1599 sessionId: src_id|ddfcce13-c6bfe5c9-3e81fde7-1a812375_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:42.177878Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:42.231055Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342510376311168:2585]
2025-10-09T21:49:42.231350Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 1598, pipe: [1:7559342510376311168:2585]
2025-10-09T21:49:42.324416Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:42.325011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.325025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.325064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342510376311168:2585], now have 1 active actors on pipe
2025-10-09T21:49:42.325254Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:42.325289Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342510376311168:2585]
2025-10-09T21:49:42.325289Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:42.325293Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:42.325326Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:42.325371Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1599 reason: error at write: Disconnected sessionId: src_id|ddfcce13-c6bfe5c9-3e81fde7-1a812375_0
2025-10-09T21:49:42.325593Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1599 sessionId: src_id|ddfcce13-c6bfe5c9-3e81fde7-1a812375_0 is DEAD
2025-10-09T21:49:42.325698Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddfcce13-c6bfe5c9-3e81fde7-1a812375_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:42.325708Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddfcce13-c6bfe5c9-3e81fde7-1a812375_0] Write session will restart in 0.007097s
2025-10-09T21:49:42.325731Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddfcce13-c6bfe5c9-3e81fde7-1a812375_0] Write session: Do CDS request
2025-10-09T21:49:42.325736Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddfcce13-c6bfe5c9-3e81fde7-1a812375_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:42.333541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ddfcce13-c6bfe5c9-3e81fde7-1a812375_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:42.333606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:42.333618Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1600
2025-10-09T21:49:42.333787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1600 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:42.333820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1600 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:42.333826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1600 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:42.333830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:42.334373Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1600 sessionId:  describe result for acl check
2025-10-09T21:49:42.334409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:42.334412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:42.334414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:42.334421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342510376311234:7804] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:42.334982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342510376311234:7804] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:42.337322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342510376311234:7804] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:42.339167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:42.342064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:42.342219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342510376311234:7804] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:42.342257Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:42.342236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342510376311234:7804] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:42.342241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342510376311234:7804] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:42.342250Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1600 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:42.342266Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:42.342278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:42.342327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:42.342368Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342510014483388:4625]
2025-10-09T21:49:42.342951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:42.343239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:42.343285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:42.343598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:42.343648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.343654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.343655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.343656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.343658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.343659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.343660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.343661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.343662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.343667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.343678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.343682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.343687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.343690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.343694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.343699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.343703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.343707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.343713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.343722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.343732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.343737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.343746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.343752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.343756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.343762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.343771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.343776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.343786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.343791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.343801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.343806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.343815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.343819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.343824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.343830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.343864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:42.350258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:42.351352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:42.351367Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:42.351369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:42.351372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:42.351378Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1599 [2:7559342510014483388:4625]
2025-10-09T21:49:42.351388Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:42.351405Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:42.351408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:42.351412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:42.351414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:42.351433Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:42.351444Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:42.351453Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:42.351555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.351564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.351566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.351569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:42.351630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:42.351636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.351645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.351651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.351661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:42.351686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342510014483383:4625]
2025-10-09T21:49:42.351693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342510014483383:4625]
2025-10-09T21:49:42.351695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342510014483383:4625]
2025-10-09T21:49:42.351705Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:42.351714Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:42.351720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342510376311359:7804], now have 1 active actors on pipe
2025-10-09T21:49:42.351723Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:42.351726Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:42.351728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342510376311175:2432], now have 1 active actors on pipe
2025-10-09T21:49:42.351734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:42.351743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:42.351745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:42.351775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342510014483383:4625]
2025-10-09T21:49:42.351788Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:42.351793Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:42.351805Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1599
2025-10-09T21:49:42.351808Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1599
2025-10-09T21:49:42.351901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.351912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.351942Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c0d512e6-8c1bb12f-b5ad4107-dfe73f8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:42.351953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:42.351956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:42.351959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:42.351969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:42.351980Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:42.352000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:42.352173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.352181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.352195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:42.352300Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1600 partition: 0 MaxSeqNo: 25 sessionId: src_id|c0d512e6-8c1bb12f-b5ad4107-dfe73f8_0
2025-10-09T21:49:42.352645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddfcce13-c6bfe5c9-3e81fde7-1a812375_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c0d512e6-8c1bb12f-b5ad4107-dfe73f8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:42.355123Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c0d512e6-8c1bb12f-b5ad4107-dfe73f8_0] Write session: try to update token
2025-10-09T21:49:42.355142Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c0d512e6-8c1bb12f-b5ad4107-dfe73f8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:42.366692Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:42.366713Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:42.366716Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:42.366718Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:42.366727Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342510014483383:4625]
2025-10-09T21:49:42.366731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342510014483383:4625]
2025-10-09T21:49:42.366735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342510014483383:4625]
2025-10-09T21:49:42.366781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:42.366789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.366791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:42.366806Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:42.366816Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:42.366821Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:42.366926Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:42.368077Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1600 sessionId: src_id|c0d512e6-8c1bb12f-b5ad4107-dfe73f8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:42.370400Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:42.425635Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342510376311371:2585]
2025-10-09T21:49:42.425959Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 1599, pipe: [1:7559342510376311371:2585]
2025-10-09T21:49:42.528102Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:42.528310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.528324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.528362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342510376311371:2585], now have 1 active actors on pipe
2025-10-09T21:49:42.528527Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:42.528574Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:42.528588Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:42.528598Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342510376311371:2585]
2025-10-09T21:49:42.528620Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:42.528653Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1600 reason: error at write: Disconnected sessionId: src_id|c0d512e6-8c1bb12f-b5ad4107-dfe73f8_0
2025-10-09T21:49:42.528828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1600 sessionId: src_id|c0d512e6-8c1bb12f-b5ad4107-dfe73f8_0 is DEAD
2025-10-09T21:49:42.528981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0d512e6-8c1bb12f-b5ad4107-dfe73f8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:42.528993Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0d512e6-8c1bb12f-b5ad4107-dfe73f8_0] Write session will restart in 0.005379s
2025-10-09T21:49:42.529020Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0d512e6-8c1bb12f-b5ad4107-dfe73f8_0] Write session: Do CDS request
2025-10-09T21:49:42.529027Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0d512e6-8c1bb12f-b5ad4107-dfe73f8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:42.535466Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c0d512e6-8c1bb12f-b5ad4107-dfe73f8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:42.535577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:42.535592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1601
2025-10-09T21:49:42.535714Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1601 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:42.535745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1601 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:42.535755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1601 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:42.535758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:42.536136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1601 sessionId:  describe result for acl check
2025-10-09T21:49:42.536169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:42.536175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:42.536177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:42.536181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342510376311429:7781] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:42.536676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342510376311429:7781] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:42.539019Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342510376311429:7781] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:42.541809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342510376311429:7781] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:42.541833Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342510376311429:7781] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:42.541837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342510376311429:7781] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:42.541845Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1601 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:42.542289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:42.545338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:42.545533Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:42.545543Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:42.545565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:42.545622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:42.545667Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342510014483605:4604]
2025-10-09T21:49:42.546273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:42.546550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:42.546591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:42.546896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:42.546943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.546949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.546950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.546951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.546952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.546953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.546955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.546956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.546957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.546962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.546973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.546978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.546982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.546986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.546990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.546995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.546999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.547003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.547009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.547018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.547024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.547029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.547035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.547040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.547045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.547050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.547055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.547064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.547070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.547079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.547084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.547089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.547098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.547104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.547113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.547118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.547147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:42.553562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:42.555056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:42.555075Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:42.555077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:42.555081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:42.555088Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1600 [2:7559342510014483605:4604]
2025-10-09T21:49:42.555101Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:42.555119Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:42.555122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:42.555125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:42.555127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:42.555143Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:42.555154Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:42.555161Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:42.555216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.555224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.555226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.555229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:42.555298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:42.555305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.555317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.555330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.555352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:42.555376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342510376311580:7781], now have 1 active actors on pipe
2025-10-09T21:49:42.555394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342510014483600:4604]
2025-10-09T21:49:42.555396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342510014483600:4604]
2025-10-09T21:49:42.555398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342510014483600:4604]
2025-10-09T21:49:42.555407Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:42.555417Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:42.555418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342510376311398:2432], now have 1 active actors on pipe
2025-10-09T21:49:42.555421Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:42.555423Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:42.555427Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:42.555441Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:42.555444Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:42.555444Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1600
2025-10-09T21:49:42.555472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342510014483600:4604]
2025-10-09T21:49:42.555489Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:42.555496Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:42.555664Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1600
2025-10-09T21:49:42.555718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.555731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.555763Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|359bfb80-8b4e211f-279d19e4-81f4784d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:42.555778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:42.555790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:42.555793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:42.555804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:42.555816Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:42.555832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:42.555981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.555994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.556014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:42.556115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1601 partition: 0 MaxSeqNo: 25 sessionId: src_id|359bfb80-8b4e211f-279d19e4-81f4784d_0
2025-10-09T21:49:42.556460Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0d512e6-8c1bb12f-b5ad4107-dfe73f8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|359bfb80-8b4e211f-279d19e4-81f4784d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:42.558930Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|359bfb80-8b4e211f-279d19e4-81f4784d_0] Write session: try to update token
2025-10-09T21:49:42.558949Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|359bfb80-8b4e211f-279d19e4-81f4784d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:42.567328Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:42.567344Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:42.567354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342510014483600:4604]
2025-10-09T21:49:42.567404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:42.567615Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:42.567768Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:42.567921Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:42.568063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:42.568268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:42.568311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:42.568389Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:42.572491Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1601 sessionId: src_id|359bfb80-8b4e211f-279d19e4-81f4784d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:42.574782Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:42.574972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.574999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.575040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:42.575139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:42.580673Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:42.580689Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:42.580692Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:42.580695Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:42.580705Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342510014483600:4604]
2025-10-09T21:49:42.580710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342510014483600:4604]
2025-10-09T21:49:42.580714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342510014483600:4604]
2025-10-09T21:49:42.580756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:42.580763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.580766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:42.580771Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:42.580777Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:42.580781Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:42.580909Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:42.628855Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342510376311592:2585]
2025-10-09T21:49:42.629137Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 1600, pipe: [1:7559342510376311592:2585]
2025-10-09T21:49:42.743424Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:42.751759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.751786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.751961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.752027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.752094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.752169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.752217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.752230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.752297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.752362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.752425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.752488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.752532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.752541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.752605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.752664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.752729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.752791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.752835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.752843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.752907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.752967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.753033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.753100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.753147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.753155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.753223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.753288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.753366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.753429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.753492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.753500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.753574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.753642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.753704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.753770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.753816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.753826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.753890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.753951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.754015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.754097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.754141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.754149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.754210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.754267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.754327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.754383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.754427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.754435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.754495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.754554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.754615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.754680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.754724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.754733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.754795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:42.754855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:42.754920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:42.754985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:42.755032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:42.755039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:42.755105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342510376311592:2585], now have 1 active actors on pipe
2025-10-09T21:49:42.755382Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342510376311592:2585]
2025-10-09T21:49:42.755392Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:42.755442Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:49:42.755445Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:42.755448Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:42.755477Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1601 reason: error at write: Disconnected sessionId: src_id|359bfb80-8b4e211f-279d19e4-81f4784d_0
2025-10-09T21:49:42.755554Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1601 sessionId: src_id|359bfb80-8b4e211f-279d19e4-81f4784d_0 is DEAD
2025-10-09T21:49:42.755568Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 16 consumer shared/user session shared/user_1_16_18381273765076421244_v1 is DEAD
2025-10-09T21:49:42.755695Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342428771917403:2586] disconnected.
2025-10-09T21:49:42.755707Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342428771917403:2586] disconnected; active server actors: 1
2025-10-09T21:49:42.755710Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342428771917403:2586] client user disconnected session shared/user_1_16_18381273765076421244_v1
2025-10-09T21:49:42.755833Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:49:42.755841Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|359bfb80-8b4e211f-279d19e4-81f4784d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:42.755865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|359bfb80-8b4e211f-279d19e4-81f4784d_0] Write session will restart in 0.009536s
2025-10-09T21:49:42.755901Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|359bfb80-8b4e211f-279d19e4-81f4784d_0] Write session: Do CDS request
2025-10-09T21:49:42.755908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|359bfb80-8b4e211f-279d19e4-81f4784d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:42.755996Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:49:42.756002Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:49:42.756009Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.006127s
2025-10-09T21:49:42.756196Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:42.763452Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:49:42.763610Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:49:42.763623Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 17
2025-10-09T21:49:42.763789Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 17 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:49:42.763843Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:49:42.763928Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 auth for : user
2025-10-09T21:49:42.764129Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 Handle describe topics response
2025-10-09T21:49:42.764177Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 auth is DEAD
2025-10-09T21:49:42.764181Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:49:42.764476Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:49:42.764571Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_17_11548234668147486070_v1
2025-10-09T21:49:42.764582Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342510376311633:7764] connected; active server actors: 1
2025-10-09T21:49:42.764599Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559342510376311633:7764] session shared/user_1_17_11548234668147486070_v1
2025-10-09T21:49:42.764609Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:49:42.764620Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:49:42.764627Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_17_11548234668147486070_v1" (Sender=[1:7559342510376311616:7764], Pipe=[1:7559342510376311633:7764], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:49:42.764631Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:49:42.764642Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:49:42.764651Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_17_11548234668147486070_v1" (Sender=[1:7559342510376311616:7764], Pipe=[1:7559342510376311633:7764], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:49:42.764662Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_17_11548234668147486070_v1" sender [1:7559342510376311616:7764] lock partition 0 for ReadingSession "shared/user_1_17_11548234668147486070_v1" (Sender=[1:7559342510376311616:7764], Pipe=[1:7559342510376311633:7764], Partitions=[], ActiveFamilyCount=1) generation 1 step 17
2025-10-09T21:49:42.764677Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:49:42.764682Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000037s
2025-10-09T21:49:42.764940Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 17 Session: "shared/user_1_17_11548234668147486070_v1" ClientId: "user" PipeClient { RawX1: 7559342510376311633 RawX2: 4503603922345556 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:49:42.764966Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:49:42.766432Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|359bfb80-8b4e211f-279d19e4-81f4784d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:42.766521Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:42.766533Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1602
2025-10-09T21:49:42.766664Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1602 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:42.766693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1602 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:42.766699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1602 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:42.766702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:42.767103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1602 sessionId:  describe result for acl check
2025-10-09T21:49:42.767140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:42.767147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:42.767149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:42.767155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342510376311703:7772] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:42.767722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342510376311703:7772] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:42.770573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342510376311703:7772] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:42.773680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:42.773954Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342510376311703:7772] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:42.773968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342510376311703:7772] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:42.773972Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342510376311703:7772] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:42.773982Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1602 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:42.774322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:42.774513Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:42.774522Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:42.774533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:42.774581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:42.774631Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342510014483788:4602]
2025-10-09T21:49:42.775288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:42.775579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:42.775621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:42.775930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:42.775979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.775985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.775987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.775988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.775990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.775991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.775992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.775994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.775995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.776000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.776011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.776016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.776020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.776024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.776028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.776032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.776036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.776040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.776047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.776055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.776062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.776067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.776072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.776077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.776082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.776088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.776093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.776098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.776107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.776112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.776122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.776128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.776132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.776138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.776143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.776148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.776178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:42.782678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:42.783814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:42.783831Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:42.783834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:42.783838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:42.783847Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1601 [2:7559342510014483788:4602]
2025-10-09T21:49:42.783860Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:42.783886Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:42.783891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:42.783895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:42.783897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:42.783913Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:42.783924Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:42.783931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:42.783991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.784000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.784002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.784005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:42.784080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:42.784087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.784097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.784110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.784129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:42.784155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342510376311788:7772], now have 1 active actors on pipe
2025-10-09T21:49:42.784167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342510014483782:4602]
2025-10-09T21:49:42.784174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342510014483782:4602]
2025-10-09T21:49:42.784176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342510014483782:4602]
2025-10-09T21:49:42.784190Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:42.784217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342510376311647:7774], now have 1 active actors on pipe
2025-10-09T21:49:42.784226Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1601
2025-10-09T21:49:42.784230Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1601, pipe: [1:7559342510376311647:7774]
2025-10-09T21:49:42.784229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342510376311595:2432], now have 1 active actors on pipe
2025-10-09T21:49:42.784253Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1601
2025-10-09T21:49:42.784234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:42.784249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:42.784250Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:42.784252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:42.784254Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:42.784260Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:42.784287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342510014483782:4602]
2025-10-09T21:49:42.784290Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:42.784313Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:42.784324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.784328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.784346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.784355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.784361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_17_11548234668147486070_v1 on pipe: [1:7559342510376311647:7774]
2025-10-09T21:49:42.784396Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a6f8e155-f4334c77-5be597eb-81df8512_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:42.784407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:42.784408Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_17_11548234668147486070_v1:1 with generation 1601
2025-10-09T21:49:42.784412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:42.784414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:42.784424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:42.784438Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:42.784454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:42.784463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:42.784465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:42.784482Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_17_11548234668147486070_v1
2025-10-09T21:49:42.784489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:42.784518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:42.784539Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:49:42.784691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.784700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.784715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:42.784806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1602 partition: 0 MaxSeqNo: 25 sessionId: src_id|a6f8e155-f4334c77-5be597eb-81df8512_0
2025-10-09T21:49:42.785185Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|359bfb80-8b4e211f-279d19e4-81f4784d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a6f8e155-f4334c77-5be597eb-81df8512_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:42.787748Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6f8e155-f4334c77-5be597eb-81df8512_0] Write session: try to update token
2025-10-09T21:49:42.787764Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6f8e155-f4334c77-5be597eb-81df8512_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:42.789061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:49:42.789091Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:49:42.789101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:42.789105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:42.789107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:42.789115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:49:42.789128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:49:42.789269Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046582784 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:49:42.789289Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:49:42.789331Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 sending to client partition status
2025-10-09T21:49:42.798981Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:42.798997Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:42.798999Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:42.799001Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:42.799011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342510014483782:4602]
2025-10-09T21:49:42.799017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342510014483782:4602]
2025-10-09T21:49:42.799021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342510014483782:4602]
2025-10-09T21:49:42.799064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:42.799066Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:42.799072Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:42.799073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.799075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:42.799076Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:42.799203Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:42.800719Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1602 sessionId: src_id|a6f8e155-f4334c77-5be597eb-81df8512_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:42.802938Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:42.966532Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:42.967102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.967118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.967355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_17_11548234668147486070_v1
2025-10-09T21:49:42.967381Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:42.967404Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_17_11548234668147486070_v1
2025-10-09T21:49:42.967453Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342510376311647:7774]
2025-10-09T21:49:42.967461Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:42.967464Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:42.967480Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:42.967515Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1602 reason: error at write: Disconnected sessionId: src_id|a6f8e155-f4334c77-5be597eb-81df8512_0
2025-10-09T21:49:42.967678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1602 sessionId: src_id|a6f8e155-f4334c77-5be597eb-81df8512_0 is DEAD
2025-10-09T21:49:42.967849Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6f8e155-f4334c77-5be597eb-81df8512_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:42.967860Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6f8e155-f4334c77-5be597eb-81df8512_0] Write session will restart in 0.007554s
2025-10-09T21:49:42.967891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6f8e155-f4334c77-5be597eb-81df8512_0] Write session: Do CDS request
2025-10-09T21:49:42.967897Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6f8e155-f4334c77-5be597eb-81df8512_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:42.976516Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a6f8e155-f4334c77-5be597eb-81df8512_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:42.976594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:42.976609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1603
2025-10-09T21:49:42.976751Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1603 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:42.976801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1603 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:42.976811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1603 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:42.976815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:42.977297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1603 sessionId:  describe result for acl check
2025-10-09T21:49:42.977333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:42.977339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:42.977340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:42.977346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342510376311914:7760] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:42.977971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342510376311914:7760] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:42.980033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342510376311914:7760] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:42.983323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:42.983482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342510376311914:7760] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:42.983495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342510376311914:7760] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:42.983500Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342510376311914:7760] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:42.983508Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1603 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:42.983819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:42.983980Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:42.983987Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:42.984003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:42.984052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:42.984082Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342510014483969:4600]
2025-10-09T21:49:42.984489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:42.984779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:42.984819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:42.985109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:42.985160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.985166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.985167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.985169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.985170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.985171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.985173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.985174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.985175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.985180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.985190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.985194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.985198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.985202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.985206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.985210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.985213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.985217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.985224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.985231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:42.985238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.985243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:42.985252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.985256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:42.985260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.985265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:42.985275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.985280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:42.985289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.985295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:42.985304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.985315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.985323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.985328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.985337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.985342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.985371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:42.991754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:42.992882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:42.992896Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:42.992898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:42.992902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:42.992909Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1602 [2:7559342510014483969:4600]
2025-10-09T21:49:42.992923Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:42.992947Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:42.992951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:42.992954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:42.992957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:42.992974Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:42.992989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:42.992996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:42.993058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.993067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.993069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.993072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:42.993143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:42.993153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:42.993164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:42.993176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:42.993192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:42.993215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342510376311807:2432], now have 1 active actors on pipe
2025-10-09T21:49:42.993223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342510376311991:7760], now have 1 active actors on pipe
2025-10-09T21:49:42.993232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342510014483963:4600]
2025-10-09T21:49:42.993238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342510014483963:4600]
2025-10-09T21:49:42.993241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342510014483963:4600]
2025-10-09T21:49:42.993255Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:42.993272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:42.993272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:42.993286Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:42.993288Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:42.993291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:42.993294Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:42.993324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342510014483963:4600]
2025-10-09T21:49:42.993332Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:42.993298Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1602
2025-10-09T21:49:42.993333Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:42.993315Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1602
2025-10-09T21:49:42.993358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.993362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.993399Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cc50472d-a0642da4-63934261-b58bcc63_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:42.993411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:42.993415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:42.993422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:42.993433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:42.993446Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:42.993468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:42.993605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:42.993616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:42.993629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:42.993750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1603 partition: 0 MaxSeqNo: 25 sessionId: src_id|cc50472d-a0642da4-63934261-b58bcc63_0
2025-10-09T21:49:42.994150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a6f8e155-f4334c77-5be597eb-81df8512_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cc50472d-a0642da4-63934261-b58bcc63_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:42.996658Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc50472d-a0642da4-63934261-b58bcc63_0] Write session: try to update token
2025-10-09T21:49:42.996675Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc50472d-a0642da4-63934261-b58bcc63_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:43.007498Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:43.007518Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:43.007521Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:43.007523Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:43.007534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342510014483963:4600]
2025-10-09T21:49:43.007539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342510014483963:4600]
2025-10-09T21:49:43.007544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342510014483963:4600]
2025-10-09T21:49:43.007595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:43.007598Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:43.007605Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:43.007608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.007609Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:43.007611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:43.007754Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:43.010103Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1603 sessionId: src_id|cc50472d-a0642da4-63934261-b58bcc63_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:43.012663Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:43.067627Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342514671279299:7774]
2025-10-09T21:49:43.067931Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 1602, pipe: [1:7559342514671279299:7774]
2025-10-09T21:49:43.174351Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:43.175121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:43.175138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:43.175185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342514671279299:7774], now have 1 active actors on pipe
2025-10-09T21:49:43.175377Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:43.175485Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:43.175497Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342514671279299:7774]
2025-10-09T21:49:43.175500Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:43.175519Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:43.175577Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1603 reason: error at write: Disconnected sessionId: src_id|cc50472d-a0642da4-63934261-b58bcc63_0
2025-10-09T21:49:43.175772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1603 sessionId: src_id|cc50472d-a0642da4-63934261-b58bcc63_0 is DEAD
2025-10-09T21:49:43.175936Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc50472d-a0642da4-63934261-b58bcc63_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:43.175962Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc50472d-a0642da4-63934261-b58bcc63_0] Write session will restart in 0.007892s
2025-10-09T21:49:43.175990Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc50472d-a0642da4-63934261-b58bcc63_0] Write session: Do CDS request
2025-10-09T21:49:43.175997Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc50472d-a0642da4-63934261-b58bcc63_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:43.185582Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc50472d-a0642da4-63934261-b58bcc63_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:43.185692Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:43.185709Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1604
2025-10-09T21:49:43.185833Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1604 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:43.185881Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1604 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:43.185893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1604 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:43.185897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:43.186417Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1604 sessionId:  describe result for acl check
2025-10-09T21:49:43.186456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:43.186466Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:43.186468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:43.186474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342514671279407:7755] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:43.187069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342514671279407:7755] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:43.189443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342514671279407:7755] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:43.192826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:43.193099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342514671279407:7755] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:43.193113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342514671279407:7755] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:43.193117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342514671279407:7755] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:43.193127Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1604 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:43.193243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:43.193428Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:43.193439Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:43.193450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:43.193491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:43.193524Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342514309451445:4588]
2025-10-09T21:49:43.193968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:43.194277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:43.194313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:43.194629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:43.194677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.194683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.194684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.194685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.194687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.194688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.194689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.194690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.194692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.194697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.194708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.194713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.194717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.194721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.194726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.194730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.194734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.194738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.194745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.194753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.194760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.194765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.194770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.194775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.194780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.194792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.194797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.194806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.194811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.194820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.194826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.194836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.194846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.194856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.194862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.194867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.194904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:43.201322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:43.202540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:43.202559Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:43.202563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:43.202567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:43.202575Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1603 [2:7559342514309451445:4588]
2025-10-09T21:49:43.202590Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:43.202611Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:43.202614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:43.202618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:43.202621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:43.202641Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:43.202647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:43.202660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:43.202721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.202726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.202728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.202731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:43.202809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:43.202821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.202833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.202844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.202863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:43.202891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342514671279306:2432], now have 1 active actors on pipe
2025-10-09T21:49:43.202900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342514671279494:7755], now have 1 active actors on pipe
2025-10-09T21:49:43.202908Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342514309451439:4588]
2025-10-09T21:49:43.202915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342514309451439:4588]
2025-10-09T21:49:43.202918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342514309451439:4588]
2025-10-09T21:49:43.202933Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:43.202952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:43.202954Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:43.202958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:43.202961Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:43.202968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:43.202972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:43.202955Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1603
2025-10-09T21:49:43.202974Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1603
2025-10-09T21:49:43.203011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342514309451439:4588]
2025-10-09T21:49:43.203019Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:43.203030Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:43.203041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:43.203050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:43.203104Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|629f1a58-eb56a7de-64784ad7-d1b339ab_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:43.203569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:43.203577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:43.203579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:43.203587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:43.203596Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:43.203621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:43.203895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:43.203904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:43.203922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:43.204020Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1604 partition: 0 MaxSeqNo: 25 sessionId: src_id|629f1a58-eb56a7de-64784ad7-d1b339ab_0
2025-10-09T21:49:43.204379Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc50472d-a0642da4-63934261-b58bcc63_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|629f1a58-eb56a7de-64784ad7-d1b339ab_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:43.206879Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|629f1a58-eb56a7de-64784ad7-d1b339ab_0] Write session: try to update token
2025-10-09T21:49:43.206900Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|629f1a58-eb56a7de-64784ad7-d1b339ab_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:43.217312Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:43.217330Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:43.217333Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:43.217335Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:43.217345Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342514309451439:4588]
2025-10-09T21:49:43.217350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342514309451439:4588]
2025-10-09T21:49:43.217354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342514309451439:4588]
2025-10-09T21:49:43.217396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:43.217396Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:43.217403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.217405Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:43.217406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:43.217409Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:43.217563Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:43.219238Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1604 sessionId: src_id|629f1a58-eb56a7de-64784ad7-d1b339ab_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:43.221468Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:43.275533Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342514671279511:7774]
2025-10-09T21:49:43.275729Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 1603, pipe: [1:7559342514671279511:7774]
2025-10-09T21:49:43.366232Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:43.366699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:43.366711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:43.366750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342514671279511:7774], now have 1 active actors on pipe
2025-10-09T21:49:43.366949Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:43.366953Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:43.366962Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:43.367013Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342514671279511:7774]
2025-10-09T21:49:43.367064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1604 reason: error at write: Disconnected sessionId: src_id|629f1a58-eb56a7de-64784ad7-d1b339ab_0
2025-10-09T21:49:43.367094Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:43.367247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1604 sessionId: src_id|629f1a58-eb56a7de-64784ad7-d1b339ab_0 is DEAD
2025-10-09T21:49:43.367421Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|629f1a58-eb56a7de-64784ad7-d1b339ab_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:43.367431Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|629f1a58-eb56a7de-64784ad7-d1b339ab_0] Write session will restart in 0.007645s
2025-10-09T21:49:43.367464Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|629f1a58-eb56a7de-64784ad7-d1b339ab_0] Write session: Do CDS request
2025-10-09T21:49:43.367470Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|629f1a58-eb56a7de-64784ad7-d1b339ab_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:43.376486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|629f1a58-eb56a7de-64784ad7-d1b339ab_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:43.376579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:43.376594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1605
2025-10-09T21:49:43.376713Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1605 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:43.376745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1605 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:43.376756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1605 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:43.376768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:43.377219Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1605 sessionId:  describe result for acl check
2025-10-09T21:49:43.377256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:43.377262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:43.377264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:43.377270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342514671279636:7754] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:43.377825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342514671279636:7754] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:43.379641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342514671279636:7754] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:43.379997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:43.382577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:43.382767Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:43.382776Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:43.382788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:43.382842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:43.382885Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342514309451625:4574]
2025-10-09T21:49:43.382918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342514671279636:7754] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:43.382930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342514671279636:7754] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:43.382934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342514671279636:7754] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:43.382942Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1605 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:43.383470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:43.383754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:43.383796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:43.384095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:43.384146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.384152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.384153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.384154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.384156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.384157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.384159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.384160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.384161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.384166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.384176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.384180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.384185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.384189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.384193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.384196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.384200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.384203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.384210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.384220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.384227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.384232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.384236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.384241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.384251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.384256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.384261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.384270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.384275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.384280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.384290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.384295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.384304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.384309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.384318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.384323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.384353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:43.390591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:43.391730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:43.391748Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:43.391751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:43.391756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:43.391762Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1604 [2:7559342514309451625:4574]
2025-10-09T21:49:43.391778Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:43.391797Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:43.391806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:43.391809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:43.391812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:43.391828Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:43.391839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:43.391846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:43.391901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.391909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.391911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.391914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:43.391984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:43.391992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.392003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.392016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.392037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:43.392062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342514671279513:2432], now have 1 active actors on pipe
2025-10-09T21:49:43.392069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342514671279702:7754], now have 1 active actors on pipe
2025-10-09T21:49:43.392078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342514309451620:4574]
2025-10-09T21:49:43.392080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342514309451620:4574]
2025-10-09T21:49:43.392082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342514309451620:4574]
2025-10-09T21:49:43.392094Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:43.392112Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:43.392114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:43.392120Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:43.392122Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:43.392127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:43.392131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:43.392112Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1604
2025-10-09T21:49:43.392113Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1604
2025-10-09T21:49:43.392160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342514309451620:4574]
2025-10-09T21:49:43.392173Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:43.392177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:43.392194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:43.392204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:43.392685Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cc04e051-20bf72c8-eba9376c-62715452_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:43.392701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:43.392705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:43.392708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:43.392718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:43.392732Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:43.392762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:43.392928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:43.392933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:43.392947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:43.393029Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1605 partition: 0 MaxSeqNo: 25 sessionId: src_id|cc04e051-20bf72c8-eba9376c-62715452_0
2025-10-09T21:49:43.393376Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|629f1a58-eb56a7de-64784ad7-d1b339ab_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cc04e051-20bf72c8-eba9376c-62715452_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:43.395749Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc04e051-20bf72c8-eba9376c-62715452_0] Write session: try to update token
2025-10-09T21:49:43.395763Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc04e051-20bf72c8-eba9376c-62715452_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:43.406270Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:43.406287Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:43.406289Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:43.406292Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:43.406302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342514309451620:4574]
2025-10-09T21:49:43.406307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342514309451620:4574]
2025-10-09T21:49:43.406311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342514309451620:4574]
2025-10-09T21:49:43.406355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:43.406355Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:43.406360Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:43.406362Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:43.406367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.406369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:43.406523Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:43.407932Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1605 sessionId: src_id|cc04e051-20bf72c8-eba9376c-62715452_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:43.410316Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:43.467017Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342514671279714:7774]
2025-10-09T21:49:43.467290Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 1604, pipe: [1:7559342514671279714:7774]
2025-10-09T21:49:43.562650Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:43.562850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:43.562863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:43.562896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342514671279714:7774], now have 1 active actors on pipe
2025-10-09T21:49:43.563040Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:43.563095Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:43.563107Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342514671279714:7774]
2025-10-09T21:49:43.563128Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:43.563205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1605 reason: status is not ok: tablet will be restarted right now sessionId: src_id|cc04e051-20bf72c8-eba9376c-62715452_0
2025-10-09T21:49:43.563387Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1605 sessionId: src_id|cc04e051-20bf72c8-eba9376c-62715452_0 is DEAD
2025-10-09T21:49:43.563540Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc04e051-20bf72c8-eba9376c-62715452_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:43.563551Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc04e051-20bf72c8-eba9376c-62715452_0] Write session will restart in 0.006687s
2025-10-09T21:49:43.563578Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc04e051-20bf72c8-eba9376c-62715452_0] Write session: Do CDS request
2025-10-09T21:49:43.563584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc04e051-20bf72c8-eba9376c-62715452_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:43.571495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc04e051-20bf72c8-eba9376c-62715452_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:43.571587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:43.571597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1606
2025-10-09T21:49:43.571692Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1606 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:43.571731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1606 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:43.571741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1606 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:43.571745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:43.572212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1606 sessionId:  describe result for acl check
2025-10-09T21:49:43.572247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:43.572253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:43.572255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:43.572261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342514671279816:7731] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:43.572806Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342514671279816:7731] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:43.575035Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342514671279816:7731] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:43.576558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:43.579188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:43.579355Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:43.579366Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:43.579377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:43.579424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:43.579463Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342514309451846:4569]
2025-10-09T21:49:43.579547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342514671279816:7731] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:43.579561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342514671279816:7731] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:43.579564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342514671279816:7731] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:43.579572Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1606 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:43.580056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:43.580317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:43.580361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:43.580673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:43.580722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.580728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.580729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.580730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.580731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.580733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.580734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.580735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.580736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.580741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.580751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.580755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.580758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.580762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.580767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.580771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.580775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.580779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.580784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.580792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.580798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.580803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.580812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.580817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.580827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.580831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.580836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.580846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.580850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.580856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.580861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.580866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.580870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.580875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.580881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.580886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.580917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:43.587240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:43.588734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:43.588746Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:43.588747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:43.588750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:43.588754Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1605 [2:7559342514309451846:4569]
2025-10-09T21:49:43.588765Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:43.588779Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:43.588780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:43.588783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:43.588784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:43.588797Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:43.588801Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:43.588807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:43.588849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.588859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.588860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.588862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:43.588915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:43.588921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.588932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.588948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.588960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:43.588981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342514309451841:4569]
2025-10-09T21:49:43.588987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342514309451841:4569]
2025-10-09T21:49:43.588989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342514309451841:4569]
2025-10-09T21:49:43.589004Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:43.589013Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:43.589017Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:43.589019Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:43.589020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342514671279742:2432], now have 1 active actors on pipe
2025-10-09T21:49:43.589023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342514671279929:7731], now have 1 active actors on pipe
2025-10-09T21:49:43.589029Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:43.589037Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:43.589039Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:43.589059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342514309451841:4569]
2025-10-09T21:49:43.589066Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:43.589073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:43.589053Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1605
2025-10-09T21:49:43.589067Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1605
2025-10-09T21:49:43.589097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:43.589103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:43.589123Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|659fac28-549a109f-87840b1a-a6d66980_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:43.589132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:43.589139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:43.589140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:43.589147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:43.589158Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:43.589168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:43.589388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:43.589397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:43.589408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:43.589482Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1606 partition: 0 MaxSeqNo: 25 sessionId: src_id|659fac28-549a109f-87840b1a-a6d66980_0
2025-10-09T21:49:43.589831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc04e051-20bf72c8-eba9376c-62715452_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|659fac28-549a109f-87840b1a-a6d66980_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:43.592211Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|659fac28-549a109f-87840b1a-a6d66980_0] Write session: try to update token
2025-10-09T21:49:43.592226Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|659fac28-549a109f-87840b1a-a6d66980_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:43.604738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1606 sessionId: src_id|659fac28-549a109f-87840b1a-a6d66980_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:43.605473Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:43.605489Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:43.605499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342514309451841:4569]
2025-10-09T21:49:43.605546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:43.605610Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:43.605722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:43.605858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:43.605990Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:43.606133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:43.606315Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:43.606354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:43.607074Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:43.607230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:43.607246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:43.607280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:43.607341Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:43.612528Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:43.612541Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:43.612544Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:43.612546Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:43.612554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342514309451841:4569]
2025-10-09T21:49:43.612558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342514309451841:4569]
2025-10-09T21:49:43.612561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342514309451841:4569]
2025-10-09T21:49:43.612597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:43.612604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.612607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:43.612612Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:43.612619Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:43.612623Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:43.612736Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:43.663113Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342514671279940:7774]
2025-10-09T21:49:43.663370Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 1605, pipe: [1:7559342514671279940:7774]
2025-10-09T21:49:43.754555Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:43.761329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:43.761350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:43.761492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:43.761561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:43.761621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:43.761680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:43.761719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:43.761727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:43.761786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:43.761844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:43.761902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:43.761959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:43.761997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:43.762003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:43.762071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:43.762122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:43.762178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:43.762233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:43.762271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:43.762278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:43.762344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:43.762405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:43.762461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:43.762517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:43.762557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:43.762562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:43.762621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:43.762685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:43.762748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:43.762810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:43.762849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:43.762854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:43.762909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:43.762969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:43.763032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:43.763088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:43.763125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:43.763130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:43.763185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:43.763236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:43.763289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:43.763346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:43.763383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:43.763391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:43.763451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:43.763504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:43.763558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:43.763613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:43.763652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:43.763660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:43.763724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:43.763786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:43.763846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:43.763901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:43.763938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:43.763943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:43.763995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:43.764048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:43.764100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:43.764155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:43.764192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:43.764197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:43.764237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342514671279940:7774], now have 1 active actors on pipe
2025-10-09T21:49:43.764431Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:43.764471Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342514671279940:7774]
2025-10-09T21:49:43.764474Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:43.764476Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:43.764505Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1606 reason: error at write: Disconnected sessionId: src_id|659fac28-549a109f-87840b1a-a6d66980_0
2025-10-09T21:49:43.764660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1606 sessionId: src_id|659fac28-549a109f-87840b1a-a6d66980_0 is DEAD
2025-10-09T21:49:43.764834Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|659fac28-549a109f-87840b1a-a6d66980_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:43.764844Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|659fac28-549a109f-87840b1a-a6d66980_0] Write session will restart in 0.007587s
2025-10-09T21:49:43.764864Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|659fac28-549a109f-87840b1a-a6d66980_0] Write session: Do CDS request
2025-10-09T21:49:43.764869Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|659fac28-549a109f-87840b1a-a6d66980_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:43.764984Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:43.773482Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|659fac28-549a109f-87840b1a-a6d66980_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:43.773534Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:43.773569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1607
2025-10-09T21:49:43.773708Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1607 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:43.773745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1607 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:43.773762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1607 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:43.773771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:43.774104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1607 sessionId:  describe result for acl check
2025-10-09T21:49:43.774140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:43.774145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:43.774147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:43.774152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342514671280074:7730] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:43.774625Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342514671280074:7730] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:43.776441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:43.776630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342514671280074:7730] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:43.778954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:43.779091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342514671280074:7730] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:43.779100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342514671280074:7730] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:43.779103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342514671280074:7730] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:43.779126Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:43.779109Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1607 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:43.779135Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:43.779146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:43.779197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:43.779232Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342514309452033:4552]
2025-10-09T21:49:43.779774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:43.780017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:43.780056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:43.780350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:43.780394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.780399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.780401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.780402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.780403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.780404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.780405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.780406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.780408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.780412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.780423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.780426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.780430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.780434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.780438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.780441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.780445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.780449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.780455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.780462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.780468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.780473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.780477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.780482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.780486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.780496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.780501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.780510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.780514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.780519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.780527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.780532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.780537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.780546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.780551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.780556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.780588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:43.786605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:43.787717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:43.787733Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:43.787735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:43.787739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:43.787745Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1606 [2:7559342514309452033:4552]
2025-10-09T21:49:43.787759Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:43.787776Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:43.787779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:43.787782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:43.787785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:43.787799Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:43.787810Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:43.787818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:43.787874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.787882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.787884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.787886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:43.787955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:43.787965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.787976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.787990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.788009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:43.788035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342514671279943:2432], now have 1 active actors on pipe
2025-10-09T21:49:43.788044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342514671280134:7730], now have 1 active actors on pipe
2025-10-09T21:49:43.788051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342514309452028:4552]
2025-10-09T21:49:43.788053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342514309452028:4552]
2025-10-09T21:49:43.788056Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342514309452028:4552]
2025-10-09T21:49:43.788068Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:43.788076Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:43.788079Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:43.788080Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:43.788089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:43.788104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:43.788111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:43.788108Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1606
2025-10-09T21:49:43.788145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342514309452028:4552]
2025-10-09T21:49:43.788157Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:43.788161Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:43.788156Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1606
2025-10-09T21:49:43.788208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:43.788217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:43.788245Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ece8178d-c51e7ba1-8912d331-b94781f0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:43.788257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:43.788264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:43.788267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:43.788277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:43.788291Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:43.788310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:43.788485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:43.788496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:43.788510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:43.788620Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1607 partition: 0 MaxSeqNo: 25 sessionId: src_id|ece8178d-c51e7ba1-8912d331-b94781f0_0
2025-10-09T21:49:43.788968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|659fac28-549a109f-87840b1a-a6d66980_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ece8178d-c51e7ba1-8912d331-b94781f0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:43.791322Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ece8178d-c51e7ba1-8912d331-b94781f0_0] Write session: try to update token
2025-10-09T21:49:43.791337Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ece8178d-c51e7ba1-8912d331-b94781f0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:43.802508Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:43.802527Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:43.802530Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:43.802532Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:43.802542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342514309452028:4552]
2025-10-09T21:49:43.802547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342514309452028:4552]
2025-10-09T21:49:43.802551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342514309452028:4552]
2025-10-09T21:49:43.802595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:43.802598Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:43.802604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.802604Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:43.802606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:43.802607Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:43.802748Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:43.804217Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1607 sessionId: src_id|ece8178d-c51e7ba1-8912d331-b94781f0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:43.806489Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:43.864829Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342514671280147:7774]
2025-10-09T21:49:43.865105Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 1606, pipe: [1:7559342514671280147:7774]
2025-10-09T21:49:43.948343Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:43.948811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:43.948829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:43.948874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342514671280147:7774], now have 1 active actors on pipe
2025-10-09T21:49:43.949039Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:43.949060Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:43.949080Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342514671280147:7774]
2025-10-09T21:49:43.949099Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:43.949166Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1607 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ece8178d-c51e7ba1-8912d331-b94781f0_0
2025-10-09T21:49:43.949324Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1607 sessionId: src_id|ece8178d-c51e7ba1-8912d331-b94781f0_0 is DEAD
2025-10-09T21:49:43.949483Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ece8178d-c51e7ba1-8912d331-b94781f0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:43.949495Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ece8178d-c51e7ba1-8912d331-b94781f0_0] Write session will restart in 0.006559s
2025-10-09T21:49:43.949524Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ece8178d-c51e7ba1-8912d331-b94781f0_0] Write session: Do CDS request
2025-10-09T21:49:43.949530Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ece8178d-c51e7ba1-8912d331-b94781f0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:43.957476Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ece8178d-c51e7ba1-8912d331-b94781f0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:43.957594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:43.957608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1608
2025-10-09T21:49:43.957715Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1608 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:43.957753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1608 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:43.957763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1608 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:43.957768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:43.958278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1608 sessionId:  describe result for acl check
2025-10-09T21:49:43.958309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:43.958311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:43.958312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:43.958316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342514671280210:7723] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:43.958785Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342514671280210:7723] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:43.961318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342514671280210:7723] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:43.965470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:43.965495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342514671280210:7723] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:43.965505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342514671280210:7723] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:43.965507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342514671280210:7723] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:43.965511Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1608 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:43.965847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:43.965998Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:43.966007Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:43.966017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:43.966075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:43.966114Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342514309452218:4547]
2025-10-09T21:49:43.966675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:43.966934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:43.966971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:43.967254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:43.967299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.967305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.967306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.967307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.967308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.967309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.967310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.967311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.967312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.967316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.967326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.967330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.967334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.967338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.967342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.967345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.967349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.967352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.967358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.967366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:43.967372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.967377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:43.967381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.967390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:43.967395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.967401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:43.967406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.967413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:43.967417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.967421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:43.967426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.967431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.967439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.967444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.967450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.967457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.967487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:43.973320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:43.974707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:43.974719Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:43.974721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:43.974725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:43.974730Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1607 [2:7559342514309452218:4547]
2025-10-09T21:49:43.974742Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:43.974758Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:43.974766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:43.974769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:43.974770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:43.974786Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:43.974795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:43.974802Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:43.974855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.974862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.974864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.974866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:43.974927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:43.974932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.974943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:43.974955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:43.974975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:43.974999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342514671280150:2432], now have 1 active actors on pipe
2025-10-09T21:49:43.975006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342514671280341:7723], now have 1 active actors on pipe
2025-10-09T21:49:43.975013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342514309452212:4547]
2025-10-09T21:49:43.975016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342514309452212:4547]
2025-10-09T21:49:43.975018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342514309452212:4547]
2025-10-09T21:49:43.975032Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:43.975033Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1607
2025-10-09T21:49:43.975049Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:43.975036Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1607
2025-10-09T21:49:43.975066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:43.975069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:43.975092Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:43.975101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:43.975104Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:43.975108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:43.975113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:43.975126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342514309452212:4547]
2025-10-09T21:49:43.975137Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:43.975143Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:43.975166Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|58e1b234-9e8738a9-bc96c8a9-6f8d426_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:43.975176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:43.975182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:43.975187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:43.975197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:43.975207Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:43.975227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:43.975359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:43.975368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:43.975382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:43.975467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1608 partition: 0 MaxSeqNo: 25 sessionId: src_id|58e1b234-9e8738a9-bc96c8a9-6f8d426_0
2025-10-09T21:49:43.975757Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ece8178d-c51e7ba1-8912d331-b94781f0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|58e1b234-9e8738a9-bc96c8a9-6f8d426_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:43.978095Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|58e1b234-9e8738a9-bc96c8a9-6f8d426_0] Write session: try to update token
2025-10-09T21:49:43.978113Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|58e1b234-9e8738a9-bc96c8a9-6f8d426_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:43.989101Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:43.989120Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:43.989122Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:43.989124Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:43.989134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342514309452212:4547]
2025-10-09T21:49:43.989140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342514309452212:4547]
2025-10-09T21:49:43.989143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342514309452212:4547]
2025-10-09T21:49:43.989187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:43.989194Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:43.989198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:43.989199Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:43.989201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:43.989202Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:43.989354Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:43.990818Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1608 sessionId: src_id|58e1b234-9e8738a9-bc96c8a9-6f8d426_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:43.992958Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:44.049134Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342518966247649:7774]
2025-10-09T21:49:44.049421Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 1607, pipe: [1:7559342518966247649:7774]
2025-10-09T21:49:44.139089Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:44.139540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.139555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.139592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342518966247649:7774], now have 1 active actors on pipe
2025-10-09T21:49:44.139740Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:44.139820Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:44.139836Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342518966247649:7774]
2025-10-09T21:49:44.139866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:44.139930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1608 reason: status is not ok: tablet will be restarted right now sessionId: src_id|58e1b234-9e8738a9-bc96c8a9-6f8d426_0
2025-10-09T21:49:44.140095Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1608 sessionId: src_id|58e1b234-9e8738a9-bc96c8a9-6f8d426_0 is DEAD
2025-10-09T21:49:44.140248Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|58e1b234-9e8738a9-bc96c8a9-6f8d426_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:44.140258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|58e1b234-9e8738a9-bc96c8a9-6f8d426_0] Write session will restart in 0.006243s
2025-10-09T21:49:44.140277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|58e1b234-9e8738a9-bc96c8a9-6f8d426_0] Write session: Do CDS request
2025-10-09T21:49:44.140283Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|58e1b234-9e8738a9-bc96c8a9-6f8d426_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:44.147508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|58e1b234-9e8738a9-bc96c8a9-6f8d426_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:44.147639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:44.147652Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1609
2025-10-09T21:49:44.147785Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1609 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:44.147817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1609 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:44.147829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1609 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:44.147832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:44.148283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1609 sessionId:  describe result for acl check
2025-10-09T21:49:44.148321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:44.148328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:44.148330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:44.148336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342518966247728:7709] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:44.148884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342518966247728:7709] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:44.151137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342518966247728:7709] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:44.155279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:44.155432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342518966247728:7709] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:44.155445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342518966247728:7709] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:44.155449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342518966247728:7709] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:44.155457Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1609 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:44.155721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:44.155885Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:44.155894Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:44.155905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:44.155949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:44.155987Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342518604419699:4543]
2025-10-09T21:49:44.156583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:44.156845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:44.156886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:44.157192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:44.157240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.157247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.157248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.157249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.157250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.157251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.157253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.157254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.157255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.157260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.157270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.157274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.157277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.157281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.157284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.157287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.157290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.157293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.157300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.157307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.157313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.157318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.157323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.157327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.157331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.157336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.157341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.157346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.157351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.157359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.157363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.157368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.157373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.157380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.157384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.157390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.157419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:44.163830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:44.165070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:44.165084Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:44.165086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:44.165091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:44.165098Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1608 [2:7559342518604419699:4543]
2025-10-09T21:49:44.165112Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:44.165130Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:44.165138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:44.165141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:44.165144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:44.165160Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:44.165171Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:44.165179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:44.165242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.165249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.165251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.165254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:44.165321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:44.165328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.165340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.165353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.165372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:44.165398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342518966247846:7709], now have 1 active actors on pipe
2025-10-09T21:49:44.165406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342518966247653:2432], now have 1 active actors on pipe
2025-10-09T21:49:44.165413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342518604419693:4543]
2025-10-09T21:49:44.165415Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342518604419693:4543]
2025-10-09T21:49:44.165417Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342518604419693:4543]
2025-10-09T21:49:44.165430Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:44.165449Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:44.165448Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:44.165451Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:44.165454Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:44.165465Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:44.165469Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:44.165458Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1608
2025-10-09T21:49:44.165499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342518604419693:4543]
2025-10-09T21:49:44.165509Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:44.165463Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1608
2025-10-09T21:49:44.165515Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:44.165526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.165535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.165599Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f71b9977-3be49df2-cfd9cf7f-aa1495f4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:44.165613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:44.165617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:44.165620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:44.165629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:44.165640Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:44.165661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:44.165817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.165827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.165842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:44.165936Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1609 partition: 0 MaxSeqNo: 25 sessionId: src_id|f71b9977-3be49df2-cfd9cf7f-aa1495f4_0
2025-10-09T21:49:44.166281Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|58e1b234-9e8738a9-bc96c8a9-6f8d426_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f71b9977-3be49df2-cfd9cf7f-aa1495f4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:44.168627Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f71b9977-3be49df2-cfd9cf7f-aa1495f4_0] Write session: try to update token
2025-10-09T21:49:44.168641Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f71b9977-3be49df2-cfd9cf7f-aa1495f4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:44.179818Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:44.179836Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:44.179845Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:44.179847Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:44.179857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342518604419693:4543]
2025-10-09T21:49:44.179862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342518604419693:4543]
2025-10-09T21:49:44.179868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342518604419693:4543]
2025-10-09T21:49:44.179912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:44.179914Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:44.179919Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:44.179920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.179923Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:44.179923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:44.180058Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:44.180649Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1609 sessionId: src_id|f71b9977-3be49df2-cfd9cf7f-aa1495f4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:44.183017Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:44.240116Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342518966247861:7774]
2025-10-09T21:49:44.240359Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 1608, pipe: [1:7559342518966247861:7774]
2025-10-09T21:49:44.326894Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:44.327552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.327569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.327617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342518966247861:7774], now have 1 active actors on pipe
2025-10-09T21:49:44.327820Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:44.327853Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:44.327856Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:44.327860Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:44.327906Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342518966247861:7774]
2025-10-09T21:49:44.327919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1609 reason: error at write: Disconnected sessionId: src_id|f71b9977-3be49df2-cfd9cf7f-aa1495f4_0
2025-10-09T21:49:44.328083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1609 sessionId: src_id|f71b9977-3be49df2-cfd9cf7f-aa1495f4_0 is DEAD
2025-10-09T21:49:44.328241Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f71b9977-3be49df2-cfd9cf7f-aa1495f4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:44.328250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f71b9977-3be49df2-cfd9cf7f-aa1495f4_0] Write session will restart in 0.009229s
2025-10-09T21:49:44.328270Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f71b9977-3be49df2-cfd9cf7f-aa1495f4_0] Write session: Do CDS request
2025-10-09T21:49:44.328275Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f71b9977-3be49df2-cfd9cf7f-aa1495f4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:44.338491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f71b9977-3be49df2-cfd9cf7f-aa1495f4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:44.338602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:44.338615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1610
2025-10-09T21:49:44.338723Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1610 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:44.338754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1610 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:44.338764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1610 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:44.338773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:44.339256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1610 sessionId:  describe result for acl check
2025-10-09T21:49:44.339290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:44.339296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:44.339297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:44.339303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342518966248025:7699] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:44.339843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342518966248025:7699] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:44.340172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:44.341855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342518966248025:7699] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:44.342660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:44.342873Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:44.342883Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:44.342896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:44.342952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:44.342993Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342518604419884:4535]
2025-10-09T21:49:44.343576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:44.343816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:44.343857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:44.344040Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342518966248025:7699] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:44.344052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342518966248025:7699] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:44.344056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342518966248025:7699] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:44.344064Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1610 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:44.344170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:44.344217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.344223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.344224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.344226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.344227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.344228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.344230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.344231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.344232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.344237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.344247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.344251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.344255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.344259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.344263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.344267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.344271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.344275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.344281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.344295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.344306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.344316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.344325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.344331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.344340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.344345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.344350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.344355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.344359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.344365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.344375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.344384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.344393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.344400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.344408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.344413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.344445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:44.350821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:44.351957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:44.351972Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:44.351974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:44.351978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:44.351985Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1609 [2:7559342518604419884:4535]
2025-10-09T21:49:44.351998Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:44.352015Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:44.352023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:44.352026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:44.352028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:44.352045Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:44.352055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:44.352063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:44.352118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.352125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.352128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.352131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:44.352201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:44.352210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.352224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.352241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.352261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:44.352287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342518966247868:2432], now have 1 active actors on pipe
2025-10-09T21:49:44.352295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342518966248064:7699], now have 1 active actors on pipe
2025-10-09T21:49:44.352302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342518604419879:4535]
2025-10-09T21:49:44.352304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342518604419879:4535]
2025-10-09T21:49:44.352307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342518604419879:4535]
2025-10-09T21:49:44.352320Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:44.352338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:44.352349Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:44.352353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:44.352356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:44.352358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:44.352360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:44.352361Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1609
2025-10-09T21:49:44.352381Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342518604419879:4535]
2025-10-09T21:49:44.352356Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1609
2025-10-09T21:49:44.352388Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:44.352391Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:44.352413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.352418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.352455Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|167b9e19-e59ac64c-6d6b73dd-19c9f936_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:44.352469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:44.352473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:44.352476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:44.352486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:44.352500Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:44.352529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:44.352690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.352700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.352714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:44.352811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1610 partition: 0 MaxSeqNo: 25 sessionId: src_id|167b9e19-e59ac64c-6d6b73dd-19c9f936_0
2025-10-09T21:49:44.353139Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f71b9977-3be49df2-cfd9cf7f-aa1495f4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|167b9e19-e59ac64c-6d6b73dd-19c9f936_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:44.355552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|167b9e19-e59ac64c-6d6b73dd-19c9f936_0] Write session: try to update token
2025-10-09T21:49:44.355567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|167b9e19-e59ac64c-6d6b73dd-19c9f936_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:44.366993Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:44.367009Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:44.367012Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:44.367014Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:44.367025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342518604419879:4535]
2025-10-09T21:49:44.367029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342518604419879:4535]
2025-10-09T21:49:44.367034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342518604419879:4535]
2025-10-09T21:49:44.367079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:44.367087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.367087Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:44.367090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:44.367092Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:44.367096Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:44.367253Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:44.368928Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1610 sessionId: src_id|167b9e19-e59ac64c-6d6b73dd-19c9f936_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:44.371518Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:44.428044Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342518966248074:7774]
2025-10-09T21:49:44.428301Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 1609, pipe: [1:7559342518966248074:7774]
2025-10-09T21:49:44.523261Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:44.524078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.524095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.524135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342518966248074:7774], now have 1 active actors on pipe
2025-10-09T21:49:44.524362Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:44.524413Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:44.524443Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342518966248074:7774]
2025-10-09T21:49:44.524468Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:44.524571Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1610 reason: status is not ok: tablet will be restarted right now sessionId: src_id|167b9e19-e59ac64c-6d6b73dd-19c9f936_0
2025-10-09T21:49:44.524750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1610 sessionId: src_id|167b9e19-e59ac64c-6d6b73dd-19c9f936_0 is DEAD
2025-10-09T21:49:44.524931Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|167b9e19-e59ac64c-6d6b73dd-19c9f936_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:44.524941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|167b9e19-e59ac64c-6d6b73dd-19c9f936_0] Write session will restart in 0.005571s
2025-10-09T21:49:44.524960Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|167b9e19-e59ac64c-6d6b73dd-19c9f936_0] Write session: Do CDS request
2025-10-09T21:49:44.524966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|167b9e19-e59ac64c-6d6b73dd-19c9f936_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:44.531505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|167b9e19-e59ac64c-6d6b73dd-19c9f936_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:44.531590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:44.531602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1611
2025-10-09T21:49:44.531716Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1611 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:44.531758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1611 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:44.531768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1611 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:44.531776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:44.532207Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1611 sessionId:  describe result for acl check
2025-10-09T21:49:44.532243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:44.532250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:44.532252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:44.532257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342518966248115:7700] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:44.532793Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342518966248115:7700] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:44.535251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342518966248115:7700] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:44.537680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342518966248115:7700] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:44.537692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342518966248115:7700] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:44.537696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342518966248115:7700] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:44.537703Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1611 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:44.540086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:44.540567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:44.540733Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:44.540741Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:44.540752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:44.540797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:44.540837Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342518604420072:4534]
2025-10-09T21:49:44.541402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:44.541666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:44.541705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:44.541999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:44.542061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.542066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.542068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.542069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.542070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.542071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.542072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.542073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.542074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.542078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.542089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.542093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.542097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.542101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.542105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.542108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.542116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.542120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.542127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.542135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.542147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.542156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.542162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.542169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.542178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.542183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.542187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.542192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.542201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.542205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.542214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.542223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.542232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.542240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.542250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.542260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.542291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:44.548158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:44.549503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:44.549518Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:44.549520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:44.549525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:44.549532Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1610 [2:7559342518604420072:4534]
2025-10-09T21:49:44.549547Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:44.549578Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:44.549585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:44.549588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:44.549591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:44.549606Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:44.549617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:44.549628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:44.549683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.549691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.549693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.549696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:44.549760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:44.549766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.549776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.549791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.549812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:44.549839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342518966248076:2432], now have 1 active actors on pipe
2025-10-09T21:49:44.549842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342518966248270:7700], now have 1 active actors on pipe
2025-10-09T21:49:44.549849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342518604420067:4534]
2025-10-09T21:49:44.549835Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1610
2025-10-09T21:49:44.549855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342518604420067:4534]
2025-10-09T21:49:44.549839Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1610
2025-10-09T21:49:44.549857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342518604420067:4534]
2025-10-09T21:49:44.549870Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:44.549886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:44.549886Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:44.549890Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:44.549892Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:44.549903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:44.549906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:44.549955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.549964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.549976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342518604420067:4534]
2025-10-09T21:49:44.549988Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:44.550007Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:44.550018Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|78aeb87d-5339374f-23a5028d-e055b324_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:44.550025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:44.550043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:44.550045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:44.550055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:44.550064Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:44.550440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:44.550616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.550623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.550637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:44.550739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1611 partition: 0 MaxSeqNo: 25 sessionId: src_id|78aeb87d-5339374f-23a5028d-e055b324_0
2025-10-09T21:49:44.551069Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|167b9e19-e59ac64c-6d6b73dd-19c9f936_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|78aeb87d-5339374f-23a5028d-e055b324_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:44.557300Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78aeb87d-5339374f-23a5028d-e055b324_0] Write session: try to update token
2025-10-09T21:49:44.557316Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78aeb87d-5339374f-23a5028d-e055b324_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:44.562679Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:44.562694Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:44.562696Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:44.562698Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:44.562712Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342518604420067:4534]
2025-10-09T21:49:44.562716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342518604420067:4534]
2025-10-09T21:49:44.562719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342518604420067:4534]
2025-10-09T21:49:44.562761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:44.562766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.562768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:44.562769Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:44.562776Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:44.562780Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:44.562893Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:44.571896Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1611 sessionId: src_id|78aeb87d-5339374f-23a5028d-e055b324_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:44.574316Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:44.624635Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342518966248305:7774]
2025-10-09T21:49:44.624917Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 1610, pipe: [1:7559342518966248305:7774]
2025-10-09T21:49:44.703255Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:44.703780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.703794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.703816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342518966248305:7774], now have 1 active actors on pipe
2025-10-09T21:49:44.704018Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:44.704021Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:44.704024Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342518966248305:7774]
2025-10-09T21:49:44.704034Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:44.704046Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:44.704104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1611 reason: error at write: Disconnected sessionId: src_id|78aeb87d-5339374f-23a5028d-e055b324_0
2025-10-09T21:49:44.704289Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1611 sessionId: src_id|78aeb87d-5339374f-23a5028d-e055b324_0 is DEAD
2025-10-09T21:49:44.704437Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78aeb87d-5339374f-23a5028d-e055b324_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:44.704444Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78aeb87d-5339374f-23a5028d-e055b324_0] Write session will restart in 0.005573s
2025-10-09T21:49:44.704463Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78aeb87d-5339374f-23a5028d-e055b324_0] Write session: Do CDS request
2025-10-09T21:49:44.704467Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78aeb87d-5339374f-23a5028d-e055b324_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:44.711488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78aeb87d-5339374f-23a5028d-e055b324_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:44.711590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:44.711605Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1612
2025-10-09T21:49:44.711739Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1612 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:44.711782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1612 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:44.711794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1612 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:44.711797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:44.712229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1612 sessionId:  describe result for acl check
2025-10-09T21:49:44.712263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:44.712269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:44.712271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:44.712277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342518966248358:7687] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:44.713246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342518966248358:7687] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:44.716287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342518966248358:7687] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:44.717647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:44.720535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:44.720745Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:44.720750Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:44.720759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:44.720798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:44.720830Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342518604420302:4506]
2025-10-09T21:49:44.721017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342518966248358:7687] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:44.721029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342518966248358:7687] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:44.721032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342518966248358:7687] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:44.721039Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1612 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:44.721183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:44.721355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:44.721380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:44.721598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:44.721643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.721648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.721649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.721649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.721650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.721651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.721651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.721652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.721653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.721656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.721662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.721664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.721667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.721669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.721671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.721673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.721675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.721677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.721681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.721686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.721690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.721694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.721696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.721699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.721701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.721704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.721707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.721710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.721712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.721715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.721718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.721721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.721724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.721727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.721729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.721732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.721749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:44.727889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:44.729405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:44.729416Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:44.729417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:44.729419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:44.729424Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1611 [2:7559342518604420302:4506]
2025-10-09T21:49:44.729432Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:44.729443Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:44.729445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:44.729447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:44.729448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:44.729460Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:44.729464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:44.729469Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:44.729514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.729523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.729524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.729527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:44.729594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:44.729600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.729611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.729625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.729634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:44.729654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342518966248509:7687], now have 1 active actors on pipe
2025-10-09T21:49:44.729656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342518966248307:2432], now have 1 active actors on pipe
2025-10-09T21:49:44.729661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342518604420296:4506]
2025-10-09T21:49:44.729662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342518604420296:4506]
2025-10-09T21:49:44.729664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342518604420296:4506]
2025-10-09T21:49:44.729673Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:44.729683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:44.729691Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:44.729693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:44.729697Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:44.729701Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:44.729703Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:44.729707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342518604420296:4506]
2025-10-09T21:49:44.729709Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:44.729705Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1611
2025-10-09T21:49:44.729727Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:44.729710Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1611
2025-10-09T21:49:44.729755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.729762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.729782Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8206a2df-fff9de6c-63d487e0-f9222cc8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:44.729791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:44.729794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:44.729795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:44.729802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:44.729812Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:44.729824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:44.729971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.729976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.729986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:44.730089Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1612 partition: 0 MaxSeqNo: 25 sessionId: src_id|8206a2df-fff9de6c-63d487e0-f9222cc8_0
2025-10-09T21:49:44.730393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78aeb87d-5339374f-23a5028d-e055b324_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8206a2df-fff9de6c-63d487e0-f9222cc8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:44.732794Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8206a2df-fff9de6c-63d487e0-f9222cc8_0] Write session: try to update token
2025-10-09T21:49:44.732808Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8206a2df-fff9de6c-63d487e0-f9222cc8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:44.746271Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:44.746287Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:44.746297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342518604420296:4506]
2025-10-09T21:49:44.746263Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1612 sessionId: src_id|8206a2df-fff9de6c-63d487e0-f9222cc8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:44.746347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:44.746367Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:44.746521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:44.746672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:44.746843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:44.746979Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:44.747174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:44.747216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:44.748438Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:44.748639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.748656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.748690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:44.748774Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:44.754116Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:44.754130Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:44.754133Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:44.754134Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:44.754143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342518604420296:4506]
2025-10-09T21:49:44.754148Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342518604420296:4506]
2025-10-09T21:49:44.754151Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342518604420296:4506]
2025-10-09T21:49:44.754180Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:44.754185Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:44.754189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:44.754189Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:44.754196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.754198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:44.754339Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:44.804291Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342518966248522:7774]
2025-10-09T21:49:44.804537Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 1611, pipe: [1:7559342518966248522:7774]
2025-10-09T21:49:44.898334Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:44.905097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.905121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.905269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:44.905325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:44.905383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:44.905439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:44.905481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:44.905491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:44.905565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:44.905626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:44.905683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:44.905740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:44.905779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:44.905785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:44.905839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:44.905892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:44.905948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:44.906005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:44.906060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:44.906066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:44.906400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:44.906459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:44.906514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:44.906574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:44.906618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:44.906627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:44.906697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:44.906760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:44.906815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:44.906871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:44.906914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:44.906922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:44.906988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:44.907053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:44.907109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:44.907165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:44.907206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:44.907214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:44.907274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:44.907327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:44.907381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:44.907436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:44.907475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:44.907480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:44.907534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:44.907585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:44.907636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:44.907691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:44.907737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:44.907747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:44.907810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:44.907862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:44.907921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:44.907982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:44.908019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:44.908024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:44.908078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:44.908131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:44.908185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:44.908240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:44.908357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:44.908364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:44.908410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342518966248522:7774], now have 1 active actors on pipe
2025-10-09T21:49:44.908604Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342518966248522:7774]
2025-10-09T21:49:44.908618Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:44.908621Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:44.908630Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:44.908904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1612 reason: error at write: Disconnected sessionId: src_id|8206a2df-fff9de6c-63d487e0-f9222cc8_0
2025-10-09T21:49:44.909100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1612 sessionId: src_id|8206a2df-fff9de6c-63d487e0-f9222cc8_0 is DEAD
2025-10-09T21:49:44.909167Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:44.909268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8206a2df-fff9de6c-63d487e0-f9222cc8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:44.909279Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8206a2df-fff9de6c-63d487e0-f9222cc8_0] Write session will restart in 0.005219s
2025-10-09T21:49:44.909304Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8206a2df-fff9de6c-63d487e0-f9222cc8_0] Write session: Do CDS request
2025-10-09T21:49:44.909311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8206a2df-fff9de6c-63d487e0-f9222cc8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:44.915434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8206a2df-fff9de6c-63d487e0-f9222cc8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:44.915555Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:44.915569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1613
2025-10-09T21:49:44.915677Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1613 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:44.915710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1613 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:44.915723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1613 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:44.915727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:44.916143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1613 sessionId:  describe result for acl check
2025-10-09T21:49:44.916176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:44.916183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:44.916186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:44.916192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342518966248567:7675] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:44.916721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342518966248567:7675] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:44.920039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342518966248567:7675] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:44.924991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:44.925110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342518966248567:7675] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:44.925124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342518966248567:7675] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:44.925127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342518966248567:7675] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:44.925135Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1613 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:44.925386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:44.925559Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:44.925569Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:44.925581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:44.925632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:44.925670Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342518604420493:4503]
2025-10-09T21:49:44.926254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:44.926518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:44.926561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:44.926866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:44.926913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.926919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.926920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.926921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.926922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.926924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.926925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.926926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.926927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.926932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.926944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.926948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.926956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.926960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.926969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.926972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.926976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.926980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.926994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.927002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:44.927008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.927014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:44.927022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.927027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:44.927030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.927035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:44.927043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.927048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:44.927056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.927061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:44.927071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.927080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.927089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.927094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.927102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.927108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.927142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:44.933318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:44.934660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:44.934676Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:44.934678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:44.934682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:44.934689Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1612 [2:7559342518604420493:4503]
2025-10-09T21:49:44.934701Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:44.934717Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:44.934725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:44.934728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:44.934730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:44.934745Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:44.934756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:44.934763Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:44.934813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.934820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.934822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.934825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:44.934888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:44.934895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.934907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:44.934924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:44.934942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:44.934966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342518966248729:7675], now have 1 active actors on pipe
2025-10-09T21:49:44.934974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342518966248529:2432], now have 1 active actors on pipe
2025-10-09T21:49:44.934980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342518604420487:4503]
2025-10-09T21:49:44.934987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342518604420487:4503]
2025-10-09T21:49:44.934989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342518604420487:4503]
2025-10-09T21:49:44.934979Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1612
2025-10-09T21:49:44.935002Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1612
2025-10-09T21:49:44.935002Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:44.935014Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:44.935017Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:44.935018Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:44.935023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:44.935038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:44.935041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:44.935061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.935065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.935100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342518604420487:4503]
2025-10-09T21:49:44.935112Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:44.935136Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cbce8018-2b0747dd-955b7ae3-463a34e5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:44.935138Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:44.935147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:44.935156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:44.935158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:44.935169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:44.935184Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:44.935219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:44.935410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.935419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.935434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:44.935515Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1613 partition: 0 MaxSeqNo: 25 sessionId: src_id|cbce8018-2b0747dd-955b7ae3-463a34e5_0
2025-10-09T21:49:44.935853Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8206a2df-fff9de6c-63d487e0-f9222cc8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cbce8018-2b0747dd-955b7ae3-463a34e5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:44.938258Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cbce8018-2b0747dd-955b7ae3-463a34e5_0] Write session: try to update token
2025-10-09T21:49:44.938270Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cbce8018-2b0747dd-955b7ae3-463a34e5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:44.946149Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:44.946162Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:44.946173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342518604420487:4503]
2025-10-09T21:49:44.946209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:44.946210Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:44.946362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:44.946483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:44.946600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:44.946718Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:44.946880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:44.946910Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:44.951958Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1613 sessionId: src_id|cbce8018-2b0747dd-955b7ae3-463a34e5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:44.954245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:44.954572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:44.954593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:44.954630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:44.954716Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:44.958866Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:44.958876Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:44.958877Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:44.958879Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:44.958887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342518604420487:4503]
2025-10-09T21:49:44.958889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342518604420487:4503]
2025-10-09T21:49:44.958892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342518604420487:4503]
2025-10-09T21:49:44.958923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:44.958928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:44.958931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:44.958933Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:44.958939Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:44.958946Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:44.959045Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:45.008638Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342523261216036:7774]
2025-10-09T21:49:45.008935Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 1612, pipe: [1:7559342523261216036:7774]
2025-10-09T21:49:45.105763Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:45.112839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:45.112865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:45.113011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:45.113061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:45.113118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:45.113182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:45.113225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:45.113235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:45.113294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:45.113344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:45.113403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:45.113463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:45.113504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:45.113510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:45.113590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:45.113659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:45.113733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:45.113793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:45.113831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:45.113836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:45.113898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:45.113961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:45.114018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:45.114095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:45.114139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:45.114142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:45.114205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:45.114291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:45.114347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:45.114402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:45.114449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:45.114458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:45.114514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:45.114566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:45.114619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:45.114676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:45.114721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:45.114729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:45.114785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:45.114836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:45.114891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:45.114949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:45.114990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:45.114996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:45.115049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:45.115099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:45.115151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:45.115212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:45.115257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:45.115262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:45.115318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:45.115370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:45.115423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:45.115487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:45.115527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:45.115533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:45.115587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:45.115637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:45.115692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:45.115752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:45.115792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:45.115798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:45.115843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342523261216036:7774], now have 1 active actors on pipe
2025-10-09T21:49:45.116080Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:45.116080Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:45.116085Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:45.116117Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1613 reason: error at write: Disconnected sessionId: src_id|cbce8018-2b0747dd-955b7ae3-463a34e5_0
2025-10-09T21:49:45.116128Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342523261216036:7774]
2025-10-09T21:49:45.116309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1613 sessionId: src_id|cbce8018-2b0747dd-955b7ae3-463a34e5_0 is DEAD
2025-10-09T21:49:45.116433Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cbce8018-2b0747dd-955b7ae3-463a34e5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:45.116444Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cbce8018-2b0747dd-955b7ae3-463a34e5_0] Write session will restart in 0.007361s
2025-10-09T21:49:45.116466Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cbce8018-2b0747dd-955b7ae3-463a34e5_0] Write session: Do CDS request
2025-10-09T21:49:45.116472Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cbce8018-2b0747dd-955b7ae3-463a34e5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:45.116706Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:45.124470Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cbce8018-2b0747dd-955b7ae3-463a34e5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:45.124593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:45.124608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1614
2025-10-09T21:49:45.124719Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1614 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:45.124748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1614 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:45.124753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1614 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:45.124757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:45.125193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1614 sessionId:  describe result for acl check
2025-10-09T21:49:45.125243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:45.125249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:45.125251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:45.125258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342523261216115:7669] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:45.125836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342523261216115:7669] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:45.129437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342523261216115:7669] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:45.130091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:45.133631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:45.133857Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:45.133871Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:45.133884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:45.133943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:45.133990Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342522899387978:4483]
2025-10-09T21:49:45.134075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342523261216115:7669] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:45.134094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342523261216115:7669] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:45.134099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342523261216115:7669] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:45.134110Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1614 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:45.134603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:45.134957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:45.135013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:45.135374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:45.135431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.135437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.135438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.135440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.135441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.135442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.135444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.135445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.135446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.135451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.135463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.135468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.135472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.135476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.135480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.135484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.135488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.135492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.135499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.135508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.135515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.135520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.135531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.135536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.135541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.135546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.135551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.135556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.135561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.135567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.135572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.135578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.135582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.135588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.135592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.135602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.135629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:45.142171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:45.143238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:45.143253Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:45.143256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:45.143261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:45.143268Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1613 [2:7559342522899387978:4483]
2025-10-09T21:49:45.143282Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:45.143299Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:45.143307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:45.143310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:45.143312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:45.143331Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:45.143343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:45.143350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:45.143414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.143421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.143423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.143425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:45.143494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:45.143504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.143515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.143532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.143554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:45.143580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342523261216237:7669], now have 1 active actors on pipe
2025-10-09T21:49:45.143588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342523261216039:2432], now have 1 active actors on pipe
2025-10-09T21:49:45.143596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342522899387973:4483]
2025-10-09T21:49:45.143602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342522899387973:4483]
2025-10-09T21:49:45.143605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342522899387973:4483]
2025-10-09T21:49:45.143618Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:45.143634Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:45.143635Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:45.143637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:45.143640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:45.143652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:45.143655Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:45.143653Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1613
2025-10-09T21:49:45.143688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342522899387973:4483]
2025-10-09T21:49:45.143656Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1613
2025-10-09T21:49:45.143700Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:45.143705Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:45.143725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:45.143734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:45.143795Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|946052f0-af095401-a13bf6a4-96e502ac_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:45.143806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:45.143814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:45.143817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:45.143827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:45.143842Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:45.143864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:45.144000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:45.144009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:45.144025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:45.144109Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1614 partition: 0 MaxSeqNo: 25 sessionId: src_id|946052f0-af095401-a13bf6a4-96e502ac_0
2025-10-09T21:49:45.144452Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cbce8018-2b0747dd-955b7ae3-463a34e5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|946052f0-af095401-a13bf6a4-96e502ac_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:45.146851Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|946052f0-af095401-a13bf6a4-96e502ac_0] Write session: try to update token
2025-10-09T21:49:45.146865Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|946052f0-af095401-a13bf6a4-96e502ac_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:45.157964Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:45.157979Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:45.157980Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:45.157982Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:45.157990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342522899387973:4483]
2025-10-09T21:49:45.157993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342522899387973:4483]
2025-10-09T21:49:45.157996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342522899387973:4483]
2025-10-09T21:49:45.158052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:45.158060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.158067Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:45.158068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:45.158073Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:45.158077Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:45.158195Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:45.159916Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1614 sessionId: src_id|946052f0-af095401-a13bf6a4-96e502ac_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:45.162234Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:45.216499Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342523261216252:7774]
2025-10-09T21:49:45.216769Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 1613, pipe: [1:7559342523261216252:7774]
2025-10-09T21:49:45.308126Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:45.308696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:45.308714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:45.308759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342523261216252:7774], now have 1 active actors on pipe
2025-10-09T21:49:45.309012Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:45.309017Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342523261216252:7774]
2025-10-09T21:49:45.309044Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:45.309047Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:45.309106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1614 reason: error at write: Disconnected sessionId: src_id|946052f0-af095401-a13bf6a4-96e502ac_0
2025-10-09T21:49:45.309153Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:45.309303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1614 sessionId: src_id|946052f0-af095401-a13bf6a4-96e502ac_0 is DEAD
2025-10-09T21:49:45.309460Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|946052f0-af095401-a13bf6a4-96e502ac_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:45.309471Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|946052f0-af095401-a13bf6a4-96e502ac_0] Write session will restart in 0.008872s
2025-10-09T21:49:45.309497Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|946052f0-af095401-a13bf6a4-96e502ac_0] Write session: Do CDS request
2025-10-09T21:49:45.309503Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|946052f0-af095401-a13bf6a4-96e502ac_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:45.319527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|946052f0-af095401-a13bf6a4-96e502ac_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:45.319625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:45.319641Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1615
2025-10-09T21:49:45.319753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1615 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:45.319785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1615 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:45.319809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1615 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:45.319818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:45.320281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1615 sessionId:  describe result for acl check
2025-10-09T21:49:45.320317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:45.320324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:45.320326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:45.320332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342523261216415:7658] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:45.320869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342523261216415:7658] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:45.322814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342523261216415:7658] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:45.325162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:45.325302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342523261216415:7658] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:45.325315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342523261216415:7658] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:45.325319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342523261216415:7658] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:45.325327Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1615 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:45.325638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:45.325819Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:45.325829Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:45.325839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:45.325882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:45.325911Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342522899388170:4481]
2025-10-09T21:49:45.326389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:45.326665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:45.326708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:45.327022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:45.327070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.327076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.327077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.327079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.327080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.327081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.327082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.327083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.327084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.327089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.327100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.327105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.327109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.327113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.327117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.327122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.327130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.327134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.327140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.327149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.327160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.327166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.327175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.327184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.327193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.327198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.327208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.327213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.327221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.327227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.327236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.327245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.327253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.327258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.327266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.327271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.327306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:45.333665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:45.334779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:45.334793Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:45.334796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:45.334800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:45.334807Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1614 [2:7559342522899388170:4481]
2025-10-09T21:49:45.334822Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:45.334839Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:45.334847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:45.334850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:45.334852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:45.334868Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:45.334879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:45.334887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:45.334940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.334947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.334949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.334953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:45.335035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:45.335043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.335053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.335070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.335093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:45.335120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342523261216458:7658], now have 1 active actors on pipe
2025-10-09T21:49:45.335123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342523261216259:2432], now have 1 active actors on pipe
2025-10-09T21:49:45.335130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342522899388164:4481]
2025-10-09T21:49:45.335133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342522899388164:4481]
2025-10-09T21:49:45.335135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342522899388164:4481]
2025-10-09T21:49:45.335149Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:45.335160Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:45.335163Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:45.335165Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:45.335169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:45.335150Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1614
2025-10-09T21:49:45.335162Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1614
2025-10-09T21:49:45.335184Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:45.335191Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:45.335239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:45.335248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:45.335260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342522899388164:4481]
2025-10-09T21:49:45.335271Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:45.335277Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:45.335304Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|42c722a2-410caffe-2b04bdfb-c6fa4ee4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:45.335316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:45.335320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:45.335322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:45.335332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:45.335342Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:45.335367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:45.335487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:45.335498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:45.335513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:45.335591Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1615 partition: 0 MaxSeqNo: 25 sessionId: src_id|42c722a2-410caffe-2b04bdfb-c6fa4ee4_0
2025-10-09T21:49:45.335944Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|946052f0-af095401-a13bf6a4-96e502ac_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|42c722a2-410caffe-2b04bdfb-c6fa4ee4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:45.338381Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42c722a2-410caffe-2b04bdfb-c6fa4ee4_0] Write session: try to update token
2025-10-09T21:49:45.338396Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42c722a2-410caffe-2b04bdfb-c6fa4ee4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:45.349662Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:45.349678Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:45.349682Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:45.349683Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:45.349693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342522899388164:4481]
2025-10-09T21:49:45.349697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342522899388164:4481]
2025-10-09T21:49:45.349703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342522899388164:4481]
2025-10-09T21:49:45.349749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:45.349761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.349760Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:45.349768Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:45.349768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:45.349771Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:45.349897Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:45.351625Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1615 sessionId: src_id|42c722a2-410caffe-2b04bdfb-c6fa4ee4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:45.354020Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:45.409019Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342523261216470:7774]
2025-10-09T21:49:45.409265Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 1614, pipe: [1:7559342523261216470:7774]
2025-10-09T21:49:45.496023Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:45.496607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:45.496621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:45.496658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342523261216470:7774], now have 1 active actors on pipe
2025-10-09T21:49:45.496875Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:45.496878Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:45.496893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:45.496895Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342523261216470:7774]
2025-10-09T21:49:45.496919Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:45.496971Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1615 reason: error at write: Disconnected sessionId: src_id|42c722a2-410caffe-2b04bdfb-c6fa4ee4_0
2025-10-09T21:49:45.497141Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1615 sessionId: src_id|42c722a2-410caffe-2b04bdfb-c6fa4ee4_0 is DEAD
2025-10-09T21:49:45.497287Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42c722a2-410caffe-2b04bdfb-c6fa4ee4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:45.497299Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42c722a2-410caffe-2b04bdfb-c6fa4ee4_0] Write session will restart in 0.008214s
2025-10-09T21:49:45.497335Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42c722a2-410caffe-2b04bdfb-c6fa4ee4_0] Write session: Do CDS request
2025-10-09T21:49:45.497341Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42c722a2-410caffe-2b04bdfb-c6fa4ee4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:45.506559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|42c722a2-410caffe-2b04bdfb-c6fa4ee4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:45.506656Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:45.506672Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1616
2025-10-09T21:49:45.506787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1616 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:45.506832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1616 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:45.506842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1616 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:45.506845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:45.507260Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1616 sessionId:  describe result for acl check
2025-10-09T21:49:45.507293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:45.507300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:45.507301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:45.507307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342523261216602:7656] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:45.507853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342523261216602:7656] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:45.509605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342523261216602:7656] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:45.512957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:45.513127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342523261216602:7656] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:45.513138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342523261216602:7656] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:45.513142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342523261216602:7656] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:45.513149Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1616 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:45.513340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:45.513509Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:45.513519Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:45.513531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:45.513593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:45.513631Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342522899388362:4465]
2025-10-09T21:49:45.514218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:45.514473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:45.514511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:45.514821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:45.514870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.514877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.514878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.514879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.514880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.514881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.514882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.514884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.514885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.514890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.514901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.514905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.514910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.514914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.514918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.514922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.514926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.514930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.514937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.514944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.514950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.514956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.514960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.514965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.514970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.514974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.514979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.514985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.514990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.514996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.515001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.515006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.515010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.515015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.515020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.515025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.515054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:45.521442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:45.522640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:45.522655Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:45.522658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:45.522662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:45.522670Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1615 [2:7559342522899388362:4465]
2025-10-09T21:49:45.522684Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:45.522705Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:45.522712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:45.522715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:45.522717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:45.522733Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:45.522744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:45.522751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:45.522810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.522817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.522821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.522824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:45.522888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:45.522895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.522906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.522924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.522944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:45.522969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342523261216472:2432], now have 1 active actors on pipe
2025-10-09T21:49:45.522977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342523261216671:7656], now have 1 active actors on pipe
2025-10-09T21:49:45.522985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342522899388356:4465]
2025-10-09T21:49:45.522992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342522899388356:4465]
2025-10-09T21:49:45.522995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342522899388356:4465]
2025-10-09T21:49:45.523009Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:45.523027Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:45.523028Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:45.523044Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:45.523048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:45.523048Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:45.523051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:45.523034Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1615
2025-10-09T21:49:45.523038Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1615
2025-10-09T21:49:45.523078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342522899388356:4465]
2025-10-09T21:49:45.523086Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:45.523087Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:45.523114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:45.523123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:45.523160Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6406bf65-5654b9d8-e7835d64-2d7ca633_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:45.523174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:45.523178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:45.523180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:45.523193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:45.523202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:45.523221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:45.523381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:45.523391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:45.523417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:45.523522Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1616 partition: 0 MaxSeqNo: 25 sessionId: src_id|6406bf65-5654b9d8-e7835d64-2d7ca633_0
2025-10-09T21:49:45.523878Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|42c722a2-410caffe-2b04bdfb-c6fa4ee4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6406bf65-5654b9d8-e7835d64-2d7ca633_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:45.526238Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6406bf65-5654b9d8-e7835d64-2d7ca633_0] Write session: try to update token
2025-10-09T21:49:45.526253Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6406bf65-5654b9d8-e7835d64-2d7ca633_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:45.537418Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:45.537436Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:45.537439Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:45.537440Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:45.537450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342522899388356:4465]
2025-10-09T21:49:45.537455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342522899388356:4465]
2025-10-09T21:49:45.537459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342522899388356:4465]
2025-10-09T21:49:45.537492Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:45.537500Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:45.537501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:45.537503Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:45.537507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.537510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:45.537652Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:45.538500Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1616 sessionId: src_id|6406bf65-5654b9d8-e7835d64-2d7ca633_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:45.541072Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:45.596893Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342523261216696:7774]
2025-10-09T21:49:45.597067Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 1615, pipe: [1:7559342523261216696:7774]
2025-10-09T21:49:45.686400Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:45.686845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:45.686860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:45.686904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342523261216696:7774], now have 1 active actors on pipe
2025-10-09T21:49:45.687266Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:45.687266Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:45.687276Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342523261216696:7774]
2025-10-09T21:49:45.687297Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:45.687301Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:45.687376Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1616 reason: error at write: Disconnected sessionId: src_id|6406bf65-5654b9d8-e7835d64-2d7ca633_0
2025-10-09T21:49:45.687562Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1616 sessionId: src_id|6406bf65-5654b9d8-e7835d64-2d7ca633_0 is DEAD
2025-10-09T21:49:45.687786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6406bf65-5654b9d8-e7835d64-2d7ca633_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:45.687800Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6406bf65-5654b9d8-e7835d64-2d7ca633_0] Write session will restart in 0.005009s
2025-10-09T21:49:45.687839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6406bf65-5654b9d8-e7835d64-2d7ca633_0] Write session: Do CDS request
2025-10-09T21:49:45.687845Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6406bf65-5654b9d8-e7835d64-2d7ca633_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:45.694578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6406bf65-5654b9d8-e7835d64-2d7ca633_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:45.694664Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:45.694679Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1617
2025-10-09T21:49:45.695269Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1617 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:45.695305Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1617 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:45.695311Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1617 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:45.695314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:45.695871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1617 sessionId:  describe result for acl check
2025-10-09T21:49:45.695907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:45.695909Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:45.695912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:45.695918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342523261216737:7642] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:45.696485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342523261216737:7642] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:45.699264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342523261216737:7642] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:45.702274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342523261216737:7642] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:45.702285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342523261216737:7642] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:45.702288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342523261216737:7642] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:45.702295Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1617 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:45.705635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:45.706206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:45.706427Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:45.706436Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:45.706448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:45.706495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:45.706533Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342522899388597:4459]
2025-10-09T21:49:45.707202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:45.707479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:45.707521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:45.707855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:45.707905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.707910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.707911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.707913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.707914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.707915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.707916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.707917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.707919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.707923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.707935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.707939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.707943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.707947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.707951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.707955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.707959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.707963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.707970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.707979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.707989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.707994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.708004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.708010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.708014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.708019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.708023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.708030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.708038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.708043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.708048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.708057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.708062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.708071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.708076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.708082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.708110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:45.714731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:45.716667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:45.716680Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:45.716682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:45.716685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:45.716691Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1616 [2:7559342522899388597:4459]
2025-10-09T21:49:45.716705Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:45.716719Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:45.716722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:45.716724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:45.716725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:45.716739Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:45.716743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:45.716749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:45.716795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.716801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.716802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.716804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:45.716868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:45.716874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.716882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.716889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.716898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:45.716914Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342522899388592:4459]
2025-10-09T21:49:45.716916Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342522899388592:4459]
2025-10-09T21:49:45.716918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342522899388592:4459]
2025-10-09T21:49:45.716924Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:45.716935Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:45.716947Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:45.716949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:45.716950Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:45.716951Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:45.716952Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:45.716987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342523261216709:2432], now have 1 active actors on pipe
2025-10-09T21:49:45.716993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342523261216904:7642], now have 1 active actors on pipe
2025-10-09T21:49:45.716995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342522899388592:4459]
2025-10-09T21:49:45.717000Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:45.717013Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:45.717079Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1616
2025-10-09T21:49:45.717083Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1616
2025-10-09T21:49:45.717278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:45.717287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:45.717309Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8b2e36e5-f0fd36af-68d3685b-43b0627c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:45.717320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:45.717323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:45.717325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:45.717334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:45.717344Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:45.717359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:45.717535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:45.717541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:45.717563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:45.717654Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1617 partition: 0 MaxSeqNo: 25 sessionId: src_id|8b2e36e5-f0fd36af-68d3685b-43b0627c_0
2025-10-09T21:49:45.718041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6406bf65-5654b9d8-e7835d64-2d7ca633_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8b2e36e5-f0fd36af-68d3685b-43b0627c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:45.722178Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2e36e5-f0fd36af-68d3685b-43b0627c_0] Write session: try to update token
2025-10-09T21:49:45.722198Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2e36e5-f0fd36af-68d3685b-43b0627c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:45.731835Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:45.731852Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:45.731854Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:45.731855Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:45.731863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342522899388592:4459]
2025-10-09T21:49:45.731866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342522899388592:4459]
2025-10-09T21:49:45.731869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342522899388592:4459]
2025-10-09T21:49:45.731904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:45.731911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.731914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:45.731912Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:45.731918Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:45.731921Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:45.732058Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:45.736085Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1617 sessionId: src_id|8b2e36e5-f0fd36af-68d3685b-43b0627c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:45.738487Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:45.787388Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342523261216919:7774]
2025-10-09T21:49:45.787670Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 1616, pipe: [1:7559342523261216919:7774]
2025-10-09T21:49:45.881758Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:45.882058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:45.882076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:45.882123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342523261216919:7774], now have 1 active actors on pipe
2025-10-09T21:49:45.882368Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:45.882358Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:45.882378Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342523261216919:7774]
2025-10-09T21:49:45.882408Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:45.882411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:45.882472Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1617 reason: error at write: Disconnected sessionId: src_id|8b2e36e5-f0fd36af-68d3685b-43b0627c_0
2025-10-09T21:49:45.882674Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1617 sessionId: src_id|8b2e36e5-f0fd36af-68d3685b-43b0627c_0 is DEAD
2025-10-09T21:49:45.882808Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2e36e5-f0fd36af-68d3685b-43b0627c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:45.882817Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2e36e5-f0fd36af-68d3685b-43b0627c_0] Write session will restart in 0.006616s
2025-10-09T21:49:45.882838Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2e36e5-f0fd36af-68d3685b-43b0627c_0] Write session: Do CDS request
2025-10-09T21:49:45.882843Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2e36e5-f0fd36af-68d3685b-43b0627c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:45.890560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2e36e5-f0fd36af-68d3685b-43b0627c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:45.890797Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:45.890819Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1618
2025-10-09T21:49:45.890955Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1618 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:45.890993Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1618 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:45.890999Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1618 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:45.891002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:45.891473Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1618 sessionId:  describe result for acl check
2025-10-09T21:49:45.891505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:45.891512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:45.891516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:45.891522Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342523261216987:7635] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:45.892046Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342523261216987:7635] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:45.894571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342523261216987:7635] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:45.896822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342523261216987:7635] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:45.896832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342523261216987:7635] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:45.896835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342523261216987:7635] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:45.896840Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1618 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:45.897162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:45.900012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:45.900248Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:45.900259Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:45.900273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:45.900339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:45.900382Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342522899388792:4436]
2025-10-09T21:49:45.900945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:45.901239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:45.901285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:45.901607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:45.901676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.901683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.901685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.901686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.901687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.901688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.901690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.901691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.901693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.901698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.901709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.901714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.901718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.901722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.901726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.901729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.901732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.901736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.901742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.901749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:45.901755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.901760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:45.901765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.901769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:45.901773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.901778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:45.901783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.901787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:45.901792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.901804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:45.901809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.901815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.901820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.901825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.901829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.901834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.901862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:45.908268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:45.909269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:45.909283Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:45.909284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:45.909287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:45.909292Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1617 [2:7559342522899388792:4436]
2025-10-09T21:49:45.909304Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:45.909323Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:45.909326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:45.909329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:45.909331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:45.909346Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:45.909352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:45.909360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:45.909407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.909412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.909414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.909416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:45.909480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:45.909489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.909499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:45.909510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:45.909530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:45.909567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342523261217126:7635], now have 1 active actors on pipe
2025-10-09T21:49:45.909573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342523261216925:2432], now have 1 active actors on pipe
2025-10-09T21:49:45.909579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342522899388787:4436]
2025-10-09T21:49:45.909581Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342522899388787:4436]
2025-10-09T21:49:45.909582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342522899388787:4436]
2025-10-09T21:49:45.909594Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:45.909609Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:45.909611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:45.909616Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:45.909618Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:45.909627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:45.909630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:45.909632Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1617
2025-10-09T21:49:45.909656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342522899388787:4436]
2025-10-09T21:49:45.909634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1617
2025-10-09T21:49:45.909664Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:45.909665Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:45.909687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:45.909691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:45.909788Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|96a0f550-df21b13b-955ec54b-4298e069_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:45.909801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:45.909805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:45.909807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:45.909817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:45.909831Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:45.909856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:45.910003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:45.910010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:45.910023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:45.910138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1618 partition: 0 MaxSeqNo: 25 sessionId: src_id|96a0f550-df21b13b-955ec54b-4298e069_0
2025-10-09T21:49:45.910555Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2e36e5-f0fd36af-68d3685b-43b0627c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|96a0f550-df21b13b-955ec54b-4298e069_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:45.912964Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96a0f550-df21b13b-955ec54b-4298e069_0] Write session: try to update token
2025-10-09T21:49:45.912982Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96a0f550-df21b13b-955ec54b-4298e069_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:45.924033Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:45.924047Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:45.924049Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:45.924050Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:45.924057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342522899388787:4436]
2025-10-09T21:49:45.924060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342522899388787:4436]
2025-10-09T21:49:45.924063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342522899388787:4436]
2025-10-09T21:49:45.924103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:45.924105Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:45.924109Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:45.924111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:45.924114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:45.924115Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:45.924318Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:45.925847Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1618 sessionId: src_id|96a0f550-df21b13b-955ec54b-4298e069_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:45.928023Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:45.982454Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342523261217138:7774]
2025-10-09T21:49:45.982673Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 1617, pipe: [1:7559342523261217138:7774]
2025-10-09T21:49:46.070369Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:46.070978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.070991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.071027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342523261217138:7774], now have 1 active actors on pipe
2025-10-09T21:49:46.071229Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:46.071255Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:46.071272Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:46.071271Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342523261217138:7774]
2025-10-09T21:49:46.071291Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:46.071336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1618 reason: error at write: Disconnected sessionId: src_id|96a0f550-df21b13b-955ec54b-4298e069_0
2025-10-09T21:49:46.071537Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1618 sessionId: src_id|96a0f550-df21b13b-955ec54b-4298e069_0 is DEAD
2025-10-09T21:49:46.071702Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96a0f550-df21b13b-955ec54b-4298e069_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:46.071708Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96a0f550-df21b13b-955ec54b-4298e069_0] Write session will restart in 0.009260s
2025-10-09T21:49:46.071731Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96a0f550-df21b13b-955ec54b-4298e069_0] Write session: Do CDS request
2025-10-09T21:49:46.071735Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96a0f550-df21b13b-955ec54b-4298e069_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:46.082497Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96a0f550-df21b13b-955ec54b-4298e069_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:46.082622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:46.082636Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1619
2025-10-09T21:49:46.082746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1619 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:46.082778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1619 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:46.082783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1619 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:46.082787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:46.083225Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1619 sessionId:  describe result for acl check
2025-10-09T21:49:46.083260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:46.083266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:46.083268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:46.083275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342527556184605:7631] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:46.083605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:46.083809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342527556184605:7631] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:46.085770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342527556184605:7631] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:46.086507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:46.086712Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:46.086717Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:46.086728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:46.086772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:46.086802Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342527194356282:4420]
2025-10-09T21:49:46.087247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:46.087564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:46.087617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:46.087899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:46.087939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.087952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.087958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.087959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.087961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.087962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.087964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.087965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.087966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.087972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.087980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.087982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.087985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.087987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.087989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.087991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.087993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.087996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.088001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.088006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.088010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.088014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.088016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.088019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.088022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.088025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.087978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342527556184605:7631] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:46.088028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.087989Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342527556184605:7631] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:46.088031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.087993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342527556184605:7631] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:46.088034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.088037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.088039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.088043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.088045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.088001Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1619 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:46.088048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.088052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.088055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.088072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:46.094571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:46.096087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:46.096100Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:46.096102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:46.096105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:46.096115Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1618 [2:7559342527194356282:4420]
2025-10-09T21:49:46.096131Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:46.096153Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:46.096156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:46.096159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:46.096161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:46.096180Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:46.096185Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:46.096191Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:46.096242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.096248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.096249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.096251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:46.096310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:46.096315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.096323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.096329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.096338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:46.096354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342527194356277:4420]
2025-10-09T21:49:46.096359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342527194356277:4420]
2025-10-09T21:49:46.096360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342527194356277:4420]
2025-10-09T21:49:46.096373Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:46.096388Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:46.096391Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:46.096393Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:46.096395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342527556184642:7631], now have 1 active actors on pipe
2025-10-09T21:49:46.096399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342527556184437:2432], now have 1 active actors on pipe
2025-10-09T21:49:46.096409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:46.096419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:46.096425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:46.096458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342527194356277:4420]
2025-10-09T21:49:46.096467Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:46.096479Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:46.096473Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1618
2025-10-09T21:49:46.096480Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1618
2025-10-09T21:49:46.096533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.096542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.096573Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ebb99d02-eed3a46c-bc806b45-df921fa5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:46.096587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:46.096590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:46.096593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:46.096602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:46.096610Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:46.096623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:46.096759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.096764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.096775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:46.096865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1619 partition: 0 MaxSeqNo: 25 sessionId: src_id|ebb99d02-eed3a46c-bc806b45-df921fa5_0
2025-10-09T21:49:46.097225Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96a0f550-df21b13b-955ec54b-4298e069_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ebb99d02-eed3a46c-bc806b45-df921fa5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:46.099736Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb99d02-eed3a46c-bc806b45-df921fa5_0] Write session: try to update token
2025-10-09T21:49:46.099756Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb99d02-eed3a46c-bc806b45-df921fa5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:46.108869Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:46.108887Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:46.108896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342527194356277:4420]
2025-10-09T21:49:46.108935Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:46.108943Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:46.109117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:46.109244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:46.109369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:46.109498Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:46.109684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:46.109719Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:46.113601Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1619 sessionId: src_id|ebb99d02-eed3a46c-bc806b45-df921fa5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:46.116064Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:46.116280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.116298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.116340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:46.116428Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:46.121937Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:46.121953Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:46.121954Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:46.121956Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:46.121965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342527194356277:4420]
2025-10-09T21:49:46.121968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342527194356277:4420]
2025-10-09T21:49:46.121970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342527194356277:4420]
2025-10-09T21:49:46.122004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:46.122011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.122013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:46.122015Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:46.122020Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:46.122024Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:46.122150Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:46.171387Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342527556184652:7774]
2025-10-09T21:49:46.171633Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 1618, pipe: [1:7559342527556184652:7774]
2025-10-09T21:49:46.269516Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:46.276264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.276286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.276430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.276481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.276541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.276597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.276724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.276733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.276791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.276845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.276900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.276957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.276996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.277005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.277058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.277106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.277161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.277221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.277264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.277272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.277335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.277385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.277437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.277491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.277530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.277535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.277601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.277651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.277710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.277770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.277807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.277813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.277865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.277916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.277969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.278021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.278157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.278165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.278216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.278263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.278314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.278374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.278410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.278415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.278466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.278516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.278567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.278628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.278665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.278670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.278728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.278785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.278836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.278893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.278930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.278935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.278985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.279032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.279081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.279142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.279193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.279198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.279233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342527556184652:7774], now have 1 active actors on pipe
2025-10-09T21:49:46.279462Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:46.279471Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:46.279476Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:46.279511Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1619 reason: error at write: Disconnected sessionId: src_id|ebb99d02-eed3a46c-bc806b45-df921fa5_0
2025-10-09T21:49:46.279523Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342527556184652:7774]
2025-10-09T21:49:46.279713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1619 sessionId: src_id|ebb99d02-eed3a46c-bc806b45-df921fa5_0 is DEAD
2025-10-09T21:49:46.279840Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb99d02-eed3a46c-bc806b45-df921fa5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:46.279852Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb99d02-eed3a46c-bc806b45-df921fa5_0] Write session will restart in 0.009906s
2025-10-09T21:49:46.279877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb99d02-eed3a46c-bc806b45-df921fa5_0] Write session: Do CDS request
2025-10-09T21:49:46.279884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb99d02-eed3a46c-bc806b45-df921fa5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:46.279984Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:46.290518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb99d02-eed3a46c-bc806b45-df921fa5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:46.290577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:46.290593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1620
2025-10-09T21:49:46.290787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1620 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:46.290829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1620 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:46.290848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1620 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:46.290852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:46.291285Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1620 sessionId:  describe result for acl check
2025-10-09T21:49:46.291324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:46.291332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:46.291334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:46.291341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342527556184822:7622] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:46.291893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342527556184822:7622] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:46.293595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:46.294268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342527556184822:7622] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:46.296355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:46.296530Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:46.296537Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:46.296546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:46.296593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:46.296635Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342527194356478:4432]
2025-10-09T21:49:46.296796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342527556184822:7622] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:46.296810Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342527556184822:7622] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:46.296814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342527556184822:7622] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:46.296822Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1620 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:46.297031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:46.297256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:46.297286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:46.297503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:46.297558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.297563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.297564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.297564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.297565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.297566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.297566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.297567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.297568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.297571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.297578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.297580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.297583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.297585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.297587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.297589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.297591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.297593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.297598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.297603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.297607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.297611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.297613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.297617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.297619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.297622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.297625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.297630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.297635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.297640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.297644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.297648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.297655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.297660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.297668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.297674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.297698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:46.304547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:46.305948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:46.305961Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:46.305962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:46.305966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:46.305972Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1619 [2:7559342527194356478:4432]
2025-10-09T21:49:46.305980Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:46.305995Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:46.305997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:46.305999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:46.306001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:46.306014Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:46.306017Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:46.306022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:46.306108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.306115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.306117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.306120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:46.306185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:46.306193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.306204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.306220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.306241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:46.306267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342527556184663:2432], now have 1 active actors on pipe
2025-10-09T21:49:46.306276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342527556184867:7622], now have 1 active actors on pipe
2025-10-09T21:49:46.306289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342527194356473:4432]
2025-10-09T21:49:46.306291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342527194356473:4432]
2025-10-09T21:49:46.306292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342527194356473:4432]
2025-10-09T21:49:46.306299Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1619
2025-10-09T21:49:46.306302Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:46.306323Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1619
2025-10-09T21:49:46.306313Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:46.306320Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:46.306322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:46.306323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:46.306324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:46.306325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:46.306348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342527194356473:4432]
2025-10-09T21:49:46.306351Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:46.306359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:46.306367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.306371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.306409Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|226f4e6b-f1c26bea-8576e3d9-494baa11_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:46.306421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:46.306425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:46.306429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:46.306439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:46.306448Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:46.306467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:46.306614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.306620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.306629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:46.306702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1620 partition: 0 MaxSeqNo: 25 sessionId: src_id|226f4e6b-f1c26bea-8576e3d9-494baa11_0
2025-10-09T21:49:46.307030Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb99d02-eed3a46c-bc806b45-df921fa5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|226f4e6b-f1c26bea-8576e3d9-494baa11_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:46.309488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|226f4e6b-f1c26bea-8576e3d9-494baa11_0] Write session: try to update token
2025-10-09T21:49:46.309506Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|226f4e6b-f1c26bea-8576e3d9-494baa11_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:46.320985Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:46.321005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:46.321007Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:46.321009Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:46.321019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342527194356473:4432]
2025-10-09T21:49:46.321023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342527194356473:4432]
2025-10-09T21:49:46.321030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342527194356473:4432]
2025-10-09T21:49:46.321073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:46.321076Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:46.321080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.321082Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:46.321083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:46.321085Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:46.321217Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:46.322341Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1620 sessionId: src_id|226f4e6b-f1c26bea-8576e3d9-494baa11_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:46.324580Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:46.379753Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342527556184879:7774]
2025-10-09T21:49:46.379994Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 1619, pipe: [1:7559342527556184879:7774]
2025-10-09T21:49:46.475835Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:46.476369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.476387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.476432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342527556184879:7774], now have 1 active actors on pipe
2025-10-09T21:49:46.476667Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:46.476673Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342527556184879:7774]
2025-10-09T21:49:46.476711Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:46.476714Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:46.476775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1620 reason: error at write: Disconnected sessionId: src_id|226f4e6b-f1c26bea-8576e3d9-494baa11_0
2025-10-09T21:49:46.476970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1620 sessionId: src_id|226f4e6b-f1c26bea-8576e3d9-494baa11_0 is DEAD
2025-10-09T21:49:46.477148Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|226f4e6b-f1c26bea-8576e3d9-494baa11_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:46.477159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|226f4e6b-f1c26bea-8576e3d9-494baa11_0] Write session will restart in 0.008684s
2025-10-09T21:49:46.477185Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|226f4e6b-f1c26bea-8576e3d9-494baa11_0] Write session: Do CDS request
2025-10-09T21:49:46.477195Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|226f4e6b-f1c26bea-8576e3d9-494baa11_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:46.477234Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:46.487604Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|226f4e6b-f1c26bea-8576e3d9-494baa11_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:46.487652Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:46.487667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1621
2025-10-09T21:49:46.487840Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1621 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:46.487886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1621 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:46.487892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1621 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:46.487896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:46.488411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1621 sessionId:  describe result for acl check
2025-10-09T21:49:46.488449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:46.488451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:46.488453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:46.488459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342527556184897:7609] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:46.489012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342527556184897:7609] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:46.491762Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342527556184897:7609] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:46.494933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342527556184897:7609] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:46.494947Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342527556184897:7609] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:46.494951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342527556184897:7609] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:46.494959Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1621 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:46.497305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:46.500097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:46.500303Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:46.500314Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:46.500326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:46.500386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:46.500434Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342527194356677:4429]
2025-10-09T21:49:46.501055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:46.501357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:46.501397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:46.501745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:46.501808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.501815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.501816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.501818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.501819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.501821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.501822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.501823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.501824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.501830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.501842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.501846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.501849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.501853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.501856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.501860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.501864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.501868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.501877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.501886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.501893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.501899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.501904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.501909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.501913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.501917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.501922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.501927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.501931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.501937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.501943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.501948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.501952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.501957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.501962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.501967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.501996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:46.511582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:46.513011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:46.513028Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:46.513031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:46.513036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:46.513043Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1620 [2:7559342527194356677:4429]
2025-10-09T21:49:46.513061Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:46.513082Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:46.513091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:46.513094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:46.513097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:46.513115Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:46.513121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:46.513129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:46.513189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.513197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.513200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.513203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:46.513311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:46.513330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.513344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.513357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.513374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:46.513403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342527556184881:2432], now have 1 active actors on pipe
2025-10-09T21:49:46.513413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342527556185082:7609], now have 1 active actors on pipe
2025-10-09T21:49:46.513426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342527194356672:4429]
2025-10-09T21:49:46.513428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342527194356672:4429]
2025-10-09T21:49:46.513430Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342527194356672:4429]
2025-10-09T21:49:46.513448Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:46.513471Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:46.513482Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:46.513484Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:46.513486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:46.513498Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:46.513459Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1620
2025-10-09T21:49:46.513503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:46.513464Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1620
2025-10-09T21:49:46.513579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.513595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.513609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342527194356672:4429]
2025-10-09T21:49:46.513620Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:46.513628Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:46.513657Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3e999081-5f00fdcd-18c0ed11-6ac03d1b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:46.513673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:46.513678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:46.513680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:46.513693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:46.513704Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:46.513737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:46.513942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.513957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.513988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:46.514108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1621 partition: 0 MaxSeqNo: 25 sessionId: src_id|3e999081-5f00fdcd-18c0ed11-6ac03d1b_0
2025-10-09T21:49:46.514502Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|226f4e6b-f1c26bea-8576e3d9-494baa11_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3e999081-5f00fdcd-18c0ed11-6ac03d1b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:46.517022Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3e999081-5f00fdcd-18c0ed11-6ac03d1b_0] Write session: try to update token
2025-10-09T21:49:46.517042Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3e999081-5f00fdcd-18c0ed11-6ac03d1b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:46.529000Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:46.529023Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:46.529026Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:46.529028Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:46.529039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342527194356672:4429]
2025-10-09T21:49:46.529045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342527194356672:4429]
2025-10-09T21:49:46.529049Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342527194356672:4429]
2025-10-09T21:49:46.529095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:46.529104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.529107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:46.529122Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:46.529128Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:46.529132Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:46.529245Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:46.533036Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1621 sessionId: src_id|3e999081-5f00fdcd-18c0ed11-6ac03d1b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:46.535710Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:46.576888Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342527556185098:7774]
2025-10-09T21:49:46.577145Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 1620, pipe: [1:7559342527556185098:7774]
2025-10-09T21:49:46.688317Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:46.688499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.688513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.688553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342527556185098:7774], now have 1 active actors on pipe
2025-10-09T21:49:46.688753Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:46.688779Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342527556185098:7774]
2025-10-09T21:49:46.688792Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:46.688793Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:46.688845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1621 reason: error at write: Disconnected sessionId: src_id|3e999081-5f00fdcd-18c0ed11-6ac03d1b_0
2025-10-09T21:49:46.689087Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1621 sessionId: src_id|3e999081-5f00fdcd-18c0ed11-6ac03d1b_0 is DEAD
2025-10-09T21:49:46.689171Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e999081-5f00fdcd-18c0ed11-6ac03d1b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:46.689193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e999081-5f00fdcd-18c0ed11-6ac03d1b_0] Write session will restart in 0.008349s
2025-10-09T21:49:46.689165Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:46.689215Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e999081-5f00fdcd-18c0ed11-6ac03d1b_0] Write session: Do CDS request
2025-10-09T21:49:46.689221Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e999081-5f00fdcd-18c0ed11-6ac03d1b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:46.698575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3e999081-5f00fdcd-18c0ed11-6ac03d1b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:46.698668Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:46.698683Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1622
2025-10-09T21:49:46.698819Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1622 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:46.698855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1622 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:46.698861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1622 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:46.698868Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:46.699451Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1622 sessionId:  describe result for acl check
2025-10-09T21:49:46.699485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:46.699487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:46.699488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:46.699494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342527556185214:7601] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:46.700070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342527556185214:7601] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:46.702556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342527556185214:7601] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:46.704348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:46.707327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:46.707649Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:46.707659Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:46.707672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:46.707713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342527556185214:7601] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:46.707724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:46.707729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342527556185214:7601] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:46.707733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342527556185214:7601] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:46.707742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1622 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:46.707766Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342527194356916:4402]
2025-10-09T21:49:46.708390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:46.708711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:46.708767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:46.709094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:46.709140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.709148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.709149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.709150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.709152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.709153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.709154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.709156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.709156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.709162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.709173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.709177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.709180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.709184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.709188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.709192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.709196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.709200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.709207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.709215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.709223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.709229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.709233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.709238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.709242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.709248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.709252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.709257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.709262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.709267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.709272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.709278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.709283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.709288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.709294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.709298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.709333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:46.715949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:46.716976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:46.716990Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:46.716992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:46.716996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:46.717001Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1621 [2:7559342527194356916:4402]
2025-10-09T21:49:46.717013Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:46.717033Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:46.717043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:46.717046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:46.717049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:46.717069Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:46.717073Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:46.717078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:46.717123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.717129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.717130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.717132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:46.717187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:46.717192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.717200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.717207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.717219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:46.717237Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342527194356911:4402]
2025-10-09T21:49:46.717242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342527194356911:4402]
2025-10-09T21:49:46.717243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342527194356911:4402]
2025-10-09T21:49:46.717249Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:46.717260Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:46.717261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342527556185327:7601], now have 1 active actors on pipe
2025-10-09T21:49:46.717267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342527556185125:2432], now have 1 active actors on pipe
2025-10-09T21:49:46.717271Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:46.717272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:46.717273Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:46.717285Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:46.717289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:46.717321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342527194356911:4402]
2025-10-09T21:49:46.717328Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:46.717339Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:46.717336Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1621
2025-10-09T21:49:46.717379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.717348Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1621
2025-10-09T21:49:46.717386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.717411Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b6fb9cef-4d70af2b-da3ea1c0-f589ed06_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:46.717421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:46.717424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:46.717425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:46.717433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:46.717440Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:46.717455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:46.717608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.717623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.717648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:46.717734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1622 partition: 0 MaxSeqNo: 25 sessionId: src_id|b6fb9cef-4d70af2b-da3ea1c0-f589ed06_0
2025-10-09T21:49:46.718097Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e999081-5f00fdcd-18c0ed11-6ac03d1b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b6fb9cef-4d70af2b-da3ea1c0-f589ed06_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:46.720412Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6fb9cef-4d70af2b-da3ea1c0-f589ed06_0] Write session: try to update token
2025-10-09T21:49:46.720428Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6fb9cef-4d70af2b-da3ea1c0-f589ed06_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:46.728997Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:46.729018Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:46.729027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342527194356911:4402]
2025-10-09T21:49:46.729073Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:46.729075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:46.729254Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:46.729395Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:46.729534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:46.729680Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:46.729875Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:46.729921Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:46.732921Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1622 sessionId: src_id|b6fb9cef-4d70af2b-da3ea1c0-f589ed06_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:46.735255Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:46.735488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.735510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.735549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:46.735623Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:46.743017Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:46.743038Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:46.743040Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:46.743042Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:46.743054Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342527194356911:4402]
2025-10-09T21:49:46.743058Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342527194356911:4402]
2025-10-09T21:49:46.743064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342527194356911:4402]
2025-10-09T21:49:46.743107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:46.743120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.743123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:46.743127Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:46.743132Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:46.743135Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:46.743303Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:46.789234Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342527556185343:7774]
2025-10-09T21:49:46.789533Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 1621, pipe: [1:7559342527556185343:7774]
2025-10-09T21:49:46.898914Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:46.906581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.906612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.906825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.906900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.906972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.907046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.907364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.907380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.907451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.907537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.907614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.907705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.907756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.907769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.907843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.907933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.908003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.908069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.908118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.908130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.908203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.908271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.908338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.908404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.908452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.908460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.908531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.908594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.908675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.908735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.908775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.908782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.908844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.908914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.908979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.909044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.909117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.909125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.909183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.909237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.909293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.909351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.909394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.909401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.909456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.909507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.909573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.909625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.909667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.909669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.909722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.909772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.909834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.909935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.909995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.910006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.910100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:46.910169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:46.910240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:46.910313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:46.910363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:46.910371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:46.910441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342527556185343:7774], now have 1 active actors on pipe
2025-10-09T21:49:46.910688Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:46.910710Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342527556185343:7774]
2025-10-09T21:49:46.910716Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:46.910732Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:46.910754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1622 reason: error at write: Disconnected sessionId: src_id|b6fb9cef-4d70af2b-da3ea1c0-f589ed06_0
2025-10-09T21:49:46.910994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1622 sessionId: src_id|b6fb9cef-4d70af2b-da3ea1c0-f589ed06_0 is DEAD
2025-10-09T21:49:46.911156Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6fb9cef-4d70af2b-da3ea1c0-f589ed06_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:46.911172Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6fb9cef-4d70af2b-da3ea1c0-f589ed06_0] Write session will restart in 0.009943s
2025-10-09T21:49:46.911197Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6fb9cef-4d70af2b-da3ea1c0-f589ed06_0] Write session: Do CDS request
2025-10-09T21:49:46.911206Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6fb9cef-4d70af2b-da3ea1c0-f589ed06_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:46.911375Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:46.922542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6fb9cef-4d70af2b-da3ea1c0-f589ed06_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:46.922646Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:46.922662Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1623
2025-10-09T21:49:46.922809Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1623 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:46.922842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1623 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:46.922852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1623 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:46.922856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:46.923369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1623 sessionId:  describe result for acl check
2025-10-09T21:49:46.923403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:46.923405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:46.923407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:46.923417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342527556185467:7592] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:46.924010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342527556185467:7592] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:46.926318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342527556185467:7592] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:46.930967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:46.931258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342527556185467:7592] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:46.931271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342527556185467:7592] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:46.931273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342527556185467:7592] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:46.931280Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1623 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:46.931402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:46.931576Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:46.931587Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:46.931599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:46.931807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:46.931853Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342527194357117:4397]
2025-10-09T21:49:46.932460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:46.932750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:46.932794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:46.933115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:46.933167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.933174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.933175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.933176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.933178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.933179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.933180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.933182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.933183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.933188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.933199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.933203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.933208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.933212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.933217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.933221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.933225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.933229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.933238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.933246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:46.933252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.933257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:46.933265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.933270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:46.933274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.933281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:46.933291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.933296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:46.933305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.933310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:46.933319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.933325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.933334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.933339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.933348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.933354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.933387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:46.940754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:46.941831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:46.941847Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:46.941850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:46.941854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:46.941861Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1622 [2:7559342527194357117:4397]
2025-10-09T21:49:46.941874Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:46.941892Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:46.941900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:46.941904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:46.941906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:46.941924Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:46.941934Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:46.941940Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:46.941995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.942002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.942004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.942007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:46.942096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:46.942110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.942123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:46.942138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:46.942162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:46.942188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342527556185555:7592], now have 1 active actors on pipe
2025-10-09T21:49:46.942196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342527556185348:2432], now have 1 active actors on pipe
2025-10-09T21:49:46.942203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342527194357111:4397]
2025-10-09T21:49:46.942206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342527194357111:4397]
2025-10-09T21:49:46.942208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342527194357111:4397]
2025-10-09T21:49:46.942223Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:46.942208Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1622
2025-10-09T21:49:46.942209Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1622
2025-10-09T21:49:46.942243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:46.942253Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:46.942257Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:46.942259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:46.942264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:46.942267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:46.942306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.942316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.942334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342527194357111:4397]
2025-10-09T21:49:46.942346Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:46.942358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:46.942379Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|daa3f759-e6831338-4bb23d78-7b98ca60_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:46.942394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:46.942403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:46.942406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:46.942416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:46.942430Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:46.942456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:46.942617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:46.942627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:46.942643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:46.942718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1623 partition: 0 MaxSeqNo: 25 sessionId: src_id|daa3f759-e6831338-4bb23d78-7b98ca60_0
2025-10-09T21:49:46.943057Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6fb9cef-4d70af2b-da3ea1c0-f589ed06_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|daa3f759-e6831338-4bb23d78-7b98ca60_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:46.945351Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|daa3f759-e6831338-4bb23d78-7b98ca60_0] Write session: try to update token
2025-10-09T21:49:46.945367Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|daa3f759-e6831338-4bb23d78-7b98ca60_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:46.958935Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1623 sessionId: src_id|daa3f759-e6831338-4bb23d78-7b98ca60_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:46.960799Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:46.960816Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:46.960818Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:46.960820Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:46.960847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342527194357111:4397]
2025-10-09T21:49:46.960852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342527194357111:4397]
2025-10-09T21:49:46.960856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342527194357111:4397]
2025-10-09T21:49:46.960900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:46.960902Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:46.960908Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:46.960912Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:46.960923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:46.960926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:46.961060Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:46.961319Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:47.010569Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342531851152862:7774]
2025-10-09T21:49:47.010820Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 1622, pipe: [1:7559342531851152862:7774]
2025-10-09T21:49:47.112236Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:47.112622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:47.112635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:47.112674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342531851152862:7774], now have 1 active actors on pipe
2025-10-09T21:49:47.112868Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:47.112919Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:47.112927Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342531851152862:7774]
2025-10-09T21:49:47.112933Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:47.112945Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:47.112991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1623 reason: error at write: Disconnected sessionId: src_id|daa3f759-e6831338-4bb23d78-7b98ca60_0
2025-10-09T21:49:47.113176Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1623 sessionId: src_id|daa3f759-e6831338-4bb23d78-7b98ca60_0 is DEAD
2025-10-09T21:49:47.113280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|daa3f759-e6831338-4bb23d78-7b98ca60_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:47.113289Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|daa3f759-e6831338-4bb23d78-7b98ca60_0] Write session will restart in 0.008462s
2025-10-09T21:49:47.113315Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|daa3f759-e6831338-4bb23d78-7b98ca60_0] Write session: Do CDS request
2025-10-09T21:49:47.113322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|daa3f759-e6831338-4bb23d78-7b98ca60_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:47.122408Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|daa3f759-e6831338-4bb23d78-7b98ca60_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:47.122515Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:47.122529Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1624
2025-10-09T21:49:47.122666Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1624 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:47.122701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1624 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:47.122709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1624 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:47.122713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:47.123154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1624 sessionId:  describe result for acl check
2025-10-09T21:49:47.123181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:47.123186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:47.123187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:47.123193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342531851152955:7588] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:47.123661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342531851152955:7588] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:47.126017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342531851152955:7588] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:47.128312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:47.128590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342531851152955:7588] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:47.128603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342531851152955:7588] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:47.128607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342531851152955:7588] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:47.128615Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1624 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:47.131694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:47.131933Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:47.131947Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:47.131960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:47.132021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:47.132093Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342531489324613:4392]
2025-10-09T21:49:47.132700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:47.133002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:47.133046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:47.133661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:47.133740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.133747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.133748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.133749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.133751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.133753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.133754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.133755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.133756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.133762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.133774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.133777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.133781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.133784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.133788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.133791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.133795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.133799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.133806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.133814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.133820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.133825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.133831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.133836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.133841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.133845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.133850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.133857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.133861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.133866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.133870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.133875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.133880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.133885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.133889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.133893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.133926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:47.141762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:47.142804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:47.142821Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:47.142822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:47.142826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:47.142832Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1623 [2:7559342531489324613:4392]
2025-10-09T21:49:47.142843Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:47.142858Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:47.142860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:47.142863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:47.142864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:47.142878Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:47.142882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:47.142888Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:47.142936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.142943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.142946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.142948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:47.143014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:47.143020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.143028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.143035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.143048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:47.143065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342531851152865:2432], now have 1 active actors on pipe
2025-10-09T21:49:47.143074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342531851153073:7588], now have 1 active actors on pipe
2025-10-09T21:49:47.143080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342531489324607:4392]
2025-10-09T21:49:47.143082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342531489324607:4392]
2025-10-09T21:49:47.143084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342531489324607:4392]
2025-10-09T21:49:47.143093Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:47.143104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:47.143106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:47.143110Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:47.143112Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:47.143112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:47.143114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:47.143153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342531489324607:4392]
2025-10-09T21:49:47.143161Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:47.143166Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:47.143173Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1623
2025-10-09T21:49:47.143180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1623
2025-10-09T21:49:47.143236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:47.143246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:47.143272Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a673fd1f-1c48ecc6-5652f1db-7958a439_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:47.143280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:47.143284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:47.143286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:47.143293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:47.143301Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:47.143312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:47.143475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:47.143488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:47.143511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:47.143603Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1624 partition: 0 MaxSeqNo: 25 sessionId: src_id|a673fd1f-1c48ecc6-5652f1db-7958a439_0
2025-10-09T21:49:47.143960Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|daa3f759-e6831338-4bb23d78-7b98ca60_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a673fd1f-1c48ecc6-5652f1db-7958a439_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:47.146580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a673fd1f-1c48ecc6-5652f1db-7958a439_0] Write session: try to update token
2025-10-09T21:49:47.146600Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a673fd1f-1c48ecc6-5652f1db-7958a439_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:47.160122Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1624 sessionId: src_id|a673fd1f-1c48ecc6-5652f1db-7958a439_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:47.162483Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:47.162865Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:47.162883Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:47.162886Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:47.162887Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:47.162896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342531489324607:4392]
2025-10-09T21:49:47.162903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342531489324607:4392]
2025-10-09T21:49:47.162907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342531489324607:4392]
2025-10-09T21:49:47.162948Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:47.162958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:47.162958Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:47.162961Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:47.162967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:47.162987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:47.162998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.163006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:47.163136Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:47.213334Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342531851153087:7774]
2025-10-09T21:49:47.213620Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 1623, pipe: [1:7559342531851153087:7774]
2025-10-09T21:49:47.315053Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:47.315411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342531851153087:7774], now have 1 active actors on pipe
2025-10-09T21:49:47.315609Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:47.315631Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:47.315633Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:47.315637Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:47.315696Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342531851153087:7774]
2025-10-09T21:49:47.315697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1624 reason: error at write: Disconnected sessionId: src_id|a673fd1f-1c48ecc6-5652f1db-7958a439_0
2025-10-09T21:49:47.315872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1624 sessionId: src_id|a673fd1f-1c48ecc6-5652f1db-7958a439_0 is DEAD
2025-10-09T21:49:47.316044Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a673fd1f-1c48ecc6-5652f1db-7958a439_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:47.316055Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a673fd1f-1c48ecc6-5652f1db-7958a439_0] Write session will restart in 0.009009s
2025-10-09T21:49:47.316073Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a673fd1f-1c48ecc6-5652f1db-7958a439_0] Write session: Do CDS request
2025-10-09T21:49:47.316077Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a673fd1f-1c48ecc6-5652f1db-7958a439_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:47.326514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a673fd1f-1c48ecc6-5652f1db-7958a439_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:47.326582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:47.326596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1625
2025-10-09T21:49:47.326747Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1625 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:47.326782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1625 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:47.326788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1625 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:47.326796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:47.327275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1625 sessionId:  describe result for acl check
2025-10-09T21:49:47.327305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:47.327313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:47.327315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:47.327322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342531851153257:7579] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:47.327864Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342531851153257:7579] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:47.329863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342531851153257:7579] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:47.332800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:47.332815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342531851153257:7579] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:47.332829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342531851153257:7579] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:47.332832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342531851153257:7579] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:47.332840Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1625 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:47.333319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:47.333508Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:47.333520Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:47.333534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:47.333630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:47.333672Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342531489324814:4390]
2025-10-09T21:49:47.334287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:47.334575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:47.334625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:47.334936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:47.335013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.335022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.335024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.335025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.335026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.335028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.335029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.335030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.335032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.335038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.335049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.335054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.335059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.335063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.335067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.335072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.335076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.335081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.335089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.335103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.335112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.335123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.335127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.335138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.335144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.335154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.335159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.335164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.335174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.335180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.335185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.335196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.335205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.335215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.335224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.335230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.335263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:47.343278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:47.344387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:47.344404Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:47.344407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:47.344412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:47.344420Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1624 [2:7559342531489324814:4390]
2025-10-09T21:49:47.344435Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:47.344460Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:47.344469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:47.344472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:47.344474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:47.344494Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:47.344509Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:47.344518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:47.344598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.344607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.344610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.344613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:47.344686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:47.344695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.344708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.344725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.344748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:47.344777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342531851153303:7579], now have 1 active actors on pipe
2025-10-09T21:49:47.344795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342531851153095:2432], now have 1 active actors on pipe
2025-10-09T21:49:47.344809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342531489324808:4390]
2025-10-09T21:49:47.344816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342531489324808:4390]
2025-10-09T21:49:47.344819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342531489324808:4390]
2025-10-09T21:49:47.344834Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:47.344851Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:47.344854Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:47.344830Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1624
2025-10-09T21:49:47.344856Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:47.344859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:47.344831Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1624
2025-10-09T21:49:47.344877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:47.344881Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:47.344936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:47.344947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:47.344966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342531489324808:4390]
2025-10-09T21:49:47.344980Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:47.344991Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:47.345015Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b51b77e6-b6ec7cf1-6d472740-24204e94_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:47.345028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:47.345038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:47.345045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:47.345058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:47.345075Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:47.345110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:47.345262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:47.345269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:47.345284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:47.345373Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1625 partition: 0 MaxSeqNo: 25 sessionId: src_id|b51b77e6-b6ec7cf1-6d472740-24204e94_0
2025-10-09T21:49:47.345762Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a673fd1f-1c48ecc6-5652f1db-7958a439_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b51b77e6-b6ec7cf1-6d472740-24204e94_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:47.348179Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b51b77e6-b6ec7cf1-6d472740-24204e94_0] Write session: try to update token
2025-10-09T21:49:47.348195Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b51b77e6-b6ec7cf1-6d472740-24204e94_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:47.361405Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1625 sessionId: src_id|b51b77e6-b6ec7cf1-6d472740-24204e94_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:47.362832Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:47.362854Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:47.362856Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:47.362857Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:47.362865Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342531489324808:4390]
2025-10-09T21:49:47.362868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342531489324808:4390]
2025-10-09T21:49:47.362871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342531489324808:4390]
2025-10-09T21:49:47.362913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:47.362919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.362921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:47.363043Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:47.363126Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:47.363141Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:47.363145Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:47.363858Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:47.416072Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342531851153316:7774]
2025-10-09T21:49:47.416326Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 1624, pipe: [1:7559342531851153316:7774]
2025-10-09T21:49:47.517637Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:47.518066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:47.518079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:47.518118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342531851153316:7774], now have 1 active actors on pipe
2025-10-09T21:49:47.518335Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:47.518342Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342531851153316:7774]
2025-10-09T21:49:47.518356Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:47.518376Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:47.518379Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:47.518444Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1625 reason: error at write: Disconnected sessionId: src_id|b51b77e6-b6ec7cf1-6d472740-24204e94_0
2025-10-09T21:49:47.518614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1625 sessionId: src_id|b51b77e6-b6ec7cf1-6d472740-24204e94_0 is DEAD
2025-10-09T21:49:47.518746Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b51b77e6-b6ec7cf1-6d472740-24204e94_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:47.518758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b51b77e6-b6ec7cf1-6d472740-24204e94_0] Write session will restart in 0.006285s
2025-10-09T21:49:47.518785Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b51b77e6-b6ec7cf1-6d472740-24204e94_0] Write session: Do CDS request
2025-10-09T21:49:47.518792Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b51b77e6-b6ec7cf1-6d472740-24204e94_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:47.526577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b51b77e6-b6ec7cf1-6d472740-24204e94_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:47.526621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:47.526635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1626
2025-10-09T21:49:47.526795Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1626 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:47.526828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1626 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:47.526838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1626 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:47.526856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:47.527301Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1626 sessionId:  describe result for acl check
2025-10-09T21:49:47.527338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:47.527344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:47.527346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:47.527352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342531851153367:7574] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:47.527934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342531851153367:7574] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:47.530504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342531851153367:7574] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:47.532787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342531851153367:7574] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:47.532813Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342531851153367:7574] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:47.532817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342531851153367:7574] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:47.532824Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1626 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:47.533449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:47.536448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:47.536636Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:47.536646Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:47.536657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:47.536709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:47.536748Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342531489325016:4381]
2025-10-09T21:49:47.537123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:47.537443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:47.537475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:47.537693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:47.537731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.537737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.537738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.537739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.537740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.537741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.537742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.537743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.537744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.537749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.537760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.537763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.537767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.537771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.537775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.537779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.537783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.537786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.537791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.537796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.537800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.537804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.537807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.537810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.537813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.537816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.537819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.537822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.537824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.537827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.537830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.537833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.537835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.537838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.537841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.537844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.537860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:47.545052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:47.546506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:47.546521Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:47.546522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:47.546525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:47.546530Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1625 [2:7559342531489325016:4381]
2025-10-09T21:49:47.546540Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:47.546555Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:47.546557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:47.546559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:47.546560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:47.546573Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:47.546577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:47.546582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:47.546629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.546637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.546638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.546640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:47.546703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:47.546711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.546723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.546734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.546748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:47.546773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342531489325011:4381]
2025-10-09T21:49:47.546780Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342531489325011:4381]
2025-10-09T21:49:47.546782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342531489325011:4381]
2025-10-09T21:49:47.546795Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:47.546805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342531851153318:2432], now have 1 active actors on pipe
2025-10-09T21:49:47.546808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342531851153527:7574], now have 1 active actors on pipe
2025-10-09T21:49:47.546812Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:47.546815Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:47.546816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:47.546817Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:47.546825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:47.546827Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:47.546852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342531489325011:4381]
2025-10-09T21:49:47.546857Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:47.546866Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:47.546877Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1625
2025-10-09T21:49:47.546885Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1625
2025-10-09T21:49:47.546927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:47.546933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:47.546955Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ad3aac6d-272eda04-88903c95-315c0931_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:47.546967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:47.546969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:47.546971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:47.546980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:47.546987Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:47.547001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:47.547117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:47.547122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:47.547132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:47.547211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1626 partition: 0 MaxSeqNo: 25 sessionId: src_id|ad3aac6d-272eda04-88903c95-315c0931_0
2025-10-09T21:49:47.547560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b51b77e6-b6ec7cf1-6d472740-24204e94_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ad3aac6d-272eda04-88903c95-315c0931_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:47.549904Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad3aac6d-272eda04-88903c95-315c0931_0] Write session: try to update token
2025-10-09T21:49:47.549919Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad3aac6d-272eda04-88903c95-315c0931_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:47.562283Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1626 sessionId: src_id|ad3aac6d-272eda04-88903c95-315c0931_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:47.563572Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:47.563591Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:47.563593Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:47.563594Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:47.563602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342531489325011:4381]
2025-10-09T21:49:47.563605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342531489325011:4381]
2025-10-09T21:49:47.563608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342531489325011:4381]
2025-10-09T21:49:47.563641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:47.563653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.563655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:47.563664Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:47.563671Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:47.563678Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:47.563807Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:47.564584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:47.618306Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342531851153540:7774]
2025-10-09T21:49:47.618555Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 1625, pipe: [1:7559342531851153540:7774]
2025-10-09T21:49:47.726674Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:47.727102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:47.727121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:47.727164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342531851153540:7774], now have 1 active actors on pipe
2025-10-09T21:49:47.727351Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:47.727393Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:47.727394Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342531851153540:7774]
2025-10-09T21:49:47.727413Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:47.727487Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1626 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ad3aac6d-272eda04-88903c95-315c0931_0
2025-10-09T21:49:47.727660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1626 sessionId: src_id|ad3aac6d-272eda04-88903c95-315c0931_0 is DEAD
2025-10-09T21:49:47.727863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad3aac6d-272eda04-88903c95-315c0931_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:47.727876Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad3aac6d-272eda04-88903c95-315c0931_0] Write session will restart in 0.007077s
2025-10-09T21:49:47.727904Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad3aac6d-272eda04-88903c95-315c0931_0] Write session: Do CDS request
2025-10-09T21:49:47.727912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad3aac6d-272eda04-88903c95-315c0931_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:47.736566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad3aac6d-272eda04-88903c95-315c0931_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:47.736643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:47.736658Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1627
2025-10-09T21:49:47.737154Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1627 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:47.737193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1627 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:47.737204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1627 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:47.737207Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:47.737663Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1627 sessionId:  describe result for acl check
2025-10-09T21:49:47.737690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:47.737696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:47.737698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:47.737703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342531851153658:7551] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:47.738207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342531851153658:7551] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:47.740389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342531851153658:7551] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:47.745481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:47.745620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342531851153658:7551] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:47.745635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342531851153658:7551] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:47.745639Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342531851153658:7551] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:47.745648Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1627 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:47.745982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:47.746129Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:47.746136Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:47.746145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:47.746185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:47.746223Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342531489325262:4360]
2025-10-09T21:49:47.746608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:47.746918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:47.746976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:47.747282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:47.747338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.747339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.747340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.747342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.747343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.747344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.747345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.747346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.747348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.747353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.747364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.747368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.747372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.747375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.747379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.747383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.747386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.747389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.747397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.747406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.747413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.747418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.747423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.747428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.747432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.747437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.747442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.747447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.747451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.747457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.747461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.747467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.747471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.747476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.747481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.747486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.747516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:47.754276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:47.755710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:47.755723Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:47.755725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:47.755728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:47.755733Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1626 [2:7559342531489325262:4360]
2025-10-09T21:49:47.755743Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:47.755760Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:47.755762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:47.755767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:47.755769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:47.755783Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:47.755787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:47.755794Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:47.755852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.755859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.755862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.755865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:47.755926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:47.755940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.755949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.755955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.755965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:47.755990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342531489325256:4360]
2025-10-09T21:49:47.755997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342531489325256:4360]
2025-10-09T21:49:47.755999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342531489325256:4360]
2025-10-09T21:49:47.756007Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:47.756018Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:47.756023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:47.756028Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:47.756031Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:47.756039Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:47.756041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:47.756053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342531851153776:7551], now have 1 active actors on pipe
2025-10-09T21:49:47.756061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342531851153568:2432], now have 1 active actors on pipe
2025-10-09T21:49:47.756093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342531489325256:4360]
2025-10-09T21:49:47.756104Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:47.756094Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1626
2025-10-09T21:49:47.756127Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:47.756107Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1626
2025-10-09T21:49:47.756175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:47.756185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:47.756211Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f13b516-cb798749-7d3ca157-f81380e5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:47.756222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:47.756226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:47.756229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:47.756238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:47.756252Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:47.756265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:47.756401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:47.756409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:47.756424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:47.756526Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1627 partition: 0 MaxSeqNo: 25 sessionId: src_id|f13b516-cb798749-7d3ca157-f81380e5_0
2025-10-09T21:49:47.756904Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad3aac6d-272eda04-88903c95-315c0931_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f13b516-cb798749-7d3ca157-f81380e5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:47.759252Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f13b516-cb798749-7d3ca157-f81380e5_0] Write session: try to update token
2025-10-09T21:49:47.759268Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f13b516-cb798749-7d3ca157-f81380e5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:47.771708Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:47.771730Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:47.771732Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:47.771734Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:47.771744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342531489325256:4360]
2025-10-09T21:49:47.771748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342531489325256:4360]
2025-10-09T21:49:47.771751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342531489325256:4360]
2025-10-09T21:49:47.771787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:47.771793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.771795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:47.771826Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:47.771849Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:47.771856Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:47.771909Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:47.772479Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1627 sessionId: src_id|f13b516-cb798749-7d3ca157-f81380e5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:47.774788Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:47.827708Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342531851153787:7774]
2025-10-09T21:49:47.827933Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 1626, pipe: [1:7559342531851153787:7774]
2025-10-09T21:49:47.920734Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:47.921157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:47.921174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:47.921221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342531851153787:7774], now have 1 active actors on pipe
2025-10-09T21:49:47.921480Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:47.921493Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:47.921517Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:47.921567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1627 reason: error at write: Disconnected sessionId: src_id|f13b516-cb798749-7d3ca157-f81380e5_0
2025-10-09T21:49:47.921570Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342531851153787:7774]
2025-10-09T21:49:47.921609Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:47.921743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1627 sessionId: src_id|f13b516-cb798749-7d3ca157-f81380e5_0 is DEAD
2025-10-09T21:49:47.921871Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f13b516-cb798749-7d3ca157-f81380e5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:47.921882Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f13b516-cb798749-7d3ca157-f81380e5_0] Write session will restart in 0.009657s
2025-10-09T21:49:47.921903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f13b516-cb798749-7d3ca157-f81380e5_0] Write session: Do CDS request
2025-10-09T21:49:47.921907Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f13b516-cb798749-7d3ca157-f81380e5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:47.932503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f13b516-cb798749-7d3ca157-f81380e5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:47.932573Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:47.932583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1628
2025-10-09T21:49:47.932702Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1628 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:47.932725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1628 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:47.932734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1628 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:47.932737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:47.933143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1628 sessionId:  describe result for acl check
2025-10-09T21:49:47.933180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:47.933186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:47.933188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:47.933194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342531851153967:7548] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:47.933618Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342531851153967:7548] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:47.934136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:47.935271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342531851153967:7548] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:47.937008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:47.937172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342531851153967:7548] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:47.937182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342531851153967:7548] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:47.937184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342531851153967:7548] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:47.937202Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:47.937189Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1628 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:47.937215Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:47.937240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:47.937309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:47.937345Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342531489325465:4357]
2025-10-09T21:49:47.937907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:47.938228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:47.938272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:47.938513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:47.938546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.938550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.938551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.938552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.938552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.938553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.938553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.938554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.938555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.938558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.938564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.938566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.938568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.938570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.938574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.938577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.938580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.938584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.938589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.938597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:47.938607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.938616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:47.938622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.938626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:47.938631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.938639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:47.938643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.938648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:47.938651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.938656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:47.938661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.938666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.938674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.938679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.938683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.938690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.938718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:47.944636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:47.945667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:47.945678Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:47.945680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:47.945682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:47.945687Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1627 [2:7559342531489325465:4357]
2025-10-09T21:49:47.945696Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:47.945708Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:47.945713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:47.945715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:47.945716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:47.945728Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:47.945735Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:47.945741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:47.945781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.945785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.945787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.945788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:47.945843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:47.945849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.945856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:47.945862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:47.945871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:47.945885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342531851154004:7548], now have 1 active actors on pipe
2025-10-09T21:49:47.945890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342531489325460:4357]
2025-10-09T21:49:47.945892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342531489325460:4357]
2025-10-09T21:49:47.945893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342531489325460:4357]
2025-10-09T21:49:47.945900Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:47.945906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342531851153792:2432], now have 1 active actors on pipe
2025-10-09T21:49:47.945912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:47.945914Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:47.945916Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:47.945920Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:47.945920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:47.945922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:47.945937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342531489325460:4357]
2025-10-09T21:49:47.945943Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:47.945945Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:47.945952Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1627
2025-10-09T21:49:47.945953Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1627
2025-10-09T21:49:47.945983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:47.945987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:47.946007Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|75a0b2f1-d777b65e-c9852769-729475ad_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:47.946016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:47.946019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:47.946020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:47.946026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:47.946059Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:47.946074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:47.946238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:47.946249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:47.946267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:47.946374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1628 partition: 0 MaxSeqNo: 25 sessionId: src_id|75a0b2f1-d777b65e-c9852769-729475ad_0
2025-10-09T21:49:47.946672Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f13b516-cb798749-7d3ca157-f81380e5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|75a0b2f1-d777b65e-c9852769-729475ad_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:47.949018Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|75a0b2f1-d777b65e-c9852769-729475ad_0] Write session: try to update token
2025-10-09T21:49:47.949033Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|75a0b2f1-d777b65e-c9852769-729475ad_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:47.961670Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:47.961684Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:47.961692Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342531489325460:4357]
2025-10-09T21:49:47.961732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:47.961769Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:47.961869Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:47.961996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:47.962182Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:47.962302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:47.962474Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:47.962510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:47.962903Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1628 sessionId: src_id|75a0b2f1-d777b65e-c9852769-729475ad_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:47.965334Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:47.965496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:47.965514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:47.965570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:47.965653Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:47.973376Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:47.973392Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:47.973394Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:47.973396Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:47.973406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342531489325460:4357]
2025-10-09T21:49:47.973411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342531489325460:4357]
2025-10-09T21:49:47.973418Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342531489325460:4357]
2025-10-09T21:49:47.973459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:47.973461Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:47.973468Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:47.973471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:47.973474Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:47.973474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:47.973607Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:48.021761Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342536146121311:7774]
2025-10-09T21:49:48.022139Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 1627, pipe: [1:7559342536146121311:7774]
2025-10-09T21:49:48.127143Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:48.142668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:48.142701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:48.142910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.142971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.143039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.143102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.143147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.143165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.143237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.143297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.143367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.143439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.143481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.143489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.143549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.143608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.143662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.143718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.143834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.143841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.143910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.143979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.144049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.144113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.144156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.144163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.144237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.144339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.144402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.144467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.144507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.144514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.144577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.144638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.144702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.144764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.144808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.144815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.144917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.144978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.145036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.145096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.145140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.145150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.145211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.145269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.145328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.145390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.145447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.145454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.145514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.145580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.145637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.145700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.145744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.145751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.145815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.145874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.145934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.145996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.146055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.146064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.146131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342536146121311:7774], now have 1 active actors on pipe
2025-10-09T21:49:48.146389Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:48.146404Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342536146121311:7774]
2025-10-09T21:49:48.146410Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:48.146442Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:48.146456Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1628 reason: error at write: Disconnected sessionId: src_id|75a0b2f1-d777b65e-c9852769-729475ad_0
2025-10-09T21:49:48.146662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1628 sessionId: src_id|75a0b2f1-d777b65e-c9852769-729475ad_0 is DEAD
2025-10-09T21:49:48.146824Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|75a0b2f1-d777b65e-c9852769-729475ad_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:48.146836Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|75a0b2f1-d777b65e-c9852769-729475ad_0] Write session will restart in 0.008864s
2025-10-09T21:49:48.146864Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|75a0b2f1-d777b65e-c9852769-729475ad_0] Write session: Do CDS request
2025-10-09T21:49:48.146870Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|75a0b2f1-d777b65e-c9852769-729475ad_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:48.147430Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:48.156525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|75a0b2f1-d777b65e-c9852769-729475ad_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:48.156676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:48.156692Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1629
2025-10-09T21:49:48.156811Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1629 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:48.156858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1629 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:48.156868Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1629 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:48.156871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:48.157306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1629 sessionId:  describe result for acl check
2025-10-09T21:49:48.157341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:48.157349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:48.157351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:48.157357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342536146121413:7542] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:48.157951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342536146121413:7542] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:48.160939Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342536146121413:7542] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:48.162161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:48.164873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:48.165160Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:48.165175Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:48.165189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:48.165164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342536146121413:7542] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:48.165175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342536146121413:7542] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:48.165177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342536146121413:7542] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:48.165184Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1629 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:48.165251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:48.165291Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342535784292966:4349]
2025-10-09T21:49:48.165921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:48.166256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:48.166309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:48.166640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:48.166709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.166716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.166718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.166719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.166720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.166721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.166723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.166724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.166725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.166731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.166741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.166745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.166749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.166753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.166757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.166760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.166764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.166767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.166774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.166782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.166788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.166793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.166803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.166807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.166811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.166816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.166821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.166826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.166835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.166840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.166849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.166853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.166862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.166866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.166870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.166880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.166917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:48.177575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:48.178617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:48.178633Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:48.178635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:48.178639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:48.178649Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1628 [2:7559342535784292966:4349]
2025-10-09T21:49:48.178661Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:48.178685Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:48.178687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:48.178690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:48.178692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:48.178708Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:48.178715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:48.178724Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:48.178777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.178784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.178785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.178788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:48.178849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:48.178856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.178865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.178871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.178881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:48.178896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342535784292961:4349]
2025-10-09T21:49:48.178898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342535784292961:4349]
2025-10-09T21:49:48.178899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342535784292961:4349]
2025-10-09T21:49:48.178911Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:48.178923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:48.178923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:48.178927Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:48.178930Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:48.178933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:48.178935Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:48.178945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342536146121314:2432], now have 1 active actors on pipe
2025-10-09T21:49:48.178948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342536146121527:7542], now have 1 active actors on pipe
2025-10-09T21:49:48.178985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342535784292961:4349]
2025-10-09T21:49:48.178998Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:48.179004Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:48.179028Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1628
2025-10-09T21:49:48.179041Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1628
2025-10-09T21:49:48.179153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:48.179161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:48.179186Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5e6d1d4a-1e5d53bc-2a45a690-c752845b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:48.179197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:48.179200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:48.179201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:48.179209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:48.179218Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:48.179240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:48.179385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:48.179392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:48.179406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:48.179487Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1629 partition: 0 MaxSeqNo: 25 sessionId: src_id|5e6d1d4a-1e5d53bc-2a45a690-c752845b_0
2025-10-09T21:49:48.179877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|75a0b2f1-d777b65e-c9852769-729475ad_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5e6d1d4a-1e5d53bc-2a45a690-c752845b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:48.186260Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5e6d1d4a-1e5d53bc-2a45a690-c752845b_0] Write session: try to update token
2025-10-09T21:49:48.186281Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5e6d1d4a-1e5d53bc-2a45a690-c752845b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:48.191472Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:48.191499Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:48.191511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342535784292961:4349]
2025-10-09T21:49:48.191566Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:48.191568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:48.191808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:48.191992Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:48.192174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:48.192351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:48.192614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:48.192670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:48.201008Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1629 sessionId: src_id|5e6d1d4a-1e5d53bc-2a45a690-c752845b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:48.206893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:48.211235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:48.211263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:48.211286Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:48.211290Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:48.211292Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:48.211294Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:48.211304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342535784292961:4349]
2025-10-09T21:49:48.211308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342535784292961:4349]
2025-10-09T21:49:48.211311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342535784292961:4349]
2025-10-09T21:49:48.211363Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:48.211372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:48.211377Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:48.211380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.211381Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:48.211383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:48.211519Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:48.246487Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342536146121542:7774]
2025-10-09T21:49:48.246785Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 1628, pipe: [1:7559342536146121542:7774]
2025-10-09T21:49:48.358936Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:48.359353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:48.359387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342536146121542:7774], now have 1 active actors on pipe
2025-10-09T21:49:48.359488Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:48.359576Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342536146121542:7774]
2025-10-09T21:49:48.359620Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:48.368782Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:48.368794Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:48.368818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1629 reason: error at write: Disconnected sessionId: src_id|5e6d1d4a-1e5d53bc-2a45a690-c752845b_0
2025-10-09T21:49:48.368947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1629 sessionId: src_id|5e6d1d4a-1e5d53bc-2a45a690-c752845b_0 is DEAD
2025-10-09T21:49:48.369086Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e6d1d4a-1e5d53bc-2a45a690-c752845b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:48.369097Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e6d1d4a-1e5d53bc-2a45a690-c752845b_0] Write session will restart in 0.009011s
2025-10-09T21:49:48.369125Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e6d1d4a-1e5d53bc-2a45a690-c752845b_0] Write session: Do CDS request
2025-10-09T21:49:48.369133Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e6d1d4a-1e5d53bc-2a45a690-c752845b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:48.378662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:48.379564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5e6d1d4a-1e5d53bc-2a45a690-c752845b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:48.379592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:48.379603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1630
2025-10-09T21:49:48.379726Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1630 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:48.379750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1630 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:48.379757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1630 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:48.379760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:48.380089Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1630 sessionId:  describe result for acl check
2025-10-09T21:49:48.380115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:48.380116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:48.380117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:48.380122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342536146121729:7534] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:48.380602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342536146121729:7534] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:48.381347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:48.381567Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:48.381578Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:48.381592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:48.381659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:48.381697Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342535784293172:4347]
2025-10-09T21:49:48.382296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:48.382574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:48.382620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:48.382921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:48.382977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.382984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.382985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.382986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.382987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.382988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.382989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.382990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.382991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.382996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.383005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.383009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.383012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.383015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.383019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.383023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.383027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.383030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.383038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.383046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.383052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.383057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.383061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.383067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.383071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.383075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.383079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.383086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.383090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.383094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.383101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.383106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.383115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.383120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.383125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.383133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.383163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:48.383360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342536146121729:7534] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:48.389136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342536146121729:7534] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:48.389150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342536146121729:7534] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:48.389155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342536146121729:7534] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:48.389165Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1630 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:48.389505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:48.390934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:48.390949Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:48.390951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:48.390954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:48.390960Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1629 [2:7559342535784293172:4347]
2025-10-09T21:49:48.390975Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:48.390994Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:48.391006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:48.391010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:48.391013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:48.391034Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:48.391049Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:48.391058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:48.391370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.391387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.391389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.391391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:48.391473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:48.391483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.391495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.391510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.391526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:48.391550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342536146121768:7534], now have 1 active actors on pipe
2025-10-09T21:49:48.391554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342536146121549:2432], now have 1 active actors on pipe
2025-10-09T21:49:48.391562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342535784293167:4347]
2025-10-09T21:49:48.391564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342535784293167:4347]
2025-10-09T21:49:48.391566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342535784293167:4347]
2025-10-09T21:49:48.391580Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:48.391596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:48.391606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:48.391610Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:48.391612Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:48.391616Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:48.391621Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:48.391604Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1629
2025-10-09T21:49:48.391626Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1629
2025-10-09T21:49:48.391653Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342535784293167:4347]
2025-10-09T21:49:48.391657Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:48.391659Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:48.391674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:48.391679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:48.391734Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2c6a1351-c270b195-d0ff0f6e-602f40bf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:48.391748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:48.391753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:48.391758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:48.391768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:48.391783Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:48.391807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:48.391966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:48.391975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:48.391990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:48.392096Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1630 partition: 0 MaxSeqNo: 25 sessionId: src_id|2c6a1351-c270b195-d0ff0f6e-602f40bf_0
2025-10-09T21:49:48.392452Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e6d1d4a-1e5d53bc-2a45a690-c752845b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2c6a1351-c270b195-d0ff0f6e-602f40bf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:48.395125Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c6a1351-c270b195-d0ff0f6e-602f40bf_0] Write session: try to update token
2025-10-09T21:49:48.395142Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c6a1351-c270b195-d0ff0f6e-602f40bf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:48.405805Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:48.405823Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:48.405826Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:48.405828Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:48.405838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342535784293167:4347]
2025-10-09T21:49:48.405843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342535784293167:4347]
2025-10-09T21:49:48.405847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342535784293167:4347]
2025-10-09T21:49:48.405890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:48.405894Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:48.405897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.405900Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:48.405900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:48.405904Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:48.406054Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:48.408001Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1630 sessionId: src_id|2c6a1351-c270b195-d0ff0f6e-602f40bf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:48.410581Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:48.460034Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342536146121775:7774]
2025-10-09T21:49:48.460319Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 1629, pipe: [1:7559342536146121775:7774]
2025-10-09T21:49:48.551748Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:48.551976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:48.551988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:48.552025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342536146121775:7774], now have 1 active actors on pipe
2025-10-09T21:49:48.552259Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:48.552246Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342536146121775:7774]
2025-10-09T21:49:48.552248Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:48.552252Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:48.552282Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:48.552323Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1630 reason: error at write: Disconnected sessionId: src_id|2c6a1351-c270b195-d0ff0f6e-602f40bf_0
2025-10-09T21:49:48.552509Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1630 sessionId: src_id|2c6a1351-c270b195-d0ff0f6e-602f40bf_0 is DEAD
2025-10-09T21:49:48.552668Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c6a1351-c270b195-d0ff0f6e-602f40bf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:48.552680Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c6a1351-c270b195-d0ff0f6e-602f40bf_0] Write session will restart in 0.005126s
2025-10-09T21:49:48.552708Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c6a1351-c270b195-d0ff0f6e-602f40bf_0] Write session: Do CDS request
2025-10-09T21:49:48.552715Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c6a1351-c270b195-d0ff0f6e-602f40bf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:48.558481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c6a1351-c270b195-d0ff0f6e-602f40bf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:48.558559Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:48.558569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1631
2025-10-09T21:49:48.558737Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1631 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:48.558779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1631 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:48.558789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1631 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:48.558793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:48.559257Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1631 sessionId:  describe result for acl check
2025-10-09T21:49:48.559301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:48.559307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:48.559309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:48.559315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342536146121795:7531] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:48.559868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342536146121795:7531] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:48.562583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342536146121795:7531] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:48.565119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342536146121795:7531] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:48.565131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342536146121795:7531] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:48.565135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342536146121795:7531] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:48.565143Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1631 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:48.571022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:48.571540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:48.571873Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:48.571885Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:48.571898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:48.571953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:48.571994Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342535784293381:4344]
2025-10-09T21:49:48.572562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:48.572887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:48.572939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:48.573266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:48.573313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.573318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.573319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.573320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.573321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.573323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.573325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.573326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.573327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.573332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.573343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.573347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.573351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.573354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.573358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.573361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.573365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.573368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.573375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.573383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.573389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.573395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.573400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.573406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.573413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.573418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.573422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.573428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.573437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.573443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.573448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.573453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.573457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.573462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.573468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.573475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.573503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:48.579674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:48.580676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:48.580692Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:48.580694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:48.580698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:48.580708Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1630 [2:7559342535784293381:4344]
2025-10-09T21:49:48.580721Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:48.580740Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:48.580743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:48.580746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:48.580748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:48.580763Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:48.580773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:48.580781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:48.580832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.580839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.580841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.580843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:48.580908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:48.580916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.580927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.580937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.580954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:48.580978Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342535784293376:4344]
2025-10-09T21:49:48.580985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342535784293376:4344]
2025-10-09T21:49:48.580987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342535784293376:4344]
2025-10-09T21:49:48.581000Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:48.581018Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:48.581021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342536146121778:2432], now have 1 active actors on pipe
2025-10-09T21:49:48.581025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342536146121978:7531], now have 1 active actors on pipe
2025-10-09T21:49:48.581028Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:48.581030Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:48.581033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:48.581048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:48.581051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:48.581076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342535784293376:4344]
2025-10-09T21:49:48.581092Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:48.581075Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1630
2025-10-09T21:49:48.581079Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1630
2025-10-09T21:49:48.581122Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:48.581166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:48.581178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:48.581206Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d03ea725-75330869-c8cc13c1-a8246ca7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:48.581220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:48.581229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:48.581232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:48.581241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:48.581252Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:48.581273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:48.581432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:48.581444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:48.581459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:48.581557Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1631 partition: 0 MaxSeqNo: 25 sessionId: src_id|d03ea725-75330869-c8cc13c1-a8246ca7_0
2025-10-09T21:49:48.581922Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c6a1351-c270b195-d0ff0f6e-602f40bf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d03ea725-75330869-c8cc13c1-a8246ca7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:48.584553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d03ea725-75330869-c8cc13c1-a8246ca7_0] Write session: try to update token
2025-10-09T21:49:48.584573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d03ea725-75330869-c8cc13c1-a8246ca7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:48.591693Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:48.591711Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:48.591720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342535784293376:4344]
2025-10-09T21:49:48.591765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:48.591775Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:48.591931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:48.592066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:48.592194Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:48.592319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:48.592491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:48.592529Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:48.596775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1631 sessionId: src_id|d03ea725-75330869-c8cc13c1-a8246ca7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:48.599283Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:48.599492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:48.599514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:48.599558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:48.599661Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:48.604695Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:48.604720Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:48.604722Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:48.604724Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:48.604734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342535784293376:4344]
2025-10-09T21:49:48.604738Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342535784293376:4344]
2025-10-09T21:49:48.604742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342535784293376:4344]
2025-10-09T21:49:48.604784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:48.604788Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:48.604790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.604793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:48.604794Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:48.604799Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:48.604925Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:48.652573Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342536146122004:7774]
2025-10-09T21:49:48.652911Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 1630, pipe: [1:7559342536146122004:7774]
2025-10-09T21:49:48.754929Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:48.762095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:48.762118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:48.762259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.762326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.762387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.762447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.762488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.762496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.762559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.762612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.762667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.762726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.762766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.762776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.762836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.762887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.762940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.762991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.763033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.763041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.763108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.763350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.763407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.763463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.763501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.763507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.763567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.763618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.763670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.763723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.763764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.763770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.763832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.763887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.763941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.763994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.764035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.764041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.764102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.764153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.764206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.764258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.764318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.764327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.764378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.764430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.764481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.764532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.764574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.764582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.764639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.764687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.764738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.764790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.764833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.764842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.764894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:48.764947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:48.765008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:48.765065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:48.765108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:48.765115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:48.765155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342536146122004:7774], now have 1 active actors on pipe
2025-10-09T21:49:48.765342Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:48.765361Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342536146122004:7774]
2025-10-09T21:49:48.765593Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:48.765602Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:48.765628Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1631 reason: error at write: Disconnected sessionId: src_id|d03ea725-75330869-c8cc13c1-a8246ca7_0
2025-10-09T21:49:48.765809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1631 sessionId: src_id|d03ea725-75330869-c8cc13c1-a8246ca7_0 is DEAD
2025-10-09T21:49:48.765956Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d03ea725-75330869-c8cc13c1-a8246ca7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:48.765968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d03ea725-75330869-c8cc13c1-a8246ca7_0] Write session will restart in 0.005104s
2025-10-09T21:49:48.765947Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:48.766006Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d03ea725-75330869-c8cc13c1-a8246ca7_0] Write session: Do CDS request
2025-10-09T21:49:48.766013Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d03ea725-75330869-c8cc13c1-a8246ca7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:48.772541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d03ea725-75330869-c8cc13c1-a8246ca7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:48.772650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:48.772685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1632
2025-10-09T21:49:48.772806Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1632 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:48.772837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1632 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:48.772844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1632 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:48.772848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:48.773298Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1632 sessionId:  describe result for acl check
2025-10-09T21:49:48.773337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:48.773344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:48.773345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:48.773351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342536146122076:7507] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:48.773891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342536146122076:7507] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:48.776192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342536146122076:7507] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:48.779482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342536146122076:7507] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:48.779495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342536146122076:7507] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:48.779499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342536146122076:7507] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:48.779507Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1632 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:48.781262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:48.784350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:48.784546Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:48.784557Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:48.784569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:48.784630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:48.784673Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342535784293631:4314]
2025-10-09T21:49:48.785230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:48.785558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:48.785604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:48.785910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:48.785957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.785963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.785964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.785965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.785966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.785968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.785969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.785970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.785971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.785976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.785988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.785992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.785996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.786004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.786008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.786017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.786020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.786024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.786049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.786062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.786072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.786078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.786086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.786091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.786097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.786106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.786111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.786120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.786124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.786129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.786134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.786139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.786144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.786149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.786159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.786164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.786196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:48.792697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:48.794166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:48.794181Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:48.794183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:48.794188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:48.794194Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1631 [2:7559342535784293631:4314]
2025-10-09T21:49:48.794206Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:48.794224Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:48.794228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:48.794230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:48.794233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:48.794251Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:48.794257Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:48.794264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:48.794322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.794330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.794332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.794335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:48.794404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:48.794412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.794424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.794441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.794463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:48.794488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342536146122241:7507], now have 1 active actors on pipe
2025-10-09T21:49:48.794495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342536146122032:2432], now have 1 active actors on pipe
2025-10-09T21:49:48.794503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342535784293626:4314]
2025-10-09T21:49:48.794506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342535784293626:4314]
2025-10-09T21:49:48.794507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342535784293626:4314]
2025-10-09T21:49:48.794520Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:48.794538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:48.794545Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:48.794549Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:48.794551Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:48.794557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:48.794565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:48.794597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342535784293626:4314]
2025-10-09T21:49:48.794605Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:48.794574Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1631
2025-10-09T21:49:48.794608Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:48.794574Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1631
2025-10-09T21:49:48.794640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:48.794651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:48.794685Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2fa5764d-f6797b39-6eb063e5-c82a0aaa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:48.794697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:48.794701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:48.794705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:48.794715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:48.794729Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:48.794748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:48.794934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:48.794943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:48.794957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:48.795050Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1632 partition: 0 MaxSeqNo: 25 sessionId: src_id|2fa5764d-f6797b39-6eb063e5-c82a0aaa_0
2025-10-09T21:49:48.795416Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d03ea725-75330869-c8cc13c1-a8246ca7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2fa5764d-f6797b39-6eb063e5-c82a0aaa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:48.797861Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2fa5764d-f6797b39-6eb063e5-c82a0aaa_0] Write session: try to update token
2025-10-09T21:49:48.797879Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2fa5764d-f6797b39-6eb063e5-c82a0aaa_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:48.808912Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:48.808934Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:48.808936Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:48.808938Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:48.808948Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342535784293626:4314]
2025-10-09T21:49:48.808952Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342535784293626:4314]
2025-10-09T21:49:48.808957Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342535784293626:4314]
2025-10-09T21:49:48.809000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:48.809005Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:48.809011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.809011Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:48.809014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:48.809015Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:48.809153Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:48.810710Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1632 sessionId: src_id|2fa5764d-f6797b39-6eb063e5-c82a0aaa_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:48.813362Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:48.865551Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342536146122258:7774]
2025-10-09T21:49:48.865878Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 1631, pipe: [1:7559342536146122258:7774]
2025-10-09T21:49:48.960958Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:48.961176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:48.961189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:48.961225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342536146122258:7774], now have 1 active actors on pipe
2025-10-09T21:49:48.961441Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:48.961467Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:48.961490Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342536146122258:7774]
2025-10-09T21:49:48.961514Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:48.961524Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1632 reason: error at write: Disconnected sessionId: src_id|2fa5764d-f6797b39-6eb063e5-c82a0aaa_0
2025-10-09T21:49:48.961597Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:48.961696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1632 sessionId: src_id|2fa5764d-f6797b39-6eb063e5-c82a0aaa_0 is DEAD
2025-10-09T21:49:48.961871Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2fa5764d-f6797b39-6eb063e5-c82a0aaa_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:48.961883Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2fa5764d-f6797b39-6eb063e5-c82a0aaa_0] Write session will restart in 0.005092s
2025-10-09T21:49:48.961918Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2fa5764d-f6797b39-6eb063e5-c82a0aaa_0] Write session: Do CDS request
2025-10-09T21:49:48.961924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2fa5764d-f6797b39-6eb063e5-c82a0aaa_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:48.968381Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2fa5764d-f6797b39-6eb063e5-c82a0aaa_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:48.968467Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:48.968485Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1633
2025-10-09T21:49:48.968608Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1633 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:48.968631Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1633 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:48.968638Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1633 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:48.968642Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:48.969055Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1633 sessionId:  describe result for acl check
2025-10-09T21:49:48.969090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:48.969098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:48.969100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:48.969106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342536146122309:7505] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:48.969657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342536146122309:7505] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:48.972157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342536146122309:7505] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:48.974670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342536146122309:7505] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:48.974680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342536146122309:7505] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:48.974683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342536146122309:7505] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:48.974690Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1633 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:48.978186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:48.978614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:48.978765Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:48.978774Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:48.978784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:48.978829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:48.978864Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342535784293840:4317]
2025-10-09T21:49:48.979411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:48.979680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:48.979719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:48.980007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:48.980060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.980066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.980067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.980068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.980069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.980070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.980071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.980072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.980073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.980078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.980088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.980092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.980096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.980100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.980104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.980108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.980111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.980114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.980120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.980128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:48.980134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.980140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:48.980144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.980149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:48.980153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.980157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:48.980161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.980167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:48.980173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.980178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:48.980183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.980188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.980197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.980202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.980207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.980212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.980244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:48.986456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:48.987437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:48.987455Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:48.987457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:48.987461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:48.987468Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1632 [2:7559342535784293840:4317]
2025-10-09T21:49:48.987482Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:48.987499Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:48.987504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:48.987507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:48.987509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:48.987525Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:48.987537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:48.987544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:48.987612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.987619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.987621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.987624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:48.987687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:48.987694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:48.987706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:48.987721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:48.987741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:48.987766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342536146122476:7505], now have 1 active actors on pipe
2025-10-09T21:49:48.987774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342536146122264:2432], now have 1 active actors on pipe
2025-10-09T21:49:48.987782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342535784293835:4317]
2025-10-09T21:49:48.987784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342535784293835:4317]
2025-10-09T21:49:48.987786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342535784293835:4317]
2025-10-09T21:49:48.987799Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:48.987813Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:48.987816Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:48.987819Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:48.987821Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:48.987831Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:48.987834Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:48.987830Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1632
2025-10-09T21:49:48.987858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342535784293835:4317]
2025-10-09T21:49:48.987834Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1632
2025-10-09T21:49:48.987865Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:48.987872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:48.987897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:48.987906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:48.987938Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e5f31d32-a8813535-402c7974-fed35697_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:48.987950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:48.987954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:48.987959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:48.987968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:48.987981Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:48.987998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:48.988137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:48.988146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:48.988159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:48.988241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1633 partition: 0 MaxSeqNo: 25 sessionId: src_id|e5f31d32-a8813535-402c7974-fed35697_0
2025-10-09T21:49:48.988543Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2fa5764d-f6797b39-6eb063e5-c82a0aaa_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e5f31d32-a8813535-402c7974-fed35697_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:48.990771Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5f31d32-a8813535-402c7974-fed35697_0] Write session: try to update token
2025-10-09T21:49:48.990787Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5f31d32-a8813535-402c7974-fed35697_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:49.001468Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:49.001488Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:49.001491Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:49.001493Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:49.001503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342535784293835:4317]
2025-10-09T21:49:49.001507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342535784293835:4317]
2025-10-09T21:49:49.001511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342535784293835:4317]
2025-10-09T21:49:49.001564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:49.001565Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:49.001571Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:49.001572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.001575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:49.001577Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:49.001706Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:49.003214Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1633 sessionId: src_id|e5f31d32-a8813535-402c7974-fed35697_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:49.005512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:49.061632Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342540441089786:7774]
2025-10-09T21:49:49.061893Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 1632, pipe: [1:7559342540441089786:7774]
2025-10-09T21:49:49.153074Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:49.153790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:49.153807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:49.153852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342540441089786:7774], now have 1 active actors on pipe
2025-10-09T21:49:49.154048Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:49.154067Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:49.154071Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342540441089786:7774]
2025-10-09T21:49:49.154083Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:49.154090Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:49.154140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1633 reason: error at write: Disconnected sessionId: src_id|e5f31d32-a8813535-402c7974-fed35697_0
2025-10-09T21:49:49.154317Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1633 sessionId: src_id|e5f31d32-a8813535-402c7974-fed35697_0 is DEAD
2025-10-09T21:49:49.154485Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5f31d32-a8813535-402c7974-fed35697_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:49.154498Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5f31d32-a8813535-402c7974-fed35697_0] Write session will restart in 0.005295s
2025-10-09T21:49:49.154525Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5f31d32-a8813535-402c7974-fed35697_0] Write session: Do CDS request
2025-10-09T21:49:49.154531Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5f31d32-a8813535-402c7974-fed35697_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:49.160501Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5f31d32-a8813535-402c7974-fed35697_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:49.160533Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:49.160548Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1634
2025-10-09T21:49:49.160747Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1634 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:49.160783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1634 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:49.160794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1634 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:49.160797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:49.161218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1634 sessionId:  describe result for acl check
2025-10-09T21:49:49.161248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:49.161250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:49.161252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:49.161257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342540441089805:7500] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:49.161784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342540441089805:7500] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:49.164166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342540441089805:7500] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:49.167438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342540441089805:7500] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:49.167451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342540441089805:7500] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:49.167454Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342540441089805:7500] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:49.167462Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1634 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:49.173192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:49.173657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:49.173828Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:49.173839Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:49.173852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:49.173906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:49.173953Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342540079261346:4315]
2025-10-09T21:49:49.174569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:49.174831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:49.174873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:49.175187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:49.175237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.175243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.175244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.175246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.175247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.175249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.175250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.175251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.175252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.175258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.175269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.175274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.175278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.175282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.175286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.175290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.175294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.175298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.175306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.175314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.175324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.175330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.175335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.175346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.175350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.175359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.175363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.175367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.175372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.175381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.175386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.175395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.175399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.175404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.175410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.175419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.175448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:49.181784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:49.182848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:49.182868Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:49.182871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:49.182876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:49.182883Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1633 [2:7559342540079261346:4315]
2025-10-09T21:49:49.182897Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:49.182917Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:49.182925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:49.182929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:49.182931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:49.182948Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:49.182959Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:49.182966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:49.183035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.183043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.183045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.183048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:49.183123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:49.183132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.183143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.183158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.183179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:49.183202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342540079261341:4315]
2025-10-09T21:49:49.183210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342540079261341:4315]
2025-10-09T21:49:49.183213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342540079261341:4315]
2025-10-09T21:49:49.183228Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:49.183244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342540441089985:7500], now have 1 active actors on pipe
2025-10-09T21:49:49.183248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342540441089790:2432], now have 1 active actors on pipe
2025-10-09T21:49:49.183254Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:49.183257Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:49.183259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:49.183262Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:49.183273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:49.183276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:49.183310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342540079261341:4315]
2025-10-09T21:49:49.183315Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:49.183323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:49.183306Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1633
2025-10-09T21:49:49.183311Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1633
2025-10-09T21:49:49.183367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:49.183379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:49.183408Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|95e68a59-9a0a1412-e16131cc-feb0cb20_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:49.183415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:49.183420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:49.183422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:49.183433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:49.183445Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:49.183459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:49.183606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:49.183618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:49.183634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:49.183732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1634 partition: 0 MaxSeqNo: 25 sessionId: src_id|95e68a59-9a0a1412-e16131cc-feb0cb20_0
2025-10-09T21:49:49.184111Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5f31d32-a8813535-402c7974-fed35697_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|95e68a59-9a0a1412-e16131cc-feb0cb20_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:49.186709Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|95e68a59-9a0a1412-e16131cc-feb0cb20_0] Write session: try to update token
2025-10-09T21:49:49.186730Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|95e68a59-9a0a1412-e16131cc-feb0cb20_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:49.200068Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1634 sessionId: src_id|95e68a59-9a0a1412-e16131cc-feb0cb20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:49.202168Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:49.202183Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:49.202192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342540079261341:4315]
2025-10-09T21:49:49.202241Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:49.202244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:49.202414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:49.202497Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:49.202555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:49.202691Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:49.202858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:49.203035Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:49.203089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:49.203103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:49.203126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:49.203167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:49.203265Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:49.208775Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:49.208793Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:49.208796Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:49.208798Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:49.208809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342540079261341:4315]
2025-10-09T21:49:49.208813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342540079261341:4315]
2025-10-09T21:49:49.208817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342540079261341:4315]
2025-10-09T21:49:49.208858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:49.208864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.208866Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:49.208867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:49.208872Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:49.208876Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:49.208997Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:49.254091Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342540441090025:7774]
2025-10-09T21:49:49.254342Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 1633, pipe: [1:7559342540441090025:7774]
2025-10-09T21:49:49.358418Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:49.370499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:49.370522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:49.370663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:49.370721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:49.370786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:49.370844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:49.370886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:49.370895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:49.371106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:49.371170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:49.371234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:49.371311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:49.371428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:49.371434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:49.371491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:49.371547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:49.371601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:49.371781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:49.371890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:49.371896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:49.371951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:49.372008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:49.372069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:49.372123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:49.372162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:49.372167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:49.372229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:49.372291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:49.372344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:49.372437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:49.372493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:49.372499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:49.372709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:49.372765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:49.372822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:49.372874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:49.372913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:49.372918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:49.372971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:49.373029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:49.373085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:49.373140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:49.373178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:49.373183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:49.373237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:49.373452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:49.373690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:49.373919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:49.373959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:49.373964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:49.374171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:49.374382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:49.374581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:49.374789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:49.374831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:49.374833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:49.375011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:49.375210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:49.375413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:49.375618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:49.375728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:49.375734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:49.375789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342540441090025:7774], now have 1 active actors on pipe
2025-10-09T21:49:49.376059Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:49.376068Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:49.376073Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:49.376110Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1634 reason: error at write: Disconnected sessionId: src_id|95e68a59-9a0a1412-e16131cc-feb0cb20_0
2025-10-09T21:49:49.376120Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342540441090025:7774]
2025-10-09T21:49:49.376313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1634 sessionId: src_id|95e68a59-9a0a1412-e16131cc-feb0cb20_0 is DEAD
2025-10-09T21:49:49.376480Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95e68a59-9a0a1412-e16131cc-feb0cb20_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:49.376504Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95e68a59-9a0a1412-e16131cc-feb0cb20_0] Write session will restart in 0.008476s
2025-10-09T21:49:49.376530Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95e68a59-9a0a1412-e16131cc-feb0cb20_0] Write session: Do CDS request
2025-10-09T21:49:49.376535Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95e68a59-9a0a1412-e16131cc-feb0cb20_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:49.376815Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:49.386547Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|95e68a59-9a0a1412-e16131cc-feb0cb20_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:49.386566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:49.386580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1635
2025-10-09T21:49:49.386812Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1635 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:49.386843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1635 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:49.386855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1635 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:49.386859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:49.387258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1635 sessionId:  describe result for acl check
2025-10-09T21:49:49.387287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:49.387289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:49.387292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:49.387297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342540441090121:7488] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:49.387830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342540441090121:7488] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:49.390820Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342540441090121:7488] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:49.398158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:49.398189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342540441090121:7488] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:49.398206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342540441090121:7488] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:49.398210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342540441090121:7488] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:49.398218Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1635 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:49.398602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:49.398784Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:49.398792Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:49.398800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:49.398836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:49.398875Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342540079261557:4310]
2025-10-09T21:49:49.399280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:49.399534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:49.399561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:49.399801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:49.399837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.399843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.399844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.399845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.399847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.399849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.399850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.399851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.399852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.399856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.399866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.399870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.399874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.399877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.399881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.399885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.399888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.399892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.399898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.399907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.399913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.399918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.399922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.399927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.399931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.399936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.399941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.399946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.399949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.399953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.399955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.399959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.399961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.399964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.399966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.399970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.399988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:49.406271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:49.407411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:49.407424Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:49.407426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:49.407430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:49.407437Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1634 [2:7559342540079261557:4310]
2025-10-09T21:49:49.407447Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:49.407462Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:49.407464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:49.407466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:49.407468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:49.407482Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:49.407490Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:49.407496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:49.407539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.407543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.407544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.407547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:49.407602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:49.407608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.407616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.407622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.407632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:49.407654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342540079261551:4310]
2025-10-09T21:49:49.407662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342540079261551:4310]
2025-10-09T21:49:49.407664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342540079261551:4310]
2025-10-09T21:49:49.407674Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:49.407682Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:49.407691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342540441090030:2432], now have 1 active actors on pipe
2025-10-09T21:49:49.407692Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:49.407695Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:49.407698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342540441090249:7488], now have 1 active actors on pipe
2025-10-09T21:49:49.407703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:49.407717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:49.407725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:49.407756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342540079261551:4310]
2025-10-09T21:49:49.407768Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:49.407750Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1634
2025-10-09T21:49:49.407787Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:49.407774Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1634
2025-10-09T21:49:49.407809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:49.407820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:49.407849Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5f19e04d-993f5466-c66de147-ff2020cc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:49.407861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:49.407865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:49.407872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:49.407883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:49.407894Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:49.407913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:49.408040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:49.408048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:49.408063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:49.408175Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1635 partition: 0 MaxSeqNo: 25 sessionId: src_id|5f19e04d-993f5466-c66de147-ff2020cc_0
2025-10-09T21:49:49.408530Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95e68a59-9a0a1412-e16131cc-feb0cb20_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5f19e04d-993f5466-c66de147-ff2020cc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:49.410887Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f19e04d-993f5466-c66de147-ff2020cc_0] Write session: try to update token
2025-10-09T21:49:49.410902Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f19e04d-993f5466-c66de147-ff2020cc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:49.421425Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:49.421445Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:49.421456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342540079261551:4310]
2025-10-09T21:49:49.421506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:49.421511Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:49.422051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:49.422346Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:49.422625Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:49.422895Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:49.423243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:49.423286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:49.428296Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1635 sessionId: src_id|5f19e04d-993f5466-c66de147-ff2020cc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:49.430786Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:49.432112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:49.432137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:49.432158Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:49.432161Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:49.432163Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:49.432166Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:49.432175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342540079261551:4310]
2025-10-09T21:49:49.432179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342540079261551:4310]
2025-10-09T21:49:49.432184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342540079261551:4310]
2025-10-09T21:49:49.432226Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:49.432229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:49.432234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.432236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:49.432241Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:49.432246Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:49.432397Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:49.476342Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342540441090261:7774]
2025-10-09T21:49:49.476592Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 1634, pipe: [1:7559342540441090261:7774]
2025-10-09T21:49:49.588731Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:49.589369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:49.589404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342540441090261:7774], now have 1 active actors on pipe
2025-10-09T21:49:49.589529Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:49.589576Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342540441090261:7774]
2025-10-09T21:49:49.589597Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:49.589635Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:49.589642Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:49.589703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1635 reason: error at write: Disconnected sessionId: src_id|5f19e04d-993f5466-c66de147-ff2020cc_0
2025-10-09T21:49:49.589840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1635 sessionId: src_id|5f19e04d-993f5466-c66de147-ff2020cc_0 is DEAD
2025-10-09T21:49:49.589980Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f19e04d-993f5466-c66de147-ff2020cc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:49.589990Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f19e04d-993f5466-c66de147-ff2020cc_0] Write session will restart in 0.008776s
2025-10-09T21:49:49.590007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f19e04d-993f5466-c66de147-ff2020cc_0] Write session: Do CDS request
2025-10-09T21:49:49.590015Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f19e04d-993f5466-c66de147-ff2020cc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:49.599532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f19e04d-993f5466-c66de147-ff2020cc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:49.599614Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:49.599635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1636
2025-10-09T21:49:49.599769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1636 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:49.599812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1636 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:49.599819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1636 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:49.599824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:49.600196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1636 sessionId:  describe result for acl check
2025-10-09T21:49:49.600223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:49.600228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:49.600229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:49.600234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342540441090403:7481] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:49.600725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342540441090403:7481] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:49.602448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342540441090403:7481] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:49.606550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:49.606765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342540441090403:7481] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:49.606778Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342540441090403:7481] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:49.606781Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342540441090403:7481] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:49.606790Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1636 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:49.607030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:49.607221Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:49.607231Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:49.607242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:49.607294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:49.607337Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342540079261770:4292]
2025-10-09T21:49:49.607903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:49.608196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:49.608242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:49.608552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:49.608602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.608609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.608610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.608611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.608612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.608613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.608614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.608615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.608616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.608621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.608632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.608637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.608641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.608645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.608649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.608653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.608656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.608660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.608666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.608674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.608681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.608686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.608691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.608696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.608701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.608713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.608719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.608728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.608732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.608737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.608746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.608751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.608755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.608765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.608769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.608775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.608805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:49.615163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:49.616751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:49.616770Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:49.616772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:49.616777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:49.616783Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1635 [2:7559342540079261770:4292]
2025-10-09T21:49:49.616797Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:49.616815Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:49.616818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:49.616822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:49.616824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:49.616842Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:49.616851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:49.616859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:49.616918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.616925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.616928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.616931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:49.617001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:49.617011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.617023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.617036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.617056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:49.617084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342540441090264:2432], now have 1 active actors on pipe
2025-10-09T21:49:49.617093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342540441090484:7481], now have 1 active actors on pipe
2025-10-09T21:49:49.617101Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342540079261764:4292]
2025-10-09T21:49:49.617103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342540079261764:4292]
2025-10-09T21:49:49.617105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342540079261764:4292]
2025-10-09T21:49:49.617120Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:49.617136Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:49.617139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:49.617144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:49.617146Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:49.617153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:49.617156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:49.617135Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1635
2025-10-09T21:49:49.617141Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1635
2025-10-09T21:49:49.617187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342540079261764:4292]
2025-10-09T21:49:49.617198Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:49.617202Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:49.617218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:49.617228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:49.617279Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7f8b2162-6626dc5-fc6ad265-679f1f6b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:49.617292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:49.617296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:49.617298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:49.617310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:49.617324Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:49.617346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:49.617525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:49.617533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:49.617555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:49.617664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1636 partition: 0 MaxSeqNo: 25 sessionId: src_id|7f8b2162-6626dc5-fc6ad265-679f1f6b_0
2025-10-09T21:49:49.617987Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f19e04d-993f5466-c66de147-ff2020cc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7f8b2162-6626dc5-fc6ad265-679f1f6b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:49.620325Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7f8b2162-6626dc5-fc6ad265-679f1f6b_0] Write session: try to update token
2025-10-09T21:49:49.620353Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7f8b2162-6626dc5-fc6ad265-679f1f6b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:49.631898Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:49.631919Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:49.631921Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:49.631923Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:49.631931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342540079261764:4292]
2025-10-09T21:49:49.631935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342540079261764:4292]
2025-10-09T21:49:49.631938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342540079261764:4292]
2025-10-09T21:49:49.631974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:49.631979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.631981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:49.631985Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:49.631991Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:49.631994Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:49.632115Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:49.634290Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1636 sessionId: src_id|7f8b2162-6626dc5-fc6ad265-679f1f6b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:49.636559Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:49.689848Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342540441090495:7774]
2025-10-09T21:49:49.690167Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 1635, pipe: [1:7559342540441090495:7774]
2025-10-09T21:49:49.775137Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:49.775571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:49.775585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:49.775623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342540441090495:7774], now have 1 active actors on pipe
2025-10-09T21:49:49.775827Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342540441090495:7774]
2025-10-09T21:49:49.775826Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:49.775864Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:49.775867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:49.775922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1636 reason: error at write: Disconnected sessionId: src_id|7f8b2162-6626dc5-fc6ad265-679f1f6b_0
2025-10-09T21:49:49.776104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1636 sessionId: src_id|7f8b2162-6626dc5-fc6ad265-679f1f6b_0 is DEAD
2025-10-09T21:49:49.776247Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f8b2162-6626dc5-fc6ad265-679f1f6b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:49.776241Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:49.776271Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f8b2162-6626dc5-fc6ad265-679f1f6b_0] Write session will restart in 0.006896s
2025-10-09T21:49:49.776311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f8b2162-6626dc5-fc6ad265-679f1f6b_0] Write session: Do CDS request
2025-10-09T21:49:49.776319Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f8b2162-6626dc5-fc6ad265-679f1f6b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:49.784605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7f8b2162-6626dc5-fc6ad265-679f1f6b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:49.784655Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:49.784682Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1637
2025-10-09T21:49:49.784845Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1637 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:49.784893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1637 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:49.784902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1637 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:49.784906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:49.785451Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1637 sessionId:  describe result for acl check
2025-10-09T21:49:49.785486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:49.785490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:49.785492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:49.785498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342540441090581:7467] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:49.786087Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342540441090581:7467] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:49.788509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342540441090581:7467] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:49.791288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342540441090581:7467] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:49.791300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342540441090581:7467] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:49.791304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342540441090581:7467] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:49.791311Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1637 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:49.791398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:49.795171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:49.795371Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:49.795381Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:49.795392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:49.795446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:49.795490Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342540079262000:4281]
2025-10-09T21:49:49.796069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:49.796406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:49.796470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:49.796901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:49.796965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.796970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.796971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.796972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.796972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.796973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.796973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.796974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.796975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.796979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.796985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.796987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.796989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.796991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.796993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.796995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.796997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.796999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.797004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.797009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.797013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.797017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.797019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.797022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.797025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.797028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.797031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.797034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.797036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.797040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.797042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.797046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.797049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.797052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.797055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.797058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.797082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:49.803352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:49.804524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:49.804539Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:49.804542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:49.804547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:49.804553Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1636 [2:7559342540079262000:4281]
2025-10-09T21:49:49.804566Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:49.804582Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:49.804588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:49.804591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:49.804593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:49.804611Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:49.804622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:49.804630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:49.804681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.804683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.804685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.804688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:49.804753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:49.804760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.804772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.804782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.804792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:49.804811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342540441090511:2432], now have 1 active actors on pipe
2025-10-09T21:49:49.804814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342540441090729:7467], now have 1 active actors on pipe
2025-10-09T21:49:49.804819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342540079261994:4281]
2025-10-09T21:49:49.804821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342540079261994:4281]
2025-10-09T21:49:49.804822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342540079261994:4281]
2025-10-09T21:49:49.804830Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:49.804842Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:49.804842Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:49.804844Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:49.804846Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:49.804855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:49.804858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:49.804869Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1636
2025-10-09T21:49:49.804892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342540079261994:4281]
2025-10-09T21:49:49.804907Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:49.804924Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:49.804936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:49.804939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:49.804966Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2ce4ca5f-a047464b-b5958697-a7168fd8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:49.804979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:49.804986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:49.804991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:49.804999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:49.805007Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:49.805020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:49.805059Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1636
2025-10-09T21:49:49.805107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:49.805114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:49.805125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:49.805205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1637 partition: 0 MaxSeqNo: 25 sessionId: src_id|2ce4ca5f-a047464b-b5958697-a7168fd8_0
2025-10-09T21:49:49.805531Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f8b2162-6626dc5-fc6ad265-679f1f6b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2ce4ca5f-a047464b-b5958697-a7168fd8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:49.807943Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ce4ca5f-a047464b-b5958697-a7168fd8_0] Write session: try to update token
2025-10-09T21:49:49.807962Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ce4ca5f-a047464b-b5958697-a7168fd8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:49.818325Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:49.818342Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:49.818343Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:49.818345Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:49.818352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342540079261994:4281]
2025-10-09T21:49:49.818356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342540079261994:4281]
2025-10-09T21:49:49.818358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342540079261994:4281]
2025-10-09T21:49:49.818389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:49.818394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.818396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:49.818419Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:49.818428Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:49.818446Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:49.818510Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:49.820452Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1637 sessionId: src_id|2ce4ca5f-a047464b-b5958697-a7168fd8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:49.822738Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:49.876175Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342540441090753:7774]
2025-10-09T21:49:49.876440Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 1636, pipe: [1:7559342540441090753:7774]
2025-10-09T21:49:49.954704Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:49.955046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:49.955065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:49.955106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342540441090753:7774], now have 1 active actors on pipe
2025-10-09T21:49:49.955348Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:49.955468Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:49.955514Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:49.955514Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342540441090753:7774]
2025-10-09T21:49:49.955603Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1637 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2ce4ca5f-a047464b-b5958697-a7168fd8_0
2025-10-09T21:49:49.955772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1637 sessionId: src_id|2ce4ca5f-a047464b-b5958697-a7168fd8_0 is DEAD
2025-10-09T21:49:49.955915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ce4ca5f-a047464b-b5958697-a7168fd8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:49.955930Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ce4ca5f-a047464b-b5958697-a7168fd8_0] Write session will restart in 0.006909s
2025-10-09T21:49:49.955956Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ce4ca5f-a047464b-b5958697-a7168fd8_0] Write session: Do CDS request
2025-10-09T21:49:49.955963Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ce4ca5f-a047464b-b5958697-a7168fd8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:49.964461Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ce4ca5f-a047464b-b5958697-a7168fd8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:49.964535Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:49.964546Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1638
2025-10-09T21:49:49.964653Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1638 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:49.964677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1638 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:49.964693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1638 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:49.964696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:49.965044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1638 sessionId:  describe result for acl check
2025-10-09T21:49:49.965067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:49.965071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:49.965073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:49.965077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342540441090855:7453] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:49.965550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342540441090855:7453] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:49.967897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342540441090855:7453] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:49.969292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:49.972114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:49.972271Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:49.972279Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:49.972288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:49.972271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342540441090855:7453] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:49.972283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342540441090855:7453] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:49.972287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342540441090855:7453] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:49.972294Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1638 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:49.972328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:49.972356Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342540079262238:4276]
2025-10-09T21:49:49.972869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:49.973121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:49.973162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:49.973430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:49.973466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.973470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.973471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.973472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.973472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.973473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.973473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.973474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.973475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.973478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.973484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.973486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.973489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.973491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.973493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.973495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.973497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.973499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.973504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.973508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:49.973513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.973516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:49.973519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.973523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:49.973525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.973528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:49.973531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.973534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:49.973545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.973550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:49.973553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.973556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.973558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.973561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.973563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.973566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.973582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:49.979824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:49.980800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:49.980812Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:49.980814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:49.980817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:49.980822Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1637 [2:7559342540079262238:4276]
2025-10-09T21:49:49.980831Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:49.980844Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:49.980847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:49.980849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:49.980850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:49.980863Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:49.980873Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:49.980879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:49.980944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.980952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.980953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.980955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:49.981044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:49.981051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.981057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:49.981064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:49.981074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:49.981092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342540441090758:2432], now have 1 active actors on pipe
2025-10-09T21:49:49.981098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342540441090977:7453], now have 1 active actors on pipe
2025-10-09T21:49:49.981103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342540079262233:4276]
2025-10-09T21:49:49.981105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342540079262233:4276]
2025-10-09T21:49:49.981106Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342540079262233:4276]
2025-10-09T21:49:49.981115Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:49.981128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:49.981137Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:49.981140Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:49.981142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:49.981146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:49.981149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:49.981170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342540079262233:4276]
2025-10-09T21:49:49.981176Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:49.981178Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:49.981150Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1637
2025-10-09T21:49:49.981199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:49.981202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:49.981179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1637
2025-10-09T21:49:49.981228Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8ea2345f-44c3f00a-aa9ec415-9df9c3ea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:49.981237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:49.981240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:49.981241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:49.981248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:49.981256Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:49.981268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:49.981382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:49.981390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:49.981403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:49.981487Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1638 partition: 0 MaxSeqNo: 25 sessionId: src_id|8ea2345f-44c3f00a-aa9ec415-9df9c3ea_0
2025-10-09T21:49:49.981814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ce4ca5f-a047464b-b5958697-a7168fd8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8ea2345f-44c3f00a-aa9ec415-9df9c3ea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:49.984113Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8ea2345f-44c3f00a-aa9ec415-9df9c3ea_0] Write session: try to update token
2025-10-09T21:49:49.984131Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8ea2345f-44c3f00a-aa9ec415-9df9c3ea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:49.994788Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:49.994807Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:49.994809Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:49.994810Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:49.994818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342540079262233:4276]
2025-10-09T21:49:49.994821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342540079262233:4276]
2025-10-09T21:49:49.994824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342540079262233:4276]
2025-10-09T21:49:49.994855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:49.994861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:49.994863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:49.994867Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:49.994872Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:49.994878Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:49.994986Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:49.996078Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1638 sessionId: src_id|8ea2345f-44c3f00a-aa9ec415-9df9c3ea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:49.998462Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:50.055424Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342544736058285:7774]
2025-10-09T21:49:50.055763Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 1637, pipe: [1:7559342544736058285:7774]
2025-10-09T21:49:50.142800Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:50.143041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.143057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.143101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342544736058285:7774], now have 1 active actors on pipe
2025-10-09T21:49:50.143274Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:50.143355Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:50.143383Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342544736058285:7774]
2025-10-09T21:49:50.143399Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:50.143486Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1638 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8ea2345f-44c3f00a-aa9ec415-9df9c3ea_0
2025-10-09T21:49:50.143637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1638 sessionId: src_id|8ea2345f-44c3f00a-aa9ec415-9df9c3ea_0 is DEAD
2025-10-09T21:49:50.143803Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ea2345f-44c3f00a-aa9ec415-9df9c3ea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:50.143815Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ea2345f-44c3f00a-aa9ec415-9df9c3ea_0] Write session will restart in 0.007251s
2025-10-09T21:49:50.143850Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ea2345f-44c3f00a-aa9ec415-9df9c3ea_0] Write session: Do CDS request
2025-10-09T21:49:50.143857Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ea2345f-44c3f00a-aa9ec415-9df9c3ea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:50.152525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8ea2345f-44c3f00a-aa9ec415-9df9c3ea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:50.152617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:50.152629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1639
2025-10-09T21:49:50.152757Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1639 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:50.152804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1639 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:50.152814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1639 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:50.152818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:50.153208Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1639 sessionId:  describe result for acl check
2025-10-09T21:49:50.153242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:50.153248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:50.153250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:50.153256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342544736058364:7447] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:50.153828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342544736058364:7447] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:50.156650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342544736058364:7447] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:50.158950Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342544736058364:7447] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:50.158964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342544736058364:7447] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:50.158968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342544736058364:7447] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:50.158976Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1639 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:50.159229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:50.162632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:50.162844Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:50.162854Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:50.162866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:50.162921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:50.162952Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342544374229750:4273]
2025-10-09T21:49:50.163369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:50.163676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:50.163704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:50.163928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:50.163962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.163967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.163968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.163968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.163970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.163971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.163972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.163972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.163973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.163976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.163983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.163986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.163988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.163990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.163992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.163994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.163997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.163999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.164003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.164008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.164012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.164016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.164018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.164022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.164024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.164027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.164030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.164033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.164035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.164039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.164041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.164044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.164047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.164051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.164057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.164061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.164089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:50.170612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:50.171638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:50.171655Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:50.171658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:50.171664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:50.171671Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1638 [2:7559342544374229750:4273]
2025-10-09T21:49:50.171685Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:50.171703Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:50.171711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:50.171714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:50.171719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:50.171738Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:50.171749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:50.171758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:50.171829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.171837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.171839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.171842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:50.172078Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1638
2025-10-09T21:49:50.172089Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1638
2025-10-09T21:49:50.172161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:50.172170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.172178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.172185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.172195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:50.172213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342544736058289:2432], now have 1 active actors on pipe
2025-10-09T21:49:50.172221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342544736058511:7447], now have 1 active actors on pipe
2025-10-09T21:49:50.172239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.172249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.172268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342544374229744:4273]
2025-10-09T21:49:50.172274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342544374229744:4273]
2025-10-09T21:49:50.172275Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342544374229744:4273]
2025-10-09T21:49:50.172286Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:50.172298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:50.172311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:50.172317Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:50.172319Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:50.172323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:50.172325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:50.172333Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e0c884d1-e3ca0d99-ca05df9d-c9933c8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:50.172338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:50.172342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:50.172344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:50.172352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:50.172362Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:50.172393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342544374229744:4273]
2025-10-09T21:49:50.172404Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:50.172414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:50.172425Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:50.172567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.172573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.172584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:50.172662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1639 partition: 0 MaxSeqNo: 25 sessionId: src_id|e0c884d1-e3ca0d99-ca05df9d-c9933c8_0
2025-10-09T21:49:50.173010Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ea2345f-44c3f00a-aa9ec415-9df9c3ea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e0c884d1-e3ca0d99-ca05df9d-c9933c8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:50.175590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e0c884d1-e3ca0d99-ca05df9d-c9933c8_0] Write session: try to update token
2025-10-09T21:49:50.175611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e0c884d1-e3ca0d99-ca05df9d-c9933c8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:50.188867Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:50.188885Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:50.188896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342544374229744:4273]
2025-10-09T21:49:50.188940Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:50.188950Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:50.189094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:50.189222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:50.189223Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1639 sessionId: src_id|e0c884d1-e3ca0d99-ca05df9d-c9933c8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:50.189342Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:50.189468Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:50.189652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:50.189690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:50.191580Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:50.191776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.191792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.191827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:50.191907Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:50.197258Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:50.197276Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:50.197279Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:50.197281Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:50.197292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342544374229744:4273]
2025-10-09T21:49:50.197296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342544374229744:4273]
2025-10-09T21:49:50.197301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342544374229744:4273]
2025-10-09T21:49:50.197343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:50.197345Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:50.197349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.197350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:50.197352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:50.197358Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:50.197484Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:50.243320Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342544736058527:7774]
2025-10-09T21:49:50.243621Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 1638, pipe: [1:7559342544736058527:7774]
2025-10-09T21:49:50.345901Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:50.352798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.352822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.352966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.353023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.353081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.353136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.353176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.353186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.353253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.353307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.353362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.353419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.353459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.353468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.353523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.353589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.353639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.353689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.353729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.353737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.353800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.353902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.353956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.354016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.354068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.354073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.354129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.354183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.354241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.354301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.354341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.354348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.354413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.354474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.354538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.354593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.354632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.354638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.354692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.354745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.354807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.354861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.354900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.354906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.354959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.355015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.355073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.355127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.355169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.355177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.355239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.355298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.355354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.355406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.355452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.355461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.355521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.355575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.355633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.355685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.355723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.355729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.355772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342544736058527:7774], now have 1 active actors on pipe
2025-10-09T21:49:50.355989Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:50.355994Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342544736058527:7774]
2025-10-09T21:49:50.356025Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:50.356028Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:50.356059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1639 reason: error at write: Disconnected sessionId: src_id|e0c884d1-e3ca0d99-ca05df9d-c9933c8_0
2025-10-09T21:49:50.356255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1639 sessionId: src_id|e0c884d1-e3ca0d99-ca05df9d-c9933c8_0 is DEAD
2025-10-09T21:49:50.356393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0c884d1-e3ca0d99-ca05df9d-c9933c8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:50.356402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0c884d1-e3ca0d99-ca05df9d-c9933c8_0] Write session will restart in 0.007114s
2025-10-09T21:49:50.356432Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0c884d1-e3ca0d99-ca05df9d-c9933c8_0] Write session: Do CDS request
2025-10-09T21:49:50.356440Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0c884d1-e3ca0d99-ca05df9d-c9933c8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:50.356759Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:50.364634Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e0c884d1-e3ca0d99-ca05df9d-c9933c8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:50.364695Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:50.364711Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1640
2025-10-09T21:49:50.364861Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1640 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:50.364895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1640 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:50.364904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1640 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:50.364907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:50.365479Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1640 sessionId:  describe result for acl check
2025-10-09T21:49:50.365519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:50.365525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:50.365527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:50.365534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342544736058579:7444] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:50.366147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342544736058579:7444] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:50.368657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342544736058579:7444] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:50.371751Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342544736058579:7444] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:50.371769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342544736058579:7444] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:50.371773Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342544736058579:7444] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:50.371782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1640 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:50.374191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:50.377369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:50.377660Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:50.377679Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:50.377694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:50.377758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:50.377800Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342544374229966:4268]
2025-10-09T21:49:50.378306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:50.378614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:50.378646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:50.378900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:50.378943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.378948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.378949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.378950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.378951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.378951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.378952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.378953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.378954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.378957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.378965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.378967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.378970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.378972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.378974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.378976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.378978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.378980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.378986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.378991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.378995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.378998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.379001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.379005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.379010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.379015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.379020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.379026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.379030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.379036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.379040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.379045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.379050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.379055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.379071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.379075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.379092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:50.385612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:50.386686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:50.386700Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:50.386704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:50.386709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:50.386716Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1639 [2:7559342544374229966:4268]
2025-10-09T21:49:50.386733Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:50.386752Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:50.386760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:50.386764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:50.386767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:50.386786Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:50.386797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:50.386805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:50.386872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.386880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.386883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.386885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:50.386959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:50.386967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.386979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.386991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.387013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:50.387038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342544374229961:4268]
2025-10-09T21:49:50.387045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342544374229961:4268]
2025-10-09T21:49:50.387047Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342544374229961:4268]
2025-10-09T21:49:50.387060Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:50.387079Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:50.387080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342544736058754:7444], now have 1 active actors on pipe
2025-10-09T21:49:50.387084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342544736058534:2432], now have 1 active actors on pipe
2025-10-09T21:49:50.387090Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:50.387092Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:50.387095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:50.387111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:50.387115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:50.387162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342544374229961:4268]
2025-10-09T21:49:50.387125Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1639
2025-10-09T21:49:50.387174Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:50.387184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:50.387179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1639
2025-10-09T21:49:50.387232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.387242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.387269Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5efc96f8-2e8b969a-c5f986d8-415e82b8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:50.387281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:50.387284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:50.387287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:50.387299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:50.387313Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:50.387332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:50.387447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.387455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.387469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:50.387593Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1640 partition: 0 MaxSeqNo: 25 sessionId: src_id|5efc96f8-2e8b969a-c5f986d8-415e82b8_0
2025-10-09T21:49:50.387938Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e0c884d1-e3ca0d99-ca05df9d-c9933c8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5efc96f8-2e8b969a-c5f986d8-415e82b8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:50.390530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5efc96f8-2e8b969a-c5f986d8-415e82b8_0] Write session: try to update token
2025-10-09T21:49:50.390552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5efc96f8-2e8b969a-c5f986d8-415e82b8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:50.398913Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:50.398929Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:50.398939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342544374229961:4268]
2025-10-09T21:49:50.398993Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:50.398992Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:50.399245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:50.399430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:50.399559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:50.399699Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:50.399903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:50.399955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:50.404435Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1640 sessionId: src_id|5efc96f8-2e8b969a-c5f986d8-415e82b8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:50.406788Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:50.407009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.407043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.407092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:50.407160Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:50.411728Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:50.411744Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:50.411746Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:50.411747Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:50.411756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342544374229961:4268]
2025-10-09T21:49:50.411759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342544374229961:4268]
2025-10-09T21:49:50.411762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342544374229961:4268]
2025-10-09T21:49:50.411798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:50.411804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.411807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:50.411813Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:50.411828Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:50.411831Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:50.411937Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:50.456310Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342544736058770:7774]
2025-10-09T21:49:50.456581Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 1639, pipe: [1:7559342544736058770:7774]
2025-10-09T21:49:50.560188Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:50.571682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.571712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.572040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.572304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.572373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.572434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.572846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.572857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.572921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.572980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.573038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.573330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.573458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.573469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.573527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.573756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.573814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.573869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.573928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.573936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.573996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.574269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.574330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.574390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.574522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.574530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.574590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.574649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.574711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.574774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.574819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.574826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.574886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.574945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.575043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.575099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.575154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.575162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.575219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.575275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.575333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.575564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.575620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.575629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.575864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.575920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.575974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.576032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.576073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.576080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.576136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.576355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.576606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.576666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.576797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.576805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.576863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:50.577114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:50.577368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:50.577427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:50.577569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:50.577577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:50.577634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342544736058770:7774], now have 1 active actors on pipe
2025-10-09T21:49:50.577928Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342544736058770:7774]
2025-10-09T21:49:50.577948Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:50.577985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:50.577988Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:50.578051Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1640 reason: error at write: Disconnected sessionId: src_id|5efc96f8-2e8b969a-c5f986d8-415e82b8_0
2025-10-09T21:49:50.578228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1640 sessionId: src_id|5efc96f8-2e8b969a-c5f986d8-415e82b8_0 is DEAD
2025-10-09T21:49:50.578388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5efc96f8-2e8b969a-c5f986d8-415e82b8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:50.578399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5efc96f8-2e8b969a-c5f986d8-415e82b8_0] Write session will restart in 0.005979s
2025-10-09T21:49:50.578450Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5efc96f8-2e8b969a-c5f986d8-415e82b8_0] Write session: Do CDS request
2025-10-09T21:49:50.578458Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5efc96f8-2e8b969a-c5f986d8-415e82b8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:50.578546Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:50.585456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5efc96f8-2e8b969a-c5f986d8-415e82b8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:50.585522Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:50.585533Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1641
2025-10-09T21:49:50.585654Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1641 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:50.585678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1641 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:50.585683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1641 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:50.585688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:50.586353Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1641 sessionId:  describe result for acl check
2025-10-09T21:49:50.586393Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:50.586399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:50.586401Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:50.586407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342544736058804:7434] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:50.587000Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342544736058804:7434] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:50.590494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342544736058804:7434] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:50.593006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342544736058804:7434] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:50.593016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342544736058804:7434] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:50.593019Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342544736058804:7434] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:50.593026Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1641 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:50.597084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:50.597568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:50.597730Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:50.597739Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:50.597750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:50.597798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:50.597850Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342544374230184:4256]
2025-10-09T21:49:50.598364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:50.598641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:50.598685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:50.599000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:50.599038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.599045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.599046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.599047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.599048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.599050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.599051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.599052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.599053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.599057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.599067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.599071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.599075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.599078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.599082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.599087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.599090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.599092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.599099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.599107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.599113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.599119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.599125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.599130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.599134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.599140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.599144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.599149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.599153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.599157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.599162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.599167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.599171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.599176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.599181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.599186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.599214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:50.605016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:50.606231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:50.606245Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:50.606247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:50.606251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:50.606257Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1640 [2:7559342544374230184:4256]
2025-10-09T21:49:50.606267Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:50.606281Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:50.606283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:50.606285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:50.606286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:50.606298Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:50.606306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:50.606312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:50.606349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.606355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.606356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.606357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:50.606405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:50.606412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.606418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.606424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.606437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:50.606453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342544736058995:7434], now have 1 active actors on pipe
2025-10-09T21:49:50.606462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342544374230179:4256]
2025-10-09T21:49:50.606464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342544374230179:4256]
2025-10-09T21:49:50.606465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342544374230179:4256]
2025-10-09T21:49:50.606474Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:50.606484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342544736058773:2432], now have 1 active actors on pipe
2025-10-09T21:49:50.606484Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:50.606487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:50.606489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:50.606495Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:50.606507Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:50.606510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:50.606533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342544374230179:4256]
2025-10-09T21:49:50.606540Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:50.606542Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:50.606509Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1640
2025-10-09T21:49:50.606528Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1640
2025-10-09T21:49:50.606557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.606564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.606595Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|aeeb37c-e064fe72-977197cd-29fc2dfe_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:50.606604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:50.606606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:50.606608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:50.606615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:50.606622Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:50.606635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:50.606789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.606800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.606815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:50.606906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1641 partition: 0 MaxSeqNo: 25 sessionId: src_id|aeeb37c-e064fe72-977197cd-29fc2dfe_0
2025-10-09T21:49:50.607237Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5efc96f8-2e8b969a-c5f986d8-415e82b8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|aeeb37c-e064fe72-977197cd-29fc2dfe_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:50.609720Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aeeb37c-e064fe72-977197cd-29fc2dfe_0] Write session: try to update token
2025-10-09T21:49:50.609738Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aeeb37c-e064fe72-977197cd-29fc2dfe_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:50.619873Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:50.619894Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:50.619896Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:50.619898Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:50.619907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342544374230179:4256]
2025-10-09T21:49:50.619911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342544374230179:4256]
2025-10-09T21:49:50.619914Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342544374230179:4256]
2025-10-09T21:49:50.619955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:50.619964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.619971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:50.619974Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:50.619981Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:50.619984Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:50.620100Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:50.625347Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1641 sessionId: src_id|aeeb37c-e064fe72-977197cd-29fc2dfe_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:50.627570Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:50.677968Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342544736059009:7774]
2025-10-09T21:49:50.678234Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 1640, pipe: [1:7559342544736059009:7774]
2025-10-09T21:49:50.761711Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:50.762293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.762307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.762343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342544736059009:7774], now have 1 active actors on pipe
2025-10-09T21:49:50.762494Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:50.762592Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:50.762606Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342544736059009:7774]
2025-10-09T21:49:50.762638Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:50.762707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1641 reason: status is not ok: tablet will be restarted right now sessionId: src_id|aeeb37c-e064fe72-977197cd-29fc2dfe_0
2025-10-09T21:49:50.762876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1641 sessionId: src_id|aeeb37c-e064fe72-977197cd-29fc2dfe_0 is DEAD
2025-10-09T21:49:50.763037Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aeeb37c-e064fe72-977197cd-29fc2dfe_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:50.763050Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aeeb37c-e064fe72-977197cd-29fc2dfe_0] Write session will restart in 0.007008s
2025-10-09T21:49:50.763087Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aeeb37c-e064fe72-977197cd-29fc2dfe_0] Write session: Do CDS request
2025-10-09T21:49:50.763095Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aeeb37c-e064fe72-977197cd-29fc2dfe_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:50.771468Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aeeb37c-e064fe72-977197cd-29fc2dfe_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:50.771545Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:50.771559Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1642
2025-10-09T21:49:50.771657Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1642 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:50.771687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1642 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:50.771697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1642 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:50.771700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:50.772147Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1642 sessionId:  describe result for acl check
2025-10-09T21:49:50.772180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:50.772188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:50.772190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:50.772197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342544736059086:7427] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:50.772653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342544736059086:7427] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:50.774749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342544736059086:7427] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:50.776892Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342544736059086:7427] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:50.776902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342544736059086:7427] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:50.776905Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342544736059086:7427] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:50.776911Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1642 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:50.777708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:50.780686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:50.780856Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:50.780865Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:50.780877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:50.780931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:50.780969Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342544374230406:4251]
2025-10-09T21:49:50.781498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:50.781819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:50.781865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:50.782199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:50.782251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.782257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.782259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.782260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.782261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.782262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.782263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.782265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.782266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.782271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.782280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.782285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.782289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.782293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.782297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.782301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.782304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.782308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.782315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.782323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.782331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.782337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.782341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.782346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.782351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.782356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.782364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.782369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.782374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.782381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.782384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.782387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.782390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.782395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.782400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.782406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.782436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:50.788608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:50.789731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:50.789745Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:50.789746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:50.789749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:50.789754Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1641 [2:7559342544374230406:4251]
2025-10-09T21:49:50.789764Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:50.789781Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:50.789783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:50.789785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:50.789786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:50.789798Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:50.789806Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:50.789811Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:50.789862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.789868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.789869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.789871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:50.789947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:50.789953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.789960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.789970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.789980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:50.789996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342544736059236:7427], now have 1 active actors on pipe
2025-10-09T21:49:50.790004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342544736059012:2432], now have 1 active actors on pipe
2025-10-09T21:49:50.790010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342544374230401:4251]
2025-10-09T21:49:50.790011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342544374230401:4251]
2025-10-09T21:49:50.790012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342544374230401:4251]
2025-10-09T21:49:50.790022Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:50.790048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:50.790052Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:50.790055Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:50.790056Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:50.790056Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:50.790058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:50.790080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342544374230401:4251]
2025-10-09T21:49:50.790083Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:50.790074Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1641
2025-10-09T21:49:50.790103Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:50.790093Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1641
2025-10-09T21:49:50.790447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.790460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.790487Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|35aab186-2d5ed6be-67a17aff-d9d302d8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:50.790500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:50.790505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:50.790507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:50.790518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:50.790533Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:50.790550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:50.790726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.790737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.790752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:50.790851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1642 partition: 0 MaxSeqNo: 25 sessionId: src_id|35aab186-2d5ed6be-67a17aff-d9d302d8_0
2025-10-09T21:49:50.791186Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aeeb37c-e064fe72-977197cd-29fc2dfe_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|35aab186-2d5ed6be-67a17aff-d9d302d8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:50.793728Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|35aab186-2d5ed6be-67a17aff-d9d302d8_0] Write session: try to update token
2025-10-09T21:49:50.793745Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|35aab186-2d5ed6be-67a17aff-d9d302d8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:50.804369Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:50.804388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:50.804390Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:50.804391Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:50.804399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342544374230401:4251]
2025-10-09T21:49:50.804402Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342544374230401:4251]
2025-10-09T21:49:50.804405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342544374230401:4251]
2025-10-09T21:49:50.804439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:50.804444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.804446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:50.804452Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:50.804458Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:50.804462Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:50.804570Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:50.807163Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1642 sessionId: src_id|35aab186-2d5ed6be-67a17aff-d9d302d8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:50.809587Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:50.862825Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342544736059276:7774]
2025-10-09T21:49:50.863079Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 1641, pipe: [1:7559342544736059276:7774]
2025-10-09T21:49:50.883263Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:49:50.883284Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 14
2025-10-09T21:49:50.955026Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:50.955376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.955390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.955431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342544736059276:7774], now have 1 active actors on pipe
2025-10-09T21:49:50.955695Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:50.955699Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:50.955703Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:50.955714Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 14
2025-10-09T21:49:50.955750Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342544736059276:7774]
2025-10-09T21:49:50.955797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1642 reason: error at write: Disconnected sessionId: src_id|35aab186-2d5ed6be-67a17aff-d9d302d8_0
2025-10-09T21:49:50.955854Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:50.955974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1642 sessionId: src_id|35aab186-2d5ed6be-67a17aff-d9d302d8_0 is DEAD
2025-10-09T21:49:50.956150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35aab186-2d5ed6be-67a17aff-d9d302d8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:50.956163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35aab186-2d5ed6be-67a17aff-d9d302d8_0] Write session will restart in 0.006287s
2025-10-09T21:49:50.956188Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35aab186-2d5ed6be-67a17aff-d9d302d8_0] Write session: Do CDS request
2025-10-09T21:49:50.956193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35aab186-2d5ed6be-67a17aff-d9d302d8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:50.963572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|35aab186-2d5ed6be-67a17aff-d9d302d8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:50.963638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:50.963653Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1643
2025-10-09T21:49:50.963811Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1643 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:50.963847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1643 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:50.963850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1643 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:50.963854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:50.964265Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1643 sessionId:  describe result for acl check
2025-10-09T21:49:50.964306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:50.964312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:50.964317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:50.964322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342544736059325:7409] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:50.964874Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342544736059325:7409] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:50.968101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342544736059325:7409] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:50.971169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342544736059325:7409] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:50.971185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342544736059325:7409] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:50.971189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342544736059325:7409] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:50.971198Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1643 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:50.971225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:50.974700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:50.974866Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:50.974874Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:50.974883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:50.974920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:50.974950Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342544374230664:4231]
2025-10-09T21:49:50.975388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:50.975687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:50.975720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:50.975940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:50.975979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.975983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.975984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.975985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.975985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.975986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.975987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.975987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.975988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.975991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.975998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.976001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.976003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.976005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.976007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.976009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.976011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.976013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.976018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.976023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:50.976030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.976035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:50.976039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.976043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:50.976047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.976052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:50.976055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.976059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:50.976065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.976075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:50.976079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.976088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.976095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.976104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.976109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.976114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.976134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:50.983054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:50.984234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:50.984254Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:50.984257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:50.984261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:50.984268Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1642 [2:7559342544374230664:4231]
2025-10-09T21:49:50.984282Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:50.984301Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:50.984304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:50.984307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:50.984310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:50.984328Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:50.984340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:50.984347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:50.984415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.984421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.984426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.984429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:50.984501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:50.984509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.984521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:50.984536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:50.984553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:50.984580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342544736059504:7409], now have 1 active actors on pipe
2025-10-09T21:49:50.984588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342544736059281:2432], now have 1 active actors on pipe
2025-10-09T21:49:50.984596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342544374230658:4231]
2025-10-09T21:49:50.984603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342544374230658:4231]
2025-10-09T21:49:50.984605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342544374230658:4231]
2025-10-09T21:49:50.984627Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:50.984645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:50.984646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:50.984651Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:50.984653Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:50.984668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:50.984671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:50.984705Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342544374230658:4231]
2025-10-09T21:49:50.984718Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:50.984705Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1642
2025-10-09T21:49:50.984713Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1642
2025-10-09T21:49:50.984739Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:50.985223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.985237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.985307Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:49:50.985346Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|25a8f095-d30307c-70513ba2-df38b777_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:50.985360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:50.985369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:50.985373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:50.985385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:50.985397Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:50.985412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:50.985571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:50.985579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:50.985593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:50.985581Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 18 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:49:50.985620Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:49:50.985679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1643 partition: 0 MaxSeqNo: 25 sessionId: src_id|25a8f095-d30307c-70513ba2-df38b777_0
2025-10-09T21:49:50.986065Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35aab186-2d5ed6be-67a17aff-d9d302d8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|25a8f095-d30307c-70513ba2-df38b777_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:50.988496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25a8f095-d30307c-70513ba2-df38b777_0] Write session: try to update token
2025-10-09T21:49:50.988513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25a8f095-d30307c-70513ba2-df38b777_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:50.999503Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:50.999521Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:50.999524Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:50.999526Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:50.999536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342544374230658:4231]
2025-10-09T21:49:50.999540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342544374230658:4231]
2025-10-09T21:49:50.999544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342544374230658:4231]
2025-10-09T21:49:50.999587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:50.999594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:50.999593Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:50.999597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:50.999600Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:50.999604Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:50.999726Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:51.001904Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1643 sessionId: src_id|25a8f095-d30307c-70513ba2-df38b777_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:51.004305Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:51.055878Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342549031026816:7774]
2025-10-09T21:49:51.056167Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 1642, pipe: [1:7559342549031026816:7774]
2025-10-09T21:49:51.153494Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:51.153801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:51.153817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:51.153863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342549031026816:7774], now have 1 active actors on pipe
2025-10-09T21:49:51.154100Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:51.154201Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342549031026816:7774]
2025-10-09T21:49:51.154202Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:51.154216Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 0
2025-10-09T21:49:51.154242Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:51.154330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1643 reason: status is not ok: tablet will be restarted right now sessionId: src_id|25a8f095-d30307c-70513ba2-df38b777_0
2025-10-09T21:49:51.154544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1643 sessionId: src_id|25a8f095-d30307c-70513ba2-df38b777_0 is DEAD
2025-10-09T21:49:51.154692Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25a8f095-d30307c-70513ba2-df38b777_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:51.154702Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25a8f095-d30307c-70513ba2-df38b777_0] Write session will restart in 0.008773s
2025-10-09T21:49:51.154739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25a8f095-d30307c-70513ba2-df38b777_0] Write session: Do CDS request
2025-10-09T21:49:51.154746Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25a8f095-d30307c-70513ba2-df38b777_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:51.164617Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25a8f095-d30307c-70513ba2-df38b777_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:51.164672Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:51.164697Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1644
2025-10-09T21:49:51.164864Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1644 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:51.164919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1644 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:51.164942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1644 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:51.164946Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:51.165485Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1644 sessionId:  describe result for acl check
2025-10-09T21:49:51.165523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:51.165530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:51.165532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:51.165551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342549031026890:7401] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:51.166157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342549031026890:7401] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:51.168949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342549031026890:7401] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:51.171768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342549031026890:7401] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:51.171783Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342549031026890:7401] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:51.171787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342549031026890:7401] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:51.171797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1644 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:51.172410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:51.175229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:51.175420Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:51.175430Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:51.175441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:51.175495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:51.175546Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342548669198181:4225]
2025-10-09T21:49:51.176176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:51.176477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:51.176535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:51.176860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:51.176909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:51.176916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:51.176918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:51.176919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:51.176920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:51.176921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:51.176922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.176923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.176925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.176929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:51.176938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:51.176940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:51.176942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:51.176944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:51.176947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:51.176949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.176951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.176953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.176960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:51.176965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:51.176970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:51.176974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:51.176978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:51.176984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:51.176988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:51.176993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:51.176998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:51.177003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:51.177007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:51.177013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:51.177018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.177021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.177024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.177027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.177029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.177032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.177051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:51.184076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:51.185683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:51.185703Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:51.185705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:51.185710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:51.185721Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1643 [2:7559342548669198181:4225]
2025-10-09T21:49:51.185737Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:51.185753Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:51.185761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:51.185763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:51.185765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:51.185780Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:51.185785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:51.185790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:51.185838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.185843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.185845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.185847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:51.185903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:51.185910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.185919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.185925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.185935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:51.185951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342548669198176:4225]
2025-10-09T21:49:51.185952Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342548669198176:4225]
2025-10-09T21:49:51.185954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342548669198176:4225]
2025-10-09T21:49:51.185960Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:51.185974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342549031027049:7401], now have 1 active actors on pipe
2025-10-09T21:49:51.185983Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:51.185989Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:51.185992Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:51.185994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:51.185999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342549031026821:2432], now have 1 active actors on pipe
2025-10-09T21:49:51.186000Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:51.186003Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:51.186050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342548669198176:4225]
2025-10-09T21:49:51.186065Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:51.186084Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1643
2025-10-09T21:49:51.186097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:51.186096Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1643
2025-10-09T21:49:51.186202Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:49:51.186236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:51.186243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:51.186263Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b0534ce1-792402ae-d4d0319e-ca7c8805_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:51.186273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:51.186276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:51.186278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:51.186286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:51.186294Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:51.186308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:51.186373Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 19 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:49:51.186410Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:49:51.186503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:51.186517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:51.186537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:51.186633Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1644 partition: 0 MaxSeqNo: 25 sessionId: src_id|b0534ce1-792402ae-d4d0319e-ca7c8805_0
2025-10-09T21:49:51.187002Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25a8f095-d30307c-70513ba2-df38b777_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b0534ce1-792402ae-d4d0319e-ca7c8805_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:51.189500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0534ce1-792402ae-d4d0319e-ca7c8805_0] Write session: try to update token
2025-10-09T21:49:51.189524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0534ce1-792402ae-d4d0319e-ca7c8805_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:51.198069Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:51.198090Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:51.198101Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342548669198176:4225]
2025-10-09T21:49:51.198149Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:51.198152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:51.198511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:51.198656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:51.198798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:51.198937Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:51.199220Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:51.199267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:51.202812Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1644 sessionId: src_id|b0534ce1-792402ae-d4d0319e-ca7c8805_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:51.205126Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:51.205313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:51.205334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:51.205375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:51.205469Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:51.211174Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:51.211196Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:51.211200Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:51.211202Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:51.211213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342548669198176:4225]
2025-10-09T21:49:51.211218Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342548669198176:4225]
2025-10-09T21:49:51.211222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342548669198176:4225]
2025-10-09T21:49:51.211271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:51.211278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.211282Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:51.211288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:51.211290Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:51.211294Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:51.211480Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:51.254502Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342549031027070:7774]
2025-10-09T21:49:51.254756Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 1643, pipe: [1:7559342549031027070:7774]
2025-10-09T21:49:51.361258Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:51.368440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:51.368465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:51.368611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.368672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.368730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.368787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.368832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.368838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.368907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.368963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.369023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.369082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.369123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.369132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.369191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.369271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.369356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.369418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.369458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.369464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.369521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.369596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.369654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.369715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.369755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.369760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.369821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.369874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.369936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.370015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.370087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.370094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.370172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.370228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.370282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.370336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.370449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.370457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.370539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.370595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.370650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.370712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.370751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.370756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.370810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.370861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.370914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.371009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.371054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.371059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.371113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.371177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.371237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.371291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.371329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.371335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.371387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.371440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.371502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.371555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.371609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.371615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.371657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342549031027070:7774], now have 1 active actors on pipe
2025-10-09T21:49:51.371873Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:51.371887Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:51.371895Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:51.371920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1644 reason: error at write: Disconnected sessionId: src_id|b0534ce1-792402ae-d4d0319e-ca7c8805_0
2025-10-09T21:49:51.371942Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342549031027070:7774]
2025-10-09T21:49:51.372110Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1644 sessionId: src_id|b0534ce1-792402ae-d4d0319e-ca7c8805_0 is DEAD
2025-10-09T21:49:51.372254Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0534ce1-792402ae-d4d0319e-ca7c8805_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:51.372267Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0534ce1-792402ae-d4d0319e-ca7c8805_0] Write session will restart in 0.005605s
2025-10-09T21:49:51.372292Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0534ce1-792402ae-d4d0319e-ca7c8805_0] Write session: Do CDS request
2025-10-09T21:49:51.372298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0534ce1-792402ae-d4d0319e-ca7c8805_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:51.372579Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:51.379551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0534ce1-792402ae-d4d0319e-ca7c8805_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:51.379672Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:51.379686Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1645
2025-10-09T21:49:51.379820Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1645 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:51.379854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1645 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:51.379864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1645 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:51.379868Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:51.380322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1645 sessionId:  describe result for acl check
2025-10-09T21:49:51.380359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:51.380365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:51.380367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:51.380375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342549031027122:7398] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:51.380940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342549031027122:7398] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:51.384001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342549031027122:7398] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:51.386905Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342549031027122:7398] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:51.386920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342549031027122:7398] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:51.386923Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342549031027122:7398] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:51.386932Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1645 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:51.388219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:51.391496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:51.391717Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:51.391726Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:51.391736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:51.391779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:51.391814Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342548669198403:4212]
2025-10-09T21:49:51.392228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:51.392524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:51.392559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:51.392810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:51.392849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:51.392854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:51.392855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:51.392855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:51.392856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:51.392857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:51.392858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.392858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.392859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.392862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:51.392870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:51.392872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:51.392874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:51.392876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:51.392879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:51.392881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.392883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.392885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.392890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:51.392896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:51.392900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:51.392905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:51.392909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:51.392914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:51.392918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:51.392923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:51.392928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:51.392933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:51.392937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:51.392943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:51.392948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.392953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.392957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.392962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.392967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.392970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.392986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:51.399816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:51.401375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:51.401394Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:51.401397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:51.401401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:51.401409Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1644 [2:7559342548669198403:4212]
2025-10-09T21:49:51.401425Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:51.401445Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:51.401453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:51.401457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:51.401459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:51.401478Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:51.401489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:51.401500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:51.401584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.401591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.401593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.401596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:51.401656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:51.401661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.401670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.401687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.401709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:51.401737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342548669198398:4212]
2025-10-09T21:49:51.401739Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342548669198398:4212]
2025-10-09T21:49:51.401742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342548669198398:4212]
2025-10-09T21:49:51.401751Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:51.401761Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:51.401768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342549031027305:7398], now have 1 active actors on pipe
2025-10-09T21:49:51.401770Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:51.401772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342549031027079:2432], now have 1 active actors on pipe
2025-10-09T21:49:51.401772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:51.401781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:51.401797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:51.401800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:51.401839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342548669198398:4212]
2025-10-09T21:49:51.401867Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:51.401889Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:51.401867Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1644
2025-10-09T21:49:51.401946Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1644
2025-10-09T21:49:51.402020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:51.402045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:51.402081Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|549570fb-5ec0e2da-6119dee4-5281f286_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:51.402095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:51.402104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:51.402107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:51.402120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:51.402137Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:51.402158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:51.402308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:51.402330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:51.402359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:51.402455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1645 partition: 0 MaxSeqNo: 25 sessionId: src_id|549570fb-5ec0e2da-6119dee4-5281f286_0
2025-10-09T21:49:51.402821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0534ce1-792402ae-d4d0319e-ca7c8805_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|549570fb-5ec0e2da-6119dee4-5281f286_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:51.405325Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|549570fb-5ec0e2da-6119dee4-5281f286_0] Write session: try to update token
2025-10-09T21:49:51.405349Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|549570fb-5ec0e2da-6119dee4-5281f286_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:51.413210Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:51.413231Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:51.413242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342548669198398:4212]
2025-10-09T21:49:51.413284Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:51.413295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:51.413471Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:51.413641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:51.413779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:51.413920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:51.414133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:51.414179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:51.421412Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1645 sessionId: src_id|549570fb-5ec0e2da-6119dee4-5281f286_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:51.424045Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:51.424255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:51.424286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:51.424322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:51.424386Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:51.426955Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:51.426971Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:51.426973Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:51.426974Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:51.426983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342548669198398:4212]
2025-10-09T21:49:51.426986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342548669198398:4212]
2025-10-09T21:49:51.426989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342548669198398:4212]
2025-10-09T21:49:51.427027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:51.427032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.427034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:51.427034Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:51.427041Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:51.427044Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:51.427162Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:51.472126Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342549031027319:7774]
2025-10-09T21:49:51.472405Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 1644, pipe: [1:7559342549031027319:7774]
2025-10-09T21:49:51.578591Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:51.585123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:51.585151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:51.585291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.585347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.585404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.585472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.585516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.585525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.585594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.585649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.585705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.585767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.585810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.585812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.585867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.585916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.585969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.586041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.586080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.586086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.586159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.586218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.586275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.586330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.586369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.586375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.586431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.586488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.586544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.586596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.586634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.586639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.586723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.586807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.586860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.586934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.586984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.586991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.587048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.587100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.587183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.587256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.587311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.587317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.587369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.587416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.587466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.587522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.587562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.587569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.587623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.587672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.587721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.587782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.587822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.587827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.587881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.587929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.587978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.588037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.588078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.588083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.588125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342549031027319:7774], now have 1 active actors on pipe
2025-10-09T21:49:51.588372Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:51.588376Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:51.588380Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:51.588414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1645 reason: error at write: Disconnected sessionId: src_id|549570fb-5ec0e2da-6119dee4-5281f286_0
2025-10-09T21:49:51.588439Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342549031027319:7774]
2025-10-09T21:49:51.588628Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1645 sessionId: src_id|549570fb-5ec0e2da-6119dee4-5281f286_0 is DEAD
2025-10-09T21:49:51.588731Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|549570fb-5ec0e2da-6119dee4-5281f286_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:51.588740Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|549570fb-5ec0e2da-6119dee4-5281f286_0] Write session will restart in 0.006103s
2025-10-09T21:49:51.588767Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|549570fb-5ec0e2da-6119dee4-5281f286_0] Write session: Do CDS request
2025-10-09T21:49:51.588772Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|549570fb-5ec0e2da-6119dee4-5281f286_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:51.588952Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:51.595507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|549570fb-5ec0e2da-6119dee4-5281f286_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:51.595576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:51.595589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1646
2025-10-09T21:49:51.595698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1646 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:51.595730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1646 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:51.595740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1646 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:51.595748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:51.596205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1646 sessionId:  describe result for acl check
2025-10-09T21:49:51.596238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:51.596240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:51.596242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:51.596248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342549031027337:7390] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:51.596792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342549031027337:7390] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:51.600002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342549031027337:7390] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:51.603061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342549031027337:7390] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:51.603076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342549031027337:7390] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:51.603081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342549031027337:7390] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:51.603090Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1646 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:51.605405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:51.608528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:51.608748Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:51.608757Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:51.608769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:51.608820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:51.608862Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342548669198626:4209]
2025-10-09T21:49:51.609481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:51.609742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:51.609782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:51.610104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:51.610153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:51.610160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:51.610161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:51.610163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:51.610164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:51.610165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:51.610166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.610168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.610169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.610174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:51.610185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:51.610190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:51.610194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:51.610198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:51.610202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:51.610206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.610210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.610214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.610221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:51.610229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:51.610236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:51.610242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:51.610246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:51.610252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:51.610256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:51.610261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:51.610266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:51.610271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:51.610275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:51.610280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:51.610285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.610290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.610294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.610299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.610304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.610309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.610340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:51.616662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:51.617659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:51.617676Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:51.617679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:51.617684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:51.617691Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1645 [2:7559342548669198626:4209]
2025-10-09T21:49:51.617709Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:51.617726Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:51.617732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:51.617735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:51.617736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:51.617751Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:51.617759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:51.617765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:51.617813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.617818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.617820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.617822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:51.617888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:51.617896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.617908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.617921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.617933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:51.617951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342549031027548:7390], now have 1 active actors on pipe
2025-10-09T21:49:51.617957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342549031027322:2432], now have 1 active actors on pipe
2025-10-09T21:49:51.617962Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342548669198621:4209]
2025-10-09T21:49:51.617964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342548669198621:4209]
2025-10-09T21:49:51.617965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342548669198621:4209]
2025-10-09T21:49:51.617975Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:51.617988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:51.617995Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:51.617999Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:51.618000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:51.618000Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:51.618002Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:51.618041Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342548669198621:4209]
2025-10-09T21:49:51.618053Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:51.618048Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1645
2025-10-09T21:49:51.618050Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1645
2025-10-09T21:49:51.618084Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:51.618166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:51.618178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:51.618210Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7ad6693d-99b5677c-60d74f90-cbef1692_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:51.618222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:51.618231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:51.618234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:51.618245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:51.618260Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:51.618278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:51.618432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:51.618443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:51.618458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:51.618539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1646 partition: 0 MaxSeqNo: 25 sessionId: src_id|7ad6693d-99b5677c-60d74f90-cbef1692_0
2025-10-09T21:49:51.618856Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|549570fb-5ec0e2da-6119dee4-5281f286_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7ad6693d-99b5677c-60d74f90-cbef1692_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:51.621292Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ad6693d-99b5677c-60d74f90-cbef1692_0] Write session: try to update token
2025-10-09T21:49:51.621310Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ad6693d-99b5677c-60d74f90-cbef1692_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:51.629807Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:51.629833Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:51.629844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342548669198621:4209]
2025-10-09T21:49:51.629887Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:51.629898Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:51.630091Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:51.630237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:51.630361Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:51.630481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:51.630656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:51.630700Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:51.634877Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1646 sessionId: src_id|7ad6693d-99b5677c-60d74f90-cbef1692_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:51.637258Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:51.637475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:51.637501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:51.637551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:51.637653Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:51.641983Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:51.642000Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:51.642003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:51.642005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:51.642015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342548669198621:4209]
2025-10-09T21:49:51.642019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342548669198621:4209]
2025-10-09T21:49:51.642023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342548669198621:4209]
2025-10-09T21:49:51.642076Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:51.642079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:51.642087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.642089Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:51.642090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:51.642093Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:51.642240Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:51.688696Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342549031027570:7774]
2025-10-09T21:49:51.688931Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 1645, pipe: [1:7559342549031027570:7774]
2025-10-09T21:49:51.783276Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:51.789938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:51.789963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:51.790128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.790191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.790263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.790327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.790379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.790386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.790455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.790514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.790572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.790639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.790677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.790684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.790781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.790837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.790892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.790947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.790985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.790996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.791057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.791112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.791168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.791226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.791268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.791276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.791333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.791385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.791440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.791494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.791543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.791551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.791605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.791656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.791707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.791767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.791827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.791833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.791886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.791933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.791989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.792047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.792107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.792115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.792191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.792285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.792376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.792429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.792503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.792513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.792567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.792617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.792669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.792720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.792758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.792764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.792825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:51.792876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:51.792928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:51.792984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:51.793022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:51.793027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:51.793076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342549031027570:7774], now have 1 active actors on pipe
2025-10-09T21:49:51.793341Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:51.793345Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342549031027570:7774]
2025-10-09T21:49:51.793355Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:51.793366Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:51.793395Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1646 reason: error at write: Disconnected sessionId: src_id|7ad6693d-99b5677c-60d74f90-cbef1692_0
2025-10-09T21:49:51.793589Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1646 sessionId: src_id|7ad6693d-99b5677c-60d74f90-cbef1692_0 is DEAD
2025-10-09T21:49:51.793717Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ad6693d-99b5677c-60d74f90-cbef1692_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:51.793729Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ad6693d-99b5677c-60d74f90-cbef1692_0] Write session will restart in 0.006854s
2025-10-09T21:49:51.793753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ad6693d-99b5677c-60d74f90-cbef1692_0] Write session: Do CDS request
2025-10-09T21:49:51.793760Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ad6693d-99b5677c-60d74f90-cbef1692_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:51.793943Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:51.801569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ad6693d-99b5677c-60d74f90-cbef1692_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:51.801684Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:51.801697Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1647
2025-10-09T21:49:51.801803Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1647 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:51.801829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1647 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:51.801836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1647 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:51.801840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:51.802238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1647 sessionId:  describe result for acl check
2025-10-09T21:49:51.802277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:51.802283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:51.802287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:51.802293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342549031027627:7375] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:51.802902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342549031027627:7375] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:51.807365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342549031027627:7375] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:51.810851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342549031027627:7375] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:51.810865Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342549031027627:7375] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:51.810869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342549031027627:7375] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:51.810879Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1647 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:51.815494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:51.816050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:51.816238Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:51.816249Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:51.816261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:51.816317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:51.816361Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342548669198854:4200]
2025-10-09T21:49:51.816919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:51.817202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:51.817246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:51.817544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:51.817593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:51.817598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:51.817600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:51.817601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:51.817603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:51.817604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:51.817605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.817607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.817608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.817612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:51.817623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:51.817627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:51.817630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:51.817634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:51.817638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:51.817642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.817645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.817648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.817654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:51.817662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:51.817669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:51.817674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:51.817678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:51.817683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:51.817687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:51.817692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:51.817696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:51.817700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:51.817704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:51.817709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:51.817714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.817719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.817731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.817736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.817740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.817746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.817776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:51.824765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:51.826146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:51.826159Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:51.826161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:51.826166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:51.826172Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1646 [2:7559342548669198854:4200]
2025-10-09T21:49:51.826185Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:51.826202Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:51.826210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:51.826213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:51.826216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:51.826233Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:51.826239Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:51.826246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:51.826302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.826310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.826312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.826315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:51.826385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:51.826392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.826403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:51.826417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:51.826436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:51.826460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342549031027578:2432], now have 1 active actors on pipe
2025-10-09T21:49:51.826468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342549031027807:7375], now have 1 active actors on pipe
2025-10-09T21:49:51.826476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342548669198849:4200]
2025-10-09T21:49:51.826479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342548669198849:4200]
2025-10-09T21:49:51.826481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342548669198849:4200]
2025-10-09T21:49:51.826493Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:51.826509Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:51.826511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:51.826519Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:51.826521Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:51.826524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:51.826527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:51.826555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342548669198849:4200]
2025-10-09T21:49:51.826562Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:51.826565Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:51.826537Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1646
2025-10-09T21:49:51.826550Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1646
2025-10-09T21:49:51.826591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:51.826600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:51.826632Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b5000e40-83ef8283-9be030af-f7dddf8c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:51.826645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:51.826648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:51.826651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:51.826661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:51.826670Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:51.826693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:51.826881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:51.826890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:51.826905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:51.827030Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1647 partition: 0 MaxSeqNo: 25 sessionId: src_id|b5000e40-83ef8283-9be030af-f7dddf8c_0
2025-10-09T21:49:51.827403Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ad6693d-99b5677c-60d74f90-cbef1692_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b5000e40-83ef8283-9be030af-f7dddf8c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:51.829776Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5000e40-83ef8283-9be030af-f7dddf8c_0] Write session: try to update token
2025-10-09T21:49:51.829795Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5000e40-83ef8283-9be030af-f7dddf8c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:51.841723Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:51.841738Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:51.841741Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:51.841743Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:51.841752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342548669198849:4200]
2025-10-09T21:49:51.841756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342548669198849:4200]
2025-10-09T21:49:51.841760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342548669198849:4200]
2025-10-09T21:49:51.841804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:51.841807Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:51.841811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:51.841812Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:51.841813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:51.841817Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:51.841944Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:51.842484Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1647 sessionId: src_id|b5000e40-83ef8283-9be030af-f7dddf8c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:51.845468Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:51.893515Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342549031027845:7774]
2025-10-09T21:49:51.893861Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 1646, pipe: [1:7559342549031027845:7774]
2025-10-09T21:49:51.994285Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:51.994826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:51.994844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:51.994889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342549031027845:7774], now have 1 active actors on pipe
2025-10-09T21:49:51.995103Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:51.995144Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:51.995166Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:51.995183Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342549031027845:7774]
2025-10-09T21:49:51.995210Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:51.995220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1647 reason: error at write: Disconnected sessionId: src_id|b5000e40-83ef8283-9be030af-f7dddf8c_0
2025-10-09T21:49:51.995391Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1647 sessionId: src_id|b5000e40-83ef8283-9be030af-f7dddf8c_0 is DEAD
2025-10-09T21:49:51.995533Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5000e40-83ef8283-9be030af-f7dddf8c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:51.995544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5000e40-83ef8283-9be030af-f7dddf8c_0] Write session will restart in 0.006689s
2025-10-09T21:49:51.995571Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5000e40-83ef8283-9be030af-f7dddf8c_0] Write session: Do CDS request
2025-10-09T21:49:51.995577Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5000e40-83ef8283-9be030af-f7dddf8c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:52.003552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5000e40-83ef8283-9be030af-f7dddf8c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:52.003750Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:52.003767Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1648
2025-10-09T21:49:52.003909Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1648 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:52.003943Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1648 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:52.003954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1648 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:52.003961Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:52.004449Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1648 sessionId:  describe result for acl check
2025-10-09T21:49:52.004497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:52.004505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:52.004507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:52.004512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342553325995229:7361] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:52.005059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342553325995229:7361] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:52.007833Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342553325995229:7361] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:52.010212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:52.010568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342553325995229:7361] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:52.010583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342553325995229:7361] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:52.010586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342553325995229:7361] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:52.010592Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1648 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:52.013723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:52.013890Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:52.013899Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:52.013912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:52.013963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:52.013995Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342552964166412:4189]
2025-10-09T21:49:52.014412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:52.014678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:52.014712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:52.014920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:52.014965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.014970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.014970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.014971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.014972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.014973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.014973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.014974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.014975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.014978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.014986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.014988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.014990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.014993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.014995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.014997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.014999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.015001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.015006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.015011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.015016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.015019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.015022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.015025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.015028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.015031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.015033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.015037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.015039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.015043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.015047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.015050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.015052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.015055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.015058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.015061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.015078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:52.021948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:52.022944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:52.022959Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:52.022961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:52.022964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:52.022969Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1647 [2:7559342552964166412:4189]
2025-10-09T21:49:52.022981Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:52.022996Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:52.022999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:52.023001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:52.023002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:52.023016Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:52.023025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:52.023032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:52.023081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.023086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.023088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.023090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:52.023144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:52.023149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.023158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.023164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.023178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:52.023197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342552964166406:4189]
2025-10-09T21:49:52.023203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342552964166406:4189]
2025-10-09T21:49:52.023205Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342552964166406:4189]
2025-10-09T21:49:52.023211Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:52.023222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:52.023224Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:52.023228Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:52.023230Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:52.023235Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:52.023238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:52.023247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342553325995374:7361], now have 1 active actors on pipe
2025-10-09T21:49:52.023253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342549031027849:2432], now have 1 active actors on pipe
2025-10-09T21:49:52.023284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342552964166406:4189]
2025-10-09T21:49:52.023295Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:52.023313Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:52.023307Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1647
2025-10-09T21:49:52.023314Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1647
2025-10-09T21:49:52.023383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:52.023392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:52.023417Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2a8f45e-f4a18b11-141dfe0b-4d759a37_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:52.023424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:52.023427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:52.023428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:52.023436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:52.023443Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:52.023456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:52.023644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:52.023661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:52.023684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:52.023798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1648 partition: 0 MaxSeqNo: 25 sessionId: src_id|2a8f45e-f4a18b11-141dfe0b-4d759a37_0
2025-10-09T21:49:52.024159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5000e40-83ef8283-9be030af-f7dddf8c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2a8f45e-f4a18b11-141dfe0b-4d759a37_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:52.026721Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a8f45e-f4a18b11-141dfe0b-4d759a37_0] Write session: try to update token
2025-10-09T21:49:52.026742Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a8f45e-f4a18b11-141dfe0b-4d759a37_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:52.035431Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:52.035453Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:52.035461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342552964166406:4189]
2025-10-09T21:49:52.035501Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:52.035502Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:52.035671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:52.035809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:52.035941Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:52.036070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:52.036246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:52.036283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:52.039514Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1648 sessionId: src_id|2a8f45e-f4a18b11-141dfe0b-4d759a37_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:52.041797Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:52.041977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:52.041998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:52.042052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:52.042132Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:52.048489Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:52.048507Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:52.048510Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:52.048511Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:52.048522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342552964166406:4189]
2025-10-09T21:49:52.048526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342552964166406:4189]
2025-10-09T21:49:52.048530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342552964166406:4189]
2025-10-09T21:49:52.048585Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:52.048586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:52.048595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.048598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:52.048599Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:52.048602Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:52.048767Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:52.095210Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342553325995386:7774]
2025-10-09T21:49:52.095419Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 1647, pipe: [1:7559342553325995386:7774]
2025-10-09T21:49:52.198633Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:52.206244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:52.206269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:52.206413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:52.206469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:52.206538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:52.206597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:52.206641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:52.206649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:52.206710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:52.206763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:52.206824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:52.206882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:52.206923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:52.206929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:52.206983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:52.207043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:52.207107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:52.207160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:52.207199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:52.207204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:52.207263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:52.207332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:52.207396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:52.207462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:52.207506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:52.207514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:52.207576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:52.207628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:52.207688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:52.207747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:52.207794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:52.207803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:52.207864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:52.207917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:52.207989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:52.208047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:52.208090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:52.208097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:52.208152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:52.208207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:52.208273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:52.208336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:52.208380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:52.208388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:52.208447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:52.208497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:52.208546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:52.208597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:52.208717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:52.208724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:52.208776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:52.208826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:52.208879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:52.208944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:52.209002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:52.209007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:52.209060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:52.209109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:52.209168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:52.209232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:52.209276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:52.209283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:52.209332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342553325995386:7774], now have 1 active actors on pipe
2025-10-09T21:49:52.209633Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342553325995386:7774]
2025-10-09T21:49:52.209666Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:52.209668Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:52.209699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1648 reason: error at write: Disconnected sessionId: src_id|2a8f45e-f4a18b11-141dfe0b-4d759a37_0
2025-10-09T21:49:52.209722Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:52.209872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1648 sessionId: src_id|2a8f45e-f4a18b11-141dfe0b-4d759a37_0 is DEAD
2025-10-09T21:49:52.210045Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a8f45e-f4a18b11-141dfe0b-4d759a37_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:52.210058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a8f45e-f4a18b11-141dfe0b-4d759a37_0] Write session will restart in 0.007556s
2025-10-09T21:49:52.210084Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a8f45e-f4a18b11-141dfe0b-4d759a37_0] Write session: Do CDS request
2025-10-09T21:49:52.210091Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a8f45e-f4a18b11-141dfe0b-4d759a37_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:52.210559Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:52.218495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a8f45e-f4a18b11-141dfe0b-4d759a37_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:52.218604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:52.218628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1649
2025-10-09T21:49:52.218758Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1649 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:52.218811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1649 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:52.218821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1649 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:52.218825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:52.219311Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1649 sessionId:  describe result for acl check
2025-10-09T21:49:52.219347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:52.219348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:52.219350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:52.219356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342553325995475:7354] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:52.219893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342553325995475:7354] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:52.222851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342553325995475:7354] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:52.225395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:52.229125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:52.229299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342553325995475:7354] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:52.229316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342553325995475:7354] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:52.229321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342553325995475:7354] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:52.229331Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1649 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:52.229373Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:52.229389Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:52.229404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:52.229478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:52.229528Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342552964166637:4183]
2025-10-09T21:49:52.230145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:52.230724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:52.230792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:52.231121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:52.231198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.231209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.231211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.231212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.231213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.231214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.231216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.231217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.231218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.231224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.231236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.231240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.231244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.231248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.231251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.231254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.231258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.231261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.231269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.231278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.231284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.231290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.231294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.231299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.231304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.231310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.231314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.231320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.231324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.231330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.231336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.231342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.231347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.231352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.231357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.231369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.231400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:52.237926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:52.239414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:52.239430Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:52.239433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:52.239437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:52.239446Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1648 [2:7559342552964166637:4183]
2025-10-09T21:49:52.239461Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:52.239479Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:52.239487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:52.239490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:52.239492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:52.239511Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:52.239523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:52.239535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:52.239600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.239609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.239610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.239613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:52.239686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:52.239694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.239706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.239719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.239741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:52.239768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342553325995392:2432], now have 1 active actors on pipe
2025-10-09T21:49:52.239776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342553325995625:7354], now have 1 active actors on pipe
2025-10-09T21:49:52.239784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342552964166631:4183]
2025-10-09T21:49:52.239786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342552964166631:4183]
2025-10-09T21:49:52.239788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342552964166631:4183]
2025-10-09T21:49:52.239803Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:52.239814Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:52.239817Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:52.239818Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:52.239822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:52.239837Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:52.239843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:52.239808Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1648
2025-10-09T21:49:52.239827Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1648
2025-10-09T21:49:52.239886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:52.239895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:52.239907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342552964166631:4183]
2025-10-09T21:49:52.239917Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:52.239922Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:52.239948Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c034900e-be625f38-58767fbd-e023d64f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:52.239960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:52.239964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:52.239967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:52.239977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:52.239992Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:52.240013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:52.240157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:52.240165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:52.240180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:52.240286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1649 partition: 0 MaxSeqNo: 25 sessionId: src_id|c034900e-be625f38-58767fbd-e023d64f_0
2025-10-09T21:49:52.240642Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a8f45e-f4a18b11-141dfe0b-4d759a37_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c034900e-be625f38-58767fbd-e023d64f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:52.242939Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c034900e-be625f38-58767fbd-e023d64f_0] Write session: try to update token
2025-10-09T21:49:52.242953Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c034900e-be625f38-58767fbd-e023d64f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:52.253854Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:52.253875Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:52.253877Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:52.253879Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:52.253889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342552964166631:4183]
2025-10-09T21:49:52.253895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342552964166631:4183]
2025-10-09T21:49:52.253912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342552964166631:4183]
2025-10-09T21:49:52.253958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:52.253970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.253973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:52.253977Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:52.253992Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:52.253996Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:52.254124Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:52.256848Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1649 sessionId: src_id|c034900e-be625f38-58767fbd-e023d64f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:52.259334Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:52.309721Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342553325995641:7774]
2025-10-09T21:49:52.310012Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 1648, pipe: [1:7559342553325995641:7774]
2025-10-09T21:49:52.399219Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:52.399734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:52.399747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:52.399787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342553325995641:7774], now have 1 active actors on pipe
2025-10-09T21:49:52.399943Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:52.400011Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:52.400044Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342553325995641:7774]
2025-10-09T21:49:52.400050Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:52.400128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1649 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c034900e-be625f38-58767fbd-e023d64f_0
2025-10-09T21:49:52.400309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1649 sessionId: src_id|c034900e-be625f38-58767fbd-e023d64f_0 is DEAD
2025-10-09T21:49:52.400448Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c034900e-be625f38-58767fbd-e023d64f_0] Counters: { Errors: 0 CurrentSessionLifetimeMs: 159 BytesWritten: 56320040 MessagesWritten: 25 BytesWrittenCompressed: 0 BytesInflightUncompressed: 22528020 BytesInflightCompressed: 0 BytesInflightTotal: 18022416 MessagesInflight: 10 }
2025-10-09T21:49:52.400481Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c034900e-be625f38-58767fbd-e023d64f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:52.400487Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c034900e-be625f38-58767fbd-e023d64f_0] Write session will restart in 0.009173s
2025-10-09T21:49:52.400506Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c034900e-be625f38-58767fbd-e023d64f_0] Write session: Do CDS request
2025-10-09T21:49:52.400511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c034900e-be625f38-58767fbd-e023d64f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:52.410517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c034900e-be625f38-58767fbd-e023d64f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:52.410614Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:52.410626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1650
2025-10-09T21:49:52.410756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1650 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:52.410799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1650 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:52.410806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1650 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:52.410810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:52.411177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1650 sessionId:  describe result for acl check
2025-10-09T21:49:52.411205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:52.411211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:52.411213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:52.411217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342553325995801:7336] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:52.411688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342553325995801:7336] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:52.413222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342553325995801:7336] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:52.416873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:52.417030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342553325995801:7336] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:52.417050Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342553325995801:7336] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:52.417053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342553325995801:7336] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:52.417060Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1650 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:52.417289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:52.417445Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:52.417456Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:52.417467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:52.417511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:52.417561Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342552964166863:4176]
2025-10-09T21:49:52.418146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:52.418403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:52.418445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:52.418744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:52.418788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.418793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.418795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.418796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.418797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.418798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.418799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.418800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.418801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.418807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.418817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.418822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.418826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.418830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.418834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.418838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.418841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.418845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.418852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.418860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.418870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.418876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.418880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.418884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.418889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.418894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.418901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.418906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.418910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.418915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.418920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.418928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.418932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.418938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.418945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.418950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.418978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:52.425057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:52.426105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:52.426118Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:52.426120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:52.426124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:52.426130Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1649 [2:7559342552964166863:4176]
2025-10-09T21:49:52.426143Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:52.426161Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:52.426169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:52.426172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:52.426174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:52.426190Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:52.426201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:52.426208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:52.426268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.426275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.426277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.426280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:52.426344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:52.426351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.426363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.426374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.426390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:52.426416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342553325995644:2432], now have 1 active actors on pipe
2025-10-09T21:49:52.426424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342553325995878:7336], now have 1 active actors on pipe
2025-10-09T21:49:52.426431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342552964166857:4176]
2025-10-09T21:49:52.426433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342552964166857:4176]
2025-10-09T21:49:52.426436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342552964166857:4176]
2025-10-09T21:49:52.426450Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:52.426444Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1649
2025-10-09T21:49:52.426449Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1649
2025-10-09T21:49:52.426468Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:52.426468Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:52.426472Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:52.426474Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:52.426486Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:52.426490Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:52.426530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:52.426539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:52.426551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342552964166857:4176]
2025-10-09T21:49:52.426563Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:52.426574Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:52.426592Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ea1ea6c3-bd5a1b9e-ecb741ac-47a4dd5a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:52.426603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:52.426607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:52.426610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:52.426621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:52.426634Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:52.426656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:52.426788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:52.426798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:52.426812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:52.426883Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1650 partition: 0 MaxSeqNo: 25 sessionId: src_id|ea1ea6c3-bd5a1b9e-ecb741ac-47a4dd5a_0
2025-10-09T21:49:52.427185Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c034900e-be625f38-58767fbd-e023d64f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ea1ea6c3-bd5a1b9e-ecb741ac-47a4dd5a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:52.429382Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea1ea6c3-bd5a1b9e-ecb741ac-47a4dd5a_0] Write session: try to update token
2025-10-09T21:49:52.429397Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea1ea6c3-bd5a1b9e-ecb741ac-47a4dd5a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:52.441205Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:52.441225Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:52.441227Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:52.441229Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:52.441239Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342552964166857:4176]
2025-10-09T21:49:52.441243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342552964166857:4176]
2025-10-09T21:49:52.441247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342552964166857:4176]
2025-10-09T21:49:52.441291Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:52.441292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:52.441296Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:52.441298Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:52.441300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.441303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:52.441433Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:52.442991Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1650 sessionId: src_id|ea1ea6c3-bd5a1b9e-ecb741ac-47a4dd5a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:52.445265Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:52.500208Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342553325995890:7774]
2025-10-09T21:49:52.500428Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 1649, pipe: [1:7559342553325995890:7774]
2025-10-09T21:49:52.587287Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:52.587905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:52.587922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:52.587963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342553325995890:7774], now have 1 active actors on pipe
2025-10-09T21:49:52.588115Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:52.588179Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:52.588206Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:52.588210Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:52.588244Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342553325995890:7774]
2025-10-09T21:49:52.588270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1650 reason: error at write: Disconnected sessionId: src_id|ea1ea6c3-bd5a1b9e-ecb741ac-47a4dd5a_0
2025-10-09T21:49:52.588438Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1650 sessionId: src_id|ea1ea6c3-bd5a1b9e-ecb741ac-47a4dd5a_0 is DEAD
2025-10-09T21:49:52.588586Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea1ea6c3-bd5a1b9e-ecb741ac-47a4dd5a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:52.588597Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea1ea6c3-bd5a1b9e-ecb741ac-47a4dd5a_0] Write session will restart in 0.008474s
2025-10-09T21:49:52.588624Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea1ea6c3-bd5a1b9e-ecb741ac-47a4dd5a_0] Write session: Do CDS request
2025-10-09T21:49:52.588630Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea1ea6c3-bd5a1b9e-ecb741ac-47a4dd5a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:52.598587Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea1ea6c3-bd5a1b9e-ecb741ac-47a4dd5a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:52.598666Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:52.598679Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1651
2025-10-09T21:49:52.598785Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1651 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:52.598823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1651 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:52.598827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1651 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:52.598830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:52.599286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1651 sessionId:  describe result for acl check
2025-10-09T21:49:52.599324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:52.599326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:52.599328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:52.599334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342553325995996:7339] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:52.599780Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342553325995996:7339] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:52.602291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342553325995996:7339] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:52.604398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:52.604697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342553325995996:7339] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:52.604711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342553325995996:7339] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:52.604715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342553325995996:7339] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:52.604723Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1651 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:52.607652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:52.607817Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:52.607828Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:52.607840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:52.607894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:52.607925Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342552964167091:4175]
2025-10-09T21:49:52.608294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:52.608544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:52.608573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:52.608772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:52.608804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.608810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.608811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.608811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.608812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.608813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.608813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.608814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.608815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.608818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.608825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.608827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.608829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.608832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.608834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.608836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.608838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.608840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.608845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.608850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.608854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.608858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.608860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.608865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.608867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.608870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.608873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.608875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.608878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.608881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.608884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.608887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.608890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.608893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.608896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.608898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.608916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:52.614999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:52.616397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:52.616409Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:52.616410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:52.616413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:52.616418Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1650 [2:7559342552964167091:4175]
2025-10-09T21:49:52.616427Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:52.616441Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:52.616443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:52.616445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:52.616446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:52.616458Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:52.616464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:52.616469Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:52.616510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.616516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.616517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.616519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:52.616571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:52.616577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.616584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.616590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.616600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:52.616616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342553325995892:2432], now have 1 active actors on pipe
2025-10-09T21:49:52.616622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342553325996126:7339], now have 1 active actors on pipe
2025-10-09T21:49:52.616628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342552964167085:4175]
2025-10-09T21:49:52.616631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342552964167085:4175]
2025-10-09T21:49:52.616633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342552964167085:4175]
2025-10-09T21:49:52.616642Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:52.616657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:52.616669Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:52.616668Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:52.616675Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:52.616675Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:52.616678Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:52.616665Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1650
2025-10-09T21:49:52.616693Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1650
2025-10-09T21:49:52.616692Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342552964167085:4175]
2025-10-09T21:49:52.616700Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:52.616713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:52.616724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:52.616728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:52.616766Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|98ff7ab3-26336723-363a39f8-8308bf70_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:52.616780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:52.616784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:52.616786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:52.616796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:52.616810Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:52.616828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:52.616938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:52.616944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:52.616957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:52.617030Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1651 partition: 0 MaxSeqNo: 25 sessionId: src_id|98ff7ab3-26336723-363a39f8-8308bf70_0
2025-10-09T21:49:52.617337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea1ea6c3-bd5a1b9e-ecb741ac-47a4dd5a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|98ff7ab3-26336723-363a39f8-8308bf70_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:52.619772Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98ff7ab3-26336723-363a39f8-8308bf70_0] Write session: try to update token
2025-10-09T21:49:52.619789Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98ff7ab3-26336723-363a39f8-8308bf70_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:52.630959Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:52.630982Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:52.630984Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:52.630986Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:52.630997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342552964167085:4175]
2025-10-09T21:49:52.631001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342552964167085:4175]
2025-10-09T21:49:52.631005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342552964167085:4175]
2025-10-09T21:49:52.631046Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:52.631051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:52.631058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.631061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:52.631063Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:52.631067Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:52.631198Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:52.633030Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1651 sessionId: src_id|98ff7ab3-26336723-363a39f8-8308bf70_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:52.635311Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:52.688627Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342553325996139:7774]
2025-10-09T21:49:52.688881Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 1650, pipe: [1:7559342553325996139:7774]
2025-10-09T21:49:52.780012Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:52.780485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:52.780502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:52.780547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342553325996139:7774], now have 1 active actors on pipe
2025-10-09T21:49:52.780736Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:52.780731Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342553325996139:7774]
2025-10-09T21:49:52.780757Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:52.780759Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:52.780760Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:52.780820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1651 reason: error at write: Disconnected sessionId: src_id|98ff7ab3-26336723-363a39f8-8308bf70_0
2025-10-09T21:49:52.781000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1651 sessionId: src_id|98ff7ab3-26336723-363a39f8-8308bf70_0 is DEAD
2025-10-09T21:49:52.781140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98ff7ab3-26336723-363a39f8-8308bf70_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:52.781148Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98ff7ab3-26336723-363a39f8-8308bf70_0] Write session will restart in 0.008046s
2025-10-09T21:49:52.781178Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98ff7ab3-26336723-363a39f8-8308bf70_0] Write session: Do CDS request
2025-10-09T21:49:52.781184Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98ff7ab3-26336723-363a39f8-8308bf70_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:52.790561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98ff7ab3-26336723-363a39f8-8308bf70_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:52.790629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:52.790644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1652
2025-10-09T21:49:52.790807Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1652 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:52.790858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1652 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:52.790867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1652 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:52.790871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:52.791447Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1652 sessionId:  describe result for acl check
2025-10-09T21:49:52.791489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:52.791491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:52.791493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:52.791499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342553325996243:7334] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:52.792069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342553325996243:7334] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:52.794752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342553325996243:7334] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:52.797360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342553325996243:7334] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:52.797373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342553325996243:7334] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:52.797377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342553325996243:7334] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:52.797385Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1652 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:52.797662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:52.800992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:52.801222Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:52.801237Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:52.801250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:52.801312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:52.801362Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342552964167323:4172]
2025-10-09T21:49:52.801983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:52.802324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:52.802367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:52.802629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:52.802685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.802692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.802693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.802694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.802696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.802697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.802699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.802700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.802701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.802707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.802715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.802717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.802719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.802721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.802724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.802726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.802728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.802730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.802736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.802741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.802745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.802749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.802756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.802760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.802762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.802770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.802772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.802776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.802780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.802785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.802795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.802806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.802815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.802820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.802831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.802840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.802862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:52.809439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:52.810657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:52.810674Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:52.810676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:52.810680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:52.810688Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1651 [2:7559342552964167323:4172]
2025-10-09T21:49:52.810701Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:52.810725Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:52.810732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:52.810735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:52.810738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:52.810756Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:52.810770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:52.810777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:52.810834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.810843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.810845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.810848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:52.810932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:52.810942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.810954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.810969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.810992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:52.811019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342553325996376:7334], now have 1 active actors on pipe
2025-10-09T21:49:52.811027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342553325996141:2432], now have 1 active actors on pipe
2025-10-09T21:49:52.811035Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342552964167318:4172]
2025-10-09T21:49:52.811042Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342552964167318:4172]
2025-10-09T21:49:52.811044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342552964167318:4172]
2025-10-09T21:49:52.811056Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:52.811067Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:52.811070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:52.811076Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:52.811080Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:52.811084Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:52.811088Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:52.811079Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1651
2025-10-09T21:49:52.811085Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1651
2025-10-09T21:49:52.811120Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342552964167318:4172]
2025-10-09T21:49:52.811129Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:52.811132Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:52.811144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:52.811153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:52.811205Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8ccebdea-a2333c2-dc4de0c8-e6da03db_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:52.811218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:52.811223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:52.811228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:52.811239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:52.811253Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:52.811272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:52.811417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:52.811426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:52.811442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:52.811509Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1652 partition: 0 MaxSeqNo: 25 sessionId: src_id|8ccebdea-a2333c2-dc4de0c8-e6da03db_0
2025-10-09T21:49:52.811827Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98ff7ab3-26336723-363a39f8-8308bf70_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8ccebdea-a2333c2-dc4de0c8-e6da03db_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:52.814349Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8ccebdea-a2333c2-dc4de0c8-e6da03db_0] Write session: try to update token
2025-10-09T21:49:52.814372Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8ccebdea-a2333c2-dc4de0c8-e6da03db_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:52.827069Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:52.827102Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:52.827107Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:52.827110Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:52.827123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342552964167318:4172]
2025-10-09T21:49:52.827131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342552964167318:4172]
2025-10-09T21:49:52.827138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342552964167318:4172]
2025-10-09T21:49:52.827197Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:52.827201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:52.827210Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:52.827211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.827214Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:52.827214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:52.827450Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:52.827798Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1652 sessionId: src_id|8ccebdea-a2333c2-dc4de0c8-e6da03db_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:52.830138Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:52.881160Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342553325996391:7774]
2025-10-09T21:49:52.881454Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 1651, pipe: [1:7559342553325996391:7774]
2025-10-09T21:49:52.977323Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:52.977569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:52.977582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:52.977621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342553325996391:7774], now have 1 active actors on pipe
2025-10-09T21:49:52.977816Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342553325996391:7774]
2025-10-09T21:49:52.977835Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:52.977848Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:52.977850Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:52.977902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1652 reason: error at write: Disconnected sessionId: src_id|8ccebdea-a2333c2-dc4de0c8-e6da03db_0
2025-10-09T21:49:52.978069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1652 sessionId: src_id|8ccebdea-a2333c2-dc4de0c8-e6da03db_0 is DEAD
2025-10-09T21:49:52.978134Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:52.978248Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ccebdea-a2333c2-dc4de0c8-e6da03db_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:52.978258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ccebdea-a2333c2-dc4de0c8-e6da03db_0] Write session will restart in 0.007890s
2025-10-09T21:49:52.978285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ccebdea-a2333c2-dc4de0c8-e6da03db_0] Write session: Do CDS request
2025-10-09T21:49:52.978291Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ccebdea-a2333c2-dc4de0c8-e6da03db_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:52.987532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8ccebdea-a2333c2-dc4de0c8-e6da03db_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:52.987613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:52.987626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1653
2025-10-09T21:49:52.987796Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1653 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:52.987827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1653 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:52.987845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1653 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:52.987849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:52.988345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1653 sessionId:  describe result for acl check
2025-10-09T21:49:52.988380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:52.988388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:52.988390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:52.988396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342553325996513:7309] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:52.988919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342553325996513:7309] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:52.991791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342553325996513:7309] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:52.997059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:52.997273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342553325996513:7309] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:52.997289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342553325996513:7309] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:52.997293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342553325996513:7309] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:52.997302Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1653 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:52.997550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:52.997754Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:52.997764Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:52.997776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:52.997839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:52.997883Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342552964167591:4145]
2025-10-09T21:49:52.998482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:52.998809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:52.998857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:52.999191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:52.999255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.999262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.999263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.999265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.999266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.999268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.999269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.999270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.999272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.999277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.999289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.999292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.999296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.999300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.999303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.999306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.999310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.999319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.999327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.999341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:52.999352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.999362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:52.999372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.999377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:52.999381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.999392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:52.999397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.999404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:52.999410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.999415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:52.999419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.999425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:52.999429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.999434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:52.999439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.999451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:52.999485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:53.005804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:53.007134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:53.007152Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:53.007155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:53.007160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:53.007168Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1652 [2:7559342552964167591:4145]
2025-10-09T21:49:53.007182Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:53.007206Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:53.007217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:53.007220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:53.007223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:53.007241Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:53.007248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:53.007255Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:53.007321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.007329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.007331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.007334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:53.007413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:53.007421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.007434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.007447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.007469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:53.007495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342553325996649:7309], now have 1 active actors on pipe
2025-10-09T21:49:53.007504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342553325996417:2432], now have 1 active actors on pipe
2025-10-09T21:49:53.007512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342552964167585:4145]
2025-10-09T21:49:53.007515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342552964167585:4145]
2025-10-09T21:49:53.007518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342552964167585:4145]
2025-10-09T21:49:53.007531Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:53.007544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:53.007557Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:53.007564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:53.007568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:53.007569Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:53.007572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:53.007600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342552964167585:4145]
2025-10-09T21:49:53.007607Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:53.007567Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1652
2025-10-09T21:49:53.007609Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:53.007576Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1652
2025-10-09T21:49:53.007635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:53.007644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:53.007686Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7e5b8230-1d9abb8a-97307aeb-58f00f56_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:53.007699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:53.007703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:53.007706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:53.007718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:53.007730Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:53.007753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:53.007870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:53.007880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:53.007894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:53.007989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1653 partition: 0 MaxSeqNo: 25 sessionId: src_id|7e5b8230-1d9abb8a-97307aeb-58f00f56_0
2025-10-09T21:49:53.008349Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8ccebdea-a2333c2-dc4de0c8-e6da03db_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7e5b8230-1d9abb8a-97307aeb-58f00f56_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:53.010726Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e5b8230-1d9abb8a-97307aeb-58f00f56_0] Write session: try to update token
2025-10-09T21:49:53.010745Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e5b8230-1d9abb8a-97307aeb-58f00f56_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:53.022009Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:53.022047Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:53.022050Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:53.022053Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:53.022062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342552964167585:4145]
2025-10-09T21:49:53.022067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342552964167585:4145]
2025-10-09T21:49:53.022071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342552964167585:4145]
2025-10-09T21:49:53.022116Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:53.022118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:53.022122Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:53.022125Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:53.022127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.022129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:53.022253Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:53.023770Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1653 sessionId: src_id|7e5b8230-1d9abb8a-97307aeb-58f00f56_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:53.026183Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:53.077723Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342557620963958:7774]
2025-10-09T21:49:53.077968Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 1652, pipe: [1:7559342557620963958:7774]
2025-10-09T21:49:53.169930Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:53.170562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:53.170578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:53.170618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342557620963958:7774], now have 1 active actors on pipe
2025-10-09T21:49:53.170782Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:53.170843Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:53.170860Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342557620963958:7774]
2025-10-09T21:49:53.170886Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:53.170891Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:53.170956Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1653 reason: error at write: Disconnected sessionId: src_id|7e5b8230-1d9abb8a-97307aeb-58f00f56_0
2025-10-09T21:49:53.171151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1653 sessionId: src_id|7e5b8230-1d9abb8a-97307aeb-58f00f56_0 is DEAD
2025-10-09T21:49:53.171339Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e5b8230-1d9abb8a-97307aeb-58f00f56_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:53.171351Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e5b8230-1d9abb8a-97307aeb-58f00f56_0] Write session will restart in 0.006743s
2025-10-09T21:49:53.171378Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e5b8230-1d9abb8a-97307aeb-58f00f56_0] Write session: Do CDS request
2025-10-09T21:49:53.171386Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e5b8230-1d9abb8a-97307aeb-58f00f56_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:53.179619Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e5b8230-1d9abb8a-97307aeb-58f00f56_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:53.179644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:53.179667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1654
2025-10-09T21:49:53.179862Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1654 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:53.179912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1654 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:53.179924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1654 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:53.179938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:53.180375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1654 sessionId:  describe result for acl check
2025-10-09T21:49:53.180415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:53.180422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:53.180424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:53.180430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342557620964025:7305] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:53.181017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342557620964025:7305] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:53.183771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342557620964025:7305] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:53.186570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342557620964025:7305] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:53.186584Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342557620964025:7305] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:53.186588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342557620964025:7305] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:53.186598Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1654 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:53.191774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:53.192361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:53.192565Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:53.192576Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:53.192589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:53.192643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:53.192688Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342557259135117:4143]
2025-10-09T21:49:53.194081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:53.194407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:53.194459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:53.195095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:53.195168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:53.195174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:53.195175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:53.195177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:53.195178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:53.195179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:53.195181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.195182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.195183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.195189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:53.195200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:53.195204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:53.195209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:53.195213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:53.195217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:53.195221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.195225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.195229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.195236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:53.195244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:53.195255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:53.195265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:53.195275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:53.195285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:53.195289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:53.195295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:53.195301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:53.195311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:53.195321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:53.195331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:53.195341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.195346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.195354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.195360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.195365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.195374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.195409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:53.201968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:53.203251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:53.203270Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:53.203272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:53.203277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:53.203284Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1653 [2:7559342557259135117:4143]
2025-10-09T21:49:53.203302Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:53.203324Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:53.203332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:53.203336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:53.203339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:53.203358Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:53.203368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:53.203377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:53.203449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.203459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.203462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.203465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:53.203544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:53.203551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.203564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.203576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.203599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:53.203625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342557620963961:2432], now have 1 active actors on pipe
2025-10-09T21:49:53.203633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342557620964208:7305], now have 1 active actors on pipe
2025-10-09T21:49:53.203641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342557259135112:4143]
2025-10-09T21:49:53.203644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342557259135112:4143]
2025-10-09T21:49:53.203646Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342557259135112:4143]
2025-10-09T21:49:53.203659Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:53.203678Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:53.203679Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:53.203682Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:53.203685Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:53.203699Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:53.203704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:53.203736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342557259135112:4143]
2025-10-09T21:49:53.203749Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:53.203731Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1653
2025-10-09T21:49:53.203739Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1653
2025-10-09T21:49:53.203769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:53.203787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:53.203798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:53.203830Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2dc8f52f-7f0f5ce4-30a33ee8-82a180d4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:53.203843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:53.203852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:53.203860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:53.203871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:53.203886Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:53.203906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:53.204067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:53.204074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:53.204088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:53.204215Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1654 partition: 0 MaxSeqNo: 25 sessionId: src_id|2dc8f52f-7f0f5ce4-30a33ee8-82a180d4_0
2025-10-09T21:49:53.204562Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e5b8230-1d9abb8a-97307aeb-58f00f56_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2dc8f52f-7f0f5ce4-30a33ee8-82a180d4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:53.205175Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] Counters: { Errors: 16 CurrentSessionLifetimeMs: 421656 BytesRead: 24780812 MessagesRead: 11 BytesReadCompressed: 24780812 BytesInflightUncompressed: 2252802 BytesInflightCompressed: 0 BytesInflightTotal: 2252802 MessagesInflight: 1 }
2025-10-09T21:49:53.207057Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2dc8f52f-7f0f5ce4-30a33ee8-82a180d4_0] Write session: try to update token
2025-10-09T21:49:53.207072Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2dc8f52f-7f0f5ce4-30a33ee8-82a180d4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:53.215634Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:53.215663Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:53.215676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342557259135112:4143]
2025-10-09T21:49:53.215731Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:53.215745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:53.216143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:53.216431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:53.216721Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:53.216997Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:53.217390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:53.217467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:53.220762Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1654 sessionId: src_id|2dc8f52f-7f0f5ce4-30a33ee8-82a180d4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:53.223315Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:53.223566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:53.223591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:53.223635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:53.223755Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:53.229437Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:53.229459Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:53.229461Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:53.229464Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:53.229475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342557259135112:4143]
2025-10-09T21:49:53.229479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342557259135112:4143]
2025-10-09T21:49:53.229484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342557259135112:4143]
2025-10-09T21:49:53.229549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:53.229552Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:53.229557Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:53.229557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.229560Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:53.229561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:53.229755Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:53.270799Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342557620964227:7774]
2025-10-09T21:49:53.271055Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 1653, pipe: [1:7559342557620964227:7774]
2025-10-09T21:49:53.389864Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:53.398129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:53.398156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:53.398328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:53.398394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:53.398461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:53.398530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:53.398579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:53.398590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:53.398671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:53.398737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:53.398807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:53.398880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:53.398928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:53.398937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:53.399000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:53.399067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:53.399131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:53.399195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:53.399242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:53.399252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:53.399322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:53.399395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:53.399460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:53.399530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:53.399578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:53.399589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:53.399656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:53.399720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:53.399781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:53.399845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:53.399890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:53.399896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:53.400031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:53.400160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:53.400288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:53.400353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:53.400399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:53.400406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:53.400467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:53.400526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:53.400594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:53.400660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:53.400704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:53.400712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:53.400778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:53.400837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:53.400898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:53.400962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:53.401001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:53.401009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:53.401071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:53.401134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:53.401197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:53.401261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:53.401302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:53.401309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:53.401373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:53.401431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:53.401500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:53.401574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:53.401620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:53.401629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:53.401695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342557620964227:7774], now have 1 active actors on pipe
2025-10-09T21:49:53.401953Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:53.401966Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:53.401968Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:53.401997Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1654 reason: error at write: Disconnected sessionId: src_id|2dc8f52f-7f0f5ce4-30a33ee8-82a180d4_0
2025-10-09T21:49:53.402019Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342557620964227:7774]
2025-10-09T21:49:53.402176Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1654 sessionId: src_id|2dc8f52f-7f0f5ce4-30a33ee8-82a180d4_0 is DEAD
2025-10-09T21:49:53.402336Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2dc8f52f-7f0f5ce4-30a33ee8-82a180d4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:53.402346Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2dc8f52f-7f0f5ce4-30a33ee8-82a180d4_0] Write session will restart in 0.007122s
2025-10-09T21:49:53.402367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2dc8f52f-7f0f5ce4-30a33ee8-82a180d4_0] Write session: Do CDS request
2025-10-09T21:49:53.402372Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2dc8f52f-7f0f5ce4-30a33ee8-82a180d4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:53.402795Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:53.410483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2dc8f52f-7f0f5ce4-30a33ee8-82a180d4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:53.410584Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:53.410607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1655
2025-10-09T21:49:53.410727Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1655 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:53.410776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1655 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:53.410789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1655 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:53.410792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:53.411233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1655 sessionId:  describe result for acl check
2025-10-09T21:49:53.411313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:53.411322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:53.411324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:53.411330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342557620964293:7295] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:53.411879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342557620964293:7295] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:53.415200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342557620964293:7295] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:53.417974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342557620964293:7295] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:53.417987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342557620964293:7295] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:53.417991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342557620964293:7295] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:53.417999Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1655 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:53.418731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:53.421796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:53.421985Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:53.421996Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:53.422007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:53.422079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:53.422123Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342557259135348:4125]
2025-10-09T21:49:53.422717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:53.423054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:53.423124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:53.423458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:53.423527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:53.423537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:53.423538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:53.423539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:53.423540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:53.423541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:53.423543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.423543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.423544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.423550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:53.423562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:53.423566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:53.423570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:53.423574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:53.423579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:53.423582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.423585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.423588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.423598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:53.423606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:53.423612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:53.423618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:53.423629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:53.423634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:53.423639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:53.423649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:53.423658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:53.423668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:53.423677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:53.423682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:53.423687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.423694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.423699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.423707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.423713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.423723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.423753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:53.430212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:53.431627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:53.431646Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:53.431648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:53.431652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:53.431659Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1654 [2:7559342557259135348:4125]
2025-10-09T21:49:53.431672Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:53.431688Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:53.431691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:53.431694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:53.431696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:53.431713Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:53.431725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:53.431737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:53.431802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.431809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.431811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.431814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:53.431881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:53.431888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.431901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.431916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.431940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:53.431965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342557620964230:2432], now have 1 active actors on pipe
2025-10-09T21:49:53.431973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342557620964467:7295], now have 1 active actors on pipe
2025-10-09T21:49:53.431980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342557259135343:4125]
2025-10-09T21:49:53.431987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342557259135343:4125]
2025-10-09T21:49:53.431990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342557259135343:4125]
2025-10-09T21:49:53.432004Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:53.432023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:53.432022Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:53.432026Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:53.432027Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:53.432041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:53.432049Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:53.432028Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1654
2025-10-09T21:49:53.432032Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1654
2025-10-09T21:49:53.432075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342557259135343:4125]
2025-10-09T21:49:53.432085Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:53.432097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:53.432110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:53.432120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:53.432167Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f48e7d70-33e76142-e853f00d-757844b8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:53.432179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:53.432183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:53.432186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:53.432197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:53.432212Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:53.432234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:53.432376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:53.432387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:53.432401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:53.432509Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1655 partition: 0 MaxSeqNo: 25 sessionId: src_id|f48e7d70-33e76142-e853f00d-757844b8_0
2025-10-09T21:49:53.432889Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2dc8f52f-7f0f5ce4-30a33ee8-82a180d4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f48e7d70-33e76142-e853f00d-757844b8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:53.435269Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f48e7d70-33e76142-e853f00d-757844b8_0] Write session: try to update token
2025-10-09T21:49:53.435283Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f48e7d70-33e76142-e853f00d-757844b8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:53.446172Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:53.446192Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:53.446194Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:53.446196Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:53.446206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342557259135343:4125]
2025-10-09T21:49:53.446211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342557259135343:4125]
2025-10-09T21:49:53.446216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342557259135343:4125]
2025-10-09T21:49:53.446259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:53.446259Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:53.446264Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:53.446267Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:53.446267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.446269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:53.446409Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:53.447531Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1655 sessionId: src_id|f48e7d70-33e76142-e853f00d-757844b8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:53.449902Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:53.502167Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342557620964481:7774]
2025-10-09T21:49:53.502388Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 1654, pipe: [1:7559342557620964481:7774]
2025-10-09T21:49:53.594323Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:53.594955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:53.594973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:53.595017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342557620964481:7774], now have 1 active actors on pipe
2025-10-09T21:49:53.595253Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:53.595296Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:53.595308Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:53.595309Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342557620964481:7774]
2025-10-09T21:49:53.595328Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:53.595370Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1655 reason: error at write: Disconnected sessionId: src_id|f48e7d70-33e76142-e853f00d-757844b8_0
2025-10-09T21:49:53.595533Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1655 sessionId: src_id|f48e7d70-33e76142-e853f00d-757844b8_0 is DEAD
2025-10-09T21:49:53.595661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f48e7d70-33e76142-e853f00d-757844b8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:53.595669Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f48e7d70-33e76142-e853f00d-757844b8_0] Write session will restart in 0.009441s
2025-10-09T21:49:53.595696Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f48e7d70-33e76142-e853f00d-757844b8_0] Write session: Do CDS request
2025-10-09T21:49:53.595702Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f48e7d70-33e76142-e853f00d-757844b8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:53.606516Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f48e7d70-33e76142-e853f00d-757844b8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:53.606585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:53.606598Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1656
2025-10-09T21:49:53.606724Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1656 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:53.606766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1656 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:53.606778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1656 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:53.606782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:53.607164Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1656 sessionId:  describe result for acl check
2025-10-09T21:49:53.607193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:53.607198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:53.607199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:53.607205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342557620964622:7291] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:53.607701Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342557620964622:7291] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:53.610155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342557620964622:7291] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:53.611044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:53.614271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:53.614461Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:53.614473Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:53.614485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:53.614548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:53.614565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342557620964622:7291] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:53.614575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342557620964622:7291] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:53.614578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342557620964622:7291] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:53.614587Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342557259135580:4133]
2025-10-09T21:49:53.614584Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1656 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:53.615193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:53.615498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:53.615547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:53.615899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:53.615952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:53.615959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:53.615960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:53.615961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:53.615962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:53.615963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:53.615964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.615966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.615967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.615972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:53.615982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:53.615986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:53.615991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:53.616000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:53.616004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:53.616012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.616016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.616020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.616031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:53.616044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:53.616056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:53.616066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:53.616071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:53.616081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:53.616089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:53.616094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:53.616104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:53.616109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:53.616114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:53.616121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:53.616126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.616138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.616142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.616147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.616152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.616157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.616186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:53.622637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:53.623823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:53.623838Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:53.623840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:53.623846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:53.623851Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1655 [2:7559342557259135580:4133]
2025-10-09T21:49:53.623865Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:53.623887Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:53.623895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:53.623898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:53.623900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:53.623920Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:53.623931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:53.623939Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:53.624004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.624011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.624013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.624016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:53.624089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:53.624098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.624111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.624128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.624149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:53.624177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342557620964483:2432], now have 1 active actors on pipe
2025-10-09T21:49:53.624185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342557620964723:7291], now have 1 active actors on pipe
2025-10-09T21:49:53.624193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342557259135575:4133]
2025-10-09T21:49:53.624198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342557259135575:4133]
2025-10-09T21:49:53.624200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342557259135575:4133]
2025-10-09T21:49:53.624213Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:53.624234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:53.624242Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:53.624246Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:53.624248Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:53.624250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:53.624253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:53.624242Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1655
2025-10-09T21:49:53.624280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342557259135575:4133]
2025-10-09T21:49:53.624255Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1655
2025-10-09T21:49:53.624285Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:53.624287Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:53.624308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:53.624318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:53.624373Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c883cc29-b4a895b5-849276e2-a3c32e0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:53.624387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:53.624392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:53.624395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:53.624407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:53.624424Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:53.624449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:53.624645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:53.624658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:53.624675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:53.624756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1656 partition: 0 MaxSeqNo: 25 sessionId: src_id|c883cc29-b4a895b5-849276e2-a3c32e0_0
2025-10-09T21:49:53.625114Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f48e7d70-33e76142-e853f00d-757844b8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c883cc29-b4a895b5-849276e2-a3c32e0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:53.627468Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c883cc29-b4a895b5-849276e2-a3c32e0_0] Write session: try to update token
2025-10-09T21:49:53.627483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c883cc29-b4a895b5-849276e2-a3c32e0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:53.638691Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:53.638713Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:53.638716Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:53.638718Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:53.638728Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342557259135575:4133]
2025-10-09T21:49:53.638734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342557259135575:4133]
2025-10-09T21:49:53.638737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342557259135575:4133]
2025-10-09T21:49:53.638780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:53.638785Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:53.638789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.638791Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:53.638792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:53.638795Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:53.638927Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:53.639809Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1656 sessionId: src_id|c883cc29-b4a895b5-849276e2-a3c32e0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:53.642171Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:53.695238Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342557620964735:7774]
2025-10-09T21:49:53.695536Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 1655, pipe: [1:7559342557620964735:7774]
2025-10-09T21:49:53.786362Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:53.787098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:53.787112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:53.787152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342557620964735:7774], now have 1 active actors on pipe
2025-10-09T21:49:53.787331Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:53.787359Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:53.787354Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:53.787357Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:53.787387Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342557620964735:7774]
2025-10-09T21:49:53.787411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1656 reason: error at write: Disconnected sessionId: src_id|c883cc29-b4a895b5-849276e2-a3c32e0_0
2025-10-09T21:49:53.787562Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1656 sessionId: src_id|c883cc29-b4a895b5-849276e2-a3c32e0_0 is DEAD
2025-10-09T21:49:53.787716Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c883cc29-b4a895b5-849276e2-a3c32e0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:53.787727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c883cc29-b4a895b5-849276e2-a3c32e0_0] Write session will restart in 0.006890s
2025-10-09T21:49:53.787746Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c883cc29-b4a895b5-849276e2-a3c32e0_0] Write session: Do CDS request
2025-10-09T21:49:53.787751Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c883cc29-b4a895b5-849276e2-a3c32e0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:53.795514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c883cc29-b4a895b5-849276e2-a3c32e0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:53.795595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:53.795607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1657
2025-10-09T21:49:53.795736Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1657 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:53.795792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1657 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:53.795803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1657 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:53.795807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:53.796245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1657 sessionId:  describe result for acl check
2025-10-09T21:49:53.796277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:53.796283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:53.796285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:53.796290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342557620964781:7287] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:53.796937Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342557620964781:7287] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:53.799554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342557620964781:7287] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:53.802587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342557620964781:7287] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:53.802601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342557620964781:7287] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:53.802605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342557620964781:7287] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:53.802612Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1657 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:53.808050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:53.808660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:53.808851Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:53.808864Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:53.808877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:53.808940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:53.808984Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342557259135818:4114]
2025-10-09T21:49:53.809567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:53.809891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:53.809935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:53.810276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:53.810323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:53.810331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:53.810332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:53.810333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:53.810333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:53.810334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:53.810335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.810335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.810336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.810340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:53.810347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:53.810350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:53.810353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:53.810357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:53.810361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:53.810365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.810368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.810372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.810379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:53.810388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:53.810394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:53.810399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:53.810403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:53.810408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:53.810418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:53.810428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:53.810437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:53.810442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:53.810451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:53.810457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:53.810467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.810472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.810476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.810480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.810485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.810489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.810525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:53.816707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:53.817824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:53.817842Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:53.817845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:53.817849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:53.817855Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1656 [2:7559342557259135818:4114]
2025-10-09T21:49:53.817868Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:53.817885Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:53.817893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:53.817896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:53.817899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:53.817914Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:53.817920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:53.817927Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:53.817980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.817987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.817989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.817992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:53.818071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:53.818079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.818092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:53.818108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:53.818129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:53.818154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342557620964975:7287], now have 1 active actors on pipe
2025-10-09T21:49:53.818162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342557620964737:2432], now have 1 active actors on pipe
2025-10-09T21:49:53.818169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342557259135813:4114]
2025-10-09T21:49:53.818176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342557259135813:4114]
2025-10-09T21:49:53.818178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342557259135813:4114]
2025-10-09T21:49:53.818193Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:53.818208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:53.818222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:53.818225Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:53.818229Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:53.818230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:53.818231Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:53.818213Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1656
2025-10-09T21:49:53.818254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342557259135813:4114]
2025-10-09T21:49:53.818221Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1656
2025-10-09T21:49:53.818262Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:53.818264Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:53.818285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:53.818295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:53.818347Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c832e161-80ee56d9-26a75e2f-aeb9a9d6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:53.818359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:53.818368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:53.818376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:53.818386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:53.818401Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:53.818420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:53.818591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:53.818599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:53.818612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:53.818698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1657 partition: 0 MaxSeqNo: 25 sessionId: src_id|c832e161-80ee56d9-26a75e2f-aeb9a9d6_0
2025-10-09T21:49:53.819066Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c883cc29-b4a895b5-849276e2-a3c32e0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c832e161-80ee56d9-26a75e2f-aeb9a9d6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:53.821471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c832e161-80ee56d9-26a75e2f-aeb9a9d6_0] Write session: try to update token
2025-10-09T21:49:53.821489Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c832e161-80ee56d9-26a75e2f-aeb9a9d6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:53.832431Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:53.832453Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:53.832456Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:53.832458Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:53.832468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342557259135813:4114]
2025-10-09T21:49:53.832474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342557259135813:4114]
2025-10-09T21:49:53.832478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342557259135813:4114]
2025-10-09T21:49:53.832520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:53.832524Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:53.832528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:53.832529Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:53.832531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:53.832533Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:53.832664Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:53.834374Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1657 sessionId: src_id|c832e161-80ee56d9-26a75e2f-aeb9a9d6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:53.836604Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:53.887754Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342557620964992:7774]
2025-10-09T21:49:53.888066Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 1656, pipe: [1:7559342557620964992:7774]
2025-10-09T21:49:53.983789Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:53.984277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:53.984293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:53.984335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342557620964992:7774], now have 1 active actors on pipe
2025-10-09T21:49:53.984533Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:53.984534Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:53.984546Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:53.984607Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1657 reason: error at write: Disconnected sessionId: src_id|c832e161-80ee56d9-26a75e2f-aeb9a9d6_0
2025-10-09T21:49:53.984740Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342557620964992:7774]
2025-10-09T21:49:53.984767Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:53.984786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1657 sessionId: src_id|c832e161-80ee56d9-26a75e2f-aeb9a9d6_0 is DEAD
2025-10-09T21:49:53.984934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c832e161-80ee56d9-26a75e2f-aeb9a9d6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:53.984954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c832e161-80ee56d9-26a75e2f-aeb9a9d6_0] Write session will restart in 0.009352s
2025-10-09T21:49:53.984980Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c832e161-80ee56d9-26a75e2f-aeb9a9d6_0] Write session: Do CDS request
2025-10-09T21:49:53.984986Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c832e161-80ee56d9-26a75e2f-aeb9a9d6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:53.995475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c832e161-80ee56d9-26a75e2f-aeb9a9d6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:53.995551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:53.995562Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1658
2025-10-09T21:49:53.995700Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1658 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:53.995745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1658 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:53.995755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1658 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:53.995759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:53.996319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1658 sessionId:  describe result for acl check
2025-10-09T21:49:53.996353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:53.996356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:53.996357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:53.996364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342557620965136:7273] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:53.996891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342557620965136:7273] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:53.999615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342557620965136:7273] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:54.004274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:54.004612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342557620965136:7273] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:54.004626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342557620965136:7273] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:54.004631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342557620965136:7273] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:54.004640Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1658 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:54.004739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:54.004914Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:54.004924Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:54.004935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:54.004982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:54.005023Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342561554103386:4088]
2025-10-09T21:49:54.005632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:54.005932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:54.005974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:54.006301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:54.006349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.006356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.006358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.006359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.006360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.006361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.006363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.006364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.006366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.006370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.006383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.006387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.006391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.006395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.006399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.006403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.006405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.006409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.006414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.006422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.006429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.006433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.006438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.006443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.006447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.006452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.006457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.006462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.006466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.006472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.006477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.006482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.006487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.006492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.006503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.006509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.006539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:54.013103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:54.014315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:54.014334Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:54.014337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:54.014341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:54.014349Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1657 [2:7559342561554103386:4088]
2025-10-09T21:49:54.014364Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:54.014385Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:54.014388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:54.014391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:54.014393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:54.014411Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:54.014421Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:54.014431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:54.014495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.014503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.014505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.014508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:54.014582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:54.014589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.014602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.014619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.014642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:54.014668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342561915932554:7273], now have 1 active actors on pipe
2025-10-09T21:49:54.014672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342557620965018:2432], now have 1 active actors on pipe
2025-10-09T21:49:54.014680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342561554103380:4088]
2025-10-09T21:49:54.014682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342561554103380:4088]
2025-10-09T21:49:54.014685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342561554103380:4088]
2025-10-09T21:49:54.014698Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:54.014717Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:54.014720Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:54.014720Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:54.014722Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:54.014734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:54.014736Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:54.014740Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1657
2025-10-09T21:49:54.014749Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1657
2025-10-09T21:49:54.014772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342561554103380:4088]
2025-10-09T21:49:54.014782Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:54.014785Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:54.014813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:54.014822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:54.014857Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|16c67c4a-c0fa366c-a8fce269-90b61c54_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:54.014869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:54.014872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:54.014875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:54.014885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:54.014894Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:54.014926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:54.015093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:54.015104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:54.015121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:54.015236Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1658 partition: 0 MaxSeqNo: 25 sessionId: src_id|16c67c4a-c0fa366c-a8fce269-90b61c54_0
2025-10-09T21:49:54.015615Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c832e161-80ee56d9-26a75e2f-aeb9a9d6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|16c67c4a-c0fa366c-a8fce269-90b61c54_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:54.018015Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16c67c4a-c0fa366c-a8fce269-90b61c54_0] Write session: try to update token
2025-10-09T21:49:54.018047Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16c67c4a-c0fa366c-a8fce269-90b61c54_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:54.029237Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:54.029258Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:54.029261Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:54.029263Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:54.029273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342561554103380:4088]
2025-10-09T21:49:54.029279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342561554103380:4088]
2025-10-09T21:49:54.029282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342561554103380:4088]
2025-10-09T21:49:54.029329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:54.029340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.029343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:54.029342Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:54.029350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:54.029354Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:54.029480Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:54.030961Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1658 sessionId: src_id|16c67c4a-c0fa366c-a8fce269-90b61c54_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:54.033366Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:54.084856Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342561915932565:7774]
2025-10-09T21:49:54.085123Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 1657, pipe: [1:7559342561915932565:7774]
2025-10-09T21:49:54.179267Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:54.179527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:54.179558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:54.179617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342561915932565:7774], now have 1 active actors on pipe
2025-10-09T21:49:54.179867Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:54.179881Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342561915932565:7774]
2025-10-09T21:49:54.179897Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:54.179906Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:54.179909Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:54.179999Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1658 reason: error at write: Disconnected sessionId: src_id|16c67c4a-c0fa366c-a8fce269-90b61c54_0
2025-10-09T21:49:54.180146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1658 sessionId: src_id|16c67c4a-c0fa366c-a8fce269-90b61c54_0 is DEAD
2025-10-09T21:49:54.180292Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16c67c4a-c0fa366c-a8fce269-90b61c54_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:54.180302Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16c67c4a-c0fa366c-a8fce269-90b61c54_0] Write session will restart in 0.009215s
2025-10-09T21:49:54.180330Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16c67c4a-c0fa366c-a8fce269-90b61c54_0] Write session: Do CDS request
2025-10-09T21:49:54.180336Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16c67c4a-c0fa366c-a8fce269-90b61c54_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:54.190573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16c67c4a-c0fa366c-a8fce269-90b61c54_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:54.190675Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:54.190689Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1659
2025-10-09T21:49:54.190836Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1659 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:54.190870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1659 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:54.190878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1659 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:54.190881Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:54.191196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1659 sessionId:  describe result for acl check
2025-10-09T21:49:54.191222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:54.191227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:54.191229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:54.191233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342561915932655:7270] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:54.191690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342561915932655:7270] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:54.194021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342561915932655:7270] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:54.196584Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342561915932655:7270] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:54.196597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342561915932655:7270] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:54.196600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342561915932655:7270] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:54.196609Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1659 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:54.201289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:54.201750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:54.201942Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:54.201953Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:54.201965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:54.202017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:54.202076Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342561554103621:4086]
2025-10-09T21:49:54.202669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:54.202981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:54.203026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:54.203336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:54.203384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.203391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.203392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.203393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.203394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.203395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.203397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.203398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.203399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.203404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.203415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.203418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.203422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.203426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.203430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.203434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.203438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.203447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.203454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.203463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.203474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.203479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.203483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.203488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.203493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.203502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.203506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.203515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.203524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.203529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.203538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.203544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.203548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.203558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.203562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.203570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.203599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:54.209812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:54.211283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:54.211302Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:54.211304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:54.211308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:54.211315Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1658 [2:7559342561554103621:4086]
2025-10-09T21:49:54.211330Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:54.211350Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:54.211357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:54.211360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:54.211362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:54.211378Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:54.211388Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:54.211396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:54.211450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.211461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.211463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.211466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:54.211539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:54.211549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.211562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.211574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.211589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:54.211612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342561915932571:2432], now have 1 active actors on pipe
2025-10-09T21:49:54.211621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342561915932812:7270], now have 1 active actors on pipe
2025-10-09T21:49:54.211628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342561554103616:4086]
2025-10-09T21:49:54.211631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342561554103616:4086]
2025-10-09T21:49:54.211633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342561554103616:4086]
2025-10-09T21:49:54.211646Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:54.211666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:54.211666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:54.211670Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:54.211673Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:54.211684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:54.211688Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:54.211692Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1658
2025-10-09T21:49:54.211714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342561554103616:4086]
2025-10-09T21:49:54.211702Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1658
2025-10-09T21:49:54.211726Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:54.211738Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:54.211755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:54.211760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:54.211795Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9cc2d30d-789835fd-fb6572f6-eaae32ba_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:54.211809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:54.211818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:54.211821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:54.211835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:54.211849Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:54.211870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:54.212030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:54.212038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:54.212053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:54.212148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1659 partition: 0 MaxSeqNo: 25 sessionId: src_id|9cc2d30d-789835fd-fb6572f6-eaae32ba_0
2025-10-09T21:49:54.212462Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16c67c4a-c0fa366c-a8fce269-90b61c54_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9cc2d30d-789835fd-fb6572f6-eaae32ba_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:54.214892Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9cc2d30d-789835fd-fb6572f6-eaae32ba_0] Write session: try to update token
2025-10-09T21:49:54.214908Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9cc2d30d-789835fd-fb6572f6-eaae32ba_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:54.225431Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:54.225449Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:54.225451Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:54.225452Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:54.225462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342561554103616:4086]
2025-10-09T21:49:54.225465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342561554103616:4086]
2025-10-09T21:49:54.225468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342561554103616:4086]
2025-10-09T21:49:54.225501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:54.225507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.225509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:54.225510Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:54.225519Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:54.225524Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:54.225645Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:54.227000Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1659 sessionId: src_id|9cc2d30d-789835fd-fb6572f6-eaae32ba_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:54.229246Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:54.279947Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342561915932830:7774]
2025-10-09T21:49:54.280207Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 1658, pipe: [1:7559342561915932830:7774]
2025-10-09T21:49:54.376301Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:54.376763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:54.376781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:54.376820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342561915932830:7774], now have 1 active actors on pipe
2025-10-09T21:49:54.377061Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:54.377089Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342561915932830:7774]
2025-10-09T21:49:54.377113Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:54.377115Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:54.377172Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1659 reason: error at write: Disconnected sessionId: src_id|9cc2d30d-789835fd-fb6572f6-eaae32ba_0
2025-10-09T21:49:54.377370Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1659 sessionId: src_id|9cc2d30d-789835fd-fb6572f6-eaae32ba_0 is DEAD
2025-10-09T21:49:54.377575Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9cc2d30d-789835fd-fb6572f6-eaae32ba_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:54.377582Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9cc2d30d-789835fd-fb6572f6-eaae32ba_0] Write session will restart in 0.009108s
2025-10-09T21:49:54.377599Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9cc2d30d-789835fd-fb6572f6-eaae32ba_0] Write session: Do CDS request
2025-10-09T21:49:54.377607Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9cc2d30d-789835fd-fb6572f6-eaae32ba_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:54.377694Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:54.387530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9cc2d30d-789835fd-fb6572f6-eaae32ba_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:54.387585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:54.387600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1660
2025-10-09T21:49:54.387734Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1660 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:54.387780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1660 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:54.387791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1660 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:54.387795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:54.388271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1660 sessionId:  describe result for acl check
2025-10-09T21:49:54.388305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:54.388311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:54.388314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:54.388320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342561915932953:7263] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:54.388932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342561915932953:7263] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:54.391057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342561915932953:7263] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:54.393635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342561915932953:7263] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:54.393647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342561915932953:7263] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:54.393651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342561915932953:7263] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:54.393659Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1660 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:54.396700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:54.397180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:54.397339Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:54.397350Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:54.397359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:54.397396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:54.397425Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342561554103857:4084]
2025-10-09T21:49:54.397835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:54.398113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:54.398145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:54.398352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:54.398384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.398390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.398390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.398391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.398391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.398392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.398393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.398393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.398394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.398397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.398404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.398406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.398408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.398410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.398413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.398414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.398416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.398418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.398423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.398428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.398432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.398435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.398438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.398441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.398443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.398447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.398449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.398452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.398455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.398458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.398461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.398464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.398467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.398470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.398472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.398475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.398490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:54.404448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:54.405447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:54.405459Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:54.405461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:54.405463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:54.405470Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1659 [2:7559342561554103857:4084]
2025-10-09T21:49:54.405478Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:54.405492Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:54.405494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:54.405496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:54.405497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:54.405510Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:54.405520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:54.405539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:54.405580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.405586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.405587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.405589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:54.405637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:54.405642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.405648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.405655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.405664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:54.405681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342561554103852:4084]
2025-10-09T21:49:54.405687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342561554103852:4084]
2025-10-09T21:49:54.405688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342561554103852:4084]
2025-10-09T21:49:54.405694Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:54.405705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:54.405709Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:54.405713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:54.405713Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:54.405715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:54.405715Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:54.405723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342561915933075:7263], now have 1 active actors on pipe
2025-10-09T21:49:54.405726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342561915932833:2432], now have 1 active actors on pipe
2025-10-09T21:49:54.405742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342561554103852:4084]
2025-10-09T21:49:54.405748Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:54.405769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:54.405786Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1659
2025-10-09T21:49:54.405797Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1659
2025-10-09T21:49:54.405847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:54.405860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:54.405888Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c26f612b-e832da94-296d77b2-f0b1d1df_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:54.405900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:54.405909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:54.405911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:54.405926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:54.405939Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:54.405957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:54.406132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:54.406147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:54.406172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:54.406274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1660 partition: 0 MaxSeqNo: 25 sessionId: src_id|c26f612b-e832da94-296d77b2-f0b1d1df_0
2025-10-09T21:49:54.406595Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9cc2d30d-789835fd-fb6572f6-eaae32ba_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c26f612b-e832da94-296d77b2-f0b1d1df_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:54.408935Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c26f612b-e832da94-296d77b2-f0b1d1df_0] Write session: try to update token
2025-10-09T21:49:54.408950Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c26f612b-e832da94-296d77b2-f0b1d1df_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:54.416695Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:54.416714Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:54.416724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342561554103852:4084]
2025-10-09T21:49:54.416771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:54.416773Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:54.416932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:54.417071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:54.417208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:54.417336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:54.417577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:54.417615Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:54.421377Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1660 sessionId: src_id|c26f612b-e832da94-296d77b2-f0b1d1df_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:54.423675Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:54.423865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:54.423887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:54.423927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:54.424021Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:54.429852Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:54.429867Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:54.429870Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:54.429872Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:54.429881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342561554103852:4084]
2025-10-09T21:49:54.429884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342561554103852:4084]
2025-10-09T21:49:54.429888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342561554103852:4084]
2025-10-09T21:49:54.429926Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:54.429928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:54.429935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.429938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:54.429938Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:54.429942Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:54.430081Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:54.477049Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342561915933088:7774]
2025-10-09T21:49:54.477295Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 1659, pipe: [1:7559342561915933088:7774]
2025-10-09T21:49:54.573175Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:54.579779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:54.579803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:54.579945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:54.580002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:54.580065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:54.580126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:54.580166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:54.580175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:54.580241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:54.580297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:54.580352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:54.580418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:54.580447Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: CLIENT_DEADLINE_EXCEEDED. Description: [ { <main>: Error: GRpc error: (4): Deadline Exceeded } { <main>: Error: Grpc error response on endpoint localhost:14704 } ]
2025-10-09T21:49:54.580462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:54.580469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:54.580530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:54.580590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:54.580645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:54.580702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:54.580741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:54.580746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:54.580801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:54.580865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:54.580927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:54.580985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:54.581022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:54.581029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:54.581092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:54.581149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:54.581208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:54.581263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:54.581302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:54.581309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:54.581369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:54.581422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:54.581472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:54.581523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:54.581571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:54.581577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:54.581633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:54.581690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:54.581753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:54.581804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:54.581841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:54.581846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:54.581897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:54.581945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:54.581993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:54.582060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:54.582105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:54.582111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:54.582162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:54.582210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:54.582259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:54.582310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:54.582348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:54.582353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:54.582404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:54.582453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:54.582503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:54.582554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:54.582590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:54.582596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:54.582632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342561915933088:7774], now have 1 active actors on pipe
2025-10-09T21:49:54.582855Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:54.582863Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:54.582868Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:54.582900Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342561915933088:7774]
2025-10-09T21:49:54.582902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1660 reason: error at write: Disconnected sessionId: src_id|c26f612b-e832da94-296d77b2-f0b1d1df_0
2025-10-09T21:49:54.583036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1660 sessionId: src_id|c26f612b-e832da94-296d77b2-f0b1d1df_0 is DEAD
2025-10-09T21:49:54.583166Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c26f612b-e832da94-296d77b2-f0b1d1df_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:54.583171Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c26f612b-e832da94-296d77b2-f0b1d1df_0] Write session will restart in 0.009272s
2025-10-09T21:49:54.583189Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c26f612b-e832da94-296d77b2-f0b1d1df_0] Write session: Do CDS request
2025-10-09T21:49:54.583193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c26f612b-e832da94-296d77b2-f0b1d1df_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:54.583486Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:54.593472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c26f612b-e832da94-296d77b2-f0b1d1df_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:54.593549Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:54.593562Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1661
2025-10-09T21:49:54.593703Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1661 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:54.593735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1661 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:54.593745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1661 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:54.593748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:54.594216Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1661 sessionId:  describe result for acl check
2025-10-09T21:49:54.594249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:54.594251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:54.594253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:54.594260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342561915933204:7258] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:54.594698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342561915933204:7258] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:54.597158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342561915933204:7258] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:54.599010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:54.602301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:54.602447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342561915933204:7258] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:54.602462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342561915933204:7258] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:54.602466Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342561915933204:7258] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:54.602477Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1661 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:54.602479Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:54.602489Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:54.602499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:54.602548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:54.602584Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342561554104094:4077]
2025-10-09T21:49:54.603169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:54.603478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:54.603520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:54.603823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:54.603871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.603876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.603878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.603879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.603880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.603882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.603883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.603884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.603885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.603890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.603902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.603906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.603910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.603914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.603918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.603922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.603926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.603929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.603936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.603951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.603961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.603972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.603981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.603986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.603995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.604005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.604013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.604018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.604028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.604033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.604038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.604044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.604048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.604053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.604058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.604062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.604092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:54.610055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:54.611307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:54.611323Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:54.611325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:54.611328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:54.611336Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1660 [2:7559342561554104094:4077]
2025-10-09T21:49:54.611349Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:54.611363Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:54.611371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:54.611373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:54.611376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:54.611391Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:54.611401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:54.611412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:54.611474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.611480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.611482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.611485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:54.611542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:54.611549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.611559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.611574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.611594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:54.611619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342561915933336:7258], now have 1 active actors on pipe
2025-10-09T21:49:54.611627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342561915933091:2432], now have 1 active actors on pipe
2025-10-09T21:49:54.611633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342561554104089:4077]
2025-10-09T21:49:54.611635Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342561554104089:4077]
2025-10-09T21:49:54.611637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342561554104089:4077]
2025-10-09T21:49:54.611651Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:54.611667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:54.611673Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:54.611676Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:54.611679Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:54.611684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:54.611688Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:54.611676Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1660
2025-10-09T21:49:54.611682Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1660
2025-10-09T21:49:54.611725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:54.611737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:54.611749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342561554104089:4077]
2025-10-09T21:49:54.611756Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:54.611772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:54.611794Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5c0c7c2d-464e6fa2-79ab2436-54dadd6f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:54.611806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:54.611809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:54.611811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:54.611820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:54.611830Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:54.611853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:54.612007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:54.612017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:54.612030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:54.612103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1661 partition: 0 MaxSeqNo: 25 sessionId: src_id|5c0c7c2d-464e6fa2-79ab2436-54dadd6f_0
2025-10-09T21:49:54.612451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c26f612b-e832da94-296d77b2-f0b1d1df_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5c0c7c2d-464e6fa2-79ab2436-54dadd6f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:54.614848Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c0c7c2d-464e6fa2-79ab2436-54dadd6f_0] Write session: try to update token
2025-10-09T21:49:54.614865Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c0c7c2d-464e6fa2-79ab2436-54dadd6f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:54.625589Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:54.625608Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:54.625610Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:54.625612Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:54.625621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342561554104089:4077]
2025-10-09T21:49:54.625626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342561554104089:4077]
2025-10-09T21:49:54.625630Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342561554104089:4077]
2025-10-09T21:49:54.625674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:54.625680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.625686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:54.625685Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:54.625692Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:54.625699Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:54.625812Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:54.627775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1661 sessionId: src_id|5c0c7c2d-464e6fa2-79ab2436-54dadd6f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:54.630163Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:54.682862Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342561915933348:7774]
2025-10-09T21:49:54.683082Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 1660, pipe: [1:7559342561915933348:7774]
2025-10-09T21:49:54.780145Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:54.780757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:54.780771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:54.780820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342561915933348:7774], now have 1 active actors on pipe
2025-10-09T21:49:54.780985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:54.780995Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:54.781026Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342561915933348:7774]
2025-10-09T21:49:54.781037Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:54.781048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1661 reason: error at write: Disconnected sessionId: src_id|5c0c7c2d-464e6fa2-79ab2436-54dadd6f_0
2025-10-09T21:49:54.781066Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:54.781206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1661 sessionId: src_id|5c0c7c2d-464e6fa2-79ab2436-54dadd6f_0 is DEAD
2025-10-09T21:49:54.781373Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c0c7c2d-464e6fa2-79ab2436-54dadd6f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:54.781384Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c0c7c2d-464e6fa2-79ab2436-54dadd6f_0] Write session will restart in 0.005545s
2025-10-09T21:49:54.781412Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c0c7c2d-464e6fa2-79ab2436-54dadd6f_0] Write session: Do CDS request
2025-10-09T21:49:54.781419Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c0c7c2d-464e6fa2-79ab2436-54dadd6f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:54.788475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c0c7c2d-464e6fa2-79ab2436-54dadd6f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:54.788571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:54.788581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1662
2025-10-09T21:49:54.788738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1662 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:54.788794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1662 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:54.788805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1662 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:54.788809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:54.789154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1662 sessionId:  describe result for acl check
2025-10-09T21:49:54.789191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:54.789197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:54.789199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:54.789204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342561915933387:7249] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:54.789673Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342561915933387:7249] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:54.792247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342561915933387:7249] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:54.794812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342561915933387:7249] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:54.794825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342561915933387:7249] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:54.794827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342561915933387:7249] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:54.794833Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1662 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:54.801779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:54.802400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:54.802580Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:54.802590Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:54.802601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:54.802651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:54.802692Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342561554104338:4079]
2025-10-09T21:49:54.803273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:54.803594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:54.803631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:54.803930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:54.803975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.803980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.803981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.803982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.803984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.803985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.803986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.803987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.803988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.803993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.804003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.804006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.804010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.804013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.804017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.804020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.804023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.804027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.804034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.804041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.804046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.804052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.804055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.804060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.804064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.804068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.804073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.804077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.804081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.804090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.804094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.804099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.804103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.804108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.804114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.804123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.804149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:54.810696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:54.811724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:54.811738Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:54.811740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:54.811744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:54.811751Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1661 [2:7559342561554104338:4079]
2025-10-09T21:49:54.811766Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:54.811784Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:54.811791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:54.811794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:54.811797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:54.811814Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:54.811824Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:54.811835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:54.811894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.811900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.811902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.811905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:54.811971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:54.811979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.811990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.812004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.812025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:54.812052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342561915933553:7249], now have 1 active actors on pipe
2025-10-09T21:49:54.812063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342561915933351:2432], now have 1 active actors on pipe
2025-10-09T21:49:54.812071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342561554104333:4079]
2025-10-09T21:49:54.812073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342561554104333:4079]
2025-10-09T21:49:54.812076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342561554104333:4079]
2025-10-09T21:49:54.812093Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:54.812108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:54.812109Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:54.812112Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:54.812114Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:54.812123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:54.812126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:54.812101Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1661
2025-10-09T21:49:54.812116Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1661
2025-10-09T21:49:54.812171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:54.812182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:54.812200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342561554104333:4079]
2025-10-09T21:49:54.812209Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:54.812223Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:54.812245Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3c025d5c-dae09dcc-3a7dfad-de5bede5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:54.812258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:54.812264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:54.812272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:54.812282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:54.812296Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:54.812316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:54.812494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:54.812503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:54.812517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:54.812591Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1662 partition: 0 MaxSeqNo: 25 sessionId: src_id|3c025d5c-dae09dcc-3a7dfad-de5bede5_0
2025-10-09T21:49:54.812932Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c0c7c2d-464e6fa2-79ab2436-54dadd6f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3c025d5c-dae09dcc-3a7dfad-de5bede5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:54.815312Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c025d5c-dae09dcc-3a7dfad-de5bede5_0] Write session: try to update token
2025-10-09T21:49:54.815329Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c025d5c-dae09dcc-3a7dfad-de5bede5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:54.826925Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:54.826943Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:54.826946Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:54.826948Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:54.826958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342561554104333:4079]
2025-10-09T21:49:54.826963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342561554104333:4079]
2025-10-09T21:49:54.826967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342561554104333:4079]
2025-10-09T21:49:54.827014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:54.827016Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:54.827022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.827022Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:54.827025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:54.827027Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:54.827160Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:54.827902Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1662 sessionId: src_id|3c025d5c-dae09dcc-3a7dfad-de5bede5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:54.830241Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:54.881002Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342561915933614:7774]
2025-10-09T21:49:54.881246Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 1661, pipe: [1:7559342561915933614:7774]
2025-10-09T21:49:54.975349Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:54.975864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:54.975875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:54.975910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342561915933614:7774], now have 1 active actors on pipe
2025-10-09T21:49:54.976057Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:54.976096Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342561915933614:7774]
2025-10-09T21:49:54.976124Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:54.976155Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:54.976277Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1662 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3c025d5c-dae09dcc-3a7dfad-de5bede5_0
2025-10-09T21:49:54.976449Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1662 sessionId: src_id|3c025d5c-dae09dcc-3a7dfad-de5bede5_0 is DEAD
2025-10-09T21:49:54.976628Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c025d5c-dae09dcc-3a7dfad-de5bede5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:54.976637Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c025d5c-dae09dcc-3a7dfad-de5bede5_0] Write session will restart in 0.007001s
2025-10-09T21:49:54.976664Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c025d5c-dae09dcc-3a7dfad-de5bede5_0] Write session: Do CDS request
2025-10-09T21:49:54.976670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c025d5c-dae09dcc-3a7dfad-de5bede5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:54.984440Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c025d5c-dae09dcc-3a7dfad-de5bede5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:54.984515Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:54.984530Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1663
2025-10-09T21:49:54.984679Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1663 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:54.984715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1663 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:54.984723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1663 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:54.984727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:54.985101Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1663 sessionId:  describe result for acl check
2025-10-09T21:49:54.985128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:54.985130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:54.985131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:54.985135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342561915933694:7223] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:54.985567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342561915933694:7223] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:54.987567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342561915933694:7223] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:54.990404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342561915933694:7223] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:54.990416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342561915933694:7223] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:54.990421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342561915933694:7223] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:54.990428Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1663 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:54.994847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:54.995273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:54.995423Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:54.995433Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:54.995443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:54.995487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:54.995522Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342561554104590:4062]
2025-10-09T21:49:54.996092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:54.996365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:54.996403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:54.996692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:54.996736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.996741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.996742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.996743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.996745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.996746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.996747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.996748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.996749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.996754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.996764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.996768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.996772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.996776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.996780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.996783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.996787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.996791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.996797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.996806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:54.996811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.996816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:54.996821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.996826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:54.996830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.996835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:54.996840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.996844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:54.996850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.996855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:54.996871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.996876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:54.996880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.996885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:54.996889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.996894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:54.996920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:55.003174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:55.004398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:55.004415Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:55.004418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:55.004422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:55.004428Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1662 [2:7559342561554104590:4062]
2025-10-09T21:49:55.004443Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:55.004461Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:55.004469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:55.004472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:55.004475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:55.004493Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:55.004504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:55.004516Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:55.004578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.004585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.004587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.004590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:55.004662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:55.004669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.004683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.004701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.004725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:55.004752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342561915933632:2432], now have 1 active actors on pipe
2025-10-09T21:49:55.004762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342561915933872:7223], now have 1 active actors on pipe
2025-10-09T21:49:55.004770Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342561554104585:4062]
2025-10-09T21:49:55.004773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342561554104585:4062]
2025-10-09T21:49:55.004775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342561554104585:4062]
2025-10-09T21:49:55.004789Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:55.004807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:55.004811Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:55.004815Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:55.004817Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:55.004787Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1662
2025-10-09T21:49:55.004826Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:55.004829Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1662
2025-10-09T21:49:55.004829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:55.004872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342561554104585:4062]
2025-10-09T21:49:55.004885Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:55.004900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.004903Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:55.004910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.004962Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3c86685a-407e2e5-8872e2c3-4412b2d0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:55.004976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:55.004980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:55.004983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:55.004994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:55.005006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:55.005033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:55.005630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.005642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.005661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:55.005735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1663 partition: 0 MaxSeqNo: 25 sessionId: src_id|3c86685a-407e2e5-8872e2c3-4412b2d0_0
2025-10-09T21:49:55.006099Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c025d5c-dae09dcc-3a7dfad-de5bede5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3c86685a-407e2e5-8872e2c3-4412b2d0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:55.008519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c86685a-407e2e5-8872e2c3-4412b2d0_0] Write session: try to update token
2025-10-09T21:49:55.008536Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c86685a-407e2e5-8872e2c3-4412b2d0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:55.019223Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:55.019238Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:55.019240Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:55.019242Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:55.019251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342561554104585:4062]
2025-10-09T21:49:55.019255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342561554104585:4062]
2025-10-09T21:49:55.019258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342561554104585:4062]
2025-10-09T21:49:55.019302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:55.019308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.019310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:55.019314Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:55.019322Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:55.019326Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:55.019456Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:55.022458Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1663 sessionId: src_id|3c86685a-407e2e5-8872e2c3-4412b2d0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:55.024640Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:55.076094Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342566210901193:7774]
2025-10-09T21:49:55.076308Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 1662, pipe: [1:7559342566210901193:7774]
2025-10-09T21:49:55.166410Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:55.166867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.166882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.166920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342566210901193:7774], now have 1 active actors on pipe
2025-10-09T21:49:55.167150Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:55.167164Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342566210901193:7774]
2025-10-09T21:49:55.167190Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:55.167192Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:55.167290Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1663 reason: error at write: Disconnected sessionId: src_id|3c86685a-407e2e5-8872e2c3-4412b2d0_0
2025-10-09T21:49:55.167477Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1663 sessionId: src_id|3c86685a-407e2e5-8872e2c3-4412b2d0_0 is DEAD
2025-10-09T21:49:55.167605Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c86685a-407e2e5-8872e2c3-4412b2d0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:55.167623Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c86685a-407e2e5-8872e2c3-4412b2d0_0] Write session will restart in 0.009495s
2025-10-09T21:49:55.167648Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c86685a-407e2e5-8872e2c3-4412b2d0_0] Write session: Do CDS request
2025-10-09T21:49:55.167659Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c86685a-407e2e5-8872e2c3-4412b2d0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:55.167691Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:55.178534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c86685a-407e2e5-8872e2c3-4412b2d0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:55.178585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:55.178597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1664
2025-10-09T21:49:55.178744Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1664 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:55.178771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1664 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:55.178779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1664 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:55.178790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:55.179116Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1664 sessionId:  describe result for acl check
2025-10-09T21:49:55.179148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:55.179154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:55.179157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:55.179163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342566210901361:7215] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:55.179591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342566210901361:7215] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:55.181481Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342566210901361:7215] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:55.185579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:55.185823Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342566210901361:7215] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:55.185837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342566210901361:7215] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:55.185841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342566210901361:7215] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:55.185849Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1664 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:55.186024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:55.186204Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:55.186216Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:55.186228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:55.186273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:55.186310Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342565849072151:4038]
2025-10-09T21:49:55.186676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:55.186919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:55.186947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:55.187148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:55.187180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.187185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.187185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.187186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.187187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.187187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.187188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.187188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.187189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.187192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.187198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.187201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.187203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.187205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.187207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.187209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.187211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.187213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.187217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.187222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.187226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.187230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.187232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.187235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.187237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.187240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.187243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.187246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.187248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.187251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.187255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.187263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.187266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.187269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.187272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.187275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.187290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:55.193315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:55.194443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:55.194453Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:55.194455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:55.194458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:55.194463Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1663 [2:7559342565849072151:4038]
2025-10-09T21:49:55.194473Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:55.194489Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:55.194491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:55.194493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:55.194494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:55.194508Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:55.194511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:55.194517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:55.194561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.194567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.194569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.194571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:55.194622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:55.194628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.194635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.194641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.194650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:55.194665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342565849072145:4038]
2025-10-09T21:49:55.194669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342565849072145:4038]
2025-10-09T21:49:55.194671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342565849072145:4038]
2025-10-09T21:49:55.194677Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:55.194687Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:55.194688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342566210901197:2432], now have 1 active actors on pipe
2025-10-09T21:49:55.194690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342566210901449:7215], now have 1 active actors on pipe
2025-10-09T21:49:55.194692Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:55.194693Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:55.194697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:55.194705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:55.194707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:55.194729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342565849072145:4038]
2025-10-09T21:49:55.194738Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:55.194733Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1663
2025-10-09T21:49:55.194760Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:55.194774Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1663
2025-10-09T21:49:55.194831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.194839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.194861Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a19e09e-d728e8f2-2176a57e-278b0761_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:55.194869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:55.194872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:55.194874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:55.194883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:55.194890Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:55.194904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:55.195028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.195035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.195046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:55.195135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1664 partition: 0 MaxSeqNo: 25 sessionId: src_id|a19e09e-d728e8f2-2176a57e-278b0761_0
2025-10-09T21:49:55.195430Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c86685a-407e2e5-8872e2c3-4412b2d0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a19e09e-d728e8f2-2176a57e-278b0761_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:55.197658Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a19e09e-d728e8f2-2176a57e-278b0761_0] Write session: try to update token
2025-10-09T21:49:55.197672Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a19e09e-d728e8f2-2176a57e-278b0761_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:55.208723Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:55.208737Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:55.208740Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:55.208742Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:55.208751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342565849072145:4038]
2025-10-09T21:49:55.208756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342565849072145:4038]
2025-10-09T21:49:55.208760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342565849072145:4038]
2025-10-09T21:49:55.208803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:55.208810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.208811Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:55.208817Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:55.208821Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:55.208822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:55.208947Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:55.210486Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1664 sessionId: src_id|a19e09e-d728e8f2-2176a57e-278b0761_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:55.212635Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:55.267574Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342566210901465:7774]
2025-10-09T21:49:55.267791Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 1663, pipe: [1:7559342566210901465:7774]
2025-10-09T21:49:55.353418Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:55.354059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.354073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.354110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342566210901465:7774], now have 1 active actors on pipe
2025-10-09T21:49:55.354271Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:55.354312Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:55.354329Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342566210901465:7774]
2025-10-09T21:49:55.354349Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:55.354412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1664 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a19e09e-d728e8f2-2176a57e-278b0761_0
2025-10-09T21:49:55.354562Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1664 sessionId: src_id|a19e09e-d728e8f2-2176a57e-278b0761_0 is DEAD
2025-10-09T21:49:55.354701Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a19e09e-d728e8f2-2176a57e-278b0761_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:55.354712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a19e09e-d728e8f2-2176a57e-278b0761_0] Write session will restart in 0.009989s
2025-10-09T21:49:55.354733Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a19e09e-d728e8f2-2176a57e-278b0761_0] Write session: Do CDS request
2025-10-09T21:49:55.354737Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a19e09e-d728e8f2-2176a57e-278b0761_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:55.365487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a19e09e-d728e8f2-2176a57e-278b0761_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:55.365570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:55.365583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1665
2025-10-09T21:49:55.365689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1665 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:55.365716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1665 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:55.365723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1665 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:55.365727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:55.366186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1665 sessionId:  describe result for acl check
2025-10-09T21:49:55.366223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:55.366229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:55.366231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:55.366237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342566210901617:7206] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:55.366798Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342566210901617:7206] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:55.368838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342566210901617:7206] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:55.373298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:55.373512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342566210901617:7206] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:55.373525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342566210901617:7206] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:55.373541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342566210901617:7206] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:55.373551Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1665 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:55.373706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:55.373928Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:55.373939Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:55.373951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:55.374001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:55.374054Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342565849072392:4032]
2025-10-09T21:49:55.374564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:55.374845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:55.374894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:55.375210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:55.375261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.375267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.375268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.375269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.375271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.375272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.375273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.375274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.375275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.375280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.375291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.375295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.375304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.375308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.375312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.375316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.375324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.375328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.375338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.375350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.375361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.375371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.375380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.375386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.375395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.375400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.375405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.375410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.375414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.375424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.375429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.375434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.375442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.375447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.375456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.375461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.375494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:55.381524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:55.382922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:55.382940Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:55.382942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:55.382946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:55.382952Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1664 [2:7559342565849072392:4032]
2025-10-09T21:49:55.382965Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:55.382983Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:55.382985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:55.382987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:55.382988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:55.383003Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:55.383015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:55.383023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:55.383086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.383096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.383098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.383101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:55.383174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:55.383181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.383193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.383207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.383230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:55.383255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342566210901468:2432], now have 1 active actors on pipe
2025-10-09T21:49:55.383264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342566210901717:7206], now have 1 active actors on pipe
2025-10-09T21:49:55.383271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342565849072386:4032]
2025-10-09T21:49:55.383274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342565849072386:4032]
2025-10-09T21:49:55.383276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342565849072386:4032]
2025-10-09T21:49:55.383289Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:55.383303Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:55.383308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:55.383312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:55.383314Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:55.383321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:55.383326Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:55.383342Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1664
2025-10-09T21:49:55.383355Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342565849072386:4032]
2025-10-09T21:49:55.383360Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1664
2025-10-09T21:49:55.383369Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:55.383377Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:55.383406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.383411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.383440Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|de7070d4-9149f0ca-485d9b8b-9b83fb60_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:55.383454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:55.383458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:55.383461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:55.383470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:55.383484Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:55.383503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:55.383642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.383650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.383665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:55.383754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1665 partition: 0 MaxSeqNo: 25 sessionId: src_id|de7070d4-9149f0ca-485d9b8b-9b83fb60_0
2025-10-09T21:49:55.384092Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a19e09e-d728e8f2-2176a57e-278b0761_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|de7070d4-9149f0ca-485d9b8b-9b83fb60_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:55.386586Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|de7070d4-9149f0ca-485d9b8b-9b83fb60_0] Write session: try to update token
2025-10-09T21:49:55.386605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|de7070d4-9149f0ca-485d9b8b-9b83fb60_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:55.398875Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1665 sessionId: src_id|de7070d4-9149f0ca-485d9b8b-9b83fb60_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:55.400501Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:55.400515Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:55.400525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342565849072386:4032]
2025-10-09T21:49:55.400569Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:55.400571Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:55.400750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:55.400892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:55.401021Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:55.401150Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:55.401270Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:55.401331Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:55.401366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:55.401415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.401424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.401447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:55.401544Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:55.407539Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:55.407550Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:55.407552Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:55.407553Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:55.407560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342565849072386:4032]
2025-10-09T21:49:55.407564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342565849072386:4032]
2025-10-09T21:49:55.407568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342565849072386:4032]
2025-10-09T21:49:55.407602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:55.407608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.407608Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:55.407610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:55.407612Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:55.407615Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:55.407720Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:55.454451Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342566210901728:7774]
2025-10-09T21:49:55.454670Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 1664, pipe: [1:7559342566210901728:7774]
2025-10-09T21:49:55.547472Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:55.553973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.553997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.554146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:55.554197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:55.554252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:55.554305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:55.554345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:55.554353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:55.554412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:55.554463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:55.554517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:55.554570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:55.554615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:55.554624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:55.554680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:55.554728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:55.554779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:55.554828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:55.554864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:55.554869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:55.554923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:55.554975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:55.555025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:55.555085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:55.555126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:55.555132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:55.555188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:55.555237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:55.555287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:55.555335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:55.555373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:55.555383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:55.555439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:55.555489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:55.555539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:55.555588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:55.555627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:55.555633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:55.555682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:55.555731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:55.555783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:55.555864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:55.555912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:55.555917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:55.555964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:55.556010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:55.556057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:55.556105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:55.556142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:55.556150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:55.556202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:55.556249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:55.556302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:55.556351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:55.556389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:55.556398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:55.556459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:55.556507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:55.556557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:55.556606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:55.556642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:55.556647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:55.556685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342566210901728:7774], now have 1 active actors on pipe
2025-10-09T21:49:55.556897Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:55.556913Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342566210901728:7774]
2025-10-09T21:49:55.556943Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:55.556945Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:55.556975Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1665 reason: error at write: Disconnected sessionId: src_id|de7070d4-9149f0ca-485d9b8b-9b83fb60_0
2025-10-09T21:49:55.557186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1665 sessionId: src_id|de7070d4-9149f0ca-485d9b8b-9b83fb60_0 is DEAD
2025-10-09T21:49:55.557283Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de7070d4-9149f0ca-485d9b8b-9b83fb60_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:55.557291Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de7070d4-9149f0ca-485d9b8b-9b83fb60_0] Write session will restart in 0.005224s
2025-10-09T21:49:55.557308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de7070d4-9149f0ca-485d9b8b-9b83fb60_0] Write session: Do CDS request
2025-10-09T21:49:55.557312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de7070d4-9149f0ca-485d9b8b-9b83fb60_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:55.557313Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:55.563503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|de7070d4-9149f0ca-485d9b8b-9b83fb60_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:55.563570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:55.563581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1666
2025-10-09T21:49:55.563717Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1666 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:55.563750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1666 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:55.563763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1666 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:55.563767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:55.564338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1666 sessionId:  describe result for acl check
2025-10-09T21:49:55.564372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:55.564377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:55.564379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:55.564384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342566210901760:7205] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:55.564929Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342566210901760:7205] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:55.568293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342566210901760:7205] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:55.571389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342566210901760:7205] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:55.571401Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342566210901760:7205] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:55.571405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342566210901760:7205] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:55.571411Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1666 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:55.572999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:55.575827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:55.576001Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:55.576011Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:55.576022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:55.576073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:55.576111Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342565849072634:4025]
2025-10-09T21:49:55.576685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:55.576939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:55.576972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:55.577217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:55.577257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.577261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.577262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.577263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.577263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.577264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.577264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.577265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.577265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.577269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.577276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.577278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.577280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.577282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.577284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.577286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.577288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.577290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.577294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.577302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.577308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.577313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.577317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.577322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.577327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.577332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.577341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.577347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.577355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.577360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.577368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.577376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.577378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.577381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.577383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.577386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.577410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:55.583703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:55.584798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:55.584808Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:55.584810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:55.584812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:55.584817Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1665 [2:7559342565849072634:4025]
2025-10-09T21:49:55.584825Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:55.584838Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:55.584841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:55.584842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:55.584844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:55.584855Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:55.584860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:55.584866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:55.584903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.584908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.584909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.584911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:55.584978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:55.584988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.584999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.585011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.585029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:55.585053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342566210901731:2432], now have 1 active actors on pipe
2025-10-09T21:49:55.585060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342566210901978:7205], now have 1 active actors on pipe
2025-10-09T21:49:55.585068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342565849072629:4025]
2025-10-09T21:49:55.585071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342565849072629:4025]
2025-10-09T21:49:55.585072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342565849072629:4025]
2025-10-09T21:49:55.585087Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:55.585106Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:55.585111Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:55.585117Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:55.585119Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:55.585123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:55.585126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:55.585122Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1665
2025-10-09T21:49:55.585155Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342565849072629:4025]
2025-10-09T21:49:55.585140Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1665
2025-10-09T21:49:55.585167Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:55.585169Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:55.585195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.585204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.585241Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2daff0a2-4debb49a-3dd25d90-7787069c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:55.585252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:55.585256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:55.585259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:55.585268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:55.585282Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:55.585316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:55.585460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.585471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.585488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:55.585624Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1666 partition: 0 MaxSeqNo: 25 sessionId: src_id|2daff0a2-4debb49a-3dd25d90-7787069c_0
2025-10-09T21:49:55.585937Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de7070d4-9149f0ca-485d9b8b-9b83fb60_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2daff0a2-4debb49a-3dd25d90-7787069c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:55.588345Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2daff0a2-4debb49a-3dd25d90-7787069c_0] Write session: try to update token
2025-10-09T21:49:55.588361Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2daff0a2-4debb49a-3dd25d90-7787069c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:55.599293Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:55.599305Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:55.599307Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:55.599308Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:55.599316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342565849072629:4025]
2025-10-09T21:49:55.599320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342565849072629:4025]
2025-10-09T21:49:55.599323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342565849072629:4025]
2025-10-09T21:49:55.599356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:55.599362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.599363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:55.599378Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:55.599385Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:55.599389Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:55.599489Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:55.601352Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1666 sessionId: src_id|2daff0a2-4debb49a-3dd25d90-7787069c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:55.603693Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:55.657207Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342566210901992:7774]
2025-10-09T21:49:55.657452Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 1665, pipe: [1:7559342566210901992:7774]
2025-10-09T21:49:55.743306Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:55.743914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.743927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.743963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342566210901992:7774], now have 1 active actors on pipe
2025-10-09T21:49:55.744130Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:55.744186Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:55.744202Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342566210901992:7774]
2025-10-09T21:49:55.744228Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:55.744308Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1666 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2daff0a2-4debb49a-3dd25d90-7787069c_0
2025-10-09T21:49:55.744484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1666 sessionId: src_id|2daff0a2-4debb49a-3dd25d90-7787069c_0 is DEAD
2025-10-09T21:49:55.744615Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2daff0a2-4debb49a-3dd25d90-7787069c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:55.744627Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2daff0a2-4debb49a-3dd25d90-7787069c_0] Write session will restart in 0.007428s
2025-10-09T21:49:55.744654Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2daff0a2-4debb49a-3dd25d90-7787069c_0] Write session: Do CDS request
2025-10-09T21:49:55.744660Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2daff0a2-4debb49a-3dd25d90-7787069c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:55.753522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2daff0a2-4debb49a-3dd25d90-7787069c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:55.753617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:55.753632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1667
2025-10-09T21:49:55.753789Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1667 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:55.753821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1667 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:55.753840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1667 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:55.753848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:55.754305Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1667 sessionId:  describe result for acl check
2025-10-09T21:49:55.754345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:55.754347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:55.754349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:55.754355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342566210902062:7200] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:55.754859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342566210902062:7200] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:55.757424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342566210902062:7200] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:55.760670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342566210902062:7200] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:55.760684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342566210902062:7200] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:55.760687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342566210902062:7200] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:55.760697Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1667 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:55.763625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:55.767009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:55.767215Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:55.767225Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:55.767236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:55.767291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:55.767400Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342565849072882:4013]
2025-10-09T21:49:55.767894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:55.768176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:55.768208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:55.768466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:55.768509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.768515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.768516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.768517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.768518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.768518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.768519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.768520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.768521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.768524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.768531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.768533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.768535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.768537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.768540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.768542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.768544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.768546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.768551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.768556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.768559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.768564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.768566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.768570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.768573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.768576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.768579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.768584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.768587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.768590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.768593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.768596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.768599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.768602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.768604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.768607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.768627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:55.775113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:55.776303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:55.776321Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:55.776324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:55.776327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:55.776335Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1666 [2:7559342565849072882:4013]
2025-10-09T21:49:55.776351Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:55.776368Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:55.776377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:55.776380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:55.776382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:55.776397Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:55.776403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:55.776416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:55.776479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.776488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.776490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.776493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:55.776568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:55.776577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.776589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.776606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.776624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:55.776640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342566210901995:2432], now have 1 active actors on pipe
2025-10-09T21:49:55.776644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342566210902233:7200], now have 1 active actors on pipe
2025-10-09T21:49:55.776651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342565849072876:4013]
2025-10-09T21:49:55.776653Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342565849072876:4013]
2025-10-09T21:49:55.776656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342565849072876:4013]
2025-10-09T21:49:55.776673Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:55.776695Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:55.776709Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:55.776716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:55.776720Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:55.776720Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:55.776722Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:55.776713Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1666
2025-10-09T21:49:55.776745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342565849072876:4013]
2025-10-09T21:49:55.776756Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:55.776758Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:55.776760Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1666
2025-10-09T21:49:55.776833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.776842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.776864Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|26444aad-ceba4b06-9cef091f-937e1c30_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:55.776876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:55.776879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:55.776882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:55.776890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:55.776904Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:55.776925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:55.777097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.777106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.777117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:55.777213Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1667 partition: 0 MaxSeqNo: 25 sessionId: src_id|26444aad-ceba4b06-9cef091f-937e1c30_0
2025-10-09T21:49:55.777562Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2daff0a2-4debb49a-3dd25d90-7787069c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|26444aad-ceba4b06-9cef091f-937e1c30_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:55.779922Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26444aad-ceba4b06-9cef091f-937e1c30_0] Write session: try to update token
2025-10-09T21:49:55.779938Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26444aad-ceba4b06-9cef091f-937e1c30_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:55.791984Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:55.792005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:55.792009Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:55.792012Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:55.792024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342565849072876:4013]
2025-10-09T21:49:55.792035Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342565849072876:4013]
2025-10-09T21:49:55.792042Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342565849072876:4013]
2025-10-09T21:49:55.792094Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:55.792105Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:55.792105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:55.792109Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:55.792115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.792119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:55.792255Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:55.792516Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1667 sessionId: src_id|26444aad-ceba4b06-9cef091f-937e1c30_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:55.794774Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:55.844609Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342566210902258:7774]
2025-10-09T21:49:55.845033Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 1666, pipe: [1:7559342566210902258:7774]
2025-10-09T21:49:55.935251Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:55.935918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.935930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.935967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342566210902258:7774], now have 1 active actors on pipe
2025-10-09T21:49:55.936145Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:55.936212Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:55.936222Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342566210902258:7774]
2025-10-09T21:49:55.936243Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:55.936289Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1667 reason: status is not ok: tablet will be restarted right now sessionId: src_id|26444aad-ceba4b06-9cef091f-937e1c30_0
2025-10-09T21:49:55.936441Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1667 sessionId: src_id|26444aad-ceba4b06-9cef091f-937e1c30_0 is DEAD
2025-10-09T21:49:55.936624Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26444aad-ceba4b06-9cef091f-937e1c30_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:55.936635Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26444aad-ceba4b06-9cef091f-937e1c30_0] Write session will restart in 0.008137s
2025-10-09T21:49:55.936657Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26444aad-ceba4b06-9cef091f-937e1c30_0] Write session: Do CDS request
2025-10-09T21:49:55.936664Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26444aad-ceba4b06-9cef091f-937e1c30_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:55.945515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26444aad-ceba4b06-9cef091f-937e1c30_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:55.945604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:55.945619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1668
2025-10-09T21:49:55.945724Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1668 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:55.945758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1668 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:55.945768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1668 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:55.945771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:55.946302Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1668 sessionId:  describe result for acl check
2025-10-09T21:49:55.946338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:55.946345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:55.946346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:55.946353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342566210902360:7187] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:55.946852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342566210902360:7187] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:55.949409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342566210902360:7187] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:55.951913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:55.952192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342566210902360:7187] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:55.952205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342566210902360:7187] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:55.952219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342566210902360:7187] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:55.952226Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1668 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:55.955341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:55.955539Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:55.955551Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:55.955563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:55.955619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:55.955660Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342565849073126:4021]
2025-10-09T21:49:55.956179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:55.956453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:55.956503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:55.956817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:55.956869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.956875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.956876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.956877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.956878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.956880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.956881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.956882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.956883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.956889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.956900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.956903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.956906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.956910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.956914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.956918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.956922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.956925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.956933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.956940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:55.956946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.956956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:55.956966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.956975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:55.956980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.956989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:55.956997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.957006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:55.957014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.957019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:55.957023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.957028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.957033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.957042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.957047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.957056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.957084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:55.963071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:55.964483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:55.964494Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:55.964495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:55.964498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:55.964503Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1667 [2:7559342565849073126:4021]
2025-10-09T21:49:55.964512Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:55.964526Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:55.964531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:55.964533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:55.964534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:55.964547Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:55.964554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:55.964559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:55.964608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.964613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.964614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.964616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:55.964676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:55.964682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.964689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:55.964696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:55.964705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:55.964721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342565849073120:4021]
2025-10-09T21:49:55.964722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342565849073120:4021]
2025-10-09T21:49:55.964723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342565849073120:4021]
2025-10-09T21:49:55.964729Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:55.964742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342566210902513:7187], now have 1 active actors on pipe
2025-10-09T21:49:55.964744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342566210902262:2432], now have 1 active actors on pipe
2025-10-09T21:49:55.964752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:55.964752Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:55.964755Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:55.964757Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:55.964768Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:55.964772Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:55.964800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342565849073120:4021]
2025-10-09T21:49:55.964811Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:55.964830Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:55.964820Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1667
2025-10-09T21:49:55.964834Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1667
2025-10-09T21:49:55.964881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.964887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.964914Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|287a2b5a-1f2e899b-f66b95d2-549bb630_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:55.964925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:55.964930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:55.964936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:55.964945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:55.964956Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:55.964972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:55.965106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.965110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.965120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:55.965201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1668 partition: 0 MaxSeqNo: 25 sessionId: src_id|287a2b5a-1f2e899b-f66b95d2-549bb630_0
2025-10-09T21:49:55.965509Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26444aad-ceba4b06-9cef091f-937e1c30_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|287a2b5a-1f2e899b-f66b95d2-549bb630_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:55.967788Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|287a2b5a-1f2e899b-f66b95d2-549bb630_0] Write session: try to update token
2025-10-09T21:49:55.967805Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|287a2b5a-1f2e899b-f66b95d2-549bb630_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:55.975495Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:55.975507Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:55.975515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342565849073120:4021]
2025-10-09T21:49:55.975555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:55.975559Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:55.975728Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:55.975854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:55.975973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:55.976093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:55.976269Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:55.976301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:55.979513Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1668 sessionId: src_id|287a2b5a-1f2e899b-f66b95d2-549bb630_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:55.981913Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:55.982195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:55.982211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:55.982250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:55.982323Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:55.988038Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:55.988051Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:55.988052Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:55.988053Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:55.988061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342565849073120:4021]
2025-10-09T21:49:55.988064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342565849073120:4021]
2025-10-09T21:49:55.988067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342565849073120:4021]
2025-10-09T21:49:55.988097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:55.988103Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:55.988106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:55.988107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:55.988108Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:55.988110Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:55.988221Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:56.036611Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342570505869835:7774]
2025-10-09T21:49:56.036918Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 1667, pipe: [1:7559342570505869835:7774]
2025-10-09T21:49:56.133239Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:56.139937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:56.139957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:56.140090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:56.140145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:56.140201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:56.140256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:56.140295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:56.140302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:56.140360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:56.140414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:56.140468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:56.140523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:56.140561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:56.140565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:56.140615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:56.140664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:56.140715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:56.140767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:56.140803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:56.140809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:56.140863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:56.140916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:56.140968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:56.141024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:56.141069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:56.141078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:56.141132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:56.141183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:56.141256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:56.141307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:56.141352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:56.141361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:56.141415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:56.141467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:56.141518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:56.141583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:56.141625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:56.141630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:56.141684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:56.141734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:56.141784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:56.141843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:56.141884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:56.141891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:56.141940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:56.141987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:56.142050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:56.142109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:56.142149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:56.142154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:56.142206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:56.142257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:56.142319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:56.142378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:56.142418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:56.142424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:56.142475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:56.142527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:56.142580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:56.142640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:56.142677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:56.142682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:56.142721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342570505869835:7774], now have 1 active actors on pipe
2025-10-09T21:49:56.142945Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:56.142958Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:56.142962Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342570505869835:7774]
2025-10-09T21:49:56.142984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1668 reason: error at write: Disconnected sessionId: src_id|287a2b5a-1f2e899b-f66b95d2-549bb630_0
2025-10-09T21:49:56.142986Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:56.143146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1668 sessionId: src_id|287a2b5a-1f2e899b-f66b95d2-549bb630_0 is DEAD
2025-10-09T21:49:56.143280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|287a2b5a-1f2e899b-f66b95d2-549bb630_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:56.143289Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|287a2b5a-1f2e899b-f66b95d2-549bb630_0] Write session will restart in 0.007222s
2025-10-09T21:49:56.143307Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|287a2b5a-1f2e899b-f66b95d2-549bb630_0] Write session: Do CDS request
2025-10-09T21:49:56.143312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|287a2b5a-1f2e899b-f66b95d2-549bb630_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:56.143425Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:56.151501Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|287a2b5a-1f2e899b-f66b95d2-549bb630_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:56.151579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:56.151591Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1669
2025-10-09T21:49:56.151713Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1669 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:56.151744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1669 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:56.151755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1669 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:56.151758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:56.152255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1669 sessionId:  describe result for acl check
2025-10-09T21:49:56.152292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:56.152299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:56.152301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:56.152307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342570505869893:7172] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:56.152860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342570505869893:7172] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:56.155639Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342570505869893:7172] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:56.158477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342570505869893:7172] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:56.158489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342570505869893:7172] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:56.158492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342570505869893:7172] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:56.158497Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1669 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:56.163755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:56.164211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:56.164364Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:56.164372Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:56.164381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:56.164416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:56.164446Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342570144040705:3991]
2025-10-09T21:49:56.164814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:56.165088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:56.165120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:56.165333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:56.165363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.165363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.165364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.165365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.165366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.165367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.165367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.165368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.165369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.165372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.165378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.165380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.165383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.165385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.165387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.165389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.165391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.165393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.165397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.165403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.165407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.165409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.165413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.165415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.165418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.165421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.165423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.165426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.165429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.165432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.165435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.165438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.165440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.165443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.165446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.165449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.165466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:56.171704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:56.172833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:56.172843Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:56.172845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:56.172848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:56.172854Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1668 [2:7559342570144040705:3991]
2025-10-09T21:49:56.172863Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:56.172879Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:56.172881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:56.172883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:56.172885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:56.172898Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:56.172905Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:56.172911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:56.172952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.172958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.172959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.172961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:56.173015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:56.173022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.173033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.173047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.173060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:56.173081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342570505870099:7172], now have 1 active actors on pipe
2025-10-09T21:49:56.173085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342570505869848:2432], now have 1 active actors on pipe
2025-10-09T21:49:56.173093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342570144040700:3991]
2025-10-09T21:49:56.173100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342570144040700:3991]
2025-10-09T21:49:56.173102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342570144040700:3991]
2025-10-09T21:49:56.173115Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:56.173125Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:56.173128Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:56.173130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:56.173138Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:56.173152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:56.173155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:56.173137Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1668
2025-10-09T21:49:56.173151Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1668
2025-10-09T21:49:56.173199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:56.173207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:56.173217Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342570144040700:3991]
2025-10-09T21:49:56.173226Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:56.173230Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:56.173247Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|57afe4f-8a2fb208-361b85f9-f41182f0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:56.173256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:56.173260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:56.173267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:56.173278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:56.173292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:56.173314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:56.173413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:56.173420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:56.173432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:56.173512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1669 partition: 0 MaxSeqNo: 25 sessionId: src_id|57afe4f-8a2fb208-361b85f9-f41182f0_0
2025-10-09T21:49:56.173854Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|287a2b5a-1f2e899b-f66b95d2-549bb630_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|57afe4f-8a2fb208-361b85f9-f41182f0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:56.176182Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57afe4f-8a2fb208-361b85f9-f41182f0_0] Write session: try to update token
2025-10-09T21:49:56.176198Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57afe4f-8a2fb208-361b85f9-f41182f0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:56.187234Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:56.187249Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:56.187250Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:56.187252Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:56.187260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342570144040700:3991]
2025-10-09T21:49:56.187264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342570144040700:3991]
2025-10-09T21:49:56.187267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342570144040700:3991]
2025-10-09T21:49:56.187305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:56.187312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.187319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:56.187321Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:56.187328Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:56.187333Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:56.187452Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:56.188285Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1669 sessionId: src_id|57afe4f-8a2fb208-361b85f9-f41182f0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:56.190543Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:56.242927Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342570505870117:7774]
2025-10-09T21:49:56.243232Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 1668, pipe: [1:7559342570505870117:7774]
2025-10-09T21:49:56.340910Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:56.341699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:56.341713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:56.341752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342570505870117:7774], now have 1 active actors on pipe
2025-10-09T21:49:56.341950Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:56.341981Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:56.341998Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:56.342003Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:56.342066Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342570505870117:7774]
2025-10-09T21:49:56.342071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1669 reason: error at write: Disconnected sessionId: src_id|57afe4f-8a2fb208-361b85f9-f41182f0_0
2025-10-09T21:49:56.342238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1669 sessionId: src_id|57afe4f-8a2fb208-361b85f9-f41182f0_0 is DEAD
2025-10-09T21:49:56.342402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57afe4f-8a2fb208-361b85f9-f41182f0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:56.342425Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57afe4f-8a2fb208-361b85f9-f41182f0_0] Write session will restart in 0.008255s
2025-10-09T21:49:56.342451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57afe4f-8a2fb208-361b85f9-f41182f0_0] Write session: Do CDS request
2025-10-09T21:49:56.342458Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57afe4f-8a2fb208-361b85f9-f41182f0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:56.351472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57afe4f-8a2fb208-361b85f9-f41182f0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:56.351635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:56.351649Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1670
2025-10-09T21:49:56.351775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1670 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:56.351813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1670 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:56.351821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1670 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:56.351825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:56.352301Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1670 sessionId:  describe result for acl check
2025-10-09T21:49:56.352343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:56.352352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:56.352354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:56.352360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342570505870216:7160] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:56.353054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342570505870216:7160] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:56.355378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342570505870216:7160] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:56.357731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342570505870216:7160] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:56.357743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342570505870216:7160] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:56.357745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342570505870216:7160] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:56.357752Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1670 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:56.362314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:56.362742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:56.362912Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:56.362921Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:56.362934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:56.362980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:56.363022Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342570144040951:3999]
2025-10-09T21:49:56.363613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:56.363902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:56.363947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:56.364239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:56.364300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.364306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.364307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.364308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.364310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.364311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.364312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.364313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.364315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.364320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.364331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.364335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.364339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.364343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.364347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.364350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.364353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.364357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.364364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.364372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.364379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.364384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.364394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.364400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.364410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.364415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.364420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.364427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.364435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.364441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.364446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.364451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.364456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.364461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.364468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.364473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.364506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:56.370609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:56.371672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:56.371687Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:56.371689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:56.371694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:56.371701Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1669 [2:7559342570144040951:3999]
2025-10-09T21:49:56.371714Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:56.371736Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:56.371744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:56.371747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:56.371749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:56.371765Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:56.371776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:56.371784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:56.371848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.371855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.371857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.371860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:56.371978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:56.371985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.371998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.372016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.372039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:56.372071Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1669
2025-10-09T21:49:56.372066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342570505870376:7160], now have 1 active actors on pipe
2025-10-09T21:49:56.372073Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1669
2025-10-09T21:49:56.372076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342570505870125:2432], now have 1 active actors on pipe
2025-10-09T21:49:56.372084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342570144040946:3999]
2025-10-09T21:49:56.372086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342570144040946:3999]
2025-10-09T21:49:56.372089Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342570144040946:3999]
2025-10-09T21:49:56.372098Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:56.372111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:56.372120Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:56.372126Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:56.372126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:56.372128Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:56.372129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:56.372172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:56.372181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:56.372198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342570144040946:3999]
2025-10-09T21:49:56.372210Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:56.372228Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:56.372242Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|df256c24-59ebca04-f3086da8-65431c31_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:56.372253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:56.372257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:56.372260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:56.372270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:56.372284Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:56.372311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:56.372458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:56.372467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:56.372482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:56.372554Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1670 partition: 0 MaxSeqNo: 25 sessionId: src_id|df256c24-59ebca04-f3086da8-65431c31_0
2025-10-09T21:49:56.372871Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57afe4f-8a2fb208-361b85f9-f41182f0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|df256c24-59ebca04-f3086da8-65431c31_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:56.375216Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df256c24-59ebca04-f3086da8-65431c31_0] Write session: try to update token
2025-10-09T21:49:56.375232Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df256c24-59ebca04-f3086da8-65431c31_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:56.386604Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:56.386622Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:56.386624Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:56.386627Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:56.386637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342570144040946:3999]
2025-10-09T21:49:56.386642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342570144040946:3999]
2025-10-09T21:49:56.386646Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342570144040946:3999]
2025-10-09T21:49:56.386694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:56.386695Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:56.386701Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:56.386705Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:56.386710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.386713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:56.386845Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:56.387286Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1670 sessionId: src_id|df256c24-59ebca04-f3086da8-65431c31_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:56.389508Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:56.442094Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342570505870390:7774]
2025-10-09T21:49:56.442343Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 1669, pipe: [1:7559342570505870390:7774]
2025-10-09T21:49:56.533917Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:56.534761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:56.534777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:56.534822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342570505870390:7774], now have 1 active actors on pipe
2025-10-09T21:49:56.535027Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:56.535024Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:56.535049Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:56.535052Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:56.535081Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342570505870390:7774]
2025-10-09T21:49:56.535122Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1670 reason: error at write: Disconnected sessionId: src_id|df256c24-59ebca04-f3086da8-65431c31_0
2025-10-09T21:49:56.535279Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1670 sessionId: src_id|df256c24-59ebca04-f3086da8-65431c31_0 is DEAD
2025-10-09T21:49:56.535455Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df256c24-59ebca04-f3086da8-65431c31_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:56.535478Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df256c24-59ebca04-f3086da8-65431c31_0] Write session will restart in 0.009914s
2025-10-09T21:49:56.535504Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df256c24-59ebca04-f3086da8-65431c31_0] Write session: Do CDS request
2025-10-09T21:49:56.535511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df256c24-59ebca04-f3086da8-65431c31_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:56.546464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df256c24-59ebca04-f3086da8-65431c31_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:56.546563Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:56.546586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1671
2025-10-09T21:49:56.546711Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1671 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:56.546755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1671 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:56.546767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1671 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:56.546771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:56.547194Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1671 sessionId:  describe result for acl check
2025-10-09T21:49:56.547226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:56.547232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:56.547234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:56.547240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342570505870518:7152] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:56.547767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342570505870518:7152] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:56.550396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342570505870518:7152] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:56.554983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:56.555153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342570505870518:7152] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:56.555168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342570505870518:7152] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:56.555172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342570505870518:7152] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:56.555181Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1671 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:56.555377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:56.555540Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:56.555550Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:56.555562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:56.555609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:56.555647Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342570144041198:3997]
2025-10-09T21:49:56.556226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:56.556498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:56.556534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:56.556835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:56.556894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.556899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.556900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.556902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.556903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.556904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.556905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.556906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.556907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.556912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.556922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.556925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.556929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.556933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.556936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.556940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.556944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.556949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.556955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.556963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.556969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.556974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.556978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.556982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.556987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.556992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.556996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.557000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.557004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.557010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.557015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.557020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.557024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.557029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.557039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.557044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.557071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:56.563074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:56.564202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:56.564217Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:56.564220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:56.564223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:56.564229Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1670 [2:7559342570144041198:3997]
2025-10-09T21:49:56.564243Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:56.564258Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:56.564261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:56.564264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:56.564267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:56.564282Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:56.564290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:56.564298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:56.564350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.564357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.564359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.564362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:56.564425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:56.564433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.564443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.564453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.564472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:56.564495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342570505870648:7152], now have 1 active actors on pipe
2025-10-09T21:49:56.564503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342570505870393:2432], now have 1 active actors on pipe
2025-10-09T21:49:56.564510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342570144041192:3997]
2025-10-09T21:49:56.564512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342570144041192:3997]
2025-10-09T21:49:56.564514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342570144041192:3997]
2025-10-09T21:49:56.564526Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:56.564544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:56.564547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:56.564550Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:56.564552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:56.564561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:56.564537Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1670
2025-10-09T21:49:56.564550Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1670
2025-10-09T21:49:56.564565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:56.564587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342570144041192:3997]
2025-10-09T21:49:56.564590Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:56.564592Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:56.564605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:56.564609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:56.564657Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2952384d-d5bb808c-b29a39c1-48180d10_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:56.564670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:56.564674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:56.564676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:56.564686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:56.564695Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:56.564716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:56.564947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:56.564957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:56.564972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:56.565087Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1671 partition: 0 MaxSeqNo: 25 sessionId: src_id|2952384d-d5bb808c-b29a39c1-48180d10_0
2025-10-09T21:49:56.565409Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df256c24-59ebca04-f3086da8-65431c31_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2952384d-d5bb808c-b29a39c1-48180d10_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:56.567915Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2952384d-d5bb808c-b29a39c1-48180d10_0] Write session: try to update token
2025-10-09T21:49:56.567934Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2952384d-d5bb808c-b29a39c1-48180d10_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:56.578656Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:56.578671Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:56.578673Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:56.578675Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:56.578685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342570144041192:3997]
2025-10-09T21:49:56.578689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342570144041192:3997]
2025-10-09T21:49:56.578693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342570144041192:3997]
2025-10-09T21:49:56.578737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:56.578740Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:56.578745Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:56.578746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.578749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:56.578749Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:56.578882Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:56.580461Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1671 sessionId: src_id|2952384d-d5bb808c-b29a39c1-48180d10_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:56.582892Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:56.635134Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342570505870663:7774]
2025-10-09T21:49:56.635439Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 1670, pipe: [1:7559342570505870663:7774]
2025-10-09T21:49:56.726803Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:56.727023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:56.727039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:56.727091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342570505870663:7774], now have 1 active actors on pipe
2025-10-09T21:49:56.727257Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:56.727283Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342570505870663:7774]
2025-10-09T21:49:56.727291Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:56.727318Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:56.727392Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1671 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2952384d-d5bb808c-b29a39c1-48180d10_0
2025-10-09T21:49:56.727545Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1671 sessionId: src_id|2952384d-d5bb808c-b29a39c1-48180d10_0 is DEAD
2025-10-09T21:49:56.727697Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2952384d-d5bb808c-b29a39c1-48180d10_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:56.727703Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2952384d-d5bb808c-b29a39c1-48180d10_0] Write session will restart in 0.008521s
2025-10-09T21:49:56.727734Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2952384d-d5bb808c-b29a39c1-48180d10_0] Write session: Do CDS request
2025-10-09T21:49:56.727739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2952384d-d5bb808c-b29a39c1-48180d10_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:56.737395Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2952384d-d5bb808c-b29a39c1-48180d10_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:56.737496Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:56.737509Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1672
2025-10-09T21:49:56.737652Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1672 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:56.737682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1672 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:56.737692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1672 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:56.737699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:56.738176Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1672 sessionId:  describe result for acl check
2025-10-09T21:49:56.738211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:56.738217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:56.738219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:56.738224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342570505870749:7146] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:56.738765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342570505870749:7146] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:56.741548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342570505870749:7146] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:56.744135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342570505870749:7146] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:56.744148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342570505870749:7146] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:56.744151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342570505870749:7146] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:56.744159Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1672 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:56.744899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:56.748144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:56.748346Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:56.748357Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:56.748371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:56.748437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:56.748468Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342570144041450:3994]
2025-10-09T21:49:56.748875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:56.749140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:56.749206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:56.749475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:56.749512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.749517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.749517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.749518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.749519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.749519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.749520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.749521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.749521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.749535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.749542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.749545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.749547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.749549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.749551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.749553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.749555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.749557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.749563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.749568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.749572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.749575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.749578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.749590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.749592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.749595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.749597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.749600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.749604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.749609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.749613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.749618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.749625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.749634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.749636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.749639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.749655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:56.755823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:56.756886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:56.756897Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:56.756899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:56.756901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:56.756906Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1671 [2:7559342570144041450:3994]
2025-10-09T21:49:56.756915Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:56.756927Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:56.756929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:56.756931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:56.756932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:56.756945Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:56.756948Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:56.756955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:56.756995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.757000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.757001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.757003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:56.757068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:56.757073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.757080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.757086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.757096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:56.757110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342570144041445:3994]
2025-10-09T21:49:56.757115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342570144041445:3994]
2025-10-09T21:49:56.757116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342570144041445:3994]
2025-10-09T21:49:56.757122Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:56.757132Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:56.757133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342570505870666:2432], now have 1 active actors on pipe
2025-10-09T21:49:56.757136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342570505870924:7146], now have 1 active actors on pipe
2025-10-09T21:49:56.757140Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:56.757142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:56.757143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:56.757151Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:56.757153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:56.757175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342570144041445:3994]
2025-10-09T21:49:56.757183Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:56.757190Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:56.757206Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1671
2025-10-09T21:49:56.757213Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1671
2025-10-09T21:49:56.757277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:56.757286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:56.757311Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e6948425-cf17ada4-7b5a987a-f00316ec_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:56.757321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:56.757324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:56.757326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:56.757333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:56.757341Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:56.757350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:56.757488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:56.757494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:56.757504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:56.757601Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1672 partition: 0 MaxSeqNo: 25 sessionId: src_id|e6948425-cf17ada4-7b5a987a-f00316ec_0
2025-10-09T21:49:56.757930Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2952384d-d5bb808c-b29a39c1-48180d10_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e6948425-cf17ada4-7b5a987a-f00316ec_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:56.760274Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e6948425-cf17ada4-7b5a987a-f00316ec_0] Write session: try to update token
2025-10-09T21:49:56.760292Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e6948425-cf17ada4-7b5a987a-f00316ec_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:56.771204Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:56.771217Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:56.771219Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:56.771221Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:56.771230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342570144041445:3994]
2025-10-09T21:49:56.771235Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342570144041445:3994]
2025-10-09T21:49:56.771239Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342570144041445:3994]
2025-10-09T21:49:56.771281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:56.771286Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:56.771288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.771290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:56.771294Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:56.771298Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:56.771404Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:56.772470Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1672 sessionId: src_id|e6948425-cf17ada4-7b5a987a-f00316ec_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:56.774788Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:56.827608Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342570505870937:7774]
2025-10-09T21:49:56.827845Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 1671, pipe: [1:7559342570505870937:7774]
2025-10-09T21:49:56.920993Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:56.921735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:56.921749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:56.921786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342570505870937:7774], now have 1 active actors on pipe
2025-10-09T21:49:56.922019Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:56.922005Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:56.922061Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:56.922065Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342570505870937:7774]
2025-10-09T21:49:56.922065Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:56.922141Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1672 reason: error at write: Disconnected sessionId: src_id|e6948425-cf17ada4-7b5a987a-f00316ec_0
2025-10-09T21:49:56.922326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1672 sessionId: src_id|e6948425-cf17ada4-7b5a987a-f00316ec_0 is DEAD
2025-10-09T21:49:56.922482Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6948425-cf17ada4-7b5a987a-f00316ec_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:56.922494Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6948425-cf17ada4-7b5a987a-f00316ec_0] Write session will restart in 0.005619s
2025-10-09T21:49:56.922522Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6948425-cf17ada4-7b5a987a-f00316ec_0] Write session: Do CDS request
2025-10-09T21:49:56.922529Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6948425-cf17ada4-7b5a987a-f00316ec_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:56.929436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e6948425-cf17ada4-7b5a987a-f00316ec_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:56.929516Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:56.929536Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1673
2025-10-09T21:49:56.929666Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1673 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:56.929706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1673 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:56.929716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1673 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:56.929720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:56.930196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1673 sessionId:  describe result for acl check
2025-10-09T21:49:56.930228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:56.930235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:56.930236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:56.930242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342570505870979:7138] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:56.930787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342570505870979:7138] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:56.934016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342570505870979:7138] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:56.937208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342570505870979:7138] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:56.937224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342570505870979:7138] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:56.937228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342570505870979:7138] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:56.937234Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1673 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:56.942980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:56.943427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:56.943586Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:56.943594Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:56.943605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:56.943652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:56.943691Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342570144041701:3990]
2025-10-09T21:49:56.944272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:56.944587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:56.944637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:56.944946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:56.945000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.945006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.945007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.945008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.945009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.945010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.945011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.945012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.945013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.945018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.945029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.945033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.945036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.945039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.945043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.945047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.945050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.945053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.945060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.945067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:56.945073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.945078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:56.945082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.945087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:56.945094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.945099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:56.945107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.945112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:56.945121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.945126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:56.945136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.945142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.945150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.945155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.945161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.945171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.945199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:56.951629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:56.952778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:56.952792Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:56.952794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:56.952797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:56.952801Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1672 [2:7559342570144041701:3990]
2025-10-09T21:49:56.952815Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:56.952833Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:56.952836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:56.952839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:56.952841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:56.952859Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:56.952872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:56.952879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:56.952937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.952945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.952947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.952950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:56.953015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:56.953023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.953035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:56.953052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:56.953075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:56.953097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342570505870942:2432], now have 1 active actors on pipe
2025-10-09T21:49:56.953106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342570505871190:7138], now have 1 active actors on pipe
2025-10-09T21:49:56.953113Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342570144041696:3990]
2025-10-09T21:49:56.953120Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342570144041696:3990]
2025-10-09T21:49:56.953122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342570144041696:3990]
2025-10-09T21:49:56.953134Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:56.953147Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:56.953150Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:56.953152Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:56.953153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:56.953168Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:56.953172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:56.953139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1672
2025-10-09T21:49:56.953165Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1672
2025-10-09T21:49:56.953215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:56.953225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:56.953238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342570144041696:3990]
2025-10-09T21:49:56.953250Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:56.953253Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:56.953282Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|91f0b080-8e7bc814-97733c7b-bb3998b2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:56.953293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:56.953301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:56.953304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:56.953314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:56.953327Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:56.953358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:56.953507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:56.953517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:56.953548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:56.953658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1673 partition: 0 MaxSeqNo: 25 sessionId: src_id|91f0b080-8e7bc814-97733c7b-bb3998b2_0
2025-10-09T21:49:56.953984Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e6948425-cf17ada4-7b5a987a-f00316ec_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|91f0b080-8e7bc814-97733c7b-bb3998b2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:56.956498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91f0b080-8e7bc814-97733c7b-bb3998b2_0] Write session: try to update token
2025-10-09T21:49:56.956520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91f0b080-8e7bc814-97733c7b-bb3998b2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:56.967067Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:56.967084Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:56.967087Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:56.967090Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:56.967100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342570144041696:3990]
2025-10-09T21:49:56.967104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342570144041696:3990]
2025-10-09T21:49:56.967108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342570144041696:3990]
2025-10-09T21:49:56.967141Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:56.967147Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:56.967148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:56.967149Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:56.967155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:56.967158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:56.967279Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:56.969257Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1673 sessionId: src_id|91f0b080-8e7bc814-97733c7b-bb3998b2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:56.971502Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:57.022230Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342574800838507:7774]
2025-10-09T21:49:57.022475Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 1672, pipe: [1:7559342574800838507:7774]
2025-10-09T21:49:57.115294Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:57.115492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.115504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.115541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342574800838507:7774], now have 1 active actors on pipe
2025-10-09T21:49:57.115696Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:57.115746Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342574800838507:7774]
2025-10-09T21:49:57.115758Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:57.115783Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:57.115860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1673 reason: status is not ok: tablet will be restarted right now sessionId: src_id|91f0b080-8e7bc814-97733c7b-bb3998b2_0
2025-10-09T21:49:57.116059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1673 sessionId: src_id|91f0b080-8e7bc814-97733c7b-bb3998b2_0 is DEAD
2025-10-09T21:49:57.116231Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91f0b080-8e7bc814-97733c7b-bb3998b2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:57.116243Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91f0b080-8e7bc814-97733c7b-bb3998b2_0] Write session will restart in 0.006993s
2025-10-09T21:49:57.116272Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91f0b080-8e7bc814-97733c7b-bb3998b2_0] Write session: Do CDS request
2025-10-09T21:49:57.116279Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91f0b080-8e7bc814-97733c7b-bb3998b2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:57.124456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91f0b080-8e7bc814-97733c7b-bb3998b2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:57.124555Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:57.124568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1674
2025-10-09T21:49:57.124733Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1674 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:57.124775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1674 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:57.124781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1674 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:57.124785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:57.125246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1674 sessionId:  describe result for acl check
2025-10-09T21:49:57.125282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:57.125288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:57.125290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:57.125296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342574800838584:7124] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:57.125863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342574800838584:7124] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:57.128770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342574800838584:7124] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:57.131673Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342574800838584:7124] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:57.131686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342574800838584:7124] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:57.131690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342574800838584:7124] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:57.131698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1674 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:57.133686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:57.137142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:57.137322Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:57.137333Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:57.137345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:57.137400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:57.137441Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342574439009285:3951]
2025-10-09T21:49:57.138046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:57.138319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:57.138365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:57.138663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:57.138710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.138717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.138718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.138719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.138720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.138721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.138722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.138724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.138725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.138730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.138741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.138750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.138754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.138758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.138762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.138766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.138770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.138778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.138785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.138798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.138805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.138809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.138816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.138820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.138824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.138830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.138834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.138839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.138844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.138849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.138854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.138859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.138868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.138873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.138882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.138891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.138922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:57.145316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:57.146503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:57.146515Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:57.146517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:57.146521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:57.146527Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1673 [2:7559342574439009285:3951]
2025-10-09T21:49:57.146541Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:57.146557Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:57.146559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:57.146562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:57.146564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:57.146581Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:57.146589Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:57.146596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:57.146646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.146653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.146654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.146657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:57.146723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:57.146731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.146742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.146753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.146769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:57.146795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342574800838533:2432], now have 1 active actors on pipe
2025-10-09T21:49:57.146804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342574800838785:7124], now have 1 active actors on pipe
2025-10-09T21:49:57.146811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342574439009280:3951]
2025-10-09T21:49:57.146814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342574439009280:3951]
2025-10-09T21:49:57.146816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342574439009280:3951]
2025-10-09T21:49:57.146828Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:57.146838Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:57.146840Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:57.146840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:57.146842Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:57.146854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:57.146857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:57.146877Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342574439009280:3951]
2025-10-09T21:49:57.146891Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:57.146884Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1673
2025-10-09T21:49:57.146886Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1673
2025-10-09T21:49:57.146916Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:57.147004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.147014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.147041Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|125d5087-25af4752-77cb5ff4-59810d1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:57.147053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:57.147057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:57.147060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:57.147074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:57.147084Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:57.147097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:57.147266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.147275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.147288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:57.147392Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1674 partition: 0 MaxSeqNo: 25 sessionId: src_id|125d5087-25af4752-77cb5ff4-59810d1_0
2025-10-09T21:49:57.147700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91f0b080-8e7bc814-97733c7b-bb3998b2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|125d5087-25af4752-77cb5ff4-59810d1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:57.150198Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|125d5087-25af4752-77cb5ff4-59810d1_0] Write session: try to update token
2025-10-09T21:49:57.150218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|125d5087-25af4752-77cb5ff4-59810d1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:57.161992Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:57.162005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:57.162008Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:57.162010Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:57.162019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342574439009280:3951]
2025-10-09T21:49:57.162024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342574439009280:3951]
2025-10-09T21:49:57.162046Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342574439009280:3951]
2025-10-09T21:49:57.162097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:57.162097Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:57.162104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.162105Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:57.162107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:57.162109Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:57.162243Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:57.163566Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1674 sessionId: src_id|125d5087-25af4752-77cb5ff4-59810d1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:57.165975Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:57.215750Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342574800838803:7774]
2025-10-09T21:49:57.216015Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 1673, pipe: [1:7559342574800838803:7774]
2025-10-09T21:49:57.313467Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:57.314283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.314304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.314352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342574800838803:7774], now have 1 active actors on pipe
2025-10-09T21:49:57.314569Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:57.314588Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:57.314632Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:57.314636Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:57.314645Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342574800838803:7774]
2025-10-09T21:49:57.314707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1674 reason: error at write: Disconnected sessionId: src_id|125d5087-25af4752-77cb5ff4-59810d1_0
2025-10-09T21:49:57.314900Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1674 sessionId: src_id|125d5087-25af4752-77cb5ff4-59810d1_0 is DEAD
2025-10-09T21:49:57.315052Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|125d5087-25af4752-77cb5ff4-59810d1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:57.315065Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|125d5087-25af4752-77cb5ff4-59810d1_0] Write session will restart in 0.008352s
2025-10-09T21:49:57.315103Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|125d5087-25af4752-77cb5ff4-59810d1_0] Write session: Do CDS request
2025-10-09T21:49:57.315110Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|125d5087-25af4752-77cb5ff4-59810d1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:57.324574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|125d5087-25af4752-77cb5ff4-59810d1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:57.324607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:57.324623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1675
2025-10-09T21:49:57.324845Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1675 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:57.324889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1675 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:57.324900Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1675 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:57.324905Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:57.325456Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1675 sessionId:  describe result for acl check
2025-10-09T21:49:57.325497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:57.325553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:57.325563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:57.325570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342574800838879:7118] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:57.326245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342574800838879:7118] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:57.328611Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342574800838879:7118] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:57.331597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342574800838879:7118] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:57.331609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342574800838879:7118] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:57.331613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342574800838879:7118] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:57.331620Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1675 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:57.337063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:57.337557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:57.337754Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:57.337762Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:57.337771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:57.337813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:57.337843Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342574439009536:3955]
2025-10-09T21:49:57.338234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:57.338587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:57.338648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:57.338975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:57.339027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.339034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.339035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.339036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.339038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.339039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.339040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.339042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.339043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.339048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.339059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.339064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.339068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.339073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.339080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.339084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.339088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.339092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.339099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.339108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.339118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.339128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.339136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.339146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.339154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.339160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.339164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.339169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.339174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.339178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.339186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.339195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.339204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.339213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.339218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.339223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.339252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:57.345833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:57.347122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:57.347135Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:57.347137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:57.347140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:57.347147Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1674 [2:7559342574439009536:3955]
2025-10-09T21:49:57.347161Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:57.347180Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:57.347183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:57.347186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:57.347189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:57.347205Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:57.347216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:57.347223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:57.347275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.347281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.347282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.347284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:57.347349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:57.347357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.347369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.347382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.347404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:57.347430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342574800838810:2432], now have 1 active actors on pipe
2025-10-09T21:49:57.347438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342574800839066:7118], now have 1 active actors on pipe
2025-10-09T21:49:57.347445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342574439009531:3955]
2025-10-09T21:49:57.347449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342574439009531:3955]
2025-10-09T21:49:57.347451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342574439009531:3955]
2025-10-09T21:49:57.347464Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:57.347483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:57.347489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:57.347493Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:57.347495Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:57.347497Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:57.347500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:57.347511Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1674
2025-10-09T21:49:57.347515Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1674
2025-10-09T21:49:57.347530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342574439009531:3955]
2025-10-09T21:49:57.347538Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:57.347555Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:57.347581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.347585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.347617Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|86ec2d0e-2b9ee0d6-ee59e314-53d53e54_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:57.347635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:57.347639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:57.347642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:57.347653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:57.347663Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:57.347681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:57.347818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.347826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.347839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:57.347952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1675 partition: 0 MaxSeqNo: 25 sessionId: src_id|86ec2d0e-2b9ee0d6-ee59e314-53d53e54_0
2025-10-09T21:49:57.348300Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|125d5087-25af4752-77cb5ff4-59810d1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|86ec2d0e-2b9ee0d6-ee59e314-53d53e54_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:57.350859Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86ec2d0e-2b9ee0d6-ee59e314-53d53e54_0] Write session: try to update token
2025-10-09T21:49:57.350876Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86ec2d0e-2b9ee0d6-ee59e314-53d53e54_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:57.362331Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:57.362346Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:57.362349Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:57.362351Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:57.362360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342574439009531:3955]
2025-10-09T21:49:57.362366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342574439009531:3955]
2025-10-09T21:49:57.362370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342574439009531:3955]
2025-10-09T21:49:57.362414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:57.362422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.362425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:57.362429Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:57.362437Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:57.362439Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:57.362565Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:57.363695Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1675 sessionId: src_id|86ec2d0e-2b9ee0d6-ee59e314-53d53e54_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:57.366076Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:57.414905Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342574800839081:7774]
2025-10-09T21:49:57.415127Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 1674, pipe: [1:7559342574800839081:7774]
2025-10-09T21:49:57.510737Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:57.511396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.511409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.511448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342574800839081:7774], now have 1 active actors on pipe
2025-10-09T21:49:57.511635Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:57.511647Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:57.511674Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:57.511675Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:57.511717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1675 reason: error at write: Disconnected sessionId: src_id|86ec2d0e-2b9ee0d6-ee59e314-53d53e54_0
2025-10-09T21:49:57.511721Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342574800839081:7774]
2025-10-09T21:49:57.511901Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1675 sessionId: src_id|86ec2d0e-2b9ee0d6-ee59e314-53d53e54_0 is DEAD
2025-10-09T21:49:57.512103Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86ec2d0e-2b9ee0d6-ee59e314-53d53e54_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:57.512113Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86ec2d0e-2b9ee0d6-ee59e314-53d53e54_0] Write session will restart in 0.005768s
2025-10-09T21:49:57.512133Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86ec2d0e-2b9ee0d6-ee59e314-53d53e54_0] Write session: Do CDS request
2025-10-09T21:49:57.512138Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86ec2d0e-2b9ee0d6-ee59e314-53d53e54_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:57.519432Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86ec2d0e-2b9ee0d6-ee59e314-53d53e54_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:57.519541Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:57.519553Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1676
2025-10-09T21:49:57.519668Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1676 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:57.519707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1676 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:57.519719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1676 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:57.519723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:57.520112Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1676 sessionId:  describe result for acl check
2025-10-09T21:49:57.520137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:57.520143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:57.520144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:57.520149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342574800839121:7109] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:57.520689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342574800839121:7109] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:57.523434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342574800839121:7109] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:57.526420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342574800839121:7109] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:57.526433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342574800839121:7109] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:57.526437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342574800839121:7109] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:57.526444Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1676 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:57.532018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:57.532470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:57.532633Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:57.532643Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:57.532654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:57.532703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:57.532741Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342574439009788:3952]
2025-10-09T21:49:57.533282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:57.533553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:57.533594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:57.533886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:57.533926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.533932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.533933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.533934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.533935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.533936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.533937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.533938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.533939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.533943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.533954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.533958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.533962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.533965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.533969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.533973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.533976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.533980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.533986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.533993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.533999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.534005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.534009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.534013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.534016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.534021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.534025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.534043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.534050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.534055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.534063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.534068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.534076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.534081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.534085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.534094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.534122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:57.539783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:57.540858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:57.540873Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:57.540875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:57.540879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:57.540886Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1675 [2:7559342574439009788:3952]
2025-10-09T21:49:57.540899Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:57.540913Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:57.540920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:57.540924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:57.540926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:57.540940Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:57.540946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:57.540953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:57.541001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.541010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.541012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.541014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:57.541198Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1675
2025-10-09T21:49:57.541197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1675
2025-10-09T21:49:57.541367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:57.541376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.541388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.541399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.541415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:57.541435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342574800839317:7109], now have 1 active actors on pipe
2025-10-09T21:49:57.541441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342574800839084:2432], now have 1 active actors on pipe
2025-10-09T21:49:57.541454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.541462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.541475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342574439009783:3952]
2025-10-09T21:49:57.541476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342574439009783:3952]
2025-10-09T21:49:57.541478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342574439009783:3952]
2025-10-09T21:49:57.541488Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:57.541498Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:57.541502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:57.541503Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:57.541505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:57.541511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:57.541513Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:57.541538Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|af2f1697-c6375362-f8366d49-a82f277c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:57.541548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:57.541551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:57.541552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:57.541559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:57.541567Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:57.541591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342574439009783:3952]
2025-10-09T21:49:57.541602Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:57.541605Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:57.541617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:57.541759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.541766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.541777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:57.541858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1676 partition: 0 MaxSeqNo: 25 sessionId: src_id|af2f1697-c6375362-f8366d49-a82f277c_0
2025-10-09T21:49:57.542174Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86ec2d0e-2b9ee0d6-ee59e314-53d53e54_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|af2f1697-c6375362-f8366d49-a82f277c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:57.544609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af2f1697-c6375362-f8366d49-a82f277c_0] Write session: try to update token
2025-10-09T21:49:57.544625Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af2f1697-c6375362-f8366d49-a82f277c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:57.557430Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1676 sessionId: src_id|af2f1697-c6375362-f8366d49-a82f277c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:57.557513Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:57.557536Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:57.557543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342574439009783:3952]
2025-10-09T21:49:57.557585Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:57.557590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:57.557759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:57.557890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:57.558015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:57.558154Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:57.558330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:57.558369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:57.559634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:57.559795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.559807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.559838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:57.559931Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:57.565222Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:57.565236Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:57.565239Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:57.565241Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:57.565250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342574439009783:3952]
2025-10-09T21:49:57.565254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342574439009783:3952]
2025-10-09T21:49:57.565258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342574439009783:3952]
2025-10-09T21:49:57.565295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:57.565298Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:57.565304Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:57.565304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.565307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:57.565308Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:57.565430Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:57.612032Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342574800839355:7774]
2025-10-09T21:49:57.612301Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 1675, pipe: [1:7559342574800839355:7774]
2025-10-09T21:49:57.706296Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:57.712909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.712932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.713066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.713117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.713171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.713227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.713320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.713330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.713387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.713438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.713493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.713565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.713608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.713616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.713680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.713745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.713799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.713848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.713886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.713891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.713947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.714000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.714067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.714119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.714156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.714161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.714216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.714266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.714319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.714371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.714407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.714412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.714465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.714520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.714573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.714623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.714662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.714669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.714722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.714769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.714820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.714870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.714907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.714912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.714960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.715016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.715078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.715130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.715233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.715239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.715291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.715340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.715390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.715444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.715479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.715484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.715536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.715586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.715637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.715688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.715730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.715738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.715783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342574800839355:7774], now have 1 active actors on pipe
2025-10-09T21:49:57.716004Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342574800839355:7774]
2025-10-09T21:49:57.716033Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:57.716037Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:57.716040Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:57.716066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1676 reason: error at write: Disconnected sessionId: src_id|af2f1697-c6375362-f8366d49-a82f277c_0
2025-10-09T21:49:57.716238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1676 sessionId: src_id|af2f1697-c6375362-f8366d49-a82f277c_0 is DEAD
2025-10-09T21:49:57.716383Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af2f1697-c6375362-f8366d49-a82f277c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:57.716393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af2f1697-c6375362-f8366d49-a82f277c_0] Write session will restart in 0.007527s
2025-10-09T21:49:57.716418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af2f1697-c6375362-f8366d49-a82f277c_0] Write session: Do CDS request
2025-10-09T21:49:57.716423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af2f1697-c6375362-f8366d49-a82f277c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:57.716488Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:57.725507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af2f1697-c6375362-f8366d49-a82f277c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:57.725540Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:57.725553Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1677
2025-10-09T21:49:57.725732Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1677 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:57.725757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1677 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:57.725760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1677 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:57.725763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:57.726171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1677 sessionId:  describe result for acl check
2025-10-09T21:49:57.726199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:57.726202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:57.726204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:57.726208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342574800839427:7101] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:57.726693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342574800839427:7101] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:57.729408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342574800839427:7101] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:57.732442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342574800839427:7101] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:57.732456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342574800839427:7101] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:57.732460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342574800839427:7101] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:57.732469Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1677 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:57.736867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:57.737308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:57.737463Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:57.737469Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:57.737487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:57.737545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:57.737577Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342574439010045:3942]
2025-10-09T21:49:57.737946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:57.738322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:57.738375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:57.738687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:57.738723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.738727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.738728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.738729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.738730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.738730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.738731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.738732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.738732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.738736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.738743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.738745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.738747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.738749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.738751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.738753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.738755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.738757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.738762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.738768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.738772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.738775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.738778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.738781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.738785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.738790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.738795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.738805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.738813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.738818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.738827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.738836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.738840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.738851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.738855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.738860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.738885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:57.744935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:57.746348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:57.746357Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:57.746359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:57.746362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:57.746367Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1676 [2:7559342574439010045:3942]
2025-10-09T21:49:57.746377Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:57.746391Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:57.746393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:57.746395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:57.746396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:57.746409Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:57.746413Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:57.746418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:57.746460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.746465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.746467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.746469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:57.746522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:57.746530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.746541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.746549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.746559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:57.746573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342574439010040:3942]
2025-10-09T21:49:57.746575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342574439010040:3942]
2025-10-09T21:49:57.746576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342574439010040:3942]
2025-10-09T21:49:57.746582Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:57.746593Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:57.746604Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:57.746611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:57.746613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:57.746616Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:57.746618Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:57.746640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342574800839616:7101], now have 1 active actors on pipe
2025-10-09T21:49:57.746647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342574800839358:2432], now have 1 active actors on pipe
2025-10-09T21:49:57.746648Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342574439010040:3942]
2025-10-09T21:49:57.746654Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:57.746681Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:57.746721Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1676
2025-10-09T21:49:57.746722Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1676
2025-10-09T21:49:57.746840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.746866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.746906Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dc579058-7fa7bb95-d4282dc0-5e16e742_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:57.746919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:57.746929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:57.746932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:57.746943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:57.746958Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:57.746992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:57.747185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.747194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.747222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:57.747306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1677 partition: 0 MaxSeqNo: 25 sessionId: src_id|dc579058-7fa7bb95-d4282dc0-5e16e742_0
2025-10-09T21:49:57.747694Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af2f1697-c6375362-f8366d49-a82f277c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dc579058-7fa7bb95-d4282dc0-5e16e742_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:57.750255Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dc579058-7fa7bb95-d4282dc0-5e16e742_0] Write session: try to update token
2025-10-09T21:49:57.750275Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dc579058-7fa7bb95-d4282dc0-5e16e742_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:57.757740Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:49:57.757752Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:57.757760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342574439010040:3942]
2025-10-09T21:49:57.757799Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:49:57.757817Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:49:57.757952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:49:57.758098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:57.758211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:57.758324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:49:57.758480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:49:57.758511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:49:57.762717Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1677 sessionId: src_id|dc579058-7fa7bb95-d4282dc0-5e16e742_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:57.764927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:57.765127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.765145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.765178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:49:57.765251Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:57.770533Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:57.770546Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:57.770548Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:57.770549Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:57.770557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342574439010040:3942]
2025-10-09T21:49:57.770562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342574439010040:3942]
2025-10-09T21:49:57.770565Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342574439010040:3942]
2025-10-09T21:49:57.770596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:57.770603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.770604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:57.770614Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:57.770620Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:57.770624Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:57.770724Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:57.816319Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342574800839630:7774]
2025-10-09T21:49:57.816553Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 1676, pipe: [1:7559342574800839630:7774]
2025-10-09T21:49:57.910174Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:57.916739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.916762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.916905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.916961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.917020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.917078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.917119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.917127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.917188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.917247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.917319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.917385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.917428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.917435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.917495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.917564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.917617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.917666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.917700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.917708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.917767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.917821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.917876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.917932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.917974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.917980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.918055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.918115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.918186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.918251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.918307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.918317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.918387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.918446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.918500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.918555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.918595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.918601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.918655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.918707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.918760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.918814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.918851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.918857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.918912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.918963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.919016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.919069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.919106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.919112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.919164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.919213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.919261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.919317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.919361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.919367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.919447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:49:57.919497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:49:57.919548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:49:57.919600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:49:57.919640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:49:57.919645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:49:57.919684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342574800839630:7774], now have 1 active actors on pipe
2025-10-09T21:49:57.919950Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342574800839630:7774]
2025-10-09T21:49:57.919978Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:57.919988Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:57.919992Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:57.920021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1677 reason: error at write: Disconnected sessionId: src_id|dc579058-7fa7bb95-d4282dc0-5e16e742_0
2025-10-09T21:49:57.920205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1677 sessionId: src_id|dc579058-7fa7bb95-d4282dc0-5e16e742_0 is DEAD
2025-10-09T21:49:57.920382Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc579058-7fa7bb95-d4282dc0-5e16e742_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:57.920399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc579058-7fa7bb95-d4282dc0-5e16e742_0] Write session will restart in 0.006186s
2025-10-09T21:49:57.920424Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc579058-7fa7bb95-d4282dc0-5e16e742_0] Write session: Do CDS request
2025-10-09T21:49:57.920431Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc579058-7fa7bb95-d4282dc0-5e16e742_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:57.920665Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:57.927460Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dc579058-7fa7bb95-d4282dc0-5e16e742_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:57.927544Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:57.927566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1678
2025-10-09T21:49:57.927672Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1678 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:57.927697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1678 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:57.927706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1678 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:57.927710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:57.928170Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1678 sessionId:  describe result for acl check
2025-10-09T21:49:57.928206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:57.928213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:57.928215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:57.928221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342574800839669:7091] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:57.928788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342574800839669:7091] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:57.931947Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342574800839669:7091] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:57.935299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342574800839669:7091] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:57.935315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342574800839669:7091] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:57.935318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342574800839669:7091] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:57.935334Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1678 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:57.940402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:57.940966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:57.941133Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:57.941144Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:57.941155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:57.941201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:57.941237Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342574439010300:3939]
2025-10-09T21:49:57.941760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:57.942060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:57.942100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:57.942381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:57.942428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.942434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.942435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.942436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.942437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.942438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.942439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.942440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.942441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.942446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.942455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.942458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.942462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.942465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.942468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.942472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.942475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.942478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.942484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.942491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:57.942498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.942508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:57.942511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.942518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:57.942521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.942525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:57.942529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.942533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:57.942541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.942546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:57.942554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.942558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.942565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.942569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.942577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.942582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.942610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:57.948838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:57.949833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:57.949849Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:57.949851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:57.949855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:57.949861Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1677 [2:7559342574439010300:3939]
2025-10-09T21:49:57.949874Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:57.949891Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:57.949899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:57.949902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:57.949905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:57.949921Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:57.949933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:57.949945Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:57.949999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.950006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.950009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.950012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:57.950095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:57.950102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.950115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:57.950133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:57.950158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:57.950186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342574800839894:7091], now have 1 active actors on pipe
2025-10-09T21:49:57.950190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342574800839635:2432], now have 1 active actors on pipe
2025-10-09T21:49:57.950197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342574439010295:3939]
2025-10-09T21:49:57.950200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342574439010295:3939]
2025-10-09T21:49:57.950202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342574439010295:3939]
2025-10-09T21:49:57.950216Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:57.950230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:57.950243Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:57.950247Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:57.950249Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:57.950252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:57.950255Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:57.950246Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1677
2025-10-09T21:49:57.950254Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1677
2025-10-09T21:49:57.950279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342574439010295:3939]
2025-10-09T21:49:57.950290Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:57.950291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:57.950314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.950320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.950357Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f5a8aff3-90cd0635-8313a0f2-ffd2a93f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:57.950370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:57.950374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:57.950378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:57.950387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:57.950401Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:57.950421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:57.950669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:57.950676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:57.950692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:57.950823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1678 partition: 0 MaxSeqNo: 25 sessionId: src_id|f5a8aff3-90cd0635-8313a0f2-ffd2a93f_0
2025-10-09T21:49:57.951226Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc579058-7fa7bb95-d4282dc0-5e16e742_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f5a8aff3-90cd0635-8313a0f2-ffd2a93f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:57.953774Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5a8aff3-90cd0635-8313a0f2-ffd2a93f_0] Write session: try to update token
2025-10-09T21:49:57.953792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5a8aff3-90cd0635-8313a0f2-ffd2a93f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:57.964730Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:57.964747Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:57.964749Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:57.964751Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:57.964760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342574439010295:3939]
2025-10-09T21:49:57.964765Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342574439010295:3939]
2025-10-09T21:49:57.964769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342574439010295:3939]
2025-10-09T21:49:57.964809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:57.964821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:57.964821Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:57.964824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:57.964826Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:57.964829Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:57.964953Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:57.966700Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1678 sessionId: src_id|f5a8aff3-90cd0635-8313a0f2-ffd2a93f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:57.969061Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:58.020101Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342579095807205:7774]
2025-10-09T21:49:58.020411Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 1677, pipe: [1:7559342579095807205:7774]
2025-10-09T21:49:58.114349Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:58.115283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:58.115296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:58.115331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342579095807205:7774], now have 1 active actors on pipe
2025-10-09T21:49:58.115441Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:58.115471Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342579095807205:7774]
2025-10-09T21:49:58.115507Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:58.115735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1678 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f5a8aff3-90cd0635-8313a0f2-ffd2a93f_0
2025-10-09T21:49:58.115912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1678 sessionId: src_id|f5a8aff3-90cd0635-8313a0f2-ffd2a93f_0 is DEAD
2025-10-09T21:49:58.116066Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5a8aff3-90cd0635-8313a0f2-ffd2a93f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:58.116079Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5a8aff3-90cd0635-8313a0f2-ffd2a93f_0] Write session will restart in 0.005237s
2025-10-09T21:49:58.116084Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:58.116102Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5a8aff3-90cd0635-8313a0f2-ffd2a93f_0] Write session: Do CDS request
2025-10-09T21:49:58.116109Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5a8aff3-90cd0635-8313a0f2-ffd2a93f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:58.122536Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5a8aff3-90cd0635-8313a0f2-ffd2a93f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:58.123351Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:58.123366Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1679
2025-10-09T21:49:58.123510Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1679 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:58.123541Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1679 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:58.123551Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1679 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:58.123555Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:58.123930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1679 sessionId:  describe result for acl check
2025-10-09T21:49:58.123964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:58.123966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:58.123967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:58.123972Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342579095807263:7074] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:58.124480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342579095807263:7074] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:58.127310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342579095807263:7074] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:58.130156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342579095807263:7074] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:58.130170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342579095807263:7074] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:58.130173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342579095807263:7074] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:58.130185Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1679 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:58.136606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:58.137102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:58.137250Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:58.137257Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:58.137266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:58.137305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:58.137336Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342578733977888:3932]
2025-10-09T21:49:58.137749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:58.138085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:58.138135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:58.138471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:58.138514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.138520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.138521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.138522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.138523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.138525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.138526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.138527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.138528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.138533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.138543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.138547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.138552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.138556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.138560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.138564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.138568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.138572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.138578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.138587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.138593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.138598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.138607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.138612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.138617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.138622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.138626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.138632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.138640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.138646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.138650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.138661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.138666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.138671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.138676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.138681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.138710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:58.145045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:58.146145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:58.146161Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:58.146163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:58.146168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:58.146176Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1678 [2:7559342578733977888:3932]
2025-10-09T21:49:58.146191Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:58.146211Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:58.146219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:58.146222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:58.146224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:58.146244Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:58.146256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:58.146265Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:58.146328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.146337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.146340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.146343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:58.146417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:58.146425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.146440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.146452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.146473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:58.146502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342579095807460:7074], now have 1 active actors on pipe
2025-10-09T21:49:58.146505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342579095807220:2432], now have 1 active actors on pipe
2025-10-09T21:49:58.146513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342578733977880:3932]
2025-10-09T21:49:58.146516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342578733977880:3932]
2025-10-09T21:49:58.146518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342578733977880:3932]
2025-10-09T21:49:58.146532Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:58.146546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:58.146559Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:58.146561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:58.146564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:58.146569Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:58.146571Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:58.146560Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1678
2025-10-09T21:49:58.146602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342578733977880:3932]
2025-10-09T21:49:58.146610Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:58.146611Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:58.146638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:58.146649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:58.146685Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6f637c0f-265480f9-346997d8-1a4c73fc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:58.146698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:58.146702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:58.146705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:58.146717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:58.146729Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:58.146755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:58.146820Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1678
2025-10-09T21:49:58.147050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:58.147062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:58.147079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:58.147200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1679 partition: 0 MaxSeqNo: 25 sessionId: src_id|6f637c0f-265480f9-346997d8-1a4c73fc_0
2025-10-09T21:49:58.147590Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5a8aff3-90cd0635-8313a0f2-ffd2a93f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6f637c0f-265480f9-346997d8-1a4c73fc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:58.150215Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6f637c0f-265480f9-346997d8-1a4c73fc_0] Write session: try to update token
2025-10-09T21:49:58.150237Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6f637c0f-265480f9-346997d8-1a4c73fc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:58.161082Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:58.161099Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:58.161101Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:58.161103Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:58.161113Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342578733977880:3932]
2025-10-09T21:49:58.161118Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342578733977880:3932]
2025-10-09T21:49:58.161122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342578733977880:3932]
2025-10-09T21:49:58.161168Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:58.161170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:58.161177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.161178Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:58.161180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:58.161182Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:58.161318Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:58.164846Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1679 sessionId: src_id|6f637c0f-265480f9-346997d8-1a4c73fc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:58.167281Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:58.215645Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342579095807506:7774]
2025-10-09T21:49:58.215894Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 1678, pipe: [1:7559342579095807506:7774]
2025-10-09T21:49:58.313919Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:58.314227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:58.314245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:58.314293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342579095807506:7774], now have 1 active actors on pipe
2025-10-09T21:49:58.314538Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:58.314690Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:58.314734Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342579095807506:7774]
2025-10-09T21:49:58.314771Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:58.314778Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:58.314857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1679 reason: error at write: Disconnected sessionId: src_id|6f637c0f-265480f9-346997d8-1a4c73fc_0
2025-10-09T21:49:58.315052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1679 sessionId: src_id|6f637c0f-265480f9-346997d8-1a4c73fc_0 is DEAD
2025-10-09T21:49:58.315177Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6f637c0f-265480f9-346997d8-1a4c73fc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:58.315199Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6f637c0f-265480f9-346997d8-1a4c73fc_0] Write session will restart in 0.008717s
2025-10-09T21:49:58.315227Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6f637c0f-265480f9-346997d8-1a4c73fc_0] Write session: Do CDS request
2025-10-09T21:49:58.315234Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6f637c0f-265480f9-346997d8-1a4c73fc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:58.325549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6f637c0f-265480f9-346997d8-1a4c73fc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:58.325663Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:58.325681Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1680
2025-10-09T21:49:58.325869Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1680 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:58.325902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1680 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:58.325914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1680 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:58.325919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:58.326463Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1680 sessionId:  describe result for acl check
2025-10-09T21:49:58.326501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:58.326503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:58.326504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:58.326510Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342579095807594:7061] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:58.327105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342579095807594:7061] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:58.329963Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342579095807594:7061] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:58.333400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342579095807594:7061] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:58.333417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342579095807594:7061] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:58.333422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342579095807594:7061] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:58.333504Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1680 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:58.334952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:58.338515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:58.338674Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:58.338683Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:58.338691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:58.338729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:58.338759Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342578733978145:3919]
2025-10-09T21:49:58.339133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:58.339432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:58.339465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:58.339674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:58.339705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.339711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.339711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.339712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.339713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.339713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.339714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.339715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.339715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.339718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.339725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.339728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.339730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.339732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.339735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.339737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.339739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.339741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.339745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.339751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.339757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.339760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.339762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.339765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.339768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.339771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.339774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.339777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.339780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.339783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.339786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.339789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.339791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.339795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.339797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.339800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.339817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:58.346117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:58.347601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:58.347618Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:58.347620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:58.347624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:58.347630Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1679 [2:7559342578733978145:3919]
2025-10-09T21:49:58.347646Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:58.347664Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:58.347667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:58.347670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:58.347673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:58.347688Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:58.347694Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:58.347703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:58.347756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.347764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.347766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.347768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:58.347833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:58.347841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.347853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.347866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.347883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:58.347908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342579095807513:2432], now have 1 active actors on pipe
2025-10-09T21:49:58.347916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342579095807774:7061], now have 1 active actors on pipe
2025-10-09T21:49:58.347923Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342578733978140:3919]
2025-10-09T21:49:58.347926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342578733978140:3919]
2025-10-09T21:49:58.347929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342578733978140:3919]
2025-10-09T21:49:58.347941Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:58.347959Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:58.347967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:58.347970Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:58.347972Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:58.347977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:58.347980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:58.348005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342578733978140:3919]
2025-10-09T21:49:58.348011Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:58.347989Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1679
2025-10-09T21:49:58.348020Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:58.348035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:58.348038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:58.348011Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1679
2025-10-09T21:49:58.348063Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1428131e-828b03f2-b6faff4-4c8f0e89_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:58.348072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:58.348076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:58.348077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:58.348086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:58.348094Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:58.348110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:58.348283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:58.348291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:58.348302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:58.348418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1680 partition: 0 MaxSeqNo: 25 sessionId: src_id|1428131e-828b03f2-b6faff4-4c8f0e89_0
2025-10-09T21:49:58.348756Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6f637c0f-265480f9-346997d8-1a4c73fc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1428131e-828b03f2-b6faff4-4c8f0e89_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:58.351027Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1428131e-828b03f2-b6faff4-4c8f0e89_0] Write session: try to update token
2025-10-09T21:49:58.351043Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1428131e-828b03f2-b6faff4-4c8f0e89_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:58.361612Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:58.361631Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:58.361633Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:58.361635Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:58.361644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342578733978140:3919]
2025-10-09T21:49:58.361649Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342578733978140:3919]
2025-10-09T21:49:58.361653Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342578733978140:3919]
2025-10-09T21:49:58.361695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:58.361697Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:58.361702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.361705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:58.361706Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:58.361708Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:58.361836Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:58.362984Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1680 sessionId: src_id|1428131e-828b03f2-b6faff4-4c8f0e89_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:58.365176Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:58.414793Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342579095807789:7774]
2025-10-09T21:49:58.415049Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 1679, pipe: [1:7559342579095807789:7774]
2025-10-09T21:49:58.509994Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:58.510220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:58.510232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:58.510267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342579095807789:7774], now have 1 active actors on pipe
2025-10-09T21:49:58.510460Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342579095807789:7774]
2025-10-09T21:49:58.510469Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:58.510490Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:58.510492Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:58.510553Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1680 reason: error at write: Disconnected sessionId: src_id|1428131e-828b03f2-b6faff4-4c8f0e89_0
2025-10-09T21:49:58.510748Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:58.510737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1680 sessionId: src_id|1428131e-828b03f2-b6faff4-4c8f0e89_0 is DEAD
2025-10-09T21:49:58.510893Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1428131e-828b03f2-b6faff4-4c8f0e89_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:58.510905Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1428131e-828b03f2-b6faff4-4c8f0e89_0] Write session will restart in 0.007437s
2025-10-09T21:49:58.510933Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1428131e-828b03f2-b6faff4-4c8f0e89_0] Write session: Do CDS request
2025-10-09T21:49:58.510940Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1428131e-828b03f2-b6faff4-4c8f0e89_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:58.519445Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1428131e-828b03f2-b6faff4-4c8f0e89_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:58.519544Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:58.519561Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1681
2025-10-09T21:49:58.519677Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1681 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:58.519712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1681 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:58.519719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1681 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:58.519723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:58.520184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1681 sessionId:  describe result for acl check
2025-10-09T21:49:58.520222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:58.520229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:58.520231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:58.520237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342579095807854:7065] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:58.520754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342579095807854:7065] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:58.523335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342579095807854:7065] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:58.526161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342579095807854:7065] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:58.526177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342579095807854:7065] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:58.526180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342579095807854:7065] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:58.526203Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1681 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:58.527863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:58.530960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:58.531118Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:58.531129Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:58.531142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:58.531200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:58.531230Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342578733978402:3917]
2025-10-09T21:49:58.531648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:58.531910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:58.531939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:58.532138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:58.532167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.532172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.532173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.532174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.532175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.532175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.532176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.532176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.532177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.532180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.532187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.532189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.532191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.532193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.532195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.532197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.532199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.532201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.532206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.532210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.532215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.532218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.532221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.532224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.532226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.532229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.532232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.532235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.532237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.532240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.532243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.532246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.532248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.532251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.532254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.532257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.532271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:58.538151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:58.539419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:58.539432Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:58.539434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:58.539437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:58.539443Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1680 [2:7559342578733978402:3917]
2025-10-09T21:49:58.539455Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:58.539470Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:58.539473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:58.539476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:58.539478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:58.539493Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:58.539499Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:58.539506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:58.539556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.539563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.539565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.539568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:58.539637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:58.539644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.539654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.539663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.539680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:58.539704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342579095808054:7065], now have 1 active actors on pipe
2025-10-09T21:49:58.539710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342579095807792:2432], now have 1 active actors on pipe
2025-10-09T21:49:58.539720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342578733978397:3917]
2025-10-09T21:49:58.539722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342578733978397:3917]
2025-10-09T21:49:58.539724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342578733978397:3917]
2025-10-09T21:49:58.539743Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1680
2025-10-09T21:49:58.539757Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1680
2025-10-09T21:49:58.539736Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:58.539753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:58.539756Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:58.539756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:58.539758Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:58.539770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:58.539773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:58.539804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:58.539808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:58.539821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342578733978397:3917]
2025-10-09T21:49:58.539828Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:58.539858Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:58.539859Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fabfc2d9-ee983d97-c975656d-e86a476e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:58.539867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:58.539871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:58.539874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:58.539883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:58.539893Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:58.539921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:58.540063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:58.540071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:58.540084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:58.540150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1681 partition: 0 MaxSeqNo: 25 sessionId: src_id|fabfc2d9-ee983d97-c975656d-e86a476e_0
2025-10-09T21:49:58.540475Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1428131e-828b03f2-b6faff4-4c8f0e89_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fabfc2d9-ee983d97-c975656d-e86a476e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:58.542942Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fabfc2d9-ee983d97-c975656d-e86a476e_0] Write session: try to update token
2025-10-09T21:49:58.542961Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fabfc2d9-ee983d97-c975656d-e86a476e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:58.554634Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:58.554649Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:58.554651Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:58.554654Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:58.554663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342578733978397:3917]
2025-10-09T21:49:58.554667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342578733978397:3917]
2025-10-09T21:49:58.554671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342578733978397:3917]
2025-10-09T21:49:58.554711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:58.554719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.554722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:58.554722Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:58.554730Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:58.554734Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:58.554847Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:58.555890Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1681 sessionId: src_id|fabfc2d9-ee983d97-c975656d-e86a476e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:58.558199Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:58.610387Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342579095808068:7774]
2025-10-09T21:49:58.610631Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 1680, pipe: [1:7559342579095808068:7774]
2025-10-09T21:49:58.699322Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:58.699562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:58.699574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:58.699613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342579095808068:7774], now have 1 active actors on pipe
2025-10-09T21:49:58.699770Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:58.699834Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342579095808068:7774]
2025-10-09T21:49:58.699852Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:58.699869Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:49:58.699930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1681 reason: status is not ok: tablet will be restarted right now sessionId: src_id|fabfc2d9-ee983d97-c975656d-e86a476e_0
2025-10-09T21:49:58.700084Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1681 sessionId: src_id|fabfc2d9-ee983d97-c975656d-e86a476e_0 is DEAD
2025-10-09T21:49:58.700250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fabfc2d9-ee983d97-c975656d-e86a476e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:49:58.700262Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fabfc2d9-ee983d97-c975656d-e86a476e_0] Write session will restart in 0.007265s
2025-10-09T21:49:58.700291Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fabfc2d9-ee983d97-c975656d-e86a476e_0] Write session: Do CDS request
2025-10-09T21:49:58.700298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fabfc2d9-ee983d97-c975656d-e86a476e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:58.708522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fabfc2d9-ee983d97-c975656d-e86a476e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:58.708585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:58.708598Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1682
2025-10-09T21:49:58.708705Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1682 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:58.708730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1682 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:58.708734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1682 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:58.708738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:58.709179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1682 sessionId:  describe result for acl check
2025-10-09T21:49:58.709209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:58.709213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:58.709215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:58.709220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342579095808120:7059] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:58.709745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342579095808120:7059] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:58.712729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342579095808120:7059] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:58.715559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342579095808120:7059] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:58.715574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342579095808120:7059] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:58.715576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342579095808120:7059] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:58.715587Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1682 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:58.717873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:58.721114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:58.721293Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:58.721303Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:58.721314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:58.721360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:58.721398Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342578733978660:3915]
2025-10-09T21:49:58.721870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:58.722198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:58.722238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:58.722515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:58.722558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.722562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.722563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.722564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.722564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.722565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.722566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.722566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.722567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.722570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.722577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.722579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.722581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.722583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.722586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.722588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.722590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.722592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.722597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.722602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.722606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.722609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.722612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.722616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.722618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.722621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.722624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.722626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.722629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.722632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.722635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.722638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.722640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.722644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.722646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.722649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.722668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:58.728930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:58.730255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:58.730266Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:58.730267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:58.730270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:58.730275Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1681 [2:7559342578733978660:3915]
2025-10-09T21:49:58.730284Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:58.730297Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:58.730299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:58.730301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:58.730302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:58.730314Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:58.730321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:58.730327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:58.730377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.730382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.730383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.730385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:58.730437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:58.730442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.730450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.730459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.730475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:58.730499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342579095808071:2432], now have 1 active actors on pipe
2025-10-09T21:49:58.730507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342579095808319:7059], now have 1 active actors on pipe
2025-10-09T21:49:58.730512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342578733978655:3915]
2025-10-09T21:49:58.730519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342578733978655:3915]
2025-10-09T21:49:58.730521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342578733978655:3915]
2025-10-09T21:49:58.730533Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:58.730547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:58.730550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:58.730557Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:58.730559Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:58.730563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:58.730566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:58.730590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342578733978655:3915]
2025-10-09T21:49:58.730598Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:58.730595Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1681
2025-10-09T21:49:58.730601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1681
2025-10-09T21:49:58.730624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:58.730630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:58.730634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:58.730659Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a203d3d9-7158c173-fa578137-73ad1eec_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:58.730666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:58.730670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:58.730672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:58.730681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:58.730691Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:58.730710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:58.730915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:58.730924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:58.730939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:58.731052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1682 partition: 0 MaxSeqNo: 25 sessionId: src_id|a203d3d9-7158c173-fa578137-73ad1eec_0
2025-10-09T21:49:58.731383Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fabfc2d9-ee983d97-c975656d-e86a476e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a203d3d9-7158c173-fa578137-73ad1eec_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:58.733728Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a203d3d9-7158c173-fa578137-73ad1eec_0] Write session: try to update token
2025-10-09T21:49:58.733747Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a203d3d9-7158c173-fa578137-73ad1eec_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:58.744526Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:58.744543Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:58.744545Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:58.744547Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:58.744556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342578733978655:3915]
2025-10-09T21:49:58.744560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342578733978655:3915]
2025-10-09T21:49:58.744564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342578733978655:3915]
2025-10-09T21:49:58.744604Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:58.744608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:58.744608Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:58.744611Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:58.744615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.744618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:58.744745Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:58.745884Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1682 sessionId: src_id|a203d3d9-7158c173-fa578137-73ad1eec_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:58.747947Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:58.799853Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342579095808348:7774]
2025-10-09T21:49:58.800109Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 1681, pipe: [1:7559342579095808348:7774]
2025-10-09T21:49:58.889019Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:58.889517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:58.889539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:58.889577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342579095808348:7774], now have 1 active actors on pipe
2025-10-09T21:49:58.889790Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:58.889829Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342579095808348:7774]
2025-10-09T21:49:58.889837Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:58.889839Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:58.889913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1682 reason: error at write: Disconnected sessionId: src_id|a203d3d9-7158c173-fa578137-73ad1eec_0
2025-10-09T21:49:58.889960Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:58.890099Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1682 sessionId: src_id|a203d3d9-7158c173-fa578137-73ad1eec_0 is DEAD
2025-10-09T21:49:58.890255Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a203d3d9-7158c173-fa578137-73ad1eec_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:58.890267Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a203d3d9-7158c173-fa578137-73ad1eec_0] Write session will restart in 0.006859s
2025-10-09T21:49:58.890300Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a203d3d9-7158c173-fa578137-73ad1eec_0] Write session: Do CDS request
2025-10-09T21:49:58.890306Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a203d3d9-7158c173-fa578137-73ad1eec_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:58.898434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a203d3d9-7158c173-fa578137-73ad1eec_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:58.898476Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:58.898492Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1683
2025-10-09T21:49:58.898687Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1683 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:58.898711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1683 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:58.898715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1683 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:58.898720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:58.899160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1683 sessionId:  describe result for acl check
2025-10-09T21:49:58.899199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:58.899210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:58.899212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:58.899218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342579095808413:7046] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:58.899757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342579095808413:7046] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:58.902656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342579095808413:7046] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:58.905374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342579095808413:7046] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:58.905389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342579095808413:7046] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:58.905392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342579095808413:7046] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:58.905401Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1683 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:58.911098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:58.911596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:58.911771Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:58.911780Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:58.911791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:58.911841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:58.911881Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342578733978923:3907]
2025-10-09T21:49:58.912470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:58.912847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:58.912892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:58.913194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:58.913250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.913256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.913258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.913259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.913260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.913261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.913262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.913263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.913263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.913268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.913279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.913283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.913287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.913290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.913294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.913298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.913301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.913305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.913312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.913320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:58.913326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.913331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:58.913335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.913340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:58.913344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.913350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:58.913355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.913359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:58.913364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.913370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:58.913375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.913380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.913387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.913393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.913397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.913402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.913430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:58.919542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:58.920652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:58.920668Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:58.920671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:58.920675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:58.920680Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1682 [2:7559342578733978923:3907]
2025-10-09T21:49:58.921089Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:58.921117Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:58.921120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:58.921123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:58.921126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:58.921142Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:58.921153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:58.921160Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:58.921219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.921226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.921229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.921232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:58.921298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:58.921305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.921318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:58.921331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:58.921354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:58.921381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342579095808353:2432], now have 1 active actors on pipe
2025-10-09T21:49:58.921393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342578733978918:3907]
2025-10-09T21:49:58.921395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342578733978918:3907]
2025-10-09T21:49:58.921397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342578733978918:3907]
2025-10-09T21:49:58.921410Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:58.921424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342579095808610:7046], now have 1 active actors on pipe
2025-10-09T21:49:58.921432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:58.921432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:58.921436Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:58.921438Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:58.921450Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:58.921454Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:58.921454Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1682
2025-10-09T21:49:58.921459Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1682
2025-10-09T21:49:58.921480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342578733978918:3907]
2025-10-09T21:49:58.921489Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:58.921492Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:58.921517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:58.921534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:58.921569Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a171e870-4c17d33d-dcd4db09-29b1ab49_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:58.921581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:58.921585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:58.921587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:58.921597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:58.921610Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:58.921627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:58.921815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:58.921826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:58.921844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:58.921945Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1683 partition: 0 MaxSeqNo: 25 sessionId: src_id|a171e870-4c17d33d-dcd4db09-29b1ab49_0
2025-10-09T21:49:58.922287Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a203d3d9-7158c173-fa578137-73ad1eec_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a171e870-4c17d33d-dcd4db09-29b1ab49_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:58.924654Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a171e870-4c17d33d-dcd4db09-29b1ab49_0] Write session: try to update token
2025-10-09T21:49:58.924672Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a171e870-4c17d33d-dcd4db09-29b1ab49_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:58.935575Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:58.935593Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:58.935595Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:58.935597Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:58.935607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342578733978918:3907]
2025-10-09T21:49:58.935612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342578733978918:3907]
2025-10-09T21:49:58.935616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342578733978918:3907]
2025-10-09T21:49:58.935659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:58.935663Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:58.935669Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:58.935672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:58.935673Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:58.935675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:58.935807Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:58.937181Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1683 sessionId: src_id|a171e870-4c17d33d-dcd4db09-29b1ab49_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:58.939496Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:58.989817Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342579095808631:7774]
2025-10-09T21:49:58.990084Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 1682, pipe: [1:7559342579095808631:7774]
2025-10-09T21:49:59.083867Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:59.084089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:59.084102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:59.084146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342579095808631:7774], now have 1 active actors on pipe
2025-10-09T21:49:59.084361Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:59.084368Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:59.084374Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:59.084404Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342579095808631:7774]
2025-10-09T21:49:59.084429Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1683 reason: error at write: Disconnected sessionId: src_id|a171e870-4c17d33d-dcd4db09-29b1ab49_0
2025-10-09T21:49:59.084592Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1683 sessionId: src_id|a171e870-4c17d33d-dcd4db09-29b1ab49_0 is DEAD
2025-10-09T21:49:59.084682Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:59.084730Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a171e870-4c17d33d-dcd4db09-29b1ab49_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:59.084742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a171e870-4c17d33d-dcd4db09-29b1ab49_0] Write session will restart in 0.009483s
2025-10-09T21:49:59.084770Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a171e870-4c17d33d-dcd4db09-29b1ab49_0] Write session: Do CDS request
2025-10-09T21:49:59.084776Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a171e870-4c17d33d-dcd4db09-29b1ab49_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:59.095543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a171e870-4c17d33d-dcd4db09-29b1ab49_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:59.095603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:59.095618Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1684
2025-10-09T21:49:59.095784Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1684 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:59.095818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1684 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:59.095828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1684 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:59.095831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:59.096298Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1684 sessionId:  describe result for acl check
2025-10-09T21:49:59.096335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:59.096337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:59.096339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:59.096346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342583390776023:7044] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:59.096860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342583390776023:7044] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:59.099350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342583390776023:7044] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:59.102093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342583390776023:7044] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:59.102108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342583390776023:7044] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:59.102111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342583390776023:7044] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:59.102120Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1684 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:59.106609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:59.107109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:59.107343Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:59.107353Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:59.107365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:59.107417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:59.107454Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342583028946479:3905]
2025-10-09T21:49:59.108034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:59.108357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:59.108403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:59.108703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:59.108764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.108770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.108772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.108773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.108774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.108775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.108776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.108777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.108779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.108784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.108794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.108798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.108802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.108806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.108810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.108814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.108817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.108821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.108828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.108837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.108843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.108849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.108853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.108859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.108863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.108873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.108878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.108883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.108888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.108893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.108899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.108905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.108912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.108917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.108922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.108931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.108966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:59.115117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:59.116484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:59.116496Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:59.116498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:59.116501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:59.116506Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1683 [2:7559342583028946479:3905]
2025-10-09T21:49:59.116516Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:59.116529Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:59.116531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:59.116533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:59.116535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:59.116547Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:59.116551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:59.116557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:59.116601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.116607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.116608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.116610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:59.116661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:59.116667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.116675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.116681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.116691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:59.116708Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342583028946474:3905]
2025-10-09T21:49:59.116713Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342583028946474:3905]
2025-10-09T21:49:59.116714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342583028946474:3905]
2025-10-09T21:49:59.116721Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:59.116732Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:59.116733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342583390776196:7044], now have 1 active actors on pipe
2025-10-09T21:49:59.116735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342583390775930:2432], now have 1 active actors on pipe
2025-10-09T21:49:59.116742Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:59.116742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:59.116744Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:59.116751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:59.116754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:59.116779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342583028946474:3905]
2025-10-09T21:49:59.116792Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:59.116800Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:59.116790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1683
2025-10-09T21:49:59.116806Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1683
2025-10-09T21:49:59.116842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:59.116853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:59.116878Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|68c7d3a4-ef3be906-9568cd0b-95f56e0e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:59.116886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:59.116890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:59.116891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:59.116898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:59.116908Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:59.116927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:59.117074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:59.117079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:59.117089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:59.117178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1684 partition: 0 MaxSeqNo: 25 sessionId: src_id|68c7d3a4-ef3be906-9568cd0b-95f56e0e_0
2025-10-09T21:49:59.117561Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a171e870-4c17d33d-dcd4db09-29b1ab49_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|68c7d3a4-ef3be906-9568cd0b-95f56e0e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:59.119934Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|68c7d3a4-ef3be906-9568cd0b-95f56e0e_0] Write session: try to update token
2025-10-09T21:49:59.119955Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|68c7d3a4-ef3be906-9568cd0b-95f56e0e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:59.130507Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:59.130524Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:59.130527Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:59.130529Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:59.130539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342583028946474:3905]
2025-10-09T21:49:59.130543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342583028946474:3905]
2025-10-09T21:49:59.130547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342583028946474:3905]
2025-10-09T21:49:59.130595Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:59.130599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:59.130602Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:59.130605Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:59.130605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.130608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:59.130738Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:59.132666Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1684 sessionId: src_id|68c7d3a4-ef3be906-9568cd0b-95f56e0e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:59.134970Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:59.184322Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342583390776235:7774]
2025-10-09T21:49:59.184530Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 1683, pipe: [1:7559342583390776235:7774]
2025-10-09T21:49:59.281291Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:59.281785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:59.281798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:59.281836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342583390776235:7774], now have 1 active actors on pipe
2025-10-09T21:49:59.282001Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:59.282046Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342583390776235:7774]
2025-10-09T21:49:59.282050Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:59.282054Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:59.282073Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:59.282118Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1684 reason: error at write: Disconnected sessionId: src_id|68c7d3a4-ef3be906-9568cd0b-95f56e0e_0
2025-10-09T21:49:59.282306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1684 sessionId: src_id|68c7d3a4-ef3be906-9568cd0b-95f56e0e_0 is DEAD
2025-10-09T21:49:59.282451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|68c7d3a4-ef3be906-9568cd0b-95f56e0e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:59.282467Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|68c7d3a4-ef3be906-9568cd0b-95f56e0e_0] Write session will restart in 0.009483s
2025-10-09T21:49:59.282487Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|68c7d3a4-ef3be906-9568cd0b-95f56e0e_0] Write session: Do CDS request
2025-10-09T21:49:59.282493Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|68c7d3a4-ef3be906-9568cd0b-95f56e0e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:59.293527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|68c7d3a4-ef3be906-9568cd0b-95f56e0e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:59.293930Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:59.293947Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1685
2025-10-09T21:49:59.294090Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1685 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:59.294123Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1685 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:59.294128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1685 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:59.294134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:59.294615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1685 sessionId:  describe result for acl check
2025-10-09T21:49:59.294656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:59.294663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:59.294665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:59.294671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342583390776372:7029] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:59.295210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342583390776372:7029] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:59.298025Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342583390776372:7029] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:59.300258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:59.303438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:59.303626Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:59.303637Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:59.303651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:59.303705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:59.303743Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342583028946779:3877]
2025-10-09T21:49:59.303820Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342583390776372:7029] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:59.303834Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342583390776372:7029] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:59.303838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342583390776372:7029] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:59.303846Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1685 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:59.304347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:59.304669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:59.304729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:59.304963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:59.304997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.305002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.305003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.305003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.305011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.305011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.305012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.305013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.305013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.305017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.305024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.305026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.305028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.305030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.305032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.305034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.305036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.305038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.305044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.305049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.305053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.305056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.305061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.305067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.305071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.305076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.305080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.305084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.305088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.305093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.305097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.305102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.305107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.305112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.305114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.305117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.305133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:59.311539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:59.313160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:59.313172Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:59.313174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:59.313178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:59.313185Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1684 [2:7559342583028946779:3877]
2025-10-09T21:49:59.313199Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:59.313217Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:59.313220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:59.313223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:59.313225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:59.313242Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:59.313252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:59.313259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:59.313313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.313320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.313322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.313325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:59.313394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:59.313401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.313413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.313430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.313451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:59.313478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342583390776510:7029], now have 1 active actors on pipe
2025-10-09T21:49:59.313486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342583390776242:2432], now have 1 active actors on pipe
2025-10-09T21:49:59.313494Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342583028946773:3877]
2025-10-09T21:49:59.313497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342583028946773:3877]
2025-10-09T21:49:59.313499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342583028946773:3877]
2025-10-09T21:49:59.313514Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:59.313542Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:59.313546Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:59.313547Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:59.313548Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:59.313562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:59.313566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:59.313596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342583028946773:3877]
2025-10-09T21:49:59.313573Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1684
2025-10-09T21:49:59.313599Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:59.313592Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1684
2025-10-09T21:49:59.313602Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:59.313615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:59.313621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:59.313954Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fe3f3928-15cbdf37-1f3111fe-54c1c173_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:59.313969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:59.313973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:59.313975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:59.313986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:59.313996Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:59.314026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:59.314261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:59.314274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:59.314296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:59.314466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1685 partition: 0 MaxSeqNo: 25 sessionId: src_id|fe3f3928-15cbdf37-1f3111fe-54c1c173_0
2025-10-09T21:49:59.314817Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|68c7d3a4-ef3be906-9568cd0b-95f56e0e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fe3f3928-15cbdf37-1f3111fe-54c1c173_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:59.317212Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe3f3928-15cbdf37-1f3111fe-54c1c173_0] Write session: try to update token
2025-10-09T21:49:59.317229Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe3f3928-15cbdf37-1f3111fe-54c1c173_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:59.328716Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:59.328736Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:59.328739Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:59.328744Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:59.328754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342583028946773:3877]
2025-10-09T21:49:59.328758Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342583028946773:3877]
2025-10-09T21:49:59.328763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342583028946773:3877]
2025-10-09T21:49:59.328808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:59.328815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.328818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:59.328830Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:59.328847Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:59.328851Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:59.328948Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:59.330866Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1685 sessionId: src_id|fe3f3928-15cbdf37-1f3111fe-54c1c173_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:59.333495Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:59.382022Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342583390776522:7774]
2025-10-09T21:49:59.382285Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 1684, pipe: [1:7559342583390776522:7774]
2025-10-09T21:49:59.477376Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:59.477629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:59.477644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:59.477690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342583390776522:7774], now have 1 active actors on pipe
2025-10-09T21:49:59.477848Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:59.477915Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342583390776522:7774]
2025-10-09T21:49:59.477929Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:59.477933Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:59.477943Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:59.478006Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1685 reason: error at write: Disconnected sessionId: src_id|fe3f3928-15cbdf37-1f3111fe-54c1c173_0
2025-10-09T21:49:59.478185Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1685 sessionId: src_id|fe3f3928-15cbdf37-1f3111fe-54c1c173_0 is DEAD
2025-10-09T21:49:59.478344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe3f3928-15cbdf37-1f3111fe-54c1c173_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:59.478356Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe3f3928-15cbdf37-1f3111fe-54c1c173_0] Write session will restart in 0.005112s
2025-10-09T21:49:59.478382Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe3f3928-15cbdf37-1f3111fe-54c1c173_0] Write session: Do CDS request
2025-10-09T21:49:59.478388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe3f3928-15cbdf37-1f3111fe-54c1c173_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:59.484543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe3f3928-15cbdf37-1f3111fe-54c1c173_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:59.484637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:59.484654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1686
2025-10-09T21:49:59.484784Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1686 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:59.484818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1686 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:59.484828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1686 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:59.484832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:59.485269Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1686 sessionId:  describe result for acl check
2025-10-09T21:49:59.485308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:59.485315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:59.485317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:59.485323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342583390776542:7018] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:59.485904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342583390776542:7018] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:59.488617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342583390776542:7018] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:59.492205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342583390776542:7018] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:59.492217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342583390776542:7018] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:59.492220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342583390776542:7018] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:59.492228Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1686 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:59.500790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:59.501350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:59.501560Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:59.501567Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:59.501577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:59.501620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:59.501656Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342583028947042:3875]
2025-10-09T21:49:59.502097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:59.502345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:59.502375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:59.502597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:59.502642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.502648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.502649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.502650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.502651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.502651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.502652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.502653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.502653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.502657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.502664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.502667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.502669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.502672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.502674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.502676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.502678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.502680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.502686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.502691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.502695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.502698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.502701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.502704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.502706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.502709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.502712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.502715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.502718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.502722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.502724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.502728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.502730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.502733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.502736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.502740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.502756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:59.509022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:59.510476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:59.510489Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:59.510492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:59.510496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:59.510503Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1685 [2:7559342583028947042:3875]
2025-10-09T21:49:59.510519Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:59.510539Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:59.510542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:59.510545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:59.510548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:59.510568Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:59.510575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:59.510584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:59.510658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.510667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.510669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.510673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:59.510750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:59.510758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.510772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.510782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.510797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:59.510826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342583390776713:7018], now have 1 active actors on pipe
2025-10-09T21:49:59.510837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342583390776524:2432], now have 1 active actors on pipe
2025-10-09T21:49:59.510845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342583028947037:3875]
2025-10-09T21:49:59.510848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342583028947037:3875]
2025-10-09T21:49:59.510850Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342583028947037:3875]
2025-10-09T21:49:59.510865Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:59.510877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:59.510892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:59.510898Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:59.510901Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:59.510903Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:59.510905Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:59.510908Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1685
2025-10-09T21:49:59.510942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342583028947037:3875]
2025-10-09T21:49:59.510950Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:59.510953Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:59.510909Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1685
2025-10-09T21:49:59.510977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:59.510987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:59.511025Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|35739907-63ce528e-8852bc3e-d8e09c30_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:59.511038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:59.511048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:59.511051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:59.511061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:59.511071Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:59.511093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:59.511259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:59.511268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:59.511283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:59.511378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1686 partition: 0 MaxSeqNo: 25 sessionId: src_id|35739907-63ce528e-8852bc3e-d8e09c30_0
2025-10-09T21:49:59.511765Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe3f3928-15cbdf37-1f3111fe-54c1c173_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|35739907-63ce528e-8852bc3e-d8e09c30_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:59.514283Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|35739907-63ce528e-8852bc3e-d8e09c30_0] Write session: try to update token
2025-10-09T21:49:59.514305Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|35739907-63ce528e-8852bc3e-d8e09c30_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:59.525108Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:59.525122Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:59.525124Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:59.525125Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:59.525134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342583028947037:3875]
2025-10-09T21:49:59.525137Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342583028947037:3875]
2025-10-09T21:49:59.525140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342583028947037:3875]
2025-10-09T21:49:59.525178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:59.525183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.525185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:59.525226Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:59.525251Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:59.525258Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:59.525312Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:59.527725Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1686 sessionId: src_id|35739907-63ce528e-8852bc3e-d8e09c30_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:59.530410Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:59.578102Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342583390776806:7774]
2025-10-09T21:49:59.578382Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 1685, pipe: [1:7559342583390776806:7774]
2025-10-09T21:49:59.666984Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:59.667507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:59.667522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:59.667568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342583390776806:7774], now have 1 active actors on pipe
2025-10-09T21:49:59.667833Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342583390776806:7774]
2025-10-09T21:49:59.667840Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:59.667844Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:59.667869Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:59.667900Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:59.667907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1686 reason: error at write: Disconnected sessionId: src_id|35739907-63ce528e-8852bc3e-d8e09c30_0
2025-10-09T21:49:59.668098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1686 sessionId: src_id|35739907-63ce528e-8852bc3e-d8e09c30_0 is DEAD
2025-10-09T21:49:59.668264Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35739907-63ce528e-8852bc3e-d8e09c30_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:59.668277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35739907-63ce528e-8852bc3e-d8e09c30_0] Write session will restart in 0.006972s
2025-10-09T21:49:59.668304Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35739907-63ce528e-8852bc3e-d8e09c30_0] Write session: Do CDS request
2025-10-09T21:49:59.668311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35739907-63ce528e-8852bc3e-d8e09c30_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:59.676471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|35739907-63ce528e-8852bc3e-d8e09c30_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:59.676569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:59.676582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1687
2025-10-09T21:49:59.676720Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1687 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:59.676753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1687 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:59.676765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1687 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:59.676768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:59.677189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1687 sessionId:  describe result for acl check
2025-10-09T21:49:59.677226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:59.677232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:59.677234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:59.677240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342583390776857:7013] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:59.677814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342583390776857:7013] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:59.680583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342583390776857:7013] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:59.683228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342583390776857:7013] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:59.683242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342583390776857:7013] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:59.683246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342583390776857:7013] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:59.683253Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1687 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:59.690164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:59.690585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:59.690927Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:59.690936Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:59.690948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:59.690996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:59.691039Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342583028947305:3870]
2025-10-09T21:49:59.691494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:59.691749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:59.691779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:59.692006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:59.692046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.692051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.692051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.692052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.692053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.692054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.692054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.692055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.692055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.692059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.692065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.692067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.692069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.692071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.692074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.692075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.692077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.692079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.692084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.692089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.692093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.692096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.692099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.692102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.692105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.692107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.692110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.692114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.692118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.692123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.692129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.692133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.692138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.692142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.692146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.692151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.692171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:59.698045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:59.699060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:59.699077Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:59.699079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:59.699084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:59.699090Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1686 [2:7559342583028947305:3870]
2025-10-09T21:49:59.699106Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:59.699124Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:59.699132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:59.699135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:59.699138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:59.699154Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:59.699164Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:59.699172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:59.699231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.699238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.699240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.699243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:59.699311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:59.699318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.699332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.699344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.699366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:59.699390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342583390777032:7013], now have 1 active actors on pipe
2025-10-09T21:49:59.699397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342583390776808:2432], now have 1 active actors on pipe
2025-10-09T21:49:59.699404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342583028947300:3870]
2025-10-09T21:49:59.699407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342583028947300:3870]
2025-10-09T21:49:59.699409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342583028947300:3870]
2025-10-09T21:49:59.699421Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:59.699438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:59.699438Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:59.699441Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:59.699443Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:59.699456Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:59.699459Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:59.699489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342583028947300:3870]
2025-10-09T21:49:59.699452Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1686
2025-10-09T21:49:59.699499Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:59.699470Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1686
2025-10-09T21:49:59.699515Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:59.699515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:59.699519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:59.699565Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|62fecc92-8ff95312-11c1bef8-b09780fb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:59.699577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:59.699581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:59.699583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:59.699592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:59.699601Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:59.699622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:59.699766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:59.699774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:59.699787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:59.699860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1687 partition: 0 MaxSeqNo: 25 sessionId: src_id|62fecc92-8ff95312-11c1bef8-b09780fb_0
2025-10-09T21:49:59.700210Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35739907-63ce528e-8852bc3e-d8e09c30_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|62fecc92-8ff95312-11c1bef8-b09780fb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:59.702655Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62fecc92-8ff95312-11c1bef8-b09780fb_0] Write session: try to update token
2025-10-09T21:49:59.702674Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62fecc92-8ff95312-11c1bef8-b09780fb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:59.713205Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:59.713220Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:59.713222Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:59.713223Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:59.713230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342583028947300:3870]
2025-10-09T21:49:59.713233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342583028947300:3870]
2025-10-09T21:49:59.713236Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342583028947300:3870]
2025-10-09T21:49:59.713270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:59.713275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.713277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:59.713289Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:59.713296Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:59.713300Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:59.713403Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:59.714337Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1687 sessionId: src_id|62fecc92-8ff95312-11c1bef8-b09780fb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:59.716771Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:59.768013Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342583390777091:7774]
2025-10-09T21:49:59.768246Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 1686, pipe: [1:7559342583390777091:7774]
2025-10-09T21:49:59.859630Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:49:59.860399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:59.860413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:59.860458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342583390777091:7774], now have 1 active actors on pipe
2025-10-09T21:49:59.860683Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:49:59.860697Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:49:59.860715Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:49:59.860770Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342583390777091:7774]
2025-10-09T21:49:59.860796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1687 reason: error at write: Disconnected sessionId: src_id|62fecc92-8ff95312-11c1bef8-b09780fb_0
2025-10-09T21:49:59.860860Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:49:59.860971Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1687 sessionId: src_id|62fecc92-8ff95312-11c1bef8-b09780fb_0 is DEAD
2025-10-09T21:49:59.861130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62fecc92-8ff95312-11c1bef8-b09780fb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:49:59.861142Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62fecc92-8ff95312-11c1bef8-b09780fb_0] Write session will restart in 0.005906s
2025-10-09T21:49:59.861170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62fecc92-8ff95312-11c1bef8-b09780fb_0] Write session: Do CDS request
2025-10-09T21:49:59.861176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62fecc92-8ff95312-11c1bef8-b09780fb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:49:59.868459Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|62fecc92-8ff95312-11c1bef8-b09780fb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:59.868547Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:49:59.868574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1688
2025-10-09T21:49:59.868710Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1688 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:49:59.868743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1688 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:49:59.868754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1688 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:49:59.868758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:49:59.869187Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1688 sessionId:  describe result for acl check
2025-10-09T21:49:59.869220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:49:59.869229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:49:59.869230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:49:59.869237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342583390777131:6998] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:49:59.869788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342583390777131:6998] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:49:59.872823Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342583390777131:6998] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:49:59.875413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342583390777131:6998] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:49:59.875428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342583390777131:6998] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:49:59.875431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342583390777131:6998] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:49:59.875439Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1688 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:49:59.878383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:49:59.881548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:49:59.881735Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:49:59.881744Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:49:59.881756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:49:59.881810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:49:59.881850Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342583028947573:3861]
2025-10-09T21:49:59.882491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:49:59.882798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:49:59.882850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:49:59.883218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:49:59.883266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.883272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.883273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.883275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.883276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.883277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.883278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.883279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.883280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.883285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.883296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.883300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.883304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.883307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.883311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.883315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.883318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.883322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.883329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.883337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:49:59.883343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.883348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:49:59.883355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.883366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:49:59.883371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.883377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:49:59.883388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.883393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:49:59.883401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.883407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:49:59.883418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.883423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.883429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.883438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.883443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.883448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.883478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:49:59.889838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:49:59.891178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:49:59.891197Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:49:59.891200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:49:59.891204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:49:59.891211Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1687 [2:7559342583028947573:3861]
2025-10-09T21:49:59.891225Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:49:59.891242Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:49:59.891252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:59.891256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:59.891258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:59.891275Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:49:59.891282Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:49:59.891293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:49:59.891356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.891364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.891367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.891370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:49:59.891436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:49:59.891443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.891456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:49:59.891474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:49:59.891492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:49:59.891517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342583390777323:6998], now have 1 active actors on pipe
2025-10-09T21:49:59.891524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342583390777096:2432], now have 1 active actors on pipe
2025-10-09T21:49:59.891531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342583028947568:3861]
2025-10-09T21:49:59.891534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342583028947568:3861]
2025-10-09T21:49:59.891536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342583028947568:3861]
2025-10-09T21:49:59.891549Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:49:59.891567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:49:59.891570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:49:59.891574Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:49:59.891577Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:49:59.891582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:49:59.891564Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1687
2025-10-09T21:49:59.891585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:49:59.891613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342583028947568:3861]
2025-10-09T21:49:59.891616Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:49:59.891566Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1687
2025-10-09T21:49:59.891619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:49:59.891631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:59.891636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:59.891688Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|12f56349-bc817826-bdb89c22-4a55c8da_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:49:59.891700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:49:59.891704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:49:59.891707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:49:59.891717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:49:59.891731Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:49:59.891755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:59.891946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:49:59.891954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:49:59.891968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:49:59.892044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1688 partition: 0 MaxSeqNo: 25 sessionId: src_id|12f56349-bc817826-bdb89c22-4a55c8da_0
2025-10-09T21:49:59.892418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|62fecc92-8ff95312-11c1bef8-b09780fb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|12f56349-bc817826-bdb89c22-4a55c8da_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:49:59.894790Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12f56349-bc817826-bdb89c22-4a55c8da_0] Write session: try to update token
2025-10-09T21:49:59.894808Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12f56349-bc817826-bdb89c22-4a55c8da_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:49:59.905578Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:49:59.905595Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:49:59.905598Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:49:59.905600Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:49:59.905609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342583028947568:3861]
2025-10-09T21:49:59.905614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342583028947568:3861]
2025-10-09T21:49:59.905617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342583028947568:3861]
2025-10-09T21:49:59.905661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:49:59.905661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:49:59.905666Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:49:59.905670Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:49:59.905674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:49:59.905677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:49:59.905814Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:49:59.906809Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1688 sessionId: src_id|12f56349-bc817826-bdb89c22-4a55c8da_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:49:59.909270Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:49:59.961055Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342583390777381:7774]
2025-10-09T21:49:59.961379Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 1687, pipe: [1:7559342583390777381:7774]
2025-10-09T21:50:00.051251Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:00.051744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:00.051760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:00.051799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342583390777381:7774], now have 1 active actors on pipe
2025-10-09T21:50:00.051958Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:00.052054Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:00.052067Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:00.052072Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342583390777381:7774]
2025-10-09T21:50:00.052094Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:00.052127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1688 reason: error at write: Disconnected sessionId: src_id|12f56349-bc817826-bdb89c22-4a55c8da_0
2025-10-09T21:50:00.052297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1688 sessionId: src_id|12f56349-bc817826-bdb89c22-4a55c8da_0 is DEAD
2025-10-09T21:50:00.052451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12f56349-bc817826-bdb89c22-4a55c8da_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:00.052463Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12f56349-bc817826-bdb89c22-4a55c8da_0] Write session will restart in 0.007810s
2025-10-09T21:50:00.052489Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12f56349-bc817826-bdb89c22-4a55c8da_0] Write session: Do CDS request
2025-10-09T21:50:00.052496Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12f56349-bc817826-bdb89c22-4a55c8da_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:00.061539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12f56349-bc817826-bdb89c22-4a55c8da_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:00.061560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:00.061576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1689
2025-10-09T21:50:00.061752Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1689 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:00.061785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1689 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:00.061795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1689 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:00.061799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:00.062284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1689 sessionId:  describe result for acl check
2025-10-09T21:50:00.062323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:00.062330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:00.062331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:00.062337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342587685744765:7001] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:00.062913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342587685744765:7001] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:00.065878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342587685744765:7001] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:00.068718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342587685744765:7001] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:00.068732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342587685744765:7001] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:00.068736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342587685744765:7001] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:00.068744Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1689 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:00.073010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:00.073443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:00.073606Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:00.073614Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:00.073623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:00.073669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:00.073709Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342587323915134:3848]
2025-10-09T21:50:00.074230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:00.074483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:00.074521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:00.074816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:00.074865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.074871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.074872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.074874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.074875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.074877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.074878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.074879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.074880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.074885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.074896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.074900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.074904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.074908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.074912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.074916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.074920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.074924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.074932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.074941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.074947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.074952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.074956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.074962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.074966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.074971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.074976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.074981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.074985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.074990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.074994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.074997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.075000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.075003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.075006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.075009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.075025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:00.081405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:00.082761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:00.082777Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:00.082780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:00.082784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:00.082791Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1688 [2:7559342587323915134:3848]
2025-10-09T21:50:00.082804Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:00.082821Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:00.082829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:00.082832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:00.082835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:00.082851Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:00.082858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:00.082865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:00.082926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.082933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.082936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.082939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:00.083008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:00.083016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.083029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.083046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.083068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:00.083095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342587685744680:2432], now have 1 active actors on pipe
2025-10-09T21:50:00.083103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342587685744951:7001], now have 1 active actors on pipe
2025-10-09T21:50:00.083110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342587323915129:3848]
2025-10-09T21:50:00.083113Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342587323915129:3848]
2025-10-09T21:50:00.083115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342587323915129:3848]
2025-10-09T21:50:00.083127Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:00.083130Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1688
2025-10-09T21:50:00.083148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:00.083141Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1688
2025-10-09T21:50:00.083149Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:00.083154Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:00.083158Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:00.083164Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:00.083167Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:00.083206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342587323915129:3848]
2025-10-09T21:50:00.083217Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:00.083228Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:00.083232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:00.083238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:00.083284Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c7d1e381-ed982db-77259f0e-dec3b7a4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:00.083297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:00.083301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:00.083303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:00.083313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:00.083328Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:00.083352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:00.083494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:00.083502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:00.083516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:00.083592Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1689 partition: 0 MaxSeqNo: 25 sessionId: src_id|c7d1e381-ed982db-77259f0e-dec3b7a4_0
2025-10-09T21:50:00.083917Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12f56349-bc817826-bdb89c22-4a55c8da_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c7d1e381-ed982db-77259f0e-dec3b7a4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:00.086250Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c7d1e381-ed982db-77259f0e-dec3b7a4_0] Write session: try to update token
2025-10-09T21:50:00.086264Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c7d1e381-ed982db-77259f0e-dec3b7a4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:00.097372Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:00.097388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:00.097390Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:00.097392Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:00.097400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342587323915129:3848]
2025-10-09T21:50:00.097404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342587323915129:3848]
2025-10-09T21:50:00.097408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342587323915129:3848]
2025-10-09T21:50:00.097446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:00.097451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.097453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:00.097459Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:00.097466Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:00.097470Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:00.097587Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:00.098780Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1689 sessionId: src_id|c7d1e381-ed982db-77259f0e-dec3b7a4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:00.101322Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:00.152077Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342587685744964:7774]
2025-10-09T21:50:00.152355Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 1688, pipe: [1:7559342587685744964:7774]
2025-10-09T21:50:00.249008Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:00.249398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:00.249413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:00.249452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342587685744964:7774], now have 1 active actors on pipe
2025-10-09T21:50:00.249707Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:00.249732Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:00.249737Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342587685744964:7774]
2025-10-09T21:50:00.249759Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:00.249761Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:00.249818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1689 reason: error at write: Disconnected sessionId: src_id|c7d1e381-ed982db-77259f0e-dec3b7a4_0
2025-10-09T21:50:00.249980Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1689 sessionId: src_id|c7d1e381-ed982db-77259f0e-dec3b7a4_0 is DEAD
2025-10-09T21:50:00.250155Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7d1e381-ed982db-77259f0e-dec3b7a4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:00.250171Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7d1e381-ed982db-77259f0e-dec3b7a4_0] Write session will restart in 0.009007s
2025-10-09T21:50:00.250191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7d1e381-ed982db-77259f0e-dec3b7a4_0] Write session: Do CDS request
2025-10-09T21:50:00.250196Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7d1e381-ed982db-77259f0e-dec3b7a4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:00.260491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c7d1e381-ed982db-77259f0e-dec3b7a4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:00.260546Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:00.260563Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1690
2025-10-09T21:50:00.260712Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1690 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:00.260738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1690 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:00.260747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1690 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:00.260751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:00.261275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1690 sessionId:  describe result for acl check
2025-10-09T21:50:00.261308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:00.261312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:00.261314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:00.261318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342587685745102:6983] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:00.261820Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342587685745102:6983] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:00.264870Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342587685745102:6983] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:00.267480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342587685745102:6983] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:00.267494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342587685745102:6983] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:00.267498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342587685745102:6983] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:00.267507Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1690 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:00.268062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:00.271773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:00.271984Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:00.271994Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:00.272007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:00.272058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:00.272099Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342587323915439:3823]
2025-10-09T21:50:00.272694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:00.273041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:00.273097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:00.273334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:00.273380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.273385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.273386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.273387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.273388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.273389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.273389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.273390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.273391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.273394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.273402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.273404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.273406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.273408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.273411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.273413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.273415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.273417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.273423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.273428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.273432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.273436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.273440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.273445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.273449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.273454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.273459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.273464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.273469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.273474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.273485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.273488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.273491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.273494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.273497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.273500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.273533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:00.279923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:00.281396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:00.281411Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:00.281414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:00.281419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:00.281426Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1689 [2:7559342587323915439:3823]
2025-10-09T21:50:00.281442Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:00.281464Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:00.281467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:00.281470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:00.281473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:00.281490Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:00.281501Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:00.281508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:00.281581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.281588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.281590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.281593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:00.281663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:00.281671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.281683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.281695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.281713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:00.281738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342587685745268:6983], now have 1 active actors on pipe
2025-10-09T21:50:00.281746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342587685744994:2432], now have 1 active actors on pipe
2025-10-09T21:50:00.281754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342587323915434:3823]
2025-10-09T21:50:00.281757Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342587323915434:3823]
2025-10-09T21:50:00.281759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342587323915434:3823]
2025-10-09T21:50:00.281771Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:00.281789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:00.281792Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:00.281796Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:00.281799Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:00.281807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:00.281810Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:00.281808Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1689
2025-10-09T21:50:00.281843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342587323915434:3823]
2025-10-09T21:50:00.281812Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1689
2025-10-09T21:50:00.281851Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:00.281853Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:00.281868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:00.281872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:00.281929Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8019c3c6-39782699-b07ef1d0-2e952251_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:00.281942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:00.281946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:00.281949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:00.281961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:00.281974Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:00.281993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:00.282145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:00.282152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:00.282166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:00.282237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1690 partition: 0 MaxSeqNo: 25 sessionId: src_id|8019c3c6-39782699-b07ef1d0-2e952251_0
2025-10-09T21:50:00.282573Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7d1e381-ed982db-77259f0e-dec3b7a4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8019c3c6-39782699-b07ef1d0-2e952251_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:00.284959Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8019c3c6-39782699-b07ef1d0-2e952251_0] Write session: try to update token
2025-10-09T21:50:00.284975Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8019c3c6-39782699-b07ef1d0-2e952251_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:00.296961Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:00.296981Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:00.296984Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:00.296986Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:00.296996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342587323915434:3823]
2025-10-09T21:50:00.297005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342587323915434:3823]
2025-10-09T21:50:00.297009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342587323915434:3823]
2025-10-09T21:50:00.297053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:00.297061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.297064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:00.297079Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:00.297089Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:00.297095Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:00.297203Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:00.297709Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1690 sessionId: src_id|8019c3c6-39782699-b07ef1d0-2e952251_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:00.300112Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:00.349716Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342587685745282:7774]
2025-10-09T21:50:00.350048Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 1689, pipe: [1:7559342587685745282:7774]
2025-10-09T21:50:00.447817Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:00.448261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:00.448281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:00.448343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342587685745282:7774], now have 1 active actors on pipe
2025-10-09T21:50:00.448560Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:00.448617Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342587685745282:7774]
2025-10-09T21:50:00.448628Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:00.448632Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:00.448661Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:00.448703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1690 reason: error at write: Disconnected sessionId: src_id|8019c3c6-39782699-b07ef1d0-2e952251_0
2025-10-09T21:50:00.448892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1690 sessionId: src_id|8019c3c6-39782699-b07ef1d0-2e952251_0 is DEAD
2025-10-09T21:50:00.449055Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8019c3c6-39782699-b07ef1d0-2e952251_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:00.449067Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8019c3c6-39782699-b07ef1d0-2e952251_0] Write session will restart in 0.007203s
2025-10-09T21:50:00.449094Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8019c3c6-39782699-b07ef1d0-2e952251_0] Write session: Do CDS request
2025-10-09T21:50:00.449100Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8019c3c6-39782699-b07ef1d0-2e952251_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:00.457530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8019c3c6-39782699-b07ef1d0-2e952251_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:00.457589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:00.457602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1691
2025-10-09T21:50:00.457756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1691 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:00.457793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1691 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:00.457801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1691 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:00.457805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:00.458238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1691 sessionId:  describe result for acl check
2025-10-09T21:50:00.458277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:00.458283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:00.458285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:00.458292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342587685745337:6978] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:00.458745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342587685745337:6978] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:00.461281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342587685745337:6978] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:00.464238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342587685745337:6978] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:00.464251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342587685745337:6978] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:00.464265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342587685745337:6978] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:00.464274Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1691 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:00.470537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:00.471022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:00.471195Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:00.471205Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:00.471215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:00.471263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:00.471307Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342587323915706:3821]
2025-10-09T21:50:00.471901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:00.472184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:00.472226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:00.472538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:00.472586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.472591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.472593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.472594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.472595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.472596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.472598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.472599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.472600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.472605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.472616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.472620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.472625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.472629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.472633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.472636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.472640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.472644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.472651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.472659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.472664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.472670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.472675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.472681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.472685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.472691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.472696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.472701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.472710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.472716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.472722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.472728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.472739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.472744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.472750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.472759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.472795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:00.478937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:00.480371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:00.480387Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:00.480390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:00.480394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:00.480401Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1690 [2:7559342587323915706:3821]
2025-10-09T21:50:00.480417Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:00.480441Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:00.480449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:00.480452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:00.480455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:00.480472Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:00.480483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:00.480495Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:00.480562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.480569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.480570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.480573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:00.480644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:00.480655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.480667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.480678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.480693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:00.480716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342587685745284:2432], now have 1 active actors on pipe
2025-10-09T21:50:00.480724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342587685745523:6978], now have 1 active actors on pipe
2025-10-09T21:50:00.480731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342587323915701:3821]
2025-10-09T21:50:00.480737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342587323915701:3821]
2025-10-09T21:50:00.480741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342587323915701:3821]
2025-10-09T21:50:00.480740Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1690
2025-10-09T21:50:00.480753Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1690
2025-10-09T21:50:00.480753Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:00.480769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:00.480771Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:00.480772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:00.480773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:00.480786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:00.480789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:00.480833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:00.480838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:00.480855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342587323915701:3821]
2025-10-09T21:50:00.480866Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:00.480881Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:00.480900Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d8370d12-644104ce-ab3d6e9a-ce610d25_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:00.480912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:00.480916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:00.480919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:00.480931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:00.480942Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:00.480966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:00.481100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:00.481111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:00.481127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:00.481216Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1691 partition: 0 MaxSeqNo: 25 sessionId: src_id|d8370d12-644104ce-ab3d6e9a-ce610d25_0
2025-10-09T21:50:00.481611Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8019c3c6-39782699-b07ef1d0-2e952251_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d8370d12-644104ce-ab3d6e9a-ce610d25_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:00.484016Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d8370d12-644104ce-ab3d6e9a-ce610d25_0] Write session: try to update token
2025-10-09T21:50:00.484035Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d8370d12-644104ce-ab3d6e9a-ce610d25_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:00.495590Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:00.495608Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:00.495611Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:00.495613Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:00.495623Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342587323915701:3821]
2025-10-09T21:50:00.495628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342587323915701:3821]
2025-10-09T21:50:00.495633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342587323915701:3821]
2025-10-09T21:50:00.495676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:00.495678Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:00.495684Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:00.495684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.495686Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:00.495687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:00.495827Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:00.496477Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1691 sessionId: src_id|d8370d12-644104ce-ab3d6e9a-ce610d25_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:00.499034Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:00.548858Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342587685745571:7774]
2025-10-09T21:50:00.549106Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 1690, pipe: [1:7559342587685745571:7774]
2025-10-09T21:50:00.642998Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:00.643568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:00.643581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:00.643620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342587685745571:7774], now have 1 active actors on pipe
2025-10-09T21:50:00.643798Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:00.643845Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:00.643858Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:00.643872Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342587685745571:7774]
2025-10-09T21:50:00.643894Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:00.643926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1691 reason: error at write: Disconnected sessionId: src_id|d8370d12-644104ce-ab3d6e9a-ce610d25_0
2025-10-09T21:50:00.644081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1691 sessionId: src_id|d8370d12-644104ce-ab3d6e9a-ce610d25_0 is DEAD
2025-10-09T21:50:00.644261Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d8370d12-644104ce-ab3d6e9a-ce610d25_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:00.644271Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d8370d12-644104ce-ab3d6e9a-ce610d25_0] Write session will restart in 0.008559s
2025-10-09T21:50:00.644291Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d8370d12-644104ce-ab3d6e9a-ce610d25_0] Write session: Do CDS request
2025-10-09T21:50:00.644296Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d8370d12-644104ce-ab3d6e9a-ce610d25_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:00.653482Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d8370d12-644104ce-ab3d6e9a-ce610d25_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:00.653628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:00.653648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1692
2025-10-09T21:50:00.653788Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1692 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:00.653813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1692 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:00.653820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1692 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:00.653824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:00.654192Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1692 sessionId:  describe result for acl check
2025-10-09T21:50:00.654223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:00.654228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:00.654230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:00.654234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342587685745653:6970] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:00.654659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342587685745653:6970] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:00.657188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342587685745653:6970] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:00.659975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342587685745653:6970] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:00.659986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342587685745653:6970] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:00.659988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342587685745653:6970] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:00.660001Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1692 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:00.664753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:00.665277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:00.665437Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:00.665448Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:00.665460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:00.665510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:00.665559Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342587323915974:3812]
2025-10-09T21:50:00.666166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:00.666447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:00.666496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:00.666804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:00.666856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.666862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.666863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.666864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.666866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.666867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.666869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.666870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.666871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.666876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.666886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.666890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.666894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.666898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.666902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.666906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.666910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.666914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.666920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.666930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.666936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.666941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.666945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.666950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.666954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.666959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.666967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.666972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.666977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.666987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.666992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.666997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.667001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.667011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.667015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.667022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.667052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:00.673177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:00.674249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:00.674266Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:00.674268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:00.674273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:00.674280Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1691 [2:7559342587323915974:3812]
2025-10-09T21:50:00.674294Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:00.674319Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:00.674322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:00.674325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:00.674327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:00.674344Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:00.674355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:00.674363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:00.674418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.674426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.674428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.674431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:00.674499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:00.674506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.674519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.674536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.674558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:00.674584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342587685745847:6970], now have 1 active actors on pipe
2025-10-09T21:50:00.674591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342587685745573:2432], now have 1 active actors on pipe
2025-10-09T21:50:00.674599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342587323915969:3812]
2025-10-09T21:50:00.674602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342587323915969:3812]
2025-10-09T21:50:00.674604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342587323915969:3812]
2025-10-09T21:50:00.674617Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:00.674604Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1691
2025-10-09T21:50:00.674636Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:00.674612Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1691
2025-10-09T21:50:00.674638Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:00.674640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:00.674642Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:00.674655Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:00.674659Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:00.674700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:00.674710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:00.674723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342587323915969:3812]
2025-10-09T21:50:00.674735Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:00.674738Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:00.674764Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5dade6-ae36c3d3-d3e07c55-509b18d3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:00.674775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:00.674779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:00.674782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:00.674792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:00.674805Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:00.674831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:00.674993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:00.675002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:00.675017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:00.675118Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1692 partition: 0 MaxSeqNo: 25 sessionId: src_id|5dade6-ae36c3d3-d3e07c55-509b18d3_0
2025-10-09T21:50:00.675475Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d8370d12-644104ce-ab3d6e9a-ce610d25_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5dade6-ae36c3d3-d3e07c55-509b18d3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:00.677834Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5dade6-ae36c3d3-d3e07c55-509b18d3_0] Write session: try to update token
2025-10-09T21:50:00.677851Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5dade6-ae36c3d3-d3e07c55-509b18d3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:00.688363Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:00.688382Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:00.688385Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:00.688387Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:00.688396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342587323915969:3812]
2025-10-09T21:50:00.688400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342587323915969:3812]
2025-10-09T21:50:00.688405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342587323915969:3812]
2025-10-09T21:50:00.688443Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:00.688448Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:00.688448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:00.688450Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:00.688456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.688458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:00.688587Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:00.689812Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1692 sessionId: src_id|5dade6-ae36c3d3-d3e07c55-509b18d3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:00.691983Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:00.743932Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342587685745861:7774]
2025-10-09T21:50:00.744203Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 1691, pipe: [1:7559342587685745861:7774]
2025-10-09T21:50:00.838383Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:00.838942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:00.838962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:00.839007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342587685745861:7774], now have 1 active actors on pipe
2025-10-09T21:50:00.839210Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:00.839266Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:00.839326Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342587685745861:7774]
2025-10-09T21:50:00.839330Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:00.839405Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1692 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5dade6-ae36c3d3-d3e07c55-509b18d3_0
2025-10-09T21:50:00.839577Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1692 sessionId: src_id|5dade6-ae36c3d3-d3e07c55-509b18d3_0 is DEAD
2025-10-09T21:50:00.839771Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5dade6-ae36c3d3-d3e07c55-509b18d3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:00.839781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5dade6-ae36c3d3-d3e07c55-509b18d3_0] Write session will restart in 0.006198s
2025-10-09T21:50:00.839819Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5dade6-ae36c3d3-d3e07c55-509b18d3_0] Write session: Do CDS request
2025-10-09T21:50:00.839825Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5dade6-ae36c3d3-d3e07c55-509b18d3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:00.847493Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5dade6-ae36c3d3-d3e07c55-509b18d3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:00.847579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:00.847596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1693
2025-10-09T21:50:00.847755Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1693 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:00.847793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1693 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:00.847804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1693 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:00.847808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:00.848266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1693 sessionId:  describe result for acl check
2025-10-09T21:50:00.848303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:00.848310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:00.848312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:00.848318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342587685745918:6967] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:00.848859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342587685745918:6967] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:00.851306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342587685745918:6967] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:00.854215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342587685745918:6967] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:00.854229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342587685745918:6967] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:00.854233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342587685745918:6967] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:00.854242Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1693 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:00.861054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:00.861502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:00.861698Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:00.861710Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:00.861722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:00.861779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:00.861823Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342587323916247:3813]
2025-10-09T21:50:00.862419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:00.862739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:00.862787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:00.863076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:00.863118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.863124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.863125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.863126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.863128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.863129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.863130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.863132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.863133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.863138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.863148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.863150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.863153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.863155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.863157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.863159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.863161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.863164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.863171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.863179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:00.863185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.863190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:00.863196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.863202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:00.863206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.863212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:00.863216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.863222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:00.863227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.863232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:00.863238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.863244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.863248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.863253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.863258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.863269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.863292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:00.869676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:00.871357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:00.871373Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:00.871375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:00.871380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:00.871387Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1692 [2:7559342587323916247:3813]
2025-10-09T21:50:00.871404Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:00.871422Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:00.871431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:00.871434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:00.871437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:00.871454Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:00.871460Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:00.871468Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:00.871529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.871536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.871538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.871542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:00.871607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:00.871612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.871621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:00.871631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:00.871651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:00.871677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342587685746092:6967], now have 1 active actors on pipe
2025-10-09T21:50:00.871691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342587323916242:3813]
2025-10-09T21:50:00.871693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342587323916242:3813]
2025-10-09T21:50:00.871695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342587323916242:3813]
2025-10-09T21:50:00.871710Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:00.871724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342587685745863:2432], now have 1 active actors on pipe
2025-10-09T21:50:00.871736Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:00.871741Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:00.871745Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:00.871747Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:00.871756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:00.871759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:00.871792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342587323916242:3813]
2025-10-09T21:50:00.871800Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:00.871804Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:00.871773Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1692
2025-10-09T21:50:00.871779Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1692
2025-10-09T21:50:00.871839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:00.871848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:00.871887Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2fd02ed0-4a1a657b-d2be1c26-5811bd1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:00.871900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:00.871910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:00.871913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:00.871924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:00.871937Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:00.871953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:00.872254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:00.872266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:00.872279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:00.872370Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1693 partition: 0 MaxSeqNo: 25 sessionId: src_id|2fd02ed0-4a1a657b-d2be1c26-5811bd1_0
2025-10-09T21:50:00.872719Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5dade6-ae36c3d3-d3e07c55-509b18d3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2fd02ed0-4a1a657b-d2be1c26-5811bd1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:00.875183Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2fd02ed0-4a1a657b-d2be1c26-5811bd1_0] Write session: try to update token
2025-10-09T21:50:00.875200Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2fd02ed0-4a1a657b-d2be1c26-5811bd1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:00.885945Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:00.885963Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:00.885966Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:00.885968Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:00.885977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342587323916242:3813]
2025-10-09T21:50:00.885981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342587323916242:3813]
2025-10-09T21:50:00.885985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342587323916242:3813]
2025-10-09T21:50:00.886025Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:00.886044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:00.886050Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:00.886052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:00.886054Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:00.886055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:00.886196Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:00.887810Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1693 sessionId: src_id|2fd02ed0-4a1a657b-d2be1c26-5811bd1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:00.890505Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:00.939516Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342587685746154:7774]
2025-10-09T21:50:00.939776Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 1692, pipe: [1:7559342587685746154:7774]
2025-10-09T21:50:01.036005Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:01.036656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.036675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.036726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342587685746154:7774], now have 1 active actors on pipe
2025-10-09T21:50:01.036954Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:01.037013Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342587685746154:7774]
2025-10-09T21:50:01.037023Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:01.037027Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:01.037048Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:01.037093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1693 reason: error at write: Disconnected sessionId: src_id|2fd02ed0-4a1a657b-d2be1c26-5811bd1_0
2025-10-09T21:50:01.037287Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1693 sessionId: src_id|2fd02ed0-4a1a657b-d2be1c26-5811bd1_0 is DEAD
2025-10-09T21:50:01.037447Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2fd02ed0-4a1a657b-d2be1c26-5811bd1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:01.037457Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2fd02ed0-4a1a657b-d2be1c26-5811bd1_0] Write session will restart in 0.008479s
2025-10-09T21:50:01.037480Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2fd02ed0-4a1a657b-d2be1c26-5811bd1_0] Write session: Do CDS request
2025-10-09T21:50:01.037488Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2fd02ed0-4a1a657b-d2be1c26-5811bd1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:01.047540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2fd02ed0-4a1a657b-d2be1c26-5811bd1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:01.047589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:01.047606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1694
2025-10-09T21:50:01.047783Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1694 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:01.047816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1694 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:01.047830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1694 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:01.047839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:01.048260Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1694 sessionId:  describe result for acl check
2025-10-09T21:50:01.048294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:01.048299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:01.048301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:01.048306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342591980713540:6953] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:01.048867Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342591980713540:6953] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:01.051546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342591980713540:6953] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:01.054520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342591980713540:6953] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:01.054538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342591980713540:6953] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:01.054542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342591980713540:6953] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:01.054551Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1694 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:01.059656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:01.060116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:01.060296Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:01.060307Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:01.060320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:01.060388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:01.060431Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342591618883813:3795]
2025-10-09T21:50:01.061087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:01.061400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:01.061453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:01.061832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:01.061913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.061921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.061923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.061924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.061925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.061926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.061928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.061929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.061931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.061936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.061946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.061950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.061954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.061957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.061960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.061964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.061968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.061971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.061979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.061987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.061995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.062000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.062004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.062009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.062014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.062020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.062024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.062054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.062059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.062064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.062075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.062081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.062091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.062097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.062101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.062110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.062139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:01.068565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:01.070178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:01.070193Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:01.070195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:01.070200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:01.070211Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1693 [2:7559342591618883813:3795]
2025-10-09T21:50:01.070226Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:01.070247Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:01.070250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:01.070254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:01.070257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:01.070276Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:01.070287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:01.070295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:01.070364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.070372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.070374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.070376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:01.070674Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1693
2025-10-09T21:50:01.070674Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1693
2025-10-09T21:50:01.070740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:01.070749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.070763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.070778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.070803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:01.070828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342591980713728:6953], now have 1 active actors on pipe
2025-10-09T21:50:01.070837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342591980713453:2432], now have 1 active actors on pipe
2025-10-09T21:50:01.070859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.070868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.070888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342591618883808:3795]
2025-10-09T21:50:01.070895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342591618883808:3795]
2025-10-09T21:50:01.070897Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342591618883808:3795]
2025-10-09T21:50:01.070911Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:01.070923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:01.070930Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:01.070932Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:01.070933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:01.070948Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:01.070952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:01.070975Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3d3234e6-61a8df8f-2c938ea8-32c4d9a7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:01.070987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:01.070993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:01.070995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:01.071006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:01.071021Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:01.071065Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342591618883808:3795]
2025-10-09T21:50:01.071079Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:01.071093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:01.071098Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:01.071328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.071338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.071357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:01.071458Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1694 partition: 0 MaxSeqNo: 25 sessionId: src_id|3d3234e6-61a8df8f-2c938ea8-32c4d9a7_0
2025-10-09T21:50:01.071889Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2fd02ed0-4a1a657b-d2be1c26-5811bd1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3d3234e6-61a8df8f-2c938ea8-32c4d9a7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:01.074356Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3234e6-61a8df8f-2c938ea8-32c4d9a7_0] Write session: try to update token
2025-10-09T21:50:01.074373Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3234e6-61a8df8f-2c938ea8-32c4d9a7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:01.087401Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1694 sessionId: src_id|3d3234e6-61a8df8f-2c938ea8-32c4d9a7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:01.088814Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:01.088833Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:01.088843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342591618883808:3795]
2025-10-09T21:50:01.088893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:01.088900Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:01.089089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:01.089254Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:01.089402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:01.089575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:01.089811Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:01.089866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:01.090389Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:01.090597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.090615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.090654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:01.090742Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:01.096683Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:01.096703Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:01.096705Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:01.096707Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:01.096721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342591618883808:3795]
2025-10-09T21:50:01.096725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342591618883808:3795]
2025-10-09T21:50:01.096729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342591618883808:3795]
2025-10-09T21:50:01.096773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:01.096775Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:01.096782Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:01.096783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.096785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:01.096787Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:01.096933Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:01.137201Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342591980713744:7774]
2025-10-09T21:50:01.137508Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 1693, pipe: [1:7559342591980713744:7774]
2025-10-09T21:50:01.247664Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:01.254643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.254668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.254815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.254872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.254940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.255003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.255044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.255052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.255114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.255169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.255224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.255280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.255321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.255326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.255383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.255436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.255497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.255561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.255600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.255606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.255665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.255721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.255775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.255831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.255871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.255877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.255936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.255992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.256047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.256101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.256142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.256147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.256202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.256256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.256311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.256370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.256410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.256416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.256471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.256535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.256604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.256670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.256712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.256718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.256773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.256822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.256877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.256931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.256970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.256976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.257028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.257085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.257150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.257220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.257260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.257267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.257342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.257406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.257462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.257531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.257573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.257579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.257617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342591980713744:7774], now have 1 active actors on pipe
2025-10-09T21:50:01.257784Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342591980713744:7774]
2025-10-09T21:50:01.257789Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:01.257792Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:01.257815Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:01.257818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1694 reason: error at write: Disconnected sessionId: src_id|3d3234e6-61a8df8f-2c938ea8-32c4d9a7_0
2025-10-09T21:50:01.258001Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1694 sessionId: src_id|3d3234e6-61a8df8f-2c938ea8-32c4d9a7_0 is DEAD
2025-10-09T21:50:01.258175Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3234e6-61a8df8f-2c938ea8-32c4d9a7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:01.258186Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3234e6-61a8df8f-2c938ea8-32c4d9a7_0] Write session will restart in 0.007954s
2025-10-09T21:50:01.258209Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3234e6-61a8df8f-2c938ea8-32c4d9a7_0] Write session: Do CDS request
2025-10-09T21:50:01.258217Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3234e6-61a8df8f-2c938ea8-32c4d9a7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:01.259072Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:01.267464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3234e6-61a8df8f-2c938ea8-32c4d9a7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:01.267552Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:01.267567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1695
2025-10-09T21:50:01.267698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1695 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:01.267731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1695 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:01.267743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1695 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:01.267747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:01.268174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1695 sessionId:  describe result for acl check
2025-10-09T21:50:01.268208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:01.268210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:01.268212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:01.268217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342591980713865:6939] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:01.268733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342591980713865:6939] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:01.270995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342591980713865:6939] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:01.274075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342591980713865:6939] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:01.274090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342591980713865:6939] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:01.274094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342591980713865:6939] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:01.274111Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1695 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:01.275351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:01.278613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:01.278804Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:01.278815Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:01.278826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:01.278873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:01.278912Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342591618884123:3768]
2025-10-09T21:50:01.279500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:01.279823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:01.279866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:01.280187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:01.280234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.280239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.280241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.280242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.280243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.280244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.280246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.280247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.280248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.280253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.280263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.280267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.280272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.280276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.280280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.280284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.280288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.280292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.280298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.280306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.280312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.280318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.280323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.280329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.280333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.280338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.280342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.280347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.280352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.280357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.280361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.280367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.280371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.280376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.280381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.280386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.280417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:01.286788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:01.288313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:01.288328Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:01.288331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:01.288335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:01.288342Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1694 [2:7559342591618884123:3768]
2025-10-09T21:50:01.288356Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:01.288374Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:01.288377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:01.288380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:01.288382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:01.288400Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:01.288409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:01.288416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:01.288477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.288485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.288487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.288489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:01.288565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:01.288572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.288585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.288595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.288615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:01.288641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342591980713774:2432], now have 1 active actors on pipe
2025-10-09T21:50:01.288649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342591980714053:6939], now have 1 active actors on pipe
2025-10-09T21:50:01.288657Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342591618884118:3768]
2025-10-09T21:50:01.288660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342591618884118:3768]
2025-10-09T21:50:01.288662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342591618884118:3768]
2025-10-09T21:50:01.288677Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:01.288697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:01.288702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:01.288705Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:01.288708Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:01.288713Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:01.288716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:01.288750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342591618884118:3768]
2025-10-09T21:50:01.288758Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:01.288760Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:01.288721Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1694
2025-10-09T21:50:01.288794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.288804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.288726Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1694
2025-10-09T21:50:01.288837Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|51fec948-8607e2a6-3d3c6338-6db8a06_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:01.288850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:01.288854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:01.288857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:01.288868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:01.288883Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:01.288900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:01.289067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.289076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.289092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:01.289200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1695 partition: 0 MaxSeqNo: 25 sessionId: src_id|51fec948-8607e2a6-3d3c6338-6db8a06_0
2025-10-09T21:50:01.289555Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3234e6-61a8df8f-2c938ea8-32c4d9a7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|51fec948-8607e2a6-3d3c6338-6db8a06_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:01.291988Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51fec948-8607e2a6-3d3c6338-6db8a06_0] Write session: try to update token
2025-10-09T21:50:01.292007Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51fec948-8607e2a6-3d3c6338-6db8a06_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:01.303019Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:01.303035Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:01.303037Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:01.303038Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:01.303047Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342591618884118:3768]
2025-10-09T21:50:01.303050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342591618884118:3768]
2025-10-09T21:50:01.303053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342591618884118:3768]
2025-10-09T21:50:01.303095Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:01.303100Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:01.303103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:01.303103Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:01.303111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.303114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:01.303264Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:01.305272Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1695 sessionId: src_id|51fec948-8607e2a6-3d3c6338-6db8a06_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:01.307758Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:01.357812Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342591980714068:7774]
2025-10-09T21:50:01.358076Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 1694, pipe: [1:7559342591980714068:7774]
2025-10-09T21:50:01.454679Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:01.455346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.455363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.455404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342591980714068:7774], now have 1 active actors on pipe
2025-10-09T21:50:01.455621Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:01.455634Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342591980714068:7774]
2025-10-09T21:50:01.455658Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:01.455660Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:01.455729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1695 reason: error at write: Disconnected sessionId: src_id|51fec948-8607e2a6-3d3c6338-6db8a06_0
2025-10-09T21:50:01.455890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1695 sessionId: src_id|51fec948-8607e2a6-3d3c6338-6db8a06_0 is DEAD
2025-10-09T21:50:01.455964Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:01.456064Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51fec948-8607e2a6-3d3c6338-6db8a06_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:01.456074Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51fec948-8607e2a6-3d3c6338-6db8a06_0] Write session will restart in 0.005009s
2025-10-09T21:50:01.456101Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51fec948-8607e2a6-3d3c6338-6db8a06_0] Write session: Do CDS request
2025-10-09T21:50:01.456108Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51fec948-8607e2a6-3d3c6338-6db8a06_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:01.462406Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51fec948-8607e2a6-3d3c6338-6db8a06_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:01.462502Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:01.462517Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1696
2025-10-09T21:50:01.462620Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1696 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:01.462653Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1696 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:01.462663Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1696 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:01.462667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:01.463063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1696 sessionId:  describe result for acl check
2025-10-09T21:50:01.463097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:01.463099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:01.463100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:01.463105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342591980714090:6931] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:01.463647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342591980714090:6931] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:01.465871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342591980714090:6931] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:01.468701Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342591980714090:6931] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:01.468714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342591980714090:6931] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:01.468718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342591980714090:6931] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:01.468724Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1696 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:01.473435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:01.476663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:01.476824Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:01.476833Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:01.476841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:01.476879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:01.476908Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342591618884395:3756]
2025-10-09T21:50:01.477320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:01.477596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:01.477629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:01.477887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:01.477920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.477925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.477925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.477926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.477927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.477927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.477928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.477929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.477930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.477933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.477940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.477942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.477944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.477946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.477948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.477950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.477952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.477954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.477960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.477966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.477970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.477973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.477976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.477979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.477982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.477985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.477988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.477990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.477993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.477996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.477999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.478002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.478004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.478007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.478010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.478013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.478046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:01.484129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:01.485466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:01.485477Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:01.485478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:01.485482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:01.485487Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1695 [2:7559342591618884395:3756]
2025-10-09T21:50:01.485498Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:01.485512Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:01.485525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:01.485528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:01.485529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:01.485545Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:01.485556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:01.485563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:01.485607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.485616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.485618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.485620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:01.485685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:01.485692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.485703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.485714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.485729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:01.485750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342591980714243:6931], now have 1 active actors on pipe
2025-10-09T21:50:01.485834Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1695
2025-10-09T21:50:01.485759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342591980714070:2432], now have 1 active actors on pipe
2025-10-09T21:50:01.485766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342591618884390:3756]
2025-10-09T21:50:01.485769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342591618884390:3756]
2025-10-09T21:50:01.485771Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342591618884390:3756]
2025-10-09T21:50:01.485784Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:01.485800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:01.485813Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:01.485816Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:01.485817Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:01.485817Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:01.485820Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:01.485848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342591618884390:3756]
2025-10-09T21:50:01.485921Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1695
2025-10-09T21:50:01.485851Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:01.485861Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:01.485884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.485889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.485919Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|86cb31cb-2ad5acb-8cb7b91a-513884d1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:01.485927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:01.485931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:01.485934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:01.485944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:01.485957Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:01.485976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:01.486118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.486126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.486137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:01.486264Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1696 partition: 0 MaxSeqNo: 25 sessionId: src_id|86cb31cb-2ad5acb-8cb7b91a-513884d1_0
2025-10-09T21:50:01.486617Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51fec948-8607e2a6-3d3c6338-6db8a06_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|86cb31cb-2ad5acb-8cb7b91a-513884d1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:01.489071Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86cb31cb-2ad5acb-8cb7b91a-513884d1_0] Write session: try to update token
2025-10-09T21:50:01.489087Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86cb31cb-2ad5acb-8cb7b91a-513884d1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:01.496842Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:01.496861Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:01.496870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342591618884390:3756]
2025-10-09T21:50:01.496908Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:01.496909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:01.497060Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:01.497189Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:01.497327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:01.497460Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:01.497666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:01.497712Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:01.501632Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1696 sessionId: src_id|86cb31cb-2ad5acb-8cb7b91a-513884d1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:01.504104Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:01.504303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.504327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.504363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:01.504459Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:01.509969Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:01.509984Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:01.509986Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:01.509988Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:01.509997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342591618884390:3756]
2025-10-09T21:50:01.510000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342591618884390:3756]
2025-10-09T21:50:01.510003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342591618884390:3756]
2025-10-09T21:50:01.510059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:01.510059Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:01.510065Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:01.510066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.510068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:01.510072Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:01.510194Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:01.555967Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342591980714362:7774]
2025-10-09T21:50:01.556227Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 1695, pipe: [1:7559342591980714362:7774]
2025-10-09T21:50:01.662491Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:01.670409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.670432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.670604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.670665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.670730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.670794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.670839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.670849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.670921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.670986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.671050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.671112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.671156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.671165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.671222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.671284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.671339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.671398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.671438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.671446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.671510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.671567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.671623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.671685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.671726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.671733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.671797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.671854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.671912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.671974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.672043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.672053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.672120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.672177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.672237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.672299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.672356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.672362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.672422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.672481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.672540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.672592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.672632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.672634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.672690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.672745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.672796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.672851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.672889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.672894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.672950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.673009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.673060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.673120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.673176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.673183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.673246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.673299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.673350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.673412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.673457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.673466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.673537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342591980714362:7774], now have 1 active actors on pipe
2025-10-09T21:50:01.673781Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:01.673786Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342591980714362:7774]
2025-10-09T21:50:01.673796Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:01.673808Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:01.673826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1696 reason: error at write: Disconnected sessionId: src_id|86cb31cb-2ad5acb-8cb7b91a-513884d1_0
2025-10-09T21:50:01.674023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1696 sessionId: src_id|86cb31cb-2ad5acb-8cb7b91a-513884d1_0 is DEAD
2025-10-09T21:50:01.674204Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86cb31cb-2ad5acb-8cb7b91a-513884d1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:01.674213Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86cb31cb-2ad5acb-8cb7b91a-513884d1_0] Write session will restart in 0.007654s
2025-10-09T21:50:01.674242Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86cb31cb-2ad5acb-8cb7b91a-513884d1_0] Write session: Do CDS request
2025-10-09T21:50:01.674249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86cb31cb-2ad5acb-8cb7b91a-513884d1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:01.674446Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:01.683533Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86cb31cb-2ad5acb-8cb7b91a-513884d1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:01.683632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:01.683649Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1697
2025-10-09T21:50:01.683796Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1697 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:01.683830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1697 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:01.683837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1697 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:01.683840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:01.684359Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1697 sessionId:  describe result for acl check
2025-10-09T21:50:01.684389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:01.684391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:01.684392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:01.684396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342591980714440:6919] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:01.684846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342591980714440:6919] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:01.688153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342591980714440:6919] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:01.691542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342591980714440:6919] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:01.691553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342591980714440:6919] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:01.691556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342591980714440:6919] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:01.691565Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1697 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:01.693900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:01.697965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:01.698222Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:01.698232Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:01.698245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:01.698312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:01.698356Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342591618884668:3750]
2025-10-09T21:50:01.698980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:01.699310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:01.699389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:01.699811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:01.699859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.699863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.699864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.699866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.699866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.699867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.699868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.699868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.699870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.699873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.699881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.699883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.699885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.699887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.699889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.699892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.699894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.699896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.699902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.699910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.699914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.699918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.699920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.699923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.699926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.699929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.699933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.699936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.699939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.699942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.699945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.699949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.699951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.699954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.699957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.699960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.699987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:01.706482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:01.707727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:01.707746Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:01.707748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:01.707754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:01.707762Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1696 [2:7559342591618884668:3750]
2025-10-09T21:50:01.707780Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:01.707800Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:01.707803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:01.707807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:01.707809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:01.707828Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:01.707839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:01.707847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:01.707906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.707913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.707914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.707917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:01.707986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:01.707991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.708001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.708007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.708017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:01.708036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342591618884663:3750]
2025-10-09T21:50:01.708041Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342591618884663:3750]
2025-10-09T21:50:01.708043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342591618884663:3750]
2025-10-09T21:50:01.708050Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:01.708059Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:01.708062Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:01.708062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:01.708064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:01.708071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:01.708078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:01.708086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342591980714640:6919], now have 1 active actors on pipe
2025-10-09T21:50:01.708097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342591980714364:2432], now have 1 active actors on pipe
2025-10-09T21:50:01.708141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342591618884663:3750]
2025-10-09T21:50:01.708154Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:01.708179Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:01.708188Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1696
2025-10-09T21:50:01.708189Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1696
2025-10-09T21:50:01.708324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.708340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.708375Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|48523c30-75f0467c-286ba986-b16f2621_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:01.708389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:01.708396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:01.708399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:01.708413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:01.708429Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:01.708447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:01.708696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.708708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.708726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:01.708968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1697 partition: 0 MaxSeqNo: 25 sessionId: src_id|48523c30-75f0467c-286ba986-b16f2621_0
2025-10-09T21:50:01.709333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86cb31cb-2ad5acb-8cb7b91a-513884d1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|48523c30-75f0467c-286ba986-b16f2621_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:01.711886Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48523c30-75f0467c-286ba986-b16f2621_0] Write session: try to update token
2025-10-09T21:50:01.711905Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48523c30-75f0467c-286ba986-b16f2621_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:01.719999Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:01.720019Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:01.720030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342591618884663:3750]
2025-10-09T21:50:01.720078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:01.720077Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:01.720248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:01.720385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:01.720505Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:01.720666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:01.720844Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:01.720876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:01.725183Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1697 sessionId: src_id|48523c30-75f0467c-286ba986-b16f2621_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:01.727513Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:01.727711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.727731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.727772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:01.727862Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:01.732953Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:01.732969Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:01.732972Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:01.732974Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:01.732985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342591618884663:3750]
2025-10-09T21:50:01.732988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342591618884663:3750]
2025-10-09T21:50:01.732992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342591618884663:3750]
2025-10-09T21:50:01.733037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:01.733041Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:01.733044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.733047Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:01.733047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:01.733054Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:01.733178Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:01.774082Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342591980714660:7774]
2025-10-09T21:50:01.774377Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 1696, pipe: [1:7559342591980714660:7774]
2025-10-09T21:50:01.883340Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:01.890660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.890684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.890833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.890887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.890942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.890997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.891039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.891047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.891108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.891174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.891234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.891290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.891332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.891342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.891405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.891456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.891507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.891559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.891597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.891602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.891661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.891717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.891771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.891825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.891863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.891868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.891925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.891989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.892044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.892097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.892136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.892140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.892196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.892248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.892304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.892358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.892400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.892405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.892458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.892507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.892560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.892615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.892652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.892659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.892713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.892764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.892816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.892868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.892907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.892913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.892964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.893009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.893062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.893115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.893169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.893175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.893227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:01.893276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:01.893325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:01.893376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:01.893413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:01.893418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:01.893457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342591980714660:7774], now have 1 active actors on pipe
2025-10-09T21:50:01.893683Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:01.893699Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:01.893707Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:01.893728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1697 reason: error at write: Disconnected sessionId: src_id|48523c30-75f0467c-286ba986-b16f2621_0
2025-10-09T21:50:01.893754Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342591980714660:7774]
2025-10-09T21:50:01.893906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1697 sessionId: src_id|48523c30-75f0467c-286ba986-b16f2621_0 is DEAD
2025-10-09T21:50:01.894061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48523c30-75f0467c-286ba986-b16f2621_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:01.894072Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48523c30-75f0467c-286ba986-b16f2621_0] Write session will restart in 0.005266s
2025-10-09T21:50:01.894096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48523c30-75f0467c-286ba986-b16f2621_0] Write session: Do CDS request
2025-10-09T21:50:01.894102Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48523c30-75f0467c-286ba986-b16f2621_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:01.894390Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:01.900512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48523c30-75f0467c-286ba986-b16f2621_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:01.900637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:01.900652Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1698
2025-10-09T21:50:01.900778Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1698 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:01.900808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1698 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:01.900823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1698 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:01.900827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:01.901262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1698 sessionId:  describe result for acl check
2025-10-09T21:50:01.901293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:01.901296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:01.901297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:01.901302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342591980714692:6913] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:01.901840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342591980714692:6913] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:01.904944Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342591980714692:6913] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:01.908267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342591980714692:6913] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:01.908282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342591980714692:6913] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:01.908285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342591980714692:6913] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:01.908295Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1698 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:01.912494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:01.915889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:01.916093Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:01.916104Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:01.916115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:01.916171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:01.916218Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342591618884947:3735]
2025-10-09T21:50:01.916872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:01.917144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:01.917185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:01.917510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:01.917573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.917579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.917580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.917582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.917583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.917584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.917586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.917587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.917589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.917593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.917605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.917609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.917614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.917618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.917622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.917626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.917630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.917633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.917641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.917654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:01.917661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.917668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:01.917674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.917679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:01.917683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.917689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:01.917695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.917700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:01.917705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.917709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:01.917716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.917722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.917726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.917732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.917738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.917743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.917768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:01.923893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:01.924846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:01.924861Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:01.924862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:01.924865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:01.924870Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1697 [2:7559342591618884947:3735]
2025-10-09T21:50:01.924880Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:01.924898Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:01.924902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:01.924905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:01.924907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:01.924924Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:01.924937Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:01.924942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:01.924990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.924996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.924997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.924999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:01.925050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:01.925055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.925063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:01.925069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:01.925078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:01.925096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342591618884942:3735]
2025-10-09T21:50:01.925104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342591618884942:3735]
2025-10-09T21:50:01.925105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342591618884942:3735]
2025-10-09T21:50:01.925111Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:01.925122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:01.925130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:01.925130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:01.925132Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:01.925136Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:01.925138Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:01.925141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342591980714873:6913], now have 1 active actors on pipe
2025-10-09T21:50:01.925159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342591980714665:2432], now have 1 active actors on pipe
2025-10-09T21:50:01.925161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342591618884942:3735]
2025-10-09T21:50:01.925166Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:01.925181Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:01.925229Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1697
2025-10-09T21:50:01.925230Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1697
2025-10-09T21:50:01.925302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.925310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.925335Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ca39f615-4fb08e6c-33363987-7873df1d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:01.925340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:01.925344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:01.925346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:01.925353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:01.925361Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:01.925372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:01.925490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:01.925495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:01.925505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:01.925618Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1698 partition: 0 MaxSeqNo: 25 sessionId: src_id|ca39f615-4fb08e6c-33363987-7873df1d_0
2025-10-09T21:50:01.925946Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48523c30-75f0467c-286ba986-b16f2621_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ca39f615-4fb08e6c-33363987-7873df1d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:01.928326Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca39f615-4fb08e6c-33363987-7873df1d_0] Write session: try to update token
2025-10-09T21:50:01.928344Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca39f615-4fb08e6c-33363987-7873df1d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:01.938938Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:01.938953Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:01.938955Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:01.938957Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:01.938965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342591618884942:3735]
2025-10-09T21:50:01.938968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342591618884942:3735]
2025-10-09T21:50:01.938971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342591618884942:3735]
2025-10-09T21:50:01.939011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:01.939014Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:01.939018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:01.939020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:01.939021Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:01.939025Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:01.939141Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:01.940368Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1698 sessionId: src_id|ca39f615-4fb08e6c-33363987-7873df1d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:01.942555Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:01.993733Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342591980714959:7774]
2025-10-09T21:50:01.993975Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 1697, pipe: [1:7559342591980714959:7774]
2025-10-09T21:50:02.087652Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:02.088256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:02.088271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:02.088308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342591980714959:7774], now have 1 active actors on pipe
2025-10-09T21:50:02.088503Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342591980714959:7774]
2025-10-09T21:50:02.088522Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:02.088526Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:02.088528Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:02.088583Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1698 reason: error at write: Disconnected sessionId: src_id|ca39f615-4fb08e6c-33363987-7873df1d_0
2025-10-09T21:50:02.088780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1698 sessionId: src_id|ca39f615-4fb08e6c-33363987-7873df1d_0 is DEAD
2025-10-09T21:50:02.088949Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca39f615-4fb08e6c-33363987-7873df1d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:02.088959Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca39f615-4fb08e6c-33363987-7873df1d_0] Write session will restart in 0.005408s
2025-10-09T21:50:02.088977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca39f615-4fb08e6c-33363987-7873df1d_0] Write session: Do CDS request
2025-10-09T21:50:02.088985Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca39f615-4fb08e6c-33363987-7873df1d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:02.089021Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:02.095494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca39f615-4fb08e6c-33363987-7873df1d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:02.095592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:02.095607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1699
2025-10-09T21:50:02.095731Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1699 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:02.095764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1699 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:02.095769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1699 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:02.095777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:02.096200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1699 sessionId:  describe result for acl check
2025-10-09T21:50:02.096242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:02.096245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:02.096247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:02.096253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342596275682283:6893] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:02.096839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342596275682283:6893] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:02.099872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342596275682283:6893] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:02.103608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342596275682283:6893] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:02.103623Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342596275682283:6893] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:02.103627Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342596275682283:6893] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:02.103637Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1699 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:02.111980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:02.112572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:02.112763Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:02.112777Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:02.112789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:02.112840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:02.112887Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342595913852518:3743]
2025-10-09T21:50:02.113444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:02.113761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:02.113812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:02.114142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:02.114199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.114205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.114207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.114208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.114209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.114211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.114213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.114214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.114215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.114220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.114232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.114236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.114240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.114244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.114249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.114253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.114257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.114261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.114268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.114276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.114281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.114287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.114292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.114296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.114300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.114306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.114312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.114317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.114322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.114328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.114333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.114338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.114342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.114348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.114353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.114358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.114394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:02.121077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:02.122236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:02.122254Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:02.122257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:02.122262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:02.122271Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1698 [2:7559342595913852518:3743]
2025-10-09T21:50:02.122287Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:02.122307Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:02.122310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:02.122314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:02.122317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:02.122337Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:02.122351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:02.122359Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:02.122424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.122432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.122435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.122438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:02.122512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:02.122521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.122536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.122549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.122567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:02.122597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342596275682467:6893], now have 1 active actors on pipe
2025-10-09T21:50:02.122601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342596275682260:2432], now have 1 active actors on pipe
2025-10-09T21:50:02.122609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342595913852513:3743]
2025-10-09T21:50:02.122611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342595913852513:3743]
2025-10-09T21:50:02.122614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342595913852513:3743]
2025-10-09T21:50:02.122626Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:02.122644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:02.122644Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:02.122647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:02.122649Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:02.122664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:02.122668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:02.122707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342595913852513:3743]
2025-10-09T21:50:02.122682Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1698
2025-10-09T21:50:02.122719Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:02.122681Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1698
2025-10-09T21:50:02.122721Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:02.122739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:02.122744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:02.122800Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3d0cc0c3-e54964e9-99ce90d3-9d34e02b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:02.122813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:02.122818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:02.122821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:02.122831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:02.122842Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:02.122869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:02.123064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:02.123075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:02.123092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:02.123205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1699 partition: 0 MaxSeqNo: 25 sessionId: src_id|3d0cc0c3-e54964e9-99ce90d3-9d34e02b_0
2025-10-09T21:50:02.123587Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca39f615-4fb08e6c-33363987-7873df1d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3d0cc0c3-e54964e9-99ce90d3-9d34e02b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:02.125933Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d0cc0c3-e54964e9-99ce90d3-9d34e02b_0] Write session: try to update token
2025-10-09T21:50:02.125950Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d0cc0c3-e54964e9-99ce90d3-9d34e02b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:02.137479Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:02.137497Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:02.137499Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:02.137502Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:02.137511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342595913852513:3743]
2025-10-09T21:50:02.137528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342595913852513:3743]
2025-10-09T21:50:02.137532Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342595913852513:3743]
2025-10-09T21:50:02.137580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:02.137584Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:02.137588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.137590Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:02.137591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:02.137594Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:02.137735Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:02.138683Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1699 sessionId: src_id|3d0cc0c3-e54964e9-99ce90d3-9d34e02b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:02.141189Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:02.188801Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342596275682558:7774]
2025-10-09T21:50:02.189013Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 1698, pipe: [1:7559342596275682558:7774]
2025-10-09T21:50:02.287069Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:02.287913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:02.287928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:02.287974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342596275682558:7774], now have 1 active actors on pipe
2025-10-09T21:50:02.288107Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:02.288232Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:02.288347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1699 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3d0cc0c3-e54964e9-99ce90d3-9d34e02b_0
2025-10-09T21:50:02.288531Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1699 sessionId: src_id|3d0cc0c3-e54964e9-99ce90d3-9d34e02b_0 is DEAD
2025-10-09T21:50:02.288682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d0cc0c3-e54964e9-99ce90d3-9d34e02b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:02.288692Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d0cc0c3-e54964e9-99ce90d3-9d34e02b_0] Write session will restart in 0.005858s
2025-10-09T21:50:02.288709Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d0cc0c3-e54964e9-99ce90d3-9d34e02b_0] Write session: Do CDS request
2025-10-09T21:50:02.288714Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d0cc0c3-e54964e9-99ce90d3-9d34e02b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:02.288832Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342596275682558:7774]
2025-10-09T21:50:02.288855Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:02.295486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d0cc0c3-e54964e9-99ce90d3-9d34e02b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:02.295600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:02.295614Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1700
2025-10-09T21:50:02.295710Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1700 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:02.295752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1700 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:02.295757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1700 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:02.295760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:02.296233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1700 sessionId:  describe result for acl check
2025-10-09T21:50:02.296272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:02.296278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:02.296279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:02.296282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342596275682615:6886] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:02.296712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342596275682615:6886] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:02.299493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342596275682615:6886] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:02.302837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342596275682615:6886] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:02.302852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342596275682615:6886] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:02.302855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342596275682615:6886] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:02.302862Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1700 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:02.310563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:02.311149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:02.311343Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:02.311355Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:02.311367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:02.311418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:02.311460Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342595913852833:3722]
2025-10-09T21:50:02.312068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:02.312355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:02.312396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:02.312729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:02.312798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.312804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.312805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.312807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.312808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.312809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.312810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.312812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.312813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.312819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.312831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.312835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.312839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.312844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.312848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.312852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.312856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.312860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.312867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.312875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.312882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.312888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.312893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.312898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.312903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.312908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.312919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.312929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.312933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.312938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.312945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.312949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.312952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.312955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.312959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.312962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.312979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:02.319435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:02.320516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:02.320535Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:02.320537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:02.320542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:02.320550Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1699 [2:7559342595913852833:3722]
2025-10-09T21:50:02.320566Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:02.320587Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:02.320590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:02.320594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:02.320596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:02.320614Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:02.320625Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:02.320638Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:02.320703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.320712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.320714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.320717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:02.320790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:02.320798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.320811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.320827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.320851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:02.320874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342596275682800:6886], now have 1 active actors on pipe
2025-10-09T21:50:02.320883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342596275682578:2432], now have 1 active actors on pipe
2025-10-09T21:50:02.320892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342595913852828:3722]
2025-10-09T21:50:02.320899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342595913852828:3722]
2025-10-09T21:50:02.320902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342595913852828:3722]
2025-10-09T21:50:02.320913Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:02.320925Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:02.320928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:02.320935Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:02.320937Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:02.320943Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:02.320946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:02.320938Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1699
2025-10-09T21:50:02.320985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342595913852828:3722]
2025-10-09T21:50:02.320997Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:02.321003Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:02.321015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:02.321024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:02.321074Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1bf3990-6d9e6c1d-f0ba0cc8-fdf9246c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:02.321086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:02.321091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:02.321097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:02.321109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:02.321118Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:02.321143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:02.321309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:02.321317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:02.321332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:02.321351Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1699
2025-10-09T21:50:02.321484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1700 partition: 0 MaxSeqNo: 25 sessionId: src_id|1bf3990-6d9e6c1d-f0ba0cc8-fdf9246c_0
2025-10-09T21:50:02.321850Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d0cc0c3-e54964e9-99ce90d3-9d34e02b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1bf3990-6d9e6c1d-f0ba0cc8-fdf9246c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:02.324316Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1bf3990-6d9e6c1d-f0ba0cc8-fdf9246c_0] Write session: try to update token
2025-10-09T21:50:02.324336Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1bf3990-6d9e6c1d-f0ba0cc8-fdf9246c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:02.335255Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:02.335276Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:02.335278Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:02.335280Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:02.335290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342595913852828:3722]
2025-10-09T21:50:02.335295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342595913852828:3722]
2025-10-09T21:50:02.335299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342595913852828:3722]
2025-10-09T21:50:02.335341Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:02.335345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:02.335352Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:02.335353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.335357Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:02.335356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:02.335502Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:02.338096Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1700 sessionId: src_id|1bf3990-6d9e6c1d-f0ba0cc8-fdf9246c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:02.340404Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:02.388985Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342596275682882:7774]
2025-10-09T21:50:02.389252Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 1699, pipe: [1:7559342596275682882:7774]
2025-10-09T21:50:02.484497Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:02.485509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:02.485534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:02.485573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342596275682882:7774], now have 1 active actors on pipe
2025-10-09T21:50:02.485781Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:02.485818Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:02.485821Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342596275682882:7774]
2025-10-09T21:50:02.485837Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:02.485837Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:02.485913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1700 reason: error at write: Disconnected sessionId: src_id|1bf3990-6d9e6c1d-f0ba0cc8-fdf9246c_0
2025-10-09T21:50:02.486096Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1700 sessionId: src_id|1bf3990-6d9e6c1d-f0ba0cc8-fdf9246c_0 is DEAD
2025-10-09T21:50:02.486299Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bf3990-6d9e6c1d-f0ba0cc8-fdf9246c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:02.486312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bf3990-6d9e6c1d-f0ba0cc8-fdf9246c_0] Write session will restart in 0.005102s
2025-10-09T21:50:02.486339Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bf3990-6d9e6c1d-f0ba0cc8-fdf9246c_0] Write session: Do CDS request
2025-10-09T21:50:02.486350Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bf3990-6d9e6c1d-f0ba0cc8-fdf9246c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:02.492459Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1bf3990-6d9e6c1d-f0ba0cc8-fdf9246c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:02.492584Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:02.492598Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1701
2025-10-09T21:50:02.492748Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1701 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:02.492783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1701 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:02.492806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1701 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:02.492816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:02.493245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1701 sessionId:  describe result for acl check
2025-10-09T21:50:02.493287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:02.493295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:02.493298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:02.493304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342596275682899:6883] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:02.493857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342596275682899:6883] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:02.496306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342596275682899:6883] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:02.500161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342596275682899:6883] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:02.500176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342596275682899:6883] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:02.500180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342596275682899:6883] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:02.500188Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1701 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:02.505483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:02.509626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:02.509864Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:02.509873Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:02.509884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:02.509929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:02.509968Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342595913853110:3711]
2025-10-09T21:50:02.510432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:02.510728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:02.510760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:02.511002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:02.511043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.511048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.511049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.511049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.511050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.511051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.511051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.511052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.511053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.511056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.511063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.511066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.511068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.511070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.511072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.511074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.511076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.511079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.511085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.511091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.511097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.511103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.511108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.511114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.511119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.511122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.511125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.511128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.511132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.511135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.511138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.511141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.511143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.511146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.511149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.511152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.511169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:02.517632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:02.519390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:02.519401Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:02.519403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:02.519406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:02.519412Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1700 [2:7559342595913853110:3711]
2025-10-09T21:50:02.519422Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:02.519441Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:02.519449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:02.519453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:02.519456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:02.519471Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:02.519481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:02.519491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:02.519537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.519546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.519548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.519551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:02.519616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:02.519622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.519630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.519640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.519653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:02.519672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342595913853105:3711]
2025-10-09T21:50:02.519677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342595913853105:3711]
2025-10-09T21:50:02.519679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342595913853105:3711]
2025-10-09T21:50:02.519685Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:02.519696Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:02.519697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342596275682884:2432], now have 1 active actors on pipe
2025-10-09T21:50:02.519700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342596275683062:6883], now have 1 active actors on pipe
2025-10-09T21:50:02.519705Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:02.519707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:02.519708Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:02.519715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:02.519717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:02.519744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342595913853105:3711]
2025-10-09T21:50:02.519753Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:02.519762Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:02.519771Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1700
2025-10-09T21:50:02.519813Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1700
2025-10-09T21:50:02.519867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:02.519874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:02.519897Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|23cf701a-5c22ae16-f568df8d-b6a8b2ac_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:02.519906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:02.519913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:02.519915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:02.519922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:02.519933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:02.519947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:02.520102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:02.520113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:02.520134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:02.520218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1701 partition: 0 MaxSeqNo: 25 sessionId: src_id|23cf701a-5c22ae16-f568df8d-b6a8b2ac_0
2025-10-09T21:50:02.520534Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bf3990-6d9e6c1d-f0ba0cc8-fdf9246c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|23cf701a-5c22ae16-f568df8d-b6a8b2ac_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:02.526942Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23cf701a-5c22ae16-f568df8d-b6a8b2ac_0] Write session: try to update token
2025-10-09T21:50:02.526958Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23cf701a-5c22ae16-f568df8d-b6a8b2ac_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:02.536164Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:02.536179Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:02.536188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342595913853105:3711]
2025-10-09T21:50:02.536230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:02.536250Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:02.536534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:02.536675Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:02.536820Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:02.536990Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:02.537168Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:02.537204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:02.540569Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1701 sessionId: src_id|23cf701a-5c22ae16-f568df8d-b6a8b2ac_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:02.542842Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:02.544113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:02.544131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:02.544150Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:02.544152Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:02.544154Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:02.544155Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:02.544161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342595913853105:3711]
2025-10-09T21:50:02.544164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342595913853105:3711]
2025-10-09T21:50:02.544166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342595913853105:3711]
2025-10-09T21:50:02.544210Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:02.544213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:02.544218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.544220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:02.544226Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:02.544230Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:02.544335Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:02.586118Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342596275683181:7774]
2025-10-09T21:50:02.586383Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 1700, pipe: [1:7559342596275683181:7774]
2025-10-09T21:50:02.695260Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:02.696034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:02.696068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342596275683181:7774], now have 1 active actors on pipe
2025-10-09T21:50:02.696171Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:02.696227Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342596275683181:7774]
2025-10-09T21:50:02.696249Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:02.696257Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:02.696259Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:02.696288Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:50:02.696336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1701 reason: error at write: Disconnected sessionId: src_id|23cf701a-5c22ae16-f568df8d-b6a8b2ac_0
2025-10-09T21:50:02.696370Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1701 sessionId: src_id|23cf701a-5c22ae16-f568df8d-b6a8b2ac_0 is DEAD
2025-10-09T21:50:02.696405Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 17 consumer shared/user session shared/user_1_17_11548234668147486070_v1 is DEAD
2025-10-09T21:50:02.696498Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342510376311633:7764] disconnected.
2025-10-09T21:50:02.696506Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342510376311633:7764] disconnected; active server actors: 1
2025-10-09T21:50:02.696508Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342510376311633:7764] client user disconnected session shared/user_1_17_11548234668147486070_v1
2025-10-09T21:50:02.696676Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:50:02.696687Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23cf701a-5c22ae16-f568df8d-b6a8b2ac_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:02.696698Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23cf701a-5c22ae16-f568df8d-b6a8b2ac_0] Write session will restart in 0.008077s
2025-10-09T21:50:02.696721Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23cf701a-5c22ae16-f568df8d-b6a8b2ac_0] Write session: Do CDS request
2025-10-09T21:50:02.696727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23cf701a-5c22ae16-f568df8d-b6a8b2ac_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:02.696820Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:50:02.696825Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:50:02.696831Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.008001s
2025-10-09T21:50:02.705457Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:50:02.705575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23cf701a-5c22ae16-f568df8d-b6a8b2ac_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:02.705686Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:02.705694Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:50:02.705699Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 18
2025-10-09T21:50:02.705701Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1702
2025-10-09T21:50:02.705832Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1702 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:02.705864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1702 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:02.705874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1702 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:02.705878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:02.705878Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 18 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:50:02.705935Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:50:02.706012Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 auth for : user
2025-10-09T21:50:02.706238Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 Handle describe topics response
2025-10-09T21:50:02.706264Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 auth is DEAD
2025-10-09T21:50:02.706286Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:50:02.706401Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1702 sessionId:  describe result for acl check
2025-10-09T21:50:02.706435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:02.706441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:02.706443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:02.706448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342596275683243:6865] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:02.706523Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:50:02.706619Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_18_7210191584091053671_v1
2025-10-09T21:50:02.707011Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342596275683247:6869] connected; active server actors: 1
2025-10-09T21:50:02.707048Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559342596275683247:6869] session shared/user_1_18_7210191584091053671_v1
2025-10-09T21:50:02.707063Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:50:02.707082Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:50:02.707088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342596275683243:6865] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:02.707093Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_18_7210191584091053671_v1" (Sender=[1:7559342596275683227:6869], Pipe=[1:7559342596275683247:6869], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:50:02.707096Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:50:02.707105Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:50:02.707112Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_18_7210191584091053671_v1" (Sender=[1:7559342596275683227:6869], Pipe=[1:7559342596275683247:6869], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:50:02.707119Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_18_7210191584091053671_v1" sender [1:7559342596275683227:6869] lock partition 0 for ReadingSession "shared/user_1_18_7210191584091053671_v1" (Sender=[1:7559342596275683227:6869], Pipe=[1:7559342596275683247:6869], Partitions=[], ActiveFamilyCount=1) generation 1 step 18
2025-10-09T21:50:02.707126Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:50:02.707129Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000022s
2025-10-09T21:50:02.707293Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 18 Session: "shared/user_1_18_7210191584091053671_v1" ClientId: "user" PipeClient { RawX1: 7559342596275683247 RawX2: 4503603922344661 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:50:02.707317Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:50:02.709491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342596275683243:6865] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:02.712664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342596275683243:6865] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:02.712677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342596275683243:6865] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:02.712680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342596275683243:6865] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:02.712689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1702 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:02.719463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:02.719904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:02.720077Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:02.720087Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:02.720099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:02.720147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:02.720195Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342595913853389:3708]
2025-10-09T21:50:02.720797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:02.721104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:02.721147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:02.721451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:02.721506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.721522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.721524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.721525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.721527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.721528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.721529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.721531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.721532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.721538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.721550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.721554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.721558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.721562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.721566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.721570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.721574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.721579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.721586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.721594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.721603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.721608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.721614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.721622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.721626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.721631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.721636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.721646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.721650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.721656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.721661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.721670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.721675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.721679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.721683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.721689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.721722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:02.727790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:02.728880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:02.728896Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:02.728899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:02.728903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:02.728909Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1701 [2:7559342595913853389:3708]
2025-10-09T21:50:02.728926Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:02.728943Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:02.728946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:02.728950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:02.728952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:02.728969Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:02.728979Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:02.728986Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:02.729159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.729167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.729169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.729172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:02.729243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:02.729251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.729264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.729281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.729303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:02.729325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342596275683433:6865], now have 1 active actors on pipe
2025-10-09T21:50:02.729334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342596275683184:2432], now have 1 active actors on pipe
2025-10-09T21:50:02.729338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342596275683268:6864], now have 1 active actors on pipe
2025-10-09T21:50:02.729346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342595913853384:3708]
2025-10-09T21:50:02.729348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342595913853384:3708]
2025-10-09T21:50:02.729350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342595913853384:3708]
2025-10-09T21:50:02.729364Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:02.729380Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:02.729381Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:02.729388Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:02.729390Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:02.729396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:02.729399Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:02.729372Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1701
2025-10-09T21:50:02.729372Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1701
2025-10-09T21:50:02.729417Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1701, pipe: [1:7559342596275683268:6864]
2025-10-09T21:50:02.729445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:02.729454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:02.729468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342595913853384:3708]
2025-10-09T21:50:02.729477Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:02.729485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:02.729488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:02.729494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_18_7210191584091053671_v1 on pipe: [1:7559342596275683268:6864]
2025-10-09T21:50:02.729495Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:02.729557Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e5fe8fe2-767b5035-98ad57df-c204208_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:02.729558Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_18_7210191584091053671_v1:1 with generation 1701
2025-10-09T21:50:02.729569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:02.729573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:02.729576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:02.729588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:02.729598Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:02.729619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:02.729629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:02.729631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:02.729647Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_18_7210191584091053671_v1
2025-10-09T21:50:02.729654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:02.729680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:02.729707Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:50:02.729803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:02.729812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:02.729825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:02.729899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1702 partition: 0 MaxSeqNo: 25 sessionId: src_id|e5fe8fe2-767b5035-98ad57df-c204208_0
2025-10-09T21:50:02.730223Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23cf701a-5c22ae16-f568df8d-b6a8b2ac_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e5fe8fe2-767b5035-98ad57df-c204208_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:02.732549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5fe8fe2-767b5035-98ad57df-c204208_0] Write session: try to update token
2025-10-09T21:50:02.732563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5fe8fe2-767b5035-98ad57df-c204208_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:02.733909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:50:02.733942Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:50:02.733952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:02.733959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:02.733962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:02.733970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:50:02.733983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:50:02.734197Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046602729 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:50:02.734221Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:50:02.734262Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 sending to client partition status
2025-10-09T21:50:02.743427Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:02.743444Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:02.743446Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:02.743447Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:02.743456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342595913853384:3708]
2025-10-09T21:50:02.743459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342595913853384:3708]
2025-10-09T21:50:02.743462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342595913853384:3708]
2025-10-09T21:50:02.743496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:02.743501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.743503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:02.743503Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:02.743510Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:02.743514Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:02.743649Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:02.744883Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1702 sessionId: src_id|e5fe8fe2-767b5035-98ad57df-c204208_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:02.747365Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:02.888855Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:02.889617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:02.889634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:02.889836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_18_7210191584091053671_v1
2025-10-09T21:50:02.889852Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:02.889843Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:02.889875Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_18_7210191584091053671_v1
2025-10-09T21:50:02.889850Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:02.889853Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:02.889895Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342596275683268:6864]
2025-10-09T21:50:02.889942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1702 reason: error at write: Disconnected sessionId: src_id|e5fe8fe2-767b5035-98ad57df-c204208_0
2025-10-09T21:50:02.890124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1702 sessionId: src_id|e5fe8fe2-767b5035-98ad57df-c204208_0 is DEAD
2025-10-09T21:50:02.890274Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5fe8fe2-767b5035-98ad57df-c204208_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:02.890286Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5fe8fe2-767b5035-98ad57df-c204208_0] Write session will restart in 0.009221s
2025-10-09T21:50:02.890304Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5fe8fe2-767b5035-98ad57df-c204208_0] Write session: Do CDS request
2025-10-09T21:50:02.890309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5fe8fe2-767b5035-98ad57df-c204208_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:02.900566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5fe8fe2-767b5035-98ad57df-c204208_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:02.900668Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:02.900690Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1703
2025-10-09T21:50:02.900808Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1703 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:02.900840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1703 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:02.900850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1703 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:02.900853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:02.901344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1703 sessionId:  describe result for acl check
2025-10-09T21:50:02.901390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:02.901408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:02.901410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:02.901418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342596275683597:6857] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:02.901907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342596275683597:6857] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:02.904718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342596275683597:6857] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:02.907354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342596275683597:6857] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:02.907374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342596275683597:6857] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:02.907378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342596275683597:6857] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:02.907388Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1703 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:02.913248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:02.913939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:02.914149Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:02.914161Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:02.914174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:02.914230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:02.914275Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342595913853674:3704]
2025-10-09T21:50:02.914938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:02.915284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:02.915332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:02.915632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:02.915674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.915679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.915680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.915681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.915682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.915683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.915684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.915684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.915685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.915689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.915697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.915700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.915702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.915704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.915707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.915709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.915711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.915713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.915719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.915724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:02.915729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.915732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:02.915735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.915738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:02.915741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.915746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:02.915752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.915757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:02.915762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.915767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:02.915772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.915779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.915783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.915789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.915794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.915800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.915828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:02.922563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:02.924106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:02.924125Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:02.924128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:02.924133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:02.924142Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1702 [2:7559342595913853674:3704]
2025-10-09T21:50:02.924159Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:02.924180Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:02.924187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:02.924191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:02.924193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:02.924213Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:02.924228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:02.924241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:02.924314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.924322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.924324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.924328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:02.924402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:02.924412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.924427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:02.924440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:02.924457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:02.924481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342596275683493:2432], now have 1 active actors on pipe
2025-10-09T21:50:02.924489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342596275683777:6857], now have 1 active actors on pipe
2025-10-09T21:50:02.924498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342595913853669:3704]
2025-10-09T21:50:02.924500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342595913853669:3704]
2025-10-09T21:50:02.924503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342595913853669:3704]
2025-10-09T21:50:02.924517Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:02.924531Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:02.924538Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:02.924541Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:02.924543Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:02.924552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:02.924555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:02.924551Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1702
2025-10-09T21:50:02.924560Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1702
2025-10-09T21:50:02.924596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342595913853669:3704]
2025-10-09T21:50:02.924604Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:02.924606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:02.924627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:02.924636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:02.924690Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6b17d139-4eae4673-23000b1-a6320c59_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:02.924703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:02.924708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:02.924710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:02.924722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:02.924736Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:02.924762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:02.924937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:02.924947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:02.924962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:02.925065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1703 partition: 0 MaxSeqNo: 25 sessionId: src_id|6b17d139-4eae4673-23000b1-a6320c59_0
2025-10-09T21:50:02.925423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5fe8fe2-767b5035-98ad57df-c204208_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6b17d139-4eae4673-23000b1-a6320c59_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:02.927851Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b17d139-4eae4673-23000b1-a6320c59_0] Write session: try to update token
2025-10-09T21:50:02.927871Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b17d139-4eae4673-23000b1-a6320c59_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:02.939137Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:02.939159Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:02.939161Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:02.939164Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:02.939173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342595913853669:3704]
2025-10-09T21:50:02.939179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342595913853669:3704]
2025-10-09T21:50:02.939184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342595913853669:3704]
2025-10-09T21:50:02.939223Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:02.939227Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:02.939230Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:02.939234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:02.939242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:02.939245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:02.939394Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:02.941390Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1703 sessionId: src_id|6b17d139-4eae4673-23000b1-a6320c59_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:02.943948Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:02.990109Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342596275683791:6864]
2025-10-09T21:50:02.990401Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 1702, pipe: [1:7559342596275683791:6864]
2025-10-09T21:50:03.086538Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:03.087217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:03.087236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:03.087281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342596275683791:6864], now have 1 active actors on pipe
2025-10-09T21:50:03.087461Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:03.087474Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342596275683791:6864]
2025-10-09T21:50:03.087483Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:03.087487Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:03.087505Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:03.087558Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1703 reason: error at write: Disconnected sessionId: src_id|6b17d139-4eae4673-23000b1-a6320c59_0
2025-10-09T21:50:03.087735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1703 sessionId: src_id|6b17d139-4eae4673-23000b1-a6320c59_0 is DEAD
2025-10-09T21:50:03.087878Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b17d139-4eae4673-23000b1-a6320c59_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:03.087888Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b17d139-4eae4673-23000b1-a6320c59_0] Write session will restart in 0.005534s
2025-10-09T21:50:03.087909Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b17d139-4eae4673-23000b1-a6320c59_0] Write session: Do CDS request
2025-10-09T21:50:03.087914Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b17d139-4eae4673-23000b1-a6320c59_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:03.094561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b17d139-4eae4673-23000b1-a6320c59_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:03.094617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:03.094632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1704
2025-10-09T21:50:03.094755Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1704 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:03.094781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1704 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:03.094786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1704 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:03.094789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:03.095351Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1704 sessionId:  describe result for acl check
2025-10-09T21:50:03.095387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:03.095393Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:03.095395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:03.095402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342600570651119:6844] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:03.095897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342600570651119:6844] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:03.098704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342600570651119:6844] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:03.102573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342600570651119:6844] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:03.102586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342600570651119:6844] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:03.102588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342600570651119:6844] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:03.102596Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1704 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:03.106945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:03.110331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:03.110547Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:03.110559Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:03.110571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:03.110625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:03.110670Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342600208821250:3702]
2025-10-09T21:50:03.111293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:03.111588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:03.111633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:03.111959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:03.112009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.112015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.112016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.112017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.112019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.112020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.112022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.112023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.112025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.112029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.112041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.112045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.112050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.112054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.112058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.112062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.112066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.112070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.112078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.112085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.112092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.112098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.112103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.112108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.112113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.112118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.112125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.112131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.112136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.112141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.112148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.112154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.112158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.112163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.112169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.112175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.112210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:03.118551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:03.119603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:03.119618Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:03.119621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:03.119624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:03.119631Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1703 [2:7559342600208821250:3702]
2025-10-09T21:50:03.119642Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:03.119659Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:03.119661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:03.119663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:03.119665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:03.119679Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:03.119685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:03.119692Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:03.119741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.119747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.119749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.119751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:03.119814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:03.119822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.119832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.119838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.119849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:03.119868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342600208821245:3702]
2025-10-09T21:50:03.119873Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342600208821245:3702]
2025-10-09T21:50:03.119875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342600208821245:3702]
2025-10-09T21:50:03.119881Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:03.119893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:03.119893Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:03.119896Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:03.119898Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:03.119903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:03.119905Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:03.119915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342600570651090:2432], now have 1 active actors on pipe
2025-10-09T21:50:03.119918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342600570651306:6844], now have 1 active actors on pipe
2025-10-09T21:50:03.119945Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342600208821245:3702]
2025-10-09T21:50:03.119954Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:03.119973Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:03.119995Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1703
2025-10-09T21:50:03.120004Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1703
2025-10-09T21:50:03.120042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:03.120052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:03.120084Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a2e47b91-ea4d0b72-67a633f7-4befaf1f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:03.120094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:03.120098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:03.120100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:03.120109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:03.120119Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:03.120132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:03.120293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:03.120303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:03.120318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:03.120450Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1704 partition: 0 MaxSeqNo: 25 sessionId: src_id|a2e47b91-ea4d0b72-67a633f7-4befaf1f_0
2025-10-09T21:50:03.120787Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b17d139-4eae4673-23000b1-a6320c59_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a2e47b91-ea4d0b72-67a633f7-4befaf1f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:03.123151Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a2e47b91-ea4d0b72-67a633f7-4befaf1f_0] Write session: try to update token
2025-10-09T21:50:03.123169Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a2e47b91-ea4d0b72-67a633f7-4befaf1f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:03.131443Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:03.131462Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:03.131473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342600208821245:3702]
2025-10-09T21:50:03.131518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:03.131527Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:03.131708Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:03.131858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:03.132117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:03.132261Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:03.132461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:03.132503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:03.135402Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1704 sessionId: src_id|a2e47b91-ea4d0b72-67a633f7-4befaf1f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:03.137710Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:03.137914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:03.137934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:03.137973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:03.138067Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:03.144063Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:03.144079Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:03.144082Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:03.144084Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:03.144094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342600208821245:3702]
2025-10-09T21:50:03.144098Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342600208821245:3702]
2025-10-09T21:50:03.144102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342600208821245:3702]
2025-10-09T21:50:03.144143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:03.144150Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:03.144153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.144155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:03.144156Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:03.144160Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:03.144283Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:03.187703Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342600570651393:6864]
2025-10-09T21:50:03.187966Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 1703, pipe: [1:7559342600570651393:6864]
2025-10-09T21:50:03.291675Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:03.299271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:03.299299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:03.299440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:03.299508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:03.299574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:03.299636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:03.300086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:03.300100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:03.300175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:03.300245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:03.300310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:03.300391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:03.300438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:03.300447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:03.300543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:03.300629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:03.300695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:03.300779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:03.300892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:03.300899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:03.300968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:03.301034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:03.301104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:03.301169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:03.301218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:03.301229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:03.301312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:03.301377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:03.301467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:03.301539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:03.301580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:03.301586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:03.301645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:03.301713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:03.301775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:03.301832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:03.301880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:03.301888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:03.301948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:03.302010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:03.302104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:03.302173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:03.302220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:03.302228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:03.302290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:03.302344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:03.302402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:03.302458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:03.302573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:03.302580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:03.302646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:03.302709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:03.302776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:03.302835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:03.302880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:03.302887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:03.302946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:03.303030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:03.303097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:03.303165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:03.303211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:03.303221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:03.303283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342600570651393:6864], now have 1 active actors on pipe
2025-10-09T21:50:03.303472Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:03.303491Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:03.303496Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:03.303525Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1704 reason: error at write: Disconnected sessionId: src_id|a2e47b91-ea4d0b72-67a633f7-4befaf1f_0
2025-10-09T21:50:03.303526Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342600570651393:6864]
2025-10-09T21:50:03.303745Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:03.303788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1704 sessionId: src_id|a2e47b91-ea4d0b72-67a633f7-4befaf1f_0 is DEAD
2025-10-09T21:50:03.303864Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2e47b91-ea4d0b72-67a633f7-4befaf1f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:03.303876Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2e47b91-ea4d0b72-67a633f7-4befaf1f_0] Write session will restart in 0.009878s
2025-10-09T21:50:03.303900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2e47b91-ea4d0b72-67a633f7-4befaf1f_0] Write session: Do CDS request
2025-10-09T21:50:03.303906Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2e47b91-ea4d0b72-67a633f7-4befaf1f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:03.314545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a2e47b91-ea4d0b72-67a633f7-4befaf1f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:03.314582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:03.314596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1705
2025-10-09T21:50:03.314796Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1705 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:03.314853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1705 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:03.314863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1705 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:03.314867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:03.315348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1705 sessionId:  describe result for acl check
2025-10-09T21:50:03.315384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:03.315385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:03.315386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:03.315391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342600570651511:6838] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:03.315889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342600570651511:6838] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:03.318526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342600570651511:6838] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:03.320841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342600570651511:6838] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:03.320857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342600570651511:6838] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:03.320860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342600570651511:6838] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:03.320878Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1705 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:03.326189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:03.326765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:03.326951Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:03.326960Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:03.326971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:03.327020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:03.327060Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342600208821545:3690]
2025-10-09T21:50:03.327658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:03.327940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:03.327983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:03.328286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:03.328329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.328336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.328338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.328339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.328340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.328341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.328342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.328343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.328344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.328348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.328358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.328362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.328367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.328371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.328375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.328379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.328383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.328387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.328394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.328402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.328408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.328414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.328419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.328425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.328430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.328435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.328444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.328450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.328459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.328466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.328475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.328481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.328490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.328496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.328506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.328515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.328546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:03.334917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:03.336272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:03.336287Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:03.336289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:03.336293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:03.336298Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1704 [2:7559342600208821545:3690]
2025-10-09T21:50:03.336311Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:03.336328Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:03.336333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:03.336336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:03.336339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:03.336356Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:03.336369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:03.336376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:03.336432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.336439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.336441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.336443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:03.336506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:03.336513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.336524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.336535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.336555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:03.336582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342600570651698:6838], now have 1 active actors on pipe
2025-10-09T21:50:03.336586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342600570651400:2432], now have 1 active actors on pipe
2025-10-09T21:50:03.336593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342600208821540:3690]
2025-10-09T21:50:03.336595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342600208821540:3690]
2025-10-09T21:50:03.336597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342600208821540:3690]
2025-10-09T21:50:03.336610Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:03.336627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:03.336639Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:03.336648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:03.336649Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:03.336651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:03.336651Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:03.336642Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1704
2025-10-09T21:50:03.336678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342600208821540:3690]
2025-10-09T21:50:03.336687Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:03.336675Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1704
2025-10-09T21:50:03.336700Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:03.336702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:03.336707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:03.336757Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1325ad7c-cc7469e0-63293261-3fec2261_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:03.336771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:03.336775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:03.336778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:03.336788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:03.336797Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:03.336817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:03.336970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:03.336979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:03.336992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:03.337053Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1705 partition: 0 MaxSeqNo: 25 sessionId: src_id|1325ad7c-cc7469e0-63293261-3fec2261_0
2025-10-09T21:50:03.337391Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2e47b91-ea4d0b72-67a633f7-4befaf1f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1325ad7c-cc7469e0-63293261-3fec2261_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:03.339807Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1325ad7c-cc7469e0-63293261-3fec2261_0] Write session: try to update token
2025-10-09T21:50:03.339822Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1325ad7c-cc7469e0-63293261-3fec2261_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:03.350728Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:03.350743Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:03.350745Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:03.350746Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:03.350752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342600208821540:3690]
2025-10-09T21:50:03.350755Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342600208821540:3690]
2025-10-09T21:50:03.350757Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342600208821540:3690]
2025-10-09T21:50:03.350789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:03.350793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.350795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:03.350797Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:03.350803Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:03.350809Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:03.350910Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:03.352394Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1705 sessionId: src_id|1325ad7c-cc7469e0-63293261-3fec2261_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:03.354545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:03.403778Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342600570651723:6864]
2025-10-09T21:50:03.404037Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 1704, pipe: [1:7559342600570651723:6864]
2025-10-09T21:50:03.497168Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:03.497414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:03.497428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:03.497466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342600570651723:6864], now have 1 active actors on pipe
2025-10-09T21:50:03.497683Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:03.497688Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:03.497693Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:03.497751Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342600570651723:6864]
2025-10-09T21:50:03.497752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1705 reason: error at write: Disconnected sessionId: src_id|1325ad7c-cc7469e0-63293261-3fec2261_0
2025-10-09T21:50:03.497919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1705 sessionId: src_id|1325ad7c-cc7469e0-63293261-3fec2261_0 is DEAD
2025-10-09T21:50:03.497642Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:03.498086Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1325ad7c-cc7469e0-63293261-3fec2261_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:03.498097Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1325ad7c-cc7469e0-63293261-3fec2261_0] Write session will restart in 0.006018s
2025-10-09T21:50:03.498118Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1325ad7c-cc7469e0-63293261-3fec2261_0] Write session: Do CDS request
2025-10-09T21:50:03.498123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1325ad7c-cc7469e0-63293261-3fec2261_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:03.505538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1325ad7c-cc7469e0-63293261-3fec2261_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:03.505616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:03.505638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1706
2025-10-09T21:50:03.505754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1706 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:03.505788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1706 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:03.505806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1706 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:03.505814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:03.506240Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1706 sessionId:  describe result for acl check
2025-10-09T21:50:03.506278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:03.506287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:03.506289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:03.506295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342600570651763:6825] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:03.506847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342600570651763:6825] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:03.510267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342600570651763:6825] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:03.514176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342600570651763:6825] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:03.514189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342600570651763:6825] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:03.514194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342600570651763:6825] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:03.514203Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1706 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:03.518321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:03.521819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:03.522024Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:03.522048Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:03.522061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:03.522115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:03.522160Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342600208821852:3682]
2025-10-09T21:50:03.522772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:03.523026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:03.523067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:03.523383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:03.523449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.523456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.523457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.523458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.523459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.523460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.523462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.523463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.523464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.523470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.523481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.523485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.523488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.523492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.523496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.523499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.523503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.523507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.523515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.523523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.523529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.523534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.523538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.523543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.523547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.523551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.523555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.523560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.523565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.523576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.523583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.523590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.523594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.523600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.523605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.523614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.523646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:03.529964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:03.531491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:03.531501Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:03.531504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:03.531508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:03.531515Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1705 [2:7559342600208821852:3682]
2025-10-09T21:50:03.531529Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:03.531546Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:03.531549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:03.531553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:03.531555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:03.531571Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:03.531582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:03.531590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:03.531652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.531660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.531662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.531666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:03.531733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:03.531742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.531754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.531770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.531795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:03.531817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342600570651959:6825], now have 1 active actors on pipe
2025-10-09T21:50:03.531825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342600570651725:2432], now have 1 active actors on pipe
2025-10-09T21:50:03.531832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342600208821847:3682]
2025-10-09T21:50:03.531835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342600208821847:3682]
2025-10-09T21:50:03.531838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342600208821847:3682]
2025-10-09T21:50:03.531846Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:03.531823Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1705
2025-10-09T21:50:03.531861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:03.531835Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1705
2025-10-09T21:50:03.531867Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:03.531870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:03.531872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:03.531880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:03.531883Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:03.531905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:03.531910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:03.531943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342600208821847:3682]
2025-10-09T21:50:03.531956Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:03.531976Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5bca851-437e24ab-122437bd-e3bcfa8c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:03.531978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:03.531991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:03.531994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:03.531997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:03.532008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:03.532018Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:03.532054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:03.532173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:03.532178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:03.532189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:03.532249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1706 partition: 0 MaxSeqNo: 25 sessionId: src_id|5bca851-437e24ab-122437bd-e3bcfa8c_0
2025-10-09T21:50:03.532609Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1325ad7c-cc7469e0-63293261-3fec2261_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5bca851-437e24ab-122437bd-e3bcfa8c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:03.534879Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5bca851-437e24ab-122437bd-e3bcfa8c_0] Write session: try to update token
2025-10-09T21:50:03.534893Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5bca851-437e24ab-122437bd-e3bcfa8c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:03.546458Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:03.546477Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:03.546480Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:03.546481Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:03.546492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342600208821847:3682]
2025-10-09T21:50:03.546497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342600208821847:3682]
2025-10-09T21:50:03.546502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342600208821847:3682]
2025-10-09T21:50:03.546544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:03.546551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.546554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:03.546567Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:03.546576Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:03.546582Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:03.546683Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:03.547109Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1706 sessionId: src_id|5bca851-437e24ab-122437bd-e3bcfa8c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:03.549277Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:03.597822Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342600570652027:6864]
2025-10-09T21:50:03.598080Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 1705, pipe: [1:7559342600570652027:6864]
2025-10-09T21:50:03.697293Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:03.698301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:03.698324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:03.698379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342600570652027:6864], now have 1 active actors on pipe
2025-10-09T21:50:03.698663Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:03.698712Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342600570652027:6864]
2025-10-09T21:50:03.698742Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:03.698752Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:03.698766Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:03.698826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1706 reason: error at write: Disconnected sessionId: src_id|5bca851-437e24ab-122437bd-e3bcfa8c_0
2025-10-09T21:50:03.699061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1706 sessionId: src_id|5bca851-437e24ab-122437bd-e3bcfa8c_0 is DEAD
2025-10-09T21:50:03.699206Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5bca851-437e24ab-122437bd-e3bcfa8c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:03.699222Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5bca851-437e24ab-122437bd-e3bcfa8c_0] Write session will restart in 0.008858s
2025-10-09T21:50:03.699268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5bca851-437e24ab-122437bd-e3bcfa8c_0] Write session: Do CDS request
2025-10-09T21:50:03.699275Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5bca851-437e24ab-122437bd-e3bcfa8c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:03.709577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5bca851-437e24ab-122437bd-e3bcfa8c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:03.709715Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:03.709733Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1707
2025-10-09T21:50:03.709917Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1707 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:03.709973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1707 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:03.709990Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1707 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:03.709995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:03.710695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1707 sessionId:  describe result for acl check
2025-10-09T21:50:03.710736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:03.710740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:03.710741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:03.710748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342600570652113:6820] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:03.711214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342600570652113:6820] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:03.714354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342600570652113:6820] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:03.717643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342600570652113:6820] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:03.717655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342600570652113:6820] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:03.717661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342600570652113:6820] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:03.717670Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1707 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:03.723084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:03.723550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:03.723946Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:03.723960Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:03.723975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:03.724046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:03.724089Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342600208822135:3680]
2025-10-09T21:50:03.724773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:03.725079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:03.725120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:03.725486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:03.725552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.725558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.725559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.725560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.725562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.725563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.725564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.725566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.725567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.725572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.725584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.725590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.725595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.725599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.725603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.725607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.725609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.725611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.725617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.725622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.725626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.725630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.725633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.725636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.725638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.725641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.725644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.725647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.725650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.725653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.725656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.725664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.725666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.725670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.725673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.725676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.725697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:03.731883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:03.733744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:03.733757Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:03.733759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:03.733763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:03.733769Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1706 [2:7559342600208822135:3680]
2025-10-09T21:50:03.733783Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:03.733801Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:03.733804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:03.733807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:03.733809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:03.733826Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:03.733832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:03.733846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:03.733903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.733910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.733912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.733915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:03.733987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:03.733998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.734009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.734022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.734056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:03.734081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342600570652318:6820], now have 1 active actors on pipe
2025-10-09T21:50:03.734090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342600570652030:2432], now have 1 active actors on pipe
2025-10-09T21:50:03.734097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342600208822130:3680]
2025-10-09T21:50:03.734100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342600208822130:3680]
2025-10-09T21:50:03.734102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342600208822130:3680]
2025-10-09T21:50:03.734115Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:03.734129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:03.734104Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1706
2025-10-09T21:50:03.734139Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:03.734127Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1706
2025-10-09T21:50:03.734142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:03.734144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:03.734148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:03.734151Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:03.734198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:03.734214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:03.734231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342600208822130:3680]
2025-10-09T21:50:03.734242Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:03.734260Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:03.734280Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c33a5794-25ce2394-bdc171c5-d1a1b92b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:03.734292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:03.734296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:03.734299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:03.734309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:03.734324Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:03.734349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:03.734472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:03.734481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:03.734495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:03.734567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1707 partition: 0 MaxSeqNo: 25 sessionId: src_id|c33a5794-25ce2394-bdc171c5-d1a1b92b_0
2025-10-09T21:50:03.734954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5bca851-437e24ab-122437bd-e3bcfa8c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c33a5794-25ce2394-bdc171c5-d1a1b92b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:03.739144Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c33a5794-25ce2394-bdc171c5-d1a1b92b_0] Write session: try to update token
2025-10-09T21:50:03.739160Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c33a5794-25ce2394-bdc171c5-d1a1b92b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:03.748646Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:03.748665Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:03.748668Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:03.748670Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:03.748679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342600208822130:3680]
2025-10-09T21:50:03.748684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342600208822130:3680]
2025-10-09T21:50:03.748688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342600208822130:3680]
2025-10-09T21:50:03.748732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:03.748740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.748739Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:03.748743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:03.748745Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:03.748749Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:03.748885Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:03.752213Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1707 sessionId: src_id|c33a5794-25ce2394-bdc171c5-d1a1b92b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:03.754572Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:03.799024Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342600570652332:6864]
2025-10-09T21:50:03.799263Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 1706, pipe: [1:7559342600570652332:6864]
2025-10-09T21:50:03.898304Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:03.898554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:03.898567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:03.898605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342600570652332:6864], now have 1 active actors on pipe
2025-10-09T21:50:03.898791Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:03.898824Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342600570652332:6864]
2025-10-09T21:50:03.898830Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:03.898860Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:03.898927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1707 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c33a5794-25ce2394-bdc171c5-d1a1b92b_0
2025-10-09T21:50:03.899073Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1707 sessionId: src_id|c33a5794-25ce2394-bdc171c5-d1a1b92b_0 is DEAD
2025-10-09T21:50:03.899217Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c33a5794-25ce2394-bdc171c5-d1a1b92b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:03.899228Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c33a5794-25ce2394-bdc171c5-d1a1b92b_0] Write session will restart in 0.007522s
2025-10-09T21:50:03.899249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c33a5794-25ce2394-bdc171c5-d1a1b92b_0] Write session: Do CDS request
2025-10-09T21:50:03.899254Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c33a5794-25ce2394-bdc171c5-d1a1b92b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:03.907501Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c33a5794-25ce2394-bdc171c5-d1a1b92b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:03.907608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:03.907623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1708
2025-10-09T21:50:03.907741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1708 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:03.907776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1708 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:03.907788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1708 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:03.907794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:03.908176Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1708 sessionId:  describe result for acl check
2025-10-09T21:50:03.908209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:03.908216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:03.908217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:03.908223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342600570652363:6815] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:03.908735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342600570652363:6815] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:03.912117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342600570652363:6815] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:03.915216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342600570652363:6815] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:03.915229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342600570652363:6815] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:03.915233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342600570652363:6815] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:03.915242Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1708 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:03.919264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:03.922533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:03.922735Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:03.922745Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:03.922758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:03.922812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:03.922850Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342600208822423:3677]
2025-10-09T21:50:03.923303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:03.923559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:03.923619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:03.923954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:03.924008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.924014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.924015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.924016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.924018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.924019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.924020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.924021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.924022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.924027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.924038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.924042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.924046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.924050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.924054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.924057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.924061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.924065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.924071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.924079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:03.924085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.924094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:03.924099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.924104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:03.924108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.924113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:03.924117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.924122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:03.924126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.924131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:03.924135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.924139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.924141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.924144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.924147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.924150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.924167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:03.930195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:03.931174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:03.931184Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:03.931185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:03.931188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:03.931193Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1707 [2:7559342600208822423:3677]
2025-10-09T21:50:03.931201Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:03.931213Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:03.931214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:03.931216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:03.931217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:03.931229Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:03.931235Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:03.931241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:03.931280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.931284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.931285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.931287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:03.931332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:03.931337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.931343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:03.931348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:03.931362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:03.931383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342600208822418:3677]
2025-10-09T21:50:03.931390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342600208822418:3677]
2025-10-09T21:50:03.931392Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342600208822418:3677]
2025-10-09T21:50:03.931399Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:03.931412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342600570652550:6815], now have 1 active actors on pipe
2025-10-09T21:50:03.931417Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:03.931418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342600570652337:2432], now have 1 active actors on pipe
2025-10-09T21:50:03.931420Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:03.931422Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:03.931422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:03.931430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:03.931432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:03.931450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342600208822418:3677]
2025-10-09T21:50:03.931460Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:03.931499Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:03.931503Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1707
2025-10-09T21:50:03.931504Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1707
2025-10-09T21:50:03.931547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:03.931554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:03.931580Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f764d26f-20e6532f-f74a1774-96582993_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:03.931592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:03.931596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:03.931602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:03.931611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:03.931619Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:03.931635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:03.931799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:03.931804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:03.931814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:03.931911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1708 partition: 0 MaxSeqNo: 25 sessionId: src_id|f764d26f-20e6532f-f74a1774-96582993_0
2025-10-09T21:50:03.932238Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c33a5794-25ce2394-bdc171c5-d1a1b92b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f764d26f-20e6532f-f74a1774-96582993_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:03.934625Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f764d26f-20e6532f-f74a1774-96582993_0] Write session: try to update token
2025-10-09T21:50:03.934640Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f764d26f-20e6532f-f74a1774-96582993_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:03.945957Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:03.945973Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:03.945976Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:03.945978Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:03.945987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342600208822418:3677]
2025-10-09T21:50:03.945992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342600208822418:3677]
2025-10-09T21:50:03.945996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342600208822418:3677]
2025-10-09T21:50:03.946049Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:03.946052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:03.946054Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:03.946056Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:03.946059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:03.946062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:03.946183Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:03.947157Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1708 sessionId: src_id|f764d26f-20e6532f-f74a1774-96582993_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:03.949454Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:03.999230Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342600570652640:6864]
2025-10-09T21:50:03.999479Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 1707, pipe: [1:7559342600570652640:6864]
2025-10-09T21:50:04.091462Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:04.091692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:04.091703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:04.091743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342600570652640:6864], now have 1 active actors on pipe
2025-10-09T21:50:04.091917Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:04.091964Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:04.091977Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:04.091997Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342600570652640:6864]
2025-10-09T21:50:04.092015Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:04.092032Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1708 reason: error at write: Disconnected sessionId: src_id|f764d26f-20e6532f-f74a1774-96582993_0
2025-10-09T21:50:04.092200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1708 sessionId: src_id|f764d26f-20e6532f-f74a1774-96582993_0 is DEAD
2025-10-09T21:50:04.092331Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f764d26f-20e6532f-f74a1774-96582993_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:04.092362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f764d26f-20e6532f-f74a1774-96582993_0] Write session will restart in 0.005743s
2025-10-09T21:50:04.092388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f764d26f-20e6532f-f74a1774-96582993_0] Write session: Do CDS request
2025-10-09T21:50:04.092394Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f764d26f-20e6532f-f74a1774-96582993_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:04.099545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f764d26f-20e6532f-f74a1774-96582993_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:04.099669Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:04.099688Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1709
2025-10-09T21:50:04.099818Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1709 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:04.099851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1709 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:04.099866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1709 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:04.099870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:04.100335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1709 sessionId:  describe result for acl check
2025-10-09T21:50:04.100372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:04.100377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:04.100379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:04.100384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342604865619967:6806] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:04.100918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342604865619967:6806] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:04.103595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342604865619967:6806] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:04.106810Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342604865619967:6806] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:04.106826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342604865619967:6806] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:04.106830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342604865619967:6806] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:04.106840Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1709 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:04.112681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:04.116746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:04.116939Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:04.116947Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:04.116956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:04.117006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:04.117042Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342604503790004:3673]
2025-10-09T21:50:04.117451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:04.117819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:04.117854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:04.118091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:04.118131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.118135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.118136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.118137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.118137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.118138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.118139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.118140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.118141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.118144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.118151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.118153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.118156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.118158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.118160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.118162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.118164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.118166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.118171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.118178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.118183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.118186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.118190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.118193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.118195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.118199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.118202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.118205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.118208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.118211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.118215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.118218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.118221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.118224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.118227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.118230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.118247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:04.124802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:04.125803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:04.125814Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:04.125816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:04.125819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:04.125825Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1708 [2:7559342604503790004:3673]
2025-10-09T21:50:04.125836Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:04.125851Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:04.125857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:04.125860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:04.125861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:04.125876Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:04.125887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:04.125894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:04.125944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.125949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.125950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.125953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:04.126019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:04.126043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.126054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.126064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.126081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:04.126108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342604865619939:2432], now have 1 active actors on pipe
2025-10-09T21:50:04.126114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342604865620112:6806], now have 1 active actors on pipe
2025-10-09T21:50:04.126121Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342604503789999:3673]
2025-10-09T21:50:04.126126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342604503789999:3673]
2025-10-09T21:50:04.126127Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342604503789999:3673]
2025-10-09T21:50:04.126137Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:04.126153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:04.126155Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:04.126159Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:04.126161Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:04.126162Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:04.126164Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:04.126181Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1708
2025-10-09T21:50:04.126204Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1708
2025-10-09T21:50:04.126195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342604503789999:3673]
2025-10-09T21:50:04.126204Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:04.126212Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:04.126241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:04.126246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:04.126278Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|99c7a6ab-772b271b-ce6c2909-810f161e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:04.126291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:04.126295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:04.126297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:04.126308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:04.126319Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:04.126337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:04.126470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:04.126477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:04.126492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:04.126561Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1709 partition: 0 MaxSeqNo: 25 sessionId: src_id|99c7a6ab-772b271b-ce6c2909-810f161e_0
2025-10-09T21:50:04.126913Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f764d26f-20e6532f-f74a1774-96582993_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|99c7a6ab-772b271b-ce6c2909-810f161e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:04.129480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99c7a6ab-772b271b-ce6c2909-810f161e_0] Write session: try to update token
2025-10-09T21:50:04.129498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99c7a6ab-772b271b-ce6c2909-810f161e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:04.140799Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:04.140815Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:04.140817Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:04.140818Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:04.140828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342604503789999:3673]
2025-10-09T21:50:04.140831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342604503789999:3673]
2025-10-09T21:50:04.140834Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342604503789999:3673]
2025-10-09T21:50:04.140872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:04.140877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.140878Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:04.140879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:04.140884Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:04.140888Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:04.141004Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:04.142816Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1709 sessionId: src_id|99c7a6ab-772b271b-ce6c2909-810f161e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:04.145149Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:04.192232Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342604865620247:6864]
2025-10-09T21:50:04.192453Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 1708, pipe: [1:7559342604865620247:6864]
2025-10-09T21:50:04.291082Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:04.291529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:04.291544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:04.291581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342604865620247:6864], now have 1 active actors on pipe
2025-10-09T21:50:04.291749Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:04.291781Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:04.291784Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342604865620247:6864]
2025-10-09T21:50:04.291810Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:04.291886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1709 reason: status is not ok: tablet will be restarted right now sessionId: src_id|99c7a6ab-772b271b-ce6c2909-810f161e_0
2025-10-09T21:50:04.292063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1709 sessionId: src_id|99c7a6ab-772b271b-ce6c2909-810f161e_0 is DEAD
2025-10-09T21:50:04.292252Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99c7a6ab-772b271b-ce6c2909-810f161e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:04.292260Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99c7a6ab-772b271b-ce6c2909-810f161e_0] Write session will restart in 0.008276s
2025-10-09T21:50:04.292289Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99c7a6ab-772b271b-ce6c2909-810f161e_0] Write session: Do CDS request
2025-10-09T21:50:04.292296Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99c7a6ab-772b271b-ce6c2909-810f161e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:04.301481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99c7a6ab-772b271b-ce6c2909-810f161e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:04.301593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:04.301621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1710
2025-10-09T21:50:04.301733Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1710 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:04.301765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1710 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:04.301775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1710 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:04.301779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:04.302239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1710 sessionId:  describe result for acl check
2025-10-09T21:50:04.302276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:04.302283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:04.302285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:04.302291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342604865620346:6799] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:04.302907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342604865620346:6799] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:04.305567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342604865620346:6799] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:04.308141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342604865620346:6799] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:04.308160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342604865620346:6799] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:04.308163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342604865620346:6799] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:04.308169Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1710 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:04.310064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:04.313406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:04.313612Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:04.313621Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:04.313634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:04.313682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:04.313711Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342604503790291:3668]
2025-10-09T21:50:04.314113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:04.314378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:04.314418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:04.314869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:04.314928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.314934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.314935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.314935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.314936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.314937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.314938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.314938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.314939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.314943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.314949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.314951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.314953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.314955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.314958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.314960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.314962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.314964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.314969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.314974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.314978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.314981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.314984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.314987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.314989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.314992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.314994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.314997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.315000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.315003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.315006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.315009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.315012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.315015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.315017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.315020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.315036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:04.321491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:04.322556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:04.322570Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:04.322571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:04.322574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:04.322578Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1709 [2:7559342604503790291:3668]
2025-10-09T21:50:04.322589Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:04.322603Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:04.322608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:04.322610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:04.322612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:04.322625Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:04.322632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:04.322638Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:04.322684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.322689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.322690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.322692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:04.322741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:04.322749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.322759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.322775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.322795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:04.322814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342604865620545:6799], now have 1 active actors on pipe
2025-10-09T21:50:04.322822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342604865620254:2432], now have 1 active actors on pipe
2025-10-09T21:50:04.322828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342604503790286:3668]
2025-10-09T21:50:04.322830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342604503790286:3668]
2025-10-09T21:50:04.322831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342604503790286:3668]
2025-10-09T21:50:04.322841Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:04.322851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:04.322859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:04.322859Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:04.322861Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:04.322862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:04.322864Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:04.322880Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342604503790286:3668]
2025-10-09T21:50:04.322885Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:04.322891Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:04.322857Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1709
2025-10-09T21:50:04.322859Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1709
2025-10-09T21:50:04.322915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:04.322924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:04.322955Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1dea24cc-ee0c7bdc-9defc7dd-167666a2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:04.322965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:04.322969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:04.322971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:04.322978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:04.322985Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:04.323000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:04.323130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:04.323136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:04.323146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:04.323214Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1710 partition: 0 MaxSeqNo: 25 sessionId: src_id|1dea24cc-ee0c7bdc-9defc7dd-167666a2_0
2025-10-09T21:50:04.323539Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99c7a6ab-772b271b-ce6c2909-810f161e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1dea24cc-ee0c7bdc-9defc7dd-167666a2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:04.325992Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1dea24cc-ee0c7bdc-9defc7dd-167666a2_0] Write session: try to update token
2025-10-09T21:50:04.326006Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1dea24cc-ee0c7bdc-9defc7dd-167666a2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:04.337274Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:04.337288Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:04.337290Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:04.337291Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:04.337299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342604503790286:3668]
2025-10-09T21:50:04.337302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342604503790286:3668]
2025-10-09T21:50:04.337305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342604503790286:3668]
2025-10-09T21:50:04.337340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:04.337346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.337348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:04.337347Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:04.337352Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:04.337356Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:04.337486Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:04.338886Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1710 sessionId: src_id|1dea24cc-ee0c7bdc-9defc7dd-167666a2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:04.341050Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:04.391916Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342604865620582:6864]
2025-10-09T21:50:04.392110Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 1709, pipe: [1:7559342604865620582:6864]
2025-10-09T21:50:04.487621Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:04.488034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:04.488047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:04.488092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342604865620582:6864], now have 1 active actors on pipe
2025-10-09T21:50:04.488285Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:04.488320Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:04.488327Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342604865620582:6864]
2025-10-09T21:50:04.488332Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:04.488350Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:04.488389Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1710 reason: error at write: Disconnected sessionId: src_id|1dea24cc-ee0c7bdc-9defc7dd-167666a2_0
2025-10-09T21:50:04.488545Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1710 sessionId: src_id|1dea24cc-ee0c7bdc-9defc7dd-167666a2_0 is DEAD
2025-10-09T21:50:04.488678Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1dea24cc-ee0c7bdc-9defc7dd-167666a2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:04.488688Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1dea24cc-ee0c7bdc-9defc7dd-167666a2_0] Write session will restart in 0.008350s
2025-10-09T21:50:04.488711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1dea24cc-ee0c7bdc-9defc7dd-167666a2_0] Write session: Do CDS request
2025-10-09T21:50:04.488717Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1dea24cc-ee0c7bdc-9defc7dd-167666a2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:04.498515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1dea24cc-ee0c7bdc-9defc7dd-167666a2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:04.498610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:04.498624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1711
2025-10-09T21:50:04.498779Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1711 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:04.498806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1711 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:04.498810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1711 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:04.498813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:04.499329Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1711 sessionId:  describe result for acl check
2025-10-09T21:50:04.499364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:04.499373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:04.499375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:04.499379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342604865620682:6776] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:04.499915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342604865620682:6776] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:04.502721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342604865620682:6776] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:04.505297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342604865620682:6776] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:04.505308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342604865620682:6776] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:04.505311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342604865620682:6776] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:04.505317Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1711 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:04.507183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:04.510637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:04.510811Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:04.510821Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:04.510831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:04.510877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:04.510932Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342604503790616:3615]
2025-10-09T21:50:04.511498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:04.511761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:04.511804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:04.512050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:04.512083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.512088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.512088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.512089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.512090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.512090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.512091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.512092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.512092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.512096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.512103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.512105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.512108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.512110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.512112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.512114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.512118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.512120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.512125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.512129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.512134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.512139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.512143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.512148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.512156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.512161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.512169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.512174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.512182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.512187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.512196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.512204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.512213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.512217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.512236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.512240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.512260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:04.518393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:04.519433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:04.519448Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:04.519450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:04.519455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:04.519461Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1710 [2:7559342604503790616:3615]
2025-10-09T21:50:04.519474Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:04.519491Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:04.519494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:04.519498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:04.519502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:04.519517Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:04.519530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:04.519537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:04.519593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.519600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.519602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.519605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:04.519671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:04.519680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.519692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.519704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.519730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:04.519755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342604865620584:2432], now have 1 active actors on pipe
2025-10-09T21:50:04.519763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342604865620876:6776], now have 1 active actors on pipe
2025-10-09T21:50:04.519771Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342604503790611:3615]
2025-10-09T21:50:04.519775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342604503790611:3615]
2025-10-09T21:50:04.519777Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342604503790611:3615]
2025-10-09T21:50:04.519790Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:04.519796Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1710
2025-10-09T21:50:04.519797Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1710
2025-10-09T21:50:04.519808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:04.519818Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:04.519821Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:04.519823Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:04.519828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:04.519831Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:04.519870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:04.519879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:04.519892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342604503790611:3615]
2025-10-09T21:50:04.519902Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:04.519925Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f148b1f5-66438134-718284f9-86100362_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:04.519925Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:04.519936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:04.519939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:04.519941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:04.519948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:04.519958Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:04.519985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:04.520086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:04.520092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:04.520104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:04.520177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1711 partition: 0 MaxSeqNo: 25 sessionId: src_id|f148b1f5-66438134-718284f9-86100362_0
2025-10-09T21:50:04.520513Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1dea24cc-ee0c7bdc-9defc7dd-167666a2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f148b1f5-66438134-718284f9-86100362_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:04.522931Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f148b1f5-66438134-718284f9-86100362_0] Write session: try to update token
2025-10-09T21:50:04.522949Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f148b1f5-66438134-718284f9-86100362_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:04.533732Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:04.533747Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:04.533749Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:04.533750Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:04.533757Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342604503790611:3615]
2025-10-09T21:50:04.533760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342604503790611:3615]
2025-10-09T21:50:04.533763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342604503790611:3615]
2025-10-09T21:50:04.533802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:04.533809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.533811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:04.533816Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:04.533822Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:04.533827Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:04.533928Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:04.535918Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1711 sessionId: src_id|f148b1f5-66438134-718284f9-86100362_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:04.538062Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:04.588521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342604865620891:6864]
2025-10-09T21:50:04.588781Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 1710, pipe: [1:7559342604865620891:6864]
2025-10-09T21:50:04.675854Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:04.676318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:04.676335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:04.676373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342604865620891:6864], now have 1 active actors on pipe
2025-10-09T21:50:04.676542Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:04.676594Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:04.676633Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342604865620891:6864]
2025-10-09T21:50:04.676650Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:04.676711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1711 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f148b1f5-66438134-718284f9-86100362_0
2025-10-09T21:50:04.676884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1711 sessionId: src_id|f148b1f5-66438134-718284f9-86100362_0 is DEAD
2025-10-09T21:50:04.677022Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f148b1f5-66438134-718284f9-86100362_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:04.677035Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f148b1f5-66438134-718284f9-86100362_0] Write session will restart in 0.007488s
2025-10-09T21:50:04.677063Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f148b1f5-66438134-718284f9-86100362_0] Write session: Do CDS request
2025-10-09T21:50:04.677070Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f148b1f5-66438134-718284f9-86100362_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:04.685582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:04.685596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1712
2025-10-09T21:50:04.685635Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f148b1f5-66438134-718284f9-86100362_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:04.685837Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1712 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:04.685869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1712 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:04.685876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1712 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:04.685880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:04.686424Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1712 sessionId:  describe result for acl check
2025-10-09T21:50:04.686456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:04.686462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:04.686468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:04.686474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342604865620951:6773] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:04.687037Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342604865620951:6773] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:04.690174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342604865620951:6773] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:04.692755Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342604865620951:6773] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:04.692769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342604865620951:6773] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:04.692772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342604865620951:6773] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:04.692780Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1712 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:04.699310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:04.699729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:04.699901Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:04.699911Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:04.699923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:04.699984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:04.700018Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342604503790904:3623]
2025-10-09T21:50:04.700509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:04.700786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:04.700840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:04.701187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:04.701237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.701243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.701244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.701246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.701247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.701248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.701249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.701251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.701252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.701257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.701268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.701272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.701277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.701281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.701285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.701289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.701293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.701297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.701308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.701321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.701331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.701341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.701349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.701358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.701367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.701377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.701382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.701387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.701392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.701396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.701406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.701410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.701420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.701425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.701430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.701436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.701472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:04.707527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:04.708594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:04.708610Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:04.708612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:04.708615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:04.708620Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1711 [2:7559342604503790904:3623]
2025-10-09T21:50:04.708630Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:04.708645Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:04.708647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:04.708650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:04.708652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:04.708665Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:04.708673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:04.708679Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:04.708726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.708731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.708732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.708734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:04.708791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:04.708796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.708803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.708815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.708824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:04.708841Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342604503790899:3623]
2025-10-09T21:50:04.708846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342604503790899:3623]
2025-10-09T21:50:04.708848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342604503790899:3623]
2025-10-09T21:50:04.708855Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:04.708866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342604865621154:6773], now have 1 active actors on pipe
2025-10-09T21:50:04.708866Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:04.708870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:04.708872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:04.708874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342604865620893:2432], now have 1 active actors on pipe
2025-10-09T21:50:04.708878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:04.708886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:04.708889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:04.708922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342604503790899:3623]
2025-10-09T21:50:04.708937Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:04.708931Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1711
2025-10-09T21:50:04.708947Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1711
2025-10-09T21:50:04.708964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:04.709006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:04.709014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:04.709039Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bc7482bf-e554b4e4-7de90513-127979d0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:04.709049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:04.709054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:04.709055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:04.709062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:04.709074Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:04.709090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:04.709229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:04.709235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:04.709247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:04.709350Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1712 partition: 0 MaxSeqNo: 25 sessionId: src_id|bc7482bf-e554b4e4-7de90513-127979d0_0
2025-10-09T21:50:04.709689Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f148b1f5-66438134-718284f9-86100362_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bc7482bf-e554b4e4-7de90513-127979d0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:04.712333Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc7482bf-e554b4e4-7de90513-127979d0_0] Write session: try to update token
2025-10-09T21:50:04.712353Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc7482bf-e554b4e4-7de90513-127979d0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:04.723379Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:04.723395Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:04.723397Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:04.723398Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:04.723406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342604503790899:3623]
2025-10-09T21:50:04.723412Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342604503790899:3623]
2025-10-09T21:50:04.723415Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342604503790899:3623]
2025-10-09T21:50:04.723452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:04.723459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.723461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:04.723471Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:04.723478Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:04.723483Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:04.723587Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:04.726212Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1712 sessionId: src_id|bc7482bf-e554b4e4-7de90513-127979d0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:04.728509Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:04.776952Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342604865621202:6864]
2025-10-09T21:50:04.777202Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 1711, pipe: [1:7559342604865621202:6864]
2025-10-09T21:50:04.872975Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:04.873484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:04.873498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:04.873551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342604865621202:6864], now have 1 active actors on pipe
2025-10-09T21:50:04.873769Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:04.873752Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:04.873808Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342604865621202:6864]
2025-10-09T21:50:04.873822Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:04.873825Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:04.873900Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1712 reason: error at write: Disconnected sessionId: src_id|bc7482bf-e554b4e4-7de90513-127979d0_0
2025-10-09T21:50:04.874108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1712 sessionId: src_id|bc7482bf-e554b4e4-7de90513-127979d0_0 is DEAD
2025-10-09T21:50:04.874323Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc7482bf-e554b4e4-7de90513-127979d0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:04.874336Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc7482bf-e554b4e4-7de90513-127979d0_0] Write session will restart in 0.006108s
2025-10-09T21:50:04.874365Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc7482bf-e554b4e4-7de90513-127979d0_0] Write session: Do CDS request
2025-10-09T21:50:04.874370Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc7482bf-e554b4e4-7de90513-127979d0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:04.881503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc7482bf-e554b4e4-7de90513-127979d0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:04.881610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:04.881623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1713
2025-10-09T21:50:04.881785Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1713 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:04.881829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1713 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:04.881834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1713 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:04.881838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:04.882322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1713 sessionId:  describe result for acl check
2025-10-09T21:50:04.882353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:04.882356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:04.882357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:04.882363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342604865621239:6768] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:04.882967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342604865621239:6768] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:04.886707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342604865621239:6768] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:04.889966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342604865621239:6768] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:04.889976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342604865621239:6768] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:04.889979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342604865621239:6768] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:04.889986Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1713 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:04.897575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:04.898076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:04.898369Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:04.898380Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:04.898393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:04.898452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:04.898498Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342604503791198:3619]
2025-10-09T21:50:04.899091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:04.899364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:04.899413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:04.899700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:04.899747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.899753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.899754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.899755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.899757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.899758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.899759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.899760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.899761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.899766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.899777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.899783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.899787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.899795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.899799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.899807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.899811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.899815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.899821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.899828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:04.899835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.899845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:04.899854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.899863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:04.899871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.899878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:04.899886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.899895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:04.899904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.899913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:04.899921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.899930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.899938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.899944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.899948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.899953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.899981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:04.907602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:04.909105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:04.909119Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:04.909121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:04.909126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:04.909132Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1712 [2:7559342604503791198:3619]
2025-10-09T21:50:04.909147Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:04.909166Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:04.909174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:04.909177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:04.909180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:04.909197Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:04.909215Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:04.909228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:04.909285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.909294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.909296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.909300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:04.909371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:04.909379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.909390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:04.909407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:04.909429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:04.909455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342604865621210:2432], now have 1 active actors on pipe
2025-10-09T21:50:04.909468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342604503791193:3619]
2025-10-09T21:50:04.909475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342604503791193:3619]
2025-10-09T21:50:04.909476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342604503791193:3619]
2025-10-09T21:50:04.909489Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:04.909504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342604865621424:6768], now have 1 active actors on pipe
2025-10-09T21:50:04.909523Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:04.909524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:04.909526Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:04.909528Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:04.909538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:04.909519Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1712
2025-10-09T21:50:04.909542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:04.909554Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1712
2025-10-09T21:50:04.909574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342604503791193:3619]
2025-10-09T21:50:04.909585Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:04.909587Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:04.909617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:04.909621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:04.909658Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|23f78da0-1e3a96ee-23e5a938-9dbd5354_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:04.909671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:04.909674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:04.909677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:04.909687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:04.909697Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:04.909714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:04.909837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:04.909842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:04.909855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:04.909932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1713 partition: 0 MaxSeqNo: 25 sessionId: src_id|23f78da0-1e3a96ee-23e5a938-9dbd5354_0
2025-10-09T21:50:04.910265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc7482bf-e554b4e4-7de90513-127979d0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|23f78da0-1e3a96ee-23e5a938-9dbd5354_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:04.912775Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23f78da0-1e3a96ee-23e5a938-9dbd5354_0] Write session: try to update token
2025-10-09T21:50:04.912792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23f78da0-1e3a96ee-23e5a938-9dbd5354_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:04.926080Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1713 sessionId: src_id|23f78da0-1e3a96ee-23e5a938-9dbd5354_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:04.926314Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:04.926331Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:04.926333Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:04.926335Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:04.926343Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342604503791193:3619]
2025-10-09T21:50:04.926347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342604503791193:3619]
2025-10-09T21:50:04.926350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342604503791193:3619]
2025-10-09T21:50:04.926391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:04.926397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:04.926399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:04.926529Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:04.926543Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:04.926544Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:04.926547Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:04.928220Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:04.974081Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342604865621519:6864]
2025-10-09T21:50:04.974330Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 1712, pipe: [1:7559342604865621519:6864]
2025-10-09T21:50:05.075598Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:05.076081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.076097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.076143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342604865621519:6864], now have 1 active actors on pipe
2025-10-09T21:50:05.076323Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:05.076386Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342604865621519:6864]
2025-10-09T21:50:05.076410Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:05.076516Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:05.076585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1713 reason: status is not ok: tablet will be restarted right now sessionId: src_id|23f78da0-1e3a96ee-23e5a938-9dbd5354_0
2025-10-09T21:50:05.076803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1713 sessionId: src_id|23f78da0-1e3a96ee-23e5a938-9dbd5354_0 is DEAD
2025-10-09T21:50:05.076913Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23f78da0-1e3a96ee-23e5a938-9dbd5354_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:05.076926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23f78da0-1e3a96ee-23e5a938-9dbd5354_0] Write session will restart in 0.006705s
2025-10-09T21:50:05.076950Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23f78da0-1e3a96ee-23e5a938-9dbd5354_0] Write session: Do CDS request
2025-10-09T21:50:05.076955Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23f78da0-1e3a96ee-23e5a938-9dbd5354_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:05.084434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23f78da0-1e3a96ee-23e5a938-9dbd5354_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:05.084508Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:05.084523Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1714
2025-10-09T21:50:05.084633Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1714 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:05.084666Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1714 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:05.084677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1714 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:05.084681Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:05.085156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1714 sessionId:  describe result for acl check
2025-10-09T21:50:05.085187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:05.085194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:05.085196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:05.085202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342609160588872:6761] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:05.085693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342609160588872:6761] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:05.088107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342609160588872:6761] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:05.091658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342609160588872:6761] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:05.091674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342609160588872:6761] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:05.091677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342609160588872:6761] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:05.091687Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1714 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:05.100336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:05.100873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:05.101320Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:05.101332Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:05.101346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:05.101408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:05.101456Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342608798758784:3618]
2025-10-09T21:50:05.102010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:05.102324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:05.102370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:05.102695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:05.102749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.102755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.102756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.102758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.102759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.102761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.102762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.102763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.102764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.102770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.102782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.102786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.102791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.102795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.102798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.102801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.102805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.102808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.102817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.102825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.102832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.102843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.102850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.102856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.102859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.102865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.102870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.102879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.102883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.102889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.102894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.102899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.102904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.102914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.102918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.102923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.102954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:05.110370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:05.111373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:05.111390Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:05.111392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:05.111396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:05.111402Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1713 [2:7559342608798758784:3618]
2025-10-09T21:50:05.111413Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:05.111433Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:05.111437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:05.111440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:05.111443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:05.111463Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:05.111476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:05.111483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:05.111539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.111546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.111547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.111549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:05.111619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:05.111627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.111639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.111650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.111671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:05.111697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342609160588818:2432], now have 1 active actors on pipe
2025-10-09T21:50:05.111705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342609160589042:6761], now have 1 active actors on pipe
2025-10-09T21:50:05.111713Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342608798758779:3618]
2025-10-09T21:50:05.111720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342608798758779:3618]
2025-10-09T21:50:05.111722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342608798758779:3618]
2025-10-09T21:50:05.111735Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:05.111749Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:05.111754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:05.111759Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:05.111761Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:05.111768Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:05.111771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:05.111809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342608798758779:3618]
2025-10-09T21:50:05.111819Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:05.111825Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:05.111810Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1713
2025-10-09T21:50:05.111811Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1713
2025-10-09T21:50:05.111871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.111881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.111915Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5963ff24-aa26c2c8-5963264d-d343bc77_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:05.111927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:05.111931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:05.111934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:05.111945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:05.111959Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:05.111979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:05.112152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.112162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.112178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:05.112267Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1714 partition: 0 MaxSeqNo: 25 sessionId: src_id|5963ff24-aa26c2c8-5963264d-d343bc77_0
2025-10-09T21:50:05.112620Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23f78da0-1e3a96ee-23e5a938-9dbd5354_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5963ff24-aa26c2c8-5963264d-d343bc77_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:05.115157Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5963ff24-aa26c2c8-5963264d-d343bc77_0] Write session: try to update token
2025-10-09T21:50:05.115176Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5963ff24-aa26c2c8-5963264d-d343bc77_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:05.126176Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:05.126193Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:05.126204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342608798758779:3618]
2025-10-09T21:50:05.126249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:05.126256Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:05.126401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:05.126581Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:05.126721Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:05.126863Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:05.127169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:05.127216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:05.128224Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1714 sessionId: src_id|5963ff24-aa26c2c8-5963264d-d343bc77_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:05.130656Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:05.131238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.131260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.131301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:05.131394Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:05.142494Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:05.142513Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:05.142516Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:05.142518Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:05.142531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342608798758779:3618]
2025-10-09T21:50:05.142535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342608798758779:3618]
2025-10-09T21:50:05.142539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342608798758779:3618]
2025-10-09T21:50:05.142579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:05.142584Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:05.142586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.142588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:05.142594Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:05.142599Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:05.142716Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:05.176819Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342609160589127:6864]
2025-10-09T21:50:05.177091Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 1713, pipe: [1:7559342609160589127:6864]
2025-10-09T21:50:05.291120Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:05.297736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.297759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.297928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:05.297989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:05.298071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:05.298132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:05.298463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:05.298482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:05.298551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:05.298610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:05.298665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:05.298736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:05.298784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:05.298793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:05.298861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:05.298920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:05.298979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:05.299043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:05.299085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:05.299093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:05.299171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:05.299234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:05.299295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:05.299350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:05.299391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:05.299396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:05.299453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:05.299507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:05.299577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:05.299635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:05.299674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:05.299680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:05.299735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:05.299797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:05.299861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:05.299917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:05.299957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:05.299962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:05.300019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:05.300069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:05.300160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:05.300217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:05.300269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:05.300276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:05.300331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:05.300382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:05.300447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:05.300502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:05.300543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:05.300548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:05.300610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:05.300668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:05.300730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:05.300797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:05.300840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:05.300850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:05.300916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:05.300980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:05.301041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:05.301112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:05.301160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:05.301167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:05.301225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342609160589127:6864], now have 1 active actors on pipe
2025-10-09T21:50:05.301419Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:05.301437Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:05.301441Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:05.301466Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342609160589127:6864]
2025-10-09T21:50:05.301474Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1714 reason: error at write: Disconnected sessionId: src_id|5963ff24-aa26c2c8-5963264d-d343bc77_0
2025-10-09T21:50:05.301684Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1714 sessionId: src_id|5963ff24-aa26c2c8-5963264d-d343bc77_0 is DEAD
2025-10-09T21:50:05.301857Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5963ff24-aa26c2c8-5963264d-d343bc77_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:05.301870Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5963ff24-aa26c2c8-5963264d-d343bc77_0] Write session will restart in 0.006218s
2025-10-09T21:50:05.301894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5963ff24-aa26c2c8-5963264d-d343bc77_0] Write session: Do CDS request
2025-10-09T21:50:05.301899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5963ff24-aa26c2c8-5963264d-d343bc77_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:05.301899Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:05.309520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5963ff24-aa26c2c8-5963264d-d343bc77_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:05.309933Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:05.309949Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1715
2025-10-09T21:50:05.310071Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1715 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:05.310097Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1715 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:05.310105Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1715 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:05.310110Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:05.310601Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1715 sessionId:  describe result for acl check
2025-10-09T21:50:05.310644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:05.310651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:05.310653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:05.310660Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342609160589180:6754] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:05.311281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342609160589180:6754] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:05.314294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342609160589180:6754] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:05.317705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342609160589180:6754] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:05.317718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342609160589180:6754] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:05.317722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342609160589180:6754] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:05.317730Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1715 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:05.321986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:05.325975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:05.326207Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:05.326219Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:05.326231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:05.326290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:05.326331Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342608798759076:3612]
2025-10-09T21:50:05.326934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:05.327260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:05.327305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:05.327640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:05.327692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.327698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.327699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.327700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.327701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.327702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.327704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.327705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.327706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.327711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.327722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.327725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.327729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.327733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.327736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.327740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.327743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.327747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.327754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.327769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.327776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.327781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.327790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.327795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.327800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.327809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.327814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.327820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.327829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.327835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.327840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.327850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.327854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.327859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.327866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.327871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.327902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:05.335199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:05.336771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:05.336788Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:05.336791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:05.336796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:05.336802Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1714 [2:7559342608798759076:3612]
2025-10-09T21:50:05.336817Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:05.336841Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:05.336845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:05.336848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:05.336850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:05.336869Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:05.336881Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:05.336893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:05.336955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.336967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.336969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.336971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:05.337046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:05.337059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.337072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.337084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.337102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:05.337129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342609160589138:2432], now have 1 active actors on pipe
2025-10-09T21:50:05.337133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342609160589373:6754], now have 1 active actors on pipe
2025-10-09T21:50:05.337141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342608798759071:3612]
2025-10-09T21:50:05.337143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342608798759071:3612]
2025-10-09T21:50:05.337145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342608798759071:3612]
2025-10-09T21:50:05.337157Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:05.337167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:05.337143Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1714
2025-10-09T21:50:05.337175Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:05.337145Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1714
2025-10-09T21:50:05.337176Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:05.337178Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:05.337190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:05.337193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:05.337230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342608798759071:3612]
2025-10-09T21:50:05.337243Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:05.337246Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:05.337258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.337269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.337327Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2476fcb1-d8c6ddb0-4faeb83e-81951753_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:05.337339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:05.337348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:05.337351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:05.337361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:05.337371Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:05.337394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:05.337578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.337590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.337611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:05.337695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1715 partition: 0 MaxSeqNo: 25 sessionId: src_id|2476fcb1-d8c6ddb0-4faeb83e-81951753_0
2025-10-09T21:50:05.338061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5963ff24-aa26c2c8-5963264d-d343bc77_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2476fcb1-d8c6ddb0-4faeb83e-81951753_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:05.340515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2476fcb1-d8c6ddb0-4faeb83e-81951753_0] Write session: try to update token
2025-10-09T21:50:05.340532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2476fcb1-d8c6ddb0-4faeb83e-81951753_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:05.352808Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1715 sessionId: src_id|2476fcb1-d8c6ddb0-4faeb83e-81951753_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:05.353760Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:05.353774Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:05.353776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:05.353777Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:05.353784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342608798759071:3612]
2025-10-09T21:50:05.353788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342608798759071:3612]
2025-10-09T21:50:05.353800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342608798759071:3612]
2025-10-09T21:50:05.353841Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:05.353844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:05.353847Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:05.353850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.353851Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:05.353852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:05.353986Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:05.355165Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:05.401573Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342609160589462:6864]
2025-10-09T21:50:05.401779Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 1714, pipe: [1:7559342609160589462:6864]
2025-10-09T21:50:05.496255Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:05.496780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.496797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.496838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342609160589462:6864], now have 1 active actors on pipe
2025-10-09T21:50:05.497054Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:05.497097Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342609160589462:6864]
2025-10-09T21:50:05.497107Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:05.497134Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:05.497208Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1715 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2476fcb1-d8c6ddb0-4faeb83e-81951753_0
2025-10-09T21:50:05.497390Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1715 sessionId: src_id|2476fcb1-d8c6ddb0-4faeb83e-81951753_0 is DEAD
2025-10-09T21:50:05.497555Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2476fcb1-d8c6ddb0-4faeb83e-81951753_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:05.497565Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2476fcb1-d8c6ddb0-4faeb83e-81951753_0] Write session will restart in 0.007313s
2025-10-09T21:50:05.497599Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2476fcb1-d8c6ddb0-4faeb83e-81951753_0] Write session: Do CDS request
2025-10-09T21:50:05.497607Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2476fcb1-d8c6ddb0-4faeb83e-81951753_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:05.506477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2476fcb1-d8c6ddb0-4faeb83e-81951753_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:05.506564Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:05.506581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1716
2025-10-09T21:50:05.506713Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1716 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:05.506747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1716 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:05.506756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1716 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:05.506761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:05.507280Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1716 sessionId:  describe result for acl check
2025-10-09T21:50:05.507316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:05.507322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:05.507323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:05.507328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342609160589543:6727] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:05.507839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342609160589543:6727] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:05.511317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342609160589543:6727] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:05.514742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342609160589543:6727] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:05.514755Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342609160589543:6727] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:05.514758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342609160589543:6727] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:05.514767Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1716 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:05.520613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:05.521107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:05.521279Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:05.521289Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:05.521300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:05.521348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:05.521388Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342608798759406:3577]
2025-10-09T21:50:05.521962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:05.522269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:05.522312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:05.522625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:05.522675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.522681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.522682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.522683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.522684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.522685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.522687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.522688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.522689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.522693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.522705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.522709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.522713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.522717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.522722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.522725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.522729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.522733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.522740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.522748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.522754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.522759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.522764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.522769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.522773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.522778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.522783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.522787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.522792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.522799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.522805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.522810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.522816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.522822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.522826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.522831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.522859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:05.530165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:05.531372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:05.531384Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:05.531385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:05.531388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:05.531393Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1715 [2:7559342608798759406:3577]
2025-10-09T21:50:05.531402Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:05.531415Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:05.531417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:05.531419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:05.531420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:05.531433Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:05.531438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:05.531446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:05.531499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.531509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.531511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.531513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:05.531568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:05.531573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.531580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.531586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.531596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:05.531613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342608798759401:3577]
2025-10-09T21:50:05.531618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342608798759401:3577]
2025-10-09T21:50:05.531619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342608798759401:3577]
2025-10-09T21:50:05.531627Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:05.531637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342609160589473:2432], now have 1 active actors on pipe
2025-10-09T21:50:05.531638Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:05.531640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342609160589770:6727], now have 1 active actors on pipe
2025-10-09T21:50:05.531641Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:05.531643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:05.531646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:05.531655Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:05.531657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:05.531675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342608798759401:3577]
2025-10-09T21:50:05.531685Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:05.531693Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:05.531719Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1715
2025-10-09T21:50:05.531726Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1715
2025-10-09T21:50:05.531785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.531794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.531816Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|efb65268-da0c79a2-f0803c3c-3caed4ed_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:05.531825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:05.531828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:05.531830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:05.531837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:05.531848Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:05.531858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:05.532020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.532028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.532040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:05.532147Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1716 partition: 0 MaxSeqNo: 25 sessionId: src_id|efb65268-da0c79a2-f0803c3c-3caed4ed_0
2025-10-09T21:50:05.532502Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2476fcb1-d8c6ddb0-4faeb83e-81951753_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|efb65268-da0c79a2-f0803c3c-3caed4ed_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:05.534800Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|efb65268-da0c79a2-f0803c3c-3caed4ed_0] Write session: try to update token
2025-10-09T21:50:05.534815Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|efb65268-da0c79a2-f0803c3c-3caed4ed_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:05.546715Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1716 sessionId: src_id|efb65268-da0c79a2-f0803c3c-3caed4ed_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:05.548021Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:05.548041Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:05.548043Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:05.548045Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:05.548054Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342608798759401:3577]
2025-10-09T21:50:05.548059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342608798759401:3577]
2025-10-09T21:50:05.548063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342608798759401:3577]
2025-10-09T21:50:05.548100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:05.548105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.548105Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:05.548107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:05.548110Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:05.548118Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:05.548233Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:05.549038Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:05.597206Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342609160589785:6864]
2025-10-09T21:50:05.597484Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 1715, pipe: [1:7559342609160589785:6864]
2025-10-09T21:50:05.688761Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:05.688975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.688990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.689026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342609160589785:6864], now have 1 active actors on pipe
2025-10-09T21:50:05.689224Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:05.689255Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:05.689281Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342609160589785:6864]
2025-10-09T21:50:05.689304Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:05.689378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1716 reason: status is not ok: tablet will be restarted right now sessionId: src_id|efb65268-da0c79a2-f0803c3c-3caed4ed_0
2025-10-09T21:50:05.689566Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1716 sessionId: src_id|efb65268-da0c79a2-f0803c3c-3caed4ed_0 is DEAD
2025-10-09T21:50:05.689700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efb65268-da0c79a2-f0803c3c-3caed4ed_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:05.689710Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efb65268-da0c79a2-f0803c3c-3caed4ed_0] Write session will restart in 0.006530s
2025-10-09T21:50:05.689732Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efb65268-da0c79a2-f0803c3c-3caed4ed_0] Write session: Do CDS request
2025-10-09T21:50:05.689737Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efb65268-da0c79a2-f0803c3c-3caed4ed_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:05.697478Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|efb65268-da0c79a2-f0803c3c-3caed4ed_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:05.697581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:05.697596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1717
2025-10-09T21:50:05.697749Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1717 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:05.697782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1717 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:05.697795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1717 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:05.697798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:05.698256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1717 sessionId:  describe result for acl check
2025-10-09T21:50:05.698286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:05.698291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:05.698292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:05.698298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342609160589836:6716] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:05.698708Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342609160589836:6716] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:05.701923Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342609160589836:6716] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:05.705120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342609160589836:6716] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:05.705135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342609160589836:6716] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:05.705139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342609160589836:6716] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:05.705148Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1717 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:05.712807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:05.713310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:05.713490Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:05.713500Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:05.713521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:05.713569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:05.713616Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342608798759699:3576]
2025-10-09T21:50:05.714222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:05.714526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:05.714570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:05.714887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:05.714939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.714945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.714946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.714948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.714949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.714950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.714952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.714953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.714954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.714959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.714972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.714976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.714979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.714984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.714988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.714992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.714996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.714999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.715006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.715014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.715020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.715026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.715032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.715038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.715047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.715053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.715057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.715062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.715066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.715077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.715082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.715086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.715091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.715097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.715102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.715113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.715142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:05.721636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:05.723202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:05.723219Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:05.723221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:05.723225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:05.723233Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1716 [2:7559342608798759699:3576]
2025-10-09T21:50:05.723246Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:05.723265Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:05.723268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:05.723271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:05.723274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:05.723294Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:05.723303Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:05.723310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:05.723371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.723378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.723380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.723383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:05.723454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:05.723463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.723477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.723490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.723515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:05.723542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342609160589788:2432], now have 1 active actors on pipe
2025-10-09T21:50:05.723551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342609160590020:6716], now have 1 active actors on pipe
2025-10-09T21:50:05.723558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342608798759694:3576]
2025-10-09T21:50:05.723561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342608798759694:3576]
2025-10-09T21:50:05.723563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342608798759694:3576]
2025-10-09T21:50:05.723578Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:05.723591Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:05.723598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:05.723602Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:05.723604Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:05.723613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:05.723617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:05.723617Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1716
2025-10-09T21:50:05.723650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342608798759694:3576]
2025-10-09T21:50:05.723660Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:05.723661Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:05.723618Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1716
2025-10-09T21:50:05.723690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.723699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.723735Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d86b46cb-5e6e4e8b-486c13cf-20b1f98d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:05.723748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:05.723754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:05.723756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:05.723767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:05.723782Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:05.723806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:05.723957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.723969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.723987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:05.724085Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1717 partition: 0 MaxSeqNo: 25 sessionId: src_id|d86b46cb-5e6e4e8b-486c13cf-20b1f98d_0
2025-10-09T21:50:05.724426Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|efb65268-da0c79a2-f0803c3c-3caed4ed_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d86b46cb-5e6e4e8b-486c13cf-20b1f98d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:05.726781Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d86b46cb-5e6e4e8b-486c13cf-20b1f98d_0] Write session: try to update token
2025-10-09T21:50:05.726798Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d86b46cb-5e6e4e8b-486c13cf-20b1f98d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:05.738986Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:05.739007Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:05.739009Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:05.739011Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:05.739021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342608798759694:3576]
2025-10-09T21:50:05.739025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342608798759694:3576]
2025-10-09T21:50:05.739029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342608798759694:3576]
2025-10-09T21:50:05.739073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:05.739073Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:05.739079Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:05.739082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.739083Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:05.739084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:05.739217Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:05.739462Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1717 sessionId: src_id|d86b46cb-5e6e4e8b-486c13cf-20b1f98d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:05.741685Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:05.789657Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342609160590100:6864]
2025-10-09T21:50:05.790011Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 1716, pipe: [1:7559342609160590100:6864]
2025-10-09T21:50:05.883911Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:05.884434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.884448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.884486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342609160590100:6864], now have 1 active actors on pipe
2025-10-09T21:50:05.884674Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:05.884688Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:05.884705Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:05.884715Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342609160590100:6864]
2025-10-09T21:50:05.884743Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:05.884764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1717 reason: error at write: Disconnected sessionId: src_id|d86b46cb-5e6e4e8b-486c13cf-20b1f98d_0
2025-10-09T21:50:05.884926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1717 sessionId: src_id|d86b46cb-5e6e4e8b-486c13cf-20b1f98d_0 is DEAD
2025-10-09T21:50:05.885060Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d86b46cb-5e6e4e8b-486c13cf-20b1f98d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:05.885081Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d86b46cb-5e6e4e8b-486c13cf-20b1f98d_0] Write session will restart in 0.006210s
2025-10-09T21:50:05.885107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d86b46cb-5e6e4e8b-486c13cf-20b1f98d_0] Write session: Do CDS request
2025-10-09T21:50:05.885113Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d86b46cb-5e6e4e8b-486c13cf-20b1f98d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:05.892479Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d86b46cb-5e6e4e8b-486c13cf-20b1f98d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:05.892572Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:05.892587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1718
2025-10-09T21:50:05.892711Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1718 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:05.892743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1718 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:05.892753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1718 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:05.892757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:05.893184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1718 sessionId:  describe result for acl check
2025-10-09T21:50:05.893220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:05.893228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:05.893230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:05.893235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342609160590139:6715] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:05.893775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342609160590139:6715] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:05.897698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342609160590139:6715] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:05.901883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342609160590139:6715] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:05.901897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342609160590139:6715] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:05.901901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342609160590139:6715] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:05.901909Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1718 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:05.908660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:05.912264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:05.912459Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:05.912470Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:05.912482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:05.912535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:05.912575Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342608798759997:3554]
2025-10-09T21:50:05.913155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:05.913452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:05.913487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:05.913792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:05.913839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.913846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.913847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.913848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.913849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.913850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.913851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.913852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.913853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.913859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.913869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.913873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.913876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.913879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.913883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.913886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.913889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.913892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.913900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.913908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:05.913914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.913919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:05.913923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.913928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:05.913932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.913937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:05.913942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.913947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:05.913954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.913960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:05.913965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.913971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.913975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.913981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.913986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.913993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.914022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:05.921049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:05.922179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:05.922192Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:05.922195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:05.922200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:05.922206Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1717 [2:7559342608798759997:3554]
2025-10-09T21:50:05.922219Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:05.922235Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:05.922245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:05.922248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:05.922250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:05.922267Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:05.922273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:05.922280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:05.922332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.922340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.922342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.922345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:05.922410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:05.922418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.922430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:05.922446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:05.922462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:05.922486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342609160590107:2432], now have 1 active actors on pipe
2025-10-09T21:50:05.922494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342609160590294:6715], now have 1 active actors on pipe
2025-10-09T21:50:05.922501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342608798759992:3554]
2025-10-09T21:50:05.922503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342608798759992:3554]
2025-10-09T21:50:05.922505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342608798759992:3554]
2025-10-09T21:50:05.922518Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:05.922532Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:05.922545Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:05.922550Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:05.922551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:05.922552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:05.922554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:05.922578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342608798759992:3554]
2025-10-09T21:50:05.922586Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:05.922587Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:05.922565Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1717
2025-10-09T21:50:05.922696Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1717
2025-10-09T21:50:05.922750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.922762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.922789Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7873559b-9c8da83a-493a4f21-8024fcef_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:05.922801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:05.922809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:05.922812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:05.922822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:05.922836Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:05.922855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:05.923011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.923021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.923036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:05.923116Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1718 partition: 0 MaxSeqNo: 25 sessionId: src_id|7873559b-9c8da83a-493a4f21-8024fcef_0
2025-10-09T21:50:05.923471Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d86b46cb-5e6e4e8b-486c13cf-20b1f98d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7873559b-9c8da83a-493a4f21-8024fcef_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:05.925722Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7873559b-9c8da83a-493a4f21-8024fcef_0] Write session: try to update token
2025-10-09T21:50:05.925736Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7873559b-9c8da83a-493a4f21-8024fcef_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:05.936379Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:05.936397Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:05.936406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342608798759992:3554]
2025-10-09T21:50:05.936448Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:05.936450Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:05.936605Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:05.936739Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:05.936871Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:05.936998Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:05.937168Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:05.937205Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:05.937353Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1718 sessionId: src_id|7873559b-9c8da83a-493a4f21-8024fcef_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:05.939504Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:05.940346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:05.940361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:05.940398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:05.940526Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:05.956324Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:05.956341Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:05.956343Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:05.956345Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:05.956355Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342608798759992:3554]
2025-10-09T21:50:05.956359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342608798759992:3554]
2025-10-09T21:50:05.956364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342608798759992:3554]
2025-10-09T21:50:05.956406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:05.956416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:05.956420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:05.956426Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:05.956432Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:05.956439Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:05.956547Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:05.984770Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342609160590420:6864]
2025-10-09T21:50:05.985005Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 1717, pipe: [1:7559342609160590420:6864]
2025-10-09T21:50:06.104196Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:06.119201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:06.119227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:06.119390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:06.119459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:06.119519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:06.119574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:06.119807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:06.119815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:06.119876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:06.119931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:06.119989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:06.120045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:06.120105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:06.120111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:06.120173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:06.120230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:06.120287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:06.120344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:06.120387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:06.120393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:06.120449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:06.120516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:06.120581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:06.120643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:06.120684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:06.120690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:06.120750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:06.120819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:06.120880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:06.120937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:06.121024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:06.121030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:06.121089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:06.121144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:06.121199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:06.121261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:06.121310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:06.121318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:06.121384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:06.121445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:06.121504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:06.121576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:06.121616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:06.121622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:06.121682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:06.121738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:06.121794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:06.121852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:06.121893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:06.121901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:06.121957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:06.122010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:06.122087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:06.122142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:06.122180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:06.122186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:06.122242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:06.122295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:06.122354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:06.122412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:06.122453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:06.122458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:06.122511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342609160590420:6864], now have 1 active actors on pipe
2025-10-09T21:50:06.122828Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342609160590420:6864]
2025-10-09T21:50:06.122832Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:06.122836Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:06.122875Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:06.122880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1718 reason: error at write: Disconnected sessionId: src_id|7873559b-9c8da83a-493a4f21-8024fcef_0
2025-10-09T21:50:06.123086Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1718 sessionId: src_id|7873559b-9c8da83a-493a4f21-8024fcef_0 is DEAD
2025-10-09T21:50:06.123248Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7873559b-9c8da83a-493a4f21-8024fcef_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:06.123258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7873559b-9c8da83a-493a4f21-8024fcef_0] Write session will restart in 0.009257s
2025-10-09T21:50:06.123253Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:06.123284Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7873559b-9c8da83a-493a4f21-8024fcef_0] Write session: Do CDS request
2025-10-09T21:50:06.123292Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7873559b-9c8da83a-493a4f21-8024fcef_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:06.133525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7873559b-9c8da83a-493a4f21-8024fcef_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:06.133650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:06.133667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1719
2025-10-09T21:50:06.133808Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1719 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:06.133857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1719 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:06.133868Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1719 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:06.133872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:06.134389Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1719 sessionId:  describe result for acl check
2025-10-09T21:50:06.134423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:06.134431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:06.134433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:06.134439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342613455557799:6710] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:06.135246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342613455557799:6710] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:06.138341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342613455557799:6710] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:06.141262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342613455557799:6710] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:06.141276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342613455557799:6710] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:06.141280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342613455557799:6710] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:06.141288Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1719 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:06.144849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:06.148362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:06.148588Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:06.148603Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:06.148615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:06.148674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:06.148719Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342613093727590:3558]
2025-10-09T21:50:06.149318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:06.149696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:06.149745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:06.150068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:06.150279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.150286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.150288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.150289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.150290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.150291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.150293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.150294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.150295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.150301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.150312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.150317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.150325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.150328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.150330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.150334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.150337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.150341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.150349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.150358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.150365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.150370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.150380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.150385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.150389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.150395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.150404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.150414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.150425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.150431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.150437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.150442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.150447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.150452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.150457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.150462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.150491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:06.160943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:06.162468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:06.162481Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:06.162483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:06.162487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:06.162492Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1718 [2:7559342613093727590:3558]
2025-10-09T21:50:06.162502Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:06.162518Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:06.162519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:06.162522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:06.162523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:06.162537Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:06.162545Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:06.162552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:06.162603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.162608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.162610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.162612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:06.162687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:06.162694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.162702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.162712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.162726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:06.162741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342613093727585:3558]
2025-10-09T21:50:06.162747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342613093727585:3558]
2025-10-09T21:50:06.162748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342613093727585:3558]
2025-10-09T21:50:06.162759Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:06.162772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:06.162775Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:06.162782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:06.162784Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:06.162784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:06.162786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:06.162795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342613455557984:6710], now have 1 active actors on pipe
2025-10-09T21:50:06.162800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342613455557719:2432], now have 1 active actors on pipe
2025-10-09T21:50:06.162825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342613093727585:3558]
2025-10-09T21:50:06.162839Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:06.162863Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1718
2025-10-09T21:50:06.162870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:06.162926Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1718
2025-10-09T21:50:06.163004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:06.163020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:06.163049Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|321a8cc2-653ac1e-ecc4de2e-1213c6ff_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:06.163057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:06.163061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:06.163064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:06.163077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:06.163087Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:06.163098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:06.163313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:06.163324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:06.163338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:06.163477Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1719 partition: 0 MaxSeqNo: 25 sessionId: src_id|321a8cc2-653ac1e-ecc4de2e-1213c6ff_0
2025-10-09T21:50:06.163839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7873559b-9c8da83a-493a4f21-8024fcef_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|321a8cc2-653ac1e-ecc4de2e-1213c6ff_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:06.170954Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|321a8cc2-653ac1e-ecc4de2e-1213c6ff_0] Write session: try to update token
2025-10-09T21:50:06.170973Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|321a8cc2-653ac1e-ecc4de2e-1213c6ff_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:06.181757Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:06.181782Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:06.181786Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:06.181789Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:06.181800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342613093727585:3558]
2025-10-09T21:50:06.181806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342613093727585:3558]
2025-10-09T21:50:06.181812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342613093727585:3558]
2025-10-09T21:50:06.181881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:06.181888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.181890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:06.181906Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:06.181915Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:06.181920Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:06.182055Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:06.185337Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1719 sessionId: src_id|321a8cc2-653ac1e-ecc4de2e-1213c6ff_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:06.188595Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:06.222846Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342613455558036:6864]
2025-10-09T21:50:06.223252Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 1718, pipe: [1:7559342613455558036:6864]
2025-10-09T21:50:06.328946Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:06.329384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:06.329397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:06.329436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342613455558036:6864], now have 1 active actors on pipe
2025-10-09T21:50:06.329666Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:06.329761Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:06.329769Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:06.329774Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:06.329824Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342613455558036:6864]
2025-10-09T21:50:06.329826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1719 reason: error at write: Disconnected sessionId: src_id|321a8cc2-653ac1e-ecc4de2e-1213c6ff_0
2025-10-09T21:50:06.329974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1719 sessionId: src_id|321a8cc2-653ac1e-ecc4de2e-1213c6ff_0 is DEAD
2025-10-09T21:50:06.330146Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|321a8cc2-653ac1e-ecc4de2e-1213c6ff_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:06.330159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|321a8cc2-653ac1e-ecc4de2e-1213c6ff_0] Write session will restart in 0.005650s
2025-10-09T21:50:06.330186Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|321a8cc2-653ac1e-ecc4de2e-1213c6ff_0] Write session: Do CDS request
2025-10-09T21:50:06.330193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|321a8cc2-653ac1e-ecc4de2e-1213c6ff_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:06.336499Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|321a8cc2-653ac1e-ecc4de2e-1213c6ff_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:06.336581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:06.336594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1720
2025-10-09T21:50:06.336694Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1720 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:06.336719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1720 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:06.336727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1720 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:06.336730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:06.337143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1720 sessionId:  describe result for acl check
2025-10-09T21:50:06.337176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:06.337188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:06.337190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:06.337196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342613455558061:6698] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:06.337754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342613455558061:6698] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:06.340497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342613455558061:6698] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:06.343828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342613455558061:6698] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:06.343843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342613455558061:6698] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:06.343846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342613455558061:6698] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:06.343869Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1720 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:06.349195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:06.352664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:06.352853Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:06.352862Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:06.352874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:06.352929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:06.352974Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342613093727886:3548]
2025-10-09T21:50:06.353606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:06.353959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:06.354020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:06.354382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:06.354437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.354443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.354444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.354445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.354447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.354448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.354449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.354450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.354452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.354457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.354468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.354473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.354477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.354485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.354489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.354498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.354502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.354511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.354518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.354531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.354541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.354548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.354555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.354565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.354573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.354578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.354583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.354592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.354601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.354606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.354617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.354622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.354626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.354631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.354637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.354642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.354677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:06.361138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:06.362639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:06.362656Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:06.362658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:06.362662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:06.362669Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1719 [2:7559342613093727886:3548]
2025-10-09T21:50:06.362681Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:06.362698Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:06.362701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:06.362704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:06.362706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:06.362723Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:06.362734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:06.362742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:06.362795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.362801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.362803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.362806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:06.362872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:06.362881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.362894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.362907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.362925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:06.362950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342613455558043:2432], now have 1 active actors on pipe
2025-10-09T21:50:06.362958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342613455558227:6698], now have 1 active actors on pipe
2025-10-09T21:50:06.362964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342613093727881:3548]
2025-10-09T21:50:06.362965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342613093727881:3548]
2025-10-09T21:50:06.362967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342613093727881:3548]
2025-10-09T21:50:06.362976Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:06.362989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:06.363002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:06.363012Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:06.363014Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:06.363017Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:06.362988Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1719
2025-10-09T21:50:06.363020Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:06.363002Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1719
2025-10-09T21:50:06.363046Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342613093727881:3548]
2025-10-09T21:50:06.363058Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:06.363065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:06.363084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:06.363094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:06.363146Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|242e501-1d85af7c-d5e83a8b-cd794412_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:06.363165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:06.363175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:06.363177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:06.363187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:06.363199Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:06.363225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:06.363411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:06.363425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:06.363450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:06.363571Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1720 partition: 0 MaxSeqNo: 25 sessionId: src_id|242e501-1d85af7c-d5e83a8b-cd794412_0
2025-10-09T21:50:06.363929Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|321a8cc2-653ac1e-ecc4de2e-1213c6ff_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|242e501-1d85af7c-d5e83a8b-cd794412_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:06.366281Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|242e501-1d85af7c-d5e83a8b-cd794412_0] Write session: try to update token
2025-10-09T21:50:06.366300Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|242e501-1d85af7c-d5e83a8b-cd794412_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:06.377321Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:06.377338Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:06.377340Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:06.377342Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:06.377351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342613093727881:3548]
2025-10-09T21:50:06.377356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342613093727881:3548]
2025-10-09T21:50:06.377359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342613093727881:3548]
2025-10-09T21:50:06.377403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:06.377411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.377413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:06.377424Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:06.377431Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:06.377435Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:06.377603Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:06.378813Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1720 sessionId: src_id|242e501-1d85af7c-d5e83a8b-cd794412_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:06.381114Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:06.430195Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342613455558359:6864]
2025-10-09T21:50:06.430432Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 1719, pipe: [1:7559342613455558359:6864]
2025-10-09T21:50:06.525148Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:06.525666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:06.525682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:06.525731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342613455558359:6864], now have 1 active actors on pipe
2025-10-09T21:50:06.525937Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:06.526011Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:06.526019Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342613455558359:6864]
2025-10-09T21:50:06.526063Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:06.526128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1720 reason: status is not ok: tablet will be restarted right now sessionId: src_id|242e501-1d85af7c-d5e83a8b-cd794412_0
2025-10-09T21:50:06.526294Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1720 sessionId: src_id|242e501-1d85af7c-d5e83a8b-cd794412_0 is DEAD
2025-10-09T21:50:06.526450Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|242e501-1d85af7c-d5e83a8b-cd794412_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:06.526461Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|242e501-1d85af7c-d5e83a8b-cd794412_0] Write session will restart in 0.009185s
2025-10-09T21:50:06.526488Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|242e501-1d85af7c-d5e83a8b-cd794412_0] Write session: Do CDS request
2025-10-09T21:50:06.526495Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|242e501-1d85af7c-d5e83a8b-cd794412_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:06.536494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|242e501-1d85af7c-d5e83a8b-cd794412_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:06.536578Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:06.536592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1721
2025-10-09T21:50:06.537170Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1721 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:06.537206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1721 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:06.537217Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1721 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:06.537221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:06.537726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1721 sessionId:  describe result for acl check
2025-10-09T21:50:06.537761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:06.537763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:06.537765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:06.537771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342613455558488:6686] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:06.538331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342613455558488:6686] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:06.541338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342613455558488:6686] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:06.544155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342613455558488:6686] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:06.544168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342613455558488:6686] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:06.544172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342613455558488:6686] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:06.544180Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1721 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:06.549709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:06.550319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:06.550515Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:06.550530Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:06.550544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:06.550614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:06.550660Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342613093728222:3517]
2025-10-09T21:50:06.551217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:06.551521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:06.551555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:06.551821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:06.551875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.551881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.551882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.551884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.551885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.551886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.551887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.551888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.551889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.551895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.551906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.551911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.551915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.551919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.551923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.551927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.551931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.551935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.551942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.551947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.551951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.551955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.551961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.551966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.551970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.551975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.551980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.551985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.551989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.551994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.551999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.552004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.552010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.552015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.552022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.552032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.552060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:06.558476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:06.559783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:06.559798Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:06.559802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:06.559805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:06.559811Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1720 [2:7559342613093728222:3517]
2025-10-09T21:50:06.559824Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:06.559839Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:06.559842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:06.559845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:06.559847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:06.559861Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:06.559872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:06.559878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:06.559930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.559937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.559939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.559942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:06.560002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:06.560010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.560022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.560034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.560056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:06.560079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342613455558684:6686], now have 1 active actors on pipe
2025-10-09T21:50:06.560089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342613455558386:2432], now have 1 active actors on pipe
2025-10-09T21:50:06.560097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342613093728217:3517]
2025-10-09T21:50:06.560099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342613093728217:3517]
2025-10-09T21:50:06.560101Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342613093728217:3517]
2025-10-09T21:50:06.560115Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:06.560132Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:06.560143Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:06.560146Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:06.560149Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:06.560151Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:06.560154Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:06.560147Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1720
2025-10-09T21:50:06.560149Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1720
2025-10-09T21:50:06.560173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342613093728217:3517]
2025-10-09T21:50:06.560181Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:06.560183Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:06.560207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:06.560212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:06.560244Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3a8a644a-1e5cd5a2-38fc4a83-83c2d752_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:06.560257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:06.560263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:06.560265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:06.560275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:06.560289Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:06.560307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:06.560445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:06.560455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:06.560469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:06.560559Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1721 partition: 0 MaxSeqNo: 25 sessionId: src_id|3a8a644a-1e5cd5a2-38fc4a83-83c2d752_0
2025-10-09T21:50:06.560863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|242e501-1d85af7c-d5e83a8b-cd794412_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3a8a644a-1e5cd5a2-38fc4a83-83c2d752_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:06.563135Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a8a644a-1e5cd5a2-38fc4a83-83c2d752_0] Write session: try to update token
2025-10-09T21:50:06.563149Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a8a644a-1e5cd5a2-38fc4a83-83c2d752_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:06.573861Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:06.573878Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:06.573879Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:06.573881Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:06.573888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342613093728217:3517]
2025-10-09T21:50:06.573891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342613093728217:3517]
2025-10-09T21:50:06.573894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342613093728217:3517]
2025-10-09T21:50:06.573929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:06.573933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.573935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:06.573950Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:06.573958Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:06.573962Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:06.574070Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:06.575019Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1721 sessionId: src_id|3a8a644a-1e5cd5a2-38fc4a83-83c2d752_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:06.577684Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:06.626283Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342613455558700:6864]
2025-10-09T21:50:06.626542Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 1720, pipe: [1:7559342613455558700:6864]
2025-10-09T21:50:06.714402Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:06.714823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:06.714837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:06.714875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342613455558700:6864], now have 1 active actors on pipe
2025-10-09T21:50:06.715030Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:06.715123Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342613455558700:6864]
2025-10-09T21:50:06.715139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:06.715152Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:06.715222Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1721 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3a8a644a-1e5cd5a2-38fc4a83-83c2d752_0
2025-10-09T21:50:06.715399Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1721 sessionId: src_id|3a8a644a-1e5cd5a2-38fc4a83-83c2d752_0 is DEAD
2025-10-09T21:50:06.715559Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a8a644a-1e5cd5a2-38fc4a83-83c2d752_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:06.715567Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a8a644a-1e5cd5a2-38fc4a83-83c2d752_0] Write session will restart in 0.007623s
2025-10-09T21:50:06.715593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a8a644a-1e5cd5a2-38fc4a83-83c2d752_0] Write session: Do CDS request
2025-10-09T21:50:06.715599Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a8a644a-1e5cd5a2-38fc4a83-83c2d752_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:06.724517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a8a644a-1e5cd5a2-38fc4a83-83c2d752_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:06.724570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:06.724584Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1722
2025-10-09T21:50:06.724717Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1722 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:06.724749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1722 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:06.724762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1722 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:06.724766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:06.725200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1722 sessionId:  describe result for acl check
2025-10-09T21:50:06.725232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:06.725236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:06.725238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:06.725242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342613455558778:6677] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:06.725691Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342613455558778:6677] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:06.727917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342613455558778:6677] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:06.730385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342613455558778:6677] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:06.730396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342613455558778:6677] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:06.730401Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342613455558778:6677] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:06.730406Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1722 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:06.733951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:06.737147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:06.737526Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:06.737538Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:06.737554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:06.737626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:06.737662Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342613093728524:3511]
2025-10-09T21:50:06.738081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:06.738342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:06.738376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:06.738581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:06.738621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.738629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.738630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.738631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.738631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.738632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.738633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.738633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.738634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.738637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.738644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.738646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.738648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.738650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.738653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.738655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.738657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.738659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.738664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.738668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.738672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.738676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.738678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.738681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.738685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.738688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.738692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.738694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.738697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.738700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.738702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.738705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.738708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.738711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.738714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.738717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.738733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:06.745117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:06.746485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:06.746497Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:06.746500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:06.746504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:06.746509Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1721 [2:7559342613093728524:3511]
2025-10-09T21:50:06.746521Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:06.746537Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:06.746544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:06.746547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:06.746549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:06.746565Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:06.746574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:06.746588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:06.746643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.746651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.746653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.746656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:06.746715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:06.746722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.746735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.746751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.746775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:06.746794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342613455558703:2432], now have 1 active actors on pipe
2025-10-09T21:50:06.746797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342613455558936:6677], now have 1 active actors on pipe
2025-10-09T21:50:06.746804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342613093728519:3511]
2025-10-09T21:50:06.746810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342613093728519:3511]
2025-10-09T21:50:06.746812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342613093728519:3511]
2025-10-09T21:50:06.746826Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:06.746814Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1721
2025-10-09T21:50:06.746828Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1721
2025-10-09T21:50:06.746839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:06.746847Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:06.746850Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:06.746852Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:06.746857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:06.746864Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:06.746891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342613093728519:3511]
2025-10-09T21:50:06.746902Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:06.746909Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:06.746920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:06.746929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:06.746974Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|72f28db9-727d83a9-3581fd3f-2770ca07_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:06.746986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:06.746990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:06.746992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:06.747002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:06.747026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:06.747044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:06.747192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:06.747199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:06.747212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:06.747299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1722 partition: 0 MaxSeqNo: 25 sessionId: src_id|72f28db9-727d83a9-3581fd3f-2770ca07_0
2025-10-09T21:50:06.747522Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a8a644a-1e5cd5a2-38fc4a83-83c2d752_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|72f28db9-727d83a9-3581fd3f-2770ca07_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:06.749843Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72f28db9-727d83a9-3581fd3f-2770ca07_0] Write session: try to update token
2025-10-09T21:50:06.749860Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72f28db9-727d83a9-3581fd3f-2770ca07_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:06.761339Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:06.761360Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:06.761361Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:06.761362Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:06.761371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342613093728519:3511]
2025-10-09T21:50:06.761375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342613093728519:3511]
2025-10-09T21:50:06.761377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342613093728519:3511]
2025-10-09T21:50:06.761410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:06.761415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.761417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:06.761417Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:06.761425Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:06.761429Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:06.761545Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:06.761816Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1722 sessionId: src_id|72f28db9-727d83a9-3581fd3f-2770ca07_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:06.764106Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:06.815233Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342613455559023:6864]
2025-10-09T21:50:06.815519Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 1721, pipe: [1:7559342613455559023:6864]
2025-10-09T21:50:06.904725Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:06.905096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:06.905108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:06.905148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342613455559023:6864], now have 1 active actors on pipe
2025-10-09T21:50:06.905296Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:06.905336Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:06.905349Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:06.905349Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342613455559023:6864]
2025-10-09T21:50:06.905367Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:06.905409Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1722 reason: error at write: Disconnected sessionId: src_id|72f28db9-727d83a9-3581fd3f-2770ca07_0
2025-10-09T21:50:06.905594Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1722 sessionId: src_id|72f28db9-727d83a9-3581fd3f-2770ca07_0 is DEAD
2025-10-09T21:50:06.905730Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72f28db9-727d83a9-3581fd3f-2770ca07_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:06.905741Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72f28db9-727d83a9-3581fd3f-2770ca07_0] Write session will restart in 0.005793s
2025-10-09T21:50:06.905761Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72f28db9-727d83a9-3581fd3f-2770ca07_0] Write session: Do CDS request
2025-10-09T21:50:06.905766Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72f28db9-727d83a9-3581fd3f-2770ca07_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:06.912465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72f28db9-727d83a9-3581fd3f-2770ca07_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:06.912545Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:06.912558Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1723
2025-10-09T21:50:06.912668Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1723 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:06.912694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1723 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:06.912709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1723 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:06.912715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:06.913137Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1723 sessionId:  describe result for acl check
2025-10-09T21:50:06.913173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:06.913179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:06.913181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:06.913186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342613455559061:6672] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:06.913731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342613455559061:6672] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:06.916233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342613455559061:6672] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:06.919659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342613455559061:6672] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:06.919672Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342613455559061:6672] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:06.919675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342613455559061:6672] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:06.919683Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1723 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:06.928862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:06.929316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:06.929442Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:06.929449Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:06.929457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:06.929490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:06.929532Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342613093728823:3499]
2025-10-09T21:50:06.929894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:06.930195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:06.930229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:06.930555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:06.930608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.930613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.930613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.930614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.930615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.930615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.930616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.930617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.930617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.930620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.930628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.930630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.930632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.930634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.930636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.930638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.930640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.930642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.930647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.930651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:06.930656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.930659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:06.930661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.930664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:06.930668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.930671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:06.930673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.930676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:06.930679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.930682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:06.930685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.930688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.930690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.930693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.930704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.930707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.930722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:06.936789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:06.937985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:06.937995Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:06.937996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:06.937998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:06.938003Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1722 [2:7559342613093728823:3499]
2025-10-09T21:50:06.938010Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:06.938022Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:06.938024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:06.938026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:06.938027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:06.938050Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:06.938054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:06.938059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:06.938095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.938100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.938101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.938103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:06.938148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:06.938153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.938159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:06.938165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:06.938178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:06.938195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342613093728818:3499]
2025-10-09T21:50:06.938200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342613093728818:3499]
2025-10-09T21:50:06.938202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342613093728818:3499]
2025-10-09T21:50:06.938210Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:06.938220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342613455559028:2432], now have 1 active actors on pipe
2025-10-09T21:50:06.938222Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:06.938224Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:06.938225Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:06.938226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342613455559203:6672], now have 1 active actors on pipe
2025-10-09T21:50:06.938232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:06.938239Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:06.938241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:06.938263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342613093728818:3499]
2025-10-09T21:50:06.938276Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:06.938278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:06.938306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:06.938284Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1722
2025-10-09T21:50:06.938310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:06.938330Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a7969a74-ec546adf-cef87419-4de1cdcc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:06.938335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:06.938338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:06.938340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:06.938348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:06.938355Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:06.938369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:06.938369Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1722
2025-10-09T21:50:06.938445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:06.938450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:06.938459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:06.938551Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1723 partition: 0 MaxSeqNo: 25 sessionId: src_id|a7969a74-ec546adf-cef87419-4de1cdcc_0
2025-10-09T21:50:06.938848Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72f28db9-727d83a9-3581fd3f-2770ca07_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a7969a74-ec546adf-cef87419-4de1cdcc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:06.941155Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7969a74-ec546adf-cef87419-4de1cdcc_0] Write session: try to update token
2025-10-09T21:50:06.941173Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7969a74-ec546adf-cef87419-4de1cdcc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:06.951813Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:06.951833Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:06.951836Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:06.951837Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:06.951847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342613093728818:3499]
2025-10-09T21:50:06.951852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342613093728818:3499]
2025-10-09T21:50:06.951855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342613093728818:3499]
2025-10-09T21:50:06.951892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:06.951901Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:06.951902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:06.951905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:06.951906Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:06.951911Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:06.952036Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:06.953867Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1723 sessionId: src_id|a7969a74-ec546adf-cef87419-4de1cdcc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:06.956163Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:07.005710Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342617750526643:6864]
2025-10-09T21:50:07.005938Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 1722, pipe: [1:7559342617750526643:6864]
2025-10-09T21:50:07.096245Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:07.096653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.096665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.096702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342617750526643:6864], now have 1 active actors on pipe
2025-10-09T21:50:07.096909Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:07.096923Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:07.096927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:07.096958Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342617750526643:6864]
2025-10-09T21:50:07.096959Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:07.096990Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1723 reason: error at write: Disconnected sessionId: src_id|a7969a74-ec546adf-cef87419-4de1cdcc_0
2025-10-09T21:50:07.097179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1723 sessionId: src_id|a7969a74-ec546adf-cef87419-4de1cdcc_0 is DEAD
2025-10-09T21:50:07.097321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7969a74-ec546adf-cef87419-4de1cdcc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:07.097333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7969a74-ec546adf-cef87419-4de1cdcc_0] Write session will restart in 0.008436s
2025-10-09T21:50:07.097359Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7969a74-ec546adf-cef87419-4de1cdcc_0] Write session: Do CDS request
2025-10-09T21:50:07.097364Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7969a74-ec546adf-cef87419-4de1cdcc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:07.106451Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7969a74-ec546adf-cef87419-4de1cdcc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:07.106581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:07.106597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1724
2025-10-09T21:50:07.106737Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1724 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:07.106777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1724 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:07.106787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1724 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:07.106790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:07.107244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1724 sessionId:  describe result for acl check
2025-10-09T21:50:07.107277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:07.107284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:07.107286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:07.107293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342617750526716:6671] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:07.107812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342617750526716:6671] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:07.110159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342617750526716:6671] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:07.113257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342617750526716:6671] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:07.113268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342617750526716:6671] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:07.113272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342617750526716:6671] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:07.113279Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1724 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:07.122016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:07.122580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:07.122758Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:07.122769Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:07.122781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:07.122833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:07.122879Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342617388696419:3498]
2025-10-09T21:50:07.123478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:07.123750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:07.123804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:07.124124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:07.124181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.124189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.124190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.124191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.124193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.124194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.124195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.124196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.124197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.124202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.124213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.124217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.124222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.124225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.124229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.124233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.124237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.124241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.124249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.124257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.124264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.124275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.124280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.124290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.124295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.124305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.124314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.124319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.124327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.124333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.124337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.124343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.124353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.124362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.124372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.124379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.124414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:07.130553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:07.132053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:07.132067Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:07.132069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:07.132072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:07.132077Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1723 [2:7559342617388696419:3498]
2025-10-09T21:50:07.132086Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:07.132104Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:07.132106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:07.132109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:07.132112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:07.132128Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:07.132140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:07.132149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:07.132207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.132215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.132217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.132220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:07.132431Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1723
2025-10-09T21:50:07.132435Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1723
2025-10-09T21:50:07.132564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:07.132576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.132591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.132611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.132630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:07.132657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342617750526877:6671], now have 1 active actors on pipe
2025-10-09T21:50:07.132666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342617750526645:2432], now have 1 active actors on pipe
2025-10-09T21:50:07.132681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.132692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.132713Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342617388696414:3498]
2025-10-09T21:50:07.132721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342617388696414:3498]
2025-10-09T21:50:07.132722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342617388696414:3498]
2025-10-09T21:50:07.132734Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:07.132749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:07.132770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:07.132773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:07.132793Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|241479a5-20b1e58c-119f17ac-7e6c124d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:07.132806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:07.132810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:07.132815Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:07.132817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:07.132819Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:07.132821Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:07.132828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:07.132839Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:07.132871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342617388696414:3498]
2025-10-09T21:50:07.132884Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:07.132890Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:07.132900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:07.133095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.133107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.133128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:07.133230Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1724 partition: 0 MaxSeqNo: 25 sessionId: src_id|241479a5-20b1e58c-119f17ac-7e6c124d_0
2025-10-09T21:50:07.133623Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7969a74-ec546adf-cef87419-4de1cdcc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|241479a5-20b1e58c-119f17ac-7e6c124d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:07.136426Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|241479a5-20b1e58c-119f17ac-7e6c124d_0] Write session: try to update token
2025-10-09T21:50:07.136444Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|241479a5-20b1e58c-119f17ac-7e6c124d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:07.143731Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:07.143749Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:07.143759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342617388696414:3498]
2025-10-09T21:50:07.143800Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:07.143803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:07.144014Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:07.144224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:07.144360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:07.144493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:07.144775Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:07.144815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:07.149431Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1724 sessionId: src_id|241479a5-20b1e58c-119f17ac-7e6c124d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:07.151643Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:07.151842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.151862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.151900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:07.151980Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:07.158051Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:07.158065Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:07.158067Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:07.158069Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:07.158079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342617388696414:3498]
2025-10-09T21:50:07.158083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342617388696414:3498]
2025-10-09T21:50:07.158086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342617388696414:3498]
2025-10-09T21:50:07.158127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:07.158128Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:07.158133Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:07.158137Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:07.158140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.158143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:07.158273Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:07.197199Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342617750526970:6864]
2025-10-09T21:50:07.197475Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 1723, pipe: [1:7559342617750526970:6864]
2025-10-09T21:50:07.305273Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:07.311883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.311908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.312049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.312104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.312163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.312221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.312263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.312275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.312331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.312385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.312443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.312501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.312540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.312546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.312600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.312653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.312706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.312759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.312798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.312804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.312862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.312928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.312998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.313067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.313115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.313123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.313189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.313256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.313320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.313382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.313427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.313435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.313496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.313572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.313639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.313692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.313732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.313737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.313794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.313845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.313899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.313951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.313991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.313999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.314071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.314122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.314174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.314230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.314268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.314274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.314334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.314386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.314439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.314494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.314535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.314547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.314603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.314654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.314714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.314768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.314806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.314812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.314853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342617750526970:6864], now have 1 active actors on pipe
2025-10-09T21:50:07.315084Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:07.315105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:07.315109Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:07.315136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1724 reason: error at write: Disconnected sessionId: src_id|241479a5-20b1e58c-119f17ac-7e6c124d_0
2025-10-09T21:50:07.315155Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342617750526970:6864]
2025-10-09T21:50:07.315315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1724 sessionId: src_id|241479a5-20b1e58c-119f17ac-7e6c124d_0 is DEAD
2025-10-09T21:50:07.315451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|241479a5-20b1e58c-119f17ac-7e6c124d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:07.315462Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|241479a5-20b1e58c-119f17ac-7e6c124d_0] Write session will restart in 0.007937s
2025-10-09T21:50:07.315489Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|241479a5-20b1e58c-119f17ac-7e6c124d_0] Write session: Do CDS request
2025-10-09T21:50:07.315493Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|241479a5-20b1e58c-119f17ac-7e6c124d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:07.315788Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:07.324459Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|241479a5-20b1e58c-119f17ac-7e6c124d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:07.324520Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:07.324532Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1725
2025-10-09T21:50:07.324664Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1725 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:07.324696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1725 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:07.324703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1725 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:07.324707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:07.325109Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1725 sessionId:  describe result for acl check
2025-10-09T21:50:07.325147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:07.325153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:07.325157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:07.325163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342617750527037:6664] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:07.325627Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342617750527037:6664] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:07.328563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342617750527037:6664] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:07.331745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342617750527037:6664] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:07.331758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342617750527037:6664] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:07.331762Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342617750527037:6664] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:07.331769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1725 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:07.338930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:07.339380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:07.339558Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:07.339568Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:07.339580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:07.339632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:07.339675Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342617388696721:3490]
2025-10-09T21:50:07.340274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:07.340547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:07.340593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:07.340907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:07.340954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.340960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.340962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.340963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.340964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.340966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.340967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.340969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.340970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.340975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.340987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.340996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.341000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.341004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.341008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.341012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.341016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.341020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.341027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.341035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.341041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.341045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.341054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.341058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.341063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.341069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.341074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.341079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.341088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.341099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.341108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.341115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.341124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.341133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.341145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.341156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.341188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:07.348052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:07.349967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:07.349984Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:07.349986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:07.349991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:07.349998Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1724 [2:7559342617388696721:3490]
2025-10-09T21:50:07.350014Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:07.350054Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:07.350062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:07.350065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:07.350068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:07.350088Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:07.350100Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:07.350108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:07.350175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.350182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.350184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.350187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:07.350264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:07.350272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.350287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.350300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.350321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:07.350350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342617750527233:6664], now have 1 active actors on pipe
2025-10-09T21:50:07.350360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342617750526977:2432], now have 1 active actors on pipe
2025-10-09T21:50:07.350368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342617388696716:3490]
2025-10-09T21:50:07.350370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342617388696716:3490]
2025-10-09T21:50:07.350373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342617388696716:3490]
2025-10-09T21:50:07.350387Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:07.350401Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:07.350404Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:07.350406Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:07.350407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:07.350423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:07.350430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:07.350471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342617388696716:3490]
2025-10-09T21:50:07.350479Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:07.350482Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:07.350445Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1724
2025-10-09T21:50:07.350445Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1724
2025-10-09T21:50:07.350519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.350529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.350563Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e1dc8b3-a281595e-906cf06b-97ccbf44_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:07.350575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:07.350579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:07.350584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:07.350595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:07.350609Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:07.350627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:07.350798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.350808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.350825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:07.350924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1725 partition: 0 MaxSeqNo: 25 sessionId: src_id|e1dc8b3-a281595e-906cf06b-97ccbf44_0
2025-10-09T21:50:07.351274Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|241479a5-20b1e58c-119f17ac-7e6c124d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e1dc8b3-a281595e-906cf06b-97ccbf44_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:07.353733Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1dc8b3-a281595e-906cf06b-97ccbf44_0] Write session: try to update token
2025-10-09T21:50:07.353754Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1dc8b3-a281595e-906cf06b-97ccbf44_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:07.365208Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:07.365229Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:07.365231Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:07.365233Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:07.365245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342617388696716:3490]
2025-10-09T21:50:07.365249Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342617388696716:3490]
2025-10-09T21:50:07.365253Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342617388696716:3490]
2025-10-09T21:50:07.365306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:07.365312Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:07.365318Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:07.365322Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:07.365324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.365327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:07.365465Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:07.367022Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1725 sessionId: src_id|e1dc8b3-a281595e-906cf06b-97ccbf44_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:07.369451Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:07.415308Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342617750527298:6864]
2025-10-09T21:50:07.415580Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 1724, pipe: [1:7559342617750527298:6864]
2025-10-09T21:50:07.518869Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:07.519103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.519122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.519162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342617750527298:6864], now have 1 active actors on pipe
2025-10-09T21:50:07.519378Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:07.519369Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342617750527298:6864]
2025-10-09T21:50:07.519370Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:07.519547Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:07.519557Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:07.519615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1725 reason: error at write: Disconnected sessionId: src_id|e1dc8b3-a281595e-906cf06b-97ccbf44_0
2025-10-09T21:50:07.519781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1725 sessionId: src_id|e1dc8b3-a281595e-906cf06b-97ccbf44_0 is DEAD
2025-10-09T21:50:07.519943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1dc8b3-a281595e-906cf06b-97ccbf44_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:07.519955Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1dc8b3-a281595e-906cf06b-97ccbf44_0] Write session will restart in 0.008976s
2025-10-09T21:50:07.519985Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1dc8b3-a281595e-906cf06b-97ccbf44_0] Write session: Do CDS request
2025-10-09T21:50:07.519992Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1dc8b3-a281595e-906cf06b-97ccbf44_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:07.530491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1dc8b3-a281595e-906cf06b-97ccbf44_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:07.530576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:07.530592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1726
2025-10-09T21:50:07.530702Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1726 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:07.530734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1726 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:07.530744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1726 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:07.530747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:07.531342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1726 sessionId:  describe result for acl check
2025-10-09T21:50:07.531381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:07.531387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:07.531389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:07.531395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342617750527387:6640] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:07.531937Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342617750527387:6640] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:07.534427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342617750527387:6640] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:07.536961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342617750527387:6640] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:07.536975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342617750527387:6640] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:07.536979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342617750527387:6640] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:07.536985Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1726 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:07.560112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:07.560806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:07.561002Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:07.561012Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:07.561023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:07.561073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:07.561113Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342617388697061:3468]
2025-10-09T21:50:07.561712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:07.562005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:07.562064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:07.562376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:07.562439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.562446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.562447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.562449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.562450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.562451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.562453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.562454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.562455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.562461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.562473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.562477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.562481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.562485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.562489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.562493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.562497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.562506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.562513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.562527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.562537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.562543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.562552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.562558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.562562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.562567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.562572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.562577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.562582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.562587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.562597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.562602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.562610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.562616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.562626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.562636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.562673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:07.569659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:07.571612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:07.571626Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:07.571628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:07.571630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:07.571636Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1725 [2:7559342617388697061:3468]
2025-10-09T21:50:07.571645Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:07.571660Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:07.571670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:07.571673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:07.571677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:07.571692Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:07.571700Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:07.571708Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:07.571765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.571773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.571775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.571778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:07.571844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:07.571855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.571868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.571880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.571898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:07.571923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342617750527518:6640], now have 1 active actors on pipe
2025-10-09T21:50:07.571932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342617750527326:2432], now have 1 active actors on pipe
2025-10-09T21:50:07.571939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342617388697056:3468]
2025-10-09T21:50:07.571946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342617388697056:3468]
2025-10-09T21:50:07.571948Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342617388697056:3468]
2025-10-09T21:50:07.571962Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:07.571976Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:07.571985Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:07.571988Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:07.571989Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:07.571997Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:07.571999Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:07.572020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342617388697056:3468]
2025-10-09T21:50:07.572028Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:07.572005Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1725
2025-10-09T21:50:07.572039Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:07.572060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.572069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.572075Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1725
2025-10-09T21:50:07.572106Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c51cc936-e20cf5c-9f7c8cba-dbadb55f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:07.572119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:07.572127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:07.572129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:07.572139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:07.572153Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:07.572171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:07.572346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.572356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.572371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:07.572475Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1726 partition: 0 MaxSeqNo: 25 sessionId: src_id|c51cc936-e20cf5c-9f7c8cba-dbadb55f_0
2025-10-09T21:50:07.572805Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1dc8b3-a281595e-906cf06b-97ccbf44_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c51cc936-e20cf5c-9f7c8cba-dbadb55f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:07.575300Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c51cc936-e20cf5c-9f7c8cba-dbadb55f_0] Write session: try to update token
2025-10-09T21:50:07.575315Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c51cc936-e20cf5c-9f7c8cba-dbadb55f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:07.586699Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:07.586723Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:07.586726Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:07.586728Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:07.586738Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342617388697056:3468]
2025-10-09T21:50:07.586743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342617388697056:3468]
2025-10-09T21:50:07.586747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342617388697056:3468]
2025-10-09T21:50:07.586798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:07.586802Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:07.586809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.586810Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:07.586813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:07.586813Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:07.587173Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:07.594955Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1726 sessionId: src_id|c51cc936-e20cf5c-9f7c8cba-dbadb55f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:07.597492Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:07.619587Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342617750527644:6864]
2025-10-09T21:50:07.619852Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 1725, pipe: [1:7559342617750527644:6864]
2025-10-09T21:50:07.736245Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:07.736909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.736927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.736951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342617750527644:6864], now have 1 active actors on pipe
2025-10-09T21:50:07.737234Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342617750527644:6864]
2025-10-09T21:50:07.737239Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:07.737243Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:07.737268Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:07.737296Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1726 reason: error at write: Disconnected sessionId: src_id|c51cc936-e20cf5c-9f7c8cba-dbadb55f_0
2025-10-09T21:50:07.737473Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1726 sessionId: src_id|c51cc936-e20cf5c-9f7c8cba-dbadb55f_0 is DEAD
2025-10-09T21:50:07.737640Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c51cc936-e20cf5c-9f7c8cba-dbadb55f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:07.737653Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c51cc936-e20cf5c-9f7c8cba-dbadb55f_0] Write session will restart in 0.006275s
2025-10-09T21:50:07.737678Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c51cc936-e20cf5c-9f7c8cba-dbadb55f_0] Write session: Do CDS request
2025-10-09T21:50:07.737688Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c51cc936-e20cf5c-9f7c8cba-dbadb55f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:07.737696Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:07.745395Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c51cc936-e20cf5c-9f7c8cba-dbadb55f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:07.745462Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:07.745474Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1727
2025-10-09T21:50:07.745616Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1727 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:07.745679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1727 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:07.745693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1727 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:07.745697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:07.746139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1727 sessionId:  describe result for acl check
2025-10-09T21:50:07.746176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:07.746183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:07.746185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:07.746191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342617750527684:6637] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:07.746742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342617750527684:6637] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:07.749876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342617750527684:6637] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:07.752879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342617750527684:6637] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:07.752891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342617750527684:6637] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:07.752894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342617750527684:6637] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:07.752901Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1727 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:07.757852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:07.763047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:07.763270Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:07.763280Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:07.763293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:07.763345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:07.763384Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342617388697377:3455]
2025-10-09T21:50:07.763979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:07.764274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:07.764336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:07.764671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:07.764716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.764721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.764722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.764723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.764723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.764724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.764725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.764725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.764726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.764729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.764736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.764738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.764741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.764743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.764745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.764747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.764749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.764751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.764758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.764766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.764772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.764777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.764782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.764787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.764791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.764794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.764796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.764799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.764802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.764805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.764808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.764811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.764813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.764816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.764819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.764822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.764838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:07.771058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:07.772334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:07.772349Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:07.772351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:07.772355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:07.772360Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1726 [2:7559342617388697377:3455]
2025-10-09T21:50:07.772371Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:07.772388Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:07.772390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:07.772393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:07.772395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:07.772410Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:07.772416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:07.772423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:07.772471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.772480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.772481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.772485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:07.772558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:07.772566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.772577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.772588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.772605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:07.772631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342617750527859:6637], now have 1 active actors on pipe
2025-10-09T21:50:07.772639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342617750527647:2432], now have 1 active actors on pipe
2025-10-09T21:50:07.772647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342617388697363:3455]
2025-10-09T21:50:07.772649Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342617388697363:3455]
2025-10-09T21:50:07.772651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342617388697363:3455]
2025-10-09T21:50:07.772667Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:07.772685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:07.772691Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:07.772694Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:07.772695Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:07.772705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:07.772708Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:07.772737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342617388697363:3455]
2025-10-09T21:50:07.772750Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:07.772721Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1726
2025-10-09T21:50:07.772723Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1726
2025-10-09T21:50:07.772759Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:07.772790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.772802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.772834Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b0722e1-ac83f193-3aede5c8-99696e93_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:07.772848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:07.772856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:07.772860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:07.772870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:07.772885Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:07.772908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:07.773045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.773055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.773073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:07.773200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1727 partition: 0 MaxSeqNo: 25 sessionId: src_id|b0722e1-ac83f193-3aede5c8-99696e93_0
2025-10-09T21:50:07.773549Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c51cc936-e20cf5c-9f7c8cba-dbadb55f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b0722e1-ac83f193-3aede5c8-99696e93_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:07.776232Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0722e1-ac83f193-3aede5c8-99696e93_0] Write session: try to update token
2025-10-09T21:50:07.776249Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0722e1-ac83f193-3aede5c8-99696e93_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:07.784330Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:07.784349Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:07.784358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342617388697363:3455]
2025-10-09T21:50:07.784395Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:07.784405Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:07.784974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:07.785317Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:07.785667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:07.786001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:07.786592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:07.786641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:07.790942Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1727 sessionId: src_id|b0722e1-ac83f193-3aede5c8-99696e93_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:07.793245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:07.793431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.793451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.793491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:07.793610Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:07.797591Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:07.797607Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:07.797609Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:07.797611Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:07.797620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342617388697363:3455]
2025-10-09T21:50:07.797624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342617388697363:3455]
2025-10-09T21:50:07.797628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342617388697363:3455]
2025-10-09T21:50:07.797667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:07.797668Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:07.797674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.797675Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:07.797677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:07.797679Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:07.797849Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:07.837152Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342617750527973:6864]
2025-10-09T21:50:07.837385Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 1726, pipe: [1:7559342617750527973:6864]
2025-10-09T21:50:07.945069Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:07.953048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.953080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.953260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.953340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.953417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.953496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.953562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.953573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.953656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.953735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.953812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.953897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.954051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.954062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.954125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.954227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.954297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.954361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.954407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.954415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.954480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.954546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.954651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.954715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.954850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.954858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.954926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.955034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.955143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.955211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.955276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.955285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.955349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.955415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.955478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.955538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.955582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.955589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.955652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.955712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.955776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.955846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.955985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.955993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.956058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.956119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.956183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.956278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.956329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.956337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.956426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.956488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.956551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.956616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.956659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.956670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.956731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:07.956788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:07.956851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:07.956922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:07.956970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:07.956978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:07.957038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342617750527973:6864], now have 1 active actors on pipe
2025-10-09T21:50:07.957315Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:07.957316Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:07.957320Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:07.957368Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1727 reason: error at write: Disconnected sessionId: src_id|b0722e1-ac83f193-3aede5c8-99696e93_0
2025-10-09T21:50:07.957369Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342617750527973:6864]
2025-10-09T21:50:07.957558Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1727 sessionId: src_id|b0722e1-ac83f193-3aede5c8-99696e93_0 is DEAD
2025-10-09T21:50:07.957714Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0722e1-ac83f193-3aede5c8-99696e93_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:07.957726Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0722e1-ac83f193-3aede5c8-99696e93_0] Write session will restart in 0.009370s
2025-10-09T21:50:07.957752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0722e1-ac83f193-3aede5c8-99696e93_0] Write session: Do CDS request
2025-10-09T21:50:07.957759Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0722e1-ac83f193-3aede5c8-99696e93_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:07.958110Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:07.964431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:07.965729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:07.965898Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:07.965909Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:07.965920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:07.965978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:07.966020Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342617388697434:3446]
2025-10-09T21:50:07.966627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:07.966882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:07.966922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:07.967235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:07.967282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.967288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.967290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.967291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.967292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.967294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.967295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.967297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.967298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.967303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.967314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.967324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.967329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.967333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.967338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.967342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.967346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.967350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.967357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.967370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:07.967380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.967390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:07.967398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.967405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:07.967413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.967419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:07.967425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.967432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:07.967436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.967441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:07.967446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.967451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.967455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.967460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.967469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.967474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.967507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:07.968434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0722e1-ac83f193-3aede5c8-99696e93_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:07.968524Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:07.968536Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1728
2025-10-09T21:50:07.968656Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1728 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:07.968696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1728 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:07.968706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1728 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:07.968710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:07.969043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1728 sessionId:  describe result for acl check
2025-10-09T21:50:07.969078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:07.969084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:07.969085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:07.969091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342617750528005:6632] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:07.969638Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342617750528005:6632] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:07.972158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342617750528005:6632] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:07.973852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:07.974524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342617750528005:6632] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:07.974534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342617750528005:6632] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:07.974537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342617750528005:6632] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:07.974544Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1728 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:07.974870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:07.974883Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:07.974885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:07.974888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:07.974893Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1727 [2:7559342617388697434:3446]
2025-10-09T21:50:07.974907Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:07.974924Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:07.974927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:07.974930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:07.974933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:07.974948Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:07.974954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:07.974962Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:07.975007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.975014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.975015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.975018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:07.975080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:07.975089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.975101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:07.975114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:07.975124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:07.975138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342617750527980:2432], now have 1 active actors on pipe
2025-10-09T21:50:07.975145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342617750528036:6632], now have 1 active actors on pipe
2025-10-09T21:50:07.975152Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342617388697429:3446]
2025-10-09T21:50:07.975154Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342617388697429:3446]
2025-10-09T21:50:07.975156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342617388697429:3446]
2025-10-09T21:50:07.975144Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1727
2025-10-09T21:50:07.975164Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:07.975152Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1727
2025-10-09T21:50:07.975175Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:07.975177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:07.975178Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:07.975179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:07.975193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:07.975197Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:07.975213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.975223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.975259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342617388697429:3446]
2025-10-09T21:50:07.975270Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:07.975274Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:07.975288Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5bf071e8-4d385dc3-67630f73-7a1d46cf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:07.975300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:07.975308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:07.975310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:07.975320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:07.975332Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:07.975371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:07.975545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.975554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.975566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:07.975630Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1728 partition: 0 MaxSeqNo: 25 sessionId: src_id|5bf071e8-4d385dc3-67630f73-7a1d46cf_0
2025-10-09T21:50:07.975903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0722e1-ac83f193-3aede5c8-99696e93_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5bf071e8-4d385dc3-67630f73-7a1d46cf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:07.978398Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5bf071e8-4d385dc3-67630f73-7a1d46cf_0] Write session: try to update token
2025-10-09T21:50:07.978412Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5bf071e8-4d385dc3-67630f73-7a1d46cf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:07.986420Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:07.986439Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:07.986448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342617388697429:3446]
2025-10-09T21:50:07.986488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:07.986509Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:07.986672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:07.986817Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:07.986954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:07.987086Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:07.987272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:07.987316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:07.993171Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1728 sessionId: src_id|5bf071e8-4d385dc3-67630f73-7a1d46cf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:07.995511Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:07.995694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:07.995716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:07.995754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:07.995815Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:07.999373Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:07.999390Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:07.999392Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:07.999395Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:07.999405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342617388697429:3446]
2025-10-09T21:50:07.999409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342617388697429:3446]
2025-10-09T21:50:07.999413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342617388697429:3446]
2025-10-09T21:50:07.999447Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:07.999452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:07.999459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:07.999459Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:07.999462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:07.999463Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:07.999638Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:08.057363Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342622045495340:6864]
2025-10-09T21:50:08.057669Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 1727, pipe: [1:7559342622045495340:6864]
2025-10-09T21:50:08.147370Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:08.154202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:08.154226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:08.154396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.154460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.154527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.154594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.154644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.154652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.154724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.154789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.154867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.154942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.154992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.154999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.155057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.155117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.155173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.155228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.155268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.155276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.155337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.155392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.155446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.155495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.155533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.155538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.155594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.155645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.155699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.155804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.155853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.155861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.155954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.156009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.156063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.156116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.156156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.156162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.156229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.156314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.156367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.156418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.156456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.156465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.156519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.156571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.156623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.156675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.156714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.156719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.156773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.156826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.156880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.156935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.156988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.156994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.157045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.157098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.157180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.157258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.157309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.157317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.157364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342622045495340:6864], now have 1 active actors on pipe
2025-10-09T21:50:08.157620Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:08.157623Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342622045495340:6864]
2025-10-09T21:50:08.157653Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:08.157656Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:08.157699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1728 reason: error at write: Disconnected sessionId: src_id|5bf071e8-4d385dc3-67630f73-7a1d46cf_0
2025-10-09T21:50:08.157897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1728 sessionId: src_id|5bf071e8-4d385dc3-67630f73-7a1d46cf_0 is DEAD
2025-10-09T21:50:08.158017Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5bf071e8-4d385dc3-67630f73-7a1d46cf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:08.158024Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5bf071e8-4d385dc3-67630f73-7a1d46cf_0] Write session will restart in 0.005120s
2025-10-09T21:50:08.158061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5bf071e8-4d385dc3-67630f73-7a1d46cf_0] Write session: Do CDS request
2025-10-09T21:50:08.158067Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5bf071e8-4d385dc3-67630f73-7a1d46cf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:08.158287Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:08.164484Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5bf071e8-4d385dc3-67630f73-7a1d46cf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:08.164559Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:08.164573Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1729
2025-10-09T21:50:08.164619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:08.164731Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1729 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:08.164762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1729 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:08.164772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1729 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:08.164776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:08.165142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1729 sessionId:  describe result for acl check
2025-10-09T21:50:08.165177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:08.165179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:08.165180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:08.165184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342622045495364:6613] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:08.165664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342622045495364:6613] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:08.166108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:08.166359Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:08.166372Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:08.166387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:08.166453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:08.166494Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342621683664787:3441]
2025-10-09T21:50:08.167065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:08.167414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:08.167469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:08.167801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:08.167874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.167881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.167883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.167884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.167886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.167887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.167888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.167891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.167892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.167897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.167907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.167912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.167916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.167920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.167925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.167929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.167932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.167937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.167943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.167951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.167958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.167971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.167975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.167980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.167984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.167989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.167994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.167999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.168005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.168010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.168014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.167995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342622045495364:6613] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:08.168019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.168022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.168028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.168033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.168038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.168068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:08.173960Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342622045495364:6613] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:08.173977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342622045495364:6613] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:08.173983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342622045495364:6613] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:08.173993Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1729 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:08.174451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:08.175602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:08.175618Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:08.175620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:08.175624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:08.175631Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1728 [2:7559342621683664787:3441]
2025-10-09T21:50:08.175644Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:08.175661Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:08.175669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:08.175673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:08.175675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:08.175690Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:08.175701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:08.175714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:08.175773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.175780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.175781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.175784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:08.175848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:08.175855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.175866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.175885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.175907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:08.175932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342622045495344:2432], now have 1 active actors on pipe
2025-10-09T21:50:08.175939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342622045495404:6613], now have 1 active actors on pipe
2025-10-09T21:50:08.175948Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342621683664782:3441]
2025-10-09T21:50:08.175955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342621683664782:3441]
2025-10-09T21:50:08.175957Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342621683664782:3441]
2025-10-09T21:50:08.175969Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:08.175961Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1728
2025-10-09T21:50:08.175965Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1728
2025-10-09T21:50:08.175987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:08.175987Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:08.175990Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:08.175992Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:08.176006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:08.176009Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:08.176127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:08.176137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:08.176150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342621683664782:3441]
2025-10-09T21:50:08.176161Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:08.176180Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:08.176197Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1b798b6f-2e898890-151e9677-bb6f1892_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:08.176209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:08.176218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:08.176222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:08.176232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:08.176246Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:08.176273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:08.176443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:08.176459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:08.176483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:08.176583Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1729 partition: 0 MaxSeqNo: 25 sessionId: src_id|1b798b6f-2e898890-151e9677-bb6f1892_0
2025-10-09T21:50:08.176889Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5bf071e8-4d385dc3-67630f73-7a1d46cf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1b798b6f-2e898890-151e9677-bb6f1892_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:08.179106Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b798b6f-2e898890-151e9677-bb6f1892_0] Write session: try to update token
2025-10-09T21:50:08.179122Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b798b6f-2e898890-151e9677-bb6f1892_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:08.190562Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:08.190582Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:08.190584Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:08.190586Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:08.190597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342621683664782:3441]
2025-10-09T21:50:08.190601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342621683664782:3441]
2025-10-09T21:50:08.190605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342621683664782:3441]
2025-10-09T21:50:08.190650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:08.190654Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:08.190659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.190660Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:08.190662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:08.190664Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:08.190796Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:08.193196Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1729 sessionId: src_id|1b798b6f-2e898890-151e9677-bb6f1892_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:08.195653Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:08.258066Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342622045495414:6864]
2025-10-09T21:50:08.258375Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 1728, pipe: [1:7559342622045495414:6864]
2025-10-09T21:50:08.338361Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:08.338647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:08.338665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:08.338709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342622045495414:6864], now have 1 active actors on pipe
2025-10-09T21:50:08.338984Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:08.338985Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342622045495414:6864]
2025-10-09T21:50:08.339000Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:08.339005Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:08.339099Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1729 reason: error at write: Disconnected sessionId: src_id|1b798b6f-2e898890-151e9677-bb6f1892_0
2025-10-09T21:50:08.339197Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:08.339267Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1729 sessionId: src_id|1b798b6f-2e898890-151e9677-bb6f1892_0 is DEAD
2025-10-09T21:50:08.339423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b798b6f-2e898890-151e9677-bb6f1892_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:08.339432Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b798b6f-2e898890-151e9677-bb6f1892_0] Write session will restart in 0.007442s
2025-10-09T21:50:08.339452Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b798b6f-2e898890-151e9677-bb6f1892_0] Write session: Do CDS request
2025-10-09T21:50:08.339457Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b798b6f-2e898890-151e9677-bb6f1892_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:08.345520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:08.347003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:08.347180Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:08.347187Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:08.347197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:08.347238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:08.347269Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342621683664845:3437]
2025-10-09T21:50:08.347702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:08.348011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:08.348049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:08.348273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:08.348305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.348309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.348310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.348311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.348312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.348312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.348313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.348313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.348316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.348320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.348327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.348329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.348331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.348333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.348336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.348338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.348340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.348342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.348348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.348353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.348357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.348360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.348363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.348366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.348369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.348372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.348375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.348378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.348380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.348383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.348386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.348389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.348391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.348394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.348397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.348400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.348416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:08.348496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b798b6f-2e898890-151e9677-bb6f1892_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:08.348571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:08.348585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1730
2025-10-09T21:50:08.348689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1730 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:08.348712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1730 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:08.348720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1730 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:08.348723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:08.349064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1730 sessionId:  describe result for acl check
2025-10-09T21:50:08.349092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:08.349097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:08.349098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:08.349102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342622045495448:6614] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:08.349553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342622045495448:6614] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:08.351478Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342622045495448:6614] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:08.354052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342622045495448:6614] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:08.354065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342622045495448:6614] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:08.354068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342622045495448:6614] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:08.354075Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1730 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:08.354583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:08.355704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:08.355719Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:08.355728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:08.355732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:08.355740Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1729 [2:7559342621683664845:3437]
2025-10-09T21:50:08.355752Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:08.355769Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:08.355778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:08.355781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:08.355784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:08.355801Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:08.355811Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:08.355818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:08.355859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.355864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.355865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.355867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:08.355917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:08.355922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.355928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.355934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.355944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:08.355960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342621683664840:3437]
2025-10-09T21:50:08.355965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342621683664840:3437]
2025-10-09T21:50:08.355966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342621683664840:3437]
2025-10-09T21:50:08.355975Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:08.355987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342622045495483:6614], now have 1 active actors on pipe
2025-10-09T21:50:08.355990Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:08.355993Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:08.355995Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:08.355995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342622045495423:2432], now have 1 active actors on pipe
2025-10-09T21:50:08.356002Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:08.356011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:08.356015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:08.356036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342621683664840:3437]
2025-10-09T21:50:08.356041Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:08.356052Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:08.356043Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1729
2025-10-09T21:50:08.356049Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1729
2025-10-09T21:50:08.356107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:08.356114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:08.356137Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|872b4847-2393d34c-63256ed5-40c18ba6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:08.356147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:08.356150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:08.356151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:08.356160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:08.356171Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:08.356182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:08.356307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:08.356312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:08.356322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:08.358093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1730 partition: 0 MaxSeqNo: 25 sessionId: src_id|872b4847-2393d34c-63256ed5-40c18ba6_0
2025-10-09T21:50:08.358401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b798b6f-2e898890-151e9677-bb6f1892_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|872b4847-2393d34c-63256ed5-40c18ba6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:08.361159Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|872b4847-2393d34c-63256ed5-40c18ba6_0] Write session: try to update token
2025-10-09T21:50:08.361178Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|872b4847-2393d34c-63256ed5-40c18ba6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:08.367795Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:08.367814Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:08.367824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342621683664840:3437]
2025-10-09T21:50:08.367862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:08.367897Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:08.368023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:08.368161Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:08.368295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:08.368524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:08.368704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:08.368740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:08.374451Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1730 sessionId: src_id|872b4847-2393d34c-63256ed5-40c18ba6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:08.376879Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:08.377085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:08.377108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:08.377161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:08.377259Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:08.380616Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:08.380633Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:08.380635Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:08.380637Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:08.380647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342621683664840:3437]
2025-10-09T21:50:08.380651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342621683664840:3437]
2025-10-09T21:50:08.380655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342621683664840:3437]
2025-10-09T21:50:08.380695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:08.380709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.380712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:08.380727Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:08.380746Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:08.380751Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:08.380844Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:08.439301Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342622045495490:6864]
2025-10-09T21:50:08.439600Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 1729, pipe: [1:7559342622045495490:6864]
2025-10-09T21:50:08.528321Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:08.536044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:08.536072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:08.536222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.536285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.536349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.536414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.536459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.536471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.536540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.536601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.536662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.536726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.536769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.536778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.536840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.536940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.537040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.537099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.537142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.537149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.537214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.537274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.537365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.537428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.537469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.537476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.537553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.537615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.537675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.537734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.537776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.537783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.537844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.537901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.537959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.538021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.538081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.538089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.538146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.538240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.538297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.538356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.538413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.538422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.538481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.538538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.538594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.538653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.538801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.538810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.538867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.538921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.538981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.539041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.539084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.539091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.539149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:08.539208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:08.539265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:08.539326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:08.539442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:08.539449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:08.539502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342622045495490:6864], now have 1 active actors on pipe
2025-10-09T21:50:08.539695Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342622045495490:6864]
2025-10-09T21:50:08.539739Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:08.539741Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:08.539767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1730 reason: error at write: Disconnected sessionId: src_id|872b4847-2393d34c-63256ed5-40c18ba6_0
2025-10-09T21:50:08.539936Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1730 sessionId: src_id|872b4847-2393d34c-63256ed5-40c18ba6_0 is DEAD
2025-10-09T21:50:08.540084Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|872b4847-2393d34c-63256ed5-40c18ba6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:08.540093Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|872b4847-2393d34c-63256ed5-40c18ba6_0] Write session will restart in 0.006174s
2025-10-09T21:50:08.540110Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|872b4847-2393d34c-63256ed5-40c18ba6_0] Write session: Do CDS request
2025-10-09T21:50:08.540115Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|872b4847-2393d34c-63256ed5-40c18ba6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:08.540108Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:08.540515Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:08.546651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:08.547429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|872b4847-2393d34c-63256ed5-40c18ba6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:08.547520Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:08.547537Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1731
2025-10-09T21:50:08.547630Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1731 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:08.547653Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1731 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:08.547659Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1731 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:08.547662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:08.548011Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1731 sessionId:  describe result for acl check
2025-10-09T21:50:08.548038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:08.548042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:08.548043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:08.548047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342622045495540:6594] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:08.548046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:08.548208Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:08.548222Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:08.548234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:08.548282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:08.548330Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342621683664943:3379]
2025-10-09T21:50:08.548668Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342622045495540:6594] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:08.548924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:08.549202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:08.549247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:08.549568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:08.549608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.549613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.549614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.549615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.549615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.549616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.549617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.549617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.549618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.549621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.549629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.549631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.549633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.549635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.549637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.549639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.549641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.549643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.549647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.549652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.549656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.549659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.549661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.549664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.549666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.549669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.549671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.549674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.549677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.549680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.549682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.549685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.549688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.549691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.549693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.549696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.549711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:08.550462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342622045495540:6594] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:08.555361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342622045495540:6594] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:08.555373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342622045495540:6594] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:08.555375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342622045495540:6594] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:08.555383Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1731 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:08.555768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:08.556800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:08.556819Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:08.556821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:08.556825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:08.556833Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1730 [2:7559342621683664943:3379]
2025-10-09T21:50:08.556846Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:08.556864Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:08.556872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:08.556875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:08.556878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:08.556894Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:08.556904Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:08.556911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:08.556968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.556977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.556978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.556981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:08.557043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:08.557051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.557066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.557082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.557101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:08.557125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342622045495575:6594], now have 1 active actors on pipe
2025-10-09T21:50:08.557132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342622045495513:2432], now have 1 active actors on pipe
2025-10-09T21:50:08.557139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342621683664938:3379]
2025-10-09T21:50:08.557145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342621683664938:3379]
2025-10-09T21:50:08.557147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342621683664938:3379]
2025-10-09T21:50:08.557159Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:08.557153Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1730
2025-10-09T21:50:08.557172Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1730
2025-10-09T21:50:08.557174Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:08.557179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:08.557184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:08.557186Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:08.557193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:08.557195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:08.557236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:08.557246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:08.557260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342621683664938:3379]
2025-10-09T21:50:08.557270Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:08.557287Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:08.557304Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bfa52694-ab111377-8e950b0a-4782952e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:08.557315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:08.557324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:08.557327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:08.557337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:08.557349Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:08.557377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:08.557516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:08.557526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:08.557541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:08.557633Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1731 partition: 0 MaxSeqNo: 25 sessionId: src_id|bfa52694-ab111377-8e950b0a-4782952e_0
2025-10-09T21:50:08.557930Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|872b4847-2393d34c-63256ed5-40c18ba6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bfa52694-ab111377-8e950b0a-4782952e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:08.560334Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bfa52694-ab111377-8e950b0a-4782952e_0] Write session: try to update token
2025-10-09T21:50:08.560350Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bfa52694-ab111377-8e950b0a-4782952e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:08.570941Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:08.570964Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:08.570965Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:08.570967Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:08.570975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342621683664938:3379]
2025-10-09T21:50:08.570980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342621683664938:3379]
2025-10-09T21:50:08.570983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342621683664938:3379]
2025-10-09T21:50:08.571017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:08.571022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.571026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:08.571029Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:08.571035Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:08.571038Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:08.571175Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:08.572853Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1731 sessionId: src_id|bfa52694-ab111377-8e950b0a-4782952e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:08.575074Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:08.639977Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342622045495582:6864]
2025-10-09T21:50:08.640210Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 1730, pipe: [1:7559342622045495582:6864]
2025-10-09T21:50:08.708514Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:08.708828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:08.708840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:08.708878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342622045495582:6864], now have 1 active actors on pipe
2025-10-09T21:50:08.709045Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:08.709110Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:08.709110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:08.709115Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:08.709167Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342622045495582:6864]
2025-10-09T21:50:08.709189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1731 reason: error at write: Disconnected sessionId: src_id|bfa52694-ab111377-8e950b0a-4782952e_0
2025-10-09T21:50:08.709368Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1731 sessionId: src_id|bfa52694-ab111377-8e950b0a-4782952e_0 is DEAD
2025-10-09T21:50:08.709538Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfa52694-ab111377-8e950b0a-4782952e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:08.709559Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfa52694-ab111377-8e950b0a-4782952e_0] Write session will restart in 0.008257s
2025-10-09T21:50:08.709587Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfa52694-ab111377-8e950b0a-4782952e_0] Write session: Do CDS request
2025-10-09T21:50:08.709593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfa52694-ab111377-8e950b0a-4782952e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:08.715636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:08.716888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:08.717058Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:08.717068Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:08.717080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:08.717132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:08.717177Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342621683665002:3378]
2025-10-09T21:50:08.717793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:08.718102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:08.718146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:08.718465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bfa52694-ab111377-8e950b0a-4782952e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:08.718465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:08.718512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.718519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.718521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.718522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.718523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.718525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.718526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.718528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.718529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.718534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.718545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.718550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.718554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.718558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.718534Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:08.718562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.718550Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1732
2025-10-09T21:50:08.718567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.718572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.718576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.718583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.718592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.718597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.718608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.718612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.718645Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1732 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:08.718623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.718632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.718638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.718642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.718647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.718651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.718657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.718663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.718668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.718672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.718671Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1732 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:08.718677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.718679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1732 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:08.718682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.718682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:08.718687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.718721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:08.718994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1732 sessionId:  describe result for acl check
2025-10-09T21:50:08.719021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:08.719022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:08.719023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:08.719027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342622045495612:6591] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:08.719457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342622045495612:6591] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:08.721109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342622045495612:6591] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:08.724549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342622045495612:6591] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:08.724563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342622045495612:6591] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:08.724567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342622045495612:6591] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:08.724575Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1732 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:08.725073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:08.726148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:08.726164Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:08.726166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:08.726170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:08.726176Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1731 [2:7559342621683665002:3378]
2025-10-09T21:50:08.726190Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:08.726211Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:08.726221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:08.726225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:08.726227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:08.726243Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:08.726250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:08.726257Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:08.726313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.726323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.726325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.726327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:08.726393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:08.726401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.726412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.726424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.726446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:08.726472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342622045495647:6591], now have 1 active actors on pipe
2025-10-09T21:50:08.726480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342622045495585:2432], now have 1 active actors on pipe
2025-10-09T21:50:08.726487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342621683664997:3378]
2025-10-09T21:50:08.726491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342621683664997:3378]
2025-10-09T21:50:08.726493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342621683664997:3378]
2025-10-09T21:50:08.726486Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1731
2025-10-09T21:50:08.726492Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1731
2025-10-09T21:50:08.726505Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:08.726517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:08.726521Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:08.726522Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:08.726526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:08.726541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:08.726544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:08.726588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:08.726599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:08.726612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342621683664997:3378]
2025-10-09T21:50:08.726625Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:08.726643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:08.726659Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3a6d40c4-130f54cc-6436f40f-69af59df_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:08.726667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:08.726676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:08.726680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:08.726690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:08.726703Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:08.726727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:08.726856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:08.726867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:08.726881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:08.726958Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1732 partition: 0 MaxSeqNo: 25 sessionId: src_id|3a6d40c4-130f54cc-6436f40f-69af59df_0
2025-10-09T21:50:08.727243Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfa52694-ab111377-8e950b0a-4782952e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3a6d40c4-130f54cc-6436f40f-69af59df_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:08.729548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a6d40c4-130f54cc-6436f40f-69af59df_0] Write session: try to update token
2025-10-09T21:50:08.729563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a6d40c4-130f54cc-6436f40f-69af59df_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:08.740931Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:08.740952Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:08.740954Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:08.740956Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:08.740965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342621683664997:3378]
2025-10-09T21:50:08.740972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342621683664997:3378]
2025-10-09T21:50:08.740976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342621683664997:3378]
2025-10-09T21:50:08.741021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:08.741022Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:08.741028Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:08.741032Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:08.741034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.741037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:08.741165Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:08.741801Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1732 sessionId: src_id|3a6d40c4-130f54cc-6436f40f-69af59df_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:08.744142Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:08.809493Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342622045495654:6864]
2025-10-09T21:50:08.809782Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 1731, pipe: [1:7559342622045495654:6864]
2025-10-09T21:50:08.886391Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:08.886659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:08.886675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:08.886719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342622045495654:6864], now have 1 active actors on pipe
2025-10-09T21:50:08.886936Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:08.886972Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:08.886950Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:08.886970Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342622045495654:6864]
2025-10-09T21:50:08.886995Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:08.887006Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1732 reason: error at write: Disconnected sessionId: src_id|3a6d40c4-130f54cc-6436f40f-69af59df_0
2025-10-09T21:50:08.887171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1732 sessionId: src_id|3a6d40c4-130f54cc-6436f40f-69af59df_0 is DEAD
2025-10-09T21:50:08.887310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a6d40c4-130f54cc-6436f40f-69af59df_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:08.887322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a6d40c4-130f54cc-6436f40f-69af59df_0] Write session will restart in 0.007638s
2025-10-09T21:50:08.887350Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a6d40c4-130f54cc-6436f40f-69af59df_0] Write session: Do CDS request
2025-10-09T21:50:08.887355Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a6d40c4-130f54cc-6436f40f-69af59df_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:08.893398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:08.894838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:08.895010Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:08.895019Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:08.895030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:08.895080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:08.895125Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342621683665067:3368]
2025-10-09T21:50:08.895703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:08.895993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:08.896037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:08.896345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:08.896395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.896401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.896402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.896404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.896405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.896406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.896409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.896411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.896413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.896417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.896428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.896432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.896436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.896439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.896443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.896446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.896449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.896471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3a6d40c4-130f54cc-6436f40f-69af59df_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:08.896452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.896459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.896467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:08.896474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.896479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:08.896484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.896489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:08.896493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.896499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:08.896503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.896508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:08.896513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.896518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:08.896523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.896529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.896538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.896543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.896548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.896553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.896561Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:08.896575Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1733
2025-10-09T21:50:08.896583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:08.896671Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1733 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:08.896694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1733 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:08.896698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1733 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:08.896701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:08.896991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1733 sessionId:  describe result for acl check
2025-10-09T21:50:08.897025Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:08.897027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:08.897029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:08.897034Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342622045495687:6582] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:08.897435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342622045495687:6582] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:08.899101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342622045495687:6582] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:08.902251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342622045495687:6582] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:08.902262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342622045495687:6582] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:08.902265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342622045495687:6582] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:08.902280Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1733 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:08.902845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:08.904549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:08.904565Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:08.904568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:08.904571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:08.904578Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1732 [2:7559342621683665067:3368]
2025-10-09T21:50:08.904595Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:08.904610Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:08.904618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:08.904621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:08.904624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:08.904640Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:08.904651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:08.904662Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:08.904716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.904722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.904724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.904727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:08.904796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:08.904803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.904810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:08.904820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:08.904833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:08.904851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342622045495722:6582], now have 1 active actors on pipe
2025-10-09T21:50:08.904857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342622045495658:2432], now have 1 active actors on pipe
2025-10-09T21:50:08.904862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342621683665062:3368]
2025-10-09T21:50:08.904866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342621683665062:3368]
2025-10-09T21:50:08.904870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342621683665062:3368]
2025-10-09T21:50:08.904879Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:08.904893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:08.904904Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1732
2025-10-09T21:50:08.904906Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1732
2025-10-09T21:50:08.904907Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:08.904909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:08.904912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:08.904919Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:08.904921Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:08.904939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342621683665062:3368]
2025-10-09T21:50:08.904942Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:08.904944Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:08.904956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:08.904959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:08.905000Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5a2ea86a-c787c692-a816dfec-e1a31b76_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:08.905015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:08.905023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:08.905026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:08.905036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:08.905050Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:08.905070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:08.905207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:08.905214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:08.905228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:08.905332Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1733 partition: 0 MaxSeqNo: 25 sessionId: src_id|5a2ea86a-c787c692-a816dfec-e1a31b76_0
2025-10-09T21:50:08.905652Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3a6d40c4-130f54cc-6436f40f-69af59df_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5a2ea86a-c787c692-a816dfec-e1a31b76_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:08.907907Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a2ea86a-c787c692-a816dfec-e1a31b76_0] Write session: try to update token
2025-10-09T21:50:08.907921Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a2ea86a-c787c692-a816dfec-e1a31b76_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:08.919483Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:08.919504Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:08.919506Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:08.919508Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:08.919517Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342621683665062:3368]
2025-10-09T21:50:08.919521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342621683665062:3368]
2025-10-09T21:50:08.919528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342621683665062:3368]
2025-10-09T21:50:08.919572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:08.919579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:08.919583Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:08.919584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:08.919590Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:08.919594Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:08.919723Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:08.922802Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1733 sessionId: src_id|5a2ea86a-c787c692-a816dfec-e1a31b76_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:08.925152Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:08.987135Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342622045495729:6864]
2025-10-09T21:50:08.987370Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 1732, pipe: [1:7559342622045495729:6864]
2025-10-09T21:50:09.065819Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:09.066090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.066107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.066150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342622045495729:6864], now have 1 active actors on pipe
2025-10-09T21:50:09.066343Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:09.066357Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342622045495729:6864]
2025-10-09T21:50:09.066381Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:09.066384Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:09.066392Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:09.066435Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1733 reason: error at write: Disconnected sessionId: src_id|5a2ea86a-c787c692-a816dfec-e1a31b76_0
2025-10-09T21:50:09.066597Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1733 sessionId: src_id|5a2ea86a-c787c692-a816dfec-e1a31b76_0 is DEAD
2025-10-09T21:50:09.066718Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a2ea86a-c787c692-a816dfec-e1a31b76_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:09.066726Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a2ea86a-c787c692-a816dfec-e1a31b76_0] Write session will restart in 0.009732s
2025-10-09T21:50:09.066754Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a2ea86a-c787c692-a816dfec-e1a31b76_0] Write session: Do CDS request
2025-10-09T21:50:09.066759Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a2ea86a-c787c692-a816dfec-e1a31b76_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:09.073327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:09.074679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:09.074865Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:09.074875Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:09.074886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:09.074939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:09.074980Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342625978632425:3372]
2025-10-09T21:50:09.075566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:09.075830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:09.075875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:09.076190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:09.076239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.076247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.076248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.076250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.076251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.076252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.076254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.076255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.076256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.076261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.076272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.076276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.076281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.076284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.076288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.076293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.076297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.076303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.076309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.076318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.076329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.076338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.076347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.076352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.076361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.076371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.076376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.076386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.076391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.076402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.076411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.076420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.076425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.076435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.076444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.076453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.076488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:09.077552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a2ea86a-c787c692-a816dfec-e1a31b76_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:09.077642Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:09.077656Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1734
2025-10-09T21:50:09.077788Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1734 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:09.077816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1734 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:09.077827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1734 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:09.077831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:09.078220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1734 sessionId:  describe result for acl check
2025-10-09T21:50:09.078258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:09.078267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:09.078268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:09.078274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342626340463057:6577] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:09.078808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342626340463057:6577] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:09.080589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342626340463057:6577] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:09.082836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342626340463057:6577] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:09.082849Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342626340463057:6577] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:09.082852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342626340463057:6577] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:09.082858Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1734 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:09.082871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:09.084111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:09.084132Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:09.084134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:09.084139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:09.084145Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1733 [2:7559342625978632425:3372]
2025-10-09T21:50:09.084157Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:09.084176Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:09.084179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:09.084182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:09.084186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:09.084209Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:09.084219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:09.084226Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:09.084282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.084288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.084291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.084295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:09.084364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:09.084372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.084384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.084397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.084418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:09.084437Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342625978632420:3372]
2025-10-09T21:50:09.084440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342625978632420:3372]
2025-10-09T21:50:09.084442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342625978632420:3372]
2025-10-09T21:50:09.084453Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:09.084466Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:09.084474Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:09.084477Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:09.084479Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:09.084481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:09.084483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:09.084512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342625978632420:3372]
2025-10-09T21:50:09.084522Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:09.084530Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:09.084532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463028:2432], now have 1 active actors on pipe
2025-10-09T21:50:09.084544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463092:6577], now have 1 active actors on pipe
2025-10-09T21:50:09.084623Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1733
2025-10-09T21:50:09.084626Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1733
2025-10-09T21:50:09.084693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.084707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.084742Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4e635e06-9ca16fd3-695ad140-dbce1c0b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:09.084754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:09.084757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:09.084758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:09.084769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:09.084778Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:09.084791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:09.084920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.084926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.084936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:09.084995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1734 partition: 0 MaxSeqNo: 25 sessionId: src_id|4e635e06-9ca16fd3-695ad140-dbce1c0b_0
2025-10-09T21:50:09.085323Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a2ea86a-c787c692-a816dfec-e1a31b76_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4e635e06-9ca16fd3-695ad140-dbce1c0b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:09.087698Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e635e06-9ca16fd3-695ad140-dbce1c0b_0] Write session: try to update token
2025-10-09T21:50:09.087714Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e635e06-9ca16fd3-695ad140-dbce1c0b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:09.099113Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:09.099131Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:09.099134Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:09.099136Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:09.099147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342625978632420:3372]
2025-10-09T21:50:09.099152Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342625978632420:3372]
2025-10-09T21:50:09.099159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342625978632420:3372]
2025-10-09T21:50:09.099199Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:09.099204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:09.099205Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:09.099209Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:09.099211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.099214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:09.099352Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:09.101724Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1734 sessionId: src_id|4e635e06-9ca16fd3-695ad140-dbce1c0b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:09.104255Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:09.166331Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342626340463099:6864]
2025-10-09T21:50:09.166667Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 1733, pipe: [1:7559342626340463099:6864]
2025-10-09T21:50:09.248545Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:09.248856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.248870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.248905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463099:6864], now have 1 active actors on pipe
2025-10-09T21:50:09.249058Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:09.249151Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:09.249170Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342626340463099:6864]
2025-10-09T21:50:09.249194Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:09.249268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1734 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4e635e06-9ca16fd3-695ad140-dbce1c0b_0
2025-10-09T21:50:09.249411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1734 sessionId: src_id|4e635e06-9ca16fd3-695ad140-dbce1c0b_0 is DEAD
2025-10-09T21:50:09.249579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e635e06-9ca16fd3-695ad140-dbce1c0b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:09.249597Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e635e06-9ca16fd3-695ad140-dbce1c0b_0] Write session will restart in 0.006489s
2025-10-09T21:50:09.249620Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e635e06-9ca16fd3-695ad140-dbce1c0b_0] Write session: Do CDS request
2025-10-09T21:50:09.249625Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e635e06-9ca16fd3-695ad140-dbce1c0b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:09.256259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:09.257423Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e635e06-9ca16fd3-695ad140-dbce1c0b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:09.257564Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:09.257579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1735
2025-10-09T21:50:09.257677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:09.257717Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1735 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:09.257748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1735 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:09.257758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1735 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:09.257762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:09.257837Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:09.257847Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:09.257857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:09.257897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:09.257930Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342625978632488:3369]
2025-10-09T21:50:09.258195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1735 sessionId:  describe result for acl check
2025-10-09T21:50:09.258234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:09.258241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:09.258242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:09.258246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342626340463131:6571] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:09.258402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:09.258659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:09.258693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:09.258714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342626340463131:6571] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:09.258936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:09.258990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.258998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.258999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.259000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.259002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.259003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.259004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.259006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.259007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.259012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.259023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.259026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.259028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.259030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.259032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.259034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.259038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.259040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.259045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.259050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.259054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.259058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.259061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.259064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.259066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.259069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.259073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.259076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.259081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.259086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.259090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.259095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.259099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.259104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.259109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.259114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.259142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:09.260694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342626340463131:6571] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:09.265172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342626340463131:6571] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:09.265191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342626340463131:6571] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:09.265194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342626340463131:6571] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:09.265205Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1735 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:09.265693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:09.266686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:09.266700Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:09.266701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:09.266705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:09.266710Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1734 [2:7559342625978632488:3369]
2025-10-09T21:50:09.266718Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:09.266734Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:09.266740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:09.266743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:09.266744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:09.266757Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:09.266761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:09.266771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:09.266829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.266835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.266836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.266838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:09.266897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:09.266902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.266913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.266929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.266947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:09.266966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463105:2432], now have 1 active actors on pipe
2025-10-09T21:50:09.266972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463175:6571], now have 1 active actors on pipe
2025-10-09T21:50:09.266977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342625978632483:3369]
2025-10-09T21:50:09.266979Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342625978632483:3369]
2025-10-09T21:50:09.266980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342625978632483:3369]
2025-10-09T21:50:09.266990Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:09.267002Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:09.267010Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:09.267013Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:09.267013Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:09.267015Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:09.267015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:09.267040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342625978632483:3369]
2025-10-09T21:50:09.267048Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:09.267038Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1734
2025-10-09T21:50:09.267064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:09.267046Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1734
2025-10-09T21:50:09.267076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.267079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.267109Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b4f6e249-d2ba268b-850fef8f-ce1c7a69_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:09.267122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:09.267127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:09.267130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:09.267140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:09.267155Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:09.267165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:09.267297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.267305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.267318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:09.267397Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1735 partition: 0 MaxSeqNo: 25 sessionId: src_id|b4f6e249-d2ba268b-850fef8f-ce1c7a69_0
2025-10-09T21:50:09.267704Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e635e06-9ca16fd3-695ad140-dbce1c0b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b4f6e249-d2ba268b-850fef8f-ce1c7a69_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:09.270382Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4f6e249-d2ba268b-850fef8f-ce1c7a69_0] Write session: try to update token
2025-10-09T21:50:09.270399Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4f6e249-d2ba268b-850fef8f-ce1c7a69_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:09.283551Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1735 sessionId: src_id|b4f6e249-d2ba268b-850fef8f-ce1c7a69_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:09.283555Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:09.283574Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:09.283582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342625978632483:3369]
2025-10-09T21:50:09.283625Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:09.283628Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:09.283802Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:09.283929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:09.284048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:09.284212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:09.284426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:09.284470Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:09.286154Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:09.286343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.286357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.286385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:09.286451Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:09.291174Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:09.291193Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:09.291196Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:09.291198Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:09.291207Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342625978632483:3369]
2025-10-09T21:50:09.291211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342625978632483:3369]
2025-10-09T21:50:09.291216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342625978632483:3369]
2025-10-09T21:50:09.291252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:09.291263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.291262Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:09.291265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:09.291267Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:09.291269Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:09.291392Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:09.349279Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342626340463183:6864]
2025-10-09T21:50:09.349498Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 1734, pipe: [1:7559342626340463183:6864]
2025-10-09T21:50:09.438475Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:09.445050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.445075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.445214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:09.445270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:09.445332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:09.445394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:09.445439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:09.445448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:09.445520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:09.445590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:09.445646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:09.445702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:09.445740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:09.445746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:09.445799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:09.445851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:09.445906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:09.445956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:09.445999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:09.446006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:09.446082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:09.446140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:09.446194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:09.446248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:09.446287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:09.446292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:09.446348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:09.446406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:09.446507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:09.446562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:09.446676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:09.446682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:09.446735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:09.446812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:09.446892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:09.446944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:09.446984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:09.446992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:09.447090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:09.447139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:09.447192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:09.447258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:09.447308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:09.447313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:09.447364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:09.447422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:09.447484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:09.447537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:09.447588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:09.447594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:09.447658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:09.447747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:09.447833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:09.447918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:09.447963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:09.447969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:09.448023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:09.448072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:09.448147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:09.448202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:09.448240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:09.448247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:09.448286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463183:6864], now have 1 active actors on pipe
2025-10-09T21:50:09.448492Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:09.448536Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342626340463183:6864]
2025-10-09T21:50:09.448601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:09.448614Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:09.448646Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1735 reason: error at write: Disconnected sessionId: src_id|b4f6e249-d2ba268b-850fef8f-ce1c7a69_0
2025-10-09T21:50:09.448841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1735 sessionId: src_id|b4f6e249-d2ba268b-850fef8f-ce1c7a69_0 is DEAD
2025-10-09T21:50:09.448967Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4f6e249-d2ba268b-850fef8f-ce1c7a69_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:09.448977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4f6e249-d2ba268b-850fef8f-ce1c7a69_0] Write session will restart in 0.008774s
2025-10-09T21:50:09.448995Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4f6e249-d2ba268b-850fef8f-ce1c7a69_0] Write session: Do CDS request
2025-10-09T21:50:09.449000Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4f6e249-d2ba268b-850fef8f-ce1c7a69_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:09.448993Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:09.454813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:09.456067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:09.456227Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:09.456237Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:09.456247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:09.456296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:09.456331Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342625978632552:3364]
2025-10-09T21:50:09.456695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:09.456933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:09.456966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:09.457187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:09.457222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.457227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.457228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.457228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.457229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.457230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.457231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.457231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.457232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.457236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.457246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.457249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.457253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.457258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.457261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.457266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.457270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.457277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.457283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.457287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.457295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.457299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.457302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.457305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.457307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.457310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.457313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.457322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.457330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.457339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.457348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.457356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.457359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.457365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.457367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.457370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.457390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:09.458384Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4f6e249-d2ba268b-850fef8f-ce1c7a69_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:09.458461Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:09.458473Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1736
2025-10-09T21:50:09.458573Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1736 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:09.458595Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1736 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:09.458602Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1736 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:09.458604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:09.458913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1736 sessionId:  describe result for acl check
2025-10-09T21:50:09.458937Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:09.458942Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:09.458943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:09.458947Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342626340463216:6560] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:09.459379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342626340463216:6560] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:09.461669Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342626340463216:6560] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:09.463506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:09.464301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342626340463216:6560] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:09.464312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342626340463216:6560] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:09.464315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342626340463216:6560] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:09.464321Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1736 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:09.464527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:09.464544Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:09.464547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:09.464552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:09.464560Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1735 [2:7559342625978632552:3364]
2025-10-09T21:50:09.464571Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:09.464591Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:09.464598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:09.464601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:09.464603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:09.464621Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:09.464632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:09.464642Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:09.464696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.464702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.464704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.464707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:09.464764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:09.464771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.464783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.464803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.464821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:09.464843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463185:2432], now have 1 active actors on pipe
2025-10-09T21:50:09.464851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463251:6560], now have 1 active actors on pipe
2025-10-09T21:50:09.464858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342625978632547:3364]
2025-10-09T21:50:09.464868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342625978632547:3364]
2025-10-09T21:50:09.464870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342625978632547:3364]
2025-10-09T21:50:09.464881Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:09.464881Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1735
2025-10-09T21:50:09.464893Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:09.464894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:09.464895Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:09.464896Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:09.464904Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1735
2025-10-09T21:50:09.464908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:09.464912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:09.464933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342625978632547:3364]
2025-10-09T21:50:09.464944Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:09.464950Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:09.464962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.464966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.465009Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|16d00f4e-c086af1d-de0f589f-f442214a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:09.465021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:09.465024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:09.465026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:09.465036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:09.465044Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:09.465060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:09.465222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.465233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.465247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:09.465331Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1736 partition: 0 MaxSeqNo: 25 sessionId: src_id|16d00f4e-c086af1d-de0f589f-f442214a_0
2025-10-09T21:50:09.465631Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4f6e249-d2ba268b-850fef8f-ce1c7a69_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|16d00f4e-c086af1d-de0f589f-f442214a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:09.468051Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16d00f4e-c086af1d-de0f589f-f442214a_0] Write session: try to update token
2025-10-09T21:50:09.468066Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16d00f4e-c086af1d-de0f589f-f442214a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:09.479228Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:09.479249Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:09.479252Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:09.479256Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:09.479266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342625978632547:3364]
2025-10-09T21:50:09.479270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342625978632547:3364]
2025-10-09T21:50:09.479274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342625978632547:3364]
2025-10-09T21:50:09.479314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:09.479321Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:09.479325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.479326Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:09.479328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:09.479329Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:09.479474Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:09.480970Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1736 sessionId: src_id|16d00f4e-c086af1d-de0f589f-f442214a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:09.483388Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:09.548784Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342626340463272:6864]
2025-10-09T21:50:09.548975Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 1735, pipe: [1:7559342626340463272:6864]
2025-10-09T21:50:09.622976Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:09.623261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.623273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.623311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463272:6864], now have 1 active actors on pipe
2025-10-09T21:50:09.623540Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:09.623570Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:09.623573Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:09.623596Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342626340463272:6864]
2025-10-09T21:50:09.623631Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1736 reason: error at write: Disconnected sessionId: src_id|16d00f4e-c086af1d-de0f589f-f442214a_0
2025-10-09T21:50:09.623745Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:09.623803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1736 sessionId: src_id|16d00f4e-c086af1d-de0f589f-f442214a_0 is DEAD
2025-10-09T21:50:09.623966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16d00f4e-c086af1d-de0f589f-f442214a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:09.623976Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16d00f4e-c086af1d-de0f589f-f442214a_0] Write session will restart in 0.006060s
2025-10-09T21:50:09.623997Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16d00f4e-c086af1d-de0f589f-f442214a_0] Write session: Do CDS request
2025-10-09T21:50:09.624002Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16d00f4e-c086af1d-de0f589f-f442214a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:09.630588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:09.631457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16d00f4e-c086af1d-de0f589f-f442214a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:09.631533Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:09.631546Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1737
2025-10-09T21:50:09.631969Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1737 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:09.632002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1737 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:09.632013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1737 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:09.632017Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:09.632128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:09.632301Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:09.632312Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:09.632324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:09.632386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:09.632427Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342625978632656:3344]
2025-10-09T21:50:09.632471Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1737 sessionId:  describe result for acl check
2025-10-09T21:50:09.632506Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:09.632514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:09.632516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:09.632522Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342626340463311:6542] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:09.633013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:09.633034Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342626340463311:6542] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:09.633314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:09.633355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:09.633681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:09.633733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.633740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.633742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.633743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.633744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.633745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.633746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.633748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.633749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.633754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.633764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.633768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.633772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.633776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.633781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.633784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.633788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.633793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.633798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.633807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.633813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.633818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.633827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.633833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.633842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.633848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.633852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.633858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.633867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.633872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.633881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.633886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.633891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.633901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.633905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.633910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.633943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:09.635253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342626340463311:6542] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:09.639807Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342626340463311:6542] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:09.639822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342626340463311:6542] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:09.639826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342626340463311:6542] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:09.639835Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1737 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:09.640144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:09.641174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:09.641194Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:09.641196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:09.641201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:09.641208Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1736 [2:7559342625978632656:3344]
2025-10-09T21:50:09.641222Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:09.641239Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:09.641242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:09.641246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:09.641249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:09.641265Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:09.641277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:09.641283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:09.641337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.641345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.641348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.641351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:09.641415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:09.641422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.641435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.641453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.641471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:09.641496Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1736
2025-10-09T21:50:09.641496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463350:6542], now have 1 active actors on pipe
2025-10-09T21:50:09.641515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463282:2432], now have 1 active actors on pipe
2025-10-09T21:50:09.641522Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1736
2025-10-09T21:50:09.641522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342625978632650:3344]
2025-10-09T21:50:09.641524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342625978632650:3344]
2025-10-09T21:50:09.641526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342625978632650:3344]
2025-10-09T21:50:09.641534Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:09.641548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:09.641557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:09.641559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:09.641572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:09.641577Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:09.641579Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:09.641592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.641595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.641604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342625978632650:3344]
2025-10-09T21:50:09.641614Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:09.641633Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:09.641651Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cc7f5be3-f1ca2279-24baf7ce-d7260910_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:09.641660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:09.641669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:09.641671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:09.641679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:09.641691Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:09.641713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:09.641834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.641842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.641854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:09.641924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1737 partition: 0 MaxSeqNo: 25 sessionId: src_id|cc7f5be3-f1ca2279-24baf7ce-d7260910_0
2025-10-09T21:50:09.642239Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16d00f4e-c086af1d-de0f589f-f442214a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cc7f5be3-f1ca2279-24baf7ce-d7260910_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:09.644611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc7f5be3-f1ca2279-24baf7ce-d7260910_0] Write session: try to update token
2025-10-09T21:50:09.644628Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc7f5be3-f1ca2279-24baf7ce-d7260910_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:09.655672Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:09.655691Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:09.655693Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:09.655694Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:09.655703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342625978632650:3344]
2025-10-09T21:50:09.655706Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342625978632650:3344]
2025-10-09T21:50:09.655709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342625978632650:3344]
2025-10-09T21:50:09.655748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:09.655754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.655754Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:09.655758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:09.655761Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:09.655764Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:09.655964Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:09.657326Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1737 sessionId: src_id|cc7f5be3-f1ca2279-24baf7ce-d7260910_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:09.659667Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:09.723899Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342626340463358:6864]
2025-10-09T21:50:09.724159Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 1736, pipe: [1:7559342626340463358:6864]
2025-10-09T21:50:09.801273Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:09.801583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.801596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.801636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463358:6864], now have 1 active actors on pipe
2025-10-09T21:50:09.801806Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:09.801879Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:09.801893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:09.801900Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:09.801949Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342626340463358:6864]
2025-10-09T21:50:09.801955Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1737 reason: error at write: Disconnected sessionId: src_id|cc7f5be3-f1ca2279-24baf7ce-d7260910_0
2025-10-09T21:50:09.802148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1737 sessionId: src_id|cc7f5be3-f1ca2279-24baf7ce-d7260910_0 is DEAD
2025-10-09T21:50:09.802311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc7f5be3-f1ca2279-24baf7ce-d7260910_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:09.802322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc7f5be3-f1ca2279-24baf7ce-d7260910_0] Write session will restart in 0.009105s
2025-10-09T21:50:09.802350Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc7f5be3-f1ca2279-24baf7ce-d7260910_0] Write session: Do CDS request
2025-10-09T21:50:09.802357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc7f5be3-f1ca2279-24baf7ce-d7260910_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:09.808747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:09.810247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:09.810448Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:09.810459Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:09.810472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:09.810529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:09.810566Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342625978632725:3317]
2025-10-09T21:50:09.811165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:09.811434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:09.811474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:09.811785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:09.811831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.811837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.811838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.811840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.811841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.811842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.811843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.811844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.811845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.811850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.811861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.811864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.811869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.811873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.811877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.811881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.811884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.811888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.811895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.811903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.811910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.811915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.811920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.811925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.811929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.811934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.811938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.811943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.811949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.811954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.811959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.811965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.811969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.811973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.811977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.811982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.812011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:09.812486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cc7f5be3-f1ca2279-24baf7ce-d7260910_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:09.812589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:09.812606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1738
2025-10-09T21:50:09.812748Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1738 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:09.812774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1738 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:09.812782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1738 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:09.812785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:09.813118Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1738 sessionId:  describe result for acl check
2025-10-09T21:50:09.813152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:09.813157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:09.813159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:09.813163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342626340463393:6544] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:09.813602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342626340463393:6544] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:09.815215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342626340463393:6544] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:09.817950Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342626340463393:6544] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:09.817963Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342626340463393:6544] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:09.817967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342626340463393:6544] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:09.817975Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1738 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:09.818469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:09.819485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:09.819500Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:09.819503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:09.819507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:09.819513Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1737 [2:7559342625978632725:3317]
2025-10-09T21:50:09.819530Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:09.819552Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:09.819559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:09.819563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:09.819567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:09.819585Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:09.819595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:09.819606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:09.819668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.819674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.819676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.819679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:09.819745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:09.819753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.819766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.819784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.819804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:09.819829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463360:2432], now have 1 active actors on pipe
2025-10-09T21:50:09.819839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463428:6544], now have 1 active actors on pipe
2025-10-09T21:50:09.819846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342625978632720:3317]
2025-10-09T21:50:09.819848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342625978632720:3317]
2025-10-09T21:50:09.819850Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342625978632720:3317]
2025-10-09T21:50:09.819856Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1737
2025-10-09T21:50:09.819871Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1737
2025-10-09T21:50:09.819865Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:09.819880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:09.819894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:09.819895Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:09.819898Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:09.819899Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:09.819901Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:09.819928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342625978632720:3317]
2025-10-09T21:50:09.819932Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:09.819945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.819946Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:09.819949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.819987Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1b7ebf30-d5f019d5-11d8ebce-ca862dc0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:09.820000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:09.820005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:09.820007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:09.820017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:09.820026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:09.820046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:09.820167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.820174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.820186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:09.820250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1738 partition: 0 MaxSeqNo: 25 sessionId: src_id|1b7ebf30-d5f019d5-11d8ebce-ca862dc0_0
2025-10-09T21:50:09.820544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cc7f5be3-f1ca2279-24baf7ce-d7260910_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1b7ebf30-d5f019d5-11d8ebce-ca862dc0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:09.822880Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b7ebf30-d5f019d5-11d8ebce-ca862dc0_0] Write session: try to update token
2025-10-09T21:50:09.822896Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b7ebf30-d5f019d5-11d8ebce-ca862dc0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:09.834227Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:09.834254Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:09.834257Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:09.834262Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:09.834275Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342625978632720:3317]
2025-10-09T21:50:09.834292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342625978632720:3317]
2025-10-09T21:50:09.834296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342625978632720:3317]
2025-10-09T21:50:09.834347Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:09.834364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:09.834364Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:09.834368Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:09.834375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.834378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:09.834526Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:09.837150Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1738 sessionId: src_id|1b7ebf30-d5f019d5-11d8ebce-ca862dc0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:09.839563Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:09.902054Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342626340463438:6864]
2025-10-09T21:50:09.902297Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 1737, pipe: [1:7559342626340463438:6864]
2025-10-09T21:50:09.982785Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:09.983025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:09.983040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:09.983079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463438:6864], now have 1 active actors on pipe
2025-10-09T21:50:09.983279Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:09.983286Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:09.983320Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:09.983324Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:09.983335Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342626340463438:6864]
2025-10-09T21:50:09.983388Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1738 reason: error at write: Disconnected sessionId: src_id|1b7ebf30-d5f019d5-11d8ebce-ca862dc0_0
2025-10-09T21:50:09.983600Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1738 sessionId: src_id|1b7ebf30-d5f019d5-11d8ebce-ca862dc0_0 is DEAD
2025-10-09T21:50:09.983745Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b7ebf30-d5f019d5-11d8ebce-ca862dc0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:09.983757Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b7ebf30-d5f019d5-11d8ebce-ca862dc0_0] Write session will restart in 0.005635s
2025-10-09T21:50:09.983797Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b7ebf30-d5f019d5-11d8ebce-ca862dc0_0] Write session: Do CDS request
2025-10-09T21:50:09.983804Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b7ebf30-d5f019d5-11d8ebce-ca862dc0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:09.990531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b7ebf30-d5f019d5-11d8ebce-ca862dc0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:09.990596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:09.990611Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1739
2025-10-09T21:50:09.990702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:09.990753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1739 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:09.990787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1739 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:09.990798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1739 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:09.990801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:09.991268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1739 sessionId:  describe result for acl check
2025-10-09T21:50:09.991304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:09.991311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:09.991312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:09.991318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342626340463472:6536] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:09.991846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342626340463472:6536] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:09.992112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:09.992278Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:09.992288Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:09.992299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:09.992347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:09.992384Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342625978632793:3315]
2025-10-09T21:50:09.992913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:09.993144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:09.993185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:09.993485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:09.993550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.993557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.993558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.993559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.993561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.993562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.993563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.993564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.993565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.993570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.993581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.993585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.993590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.993594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.993598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.993602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.993606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.993610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.993616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.993624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:09.993630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.993635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:09.993639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.993644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:09.993648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.993653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:09.993659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.993664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:09.993668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.993673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:09.993677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.993682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:09.993685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.993689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:09.993693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.993703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:09.993731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:09.993969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342626340463472:6536] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:09.999201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342626340463472:6536] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:09.999217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342626340463472:6536] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:09.999220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342626340463472:6536] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:09.999229Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1739 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:09.999678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:10.000972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:10.000993Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:10.000996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:10.001000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:10.001008Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1738 [2:7559342625978632793:3315]
2025-10-09T21:50:10.001023Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:10.001044Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:10.001048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:10.001052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:10.001054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:10.001075Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:10.001094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:10.001106Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:10.001200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.001216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.001218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.001221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:10.001309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:10.001320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.001334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.001345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.001360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:10.001384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463442:2432], now have 1 active actors on pipe
2025-10-09T21:50:10.001393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342626340463508:6536], now have 1 active actors on pipe
2025-10-09T21:50:10.001401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342625978632788:3315]
2025-10-09T21:50:10.001404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342625978632788:3315]
2025-10-09T21:50:10.001406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342625978632788:3315]
2025-10-09T21:50:10.001420Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:10.001438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:10.001447Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:10.001450Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:10.001453Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:10.001454Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:10.001441Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1738
2025-10-09T21:50:10.001458Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:10.001447Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1738
2025-10-09T21:50:10.001492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342625978632788:3315]
2025-10-09T21:50:10.001513Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:10.001539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.001547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:10.001549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.001597Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a3fa55f1-e21f81e5-c856b24-a3e2cd2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:10.001613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:10.001617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:10.001619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:10.001630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:10.001640Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:10.001657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:10.001842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.001851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.001865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:10.001928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1739 partition: 0 MaxSeqNo: 25 sessionId: src_id|a3fa55f1-e21f81e5-c856b24-a3e2cd2_0
2025-10-09T21:50:10.002297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b7ebf30-d5f019d5-11d8ebce-ca862dc0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a3fa55f1-e21f81e5-c856b24-a3e2cd2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:10.004868Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a3fa55f1-e21f81e5-c856b24-a3e2cd2_0] Write session: try to update token
2025-10-09T21:50:10.004886Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a3fa55f1-e21f81e5-c856b24-a3e2cd2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:10.016509Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:10.016531Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:10.016534Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:10.016537Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:10.016546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342625978632788:3315]
2025-10-09T21:50:10.016549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342625978632788:3315]
2025-10-09T21:50:10.016552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342625978632788:3315]
2025-10-09T21:50:10.016590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:10.016598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.016600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:10.016595Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:10.016609Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:10.016613Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:10.016753Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:10.019461Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1739 sessionId: src_id|a3fa55f1-e21f81e5-c856b24-a3e2cd2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:10.021959Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:10.083282Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342630635430812:6864]
2025-10-09T21:50:10.083531Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 1738, pipe: [1:7559342630635430812:6864]
2025-10-09T21:50:10.177371Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:10.177796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.177814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.177861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342630635430812:6864], now have 1 active actors on pipe
2025-10-09T21:50:10.178112Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:10.178156Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:10.178179Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342630635430812:6864]
2025-10-09T21:50:10.178207Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:10.178298Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1739 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a3fa55f1-e21f81e5-c856b24-a3e2cd2_0
2025-10-09T21:50:10.178480Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1739 sessionId: src_id|a3fa55f1-e21f81e5-c856b24-a3e2cd2_0 is DEAD
2025-10-09T21:50:10.178648Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3fa55f1-e21f81e5-c856b24-a3e2cd2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:10.178660Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3fa55f1-e21f81e5-c856b24-a3e2cd2_0] Write session will restart in 0.006170s
2025-10-09T21:50:10.178705Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3fa55f1-e21f81e5-c856b24-a3e2cd2_0] Write session: Do CDS request
2025-10-09T21:50:10.178714Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3fa55f1-e21f81e5-c856b24-a3e2cd2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:10.186635Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a3fa55f1-e21f81e5-c856b24-a3e2cd2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:10.186747Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:10.186765Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1740
2025-10-09T21:50:10.186915Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1740 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:10.186961Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1740 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:10.186973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1740 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:10.186977Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:10.187295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:10.187411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1740 sessionId:  describe result for acl check
2025-10-09T21:50:10.187445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:10.187448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:10.187449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:10.187456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342630635430848:6533] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:10.188024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342630635430848:6533] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:10.188895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:10.189093Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:10.189103Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:10.189114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:10.189167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:10.189210Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342630273600157:3306]
2025-10-09T21:50:10.189844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:10.190145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:10.190193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:10.190205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342630635430848:6533] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:10.190512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:10.190563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.190568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.190570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.190571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.190572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.190573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.190575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.190576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.190577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.190582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.190593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.190598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.190603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.190620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.190629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.190633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.190637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.190641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.190649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.190658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.190669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.190676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.190685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.190691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.190696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.190710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.190715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.190720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.190724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.190731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.190736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.190742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.190746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.190752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.190757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.190762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.190792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:10.196055Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342630635430848:6533] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:10.196072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342630635430848:6533] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:10.196077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342630635430848:6533] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:10.196087Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1740 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:10.197545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:10.198662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:10.198684Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:10.198687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:10.198691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:10.198699Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1739 [2:7559342630273600157:3306]
2025-10-09T21:50:10.198722Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:10.198743Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:10.198752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:10.198756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:10.198759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:10.198778Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:10.198793Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:10.198801Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:10.198865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.198875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.198877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.198880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:10.198953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:10.198964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.198978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.198990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.199009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:10.199039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342630635430888:6533], now have 1 active actors on pipe
2025-10-09T21:50:10.199043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342630635430815:2432], now have 1 active actors on pipe
2025-10-09T21:50:10.199052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342630273600152:3306]
2025-10-09T21:50:10.199060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342630273600152:3306]
2025-10-09T21:50:10.199062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342630273600152:3306]
2025-10-09T21:50:10.199078Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:10.199093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:10.199107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:10.199110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:10.199110Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:10.199113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:10.199113Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:10.199120Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1739
2025-10-09T21:50:10.199152Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342630273600152:3306]
2025-10-09T21:50:10.199127Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1739
2025-10-09T21:50:10.199160Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:10.199162Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:10.199195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.199202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.199237Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4b5012e5-14e95e90-66fb79f6-34c14807_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:10.199249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:10.199253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:10.199260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:10.199273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:10.199288Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:10.199313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:10.199509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.199524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.199556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:10.199684Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1740 partition: 0 MaxSeqNo: 25 sessionId: src_id|4b5012e5-14e95e90-66fb79f6-34c14807_0
2025-10-09T21:50:10.200049Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3fa55f1-e21f81e5-c856b24-a3e2cd2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4b5012e5-14e95e90-66fb79f6-34c14807_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:10.202553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b5012e5-14e95e90-66fb79f6-34c14807_0] Write session: try to update token
2025-10-09T21:50:10.202572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b5012e5-14e95e90-66fb79f6-34c14807_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:10.214430Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:10.214453Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:10.214460Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:10.214570Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:10.214584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342630273600152:3306]
2025-10-09T21:50:10.214594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342630273600152:3306]
2025-10-09T21:50:10.214598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342630273600152:3306]
2025-10-09T21:50:10.214647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:10.214654Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:10.214660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.214661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:10.214664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:10.214665Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:10.214816Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:10.216407Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1740 sessionId: src_id|4b5012e5-14e95e90-66fb79f6-34c14807_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:10.219089Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:10.278399Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342630635430900:6864]
2025-10-09T21:50:10.278646Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 1739, pipe: [1:7559342630635430900:6864]
2025-10-09T21:50:10.368564Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:10.368809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.368826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.368869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342630635430900:6864], now have 1 active actors on pipe
2025-10-09T21:50:10.369161Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:10.369118Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:10.369175Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:10.369200Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:10.369247Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342630635430900:6864]
2025-10-09T21:50:10.369264Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1740 reason: error at write: Disconnected sessionId: src_id|4b5012e5-14e95e90-66fb79f6-34c14807_0
2025-10-09T21:50:10.369525Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1740 sessionId: src_id|4b5012e5-14e95e90-66fb79f6-34c14807_0 is DEAD
2025-10-09T21:50:10.369640Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b5012e5-14e95e90-66fb79f6-34c14807_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:10.369652Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b5012e5-14e95e90-66fb79f6-34c14807_0] Write session will restart in 0.008722s
2025-10-09T21:50:10.369678Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b5012e5-14e95e90-66fb79f6-34c14807_0] Write session: Do CDS request
2025-10-09T21:50:10.369685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b5012e5-14e95e90-66fb79f6-34c14807_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:10.378055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:10.379592Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b5012e5-14e95e90-66fb79f6-34c14807_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:10.379592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:10.379604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1741
2025-10-09T21:50:10.379799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1741 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:10.379835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1741 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:10.379846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1741 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:10.379850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:10.379940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:10.380121Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:10.380129Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:10.380138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:10.380184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:10.380229Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342630273600226:3310]
2025-10-09T21:50:10.380378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1741 sessionId:  describe result for acl check
2025-10-09T21:50:10.380411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:10.380425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:10.380435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:10.380441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342630635430935:6517] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:10.380687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:10.380972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:10.381017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342630635430935:6517] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:10.381042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:10.381363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:10.381416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.381422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.381423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.381424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.381425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.381425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.381426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.381426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.381427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.381431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.381436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.381439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.381441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.381443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.381445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.381447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.381449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.381451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.381457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.381463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.381467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.381470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.381474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.381480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.381482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.381485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.381487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.381490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.381493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.381496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.381510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.381513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.381517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.381522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.381526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.381531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.381557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:10.383292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342630635430935:6517] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:10.387404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342630635430935:6517] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:10.387425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342630635430935:6517] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:10.387430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342630635430935:6517] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:10.387440Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1741 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:10.387853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:10.388815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:10.388840Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:10.388842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:10.388846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:10.388853Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1740 [2:7559342630273600226:3310]
2025-10-09T21:50:10.388863Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:10.388878Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:10.388885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:10.388887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:10.388890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:10.388903Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:10.388908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:10.388914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:10.388957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.388963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.388965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.388967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:10.389015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:10.389021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.389028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.389034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.389043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:10.389064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342630273600221:3310]
2025-10-09T21:50:10.389065Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342630273600221:3310]
2025-10-09T21:50:10.389066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342630273600221:3310]
2025-10-09T21:50:10.389072Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:10.389085Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:10.389086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342630635430974:6517], now have 1 active actors on pipe
2025-10-09T21:50:10.389088Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:10.389090Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:10.389091Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:10.389099Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:10.389101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:10.389104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342630635430903:2432], now have 1 active actors on pipe
2025-10-09T21:50:10.389131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342630273600221:3310]
2025-10-09T21:50:10.389144Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:10.389166Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:10.389164Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1740
2025-10-09T21:50:10.389172Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1740
2025-10-09T21:50:10.389263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.389280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.389315Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c1a3f26-caf468-551dd30-4924a529_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:10.389329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:10.389339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:10.389342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:10.389362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:10.389378Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:10.389399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:10.389574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.389581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.389594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:10.389683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1741 partition: 0 MaxSeqNo: 25 sessionId: src_id|c1a3f26-caf468-551dd30-4924a529_0
2025-10-09T21:50:10.390023Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b5012e5-14e95e90-66fb79f6-34c14807_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c1a3f26-caf468-551dd30-4924a529_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:10.392416Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1a3f26-caf468-551dd30-4924a529_0] Write session: try to update token
2025-10-09T21:50:10.392431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1a3f26-caf468-551dd30-4924a529_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:10.403424Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:10.403444Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:10.403447Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:10.403449Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:10.403458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342630273600221:3310]
2025-10-09T21:50:10.403464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342630273600221:3310]
2025-10-09T21:50:10.403468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342630273600221:3310]
2025-10-09T21:50:10.403517Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:10.403533Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:10.403534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:10.403541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.403544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:10.403546Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:10.403695Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:10.405521Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1741 sessionId: src_id|c1a3f26-caf468-551dd30-4924a529_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:10.408449Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:10.469371Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342630635430981:6864]
2025-10-09T21:50:10.469681Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 1740, pipe: [1:7559342630635430981:6864]
2025-10-09T21:50:10.556889Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:10.557153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.557165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.557204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342630635430981:6864], now have 1 active actors on pipe
2025-10-09T21:50:10.557479Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342630635430981:6864]
2025-10-09T21:50:10.557495Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:10.557525Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:10.557528Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:10.557589Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1741 reason: error at write: Disconnected sessionId: src_id|c1a3f26-caf468-551dd30-4924a529_0
2025-10-09T21:50:10.557756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1741 sessionId: src_id|c1a3f26-caf468-551dd30-4924a529_0 is DEAD
2025-10-09T21:50:10.557871Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:10.557913Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1a3f26-caf468-551dd30-4924a529_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:10.557924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1a3f26-caf468-551dd30-4924a529_0] Write session will restart in 0.009839s
2025-10-09T21:50:10.557954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1a3f26-caf468-551dd30-4924a529_0] Write session: Do CDS request
2025-10-09T21:50:10.557960Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1a3f26-caf468-551dd30-4924a529_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:10.565793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:10.567239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:10.567437Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:10.567447Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:10.567460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:10.567518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:10.567567Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342630273600296:3307]
2025-10-09T21:50:10.568146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:10.568494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:10.568561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1a3f26-caf468-551dd30-4924a529_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:10.568556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:10.568662Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:10.568679Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1742
2025-10-09T21:50:10.568803Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1742 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:10.568842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1742 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:10.568864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1742 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:10.568868Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:10.568909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:10.568992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.569001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.569002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.569003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.569004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.569005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.569007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.569008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.569009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.569016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.569025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.569028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.569032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.569036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.569040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.569044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.569048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.569054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.569061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.569083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.569090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.569102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.569108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.569116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.569120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.569126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.569130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.569140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.569144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.569150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.569154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.569160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.569164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.569169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.569173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.569179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.569208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:10.569317Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1742 sessionId:  describe result for acl check
2025-10-09T21:50:10.569353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:10.569355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:10.569357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:10.569362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342630635431020:6516] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:10.569860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342630635431020:6516] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:10.571745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342630635431020:6516] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:10.575907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342630635431020:6516] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:10.575923Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342630635431020:6516] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:10.575927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342630635431020:6516] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:10.575936Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1742 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:10.576379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:10.577565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:10.577582Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:10.577586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:10.577591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:10.577598Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1741 [2:7559342630273600296:3307]
2025-10-09T21:50:10.577612Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:10.577630Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:10.577634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:10.577637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:10.577639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:10.577656Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:10.577667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:10.577674Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:10.577727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.577737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.577739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.577742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:10.577805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:10.577813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.577827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.577839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.577862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:10.577891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342630635431061:6516], now have 1 active actors on pipe
2025-10-09T21:50:10.577899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342630635430984:2432], now have 1 active actors on pipe
2025-10-09T21:50:10.577907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342630273600291:3307]
2025-10-09T21:50:10.577909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342630273600291:3307]
2025-10-09T21:50:10.577912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342630273600291:3307]
2025-10-09T21:50:10.577928Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:10.577925Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1741
2025-10-09T21:50:10.577941Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:10.577952Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:10.577959Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1741
2025-10-09T21:50:10.577956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:10.577958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:10.577960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:10.577963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:10.577995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342630273600291:3307]
2025-10-09T21:50:10.578006Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:10.578016Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:10.578022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.578026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.578096Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b0d54e53-7f61cd4f-90d2225e-f331bad5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:10.578108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:10.578112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:10.578114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:10.578124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:10.578134Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:10.578155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:10.578337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.578349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.578365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:10.578481Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1742 partition: 0 MaxSeqNo: 25 sessionId: src_id|b0d54e53-7f61cd4f-90d2225e-f331bad5_0
2025-10-09T21:50:10.578833Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1a3f26-caf468-551dd30-4924a529_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b0d54e53-7f61cd4f-90d2225e-f331bad5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:10.581227Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0d54e53-7f61cd4f-90d2225e-f331bad5_0] Write session: try to update token
2025-10-09T21:50:10.581244Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0d54e53-7f61cd4f-90d2225e-f331bad5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:10.592877Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:10.592902Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:10.592904Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:10.592906Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:10.592915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342630273600291:3307]
2025-10-09T21:50:10.592919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342630273600291:3307]
2025-10-09T21:50:10.592923Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342630273600291:3307]
2025-10-09T21:50:10.592969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:10.592968Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:10.592974Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:10.592977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.592978Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:10.592980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:10.593136Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:10.594474Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1742 sessionId: src_id|b0d54e53-7f61cd4f-90d2225e-f331bad5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:10.596893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:10.657800Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342630635431085:6864]
2025-10-09T21:50:10.658121Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 1741, pipe: [1:7559342630635431085:6864]
2025-10-09T21:50:10.749879Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:10.750188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.750202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.750244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342630635431085:6864], now have 1 active actors on pipe
2025-10-09T21:50:10.750453Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342630635431085:6864]
2025-10-09T21:50:10.750484Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:10.750492Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:10.750495Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:10.750653Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1742 reason: error at write: Disconnected sessionId: src_id|b0d54e53-7f61cd4f-90d2225e-f331bad5_0
2025-10-09T21:50:10.750753Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:10.750832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1742 sessionId: src_id|b0d54e53-7f61cd4f-90d2225e-f331bad5_0 is DEAD
2025-10-09T21:50:10.750998Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0d54e53-7f61cd4f-90d2225e-f331bad5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:10.751007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0d54e53-7f61cd4f-90d2225e-f331bad5_0] Write session will restart in 0.009593s
2025-10-09T21:50:10.751034Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0d54e53-7f61cd4f-90d2225e-f331bad5_0] Write session: Do CDS request
2025-10-09T21:50:10.751041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0d54e53-7f61cd4f-90d2225e-f331bad5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:10.758749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:10.760255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:10.760423Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:10.760431Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:10.760440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:10.760478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:10.760510Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342630273600409:3265]
2025-10-09T21:50:10.760936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:10.761228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:10.761270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:10.761531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:10.761563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b0d54e53-7f61cd4f-90d2225e-f331bad5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:10.761580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.761586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.761587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.761589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.761590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.761591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.761592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.761593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.761595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.761600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.761611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.761615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.761619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.761622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.761626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.761632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.761636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.761639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.761647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.761655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.761661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.761666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.761676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.761680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.761647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:10.761684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.761663Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1743
2025-10-09T21:50:10.761689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.761693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.761698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.761702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.761706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.761713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.761718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.761722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.761732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.761737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.761742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.761771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:10.761808Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1743 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:10.761838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1743 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:10.761848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1743 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:10.761852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:10.762211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1743 sessionId:  describe result for acl check
2025-10-09T21:50:10.762244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:10.762251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:10.762252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:10.762258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342630635431126:6504] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:10.762742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342630635431126:6504] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:10.764752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342630635431126:6504] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:10.767761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342630635431126:6504] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:10.767775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342630635431126:6504] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:10.767779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342630635431126:6504] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:10.767787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1743 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:10.768157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:10.769371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:10.769386Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:10.769389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:10.769393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:10.769399Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1742 [2:7559342630273600409:3265]
2025-10-09T21:50:10.769411Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:10.769428Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:10.769436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:10.769439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:10.769442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:10.769459Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:10.769471Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:10.769478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:10.769541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.769548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.769550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.769553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:10.769618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:10.769626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.769638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.769655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.769678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:10.769702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342630635431088:2432], now have 1 active actors on pipe
2025-10-09T21:50:10.769709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342630635431165:6504], now have 1 active actors on pipe
2025-10-09T21:50:10.769717Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342630273600404:3265]
2025-10-09T21:50:10.769719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342630273600404:3265]
2025-10-09T21:50:10.769721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342630273600404:3265]
2025-10-09T21:50:10.769732Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:10.769747Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:10.769751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:10.769758Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:10.769760Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:10.769764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:10.769766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:10.769759Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1742
2025-10-09T21:50:10.769790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342630273600404:3265]
2025-10-09T21:50:10.769797Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:10.769799Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:10.769869Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1742
2025-10-09T21:50:10.769917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.769926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.769946Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|16dfa2c7-a888fac4-7edc80fb-d4d61a96_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:10.769952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:10.769955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:10.769958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:10.769967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:10.769976Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:10.769990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:10.770236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.770265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.770285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:10.770416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1743 partition: 0 MaxSeqNo: 25 sessionId: src_id|16dfa2c7-a888fac4-7edc80fb-d4d61a96_0
2025-10-09T21:50:10.770748Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b0d54e53-7f61cd4f-90d2225e-f331bad5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|16dfa2c7-a888fac4-7edc80fb-d4d61a96_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:10.773302Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16dfa2c7-a888fac4-7edc80fb-d4d61a96_0] Write session: try to update token
2025-10-09T21:50:10.773319Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16dfa2c7-a888fac4-7edc80fb-d4d61a96_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:10.781081Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:10.781097Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:10.781107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342630273600404:3265]
2025-10-09T21:50:10.781150Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:10.781151Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:10.781318Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:10.781449Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:10.781594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:10.781738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:10.782012Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:10.782092Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:10.786059Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1743 sessionId: src_id|16dfa2c7-a888fac4-7edc80fb-d4d61a96_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:10.788436Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:10.788587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.788609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.788663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:10.788774Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:10.793916Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:10.793935Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:10.793941Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:10.793943Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:10.793954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342630273600404:3265]
2025-10-09T21:50:10.793958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342630273600404:3265]
2025-10-09T21:50:10.793961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342630273600404:3265]
2025-10-09T21:50:10.794001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:10.794014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.794014Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:10.794017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:10.794019Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:10.794022Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:10.794154Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:10.850821Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342630635431172:6864]
2025-10-09T21:50:10.851111Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 1742, pipe: [1:7559342630635431172:6864]
2025-10-09T21:50:10.936862Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:10.943527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.943552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.943687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:10.943740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:10.943797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:10.943866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:10.943915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:10.943924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:10.943984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:10.944044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:10.944110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:10.944168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:10.944207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:10.944212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:10.944268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:10.944332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:10.944387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:10.944437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:10.944474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:10.944479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:10.944534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:10.944586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:10.944638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:10.944690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:10.944733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:10.944738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:10.944794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:10.944844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:10.944931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:10.944982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:10.945024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:10.945032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:10.945094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:10.945174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:10.945229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:10.945283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:10.945322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:10.945330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:10.945390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:10.945439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:10.945492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:10.945557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:10.945610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:10.945616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:10.945668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:10.945716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:10.945769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:10.945822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:10.945935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:10.945940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:10.945991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:10.946056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:10.946108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:10.946160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:10.946197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:10.946202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:10.946253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:10.946303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:10.946355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:10.946409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:10.946522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:10.946528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:10.946574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342630635431172:6864], now have 1 active actors on pipe
2025-10-09T21:50:10.946846Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:10.946849Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:10.946862Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:10.946894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1743 reason: error at write: Disconnected sessionId: src_id|16dfa2c7-a888fac4-7edc80fb-d4d61a96_0
2025-10-09T21:50:10.946903Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342630635431172:6864]
2025-10-09T21:50:10.947064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1743 sessionId: src_id|16dfa2c7-a888fac4-7edc80fb-d4d61a96_0 is DEAD
2025-10-09T21:50:10.947208Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16dfa2c7-a888fac4-7edc80fb-d4d61a96_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:10.947214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16dfa2c7-a888fac4-7edc80fb-d4d61a96_0] Write session will restart in 0.006952s
2025-10-09T21:50:10.947231Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16dfa2c7-a888fac4-7edc80fb-d4d61a96_0] Write session: Do CDS request
2025-10-09T21:50:10.947235Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16dfa2c7-a888fac4-7edc80fb-d4d61a96_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:10.947472Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:10.955520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16dfa2c7-a888fac4-7edc80fb-d4d61a96_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:10.955607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:10.955623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1744
2025-10-09T21:50:10.955756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1744 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:10.955796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1744 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:10.955807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1744 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:10.955811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:10.955849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:10.956248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1744 sessionId:  describe result for acl check
2025-10-09T21:50:10.956280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:10.956286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:10.956289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:10.956296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342630635431214:6495] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:10.956813Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342630635431214:6495] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:10.957161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:10.957330Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:10.957356Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:10.957370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:10.957436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:10.957481Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342630273600481:3273]
2025-10-09T21:50:10.957998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:10.958272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:10.958323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:10.958628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:10.958696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.958703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.958704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.958705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.958706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.958707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.958708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.958709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.958710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.958716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.958725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.958729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.958733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.958737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.958740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.958744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.958748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.958751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.958758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.958765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:10.958771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.958776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:10.958781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.958788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:10.958791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.958796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:10.958801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.958805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:10.958811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.958816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:10.958821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.958826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.958830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.958841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.958846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.958850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.958881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:10.959053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342630635431214:6495] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:10.964305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342630635431214:6495] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:10.964317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342630635431214:6495] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:10.964320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342630635431214:6495] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:10.964327Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1744 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:10.964751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:10.965869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:10.965887Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:10.965890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:10.965895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:10.965902Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1743 [2:7559342630273600481:3273]
2025-10-09T21:50:10.965916Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:10.965935Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:10.965943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:10.965946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:10.965948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:10.965965Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:10.965976Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:10.965983Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:10.966059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.966067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.966069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.966072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:10.966136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:10.966144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.966155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:10.966173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:10.966195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:10.966217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342630635431252:6495], now have 1 active actors on pipe
2025-10-09T21:50:10.966225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342630635431177:2432], now have 1 active actors on pipe
2025-10-09T21:50:10.966233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342630273600476:3273]
2025-10-09T21:50:10.966235Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342630273600476:3273]
2025-10-09T21:50:10.966237Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342630273600476:3273]
2025-10-09T21:50:10.966250Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:10.966236Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1743
2025-10-09T21:50:10.966251Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1743
2025-10-09T21:50:10.966266Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:10.966269Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:10.966277Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:10.966279Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:10.966284Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:10.966287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:10.966310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342630273600476:3273]
2025-10-09T21:50:10.966314Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:10.966323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:10.966333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.966349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.966396Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|90509676-1d41eda9-f5312993-a9af23f5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:10.966409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:10.966418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:10.966420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:10.966430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:10.966444Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:10.966472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:10.966694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:10.966706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:10.966725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:10.966838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1744 partition: 0 MaxSeqNo: 25 sessionId: src_id|90509676-1d41eda9-f5312993-a9af23f5_0
2025-10-09T21:50:10.967142Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16dfa2c7-a888fac4-7edc80fb-d4d61a96_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|90509676-1d41eda9-f5312993-a9af23f5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:10.969583Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90509676-1d41eda9-f5312993-a9af23f5_0] Write session: try to update token
2025-10-09T21:50:10.969599Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90509676-1d41eda9-f5312993-a9af23f5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:10.980638Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:10.980658Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:10.980660Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:10.980661Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:10.980669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342630273600476:3273]
2025-10-09T21:50:10.980672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342630273600476:3273]
2025-10-09T21:50:10.980675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342630273600476:3273]
2025-10-09T21:50:10.980713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:10.980716Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:10.980720Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:10.980720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:10.980723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:10.980723Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:10.980860Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:10.982398Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1744 sessionId: src_id|90509676-1d41eda9-f5312993-a9af23f5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:10.984576Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:11.046859Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342634930398555:6864]
2025-10-09T21:50:11.047138Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 1743, pipe: [1:7559342634930398555:6864]
2025-10-09T21:50:11.132702Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:11.133055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:11.133087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:11.133145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342634930398555:6864], now have 1 active actors on pipe
2025-10-09T21:50:11.133380Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:11.133413Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342634930398555:6864]
2025-10-09T21:50:11.133440Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:11.133446Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:11.133514Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1744 reason: error at write: Disconnected sessionId: src_id|90509676-1d41eda9-f5312993-a9af23f5_0
2025-10-09T21:50:11.133645Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:11.133690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1744 sessionId: src_id|90509676-1d41eda9-f5312993-a9af23f5_0 is DEAD
2025-10-09T21:50:11.133830Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90509676-1d41eda9-f5312993-a9af23f5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:11.133839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90509676-1d41eda9-f5312993-a9af23f5_0] Write session will restart in 0.007691s
2025-10-09T21:50:11.133880Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90509676-1d41eda9-f5312993-a9af23f5_0] Write session: Do CDS request
2025-10-09T21:50:11.133887Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90509676-1d41eda9-f5312993-a9af23f5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:11.140921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:11.142532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90509676-1d41eda9-f5312993-a9af23f5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:11.142624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:11.142643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1745
2025-10-09T21:50:11.142745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:11.142832Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1745 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:11.142867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1745 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:11.142872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1745 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:11.142876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:11.142946Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:11.142959Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:11.142970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:11.143023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:11.143066Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342634568567850:3271]
2025-10-09T21:50:11.143317Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1745 sessionId:  describe result for acl check
2025-10-09T21:50:11.143356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:11.143362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:11.143364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:11.143370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342634930398594:6487] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:11.143667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:11.143910Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342634930398594:6487] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:11.144044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:11.144091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:11.144411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:11.144467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.144474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.144475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.144476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.144477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.144479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.144480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.144483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.144484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.144490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.144500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.144504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.144509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.144513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.144517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.144521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.144525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.144533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.144540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.144552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.144561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.144571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.144580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.144586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.144595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.144600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.144605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.144614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.144623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.144629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.144634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.144641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.144650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.144655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.144665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.144674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.144708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:11.146180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342634930398594:6487] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:11.150739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342634930398594:6487] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:11.150758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342634930398594:6487] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:11.150763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342634930398594:6487] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:11.150773Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1745 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:11.151225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:11.152391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:11.152407Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:11.152410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:11.152416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:11.152424Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1744 [2:7559342634568567850:3271]
2025-10-09T21:50:11.152440Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:11.152463Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:11.152470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:11.152474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:11.152476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:11.152494Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:11.152504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:11.152512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:11.152572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.152578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.152579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.152581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:11.152653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:11.152660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.152672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.152682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.152696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:11.152717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342634930398633:6487], now have 1 active actors on pipe
2025-10-09T21:50:11.152724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342634930398558:2432], now have 1 active actors on pipe
2025-10-09T21:50:11.152730Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342634568567845:3271]
2025-10-09T21:50:11.152732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342634568567845:3271]
2025-10-09T21:50:11.152733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342634568567845:3271]
2025-10-09T21:50:11.152744Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:11.152755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:11.152767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:11.152773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:11.152762Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1744
2025-10-09T21:50:11.152763Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1744
2025-10-09T21:50:11.152774Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:11.152798Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:11.152801Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:11.152806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342634568567845:3271]
2025-10-09T21:50:11.152809Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:11.152812Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:11.152827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:11.152836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:11.152872Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ed612473-fa0dc84a-20061f07-5d30506b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:11.152884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:11.152891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:11.152894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:11.152904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:11.152917Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:11.152938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:11.153073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:11.153080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:11.153093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:11.153169Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1745 partition: 0 MaxSeqNo: 25 sessionId: src_id|ed612473-fa0dc84a-20061f07-5d30506b_0
2025-10-09T21:50:11.153538Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90509676-1d41eda9-f5312993-a9af23f5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ed612473-fa0dc84a-20061f07-5d30506b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:11.155952Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ed612473-fa0dc84a-20061f07-5d30506b_0] Write session: try to update token
2025-10-09T21:50:11.155972Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ed612473-fa0dc84a-20061f07-5d30506b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:11.168300Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:11.168333Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:11.168337Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:11.168339Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:11.168371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342634568567845:3271]
2025-10-09T21:50:11.168377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342634568567845:3271]
2025-10-09T21:50:11.168384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342634568567845:3271]
2025-10-09T21:50:11.168435Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:11.168448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:11.168450Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:11.168456Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:11.168459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.168462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:11.168738Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:11.169617Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1745 sessionId: src_id|ed612473-fa0dc84a-20061f07-5d30506b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:11.172064Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:11.233752Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342634930398643:6864]
2025-10-09T21:50:11.233986Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 1744, pipe: [1:7559342634930398643:6864]
2025-10-09T21:50:11.314228Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:11.314461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:11.314473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:11.314512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342634930398643:6864], now have 1 active actors on pipe
2025-10-09T21:50:11.314771Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:11.314776Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342634930398643:6864]
2025-10-09T21:50:11.314803Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:11.314805Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:11.314857Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:11.314869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1745 reason: error at write: Disconnected sessionId: src_id|ed612473-fa0dc84a-20061f07-5d30506b_0
2025-10-09T21:50:11.315020Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1745 sessionId: src_id|ed612473-fa0dc84a-20061f07-5d30506b_0 is DEAD
2025-10-09T21:50:11.315167Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed612473-fa0dc84a-20061f07-5d30506b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:11.315174Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed612473-fa0dc84a-20061f07-5d30506b_0] Write session will restart in 0.008370s
2025-10-09T21:50:11.315196Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed612473-fa0dc84a-20061f07-5d30506b_0] Write session: Do CDS request
2025-10-09T21:50:11.315201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed612473-fa0dc84a-20061f07-5d30506b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:11.322096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:11.323678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:11.323867Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:11.323879Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:11.323892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:11.323954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:11.323993Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342634568567925:3269]
2025-10-09T21:50:11.324488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ed612473-fa0dc84a-20061f07-5d30506b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:11.324565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:11.324554Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:11.324569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1746
2025-10-09T21:50:11.324709Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1746 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:11.324738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1746 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:11.324748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1746 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:11.324761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:11.324873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:11.324919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:11.325154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1746 sessionId:  describe result for acl check
2025-10-09T21:50:11.325187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:11.325188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:11.325190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:11.325194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342634930398691:6486] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:11.325222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:11.325272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.325278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.325280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.325281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.325282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.325283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.325285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.325286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.325287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.325292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.325301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.325305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.325311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.325314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.325318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.325321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.325324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.325327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.325334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.325342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.325350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.325360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.325365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.325369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.325378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.325382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.325387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.325392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.325400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.325406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.325412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.325419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.325424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.325430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.325435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.325440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.325470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:11.325697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342634930398691:6486] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:11.327820Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342634930398691:6486] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:11.331388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342634930398691:6486] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:11.331405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342634930398691:6486] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:11.331408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342634930398691:6486] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:11.331417Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1746 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:11.331957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:11.333224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:11.333243Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:11.333247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:11.333253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:11.333260Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1745 [2:7559342634568567925:3269]
2025-10-09T21:50:11.333275Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:11.333293Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:11.333301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:11.333304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:11.333306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:11.333324Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:11.333336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:11.333344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:11.333400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.333407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.333408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.333411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:11.333483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:11.333492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.333518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.333535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.333557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:11.333583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342634930398652:2432], now have 1 active actors on pipe
2025-10-09T21:50:11.333590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342634930398728:6486], now have 1 active actors on pipe
2025-10-09T21:50:11.333598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342634568567920:3269]
2025-10-09T21:50:11.333604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342634568567920:3269]
2025-10-09T21:50:11.333606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342634568567920:3269]
2025-10-09T21:50:11.333592Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1745
2025-10-09T21:50:11.333617Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:11.333622Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:11.333624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:11.333622Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1745
2025-10-09T21:50:11.333625Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:11.333635Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:11.333650Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:11.333657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:11.333702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:11.333710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:11.333722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342634568567920:3269]
2025-10-09T21:50:11.333733Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:11.333741Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:11.333765Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f7e5c272-861dd132-4cea36ff-af36c933_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:11.333777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:11.333781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:11.333783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:11.333793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:11.333803Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:11.333829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:11.334003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:11.334019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:11.334056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:11.334175Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1746 partition: 0 MaxSeqNo: 25 sessionId: src_id|f7e5c272-861dd132-4cea36ff-af36c933_0
2025-10-09T21:50:11.334507Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed612473-fa0dc84a-20061f07-5d30506b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f7e5c272-861dd132-4cea36ff-af36c933_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:11.336868Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7e5c272-861dd132-4cea36ff-af36c933_0] Write session: try to update token
2025-10-09T21:50:11.336885Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7e5c272-861dd132-4cea36ff-af36c933_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:11.347731Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:11.347759Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:11.347762Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:11.347764Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:11.347773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342634568567920:3269]
2025-10-09T21:50:11.347777Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342634568567920:3269]
2025-10-09T21:50:11.347782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342634568567920:3269]
2025-10-09T21:50:11.347822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:11.347823Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:11.347828Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:11.347831Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:11.347835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.347838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:11.347967Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:11.349646Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1746 sessionId: src_id|f7e5c272-861dd132-4cea36ff-af36c933_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:11.352149Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:11.415040Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342634930398735:6864]
2025-10-09T21:50:11.415299Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 1745, pipe: [1:7559342634930398735:6864]
2025-10-09T21:50:11.492312Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:11.492559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:11.492572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:11.492609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342634930398735:6864], now have 1 active actors on pipe
2025-10-09T21:50:11.492773Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:11.492827Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:11.492850Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342634930398735:6864]
2025-10-09T21:50:11.492876Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:11.493037Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1746 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f7e5c272-861dd132-4cea36ff-af36c933_0
2025-10-09T21:50:11.493221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1746 sessionId: src_id|f7e5c272-861dd132-4cea36ff-af36c933_0 is DEAD
2025-10-09T21:50:11.493349Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7e5c272-861dd132-4cea36ff-af36c933_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:11.493363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7e5c272-861dd132-4cea36ff-af36c933_0] Write session will restart in 0.008256s
2025-10-09T21:50:11.493389Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7e5c272-861dd132-4cea36ff-af36c933_0] Write session: Do CDS request
2025-10-09T21:50:11.493396Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7e5c272-861dd132-4cea36ff-af36c933_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:11.499933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:11.501366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:11.501560Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:11.501571Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:11.501582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:11.501636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:11.501674Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342634568568001:3268]
2025-10-09T21:50:11.502262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:11.502447Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7e5c272-861dd132-4cea36ff-af36c933_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:11.502538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:11.502537Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:11.502553Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1747
2025-10-09T21:50:11.502580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:11.502655Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1747 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:11.502683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1747 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:11.502693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1747 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:11.502697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:11.502885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:11.502931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.502938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.502939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.502940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.502941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.502942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.502943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.502944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.502945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.502950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.502960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.502964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.502968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.502972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.502976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.502980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.502983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.502987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.502994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.503002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.503008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.503013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.503018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.503023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.503027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.503032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.503037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.503041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.503046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.503042Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1747 sessionId:  describe result for acl check
2025-10-09T21:50:11.503051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.503058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.503064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.503067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.503072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.503070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:11.503076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.503076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:11.503081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.503078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:11.503084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342634930398778:6476] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:11.503109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:11.503578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342634930398778:6476] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:11.505304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342634930398778:6476] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:11.508749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342634930398778:6476] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:11.508761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342634930398778:6476] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:11.508765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342634930398778:6476] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:11.508772Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1747 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:11.509241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:11.510354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:11.510365Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:11.510367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:11.510371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:11.510377Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1746 [2:7559342634568568001:3268]
2025-10-09T21:50:11.510389Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:11.510404Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:11.510407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:11.510410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:11.510412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:11.510429Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:11.510443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:11.510449Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:11.510502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.510509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.510510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.510513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:11.510574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:11.510581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.510592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.510608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.510629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:11.510652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342634930398737:2432], now have 1 active actors on pipe
2025-10-09T21:50:11.510660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342634930398815:6476], now have 1 active actors on pipe
2025-10-09T21:50:11.510668Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342634568567996:3268]
2025-10-09T21:50:11.510670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342634568567996:3268]
2025-10-09T21:50:11.510671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342634568567996:3268]
2025-10-09T21:50:11.510685Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:11.510701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:11.510707Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:11.510689Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1746
2025-10-09T21:50:11.510711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:11.510690Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1746
2025-10-09T21:50:11.510713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:11.510721Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:11.510723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:11.510750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342634568567996:3268]
2025-10-09T21:50:11.510757Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:11.510763Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:11.510770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:11.510775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:11.510822Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bd4d7ef2-8fc6bb18-24990c44-61a3832c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:11.510834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:11.510838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:11.510840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:11.510850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:11.510863Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:11.510885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:11.511019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:11.511028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:11.511044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:11.511135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1747 partition: 0 MaxSeqNo: 25 sessionId: src_id|bd4d7ef2-8fc6bb18-24990c44-61a3832c_0
2025-10-09T21:50:11.511425Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7e5c272-861dd132-4cea36ff-af36c933_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bd4d7ef2-8fc6bb18-24990c44-61a3832c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:11.513801Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd4d7ef2-8fc6bb18-24990c44-61a3832c_0] Write session: try to update token
2025-10-09T21:50:11.513818Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd4d7ef2-8fc6bb18-24990c44-61a3832c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:11.525058Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:11.525084Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:11.525086Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:11.525088Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:11.525098Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342634568567996:3268]
2025-10-09T21:50:11.525103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342634568567996:3268]
2025-10-09T21:50:11.525106Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342634568567996:3268]
2025-10-09T21:50:11.525151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:11.525155Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:11.525159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.525161Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:11.525162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:11.525165Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:11.525303Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:11.526980Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1747 sessionId: src_id|bd4d7ef2-8fc6bb18-24990c44-61a3832c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:11.529430Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:11.593166Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342634930398822:6864]
2025-10-09T21:50:11.593449Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 1746, pipe: [1:7559342634930398822:6864]
2025-10-09T21:50:11.667248Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:11.667443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:11.667458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:11.667494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342634930398822:6864], now have 1 active actors on pipe
2025-10-09T21:50:11.667644Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:11.667653Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:11.667688Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342634930398822:6864]
2025-10-09T21:50:11.667697Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:11.667762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1747 reason: status is not ok: tablet will be restarted right now sessionId: src_id|bd4d7ef2-8fc6bb18-24990c44-61a3832c_0
2025-10-09T21:50:11.667913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1747 sessionId: src_id|bd4d7ef2-8fc6bb18-24990c44-61a3832c_0 is DEAD
2025-10-09T21:50:11.668076Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd4d7ef2-8fc6bb18-24990c44-61a3832c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:11.668088Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd4d7ef2-8fc6bb18-24990c44-61a3832c_0] Write session will restart in 0.006769s
2025-10-09T21:50:11.668123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd4d7ef2-8fc6bb18-24990c44-61a3832c_0] Write session: Do CDS request
2025-10-09T21:50:11.668130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd4d7ef2-8fc6bb18-24990c44-61a3832c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:11.674678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:11.676086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:11.676262Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:11.676272Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:11.676283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:11.676338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:11.676372Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342634568568115:3244]
2025-10-09T21:50:11.676390Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd4d7ef2-8fc6bb18-24990c44-61a3832c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:11.676484Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:11.676497Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1748
2025-10-09T21:50:11.676599Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1748 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:11.676625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1748 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:11.676635Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1748 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:11.676641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:11.676943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:11.676974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1748 sessionId:  describe result for acl check
2025-10-09T21:50:11.676997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:11.677002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:11.677003Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:11.677007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342634930398888:6461] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:11.677224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:11.677268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:11.677474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342634930398888:6461] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:11.677574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:11.677618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.677624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.677625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.677626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.677627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.677628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.677629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.677630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.677631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.677636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.677646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.677650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.677654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.677657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.677661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.677665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.677668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.677674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.677679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.677686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.677692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.677697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.677701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.677706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.677709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.677714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.677718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.677722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.677726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.677730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.677734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.677739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.677743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.677748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.677752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.677760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.677788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:11.679472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342634930398888:6461] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:11.683277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342634930398888:6461] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:11.683287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342634930398888:6461] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:11.683291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342634930398888:6461] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:11.683298Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1748 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:11.683718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:11.684734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:11.684751Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:11.684755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:11.684758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:11.684764Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1747 [2:7559342634568568115:3244]
2025-10-09T21:50:11.684776Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:11.684795Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:11.684799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:11.684801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:11.684804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:11.684820Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:11.684826Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:11.684832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:11.684885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.684892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.684895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.684898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:11.684957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:11.684964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.684975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.684991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.685010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:11.685034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342634930398923:6461], now have 1 active actors on pipe
2025-10-09T21:50:11.685042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342634930398849:2432], now have 1 active actors on pipe
2025-10-09T21:50:11.685049Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342634568568110:3244]
2025-10-09T21:50:11.685055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342634568568110:3244]
2025-10-09T21:50:11.685058Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342634568568110:3244]
2025-10-09T21:50:11.685062Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1747
2025-10-09T21:50:11.685068Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1747
2025-10-09T21:50:11.685071Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:11.685090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:11.685099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:11.685104Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:11.685105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:11.685106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:11.685109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:11.685146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:11.685156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:11.685167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342634568568110:3244]
2025-10-09T21:50:11.685178Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:11.685189Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:11.685211Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|eee0b376-de826d37-ce4de955-7594f6fa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:11.685222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:11.685226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:11.685227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:11.685237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:11.685250Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:11.685273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:11.685398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:11.685406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:11.685419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:11.685575Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1748 partition: 0 MaxSeqNo: 25 sessionId: src_id|eee0b376-de826d37-ce4de955-7594f6fa_0
2025-10-09T21:50:11.685871Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd4d7ef2-8fc6bb18-24990c44-61a3832c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|eee0b376-de826d37-ce4de955-7594f6fa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:11.688287Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eee0b376-de826d37-ce4de955-7594f6fa_0] Write session: try to update token
2025-10-09T21:50:11.688306Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eee0b376-de826d37-ce4de955-7594f6fa_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:11.698978Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:11.699000Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:11.699002Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:11.699004Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:11.699014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342634568568110:3244]
2025-10-09T21:50:11.699018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342634568568110:3244]
2025-10-09T21:50:11.699023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342634568568110:3244]
2025-10-09T21:50:11.699068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:11.699069Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:11.699083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.699085Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:11.699086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:11.699090Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:11.699224Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:11.700811Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1748 sessionId: src_id|eee0b376-de826d37-ce4de955-7594f6fa_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:11.702999Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:11.767788Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342634930398934:6864]
2025-10-09T21:50:11.768028Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 1747, pipe: [1:7559342634930398934:6864]
2025-10-09T21:50:11.842165Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:11.842479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:11.842491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:11.842530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342634930398934:6864], now have 1 active actors on pipe
2025-10-09T21:50:11.842713Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:11.842731Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342634930398934:6864]
2025-10-09T21:50:11.842736Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:11.842740Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:11.842763Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:11.842799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1748 reason: error at write: Disconnected sessionId: src_id|eee0b376-de826d37-ce4de955-7594f6fa_0
2025-10-09T21:50:11.842955Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1748 sessionId: src_id|eee0b376-de826d37-ce4de955-7594f6fa_0 is DEAD
2025-10-09T21:50:11.843106Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eee0b376-de826d37-ce4de955-7594f6fa_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:11.843116Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eee0b376-de826d37-ce4de955-7594f6fa_0] Write session will restart in 0.007663s
2025-10-09T21:50:11.843145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eee0b376-de826d37-ce4de955-7594f6fa_0] Write session: Do CDS request
2025-10-09T21:50:11.843152Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eee0b376-de826d37-ce4de955-7594f6fa_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:11.850339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:11.851532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eee0b376-de826d37-ce4de955-7594f6fa_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:11.851636Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:11.851654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1749
2025-10-09T21:50:11.851771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:11.851799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1749 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:11.851840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1749 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:11.851852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1749 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:11.851861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:11.851947Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:11.851956Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:11.851969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:11.852032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:11.852064Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342634568568196:3218]
2025-10-09T21:50:11.852245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1749 sessionId:  describe result for acl check
2025-10-09T21:50:11.852281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:11.852283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:11.852285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:11.852290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342634930398977:6453] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:11.852639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:11.852888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342634930398977:6453] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:11.852974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:11.853016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:11.853325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:11.853377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.853383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.853384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.853385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.853386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.853387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.853389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.853390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.853391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.853396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.853406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.853410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.853415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.853419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.853423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.853428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.853432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.853436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.853442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.853450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:11.853460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.853466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:11.853471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.853481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:11.853489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.853507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:11.853516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.853521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:11.853530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.853536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:11.853544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.853554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.853562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.853567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.853576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.853583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.853615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:11.854807Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342634930398977:6453] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:11.859268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342634930398977:6453] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:11.859282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342634930398977:6453] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:11.859285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342634930398977:6453] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:11.859293Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1749 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:11.859800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:11.860854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:11.860871Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:11.860876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:11.860880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:11.860887Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1748 [2:7559342634568568196:3218]
2025-10-09T21:50:11.860902Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:11.860917Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:11.860920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:11.860924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:11.860925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:11.860941Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:11.860951Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:11.860962Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:11.861017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.861024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.861026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.861030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:11.861091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:11.861098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.861110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:11.861126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:11.861143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:11.861166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342634930398937:2432], now have 1 active actors on pipe
2025-10-09T21:50:11.861174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342634930399016:6453], now have 1 active actors on pipe
2025-10-09T21:50:11.861182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342634568568191:3218]
2025-10-09T21:50:11.861184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342634568568191:3218]
2025-10-09T21:50:11.861186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342634568568191:3218]
2025-10-09T21:50:11.861200Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:11.861214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:11.861220Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:11.861223Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:11.861226Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:11.861231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:11.861239Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:11.861231Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1748
2025-10-09T21:50:11.861266Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1748
2025-10-09T21:50:11.861266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342634568568191:3218]
2025-10-09T21:50:11.861274Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:11.861287Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:11.861305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:11.861314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:11.861345Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|275a5ac2-976850f9-fb9c956f-b76bd658_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:11.861358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:11.861362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:11.861365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:11.861374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:11.861383Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:11.861396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:11.861556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:11.861566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:11.861580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:11.861664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1749 partition: 0 MaxSeqNo: 25 sessionId: src_id|275a5ac2-976850f9-fb9c956f-b76bd658_0
2025-10-09T21:50:11.861952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eee0b376-de826d37-ce4de955-7594f6fa_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|275a5ac2-976850f9-fb9c956f-b76bd658_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:11.864413Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|275a5ac2-976850f9-fb9c956f-b76bd658_0] Write session: try to update token
2025-10-09T21:50:11.864430Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|275a5ac2-976850f9-fb9c956f-b76bd658_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:11.875936Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:11.875957Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:11.875960Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:11.875962Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:11.875971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342634568568191:3218]
2025-10-09T21:50:11.875976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342634568568191:3218]
2025-10-09T21:50:11.875980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342634568568191:3218]
2025-10-09T21:50:11.876021Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:11.876025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:11.876032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:11.876033Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:11.876035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:11.876036Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:11.876161Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:11.876842Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1749 sessionId: src_id|275a5ac2-976850f9-fb9c956f-b76bd658_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:11.879316Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:11.942859Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342634930399026:6864]
2025-10-09T21:50:11.943095Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 1748, pipe: [1:7559342634930399026:6864]
2025-10-09T21:50:12.025130Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:12.025387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.025400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.025439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342634930399026:6864], now have 1 active actors on pipe
2025-10-09T21:50:12.025616Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:12.025651Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:12.025690Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342634930399026:6864]
2025-10-09T21:50:12.025712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:12.025797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1749 reason: status is not ok: tablet will be restarted right now sessionId: src_id|275a5ac2-976850f9-fb9c956f-b76bd658_0
2025-10-09T21:50:12.025972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1749 sessionId: src_id|275a5ac2-976850f9-fb9c956f-b76bd658_0 is DEAD
2025-10-09T21:50:12.026162Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|275a5ac2-976850f9-fb9c956f-b76bd658_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:12.026176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|275a5ac2-976850f9-fb9c956f-b76bd658_0] Write session will restart in 0.008885s
2025-10-09T21:50:12.026206Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|275a5ac2-976850f9-fb9c956f-b76bd658_0] Write session: Do CDS request
2025-10-09T21:50:12.026213Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|275a5ac2-976850f9-fb9c956f-b76bd658_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:12.033829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:12.035220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:12.035393Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:12.035404Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:12.035415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:12.035464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:12.035504Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342638863535570:3214]
2025-10-09T21:50:12.036095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:12.036549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|275a5ac2-976850f9-fb9c956f-b76bd658_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:12.036379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:12.036417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:12.036651Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:12.036668Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1750
2025-10-09T21:50:12.036715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:12.036763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.036769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.036770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.036771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.036773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.036774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.036775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.036776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.036779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.036784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.036793Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1750 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:12.036795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.036799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.036803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.036806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.036810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.036813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.036817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.036821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.036829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.036828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1750 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:12.036836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.036839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1750 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:12.036842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.036848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.036852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.036843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:12.036857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.036861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.036867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.036871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.036884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.036889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.036899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.036904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.036910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.036917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.036923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.036929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.036933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.036962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:12.037225Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1750 sessionId:  describe result for acl check
2025-10-09T21:50:12.037261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:12.037266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:12.037267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:12.037273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342639225366370:6443] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:12.037759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342639225366370:6443] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:12.039968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342639225366370:6443] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:12.042811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342639225366370:6443] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:12.042825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342639225366370:6443] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:12.042828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342639225366370:6443] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:12.042837Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1750 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:12.043310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:12.044349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:12.044367Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:12.044372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:12.044376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:12.044382Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1749 [2:7559342638863535570:3214]
2025-10-09T21:50:12.044396Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:12.044416Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:12.044423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:12.044427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:12.044434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:12.044450Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:12.044464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:12.044471Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:12.044521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.044527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.044530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.044533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:12.044596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:12.044604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.044615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.044631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.044648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:12.044674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366324:2432], now have 1 active actors on pipe
2025-10-09T21:50:12.044683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366404:6443], now have 1 active actors on pipe
2025-10-09T21:50:12.044690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342638863535565:3214]
2025-10-09T21:50:12.044695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342638863535565:3214]
2025-10-09T21:50:12.044697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342638863535565:3214]
2025-10-09T21:50:12.044710Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:12.044719Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:12.044696Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1749
2025-10-09T21:50:12.044723Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:12.044699Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1749
2025-10-09T21:50:12.044725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:12.044737Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:12.044741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:12.044744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:12.044782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342638863535565:3214]
2025-10-09T21:50:12.044795Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:12.044803Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:12.044808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.044813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.044856Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2d57a796-feb96754-9effa142-48b7c8b8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:12.044870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:12.044878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:12.044881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:12.044890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:12.044904Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:12.044924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:12.045097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.045107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.045123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:12.045218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1750 partition: 0 MaxSeqNo: 25 sessionId: src_id|2d57a796-feb96754-9effa142-48b7c8b8_0
2025-10-09T21:50:12.045542Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|275a5ac2-976850f9-fb9c956f-b76bd658_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2d57a796-feb96754-9effa142-48b7c8b8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:12.048000Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d57a796-feb96754-9effa142-48b7c8b8_0] Write session: try to update token
2025-10-09T21:50:12.048019Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d57a796-feb96754-9effa142-48b7c8b8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:12.059490Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:12.059511Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:12.059514Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:12.059516Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:12.059523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342638863535565:3214]
2025-10-09T21:50:12.059527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342638863535565:3214]
2025-10-09T21:50:12.059530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342638863535565:3214]
2025-10-09T21:50:12.059568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:12.059576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.059578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:12.059579Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:12.059586Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:12.059589Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:12.059701Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:12.061455Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1750 sessionId: src_id|2d57a796-feb96754-9effa142-48b7c8b8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:12.063914Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:12.125691Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342639225366412:6864]
2025-10-09T21:50:12.125939Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 1749, pipe: [1:7559342639225366412:6864]
2025-10-09T21:50:12.211009Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:12.211325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.211344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.211390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366412:6864], now have 1 active actors on pipe
2025-10-09T21:50:12.211614Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:12.211640Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:12.211644Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:12.211662Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342639225366412:6864]
2025-10-09T21:50:12.211621Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:12.211704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1750 reason: error at write: Disconnected sessionId: src_id|2d57a796-feb96754-9effa142-48b7c8b8_0
2025-10-09T21:50:12.211891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1750 sessionId: src_id|2d57a796-feb96754-9effa142-48b7c8b8_0 is DEAD
2025-10-09T21:50:12.212056Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d57a796-feb96754-9effa142-48b7c8b8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:12.212064Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d57a796-feb96754-9effa142-48b7c8b8_0] Write session will restart in 0.009938s
2025-10-09T21:50:12.212084Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d57a796-feb96754-9effa142-48b7c8b8_0] Write session: Do CDS request
2025-10-09T21:50:12.212089Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d57a796-feb96754-9effa142-48b7c8b8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:12.219573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:12.221099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:12.221305Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:12.221314Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:12.221327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:12.221378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:12.221424Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342638863535649:3211]
2025-10-09T21:50:12.222049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:12.222319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:12.222361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:12.222671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:12.222722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.222728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.222729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.222730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.222731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.222732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.222734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.222735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.222736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.222741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.222753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.222758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.222762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.222766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.222770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.222774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.222778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.222782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.222789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.222797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.222803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.222814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.222822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.222827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.222831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.222837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.222841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.222847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.222857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.222862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.222871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.222876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.222880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.222890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.222894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.222899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.222930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:12.223558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d57a796-feb96754-9effa142-48b7c8b8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:12.223660Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:12.223674Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1751
2025-10-09T21:50:12.223789Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1751 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:12.223815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1751 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:12.223818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1751 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:12.223825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:12.224128Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1751 sessionId:  describe result for acl check
2025-10-09T21:50:12.224155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:12.224156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:12.224157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:12.224161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342639225366464:6439] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:12.224651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342639225366464:6439] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:12.226783Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342639225366464:6439] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:12.229116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342639225366464:6439] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:12.229129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342639225366464:6439] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:12.229133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342639225366464:6439] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:12.229152Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1751 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:12.229400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:12.230403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:12.230418Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:12.230420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:12.230423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:12.230429Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1750 [2:7559342638863535649:3211]
2025-10-09T21:50:12.230442Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:12.230461Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:12.230470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:12.230473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:12.230476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:12.230492Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:12.230500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:12.230506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:12.230553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.230558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.230560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.230562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:12.230616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:12.230622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.230629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.230639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.230652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:12.230667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366498:6439], now have 1 active actors on pipe
2025-10-09T21:50:12.230674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366417:2432], now have 1 active actors on pipe
2025-10-09T21:50:12.230679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342638863535644:3211]
2025-10-09T21:50:12.230680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342638863535644:3211]
2025-10-09T21:50:12.230682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342638863535644:3211]
2025-10-09T21:50:12.230693Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:12.230708Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:12.230710Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:12.230713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:12.230715Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:12.230724Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:12.230727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:12.230742Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1750
2025-10-09T21:50:12.230751Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1750
2025-10-09T21:50:12.230760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342638863535644:3211]
2025-10-09T21:50:12.230769Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:12.230771Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:12.230790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.230793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.230818Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|31e55912-a2979a56-7c419c07-61d06d4f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:12.230827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:12.230832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:12.230833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:12.230841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:12.230852Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:12.230876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:12.230995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.231005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.231018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:12.231072Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1751 partition: 0 MaxSeqNo: 25 sessionId: src_id|31e55912-a2979a56-7c419c07-61d06d4f_0
2025-10-09T21:50:12.231375Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d57a796-feb96754-9effa142-48b7c8b8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|31e55912-a2979a56-7c419c07-61d06d4f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:12.233865Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|31e55912-a2979a56-7c419c07-61d06d4f_0] Write session: try to update token
2025-10-09T21:50:12.233882Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|31e55912-a2979a56-7c419c07-61d06d4f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:12.245597Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:12.245617Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:12.245619Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:12.245621Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:12.245632Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342638863535644:3211]
2025-10-09T21:50:12.245639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342638863535644:3211]
2025-10-09T21:50:12.245643Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342638863535644:3211]
2025-10-09T21:50:12.245681Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:12.245686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:12.245693Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:12.245693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.245695Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:12.245696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:12.245826Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:12.246665Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1751 sessionId: src_id|31e55912-a2979a56-7c419c07-61d06d4f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:12.248990Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:12.312056Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342639225366511:6864]
2025-10-09T21:50:12.312349Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 1750, pipe: [1:7559342639225366511:6864]
2025-10-09T21:50:12.397856Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:12.398264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.398280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.398324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366511:6864], now have 1 active actors on pipe
2025-10-09T21:50:12.398538Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:12.398554Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:12.398565Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:12.398559Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:12.398617Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342639225366511:6864]
2025-10-09T21:50:12.398644Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1751 reason: error at write: Disconnected sessionId: src_id|31e55912-a2979a56-7c419c07-61d06d4f_0
2025-10-09T21:50:12.398827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1751 sessionId: src_id|31e55912-a2979a56-7c419c07-61d06d4f_0 is DEAD
2025-10-09T21:50:12.398986Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31e55912-a2979a56-7c419c07-61d06d4f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:12.398998Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31e55912-a2979a56-7c419c07-61d06d4f_0] Write session will restart in 0.008210s
2025-10-09T21:50:12.399028Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31e55912-a2979a56-7c419c07-61d06d4f_0] Write session: Do CDS request
2025-10-09T21:50:12.399035Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31e55912-a2979a56-7c419c07-61d06d4f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:12.406604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:12.408177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:12.408340Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:12.408349Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:12.408360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:12.408409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:12.408447Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342638863535729:3208]
2025-10-09T21:50:12.408510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|31e55912-a2979a56-7c419c07-61d06d4f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:12.408587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:12.408603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1752
2025-10-09T21:50:12.408723Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1752 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:12.408764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1752 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:12.408773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1752 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:12.408777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:12.409036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:12.409179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1752 sessionId:  describe result for acl check
2025-10-09T21:50:12.409215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:12.409219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:12.409220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:12.409226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342639225366558:6432] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:12.409290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:12.409330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:12.409645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:12.409691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.409696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.409698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.409699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.409700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.409701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.409702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.409703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.409705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.409710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.409720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.409724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.409728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.409732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.409738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.409741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.409745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.409749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.409755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.409763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.409768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.409772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.409776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.409767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342639225366558:6432] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:12.409782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.409786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.409791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.409795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.409800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.409810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.409814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.409819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.409828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.409832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.409837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.409841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.409846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.409873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:12.411507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342639225366558:6432] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:12.415671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342639225366558:6432] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:12.415682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342639225366558:6432] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:12.415686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342639225366558:6432] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:12.415695Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1752 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:12.416098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:12.417195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:12.417213Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:12.417217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:12.417222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:12.417230Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1751 [2:7559342638863535729:3208]
2025-10-09T21:50:12.417242Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:12.417259Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:12.417262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:12.417265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:12.417267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:12.417283Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:12.417288Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:12.417294Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:12.417346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.417353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.417355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.417358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:12.417419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:12.417427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.417439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.417451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.417460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:12.417480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366594:6432], now have 1 active actors on pipe
2025-10-09T21:50:12.417486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366513:2432], now have 1 active actors on pipe
2025-10-09T21:50:12.417491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342638863535724:3208]
2025-10-09T21:50:12.417504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342638863535724:3208]
2025-10-09T21:50:12.417506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342638863535724:3208]
2025-10-09T21:50:12.417518Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:12.417541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:12.417545Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:12.417549Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:12.417551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:12.417552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:12.417554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:12.417550Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1751
2025-10-09T21:50:12.417558Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1751
2025-10-09T21:50:12.417579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342638863535724:3208]
2025-10-09T21:50:12.417588Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:12.417590Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:12.417607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.417616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.417658Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|633a1912-9eae183f-7bf39b38-c9a86ac0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:12.417672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:12.417675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:12.417678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:12.417689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:12.417698Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:12.417714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:12.417890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.417905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.417926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:12.418048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1752 partition: 0 MaxSeqNo: 25 sessionId: src_id|633a1912-9eae183f-7bf39b38-c9a86ac0_0
2025-10-09T21:50:12.418323Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|31e55912-a2979a56-7c419c07-61d06d4f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|633a1912-9eae183f-7bf39b38-c9a86ac0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:12.420737Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|633a1912-9eae183f-7bf39b38-c9a86ac0_0] Write session: try to update token
2025-10-09T21:50:12.420757Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|633a1912-9eae183f-7bf39b38-c9a86ac0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:12.431790Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:12.431809Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:12.431812Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:12.431813Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:12.431822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342638863535724:3208]
2025-10-09T21:50:12.431825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342638863535724:3208]
2025-10-09T21:50:12.431829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342638863535724:3208]
2025-10-09T21:50:12.431863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:12.431868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.431873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:12.431872Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:12.431878Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:12.431882Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:12.431991Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:12.433682Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1752 sessionId: src_id|633a1912-9eae183f-7bf39b38-c9a86ac0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:12.436362Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:12.498932Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342639225366603:6864]
2025-10-09T21:50:12.499173Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 1751, pipe: [1:7559342639225366603:6864]
2025-10-09T21:50:12.585090Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:12.585341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.585353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.585393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366603:6864], now have 1 active actors on pipe
2025-10-09T21:50:12.585572Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:12.585662Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:12.585702Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342639225366603:6864]
2025-10-09T21:50:12.585722Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:12.585808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1752 reason: status is not ok: tablet will be restarted right now sessionId: src_id|633a1912-9eae183f-7bf39b38-c9a86ac0_0
2025-10-09T21:50:12.586009Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1752 sessionId: src_id|633a1912-9eae183f-7bf39b38-c9a86ac0_0 is DEAD
2025-10-09T21:50:12.586201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|633a1912-9eae183f-7bf39b38-c9a86ac0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:12.586214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|633a1912-9eae183f-7bf39b38-c9a86ac0_0] Write session will restart in 0.007518s
2025-10-09T21:50:12.586244Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|633a1912-9eae183f-7bf39b38-c9a86ac0_0] Write session: Do CDS request
2025-10-09T21:50:12.586251Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|633a1912-9eae183f-7bf39b38-c9a86ac0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:12.593144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:12.594521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|633a1912-9eae183f-7bf39b38-c9a86ac0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:12.594603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:12.594619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1753
2025-10-09T21:50:12.594657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:12.594759Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1753 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:12.594793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1753 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:12.594801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1753 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:12.594814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:12.594811Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:12.594818Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:12.594827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:12.594863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:12.594896Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342638863535810:3206]
2025-10-09T21:50:12.595204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1753 sessionId:  describe result for acl check
2025-10-09T21:50:12.595238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:12.595246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:12.595261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:12.595267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342639225366649:6428] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:12.595285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:12.595511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:12.595545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:12.595745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:12.595790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.595796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.595796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.595797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.595800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.595801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.595801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.595802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.595803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.595806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.595812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.595814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.595817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.595819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.595821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.595823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.595825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.595827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.595831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.595836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.595840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.595843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.595846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.595849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.595852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.595845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342639225366649:6428] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:12.595855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.595857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.595860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.595863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.595868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.595873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.595878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.595881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.595886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.595891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.595903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.595927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:12.597979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342639225366649:6428] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:12.601747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342639225366649:6428] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:12.601763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342639225366649:6428] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:12.601767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342639225366649:6428] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:12.601778Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1753 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:12.602190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:12.603232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:12.603248Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:12.603249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:12.603253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:12.603257Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1752 [2:7559342638863535810:3206]
2025-10-09T21:50:12.603270Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:12.603286Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:12.603289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:12.603291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:12.603292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:12.603307Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:12.603311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:12.603317Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:12.603358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.603364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.603365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.603367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:12.603414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:12.603419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.603426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.603432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.603442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:12.603456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342638863535805:3206]
2025-10-09T21:50:12.603457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342638863535805:3206]
2025-10-09T21:50:12.603458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342638863535805:3206]
2025-10-09T21:50:12.603464Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:12.603475Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:12.603476Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:12.603478Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:12.603480Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:12.603490Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:12.603494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:12.603504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366688:6428], now have 1 active actors on pipe
2025-10-09T21:50:12.603508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366605:2432], now have 1 active actors on pipe
2025-10-09T21:50:12.603534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342638863535805:3206]
2025-10-09T21:50:12.603545Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:12.603550Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1752
2025-10-09T21:50:12.603551Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1752
2025-10-09T21:50:12.603600Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:12.603658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.603666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.603691Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5a566ceb-3976a715-6f4a6d26-134c582e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:12.603701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:12.603705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:12.603706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:12.603713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:12.603720Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:12.603734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:12.603839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.603844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.603855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:12.603926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1753 partition: 0 MaxSeqNo: 25 sessionId: src_id|5a566ceb-3976a715-6f4a6d26-134c582e_0
2025-10-09T21:50:12.604275Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|633a1912-9eae183f-7bf39b38-c9a86ac0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5a566ceb-3976a715-6f4a6d26-134c582e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:12.606620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a566ceb-3976a715-6f4a6d26-134c582e_0] Write session: try to update token
2025-10-09T21:50:12.606637Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a566ceb-3976a715-6f4a6d26-134c582e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:12.618252Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:12.618273Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:12.618276Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:12.618278Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:12.618288Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342638863535805:3206]
2025-10-09T21:50:12.618292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342638863535805:3206]
2025-10-09T21:50:12.618296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342638863535805:3206]
2025-10-09T21:50:12.618332Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:12.618334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:12.618337Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:12.618340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.618343Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:12.618345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:12.618500Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:12.619008Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1753 sessionId: src_id|5a566ceb-3976a715-6f4a6d26-134c582e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:12.621218Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:12.685781Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342639225366709:6864]
2025-10-09T21:50:12.686082Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 1752, pipe: [1:7559342639225366709:6864]
2025-10-09T21:50:12.772168Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:12.772492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.772506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.772543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366709:6864], now have 1 active actors on pipe
2025-10-09T21:50:12.772732Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:12.772747Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:12.772781Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342639225366709:6864]
2025-10-09T21:50:12.772808Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:12.772818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1753 reason: error at write: Disconnected sessionId: src_id|5a566ceb-3976a715-6f4a6d26-134c582e_0
2025-10-09T21:50:12.772999Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1753 sessionId: src_id|5a566ceb-3976a715-6f4a6d26-134c582e_0 is DEAD
2025-10-09T21:50:12.773163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a566ceb-3976a715-6f4a6d26-134c582e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:12.773176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a566ceb-3976a715-6f4a6d26-134c582e_0] Write session will restart in 0.008488s
2025-10-09T21:50:12.773200Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a566ceb-3976a715-6f4a6d26-134c582e_0] Write session: Do CDS request
2025-10-09T21:50:12.773209Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a566ceb-3976a715-6f4a6d26-134c582e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:12.773833Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:12.782233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:12.782484Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5a566ceb-3976a715-6f4a6d26-134c582e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:12.782554Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:12.782570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1754
2025-10-09T21:50:12.782692Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1754 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:12.782724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1754 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:12.782743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1754 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:12.782751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:12.783148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1754 sessionId:  describe result for acl check
2025-10-09T21:50:12.783188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:12.783194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:12.783195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:12.783199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342639225366766:6410] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:12.783592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342639225366766:6410] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:12.783734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:12.783891Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:12.783899Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:12.783909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:12.783963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:12.784004Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342638863535934:3179]
2025-10-09T21:50:12.784557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:12.784833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:12.784878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:12.785116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:12.785156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.785163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.785164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.785164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.785165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.785166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.785166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.785167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.785168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.785171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.785177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.785180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.785182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.785184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.785186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.785188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.785190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.785192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.785197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.785205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.785210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.785215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.785218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.785223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.785227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.785232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.785235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.785240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.785244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.785249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.785255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.785260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.785264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.785269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.785274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.785279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.785307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:12.785430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342639225366766:6410] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:12.791547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342639225366766:6410] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:12.791560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342639225366766:6410] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:12.791563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342639225366766:6410] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:12.791570Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1754 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:12.791653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:12.792861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:12.792877Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:12.792880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:12.792885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:12.792890Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1753 [2:7559342638863535934:3179]
2025-10-09T21:50:12.792901Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:12.792914Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:12.792916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:12.792918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:12.792920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:12.792932Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:12.792940Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:12.792946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:12.792990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.792995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.792998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.793001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:12.793066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:12.793074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.793086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.793098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.793119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:12.793144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366721:2432], now have 1 active actors on pipe
2025-10-09T21:50:12.793153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366805:6410], now have 1 active actors on pipe
2025-10-09T21:50:12.793160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342638863535929:3179]
2025-10-09T21:50:12.793168Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342638863535929:3179]
2025-10-09T21:50:12.793170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342638863535929:3179]
2025-10-09T21:50:12.793185Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:12.793188Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1753
2025-10-09T21:50:12.793199Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:12.793223Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1753
2025-10-09T21:50:12.793204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:12.793211Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:12.793213Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:12.793217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:12.793220Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:12.793249Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342638863535929:3179]
2025-10-09T21:50:12.793254Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:12.793260Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:12.793275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.793287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.793333Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c12f6e6b-36654f73-398fe8ad-c5447fdc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:12.793346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:12.793353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:12.793356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:12.793366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:12.793377Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:12.793392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:12.793582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.793593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.793609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:12.793687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1754 partition: 0 MaxSeqNo: 25 sessionId: src_id|c12f6e6b-36654f73-398fe8ad-c5447fdc_0
2025-10-09T21:50:12.793992Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5a566ceb-3976a715-6f4a6d26-134c582e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c12f6e6b-36654f73-398fe8ad-c5447fdc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:12.796395Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c12f6e6b-36654f73-398fe8ad-c5447fdc_0] Write session: try to update token
2025-10-09T21:50:12.796410Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c12f6e6b-36654f73-398fe8ad-c5447fdc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:12.808357Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:12.808384Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:12.808386Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:12.808388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:12.808398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342638863535929:3179]
2025-10-09T21:50:12.808401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342638863535929:3179]
2025-10-09T21:50:12.808404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342638863535929:3179]
2025-10-09T21:50:12.808442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:12.808458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.808465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:12.808465Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:12.808470Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:12.808475Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:12.808600Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:12.809386Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1754 sessionId: src_id|c12f6e6b-36654f73-398fe8ad-c5447fdc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:12.811849Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:12.872674Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342639225366814:6864]
2025-10-09T21:50:12.872959Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 1753, pipe: [1:7559342639225366814:6864]
2025-10-09T21:50:12.957303Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:12.957601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.957613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.957648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366814:6864], now have 1 active actors on pipe
2025-10-09T21:50:12.957798Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:12.957862Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:12.957887Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:12.957918Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342639225366814:6864]
2025-10-09T21:50:12.957950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1754 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c12f6e6b-36654f73-398fe8ad-c5447fdc_0
2025-10-09T21:50:12.958103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1754 sessionId: src_id|c12f6e6b-36654f73-398fe8ad-c5447fdc_0 is DEAD
2025-10-09T21:50:12.958230Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c12f6e6b-36654f73-398fe8ad-c5447fdc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:12.958240Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c12f6e6b-36654f73-398fe8ad-c5447fdc_0] Write session will restart in 0.005074s
2025-10-09T21:50:12.958260Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c12f6e6b-36654f73-398fe8ad-c5447fdc_0] Write session: Do CDS request
2025-10-09T21:50:12.958266Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c12f6e6b-36654f73-398fe8ad-c5447fdc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:12.964441Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c12f6e6b-36654f73-398fe8ad-c5447fdc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:12.964520Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:12.964533Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1755
2025-10-09T21:50:12.964652Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1755 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:12.964683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1755 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:12.964693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1755 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:12.964696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:12.964914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:12.965056Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1755 sessionId:  describe result for acl check
2025-10-09T21:50:12.965084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:12.965086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:12.965089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:12.965094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342639225366865:6401] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:12.965661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342639225366865:6401] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:12.966264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:12.966491Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:12.966503Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:12.966516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:12.966582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:12.966620Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342638863536017:3172]
2025-10-09T21:50:12.967029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:12.967254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:12.967290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:12.967571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342639225366865:6401] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:12.967579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:12.967623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.967628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.967630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.967631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.967632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.967633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.967634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.967635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.967636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.967641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.967651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.967659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.967662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.967670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.967674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.967678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.967681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.967685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.967690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.967697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:12.967703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.967708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:12.967713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.967718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:12.967722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.967727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:12.967733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.967742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:12.967746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.967755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:12.967764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.967772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.967776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.967785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.967793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.967800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.967829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:12.972854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342639225366865:6401] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:12.972872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342639225366865:6401] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:12.972876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342639225366865:6401] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:12.972887Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1755 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:12.974694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:12.975804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:12.975818Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:12.975821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:12.975824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:12.975829Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1754 [2:7559342638863536017:3172]
2025-10-09T21:50:12.975840Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:12.975853Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:12.975858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:12.975862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:12.975863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:12.975877Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:12.975881Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:12.975886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:12.975939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.975946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.975948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.975951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:12.976019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:12.976027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.976038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:12.976049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:12.976065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:12.976085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366900:6401], now have 1 active actors on pipe
2025-10-09T21:50:12.976095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342639225366819:2432], now have 1 active actors on pipe
2025-10-09T21:50:12.976102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342638863536012:3172]
2025-10-09T21:50:12.976105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342638863536012:3172]
2025-10-09T21:50:12.976107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342638863536012:3172]
2025-10-09T21:50:12.976121Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:12.976115Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1754
2025-10-09T21:50:12.976140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:12.976146Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:12.976150Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:12.976153Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:12.976161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1754
2025-10-09T21:50:12.976158Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:12.976161Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:12.976196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342638863536012:3172]
2025-10-09T21:50:12.976207Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:12.976221Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:12.976225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.976229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.976303Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b6c7a971-a3d6a077-2fc3f2fd-fa632107_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:12.976316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:12.976324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:12.976326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:12.976337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:12.976351Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:12.976372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:12.976519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:12.976528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:12.976542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:12.976660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1755 partition: 0 MaxSeqNo: 25 sessionId: src_id|b6c7a971-a3d6a077-2fc3f2fd-fa632107_0
2025-10-09T21:50:12.976978Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c12f6e6b-36654f73-398fe8ad-c5447fdc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b6c7a971-a3d6a077-2fc3f2fd-fa632107_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:12.979398Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6c7a971-a3d6a077-2fc3f2fd-fa632107_0] Write session: try to update token
2025-10-09T21:50:12.979417Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6c7a971-a3d6a077-2fc3f2fd-fa632107_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:12.990373Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:12.990395Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:12.990398Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:12.990400Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:12.990413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342638863536012:3172]
2025-10-09T21:50:12.990418Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342638863536012:3172]
2025-10-09T21:50:12.990424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342638863536012:3172]
2025-10-09T21:50:12.990467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:12.990476Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:12.990479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:12.990482Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:12.990483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:12.990486Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:12.990613Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:12.992033Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1755 sessionId: src_id|b6c7a971-a3d6a077-2fc3f2fd-fa632107_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:12.994232Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:13.058081Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342643520334204:6864]
2025-10-09T21:50:13.058374Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 1754, pipe: [1:7559342643520334204:6864]
2025-10-09T21:50:13.142242Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:13.142552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:13.142569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:13.142618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342643520334204:6864], now have 1 active actors on pipe
2025-10-09T21:50:13.142803Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:13.142824Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342643520334204:6864]
2025-10-09T21:50:13.142852Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:13.142923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1755 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b6c7a971-a3d6a077-2fc3f2fd-fa632107_0
2025-10-09T21:50:13.143029Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:13.143079Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1755 sessionId: src_id|b6c7a971-a3d6a077-2fc3f2fd-fa632107_0 is DEAD
2025-10-09T21:50:13.143245Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6c7a971-a3d6a077-2fc3f2fd-fa632107_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:13.143258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6c7a971-a3d6a077-2fc3f2fd-fa632107_0] Write session will restart in 0.006333s
2025-10-09T21:50:13.143287Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6c7a971-a3d6a077-2fc3f2fd-fa632107_0] Write session: Do CDS request
2025-10-09T21:50:13.143293Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6c7a971-a3d6a077-2fc3f2fd-fa632107_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:13.149337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:13.150432Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6c7a971-a3d6a077-2fc3f2fd-fa632107_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:13.150513Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:13.150525Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1756
2025-10-09T21:50:13.150659Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1756 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:13.150702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1756 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:13.150712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1756 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:13.150715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:13.150728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:13.150898Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:13.150908Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:13.150920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:13.150967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:13.151017Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342643158503398:3169]
2025-10-09T21:50:13.151126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1756 sessionId:  describe result for acl check
2025-10-09T21:50:13.151158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:13.151166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:13.151168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:13.151174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342643520334256:6396] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:13.151587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:13.151694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342643520334256:6396] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:13.151830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:13.151871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:13.152171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:13.152217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.152224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.152225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.152226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.152227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.152229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.152230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.152231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.152233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.152238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.152248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.152254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.152258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.152267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.152271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.152279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.152283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.152286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.152299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.152311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.152321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.152331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.152339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.152346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.152353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.152358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.152367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.152377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.152385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.152390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.152395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.152404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.152413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.152418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.152427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.152437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.152471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:13.153446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342643520334256:6396] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:13.158048Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342643520334256:6396] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:13.158063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342643520334256:6396] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:13.158067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342643520334256:6396] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:13.158076Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1756 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:13.158523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:13.159637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:13.159652Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:13.159654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:13.159657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:13.159662Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1755 [2:7559342643158503398:3169]
2025-10-09T21:50:13.159677Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:13.159694Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:13.159702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:13.159706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:13.159713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:13.159729Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:13.159740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:13.159751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:13.159802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.159810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.159812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.159815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:13.159876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:13.159884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.159891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.159898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.159914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:13.159933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342643520334294:6396], now have 1 active actors on pipe
2025-10-09T21:50:13.159942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342643520334209:2432], now have 1 active actors on pipe
2025-10-09T21:50:13.159950Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342643158503393:3169]
2025-10-09T21:50:13.159952Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342643158503393:3169]
2025-10-09T21:50:13.159954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342643158503393:3169]
2025-10-09T21:50:13.159968Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:13.159989Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:13.159991Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:13.160003Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:13.160005Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:13.160007Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:13.160011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:13.160036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342643158503393:3169]
2025-10-09T21:50:13.160045Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:13.160047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:13.160067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:13.160001Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1755
2025-10-09T21:50:13.160072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:13.160007Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1755
2025-10-09T21:50:13.160108Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|32efc3ab-c1c4a4af-d88f8de9-625ab1e6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:13.160121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:13.160127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:13.160129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:13.160139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:13.160148Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:13.160164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:13.160314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:13.160325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:13.160341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:13.160428Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1756 partition: 0 MaxSeqNo: 25 sessionId: src_id|32efc3ab-c1c4a4af-d88f8de9-625ab1e6_0
2025-10-09T21:50:13.160730Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6c7a971-a3d6a077-2fc3f2fd-fa632107_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|32efc3ab-c1c4a4af-d88f8de9-625ab1e6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:13.163481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32efc3ab-c1c4a4af-d88f8de9-625ab1e6_0] Write session: try to update token
2025-10-09T21:50:13.163499Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32efc3ab-c1c4a4af-d88f8de9-625ab1e6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:13.174372Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:13.174395Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:13.174398Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:13.174400Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:13.174410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342643158503393:3169]
2025-10-09T21:50:13.174415Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342643158503393:3169]
2025-10-09T21:50:13.174421Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342643158503393:3169]
2025-10-09T21:50:13.174465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:13.174467Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:13.174472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.174472Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:13.174475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:13.174476Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:13.174616Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:13.177050Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1756 sessionId: src_id|32efc3ab-c1c4a4af-d88f8de9-625ab1e6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:13.179578Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:13.242867Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342643520334304:6864]
2025-10-09T21:50:13.243134Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 1755, pipe: [1:7559342643520334304:6864]
2025-10-09T21:50:13.321445Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:13.321751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:13.321765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:13.321800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342643520334304:6864], now have 1 active actors on pipe
2025-10-09T21:50:13.321987Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:13.322061Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:13.322075Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:13.322076Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:13.322132Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342643520334304:6864]
2025-10-09T21:50:13.322136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1756 reason: error at write: Disconnected sessionId: src_id|32efc3ab-c1c4a4af-d88f8de9-625ab1e6_0
2025-10-09T21:50:13.322314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1756 sessionId: src_id|32efc3ab-c1c4a4af-d88f8de9-625ab1e6_0 is DEAD
2025-10-09T21:50:13.322491Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32efc3ab-c1c4a4af-d88f8de9-625ab1e6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:13.322504Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32efc3ab-c1c4a4af-d88f8de9-625ab1e6_0] Write session will restart in 0.006775s
2025-10-09T21:50:13.322540Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32efc3ab-c1c4a4af-d88f8de9-625ab1e6_0] Write session: Do CDS request
2025-10-09T21:50:13.322547Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32efc3ab-c1c4a4af-d88f8de9-625ab1e6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:13.330522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32efc3ab-c1c4a4af-d88f8de9-625ab1e6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:13.330569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:13.330610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:13.330626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1757
2025-10-09T21:50:13.330738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1757 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:13.330772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1757 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:13.330783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1757 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:13.330787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:13.331245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1757 sessionId:  describe result for acl check
2025-10-09T21:50:13.331289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:13.331296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:13.331297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:13.331304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342643520334360:6387] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:13.331935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342643520334360:6387] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:13.332028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:13.332204Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:13.332220Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:13.332232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:13.332282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:13.332328Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342643158503483:3159]
2025-10-09T21:50:13.332896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:13.333177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:13.333222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:13.333554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:13.333617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.333623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.333624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.333626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.333626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.333627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.333628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.333629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.333630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.333634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.333644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.333648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.333651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.333655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.333658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.333662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.333665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.333668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.333674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.333682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.333688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.333693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.333697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.333701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.333705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.333710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.333714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.333718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.333722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.333726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.333731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.333735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.333739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.333743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.333747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.333751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.333742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342643520334360:6387] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:13.333777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:13.339754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342643520334360:6387] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:13.339769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342643520334360:6387] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:13.339773Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342643520334360:6387] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:13.339786Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1757 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:13.340221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:13.341409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:13.341423Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:13.341426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:13.341430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:13.341436Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1756 [2:7559342643158503483:3159]
2025-10-09T21:50:13.341449Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:13.341467Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:13.341472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:13.341476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:13.341478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:13.341507Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:13.341520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:13.341527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:13.341580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.341587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.341589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.341592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:13.341657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:13.341666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.341679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.341693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.341711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:13.341737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342643520334312:2432], now have 1 active actors on pipe
2025-10-09T21:50:13.341745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342643520334399:6387], now have 1 active actors on pipe
2025-10-09T21:50:13.341753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342643158503478:3159]
2025-10-09T21:50:13.341756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342643158503478:3159]
2025-10-09T21:50:13.341759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342643158503478:3159]
2025-10-09T21:50:13.341771Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:13.341786Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:13.341790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:13.341798Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:13.341800Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:13.341805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:13.341808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:13.341798Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1756
2025-10-09T21:50:13.341803Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1756
2025-10-09T21:50:13.341835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342643158503478:3159]
2025-10-09T21:50:13.341845Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:13.341846Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:13.341860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:13.341869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:13.341918Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|69bd3f99-19cf68d7-cee8ac8-32379a55_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:13.341932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:13.341936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:13.341938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:13.341948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:13.341959Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:13.341979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:13.342169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:13.342177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:13.342191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:13.342281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1757 partition: 0 MaxSeqNo: 25 sessionId: src_id|69bd3f99-19cf68d7-cee8ac8-32379a55_0
2025-10-09T21:50:13.342587Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32efc3ab-c1c4a4af-d88f8de9-625ab1e6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|69bd3f99-19cf68d7-cee8ac8-32379a55_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:13.345065Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|69bd3f99-19cf68d7-cee8ac8-32379a55_0] Write session: try to update token
2025-10-09T21:50:13.345082Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|69bd3f99-19cf68d7-cee8ac8-32379a55_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:13.356735Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:13.356756Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:13.356757Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:13.356759Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:13.356766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342643158503478:3159]
2025-10-09T21:50:13.356771Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342643158503478:3159]
2025-10-09T21:50:13.356775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342643158503478:3159]
2025-10-09T21:50:13.356825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:13.356833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.356836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:13.356836Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:13.356843Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:13.356846Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:13.357007Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:13.358215Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1757 sessionId: src_id|69bd3f99-19cf68d7-cee8ac8-32379a55_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:13.360512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:13.422025Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342643520334406:6864]
2025-10-09T21:50:13.422293Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 1756, pipe: [1:7559342643520334406:6864]
2025-10-09T21:50:13.502285Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:13.502525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:13.502540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:13.502576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342643520334406:6864], now have 1 active actors on pipe
2025-10-09T21:50:13.502697Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:13.502712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:13.502718Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:13.502728Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:13.502760Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342643520334406:6864]
2025-10-09T21:50:13.502802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1757 reason: error at write: Disconnected sessionId: src_id|69bd3f99-19cf68d7-cee8ac8-32379a55_0
2025-10-09T21:50:13.502976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1757 sessionId: src_id|69bd3f99-19cf68d7-cee8ac8-32379a55_0 is DEAD
2025-10-09T21:50:13.503140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69bd3f99-19cf68d7-cee8ac8-32379a55_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:13.503148Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69bd3f99-19cf68d7-cee8ac8-32379a55_0] Write session will restart in 0.007688s
2025-10-09T21:50:13.503175Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69bd3f99-19cf68d7-cee8ac8-32379a55_0] Write session: Do CDS request
2025-10-09T21:50:13.503182Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69bd3f99-19cf68d7-cee8ac8-32379a55_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:13.510175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:13.511457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:13.511487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|69bd3f99-19cf68d7-cee8ac8-32379a55_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:13.511569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:13.511583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1758
2025-10-09T21:50:13.511626Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:13.511639Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:13.511650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:13.511688Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1758 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:13.511704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:13.511718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1758 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:13.511729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1758 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:13.511732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:13.511742Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342643158503569:3157]
2025-10-09T21:50:13.512113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1758 sessionId:  describe result for acl check
2025-10-09T21:50:13.512146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:13.512152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:13.512154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:13.512159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342643520334458:6381] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:13.512302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:13.512568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:13.512608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:13.512682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342643520334458:6381] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:13.512907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:13.512960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.512965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.512967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.512968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.512969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.512972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.512973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.512974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.512976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.512980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.512990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.512994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.512998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.513001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.513005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.513009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.513013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.513017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.513023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.513031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.513036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.513041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.513046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.513051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.513055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.513060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.513069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.513073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.513079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.513083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.513088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.513097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.513102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.513111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.513116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.513120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.513154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:13.514465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342643520334458:6381] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:13.518632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342643520334458:6381] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:13.518647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342643520334458:6381] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:13.518651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342643520334458:6381] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:13.518658Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1758 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:13.519117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:13.520232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:13.520247Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:13.520249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:13.520253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:13.520260Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1757 [2:7559342643158503569:3157]
2025-10-09T21:50:13.520275Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:13.520297Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:13.520304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:13.520307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:13.520309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:13.520324Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:13.520330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:13.520337Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:13.520389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.520395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.520397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.520400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:13.520462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:13.520470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.520481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.520496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.520510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:13.520529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342643520334409:2432], now have 1 active actors on pipe
2025-10-09T21:50:13.520535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342643520334497:6381], now have 1 active actors on pipe
2025-10-09T21:50:13.520539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342643158503564:3157]
2025-10-09T21:50:13.520541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342643158503564:3157]
2025-10-09T21:50:13.520542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342643158503564:3157]
2025-10-09T21:50:13.520552Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:13.520563Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:13.520570Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:13.520582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:13.520583Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:13.520584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:13.520586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:13.520571Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1757
2025-10-09T21:50:13.520584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1757
2025-10-09T21:50:13.520611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342643158503564:3157]
2025-10-09T21:50:13.520618Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:13.520620Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:13.520633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:13.520639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:13.520673Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|63ddb9d5-4a2dd46d-5e2fbccb-aacdbf09_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:13.520682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:13.520684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:13.520685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:13.520694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:13.520701Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:13.520713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:13.520865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:13.520875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:13.520886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:13.520975Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1758 partition: 0 MaxSeqNo: 25 sessionId: src_id|63ddb9d5-4a2dd46d-5e2fbccb-aacdbf09_0
2025-10-09T21:50:13.521245Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|69bd3f99-19cf68d7-cee8ac8-32379a55_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|63ddb9d5-4a2dd46d-5e2fbccb-aacdbf09_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:13.523498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63ddb9d5-4a2dd46d-5e2fbccb-aacdbf09_0] Write session: try to update token
2025-10-09T21:50:13.523513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63ddb9d5-4a2dd46d-5e2fbccb-aacdbf09_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:13.534471Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:13.534491Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:13.534493Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:13.534496Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:13.534505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342643158503564:3157]
2025-10-09T21:50:13.534509Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342643158503564:3157]
2025-10-09T21:50:13.534513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342643158503564:3157]
2025-10-09T21:50:13.534550Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:13.534554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:13.534554Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:13.534557Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:13.534561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.534564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:13.534687Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:13.534979Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1758 sessionId: src_id|63ddb9d5-4a2dd46d-5e2fbccb-aacdbf09_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:13.537414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:13.602822Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342643520334504:6864]
2025-10-09T21:50:13.603142Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 1757, pipe: [1:7559342643520334504:6864]
2025-10-09T21:50:13.674468Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:13.674805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:13.674821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:13.674861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342643520334504:6864], now have 1 active actors on pipe
2025-10-09T21:50:13.675068Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:13.675053Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:13.675079Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342643520334504:6864]
2025-10-09T21:50:13.675103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:13.675106Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:13.675165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1758 reason: error at write: Disconnected sessionId: src_id|63ddb9d5-4a2dd46d-5e2fbccb-aacdbf09_0
2025-10-09T21:50:13.675393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1758 sessionId: src_id|63ddb9d5-4a2dd46d-5e2fbccb-aacdbf09_0 is DEAD
2025-10-09T21:50:13.675512Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63ddb9d5-4a2dd46d-5e2fbccb-aacdbf09_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:13.675523Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63ddb9d5-4a2dd46d-5e2fbccb-aacdbf09_0] Write session will restart in 0.008883s
2025-10-09T21:50:13.675542Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63ddb9d5-4a2dd46d-5e2fbccb-aacdbf09_0] Write session: Do CDS request
2025-10-09T21:50:13.675547Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63ddb9d5-4a2dd46d-5e2fbccb-aacdbf09_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:13.682905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:13.684468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:13.684643Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:13.684652Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:13.684662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:13.684723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:13.684761Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342643158503656:3150]
2025-10-09T21:50:13.685210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:13.685505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63ddb9d5-4a2dd46d-5e2fbccb-aacdbf09_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:13.685539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:13.685579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:13.685609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:13.685621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1759
2025-10-09T21:50:13.685751Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1759 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:13.685778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1759 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:13.685787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1759 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:13.685789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:13.685867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:13.685913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.685918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.685920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.685921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.685922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.685923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.685925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.685926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.685927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.685931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.685942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.685946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.685949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.685953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.685956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.685960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.685963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.685966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.685973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.685980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.685986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.685992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.685996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.686001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.686004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.686009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.686014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.686019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.686023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.686044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.686055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.686060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.686064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.686068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.686076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.686081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.686110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:13.686172Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1759 sessionId:  describe result for acl check
2025-10-09T21:50:13.686207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:13.686213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:13.686216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:13.686221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342643520334559:6376] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:13.686732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342643520334559:6376] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:13.688600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342643520334559:6376] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:13.691436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342643520334559:6376] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:13.691449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342643520334559:6376] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:13.691453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342643520334559:6376] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:13.691464Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1759 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:13.691915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:13.692945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:13.692961Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:13.692963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:13.692967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:13.692974Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1758 [2:7559342643158503656:3150]
2025-10-09T21:50:13.692985Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:13.692999Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:13.693006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:13.693010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:13.693012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:13.693027Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:13.693034Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:13.693041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:13.693090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.693096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.693098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.693101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:13.693161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:13.693167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.693178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.693189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.693211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:13.693236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342643520334596:6376], now have 1 active actors on pipe
2025-10-09T21:50:13.693240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342643520334506:2432], now have 1 active actors on pipe
2025-10-09T21:50:13.693247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342643158503651:3150]
2025-10-09T21:50:13.693248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342643158503651:3150]
2025-10-09T21:50:13.693252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342643158503651:3150]
2025-10-09T21:50:13.693265Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:13.693243Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1758
2025-10-09T21:50:13.693269Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1758
2025-10-09T21:50:13.693274Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:13.693277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:13.693282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:13.693284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:13.693289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:13.693292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:13.693325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:13.693334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:13.693351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342643158503651:3150]
2025-10-09T21:50:13.693363Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:13.693379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:13.693393Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|16f62660-78c20597-19c03701-5d42d531_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:13.693404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:13.693408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:13.693415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:13.693425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:13.693439Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:13.693466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:13.693629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:13.693640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:13.693654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:13.693725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1759 partition: 0 MaxSeqNo: 25 sessionId: src_id|16f62660-78c20597-19c03701-5d42d531_0
2025-10-09T21:50:13.693956Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63ddb9d5-4a2dd46d-5e2fbccb-aacdbf09_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|16f62660-78c20597-19c03701-5d42d531_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:13.696132Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16f62660-78c20597-19c03701-5d42d531_0] Write session: try to update token
2025-10-09T21:50:13.696146Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16f62660-78c20597-19c03701-5d42d531_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:13.706795Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:13.706812Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:13.706814Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:13.706821Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:13.706830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342643158503651:3150]
2025-10-09T21:50:13.706834Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342643158503651:3150]
2025-10-09T21:50:13.706838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342643158503651:3150]
2025-10-09T21:50:13.706882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:13.706885Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:13.706888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.706890Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:13.706891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:13.706894Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:13.707067Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:13.707929Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1759 sessionId: src_id|16f62660-78c20597-19c03701-5d42d531_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:13.710246Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:13.775332Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342643520334626:6864]
2025-10-09T21:50:13.775561Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 1758, pipe: [1:7559342643520334626:6864]
2025-10-09T21:50:13.848059Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:13.848319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:13.848331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:13.848376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342643520334626:6864], now have 1 active actors on pipe
2025-10-09T21:50:13.848598Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:13.848624Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:13.848643Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342643520334626:6864]
2025-10-09T21:50:13.848668Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:13.848741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1759 reason: status is not ok: tablet will be restarted right now sessionId: src_id|16f62660-78c20597-19c03701-5d42d531_0
2025-10-09T21:50:13.848920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1759 sessionId: src_id|16f62660-78c20597-19c03701-5d42d531_0 is DEAD
2025-10-09T21:50:13.849077Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16f62660-78c20597-19c03701-5d42d531_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:13.849087Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16f62660-78c20597-19c03701-5d42d531_0] Write session will restart in 0.009377s
2025-10-09T21:50:13.849105Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16f62660-78c20597-19c03701-5d42d531_0] Write session: Do CDS request
2025-10-09T21:50:13.849110Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16f62660-78c20597-19c03701-5d42d531_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:13.855862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:13.857266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:13.857468Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:13.857477Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:13.857489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:13.857550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:13.857589Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342643158503786:3123]
2025-10-09T21:50:13.858152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:13.858409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:13.858449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:13.858741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:13.858784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.858790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.858791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.858792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.858793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.858795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.858796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.858797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.858798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.858802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.858812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.858816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.858820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.858823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.858827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.858831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.858834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.858838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.858845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.858853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:13.858858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.858863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:13.858867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.858872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:13.858879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.858884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:13.858888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.858893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:13.858897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.858906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:13.858912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.858921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.858924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.858928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.858934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.858943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.858970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:13.859444Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16f62660-78c20597-19c03701-5d42d531_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:13.859566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:13.859579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1760
2025-10-09T21:50:13.859680Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1760 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:13.859707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1760 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:13.859717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1760 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:13.859725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:13.860055Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1760 sessionId:  describe result for acl check
2025-10-09T21:50:13.860086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:13.860087Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:13.860088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:13.860092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342643520334683:6361] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:13.860474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342643520334683:6361] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:13.862312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342643520334683:6361] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:13.864553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:13.864576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342643520334683:6361] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:13.864590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342643520334683:6361] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:13.864594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342643520334683:6361] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:13.864601Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1760 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:13.865610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:13.865627Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:13.865629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:13.865632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:13.865638Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1759 [2:7559342643158503786:3123]
2025-10-09T21:50:13.865648Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:13.865663Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:13.865668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:13.865671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:13.865672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:13.865683Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:13.865690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:13.865697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:13.865735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.865739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.865742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.865744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:13.865796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:13.865803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.865813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:13.865827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:13.865848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:13.865870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342643520334627:2432], now have 1 active actors on pipe
2025-10-09T21:50:13.865878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342643520334720:6361], now have 1 active actors on pipe
2025-10-09T21:50:13.865885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342643158503781:3123]
2025-10-09T21:50:13.865892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342643158503781:3123]
2025-10-09T21:50:13.865894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342643158503781:3123]
2025-10-09T21:50:13.865880Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1759
2025-10-09T21:50:13.865902Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:13.865887Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1759
2025-10-09T21:50:13.865917Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:13.865918Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:13.865926Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:13.865928Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:13.865933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:13.865936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:13.865968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:13.865976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:13.865985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342643158503781:3123]
2025-10-09T21:50:13.865992Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:13.866004Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:13.866013Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d98bf134-a789dba0-4cd09de5-356d9037_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:13.866018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:13.866022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:13.866023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:13.866044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:13.866051Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:13.866099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:13.866225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:13.866235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:13.866250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:13.866359Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1760 partition: 0 MaxSeqNo: 25 sessionId: src_id|d98bf134-a789dba0-4cd09de5-356d9037_0
2025-10-09T21:50:13.866623Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16f62660-78c20597-19c03701-5d42d531_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d98bf134-a789dba0-4cd09de5-356d9037_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:13.868830Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d98bf134-a789dba0-4cd09de5-356d9037_0] Write session: try to update token
2025-10-09T21:50:13.868846Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d98bf134-a789dba0-4cd09de5-356d9037_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:13.879169Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:13.879191Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:13.879193Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:13.879198Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:13.879209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342643158503781:3123]
2025-10-09T21:50:13.879212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342643158503781:3123]
2025-10-09T21:50:13.879216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342643158503781:3123]
2025-10-09T21:50:13.879259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:13.879265Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:13.879270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:13.879272Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:13.879272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:13.879275Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:13.879404Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:13.880569Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1760 sessionId: src_id|d98bf134-a789dba0-4cd09de5-356d9037_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:13.882970Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:13.948921Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342643520334728:6864]
2025-10-09T21:50:13.949144Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 1759, pipe: [1:7559342643520334728:6864]
2025-10-09T21:50:14.015029Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:14.015368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.015384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.015432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342643520334728:6864], now have 1 active actors on pipe
2025-10-09T21:50:14.015644Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:14.015693Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342643520334728:6864]
2025-10-09T21:50:14.015716Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:14.015720Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:14.015726Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:14.015781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1760 reason: error at write: Disconnected sessionId: src_id|d98bf134-a789dba0-4cd09de5-356d9037_0
2025-10-09T21:50:14.015961Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1760 sessionId: src_id|d98bf134-a789dba0-4cd09de5-356d9037_0 is DEAD
2025-10-09T21:50:14.016117Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d98bf134-a789dba0-4cd09de5-356d9037_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:14.016127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d98bf134-a789dba0-4cd09de5-356d9037_0] Write session will restart in 0.009773s
2025-10-09T21:50:14.016155Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d98bf134-a789dba0-4cd09de5-356d9037_0] Write session: Do CDS request
2025-10-09T21:50:14.016160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d98bf134-a789dba0-4cd09de5-356d9037_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:14.023189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:14.024721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:14.024987Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:14.024999Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:14.025012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:14.025076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:14.025121Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342647453471171:3122]
2025-10-09T21:50:14.025725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:14.025993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:14.026048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:14.026300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:14.026334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.026338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.026339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.026340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.026340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.026341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.026342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.026343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.026343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.026347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.026354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.026356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.026358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.026360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.026363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.026365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.026367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.026369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.026374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.026380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.026384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.026387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.026390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.026393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.026395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.026399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.026404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.026407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.026413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.026416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.026419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.026424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.026427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.026435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.026438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.026441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.026458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:14.027475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d98bf134-a789dba0-4cd09de5-356d9037_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:14.027532Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:14.027551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1761
2025-10-09T21:50:14.027693Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1761 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:14.027724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1761 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:14.027736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1761 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:14.027740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:14.028116Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1761 sessionId:  describe result for acl check
2025-10-09T21:50:14.028150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:14.028156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:14.028158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:14.028164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342647815302083:6355] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:14.028721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342647815302083:6355] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:14.030564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342647815302083:6355] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:14.032574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:14.032914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342647815302083:6355] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:14.032927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342647815302083:6355] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:14.032932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342647815302083:6355] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:14.032944Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1761 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:14.033654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:14.033665Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:14.033667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:14.033670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:14.033675Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1760 [2:7559342647453471171:3122]
2025-10-09T21:50:14.033684Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:14.033698Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:14.033700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:14.033702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:14.033703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:14.033715Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:14.033720Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:14.033725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:14.033768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.033774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.033776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.033777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:14.033827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:14.033832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.033839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.033848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.033860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:14.033879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302025:2432], now have 1 active actors on pipe
2025-10-09T21:50:14.033886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302117:6355], now have 1 active actors on pipe
2025-10-09T21:50:14.033892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342647453471166:3122]
2025-10-09T21:50:14.033893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342647453471166:3122]
2025-10-09T21:50:14.033894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342647453471166:3122]
2025-10-09T21:50:14.033904Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:14.033914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:14.033922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:14.033928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:14.033932Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:14.033918Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1760
2025-10-09T21:50:14.033936Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:14.033933Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1760
2025-10-09T21:50:14.033943Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:14.033950Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342647453471166:3122]
2025-10-09T21:50:14.033951Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:14.033966Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:14.033980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.033983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.034005Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|218b00c8-1b88261-816708e6-6ffa67f6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:14.034015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:14.034017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:14.034019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:14.034026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:14.034053Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:14.034072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:14.034247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.034261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.034285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:14.034383Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1761 partition: 0 MaxSeqNo: 25 sessionId: src_id|218b00c8-1b88261-816708e6-6ffa67f6_0
2025-10-09T21:50:14.034672Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d98bf134-a789dba0-4cd09de5-356d9037_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|218b00c8-1b88261-816708e6-6ffa67f6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:14.036984Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|218b00c8-1b88261-816708e6-6ffa67f6_0] Write session: try to update token
2025-10-09T21:50:14.037000Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|218b00c8-1b88261-816708e6-6ffa67f6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:14.048011Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:14.048063Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:14.048065Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:14.048067Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:14.048075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342647453471166:3122]
2025-10-09T21:50:14.048078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342647453471166:3122]
2025-10-09T21:50:14.048082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342647453471166:3122]
2025-10-09T21:50:14.048121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:14.048126Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:14.048130Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:14.048132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.048133Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:14.048134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:14.048262Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:14.049238Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1761 sessionId: src_id|218b00c8-1b88261-816708e6-6ffa67f6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:14.051640Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:14.115838Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342647815302127:6864]
2025-10-09T21:50:14.116126Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 1760, pipe: [1:7559342647815302127:6864]
2025-10-09T21:50:14.195401Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:14.195695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.195707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.195746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302127:6864], now have 1 active actors on pipe
2025-10-09T21:50:14.195931Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:14.195981Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342647815302127:6864]
2025-10-09T21:50:14.195992Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:14.196012Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:14.196078Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1761 reason: status is not ok: tablet will be restarted right now sessionId: src_id|218b00c8-1b88261-816708e6-6ffa67f6_0
2025-10-09T21:50:14.196234Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1761 sessionId: src_id|218b00c8-1b88261-816708e6-6ffa67f6_0 is DEAD
2025-10-09T21:50:14.196410Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|218b00c8-1b88261-816708e6-6ffa67f6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:14.196423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|218b00c8-1b88261-816708e6-6ffa67f6_0] Write session will restart in 0.008340s
2025-10-09T21:50:14.196452Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|218b00c8-1b88261-816708e6-6ffa67f6_0] Write session: Do CDS request
2025-10-09T21:50:14.196468Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|218b00c8-1b88261-816708e6-6ffa67f6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:14.204057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:14.205465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|218b00c8-1b88261-816708e6-6ffa67f6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:14.205466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:14.205500Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:14.205513Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1762
2025-10-09T21:50:14.205630Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:14.205636Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:14.205644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:14.205679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:14.205685Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1762 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:14.205708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1762 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:14.205713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1762 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:14.205716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:14.205708Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342647453471261:3120]
2025-10-09T21:50:14.206075Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1762 sessionId:  describe result for acl check
2025-10-09T21:50:14.206102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:14.206108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:14.206110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:14.206114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342647815302185:6352] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:14.206155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:14.206425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:14.206472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:14.206523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342647815302185:6352] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:14.206728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:14.206780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.206785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.206787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.206788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.206789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.206789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.206790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.206790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.206791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.206799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.206806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.206808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.206810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.206812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.206814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.206816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.206818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.206820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.206826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.206831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.206836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.206839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.206842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.206845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.206847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.206851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.206853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.206856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.206858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.206862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.206870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.206875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.206883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.206889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.206897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.206902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.206929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:14.208206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342647815302185:6352] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:14.212417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342647815302185:6352] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:14.212430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342647815302185:6352] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:14.212434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342647815302185:6352] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:14.212442Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1762 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:14.212843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:14.213896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:14.213907Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:14.213909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:14.213912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:14.213916Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1761 [2:7559342647453471261:3120]
2025-10-09T21:50:14.213924Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:14.213937Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:14.213938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:14.213941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:14.213942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:14.213953Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:14.213957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:14.213962Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:14.214009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.214016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.214018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.214021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:14.214126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:14.214134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.214145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.214160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.214184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:14.214211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302131:2432], now have 1 active actors on pipe
2025-10-09T21:50:14.214220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302224:6352], now have 1 active actors on pipe
2025-10-09T21:50:14.214227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342647453471256:3120]
2025-10-09T21:50:14.214229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342647453471256:3120]
2025-10-09T21:50:14.214231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342647453471256:3120]
2025-10-09T21:50:14.214216Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1761
2025-10-09T21:50:14.214241Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:14.214237Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1761
2025-10-09T21:50:14.214255Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:14.214270Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:14.214270Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:14.214273Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:14.214275Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:14.214278Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:14.214308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342647453471256:3120]
2025-10-09T21:50:14.214322Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:14.214340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.214347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.214347Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:14.214390Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cca0d292-992c576d-984352f8-5b3ce23a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:14.214403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:14.214408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:14.214411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:14.214421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:14.214437Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:14.214451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:14.214622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.214634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.214664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:14.214788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1762 partition: 0 MaxSeqNo: 25 sessionId: src_id|cca0d292-992c576d-984352f8-5b3ce23a_0
2025-10-09T21:50:14.215039Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|218b00c8-1b88261-816708e6-6ffa67f6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cca0d292-992c576d-984352f8-5b3ce23a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:14.217507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cca0d292-992c576d-984352f8-5b3ce23a_0] Write session: try to update token
2025-10-09T21:50:14.217523Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cca0d292-992c576d-984352f8-5b3ce23a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:14.228161Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:14.228180Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:14.228182Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:14.228184Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:14.228195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342647453471256:3120]
2025-10-09T21:50:14.228199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342647453471256:3120]
2025-10-09T21:50:14.228202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342647453471256:3120]
2025-10-09T21:50:14.228247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:14.228253Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:14.228258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.228260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:14.228262Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:14.228266Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:14.228385Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:14.229488Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1762 sessionId: src_id|cca0d292-992c576d-984352f8-5b3ce23a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:14.231818Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:14.296027Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342647815302237:6864]
2025-10-09T21:50:14.296268Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 1761, pipe: [1:7559342647815302237:6864]
2025-10-09T21:50:14.373751Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:14.373992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.374004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.374059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302237:6864], now have 1 active actors on pipe
2025-10-09T21:50:14.374265Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:14.374340Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342647815302237:6864]
2025-10-09T21:50:14.374348Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:14.374380Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:14.374474Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1762 reason: status is not ok: tablet will be restarted right now sessionId: src_id|cca0d292-992c576d-984352f8-5b3ce23a_0
2025-10-09T21:50:14.374653Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1762 sessionId: src_id|cca0d292-992c576d-984352f8-5b3ce23a_0 is DEAD
2025-10-09T21:50:14.374798Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cca0d292-992c576d-984352f8-5b3ce23a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:14.374808Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cca0d292-992c576d-984352f8-5b3ce23a_0] Write session will restart in 0.009968s
2025-10-09T21:50:14.374826Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cca0d292-992c576d-984352f8-5b3ce23a_0] Write session: Do CDS request
2025-10-09T21:50:14.374830Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cca0d292-992c576d-984352f8-5b3ce23a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:14.382248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:14.383910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:14.384049Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:14.384058Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:14.384066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:14.384112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:14.384140Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342647453471352:3114]
2025-10-09T21:50:14.384524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:14.384796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:14.384830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:14.385063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:14.385105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.385111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.385112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.385113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.385114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.385114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.385115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.385116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.385116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.385120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.385127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.385129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.385131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.385133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.385135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.385138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.385140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.385142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.385147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.385152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.385158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.385163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.385167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.385173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.385177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.385187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.385191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.385201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.385203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.385206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.385209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.385213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.385216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.385219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.385222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.385225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.385241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:14.385452Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cca0d292-992c576d-984352f8-5b3ce23a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:14.385807Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:14.385822Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1763
2025-10-09T21:50:14.385977Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1763 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:14.386011Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1763 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:14.386014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1763 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:14.386017Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:14.386375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1763 sessionId:  describe result for acl check
2025-10-09T21:50:14.386405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:14.386411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:14.386412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:14.386416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342647815302298:6342] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:14.386908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342647815302298:6342] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:14.388940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342647815302298:6342] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:14.391342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342647815302298:6342] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:14.391357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342647815302298:6342] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:14.391360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342647815302298:6342] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:14.391367Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1763 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:14.391609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:14.392637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:14.392648Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:14.392650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:14.392652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:14.392657Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1762 [2:7559342647453471352:3114]
2025-10-09T21:50:14.392665Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:14.392677Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:14.392679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:14.392682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:14.392684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:14.392696Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:14.392699Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:14.392704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:14.392757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.392763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.392764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.392766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:14.392814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:14.392819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.392825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.392832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.392841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:14.392861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302332:6342], now have 1 active actors on pipe
2025-10-09T21:50:14.392868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302238:2432], now have 1 active actors on pipe
2025-10-09T21:50:14.392875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342647453471347:3114]
2025-10-09T21:50:14.392882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342647453471347:3114]
2025-10-09T21:50:14.392884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342647453471347:3114]
2025-10-09T21:50:14.392898Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:14.392917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:14.392922Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1762
2025-10-09T21:50:14.392918Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:14.392922Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:14.392929Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1762
2025-10-09T21:50:14.392924Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:14.392937Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:14.392940Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:14.392967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342647453471347:3114]
2025-10-09T21:50:14.392976Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:14.392981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:14.392995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.393003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.393046Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b3d84aa8-674f1d93-37303973-8a399ec5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:14.393058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:14.393063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:14.393065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:14.393072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:14.393093Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:14.393110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:14.393228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.393234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.393247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:14.393341Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1763 partition: 0 MaxSeqNo: 25 sessionId: src_id|b3d84aa8-674f1d93-37303973-8a399ec5_0
2025-10-09T21:50:14.393617Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cca0d292-992c576d-984352f8-5b3ce23a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b3d84aa8-674f1d93-37303973-8a399ec5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:14.395993Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3d84aa8-674f1d93-37303973-8a399ec5_0] Write session: try to update token
2025-10-09T21:50:14.396009Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3d84aa8-674f1d93-37303973-8a399ec5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:14.407111Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:14.407126Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:14.407128Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:14.407130Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:14.407140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342647453471347:3114]
2025-10-09T21:50:14.407148Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342647453471347:3114]
2025-10-09T21:50:14.407153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342647453471347:3114]
2025-10-09T21:50:14.407197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:14.407200Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:14.407206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.407206Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:14.407209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:14.407210Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:14.407361Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:14.408922Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1763 sessionId: src_id|b3d84aa8-674f1d93-37303973-8a399ec5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:14.411290Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:14.474718Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342647815302340:6864]
2025-10-09T21:50:14.474982Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 1762, pipe: [1:7559342647815302340:6864]
2025-10-09T21:50:14.552392Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:14.552692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.552704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.552743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302340:6864], now have 1 active actors on pipe
2025-10-09T21:50:14.552998Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:14.553018Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:14.553022Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:14.553061Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342647815302340:6864]
2025-10-09T21:50:14.553079Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:14.553086Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1763 reason: error at write: Disconnected sessionId: src_id|b3d84aa8-674f1d93-37303973-8a399ec5_0
2025-10-09T21:50:14.553270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1763 sessionId: src_id|b3d84aa8-674f1d93-37303973-8a399ec5_0 is DEAD
2025-10-09T21:50:14.553425Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3d84aa8-674f1d93-37303973-8a399ec5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:14.553438Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3d84aa8-674f1d93-37303973-8a399ec5_0] Write session will restart in 0.006881s
2025-10-09T21:50:14.553477Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3d84aa8-674f1d93-37303973-8a399ec5_0] Write session: Do CDS request
2025-10-09T21:50:14.553484Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3d84aa8-674f1d93-37303973-8a399ec5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:14.561209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:14.561510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3d84aa8-674f1d93-37303973-8a399ec5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:14.561580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:14.561597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1764
2025-10-09T21:50:14.561721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1764 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:14.561752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1764 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:14.561762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1764 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:14.561766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:14.562206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1764 sessionId:  describe result for acl check
2025-10-09T21:50:14.562249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:14.562256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:14.562258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:14.562264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342647815302397:6333] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:14.562828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342647815302397:6333] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:14.562881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:14.563044Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:14.563053Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:14.563065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:14.563113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:14.563151Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342647453471444:3118]
2025-10-09T21:50:14.563717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:14.564000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:14.564053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:14.564272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:14.564318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.564323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.564324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.564325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.564325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.564326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.564327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.564327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.564328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.564332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.564338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.564340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.564342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.564344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.564346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.564348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.564350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.564352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.564357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.564362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.564366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.564370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.564372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.564375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.564377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.564381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.564383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.564387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.564390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.564393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.564396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.564400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.564402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.564405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.564407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.564410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.564427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:14.564554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342647815302397:6333] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:14.570404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342647815302397:6333] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:14.570420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342647815302397:6333] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:14.570424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342647815302397:6333] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:14.570436Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1764 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:14.570845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:14.571968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:14.571986Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:14.571989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:14.571993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:14.571999Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1763 [2:7559342647453471444:3118]
2025-10-09T21:50:14.572012Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:14.572030Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:14.572033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:14.572036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:14.572039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:14.572057Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:14.572069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:14.572076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:14.572136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.572143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.572145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.572148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:14.572214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:14.572224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.572237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.572248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.572266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:14.572287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302436:6333], now have 1 active actors on pipe
2025-10-09T21:50:14.572299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302341:2432], now have 1 active actors on pipe
2025-10-09T21:50:14.572307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342647453471439:3118]
2025-10-09T21:50:14.572311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342647453471439:3118]
2025-10-09T21:50:14.572313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342647453471439:3118]
2025-10-09T21:50:14.572325Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:14.572340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:14.572354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:14.572357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:14.572392Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342647453471439:3118]
2025-10-09T21:50:14.572400Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:14.572386Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1763
2025-10-09T21:50:14.572387Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1763
2025-10-09T21:50:14.572416Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:14.572425Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:14.572428Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:14.572435Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:14.572518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.572532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.572561Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c8003b93-50387154-8d03a927-abd783b8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:14.572572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:14.572580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:14.572582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:14.572591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:14.572601Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:14.572618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:14.572910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.572922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.572946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:14.573057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1764 partition: 0 MaxSeqNo: 25 sessionId: src_id|c8003b93-50387154-8d03a927-abd783b8_0
2025-10-09T21:50:14.573404Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3d84aa8-674f1d93-37303973-8a399ec5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c8003b93-50387154-8d03a927-abd783b8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:14.576220Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8003b93-50387154-8d03a927-abd783b8_0] Write session: try to update token
2025-10-09T21:50:14.576240Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8003b93-50387154-8d03a927-abd783b8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:14.584383Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:14.584399Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:14.584410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342647453471439:3118]
2025-10-09T21:50:14.584453Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:14.584458Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:14.584654Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:14.584802Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:14.584943Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:14.585094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:14.585287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:14.585333Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:14.589700Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1764 sessionId: src_id|c8003b93-50387154-8d03a927-abd783b8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:14.592148Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:14.592349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.592369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.592419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:14.592504Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:14.597688Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:14.597704Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:14.597706Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:14.597710Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:14.597725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342647453471439:3118]
2025-10-09T21:50:14.597729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342647453471439:3118]
2025-10-09T21:50:14.597733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342647453471439:3118]
2025-10-09T21:50:14.597778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:14.597782Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:14.597788Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:14.597789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.597791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:14.597792Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:14.597929Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:14.653414Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342647815302444:6864]
2025-10-09T21:50:14.653721Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 1763, pipe: [1:7559342647815302444:6864]
2025-10-09T21:50:14.740511Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:14.748801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.748832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.748979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:14.749047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:14.749110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:14.749185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:14.749410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:14.749423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:14.749504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:14.749571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:14.749635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:14.749701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:14.749745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:14.749753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:14.749814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:14.749873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:14.749938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:14.750001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:14.750065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:14.750073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:14.750144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:14.750214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:14.750322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:14.750386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:14.750432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:14.750440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:14.750509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:14.750575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:14.750637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:14.750732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:14.750792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:14.750799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:14.750859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:14.750919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:14.750984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:14.751048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:14.751093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:14.751101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:14.751162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:14.751224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:14.751289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:14.751350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:14.751396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:14.751404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:14.751463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:14.751524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:14.751585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:14.751650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:14.751696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:14.751702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:14.751765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:14.751823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:14.751879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:14.751939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:14.751981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:14.751988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:14.752047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:14.752102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:14.752161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:14.752226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:14.752367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:14.752377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:14.752427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302444:6864], now have 1 active actors on pipe
2025-10-09T21:50:14.752636Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:14.752652Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:14.752680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1764 reason: error at write: Disconnected sessionId: src_id|c8003b93-50387154-8d03a927-abd783b8_0
2025-10-09T21:50:14.752862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1764 sessionId: src_id|c8003b93-50387154-8d03a927-abd783b8_0 is DEAD
2025-10-09T21:50:14.753017Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8003b93-50387154-8d03a927-abd783b8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:14.753025Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8003b93-50387154-8d03a927-abd783b8_0] Write session will restart in 0.009534s
2025-10-09T21:50:14.753043Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8003b93-50387154-8d03a927-abd783b8_0] Write session: Do CDS request
2025-10-09T21:50:14.753048Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8003b93-50387154-8d03a927-abd783b8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:14.753049Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:14.753086Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342647815302444:6864]
2025-10-09T21:50:14.753214Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:14.761666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:14.763439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:14.763507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8003b93-50387154-8d03a927-abd783b8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:14.763603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:14.763641Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:14.763652Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:14.763665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:14.763621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1765
2025-10-09T21:50:14.763718Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1765 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:14.763747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1765 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:14.763724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:14.763756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1765 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:14.763769Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342647453471555:3102]
2025-10-09T21:50:14.763761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:14.764157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1765 sessionId:  describe result for acl check
2025-10-09T21:50:14.764194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:14.764202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:14.764203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:14.764209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342647815302516:6319] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:14.764381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:14.764679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:14.764724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:14.764771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342647815302516:6319] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:14.765051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:14.765115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.765122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.765124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.765125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.765126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.765128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.765129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.765131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.765132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.765137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.765147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.765152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.765157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.765161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.765166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.765170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.765175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.765179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.765185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.765195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.765207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.765213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.765222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.765228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.765237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.765242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.765247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.765252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.765257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.765263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.765270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.765281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.765286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.765296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.765301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.765311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.765342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:14.767438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342647815302516:6319] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:14.771818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342647815302516:6319] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:14.771833Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342647815302516:6319] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:14.771838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342647815302516:6319] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:14.771848Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1765 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:14.772294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:14.773454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:14.773475Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:14.773477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:14.773482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:14.773503Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1764 [2:7559342647453471555:3102]
2025-10-09T21:50:14.773519Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:14.773537Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:14.773548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:14.773551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:14.773554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:14.773572Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:14.773578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:14.773590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:14.773651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.773658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.773661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.773664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:14.773736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:14.773743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.773757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.773774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.773797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:14.773825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302554:6319], now have 1 active actors on pipe
2025-10-09T21:50:14.773834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302460:2432], now have 1 active actors on pipe
2025-10-09T21:50:14.773843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342647453471550:3102]
2025-10-09T21:50:14.773845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342647453471550:3102]
2025-10-09T21:50:14.773847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342647453471550:3102]
2025-10-09T21:50:14.773861Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:14.773879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:14.773884Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:14.773889Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:14.773865Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1764
2025-10-09T21:50:14.773891Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:14.773895Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1764
2025-10-09T21:50:14.773899Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:14.773902Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:14.773951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.773961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.773974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342647453471550:3102]
2025-10-09T21:50:14.773986Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:14.773992Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:14.774024Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|baa00e0-66e5882b-c873046b-4a2b6e06_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:14.774055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:14.774060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:14.774063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:14.774074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:14.774091Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:14.774117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:14.774284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.774294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.774310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:14.774397Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1765 partition: 0 MaxSeqNo: 25 sessionId: src_id|baa00e0-66e5882b-c873046b-4a2b6e06_0
2025-10-09T21:50:14.774770Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8003b93-50387154-8d03a927-abd783b8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|baa00e0-66e5882b-c873046b-4a2b6e06_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:14.777156Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|baa00e0-66e5882b-c873046b-4a2b6e06_0] Write session: try to update token
2025-10-09T21:50:14.777172Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|baa00e0-66e5882b-c873046b-4a2b6e06_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:14.788839Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:14.788856Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:14.788858Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:14.788859Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:14.788867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342647453471550:3102]
2025-10-09T21:50:14.788871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342647453471550:3102]
2025-10-09T21:50:14.788874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342647453471550:3102]
2025-10-09T21:50:14.788912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:14.788914Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:14.788917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.788919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:14.788920Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:14.788924Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:14.789065Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:14.790573Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1765 sessionId: src_id|baa00e0-66e5882b-c873046b-4a2b6e06_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:14.793148Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:14.853060Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342647815302575:6864]
2025-10-09T21:50:14.853266Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 1764, pipe: [1:7559342647815302575:6864]
2025-10-09T21:50:14.935390Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:14.935752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.935764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.935803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302575:6864], now have 1 active actors on pipe
2025-10-09T21:50:14.935969Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:14.936040Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:14.936056Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:14.936118Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1765 reason: error at write: Disconnected sessionId: src_id|baa00e0-66e5882b-c873046b-4a2b6e06_0
2025-10-09T21:50:14.936197Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342647815302575:6864]
2025-10-09T21:50:14.936222Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:14.936303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1765 sessionId: src_id|baa00e0-66e5882b-c873046b-4a2b6e06_0 is DEAD
2025-10-09T21:50:14.936413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baa00e0-66e5882b-c873046b-4a2b6e06_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:14.936423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baa00e0-66e5882b-c873046b-4a2b6e06_0] Write session will restart in 0.008718s
2025-10-09T21:50:14.936443Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baa00e0-66e5882b-c873046b-4a2b6e06_0] Write session: Do CDS request
2025-10-09T21:50:14.936456Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baa00e0-66e5882b-c873046b-4a2b6e06_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:14.943260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:14.944651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:14.944811Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:14.944820Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:14.944832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:14.944880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:14.944917Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342647453471674:3096]
2025-10-09T21:50:14.945448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:14.945693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:14.945731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:14.946019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:14.946080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.946085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.946086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.946087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.946088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.946089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.946091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.946092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.946093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.946098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.946108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.946112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.946117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.946121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.946125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.946129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.946132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.946136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.946144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.946151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:14.946157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.946164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:14.946169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.946174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:14.946179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.946184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:14.946193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.946198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:14.946203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.946208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:14.946213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.946222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.946225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.946230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.946235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.946244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.946273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:14.946419Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|baa00e0-66e5882b-c873046b-4a2b6e06_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:14.946516Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:14.946528Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1766
2025-10-09T21:50:14.946629Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1766 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:14.946657Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1766 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:14.946668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1766 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:14.946671Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:14.946975Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1766 sessionId:  describe result for acl check
2025-10-09T21:50:14.946999Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:14.947000Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:14.947001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:14.947005Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342647815302644:6314] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:14.947555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342647815302644:6314] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:14.949585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342647815302644:6314] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:14.952276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342647815302644:6314] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:14.952290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342647815302644:6314] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:14.952293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342647815302644:6314] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:14.952306Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1766 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:14.952774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:14.953935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:14.953948Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:14.953951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:14.953956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:14.953962Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1765 [2:7559342647453471674:3096]
2025-10-09T21:50:14.953975Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:14.953992Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:14.953995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:14.953998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:14.954000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:14.954016Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:14.954044Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:14.954051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:14.954100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.954107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.954108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.954111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:14.954165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:14.954173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.954180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:14.954187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:14.954196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:14.954212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302679:6314], now have 1 active actors on pipe
2025-10-09T21:50:14.954217Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342647453471669:3096]
2025-10-09T21:50:14.954219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342647453471669:3096]
2025-10-09T21:50:14.954220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342647453471669:3096]
2025-10-09T21:50:14.954227Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:14.954233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342647815302582:2432], now have 1 active actors on pipe
2025-10-09T21:50:14.954240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:14.954248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:14.954248Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:14.954250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:14.954251Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:14.954253Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:14.954273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342647453471669:3096]
2025-10-09T21:50:14.954277Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:14.954283Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:14.954261Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1765
2025-10-09T21:50:14.954265Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1765
2025-10-09T21:50:14.954304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.954314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.954341Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6d8dbd2c-4c88b0e8-ba3089ee-f0038a7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:14.954351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:14.954354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:14.954355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:14.954363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:14.954371Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:14.954386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:14.954540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:14.954547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:14.954559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:14.954646Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1766 partition: 0 MaxSeqNo: 25 sessionId: src_id|6d8dbd2c-4c88b0e8-ba3089ee-f0038a7_0
2025-10-09T21:50:14.954939Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baa00e0-66e5882b-c873046b-4a2b6e06_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6d8dbd2c-4c88b0e8-ba3089ee-f0038a7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:14.957202Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6d8dbd2c-4c88b0e8-ba3089ee-f0038a7_0] Write session: try to update token
2025-10-09T21:50:14.957217Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6d8dbd2c-4c88b0e8-ba3089ee-f0038a7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:14.968428Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:14.968444Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:14.968447Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:14.968449Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:14.968459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342647453471669:3096]
2025-10-09T21:50:14.968464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342647453471669:3096]
2025-10-09T21:50:14.968472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342647453471669:3096]
2025-10-09T21:50:14.968517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:14.968523Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:14.968529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:14.968531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:14.968533Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:14.968537Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:14.968687Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:14.968931Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1766 sessionId: src_id|6d8dbd2c-4c88b0e8-ba3089ee-f0038a7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:14.971244Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:15.036418Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342652110269982:6864]
2025-10-09T21:50:15.036721Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 1765, pipe: [1:7559342652110269982:6864]
2025-10-09T21:50:15.121833Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:15.122372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:15.122392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:15.122440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342652110269982:6864], now have 1 active actors on pipe
2025-10-09T21:50:15.122760Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:15.122808Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:15.122827Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:15.122834Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:15.122882Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342652110269982:6864]
2025-10-09T21:50:15.122962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1766 reason: error at write: Disconnected sessionId: src_id|6d8dbd2c-4c88b0e8-ba3089ee-f0038a7_0
2025-10-09T21:50:15.123159Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1766 sessionId: src_id|6d8dbd2c-4c88b0e8-ba3089ee-f0038a7_0 is DEAD
2025-10-09T21:50:15.123358Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d8dbd2c-4c88b0e8-ba3089ee-f0038a7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:15.123370Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d8dbd2c-4c88b0e8-ba3089ee-f0038a7_0] Write session will restart in 0.007029s
2025-10-09T21:50:15.123406Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d8dbd2c-4c88b0e8-ba3089ee-f0038a7_0] Write session: Do CDS request
2025-10-09T21:50:15.123414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d8dbd2c-4c88b0e8-ba3089ee-f0038a7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:15.131208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:15.131517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6d8dbd2c-4c88b0e8-ba3089ee-f0038a7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:15.131626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:15.131650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1767
2025-10-09T21:50:15.131783Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1767 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:15.131818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1767 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:15.131828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1767 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:15.131831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:15.132186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1767 sessionId:  describe result for acl check
2025-10-09T21:50:15.132216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:15.132221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:15.132222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:15.132226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342652110270043:6308] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:15.132675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342652110270043:6308] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:15.132722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:15.132906Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:15.132916Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:15.132928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:15.132980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:15.133034Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342651748439065:3082]
2025-10-09T21:50:15.133632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:15.133879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:15.133924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:15.134257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:15.134310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.134316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.134317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.134318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.134319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.134320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.134321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.134323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.134324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.134329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.134339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.134343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.134347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.134351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.134356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.134360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.134364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.134368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.134378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.134386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.134395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.134404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.134409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.134419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.134427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.134433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.134439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.134442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.134445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.134448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.134456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.134463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.134467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.134472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.134477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.134483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.134509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:15.134529Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342652110270043:6308] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:15.140654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342652110270043:6308] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:15.140670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342652110270043:6308] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:15.140674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342652110270043:6308] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:15.140684Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1767 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:15.141075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:15.142328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:15.142342Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:15.142345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:15.142349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:15.142356Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1766 [2:7559342651748439065:3082]
2025-10-09T21:50:15.142369Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:15.142386Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:15.142394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:15.142397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:15.142400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:15.142418Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:15.142425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:15.142432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:15.142491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.142502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.142504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.142508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:15.142577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:15.142589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.142601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.142614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.142630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:15.142652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342652110269985:2432], now have 1 active actors on pipe
2025-10-09T21:50:15.142662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342652110270082:6308], now have 1 active actors on pipe
2025-10-09T21:50:15.142670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342651748439060:3082]
2025-10-09T21:50:15.142672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342651748439060:3082]
2025-10-09T21:50:15.142675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342651748439060:3082]
2025-10-09T21:50:15.142693Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:15.142709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:15.142711Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1766
2025-10-09T21:50:15.142723Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:15.142720Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1766
2025-10-09T21:50:15.142727Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:15.142730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:15.142730Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:15.142733Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:15.142765Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342651748439060:3082]
2025-10-09T21:50:15.142769Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:15.142771Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:15.142794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:15.142799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:15.142850Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2b4753d9-49cd7c80-858232e6-541bf78_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:15.142862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:15.142866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:15.142869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:15.142878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:15.142887Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:15.142909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:15.143074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:15.143083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:15.143098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:15.143232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1767 partition: 0 MaxSeqNo: 25 sessionId: src_id|2b4753d9-49cd7c80-858232e6-541bf78_0
2025-10-09T21:50:15.143569Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6d8dbd2c-4c88b0e8-ba3089ee-f0038a7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2b4753d9-49cd7c80-858232e6-541bf78_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:15.150194Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b4753d9-49cd7c80-858232e6-541bf78_0] Write session: try to update token
2025-10-09T21:50:15.150213Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b4753d9-49cd7c80-858232e6-541bf78_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:15.157424Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:15.157440Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:15.157443Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:15.157445Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:15.157454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342651748439060:3082]
2025-10-09T21:50:15.157461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342651748439060:3082]
2025-10-09T21:50:15.157466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342651748439060:3082]
2025-10-09T21:50:15.157530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:15.157542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.157549Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:15.157551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:15.157557Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:15.157563Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:15.157875Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:15.164470Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1767 sessionId: src_id|2b4753d9-49cd7c80-858232e6-541bf78_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:15.167307Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:15.223321Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342652110270092:6864]
2025-10-09T21:50:15.223606Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 1766, pipe: [1:7559342652110270092:6864]
2025-10-09T21:50:15.307537Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:15.307830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:15.307843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:15.307864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342652110270092:6864], now have 1 active actors on pipe
2025-10-09T21:50:15.308129Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:15.308133Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342652110270092:6864]
2025-10-09T21:50:15.308160Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:15.308236Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1767 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2b4753d9-49cd7c80-858232e6-541bf78_0
2025-10-09T21:50:15.308420Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1767 sessionId: src_id|2b4753d9-49cd7c80-858232e6-541bf78_0 is DEAD
2025-10-09T21:50:15.308579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b4753d9-49cd7c80-858232e6-541bf78_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:15.308590Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b4753d9-49cd7c80-858232e6-541bf78_0] Write session will restart in 0.007120s
2025-10-09T21:50:15.308057Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:15.308614Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b4753d9-49cd7c80-858232e6-541bf78_0] Write session: Do CDS request
2025-10-09T21:50:15.308622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b4753d9-49cd7c80-858232e6-541bf78_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:15.316440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:15.316539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b4753d9-49cd7c80-858232e6-541bf78_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:15.316580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:15.316594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1768
2025-10-09T21:50:15.316748Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1768 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:15.316773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1768 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:15.316777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1768 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:15.316780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:15.317225Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1768 sessionId:  describe result for acl check
2025-10-09T21:50:15.317271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:15.317275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:15.317277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:15.317283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342652110270159:6304] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:15.317744Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342652110270159:6304] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:15.317993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:15.318178Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:15.318188Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:15.318200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:15.318248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:15.318287Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342651748439161:3071]
2025-10-09T21:50:15.318783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:15.319057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:15.319098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:15.319323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:15.319365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.319371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.319371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.319372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.319372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.319373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.319374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.319374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.319375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.319379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.319387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.319390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.319394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.319397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.319402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.319406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.319410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.319414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.319422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.319427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.319431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.319434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.319437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.319440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.319442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.319445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.319449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.319452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.319454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.319457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.319460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.319463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.319465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.319468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.319470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.319473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.319489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:15.319744Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342652110270159:6304] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:15.325404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342652110270159:6304] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:15.325417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342652110270159:6304] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:15.325420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342652110270159:6304] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:15.325428Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1768 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:15.325894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:15.327148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:15.327162Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:15.327165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:15.327169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:15.327176Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1767 [2:7559342651748439161:3071]
2025-10-09T21:50:15.327193Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:15.327216Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:15.327224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:15.327228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:15.327234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:15.327253Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:15.327264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:15.327274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:15.327337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.327346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.327348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.327350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:15.327418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:15.327426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.327438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.327450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.327468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:15.327493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342652110270198:6304], now have 1 active actors on pipe
2025-10-09T21:50:15.327504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342652110270100:2432], now have 1 active actors on pipe
2025-10-09T21:50:15.327511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342651748439156:3071]
2025-10-09T21:50:15.327513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342651748439156:3071]
2025-10-09T21:50:15.327515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342651748439156:3071]
2025-10-09T21:50:15.327528Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:15.327541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:15.327541Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:15.327545Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:15.327547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:15.327527Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1767
2025-10-09T21:50:15.327544Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1767
2025-10-09T21:50:15.327565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:15.327573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:15.327620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:15.327628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:15.327643Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342651748439156:3071]
2025-10-09T21:50:15.327656Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:15.327662Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:15.327684Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|83d75b78-f01ffa75-31d6d6a3-ee40eacf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:15.327695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:15.327699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:15.327701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:15.327710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:15.327720Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:15.327740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:15.327901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:15.327908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:15.327922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:15.327994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1768 partition: 0 MaxSeqNo: 25 sessionId: src_id|83d75b78-f01ffa75-31d6d6a3-ee40eacf_0
2025-10-09T21:50:15.328346Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b4753d9-49cd7c80-858232e6-541bf78_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|83d75b78-f01ffa75-31d6d6a3-ee40eacf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:15.330635Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|83d75b78-f01ffa75-31d6d6a3-ee40eacf_0] Write session: try to update token
2025-10-09T21:50:15.330649Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|83d75b78-f01ffa75-31d6d6a3-ee40eacf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:15.342361Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:15.342382Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:15.342385Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:15.342387Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:15.342397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342651748439156:3071]
2025-10-09T21:50:15.342403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342651748439156:3071]
2025-10-09T21:50:15.342407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342651748439156:3071]
2025-10-09T21:50:15.342451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:15.342453Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:15.342460Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:15.342462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.342464Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:15.342465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:15.342601Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:15.343777Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1768 sessionId: src_id|83d75b78-f01ffa75-31d6d6a3-ee40eacf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:15.346238Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:15.408137Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342652110270205:6864]
2025-10-09T21:50:15.408383Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 1767, pipe: [1:7559342652110270205:6864]
2025-10-09T21:50:15.488926Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:15.489176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:15.489190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:15.489235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342652110270205:6864], now have 1 active actors on pipe
2025-10-09T21:50:15.489401Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:15.489433Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:15.489441Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342652110270205:6864]
2025-10-09T21:50:15.489448Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:15.489460Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:15.489536Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1768 reason: error at write: Disconnected sessionId: src_id|83d75b78-f01ffa75-31d6d6a3-ee40eacf_0
2025-10-09T21:50:15.489725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1768 sessionId: src_id|83d75b78-f01ffa75-31d6d6a3-ee40eacf_0 is DEAD
2025-10-09T21:50:15.489885Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83d75b78-f01ffa75-31d6d6a3-ee40eacf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:15.489895Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83d75b78-f01ffa75-31d6d6a3-ee40eacf_0] Write session will restart in 0.006542s
2025-10-09T21:50:15.489913Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83d75b78-f01ffa75-31d6d6a3-ee40eacf_0] Write session: Do CDS request
2025-10-09T21:50:15.489918Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83d75b78-f01ffa75-31d6d6a3-ee40eacf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:15.497434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|83d75b78-f01ffa75-31d6d6a3-ee40eacf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:15.497504Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:15.497523Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1769
2025-10-09T21:50:15.497651Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1769 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:15.497684Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1769 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:15.497695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1769 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:15.497707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:15.497793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:15.498150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1769 sessionId:  describe result for acl check
2025-10-09T21:50:15.498188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:15.498195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:15.498199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:15.498205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342652110270268:6293] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:15.498762Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342652110270268:6293] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:15.499447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:15.499621Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:15.499629Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:15.499638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:15.499678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:15.499715Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342651748439258:3079]
2025-10-09T21:50:15.500111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:15.500381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:15.500413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:15.500440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342652110270268:6293] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:15.500647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:15.500699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.500705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.500706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.500708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.500709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.500710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.500711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.500712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.500714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.500719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.500729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.500733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.500737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.500740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.500744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.500748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.500752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.500756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.500762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.500771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.500782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.500787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.500794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.500801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.500804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.500807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.500809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.500812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.500815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.500818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.500821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.500824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.500826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.500829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.500832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.500835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.500851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:15.506643Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342652110270268:6293] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:15.506659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342652110270268:6293] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:15.506663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342652110270268:6293] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:15.506672Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1769 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:15.507081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:15.508131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:15.508143Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:15.508145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:15.508148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:15.508153Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1768 [2:7559342651748439258:3079]
2025-10-09T21:50:15.508163Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:15.508177Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:15.508181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:15.508183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:15.508184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:15.508197Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:15.508201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:15.508208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:15.508259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.508268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.508269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.508271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:15.508318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:15.508320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.508327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.508337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.508346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:15.508365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342652110270307:6293], now have 1 active actors on pipe
2025-10-09T21:50:15.508371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342652110270208:2432], now have 1 active actors on pipe
2025-10-09T21:50:15.508376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342651748439253:3079]
2025-10-09T21:50:15.508377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342651748439253:3079]
2025-10-09T21:50:15.508379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342651748439253:3079]
2025-10-09T21:50:15.508387Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:15.508397Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:15.508403Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:15.508405Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:15.508407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:15.508409Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:15.508411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:15.508428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342651748439253:3079]
2025-10-09T21:50:15.508430Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:15.508424Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1768
2025-10-09T21:50:15.508448Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:15.508449Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1768
2025-10-09T21:50:15.508470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:15.508477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:15.508500Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|43e8adb6-8542a008-d7f863c8-c7c1f486_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:15.508513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:15.508517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:15.508519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:15.508529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:15.508542Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:15.508557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:15.508651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:15.508656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:15.508665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:15.508723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1769 partition: 0 MaxSeqNo: 25 sessionId: src_id|43e8adb6-8542a008-d7f863c8-c7c1f486_0
2025-10-09T21:50:15.509067Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83d75b78-f01ffa75-31d6d6a3-ee40eacf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|43e8adb6-8542a008-d7f863c8-c7c1f486_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:15.511752Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|43e8adb6-8542a008-d7f863c8-c7c1f486_0] Write session: try to update token
2025-10-09T21:50:15.511769Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|43e8adb6-8542a008-d7f863c8-c7c1f486_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:15.519355Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:15.519370Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:15.519378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342651748439253:3079]
2025-10-09T21:50:15.519417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:15.519440Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:15.519564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:15.519685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:15.519800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:15.519915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:15.520092Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:15.520131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:15.526776Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1769 sessionId: src_id|43e8adb6-8542a008-d7f863c8-c7c1f486_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:15.529250Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:15.529500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:15.529517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:15.529565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:15.529658Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:15.532005Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:15.532020Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:15.532022Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:15.532024Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:15.532033Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342651748439253:3079]
2025-10-09T21:50:15.532037Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342651748439253:3079]
2025-10-09T21:50:15.532041Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342651748439253:3079]
2025-10-09T21:50:15.532079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:15.532082Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:15.532087Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:15.532093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.532093Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:15.532096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:15.532234Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:15.589372Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342652110270314:6864]
2025-10-09T21:50:15.589687Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 1768, pipe: [1:7559342652110270314:6864]
2025-10-09T21:50:15.674395Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:15.680841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:15.680863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:15.681003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:15.681058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:15.681110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:15.681168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:15.681208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:15.681216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:15.681273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:15.681325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:15.681378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:15.681436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:15.681474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:15.681481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:15.681548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:15.681598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:15.681648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:15.681700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:15.681738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:15.681746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:15.681817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:15.682113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:15.682185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:15.682241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:15.682280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:15.682286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:15.682340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:15.682398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:15.682452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:15.682532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:15.682582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:15.682587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:15.682639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:15.682690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:15.682741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:15.682868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:15.682919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:15.682927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:15.683032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:15.683080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:15.683131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:15.683181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:15.683219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:15.683224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:15.683274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:15.683350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:15.683429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:15.683492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:15.683541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:15.683547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:15.683597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:15.683646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:15.683695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:15.683751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:15.683794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:15.683803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:15.683861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:15.683909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:15.683959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:15.684008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:15.684044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:15.684050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:15.684089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342652110270314:6864], now have 1 active actors on pipe
2025-10-09T21:50:15.684340Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:15.684342Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:15.684356Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:15.684384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1769 reason: error at write: Disconnected sessionId: src_id|43e8adb6-8542a008-d7f863c8-c7c1f486_0
2025-10-09T21:50:15.684393Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342652110270314:6864]
2025-10-09T21:50:15.684575Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1769 sessionId: src_id|43e8adb6-8542a008-d7f863c8-c7c1f486_0 is DEAD
2025-10-09T21:50:15.684688Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43e8adb6-8542a008-d7f863c8-c7c1f486_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:15.684700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43e8adb6-8542a008-d7f863c8-c7c1f486_0] Write session will restart in 0.009486s
2025-10-09T21:50:15.684691Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:15.684724Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43e8adb6-8542a008-d7f863c8-c7c1f486_0] Write session: Do CDS request
2025-10-09T21:50:15.684730Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43e8adb6-8542a008-d7f863c8-c7c1f486_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:15.693427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:15.695117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:15.695267Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:15.695274Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:15.695282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:15.695323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:15.695351Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342651748439357:3077]
2025-10-09T21:50:15.695478Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|43e8adb6-8542a008-d7f863c8-c7c1f486_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:15.695510Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:15.695523Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1770
2025-10-09T21:50:15.695691Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1770 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:15.695732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1770 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:15.695732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:15.695742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1770 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:15.695760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:15.696052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:15.696107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:15.696184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1770 sessionId:  describe result for acl check
2025-10-09T21:50:15.696224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:15.696226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:15.696227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:15.696233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342652110270381:6287] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:15.696360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:15.696403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.696409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.696410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.696411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.696413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.696414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.696415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.696416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.696417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.696422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.696432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.696436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.696440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.696447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.696451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.696459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.696463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.696466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.696471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.696479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.696483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.696488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.696498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.696507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.696515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.696524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.696534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.696543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.696552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.696570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.696578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.696583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.696589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.696592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.696594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.696598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.696616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:15.696753Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342652110270381:6287] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:15.699331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342652110270381:6287] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:15.702303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342652110270381:6287] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:15.702315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342652110270381:6287] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:15.702319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342652110270381:6287] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:15.702327Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1770 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:15.702802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:15.703868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:15.703880Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:15.703882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:15.703885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:15.703891Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1769 [2:7559342651748439357:3077]
2025-10-09T21:50:15.703900Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:15.703914Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:15.703919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:15.703922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:15.703924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:15.703936Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:15.703946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:15.703955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:15.704013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.704021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.704023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.704027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:15.704087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:15.704092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.704100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.704106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.704116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:15.704132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342652110270417:6287], now have 1 active actors on pipe
2025-10-09T21:50:15.704137Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342651748439352:3077]
2025-10-09T21:50:15.704139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342651748439352:3077]
2025-10-09T21:50:15.704140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342651748439352:3077]
2025-10-09T21:50:15.704149Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:15.704157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342652110270317:2432], now have 1 active actors on pipe
2025-10-09T21:50:15.704160Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:15.704162Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:15.704163Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:15.704164Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:15.704172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:15.704174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:15.704195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342651748439352:3077]
2025-10-09T21:50:15.704207Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:15.704227Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:15.704213Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1769
2025-10-09T21:50:15.704229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:15.704232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:15.704221Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1769
2025-10-09T21:50:15.704258Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a4e59963-7403adcc-53cadd40-dd2585e7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:15.704270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:15.704273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:15.704274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:15.704281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:15.704288Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:15.704299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:15.704423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:15.704429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:15.704441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:15.704560Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1770 partition: 0 MaxSeqNo: 25 sessionId: src_id|a4e59963-7403adcc-53cadd40-dd2585e7_0
2025-10-09T21:50:15.704869Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|43e8adb6-8542a008-d7f863c8-c7c1f486_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a4e59963-7403adcc-53cadd40-dd2585e7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:15.707171Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a4e59963-7403adcc-53cadd40-dd2585e7_0] Write session: try to update token
2025-10-09T21:50:15.707188Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a4e59963-7403adcc-53cadd40-dd2585e7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:15.718244Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:15.718260Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:15.718262Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:15.718264Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:15.718274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342651748439352:3077]
2025-10-09T21:50:15.718279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342651748439352:3077]
2025-10-09T21:50:15.718283Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342651748439352:3077]
2025-10-09T21:50:15.718331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:15.718319Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:15.718337Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:15.718341Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:15.718343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.718346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:15.718484Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:15.720913Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1770 sessionId: src_id|a4e59963-7403adcc-53cadd40-dd2585e7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:15.723198Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:15.784374Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342652110270429:6864]
2025-10-09T21:50:15.784590Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 1769, pipe: [1:7559342652110270429:6864]
2025-10-09T21:50:15.868156Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:15.868428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:15.868441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:15.868478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342652110270429:6864], now have 1 active actors on pipe
2025-10-09T21:50:15.868691Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:15.868708Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:15.868718Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:15.868715Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342652110270429:6864]
2025-10-09T21:50:15.868741Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:15.868766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1770 reason: error at write: Disconnected sessionId: src_id|a4e59963-7403adcc-53cadd40-dd2585e7_0
2025-10-09T21:50:15.868954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1770 sessionId: src_id|a4e59963-7403adcc-53cadd40-dd2585e7_0 is DEAD
2025-10-09T21:50:15.869086Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4e59963-7403adcc-53cadd40-dd2585e7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:15.869093Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4e59963-7403adcc-53cadd40-dd2585e7_0] Write session will restart in 0.006839s
2025-10-09T21:50:15.869114Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4e59963-7403adcc-53cadd40-dd2585e7_0] Write session: Do CDS request
2025-10-09T21:50:15.869119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4e59963-7403adcc-53cadd40-dd2585e7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:15.877076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:15.877442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a4e59963-7403adcc-53cadd40-dd2585e7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:15.877551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:15.877565Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1771
2025-10-09T21:50:15.877696Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1771 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:15.877728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1771 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:15.877733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1771 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:15.877737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:15.879212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:15.879399Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:15.879408Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:15.879419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:15.879405Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1771 sessionId:  describe result for acl check
2025-10-09T21:50:15.879445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:15.879453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:15.879455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:15.879461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342652110270523:6271] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:15.879474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:15.879515Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342651748439498:3044]
2025-10-09T21:50:15.880029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342652110270523:6271] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:15.880106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:15.882474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342652110270523:6271] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:15.883494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:15.883558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:15.883811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:15.883849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.883855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.883856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.883857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.883857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.883858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.883858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.883859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.883860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.883863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.883870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.883872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.883874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.883876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.883879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.883881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.883883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.883884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.883889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.883894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:15.883898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.883902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:15.883904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.883907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:15.883910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.883913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:15.883916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.883919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:15.883922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.883925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:15.883928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.883931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.883933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.883936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.883940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.883943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.883961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:15.887169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342652110270523:6271] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:15.887184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342652110270523:6271] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:15.887188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342652110270523:6271] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:15.887197Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1771 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:15.903920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:15.905251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:15.905264Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:15.905266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:15.905270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:15.905276Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1770 [2:7559342651748439498:3044]
2025-10-09T21:50:15.905287Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:15.905303Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:15.905304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:15.905307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:15.905308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:15.905322Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:15.905326Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:15.905332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:15.905382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.905387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.905388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.905390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:15.905447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:15.905453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.905461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:15.905470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:15.905498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:15.905522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342652110270453:2432], now have 1 active actors on pipe
2025-10-09T21:50:15.905529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342652110270568:6271], now have 1 active actors on pipe
2025-10-09T21:50:15.905537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342651748439493:3044]
2025-10-09T21:50:15.905538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342651748439493:3044]
2025-10-09T21:50:15.905540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342651748439493:3044]
2025-10-09T21:50:15.905549Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:15.905561Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:15.905564Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:15.905564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:15.905566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:15.905574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:15.905576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:15.905613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342651748439493:3044]
2025-10-09T21:50:15.905625Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:15.905631Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:15.905630Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1770
2025-10-09T21:50:15.905676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:15.905647Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1770
2025-10-09T21:50:15.905686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:15.905710Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|23e19005-d0f72383-89248fad-aed72e40_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:15.905720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:15.905724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:15.905726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:15.905737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:15.905754Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:15.905769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:15.905922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:15.905933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:15.905952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:15.906023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1771 partition: 0 MaxSeqNo: 25 sessionId: src_id|23e19005-d0f72383-89248fad-aed72e40_0
2025-10-09T21:50:15.906357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4e59963-7403adcc-53cadd40-dd2585e7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|23e19005-d0f72383-89248fad-aed72e40_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:15.908598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23e19005-d0f72383-89248fad-aed72e40_0] Write session: try to update token
2025-10-09T21:50:15.908611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23e19005-d0f72383-89248fad-aed72e40_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:15.919174Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:15.919189Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:15.919191Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:15.919192Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:15.919203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342651748439493:3044]
2025-10-09T21:50:15.919206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342651748439493:3044]
2025-10-09T21:50:15.919209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342651748439493:3044]
2025-10-09T21:50:15.919250Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:15.919251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:15.919254Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:15.919256Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:15.919258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:15.919260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:15.919390Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:15.920246Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1771 sessionId: src_id|23e19005-d0f72383-89248fad-aed72e40_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:15.922555Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:15.968755Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342652110270581:6864]
2025-10-09T21:50:15.969005Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 1770, pipe: [1:7559342652110270581:6864]
2025-10-09T21:50:16.067655Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:16.068029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.068041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.068079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342652110270581:6864], now have 1 active actors on pipe
2025-10-09T21:50:16.068361Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342652110270581:6864]
2025-10-09T21:50:16.068374Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:16.068380Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:16.068393Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:16.068454Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1771 reason: error at write: Disconnected sessionId: src_id|23e19005-d0f72383-89248fad-aed72e40_0
2025-10-09T21:50:16.068469Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:16.068637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1771 sessionId: src_id|23e19005-d0f72383-89248fad-aed72e40_0 is DEAD
2025-10-09T21:50:16.068817Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23e19005-d0f72383-89248fad-aed72e40_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:16.068825Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23e19005-d0f72383-89248fad-aed72e40_0] Write session will restart in 0.006298s
2025-10-09T21:50:16.068846Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23e19005-d0f72383-89248fad-aed72e40_0] Write session: Do CDS request
2025-10-09T21:50:16.068852Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23e19005-d0f72383-89248fad-aed72e40_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:16.076573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23e19005-d0f72383-89248fad-aed72e40_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:16.076653Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:16.076670Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1772
2025-10-09T21:50:16.076765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:16.076826Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1772 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:16.076861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1772 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:16.076873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1772 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:16.076877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:16.077294Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1772 sessionId:  describe result for acl check
2025-10-09T21:50:16.077338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:16.077347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:16.077349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:16.077356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342656405237940:6262] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:16.077917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342656405237940:6262] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:16.078387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:16.078553Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:16.078561Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:16.078571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:16.078612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:16.078649Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342656043406894:3042]
2025-10-09T21:50:16.079013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:16.079241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:16.079276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:16.079480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:16.079510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.079515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.079515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.079516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.079517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.079517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.079518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.079518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.079519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.079522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.079529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.079531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.079533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.079536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.079538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.079540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.079542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.079553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342656405237940:6262] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:16.079544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.079548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.079553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.079558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.079561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.079564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.079567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.079569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.079572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.079575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.079578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.079581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.079584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.079587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.079591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.079594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.079597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.079600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.079603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.079620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:16.085134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342656405237940:6262] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:16.085152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342656405237940:6262] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:16.085156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342656405237940:6262] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:16.085172Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1772 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:16.085919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:16.087057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:16.087069Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:16.087072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:16.087076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:16.087081Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1771 [2:7559342656043406894:3042]
2025-10-09T21:50:16.087091Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:16.087105Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:16.087108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:16.087110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:16.087112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:16.087125Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:16.087129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:16.087135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:16.087188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.087195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.087196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.087198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:16.087246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:16.087251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.087259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.087266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.087279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:16.087298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405237878:2432], now have 1 active actors on pipe
2025-10-09T21:50:16.087307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342656043406889:3042]
2025-10-09T21:50:16.087309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342656043406889:3042]
2025-10-09T21:50:16.087310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342656043406889:3042]
2025-10-09T21:50:16.087319Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:16.087325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405237980:6262], now have 1 active actors on pipe
2025-10-09T21:50:16.087331Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:16.087336Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:16.087339Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:16.087340Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:16.087343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:16.087346Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:16.087371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342656043406889:3042]
2025-10-09T21:50:16.087381Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:16.087392Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:16.087397Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1771
2025-10-09T21:50:16.087437Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1771
2025-10-09T21:50:16.087499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.087508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.087535Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e89be623-2236dff7-700b470d-1f001cf7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:16.087547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:16.087555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:16.087557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:16.087568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:16.087582Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:16.087601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:16.087748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.087757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.087769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:16.087845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1772 partition: 0 MaxSeqNo: 25 sessionId: src_id|e89be623-2236dff7-700b470d-1f001cf7_0
2025-10-09T21:50:16.088165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23e19005-d0f72383-89248fad-aed72e40_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e89be623-2236dff7-700b470d-1f001cf7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:16.090687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e89be623-2236dff7-700b470d-1f001cf7_0] Write session: try to update token
2025-10-09T21:50:16.090704Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e89be623-2236dff7-700b470d-1f001cf7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:16.102143Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:16.102157Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:16.102161Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:16.102162Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:16.102170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342656043406889:3042]
2025-10-09T21:50:16.102173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342656043406889:3042]
2025-10-09T21:50:16.102176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342656043406889:3042]
2025-10-09T21:50:16.102215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:16.102221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.102220Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:16.102223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:16.102226Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:16.102228Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:16.102353Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:16.106852Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1772 sessionId: src_id|e89be623-2236dff7-700b470d-1f001cf7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:16.109421Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:16.168447Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342656405237988:6864]
2025-10-09T21:50:16.168787Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 1771, pipe: [1:7559342656405237988:6864]
2025-10-09T21:50:16.248417Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:16.248700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.248713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.248751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405237988:6864], now have 1 active actors on pipe
2025-10-09T21:50:16.248943Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:16.248948Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:16.248960Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342656405237988:6864]
2025-10-09T21:50:16.248982Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:16.248985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:16.249044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1772 reason: error at write: Disconnected sessionId: src_id|e89be623-2236dff7-700b470d-1f001cf7_0
2025-10-09T21:50:16.249234Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1772 sessionId: src_id|e89be623-2236dff7-700b470d-1f001cf7_0 is DEAD
2025-10-09T21:50:16.249339Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e89be623-2236dff7-700b470d-1f001cf7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:16.249348Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e89be623-2236dff7-700b470d-1f001cf7_0] Write session will restart in 0.005615s
2025-10-09T21:50:16.249367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e89be623-2236dff7-700b470d-1f001cf7_0] Write session: Do CDS request
2025-10-09T21:50:16.249372Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e89be623-2236dff7-700b470d-1f001cf7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:16.256496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e89be623-2236dff7-700b470d-1f001cf7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:16.256509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:16.256549Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:16.256563Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1773
2025-10-09T21:50:16.256701Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1773 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:16.256734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1773 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:16.256745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1773 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:16.256749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:16.257160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1773 sessionId:  describe result for acl check
2025-10-09T21:50:16.257196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:16.257197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:16.257199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:16.257204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342656405238057:6258] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:16.257745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342656405238057:6258] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:16.258052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:16.258205Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:16.258211Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:16.258220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:16.258258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:16.258285Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342656043406995:3027]
2025-10-09T21:50:16.258703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:16.258897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:16.258934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:16.259150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:16.259180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.259181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.259182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.259182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.259183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.259184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.259184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.259185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.259185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.259188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.259194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.259196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.259198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.259201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.259203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.259205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.259207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.259209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.259214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.259219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.259224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.259228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.259230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.259233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.259236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.259239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.259242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.259244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.259247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.259250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.259253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.259256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.259259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.259262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.259265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.259268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.259284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:16.259796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342656405238057:6258] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:16.265175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342656405238057:6258] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:16.265194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342656405238057:6258] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:16.265198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342656405238057:6258] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:16.265210Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1773 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:16.265635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:16.266601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:16.266613Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:16.266614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:16.266617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:16.266622Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1772 [2:7559342656043406995:3027]
2025-10-09T21:50:16.266631Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:16.266645Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:16.266651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:16.266653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:16.266654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:16.266666Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:16.266670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:16.266675Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:16.266715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.266720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.266721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.266723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:16.266855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:16.266865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.266873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.266874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1772
2025-10-09T21:50:16.266880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.266891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:16.266895Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1772
2025-10-09T21:50:16.266905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405238097:6258], now have 1 active actors on pipe
2025-10-09T21:50:16.266908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405237993:2432], now have 1 active actors on pipe
2025-10-09T21:50:16.266914Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342656043406990:3027]
2025-10-09T21:50:16.266917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342656043406990:3027]
2025-10-09T21:50:16.266918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342656043406990:3027]
2025-10-09T21:50:16.266924Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:16.266934Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:16.266935Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:16.266937Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:16.266939Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:16.266945Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:16.266947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:16.266960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.266963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.266993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342656043406990:3027]
2025-10-09T21:50:16.267005Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:16.267018Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6589918-31f453f7-18cfa96d-df53ba00_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:16.267021Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:16.267028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:16.267031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:16.267033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:16.267040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:16.267048Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:16.267078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:16.267248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.267254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.267264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:16.267338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1773 partition: 0 MaxSeqNo: 25 sessionId: src_id|6589918-31f453f7-18cfa96d-df53ba00_0
2025-10-09T21:50:16.267616Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e89be623-2236dff7-700b470d-1f001cf7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6589918-31f453f7-18cfa96d-df53ba00_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:16.269845Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6589918-31f453f7-18cfa96d-df53ba00_0] Write session: try to update token
2025-10-09T21:50:16.269859Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6589918-31f453f7-18cfa96d-df53ba00_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:16.281326Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:16.281341Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:16.281343Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:16.281344Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:16.281353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342656043406990:3027]
2025-10-09T21:50:16.281356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342656043406990:3027]
2025-10-09T21:50:16.281360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342656043406990:3027]
2025-10-09T21:50:16.281396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:16.281401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.281399Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:16.281402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:16.281406Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:16.281410Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:16.281529Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:16.281701Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1773 sessionId: src_id|6589918-31f453f7-18cfa96d-df53ba00_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:16.284064Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:16.349118Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342656405238109:6864]
2025-10-09T21:50:16.349370Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 1772, pipe: [1:7559342656405238109:6864]
2025-10-09T21:50:16.423813Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:16.424064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.424076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.424115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405238109:6864], now have 1 active actors on pipe
2025-10-09T21:50:16.424366Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342656405238109:6864]
2025-10-09T21:50:16.424391Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:16.424395Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:16.424397Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:16.424489Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1773 reason: error at write: Disconnected sessionId: src_id|6589918-31f453f7-18cfa96d-df53ba00_0
2025-10-09T21:50:16.424554Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:16.424658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1773 sessionId: src_id|6589918-31f453f7-18cfa96d-df53ba00_0 is DEAD
2025-10-09T21:50:16.424800Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6589918-31f453f7-18cfa96d-df53ba00_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:16.424807Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6589918-31f453f7-18cfa96d-df53ba00_0] Write session will restart in 0.005306s
2025-10-09T21:50:16.424826Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6589918-31f453f7-18cfa96d-df53ba00_0] Write session: Do CDS request
2025-10-09T21:50:16.424830Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6589918-31f453f7-18cfa96d-df53ba00_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:16.431465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6589918-31f453f7-18cfa96d-df53ba00_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:16.431528Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:16.431540Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1774
2025-10-09T21:50:16.431653Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1774 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:16.431679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1774 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:16.431683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1774 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:16.431686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:16.432073Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1774 sessionId:  describe result for acl check
2025-10-09T21:50:16.432102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:16.432108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:16.432110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:16.432113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342656405238175:6244] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:16.432163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:16.432524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342656405238175:6244] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:16.433677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:16.433857Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:16.433865Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:16.433875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:16.433919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:16.433954Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342656043407097:3035]
2025-10-09T21:50:16.434268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342656405238175:6244] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:16.434537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:16.434801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:16.434851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:16.435193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:16.435244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.435251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.435252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.435253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.435254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.435255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.435256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.435257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.435258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.435263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.435271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.435274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.435278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.435282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.435285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.435289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.435292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.435296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.435303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.435310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.435317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.435322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.435326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.435331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.435338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.435343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.435348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.435367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.435376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.435381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.435386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.435392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.435400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.435406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.435411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.435416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.435444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:16.439998Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342656405238175:6244] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:16.440011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342656405238175:6244] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:16.440013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342656405238175:6244] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:16.440019Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1774 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:16.441392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:16.442600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:16.442613Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:16.442614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:16.442617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:16.442623Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1773 [2:7559342656043407097:3035]
2025-10-09T21:50:16.442632Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:16.442645Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:16.442647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:16.442649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:16.442650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:16.442662Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:16.442667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:16.442673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:16.442714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.442720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.442721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.442722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:16.442773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:16.442778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.442785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.442796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.442817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:16.442837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405238215:6244], now have 1 active actors on pipe
2025-10-09T21:50:16.442842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405238110:2432], now have 1 active actors on pipe
2025-10-09T21:50:16.442847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342656043407092:3035]
2025-10-09T21:50:16.442849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342656043407092:3035]
2025-10-09T21:50:16.442850Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342656043407092:3035]
2025-10-09T21:50:16.442859Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:16.442873Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:16.442885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:16.442890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:16.442889Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:16.442894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:16.442896Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:16.442882Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1773
2025-10-09T21:50:16.442910Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342656043407092:3035]
2025-10-09T21:50:16.442896Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1773
2025-10-09T21:50:16.442912Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:16.442921Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:16.442929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.442932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.442957Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c797acae-796656b-ac4da39d-212c65f4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:16.442967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:16.442970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:16.442972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:16.442980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:16.442990Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:16.443003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:16.443123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.443128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.443140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:16.443224Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1774 partition: 0 MaxSeqNo: 25 sessionId: src_id|c797acae-796656b-ac4da39d-212c65f4_0
2025-10-09T21:50:16.443511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6589918-31f453f7-18cfa96d-df53ba00_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c797acae-796656b-ac4da39d-212c65f4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:16.445922Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c797acae-796656b-ac4da39d-212c65f4_0] Write session: try to update token
2025-10-09T21:50:16.445939Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c797acae-796656b-ac4da39d-212c65f4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:16.456398Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:16.456414Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:16.456416Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:16.456417Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:16.456425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342656043407092:3035]
2025-10-09T21:50:16.456428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342656043407092:3035]
2025-10-09T21:50:16.456433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342656043407092:3035]
2025-10-09T21:50:16.456465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:16.456466Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:16.456470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.456471Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:16.456472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:16.456474Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:16.456675Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:16.459181Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1774 sessionId: src_id|c797acae-796656b-ac4da39d-212c65f4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:16.461289Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:16.524233Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342656405238223:6864]
2025-10-09T21:50:16.524482Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 1773, pipe: [1:7559342656405238223:6864]
2025-10-09T21:50:16.596158Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:16.596432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.596444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.596480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405238223:6864], now have 1 active actors on pipe
2025-10-09T21:50:16.596638Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:16.596682Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342656405238223:6864]
2025-10-09T21:50:16.596704Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:16.596713Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:16.596782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1774 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c797acae-796656b-ac4da39d-212c65f4_0
2025-10-09T21:50:16.596979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1774 sessionId: src_id|c797acae-796656b-ac4da39d-212c65f4_0 is DEAD
2025-10-09T21:50:16.597082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c797acae-796656b-ac4da39d-212c65f4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:16.597092Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c797acae-796656b-ac4da39d-212c65f4_0] Write session will restart in 0.009034s
2025-10-09T21:50:16.597120Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c797acae-796656b-ac4da39d-212c65f4_0] Write session: Do CDS request
2025-10-09T21:50:16.597126Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c797acae-796656b-ac4da39d-212c65f4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:16.605184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:16.607119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:16.607320Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:16.607328Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:16.607339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:16.607392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:16.607439Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342656043407200:3037]
2025-10-09T21:50:16.607522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c797acae-796656b-ac4da39d-212c65f4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:16.607654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:16.607684Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1775
2025-10-09T21:50:16.607850Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1775 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:16.607894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1775 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:16.607900Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1775 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:16.607908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:16.608001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:16.608323Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1775 sessionId:  describe result for acl check
2025-10-09T21:50:16.608360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:16.608369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:16.608371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:16.608377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:16.608377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342656405238295:6226] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:16.608439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:16.608792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:16.608873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.608881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.608882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.608883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.608884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.608885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.608887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.608888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.608889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.608894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.608905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.608908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.608911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.608915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.608943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342656405238295:6226] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:16.608918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.608921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.608924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.608927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.608936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.608944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.608950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.608954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.608958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.608963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.608968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.608973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.608977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.608982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.608986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.608990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.608995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.608999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.609011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.609016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.609021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.609027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.609053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:16.611542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342656405238295:6226] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:16.615024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342656405238295:6226] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:16.615039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342656405238295:6226] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:16.615042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342656405238295:6226] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:16.615052Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1775 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:16.615589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:16.616737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:16.616753Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:16.616756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:16.616760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:16.616765Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1774 [2:7559342656043407200:3037]
2025-10-09T21:50:16.616781Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:16.616798Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:16.616805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:16.616808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:16.616811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:16.616826Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:16.616832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:16.616841Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:16.616895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.616902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.616904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.616906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:16.616958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:16.616964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.616971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.616977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.616997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:16.617019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342656043407195:3037]
2025-10-09T21:50:16.617023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342656043407195:3037]
2025-10-09T21:50:16.617025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342656043407195:3037]
2025-10-09T21:50:16.617033Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:16.617046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405238225:2432], now have 1 active actors on pipe
2025-10-09T21:50:16.617049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:16.617051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405238332:6226], now have 1 active actors on pipe
2025-10-09T21:50:16.617059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:16.617063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:16.617065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:16.617067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:16.617069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:16.617065Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1774
2025-10-09T21:50:16.617078Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1774
2025-10-09T21:50:16.617100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342656043407195:3037]
2025-10-09T21:50:16.617106Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:16.617109Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:16.617121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.617127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.617162Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|acb95f61-e489d166-e765f4db-155ee54_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:16.617172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:16.617175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:16.617176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:16.617184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:16.617191Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:16.617208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:16.617311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.617317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.617328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:16.617411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1775 partition: 0 MaxSeqNo: 25 sessionId: src_id|acb95f61-e489d166-e765f4db-155ee54_0
2025-10-09T21:50:16.617753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c797acae-796656b-ac4da39d-212c65f4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|acb95f61-e489d166-e765f4db-155ee54_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:16.620240Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|acb95f61-e489d166-e765f4db-155ee54_0] Write session: try to update token
2025-10-09T21:50:16.620257Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|acb95f61-e489d166-e765f4db-155ee54_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:16.631870Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:16.631888Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:16.631891Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:16.631893Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:16.631902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342656043407195:3037]
2025-10-09T21:50:16.631907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342656043407195:3037]
2025-10-09T21:50:16.631911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342656043407195:3037]
2025-10-09T21:50:16.631956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:16.631963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.631965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:16.631968Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:16.631975Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:16.631978Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:16.632142Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:16.633681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1775 sessionId: src_id|acb95f61-e489d166-e765f4db-155ee54_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:16.636055Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:16.696809Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342656405238339:6864]
2025-10-09T21:50:16.697060Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 1774, pipe: [1:7559342656405238339:6864]
2025-10-09T21:50:16.786476Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:16.787130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.787147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.787197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405238339:6864], now have 1 active actors on pipe
2025-10-09T21:50:16.787391Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:16.787407Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:16.787411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:16.787455Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342656405238339:6864]
2025-10-09T21:50:16.787465Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1775 reason: error at write: Disconnected sessionId: src_id|acb95f61-e489d166-e765f4db-155ee54_0
2025-10-09T21:50:16.787618Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1775 sessionId: src_id|acb95f61-e489d166-e765f4db-155ee54_0 is DEAD
2025-10-09T21:50:16.787765Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|acb95f61-e489d166-e765f4db-155ee54_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:16.787778Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|acb95f61-e489d166-e765f4db-155ee54_0] Write session will restart in 0.006936s
2025-10-09T21:50:16.787815Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|acb95f61-e489d166-e765f4db-155ee54_0] Write session: Do CDS request
2025-10-09T21:50:16.787825Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|acb95f61-e489d166-e765f4db-155ee54_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:16.788036Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:16.795485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|acb95f61-e489d166-e765f4db-155ee54_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:16.795497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:16.795551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:16.795563Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1776
2025-10-09T21:50:16.795678Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1776 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:16.795702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1776 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:16.795705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1776 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:16.795708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:16.796044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1776 sessionId:  describe result for acl check
2025-10-09T21:50:16.796070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:16.796077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:16.796078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:16.796085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342656405238412:6222] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:16.796530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342656405238412:6222] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:16.797070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:16.797228Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:16.797236Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:16.797245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:16.797295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:16.797324Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342656043407309:3032]
2025-10-09T21:50:16.797736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:16.798001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:16.798043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:16.798246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342656405238412:6222] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:16.798267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:16.798305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.798309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.798310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.798311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.798311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.798312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.798313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.798313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.798314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.798317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.798323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.798325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.798327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.798329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.798331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.798333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.798335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.798337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.798341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.798347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.798351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.798354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.798357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.798360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.798365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.798368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.798371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.798373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.798376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.798379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.798382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.798385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.798387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.798390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.798393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.798396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.798411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:16.803064Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342656405238412:6222] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:16.803079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342656405238412:6222] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:16.803083Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342656405238412:6222] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:16.803090Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1776 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:16.804465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:16.805647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:16.805660Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:16.805662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:16.805665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:16.805670Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1775 [2:7559342656043407309:3032]
2025-10-09T21:50:16.805679Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:16.805692Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:16.805695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:16.805698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:16.805701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:16.805718Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:16.805725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:16.805739Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:16.805794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.805802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.805804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.805806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:16.805860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:16.805866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.805873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.805879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.805890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:16.805903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342656043407304:3032]
2025-10-09T21:50:16.805908Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342656043407304:3032]
2025-10-09T21:50:16.805909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342656043407304:3032]
2025-10-09T21:50:16.805916Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:16.805926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405238345:2432], now have 1 active actors on pipe
2025-10-09T21:50:16.805927Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:16.805930Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:16.805933Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:16.805934Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:16.805943Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:16.805945Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:16.805949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405238451:6222], now have 1 active actors on pipe
2025-10-09T21:50:16.805969Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342656043407304:3032]
2025-10-09T21:50:16.805977Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:16.805996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:16.806000Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1775
2025-10-09T21:50:16.806017Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1775
2025-10-09T21:50:16.806105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.806117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.806146Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5c6a835d-500b0566-50ec4116-b210fb52_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:16.806160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:16.806166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:16.806168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:16.806179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:16.806200Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:16.806216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:16.806374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.806382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.806396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:16.806498Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1776 partition: 0 MaxSeqNo: 25 sessionId: src_id|5c6a835d-500b0566-50ec4116-b210fb52_0
2025-10-09T21:50:16.806828Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|acb95f61-e489d166-e765f4db-155ee54_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5c6a835d-500b0566-50ec4116-b210fb52_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:16.809262Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c6a835d-500b0566-50ec4116-b210fb52_0] Write session: try to update token
2025-10-09T21:50:16.809278Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c6a835d-500b0566-50ec4116-b210fb52_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:16.819713Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:16.819728Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:16.819730Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:16.819731Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:16.819742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342656043407304:3032]
2025-10-09T21:50:16.819745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342656043407304:3032]
2025-10-09T21:50:16.819748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342656043407304:3032]
2025-10-09T21:50:16.819796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:16.819804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.819803Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:16.819806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:16.819809Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:16.819814Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:16.819928Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:16.822671Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1776 sessionId: src_id|5c6a835d-500b0566-50ec4116-b210fb52_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:16.824988Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:16.888260Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342656405238482:6864]
2025-10-09T21:50:16.888539Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 1775, pipe: [1:7559342656405238482:6864]
2025-10-09T21:50:16.970511Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:16.970867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.970878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.970915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405238482:6864], now have 1 active actors on pipe
2025-10-09T21:50:16.971180Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:16.971177Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:16.971196Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:16.971199Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:16.971243Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342656405238482:6864]
2025-10-09T21:50:16.971272Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1776 reason: error at write: Disconnected sessionId: src_id|5c6a835d-500b0566-50ec4116-b210fb52_0
2025-10-09T21:50:16.971445Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1776 sessionId: src_id|5c6a835d-500b0566-50ec4116-b210fb52_0 is DEAD
2025-10-09T21:50:16.971603Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c6a835d-500b0566-50ec4116-b210fb52_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:16.971615Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c6a835d-500b0566-50ec4116-b210fb52_0] Write session will restart in 0.005883s
2025-10-09T21:50:16.971641Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c6a835d-500b0566-50ec4116-b210fb52_0] Write session: Do CDS request
2025-10-09T21:50:16.971648Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c6a835d-500b0566-50ec4116-b210fb52_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:16.978492Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c6a835d-500b0566-50ec4116-b210fb52_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:16.978550Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:16.978565Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1777
2025-10-09T21:50:16.978713Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1777 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:16.978758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1777 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:16.978766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1777 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:16.978778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:16.979217Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1777 sessionId:  describe result for acl check
2025-10-09T21:50:16.979255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:16.979259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:16.979262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:16.979267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342656405238552:6207] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:16.979826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342656405238552:6207] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:16.979885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:16.981533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:16.981707Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:16.981715Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:16.981723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:16.981771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:16.981804Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342656043407452:3011]
2025-10-09T21:50:16.981889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342656405238552:6207] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:16.982253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:16.982503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:16.982542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:16.982842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:16.982890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.982896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.982897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.982898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.982900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.982901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.982902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.982903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.982904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.982909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.982919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.982924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.982928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.982932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.982936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.982940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.982944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.982948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.982952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.982958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:16.982962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.982965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:16.982970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.982974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:16.982976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.982979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:16.982982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.982985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:16.982988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.982991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:16.982994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.982997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.982999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.983002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.983006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.983009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.983030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:16.987763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342656405238552:6207] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:16.987777Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342656405238552:6207] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:16.987782Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342656405238552:6207] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:16.987790Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1777 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:16.989077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:16.990196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:16.990207Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:16.990209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:16.990212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:16.990216Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1776 [2:7559342656043407452:3011]
2025-10-09T21:50:16.990224Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:16.990236Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:16.990238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:16.990241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:16.990242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:16.990254Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:16.990258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:16.990263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:16.990304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.990311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.990313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.990316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:16.990367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:16.990372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:16.990379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:16.990386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:16.990395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:16.990411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342656043407447:3011]
2025-10-09T21:50:16.990416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342656043407447:3011]
2025-10-09T21:50:16.990419Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342656043407447:3011]
2025-10-09T21:50:16.990426Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:16.990438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405238591:6207], now have 1 active actors on pipe
2025-10-09T21:50:16.990439Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:16.990441Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:16.990443Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:16.990446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342656405238487:2432], now have 1 active actors on pipe
2025-10-09T21:50:16.990453Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:16.990461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:16.990463Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:16.990481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342656043407447:3011]
2025-10-09T21:50:16.990486Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:16.990494Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:16.990499Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1776
2025-10-09T21:50:16.990527Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1776
2025-10-09T21:50:16.990586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.990594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.990624Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f0c3ca71-db0e4722-5ecf966e-b091f978_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:16.990636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:16.990642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:16.990643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:16.990650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:16.990659Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:16.990678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:16.990835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:16.990845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:16.990862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:16.990948Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1777 partition: 0 MaxSeqNo: 25 sessionId: src_id|f0c3ca71-db0e4722-5ecf966e-b091f978_0
2025-10-09T21:50:16.991244Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c6a835d-500b0566-50ec4116-b210fb52_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f0c3ca71-db0e4722-5ecf966e-b091f978_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:16.993991Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0c3ca71-db0e4722-5ecf966e-b091f978_0] Write session: try to update token
2025-10-09T21:50:16.994007Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0c3ca71-db0e4722-5ecf966e-b091f978_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:17.001602Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:17.001615Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:17.001624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342656043407447:3011]
2025-10-09T21:50:17.001670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:17.001672Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:17.001834Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:17.001967Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:17.002465Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:17.002620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:17.002799Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:17.002840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:17.007306Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1777 sessionId: src_id|f0c3ca71-db0e4722-5ecf966e-b091f978_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:17.009696Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:17.009891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:17.009909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:17.009952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:17.010049Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:17.014379Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:17.014400Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:17.014403Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:17.014408Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:17.014421Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342656043407447:3011]
2025-10-09T21:50:17.014425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342656043407447:3011]
2025-10-09T21:50:17.014434Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342656043407447:3011]
2025-10-09T21:50:17.014479Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:17.014486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:17.014490Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:17.014493Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:17.014495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.014497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:17.014696Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:17.071616Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342660700205894:6864]
2025-10-09T21:50:17.071911Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 1776, pipe: [1:7559342660700205894:6864]
2025-10-09T21:50:17.158869Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:17.166327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:17.166350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:17.166516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:17.166576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:17.166641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:17.166703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:17.166899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:17.166913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:17.166978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:17.167035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:17.167096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:17.167158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:17.167199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:17.167205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:17.167262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:17.167320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:17.167378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:17.167433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:17.167785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:17.167794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:17.167859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:17.167917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:17.167975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:17.168034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:17.168074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:17.168085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:17.168146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:17.168203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:17.168262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:17.168314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:17.168357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:17.168364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:17.168425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:17.168486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:17.168543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:17.168604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:17.168664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:17.168674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:17.168733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:17.168786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:17.168845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:17.168904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:17.168959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:17.168964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:17.169016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:17.169081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:17.169157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:17.169222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:17.169277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:17.169285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:17.169343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:17.169398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:17.169461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:17.169528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:17.169567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:17.169574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:17.169633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:17.169687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:17.169741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:17.169792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:17.169831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:17.169836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:17.169893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342660700205894:6864], now have 1 active actors on pipe
2025-10-09T21:50:17.170170Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:17.170188Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:17.170188Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342660700205894:6864]
2025-10-09T21:50:17.170220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1777 reason: error at write: Disconnected sessionId: src_id|f0c3ca71-db0e4722-5ecf966e-b091f978_0
2025-10-09T21:50:17.170221Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:17.170300Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:17.170418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1777 sessionId: src_id|f0c3ca71-db0e4722-5ecf966e-b091f978_0 is DEAD
2025-10-09T21:50:17.170551Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0c3ca71-db0e4722-5ecf966e-b091f978_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:17.170576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0c3ca71-db0e4722-5ecf966e-b091f978_0] Write session will restart in 0.007008s
2025-10-09T21:50:17.170599Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0c3ca71-db0e4722-5ecf966e-b091f978_0] Write session: Do CDS request
2025-10-09T21:50:17.170605Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0c3ca71-db0e4722-5ecf966e-b091f978_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:17.178486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0c3ca71-db0e4722-5ecf966e-b091f978_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:17.178898Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:17.178912Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1778
2025-10-09T21:50:17.179046Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1778 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:17.179086Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1778 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:17.179105Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1778 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:17.179111Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:17.179495Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1778 sessionId:  describe result for acl check
2025-10-09T21:50:17.179522Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:17.179524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:17.179526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:17.179529Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342660700205965:6204] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:17.179651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:17.180069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342660700205965:6204] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:17.181310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:17.181496Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:17.181506Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:17.181519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:17.181576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:17.181613Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342660338374854:3004]
2025-10-09T21:50:17.182214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:17.182465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:17.182482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342660700205965:6204] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:17.182506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:17.182809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:17.182862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.182868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.182869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.182870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.182871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.182872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.182873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.182874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.182876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.182881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.182891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.182895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.182899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.182903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.182907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.182910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.182913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.182916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.182922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.182929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.182935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.182940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.182950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.182955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.182964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.182969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.182974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.182979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.182988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.182993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.183002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.183012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.183020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.183025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.183034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.183039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.183067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:17.188057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342660700205965:6204] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:17.188074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342660700205965:6204] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:17.188078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342660700205965:6204] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:17.188087Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1778 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:17.189365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:17.190460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:17.190476Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:17.190478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:17.190483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:17.190490Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1777 [2:7559342660338374854:3004]
2025-10-09T21:50:17.190503Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:17.190521Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:17.190524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:17.190528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:17.190530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:17.190547Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:17.190555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:17.190567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:17.190615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.190621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.190622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.190623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:17.190684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:17.190692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.190704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.190715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.190736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:17.190762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342660700206008:6204], now have 1 active actors on pipe
2025-10-09T21:50:17.190771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342660700205897:2432], now have 1 active actors on pipe
2025-10-09T21:50:17.190779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342660338374849:3004]
2025-10-09T21:50:17.190782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342660338374849:3004]
2025-10-09T21:50:17.190784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342660338374849:3004]
2025-10-09T21:50:17.190799Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:17.190812Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:17.190817Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:17.190820Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:17.190822Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:17.190831Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:17.190835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:17.190829Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1777
2025-10-09T21:50:17.190830Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1777
2025-10-09T21:50:17.190872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342660338374849:3004]
2025-10-09T21:50:17.190880Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:17.190882Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:17.190897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:17.190906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:17.190954Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a18632af-c6377c09-96bbdd7-ad0cb884_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:17.190967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:17.190972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:17.190973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:17.190981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:17.190993Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:17.191010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:17.191184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:17.191193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:17.191207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:17.191299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1778 partition: 0 MaxSeqNo: 25 sessionId: src_id|a18632af-c6377c09-96bbdd7-ad0cb884_0
2025-10-09T21:50:17.191625Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0c3ca71-db0e4722-5ecf966e-b091f978_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a18632af-c6377c09-96bbdd7-ad0cb884_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:17.193933Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a18632af-c6377c09-96bbdd7-ad0cb884_0] Write session: try to update token
2025-10-09T21:50:17.193948Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a18632af-c6377c09-96bbdd7-ad0cb884_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:17.204831Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:17.204848Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:17.204851Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:17.204853Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:17.204861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342660338374849:3004]
2025-10-09T21:50:17.204866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342660338374849:3004]
2025-10-09T21:50:17.204871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342660338374849:3004]
2025-10-09T21:50:17.204917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:17.204931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.204934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:17.204950Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:17.204974Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:17.204988Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:17.205103Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:17.207766Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1778 sessionId: src_id|a18632af-c6377c09-96bbdd7-ad0cb884_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:17.210084Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:17.270254Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342660700206020:6864]
2025-10-09T21:50:17.270527Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 1777, pipe: [1:7559342660700206020:6864]
2025-10-09T21:50:17.349647Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:17.349968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:17.349981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:17.350020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342660700206020:6864], now have 1 active actors on pipe
2025-10-09T21:50:17.350187Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:17.350212Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:17.350230Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:17.350238Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342660700206020:6864]
2025-10-09T21:50:17.350258Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:17.350291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1778 reason: error at write: Disconnected sessionId: src_id|a18632af-c6377c09-96bbdd7-ad0cb884_0
2025-10-09T21:50:17.350473Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1778 sessionId: src_id|a18632af-c6377c09-96bbdd7-ad0cb884_0 is DEAD
2025-10-09T21:50:17.350605Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a18632af-c6377c09-96bbdd7-ad0cb884_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:17.350621Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a18632af-c6377c09-96bbdd7-ad0cb884_0] Write session will restart in 0.008089s
2025-10-09T21:50:17.350641Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a18632af-c6377c09-96bbdd7-ad0cb884_0] Write session: Do CDS request
2025-10-09T21:50:17.350647Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a18632af-c6377c09-96bbdd7-ad0cb884_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:17.359233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:17.359462Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a18632af-c6377c09-96bbdd7-ad0cb884_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:17.359553Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:17.359570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1779
2025-10-09T21:50:17.359710Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1779 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:17.359745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1779 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:17.359755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1779 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:17.359759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:17.360129Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1779 sessionId:  describe result for acl check
2025-10-09T21:50:17.360163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:17.360169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:17.360170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:17.360174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342660700206093:6198] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:17.360631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342660700206093:6198] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:17.360918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:17.361087Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:17.361098Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:17.361109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:17.361161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:17.361196Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342660338374962:3008]
2025-10-09T21:50:17.361768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:17.362092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:17.362139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:17.362384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:17.362437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.362444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.362445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.362446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.362447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.362448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.362449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.362450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.362452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.362457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.362465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.362470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.362474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.362477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.362481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.362484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.362488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.362491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.362497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.362506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.362512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.362518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.362523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.362528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.362534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.362537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.362540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.362543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.362547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.362552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.362555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.362558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.362560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.362563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.362568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.362571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.362587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:17.362802Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342660700206093:6198] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:17.368397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342660700206093:6198] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:17.368414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342660700206093:6198] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:17.368435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342660700206093:6198] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:17.368448Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1779 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:17.368852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:17.370005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:17.370020Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:17.370023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:17.370026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:17.370048Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1778 [2:7559342660338374962:3008]
2025-10-09T21:50:17.370060Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:17.370079Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:17.370088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:17.370091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:17.370095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:17.370111Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:17.370121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:17.370128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:17.370188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.370195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.370197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.370200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:17.370264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:17.370274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.370287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.370300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.370317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:17.370344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342660700206023:2432], now have 1 active actors on pipe
2025-10-09T21:50:17.370352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342660700206132:6198], now have 1 active actors on pipe
2025-10-09T21:50:17.370361Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342660338374957:3008]
2025-10-09T21:50:17.370368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342660338374957:3008]
2025-10-09T21:50:17.370370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342660338374957:3008]
2025-10-09T21:50:17.370384Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:17.370397Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:17.370404Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:17.370405Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:17.370406Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:17.370421Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:17.370429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:17.370418Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1778
2025-10-09T21:50:17.370444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1778
2025-10-09T21:50:17.370455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342660338374957:3008]
2025-10-09T21:50:17.370470Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:17.370490Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:17.370507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:17.370512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:17.370543Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1ec2c71b-38cc2857-c333bda0-736323a9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:17.370557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:17.370561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:17.370564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:17.370573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:17.370589Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:17.370609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:17.370756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:17.370767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:17.370785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:17.370877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1779 partition: 0 MaxSeqNo: 25 sessionId: src_id|1ec2c71b-38cc2857-c333bda0-736323a9_0
2025-10-09T21:50:17.371142Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a18632af-c6377c09-96bbdd7-ad0cb884_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1ec2c71b-38cc2857-c333bda0-736323a9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:17.373413Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ec2c71b-38cc2857-c333bda0-736323a9_0] Write session: try to update token
2025-10-09T21:50:17.373426Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ec2c71b-38cc2857-c333bda0-736323a9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:17.384795Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:17.384817Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:17.384820Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:17.384822Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:17.384831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342660338374957:3008]
2025-10-09T21:50:17.384836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342660338374957:3008]
2025-10-09T21:50:17.384840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342660338374957:3008]
2025-10-09T21:50:17.384876Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:17.384880Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:17.384883Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:17.384884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:17.384893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.384895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:17.385033Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:17.385359Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1779 sessionId: src_id|1ec2c71b-38cc2857-c333bda0-736323a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:17.387892Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:17.450463Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342660700206140:6864]
2025-10-09T21:50:17.450837Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 1778, pipe: [1:7559342660700206140:6864]
2025-10-09T21:50:17.545258Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:17.545671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:17.545686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:17.545730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342660700206140:6864], now have 1 active actors on pipe
2025-10-09T21:50:17.546009Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:17.545986Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:17.546041Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342660700206140:6864]
2025-10-09T21:50:17.546071Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:17.546079Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:17.546166Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1779 reason: error at write: Disconnected sessionId: src_id|1ec2c71b-38cc2857-c333bda0-736323a9_0
2025-10-09T21:50:17.546366Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1779 sessionId: src_id|1ec2c71b-38cc2857-c333bda0-736323a9_0 is DEAD
2025-10-09T21:50:17.546541Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ec2c71b-38cc2857-c333bda0-736323a9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:17.546552Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ec2c71b-38cc2857-c333bda0-736323a9_0] Write session will restart in 0.009572s
2025-10-09T21:50:17.546577Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ec2c71b-38cc2857-c333bda0-736323a9_0] Write session: Do CDS request
2025-10-09T21:50:17.546582Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ec2c71b-38cc2857-c333bda0-736323a9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:17.555468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:17.556920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:17.557147Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:17.557157Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:17.557170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:17.557226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:17.557268Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342660338375070:3006]
2025-10-09T21:50:17.557548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ec2c71b-38cc2857-c333bda0-736323a9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:17.557624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:17.557641Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1780
2025-10-09T21:50:17.557742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1780 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:17.557767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1780 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:17.557777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1780 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:17.557780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:17.557976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:17.558244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:17.558286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:17.558349Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1780 sessionId:  describe result for acl check
2025-10-09T21:50:17.558384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:17.558388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:17.558390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:17.558397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342660700206216:6192] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:17.558662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:17.558714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.558722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.558723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.558724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.558725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.558726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.558727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.558728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.558730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.558734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.558744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.558748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.558752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.558755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.558759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.558763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.558767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.558772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.558778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.558785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.558792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.558798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.558802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.558807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.558812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.558817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.558826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.558831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.558836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.558841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.558846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.558851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.558856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.558863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.558873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.558878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.558904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:17.559014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342660700206216:6192] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:17.560960Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342660700206216:6192] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:17.565424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342660700206216:6192] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:17.565441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342660700206216:6192] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:17.565447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342660700206216:6192] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:17.565456Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1780 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:17.566008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:17.567583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:17.567597Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:17.567599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:17.567603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:17.567610Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1779 [2:7559342660338375070:3006]
2025-10-09T21:50:17.567622Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:17.567639Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:17.567642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:17.567645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:17.567647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:17.567665Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:17.567673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:17.567684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:17.567743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.567750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.567752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.567755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:17.567819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:17.567826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.567838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.567854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.567877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:17.567901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342660700206252:6192], now have 1 active actors on pipe
2025-10-09T21:50:17.567909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342660700206142:2432], now have 1 active actors on pipe
2025-10-09T21:50:17.567916Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342660338375065:3006]
2025-10-09T21:50:17.567918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342660338375065:3006]
2025-10-09T21:50:17.567921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342660338375065:3006]
2025-10-09T21:50:17.567934Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:17.567917Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1779
2025-10-09T21:50:17.567953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:17.567965Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:17.567967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:17.567961Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1779
2025-10-09T21:50:17.567969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:17.567973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:17.567976Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:17.568011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342660338375065:3006]
2025-10-09T21:50:17.568023Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:17.568026Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:17.568040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:17.568049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:17.568101Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|401929d8-c7c64276-f1aab3a5-4a5861da_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:17.568112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:17.568116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:17.568118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:17.568128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:17.568140Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:17.568162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:17.568283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:17.568291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:17.568304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:17.568376Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1780 partition: 0 MaxSeqNo: 25 sessionId: src_id|401929d8-c7c64276-f1aab3a5-4a5861da_0
2025-10-09T21:50:17.568717Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ec2c71b-38cc2857-c333bda0-736323a9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|401929d8-c7c64276-f1aab3a5-4a5861da_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:17.573662Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|401929d8-c7c64276-f1aab3a5-4a5861da_0] Write session: try to update token
2025-10-09T21:50:17.573683Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|401929d8-c7c64276-f1aab3a5-4a5861da_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:17.584272Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:17.584289Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:17.584291Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:17.584293Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:17.584303Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342660338375065:3006]
2025-10-09T21:50:17.584308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342660338375065:3006]
2025-10-09T21:50:17.584312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342660338375065:3006]
2025-10-09T21:50:17.584360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:17.584358Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:17.584367Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:17.584370Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:17.584374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.584377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:17.584550Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:17.588750Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1780 sessionId: src_id|401929d8-c7c64276-f1aab3a5-4a5861da_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:17.591996Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:17.646094Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342660700206260:6864]
2025-10-09T21:50:17.646427Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 1779, pipe: [1:7559342660700206260:6864]
2025-10-09T21:50:17.733589Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:17.733868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:17.733880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:17.733918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342660700206260:6864], now have 1 active actors on pipe
2025-10-09T21:50:17.734103Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:17.734167Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:17.734179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:17.734239Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342660700206260:6864]
2025-10-09T21:50:17.734261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1780 reason: status is not ok: tablet will be restarted right now sessionId: src_id|401929d8-c7c64276-f1aab3a5-4a5861da_0
2025-10-09T21:50:17.734448Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1780 sessionId: src_id|401929d8-c7c64276-f1aab3a5-4a5861da_0 is DEAD
2025-10-09T21:50:17.734624Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|401929d8-c7c64276-f1aab3a5-4a5861da_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:17.734641Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|401929d8-c7c64276-f1aab3a5-4a5861da_0] Write session will restart in 0.009301s
2025-10-09T21:50:17.734672Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|401929d8-c7c64276-f1aab3a5-4a5861da_0] Write session: Do CDS request
2025-10-09T21:50:17.734679Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|401929d8-c7c64276-f1aab3a5-4a5861da_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:17.743080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:17.744758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:17.744927Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:17.744935Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:17.744943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:17.744979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:17.745007Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342660338375183:2998]
2025-10-09T21:50:17.745402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:17.745473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|401929d8-c7c64276-f1aab3a5-4a5861da_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:17.745528Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:17.745544Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1781
2025-10-09T21:50:17.745692Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1781 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:17.745717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1781 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:17.745720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1781 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:17.745724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:17.745715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:17.745760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:17.746083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:17.746135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.746140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.746142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.746143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.746134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1781 sessionId:  describe result for acl check
2025-10-09T21:50:17.746144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.746145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.746146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.746147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.746148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.746153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.746169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:17.746164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.746168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.746176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:17.746172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.746176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.746180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.746178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:17.746184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.746183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342660700206337:6188] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:17.746188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.746191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.746198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.746206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.746212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.746217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.746221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.746230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.746238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.746244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.746253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.746259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.746274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.746284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.746293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.746303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.746312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.746321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.746326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.746336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.746370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:17.746779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342660700206337:6188] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:17.748641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342660700206337:6188] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:17.752489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342660700206337:6188] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:17.752502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342660700206337:6188] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:17.752506Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342660700206337:6188] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:17.752513Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1781 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:17.752981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:17.754018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:17.754048Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:17.754050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:17.754054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:17.754061Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1780 [2:7559342660338375183:2998]
2025-10-09T21:50:17.754073Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:17.754093Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:17.754100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:17.754104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:17.754106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:17.754121Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:17.754127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:17.754137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:17.754192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.754198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.754200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.754203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:17.754263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:17.754270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.754280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.754295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.754314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:17.754339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342660700206373:6188], now have 1 active actors on pipe
2025-10-09T21:50:17.754346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342660700206262:2432], now have 1 active actors on pipe
2025-10-09T21:50:17.754354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342660338375178:2998]
2025-10-09T21:50:17.754360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342660338375178:2998]
2025-10-09T21:50:17.754362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342660338375178:2998]
2025-10-09T21:50:17.754375Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:17.754359Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1780
2025-10-09T21:50:17.754386Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:17.754368Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1780
2025-10-09T21:50:17.754394Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:17.754394Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:17.754407Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:17.754408Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:17.754411Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:17.754442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:17.754451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:17.754466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342660338375178:2998]
2025-10-09T21:50:17.754477Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:17.754484Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:17.754501Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|eeea3e49-9dcab742-b2e27d60-21bc7371_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:17.754515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:17.754519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:17.754522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:17.754532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:17.754546Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:17.754578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:17.754714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:17.754723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:17.754738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:17.754814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1781 partition: 0 MaxSeqNo: 25 sessionId: src_id|eeea3e49-9dcab742-b2e27d60-21bc7371_0
2025-10-09T21:50:17.755055Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|401929d8-c7c64276-f1aab3a5-4a5861da_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|eeea3e49-9dcab742-b2e27d60-21bc7371_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:17.757325Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eeea3e49-9dcab742-b2e27d60-21bc7371_0] Write session: try to update token
2025-10-09T21:50:17.757339Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eeea3e49-9dcab742-b2e27d60-21bc7371_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:17.768978Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1781 sessionId: src_id|eeea3e49-9dcab742-b2e27d60-21bc7371_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:17.769745Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:17.769765Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:17.769767Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:17.769771Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:17.769780Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342660338375178:2998]
2025-10-09T21:50:17.769785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342660338375178:2998]
2025-10-09T21:50:17.769789Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342660338375178:2998]
2025-10-09T21:50:17.769823Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:17.769832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:17.769833Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:17.769836Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:17.769839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.769841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:17.769976Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:17.771249Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:17.834489Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342660700206381:6864]
2025-10-09T21:50:17.834782Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 1780, pipe: [1:7559342660700206381:6864]
2025-10-09T21:50:17.908332Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:17.908598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:17.908609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:17.908643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342660700206381:6864], now have 1 active actors on pipe
2025-10-09T21:50:17.908837Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:17.908823Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:17.908833Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:17.908837Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:17.908872Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342660700206381:6864]
2025-10-09T21:50:17.908892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1781 reason: error at write: Disconnected sessionId: src_id|eeea3e49-9dcab742-b2e27d60-21bc7371_0
2025-10-09T21:50:17.909059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1781 sessionId: src_id|eeea3e49-9dcab742-b2e27d60-21bc7371_0 is DEAD
2025-10-09T21:50:17.909190Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eeea3e49-9dcab742-b2e27d60-21bc7371_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:17.909199Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eeea3e49-9dcab742-b2e27d60-21bc7371_0] Write session will restart in 0.007766s
2025-10-09T21:50:17.909218Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eeea3e49-9dcab742-b2e27d60-21bc7371_0] Write session: Do CDS request
2025-10-09T21:50:17.909223Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eeea3e49-9dcab742-b2e27d60-21bc7371_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:17.916887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:17.918435Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eeea3e49-9dcab742-b2e27d60-21bc7371_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:17.918572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:17.918577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:17.918614Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1782
2025-10-09T21:50:17.918714Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:17.918721Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:17.918729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:17.918766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:17.918747Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1782 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:17.918777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1782 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:17.918780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1782 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:17.918784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:17.918801Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342660338375331:2959]
2025-10-09T21:50:17.919124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1782 sessionId:  describe result for acl check
2025-10-09T21:50:17.919153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:17.919160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:17.919164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:17.919169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342660700206482:6173] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:17.919207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:17.919466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:17.919496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:17.919676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342660700206482:6173] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:17.919738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:17.919772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.919777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.919778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.919779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.919781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.919782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.919783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.919784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.919785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.919789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.919799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.919802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.919804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.919806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.919808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.919810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.919811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.919813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.919817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.919822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:17.919826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.919829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:17.919831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.919834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:17.919837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.919840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:17.919842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.919845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:17.919847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.919850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:17.919853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.919856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.919859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.919861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.919864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.919867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.919881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:17.921853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342660700206482:6173] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:17.925146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342660700206482:6173] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:17.925156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342660700206482:6173] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:17.925158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342660700206482:6173] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:17.925164Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1782 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:17.925565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:17.926619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:17.926632Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:17.926634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:17.926637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:17.926644Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1781 [2:7559342660338375331:2959]
2025-10-09T21:50:17.926657Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:17.926672Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:17.926679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:17.926682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:17.926684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:17.926702Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:17.926711Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:17.926718Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:17.926766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.926779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.926781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.926783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:17.926841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:17.926847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.926858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:17.926870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:17.926891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:17.926913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342660700206518:6173], now have 1 active actors on pipe
2025-10-09T21:50:17.926921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342660700206409:2432], now have 1 active actors on pipe
2025-10-09T21:50:17.926927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342660338375326:2959]
2025-10-09T21:50:17.926933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342660338375326:2959]
2025-10-09T21:50:17.926935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342660338375326:2959]
2025-10-09T21:50:17.926946Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:17.926962Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:17.926964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:17.926947Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1781
2025-10-09T21:50:17.926955Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1781
2025-10-09T21:50:17.926965Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:17.926967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:17.926978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:17.926981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:17.927014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342660338375326:2959]
2025-10-09T21:50:17.927027Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:17.927044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:17.927045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:17.927052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:17.927094Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bf6eba65-1415bf95-aeae3e7e-e885695d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:17.927105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:17.927113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:17.927116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:17.927128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:17.927141Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:17.927159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:17.927296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:17.927303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:17.927316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:17.927418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1782 partition: 0 MaxSeqNo: 25 sessionId: src_id|bf6eba65-1415bf95-aeae3e7e-e885695d_0
2025-10-09T21:50:17.927712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eeea3e49-9dcab742-b2e27d60-21bc7371_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bf6eba65-1415bf95-aeae3e7e-e885695d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:17.929983Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf6eba65-1415bf95-aeae3e7e-e885695d_0] Write session: try to update token
2025-10-09T21:50:17.929997Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf6eba65-1415bf95-aeae3e7e-e885695d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:17.940187Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:17.940204Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:17.940209Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:17.940210Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:17.940217Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342660338375326:2959]
2025-10-09T21:50:17.940220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342660338375326:2959]
2025-10-09T21:50:17.940223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342660338375326:2959]
2025-10-09T21:50:17.940256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:17.940258Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:17.940262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:17.940263Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:17.940264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:17.940266Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:17.940384Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:17.943104Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1782 sessionId: src_id|bf6eba65-1415bf95-aeae3e7e-e885695d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:17.945369Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:18.009091Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342664995173822:6864]
2025-10-09T21:50:18.009313Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 1781, pipe: [1:7559342664995173822:6864]
2025-10-09T21:50:18.085880Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:18.086255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:18.086272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:18.086322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995173822:6864], now have 1 active actors on pipe
2025-10-09T21:50:18.086553Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:18.086580Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:18.086585Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:18.086590Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:18.086649Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1782 reason: error at write: Disconnected sessionId: src_id|bf6eba65-1415bf95-aeae3e7e-e885695d_0
2025-10-09T21:50:18.086652Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342664995173822:6864]
2025-10-09T21:50:18.086829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1782 sessionId: src_id|bf6eba65-1415bf95-aeae3e7e-e885695d_0 is DEAD
2025-10-09T21:50:18.086981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf6eba65-1415bf95-aeae3e7e-e885695d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:18.086991Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf6eba65-1415bf95-aeae3e7e-e885695d_0] Write session will restart in 0.005518s
2025-10-09T21:50:18.087034Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf6eba65-1415bf95-aeae3e7e-e885695d_0] Write session: Do CDS request
2025-10-09T21:50:18.087039Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf6eba65-1415bf95-aeae3e7e-e885695d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:18.093503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf6eba65-1415bf95-aeae3e7e-e885695d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:18.093584Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:18.093603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1783
2025-10-09T21:50:18.093843Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1783 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:18.093877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1783 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:18.093884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1783 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:18.093893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:18.094346Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1783 sessionId:  describe result for acl check
2025-10-09T21:50:18.094385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:18.094391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:18.094393Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:18.094398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342664995173888:6168] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:18.094984Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342664995173888:6168] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:18.096053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:18.096878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342664995173888:6168] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:18.097580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:18.097778Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:18.097791Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:18.097804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:18.097860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:18.097899Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342664633342738:2963]
2025-10-09T21:50:18.098483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:18.098783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:18.098827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:18.099114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:18.099170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.099177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.099178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.099179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.099181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.099182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.099183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.099184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.099185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.099190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.099200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.099203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.099207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.099210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.099213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.099205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342664995173888:6168] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:18.099220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342664995173888:6168] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:18.099217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.099224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342664995173888:6168] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:18.099220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.099233Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1783 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:18.099224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.099229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.099237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.099244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.099249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.099253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.099257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.099261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.099265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.099269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.099274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.099277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.099281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.099285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.099289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.099293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.099298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.099304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.099307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.099332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:18.105729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:18.106853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:18.106868Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:18.106871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:18.106876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:18.106881Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1782 [2:7559342664633342738:2963]
2025-10-09T21:50:18.106895Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:18.106918Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:18.106926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:18.106929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:18.106930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:18.106943Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:18.106947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:18.106952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:18.107102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.107108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.107109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.107111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:18.107182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:18.107190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.107200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.107206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.107220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:18.107244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995173935:6168], now have 1 active actors on pipe
2025-10-09T21:50:18.107252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995173824:2432], now have 1 active actors on pipe
2025-10-09T21:50:18.107260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342664633342733:2963]
2025-10-09T21:50:18.107261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342664633342733:2963]
2025-10-09T21:50:18.107263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342664633342733:2963]
2025-10-09T21:50:18.107279Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:18.107301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:18.107308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:18.107291Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1782
2025-10-09T21:50:18.107311Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:18.107314Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:18.107316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:18.107303Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1782
2025-10-09T21:50:18.107318Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:18.107343Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342664633342733:2963]
2025-10-09T21:50:18.107347Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:18.107349Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:18.107362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:18.107367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:18.107419Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a0141e2c-5bc844d1-ea0d5bd8-3d258ea1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:18.107431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:18.107440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:18.107447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:18.107457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:18.107468Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:18.107493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:18.107633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:18.107643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:18.107659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:18.107731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1783 partition: 0 MaxSeqNo: 25 sessionId: src_id|a0141e2c-5bc844d1-ea0d5bd8-3d258ea1_0
2025-10-09T21:50:18.108044Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf6eba65-1415bf95-aeae3e7e-e885695d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a0141e2c-5bc844d1-ea0d5bd8-3d258ea1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:18.110514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a0141e2c-5bc844d1-ea0d5bd8-3d258ea1_0] Write session: try to update token
2025-10-09T21:50:18.110532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a0141e2c-5bc844d1-ea0d5bd8-3d258ea1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:18.121614Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:18.121633Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:18.121635Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:18.121638Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:18.121648Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342664633342733:2963]
2025-10-09T21:50:18.121653Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342664633342733:2963]
2025-10-09T21:50:18.121658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342664633342733:2963]
2025-10-09T21:50:18.121695Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:18.121699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:18.121701Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:18.121707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.121710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:18.121711Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:18.121837Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:18.123415Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1783 sessionId: src_id|a0141e2c-5bc844d1-ea0d5bd8-3d258ea1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:18.125838Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:18.186737Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342664995173945:6864]
2025-10-09T21:50:18.186997Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 1782, pipe: [1:7559342664995173945:6864]
2025-10-09T21:50:18.271204Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:18.271422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:18.271433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:18.271472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995173945:6864], now have 1 active actors on pipe
2025-10-09T21:50:18.271635Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:18.271693Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342664995173945:6864]
2025-10-09T21:50:18.271698Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:18.271708Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:18.271727Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:18.271794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1783 reason: error at write: Disconnected sessionId: src_id|a0141e2c-5bc844d1-ea0d5bd8-3d258ea1_0
2025-10-09T21:50:18.271964Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1783 sessionId: src_id|a0141e2c-5bc844d1-ea0d5bd8-3d258ea1_0 is DEAD
2025-10-09T21:50:18.272104Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0141e2c-5bc844d1-ea0d5bd8-3d258ea1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:18.272124Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0141e2c-5bc844d1-ea0d5bd8-3d258ea1_0] Write session will restart in 0.006074s
2025-10-09T21:50:18.272145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0141e2c-5bc844d1-ea0d5bd8-3d258ea1_0] Write session: Do CDS request
2025-10-09T21:50:18.272150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0141e2c-5bc844d1-ea0d5bd8-3d258ea1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:18.279420Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a0141e2c-5bc844d1-ea0d5bd8-3d258ea1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:18.279489Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:18.279501Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1784
2025-10-09T21:50:18.279624Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1784 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:18.279649Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1784 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:18.279658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1784 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:18.279671Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:18.280030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:18.280059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1784 sessionId:  describe result for acl check
2025-10-09T21:50:18.280094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:18.280100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:18.280102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:18.280108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342664995174030:6164] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:18.280652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342664995174030:6164] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:18.281798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:18.281971Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:18.281982Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:18.281992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:18.282059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:18.282100Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342664633342850:2961]
2025-10-09T21:50:18.282559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:18.282740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342664995174030:6164] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:18.282817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:18.282851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:18.283151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:18.283198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.283206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.283207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.283208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.283209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.283210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.283211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.283213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.283214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.283218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.283229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.283233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.283237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.283241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.283245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.283249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.283253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.283257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.283263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.283271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.283277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.283282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.283287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.283291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.283295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.283304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.283313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.283318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.283327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.283333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.283341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.283346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.283351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.283356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.283364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.283370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.283398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:18.288597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342664995174030:6164] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:18.288612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342664995174030:6164] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:18.288616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342664995174030:6164] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:18.288623Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1784 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:18.289422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:18.290445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:18.290460Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:18.290462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:18.290466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:18.290472Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1783 [2:7559342664633342850:2961]
2025-10-09T21:50:18.290488Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:18.290505Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:18.290512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:18.290515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:18.290517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:18.290534Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:18.290543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:18.290550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:18.290606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.290614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.290616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.290619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:18.290681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:18.290688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.290699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.290716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.290738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:18.290765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995174070:6164], now have 1 active actors on pipe
2025-10-09T21:50:18.290773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995173955:2432], now have 1 active actors on pipe
2025-10-09T21:50:18.290779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342664633342845:2961]
2025-10-09T21:50:18.290785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342664633342845:2961]
2025-10-09T21:50:18.290787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342664633342845:2961]
2025-10-09T21:50:18.290785Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1783
2025-10-09T21:50:18.290800Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:18.290804Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1783
2025-10-09T21:50:18.290819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:18.290819Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:18.290822Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:18.290824Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:18.290837Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:18.290845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:18.290861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:18.290869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:18.290904Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342664633342845:2961]
2025-10-09T21:50:18.290915Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:18.290924Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:18.290937Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3d3439b8-d42f4dd7-49a1710c-ea41a074_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:18.290945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:18.290955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:18.290961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:18.290970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:18.290983Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:18.291018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:18.291126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:18.291133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:18.291146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:18.291203Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1784 partition: 0 MaxSeqNo: 25 sessionId: src_id|3d3439b8-d42f4dd7-49a1710c-ea41a074_0
2025-10-09T21:50:18.291465Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a0141e2c-5bc844d1-ea0d5bd8-3d258ea1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3d3439b8-d42f4dd7-49a1710c-ea41a074_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:18.293792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3439b8-d42f4dd7-49a1710c-ea41a074_0] Write session: try to update token
2025-10-09T21:50:18.293808Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3439b8-d42f4dd7-49a1710c-ea41a074_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:18.304862Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:18.304880Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:18.304882Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:18.304885Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:18.304894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342664633342845:2961]
2025-10-09T21:50:18.304899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342664633342845:2961]
2025-10-09T21:50:18.304903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342664633342845:2961]
2025-10-09T21:50:18.304944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:18.304943Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:18.304949Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:18.304952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.304952Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:18.304955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:18.305158Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:18.306334Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1784 sessionId: src_id|3d3439b8-d42f4dd7-49a1710c-ea41a074_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:18.308626Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:18.372089Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342664995174077:6864]
2025-10-09T21:50:18.372440Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 1783, pipe: [1:7559342664995174077:6864]
2025-10-09T21:50:18.451188Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:18.451506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:18.451517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:18.451554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995174077:6864], now have 1 active actors on pipe
2025-10-09T21:50:18.451721Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:18.451758Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:18.451774Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:18.451785Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:18.451842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1784 reason: error at write: Disconnected sessionId: src_id|3d3439b8-d42f4dd7-49a1710c-ea41a074_0
2025-10-09T21:50:18.451843Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342664995174077:6864]
2025-10-09T21:50:18.451980Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1784 sessionId: src_id|3d3439b8-d42f4dd7-49a1710c-ea41a074_0 is DEAD
2025-10-09T21:50:18.452123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3439b8-d42f4dd7-49a1710c-ea41a074_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:18.452132Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3439b8-d42f4dd7-49a1710c-ea41a074_0] Write session will restart in 0.005716s
2025-10-09T21:50:18.452160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3439b8-d42f4dd7-49a1710c-ea41a074_0] Write session: Do CDS request
2025-10-09T21:50:18.452166Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3439b8-d42f4dd7-49a1710c-ea41a074_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:18.458464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3439b8-d42f4dd7-49a1710c-ea41a074_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:18.458576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:18.458590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1785
2025-10-09T21:50:18.458710Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1785 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:18.458752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1785 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:18.458762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1785 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:18.458767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:18.459265Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1785 sessionId:  describe result for acl check
2025-10-09T21:50:18.459302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:18.459308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:18.459310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:18.459315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342664995174139:6154] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:18.459871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342664995174139:6154] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:18.460729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:18.461816Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342664995174139:6154] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:18.462464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:18.462638Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:18.462647Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:18.462658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:18.462704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:18.462745Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342664633342963:2952]
2025-10-09T21:50:18.463312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:18.463536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:18.463587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:18.463903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:18.463961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.463967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.463968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.463969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.463970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.463971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.463973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.463974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.463975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.463980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.463991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.463999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.464002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.464005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.464009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.464012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.464021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.464025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.464033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.464040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.464055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.464060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.464064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.464069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.464073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.464077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.464082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.464086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.464090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.464094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.464100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.464105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.464110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.464114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.464119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.464129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.464159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:18.464265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342664995174139:6154] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:18.464278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342664995174139:6154] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:18.464281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342664995174139:6154] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:18.464289Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1785 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:18.478151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:18.479299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:18.479314Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:18.479317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:18.479321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:18.479326Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1784 [2:7559342664633342963:2952]
2025-10-09T21:50:18.479339Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:18.479359Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:18.479365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:18.479368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:18.479370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:18.479383Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:18.479387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:18.479393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:18.479442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.479448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.479451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.479454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:18.479517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:18.479524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.479536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.479543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.479553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:18.479568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995174080:2432], now have 1 active actors on pipe
2025-10-09T21:50:18.479576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995174194:6154], now have 1 active actors on pipe
2025-10-09T21:50:18.479584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342664633342958:2952]
2025-10-09T21:50:18.479591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342664633342958:2952]
2025-10-09T21:50:18.479594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342664633342958:2952]
2025-10-09T21:50:18.479603Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:18.479619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:18.479620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:18.479624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:18.479627Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:18.479634Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:18.479638Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:18.479635Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1784
2025-10-09T21:50:18.479665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342664633342958:2952]
2025-10-09T21:50:18.479642Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1784
2025-10-09T21:50:18.479669Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:18.479672Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:18.479685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:18.479689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:18.479834Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|60b385a1-773202bb-ca961166-8cf27202_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:18.479850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:18.479853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:18.479855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:18.479864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:18.479871Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:18.479889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:18.480015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:18.480021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:18.480031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:18.480104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1785 partition: 0 MaxSeqNo: 25 sessionId: src_id|60b385a1-773202bb-ca961166-8cf27202_0
2025-10-09T21:50:18.480405Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d3439b8-d42f4dd7-49a1710c-ea41a074_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|60b385a1-773202bb-ca961166-8cf27202_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:18.482699Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|60b385a1-773202bb-ca961166-8cf27202_0] Write session: try to update token
2025-10-09T21:50:18.482713Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|60b385a1-773202bb-ca961166-8cf27202_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:18.496564Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:18.496578Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:18.496580Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:18.496581Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:18.496590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342664633342958:2952]
2025-10-09T21:50:18.496593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342664633342958:2952]
2025-10-09T21:50:18.496610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342664633342958:2952]
2025-10-09T21:50:18.496644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:18.496649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.496651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:18.496650Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:18.496657Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:18.496661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:18.496773Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:18.499597Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1785 sessionId: src_id|60b385a1-773202bb-ca961166-8cf27202_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:18.501804Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:18.551833Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342664995174202:6864]
2025-10-09T21:50:18.552089Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 1784, pipe: [1:7559342664995174202:6864]
2025-10-09T21:50:18.633990Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:18.634268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:18.634280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:18.634317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995174202:6864], now have 1 active actors on pipe
2025-10-09T21:50:18.634506Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:18.634537Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:18.634552Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:18.634552Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342664995174202:6864]
2025-10-09T21:50:18.634571Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:18.634619Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1785 reason: error at write: Disconnected sessionId: src_id|60b385a1-773202bb-ca961166-8cf27202_0
2025-10-09T21:50:18.634789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1785 sessionId: src_id|60b385a1-773202bb-ca961166-8cf27202_0 is DEAD
2025-10-09T21:50:18.634914Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60b385a1-773202bb-ca961166-8cf27202_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:18.634923Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60b385a1-773202bb-ca961166-8cf27202_0] Write session will restart in 0.009963s
2025-10-09T21:50:18.634946Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60b385a1-773202bb-ca961166-8cf27202_0] Write session: Do CDS request
2025-10-09T21:50:18.634952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60b385a1-773202bb-ca961166-8cf27202_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:18.645630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:18.646512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|60b385a1-773202bb-ca961166-8cf27202_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:18.646587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:18.646604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1786
2025-10-09T21:50:18.646728Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1786 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:18.646764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1786 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:18.646778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1786 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:18.646786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:18.647230Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1786 sessionId:  describe result for acl check
2025-10-09T21:50:18.647262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:18.647264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:18.647266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:18.647273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342664995174283:6149] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:18.647451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:18.647628Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:18.647638Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:18.647651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:18.647708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:18.647748Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342664633343078:2932]
2025-10-09T21:50:18.647906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342664995174283:6149] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:18.648315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:18.648562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:18.648604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:18.648902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:18.648958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.648965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.648967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.648968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.648969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.648970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.648971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.648972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.648973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.648978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.648988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.648992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.648996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.649000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.649004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.649008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.649012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.649016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.649022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.649030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.649036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.649041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.649046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.649052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.649056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.649060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.649065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.649070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.649075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.649085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.649094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.649099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.649103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.649109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.649117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.649123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.649156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:18.649811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342664995174283:6149] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:18.656276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342664995174283:6149] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:18.656290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342664995174283:6149] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:18.656292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342664995174283:6149] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:18.656299Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1786 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:18.656809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:18.657902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:18.657914Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:18.657916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:18.657918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:18.657924Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1785 [2:7559342664633343078:2932]
2025-10-09T21:50:18.657935Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:18.657948Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:18.657949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:18.657952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:18.657953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:18.657964Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:18.657968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:18.657974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:18.658015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.658019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.658020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.658022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:18.658087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:18.658095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.658104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.658114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.658123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:18.658142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995174321:6149], now have 1 active actors on pipe
2025-10-09T21:50:18.658151Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342664633343072:2932]
2025-10-09T21:50:18.658152Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342664633343072:2932]
2025-10-09T21:50:18.658154Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342664633343072:2932]
2025-10-09T21:50:18.658163Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:18.658173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995174205:2432], now have 1 active actors on pipe
2025-10-09T21:50:18.658178Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:18.658180Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:18.658181Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:18.658183Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:18.658188Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:18.658190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:18.658194Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1785
2025-10-09T21:50:18.658210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342664633343072:2932]
2025-10-09T21:50:18.658211Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1785
2025-10-09T21:50:18.658218Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:18.658220Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:18.658237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:18.658241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:18.658265Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7fe6907-4e3069c6-895066fc-79bb04b0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:18.658274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:18.658279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:18.658280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:18.658288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:18.658299Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:18.658312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:18.658469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:18.658484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:18.658508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:18.658603Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1786 partition: 0 MaxSeqNo: 25 sessionId: src_id|7fe6907-4e3069c6-895066fc-79bb04b0_0
2025-10-09T21:50:18.658917Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60b385a1-773202bb-ca961166-8cf27202_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7fe6907-4e3069c6-895066fc-79bb04b0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:18.661193Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7fe6907-4e3069c6-895066fc-79bb04b0_0] Write session: try to update token
2025-10-09T21:50:18.661207Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7fe6907-4e3069c6-895066fc-79bb04b0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:18.674272Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1786 sessionId: src_id|7fe6907-4e3069c6-895066fc-79bb04b0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:18.676383Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:18.676400Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:18.676402Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:18.676404Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:18.676413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342664633343072:2932]
2025-10-09T21:50:18.676419Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342664633343072:2932]
2025-10-09T21:50:18.676423Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342664633343072:2932]
2025-10-09T21:50:18.676466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:18.676466Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:18.676477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.676480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:18.676484Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:18.676499Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:18.676546Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:18.676645Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:18.734585Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342664995174328:6864]
2025-10-09T21:50:18.734839Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 1785, pipe: [1:7559342664995174328:6864]
2025-10-09T21:50:18.816690Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:18.816970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:18.816984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:18.817025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995174328:6864], now have 1 active actors on pipe
2025-10-09T21:50:18.817227Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:18.817245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:18.817259Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:18.817263Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342664995174328:6864]
2025-10-09T21:50:18.817281Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:18.817315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1786 reason: error at write: Disconnected sessionId: src_id|7fe6907-4e3069c6-895066fc-79bb04b0_0
2025-10-09T21:50:18.817495Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1786 sessionId: src_id|7fe6907-4e3069c6-895066fc-79bb04b0_0 is DEAD
2025-10-09T21:50:18.817651Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fe6907-4e3069c6-895066fc-79bb04b0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:18.817660Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fe6907-4e3069c6-895066fc-79bb04b0_0] Write session will restart in 0.009684s
2025-10-09T21:50:18.817681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fe6907-4e3069c6-895066fc-79bb04b0_0] Write session: Do CDS request
2025-10-09T21:50:18.817686Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fe6907-4e3069c6-895066fc-79bb04b0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:18.826076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:18.827549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:18.827716Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:18.827725Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:18.827736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:18.827783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:18.827823Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342664633343196:2944]
2025-10-09T21:50:18.828377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:18.828437Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7fe6907-4e3069c6-895066fc-79bb04b0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:18.828542Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:18.828558Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1787
2025-10-09T21:50:18.828659Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1787 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:18.828681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:18.828688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1787 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:18.828697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1787 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:18.828701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:18.828725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:18.829021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:18.829067Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1787 sessionId:  describe result for acl check
2025-10-09T21:50:18.829064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.829070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.829071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.829072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.829074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.829075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.829076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.829077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.829078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.829082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.829099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:18.829092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.829096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.829105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:18.829099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.829106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:18.829112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342664995174412:6145] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:18.829103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.829107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.829111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.829114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.829118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.829125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.829132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:18.829138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.829143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:18.829150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.829155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:18.829159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.829164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:18.829170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.829179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:18.829186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.829191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:18.829199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.829208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.829216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.829221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.829229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.829234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.829267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:18.829634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342664995174412:6145] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:18.831339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342664995174412:6145] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:18.835556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342664995174412:6145] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:18.835570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342664995174412:6145] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:18.835574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342664995174412:6145] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:18.835581Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1787 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:18.836149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:18.837144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:18.837156Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:18.837159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:18.837162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:18.837169Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1786 [2:7559342664633343196:2944]
2025-10-09T21:50:18.837180Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:18.837193Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:18.837196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:18.837199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:18.837201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:18.837215Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:18.837223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:18.837229Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:18.837280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.837286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.837288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.837291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:18.837351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:18.837358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.837369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:18.837384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:18.837405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:18.837429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995174448:6145], now have 1 active actors on pipe
2025-10-09T21:50:18.837436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995174331:2432], now have 1 active actors on pipe
2025-10-09T21:50:18.837442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342664633343191:2944]
2025-10-09T21:50:18.837448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342664633343191:2944]
2025-10-09T21:50:18.837450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342664633343191:2944]
2025-10-09T21:50:18.837462Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:18.837453Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1786
2025-10-09T21:50:18.837480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:18.837456Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1786
2025-10-09T21:50:18.837497Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:18.837501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:18.837503Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:18.837510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:18.837514Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:18.837539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342664633343191:2944]
2025-10-09T21:50:18.837550Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:18.837567Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:18.837568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:18.837573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:18.837608Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e1cbfe1a-6f0aa21d-32e952ba-d9f9925_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:18.837619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:18.837626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:18.837628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:18.837638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:18.837650Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:18.837670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:18.837805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:18.837814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:18.837829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:18.837910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1787 partition: 0 MaxSeqNo: 25 sessionId: src_id|e1cbfe1a-6f0aa21d-32e952ba-d9f9925_0
2025-10-09T21:50:18.838185Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fe6907-4e3069c6-895066fc-79bb04b0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e1cbfe1a-6f0aa21d-32e952ba-d9f9925_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:18.840427Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1cbfe1a-6f0aa21d-32e952ba-d9f9925_0] Write session: try to update token
2025-10-09T21:50:18.840440Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1cbfe1a-6f0aa21d-32e952ba-d9f9925_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:18.852260Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1787 sessionId: src_id|e1cbfe1a-6f0aa21d-32e952ba-d9f9925_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:18.853975Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:18.853989Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:18.853991Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:18.853992Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:18.854000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342664633343191:2944]
2025-10-09T21:50:18.854004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342664633343191:2944]
2025-10-09T21:50:18.854007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342664633343191:2944]
2025-10-09T21:50:18.854064Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:18.854065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:18.854067Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:18.854074Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:18.854075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:18.854078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:18.854220Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:18.854595Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:18.917365Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342664995174470:6864]
2025-10-09T21:50:18.917583Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 1786, pipe: [1:7559342664995174470:6864]
2025-10-09T21:50:18.992092Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:18.992323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:18.992334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:18.992372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995174470:6864], now have 1 active actors on pipe
2025-10-09T21:50:18.992619Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342664995174470:6864]
2025-10-09T21:50:18.992638Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:18.992624Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:18.992628Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:18.992657Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:18.992705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1787 reason: error at write: Disconnected sessionId: src_id|e1cbfe1a-6f0aa21d-32e952ba-d9f9925_0
2025-10-09T21:50:18.992955Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1787 sessionId: src_id|e1cbfe1a-6f0aa21d-32e952ba-d9f9925_0 is DEAD
2025-10-09T21:50:18.993046Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1cbfe1a-6f0aa21d-32e952ba-d9f9925_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:18.993056Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1cbfe1a-6f0aa21d-32e952ba-d9f9925_0] Write session will restart in 0.009761s
2025-10-09T21:50:18.993074Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1cbfe1a-6f0aa21d-32e952ba-d9f9925_0] Write session: Do CDS request
2025-10-09T21:50:18.993079Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1cbfe1a-6f0aa21d-32e952ba-d9f9925_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:19.001711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:19.003359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:19.003482Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1cbfe1a-6f0aa21d-32e952ba-d9f9925_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:19.003546Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:19.003556Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:19.003568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:19.003624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:19.003662Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342668928310647:2912]
2025-10-09T21:50:19.003812Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:19.003826Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1788
2025-10-09T21:50:19.003963Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1788 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:19.003997Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1788 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:19.004007Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1788 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:19.004015Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:19.004251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:19.004422Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1788 sessionId:  describe result for acl check
2025-10-09T21:50:19.004465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:19.004473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:19.004475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:19.004481Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342669290141860:6128] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:19.004516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:19.004561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:19.004868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:19.004917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.004922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.004924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.004925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.004926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.004927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.004928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.004929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.004931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.004936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.004946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.004950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.004954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.004958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.004962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.004966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.004970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.004974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.004980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.004989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.004995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.005000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.005005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.005010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.005014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.005019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.005024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.005029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.005022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342669290141860:6128] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:19.005033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.005039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.005043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.005049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.005053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.005058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.005063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.005068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.005099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:19.007074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342669290141860:6128] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:19.012428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342669290141860:6128] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:19.012443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342669290141860:6128] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:19.012448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342669290141860:6128] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:19.012455Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1788 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:19.012948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:19.014025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:19.014054Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:19.014057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:19.014061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:19.014068Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1787 [2:7559342668928310647:2912]
2025-10-09T21:50:19.014081Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:19.014099Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:19.014107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:19.014110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:19.014112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:19.014130Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:19.014140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:19.014148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:19.014209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.014214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.014216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.014220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:19.014290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:19.014299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.014312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.014329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.014352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:19.014379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342664995174481:2432], now have 1 active actors on pipe
2025-10-09T21:50:19.014387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342669290141896:6128], now have 1 active actors on pipe
2025-10-09T21:50:19.014394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342668928310642:2912]
2025-10-09T21:50:19.014396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342668928310642:2912]
2025-10-09T21:50:19.014399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342668928310642:2912]
2025-10-09T21:50:19.014381Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1787
2025-10-09T21:50:19.014408Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:19.014385Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1787
2025-10-09T21:50:19.014428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:19.014434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:19.014437Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:19.014438Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:19.014444Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:19.014447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:19.014506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.014515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.014528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342668928310642:2912]
2025-10-09T21:50:19.014540Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:19.014544Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:19.014574Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a11b07ac-b9bf5a98-86cb4659-64a824d9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:19.014587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:19.014598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:19.014601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:19.014611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:19.014622Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:19.014653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:19.014850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.014860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.014875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:19.014948Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1788 partition: 0 MaxSeqNo: 25 sessionId: src_id|a11b07ac-b9bf5a98-86cb4659-64a824d9_0
2025-10-09T21:50:19.015308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1cbfe1a-6f0aa21d-32e952ba-d9f9925_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a11b07ac-b9bf5a98-86cb4659-64a824d9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:19.017654Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a11b07ac-b9bf5a98-86cb4659-64a824d9_0] Write session: try to update token
2025-10-09T21:50:19.017668Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a11b07ac-b9bf5a98-86cb4659-64a824d9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:19.030358Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1788 sessionId: src_id|a11b07ac-b9bf5a98-86cb4659-64a824d9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:19.031986Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:19.032004Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:19.032005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:19.032007Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:19.032014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342668928310642:2912]
2025-10-09T21:50:19.032017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342668928310642:2912]
2025-10-09T21:50:19.032021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342668928310642:2912]
2025-10-09T21:50:19.032057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:19.032060Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:19.032062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.032064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:19.032065Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:19.032068Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:19.032187Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:19.032815Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:19.092990Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342669290141903:6864]
2025-10-09T21:50:19.093231Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 1787, pipe: [1:7559342669290141903:6864]
2025-10-09T21:50:19.178854Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:19.179100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.179114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.179154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342669290141903:6864], now have 1 active actors on pipe
2025-10-09T21:50:19.179340Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:19.179397Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:19.179402Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342669290141903:6864]
2025-10-09T21:50:19.179427Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:19.179505Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1788 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a11b07ac-b9bf5a98-86cb4659-64a824d9_0
2025-10-09T21:50:19.179698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1788 sessionId: src_id|a11b07ac-b9bf5a98-86cb4659-64a824d9_0 is DEAD
2025-10-09T21:50:19.179852Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a11b07ac-b9bf5a98-86cb4659-64a824d9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:19.179864Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a11b07ac-b9bf5a98-86cb4659-64a824d9_0] Write session will restart in 0.009068s
2025-10-09T21:50:19.179894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a11b07ac-b9bf5a98-86cb4659-64a824d9_0] Write session: Do CDS request
2025-10-09T21:50:19.179902Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a11b07ac-b9bf5a98-86cb4659-64a824d9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:19.190417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:19.190601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a11b07ac-b9bf5a98-86cb4659-64a824d9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:19.190639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:19.190657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1789
2025-10-09T21:50:19.190836Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1789 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:19.190869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1789 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:19.190880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1789 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:19.190884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:19.191327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1789 sessionId:  describe result for acl check
2025-10-09T21:50:19.191377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:19.191385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:19.191387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:19.191393Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342669290141992:6122] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:19.191879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342669290141992:6122] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:19.192304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:19.192501Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:19.192510Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:19.192522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:19.192566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:19.192596Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342668928310764:2921]
2025-10-09T21:50:19.193001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:19.193253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:19.193281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:19.193530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:19.193579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.193584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.193584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.193585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.193586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.193587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.193587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.193588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.193589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.193592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.193602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.193605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.193609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.193613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.193617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.193624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342669290141992:6122] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:19.193622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.193626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.193629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.193636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.193643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.193649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.193655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.193660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.193665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.193670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.193674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.193676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.193679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.193682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.193685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.193688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.193691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.193694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.193697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.193700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.193703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.193719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:19.199869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342669290141992:6122] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:19.199892Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342669290141992:6122] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:19.199897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342669290141992:6122] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:19.199907Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1789 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:19.201598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:19.202650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:19.202662Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:19.202665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:19.202668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:19.202674Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1788 [2:7559342668928310764:2921]
2025-10-09T21:50:19.202687Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:19.202705Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:19.202707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:19.202709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:19.202711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:19.202726Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:19.202734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:19.202740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:19.202785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.202790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.202791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.202794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:19.202850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:19.202855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.202863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.202874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.202888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:19.202908Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342668928310758:2921]
2025-10-09T21:50:19.202913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342668928310758:2921]
2025-10-09T21:50:19.202915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342668928310758:2921]
2025-10-09T21:50:19.202922Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:19.202934Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:19.202935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342669290141908:2432], now have 1 active actors on pipe
2025-10-09T21:50:19.202938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342669290142030:6122], now have 1 active actors on pipe
2025-10-09T21:50:19.202945Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:19.202946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:19.202948Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:19.202954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:19.202956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:19.202988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342668928310758:2921]
2025-10-09T21:50:19.202998Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:19.202996Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1788
2025-10-09T21:50:19.203023Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:19.203001Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1788
2025-10-09T21:50:19.203057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.203064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.203087Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4ad54908-2f1ebf27-5767e45d-30a9a597_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:19.203096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:19.203099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:19.203101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:19.203108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:19.203115Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:19.203125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:19.203289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.203307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.203326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:19.203458Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1789 partition: 0 MaxSeqNo: 25 sessionId: src_id|4ad54908-2f1ebf27-5767e45d-30a9a597_0
2025-10-09T21:50:19.203813Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a11b07ac-b9bf5a98-86cb4659-64a824d9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4ad54908-2f1ebf27-5767e45d-30a9a597_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:19.206362Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ad54908-2f1ebf27-5767e45d-30a9a597_0] Write session: try to update token
2025-10-09T21:50:19.206376Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ad54908-2f1ebf27-5767e45d-30a9a597_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:19.218448Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:19.218465Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:19.218472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342668928310758:2921]
2025-10-09T21:50:19.218522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:19.218530Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:19.218677Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:19.218828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:19.218952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:19.219124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:19.219241Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1789 sessionId: src_id|4ad54908-2f1ebf27-5767e45d-30a9a597_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:19.219308Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:19.219347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:19.221732Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:19.221913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.221927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.221958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:19.222052Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:19.229515Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:19.229528Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:19.229530Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:19.229531Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:19.229539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342668928310758:2921]
2025-10-09T21:50:19.229542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342668928310758:2921]
2025-10-09T21:50:19.229545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342668928310758:2921]
2025-10-09T21:50:19.229578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:19.229584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.229585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:19.229593Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:19.229599Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:19.229603Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:19.229727Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:19.279316Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342669290142042:6864]
2025-10-09T21:50:19.279566Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 1788, pipe: [1:7559342669290142042:6864]
2025-10-09T21:50:19.374134Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:19.380736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.380758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.380916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:19.380981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:19.381055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:19.381137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:19.381444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:19.381458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:19.381529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:19.381583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:19.381642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:19.381701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:19.381739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:19.381745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:19.381799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:19.381854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:19.381937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:19.381993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:19.382045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:19.382053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:19.382110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:19.382168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:19.382225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:19.382280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:19.382318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:19.382324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:19.382379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:19.382528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:19.382592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:19.382648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:19.382685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:19.382690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:19.382747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:19.382799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:19.382856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:19.382911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:19.382950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:19.382957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:19.383012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:19.383066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:19.383123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:19.383179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:19.383217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:19.383224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:19.383276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:19.383331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:19.383386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:19.383497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:19.383549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:19.383554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:19.383663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:19.383779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:19.383836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:19.383891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:19.383931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:19.383933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:19.383987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:19.384040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:19.384096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:19.384153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:19.384191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:19.384197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:19.384236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342669290142042:6864], now have 1 active actors on pipe
2025-10-09T21:50:19.384497Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342669290142042:6864]
2025-10-09T21:50:19.384519Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:19.384537Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:19.384540Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:19.384571Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1789 reason: error at write: Disconnected sessionId: src_id|4ad54908-2f1ebf27-5767e45d-30a9a597_0
2025-10-09T21:50:19.384743Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:19.384749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1789 sessionId: src_id|4ad54908-2f1ebf27-5767e45d-30a9a597_0 is DEAD
2025-10-09T21:50:19.384903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ad54908-2f1ebf27-5767e45d-30a9a597_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:19.384915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ad54908-2f1ebf27-5767e45d-30a9a597_0] Write session will restart in 0.005302s
2025-10-09T21:50:19.384933Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ad54908-2f1ebf27-5767e45d-30a9a597_0] Write session: Do CDS request
2025-10-09T21:50:19.384937Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ad54908-2f1ebf27-5767e45d-30a9a597_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:19.391392Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ad54908-2f1ebf27-5767e45d-30a9a597_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:19.391516Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:19.391531Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1790
2025-10-09T21:50:19.391634Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1790 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:19.391667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1790 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:19.391672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1790 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:19.391678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:19.392079Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1790 sessionId:  describe result for acl check
2025-10-09T21:50:19.392116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:19.392123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:19.392126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:19.392131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342669290142097:6116] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:19.392845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342669290142097:6116] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:19.394200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:19.395419Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342669290142097:6116] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:19.395852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:19.395993Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:19.396001Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:19.396019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:19.396057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:19.396086Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342668928310883:2922]
2025-10-09T21:50:19.396498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:19.396866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:19.396904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:19.397211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:19.397255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.397259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.397260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.397261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.397262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.397263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.397264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.397265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.397266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.397271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.397280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.397285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.397289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.397293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.397297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.397301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.397304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.397308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.397314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.397321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.397328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.397333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.397337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.397342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.397347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.397352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.397359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.397364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.397368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.397373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.397380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.397385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.397389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.397395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.397402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.397406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.397437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:19.398282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342669290142097:6116] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:19.398297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342669290142097:6116] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:19.398301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342669290142097:6116] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:19.398310Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1790 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:19.404295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:19.405282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:19.405297Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:19.405299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:19.405303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:19.405309Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1789 [2:7559342668928310883:2922]
2025-10-09T21:50:19.405322Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:19.405341Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:19.405347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:19.405350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:19.405357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:19.405374Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:19.405384Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:19.405391Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:19.405443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.405452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.405454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.405457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:19.405539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:19.405546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.405557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.405567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.405585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:19.405610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342669290142045:2432], now have 1 active actors on pipe
2025-10-09T21:50:19.405613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342669290142165:6116], now have 1 active actors on pipe
2025-10-09T21:50:19.405620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342668928310877:2922]
2025-10-09T21:50:19.405622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342668928310877:2922]
2025-10-09T21:50:19.405625Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342668928310877:2922]
2025-10-09T21:50:19.405638Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:19.405653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:19.405663Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:19.405666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:19.405668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:19.405669Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:19.405671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:19.405685Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1789
2025-10-09T21:50:19.405698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342668928310877:2922]
2025-10-09T21:50:19.405695Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1789
2025-10-09T21:50:19.405707Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:19.405716Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:19.405741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.405746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.405781Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|88d92dc0-7818b8a6-95fa4a59-2003c527_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:19.405792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:19.405796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:19.405798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:19.405807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:19.405821Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:19.405838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:19.406003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.406011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.406024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:19.406160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1790 partition: 0 MaxSeqNo: 25 sessionId: src_id|88d92dc0-7818b8a6-95fa4a59-2003c527_0
2025-10-09T21:50:19.406520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ad54908-2f1ebf27-5767e45d-30a9a597_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|88d92dc0-7818b8a6-95fa4a59-2003c527_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:19.408756Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88d92dc0-7818b8a6-95fa4a59-2003c527_0] Write session: try to update token
2025-10-09T21:50:19.408771Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88d92dc0-7818b8a6-95fa4a59-2003c527_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:19.420585Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1790 sessionId: src_id|88d92dc0-7818b8a6-95fa4a59-2003c527_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:19.421546Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:19.421562Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:19.421565Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:19.421567Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:19.421574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342668928310877:2922]
2025-10-09T21:50:19.421579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342668928310877:2922]
2025-10-09T21:50:19.421582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342668928310877:2922]
2025-10-09T21:50:19.421624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:19.421632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.421635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:19.421768Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:19.421767Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:19.421772Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:19.421776Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:19.423224Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:19.484680Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342669290142173:6864]
2025-10-09T21:50:19.484974Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 1789, pipe: [1:7559342669290142173:6864]
2025-10-09T21:50:19.560969Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:19.561251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.561264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.561301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342669290142173:6864], now have 1 active actors on pipe
2025-10-09T21:50:19.561546Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:19.561544Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342669290142173:6864]
2025-10-09T21:50:19.561573Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:19.561578Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:19.561586Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:19.561652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1790 reason: error at write: Disconnected sessionId: src_id|88d92dc0-7818b8a6-95fa4a59-2003c527_0
2025-10-09T21:50:19.561859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1790 sessionId: src_id|88d92dc0-7818b8a6-95fa4a59-2003c527_0 is DEAD
2025-10-09T21:50:19.562005Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88d92dc0-7818b8a6-95fa4a59-2003c527_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:19.562026Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88d92dc0-7818b8a6-95fa4a59-2003c527_0] Write session will restart in 0.006346s
2025-10-09T21:50:19.562062Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88d92dc0-7818b8a6-95fa4a59-2003c527_0] Write session: Do CDS request
2025-10-09T21:50:19.562068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88d92dc0-7818b8a6-95fa4a59-2003c527_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:19.569454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88d92dc0-7818b8a6-95fa4a59-2003c527_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:19.569529Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:19.569541Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1791
2025-10-09T21:50:19.569649Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1791 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:19.569672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1791 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:19.569682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1791 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:19.569684Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:19.570103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1791 sessionId:  describe result for acl check
2025-10-09T21:50:19.570137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:19.570139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:19.570141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:19.570146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342669290142247:6109] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:19.570683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342669290142247:6109] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:19.571623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:19.572481Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342669290142247:6109] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:19.573345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:19.573502Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:19.573509Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:19.573518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:19.573554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:19.573582Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342668928311002:2920]
2025-10-09T21:50:19.573918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:19.574215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:19.574243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:19.574500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:19.574547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.574553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.574554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.574555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.574556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.574557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.574558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.574559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.574560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.574564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.574572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.574576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.574579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.574582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.574586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.574589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.574592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.574595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.574600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.574607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.574613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.574618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.574622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.574627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.574631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.574636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.574641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.574646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.574649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.574656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.574659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.574662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.574666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.574668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.574671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.574674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.574689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:19.574840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342669290142247:6109] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:19.574854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342669290142247:6109] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:19.574857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342669290142247:6109] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:19.574864Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1791 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:19.581162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:19.582179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:19.582191Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:19.582193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:19.582196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:19.582202Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1790 [2:7559342668928311002:2920]
2025-10-09T21:50:19.582215Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:19.582234Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:19.582242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:19.582245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:19.582247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:19.582261Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:19.582267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:19.582273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:19.582322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.582329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.582331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.582333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:19.582392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:19.582401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.582413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.582425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.582442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:19.582465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342669290142296:6109], now have 1 active actors on pipe
2025-10-09T21:50:19.582472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342669290142175:2432], now have 1 active actors on pipe
2025-10-09T21:50:19.582479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342668928310996:2920]
2025-10-09T21:50:19.582487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342668928310996:2920]
2025-10-09T21:50:19.582489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342668928310996:2920]
2025-10-09T21:50:19.582501Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:19.582513Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:19.582524Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:19.582527Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:19.582528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:19.582529Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:19.582530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:19.582552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342668928310996:2920]
2025-10-09T21:50:19.582559Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:19.582545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1790
2025-10-09T21:50:19.582571Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1790
2025-10-09T21:50:19.582568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:19.582579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.582584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.582617Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|df6d48b2-bc2828b6-c220ddb6-ee6fc946_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:19.582628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:19.582632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:19.582634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:19.582643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:19.582656Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:19.582677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:19.582822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.582828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.582840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:19.582925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1791 partition: 0 MaxSeqNo: 25 sessionId: src_id|df6d48b2-bc2828b6-c220ddb6-ee6fc946_0
2025-10-09T21:50:19.583199Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88d92dc0-7818b8a6-95fa4a59-2003c527_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|df6d48b2-bc2828b6-c220ddb6-ee6fc946_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:19.585497Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df6d48b2-bc2828b6-c220ddb6-ee6fc946_0] Write session: try to update token
2025-10-09T21:50:19.585512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df6d48b2-bc2828b6-c220ddb6-ee6fc946_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:19.597297Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1791 sessionId: src_id|df6d48b2-bc2828b6-c220ddb6-ee6fc946_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:19.597760Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:19.597777Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:19.597779Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:19.597780Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:19.597787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342668928310996:2920]
2025-10-09T21:50:19.597790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342668928310996:2920]
2025-10-09T21:50:19.597795Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342668928310996:2920]
2025-10-09T21:50:19.597832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:19.597837Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:19.597841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.597842Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:19.597843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:19.597846Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:19.597965Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:19.599513Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:19.661791Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342669290142304:6864]
2025-10-09T21:50:19.662059Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 1790, pipe: [1:7559342669290142304:6864]
2025-10-09T21:50:19.742558Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:19.742972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.742987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.743027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342669290142304:6864], now have 1 active actors on pipe
2025-10-09T21:50:19.743193Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:19.743200Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:19.743204Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342669290142304:6864]
2025-10-09T21:50:19.743216Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:19.743219Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:19.743275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1791 reason: error at write: Disconnected sessionId: src_id|df6d48b2-bc2828b6-c220ddb6-ee6fc946_0
2025-10-09T21:50:19.743434Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1791 sessionId: src_id|df6d48b2-bc2828b6-c220ddb6-ee6fc946_0 is DEAD
2025-10-09T21:50:19.743560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df6d48b2-bc2828b6-c220ddb6-ee6fc946_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:19.743579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df6d48b2-bc2828b6-c220ddb6-ee6fc946_0] Write session will restart in 0.006838s
2025-10-09T21:50:19.743600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df6d48b2-bc2828b6-c220ddb6-ee6fc946_0] Write session: Do CDS request
2025-10-09T21:50:19.743605Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df6d48b2-bc2828b6-c220ddb6-ee6fc946_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:19.751417Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df6d48b2-bc2828b6-c220ddb6-ee6fc946_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:19.751483Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:19.751496Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1792
2025-10-09T21:50:19.751645Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1792 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:19.751676Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1792 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:19.751683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1792 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:19.751685Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:19.751991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1792 sessionId:  describe result for acl check
2025-10-09T21:50:19.752021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:19.752027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:19.752029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:19.752035Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342669290142388:6099] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:19.752096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:19.752466Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342669290142388:6099] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:19.753494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:19.753629Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:19.753637Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:19.753657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:19.753698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:19.753724Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342668928311125:2915]
2025-10-09T21:50:19.754171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:19.754273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342669290142388:6099] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:19.754408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:19.754441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:19.754755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:19.754796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.754802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.754803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.754804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.754805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.754806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.754807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.754808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.754809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.754813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.754822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.754825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.754828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.754831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.754835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.754838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.754841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.754845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.754851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.754859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.754865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.754870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.754874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.754879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.754885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.754890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.754897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.754903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.754912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.754916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.754921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.754928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.754932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.754938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.754942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.754947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.754974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:19.759585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342669290142388:6099] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:19.759601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342669290142388:6099] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:19.759605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342669290142388:6099] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:19.759613Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1792 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:19.761230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:19.762326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:19.762340Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:19.762342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:19.762346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:19.762352Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1791 [2:7559342668928311125:2915]
2025-10-09T21:50:19.762364Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:19.762380Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:19.762388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:19.762391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:19.762393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:19.762408Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:19.762418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:19.762424Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:19.762479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.762485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.762488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.762490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:19.762555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:19.762562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.762572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.762584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.762604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:19.762626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342669290142428:6099], now have 1 active actors on pipe
2025-10-09T21:50:19.762634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342669290142306:2432], now have 1 active actors on pipe
2025-10-09T21:50:19.762641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342668928311120:2915]
2025-10-09T21:50:19.762643Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342668928311120:2915]
2025-10-09T21:50:19.762644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342668928311120:2915]
2025-10-09T21:50:19.762656Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:19.762653Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1791
2025-10-09T21:50:19.762665Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1791
2025-10-09T21:50:19.762670Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:19.762672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:19.762679Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:19.762681Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:19.762686Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:19.762689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:19.762722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.762731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.762746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342668928311120:2915]
2025-10-09T21:50:19.762757Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:19.762767Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:19.762782Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2d1cb544-1889d9de-cb63b145-b66d49ab_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:19.762789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:19.762793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:19.762795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:19.762804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:19.762818Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:19.762913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:19.763061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.763068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.763081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:19.763167Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1792 partition: 0 MaxSeqNo: 25 sessionId: src_id|2d1cb544-1889d9de-cb63b145-b66d49ab_0
2025-10-09T21:50:19.763420Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df6d48b2-bc2828b6-c220ddb6-ee6fc946_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2d1cb544-1889d9de-cb63b145-b66d49ab_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:19.765735Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d1cb544-1889d9de-cb63b145-b66d49ab_0] Write session: try to update token
2025-10-09T21:50:19.765749Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d1cb544-1889d9de-cb63b145-b66d49ab_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:19.777046Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:19.777064Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:19.777066Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:19.777069Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:19.777080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342668928311120:2915]
2025-10-09T21:50:19.777083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342668928311120:2915]
2025-10-09T21:50:19.777086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342668928311120:2915]
2025-10-09T21:50:19.777122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:19.777132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.777134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:19.777147Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:19.777163Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:19.777168Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:19.777254Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:19.777569Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1792 sessionId: src_id|2d1cb544-1889d9de-cb63b145-b66d49ab_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:19.779811Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:19.843566Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342669290142436:6864]
2025-10-09T21:50:19.843818Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 1791, pipe: [1:7559342669290142436:6864]
2025-10-09T21:50:19.919408Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:19.919644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.919655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.919696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342669290142436:6864], now have 1 active actors on pipe
2025-10-09T21:50:19.919867Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:19.919940Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:19.919954Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:19.919956Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342669290142436:6864]
2025-10-09T21:50:19.919976Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:19.920010Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1792 reason: error at write: Disconnected sessionId: src_id|2d1cb544-1889d9de-cb63b145-b66d49ab_0
2025-10-09T21:50:19.920171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1792 sessionId: src_id|2d1cb544-1889d9de-cb63b145-b66d49ab_0 is DEAD
2025-10-09T21:50:19.920320Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d1cb544-1889d9de-cb63b145-b66d49ab_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:19.920327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d1cb544-1889d9de-cb63b145-b66d49ab_0] Write session will restart in 0.006726s
2025-10-09T21:50:19.920349Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d1cb544-1889d9de-cb63b145-b66d49ab_0] Write session: Do CDS request
2025-10-09T21:50:19.920353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d1cb544-1889d9de-cb63b145-b66d49ab_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:19.928466Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d1cb544-1889d9de-cb63b145-b66d49ab_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:19.928543Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:19.928557Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1793
2025-10-09T21:50:19.928572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:19.928703Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1793 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:19.928744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1793 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:19.928756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1793 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:19.928761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:19.929172Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1793 sessionId:  describe result for acl check
2025-10-09T21:50:19.929208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:19.929215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:19.929216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:19.929222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342669290142524:6087] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:19.929793Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342669290142524:6087] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:19.930086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:19.930243Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:19.930252Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:19.930260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:19.930301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:19.930343Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342668928311246:2911]
2025-10-09T21:50:19.930733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:19.930959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:19.930989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:19.931190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:19.931221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.931226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.931227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.931228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.931228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.931229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.931229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.931230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.931231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.931234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.931240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.931242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.931244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.931246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.931249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.931251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.931253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.931255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.931259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.931264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:19.931268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.931271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:19.931274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.931277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:19.931279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.931282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:19.931285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.931288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:19.931290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.931293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:19.931296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.931299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.931302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.931305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.931308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.931311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.931326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:19.931772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342669290142524:6087] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:19.937738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342669290142524:6087] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:19.937753Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342669290142524:6087] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:19.937759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342669290142524:6087] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:19.937767Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1793 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:19.938242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:19.939299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:19.939313Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:19.939315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:19.939319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:19.939325Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1792 [2:7559342668928311246:2911]
2025-10-09T21:50:19.939337Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:19.939353Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:19.939356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:19.939359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:19.939361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:19.939377Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:19.939388Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:19.939400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:19.939455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.939469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.939471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.939474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:19.939525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:19.939534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.939545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:19.939557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:19.939576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:19.939601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342669290142440:2432], now have 1 active actors on pipe
2025-10-09T21:50:19.939609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342669290142566:6087], now have 1 active actors on pipe
2025-10-09T21:50:19.939615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342668928311241:2911]
2025-10-09T21:50:19.939618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342668928311241:2911]
2025-10-09T21:50:19.939620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342668928311241:2911]
2025-10-09T21:50:19.939633Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:19.939651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:19.939657Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:19.939660Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:19.939662Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:19.939670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:19.939677Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:19.939653Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1792
2025-10-09T21:50:19.939691Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1792
2025-10-09T21:50:19.939701Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342668928311241:2911]
2025-10-09T21:50:19.939709Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:19.939712Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:19.939722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.939726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.939770Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c5fd2f23-565340fb-62c3a72-bb5520cc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:19.939780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:19.939788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:19.939790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:19.939799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:19.939810Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:19.939827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:19.939970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:19.939977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:19.939988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:19.940090Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1793 partition: 0 MaxSeqNo: 25 sessionId: src_id|c5fd2f23-565340fb-62c3a72-bb5520cc_0
2025-10-09T21:50:19.940411Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d1cb544-1889d9de-cb63b145-b66d49ab_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c5fd2f23-565340fb-62c3a72-bb5520cc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:19.942600Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c5fd2f23-565340fb-62c3a72-bb5520cc_0] Write session: try to update token
2025-10-09T21:50:19.942614Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c5fd2f23-565340fb-62c3a72-bb5520cc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:19.954597Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1793 sessionId: src_id|c5fd2f23-565340fb-62c3a72-bb5520cc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:19.954796Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:19.954818Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:19.954820Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:19.954822Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:19.954830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342668928311241:2911]
2025-10-09T21:50:19.954834Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342668928311241:2911]
2025-10-09T21:50:19.954838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342668928311241:2911]
2025-10-09T21:50:19.954880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:19.954888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:19.954890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:19.954905Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:19.954912Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:19.954916Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:19.955051Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:19.956894Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:20.020213Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342673585109889:6864]
2025-10-09T21:50:20.020501Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 1792, pipe: [1:7559342673585109889:6864]
2025-10-09T21:50:20.101705Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:20.102010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:20.102042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:20.102092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342673585109889:6864], now have 1 active actors on pipe
2025-10-09T21:50:20.102301Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342673585109889:6864]
2025-10-09T21:50:20.102313Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:20.102317Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:20.102331Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:20.102350Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:20.102371Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1793 reason: error at write: Disconnected sessionId: src_id|c5fd2f23-565340fb-62c3a72-bb5520cc_0
2025-10-09T21:50:20.102554Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1793 sessionId: src_id|c5fd2f23-565340fb-62c3a72-bb5520cc_0 is DEAD
2025-10-09T21:50:20.102741Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5fd2f23-565340fb-62c3a72-bb5520cc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:20.102752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5fd2f23-565340fb-62c3a72-bb5520cc_0] Write session will restart in 0.009171s
2025-10-09T21:50:20.102779Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5fd2f23-565340fb-62c3a72-bb5520cc_0] Write session: Do CDS request
2025-10-09T21:50:20.102786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5fd2f23-565340fb-62c3a72-bb5520cc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:20.111517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:20.113146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:20.113326Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:20.113337Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:20.113349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:20.113411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:20.113444Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342673223278702:2877]
2025-10-09T21:50:20.113509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c5fd2f23-565340fb-62c3a72-bb5520cc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:20.113564Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:20.113577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1794
2025-10-09T21:50:20.113924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:20.113956Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1794 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:20.114010Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1794 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:20.114020Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1794 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:20.114024Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:20.114165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:20.114202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:20.114403Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1794 sessionId:  describe result for acl check
2025-10-09T21:50:20.114410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:20.114433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:20.114434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:20.114436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:20.114444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.114441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342673585109982:6082] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:20.114444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.114445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.114446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.114446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.114447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.114447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.114448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.114449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.114452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.114458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.114460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.114463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.114465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.114467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.114469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.114471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.114473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.114478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.114483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.114488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.114491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.114494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.114497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.114499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.114502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.114505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.114508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.114510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.114514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.114516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.114519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.114522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.114525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.114528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.114531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.114547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:20.114955Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342673585109982:6082] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:20.116756Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342673585109982:6082] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:20.120405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342673585109982:6082] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:20.120416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342673585109982:6082] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:20.120419Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342673585109982:6082] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:20.120426Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1794 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:20.120967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:20.122231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:20.122242Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:20.122244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:20.122247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:20.122252Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1793 [2:7559342673223278702:2877]
2025-10-09T21:50:20.122261Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:20.122275Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:20.122281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:20.122283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:20.122285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:20.122297Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:20.122304Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:20.122310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:20.122351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.122356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.122357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.122359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:20.122410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:20.122418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.122425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.122431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.122441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:20.122456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342673585110018:6082], now have 1 active actors on pipe
2025-10-09T21:50:20.122461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342673585109893:2432], now have 1 active actors on pipe
2025-10-09T21:50:20.122467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342673223278697:2877]
2025-10-09T21:50:20.122470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342673223278697:2877]
2025-10-09T21:50:20.122471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342673223278697:2877]
2025-10-09T21:50:20.122480Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:20.122492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:20.122495Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:20.122498Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:20.122499Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:20.122504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:20.122506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:20.122502Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1793
2025-10-09T21:50:20.122527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342673223278697:2877]
2025-10-09T21:50:20.122533Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:20.122534Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:20.122513Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1793
2025-10-09T21:50:20.122550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:20.122557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:20.122583Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|22edbdc7-fc31a09e-3bd144b0-44cb5b55_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:20.122593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:20.122596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:20.122597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:20.122604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:20.122610Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:20.122625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:20.122728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:20.122735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:20.122745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:20.122801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1794 partition: 0 MaxSeqNo: 25 sessionId: src_id|22edbdc7-fc31a09e-3bd144b0-44cb5b55_0
2025-10-09T21:50:20.123069Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5fd2f23-565340fb-62c3a72-bb5520cc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|22edbdc7-fc31a09e-3bd144b0-44cb5b55_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:20.125439Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|22edbdc7-fc31a09e-3bd144b0-44cb5b55_0] Write session: try to update token
2025-10-09T21:50:20.125457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|22edbdc7-fc31a09e-3bd144b0-44cb5b55_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:20.137059Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:20.137077Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:20.137080Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:20.137082Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:20.137091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342673223278697:2877]
2025-10-09T21:50:20.137096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342673223278697:2877]
2025-10-09T21:50:20.137102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342673223278697:2877]
2025-10-09T21:50:20.137144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:20.137144Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:20.137150Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:20.137150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.137152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:20.137153Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:20.137285Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:20.138138Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1794 sessionId: src_id|22edbdc7-fc31a09e-3bd144b0-44cb5b55_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:20.140347Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:20.202436Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342673585110028:6864]
2025-10-09T21:50:20.202659Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 1793, pipe: [1:7559342673585110028:6864]
2025-10-09T21:50:20.285314Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:20.285591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:20.285612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:20.285657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342673585110028:6864], now have 1 active actors on pipe
2025-10-09T21:50:20.285843Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:20.285868Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:20.285906Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:20.285911Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:20.285936Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342673585110028:6864]
2025-10-09T21:50:20.285981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1794 reason: error at write: Disconnected sessionId: src_id|22edbdc7-fc31a09e-3bd144b0-44cb5b55_0
2025-10-09T21:50:20.286204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1794 sessionId: src_id|22edbdc7-fc31a09e-3bd144b0-44cb5b55_0 is DEAD
2025-10-09T21:50:20.286374Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|22edbdc7-fc31a09e-3bd144b0-44cb5b55_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:20.286385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|22edbdc7-fc31a09e-3bd144b0-44cb5b55_0] Write session will restart in 0.009019s
2025-10-09T21:50:20.286414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|22edbdc7-fc31a09e-3bd144b0-44cb5b55_0] Write session: Do CDS request
2025-10-09T21:50:20.286421Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|22edbdc7-fc31a09e-3bd144b0-44cb5b55_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:20.296146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:20.296476Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|22edbdc7-fc31a09e-3bd144b0-44cb5b55_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:20.296652Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:20.296664Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1795
2025-10-09T21:50:20.296806Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1795 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:20.296851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1795 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:20.296861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1795 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:20.296865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:20.297319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1795 sessionId:  describe result for acl check
2025-10-09T21:50:20.297352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:20.297358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:20.297359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:20.297364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342673585110122:6075] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:20.297726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:20.297913Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:20.297920Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:20.297931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:20.297977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:20.298014Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342673223278826:2875]
2025-10-09T21:50:20.298013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342673585110122:6075] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:20.298398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:20.298617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:20.298644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:20.298836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:20.298869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.298876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.298877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.298878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.298878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.298879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.298879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.298880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.298880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.298883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.298889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.298891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.298894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.298896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.298898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.298900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.298902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.298904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.298908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.298912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.298917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.298920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.298923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.298926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.298928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.298931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.298934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.298936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.298939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.298942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.298945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.298948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.298951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.298954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.298956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.298959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.298975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:20.300089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342673585110122:6075] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:20.304536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342673585110122:6075] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:20.304552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342673585110122:6075] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:20.304555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342673585110122:6075] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:20.304563Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1795 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:20.306263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:20.307247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:20.307261Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:20.307264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:20.307267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:20.307274Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1794 [2:7559342673223278826:2875]
2025-10-09T21:50:20.307289Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:20.307307Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:20.307308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:20.307311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:20.307312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:20.307323Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:20.307332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:20.307337Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:20.307374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.307379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.307380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.307382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:20.307431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:20.307436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.307522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.307532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.307542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:20.307557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342673585110036:2432], now have 1 active actors on pipe
2025-10-09T21:50:20.307563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342673585110161:6075], now have 1 active actors on pipe
2025-10-09T21:50:20.307568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342673223278820:2875]
2025-10-09T21:50:20.307570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342673223278820:2875]
2025-10-09T21:50:20.307571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342673223278820:2875]
2025-10-09T21:50:20.307582Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:20.307592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:20.307604Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:20.307603Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:20.307605Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:20.307584Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1794
2025-10-09T21:50:20.307607Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1794
2025-10-09T21:50:20.307606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:20.307609Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:20.307632Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342673223278820:2875]
2025-10-09T21:50:20.307635Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:20.307637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:20.307654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:20.307658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:20.307709Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|91879b10-dd80dc3e-bb0e1135-2469018e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:20.307722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:20.307726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:20.307728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:20.307739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:20.307752Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:20.307774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:20.307942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:20.307953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:20.307982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:20.308091Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1795 partition: 0 MaxSeqNo: 25 sessionId: src_id|91879b10-dd80dc3e-bb0e1135-2469018e_0
2025-10-09T21:50:20.308376Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|22edbdc7-fc31a09e-3bd144b0-44cb5b55_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|91879b10-dd80dc3e-bb0e1135-2469018e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:20.317551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91879b10-dd80dc3e-bb0e1135-2469018e_0] Write session: try to update token
2025-10-09T21:50:20.317568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91879b10-dd80dc3e-bb0e1135-2469018e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:20.325240Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:20.325263Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:20.325266Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:20.325268Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:20.325277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342673223278820:2875]
2025-10-09T21:50:20.325282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342673223278820:2875]
2025-10-09T21:50:20.325287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342673223278820:2875]
2025-10-09T21:50:20.325329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:20.325332Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:20.325336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.325337Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:20.325339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:20.325339Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:20.325559Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:20.330048Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1795 sessionId: src_id|91879b10-dd80dc3e-bb0e1135-2469018e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:20.336674Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:20.386299Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342673585110168:6864]
2025-10-09T21:50:20.386545Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 1794, pipe: [1:7559342673585110168:6864]
2025-10-09T21:50:20.469741Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:20.469988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:20.470002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:20.470021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342673585110168:6864], now have 1 active actors on pipe
2025-10-09T21:50:20.470220Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:20.470244Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:20.470247Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342673585110168:6864]
2025-10-09T21:50:20.470272Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:20.470347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1795 reason: status is not ok: tablet will be restarted right now sessionId: src_id|91879b10-dd80dc3e-bb0e1135-2469018e_0
2025-10-09T21:50:20.470521Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1795 sessionId: src_id|91879b10-dd80dc3e-bb0e1135-2469018e_0 is DEAD
2025-10-09T21:50:20.470674Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91879b10-dd80dc3e-bb0e1135-2469018e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:20.470687Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91879b10-dd80dc3e-bb0e1135-2469018e_0] Write session will restart in 0.005207s
2025-10-09T21:50:20.470717Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91879b10-dd80dc3e-bb0e1135-2469018e_0] Write session: Do CDS request
2025-10-09T21:50:20.470724Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91879b10-dd80dc3e-bb0e1135-2469018e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:20.477471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91879b10-dd80dc3e-bb0e1135-2469018e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:20.477570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:20.477585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1796
2025-10-09T21:50:20.477693Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1796 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:20.477724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1796 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:20.477735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1796 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:20.477738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:20.478161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1796 sessionId:  describe result for acl check
2025-10-09T21:50:20.478194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:20.478201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:20.478203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:20.478209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342673585110255:6057] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:20.478785Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342673585110255:6057] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:20.479272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:20.480756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:20.480837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342673585110255:6057] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:20.480939Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:20.480953Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:20.480966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:20.481023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:20.481064Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342673223278949:2874]
2025-10-09T21:50:20.481644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:20.482003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:20.482067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:20.482387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:20.482433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.482439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.482440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.482441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.482442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.482443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.482444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.482445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.482447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.482451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.482462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.482466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.482470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.482474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.482478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.482482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.482486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.482489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.482500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.482508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.482518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.482526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.482535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.482544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.482549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.482557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.482560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.482564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.482570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.482573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.482577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.482582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.482591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.482597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.482603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.482610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.482639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:20.483105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342673585110255:6057] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:20.483118Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342673585110255:6057] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:20.483122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342673585110255:6057] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:20.483129Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1796 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:20.488950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:20.490011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:20.490048Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:20.490050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:20.490054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:20.490059Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1795 [2:7559342673223278949:2874]
2025-10-09T21:50:20.490072Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:20.490087Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:20.490091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:20.490093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:20.490096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:20.490110Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:20.490121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:20.490127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:20.490180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.490187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.490190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.490192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:20.490251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:20.490258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.490270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.490286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.490308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:20.490332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342673585110297:6057], now have 1 active actors on pipe
2025-10-09T21:50:20.490340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342673585110171:2432], now have 1 active actors on pipe
2025-10-09T21:50:20.490347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342673223278944:2874]
2025-10-09T21:50:20.490349Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342673223278944:2874]
2025-10-09T21:50:20.490351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342673223278944:2874]
2025-10-09T21:50:20.490362Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:20.490357Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1795
2025-10-09T21:50:20.490373Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:20.490361Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1795
2025-10-09T21:50:20.490381Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:20.490381Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:20.490383Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:20.490396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:20.490400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:20.490435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:20.490444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:20.490456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342673223278944:2874]
2025-10-09T21:50:20.490469Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:20.490496Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:20.490503Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7cee57ab-32083020-e53a5411-e07c196e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:20.490510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:20.490513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:20.490520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:20.490529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:20.490543Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:20.490571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:20.490716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:20.490726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:20.490740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:20.490839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1796 partition: 0 MaxSeqNo: 25 sessionId: src_id|7cee57ab-32083020-e53a5411-e07c196e_0
2025-10-09T21:50:20.491141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91879b10-dd80dc3e-bb0e1135-2469018e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7cee57ab-32083020-e53a5411-e07c196e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:20.493442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7cee57ab-32083020-e53a5411-e07c196e_0] Write session: try to update token
2025-10-09T21:50:20.493457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7cee57ab-32083020-e53a5411-e07c196e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:20.505104Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:20.505123Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:20.505124Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:20.505126Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:20.505134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342673223278944:2874]
2025-10-09T21:50:20.505138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342673223278944:2874]
2025-10-09T21:50:20.505141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342673223278944:2874]
2025-10-09T21:50:20.505182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:20.505188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.505188Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:20.505191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:20.505192Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:20.505195Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:20.505358Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:20.505768Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1796 sessionId: src_id|7cee57ab-32083020-e53a5411-e07c196e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:20.508415Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:20.570629Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342673585110304:6864]
2025-10-09T21:50:20.570875Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 1795, pipe: [1:7559342673585110304:6864]
2025-10-09T21:50:20.646971Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:20.647197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:20.647211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:20.647250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342673585110304:6864], now have 1 active actors on pipe
2025-10-09T21:50:20.647475Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:20.647490Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:20.647496Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342673585110304:6864]
2025-10-09T21:50:20.647523Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:20.647554Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1796 reason: error at write: Disconnected sessionId: src_id|7cee57ab-32083020-e53a5411-e07c196e_0
2025-10-09T21:50:20.647672Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:20.647735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1796 sessionId: src_id|7cee57ab-32083020-e53a5411-e07c196e_0 is DEAD
2025-10-09T21:50:20.647891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cee57ab-32083020-e53a5411-e07c196e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:20.647917Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cee57ab-32083020-e53a5411-e07c196e_0] Write session will restart in 0.008885s
2025-10-09T21:50:20.647945Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cee57ab-32083020-e53a5411-e07c196e_0] Write session: Do CDS request
2025-10-09T21:50:20.647951Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cee57ab-32083020-e53a5411-e07c196e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:20.657099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:20.657457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7cee57ab-32083020-e53a5411-e07c196e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:20.657560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:20.657572Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1797
2025-10-09T21:50:20.657694Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1797 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:20.657732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1797 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:20.657741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1797 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:20.657749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:20.658144Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1797 sessionId:  describe result for acl check
2025-10-09T21:50:20.658174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:20.658180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:20.658182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:20.658187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342673585110395:6067] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:20.658715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342673585110395:6067] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:20.658815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:20.658997Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:20.659008Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:20.659027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:20.659083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:20.659121Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342673223279074:2872]
2025-10-09T21:50:20.659680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:20.659896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:20.659933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:20.660227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:20.660278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.660284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.660285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.660287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.660288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.660289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.660290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.660291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.660292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.660297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.660306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.660311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.660314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.660322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.660327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.660334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.660338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.660346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.660352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.660360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.660365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.660370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.660378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.660383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.660389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.660394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.660403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.660408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.660417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.660422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.660431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.660435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.660441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.660425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342673585110395:6067] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:20.660445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.660451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.660456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.660487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:20.665195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342673585110395:6067] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:20.665208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342673585110395:6067] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:20.665211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342673585110395:6067] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:20.665218Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1797 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:20.666815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:20.668027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:20.668044Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:20.668046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:20.668050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:20.668056Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1796 [2:7559342673223279074:2872]
2025-10-09T21:50:20.668069Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:20.668087Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:20.668090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:20.668093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:20.668096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:20.668113Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:20.668128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:20.668136Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:20.668197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.668203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.668206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.668209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:20.668281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:20.668292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.668305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.668317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.668342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:20.668362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342673223279069:2872]
2025-10-09T21:50:20.668372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342673223279069:2872]
2025-10-09T21:50:20.668374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342673223279069:2872]
2025-10-09T21:50:20.668386Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:20.668395Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:20.668400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:20.668407Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:20.668409Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:20.668414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:20.668417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:20.668430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342673585110434:6067], now have 1 active actors on pipe
2025-10-09T21:50:20.668434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342673585110307:2432], now have 1 active actors on pipe
2025-10-09T21:50:20.668463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342673223279069:2872]
2025-10-09T21:50:20.668478Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:20.668491Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:20.668496Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1796
2025-10-09T21:50:20.668520Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1796
2025-10-09T21:50:20.668615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:20.668629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:20.668659Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b7dce317-4cce2a6c-fe1d4a8c-138f1d47_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:20.668686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:20.668696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:20.668698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:20.668710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:20.668721Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:20.668738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:20.668897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:20.668907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:20.668924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:20.669013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1797 partition: 0 MaxSeqNo: 25 sessionId: src_id|b7dce317-4cce2a6c-fe1d4a8c-138f1d47_0
2025-10-09T21:50:20.669363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cee57ab-32083020-e53a5411-e07c196e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b7dce317-4cce2a6c-fe1d4a8c-138f1d47_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:20.671846Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7dce317-4cce2a6c-fe1d4a8c-138f1d47_0] Write session: try to update token
2025-10-09T21:50:20.671863Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7dce317-4cce2a6c-fe1d4a8c-138f1d47_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:20.684337Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:20.684353Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:20.684355Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:20.684357Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:20.684367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342673223279069:2872]
2025-10-09T21:50:20.684371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342673223279069:2872]
2025-10-09T21:50:20.684375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342673223279069:2872]
2025-10-09T21:50:20.684419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:20.684421Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:20.684427Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:20.684429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.684430Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:20.684431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:20.684549Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1797 sessionId: src_id|b7dce317-4cce2a6c-fe1d4a8c-138f1d47_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:20.684562Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:20.686892Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:20.747773Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342673585110441:6864]
2025-10-09T21:50:20.748029Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 1796, pipe: [1:7559342673585110441:6864]
2025-10-09T21:50:20.836655Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:20.836944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:20.836960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:20.836998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342673585110441:6864], now have 1 active actors on pipe
2025-10-09T21:50:20.837211Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:20.837257Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:20.837263Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:20.837274Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:20.837306Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342673585110441:6864]
2025-10-09T21:50:20.837341Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1797 reason: error at write: Disconnected sessionId: src_id|b7dce317-4cce2a6c-fe1d4a8c-138f1d47_0
2025-10-09T21:50:20.837529Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1797 sessionId: src_id|b7dce317-4cce2a6c-fe1d4a8c-138f1d47_0 is DEAD
2025-10-09T21:50:20.837685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7dce317-4cce2a6c-fe1d4a8c-138f1d47_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:20.837695Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7dce317-4cce2a6c-fe1d4a8c-138f1d47_0] Write session will restart in 0.007272s
2025-10-09T21:50:20.837722Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7dce317-4cce2a6c-fe1d4a8c-138f1d47_0] Write session: Do CDS request
2025-10-09T21:50:20.837729Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7dce317-4cce2a6c-fe1d4a8c-138f1d47_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:20.846556Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7dce317-4cce2a6c-fe1d4a8c-138f1d47_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:20.846764Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:20.846784Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1798
2025-10-09T21:50:20.846950Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1798 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:20.846987Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1798 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:20.846998Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1798 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:20.847002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:20.847165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:20.847508Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1798 sessionId:  describe result for acl check
2025-10-09T21:50:20.847563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:20.847570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:20.847572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:20.847579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342673585110536:6058] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:20.848152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342673585110536:6058] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:20.849028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:20.849224Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:20.849237Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:20.849248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:20.849304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:20.849356Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342673223279205:2870]
2025-10-09T21:50:20.849905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:20.850072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342673585110536:6058] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:20.850291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:20.850342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:20.850674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:20.850729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.850734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.850735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.850736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.850737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.850737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.850738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.850738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.850739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.850743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.850750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.850752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.850754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.850756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.850758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.850760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.850762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.850764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.850770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.850775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:20.850780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.850783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:20.850786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.850789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:20.850791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.850795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:20.850797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.850800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:20.850803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.850807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:20.850809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.850813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.850817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.850827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.850840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.850849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.850866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:20.856078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342673585110536:6058] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:20.856095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342673585110536:6058] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:20.856098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342673585110536:6058] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:20.856105Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1798 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:20.857699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:20.859087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:20.859102Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:20.859105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:20.859109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:20.859118Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1797 [2:7559342673223279205:2870]
2025-10-09T21:50:20.859131Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:20.859150Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:20.859158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:20.859161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:20.859162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:20.859179Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:20.859190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:20.859201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:20.859264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.859270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.859273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.859276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:20.859345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:20.859355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.859367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:20.859379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:20.859397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:20.859424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342673585110447:2432], now have 1 active actors on pipe
2025-10-09T21:50:20.859432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342673585110575:6058], now have 1 active actors on pipe
2025-10-09T21:50:20.859441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342673223279199:2870]
2025-10-09T21:50:20.859443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342673223279199:2870]
2025-10-09T21:50:20.859445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342673223279199:2870]
2025-10-09T21:50:20.859458Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:20.859474Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:20.859479Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:20.859484Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:20.859487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:20.859496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:20.859485Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1797
2025-10-09T21:50:20.859499Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:20.859534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342673223279199:2870]
2025-10-09T21:50:20.859537Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:20.859499Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1797
2025-10-09T21:50:20.859539Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:20.859558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:20.859563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:20.859611Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|134f411f-af8d8611-5da53413-1654c6bf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:20.859623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:20.859628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:20.859630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:20.859640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:20.859649Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:20.859671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:20.859828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:20.859844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:20.859868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:20.859977Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1798 partition: 0 MaxSeqNo: 25 sessionId: src_id|134f411f-af8d8611-5da53413-1654c6bf_0
2025-10-09T21:50:20.860342Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7dce317-4cce2a6c-fe1d4a8c-138f1d47_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|134f411f-af8d8611-5da53413-1654c6bf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:20.862938Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|134f411f-af8d8611-5da53413-1654c6bf_0] Write session: try to update token
2025-10-09T21:50:20.862956Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|134f411f-af8d8611-5da53413-1654c6bf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:20.874973Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:20.874995Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:20.874997Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:20.875000Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:20.875010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342673223279199:2870]
2025-10-09T21:50:20.875015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342673223279199:2870]
2025-10-09T21:50:20.875019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342673223279199:2870]
2025-10-09T21:50:20.875067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:20.875083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:20.875086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:20.875107Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:20.875121Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:20.875125Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:20.875266Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:20.876442Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1798 sessionId: src_id|134f411f-af8d8611-5da53413-1654c6bf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:20.879237Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:20.883397Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:50:20.883423Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 15
2025-10-09T21:50:20.937712Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342673585110584:6864]
2025-10-09T21:50:20.937971Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 1797, pipe: [1:7559342673585110584:6864]
2025-10-09T21:50:21.030859Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:21.031146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.031161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.031218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342673585110584:6864], now have 1 active actors on pipe
2025-10-09T21:50:21.031380Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:21.031439Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342673585110584:6864]
2025-10-09T21:50:21.031444Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:21.031459Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 15
2025-10-09T21:50:21.031472Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:21.031541Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1798 reason: status is not ok: tablet will be restarted right now sessionId: src_id|134f411f-af8d8611-5da53413-1654c6bf_0
2025-10-09T21:50:21.031728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1798 sessionId: src_id|134f411f-af8d8611-5da53413-1654c6bf_0 is DEAD
2025-10-09T21:50:21.031883Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|134f411f-af8d8611-5da53413-1654c6bf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:21.031895Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|134f411f-af8d8611-5da53413-1654c6bf_0] Write session will restart in 0.005232s
2025-10-09T21:50:21.031918Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|134f411f-af8d8611-5da53413-1654c6bf_0] Write session: Do CDS request
2025-10-09T21:50:21.031925Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|134f411f-af8d8611-5da53413-1654c6bf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:21.038519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|134f411f-af8d8611-5da53413-1654c6bf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:21.038593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:21.038611Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1799
2025-10-09T21:50:21.038779Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1799 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:21.038815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1799 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:21.038825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1799 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:21.038829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:21.039331Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1799 sessionId:  describe result for acl check
2025-10-09T21:50:21.039371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:21.039380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:21.039382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:21.039389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342677880077948:6047] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:21.039976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342677880077948:6047] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:21.041962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:21.042398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342677880077948:6047] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:21.043833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:21.044010Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:21.044018Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:21.044027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:21.044070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:21.044101Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342677518246668:2844]
2025-10-09T21:50:21.044489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:21.044750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:21.044803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:21.044913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342677880077948:6047] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:21.044924Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342677880077948:6047] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:21.044928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342677880077948:6047] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:21.044936Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1799 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:21.045097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:21.045170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.045177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.045179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.045180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.045181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.045183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.045184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.045185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.045186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.045192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.045202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.045206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.045210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.045214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.045218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.045223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.045226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.045230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.045237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.045245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.045251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.045256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.045261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.045266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.045269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.045274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.045279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.045283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.045288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.045293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.045297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.045303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.045310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.045315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.045321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.045326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.045362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:21.051889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:21.053159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:21.053172Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:21.053175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:21.053180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:21.053187Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1798 [2:7559342677518246668:2844]
2025-10-09T21:50:21.053200Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:21.053217Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:21.053220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:21.053223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:21.053226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:21.053242Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:21.053258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:21.053266Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:21.053319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.053324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.053326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.053328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:21.053381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:21.053386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.053394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.053400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.053412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:21.053429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342677518246662:2844]
2025-10-09T21:50:21.053434Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342677518246662:2844]
2025-10-09T21:50:21.053435Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342677518246662:2844]
2025-10-09T21:50:21.053446Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:21.053455Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:21.053458Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:21.053461Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:21.053467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880077908:2432], now have 1 active actors on pipe
2025-10-09T21:50:21.053475Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:21.053502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:21.053508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:21.053513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880078032:6047], now have 1 active actors on pipe
2025-10-09T21:50:21.053544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342677518246662:2844]
2025-10-09T21:50:21.053554Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:21.053574Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:21.053555Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1798
2025-10-09T21:50:21.053572Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1798
2025-10-09T21:50:21.053649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.053659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.053729Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:50:21.053768Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7bdaaa84-ab1c9f32-b660198b-28851801_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:21.053777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:21.053780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:21.053782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:21.053790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:21.053798Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:21.053809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:21.053997Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 20 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:50:21.053991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.054007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.054025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:21.054052Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:50:21.054118Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1799 partition: 0 MaxSeqNo: 25 sessionId: src_id|7bdaaa84-ab1c9f32-b660198b-28851801_0
2025-10-09T21:50:21.054461Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|134f411f-af8d8611-5da53413-1654c6bf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7bdaaa84-ab1c9f32-b660198b-28851801_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:21.056944Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7bdaaa84-ab1c9f32-b660198b-28851801_0] Write session: try to update token
2025-10-09T21:50:21.056963Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7bdaaa84-ab1c9f32-b660198b-28851801_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:21.067937Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:21.067955Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:21.067957Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:21.067958Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:21.067967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342677518246662:2844]
2025-10-09T21:50:21.067970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342677518246662:2844]
2025-10-09T21:50:21.067973Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342677518246662:2844]
2025-10-09T21:50:21.068011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:21.068018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.068022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:21.068024Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:21.068031Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:21.068043Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:21.068155Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:21.070027Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1799 sessionId: src_id|7bdaaa84-ab1c9f32-b660198b-28851801_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:21.072209Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:21.131741Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342677880078044:6864]
2025-10-09T21:50:21.132053Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 1798, pipe: [1:7559342677880078044:6864]
2025-10-09T21:50:21.219388Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:21.219702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.219718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.219763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880078044:6864], now have 1 active actors on pipe
2025-10-09T21:50:21.219985Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:21.220023Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:21.220033Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:21.220038Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:21.220099Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342677880078044:6864]
2025-10-09T21:50:21.220113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1799 reason: error at write: Disconnected sessionId: src_id|7bdaaa84-ab1c9f32-b660198b-28851801_0
2025-10-09T21:50:21.220294Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1799 sessionId: src_id|7bdaaa84-ab1c9f32-b660198b-28851801_0 is DEAD
2025-10-09T21:50:21.220452Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bdaaa84-ab1c9f32-b660198b-28851801_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:21.220463Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bdaaa84-ab1c9f32-b660198b-28851801_0] Write session will restart in 0.006581s
2025-10-09T21:50:21.220500Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bdaaa84-ab1c9f32-b660198b-28851801_0] Write session: Do CDS request
2025-10-09T21:50:21.220507Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bdaaa84-ab1c9f32-b660198b-28851801_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:21.228698Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7bdaaa84-ab1c9f32-b660198b-28851801_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:21.228717Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:21.228734Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1800
2025-10-09T21:50:21.228901Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1800 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:21.228938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1800 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:21.228946Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1800 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:21.228955Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:21.229463Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1800 sessionId:  describe result for acl check
2025-10-09T21:50:21.229516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:21.229523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:21.229524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:21.229531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342677880078128:6034] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:21.230174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342677880078128:6034] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:21.231306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:21.231886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342677880078128:6034] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:21.233245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:21.233418Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:21.233427Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:21.233436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:21.233475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:21.233518Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342677518246797:2845]
2025-10-09T21:50:21.234018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:21.234254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342677880078128:6034] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:21.234268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342677880078128:6034] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:21.234273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342677880078128:6034] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:21.234281Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1800 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:21.234345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:21.234403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:21.234714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:21.234756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.234761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.234762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.234762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.234763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.234764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.234765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.234766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.234766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.234770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.234777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.234779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.234781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.234783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.234785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.234788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.234792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.234796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.234803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.234813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.234820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.234826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.234830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.234836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.234840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.234846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.234850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.234857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.234866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.234871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.234880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.234885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.234894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.234899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.234903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.234906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.234924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:21.241432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:21.242512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:21.242530Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:21.242533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:21.242537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:21.242544Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1799 [2:7559342677518246797:2845]
2025-10-09T21:50:21.242558Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:21.242575Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:21.242579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:21.242583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:21.242585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:21.242600Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:21.242604Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:21.242609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:21.242652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.242659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.242660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.242662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:21.242721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:21.242729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.242740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.242758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.242781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:21.242804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880078181:6034], now have 1 active actors on pipe
2025-10-09T21:50:21.242814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880078053:2432], now have 1 active actors on pipe
2025-10-09T21:50:21.242821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342677518246791:2845]
2025-10-09T21:50:21.242823Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342677518246791:2845]
2025-10-09T21:50:21.242825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342677518246791:2845]
2025-10-09T21:50:21.242833Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:21.242852Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:21.242873Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:21.242872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:21.242876Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:21.242879Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:21.242882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:21.242913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342677518246791:2845]
2025-10-09T21:50:21.242915Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:21.242920Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:21.242893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1799
2025-10-09T21:50:21.242919Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1799
2025-10-09T21:50:21.243105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.243121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.243161Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ba1d668d-e9661f87-d1c2961a-ae151a1f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:21.243174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:21.243178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:21.243181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:21.243192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:21.243200Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:21.243212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:21.243373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.243387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.243410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:21.243527Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1800 partition: 0 MaxSeqNo: 25 sessionId: src_id|ba1d668d-e9661f87-d1c2961a-ae151a1f_0
2025-10-09T21:50:21.243884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7bdaaa84-ab1c9f32-b660198b-28851801_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ba1d668d-e9661f87-d1c2961a-ae151a1f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:21.246377Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ba1d668d-e9661f87-d1c2961a-ae151a1f_0] Write session: try to update token
2025-10-09T21:50:21.246395Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ba1d668d-e9661f87-d1c2961a-ae151a1f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:21.257296Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:21.257314Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:21.257317Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:21.257319Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:21.257329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342677518246791:2845]
2025-10-09T21:50:21.257334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342677518246791:2845]
2025-10-09T21:50:21.257339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342677518246791:2845]
2025-10-09T21:50:21.257374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:21.257380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.257382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:21.257387Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:21.257394Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:21.257398Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:21.257512Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:21.259154Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1800 sessionId: src_id|ba1d668d-e9661f87-d1c2961a-ae151a1f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:21.261870Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:21.320159Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342677880078195:6864]
2025-10-09T21:50:21.320449Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 1799, pipe: [1:7559342677880078195:6864]
2025-10-09T21:50:21.409134Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:21.409441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.409456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.409511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880078195:6864], now have 1 active actors on pipe
2025-10-09T21:50:21.409753Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:21.409792Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342677880078195:6864]
2025-10-09T21:50:21.409831Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:21.409906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1800 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ba1d668d-e9661f87-d1c2961a-ae151a1f_0
2025-10-09T21:50:21.409946Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:21.410105Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1800 sessionId: src_id|ba1d668d-e9661f87-d1c2961a-ae151a1f_0 is DEAD
2025-10-09T21:50:21.410294Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba1d668d-e9661f87-d1c2961a-ae151a1f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:21.410309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba1d668d-e9661f87-d1c2961a-ae151a1f_0] Write session will restart in 0.007754s
2025-10-09T21:50:21.410337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba1d668d-e9661f87-d1c2961a-ae151a1f_0] Write session: Do CDS request
2025-10-09T21:50:21.410344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba1d668d-e9661f87-d1c2961a-ae151a1f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:21.419528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ba1d668d-e9661f87-d1c2961a-ae151a1f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:21.419610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:21.419628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1801
2025-10-09T21:50:21.419862Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1801 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:21.419909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1801 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:21.419928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1801 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:21.419932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:21.420385Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1801 sessionId:  describe result for acl check
2025-10-09T21:50:21.420426Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:21.420433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:21.420435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:21.420441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342677880078289:6031] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:21.420751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:21.421020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342677880078289:6031] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:21.422541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:21.422711Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:21.422721Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:21.422733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:21.422783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:21.422822Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342677518246927:2840]
2025-10-09T21:50:21.423009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342677880078289:6031] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:21.423379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:21.423632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:21.423675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:21.423980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:21.424020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.424027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.424028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.424029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.424030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.424030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.424031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.424031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.424032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.424035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.424041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.424044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.424046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.424048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.424050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.424052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.424054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.424056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.424060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.424065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.424070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.424073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.424076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.424079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.424081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.424084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.424089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.424092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.424094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.424098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.424101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.424104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.424107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.424110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.424112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.424115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.424131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:21.428552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342677880078289:6031] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:21.428567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342677880078289:6031] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:21.428570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342677880078289:6031] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:21.428582Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1801 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:21.429933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:21.431138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:21.431154Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:21.431156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:21.431160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:21.431167Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1800 [2:7559342677518246927:2840]
2025-10-09T21:50:21.431179Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:21.431196Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:21.431199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:21.431202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:21.431204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:21.431221Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:21.431227Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:21.431233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:21.431296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.431304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.431306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.431309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:21.431373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:21.431381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.431392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.431408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.431423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:21.431443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880078329:6031], now have 1 active actors on pipe
2025-10-09T21:50:21.431452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880078198:2432], now have 1 active actors on pipe
2025-10-09T21:50:21.431460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342677518246921:2840]
2025-10-09T21:50:21.431462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342677518246921:2840]
2025-10-09T21:50:21.431464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342677518246921:2840]
2025-10-09T21:50:21.431477Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:21.431491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:21.431491Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:21.431495Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:21.431497Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:21.431496Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1800
2025-10-09T21:50:21.431514Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1800
2025-10-09T21:50:21.431505Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:21.431507Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:21.431530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342677518246921:2840]
2025-10-09T21:50:21.431533Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:21.431535Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:21.431555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.431560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.431595Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|63a3c01a-8c18268c-e59c1b2d-da5af5c7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:21.431609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:21.431613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:21.431616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:21.431626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:21.431640Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:21.431662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:21.431821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.431831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.431844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:21.431929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1801 partition: 0 MaxSeqNo: 25 sessionId: src_id|63a3c01a-8c18268c-e59c1b2d-da5af5c7_0
2025-10-09T21:50:21.432276Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ba1d668d-e9661f87-d1c2961a-ae151a1f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|63a3c01a-8c18268c-e59c1b2d-da5af5c7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:21.435063Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63a3c01a-8c18268c-e59c1b2d-da5af5c7_0] Write session: try to update token
2025-10-09T21:50:21.435083Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63a3c01a-8c18268c-e59c1b2d-da5af5c7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:21.445260Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:21.445277Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:21.445279Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:21.445281Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:21.445291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342677518246921:2840]
2025-10-09T21:50:21.445296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342677518246921:2840]
2025-10-09T21:50:21.445300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342677518246921:2840]
2025-10-09T21:50:21.445338Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:21.445342Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:21.445345Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:21.445345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:21.445353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.445356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:21.445498Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:21.447971Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1801 sessionId: src_id|63a3c01a-8c18268c-e59c1b2d-da5af5c7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:21.450248Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:21.510159Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342677880078337:6864]
2025-10-09T21:50:21.510526Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 1800, pipe: [1:7559342677880078337:6864]
2025-10-09T21:50:21.596660Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:21.596965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.596980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.597025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880078337:6864], now have 1 active actors on pipe
2025-10-09T21:50:21.597315Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:21.597315Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:21.597319Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342677880078337:6864]
2025-10-09T21:50:21.597332Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:21.597350Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:21.597380Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:50:21.597395Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1801 reason: error at write: Disconnected sessionId: src_id|63a3c01a-8c18268c-e59c1b2d-da5af5c7_0
2025-10-09T21:50:21.597532Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 18 consumer shared/user session shared/user_1_18_7210191584091053671_v1 is DEAD
2025-10-09T21:50:21.597585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1801 sessionId: src_id|63a3c01a-8c18268c-e59c1b2d-da5af5c7_0 is DEAD
2025-10-09T21:50:21.597664Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342596275683247:6869] disconnected.
2025-10-09T21:50:21.597674Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342596275683247:6869] disconnected; active server actors: 1
2025-10-09T21:50:21.597678Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342596275683247:6869] client user disconnected session shared/user_1_18_7210191584091053671_v1
2025-10-09T21:50:21.597737Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63a3c01a-8c18268c-e59c1b2d-da5af5c7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:21.597748Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63a3c01a-8c18268c-e59c1b2d-da5af5c7_0] Write session will restart in 0.009053s
2025-10-09T21:50:21.597785Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63a3c01a-8c18268c-e59c1b2d-da5af5c7_0] Write session: Do CDS request
2025-10-09T21:50:21.597792Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63a3c01a-8c18268c-e59c1b2d-da5af5c7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:21.597803Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:50:21.597856Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:50:21.597861Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:50:21.597870Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.006784s
2025-10-09T21:50:21.605406Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:50:21.605725Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:50:21.605738Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 19
2025-10-09T21:50:21.605932Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 19 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:50:21.605987Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:50:21.606078Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 auth for : user
2025-10-09T21:50:21.606244Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 Handle describe topics response
2025-10-09T21:50:21.606280Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 auth is DEAD
2025-10-09T21:50:21.606296Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:50:21.606606Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:50:21.606692Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_19_3411058432498014305_v1
2025-10-09T21:50:21.606731Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342677880078425:6025] connected; active server actors: 1
2025-10-09T21:50:21.606759Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559342677880078425:6025] session shared/user_1_19_3411058432498014305_v1
2025-10-09T21:50:21.606770Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:50:21.606780Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:50:21.606803Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_19_3411058432498014305_v1" (Sender=[1:7559342677880078389:6025], Pipe=[1:7559342677880078425:6025], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:50:21.606807Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:50:21.606820Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:50:21.606828Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_19_3411058432498014305_v1" (Sender=[1:7559342677880078389:6025], Pipe=[1:7559342677880078425:6025], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:50:21.606839Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_19_3411058432498014305_v1" sender [1:7559342677880078389:6025] lock partition 0 for ReadingSession "shared/user_1_19_3411058432498014305_v1" (Sender=[1:7559342677880078389:6025], Pipe=[1:7559342677880078425:6025], Partitions=[], ActiveFamilyCount=1) generation 1 step 19
2025-10-09T21:50:21.606852Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:50:21.606862Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000039s
2025-10-09T21:50:21.607081Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 19 Session: "shared/user_1_19_3411058432498014305_v1" ClientId: "user" PipeClient { RawX1: 7559342677880078425 RawX2: 4503603922343817 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:50:21.607113Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:50:21.607380Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|63a3c01a-8c18268c-e59c1b2d-da5af5c7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:21.607460Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:21.607472Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1802
2025-10-09T21:50:21.607579Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1802 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:21.607602Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1802 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:21.607608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1802 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:21.607611Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:21.607951Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1802 sessionId:  describe result for acl check
2025-10-09T21:50:21.607985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:21.607986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:21.607988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:21.607993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342677880078439:6015] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:21.608201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:21.608535Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342677880078439:6015] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:21.610103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:21.610296Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:21.610306Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:21.610318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:21.610373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:21.610413Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342677518247058:2835]
2025-10-09T21:50:21.610403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342677880078439:6015] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:21.610950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:21.611245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:21.611296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:21.611584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:21.611651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.611657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.611658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.611659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.611661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.611662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.611664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.611665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.611666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.611671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.611682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.611686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.611690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.611694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.611697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.611700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.611703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.611706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.611713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.611720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.611726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.611731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.611735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.611739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.611742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.611747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.611751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.611756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.611760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.611765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.611770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.611776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.611785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.611791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.611796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.611801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.611832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:21.612644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342677880078439:6015] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:21.612658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342677880078439:6015] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:21.612662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342677880078439:6015] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:21.612671Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1802 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:21.618420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:21.619652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:21.619671Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:21.619675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:21.619679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:21.619689Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1801 [2:7559342677518247058:2835]
2025-10-09T21:50:21.619706Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:21.619724Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:21.619733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:21.619737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:21.619739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:21.619759Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:21.619765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:21.619771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:21.619830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.619838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.619840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.619844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:21.619914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:21.619925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.619938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.619952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.619970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:21.619993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880078478:6015], now have 1 active actors on pipe
2025-10-09T21:50:21.620003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880078339:2432], now have 1 active actors on pipe
2025-10-09T21:50:21.620006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880078433:6018], now have 1 active actors on pipe
2025-10-09T21:50:21.620013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342677518247052:2835]
2025-10-09T21:50:21.620015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342677518247052:2835]
2025-10-09T21:50:21.620019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342677518247052:2835]
2025-10-09T21:50:21.620034Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:21.620055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:21.620054Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:21.620058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:21.620061Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:21.620069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:21.620072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:21.620072Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1801
2025-10-09T21:50:21.620078Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1801
2025-10-09T21:50:21.620111Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1801, pipe: [1:7559342677880078433:6018]
2025-10-09T21:50:21.620108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342677518247052:2835]
2025-10-09T21:50:21.620117Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:21.620127Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:21.620153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.620162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.620179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.620182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.620188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_19_3411058432498014305_v1 on pipe: [1:7559342677880078433:6018]
2025-10-09T21:50:21.620217Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_19_3411058432498014305_v1:1 with generation 1801
2025-10-09T21:50:21.620225Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|38290960-8ed38619-d94987cb-4e16dd7c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:21.620234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:21.620237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:21.620240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:21.620518Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1802 partition: 0 MaxSeqNo: 25 sessionId: src_id|38290960-8ed38619-d94987cb-4e16dd7c_0
2025-10-09T21:50:21.620734Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|63a3c01a-8c18268c-e59c1b2d-da5af5c7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|38290960-8ed38619-d94987cb-4e16dd7c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:21.620250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:21.620266Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:21.620279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:21.620284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:21.620286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:21.620301Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_19_3411058432498014305_v1
2025-10-09T21:50:21.620303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:21.620331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:21.620346Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:50:21.620442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.620444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.620455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:21.623074Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38290960-8ed38619-d94987cb-4e16dd7c_0] Write session: try to update token
2025-10-09T21:50:21.623089Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38290960-8ed38619-d94987cb-4e16dd7c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:21.624598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:50:21.624631Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:50:21.624641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:21.624645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:21.624647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:21.624655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:50:21.624669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:50:21.624804Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046621620 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:50:21.624830Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:50:21.624865Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 sending to client partition status
2025-10-09T21:50:21.633689Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:21.633705Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:21.633707Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:21.633712Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:21.633721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342677518247052:2835]
2025-10-09T21:50:21.633727Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342677518247052:2835]
2025-10-09T21:50:21.633729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342677518247052:2835]
2025-10-09T21:50:21.633762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:21.633767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.633769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:21.633779Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:21.633793Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:21.633797Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:21.633893Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:21.635396Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1802 sessionId: src_id|38290960-8ed38619-d94987cb-4e16dd7c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:21.637712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:21.781380Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:21.781656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.781669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.781901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_19_3411058432498014305_v1
2025-10-09T21:50:21.781922Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:21.781946Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_19_3411058432498014305_v1
2025-10-09T21:50:21.781935Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:21.781951Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:21.781942Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:21.781999Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342677880078433:6018]
2025-10-09T21:50:21.782018Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1802 reason: error at write: Disconnected sessionId: src_id|38290960-8ed38619-d94987cb-4e16dd7c_0
2025-10-09T21:50:21.782183Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1802 sessionId: src_id|38290960-8ed38619-d94987cb-4e16dd7c_0 is DEAD
2025-10-09T21:50:21.782359Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38290960-8ed38619-d94987cb-4e16dd7c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:21.782380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38290960-8ed38619-d94987cb-4e16dd7c_0] Write session will restart in 0.008161s
2025-10-09T21:50:21.782422Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38290960-8ed38619-d94987cb-4e16dd7c_0] Write session: Do CDS request
2025-10-09T21:50:21.782430Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38290960-8ed38619-d94987cb-4e16dd7c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:21.791306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:21.791629Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38290960-8ed38619-d94987cb-4e16dd7c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:21.791677Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:21.791691Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1803
2025-10-09T21:50:21.791933Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1803 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:21.791970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1803 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:21.791980Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1803 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:21.791984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:21.792400Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1803 sessionId:  describe result for acl check
2025-10-09T21:50:21.792433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:21.792439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:21.792442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:21.792448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342677880078585:6008] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:21.792921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:21.793018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342677880078585:6008] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:21.793092Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:21.793106Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:21.793118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:21.793173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:21.793209Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342677518247194:2820]
2025-10-09T21:50:21.793654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:21.794124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:21.794161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:21.794361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:21.794400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.794404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.794405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.794405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.794406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.794407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.794407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.794408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.794408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.794412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.794418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.794420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.794422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.794424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.794426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.794428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.794430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.794432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.794435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.794442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.794446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.794449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.794452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.794455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.794457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.794460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.794463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.794465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.794468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.794472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.794475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.794478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.794480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.794483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.794486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.794489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.794504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:21.795051Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342677880078585:6008] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:21.800228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342677880078585:6008] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:21.800243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342677880078585:6008] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:21.800247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342677880078585:6008] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:21.800256Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1803 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:21.800776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:21.801840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:21.801853Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:21.801855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:21.801858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:21.801863Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1802 [2:7559342677518247194:2820]
2025-10-09T21:50:21.801874Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:21.801887Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:21.801889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:21.801892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:21.801894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:21.801910Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:21.801921Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:21.801928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:21.801987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.801995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.801997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.802001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:21.802072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:21.802080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.802087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.802094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.802104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:21.802117Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342677518247189:2820]
2025-10-09T21:50:21.802119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342677518247189:2820]
2025-10-09T21:50:21.802120Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342677518247189:2820]
2025-10-09T21:50:21.802127Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:21.802139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:21.802143Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:21.802149Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:21.802151Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:21.802151Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:21.802154Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:21.802165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880078624:6008], now have 1 active actors on pipe
2025-10-09T21:50:21.802170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880078491:2432], now have 1 active actors on pipe
2025-10-09T21:50:21.802196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342677518247189:2820]
2025-10-09T21:50:21.802211Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:21.802228Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:21.802229Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1802
2025-10-09T21:50:21.802233Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1802
2025-10-09T21:50:21.802335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.802344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.802366Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8cf7a09a-d011b3bb-51970562-ddb8e4e4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:21.802377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:21.802383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:21.802384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:21.802392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:21.802404Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:21.802415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:21.802591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.802598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.802608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:21.802717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1803 partition: 0 MaxSeqNo: 25 sessionId: src_id|8cf7a09a-d011b3bb-51970562-ddb8e4e4_0
2025-10-09T21:50:21.803016Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38290960-8ed38619-d94987cb-4e16dd7c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8cf7a09a-d011b3bb-51970562-ddb8e4e4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:21.805673Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8cf7a09a-d011b3bb-51970562-ddb8e4e4_0] Write session: try to update token
2025-10-09T21:50:21.805691Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8cf7a09a-d011b3bb-51970562-ddb8e4e4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:21.814370Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:21.814387Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:21.814397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342677518247189:2820]
2025-10-09T21:50:21.814446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:21.814452Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:21.814634Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:21.814773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:21.814911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:21.815045Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:21.815222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:21.815258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:21.818955Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1803 sessionId: src_id|8cf7a09a-d011b3bb-51970562-ddb8e4e4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:21.821180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:21.821380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.821398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.821434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:21.821540Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:21.828401Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:21.828415Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:21.828417Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:21.828418Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:21.828427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342677518247189:2820]
2025-10-09T21:50:21.828430Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342677518247189:2820]
2025-10-09T21:50:21.828433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342677518247189:2820]
2025-10-09T21:50:21.828469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:21.828490Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:21.828513Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:21.828519Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:21.828599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.828607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:21.828724Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:21.882366Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342677880078635:6018]
2025-10-09T21:50:21.882610Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 1802, pipe: [1:7559342677880078635:6018]
2025-10-09T21:50:21.971010Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:21.977860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:21.977890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:21.978053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:21.978107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:21.978170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:21.978231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:21.978571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:21.978581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:21.978645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:21.978708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:21.978764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:21.978821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:21.978863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:21.978870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:21.978927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:21.978981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:21.979035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:21.979087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:21.979128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:21.979133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:21.979193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:21.979248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:21.979301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:21.979367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:21.979405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:21.979411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:21.979472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:21.979524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:21.979581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:21.979634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:21.979674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:21.979683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:21.979742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:21.979793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:21.979848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:21.979904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:21.979944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:21.979949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:21.980014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:21.980066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:21.980120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:21.980175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:21.980218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:21.980225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:21.980280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:21.980359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:21.980411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:21.980484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:21.980522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:21.980527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:21.980592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:21.980639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:21.980689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:21.980749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:21.980790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:21.980796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:21.980852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:21.980901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:21.980952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:21.981012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:21.981051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:21.981057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:21.981097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880078635:6018], now have 1 active actors on pipe
2025-10-09T21:50:21.981405Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:21.981425Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342677880078635:6018]
2025-10-09T21:50:21.981456Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:21.981466Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:21.981504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1803 reason: error at write: Disconnected sessionId: src_id|8cf7a09a-d011b3bb-51970562-ddb8e4e4_0
2025-10-09T21:50:21.981578Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:21.981692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1803 sessionId: src_id|8cf7a09a-d011b3bb-51970562-ddb8e4e4_0 is DEAD
2025-10-09T21:50:21.981821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8cf7a09a-d011b3bb-51970562-ddb8e4e4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:21.981832Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8cf7a09a-d011b3bb-51970562-ddb8e4e4_0] Write session will restart in 0.006856s
2025-10-09T21:50:21.981865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8cf7a09a-d011b3bb-51970562-ddb8e4e4_0] Write session: Do CDS request
2025-10-09T21:50:21.981871Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8cf7a09a-d011b3bb-51970562-ddb8e4e4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:21.989496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8cf7a09a-d011b3bb-51970562-ddb8e4e4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:21.989585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:21.989600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1804
2025-10-09T21:50:21.989737Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1804 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:21.989769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1804 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:21.989773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1804 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:21.989776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:21.990118Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1804 sessionId:  describe result for acl check
2025-10-09T21:50:21.990145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:21.990150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:21.990151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:21.990155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342677880078732:6003] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:21.990605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342677880078732:6003] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:21.990608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:21.992230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:21.992389Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:21.992396Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:21.992405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:21.992446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:21.992473Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342677518247326:2831]
2025-10-09T21:50:21.992856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:21.992947Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342677880078732:6003] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:21.993197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:21.993242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:21.993528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:21.993587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.993595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.993596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.993597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.993599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.993600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.993601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.993602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.993603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.993608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.993618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.993622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.993625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.993629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.993632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.993636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.993640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.993643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.993650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.993657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:21.993663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.993671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:21.993674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.993677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:21.993679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.993682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:21.993685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.993688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:21.993690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.993693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:21.993696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.993699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:21.993702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.993705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:21.993711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.993714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:21.993738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:21.998553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342677880078732:6003] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:21.998568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342677880078732:6003] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:21.998572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342677880078732:6003] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:21.998580Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1804 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:21.999941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:22.000982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:22.000995Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:22.000997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:22.001000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:22.001005Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1803 [2:7559342677518247326:2831]
2025-10-09T21:50:22.001024Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:22.001037Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:22.001045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:22.001048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:22.001050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:22.001066Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:22.001075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:22.001082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:22.001140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.001147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.001149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.001152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:22.001204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:22.001211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.001223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.001233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.001247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:22.001270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342677518247321:2831]
2025-10-09T21:50:22.001277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342677518247321:2831]
2025-10-09T21:50:22.001280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342677518247321:2831]
2025-10-09T21:50:22.001289Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:22.001297Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:22.001304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880078771:6003], now have 1 active actors on pipe
2025-10-09T21:50:22.001307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342677880078639:2432], now have 1 active actors on pipe
2025-10-09T21:50:22.001310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:22.001312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:22.001315Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:22.001329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:22.001337Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:22.001342Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1803
2025-10-09T21:50:22.001345Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1803
2025-10-09T21:50:22.001366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342677518247321:2831]
2025-10-09T21:50:22.001375Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:22.001394Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:22.001404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.001409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.001442Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ec907383-5c945eaa-73315ff4-d0566759_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:22.001452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:22.001461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:22.001463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:22.001472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:22.001492Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:22.001511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:22.001668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.001678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.001697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:22.001805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1804 partition: 0 MaxSeqNo: 25 sessionId: src_id|ec907383-5c945eaa-73315ff4-d0566759_0
2025-10-09T21:50:22.002125Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8cf7a09a-d011b3bb-51970562-ddb8e4e4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ec907383-5c945eaa-73315ff4-d0566759_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:22.004605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ec907383-5c945eaa-73315ff4-d0566759_0] Write session: try to update token
2025-10-09T21:50:22.004625Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ec907383-5c945eaa-73315ff4-d0566759_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:22.015970Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:22.015989Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:22.015991Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:22.015994Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:22.016004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342677518247321:2831]
2025-10-09T21:50:22.016009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342677518247321:2831]
2025-10-09T21:50:22.016011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342677518247321:2831]
2025-10-09T21:50:22.016051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:22.016056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.016058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:22.016081Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:22.016088Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:22.016096Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:22.016176Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:22.018122Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1804 sessionId: src_id|ec907383-5c945eaa-73315ff4-d0566759_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:22.020537Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:22.081554Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342682175046096:6018]
2025-10-09T21:50:22.081865Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 1803, pipe: [1:7559342682175046096:6018]
2025-10-09T21:50:22.165456Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:22.166021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.166049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.166094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342682175046096:6018], now have 1 active actors on pipe
2025-10-09T21:50:22.166311Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:22.166306Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:22.166324Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:22.166325Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:22.166370Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342682175046096:6018]
2025-10-09T21:50:22.166390Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1804 reason: error at write: Disconnected sessionId: src_id|ec907383-5c945eaa-73315ff4-d0566759_0
2025-10-09T21:50:22.166577Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1804 sessionId: src_id|ec907383-5c945eaa-73315ff4-d0566759_0 is DEAD
2025-10-09T21:50:22.166745Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec907383-5c945eaa-73315ff4-d0566759_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:22.166756Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec907383-5c945eaa-73315ff4-d0566759_0] Write session will restart in 0.007108s
2025-10-09T21:50:22.166785Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec907383-5c945eaa-73315ff4-d0566759_0] Write session: Do CDS request
2025-10-09T21:50:22.166791Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec907383-5c945eaa-73315ff4-d0566759_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:22.175547Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ec907383-5c945eaa-73315ff4-d0566759_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:22.175619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:22.175649Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1805
2025-10-09T21:50:22.175758Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1805 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:22.175791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1805 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:22.175801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1805 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:22.175809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:22.176190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1805 sessionId:  describe result for acl check
2025-10-09T21:50:22.176230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:22.176235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:22.176236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:22.176241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342682175046192:5982] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:22.176674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342682175046192:5982] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:22.176912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:22.178880Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342682175046192:5982] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:22.178940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:22.179137Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:22.179146Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:22.179157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:22.179208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:22.179248Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342681813214794:2809]
2025-10-09T21:50:22.179757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:22.180031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:22.180070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:22.180290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:22.180324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.180328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.180329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.180329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.180330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.180331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.180332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.180332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.180333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.180336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.180343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.180346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.180348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.180350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.180352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.180354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.180356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.180358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.180362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.180368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.180372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.180375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.180378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.180381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.180384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.180387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.180389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.180392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.180395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.180398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.180401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.180404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.180407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.180410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.180414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.180417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.180433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:22.185356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342682175046192:5982] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:22.185370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342682175046192:5982] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:22.185373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342682175046192:5982] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:22.185380Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1805 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:22.186863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:22.187912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:22.187923Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:22.187926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:22.187929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:22.187934Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1804 [2:7559342681813214794:2809]
2025-10-09T21:50:22.187945Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:22.187960Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:22.187965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:22.187968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:22.187969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:22.187982Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:22.187986Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:22.187991Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:22.188045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.188050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.188052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.188053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:22.188108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:22.188115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.188122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.188129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.188138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:22.188156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342681813214788:2809]
2025-10-09T21:50:22.188163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342681813214788:2809]
2025-10-09T21:50:22.188165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342681813214788:2809]
2025-10-09T21:50:22.188178Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:22.188192Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:22.188193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342682175046099:2432], now have 1 active actors on pipe
2025-10-09T21:50:22.188197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342682175046234:5982], now have 1 active actors on pipe
2025-10-09T21:50:22.188202Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:22.188204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:22.188205Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:22.188218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:22.188221Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:22.188248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342681813214788:2809]
2025-10-09T21:50:22.188261Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:22.188270Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:22.188264Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1804
2025-10-09T21:50:22.188265Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1804
2025-10-09T21:50:22.188304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.188314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.188341Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1a60e439-beb5888c-b7bbb2e7-1e379e13_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:22.188352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:22.188356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:22.188358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:22.188369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:22.188384Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:22.188402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:22.188605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.188616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.188635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:22.188724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1805 partition: 0 MaxSeqNo: 25 sessionId: src_id|1a60e439-beb5888c-b7bbb2e7-1e379e13_0
2025-10-09T21:50:22.189036Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec907383-5c945eaa-73315ff4-d0566759_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1a60e439-beb5888c-b7bbb2e7-1e379e13_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:22.191688Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a60e439-beb5888c-b7bbb2e7-1e379e13_0] Write session: try to update token
2025-10-09T21:50:22.191702Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a60e439-beb5888c-b7bbb2e7-1e379e13_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:22.199683Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:22.199704Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:22.199714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342681813214788:2809]
2025-10-09T21:50:22.199751Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:22.199762Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:22.199961Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:22.200111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:22.200250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:22.200391Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:22.200585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:22.200632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:22.204555Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1805 sessionId: src_id|1a60e439-beb5888c-b7bbb2e7-1e379e13_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:22.207120Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:22.207345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.207364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.207403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:22.207465Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:22.212666Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:22.212682Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:22.212684Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:22.212686Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:22.212694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342681813214788:2809]
2025-10-09T21:50:22.212698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342681813214788:2809]
2025-10-09T21:50:22.212701Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342681813214788:2809]
2025-10-09T21:50:22.212735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:22.212741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.212744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:22.212759Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:22.212766Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:22.212773Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:22.212887Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:22.266359Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342682175046244:6018]
2025-10-09T21:50:22.266663Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 1804, pipe: [1:7559342682175046244:6018]
2025-10-09T21:50:22.360180Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:22.367464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.367488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.367649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.367714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.367785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.367857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.367914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.367925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.367994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.368052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.368115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.368186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.368232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.368240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.368304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.368360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.368420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.368478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.368525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.368533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.368598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.368653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.368712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.368776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.368882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.368891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.368951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.369012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.369072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.369131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.369170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.369177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.369235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.369296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.369360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.369429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.369485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.369492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.369553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.369607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.369666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.369729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.369838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.369845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.369902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.369956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.370008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.370111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.370167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.370172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.370251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.370304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.370357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.370414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.370456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.370462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.370513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.370571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.370624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.370676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.370719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.370726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.370785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342682175046244:6018], now have 1 active actors on pipe
2025-10-09T21:50:22.371009Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:22.371034Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342682175046244:6018]
2025-10-09T21:50:22.371063Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:22.371078Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:22.371112Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1805 reason: error at write: Disconnected sessionId: src_id|1a60e439-beb5888c-b7bbb2e7-1e379e13_0
2025-10-09T21:50:22.371306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1805 sessionId: src_id|1a60e439-beb5888c-b7bbb2e7-1e379e13_0 is DEAD
2025-10-09T21:50:22.371443Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a60e439-beb5888c-b7bbb2e7-1e379e13_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:22.371454Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a60e439-beb5888c-b7bbb2e7-1e379e13_0] Write session will restart in 0.007316s
2025-10-09T21:50:22.371479Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a60e439-beb5888c-b7bbb2e7-1e379e13_0] Write session: Do CDS request
2025-10-09T21:50:22.371485Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a60e439-beb5888c-b7bbb2e7-1e379e13_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:22.371680Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:22.379512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1a60e439-beb5888c-b7bbb2e7-1e379e13_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:22.379562Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:22.379573Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1806
2025-10-09T21:50:22.379725Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1806 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:22.379769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1806 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:22.379776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1806 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:22.379779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:22.380125Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1806 sessionId:  describe result for acl check
2025-10-09T21:50:22.380153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:22.380160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:22.380162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:22.380167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342682175046346:5976] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:22.380713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342682175046346:5976] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:22.380944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:22.382903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:22.383095Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:22.383105Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:22.383115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:22.383159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:22.383186Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342681813214929:2807]
2025-10-09T21:50:22.383294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342682175046346:5976] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:22.383545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:22.383794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:22.383830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:22.384038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:22.384067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.384071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.384072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.384073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.384073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.384074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.384074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.384075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.384075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.384079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.384085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.384087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.384089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.384091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.384093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.384095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.384097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.384099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.384103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.384107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.384111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.384114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.384117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.384120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.384122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.384125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.384128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.384130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.384133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.384136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.384139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.384141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.384144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.384147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.384149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.384153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.384169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:22.388956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342682175046346:5976] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:22.388971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342682175046346:5976] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:22.388975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342682175046346:5976] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:22.388983Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1806 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:22.390477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:22.391522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:22.391537Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:22.391540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:22.391543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:22.391550Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1805 [2:7559342681813214929:2807]
2025-10-09T21:50:22.391562Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:22.391577Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:22.391579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:22.391582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:22.391583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:22.391595Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:22.391598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:22.391604Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:22.391643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.391649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.391651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.391653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:22.391711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:22.391720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.391732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.391743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.391753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:22.391774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342682175046388:5976], now have 1 active actors on pipe
2025-10-09T21:50:22.391782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342682175046252:2432], now have 1 active actors on pipe
2025-10-09T21:50:22.391789Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342681813214923:2807]
2025-10-09T21:50:22.391791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342681813214923:2807]
2025-10-09T21:50:22.391792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342681813214923:2807]
2025-10-09T21:50:22.391804Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:22.391816Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:22.391821Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:22.391824Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:22.391825Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:22.391816Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1805
2025-10-09T21:50:22.391833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:22.391844Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1805
2025-10-09T21:50:22.391836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:22.391872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.391882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.391895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342681813214923:2807]
2025-10-09T21:50:22.391906Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:22.391918Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:22.391940Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bd527ed9-b2101c0-c09f1f16-bda70f19_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:22.391953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:22.391956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:22.391957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:22.391964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:22.391972Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:22.391988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:22.392129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.392134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.392144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:22.392225Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1806 partition: 0 MaxSeqNo: 25 sessionId: src_id|bd527ed9-b2101c0-c09f1f16-bda70f19_0
2025-10-09T21:50:22.392503Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1a60e439-beb5888c-b7bbb2e7-1e379e13_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bd527ed9-b2101c0-c09f1f16-bda70f19_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:22.394865Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd527ed9-b2101c0-c09f1f16-bda70f19_0] Write session: try to update token
2025-10-09T21:50:22.394879Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd527ed9-b2101c0-c09f1f16-bda70f19_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:22.405701Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:22.405719Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:22.405721Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:22.405722Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:22.405730Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342681813214923:2807]
2025-10-09T21:50:22.405732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342681813214923:2807]
2025-10-09T21:50:22.405735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342681813214923:2807]
2025-10-09T21:50:22.405770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:22.405777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.405778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:22.405781Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:22.405788Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:22.405792Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:22.405931Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:22.407182Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1806 sessionId: src_id|bd527ed9-b2101c0-c09f1f16-bda70f19_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:22.409504Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:22.471155Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342682175046395:6018]
2025-10-09T21:50:22.471429Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 1805, pipe: [1:7559342682175046395:6018]
2025-10-09T21:50:22.549943Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:22.550413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.550431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.550475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342682175046395:6018], now have 1 active actors on pipe
2025-10-09T21:50:22.550659Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:22.550729Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:22.550738Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342682175046395:6018]
2025-10-09T21:50:22.550756Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:22.550760Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:22.550819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1806 reason: error at write: Disconnected sessionId: src_id|bd527ed9-b2101c0-c09f1f16-bda70f19_0
2025-10-09T21:50:22.550980Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1806 sessionId: src_id|bd527ed9-b2101c0-c09f1f16-bda70f19_0 is DEAD
2025-10-09T21:50:22.551152Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd527ed9-b2101c0-c09f1f16-bda70f19_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:22.551163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd527ed9-b2101c0-c09f1f16-bda70f19_0] Write session will restart in 0.009333s
2025-10-09T21:50:22.551190Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd527ed9-b2101c0-c09f1f16-bda70f19_0] Write session: Do CDS request
2025-10-09T21:50:22.551196Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd527ed9-b2101c0-c09f1f16-bda70f19_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:22.561064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:22.561435Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd527ed9-b2101c0-c09f1f16-bda70f19_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:22.561574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:22.561588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1807
2025-10-09T21:50:22.561716Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1807 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:22.561739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1807 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:22.561746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1807 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:22.561749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:22.562135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1807 sessionId:  describe result for acl check
2025-10-09T21:50:22.562167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:22.562172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:22.562174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:22.562180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342682175046497:5968] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:22.562724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342682175046497:5968] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:22.563847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:22.564023Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:22.564032Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:22.564042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:22.564089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:22.564122Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342681813215064:2803]
2025-10-09T21:50:22.564442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342682175046497:5968] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:22.564508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:22.564806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:22.564845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:22.565077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:22.565112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.565117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.565118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.565119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.565119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.565120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.565121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.565121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.565122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.565125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.565131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.565133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.565135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.565136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.565138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.565140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.565142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.565144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.565148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.565153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.565157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.565160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.565163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.565166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.565168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.565171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.565173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.565176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.565178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.565181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.565183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.565187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.565191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.565195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.565199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.565203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.565230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:22.570072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342682175046497:5968] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:22.570086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342682175046497:5968] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:22.570098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342682175046497:5968] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:22.570106Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1807 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:22.571457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:22.572497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:22.572514Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:22.572516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:22.572520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:22.572526Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1806 [2:7559342681813215064:2803]
2025-10-09T21:50:22.572543Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:22.572563Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:22.572570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:22.572573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:22.572575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:22.572591Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:22.572601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:22.572612Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:22.572669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.572675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.572678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.572680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:22.572745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:22.572752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.572764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.572782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.572799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:22.572819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342682175046398:2432], now have 1 active actors on pipe
2025-10-09T21:50:22.572828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342682175046535:5968], now have 1 active actors on pipe
2025-10-09T21:50:22.572834Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342681813215058:2803]
2025-10-09T21:50:22.572836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342681813215058:2803]
2025-10-09T21:50:22.572838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342681813215058:2803]
2025-10-09T21:50:22.572850Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:22.572869Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:22.572872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:22.572872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:22.572874Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:22.572885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:22.572888Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:22.572867Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1806
2025-10-09T21:50:22.572900Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1806
2025-10-09T21:50:22.572913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342681813215058:2803]
2025-10-09T21:50:22.572922Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:22.572924Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:22.572950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.572959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.572993Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|96a487e6-4d91e999-61d2941-dacc7c69_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:22.573004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:22.573008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:22.573010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:22.573020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:22.573034Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:22.573049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:22.573240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.573247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.573260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:22.573390Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1807 partition: 0 MaxSeqNo: 25 sessionId: src_id|96a487e6-4d91e999-61d2941-dacc7c69_0
2025-10-09T21:50:22.573729Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd527ed9-b2101c0-c09f1f16-bda70f19_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|96a487e6-4d91e999-61d2941-dacc7c69_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:22.576099Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96a487e6-4d91e999-61d2941-dacc7c69_0] Write session: try to update token
2025-10-09T21:50:22.576113Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96a487e6-4d91e999-61d2941-dacc7c69_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:22.587726Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:22.587755Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:22.587759Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:22.587762Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:22.587774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342681813215058:2803]
2025-10-09T21:50:22.587781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342681813215058:2803]
2025-10-09T21:50:22.587787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342681813215058:2803]
2025-10-09T21:50:22.587839Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:22.587843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:22.587853Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:22.587854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.587857Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:22.587857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:22.588122Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:22.588728Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1807 sessionId: src_id|96a487e6-4d91e999-61d2941-dacc7c69_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:22.591023Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:22.650892Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342682175046542:6018]
2025-10-09T21:50:22.651187Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 1806, pipe: [1:7559342682175046542:6018]
2025-10-09T21:50:22.735779Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:22.736462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.736479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.736527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342682175046542:6018], now have 1 active actors on pipe
2025-10-09T21:50:22.736774Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:22.736790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:22.736789Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342682175046542:6018]
2025-10-09T21:50:22.736807Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:22.736845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1807 reason: error at write: Disconnected sessionId: src_id|96a487e6-4d91e999-61d2941-dacc7c69_0
2025-10-09T21:50:22.736829Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:22.737017Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1807 sessionId: src_id|96a487e6-4d91e999-61d2941-dacc7c69_0 is DEAD
2025-10-09T21:50:22.737168Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96a487e6-4d91e999-61d2941-dacc7c69_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:22.737179Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96a487e6-4d91e999-61d2941-dacc7c69_0] Write session will restart in 0.005117s
2025-10-09T21:50:22.737203Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96a487e6-4d91e999-61d2941-dacc7c69_0] Write session: Do CDS request
2025-10-09T21:50:22.737209Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96a487e6-4d91e999-61d2941-dacc7c69_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:22.743370Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|96a487e6-4d91e999-61d2941-dacc7c69_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:22.743461Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:22.743475Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1808
2025-10-09T21:50:22.743564Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1808 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:22.743589Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1808 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:22.743592Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1808 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:22.743595Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:22.743874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1808 sessionId:  describe result for acl check
2025-10-09T21:50:22.743902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:22.743906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:22.743915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:22.743925Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342682175046560:5969] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:22.744430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342682175046560:5969] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:22.746930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342682175046560:5969] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:22.748441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:22.752048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:22.752247Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:22.752259Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:22.752271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:22.752325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:22.752372Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342681813215204:2796]
2025-10-09T21:50:22.752524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342682175046560:5969] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:22.752543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342682175046560:5969] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:22.752547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342682175046560:5969] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:22.752556Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1808 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:22.752962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:22.753248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:22.753291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:22.753569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:22.753612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.753619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.753619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.753620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.753621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.753621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.753622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.753623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.753623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.753626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.753637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.753640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.753644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.753649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.753653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.753657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.753661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.753664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.753670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.753675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.753679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.753682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.753684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.753688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.753690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.753693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.753704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.753709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.753718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.753725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.753734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.753743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.753750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.753753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.753756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.753759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.753784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:22.759974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:22.761304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:22.761320Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:22.761322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:22.761325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:22.761331Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1807 [2:7559342681813215204:2796]
2025-10-09T21:50:22.761339Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:22.761356Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:22.761364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:22.761368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:22.761370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:22.761392Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:22.761403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:22.761413Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:22.761466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.761473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.761486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.761489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:22.761556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:22.761566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.761577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.761583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.761593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:22.761610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342682175046679:5969], now have 1 active actors on pipe
2025-10-09T21:50:22.761615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342682175046545:2432], now have 1 active actors on pipe
2025-10-09T21:50:22.761623Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342681813215198:2796]
2025-10-09T21:50:22.761633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342681813215198:2796]
2025-10-09T21:50:22.761635Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342681813215198:2796]
2025-10-09T21:50:22.761648Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:22.761658Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:22.761668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:22.761668Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:22.761670Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:22.761684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:22.761686Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:22.761710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342681813215198:2796]
2025-10-09T21:50:22.761717Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:22.761724Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:22.761749Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1807
2025-10-09T21:50:22.761759Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1807
2025-10-09T21:50:22.761842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.761852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.761884Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|10d9cfb5-a805360d-b71c3e16-a99897d9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:22.761896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:22.761900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:22.761903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:22.761914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:22.761924Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:22.761938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:22.762184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.762196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.762218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:22.762334Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1808 partition: 0 MaxSeqNo: 25 sessionId: src_id|10d9cfb5-a805360d-b71c3e16-a99897d9_0
2025-10-09T21:50:22.762715Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|96a487e6-4d91e999-61d2941-dacc7c69_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|10d9cfb5-a805360d-b71c3e16-a99897d9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:22.765520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10d9cfb5-a805360d-b71c3e16-a99897d9_0] Write session: try to update token
2025-10-09T21:50:22.765538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10d9cfb5-a805360d-b71c3e16-a99897d9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:22.773220Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:22.773237Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:22.773248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342681813215198:2796]
2025-10-09T21:50:22.773292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:22.773293Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:22.773464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:22.773618Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:22.773754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:22.773890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:22.774095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:22.774138Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:22.780396Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1808 sessionId: src_id|10d9cfb5-a805360d-b71c3e16-a99897d9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:22.783063Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:22.783319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.783347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.783385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:22.783475Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:22.785430Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:22.785449Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:22.785452Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:22.785454Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:22.785464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342681813215198:2796]
2025-10-09T21:50:22.785470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342681813215198:2796]
2025-10-09T21:50:22.785486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342681813215198:2796]
2025-10-09T21:50:22.785533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:22.785542Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:22.785548Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:22.785551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.785553Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:22.785554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:22.785869Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:22.836702Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342682175046691:6018]
2025-10-09T21:50:22.836963Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 1807, pipe: [1:7559342682175046691:6018]
2025-10-09T21:50:22.932028Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:22.938884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.938906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.939053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.939112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.939172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.939237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.939285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.939294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.939353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.939410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.939471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.939531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.939571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.939579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.939632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.939682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.939738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.939793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.939830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.939837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.939892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.939948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.940011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.940066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.940106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.940109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.940205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.940263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.940327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.940384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.940475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.940485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.940543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.940631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.940717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.940811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.940848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.940858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.940956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.941036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.941113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.941201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.941277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.941285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.941340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.941394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.941448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.941515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.941635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.941641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.941693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.941746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.941804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.941858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.941917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.941923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.941976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:22.942045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:22.942100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:22.942151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:22.942187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:22.942192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:22.942236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342682175046691:6018], now have 1 active actors on pipe
2025-10-09T21:50:22.942536Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342682175046691:6018]
2025-10-09T21:50:22.942582Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:22.942615Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:22.942633Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:22.942672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1808 reason: error at write: Disconnected sessionId: src_id|10d9cfb5-a805360d-b71c3e16-a99897d9_0
2025-10-09T21:50:22.942937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1808 sessionId: src_id|10d9cfb5-a805360d-b71c3e16-a99897d9_0 is DEAD
2025-10-09T21:50:22.943052Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10d9cfb5-a805360d-b71c3e16-a99897d9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:22.943062Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10d9cfb5-a805360d-b71c3e16-a99897d9_0] Write session will restart in 0.006239s
2025-10-09T21:50:22.943083Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10d9cfb5-a805360d-b71c3e16-a99897d9_0] Write session: Do CDS request
2025-10-09T21:50:22.943090Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10d9cfb5-a805360d-b71c3e16-a99897d9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:22.943185Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:22.950539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10d9cfb5-a805360d-b71c3e16-a99897d9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:22.950653Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:22.950670Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1809
2025-10-09T21:50:22.950823Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1809 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:22.950870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1809 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:22.950878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1809 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:22.950884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:22.951542Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1809 sessionId:  describe result for acl check
2025-10-09T21:50:22.951581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:22.951588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:22.951590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:22.951596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342682175046732:5962] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:22.952241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342682175046732:5962] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:22.954604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:22.955088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342682175046732:5962] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:22.956403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:22.956578Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:22.956585Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:22.956601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:22.956650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:22.956693Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342681813215341:2776]
2025-10-09T21:50:22.957197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:22.957512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:22.957553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:22.957667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342682175046732:5962] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:22.957683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342682175046732:5962] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:22.957687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342682175046732:5962] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:22.957696Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1809 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:22.957872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:22.957918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.957923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.957936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.957937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.957939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.957940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.957941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.957942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.957943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.957948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.957959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.957963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.957967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.957975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.957978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.957981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.957984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.957987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.957994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.958001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:22.958007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.958012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:22.958021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.958027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:22.958044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.958049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:22.958054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.958063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:22.958067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.958072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:22.958078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.958088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.958094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.958104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.958109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.958114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.958144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:22.964675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:22.966007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:22.966025Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:22.966044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:22.966048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:22.966058Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1808 [2:7559342681813215341:2776]
2025-10-09T21:50:22.966071Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:22.966089Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:22.966098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:22.966101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:22.966104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:22.966121Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:22.966133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:22.966139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:22.966195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.966203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.966205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.966208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:22.966286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:22.966296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.966308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:22.966325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:22.966348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:22.966374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342682175046832:5962], now have 1 active actors on pipe
2025-10-09T21:50:22.966382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342682175046695:2432], now have 1 active actors on pipe
2025-10-09T21:50:22.966390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342681813215335:2776]
2025-10-09T21:50:22.966393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342681813215335:2776]
2025-10-09T21:50:22.966396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342681813215335:2776]
2025-10-09T21:50:22.966406Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:22.966419Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:22.966425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:22.966430Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:22.966432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:22.966441Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:22.966444Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:22.966422Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1808
2025-10-09T21:50:22.966428Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1808
2025-10-09T21:50:22.966478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342681813215335:2776]
2025-10-09T21:50:22.966491Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:22.966517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.966526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.966569Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ac732cc3-c74fed04-fce069ab-ea4c32d9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:22.966579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:22.966582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:22.966583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:22.966591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:22.966603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:22.966602Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:22.966619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:22.966794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:22.966802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:22.966814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:22.966878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1809 partition: 0 MaxSeqNo: 25 sessionId: src_id|ac732cc3-c74fed04-fce069ab-ea4c32d9_0
2025-10-09T21:50:22.967196Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10d9cfb5-a805360d-b71c3e16-a99897d9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ac732cc3-c74fed04-fce069ab-ea4c32d9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:22.969460Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac732cc3-c74fed04-fce069ab-ea4c32d9_0] Write session: try to update token
2025-10-09T21:50:22.969488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac732cc3-c74fed04-fce069ab-ea4c32d9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:22.981216Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:22.981234Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:22.981236Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:22.981237Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:22.981244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342681813215335:2776]
2025-10-09T21:50:22.981248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342681813215335:2776]
2025-10-09T21:50:22.981253Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342681813215335:2776]
2025-10-09T21:50:22.981289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:22.981294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:22.981296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:22.981313Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:22.981328Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:22.981332Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:22.981426Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:22.983495Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1809 sessionId: src_id|ac732cc3-c74fed04-fce069ab-ea4c32d9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:22.985953Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:23.042575Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342686470014138:6018]
2025-10-09T21:50:23.042924Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 1808, pipe: [1:7559342686470014138:6018]
2025-10-09T21:50:23.133237Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:23.133563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:23.133583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:23.133635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342686470014138:6018], now have 1 active actors on pipe
2025-10-09T21:50:23.133896Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342686470014138:6018]
2025-10-09T21:50:23.133915Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:23.133920Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:23.133932Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:23.133974Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:23.133979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1809 reason: error at write: Disconnected sessionId: src_id|ac732cc3-c74fed04-fce069ab-ea4c32d9_0
2025-10-09T21:50:23.134170Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1809 sessionId: src_id|ac732cc3-c74fed04-fce069ab-ea4c32d9_0 is DEAD
2025-10-09T21:50:23.134321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac732cc3-c74fed04-fce069ab-ea4c32d9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:23.134335Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac732cc3-c74fed04-fce069ab-ea4c32d9_0] Write session will restart in 0.006521s
2025-10-09T21:50:23.134363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac732cc3-c74fed04-fce069ab-ea4c32d9_0] Write session: Do CDS request
2025-10-09T21:50:23.134369Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac732cc3-c74fed04-fce069ab-ea4c32d9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:23.142508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac732cc3-c74fed04-fce069ab-ea4c32d9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:23.142609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:23.142625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1810
2025-10-09T21:50:23.142743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1810 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:23.142776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1810 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:23.142785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1810 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:23.142789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:23.143173Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1810 sessionId:  describe result for acl check
2025-10-09T21:50:23.143204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:23.143205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:23.143206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:23.143211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342686470014262:5942] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:23.143660Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342686470014262:5942] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:23.144313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:23.145982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342686470014262:5942] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:23.146153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:23.146339Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:23.146349Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:23.146361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:23.146410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:23.146445Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342686108182814:2769]
2025-10-09T21:50:23.147036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:23.147343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:23.147395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:23.147710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:23.147780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.147787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.147788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.147789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.147790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.147791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.147792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.147793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.147794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.147799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.147810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.147814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.147818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.147822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.147826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.147829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.147832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.147836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.147842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.147851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.147861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.147865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.147870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.147875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.147879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.147883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.147887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.147891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.147895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.147899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.147910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.147914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.147918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.147922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.147930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.147934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.147968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:23.148768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342686470014262:5942] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:23.148782Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342686470014262:5942] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:23.148786Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342686470014262:5942] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:23.148794Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1810 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:23.154614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:23.155717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:23.155736Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:23.155739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:23.155743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:23.155751Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1809 [2:7559342686108182814:2769]
2025-10-09T21:50:23.155765Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:23.155786Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:23.155789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:23.155793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:23.155795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:23.155813Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:23.155820Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:23.155827Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:23.155883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.155891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.155893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.155896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:23.155968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:23.155977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.155991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.156004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.156021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:23.156047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342686470014299:5942], now have 1 active actors on pipe
2025-10-09T21:50:23.156057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342686470014164:2432], now have 1 active actors on pipe
2025-10-09T21:50:23.156065Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342686108182808:2769]
2025-10-09T21:50:23.156067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342686108182808:2769]
2025-10-09T21:50:23.156070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342686108182808:2769]
2025-10-09T21:50:23.156084Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:23.156103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:23.156105Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:23.156108Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:23.156109Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:23.156121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:23.156125Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:23.156116Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1809
2025-10-09T21:50:23.156121Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1809
2025-10-09T21:50:23.156161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342686108182808:2769]
2025-10-09T21:50:23.156173Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:23.156177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:23.156193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:23.156203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:23.156251Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b3ab3217-2002ca13-aec0a7fa-93a3b427_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:23.156263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:23.156267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:23.156270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:23.156281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:23.156294Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:23.156317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:23.156479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:23.156488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:23.156502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:23.156642Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1810 partition: 0 MaxSeqNo: 25 sessionId: src_id|b3ab3217-2002ca13-aec0a7fa-93a3b427_0
2025-10-09T21:50:23.157036Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac732cc3-c74fed04-fce069ab-ea4c32d9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b3ab3217-2002ca13-aec0a7fa-93a3b427_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:23.159600Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3ab3217-2002ca13-aec0a7fa-93a3b427_0] Write session: try to update token
2025-10-09T21:50:23.159617Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3ab3217-2002ca13-aec0a7fa-93a3b427_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:23.170811Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:23.170832Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:23.170834Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:23.170835Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:23.170842Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342686108182808:2769]
2025-10-09T21:50:23.170845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342686108182808:2769]
2025-10-09T21:50:23.170847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342686108182808:2769]
2025-10-09T21:50:23.170883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:23.170890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.170892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:23.170896Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:23.170903Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:23.170907Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:23.171072Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:23.172388Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1810 sessionId: src_id|b3ab3217-2002ca13-aec0a7fa-93a3b427_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:23.174798Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:23.234068Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342686470014311:6018]
2025-10-09T21:50:23.234315Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 1809, pipe: [1:7559342686470014311:6018]
2025-10-09T21:50:23.320109Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:23.320538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:23.320556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:23.320596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342686470014311:6018], now have 1 active actors on pipe
2025-10-09T21:50:23.320802Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:23.320898Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:23.320912Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342686470014311:6018]
2025-10-09T21:50:23.320916Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:23.320935Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:23.320987Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1810 reason: error at write: Disconnected sessionId: src_id|b3ab3217-2002ca13-aec0a7fa-93a3b427_0
2025-10-09T21:50:23.321160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1810 sessionId: src_id|b3ab3217-2002ca13-aec0a7fa-93a3b427_0 is DEAD
2025-10-09T21:50:23.321312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3ab3217-2002ca13-aec0a7fa-93a3b427_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:23.321330Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3ab3217-2002ca13-aec0a7fa-93a3b427_0] Write session will restart in 0.009957s
2025-10-09T21:50:23.321352Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3ab3217-2002ca13-aec0a7fa-93a3b427_0] Write session: Do CDS request
2025-10-09T21:50:23.321357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3ab3217-2002ca13-aec0a7fa-93a3b427_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:23.332220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:23.332563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3ab3217-2002ca13-aec0a7fa-93a3b427_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:23.332627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:23.332645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1811
2025-10-09T21:50:23.332764Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1811 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:23.332799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1811 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:23.332811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1811 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:23.332815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:23.333266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1811 sessionId:  describe result for acl check
2025-10-09T21:50:23.333314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:23.333318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:23.333320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:23.333325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342686470014421:5940] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:23.333917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342686470014421:5940] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:23.334360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:23.334550Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:23.334559Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:23.334571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:23.334631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:23.334671Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342686108182953:2748]
2025-10-09T21:50:23.335261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:23.335547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:23.335595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:23.335639Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342686470014421:5940] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:23.335925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:23.335986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.335992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.335994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.335995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.335996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.335997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.335999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.336000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.336001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.336006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.336017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.336021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.336026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.336030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.336034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.336038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.336042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.336046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.336052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.336060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.336066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.336071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.336076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.336084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.336089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.336094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.336102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.336107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.336116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.336121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.336126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.336133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.336140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.336145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.336152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.336157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.336189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:23.341198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342686470014421:5940] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:23.341218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342686470014421:5940] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:23.341222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342686470014421:5940] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:23.341232Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1811 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:23.342759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:23.344029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:23.344046Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:23.344048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:23.344052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:23.344058Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1810 [2:7559342686108182953:2748]
2025-10-09T21:50:23.344075Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:23.344093Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:23.344096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:23.344099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:23.344102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:23.344120Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:23.344132Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:23.344139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:23.344196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.344203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.344205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.344207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:23.344279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:23.344287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.344298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.344309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.344331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:23.344356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342686470014318:2432], now have 1 active actors on pipe
2025-10-09T21:50:23.344364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342686470014460:5940], now have 1 active actors on pipe
2025-10-09T21:50:23.344371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342686108182947:2748]
2025-10-09T21:50:23.344374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342686108182947:2748]
2025-10-09T21:50:23.344376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342686108182947:2748]
2025-10-09T21:50:23.344391Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:23.344410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:23.344391Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1810
2025-10-09T21:50:23.344417Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:23.344421Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:23.344423Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:23.344429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:23.344393Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1810
2025-10-09T21:50:23.344433Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:23.344477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:23.344486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:23.344499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342686108182947:2748]
2025-10-09T21:50:23.344510Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:23.344522Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:23.344544Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6ca6265e-7cfcaea9-6d65b908-64168b72_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:23.344555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:23.344559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:23.344561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:23.344572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:23.344586Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:23.344612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:23.344852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:23.344867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:23.344889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:23.344989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1811 partition: 0 MaxSeqNo: 25 sessionId: src_id|6ca6265e-7cfcaea9-6d65b908-64168b72_0
2025-10-09T21:50:23.345372Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3ab3217-2002ca13-aec0a7fa-93a3b427_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6ca6265e-7cfcaea9-6d65b908-64168b72_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:23.348063Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ca6265e-7cfcaea9-6d65b908-64168b72_0] Write session: try to update token
2025-10-09T21:50:23.348079Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ca6265e-7cfcaea9-6d65b908-64168b72_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:23.358675Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:23.358693Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:23.358694Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:23.358696Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:23.358704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342686108182947:2748]
2025-10-09T21:50:23.358707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342686108182947:2748]
2025-10-09T21:50:23.358710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342686108182947:2748]
2025-10-09T21:50:23.358746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:23.358751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.358754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:23.358770Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:23.358776Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:23.358783Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:23.358880Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:23.361050Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1811 sessionId: src_id|6ca6265e-7cfcaea9-6d65b908-64168b72_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:23.363346Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:23.420919Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342686470014467:6018]
2025-10-09T21:50:23.421245Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 1810, pipe: [1:7559342686470014467:6018]
2025-10-09T21:50:23.510026Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:23.510352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:23.510365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:23.510403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342686470014467:6018], now have 1 active actors on pipe
2025-10-09T21:50:23.510619Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:23.510651Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:23.510660Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:23.510666Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:23.510723Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342686470014467:6018]
2025-10-09T21:50:23.510768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1811 reason: error at write: Disconnected sessionId: src_id|6ca6265e-7cfcaea9-6d65b908-64168b72_0
2025-10-09T21:50:23.511012Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1811 sessionId: src_id|6ca6265e-7cfcaea9-6d65b908-64168b72_0 is DEAD
2025-10-09T21:50:23.511133Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ca6265e-7cfcaea9-6d65b908-64168b72_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:23.511143Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ca6265e-7cfcaea9-6d65b908-64168b72_0] Write session will restart in 0.007197s
2025-10-09T21:50:23.511162Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ca6265e-7cfcaea9-6d65b908-64168b72_0] Write session: Do CDS request
2025-10-09T21:50:23.511167Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ca6265e-7cfcaea9-6d65b908-64168b72_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:23.519507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ca6265e-7cfcaea9-6d65b908-64168b72_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:23.519591Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:23.519609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1812
2025-10-09T21:50:23.519766Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1812 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:23.519796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1812 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:23.519806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1812 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:23.519811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:23.520213Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1812 sessionId:  describe result for acl check
2025-10-09T21:50:23.520250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:23.520256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:23.520257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:23.520262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342686470014565:5935] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:23.520794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342686470014565:5935] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:23.521421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:23.522764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342686470014565:5935] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:23.523472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:23.523662Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:23.523671Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:23.523680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:23.523728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:23.523768Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342686108183093:2746]
2025-10-09T21:50:23.524176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:23.524542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:23.524603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:23.524841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342686470014565:5935] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:23.524862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342686470014565:5935] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:23.524865Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342686470014565:5935] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:23.524872Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1812 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:23.524955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:23.525005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.525011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.525012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.525013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.525015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.525016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.525017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.525018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.525019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.525024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.525035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.525039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.525043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.525047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.525051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.525055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.525059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.525063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.525069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.525077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.525084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.525089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.525093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.525098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.525103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.525108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.525113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.525123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.525127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.525132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.525136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.525151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.525155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.525164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.525169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.525174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.525204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:23.531541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:23.532503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:23.532515Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:23.532517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:23.532521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:23.532527Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1811 [2:7559342686108183093:2746]
2025-10-09T21:50:23.532540Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:23.532555Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:23.532557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:23.532559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:23.532561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:23.532573Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:23.532578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:23.532583Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:23.532623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.532630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.532632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.532635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:23.532698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:23.532707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.532718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.532729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.532750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:23.532775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342686470014621:5935], now have 1 active actors on pipe
2025-10-09T21:50:23.532783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342686470014470:2432], now have 1 active actors on pipe
2025-10-09T21:50:23.532789Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342686108183087:2746]
2025-10-09T21:50:23.532791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342686108183087:2746]
2025-10-09T21:50:23.532792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342686108183087:2746]
2025-10-09T21:50:23.532801Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:23.532815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:23.532823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:23.532829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:23.532829Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:23.532833Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:23.532835Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:23.532849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342686108183087:2746]
2025-10-09T21:50:23.532854Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:23.532864Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:23.532849Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1811
2025-10-09T21:50:23.532850Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1811
2025-10-09T21:50:23.532889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:23.532895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:23.532918Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7dbfaf2b-c333276a-a0980cfa-41d2e10f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:23.532926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:23.532929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:23.532931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:23.532938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:23.532946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:23.532957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:23.533065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:23.533071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:23.533081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:23.533147Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1812 partition: 0 MaxSeqNo: 25 sessionId: src_id|7dbfaf2b-c333276a-a0980cfa-41d2e10f_0
2025-10-09T21:50:23.533440Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ca6265e-7cfcaea9-6d65b908-64168b72_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7dbfaf2b-c333276a-a0980cfa-41d2e10f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:23.535889Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7dbfaf2b-c333276a-a0980cfa-41d2e10f_0] Write session: try to update token
2025-10-09T21:50:23.535901Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7dbfaf2b-c333276a-a0980cfa-41d2e10f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:23.544224Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:23.544243Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:23.544254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342686108183087:2746]
2025-10-09T21:50:23.544299Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:23.544309Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:23.544477Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:23.544634Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:23.544774Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:23.544921Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:23.545118Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:23.545162Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:23.548293Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1812 sessionId: src_id|7dbfaf2b-c333276a-a0980cfa-41d2e10f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:23.550568Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:23.550772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:23.550794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:23.550835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:23.550937Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:23.556836Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:23.556851Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:23.556854Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:23.556856Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:23.556865Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342686108183087:2746]
2025-10-09T21:50:23.556869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342686108183087:2746]
2025-10-09T21:50:23.556873Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342686108183087:2746]
2025-10-09T21:50:23.556904Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:23.556912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:23.556913Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:23.556916Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:23.556918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.556920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:23.557048Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:23.610868Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342686470014630:6018]
2025-10-09T21:50:23.611159Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 1811, pipe: [1:7559342686470014630:6018]
2025-10-09T21:50:23.700656Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:23.708568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:23.708597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:23.708776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:23.708848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:23.708924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:23.708996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:23.709050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:23.709062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:23.709150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:23.709220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:23.709293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:23.709363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:23.709409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:23.709417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:23.709497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:23.709569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:23.709633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:23.709700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:23.709742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:23.709752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:23.709828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:23.709895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:23.709963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:23.710086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:23.710132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:23.710139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:23.710207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:23.710271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:23.710340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:23.710403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:23.710454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:23.710461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:23.710526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:23.710606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:23.710668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:23.710731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:23.710775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:23.710782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:23.710847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:23.710907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:23.710973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:23.711037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:23.711082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:23.711089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:23.711155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:23.711219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:23.711281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:23.711348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:23.711407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:23.711414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:23.711477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:23.711537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:23.711603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:23.711670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:23.711713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:23.711719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:23.711785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:23.711849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:23.711914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:23.712024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:23.712069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:23.712076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:23.712137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342686470014630:6018], now have 1 active actors on pipe
2025-10-09T21:50:23.712395Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:23.712406Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342686470014630:6018]
2025-10-09T21:50:23.712411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:23.712453Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1812 reason: error at write: Disconnected sessionId: src_id|7dbfaf2b-c333276a-a0980cfa-41d2e10f_0
2025-10-09T21:50:23.712473Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:23.712611Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1812 sessionId: src_id|7dbfaf2b-c333276a-a0980cfa-41d2e10f_0 is DEAD
2025-10-09T21:50:23.712757Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7dbfaf2b-c333276a-a0980cfa-41d2e10f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:23.712767Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7dbfaf2b-c333276a-a0980cfa-41d2e10f_0] Write session will restart in 0.007011s
2025-10-09T21:50:23.712785Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7dbfaf2b-c333276a-a0980cfa-41d2e10f_0] Write session: Do CDS request
2025-10-09T21:50:23.712791Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7dbfaf2b-c333276a-a0980cfa-41d2e10f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:23.712969Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:23.720563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7dbfaf2b-c333276a-a0980cfa-41d2e10f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:23.720648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:23.720676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1813
2025-10-09T21:50:23.720838Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1813 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:23.720897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1813 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:23.720910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1813 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:23.720914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:23.721445Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1813 sessionId:  describe result for acl check
2025-10-09T21:50:23.721498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:23.721500Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:23.721502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:23.721508Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342686470014648:5924] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:23.722153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342686470014648:5924] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:23.725517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342686470014648:5924] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:23.728155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342686470014648:5924] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:23.728168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342686470014648:5924] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:23.728172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342686470014648:5924] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:23.728180Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1813 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:23.728331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:23.729999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:23.730193Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:23.730203Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:23.730212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:23.730260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:23.730295Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342686108183236:2743]
2025-10-09T21:50:23.730887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:23.731150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:23.731201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:23.731510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:23.731567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.731575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.731577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.731578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.731579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.731581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.731582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.731584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.731586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.731591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.731602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.731606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.731610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.731614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.731618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.731622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.731626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.731636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.731643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.731654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.731658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.731662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.731670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.731674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.731677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.731681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.731689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.731693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.731696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.731699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.731702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.731706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.731709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.731713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.731716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.731720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.731739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:23.738972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:23.740110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:23.740123Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:23.740125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:23.740129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:23.740135Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1812 [2:7559342686108183236:2743]
2025-10-09T21:50:23.740148Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:23.740163Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:23.740174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:23.740178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:23.740180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:23.740199Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:23.740206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:23.740213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:23.740280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.740287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.740289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.740292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:23.740366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:23.740373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.740385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.740396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.740474Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1812
2025-10-09T21:50:23.740418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:23.740482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1812
2025-10-09T21:50:23.740443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342686470014633:2432], now have 1 active actors on pipe
2025-10-09T21:50:23.740447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342686470014772:5924], now have 1 active actors on pipe
2025-10-09T21:50:23.740455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342686108183230:2743]
2025-10-09T21:50:23.740458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342686108183230:2743]
2025-10-09T21:50:23.740460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342686108183230:2743]
2025-10-09T21:50:23.740478Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:23.740492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:23.740508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:23.740511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:23.740510Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:23.740517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:23.740521Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:23.740553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342686108183230:2743]
2025-10-09T21:50:23.740557Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:23.740571Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:23.740574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:23.740579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:23.740634Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|39a9eb2d-3d47cfe-e531b4b3-ed784fba_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:23.740647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:23.740655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:23.740659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:23.740669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:23.740684Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:23.740707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:23.740844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:23.740852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:23.740865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:23.740953Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1813 partition: 0 MaxSeqNo: 25 sessionId: src_id|39a9eb2d-3d47cfe-e531b4b3-ed784fba_0
2025-10-09T21:50:23.741269Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7dbfaf2b-c333276a-a0980cfa-41d2e10f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|39a9eb2d-3d47cfe-e531b4b3-ed784fba_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:23.743908Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|39a9eb2d-3d47cfe-e531b4b3-ed784fba_0] Write session: try to update token
2025-10-09T21:50:23.743929Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|39a9eb2d-3d47cfe-e531b4b3-ed784fba_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:23.755656Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:23.755677Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:23.755679Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:23.755681Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:23.755690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342686108183230:2743]
2025-10-09T21:50:23.755693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342686108183230:2743]
2025-10-09T21:50:23.755697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342686108183230:2743]
2025-10-09T21:50:23.755736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:23.755736Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:23.755742Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:23.755743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.755745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:23.755745Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:23.755885Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:23.757188Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1813 sessionId: src_id|39a9eb2d-3d47cfe-e531b4b3-ed784fba_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:23.759589Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:23.812605Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342686470014784:6018]
2025-10-09T21:50:23.812901Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 1812, pipe: [1:7559342686470014784:6018]
2025-10-09T21:50:23.903564Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:23.903903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:23.903914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:23.903952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342686470014784:6018], now have 1 active actors on pipe
2025-10-09T21:50:23.904115Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:23.904183Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:23.904201Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:23.904231Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342686470014784:6018]
2025-10-09T21:50:23.904249Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:23.904268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1813 reason: error at write: Disconnected sessionId: src_id|39a9eb2d-3d47cfe-e531b4b3-ed784fba_0
2025-10-09T21:50:23.904445Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1813 sessionId: src_id|39a9eb2d-3d47cfe-e531b4b3-ed784fba_0 is DEAD
2025-10-09T21:50:23.904601Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39a9eb2d-3d47cfe-e531b4b3-ed784fba_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:23.904612Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39a9eb2d-3d47cfe-e531b4b3-ed784fba_0] Write session will restart in 0.007904s
2025-10-09T21:50:23.904639Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39a9eb2d-3d47cfe-e531b4b3-ed784fba_0] Write session: Do CDS request
2025-10-09T21:50:23.904654Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39a9eb2d-3d47cfe-e531b4b3-ed784fba_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:23.913523Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|39a9eb2d-3d47cfe-e531b4b3-ed784fba_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:23.913589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:23.913604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1814
2025-10-09T21:50:23.913714Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1814 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:23.913758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1814 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:23.913768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1814 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:23.913772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:23.914265Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1814 sessionId:  describe result for acl check
2025-10-09T21:50:23.914309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:23.914311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:23.914313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:23.914319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342686470014885:5913] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:23.914927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342686470014885:5913] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:23.915645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:23.917002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342686470014885:5913] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:23.917709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:23.917871Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:23.917881Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:23.917890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:23.917929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:23.917959Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342686108183381:2740]
2025-10-09T21:50:23.918366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:23.918676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:23.918725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:23.919040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:23.919104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.919110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.919112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.919113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.919114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.919115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.919116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.919118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.919119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.919124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.919131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.919133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.919149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342686470014885:5913] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:23.919135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.919161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342686470014885:5913] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:23.919137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.919164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342686470014885:5913] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:23.919170Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1814 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:23.919139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.919141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.919143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.919145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.919150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.919155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:23.919159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.919163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:23.919166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.919169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:23.919171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.919174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:23.919177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.919181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:23.919183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.919186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:23.919191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.919194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.919197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.919200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.919202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.919205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.919221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:23.925973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:23.927017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:23.927032Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:23.927035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:23.927039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:23.927046Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1813 [2:7559342686108183381:2740]
2025-10-09T21:50:23.927059Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:23.927076Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:23.927084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:23.927087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:23.927090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:23.927106Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:23.927117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:23.927125Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:23.927191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.927198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.927200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.927203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:23.927272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:23.927280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.927292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:23.927309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:23.927331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:23.927357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342686470014931:5913], now have 1 active actors on pipe
2025-10-09T21:50:23.927366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342686470014789:2432], now have 1 active actors on pipe
2025-10-09T21:50:23.927373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342686108183375:2740]
2025-10-09T21:50:23.927380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342686108183375:2740]
2025-10-09T21:50:23.927382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342686108183375:2740]
2025-10-09T21:50:23.927396Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:23.927410Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:23.927415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:23.927419Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:23.927421Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:23.927431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:23.927395Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1813
2025-10-09T21:50:23.927434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:23.927406Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1813
2025-10-09T21:50:23.927480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:23.927489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:23.927502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342686108183375:2740]
2025-10-09T21:50:23.927513Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:23.927516Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:23.927548Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5fa5303f-685e24a6-53f4cb67-e22740bd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:23.927560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:23.927568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:23.927571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:23.927581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:23.927595Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:23.927623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:23.927779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:23.927787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:23.927801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:23.927880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1814 partition: 0 MaxSeqNo: 25 sessionId: src_id|5fa5303f-685e24a6-53f4cb67-e22740bd_0
2025-10-09T21:50:23.928205Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|39a9eb2d-3d47cfe-e531b4b3-ed784fba_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5fa5303f-685e24a6-53f4cb67-e22740bd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:23.930456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5fa5303f-685e24a6-53f4cb67-e22740bd_0] Write session: try to update token
2025-10-09T21:50:23.930472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5fa5303f-685e24a6-53f4cb67-e22740bd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:23.942236Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1814 sessionId: src_id|5fa5303f-685e24a6-53f4cb67-e22740bd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:23.942845Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:23.942864Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:23.942866Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:23.942868Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:23.942877Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342686108183375:2740]
2025-10-09T21:50:23.942882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342686108183375:2740]
2025-10-09T21:50:23.942886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342686108183375:2740]
2025-10-09T21:50:23.942934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:23.942933Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:23.942946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:23.942949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:23.942950Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:23.942954Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:23.943081Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:23.944396Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:24.004542Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342690764982237:6018]
2025-10-09T21:50:24.004793Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 1813, pipe: [1:7559342690764982237:6018]
2025-10-09T21:50:24.090862Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:24.091348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:24.091362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:24.091401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342690764982237:6018], now have 1 active actors on pipe
2025-10-09T21:50:24.091557Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:24.091592Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:24.091593Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:24.091597Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:24.091668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1814 reason: error at write: Disconnected sessionId: src_id|5fa5303f-685e24a6-53f4cb67-e22740bd_0
2025-10-09T21:50:24.091668Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342690764982237:6018]
2025-10-09T21:50:24.091852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1814 sessionId: src_id|5fa5303f-685e24a6-53f4cb67-e22740bd_0 is DEAD
2025-10-09T21:50:24.091991Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fa5303f-685e24a6-53f4cb67-e22740bd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:24.092002Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fa5303f-685e24a6-53f4cb67-e22740bd_0] Write session will restart in 0.008686s
2025-10-09T21:50:24.092028Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fa5303f-685e24a6-53f4cb67-e22740bd_0] Write session: Do CDS request
2025-10-09T21:50:24.092035Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fa5303f-685e24a6-53f4cb67-e22740bd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:24.101496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5fa5303f-685e24a6-53f4cb67-e22740bd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:24.101590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:24.101608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1815
2025-10-09T21:50:24.101731Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1815 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:24.101763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1815 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:24.101773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1815 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:24.101777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:24.102238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1815 sessionId:  describe result for acl check
2025-10-09T21:50:24.102279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:24.102286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:24.102289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:24.102295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342690764982344:5922] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:24.102441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:24.102768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342690764982344:5922] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:24.104350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:24.104552Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:24.104564Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:24.104577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:24.104642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:24.104687Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342690403150826:2734]
2025-10-09T21:50:24.104823Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342690764982344:5922] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:24.105216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:24.105490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:24.105540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:24.105823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:24.105887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.105893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.105895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.105896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.105897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.105899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.105900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.105901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.105902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.105906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.105912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.105914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.105916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.105918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.105921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.105923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.105925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.105927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.105931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.105936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.105940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.105943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.105946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.105949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.105953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.105956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.105960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.105964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.105969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.105973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.105980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.105985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.105989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.105995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.106002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.106007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.106051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:24.111182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342690764982344:5922] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:24.111193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342690764982344:5922] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:24.111196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342690764982344:5922] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:24.111202Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1815 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:24.112890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:24.113976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:24.113990Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:24.113992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:24.113996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:24.114001Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1814 [2:7559342690403150826:2734]
2025-10-09T21:50:24.114015Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:24.114049Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:24.114057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:24.114060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:24.114062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:24.114077Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:24.114082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:24.114092Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:24.114143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.114150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.114152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.114155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:24.114219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:24.114227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.114239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.114252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.114272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:24.114298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342690764982396:5922], now have 1 active actors on pipe
2025-10-09T21:50:24.114306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342690764982239:2432], now have 1 active actors on pipe
2025-10-09T21:50:24.114313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342690403150818:2734]
2025-10-09T21:50:24.114316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342690403150818:2734]
2025-10-09T21:50:24.114317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342690403150818:2734]
2025-10-09T21:50:24.114329Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:24.114345Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:24.114346Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:24.114348Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:24.114351Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:24.114360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:24.114364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:24.114374Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1814
2025-10-09T21:50:24.114409Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1814
2025-10-09T21:50:24.114390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342690403150818:2734]
2025-10-09T21:50:24.114397Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:24.114399Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:24.114421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:24.114426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:24.114519Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|18950162-480739f5-116cd4b8-de995f2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:24.114533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:24.114537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:24.114539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:24.114550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:24.114559Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:24.114584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:24.114822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:24.114832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:24.114851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:24.114992Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1815 partition: 0 MaxSeqNo: 25 sessionId: src_id|18950162-480739f5-116cd4b8-de995f2_0
2025-10-09T21:50:24.115338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5fa5303f-685e24a6-53f4cb67-e22740bd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|18950162-480739f5-116cd4b8-de995f2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:24.117679Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|18950162-480739f5-116cd4b8-de995f2_0] Write session: try to update token
2025-10-09T21:50:24.117697Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|18950162-480739f5-116cd4b8-de995f2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:24.128897Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:24.128915Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:24.128916Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:24.128919Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:24.128928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342690403150818:2734]
2025-10-09T21:50:24.128940Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342690403150818:2734]
2025-10-09T21:50:24.128942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342690403150818:2734]
2025-10-09T21:50:24.128977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:24.128982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.128983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:24.129015Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:24.129029Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:24.129033Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:24.129136Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:24.130677Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1815 sessionId: src_id|18950162-480739f5-116cd4b8-de995f2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:24.133158Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:24.191949Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342690764982417:6018]
2025-10-09T21:50:24.192206Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 1814, pipe: [1:7559342690764982417:6018]
2025-10-09T21:50:24.278640Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:24.278971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:24.278985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:24.279027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342690764982417:6018], now have 1 active actors on pipe
2025-10-09T21:50:24.279271Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:24.279266Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342690764982417:6018]
2025-10-09T21:50:24.279275Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:24.279280Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:24.279298Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:24.279345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1815 reason: error at write: Disconnected sessionId: src_id|18950162-480739f5-116cd4b8-de995f2_0
2025-10-09T21:50:24.279560Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1815 sessionId: src_id|18950162-480739f5-116cd4b8-de995f2_0 is DEAD
2025-10-09T21:50:24.279643Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18950162-480739f5-116cd4b8-de995f2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:24.279652Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18950162-480739f5-116cd4b8-de995f2_0] Write session will restart in 0.008391s
2025-10-09T21:50:24.279673Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18950162-480739f5-116cd4b8-de995f2_0] Write session: Do CDS request
2025-10-09T21:50:24.279678Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18950162-480739f5-116cd4b8-de995f2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:24.289379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:24.289495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|18950162-480739f5-116cd4b8-de995f2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:24.289966Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:24.289984Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1816
2025-10-09T21:50:24.290140Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1816 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:24.290169Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1816 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:24.290176Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1816 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:24.290179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:24.290540Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1816 sessionId:  describe result for acl check
2025-10-09T21:50:24.290569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:24.290570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:24.290572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:24.290576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342690764982531:5903] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:24.290991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342690764982531:5903] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:24.291205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:24.291366Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:24.291373Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:24.291382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:24.291424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:24.291460Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342690403151002:2717]
2025-10-09T21:50:24.291818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:24.292174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:24.292229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:24.292455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:24.292489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.292493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.292494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.292495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.292496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.292496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.292497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.292498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.292499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.292502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.292508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.292510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.292512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.292514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.292517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.292519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.292521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.292523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.292528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.292533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.292538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.292541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.292544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.292547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.292549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.292552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.292555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.292558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.292560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.292563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.292566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.292570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.292572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.292575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.292578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.292580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.292596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:24.293310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342690764982531:5903] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:24.298422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342690764982531:5903] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:24.298437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342690764982531:5903] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:24.298440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342690764982531:5903] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:24.298446Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1816 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:24.298730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:24.299946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:24.299956Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:24.299958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:24.299960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:24.299966Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1815 [2:7559342690403151002:2717]
2025-10-09T21:50:24.299974Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:24.299988Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:24.299989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:24.299991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:24.299993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:24.300005Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:24.300009Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:24.300015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:24.300055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.300063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.300064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.300066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:24.300117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:24.300124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.300130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.300137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.300146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:24.300159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342690403150996:2717]
2025-10-09T21:50:24.300160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342690403150996:2717]
2025-10-09T21:50:24.300163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342690403150996:2717]
2025-10-09T21:50:24.300168Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:24.300185Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:24.300186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:24.300188Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:24.300190Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:24.300194Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:24.300196Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:24.300205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342690764982424:2432], now have 1 active actors on pipe
2025-10-09T21:50:24.300208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342690764982570:5903], now have 1 active actors on pipe
2025-10-09T21:50:24.300227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342690403150996:2717]
2025-10-09T21:50:24.300233Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:24.300251Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:24.300299Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1815
2025-10-09T21:50:24.300303Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1815
2025-10-09T21:50:24.300358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:24.300370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:24.300402Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7667fc45-23ae10d1-f786be08-43fa3853_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:24.300416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:24.300420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:24.300422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:24.300433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:24.300444Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:24.300456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:24.300675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:24.300682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:24.300692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:24.300762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1816 partition: 0 MaxSeqNo: 25 sessionId: src_id|7667fc45-23ae10d1-f786be08-43fa3853_0
2025-10-09T21:50:24.301084Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|18950162-480739f5-116cd4b8-de995f2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7667fc45-23ae10d1-f786be08-43fa3853_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:24.303544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7667fc45-23ae10d1-f786be08-43fa3853_0] Write session: try to update token
2025-10-09T21:50:24.303558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7667fc45-23ae10d1-f786be08-43fa3853_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:24.314277Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:24.314296Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:24.314298Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:24.314300Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:24.314309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342690403150996:2717]
2025-10-09T21:50:24.314313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342690403150996:2717]
2025-10-09T21:50:24.314317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342690403150996:2717]
2025-10-09T21:50:24.314361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:24.314376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.314379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:24.314384Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:24.314390Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:24.314395Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:24.314513Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:24.316214Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1816 sessionId: src_id|7667fc45-23ae10d1-f786be08-43fa3853_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:24.318569Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:24.379376Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342690764982578:6018]
2025-10-09T21:50:24.379577Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 1815, pipe: [1:7559342690764982578:6018]
2025-10-09T21:50:24.460297Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:24.460713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:24.460726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:24.460760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342690764982578:6018], now have 1 active actors on pipe
2025-10-09T21:50:24.460955Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:24.460959Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342690764982578:6018]
2025-10-09T21:50:24.460982Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:24.460978Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:24.460984Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:24.461055Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1816 reason: error at write: Disconnected sessionId: src_id|7667fc45-23ae10d1-f786be08-43fa3853_0
2025-10-09T21:50:24.461237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1816 sessionId: src_id|7667fc45-23ae10d1-f786be08-43fa3853_0 is DEAD
2025-10-09T21:50:24.461379Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7667fc45-23ae10d1-f786be08-43fa3853_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:24.461391Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7667fc45-23ae10d1-f786be08-43fa3853_0] Write session will restart in 0.007551s
2025-10-09T21:50:24.461414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7667fc45-23ae10d1-f786be08-43fa3853_0] Write session: Do CDS request
2025-10-09T21:50:24.461420Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7667fc45-23ae10d1-f786be08-43fa3853_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:24.470411Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7667fc45-23ae10d1-f786be08-43fa3853_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:24.470537Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:24.470551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1817
2025-10-09T21:50:24.470619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:24.470647Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1817 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:24.470677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1817 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:24.470688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1817 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:24.470696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:24.471113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1817 sessionId:  describe result for acl check
2025-10-09T21:50:24.471149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:24.471156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:24.471158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:24.471164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342690764982687:5891] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:24.471758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342690764982687:5891] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:24.472256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:24.472439Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:24.472450Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:24.472464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:24.472534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:24.472572Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342690403151146:2715]
2025-10-09T21:50:24.473144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:24.473372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:24.473422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:24.473655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342690764982687:5891] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:24.473744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:24.473802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.473809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.473810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.473811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.473812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.473814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.473815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.473816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.473817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.473821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.473832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.473837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.473841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.473845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.473849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.473853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.473858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.473861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.473868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.473876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.473885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.473891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.473898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.473903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.473907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.473914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.473919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.473930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.473936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.473943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.473947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.473952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.473957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.473962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.473967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.473977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.474008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:24.478694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342690764982687:5891] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:24.478709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342690764982687:5891] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:24.478712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342690764982687:5891] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:24.478735Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1817 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:24.480126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:24.481162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:24.481179Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:24.481181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:24.481185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:24.481191Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1816 [2:7559342690403151146:2715]
2025-10-09T21:50:24.481204Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:24.481226Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:24.481232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:24.481235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:24.481238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:24.481253Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:24.481265Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:24.481272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:24.481324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.481333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.481336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.481339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:24.481402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:24.481409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.481421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.481433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.481454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:24.481487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342690764982580:2432], now have 1 active actors on pipe
2025-10-09T21:50:24.481498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342690764982727:5891], now have 1 active actors on pipe
2025-10-09T21:50:24.481506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342690403151141:2715]
2025-10-09T21:50:24.481508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342690403151141:2715]
2025-10-09T21:50:24.481510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342690403151141:2715]
2025-10-09T21:50:24.481523Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:24.481507Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1816
2025-10-09T21:50:24.481518Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1816
2025-10-09T21:50:24.481536Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:24.481541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:24.481546Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:24.481549Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:24.481550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:24.481552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:24.481589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:24.481596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:24.481606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342690403151141:2715]
2025-10-09T21:50:24.481612Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:24.481626Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:24.481641Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|19f4e4c-b82249c8-f2919b90-4148144c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:24.481653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:24.481661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:24.481664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:24.481673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:24.481687Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:24.481715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:24.481851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:24.481862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:24.481877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:24.481947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1817 partition: 0 MaxSeqNo: 25 sessionId: src_id|19f4e4c-b82249c8-f2919b90-4148144c_0
2025-10-09T21:50:24.482214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7667fc45-23ae10d1-f786be08-43fa3853_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|19f4e4c-b82249c8-f2919b90-4148144c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:24.484717Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19f4e4c-b82249c8-f2919b90-4148144c_0] Write session: try to update token
2025-10-09T21:50:24.484734Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19f4e4c-b82249c8-f2919b90-4148144c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:24.495904Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:24.495923Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:24.495925Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:24.495927Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:24.495935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342690403151141:2715]
2025-10-09T21:50:24.495940Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342690403151141:2715]
2025-10-09T21:50:24.495946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342690403151141:2715]
2025-10-09T21:50:24.495982Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:24.495988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:24.495994Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:24.495994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.495997Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:24.495997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:24.496126Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:24.497309Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1817 sessionId: src_id|19f4e4c-b82249c8-f2919b90-4148144c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:24.499589Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:24.561108Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342690764982739:6018]
2025-10-09T21:50:24.561358Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 1816, pipe: [1:7559342690764982739:6018]
2025-10-09T21:50:24.637939Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:24.638412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:24.638426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:24.638464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342690764982739:6018], now have 1 active actors on pipe
2025-10-09T21:50:24.638616Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:24.638656Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:24.638658Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:24.638677Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:24.638710Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342690764982739:6018]
2025-10-09T21:50:24.638745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1817 reason: error at write: Disconnected sessionId: src_id|19f4e4c-b82249c8-f2919b90-4148144c_0
2025-10-09T21:50:24.638928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1817 sessionId: src_id|19f4e4c-b82249c8-f2919b90-4148144c_0 is DEAD
2025-10-09T21:50:24.639099Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19f4e4c-b82249c8-f2919b90-4148144c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:24.639105Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19f4e4c-b82249c8-f2919b90-4148144c_0] Write session will restart in 0.007391s
2025-10-09T21:50:24.639129Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19f4e4c-b82249c8-f2919b90-4148144c_0] Write session: Do CDS request
2025-10-09T21:50:24.639134Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19f4e4c-b82249c8-f2919b90-4148144c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:24.647509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19f4e4c-b82249c8-f2919b90-4148144c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:24.647615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:24.647630Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1818
2025-10-09T21:50:24.647752Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1818 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:24.647784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1818 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:24.647794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1818 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:24.647798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:24.648171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1818 sessionId:  describe result for acl check
2025-10-09T21:50:24.648202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:24.648208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:24.648210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:24.648216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342690764982847:5886] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:24.648744Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342690764982847:5886] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:24.648866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:24.650468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342690764982847:5886] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:24.650687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:24.650835Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:24.650843Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:24.650851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:24.650888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:24.650915Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342690403151293:2713]
2025-10-09T21:50:24.651258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:24.651485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:24.651511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:24.651688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:24.651719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.651723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.651723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.651724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.651725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.651725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.651726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.651726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.651727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.651730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.651735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.651737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.651739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.651741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.651743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.651745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.651747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.651748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.651753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.651758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.651761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.651764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.651766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.651769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.651771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.651774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.651776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.651779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.651781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.651784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.651786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.651789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.651791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.651794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.651797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.651799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.651814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:24.656159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342690764982847:5886] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:24.656172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342690764982847:5886] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:24.656175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342690764982847:5886] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:24.656183Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1818 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:24.657532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:24.658514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:24.658526Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:24.658528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:24.658530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:24.658534Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1817 [2:7559342690403151293:2713]
2025-10-09T21:50:24.658542Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:24.658554Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:24.658556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:24.658558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:24.658559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:24.658571Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:24.658576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:24.658581Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:24.658616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.658624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.658625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.658627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:24.658673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:24.658679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.658685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.658691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.658700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:24.658713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342690764982740:2432], now have 1 active actors on pipe
2025-10-09T21:50:24.658719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342690764982889:5886], now have 1 active actors on pipe
2025-10-09T21:50:24.658724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342690403151287:2713]
2025-10-09T21:50:24.658726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342690403151287:2713]
2025-10-09T21:50:24.658727Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342690403151287:2713]
2025-10-09T21:50:24.658735Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:24.658744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:24.658744Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:24.658746Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:24.658747Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:24.658751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:24.658753Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:24.658769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342690403151287:2713]
2025-10-09T21:50:24.658777Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:24.658781Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:24.658786Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1817
2025-10-09T21:50:24.658795Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1817
2025-10-09T21:50:24.658822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:24.658830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:24.658851Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fea5f1de-b07cd69e-b3c3c8c-eb295c2b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:24.658859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:24.658862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:24.658863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:24.658871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:24.658880Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:24.658893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:24.659024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:24.659029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:24.659038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:24.659125Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1818 partition: 0 MaxSeqNo: 25 sessionId: src_id|fea5f1de-b07cd69e-b3c3c8c-eb295c2b_0
2025-10-09T21:50:24.659395Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19f4e4c-b82249c8-f2919b90-4148144c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fea5f1de-b07cd69e-b3c3c8c-eb295c2b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:24.662126Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fea5f1de-b07cd69e-b3c3c8c-eb295c2b_0] Write session: try to update token
2025-10-09T21:50:24.662140Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fea5f1de-b07cd69e-b3c3c8c-eb295c2b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:24.669080Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:24.669096Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:24.669105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342690403151287:2713]
2025-10-09T21:50:24.669141Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:24.669145Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:24.669303Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:24.669440Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:24.669582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:24.669709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:24.670187Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:24.670231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:24.674965Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1818 sessionId: src_id|fea5f1de-b07cd69e-b3c3c8c-eb295c2b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:24.677160Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:24.677346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:24.677365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:24.677402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:24.677495Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:24.681172Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:24.681185Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:24.681188Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:24.681190Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:24.681200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342690403151287:2713]
2025-10-09T21:50:24.681204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342690403151287:2713]
2025-10-09T21:50:24.681208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342690403151287:2713]
2025-10-09T21:50:24.681245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:24.681250Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:24.681257Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:24.681260Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:24.681260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.681263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:24.681390Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:24.738816Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342690764982898:6018]
2025-10-09T21:50:24.739086Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 1817, pipe: [1:7559342690764982898:6018]
2025-10-09T21:50:24.823132Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:24.829836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:24.829858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:24.830002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:24.830075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:24.830132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:24.830183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:24.830226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:24.830233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:24.830292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:24.830347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:24.830405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:24.830465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:24.830506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:24.830509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:24.830562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:24.830611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:24.830662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:24.830716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:24.830754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:24.830759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:24.830813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:24.830866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:24.830925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:24.830982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:24.831020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:24.831025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:24.831080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:24.831131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:24.831182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:24.831234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:24.831271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:24.831277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:24.831327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:24.831378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:24.831434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:24.831488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:24.831525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:24.831530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:24.831582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:24.831642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:24.831693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:24.831746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:24.831785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:24.831790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:24.831841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:24.831892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:24.831947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:24.832001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:24.832037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:24.832044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:24.832092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:24.832140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:24.832255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:24.832317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:24.832357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:24.832365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:24.832420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:24.832541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:24.832600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:24.832656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:24.832709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:24.832716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:24.832753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342690764982898:6018], now have 1 active actors on pipe
2025-10-09T21:50:24.832968Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:24.832976Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:24.832982Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:24.833009Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1818 reason: error at write: Disconnected sessionId: src_id|fea5f1de-b07cd69e-b3c3c8c-eb295c2b_0
2025-10-09T21:50:24.833016Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342690764982898:6018]
2025-10-09T21:50:24.833172Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1818 sessionId: src_id|fea5f1de-b07cd69e-b3c3c8c-eb295c2b_0 is DEAD
2025-10-09T21:50:24.833291Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fea5f1de-b07cd69e-b3c3c8c-eb295c2b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:24.833302Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fea5f1de-b07cd69e-b3c3c8c-eb295c2b_0] Write session will restart in 0.007902s
2025-10-09T21:50:24.833326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fea5f1de-b07cd69e-b3c3c8c-eb295c2b_0] Write session: Do CDS request
2025-10-09T21:50:24.833332Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fea5f1de-b07cd69e-b3c3c8c-eb295c2b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:24.833606Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:24.842427Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fea5f1de-b07cd69e-b3c3c8c-eb295c2b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:24.842497Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:24.842511Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1819
2025-10-09T21:50:24.842612Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1819 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:24.842643Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1819 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:24.842654Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1819 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:24.842657Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:24.842979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1819 sessionId:  describe result for acl check
2025-10-09T21:50:24.843007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:24.843010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:24.843011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:24.843017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342690764983011:5882] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:24.843458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342690764983011:5882] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:24.843862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:24.845426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:24.845584Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:24.845594Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:24.845605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:24.845653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:24.845691Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342690403151445:2705]
2025-10-09T21:50:24.846006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342690764983011:5882] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:24.846240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:24.846509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:24.846549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:24.846838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:24.846880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.846885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.846886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.846887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.846888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.846890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.846891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.846892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.846893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.846897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.846906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.846910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.846913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.846917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.846921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.846924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.846928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.846936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.846942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.846949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:24.846959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.846968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:24.846976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.846981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:24.846989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.846993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:24.847002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.847011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:24.847016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.847021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:24.847026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.847031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.847035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.847040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.847048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.847057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.847089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:24.851574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342690764983011:5882] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:24.851586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342690764983011:5882] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:24.851589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342690764983011:5882] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:24.851596Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1819 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:24.853087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:24.854130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:24.854144Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:24.854146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:24.854150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:24.854156Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1818 [2:7559342690403151445:2705]
2025-10-09T21:50:24.854169Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:24.854184Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:24.854193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:24.854196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:24.854203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:24.854218Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:24.854228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:24.854234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:24.854287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.854293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.854295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.854298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:24.854356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:24.854364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.854374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:24.854388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:24.854407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:24.854429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342690764982903:2432], now have 1 active actors on pipe
2025-10-09T21:50:24.854436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342690764983053:5882], now have 1 active actors on pipe
2025-10-09T21:50:24.854442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342690403151439:2705]
2025-10-09T21:50:24.854449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342690403151439:2705]
2025-10-09T21:50:24.854451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342690403151439:2705]
2025-10-09T21:50:24.854463Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:24.854479Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:24.854488Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:24.854496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:24.854499Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:24.854499Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:24.854502Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:24.854486Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1818
2025-10-09T21:50:24.854495Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1818
2025-10-09T21:50:24.854522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342690403151439:2705]
2025-10-09T21:50:24.854529Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:24.854532Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:24.854554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:24.854563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:24.854592Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e8c7c89d-19225cb9-caa90898-accad069_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:24.854603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:24.854610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:24.854612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:24.854621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:24.854633Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:24.854658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:24.854789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:24.854798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:24.854811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:24.854891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1819 partition: 0 MaxSeqNo: 25 sessionId: src_id|e8c7c89d-19225cb9-caa90898-accad069_0
2025-10-09T21:50:24.855131Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fea5f1de-b07cd69e-b3c3c8c-eb295c2b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e8c7c89d-19225cb9-caa90898-accad069_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:24.857424Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8c7c89d-19225cb9-caa90898-accad069_0] Write session: try to update token
2025-10-09T21:50:24.857443Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8c7c89d-19225cb9-caa90898-accad069_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:24.868089Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:24.868105Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:24.868106Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:24.868108Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:24.868115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342690403151439:2705]
2025-10-09T21:50:24.868118Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342690403151439:2705]
2025-10-09T21:50:24.868121Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342690403151439:2705]
2025-10-09T21:50:24.868154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:24.868162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:24.868164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:24.868177Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:24.868183Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:24.868186Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:24.868283Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:24.870640Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1819 sessionId: src_id|e8c7c89d-19225cb9-caa90898-accad069_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:24.872857Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:24.933359Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342690764983063:6018]
2025-10-09T21:50:24.933634Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 1818, pipe: [1:7559342690764983063:6018]
2025-10-09T21:50:25.007117Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:25.007413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.007427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.007465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342690764983063:6018], now have 1 active actors on pipe
2025-10-09T21:50:25.007640Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:25.007638Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342690764983063:6018]
2025-10-09T21:50:25.007664Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:25.007666Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:25.007667Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:25.007722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1819 reason: error at write: Disconnected sessionId: src_id|e8c7c89d-19225cb9-caa90898-accad069_0
2025-10-09T21:50:25.007876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1819 sessionId: src_id|e8c7c89d-19225cb9-caa90898-accad069_0 is DEAD
2025-10-09T21:50:25.008034Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8c7c89d-19225cb9-caa90898-accad069_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:25.008041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8c7c89d-19225cb9-caa90898-accad069_0] Write session will restart in 0.006141s
2025-10-09T21:50:25.008062Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8c7c89d-19225cb9-caa90898-accad069_0] Write session: Do CDS request
2025-10-09T21:50:25.008067Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8c7c89d-19225cb9-caa90898-accad069_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:25.015539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8c7c89d-19225cb9-caa90898-accad069_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:25.015611Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:25.015628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1820
2025-10-09T21:50:25.015761Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1820 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:25.015793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1820 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:25.015804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1820 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:25.015807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:25.016226Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1820 sessionId:  describe result for acl check
2025-10-09T21:50:25.016261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:25.016267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:25.016269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:25.016273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342695059950460:5869] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:25.016716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342695059950460:5869] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:25.018555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:25.018565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342695059950460:5869] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:25.020235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:25.020413Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:25.020422Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:25.020433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:25.020482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:25.020521Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342694698118889:2702]
2025-10-09T21:50:25.020884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342695059950460:5869] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:25.020898Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342695059950460:5869] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:25.020901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342695059950460:5869] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:25.020907Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1820 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:25.021098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:25.021378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:25.021423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:25.021737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:25.021792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.021798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.021799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.021800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.021801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.021802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.021804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.021805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.021807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.021812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.021822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.021826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.021830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.021834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.021838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.021842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.021846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.021850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.021857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.021864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.021871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.021881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.021890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.021895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.021900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.021909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.021914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.021919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.021927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.021933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.021943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.021953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.021961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.021965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.021975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.021980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.022013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:25.028607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:25.029639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:25.029655Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:25.029658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:25.029662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:25.029668Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1819 [2:7559342694698118889:2702]
2025-10-09T21:50:25.029683Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:25.029700Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:25.029710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:25.029712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:25.029716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:25.029732Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:25.029738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:25.029746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:25.029800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.029807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.029809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.029813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:25.029882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:25.029889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.029901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.029914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.029935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:25.029958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342694698118883:2702]
2025-10-09T21:50:25.029965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342694698118883:2702]
2025-10-09T21:50:25.029967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342694698118883:2702]
2025-10-09T21:50:25.029976Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:25.029991Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:25.029994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342695059950361:2432], now have 1 active actors on pipe
2025-10-09T21:50:25.029998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342695059950508:5869], now have 1 active actors on pipe
2025-10-09T21:50:25.030002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:25.030005Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:25.030006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:25.030020Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:25.030023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:25.030068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342694698118883:2702]
2025-10-09T21:50:25.030076Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:25.030056Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1819
2025-10-09T21:50:25.030074Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1819
2025-10-09T21:50:25.030095Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:25.030119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.030128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.030154Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1c7cf2de-be0f704e-44d06ee6-2926acbb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:25.030170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:25.030174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:25.030176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:25.030189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:25.030199Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:25.030217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:25.030426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.030437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.030454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:25.030562Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1820 partition: 0 MaxSeqNo: 25 sessionId: src_id|1c7cf2de-be0f704e-44d06ee6-2926acbb_0
2025-10-09T21:50:25.030908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8c7c89d-19225cb9-caa90898-accad069_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1c7cf2de-be0f704e-44d06ee6-2926acbb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:25.033196Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c7cf2de-be0f704e-44d06ee6-2926acbb_0] Write session: try to update token
2025-10-09T21:50:25.033209Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c7cf2de-be0f704e-44d06ee6-2926acbb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:25.046258Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1820 sessionId: src_id|1c7cf2de-be0f704e-44d06ee6-2926acbb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:25.048555Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:25.048700Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:25.048718Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:25.048729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342694698118883:2702]
2025-10-09T21:50:25.048772Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:25.048778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.048787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.048814Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:25.048985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:25.049126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:25.049260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:25.049395Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:25.049671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:25.049727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:25.049757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:25.049855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:25.056585Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:25.056600Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:25.056603Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:25.056605Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:25.056614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342694698118883:2702]
2025-10-09T21:50:25.056618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342694698118883:2702]
2025-10-09T21:50:25.056622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342694698118883:2702]
2025-10-09T21:50:25.056661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:25.056672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.056675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:25.056684Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:25.056690Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:25.056695Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:25.056799Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:25.107969Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342695059950528:6018]
2025-10-09T21:50:25.108305Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 1819, pipe: [1:7559342695059950528:6018]
2025-10-09T21:50:25.209908Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:25.217660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.217687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.217854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.217924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.217991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.218091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.218141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.218149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.218212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.218273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.218332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.218390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.218430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.218436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.218490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.218542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.218601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.218655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.218695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.218703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.218759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.218823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.218882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.218939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.218979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.218985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.219040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.219095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.219151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.219208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.219248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.219257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.219315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.219373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.219430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.219481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.219520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.219527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.219584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.219638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.219710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.219766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.219806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.219811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.219868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.219920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.219973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.220030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.220070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.220076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.220132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.220187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.220244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.220298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.220339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.220346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.220402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.220456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.220511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.220599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.220641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.220646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.220709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342695059950528:6018], now have 1 active actors on pipe
2025-10-09T21:50:25.220998Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342695059950528:6018]
2025-10-09T21:50:25.221000Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:25.221030Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:25.221032Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:25.221066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1820 reason: error at write: Disconnected sessionId: src_id|1c7cf2de-be0f704e-44d06ee6-2926acbb_0
2025-10-09T21:50:25.221252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1820 sessionId: src_id|1c7cf2de-be0f704e-44d06ee6-2926acbb_0 is DEAD
2025-10-09T21:50:25.221379Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c7cf2de-be0f704e-44d06ee6-2926acbb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:25.221392Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c7cf2de-be0f704e-44d06ee6-2926acbb_0] Write session will restart in 0.007493s
2025-10-09T21:50:25.221411Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c7cf2de-be0f704e-44d06ee6-2926acbb_0] Write session: Do CDS request
2025-10-09T21:50:25.221415Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c7cf2de-be0f704e-44d06ee6-2926acbb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:25.221588Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:25.230500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c7cf2de-be0f704e-44d06ee6-2926acbb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:25.230562Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:25.230578Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1821
2025-10-09T21:50:25.230733Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1821 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:25.230760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1821 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:25.230764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1821 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:25.230767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:25.231729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1821 sessionId:  describe result for acl check
2025-10-09T21:50:25.231768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:25.231775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:25.231777Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:25.231782Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342695059950671:5859] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:25.232179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:25.232329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342695059950671:5859] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:25.234052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:25.234223Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:25.234235Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:25.234246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:25.234296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:25.234335Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342694698119077:2673]
2025-10-09T21:50:25.234576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342695059950671:5859] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:25.234913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:25.235204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:25.235252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:25.235567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:25.235622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.235628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.235629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.235630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.235631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.235632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.235633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.235634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.235635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.235640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.235651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.235655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.235659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.235663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.235667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.235671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.235675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.235679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.235685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.235693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.235698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.235704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.235708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.235717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.235726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.235737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.235745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.235750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.235755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.235764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.235769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.235778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.235786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.235790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.235798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.235803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.235833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:25.240965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342695059950671:5859] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:25.240986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342695059950671:5859] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:25.240991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342695059950671:5859] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:25.241000Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1821 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:25.242340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:25.243411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:25.243429Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:25.243432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:25.243436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:25.243443Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1820 [2:7559342694698119077:2673]
2025-10-09T21:50:25.243460Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:25.243478Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:25.243481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:25.243484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:25.243486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:25.243504Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:25.243511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:25.243523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:25.243581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.243591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.243593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.243596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:25.243665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:25.243673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.243685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.243697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.243719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:25.243745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342695059950708:5859], now have 1 active actors on pipe
2025-10-09T21:50:25.243753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342695059950560:2432], now have 1 active actors on pipe
2025-10-09T21:50:25.243760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342694698119071:2673]
2025-10-09T21:50:25.243763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342694698119071:2673]
2025-10-09T21:50:25.243766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342694698119071:2673]
2025-10-09T21:50:25.243778Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:25.243793Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:25.243796Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:25.243808Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:25.243811Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:25.243811Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:25.243814Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:25.243847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342694698119071:2673]
2025-10-09T21:50:25.243850Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:25.243829Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1820
2025-10-09T21:50:25.243866Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:25.243873Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1820
2025-10-09T21:50:25.243890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.243901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.243931Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|645fbe9a-13dfbe8c-9e33cf8d-dce7ca59_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:25.243943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:25.243947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:25.243950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:25.243961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:25.243975Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:25.243995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:25.244182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.244193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.244209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:25.244309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1821 partition: 0 MaxSeqNo: 25 sessionId: src_id|645fbe9a-13dfbe8c-9e33cf8d-dce7ca59_0
2025-10-09T21:50:25.244627Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c7cf2de-be0f704e-44d06ee6-2926acbb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|645fbe9a-13dfbe8c-9e33cf8d-dce7ca59_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:25.247424Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|645fbe9a-13dfbe8c-9e33cf8d-dce7ca59_0] Write session: try to update token
2025-10-09T21:50:25.247444Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|645fbe9a-13dfbe8c-9e33cf8d-dce7ca59_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:25.256037Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:25.256059Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:25.256070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342694698119071:2673]
2025-10-09T21:50:25.256113Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:25.256118Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:25.256293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:25.256446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:25.256592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:25.256741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:25.256935Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:25.256982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:25.264047Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1821 sessionId: src_id|645fbe9a-13dfbe8c-9e33cf8d-dce7ca59_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:25.266542Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:25.266748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.266773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.266816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:25.266914Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:25.269627Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:25.269645Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:25.269648Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:25.269651Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:25.269661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342694698119071:2673]
2025-10-09T21:50:25.269666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342694698119071:2673]
2025-10-09T21:50:25.269670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342694698119071:2673]
2025-10-09T21:50:25.269713Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:25.269715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:25.269717Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:25.269720Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:25.269723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.269725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:25.269897Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:25.320970Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342695059950720:6018]
2025-10-09T21:50:25.321310Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 1820, pipe: [1:7559342695059950720:6018]
2025-10-09T21:50:25.418545Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:25.429907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.429941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.430133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.430202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.430267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.430341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.430399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.430409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.430618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.430786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.430850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.431066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.431178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.431193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.431248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.431295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.431344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.431402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.431445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.431453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.431511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.431748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.431800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.431849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.431893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.431900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.431972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.432020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.432077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.432128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.432164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.432169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.432226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.432280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.432471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.432685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.432798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.432803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.432853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.432901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.432946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.432995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.433031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.433038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.433230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.433423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.433606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.433664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.433778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.433783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.433833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.433879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.433933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.433991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.434042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.434051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.434101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.434149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.434201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.434419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.434463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.434469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.434516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342695059950720:6018], now have 1 active actors on pipe
2025-10-09T21:50:25.434905Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:25.434910Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:25.434924Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:25.434960Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1821 reason: error at write: Disconnected sessionId: src_id|645fbe9a-13dfbe8c-9e33cf8d-dce7ca59_0
2025-10-09T21:50:25.434965Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342695059950720:6018]
2025-10-09T21:50:25.435148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1821 sessionId: src_id|645fbe9a-13dfbe8c-9e33cf8d-dce7ca59_0 is DEAD
2025-10-09T21:50:25.435257Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|645fbe9a-13dfbe8c-9e33cf8d-dce7ca59_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:25.435265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|645fbe9a-13dfbe8c-9e33cf8d-dce7ca59_0] Write session will restart in 0.009911s
2025-10-09T21:50:25.435290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|645fbe9a-13dfbe8c-9e33cf8d-dce7ca59_0] Write session: Do CDS request
2025-10-09T21:50:25.435295Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|645fbe9a-13dfbe8c-9e33cf8d-dce7ca59_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:25.435300Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:25.446519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|645fbe9a-13dfbe8c-9e33cf8d-dce7ca59_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:25.446577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:25.446593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1822
2025-10-09T21:50:25.446838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:25.446959Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1822 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:25.446994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1822 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:25.447001Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1822 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:25.447004Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:25.447402Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1822 sessionId:  describe result for acl check
2025-10-09T21:50:25.447441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:25.447449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:25.447451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:25.447457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342695059950837:5854] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:25.448034Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342695059950837:5854] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:25.448753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:25.448915Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:25.448923Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:25.448932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:25.448980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:25.449024Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342694698119227:2671]
2025-10-09T21:50:25.449428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:25.449724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:25.449776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:25.450005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:25.450068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342695059950837:5854] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:25.450088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.450095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.450096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.450097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.450098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.450099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.450100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.450101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.450102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.450107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.450116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.450119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.450122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.450126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.450129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.450132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.450135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.450138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.450144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.450150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.450157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.450162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.450167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.450171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.450175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.450179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.450183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.450187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.450191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.450195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.450199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.450204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.450207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.450211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.450215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.450219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.450242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:25.456046Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342695059950837:5854] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:25.456067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342695059950837:5854] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:25.456071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342695059950837:5854] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:25.456079Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1822 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:25.456478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:25.457841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:25.457854Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:25.457856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:25.457859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:25.457863Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1821 [2:7559342694698119227:2671]
2025-10-09T21:50:25.457871Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:25.457883Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:25.457884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:25.457886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:25.457888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:25.457900Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:25.457903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:25.457909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:25.457950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.457956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.457958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.457959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:25.458005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:25.458011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.458018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.458024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.458055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:25.458080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342695059950875:5854], now have 1 active actors on pipe
2025-10-09T21:50:25.458086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342695059950723:2432], now have 1 active actors on pipe
2025-10-09T21:50:25.458091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342694698119222:2671]
2025-10-09T21:50:25.458093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342694698119222:2671]
2025-10-09T21:50:25.458094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342694698119222:2671]
2025-10-09T21:50:25.458103Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:25.458113Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:25.458121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:25.458123Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:25.458126Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:25.458127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:25.458129Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:25.458147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342694698119222:2671]
2025-10-09T21:50:25.458157Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:25.458147Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1821
2025-10-09T21:50:25.458156Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1821
2025-10-09T21:50:25.458177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:25.458207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.458210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.458232Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c5729da9-f9321fb8-17574c56-b3eea791_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:25.458247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:25.458251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:25.458254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:25.458263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:25.458272Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:25.458281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:25.458420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.458430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.458441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:25.458530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1822 partition: 0 MaxSeqNo: 25 sessionId: src_id|c5729da9-f9321fb8-17574c56-b3eea791_0
2025-10-09T21:50:25.458812Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|645fbe9a-13dfbe8c-9e33cf8d-dce7ca59_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c5729da9-f9321fb8-17574c56-b3eea791_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:25.461336Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c5729da9-f9321fb8-17574c56-b3eea791_0] Write session: try to update token
2025-10-09T21:50:25.461347Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c5729da9-f9321fb8-17574c56-b3eea791_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:25.469349Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:25.469362Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:25.469368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342694698119222:2671]
2025-10-09T21:50:25.469400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:25.469412Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:25.469562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:25.469694Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:25.469818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:25.469938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:25.470126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:25.470169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:25.479228Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1822 sessionId: src_id|c5729da9-f9321fb8-17574c56-b3eea791_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:25.481682Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:25.482670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.482697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.482720Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:25.482723Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:25.482726Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:25.482728Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:25.482737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342694698119222:2671]
2025-10-09T21:50:25.482743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342694698119222:2671]
2025-10-09T21:50:25.482746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342694698119222:2671]
2025-10-09T21:50:25.482798Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:25.482800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:25.482808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.482810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:25.482812Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:25.482816Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:25.482935Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:25.534973Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342695059950883:6018]
2025-10-09T21:50:25.535270Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 1821, pipe: [1:7559342695059950883:6018]
2025-10-09T21:50:25.624047Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:25.624343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:25.624393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342695059950883:6018], now have 1 active actors on pipe
2025-10-09T21:50:25.624489Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:25.624572Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:25.624622Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342695059950883:6018]
2025-10-09T21:50:25.633949Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:25.633963Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:25.633981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1822 reason: error at write: Disconnected sessionId: src_id|c5729da9-f9321fb8-17574c56-b3eea791_0
2025-10-09T21:50:25.634120Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1822 sessionId: src_id|c5729da9-f9321fb8-17574c56-b3eea791_0 is DEAD
2025-10-09T21:50:25.634279Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5729da9-f9321fb8-17574c56-b3eea791_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:25.634285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5729da9-f9321fb8-17574c56-b3eea791_0] Write session will restart in 0.008155s
2025-10-09T21:50:25.634303Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5729da9-f9321fb8-17574c56-b3eea791_0] Write session: Do CDS request
2025-10-09T21:50:25.634311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5729da9-f9321fb8-17574c56-b3eea791_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:25.638987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:25.640824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:25.641011Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:25.641022Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:25.641034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:25.641091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:25.641132Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342694698119378:2665]
2025-10-09T21:50:25.641747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:25.642055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:25.642097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:25.642419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:25.642472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.642478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.642479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.642480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.642481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.642482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.642483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.642484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.642485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.642490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.642500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.642504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.642507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.642511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.642514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.642517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.642520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.642523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.642530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.642537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.642545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.642550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.642553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.642560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.642563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.642568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.642577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.642581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.642585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.642589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.642594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.642598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.642602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.642606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.642610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.642614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.642638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:25.643445Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c5729da9-f9321fb8-17574c56-b3eea791_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:25.643535Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:25.643551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1823
2025-10-09T21:50:25.643662Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1823 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:25.643688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1823 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:25.643698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1823 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:25.643702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:25.644162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1823 sessionId:  describe result for acl check
2025-10-09T21:50:25.644208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:25.644214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:25.644216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:25.644221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342695059951004:5837] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:25.644760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342695059951004:5837] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:25.647477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342695059951004:5837] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:25.648797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:25.649846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:25.649860Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:25.649863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:25.649866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:25.649873Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1822 [2:7559342694698119378:2665]
2025-10-09T21:50:25.649886Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:25.649907Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:25.649914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:25.649918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:25.649920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:25.649936Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:25.649942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:25.649948Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:25.650000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.650009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.650011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.650014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:25.650088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:25.650095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.650105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.650121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.650142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:25.650165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342695059950885:2432], now have 1 active actors on pipe
2025-10-09T21:50:25.650178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342694698119373:2665]
2025-10-09T21:50:25.650180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342694698119373:2665]
2025-10-09T21:50:25.650182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342694698119373:2665]
2025-10-09T21:50:25.650165Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1822
2025-10-09T21:50:25.650190Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:25.650197Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:25.650201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:25.650202Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:25.650208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:25.650222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:25.650229Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:25.650243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342695059951004:5837] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:25.650255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342695059951004:5837] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:25.650257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342695059951004:5837] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:25.650263Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1823 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:25.650258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342694698119373:2665]
2025-10-09T21:50:25.650270Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:25.650273Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:25.650441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342695059951038:5837], now have 1 active actors on pipe
2025-10-09T21:50:25.650471Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1822
2025-10-09T21:50:25.650501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.650510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.650535Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5557e915-3c43ada-68d70d5d-256af0d3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:25.650548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:25.650555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:25.650558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:25.650569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:25.650582Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:25.650600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:25.650767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.650775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.650788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:25.650869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1823 partition: 0 MaxSeqNo: 25 sessionId: src_id|5557e915-3c43ada-68d70d5d-256af0d3_0
2025-10-09T21:50:25.651123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5729da9-f9321fb8-17574c56-b3eea791_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5557e915-3c43ada-68d70d5d-256af0d3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:25.653571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5557e915-3c43ada-68d70d5d-256af0d3_0] Write session: try to update token
2025-10-09T21:50:25.653583Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5557e915-3c43ada-68d70d5d-256af0d3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:25.666264Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:25.666281Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:25.666291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342694698119373:2665]
2025-10-09T21:50:25.666336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:25.666338Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:25.666487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:25.666613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:25.666749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:25.666903Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1823 sessionId: src_id|5557e915-3c43ada-68d70d5d-256af0d3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:25.667036Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:25.667264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:25.667305Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:25.669275Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:25.669438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.669458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.669505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:25.669595Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:25.672614Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:25.672627Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:25.672629Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:25.672633Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:25.672641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342694698119373:2665]
2025-10-09T21:50:25.672645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342694698119373:2665]
2025-10-09T21:50:25.672647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342694698119373:2665]
2025-10-09T21:50:25.672683Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:25.672684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:25.672690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.672692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:25.672696Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:25.672700Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:25.672811Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:25.724909Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342695059951046:6018]
2025-10-09T21:50:25.725148Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 1822, pipe: [1:7559342695059951046:6018]
2025-10-09T21:50:25.817607Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:25.824870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.824896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.825037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.825095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.825151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.825213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.825464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.825488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.825550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.825605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.825663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.825721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.825760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.825766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.825820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.825893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.825954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.826012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.826067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.826074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.826147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.826208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.826265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.826330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.826391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.826396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.826452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.826505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.826560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.826613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.826668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.826674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.826727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.826778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.826841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.826895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.826945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.826951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.827035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.827083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.827138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.827219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.827272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.827277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.827359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.827436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.827487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.827537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.827576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.827584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.827714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.827764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.827826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.827881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.827922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.827925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.827978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:25.828029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:25.828084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:25.828137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:25.828250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:25.828256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:25.828331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342695059951046:6018], now have 1 active actors on pipe
2025-10-09T21:50:25.828524Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:25.828602Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:25.828617Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:25.828610Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342695059951046:6018]
2025-10-09T21:50:25.828643Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1823 reason: error at write: Disconnected sessionId: src_id|5557e915-3c43ada-68d70d5d-256af0d3_0
2025-10-09T21:50:25.828647Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:25.828792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1823 sessionId: src_id|5557e915-3c43ada-68d70d5d-256af0d3_0 is DEAD
2025-10-09T21:50:25.828916Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5557e915-3c43ada-68d70d5d-256af0d3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:25.828925Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5557e915-3c43ada-68d70d5d-256af0d3_0] Write session will restart in 0.007935s
2025-10-09T21:50:25.828946Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5557e915-3c43ada-68d70d5d-256af0d3_0] Write session: Do CDS request
2025-10-09T21:50:25.828952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5557e915-3c43ada-68d70d5d-256af0d3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:25.838480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5557e915-3c43ada-68d70d5d-256af0d3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:25.838568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:25.838585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1824
2025-10-09T21:50:25.838708Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1824 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:25.838741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1824 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:25.838760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1824 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:25.838770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:25.839235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1824 sessionId:  describe result for acl check
2025-10-09T21:50:25.839277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:25.839284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:25.839285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:25.839292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342695059951162:5835] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:25.839714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:25.839801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342695059951162:5835] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:25.841541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:25.841769Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:25.841779Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:25.841790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:25.841848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:25.841903Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342694698119534:2666]
2025-10-09T21:50:25.842275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342695059951162:5835] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:25.842527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:25.842824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:25.842873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:25.843215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:25.843264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.843270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.843271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.843273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.843274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.843275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.843276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.843277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.843278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.843284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.843296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.843301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.843305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.843309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.843313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.843318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.843321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.843326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.843332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.843341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:25.843347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.843353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:25.843358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.843363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:25.843367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.843378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:25.843383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.843388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:25.843398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.843403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:25.843413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.843418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.843429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.843434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.843443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.843449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.843487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:25.848011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342695059951162:5835] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:25.848032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342695059951162:5835] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:25.848037Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342695059951162:5835] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:25.848047Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1824 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:25.850084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:25.851199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:25.851225Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:25.851229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:25.851233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:25.851240Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1823 [2:7559342694698119534:2666]
2025-10-09T21:50:25.851254Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:25.851273Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:25.851280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:25.851283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:25.851284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:25.851299Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:25.851303Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:25.851310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:25.851355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.851361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.851362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.851365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:25.851417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:25.851422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.851430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:25.851437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:25.851449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:25.851467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342694698119528:2666]
2025-10-09T21:50:25.851469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342694698119528:2666]
2025-10-09T21:50:25.851471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342694698119528:2666]
2025-10-09T21:50:25.851477Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:25.851487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:25.851488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:25.851489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:25.851491Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:25.851498Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:25.851500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:25.851509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342695059951048:2432], now have 1 active actors on pipe
2025-10-09T21:50:25.851512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342695059951202:5835], now have 1 active actors on pipe
2025-10-09T21:50:25.851538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342694698119528:2666]
2025-10-09T21:50:25.851544Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:25.851559Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:25.851581Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1823
2025-10-09T21:50:25.851606Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1823
2025-10-09T21:50:25.851620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.851628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.851651Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1f98a667-9ba2caf5-c05bc73-9d5b9a73_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:25.851660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:25.851663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:25.851666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:25.851675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:25.851689Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:25.851721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:25.852153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:25.852175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:25.852198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:25.852337Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1824 partition: 0 MaxSeqNo: 25 sessionId: src_id|1f98a667-9ba2caf5-c05bc73-9d5b9a73_0
2025-10-09T21:50:25.852720Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5557e915-3c43ada-68d70d5d-256af0d3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1f98a667-9ba2caf5-c05bc73-9d5b9a73_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:25.855085Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1f98a667-9ba2caf5-c05bc73-9d5b9a73_0] Write session: try to update token
2025-10-09T21:50:25.855103Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1f98a667-9ba2caf5-c05bc73-9d5b9a73_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:25.865801Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:25.865832Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:25.865835Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:25.865837Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:25.865848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342694698119528:2666]
2025-10-09T21:50:25.865853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342694698119528:2666]
2025-10-09T21:50:25.865857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342694698119528:2666]
2025-10-09T21:50:25.865893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:25.865900Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:25.865903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:25.865905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:25.865906Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:25.865910Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:25.866024Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:25.868467Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1824 sessionId: src_id|1f98a667-9ba2caf5-c05bc73-9d5b9a73_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:25.870935Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:25.928716Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342695059951212:6018]
2025-10-09T21:50:25.929039Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 1823, pipe: [1:7559342695059951212:6018]
2025-10-09T21:50:26.012609Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:26.013014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.013030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.013071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342695059951212:6018], now have 1 active actors on pipe
2025-10-09T21:50:26.013255Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:26.013294Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342695059951212:6018]
2025-10-09T21:50:26.013302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:26.013306Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:26.013325Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:26.013370Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1824 reason: error at write: Disconnected sessionId: src_id|1f98a667-9ba2caf5-c05bc73-9d5b9a73_0
2025-10-09T21:50:26.013568Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1824 sessionId: src_id|1f98a667-9ba2caf5-c05bc73-9d5b9a73_0 is DEAD
2025-10-09T21:50:26.013703Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1f98a667-9ba2caf5-c05bc73-9d5b9a73_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:26.013715Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1f98a667-9ba2caf5-c05bc73-9d5b9a73_0] Write session will restart in 0.008825s
2025-10-09T21:50:26.013742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1f98a667-9ba2caf5-c05bc73-9d5b9a73_0] Write session: Do CDS request
2025-10-09T21:50:26.013748Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1f98a667-9ba2caf5-c05bc73-9d5b9a73_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:26.023514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1f98a667-9ba2caf5-c05bc73-9d5b9a73_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:26.023620Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:26.023636Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1825
2025-10-09T21:50:26.023784Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1825 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:26.023818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1825 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:26.023830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1825 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:26.023833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:26.024186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:26.024282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1825 sessionId:  describe result for acl check
2025-10-09T21:50:26.024321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:26.024328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:26.024330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:26.024337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342699354918626:5828] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:26.024909Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342699354918626:5828] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:26.026175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:26.026349Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:26.026356Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:26.026366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:26.026408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:26.026444Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342698993086984:2658]
2025-10-09T21:50:26.026836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:26.027082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:26.027134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:26.027136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342699354918626:5828] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:26.027400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:26.027446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.027451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.027452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.027452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.027453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.027454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.027454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.027455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.027456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.027459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.027465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.027467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.027469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.027471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.027474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.027476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.027478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.027480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.027484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.027491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.027495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.027498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.027501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.027504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.027511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.027514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.027517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.027521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.027524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.027527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.027530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.027533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.027536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.027539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.027541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.027544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.027567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:26.033531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342699354918626:5828] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:26.033545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342699354918626:5828] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:26.033549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342699354918626:5828] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:26.033557Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1825 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:26.033969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:26.035094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:26.035114Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:26.035116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:26.035121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:26.035128Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1824 [2:7559342698993086984:2658]
2025-10-09T21:50:26.035142Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:26.035165Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:26.035168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:26.035172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:26.035174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:26.035192Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:26.035210Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:26.035218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:26.035277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.035285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.035287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.035289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:26.035371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:26.035377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.035385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.035392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.035403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:26.035420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354918666:5828], now have 1 active actors on pipe
2025-10-09T21:50:26.035426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354918511:2432], now have 1 active actors on pipe
2025-10-09T21:50:26.035433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342698993086978:2658]
2025-10-09T21:50:26.035444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342698993086978:2658]
2025-10-09T21:50:26.035446Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342698993086978:2658]
2025-10-09T21:50:26.035460Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:26.035476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:26.035477Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:26.035480Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:26.035482Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:26.035490Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:26.035493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:26.035492Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1824
2025-10-09T21:50:26.035512Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1824
2025-10-09T21:50:26.035525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342698993086978:2658]
2025-10-09T21:50:26.035534Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:26.035535Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:26.035550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.035554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.035605Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1bc22aa-3618b67e-2b12d85d-cffa9474_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:26.035621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:26.035625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:26.035628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:26.035639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:26.035650Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:26.035665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:26.035816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.035829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.035853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:26.035931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1825 partition: 0 MaxSeqNo: 25 sessionId: src_id|1bc22aa-3618b67e-2b12d85d-cffa9474_0
2025-10-09T21:50:26.036290Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1f98a667-9ba2caf5-c05bc73-9d5b9a73_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1bc22aa-3618b67e-2b12d85d-cffa9474_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:26.038852Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1bc22aa-3618b67e-2b12d85d-cffa9474_0] Write session: try to update token
2025-10-09T21:50:26.038874Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1bc22aa-3618b67e-2b12d85d-cffa9474_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:26.050214Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:26.050234Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:26.050236Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:26.050239Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:26.050249Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342698993086978:2658]
2025-10-09T21:50:26.050255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342698993086978:2658]
2025-10-09T21:50:26.050260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342698993086978:2658]
2025-10-09T21:50:26.050302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:26.050313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.050320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:26.050321Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:26.050327Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:26.050331Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:26.050455Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:26.051946Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1825 sessionId: src_id|1bc22aa-3618b67e-2b12d85d-cffa9474_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:26.054504Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:26.113564Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342699354918675:6018]
2025-10-09T21:50:26.113853Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 1824, pipe: [1:7559342699354918675:6018]
2025-10-09T21:50:26.202989Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:26.203484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.203498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.203534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354918675:6018], now have 1 active actors on pipe
2025-10-09T21:50:26.203718Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:26.203823Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342699354918675:6018]
2025-10-09T21:50:26.203856Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:26.203894Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:26.203981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1825 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1bc22aa-3618b67e-2b12d85d-cffa9474_0
2025-10-09T21:50:26.204146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1825 sessionId: src_id|1bc22aa-3618b67e-2b12d85d-cffa9474_0 is DEAD
2025-10-09T21:50:26.204317Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bc22aa-3618b67e-2b12d85d-cffa9474_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:26.204325Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bc22aa-3618b67e-2b12d85d-cffa9474_0] Write session will restart in 0.008805s
2025-10-09T21:50:26.204351Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bc22aa-3618b67e-2b12d85d-cffa9474_0] Write session: Do CDS request
2025-10-09T21:50:26.204357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bc22aa-3618b67e-2b12d85d-cffa9474_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:26.214525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1bc22aa-3618b67e-2b12d85d-cffa9474_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:26.214847Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:26.214870Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1826
2025-10-09T21:50:26.215100Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1826 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:26.215135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1826 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:26.215141Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1826 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:26.215143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:26.215445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:26.215484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1826 sessionId:  describe result for acl check
2025-10-09T21:50:26.215521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:26.215530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:26.215533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:26.215538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342699354918822:5809] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:26.216082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342699354918822:5809] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:26.217310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342699354918822:5809] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:26.217801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:26.217997Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:26.218009Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:26.218022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:26.218098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:26.218136Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342698993087176:2639]
2025-10-09T21:50:26.218679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:26.218962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:26.219003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:26.219241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:26.219286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.219293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.219294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.219295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.219296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.219297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.219298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.219299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.219299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.219304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.219315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.219319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.219323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.219327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.219331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.219335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.219339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.219342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.219349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.219357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.219363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.219368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.219372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.219377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.219382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.219387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.219392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.219397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.219401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.219406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.219411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.219416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.219420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.219425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.219430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.219434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.219460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:26.219721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342699354918822:5809] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:26.219733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342699354918822:5809] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:26.219736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342699354918822:5809] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:26.219741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1826 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:26.226084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:26.227139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:26.227155Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:26.227158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:26.227162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:26.227169Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1825 [2:7559342698993087176:2639]
2025-10-09T21:50:26.227181Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:26.227198Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:26.227201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:26.227204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:26.227207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:26.227224Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:26.227237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:26.227244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:26.227303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.227310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.227312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.227315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:26.227384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:26.227392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.227405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.227417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.227439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:26.227460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354918859:5809], now have 1 active actors on pipe
2025-10-09T21:50:26.227466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354918694:2432], now have 1 active actors on pipe
2025-10-09T21:50:26.227473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342698993087163:2639]
2025-10-09T21:50:26.227474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342698993087163:2639]
2025-10-09T21:50:26.227476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342698993087163:2639]
2025-10-09T21:50:26.227491Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:26.227505Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:26.227518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:26.227518Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:26.227522Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:26.227524Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:26.227524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:26.227554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342698993087163:2639]
2025-10-09T21:50:26.227560Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:26.227535Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1825
2025-10-09T21:50:26.227570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:26.227588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.227543Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1825
2025-10-09T21:50:26.227592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.227617Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|15eb7bf4-f50b23a8-b3f5763d-33de1f74_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:26.227627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:26.227630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:26.227632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:26.227644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:26.227660Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:26.227678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:26.227821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.227827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.227838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:26.227925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1826 partition: 0 MaxSeqNo: 25 sessionId: src_id|15eb7bf4-f50b23a8-b3f5763d-33de1f74_0
2025-10-09T21:50:26.228266Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bc22aa-3618b67e-2b12d85d-cffa9474_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|15eb7bf4-f50b23a8-b3f5763d-33de1f74_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:26.230767Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15eb7bf4-f50b23a8-b3f5763d-33de1f74_0] Write session: try to update token
2025-10-09T21:50:26.230786Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15eb7bf4-f50b23a8-b3f5763d-33de1f74_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:26.242379Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:26.242400Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:26.242403Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:26.242405Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:26.242414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342698993087163:2639]
2025-10-09T21:50:26.242419Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342698993087163:2639]
2025-10-09T21:50:26.242422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342698993087163:2639]
2025-10-09T21:50:26.242467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:26.242477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.242481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:26.242483Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:26.242490Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:26.242493Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:26.242620Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:26.243233Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1826 sessionId: src_id|15eb7bf4-f50b23a8-b3f5763d-33de1f74_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:26.245798Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:26.303995Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342699354918871:6018]
2025-10-09T21:50:26.304282Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 1825, pipe: [1:7559342699354918871:6018]
2025-10-09T21:50:26.385419Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:26.385887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.385904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.385947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354918871:6018], now have 1 active actors on pipe
2025-10-09T21:50:26.386232Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:26.386225Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342699354918871:6018]
2025-10-09T21:50:26.386247Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:26.386257Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:26.386309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1826 reason: error at write: Disconnected sessionId: src_id|15eb7bf4-f50b23a8-b3f5763d-33de1f74_0
2025-10-09T21:50:26.386394Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:26.386472Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1826 sessionId: src_id|15eb7bf4-f50b23a8-b3f5763d-33de1f74_0 is DEAD
2025-10-09T21:50:26.386620Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15eb7bf4-f50b23a8-b3f5763d-33de1f74_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:26.386636Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15eb7bf4-f50b23a8-b3f5763d-33de1f74_0] Write session will restart in 0.008175s
2025-10-09T21:50:26.386661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15eb7bf4-f50b23a8-b3f5763d-33de1f74_0] Write session: Do CDS request
2025-10-09T21:50:26.386668Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15eb7bf4-f50b23a8-b3f5763d-33de1f74_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:26.395490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15eb7bf4-f50b23a8-b3f5763d-33de1f74_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:26.395544Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:26.395573Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1827
2025-10-09T21:50:26.395688Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1827 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:26.395719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1827 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:26.395729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1827 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:26.395733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:26.396081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1827 sessionId:  describe result for acl check
2025-10-09T21:50:26.396113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:26.396120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:26.396121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:26.396125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342699354918991:5799] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:26.396181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:26.396598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342699354918991:5799] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:26.398357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:26.398491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342699354918991:5799] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:26.398532Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:26.398541Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:26.398552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:26.398601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:26.398643Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342698993087331:2629]
2025-10-09T21:50:26.399191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:26.399459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:26.399497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:26.399786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:26.399831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.399837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.399838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.399839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.399840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.399841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.399842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.399843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.399844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.399849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.399859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.399863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.399867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.399871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.399875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.399878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.399881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.399885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.399891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.399898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.399904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.399908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.399912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.399917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.399921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.399926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.399930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.399935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.399939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.399945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.399955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.399960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.399968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.399974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.399981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.399987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.400017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:26.402454Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342699354918991:5799] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:26.402468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342699354918991:5799] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:26.402472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342699354918991:5799] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:26.402480Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1827 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:26.405989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:26.407015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:26.407030Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:26.407032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:26.407036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:26.407042Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1826 [2:7559342698993087331:2629]
2025-10-09T21:50:26.407055Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:26.407071Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:26.407079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:26.407082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:26.407084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:26.407101Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:26.407112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:26.407119Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:26.407176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.407184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.407186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.407188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:26.407255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:26.407262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.407275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.407285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.407301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:26.407322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354919031:5799], now have 1 active actors on pipe
2025-10-09T21:50:26.407329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354918874:2432], now have 1 active actors on pipe
2025-10-09T21:50:26.407337Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342698993087325:2629]
2025-10-09T21:50:26.407346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342698993087325:2629]
2025-10-09T21:50:26.407348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342698993087325:2629]
2025-10-09T21:50:26.407362Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:26.407376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:26.407380Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:26.407383Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:26.407385Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:26.407390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:26.407393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:26.407376Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1826
2025-10-09T21:50:26.407378Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1826
2025-10-09T21:50:26.407417Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342698993087325:2629]
2025-10-09T21:50:26.407425Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:26.407427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:26.407439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.407444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.407490Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|346064ab-972eacac-66749c4c-4409a89b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:26.407502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:26.407506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:26.407509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:26.407519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:26.407529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:26.407550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:26.407731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.407741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.407755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:26.407856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1827 partition: 0 MaxSeqNo: 25 sessionId: src_id|346064ab-972eacac-66749c4c-4409a89b_0
2025-10-09T21:50:26.408179Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15eb7bf4-f50b23a8-b3f5763d-33de1f74_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|346064ab-972eacac-66749c4c-4409a89b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:26.410611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|346064ab-972eacac-66749c4c-4409a89b_0] Write session: try to update token
2025-10-09T21:50:26.410629Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|346064ab-972eacac-66749c4c-4409a89b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:26.420876Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:26.420894Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:26.420896Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:26.420898Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:26.420908Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342698993087325:2629]
2025-10-09T21:50:26.420912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342698993087325:2629]
2025-10-09T21:50:26.420915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342698993087325:2629]
2025-10-09T21:50:26.420957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:26.420961Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:26.420964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.420967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:26.420976Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:26.420981Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:26.421096Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:26.423809Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1827 sessionId: src_id|346064ab-972eacac-66749c4c-4409a89b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:26.426303Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:26.486283Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342699354919038:6018]
2025-10-09T21:50:26.486560Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 1826, pipe: [1:7559342699354919038:6018]
2025-10-09T21:50:26.562847Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:26.563383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.563398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.563445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354919038:6018], now have 1 active actors on pipe
2025-10-09T21:50:26.563654Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:26.563658Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342699354919038:6018]
2025-10-09T21:50:26.563669Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:26.563679Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:26.563742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1827 reason: error at write: Disconnected sessionId: src_id|346064ab-972eacac-66749c4c-4409a89b_0
2025-10-09T21:50:26.563776Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:26.563939Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1827 sessionId: src_id|346064ab-972eacac-66749c4c-4409a89b_0 is DEAD
2025-10-09T21:50:26.564092Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|346064ab-972eacac-66749c4c-4409a89b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:26.564105Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|346064ab-972eacac-66749c4c-4409a89b_0] Write session will restart in 0.009494s
2025-10-09T21:50:26.564130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|346064ab-972eacac-66749c4c-4409a89b_0] Write session: Do CDS request
2025-10-09T21:50:26.564137Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|346064ab-972eacac-66749c4c-4409a89b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:26.574468Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|346064ab-972eacac-66749c4c-4409a89b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:26.574472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:26.574544Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:26.574559Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1828
2025-10-09T21:50:26.574691Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1828 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:26.574724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1828 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:26.574736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1828 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:26.574740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:26.575150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1828 sessionId:  describe result for acl check
2025-10-09T21:50:26.575183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:26.575192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:26.575194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:26.575199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342699354919160:5795] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:26.575697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342699354919160:5795] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:26.576281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:26.576445Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:26.576454Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:26.576465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:26.576516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:26.576555Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342698993087487:2627]
2025-10-09T21:50:26.577098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:26.577498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:26.577532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342699354919160:5795] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:26.577544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:26.577846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:26.577884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.577888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.577889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.577889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.577890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.577890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.577891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.577892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.577892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.577895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.577901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.577903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.577905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.577907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.577909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.577911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.577913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.577915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.577920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.577925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.577929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.577934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.577938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.577943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.577946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.577950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.577955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.577960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.577964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.577969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.577974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.577978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.577981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.577984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.577987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.577990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.578006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:26.582886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342699354919160:5795] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:26.582902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342699354919160:5795] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:26.582906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342699354919160:5795] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:26.582913Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1828 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:26.584357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:26.585367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:26.585382Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:26.585384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:26.585389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:26.585395Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1827 [2:7559342698993087487:2627]
2025-10-09T21:50:26.585408Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:26.585424Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:26.585426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:26.585429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:26.585431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:26.585446Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:26.585451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:26.585458Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:26.585519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.585528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.585529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.585532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:26.585597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:26.585604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.585616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.585633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.585656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:26.585679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354919199:5795], now have 1 active actors on pipe
2025-10-09T21:50:26.585687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354919040:2432], now have 1 active actors on pipe
2025-10-09T21:50:26.585694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342698993087481:2627]
2025-10-09T21:50:26.585696Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342698993087481:2627]
2025-10-09T21:50:26.585698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342698993087481:2627]
2025-10-09T21:50:26.585711Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:26.585728Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:26.585731Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:26.585734Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:26.585735Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:26.585747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:26.585750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:26.585724Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1827
2025-10-09T21:50:26.585741Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1827
2025-10-09T21:50:26.585776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342698993087481:2627]
2025-10-09T21:50:26.585788Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:26.585796Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:26.585808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.585817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.586015Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b19306a1-9f4f051d-1a018698-b5c74be8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:26.586043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:26.586047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:26.586050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:26.586062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:26.586077Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:26.586105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:26.586265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.586274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.586288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:26.586367Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1828 partition: 0 MaxSeqNo: 25 sessionId: src_id|b19306a1-9f4f051d-1a018698-b5c74be8_0
2025-10-09T21:50:26.586682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|346064ab-972eacac-66749c4c-4409a89b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b19306a1-9f4f051d-1a018698-b5c74be8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:26.589098Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b19306a1-9f4f051d-1a018698-b5c74be8_0] Write session: try to update token
2025-10-09T21:50:26.589116Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b19306a1-9f4f051d-1a018698-b5c74be8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:26.599712Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:26.599729Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:26.599731Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:26.599733Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:26.599740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342698993087481:2627]
2025-10-09T21:50:26.599743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342698993087481:2627]
2025-10-09T21:50:26.599746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342698993087481:2627]
2025-10-09T21:50:26.599779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:26.599786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.599788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:26.599818Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:26.599839Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:26.599852Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:26.599914Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:26.602196Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1828 sessionId: src_id|b19306a1-9f4f051d-1a018698-b5c74be8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:26.604731Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:26.663930Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342699354919207:6018]
2025-10-09T21:50:26.664231Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 1827, pipe: [1:7559342699354919207:6018]
2025-10-09T21:50:26.741119Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:26.741406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.741427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.741478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354919207:6018], now have 1 active actors on pipe
2025-10-09T21:50:26.741692Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:26.741708Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:26.741713Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342699354919207:6018]
2025-10-09T21:50:26.741741Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:26.741800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1828 reason: error at write: Disconnected sessionId: src_id|b19306a1-9f4f051d-1a018698-b5c74be8_0
2025-10-09T21:50:26.741885Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:26.741984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1828 sessionId: src_id|b19306a1-9f4f051d-1a018698-b5c74be8_0 is DEAD
2025-10-09T21:50:26.742117Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b19306a1-9f4f051d-1a018698-b5c74be8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:26.742128Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b19306a1-9f4f051d-1a018698-b5c74be8_0] Write session will restart in 0.005051s
2025-10-09T21:50:26.742152Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b19306a1-9f4f051d-1a018698-b5c74be8_0] Write session: Do CDS request
2025-10-09T21:50:26.742158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b19306a1-9f4f051d-1a018698-b5c74be8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:26.748370Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b19306a1-9f4f051d-1a018698-b5c74be8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:26.748403Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:26.748419Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1829
2025-10-09T21:50:26.748543Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1829 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:26.748575Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1829 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:26.748587Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1829 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:26.748591Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:26.748930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1829 sessionId:  describe result for acl check
2025-10-09T21:50:26.748967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:26.748973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:26.748974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:26.748978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342699354919234:5789] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:26.749436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342699354919234:5789] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:26.751499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342699354919234:5789] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:26.753671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:26.755536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:26.755669Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342699354919234:5789] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:26.755685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342699354919234:5789] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:26.755689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342699354919234:5789] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:26.755697Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1829 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:26.755712Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:26.755725Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:26.755738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:26.755797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:26.755836Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342698993087649:2621]
2025-10-09T21:50:26.756431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:26.756733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:26.756784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:26.757097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:26.757152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.757159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.757161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.757162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.757163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.757164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.757165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.757166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.757167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.757172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.757181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.757186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.757190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.757194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.757198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.757202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.757206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.757210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.757216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.757224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.757230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.757236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.757240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.757245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.757249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.757255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.757259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.757265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.757269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.757274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.757279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.757284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.757288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.757294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.757298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.757302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.757331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:26.764023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:26.765206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:26.765222Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:26.765224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:26.765228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:26.765234Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1828 [2:7559342698993087649:2621]
2025-10-09T21:50:26.765248Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:26.765265Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:26.765270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:26.765274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:26.765276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:26.765293Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:26.765305Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:26.765314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:26.765380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.765388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.765390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.765393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:26.765463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:26.765480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.765492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.765502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.765517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:26.765542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342698993087643:2621]
2025-10-09T21:50:26.765550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342698993087643:2621]
2025-10-09T21:50:26.765552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342698993087643:2621]
2025-10-09T21:50:26.765567Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:26.765585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354919368:5789], now have 1 active actors on pipe
2025-10-09T21:50:26.765588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354919209:2432], now have 1 active actors on pipe
2025-10-09T21:50:26.765591Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:26.765593Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:26.765596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:26.765597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:26.765610Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:26.765614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:26.765646Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342698993087643:2621]
2025-10-09T21:50:26.765659Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:26.765667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:26.765648Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1828
2025-10-09T21:50:26.765663Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1828
2025-10-09T21:50:26.765828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.765842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.765875Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|843e3925-cfeecf7a-ec44b1d2-a52c7ce7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:26.765884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:26.765888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:26.765892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:26.765903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:26.765914Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:26.765931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:26.766169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.766180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.766197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:26.766311Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1829 partition: 0 MaxSeqNo: 25 sessionId: src_id|843e3925-cfeecf7a-ec44b1d2-a52c7ce7_0
2025-10-09T21:50:26.766676Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b19306a1-9f4f051d-1a018698-b5c74be8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|843e3925-cfeecf7a-ec44b1d2-a52c7ce7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:26.769145Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|843e3925-cfeecf7a-ec44b1d2-a52c7ce7_0] Write session: try to update token
2025-10-09T21:50:26.769165Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|843e3925-cfeecf7a-ec44b1d2-a52c7ce7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:26.780134Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:26.780150Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:26.780153Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:26.780155Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:26.780164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342698993087643:2621]
2025-10-09T21:50:26.780169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342698993087643:2621]
2025-10-09T21:50:26.780172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342698993087643:2621]
2025-10-09T21:50:26.780218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:26.780225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.780228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:26.780245Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:26.780270Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:26.780277Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:26.780355Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:26.781890Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1829 sessionId: src_id|843e3925-cfeecf7a-ec44b1d2-a52c7ce7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:26.784267Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:26.842072Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342699354919380:6018]
2025-10-09T21:50:26.842323Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 1828, pipe: [1:7559342699354919380:6018]
2025-10-09T21:50:26.920520Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:26.920873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.920886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.920922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354919380:6018], now have 1 active actors on pipe
2025-10-09T21:50:26.921139Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:26.921148Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342699354919380:6018]
2025-10-09T21:50:26.921175Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:26.921178Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:26.921183Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:26.921265Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1829 reason: error at write: Disconnected sessionId: src_id|843e3925-cfeecf7a-ec44b1d2-a52c7ce7_0
2025-10-09T21:50:26.921426Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1829 sessionId: src_id|843e3925-cfeecf7a-ec44b1d2-a52c7ce7_0 is DEAD
2025-10-09T21:50:26.921588Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|843e3925-cfeecf7a-ec44b1d2-a52c7ce7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:26.921600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|843e3925-cfeecf7a-ec44b1d2-a52c7ce7_0] Write session will restart in 0.009336s
2025-10-09T21:50:26.921628Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|843e3925-cfeecf7a-ec44b1d2-a52c7ce7_0] Write session: Do CDS request
2025-10-09T21:50:26.921634Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|843e3925-cfeecf7a-ec44b1d2-a52c7ce7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:26.931870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:26.932450Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|843e3925-cfeecf7a-ec44b1d2-a52c7ce7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:26.932539Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:26.932553Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1830
2025-10-09T21:50:26.932661Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1830 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:26.932696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1830 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:26.932703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1830 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:26.932706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:26.933094Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1830 sessionId:  describe result for acl check
2025-10-09T21:50:26.933128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:26.933130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:26.933132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:26.933138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342699354919506:5779] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:26.933698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342699354919506:5779] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:26.933881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:26.934080Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:26.934091Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:26.934105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:26.934169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:26.934212Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342698993087807:2612]
2025-10-09T21:50:26.934787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:26.935071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:26.935106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:26.935335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:26.935371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.935375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.935376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.935377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.935377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.935378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.935379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.935379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.935380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.935383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.935390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.935392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.935394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.935396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.935398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.935400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.935402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.935404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.935408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.935413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:26.935418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.935424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:26.935428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.935433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:26.935437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.935442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:26.935452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.935459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:26.935468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.935473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:26.935481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.935484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.935487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.935493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.935496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.935499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.935519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:26.935540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342699354919506:5779] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:26.940992Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342699354919506:5779] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:26.941007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342699354919506:5779] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:26.941011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342699354919506:5779] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:26.941019Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1830 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:26.941458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:26.942536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:26.942546Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:26.942548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:26.942550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:26.942555Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1829 [2:7559342698993087807:2612]
2025-10-09T21:50:26.942563Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:26.942576Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:26.942581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:26.942583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:26.942584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:26.942596Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:26.942599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:26.942604Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:26.942640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.942645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.942646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.942648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:26.942701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:26.942707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.942714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:26.942720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:26.942729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:26.942744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354919384:2432], now have 1 active actors on pipe
2025-10-09T21:50:26.942746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342699354919544:5779], now have 1 active actors on pipe
2025-10-09T21:50:26.942751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342698993087801:2612]
2025-10-09T21:50:26.942752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342698993087801:2612]
2025-10-09T21:50:26.942753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342698993087801:2612]
2025-10-09T21:50:26.942759Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:26.942768Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:26.942770Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:26.942780Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:26.942782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:26.942785Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:26.942787Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:26.942800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342698993087801:2612]
2025-10-09T21:50:26.942805Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:26.942807Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:26.942813Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1829
2025-10-09T21:50:26.942830Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1829
2025-10-09T21:50:26.942871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.942879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.942900Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6fb7c934-f35f78e0-ace2b03e-e21760f1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:26.942909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:26.942912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:26.942914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:26.942920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:26.942927Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:26.942950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:26.943092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:26.943104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:26.943120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:26.943193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1830 partition: 0 MaxSeqNo: 25 sessionId: src_id|6fb7c934-f35f78e0-ace2b03e-e21760f1_0
2025-10-09T21:50:26.943469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|843e3925-cfeecf7a-ec44b1d2-a52c7ce7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6fb7c934-f35f78e0-ace2b03e-e21760f1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:26.945822Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6fb7c934-f35f78e0-ace2b03e-e21760f1_0] Write session: try to update token
2025-10-09T21:50:26.945840Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6fb7c934-f35f78e0-ace2b03e-e21760f1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:26.956939Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:26.956962Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:26.956964Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:26.956967Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:26.956976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342698993087801:2612]
2025-10-09T21:50:26.956980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342698993087801:2612]
2025-10-09T21:50:26.956984Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342698993087801:2612]
2025-10-09T21:50:26.957029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:26.957034Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:26.957036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:26.957039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:26.957041Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:26.957045Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:26.957169Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:26.958388Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1830 sessionId: src_id|6fb7c934-f35f78e0-ace2b03e-e21760f1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:26.960657Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:27.021305Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342703649886848:6018]
2025-10-09T21:50:27.021647Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 1829, pipe: [1:7559342703649886848:6018]
2025-10-09T21:50:27.107853Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:27.108131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:27.108147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:27.108186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342703649886848:6018], now have 1 active actors on pipe
2025-10-09T21:50:27.108375Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:27.108475Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342703649886848:6018]
2025-10-09T21:50:27.108504Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:27.108515Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:27.108582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1830 reason: status is not ok: tablet will be restarted right now sessionId: src_id|6fb7c934-f35f78e0-ace2b03e-e21760f1_0
2025-10-09T21:50:27.108764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1830 sessionId: src_id|6fb7c934-f35f78e0-ace2b03e-e21760f1_0 is DEAD
2025-10-09T21:50:27.108934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6fb7c934-f35f78e0-ace2b03e-e21760f1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:27.108948Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6fb7c934-f35f78e0-ace2b03e-e21760f1_0] Write session will restart in 0.005228s
2025-10-09T21:50:27.108973Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6fb7c934-f35f78e0-ace2b03e-e21760f1_0] Write session: Do CDS request
2025-10-09T21:50:27.108980Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6fb7c934-f35f78e0-ace2b03e-e21760f1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:27.115558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6fb7c934-f35f78e0-ace2b03e-e21760f1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:27.115640Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:27.115658Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1831
2025-10-09T21:50:27.115787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1831 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:27.115823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1831 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:27.115836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1831 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:27.115840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:27.116312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1831 sessionId:  describe result for acl check
2025-10-09T21:50:27.116352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:27.116354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:27.116355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:27.116361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342703649886874:5776] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:27.116916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342703649886874:5776] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:27.119370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342703649886874:5776] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:27.121230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:27.123252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:27.123450Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:27.123460Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:27.123472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:27.123474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342703649886874:5776] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:27.123499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342703649886874:5776] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:27.123504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342703649886874:5776] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:27.123513Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1831 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:27.123525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:27.123569Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342703288055262:2610]
2025-10-09T21:50:27.124149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:27.124456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:27.124500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:27.124807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:27.124857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.124864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.124865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.124866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.124867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.124869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.124870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.124871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.124872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.124878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.124890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.124894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.124897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.124900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.124904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.124908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.124911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.124914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.124921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.124929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.124936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.124942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.124946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.124951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.124956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.124961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.124965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.124970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.124977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.124982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.124987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.124992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.125001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.125006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.125014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.125019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.125048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:27.131519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:27.132820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:27.132835Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:27.132837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:27.132842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:27.132848Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1830 [2:7559342703288055262:2610]
2025-10-09T21:50:27.132869Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:27.132887Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:27.132890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:27.132893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:27.132896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:27.132913Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:27.132926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:27.132932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:27.132989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.132997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.132999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.133002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:27.133069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:27.133077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.133088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.133104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.133128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:27.133152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342703649887007:5776], now have 1 active actors on pipe
2025-10-09T21:50:27.133160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342703649886850:2432], now have 1 active actors on pipe
2025-10-09T21:50:27.133168Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342703288055257:2610]
2025-10-09T21:50:27.133173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342703288055257:2610]
2025-10-09T21:50:27.133177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342703288055257:2610]
2025-10-09T21:50:27.133189Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:27.133204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:27.133207Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:27.133214Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:27.133216Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:27.133222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:27.133225Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:27.133206Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1830
2025-10-09T21:50:27.133218Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1830
2025-10-09T21:50:27.133250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342703288055257:2610]
2025-10-09T21:50:27.133263Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:27.133273Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:27.133280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:27.133290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:27.133343Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5509b3da-3970315a-13d7c4c1-2c9e03e8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:27.133356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:27.133362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:27.133365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:27.133376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:27.133391Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:27.133414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:27.133595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:27.133606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:27.133623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:27.133741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1831 partition: 0 MaxSeqNo: 25 sessionId: src_id|5509b3da-3970315a-13d7c4c1-2c9e03e8_0
2025-10-09T21:50:27.134086Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6fb7c934-f35f78e0-ace2b03e-e21760f1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5509b3da-3970315a-13d7c4c1-2c9e03e8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:27.136490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5509b3da-3970315a-13d7c4c1-2c9e03e8_0] Write session: try to update token
2025-10-09T21:50:27.136509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5509b3da-3970315a-13d7c4c1-2c9e03e8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:27.147723Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:27.147742Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:27.147744Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:27.147745Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:27.147754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342703288055257:2610]
2025-10-09T21:50:27.147758Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342703288055257:2610]
2025-10-09T21:50:27.147761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342703288055257:2610]
2025-10-09T21:50:27.147797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:27.147802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.147804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:27.147803Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:27.147808Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:27.147810Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:27.147923Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:27.149091Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1831 sessionId: src_id|5509b3da-3970315a-13d7c4c1-2c9e03e8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:27.151261Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:27.208697Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342703649887022:6018]
2025-10-09T21:50:27.208984Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 1830, pipe: [1:7559342703649887022:6018]
2025-10-09T21:50:27.301049Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:27.301446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:27.301485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:27.301545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342703649887022:6018], now have 1 active actors on pipe
2025-10-09T21:50:27.301781Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:27.301812Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:27.301816Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:27.301841Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342703649887022:6018]
2025-10-09T21:50:27.301875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1831 reason: error at write: Disconnected sessionId: src_id|5509b3da-3970315a-13d7c4c1-2c9e03e8_0
2025-10-09T21:50:27.302021Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:27.302095Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1831 sessionId: src_id|5509b3da-3970315a-13d7c4c1-2c9e03e8_0 is DEAD
2025-10-09T21:50:27.302236Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5509b3da-3970315a-13d7c4c1-2c9e03e8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:27.302249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5509b3da-3970315a-13d7c4c1-2c9e03e8_0] Write session will restart in 0.007933s
2025-10-09T21:50:27.302285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5509b3da-3970315a-13d7c4c1-2c9e03e8_0] Write session: Do CDS request
2025-10-09T21:50:27.302292Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5509b3da-3970315a-13d7c4c1-2c9e03e8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:27.311522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5509b3da-3970315a-13d7c4c1-2c9e03e8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:27.311594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:27.311611Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1832
2025-10-09T21:50:27.311771Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1832 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:27.311806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1832 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:27.311817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1832 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:27.311821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:27.312236Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1832 sessionId:  describe result for acl check
2025-10-09T21:50:27.312273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:27.312279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:27.312281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:27.312287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342703649887173:5753] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:27.312852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342703649887173:5753] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:27.313339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:27.315215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:27.315359Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:27.315367Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:27.315376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:27.315415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:27.315420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342703649887173:5753] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:27.315450Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342703288055460:2572]
2025-10-09T21:50:27.315837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:27.316148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:27.316223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:27.316577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:27.316627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.316633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.316634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.316635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.316636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.316637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.316638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.316639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.316640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.316646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.316655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.316659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.316663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.316667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.316670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.316674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.316678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.316682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.316689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.316698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.316704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.316709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.316725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.316730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.316735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.316741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.316749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.316755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.316761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.316767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.316773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.316779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.316784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.316789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.316798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.316803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.316835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:27.321903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342703649887173:5753] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:27.321916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342703649887173:5753] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:27.321918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342703649887173:5753] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:27.321926Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1832 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:27.323285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:27.324439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:27.324457Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:27.324460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:27.324464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:27.324471Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1831 [2:7559342703288055460:2572]
2025-10-09T21:50:27.324484Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:27.324501Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:27.324506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:27.324510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:27.324512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:27.324530Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:27.324541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:27.324550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:27.324601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.324607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.324609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.324612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:27.324675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:27.324682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.324693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.324705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.324723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:27.324748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342703649887212:5753], now have 1 active actors on pipe
2025-10-09T21:50:27.324755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342703649887053:2432], now have 1 active actors on pipe
2025-10-09T21:50:27.324763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342703288055454:2572]
2025-10-09T21:50:27.324767Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342703288055454:2572]
2025-10-09T21:50:27.324769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342703288055454:2572]
2025-10-09T21:50:27.324783Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:27.324799Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:27.324795Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1831
2025-10-09T21:50:27.324799Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1831
2025-10-09T21:50:27.324803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:27.324810Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:27.324812Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:27.324818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:27.324821Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:27.324855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342703288055454:2572]
2025-10-09T21:50:27.324863Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:27.324868Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:27.324880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:27.324889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:27.324934Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|87aa322d-b1dfbf49-b4b030ad-cec7ea8e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:27.324945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:27.324950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:27.324952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:27.324962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:27.324975Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:27.324995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:27.325133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:27.325140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:27.325153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:27.325220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1832 partition: 0 MaxSeqNo: 25 sessionId: src_id|87aa322d-b1dfbf49-b4b030ad-cec7ea8e_0
2025-10-09T21:50:27.325526Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5509b3da-3970315a-13d7c4c1-2c9e03e8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|87aa322d-b1dfbf49-b4b030ad-cec7ea8e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:27.327773Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87aa322d-b1dfbf49-b4b030ad-cec7ea8e_0] Write session: try to update token
2025-10-09T21:50:27.327788Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87aa322d-b1dfbf49-b4b030ad-cec7ea8e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:27.339154Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:27.339176Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:27.339178Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:27.339179Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:27.339187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342703288055454:2572]
2025-10-09T21:50:27.339191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342703288055454:2572]
2025-10-09T21:50:27.339194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342703288055454:2572]
2025-10-09T21:50:27.339230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:27.339235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.339237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:27.339241Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:27.339248Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:27.339252Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:27.339371Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:27.339536Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1832 sessionId: src_id|87aa322d-b1dfbf49-b4b030ad-cec7ea8e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:27.341992Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:27.402245Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342703649887220:6018]
2025-10-09T21:50:27.402467Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 1831, pipe: [1:7559342703649887220:6018]
2025-10-09T21:50:27.487791Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:27.488123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:27.488137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:27.488177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342703649887220:6018], now have 1 active actors on pipe
2025-10-09T21:50:27.488371Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:27.488428Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:27.488444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:27.488452Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:27.488489Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342703649887220:6018]
2025-10-09T21:50:27.488510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1832 reason: error at write: Disconnected sessionId: src_id|87aa322d-b1dfbf49-b4b030ad-cec7ea8e_0
2025-10-09T21:50:27.488688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1832 sessionId: src_id|87aa322d-b1dfbf49-b4b030ad-cec7ea8e_0 is DEAD
2025-10-09T21:50:27.488813Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87aa322d-b1dfbf49-b4b030ad-cec7ea8e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:27.488823Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87aa322d-b1dfbf49-b4b030ad-cec7ea8e_0] Write session will restart in 0.008418s
2025-10-09T21:50:27.488844Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87aa322d-b1dfbf49-b4b030ad-cec7ea8e_0] Write session: Do CDS request
2025-10-09T21:50:27.488849Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87aa322d-b1dfbf49-b4b030ad-cec7ea8e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:27.498446Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87aa322d-b1dfbf49-b4b030ad-cec7ea8e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:27.498545Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:27.498560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1833
2025-10-09T21:50:27.498685Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1833 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:27.498720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1833 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:27.498730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1833 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:27.498733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:27.499173Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1833 sessionId:  describe result for acl check
2025-10-09T21:50:27.499208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:27.499214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:27.499216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:27.499221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342703649887345:5746] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:27.499577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:27.499792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342703649887345:5746] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:27.501366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:27.501603Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:27.501615Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:27.501628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:27.501687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:27.501725Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342703288055621:2573]
2025-10-09T21:50:27.501952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342703649887345:5746] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:27.502329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:27.502691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:27.502761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:27.503084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:27.503154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.503161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.503163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.503164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.503166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.503167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.503168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.503170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.503171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.503177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.503188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.503192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.503197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.503200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.503204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.503208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.503212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.503215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.503223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.503231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.503237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.503249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.503254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.503259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.503265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.503270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.503280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.503286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.503290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.503295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.503300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.503319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.503328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.503333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.503338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.503343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.503390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:27.505959Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342703649887345:5746] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:27.505973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342703649887345:5746] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:27.505977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342703649887345:5746] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:27.505986Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1833 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:27.509848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:27.511099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:27.511112Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:27.511114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:27.511116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:27.511121Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1832 [2:7559342703288055621:2573]
2025-10-09T21:50:27.511130Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:27.511144Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:27.511146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:27.511149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:27.511150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:27.511162Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:27.511166Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:27.511173Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:27.511214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.511219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.511220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.511223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:27.511270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:27.511275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.511283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.511296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.511314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:27.511337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342703649887385:5746], now have 1 active actors on pipe
2025-10-09T21:50:27.511346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342703649887222:2432], now have 1 active actors on pipe
2025-10-09T21:50:27.511351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342703288055615:2573]
2025-10-09T21:50:27.511353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342703288055615:2573]
2025-10-09T21:50:27.511354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342703288055615:2573]
2025-10-09T21:50:27.511367Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:27.511377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:27.511387Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:27.511391Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:27.511392Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:27.511393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:27.511396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:27.511422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342703288055615:2573]
2025-10-09T21:50:27.511429Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:27.511409Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1832
2025-10-09T21:50:27.511421Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1832
2025-10-09T21:50:27.511434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:27.511458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:27.511462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:27.511500Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2f51e00a-519565ea-671b9a3d-e26eb544_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:27.511510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:27.511514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:27.511516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:27.511523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:27.511531Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:27.511545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:27.511681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:27.511688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:27.511701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:27.511773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1833 partition: 0 MaxSeqNo: 25 sessionId: src_id|2f51e00a-519565ea-671b9a3d-e26eb544_0
2025-10-09T21:50:27.512101Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87aa322d-b1dfbf49-b4b030ad-cec7ea8e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2f51e00a-519565ea-671b9a3d-e26eb544_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:27.514617Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2f51e00a-519565ea-671b9a3d-e26eb544_0] Write session: try to update token
2025-10-09T21:50:27.514632Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2f51e00a-519565ea-671b9a3d-e26eb544_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:27.525837Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:27.525857Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:27.525859Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:27.525862Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:27.525871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342703288055615:2573]
2025-10-09T21:50:27.525876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342703288055615:2573]
2025-10-09T21:50:27.525880Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342703288055615:2573]
2025-10-09T21:50:27.525923Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:27.525924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:27.525929Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:27.525931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.525932Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:27.525933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:27.526088Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:27.527629Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1833 sessionId: src_id|2f51e00a-519565ea-671b9a3d-e26eb544_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:27.529924Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:27.588589Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342703649887393:6018]
2025-10-09T21:50:27.588857Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 1832, pipe: [1:7559342703649887393:6018]
2025-10-09T21:50:27.667447Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:27.667799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:27.667812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:27.667848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342703649887393:6018], now have 1 active actors on pipe
2025-10-09T21:50:27.668022Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:27.668023Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:27.668038Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:27.668073Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342703649887393:6018]
2025-10-09T21:50:27.668093Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:27.668119Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1833 reason: error at write: Disconnected sessionId: src_id|2f51e00a-519565ea-671b9a3d-e26eb544_0
2025-10-09T21:50:27.668283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1833 sessionId: src_id|2f51e00a-519565ea-671b9a3d-e26eb544_0 is DEAD
2025-10-09T21:50:27.668433Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f51e00a-519565ea-671b9a3d-e26eb544_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:27.668444Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f51e00a-519565ea-671b9a3d-e26eb544_0] Write session will restart in 0.009494s
2025-10-09T21:50:27.668471Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f51e00a-519565ea-671b9a3d-e26eb544_0] Write session: Do CDS request
2025-10-09T21:50:27.668478Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f51e00a-519565ea-671b9a3d-e26eb544_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:27.679524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2f51e00a-519565ea-671b9a3d-e26eb544_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:27.679540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:27.679643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:27.679659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1834
2025-10-09T21:50:27.679776Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1834 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:27.679810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1834 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:27.679820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1834 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:27.679824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:27.680367Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1834 sessionId:  describe result for acl check
2025-10-09T21:50:27.680404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:27.680409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:27.680411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:27.680417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342703649887522:5736] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:27.680915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342703649887522:5736] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:27.682104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:27.682291Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:27.682302Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:27.682311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:27.682359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:27.682400Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342703288055783:2583]
2025-10-09T21:50:27.682850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:27.683127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:27.683185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:27.683563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:27.683623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.683630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.683631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.683632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.683633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.683634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.683635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.683637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.683638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.683643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.683653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.683661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.683665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.683668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.683672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.683675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.683679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.683662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342703649887522:5736] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:27.683687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.683694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.683704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.683709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.683719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.683740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.683746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.683749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.683754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.683758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.683763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.683766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.683771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.683775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.683780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.683784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.683788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.683793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.683798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.683828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:27.689604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342703649887522:5736] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:27.689616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342703649887522:5736] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:27.689619Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342703649887522:5736] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:27.689625Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1834 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:27.690060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:27.691239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:27.691255Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:27.691257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:27.691262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:27.691268Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1833 [2:7559342703288055783:2583]
2025-10-09T21:50:27.691279Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:27.691295Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:27.691297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:27.691300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:27.691302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:27.691319Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:27.691328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:27.691339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:27.691591Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1833
2025-10-09T21:50:27.691594Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1833
2025-10-09T21:50:27.691401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.691411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.691413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.691416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:27.691479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:27.691482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.691496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.691508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.691525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:27.691544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342703649887395:2432], now have 1 active actors on pipe
2025-10-09T21:50:27.691548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342703649887559:5736], now have 1 active actors on pipe
2025-10-09T21:50:27.691554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342703288055777:2583]
2025-10-09T21:50:27.691557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342703288055777:2583]
2025-10-09T21:50:27.691559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342703288055777:2583]
2025-10-09T21:50:27.691573Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:27.691586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:27.691591Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:27.691594Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:27.691596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:27.691598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:27.691601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:27.691634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342703288055777:2583]
2025-10-09T21:50:27.691644Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:27.691658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:27.691662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:27.691665Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:27.691699Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ecf50fee-d529f27f-30808dd4-18a3f776_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:27.691706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:27.691710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:27.691712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:27.691722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:27.691731Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:27.691748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:27.691880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:27.691889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:27.691901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:27.691981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1834 partition: 0 MaxSeqNo: 25 sessionId: src_id|ecf50fee-d529f27f-30808dd4-18a3f776_0
2025-10-09T21:50:27.692289Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f51e00a-519565ea-671b9a3d-e26eb544_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ecf50fee-d529f27f-30808dd4-18a3f776_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:27.694560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf50fee-d529f27f-30808dd4-18a3f776_0] Write session: try to update token
2025-10-09T21:50:27.694576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf50fee-d529f27f-30808dd4-18a3f776_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:27.705666Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:27.705686Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:27.705688Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:27.705690Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:27.705700Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342703288055777:2583]
2025-10-09T21:50:27.705705Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342703288055777:2583]
2025-10-09T21:50:27.705709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342703288055777:2583]
2025-10-09T21:50:27.705744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:27.705750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.705751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:27.705752Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:27.705757Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:27.705767Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:27.705884Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:27.706742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1834 sessionId: src_id|ecf50fee-d529f27f-30808dd4-18a3f776_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:27.708904Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:27.768307Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342703649887567:6018]
2025-10-09T21:50:27.768548Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 1833, pipe: [1:7559342703649887567:6018]
2025-10-09T21:50:27.857575Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:27.857823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:27.857839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:27.857880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342703649887567:6018], now have 1 active actors on pipe
2025-10-09T21:50:27.858091Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:27.858120Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:27.858143Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342703649887567:6018]
2025-10-09T21:50:27.858180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:27.858259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1834 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ecf50fee-d529f27f-30808dd4-18a3f776_0
2025-10-09T21:50:27.858443Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1834 sessionId: src_id|ecf50fee-d529f27f-30808dd4-18a3f776_0 is DEAD
2025-10-09T21:50:27.858592Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf50fee-d529f27f-30808dd4-18a3f776_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:27.858602Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf50fee-d529f27f-30808dd4-18a3f776_0] Write session will restart in 0.006155s
2025-10-09T21:50:27.858622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf50fee-d529f27f-30808dd4-18a3f776_0] Write session: Do CDS request
2025-10-09T21:50:27.858627Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf50fee-d529f27f-30808dd4-18a3f776_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:27.865434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf50fee-d529f27f-30808dd4-18a3f776_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:27.865518Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:27.865534Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1835
2025-10-09T21:50:27.865653Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1835 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:27.865687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1835 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:27.865698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1835 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:27.865715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:27.866273Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1835 sessionId:  describe result for acl check
2025-10-09T21:50:27.866311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:27.866319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:27.866320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:27.866325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342703649887610:5725] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:27.866919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342703649887610:5725] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:27.869361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342703649887610:5725] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:27.870195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:27.871968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:27.872129Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:27.872138Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:27.872149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:27.872204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:27.872231Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342703288055950:2579]
2025-10-09T21:50:27.872240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342703649887610:5725] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:27.872251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342703649887610:5725] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:27.872254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342703649887610:5725] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:27.872266Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1835 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:27.872623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:27.872930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:27.872972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:27.873235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:27.873274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.873279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.873279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.873280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.873281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.873281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.873282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.873282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.873283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.873287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.873293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.873296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.873298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.873302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.873306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.873310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.873314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.873318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.873323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.873333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:27.873338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.873342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:27.873344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.873347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:27.873349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.873352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:27.873355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.873357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:27.873360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.873363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:27.873366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.873369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.873371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.873374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.873377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.873380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.873396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:27.879636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:27.880779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:27.880792Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:27.880794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:27.880797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:27.880801Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1834 [2:7559342703288055950:2579]
2025-10-09T21:50:27.880814Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:27.880828Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:27.880830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:27.880832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:27.880834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:27.880846Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:27.880849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:27.880855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:27.880895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.880900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.880901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.880903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:27.880961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:27.880973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.880979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:27.880986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:27.880995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:27.881009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342703288055945:2579]
2025-10-09T21:50:27.881010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342703288055945:2579]
2025-10-09T21:50:27.881012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342703288055945:2579]
2025-10-09T21:50:27.881018Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:27.881028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:27.881028Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:27.881031Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:27.881033Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:27.881036Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:27.881038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:27.881047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342703649887570:2432], now have 1 active actors on pipe
2025-10-09T21:50:27.881049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342703649887733:5725], now have 1 active actors on pipe
2025-10-09T21:50:27.881067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342703288055945:2579]
2025-10-09T21:50:27.881073Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:27.881081Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:27.881118Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1834
2025-10-09T21:50:27.881121Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1834
2025-10-09T21:50:27.881185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:27.881202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:27.881229Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e153e228-8c80ab3-fb51b95e-b770ec8b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:27.881241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:27.881244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:27.881246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:27.881253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:27.881260Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:27.881276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:27.881446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:27.881458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:27.881488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:27.881604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1835 partition: 0 MaxSeqNo: 25 sessionId: src_id|e153e228-8c80ab3-fb51b95e-b770ec8b_0
2025-10-09T21:50:27.881909Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf50fee-d529f27f-30808dd4-18a3f776_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e153e228-8c80ab3-fb51b95e-b770ec8b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:27.884701Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e153e228-8c80ab3-fb51b95e-b770ec8b_0] Write session: try to update token
2025-10-09T21:50:27.884714Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e153e228-8c80ab3-fb51b95e-b770ec8b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:27.892484Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:27.892502Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:27.892513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342703288055945:2579]
2025-10-09T21:50:27.892562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:27.892566Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:27.892750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:27.892901Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:27.893033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:27.893174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:27.893366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:27.893416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:27.898200Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1835 sessionId: src_id|e153e228-8c80ab3-fb51b95e-b770ec8b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:27.900616Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:27.900811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:27.900835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:27.900876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:27.900957Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:27.905528Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:27.905543Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:27.905545Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:27.905546Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:27.905554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342703288055945:2579]
2025-10-09T21:50:27.905556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342703288055945:2579]
2025-10-09T21:50:27.905559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342703288055945:2579]
2025-10-09T21:50:27.905592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:27.905597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:27.905601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:27.905602Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:27.905611Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:27.905615Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:27.905722Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:27.958261Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342703649887744:6018]
2025-10-09T21:50:27.958517Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 1834, pipe: [1:7559342703649887744:6018]
2025-10-09T21:50:28.051010Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:28.058486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.058513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.058666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.058734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.058802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.058861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.058902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.058911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.058975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.059036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.059096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.059163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.059203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.059205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.059266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.059323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.059421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.059481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.059918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.059929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.059996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.060063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.060125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.060187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.060233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.060240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.060309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.060375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.060435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.060536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.060575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.060587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.060689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.060750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.060811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.060877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.060923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.060931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.060991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.061057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.061124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.061188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.061231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.061238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.061302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.061358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.061414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.061485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.061547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.061555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.061612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.061666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.061722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.061786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.061840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.061845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.061905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.061963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.062017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.062082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.062122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.062128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.062190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342703649887744:6018], now have 1 active actors on pipe
2025-10-09T21:50:28.062457Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:28.062472Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:28.062483Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342703649887744:6018]
2025-10-09T21:50:28.062510Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1835 reason: error at write: Disconnected sessionId: src_id|e153e228-8c80ab3-fb51b95e-b770ec8b_0
2025-10-09T21:50:28.062515Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:28.062548Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:28.062687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1835 sessionId: src_id|e153e228-8c80ab3-fb51b95e-b770ec8b_0 is DEAD
2025-10-09T21:50:28.062828Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e153e228-8c80ab3-fb51b95e-b770ec8b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:28.062839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e153e228-8c80ab3-fb51b95e-b770ec8b_0] Write session will restart in 0.009798s
2025-10-09T21:50:28.062865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e153e228-8c80ab3-fb51b95e-b770ec8b_0] Write session: Do CDS request
2025-10-09T21:50:28.062871Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e153e228-8c80ab3-fb51b95e-b770ec8b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:28.073515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e153e228-8c80ab3-fb51b95e-b770ec8b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:28.073585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:28.073599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1836
2025-10-09T21:50:28.073754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1836 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:28.073790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1836 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:28.073799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1836 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:28.073803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:28.074140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1836 sessionId:  describe result for acl check
2025-10-09T21:50:28.074166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:28.074172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:28.074173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:28.074177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342707944855170:5731] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:28.074395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:28.074627Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342707944855170:5731] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:28.076184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:28.076383Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:28.076393Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:28.076403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:28.076452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:28.076490Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342707583023411:2578]
2025-10-09T21:50:28.077069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:28.077332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:28.077375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:28.077445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342707944855170:5731] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:28.077669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:28.077709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.077713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.077714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.077715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.077715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.077716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.077717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.077717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.077718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.077722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.077727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.077730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.077732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.077734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.077736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.077738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.077740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.077742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.077747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.077752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.077756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.077759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.077762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.077765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.077768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.077771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.077773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.077776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.077778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.077782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.077785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.077788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.077790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.077793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.077796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.077799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.077826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:28.083681Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342707944855170:5731] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:28.083695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342707944855170:5731] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:28.083698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342707944855170:5731] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:28.083706Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1836 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:28.084177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:28.085648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:28.085663Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:28.085666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:28.085670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:28.085678Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1835 [2:7559342707583023411:2578]
2025-10-09T21:50:28.085693Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:28.085710Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:28.085713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:28.085717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:28.085719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:28.085736Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:28.085743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:28.085751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:28.085816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.085824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.085826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.085829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:28.085906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:28.085911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.085924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.085935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.085956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:28.085984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944855208:5731], now have 1 active actors on pipe
2025-10-09T21:50:28.085988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944855043:2432], now have 1 active actors on pipe
2025-10-09T21:50:28.085996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342707583023406:2578]
2025-10-09T21:50:28.085999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342707583023406:2578]
2025-10-09T21:50:28.086001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342707583023406:2578]
2025-10-09T21:50:28.086016Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:28.085997Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1835
2025-10-09T21:50:28.086008Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1835
2025-10-09T21:50:28.086027Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:28.086050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:28.086056Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:28.086058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:28.086066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:28.086070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:28.086113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.086124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.086141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342707583023406:2578]
2025-10-09T21:50:28.086154Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:28.086166Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:28.086186Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d712f4e2-df719e3e-3c7a7ea3-5ec63311_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:28.086194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:28.086199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:28.086201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:28.086212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:28.086230Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:28.086255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:28.086435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.086444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.086460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:28.086536Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1836 partition: 0 MaxSeqNo: 25 sessionId: src_id|d712f4e2-df719e3e-3c7a7ea3-5ec63311_0
2025-10-09T21:50:28.086908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e153e228-8c80ab3-fb51b95e-b770ec8b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d712f4e2-df719e3e-3c7a7ea3-5ec63311_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:28.089418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d712f4e2-df719e3e-3c7a7ea3-5ec63311_0] Write session: try to update token
2025-10-09T21:50:28.089436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d712f4e2-df719e3e-3c7a7ea3-5ec63311_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:28.100800Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:28.100818Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:28.100820Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:28.100823Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:28.100832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342707583023406:2578]
2025-10-09T21:50:28.100837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342707583023406:2578]
2025-10-09T21:50:28.100841Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342707583023406:2578]
2025-10-09T21:50:28.100885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:28.100886Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:28.100893Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:28.100893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.100896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:28.100897Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:28.101048Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:28.103118Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1836 sessionId: src_id|d712f4e2-df719e3e-3c7a7ea3-5ec63311_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:28.105352Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:28.162550Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342707944855216:6018]
2025-10-09T21:50:28.162784Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 1835, pipe: [1:7559342707944855216:6018]
2025-10-09T21:50:28.244526Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:28.244752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.244765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.244801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944855216:6018], now have 1 active actors on pipe
2025-10-09T21:50:28.245021Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342707944855216:6018]
2025-10-09T21:50:28.245026Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:28.245030Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:28.245056Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:28.245089Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1836 reason: error at write: Disconnected sessionId: src_id|d712f4e2-df719e3e-3c7a7ea3-5ec63311_0
2025-10-09T21:50:28.245134Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:28.245256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1836 sessionId: src_id|d712f4e2-df719e3e-3c7a7ea3-5ec63311_0 is DEAD
2025-10-09T21:50:28.245418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d712f4e2-df719e3e-3c7a7ea3-5ec63311_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:28.245429Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d712f4e2-df719e3e-3c7a7ea3-5ec63311_0] Write session will restart in 0.007622s
2025-10-09T21:50:28.245449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d712f4e2-df719e3e-3c7a7ea3-5ec63311_0] Write session: Do CDS request
2025-10-09T21:50:28.245453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d712f4e2-df719e3e-3c7a7ea3-5ec63311_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:28.254423Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d712f4e2-df719e3e-3c7a7ea3-5ec63311_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:28.254554Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:28.254567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1837
2025-10-09T21:50:28.254662Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1837 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:28.254691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1837 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:28.254701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1837 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:28.254704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:28.255056Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1837 sessionId:  describe result for acl check
2025-10-09T21:50:28.255087Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:28.255092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:28.255093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:28.255097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342707944855349:5720] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:28.255136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:28.255513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342707944855349:5720] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:28.256693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:28.256856Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:28.256866Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:28.256876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:28.256933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:28.256972Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342707583023576:2576]
2025-10-09T21:50:28.257408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:28.257428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342707944855349:5720] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:28.257669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:28.257709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:28.257973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:28.258017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.258022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.258024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.258024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.258025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.258026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.258041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.258042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.258043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.258048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.258056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.258060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.258063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.258066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.258070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.258073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.258076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.258079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.258086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.258093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.258099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.258104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.258108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.258112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.258117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.258122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.258126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.258130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.258134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.258144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.258147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.258150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.258152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.258155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.258158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.258160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.258184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:28.262715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342707944855349:5720] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:28.262733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342707944855349:5720] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:28.262739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342707944855349:5720] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:28.262748Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1837 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:28.264127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:28.265188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:28.265201Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:28.265204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:28.265208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:28.265214Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1836 [2:7559342707583023576:2576]
2025-10-09T21:50:28.265231Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:28.265246Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:28.265249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:28.265252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:28.265254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:28.265268Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:28.265278Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:28.265286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:28.265343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.265350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.265352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.265354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:28.265412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:28.265420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.265432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.265444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.265476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:28.265500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944855392:5720], now have 1 active actors on pipe
2025-10-09T21:50:28.265507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944855222:2432], now have 1 active actors on pipe
2025-10-09T21:50:28.265514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342707583023570:2576]
2025-10-09T21:50:28.265516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342707583023570:2576]
2025-10-09T21:50:28.265519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342707583023570:2576]
2025-10-09T21:50:28.265531Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:28.265524Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1836
2025-10-09T21:50:28.265546Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1836
2025-10-09T21:50:28.265549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:28.265554Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:28.265556Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:28.265557Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:28.265568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:28.265573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:28.265607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.265616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.265631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342707583023570:2576]
2025-10-09T21:50:28.265639Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:28.265657Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:28.265672Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7aa3d625-3b31fe0a-acecdf87-369c5f98_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:28.265684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:28.265691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:28.265693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:28.265704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:28.265717Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:28.265737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:28.265846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.265852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.265865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:28.265946Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1837 partition: 0 MaxSeqNo: 25 sessionId: src_id|7aa3d625-3b31fe0a-acecdf87-369c5f98_0
2025-10-09T21:50:28.266273Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d712f4e2-df719e3e-3c7a7ea3-5ec63311_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7aa3d625-3b31fe0a-acecdf87-369c5f98_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:28.268643Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7aa3d625-3b31fe0a-acecdf87-369c5f98_0] Write session: try to update token
2025-10-09T21:50:28.268656Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7aa3d625-3b31fe0a-acecdf87-369c5f98_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:28.279749Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:28.279763Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:28.279765Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:28.279766Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:28.279773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342707583023570:2576]
2025-10-09T21:50:28.279776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342707583023570:2576]
2025-10-09T21:50:28.279779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342707583023570:2576]
2025-10-09T21:50:28.279821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:28.279825Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:28.279827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.279830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:28.279831Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:28.279835Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:28.279957Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:28.280685Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1837 sessionId: src_id|7aa3d625-3b31fe0a-acecdf87-369c5f98_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:28.282954Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:28.345290Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342707944855424:6018]
2025-10-09T21:50:28.345531Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 1836, pipe: [1:7559342707944855424:6018]
2025-10-09T21:50:28.422943Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:28.423256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.423270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.423308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944855424:6018], now have 1 active actors on pipe
2025-10-09T21:50:28.423489Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:28.423491Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342707944855424:6018]
2025-10-09T21:50:28.423507Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:28.423510Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:28.423525Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:28.423564Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1837 reason: error at write: Disconnected sessionId: src_id|7aa3d625-3b31fe0a-acecdf87-369c5f98_0
2025-10-09T21:50:28.423715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1837 sessionId: src_id|7aa3d625-3b31fe0a-acecdf87-369c5f98_0 is DEAD
2025-10-09T21:50:28.423870Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7aa3d625-3b31fe0a-acecdf87-369c5f98_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:28.423880Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7aa3d625-3b31fe0a-acecdf87-369c5f98_0] Write session will restart in 0.008484s
2025-10-09T21:50:28.423905Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7aa3d625-3b31fe0a-acecdf87-369c5f98_0] Write session: Do CDS request
2025-10-09T21:50:28.423910Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7aa3d625-3b31fe0a-acecdf87-369c5f98_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:28.433442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7aa3d625-3b31fe0a-acecdf87-369c5f98_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:28.433526Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:28.433540Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1838
2025-10-09T21:50:28.433973Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1838 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:28.433947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:28.434008Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1838 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:28.434018Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1838 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:28.434021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:28.434384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1838 sessionId:  describe result for acl check
2025-10-09T21:50:28.434422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:28.434428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:28.434429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:28.434433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342707944855555:5713] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:28.434856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342707944855555:5713] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:28.435420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:28.435549Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:28.435556Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:28.435567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:28.435608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:28.435636Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342707583023780:2556]
2025-10-09T21:50:28.435994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:28.436184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:28.436210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:28.436405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:28.436437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.436442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.436442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.436443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.436443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.436444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.436444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.436445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.436445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.436448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.436454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.436456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.436458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.436460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.436462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.436464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.436466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.436468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.436471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.436477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.436480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.436483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.436486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.436489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.436491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.436493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.436496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.436499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.436502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.436504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.436507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.436510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.436512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.436515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.436519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.436522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.436538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:28.436593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342707944855555:5713] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:28.442050Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342707944855555:5713] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:28.442062Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342707944855555:5713] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:28.442065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342707944855555:5713] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:28.442071Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1838 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:28.442350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:28.443322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:28.443332Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:28.443334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:28.443336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:28.443341Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1837 [2:7559342707583023780:2556]
2025-10-09T21:50:28.443349Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:28.443362Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:28.443363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:28.443366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:28.443367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:28.443380Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:28.443383Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:28.443389Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:28.443428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.443433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.443434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.443437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:28.443495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:28.443500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.443506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.443512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.443520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:28.443533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342707583023774:2556]
2025-10-09T21:50:28.443536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342707583023774:2556]
2025-10-09T21:50:28.443537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342707583023774:2556]
2025-10-09T21:50:28.443542Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:28.443551Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:28.443553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944855426:2432], now have 1 active actors on pipe
2025-10-09T21:50:28.443553Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:28.443555Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:28.443560Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:28.443567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:28.443569Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:28.443574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944855594:5713], now have 1 active actors on pipe
2025-10-09T21:50:28.443590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342707583023774:2556]
2025-10-09T21:50:28.443599Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:28.443604Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:28.443590Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1837
2025-10-09T21:50:28.443595Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1837
2025-10-09T21:50:28.443628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.443634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.443654Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2722e00d-d7fee158-fa77e53e-ceeac77c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:28.443662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:28.443665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:28.443666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:28.443674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:28.443680Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:28.443690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:28.443800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.443805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.443817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:28.443872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1838 partition: 0 MaxSeqNo: 25 sessionId: src_id|2722e00d-d7fee158-fa77e53e-ceeac77c_0
2025-10-09T21:50:28.444116Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7aa3d625-3b31fe0a-acecdf87-369c5f98_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2722e00d-d7fee158-fa77e53e-ceeac77c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:28.446575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2722e00d-d7fee158-fa77e53e-ceeac77c_0] Write session: try to update token
2025-10-09T21:50:28.446588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2722e00d-d7fee158-fa77e53e-ceeac77c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:28.453974Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:28.453990Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:28.453999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342707583023774:2556]
2025-10-09T21:50:28.454053Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:28.454059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:28.454223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:28.454351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:28.454479Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:28.454599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:28.454781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:28.454814Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:28.458294Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1838 sessionId: src_id|2722e00d-d7fee158-fa77e53e-ceeac77c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:28.460562Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:28.460715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.460730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.460774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:28.460853Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:28.465680Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:28.465691Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:28.465693Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:28.465696Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:28.465706Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342707583023774:2556]
2025-10-09T21:50:28.465710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342707583023774:2556]
2025-10-09T21:50:28.465714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342707583023774:2556]
2025-10-09T21:50:28.465751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:28.465761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.465764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:28.465770Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:28.465782Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:28.465786Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:28.465893Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:28.523506Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342707944855602:6018]
2025-10-09T21:50:28.523743Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 1837, pipe: [1:7559342707944855602:6018]
2025-10-09T21:50:28.600300Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:28.606959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.606980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.607135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.607197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.607255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.607311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.607350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.607358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.607426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.607489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.607544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.607599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.607637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.607642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.607696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.607746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.607799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.607850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.607888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.607894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.607956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.608014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.608069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.608122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.608162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.608168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.608224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.608289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.608346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.608399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.608439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.608446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.608509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.608560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.608611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.608665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.608704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.608711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.608778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.608829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.608884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.608937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.609001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.609007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.609079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.609127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.609179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.609229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.609266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.609271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.609340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.609389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.609438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.609507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.609546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.609551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.609605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:28.609652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:28.609703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:28.609754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:28.609861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:28.609869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:28.609908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944855602:6018], now have 1 active actors on pipe
2025-10-09T21:50:28.610238Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:28.610261Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342707944855602:6018]
2025-10-09T21:50:28.610289Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:28.610291Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:28.610320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1838 reason: error at write: Disconnected sessionId: src_id|2722e00d-d7fee158-fa77e53e-ceeac77c_0
2025-10-09T21:50:28.610507Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1838 sessionId: src_id|2722e00d-d7fee158-fa77e53e-ceeac77c_0 is DEAD
2025-10-09T21:50:28.610645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2722e00d-d7fee158-fa77e53e-ceeac77c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:28.610650Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2722e00d-d7fee158-fa77e53e-ceeac77c_0] Write session will restart in 0.007065s
2025-10-09T21:50:28.610668Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2722e00d-d7fee158-fa77e53e-ceeac77c_0] Write session: Do CDS request
2025-10-09T21:50:28.610672Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2722e00d-d7fee158-fa77e53e-ceeac77c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:28.610727Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:28.618437Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2722e00d-d7fee158-fa77e53e-ceeac77c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:28.618509Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:28.618525Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1839
2025-10-09T21:50:28.618627Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1839 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:28.618652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1839 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:28.618659Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1839 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:28.618662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:28.619066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1839 sessionId:  describe result for acl check
2025-10-09T21:50:28.619096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:28.619101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:28.619102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:28.619107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342707944855706:5708] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:28.619554Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342707944855706:5708] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:28.620733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:28.621962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342707944855706:5708] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:28.622774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:28.622943Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:28.622952Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:28.622962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:28.623009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:28.623044Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342707583023947:2553]
2025-10-09T21:50:28.623579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:28.623877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:28.623920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:28.624223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:28.624267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.624272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.624274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.624275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.624276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.624277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.624278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.624279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.624280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.624284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.624293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.624296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.624299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.624302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.624306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.624310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.624313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.624317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.624324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.624332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.624337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.624342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.624347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.624352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.624359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.624364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.624368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.624355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342707944855706:5708] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:28.624369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342707944855706:5708] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:28.624374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.624372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342707944855706:5708] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:28.624379Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1839 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:28.624378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.624382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.624387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.624391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.624395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.624400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.624406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.624410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.624437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:28.630554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:28.631579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:28.631591Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:28.631593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:28.631597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:28.631603Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1838 [2:7559342707583023947:2553]
2025-10-09T21:50:28.631614Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:28.631631Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:28.631639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:28.631643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:28.631645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:28.631661Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:28.631673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:28.631681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:28.631731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.631737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.631740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.631742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:28.631804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:28.631812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.631822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.631834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.631854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:28.631879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944855604:2432], now have 1 active actors on pipe
2025-10-09T21:50:28.631887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944855771:5708], now have 1 active actors on pipe
2025-10-09T21:50:28.631894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342707583023941:2553]
2025-10-09T21:50:28.631897Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342707583023941:2553]
2025-10-09T21:50:28.631899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342707583023941:2553]
2025-10-09T21:50:28.631911Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:28.631925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:28.631937Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:28.631924Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1838
2025-10-09T21:50:28.631943Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:28.631945Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:28.631945Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:28.631947Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:28.631932Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1838
2025-10-09T21:50:28.631966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342707583023941:2553]
2025-10-09T21:50:28.631969Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:28.631971Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:28.632001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.632010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.632038Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|10a04d82-abf36829-62764382-f09cfb71_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:28.632049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:28.632053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:28.632055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:28.632064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:28.632077Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:28.632095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:28.632234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.632241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.632252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:28.632373Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1839 partition: 0 MaxSeqNo: 25 sessionId: src_id|10a04d82-abf36829-62764382-f09cfb71_0
2025-10-09T21:50:28.632680Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2722e00d-d7fee158-fa77e53e-ceeac77c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|10a04d82-abf36829-62764382-f09cfb71_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:28.635048Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10a04d82-abf36829-62764382-f09cfb71_0] Write session: try to update token
2025-10-09T21:50:28.635064Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10a04d82-abf36829-62764382-f09cfb71_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:28.645310Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:28.645325Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:28.645326Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:28.645327Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:28.645335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342707583023941:2553]
2025-10-09T21:50:28.645339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342707583023941:2553]
2025-10-09T21:50:28.645342Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342707583023941:2553]
2025-10-09T21:50:28.645377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:28.645381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.645383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:28.645384Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:28.645390Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:28.645393Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:28.645824Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:28.649593Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1839 sessionId: src_id|10a04d82-abf36829-62764382-f09cfb71_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:28.651866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:28.710388Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342707944855781:6018]
2025-10-09T21:50:28.710640Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 1838, pipe: [1:7559342707944855781:6018]
2025-10-09T21:50:28.785698Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:28.786242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.786256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.786294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944855781:6018], now have 1 active actors on pipe
2025-10-09T21:50:28.786516Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:28.786535Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:28.786576Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:28.786582Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342707944855781:6018]
2025-10-09T21:50:28.786591Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:28.786670Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1839 reason: error at write: Disconnected sessionId: src_id|10a04d82-abf36829-62764382-f09cfb71_0
2025-10-09T21:50:28.786866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1839 sessionId: src_id|10a04d82-abf36829-62764382-f09cfb71_0 is DEAD
2025-10-09T21:50:28.787030Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10a04d82-abf36829-62764382-f09cfb71_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:28.787040Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10a04d82-abf36829-62764382-f09cfb71_0] Write session will restart in 0.005090s
2025-10-09T21:50:28.787068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10a04d82-abf36829-62764382-f09cfb71_0] Write session: Do CDS request
2025-10-09T21:50:28.787075Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10a04d82-abf36829-62764382-f09cfb71_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:28.793575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10a04d82-abf36829-62764382-f09cfb71_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:28.793621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:28.793647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1840
2025-10-09T21:50:28.793845Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1840 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:28.793885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1840 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:28.793891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1840 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:28.793895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:28.794365Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1840 sessionId:  describe result for acl check
2025-10-09T21:50:28.794410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:28.794413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:28.794415Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:28.794423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342707944855797:5704] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:28.795090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342707944855797:5704] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:28.798121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342707944855797:5704] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:28.800609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342707944855797:5704] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:28.800624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342707944855797:5704] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:28.800629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342707944855797:5704] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:28.800636Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1840 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:28.801624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:28.803504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:28.803670Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:28.803678Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:28.803687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:28.803723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:28.803751Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342707583024120:2528]
2025-10-09T21:50:28.804174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:28.804457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:28.804499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:28.804808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:28.804851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.804860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.804861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.804862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.804864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.804865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.804866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.804867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.804868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.804873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.804884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.804888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.804892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.804896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.804900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.804904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.804908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.804912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.804918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.804926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.804933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.804938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.804943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.804953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.804962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.804972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.804981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.804991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.805000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.805005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.805010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.805015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.805020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.805029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.805034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.805043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.805076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:28.811399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:28.812462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:28.812474Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:28.812477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:28.812481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:28.812487Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1839 [2:7559342707583024120:2528]
2025-10-09T21:50:28.812501Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:28.812521Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:28.812523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:28.812526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:28.812529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:28.812547Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:28.812559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:28.812566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:28.812634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.812641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.812644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.812646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:28.812712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:28.812719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.812731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.812750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.812768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:28.812789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944855946:5704], now have 1 active actors on pipe
2025-10-09T21:50:28.812798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944855782:2432], now have 1 active actors on pipe
2025-10-09T21:50:28.812805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342707583024115:2528]
2025-10-09T21:50:28.812809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342707583024115:2528]
2025-10-09T21:50:28.812811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342707583024115:2528]
2025-10-09T21:50:28.812824Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:28.812829Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1839
2025-10-09T21:50:28.812841Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:28.812842Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:28.812850Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:28.812853Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:28.812858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:28.812861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:28.812856Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1839
2025-10-09T21:50:28.812906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.812915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.812927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342707583024115:2528]
2025-10-09T21:50:28.812935Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:28.812947Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:28.812969Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2d78f0a1-1be23222-760f4169-2714d706_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:28.812981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:28.812985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:28.812992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:28.813003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:28.813017Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:28.813043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:28.813176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.813186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.813203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:28.813281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1840 partition: 0 MaxSeqNo: 25 sessionId: src_id|2d78f0a1-1be23222-760f4169-2714d706_0
2025-10-09T21:50:28.813630Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10a04d82-abf36829-62764382-f09cfb71_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2d78f0a1-1be23222-760f4169-2714d706_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:28.816059Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d78f0a1-1be23222-760f4169-2714d706_0] Write session: try to update token
2025-10-09T21:50:28.816077Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d78f0a1-1be23222-760f4169-2714d706_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:28.827169Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:28.827195Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:28.827197Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:28.827198Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:28.827206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342707583024115:2528]
2025-10-09T21:50:28.827209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342707583024115:2528]
2025-10-09T21:50:28.827212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342707583024115:2528]
2025-10-09T21:50:28.827252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:28.827259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.827262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:28.827269Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:28.827275Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:28.827282Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:28.827389Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:28.829137Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1840 sessionId: src_id|2d78f0a1-1be23222-760f4169-2714d706_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:28.831580Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:28.886521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342707944855963:6018]
2025-10-09T21:50:28.886759Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 1839, pipe: [1:7559342707944855963:6018]
2025-10-09T21:50:28.971462Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:28.971887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.971902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.971940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944855963:6018], now have 1 active actors on pipe
2025-10-09T21:50:28.972127Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:28.972200Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:28.972215Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:28.972214Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342707944855963:6018]
2025-10-09T21:50:28.972278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1840 reason: error at write: Disconnected sessionId: src_id|2d78f0a1-1be23222-760f4169-2714d706_0
2025-10-09T21:50:28.972306Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:28.972433Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1840 sessionId: src_id|2d78f0a1-1be23222-760f4169-2714d706_0 is DEAD
2025-10-09T21:50:28.972594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d78f0a1-1be23222-760f4169-2714d706_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:28.972618Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d78f0a1-1be23222-760f4169-2714d706_0] Write session will restart in 0.009608s
2025-10-09T21:50:28.972643Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d78f0a1-1be23222-760f4169-2714d706_0] Write session: Do CDS request
2025-10-09T21:50:28.972650Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d78f0a1-1be23222-760f4169-2714d706_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:28.983444Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d78f0a1-1be23222-760f4169-2714d706_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:28.983508Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:28.983521Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1841
2025-10-09T21:50:28.983626Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1841 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:28.983658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1841 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:28.983666Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1841 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:28.983638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:28.983669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:28.984024Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1841 sessionId:  describe result for acl check
2025-10-09T21:50:28.984059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:28.984064Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:28.984065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:28.984069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342707944856100:5690] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:28.984504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342707944856100:5690] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:28.985305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:28.985474Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:28.985485Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:28.985497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:28.985553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:28.985591Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342707583024288:2520]
2025-10-09T21:50:28.986187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:28.986439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:28.986478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:28.986482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342707944856100:5690] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:28.986766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:28.986823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.986829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.986830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.986831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.986833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.986834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.986835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.986836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.986837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.986842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.986851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.986854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.986858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.986862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.986866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.986869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.986873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.986876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.986882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.986890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:28.986895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.986900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:28.986904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.986909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:28.986913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.986918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:28.986922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.986927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:28.986931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.986936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:28.986941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.986945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.986956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.986960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.986966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.986971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.986997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:28.991699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342707944856100:5690] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:28.991715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342707944856100:5690] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:28.991719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342707944856100:5690] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:28.991737Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1841 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:28.993212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:28.994421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:28.994435Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:28.994437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:28.994441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:28.994446Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1840 [2:7559342707583024288:2520]
2025-10-09T21:50:28.994459Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:28.994474Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:28.994477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:28.994479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:28.994482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:28.994497Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:28.994504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:28.994511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:28.994562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.994569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.994572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.994575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:28.994643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:28.994650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:28.994662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:28.994675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:28.994692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:28.994716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944855967:2432], now have 1 active actors on pipe
2025-10-09T21:50:28.994724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342707944856135:5690], now have 1 active actors on pipe
2025-10-09T21:50:28.994731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342707583024282:2520]
2025-10-09T21:50:28.994733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342707583024282:2520]
2025-10-09T21:50:28.994735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342707583024282:2520]
2025-10-09T21:50:28.994747Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:28.994745Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1840
2025-10-09T21:50:28.994767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:28.994774Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:28.994753Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1840
2025-10-09T21:50:28.994777Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:28.994780Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:28.994785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:28.994788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:28.994825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.994834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.994846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342707583024282:2520]
2025-10-09T21:50:28.994854Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:28.994870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:28.994883Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|33543f0-f3dedab9-9eb083eb-fc1de4e8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:28.994890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:28.994894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:28.994897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:28.994907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:28.994922Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:28.994942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:28.995079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:28.995092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:28.995117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:28.995184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1841 partition: 0 MaxSeqNo: 25 sessionId: src_id|33543f0-f3dedab9-9eb083eb-fc1de4e8_0
2025-10-09T21:50:28.995478Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d78f0a1-1be23222-760f4169-2714d706_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|33543f0-f3dedab9-9eb083eb-fc1de4e8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:28.997724Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33543f0-f3dedab9-9eb083eb-fc1de4e8_0] Write session: try to update token
2025-10-09T21:50:28.997737Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33543f0-f3dedab9-9eb083eb-fc1de4e8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:29.009187Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:29.009203Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:29.009205Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:29.009207Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:29.009219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342707583024282:2520]
2025-10-09T21:50:29.009223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342707583024282:2520]
2025-10-09T21:50:29.009227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342707583024282:2520]
2025-10-09T21:50:29.009271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:29.009273Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:29.009278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.009280Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:29.009282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:29.009284Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:29.009418Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:29.010681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1841 sessionId: src_id|33543f0-f3dedab9-9eb083eb-fc1de4e8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:29.013073Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:29.072377Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342712239823439:6018]
2025-10-09T21:50:29.072686Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 1840, pipe: [1:7559342712239823439:6018]
2025-10-09T21:50:29.161353Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:29.161604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:29.161617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:29.161655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342712239823439:6018], now have 1 active actors on pipe
2025-10-09T21:50:29.161838Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:29.161923Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:29.161930Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:29.161989Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342712239823439:6018]
2025-10-09T21:50:29.161995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1841 reason: status is not ok: tablet will be restarted right now sessionId: src_id|33543f0-f3dedab9-9eb083eb-fc1de4e8_0
2025-10-09T21:50:29.162179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1841 sessionId: src_id|33543f0-f3dedab9-9eb083eb-fc1de4e8_0 is DEAD
2025-10-09T21:50:29.162342Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33543f0-f3dedab9-9eb083eb-fc1de4e8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:29.162352Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33543f0-f3dedab9-9eb083eb-fc1de4e8_0] Write session will restart in 0.005030s
2025-10-09T21:50:29.162379Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33543f0-f3dedab9-9eb083eb-fc1de4e8_0] Write session: Do CDS request
2025-10-09T21:50:29.162384Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33543f0-f3dedab9-9eb083eb-fc1de4e8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:29.168460Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33543f0-f3dedab9-9eb083eb-fc1de4e8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:29.168562Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:29.168577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1842
2025-10-09T21:50:29.168688Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1842 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:29.168718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1842 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:29.168737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1842 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:29.168740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:29.169085Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1842 sessionId:  describe result for acl check
2025-10-09T21:50:29.169123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:29.169124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:29.169125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:29.169129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342712239823499:5687] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:29.169566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342712239823499:5687] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:29.171757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342712239823499:5687] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:29.172846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:29.174909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:29.175058Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:29.175065Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:29.175074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:29.175110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:29.175147Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342711877991754:2507]
2025-10-09T21:50:29.175197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342712239823499:5687] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:29.175210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342712239823499:5687] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:29.175213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342712239823499:5687] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:29.175221Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1842 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:29.175523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:29.175754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:29.175781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:29.176013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:29.176050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.176054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.176055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.176056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.176056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.176057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.176057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.176058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.176058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.176062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.176068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.176071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.176073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.176075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.176077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.176079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.176081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.176083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.176087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.176092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.176096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.176099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.176101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.176104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.176107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.176110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.176112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.176115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.176118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.176121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.176124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.176127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.176129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.176132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.176135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.176138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.176155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:29.182414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:29.183418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:29.183430Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:29.183432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:29.183435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:29.183441Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1841 [2:7559342711877991754:2507]
2025-10-09T21:50:29.183455Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:29.183471Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:29.183473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:29.183476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:29.183479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:29.183492Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:29.183496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:29.183502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:29.183539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.183547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.183548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.183550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:29.183708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:29.183714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.183721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.183727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.183737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:29.183752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342712239823441:2432], now have 1 active actors on pipe
2025-10-09T21:50:29.183754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342712239823616:5687], now have 1 active actors on pipe
2025-10-09T21:50:29.183759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342711877991749:2507]
2025-10-09T21:50:29.183760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342711877991749:2507]
2025-10-09T21:50:29.183733Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1841
2025-10-09T21:50:29.183762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342711877991749:2507]
2025-10-09T21:50:29.183739Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1841
2025-10-09T21:50:29.183768Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:29.183779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:29.183782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:29.183785Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:29.183787Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:29.183789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:29.183792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:29.183816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:29.183826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:29.183852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342711877991749:2507]
2025-10-09T21:50:29.183861Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:29.183867Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:29.183877Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|152ba22d-4bafb8c-b711e911-5636ab04_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:29.183887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:29.183890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:29.183891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:29.183899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:29.183906Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:29.183932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:29.184066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:29.184072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:29.184083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:29.184157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1842 partition: 0 MaxSeqNo: 25 sessionId: src_id|152ba22d-4bafb8c-b711e911-5636ab04_0
2025-10-09T21:50:29.184494Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33543f0-f3dedab9-9eb083eb-fc1de4e8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|152ba22d-4bafb8c-b711e911-5636ab04_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:29.187084Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|152ba22d-4bafb8c-b711e911-5636ab04_0] Write session: try to update token
2025-10-09T21:50:29.187102Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|152ba22d-4bafb8c-b711e911-5636ab04_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:29.198177Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:29.198194Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:29.198196Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:29.198197Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:29.198204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342711877991749:2507]
2025-10-09T21:50:29.198207Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342711877991749:2507]
2025-10-09T21:50:29.198211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342711877991749:2507]
2025-10-09T21:50:29.198247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:29.198252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.198253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:29.198274Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:29.198281Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:29.198285Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:29.198386Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:29.200027Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1842 sessionId: src_id|152ba22d-4bafb8c-b711e911-5636ab04_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:29.202441Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:29.262322Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342712239823632:6018]
2025-10-09T21:50:29.262551Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 1841, pipe: [1:7559342712239823632:6018]
2025-10-09T21:50:29.343425Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:29.343674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:29.343684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:29.343719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342712239823632:6018], now have 1 active actors on pipe
2025-10-09T21:50:29.343909Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:29.343913Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:29.343917Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:29.343970Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342712239823632:6018]
2025-10-09T21:50:29.343974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1842 reason: error at write: Disconnected sessionId: src_id|152ba22d-4bafb8c-b711e911-5636ab04_0
2025-10-09T21:50:29.344184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1842 sessionId: src_id|152ba22d-4bafb8c-b711e911-5636ab04_0 is DEAD
2025-10-09T21:50:29.344293Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|152ba22d-4bafb8c-b711e911-5636ab04_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:29.344304Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|152ba22d-4bafb8c-b711e911-5636ab04_0] Write session will restart in 0.006575s
2025-10-09T21:50:29.344333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|152ba22d-4bafb8c-b711e911-5636ab04_0] Write session: Do CDS request
2025-10-09T21:50:29.344344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|152ba22d-4bafb8c-b711e911-5636ab04_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:29.344569Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:29.352473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|152ba22d-4bafb8c-b711e911-5636ab04_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:29.352578Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:29.352601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1843
2025-10-09T21:50:29.352720Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1843 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:29.352753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1843 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:29.352762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1843 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:29.352766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:29.353091Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1843 sessionId:  describe result for acl check
2025-10-09T21:50:29.353119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:29.353124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:29.353125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:29.353129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342712239823764:5673] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:29.353580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342712239823764:5673] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:29.355083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:29.355321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342712239823764:5673] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:29.356889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:29.357055Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:29.357064Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:29.357076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:29.357121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:29.357152Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342711877991963:2482]
2025-10-09T21:50:29.357580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342712239823764:5673] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:29.357604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342712239823764:5673] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:29.357608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342712239823764:5673] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:29.357621Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1843 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:29.357685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:29.357956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:29.357989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:29.358241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:29.358281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.358285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.358286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.358287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.358287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.358288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.358288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.358289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.358289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.358293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.358299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.358302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.358304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.358306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.358308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.358310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.358312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.358314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.358318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.358323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.358327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.358331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.358333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.358337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.358339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.358342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.358344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.358347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.358349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.358352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.358355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.358358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.358360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.358363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.358368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.358373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.358400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:29.364527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:29.365692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:29.365706Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:29.365708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:29.365713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:29.365718Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1842 [2:7559342711877991963:2482]
2025-10-09T21:50:29.365730Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:29.365745Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:29.365748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:29.365751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:29.365753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:29.365770Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:29.365775Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:29.365781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:29.365832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.365839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.365841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.365844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:29.365905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:29.365914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.365924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.365935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.365950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:29.365974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342712239823664:2432], now have 1 active actors on pipe
2025-10-09T21:50:29.365978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342712239823830:5673], now have 1 active actors on pipe
2025-10-09T21:50:29.365985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342711877991957:2482]
2025-10-09T21:50:29.365987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342711877991957:2482]
2025-10-09T21:50:29.365989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342711877991957:2482]
2025-10-09T21:50:29.366000Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:29.366015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:29.366041Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:29.366047Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:29.366049Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:29.366050Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:29.366052Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:29.366072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342711877991957:2482]
2025-10-09T21:50:29.366075Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:29.366077Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:29.366046Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1842
2025-10-09T21:50:29.366064Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1842
2025-10-09T21:50:29.366105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:29.366116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:29.366144Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b4e00d4c-47fcd61f-ac9f5281-221b5b0e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:29.366155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:29.366159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:29.366161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:29.366172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:29.366185Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:29.366203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:29.366367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:29.366376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:29.366390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:29.366459Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1843 partition: 0 MaxSeqNo: 25 sessionId: src_id|b4e00d4c-47fcd61f-ac9f5281-221b5b0e_0
2025-10-09T21:50:29.366741Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|152ba22d-4bafb8c-b711e911-5636ab04_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b4e00d4c-47fcd61f-ac9f5281-221b5b0e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:29.368863Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4e00d4c-47fcd61f-ac9f5281-221b5b0e_0] Write session: try to update token
2025-10-09T21:50:29.368875Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4e00d4c-47fcd61f-ac9f5281-221b5b0e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:29.380248Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:29.380264Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:29.380266Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:29.380268Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:29.380277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342711877991957:2482]
2025-10-09T21:50:29.380282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342711877991957:2482]
2025-10-09T21:50:29.380286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342711877991957:2482]
2025-10-09T21:50:29.380329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:29.380332Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:29.380336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.380339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:29.380345Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:29.380349Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:29.380462Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:29.380715Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1843 sessionId: src_id|b4e00d4c-47fcd61f-ac9f5281-221b5b0e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:29.383139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:29.444096Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342712239823841:6018]
2025-10-09T21:50:29.444377Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 1842, pipe: [1:7559342712239823841:6018]
2025-10-09T21:50:29.524565Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:29.524944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:29.524965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:29.525024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342712239823841:6018], now have 1 active actors on pipe
2025-10-09T21:50:29.525218Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:29.525243Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:29.525291Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:29.525414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1843 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b4e00d4c-47fcd61f-ac9f5281-221b5b0e_0
2025-10-09T21:50:29.525607Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1843 sessionId: src_id|b4e00d4c-47fcd61f-ac9f5281-221b5b0e_0 is DEAD
2025-10-09T21:50:29.525740Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342712239823841:6018]
2025-10-09T21:50:29.525780Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4e00d4c-47fcd61f-ac9f5281-221b5b0e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:29.525791Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4e00d4c-47fcd61f-ac9f5281-221b5b0e_0] Write session will restart in 0.005138s
2025-10-09T21:50:29.525817Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4e00d4c-47fcd61f-ac9f5281-221b5b0e_0] Write session: Do CDS request
2025-10-09T21:50:29.525824Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4e00d4c-47fcd61f-ac9f5281-221b5b0e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:29.532435Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4e00d4c-47fcd61f-ac9f5281-221b5b0e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:29.532563Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:29.532576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1844
2025-10-09T21:50:29.532699Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1844 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:29.532730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1844 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:29.532740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1844 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:29.532747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:29.533200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1844 sessionId:  describe result for acl check
2025-10-09T21:50:29.533239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:29.533243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:29.533244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:29.533248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342712239823889:5664] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:29.533681Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342712239823889:5664] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:29.535742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342712239823889:5664] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:29.537000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:29.538680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:29.538744Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342712239823889:5664] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:29.538755Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342712239823889:5664] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:29.538757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342712239823889:5664] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:29.538764Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1844 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:29.538839Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:29.538850Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:29.538863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:29.538918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:29.538957Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342711877992135:2494]
2025-10-09T21:50:29.539501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:29.539769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:29.539811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:29.540089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:29.540141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.540147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.540148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.540149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.540150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.540151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.540152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.540153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.540154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.540160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.540169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.540178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.540182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.540185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.540191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.540195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.540203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.540207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.540212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.540224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.540234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.540243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.540252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.540257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.540266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.540274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.540280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.540285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.540289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.540294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.540299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.540311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.540315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.540320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.540323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.540328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.540359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:29.546358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:29.547496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:29.547510Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:29.547512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:29.547516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:29.547522Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1843 [2:7559342711877992135:2494]
2025-10-09T21:50:29.547533Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:29.547546Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:29.547549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:29.547551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:29.547554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:29.547569Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:29.547581Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:29.547588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:29.547638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.547645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.547647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.547649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:29.547711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:29.547719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.547730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.547741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.547758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:29.547786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342712239824014:5664], now have 1 active actors on pipe
2025-10-09T21:50:29.547793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342712239823844:2432], now have 1 active actors on pipe
2025-10-09T21:50:29.547800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342711877992130:2494]
2025-10-09T21:50:29.547804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342711877992130:2494]
2025-10-09T21:50:29.547806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342711877992130:2494]
2025-10-09T21:50:29.547819Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:29.547827Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:29.547830Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:29.547831Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:29.547837Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:29.547850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:29.547858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:29.547878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342711877992130:2494]
2025-10-09T21:50:29.547885Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:29.547858Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1843
2025-10-09T21:50:29.547858Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1843
2025-10-09T21:50:29.547894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:29.547925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:29.547935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:29.547960Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c1d8a924-9a6455e5-23bbd9a-a0bb6106_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:29.547973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:29.547977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:29.547979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:29.547990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:29.547999Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:29.548015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:29.548238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:29.548249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:29.548265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:29.548368Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1844 partition: 0 MaxSeqNo: 25 sessionId: src_id|c1d8a924-9a6455e5-23bbd9a-a0bb6106_0
2025-10-09T21:50:29.548676Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4e00d4c-47fcd61f-ac9f5281-221b5b0e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c1d8a924-9a6455e5-23bbd9a-a0bb6106_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:29.551542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1d8a924-9a6455e5-23bbd9a-a0bb6106_0] Write session: try to update token
2025-10-09T21:50:29.551558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1d8a924-9a6455e5-23bbd9a-a0bb6106_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:29.558926Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:29.558944Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:29.558954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342711877992130:2494]
2025-10-09T21:50:29.559000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:29.559005Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:29.559217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:29.559364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:29.559514Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:29.559647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:29.559827Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:29.559870Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:29.563631Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1844 sessionId: src_id|c1d8a924-9a6455e5-23bbd9a-a0bb6106_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:29.566002Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:29.566204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:29.566223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:29.566260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:29.566341Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:29.571132Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:29.571148Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:29.571150Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:29.571152Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:29.571160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342711877992130:2494]
2025-10-09T21:50:29.571165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342711877992130:2494]
2025-10-09T21:50:29.571170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342711877992130:2494]
2025-10-09T21:50:29.571208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:29.571216Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:29.571220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.571221Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:29.571223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:29.571225Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:29.571349Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:29.625853Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342712239824025:6018]
2025-10-09T21:50:29.626157Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 1843, pipe: [1:7559342712239824025:6018]
2025-10-09T21:50:29.706582Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:29.713040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:29.713060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:29.713192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:29.713247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:29.713305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:29.713360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:29.713398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:29.713404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:29.713479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:29.713539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:29.713593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:29.713650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:29.713690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:29.713695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:29.713747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:29.713806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:29.713859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:29.713908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:29.713946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:29.713951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:29.714007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:29.714081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:29.714133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:29.714186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:29.714223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:29.714231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:29.714289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:29.714348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:29.714403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:29.714456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:29.714494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:29.714499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:29.714558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:29.714611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:29.714661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:29.714712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:29.714763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:29.714770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:29.714825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:29.714874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:29.714926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:29.714973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:29.715008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:29.715013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:29.715073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:29.715126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:29.715178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:29.715228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:29.715264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:29.715269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:29.715324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:29.715375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:29.715424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:29.715474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:29.715510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:29.715516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:29.715572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:29.715626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:29.715732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:29.715840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:29.715917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:29.715922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:29.715958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342712239824025:6018], now have 1 active actors on pipe
2025-10-09T21:50:29.716185Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:29.716193Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:29.716197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:29.716229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1844 reason: error at write: Disconnected sessionId: src_id|c1d8a924-9a6455e5-23bbd9a-a0bb6106_0
2025-10-09T21:50:29.716248Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342712239824025:6018]
2025-10-09T21:50:29.716425Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1844 sessionId: src_id|c1d8a924-9a6455e5-23bbd9a-a0bb6106_0 is DEAD
2025-10-09T21:50:29.716552Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1d8a924-9a6455e5-23bbd9a-a0bb6106_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:29.716560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1d8a924-9a6455e5-23bbd9a-a0bb6106_0] Write session will restart in 0.008923s
2025-10-09T21:50:29.716585Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1d8a924-9a6455e5-23bbd9a-a0bb6106_0] Write session: Do CDS request
2025-10-09T21:50:29.716590Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1d8a924-9a6455e5-23bbd9a-a0bb6106_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:29.716603Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:29.726473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1d8a924-9a6455e5-23bbd9a-a0bb6106_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:29.726542Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:29.726559Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1845
2025-10-09T21:50:29.726695Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1845 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:29.726727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1845 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:29.726733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1845 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:29.726742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:29.727243Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1845 sessionId:  describe result for acl check
2025-10-09T21:50:29.727277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:29.727285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:29.727287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:29.727292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342712239824163:5663] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:29.727645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:29.727798Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342712239824163:5663] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:29.729486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:29.729611Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:29.729620Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:29.729630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:29.729671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:29.729708Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342711877992309:2492]
2025-10-09T21:50:29.730068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342712239824163:5663] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:29.730214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:29.730480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:29.730518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:29.730801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:29.730841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.730847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.730848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.730849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.730850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.730851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.730852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.730853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.730854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.730858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.730867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.730870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.730873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.730876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.730879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.730882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.730885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.730888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.730894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.730900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.730905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.730910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.730913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.730918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.730922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.730931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.730935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.730939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.730943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.730947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.730952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.730957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.730961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.730966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.730971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.730976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.731008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:29.735298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342712239824163:5663] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:29.735311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342712239824163:5663] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:29.735315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342712239824163:5663] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:29.735325Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1845 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:29.736672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:29.737680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:29.737693Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:29.737695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:29.737699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:29.737704Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1844 [2:7559342711877992309:2492]
2025-10-09T21:50:29.737717Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:29.737731Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:29.737738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:29.737741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:29.737743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:29.737757Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:29.737766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:29.737776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:29.737821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.737828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.737829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.737832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:29.737890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:29.737901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.737912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.737923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.737940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:29.737962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342712239824203:5663], now have 1 active actors on pipe
2025-10-09T21:50:29.737970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342712239824028:2432], now have 1 active actors on pipe
2025-10-09T21:50:29.737976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342711877992303:2492]
2025-10-09T21:50:29.737978Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342711877992303:2492]
2025-10-09T21:50:29.737980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342711877992303:2492]
2025-10-09T21:50:29.737991Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:29.738009Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:29.738017Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:29.738020Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:29.738022Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:29.738040Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:29.738044Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:29.738013Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1844
2025-10-09T21:50:29.738062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342711877992303:2492]
2025-10-09T21:50:29.738024Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1844
2025-10-09T21:50:29.738070Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:29.738071Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:29.738085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:29.738093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:29.738131Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cca9e4df-93640168-b959199f-b248bae0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:29.738143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:29.738150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:29.738152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:29.738161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:29.738173Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:29.738191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:29.738345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:29.738354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:29.738368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:29.738447Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1845 partition: 0 MaxSeqNo: 25 sessionId: src_id|cca9e4df-93640168-b959199f-b248bae0_0
2025-10-09T21:50:29.738699Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1d8a924-9a6455e5-23bbd9a-a0bb6106_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cca9e4df-93640168-b959199f-b248bae0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:29.740893Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cca9e4df-93640168-b959199f-b248bae0_0] Write session: try to update token
2025-10-09T21:50:29.740907Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cca9e4df-93640168-b959199f-b248bae0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:29.751627Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:29.751642Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:29.751645Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:29.751646Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:29.751655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342711877992303:2492]
2025-10-09T21:50:29.751659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342711877992303:2492]
2025-10-09T21:50:29.751663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342711877992303:2492]
2025-10-09T21:50:29.751703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:29.751713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.751713Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:29.751719Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:29.751720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:29.751723Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:29.751843Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:29.752235Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1845 sessionId: src_id|cca9e4df-93640168-b959199f-b248bae0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:29.754602Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:29.815391Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: CLIENT_DEADLINE_EXCEEDED. Description: [ { <main>: Error: GRpc error: (4): Deadline Exceeded } { <main>: Error: Grpc error response on endpoint localhost:14704 } ]
2025-10-09T21:50:29.816308Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342712239824210:6018]
2025-10-09T21:50:29.816548Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 1844, pipe: [1:7559342712239824210:6018]
2025-10-09T21:50:29.896996Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:29.897293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:29.897305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:29.897341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342712239824210:6018], now have 1 active actors on pipe
2025-10-09T21:50:29.897590Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:29.897606Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342712239824210:6018]
2025-10-09T21:50:29.897634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:29.897636Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:29.897707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1845 reason: error at write: Disconnected sessionId: src_id|cca9e4df-93640168-b959199f-b248bae0_0
2025-10-09T21:50:29.897709Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:29.897892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1845 sessionId: src_id|cca9e4df-93640168-b959199f-b248bae0_0 is DEAD
2025-10-09T21:50:29.898070Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cca9e4df-93640168-b959199f-b248bae0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:29.898080Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cca9e4df-93640168-b959199f-b248bae0_0] Write session will restart in 0.008239s
2025-10-09T21:50:29.898107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cca9e4df-93640168-b959199f-b248bae0_0] Write session: Do CDS request
2025-10-09T21:50:29.898113Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cca9e4df-93640168-b959199f-b248bae0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:29.907495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cca9e4df-93640168-b959199f-b248bae0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:29.907567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:29.907583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1846
2025-10-09T21:50:29.907690Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1846 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:29.907722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1846 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:29.907734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1846 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:29.907743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:29.908137Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1846 sessionId:  describe result for acl check
2025-10-09T21:50:29.908170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:29.908177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:29.908179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:29.908185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342712239824339:5647] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:29.908678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342712239824339:5647] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:29.909659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:29.910774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342712239824339:5647] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:29.911530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:29.911706Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:29.911716Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:29.911727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:29.911778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:29.911815Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342711877992487:2484]
2025-10-09T21:50:29.912394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:29.912641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:29.912683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:29.912980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:29.913028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.913034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.913036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.913037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.913038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.913039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.913040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.913041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.913042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.913047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.913056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.913059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.913063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.913066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.913070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.913073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.913076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.913080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.913087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.913094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:29.913100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.913111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:29.913120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.913088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342712239824339:5647] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:29.913101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342712239824339:5647] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:29.913124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:29.913103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342712239824339:5647] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:29.913133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.913122Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1846 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:29.913138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:29.913146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.913157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:29.913162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.913168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:29.913173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.913178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.913183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.913188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.913193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.913198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.913228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:29.919194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:29.920194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:29.920209Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:29.920211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:29.920216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:29.920221Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1845 [2:7559342711877992487:2484]
2025-10-09T21:50:29.920232Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:29.920247Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:29.920250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:29.920253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:29.920256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:29.920270Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:29.920275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:29.920282Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:29.920332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.920342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.920344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.920347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:29.920411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:29.920418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.920429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:29.920440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:29.920457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:29.920480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342712239824389:5647], now have 1 active actors on pipe
2025-10-09T21:50:29.920488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342712239824215:2432], now have 1 active actors on pipe
2025-10-09T21:50:29.920494Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342711877992481:2484]
2025-10-09T21:50:29.920497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342711877992481:2484]
2025-10-09T21:50:29.920499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342711877992481:2484]
2025-10-09T21:50:29.920511Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:29.920523Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:29.920526Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:29.920527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:29.920528Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:29.920541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:29.920548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:29.920570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342711877992481:2484]
2025-10-09T21:50:29.920577Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:29.920558Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1845
2025-10-09T21:50:29.920574Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1845
2025-10-09T21:50:29.920604Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:29.920609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:29.920614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:29.920641Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f41fbece-f8fdcd48-88a5226b-f4c3752_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:29.920652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:29.920656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:29.920658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:29.920667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:29.920680Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:29.920696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:29.920810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:29.920819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:29.920833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:29.920902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1846 partition: 0 MaxSeqNo: 25 sessionId: src_id|f41fbece-f8fdcd48-88a5226b-f4c3752_0
2025-10-09T21:50:29.921172Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cca9e4df-93640168-b959199f-b248bae0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f41fbece-f8fdcd48-88a5226b-f4c3752_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:29.923569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f41fbece-f8fdcd48-88a5226b-f4c3752_0] Write session: try to update token
2025-10-09T21:50:29.923588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f41fbece-f8fdcd48-88a5226b-f4c3752_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:29.933444Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:29.933459Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:29.933476Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:29.933477Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:29.933487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342711877992481:2484]
2025-10-09T21:50:29.933490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342711877992481:2484]
2025-10-09T21:50:29.933494Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342711877992481:2484]
2025-10-09T21:50:29.933534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:29.933535Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:29.933542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:29.933544Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:29.933545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:29.933548Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:29.933673Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:29.936254Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1846 sessionId: src_id|f41fbece-f8fdcd48-88a5226b-f4c3752_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:29.938726Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:29.997548Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342712239824400:6018]
2025-10-09T21:50:29.997819Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 1845, pipe: [1:7559342712239824400:6018]
2025-10-09T21:50:30.076223Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:30.076698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:30.076713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:30.076755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342712239824400:6018], now have 1 active actors on pipe
2025-10-09T21:50:30.076933Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:30.076977Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:30.077003Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342712239824400:6018]
2025-10-09T21:50:30.077019Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:30.077021Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:30.077080Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1846 reason: error at write: Disconnected sessionId: src_id|f41fbece-f8fdcd48-88a5226b-f4c3752_0
2025-10-09T21:50:30.077250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1846 sessionId: src_id|f41fbece-f8fdcd48-88a5226b-f4c3752_0 is DEAD
2025-10-09T21:50:30.077399Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f41fbece-f8fdcd48-88a5226b-f4c3752_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:30.077409Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f41fbece-f8fdcd48-88a5226b-f4c3752_0] Write session will restart in 0.007603s
2025-10-09T21:50:30.077434Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f41fbece-f8fdcd48-88a5226b-f4c3752_0] Write session: Do CDS request
2025-10-09T21:50:30.077440Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f41fbece-f8fdcd48-88a5226b-f4c3752_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:30.086525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f41fbece-f8fdcd48-88a5226b-f4c3752_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:30.086547Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:30.086561Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1847
2025-10-09T21:50:30.086766Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1847 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:30.086805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1847 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:30.086817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1847 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:30.086820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:30.087274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1847 sessionId:  describe result for acl check
2025-10-09T21:50:30.087312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:30.087314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:30.087316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:30.087322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342716534791820:5655] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:30.087869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342716534791820:5655] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:30.089147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:30.090089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342716534791820:5655] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:30.091205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:30.091378Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:30.091394Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:30.091405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:30.091462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:30.091504Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342716172959958:2488]
2025-10-09T21:50:30.092080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:30.092389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:30.092406Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342716534791820:5655] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:30.092423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342716534791820:5655] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:30.092427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342716534791820:5655] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:30.092434Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1847 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:30.092435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:30.092754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:30.092813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.092819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.092821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.092822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.092823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.092824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.092825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.092826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.092827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.092832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.092843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.092847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.092851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.092855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.092859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.092863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.092866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.092870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.092876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.092884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.092891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.092896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.092900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.092905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.092909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.092915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.092921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.092927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.092933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.092943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.092948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.092953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.092957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.092962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.092967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.092974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.093002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:30.099450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:30.100787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:30.100807Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:30.100810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:30.100814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:30.100821Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1846 [2:7559342716172959958:2488]
2025-10-09T21:50:30.100835Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:30.100854Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:30.100858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:30.100861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:30.100864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:30.100881Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:30.100892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:30.100903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:30.100964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.100971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.100973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.100977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:30.101052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:30.101064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.101077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.101089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.101107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:30.101133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342716534791874:5655], now have 1 active actors on pipe
2025-10-09T21:50:30.101145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342716534791699:2432], now have 1 active actors on pipe
2025-10-09T21:50:30.101157Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342716172959953:2488]
2025-10-09T21:50:30.101159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342716172959953:2488]
2025-10-09T21:50:30.101162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342716172959953:2488]
2025-10-09T21:50:30.101177Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:30.101194Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:30.101198Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:30.101201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:30.101203Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:30.101215Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:30.101218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:30.101210Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1846
2025-10-09T21:50:30.101214Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1846
2025-10-09T21:50:30.101254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342716172959953:2488]
2025-10-09T21:50:30.101265Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:30.101267Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:30.101283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:30.101288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:30.101341Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4ad3c2b8-258bbd06-e146071f-673b944e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:30.101354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:30.101358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:30.101360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:30.101370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:30.101386Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:30.101409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:30.101577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:30.101588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:30.101602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:30.101707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1847 partition: 0 MaxSeqNo: 25 sessionId: src_id|4ad3c2b8-258bbd06-e146071f-673b944e_0
2025-10-09T21:50:30.102056Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f41fbece-f8fdcd48-88a5226b-f4c3752_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4ad3c2b8-258bbd06-e146071f-673b944e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:30.104655Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ad3c2b8-258bbd06-e146071f-673b944e_0] Write session: try to update token
2025-10-09T21:50:30.104669Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ad3c2b8-258bbd06-e146071f-673b944e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:30.115366Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:30.115383Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:30.115386Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:30.115388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:30.115398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342716172959953:2488]
2025-10-09T21:50:30.115403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342716172959953:2488]
2025-10-09T21:50:30.115407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342716172959953:2488]
2025-10-09T21:50:30.115451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:30.115453Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:30.115458Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:30.115461Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:30.115462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.115465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:30.115599Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:30.117527Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1847 sessionId: src_id|4ad3c2b8-258bbd06-e146071f-673b944e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:30.119974Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:30.177254Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342716534791884:6018]
2025-10-09T21:50:30.177537Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 1846, pipe: [1:7559342716534791884:6018]
2025-10-09T21:50:30.266133Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:30.266471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:30.266483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:30.266522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342716534791884:6018], now have 1 active actors on pipe
2025-10-09T21:50:30.266778Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:30.266765Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:30.266782Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342716534791884:6018]
2025-10-09T21:50:30.266805Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:30.266807Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:30.266874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1847 reason: error at write: Disconnected sessionId: src_id|4ad3c2b8-258bbd06-e146071f-673b944e_0
2025-10-09T21:50:30.267071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1847 sessionId: src_id|4ad3c2b8-258bbd06-e146071f-673b944e_0 is DEAD
2025-10-09T21:50:30.267236Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ad3c2b8-258bbd06-e146071f-673b944e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:30.267246Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ad3c2b8-258bbd06-e146071f-673b944e_0] Write session will restart in 0.006765s
2025-10-09T21:50:30.267275Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ad3c2b8-258bbd06-e146071f-673b944e_0] Write session: Do CDS request
2025-10-09T21:50:30.267280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ad3c2b8-258bbd06-e146071f-673b944e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:30.275483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ad3c2b8-258bbd06-e146071f-673b944e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:30.275593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:30.275609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1848
2025-10-09T21:50:30.275747Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1848 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:30.275782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1848 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:30.275791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1848 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:30.275794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:30.276211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1848 sessionId:  describe result for acl check
2025-10-09T21:50:30.276248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:30.276255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:30.276256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:30.276262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342716534791969:5646] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:30.276810Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342716534791969:5646] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:30.279014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342716534791969:5646] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:30.279227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:30.281274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:30.281451Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:30.281472Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:30.281484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:30.281532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:30.281572Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342716172960134:2485]
2025-10-09T21:50:30.281571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342716534791969:5646] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:30.281585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342716534791969:5646] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:30.281589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342716534791969:5646] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:30.281597Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1848 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:30.282157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:30.282441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:30.282486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:30.282806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:30.282858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.282865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.282866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.282867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.282868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.282869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.282870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.282871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.282872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.282876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.282888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.282892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.282896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.282900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.282905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.282908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.282912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.282916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.282923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.282931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.282937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.282943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.282947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.282952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.282956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.282961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.282965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.282970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.282975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.282980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.282986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.282992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.282996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.283001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.283006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.283011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.283041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:30.289401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:30.290677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:30.290689Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:30.290692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:30.290696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:30.290703Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1847 [2:7559342716172960134:2485]
2025-10-09T21:50:30.290717Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:30.290734Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:30.290737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:30.290740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:30.290743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:30.290759Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:30.290770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:30.290777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:30.290837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.290844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.290846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.290849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:30.290915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:30.290923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.290935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.290948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.290970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:30.290996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342716534792068:5646], now have 1 active actors on pipe
2025-10-09T21:50:30.291003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342716534791889:2432], now have 1 active actors on pipe
2025-10-09T21:50:30.291010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342716172960129:2485]
2025-10-09T21:50:30.291012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342716172960129:2485]
2025-10-09T21:50:30.291014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342716172960129:2485]
2025-10-09T21:50:30.291028Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:30.291042Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:30.291052Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:30.291056Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:30.291058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:30.291060Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:30.291064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:30.291088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342716172960129:2485]
2025-10-09T21:50:30.291072Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1847
2025-10-09T21:50:30.291096Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:30.291072Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1847
2025-10-09T21:50:30.291098Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:30.291120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:30.291125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:30.291163Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d56a67ad-34969441-8fd46db5-1d3844bd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:30.291177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:30.291181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:30.291184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:30.291194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:30.291204Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:30.291231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:30.291400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:30.291408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:30.291423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:30.291543Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1848 partition: 0 MaxSeqNo: 25 sessionId: src_id|d56a67ad-34969441-8fd46db5-1d3844bd_0
2025-10-09T21:50:30.291915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ad3c2b8-258bbd06-e146071f-673b944e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d56a67ad-34969441-8fd46db5-1d3844bd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:30.294244Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d56a67ad-34969441-8fd46db5-1d3844bd_0] Write session: try to update token
2025-10-09T21:50:30.294261Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d56a67ad-34969441-8fd46db5-1d3844bd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:30.305857Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:30.305874Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:30.305877Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:30.305878Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:30.305888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342716172960129:2485]
2025-10-09T21:50:30.305893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342716172960129:2485]
2025-10-09T21:50:30.305897Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342716172960129:2485]
2025-10-09T21:50:30.305943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:30.305948Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:30.305951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.305953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:30.305954Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:30.305958Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:30.306136Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:30.306616Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1848 sessionId: src_id|d56a67ad-34969441-8fd46db5-1d3844bd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:30.309070Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:30.366674Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342716534792093:6018]
2025-10-09T21:50:30.367119Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 1847, pipe: [1:7559342716534792093:6018]
2025-10-09T21:50:30.452419Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:30.452643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:30.452656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:30.452693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342716534792093:6018], now have 1 active actors on pipe
2025-10-09T21:50:30.452857Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:30.452915Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:30.452921Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342716534792093:6018]
2025-10-09T21:50:30.452956Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:30.453041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1848 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d56a67ad-34969441-8fd46db5-1d3844bd_0
2025-10-09T21:50:30.453231Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1848 sessionId: src_id|d56a67ad-34969441-8fd46db5-1d3844bd_0 is DEAD
2025-10-09T21:50:30.453371Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d56a67ad-34969441-8fd46db5-1d3844bd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:30.453382Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d56a67ad-34969441-8fd46db5-1d3844bd_0] Write session will restart in 0.005987s
2025-10-09T21:50:30.453402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d56a67ad-34969441-8fd46db5-1d3844bd_0] Write session: Do CDS request
2025-10-09T21:50:30.453406Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d56a67ad-34969441-8fd46db5-1d3844bd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:30.460483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d56a67ad-34969441-8fd46db5-1d3844bd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:30.460570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:30.460581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1849
2025-10-09T21:50:30.460668Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1849 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:30.460705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1849 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:30.460713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1849 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:30.460716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:30.461134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1849 sessionId:  describe result for acl check
2025-10-09T21:50:30.461172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:30.461179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:30.461181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:30.461187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342716534792168:5632] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:30.461735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342716534792168:5632] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:30.464030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342716534792168:5632] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:30.464765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:30.466706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:30.466885Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:30.466895Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:30.466906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:30.466959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:30.466997Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342716172960349:2453]
2025-10-09T21:50:30.467006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342716534792168:5632] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:30.467017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342716534792168:5632] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:30.467020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342716534792168:5632] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:30.467027Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1849 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:30.467567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:30.467861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:30.467906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:30.468213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:30.468258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.468264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.468265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.468266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.468267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.468268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.468270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.468271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.468272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.468277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.468288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.468292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.468296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.468300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.468304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.468308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.468311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.468315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.468321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.468329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.468335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.468345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.468353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.468364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.468372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.468389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.468393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.468398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.468403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.468408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.468412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.468417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.468422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.468426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.468435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.468441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.468469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:30.474536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:30.475711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:30.475725Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:30.475727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:30.475732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:30.475737Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1848 [2:7559342716172960349:2453]
2025-10-09T21:50:30.475749Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:30.475765Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:30.475773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:30.475776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:30.475779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:30.475798Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:30.475808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:30.475815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:30.475870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.475878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.475880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.475882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:30.475931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:30.475936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.475944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.475950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.475959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:30.475980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342716534792104:2432], now have 1 active actors on pipe
2025-10-09T21:50:30.475992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342716172960344:2453]
2025-10-09T21:50:30.475999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342716172960344:2453]
2025-10-09T21:50:30.476001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342716172960344:2453]
2025-10-09T21:50:30.476013Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:30.476023Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:30.476025Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:30.476026Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:30.476028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342716534792279:5632], now have 1 active actors on pipe
2025-10-09T21:50:30.476037Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:30.476046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:30.476050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:30.476069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342716172960344:2453]
2025-10-09T21:50:30.476057Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1848
2025-10-09T21:50:30.476075Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:30.476077Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:30.476075Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1848
2025-10-09T21:50:30.476096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:30.476102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:30.476128Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5b87215c-5bfdad3c-bc80d635-ddc527bd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:30.476137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:30.476139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:30.476144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:30.476151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:30.476162Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:30.476175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:30.476301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:30.476309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:30.476321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:30.476396Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1849 partition: 0 MaxSeqNo: 25 sessionId: src_id|5b87215c-5bfdad3c-bc80d635-ddc527bd_0
2025-10-09T21:50:30.476701Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d56a67ad-34969441-8fd46db5-1d3844bd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5b87215c-5bfdad3c-bc80d635-ddc527bd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:30.478923Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b87215c-5bfdad3c-bc80d635-ddc527bd_0] Write session: try to update token
2025-10-09T21:50:30.478937Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b87215c-5bfdad3c-bc80d635-ddc527bd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:30.490476Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:30.490492Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:30.490494Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:30.490497Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:30.490505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342716172960344:2453]
2025-10-09T21:50:30.490508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342716172960344:2453]
2025-10-09T21:50:30.490511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342716172960344:2453]
2025-10-09T21:50:30.490556Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:30.490557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:30.490565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.490568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:30.490568Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:30.490572Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:30.490685Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:30.491196Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1849 sessionId: src_id|5b87215c-5bfdad3c-bc80d635-ddc527bd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:30.493529Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:30.553076Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342716534792291:6018]
2025-10-09T21:50:30.553297Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 1848, pipe: [1:7559342716534792291:6018]
2025-10-09T21:50:30.639395Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:30.639714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:30.639726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:30.639761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342716534792291:6018], now have 1 active actors on pipe
2025-10-09T21:50:30.639967Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:30.640024Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:30.640034Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342716534792291:6018]
2025-10-09T21:50:30.640040Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:30.640061Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:30.640094Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1849 reason: error at write: Disconnected sessionId: src_id|5b87215c-5bfdad3c-bc80d635-ddc527bd_0
2025-10-09T21:50:30.640274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1849 sessionId: src_id|5b87215c-5bfdad3c-bc80d635-ddc527bd_0 is DEAD
2025-10-09T21:50:30.640448Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b87215c-5bfdad3c-bc80d635-ddc527bd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:30.640461Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b87215c-5bfdad3c-bc80d635-ddc527bd_0] Write session will restart in 0.006358s
2025-10-09T21:50:30.640487Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b87215c-5bfdad3c-bc80d635-ddc527bd_0] Write session: Do CDS request
2025-10-09T21:50:30.640494Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b87215c-5bfdad3c-bc80d635-ddc527bd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:30.647476Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5b87215c-5bfdad3c-bc80d635-ddc527bd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:30.647529Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:30.647543Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1850
2025-10-09T21:50:30.647654Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1850 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:30.647678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1850 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:30.647688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1850 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:30.647695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:30.648071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1850 sessionId:  describe result for acl check
2025-10-09T21:50:30.648108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:30.648109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:30.648110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:30.648116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342716534792327:5622] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:30.648664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342716534792327:5622] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:30.650897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342716534792327:5622] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:30.653523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:30.653560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342716534792327:5622] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:30.653572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342716534792327:5622] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:30.653575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342716534792327:5622] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:30.653580Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1850 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:30.655490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:30.655663Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:30.655673Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:30.655685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:30.655741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:30.655779Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342716172960528:2448]
2025-10-09T21:50:30.656358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:30.656658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:30.656704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:30.657003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:30.657057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.657063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.657064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.657066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.657067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.657068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.657069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.657071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.657072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.657077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.657088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.657096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.657101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.657105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.657113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.657117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.657122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.657125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.657133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.657145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.657155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.657164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.657174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.657184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.657188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.657194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.657203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.657208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.657216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.657221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.657228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.657233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.657242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.657247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.657251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.657256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.657287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:30.663643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:30.664844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:30.664856Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:30.664857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:30.664860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:30.664867Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1849 [2:7559342716172960528:2448]
2025-10-09T21:50:30.664879Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:30.664894Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:30.664896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:30.664899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:30.664900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:30.664911Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:30.664915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:30.664920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:30.664958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.664965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.664966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.664968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:30.665017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:30.665022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.665030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.665040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.665051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:30.665070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342716534792468:5622], now have 1 active actors on pipe
2025-10-09T21:50:30.665075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342716534792293:2432], now have 1 active actors on pipe
2025-10-09T21:50:30.665081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342716172960522:2448]
2025-10-09T21:50:30.665086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342716172960522:2448]
2025-10-09T21:50:30.665087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342716172960522:2448]
2025-10-09T21:50:30.665097Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:30.665111Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:30.665116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:30.665118Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:30.665119Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:30.665130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:30.665149Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1849
2025-10-09T21:50:30.665154Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1849
2025-10-09T21:50:30.665132Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:30.665155Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342716172960522:2448]
2025-10-09T21:50:30.665159Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:30.665172Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:30.665204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:30.665222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:30.665255Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|adf145a7-e90d60eb-e6638a10-a7767fa8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:30.665268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:30.665273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:30.665275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:30.665285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:30.665296Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:30.665312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:30.665449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:30.665458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:30.665481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:30.665572Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1850 partition: 0 MaxSeqNo: 25 sessionId: src_id|adf145a7-e90d60eb-e6638a10-a7767fa8_0
2025-10-09T21:50:30.665834Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5b87215c-5bfdad3c-bc80d635-ddc527bd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|adf145a7-e90d60eb-e6638a10-a7767fa8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:30.668358Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|adf145a7-e90d60eb-e6638a10-a7767fa8_0] Write session: try to update token
2025-10-09T21:50:30.668371Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|adf145a7-e90d60eb-e6638a10-a7767fa8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:30.676145Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:30.676160Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:30.676170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342716172960522:2448]
2025-10-09T21:50:30.676217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:30.676218Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:30.676388Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:30.676522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:30.676653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:30.676780Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:30.676963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:30.677005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:30.680652Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1850 sessionId: src_id|adf145a7-e90d60eb-e6638a10-a7767fa8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:30.682986Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:30.683327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:30.683347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:30.683385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:30.683456Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:30.689109Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:30.689121Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:30.689123Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:30.689124Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:30.689132Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342716172960522:2448]
2025-10-09T21:50:30.689135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342716172960522:2448]
2025-10-09T21:50:30.689138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342716172960522:2448]
2025-10-09T21:50:30.689170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:30.689178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.689179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:30.689178Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:30.689184Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:30.689191Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:30.689299Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:30.739947Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342716534792481:6018]
2025-10-09T21:50:30.740179Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 1849, pipe: [1:7559342716534792481:6018]
2025-10-09T21:50:30.836882Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:30.843460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:30.843481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:30.843618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:30.843678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:30.843732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:30.843790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:30.843829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:30.843837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:30.843895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:30.843958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:30.844020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:30.844084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:30.844123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:30.844129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:30.844190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:30.844246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:30.844304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:30.844355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:30.844393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:30.844400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:30.844454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:30.844515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:30.844571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:30.844620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:30.844663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:30.844672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:30.844746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:30.844799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:30.844852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:30.844905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:30.844951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:30.844960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:30.845019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:30.845066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:30.845127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:30.845181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:30.845218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:30.845225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:30.845275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:30.845325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:30.845377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:30.845427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:30.845475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:30.845482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:30.845533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:30.845591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:30.845642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:30.845693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:30.845729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:30.845734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:30.845792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:30.845847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:30.845897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:30.845947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:30.845985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:30.845989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:30.846060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:30.846110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:30.846163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:30.846218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:30.846257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:30.846262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:30.846301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342716534792481:6018], now have 1 active actors on pipe
2025-10-09T21:50:30.846522Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:30.846529Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342716534792481:6018]
2025-10-09T21:50:30.846538Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:30.846550Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:30.846581Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1850 reason: error at write: Disconnected sessionId: src_id|adf145a7-e90d60eb-e6638a10-a7767fa8_0
2025-10-09T21:50:30.846752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1850 sessionId: src_id|adf145a7-e90d60eb-e6638a10-a7767fa8_0 is DEAD
2025-10-09T21:50:30.846907Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adf145a7-e90d60eb-e6638a10-a7767fa8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:30.846921Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adf145a7-e90d60eb-e6638a10-a7767fa8_0] Write session will restart in 0.006421s
2025-10-09T21:50:30.846945Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adf145a7-e90d60eb-e6638a10-a7767fa8_0] Write session: Do CDS request
2025-10-09T21:50:30.846951Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adf145a7-e90d60eb-e6638a10-a7767fa8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:30.847085Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:30.854577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|adf145a7-e90d60eb-e6638a10-a7767fa8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:30.854641Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:30.854656Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1851
2025-10-09T21:50:30.854765Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1851 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:30.854812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1851 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:30.854823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1851 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:30.854833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:30.855181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1851 sessionId:  describe result for acl check
2025-10-09T21:50:30.855210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:30.855211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:30.855212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:30.855216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342716534792533:5617] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:30.855658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342716534792533:5617] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:30.858461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342716534792533:5617] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:30.859249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:30.861107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:30.861320Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:30.861331Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:30.861342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:30.861392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:30.861434Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342716172960710:2446]
2025-10-09T21:50:30.861494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342716534792533:5617] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:30.861507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342716534792533:5617] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:30.861510Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342716534792533:5617] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:30.861518Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1851 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:30.862002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:30.862291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:30.862332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:30.862645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:30.862689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.862694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.862696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.862697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.862698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.862699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.862701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.862702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.862703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.862708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.862719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.862724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.862728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.862732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.862736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.862740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.862744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.862747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.862754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.862762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:30.862768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.862773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:30.862777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.862783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:30.862787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.862793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:30.862797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.862802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:30.862808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.862813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:30.862818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.862823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.862832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.862837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.862842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.862847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.862876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:30.869426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:30.870558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:30.870573Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:30.870576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:30.870580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:30.870586Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1850 [2:7559342716172960710:2446]
2025-10-09T21:50:30.870599Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:30.870619Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:30.870626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:30.870629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:30.870632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:30.870648Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:30.870659Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:30.870671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:30.870730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.870737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.870740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.870742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:30.870803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:30.870810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.870823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:30.870834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:30.870854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:30.870881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342716534792484:2432], now have 1 active actors on pipe
2025-10-09T21:50:30.870884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342716534792661:5617], now have 1 active actors on pipe
2025-10-09T21:50:30.870892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342716172960705:2446]
2025-10-09T21:50:30.870894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342716172960705:2446]
2025-10-09T21:50:30.870896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342716172960705:2446]
2025-10-09T21:50:30.870908Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:30.870922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:30.870922Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:30.870925Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:30.870928Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:30.870941Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:30.870944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:30.870907Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1850
2025-10-09T21:50:30.870929Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1850
2025-10-09T21:50:30.870985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:30.870995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:30.871007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342716172960705:2446]
2025-10-09T21:50:30.871018Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:30.871029Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:30.871049Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|535ad273-124c3e7b-9922fd1d-24736b31_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:30.871060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:30.871065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:30.871067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:30.871077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:30.871088Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:30.871113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:30.871238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:30.871248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:30.871262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:30.871318Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1851 partition: 0 MaxSeqNo: 25 sessionId: src_id|535ad273-124c3e7b-9922fd1d-24736b31_0
2025-10-09T21:50:30.871681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adf145a7-e90d60eb-e6638a10-a7767fa8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|535ad273-124c3e7b-9922fd1d-24736b31_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:30.874331Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|535ad273-124c3e7b-9922fd1d-24736b31_0] Write session: try to update token
2025-10-09T21:50:30.874350Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|535ad273-124c3e7b-9922fd1d-24736b31_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:30.885908Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:30.885923Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:30.885926Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:30.885928Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:30.885937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342716172960705:2446]
2025-10-09T21:50:30.885941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342716172960705:2446]
2025-10-09T21:50:30.885945Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342716172960705:2446]
2025-10-09T21:50:30.885985Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:30.885985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:30.885993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:30.885995Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:30.885996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:30.885999Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:30.886150Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:30.888613Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1851 sessionId: src_id|535ad273-124c3e7b-9922fd1d-24736b31_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:30.891246Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:30.946810Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342716534792674:6018]
2025-10-09T21:50:30.947050Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 1850, pipe: [1:7559342716534792674:6018]
2025-10-09T21:50:31.033537Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:31.034054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.034068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.034108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342716534792674:6018], now have 1 active actors on pipe
2025-10-09T21:50:31.034301Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:31.034321Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:31.034330Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:31.034370Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342716534792674:6018]
2025-10-09T21:50:31.034420Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1851 reason: error at write: Disconnected sessionId: src_id|535ad273-124c3e7b-9922fd1d-24736b31_0
2025-10-09T21:50:31.034473Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:31.034600Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1851 sessionId: src_id|535ad273-124c3e7b-9922fd1d-24736b31_0 is DEAD
2025-10-09T21:50:31.034767Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|535ad273-124c3e7b-9922fd1d-24736b31_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:31.034780Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|535ad273-124c3e7b-9922fd1d-24736b31_0] Write session will restart in 0.008843s
2025-10-09T21:50:31.034807Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|535ad273-124c3e7b-9922fd1d-24736b31_0] Write session: Do CDS request
2025-10-09T21:50:31.034814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|535ad273-124c3e7b-9922fd1d-24736b31_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:31.044572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|535ad273-124c3e7b-9922fd1d-24736b31_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:31.044639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:31.044658Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1852
2025-10-09T21:50:31.044827Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1852 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:31.044878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1852 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:31.044889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1852 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:31.044893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:31.045270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1852 sessionId:  describe result for acl check
2025-10-09T21:50:31.045304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:31.045309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:31.045310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:31.045315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342720829760113:5612] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:31.045808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342720829760113:5612] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:31.046433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:31.047926Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342720829760113:5612] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:31.048407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:31.048593Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:31.048605Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:31.048617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:31.048674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:31.048715Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342720467928187:2444]
2025-10-09T21:50:31.049255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:31.049570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:31.049612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:31.049921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:31.049973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.049981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.049982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.049984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.049985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.049986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.049987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.049988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.049990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.049995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.050005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.050009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.050013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.050016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.050021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.050025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.050046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.050050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.050056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.050064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.050070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.050076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.050081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.050087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.050091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.050096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.050103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.050108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.050113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.050118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.050123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.050131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.050135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.050141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.050150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.050155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.050183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:31.050235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342720829760113:5612] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:31.050251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342720829760113:5612] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:31.050255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342720829760113:5612] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:31.050264Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1852 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:31.056743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:31.057777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:31.057793Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:31.057795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:31.057799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:31.057806Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1851 [2:7559342720467928187:2444]
2025-10-09T21:50:31.057819Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:31.057838Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:31.057844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:31.057847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:31.057850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:31.057866Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:31.057879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:31.057886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:31.057951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.057959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.057962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.057965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:31.058062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:31.058071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.058084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.058097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.058115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:31.058141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342720829759973:2432], now have 1 active actors on pipe
2025-10-09T21:50:31.058149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342720829760153:5612], now have 1 active actors on pipe
2025-10-09T21:50:31.058157Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342720467928182:2444]
2025-10-09T21:50:31.058159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342720467928182:2444]
2025-10-09T21:50:31.058162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342720467928182:2444]
2025-10-09T21:50:31.058178Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:31.058186Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:31.058190Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:31.058192Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:31.058162Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1851
2025-10-09T21:50:31.058195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:31.058210Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:31.058214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:31.058174Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1851
2025-10-09T21:50:31.058245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342720467928182:2444]
2025-10-09T21:50:31.058253Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:31.058255Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:31.058283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.058292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.058326Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b55719b3-d8602cf-c1b0ef82-87aea799_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:31.058340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:31.058344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:31.058351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:31.058363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:31.058373Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:31.058396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:31.058551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.058562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.058578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:31.058693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1852 partition: 0 MaxSeqNo: 25 sessionId: src_id|b55719b3-d8602cf-c1b0ef82-87aea799_0
2025-10-09T21:50:31.059025Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|535ad273-124c3e7b-9922fd1d-24736b31_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b55719b3-d8602cf-c1b0ef82-87aea799_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:31.061520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b55719b3-d8602cf-c1b0ef82-87aea799_0] Write session: try to update token
2025-10-09T21:50:31.061538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b55719b3-d8602cf-c1b0ef82-87aea799_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:31.074581Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:31.074609Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:31.074612Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:31.074616Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:31.074629Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342720467928182:2444]
2025-10-09T21:50:31.074636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342720467928182:2444]
2025-10-09T21:50:31.074643Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342720467928182:2444]
2025-10-09T21:50:31.074700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:31.074709Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:31.074713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.074715Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:31.074716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:31.074719Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:31.074925Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:31.075076Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1852 sessionId: src_id|b55719b3-d8602cf-c1b0ef82-87aea799_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:31.077389Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:31.134714Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342720829760164:6018]
2025-10-09T21:50:31.134995Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 1851, pipe: [1:7559342720829760164:6018]
2025-10-09T21:50:31.220107Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:31.220581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.220596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.220642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342720829760164:6018], now have 1 active actors on pipe
2025-10-09T21:50:31.220821Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:31.220835Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342720829760164:6018]
2025-10-09T21:50:31.220837Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:31.220840Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:31.220865Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:31.220896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1852 reason: error at write: Disconnected sessionId: src_id|b55719b3-d8602cf-c1b0ef82-87aea799_0
2025-10-09T21:50:31.221059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1852 sessionId: src_id|b55719b3-d8602cf-c1b0ef82-87aea799_0 is DEAD
2025-10-09T21:50:31.221226Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b55719b3-d8602cf-c1b0ef82-87aea799_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:31.221239Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b55719b3-d8602cf-c1b0ef82-87aea799_0] Write session will restart in 0.005302s
2025-10-09T21:50:31.221266Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b55719b3-d8602cf-c1b0ef82-87aea799_0] Write session: Do CDS request
2025-10-09T21:50:31.221273Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b55719b3-d8602cf-c1b0ef82-87aea799_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:31.227524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b55719b3-d8602cf-c1b0ef82-87aea799_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:31.227606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:31.227621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1853
2025-10-09T21:50:31.227749Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1853 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:31.227780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1853 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:31.227791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1853 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:31.227795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:31.228144Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1853 sessionId:  describe result for acl check
2025-10-09T21:50:31.228178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:31.228184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:31.228186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:31.228191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342720829760193:5600] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:31.228642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342720829760193:5600] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:31.231182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342720829760193:5600] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:31.233198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:31.234973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342720829760193:5600] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:31.234976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:31.234987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342720829760193:5600] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:31.235006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342720829760193:5600] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:31.235014Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1853 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:31.235154Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:31.235164Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:31.235176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:31.235231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:31.235267Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342720467928368:2442]
2025-10-09T21:50:31.235849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:31.236138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:31.236181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:31.236490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:31.236535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.236541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.236542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.236543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.236545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.236546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.236547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.236548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.236549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.236554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.236565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.236569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.236572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.236576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.236580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.236584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.236588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.236592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.236598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.236606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.236612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.236617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.236622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.236628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.236635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.236641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.236645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.236650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.236659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.236663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.236668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.236674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.236678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.236682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.236689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.236693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.236722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:31.242840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:31.243883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:31.243898Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:31.243900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:31.243905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:31.243911Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1852 [2:7559342720467928368:2442]
2025-10-09T21:50:31.243927Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:31.243945Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:31.243948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:31.243951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:31.243954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:31.243968Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:31.243974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:31.243981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:31.244038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.244046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.244048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.244051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:31.244118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:31.244126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.244138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.244151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.244172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:31.244196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342720467928363:2442]
2025-10-09T21:50:31.244202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342720467928363:2442]
2025-10-09T21:50:31.244205Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342720467928363:2442]
2025-10-09T21:50:31.244218Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:31.244234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342720829760170:2432], now have 1 active actors on pipe
2025-10-09T21:50:31.244235Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:31.244238Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:31.244239Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:31.244242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342720829760349:5600], now have 1 active actors on pipe
2025-10-09T21:50:31.244252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:31.244266Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:31.244272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:31.244294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342720467928363:2442]
2025-10-09T21:50:31.244301Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:31.244268Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1852
2025-10-09T21:50:31.244314Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:31.244270Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1852
2025-10-09T21:50:31.244528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.244541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.244572Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7b21bebb-39ea4b3c-e2cf71e5-b8884c33_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:31.244584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:31.244588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:31.244590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:31.244600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:31.244615Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:31.244633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:31.244811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.244820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.244835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:31.244921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1853 partition: 0 MaxSeqNo: 25 sessionId: src_id|7b21bebb-39ea4b3c-e2cf71e5-b8884c33_0
2025-10-09T21:50:31.245244Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b55719b3-d8602cf-c1b0ef82-87aea799_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7b21bebb-39ea4b3c-e2cf71e5-b8884c33_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:31.247555Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b21bebb-39ea4b3c-e2cf71e5-b8884c33_0] Write session: try to update token
2025-10-09T21:50:31.247572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b21bebb-39ea4b3c-e2cf71e5-b8884c33_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:31.258488Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:31.258506Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:31.258508Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:31.258511Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:31.258521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342720467928363:2442]
2025-10-09T21:50:31.258526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342720467928363:2442]
2025-10-09T21:50:31.258532Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342720467928363:2442]
2025-10-09T21:50:31.258575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:31.258582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.258586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:31.258597Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:31.258611Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:31.258615Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:31.258714Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:31.259839Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1853 sessionId: src_id|7b21bebb-39ea4b3c-e2cf71e5-b8884c33_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:31.262146Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:31.321089Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342720829760365:6018]
2025-10-09T21:50:31.321370Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 1852, pipe: [1:7559342720829760365:6018]
2025-10-09T21:50:31.405193Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:31.405675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.405691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.405727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342720829760365:6018], now have 1 active actors on pipe
2025-10-09T21:50:31.405936Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:31.405976Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:31.406056Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:31.406125Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1853 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7b21bebb-39ea4b3c-e2cf71e5-b8884c33_0
2025-10-09T21:50:31.406309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1853 sessionId: src_id|7b21bebb-39ea4b3c-e2cf71e5-b8884c33_0 is DEAD
2025-10-09T21:50:31.406458Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b21bebb-39ea4b3c-e2cf71e5-b8884c33_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:31.406468Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b21bebb-39ea4b3c-e2cf71e5-b8884c33_0] Write session will restart in 0.009507s
2025-10-09T21:50:31.406486Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b21bebb-39ea4b3c-e2cf71e5-b8884c33_0] Write session: Do CDS request
2025-10-09T21:50:31.406466Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342720829760365:6018]
2025-10-09T21:50:31.406499Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b21bebb-39ea4b3c-e2cf71e5-b8884c33_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:31.417554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b21bebb-39ea4b3c-e2cf71e5-b8884c33_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:31.417642Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:31.417659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1854
2025-10-09T21:50:31.417758Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1854 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:31.417802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1854 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:31.417813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1854 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:31.417817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:31.418212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1854 sessionId:  describe result for acl check
2025-10-09T21:50:31.418244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:31.418250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:31.418252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:31.418257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342720829760537:5578] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:31.418695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342720829760537:5578] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:31.418730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:31.420280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342720829760537:5578] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:31.420781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:31.420968Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:31.420979Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:31.420992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:31.421050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:31.421079Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342720467928588:2424]
2025-10-09T21:50:31.421493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:31.421714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:31.421744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:31.421994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:31.422069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.422075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.422076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.422077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.422078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.422080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.422081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.422082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.422083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.422088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.422096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.422099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.422101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.422103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.422105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.422107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.422109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.422111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.422116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.422121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.422126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.422129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.422131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.422134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.422137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.422140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.422142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.422145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.422147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.422150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.422153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.422156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.422158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.422161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.422164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.422167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.422184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:31.422626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342720829760537:5578] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:31.422640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342720829760537:5578] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:31.422646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342720829760537:5578] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:31.422653Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1854 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:31.428284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:31.429263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:31.429276Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:31.429278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:31.429281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:31.429285Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1853 [2:7559342720467928588:2424]
2025-10-09T21:50:31.429295Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:31.429310Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:31.429312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:31.429314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:31.429316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:31.429329Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:31.429333Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:31.429339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:31.429379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.429385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.429387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.429389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:31.429442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:31.429449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.429456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.429477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.429487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:31.429502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342720829760574:5578], now have 1 active actors on pipe
2025-10-09T21:50:31.429508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342720829760381:2432], now have 1 active actors on pipe
2025-10-09T21:50:31.429514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342720467928583:2424]
2025-10-09T21:50:31.429516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342720467928583:2424]
2025-10-09T21:50:31.429517Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342720467928583:2424]
2025-10-09T21:50:31.429526Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:31.429538Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:31.429539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:31.429545Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:31.429547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:31.429547Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:31.429549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:31.429572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342720467928583:2424]
2025-10-09T21:50:31.429578Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:31.429581Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:31.429602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.429605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.429579Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1853
2025-10-09T21:50:31.429628Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c46e965a-fa1ad230-7d26c681-2053a10a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:31.429638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:31.429641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:31.429643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:31.429650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:31.429658Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:31.429669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:31.429796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.429802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.429812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:31.429883Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1854 partition: 0 MaxSeqNo: 25 sessionId: src_id|c46e965a-fa1ad230-7d26c681-2053a10a_0
2025-10-09T21:50:31.429953Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1853
2025-10-09T21:50:31.430168Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b21bebb-39ea4b3c-e2cf71e5-b8884c33_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c46e965a-fa1ad230-7d26c681-2053a10a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:31.432756Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c46e965a-fa1ad230-7d26c681-2053a10a_0] Write session: try to update token
2025-10-09T21:50:31.432773Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c46e965a-fa1ad230-7d26c681-2053a10a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:31.440534Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:31.440548Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:31.440558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342720467928583:2424]
2025-10-09T21:50:31.440604Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:31.440606Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:31.441545Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:31.441681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:31.442198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:31.442334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:31.442517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:31.442560Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:31.446862Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1854 sessionId: src_id|c46e965a-fa1ad230-7d26c681-2053a10a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:31.449238Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:31.449447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.449473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.449511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:31.449578Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:31.453096Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:31.453110Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:31.453113Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:31.453115Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:31.453125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342720467928583:2424]
2025-10-09T21:50:31.453130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342720467928583:2424]
2025-10-09T21:50:31.453134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342720467928583:2424]
2025-10-09T21:50:31.453173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:31.453187Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:31.453193Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:31.453195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.453197Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:31.453197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:31.453322Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:31.506416Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342720829760581:6018]
2025-10-09T21:50:31.506678Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 1853, pipe: [1:7559342720829760581:6018]
2025-10-09T21:50:31.600980Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:31.607779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.607805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.607949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.608004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.608054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.608111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.608147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.608155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.608216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.608269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.608321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.608377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.608416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.608423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.608476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.608529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.608580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.608633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.608671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.608676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.608731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.608782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.608834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.608904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.608944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.608950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.609004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.609054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.609106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.609159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.609201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.609207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.609261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.609311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.609366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.609419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.609469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.609475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.609527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.609576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.609630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.609684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.609810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.609816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.609867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.609915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.609964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.610103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.610148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.610154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.610254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.610303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.610354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.610411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.610450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.610456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.610509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.610558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.610610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.610664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.610717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.610723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.610765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342720829760581:6018], now have 1 active actors on pipe
2025-10-09T21:50:31.610970Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342720829760581:6018]
2025-10-09T21:50:31.610983Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:31.610988Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:31.610998Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:31.611021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1854 reason: error at write: Disconnected sessionId: src_id|c46e965a-fa1ad230-7d26c681-2053a10a_0
2025-10-09T21:50:31.611268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1854 sessionId: src_id|c46e965a-fa1ad230-7d26c681-2053a10a_0 is DEAD
2025-10-09T21:50:31.611325Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c46e965a-fa1ad230-7d26c681-2053a10a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:31.611334Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c46e965a-fa1ad230-7d26c681-2053a10a_0] Write session will restart in 0.009716s
2025-10-09T21:50:31.611353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c46e965a-fa1ad230-7d26c681-2053a10a_0] Write session: Do CDS request
2025-10-09T21:50:31.611358Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c46e965a-fa1ad230-7d26c681-2053a10a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:31.611592Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:31.622378Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c46e965a-fa1ad230-7d26c681-2053a10a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:31.622476Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:31.622495Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1855
2025-10-09T21:50:31.622613Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1855 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:31.622645Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1855 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:31.622656Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1855 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:31.622660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:31.623028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:31.623075Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1855 sessionId:  describe result for acl check
2025-10-09T21:50:31.623113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:31.623115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:31.623116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:31.623122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342720829760730:5583] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:31.623544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342720829760730:5583] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:31.625038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:31.625201Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:31.625210Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:31.625219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:31.625277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:31.625324Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342720467928772:2418]
2025-10-09T21:50:31.625886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:31.626173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:31.626212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:31.626384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342720829760730:5583] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:31.626511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:31.626577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.626583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.626584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.626586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.626587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.626588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.626589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.626590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.626592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.626598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.626608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.626611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.626615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.626619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.626623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.626627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.626630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.626634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.626641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.626655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.626665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.626676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.626681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.626686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.626691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.626696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.626706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.626713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.626722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.626727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.626737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.626742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.626745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.626750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.626753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.626756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.626776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:31.632572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342720829760730:5583] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:31.632589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342720829760730:5583] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:31.632592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342720829760730:5583] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:31.632601Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1855 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:31.632921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:31.634130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:31.634140Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:31.634141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:31.634144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:31.634150Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1854 [2:7559342720467928772:2418]
2025-10-09T21:50:31.634160Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:31.634173Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:31.634175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:31.634177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:31.634179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:31.634191Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:31.634199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:31.634204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:31.634248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.634253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.634254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.634256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:31.634305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:31.634310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.634317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.634327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.634341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:31.634364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342720829760768:5583], now have 1 active actors on pipe
2025-10-09T21:50:31.634372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342720829760584:2432], now have 1 active actors on pipe
2025-10-09T21:50:31.634379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342720467928767:2418]
2025-10-09T21:50:31.634385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342720467928767:2418]
2025-10-09T21:50:31.634387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342720467928767:2418]
2025-10-09T21:50:31.634400Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:31.634418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:31.634421Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:31.634424Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:31.634426Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:31.634434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:31.634442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:31.634450Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1854
2025-10-09T21:50:31.634469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342720467928767:2418]
2025-10-09T21:50:31.634478Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:31.634458Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1854
2025-10-09T21:50:31.634480Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:31.634503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.634512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.634548Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f40b8a72-5f04d8f5-bcd01717-f3c9aa5b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:31.634561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:31.634565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:31.634567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:31.634577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:31.634587Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:31.634609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:31.634731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.634739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.634754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:31.634823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1855 partition: 0 MaxSeqNo: 25 sessionId: src_id|f40b8a72-5f04d8f5-bcd01717-f3c9aa5b_0
2025-10-09T21:50:31.635113Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c46e965a-fa1ad230-7d26c681-2053a10a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f40b8a72-5f04d8f5-bcd01717-f3c9aa5b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:31.637428Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f40b8a72-5f04d8f5-bcd01717-f3c9aa5b_0] Write session: try to update token
2025-10-09T21:50:31.637443Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f40b8a72-5f04d8f5-bcd01717-f3c9aa5b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:31.648258Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:31.648275Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:31.648277Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:31.648278Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:31.648287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342720467928767:2418]
2025-10-09T21:50:31.648291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342720467928767:2418]
2025-10-09T21:50:31.648295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342720467928767:2418]
2025-10-09T21:50:31.648339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:31.648345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.648348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:31.648349Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:31.648355Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:31.648370Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:31.648467Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:31.654062Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1855 sessionId: src_id|f40b8a72-5f04d8f5-bcd01717-f3c9aa5b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:31.656337Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:31.711225Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342720829760777:6018]
2025-10-09T21:50:31.711536Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 1854, pipe: [1:7559342720829760777:6018]
2025-10-09T21:50:31.788129Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:31.788351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.788364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.788400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342720829760777:6018], now have 1 active actors on pipe
2025-10-09T21:50:31.788623Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:31.788637Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:31.788655Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342720829760777:6018]
2025-10-09T21:50:31.788676Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:31.788691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1855 reason: error at write: Disconnected sessionId: src_id|f40b8a72-5f04d8f5-bcd01717-f3c9aa5b_0
2025-10-09T21:50:31.788844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1855 sessionId: src_id|f40b8a72-5f04d8f5-bcd01717-f3c9aa5b_0 is DEAD
2025-10-09T21:50:31.788942Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:31.788972Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f40b8a72-5f04d8f5-bcd01717-f3c9aa5b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:31.788981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f40b8a72-5f04d8f5-bcd01717-f3c9aa5b_0] Write session will restart in 0.006892s
2025-10-09T21:50:31.789002Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f40b8a72-5f04d8f5-bcd01717-f3c9aa5b_0] Write session: Do CDS request
2025-10-09T21:50:31.789007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f40b8a72-5f04d8f5-bcd01717-f3c9aa5b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:31.796480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f40b8a72-5f04d8f5-bcd01717-f3c9aa5b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:31.796542Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:31.796559Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1856
2025-10-09T21:50:31.796698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1856 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:31.796733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1856 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:31.796741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1856 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:31.796744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:31.797195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1856 sessionId:  describe result for acl check
2025-10-09T21:50:31.797230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:31.797239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:31.797241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:31.797246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342720829760864:5572] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:31.797808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342720829760864:5572] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:31.799672Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342720829760864:5572] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:31.800292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:31.802268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:31.802359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342720829760864:5572] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:31.802372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342720829760864:5572] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:31.802375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342720829760864:5572] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:31.802381Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1856 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:31.802438Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:31.802448Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:31.802459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:31.802508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:31.802550Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342720467928960:2410]
2025-10-09T21:50:31.802922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:31.803168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:31.803198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:31.803408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:31.803439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.803443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.803444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.803445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.803446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.803447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.803447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.803448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.803448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.803452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.803459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.803461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.803463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.803465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.803467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.803469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.803471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.803473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.803477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.803482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:31.803486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.803489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:31.803492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.803495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:31.803497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.803500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:31.803503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.803505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:31.803508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.803511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:31.803515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.803519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.803521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.803524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.803527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.803530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.803545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:31.810207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:31.811202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:31.811214Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:31.811217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:31.811222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:31.811231Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1855 [2:7559342720467928960:2410]
2025-10-09T21:50:31.811245Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:31.811263Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:31.811269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:31.811272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:31.811273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:31.811287Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:31.811294Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:31.811300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:31.811347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.811352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.811353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.811356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:31.811413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:31.811418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.811426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:31.811436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:31.811450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:31.811467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342720467928954:2410]
2025-10-09T21:50:31.811474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342720467928954:2410]
2025-10-09T21:50:31.811476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342720467928954:2410]
2025-10-09T21:50:31.811485Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:31.811496Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:31.811505Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:31.811505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:31.811508Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:31.811514Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:31.811516Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:31.811525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342720829760781:2432], now have 1 active actors on pipe
2025-10-09T21:50:31.811528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342720829760964:5572], now have 1 active actors on pipe
2025-10-09T21:50:31.811552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342720467928954:2410]
2025-10-09T21:50:31.811561Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:31.811568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:31.811572Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1855
2025-10-09T21:50:31.811578Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1855
2025-10-09T21:50:31.811616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.811620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.811642Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5c9fecfb-719602bb-52a352a9-4fda7238_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:31.811648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:31.811651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:31.811652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:31.811661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:31.811670Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:31.811681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:31.811860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.811871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.811893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:31.811995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1856 partition: 0 MaxSeqNo: 25 sessionId: src_id|5c9fecfb-719602bb-52a352a9-4fda7238_0
2025-10-09T21:50:31.812338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f40b8a72-5f04d8f5-bcd01717-f3c9aa5b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5c9fecfb-719602bb-52a352a9-4fda7238_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:31.814618Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9fecfb-719602bb-52a352a9-4fda7238_0] Write session: try to update token
2025-10-09T21:50:31.814634Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9fecfb-719602bb-52a352a9-4fda7238_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:31.824329Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:31.824347Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:31.824354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342720467928954:2410]
2025-10-09T21:50:31.824389Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:31.824395Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:31.824562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:31.824702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:31.824829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:31.824951Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:31.825137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:31.825179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:31.826775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1856 sessionId: src_id|5c9fecfb-719602bb-52a352a9-4fda7238_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:31.829156Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:31.829362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.829378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.829419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:31.829518Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:31.838377Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:31.838394Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:31.838396Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:31.838398Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:31.838409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342720467928954:2410]
2025-10-09T21:50:31.838413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342720467928954:2410]
2025-10-09T21:50:31.838417Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342720467928954:2410]
2025-10-09T21:50:31.838460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:31.838467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:31.838469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:31.838469Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:31.838476Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:31.838481Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:31.838604Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:31.888911Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342720829760979:6018]
2025-10-09T21:50:31.889192Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 1855, pipe: [1:7559342720829760979:6018]
2025-10-09T21:50:31.980783Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:31.987449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:31.987472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:31.987612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.987670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.987723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.987772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.987814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.987822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.987878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.987936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.987993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.988049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.988091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.988097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.988156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.988209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.988261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.988314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.988352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.988357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.988453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.988563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.988685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.988739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.988835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.988842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.988894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.988952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.989007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.989063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.989102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.989107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.989165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.989219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.989269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.989319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.989357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.989363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.989423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.989490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.989544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.989594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.989633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.989638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.989695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.989746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.989796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.989845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.989955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.989963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.990016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.990077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.990128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.990182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.990227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.990232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.990283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:31.990332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:31.990385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:31.990435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:31.990551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:31.990559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:31.990600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342720829760979:6018], now have 1 active actors on pipe
2025-10-09T21:50:31.990894Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:31.990895Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342720829760979:6018]
2025-10-09T21:50:31.990928Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:31.990931Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:31.990963Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1856 reason: error at write: Disconnected sessionId: src_id|5c9fecfb-719602bb-52a352a9-4fda7238_0
2025-10-09T21:50:31.991156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1856 sessionId: src_id|5c9fecfb-719602bb-52a352a9-4fda7238_0 is DEAD
2025-10-09T21:50:31.991308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9fecfb-719602bb-52a352a9-4fda7238_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:31.991318Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9fecfb-719602bb-52a352a9-4fda7238_0] Write session will restart in 0.006618s
2025-10-09T21:50:31.991345Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9fecfb-719602bb-52a352a9-4fda7238_0] Write session: Do CDS request
2025-10-09T21:50:31.991350Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9fecfb-719602bb-52a352a9-4fda7238_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:31.991357Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:31.999539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9fecfb-719602bb-52a352a9-4fda7238_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:31.999638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:31.999657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1857
2025-10-09T21:50:31.999788Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1857 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:31.999825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1857 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:31.999832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1857 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:31.999843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:32.000249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1857 sessionId:  describe result for acl check
2025-10-09T21:50:32.000286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:32.000288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:32.000292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:32.000298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342720829761028:5567] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:32.000875Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342720829761028:5567] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:32.003948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342720829761028:5567] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:32.005424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:32.007608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:32.007856Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:32.007868Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:32.007883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:32.007951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:32.007996Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342724762896441:2394]
2025-10-09T21:50:32.008093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342720829761028:5567] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:32.008113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342720829761028:5567] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:32.008116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342720829761028:5567] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:32.008125Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1857 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:32.008606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:32.008894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:32.008937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:32.009258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:32.009309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.009315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.009316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.009318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.009319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.009320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.009322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.009323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.009324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.009330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.009341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.009345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.009350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.009354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.009358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.009362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.009366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.009370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.009376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.009386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.009397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.009403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.009412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.009417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.009421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.009427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.009436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.009441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.009446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.009467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.009477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.009483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.009486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.009491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.009501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.009507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.009541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:32.016120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:32.017286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:32.017302Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:32.017304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:32.017308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:32.017315Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1856 [2:7559342724762896441:2394]
2025-10-09T21:50:32.017325Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:32.017342Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:32.017344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:32.017346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:32.017348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:32.017362Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:32.017370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:32.017375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:32.017426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.017432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.017433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.017436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:32.017510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:32.017519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.017531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.017541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.017563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:32.017584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342720829760983:2432], now have 1 active actors on pipe
2025-10-09T21:50:32.017590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342725124728459:5567], now have 1 active actors on pipe
2025-10-09T21:50:32.017596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342724762896436:2394]
2025-10-09T21:50:32.017598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342724762896436:2394]
2025-10-09T21:50:32.017599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342724762896436:2394]
2025-10-09T21:50:32.017609Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:32.017621Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:32.017621Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:32.017624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:32.017626Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:32.017634Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:32.017636Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:32.017670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342724762896436:2394]
2025-10-09T21:50:32.017675Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:32.017677Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:32.017661Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1856
2025-10-09T21:50:32.017706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.017675Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1856
2025-10-09T21:50:32.017714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.017738Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|892acc13-e42cad2c-6524fb24-ca831240_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:32.017746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:32.017749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:32.017751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:32.017759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:32.017768Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:32.017782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:32.017911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.017921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.017936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:32.018039Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1857 partition: 0 MaxSeqNo: 25 sessionId: src_id|892acc13-e42cad2c-6524fb24-ca831240_0
2025-10-09T21:50:32.018392Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9fecfb-719602bb-52a352a9-4fda7238_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|892acc13-e42cad2c-6524fb24-ca831240_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:32.020719Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|892acc13-e42cad2c-6524fb24-ca831240_0] Write session: try to update token
2025-10-09T21:50:32.020736Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|892acc13-e42cad2c-6524fb24-ca831240_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:32.032223Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:32.032242Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:32.032245Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:32.032247Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:32.032256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342724762896436:2394]
2025-10-09T21:50:32.032261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342724762896436:2394]
2025-10-09T21:50:32.032265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342724762896436:2394]
2025-10-09T21:50:32.032311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:32.032318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.032321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:32.032323Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:32.032330Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:32.032334Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:32.032453Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:32.033468Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1857 sessionId: src_id|892acc13-e42cad2c-6524fb24-ca831240_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:32.035866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:32.091113Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342725124728471:6018]
2025-10-09T21:50:32.091391Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 1856, pipe: [1:7559342725124728471:6018]
2025-10-09T21:50:32.179987Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:32.180254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.180271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.180315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342725124728471:6018], now have 1 active actors on pipe
2025-10-09T21:50:32.180554Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:32.180563Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:32.180572Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:32.180614Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342725124728471:6018]
2025-10-09T21:50:32.180650Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1857 reason: error at write: Disconnected sessionId: src_id|892acc13-e42cad2c-6524fb24-ca831240_0
2025-10-09T21:50:32.180846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1857 sessionId: src_id|892acc13-e42cad2c-6524fb24-ca831240_0 is DEAD
2025-10-09T21:50:32.180986Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|892acc13-e42cad2c-6524fb24-ca831240_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:32.180998Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|892acc13-e42cad2c-6524fb24-ca831240_0] Write session will restart in 0.009002s
2025-10-09T21:50:32.181022Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|892acc13-e42cad2c-6524fb24-ca831240_0] Write session: Do CDS request
2025-10-09T21:50:32.181031Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|892acc13-e42cad2c-6524fb24-ca831240_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:32.181327Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:32.191533Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|892acc13-e42cad2c-6524fb24-ca831240_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:32.191564Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:32.191582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1858
2025-10-09T21:50:32.191750Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1858 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:32.191785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1858 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:32.191794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1858 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:32.191797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:32.192132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1858 sessionId:  describe result for acl check
2025-10-09T21:50:32.192161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:32.192166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:32.192167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:32.192172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342725124728605:5562] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:32.192617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342725124728605:5562] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:32.194070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:32.194955Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342725124728605:5562] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:32.196057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:32.196237Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:32.196246Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:32.196258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:32.196309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:32.196348Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342724762896627:2398]
2025-10-09T21:50:32.196767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:32.197005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:32.197052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:32.197164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342725124728605:5562] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:32.197178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342725124728605:5562] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:32.197182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342725124728605:5562] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:32.197190Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1858 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:32.197360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:32.197436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.197443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.197445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.197446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.197448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.197449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.197450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.197452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.197453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.197470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.197480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.197485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.197489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.197493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.197497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.197501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.197505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.197510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.197518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.197526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.197532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.197538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.197543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.197549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.197553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.197560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.197564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.197570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.197576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.197589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.197594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.197602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.197607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.197615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.197620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.197626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.197660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:32.204167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:32.205243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:32.205263Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:32.205267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:32.205271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:32.205280Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1857 [2:7559342724762896627:2398]
2025-10-09T21:50:32.205300Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:32.205320Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:32.205328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:32.205332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:32.205334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:32.205355Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:32.205366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:32.205374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:32.205442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.205449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.205451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.205453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:32.205556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:32.205565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.205578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.205591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.205613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:32.205642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342725124728665:5562], now have 1 active actors on pipe
2025-10-09T21:50:32.205651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342725124728476:2432], now have 1 active actors on pipe
2025-10-09T21:50:32.205660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342724762896622:2398]
2025-10-09T21:50:32.205668Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342724762896622:2398]
2025-10-09T21:50:32.205670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342724762896622:2398]
2025-10-09T21:50:32.205685Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:32.205706Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:32.205707Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:32.205711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:32.205713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:32.205726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:32.205730Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:32.205747Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1857
2025-10-09T21:50:32.205774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342724762896622:2398]
2025-10-09T21:50:32.205748Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1857
2025-10-09T21:50:32.205782Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:32.205784Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:32.205801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.205806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.205867Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5168d9b5-d499646f-94ab6a9c-558888ad_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:32.205881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:32.205885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:32.205888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:32.205901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:32.205917Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:32.205949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:32.206131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.206144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.206164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:32.206260Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1858 partition: 0 MaxSeqNo: 25 sessionId: src_id|5168d9b5-d499646f-94ab6a9c-558888ad_0
2025-10-09T21:50:32.206645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|892acc13-e42cad2c-6524fb24-ca831240_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5168d9b5-d499646f-94ab6a9c-558888ad_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:32.209072Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5168d9b5-d499646f-94ab6a9c-558888ad_0] Write session: try to update token
2025-10-09T21:50:32.209093Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5168d9b5-d499646f-94ab6a9c-558888ad_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:32.220401Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:32.220420Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:32.220422Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:32.220424Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:32.220433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342724762896622:2398]
2025-10-09T21:50:32.220437Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342724762896622:2398]
2025-10-09T21:50:32.220441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342724762896622:2398]
2025-10-09T21:50:32.220478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:32.220485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.220487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:32.220495Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:32.220502Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:32.220506Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:32.220625Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:32.222332Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1858 sessionId: src_id|5168d9b5-d499646f-94ab6a9c-558888ad_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:32.224906Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:32.280521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342725124728679:6018]
2025-10-09T21:50:32.280772Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 1857, pipe: [1:7559342725124728679:6018]
2025-10-09T21:50:32.367826Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:32.368084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.368097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.368136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342725124728679:6018], now have 1 active actors on pipe
2025-10-09T21:50:32.368309Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:32.368384Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:32.368399Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:32.368418Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342725124728679:6018]
2025-10-09T21:50:32.368443Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:32.368462Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1858 reason: error at write: Disconnected sessionId: src_id|5168d9b5-d499646f-94ab6a9c-558888ad_0
2025-10-09T21:50:32.368607Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1858 sessionId: src_id|5168d9b5-d499646f-94ab6a9c-558888ad_0 is DEAD
2025-10-09T21:50:32.368750Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5168d9b5-d499646f-94ab6a9c-558888ad_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:32.368762Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5168d9b5-d499646f-94ab6a9c-558888ad_0] Write session will restart in 0.007296s
2025-10-09T21:50:32.368796Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5168d9b5-d499646f-94ab6a9c-558888ad_0] Write session: Do CDS request
2025-10-09T21:50:32.368802Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5168d9b5-d499646f-94ab6a9c-558888ad_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:32.377550Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5168d9b5-d499646f-94ab6a9c-558888ad_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:32.377604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:32.377620Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1859
2025-10-09T21:50:32.377756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1859 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:32.377793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1859 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:32.377799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1859 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:32.377803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:32.378196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1859 sessionId:  describe result for acl check
2025-10-09T21:50:32.378237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:32.378244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:32.378245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:32.378250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342725124728783:5557] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:32.378778Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342725124728783:5557] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:32.380636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:32.380707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342725124728783:5557] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:32.382591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:32.382757Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:32.382768Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:32.382779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:32.382824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:32.382861Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342724762896815:2386]
2025-10-09T21:50:32.382962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342725124728783:5557] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:32.382974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342725124728783:5557] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:32.382976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342725124728783:5557] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:32.382987Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1859 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:32.383423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:32.383693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:32.383736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:32.384034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:32.384076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.384082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.384083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.384084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.384085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.384086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.384087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.384088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.384089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.384094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.384103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.384106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.384110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.384113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.384116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.384120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.384123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.384126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.384133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.384140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.384146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.384151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.384155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.384161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.384165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.384170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.384179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.384183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.384189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.384194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.384201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.384206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.384211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.384215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.384222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.384227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.384255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:32.390295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:32.391319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:32.391333Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:32.391335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:32.391339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:32.391345Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1858 [2:7559342724762896815:2386]
2025-10-09T21:50:32.391358Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:32.391375Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:32.391384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:32.391388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:32.391391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:32.391406Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:32.391412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:32.391419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:32.391471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.391478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.391481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.391483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:32.391623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:32.391631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.391644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.391661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.391683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:32.391709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342725124728867:5557], now have 1 active actors on pipe
2025-10-09T21:50:32.391717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342725124728682:2432], now have 1 active actors on pipe
2025-10-09T21:50:32.391724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342724762896810:2386]
2025-10-09T21:50:32.391726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342724762896810:2386]
2025-10-09T21:50:32.391728Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342724762896810:2386]
2025-10-09T21:50:32.391710Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1858
2025-10-09T21:50:32.391737Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:32.391753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:32.391755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:32.391762Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:32.391765Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:32.391769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:32.391772Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:32.391778Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1858
2025-10-09T21:50:32.391803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342724762896810:2386]
2025-10-09T21:50:32.391814Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:32.391830Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:32.391832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.391836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.391880Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e46c7102-8f078397-cc38df75-27209c66_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:32.391892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:32.391899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:32.391902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:32.391912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:32.391925Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:32.391939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:32.392090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.392097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.392109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:32.392197Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1859 partition: 0 MaxSeqNo: 25 sessionId: src_id|e46c7102-8f078397-cc38df75-27209c66_0
2025-10-09T21:50:32.392532Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5168d9b5-d499646f-94ab6a9c-558888ad_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e46c7102-8f078397-cc38df75-27209c66_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:32.394828Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e46c7102-8f078397-cc38df75-27209c66_0] Write session: try to update token
2025-10-09T21:50:32.394843Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e46c7102-8f078397-cc38df75-27209c66_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:32.405631Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:32.405648Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:32.405651Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:32.405653Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:32.405663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342724762896810:2386]
2025-10-09T21:50:32.405668Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342724762896810:2386]
2025-10-09T21:50:32.405672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342724762896810:2386]
2025-10-09T21:50:32.405717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:32.405725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.405724Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:32.405730Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:32.405733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:32.405737Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:32.405868Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:32.407106Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1859 sessionId: src_id|e46c7102-8f078397-cc38df75-27209c66_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:32.409482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:32.468460Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342725124728900:6018]
2025-10-09T21:50:32.468704Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 1858, pipe: [1:7559342725124728900:6018]
2025-10-09T21:50:32.548273Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:32.548548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.548563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.548606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342725124728900:6018], now have 1 active actors on pipe
2025-10-09T21:50:32.548807Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:32.548811Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:32.548827Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342725124728900:6018]
2025-10-09T21:50:32.548831Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:32.548849Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:32.549217Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1859 reason: error at write: Disconnected sessionId: src_id|e46c7102-8f078397-cc38df75-27209c66_0
2025-10-09T21:50:32.549425Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1859 sessionId: src_id|e46c7102-8f078397-cc38df75-27209c66_0 is DEAD
2025-10-09T21:50:32.549554Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e46c7102-8f078397-cc38df75-27209c66_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:32.549570Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e46c7102-8f078397-cc38df75-27209c66_0] Write session will restart in 0.006648s
2025-10-09T21:50:32.549593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e46c7102-8f078397-cc38df75-27209c66_0] Write session: Do CDS request
2025-10-09T21:50:32.549600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e46c7102-8f078397-cc38df75-27209c66_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:32.557468Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e46c7102-8f078397-cc38df75-27209c66_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:32.557548Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:32.557563Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1860
2025-10-09T21:50:32.557657Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1860 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:32.557688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1860 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:32.557699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1860 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:32.557707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:32.558123Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1860 sessionId:  describe result for acl check
2025-10-09T21:50:32.558161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:32.558169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:32.558170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:32.558176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342725124728988:5539] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:32.558609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342725124728988:5539] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:32.560588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342725124728988:5539] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:32.561147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:32.563109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:32.563270Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:32.563279Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:32.563268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342725124728988:5539] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:32.563290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:32.563279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342725124728988:5539] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:32.563282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342725124728988:5539] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:32.563287Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1860 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:32.563340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:32.563376Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342724762897041:2361]
2025-10-09T21:50:32.563931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:32.564176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:32.564217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:32.564516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:32.564566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.564572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.564573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.564574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.564575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.564576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.564577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.564578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.564579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.564584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.564594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.564598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.564602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.564606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.564610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.564613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.564617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.564620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.564626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.564638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.564644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.564653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.564661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.564666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.564675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.564680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.564684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.564693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.564697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.564703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.564710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.564715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.564723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.564728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.564732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.564737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.564771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:32.570970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:32.571942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:32.571958Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:32.571959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:32.571962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:32.571966Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1859 [2:7559342724762897041:2361]
2025-10-09T21:50:32.571977Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:32.571989Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:32.571992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:32.571994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:32.571997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:32.572013Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:32.572018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:32.572025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:32.572076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.572084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.572086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.572089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:32.572139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:32.572145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.572155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.572166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.572187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:32.572211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342725124728903:2432], now have 1 active actors on pipe
2025-10-09T21:50:32.572217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342725124729089:5539], now have 1 active actors on pipe
2025-10-09T21:50:32.572222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342724762897036:2361]
2025-10-09T21:50:32.572224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342724762897036:2361]
2025-10-09T21:50:32.572225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342724762897036:2361]
2025-10-09T21:50:32.572235Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:32.572245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:32.572253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:32.572259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:32.572259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:32.572262Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:32.572264Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:32.572276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342724762897036:2361]
2025-10-09T21:50:32.572282Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:32.572285Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:32.572274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1859
2025-10-09T21:50:32.572295Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1859
2025-10-09T21:50:32.572317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.572326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.572350Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8a0d087c-f63bc8f7-3604e56a-8d116ff4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:32.572362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:32.572366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:32.572368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:32.572378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:32.572391Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:32.572402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:32.572573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.572582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.572595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:32.572674Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1860 partition: 0 MaxSeqNo: 25 sessionId: src_id|8a0d087c-f63bc8f7-3604e56a-8d116ff4_0
2025-10-09T21:50:32.572970Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e46c7102-8f078397-cc38df75-27209c66_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8a0d087c-f63bc8f7-3604e56a-8d116ff4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:32.575675Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a0d087c-f63bc8f7-3604e56a-8d116ff4_0] Write session: try to update token
2025-10-09T21:50:32.575690Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a0d087c-f63bc8f7-3604e56a-8d116ff4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:32.582910Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:32.582926Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:32.582935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342724762897036:2361]
2025-10-09T21:50:32.582976Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:32.582977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:32.583136Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:32.583268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:32.583393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:32.583513Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:32.583682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:32.583715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:32.587278Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1860 sessionId: src_id|8a0d087c-f63bc8f7-3604e56a-8d116ff4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:32.589540Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:32.589727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.589746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.589781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:32.589877Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:32.595229Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:32.595246Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:32.595248Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:32.595250Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:32.595260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342724762897036:2361]
2025-10-09T21:50:32.595265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342724762897036:2361]
2025-10-09T21:50:32.595269Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342724762897036:2361]
2025-10-09T21:50:32.595304Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:32.595306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:32.595309Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:32.595312Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:32.595313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.595315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:32.595440Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:32.649203Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342725124729100:6018]
2025-10-09T21:50:32.649474Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 1859, pipe: [1:7559342725124729100:6018]
2025-10-09T21:50:32.731496Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:32.738225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.738245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.738383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:32.738437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:32.738491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:32.738546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:32.738590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:32.738597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:32.738655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:32.738707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:32.738776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:32.738837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:32.739266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:32.739274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:32.739326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:32.739376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:32.739431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:32.739484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:32.739586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:32.739592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:32.739650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:32.739703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:32.739757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:32.739811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:32.739849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:32.739856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:32.739910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:32.739962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:32.740013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:32.740069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:32.740105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:32.740110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:32.740160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:32.740214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:32.740269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:32.740321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:32.740357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:32.740362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:32.740412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:32.740462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:32.740512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:32.740563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:32.740600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:32.740605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:32.740655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:32.740704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:32.740754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:32.740809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:32.740845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:32.740850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:32.740901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:32.740950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:32.740998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:32.741053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:32.741096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:32.741104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:32.741161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:32.741212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:32.741260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:32.741313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:32.741357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:32.741363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:32.741399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342725124729100:6018], now have 1 active actors on pipe
2025-10-09T21:50:32.741596Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342725124729100:6018]
2025-10-09T21:50:32.741612Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:32.741629Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:32.741631Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:32.741665Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1860 reason: error at write: Disconnected sessionId: src_id|8a0d087c-f63bc8f7-3604e56a-8d116ff4_0
2025-10-09T21:50:32.741795Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:32.741825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1860 sessionId: src_id|8a0d087c-f63bc8f7-3604e56a-8d116ff4_0 is DEAD
2025-10-09T21:50:32.741966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a0d087c-f63bc8f7-3604e56a-8d116ff4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:32.741971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a0d087c-f63bc8f7-3604e56a-8d116ff4_0] Write session will restart in 0.006391s
2025-10-09T21:50:32.741989Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a0d087c-f63bc8f7-3604e56a-8d116ff4_0] Write session: Do CDS request
2025-10-09T21:50:32.741994Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a0d087c-f63bc8f7-3604e56a-8d116ff4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:32.749422Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a0d087c-f63bc8f7-3604e56a-8d116ff4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:32.749564Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:32.749580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1861
2025-10-09T21:50:32.749690Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1861 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:32.749720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1861 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:32.749726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1861 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:32.749729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:32.750216Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1861 sessionId:  describe result for acl check
2025-10-09T21:50:32.750254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:32.750261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:32.750262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:32.750267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342725124729163:5524] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:32.750798Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342725124729163:5524] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:32.753467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342725124729163:5524] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:32.754361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:32.756255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:32.756401Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342725124729163:5524] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:32.756414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342725124729163:5524] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:32.756418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342725124729163:5524] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:32.756425Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1861 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:32.756422Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:32.756434Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:32.756447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:32.756502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:32.756538Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342724762897234:2344]
2025-10-09T21:50:32.757111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:32.757377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:32.757417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:32.757745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:32.757795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.757801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.757802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.757803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.757804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.757806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.757807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.757808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.757809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.757813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.757824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.757828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.757832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.757836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.757840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.757844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.757848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.757852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.757858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.757865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.757870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.757875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.757879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.757883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.757887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.757892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.757896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.757901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.757911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.757916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.757920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.757925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.757930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.757934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.757938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.757943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.757976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:32.764157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:32.765149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:32.765164Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:32.765166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:32.765170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:32.765176Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1860 [2:7559342724762897234:2344]
2025-10-09T21:50:32.765189Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:32.765204Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:32.765207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:32.765210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:32.765212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:32.765228Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:32.765238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:32.765244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:32.765441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.765451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.765453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.765467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:32.765533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:32.765541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.765552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.765564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.765580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:32.765604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342725124729290:5524], now have 1 active actors on pipe
2025-10-09T21:50:32.765612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342725124729103:2432], now have 1 active actors on pipe
2025-10-09T21:50:32.765618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342724762897229:2344]
2025-10-09T21:50:32.765620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342724762897229:2344]
2025-10-09T21:50:32.765623Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342724762897229:2344]
2025-10-09T21:50:32.765636Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:32.765653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:32.765661Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:32.765664Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:32.765666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:32.765670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:32.765673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:32.765670Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1860
2025-10-09T21:50:32.765694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342724762897229:2344]
2025-10-09T21:50:32.765701Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:32.765704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:32.765728Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1860
2025-10-09T21:50:32.765785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.765794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.765818Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1ebf533c-cdc6f57d-541a2ef-d885627b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:32.765829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:32.765833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:32.765835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:32.765844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:32.765858Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:32.765876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:32.766049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.766058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.766072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:32.766160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1861 partition: 0 MaxSeqNo: 25 sessionId: src_id|1ebf533c-cdc6f57d-541a2ef-d885627b_0
2025-10-09T21:50:32.766434Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a0d087c-f63bc8f7-3604e56a-8d116ff4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1ebf533c-cdc6f57d-541a2ef-d885627b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:32.768782Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ebf533c-cdc6f57d-541a2ef-d885627b_0] Write session: try to update token
2025-10-09T21:50:32.768797Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ebf533c-cdc6f57d-541a2ef-d885627b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:32.779431Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:32.779448Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:32.779451Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:32.779453Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:32.779462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342724762897229:2344]
2025-10-09T21:50:32.779466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342724762897229:2344]
2025-10-09T21:50:32.779471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342724762897229:2344]
2025-10-09T21:50:32.779506Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:32.779510Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:32.779512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:32.779513Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:32.779518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.779521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:32.779651Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:32.780724Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1861 sessionId: src_id|1ebf533c-cdc6f57d-541a2ef-d885627b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:32.782867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:32.841726Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342725124729301:6018]
2025-10-09T21:50:32.842020Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 1860, pipe: [1:7559342725124729301:6018]
2025-10-09T21:50:32.926389Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:32.926748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.926761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.926797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342725124729301:6018], now have 1 active actors on pipe
2025-10-09T21:50:32.926988Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:32.926993Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342725124729301:6018]
2025-10-09T21:50:32.927012Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:32.927015Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:32.927070Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1861 reason: error at write: Disconnected sessionId: src_id|1ebf533c-cdc6f57d-541a2ef-d885627b_0
2025-10-09T21:50:32.927099Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:32.927214Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1861 sessionId: src_id|1ebf533c-cdc6f57d-541a2ef-d885627b_0 is DEAD
2025-10-09T21:50:32.927362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ebf533c-cdc6f57d-541a2ef-d885627b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:32.927378Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ebf533c-cdc6f57d-541a2ef-d885627b_0] Write session will restart in 0.006049s
2025-10-09T21:50:32.927398Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ebf533c-cdc6f57d-541a2ef-d885627b_0] Write session: Do CDS request
2025-10-09T21:50:32.927402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ebf533c-cdc6f57d-541a2ef-d885627b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:32.934423Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ebf533c-cdc6f57d-541a2ef-d885627b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:32.934479Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:32.934493Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1862
2025-10-09T21:50:32.934606Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1862 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:32.934635Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1862 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:32.934645Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1862 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:32.934652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:32.935077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1862 sessionId:  describe result for acl check
2025-10-09T21:50:32.935110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:32.935119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:32.935121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:32.935128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342725124729366:5525] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:32.935673Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342725124729366:5525] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:32.938052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342725124729366:5525] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:32.938547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:32.940318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:32.940448Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:32.940455Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:32.940463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:32.940494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:32.940535Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342724762897424:2350]
2025-10-09T21:50:32.940588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342725124729366:5525] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:32.940601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342725124729366:5525] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:32.940605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342725124729366:5525] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:32.940611Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1862 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:32.940882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:32.941097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:32.941122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:32.941304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:32.941331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.941335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.941335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.941336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.941337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.941337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.941338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.941338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.941339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.941342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.941348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.941350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.941352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.941354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.941356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.941358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.941360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.941361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.941365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.941370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:32.941374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.941377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:32.941379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.941382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:32.941384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.941387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:32.941389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.941392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:32.941394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.941398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:32.941400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.941403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.941405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.941407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.941410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.941413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.941428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:32.947225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:32.948144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:32.948157Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:32.948158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:32.948160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:32.948165Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1861 [2:7559342724762897424:2350]
2025-10-09T21:50:32.948173Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:32.948186Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:32.948187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:32.948189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:32.948192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:32.948206Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:32.948211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:32.948218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:32.948263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.948269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.948270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.948272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:32.948417Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1861
2025-10-09T21:50:32.948437Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1861
2025-10-09T21:50:32.948433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:32.948443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.948451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:32.948457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:32.948467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:32.948482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342725124729306:2432], now have 1 active actors on pipe
2025-10-09T21:50:32.948488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342725124729494:5525], now have 1 active actors on pipe
2025-10-09T21:50:32.948499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.948505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.948520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342724762897419:2350]
2025-10-09T21:50:32.948525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342724762897419:2350]
2025-10-09T21:50:32.948527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342724762897419:2350]
2025-10-09T21:50:32.948535Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:32.948545Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:32.948557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:32.948563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:32.948564Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:32.948568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:32.948570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:32.948576Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3fcfcda3-91f41e9b-936f9728-e89de588_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:32.948582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:32.948585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:32.948586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:32.948592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:32.948600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:32.948616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342724762897419:2350]
2025-10-09T21:50:32.948622Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:32.948630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:32.948630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:32.948753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.948779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.948801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:32.948883Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1862 partition: 0 MaxSeqNo: 25 sessionId: src_id|3fcfcda3-91f41e9b-936f9728-e89de588_0
2025-10-09T21:50:32.949145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ebf533c-cdc6f57d-541a2ef-d885627b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3fcfcda3-91f41e9b-936f9728-e89de588_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:32.951966Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3fcfcda3-91f41e9b-936f9728-e89de588_0] Write session: try to update token
2025-10-09T21:50:32.951983Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3fcfcda3-91f41e9b-936f9728-e89de588_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:32.959482Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:32.959496Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:32.959504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342724762897419:2350]
2025-10-09T21:50:32.959537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:32.959539Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:32.959692Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:32.959817Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:32.959948Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:32.960079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:32.960243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:32.960275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:32.963845Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1862 sessionId: src_id|3fcfcda3-91f41e9b-936f9728-e89de588_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:32.966020Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:32.966190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:32.966205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:32.966236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:32.966292Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:32.971278Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:32.971287Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:32.971289Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:32.971290Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:32.971298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342724762897419:2350]
2025-10-09T21:50:32.971301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342724762897419:2350]
2025-10-09T21:50:32.971304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342724762897419:2350]
2025-10-09T21:50:32.971331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:32.971336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:32.971335Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:32.971338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:32.971340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:32.971343Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:32.971463Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:33.027065Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342729419696802:6018]
2025-10-09T21:50:33.027311Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 1861, pipe: [1:7559342729419696802:6018]
2025-10-09T21:50:33.116519Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:33.123283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:33.123306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:33.123456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.123511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.123566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.123622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.123667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.123675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.123734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.123788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.123843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.123901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.123946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.123952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.124005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.124055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.124110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.124169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.124207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.124213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.124268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.124321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.124390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.124450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.124488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.124494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.124550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.124599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.124655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.124711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.124774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.124780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.124832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.124883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.124945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.124999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.125037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.125046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.125096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.125146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.125207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.125260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.125300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.125306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.125357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.125409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.125476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.125529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.125582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.125587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.125638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.125691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.125746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.125799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.125839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.125846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.125898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.125954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.126012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.126074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.126111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.126116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.126165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342729419696802:6018], now have 1 active actors on pipe
2025-10-09T21:50:33.126377Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342729419696802:6018]
2025-10-09T21:50:33.126401Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:33.126412Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:33.126414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:33.126450Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1862 reason: error at write: Disconnected sessionId: src_id|3fcfcda3-91f41e9b-936f9728-e89de588_0
2025-10-09T21:50:33.126618Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1862 sessionId: src_id|3fcfcda3-91f41e9b-936f9728-e89de588_0 is DEAD
2025-10-09T21:50:33.126777Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3fcfcda3-91f41e9b-936f9728-e89de588_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:33.126789Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3fcfcda3-91f41e9b-936f9728-e89de588_0] Write session will restart in 0.006913s
2025-10-09T21:50:33.126814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3fcfcda3-91f41e9b-936f9728-e89de588_0] Write session: Do CDS request
2025-10-09T21:50:33.126821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3fcfcda3-91f41e9b-936f9728-e89de588_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:33.127005Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:33.134451Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3fcfcda3-91f41e9b-936f9728-e89de588_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:33.134529Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:33.134544Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1863
2025-10-09T21:50:33.134665Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1863 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:33.134691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1863 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:33.134698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1863 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:33.134702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:33.135135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1863 sessionId:  describe result for acl check
2025-10-09T21:50:33.135176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:33.135183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:33.135186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:33.135192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342729419696880:5518] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:33.135732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342729419696880:5518] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:33.138691Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342729419696880:5518] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:33.138820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:33.140826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:33.141006Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:33.141018Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:33.141030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:33.141082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:33.141122Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342729057864911:2348]
2025-10-09T21:50:33.141260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342729419696880:5518] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:33.141273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342729419696880:5518] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:33.141277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342729419696880:5518] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:33.141285Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1863 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:33.141713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:33.141960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:33.142002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:33.142334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:33.142385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.142391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.142392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.142393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.142395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.142396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.142397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.142398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.142399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.142405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.142415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.142424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.142428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.142432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.142440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.142444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.142452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.142456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.142462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.142469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.142480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.142490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.142500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.142509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.142513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.142518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.142528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.142533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.142541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.142547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.142556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.142563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.142571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.142576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.142585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.142590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.142623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:33.148667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:33.149643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:33.149658Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:33.149660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:33.149663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:33.149670Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1862 [2:7559342729057864911:2348]
2025-10-09T21:50:33.149680Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:33.149692Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:33.149694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:33.149696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:33.149698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:33.149710Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:33.149719Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:33.149725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:33.149763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.149768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.149769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.149771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:33.149944Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1862
2025-10-09T21:50:33.149956Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1862
2025-10-09T21:50:33.149993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:33.150000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.150007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.150013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.150023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:33.150056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342729419696804:2432], now have 1 active actors on pipe
2025-10-09T21:50:33.150059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342729419696993:5518], now have 1 active actors on pipe
2025-10-09T21:50:33.150075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:33.150082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:33.150099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342729057864906:2348]
2025-10-09T21:50:33.150105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342729057864906:2348]
2025-10-09T21:50:33.150106Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342729057864906:2348]
2025-10-09T21:50:33.150116Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:33.150126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:33.150139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:33.150138Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:33.150141Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:33.150142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:33.150144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:33.150159Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c889cf19-f91e9d15-f157ce49-759d3add_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:33.150167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:33.150170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:33.150172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:33.150179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:33.150190Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:33.150217Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342729057864906:2348]
2025-10-09T21:50:33.150226Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:33.150234Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:33.150236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:33.150547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:33.150555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:33.150571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:33.150675Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1863 partition: 0 MaxSeqNo: 25 sessionId: src_id|c889cf19-f91e9d15-f157ce49-759d3add_0
2025-10-09T21:50:33.150973Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3fcfcda3-91f41e9b-936f9728-e89de588_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c889cf19-f91e9d15-f157ce49-759d3add_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:33.153668Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c889cf19-f91e9d15-f157ce49-759d3add_0] Write session: try to update token
2025-10-09T21:50:33.153683Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c889cf19-f91e9d15-f157ce49-759d3add_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:33.166742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1863 sessionId: src_id|c889cf19-f91e9d15-f157ce49-759d3add_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:33.168332Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:33.168348Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:33.168357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342729057864906:2348]
2025-10-09T21:50:33.168398Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:33.168406Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:33.168564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:33.168705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:33.168836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:33.168977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:33.169069Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:33.169166Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:33.169225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:33.169238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:33.169263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:33.169305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:33.169401Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:33.175868Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:33.175883Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:33.175885Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:33.175887Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:33.175895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342729057864906:2348]
2025-10-09T21:50:33.175899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342729057864906:2348]
2025-10-09T21:50:33.175903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342729057864906:2348]
2025-10-09T21:50:33.175940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:33.175947Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:33.175952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.175953Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:33.175955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:33.175957Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:33.176079Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:33.226763Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342729419697008:6018]
2025-10-09T21:50:33.227039Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 1862, pipe: [1:7559342729419697008:6018]
2025-10-09T21:50:33.323664Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:33.330288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:33.330315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:33.330459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.330514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.330576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.330635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.330674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.330683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.330741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.330801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.330860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.330916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.330955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.330960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.331014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.331069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.331128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.331179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.331217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.331223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.331280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.331343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.331396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.331451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.331490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.331497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.331553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.331619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.331675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.331726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.331765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.331772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.331839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.331901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.331956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.332006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.332113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.332124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.332178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.332227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.332281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.332336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.332453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.332460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.332512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.332561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.332614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.332675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.332719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.332725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.332778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.332827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.332883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.332940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.332978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.332986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.333038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:33.333090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:33.333149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:33.333203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:33.333264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:33.333271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:33.333311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342729419697008:6018], now have 1 active actors on pipe
2025-10-09T21:50:33.333513Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:33.333525Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342729419697008:6018]
2025-10-09T21:50:33.333553Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:33.333558Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:33.333586Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1863 reason: error at write: Disconnected sessionId: src_id|c889cf19-f91e9d15-f157ce49-759d3add_0
2025-10-09T21:50:33.333790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1863 sessionId: src_id|c889cf19-f91e9d15-f157ce49-759d3add_0 is DEAD
2025-10-09T21:50:33.333924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c889cf19-f91e9d15-f157ce49-759d3add_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:33.333934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c889cf19-f91e9d15-f157ce49-759d3add_0] Write session will restart in 0.005896s
2025-10-09T21:50:33.333958Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c889cf19-f91e9d15-f157ce49-759d3add_0] Write session: Do CDS request
2025-10-09T21:50:33.333964Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c889cf19-f91e9d15-f157ce49-759d3add_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:33.334052Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:33.340426Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c889cf19-f91e9d15-f157ce49-759d3add_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:33.340544Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:33.340560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1864
2025-10-09T21:50:33.340679Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1864 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:33.340712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1864 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:33.340732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1864 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:33.340739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:33.341118Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1864 sessionId:  describe result for acl check
2025-10-09T21:50:33.341148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:33.341153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:33.341154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:33.341158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342729419697052:5512] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:33.341608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342729419697052:5512] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:33.344531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342729419697052:5512] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:33.346822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:33.347195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342729419697052:5512] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:33.347208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342729419697052:5512] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:33.347212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342729419697052:5512] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:33.347219Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1864 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:33.348851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:33.349022Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:33.349032Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:33.349043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:33.349092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:33.349131Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342729057865104:2331]
2025-10-09T21:50:33.349690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:33.349953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:33.349993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:33.350300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:33.350341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.350346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.350347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.350348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.350349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.350350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.350352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.350353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.350354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.350359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.350369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.350373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.350377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.350380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.350384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.350388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.350392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.350395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.350401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.350412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.350423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.350432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.350436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.350441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.350445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.350450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.350454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.350459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.350462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.350470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.350475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.350483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.350487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.350495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.350504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.350512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.350539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:33.356457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:33.357773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:33.357786Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:33.357787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:33.357791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:33.357797Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1863 [2:7559342729057865104:2331]
2025-10-09T21:50:33.357811Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:33.357829Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:33.357832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:33.357835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:33.357841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:33.357856Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:33.357866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:33.357877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:33.357935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.357942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.357943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.357946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:33.358010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:33.358018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.358044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.358054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.358068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:33.358093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342729419697015:2432], now have 1 active actors on pipe
2025-10-09T21:50:33.358101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342729419697205:5512], now have 1 active actors on pipe
2025-10-09T21:50:33.358109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342729057865098:2331]
2025-10-09T21:50:33.358111Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342729057865098:2331]
2025-10-09T21:50:33.358113Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342729057865098:2331]
2025-10-09T21:50:33.358128Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:33.358143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:33.358133Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1863
2025-10-09T21:50:33.358142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:33.358145Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:33.358152Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1863
2025-10-09T21:50:33.358148Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:33.358163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:33.358166Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:33.358203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:33.358213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:33.358230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342729057865098:2331]
2025-10-09T21:50:33.358243Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:33.358261Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:33.358276Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|19ea77d5-943d3c33-9d9d27d0-87c06f48_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:33.358284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:33.358291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:33.358294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:33.358305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:33.358318Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:33.358341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:33.358498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:33.358507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:33.358523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:33.358599Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1864 partition: 0 MaxSeqNo: 25 sessionId: src_id|19ea77d5-943d3c33-9d9d27d0-87c06f48_0
2025-10-09T21:50:33.358889Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c889cf19-f91e9d15-f157ce49-759d3add_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|19ea77d5-943d3c33-9d9d27d0-87c06f48_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:33.361153Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19ea77d5-943d3c33-9d9d27d0-87c06f48_0] Write session: try to update token
2025-10-09T21:50:33.361169Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19ea77d5-943d3c33-9d9d27d0-87c06f48_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:33.371844Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:33.371864Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:33.371867Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:33.371869Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:33.371877Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342729057865098:2331]
2025-10-09T21:50:33.371882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342729057865098:2331]
2025-10-09T21:50:33.371886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342729057865098:2331]
2025-10-09T21:50:33.371933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:33.371932Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:33.371939Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:33.371943Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:33.371943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.371946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:33.372076Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:33.373043Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1864 sessionId: src_id|19ea77d5-943d3c33-9d9d27d0-87c06f48_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:33.375502Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:33.433602Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342729419697217:6018]
2025-10-09T21:50:33.433957Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 1863, pipe: [1:7559342729419697217:6018]
2025-10-09T21:50:33.520936Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:33.521291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:33.521304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:33.521342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342729419697217:6018], now have 1 active actors on pipe
2025-10-09T21:50:33.521573Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:33.521575Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:33.521604Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342729419697217:6018]
2025-10-09T21:50:33.521624Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:33.521627Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:33.521688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1864 reason: error at write: Disconnected sessionId: src_id|19ea77d5-943d3c33-9d9d27d0-87c06f48_0
2025-10-09T21:50:33.521839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1864 sessionId: src_id|19ea77d5-943d3c33-9d9d27d0-87c06f48_0 is DEAD
2025-10-09T21:50:33.522024Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19ea77d5-943d3c33-9d9d27d0-87c06f48_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:33.522053Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19ea77d5-943d3c33-9d9d27d0-87c06f48_0] Write session will restart in 0.005923s
2025-10-09T21:50:33.522081Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19ea77d5-943d3c33-9d9d27d0-87c06f48_0] Write session: Do CDS request
2025-10-09T21:50:33.522087Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19ea77d5-943d3c33-9d9d27d0-87c06f48_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:33.529503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|19ea77d5-943d3c33-9d9d27d0-87c06f48_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:33.529581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:33.529596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1865
2025-10-09T21:50:33.530261Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1865 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:33.530299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1865 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:33.530310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1865 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:33.530314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:33.530862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1865 sessionId:  describe result for acl check
2025-10-09T21:50:33.530898Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:33.530900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:33.530902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:33.530908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342729419697305:5496] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:33.531414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342729419697305:5496] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:33.533629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342729419697305:5496] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:33.535064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:33.537235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:33.537403Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:33.537412Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:33.537422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:33.537476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:33.537512Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342729057865334:2307]
2025-10-09T21:50:33.537558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342729419697305:5496] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:33.537571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342729419697305:5496] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:33.537574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342729419697305:5496] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:33.537582Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1865 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:33.538087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:33.538365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:33.538401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:33.538700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:33.538740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.538746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.538747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.538748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.538749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.538751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.538751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.538753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.538754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.538758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.538768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.538772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.538776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.538779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.538783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.538787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.538790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.538794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.538799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.538807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.538813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.538818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.538822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.538831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.538836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.538845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.538851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.538854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.538858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.538863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.538874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.538879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.538883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.538887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.538893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.538898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.538925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:33.545339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:33.546614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:33.546629Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:33.546631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:33.546635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:33.546642Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1864 [2:7559342729057865334:2307]
2025-10-09T21:50:33.546655Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:33.546678Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:33.546680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:33.546683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:33.546686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:33.546705Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:33.546711Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:33.546723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:33.546786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.546793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.546796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.546799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:33.546865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:33.546875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.546886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.546899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.546921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:33.546947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342729419697432:5496], now have 1 active actors on pipe
2025-10-09T21:50:33.546955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342729419697244:2432], now have 1 active actors on pipe
2025-10-09T21:50:33.546963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342729057865329:2307]
2025-10-09T21:50:33.546970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342729057865329:2307]
2025-10-09T21:50:33.546972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342729057865329:2307]
2025-10-09T21:50:33.546986Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:33.547005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:33.547004Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:33.547008Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:33.547010Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:33.546993Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1864
2025-10-09T21:50:33.547024Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:33.547027Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:33.547009Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1864
2025-10-09T21:50:33.547060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342729057865329:2307]
2025-10-09T21:50:33.547071Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:33.547075Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:33.547086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:33.547096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:33.547141Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1c4b5e6d-86a80040-6a52f42d-1ef43398_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:33.547151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:33.547158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:33.547161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:33.547170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:33.547184Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:33.547204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:33.547378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:33.547385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:33.547397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:33.547481Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1865 partition: 0 MaxSeqNo: 25 sessionId: src_id|1c4b5e6d-86a80040-6a52f42d-1ef43398_0
2025-10-09T21:50:33.547878Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|19ea77d5-943d3c33-9d9d27d0-87c06f48_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1c4b5e6d-86a80040-6a52f42d-1ef43398_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:33.550447Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c4b5e6d-86a80040-6a52f42d-1ef43398_0] Write session: try to update token
2025-10-09T21:50:33.550463Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c4b5e6d-86a80040-6a52f42d-1ef43398_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:33.562633Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:33.562655Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:33.562657Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:33.562659Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:33.562671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342729057865329:2307]
2025-10-09T21:50:33.562675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342729057865329:2307]
2025-10-09T21:50:33.562679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342729057865329:2307]
2025-10-09T21:50:33.562722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:33.562723Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:33.562729Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:33.562730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.562732Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:33.562733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:33.562883Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:33.564083Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1865 sessionId: src_id|1c4b5e6d-86a80040-6a52f42d-1ef43398_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:33.567182Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:33.622002Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342729419697443:6018]
2025-10-09T21:50:33.622369Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 1864, pipe: [1:7559342729419697443:6018]
2025-10-09T21:50:33.716278Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:33.716725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:33.716745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:33.716791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342729419697443:6018], now have 1 active actors on pipe
2025-10-09T21:50:33.716998Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:33.717191Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:33.717208Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:33.717214Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342729419697443:6018]
2025-10-09T21:50:33.717235Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:33.717285Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1865 reason: error at write: Disconnected sessionId: src_id|1c4b5e6d-86a80040-6a52f42d-1ef43398_0
2025-10-09T21:50:33.717492Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1865 sessionId: src_id|1c4b5e6d-86a80040-6a52f42d-1ef43398_0 is DEAD
2025-10-09T21:50:33.717658Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c4b5e6d-86a80040-6a52f42d-1ef43398_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:33.717672Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c4b5e6d-86a80040-6a52f42d-1ef43398_0] Write session will restart in 0.009870s
2025-10-09T21:50:33.717713Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c4b5e6d-86a80040-6a52f42d-1ef43398_0] Write session: Do CDS request
2025-10-09T21:50:33.717720Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c4b5e6d-86a80040-6a52f42d-1ef43398_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:33.728572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c4b5e6d-86a80040-6a52f42d-1ef43398_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:33.728685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:33.728705Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1866
2025-10-09T21:50:33.728856Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1866 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:33.728896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1866 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:33.728907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1866 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:33.728911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:33.729314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1866 sessionId:  describe result for acl check
2025-10-09T21:50:33.729347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:33.729352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:33.729353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:33.729359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342729419697536:5488] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:33.729907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342729419697536:5488] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:33.732289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342729419697536:5488] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:33.733026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:33.735337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:33.735483Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:33.735492Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:33.735504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:33.735555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:33.735588Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342729057865529:10240]
2025-10-09T21:50:33.735656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342729419697536:5488] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:33.735669Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342729419697536:5488] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:33.735674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342729419697536:5488] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:33.735690Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1866 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:33.735986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:33.736364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:33.736456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:33.736834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:33.736919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.736927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.736928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.736928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.736929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.736930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.736931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.736931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.736932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.736936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.736943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.736946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.736948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.736950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.736952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.736954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.736956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.736958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.736963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.736974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.736980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.736985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.736990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.736996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.736998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.737001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.737004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.737007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.737011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.737014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.737017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.737020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.737022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.737025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.737028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.737031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.737048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:33.743401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:33.744700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:33.744717Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:33.744720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:33.744725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:33.744731Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1865 [2:7559342729057865529:10240]
2025-10-09T21:50:33.744745Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:33.744763Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:33.744770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:33.744773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:33.744774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:33.744788Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:33.744797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:33.744803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:33.744865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.744873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.744875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.744877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:33.744934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:33.744945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.744956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.744969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.744987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:33.745013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342729419697446:2432], now have 1 active actors on pipe
2025-10-09T21:50:33.745020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342729057865524:10240]
2025-10-09T21:50:33.745030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342729057865524:10240]
2025-10-09T21:50:33.745031Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342729057865524:10240]
2025-10-09T21:50:33.745042Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:33.745049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342729419697638:5488], now have 1 active actors on pipe
2025-10-09T21:50:33.745056Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:33.745062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:33.745064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:33.745065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:33.745067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:33.745068Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:33.745069Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1865
2025-10-09T21:50:33.745093Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1865
2025-10-09T21:50:33.745094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342729057865524:10240]
2025-10-09T21:50:33.745096Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:33.745110Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:33.745124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:33.745128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:33.745153Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|beacb797-a854e1e2-8625b55e-cb0d45c9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:33.745165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:33.745168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:33.745169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:33.745177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:33.745186Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:33.745199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:33.745336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:33.745345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:33.745356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:33.745422Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1866 partition: 0 MaxSeqNo: 25 sessionId: src_id|beacb797-a854e1e2-8625b55e-cb0d45c9_0
2025-10-09T21:50:33.745789Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c4b5e6d-86a80040-6a52f42d-1ef43398_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|beacb797-a854e1e2-8625b55e-cb0d45c9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:33.748084Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|beacb797-a854e1e2-8625b55e-cb0d45c9_0] Write session: try to update token
2025-10-09T21:50:33.748100Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|beacb797-a854e1e2-8625b55e-cb0d45c9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:33.759478Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:33.759503Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:33.759507Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:33.759509Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:33.759520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342729057865524:10240]
2025-10-09T21:50:33.759526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342729057865524:10240]
2025-10-09T21:50:33.759531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342729057865524:10240]
2025-10-09T21:50:33.759581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:33.759590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.759593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:33.759591Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:33.759600Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:33.759604Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:33.759756Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:33.760549Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1866 sessionId: src_id|beacb797-a854e1e2-8625b55e-cb0d45c9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:33.762994Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:33.817622Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342729419697649:6018]
2025-10-09T21:50:33.817937Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 1865, pipe: [1:7559342729419697649:6018]
2025-10-09T21:50:33.916444Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:33.916719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:33.916732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:33.916773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342729419697649:6018], now have 1 active actors on pipe
2025-10-09T21:50:33.916963Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:33.917042Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:33.917044Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342729419697649:6018]
2025-10-09T21:50:33.917070Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:33.917146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1866 reason: status is not ok: tablet will be restarted right now sessionId: src_id|beacb797-a854e1e2-8625b55e-cb0d45c9_0
2025-10-09T21:50:33.917323Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1866 sessionId: src_id|beacb797-a854e1e2-8625b55e-cb0d45c9_0 is DEAD
2025-10-09T21:50:33.917523Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|beacb797-a854e1e2-8625b55e-cb0d45c9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:33.917535Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|beacb797-a854e1e2-8625b55e-cb0d45c9_0] Write session will restart in 0.008762s
2025-10-09T21:50:33.917563Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|beacb797-a854e1e2-8625b55e-cb0d45c9_0] Write session: Do CDS request
2025-10-09T21:50:33.917570Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|beacb797-a854e1e2-8625b55e-cb0d45c9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:33.927509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|beacb797-a854e1e2-8625b55e-cb0d45c9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:33.927556Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:33.927572Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1867
2025-10-09T21:50:33.927712Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1867 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:33.927747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1867 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:33.927758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1867 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:33.927762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:33.928191Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1867 sessionId:  describe result for acl check
2025-10-09T21:50:33.928226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:33.928232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:33.928233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:33.928238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342729419697809:5486] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:33.928767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342729419697809:5486] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:33.928956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:33.930604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342729419697809:5486] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:33.931026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:33.931201Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:33.931210Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:33.931220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:33.931267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:33.931308Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342729057865728:10244]
2025-10-09T21:50:33.931854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:33.932121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:33.932163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:33.932454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:33.932497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.932504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.932505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.932506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.932507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.932508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.932509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.932510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.932511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.932516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.932527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.932531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.932535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.932538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.932542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.932546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.932550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.932554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.932560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.932568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:33.932574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.932579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:33.932584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.932589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:33.932593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.932598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:33.932602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.932607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:33.932612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.932617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:33.932622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.932627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.932631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.932636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.932640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.932645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.932672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:33.932913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342729419697809:5486] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:33.932925Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342729419697809:5486] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:33.932928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342729419697809:5486] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:33.932936Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1867 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:33.938697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:33.939734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:33.939754Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:33.939756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:33.939759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:33.939766Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1866 [2:7559342729057865728:10244]
2025-10-09T21:50:33.939777Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:33.939794Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:33.939797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:33.939799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:33.939802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:33.939817Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:33.939822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:33.939829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:33.939882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.939891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.939892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.939895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:33.939962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:33.939970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.939982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:33.939999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:33.940021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:33.940047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342729419697851:5486], now have 1 active actors on pipe
2025-10-09T21:50:33.940056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342729419697654:2432], now have 1 active actors on pipe
2025-10-09T21:50:33.940062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342729057865723:10244]
2025-10-09T21:50:33.940065Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342729057865723:10244]
2025-10-09T21:50:33.940067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342729057865723:10244]
2025-10-09T21:50:33.940079Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:33.940098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:33.940097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:33.940100Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:33.940104Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:33.940117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:33.940120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:33.940102Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1866
2025-10-09T21:50:33.940102Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1866
2025-10-09T21:50:33.940143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342729057865723:10244]
2025-10-09T21:50:33.940154Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:33.940156Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:33.940169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:33.940174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:33.940219Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cb48b5d5-a3f6df7f-a1fa0b4e-5c6dc9ea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:33.940231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:33.940235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:33.940238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:33.940247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:33.940256Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:33.940274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:33.940439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:33.940455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:33.940476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:33.940580Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1867 partition: 0 MaxSeqNo: 25 sessionId: src_id|cb48b5d5-a3f6df7f-a1fa0b4e-5c6dc9ea_0
2025-10-09T21:50:33.940933Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|beacb797-a854e1e2-8625b55e-cb0d45c9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cb48b5d5-a3f6df7f-a1fa0b4e-5c6dc9ea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:33.943351Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cb48b5d5-a3f6df7f-a1fa0b4e-5c6dc9ea_0] Write session: try to update token
2025-10-09T21:50:33.943371Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cb48b5d5-a3f6df7f-a1fa0b4e-5c6dc9ea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:33.953939Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:33.953956Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:33.953958Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:33.953960Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:33.953967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342729057865723:10244]
2025-10-09T21:50:33.953970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342729057865723:10244]
2025-10-09T21:50:33.953974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342729057865723:10244]
2025-10-09T21:50:33.954007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:33.954013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:33.954014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:33.954058Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:33.954076Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:33.954079Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:33.954140Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:33.955990Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1867 sessionId: src_id|cb48b5d5-a3f6df7f-a1fa0b4e-5c6dc9ea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:33.958340Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:34.017271Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342733714665155:6018]
2025-10-09T21:50:34.017541Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 1866, pipe: [1:7559342733714665155:6018]
2025-10-09T21:50:34.098617Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:34.099000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:34.099015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:34.099054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342733714665155:6018], now have 1 active actors on pipe
2025-10-09T21:50:34.099231Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:34.099246Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:34.099264Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342733714665155:6018]
2025-10-09T21:50:34.099269Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:34.099284Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:34.099311Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1867 reason: error at write: Disconnected sessionId: src_id|cb48b5d5-a3f6df7f-a1fa0b4e-5c6dc9ea_0
2025-10-09T21:50:34.099471Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1867 sessionId: src_id|cb48b5d5-a3f6df7f-a1fa0b4e-5c6dc9ea_0 is DEAD
2025-10-09T21:50:34.099642Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb48b5d5-a3f6df7f-a1fa0b4e-5c6dc9ea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:34.099655Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb48b5d5-a3f6df7f-a1fa0b4e-5c6dc9ea_0] Write session will restart in 0.008376s
2025-10-09T21:50:34.099684Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb48b5d5-a3f6df7f-a1fa0b4e-5c6dc9ea_0] Write session: Do CDS request
2025-10-09T21:50:34.099690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb48b5d5-a3f6df7f-a1fa0b4e-5c6dc9ea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:34.109617Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cb48b5d5-a3f6df7f-a1fa0b4e-5c6dc9ea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:34.109697Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:34.109716Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1868
2025-10-09T21:50:34.109859Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1868 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:34.109914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1868 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:34.109925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1868 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:34.109931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:34.110381Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1868 sessionId:  describe result for acl check
2025-10-09T21:50:34.110421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:34.110428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:34.110432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:34.110438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342733714665301:5482] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:34.110987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342733714665301:5482] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:34.112358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:34.112862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342733714665301:5482] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:34.114587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:34.114903Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:34.114914Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:34.114926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:34.114986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:34.115031Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342733352833220:10246]
2025-10-09T21:50:34.115284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342733714665301:5482] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:34.115300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342733714665301:5482] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:34.115305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342733714665301:5482] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:34.115313Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1868 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:34.115626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:34.115936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:34.115983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:34.116248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:34.116295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.116301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.116303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.116304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.116305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.116306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.116307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.116308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.116309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.116314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.116325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.116329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.116333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.116337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.116341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.116345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.116349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.116353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.116360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.116368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.116375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.116386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.116390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.116395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.116400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.116406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.116410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.116416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.116421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.116429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.116434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.116437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.116440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.116443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.116446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.116449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.116465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:34.122742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:34.123934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:34.123949Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:34.123951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:34.123956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:34.123963Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1867 [2:7559342733352833220:10246]
2025-10-09T21:50:34.123976Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:34.123999Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:34.124007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:34.124011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:34.124013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:34.124029Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:34.124039Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:34.124051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:34.124109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.124117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.124119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.124122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:34.124191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:34.124200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.124212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.124225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.124242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:34.124266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342733714665157:2432], now have 1 active actors on pipe
2025-10-09T21:50:34.124274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342733714665351:5482], now have 1 active actors on pipe
2025-10-09T21:50:34.124283Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342733352833215:10246]
2025-10-09T21:50:34.124286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342733352833215:10246]
2025-10-09T21:50:34.124288Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342733352833215:10246]
2025-10-09T21:50:34.124301Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:34.124317Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:34.124320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:34.124325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:34.124328Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:34.124334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:34.124337Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:34.124316Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1867
2025-10-09T21:50:34.124334Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1867
2025-10-09T21:50:34.124381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:34.124391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:34.124403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342733352833215:10246]
2025-10-09T21:50:34.124416Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:34.124433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:34.124451Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1fdae9d-574c52f3-d8263801-c954a052_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:34.124460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:34.124464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:34.124466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:34.124477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:34.124486Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:34.124512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:34.124655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:34.124665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:34.124681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:34.124783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1868 partition: 0 MaxSeqNo: 25 sessionId: src_id|1fdae9d-574c52f3-d8263801-c954a052_0
2025-10-09T21:50:34.125131Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cb48b5d5-a3f6df7f-a1fa0b4e-5c6dc9ea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1fdae9d-574c52f3-d8263801-c954a052_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:34.127668Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1fdae9d-574c52f3-d8263801-c954a052_0] Write session: try to update token
2025-10-09T21:50:34.127688Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1fdae9d-574c52f3-d8263801-c954a052_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:34.138703Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:34.138721Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:34.138723Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:34.138725Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:34.138739Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342733352833215:10246]
2025-10-09T21:50:34.138743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342733352833215:10246]
2025-10-09T21:50:34.138746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342733352833215:10246]
2025-10-09T21:50:34.138791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:34.138798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.138802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:34.138800Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:34.138807Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:34.138811Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:34.138934Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:34.140406Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1868 sessionId: src_id|1fdae9d-574c52f3-d8263801-c954a052_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:34.142710Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:34.199602Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342733714665366:6018]
2025-10-09T21:50:34.199842Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 1867, pipe: [1:7559342733714665366:6018]
2025-10-09T21:50:34.283410Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:34.283754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:34.283768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:34.283805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342733714665366:6018], now have 1 active actors on pipe
2025-10-09T21:50:34.283974Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:34.284049Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:34.284074Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:34.284078Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:34.284103Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342733714665366:6018]
2025-10-09T21:50:34.284145Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1868 reason: error at write: Disconnected sessionId: src_id|1fdae9d-574c52f3-d8263801-c954a052_0
2025-10-09T21:50:34.284326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1868 sessionId: src_id|1fdae9d-574c52f3-d8263801-c954a052_0 is DEAD
2025-10-09T21:50:34.284500Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fdae9d-574c52f3-d8263801-c954a052_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:34.284511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fdae9d-574c52f3-d8263801-c954a052_0] Write session will restart in 0.007467s
2025-10-09T21:50:34.284535Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fdae9d-574c52f3-d8263801-c954a052_0] Write session: Do CDS request
2025-10-09T21:50:34.284542Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fdae9d-574c52f3-d8263801-c954a052_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:34.293474Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1fdae9d-574c52f3-d8263801-c954a052_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:34.293535Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:34.293549Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1869
2025-10-09T21:50:34.293695Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1869 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:34.293727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1869 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:34.293734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1869 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:34.293737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:34.294130Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1869 sessionId:  describe result for acl check
2025-10-09T21:50:34.294169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:34.294177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:34.294178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:34.294184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342733714665499:5467] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:34.294733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342733714665499:5467] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:34.296082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:34.296597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342733714665499:5467] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:34.298094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:34.298277Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:34.298289Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:34.298301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:34.298348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:34.298387Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342733352833417:10248]
2025-10-09T21:50:34.298830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342733714665499:5467] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:34.298846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342733714665499:5467] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:34.298850Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342733714665499:5467] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:34.298856Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1869 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:34.298959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:34.299190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:34.299227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:34.299544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:34.299593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.299599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.299600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.299601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.299602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.299604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.299605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.299606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.299607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.299612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.299622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.299626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.299630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.299638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.299643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.299651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.299655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.299659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.299670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.299681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.299692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.299701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.299710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.299715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.299724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.299734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.299744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.299749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.299757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.299763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.299767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.299777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.299785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.299790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.299795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.299804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.299834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:34.305847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:34.307219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:34.307233Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:34.307235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:34.307240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:34.307247Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1868 [2:7559342733352833417:10248]
2025-10-09T21:50:34.307260Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:34.307276Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:34.307284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:34.307287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:34.307290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:34.307306Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:34.307316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:34.307324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:34.307377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.307383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.307386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.307389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:34.307453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:34.307460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.307472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.307484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.307505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:34.307529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342733714665373:2432], now have 1 active actors on pipe
2025-10-09T21:50:34.307537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342733714665568:5467], now have 1 active actors on pipe
2025-10-09T21:50:34.307544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342733352833412:10248]
2025-10-09T21:50:34.307549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342733352833412:10248]
2025-10-09T21:50:34.307552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342733352833412:10248]
2025-10-09T21:50:34.307564Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:34.307582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:34.307583Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:34.307586Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:34.307589Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:34.307600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:34.307607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:34.307564Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1868
2025-10-09T21:50:34.307590Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1868
2025-10-09T21:50:34.307645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:34.307654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:34.307671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342733352833412:10248]
2025-10-09T21:50:34.307683Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:34.307694Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:34.307711Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|56ef940f-b482d851-32577e1c-127e5ca9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:34.307723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:34.307728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:34.307731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:34.307741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:34.307756Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:34.307780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:34.307905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:34.307912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:34.307926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:34.307994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1869 partition: 0 MaxSeqNo: 25 sessionId: src_id|56ef940f-b482d851-32577e1c-127e5ca9_0
2025-10-09T21:50:34.308323Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fdae9d-574c52f3-d8263801-c954a052_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|56ef940f-b482d851-32577e1c-127e5ca9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:34.310763Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56ef940f-b482d851-32577e1c-127e5ca9_0] Write session: try to update token
2025-10-09T21:50:34.310783Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56ef940f-b482d851-32577e1c-127e5ca9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:34.320728Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:34.320745Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:34.320747Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:34.320748Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:34.320755Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342733352833412:10248]
2025-10-09T21:50:34.320759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342733352833412:10248]
2025-10-09T21:50:34.320762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342733352833412:10248]
2025-10-09T21:50:34.320796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:34.320805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.320806Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:34.320807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:34.320811Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:34.320815Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:34.320926Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:34.323153Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1869 sessionId: src_id|56ef940f-b482d851-32577e1c-127e5ca9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:34.325357Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:34.384316Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342733714665580:6018]
2025-10-09T21:50:34.384515Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 1868, pipe: [1:7559342733714665580:6018]
2025-10-09T21:50:34.461586Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:34.461978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:34.461994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:34.462043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342733714665580:6018], now have 1 active actors on pipe
2025-10-09T21:50:34.462204Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:34.462243Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:34.462290Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:34.462290Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342733714665580:6018]
2025-10-09T21:50:34.462382Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1869 reason: status is not ok: tablet will be restarted right now sessionId: src_id|56ef940f-b482d851-32577e1c-127e5ca9_0
2025-10-09T21:50:34.462557Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1869 sessionId: src_id|56ef940f-b482d851-32577e1c-127e5ca9_0 is DEAD
2025-10-09T21:50:34.462728Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56ef940f-b482d851-32577e1c-127e5ca9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:34.462738Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56ef940f-b482d851-32577e1c-127e5ca9_0] Write session will restart in 0.008283s
2025-10-09T21:50:34.462765Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56ef940f-b482d851-32577e1c-127e5ca9_0] Write session: Do CDS request
2025-10-09T21:50:34.462772Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56ef940f-b482d851-32577e1c-127e5ca9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:34.472502Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56ef940f-b482d851-32577e1c-127e5ca9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:34.472580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:34.472596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1870
2025-10-09T21:50:34.472711Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1870 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:34.472747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1870 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:34.472757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1870 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:34.472761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:34.473180Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1870 sessionId:  describe result for acl check
2025-10-09T21:50:34.473220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:34.473227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:34.473229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:34.473234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342733714665740:5468] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:34.473812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342733714665740:5468] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:34.474913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:34.475750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342733714665740:5468] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:34.476630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:34.476802Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:34.476811Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:34.476822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:34.476871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:34.476912Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342733352833615:10250]
2025-10-09T21:50:34.477509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:34.477752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:34.477795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:34.478020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342733714665740:5468] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:34.478046Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342733714665740:5468] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:34.478055Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342733714665740:5468] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:34.478061Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1870 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:34.478119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:34.478166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.478172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.478173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.478174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.478175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.478176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.478177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.478179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.478180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.478185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.478195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.478199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.478204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.478208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.478212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.478215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.478219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.478221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.478225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.478230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.478235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.478238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.478241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.478244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.478246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.478250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.478252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.478255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.478258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.478261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.478264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.478267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.478269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.478272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.478275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.478278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.478293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:34.484176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:34.485349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:34.485364Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:34.485366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:34.485368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:34.485373Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1869 [2:7559342733352833615:10250]
2025-10-09T21:50:34.485381Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:34.485394Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:34.485395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:34.485397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:34.485398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:34.485410Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:34.485414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:34.485420Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:34.485480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.485486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.485487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.485489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:34.485536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:34.485541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.485548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.485554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.485563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:34.485579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342733352833610:10250]
2025-10-09T21:50:34.485584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342733352833610:10250]
2025-10-09T21:50:34.485586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342733352833610:10250]
2025-10-09T21:50:34.485592Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:34.485601Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:34.485602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342733714665780:5468], now have 1 active actors on pipe
2025-10-09T21:50:34.485605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342733714665582:2432], now have 1 active actors on pipe
2025-10-09T21:50:34.485610Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:34.485611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:34.485611Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:34.485619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:34.485621Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:34.485639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342733352833610:10250]
2025-10-09T21:50:34.485644Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:34.485649Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:34.485674Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1869
2025-10-09T21:50:34.485699Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1869
2025-10-09T21:50:34.485733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:34.485740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:34.485767Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|66dd58a7-f07dfa4d-70f2c179-4f241c53_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:34.485779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:34.485783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:34.485788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:34.485795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:34.485809Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:34.485827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:34.485970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:34.485979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:34.485991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:34.486125Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1870 partition: 0 MaxSeqNo: 25 sessionId: src_id|66dd58a7-f07dfa4d-70f2c179-4f241c53_0
2025-10-09T21:50:34.486437Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56ef940f-b482d851-32577e1c-127e5ca9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|66dd58a7-f07dfa4d-70f2c179-4f241c53_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:34.488737Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66dd58a7-f07dfa4d-70f2c179-4f241c53_0] Write session: try to update token
2025-10-09T21:50:34.488750Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66dd58a7-f07dfa4d-70f2c179-4f241c53_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:34.499337Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:34.499363Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:34.499366Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:34.499368Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:34.499379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342733352833610:10250]
2025-10-09T21:50:34.499384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342733352833610:10250]
2025-10-09T21:50:34.499387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342733352833610:10250]
2025-10-09T21:50:34.499427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:34.499439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.499440Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:34.499446Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:34.499446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:34.499450Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:34.499579Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:34.500817Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1870 sessionId: src_id|66dd58a7-f07dfa4d-70f2c179-4f241c53_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:34.503369Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:34.562558Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342733714665812:6018]
2025-10-09T21:50:34.562775Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 1869, pipe: [1:7559342733714665812:6018]
2025-10-09T21:50:34.646515Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:34.646899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:34.646913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:34.646951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342733714665812:6018], now have 1 active actors on pipe
2025-10-09T21:50:34.647134Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:34.647190Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342733714665812:6018]
2025-10-09T21:50:34.647218Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:34.647221Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:34.647282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1870 reason: status is not ok: tablet will be restarted right now sessionId: src_id|66dd58a7-f07dfa4d-70f2c179-4f241c53_0
2025-10-09T21:50:34.647440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1870 sessionId: src_id|66dd58a7-f07dfa4d-70f2c179-4f241c53_0 is DEAD
2025-10-09T21:50:34.647603Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66dd58a7-f07dfa4d-70f2c179-4f241c53_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:34.647611Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66dd58a7-f07dfa4d-70f2c179-4f241c53_0] Write session will restart in 0.005511s
2025-10-09T21:50:34.647645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66dd58a7-f07dfa4d-70f2c179-4f241c53_0] Write session: Do CDS request
2025-10-09T21:50:34.647651Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66dd58a7-f07dfa4d-70f2c179-4f241c53_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:34.654405Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66dd58a7-f07dfa4d-70f2c179-4f241c53_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:34.654453Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:34.654467Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1871
2025-10-09T21:50:34.654576Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1871 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:34.654606Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1871 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:34.654613Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1871 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:34.654617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:34.654905Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1871 sessionId:  describe result for acl check
2025-10-09T21:50:34.654930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:34.654931Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:34.654932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:34.654935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342733714665856:5455] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:34.655353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342733714665856:5455] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:34.657537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342733714665856:5455] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:34.659622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:34.661323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:34.661359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342733714665856:5455] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:34.661373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342733714665856:5455] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:34.661376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342733714665856:5455] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:34.661383Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1871 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:34.661494Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:34.661504Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:34.661515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:34.661561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:34.661590Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342733352833852:10264]
2025-10-09T21:50:34.661918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:34.662154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:34.662184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:34.662427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:34.662465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.662470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.662470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.662471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.662471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.662472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.662473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.662473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.662474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.662477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.662483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.662485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.662487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.662488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.662490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.662492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.662494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.662496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.662500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.662505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.662509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.662512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.662514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.662517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.662519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.662522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.662524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.662527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.662529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.662533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.662536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.662539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.662541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.662544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.662546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.662549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.662563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:34.668224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:34.669399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:34.669412Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:34.669414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:34.669418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:34.669424Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1870 [2:7559342733352833852:10264]
2025-10-09T21:50:34.669435Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:34.669450Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:34.669467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:34.669470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:34.669472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:34.669488Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:34.669493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:34.669500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:34.669552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.669559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.669561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.669564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:34.669624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:34.669630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.669642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.669657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.669675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:34.669700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342733714665814:2432], now have 1 active actors on pipe
2025-10-09T21:50:34.669708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342733714666011:5455], now have 1 active actors on pipe
2025-10-09T21:50:34.669714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342733352833847:10264]
2025-10-09T21:50:34.669716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342733352833847:10264]
2025-10-09T21:50:34.669718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342733352833847:10264]
2025-10-09T21:50:34.669730Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:34.669744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:34.669747Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:34.669750Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:34.669752Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:34.669759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:34.669761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:34.669781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342733352833847:10264]
2025-10-09T21:50:34.669793Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:34.669799Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:34.669780Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1870
2025-10-09T21:50:34.669836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:34.669845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:34.669870Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fbc89a06-fe17dc98-c0d1977b-832131af_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:34.669867Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1870
2025-10-09T21:50:34.669880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:34.669884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:34.669887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:34.669895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:34.669908Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:34.669924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:34.670059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:34.670066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:34.670076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:34.670145Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1871 partition: 0 MaxSeqNo: 25 sessionId: src_id|fbc89a06-fe17dc98-c0d1977b-832131af_0
2025-10-09T21:50:34.670443Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66dd58a7-f07dfa4d-70f2c179-4f241c53_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fbc89a06-fe17dc98-c0d1977b-832131af_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:34.673187Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fbc89a06-fe17dc98-c0d1977b-832131af_0] Write session: try to update token
2025-10-09T21:50:34.673206Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fbc89a06-fe17dc98-c0d1977b-832131af_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:34.680123Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:34.680138Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:34.680145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342733352833847:10264]
2025-10-09T21:50:34.680183Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:34.680195Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:34.680330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:34.680456Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:34.680574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:34.680696Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:34.680867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:34.680908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:34.686109Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1871 sessionId: src_id|fbc89a06-fe17dc98-c0d1977b-832131af_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:34.688244Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:34.688420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:34.688436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:34.688467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:34.688517Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:34.692230Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:34.692244Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:34.692246Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:34.692248Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:34.692258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342733352833847:10264]
2025-10-09T21:50:34.692262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342733352833847:10264]
2025-10-09T21:50:34.692265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342733352833847:10264]
2025-10-09T21:50:34.692295Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:34.692300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:34.692306Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:34.692308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.692311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:34.692318Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:34.692432Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:34.747392Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342733714666024:6018]
2025-10-09T21:50:34.747664Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 1870, pipe: [1:7559342733714666024:6018]
2025-10-09T21:50:34.840312Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:34.846762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:34.846784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:34.846926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:34.846980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:34.847033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:34.847087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:34.847129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:34.847137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:34.847194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:34.847249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:34.847319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:34.847389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:34.847427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:34.847433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:34.847486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:34.847535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:34.847593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:34.847706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:34.847744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:34.847752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:34.847828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:34.847880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:34.847930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:34.847989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:34.848033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:34.848039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:34.848093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:34.848144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:34.848195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:34.848246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:34.848282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:34.848289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:34.848340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:34.848390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:34.848443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:34.848493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:34.848530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:34.848536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:34.848585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:34.848631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:34.848682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:34.848744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:34.848782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:34.848791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:34.848840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:34.848892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:34.848940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:34.848991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:34.849027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:34.849033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:34.849081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:34.849130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:34.849179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:34.849227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:34.849262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:34.849267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:34.849313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:34.849360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:34.849411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:34.849473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:34.849513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:34.849518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:34.849551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342733714666024:6018], now have 1 active actors on pipe
2025-10-09T21:50:34.849752Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:34.849763Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342733714666024:6018]
2025-10-09T21:50:34.849786Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:34.849788Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:34.849813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1871 reason: error at write: Disconnected sessionId: src_id|fbc89a06-fe17dc98-c0d1977b-832131af_0
2025-10-09T21:50:34.849998Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1871 sessionId: src_id|fbc89a06-fe17dc98-c0d1977b-832131af_0 is DEAD
2025-10-09T21:50:34.850180Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fbc89a06-fe17dc98-c0d1977b-832131af_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:34.850193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fbc89a06-fe17dc98-c0d1977b-832131af_0] Write session will restart in 0.005034s
2025-10-09T21:50:34.850194Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:34.850219Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fbc89a06-fe17dc98-c0d1977b-832131af_0] Write session: Do CDS request
2025-10-09T21:50:34.850237Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fbc89a06-fe17dc98-c0d1977b-832131af_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:34.856529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fbc89a06-fe17dc98-c0d1977b-832131af_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:34.856589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:34.856602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1872
2025-10-09T21:50:34.856759Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1872 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:34.856785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1872 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:34.856789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1872 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:34.856792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:34.857094Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1872 sessionId:  describe result for acl check
2025-10-09T21:50:34.857120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:34.857126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:34.857127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:34.857131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342733714666060:5447] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:34.857637Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342733714666060:5447] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:34.860526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342733714666060:5447] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:34.862668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:34.864586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:34.864741Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:34.864749Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:34.864757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:34.864743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342733714666060:5447] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:34.864759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342733714666060:5447] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:34.864763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342733714666060:5447] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:34.864782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1872 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:34.864797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:34.864828Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342733352834057:10268]
2025-10-09T21:50:34.865251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:34.865471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:34.865502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:34.865710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:34.865754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.865760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.865761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.865763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.865764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.865765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.865766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.865766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.865767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.865770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.865776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.865779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.865781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.865783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.865785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.865787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.865789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.865791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.865796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.865801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:34.865804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.865808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:34.865810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.865813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:34.865815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.865818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:34.865821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.865823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:34.865826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.865829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:34.865832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.865834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.865837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.865840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.865843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.865845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.865867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:34.871902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:34.872917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:34.872932Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:34.872933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:34.872936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:34.872941Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1871 [2:7559342733352834057:10268]
2025-10-09T21:50:34.872949Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:34.872963Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:34.872965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:34.872967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:34.872968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:34.872979Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:34.872989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:34.872994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:34.873034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.873040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.873041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.873043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:34.873101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:34.873108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.873118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:34.873134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:34.873146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:34.873164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342733714666228:5447], now have 1 active actors on pipe
2025-10-09T21:50:34.873171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342733714666029:2432], now have 1 active actors on pipe
2025-10-09T21:50:34.873177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342733352834052:10268]
2025-10-09T21:50:34.873178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342733352834052:10268]
2025-10-09T21:50:34.873179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342733352834052:10268]
2025-10-09T21:50:34.873188Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:34.873199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:34.873207Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:34.873209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:34.873202Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1871
2025-10-09T21:50:34.873229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342733352834052:10268]
2025-10-09T21:50:34.873203Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1871
2025-10-09T21:50:34.873230Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:34.873246Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:34.873257Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:34.873259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:34.873262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:34.873267Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:34.873267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:34.873289Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5f532040-fee6307c-7f43883f-4b463417_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:34.873298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:34.873301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:34.873303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:34.873309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:34.873317Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:34.873327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:34.873447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:34.873462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:34.873476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:34.873581Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1872 partition: 0 MaxSeqNo: 25 sessionId: src_id|5f532040-fee6307c-7f43883f-4b463417_0
2025-10-09T21:50:34.873901Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fbc89a06-fe17dc98-c0d1977b-832131af_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5f532040-fee6307c-7f43883f-4b463417_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:34.876339Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f532040-fee6307c-7f43883f-4b463417_0] Write session: try to update token
2025-10-09T21:50:34.876358Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f532040-fee6307c-7f43883f-4b463417_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:34.886680Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:34.886703Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:34.886705Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:34.886707Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:34.886717Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342733352834052:10268]
2025-10-09T21:50:34.886721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342733352834052:10268]
2025-10-09T21:50:34.886725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342733352834052:10268]
2025-10-09T21:50:34.886746Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:34.886750Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:34.886753Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:34.886762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:34.886768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:34.886770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:34.886896Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:34.889900Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1872 sessionId: src_id|5f532040-fee6307c-7f43883f-4b463417_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:34.892360Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:34.950123Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342733714666242:6018]
2025-10-09T21:50:34.950403Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 1871, pipe: [1:7559342733714666242:6018]
2025-10-09T21:50:35.037965Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:35.038686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:35.038711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:35.038762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342733714666242:6018], now have 1 active actors on pipe
2025-10-09T21:50:35.038944Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:35.038988Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:35.039004Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:35.039052Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342733714666242:6018]
2025-10-09T21:50:35.039095Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1872 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5f532040-fee6307c-7f43883f-4b463417_0
2025-10-09T21:50:35.039287Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1872 sessionId: src_id|5f532040-fee6307c-7f43883f-4b463417_0 is DEAD
2025-10-09T21:50:35.039461Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f532040-fee6307c-7f43883f-4b463417_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:35.039472Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f532040-fee6307c-7f43883f-4b463417_0] Write session will restart in 0.007498s
2025-10-09T21:50:35.039510Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f532040-fee6307c-7f43883f-4b463417_0] Write session: Do CDS request
2025-10-09T21:50:35.039518Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f532040-fee6307c-7f43883f-4b463417_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:35.048581Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f532040-fee6307c-7f43883f-4b463417_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:35.048678Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:35.048702Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1873
2025-10-09T21:50:35.048803Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1873 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:35.048829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1873 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:35.048843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1873 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:35.048846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:35.049307Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1873 sessionId:  describe result for acl check
2025-10-09T21:50:35.049345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:35.049352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:35.049354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:35.049360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342738009633630:5442] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:35.049941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342738009633630:5442] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:35.052091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342738009633630:5442] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:35.052980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:35.055205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:35.055409Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:35.055420Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:35.055432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:35.055489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342738009633630:5442] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:35.055511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342738009633630:5442] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:35.055492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:35.055514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342738009633630:5442] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:35.055521Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1873 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:35.055539Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342737647801555:10270]
2025-10-09T21:50:35.056136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:35.056429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:35.056491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:35.056858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:35.056907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.056914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.056915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.056916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.056918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.056919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.056920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.056921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.056923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.056928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.056938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.056942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.056946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.056950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.056954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.056958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.056962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.056965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.056972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.056980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.056986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.056996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.057004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.057014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.057024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.057034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.057043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.057052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.057060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.057066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.057071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.057076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.057081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.057091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.057100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.057105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.057146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:35.063428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:35.064439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:35.064452Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:35.064453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:35.064457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:35.064462Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1872 [2:7559342737647801555:10270]
2025-10-09T21:50:35.064472Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:35.064487Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:35.064491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:35.064494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:35.064496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:35.064510Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:35.064513Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:35.064519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:35.064562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.064569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.064570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.064572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:35.064626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:35.064632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.064639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.064646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.064659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:35.064676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342737647801550:10270]
2025-10-09T21:50:35.064682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342737647801550:10270]
2025-10-09T21:50:35.064683Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342737647801550:10270]
2025-10-09T21:50:35.064690Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:35.064702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:35.064701Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:35.064705Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:35.064710Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:35.064711Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:35.064712Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:35.064721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342738009633739:5442], now have 1 active actors on pipe
2025-10-09T21:50:35.064723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342738009633541:2432], now have 1 active actors on pipe
2025-10-09T21:50:35.064748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342737647801550:10270]
2025-10-09T21:50:35.064758Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:35.064767Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:35.064790Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1872
2025-10-09T21:50:35.064791Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1872
2025-10-09T21:50:35.064866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:35.064879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:35.064910Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d2afd539-14c37448-6f1a95a4-eaef1f3b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:35.064924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:35.064931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:35.064933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:35.064940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:35.064948Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:35.064963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:35.065102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:35.065109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:35.065120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:35.065221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1873 partition: 0 MaxSeqNo: 25 sessionId: src_id|d2afd539-14c37448-6f1a95a4-eaef1f3b_0
2025-10-09T21:50:35.065579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f532040-fee6307c-7f43883f-4b463417_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d2afd539-14c37448-6f1a95a4-eaef1f3b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:35.067902Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d2afd539-14c37448-6f1a95a4-eaef1f3b_0] Write session: try to update token
2025-10-09T21:50:35.067918Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d2afd539-14c37448-6f1a95a4-eaef1f3b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:35.079098Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:35.079118Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:35.079121Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:35.079123Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:35.079133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342737647801550:10270]
2025-10-09T21:50:35.079138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342737647801550:10270]
2025-10-09T21:50:35.079142Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342737647801550:10270]
2025-10-09T21:50:35.079184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:35.079191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.079194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:35.079196Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:35.079215Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:35.079224Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:35.079330Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:35.080573Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1873 sessionId: src_id|d2afd539-14c37448-6f1a95a4-eaef1f3b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:35.083054Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:35.139097Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342738009633750:6018]
2025-10-09T21:50:35.139412Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 1872, pipe: [1:7559342738009633750:6018]
2025-10-09T21:50:35.234097Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:35.234339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:35.234352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:35.234393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342738009633750:6018], now have 1 active actors on pipe
2025-10-09T21:50:35.234612Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:35.234610Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:35.234614Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:35.234617Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:35.234681Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1873 reason: error at write: Disconnected sessionId: src_id|d2afd539-14c37448-6f1a95a4-eaef1f3b_0
2025-10-09T21:50:35.234683Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342738009633750:6018]
2025-10-09T21:50:35.234841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1873 sessionId: src_id|d2afd539-14c37448-6f1a95a4-eaef1f3b_0 is DEAD
2025-10-09T21:50:35.234990Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d2afd539-14c37448-6f1a95a4-eaef1f3b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:35.235003Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d2afd539-14c37448-6f1a95a4-eaef1f3b_0] Write session will restart in 0.008317s
2025-10-09T21:50:35.235045Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d2afd539-14c37448-6f1a95a4-eaef1f3b_0] Write session: Do CDS request
2025-10-09T21:50:35.235052Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d2afd539-14c37448-6f1a95a4-eaef1f3b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:35.244535Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d2afd539-14c37448-6f1a95a4-eaef1f3b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:35.244612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:35.244628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1874
2025-10-09T21:50:35.244764Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1874 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:35.244799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1874 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:35.244811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1874 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:35.244815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:35.245347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1874 sessionId:  describe result for acl check
2025-10-09T21:50:35.245383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:35.245389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:35.245391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:35.245395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342738009633866:5437] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:35.245890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342738009633866:5437] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:35.247819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342738009633866:5437] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:35.248041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:35.250051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:35.250235Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:35.250244Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:35.250255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:35.250278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342738009633866:5437] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:35.250290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342738009633866:5437] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:35.250292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342738009633866:5437] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:35.250298Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1874 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:35.250303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:35.250333Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342737647801757:10272]
2025-10-09T21:50:35.250714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:35.250989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:35.251028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:35.251301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:35.251341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.251345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.251346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.251347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.251348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.251348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.251349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.251349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.251350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.251353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.251361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.251366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.251370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.251373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.251376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.251380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.251383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.251386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.251392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.251400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.251405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.251411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.251421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.251425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.251434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.251439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.251449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.251455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.251464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.251469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.251474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.251481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.251486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.251494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.251498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.251503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.251532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:35.257791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:35.258850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:35.258871Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:35.258874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:35.258878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:35.258884Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1873 [2:7559342737647801757:10272]
2025-10-09T21:50:35.258896Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:35.258914Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:35.258922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:35.258925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:35.258927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:35.258944Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:35.258955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:35.258962Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:35.259022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.259029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.259032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.259034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:35.259100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:35.259108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.259120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.259131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.259151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:35.259177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342738009633956:5437], now have 1 active actors on pipe
2025-10-09T21:50:35.259185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342738009633756:2432], now have 1 active actors on pipe
2025-10-09T21:50:35.259193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342737647801752:10272]
2025-10-09T21:50:35.259200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342737647801752:10272]
2025-10-09T21:50:35.259202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342737647801752:10272]
2025-10-09T21:50:35.259216Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:35.259234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:35.259233Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:35.259237Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:35.259239Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:35.259234Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1873
2025-10-09T21:50:35.259253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:35.259235Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1873
2025-10-09T21:50:35.259261Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:35.259287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342737647801752:10272]
2025-10-09T21:50:35.259302Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:35.259319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:35.259324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:35.259334Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:35.259371Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|26f6a258-ec8788e3-abe719ec-f3f81b9e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:35.259382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:35.259387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:35.259394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:35.259404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:35.259414Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:35.259440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:35.259579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:35.259587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:35.259602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:35.259688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1874 partition: 0 MaxSeqNo: 25 sessionId: src_id|26f6a258-ec8788e3-abe719ec-f3f81b9e_0
2025-10-09T21:50:35.259997Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d2afd539-14c37448-6f1a95a4-eaef1f3b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|26f6a258-ec8788e3-abe719ec-f3f81b9e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:35.262311Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26f6a258-ec8788e3-abe719ec-f3f81b9e_0] Write session: try to update token
2025-10-09T21:50:35.262327Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26f6a258-ec8788e3-abe719ec-f3f81b9e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:35.273435Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:35.273468Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:35.273471Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:35.273473Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:35.273483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342737647801752:10272]
2025-10-09T21:50:35.273490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342737647801752:10272]
2025-10-09T21:50:35.273494Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342737647801752:10272]
2025-10-09T21:50:35.273541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:35.273546Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:35.273552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.273555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:35.273558Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:35.273563Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:35.273694Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:35.275110Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1874 sessionId: src_id|26f6a258-ec8788e3-abe719ec-f3f81b9e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:35.277414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:35.334642Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342738009633972:6018]
2025-10-09T21:50:35.334921Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 1873, pipe: [1:7559342738009633972:6018]
2025-10-09T21:50:35.420668Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:35.421200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:35.421224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:35.421264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342738009633972:6018], now have 1 active actors on pipe
2025-10-09T21:50:35.421457Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:35.421468Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342738009633972:6018]
2025-10-09T21:50:35.421477Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:35.421497Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:35.421525Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1874 reason: error at write: Disconnected sessionId: src_id|26f6a258-ec8788e3-abe719ec-f3f81b9e_0
2025-10-09T21:50:35.421639Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:35.421682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1874 sessionId: src_id|26f6a258-ec8788e3-abe719ec-f3f81b9e_0 is DEAD
2025-10-09T21:50:35.421848Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26f6a258-ec8788e3-abe719ec-f3f81b9e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:35.421860Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26f6a258-ec8788e3-abe719ec-f3f81b9e_0] Write session will restart in 0.008119s
2025-10-09T21:50:35.421887Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26f6a258-ec8788e3-abe719ec-f3f81b9e_0] Write session: Do CDS request
2025-10-09T21:50:35.421894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26f6a258-ec8788e3-abe719ec-f3f81b9e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:35.431517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|26f6a258-ec8788e3-abe719ec-f3f81b9e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:35.431605Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:35.431630Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1875
2025-10-09T21:50:35.431748Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1875 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:35.431774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1875 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:35.431778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1875 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:35.431780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:35.432204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1875 sessionId:  describe result for acl check
2025-10-09T21:50:35.432241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:35.432247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:35.432248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:35.432253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342738009634114:5428] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:35.432698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342738009634114:5428] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:35.434074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:35.434658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342738009634114:5428] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:35.436242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:35.436395Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:35.436406Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:35.436414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:35.436454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:35.436453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342738009634114:5428] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:35.436467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342738009634114:5428] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:35.436470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342738009634114:5428] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:35.436484Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342737647801960:10274]
2025-10-09T21:50:35.436476Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1875 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:35.436826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:35.437049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:35.437080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:35.437297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:35.437332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.437337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.437338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.437339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.437339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.437340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.437340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.437341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.437341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.437344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.437350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.437352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.437360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.437363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.437365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.437367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.437369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.437371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.437375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.437380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.437385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.437388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.437390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.437393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.437395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.437398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.437401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.437404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.437407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.437409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.437413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.437416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.437418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.437421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.437423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.437426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.437441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:35.443756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:35.444789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:35.444808Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:35.444811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:35.444816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:35.444822Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1874 [2:7559342737647801960:10274]
2025-10-09T21:50:35.444836Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:35.444854Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:35.444862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:35.444865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:35.444868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:35.444885Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:35.444896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:35.444904Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:35.444965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.444971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.444974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.444977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:35.445046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:35.445053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.445063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.445080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.445101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:35.445128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342738009633975:2432], now have 1 active actors on pipe
2025-10-09T21:50:35.445137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342738009634175:5428], now have 1 active actors on pipe
2025-10-09T21:50:35.445146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342737647801955:10274]
2025-10-09T21:50:35.445148Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342737647801955:10274]
2025-10-09T21:50:35.445150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342737647801955:10274]
2025-10-09T21:50:35.445163Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:35.445159Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1874
2025-10-09T21:50:35.445184Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1874
2025-10-09T21:50:35.445184Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:35.445185Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:35.445188Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:35.445190Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:35.445203Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:35.445206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:35.445250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:35.445260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:35.445277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342737647801955:10274]
2025-10-09T21:50:35.445289Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:35.445301Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:35.445324Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|aa5d5c99-3f6819e5-878faa08-37267a86_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:35.445336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:35.445344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:35.445347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:35.445356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:35.445369Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:35.445393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:35.445583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:35.445594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:35.445609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:35.445706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1875 partition: 0 MaxSeqNo: 25 sessionId: src_id|aa5d5c99-3f6819e5-878faa08-37267a86_0
2025-10-09T21:50:35.446055Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|26f6a258-ec8788e3-abe719ec-f3f81b9e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|aa5d5c99-3f6819e5-878faa08-37267a86_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:35.448423Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aa5d5c99-3f6819e5-878faa08-37267a86_0] Write session: try to update token
2025-10-09T21:50:35.448443Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aa5d5c99-3f6819e5-878faa08-37267a86_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:35.460492Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:35.460520Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:35.460525Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:35.460527Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:35.460540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342737647801955:10274]
2025-10-09T21:50:35.460548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342737647801955:10274]
2025-10-09T21:50:35.460555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342737647801955:10274]
2025-10-09T21:50:35.460610Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:35.460611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:35.460621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.460623Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:35.460624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:35.460627Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:35.460882Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:35.461872Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1875 sessionId: src_id|aa5d5c99-3f6819e5-878faa08-37267a86_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:35.464022Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:35.521518Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342738009634188:6018]
2025-10-09T21:50:35.521795Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 1874, pipe: [1:7559342738009634188:6018]
2025-10-09T21:50:35.602378Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:35.602752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:35.602773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:35.602824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342738009634188:6018], now have 1 active actors on pipe
2025-10-09T21:50:35.603046Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342738009634188:6018]
2025-10-09T21:50:35.603066Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:35.603082Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:35.603085Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:35.603151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1875 reason: error at write: Disconnected sessionId: src_id|aa5d5c99-3f6819e5-878faa08-37267a86_0
2025-10-09T21:50:35.603336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1875 sessionId: src_id|aa5d5c99-3f6819e5-878faa08-37267a86_0 is DEAD
2025-10-09T21:50:35.603474Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa5d5c99-3f6819e5-878faa08-37267a86_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:35.603486Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa5d5c99-3f6819e5-878faa08-37267a86_0] Write session will restart in 0.009757s
2025-10-09T21:50:35.603514Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa5d5c99-3f6819e5-878faa08-37267a86_0] Write session: Do CDS request
2025-10-09T21:50:35.603520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa5d5c99-3f6819e5-878faa08-37267a86_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:35.603522Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:35.614598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aa5d5c99-3f6819e5-878faa08-37267a86_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:35.614680Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:35.614706Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1876
2025-10-09T21:50:35.614810Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1876 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:35.614847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1876 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:35.614858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1876 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:35.614862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:35.615316Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1876 sessionId:  describe result for acl check
2025-10-09T21:50:35.615352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:35.615357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:35.615358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:35.615364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342738009634351:5403] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:35.615915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342738009634351:5403] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:35.617393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:35.618288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342738009634351:5403] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:35.619960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:35.620143Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:35.620154Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:35.620165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:35.620379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:35.620425Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342737647802202:10288]
2025-10-09T21:50:35.620893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:35.620987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342738009634351:5403] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:35.621001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342738009634351:5403] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:35.621005Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342738009634351:5403] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:35.621013Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1876 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:35.621155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:35.621200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:35.621516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:35.621555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.621560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.621561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.621561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.621562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.621563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.621563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.621564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.621564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.621567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.621575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.621577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.621579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.621581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.621583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.621585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.621588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.621590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.621594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.621599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.621603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.621606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.621609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.621612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.621614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.621617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.621621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.621624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.621627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.621630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.621633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.621636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.621639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.621642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.621644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.621647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.621672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:35.627909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:35.628885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:35.628900Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:35.628902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:35.628907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:35.628915Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1875 [2:7559342737647802202:10288]
2025-10-09T21:50:35.628929Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:35.628944Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:35.628953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:35.628956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:35.628959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:35.628974Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:35.628985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:35.628993Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:35.629046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.629054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.629056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.629059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:35.629125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:35.629132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.629145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.629157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.629177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:35.629204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342738009634415:5403], now have 1 active actors on pipe
2025-10-09T21:50:35.629211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342738009634214:2432], now have 1 active actors on pipe
2025-10-09T21:50:35.629218Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342737647802197:10288]
2025-10-09T21:50:35.629221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342737647802197:10288]
2025-10-09T21:50:35.629223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342737647802197:10288]
2025-10-09T21:50:35.629235Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:35.629249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:35.629235Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1875
2025-10-09T21:50:35.629255Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:35.629259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:35.629261Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:35.629256Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1875
2025-10-09T21:50:35.629268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:35.629271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:35.629308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:35.629313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:35.629324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342737647802197:10288]
2025-10-09T21:50:35.629333Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:35.629338Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:35.629363Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|60ea9f4b-e246d647-99c0ec30-6817e362_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:35.629370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:35.629375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:35.629377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:35.629389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:35.629404Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:35.629425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:35.629550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:35.629559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:35.629572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:35.629643Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1876 partition: 0 MaxSeqNo: 25 sessionId: src_id|60ea9f4b-e246d647-99c0ec30-6817e362_0
2025-10-09T21:50:35.629984Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aa5d5c99-3f6819e5-878faa08-37267a86_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|60ea9f4b-e246d647-99c0ec30-6817e362_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:35.632437Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|60ea9f4b-e246d647-99c0ec30-6817e362_0] Write session: try to update token
2025-10-09T21:50:35.632456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|60ea9f4b-e246d647-99c0ec30-6817e362_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:35.643473Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:35.643493Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:35.643496Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:35.643498Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:35.643508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342737647802197:10288]
2025-10-09T21:50:35.643512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342737647802197:10288]
2025-10-09T21:50:35.643517Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342737647802197:10288]
2025-10-09T21:50:35.643559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:35.643559Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:35.643564Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:35.643571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.643574Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:35.643575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:35.643710Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:35.645036Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1876 sessionId: src_id|60ea9f4b-e246d647-99c0ec30-6817e362_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:35.647548Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:35.703297Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342738009634425:6018]
2025-10-09T21:50:35.703577Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 1875, pipe: [1:7559342738009634425:6018]
2025-10-09T21:50:35.790242Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:35.790590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:35.790603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:35.790642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342738009634425:6018], now have 1 active actors on pipe
2025-10-09T21:50:35.790821Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:35.790899Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:35.790913Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:35.790916Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:35.790965Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342738009634425:6018]
2025-10-09T21:50:35.790976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1876 reason: error at write: Disconnected sessionId: src_id|60ea9f4b-e246d647-99c0ec30-6817e362_0
2025-10-09T21:50:35.791125Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1876 sessionId: src_id|60ea9f4b-e246d647-99c0ec30-6817e362_0 is DEAD
2025-10-09T21:50:35.791310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60ea9f4b-e246d647-99c0ec30-6817e362_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:35.791322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60ea9f4b-e246d647-99c0ec30-6817e362_0] Write session will restart in 0.005846s
2025-10-09T21:50:35.791362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60ea9f4b-e246d647-99c0ec30-6817e362_0] Write session: Do CDS request
2025-10-09T21:50:35.791369Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60ea9f4b-e246d647-99c0ec30-6817e362_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:35.798541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|60ea9f4b-e246d647-99c0ec30-6817e362_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:35.798622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:35.798642Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1877
2025-10-09T21:50:35.798780Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1877 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:35.798816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1877 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:35.798827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1877 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:35.798833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:35.799347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1877 sessionId:  describe result for acl check
2025-10-09T21:50:35.799390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:35.799398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:35.799400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:35.799407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342738009634462:5401] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:35.799953Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342738009634462:5401] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:35.802565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342738009634462:5401] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:35.805323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342738009634462:5401] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:35.805336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342738009634462:5401] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:35.805340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342738009634462:5401] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:35.805349Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1877 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:35.805913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:35.808123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:35.808326Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:35.808337Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:35.808349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:35.808404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:35.808449Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342737647802412:10292]
2025-10-09T21:50:35.809005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:35.809293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:35.809334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:35.809635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:35.809690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.809696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.809697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.809699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.809700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.809701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.809702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.809703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.809704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.809709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.809720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.809723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.809726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.809729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.809732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.809735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.809738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.809741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.809748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.809760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:35.809769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.809774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:35.809782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.809786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:35.809791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.809799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:35.809803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.809808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:35.809812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.809816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:35.809820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.809824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.809828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.809832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.809838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.809843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.809867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:35.816402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:35.817811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:35.817831Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:35.817833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:35.817838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:35.817846Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1876 [2:7559342737647802412:10292]
2025-10-09T21:50:35.817860Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:35.817881Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:35.817884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:35.817887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:35.817889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:35.817907Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:35.817918Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:35.817925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:35.817986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.817993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.817995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.817997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:35.818091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:35.818099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.818114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:35.818125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:35.818141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:35.818169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342738009634427:2432], now have 1 active actors on pipe
2025-10-09T21:50:35.818177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342738009634628:5401], now have 1 active actors on pipe
2025-10-09T21:50:35.818185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342737647802407:10292]
2025-10-09T21:50:35.818187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342737647802407:10292]
2025-10-09T21:50:35.818189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342737647802407:10292]
2025-10-09T21:50:35.818205Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:35.818220Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:35.818229Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:35.818233Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:35.818236Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:35.818238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:35.818242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:35.818282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342737647802407:10292]
2025-10-09T21:50:35.818294Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:35.818299Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:35.818275Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1876
2025-10-09T21:50:35.818286Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1876
2025-10-09T21:50:35.818340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:35.818354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:35.818386Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7a20028e-c81dd14e-3766adc7-bbd8e091_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:35.818394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:35.818398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:35.818401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:35.818412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:35.818422Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:35.818445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:35.818640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:35.818648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:35.818661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:35.818758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1877 partition: 0 MaxSeqNo: 25 sessionId: src_id|7a20028e-c81dd14e-3766adc7-bbd8e091_0
2025-10-09T21:50:35.819143Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60ea9f4b-e246d647-99c0ec30-6817e362_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7a20028e-c81dd14e-3766adc7-bbd8e091_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:35.821771Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a20028e-c81dd14e-3766adc7-bbd8e091_0] Write session: try to update token
2025-10-09T21:50:35.821790Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a20028e-c81dd14e-3766adc7-bbd8e091_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:35.830415Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:35.830437Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:35.830445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342737647802407:10292]
2025-10-09T21:50:35.830497Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:35.830500Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:35.830673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:35.830831Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:35.830978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:35.831105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:35.831291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:35.831332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:35.835765Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1877 sessionId: src_id|7a20028e-c81dd14e-3766adc7-bbd8e091_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:35.838211Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:35.838430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:35.838454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:35.838495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:35.838591Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:35.843163Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:35.843179Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:35.843181Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:35.843182Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:35.843191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342737647802407:10292]
2025-10-09T21:50:35.843194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342737647802407:10292]
2025-10-09T21:50:35.843196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342737647802407:10292]
2025-10-09T21:50:35.843227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:35.843233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:35.843235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:35.843237Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:35.843243Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:35.843247Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:35.843358Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:35.891177Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342738009634646:6018]
2025-10-09T21:50:35.891467Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 1876, pipe: [1:7559342738009634646:6018]
2025-10-09T21:50:35.994165Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:36.001637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.001664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.001835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.001903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.001962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.002024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.002090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.002101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.002168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.002229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.002293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.002360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.002406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.002420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.002485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.002541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.002599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.002661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.002974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.002983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.003051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.003113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.003176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.003241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.003283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.003292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.003364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.003434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.003502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.003563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.003604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.003611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.003677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.003741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.003801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.003882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.003930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.003936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.003992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.004043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.004096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.004147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.004193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.004203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.004266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.004331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.004387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.004454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.004500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.004505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.004557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.004605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.004655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.004712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.004810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.004816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.004870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.004922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.004985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.005049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.005167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.005174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.005231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342738009634646:6018], now have 1 active actors on pipe
2025-10-09T21:50:36.005488Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342738009634646:6018]
2025-10-09T21:50:36.005499Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:36.005528Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:36.005531Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:36.005561Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1877 reason: error at write: Disconnected sessionId: src_id|7a20028e-c81dd14e-3766adc7-bbd8e091_0
2025-10-09T21:50:36.005685Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:36.005784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1877 sessionId: src_id|7a20028e-c81dd14e-3766adc7-bbd8e091_0 is DEAD
2025-10-09T21:50:36.005895Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a20028e-c81dd14e-3766adc7-bbd8e091_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:36.005905Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a20028e-c81dd14e-3766adc7-bbd8e091_0] Write session will restart in 0.008168s
2025-10-09T21:50:36.005943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a20028e-c81dd14e-3766adc7-bbd8e091_0] Write session: Do CDS request
2025-10-09T21:50:36.005950Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a20028e-c81dd14e-3766adc7-bbd8e091_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:36.015618Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a20028e-c81dd14e-3766adc7-bbd8e091_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:36.015654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:36.015675Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1878
2025-10-09T21:50:36.015868Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1878 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:36.015930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1878 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:36.015946Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1878 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:36.015950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:36.016517Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1878 sessionId:  describe result for acl check
2025-10-09T21:50:36.016558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:36.016572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:36.016574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:36.016581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342742304602009:5394] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:36.017182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342742304602009:5394] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:36.020430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342742304602009:5394] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:36.022406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:36.024847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:36.025074Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:36.025080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342742304602009:5394] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:36.025099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342742304602009:5394] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:36.025102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342742304602009:5394] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:36.025086Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:36.025112Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1878 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:36.025099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:36.025164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:36.025203Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342741942769913:10294]
2025-10-09T21:50:36.025799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:36.026128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:36.026185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:36.026525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:36.026576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.026581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.026582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.026583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.026584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.026584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.026585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.026586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.026586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.026590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.026597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.026600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.026602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.026604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.026606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.026609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.026611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.026613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.026618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.026623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.026628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.026631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.026634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.026637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.026640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.026643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.026645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.026648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.026651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.026654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.026657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.026660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.026662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.026665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.026668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.026671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.026688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:36.033838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:36.035288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:36.035306Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:36.035308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:36.035312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:36.035319Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1877 [2:7559342741942769913:10294]
2025-10-09T21:50:36.035333Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:36.035350Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:36.035356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:36.035359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:36.035360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:36.035376Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:36.035381Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:36.035388Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:36.035442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.035448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.035449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.035452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:36.035532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:36.035540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.035553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.035563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.035579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:36.035600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342742304602148:5394], now have 1 active actors on pipe
2025-10-09T21:50:36.035610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342742304601947:2432], now have 1 active actors on pipe
2025-10-09T21:50:36.035617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342741942769908:10294]
2025-10-09T21:50:36.035619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342741942769908:10294]
2025-10-09T21:50:36.035622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342741942769908:10294]
2025-10-09T21:50:36.035633Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:36.035654Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:36.035659Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:36.035664Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:36.035667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:36.035673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:36.035676Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:36.035710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342741942769908:10294]
2025-10-09T21:50:36.035723Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:36.035706Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1877
2025-10-09T21:50:36.035743Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:36.035711Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1877
2025-10-09T21:50:36.035764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.035775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.035807Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|23aeac73-300e9ba7-a39ea706-596096fc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:36.035818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:36.035821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:36.035824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:36.035832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:36.035840Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:36.035856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:36.036058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.036071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.036090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:36.036182Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1878 partition: 0 MaxSeqNo: 25 sessionId: src_id|23aeac73-300e9ba7-a39ea706-596096fc_0
2025-10-09T21:50:36.036560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a20028e-c81dd14e-3766adc7-bbd8e091_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|23aeac73-300e9ba7-a39ea706-596096fc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:36.039320Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23aeac73-300e9ba7-a39ea706-596096fc_0] Write session: try to update token
2025-10-09T21:50:36.039336Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23aeac73-300e9ba7-a39ea706-596096fc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:36.047709Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:36.047725Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:36.047734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342741942769908:10294]
2025-10-09T21:50:36.047776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:36.047788Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:36.047960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:36.048099Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:36.048247Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:36.048404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:36.048600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:36.048671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:36.053472Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1878 sessionId: src_id|23aeac73-300e9ba7-a39ea706-596096fc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:36.056099Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:36.056298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.056319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.056356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:36.056433Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:36.061121Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:36.061142Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:36.061145Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:36.061147Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:36.061159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342741942769908:10294]
2025-10-09T21:50:36.061164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342741942769908:10294]
2025-10-09T21:50:36.061168Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342741942769908:10294]
2025-10-09T21:50:36.061213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:36.061220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.061221Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:36.061223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:36.061228Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:36.061232Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:36.061362Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:36.105685Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342742304602159:6018]
2025-10-09T21:50:36.105982Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 1877, pipe: [1:7559342742304602159:6018]
2025-10-09T21:50:36.218473Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:36.226944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.226976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.227147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.227215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.227282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.227352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.227397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.227407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.227477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.227541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.227606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.227671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.227715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.227724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.227789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.227856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.227922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.227986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.228031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.228039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.228108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.228173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.228238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.228308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.228353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.228364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.228435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.228507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.228575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.228643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.228696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.228705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.228773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.228838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.228903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.228969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.229014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.229023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.229113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.229176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.229241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.229303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.229351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.229356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.229420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.229494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.229560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.229626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.229681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.229689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.229754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.229811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.229877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.229950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.230100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.230110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.230182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.230244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.230305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.230365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.230409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.230416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.230470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342742304602159:6018], now have 1 active actors on pipe
2025-10-09T21:50:36.230706Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:36.230725Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:36.230725Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342742304602159:6018]
2025-10-09T21:50:36.230751Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:36.230758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1878 reason: error at write: Disconnected sessionId: src_id|23aeac73-300e9ba7-a39ea706-596096fc_0
2025-10-09T21:50:36.230984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1878 sessionId: src_id|23aeac73-300e9ba7-a39ea706-596096fc_0 is DEAD
2025-10-09T21:50:36.231102Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23aeac73-300e9ba7-a39ea706-596096fc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:36.231115Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23aeac73-300e9ba7-a39ea706-596096fc_0] Write session will restart in 0.006872s
2025-10-09T21:50:36.231141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23aeac73-300e9ba7-a39ea706-596096fc_0] Write session: Do CDS request
2025-10-09T21:50:36.231149Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23aeac73-300e9ba7-a39ea706-596096fc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:36.231510Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:36.239544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23aeac73-300e9ba7-a39ea706-596096fc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:36.239673Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:36.239693Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1879
2025-10-09T21:50:36.239824Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1879 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:36.239866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1879 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:36.239877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1879 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:36.239882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:36.240379Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1879 sessionId:  describe result for acl check
2025-10-09T21:50:36.240423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:36.240430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:36.240432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:36.240439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342742304602213:5389] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:36.241045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342742304602213:5389] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:36.244201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342742304602213:5389] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:36.247278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:36.247365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342742304602213:5389] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:36.247385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342742304602213:5389] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:36.247389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342742304602213:5389] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:36.247398Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1879 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:36.249961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:36.250228Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:36.250242Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:36.250255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:36.250321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:36.250366Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342741942770123:10296]
2025-10-09T21:50:36.250956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:36.251293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:36.251341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:36.251684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:36.251728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.251733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.251733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.251734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.251735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.251736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.251737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.251737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.251738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.251742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.251751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.251755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.251759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.251764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.251768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.251771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.251775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.251780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.251787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.251795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.251801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.251806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.251811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.251816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.251820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.251825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.251830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.251835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.251839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.251844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.251851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.251856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.251860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.251872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.251877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.251882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.251917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:36.258947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:36.260047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:36.260069Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:36.260072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:36.260078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:36.260085Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1878 [2:7559342741942770123:10296]
2025-10-09T21:50:36.260102Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:36.260120Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:36.260123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:36.260126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:36.260129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:36.260150Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:36.260157Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:36.260164Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:36.260241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.260251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.260253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.260256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:36.260328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:36.260336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.260350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.260362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.260384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:36.260407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342742304602165:2432], now have 1 active actors on pipe
2025-10-09T21:50:36.260411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342742304602369:5389], now have 1 active actors on pipe
2025-10-09T21:50:36.260419Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342741942770117:10296]
2025-10-09T21:50:36.260421Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342741942770117:10296]
2025-10-09T21:50:36.260424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342741942770117:10296]
2025-10-09T21:50:36.260441Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:36.260461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:36.260467Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:36.260471Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:36.260474Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:36.260482Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:36.260486Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:36.260488Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1878
2025-10-09T21:50:36.260530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342741942770117:10296]
2025-10-09T21:50:36.260496Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1878
2025-10-09T21:50:36.260539Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:36.260549Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:36.260577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.260590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.260629Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a3b7bd00-e508e685-d43b8768-75f920a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:36.260637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:36.260641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:36.260644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:36.260654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:36.260666Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:36.260686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:36.260830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.260838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.260854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:36.260963Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1879 partition: 0 MaxSeqNo: 25 sessionId: src_id|a3b7bd00-e508e685-d43b8768-75f920a_0
2025-10-09T21:50:36.261376Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23aeac73-300e9ba7-a39ea706-596096fc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a3b7bd00-e508e685-d43b8768-75f920a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:36.263972Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a3b7bd00-e508e685-d43b8768-75f920a_0] Write session: try to update token
2025-10-09T21:50:36.263995Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a3b7bd00-e508e685-d43b8768-75f920a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:36.275646Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:36.275670Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:36.275673Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:36.275674Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:36.275685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342741942770117:10296]
2025-10-09T21:50:36.275691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342741942770117:10296]
2025-10-09T21:50:36.275695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342741942770117:10296]
2025-10-09T21:50:36.275742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:36.275749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.275752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:36.275751Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:36.275757Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:36.275761Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:36.275888Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:36.277977Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1879 sessionId: src_id|a3b7bd00-e508e685-d43b8768-75f920a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:36.280453Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:36.330989Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342742304602388:6018]
2025-10-09T21:50:36.331319Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 1878, pipe: [1:7559342742304602388:6018]
2025-10-09T21:50:36.447664Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:36.448501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.448520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.448568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342742304602388:6018], now have 1 active actors on pipe
2025-10-09T21:50:36.448783Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:36.448799Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:36.448821Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342742304602388:6018]
2025-10-09T21:50:36.448851Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:36.448876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1879 reason: error at write: Disconnected sessionId: src_id|a3b7bd00-e508e685-d43b8768-75f920a_0
2025-10-09T21:50:36.448923Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:36.449065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1879 sessionId: src_id|a3b7bd00-e508e685-d43b8768-75f920a_0 is DEAD
2025-10-09T21:50:36.449212Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3b7bd00-e508e685-d43b8768-75f920a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:36.449224Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3b7bd00-e508e685-d43b8768-75f920a_0] Write session will restart in 0.005650s
2025-10-09T21:50:36.449259Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3b7bd00-e508e685-d43b8768-75f920a_0] Write session: Do CDS request
2025-10-09T21:50:36.449266Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3b7bd00-e508e685-d43b8768-75f920a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:36.456507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a3b7bd00-e508e685-d43b8768-75f920a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:36.456609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:36.456628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1880
2025-10-09T21:50:36.456842Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1880 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:36.456875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1880 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:36.456885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1880 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:36.456889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:36.457425Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1880 sessionId:  describe result for acl check
2025-10-09T21:50:36.457476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:36.457482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:36.457484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:36.457489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342742304602423:5374] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:36.458073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342742304602423:5374] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:36.460826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342742304602423:5374] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:36.463169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342742304602423:5374] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:36.463186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342742304602423:5374] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:36.463190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342742304602423:5374] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:36.463198Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1880 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:36.463731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:36.466277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:36.466468Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:36.466479Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:36.466492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:36.466551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:36.466592Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342741942770331:10298]
2025-10-09T21:50:36.467104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:36.467379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:36.467436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:36.467726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:36.467770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.467771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.467771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.467772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.467773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.467773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.467774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.467775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.467775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.467779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.467786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.467788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.467790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.467800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.467803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.467805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.467807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.467809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.467815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.467819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.467824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.467827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.467829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.467832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.467835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.467838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.467840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.467845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.467849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.467854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.467858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.467863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.467870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.467875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.467880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.467885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.467914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:36.474400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:36.475756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:36.475771Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:36.475774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:36.475779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:36.475785Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1879 [2:7559342741942770331:10298]
2025-10-09T21:50:36.475800Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:36.475818Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:36.475825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:36.475828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:36.475831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:36.475847Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:36.475859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:36.475867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:36.475928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.475935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.475937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.475940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:36.476015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:36.476022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.476034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.476051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.476073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:36.476101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342742304602390:2432], now have 1 active actors on pipe
2025-10-09T21:50:36.476108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342742304602594:5374], now have 1 active actors on pipe
2025-10-09T21:50:36.476116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342741942770326:10298]
2025-10-09T21:50:36.476119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342741942770326:10298]
2025-10-09T21:50:36.476123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342741942770326:10298]
2025-10-09T21:50:36.476135Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:36.476154Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:36.476155Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:36.476158Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:36.476160Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:36.476168Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:36.476171Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:36.476204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342741942770326:10298]
2025-10-09T21:50:36.476212Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:36.476175Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1879
2025-10-09T21:50:36.476222Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:36.476177Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1879
2025-10-09T21:50:36.476249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.476258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.476292Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ecf72172-5dc9a27c-aec73344-77273d5e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:36.476305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:36.476313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:36.476316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:36.476325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:36.476339Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:36.476359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:36.476661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.476677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.476697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:36.476827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1880 partition: 0 MaxSeqNo: 25 sessionId: src_id|ecf72172-5dc9a27c-aec73344-77273d5e_0
2025-10-09T21:50:36.477169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a3b7bd00-e508e685-d43b8768-75f920a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ecf72172-5dc9a27c-aec73344-77273d5e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:36.479558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf72172-5dc9a27c-aec73344-77273d5e_0] Write session: try to update token
2025-10-09T21:50:36.479574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf72172-5dc9a27c-aec73344-77273d5e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:36.491071Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:36.491090Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:36.491093Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:36.491094Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:36.491104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342741942770326:10298]
2025-10-09T21:50:36.491110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342741942770326:10298]
2025-10-09T21:50:36.491113Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342741942770326:10298]
2025-10-09T21:50:36.491157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:36.491161Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:36.491164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.491167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:36.491168Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:36.491172Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:36.491303Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:36.493257Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1880 sessionId: src_id|ecf72172-5dc9a27c-aec73344-77273d5e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:36.496031Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:36.549129Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342742304602608:6018]
2025-10-09T21:50:36.549440Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 1879, pipe: [1:7559342742304602608:6018]
2025-10-09T21:50:36.648764Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:36.649222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.649240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.649284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342742304602608:6018], now have 1 active actors on pipe
2025-10-09T21:50:36.649508Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:36.649523Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:36.649533Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:36.649537Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:36.649586Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342742304602608:6018]
2025-10-09T21:50:36.649602Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1880 reason: error at write: Disconnected sessionId: src_id|ecf72172-5dc9a27c-aec73344-77273d5e_0
2025-10-09T21:50:36.649795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1880 sessionId: src_id|ecf72172-5dc9a27c-aec73344-77273d5e_0 is DEAD
2025-10-09T21:50:36.649976Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf72172-5dc9a27c-aec73344-77273d5e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:36.649988Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf72172-5dc9a27c-aec73344-77273d5e_0] Write session will restart in 0.005504s
2025-10-09T21:50:36.650024Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf72172-5dc9a27c-aec73344-77273d5e_0] Write session: Do CDS request
2025-10-09T21:50:36.650055Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf72172-5dc9a27c-aec73344-77273d5e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:36.656454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf72172-5dc9a27c-aec73344-77273d5e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:36.656551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:36.656586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1881
2025-10-09T21:50:36.656821Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1881 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:36.656848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1881 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:36.656852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1881 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:36.656855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:36.657303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1881 sessionId:  describe result for acl check
2025-10-09T21:50:36.657344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:36.657350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:36.657352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:36.657358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342742304602649:5368] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:36.657909Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342742304602649:5368] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:36.660354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342742304602649:5368] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:36.663069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342742304602649:5368] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:36.663081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342742304602649:5368] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:36.663086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342742304602649:5368] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:36.663094Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1881 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:36.665050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:36.667581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:36.667772Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:36.667783Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:36.667794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:36.667846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:36.667888Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342741942770578:10312]
2025-10-09T21:50:36.668457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:36.668760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:36.668808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:36.669117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:36.669176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.669182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.669183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.669184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.669186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.669187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.669188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.669189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.669190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.669195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.669205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.669209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.669213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.669217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.669221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.669225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.669228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.669232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.669239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.669248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.669259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.669265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.669269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.669278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.669282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.669287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.669296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.669301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.669304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.669309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.669313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.669317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.669322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.669330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.669335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.669344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.669373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:36.675842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:36.677075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:36.677092Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:36.677093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:36.677097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:36.677104Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1880 [2:7559342741942770578:10312]
2025-10-09T21:50:36.677114Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:36.677129Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:36.677130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:36.677133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:36.677134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:36.677146Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:36.677150Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:36.677156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:36.677198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.677204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.677206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.677207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:36.677260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:36.677265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.677273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.677279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.677293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:36.677313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342741942770573:10312]
2025-10-09T21:50:36.677317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342741942770573:10312]
2025-10-09T21:50:36.677319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342741942770573:10312]
2025-10-09T21:50:36.677325Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:36.677334Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:36.677337Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:36.677343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:36.677345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:36.677345Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:36.677347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:36.677356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342742304602812:5368], now have 1 active actors on pipe
2025-10-09T21:50:36.677359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342742304602634:2432], now have 1 active actors on pipe
2025-10-09T21:50:36.677381Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342741942770573:10312]
2025-10-09T21:50:36.677390Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:36.677395Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:36.677435Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1880
2025-10-09T21:50:36.677437Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1880
2025-10-09T21:50:36.677529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.677536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.677559Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cd800592-87e96ff4-75af4615-d06bb01d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:36.677567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:36.677570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:36.677572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:36.677580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:36.677591Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:36.677610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:36.677790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.677804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.677843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:36.677963Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1881 partition: 0 MaxSeqNo: 25 sessionId: src_id|cd800592-87e96ff4-75af4615-d06bb01d_0
2025-10-09T21:50:36.678349Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecf72172-5dc9a27c-aec73344-77273d5e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cd800592-87e96ff4-75af4615-d06bb01d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:36.680899Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd800592-87e96ff4-75af4615-d06bb01d_0] Write session: try to update token
2025-10-09T21:50:36.680918Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd800592-87e96ff4-75af4615-d06bb01d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:36.688731Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:36.688750Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:36.688760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342741942770573:10312]
2025-10-09T21:50:36.688805Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:36.688812Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:36.688985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:36.689152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:36.689301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:36.689438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:36.689622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:36.689657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:36.693877Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1881 sessionId: src_id|cd800592-87e96ff4-75af4615-d06bb01d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:36.696451Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:36.696665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.696686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.696724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:36.696822Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:36.701706Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:36.701723Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:36.701725Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:36.701727Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:36.701738Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342741942770573:10312]
2025-10-09T21:50:36.701742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342741942770573:10312]
2025-10-09T21:50:36.701747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342741942770573:10312]
2025-10-09T21:50:36.701790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:36.701794Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:36.701798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.701801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:36.701811Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:36.701816Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:36.701946Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:36.749800Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342742304602852:6018]
2025-10-09T21:50:36.750012Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 1880, pipe: [1:7559342742304602852:6018]
2025-10-09T21:50:36.853234Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:36.859990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.860016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.860160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.860214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.860273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.860340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.860387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.860399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.860461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.860512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.860573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.860634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.860676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.860682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.860736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.860786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.860848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.860901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.860942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.860947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.861002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.861054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.861119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.861175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.861407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.861415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.861481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.861535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.861593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.861651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.861688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.861694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.861746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.861800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.861859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.861946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.861985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.861990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.862055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.862111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.862180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.862232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.862270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.862275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.862328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.862385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.862437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.862488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.862526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.862536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.862587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.862643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.862696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.862750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.862788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.862796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.862851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:36.862907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:36.862958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:36.863011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:36.863046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:36.863052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:36.863093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342742304602852:6018], now have 1 active actors on pipe
2025-10-09T21:50:36.863355Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:36.863369Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:36.863395Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1881 reason: error at write: Disconnected sessionId: src_id|cd800592-87e96ff4-75af4615-d06bb01d_0
2025-10-09T21:50:36.863482Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:36.863511Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:36.863536Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342742304602852:6018]
2025-10-09T21:50:36.863590Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1881 sessionId: src_id|cd800592-87e96ff4-75af4615-d06bb01d_0 is DEAD
2025-10-09T21:50:36.863758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd800592-87e96ff4-75af4615-d06bb01d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:36.863766Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd800592-87e96ff4-75af4615-d06bb01d_0] Write session will restart in 0.006434s
2025-10-09T21:50:36.863784Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd800592-87e96ff4-75af4615-d06bb01d_0] Write session: Do CDS request
2025-10-09T21:50:36.863789Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd800592-87e96ff4-75af4615-d06bb01d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:36.871486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd800592-87e96ff4-75af4615-d06bb01d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:36.871601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:36.871614Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1882
2025-10-09T21:50:36.871744Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1882 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:36.871798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1882 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:36.871808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1882 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:36.871812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:36.872212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1882 sessionId:  describe result for acl check
2025-10-09T21:50:36.872246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:36.872248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:36.872249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:36.872255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342742304602896:5357] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:36.872702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342742304602896:5357] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:36.875321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342742304602896:5357] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:36.877706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342742304602896:5357] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:36.877722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342742304602896:5357] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:36.877726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342742304602896:5357] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:36.877733Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1882 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:36.878110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:36.880222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:36.880427Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:36.880435Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:36.880446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:36.880496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:36.880527Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342741942770792:10316]
2025-10-09T21:50:36.880990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:36.881264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:36.881315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:36.881631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:36.881684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.881693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.881694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.881695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.881697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.881698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.881699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.881700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.881701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.881707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.881718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.881722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.881727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.881730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.881734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.881738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.881742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.881746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.881753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.881762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:36.881769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.881774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:36.881778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.881784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:36.881788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.881792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:36.881797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.881802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:36.881806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.881812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:36.881816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.881821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.881825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.881829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.881834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.881840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.881869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:36.887972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:36.888975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:36.888991Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:36.888993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:36.888998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:36.889004Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1881 [2:7559342741942770792:10316]
2025-10-09T21:50:36.889017Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:36.889036Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:36.889039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:36.889042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:36.889045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:36.889062Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:36.889075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:36.889087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:36.889139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.889148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.889150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.889152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:36.889219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:36.889226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.889236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:36.889247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:36.889266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:36.889292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342742304603065:5357], now have 1 active actors on pipe
2025-10-09T21:50:36.889300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342742304602856:2432], now have 1 active actors on pipe
2025-10-09T21:50:36.889306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342741942770787:10316]
2025-10-09T21:50:36.889313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342741942770787:10316]
2025-10-09T21:50:36.889315Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342741942770787:10316]
2025-10-09T21:50:36.889326Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:36.889336Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:36.889344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:36.889344Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:36.889347Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:36.889357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:36.889338Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1881
2025-10-09T21:50:36.889360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:36.889363Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1881
2025-10-09T21:50:36.889388Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342741942770787:10316]
2025-10-09T21:50:36.889396Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:36.889397Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:36.889414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.889422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.889482Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|107d961-7322c5b6-8037c312-426a134e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:36.889494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:36.889502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:36.889504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:36.889515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:36.889528Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:36.889547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:36.889694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:36.889703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:36.889718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:36.889810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1882 partition: 0 MaxSeqNo: 25 sessionId: src_id|107d961-7322c5b6-8037c312-426a134e_0
2025-10-09T21:50:36.890129Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd800592-87e96ff4-75af4615-d06bb01d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|107d961-7322c5b6-8037c312-426a134e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:36.892412Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|107d961-7322c5b6-8037c312-426a134e_0] Write session: try to update token
2025-10-09T21:50:36.892424Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|107d961-7322c5b6-8037c312-426a134e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:36.903471Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:36.903490Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:36.903492Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:36.903493Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:36.903501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342741942770787:10316]
2025-10-09T21:50:36.903505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342741942770787:10316]
2025-10-09T21:50:36.903507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342741942770787:10316]
2025-10-09T21:50:36.903549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:36.903559Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:36.903563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:36.903564Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:36.903566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:36.903568Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:36.903701Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:36.904847Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1882 sessionId: src_id|107d961-7322c5b6-8037c312-426a134e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:36.907580Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:36.963842Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342742304603077:6018]
2025-10-09T21:50:36.964136Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 1881, pipe: [1:7559342742304603077:6018]
2025-10-09T21:50:37.050690Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:37.051328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:37.051344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:37.051384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342742304603077:6018], now have 1 active actors on pipe
2025-10-09T21:50:37.051516Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:37.051645Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:37.051647Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:37.051662Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:37.051724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1882 reason: error at write: Disconnected sessionId: src_id|107d961-7322c5b6-8037c312-426a134e_0
2025-10-09T21:50:37.051777Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342742304603077:6018]
2025-10-09T21:50:37.051917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1882 sessionId: src_id|107d961-7322c5b6-8037c312-426a134e_0 is DEAD
2025-10-09T21:50:37.052053Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|107d961-7322c5b6-8037c312-426a134e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:37.052063Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|107d961-7322c5b6-8037c312-426a134e_0] Write session will restart in 0.005783s
2025-10-09T21:50:37.052084Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|107d961-7322c5b6-8037c312-426a134e_0] Write session: Do CDS request
2025-10-09T21:50:37.052089Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|107d961-7322c5b6-8037c312-426a134e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:37.058430Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|107d961-7322c5b6-8037c312-426a134e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:37.058511Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:37.058525Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1883
2025-10-09T21:50:37.058631Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1883 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:37.058662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1883 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:37.058672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1883 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:37.058676Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:37.059070Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1883 sessionId:  describe result for acl check
2025-10-09T21:50:37.059112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:37.059118Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:37.059120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:37.059126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342746599570398:5356] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:37.059682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342746599570398:5356] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:37.062556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342746599570398:5356] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:37.065330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342746599570398:5356] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:37.065345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342746599570398:5356] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:37.065348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342746599570398:5356] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:37.065357Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1883 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:37.066434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:37.068359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:37.068520Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:37.068528Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:37.068545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:37.068586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:37.068617Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342746237738300:10318]
2025-10-09T21:50:37.069036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:37.069366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:37.069422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:37.069806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:37.069865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.069872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.069873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.069875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.069876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.069877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.069878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.069879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.069881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.069886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.069896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.069901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.069905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.069909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.069914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.069918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.069922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.069926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.069937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.069945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.069951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.069957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.069961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.069966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.069972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.069977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.069987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.069992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.070001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.070007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.070018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.070023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.070044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.070050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.070054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.070058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.070090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:37.076364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:37.077420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:37.077437Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:37.077440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:37.077444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:37.077463Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1882 [2:7559342746237738300:10318]
2025-10-09T21:50:37.077477Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:37.077494Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:37.077498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:37.077501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:37.077504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:37.077519Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:37.077533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:37.077540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:37.077594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.077599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.077601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.077604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:37.077672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:37.077681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.077692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.077710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.077733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:37.077757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342746599570581:5356], now have 1 active actors on pipe
2025-10-09T21:50:37.077765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342746599570376:2432], now have 1 active actors on pipe
2025-10-09T21:50:37.077774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342746237738295:10318]
2025-10-09T21:50:37.077781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342746237738295:10318]
2025-10-09T21:50:37.077784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342746237738295:10318]
2025-10-09T21:50:37.077796Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:37.077807Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:37.077811Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:37.077813Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:37.077819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:37.077834Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:37.077837Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:37.077815Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1882
2025-10-09T21:50:37.077824Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1882
2025-10-09T21:50:37.077877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:37.077887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:37.077904Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342746237738295:10318]
2025-10-09T21:50:37.077918Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:37.077939Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:37.077947Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|72f59cdd-c0918dad-d132ad7e-bb5d22fc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:37.077955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:37.077959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:37.077962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:37.077972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:37.077987Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:37.078009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:37.078193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:37.078202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:37.078217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:37.078338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1883 partition: 0 MaxSeqNo: 25 sessionId: src_id|72f59cdd-c0918dad-d132ad7e-bb5d22fc_0
2025-10-09T21:50:37.078714Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|107d961-7322c5b6-8037c312-426a134e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|72f59cdd-c0918dad-d132ad7e-bb5d22fc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:37.081128Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72f59cdd-c0918dad-d132ad7e-bb5d22fc_0] Write session: try to update token
2025-10-09T21:50:37.081148Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72f59cdd-c0918dad-d132ad7e-bb5d22fc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:37.092724Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:37.092743Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:37.092746Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:37.092748Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:37.092758Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342746237738295:10318]
2025-10-09T21:50:37.092762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342746237738295:10318]
2025-10-09T21:50:37.092767Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342746237738295:10318]
2025-10-09T21:50:37.092830Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:37.092837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:37.092845Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:37.092849Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:37.092922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.092931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:37.093080Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:37.094506Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1883 sessionId: src_id|72f59cdd-c0918dad-d132ad7e-bb5d22fc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:37.096889Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:37.151738Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342746599570598:6018]
2025-10-09T21:50:37.151979Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 1882, pipe: [1:7559342746599570598:6018]
2025-10-09T21:50:37.247052Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:37.247527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:37.247544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:37.247589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342746599570598:6018], now have 1 active actors on pipe
2025-10-09T21:50:37.247738Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:37.247832Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342746599570598:6018]
2025-10-09T21:50:37.247863Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:37.247868Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:37.247872Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:37.247926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1883 reason: error at write: Disconnected sessionId: src_id|72f59cdd-c0918dad-d132ad7e-bb5d22fc_0
2025-10-09T21:50:37.248107Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1883 sessionId: src_id|72f59cdd-c0918dad-d132ad7e-bb5d22fc_0 is DEAD
2025-10-09T21:50:37.248284Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72f59cdd-c0918dad-d132ad7e-bb5d22fc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:37.248296Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72f59cdd-c0918dad-d132ad7e-bb5d22fc_0] Write session will restart in 0.009664s
2025-10-09T21:50:37.248322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72f59cdd-c0918dad-d132ad7e-bb5d22fc_0] Write session: Do CDS request
2025-10-09T21:50:37.248328Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72f59cdd-c0918dad-d132ad7e-bb5d22fc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:37.259597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72f59cdd-c0918dad-d132ad7e-bb5d22fc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:37.259647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:37.259665Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1884
2025-10-09T21:50:37.259838Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1884 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:37.259886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1884 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:37.259898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1884 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:37.259902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:37.260452Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1884 sessionId:  describe result for acl check
2025-10-09T21:50:37.260488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:37.260493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:37.260494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:37.260499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342746599570709:5352] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:37.261017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342746599570709:5352] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:37.263492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342746599570709:5352] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:37.264631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:37.267012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:37.267202Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:37.267216Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:37.267228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:37.267292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:37.267386Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342746237738511:10320]
2025-10-09T21:50:37.267430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342746599570709:5352] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:37.267448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342746599570709:5352] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:37.267452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342746599570709:5352] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:37.267460Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1884 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:37.267970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:37.268234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:37.268281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:37.268587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:37.268640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.268647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.268648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.268649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.268650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.268652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.268653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.268654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.268655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.268660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.268671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.268675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.268679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.268683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.268687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.268691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.268695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.268699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.268706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.268714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.268720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.268725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.268730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.268736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.268740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.268745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.268750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.268755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.268759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.268766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.268771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.268776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.268780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.268785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.268789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.268794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.268823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:37.275286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:37.276304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:37.276321Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:37.276323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:37.276326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:37.276332Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1883 [2:7559342746237738511:10320]
2025-10-09T21:50:37.276342Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:37.276356Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:37.276360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:37.276363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:37.276364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:37.276377Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:37.276381Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:37.276387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:37.276429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.276435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.276437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.276439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:37.276500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:37.276509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.276520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.276535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.276559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:37.276580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342746599570814:5352], now have 1 active actors on pipe
2025-10-09T21:50:37.276589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342746599570604:2432], now have 1 active actors on pipe
2025-10-09T21:50:37.276596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342746237738506:10320]
2025-10-09T21:50:37.276603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342746237738506:10320]
2025-10-09T21:50:37.276605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342746237738506:10320]
2025-10-09T21:50:37.276617Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:37.276627Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:37.276629Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:37.276632Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:37.276632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:37.276647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:37.276655Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:37.276623Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1883
2025-10-09T21:50:37.276646Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1883
2025-10-09T21:50:37.276683Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342746237738506:10320]
2025-10-09T21:50:37.276691Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:37.276700Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:37.276724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:37.276733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:37.276769Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|34d81316-c1a0558e-ff84638-fb1cf323_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:37.276781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:37.276785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:37.276788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:37.276797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:37.276811Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:37.276826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:37.276985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:37.276992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:37.277003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:37.277094Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1884 partition: 0 MaxSeqNo: 25 sessionId: src_id|34d81316-c1a0558e-ff84638-fb1cf323_0
2025-10-09T21:50:37.277486Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72f59cdd-c0918dad-d132ad7e-bb5d22fc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|34d81316-c1a0558e-ff84638-fb1cf323_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:37.279962Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|34d81316-c1a0558e-ff84638-fb1cf323_0] Write session: try to update token
2025-10-09T21:50:37.279981Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|34d81316-c1a0558e-ff84638-fb1cf323_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:37.290816Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:37.290840Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:37.290843Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:37.290844Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:37.290854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342746237738506:10320]
2025-10-09T21:50:37.290858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342746237738506:10320]
2025-10-09T21:50:37.290862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342746237738506:10320]
2025-10-09T21:50:37.290906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:37.290906Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:37.290911Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:37.290914Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:37.290915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.290917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:37.291046Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:37.292543Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1884 sessionId: src_id|34d81316-c1a0558e-ff84638-fb1cf323_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:37.294974Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:37.347800Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342746599570831:6018]
2025-10-09T21:50:37.348087Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 1883, pipe: [1:7559342746599570831:6018]
2025-10-09T21:50:37.438941Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:37.439349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:37.439364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:37.439407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342746599570831:6018], now have 1 active actors on pipe
2025-10-09T21:50:37.439565Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:37.439614Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:37.439638Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:37.439671Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342746599570831:6018]
2025-10-09T21:50:37.439731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1884 reason: status is not ok: tablet will be restarted right now sessionId: src_id|34d81316-c1a0558e-ff84638-fb1cf323_0
2025-10-09T21:50:37.439919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1884 sessionId: src_id|34d81316-c1a0558e-ff84638-fb1cf323_0 is DEAD
2025-10-09T21:50:37.440090Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34d81316-c1a0558e-ff84638-fb1cf323_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:37.440098Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34d81316-c1a0558e-ff84638-fb1cf323_0] Write session will restart in 0.007627s
2025-10-09T21:50:37.440127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34d81316-c1a0558e-ff84638-fb1cf323_0] Write session: Do CDS request
2025-10-09T21:50:37.440134Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34d81316-c1a0558e-ff84638-fb1cf323_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:37.448502Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|34d81316-c1a0558e-ff84638-fb1cf323_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:37.448554Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:37.448569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1885
2025-10-09T21:50:37.448689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1885 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:37.448720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1885 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:37.448730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1885 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:37.448733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:37.449218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1885 sessionId:  describe result for acl check
2025-10-09T21:50:37.449256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:37.449262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:37.449263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:37.449268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342746599570908:5343] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:37.449805Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342746599570908:5343] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:37.452635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342746599570908:5343] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:37.453996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:37.456187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:37.456386Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:37.456398Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:37.456411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:37.456459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:37.456492Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342746237738724:10322]
2025-10-09T21:50:37.456509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342746599570908:5343] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:37.456525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342746599570908:5343] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:37.456529Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342746599570908:5343] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:37.456537Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1885 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:37.457001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:37.457507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:37.457573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:37.457850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:37.457917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.457926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.457927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.457928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.457929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.457930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.457931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.457932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.457934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.457939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.457949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.457953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.457957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.457960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.457964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.457968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.457972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.457976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.457983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.457992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.457999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.458004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.458009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.458014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.458018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.458022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.458027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.458051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.458055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.458060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.458065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.458070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.458074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.458079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.458085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.458089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.458132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:37.464680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:37.465775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:37.465792Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:37.465795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:37.465800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:37.465807Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1884 [2:7559342746237738724:10322]
2025-10-09T21:50:37.465822Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:37.465840Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:37.465843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:37.465846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:37.465848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:37.465864Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:37.465870Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:37.465877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:37.465933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.465941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.465944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.465947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:37.466014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:37.466022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.466050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.466067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.466089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:37.466116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342746599571044:5343], now have 1 active actors on pipe
2025-10-09T21:50:37.466124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342746599570833:2432], now have 1 active actors on pipe
2025-10-09T21:50:37.466132Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342746237738719:10322]
2025-10-09T21:50:37.466138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342746237738719:10322]
2025-10-09T21:50:37.466140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342746237738719:10322]
2025-10-09T21:50:37.466153Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:37.466170Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:37.466171Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:37.466182Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:37.466184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:37.466185Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:37.466188Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:37.466186Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1884
2025-10-09T21:50:37.466216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342746237738719:10322]
2025-10-09T21:50:37.466224Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:37.466226Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:37.466186Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1884
2025-10-09T21:50:37.466242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:37.466251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:37.466301Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a516d93-21ddd848-c7ab5084-2575a321_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:37.466313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:37.466317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:37.466320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:37.466330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:37.466344Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:37.466368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:37.466486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:37.466493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:37.466506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:37.466576Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1885 partition: 0 MaxSeqNo: 25 sessionId: src_id|a516d93-21ddd848-c7ab5084-2575a321_0
2025-10-09T21:50:37.466896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|34d81316-c1a0558e-ff84638-fb1cf323_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a516d93-21ddd848-c7ab5084-2575a321_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:37.469104Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a516d93-21ddd848-c7ab5084-2575a321_0] Write session: try to update token
2025-10-09T21:50:37.469119Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a516d93-21ddd848-c7ab5084-2575a321_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:37.480574Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1885 sessionId: src_id|a516d93-21ddd848-c7ab5084-2575a321_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:37.480595Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:37.480616Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:37.480619Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:37.480621Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:37.480630Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342746237738719:10322]
2025-10-09T21:50:37.480635Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342746237738719:10322]
2025-10-09T21:50:37.480639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342746237738719:10322]
2025-10-09T21:50:37.480683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:37.480693Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:37.480696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.480698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:37.480709Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:37.480714Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:37.480829Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:37.482987Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:37.539793Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342746599571056:6018]
2025-10-09T21:50:37.540074Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 1884, pipe: [1:7559342746599571056:6018]
2025-10-09T21:50:37.630722Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:37.631189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:37.631204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:37.631242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342746599571056:6018], now have 1 active actors on pipe
2025-10-09T21:50:37.631408Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:37.631491Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342746599571056:6018]
2025-10-09T21:50:37.631534Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:37.631618Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1885 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a516d93-21ddd848-c7ab5084-2575a321_0
2025-10-09T21:50:37.631787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1885 sessionId: src_id|a516d93-21ddd848-c7ab5084-2575a321_0 is DEAD
2025-10-09T21:50:37.631968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a516d93-21ddd848-c7ab5084-2575a321_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:37.631977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a516d93-21ddd848-c7ab5084-2575a321_0] Write session will restart in 0.007371s
2025-10-09T21:50:37.631942Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:37.632001Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a516d93-21ddd848-c7ab5084-2575a321_0] Write session: Do CDS request
2025-10-09T21:50:37.632007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a516d93-21ddd848-c7ab5084-2575a321_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:37.640473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a516d93-21ddd848-c7ab5084-2575a321_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:37.640603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:37.640619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1886
2025-10-09T21:50:37.640749Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1886 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:37.640783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1886 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:37.640788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1886 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:37.640791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:37.641184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1886 sessionId:  describe result for acl check
2025-10-09T21:50:37.641233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:37.641240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:37.641242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:37.641247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342746599571124:3868] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:37.641732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342746599571124:3868] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:37.644384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342746599571124:3868] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:37.646893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342746599571124:3868] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:37.646904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342746599571124:3868] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:37.646907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342746599571124:3868] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:37.646912Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1886 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:37.647307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:37.649476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:37.649636Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:37.649644Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:37.649652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:37.649689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:37.649716Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342746237738952:10330]
2025-10-09T21:50:37.650121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:37.650438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:37.650483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:37.650751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:37.650808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.650815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.650817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.650818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.650819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.650820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.650821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.650822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.650823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.650828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.650838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.650842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.650847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.650851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.650854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.650856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.650858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.650860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.650865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.650874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.650883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.650886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.650889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.650892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.650894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.650898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.650901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.650904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.650906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.650909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.650912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.650919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.650922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.650925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.650927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.650930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.650948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:37.657574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:37.658838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:37.658852Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:37.658854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:37.658858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:37.658864Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1885 [2:7559342746237738952:10330]
2025-10-09T21:50:37.658876Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:37.658899Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:37.658906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:37.658909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:37.658911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:37.658927Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:37.658938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:37.658944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:37.658989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.658994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.658995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.658997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:37.659045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:37.659053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.659060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.659067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.659076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:37.659090Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342746237738947:10330]
2025-10-09T21:50:37.659092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342746237738947:10330]
2025-10-09T21:50:37.659093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342746237738947:10330]
2025-10-09T21:50:37.659100Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:37.659114Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:37.659115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342746599571071:2432], now have 1 active actors on pipe
2025-10-09T21:50:37.659117Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:37.659121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:37.659122Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:37.659134Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:37.659137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:37.659143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342746599571280:3868], now have 1 active actors on pipe
2025-10-09T21:50:37.659164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342746237738947:10330]
2025-10-09T21:50:37.659174Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:37.659195Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:37.659175Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1885
2025-10-09T21:50:37.659199Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1885
2025-10-09T21:50:37.659515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:37.659530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:37.659559Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|529aa724-77938274-d55a2c16-310ccb64_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:37.659575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:37.659579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:37.659581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:37.659591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:37.659602Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:37.659620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:37.659901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:37.659915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:37.659934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:37.660023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1886 partition: 0 MaxSeqNo: 25 sessionId: src_id|529aa724-77938274-d55a2c16-310ccb64_0
2025-10-09T21:50:37.660413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a516d93-21ddd848-c7ab5084-2575a321_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|529aa724-77938274-d55a2c16-310ccb64_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:37.663052Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|529aa724-77938274-d55a2c16-310ccb64_0] Write session: try to update token
2025-10-09T21:50:37.663072Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|529aa724-77938274-d55a2c16-310ccb64_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:37.675628Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1886 sessionId: src_id|529aa724-77938274-d55a2c16-310ccb64_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:37.677458Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:37.677475Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:37.677485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342746237738947:10330]
2025-10-09T21:50:37.677534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:37.677560Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:37.677719Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:37.677879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:37.678002Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:37.678048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:37.678180Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:37.678488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:37.678539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:37.678557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:37.678577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:37.678741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:37.678847Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:37.684066Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:37.684086Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:37.684089Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:37.684090Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:37.684100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342746237738947:10330]
2025-10-09T21:50:37.684105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342746237738947:10330]
2025-10-09T21:50:37.684109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342746237738947:10330]
2025-10-09T21:50:37.684147Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:37.684152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:37.684158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.684161Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:37.684161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:37.684164Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:37.684294Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:37.731818Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342746599571303:6018]
2025-10-09T21:50:37.732062Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 1885, pipe: [1:7559342746599571303:6018]
2025-10-09T21:50:37.834323Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:37.840884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:37.840913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:37.841056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:37.841112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:37.841170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:37.841233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:37.841280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:37.841291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:37.841349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:37.841403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:37.841474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:37.841532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:37.841625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:37.841632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:37.841686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:37.841744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:37.841806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:37.841859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:37.841899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:37.841905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:37.841962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:37.842019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:37.842090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:37.842146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:37.842188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:37.842193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:37.842251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:37.842314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:37.842373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:37.842427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:37.842466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:37.842471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:37.842528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:37.842581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:37.842635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:37.842688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:37.842731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:37.842737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:37.842792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:37.842844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:37.842899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:37.842953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:37.842990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:37.842995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:37.843048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:37.843101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:37.843155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:37.843206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:37.843243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:37.843248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:37.843300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:37.843352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:37.843417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:37.843479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:37.843524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:37.843530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:37.843586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:37.843637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:37.843691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:37.843745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:37.843784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:37.843789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:37.843835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342746599571303:6018], now have 1 active actors on pipe
2025-10-09T21:50:37.844050Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:37.844072Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:37.844100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1886 reason: error at write: Disconnected sessionId: src_id|529aa724-77938274-d55a2c16-310ccb64_0
2025-10-09T21:50:37.844274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1886 sessionId: src_id|529aa724-77938274-d55a2c16-310ccb64_0 is DEAD
2025-10-09T21:50:37.844406Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|529aa724-77938274-d55a2c16-310ccb64_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:37.844417Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|529aa724-77938274-d55a2c16-310ccb64_0] Write session will restart in 0.009502s
2025-10-09T21:50:37.844437Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|529aa724-77938274-d55a2c16-310ccb64_0] Write session: Do CDS request
2025-10-09T21:50:37.844418Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:37.844449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|529aa724-77938274-d55a2c16-310ccb64_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:37.844477Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342746599571303:6018]
2025-10-09T21:50:37.844561Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:37.855574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|529aa724-77938274-d55a2c16-310ccb64_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:37.855663Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:37.855683Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1887
2025-10-09T21:50:37.855834Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1887 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:37.855872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1887 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:37.855884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1887 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:37.855888Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:37.856328Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1887 sessionId:  describe result for acl check
2025-10-09T21:50:37.856368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:37.856375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:37.856377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:37.856383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342746599571471:3862] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:37.856938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342746599571471:3862] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:37.858495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:37.859599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342746599571471:3862] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:37.860966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:37.861258Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:37.861270Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:37.861287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:37.861351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:37.861392Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342746237739195:10340]
2025-10-09T21:50:37.861957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342746599571471:3862] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:37.861972Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342746599571471:3862] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:37.861976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342746599571471:3862] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:37.861984Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1887 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:37.862003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:37.862275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:37.862318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:37.862625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:37.862672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.862678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.862679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.862680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.862682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.862683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.862684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.862685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.862687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.862692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.862703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.862708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.862712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.862716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.862720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.862724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.862728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.862732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.862740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.862748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:37.862758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.862764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:37.862774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.862780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:37.862786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.862792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:37.862796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.862802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:37.862811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.862816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:37.862823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.862832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.862838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.862848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.862853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.862858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.862886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:37.869413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:37.870433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:37.870447Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:37.870448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:37.870452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:37.870457Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1886 [2:7559342746237739195:10340]
2025-10-09T21:50:37.870466Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:37.870484Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:37.870487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:37.870491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:37.870493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:37.870512Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:37.870528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:37.870536Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:37.870604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.870613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.870615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.870618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:37.870676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:37.870684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.870696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:37.870706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:37.870722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:37.870748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342746599571519:3862], now have 1 active actors on pipe
2025-10-09T21:50:37.870757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342746599571306:2432], now have 1 active actors on pipe
2025-10-09T21:50:37.870764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342746237739190:10340]
2025-10-09T21:50:37.870766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342746237739190:10340]
2025-10-09T21:50:37.870769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342746237739190:10340]
2025-10-09T21:50:37.870782Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:37.870802Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:37.870807Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:37.870810Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:37.870812Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:37.870820Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:37.870823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:37.870804Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1886
2025-10-09T21:50:37.870807Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1886
2025-10-09T21:50:37.870851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342746237739190:10340]
2025-10-09T21:50:37.870859Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:37.870861Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:37.870879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:37.870888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:37.870937Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6e8db925-27f701a6-b9a9d4bf-1a0fe0e9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:37.870949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:37.870952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:37.870955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:37.870965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:37.870980Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:37.871001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:37.871146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:37.871154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:37.871168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:37.871242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1887 partition: 0 MaxSeqNo: 25 sessionId: src_id|6e8db925-27f701a6-b9a9d4bf-1a0fe0e9_0
2025-10-09T21:50:37.871581Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|529aa724-77938274-d55a2c16-310ccb64_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6e8db925-27f701a6-b9a9d4bf-1a0fe0e9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:37.873999Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e8db925-27f701a6-b9a9d4bf-1a0fe0e9_0] Write session: try to update token
2025-10-09T21:50:37.874020Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e8db925-27f701a6-b9a9d4bf-1a0fe0e9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:37.885696Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:37.885720Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:37.885722Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:37.885724Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:37.885734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342746237739190:10340]
2025-10-09T21:50:37.885739Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342746237739190:10340]
2025-10-09T21:50:37.885743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342746237739190:10340]
2025-10-09T21:50:37.885790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:37.885799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:37.885798Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:37.885807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:37.885814Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:37.885818Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:37.885936Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:37.887431Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1887 sessionId: src_id|6e8db925-27f701a6-b9a9d4bf-1a0fe0e9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:37.889872Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:37.944840Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342746599571532:6018]
2025-10-09T21:50:37.945081Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 1886, pipe: [1:7559342746599571532:6018]
2025-10-09T21:50:38.036982Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:38.037421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.037434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.037487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342746599571532:6018], now have 1 active actors on pipe
2025-10-09T21:50:38.037745Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:38.037771Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342746599571532:6018]
2025-10-09T21:50:38.037798Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:38.037801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:38.037879Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1887 reason: error at write: Disconnected sessionId: src_id|6e8db925-27f701a6-b9a9d4bf-1a0fe0e9_0
2025-10-09T21:50:38.037911Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:38.038094Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1887 sessionId: src_id|6e8db925-27f701a6-b9a9d4bf-1a0fe0e9_0 is DEAD
2025-10-09T21:50:38.038277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e8db925-27f701a6-b9a9d4bf-1a0fe0e9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:38.038285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e8db925-27f701a6-b9a9d4bf-1a0fe0e9_0] Write session will restart in 0.008914s
2025-10-09T21:50:38.038322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e8db925-27f701a6-b9a9d4bf-1a0fe0e9_0] Write session: Do CDS request
2025-10-09T21:50:38.038328Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e8db925-27f701a6-b9a9d4bf-1a0fe0e9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:38.048483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e8db925-27f701a6-b9a9d4bf-1a0fe0e9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:38.048618Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:38.048636Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1888
2025-10-09T21:50:38.048776Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1888 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:38.048826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1888 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:38.048837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1888 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:38.048846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:38.049277Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1888 sessionId:  describe result for acl check
2025-10-09T21:50:38.049318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:38.049325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:38.049327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:38.049333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342750894538980:3890] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:38.049896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342750894538980:3890] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:38.051369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:38.051759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342750894538980:3890] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:38.053684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:38.053874Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:38.053884Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:38.053894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:38.053945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:38.053986Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342750532706708:10342]
2025-10-09T21:50:38.053987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342750894538980:3890] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:38.054001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342750894538980:3890] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:38.054005Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342750894538980:3890] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:38.054014Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1888 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:38.054603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:38.054875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:38.054923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:38.055237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:38.055281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.055287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.055289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.055290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.055291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.055292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.055294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.055295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.055296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.055301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.055312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.055316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.055319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.055323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.055327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.055332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.055336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.055340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.055352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.055359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.055366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.055376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.055385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.055397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.055406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.055411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.055420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.055426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.055435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.055446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.055451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.055459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.055468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.055473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.055482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.055486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.055517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:38.061888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:38.062973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:38.062990Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:38.062992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:38.062997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:38.063005Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1887 [2:7559342750532706708:10342]
2025-10-09T21:50:38.063020Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:38.063038Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:38.063046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:38.063050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:38.063053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:38.063071Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:38.063083Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:38.063091Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:38.063154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.063162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.063165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.063167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:38.063234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:38.063240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.063247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.063254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.063263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:38.063283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342750894538831:2432], now have 1 active actors on pipe
2025-10-09T21:50:38.063289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342750894539045:3890], now have 1 active actors on pipe
2025-10-09T21:50:38.063295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342750532706702:10342]
2025-10-09T21:50:38.063298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342750532706702:10342]
2025-10-09T21:50:38.063299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342750532706702:10342]
2025-10-09T21:50:38.063306Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:38.063317Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:38.063320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:38.063326Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:38.063327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:38.063329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:38.063331Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:38.063360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342750532706702:10342]
2025-10-09T21:50:38.063374Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:38.063369Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1887
2025-10-09T21:50:38.063383Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1887
2025-10-09T21:50:38.063407Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:38.063493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.063501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.063524Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4279d587-94fe650f-aee81cd5-c350a817_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:38.063537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:38.063539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:38.063541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:38.063550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:38.063559Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:38.063571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:38.063726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.063733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.063746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:38.063833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1888 partition: 0 MaxSeqNo: 25 sessionId: src_id|4279d587-94fe650f-aee81cd5-c350a817_0
2025-10-09T21:50:38.064177Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e8db925-27f701a6-b9a9d4bf-1a0fe0e9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4279d587-94fe650f-aee81cd5-c350a817_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:38.066769Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4279d587-94fe650f-aee81cd5-c350a817_0] Write session: try to update token
2025-10-09T21:50:38.066791Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4279d587-94fe650f-aee81cd5-c350a817_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:38.075055Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:38.075071Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:38.075081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342750532706702:10342]
2025-10-09T21:50:38.075137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:38.075145Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:38.075326Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:38.075474Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:38.075614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:38.075752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:38.075933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:38.075975Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:38.079841Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1888 sessionId: src_id|4279d587-94fe650f-aee81cd5-c350a817_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:38.082387Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:38.082593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.082611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.082650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:38.082750Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:38.087888Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:38.087903Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:38.087905Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:38.087907Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:38.087917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342750532706702:10342]
2025-10-09T21:50:38.087922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342750532706702:10342]
2025-10-09T21:50:38.087926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342750532706702:10342]
2025-10-09T21:50:38.087968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:38.087971Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:38.087976Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:38.087979Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:38.087983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.087986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:38.088114Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:38.137860Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342750894539058:6018]
2025-10-09T21:50:38.138124Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 1887, pipe: [1:7559342750894539058:6018]
2025-10-09T21:50:38.240295Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:38.247167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.247188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.247327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.247380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.247438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.247495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.247543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.247554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.247614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.247668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.247732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.247789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.247828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.247834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.247887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.247951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.248008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.248065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.248103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.248108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.248168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.248222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.248275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.248330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.248372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.248378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.248435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.248488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.248545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.248600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.248638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.248643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.248701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.248763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.248821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.248874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.248914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.248919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.248971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.249019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.249072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.249125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.249162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.249168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.249221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.249272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.249324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.249374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.249489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.249494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.249547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.249598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.249649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.249701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.249740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.249745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.249798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.249847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.249897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.249953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.250100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.250109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.250156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342750894539058:6018], now have 1 active actors on pipe
2025-10-09T21:50:38.250402Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:38.250414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:38.250417Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:38.250447Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1888 reason: error at write: Disconnected sessionId: src_id|4279d587-94fe650f-aee81cd5-c350a817_0
2025-10-09T21:50:38.250469Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342750894539058:6018]
2025-10-09T21:50:38.250643Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1888 sessionId: src_id|4279d587-94fe650f-aee81cd5-c350a817_0 is DEAD
2025-10-09T21:50:38.250769Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4279d587-94fe650f-aee81cd5-c350a817_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:38.250789Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4279d587-94fe650f-aee81cd5-c350a817_0] Write session will restart in 0.009577s
2025-10-09T21:50:38.250808Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4279d587-94fe650f-aee81cd5-c350a817_0] Write session: Do CDS request
2025-10-09T21:50:38.250814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4279d587-94fe650f-aee81cd5-c350a817_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:38.251198Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:38.261540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4279d587-94fe650f-aee81cd5-c350a817_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:38.261673Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:38.261693Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1889
2025-10-09T21:50:38.261823Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1889 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:38.261860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1889 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:38.261871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1889 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:38.261874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:38.262393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1889 sessionId:  describe result for acl check
2025-10-09T21:50:38.262445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:38.262451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:38.262453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:38.262459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342750894539185:3889] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:38.263032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342750894539185:3889] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:38.265850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:38.265946Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342750894539185:3889] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:38.268562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:38.268773Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:38.268783Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:38.268795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:38.268858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:38.268899Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342750532706925:10344]
2025-10-09T21:50:38.269147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342750894539185:3889] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:38.269168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342750894539185:3889] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:38.269172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342750894539185:3889] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:38.269182Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1889 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:38.269466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:38.269768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:38.269815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:38.270108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:38.270146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.270150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.270151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.270152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.270153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.270153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.270154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.270155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.270155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.270159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.270166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.270169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.270171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.270173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.270175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.270179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.270183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.270187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.270193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.270203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.270210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.270216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.270220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.270225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.270230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.270235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.270240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.270245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.270250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.270261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.270270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.270280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.270288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.270294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.270303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.270312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.270347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:38.277146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:38.278231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:38.278248Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:38.278250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:38.278254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:38.278260Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1888 [2:7559342750532706925:10344]
2025-10-09T21:50:38.278270Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:38.278287Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:38.278289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:38.278291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:38.278293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:38.278305Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:38.278310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:38.278316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:38.278369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.278376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.278378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.278379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:38.278436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:38.278441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.278451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.278465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.278485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:38.278514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342750532706920:10344]
2025-10-09T21:50:38.278517Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342750532706920:10344]
2025-10-09T21:50:38.278519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342750532706920:10344]
2025-10-09T21:50:38.278527Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:38.278538Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:38.278541Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:38.278542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:38.278545Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:38.278557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:38.278561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:38.278574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342750894539064:2432], now have 1 active actors on pipe
2025-10-09T21:50:38.278584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342750894539279:3889], now have 1 active actors on pipe
2025-10-09T21:50:38.278617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342750532706920:10344]
2025-10-09T21:50:38.278632Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:38.278637Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1888
2025-10-09T21:50:38.278664Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:38.278639Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1888
2025-10-09T21:50:38.278714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.278726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.278758Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cf45517d-4e6e1856-b4a50e43-4b34a17_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:38.278768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:38.278771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:38.278773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:38.278782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:38.278797Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:38.278818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:38.278986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.278997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.279013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:38.279110Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1889 partition: 0 MaxSeqNo: 25 sessionId: src_id|cf45517d-4e6e1856-b4a50e43-4b34a17_0
2025-10-09T21:50:38.279449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4279d587-94fe650f-aee81cd5-c350a817_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cf45517d-4e6e1856-b4a50e43-4b34a17_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:38.282117Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf45517d-4e6e1856-b4a50e43-4b34a17_0] Write session: try to update token
2025-10-09T21:50:38.282138Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf45517d-4e6e1856-b4a50e43-4b34a17_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:38.290528Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:38.290545Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:38.290556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342750532706920:10344]
2025-10-09T21:50:38.290604Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:38.290607Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:38.290777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:38.290918Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:38.291060Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:38.291195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:38.291381Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:38.291427Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:38.295656Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1889 sessionId: src_id|cf45517d-4e6e1856-b4a50e43-4b34a17_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:38.298147Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:38.298368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.298390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.298433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:38.298520Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:38.303477Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:38.303493Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:38.303496Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:38.303498Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:38.303510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342750532706920:10344]
2025-10-09T21:50:38.303514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342750532706920:10344]
2025-10-09T21:50:38.303519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342750532706920:10344]
2025-10-09T21:50:38.303565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:38.303572Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:38.303574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.303577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:38.303583Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:38.303586Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:38.303718Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:38.350844Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342750894539296:6018]
2025-10-09T21:50:38.351110Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 1888, pipe: [1:7559342750894539296:6018]
2025-10-09T21:50:38.455386Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:38.462184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.462206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.462350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.462404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.462461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.462519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.462801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.462809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.462880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.462930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.462985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.463252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.463292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.463298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.463353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.463407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.463466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.463517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.463553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.463558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.463614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.463669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.463730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.463784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.463823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.463829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.463884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.463935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.464009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.464064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.464104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.464109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.464164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.464220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.464273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.464336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.464375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.464380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.464435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.464491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.464546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.464598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.464686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.464691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.464745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.464798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.464855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.464906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.464952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.464957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.465008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.465054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.465104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.465157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.465202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.465208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.465267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.465319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.465371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.465421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.465486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.465493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.465534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342750894539296:6018], now have 1 active actors on pipe
2025-10-09T21:50:38.465773Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:38.465787Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342750894539296:6018]
2025-10-09T21:50:38.465827Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:38.465830Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:38.465863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1889 reason: error at write: Disconnected sessionId: src_id|cf45517d-4e6e1856-b4a50e43-4b34a17_0
2025-10-09T21:50:38.465989Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:38.466060Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1889 sessionId: src_id|cf45517d-4e6e1856-b4a50e43-4b34a17_0 is DEAD
2025-10-09T21:50:38.466193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf45517d-4e6e1856-b4a50e43-4b34a17_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:38.466201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf45517d-4e6e1856-b4a50e43-4b34a17_0] Write session will restart in 0.007383s
2025-10-09T21:50:38.466225Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf45517d-4e6e1856-b4a50e43-4b34a17_0] Write session: Do CDS request
2025-10-09T21:50:38.466231Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf45517d-4e6e1856-b4a50e43-4b34a17_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:38.474518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf45517d-4e6e1856-b4a50e43-4b34a17_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:38.474603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:38.474623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1890
2025-10-09T21:50:38.474780Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1890 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:38.474817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1890 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:38.474828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1890 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:38.474831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:38.475330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1890 sessionId:  describe result for acl check
2025-10-09T21:50:38.475373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:38.475381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:38.475382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:38.475388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342750894539348:3905] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:38.475952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342750894539348:3905] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:38.478737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342750894539348:3905] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:38.482213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342750894539348:3905] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:38.482229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342750894539348:3905] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:38.482233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342750894539348:3905] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:38.482243Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1890 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:38.482355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:38.485836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:38.486071Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:38.486082Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:38.486094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:38.486151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:38.486197Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342750532707144:10346]
2025-10-09T21:50:38.487883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:38.488301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:38.488365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:38.488631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:38.488692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.488698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.488699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.488699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.488700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.488701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.488702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.488702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.488703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.488708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.488716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.488718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.488720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.488723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.488725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.488727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.488729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.488731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.488738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.488743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.488748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.488751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.488754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.488757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.488760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.488763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.488766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.488769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.488772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.488776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.488779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.488782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.488785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.488788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.488790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.488793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.488810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:38.495614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:38.496676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:38.496686Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:38.496688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:38.496692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:38.496698Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1889 [2:7559342750532707144:10346]
2025-10-09T21:50:38.496709Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:38.496726Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:38.496729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:38.496731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:38.496732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:38.496748Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:38.496756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:38.496762Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:38.496817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.496824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.496827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.496830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:38.496898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:38.496904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.496912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.496919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.496929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:38.496948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342750894539298:2432], now have 1 active actors on pipe
2025-10-09T21:50:38.496959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342750532707138:10346]
2025-10-09T21:50:38.496960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342750532707138:10346]
2025-10-09T21:50:38.496962Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342750532707138:10346]
2025-10-09T21:50:38.496972Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:38.496982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342750894539513:3905], now have 1 active actors on pipe
2025-10-09T21:50:38.496986Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:38.496990Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:38.496994Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:38.496996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:38.496999Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:38.497001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:38.497031Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342750532707138:10346]
2025-10-09T21:50:38.497040Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:38.497045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:38.497044Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1889
2025-10-09T21:50:38.497064Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1889
2025-10-09T21:50:38.497100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.497109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.497132Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dfaf2399-394344bd-7f2d7ec1-f11fde11_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:38.497141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:38.497145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:38.497147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:38.497156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:38.497164Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:38.497175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:38.497301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.497307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.497319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:38.497405Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1890 partition: 0 MaxSeqNo: 25 sessionId: src_id|dfaf2399-394344bd-7f2d7ec1-f11fde11_0
2025-10-09T21:50:38.497771Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf45517d-4e6e1856-b4a50e43-4b34a17_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dfaf2399-394344bd-7f2d7ec1-f11fde11_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:38.500481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfaf2399-394344bd-7f2d7ec1-f11fde11_0] Write session: try to update token
2025-10-09T21:50:38.500499Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfaf2399-394344bd-7f2d7ec1-f11fde11_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:38.514854Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1890 sessionId: src_id|dfaf2399-394344bd-7f2d7ec1-f11fde11_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:38.516966Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:38.516984Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:38.516996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342750532707138:10346]
2025-10-09T21:50:38.517054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:38.517068Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:38.517242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:38.517405Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:38.517465Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:38.517555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:38.517693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:38.517882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:38.517938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.517952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.517971Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:38.518018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:38.518110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:38.522816Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:38.522832Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:38.522834Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:38.522837Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:38.522846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342750532707138:10346]
2025-10-09T21:50:38.522851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342750532707138:10346]
2025-10-09T21:50:38.522855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342750532707138:10346]
2025-10-09T21:50:38.522897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:38.522904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.522912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:38.522917Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:38.522924Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:38.522929Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:38.523042Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:38.565903Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342750894539526:6018]
2025-10-09T21:50:38.566173Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 1889, pipe: [1:7559342750894539526:6018]
2025-10-09T21:50:38.676424Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:38.684322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.684349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.684510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.684572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.684635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.684699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.684753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.684764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.684832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.684895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.684959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.685030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.685074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.685085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.685146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.685206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.685268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.685338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.685385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.685394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.685474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.685545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.685610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.685670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.685708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.685714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.685792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.685849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.685918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.685985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.686047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.686053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.686118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.686178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.686246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.686317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.686361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.686367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.686422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.686480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.686549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.686623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.686669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.686677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.686740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.686798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.686861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.686924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.686967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.686974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.687034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.687088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.687158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.687222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.687263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.687268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.687325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:38.687380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:38.687435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:38.687490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:38.687538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:38.687541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:38.687607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342750894539526:6018], now have 1 active actors on pipe
2025-10-09T21:50:38.687821Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:38.687835Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:38.687866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1890 reason: error at write: Disconnected sessionId: src_id|dfaf2399-394344bd-7f2d7ec1-f11fde11_0
2025-10-09T21:50:38.688052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1890 sessionId: src_id|dfaf2399-394344bd-7f2d7ec1-f11fde11_0 is DEAD
2025-10-09T21:50:38.688244Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfaf2399-394344bd-7f2d7ec1-f11fde11_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:38.688257Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfaf2399-394344bd-7f2d7ec1-f11fde11_0] Write session will restart in 0.006455s
2025-10-09T21:50:38.688257Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:38.688280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfaf2399-394344bd-7f2d7ec1-f11fde11_0] Write session: Do CDS request
2025-10-09T21:50:38.688286Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfaf2399-394344bd-7f2d7ec1-f11fde11_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:38.688310Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342750894539526:6018]
2025-10-09T21:50:38.688591Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:38.695550Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfaf2399-394344bd-7f2d7ec1-f11fde11_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:38.695615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:38.695635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1891
2025-10-09T21:50:38.695760Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1891 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:38.695792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1891 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:38.695802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1891 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:38.695806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:38.696165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1891 sessionId:  describe result for acl check
2025-10-09T21:50:38.696200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:38.696206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:38.696208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:38.696213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342750894539561:3915] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:38.696739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342750894539561:3915] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:38.699586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342750894539561:3915] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:38.703122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342750894539561:3915] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:38.703140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342750894539561:3915] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:38.703144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342750894539561:3915] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:38.703154Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1891 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:38.707912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:38.710994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:38.711280Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:38.711291Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:38.711305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:38.711374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:38.711427Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342750532707401:10354]
2025-10-09T21:50:38.712061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:38.712395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:38.712439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:38.712765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:38.712815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.712822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.712823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.712825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.712826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.712827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.712828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.712829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.712831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.712837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.712849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.712854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.712858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.712861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.712866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.712870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.712873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.712877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.712895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.712903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.712930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.712941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.712946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.712957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.712961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.712967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.712976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.712981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.712987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.712992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.712997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.713007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.713011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.713016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.713022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.713031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.713062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:38.720289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:38.721765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:38.721781Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:38.721785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:38.721790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:38.721798Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1890 [2:7559342750532707401:10354]
2025-10-09T21:50:38.721815Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:38.721835Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:38.721845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:38.721849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:38.721851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:38.721869Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:38.721875Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:38.721882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:38.721957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.721966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.721968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.721972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:38.722062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:38.722070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.722086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.722105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.722123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:38.722150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342750894539674:3915], now have 1 active actors on pipe
2025-10-09T21:50:38.722158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342750894539542:2432], now have 1 active actors on pipe
2025-10-09T21:50:38.722167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342750532707396:10354]
2025-10-09T21:50:38.722173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342750532707396:10354]
2025-10-09T21:50:38.722176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342750532707396:10354]
2025-10-09T21:50:38.722189Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:38.722209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:38.722208Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:38.722212Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:38.722215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:38.722228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:38.722232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:38.722225Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1890
2025-10-09T21:50:38.722232Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1890
2025-10-09T21:50:38.722267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342750532707396:10354]
2025-10-09T21:50:38.722276Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:38.722283Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:38.722299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.722306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.722351Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7ba62218-ee12e9d0-c51a1fc1-54f63ab7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:38.722359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:38.722364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:38.722366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:38.722373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:38.722386Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:38.722410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:38.722563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.722572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.722586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:38.722668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1891 partition: 0 MaxSeqNo: 25 sessionId: src_id|7ba62218-ee12e9d0-c51a1fc1-54f63ab7_0
2025-10-09T21:50:38.723029Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfaf2399-394344bd-7f2d7ec1-f11fde11_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7ba62218-ee12e9d0-c51a1fc1-54f63ab7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:38.730099Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ba62218-ee12e9d0-c51a1fc1-54f63ab7_0] Write session: try to update token
2025-10-09T21:50:38.730122Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ba62218-ee12e9d0-c51a1fc1-54f63ab7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:38.737507Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:38.737525Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:38.737528Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:38.737530Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:38.737541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342750532707396:10354]
2025-10-09T21:50:38.737546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342750532707396:10354]
2025-10-09T21:50:38.737550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342750532707396:10354]
2025-10-09T21:50:38.737599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:38.737607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.737610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:38.737616Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:38.737623Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:38.737628Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:38.737756Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:38.745615Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1891 sessionId: src_id|7ba62218-ee12e9d0-c51a1fc1-54f63ab7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:38.748195Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:38.788564Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342750894539779:6018]
2025-10-09T21:50:38.788830Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 1890, pipe: [1:7559342750894539779:6018]
2025-10-09T21:50:38.893117Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:38.893662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.893675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.893695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342750894539779:6018], now have 1 active actors on pipe
2025-10-09T21:50:38.893920Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:38.893921Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:38.893937Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:38.893984Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342750894539779:6018]
2025-10-09T21:50:38.894008Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1891 reason: error at write: Disconnected sessionId: src_id|7ba62218-ee12e9d0-c51a1fc1-54f63ab7_0
2025-10-09T21:50:38.894211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1891 sessionId: src_id|7ba62218-ee12e9d0-c51a1fc1-54f63ab7_0 is DEAD
2025-10-09T21:50:38.894372Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ba62218-ee12e9d0-c51a1fc1-54f63ab7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:38.894383Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ba62218-ee12e9d0-c51a1fc1-54f63ab7_0] Write session will restart in 0.005381s
2025-10-09T21:50:38.894400Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ba62218-ee12e9d0-c51a1fc1-54f63ab7_0] Write session: Do CDS request
2025-10-09T21:50:38.894407Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ba62218-ee12e9d0-c51a1fc1-54f63ab7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:38.894541Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:38.900547Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ba62218-ee12e9d0-c51a1fc1-54f63ab7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:38.900574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:38.900590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1892
2025-10-09T21:50:38.900754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1892 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:38.900791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1892 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:38.900803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1892 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:38.900811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:38.901278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1892 sessionId:  describe result for acl check
2025-10-09T21:50:38.901318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:38.901324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:38.901326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:38.901332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342750894539799:3926] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:38.901950Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342750894539799:3926] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:38.905102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342750894539799:3926] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:38.908511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342750894539799:3926] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:38.908531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342750894539799:3926] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:38.908534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342750894539799:3926] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:38.908543Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1892 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:38.910534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:38.912727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:38.912952Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:38.912962Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:38.912986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:38.913044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:38.913085Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342750532707626:10364]
2025-10-09T21:50:38.913540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:38.913840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:38.913886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:38.914161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:38.914207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.914211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.914212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.914213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.914214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.914215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.914215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.914216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.914216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.914220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.914227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.914229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.914232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.914234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.914236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.914238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.914240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.914242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.914247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.914252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:38.914257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.914260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:38.914263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.914266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:38.914269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.914272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:38.914274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.914277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:38.914280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.914285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:38.914288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.914291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.914293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.914296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.914299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.914302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.914318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:38.920869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:38.922108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:38.922121Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:38.922123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:38.922127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:38.922133Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1891 [2:7559342750532707626:10364]
2025-10-09T21:50:38.922144Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:38.922159Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:38.922160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:38.922163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:38.922164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:38.922179Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:38.922183Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:38.922189Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:38.922233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.922239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.922241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.922243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:38.922305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:38.922313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.922325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:38.922336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:38.922351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:38.922375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342750532707620:10364]
2025-10-09T21:50:38.922383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342750532707620:10364]
2025-10-09T21:50:38.922385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342750532707620:10364]
2025-10-09T21:50:38.922394Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:38.922404Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:38.922407Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:38.922410Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:38.922412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:38.922426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:38.922429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:38.922441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342750894539995:3926], now have 1 active actors on pipe
2025-10-09T21:50:38.922450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342750894539784:2432], now have 1 active actors on pipe
2025-10-09T21:50:38.922493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342750532707620:10364]
2025-10-09T21:50:38.922506Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:38.922484Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1891
2025-10-09T21:50:38.922488Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1891
2025-10-09T21:50:38.922539Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:38.922583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.922592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.922620Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4eba5d0f-869c12cd-5780ed3c-f194d703_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:38.922631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:38.922635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:38.922643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:38.922653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:38.922666Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:38.922681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:38.922843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:38.922856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:38.922882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:38.922986Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1892 partition: 0 MaxSeqNo: 25 sessionId: src_id|4eba5d0f-869c12cd-5780ed3c-f194d703_0
2025-10-09T21:50:38.923354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ba62218-ee12e9d0-c51a1fc1-54f63ab7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4eba5d0f-869c12cd-5780ed3c-f194d703_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:38.925841Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4eba5d0f-869c12cd-5780ed3c-f194d703_0] Write session: try to update token
2025-10-09T21:50:38.925863Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4eba5d0f-869c12cd-5780ed3c-f194d703_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:38.937014Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:38.937038Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:38.937040Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:38.937043Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:38.937054Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342750532707620:10364]
2025-10-09T21:50:38.937059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342750532707620:10364]
2025-10-09T21:50:38.937062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342750532707620:10364]
2025-10-09T21:50:38.937108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:38.937118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:38.937122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:38.937131Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:38.937137Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:38.937141Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:38.937288Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:38.939329Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1892 sessionId: src_id|4eba5d0f-869c12cd-5780ed3c-f194d703_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:38.941761Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:38.993935Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342750894540013:6018]
2025-10-09T21:50:38.994189Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 1891, pipe: [1:7559342750894540013:6018]
2025-10-09T21:50:39.094124Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:39.094418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:39.094434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:39.094479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342750894540013:6018], now have 1 active actors on pipe
2025-10-09T21:50:39.094692Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:39.094718Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:39.094737Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:39.094738Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:39.094786Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342750894540013:6018]
2025-10-09T21:50:39.094801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1892 reason: error at write: Disconnected sessionId: src_id|4eba5d0f-869c12cd-5780ed3c-f194d703_0
2025-10-09T21:50:39.094993Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1892 sessionId: src_id|4eba5d0f-869c12cd-5780ed3c-f194d703_0 is DEAD
2025-10-09T21:50:39.095141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4eba5d0f-869c12cd-5780ed3c-f194d703_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:39.095154Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4eba5d0f-869c12cd-5780ed3c-f194d703_0] Write session will restart in 0.008653s
2025-10-09T21:50:39.095182Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4eba5d0f-869c12cd-5780ed3c-f194d703_0] Write session: Do CDS request
2025-10-09T21:50:39.095189Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4eba5d0f-869c12cd-5780ed3c-f194d703_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:39.104554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4eba5d0f-869c12cd-5780ed3c-f194d703_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:39.104657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:39.104689Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1893
2025-10-09T21:50:39.104807Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1893 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:39.104840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1893 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:39.104846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1893 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:39.104854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:39.105269Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1893 sessionId:  describe result for acl check
2025-10-09T21:50:39.105301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:39.105304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:39.105306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:39.105313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342755189507373:3931] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:39.105901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342755189507373:3931] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:39.108345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342755189507373:3931] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:39.111045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342755189507373:3931] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:39.111048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:39.111062Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342755189507373:3931] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:39.111067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342755189507373:3931] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:39.111077Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1893 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:39.113806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:39.114019Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:39.114046Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:39.114064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:39.114124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:39.114168Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342754827675143:10366]
2025-10-09T21:50:39.114752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:39.115053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:39.115093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:39.115407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:39.115457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.115463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.115464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.115466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.115467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.115468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.115469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.115471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.115472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.115477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.115488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.115492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.115497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.115501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.115505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.115509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.115513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.115517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.115524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.115533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.115539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.115552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.115556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.115561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.115571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.115576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.115581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.115587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.115591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.115601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.115610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.115621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.115630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.115639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.115645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.115654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.115683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:39.122356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:39.123421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:39.123438Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:39.123441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:39.123445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:39.123452Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1892 [2:7559342754827675143:10366]
2025-10-09T21:50:39.123467Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:39.123493Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:39.123500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:39.123503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:39.123506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:39.123524Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:39.123537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:39.123545Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:39.123909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.123917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.123919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.123921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:39.123980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:39.123989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.124000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.124018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.124036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:39.124060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342755189507530:3931], now have 1 active actors on pipe
2025-10-09T21:50:39.124069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342755189507312:2432], now have 1 active actors on pipe
2025-10-09T21:50:39.124078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342754827675137:10366]
2025-10-09T21:50:39.124085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342754827675137:10366]
2025-10-09T21:50:39.124088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342754827675137:10366]
2025-10-09T21:50:39.124101Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:39.124118Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:39.124120Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:39.124124Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:39.124126Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:39.124128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:39.124130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:39.124169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342754827675137:10366]
2025-10-09T21:50:39.124172Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:39.124142Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1892
2025-10-09T21:50:39.124187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:39.124145Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1892
2025-10-09T21:50:39.124212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:39.124223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:39.124260Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|275dfec5-44904430-6e19004f-a5cf07dc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:39.124274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:39.124282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:39.124285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:39.124296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:39.124311Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:39.124331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:39.124469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:39.124481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:39.124495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:39.124595Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1893 partition: 0 MaxSeqNo: 25 sessionId: src_id|275dfec5-44904430-6e19004f-a5cf07dc_0
2025-10-09T21:50:39.124929Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4eba5d0f-869c12cd-5780ed3c-f194d703_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|275dfec5-44904430-6e19004f-a5cf07dc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:39.127223Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|275dfec5-44904430-6e19004f-a5cf07dc_0] Write session: try to update token
2025-10-09T21:50:39.127244Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|275dfec5-44904430-6e19004f-a5cf07dc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:39.139019Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:39.139039Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:39.139042Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:39.139044Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:39.139054Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342754827675137:10366]
2025-10-09T21:50:39.139058Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342754827675137:10366]
2025-10-09T21:50:39.139063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342754827675137:10366]
2025-10-09T21:50:39.139106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:39.139109Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:39.139115Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:39.139115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.139118Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:39.139118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:39.139257Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:39.140543Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1893 sessionId: src_id|275dfec5-44904430-6e19004f-a5cf07dc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:39.143275Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:39.195146Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342755189507543:6018]
2025-10-09T21:50:39.195591Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 1892, pipe: [1:7559342755189507543:6018]
2025-10-09T21:50:39.299535Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:39.299796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:39.299815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:39.299861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342755189507543:6018], now have 1 active actors on pipe
2025-10-09T21:50:39.300043Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:39.300070Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342755189507543:6018]
2025-10-09T21:50:39.300099Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:39.300103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:39.300109Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:39.300182Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1893 reason: error at write: Disconnected sessionId: src_id|275dfec5-44904430-6e19004f-a5cf07dc_0
2025-10-09T21:50:39.300363Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1893 sessionId: src_id|275dfec5-44904430-6e19004f-a5cf07dc_0 is DEAD
2025-10-09T21:50:39.300521Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|275dfec5-44904430-6e19004f-a5cf07dc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:39.300532Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|275dfec5-44904430-6e19004f-a5cf07dc_0] Write session will restart in 0.007014s
2025-10-09T21:50:39.300560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|275dfec5-44904430-6e19004f-a5cf07dc_0] Write session: Do CDS request
2025-10-09T21:50:39.300576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|275dfec5-44904430-6e19004f-a5cf07dc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:39.308467Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|275dfec5-44904430-6e19004f-a5cf07dc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:39.308586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:39.308606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1894
2025-10-09T21:50:39.308782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1894 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:39.308820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1894 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:39.308826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1894 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:39.308831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:39.309256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1894 sessionId:  describe result for acl check
2025-10-09T21:50:39.309283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:39.309289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:39.309291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:39.309295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342755189507592:3938] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:39.309750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342755189507592:3938] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:39.312655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342755189507592:3938] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:39.315057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342755189507592:3938] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:39.315075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342755189507592:3938] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:39.315079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342755189507592:3938] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:39.315094Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1894 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:39.316631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:39.318962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:39.319187Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:39.319202Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:39.319214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:39.319275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:39.319327Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342754827675365:10368]
2025-10-09T21:50:39.319984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:39.320302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:39.320348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:39.320705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:39.320768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.320775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.320777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.320778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.320779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.320780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.320782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.320783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.320784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.320790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.320802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.320806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.320811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.320815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.320820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.320824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.320828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.320832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.320840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.320848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.320860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.320866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.320875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.320881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.320890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.320895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.320906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.320911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.320915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.320921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.320926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.320932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.320936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.320941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.320951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.320956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.320988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:39.327775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:39.328871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:39.328893Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:39.328896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:39.328901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:39.328909Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1893 [2:7559342754827675365:10368]
2025-10-09T21:50:39.328925Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:39.328950Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:39.328958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:39.328962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:39.328965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:39.328984Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:39.328993Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:39.329003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:39.329064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.329072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.329074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.329077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:39.329153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:39.329165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.329178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.329192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.329211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:39.329237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342755189507553:2432], now have 1 active actors on pipe
2025-10-09T21:50:39.329245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342755189507770:3938], now have 1 active actors on pipe
2025-10-09T21:50:39.329253Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342754827675360:10368]
2025-10-09T21:50:39.329256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342754827675360:10368]
2025-10-09T21:50:39.329258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342754827675360:10368]
2025-10-09T21:50:39.329273Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:39.329289Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:39.329293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:39.329296Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:39.329298Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:39.329309Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:39.329312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:39.329306Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1893
2025-10-09T21:50:39.329306Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1893
2025-10-09T21:50:39.329349Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342754827675360:10368]
2025-10-09T21:50:39.329358Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:39.329360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:39.329380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:39.329386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:39.329441Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2eafcf64-40cd6232-2eab410e-dd2fc779_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:39.329468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:39.329478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:39.329481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:39.329496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:39.329511Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:39.329536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:39.329699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:39.329709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:39.329726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:39.329804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1894 partition: 0 MaxSeqNo: 25 sessionId: src_id|2eafcf64-40cd6232-2eab410e-dd2fc779_0
2025-10-09T21:50:39.330142Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|275dfec5-44904430-6e19004f-a5cf07dc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2eafcf64-40cd6232-2eab410e-dd2fc779_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:39.332418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2eafcf64-40cd6232-2eab410e-dd2fc779_0] Write session: try to update token
2025-10-09T21:50:39.332435Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2eafcf64-40cd6232-2eab410e-dd2fc779_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:39.344238Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:39.344260Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:39.344263Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:39.344265Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:39.344276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342754827675360:10368]
2025-10-09T21:50:39.344280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342754827675360:10368]
2025-10-09T21:50:39.344285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342754827675360:10368]
2025-10-09T21:50:39.344331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:39.344332Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:39.344342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.344346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:39.344347Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:39.344350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:39.344484Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:39.345166Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1894 sessionId: src_id|2eafcf64-40cd6232-2eab410e-dd2fc779_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:39.347576Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:39.400447Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342755189507785:6018]
2025-10-09T21:50:39.400745Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 1893, pipe: [1:7559342755189507785:6018]
2025-10-09T21:50:39.492909Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:39.493136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:39.493149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:39.493188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342755189507785:6018], now have 1 active actors on pipe
2025-10-09T21:50:39.493353Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:39.493401Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342755189507785:6018]
2025-10-09T21:50:39.493409Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:39.493414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:39.493439Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:39.493490Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1894 reason: error at write: Disconnected sessionId: src_id|2eafcf64-40cd6232-2eab410e-dd2fc779_0
2025-10-09T21:50:39.493665Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1894 sessionId: src_id|2eafcf64-40cd6232-2eab410e-dd2fc779_0 is DEAD
2025-10-09T21:50:39.493807Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2eafcf64-40cd6232-2eab410e-dd2fc779_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:39.493817Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2eafcf64-40cd6232-2eab410e-dd2fc779_0] Write session will restart in 0.005495s
2025-10-09T21:50:39.493837Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2eafcf64-40cd6232-2eab410e-dd2fc779_0] Write session: Do CDS request
2025-10-09T21:50:39.493841Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2eafcf64-40cd6232-2eab410e-dd2fc779_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:39.500477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2eafcf64-40cd6232-2eab410e-dd2fc779_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:39.500557Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:39.500571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1895
2025-10-09T21:50:39.500685Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1895 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:39.500716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1895 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:39.500725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1895 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:39.500735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:39.501118Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1895 sessionId:  describe result for acl check
2025-10-09T21:50:39.501156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:39.501165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:39.501170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:39.501175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342755189507808:3944] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:39.501620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342755189507808:3944] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:39.504411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342755189507808:3944] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:39.507391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342755189507808:3944] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:39.507405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342755189507808:3944] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:39.507410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342755189507808:3944] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:39.507417Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1895 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:39.509395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:39.511794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:39.511992Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:39.512004Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:39.512017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:39.512078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:39.512118Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342754827675588:10370]
2025-10-09T21:50:39.512558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:39.512864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:39.512918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:39.513235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:39.513295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.513301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.513302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.513303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.513305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.513306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.513307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.513308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.513310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.513315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.513327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.513331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.513340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.513344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.513352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.513356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.513365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.513368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.513375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.513384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.513391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.513396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.513403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.513408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.513412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.513417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.513424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.513434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.513440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.513461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.513466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.513471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.513476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.513486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.513491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.513497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.513527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:39.519701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:39.520894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:39.520907Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:39.520910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:39.520914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:39.520920Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1894 [2:7559342754827675588:10370]
2025-10-09T21:50:39.520933Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:39.520951Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:39.520959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:39.520962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:39.520964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:39.520979Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:39.520989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:39.520996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:39.521058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.521068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.521070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.521073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:39.521136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:39.521145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.521157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.521169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.521186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:39.521210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342755189507787:2432], now have 1 active actors on pipe
2025-10-09T21:50:39.521220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342755189507995:3944], now have 1 active actors on pipe
2025-10-09T21:50:39.521228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342754827675583:10370]
2025-10-09T21:50:39.521231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342754827675583:10370]
2025-10-09T21:50:39.521233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342754827675583:10370]
2025-10-09T21:50:39.521245Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:39.521260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:39.521265Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:39.521269Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:39.521271Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:39.521274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:39.521278Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:39.521305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342754827675583:10370]
2025-10-09T21:50:39.521312Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:39.521292Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1894
2025-10-09T21:50:39.521321Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:39.521299Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1894
2025-10-09T21:50:39.521344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:39.521354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:39.521387Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a779dfd9-c0f85585-4fafe689-a2d6746d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:39.521397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:39.521405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:39.521407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:39.521417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:39.521430Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:39.521458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:39.521589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:39.521595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:39.521607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:39.521719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1895 partition: 0 MaxSeqNo: 25 sessionId: src_id|a779dfd9-c0f85585-4fafe689-a2d6746d_0
2025-10-09T21:50:39.522065Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2eafcf64-40cd6232-2eab410e-dd2fc779_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a779dfd9-c0f85585-4fafe689-a2d6746d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:39.524326Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a779dfd9-c0f85585-4fafe689-a2d6746d_0] Write session: try to update token
2025-10-09T21:50:39.524341Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a779dfd9-c0f85585-4fafe689-a2d6746d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:39.535555Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:39.535572Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:39.535574Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:39.535576Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:39.535586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342754827675583:10370]
2025-10-09T21:50:39.535590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342754827675583:10370]
2025-10-09T21:50:39.535594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342754827675583:10370]
2025-10-09T21:50:39.535637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:39.535644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.535647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:39.535654Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:39.535660Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:39.535665Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:39.535771Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:39.536288Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1895 sessionId: src_id|a779dfd9-c0f85585-4fafe689-a2d6746d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:39.538595Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:39.593470Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342755189508020:6018]
2025-10-09T21:50:39.593721Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 1894, pipe: [1:7559342755189508020:6018]
2025-10-09T21:50:39.680188Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:39.680539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:39.680550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:39.680587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342755189508020:6018], now have 1 active actors on pipe
2025-10-09T21:50:39.680758Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:39.680794Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:39.680796Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:39.680809Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:39.680839Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342755189508020:6018]
2025-10-09T21:50:39.680879Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1895 reason: error at write: Disconnected sessionId: src_id|a779dfd9-c0f85585-4fafe689-a2d6746d_0
2025-10-09T21:50:39.681042Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1895 sessionId: src_id|a779dfd9-c0f85585-4fafe689-a2d6746d_0 is DEAD
2025-10-09T21:50:39.681193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a779dfd9-c0f85585-4fafe689-a2d6746d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:39.681203Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a779dfd9-c0f85585-4fafe689-a2d6746d_0] Write session will restart in 0.005256s
2025-10-09T21:50:39.681223Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a779dfd9-c0f85585-4fafe689-a2d6746d_0] Write session: Do CDS request
2025-10-09T21:50:39.681228Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a779dfd9-c0f85585-4fafe689-a2d6746d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:39.687387Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a779dfd9-c0f85585-4fafe689-a2d6746d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:39.687479Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:39.687495Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1896
2025-10-09T21:50:39.687621Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1896 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:39.687657Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1896 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:39.687668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1896 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:39.687672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:39.688100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1896 sessionId:  describe result for acl check
2025-10-09T21:50:39.688132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:39.688140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:39.688142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:39.688149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342755189508041:3958] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:39.688725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342755189508041:3958] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:39.690499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342755189508041:3958] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:39.693613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342755189508041:3958] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:39.693625Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342755189508041:3958] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:39.693628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342755189508041:3958] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:39.693633Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1896 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:39.696563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:39.698619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:39.698777Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:39.698786Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:39.698794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:39.698837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:39.698873Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342754827675812:10372]
2025-10-09T21:50:39.699388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:39.699696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:39.699737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:39.700043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:39.700086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.700091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.700093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.700094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.700095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.700096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.700097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.700099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.700100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.700104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.700115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.700118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.700121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.700125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.700128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.700132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.700136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.700140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.700151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.700164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.700175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.700184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.700189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.700194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.700198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.700203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.700211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.700217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.700226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.700236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.700245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.700254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.700262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.700267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.700272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.700276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.700304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:39.706397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:39.707616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:39.707629Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:39.707630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:39.707633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:39.707638Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1895 [2:7559342754827675812:10372]
2025-10-09T21:50:39.707647Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:39.707659Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:39.707662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:39.707664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:39.707665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:39.707677Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:39.707680Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:39.707687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:39.707737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.707741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.707743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.707745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:39.707795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:39.707801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.707817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.707827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.707838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:39.707854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342754827675807:10372]
2025-10-09T21:50:39.707859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342754827675807:10372]
2025-10-09T21:50:39.707860Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342754827675807:10372]
2025-10-09T21:50:39.707867Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:39.707878Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:39.707880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342755189508023:2432], now have 1 active actors on pipe
2025-10-09T21:50:39.707881Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:39.707883Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:39.707888Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:39.707896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:39.707899Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:39.707903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342755189508190:3958], now have 1 active actors on pipe
2025-10-09T21:50:39.707921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342754827675807:10372]
2025-10-09T21:50:39.707930Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:39.707944Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:39.707952Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1895
2025-10-09T21:50:39.708002Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1895
2025-10-09T21:50:39.708031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:39.708039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:39.708059Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dbe76178-b64c774-769d95b7-d745d79d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:39.708064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:39.708067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:39.708068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:39.708076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:39.708086Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:39.708099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:39.708221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:39.708226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:39.708235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:39.708310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1896 partition: 0 MaxSeqNo: 25 sessionId: src_id|dbe76178-b64c774-769d95b7-d745d79d_0
2025-10-09T21:50:39.708640Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a779dfd9-c0f85585-4fafe689-a2d6746d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dbe76178-b64c774-769d95b7-d745d79d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:39.711158Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dbe76178-b64c774-769d95b7-d745d79d_0] Write session: try to update token
2025-10-09T21:50:39.711175Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dbe76178-b64c774-769d95b7-d745d79d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:39.721468Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:39.721486Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:39.721488Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:39.721490Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:39.721499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342754827675807:10372]
2025-10-09T21:50:39.721504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342754827675807:10372]
2025-10-09T21:50:39.721511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342754827675807:10372]
2025-10-09T21:50:39.721551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:39.721551Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:39.721556Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:39.721558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.721560Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:39.721561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:39.721699Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:39.724667Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1896 sessionId: src_id|dbe76178-b64c774-769d95b7-d745d79d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:39.727078Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:39.780846Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342755189508278:6018]
2025-10-09T21:50:39.781099Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 1895, pipe: [1:7559342755189508278:6018]
2025-10-09T21:50:39.865542Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:39.865815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:39.865830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:39.865873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342755189508278:6018], now have 1 active actors on pipe
2025-10-09T21:50:39.866140Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:39.866112Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342755189508278:6018]
2025-10-09T21:50:39.866116Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:39.866152Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:39.866155Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:39.866237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1896 reason: error at write: Disconnected sessionId: src_id|dbe76178-b64c774-769d95b7-d745d79d_0
2025-10-09T21:50:39.866419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1896 sessionId: src_id|dbe76178-b64c774-769d95b7-d745d79d_0 is DEAD
2025-10-09T21:50:39.866539Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbe76178-b64c774-769d95b7-d745d79d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:39.866550Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbe76178-b64c774-769d95b7-d745d79d_0] Write session will restart in 0.006518s
2025-10-09T21:50:39.866571Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbe76178-b64c774-769d95b7-d745d79d_0] Write session: Do CDS request
2025-10-09T21:50:39.866575Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbe76178-b64c774-769d95b7-d745d79d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:39.874491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dbe76178-b64c774-769d95b7-d745d79d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:39.874568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:39.874582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1897
2025-10-09T21:50:39.874689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1897 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:39.874714Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1897 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:39.874718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1897 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:39.874721Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:39.875100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1897 sessionId:  describe result for acl check
2025-10-09T21:50:39.875142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:39.875148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:39.875150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:39.875155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342755189508338:3978] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:39.875719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342755189508338:3978] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:39.878402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342755189508338:3978] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:39.880748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342755189508338:3978] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:39.880760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342755189508338:3978] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:39.880762Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342755189508338:3978] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:39.880768Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1897 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:39.881336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:39.883540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:39.883732Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:39.883744Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:39.883756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:39.883808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:39.883864Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342754827676079:10388]
2025-10-09T21:50:39.884445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:39.884730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:39.884775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:39.885087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:39.885140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.885146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.885147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.885149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.885150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.885152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.885153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.885154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.885155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.885160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.885171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.885176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.885179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.885184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.885188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.885192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.885196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.885200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.885207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.885214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:39.885225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.885231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:39.885235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.885245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:39.885254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.885264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:39.885267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.885273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:39.885281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.885285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:39.885289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.885293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.885302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.885308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.885312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.885318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.885351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:39.891799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:39.893122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:39.893140Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:39.893143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:39.893148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:39.893154Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1896 [2:7559342754827676079:10388]
2025-10-09T21:50:39.893167Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:39.893187Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:39.893190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:39.893193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:39.893196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:39.893214Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:39.893227Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:39.893234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:39.893287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.893294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.893296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.893299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:39.893369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:39.893377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.893388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:39.893404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:39.893426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:39.893462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342755189508281:2432], now have 1 active actors on pipe
2025-10-09T21:50:39.893466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342755189508504:3978], now have 1 active actors on pipe
2025-10-09T21:50:39.893474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342754827676074:10388]
2025-10-09T21:50:39.893476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342754827676074:10388]
2025-10-09T21:50:39.893479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342754827676074:10388]
2025-10-09T21:50:39.893492Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:39.893507Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:39.893513Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:39.893516Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:39.893519Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:39.893526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:39.893529Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:39.893556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342754827676074:10388]
2025-10-09T21:50:39.893567Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:39.893575Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:39.893585Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1896
2025-10-09T21:50:39.893589Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1896
2025-10-09T21:50:39.893661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:39.893671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:39.893700Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d9789274-22881998-cf6c7537-e7b401dc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:39.893712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:39.893716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:39.893718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:39.893729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:39.893744Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:39.893761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:39.893921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:39.893929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:39.893943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:39.894024Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1897 partition: 0 MaxSeqNo: 25 sessionId: src_id|d9789274-22881998-cf6c7537-e7b401dc_0
2025-10-09T21:50:39.894363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbe76178-b64c774-769d95b7-d745d79d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d9789274-22881998-cf6c7537-e7b401dc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:39.896593Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d9789274-22881998-cf6c7537-e7b401dc_0] Write session: try to update token
2025-10-09T21:50:39.896607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d9789274-22881998-cf6c7537-e7b401dc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:39.907653Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:39.907671Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:39.907673Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:39.907676Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:39.907686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342754827676074:10388]
2025-10-09T21:50:39.907693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342754827676074:10388]
2025-10-09T21:50:39.907697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342754827676074:10388]
2025-10-09T21:50:39.907742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:39.907744Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:39.907750Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:39.907750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:39.907753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:39.907754Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:39.907889Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:39.908639Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1897 sessionId: src_id|d9789274-22881998-cf6c7537-e7b401dc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:39.910972Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:39.966221Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342755189508517:6018]
2025-10-09T21:50:39.966516Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 1896, pipe: [1:7559342755189508517:6018]
2025-10-09T21:50:40.057811Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:40.058275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.058291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.058335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342755189508517:6018], now have 1 active actors on pipe
2025-10-09T21:50:40.058537Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:40.058568Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342755189508517:6018]
2025-10-09T21:50:40.058589Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:40.058591Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:40.058660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1897 reason: error at write: Disconnected sessionId: src_id|d9789274-22881998-cf6c7537-e7b401dc_0
2025-10-09T21:50:40.058643Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:40.058825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1897 sessionId: src_id|d9789274-22881998-cf6c7537-e7b401dc_0 is DEAD
2025-10-09T21:50:40.059007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9789274-22881998-cf6c7537-e7b401dc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:40.059019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9789274-22881998-cf6c7537-e7b401dc_0] Write session will restart in 0.009866s
2025-10-09T21:50:40.059047Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9789274-22881998-cf6c7537-e7b401dc_0] Write session: Do CDS request
2025-10-09T21:50:40.059054Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9789274-22881998-cf6c7537-e7b401dc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:40.070509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d9789274-22881998-cf6c7537-e7b401dc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:40.070657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:40.070670Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1898
2025-10-09T21:50:40.070789Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1898 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:40.070816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1898 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:40.070825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1898 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:40.070828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:40.071524Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1898 sessionId:  describe result for acl check
2025-10-09T21:50:40.071557Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:40.071563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:40.071565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:40.071571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342759484475968:3984] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:40.072131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342759484475968:3984] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:40.073822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:40.073941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342759484475968:3984] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:40.076036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:40.076311Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:40.076324Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:40.076337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:40.076332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342759484475968:3984] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:40.076347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342759484475968:3984] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:40.076351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342759484475968:3984] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:40.076358Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1898 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:40.076409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:40.076446Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342759122643601:10390]
2025-10-09T21:50:40.076986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:40.077218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:40.077257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:40.077503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:40.077540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.077545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.077546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.077546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.077547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.077548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.077549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.077549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.077550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.077553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.077559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.077561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.077563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.077565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.077568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.077570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.077572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.077574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.077579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.077584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.077589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.077592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.077595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.077598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.077600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.077604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.077606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.077609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.077612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.077615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.077618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.077621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.077623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.077626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.077629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.077633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.077647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:40.083769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:40.084920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:40.084934Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:40.084937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:40.084941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:40.084947Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1897 [2:7559342759122643601:10390]
2025-10-09T21:50:40.084959Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:40.084975Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:40.084978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:40.084981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:40.084983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:40.084998Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:40.085004Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:40.085012Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:40.085063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.085070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.085072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.085074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:40.085138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:40.085146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.085158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.085171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.085193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:40.085217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342759484476039:3984], now have 1 active actors on pipe
2025-10-09T21:50:40.085225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342759484475816:2432], now have 1 active actors on pipe
2025-10-09T21:50:40.085232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342759122643595:10390]
2025-10-09T21:50:40.085238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342759122643595:10390]
2025-10-09T21:50:40.085241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342759122643595:10390]
2025-10-09T21:50:40.085253Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:40.085270Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:40.085276Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:40.085279Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:40.085281Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:40.085289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:40.085292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:40.085272Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1897
2025-10-09T21:50:40.085314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342759122643595:10390]
2025-10-09T21:50:40.085322Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:40.085324Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:40.085347Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1897
2025-10-09T21:50:40.085391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.085400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.085425Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d1bc322e-ad5d7885-37de4a03-5c50b510_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:40.085436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:40.085453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:40.085461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:40.085471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:40.085487Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:40.085500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:40.085702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.085713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.085730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:40.085833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1898 partition: 0 MaxSeqNo: 25 sessionId: src_id|d1bc322e-ad5d7885-37de4a03-5c50b510_0
2025-10-09T21:50:40.086164Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d9789274-22881998-cf6c7537-e7b401dc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d1bc322e-ad5d7885-37de4a03-5c50b510_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:40.088445Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1bc322e-ad5d7885-37de4a03-5c50b510_0] Write session: try to update token
2025-10-09T21:50:40.088456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1bc322e-ad5d7885-37de4a03-5c50b510_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:40.099226Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:40.099242Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:40.099244Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:40.099246Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:40.099256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342759122643595:10390]
2025-10-09T21:50:40.099262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342759122643595:10390]
2025-10-09T21:50:40.099265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342759122643595:10390]
2025-10-09T21:50:40.099307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:40.099315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.099317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:40.099319Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:40.099324Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:40.099327Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:40.099440Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:40.101365Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1898 sessionId: src_id|d1bc322e-ad5d7885-37de4a03-5c50b510_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:40.104877Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:40.158706Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342759484476051:6018]
2025-10-09T21:50:40.159003Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 1897, pipe: [1:7559342759484476051:6018]
2025-10-09T21:50:40.246047Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:40.246569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.246584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.246629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342759484476051:6018], now have 1 active actors on pipe
2025-10-09T21:50:40.246831Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342759484476051:6018]
2025-10-09T21:50:40.246832Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:40.246835Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:40.246877Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:40.246901Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1898 reason: error at write: Disconnected sessionId: src_id|d1bc322e-ad5d7885-37de4a03-5c50b510_0
2025-10-09T21:50:40.246924Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:40.247095Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1898 sessionId: src_id|d1bc322e-ad5d7885-37de4a03-5c50b510_0 is DEAD
2025-10-09T21:50:40.247280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1bc322e-ad5d7885-37de4a03-5c50b510_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:40.247294Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1bc322e-ad5d7885-37de4a03-5c50b510_0] Write session will restart in 0.009907s
2025-10-09T21:50:40.247322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1bc322e-ad5d7885-37de4a03-5c50b510_0] Write session: Do CDS request
2025-10-09T21:50:40.247327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1bc322e-ad5d7885-37de4a03-5c50b510_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:40.258534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1bc322e-ad5d7885-37de4a03-5c50b510_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:40.258656Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:40.258674Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1899
2025-10-09T21:50:40.258782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1899 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:40.258815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1899 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:40.258830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1899 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:40.258834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:40.259406Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1899 sessionId:  describe result for acl check
2025-10-09T21:50:40.259449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:40.259458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:40.259460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:40.259465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342759484476181:3987] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:40.260069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342759484476181:3987] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:40.262188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342759484476181:3987] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:40.262959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:40.265227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:40.265438Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:40.265460Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:40.265472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:40.265523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:40.265551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342759484476181:3987] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:40.265566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342759484476181:3987] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:40.265570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342759484476181:3987] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:40.265576Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1899 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:40.265560Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342759122643828:10392]
2025-10-09T21:50:40.266172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:40.266479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:40.266517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:40.266834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:40.266894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.266902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.266903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.266904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.266906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.266907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.266908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.266909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.266910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.266916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.266926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.266930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.266933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.266936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.266939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.266943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.266946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.266949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.266956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.266964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.266970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.266975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.266979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.266985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.266989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.266999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.267003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.267012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.267020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.267026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.267035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.267042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.267047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.267055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.267060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.267067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.267100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:40.273252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:40.274893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:40.274905Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:40.274907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:40.274911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:40.274918Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1898 [2:7559342759122643828:10392]
2025-10-09T21:50:40.275168Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:40.275190Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:40.275197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:40.275200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:40.275202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:40.275218Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:40.275223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:40.275234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:40.275288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.275295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.275298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.275300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:40.275364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:40.275373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.275385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.275397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.275414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:40.275438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342759484476282:3987], now have 1 active actors on pipe
2025-10-09T21:50:40.275446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342759484476057:2432], now have 1 active actors on pipe
2025-10-09T21:50:40.275454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342759122643823:10392]
2025-10-09T21:50:40.275456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342759122643823:10392]
2025-10-09T21:50:40.275459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342759122643823:10392]
2025-10-09T21:50:40.275471Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:40.275488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:40.275497Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:40.275501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:40.275503Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:40.275508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:40.275512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:40.275538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342759122643823:10392]
2025-10-09T21:50:40.275546Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:40.275548Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:40.275525Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1898
2025-10-09T21:50:40.275548Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1898
2025-10-09T21:50:40.275575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.275583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.275640Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9618fc3b-1cace4ae-f82274ed-e17ba309_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:40.275652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:40.275656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:40.275658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:40.275669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:40.275678Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:40.275704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:40.275849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.275854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.275867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:40.275955Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1899 partition: 0 MaxSeqNo: 25 sessionId: src_id|9618fc3b-1cace4ae-f82274ed-e17ba309_0
2025-10-09T21:50:40.276273Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1bc322e-ad5d7885-37de4a03-5c50b510_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9618fc3b-1cace4ae-f82274ed-e17ba309_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:40.281062Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9618fc3b-1cace4ae-f82274ed-e17ba309_0] Write session: try to update token
2025-10-09T21:50:40.281091Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9618fc3b-1cace4ae-f82274ed-e17ba309_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:40.289273Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:40.289293Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:40.289296Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:40.289298Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:40.289308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342759122643823:10392]
2025-10-09T21:50:40.289313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342759122643823:10392]
2025-10-09T21:50:40.289317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342759122643823:10392]
2025-10-09T21:50:40.289359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:40.289364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.289366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:40.289375Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:40.289393Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:40.289402Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:40.289673Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:40.295388Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1899 sessionId: src_id|9618fc3b-1cace4ae-f82274ed-e17ba309_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:40.297728Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:40.347126Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342759484476299:6018]
2025-10-09T21:50:40.347376Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 1898, pipe: [1:7559342759484476299:6018]
2025-10-09T21:50:40.431394Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:40.431873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.431886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.431924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342759484476299:6018], now have 1 active actors on pipe
2025-10-09T21:50:40.432086Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:40.432153Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:40.432182Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:40.432226Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342759484476299:6018]
2025-10-09T21:50:40.432236Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1899 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9618fc3b-1cace4ae-f82274ed-e17ba309_0
2025-10-09T21:50:40.432397Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1899 sessionId: src_id|9618fc3b-1cace4ae-f82274ed-e17ba309_0 is DEAD
2025-10-09T21:50:40.432554Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9618fc3b-1cace4ae-f82274ed-e17ba309_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:40.432565Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9618fc3b-1cace4ae-f82274ed-e17ba309_0] Write session will restart in 0.005157s
2025-10-09T21:50:40.432584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9618fc3b-1cace4ae-f82274ed-e17ba309_0] Write session: Do CDS request
2025-10-09T21:50:40.432589Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9618fc3b-1cace4ae-f82274ed-e17ba309_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:40.438375Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9618fc3b-1cace4ae-f82274ed-e17ba309_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:40.438456Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:40.438468Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1900
2025-10-09T21:50:40.438588Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1900 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:40.438627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1900 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:40.438637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1900 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:40.438641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:40.439041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1900 sessionId:  describe result for acl check
2025-10-09T21:50:40.439075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:40.439080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:40.439081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:40.439087Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342759484476325:4004] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:40.439637Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342759484476325:4004] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:40.442440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342759484476325:4004] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:40.445269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342759484476325:4004] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:40.445282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342759484476325:4004] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:40.445286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342759484476325:4004] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:40.445293Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1900 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:40.446765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:40.449919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:40.450135Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:40.450144Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:40.450155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:40.450209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:40.450249Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342759122644057:10394]
2025-10-09T21:50:40.450831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:40.451159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:40.451209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:40.451531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:40.451595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.451601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.451602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.451603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.451605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.451606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.451607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.451608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.451610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.451614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.451626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.451630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.451634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.451638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.451642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.451648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.451652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.451655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.451662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.451675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.451686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.451695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.451704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.451709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.451719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.451729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.451737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.451743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.451751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.451759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.451767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.451773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.451782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.451787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.451792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.451797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.451830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:40.457960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:40.459199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:40.459214Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:40.459217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:40.459220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:40.459227Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1899 [2:7559342759122644057:10394]
2025-10-09T21:50:40.459239Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:40.459259Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:40.459265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:40.459268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:40.459270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:40.459285Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:40.459295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:40.459305Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:40.459355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.459362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.459364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.459367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:40.459426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:40.459433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.459444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.459461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.459478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:40.459501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342759484476523:4004], now have 1 active actors on pipe
2025-10-09T21:50:40.459508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342759484476301:2432], now have 1 active actors on pipe
2025-10-09T21:50:40.459515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342759122644052:10394]
2025-10-09T21:50:40.459522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342759122644052:10394]
2025-10-09T21:50:40.459524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342759122644052:10394]
2025-10-09T21:50:40.459537Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:40.459554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:40.459558Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:40.459561Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:40.459563Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:40.459572Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:40.459579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:40.459603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342759122644052:10394]
2025-10-09T21:50:40.459566Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1899
2025-10-09T21:50:40.459611Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:40.459613Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:40.459586Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1899
2025-10-09T21:50:40.459631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.459640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.459687Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6ccebdc3-7178cef7-a33f1213-edf213d3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:40.459701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:40.459705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:40.459708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:40.459718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:40.459726Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:40.459748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:40.459942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.459952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.459969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:40.460049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1900 partition: 0 MaxSeqNo: 25 sessionId: src_id|6ccebdc3-7178cef7-a33f1213-edf213d3_0
2025-10-09T21:50:40.460386Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9618fc3b-1cace4ae-f82274ed-e17ba309_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6ccebdc3-7178cef7-a33f1213-edf213d3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:40.462815Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ccebdc3-7178cef7-a33f1213-edf213d3_0] Write session: try to update token
2025-10-09T21:50:40.462832Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ccebdc3-7178cef7-a33f1213-edf213d3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:40.473303Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:40.473319Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:40.473320Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:40.473321Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:40.473329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342759122644052:10394]
2025-10-09T21:50:40.473332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342759122644052:10394]
2025-10-09T21:50:40.473335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342759122644052:10394]
2025-10-09T21:50:40.473371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:40.473376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.473377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:40.473381Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:40.473387Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:40.473391Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:40.473516Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:40.475586Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1900 sessionId: src_id|6ccebdc3-7178cef7-a33f1213-edf213d3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:40.477982Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:40.532462Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342759484476539:6018]
2025-10-09T21:50:40.532749Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 1899, pipe: [1:7559342759484476539:6018]
2025-10-09T21:50:40.617127Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:40.617556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.617572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.617612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342759484476539:6018], now have 1 active actors on pipe
2025-10-09T21:50:40.617813Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:40.617838Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342759484476539:6018]
2025-10-09T21:50:40.617858Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:40.617878Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:40.617949Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1900 reason: status is not ok: tablet will be restarted right now sessionId: src_id|6ccebdc3-7178cef7-a33f1213-edf213d3_0
2025-10-09T21:50:40.618145Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1900 sessionId: src_id|6ccebdc3-7178cef7-a33f1213-edf213d3_0 is DEAD
2025-10-09T21:50:40.618314Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ccebdc3-7178cef7-a33f1213-edf213d3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:40.618327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ccebdc3-7178cef7-a33f1213-edf213d3_0] Write session will restart in 0.009145s
2025-10-09T21:50:40.618354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ccebdc3-7178cef7-a33f1213-edf213d3_0] Write session: Do CDS request
2025-10-09T21:50:40.618361Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ccebdc3-7178cef7-a33f1213-edf213d3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:40.628441Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ccebdc3-7178cef7-a33f1213-edf213d3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:40.628534Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:40.628548Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1901
2025-10-09T21:50:40.628656Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1901 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:40.628693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1901 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:40.628704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1901 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:40.628708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:40.629133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1901 sessionId:  describe result for acl check
2025-10-09T21:50:40.629175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:40.629181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:40.629183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:40.629189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342759484476654:4018] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:40.630098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342759484476654:4018] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:40.632149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342759484476654:4018] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:40.632883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:40.635124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:40.635306Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:40.635315Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:40.635327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:40.635345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342759484476654:4018] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:40.635358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342759484476654:4018] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:40.635362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342759484476654:4018] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:40.635369Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1901 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:40.635380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:40.635418Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342759122644285:10396]
2025-10-09T21:50:40.635978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:40.636279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:40.636325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:40.636650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:40.636715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.636721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.636722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.636723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.636724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.636725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.636726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.636727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.636728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.636733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.636743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.636747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.636750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.636753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.636757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.636760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.636764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.636767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.636774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.636782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.636791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.636802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.636813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.636819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.636823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.636829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.636833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.636838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.636842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.636847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.636851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.636855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.636865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.636870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.636879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.636885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.636913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:40.643237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:40.644491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:40.644505Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:40.644507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:40.644512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:40.644518Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1900 [2:7559342759122644285:10396]
2025-10-09T21:50:40.644529Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:40.644545Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:40.644552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:40.644555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:40.644557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:40.644572Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:40.644583Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:40.644593Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:40.644649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.644655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.644657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.644659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:40.644722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:40.644728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.644735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.644743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.644755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:40.644772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342759122644280:10396]
2025-10-09T21:50:40.644774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342759122644280:10396]
2025-10-09T21:50:40.644776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342759122644280:10396]
2025-10-09T21:50:40.644781Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:40.644792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:40.644803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:40.644808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:40.644809Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:40.644815Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:40.644817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342759484476768:4018], now have 1 active actors on pipe
2025-10-09T21:50:40.644818Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:40.644837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342759484476541:2432], now have 1 active actors on pipe
2025-10-09T21:50:40.644842Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342759122644280:10396]
2025-10-09T21:50:40.644848Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:40.644858Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1900
2025-10-09T21:50:40.644876Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:40.644864Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1900
2025-10-09T21:50:40.644957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.644969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.644998Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ae58f354-5c6eb808-ecd588be-f15e79ac_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:40.645011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:40.645019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:40.645022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:40.645035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:40.645048Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:40.645071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:40.645250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.645260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.645274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:40.645372Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1901 partition: 0 MaxSeqNo: 25 sessionId: src_id|ae58f354-5c6eb808-ecd588be-f15e79ac_0
2025-10-09T21:50:40.645682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ccebdc3-7178cef7-a33f1213-edf213d3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ae58f354-5c6eb808-ecd588be-f15e79ac_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:40.648015Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae58f354-5c6eb808-ecd588be-f15e79ac_0] Write session: try to update token
2025-10-09T21:50:40.648030Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae58f354-5c6eb808-ecd588be-f15e79ac_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:40.659053Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:40.659073Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:40.659076Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:40.659078Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:40.659087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342759122644280:10396]
2025-10-09T21:50:40.659093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342759122644280:10396]
2025-10-09T21:50:40.659097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342759122644280:10396]
2025-10-09T21:50:40.659144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:40.659157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.659155Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:40.659160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:40.659162Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:40.659165Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:40.659326Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:40.660002Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1901 sessionId: src_id|ae58f354-5c6eb808-ecd588be-f15e79ac_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:40.662374Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:40.717815Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342759484476780:6018]
2025-10-09T21:50:40.718136Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 1900, pipe: [1:7559342759484476780:6018]
2025-10-09T21:50:40.806173Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:40.806414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.806426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.806466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342759484476780:6018], now have 1 active actors on pipe
2025-10-09T21:50:40.806673Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:40.806691Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:40.806701Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342759484476780:6018]
2025-10-09T21:50:40.806725Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:40.806729Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:40.806755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1901 reason: error at write: Disconnected sessionId: src_id|ae58f354-5c6eb808-ecd588be-f15e79ac_0
2025-10-09T21:50:40.806756Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:50:40.806844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1901 sessionId: src_id|ae58f354-5c6eb808-ecd588be-f15e79ac_0 is DEAD
2025-10-09T21:50:40.806860Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 19 consumer shared/user session shared/user_1_19_3411058432498014305_v1 is DEAD
2025-10-09T21:50:40.806976Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342677880078425:6025] disconnected.
2025-10-09T21:50:40.806985Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342677880078425:6025] disconnected; active server actors: 1
2025-10-09T21:50:40.806988Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342677880078425:6025] client user disconnected session shared/user_1_19_3411058432498014305_v1
2025-10-09T21:50:40.807130Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:50:40.807147Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae58f354-5c6eb808-ecd588be-f15e79ac_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:40.807158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae58f354-5c6eb808-ecd588be-f15e79ac_0] Write session will restart in 0.009200s
2025-10-09T21:50:40.807176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae58f354-5c6eb808-ecd588be-f15e79ac_0] Write session: Do CDS request
2025-10-09T21:50:40.807181Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae58f354-5c6eb808-ecd588be-f15e79ac_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:40.807275Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:50:40.807279Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:50:40.807285Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.005891s
2025-10-09T21:50:40.814473Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:50:40.814674Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:50:40.814691Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 20
2025-10-09T21:50:40.814891Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 20 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:50:40.814953Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:50:40.815033Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 auth for : user
2025-10-09T21:50:40.815229Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 Handle describe topics response
2025-10-09T21:50:40.815268Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 auth is DEAD
2025-10-09T21:50:40.815271Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:50:40.815505Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:50:40.815612Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_20_4254230261340021184_v1
2025-10-09T21:50:40.815640Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342759484476839:4037] connected; active server actors: 1
2025-10-09T21:50:40.815706Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559342759484476839:4037] session shared/user_1_20_4254230261340021184_v1
2025-10-09T21:50:40.815720Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:50:40.815732Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:50:40.815739Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_20_4254230261340021184_v1" (Sender=[1:7559342759484476829:4037], Pipe=[1:7559342759484476839:4037], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:50:40.815743Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:50:40.815766Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:50:40.815782Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_20_4254230261340021184_v1" (Sender=[1:7559342759484476829:4037], Pipe=[1:7559342759484476839:4037], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:50:40.815792Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_20_4254230261340021184_v1" sender [1:7559342759484476829:4037] lock partition 0 for ReadingSession "shared/user_1_20_4254230261340021184_v1" (Sender=[1:7559342759484476829:4037], Pipe=[1:7559342759484476839:4037], Partitions=[], ActiveFamilyCount=1) generation 1 step 20
2025-10-09T21:50:40.815806Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:50:40.815815Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000040s
2025-10-09T21:50:40.816037Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 20 Session: "shared/user_1_20_4254230261340021184_v1" ClientId: "user" PipeClient { RawX1: 7559342759484476839 RawX2: 4503603922341829 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:50:40.816063Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:50:40.817410Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae58f354-5c6eb808-ecd588be-f15e79ac_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:40.817505Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:40.817519Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1902
2025-10-09T21:50:40.817631Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1902 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:40.817661Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1902 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:40.817669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1902 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:40.817672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:40.818107Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1902 sessionId:  describe result for acl check
2025-10-09T21:50:40.818140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:40.818147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:40.818149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:40.818154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342759484476913:4042] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:40.818704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342759484476913:4042] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:40.820881Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342759484476913:4042] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:40.822193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:40.824952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:40.825177Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:40.825188Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:40.825200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:40.825262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:40.825273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342759484476913:4042] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:40.825306Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342759122644558:10412]
2025-10-09T21:50:40.825288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342759484476913:4042] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:40.825291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342759484476913:4042] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:40.825298Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1902 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:40.825926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:40.826211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:40.826255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:40.826759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:40.826818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.826824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.826825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.826827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.826829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.826830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.826831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.826832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.826833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.826838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.826850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.826854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.826858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.826862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.826866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.826870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.826874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.826878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.826884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.826892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:40.826899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.826905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:40.826909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.826915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:40.826919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.826924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:40.826930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.826935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:40.826939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.826944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:40.826949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.826960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.826966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.826971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.826981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.826988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.827022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:40.833432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:40.834490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:40.834507Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:40.834509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:40.834514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:40.834521Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1901 [2:7559342759122644558:10412]
2025-10-09T21:50:40.834538Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:40.834558Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:40.834564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:40.834566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:40.834568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:40.834583Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:40.834592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:40.834599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:40.834926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.834934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.834935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.834938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:40.835002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:40.835010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.835021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:40.835039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:40.835055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:40.835073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342759484476856:4041], now have 1 active actors on pipe
2025-10-09T21:50:40.835076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342759484476806:2432], now have 1 active actors on pipe
2025-10-09T21:50:40.835078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342759484477037:4042], now have 1 active actors on pipe
2025-10-09T21:50:40.835084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342759122644553:10412]
2025-10-09T21:50:40.835085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342759122644553:10412]
2025-10-09T21:50:40.835087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342759122644553:10412]
2025-10-09T21:50:40.835095Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:40.835109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:40.835116Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:40.835119Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:40.835122Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:40.835124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:40.835127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:40.835165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342759122644553:10412]
2025-10-09T21:50:40.835178Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:40.835199Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:40.835197Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1901
2025-10-09T21:50:40.835206Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1901, pipe: [1:7559342759484476856:4041]
2025-10-09T21:50:40.835235Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1901
2025-10-09T21:50:40.835320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.835330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.835341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_20_4254230261340021184_v1 on pipe: [1:7559342759484476856:4041]
2025-10-09T21:50:40.835358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.835363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.835371Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_20_4254230261340021184_v1:1 with generation 1901
2025-10-09T21:50:40.835373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:40.835380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:40.835382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:40.835397Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_20_4254230261340021184_v1
2025-10-09T21:50:40.835403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:40.835431Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|473cfe00-7ece8819-a1a1c82c-f944a27d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:40.835440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:50:40.835443Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:50:40.840468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:50:40.840504Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:50:40.840514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:40.840522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:40.840525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:40.840538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:40.840553Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:40.840561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:50:40.840580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:50:40.840602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:40.840723Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046640834 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:50:40.840745Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:50:40.840775Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 sending to client partition status
2025-10-09T21:50:40.840811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.840825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.840846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:40.840969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1902 partition: 0 MaxSeqNo: 25 sessionId: src_id|473cfe00-7ece8819-a1a1c82c-f944a27d_0
2025-10-09T21:50:40.841280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae58f354-5c6eb808-ecd588be-f15e79ac_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|473cfe00-7ece8819-a1a1c82c-f944a27d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:40.843776Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|473cfe00-7ece8819-a1a1c82c-f944a27d_0] Write session: try to update token
2025-10-09T21:50:40.843793Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|473cfe00-7ece8819-a1a1c82c-f944a27d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:40.853264Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:40.853287Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:40.853298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342759122644553:10412]
2025-10-09T21:50:40.853352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:40.853359Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:40.853575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:40.853742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:40.853901Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:40.854150Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:40.854360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:40.854412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:40.857974Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1902 sessionId: src_id|473cfe00-7ece8819-a1a1c82c-f944a27d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:40.860383Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:40.861227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:40.861251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:40.861272Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:40.861274Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:40.861276Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:40.861278Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:40.861286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342759122644553:10412]
2025-10-09T21:50:40.861289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342759122644553:10412]
2025-10-09T21:50:40.861292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342759122644553:10412]
2025-10-09T21:50:40.861340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:40.861349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:40.861354Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:40.861354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:40.861356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:40.861357Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:40.861485Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:41.010006Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:41.010394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:41.010536Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:41.010575Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342759484476856:4041]
2025-10-09T21:50:41.010602Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:41.010626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_20_4254230261340021184_v1
2025-10-09T21:50:41.010649Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:41.010675Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:41.010686Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_20_4254230261340021184_v1
2025-10-09T21:50:41.010758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1902 reason: error at write: Disconnected sessionId: src_id|473cfe00-7ece8819-a1a1c82c-f944a27d_0
2025-10-09T21:50:41.010899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1902 sessionId: src_id|473cfe00-7ece8819-a1a1c82c-f944a27d_0 is DEAD
2025-10-09T21:50:41.011030Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|473cfe00-7ece8819-a1a1c82c-f944a27d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:41.011054Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|473cfe00-7ece8819-a1a1c82c-f944a27d_0] Write session will restart in 0.007876s
2025-10-09T21:50:41.011080Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|473cfe00-7ece8819-a1a1c82c-f944a27d_0] Write session: Do CDS request
2025-10-09T21:50:41.011087Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|473cfe00-7ece8819-a1a1c82c-f944a27d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:41.020555Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|473cfe00-7ece8819-a1a1c82c-f944a27d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:41.020644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:41.020681Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1903
2025-10-09T21:50:41.020790Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1903 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:41.020818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1903 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:41.020828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1903 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:41.020831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:41.021411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1903 sessionId:  describe result for acl check
2025-10-09T21:50:41.021463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:41.021471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:41.021473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:41.021479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342763779444412:4054] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:41.022052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342763779444412:4054] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:41.024717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342763779444412:4054] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:41.027325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342763779444412:4054] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:41.027342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342763779444412:4054] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:41.027346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342763779444412:4054] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:41.027357Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1903 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:41.027905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:41.030437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:41.030628Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:41.030636Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:41.030646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:41.030702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:41.030738Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342763417612088:10414]
2025-10-09T21:50:41.031198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:41.031513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:41.031549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:41.031784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:41.031826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.031831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.031832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.031833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.031833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.031834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.031835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.031836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.031837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.031840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.031848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.031850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.031852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.031854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.031856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.031858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.031861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.031863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.031868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.031874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.031878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.031881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.031884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.031887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.031890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.031893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.031896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.031899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.031902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.031905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.031908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.031913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.031916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.031919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.031921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.031924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.031950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:41.038458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:41.039611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:41.039623Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:41.039625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:41.039628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:41.039634Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1902 [2:7559342763417612088:10414]
2025-10-09T21:50:41.039646Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:41.039663Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:41.039665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:41.039667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:41.039669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:41.039686Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:41.039693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:41.039701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:41.039766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.039775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.039777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.039781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:41.039987Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1902
2025-10-09T21:50:41.039990Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1902
2025-10-09T21:50:41.040098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:41.040106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.040115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.040122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.040132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:41.040150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779444576:4054], now have 1 active actors on pipe
2025-10-09T21:50:41.040156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779444352:2432], now have 1 active actors on pipe
2025-10-09T21:50:41.040172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.040181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.040199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342763417612083:10414]
2025-10-09T21:50:41.040201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342763417612083:10414]
2025-10-09T21:50:41.040203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342763417612083:10414]
2025-10-09T21:50:41.040217Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:41.040230Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:41.040232Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:41.040233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:41.040234Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:41.040244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:41.040246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:41.040262Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9a0c0b2f-d3bcdfc7-49b67128-3e26b26e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:41.040272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:41.040275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:41.040277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:41.040285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:41.040295Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:41.040339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342763417612083:10414]
2025-10-09T21:50:41.040352Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:41.040358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:41.040361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:41.040507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.040518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.040531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:41.040613Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1903 partition: 0 MaxSeqNo: 25 sessionId: src_id|9a0c0b2f-d3bcdfc7-49b67128-3e26b26e_0
2025-10-09T21:50:41.040930Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|473cfe00-7ece8819-a1a1c82c-f944a27d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9a0c0b2f-d3bcdfc7-49b67128-3e26b26e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:41.043208Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9a0c0b2f-d3bcdfc7-49b67128-3e26b26e_0] Write session: try to update token
2025-10-09T21:50:41.043222Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9a0c0b2f-d3bcdfc7-49b67128-3e26b26e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:41.054600Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:41.054619Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:41.054622Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:41.054624Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:41.054634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342763417612083:10414]
2025-10-09T21:50:41.054638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342763417612083:10414]
2025-10-09T21:50:41.054643Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342763417612083:10414]
2025-10-09T21:50:41.054686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:41.054685Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:41.054692Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:41.054697Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:41.054697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.054700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:41.054835Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:41.056090Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1903 sessionId: src_id|9a0c0b2f-d3bcdfc7-49b67128-3e26b26e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:41.058384Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:41.110535Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342763779444589:4041]
2025-10-09T21:50:41.110786Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 1902, pipe: [1:7559342763779444589:4041]
2025-10-09T21:50:41.200694Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:41.200935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.200952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.200992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779444589:4041], now have 1 active actors on pipe
2025-10-09T21:50:41.201186Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:41.201224Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342763779444589:4041]
2025-10-09T21:50:41.201224Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:41.201229Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:41.201261Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:41.201294Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1903 reason: error at write: Disconnected sessionId: src_id|9a0c0b2f-d3bcdfc7-49b67128-3e26b26e_0
2025-10-09T21:50:41.201511Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1903 sessionId: src_id|9a0c0b2f-d3bcdfc7-49b67128-3e26b26e_0 is DEAD
2025-10-09T21:50:41.201650Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a0c0b2f-d3bcdfc7-49b67128-3e26b26e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:41.201660Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a0c0b2f-d3bcdfc7-49b67128-3e26b26e_0] Write session will restart in 0.005407s
2025-10-09T21:50:41.201681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a0c0b2f-d3bcdfc7-49b67128-3e26b26e_0] Write session: Do CDS request
2025-10-09T21:50:41.201686Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a0c0b2f-d3bcdfc7-49b67128-3e26b26e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:41.208473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9a0c0b2f-d3bcdfc7-49b67128-3e26b26e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:41.208623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:41.208641Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1904
2025-10-09T21:50:41.208755Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1904 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:41.208791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1904 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:41.208804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1904 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:41.208813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:41.209178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1904 sessionId:  describe result for acl check
2025-10-09T21:50:41.209209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:41.209214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:41.209223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:41.209227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342763779444623:4064] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:41.209701Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342763779444623:4064] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:41.212053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342763779444623:4064] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:41.215265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342763779444623:4064] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:41.215281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342763779444623:4064] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:41.215285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342763779444623:4064] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:41.215294Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1904 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:41.216309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:41.218713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:41.218882Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:41.218890Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:41.218900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:41.218940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:41.218970Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342763417612320:10416]
2025-10-09T21:50:41.219365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:41.219624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:41.219665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:41.219896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:41.219932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.219936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.219937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.219938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.219938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.219939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.219939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.219940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.219940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.219943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.219950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.219952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.219955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.219957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.219959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.219961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.219963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.219965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.219969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.219974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.219979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.219982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.219985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.219988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.219990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.219993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.219996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.219999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.220001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.220004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.220007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.220010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.220013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.220016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.220019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.220025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.220040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:41.226623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:41.227658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:41.227671Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:41.227682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:41.227685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:41.227693Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1903 [2:7559342763417612320:10416]
2025-10-09T21:50:41.227707Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:41.227729Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:41.227731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:41.227732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:41.227734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:41.227746Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:41.227752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:41.227757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:41.227797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.227803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.227804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.227806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:41.227857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:41.227863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.227870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.227876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.227885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:41.227900Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342763417612315:10416]
2025-10-09T21:50:41.227906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342763417612315:10416]
2025-10-09T21:50:41.227907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342763417612315:10416]
2025-10-09T21:50:41.227914Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:41.227924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779444595:2432], now have 1 active actors on pipe
2025-10-09T21:50:41.227924Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:41.227927Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:41.227929Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:41.227931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779444822:4064], now have 1 active actors on pipe
2025-10-09T21:50:41.227936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:41.227949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:41.227952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:41.227979Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342763417612315:10416]
2025-10-09T21:50:41.227989Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:41.228016Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:41.228012Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1903
2025-10-09T21:50:41.228016Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1903
2025-10-09T21:50:41.228256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.228265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.228287Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|95d039e4-53cf987-e39e2575-b6d0c02c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:41.228297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:41.228300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:41.228302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:41.228320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:41.228331Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:41.228342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:41.228492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.228497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.228508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:41.228589Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1904 partition: 0 MaxSeqNo: 25 sessionId: src_id|95d039e4-53cf987-e39e2575-b6d0c02c_0
2025-10-09T21:50:41.228944Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a0c0b2f-d3bcdfc7-49b67128-3e26b26e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|95d039e4-53cf987-e39e2575-b6d0c02c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:41.231245Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|95d039e4-53cf987-e39e2575-b6d0c02c_0] Write session: try to update token
2025-10-09T21:50:41.231260Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|95d039e4-53cf987-e39e2575-b6d0c02c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:41.242686Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:41.242702Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:41.242703Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:41.242705Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:41.242712Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342763417612315:10416]
2025-10-09T21:50:41.242716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342763417612315:10416]
2025-10-09T21:50:41.242720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342763417612315:10416]
2025-10-09T21:50:41.242752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:41.242757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.242759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:41.242773Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:41.242782Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:41.242786Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:41.242874Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:41.243309Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1904 sessionId: src_id|95d039e4-53cf987-e39e2575-b6d0c02c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:41.245756Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:41.301510Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342763779444841:4041]
2025-10-09T21:50:41.301797Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 1903, pipe: [1:7559342763779444841:4041]
2025-10-09T21:50:41.382107Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:41.382351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.382366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.382408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779444841:4041], now have 1 active actors on pipe
2025-10-09T21:50:41.382618Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:41.382631Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342763779444841:4041]
2025-10-09T21:50:41.382664Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:41.382667Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:41.382734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1904 reason: error at write: Disconnected sessionId: src_id|95d039e4-53cf987-e39e2575-b6d0c02c_0
2025-10-09T21:50:41.382783Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:41.382887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1904 sessionId: src_id|95d039e4-53cf987-e39e2575-b6d0c02c_0 is DEAD
2025-10-09T21:50:41.383049Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95d039e4-53cf987-e39e2575-b6d0c02c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:41.383060Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95d039e4-53cf987-e39e2575-b6d0c02c_0] Write session will restart in 0.009784s
2025-10-09T21:50:41.383089Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95d039e4-53cf987-e39e2575-b6d0c02c_0] Write session: Do CDS request
2025-10-09T21:50:41.383095Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95d039e4-53cf987-e39e2575-b6d0c02c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:41.394453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|95d039e4-53cf987-e39e2575-b6d0c02c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:41.394546Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:41.394564Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1905
2025-10-09T21:50:41.394692Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1905 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:41.394727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1905 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:41.394736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1905 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:41.394739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:41.395195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1905 sessionId:  describe result for acl check
2025-10-09T21:50:41.395236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:41.395244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:41.395245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:41.395251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342763779445038:4074] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:41.395798Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342763779445038:4074] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:41.396672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:41.397757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342763779445038:4074] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:41.398831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:41.399012Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:41.399022Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:41.399033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:41.399078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:41.399116Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342763417612552:10418]
2025-10-09T21:50:41.399707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:41.399869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342763779445038:4074] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:41.399883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342763779445038:4074] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:41.399886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342763779445038:4074] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:41.399893Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1905 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:41.399955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:41.399991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:41.400295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:41.400340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.400346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.400347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.400348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.400349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.400350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.400351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.400352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.400354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.400358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.400368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.400373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.400377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.400381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.400385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.400389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.400392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.400396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.400402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.400410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.400415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.400420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.400425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.400430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.400434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.400439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.400449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.400454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.400458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.400463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.400473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.400478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.400487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.400491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.400496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.400506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.400536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:41.406782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:41.407725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:41.407737Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:41.407739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:41.407742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:41.407747Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1904 [2:7559342763417612552:10418]
2025-10-09T21:50:41.407756Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:41.407771Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:41.407772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:41.407774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:41.407776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:41.407789Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:41.407792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:41.407797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:41.407841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.407846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.407847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.407849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:41.407899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:41.407906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.407916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.407928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.407937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:41.407954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342763417612547:10418]
2025-10-09T21:50:41.407955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342763417612547:10418]
2025-10-09T21:50:41.407957Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342763417612547:10418]
2025-10-09T21:50:41.407962Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:41.407970Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:41.407975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779445077:4074], now have 1 active actors on pipe
2025-10-09T21:50:41.407978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779444844:2432], now have 1 active actors on pipe
2025-10-09T21:50:41.407978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:41.407980Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:41.407981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:41.407989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:41.407991Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:41.408010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342763417612547:10418]
2025-10-09T21:50:41.408018Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:41.408022Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:41.408062Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1904
2025-10-09T21:50:41.408071Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1904
2025-10-09T21:50:41.408148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.408163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.408195Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|12c9ffaa-44827ce6-8f58c3b1-99b9aa47_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:41.408211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:41.408228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:41.408235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:41.408246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:41.408260Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:41.408276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:41.408397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.408404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.408416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:41.408485Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1905 partition: 0 MaxSeqNo: 25 sessionId: src_id|12c9ffaa-44827ce6-8f58c3b1-99b9aa47_0
2025-10-09T21:50:41.408840Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95d039e4-53cf987-e39e2575-b6d0c02c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|12c9ffaa-44827ce6-8f58c3b1-99b9aa47_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:41.411298Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12c9ffaa-44827ce6-8f58c3b1-99b9aa47_0] Write session: try to update token
2025-10-09T21:50:41.411317Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12c9ffaa-44827ce6-8f58c3b1-99b9aa47_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:41.421624Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:41.421640Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:41.421642Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:41.421643Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:41.421650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342763417612547:10418]
2025-10-09T21:50:41.421653Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342763417612547:10418]
2025-10-09T21:50:41.421656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342763417612547:10418]
2025-10-09T21:50:41.421688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:41.421687Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:41.421692Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:41.421692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.421694Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:41.421694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:41.421808Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:41.423603Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1905 sessionId: src_id|12c9ffaa-44827ce6-8f58c3b1-99b9aa47_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:41.425898Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:41.482759Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342763779445086:4041]
2025-10-09T21:50:41.483044Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 1904, pipe: [1:7559342763779445086:4041]
2025-10-09T21:50:41.558343Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:41.558616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.558629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.558669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779445086:4041], now have 1 active actors on pipe
2025-10-09T21:50:41.558855Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:41.558934Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342763779445086:4041]
2025-10-09T21:50:41.558941Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:41.558969Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:41.559031Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1905 reason: status is not ok: tablet will be restarted right now sessionId: src_id|12c9ffaa-44827ce6-8f58c3b1-99b9aa47_0
2025-10-09T21:50:41.559227Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1905 sessionId: src_id|12c9ffaa-44827ce6-8f58c3b1-99b9aa47_0 is DEAD
2025-10-09T21:50:41.559361Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12c9ffaa-44827ce6-8f58c3b1-99b9aa47_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:41.559373Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12c9ffaa-44827ce6-8f58c3b1-99b9aa47_0] Write session will restart in 0.009112s
2025-10-09T21:50:41.559408Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12c9ffaa-44827ce6-8f58c3b1-99b9aa47_0] Write session: Do CDS request
2025-10-09T21:50:41.559413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12c9ffaa-44827ce6-8f58c3b1-99b9aa47_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:41.569476Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12c9ffaa-44827ce6-8f58c3b1-99b9aa47_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:41.569575Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:41.569590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1906
2025-10-09T21:50:41.569716Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1906 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:41.569746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1906 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:41.569756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1906 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:41.569760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:41.570297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1906 sessionId:  describe result for acl check
2025-10-09T21:50:41.570329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:41.570336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:41.570337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:41.570343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342763779445182:4079] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:41.570879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342763779445182:4079] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:41.573602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342763779445182:4079] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:41.575350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:41.577638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:41.577804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342763779445182:4079] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:41.577818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342763779445182:4079] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:41.577825Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:41.577835Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:41.577822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342763779445182:4079] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:41.577829Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1906 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:41.577845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:41.577899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:41.577938Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342763417612786:10420]
2025-10-09T21:50:41.578513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:41.578779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:41.578821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:41.579146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:41.579189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.579194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.579194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.579195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.579196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.579196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.579197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.579197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.579198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.579202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.579208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.579210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.579212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.579214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.579217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.579219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.579221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.579223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.579229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.579238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.579244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.579250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.579253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.579256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.579258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.579261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.579264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.579267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.579269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.579272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.579275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.579278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.579281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.579284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.579286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.579289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.579305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:41.585266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:41.586579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:41.586593Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:41.586595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:41.586599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:41.586605Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1905 [2:7559342763417612786:10420]
2025-10-09T21:50:41.586618Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:41.586634Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:41.586641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:41.586644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:41.586647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:41.586663Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:41.586672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:41.586678Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:41.586729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.586735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.586736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.586739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:41.586805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:41.586811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.586823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.586838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.586857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:41.586874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779445321:4079], now have 1 active actors on pipe
2025-10-09T21:50:41.586880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779445088:2432], now have 1 active actors on pipe
2025-10-09T21:50:41.586886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342763417612781:10420]
2025-10-09T21:50:41.586888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342763417612781:10420]
2025-10-09T21:50:41.586889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342763417612781:10420]
2025-10-09T21:50:41.586898Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:41.586912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:41.586913Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:41.586916Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:41.586918Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:41.586929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:41.586933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:41.586921Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1905
2025-10-09T21:50:41.586921Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1905
2025-10-09T21:50:41.586956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342763417612781:10420]
2025-10-09T21:50:41.586963Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:41.586965Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:41.586977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.586983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.587020Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|743b0e79-996b7b42-5841a875-76a53586_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:41.587030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:41.587033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:41.587035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:41.587041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:41.587052Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:41.587065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:41.587204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.587216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.587231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:41.587330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1906 partition: 0 MaxSeqNo: 25 sessionId: src_id|743b0e79-996b7b42-5841a875-76a53586_0
2025-10-09T21:50:41.587661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12c9ffaa-44827ce6-8f58c3b1-99b9aa47_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|743b0e79-996b7b42-5841a875-76a53586_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:41.589833Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|743b0e79-996b7b42-5841a875-76a53586_0] Write session: try to update token
2025-10-09T21:50:41.589845Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|743b0e79-996b7b42-5841a875-76a53586_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:41.601363Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:41.601384Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:41.601386Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:41.601388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:41.601398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342763417612781:10420]
2025-10-09T21:50:41.601402Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342763417612781:10420]
2025-10-09T21:50:41.601406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342763417612781:10420]
2025-10-09T21:50:41.601460Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:41.601461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:41.601465Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:41.601468Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:41.601470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.601473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:41.601606Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:41.602023Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1906 sessionId: src_id|743b0e79-996b7b42-5841a875-76a53586_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:41.604415Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:41.659405Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342763779445333:4041]
2025-10-09T21:50:41.659691Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 1905, pipe: [1:7559342763779445333:4041]
2025-10-09T21:50:41.745045Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:41.745365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.745379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.745415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779445333:4041], now have 1 active actors on pipe
2025-10-09T21:50:41.745598Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:41.745630Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:41.745635Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342763779445333:4041]
2025-10-09T21:50:41.745645Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:41.745657Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:41.745706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1906 reason: error at write: Disconnected sessionId: src_id|743b0e79-996b7b42-5841a875-76a53586_0
2025-10-09T21:50:41.745880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1906 sessionId: src_id|743b0e79-996b7b42-5841a875-76a53586_0 is DEAD
2025-10-09T21:50:41.746018Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|743b0e79-996b7b42-5841a875-76a53586_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:41.746045Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|743b0e79-996b7b42-5841a875-76a53586_0] Write session will restart in 0.009684s
2025-10-09T21:50:41.746072Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|743b0e79-996b7b42-5841a875-76a53586_0] Write session: Do CDS request
2025-10-09T21:50:41.746080Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|743b0e79-996b7b42-5841a875-76a53586_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:41.756410Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|743b0e79-996b7b42-5841a875-76a53586_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:41.756481Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:41.756497Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1907
2025-10-09T21:50:41.756601Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1907 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:41.756634Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1907 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:41.756645Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1907 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:41.756652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:41.757086Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1907 sessionId:  describe result for acl check
2025-10-09T21:50:41.757127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:41.757134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:41.757136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:41.757141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342763779445457:4092] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:41.757728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342763779445457:4092] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:41.759841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342763779445457:4092] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:41.761186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:41.763253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:41.763486Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:41.763495Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:41.763505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:41.763552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:41.763547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342763779445457:4092] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:41.763561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342763779445457:4092] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:41.763579Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342763417613027:10422]
2025-10-09T21:50:41.763564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342763779445457:4092] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:41.763572Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1907 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:41.763958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:41.764180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:41.764207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:41.764435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:41.764470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.764474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.764475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.764476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.764477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.764477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.764478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.764478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.764479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.764482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.764488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.764491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.764493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.764495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.764498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.764500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.764502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.764504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.764508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.764513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.764516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.764520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.764522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.764525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.764528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.764530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.764533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.764535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.764538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.764541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.764544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.764547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.764549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.764552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.764555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.764558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.764573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:41.770810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:41.771957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:41.771970Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:41.771972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:41.771976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:41.771983Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1906 [2:7559342763417613027:10422]
2025-10-09T21:50:41.771994Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:41.772015Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:41.772018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:41.772022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:41.772024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:41.772040Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:41.772052Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:41.772064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:41.772116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.772124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.772125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.772128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:41.772187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:41.772193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.772200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.772210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.772227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:41.772252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779445572:4092], now have 1 active actors on pipe
2025-10-09T21:50:41.772260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779445336:2432], now have 1 active actors on pipe
2025-10-09T21:50:41.772267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342763417613022:10422]
2025-10-09T21:50:41.772270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342763417613022:10422]
2025-10-09T21:50:41.772272Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342763417613022:10422]
2025-10-09T21:50:41.772284Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:41.772299Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:41.772301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:41.772307Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:41.772309Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:41.772315Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:41.772318Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:41.772340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342763417613022:10422]
2025-10-09T21:50:41.772348Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:41.772338Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1906
2025-10-09T21:50:41.772359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:41.772350Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1906
2025-10-09T21:50:41.772377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.772386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.772422Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e1920515-8c0d01e1-15ab250b-a634be6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:41.772433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:41.772438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:41.772440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:41.772451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:41.772465Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:41.772477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:41.772611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.772618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.772632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:41.772728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1907 partition: 0 MaxSeqNo: 25 sessionId: src_id|e1920515-8c0d01e1-15ab250b-a634be6_0
2025-10-09T21:50:41.773034Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|743b0e79-996b7b42-5841a875-76a53586_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e1920515-8c0d01e1-15ab250b-a634be6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:41.775447Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1920515-8c0d01e1-15ab250b-a634be6_0] Write session: try to update token
2025-10-09T21:50:41.775466Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1920515-8c0d01e1-15ab250b-a634be6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:41.786629Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:41.786649Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:41.786651Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:41.786653Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:41.786665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342763417613022:10422]
2025-10-09T21:50:41.786670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342763417613022:10422]
2025-10-09T21:50:41.786674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342763417613022:10422]
2025-10-09T21:50:41.786709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:41.786715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.786714Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:41.786716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:41.786719Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:41.786721Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:41.786843Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:41.787862Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1907 sessionId: src_id|e1920515-8c0d01e1-15ab250b-a634be6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:41.790193Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:41.845759Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342763779445606:4041]
2025-10-09T21:50:41.845979Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 1906, pipe: [1:7559342763779445606:4041]
2025-10-09T21:50:41.928892Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:41.929223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.929235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.929272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779445606:4041], now have 1 active actors on pipe
2025-10-09T21:50:41.929427Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:41.929467Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342763779445606:4041]
2025-10-09T21:50:41.929477Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:41.929482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:41.929499Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:41.929544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1907 reason: error at write: Disconnected sessionId: src_id|e1920515-8c0d01e1-15ab250b-a634be6_0
2025-10-09T21:50:41.929730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1907 sessionId: src_id|e1920515-8c0d01e1-15ab250b-a634be6_0 is DEAD
2025-10-09T21:50:41.929891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1920515-8c0d01e1-15ab250b-a634be6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:41.929905Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1920515-8c0d01e1-15ab250b-a634be6_0] Write session will restart in 0.006126s
2025-10-09T21:50:41.929926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1920515-8c0d01e1-15ab250b-a634be6_0] Write session: Do CDS request
2025-10-09T21:50:41.929931Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1920515-8c0d01e1-15ab250b-a634be6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:41.937425Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1920515-8c0d01e1-15ab250b-a634be6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:41.937534Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:41.937551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1908
2025-10-09T21:50:41.937657Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1908 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:41.937691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1908 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:41.937702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1908 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:41.937707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:41.938179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1908 sessionId:  describe result for acl check
2025-10-09T21:50:41.938213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:41.938254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:41.938261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:41.938267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342763779445666:4120] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:41.938815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342763779445666:4120] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:41.941715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342763779445666:4120] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:41.944164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342763779445666:4120] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:41.944177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342763779445666:4120] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:41.944181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342763779445666:4120] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:41.944188Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1908 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:41.947412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:41.947945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:41.948136Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:41.948147Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:41.948160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:41.948213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:41.948252Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342763417613302:10438]
2025-10-09T21:50:41.948734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:41.948985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:41.949032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:41.949342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:41.949397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.949403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.949404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.949405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.949406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.949407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.949409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.949410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.949411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.949416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.949428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.949432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.949435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.949450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.949453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.949457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.949460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.949464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.949472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.949482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:41.949489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.949494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:41.949498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.949504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:41.949508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.949514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:41.949521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.949527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:41.949532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.949537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:41.949544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.949550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.949554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.949559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.949565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.949577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.949606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:41.956395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:41.957681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:41.957698Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:41.957701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:41.957705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:41.957711Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1907 [2:7559342763417613302:10438]
2025-10-09T21:50:41.957726Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:41.957746Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:41.957750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:41.957754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:41.957756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:41.957775Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:41.957782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:41.957789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:41.957853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.957861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.957866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.957869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:41.957946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:41.957956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.957969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:41.957981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:41.957999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:41.958048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779445610:2432], now have 1 active actors on pipe
2025-10-09T21:50:41.958059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342763779445842:4120], now have 1 active actors on pipe
2025-10-09T21:50:41.958067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342763417613297:10438]
2025-10-09T21:50:41.958069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342763417613297:10438]
2025-10-09T21:50:41.958072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342763417613297:10438]
2025-10-09T21:50:41.958087Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:41.958104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:41.958107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:41.958110Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:41.958113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:41.958114Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1907
2025-10-09T21:50:41.958120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:41.958116Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1907
2025-10-09T21:50:41.958123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:41.958164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342763417613297:10438]
2025-10-09T21:50:41.958167Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:41.958180Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:41.958200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.958210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.958244Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|92071e08-8f5de0a4-ce619aad-e355ce95_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:41.958258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:41.958262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:41.958265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:41.958276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:41.958292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:41.958314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:41.958480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:41.958489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:41.958504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:41.958601Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1908 partition: 0 MaxSeqNo: 25 sessionId: src_id|92071e08-8f5de0a4-ce619aad-e355ce95_0
2025-10-09T21:50:41.958952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1920515-8c0d01e1-15ab250b-a634be6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|92071e08-8f5de0a4-ce619aad-e355ce95_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:41.961284Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92071e08-8f5de0a4-ce619aad-e355ce95_0] Write session: try to update token
2025-10-09T21:50:41.961299Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92071e08-8f5de0a4-ce619aad-e355ce95_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:41.972396Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:41.972414Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:41.972417Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:41.972422Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:41.972431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342763417613297:10438]
2025-10-09T21:50:41.972435Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342763417613297:10438]
2025-10-09T21:50:41.972439Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342763417613297:10438]
2025-10-09T21:50:41.972480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:41.972485Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:41.972487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:41.972489Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:41.972489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:41.972493Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:41.972626Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:41.973073Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1908 sessionId: src_id|92071e08-8f5de0a4-ce619aad-e355ce95_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:41.975434Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:42.029574Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342768074413152:4041]
2025-10-09T21:50:42.029844Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 1907, pipe: [1:7559342768074413152:4041]
2025-10-09T21:50:42.122797Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:42.123218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:42.123233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:42.123274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342768074413152:4041], now have 1 active actors on pipe
2025-10-09T21:50:42.123487Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:42.123527Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342768074413152:4041]
2025-10-09T21:50:42.123547Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:42.123550Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:42.123618Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1908 reason: error at write: Disconnected sessionId: src_id|92071e08-8f5de0a4-ce619aad-e355ce95_0
2025-10-09T21:50:42.123803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1908 sessionId: src_id|92071e08-8f5de0a4-ce619aad-e355ce95_0 is DEAD
2025-10-09T21:50:42.123965Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92071e08-8f5de0a4-ce619aad-e355ce95_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:42.123981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92071e08-8f5de0a4-ce619aad-e355ce95_0] Write session will restart in 0.007699s
2025-10-09T21:50:42.123999Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92071e08-8f5de0a4-ce619aad-e355ce95_0] Write session: Do CDS request
2025-10-09T21:50:42.124009Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92071e08-8f5de0a4-ce619aad-e355ce95_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:42.124038Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:42.132568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92071e08-8f5de0a4-ce619aad-e355ce95_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:42.132646Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:42.132662Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1909
2025-10-09T21:50:42.132776Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1909 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:42.132816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1909 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:42.132824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1909 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:42.132827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:42.133239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1909 sessionId:  describe result for acl check
2025-10-09T21:50:42.133267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:42.133268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:42.133269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:42.133274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342768074413189:4136] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:42.133741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342768074413189:4136] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:42.136572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342768074413189:4136] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:42.139412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342768074413189:4136] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:42.139427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342768074413189:4136] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:42.139430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342768074413189:4136] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:42.139439Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1909 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:42.143612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:42.144113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:42.144329Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:42.144341Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:42.144355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:42.144422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:42.144474Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342767712580835:10440]
2025-10-09T21:50:42.145073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:42.145360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:42.145406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:42.145732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:42.145807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.145816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.145817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.145819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.145820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.145821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.145822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.145824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.145825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.145830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.145842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.145847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.145852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.145856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.145860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.145864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.145868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.145872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.145881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.145892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.145901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.145909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.145921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.145931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.145937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.145944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.145949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.145958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.145963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.145972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.145977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.145986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.145990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.146001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.146007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.146013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.146061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:42.153911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:42.155100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:42.155115Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:42.155116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:42.155121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:42.155130Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1908 [2:7559342767712580835:10440]
2025-10-09T21:50:42.155145Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:42.155164Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:42.155167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:42.155171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:42.155173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:42.155190Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:42.155197Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:42.155205Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:42.155262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.155271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.155273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.155275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:42.155344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:42.155350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.155358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.155364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.155373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:42.155391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342768074413386:4136], now have 1 active actors on pipe
2025-10-09T21:50:42.155401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342767712580830:10440]
2025-10-09T21:50:42.155403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342767712580830:10440]
2025-10-09T21:50:42.155404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342767712580830:10440]
2025-10-09T21:50:42.155412Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:42.155423Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:42.155427Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:42.155433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:42.155436Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:42.155441Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:42.155444Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:42.155451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342768074413154:2432], now have 1 active actors on pipe
2025-10-09T21:50:42.155482Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342767712580830:10440]
2025-10-09T21:50:42.155470Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1908
2025-10-09T21:50:42.155492Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:42.155484Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1908
2025-10-09T21:50:42.155495Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:42.155523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:42.155527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:42.155763Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7cc58ef1-6ab64ac0-e69d5e74-24ecf8d7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:42.155782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:42.155786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:42.155788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:42.155799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:42.155809Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:42.155836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:42.155988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:42.155997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:42.156012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:42.156095Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1909 partition: 0 MaxSeqNo: 25 sessionId: src_id|7cc58ef1-6ab64ac0-e69d5e74-24ecf8d7_0
2025-10-09T21:50:42.156487Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92071e08-8f5de0a4-ce619aad-e355ce95_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7cc58ef1-6ab64ac0-e69d5e74-24ecf8d7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:42.159133Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7cc58ef1-6ab64ac0-e69d5e74-24ecf8d7_0] Write session: try to update token
2025-10-09T21:50:42.159150Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7cc58ef1-6ab64ac0-e69d5e74-24ecf8d7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:42.173665Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1909 sessionId: src_id|7cc58ef1-6ab64ac0-e69d5e74-24ecf8d7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:42.175273Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:42.175293Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:42.175296Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:42.175298Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:42.175308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342767712580830:10440]
2025-10-09T21:50:42.175315Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342767712580830:10440]
2025-10-09T21:50:42.175320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342767712580830:10440]
2025-10-09T21:50:42.175363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:42.175373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.175375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:42.175377Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:42.175385Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:42.175389Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:42.175524Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:42.176073Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:42.223626Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342768074413404:4041]
2025-10-09T21:50:42.223938Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 1908, pipe: [1:7559342768074413404:4041]
2025-10-09T21:50:42.322394Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:42.322719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:42.322732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:42.322776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342768074413404:4041], now have 1 active actors on pipe
2025-10-09T21:50:42.322964Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:42.323014Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342768074413404:4041]
2025-10-09T21:50:42.323017Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:42.323051Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:42.323129Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1909 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7cc58ef1-6ab64ac0-e69d5e74-24ecf8d7_0
2025-10-09T21:50:42.323292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1909 sessionId: src_id|7cc58ef1-6ab64ac0-e69d5e74-24ecf8d7_0 is DEAD
2025-10-09T21:50:42.323421Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cc58ef1-6ab64ac0-e69d5e74-24ecf8d7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:42.323432Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cc58ef1-6ab64ac0-e69d5e74-24ecf8d7_0] Write session will restart in 0.006150s
2025-10-09T21:50:42.323453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cc58ef1-6ab64ac0-e69d5e74-24ecf8d7_0] Write session: Do CDS request
2025-10-09T21:50:42.323458Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cc58ef1-6ab64ac0-e69d5e74-24ecf8d7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:42.330458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7cc58ef1-6ab64ac0-e69d5e74-24ecf8d7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:42.330555Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:42.330572Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1910
2025-10-09T21:50:42.330677Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1910 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:42.330702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1910 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:42.330710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1910 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:42.330712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:42.331053Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1910 sessionId:  describe result for acl check
2025-10-09T21:50:42.331077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:42.331080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:42.331081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:42.331086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342768074413435:4157] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:42.331536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342768074413435:4157] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:42.333865Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342768074413435:4157] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:42.336523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342768074413435:4157] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:42.336538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342768074413435:4157] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:42.336541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342768074413435:4157] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:42.336549Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1910 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:42.339227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:42.341541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:42.341698Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:42.341706Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:42.341716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:42.341762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:42.341799Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342767712581073:10442]
2025-10-09T21:50:42.342199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:42.342484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:42.342521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:42.342730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:42.342772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.342778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.342779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.342780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.342782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.342783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.342784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.342785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.342786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.342791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.342800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.342802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.342804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.342806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.342808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.342811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.342813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.342815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.342821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.342827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.342831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.342835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.342838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.342840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.342843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.342846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.342848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.342851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.342854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.342857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.342860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.342863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.342865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.342868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.342871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.342874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.342892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:42.350902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:42.352278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:42.352291Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:42.352293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:42.352296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:42.352302Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1909 [2:7559342767712581073:10442]
2025-10-09T21:50:42.352312Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:42.352328Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:42.352329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:42.352332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:42.352334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:42.352348Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:42.352356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:42.352363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:42.352410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.352418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.352419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.352421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:42.352475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:42.352481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.352490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.352502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.352522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:42.352539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342768074413599:4157], now have 1 active actors on pipe
2025-10-09T21:50:42.352546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342768074413411:2432], now have 1 active actors on pipe
2025-10-09T21:50:42.352552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342767712581068:10442]
2025-10-09T21:50:42.352554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342767712581068:10442]
2025-10-09T21:50:42.352555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342767712581068:10442]
2025-10-09T21:50:42.352564Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:42.352575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:42.352576Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:42.352579Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:42.352580Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:42.352587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:42.352589Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:42.352617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342767712581068:10442]
2025-10-09T21:50:42.352625Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:42.352630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:42.352683Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1909
2025-10-09T21:50:42.352694Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1909
2025-10-09T21:50:42.352790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:42.352803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:42.352837Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|46e71373-69133eb3-9bec0e6a-f1edb6c7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:42.352850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:42.352854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:42.352858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:42.352869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:42.352884Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:42.352907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:42.353221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:42.353229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:42.353243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:42.353338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1910 partition: 0 MaxSeqNo: 25 sessionId: src_id|46e71373-69133eb3-9bec0e6a-f1edb6c7_0
2025-10-09T21:50:42.353730Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7cc58ef1-6ab64ac0-e69d5e74-24ecf8d7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|46e71373-69133eb3-9bec0e6a-f1edb6c7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:42.356244Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46e71373-69133eb3-9bec0e6a-f1edb6c7_0] Write session: try to update token
2025-10-09T21:50:42.356262Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46e71373-69133eb3-9bec0e6a-f1edb6c7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:42.370566Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1910 sessionId: src_id|46e71373-69133eb3-9bec0e6a-f1edb6c7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:42.372225Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:42.372247Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:42.372250Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:42.372252Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:42.372262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342767712581068:10442]
2025-10-09T21:50:42.372266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342767712581068:10442]
2025-10-09T21:50:42.372270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342767712581068:10442]
2025-10-09T21:50:42.372315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:42.372327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.372330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:42.372337Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:42.372354Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:42.372362Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:42.372466Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:42.372840Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:42.423288Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342768074413659:4041]
2025-10-09T21:50:42.423569Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 1909, pipe: [1:7559342768074413659:4041]
2025-10-09T21:50:42.517757Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:42.518185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:42.518200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:42.518239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342768074413659:4041], now have 1 active actors on pipe
2025-10-09T21:50:42.518410Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:42.518502Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342768074413659:4041]
2025-10-09T21:50:42.518517Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:42.518534Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:42.518662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1910 reason: status is not ok: tablet will be restarted right now sessionId: src_id|46e71373-69133eb3-9bec0e6a-f1edb6c7_0
2025-10-09T21:50:42.518846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1910 sessionId: src_id|46e71373-69133eb3-9bec0e6a-f1edb6c7_0 is DEAD
2025-10-09T21:50:42.519004Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46e71373-69133eb3-9bec0e6a-f1edb6c7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:42.519016Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46e71373-69133eb3-9bec0e6a-f1edb6c7_0] Write session will restart in 0.007715s
2025-10-09T21:50:42.519053Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46e71373-69133eb3-9bec0e6a-f1edb6c7_0] Write session: Do CDS request
2025-10-09T21:50:42.519060Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46e71373-69133eb3-9bec0e6a-f1edb6c7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:42.527460Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46e71373-69133eb3-9bec0e6a-f1edb6c7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:42.527590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:42.527608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1911
2025-10-09T21:50:42.527735Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1911 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:42.527768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1911 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:42.527779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1911 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:42.527782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:42.528219Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1911 sessionId:  describe result for acl check
2025-10-09T21:50:42.528257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:42.528263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:42.528265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:42.528271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342768074413714:4151] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:42.528907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342768074413714:4151] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:42.531591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342768074413714:4151] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:42.534477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342768074413714:4151] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:42.534490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342768074413714:4151] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:42.534493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342768074413714:4151] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:42.534498Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1911 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:42.535399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:42.537909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:42.538109Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:42.538118Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:42.538129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:42.538181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:42.538225Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342767712581312:10444]
2025-10-09T21:50:42.538801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:42.539130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:42.539174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:42.539473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:42.539529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.539535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.539536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.539538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.539539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.539540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.539541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.539543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.539544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.539548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.539560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.539564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.539568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.539572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.539577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.539581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.539584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.539588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.539594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.539603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.539609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.539619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.539629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.539639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.539646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.539656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.539660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.539666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.539670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.539680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.539690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.539698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.539713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.539722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.539727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.539732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.539762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:42.547831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:42.549300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:42.549315Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:42.549318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:42.549322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:42.549329Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1910 [2:7559342767712581312:10444]
2025-10-09T21:50:42.549343Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:42.549362Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:42.549365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:42.549368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:42.549371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:42.549389Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:42.549395Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:42.549403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:42.549476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.549485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.549487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.549490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:42.549556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:42.549564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.549576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.549589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.549611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:42.549638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342768074413897:4151], now have 1 active actors on pipe
2025-10-09T21:50:42.549646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342768074413662:2432], now have 1 active actors on pipe
2025-10-09T21:50:42.549653Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342767712581307:10444]
2025-10-09T21:50:42.549660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342767712581307:10444]
2025-10-09T21:50:42.549663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342767712581307:10444]
2025-10-09T21:50:42.549675Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:42.549694Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:42.549696Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:42.549702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:42.549704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:42.549714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:42.549717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:42.549751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342767712581307:10444]
2025-10-09T21:50:42.549762Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:42.549719Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1910
2025-10-09T21:50:42.549734Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1910
2025-10-09T21:50:42.549768Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:42.549783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:42.549793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:42.549841Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ed72b40-50ede447-89c54589-731101a9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:42.549855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:42.549859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:42.549861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:42.549872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:42.549881Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:42.549901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:42.550065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:42.550077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:42.550096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:42.550187Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1911 partition: 0 MaxSeqNo: 25 sessionId: src_id|ed72b40-50ede447-89c54589-731101a9_0
2025-10-09T21:50:42.550517Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46e71373-69133eb3-9bec0e6a-f1edb6c7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ed72b40-50ede447-89c54589-731101a9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:42.552862Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ed72b40-50ede447-89c54589-731101a9_0] Write session: try to update token
2025-10-09T21:50:42.552879Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ed72b40-50ede447-89c54589-731101a9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:42.565200Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1911 sessionId: src_id|ed72b40-50ede447-89c54589-731101a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:42.566866Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:42.566883Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:42.566884Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:42.566886Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:42.566894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342767712581307:10444]
2025-10-09T21:50:42.566897Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342767712581307:10444]
2025-10-09T21:50:42.566901Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342767712581307:10444]
2025-10-09T21:50:42.566943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:42.566953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.566955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:42.566959Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:42.566966Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:42.566972Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:42.567074Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:42.567704Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:42.618880Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342768074413910:4041]
2025-10-09T21:50:42.619174Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 1910, pipe: [1:7559342768074413910:4041]
2025-10-09T21:50:42.712278Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:42.712561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:42.712589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:42.712646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342768074413910:4041], now have 1 active actors on pipe
2025-10-09T21:50:42.712840Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:42.712866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:42.712892Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:42.712893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:42.712929Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342768074413910:4041]
2025-10-09T21:50:42.712957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1911 reason: error at write: Disconnected sessionId: src_id|ed72b40-50ede447-89c54589-731101a9_0
2025-10-09T21:50:42.713130Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1911 sessionId: src_id|ed72b40-50ede447-89c54589-731101a9_0 is DEAD
2025-10-09T21:50:42.713324Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed72b40-50ede447-89c54589-731101a9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:42.713336Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed72b40-50ede447-89c54589-731101a9_0] Write session will restart in 0.005351s
2025-10-09T21:50:42.713363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed72b40-50ede447-89c54589-731101a9_0] Write session: Do CDS request
2025-10-09T21:50:42.713370Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed72b40-50ede447-89c54589-731101a9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:42.719449Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ed72b40-50ede447-89c54589-731101a9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:42.719529Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:42.719554Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1912
2025-10-09T21:50:42.719675Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1912 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:42.719700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1912 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:42.719704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1912 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:42.719709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:42.720146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1912 sessionId:  describe result for acl check
2025-10-09T21:50:42.720186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:42.720193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:42.720195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:42.720201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342768074413929:4167] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:42.720763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342768074413929:4167] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:42.723290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342768074413929:4167] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:42.726258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342768074413929:4167] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:42.726271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342768074413929:4167] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:42.726275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342768074413929:4167] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:42.726290Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1912 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:42.729024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:42.731610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:42.731800Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:42.731811Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:42.731823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:42.731876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:42.731919Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342767712581552:10446]
2025-10-09T21:50:42.732529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:42.732806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:42.732845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:42.733292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:42.733346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.733353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.733354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.733356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.733357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.733358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.733359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.733360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.733361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.733366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.733378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.733382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.733386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.733390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.733394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.733397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.733401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.733405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.733413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.733421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.733427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.733433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.733448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.733454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.733458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.733463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.733468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.733473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.733479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.733485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.733490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.733496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.733500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.733505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.733509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.733514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.733543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:42.741318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:42.742518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:42.742531Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:42.742533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:42.742536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:42.742540Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1911 [2:7559342767712581552:10446]
2025-10-09T21:50:42.742550Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:42.742563Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:42.742565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:42.742567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:42.742569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:42.742582Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:42.742586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:42.742591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:42.742632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.742637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.742639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.742641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:42.742690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:42.742697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.742708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.742720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.742732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:42.742747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342768074413913:2432], now have 1 active actors on pipe
2025-10-09T21:50:42.742754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342768074414092:4167], now have 1 active actors on pipe
2025-10-09T21:50:42.742760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342767712581547:10446]
2025-10-09T21:50:42.742761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342767712581547:10446]
2025-10-09T21:50:42.742763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342767712581547:10446]
2025-10-09T21:50:42.742769Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:42.742779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:42.742780Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:42.742782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:42.742784Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:42.742789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:42.742791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:42.742815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342767712581547:10446]
2025-10-09T21:50:42.742824Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:42.742840Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:42.742831Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1911
2025-10-09T21:50:42.742868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:42.742876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:42.742840Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1911
2025-10-09T21:50:42.742897Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ed313f60-4428e19-50ae8077-5f9f81d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:42.742910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:42.742914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:42.742916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:42.742923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:42.742930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:42.742940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:42.743088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:42.743095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:42.743107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:42.743188Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1912 partition: 0 MaxSeqNo: 25 sessionId: src_id|ed313f60-4428e19-50ae8077-5f9f81d_0
2025-10-09T21:50:42.743504Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed72b40-50ede447-89c54589-731101a9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ed313f60-4428e19-50ae8077-5f9f81d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:42.745832Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ed313f60-4428e19-50ae8077-5f9f81d_0] Write session: try to update token
2025-10-09T21:50:42.745844Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ed313f60-4428e19-50ae8077-5f9f81d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:42.758067Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1912 sessionId: src_id|ed313f60-4428e19-50ae8077-5f9f81d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:42.760404Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:42.760605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:42.760627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:42.760668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:42.760748Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:42.763287Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:42.763305Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:42.763307Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:42.763309Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:42.763318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342767712581547:10446]
2025-10-09T21:50:42.763323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342767712581547:10446]
2025-10-09T21:50:42.763326Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342767712581547:10446]
2025-10-09T21:50:42.763366Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:42.763369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:42.763376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.763378Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:42.763379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:42.763381Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:42.763500Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:42.812931Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342768074414162:4041]
2025-10-09T21:50:42.813176Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 1911, pipe: [1:7559342768074414162:4041]
2025-10-09T21:50:42.913623Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:42.921156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:42.921183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:42.921335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:42.921400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:42.921491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:42.921572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:42.921811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:42.921824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:42.921900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:42.921963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:42.922041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:42.922117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:42.922166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:42.922176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:42.922234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:42.922288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:42.922341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:42.922393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:42.922434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:42.922436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:42.922492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:42.922555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:42.922622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:42.922681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:42.922725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:42.922732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:42.922789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:42.922848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:42.922913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:42.922969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:42.923008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:42.923013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:42.923065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:42.923113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:42.923168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:42.923247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:42.923294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:42.923301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:42.923363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:42.923413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:42.923477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:42.923582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:42.923625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:42.923634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:42.923696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:42.923781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:42.923837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:42.923889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:42.923938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:42.923946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:42.924007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:42.924064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:42.924120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:42.924185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:42.924228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:42.924236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:42.924298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:42.924358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:42.924413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:42.924469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:42.924511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:42.924521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:42.924582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342768074414162:4041], now have 1 active actors on pipe
2025-10-09T21:50:42.924787Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:42.924805Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:42.924807Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342768074414162:4041]
2025-10-09T21:50:42.924835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1912 reason: error at write: Disconnected sessionId: src_id|ed313f60-4428e19-50ae8077-5f9f81d_0
2025-10-09T21:50:42.924841Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:42.925035Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1912 sessionId: src_id|ed313f60-4428e19-50ae8077-5f9f81d_0 is DEAD
2025-10-09T21:50:42.925188Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed313f60-4428e19-50ae8077-5f9f81d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:42.925199Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed313f60-4428e19-50ae8077-5f9f81d_0] Write session will restart in 0.005471s
2025-10-09T21:50:42.925224Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed313f60-4428e19-50ae8077-5f9f81d_0] Write session: Do CDS request
2025-10-09T21:50:42.925230Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed313f60-4428e19-50ae8077-5f9f81d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:42.925346Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:42.931416Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ed313f60-4428e19-50ae8077-5f9f81d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:42.931485Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:42.931498Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1913
2025-10-09T21:50:42.931637Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1913 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:42.931661Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1913 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:42.931668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1913 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:42.931671Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:42.932366Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1913 sessionId:  describe result for acl check
2025-10-09T21:50:42.932398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:42.932399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:42.932400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:42.932404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342768074414215:4202] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:42.932810Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342768074414215:4202] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:42.935794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342768074414215:4202] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:42.938870Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342768074414215:4202] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:42.938885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342768074414215:4202] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:42.938888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342768074414215:4202] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:42.938895Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1913 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:42.940489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:42.942615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:42.942777Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:42.942786Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:42.942796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:42.942841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:42.942880Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342767712581836:10462]
2025-10-09T21:50:42.943416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:42.943667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:42.943707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:42.943991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:42.944034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.944039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.944040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.944041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.944042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.944043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.944044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.944045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.944046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.944051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.944061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.944065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.944069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.944073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.944077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.944081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.944085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.944089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.944095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.944106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:42.944116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.944124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:42.944131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.944136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:42.944140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.944146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:42.944153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.944158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:42.944162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.944167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:42.944171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.944182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.944186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.944190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.944195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.944200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.944215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:42.951652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:42.952801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:42.952815Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:42.952817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:42.952821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:42.952826Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1912 [2:7559342767712581836:10462]
2025-10-09T21:50:42.952839Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:42.952855Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:42.952862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:42.952866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:42.952868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:42.952884Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:42.952893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:42.952900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:42.952955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.952962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.952964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.952966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:42.953017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:42.953024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.953035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:42.953048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:42.953069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:42.953091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342768074414424:4202], now have 1 active actors on pipe
2025-10-09T21:50:42.953099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342768074414191:2432], now have 1 active actors on pipe
2025-10-09T21:50:42.953106Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342767712581831:10462]
2025-10-09T21:50:42.953108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342767712581831:10462]
2025-10-09T21:50:42.953111Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342767712581831:10462]
2025-10-09T21:50:42.953123Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:42.953140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:42.953141Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:42.953144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:42.953146Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:42.953158Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:42.953162Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:42.953145Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1912
2025-10-09T21:50:42.953167Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1912
2025-10-09T21:50:42.953186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342767712581831:10462]
2025-10-09T21:50:42.953191Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:42.953211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:42.953219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:42.953260Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:42.953261Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bd0e34b1-a10c1ccb-93d94cfd-6d48f252_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:42.953266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:42.953270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:42.953272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:42.953282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:42.953291Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:42.953309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:42.953476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:42.953484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:42.953499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:42.953588Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1913 partition: 0 MaxSeqNo: 25 sessionId: src_id|bd0e34b1-a10c1ccb-93d94cfd-6d48f252_0
2025-10-09T21:50:42.953949Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ed313f60-4428e19-50ae8077-5f9f81d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bd0e34b1-a10c1ccb-93d94cfd-6d48f252_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:42.956329Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd0e34b1-a10c1ccb-93d94cfd-6d48f252_0] Write session: try to update token
2025-10-09T21:50:42.956346Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd0e34b1-a10c1ccb-93d94cfd-6d48f252_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:42.967552Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:42.967570Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:42.967572Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:42.967574Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:42.967584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342767712581831:10462]
2025-10-09T21:50:42.967588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342767712581831:10462]
2025-10-09T21:50:42.967592Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342767712581831:10462]
2025-10-09T21:50:42.967633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:42.967636Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:42.967640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:42.967642Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:42.967643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:42.967644Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:42.967773Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:42.968726Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1913 sessionId: src_id|bd0e34b1-a10c1ccb-93d94cfd-6d48f252_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:42.970957Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:43.024870Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342772369381735:4041]
2025-10-09T21:50:43.025146Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 1912, pipe: [1:7559342772369381735:4041]
2025-10-09T21:50:43.118242Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:43.118712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:43.118727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:43.118774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342772369381735:4041], now have 1 active actors on pipe
2025-10-09T21:50:43.118938Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:43.119052Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:43.119071Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:43.119090Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:43.119135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1913 reason: error at write: Disconnected sessionId: src_id|bd0e34b1-a10c1ccb-93d94cfd-6d48f252_0
2025-10-09T21:50:43.119154Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342772369381735:4041]
2025-10-09T21:50:43.119320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1913 sessionId: src_id|bd0e34b1-a10c1ccb-93d94cfd-6d48f252_0 is DEAD
2025-10-09T21:50:43.119479Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd0e34b1-a10c1ccb-93d94cfd-6d48f252_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:43.119491Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd0e34b1-a10c1ccb-93d94cfd-6d48f252_0] Write session will restart in 0.009888s
2025-10-09T21:50:43.119528Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd0e34b1-a10c1ccb-93d94cfd-6d48f252_0] Write session: Do CDS request
2025-10-09T21:50:43.119534Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd0e34b1-a10c1ccb-93d94cfd-6d48f252_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:43.130473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd0e34b1-a10c1ccb-93d94cfd-6d48f252_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:43.130525Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:43.130543Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1914
2025-10-09T21:50:43.130675Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1914 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:43.130707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1914 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:43.130718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1914 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:43.130722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:43.131407Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1914 sessionId:  describe result for acl check
2025-10-09T21:50:43.131443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:43.131449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:43.131451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:43.131457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342772369381855:4203] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:43.132003Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342772369381855:4203] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:43.134491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342772369381855:4203] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:43.135761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:43.137991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:43.138161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342772369381855:4203] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:43.138175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342772369381855:4203] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:43.138179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342772369381855:4203] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:43.138186Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1914 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:43.138186Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:43.138200Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:43.138213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:43.138269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:43.138308Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342772007549373:10464]
2025-10-09T21:50:43.138693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:43.138964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:43.138994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:43.139203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:43.139236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.139242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.139243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.139244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.139245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.139245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.139246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.139246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.139247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.139250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.139257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.139259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.139261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.139263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.139265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.139267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.139269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.139271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.139276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.139281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.139285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.139288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.139290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.139293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.139296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.139299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.139302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.139305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.139307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.139310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.139313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.139316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.139319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.139322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.139325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.139328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.139344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:43.146121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:43.147333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:43.147345Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:43.147346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:43.147350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:43.147355Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1913 [2:7559342772007549373:10464]
2025-10-09T21:50:43.147364Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:43.147379Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:43.147381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:43.147383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:43.147385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:43.147397Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:43.147401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:43.147407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:43.147453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.147459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.147461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.147463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:43.147522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:43.147529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.147537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.147544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.147560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:43.147576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342772369381979:4203], now have 1 active actors on pipe
2025-10-09T21:50:43.147583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342772369381737:2432], now have 1 active actors on pipe
2025-10-09T21:50:43.147588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342772007549368:10464]
2025-10-09T21:50:43.147590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342772007549368:10464]
2025-10-09T21:50:43.147591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342772007549368:10464]
2025-10-09T21:50:43.147602Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:43.147614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:43.147624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:43.147626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:43.147628Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:43.147629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:43.147630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:43.147652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342772007549368:10464]
2025-10-09T21:50:43.147659Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:43.147649Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1913
2025-10-09T21:50:43.147651Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1913
2025-10-09T21:50:43.147678Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:43.147707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:43.147715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:43.147739Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|624a9ef1-1887e1f2-9bac1ead-7e4aab36_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:43.147752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:43.147760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:43.147763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:43.147770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:43.147780Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:43.147793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:43.147927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:43.147933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:43.147943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:43.148038Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1914 partition: 0 MaxSeqNo: 25 sessionId: src_id|624a9ef1-1887e1f2-9bac1ead-7e4aab36_0
2025-10-09T21:50:43.148356Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd0e34b1-a10c1ccb-93d94cfd-6d48f252_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|624a9ef1-1887e1f2-9bac1ead-7e4aab36_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:43.150975Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|624a9ef1-1887e1f2-9bac1ead-7e4aab36_0] Write session: try to update token
2025-10-09T21:50:43.150994Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|624a9ef1-1887e1f2-9bac1ead-7e4aab36_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:43.160333Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:43.160353Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:43.160362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342772007549368:10464]
2025-10-09T21:50:43.160408Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:43.160409Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:43.160587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:43.160725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:43.160862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:43.161037Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:43.161228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:43.161270Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:43.163885Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1914 sessionId: src_id|624a9ef1-1887e1f2-9bac1ead-7e4aab36_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:43.166231Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:43.166410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:43.166426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:43.166458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:43.166542Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:43.173756Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:43.173770Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:43.173771Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:43.173773Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:43.173781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342772007549368:10464]
2025-10-09T21:50:43.173784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342772007549368:10464]
2025-10-09T21:50:43.173787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342772007549368:10464]
2025-10-09T21:50:43.173820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:43.173825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.173827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:43.173831Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:43.173837Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:43.173841Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:43.173958Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:43.219401Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342772369381994:4041]
2025-10-09T21:50:43.219700Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 1913, pipe: [1:7559342772369381994:4041]
2025-10-09T21:50:43.317163Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:43.324439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:43.324466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:43.324633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:43.324697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:43.324771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:43.324836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:43.324891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:43.324899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:43.324972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:43.325043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:43.325109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:43.325178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:43.325221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:43.325230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:43.325294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:43.325346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:43.325401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:43.325467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:43.325513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:43.325521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:43.325582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:43.325637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:43.325690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:43.325775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:43.325820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:43.325829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:43.325883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:43.325935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:43.326025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:43.326105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:43.326144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:43.326151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:43.326234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:43.326322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:43.326372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:43.326456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:43.326505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:43.326510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:43.326601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:43.326686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:43.326773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:43.326834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:43.326873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:43.326879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:43.326932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:43.326984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:43.327046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:43.327102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:43.327141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:43.327146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:43.327199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:43.327261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:43.327319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:43.327377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:43.327415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:43.327421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:43.327475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:43.327527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:43.327588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:43.327643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:43.327695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:43.327702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:43.327742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342772369381994:4041], now have 1 active actors on pipe
2025-10-09T21:50:43.327976Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:43.327977Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:43.327981Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:43.328015Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1914 reason: error at write: Disconnected sessionId: src_id|624a9ef1-1887e1f2-9bac1ead-7e4aab36_0
2025-10-09T21:50:43.328045Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342772369381994:4041]
2025-10-09T21:50:43.328195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1914 sessionId: src_id|624a9ef1-1887e1f2-9bac1ead-7e4aab36_0 is DEAD
2025-10-09T21:50:43.328352Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|624a9ef1-1887e1f2-9bac1ead-7e4aab36_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:43.328364Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|624a9ef1-1887e1f2-9bac1ead-7e4aab36_0] Write session will restart in 0.006837s
2025-10-09T21:50:43.328403Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|624a9ef1-1887e1f2-9bac1ead-7e4aab36_0] Write session: Do CDS request
2025-10-09T21:50:43.328410Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|624a9ef1-1887e1f2-9bac1ead-7e4aab36_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:43.328640Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:43.336418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|624a9ef1-1887e1f2-9bac1ead-7e4aab36_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:43.336516Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:43.336530Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1915
2025-10-09T21:50:43.336633Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1915 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:43.336665Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1915 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:43.336677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1915 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:43.336681Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:43.337344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1915 sessionId:  describe result for acl check
2025-10-09T21:50:43.337377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:43.337385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:43.337387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:43.337392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342772369382041:4205] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:43.337924Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342772369382041:4205] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:43.341098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342772369382041:4205] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:43.343916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342772369382041:4205] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:43.343930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342772369382041:4205] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:43.343934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342772369382041:4205] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:43.343942Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1915 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:43.345140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:43.347365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:43.347573Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:43.347584Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:43.347596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:43.347645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:43.347687Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342772007549617:10466]
2025-10-09T21:50:43.348300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:43.348560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:43.348598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:43.348905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:43.348951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.348958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.348959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.348960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.348961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.348962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.348964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.348965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.348966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.348971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.348982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.348986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.348991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.348995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.348999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.349003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.349007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.349011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.349017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.349025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.349032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.349042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.349050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.349056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.349060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.349070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.349078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.349084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.349093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.349098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.349103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.349108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.349117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.349122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.349131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.349136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.349169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:43.355868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:43.357057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:43.357072Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:43.357075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:43.357079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:43.357087Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1914 [2:7559342772007549617:10466]
2025-10-09T21:50:43.357100Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:43.357117Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:43.357120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:43.357123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:43.357126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:43.357143Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:43.357152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:43.357159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:43.357217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.357224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.357226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.357228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:43.357293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:43.357301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.357313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.357324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.357341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:43.357362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342772369382001:2432], now have 1 active actors on pipe
2025-10-09T21:50:43.357370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342772369382240:4205], now have 1 active actors on pipe
2025-10-09T21:50:43.357377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342772007549612:10466]
2025-10-09T21:50:43.357380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342772007549612:10466]
2025-10-09T21:50:43.357382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342772007549612:10466]
2025-10-09T21:50:43.357397Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:43.357416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:43.357421Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:43.357425Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:43.357429Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:43.357434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:43.357450Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:43.357424Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1914
2025-10-09T21:50:43.357450Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1914
2025-10-09T21:50:43.357479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342772007549612:10466]
2025-10-09T21:50:43.357488Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:43.357491Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:43.357517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:43.357527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:43.357559Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|97849bd7-e1bf12ce-dee1e1ca-b08280d3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:43.357570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:43.357574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:43.357576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:43.357586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:43.357600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:43.357624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:43.357791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:43.357800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:43.357814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:43.357922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1915 partition: 0 MaxSeqNo: 25 sessionId: src_id|97849bd7-e1bf12ce-dee1e1ca-b08280d3_0
2025-10-09T21:50:43.358244Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|624a9ef1-1887e1f2-9bac1ead-7e4aab36_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|97849bd7-e1bf12ce-dee1e1ca-b08280d3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:43.360511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97849bd7-e1bf12ce-dee1e1ca-b08280d3_0] Write session: try to update token
2025-10-09T21:50:43.360524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97849bd7-e1bf12ce-dee1e1ca-b08280d3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:43.372263Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:43.372282Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:43.372284Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:43.372286Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:43.372295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342772007549612:10466]
2025-10-09T21:50:43.372300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342772007549612:10466]
2025-10-09T21:50:43.372305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342772007549612:10466]
2025-10-09T21:50:43.372348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:43.372353Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:43.372359Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:43.372360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.372362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:43.372366Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:43.372486Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:43.373113Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1915 sessionId: src_id|97849bd7-e1bf12ce-dee1e1ca-b08280d3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:43.375488Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:43.427862Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342772369382254:4041]
2025-10-09T21:50:43.428150Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 1914, pipe: [1:7559342772369382254:4041]
2025-10-09T21:50:43.517538Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:43.518154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:43.518173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:43.518219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342772369382254:4041], now have 1 active actors on pipe
2025-10-09T21:50:43.518499Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:43.518503Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:43.518563Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342772369382254:4041]
2025-10-09T21:50:43.518592Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:43.518606Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:43.518713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1915 reason: error at write: Disconnected sessionId: src_id|97849bd7-e1bf12ce-dee1e1ca-b08280d3_0
2025-10-09T21:50:43.518912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1915 sessionId: src_id|97849bd7-e1bf12ce-dee1e1ca-b08280d3_0 is DEAD
2025-10-09T21:50:43.519059Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97849bd7-e1bf12ce-dee1e1ca-b08280d3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:43.519071Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97849bd7-e1bf12ce-dee1e1ca-b08280d3_0] Write session will restart in 0.008897s
2025-10-09T21:50:43.519104Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97849bd7-e1bf12ce-dee1e1ca-b08280d3_0] Write session: Do CDS request
2025-10-09T21:50:43.519111Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97849bd7-e1bf12ce-dee1e1ca-b08280d3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:43.529543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97849bd7-e1bf12ce-dee1e1ca-b08280d3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:43.529693Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:43.529714Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1916
2025-10-09T21:50:43.529845Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1916 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:43.529887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1916 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:43.529900Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1916 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:43.529909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:43.530530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1916 sessionId:  describe result for acl check
2025-10-09T21:50:43.530576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:43.530582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:43.530584Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:43.530590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342772369382364:4221] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:43.531318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342772369382364:4221] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:43.533571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342772369382364:4221] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:43.534908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:43.537294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:43.537483Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:43.537493Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:43.537504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:43.537553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:43.537592Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342772007549860:10468]
2025-10-09T21:50:43.537595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342772369382364:4221] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:43.537607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342772369382364:4221] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:43.537609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342772369382364:4221] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:43.537615Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1916 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:43.538138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:43.538476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:43.538526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:43.538766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:43.538812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.538818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.538819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.538820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.538821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.538822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.538824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.538825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.538826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.538831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.538842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.538847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.538851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.538856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.538859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.538864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.538867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.538870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.538876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.538884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.538895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.538900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.538905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.538915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.538921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.538932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.538936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.538941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.538945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.538950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.538956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.538961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.538965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.538971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.538980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.538985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.539015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:43.551079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:43.552525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:43.552539Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:43.552540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:43.552544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:43.552550Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1915 [2:7559342772007549860:10468]
2025-10-09T21:50:43.552564Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:43.552580Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:43.552582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:43.552585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:43.552586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:43.552601Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:43.552605Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:43.552610Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:43.552662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.552668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.552669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.552671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:43.552732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:43.552738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.552745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.552752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.552761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:43.552780Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342772007549855:10468]
2025-10-09T21:50:43.552785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342772007549855:10468]
2025-10-09T21:50:43.552786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342772007549855:10468]
2025-10-09T21:50:43.552796Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:43.552808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:43.552814Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:43.552816Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:43.552820Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:43.552822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:43.552824Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:43.552833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342772369382499:4221], now have 1 active actors on pipe
2025-10-09T21:50:43.552835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342772369382257:2432], now have 1 active actors on pipe
2025-10-09T21:50:43.552864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342772007549855:10468]
2025-10-09T21:50:43.552875Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:43.552898Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:43.552909Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1915
2025-10-09T21:50:43.552914Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1915
2025-10-09T21:50:43.553025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:43.553033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:43.553056Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6649ff14-7a253902-64e2d515-a435889c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:43.553062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:43.553065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:43.553067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:43.553076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:43.553091Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:43.553105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:43.553275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:43.553281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:43.553292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:43.553380Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1916 partition: 0 MaxSeqNo: 25 sessionId: src_id|6649ff14-7a253902-64e2d515-a435889c_0
2025-10-09T21:50:43.553767Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97849bd7-e1bf12ce-dee1e1ca-b08280d3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6649ff14-7a253902-64e2d515-a435889c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:43.563089Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6649ff14-7a253902-64e2d515-a435889c_0] Write session: try to update token
2025-10-09T21:50:43.563116Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6649ff14-7a253902-64e2d515-a435889c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:43.571261Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:43.571284Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:43.571287Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:43.571289Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:43.571300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342772007549855:10468]
2025-10-09T21:50:43.571303Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342772007549855:10468]
2025-10-09T21:50:43.571306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342772007549855:10468]
2025-10-09T21:50:43.571350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:43.571352Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:43.571358Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:43.571362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.571365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:43.571368Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:43.571516Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:43.576787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1916 sessionId: src_id|6649ff14-7a253902-64e2d515-a435889c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:43.583906Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:43.618825Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342772369382510:4041]
2025-10-09T21:50:43.619145Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 1915, pipe: [1:7559342772369382510:4041]
2025-10-09T21:50:43.714729Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:43.715026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:43.715038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:43.715059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342772369382510:4041], now have 1 active actors on pipe
2025-10-09T21:50:43.715251Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:43.715289Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:43.715298Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:43.715304Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:43.715345Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342772369382510:4041]
2025-10-09T21:50:43.715372Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1916 reason: error at write: Disconnected sessionId: src_id|6649ff14-7a253902-64e2d515-a435889c_0
2025-10-09T21:50:43.715525Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1916 sessionId: src_id|6649ff14-7a253902-64e2d515-a435889c_0 is DEAD
2025-10-09T21:50:43.715686Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6649ff14-7a253902-64e2d515-a435889c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:43.715696Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6649ff14-7a253902-64e2d515-a435889c_0] Write session will restart in 0.009674s
2025-10-09T21:50:43.715717Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6649ff14-7a253902-64e2d515-a435889c_0] Write session: Do CDS request
2025-10-09T21:50:43.715721Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6649ff14-7a253902-64e2d515-a435889c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:43.726537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6649ff14-7a253902-64e2d515-a435889c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:43.726631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:43.726647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1917
2025-10-09T21:50:43.726747Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1917 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:43.726773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1917 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:43.726778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1917 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:43.726780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:43.727113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1917 sessionId:  describe result for acl check
2025-10-09T21:50:43.727141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:43.727148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:43.727149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:43.727153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342772369382650:4226] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:43.727573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342772369382650:4226] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:43.729927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342772369382650:4226] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:43.731091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:43.733271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:43.733461Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:43.733471Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:43.733482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:43.733535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:43.733573Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342772007550105:10470]
2025-10-09T21:50:43.733607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342772369382650:4226] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:43.733621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342772369382650:4226] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:43.733625Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342772369382650:4226] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:43.733633Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1917 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:43.734237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:43.734543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:43.734588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:43.734934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:43.735005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.735011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.735012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.735014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.735015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.735016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.735017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.735018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.735019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.735024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.735035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.735040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.735043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.735047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.735052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.735056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.735060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.735063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.735069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.735076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.735082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.735088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.735093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.735099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.735103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.735108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.735114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.735124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.735128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.735133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.735138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.735143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.735150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.735155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.735160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.735165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.735194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:43.741933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:43.743149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:43.743165Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:43.743167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:43.743171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:43.743177Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1916 [2:7559342772007550105:10470]
2025-10-09T21:50:43.743190Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:43.743205Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:43.743208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:43.743211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:43.743213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:43.743229Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:43.743235Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:43.743244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:43.743294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.743302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.743303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.743306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:43.743370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:43.743378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.743389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.743399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.743415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:43.743441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342772369382513:2432], now have 1 active actors on pipe
2025-10-09T21:50:43.743448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342772369382756:4226], now have 1 active actors on pipe
2025-10-09T21:50:43.743456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342772007550100:10470]
2025-10-09T21:50:43.743458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342772007550100:10470]
2025-10-09T21:50:43.743461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342772007550100:10470]
2025-10-09T21:50:43.743474Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:43.743492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:43.743512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:43.743495Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1916
2025-10-09T21:50:43.743516Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:43.743507Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1916
2025-10-09T21:50:43.743519Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:43.743519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:43.743522Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:43.743559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:43.743564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:43.743576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342772007550100:10470]
2025-10-09T21:50:43.743588Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:43.743601Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:43.743619Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|87ccb803-540511-5ce22cf7-28c41fe2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:43.743628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:43.743632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:43.743635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:43.743644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:43.743657Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:43.743677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:43.743864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:43.743874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:43.743890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:43.743972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1917 partition: 0 MaxSeqNo: 25 sessionId: src_id|87ccb803-540511-5ce22cf7-28c41fe2_0
2025-10-09T21:50:43.744306Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6649ff14-7a253902-64e2d515-a435889c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|87ccb803-540511-5ce22cf7-28c41fe2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:43.746682Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87ccb803-540511-5ce22cf7-28c41fe2_0] Write session: try to update token
2025-10-09T21:50:43.746698Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87ccb803-540511-5ce22cf7-28c41fe2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:43.758935Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:43.758953Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:43.758956Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:43.758958Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:43.758967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342772007550100:10470]
2025-10-09T21:50:43.758971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342772007550100:10470]
2025-10-09T21:50:43.758975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342772007550100:10470]
2025-10-09T21:50:43.759017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:43.759017Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:43.759022Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:43.759024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.759026Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:43.759027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:43.759154Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:43.759428Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1917 sessionId: src_id|87ccb803-540511-5ce22cf7-28c41fe2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:43.761880Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:43.815414Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342772369382767:4041]
2025-10-09T21:50:43.815723Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 1916, pipe: [1:7559342772369382767:4041]
2025-10-09T21:50:43.905101Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:43.905475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:43.905488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:43.905521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342772369382767:4041], now have 1 active actors on pipe
2025-10-09T21:50:43.905667Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:43.905694Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342772369382767:4041]
2025-10-09T21:50:43.905723Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:43.905725Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:43.905793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1917 reason: status is not ok: tablet will be restarted right now sessionId: src_id|87ccb803-540511-5ce22cf7-28c41fe2_0
2025-10-09T21:50:43.905970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1917 sessionId: src_id|87ccb803-540511-5ce22cf7-28c41fe2_0 is DEAD
2025-10-09T21:50:43.906157Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87ccb803-540511-5ce22cf7-28c41fe2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:43.906170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87ccb803-540511-5ce22cf7-28c41fe2_0] Write session will restart in 0.007757s
2025-10-09T21:50:43.906198Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87ccb803-540511-5ce22cf7-28c41fe2_0] Write session: Do CDS request
2025-10-09T21:50:43.906205Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87ccb803-540511-5ce22cf7-28c41fe2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:43.915564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87ccb803-540511-5ce22cf7-28c41fe2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:43.915612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:43.915628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1918
2025-10-09T21:50:43.915773Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1918 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:43.915797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1918 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:43.915806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1918 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:43.915809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:43.916286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1918 sessionId:  describe result for acl check
2025-10-09T21:50:43.916324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:43.916332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:43.916333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:43.916340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342772369382869:4259] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:43.916862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342772369382869:4259] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:43.919060Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342772369382869:4259] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:43.921656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342772369382869:4259] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:43.921669Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342772369382869:4259] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:43.921674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342772369382869:4259] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:43.921681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1918 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:43.922131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:43.924494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:43.924660Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:43.924668Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:43.924678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:43.924725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:43.924759Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342772007550395:10486]
2025-10-09T21:50:43.925185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:43.925409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:43.925454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:43.925695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:43.925730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.925734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.925735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.925735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.925736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.925737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.925737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.925738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.925738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.925742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.925752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.925755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.925759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.925763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.925767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.925770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.925774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.925778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.925784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.925791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:43.925797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.925802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:43.925806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.925811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:43.925815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.925819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:43.925823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.925827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:43.925831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.925837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:43.925841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.925846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.925850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.925855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.925860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.925869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.925895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:43.931841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:43.932926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:43.932941Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:43.932944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:43.932947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:43.932952Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1917 [2:7559342772007550395:10486]
2025-10-09T21:50:43.932962Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:43.932976Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:43.932979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:43.932982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:43.932984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:43.932998Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:43.933003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:43.933010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:43.933053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.933061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.933063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.933065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:43.933121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:43.933129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.933139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:43.933151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:43.933168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:43.933191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342772369383035:4259], now have 1 active actors on pipe
2025-10-09T21:50:43.933199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342772369382797:2432], now have 1 active actors on pipe
2025-10-09T21:50:43.933205Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342772007550390:10486]
2025-10-09T21:50:43.933208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342772007550390:10486]
2025-10-09T21:50:43.933210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342772007550390:10486]
2025-10-09T21:50:43.933225Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:43.933242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:43.933259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:43.933259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:43.933262Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:43.933264Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:43.933266Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:43.933259Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1917
2025-10-09T21:50:43.933285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342772007550390:10486]
2025-10-09T21:50:43.933275Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1917
2025-10-09T21:50:43.933293Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:43.933294Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:43.933312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:43.933319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:43.933353Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|91fe62a3-4c687dfa-b63d6b98-31f49e0e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:43.933364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:43.933368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:43.933370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:43.933380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:43.933394Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:43.933414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:43.933579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:43.933588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:43.933604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:43.933689Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1918 partition: 0 MaxSeqNo: 25 sessionId: src_id|91fe62a3-4c687dfa-b63d6b98-31f49e0e_0
2025-10-09T21:50:43.933997Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87ccb803-540511-5ce22cf7-28c41fe2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|91fe62a3-4c687dfa-b63d6b98-31f49e0e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:43.936361Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91fe62a3-4c687dfa-b63d6b98-31f49e0e_0] Write session: try to update token
2025-10-09T21:50:43.936374Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91fe62a3-4c687dfa-b63d6b98-31f49e0e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:43.946264Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:43.946282Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:43.946284Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:43.946287Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:43.946296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342772007550390:10486]
2025-10-09T21:50:43.946300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342772007550390:10486]
2025-10-09T21:50:43.946304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342772007550390:10486]
2025-10-09T21:50:43.946344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:43.946345Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:43.946350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:43.946350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:43.946352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:43.946353Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:43.946479Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:43.947993Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1918 sessionId: src_id|91fe62a3-4c687dfa-b63d6b98-31f49e0e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:43.950311Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:44.005846Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342776664350345:4041]
2025-10-09T21:50:44.006145Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 1917, pipe: [1:7559342776664350345:4041]
2025-10-09T21:50:44.093457Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:44.093684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:44.093701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:44.093741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342776664350345:4041], now have 1 active actors on pipe
2025-10-09T21:50:44.093906Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:44.093927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:44.093942Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:44.093944Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:44.093987Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342776664350345:4041]
2025-10-09T21:50:44.094013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1918 reason: error at write: Disconnected sessionId: src_id|91fe62a3-4c687dfa-b63d6b98-31f49e0e_0
2025-10-09T21:50:44.094206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1918 sessionId: src_id|91fe62a3-4c687dfa-b63d6b98-31f49e0e_0 is DEAD
2025-10-09T21:50:44.094346Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91fe62a3-4c687dfa-b63d6b98-31f49e0e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:44.094359Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91fe62a3-4c687dfa-b63d6b98-31f49e0e_0] Write session will restart in 0.008549s
2025-10-09T21:50:44.094380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91fe62a3-4c687dfa-b63d6b98-31f49e0e_0] Write session: Do CDS request
2025-10-09T21:50:44.094401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91fe62a3-4c687dfa-b63d6b98-31f49e0e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:44.104532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91fe62a3-4c687dfa-b63d6b98-31f49e0e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:44.104614Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:44.104639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1919
2025-10-09T21:50:44.104748Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1919 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:44.104780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1919 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:44.104790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1919 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:44.104794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:44.105282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1919 sessionId:  describe result for acl check
2025-10-09T21:50:44.105314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:44.105319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:44.105320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:44.105325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342776664350446:4255] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:44.105755Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342776664350446:4255] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:44.107852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342776664350446:4255] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:44.110346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342776664350446:4255] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:44.110358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342776664350446:4255] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:44.110363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342776664350446:4255] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:44.110336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:44.110369Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1919 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:44.112596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:44.112795Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:44.112806Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:44.112817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:44.112867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:44.112904Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342776302517938:10488]
2025-10-09T21:50:44.113490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:44.113804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:44.113848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:44.114173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:44.114229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.114236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.114237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.114238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.114239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.114240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.114241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.114242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.114244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.114248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.114260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.114264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.114268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.114272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.114276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.114280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.114284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.114287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.114294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.114303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.114309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.114314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.114318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.114324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.114328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.114339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.114344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.114349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.114358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.114363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.114368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.114373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.114377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.114387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.114392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.114402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.114431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:44.121051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:44.122362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:44.122377Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:44.122380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:44.122384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:44.122390Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1918 [2:7559342776302517938:10488]
2025-10-09T21:50:44.122403Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:44.122420Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:44.122423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:44.122426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:44.122428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:44.122445Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:44.122457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:44.122464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:44.122519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.122526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.122529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.122531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:44.122597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:44.122605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.122615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.122626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.122649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:44.122675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342776664350595:4255], now have 1 active actors on pipe
2025-10-09T21:50:44.122682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342776664350348:2432], now have 1 active actors on pipe
2025-10-09T21:50:44.122691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342776302517932:10488]
2025-10-09T21:50:44.122697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342776302517932:10488]
2025-10-09T21:50:44.122699Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342776302517932:10488]
2025-10-09T21:50:44.122712Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:44.122730Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:44.122730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:44.122734Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:44.122736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:44.122749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:44.122717Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1918
2025-10-09T21:50:44.122752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:44.122722Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1918
2025-10-09T21:50:44.122789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:44.122799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:44.122817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342776302517932:10488]
2025-10-09T21:50:44.122828Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:44.122838Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:44.122859Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b6323946-b0cdccd2-b34f134c-f64797ae_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:44.122871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:44.122880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:44.122883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:44.122894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:44.122909Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:44.122942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:44.123114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:44.123124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:44.123142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:44.123218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1919 partition: 0 MaxSeqNo: 25 sessionId: src_id|b6323946-b0cdccd2-b34f134c-f64797ae_0
2025-10-09T21:50:44.123543Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91fe62a3-4c687dfa-b63d6b98-31f49e0e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b6323946-b0cdccd2-b34f134c-f64797ae_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:44.126069Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6323946-b0cdccd2-b34f134c-f64797ae_0] Write session: try to update token
2025-10-09T21:50:44.126087Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6323946-b0cdccd2-b34f134c-f64797ae_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:44.137030Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:44.137050Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:44.137052Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:44.137054Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:44.137064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342776302517932:10488]
2025-10-09T21:50:44.137069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342776302517932:10488]
2025-10-09T21:50:44.137072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342776302517932:10488]
2025-10-09T21:50:44.137109Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:44.137115Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:44.137116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:44.137119Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:44.137123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.137125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:44.137253Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:44.138994Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1919 sessionId: src_id|b6323946-b0cdccd2-b34f134c-f64797ae_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:44.141216Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:44.194303Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342776664350607:4041]
2025-10-09T21:50:44.194571Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 1918, pipe: [1:7559342776664350607:4041]
2025-10-09T21:50:44.291721Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:44.292138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:44.292151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:44.292190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342776664350607:4041], now have 1 active actors on pipe
2025-10-09T21:50:44.292387Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:44.292446Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342776664350607:4041]
2025-10-09T21:50:44.292452Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:44.292456Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:44.292477Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:44.292508Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1919 reason: error at write: Disconnected sessionId: src_id|b6323946-b0cdccd2-b34f134c-f64797ae_0
2025-10-09T21:50:44.292664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1919 sessionId: src_id|b6323946-b0cdccd2-b34f134c-f64797ae_0 is DEAD
2025-10-09T21:50:44.292823Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6323946-b0cdccd2-b34f134c-f64797ae_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:44.292835Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6323946-b0cdccd2-b34f134c-f64797ae_0] Write session will restart in 0.008670s
2025-10-09T21:50:44.292863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6323946-b0cdccd2-b34f134c-f64797ae_0] Write session: Do CDS request
2025-10-09T21:50:44.292869Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6323946-b0cdccd2-b34f134c-f64797ae_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:44.302508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b6323946-b0cdccd2-b34f134c-f64797ae_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:44.302568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:44.302584Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1920
2025-10-09T21:50:44.302742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1920 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:44.302780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1920 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:44.302789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1920 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:44.302792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:44.303150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1920 sessionId:  describe result for acl check
2025-10-09T21:50:44.303181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:44.303186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:44.303187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:44.303191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342776664350698:4257] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:44.303716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342776664350698:4257] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:44.306193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342776664350698:4257] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:44.308520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342776664350698:4257] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:44.308533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342776664350698:4257] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:44.308536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342776664350698:4257] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:44.308541Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1920 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:44.308811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:44.311287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:44.311473Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:44.311488Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:44.311502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:44.311567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:44.311610Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342776302518186:10490]
2025-10-09T21:50:44.312164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:44.312483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:44.312533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:44.312835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:44.312871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.312872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.312873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.312874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.312874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.312875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.312876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.312876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.312877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.312880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.312887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.312889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.312891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.312893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.312895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.312897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.312899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.312901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.312906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.312911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.312915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.312919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.312921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.312924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.312927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.312930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.312932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.312935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.312939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.312942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.312944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.312947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.312951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.312954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.312956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.312959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.312976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:44.318903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:44.319882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:44.319899Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:44.319902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:44.319905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:44.319912Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1919 [2:7559342776302518186:10490]
2025-10-09T21:50:44.319925Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:44.319940Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:44.319948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:44.319951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:44.319953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:44.319969Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:44.319974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:44.319981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:44.320034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.320041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.320044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.320046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:44.320111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:44.320118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.320129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.320140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.320153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:44.320177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342776664350862:4257], now have 1 active actors on pipe
2025-10-09T21:50:44.320185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342776664350617:2432], now have 1 active actors on pipe
2025-10-09T21:50:44.320192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342776302518181:10490]
2025-10-09T21:50:44.320195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342776302518181:10490]
2025-10-09T21:50:44.320197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342776302518181:10490]
2025-10-09T21:50:44.320209Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:44.320222Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:44.320241Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1919
2025-10-09T21:50:44.320228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:44.320245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1919
2025-10-09T21:50:44.320233Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:44.320237Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:44.320242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:44.320245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:44.320267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342776302518181:10490]
2025-10-09T21:50:44.320271Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:44.320272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:44.320294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:44.320299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:44.320334Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ca44f4cb-52cc4813-bc2195d9-776faf75_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:44.320342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:44.320352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:44.320355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:44.320365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:44.320381Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:44.320409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:44.320553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:44.320561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:44.320574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:44.320650Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1920 partition: 0 MaxSeqNo: 25 sessionId: src_id|ca44f4cb-52cc4813-bc2195d9-776faf75_0
2025-10-09T21:50:44.321010Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b6323946-b0cdccd2-b34f134c-f64797ae_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ca44f4cb-52cc4813-bc2195d9-776faf75_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:44.323379Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca44f4cb-52cc4813-bc2195d9-776faf75_0] Write session: try to update token
2025-10-09T21:50:44.323398Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca44f4cb-52cc4813-bc2195d9-776faf75_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:44.333981Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:44.334000Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:44.334002Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:44.334003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:44.334014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342776302518181:10490]
2025-10-09T21:50:44.334017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342776302518181:10490]
2025-10-09T21:50:44.334024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342776302518181:10490]
2025-10-09T21:50:44.334080Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:44.334084Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:44.334085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:44.334093Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:44.334094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.334097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:44.334228Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:44.335534Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1920 sessionId: src_id|ca44f4cb-52cc4813-bc2195d9-776faf75_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:44.337889Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:44.392401Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342776664350875:4041]
2025-10-09T21:50:44.392608Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 1919, pipe: [1:7559342776664350875:4041]
2025-10-09T21:50:44.472730Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:44.473307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:44.473320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:44.473357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342776664350875:4041], now have 1 active actors on pipe
2025-10-09T21:50:44.473515Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:44.473591Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342776664350875:4041]
2025-10-09T21:50:44.473621Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:44.473643Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:44.473647Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:44.473705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1920 reason: error at write: Disconnected sessionId: src_id|ca44f4cb-52cc4813-bc2195d9-776faf75_0
2025-10-09T21:50:44.473861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1920 sessionId: src_id|ca44f4cb-52cc4813-bc2195d9-776faf75_0 is DEAD
2025-10-09T21:50:44.474012Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca44f4cb-52cc4813-bc2195d9-776faf75_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:44.474025Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca44f4cb-52cc4813-bc2195d9-776faf75_0] Write session will restart in 0.006464s
2025-10-09T21:50:44.474062Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca44f4cb-52cc4813-bc2195d9-776faf75_0] Write session: Do CDS request
2025-10-09T21:50:44.474069Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca44f4cb-52cc4813-bc2195d9-776faf75_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:44.481528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca44f4cb-52cc4813-bc2195d9-776faf75_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:44.481600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:44.481615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1921
2025-10-09T21:50:44.481736Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1921 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:44.481764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1921 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:44.481768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1921 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:44.481771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:44.482164Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1921 sessionId:  describe result for acl check
2025-10-09T21:50:44.482199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:44.482216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:44.482217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:44.482222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342776664350905:4266] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:44.482717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342776664350905:4266] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:44.485597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342776664350905:4266] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:44.488574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342776664350905:4266] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:44.488586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342776664350905:4266] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:44.488589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342776664350905:4266] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:44.488595Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1921 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:44.492759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:44.493199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:44.493358Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:44.493369Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:44.493380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:44.493431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:44.493479Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342776302518435:10492]
2025-10-09T21:50:44.494067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:44.494471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:44.494519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:44.494844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:44.494901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.494908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.494909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.494910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.494911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.494912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.494913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.494915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.494916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.494920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.494931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.494934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.494938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.494941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.494944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.494948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.494951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.494954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.494960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.494973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.494983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.494990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.494998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.495002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.495006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.495018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.495023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.495028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.495033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.495037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.495041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.495044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.495046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.495049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.495052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.495054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.495072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:44.500889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:44.501922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:44.501938Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:44.501940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:44.501944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:44.501950Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1920 [2:7559342776302518435:10492]
2025-10-09T21:50:44.501962Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:44.501979Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:44.501987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:44.501990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:44.501992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:44.502008Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:44.502019Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:44.502026Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:44.502091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.502100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.502101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.502104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:44.502164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:44.502171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.502183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.502195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.502216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:44.502240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342776664350877:2432], now have 1 active actors on pipe
2025-10-09T21:50:44.502247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342776664351101:4266], now have 1 active actors on pipe
2025-10-09T21:50:44.502254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342776302518430:10492]
2025-10-09T21:50:44.502256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342776302518430:10492]
2025-10-09T21:50:44.502258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342776302518430:10492]
2025-10-09T21:50:44.502270Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:44.502283Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:44.502285Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:44.502287Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:44.502287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:44.502302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:44.502304Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:44.502283Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1920
2025-10-09T21:50:44.502301Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1920
2025-10-09T21:50:44.502328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342776302518430:10492]
2025-10-09T21:50:44.502339Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:44.502343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:44.502354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:44.502363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:44.502588Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fe18b0e0-8ef72255-fc6526d2-6a0037c1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:44.502602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:44.502606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:44.502608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:44.502618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:44.502628Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:44.502656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:44.502845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:44.502853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:44.502865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:44.502956Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1921 partition: 0 MaxSeqNo: 25 sessionId: src_id|fe18b0e0-8ef72255-fc6526d2-6a0037c1_0
2025-10-09T21:50:44.503309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca44f4cb-52cc4813-bc2195d9-776faf75_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fe18b0e0-8ef72255-fc6526d2-6a0037c1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:44.505818Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe18b0e0-8ef72255-fc6526d2-6a0037c1_0] Write session: try to update token
2025-10-09T21:50:44.505836Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe18b0e0-8ef72255-fc6526d2-6a0037c1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:44.515605Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:44.515623Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:44.515625Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:44.515627Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:44.515636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342776302518430:10492]
2025-10-09T21:50:44.515641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342776302518430:10492]
2025-10-09T21:50:44.515645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342776302518430:10492]
2025-10-09T21:50:44.515684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:44.515687Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:44.515688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.515690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:44.515693Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:44.515697Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:44.515823Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:44.518591Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1921 sessionId: src_id|fe18b0e0-8ef72255-fc6526d2-6a0037c1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:44.521028Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:44.573651Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342776664351136:4041]
2025-10-09T21:50:44.574012Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 1920, pipe: [1:7559342776664351136:4041]
2025-10-09T21:50:44.655203Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:44.655606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:44.655624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:44.655673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342776664351136:4041], now have 1 active actors on pipe
2025-10-09T21:50:44.655849Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:44.655888Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:44.655887Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342776664351136:4041]
2025-10-09T21:50:44.655903Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:44.655918Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:44.655957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1921 reason: error at write: Disconnected sessionId: src_id|fe18b0e0-8ef72255-fc6526d2-6a0037c1_0
2025-10-09T21:50:44.656113Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1921 sessionId: src_id|fe18b0e0-8ef72255-fc6526d2-6a0037c1_0 is DEAD
2025-10-09T21:50:44.656265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe18b0e0-8ef72255-fc6526d2-6a0037c1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:44.656274Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe18b0e0-8ef72255-fc6526d2-6a0037c1_0] Write session will restart in 0.006460s
2025-10-09T21:50:44.656301Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe18b0e0-8ef72255-fc6526d2-6a0037c1_0] Write session: Do CDS request
2025-10-09T21:50:44.656308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe18b0e0-8ef72255-fc6526d2-6a0037c1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:44.663542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe18b0e0-8ef72255-fc6526d2-6a0037c1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:44.663578Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:44.663594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1922
2025-10-09T21:50:44.663716Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1922 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:44.663749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1922 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:44.663761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1922 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:44.663765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:44.664451Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1922 sessionId:  describe result for acl check
2025-10-09T21:50:44.664484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:44.664490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:44.664491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:44.664495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342776664351175:4272] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:44.664937Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342776664351175:4272] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:44.667617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342776664351175:4272] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:44.670476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342776664351175:4272] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:44.670488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342776664351175:4272] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:44.670492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342776664351175:4272] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:44.670498Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1922 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:44.674606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:44.680226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:44.680419Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:44.680432Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:44.680445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:44.680510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:44.680558Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342776302518685:10494]
2025-10-09T21:50:44.681130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:44.681386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:44.681425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:44.681734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:44.681780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.681784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.681785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.681786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.681788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.681789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.681790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.681791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.681792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.681797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.681806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.681810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.681813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.681817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.681820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.681823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.681827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.681830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.681837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.681845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.681851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.681856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.681860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.681865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.681869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.681874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.681879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.681884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.681888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.681899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.681910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.681915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.681919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.681929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.681934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.681940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.681977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:44.688011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:44.689717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:44.689731Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:44.689734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:44.689738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:44.689744Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1921 [2:7559342776302518685:10494]
2025-10-09T21:50:44.689762Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:44.689782Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:44.689789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:44.689792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:44.689794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:44.689810Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:44.689821Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:44.689828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:44.689883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.689891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.689893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.689895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:44.689949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:44.689954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.689964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.689974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.689988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:44.690008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342776664351350:4272], now have 1 active actors on pipe
2025-10-09T21:50:44.690014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342776664351138:2432], now have 1 active actors on pipe
2025-10-09T21:50:44.690022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342776302518680:10494]
2025-10-09T21:50:44.690025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342776302518680:10494]
2025-10-09T21:50:44.690043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342776302518680:10494]
2025-10-09T21:50:44.690057Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:44.690076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:44.690095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:44.690098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:44.690125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342776302518680:10494]
2025-10-09T21:50:44.690125Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:44.690128Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:44.690130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:44.690133Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:44.690137Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:44.690106Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1921
2025-10-09T21:50:44.690115Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1921
2025-10-09T21:50:44.690241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:44.690259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:44.690295Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|57421bfa-7af19b9a-7672f41-5aee97da_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:44.690310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:44.690315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:44.690317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:44.690327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:44.690339Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:44.690366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:44.690525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:44.690535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:44.690552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:44.690659Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1922 partition: 0 MaxSeqNo: 25 sessionId: src_id|57421bfa-7af19b9a-7672f41-5aee97da_0
2025-10-09T21:50:44.691015Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe18b0e0-8ef72255-fc6526d2-6a0037c1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|57421bfa-7af19b9a-7672f41-5aee97da_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:44.693330Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57421bfa-7af19b9a-7672f41-5aee97da_0] Write session: try to update token
2025-10-09T21:50:44.693345Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57421bfa-7af19b9a-7672f41-5aee97da_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:44.707546Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:44.707565Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:44.707575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342776302518680:10494]
2025-10-09T21:50:44.707616Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:44.707625Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:44.707789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:44.707972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:44.708109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:44.708178Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1922 sessionId: src_id|57421bfa-7af19b9a-7672f41-5aee97da_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:44.708243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:44.708441Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:44.708486Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:44.710565Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:44.710731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:44.710747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:44.710775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:44.710843Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:44.716770Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:44.716785Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:44.716787Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:44.716789Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:44.716799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342776302518680:10494]
2025-10-09T21:50:44.716803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342776302518680:10494]
2025-10-09T21:50:44.716806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342776302518680:10494]
2025-10-09T21:50:44.716843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:44.716848Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:44.716853Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:44.716856Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:44.716857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.716860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:44.716988Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:44.755944Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342776664351399:4041]
2025-10-09T21:50:44.756147Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 1921, pipe: [1:7559342776664351399:4041]
2025-10-09T21:50:44.856404Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:44.863595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:44.863620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:44.863761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:44.863817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:44.863873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:44.863940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:44.864061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:44.864069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:44.864134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:44.864194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:44.864250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:44.864305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:44.864343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:44.864350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:44.864404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:44.864457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:44.864522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:44.864579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:44.864624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:44.864630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:44.864686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:44.864742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:44.864795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:44.864852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:44.864951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:44.864957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:44.865046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:44.865130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:44.865215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:44.865271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:44.865324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:44.865330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:44.865386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:44.865452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:44.865559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:44.865622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:44.865722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:44.865731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:44.865784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:44.865834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:44.865901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:44.865956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:44.866007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:44.866012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:44.866077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:44.866129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:44.866184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:44.866236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:44.866275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:44.866281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:44.866335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:44.866397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:44.866458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:44.866511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:44.866550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:44.866555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:44.866607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:44.866657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:44.866729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:44.866789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:44.866842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:44.866851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:44.866900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342776664351399:4041], now have 1 active actors on pipe
2025-10-09T21:50:44.867123Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342776664351399:4041]
2025-10-09T21:50:44.867123Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:44.867129Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:44.867160Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:44.867162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1922 reason: error at write: Disconnected sessionId: src_id|57421bfa-7af19b9a-7672f41-5aee97da_0
2025-10-09T21:50:44.867353Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1922 sessionId: src_id|57421bfa-7af19b9a-7672f41-5aee97da_0 is DEAD
2025-10-09T21:50:44.867518Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57421bfa-7af19b9a-7672f41-5aee97da_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:44.867529Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57421bfa-7af19b9a-7672f41-5aee97da_0] Write session will restart in 0.007000s
2025-10-09T21:50:44.867554Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57421bfa-7af19b9a-7672f41-5aee97da_0] Write session: Do CDS request
2025-10-09T21:50:44.867560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57421bfa-7af19b9a-7672f41-5aee97da_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:44.867801Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:44.875479Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57421bfa-7af19b9a-7672f41-5aee97da_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:44.875531Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:44.875547Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1923
2025-10-09T21:50:44.875719Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1923 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:44.875751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1923 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:44.875760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1923 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:44.875762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:44.876177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1923 sessionId:  describe result for acl check
2025-10-09T21:50:44.876215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:44.876222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:44.876224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:44.876230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342776664351440:4284] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:44.876774Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342776664351440:4284] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:44.880068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342776664351440:4284] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:44.883262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342776664351440:4284] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:44.883279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342776664351440:4284] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:44.883283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342776664351440:4284] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:44.883292Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1923 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:44.885458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:44.887974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:44.888185Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:44.888196Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:44.888209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:44.888280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:44.888321Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342776302518941:10498]
2025-10-09T21:50:44.888948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:44.889236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:44.889278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:44.889564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:44.889610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.889614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.889615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.889616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.889617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.889617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.889618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.889619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.889619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.889623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.889630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.889632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.889634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.889636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.889638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.889640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.889642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.889644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.889650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.889655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:44.889659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.889662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:44.889665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.889668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:44.889670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.889673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:44.889676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.889680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:44.889684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.889688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:44.889694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.889698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.889704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.889709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.889714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.889719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.889743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:44.896088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:44.897384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:44.897402Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:44.897405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:44.897410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:44.897417Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1922 [2:7559342776302518941:10498]
2025-10-09T21:50:44.897446Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:44.897467Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:44.897474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:44.897478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:44.897480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:44.897499Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:44.897511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:44.897519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:44.897582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.897590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.897591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.897594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:44.897672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:44.897680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.897693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:44.897706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:44.897727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:44.897754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342776664351645:4284], now have 1 active actors on pipe
2025-10-09T21:50:44.897762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342776664351405:2432], now have 1 active actors on pipe
2025-10-09T21:50:44.897770Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342776302518936:10498]
2025-10-09T21:50:44.897772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342776302518936:10498]
2025-10-09T21:50:44.897774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342776302518936:10498]
2025-10-09T21:50:44.897790Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:44.897804Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:44.897808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:44.897813Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:44.897815Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:44.897791Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1922
2025-10-09T21:50:44.897822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:44.897825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:44.897790Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1922
2025-10-09T21:50:44.897860Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342776302518936:10498]
2025-10-09T21:50:44.897871Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:44.897885Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:44.897892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:44.897897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:44.897940Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|77f2f45e-8a956b17-86a7cf31-6350db8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:44.897952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:44.897956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:44.897958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:44.897976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:44.897986Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:44.898002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:44.898136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:44.898144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:44.898158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:44.898227Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1923 partition: 0 MaxSeqNo: 25 sessionId: src_id|77f2f45e-8a956b17-86a7cf31-6350db8_0
2025-10-09T21:50:44.898593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57421bfa-7af19b9a-7672f41-5aee97da_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|77f2f45e-8a956b17-86a7cf31-6350db8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:44.901115Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|77f2f45e-8a956b17-86a7cf31-6350db8_0] Write session: try to update token
2025-10-09T21:50:44.901133Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|77f2f45e-8a956b17-86a7cf31-6350db8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:44.911827Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:44.911844Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:44.911848Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:44.911850Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:44.911859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342776302518936:10498]
2025-10-09T21:50:44.911863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342776302518936:10498]
2025-10-09T21:50:44.911867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342776302518936:10498]
2025-10-09T21:50:44.911910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:44.911921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:44.911924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:44.911924Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:44.911929Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:44.911931Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:44.912083Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:44.916350Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1923 sessionId: src_id|77f2f45e-8a956b17-86a7cf31-6350db8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:44.918694Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:44.967414Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342776664351692:4041]
2025-10-09T21:50:44.967626Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 1922, pipe: [1:7559342776664351692:4041]
2025-10-09T21:50:45.064981Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:45.065287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:45.065300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:45.065338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342776664351692:4041], now have 1 active actors on pipe
2025-10-09T21:50:45.065574Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:45.065593Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:45.065589Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:45.065602Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:45.065649Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342776664351692:4041]
2025-10-09T21:50:45.065671Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1923 reason: error at write: Disconnected sessionId: src_id|77f2f45e-8a956b17-86a7cf31-6350db8_0
2025-10-09T21:50:45.065842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1923 sessionId: src_id|77f2f45e-8a956b17-86a7cf31-6350db8_0 is DEAD
2025-10-09T21:50:45.066006Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|77f2f45e-8a956b17-86a7cf31-6350db8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:45.066018Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|77f2f45e-8a956b17-86a7cf31-6350db8_0] Write session will restart in 0.009768s
2025-10-09T21:50:45.066060Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|77f2f45e-8a956b17-86a7cf31-6350db8_0] Write session: Do CDS request
2025-10-09T21:50:45.066068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|77f2f45e-8a956b17-86a7cf31-6350db8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:45.076519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|77f2f45e-8a956b17-86a7cf31-6350db8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:45.076608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:45.076626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1924
2025-10-09T21:50:45.076748Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1924 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:45.076780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1924 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:45.076784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1924 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:45.076787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:45.077163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1924 sessionId:  describe result for acl check
2025-10-09T21:50:45.077202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:45.077208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:45.077210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:45.077214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342780959319068:4308] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:45.077725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342780959319068:4308] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:45.079983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342780959319068:4308] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:45.083054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342780959319068:4308] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:45.083070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342780959319068:4308] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:45.083074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342780959319068:4308] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:45.083083Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1924 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:45.083336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:45.085499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:45.085703Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:45.085712Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:45.085722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:45.085766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:45.085796Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342780597486528:10512]
2025-10-09T21:50:45.086203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:45.086466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:45.086497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:45.086719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:45.086755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.086759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.086760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.086760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.086761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.086768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.086769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.086770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.086770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.086773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.086780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.086783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.086785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.086787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.086789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.086791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.086793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.086795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.086799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.086804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.086808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.086812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.086816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.086819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.086822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.086825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.086828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.086831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.086834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.086837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.086840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.086843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.086845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.086848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.086853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.086856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.086871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:45.092947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:45.094153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:45.094166Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:45.094168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:45.094171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:45.094176Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1923 [2:7559342780597486528:10512]
2025-10-09T21:50:45.094186Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:45.094199Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:45.094201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:45.094203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:45.094205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:45.094222Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:45.094235Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:45.094246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:45.094289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.094294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.094295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.094297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:45.094349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:45.094354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.094362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.094371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.094383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:45.094401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342780597486523:10512]
2025-10-09T21:50:45.094408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342780597486523:10512]
2025-10-09T21:50:45.094410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342780597486523:10512]
2025-10-09T21:50:45.094421Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:45.094432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:45.094434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342780959318991:2432], now have 1 active actors on pipe
2025-10-09T21:50:45.094438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342780959319240:4308], now have 1 active actors on pipe
2025-10-09T21:50:45.094443Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:45.094445Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:45.094446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:45.094461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:45.094468Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:45.094494Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342780597486523:10512]
2025-10-09T21:50:45.094513Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:45.094503Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1923
2025-10-09T21:50:45.094546Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:45.094755Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1923
2025-10-09T21:50:45.094803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:45.094815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:45.094840Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ad2bc534-f6097395-c64425a4-221c3576_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:45.094854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:45.094857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:45.094860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:45.094878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:45.094887Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:45.094902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:45.095037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:45.095045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:45.095058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:45.095135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1924 partition: 0 MaxSeqNo: 25 sessionId: src_id|ad2bc534-f6097395-c64425a4-221c3576_0
2025-10-09T21:50:45.095480Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|77f2f45e-8a956b17-86a7cf31-6350db8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ad2bc534-f6097395-c64425a4-221c3576_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:45.097971Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad2bc534-f6097395-c64425a4-221c3576_0] Write session: try to update token
2025-10-09T21:50:45.097990Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad2bc534-f6097395-c64425a4-221c3576_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:45.108716Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:45.108736Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:45.108738Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:45.108741Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:45.108750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342780597486523:10512]
2025-10-09T21:50:45.108756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342780597486523:10512]
2025-10-09T21:50:45.108759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342780597486523:10512]
2025-10-09T21:50:45.108806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:45.108813Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:45.108818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.108820Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:45.108821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:45.108824Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:45.108954Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:45.111976Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1924 sessionId: src_id|ad2bc534-f6097395-c64425a4-221c3576_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:45.114550Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:45.165560Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342780959319252:4041]
2025-10-09T21:50:45.165803Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 1923, pipe: [1:7559342780959319252:4041]
2025-10-09T21:50:45.264610Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:45.265203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:45.265220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:45.265264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342780959319252:4041], now have 1 active actors on pipe
2025-10-09T21:50:45.265497Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:45.265524Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342780959319252:4041]
2025-10-09T21:50:45.265547Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:45.265549Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:45.265614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1924 reason: error at write: Disconnected sessionId: src_id|ad2bc534-f6097395-c64425a4-221c3576_0
2025-10-09T21:50:45.265800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1924 sessionId: src_id|ad2bc534-f6097395-c64425a4-221c3576_0 is DEAD
2025-10-09T21:50:45.265959Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad2bc534-f6097395-c64425a4-221c3576_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:45.265970Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad2bc534-f6097395-c64425a4-221c3576_0] Write session will restart in 0.007886s
2025-10-09T21:50:45.265994Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad2bc534-f6097395-c64425a4-221c3576_0] Write session: Do CDS request
2025-10-09T21:50:45.266004Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad2bc534-f6097395-c64425a4-221c3576_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:45.266734Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:45.275548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad2bc534-f6097395-c64425a4-221c3576_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:45.275616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:45.275631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1925
2025-10-09T21:50:45.275756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1925 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:45.275791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1925 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:45.275802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1925 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:45.275806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:45.276283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1925 sessionId:  describe result for acl check
2025-10-09T21:50:45.276325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:45.276333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:45.276335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:45.276341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342780959319326:4312] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:45.276915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342780959319326:4312] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:45.279515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342780959319326:4312] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:45.281996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342780959319326:4312] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:45.282019Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342780959319326:4312] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:45.282022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342780959319326:4312] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:45.282045Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1925 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:45.283191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:45.285521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:45.285720Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:45.285730Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:45.285742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:45.285793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:45.285833Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342780597486781:10514]
2025-10-09T21:50:45.286433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:45.286723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:45.286764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:45.287068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:45.287128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.287137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.287138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.287140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.287141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.287142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.287143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.287144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.287145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.287151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.287162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.287166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.287170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.287173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.287177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.287181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.287185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.287189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.287196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.287205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.287212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.287217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.287227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.287232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.287241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.287246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.287255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.287260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.287265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.287275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.287284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.287292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.287301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.287306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.287312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.287318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.287348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:45.293644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:45.294736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:45.294756Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:45.294759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:45.294764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:45.294771Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1924 [2:7559342780597486781:10514]
2025-10-09T21:50:45.294786Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:45.294806Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:45.294810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:45.294813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:45.294815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:45.294835Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:45.294846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:45.294855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:45.294911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.294918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.294921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.294923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:45.294999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:45.295009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.295022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.295038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.295057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:45.295085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342780959319258:2432], now have 1 active actors on pipe
2025-10-09T21:50:45.295091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342780959319511:4312], now have 1 active actors on pipe
2025-10-09T21:50:45.295102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342780597486776:10514]
2025-10-09T21:50:45.295108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342780597486776:10514]
2025-10-09T21:50:45.295111Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342780597486776:10514]
2025-10-09T21:50:45.295126Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:45.295137Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1924
2025-10-09T21:50:45.295162Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1924
2025-10-09T21:50:45.295146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:45.295162Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:45.295165Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:45.295167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:45.295169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:45.295172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:45.295209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342780597486776:10514]
2025-10-09T21:50:45.295213Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:45.295215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:45.295246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:45.295256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:45.295292Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|86c5a4fb-a786c184-6b595fb7-51f2e4e1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:45.295304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:45.295308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:45.295312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:45.295321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:45.295334Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:45.295372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:45.295517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:45.295524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:45.295538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:45.295639Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1925 partition: 0 MaxSeqNo: 25 sessionId: src_id|86c5a4fb-a786c184-6b595fb7-51f2e4e1_0
2025-10-09T21:50:45.295986Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad2bc534-f6097395-c64425a4-221c3576_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|86c5a4fb-a786c184-6b595fb7-51f2e4e1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:45.298494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86c5a4fb-a786c184-6b595fb7-51f2e4e1_0] Write session: try to update token
2025-10-09T21:50:45.298513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86c5a4fb-a786c184-6b595fb7-51f2e4e1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:45.311274Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:45.311294Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:45.311297Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:45.311298Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:45.311308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342780597486776:10514]
2025-10-09T21:50:45.311313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342780597486776:10514]
2025-10-09T21:50:45.311317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342780597486776:10514]
2025-10-09T21:50:45.311362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:45.311370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.311372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:45.311373Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:45.311378Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:45.311381Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:45.311581Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:45.312278Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1925 sessionId: src_id|86c5a4fb-a786c184-6b595fb7-51f2e4e1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:45.314779Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:45.365759Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342780959319531:4041]
2025-10-09T21:50:45.366128Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 1924, pipe: [1:7559342780959319531:4041]
2025-10-09T21:50:45.463959Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:45.464249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:45.464269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:45.464312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342780959319531:4041], now have 1 active actors on pipe
2025-10-09T21:50:45.464533Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:45.464551Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342780959319531:4041]
2025-10-09T21:50:45.464584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:45.464586Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:45.464673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1925 reason: status is not ok: tablet will be restarted right now sessionId: src_id|86c5a4fb-a786c184-6b595fb7-51f2e4e1_0
2025-10-09T21:50:45.464863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1925 sessionId: src_id|86c5a4fb-a786c184-6b595fb7-51f2e4e1_0 is DEAD
2025-10-09T21:50:45.465005Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86c5a4fb-a786c184-6b595fb7-51f2e4e1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:45.465016Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86c5a4fb-a786c184-6b595fb7-51f2e4e1_0] Write session will restart in 0.007232s
2025-10-09T21:50:45.465044Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86c5a4fb-a786c184-6b595fb7-51f2e4e1_0] Write session: Do CDS request
2025-10-09T21:50:45.465051Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86c5a4fb-a786c184-6b595fb7-51f2e4e1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:45.473541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|86c5a4fb-a786c184-6b595fb7-51f2e4e1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:45.473683Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:45.473701Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1926
2025-10-09T21:50:45.473860Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1926 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:45.473897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1926 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:45.473908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1926 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:45.473912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:45.474276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1926 sessionId:  describe result for acl check
2025-10-09T21:50:45.474308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:45.474313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:45.474314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:45.474318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342780959319565:4319] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:45.474841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342780959319565:4319] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:45.477895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342780959319565:4319] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:45.480965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342780959319565:4319] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:45.480980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342780959319565:4319] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:45.480983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342780959319565:4319] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:45.480989Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1926 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:45.483371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:45.485553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:45.485888Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:45.485900Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:45.485912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:45.485967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:45.485998Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342780597487035:10516]
2025-10-09T21:50:45.486489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:45.486762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:45.486793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:45.487025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:45.487057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.487069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.487072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.487073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.487073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.487074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.487075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.487076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.487076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.487079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.487089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.487093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.487097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.487101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.487105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.487108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.487112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.487116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.487127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.487136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.487141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.487144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.487148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.487152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.487154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.487158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.487160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.487163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.487166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.487169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.487172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.487176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.487179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.487189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.487194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.487203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.487223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:45.493471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:45.494478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:45.494496Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:45.494499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:45.494503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:45.494508Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1925 [2:7559342780597487035:10516]
2025-10-09T21:50:45.494518Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:45.494533Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:45.494535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:45.494537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:45.494538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:45.494551Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:45.494556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:45.494562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:45.494601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.494607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.494608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.494610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:45.494662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:45.494668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.494675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.494681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.494691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:45.494709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342780597487030:10516]
2025-10-09T21:50:45.494714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342780597487030:10516]
2025-10-09T21:50:45.494715Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342780597487030:10516]
2025-10-09T21:50:45.494721Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:45.494731Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:45.494733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:45.494733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342780959319533:2432], now have 1 active actors on pipe
2025-10-09T21:50:45.494735Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:45.494740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342780959319764:4319], now have 1 active actors on pipe
2025-10-09T21:50:45.494745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:45.494757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:45.494765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:45.494790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342780597487030:10516]
2025-10-09T21:50:45.494799Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:45.494818Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:45.494820Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1925
2025-10-09T21:50:45.494837Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1925
2025-10-09T21:50:45.494887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:45.494896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:45.494919Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f90b716-c2f59323-41e0dc9-bbed334c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:45.494928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:45.494930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:45.494932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:45.494940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:45.494951Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:45.494967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:45.495134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:45.495145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:45.495157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:45.495249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1926 partition: 0 MaxSeqNo: 25 sessionId: src_id|f90b716-c2f59323-41e0dc9-bbed334c_0
2025-10-09T21:50:45.495641Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|86c5a4fb-a786c184-6b595fb7-51f2e4e1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f90b716-c2f59323-41e0dc9-bbed334c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:45.498019Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f90b716-c2f59323-41e0dc9-bbed334c_0] Write session: try to update token
2025-10-09T21:50:45.498072Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f90b716-c2f59323-41e0dc9-bbed334c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:45.508751Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:45.508770Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:45.508772Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:45.508774Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:45.508783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342780597487030:10516]
2025-10-09T21:50:45.508787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342780597487030:10516]
2025-10-09T21:50:45.508791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342780597487030:10516]
2025-10-09T21:50:45.508839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:45.508838Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:45.508844Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:45.508848Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:45.508851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.508854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:45.509018Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:45.511228Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1926 sessionId: src_id|f90b716-c2f59323-41e0dc9-bbed334c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:45.513420Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:45.564942Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342780959319797:4041]
2025-10-09T21:50:45.565207Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 1925, pipe: [1:7559342780959319797:4041]
2025-10-09T21:50:45.653142Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:45.653615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:45.653638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:45.653695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342780959319797:4041], now have 1 active actors on pipe
2025-10-09T21:50:45.653902Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:45.653913Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:45.653917Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:45.653967Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342780959319797:4041]
2025-10-09T21:50:45.653977Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1926 reason: error at write: Disconnected sessionId: src_id|f90b716-c2f59323-41e0dc9-bbed334c_0
2025-10-09T21:50:45.654072Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:45.654185Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1926 sessionId: src_id|f90b716-c2f59323-41e0dc9-bbed334c_0 is DEAD
2025-10-09T21:50:45.654348Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f90b716-c2f59323-41e0dc9-bbed334c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:45.654357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f90b716-c2f59323-41e0dc9-bbed334c_0] Write session will restart in 0.005298s
2025-10-09T21:50:45.654384Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f90b716-c2f59323-41e0dc9-bbed334c_0] Write session: Do CDS request
2025-10-09T21:50:45.654391Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f90b716-c2f59323-41e0dc9-bbed334c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:45.660494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f90b716-c2f59323-41e0dc9-bbed334c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:45.660596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:45.660613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1927
2025-10-09T21:50:45.660741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1927 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:45.660772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1927 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:45.660781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1927 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:45.660784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:45.661211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1927 sessionId:  describe result for acl check
2025-10-09T21:50:45.661244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:45.661251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:45.661255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:45.661261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342780959319816:4331] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:45.661759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342780959319816:4331] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:45.664148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342780959319816:4331] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:45.667446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342780959319816:4331] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:45.667463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342780959319816:4331] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:45.667466Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342780959319816:4331] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:45.667475Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1927 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:45.671865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:45.674331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:45.674532Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:45.674540Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:45.674548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:45.674589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:45.674620Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342780597487295:10522]
2025-10-09T21:50:45.675045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:45.675332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:45.675376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:45.675684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:45.675747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.675754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.675755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.675756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.675757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.675759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.675760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.675761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.675762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.675768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.675780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.675784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.675789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.675793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.675797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.675801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.675805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.675808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.675815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.675822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.675833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.675838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.675847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.675853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.675861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.675866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.675870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.675876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.675882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.675890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.675900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.675905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.675913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.675918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.675922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.675931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.675960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:45.682086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:45.683163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:45.683183Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:45.683186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:45.683191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:45.683199Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1926 [2:7559342780597487295:10522]
2025-10-09T21:50:45.683214Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:45.683239Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:45.683261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:45.683273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:45.683275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:45.683295Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:45.683302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:45.683311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:45.683376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.683388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.683390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.683393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:45.683469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:45.683478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.683491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.683503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.683523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:45.683550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342780959319963:4331], now have 1 active actors on pipe
2025-10-09T21:50:45.683557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342780959319799:2432], now have 1 active actors on pipe
2025-10-09T21:50:45.683563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342780597487290:10522]
2025-10-09T21:50:45.683568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342780597487290:10522]
2025-10-09T21:50:45.683570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342780597487290:10522]
2025-10-09T21:50:45.683582Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:45.683601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:45.683609Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:45.683612Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:45.683615Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:45.683623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:45.683626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:45.683626Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1926
2025-10-09T21:50:45.683627Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1926
2025-10-09T21:50:45.683664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342780597487290:10522]
2025-10-09T21:50:45.683676Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:45.683681Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:45.683697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:45.683707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:45.683760Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|638e9165-599078ea-70f72fe9-49a7ab4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:45.683772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:45.683781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:45.683783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:45.683794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:45.683808Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:45.683833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:45.684007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:45.684019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:45.684037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:45.684135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1927 partition: 0 MaxSeqNo: 25 sessionId: src_id|638e9165-599078ea-70f72fe9-49a7ab4_0
2025-10-09T21:50:45.684468Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f90b716-c2f59323-41e0dc9-bbed334c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|638e9165-599078ea-70f72fe9-49a7ab4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:45.686786Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|638e9165-599078ea-70f72fe9-49a7ab4_0] Write session: try to update token
2025-10-09T21:50:45.686803Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|638e9165-599078ea-70f72fe9-49a7ab4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:45.697495Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:45.697521Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:45.697523Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:45.697526Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:45.697535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342780597487290:10522]
2025-10-09T21:50:45.697539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342780597487290:10522]
2025-10-09T21:50:45.697543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342780597487290:10522]
2025-10-09T21:50:45.697589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:45.697596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.697598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:45.697600Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:45.697606Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:45.697610Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:45.697736Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:45.699511Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1927 sessionId: src_id|638e9165-599078ea-70f72fe9-49a7ab4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:45.702020Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:45.754358Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342780959320064:4041]
2025-10-09T21:50:45.754565Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 1926, pipe: [1:7559342780959320064:4041]
2025-10-09T21:50:45.845778Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:45.846012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:45.846039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:45.846080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342780959320064:4041], now have 1 active actors on pipe
2025-10-09T21:50:45.846254Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:45.846282Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342780959320064:4041]
2025-10-09T21:50:45.846292Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:45.846296Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:45.846315Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:45.846361Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1927 reason: error at write: Disconnected sessionId: src_id|638e9165-599078ea-70f72fe9-49a7ab4_0
2025-10-09T21:50:45.846532Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1927 sessionId: src_id|638e9165-599078ea-70f72fe9-49a7ab4_0 is DEAD
2025-10-09T21:50:45.846682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|638e9165-599078ea-70f72fe9-49a7ab4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:45.846694Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|638e9165-599078ea-70f72fe9-49a7ab4_0] Write session will restart in 0.005287s
2025-10-09T21:50:45.846721Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|638e9165-599078ea-70f72fe9-49a7ab4_0] Write session: Do CDS request
2025-10-09T21:50:45.846728Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|638e9165-599078ea-70f72fe9-49a7ab4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:45.853535Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|638e9165-599078ea-70f72fe9-49a7ab4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:45.853610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:45.853627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1928
2025-10-09T21:50:45.853753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1928 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:45.853779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1928 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:45.853784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1928 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:45.853788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:45.854693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1928 sessionId:  describe result for acl check
2025-10-09T21:50:45.854733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:45.854735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:45.854736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:45.854742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342780959320092:4339] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:45.855301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342780959320092:4339] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:45.858335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342780959320092:4339] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:45.861685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342780959320092:4339] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:45.861699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342780959320092:4339] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:45.861703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342780959320092:4339] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:45.861711Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1928 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:45.864113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:45.866405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:45.866613Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:45.866624Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:45.866636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:45.866689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:45.866730Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342780597487555:10526]
2025-10-09T21:50:45.867220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:45.867487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:45.867528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:45.867766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:45.867804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.867808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.867809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.867809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.867810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.867811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.867811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.867812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.867813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.867816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.867823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.867826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.867828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.867830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.867832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.867834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.867836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.867838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.867843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.867848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:45.867853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.867856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:45.867860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.867863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:45.867865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.867868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:45.867870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.867873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:45.867875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.867878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:45.867881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.867884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.867887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.867890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.867892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.867896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.867917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:45.874158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:45.875485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:45.875503Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:45.875505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:45.875510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:45.875517Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1927 [2:7559342780597487555:10526]
2025-10-09T21:50:45.875530Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:45.875551Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:45.875559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:45.875562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:45.875565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:45.875583Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:45.875589Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:45.875596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:45.875654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.875663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.875665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.875668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:45.875739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:45.875747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.875759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:45.875776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:45.875798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:45.875828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342780959320290:4339], now have 1 active actors on pipe
2025-10-09T21:50:45.875832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342780959320067:2432], now have 1 active actors on pipe
2025-10-09T21:50:45.875839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342780597487550:10526]
2025-10-09T21:50:45.875847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342780597487550:10526]
2025-10-09T21:50:45.875849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342780597487550:10526]
2025-10-09T21:50:45.875866Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:45.875880Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1927
2025-10-09T21:50:45.875889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:45.875894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:45.875887Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1927
2025-10-09T21:50:45.875898Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:45.875901Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:45.875903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:45.875906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:45.875938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342780597487550:10526]
2025-10-09T21:50:45.875941Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:45.875943Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:45.875960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:45.875965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:45.876013Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|83956de6-4b11a802-e2965285-e21e3c73_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:45.876025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:45.876033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:45.876035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:45.876046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:45.876060Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:45.876085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:45.876293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:45.876303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:45.876317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:45.876399Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1928 partition: 0 MaxSeqNo: 25 sessionId: src_id|83956de6-4b11a802-e2965285-e21e3c73_0
2025-10-09T21:50:45.876731Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|638e9165-599078ea-70f72fe9-49a7ab4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|83956de6-4b11a802-e2965285-e21e3c73_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:45.879053Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|83956de6-4b11a802-e2965285-e21e3c73_0] Write session: try to update token
2025-10-09T21:50:45.879067Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|83956de6-4b11a802-e2965285-e21e3c73_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:45.890009Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:45.890048Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:45.890051Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:45.890053Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:45.890063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342780597487550:10526]
2025-10-09T21:50:45.890068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342780597487550:10526]
2025-10-09T21:50:45.890072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342780597487550:10526]
2025-10-09T21:50:45.890115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:45.890113Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:45.890119Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:45.890121Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:45.890121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:45.890124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:45.890257Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:45.893124Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1928 sessionId: src_id|83956de6-4b11a802-e2965285-e21e3c73_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:45.895364Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:45.946356Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342780959320334:4041]
2025-10-09T21:50:45.946569Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 1927, pipe: [1:7559342780959320334:4041]
2025-10-09T21:50:46.040588Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:46.041068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.041084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.041123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342780959320334:4041], now have 1 active actors on pipe
2025-10-09T21:50:46.041311Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:46.041338Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:46.041357Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:46.041367Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342780959320334:4041]
2025-10-09T21:50:46.041398Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:46.041423Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1928 reason: error at write: Disconnected sessionId: src_id|83956de6-4b11a802-e2965285-e21e3c73_0
2025-10-09T21:50:46.041600Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1928 sessionId: src_id|83956de6-4b11a802-e2965285-e21e3c73_0 is DEAD
2025-10-09T21:50:46.041743Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83956de6-4b11a802-e2965285-e21e3c73_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:46.041753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83956de6-4b11a802-e2965285-e21e3c73_0] Write session will restart in 0.005157s
2025-10-09T21:50:46.041778Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83956de6-4b11a802-e2965285-e21e3c73_0] Write session: Do CDS request
2025-10-09T21:50:46.041785Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83956de6-4b11a802-e2965285-e21e3c73_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:46.048488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|83956de6-4b11a802-e2965285-e21e3c73_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:46.048560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:46.048574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1929
2025-10-09T21:50:46.048715Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1929 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:46.048762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1929 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:46.048769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1929 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:46.048772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:46.049127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1929 sessionId:  describe result for acl check
2025-10-09T21:50:46.049160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:46.049167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:46.049169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:46.049174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342785254287673:4367] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:46.049609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342785254287673:4367] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:46.051724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342785254287673:4367] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:46.055058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342785254287673:4367] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:46.055075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342785254287673:4367] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:46.055079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342785254287673:4367] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:46.055087Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1929 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:46.061686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:46.064454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:46.064653Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:46.064663Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:46.064675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:46.064725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:46.064764Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342784892455146:10540]
2025-10-09T21:50:46.065379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:46.065651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:46.065688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:46.065986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:46.066045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.066051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.066053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.066054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.066055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.066057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.066058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.066058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.066060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.066065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.066077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.066081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.066085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.066089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.066093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.066097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.066101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.066105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.066116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.066130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.066140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.066150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.066158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.066164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.066169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.066174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.066183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.066189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.066197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.066203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.066212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.066221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.066231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.066240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.066244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.066254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.066286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:46.072489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:46.073856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:46.073870Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:46.073871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:46.073875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:46.073879Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1928 [2:7559342784892455146:10540]
2025-10-09T21:50:46.073889Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:46.073903Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:46.073905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:46.073907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:46.073909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:46.073922Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:46.073926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:46.073932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:46.073978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.073984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.073987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.073990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:46.074074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:46.074080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.074088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.074094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.074104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:46.074123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342785254287808:4367], now have 1 active actors on pipe
2025-10-09T21:50:46.074129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342785254287657:2432], now have 1 active actors on pipe
2025-10-09T21:50:46.074134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342784892455141:10540]
2025-10-09T21:50:46.074136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342784892455141:10540]
2025-10-09T21:50:46.074138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342784892455141:10540]
2025-10-09T21:50:46.074148Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:46.074161Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:46.074172Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:46.074173Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:46.074176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:46.074182Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:46.074202Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1928
2025-10-09T21:50:46.074185Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:46.074201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342784892455141:10540]
2025-10-09T21:50:46.074207Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:46.074208Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:46.074317Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1928
2025-10-09T21:50:46.074350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.074358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.074379Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8524801-a9f69293-76b189f2-295c1481_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:46.074388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:46.074391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:46.074394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:46.074401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:46.074409Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:46.074424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:46.074568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.074575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.074586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:46.074669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1929 partition: 0 MaxSeqNo: 25 sessionId: src_id|8524801-a9f69293-76b189f2-295c1481_0
2025-10-09T21:50:46.075058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83956de6-4b11a802-e2965285-e21e3c73_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8524801-a9f69293-76b189f2-295c1481_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:46.077371Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8524801-a9f69293-76b189f2-295c1481_0] Write session: try to update token
2025-10-09T21:50:46.077388Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8524801-a9f69293-76b189f2-295c1481_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:46.086179Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:46.086195Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:46.086205Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342784892455141:10540]
2025-10-09T21:50:46.086243Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:46.086251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:46.086417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:46.086597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:46.086734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:46.086870Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:46.087539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:46.087582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:46.089935Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1929 sessionId: src_id|8524801-a9f69293-76b189f2-295c1481_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:46.092182Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:46.092504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.092524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.092571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:46.092640Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:46.099515Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:46.099532Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:46.099534Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:46.099536Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:46.099546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342784892455141:10540]
2025-10-09T21:50:46.099551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342784892455141:10540]
2025-10-09T21:50:46.099554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342784892455141:10540]
2025-10-09T21:50:46.099586Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:46.099588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:46.099591Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:46.099594Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:46.099595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.099596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:46.099716Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:46.141479Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342785254287920:4041]
2025-10-09T21:50:46.141865Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 1928, pipe: [1:7559342785254287920:4041]
2025-10-09T21:50:46.248370Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:46.260116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.260148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.260302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:46.260360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:46.260583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:46.260644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:46.260684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:46.260695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:46.260762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:46.260821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:46.260879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:46.260939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:46.260986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:46.260995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:46.261213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:46.261424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:46.261519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:46.261578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:46.261691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:46.261697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:46.261908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:46.262135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:46.262197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:46.262281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:46.262402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:46.262408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:46.262627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:46.262855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:46.262913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:46.262966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:46.263088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:46.263094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:46.263312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:46.263556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:46.263772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:46.263832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:46.263879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:46.263885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:46.263965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:46.264045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:46.264101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:46.264189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:46.264228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:46.264233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:46.264323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:46.264505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:46.264695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:46.264752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:46.264864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:46.264873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:46.264927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:46.264989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:46.265045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:46.265270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:46.265314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:46.265321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:46.265546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:46.265604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:46.265663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:46.265869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:46.265925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:46.265932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:46.265991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342785254287920:4041], now have 1 active actors on pipe
2025-10-09T21:50:46.266260Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342785254287920:4041]
2025-10-09T21:50:46.266298Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:46.266322Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:46.266325Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:46.266362Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1929 reason: error at write: Disconnected sessionId: src_id|8524801-a9f69293-76b189f2-295c1481_0
2025-10-09T21:50:46.266566Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1929 sessionId: src_id|8524801-a9f69293-76b189f2-295c1481_0 is DEAD
2025-10-09T21:50:46.266704Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8524801-a9f69293-76b189f2-295c1481_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:46.266713Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8524801-a9f69293-76b189f2-295c1481_0] Write session will restart in 0.009832s
2025-10-09T21:50:46.266732Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8524801-a9f69293-76b189f2-295c1481_0] Write session: Do CDS request
2025-10-09T21:50:46.266736Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8524801-a9f69293-76b189f2-295c1481_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:46.267180Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:46.277554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8524801-a9f69293-76b189f2-295c1481_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:46.277673Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:46.277692Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1930
2025-10-09T21:50:46.277811Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1930 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:46.277844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1930 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:46.277858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1930 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:46.277862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:46.278323Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1930 sessionId:  describe result for acl check
2025-10-09T21:50:46.278365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:46.278372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:46.278374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:46.278380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342785254288004:4376] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:46.278916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342785254288004:4376] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:46.282008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342785254288004:4376] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:46.284796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342785254288004:4376] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:46.284808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342785254288004:4376] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:46.284811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342785254288004:4376] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:46.284817Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1930 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:46.284937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:46.287213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:46.287389Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:46.287399Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:46.287419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:46.287473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:46.287510Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342784892455405:10542]
2025-10-09T21:50:46.288102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:46.288340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:46.288376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:46.288681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:46.288727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.288732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.288734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.288735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.288736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.288737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.288738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.288740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.288741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.288745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.288757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.288761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.288765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.288769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.288773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.288777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.288780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.288784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.288791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.288798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.288805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.288815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.288820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.288825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.288834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.288839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.288843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.288848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.288857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.288862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.288867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.288877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.288881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.288886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.288896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.288901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.288934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:46.295091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:46.296289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:46.296301Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:46.296303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:46.296306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:46.296312Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1929 [2:7559342784892455405:10542]
2025-10-09T21:50:46.296325Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:46.296340Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:46.296341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:46.296344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:46.296345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:46.296358Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:46.296363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:46.296368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:46.296420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.296426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.296427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.296429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:46.296483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:46.296489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.296496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.296502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.296522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:46.296542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342785254287926:2432], now have 1 active actors on pipe
2025-10-09T21:50:46.296549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342785254288186:4376], now have 1 active actors on pipe
2025-10-09T21:50:46.296555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342784892455399:10542]
2025-10-09T21:50:46.296556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342784892455399:10542]
2025-10-09T21:50:46.296558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342784892455399:10542]
2025-10-09T21:50:46.296567Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:46.296581Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:46.296583Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:46.296586Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:46.296587Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:46.296593Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:46.296594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:46.296618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342784892455399:10542]
2025-10-09T21:50:46.296628Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:46.296634Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:46.296625Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1929
2025-10-09T21:50:46.296649Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1929
2025-10-09T21:50:46.296718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.296726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.296747Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|baa72546-d55e6c12-9709cf8f-cf4cf8fe_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:46.296756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:46.296759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:46.296760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:46.296767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:46.296776Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:46.296789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:46.296936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.296949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.296969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:46.297040Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1930 partition: 0 MaxSeqNo: 25 sessionId: src_id|baa72546-d55e6c12-9709cf8f-cf4cf8fe_0
2025-10-09T21:50:46.297388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8524801-a9f69293-76b189f2-295c1481_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|baa72546-d55e6c12-9709cf8f-cf4cf8fe_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:46.299854Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|baa72546-d55e6c12-9709cf8f-cf4cf8fe_0] Write session: try to update token
2025-10-09T21:50:46.299874Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|baa72546-d55e6c12-9709cf8f-cf4cf8fe_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:46.308673Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:46.308689Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:46.308699Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342784892455399:10542]
2025-10-09T21:50:46.308745Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:46.308747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:46.308924Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:46.309064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:46.309202Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:46.309338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:46.309551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:46.309594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:46.318354Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1930 sessionId: src_id|baa72546-d55e6c12-9709cf8f-cf4cf8fe_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:46.320659Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:46.321850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.321874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.321893Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:46.321895Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:46.321897Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:46.321898Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:46.321905Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342784892455399:10542]
2025-10-09T21:50:46.321908Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342784892455399:10542]
2025-10-09T21:50:46.321911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342784892455399:10542]
2025-10-09T21:50:46.321951Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:46.321952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:46.321957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.321959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:46.321964Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:46.321967Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:46.322088Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:46.366211Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342785254288199:4041]
2025-10-09T21:50:46.366490Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 1929, pipe: [1:7559342785254288199:4041]
2025-10-09T21:50:46.464552Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:46.465012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:46.465046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342785254288199:4041], now have 1 active actors on pipe
2025-10-09T21:50:46.465157Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:46.465191Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:46.465234Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342785254288199:4041]
2025-10-09T21:50:46.465272Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:46.465281Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:46.465358Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1930 reason: error at write: Disconnected sessionId: src_id|baa72546-d55e6c12-9709cf8f-cf4cf8fe_0
2025-10-09T21:50:46.465517Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1930 sessionId: src_id|baa72546-d55e6c12-9709cf8f-cf4cf8fe_0 is DEAD
2025-10-09T21:50:46.465673Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baa72546-d55e6c12-9709cf8f-cf4cf8fe_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:46.465685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baa72546-d55e6c12-9709cf8f-cf4cf8fe_0] Write session will restart in 0.008251s
2025-10-09T21:50:46.465708Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baa72546-d55e6c12-9709cf8f-cf4cf8fe_0] Write session: Do CDS request
2025-10-09T21:50:46.465727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baa72546-d55e6c12-9709cf8f-cf4cf8fe_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:46.475521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|baa72546-d55e6c12-9709cf8f-cf4cf8fe_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:46.475576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:46.475594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1931
2025-10-09T21:50:46.475741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1931 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:46.475768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1931 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:46.475775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1931 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:46.475778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:46.476293Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1931 sessionId:  describe result for acl check
2025-10-09T21:50:46.476334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:46.476341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:46.476343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:46.476349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342785254288296:4400] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:46.476882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342785254288296:4400] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:46.479397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342785254288296:4400] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:46.481949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342785254288296:4400] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:46.481964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342785254288296:4400] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:46.481967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342785254288296:4400] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:46.481975Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1931 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:46.482420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:46.484867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:46.485066Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:46.485076Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:46.485087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:46.485139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:46.485178Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342784892455664:10544]
2025-10-09T21:50:46.485794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:46.486167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:46.486224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:46.486562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:46.486610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.486616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.486618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.486619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.486620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.486621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.486622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.486623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.486624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.486630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.486641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.486645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.486649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.486654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.486658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.486662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.486665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.486669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.486676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.486684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.486690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.486695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.486699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.486704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.486708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.486713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.486718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.486730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.486734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.486744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.486748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.486753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.486757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.486762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.486766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.486772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.486798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:46.493301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:46.494832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:46.494845Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:46.494847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:46.494850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:46.494855Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1930 [2:7559342784892455664:10544]
2025-10-09T21:50:46.494866Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:46.494885Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:46.494888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:46.494891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:46.494894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:46.494912Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:46.494917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:46.494925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:46.494999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.495007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.495010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.495012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:46.495077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:46.495086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.495099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.495110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.495126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:46.495154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342785254288201:2432], now have 1 active actors on pipe
2025-10-09T21:50:46.495165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342785254288457:4400], now have 1 active actors on pipe
2025-10-09T21:50:46.495173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342784892455659:10544]
2025-10-09T21:50:46.495176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342784892455659:10544]
2025-10-09T21:50:46.495178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342784892455659:10544]
2025-10-09T21:50:46.495190Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:46.495172Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1930
2025-10-09T21:50:46.495203Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:46.495195Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1930
2025-10-09T21:50:46.495215Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:46.495217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:46.495218Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:46.495221Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:46.495223Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:46.495245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342784892455659:10544]
2025-10-09T21:50:46.495247Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:46.495250Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:46.495266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.495269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.495308Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|608b2de5-8ea13081-cb789963-e5e374a9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:46.495320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:46.495325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:46.495332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:46.495344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:46.495358Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:46.495383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:46.495528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.495533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.495544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:46.495610Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1931 partition: 0 MaxSeqNo: 25 sessionId: src_id|608b2de5-8ea13081-cb789963-e5e374a9_0
2025-10-09T21:50:46.495954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|baa72546-d55e6c12-9709cf8f-cf4cf8fe_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|608b2de5-8ea13081-cb789963-e5e374a9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:46.498312Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|608b2de5-8ea13081-cb789963-e5e374a9_0] Write session: try to update token
2025-10-09T21:50:46.498332Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|608b2de5-8ea13081-cb789963-e5e374a9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:46.509974Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:46.509991Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:46.509993Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:46.509994Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:46.510002Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342784892455659:10544]
2025-10-09T21:50:46.510006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342784892455659:10544]
2025-10-09T21:50:46.510009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342784892455659:10544]
2025-10-09T21:50:46.510065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:46.510071Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:46.510072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.510075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:46.510076Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:46.510079Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:46.510200Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:46.512956Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1931 sessionId: src_id|608b2de5-8ea13081-cb789963-e5e374a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:46.515310Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:46.565358Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342785254288470:4041]
2025-10-09T21:50:46.565734Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 1930, pipe: [1:7559342785254288470:4041]
2025-10-09T21:50:46.653156Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:46.653555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.653573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.653615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342785254288470:4041], now have 1 active actors on pipe
2025-10-09T21:50:46.653778Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:46.653837Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:46.653855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:46.653868Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342785254288470:4041]
2025-10-09T21:50:46.653886Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:46.653929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1931 reason: error at write: Disconnected sessionId: src_id|608b2de5-8ea13081-cb789963-e5e374a9_0
2025-10-09T21:50:46.654136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1931 sessionId: src_id|608b2de5-8ea13081-cb789963-e5e374a9_0 is DEAD
2025-10-09T21:50:46.654271Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|608b2de5-8ea13081-cb789963-e5e374a9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:46.654281Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|608b2de5-8ea13081-cb789963-e5e374a9_0] Write session will restart in 0.009302s
2025-10-09T21:50:46.654301Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|608b2de5-8ea13081-cb789963-e5e374a9_0] Write session: Do CDS request
2025-10-09T21:50:46.654305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|608b2de5-8ea13081-cb789963-e5e374a9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:46.664473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|608b2de5-8ea13081-cb789963-e5e374a9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:46.664527Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:46.664544Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1932
2025-10-09T21:50:46.664712Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1932 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:46.664750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1932 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:46.664762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1932 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:46.664766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:46.665242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1932 sessionId:  describe result for acl check
2025-10-09T21:50:46.665273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:46.665277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:46.665279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:46.665283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342785254288566:4391] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:46.665700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342785254288566:4391] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:46.668302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342785254288566:4391] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:46.670843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342785254288566:4391] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:46.670835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:46.670857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342785254288566:4391] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:46.670861Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342785254288566:4391] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:46.670867Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1932 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:46.673309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:46.673513Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:46.673523Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:46.673534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:46.673576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:46.673607Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342784892455924:10546]
2025-10-09T21:50:46.673985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:46.674247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:46.674273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:46.674503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:46.674543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.674548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.674548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.674549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.674550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.674550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.674551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.674551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.674552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.674555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.674561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.674563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.674565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.674567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.674569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.674571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.674573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.674575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.674580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.674585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.674588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.674591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.674594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.674597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.674599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.674602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.674604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.674607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.674609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.674612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.674614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.674617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.674619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.674622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.674626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.674629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.674657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:46.680389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:46.681631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:46.681645Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:46.681647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:46.681652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:46.681657Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1931 [2:7559342784892455924:10546]
2025-10-09T21:50:46.681669Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:46.681683Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:46.681689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:46.681691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:46.681692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:46.681704Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:46.681711Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:46.681718Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:46.681769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.681774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.681775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.681777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:46.681827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:46.681834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.681841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.681847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.681856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:46.681869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342784892455918:10546]
2025-10-09T21:50:46.681874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342784892455918:10546]
2025-10-09T21:50:46.681875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342784892455918:10546]
2025-10-09T21:50:46.681882Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:46.681891Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:46.681901Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:46.681902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342785254288730:4391], now have 1 active actors on pipe
2025-10-09T21:50:46.681903Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:46.681909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:46.681917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:46.681919Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:46.681923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342785254288473:2432], now have 1 active actors on pipe
2025-10-09T21:50:46.681940Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342784892455918:10546]
2025-10-09T21:50:46.681945Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:46.681951Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:46.681950Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1931
2025-10-09T21:50:46.681955Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1931
2025-10-09T21:50:46.682009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.682016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.682053Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c9cca8fe-d6ec5843-d8f7843d-6cd3583_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:46.682065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:46.682069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:46.682074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:46.682081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:46.682091Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:46.682105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:46.682255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.682266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.682285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:46.682376Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1932 partition: 0 MaxSeqNo: 25 sessionId: src_id|c9cca8fe-d6ec5843-d8f7843d-6cd3583_0
2025-10-09T21:50:46.682666Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|608b2de5-8ea13081-cb789963-e5e374a9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c9cca8fe-d6ec5843-d8f7843d-6cd3583_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:46.685058Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c9cca8fe-d6ec5843-d8f7843d-6cd3583_0] Write session: try to update token
2025-10-09T21:50:46.685074Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c9cca8fe-d6ec5843-d8f7843d-6cd3583_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:46.697505Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:46.697525Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:46.697536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342784892455918:10546]
2025-10-09T21:50:46.697582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:46.697591Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:46.697732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:46.697855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:46.697977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:46.698125Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:46.698294Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:46.698328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:46.701806Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1932 sessionId: src_id|c9cca8fe-d6ec5843-d8f7843d-6cd3583_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:46.704124Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:46.704491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.704510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.704528Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:46.704529Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:46.704531Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:46.704532Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:46.704538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342784892455918:10546]
2025-10-09T21:50:46.704541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342784892455918:10546]
2025-10-09T21:50:46.704544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342784892455918:10546]
2025-10-09T21:50:46.704577Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:46.704580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:46.704585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.704586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:46.704592Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:46.704595Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:46.704698Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:46.753770Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342785254288746:4041]
2025-10-09T21:50:46.753945Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 1931, pipe: [1:7559342785254288746:4041]
2025-10-09T21:50:46.851459Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:46.852227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:46.852272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342785254288746:4041], now have 1 active actors on pipe
2025-10-09T21:50:46.852376Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:46.852412Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342785254288746:4041]
2025-10-09T21:50:46.852433Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:46.852489Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:46.852497Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:46.852589Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1932 reason: error at write: Disconnected sessionId: src_id|c9cca8fe-d6ec5843-d8f7843d-6cd3583_0
2025-10-09T21:50:46.852747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1932 sessionId: src_id|c9cca8fe-d6ec5843-d8f7843d-6cd3583_0 is DEAD
2025-10-09T21:50:46.852862Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9cca8fe-d6ec5843-d8f7843d-6cd3583_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:46.852869Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9cca8fe-d6ec5843-d8f7843d-6cd3583_0] Write session will restart in 0.007689s
2025-10-09T21:50:46.852886Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9cca8fe-d6ec5843-d8f7843d-6cd3583_0] Write session: Do CDS request
2025-10-09T21:50:46.852894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9cca8fe-d6ec5843-d8f7843d-6cd3583_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:46.861483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c9cca8fe-d6ec5843-d8f7843d-6cd3583_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:46.861740Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:46.861759Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1933
2025-10-09T21:50:46.861853Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1933 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:46.861898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1933 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:46.861909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1933 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:46.861913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:46.862466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1933 sessionId:  describe result for acl check
2025-10-09T21:50:46.862505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:46.862511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:46.862513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:46.862519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342785254288785:4405] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:46.863068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342785254288785:4405] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:46.865968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342785254288785:4405] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:46.869381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342785254288785:4405] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:46.869399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342785254288785:4405] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:46.869402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342785254288785:4405] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:46.869411Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1933 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:46.873380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:46.875939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:46.876122Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:46.876131Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:46.876144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:46.876197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:46.876236Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342784892456189:10550]
2025-10-09T21:50:46.876826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:46.877130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:46.877174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:46.877498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:46.877557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.877564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.877565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.877566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.877567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.877569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.877570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.877571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.877573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.877578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.877589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.877593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.877604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.877608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.877612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.877616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.877620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.877623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.877630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.877639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:46.877645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.877651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:46.877656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.877661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:46.877665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.877673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:46.877683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.877689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:46.877697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.877703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:46.877712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.877719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.877728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.877733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.877737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.877742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.877772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:46.883876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:46.884908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:46.884923Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:46.884926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:46.884930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:46.884937Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1932 [2:7559342784892456189:10550]
2025-10-09T21:50:46.884951Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:46.884966Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:46.884972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:46.884974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:46.884975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:46.884989Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:46.884994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:46.885004Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:46.885152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.885160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.885162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.885165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:46.885221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:46.885228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.885236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:46.885242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:46.885252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:46.885267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342784892456184:10550]
2025-10-09T21:50:46.885269Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342784892456184:10550]
2025-10-09T21:50:46.885270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342784892456184:10550]
2025-10-09T21:50:46.885277Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:46.885289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:46.885289Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:46.885293Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:46.885295Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:46.885302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:46.885304Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:46.885312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342785254288749:2432], now have 1 active actors on pipe
2025-10-09T21:50:46.885315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342785254288940:4405], now have 1 active actors on pipe
2025-10-09T21:50:46.885338Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342784892456184:10550]
2025-10-09T21:50:46.885348Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:46.885361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:46.885351Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1932
2025-10-09T21:50:46.885394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.885365Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1932
2025-10-09T21:50:46.885400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.885443Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d506f768-af80e547-a9869884-fd60171e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:46.885457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:46.885462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:46.885463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:46.885471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:46.885479Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:46.885500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:46.885663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.885668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.885679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:46.885784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1933 partition: 0 MaxSeqNo: 25 sessionId: src_id|d506f768-af80e547-a9869884-fd60171e_0
2025-10-09T21:50:46.886125Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9cca8fe-d6ec5843-d8f7843d-6cd3583_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d506f768-af80e547-a9869884-fd60171e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:46.888524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d506f768-af80e547-a9869884-fd60171e_0] Write session: try to update token
2025-10-09T21:50:46.888542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d506f768-af80e547-a9869884-fd60171e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:46.897200Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:46.897217Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:46.897225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342784892456184:10550]
2025-10-09T21:50:46.897268Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:46.897273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:46.897448Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:46.897593Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:46.897763Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:46.897923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:46.898139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:46.898179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:46.903494Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1933 sessionId: src_id|d506f768-af80e547-a9869884-fd60171e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:46.905847Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:46.906146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:46.906169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:46.906212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:46.906309Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:46.910626Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:46.910643Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:46.910646Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:46.910648Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:46.910658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342784892456184:10550]
2025-10-09T21:50:46.910662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342784892456184:10550]
2025-10-09T21:50:46.910667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342784892456184:10550]
2025-10-09T21:50:46.910701Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:46.910706Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:46.910709Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:46.910719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:46.910727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:46.910734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:46.910943Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:46.952436Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342785254289021:4041]
2025-10-09T21:50:46.952666Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 1932, pipe: [1:7559342785254289021:4041]
2025-10-09T21:50:47.058694Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:47.066567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:47.066594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:47.066753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.066827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.066901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.066978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.067033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.067043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.067110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.067174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.067239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.067299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.067340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.067345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.067401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.067461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.067528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.067598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.067643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.067649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.067711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.067786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.067868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.067939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.068000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.068007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.068066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.068127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.068185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.068242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.068298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.068304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.068379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.068450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.068539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.068599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.068638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.068643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.068699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.068771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.068837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.068905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.068945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.068951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.069012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.069075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.069138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.069192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.069234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.069240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.069294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.069355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.069424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.069495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.069552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.069563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.069627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.069692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.069756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.069820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.069865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.069873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.069938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342785254289021:4041], now have 1 active actors on pipe
2025-10-09T21:50:47.070193Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:47.070213Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342785254289021:4041]
2025-10-09T21:50:47.070245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:47.070248Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:47.070296Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1933 reason: error at write: Disconnected sessionId: src_id|d506f768-af80e547-a9869884-fd60171e_0
2025-10-09T21:50:47.070506Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1933 sessionId: src_id|d506f768-af80e547-a9869884-fd60171e_0 is DEAD
2025-10-09T21:50:47.070637Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d506f768-af80e547-a9869884-fd60171e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:47.070650Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d506f768-af80e547-a9869884-fd60171e_0] Write session will restart in 0.008363s
2025-10-09T21:50:47.070675Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d506f768-af80e547-a9869884-fd60171e_0] Write session: Do CDS request
2025-10-09T21:50:47.070682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d506f768-af80e547-a9869884-fd60171e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:47.071236Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:47.080601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d506f768-af80e547-a9869884-fd60171e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:47.080660Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:47.080679Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1934
2025-10-09T21:50:47.080808Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1934 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:47.080841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1934 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:47.080851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1934 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:47.080856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:47.081293Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1934 sessionId:  describe result for acl check
2025-10-09T21:50:47.081337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:47.081344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:47.081346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:47.081353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342789549256393:4431] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:47.081908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342789549256393:4431] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:47.084414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342789549256393:4431] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:47.088210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342789549256393:4431] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:47.088224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342789549256393:4431] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:47.088227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342789549256393:4431] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:47.088236Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1934 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:47.090069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:47.092940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:47.093138Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:47.093148Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:47.093157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:47.093201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:47.093233Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342789187423787:10564]
2025-10-09T21:50:47.093714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:47.093981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:47.094013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:47.094259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:47.094312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.094318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.094320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.094321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.094323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.094324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.094325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.094327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.094328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.094333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.094343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.094347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.094351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.094354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.094358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.094361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.094365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.094368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.094376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.094384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.094390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.094395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.094400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.094409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.094413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.094421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.094424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.094427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.094429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.094432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.094445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.094452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.094454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.094457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.094462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.094465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.094482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:47.100897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:47.102131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:47.102147Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:47.102150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:47.102155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:47.102162Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1933 [2:7559342789187423787:10564]
2025-10-09T21:50:47.102178Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:47.102197Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:47.102199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:47.102203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:47.102205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:47.102220Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:47.102233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:47.102240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:47.102288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.102293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.102294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.102296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:47.102358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:47.102365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.102380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.102390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.102407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:47.102430Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342789187423782:10564]
2025-10-09T21:50:47.102435Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342789187423782:10564]
2025-10-09T21:50:47.102437Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342789187423782:10564]
2025-10-09T21:50:47.102445Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:47.102458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342789549256587:4431], now have 1 active actors on pipe
2025-10-09T21:50:47.102464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342789549256344:2432], now have 1 active actors on pipe
2025-10-09T21:50:47.102467Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:47.102470Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:47.102472Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:47.102472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:47.102481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:47.102483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:47.102513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342789187423782:10564]
2025-10-09T21:50:47.102522Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:47.102528Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:47.102525Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1933
2025-10-09T21:50:47.102573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:47.102534Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1933
2025-10-09T21:50:47.102580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:47.102602Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|59719a4c-c1e83748-f0c1fa33-db5873e9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:47.102611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:47.102615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:47.102617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:47.102625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:47.102633Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:47.102644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:47.102771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:47.102778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:47.102787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:47.102860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1934 partition: 0 MaxSeqNo: 25 sessionId: src_id|59719a4c-c1e83748-f0c1fa33-db5873e9_0
2025-10-09T21:50:47.103481Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d506f768-af80e547-a9869884-fd60171e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|59719a4c-c1e83748-f0c1fa33-db5873e9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:47.106166Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|59719a4c-c1e83748-f0c1fa33-db5873e9_0] Write session: try to update token
2025-10-09T21:50:47.106185Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|59719a4c-c1e83748-f0c1fa33-db5873e9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:47.114586Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:47.114606Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:47.114614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342789187423782:10564]
2025-10-09T21:50:47.114657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:47.114677Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:47.114887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:47.115031Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:47.115155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:47.115289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:47.115582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:47.115623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:47.120293Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1934 sessionId: src_id|59719a4c-c1e83748-f0c1fa33-db5873e9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:47.122740Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:47.122966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:47.122991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:47.123032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:47.123151Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:47.128823Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:47.128844Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:47.128847Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:47.128849Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:47.128861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342789187423782:10564]
2025-10-09T21:50:47.128866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342789187423782:10564]
2025-10-09T21:50:47.128871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342789187423782:10564]
2025-10-09T21:50:47.128915Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:47.128919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:47.128927Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:47.128927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.128931Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:47.128931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:47.129138Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:47.170584Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342789549256612:4041]
2025-10-09T21:50:47.170864Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 1933, pipe: [1:7559342789549256612:4041]
2025-10-09T21:50:47.276448Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:47.283050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:47.283074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:47.283207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.283261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.283329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.283397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.283444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.283454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.283524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.283583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.283651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.283716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.283753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.283758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.283810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.283859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.283911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.283961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.284000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.284007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.284062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.284117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.284180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.284238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.284285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.284288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.284344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.284404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.284495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.284591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.284632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.284639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.284735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.284821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.284874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.284935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.285054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.285060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.285117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.285171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.285233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.285289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.285413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.285422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.285488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.285536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.285595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.285650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.285766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.285772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.285833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.285887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.285945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.286002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.286070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.286075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.286126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:47.286172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:47.286223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:47.286280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:47.286338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:47.286348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:47.286393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342789549256612:4041], now have 1 active actors on pipe
2025-10-09T21:50:47.286582Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:47.286587Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:47.286590Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:47.286625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1934 reason: error at write: Disconnected sessionId: src_id|59719a4c-c1e83748-f0c1fa33-db5873e9_0
2025-10-09T21:50:47.286633Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342789549256612:4041]
2025-10-09T21:50:47.286802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1934 sessionId: src_id|59719a4c-c1e83748-f0c1fa33-db5873e9_0 is DEAD
2025-10-09T21:50:47.286906Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|59719a4c-c1e83748-f0c1fa33-db5873e9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:47.286915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|59719a4c-c1e83748-f0c1fa33-db5873e9_0] Write session will restart in 0.008807s
2025-10-09T21:50:47.286939Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|59719a4c-c1e83748-f0c1fa33-db5873e9_0] Write session: Do CDS request
2025-10-09T21:50:47.286955Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|59719a4c-c1e83748-f0c1fa33-db5873e9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:47.287300Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:47.296551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|59719a4c-c1e83748-f0c1fa33-db5873e9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:47.296638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:47.296654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1935
2025-10-09T21:50:47.296776Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1935 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:47.296810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1935 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:47.296820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1935 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:47.296824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:47.297465Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1935 sessionId:  describe result for acl check
2025-10-09T21:50:47.297502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:47.297504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:47.297506Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:47.297512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342789549256673:4450] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:47.298121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342789549256673:4450] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:47.300201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342789549256673:4450] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:47.303429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342789549256673:4450] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:47.303441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342789549256673:4450] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:47.303444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342789549256673:4450] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:47.303449Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1935 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:47.306284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:47.308659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:47.308837Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:47.308848Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:47.308861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:47.308918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:47.308955Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342789187424050:10566]
2025-10-09T21:50:47.309516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:47.309797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:47.309835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:47.310126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:47.310175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.310180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.310181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.310181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.310182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.310182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.310183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.310184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.310184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.310187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.310197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.310200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.310203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.310207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.310210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.310214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.310218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.310222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.310229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.310236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.310242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.310246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.310250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.310255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.310260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.310264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.310269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.310281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.310285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.310294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.310302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.310305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.310311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.310314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.310317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.310319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.310338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:47.316537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:47.317804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:47.317823Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:47.317825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:47.317830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:47.317837Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1934 [2:7559342789187424050:10566]
2025-10-09T21:50:47.317851Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:47.317870Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:47.317878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:47.317881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:47.317883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:47.317901Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:47.317912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:47.317919Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:47.317980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.317989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.317991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.317994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:47.318083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:47.318093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.318106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.318122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.318145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:47.318167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342789549256623:2432], now have 1 active actors on pipe
2025-10-09T21:50:47.318170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342789549256838:4450], now have 1 active actors on pipe
2025-10-09T21:50:47.318178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342789187424045:10566]
2025-10-09T21:50:47.318181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342789187424045:10566]
2025-10-09T21:50:47.318183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342789187424045:10566]
2025-10-09T21:50:47.318196Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:47.318211Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:47.318214Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:47.318216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:47.318216Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:47.318231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:47.318234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:47.318266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342789187424045:10566]
2025-10-09T21:50:47.318277Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:47.318283Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:47.318256Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1934
2025-10-09T21:50:47.318312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:47.318323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:47.318271Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1934
2025-10-09T21:50:47.318357Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ffb118a0-8d9bb134-a9071757-17da6d7b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:47.318370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:47.318379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:47.318382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:47.318393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:47.318409Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:47.318428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:47.318600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:47.318611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:47.318628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:47.318749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1935 partition: 0 MaxSeqNo: 25 sessionId: src_id|ffb118a0-8d9bb134-a9071757-17da6d7b_0
2025-10-09T21:50:47.319148Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|59719a4c-c1e83748-f0c1fa33-db5873e9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ffb118a0-8d9bb134-a9071757-17da6d7b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:47.321630Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ffb118a0-8d9bb134-a9071757-17da6d7b_0] Write session: try to update token
2025-10-09T21:50:47.321650Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ffb118a0-8d9bb134-a9071757-17da6d7b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:47.332756Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:47.332774Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:47.332776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:47.332777Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:47.332785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342789187424045:10566]
2025-10-09T21:50:47.332789Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342789187424045:10566]
2025-10-09T21:50:47.332791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342789187424045:10566]
2025-10-09T21:50:47.332827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:47.332832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.332834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:47.332838Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:47.332843Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:47.332846Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:47.332951Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:47.335701Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1935 sessionId: src_id|ffb118a0-8d9bb134-a9071757-17da6d7b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:47.338380Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:47.386578Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342789549256896:4041]
2025-10-09T21:50:47.386843Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 1934, pipe: [1:7559342789549256896:4041]
2025-10-09T21:50:47.481796Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:47.482465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:47.482485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:47.482534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342789549256896:4041], now have 1 active actors on pipe
2025-10-09T21:50:47.482757Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:47.482754Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342789549256896:4041]
2025-10-09T21:50:47.482754Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:47.482793Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:47.482795Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:47.482858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1935 reason: error at write: Disconnected sessionId: src_id|ffb118a0-8d9bb134-a9071757-17da6d7b_0
2025-10-09T21:50:47.483038Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1935 sessionId: src_id|ffb118a0-8d9bb134-a9071757-17da6d7b_0 is DEAD
2025-10-09T21:50:47.483265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffb118a0-8d9bb134-a9071757-17da6d7b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:47.483277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffb118a0-8d9bb134-a9071757-17da6d7b_0] Write session will restart in 0.005182s
2025-10-09T21:50:47.483306Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffb118a0-8d9bb134-a9071757-17da6d7b_0] Write session: Do CDS request
2025-10-09T21:50:47.483323Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffb118a0-8d9bb134-a9071757-17da6d7b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:47.489496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ffb118a0-8d9bb134-a9071757-17da6d7b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:47.489552Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:47.489577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1936
2025-10-09T21:50:47.489698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1936 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:47.489734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1936 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:47.489740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1936 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:47.489744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:47.490174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1936 sessionId:  describe result for acl check
2025-10-09T21:50:47.490214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:47.490224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:47.490226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:47.490232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342789549256913:4465] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:47.490788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342789549256913:4465] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:47.493123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342789549256913:4465] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:47.496323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342789549256913:4465] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:47.496341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342789549256913:4465] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:47.496345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342789549256913:4465] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:47.496356Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1936 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:47.502676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:47.505049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:47.505264Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:47.505275Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:47.505287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:47.505337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:47.505378Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342789187424314:10568]
2025-10-09T21:50:47.505949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:47.506251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:47.506299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:47.506612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:47.506657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.506661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.506662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.506663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.506664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.506664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.506665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.506666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.506667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.506670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.506678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.506680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.506682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.506684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.506686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.506688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.506690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.506692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.506698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.506704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.506708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.506711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.506714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.506718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.506721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.506724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.506727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.506730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.506734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.506739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.506744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.506748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.506753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.506757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.506762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.506766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.506791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:47.513165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:47.514219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:47.514234Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:47.514236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:47.514239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:47.514244Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1935 [2:7559342789187424314:10568]
2025-10-09T21:50:47.514257Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:47.514272Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:47.514274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:47.514276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:47.514278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:47.514291Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:47.514302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:47.514311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:47.514365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.514371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.514372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.514374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:47.514432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:47.514438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.514445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.514457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.514478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:47.514502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342789549256898:2432], now have 1 active actors on pipe
2025-10-09T21:50:47.514504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342789549257036:4465], now have 1 active actors on pipe
2025-10-09T21:50:47.514509Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342789187424309:10568]
2025-10-09T21:50:47.514511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342789187424309:10568]
2025-10-09T21:50:47.514512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342789187424309:10568]
2025-10-09T21:50:47.514523Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:47.514539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:47.514556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:47.514557Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:47.514561Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:47.514563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:47.514564Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:47.514553Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1935
2025-10-09T21:50:47.514595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342789187424309:10568]
2025-10-09T21:50:47.514572Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1935
2025-10-09T21:50:47.514605Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:47.514607Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:47.514629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:47.514640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:47.514684Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f84cfc6-4abe7f7f-378a5fb5-4f48d9a1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:47.514694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:47.514698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:47.514700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:47.514708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:47.514717Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:47.514734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:47.514865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:47.514875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:47.514890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:47.514996Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1936 partition: 0 MaxSeqNo: 25 sessionId: src_id|f84cfc6-4abe7f7f-378a5fb5-4f48d9a1_0
2025-10-09T21:50:47.515352Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffb118a0-8d9bb134-a9071757-17da6d7b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f84cfc6-4abe7f7f-378a5fb5-4f48d9a1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:47.517714Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f84cfc6-4abe7f7f-378a5fb5-4f48d9a1_0] Write session: try to update token
2025-10-09T21:50:47.517732Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f84cfc6-4abe7f7f-378a5fb5-4f48d9a1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:47.528439Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:47.528459Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:47.528462Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:47.528464Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:47.528475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342789187424309:10568]
2025-10-09T21:50:47.528480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342789187424309:10568]
2025-10-09T21:50:47.528484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342789187424309:10568]
2025-10-09T21:50:47.528530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:47.528542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.528545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:47.528546Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:47.528553Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:47.528558Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:47.528720Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:47.530532Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1936 sessionId: src_id|f84cfc6-4abe7f7f-378a5fb5-4f48d9a1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:47.532843Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:47.582703Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342789549257172:4041]
2025-10-09T21:50:47.582971Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 1935, pipe: [1:7559342789549257172:4041]
2025-10-09T21:50:47.670333Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:47.670578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:47.670590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:47.670631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342789549257172:4041], now have 1 active actors on pipe
2025-10-09T21:50:47.670795Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:47.670828Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:47.670843Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:47.670863Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:47.670907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1936 reason: error at write: Disconnected sessionId: src_id|f84cfc6-4abe7f7f-378a5fb5-4f48d9a1_0
2025-10-09T21:50:47.670914Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342789549257172:4041]
2025-10-09T21:50:47.671087Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1936 sessionId: src_id|f84cfc6-4abe7f7f-378a5fb5-4f48d9a1_0 is DEAD
2025-10-09T21:50:47.671234Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f84cfc6-4abe7f7f-378a5fb5-4f48d9a1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:47.671246Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f84cfc6-4abe7f7f-378a5fb5-4f48d9a1_0] Write session will restart in 0.007108s
2025-10-09T21:50:47.671273Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f84cfc6-4abe7f7f-378a5fb5-4f48d9a1_0] Write session: Do CDS request
2025-10-09T21:50:47.671279Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f84cfc6-4abe7f7f-378a5fb5-4f48d9a1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:47.679471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f84cfc6-4abe7f7f-378a5fb5-4f48d9a1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:47.679558Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:47.679574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1937
2025-10-09T21:50:47.679697Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1937 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:47.679737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1937 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:47.679747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1937 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:47.679755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:47.680212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1937 sessionId:  describe result for acl check
2025-10-09T21:50:47.680248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:47.680252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:47.680254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:47.680259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342789549257216:4459] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:47.680781Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342789549257216:4459] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:47.683211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342789549257216:4459] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:47.686473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342789549257216:4459] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:47.686485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342789549257216:4459] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:47.686488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342789549257216:4459] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:47.686493Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1937 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:47.688757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:47.690990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:47.691131Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:47.691138Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:47.691147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:47.691180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:47.691218Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342789187424579:10570]
2025-10-09T21:50:47.691614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:47.691866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:47.691896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:47.692102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:47.692134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.692138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.692138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.692139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.692140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.692141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.692141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.692142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.692142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.692145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.692152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.692154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.692156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.692158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.692160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.692161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.692163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.692165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.692170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.692175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.692179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.692182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.692184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.692187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.692189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.692192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.692194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.692197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.692199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.692202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.692204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.692208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.692217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.692222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.692230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.692239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.692259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:47.698162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:47.699140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:47.699154Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:47.699156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:47.699159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:47.699164Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1936 [2:7559342789187424579:10570]
2025-10-09T21:50:47.699173Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:47.699187Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:47.699189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:47.699191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:47.699193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:47.699206Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:47.699211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:47.699218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:47.699264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.699270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.699272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.699274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:47.699323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:47.699329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.699335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.699341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.699351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:47.699368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342789187424574:10570]
2025-10-09T21:50:47.699369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342789187424574:10570]
2025-10-09T21:50:47.699371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342789187424574:10570]
2025-10-09T21:50:47.699379Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:47.699388Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:47.699395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342789549257174:2432], now have 1 active actors on pipe
2025-10-09T21:50:47.699399Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:47.699401Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:47.699401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342789549257407:4459], now have 1 active actors on pipe
2025-10-09T21:50:47.699407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:47.699425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:47.699427Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:47.699448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342789187424574:10570]
2025-10-09T21:50:47.699462Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:47.699467Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:47.699475Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1936
2025-10-09T21:50:47.699477Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1936
2025-10-09T21:50:47.699532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:47.699540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:47.699562Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d8fd5fcf-4bab8178-8cf38c02-295a242_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:47.699568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:47.699571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:47.699573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:47.699582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:47.699592Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:47.699602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:47.699779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:47.699794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:47.699820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:47.699923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1937 partition: 0 MaxSeqNo: 25 sessionId: src_id|d8fd5fcf-4bab8178-8cf38c02-295a242_0
2025-10-09T21:50:47.700270Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f84cfc6-4abe7f7f-378a5fb5-4f48d9a1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d8fd5fcf-4bab8178-8cf38c02-295a242_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:47.702649Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d8fd5fcf-4bab8178-8cf38c02-295a242_0] Write session: try to update token
2025-10-09T21:50:47.702664Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d8fd5fcf-4bab8178-8cf38c02-295a242_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:47.713279Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:47.713299Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:47.713302Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:47.713304Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:47.713313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342789187424574:10570]
2025-10-09T21:50:47.713319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342789187424574:10570]
2025-10-09T21:50:47.713323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342789187424574:10570]
2025-10-09T21:50:47.713368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:47.713372Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:47.713376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.713378Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:47.713379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:47.713381Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:47.713541Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:47.715709Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1937 sessionId: src_id|d8fd5fcf-4bab8178-8cf38c02-295a242_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:47.717989Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:47.771106Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342789549257449:4041]
2025-10-09T21:50:47.771339Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 1936, pipe: [1:7559342789549257449:4041]
2025-10-09T21:50:47.860355Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:47.860595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:47.860617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:47.860665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342789549257449:4041], now have 1 active actors on pipe
2025-10-09T21:50:47.860835Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:47.860958Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342789549257449:4041]
2025-10-09T21:50:47.860972Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:47.860993Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:47.861063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1937 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d8fd5fcf-4bab8178-8cf38c02-295a242_0
2025-10-09T21:50:47.861245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1937 sessionId: src_id|d8fd5fcf-4bab8178-8cf38c02-295a242_0 is DEAD
2025-10-09T21:50:47.861393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d8fd5fcf-4bab8178-8cf38c02-295a242_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:47.861402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d8fd5fcf-4bab8178-8cf38c02-295a242_0] Write session will restart in 0.007287s
2025-10-09T21:50:47.861424Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d8fd5fcf-4bab8178-8cf38c02-295a242_0] Write session: Do CDS request
2025-10-09T21:50:47.861448Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d8fd5fcf-4bab8178-8cf38c02-295a242_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:47.869507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d8fd5fcf-4bab8178-8cf38c02-295a242_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:47.869590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:47.869605Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1938
2025-10-09T21:50:47.869718Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1938 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:47.869752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1938 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:47.869759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1938 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:47.869768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:47.870147Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1938 sessionId:  describe result for acl check
2025-10-09T21:50:47.870175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:47.870180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:47.870181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:47.870184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342789549257504:4480] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:47.870654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342789549257504:4480] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:47.873476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342789549257504:4480] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:47.876148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342789549257504:4480] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:47.876163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342789549257504:4480] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:47.876167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342789549257504:4480] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:47.876173Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1938 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:47.878081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:47.880320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:47.880489Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:47.880502Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:47.880513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:47.880562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:47.880597Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342789187424849:10574]
2025-10-09T21:50:47.881141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:47.881449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:47.881491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:47.881783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:47.881832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.881839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.881840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.881841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.881841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.881843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.881844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.881845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.881846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.881851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.881861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.881865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.881870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.881874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.881877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.881882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.881885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.881889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.881895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.881902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:47.881908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.881914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:47.881918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.881923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:47.881927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.881932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:47.881937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.881942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:47.881946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.881951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:47.881956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.881966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.881971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.881977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.881982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.881989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.882018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:47.888133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:47.889145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:47.889163Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:47.889165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:47.889170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:47.889177Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1937 [2:7559342789187424849:10574]
2025-10-09T21:50:47.889193Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:47.889209Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:47.889217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:47.889220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:47.889223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:47.889239Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:47.889250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:47.889258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:47.889315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.889323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.889325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.889328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:47.889413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:47.889421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.889446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:47.889466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:47.889487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:47.889508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342789549257452:2432], now have 1 active actors on pipe
2025-10-09T21:50:47.889512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342789549257714:4480], now have 1 active actors on pipe
2025-10-09T21:50:47.889520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342789187424844:10574]
2025-10-09T21:50:47.889522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342789187424844:10574]
2025-10-09T21:50:47.889525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342789187424844:10574]
2025-10-09T21:50:47.889539Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:47.889554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:47.889562Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:47.889565Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:47.889570Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:47.889573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:47.889584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:47.889596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342789187424844:10574]
2025-10-09T21:50:47.889593Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1937
2025-10-09T21:50:47.889600Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:47.889606Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1937
2025-10-09T21:50:47.889611Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:47.889904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:47.889917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:47.889958Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6c3bb3c-d4b89886-bae7defe-d6728e23_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:47.889970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:47.889976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:47.889979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:47.889993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:47.890008Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:47.890046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:47.890254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:47.890264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:47.890279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:47.890369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1938 partition: 0 MaxSeqNo: 25 sessionId: src_id|6c3bb3c-d4b89886-bae7defe-d6728e23_0
2025-10-09T21:50:47.890715Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d8fd5fcf-4bab8178-8cf38c02-295a242_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6c3bb3c-d4b89886-bae7defe-d6728e23_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:47.893193Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6c3bb3c-d4b89886-bae7defe-d6728e23_0] Write session: try to update token
2025-10-09T21:50:47.893208Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6c3bb3c-d4b89886-bae7defe-d6728e23_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:47.904014Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:47.904035Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:47.904037Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:47.904039Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:47.904048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342789187424844:10574]
2025-10-09T21:50:47.904052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342789187424844:10574]
2025-10-09T21:50:47.904055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342789187424844:10574]
2025-10-09T21:50:47.904099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:47.904098Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:47.904103Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:47.904106Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:47.904113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:47.904116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:47.904243Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:47.906132Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1938 sessionId: src_id|6c3bb3c-d4b89886-bae7defe-d6728e23_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:47.908366Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:47.961044Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342789549257729:4041]
2025-10-09T21:50:47.961315Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 1937, pipe: [1:7559342789549257729:4041]
2025-10-09T21:50:48.054557Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:48.054821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.054837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.054880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342789549257729:4041], now have 1 active actors on pipe
2025-10-09T21:50:48.055082Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:48.055090Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342789549257729:4041]
2025-10-09T21:50:48.055119Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:48.055145Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:48.055147Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:48.055201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1938 reason: error at write: Disconnected sessionId: src_id|6c3bb3c-d4b89886-bae7defe-d6728e23_0
2025-10-09T21:50:48.055366Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1938 sessionId: src_id|6c3bb3c-d4b89886-bae7defe-d6728e23_0 is DEAD
2025-10-09T21:50:48.055499Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c3bb3c-d4b89886-bae7defe-d6728e23_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:48.055509Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c3bb3c-d4b89886-bae7defe-d6728e23_0] Write session will restart in 0.008033s
2025-10-09T21:50:48.055533Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c3bb3c-d4b89886-bae7defe-d6728e23_0] Write session: Do CDS request
2025-10-09T21:50:48.055539Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c3bb3c-d4b89886-bae7defe-d6728e23_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:48.064483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6c3bb3c-d4b89886-bae7defe-d6728e23_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:48.064513Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:48.064530Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1939
2025-10-09T21:50:48.064723Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1939 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:48.064755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1939 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:48.064764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1939 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:48.064767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:48.065106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1939 sessionId:  describe result for acl check
2025-10-09T21:50:48.065139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:48.065146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:48.065147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:48.065152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342793844225112:4506] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:48.065804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342793844225112:4506] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:48.068218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342793844225112:4506] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:48.071041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342793844225112:4506] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:48.071055Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342793844225112:4506] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:48.071058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342793844225112:4506] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:48.071065Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1939 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:48.072673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:48.074806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:48.074977Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:48.074989Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:48.075000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:48.075046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:48.075088Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342793482392450:10582]
2025-10-09T21:50:48.075584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:48.075857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:48.075897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:48.076166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:48.076210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.076215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.076216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.076217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.076218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.076218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.076219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.076220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.076221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.076225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.076233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.076236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.076239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.076242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.076245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.076248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.076251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.076253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.076260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.076266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.076271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.076275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.076278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.076282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.076288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.076292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.076296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.076305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.076308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.076313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.076317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.076325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.076328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.076332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.076340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.076344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.076371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:48.082591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:48.083733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:48.083749Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:48.083750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:48.083754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:48.083761Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1938 [2:7559342793482392450:10582]
2025-10-09T21:50:48.083771Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:48.083785Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:48.083787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:48.083789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:48.083790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:48.083802Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:48.083806Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:48.083811Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:48.083866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.083877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.083879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.083882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:48.083950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:48.083958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.083971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.083985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.083999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:48.084019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342793844225312:4506], now have 1 active actors on pipe
2025-10-09T21:50:48.084026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342793844225041:2432], now have 1 active actors on pipe
2025-10-09T21:50:48.084031Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342793482392445:10582]
2025-10-09T21:50:48.084033Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342793482392445:10582]
2025-10-09T21:50:48.084034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342793482392445:10582]
2025-10-09T21:50:48.084044Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:48.084054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:48.084067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:48.084069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:48.084069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:48.084071Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:48.084073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:48.084093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342793482392445:10582]
2025-10-09T21:50:48.084098Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:48.084101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:48.084103Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1938
2025-10-09T21:50:48.084108Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1938
2025-10-09T21:50:48.084228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.084239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.084269Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e4ec89d8-4254a44f-283eeb4c-8345017f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:48.084281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:48.084292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:48.084294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:48.084317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:48.084331Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:48.084345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:48.084558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.084571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.084593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:48.084712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1939 partition: 0 MaxSeqNo: 25 sessionId: src_id|e4ec89d8-4254a44f-283eeb4c-8345017f_0
2025-10-09T21:50:48.085097Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c3bb3c-d4b89886-bae7defe-d6728e23_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e4ec89d8-4254a44f-283eeb4c-8345017f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:48.087700Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4ec89d8-4254a44f-283eeb4c-8345017f_0] Write session: try to update token
2025-10-09T21:50:48.087720Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4ec89d8-4254a44f-283eeb4c-8345017f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:48.096670Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:48.096691Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:48.096701Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342793482392445:10582]
2025-10-09T21:50:48.096748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:48.096757Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:48.096927Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:48.097075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:48.097211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:48.097355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:48.097569Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:48.097610Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:48.101496Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1939 sessionId: src_id|e4ec89d8-4254a44f-283eeb4c-8345017f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:48.103843Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:48.104022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.104039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.104072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:48.104140Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:48.110465Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:48.110481Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:48.110483Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:48.110486Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:48.110497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342793482392445:10582]
2025-10-09T21:50:48.110501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342793482392445:10582]
2025-10-09T21:50:48.110505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342793482392445:10582]
2025-10-09T21:50:48.110549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:48.110556Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:48.110561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.110562Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:48.110563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:48.110566Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:48.110697Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:48.155145Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342793844225326:4041]
2025-10-09T21:50:48.155458Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 1938, pipe: [1:7559342793844225326:4041]
2025-10-09T21:50:48.265746Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:48.273200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.273227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.273376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.273448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.273508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.273567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.273884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.273894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.273961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.274013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.274092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.274148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.274186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.274196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.274256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.274318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.274375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.274437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.274479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.274483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.274545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.274609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.274666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.274773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.274832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.274842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.274903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.274962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.275021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.275077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.275120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.275127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.275185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.275247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.275306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.275385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.275443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.275450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.275531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.275594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.275650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.275708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.275762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.275768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.275826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.275890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.275944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.275997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.276040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.276046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.276101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.276153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.276211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.276268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.276312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.276318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.276377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.276433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.276493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.276549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.276595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.276601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.276658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342793844225326:4041], now have 1 active actors on pipe
2025-10-09T21:50:48.276877Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:48.276885Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:48.276890Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:48.276933Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1939 reason: error at write: Disconnected sessionId: src_id|e4ec89d8-4254a44f-283eeb4c-8345017f_0
2025-10-09T21:50:48.276948Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342793844225326:4041]
2025-10-09T21:50:48.277193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1939 sessionId: src_id|e4ec89d8-4254a44f-283eeb4c-8345017f_0 is DEAD
2025-10-09T21:50:48.277341Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4ec89d8-4254a44f-283eeb4c-8345017f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:48.277363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4ec89d8-4254a44f-283eeb4c-8345017f_0] Write session will restart in 0.007378s
2025-10-09T21:50:48.277348Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:48.277389Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4ec89d8-4254a44f-283eeb4c-8345017f_0] Write session: Do CDS request
2025-10-09T21:50:48.277396Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4ec89d8-4254a44f-283eeb4c-8345017f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:48.285543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4ec89d8-4254a44f-283eeb4c-8345017f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:48.285582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:48.285600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1940
2025-10-09T21:50:48.285754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1940 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:48.285801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1940 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:48.285813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1940 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:48.285816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:48.286280Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1940 sessionId:  describe result for acl check
2025-10-09T21:50:48.286317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:48.286323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:48.286325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:48.286330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342793844225370:4517] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:48.286864Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342793844225370:4517] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:48.290230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342793844225370:4517] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:48.293711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342793844225370:4517] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:48.293726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342793844225370:4517] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:48.293730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342793844225370:4517] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:48.293738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1940 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:48.298636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:48.299076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:48.299255Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:48.299276Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:48.299293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:48.299351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:48.299404Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342793482392719:10590]
2025-10-09T21:50:48.299972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:48.300298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:48.300361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:48.300710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:48.300769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.300776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.300777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.300778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.300779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.300781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.300782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.300783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.300784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.300790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.300801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.300805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.300809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.300813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.300817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.300821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.300825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.300829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.300835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.300844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.300849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.300855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.300859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.300870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.300875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.300881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.300887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.300896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.300901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.300906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.300916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.300921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.300930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.300935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.300939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.300944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.300976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:48.307319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:48.308735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:48.308754Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:48.308757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:48.308762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:48.308769Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1939 [2:7559342793482392719:10590]
2025-10-09T21:50:48.308783Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:48.308803Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:48.308810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:48.308814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:48.308816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:48.308834Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:48.308842Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:48.308850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:48.308913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.308920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.308922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.308924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:48.308993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:48.309000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.309013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.309026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.309048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:48.309074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342793844225579:4517], now have 1 active actors on pipe
2025-10-09T21:50:48.309082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342793844225332:2432], now have 1 active actors on pipe
2025-10-09T21:50:48.309090Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342793482392714:10590]
2025-10-09T21:50:48.309092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342793482392714:10590]
2025-10-09T21:50:48.309094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342793482392714:10590]
2025-10-09T21:50:48.309107Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:48.309121Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:48.309123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:48.309131Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:48.309133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:48.309134Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:48.309135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:48.309160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342793482392714:10590]
2025-10-09T21:50:48.309171Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:48.309196Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:48.309172Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1939
2025-10-09T21:50:48.309247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.309255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.309278Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1ad1e43-85d94c88-b66f358a-5c37b71c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:48.309288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:48.309291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:48.309292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:48.309301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:48.309315Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:48.309336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:48.309320Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1939
2025-10-09T21:50:48.309464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.309471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.309483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:48.309576Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1940 partition: 0 MaxSeqNo: 25 sessionId: src_id|1ad1e43-85d94c88-b66f358a-5c37b71c_0
2025-10-09T21:50:48.309953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4ec89d8-4254a44f-283eeb4c-8345017f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1ad1e43-85d94c88-b66f358a-5c37b71c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:48.312483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad1e43-85d94c88-b66f358a-5c37b71c_0] Write session: try to update token
2025-10-09T21:50:48.312502Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad1e43-85d94c88-b66f358a-5c37b71c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:48.321477Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:48.321497Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:48.321506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342793482392714:10590]
2025-10-09T21:50:48.321553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:48.321559Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:48.321735Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:48.321880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:48.322013Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:48.322176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:48.322375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:48.322421Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:48.325610Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1940 sessionId: src_id|1ad1e43-85d94c88-b66f358a-5c37b71c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:48.327913Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:48.328084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.328122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.328161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:48.328261Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:48.334524Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:48.334541Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:48.334543Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:48.334545Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:48.334556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342793482392714:10590]
2025-10-09T21:50:48.334560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342793482392714:10590]
2025-10-09T21:50:48.334570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342793482392714:10590]
2025-10-09T21:50:48.334611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:48.334617Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:48.334622Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:48.334622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.334626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:48.334626Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:48.334757Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:48.377356Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342793844225615:4041]
2025-10-09T21:50:48.377622Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 1939, pipe: [1:7559342793844225615:4041]
2025-10-09T21:50:48.482089Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:48.489904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.489932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.490100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.490152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.490210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.490276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.490321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.490330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.490401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.490470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.490537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.490620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.490660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.490668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.490731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.490824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.490884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.491091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.491134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.491142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.491205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.491268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.491330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.491397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.491442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.491454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.491518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.491580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.491641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.491698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.491740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.491747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.491807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.491865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.491922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.491978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.492021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.492031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.492087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.492138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.492200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.492256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.492295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.492303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.492540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.492735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.492981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.493042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.493092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.493100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.493157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.493210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.493264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.493322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.493379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.493386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.493457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.493513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.493572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.493630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.493668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.493676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.493734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342793844225615:4041], now have 1 active actors on pipe
2025-10-09T21:50:48.494088Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:48.494105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:48.494107Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342793844225615:4041]
2025-10-09T21:50:48.494134Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:48.494136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1940 reason: error at write: Disconnected sessionId: src_id|1ad1e43-85d94c88-b66f358a-5c37b71c_0
2025-10-09T21:50:48.494328Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1940 sessionId: src_id|1ad1e43-85d94c88-b66f358a-5c37b71c_0 is DEAD
2025-10-09T21:50:48.494497Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad1e43-85d94c88-b66f358a-5c37b71c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:48.494506Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad1e43-85d94c88-b66f358a-5c37b71c_0] Write session will restart in 0.007964s
2025-10-09T21:50:48.494524Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad1e43-85d94c88-b66f358a-5c37b71c_0] Write session: Do CDS request
2025-10-09T21:50:48.494529Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad1e43-85d94c88-b66f358a-5c37b71c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:48.494718Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:48.503488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad1e43-85d94c88-b66f358a-5c37b71c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:48.503607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:48.503624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1941
2025-10-09T21:50:48.503736Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1941 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:48.503761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1941 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:48.503768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1941 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:48.503771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:48.504158Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1941 sessionId:  describe result for acl check
2025-10-09T21:50:48.504199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:48.504206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:48.504208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:48.504214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342793844225654:4526] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:48.504732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342793844225654:4526] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:48.508705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342793844225654:4526] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:48.511811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342793844225654:4526] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:48.511824Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342793844225654:4526] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:48.511827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342793844225654:4526] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:48.511836Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1941 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:48.513144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:48.515767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:48.515934Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:48.515941Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:48.515950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:48.515989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:48.516021Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342793482392988:10592]
2025-10-09T21:50:48.516428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:48.516675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:48.516703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:48.516902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:48.516935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.516940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.516941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.516941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.516942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.516943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.516943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.516944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.516945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.516948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.516955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.516958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.516960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.516962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.516964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.516966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.516968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.516970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.516975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.516980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.516984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.516987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.516990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.516997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.516999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.517002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.517005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.517008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.517010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.517013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.517016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.517019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.517025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.517028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.517031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.517035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.517052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:48.523291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:48.524278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:48.524292Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:48.524294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:48.524297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:48.524304Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1940 [2:7559342793482392988:10592]
2025-10-09T21:50:48.524314Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:48.524330Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:48.524336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:48.524338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:48.524340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:48.524355Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:48.524363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:48.524368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:48.524420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.524425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.524427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.524429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:48.524484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:48.524489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.524496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.524506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.524519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:48.524538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342793482392983:10592]
2025-10-09T21:50:48.524544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342793482392983:10592]
2025-10-09T21:50:48.524545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342793482392983:10592]
2025-10-09T21:50:48.524551Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:48.524562Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:48.524563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342793844225880:4526], now have 1 active actors on pipe
2025-10-09T21:50:48.524566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342793844225617:2432], now have 1 active actors on pipe
2025-10-09T21:50:48.524571Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:48.524573Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:48.524573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:48.524582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:48.524584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:48.524620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342793482392983:10592]
2025-10-09T21:50:48.524632Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:48.524636Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:48.524667Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1940
2025-10-09T21:50:48.524669Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1940
2025-10-09T21:50:48.524764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.524775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.524798Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8c4957f7-e72a7da7-e784d342-4acc7ce7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:48.524804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:48.524812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:48.524814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:48.524822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:48.524830Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:48.524842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:48.524971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.524979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.524990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:48.525073Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1941 partition: 0 MaxSeqNo: 25 sessionId: src_id|8c4957f7-e72a7da7-e784d342-4acc7ce7_0
2025-10-09T21:50:48.525473Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1ad1e43-85d94c88-b66f358a-5c37b71c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8c4957f7-e72a7da7-e784d342-4acc7ce7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:48.527753Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c4957f7-e72a7da7-e784d342-4acc7ce7_0] Write session: try to update token
2025-10-09T21:50:48.527770Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c4957f7-e72a7da7-e784d342-4acc7ce7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:48.536005Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:48.536025Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:48.536035Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342793482392983:10592]
2025-10-09T21:50:48.536072Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:48.536084Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:48.536258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:48.536392Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:48.536517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:48.536662Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:48.536850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:48.536899Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:48.539984Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1941 sessionId: src_id|8c4957f7-e72a7da7-e784d342-4acc7ce7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:48.542346Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:48.542542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.542563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.542603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:48.542684Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:48.548231Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:48.548245Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:48.548247Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:48.548248Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:48.548256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342793482392983:10592]
2025-10-09T21:50:48.548259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342793482392983:10592]
2025-10-09T21:50:48.548262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342793482392983:10592]
2025-10-09T21:50:48.548300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:48.548299Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:48.548304Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:48.548307Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:48.548308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.548311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:48.548437Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:48.594427Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342793844225896:4041]
2025-10-09T21:50:48.594685Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 1940, pipe: [1:7559342793844225896:4041]
2025-10-09T21:50:48.695834Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:48.702570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.702596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.702742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.702811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.702882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.702952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.702998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.703008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.703076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.703144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.703209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.703269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.703443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.703450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.703504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.703559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.703612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.703663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.703703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.703706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.703765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.703823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.703887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.703956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.704008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.704015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.704072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.704123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.704190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.704259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.704302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.704308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.704367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.704419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.704473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.704525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.704567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.704573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.704628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.704679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.704738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.704801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.704841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.704849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.704912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.704970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.705024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.705075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.705114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.705116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.705173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.705232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.705289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.705347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.705386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.705392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.705459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:48.705509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:48.705566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:48.705628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:48.705671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:48.705678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:48.705724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342793844225896:4041], now have 1 active actors on pipe
2025-10-09T21:50:48.706012Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:48.706098Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342793844225896:4041]
2025-10-09T21:50:48.706258Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:48.706271Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:48.706300Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1941 reason: error at write: Disconnected sessionId: src_id|8c4957f7-e72a7da7-e784d342-4acc7ce7_0
2025-10-09T21:50:48.706509Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1941 sessionId: src_id|8c4957f7-e72a7da7-e784d342-4acc7ce7_0 is DEAD
2025-10-09T21:50:48.706609Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c4957f7-e72a7da7-e784d342-4acc7ce7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:48.706621Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c4957f7-e72a7da7-e784d342-4acc7ce7_0] Write session will restart in 0.006070s
2025-10-09T21:50:48.706654Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c4957f7-e72a7da7-e784d342-4acc7ce7_0] Write session: Do CDS request
2025-10-09T21:50:48.706661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c4957f7-e72a7da7-e784d342-4acc7ce7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:48.706737Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:48.713565Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c4957f7-e72a7da7-e784d342-4acc7ce7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:48.713677Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:48.713695Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1942
2025-10-09T21:50:48.713834Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1942 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:48.713870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1942 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:48.713882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1942 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:48.713885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:48.714332Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1942 sessionId:  describe result for acl check
2025-10-09T21:50:48.714367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:48.714372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:48.714374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:48.714378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342793844225915:4531] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:48.714881Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342793844225915:4531] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:48.717491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342793844225915:4531] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:48.720841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342793844225915:4531] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:48.720856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342793844225915:4531] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:48.720860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342793844225915:4531] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:48.720868Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1942 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:48.725383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:48.727859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:48.728062Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:48.728075Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:48.728086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:48.728152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:48.728193Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342793482393258:10594]
2025-10-09T21:50:48.728809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:48.729078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:48.729122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:48.729388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:48.729422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.729436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.729437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.729438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.729439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.729439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.729440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.729441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.729441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.729445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.729452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.729454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.729456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.729458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.729460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.729462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.729464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.729466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.729471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.729475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.729480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.729483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.729485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.729489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.729491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.729494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.729497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.729500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.729503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.729506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.729509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.729512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.729514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.729517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.729521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.729524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.729540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:48.735771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:48.737023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:48.737038Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:48.737040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:48.737044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:48.737051Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1941 [2:7559342793482393258:10594]
2025-10-09T21:50:48.737064Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:48.737082Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:48.737087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:48.737090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:48.737092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:48.737111Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:48.737123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:48.737130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:48.737182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.737188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.737190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.737192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:48.737258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:48.737265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.737275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.737289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.737311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:48.737332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342793844226068:4531], now have 1 active actors on pipe
2025-10-09T21:50:48.737340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342793844225899:2432], now have 1 active actors on pipe
2025-10-09T21:50:48.737348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342793482393253:10594]
2025-10-09T21:50:48.737350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342793482393253:10594]
2025-10-09T21:50:48.737352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342793482393253:10594]
2025-10-09T21:50:48.737364Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:48.737383Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:48.737397Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:48.737397Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:48.737401Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:48.737403Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:48.737405Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:48.737443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342793482393253:10594]
2025-10-09T21:50:48.737450Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:48.737452Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:48.737411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1941
2025-10-09T21:50:48.737418Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1941
2025-10-09T21:50:48.737486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.737495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.737533Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|23179f8a-687b93a8-46262283-6797120f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:48.737546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:48.737555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:48.737557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:48.737569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:48.737583Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:48.737602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:48.737744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.737753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.737770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:48.737885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1942 partition: 0 MaxSeqNo: 25 sessionId: src_id|23179f8a-687b93a8-46262283-6797120f_0
2025-10-09T21:50:48.738268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c4957f7-e72a7da7-e784d342-4acc7ce7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|23179f8a-687b93a8-46262283-6797120f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:48.740649Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23179f8a-687b93a8-46262283-6797120f_0] Write session: try to update token
2025-10-09T21:50:48.740666Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23179f8a-687b93a8-46262283-6797120f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:48.751518Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:48.751537Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:48.751539Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:48.751541Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:48.751552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342793482393253:10594]
2025-10-09T21:50:48.751556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342793482393253:10594]
2025-10-09T21:50:48.751560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342793482393253:10594]
2025-10-09T21:50:48.751603Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:48.751605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:48.751610Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:48.751613Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:48.751613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.751616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:48.751762Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:48.753370Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1942 sessionId: src_id|23179f8a-687b93a8-46262283-6797120f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:48.755770Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:48.806381Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342793844226178:4041]
2025-10-09T21:50:48.806621Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 1941, pipe: [1:7559342793844226178:4041]
2025-10-09T21:50:48.896471Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:48.896961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.896977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.897021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342793844226178:4041], now have 1 active actors on pipe
2025-10-09T21:50:48.897195Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:48.897246Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342793844226178:4041]
2025-10-09T21:50:48.897252Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:48.897256Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:48.897281Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:48.897315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1942 reason: error at write: Disconnected sessionId: src_id|23179f8a-687b93a8-46262283-6797120f_0
2025-10-09T21:50:48.897493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1942 sessionId: src_id|23179f8a-687b93a8-46262283-6797120f_0 is DEAD
2025-10-09T21:50:48.897670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23179f8a-687b93a8-46262283-6797120f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:48.897682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23179f8a-687b93a8-46262283-6797120f_0] Write session will restart in 0.006331s
2025-10-09T21:50:48.897712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23179f8a-687b93a8-46262283-6797120f_0] Write session: Do CDS request
2025-10-09T21:50:48.897720Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23179f8a-687b93a8-46262283-6797120f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:48.905513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23179f8a-687b93a8-46262283-6797120f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:48.905559Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:48.905578Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1943
2025-10-09T21:50:48.905725Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1943 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:48.905767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1943 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:48.905776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1943 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:48.905780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:48.906156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1943 sessionId:  describe result for acl check
2025-10-09T21:50:48.906183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:48.906187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:48.906189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:48.906193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342793844226218:4546] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:48.906632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342793844226218:4546] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:48.909136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342793844226218:4546] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:48.912275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342793844226218:4546] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:48.912290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342793844226218:4546] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:48.912293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342793844226218:4546] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:48.912301Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1943 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:48.915224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:48.917812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:48.917993Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:48.918003Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:48.918014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:48.918081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:48.918121Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342793482393533:10598]
2025-10-09T21:50:48.918703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:48.918998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:48.919042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:48.919323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:48.919376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.919382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.919383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.919384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.919386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.919387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.919388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.919389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.919391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.919396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.919406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.919409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.919413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.919417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.919421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.919429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.919433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.919441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.919448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.919457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:48.919468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.919477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:48.919486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.919491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:48.919501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.919506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:48.919511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.919517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:48.919522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.919532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:48.919537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.919542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.919550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.919556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.919561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.919571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.919593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:48.925531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:48.926504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:48.926520Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:48.926522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:48.926525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:48.926529Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1942 [2:7559342793482393533:10598]
2025-10-09T21:50:48.926541Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:48.926553Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:48.926555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:48.926557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:48.926558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:48.926570Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:48.926574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:48.926582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:48.926621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.926627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.926628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.926630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:48.926678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:48.926684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.926691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:48.926701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:48.926713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:48.926733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342793482393528:10598]
2025-10-09T21:50:48.926740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342793482393528:10598]
2025-10-09T21:50:48.926742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342793482393528:10598]
2025-10-09T21:50:48.926754Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:48.926768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342793844226397:4546], now have 1 active actors on pipe
2025-10-09T21:50:48.926769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:48.926772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:48.926774Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:48.926774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342793844226183:2432], now have 1 active actors on pipe
2025-10-09T21:50:48.926782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:48.926790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:48.926792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:48.926810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342793482393528:10598]
2025-10-09T21:50:48.926817Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:48.926808Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1942
2025-10-09T21:50:48.926838Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:48.926830Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1942
2025-10-09T21:50:48.926845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.926848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.926872Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c8306b86-1c315d8a-577da3a5-88471ff3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:48.926877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:48.926880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:48.926881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:48.926888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:48.926896Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:48.926906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:48.927044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.927054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.927065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:48.927154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1943 partition: 0 MaxSeqNo: 25 sessionId: src_id|c8306b86-1c315d8a-577da3a5-88471ff3_0
2025-10-09T21:50:48.927480Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23179f8a-687b93a8-46262283-6797120f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c8306b86-1c315d8a-577da3a5-88471ff3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:48.929860Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8306b86-1c315d8a-577da3a5-88471ff3_0] Write session: try to update token
2025-10-09T21:50:48.929875Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8306b86-1c315d8a-577da3a5-88471ff3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:48.938061Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:48.938079Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:48.938088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342793482393528:10598]
2025-10-09T21:50:48.938130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:48.938135Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:48.938285Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:48.938415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:48.938540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:48.938666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:48.938832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:48.938863Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:48.941843Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1943 sessionId: src_id|c8306b86-1c315d8a-577da3a5-88471ff3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:48.944352Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:48.944567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:48.944585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:48.944621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:48.944714Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:48.950557Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:48.950577Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:48.950579Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:48.950581Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:48.950592Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342793482393528:10598]
2025-10-09T21:50:48.950595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342793482393528:10598]
2025-10-09T21:50:48.950598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342793482393528:10598]
2025-10-09T21:50:48.950638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:48.950645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:48.950648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:48.950684Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:48.950701Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:48.950705Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:48.950799Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:48.997373Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342793844226463:4041]
2025-10-09T21:50:48.997630Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 1942, pipe: [1:7559342793844226463:4041]
2025-10-09T21:50:49.096928Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:49.105326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.105359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.105544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.105615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.105696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.105778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.105830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.105844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.105914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.105986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.106080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.106159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.106266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.106279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.106352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.106425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.106524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.106592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.106644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.106657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.106725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.106788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.106859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.106936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.106982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.106991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.107064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.107139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.107220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.107302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.107358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.107368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.107445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.107523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.107604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.107673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.107717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.107723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.107791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.107851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.107914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.107980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.108022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.108024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.108097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.108158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.108221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.108283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.108343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.108346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.108415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.108475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.108536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.108605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.108668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.108672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.108736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.108790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.108850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.108923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.108968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.108971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.109045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342793844226463:4041], now have 1 active actors on pipe
2025-10-09T21:50:49.109256Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342793844226463:4041]
2025-10-09T21:50:49.109259Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:49.109263Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:49.109288Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:49.109291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1943 reason: error at write: Disconnected sessionId: src_id|c8306b86-1c315d8a-577da3a5-88471ff3_0
2025-10-09T21:50:49.109497Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1943 sessionId: src_id|c8306b86-1c315d8a-577da3a5-88471ff3_0 is DEAD
2025-10-09T21:50:49.109544Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:49.109651Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8306b86-1c315d8a-577da3a5-88471ff3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:49.109662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8306b86-1c315d8a-577da3a5-88471ff3_0] Write session will restart in 0.005816s
2025-10-09T21:50:49.109688Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8306b86-1c315d8a-577da3a5-88471ff3_0] Write session: Do CDS request
2025-10-09T21:50:49.109695Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8306b86-1c315d8a-577da3a5-88471ff3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:49.116577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8306b86-1c315d8a-577da3a5-88471ff3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:49.116655Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:49.116673Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1944
2025-10-09T21:50:49.116799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1944 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:49.116835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1944 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:49.116859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1944 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:49.116863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:49.117321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1944 sessionId:  describe result for acl check
2025-10-09T21:50:49.117362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:49.117369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:49.117371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:49.117377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342798139193802:4568] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:49.117964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342798139193802:4568] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:49.120423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342798139193802:4568] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:49.124098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342798139193802:4568] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:49.124120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342798139193802:4568] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:49.124123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342798139193802:4568] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:49.124134Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1944 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:49.133679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:49.134135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:49.134308Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:49.134319Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:49.134331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:49.134384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:49.134426Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342797777361140:10612]
2025-10-09T21:50:49.134869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:49.135090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:49.135124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:49.135338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:49.135375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.135380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.135380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.135381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.135382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.135383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.135383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.135384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.135385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.135388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.135396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.135398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.135401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.135403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.135405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.135407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.135409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.135411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.135417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.135422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.135426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.135430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.135436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.135443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.135445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.135449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.135451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.135454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.135460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.135463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.135467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.135473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.135476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.135479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.135482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.135485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.135502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:49.141863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:49.143188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:49.143206Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:49.143208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:49.143212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:49.143220Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1943 [2:7559342797777361140:10612]
2025-10-09T21:50:49.143232Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:49.143251Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:49.143257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:49.143260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:49.143261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:49.143277Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:49.143288Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:49.143295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:49.143364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.143371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.143373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.143376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:49.143453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:49.143458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.143467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.143478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.143499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:49.143528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342798139193787:2432], now have 1 active actors on pipe
2025-10-09T21:50:49.143534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342798139193934:4568], now have 1 active actors on pipe
2025-10-09T21:50:49.143540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342797777361135:10612]
2025-10-09T21:50:49.143542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342797777361135:10612]
2025-10-09T21:50:49.143543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342797777361135:10612]
2025-10-09T21:50:49.143554Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:49.143568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:49.143570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:49.143574Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:49.143576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:49.143576Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:49.143579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:49.143621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342797777361135:10612]
2025-10-09T21:50:49.143633Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:49.143643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:49.143683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.143689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.143669Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1943
2025-10-09T21:50:49.143712Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|10333c6b-6cbdf66c-a5455aee-ae6fdb58_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:49.143722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:49.143725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:49.143727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:49.143735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:49.143743Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:49.143755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:49.143825Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1943
2025-10-09T21:50:49.143946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.143953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.143967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:49.144055Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1944 partition: 0 MaxSeqNo: 25 sessionId: src_id|10333c6b-6cbdf66c-a5455aee-ae6fdb58_0
2025-10-09T21:50:49.144453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8306b86-1c315d8a-577da3a5-88471ff3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|10333c6b-6cbdf66c-a5455aee-ae6fdb58_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:49.147106Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10333c6b-6cbdf66c-a5455aee-ae6fdb58_0] Write session: try to update token
2025-10-09T21:50:49.147127Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10333c6b-6cbdf66c-a5455aee-ae6fdb58_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:49.158873Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:49.158893Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:49.158895Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:49.158896Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:49.158903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342797777361135:10612]
2025-10-09T21:50:49.158908Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342797777361135:10612]
2025-10-09T21:50:49.158910Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342797777361135:10612]
2025-10-09T21:50:49.158945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:49.158952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.158954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:49.158963Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:49.158970Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:49.158974Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:49.159077Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:49.160593Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1944 sessionId: src_id|10333c6b-6cbdf66c-a5455aee-ae6fdb58_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:49.163099Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:49.209370Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342798139194071:4041]
2025-10-09T21:50:49.209686Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 1943, pipe: [1:7559342798139194071:4041]
2025-10-09T21:50:49.311680Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:49.312165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.312177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.312223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342798139194071:4041], now have 1 active actors on pipe
2025-10-09T21:50:49.312413Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342798139194071:4041]
2025-10-09T21:50:49.312436Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:49.312422Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:49.312448Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:49.312451Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:49.312512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1944 reason: error at write: Disconnected sessionId: src_id|10333c6b-6cbdf66c-a5455aee-ae6fdb58_0
2025-10-09T21:50:49.312671Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1944 sessionId: src_id|10333c6b-6cbdf66c-a5455aee-ae6fdb58_0 is DEAD
2025-10-09T21:50:49.312863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10333c6b-6cbdf66c-a5455aee-ae6fdb58_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:49.312874Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10333c6b-6cbdf66c-a5455aee-ae6fdb58_0] Write session will restart in 0.007867s
2025-10-09T21:50:49.312903Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10333c6b-6cbdf66c-a5455aee-ae6fdb58_0] Write session: Do CDS request
2025-10-09T21:50:49.312910Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10333c6b-6cbdf66c-a5455aee-ae6fdb58_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:49.321509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10333c6b-6cbdf66c-a5455aee-ae6fdb58_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:49.321625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:49.321642Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1945
2025-10-09T21:50:49.321787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1945 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:49.321820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1945 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:49.321826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1945 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:49.321829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:49.322481Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1945 sessionId:  describe result for acl check
2025-10-09T21:50:49.322524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:49.322531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:49.322533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:49.322539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342798139194120:4567] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:49.323116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342798139194120:4567] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:49.326199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342798139194120:4567] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:49.329153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342798139194120:4567] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:49.329169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342798139194120:4567] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:49.329174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342798139194120:4567] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:49.329183Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1945 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:49.332387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:49.335004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:49.335170Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:49.335178Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:49.335188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:49.335248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:49.335286Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342797777361413:10614]
2025-10-09T21:50:49.335659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:49.335977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:49.336050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:49.336400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:49.336451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.336458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.336460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.336461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.336462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.336463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.336464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.336465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.336467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.336472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.336483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.336487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.336491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.336494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.336498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.336502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.336506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.336510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.336517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.336525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.336531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.336536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.336541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.336546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.336551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.336556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.336560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.336565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.336570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.336575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.336580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.336585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.336593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.336599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.336603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.336609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.336650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:49.342837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:49.343935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:49.343948Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:49.343950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:49.343954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:49.343960Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1944 [2:7559342797777361413:10614]
2025-10-09T21:50:49.343975Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:49.343992Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:49.343995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:49.343998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:49.344002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:49.344021Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:49.344026Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:49.344033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:49.344106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.344114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.344116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.344119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:49.344183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:49.344190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.344202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.344218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.344239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:49.344264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342798139194078:2432], now have 1 active actors on pipe
2025-10-09T21:50:49.344272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342798139194313:4567], now have 1 active actors on pipe
2025-10-09T21:50:49.344279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342797777361408:10614]
2025-10-09T21:50:49.344283Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342797777361408:10614]
2025-10-09T21:50:49.344285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342797777361408:10614]
2025-10-09T21:50:49.344297Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:49.344316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:49.344321Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:49.344325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:49.344327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:49.344334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:49.344338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:49.344325Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1944
2025-10-09T21:50:49.344344Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1944
2025-10-09T21:50:49.344369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342797777361408:10614]
2025-10-09T21:50:49.344377Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:49.344379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:49.344397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.344406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.344452Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|16f83fa6-130a7539-3b256706-3990242a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:49.344464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:49.344467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:49.344470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:49.344480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:49.344494Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:49.344516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:49.344696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.344709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.344730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:49.344808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1945 partition: 0 MaxSeqNo: 25 sessionId: src_id|16f83fa6-130a7539-3b256706-3990242a_0
2025-10-09T21:50:49.345169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10333c6b-6cbdf66c-a5455aee-ae6fdb58_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|16f83fa6-130a7539-3b256706-3990242a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:49.347569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16f83fa6-130a7539-3b256706-3990242a_0] Write session: try to update token
2025-10-09T21:50:49.347584Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16f83fa6-130a7539-3b256706-3990242a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:49.358330Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:49.358349Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:49.358351Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:49.358353Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:49.358362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342797777361408:10614]
2025-10-09T21:50:49.358367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342797777361408:10614]
2025-10-09T21:50:49.358371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342797777361408:10614]
2025-10-09T21:50:49.358418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:49.358430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.358430Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:49.358433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:49.358437Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:49.358441Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:49.358561Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:49.360763Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1945 sessionId: src_id|16f83fa6-130a7539-3b256706-3990242a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:49.363030Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:49.412597Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342798139194361:4041]
2025-10-09T21:50:49.412889Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 1944, pipe: [1:7559342798139194361:4041]
2025-10-09T21:50:49.501816Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:49.502118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.502133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.502180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342798139194361:4041], now have 1 active actors on pipe
2025-10-09T21:50:49.502403Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:49.502472Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342798139194361:4041]
2025-10-09T21:50:49.502485Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:49.502507Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:49.502568Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1945 reason: status is not ok: tablet will be restarted right now sessionId: src_id|16f83fa6-130a7539-3b256706-3990242a_0
2025-10-09T21:50:49.502758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1945 sessionId: src_id|16f83fa6-130a7539-3b256706-3990242a_0 is DEAD
2025-10-09T21:50:49.502927Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16f83fa6-130a7539-3b256706-3990242a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:49.502938Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16f83fa6-130a7539-3b256706-3990242a_0] Write session will restart in 0.009550s
2025-10-09T21:50:49.502974Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16f83fa6-130a7539-3b256706-3990242a_0] Write session: Do CDS request
2025-10-09T21:50:49.502981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16f83fa6-130a7539-3b256706-3990242a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:49.513581Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16f83fa6-130a7539-3b256706-3990242a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:49.513625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:49.513645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1946
2025-10-09T21:50:49.513803Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1946 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:49.513844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1946 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:49.513856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1946 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:49.513860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:49.514285Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1946 sessionId:  describe result for acl check
2025-10-09T21:50:49.514319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:49.514326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:49.514327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:49.514333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342798139194453:4579] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:49.514873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342798139194453:4579] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:49.517453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342798139194453:4579] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:49.519743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342798139194453:4579] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:49.519758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342798139194453:4579] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:49.519761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342798139194453:4579] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:49.519769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1946 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:49.520845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:49.523496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:49.523700Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:49.523710Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:49.523721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:49.523774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:49.523814Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342797777361687:10616]
2025-10-09T21:50:49.524239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:49.524503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:49.524544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:49.524767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:49.524807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.524812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.524814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.524815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.524817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.524818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.524819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.524820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.524821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.524826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.524838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.524842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.524846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.524850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.524854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.524858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.524862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.524866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.524872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.524880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.524892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.524902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.524907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.524915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.524919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.524925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.524934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.524939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.524948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.524953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.524959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.524965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.524973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.524979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.524983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.524988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.525015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:49.531031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:49.532348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:49.532359Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:49.532361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:49.532364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:49.532370Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1945 [2:7559342797777361687:10616]
2025-10-09T21:50:49.532379Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:49.532394Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:49.532396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:49.532398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:49.532399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:49.532413Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:49.532417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:49.532423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:49.532478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.532485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.532486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.532488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:49.532544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:49.532550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.532557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.532563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.532572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:49.532586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342797777361682:10616]
2025-10-09T21:50:49.532591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342797777361682:10616]
2025-10-09T21:50:49.532592Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342797777361682:10616]
2025-10-09T21:50:49.532599Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:49.532610Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:49.532611Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:49.532614Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:49.532616Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:49.532621Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:49.532623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:49.532632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342798139194363:2432], now have 1 active actors on pipe
2025-10-09T21:50:49.532634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342798139194633:4579], now have 1 active actors on pipe
2025-10-09T21:50:49.532659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342797777361682:10616]
2025-10-09T21:50:49.532669Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:49.532700Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:49.532695Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1945
2025-10-09T21:50:49.532708Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1945
2025-10-09T21:50:49.532773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.532782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.532808Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|54638c64-bb3fab4b-2aa3ac41-bc0a8d51_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:49.532818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:49.532821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:49.532823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:49.532831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:49.532838Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:49.532852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:49.533019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.533032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.533055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:49.533144Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1946 partition: 0 MaxSeqNo: 25 sessionId: src_id|54638c64-bb3fab4b-2aa3ac41-bc0a8d51_0
2025-10-09T21:50:49.533518Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16f83fa6-130a7539-3b256706-3990242a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|54638c64-bb3fab4b-2aa3ac41-bc0a8d51_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:49.535840Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54638c64-bb3fab4b-2aa3ac41-bc0a8d51_0] Write session: try to update token
2025-10-09T21:50:49.535857Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54638c64-bb3fab4b-2aa3ac41-bc0a8d51_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:49.546665Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:49.546679Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:49.546681Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:49.546683Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:49.546692Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342797777361682:10616]
2025-10-09T21:50:49.546697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342797777361682:10616]
2025-10-09T21:50:49.546701Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342797777361682:10616]
2025-10-09T21:50:49.546746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:49.546752Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:49.546754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.546757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:49.546757Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:49.546761Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:49.546881Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:49.547815Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1946 sessionId: src_id|54638c64-bb3fab4b-2aa3ac41-bc0a8d51_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:49.550205Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:49.602550Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342798139194647:4041]
2025-10-09T21:50:49.602803Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 1945, pipe: [1:7559342798139194647:4041]
2025-10-09T21:50:49.688338Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:49.688823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.688839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.688886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342798139194647:4041], now have 1 active actors on pipe
2025-10-09T21:50:49.689087Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:49.689094Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:49.689108Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342798139194647:4041]
2025-10-09T21:50:49.689139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:49.689141Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:49.689200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1946 reason: error at write: Disconnected sessionId: src_id|54638c64-bb3fab4b-2aa3ac41-bc0a8d51_0
2025-10-09T21:50:49.689384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1946 sessionId: src_id|54638c64-bb3fab4b-2aa3ac41-bc0a8d51_0 is DEAD
2025-10-09T21:50:49.689541Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54638c64-bb3fab4b-2aa3ac41-bc0a8d51_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:49.689553Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54638c64-bb3fab4b-2aa3ac41-bc0a8d51_0] Write session will restart in 0.006150s
2025-10-09T21:50:49.689576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54638c64-bb3fab4b-2aa3ac41-bc0a8d51_0] Write session: Do CDS request
2025-10-09T21:50:49.689591Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54638c64-bb3fab4b-2aa3ac41-bc0a8d51_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:49.696409Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54638c64-bb3fab4b-2aa3ac41-bc0a8d51_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:49.696501Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:49.696516Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1947
2025-10-09T21:50:49.696617Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1947 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:49.696650Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1947 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:49.696656Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1947 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:49.696659Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:49.697100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1947 sessionId:  describe result for acl check
2025-10-09T21:50:49.697134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:49.697140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:49.697141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:49.697147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342798139194676:4591] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:49.697727Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342798139194676:4591] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:49.700536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342798139194676:4591] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:49.703982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342798139194676:4591] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:49.704001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342798139194676:4591] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:49.704004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342798139194676:4591] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:49.704012Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1947 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:49.707920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:49.710629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:49.710814Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:49.710823Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:49.710834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:49.710882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:49.710926Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342797777361962:10618]
2025-10-09T21:50:49.711488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:49.711826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:49.711871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:49.712162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:49.712213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.712218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.712220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.712221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.712222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.712223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.712224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.712225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.712226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.712231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.712242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.712246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.712249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.712253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.712257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.712261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.712265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.712268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.712275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.712283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.712289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.712294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.712298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.712303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.712307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.712312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.712316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.712326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.712332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.712341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.712346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.712351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.712359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.712364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.712369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.712375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.712407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:49.718557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:49.719647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:49.719657Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:49.719665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:49.719669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:49.719674Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1946 [2:7559342797777361962:10618]
2025-10-09T21:50:49.719684Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:49.719698Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:49.719706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:49.719709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:49.719710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:49.719723Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:49.719728Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:49.719735Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:49.719792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.719798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.719799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.719801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:49.719865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:49.719874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.719884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.719890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.719899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:49.719914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342798139194863:4591], now have 1 active actors on pipe
2025-10-09T21:50:49.719920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342798139194650:2432], now have 1 active actors on pipe
2025-10-09T21:50:49.719926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342797777361957:10618]
2025-10-09T21:50:49.719928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342797777361957:10618]
2025-10-09T21:50:49.719929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342797777361957:10618]
2025-10-09T21:50:49.719939Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:49.719954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:49.719956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:49.719959Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:49.719961Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:49.719965Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:49.719967Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:49.719994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342797777361957:10618]
2025-10-09T21:50:49.720000Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:49.719993Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1946
2025-10-09T21:50:49.720023Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:49.720050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.720003Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1946
2025-10-09T21:50:49.720054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.720079Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ac401c65-75570032-b22995b7-32559d9f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:49.720092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:49.720104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:49.720106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:49.720117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:49.720131Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:49.720148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:49.720321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.720326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.720340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:49.720437Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1947 partition: 0 MaxSeqNo: 25 sessionId: src_id|ac401c65-75570032-b22995b7-32559d9f_0
2025-10-09T21:50:49.720774Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54638c64-bb3fab4b-2aa3ac41-bc0a8d51_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ac401c65-75570032-b22995b7-32559d9f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:49.723233Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac401c65-75570032-b22995b7-32559d9f_0] Write session: try to update token
2025-10-09T21:50:49.723250Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac401c65-75570032-b22995b7-32559d9f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:49.731653Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:49.731669Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:49.731679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342797777361957:10618]
2025-10-09T21:50:49.731729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:49.731728Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:49.731977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:49.732139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:49.732316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:49.732459Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:49.732649Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:49.732693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:49.736456Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1947 sessionId: src_id|ac401c65-75570032-b22995b7-32559d9f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:49.738865Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:49.739082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.739119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.739156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:49.739246Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:49.744455Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:49.744469Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:49.744471Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:49.744473Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:49.744483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342797777361957:10618]
2025-10-09T21:50:49.744486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342797777361957:10618]
2025-10-09T21:50:49.744490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342797777361957:10618]
2025-10-09T21:50:49.744529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:49.744540Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:49.744546Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:49.744550Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:49.744568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.744572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:49.744709Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:49.789413Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342798139194934:4041]
2025-10-09T21:50:49.789708Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 1946, pipe: [1:7559342798139194934:4041]
2025-10-09T21:50:49.886241Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:49.892819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.892843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.892997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.893056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.893120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.893178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.893216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.893222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.893285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.893349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.893411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.893480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.893513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.893524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.893577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.893646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.893703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.893759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.893796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.893802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.893860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.893921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.893979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.894046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.894084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.894092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.894155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.894207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.894258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.894314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.894377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.894384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.894446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.894503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.894558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.894610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.894655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.894663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.894723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.894776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.894837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.894892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.894931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.894936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.894999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.895053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.895105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.895162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.895203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.895208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.895260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.895306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.895357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.895417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.895465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.895475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.895542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:49.895602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:49.895663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:49.895719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:49.895755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:49.895760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:49.895797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342798139194934:4041], now have 1 active actors on pipe
2025-10-09T21:50:49.896019Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342798139194934:4041]
2025-10-09T21:50:49.896024Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:49.896028Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:49.896059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1947 reason: error at write: Disconnected sessionId: src_id|ac401c65-75570032-b22995b7-32559d9f_0
2025-10-09T21:50:49.896068Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:49.896274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1947 sessionId: src_id|ac401c65-75570032-b22995b7-32559d9f_0 is DEAD
2025-10-09T21:50:49.896416Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac401c65-75570032-b22995b7-32559d9f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:49.896428Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac401c65-75570032-b22995b7-32559d9f_0] Write session will restart in 0.009246s
2025-10-09T21:50:49.896459Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac401c65-75570032-b22995b7-32559d9f_0] Write session: Do CDS request
2025-10-09T21:50:49.896466Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac401c65-75570032-b22995b7-32559d9f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:49.896544Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:49.906507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac401c65-75570032-b22995b7-32559d9f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:49.906581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:49.906598Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1948
2025-10-09T21:50:49.906712Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1948 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:49.906752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1948 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:49.906765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1948 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:49.906769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:49.907229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1948 sessionId:  describe result for acl check
2025-10-09T21:50:49.907267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:49.907275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:49.907278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:49.907284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342798139195022:4594] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:49.907832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342798139195022:4594] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:49.911127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342798139195022:4594] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:49.913727Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342798139195022:4594] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:49.913742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342798139195022:4594] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:49.913745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342798139195022:4594] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:49.913752Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1948 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:49.914178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:49.916427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:49.916657Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:49.916665Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:49.916677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:49.916732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:49.916774Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342797777362242:10622]
2025-10-09T21:50:49.917310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:49.917644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:49.917689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:49.917997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:49.918060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.918065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.918066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.918067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.918068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.918069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.918070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.918071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.918072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.918077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.918086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.918090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.918093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.918097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.918101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.918104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.918107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.918110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.918117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.918123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:49.918129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.918135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:49.918139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.918144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:49.918148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.918153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:49.918157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.918161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:49.918171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.918176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:49.918185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.918189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.918193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.918197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.918202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.918206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.918232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:49.924378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:49.925616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:49.925627Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:49.925629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:49.925632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:49.925637Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1947 [2:7559342797777362242:10622]
2025-10-09T21:50:49.925646Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:49.925660Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:49.925661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:49.925663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:49.925664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:49.925677Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:49.925680Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:49.925695Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:49.925736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.925741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.925742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.925744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:49.925792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:49.925797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.925804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:49.925815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:49.925836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:49.925856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342798139195212:4594], now have 1 active actors on pipe
2025-10-09T21:50:49.925861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342798139194939:2432], now have 1 active actors on pipe
2025-10-09T21:50:49.925867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342797777362237:10622]
2025-10-09T21:50:49.925868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342797777362237:10622]
2025-10-09T21:50:49.925869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342797777362237:10622]
2025-10-09T21:50:49.925878Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:49.925888Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:49.925899Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:49.925902Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:49.925904Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:49.925908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:49.925910Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:49.925929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342797777362237:10622]
2025-10-09T21:50:49.925935Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:49.925955Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:49.925935Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1947
2025-10-09T21:50:49.925969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.925939Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1947
2025-10-09T21:50:49.925978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.926004Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6555ac6e-a8468f5c-b880c67f-511d0a2f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:49.926014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:49.926017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:49.926018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:49.926026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:49.926049Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:49.926061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:49.926233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.926239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.926251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:49.926315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1948 partition: 0 MaxSeqNo: 25 sessionId: src_id|6555ac6e-a8468f5c-b880c67f-511d0a2f_0
2025-10-09T21:50:49.926610Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac401c65-75570032-b22995b7-32559d9f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6555ac6e-a8468f5c-b880c67f-511d0a2f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:49.929099Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6555ac6e-a8468f5c-b880c67f-511d0a2f_0] Write session: try to update token
2025-10-09T21:50:49.929115Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6555ac6e-a8468f5c-b880c67f-511d0a2f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:49.937930Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:49.937952Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:49.937964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342797777362237:10622]
2025-10-09T21:50:49.938002Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:49.938022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:49.938422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:49.938707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:49.938972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:49.939232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:49.939606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:49.939670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:49.941592Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1948 sessionId: src_id|6555ac6e-a8468f5c-b880c67f-511d0a2f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:49.943711Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:49.943895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:49.943910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:49.943940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:49.944014Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:49.949879Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:49.949889Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:49.949890Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:49.949891Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:49.949899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342797777362237:10622]
2025-10-09T21:50:49.949902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342797777362237:10622]
2025-10-09T21:50:49.949905Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342797777362237:10622]
2025-10-09T21:50:49.949934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:49.949940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:49.949942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:49.949942Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:49.949948Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:49.949952Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:49.950080Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:49.996270Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342798139195224:4041]
2025-10-09T21:50:49.996486Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 1947, pipe: [1:7559342798139195224:4041]
2025-10-09T21:50:50.102022Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:50.109001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:50.109021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:50.109144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:50.109202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:50.109263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:50.109320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:50.109360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:50.109367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:50.109449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:50.109509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:50.109568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:50.109629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:50.109668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:50.109674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:50.109729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:50.109812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:50.109865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:50.109932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:50.109976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:50.109982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:50.110057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:50.110114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:50.110178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:50.110237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:50.110281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:50.110283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:50.110341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:50.110402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:50.110464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:50.110524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:50.110573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:50.110576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:50.110643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:50.110704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:50.110767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:50.110826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:50.110866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:50.110872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:50.110928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:50.110989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:50.111050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:50.111106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:50.111144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:50.111150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:50.111212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:50.111265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:50.111316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:50.111369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:50.111407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:50.111415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:50.111475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:50.111529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:50.111582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:50.111642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:50.111686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:50.111694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:50.111755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:50.111807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:50.111860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:50.111914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:50.111970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:50.111980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:50.112021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342798139195224:4041], now have 1 active actors on pipe
2025-10-09T21:50:50.112216Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342798139195224:4041]
2025-10-09T21:50:50.112269Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:50.112271Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:50.112299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1948 reason: error at write: Disconnected sessionId: src_id|6555ac6e-a8468f5c-b880c67f-511d0a2f_0
2025-10-09T21:50:50.112560Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1948 sessionId: src_id|6555ac6e-a8468f5c-b880c67f-511d0a2f_0 is DEAD
2025-10-09T21:50:50.112644Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6555ac6e-a8468f5c-b880c67f-511d0a2f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:50.112656Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6555ac6e-a8468f5c-b880c67f-511d0a2f_0] Write session will restart in 0.006103s
2025-10-09T21:50:50.112699Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6555ac6e-a8468f5c-b880c67f-511d0a2f_0] Write session: Do CDS request
2025-10-09T21:50:50.112706Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6555ac6e-a8468f5c-b880c67f-511d0a2f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:50.112675Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:50.113085Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:50.119613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6555ac6e-a8468f5c-b880c67f-511d0a2f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:50.119680Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:50.119696Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1949
2025-10-09T21:50:50.119850Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1949 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:50.119885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1949 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:50.119890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1949 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:50.119894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:50.120318Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1949 sessionId:  describe result for acl check
2025-10-09T21:50:50.120359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:50.120361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:50.120363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:50.120368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342802434162553:4616] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:50.120917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342802434162553:4616] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:50.123787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342802434162553:4616] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:50.127775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342802434162553:4616] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:50.127789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342802434162553:4616] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:50.127791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342802434162553:4616] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:50.127797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1949 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:50.133030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:50.135974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:50.136358Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:50.136371Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:50.136384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:50.136449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:50.136488Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342802072329829:10630]
2025-10-09T21:50:50.137078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:50.137359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:50.137405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:50.137732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:50.137783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.137791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.137792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.137793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.137795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.137796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.137797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.137798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.137799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.137804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.137815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.137818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.137822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.137826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.137830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.137834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.137838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.137842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.137849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.137857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.137863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.137869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.137874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.137879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.137883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.137888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.137893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.137899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.137903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.137908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.137913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.137919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.137922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.137926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.137932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.137937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.137966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:50.144116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:50.145329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:50.145341Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:50.145342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:50.145346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:50.145350Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1948 [2:7559342802072329829:10630]
2025-10-09T21:50:50.145360Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:50.145374Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:50.145376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:50.145378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:50.145379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:50.145391Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:50.145394Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:50.145400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:50.145456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.145463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.145465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.145468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:50.145529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:50.145535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.145542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.145548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.145558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:50.145575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342802434162683:4616], now have 1 active actors on pipe
2025-10-09T21:50:50.145583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342802434162531:2432], now have 1 active actors on pipe
2025-10-09T21:50:50.145589Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342802072329824:10630]
2025-10-09T21:50:50.145590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342802072329824:10630]
2025-10-09T21:50:50.145592Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342802072329824:10630]
2025-10-09T21:50:50.145604Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:50.145614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:50.145623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:50.145624Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:50.145624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:50.145628Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:50.145631Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:50.145645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342802072329824:10630]
2025-10-09T21:50:50.145647Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:50.145650Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:50.145683Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1948
2025-10-09T21:50:50.145722Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1948
2025-10-09T21:50:50.145851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:50.145862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:50.145887Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b9d49c10-8f3f1309-71058597-962e9910_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:50.145900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:50.145903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:50.145904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:50.145911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:50.145921Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:50.145933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:50.146214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:50.146225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:50.146243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:50.146334Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1949 partition: 0 MaxSeqNo: 25 sessionId: src_id|b9d49c10-8f3f1309-71058597-962e9910_0
2025-10-09T21:50:50.146681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6555ac6e-a8468f5c-b880c67f-511d0a2f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b9d49c10-8f3f1309-71058597-962e9910_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:50.149043Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b9d49c10-8f3f1309-71058597-962e9910_0] Write session: try to update token
2025-10-09T21:50:50.149062Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b9d49c10-8f3f1309-71058597-962e9910_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:50.159853Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:50.159870Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:50.159873Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:50.159875Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:50.159884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342802072329824:10630]
2025-10-09T21:50:50.159888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342802072329824:10630]
2025-10-09T21:50:50.159893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342802072329824:10630]
2025-10-09T21:50:50.159933Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:50.159938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:50.159938Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:50.159941Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:50.159946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.159948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:50.160083Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:50.161866Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1949 sessionId: src_id|b9d49c10-8f3f1309-71058597-962e9910_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:50.164180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:50.212351Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342802434162837:4041]
2025-10-09T21:50:50.212673Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 1948, pipe: [1:7559342802434162837:4041]
2025-10-09T21:50:50.308146Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:50.308588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:50.308606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:50.308650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342802434162837:4041], now have 1 active actors on pipe
2025-10-09T21:50:50.308858Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:50.308895Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:50.308911Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:50.308934Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342802434162837:4041]
2025-10-09T21:50:50.308957Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:50.308972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1949 reason: error at write: Disconnected sessionId: src_id|b9d49c10-8f3f1309-71058597-962e9910_0
2025-10-09T21:50:50.309121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1949 sessionId: src_id|b9d49c10-8f3f1309-71058597-962e9910_0 is DEAD
2025-10-09T21:50:50.309300Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9d49c10-8f3f1309-71058597-962e9910_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:50.309308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9d49c10-8f3f1309-71058597-962e9910_0] Write session will restart in 0.008970s
2025-10-09T21:50:50.309337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9d49c10-8f3f1309-71058597-962e9910_0] Write session: Do CDS request
2025-10-09T21:50:50.309343Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9d49c10-8f3f1309-71058597-962e9910_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:50.319526Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b9d49c10-8f3f1309-71058597-962e9910_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:50.319624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:50.319642Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1950
2025-10-09T21:50:50.319766Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1950 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:50.319799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1950 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:50.319809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1950 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:50.319814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:50.320403Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1950 sessionId:  describe result for acl check
2025-10-09T21:50:50.320445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:50.320447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:50.320448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:50.320454Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342802434162943:4626] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:50.320927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342802434162943:4626] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:50.323499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342802434162943:4626] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:50.326143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342802434162943:4626] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:50.326159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342802434162943:4626] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:50.326163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342802434162943:4626] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:50.326172Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1950 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:50.327143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:50.329260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:50.329612Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:50.329623Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:50.329638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:50.329705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:50.329746Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342802072330132:10638]
2025-10-09T21:50:50.330332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:50.330593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:50.330623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:50.330838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:50.330875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.330879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.330880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.330881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.330882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.330882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.330883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.330883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.330884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.330888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.330895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.330899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.330903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.330906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.330910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.330913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.330916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.330919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.330926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.330934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.330940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.330945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.330949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.330953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.330957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.330962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.330966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.330971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.330975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.330980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.330990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.330995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.330999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.331004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.331008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.331011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.331028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:50.337101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:50.338226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:50.338240Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:50.338243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:50.338248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:50.338254Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1949 [2:7559342802072330132:10638]
2025-10-09T21:50:50.338268Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:50.338285Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:50.338288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:50.338291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:50.338293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:50.338312Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:50.338323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:50.338333Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:50.338392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.338401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.338403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.338406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:50.338612Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1949
2025-10-09T21:50:50.338628Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1949
2025-10-09T21:50:50.338681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:50.338687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.338695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.338701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.338710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:50.338728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342802434163118:4626], now have 1 active actors on pipe
2025-10-09T21:50:50.338734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342802434162844:2432], now have 1 active actors on pipe
2025-10-09T21:50:50.338752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:50.338759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:50.338774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342802072330127:10638]
2025-10-09T21:50:50.338778Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342802072330127:10638]
2025-10-09T21:50:50.338780Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342802072330127:10638]
2025-10-09T21:50:50.338791Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:50.338803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:50.338814Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:50.338816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:50.338821Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:50.338825Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:50.338827Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:50.338833Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|29a8bb0-ee52cb14-efcbd38a-2c954ffe_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:50.338838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:50.338841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:50.338843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:50.338853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:50.338863Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:50.338893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342802072330127:10638]
2025-10-09T21:50:50.338902Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:50.338908Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:50.338911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:50.339037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:50.339045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:50.339056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:50.339138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1950 partition: 0 MaxSeqNo: 25 sessionId: src_id|29a8bb0-ee52cb14-efcbd38a-2c954ffe_0
2025-10-09T21:50:50.339486Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9d49c10-8f3f1309-71058597-962e9910_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|29a8bb0-ee52cb14-efcbd38a-2c954ffe_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:50.341945Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29a8bb0-ee52cb14-efcbd38a-2c954ffe_0] Write session: try to update token
2025-10-09T21:50:50.341967Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29a8bb0-ee52cb14-efcbd38a-2c954ffe_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:50.353142Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:50.353160Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:50.353163Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:50.353164Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:50.353175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342802072330127:10638]
2025-10-09T21:50:50.353179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342802072330127:10638]
2025-10-09T21:50:50.353183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342802072330127:10638]
2025-10-09T21:50:50.353227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:50.353233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.353235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:50.353237Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:50.353244Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:50.353254Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:50.353375Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:50.356107Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1950 sessionId: src_id|29a8bb0-ee52cb14-efcbd38a-2c954ffe_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:50.358482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:50.408921Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342802434163132:4041]
2025-10-09T21:50:50.409185Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 1949, pipe: [1:7559342802434163132:4041]
2025-10-09T21:50:50.500387Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:50.500745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:50.500757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:50.500797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342802434163132:4041], now have 1 active actors on pipe
2025-10-09T21:50:50.500986Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:50.500997Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:50.501119Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342802434163132:4041]
2025-10-09T21:50:50.501177Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:50.501250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1950 reason: status is not ok: tablet will be restarted right now sessionId: src_id|29a8bb0-ee52cb14-efcbd38a-2c954ffe_0
2025-10-09T21:50:50.501419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1950 sessionId: src_id|29a8bb0-ee52cb14-efcbd38a-2c954ffe_0 is DEAD
2025-10-09T21:50:50.501581Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29a8bb0-ee52cb14-efcbd38a-2c954ffe_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:50.501592Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29a8bb0-ee52cb14-efcbd38a-2c954ffe_0] Write session will restart in 0.009110s
2025-10-09T21:50:50.501616Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29a8bb0-ee52cb14-efcbd38a-2c954ffe_0] Write session: Do CDS request
2025-10-09T21:50:50.501623Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29a8bb0-ee52cb14-efcbd38a-2c954ffe_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:50.511546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29a8bb0-ee52cb14-efcbd38a-2c954ffe_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:50.511644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:50.511662Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1951
2025-10-09T21:50:50.511790Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1951 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:50.511820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1951 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:50.511826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1951 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:50.511829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:50.512221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1951 sessionId:  describe result for acl check
2025-10-09T21:50:50.512258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:50.512264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:50.512266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:50.512271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342802434163206:4631] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:50.512792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342802434163206:4631] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:50.515837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342802434163206:4631] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:50.518433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342802434163206:4631] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:50.518448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342802434163206:4631] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:50.518451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342802434163206:4631] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:50.518457Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1951 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:50.520594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:50.524322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:50.524511Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:50.524519Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:50.524530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:50.524584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:50.524622Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342802072330411:10640]
2025-10-09T21:50:50.525169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:50.525444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:50.525484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:50.525778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:50.525833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.525839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.525840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.525841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.525842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.525844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.525845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.525846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.525847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.525852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.525862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.525866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.525869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.525872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.525876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.525879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.525883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.525887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.525894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.525902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.525908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.525913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.525917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.525921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.525931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.525935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.525940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.525949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.525952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.525957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.525966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.525971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.525975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.525980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.525987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.525992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.526016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:50.532090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:50.533286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:50.533301Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:50.533303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:50.533307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:50.533314Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1950 [2:7559342802072330411:10640]
2025-10-09T21:50:50.533328Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:50.533346Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:50.533349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:50.533352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:50.533354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:50.533371Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:50.533380Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:50.533387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:50.533456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.533462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.533465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.533468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:50.533536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:50.533542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.533553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.533568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.533578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:50.533603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342802434163403:4631], now have 1 active actors on pipe
2025-10-09T21:50:50.533611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342802434163134:2432], now have 1 active actors on pipe
2025-10-09T21:50:50.533618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342802072330406:10640]
2025-10-09T21:50:50.533621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342802072330406:10640]
2025-10-09T21:50:50.533622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342802072330406:10640]
2025-10-09T21:50:50.533633Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:50.533643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:50.533645Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:50.533648Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:50.533649Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:50.533656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:50.533658Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:50.533669Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1950
2025-10-09T21:50:50.533680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342802072330406:10640]
2025-10-09T21:50:50.533687Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:50.533673Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1950
2025-10-09T21:50:50.533688Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:50.533716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:50.533719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:50.533746Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e2e45a5d-e2a04f75-7eae7b7-f9852d0a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:50.533755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:50.533758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:50.533760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:50.533768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:50.533778Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:50.533793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:50.533933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:50.533938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:50.533949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:50.534077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1951 partition: 0 MaxSeqNo: 25 sessionId: src_id|e2e45a5d-e2a04f75-7eae7b7-f9852d0a_0
2025-10-09T21:50:50.534434Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29a8bb0-ee52cb14-efcbd38a-2c954ffe_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e2e45a5d-e2a04f75-7eae7b7-f9852d0a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:50.536662Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2e45a5d-e2a04f75-7eae7b7-f9852d0a_0] Write session: try to update token
2025-10-09T21:50:50.536680Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2e45a5d-e2a04f75-7eae7b7-f9852d0a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:50.548142Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:50.548164Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:50.548167Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:50.548170Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:50.548185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342802072330406:10640]
2025-10-09T21:50:50.548192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342802072330406:10640]
2025-10-09T21:50:50.548198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342802072330406:10640]
2025-10-09T21:50:50.548250Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:50.548263Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:50.548267Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:50.548279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:50.548289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.548292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:50.548546Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:50.548925Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1951 sessionId: src_id|e2e45a5d-e2a04f75-7eae7b7-f9852d0a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:50.551204Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:50.601471Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342802434163423:4041]
2025-10-09T21:50:50.601740Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 1950, pipe: [1:7559342802434163423:4041]
2025-10-09T21:50:50.686918Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:50.687127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:50.687140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:50.687175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342802434163423:4041], now have 1 active actors on pipe
2025-10-09T21:50:50.687383Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342802434163423:4041]
2025-10-09T21:50:50.687385Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:50.687390Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:50.687417Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:50.687496Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1951 reason: error at write: Disconnected sessionId: src_id|e2e45a5d-e2a04f75-7eae7b7-f9852d0a_0
2025-10-09T21:50:50.687575Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:50.687668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1951 sessionId: src_id|e2e45a5d-e2a04f75-7eae7b7-f9852d0a_0 is DEAD
2025-10-09T21:50:50.687821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2e45a5d-e2a04f75-7eae7b7-f9852d0a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:50.687831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2e45a5d-e2a04f75-7eae7b7-f9852d0a_0] Write session will restart in 0.005263s
2025-10-09T21:50:50.687859Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2e45a5d-e2a04f75-7eae7b7-f9852d0a_0] Write session: Do CDS request
2025-10-09T21:50:50.687863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2e45a5d-e2a04f75-7eae7b7-f9852d0a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:50.694446Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e2e45a5d-e2a04f75-7eae7b7-f9852d0a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:50.694507Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:50.694523Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1952
2025-10-09T21:50:50.694700Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1952 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:50.694733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1952 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:50.694739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1952 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:50.694748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:50.695292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1952 sessionId:  describe result for acl check
2025-10-09T21:50:50.695327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:50.695333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:50.695334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:50.695339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342802434163450:4644] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:50.695865Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342802434163450:4644] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:50.699362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342802434163450:4644] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:50.702758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342802434163450:4644] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:50.702773Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342802434163450:4644] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:50.702776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342802434163450:4644] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:50.702784Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1952 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:50.706925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:50.709331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:50.709495Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:50.709503Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:50.709515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:50.709562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:50.709602Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342802072330691:10642]
2025-10-09T21:50:50.710020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:50.710287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:50.710314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:50.710514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:50.710564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.710570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.710571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.710572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.710573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.710574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.710576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.710576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.710577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.710582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.710594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.710597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.710600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.710602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.710604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.710606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.710608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.710610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.710615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.710623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.710628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.710632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.710634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.710637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.710640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.710643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.710645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.710651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.710653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.710656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.710659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.710662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.710666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.710668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.710672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.710675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.710690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:50.716538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:50.717935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:50.717950Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:50.717953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:50.717957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:50.717963Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1951 [2:7559342802072330691:10642]
2025-10-09T21:50:50.717979Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:50.717997Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:50.718005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:50.718008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:50.718010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:50.718044Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:50.718054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:50.718066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:50.718128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.718136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.718138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.718141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:50.718208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:50.718217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.718229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.718241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.718256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:50.718283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342802434163620:4644], now have 1 active actors on pipe
2025-10-09T21:50:50.718297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342802072330686:10642]
2025-10-09T21:50:50.718300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342802072330686:10642]
2025-10-09T21:50:50.718302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342802072330686:10642]
2025-10-09T21:50:50.718315Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:50.718324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342802434163426:2432], now have 1 active actors on pipe
2025-10-09T21:50:50.718328Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:50.718330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:50.718332Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:50.718334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:50.718348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:50.718351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:50.718379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342802072330686:10642]
2025-10-09T21:50:50.718386Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:50.718363Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1951
2025-10-09T21:50:50.718402Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:50.718373Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1951
2025-10-09T21:50:50.718415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:50.718420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:50.718455Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a55657c9-51ed1395-5bf793a0-6089d3b2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:50.718469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:50.718477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:50.718480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:50.718490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:50.718505Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:50.718526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:50.718663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:50.718681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:50.718698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:50.718788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1952 partition: 0 MaxSeqNo: 25 sessionId: src_id|a55657c9-51ed1395-5bf793a0-6089d3b2_0
2025-10-09T21:50:50.719103Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e2e45a5d-e2a04f75-7eae7b7-f9852d0a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a55657c9-51ed1395-5bf793a0-6089d3b2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:50.721462Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a55657c9-51ed1395-5bf793a0-6089d3b2_0] Write session: try to update token
2025-10-09T21:50:50.721479Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a55657c9-51ed1395-5bf793a0-6089d3b2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:50.732170Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:50.732186Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:50.732188Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:50.732189Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:50.732197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342802072330686:10642]
2025-10-09T21:50:50.732200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342802072330686:10642]
2025-10-09T21:50:50.732202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342802072330686:10642]
2025-10-09T21:50:50.732241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:50.732249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.732251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:50.732251Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:50.732256Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:50.732262Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:50.732369Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:50.733859Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1952 sessionId: src_id|a55657c9-51ed1395-5bf793a0-6089d3b2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:50.736140Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:50.787305Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342802434163715:4041]
2025-10-09T21:50:50.787556Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 1951, pipe: [1:7559342802434163715:4041]
2025-10-09T21:50:50.873875Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:50.874167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:50.874184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:50.874232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342802434163715:4041], now have 1 active actors on pipe
2025-10-09T21:50:50.874441Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:50.874453Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:50.874473Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:50.874477Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:50.874512Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342802434163715:4041]
2025-10-09T21:50:50.874548Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1952 reason: error at write: Disconnected sessionId: src_id|a55657c9-51ed1395-5bf793a0-6089d3b2_0
2025-10-09T21:50:50.874708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1952 sessionId: src_id|a55657c9-51ed1395-5bf793a0-6089d3b2_0 is DEAD
2025-10-09T21:50:50.874856Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a55657c9-51ed1395-5bf793a0-6089d3b2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:50.874867Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a55657c9-51ed1395-5bf793a0-6089d3b2_0] Write session will restart in 0.006713s
2025-10-09T21:50:50.874905Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a55657c9-51ed1395-5bf793a0-6089d3b2_0] Write session: Do CDS request
2025-10-09T21:50:50.874912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a55657c9-51ed1395-5bf793a0-6089d3b2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:50.882498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a55657c9-51ed1395-5bf793a0-6089d3b2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:50.882556Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:50.882573Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1953
2025-10-09T21:50:50.882708Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1953 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:50.882750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1953 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:50.882760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1953 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:50.882768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:50.883204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1953 sessionId:  describe result for acl check
2025-10-09T21:50:50.883242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:50.883252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:50.883254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:50.883261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342802434163776:4653] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:50.883547Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:50:50.883565Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 16
2025-10-09T21:50:50.883773Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342802434163776:4653] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:50.886755Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342802434163776:4653] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:50.889840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342802434163776:4653] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:50.889868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342802434163776:4653] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:50.889872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342802434163776:4653] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:50.889880Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1953 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:50.893789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:50.896527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:50.896734Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:50.896743Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:50.896754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:50.896806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:50.896844Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342802072330976:10646]
2025-10-09T21:50:50.897467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:50.897689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:50.897729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:50.898045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:50.898092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.898099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.898101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.898102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.898103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.898104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.898105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.898106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.898107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.898111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.898122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.898126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.898130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.898134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.898139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.898142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.898146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.898150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.898158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.898165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:50.898172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.898178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:50.898187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.898192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:50.898196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.898206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:50.898216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.898221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:50.898225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.898230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:50.898239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.898248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.898257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.898262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.898271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.898281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.898310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:50.904575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:50.905746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:50.905756Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:50.905758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:50.905761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:50.905767Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1952 [2:7559342802072330976:10646]
2025-10-09T21:50:50.905778Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:50.905791Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:50.905793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:50.905795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:50.905797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:50.905810Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:50.905816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:50.905823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:50.905878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.905886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.905888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.905891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:50.905945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:50.905951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.905957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:50.905967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:50.905976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:50.905993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342802434163945:4653], now have 1 active actors on pipe
2025-10-09T21:50:50.905999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342802434163747:2432], now have 1 active actors on pipe
2025-10-09T21:50:50.906005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342802072330971:10646]
2025-10-09T21:50:50.906011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342802072330971:10646]
2025-10-09T21:50:50.906012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342802072330971:10646]
2025-10-09T21:50:50.906021Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:50.906045Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:50.906046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:50.906049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:50.906051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:50.906064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:50.906067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:50.906095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342802072330971:10646]
2025-10-09T21:50:50.906098Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:50.906113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:50.906086Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1952
2025-10-09T21:50:50.906100Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1952
2025-10-09T21:50:50.906129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:50.906138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:50.906190Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|60c38be7-c246df62-4bb9d861-82698390_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:50.906203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:50.906206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:50.906208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:50.906218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:50.906229Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:50.906285Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:50:50.906314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:50.906467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:50.906475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:50.906489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:50.906508Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 21 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:50:50.906545Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:50:50.906592Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1953 partition: 0 MaxSeqNo: 25 sessionId: src_id|60c38be7-c246df62-4bb9d861-82698390_0
2025-10-09T21:50:50.906912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a55657c9-51ed1395-5bf793a0-6089d3b2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|60c38be7-c246df62-4bb9d861-82698390_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:50.909353Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|60c38be7-c246df62-4bb9d861-82698390_0] Write session: try to update token
2025-10-09T21:50:50.909369Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|60c38be7-c246df62-4bb9d861-82698390_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:50.917695Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:50.917719Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:50.917726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342802072330971:10646]
2025-10-09T21:50:50.917764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:50.917780Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:50.917957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:50.918109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:50.918245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:50.918384Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:50.918577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:50.918622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:50.921925Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1953 sessionId: src_id|60c38be7-c246df62-4bb9d861-82698390_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:50.924096Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:50.924610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:50.924628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:50.924667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:50.924754Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:50.930927Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:50.930941Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:50.930943Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:50.930945Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:50.930955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342802072330971:10646]
2025-10-09T21:50:50.930959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342802072330971:10646]
2025-10-09T21:50:50.930963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342802072330971:10646]
2025-10-09T21:50:50.931000Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:50.931005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:50.931005Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:50.931008Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:50.931013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:50.931015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:50.931164Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:50.974722Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342802434164042:4041]
2025-10-09T21:50:50.974958Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 1952, pipe: [1:7559342802434164042:4041]
2025-10-09T21:50:51.082886Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:51.091384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:51.091407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:51.091558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:51.091627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:51.091694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:51.091763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:51.092081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:51.092091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:51.092163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:51.092234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:51.092298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:51.092359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:51.092399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:51.092405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:51.092465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:51.092536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:51.092603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:51.092679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:51.092734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:51.092742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:51.092807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:51.092867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:51.092930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:51.092995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:51.093036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:51.093044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:51.093122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:51.093183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:51.093248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:51.093313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:51.093356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:51.093366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:51.093446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:51.093509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:51.093584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:51.093659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:51.093711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:51.093719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:51.093785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:51.093843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:51.093904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:51.093959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:51.093999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:51.094001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:51.094081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:51.094140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:51.094199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:51.094257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:51.094296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:51.094304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:51.094360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:51.094420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:51.094486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:51.094551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:51.094623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:51.094631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:51.094691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:51.094747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:51.094820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:51.094881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:51.094934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:51.094942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:51.095016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342802434164042:4041], now have 1 active actors on pipe
2025-10-09T21:50:51.095225Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:51.095274Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:51.095291Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:51.095306Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342802434164042:4041]
2025-10-09T21:50:51.095321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1953 reason: error at write: Disconnected sessionId: src_id|60c38be7-c246df62-4bb9d861-82698390_0
2025-10-09T21:50:51.095330Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:51.095520Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1953 sessionId: src_id|60c38be7-c246df62-4bb9d861-82698390_0 is DEAD
2025-10-09T21:50:51.095648Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60c38be7-c246df62-4bb9d861-82698390_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:51.095660Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60c38be7-c246df62-4bb9d861-82698390_0] Write session will restart in 0.008711s
2025-10-09T21:50:51.095685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60c38be7-c246df62-4bb9d861-82698390_0] Write session: Do CDS request
2025-10-09T21:50:51.095692Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60c38be7-c246df62-4bb9d861-82698390_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:51.105663Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|60c38be7-c246df62-4bb9d861-82698390_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:51.105657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:51.105678Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1954
2025-10-09T21:50:51.105904Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1954 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:51.105945Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1954 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:51.105949Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1954 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:51.105954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:51.106470Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1954 sessionId:  describe result for acl check
2025-10-09T21:50:51.106520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:51.106530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:51.106533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:51.106540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342806729131400:4670] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:51.107124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342806729131400:4670] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:51.110913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342806729131400:4670] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:51.114279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342806729131400:4670] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:51.114295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342806729131400:4670] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:51.114300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342806729131400:4670] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:51.114308Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1954 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:51.115875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:51.118901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:51.119304Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:51.119317Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:51.119337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:51.119418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:51.119468Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342806367298556:10649]
2025-10-09T21:50:51.120094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:51.120431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:51.120482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:51.120791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:51.120860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.120867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.120868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.120870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.120871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.120872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.120873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.120873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.120874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.120878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.120886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.120889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.120891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.120893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.120895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.120898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.120901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.120905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.120912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.120922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.120928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.120933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.120938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.120943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.120948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.120953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.120966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.120969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.120971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.120975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.120978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.120984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.120988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.120996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.121001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.121006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.121035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:51.127472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:51.128598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:51.128614Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:51.128617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:51.128622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:51.128629Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1953 [2:7559342806367298556:10649]
2025-10-09T21:50:51.128647Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:51.128666Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:51.128674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:51.128678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:51.128680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:51.128698Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:51.128709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:51.128720Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:51.128785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.128793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.128796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.128798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:51.128881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:51.128890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.128903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.128915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.128938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:51.128963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342806729131344:2432], now have 1 active actors on pipe
2025-10-09T21:50:51.128972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342806729131620:4670], now have 1 active actors on pipe
2025-10-09T21:50:51.128979Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342806367298551:10649]
2025-10-09T21:50:51.128987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342806367298551:10649]
2025-10-09T21:50:51.128990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342806367298551:10649]
2025-10-09T21:50:51.129003Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:51.129017Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:51.129024Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:51.129026Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:51.129029Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:51.129038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:51.129045Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:51.129083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342806367298551:10649]
2025-10-09T21:50:51.129091Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:51.129053Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1953
2025-10-09T21:50:51.129077Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1953
2025-10-09T21:50:51.129102Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:51.129452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:51.129465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:51.129509Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d2e67668-11d1fdfb-3f9ac890-1a8dc54_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:51.129524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:51.129528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:51.129531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:51.129545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:51.129557Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:51.129581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:51.129739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:51.129748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:51.129764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:51.129876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1954 partition: 0 MaxSeqNo: 25 sessionId: src_id|d2e67668-11d1fdfb-3f9ac890-1a8dc54_0
2025-10-09T21:50:51.130209Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60c38be7-c246df62-4bb9d861-82698390_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d2e67668-11d1fdfb-3f9ac890-1a8dc54_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:51.132719Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d2e67668-11d1fdfb-3f9ac890-1a8dc54_0] Write session: try to update token
2025-10-09T21:50:51.132740Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d2e67668-11d1fdfb-3f9ac890-1a8dc54_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:51.144109Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:51.144132Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:51.144135Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:51.144138Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:51.144147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342806367298551:10649]
2025-10-09T21:50:51.144152Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342806367298551:10649]
2025-10-09T21:50:51.144156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342806367298551:10649]
2025-10-09T21:50:51.144205Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:51.144208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:51.144211Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:51.144215Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:51.144216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.144219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:51.144373Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:51.147566Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1954 sessionId: src_id|d2e67668-11d1fdfb-3f9ac890-1a8dc54_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:51.150291Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:51.195353Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342806729131658:4041]
2025-10-09T21:50:51.195516Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 1953, pipe: [1:7559342806729131658:4041]
2025-10-09T21:50:51.301913Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:51.302366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:51.302381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:51.302426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342806729131658:4041], now have 1 active actors on pipe
2025-10-09T21:50:51.302658Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342806729131658:4041]
2025-10-09T21:50:51.302675Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:51.302694Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:51.302697Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:51.302755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1954 reason: error at write: Disconnected sessionId: src_id|d2e67668-11d1fdfb-3f9ac890-1a8dc54_0
2025-10-09T21:50:51.302812Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:51.302955Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1954 sessionId: src_id|d2e67668-11d1fdfb-3f9ac890-1a8dc54_0 is DEAD
2025-10-09T21:50:51.303126Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d2e67668-11d1fdfb-3f9ac890-1a8dc54_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:51.303138Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d2e67668-11d1fdfb-3f9ac890-1a8dc54_0] Write session will restart in 0.008902s
2025-10-09T21:50:51.303184Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d2e67668-11d1fdfb-3f9ac890-1a8dc54_0] Write session: Do CDS request
2025-10-09T21:50:51.303190Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d2e67668-11d1fdfb-3f9ac890-1a8dc54_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:51.313548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d2e67668-11d1fdfb-3f9ac890-1a8dc54_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:51.313624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:51.313642Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1955
2025-10-09T21:50:51.313764Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1955 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:51.313801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1955 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:51.313811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1955 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:51.313815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:51.314278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1955 sessionId:  describe result for acl check
2025-10-09T21:50:51.314315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:51.314325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:51.314327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:51.314333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342806729131730:4671] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:51.314889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342806729131730:4671] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:51.317990Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342806729131730:4671] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:51.321855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342806729131730:4671] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:51.321871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342806729131730:4671] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:51.321876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342806729131730:4671] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:51.321886Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1955 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:51.323980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:51.326312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:51.326591Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:51.326602Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:51.326617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:51.326680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:51.326726Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342806367298877:10663]
2025-10-09T21:50:51.327309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:51.327605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:51.327654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:51.328009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:51.328073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.328081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.328083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.328084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.328085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.328086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.328087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.328088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.328089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.328095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.328107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.328111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.328116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.328119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.328123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.328127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.328130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.328134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.328141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.328149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.328155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.328160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.328164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.328169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.328187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.328193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.328198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.328203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.328207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.328212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.328217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.328222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.328226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.328231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.328236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.328241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.328273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:51.334424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:51.335771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:51.335784Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:51.335786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:51.335790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:51.335796Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1954 [2:7559342806367298877:10663]
2025-10-09T21:50:51.335808Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:51.335825Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:51.335832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:51.335834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:51.335837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:51.335855Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:51.335868Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:51.335876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:51.335929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.335936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.335938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.335940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:51.336010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:51.336017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.336028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.336038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.336052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:51.336073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342806729131669:2432], now have 1 active actors on pipe
2025-10-09T21:50:51.336081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342806729131945:4671], now have 1 active actors on pipe
2025-10-09T21:50:51.336088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342806367298872:10663]
2025-10-09T21:50:51.336091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342806367298872:10663]
2025-10-09T21:50:51.336093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342806367298872:10663]
2025-10-09T21:50:51.336105Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:51.336122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:51.336128Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:51.336121Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1954
2025-10-09T21:50:51.336131Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:51.336133Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:51.336140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:51.336143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:51.336168Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342806367298872:10663]
2025-10-09T21:50:51.336140Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1954
2025-10-09T21:50:51.336175Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:51.336188Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:51.336195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:51.336200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:51.336256Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cd6898ea-71c0583e-5f0d1b4-b4094e3b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:51.336270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:51.336274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:51.336276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:51.336285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:51.336294Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:51.336311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:51.336478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:51.336499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:51.336520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:51.336614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1955 partition: 0 MaxSeqNo: 25 sessionId: src_id|cd6898ea-71c0583e-5f0d1b4-b4094e3b_0
2025-10-09T21:50:51.336994Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d2e67668-11d1fdfb-3f9ac890-1a8dc54_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cd6898ea-71c0583e-5f0d1b4-b4094e3b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:51.339425Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd6898ea-71c0583e-5f0d1b4-b4094e3b_0] Write session: try to update token
2025-10-09T21:50:51.339445Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd6898ea-71c0583e-5f0d1b4-b4094e3b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:51.350375Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:51.350395Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:51.350397Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:51.350399Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:51.350410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342806367298872:10663]
2025-10-09T21:50:51.350414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342806367298872:10663]
2025-10-09T21:50:51.350419Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342806367298872:10663]
2025-10-09T21:50:51.350465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:51.350472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.350472Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:51.350476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:51.350479Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:51.350483Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:51.350601Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:51.353521Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1955 sessionId: src_id|cd6898ea-71c0583e-5f0d1b4-b4094e3b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:51.356035Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:51.402755Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342806729131962:4041]
2025-10-09T21:50:51.403001Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 1954, pipe: [1:7559342806729131962:4041]
2025-10-09T21:50:51.495758Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:51.496190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:51.496202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:51.496236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342806729131962:4041], now have 1 active actors on pipe
2025-10-09T21:50:51.496526Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:51.496529Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342806729131962:4041]
2025-10-09T21:50:51.496566Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:51.496569Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:51.496605Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:51.496637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1955 reason: error at write: Disconnected sessionId: src_id|cd6898ea-71c0583e-5f0d1b4-b4094e3b_0
2025-10-09T21:50:51.496835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1955 sessionId: src_id|cd6898ea-71c0583e-5f0d1b4-b4094e3b_0 is DEAD
2025-10-09T21:50:51.496992Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd6898ea-71c0583e-5f0d1b4-b4094e3b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:51.497001Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd6898ea-71c0583e-5f0d1b4-b4094e3b_0] Write session will restart in 0.008376s
2025-10-09T21:50:51.497027Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd6898ea-71c0583e-5f0d1b4-b4094e3b_0] Write session: Do CDS request
2025-10-09T21:50:51.497031Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd6898ea-71c0583e-5f0d1b4-b4094e3b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:51.506485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd6898ea-71c0583e-5f0d1b4-b4094e3b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:51.506588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:51.506603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1956
2025-10-09T21:50:51.506732Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1956 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:51.506777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1956 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:51.506787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1956 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:51.506791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:51.507198Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1956 sessionId:  describe result for acl check
2025-10-09T21:50:51.507234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:51.507240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:51.507241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:51.507246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342806729132035:4692] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:51.507792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342806729132035:4692] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:51.510803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342806729132035:4692] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:51.513353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342806729132035:4692] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:51.513365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342806729132035:4692] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:51.513369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342806729132035:4692] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:51.513376Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1956 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:51.514476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:51.517127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:51.517315Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:51.517326Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:51.517338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:51.517387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:51.517439Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342806367299161:10665]
2025-10-09T21:50:51.517885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:51.518209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:51.518253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:51.518554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:51.518599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.518604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.518606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.518607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.518608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.518609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.518610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.518611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.518612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.518617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.518628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.518632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.518636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.518640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.518644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.518648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.518652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.518655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.518662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.518670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.518676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.518681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.518685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.518690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.518699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.518704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.518709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.518718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.518722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.518728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.518732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.518738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.518744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.518749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.518754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.518764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.518790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:51.524772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:51.525815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:51.525832Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:51.525834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:51.525839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:51.525846Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1955 [2:7559342806367299161:10665]
2025-10-09T21:50:51.525861Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:51.525880Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:51.525883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:51.525885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:51.525888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:51.525905Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:51.525917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:51.525924Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:51.526334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.526344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.526346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.526348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:51.526419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:51.526428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.526441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.526451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.526465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:51.526485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342806729132245:4692], now have 1 active actors on pipe
2025-10-09T21:50:51.526494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342806729131964:2432], now have 1 active actors on pipe
2025-10-09T21:50:51.526500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342806367299156:10665]
2025-10-09T21:50:51.526504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342806367299156:10665]
2025-10-09T21:50:51.526507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342806367299156:10665]
2025-10-09T21:50:51.526520Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:51.526537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:51.526545Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:51.526549Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:51.526550Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:51.526529Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1955
2025-10-09T21:50:51.526551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:51.526531Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1955
2025-10-09T21:50:51.526555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:51.526591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:51.526600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:51.526614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342806367299156:10665]
2025-10-09T21:50:51.526624Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:51.526628Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:51.526652Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|12aa8254-69e30439-55897020-23366d02_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:51.526664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:51.526668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:51.526676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:51.526687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:51.526702Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:51.526726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:51.526860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:51.526871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:51.526885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:51.526980Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1956 partition: 0 MaxSeqNo: 25 sessionId: src_id|12aa8254-69e30439-55897020-23366d02_0
2025-10-09T21:50:51.527337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd6898ea-71c0583e-5f0d1b4-b4094e3b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|12aa8254-69e30439-55897020-23366d02_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:51.529954Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12aa8254-69e30439-55897020-23366d02_0] Write session: try to update token
2025-10-09T21:50:51.529970Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12aa8254-69e30439-55897020-23366d02_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:51.540144Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:51.540162Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:51.540164Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:51.540166Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:51.540176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342806367299156:10665]
2025-10-09T21:50:51.540180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342806367299156:10665]
2025-10-09T21:50:51.540185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342806367299156:10665]
2025-10-09T21:50:51.540231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:51.540236Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:51.540239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.540242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:51.540258Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:51.540262Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:51.540371Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:51.543565Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1956 sessionId: src_id|12aa8254-69e30439-55897020-23366d02_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:51.545834Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:51.596794Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342806729132258:4041]
2025-10-09T21:50:51.597104Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 1955, pipe: [1:7559342806729132258:4041]
2025-10-09T21:50:51.684766Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:51.684989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:51.685003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:51.685048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342806729132258:4041], now have 1 active actors on pipe
2025-10-09T21:50:51.685229Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:51.685330Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342806729132258:4041]
2025-10-09T21:50:51.685353Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:51.685362Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:51.685440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1956 reason: status is not ok: tablet will be restarted right now sessionId: src_id|12aa8254-69e30439-55897020-23366d02_0
2025-10-09T21:50:51.685552Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1956 sessionId: src_id|12aa8254-69e30439-55897020-23366d02_0 is DEAD
2025-10-09T21:50:51.685671Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12aa8254-69e30439-55897020-23366d02_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:51.685683Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12aa8254-69e30439-55897020-23366d02_0] Write session will restart in 0.005881s
2025-10-09T21:50:51.685709Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12aa8254-69e30439-55897020-23366d02_0] Write session: Do CDS request
2025-10-09T21:50:51.685716Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12aa8254-69e30439-55897020-23366d02_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:51.692418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12aa8254-69e30439-55897020-23366d02_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:51.692448Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:51.692461Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1957
2025-10-09T21:50:51.692640Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1957 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:51.692684Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1957 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:51.692692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1957 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:51.692695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:51.693056Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1957 sessionId:  describe result for acl check
2025-10-09T21:50:51.693090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:51.693092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:51.693094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:51.693099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342806729132282:4698] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:51.693594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342806729132282:4698] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:51.695837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342806729132282:4698] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:51.699143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342806729132282:4698] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:51.699157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342806729132282:4698] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:51.699161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342806729132282:4698] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:51.699168Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1957 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:51.706436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:51.709113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:51.709317Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:51.709327Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:51.709341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:51.709403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:51.709456Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342806367299447:10667]
2025-10-09T21:50:51.710074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:51.710397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:51.710439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:51.710753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:51.710799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.710804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.710805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.710805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.710806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.710807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.710807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.710808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.710809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.710812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.710819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.710822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.710824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.710826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.710828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.710830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.710832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.710834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.710839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.710844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.710849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.710852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.710854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.710857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.710860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.710864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.710866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.710870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.710872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.710876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.710880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.710885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.710889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.710894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.710900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.710906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.710933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:51.717359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:51.718894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:51.718908Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:51.718910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:51.718914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:51.718920Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1956 [2:7559342806367299447:10667]
2025-10-09T21:50:51.718933Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:51.718952Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:51.718959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:51.718962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:51.718965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:51.718981Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:51.718992Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:51.718999Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:51.719055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.719063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.719065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.719068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:51.719135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:51.719143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.719156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.719173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.719192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:51.719219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342806729132264:2432], now have 1 active actors on pipe
2025-10-09T21:50:51.719223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342806729132395:4698], now have 1 active actors on pipe
2025-10-09T21:50:51.719230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342806367299442:10667]
2025-10-09T21:50:51.719232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342806367299442:10667]
2025-10-09T21:50:51.719234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342806367299442:10667]
2025-10-09T21:50:51.719250Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:51.719272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:51.719279Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:51.719283Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:51.719286Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:51.719291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:51.719295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:51.719288Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1956
2025-10-09T21:50:51.719329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342806367299442:10667]
2025-10-09T21:50:51.719337Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:51.719298Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1956
2025-10-09T21:50:51.719339Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:51.719360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:51.719370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:51.719418Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ffdcae14-feaef319-b62f8f78-319813_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:51.719429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:51.719433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:51.719435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:51.719445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:51.719454Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:51.719478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:51.719639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:51.719648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:51.719666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:51.719786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1957 partition: 0 MaxSeqNo: 25 sessionId: src_id|ffdcae14-feaef319-b62f8f78-319813_0
2025-10-09T21:50:51.720125Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12aa8254-69e30439-55897020-23366d02_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ffdcae14-feaef319-b62f8f78-319813_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:51.722656Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ffdcae14-feaef319-b62f8f78-319813_0] Write session: try to update token
2025-10-09T21:50:51.722676Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ffdcae14-feaef319-b62f8f78-319813_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:51.734824Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:51.734852Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:51.734855Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:51.734859Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:51.734873Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342806367299442:10667]
2025-10-09T21:50:51.734879Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342806367299442:10667]
2025-10-09T21:50:51.734885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342806367299442:10667]
2025-10-09T21:50:51.734943Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:51.734950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:51.734957Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:51.734960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.734963Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:51.734964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:51.735259Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:51.736346Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1957 sessionId: src_id|ffdcae14-feaef319-b62f8f78-319813_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:51.738741Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:51.785700Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342806729132567:4041]
2025-10-09T21:50:51.785899Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 1956, pipe: [1:7559342806729132567:4041]
2025-10-09T21:50:51.881264Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:51.881714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:51.881728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:51.881768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342806729132567:4041], now have 1 active actors on pipe
2025-10-09T21:50:51.881933Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:51.881994Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342806729132567:4041]
2025-10-09T21:50:51.882008Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:51.882055Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:51.882131Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1957 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ffdcae14-feaef319-b62f8f78-319813_0
2025-10-09T21:50:51.882309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1957 sessionId: src_id|ffdcae14-feaef319-b62f8f78-319813_0 is DEAD
2025-10-09T21:50:51.882459Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffdcae14-feaef319-b62f8f78-319813_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:51.882471Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffdcae14-feaef319-b62f8f78-319813_0] Write session will restart in 0.007098s
2025-10-09T21:50:51.882506Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffdcae14-feaef319-b62f8f78-319813_0] Write session: Do CDS request
2025-10-09T21:50:51.882511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffdcae14-feaef319-b62f8f78-319813_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:51.890448Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ffdcae14-feaef319-b62f8f78-319813_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:51.890547Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:51.890564Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1958
2025-10-09T21:50:51.890692Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1958 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:51.890722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1958 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:51.890730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1958 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:51.890734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:51.891173Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1958 sessionId:  describe result for acl check
2025-10-09T21:50:51.891209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:51.891216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:51.891218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:51.891224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342806729132614:4720] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:51.891792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342806729132614:4720] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:51.894588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342806729132614:4720] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:51.897628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342806729132614:4720] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:51.897644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342806729132614:4720] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:51.897647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342806729132614:4720] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:51.897657Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1958 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:51.900841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:51.903240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:51.903410Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:51.903420Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:51.903431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:51.903475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:51.903508Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342806367299737:10671]
2025-10-09T21:50:51.903920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:51.904170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:51.904210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:51.904502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:51.904548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.904554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.904555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.904556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.904558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.904559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.904560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.904561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.904563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.904567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.904578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.904582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.904585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.904588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.904593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.904597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.904600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.904604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.904610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.904618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:51.904624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.904629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:51.904639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.904644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:51.904652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.904657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:51.904662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.904671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:51.904675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.904681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:51.904686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.904691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.904695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.904700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.904706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.904710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.904733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:51.910724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:51.912096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:51.912106Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:51.912108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:51.912113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:51.912120Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1957 [2:7559342806367299737:10671]
2025-10-09T21:50:51.912132Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:51.912148Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:51.912151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:51.912154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:51.912156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:51.912173Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:51.912186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:51.912195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:51.912250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.912260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.912262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.912264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:51.912332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:51.912343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.912355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:51.912368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:51.912385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:51.912410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342806729132809:4720], now have 1 active actors on pipe
2025-10-09T21:50:51.912416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342806729132572:2432], now have 1 active actors on pipe
2025-10-09T21:50:51.912424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342806367299732:10671]
2025-10-09T21:50:51.912427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342806367299732:10671]
2025-10-09T21:50:51.912430Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342806367299732:10671]
2025-10-09T21:50:51.912442Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:51.912462Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:51.912443Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1957
2025-10-09T21:50:51.912475Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:51.912477Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:51.912444Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1957
2025-10-09T21:50:51.912481Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:51.912483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:51.912483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:51.912524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:51.912533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:51.912546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342806367299732:10671]
2025-10-09T21:50:51.912557Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:51.912561Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:51.912584Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|95439dac-9a08d344-c5298595-e96e0276_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:51.912596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:51.912600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:51.912607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:51.912617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:51.912627Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:51.912646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:51.912798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:51.912806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:51.912819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:51.912909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1958 partition: 0 MaxSeqNo: 25 sessionId: src_id|95439dac-9a08d344-c5298595-e96e0276_0
2025-10-09T21:50:51.913240Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ffdcae14-feaef319-b62f8f78-319813_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|95439dac-9a08d344-c5298595-e96e0276_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:51.915654Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|95439dac-9a08d344-c5298595-e96e0276_0] Write session: try to update token
2025-10-09T21:50:51.915675Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|95439dac-9a08d344-c5298595-e96e0276_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:51.926621Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:51.926638Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:51.926640Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:51.926642Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:51.926650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342806367299732:10671]
2025-10-09T21:50:51.926655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342806367299732:10671]
2025-10-09T21:50:51.926659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342806367299732:10671]
2025-10-09T21:50:51.926703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:51.926710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:51.926713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:51.926715Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:51.926721Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:51.926725Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:51.926851Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:51.927764Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1958 sessionId: src_id|95439dac-9a08d344-c5298595-e96e0276_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:51.930004Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:51.982317Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342806729132867:4041]
2025-10-09T21:50:51.982554Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 1957, pipe: [1:7559342806729132867:4041]
2025-10-09T21:50:52.073698Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:52.073990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.074007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.074064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342806729132867:4041], now have 1 active actors on pipe
2025-10-09T21:50:52.074275Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:52.074280Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:52.074295Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:52.074305Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342806729132867:4041]
2025-10-09T21:50:52.074327Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:52.074350Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1958 reason: error at write: Disconnected sessionId: src_id|95439dac-9a08d344-c5298595-e96e0276_0
2025-10-09T21:50:52.074508Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1958 sessionId: src_id|95439dac-9a08d344-c5298595-e96e0276_0 is DEAD
2025-10-09T21:50:52.074654Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95439dac-9a08d344-c5298595-e96e0276_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:52.074663Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95439dac-9a08d344-c5298595-e96e0276_0] Write session will restart in 0.005850s
2025-10-09T21:50:52.074690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95439dac-9a08d344-c5298595-e96e0276_0] Write session: Do CDS request
2025-10-09T21:50:52.074696Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95439dac-9a08d344-c5298595-e96e0276_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:52.081545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|95439dac-9a08d344-c5298595-e96e0276_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:52.081629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:52.081645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1959
2025-10-09T21:50:52.081766Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1959 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:52.081801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1959 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:52.081814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1959 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:52.081818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:52.082255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1959 sessionId:  describe result for acl check
2025-10-09T21:50:52.082296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:52.082298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:52.082300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:52.082306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342811024100183:4738] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:52.082853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342811024100183:4738] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:52.085986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342811024100183:4738] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:52.089534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342811024100183:4738] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:52.089548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342811024100183:4738] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:52.089552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342811024100183:4738] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:52.089562Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1959 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:52.094406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:52.096813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:52.096972Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:52.096983Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:52.096995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:52.097043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:52.097079Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342810662267320:10673]
2025-10-09T21:50:52.097619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:52.097920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:52.097951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:52.098242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:52.098294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.098299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.098300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.098302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.098303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.098304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.098306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.098307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.098308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.098313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.098324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.098329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.098333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.098337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.098341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.098344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.098347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.098351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.098358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.098366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.098372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.098378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.098383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.098389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.098393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.098398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.098407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.098412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.098416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.098426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.098432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.098437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.098442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.098445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.098448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.098451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.098468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:52.104843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:52.105952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:52.105967Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:52.105969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:52.105973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:52.105978Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1958 [2:7559342810662267320:10673]
2025-10-09T21:50:52.105988Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:52.106003Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:52.106005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:52.106007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:52.106008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:52.106021Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:52.106025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:52.106053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:52.106426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.106440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.106442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.106444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:52.106499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:52.106505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.106513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.106519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.106530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:52.106551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342810662267315:10673]
2025-10-09T21:50:52.106553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342810662267315:10673]
2025-10-09T21:50:52.106554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342810662267315:10673]
2025-10-09T21:50:52.106560Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:52.106572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:52.106574Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:52.106575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342811024100166:2432], now have 1 active actors on pipe
2025-10-09T21:50:52.106577Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:52.106580Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:52.106587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:52.106589Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:52.106595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342811024100344:4738], now have 1 active actors on pipe
2025-10-09T21:50:52.106617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342810662267315:10673]
2025-10-09T21:50:52.106626Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:52.106634Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:52.106646Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1958
2025-10-09T21:50:52.106681Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1958
2025-10-09T21:50:52.106867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.106878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.106902Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4e310bc4-a11cd5a5-e64e695-751dd61f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:52.106912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:52.106916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:52.106918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:52.106927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:52.106942Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:52.106958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:52.107152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.107158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.107172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:52.107295Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1959 partition: 0 MaxSeqNo: 25 sessionId: src_id|4e310bc4-a11cd5a5-e64e695-751dd61f_0
2025-10-09T21:50:52.107684Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|95439dac-9a08d344-c5298595-e96e0276_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4e310bc4-a11cd5a5-e64e695-751dd61f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:52.110161Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e310bc4-a11cd5a5-e64e695-751dd61f_0] Write session: try to update token
2025-10-09T21:50:52.110177Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e310bc4-a11cd5a5-e64e695-751dd61f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:52.118444Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:52.118459Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:52.118469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342810662267315:10673]
2025-10-09T21:50:52.118517Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:52.118519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:52.118700Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:52.118845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:52.118989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:52.119131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:52.119323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:52.119362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:52.122829Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1959 sessionId: src_id|4e310bc4-a11cd5a5-e64e695-751dd61f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:52.125322Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:52.125702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.125722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.125761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:52.125851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:52.131799Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:52.131816Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:52.131819Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:52.131821Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:52.131832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342810662267315:10673]
2025-10-09T21:50:52.131836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342810662267315:10673]
2025-10-09T21:50:52.131840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342810662267315:10673]
2025-10-09T21:50:52.131882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:52.131886Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:52.131889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.131891Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:52.131892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:52.131894Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:52.132064Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:52.174329Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342811024100463:4041]
2025-10-09T21:50:52.174661Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 1958, pipe: [1:7559342811024100463:4041]
2025-10-09T21:50:52.279720Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:52.286257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.286277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.286415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.286468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.286526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.286584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.286625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.286632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.286693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.286749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.286803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.286862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.286902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.286908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.286961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.287014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.287068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.287147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.287187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.287195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.287254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.287312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.287369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.287427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.287466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.287471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.287530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.287584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.287640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.287701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.287743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.287749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.287804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.287856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.287910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.287963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.288005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.288011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.288068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.288131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.288197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.288254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.288294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.288301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.288354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.288402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.288455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.288521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.288563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.288569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.288621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.288670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.288721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.288773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.288810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.288816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.288883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.288938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.288989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.289042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.289093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.289099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.289140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342811024100463:4041], now have 1 active actors on pipe
2025-10-09T21:50:52.289335Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342811024100463:4041]
2025-10-09T21:50:52.289365Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:52.289853Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:52.289866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:52.289894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1959 reason: error at write: Disconnected sessionId: src_id|4e310bc4-a11cd5a5-e64e695-751dd61f_0
2025-10-09T21:50:52.289973Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:52.290089Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1959 sessionId: src_id|4e310bc4-a11cd5a5-e64e695-751dd61f_0 is DEAD
2025-10-09T21:50:52.290215Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e310bc4-a11cd5a5-e64e695-751dd61f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:52.290228Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e310bc4-a11cd5a5-e64e695-751dd61f_0] Write session will restart in 0.009637s
2025-10-09T21:50:52.290252Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e310bc4-a11cd5a5-e64e695-751dd61f_0] Write session: Do CDS request
2025-10-09T21:50:52.290258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e310bc4-a11cd5a5-e64e695-751dd61f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:52.301539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e310bc4-a11cd5a5-e64e695-751dd61f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:52.301520Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:52.301539Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1960
2025-10-09T21:50:52.301724Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1960 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:52.301748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1960 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:52.301756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1960 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:52.301759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:52.302162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1960 sessionId:  describe result for acl check
2025-10-09T21:50:52.302203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:52.302205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:52.302206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:52.302212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342811024100595:4744] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:52.302764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342811024100595:4744] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:52.305509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342811024100595:4744] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:52.308004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342811024100595:4744] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:52.308018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342811024100595:4744] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:52.308022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342811024100595:4744] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:52.308031Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1960 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:52.308817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:52.311509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:52.311700Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:52.311710Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:52.311722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:52.311772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:52.311812Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342810662267647:10687]
2025-10-09T21:50:52.312403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:52.312674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:52.312714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:52.313022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:52.313070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.313075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.313076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.313078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.313079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.313080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.313081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.313083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.313085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.313089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.313100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.313104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.313108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.313112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.313116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.313120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.313124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.313127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.313134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.313147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.313158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.313167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.313176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.313182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.313186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.313191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.313197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.313202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.313206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.313211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.313217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.313226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.313230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.313235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.313244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.313249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.313277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:52.319695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:52.321087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:52.321101Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:52.321103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:52.321107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:52.321116Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1959 [2:7559342810662267647:10687]
2025-10-09T21:50:52.321129Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:52.321146Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:52.321154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:52.321156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:52.321159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:52.321177Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:52.321187Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:52.321202Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:52.321264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.321271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.321273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.321277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:52.321344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:52.321351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.321362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.321378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.321393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:52.321431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342811024100497:2432], now have 1 active actors on pipe
2025-10-09T21:50:52.321437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342811024100778:4744], now have 1 active actors on pipe
2025-10-09T21:50:52.321445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342810662267642:10687]
2025-10-09T21:50:52.321449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342810662267642:10687]
2025-10-09T21:50:52.321451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342810662267642:10687]
2025-10-09T21:50:52.321463Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:52.321478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:52.321486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:52.321488Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:52.321491Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:52.321497Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:52.321500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:52.321480Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1959
2025-10-09T21:50:52.321530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342810662267642:10687]
2025-10-09T21:50:52.321537Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:52.321539Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:52.321732Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1959
2025-10-09T21:50:52.321902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.321917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.321950Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5eeb9a98-c25e6817-958eeb59-8922128e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:52.321973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:52.321982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:52.321985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:52.321996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:52.322008Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:52.322044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:52.322194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.322201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.322215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:52.322291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1960 partition: 0 MaxSeqNo: 25 sessionId: src_id|5eeb9a98-c25e6817-958eeb59-8922128e_0
2025-10-09T21:50:52.322625Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e310bc4-a11cd5a5-e64e695-751dd61f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5eeb9a98-c25e6817-958eeb59-8922128e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:52.325005Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5eeb9a98-c25e6817-958eeb59-8922128e_0] Write session: try to update token
2025-10-09T21:50:52.325018Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5eeb9a98-c25e6817-958eeb59-8922128e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:52.337396Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1960 sessionId: src_id|5eeb9a98-c25e6817-958eeb59-8922128e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:52.339481Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:52.339497Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:52.339506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342810662267642:10687]
2025-10-09T21:50:52.339554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:52.339559Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:52.339692Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:52.339739Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:52.339879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:52.340015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:52.340155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:52.340345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:52.340395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.340408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.340430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:52.340462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:52.340570Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:52.346145Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:52.346159Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:52.346161Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:52.346163Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:52.346171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342810662267642:10687]
2025-10-09T21:50:52.346175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342810662267642:10687]
2025-10-09T21:50:52.346179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342810662267642:10687]
2025-10-09T21:50:52.346213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:52.346220Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:52.346226Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:52.346227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.346229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:52.346230Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:52.346354Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:52.389342Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342811024100796:4041]
2025-10-09T21:50:52.389598Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 1959, pipe: [1:7559342811024100796:4041]
2025-10-09T21:50:52.490660Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:52.498262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.498287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.498449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.498516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.498582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.498649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.498700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.498713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.498785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.498854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.498918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.498984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.499025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.499035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.499094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.499150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.499216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.499283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.499327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.499334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.499399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.499466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.499529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.499590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.499635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.499638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.499705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.499765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.499827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.499892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.499934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.499941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.500004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.500066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.500126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.500186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.500231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.500238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.500295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.500350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.500408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.500471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.500583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.500590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.500647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.500702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.500764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.500823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.500926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.500933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.500993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.501048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.501110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.501169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.501209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.501219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.501282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:52.501337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:52.501395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:52.501465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:52.501510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:52.501516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:52.501570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342811024100796:4041], now have 1 active actors on pipe
2025-10-09T21:50:52.501837Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342811024100796:4041]
2025-10-09T21:50:52.501842Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:52.501845Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:52.501876Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:52.501877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1960 reason: error at write: Disconnected sessionId: src_id|5eeb9a98-c25e6817-958eeb59-8922128e_0
2025-10-09T21:50:52.502066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1960 sessionId: src_id|5eeb9a98-c25e6817-958eeb59-8922128e_0 is DEAD
2025-10-09T21:50:52.502170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5eeb9a98-c25e6817-958eeb59-8922128e_0] Counters: { Errors: 0 CurrentSessionLifetimeMs: 179 BytesWritten: 56320040 MessagesWritten: 25 BytesWrittenCompressed: 0 BytesInflightUncompressed: 22528020 BytesInflightCompressed: 0 BytesInflightTotal: 18022416 MessagesInflight: 10 }
2025-10-09T21:50:52.502194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5eeb9a98-c25e6817-958eeb59-8922128e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:52.502197Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5eeb9a98-c25e6817-958eeb59-8922128e_0] Write session will restart in 0.009422s
2025-10-09T21:50:52.502223Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5eeb9a98-c25e6817-958eeb59-8922128e_0] Write session: Do CDS request
2025-10-09T21:50:52.502229Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5eeb9a98-c25e6817-958eeb59-8922128e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:52.502275Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:52.512442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5eeb9a98-c25e6817-958eeb59-8922128e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:52.512495Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:52.512511Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1961
2025-10-09T21:50:52.512635Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1961 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:52.512667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1961 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:52.512679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1961 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:52.512683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:52.513094Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1961 sessionId:  describe result for acl check
2025-10-09T21:50:52.513131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:52.513139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:52.513140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:52.513146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342811024100863:4764] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:52.513690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342811024100863:4764] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:52.516954Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342811024100863:4764] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:52.520152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342811024100863:4764] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:52.520163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342811024100863:4764] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:52.520166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342811024100863:4764] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:52.520172Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1961 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:52.522500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:52.525297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:52.525499Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:52.525508Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:52.525521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:52.525572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:52.525614Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342810662267936:10689]
2025-10-09T21:50:52.526233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:52.526549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:52.526594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:52.526912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:52.526961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.526967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.526969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.526970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.526972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.526973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.526974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.526975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.526976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.526981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.526992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.526996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.527001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.527004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.527009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.527013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.527018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.527021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.527028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.527035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.527041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.527047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.527051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.527055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.527059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.527063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.527067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.527072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.527076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.527081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.527086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.527091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.527095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.527100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.527105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.527110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.527144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:52.533560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:52.534989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:52.535004Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:52.535007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:52.535011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:52.535017Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1960 [2:7559342810662267936:10689]
2025-10-09T21:50:52.535029Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:52.535047Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:52.535050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:52.535053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:52.535056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:52.535073Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:52.535082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:52.535089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:52.535151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.535159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.535161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.535164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:52.535233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:52.535241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.535253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.535265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.535287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:52.535313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342811024101064:4764], now have 1 active actors on pipe
2025-10-09T21:50:52.535323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342811024100798:2432], now have 1 active actors on pipe
2025-10-09T21:50:52.535331Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342810662267931:10689]
2025-10-09T21:50:52.535333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342810662267931:10689]
2025-10-09T21:50:52.535335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342810662267931:10689]
2025-10-09T21:50:52.535348Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:52.535362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:52.535373Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:52.535376Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:52.535377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:52.535378Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:52.535381Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:52.535375Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1960
2025-10-09T21:50:52.535383Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1960
2025-10-09T21:50:52.535410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342810662267931:10689]
2025-10-09T21:50:52.535419Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:52.535421Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:52.535435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.535441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.535492Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7a6ae777-c414084c-70a3fce2-55abc055_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:52.535505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:52.535509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:52.535511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:52.535521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:52.535532Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:52.535551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:52.535717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.535725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.535740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:52.535850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1961 partition: 0 MaxSeqNo: 25 sessionId: src_id|7a6ae777-c414084c-70a3fce2-55abc055_0
2025-10-09T21:50:52.536194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5eeb9a98-c25e6817-958eeb59-8922128e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7a6ae777-c414084c-70a3fce2-55abc055_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:52.538557Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a6ae777-c414084c-70a3fce2-55abc055_0] Write session: try to update token
2025-10-09T21:50:52.538571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a6ae777-c414084c-70a3fce2-55abc055_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:52.549675Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:52.549689Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:52.549692Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:52.549694Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:52.549704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342810662267931:10689]
2025-10-09T21:50:52.549709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342810662267931:10689]
2025-10-09T21:50:52.549714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342810662267931:10689]
2025-10-09T21:50:52.549757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:52.549764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.549764Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:52.549767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:52.549779Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:52.549784Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:52.549895Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:52.551208Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1961 sessionId: src_id|7a6ae777-c414084c-70a3fce2-55abc055_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:52.553586Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:52.601848Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342811024101098:4041]
2025-10-09T21:50:52.602144Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 1960, pipe: [1:7559342811024101098:4041]
2025-10-09T21:50:52.699196Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:52.699676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.699689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.699735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342811024101098:4041], now have 1 active actors on pipe
2025-10-09T21:50:52.699912Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:52.699964Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:52.699980Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:52.700022Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342811024101098:4041]
2025-10-09T21:50:52.700048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1961 reason: error at write: Disconnected sessionId: src_id|7a6ae777-c414084c-70a3fce2-55abc055_0
2025-10-09T21:50:52.700056Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:52.700228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1961 sessionId: src_id|7a6ae777-c414084c-70a3fce2-55abc055_0 is DEAD
2025-10-09T21:50:52.700402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a6ae777-c414084c-70a3fce2-55abc055_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:52.700416Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a6ae777-c414084c-70a3fce2-55abc055_0] Write session will restart in 0.007028s
2025-10-09T21:50:52.700445Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a6ae777-c414084c-70a3fce2-55abc055_0] Write session: Do CDS request
2025-10-09T21:50:52.700453Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a6ae777-c414084c-70a3fce2-55abc055_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:52.708533Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a6ae777-c414084c-70a3fce2-55abc055_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:52.708645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:52.708663Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1962
2025-10-09T21:50:52.708819Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1962 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:52.708846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1962 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:52.708858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1962 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:52.708864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:52.709209Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1962 sessionId:  describe result for acl check
2025-10-09T21:50:52.709238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:52.709239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:52.709240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:52.709244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342811024101128:4776] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:52.709814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342811024101128:4776] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:52.712808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342811024101128:4776] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:52.715843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342811024101128:4776] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:52.715861Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342811024101128:4776] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:52.715864Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342811024101128:4776] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:52.715873Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1962 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:52.721164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:52.723916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:52.724112Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:52.724122Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:52.724134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:52.724200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:52.724241Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342810662268226:10691]
2025-10-09T21:50:52.724718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:52.725026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:52.725065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:52.725367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:52.725433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.725439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.725440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.725442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.725443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.725444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.725445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.725446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.725448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.725453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.725464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.725468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.725472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.725476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.725480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.725484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.725488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.725491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.725498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.725506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.725512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.725518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.725523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.725528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.725532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.725537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.725541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.725546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.725551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.725556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.725561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.725565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.725570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.725576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.725581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.725586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.725612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:52.731939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:52.732928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:52.732943Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:52.732945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:52.732949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:52.732959Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1961 [2:7559342810662268226:10691]
2025-10-09T21:50:52.732969Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:52.732985Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:52.732987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:52.732989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:52.732991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:52.733004Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:52.733008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:52.733015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:52.733060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.733066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.733067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.733069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:52.733133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:52.733141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.733152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.733163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.733184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:52.733208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342811024101282:4776], now have 1 active actors on pipe
2025-10-09T21:50:52.733216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342811024101101:2432], now have 1 active actors on pipe
2025-10-09T21:50:52.733224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342810662268221:10691]
2025-10-09T21:50:52.733227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342810662268221:10691]
2025-10-09T21:50:52.733229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342810662268221:10691]
2025-10-09T21:50:52.733241Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:52.733257Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:52.733259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:52.733265Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:52.733267Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:52.733273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:52.733276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:52.733292Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1961
2025-10-09T21:50:52.733294Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1961
2025-10-09T21:50:52.733305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342810662268221:10691]
2025-10-09T21:50:52.733314Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:52.733316Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:52.733338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.733343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.733380Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9d8cea12-ecc7f232-d0de0843-de66afba_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:52.733392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:52.733400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:52.733403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:52.733412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:52.733437Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:52.733461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:52.733787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.733798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.733817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:52.733928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1962 partition: 0 MaxSeqNo: 25 sessionId: src_id|9d8cea12-ecc7f232-d0de0843-de66afba_0
2025-10-09T21:50:52.734328Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a6ae777-c414084c-70a3fce2-55abc055_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9d8cea12-ecc7f232-d0de0843-de66afba_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:52.736750Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d8cea12-ecc7f232-d0de0843-de66afba_0] Write session: try to update token
2025-10-09T21:50:52.736768Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d8cea12-ecc7f232-d0de0843-de66afba_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:52.747473Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:52.747487Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:52.747489Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:52.747491Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:52.747498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342810662268221:10691]
2025-10-09T21:50:52.747501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342810662268221:10691]
2025-10-09T21:50:52.747505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342810662268221:10691]
2025-10-09T21:50:52.747547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:52.747555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.747561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:52.747585Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:52.747598Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:52.747603Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:52.747689Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:52.749755Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1962 sessionId: src_id|9d8cea12-ecc7f232-d0de0843-de66afba_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:52.751931Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:52.799960Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342811024101409:4041]
2025-10-09T21:50:52.800242Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 1961, pipe: [1:7559342811024101409:4041]
2025-10-09T21:50:52.890894Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:52.891586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.891599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.891634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342811024101409:4041], now have 1 active actors on pipe
2025-10-09T21:50:52.891829Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:52.891901Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342811024101409:4041]
2025-10-09T21:50:52.891938Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:52.891940Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:52.892023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1962 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9d8cea12-ecc7f232-d0de0843-de66afba_0
2025-10-09T21:50:52.892205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1962 sessionId: src_id|9d8cea12-ecc7f232-d0de0843-de66afba_0 is DEAD
2025-10-09T21:50:52.892355Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d8cea12-ecc7f232-d0de0843-de66afba_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:52.892369Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d8cea12-ecc7f232-d0de0843-de66afba_0] Write session will restart in 0.006602s
2025-10-09T21:50:52.892398Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d8cea12-ecc7f232-d0de0843-de66afba_0] Write session: Do CDS request
2025-10-09T21:50:52.892404Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d8cea12-ecc7f232-d0de0843-de66afba_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:52.900481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d8cea12-ecc7f232-d0de0843-de66afba_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:52.900582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:52.900599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1963
2025-10-09T21:50:52.900740Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1963 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:52.900782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1963 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:52.900793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1963 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:52.900798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:52.901233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1963 sessionId:  describe result for acl check
2025-10-09T21:50:52.901272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:52.901278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:52.901280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:52.901286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342811024101450:4785] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:52.901860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342811024101450:4785] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:52.905397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342811024101450:4785] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:52.908299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342811024101450:4785] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:52.908316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342811024101450:4785] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:52.908319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342811024101450:4785] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:52.908333Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1963 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:52.911376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:52.914020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:52.914193Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:52.914203Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:52.914214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:52.914260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:52.914287Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342810662268521:10695]
2025-10-09T21:50:52.914703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:52.914998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:52.915029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:52.915266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:52.915303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.915309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.915309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.915310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.915310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.915311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.915312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.915312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.915313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.915316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.915323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.915325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.915327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.915329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.915331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.915333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.915335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.915337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.915342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.915350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:52.915356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.915361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:52.915365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.915369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:52.915373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.915377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:52.915381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.915385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:52.915389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.915394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:52.915400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.915404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.915408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.915413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.915417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.915421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.915447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:52.921525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:52.922576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:52.922591Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:52.922592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:52.922595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:52.922599Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1962 [2:7559342810662268521:10695]
2025-10-09T21:50:52.922610Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:52.922623Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:52.922628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:52.922630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:52.922631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:52.922642Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:52.922647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:52.922652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:52.922695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.922699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.922700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.922702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:52.922755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:52.922761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.922768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:52.922774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:52.922783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:52.922797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342811024101636:4785], now have 1 active actors on pipe
2025-10-09T21:50:52.922800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342811024101414:2432], now have 1 active actors on pipe
2025-10-09T21:50:52.922805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342810662268516:10695]
2025-10-09T21:50:52.922807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342810662268516:10695]
2025-10-09T21:50:52.922808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342810662268516:10695]
2025-10-09T21:50:52.922817Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:52.922827Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:52.922834Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:52.922837Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:52.922838Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:52.922839Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:52.922840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:52.922857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342810662268516:10695]
2025-10-09T21:50:52.922859Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:52.922861Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:52.922885Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1962
2025-10-09T21:50:52.922895Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1962
2025-10-09T21:50:52.922926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.922935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.922964Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|27216423-2bf3003a-4bea7f73-47e451f6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:52.922978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:52.922986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:52.922989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:52.922998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:52.923012Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:52.923031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:52.923188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.923196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.923210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:52.923301Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1963 partition: 0 MaxSeqNo: 25 sessionId: src_id|27216423-2bf3003a-4bea7f73-47e451f6_0
2025-10-09T21:50:52.923612Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d8cea12-ecc7f232-d0de0843-de66afba_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|27216423-2bf3003a-4bea7f73-47e451f6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:52.926012Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|27216423-2bf3003a-4bea7f73-47e451f6_0] Write session: try to update token
2025-10-09T21:50:52.926046Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|27216423-2bf3003a-4bea7f73-47e451f6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:52.934673Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:52.934687Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:52.934696Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342810662268516:10695]
2025-10-09T21:50:52.934742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:52.934743Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:52.934905Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:52.935036Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:52.935164Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:52.935286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:52.935451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:52.935495Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:52.938110Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1963 sessionId: src_id|27216423-2bf3003a-4bea7f73-47e451f6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:52.940245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:52.940440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:52.940457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:52.940498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:52.940598Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:52.946966Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:52.946981Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:52.946983Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:52.946985Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:52.946996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342810662268516:10695]
2025-10-09T21:50:52.947000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342810662268516:10695]
2025-10-09T21:50:52.947006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342810662268516:10695]
2025-10-09T21:50:52.947042Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:52.947052Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:52.947054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:52.947055Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:52.947061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:52.947064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:52.947183Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:52.991980Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342811024101714:4041]
2025-10-09T21:50:52.992269Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 1962, pipe: [1:7559342811024101714:4041]
2025-10-09T21:50:53.091880Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:53.099447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.099470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.099627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.099701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.099762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.099822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.099866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.099876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.099942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.100001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.100059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.100122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.100162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.100170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.100231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.100285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.100344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.100401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.100445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.100453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.100514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.100569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.100631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.100690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.100732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.100737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.100800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.100864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.100928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.100985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.101044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.101052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.101111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.101172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.101235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.101296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.101338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.101341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.101403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.101474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.101534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.101591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.101635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.101642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.101701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.101760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.101817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.101879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.101920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.101930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.101994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.102062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.102120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.102182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.102223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.102230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.102288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.102346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.102410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.102471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.102516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.102524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.102583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342811024101714:4041], now have 1 active actors on pipe
2025-10-09T21:50:53.102740Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:53.102742Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342811024101714:4041]
2025-10-09T21:50:53.102758Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:53.102762Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:53.102791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1963 reason: error at write: Disconnected sessionId: src_id|27216423-2bf3003a-4bea7f73-47e451f6_0
2025-10-09T21:50:53.102981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1963 sessionId: src_id|27216423-2bf3003a-4bea7f73-47e451f6_0 is DEAD
2025-10-09T21:50:53.103118Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27216423-2bf3003a-4bea7f73-47e451f6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:53.103128Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27216423-2bf3003a-4bea7f73-47e451f6_0] Write session will restart in 0.005369s
2025-10-09T21:50:53.103153Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27216423-2bf3003a-4bea7f73-47e451f6_0] Write session: Do CDS request
2025-10-09T21:50:53.103159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27216423-2bf3003a-4bea7f73-47e451f6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:53.103403Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:53.109539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|27216423-2bf3003a-4bea7f73-47e451f6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:53.109647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:53.109663Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1964
2025-10-09T21:50:53.109925Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1964 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:53.109969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1964 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:53.109990Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1964 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:53.109994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:53.110429Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1964 sessionId:  describe result for acl check
2025-10-09T21:50:53.110460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:53.110466Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:53.110468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:53.110473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342815319069028:4795] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:53.111039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342815319069028:4795] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:53.114129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342815319069028:4795] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:53.117492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342815319069028:4795] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:53.117508Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342815319069028:4795] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:53.117512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342815319069028:4795] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:53.117519Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1964 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:53.123131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:53.126063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:53.126279Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:53.126289Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:53.126298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:53.126341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:53.126372Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342814957236110:10697]
2025-10-09T21:50:53.126772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:53.127059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:53.127114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:53.127465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:53.127512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.127518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.127520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.127521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.127522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.127524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.127525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.127526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.127527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.127532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.127543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.127547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.127551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.127555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.127559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.127563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.127567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.127571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.127578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.127587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.127598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.127603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.127608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.127617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.127621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.127626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.127631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.127636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.127640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.127645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.127654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.127660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.127668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.127673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.127678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.127687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.127716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:53.133880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:53.135135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:53.135152Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:53.135154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:53.135159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:53.135165Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1963 [2:7559342814957236110:10697]
2025-10-09T21:50:53.135177Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:53.135191Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:53.135193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:53.135195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:53.135196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:53.135209Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:53.135218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:53.135224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:53.135265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.135270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.135271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.135274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:53.135324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:53.135329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.135336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.135342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.135352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:53.135370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342815319069013:2432], now have 1 active actors on pipe
2025-10-09T21:50:53.135376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342815319069160:4795], now have 1 active actors on pipe
2025-10-09T21:50:53.135383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342814957236105:10697]
2025-10-09T21:50:53.135385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342814957236105:10697]
2025-10-09T21:50:53.135387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342814957236105:10697]
2025-10-09T21:50:53.135400Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:53.135418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:53.135430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:53.135430Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:53.135434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:53.135436Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:53.135436Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:53.135421Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1963
2025-10-09T21:50:53.135424Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1963
2025-10-09T21:50:53.135458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342814957236105:10697]
2025-10-09T21:50:53.135464Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:53.135466Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:53.135481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.135487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.135522Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d153089b-82695e07-c97ea359-649f9c43_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:53.135532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:53.135535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:53.135536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:53.135545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:53.135555Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:53.135569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:53.135725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.135733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.135744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:53.135821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1964 partition: 0 MaxSeqNo: 25 sessionId: src_id|d153089b-82695e07-c97ea359-649f9c43_0
2025-10-09T21:50:53.136183Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|27216423-2bf3003a-4bea7f73-47e451f6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d153089b-82695e07-c97ea359-649f9c43_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:53.138571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d153089b-82695e07-c97ea359-649f9c43_0] Write session: try to update token
2025-10-09T21:50:53.138590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d153089b-82695e07-c97ea359-649f9c43_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:53.149415Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:53.149445Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:53.149448Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:53.149450Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:53.149460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342814957236105:10697]
2025-10-09T21:50:53.149465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342814957236105:10697]
2025-10-09T21:50:53.149468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342814957236105:10697]
2025-10-09T21:50:53.149508Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:53.149514Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:53.149515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:53.149522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.149524Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:53.149525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:53.149656Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:53.151495Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1964 sessionId: src_id|d153089b-82695e07-c97ea359-649f9c43_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:53.153779Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:53.202948Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342815319069317:4041]
2025-10-09T21:50:53.203227Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 1963, pipe: [1:7559342815319069317:4041]
2025-10-09T21:50:53.265199Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] Counters: { Errors: 19 CurrentSessionLifetimeMs: 481716 BytesRead: 24780812 MessagesRead: 11 BytesReadCompressed: 24780812 BytesInflightUncompressed: 2252802 BytesInflightCompressed: 0 BytesInflightTotal: 2252802 MessagesInflight: 1 }
2025-10-09T21:50:53.302367Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:53.302788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.302801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.302838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342815319069317:4041], now have 1 active actors on pipe
2025-10-09T21:50:53.303015Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342815319069317:4041]
2025-10-09T21:50:53.303015Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:53.303054Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:53.303057Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:53.303139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1964 reason: error at write: Disconnected sessionId: src_id|d153089b-82695e07-c97ea359-649f9c43_0
2025-10-09T21:50:53.303319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1964 sessionId: src_id|d153089b-82695e07-c97ea359-649f9c43_0 is DEAD
2025-10-09T21:50:53.303457Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d153089b-82695e07-c97ea359-649f9c43_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:53.303471Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d153089b-82695e07-c97ea359-649f9c43_0] Write session will restart in 0.007447s
2025-10-09T21:50:53.303495Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d153089b-82695e07-c97ea359-649f9c43_0] Write session: Do CDS request
2025-10-09T21:50:53.303501Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d153089b-82695e07-c97ea359-649f9c43_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:53.303636Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:53.312429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d153089b-82695e07-c97ea359-649f9c43_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:53.312510Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:53.312527Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1965
2025-10-09T21:50:53.312664Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1965 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:53.312713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1965 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:53.312724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1965 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:53.312728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:53.313184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1965 sessionId:  describe result for acl check
2025-10-09T21:50:53.313221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:53.313228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:53.313230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:53.313235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342815319069422:4810] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:53.313787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342815319069422:4810] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:53.316834Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342815319069422:4810] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:53.320073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342815319069422:4810] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:53.320088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342815319069422:4810] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:53.320092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342815319069422:4810] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:53.320107Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1965 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:53.321681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:53.324048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:53.324197Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:53.324206Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:53.324215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:53.324253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:53.324281Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342814957236441:10711]
2025-10-09T21:50:53.324695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:53.324925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:53.324953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:53.325149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:53.325179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.325185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.325186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.325186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.325187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.325187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.325188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.325189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.325190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.325193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.325200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.325202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.325204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.325206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.325208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.325210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.325212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.325214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.325219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.325224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.325228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.325231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.325234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.325237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.325239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.325242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.325245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.325248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.325251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.325254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.325257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.325260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.325262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.325265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.325269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.325271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.325287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:53.331276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:53.332217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:53.332227Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:53.332229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:53.332231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:53.332236Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1964 [2:7559342814957236441:10711]
2025-10-09T21:50:53.332245Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:53.332258Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:53.332259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:53.332262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:53.332263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:53.332275Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:53.332282Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:53.332287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:53.332326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.332331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.332332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.332334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:53.332387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:53.332394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.332404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.332419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.332434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:53.332449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342815319069634:4810], now have 1 active actors on pipe
2025-10-09T21:50:53.332451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342815319069349:2432], now have 1 active actors on pipe
2025-10-09T21:50:53.332456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342814957236436:10711]
2025-10-09T21:50:53.332458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342814957236436:10711]
2025-10-09T21:50:53.332459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342814957236436:10711]
2025-10-09T21:50:53.332467Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:53.332479Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:53.332483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:53.332486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:53.332487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:53.332491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:53.332493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:53.332494Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1964
2025-10-09T21:50:53.332511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342814957236436:10711]
2025-10-09T21:50:53.332524Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1964
2025-10-09T21:50:53.332517Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:53.332520Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:53.332543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.332549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.332570Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7c96f4ec-ac8259ba-705adfd1-787d82b6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:53.332578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:53.332581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:53.332583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:53.332590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:53.332598Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:53.332607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:53.332735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.332740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.332750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:53.332843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1965 partition: 0 MaxSeqNo: 25 sessionId: src_id|7c96f4ec-ac8259ba-705adfd1-787d82b6_0
2025-10-09T21:50:53.333163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d153089b-82695e07-c97ea359-649f9c43_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7c96f4ec-ac8259ba-705adfd1-787d82b6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:53.335576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c96f4ec-ac8259ba-705adfd1-787d82b6_0] Write session: try to update token
2025-10-09T21:50:53.335590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c96f4ec-ac8259ba-705adfd1-787d82b6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:53.342993Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:53.343009Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:53.343017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342814957236436:10711]
2025-10-09T21:50:53.343051Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:53.343057Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:53.343258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:53.343372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:53.343477Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:53.343593Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:53.343755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:53.343787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:53.347881Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1965 sessionId: src_id|7c96f4ec-ac8259ba-705adfd1-787d82b6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:53.350210Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:53.350365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.350382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.350421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:53.350494Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:53.355349Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:53.355365Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:53.355367Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:53.355369Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:53.355380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342814957236436:10711]
2025-10-09T21:50:53.355384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342814957236436:10711]
2025-10-09T21:50:53.355387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342814957236436:10711]
2025-10-09T21:50:53.355426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:53.355439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.355441Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:53.355445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:53.355448Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:53.355452Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:53.355570Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:53.403121Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342815319069649:4041]
2025-10-09T21:50:53.403380Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 1964, pipe: [1:7559342815319069649:4041]
2025-10-09T21:50:53.494704Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:53.501538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.501560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.501694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.501745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.501800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.501856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.501905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.501914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.501974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.502045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.502104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.502160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.502362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.502370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.502419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.502465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.502517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.502581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.502619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.502624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.502677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.502726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.502778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.502841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.502876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.502882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.502935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.502986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.503048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.503102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.503138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.503143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.503193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.503241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.503293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.503343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.503381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.503387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.503437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.503486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.503536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.503585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.503620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.503625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.503671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.503719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.503770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.503828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.503868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.503874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.503920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.503966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.504016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.504065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.504100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.504105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.504152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.504198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.504245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.504294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.504403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.504409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.504445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342815319069649:4041], now have 1 active actors on pipe
2025-10-09T21:50:53.504678Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:53.504745Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342815319069649:4041]
2025-10-09T21:50:53.505004Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:53.505016Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:53.505040Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1965 reason: error at write: Disconnected sessionId: src_id|7c96f4ec-ac8259ba-705adfd1-787d82b6_0
2025-10-09T21:50:53.505177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1965 sessionId: src_id|7c96f4ec-ac8259ba-705adfd1-787d82b6_0 is DEAD
2025-10-09T21:50:53.505209Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:53.505324Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c96f4ec-ac8259ba-705adfd1-787d82b6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:53.505330Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c96f4ec-ac8259ba-705adfd1-787d82b6_0] Write session will restart in 0.009931s
2025-10-09T21:50:53.505353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c96f4ec-ac8259ba-705adfd1-787d82b6_0] Write session: Do CDS request
2025-10-09T21:50:53.505359Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c96f4ec-ac8259ba-705adfd1-787d82b6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:53.516384Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c96f4ec-ac8259ba-705adfd1-787d82b6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:53.516466Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:53.516482Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1966
2025-10-09T21:50:53.516594Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1966 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:53.516626Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1966 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:53.516637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1966 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:53.516640Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:53.517190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1966 sessionId:  describe result for acl check
2025-10-09T21:50:53.517227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:53.517233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:53.517235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:53.517241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342815319069775:4829] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:53.517781Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342815319069775:4829] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:53.520621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342815319069775:4829] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:53.522914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342815319069775:4829] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:53.522927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342815319069775:4829] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:53.522929Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342815319069775:4829] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:53.522935Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1966 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:53.523248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:53.525472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:53.525617Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:53.525624Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:53.525633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:53.525668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:53.525696Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342814957236735:10713]
2025-10-09T21:50:53.526063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:53.526318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:53.526352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:53.526561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:53.526590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.526595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.526595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.526596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.526597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.526597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.526598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.526598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.526599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.526602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.526608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.526610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.526613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.526614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.526616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.526618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.526620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.526622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.526626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.526630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.526635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.526638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.526640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.526643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.526645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.526648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.526651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.526654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.526656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.526659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.526662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.526664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.526667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.526670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.526672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.526675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.526690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:53.532541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:53.533657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:53.533668Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:53.533669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:53.533672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:53.533679Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1965 [2:7559342814957236735:10713]
2025-10-09T21:50:53.533688Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:53.533701Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:53.533703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:53.533705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:53.533706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:53.533719Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:53.533726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:53.533731Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:53.533768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.533773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.533774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.533776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:53.533827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:53.533832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.533839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.533845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.533856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:53.533871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342814957236730:10713]
2025-10-09T21:50:53.533876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342814957236730:10713]
2025-10-09T21:50:53.533877Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342814957236730:10713]
2025-10-09T21:50:53.533882Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:53.533893Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:53.533896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342815319069651:2432], now have 1 active actors on pipe
2025-10-09T21:50:53.533900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:53.533901Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:53.533903Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:53.533908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:53.533910Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:53.533913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342815319069942:4829], now have 1 active actors on pipe
2025-10-09T21:50:53.533930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342814957236730:10713]
2025-10-09T21:50:53.533939Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:53.533952Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:53.533968Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1965
2025-10-09T21:50:53.533980Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1965
2025-10-09T21:50:53.534060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.534071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.534101Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f69c565-8a5da05-7dbb0074-989040e9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:53.534114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:53.534122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:53.534123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:53.534132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:53.534141Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:53.534156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:53.534298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.534305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.534318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:53.534429Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1966 partition: 0 MaxSeqNo: 25 sessionId: src_id|f69c565-8a5da05-7dbb0074-989040e9_0
2025-10-09T21:50:53.534734Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c96f4ec-ac8259ba-705adfd1-787d82b6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f69c565-8a5da05-7dbb0074-989040e9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:53.537274Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f69c565-8a5da05-7dbb0074-989040e9_0] Write session: try to update token
2025-10-09T21:50:53.537293Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f69c565-8a5da05-7dbb0074-989040e9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:53.547188Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:53.547203Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:53.547206Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:53.547208Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:53.547218Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342814957236730:10713]
2025-10-09T21:50:53.547222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342814957236730:10713]
2025-10-09T21:50:53.547226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342814957236730:10713]
2025-10-09T21:50:53.547261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:53.547267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.547268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:53.547273Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:53.547279Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:53.547283Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:53.547404Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:53.550594Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1966 sessionId: src_id|f69c565-8a5da05-7dbb0074-989040e9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:53.552904Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:53.604863Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342815319069955:4041]
2025-10-09T21:50:53.605111Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 1965, pipe: [1:7559342815319069955:4041]
2025-10-09T21:50:53.693627Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:53.693870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.693884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.693921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342815319069955:4041], now have 1 active actors on pipe
2025-10-09T21:50:53.694181Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:53.694187Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342815319069955:4041]
2025-10-09T21:50:53.694198Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:53.694223Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:53.694226Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:53.694276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1966 reason: error at write: Disconnected sessionId: src_id|f69c565-8a5da05-7dbb0074-989040e9_0
2025-10-09T21:50:53.694420Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1966 sessionId: src_id|f69c565-8a5da05-7dbb0074-989040e9_0 is DEAD
2025-10-09T21:50:53.694581Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f69c565-8a5da05-7dbb0074-989040e9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:53.694593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f69c565-8a5da05-7dbb0074-989040e9_0] Write session will restart in 0.005119s
2025-10-09T21:50:53.694621Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f69c565-8a5da05-7dbb0074-989040e9_0] Write session: Do CDS request
2025-10-09T21:50:53.694626Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f69c565-8a5da05-7dbb0074-989040e9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:53.700419Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f69c565-8a5da05-7dbb0074-989040e9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:53.700491Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:53.700507Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1967
2025-10-09T21:50:53.700611Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1967 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:53.700645Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1967 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:53.700650Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1967 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:53.700655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:53.701071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1967 sessionId:  describe result for acl check
2025-10-09T21:50:53.701105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:53.701112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:53.701113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:53.701118Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342815319069972:4826] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:53.701650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342815319069972:4826] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:53.704154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342815319069972:4826] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:53.707389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342815319069972:4826] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:53.707405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342815319069972:4826] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:53.707408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342815319069972:4826] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:53.707414Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1967 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:53.712383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:53.714819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:53.714995Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:53.715004Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:53.715016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:53.715068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:53.715107Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342814957237030:10715]
2025-10-09T21:50:53.715572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:53.715840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:53.715879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:53.716097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:53.716134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.716139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.716139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.716140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.716141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.716141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.716142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.716143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.716143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.716147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.716154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.716156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.716158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.716160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.716162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.716164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.716166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.716169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.716174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.716179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.716183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.716187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.716189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.716194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.716198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.716203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.716207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.716212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.716216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.716221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.716225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.716229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.716234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.716240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.716244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.716248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.716277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:53.722532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:53.723478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:53.723489Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:53.723491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:53.723496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:53.723502Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1966 [2:7559342814957237030:10715]
2025-10-09T21:50:53.723515Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:53.723532Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:53.723541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:53.723544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:53.723547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:53.723564Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:53.723574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:53.723580Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:53.723628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.723633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.723634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.723636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:53.723693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:53.723700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.723711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.723722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.723742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:53.723765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342815319069957:2432], now have 1 active actors on pipe
2025-10-09T21:50:53.723773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342815319070121:4826], now have 1 active actors on pipe
2025-10-09T21:50:53.723781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342814957237025:10715]
2025-10-09T21:50:53.723788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342814957237025:10715]
2025-10-09T21:50:53.723791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342814957237025:10715]
2025-10-09T21:50:53.723801Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:53.723815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:53.723819Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:53.723822Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:53.723824Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:53.723827Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:53.723829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:53.723853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342814957237025:10715]
2025-10-09T21:50:53.723858Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:53.723847Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1966
2025-10-09T21:50:53.723881Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:53.723853Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1966
2025-10-09T21:50:53.723907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.723918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.723945Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3d628a3e-fa209977-e4c19e4d-446b9cb9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:53.723957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:53.723965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:53.723968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:53.723981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:53.723996Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:53.724014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:53.724203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.724212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.724227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:53.724317Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1967 partition: 0 MaxSeqNo: 25 sessionId: src_id|3d628a3e-fa209977-e4c19e4d-446b9cb9_0
2025-10-09T21:50:53.724646Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f69c565-8a5da05-7dbb0074-989040e9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3d628a3e-fa209977-e4c19e4d-446b9cb9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:53.727196Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d628a3e-fa209977-e4c19e4d-446b9cb9_0] Write session: try to update token
2025-10-09T21:50:53.727213Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d628a3e-fa209977-e4c19e4d-446b9cb9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:53.735225Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:53.735242Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:53.735253Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342814957237025:10715]
2025-10-09T21:50:53.735299Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:53.735300Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:53.735438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:53.735568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:53.735701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:53.735828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:53.735996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:53.736029Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:53.739501Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1967 sessionId: src_id|3d628a3e-fa209977-e4c19e4d-446b9cb9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:53.741822Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:53.742492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.742508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.742543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:53.742632Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:53.748001Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:53.748013Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:53.748015Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:53.748016Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:53.748024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342814957237025:10715]
2025-10-09T21:50:53.748027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342814957237025:10715]
2025-10-09T21:50:53.748030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342814957237025:10715]
2025-10-09T21:50:53.748062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:53.748070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.748078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:53.748082Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:53.748088Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:53.748092Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:53.748206Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:53.794251Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342815319070262:4041]
2025-10-09T21:50:53.794519Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 1966, pipe: [1:7559342815319070262:4041]
2025-10-09T21:50:53.893745Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:53.901494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.901517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.901680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.901740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.901806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.901870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.901917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.901921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.901987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.902063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.902128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.902191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.902231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.902238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.902297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.902358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.902414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.902470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.902509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.902517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.902582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.902642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.902706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.902766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.902809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.902816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.902881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.902941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.903000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.903058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.903097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.903104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.903168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.903225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.903281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.903338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.903384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.903392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.903451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.903507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.903566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.903625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.903683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.903691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.903746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.903801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.903860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.903921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.903970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.903977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.904034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.904092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.904152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.904213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.904252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.904259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.904322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:53.904380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:53.904437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:53.904498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:53.904542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:53.904551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:53.904609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342815319070262:4041], now have 1 active actors on pipe
2025-10-09T21:50:53.904872Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:53.904900Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342815319070262:4041]
2025-10-09T21:50:53.904935Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:53.904948Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:53.904978Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1967 reason: error at write: Disconnected sessionId: src_id|3d628a3e-fa209977-e4c19e4d-446b9cb9_0
2025-10-09T21:50:53.905214Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1967 sessionId: src_id|3d628a3e-fa209977-e4c19e4d-446b9cb9_0 is DEAD
2025-10-09T21:50:53.905385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d628a3e-fa209977-e4c19e4d-446b9cb9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:53.905393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d628a3e-fa209977-e4c19e4d-446b9cb9_0] Write session will restart in 0.005654s
2025-10-09T21:50:53.905432Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d628a3e-fa209977-e4c19e4d-446b9cb9_0] Write session: Do CDS request
2025-10-09T21:50:53.905437Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d628a3e-fa209977-e4c19e4d-446b9cb9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:53.905707Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:53.912458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d628a3e-fa209977-e4c19e4d-446b9cb9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:53.912552Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:53.912568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1968
2025-10-09T21:50:53.912732Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1968 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:53.912780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1968 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:53.912792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1968 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:53.912796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:53.913224Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1968 sessionId:  describe result for acl check
2025-10-09T21:50:53.913258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:53.913267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:53.913269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:53.913276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342815319070293:4841] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:53.914157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342815319070293:4841] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:53.917358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342815319070293:4841] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:53.920729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342815319070293:4841] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:53.920743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342815319070293:4841] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:53.920747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342815319070293:4841] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:53.920755Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1968 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:53.924857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:53.927396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:53.927581Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:53.927591Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:53.927603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:53.927657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:53.927698Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342814957237330:10719]
2025-10-09T21:50:53.928286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:53.928537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:53.928577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:53.928894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:53.928944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.928951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.928953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.928954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.928955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.928956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.928958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.928959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.928960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.928965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.928976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.928980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.928984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.928988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.928992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.928996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.929001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.929004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.929011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.929019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:53.929024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.929027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:53.929030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.929033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:53.929035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.929039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:53.929041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.929044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:53.929046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.929050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:53.929053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.929056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.929060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.929063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.929065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.929069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.929084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:53.935313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:53.936394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:53.936407Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:53.936409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:53.936413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:53.936419Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1967 [2:7559342814957237330:10719]
2025-10-09T21:50:53.936433Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:53.936450Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:53.936453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:53.936456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:53.936458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:53.936471Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:53.936474Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:53.936480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:53.936519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.936527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.936528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.936530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:53.936591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:53.936600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.936611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:53.936623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:53.936638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:53.936656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342815319070267:2432], now have 1 active actors on pipe
2025-10-09T21:50:53.936662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342815319070467:4841], now have 1 active actors on pipe
2025-10-09T21:50:53.936668Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342814957237325:10719]
2025-10-09T21:50:53.936671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342814957237325:10719]
2025-10-09T21:50:53.936673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342814957237325:10719]
2025-10-09T21:50:53.936687Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:53.936704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:53.936707Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:53.936710Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:53.936713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:53.936723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:53.936726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:53.936717Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1967
2025-10-09T21:50:53.936745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342814957237325:10719]
2025-10-09T21:50:53.936753Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:53.936755Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:53.936769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.936774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.936785Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1967
2025-10-09T21:50:53.936811Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6c48d475-bda546a2-8d2dc358-c066d078_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:53.936823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:53.936827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:53.936829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:53.936838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:53.936849Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:53.936872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:53.937013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:53.937021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:53.937034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:53.937115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1968 partition: 0 MaxSeqNo: 25 sessionId: src_id|6c48d475-bda546a2-8d2dc358-c066d078_0
2025-10-09T21:50:53.937469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d628a3e-fa209977-e4c19e4d-446b9cb9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6c48d475-bda546a2-8d2dc358-c066d078_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:53.939823Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6c48d475-bda546a2-8d2dc358-c066d078_0] Write session: try to update token
2025-10-09T21:50:53.939838Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6c48d475-bda546a2-8d2dc358-c066d078_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:53.950911Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:53.950931Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:53.950934Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:53.950936Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:53.950946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342814957237325:10719]
2025-10-09T21:50:53.950951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342814957237325:10719]
2025-10-09T21:50:53.950955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342814957237325:10719]
2025-10-09T21:50:53.951005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:53.951014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:53.951018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:53.951200Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:53.951275Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:53.951289Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:53.951293Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:53.952864Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1968 sessionId: src_id|6c48d475-bda546a2-8d2dc358-c066d078_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:53.955482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:54.005259Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342819614037868:4041]
2025-10-09T21:50:54.005543Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 1967, pipe: [1:7559342819614037868:4041]
2025-10-09T21:50:54.103444Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:54.104196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.104210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.104263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342819614037868:4041], now have 1 active actors on pipe
2025-10-09T21:50:54.104428Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:54.104442Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342819614037868:4041]
2025-10-09T21:50:54.104463Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:54.104465Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:54.104432Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:54.104530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1968 reason: error at write: Disconnected sessionId: src_id|6c48d475-bda546a2-8d2dc358-c066d078_0
2025-10-09T21:50:54.104697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1968 sessionId: src_id|6c48d475-bda546a2-8d2dc358-c066d078_0 is DEAD
2025-10-09T21:50:54.104826Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c48d475-bda546a2-8d2dc358-c066d078_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:54.104838Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c48d475-bda546a2-8d2dc358-c066d078_0] Write session will restart in 0.009555s
2025-10-09T21:50:54.104875Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c48d475-bda546a2-8d2dc358-c066d078_0] Write session: Do CDS request
2025-10-09T21:50:54.104882Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c48d475-bda546a2-8d2dc358-c066d078_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:54.115544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6c48d475-bda546a2-8d2dc358-c066d078_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:54.115621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:54.115639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1969
2025-10-09T21:50:54.115765Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1969 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:54.115806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1969 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:54.115816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1969 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:54.115820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:54.116333Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1969 sessionId:  describe result for acl check
2025-10-09T21:50:54.116376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:54.116382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:54.116384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:54.116390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342819614037944:4854] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:54.116956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342819614037944:4854] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:54.119786Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342819614037944:4854] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:54.122789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342819614037944:4854] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:54.122805Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342819614037944:4854] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:54.122808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342819614037944:4854] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:54.122816Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1969 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:54.124497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:54.127055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:54.127216Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:54.127222Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:54.127231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:54.127277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:54.127305Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342819252204924:10721]
2025-10-09T21:50:54.127677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:54.127941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:54.127969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:54.128168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:54.128199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.128204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.128204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.128205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.128206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.128206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.128207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.128208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.128209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.128212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.128219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.128221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.128223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.128225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.128227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.128229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.128231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.128233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.128238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.128243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.128247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.128250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.128253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.128256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.128258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.128261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.128263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.128267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.128269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.128272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.128275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.128278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.128281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.128284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.128286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.128289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.128305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:54.134454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:54.135440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:54.135456Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:54.135458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:54.135462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:54.135468Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1968 [2:7559342819252204924:10721]
2025-10-09T21:50:54.135480Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:54.135496Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:54.135504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:54.135507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:54.135509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:54.135526Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:54.135537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:54.135549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:54.135594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.135601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.135602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.135604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:54.135661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:54.135669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.135687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.135698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.135707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:54.135722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342819614038163:4854], now have 1 active actors on pipe
2025-10-09T21:50:54.135728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342819614037871:2432], now have 1 active actors on pipe
2025-10-09T21:50:54.135733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342819252204919:10721]
2025-10-09T21:50:54.135735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342819252204919:10721]
2025-10-09T21:50:54.135736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342819252204919:10721]
2025-10-09T21:50:54.135747Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:54.135764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:54.135765Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:54.135768Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:54.135771Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:54.135781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:54.135784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:54.135807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342819252204919:10721]
2025-10-09T21:50:54.135817Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:54.135827Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1968
2025-10-09T21:50:54.135833Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1968
2025-10-09T21:50:54.135866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.135874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.135904Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b7683603-b8c3799f-6f57490e-cf262d8e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:54.135913Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:54.135917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:54.135921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:54.135923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:54.135933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:54.135949Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:54.135963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:54.136075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.136083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.136094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:54.137061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1969 partition: 0 MaxSeqNo: 25 sessionId: src_id|b7683603-b8c3799f-6f57490e-cf262d8e_0
2025-10-09T21:50:54.137432Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c48d475-bda546a2-8d2dc358-c066d078_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b7683603-b8c3799f-6f57490e-cf262d8e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:54.139872Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7683603-b8c3799f-6f57490e-cf262d8e_0] Write session: try to update token
2025-10-09T21:50:54.139889Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7683603-b8c3799f-6f57490e-cf262d8e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:54.146996Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:54.147007Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:54.147015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342819252204919:10721]
2025-10-09T21:50:54.147058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:54.147063Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:54.147216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:54.147347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:54.147515Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:54.147637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:54.147813Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:54.147850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:54.151899Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1969 sessionId: src_id|b7683603-b8c3799f-6f57490e-cf262d8e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:54.154397Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:54.154642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.154660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.154695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:54.154777Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:54.160606Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:54.160622Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:54.160624Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:54.160626Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:54.160636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342819252204919:10721]
2025-10-09T21:50:54.160640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342819252204919:10721]
2025-10-09T21:50:54.160644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342819252204919:10721]
2025-10-09T21:50:54.160685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:54.160691Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:54.160696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.160697Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:54.160700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:54.160702Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:54.160836Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:54.204393Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342819614038180:4041]
2025-10-09T21:50:54.204644Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 1968, pipe: [1:7559342819614038180:4041]
2025-10-09T21:50:54.308692Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:54.316541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.316565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.316722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.316789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.316854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.316920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.316967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.316976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.317046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.317113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.317183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.317253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.317296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.317305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.317372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.317444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.317505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.317569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.317611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.317622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.317686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.317750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.317813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.317869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.317909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.317914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.317981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.318053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.318114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.318179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.318231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.318240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.318308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.318369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.318428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.318491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.318539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.318547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.318638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.318701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.318764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.318828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.318877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.318888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.318953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.319012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.319071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.319140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.319187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.319194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.319261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.319324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.319385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.319452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.319500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.319508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.319571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.319630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.319688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.319746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.319787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.319794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.319846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342819614038180:4041], now have 1 active actors on pipe
2025-10-09T21:50:54.319985Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342819614038180:4041]
2025-10-09T21:50:54.319995Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:54.319998Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:54.320011Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:54.320025Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1969 reason: error at write: Disconnected sessionId: src_id|b7683603-b8c3799f-6f57490e-cf262d8e_0
2025-10-09T21:50:54.320214Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1969 sessionId: src_id|b7683603-b8c3799f-6f57490e-cf262d8e_0 is DEAD
2025-10-09T21:50:54.320326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7683603-b8c3799f-6f57490e-cf262d8e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:54.320337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7683603-b8c3799f-6f57490e-cf262d8e_0] Write session will restart in 0.009950s
2025-10-09T21:50:54.320360Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7683603-b8c3799f-6f57490e-cf262d8e_0] Write session: Do CDS request
2025-10-09T21:50:54.320367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7683603-b8c3799f-6f57490e-cf262d8e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:54.321052Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:54.331540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b7683603-b8c3799f-6f57490e-cf262d8e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:54.331747Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:54.331765Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1970
2025-10-09T21:50:54.331888Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1970 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:54.331922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1970 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:54.331929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1970 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:54.331932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:54.332284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1970 sessionId:  describe result for acl check
2025-10-09T21:50:54.332318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:54.332323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:54.332324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:54.332329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342819614038305:4873] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:54.332786Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342819614038305:4873] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:54.335279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342819614038305:4873] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:54.338813Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342819614038305:4873] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:54.338828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342819614038305:4873] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:54.338831Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342819614038305:4873] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:54.338840Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1970 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:54.339972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:54.342454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:54.342676Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:54.342686Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:54.342700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:54.342766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:54.342808Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342819252205260:10735]
2025-10-09T21:50:54.343359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:54.343665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:54.343714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:54.344027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:54.344080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.344086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.344087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.344088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.344089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.344091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.344092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.344093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.344094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.344099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.344110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.344126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.344130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.344134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.344137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.344141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.344146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.344149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.344159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.344167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.344177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.344187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.344191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.344200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.344203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.344208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.344217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.344223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.344227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.344232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.344241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.344248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.344252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.344258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.344263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.344268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.344294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:54.350644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:54.352155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:54.352169Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:54.352172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:54.352176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:54.352183Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1969 [2:7559342819252205260:10735]
2025-10-09T21:50:54.352197Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:54.352213Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:54.352217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:54.352220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:54.352223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:54.352241Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:54.352252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:54.352259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:54.352322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.352331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.352333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.352336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:54.352405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:54.352411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.352425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.352441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.352463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:54.352483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342819614038502:4873], now have 1 active actors on pipe
2025-10-09T21:50:54.352487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342819614038213:2432], now have 1 active actors on pipe
2025-10-09T21:50:54.352493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342819252205255:10735]
2025-10-09T21:50:54.352495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342819252205255:10735]
2025-10-09T21:50:54.352496Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342819252205255:10735]
2025-10-09T21:50:54.352510Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:54.352529Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:54.352533Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:54.352535Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:54.352537Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:54.352540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:54.352542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:54.352568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342819252205255:10735]
2025-10-09T21:50:54.352573Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:54.352581Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:54.352557Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1969
2025-10-09T21:50:54.352563Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1969
2025-10-09T21:50:54.352615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.352626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.352653Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|53001c56-5f9f8d77-dabf68b-14831660_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:54.352664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:54.352667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:54.352669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:54.352676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:54.352692Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:54.352708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:54.352859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.352871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.352896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:54.352985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1970 partition: 0 MaxSeqNo: 25 sessionId: src_id|53001c56-5f9f8d77-dabf68b-14831660_0
2025-10-09T21:50:54.353333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b7683603-b8c3799f-6f57490e-cf262d8e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|53001c56-5f9f8d77-dabf68b-14831660_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:54.355689Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53001c56-5f9f8d77-dabf68b-14831660_0] Write session: try to update token
2025-10-09T21:50:54.355705Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53001c56-5f9f8d77-dabf68b-14831660_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:54.367128Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:54.367144Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:54.367147Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:54.367149Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:54.367159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342819252205255:10735]
2025-10-09T21:50:54.367164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342819252205255:10735]
2025-10-09T21:50:54.367168Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342819252205255:10735]
2025-10-09T21:50:54.367208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:54.367213Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:54.367215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.367217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:54.367220Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:54.367223Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:54.367337Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:54.368484Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1970 sessionId: src_id|53001c56-5f9f8d77-dabf68b-14831660_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:54.370718Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:54.419921Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342819614038518:4041]
2025-10-09T21:50:54.420205Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 1969, pipe: [1:7559342819614038518:4041]
2025-10-09T21:50:54.511721Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:54.511948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.511963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.512008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342819614038518:4041], now have 1 active actors on pipe
2025-10-09T21:50:54.512177Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:54.512262Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:54.512280Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:54.512285Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342819614038518:4041]
2025-10-09T21:50:54.512307Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:54.512342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1970 reason: error at write: Disconnected sessionId: src_id|53001c56-5f9f8d77-dabf68b-14831660_0
2025-10-09T21:50:54.512513Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1970 sessionId: src_id|53001c56-5f9f8d77-dabf68b-14831660_0 is DEAD
2025-10-09T21:50:54.512650Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53001c56-5f9f8d77-dabf68b-14831660_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:54.512662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53001c56-5f9f8d77-dabf68b-14831660_0] Write session will restart in 0.006039s
2025-10-09T21:50:54.512690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53001c56-5f9f8d77-dabf68b-14831660_0] Write session: Do CDS request
2025-10-09T21:50:54.512697Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53001c56-5f9f8d77-dabf68b-14831660_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:54.519502Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53001c56-5f9f8d77-dabf68b-14831660_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:54.519573Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:54.519590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1971
2025-10-09T21:50:54.519715Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1971 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:54.519758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1971 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:54.519768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1971 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:54.519776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:54.520132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1971 sessionId:  describe result for acl check
2025-10-09T21:50:54.520158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:54.520160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:54.520161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:54.520166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342819614038545:4886] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:54.520740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342819614038545:4886] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:54.523612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342819614038545:4886] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:54.526483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342819614038545:4886] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:54.526511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342819614038545:4886] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:54.526515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342819614038545:4886] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:54.526527Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1971 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:54.532469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:54.534810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:54.534998Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:54.535008Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:54.535019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:54.535064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:54.535111Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342819252205559:10737]
2025-10-09T21:50:54.535690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:54.535961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:54.536008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:54.536239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:54.536283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.536289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.536289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.536290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.536291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.536291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.536292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.536292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.536293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.536297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.536304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.536306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.536308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.536310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.536312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.536314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.536316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.536318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.536322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.536328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.536332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.536335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.536338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.536341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.536344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.536346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.536349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.536352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.536354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.536357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.536360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.536363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.536365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.536368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.536372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.536375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.536391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:54.542342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:54.543317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:54.543331Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:54.543334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:54.543338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:54.543345Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1970 [2:7559342819252205559:10737]
2025-10-09T21:50:54.543356Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:54.543371Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:54.543378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:54.543381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:54.543384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:54.543399Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:54.543409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:54.543419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:54.543476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.543483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.543485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.543488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:54.543550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:54.543557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.543567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.543579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.543600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:54.543623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342819614038520:2432], now have 1 active actors on pipe
2025-10-09T21:50:54.543630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342819614038699:4886], now have 1 active actors on pipe
2025-10-09T21:50:54.543637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342819252205554:10737]
2025-10-09T21:50:54.543639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342819252205554:10737]
2025-10-09T21:50:54.543641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342819252205554:10737]
2025-10-09T21:50:54.543652Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:54.543667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:54.543677Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:54.543684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:54.543686Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:54.543687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:54.543688Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:54.543714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342819252205554:10737]
2025-10-09T21:50:54.543724Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:54.543731Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:54.543760Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1970
2025-10-09T21:50:54.543900Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1970
2025-10-09T21:50:54.543963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.543977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.544004Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5e0ad065-5755d441-27421d2f-7e07d1ff_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:54.544014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:54.544018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:54.544020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:54.544029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:54.544046Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:54.544061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:54.544220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.544227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.544240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:54.544318Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1971 partition: 0 MaxSeqNo: 25 sessionId: src_id|5e0ad065-5755d441-27421d2f-7e07d1ff_0
2025-10-09T21:50:54.544655Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53001c56-5f9f8d77-dabf68b-14831660_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5e0ad065-5755d441-27421d2f-7e07d1ff_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:54.547031Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5e0ad065-5755d441-27421d2f-7e07d1ff_0] Write session: try to update token
2025-10-09T21:50:54.547049Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5e0ad065-5755d441-27421d2f-7e07d1ff_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:54.554394Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:54.554412Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:54.554422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342819252205554:10737]
2025-10-09T21:50:54.554461Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:54.554465Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:54.554607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:54.554736Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:54.554856Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:54.554982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:54.555169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:54.555210Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:54.559417Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1971 sessionId: src_id|5e0ad065-5755d441-27421d2f-7e07d1ff_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:54.561542Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:54.561952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.561966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.561998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:54.562097Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:54.567074Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:54.567086Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:54.567087Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:54.567088Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:54.567096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342819252205554:10737]
2025-10-09T21:50:54.567099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342819252205554:10737]
2025-10-09T21:50:54.567102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342819252205554:10737]
2025-10-09T21:50:54.567131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:54.567136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.567137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:54.567145Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:54.567151Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:54.567155Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:54.567261Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:54.612473Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342819614038829:4041]
2025-10-09T21:50:54.612731Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 1970, pipe: [1:7559342819614038829:4041]
2025-10-09T21:50:54.705797Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:54.712769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.712790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.712930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.713001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.713072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.713137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.713178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.713188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.713251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.713318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.713384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.713460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.713845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.713857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.713921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.713975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.714046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.714113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.714156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.714163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.714230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.714314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.714376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.714434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.714480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.714487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.714552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.714612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.714671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.714729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.714775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.714782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.714849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.714904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.714959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.715021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.715075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.715084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.715144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.715193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.715246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.715299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.715337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.715342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.715401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.715463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.715519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.715581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.715732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.715740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.715802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.715855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.715913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.715976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.716017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.716023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.716086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:54.716145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:54.716207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:54.716267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:54.716314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:54.716321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:54.716369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342819614038829:4041], now have 1 active actors on pipe
2025-10-09T21:50:54.716619Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342819614038829:4041]
2025-10-09T21:50:54.716645Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:54.716648Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:54.716661Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:54.716674Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1971 reason: error at write: Disconnected sessionId: src_id|5e0ad065-5755d441-27421d2f-7e07d1ff_0
2025-10-09T21:50:54.716861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1971 sessionId: src_id|5e0ad065-5755d441-27421d2f-7e07d1ff_0 is DEAD
2025-10-09T21:50:54.716967Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:54.716987Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e0ad065-5755d441-27421d2f-7e07d1ff_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:54.716997Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e0ad065-5755d441-27421d2f-7e07d1ff_0] Write session will restart in 0.008124s
2025-10-09T21:50:54.717021Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e0ad065-5755d441-27421d2f-7e07d1ff_0] Write session: Do CDS request
2025-10-09T21:50:54.717027Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e0ad065-5755d441-27421d2f-7e07d1ff_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:54.726485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5e0ad065-5755d441-27421d2f-7e07d1ff_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:54.726542Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:54.726556Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1972
2025-10-09T21:50:54.726707Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1972 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:54.726740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1972 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:54.726749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1972 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:54.726752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:54.727311Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1972 sessionId:  describe result for acl check
2025-10-09T21:50:54.727345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:54.727352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:54.727354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:54.727357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342819614038888:4889] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:54.727753Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342819614038888:4889] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:54.730914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342819614038888:4889] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:54.734299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342819614038888:4889] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:54.734311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342819614038888:4889] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:54.734315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342819614038888:4889] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:54.734335Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1972 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:54.737738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:54.740223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:54.740415Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:54.740424Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:54.740436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:54.740501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:54.740545Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342819252205860:10739]
2025-10-09T21:50:54.740972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:54.741256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:54.741294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:54.741609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:54.741653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.741658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.741658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.741659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.741660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.741660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.741661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.741661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.741662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.741665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.741672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.741674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.741676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.741678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.741681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.741682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.741684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.741686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.741691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.741695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.741699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.741703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.741705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.741708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.741710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.741713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.741716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.741719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.741721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.741724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.741727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.741730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.741732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.741735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.741738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.741741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.741756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:54.747746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:54.748874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:54.748892Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:54.748894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:54.748898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:54.748903Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1971 [2:7559342819252205860:10739]
2025-10-09T21:50:54.748915Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:54.748930Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:54.748933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:54.748935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:54.748937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:54.748955Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:54.748965Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:54.748974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:54.749030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.749037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.749039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.749041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:54.749104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:54.749111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.749122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.749139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.749159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:54.749183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342819614039067:4889], now have 1 active actors on pipe
2025-10-09T21:50:54.749192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342819614038831:2432], now have 1 active actors on pipe
2025-10-09T21:50:54.749198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342819252205855:10739]
2025-10-09T21:50:54.749201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342819252205855:10739]
2025-10-09T21:50:54.749203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342819252205855:10739]
2025-10-09T21:50:54.749214Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:54.749226Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:54.749229Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:54.749232Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:54.749234Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:54.749240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:54.749243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:54.749236Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1971
2025-10-09T21:50:54.749239Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1971
2025-10-09T21:50:54.749265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342819252205855:10739]
2025-10-09T21:50:54.749274Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:54.749276Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:54.749299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.749302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.749326Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|356bb567-94842b3c-7aa99eff-3989fd43_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:54.749335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:54.749337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:54.749339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:54.749347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:54.749354Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:54.749371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:54.749534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.749543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.749558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:54.749650Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1972 partition: 0 MaxSeqNo: 25 sessionId: src_id|356bb567-94842b3c-7aa99eff-3989fd43_0
2025-10-09T21:50:54.749993Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5e0ad065-5755d441-27421d2f-7e07d1ff_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|356bb567-94842b3c-7aa99eff-3989fd43_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:54.752292Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|356bb567-94842b3c-7aa99eff-3989fd43_0] Write session: try to update token
2025-10-09T21:50:54.752307Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|356bb567-94842b3c-7aa99eff-3989fd43_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:54.763678Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:54.763694Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:54.763697Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:54.763699Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:54.763709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342819252205855:10739]
2025-10-09T21:50:54.763713Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342819252205855:10739]
2025-10-09T21:50:54.763717Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342819252205855:10739]
2025-10-09T21:50:54.763759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:54.763766Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:54.763770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.763772Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:54.763773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:54.763775Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:54.763911Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:54.764455Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1972 sessionId: src_id|356bb567-94842b3c-7aa99eff-3989fd43_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:54.766751Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:54.816734Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342819614039145:4041]
2025-10-09T21:50:54.816990Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 1971, pipe: [1:7559342819614039145:4041]
2025-10-09T21:50:54.913066Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:54.913295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.913308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.913347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342819614039145:4041], now have 1 active actors on pipe
2025-10-09T21:50:54.913518Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:54.913544Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:54.913549Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342819614039145:4041]
2025-10-09T21:50:54.913560Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:54.913572Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:54.913635Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1972 reason: error at write: Disconnected sessionId: src_id|356bb567-94842b3c-7aa99eff-3989fd43_0
2025-10-09T21:50:54.913797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1972 sessionId: src_id|356bb567-94842b3c-7aa99eff-3989fd43_0 is DEAD
2025-10-09T21:50:54.913948Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|356bb567-94842b3c-7aa99eff-3989fd43_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:54.913959Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|356bb567-94842b3c-7aa99eff-3989fd43_0] Write session will restart in 0.008607s
2025-10-09T21:50:54.913985Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|356bb567-94842b3c-7aa99eff-3989fd43_0] Write session: Do CDS request
2025-10-09T21:50:54.913991Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|356bb567-94842b3c-7aa99eff-3989fd43_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:54.923470Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|356bb567-94842b3c-7aa99eff-3989fd43_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:54.923569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:54.923582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1973
2025-10-09T21:50:54.923713Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1973 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:54.923758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1973 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:54.923766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1973 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:54.923771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:54.924199Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1973 sessionId:  describe result for acl check
2025-10-09T21:50:54.924237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:54.924243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:54.924245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:54.924251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342819614039215:4893] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:54.924791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342819614039215:4893] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:54.926998Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342819614039215:4893] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:54.929684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342819614039215:4893] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:54.929700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342819614039215:4893] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:54.929703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342819614039215:4893] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:54.929709Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1973 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:54.932367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:54.934594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:54.934775Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:54.934782Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:54.934790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:54.934827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:54.934854Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342819252206165:10743]
2025-10-09T21:50:54.935222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:54.935532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:54.935573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:54.935782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:54.935814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.935819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.935819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.935820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.935821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.935821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.935822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.935822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.935823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.935826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.935833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.935835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.935837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.935839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.935841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.935843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.935844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.935846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.935851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.935855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:54.935860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.935865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:54.935869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.935873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:54.935877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.935881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:54.935885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.935890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:54.935895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.935899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:54.935902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.935905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.935907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.935910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.935912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.935915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.935936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:54.941809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:54.942812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:54.942826Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:54.942828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:54.942832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:54.942838Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1972 [2:7559342819252206165:10743]
2025-10-09T21:50:54.942849Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:54.942861Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:54.942864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:54.942867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:54.942871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:54.942886Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:54.942896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:54.942903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:54.942972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.942980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.942982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.942984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:54.943045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:54.943053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.943066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:54.943082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:54.943102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:54.943127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342819614039151:2432], now have 1 active actors on pipe
2025-10-09T21:50:54.943135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342819614039396:4893], now have 1 active actors on pipe
2025-10-09T21:50:54.943143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342819252206160:10743]
2025-10-09T21:50:54.943145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342819252206160:10743]
2025-10-09T21:50:54.943148Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342819252206160:10743]
2025-10-09T21:50:54.943160Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:54.943155Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1972
2025-10-09T21:50:54.943176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:54.943184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:54.943161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1972
2025-10-09T21:50:54.943186Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:54.943188Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:54.943196Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:54.943199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:54.943225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342819252206160:10743]
2025-10-09T21:50:54.943237Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:54.943250Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:54.943257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.943261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.943292Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7e680e1f-e5720d9b-6ba569fa-da14b74e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:54.943301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:54.943304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:54.943307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:54.943314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:54.943322Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:54.943337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:54.943490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:54.943501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:54.943525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:54.943608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1973 partition: 0 MaxSeqNo: 25 sessionId: src_id|7e680e1f-e5720d9b-6ba569fa-da14b74e_0
2025-10-09T21:50:54.943872Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|356bb567-94842b3c-7aa99eff-3989fd43_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7e680e1f-e5720d9b-6ba569fa-da14b74e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:54.946137Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e680e1f-e5720d9b-6ba569fa-da14b74e_0] Write session: try to update token
2025-10-09T21:50:54.946150Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e680e1f-e5720d9b-6ba569fa-da14b74e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:54.956547Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:54.956563Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:54.956566Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:54.956567Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:54.956577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342819252206160:10743]
2025-10-09T21:50:54.956581Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342819252206160:10743]
2025-10-09T21:50:54.956585Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342819252206160:10743]
2025-10-09T21:50:54.956627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:54.956627Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:54.956633Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:54.956633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:54.956636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:54.956636Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:54.956765Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:54.958216Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1973 sessionId: src_id|7e680e1f-e5720d9b-6ba569fa-da14b74e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:54.960239Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:55.013879Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342823909006757:4041]
2025-10-09T21:50:55.014144Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 1972, pipe: [1:7559342823909006757:4041]
2025-10-09T21:50:55.101647Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:55.102430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:55.102448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:55.102507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342823909006757:4041], now have 1 active actors on pipe
2025-10-09T21:50:55.102700Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342823909006757:4041]
2025-10-09T21:50:55.102700Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:55.102705Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:55.102736Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:55.102721Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:55.102764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1973 reason: error at write: Disconnected sessionId: src_id|7e680e1f-e5720d9b-6ba569fa-da14b74e_0
2025-10-09T21:50:55.102947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1973 sessionId: src_id|7e680e1f-e5720d9b-6ba569fa-da14b74e_0 is DEAD
2025-10-09T21:50:55.103127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e680e1f-e5720d9b-6ba569fa-da14b74e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:55.103139Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e680e1f-e5720d9b-6ba569fa-da14b74e_0] Write session will restart in 0.009512s
2025-10-09T21:50:55.103168Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e680e1f-e5720d9b-6ba569fa-da14b74e_0] Write session: Do CDS request
2025-10-09T21:50:55.103175Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e680e1f-e5720d9b-6ba569fa-da14b74e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:55.113514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e680e1f-e5720d9b-6ba569fa-da14b74e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:55.113632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:55.113648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1974
2025-10-09T21:50:55.113774Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1974 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:55.113806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1974 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:55.113812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1974 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:55.113815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:55.114284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1974 sessionId:  describe result for acl check
2025-10-09T21:50:55.114328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:55.114332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:55.114333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:55.114339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342823909006855:4917] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:55.114826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342823909006855:4917] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:55.117002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342823909006855:4917] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:55.119598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342823909006855:4917] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:55.119613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342823909006855:4917] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:55.119616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342823909006855:4917] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:55.119627Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1974 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:55.121738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:55.124783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:55.124975Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:55.124985Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:55.124997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:55.125050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:55.125090Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342823547173764:10745]
2025-10-09T21:50:55.125690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:55.125977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:55.126021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:55.126355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:55.126406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.126412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.126414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.126415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.126416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.126417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.126419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.126420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.126421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.126426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.126438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.126443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.126447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.126450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.126454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.126458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.126462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.126465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.126472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.126481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.126486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.126497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.126506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.126511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.126520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.126525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.126530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.126540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.126549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.126554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.126563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.126569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.126581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.126587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.126592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.126597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.126632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:55.133018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:55.134249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:55.134264Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:55.134266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:55.134271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:55.134278Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1973 [2:7559342823547173764:10745]
2025-10-09T21:50:55.134292Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:55.134309Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:55.134317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:55.134320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:55.134323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:55.134338Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:55.134350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:55.134358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:55.134416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.134422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.134425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.134428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:55.134497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:55.134505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.134516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.134531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.134551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:55.134576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342823909007049:4917], now have 1 active actors on pipe
2025-10-09T21:50:55.134583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342823909006760:2432], now have 1 active actors on pipe
2025-10-09T21:50:55.134591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342823547173759:10745]
2025-10-09T21:50:55.134598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342823547173759:10745]
2025-10-09T21:50:55.134600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342823547173759:10745]
2025-10-09T21:50:55.134613Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:55.134616Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:55.134618Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:55.134620Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:55.134631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:55.134602Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1973
2025-10-09T21:50:55.134608Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1973
2025-10-09T21:50:55.134646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:55.134655Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:55.134695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:55.134706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:55.134722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342823547173759:10745]
2025-10-09T21:50:55.134732Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:55.134738Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:55.134763Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e7bbb6f6-24d94415-37c4fa4e-97ab6717_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:55.134775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:55.134782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:55.134784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:55.134796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:55.134810Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:55.135198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:55.135393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:55.135404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:55.135419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:55.135508Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1974 partition: 0 MaxSeqNo: 25 sessionId: src_id|e7bbb6f6-24d94415-37c4fa4e-97ab6717_0
2025-10-09T21:50:55.135848Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e680e1f-e5720d9b-6ba569fa-da14b74e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e7bbb6f6-24d94415-37c4fa4e-97ab6717_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:55.138202Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e7bbb6f6-24d94415-37c4fa4e-97ab6717_0] Write session: try to update token
2025-10-09T21:50:55.138216Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e7bbb6f6-24d94415-37c4fa4e-97ab6717_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:55.149217Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:55.149238Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:55.149241Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:55.149243Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:55.149253Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342823547173759:10745]
2025-10-09T21:50:55.149259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342823547173759:10745]
2025-10-09T21:50:55.149263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342823547173759:10745]
2025-10-09T21:50:55.149299Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:55.149303Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:55.149306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:55.149306Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:55.149315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.149318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:55.149474Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:55.151164Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1974 sessionId: src_id|e7bbb6f6-24d94415-37c4fa4e-97ab6717_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:55.153520Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:55.202827Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342823909007073:4041]
2025-10-09T21:50:55.203119Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 1973, pipe: [1:7559342823909007073:4041]
2025-10-09T21:50:55.298280Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:55.298507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:55.298522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:55.298560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342823909007073:4041], now have 1 active actors on pipe
2025-10-09T21:50:55.298789Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342823909007073:4041]
2025-10-09T21:50:55.298789Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:55.298794Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:55.298822Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:55.298857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1974 reason: error at write: Disconnected sessionId: src_id|e7bbb6f6-24d94415-37c4fa4e-97ab6717_0
2025-10-09T21:50:55.298733Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:55.299035Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1974 sessionId: src_id|e7bbb6f6-24d94415-37c4fa4e-97ab6717_0 is DEAD
2025-10-09T21:50:55.299199Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e7bbb6f6-24d94415-37c4fa4e-97ab6717_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:55.299212Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e7bbb6f6-24d94415-37c4fa4e-97ab6717_0] Write session will restart in 0.009052s
2025-10-09T21:50:55.299239Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e7bbb6f6-24d94415-37c4fa4e-97ab6717_0] Write session: Do CDS request
2025-10-09T21:50:55.299246Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e7bbb6f6-24d94415-37c4fa4e-97ab6717_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:55.309487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e7bbb6f6-24d94415-37c4fa4e-97ab6717_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:55.309612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:55.309629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1975
2025-10-09T21:50:55.309746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1975 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:55.309776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1975 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:55.309781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1975 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:55.309785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:55.310273Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1975 sessionId:  describe result for acl check
2025-10-09T21:50:55.310328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:55.310338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:55.310340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:55.310346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342823909007165:4932] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:55.310815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342823909007165:4932] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:55.313364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342823909007165:4932] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:55.316242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342823909007165:4932] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:55.316255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342823909007165:4932] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:55.316257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342823909007165:4932] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:55.316264Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1975 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:55.318467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:55.320776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:55.320967Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:55.320979Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:55.320993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:55.321060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:55.321099Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342823547174071:10747]
2025-10-09T21:50:55.321672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:55.321987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:55.322050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:55.322346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:55.322406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.322415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.322416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.322417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.322418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.322419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.322420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.322421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.322423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.322427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.322437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.322441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.322445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.322449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.322453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.322456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.322460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.322463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.322472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.322480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.322486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.322492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.322499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.322504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.322508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.322513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.322517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.322521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.322523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.322526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.322533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.322538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.322547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.322552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.322556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.322567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.322595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:55.328748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:55.329941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:55.329955Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:55.329958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:55.329962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:55.329969Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1974 [2:7559342823547174071:10747]
2025-10-09T21:50:55.329981Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:55.330001Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:55.330004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:55.330007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:55.330009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:55.330046Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:55.330052Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:55.330059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:55.330111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.330120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.330124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.330126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:55.330192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:55.330201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.330213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.330226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.330243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:55.330269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342823909007361:4932], now have 1 active actors on pipe
2025-10-09T21:50:55.330277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342823909007084:2432], now have 1 active actors on pipe
2025-10-09T21:50:55.330284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342823547174062:10747]
2025-10-09T21:50:55.330290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342823547174062:10747]
2025-10-09T21:50:55.330292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342823547174062:10747]
2025-10-09T21:50:55.330307Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:55.330301Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1974
2025-10-09T21:50:55.330318Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:55.330335Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1974
2025-10-09T21:50:55.330324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:55.330327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:55.330329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:55.330339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:55.330342Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:55.330370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342823547174062:10747]
2025-10-09T21:50:55.330373Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:55.330398Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:55.330403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:55.330408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:55.330440Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a7c294ce-ea4a6805-49114e1f-43ca6e0f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:55.330455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:55.330459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:55.330462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:55.330472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:55.330487Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:55.330506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:55.330716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:55.330735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:55.330750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:55.330837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1975 partition: 0 MaxSeqNo: 25 sessionId: src_id|a7c294ce-ea4a6805-49114e1f-43ca6e0f_0
2025-10-09T21:50:55.331191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e7bbb6f6-24d94415-37c4fa4e-97ab6717_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a7c294ce-ea4a6805-49114e1f-43ca6e0f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:55.333593Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c294ce-ea4a6805-49114e1f-43ca6e0f_0] Write session: try to update token
2025-10-09T21:50:55.333613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c294ce-ea4a6805-49114e1f-43ca6e0f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:55.343950Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:55.343963Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:55.343964Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:55.343966Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:55.343973Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342823547174062:10747]
2025-10-09T21:50:55.343976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342823547174062:10747]
2025-10-09T21:50:55.343979Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342823547174062:10747]
2025-10-09T21:50:55.344012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:55.344018Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:55.344020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.344022Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:55.344022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:55.344025Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:55.344136Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:55.346569Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1975 sessionId: src_id|a7c294ce-ea4a6805-49114e1f-43ca6e0f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:55.348790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:55.398855Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342823909007419:4041]
2025-10-09T21:50:55.399093Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 1974, pipe: [1:7559342823909007419:4041]
2025-10-09T21:50:55.495647Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:55.495993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:55.496010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:55.496057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342823909007419:4041], now have 1 active actors on pipe
2025-10-09T21:50:55.496298Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:55.496313Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342823909007419:4041]
2025-10-09T21:50:55.496317Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:55.496329Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:55.496410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1975 reason: error at write: Disconnected sessionId: src_id|a7c294ce-ea4a6805-49114e1f-43ca6e0f_0
2025-10-09T21:50:55.496588Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1975 sessionId: src_id|a7c294ce-ea4a6805-49114e1f-43ca6e0f_0 is DEAD
2025-10-09T21:50:55.496769Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c294ce-ea4a6805-49114e1f-43ca6e0f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:55.496780Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c294ce-ea4a6805-49114e1f-43ca6e0f_0] Write session will restart in 0.007482s
2025-10-09T21:50:55.496804Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c294ce-ea4a6805-49114e1f-43ca6e0f_0] Write session: Do CDS request
2025-10-09T21:50:55.496813Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c294ce-ea4a6805-49114e1f-43ca6e0f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:55.496819Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:55.505463Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c294ce-ea4a6805-49114e1f-43ca6e0f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:55.505552Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:55.505567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1976
2025-10-09T21:50:55.505692Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1976 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:55.505733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1976 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:55.505740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1976 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:55.505744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:55.506135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1976 sessionId:  describe result for acl check
2025-10-09T21:50:55.506162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:55.506164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:55.506165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:55.506169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342823909007467:4944] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:55.506667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342823909007467:4944] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:55.510133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342823909007467:4944] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:55.513822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342823909007467:4944] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:55.513838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342823909007467:4944] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:55.513842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342823909007467:4944] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:55.513858Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1976 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:55.516547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:55.519105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:55.519302Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:55.519380Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:55.519954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:55.520024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:55.520059Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342823547174414:10761]
2025-10-09T21:50:55.520589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:55.520870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:55.520911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:55.521250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:55.521300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.521307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.521309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.521310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.521311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.521312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.521313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.521315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.521316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.521321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.521332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.521336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.521338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.521340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.521342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.521344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.521346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.521348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.521353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.521358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.521362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.521366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.521368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.521371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.521374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.521377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.521379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.521382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.521385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.521388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.521391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.521394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.521396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.521399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.521403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.521406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.521447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:55.527708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:55.528682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:55.528699Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:55.528702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:55.528706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:55.528713Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1975 [2:7559342823547174414:10761]
2025-10-09T21:50:55.528725Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:55.528741Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:55.528745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:55.528748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:55.528750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:55.528765Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:55.528776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:55.528783Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:55.528828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.528835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.528836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.528838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:55.528887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:55.528895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.528902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.528907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.528917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:55.528936Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342823547174404:10761]
2025-10-09T21:50:55.528942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342823547174404:10761]
2025-10-09T21:50:55.528944Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342823547174404:10761]
2025-10-09T21:50:55.528953Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:55.528964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:55.528968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:55.528975Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:55.528978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:55.528982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:55.528985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:55.528997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342823909007421:2432], now have 1 active actors on pipe
2025-10-09T21:50:55.529001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342823909007667:4944], now have 1 active actors on pipe
2025-10-09T21:50:55.529023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342823547174404:10761]
2025-10-09T21:50:55.529028Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1975
2025-10-09T21:50:55.529040Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1975
2025-10-09T21:50:55.529034Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:55.529042Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:55.529072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:55.529077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:55.529128Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f0f143db-fde3cebd-6714a1e-92d0173_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:55.529139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:55.529142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:55.529144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:55.529150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:55.529162Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:55.529172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:55.529302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:55.529311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:55.529325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:55.529424Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1976 partition: 0 MaxSeqNo: 25 sessionId: src_id|f0f143db-fde3cebd-6714a1e-92d0173_0
2025-10-09T21:50:55.529744Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c294ce-ea4a6805-49114e1f-43ca6e0f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f0f143db-fde3cebd-6714a1e-92d0173_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:55.532063Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0f143db-fde3cebd-6714a1e-92d0173_0] Write session: try to update token
2025-10-09T21:50:55.532095Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0f143db-fde3cebd-6714a1e-92d0173_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:55.543114Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:55.543131Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:55.543134Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:55.543136Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:55.543145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342823547174404:10761]
2025-10-09T21:50:55.543149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342823547174404:10761]
2025-10-09T21:50:55.543153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342823547174404:10761]
2025-10-09T21:50:55.543195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:55.543202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.543205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:55.543230Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:55.543239Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:55.543244Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:55.543332Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:55.544143Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1976 sessionId: src_id|f0f143db-fde3cebd-6714a1e-92d0173_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:55.546254Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:55.596544Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342823909007738:4041]
2025-10-09T21:50:55.596817Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 1975, pipe: [1:7559342823909007738:4041]
2025-10-09T21:50:55.682885Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:55.683515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:55.683531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:55.683571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342823909007738:4041], now have 1 active actors on pipe
2025-10-09T21:50:55.683796Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342823909007738:4041]
2025-10-09T21:50:55.683809Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:55.683833Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:55.683836Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:55.683889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1976 reason: error at write: Disconnected sessionId: src_id|f0f143db-fde3cebd-6714a1e-92d0173_0
2025-10-09T21:50:55.683888Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:55.684056Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1976 sessionId: src_id|f0f143db-fde3cebd-6714a1e-92d0173_0 is DEAD
2025-10-09T21:50:55.684212Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0f143db-fde3cebd-6714a1e-92d0173_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:55.684223Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0f143db-fde3cebd-6714a1e-92d0173_0] Write session will restart in 0.005383s
2025-10-09T21:50:55.684249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0f143db-fde3cebd-6714a1e-92d0173_0] Write session: Do CDS request
2025-10-09T21:50:55.684255Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0f143db-fde3cebd-6714a1e-92d0173_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:55.690425Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f0f143db-fde3cebd-6714a1e-92d0173_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:55.690478Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:55.690494Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1977
2025-10-09T21:50:55.690635Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1977 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:55.690676Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1977 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:55.690684Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1977 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:55.690687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:55.691074Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1977 sessionId:  describe result for acl check
2025-10-09T21:50:55.691108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:55.691116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:55.691117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:55.691123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342823909007759:4950] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:55.691143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:55.691626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342823909007759:4950] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:55.692388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:55.692535Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:55.692557Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:55.692571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:55.692614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:55.692643Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342823547174474:10763]
2025-10-09T21:50:55.692979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:55.693205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:55.693235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:55.693411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342823909007759:4950] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:55.693460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:55.693493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.693497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.693498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.693498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.693499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.693500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.693501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.693501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.693502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.693505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.693511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.693513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.693515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.693517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.693519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.693521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.693523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.693525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.693529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.693535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.693540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.693545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.693552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.693556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.693560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.693566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.693571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.693576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.693581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.693586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.693589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.693592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.693594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.693596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.693599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.693601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.693616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:55.697874Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342823909007759:4950] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:55.697887Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342823909007759:4950] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:55.697891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342823909007759:4950] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:55.697898Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1977 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:55.699129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:55.700105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:55.700122Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:55.700123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:55.700130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:55.700137Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1976 [2:7559342823547174474:10763]
2025-10-09T21:50:55.700150Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:55.700165Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:55.700172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:55.700175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:55.700177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:55.700188Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:55.700195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:55.700200Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:55.700247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.700251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.700252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.700254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:55.700300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:55.700304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.700311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.700317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.700325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:55.700339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342823547174469:10763]
2025-10-09T21:50:55.700344Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342823547174469:10763]
2025-10-09T21:50:55.700345Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342823547174469:10763]
2025-10-09T21:50:55.700351Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:55.700360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:55.700362Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:55.700364Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:55.700366Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:55.700371Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:55.700373Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:55.700382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342823909007740:2432], now have 1 active actors on pipe
2025-10-09T21:50:55.700388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342823909007799:4950], now have 1 active actors on pipe
2025-10-09T21:50:55.700402Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342823547174469:10763]
2025-10-09T21:50:55.700413Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:55.700425Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:55.700432Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1976
2025-10-09T21:50:55.700454Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1976
2025-10-09T21:50:55.700503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:55.700511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:55.700534Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9152507f-30632a6d-a2d41b0-245ec239_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:55.700542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:55.700545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:55.700546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:55.700553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:55.700561Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:55.700573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:55.700725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:55.700730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:55.700740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:55.700813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1977 partition: 0 MaxSeqNo: 25 sessionId: src_id|9152507f-30632a6d-a2d41b0-245ec239_0
2025-10-09T21:50:55.701061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f0f143db-fde3cebd-6714a1e-92d0173_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9152507f-30632a6d-a2d41b0-245ec239_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:55.703325Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9152507f-30632a6d-a2d41b0-245ec239_0] Write session: try to update token
2025-10-09T21:50:55.703341Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9152507f-30632a6d-a2d41b0-245ec239_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:55.714181Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:55.714197Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:55.714198Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:55.714201Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:55.714208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342823547174469:10763]
2025-10-09T21:50:55.714211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342823547174469:10763]
2025-10-09T21:50:55.714215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342823547174469:10763]
2025-10-09T21:50:55.714248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:55.714253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.714255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:55.714260Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:55.714266Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:55.714269Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:55.714373Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:55.715437Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1977 sessionId: src_id|9152507f-30632a6d-a2d41b0-245ec239_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:55.717659Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:55.783925Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342823909007807:4041]
2025-10-09T21:50:55.784136Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 1976, pipe: [1:7559342823909007807:4041]
2025-10-09T21:50:55.853682Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:55.853924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:55.853936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:55.853974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342823909007807:4041], now have 1 active actors on pipe
2025-10-09T21:50:55.854195Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:55.854176Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:55.854194Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:55.854199Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:55.854245Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342823909007807:4041]
2025-10-09T21:50:55.854262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1977 reason: error at write: Disconnected sessionId: src_id|9152507f-30632a6d-a2d41b0-245ec239_0
2025-10-09T21:50:55.854431Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1977 sessionId: src_id|9152507f-30632a6d-a2d41b0-245ec239_0 is DEAD
2025-10-09T21:50:55.854586Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9152507f-30632a6d-a2d41b0-245ec239_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:55.854598Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9152507f-30632a6d-a2d41b0-245ec239_0] Write session will restart in 0.008733s
2025-10-09T21:50:55.854625Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9152507f-30632a6d-a2d41b0-245ec239_0] Write session: Do CDS request
2025-10-09T21:50:55.854631Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9152507f-30632a6d-a2d41b0-245ec239_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:55.861249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:55.862803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:55.862990Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:55.863001Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:55.863012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:55.863067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:55.863110Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342823547174535:10767]
2025-10-09T21:50:55.863679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:55.863958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:55.863998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:55.864317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:55.864367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.864373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.864374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.864375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.864377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.864378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.864379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.864383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.864384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.864389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.864400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.864404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.864408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.864412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.864416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.864419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.864423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.864427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.864435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.864442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:55.864449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.864455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:55.864459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.864463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:55.864467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.864472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:55.864476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.864481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:55.864490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.864495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:55.864500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.864505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9152507f-30632a6d-a2d41b0-245ec239_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:55.864504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.864508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.864513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.864517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.864522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.864551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:55.864568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:55.864582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1978
2025-10-09T21:50:55.864715Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1978 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:55.864758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1978 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:55.864768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1978 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:55.864771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:55.865124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1978 sessionId:  describe result for acl check
2025-10-09T21:50:55.865155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:55.865162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:55.865164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:55.865177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342823909007834:4964] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:55.865645Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342823909007834:4964] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:55.867646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342823909007834:4964] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:55.870125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342823909007834:4964] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:55.870139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342823909007834:4964] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:55.870143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342823909007834:4964] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:55.870151Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1978 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:55.870578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:55.871679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:55.871692Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:55.871694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:55.871697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:55.871702Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1977 [2:7559342823547174535:10767]
2025-10-09T21:50:55.871713Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:55.871727Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:55.871733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:55.871735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:55.871738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:55.871753Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:55.871764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:55.871771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:55.871825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.871833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.871835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.871838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:55.871900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:55.871908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.871920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:55.871931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:55.871962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:55.871989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342823909007808:2432], now have 1 active actors on pipe
2025-10-09T21:50:55.871993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342823909007869:4964], now have 1 active actors on pipe
2025-10-09T21:50:55.871999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342823547174530:10767]
2025-10-09T21:50:55.872002Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342823547174530:10767]
2025-10-09T21:50:55.872004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342823547174530:10767]
2025-10-09T21:50:55.872019Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:55.872004Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1977
2025-10-09T21:50:55.872022Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1977
2025-10-09T21:50:55.872032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:55.872044Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:55.872047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:55.872048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:55.872049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:55.872050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:55.872087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:55.872097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:55.872109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342823547174530:10767]
2025-10-09T21:50:55.872122Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:55.872139Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:55.872153Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e5798fd1-c1b5123b-ac52ec62-124b10b0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:55.872161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:55.872164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:55.872166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:55.872177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:55.872191Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:55.872216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:55.872396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:55.872410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:55.872433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:55.872545Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1978 partition: 0 MaxSeqNo: 25 sessionId: src_id|e5798fd1-c1b5123b-ac52ec62-124b10b0_0
2025-10-09T21:50:55.872834Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9152507f-30632a6d-a2d41b0-245ec239_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e5798fd1-c1b5123b-ac52ec62-124b10b0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:55.875069Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5798fd1-c1b5123b-ac52ec62-124b10b0_0] Write session: try to update token
2025-10-09T21:50:55.875082Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5798fd1-c1b5123b-ac52ec62-124b10b0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:55.885877Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:55.885890Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:55.885892Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:55.885893Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:55.885900Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342823547174530:10767]
2025-10-09T21:50:55.885903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342823547174530:10767]
2025-10-09T21:50:55.885906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342823547174530:10767]
2025-10-09T21:50:55.885941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:55.885954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:55.885957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:55.885963Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:55.885970Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:55.885976Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:55.886109Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:55.887185Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1978 sessionId: src_id|e5798fd1-c1b5123b-ac52ec62-124b10b0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:55.889490Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:55.954410Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342823909007879:4041]
2025-10-09T21:50:55.954655Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 1977, pipe: [1:7559342823909007879:4041]
2025-10-09T21:50:56.025116Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:56.025410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.025436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.025477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342823909007879:4041], now have 1 active actors on pipe
2025-10-09T21:50:56.025663Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:56.025735Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:56.025751Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:56.025758Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342823909007879:4041]
2025-10-09T21:50:56.025781Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:56.025815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1978 reason: error at write: Disconnected sessionId: src_id|e5798fd1-c1b5123b-ac52ec62-124b10b0_0
2025-10-09T21:50:56.025954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1978 sessionId: src_id|e5798fd1-c1b5123b-ac52ec62-124b10b0_0 is DEAD
2025-10-09T21:50:56.026121Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5798fd1-c1b5123b-ac52ec62-124b10b0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:56.026135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5798fd1-c1b5123b-ac52ec62-124b10b0_0] Write session will restart in 0.005751s
2025-10-09T21:50:56.026164Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5798fd1-c1b5123b-ac52ec62-124b10b0_0] Write session: Do CDS request
2025-10-09T21:50:56.026170Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5798fd1-c1b5123b-ac52ec62-124b10b0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:56.033527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5798fd1-c1b5123b-ac52ec62-124b10b0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:56.033588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:56.033605Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1979
2025-10-09T21:50:56.033614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:56.033775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1979 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:56.033819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1979 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:56.033835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1979 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:56.033839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:56.034204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1979 sessionId:  describe result for acl check
2025-10-09T21:50:56.034232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:56.034237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:56.034238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:56.034243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342828203975198:4974] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:56.034769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342828203975198:4974] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:56.035198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:56.035372Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:56.035382Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:56.035394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:56.035450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:56.035490Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342827842141889:10769]
2025-10-09T21:50:56.036074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:56.036368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:56.036420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:56.036593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342828203975198:4974] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:56.036737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:56.036805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.036812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.036813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.036815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.036817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.036818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.036820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.036821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.036822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.036827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.036837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.036845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.036849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.036854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.036859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.036868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.036872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.036880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.036888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.036896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.036902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.036908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.036913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.036918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.036922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.036927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.036931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.036944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.036948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.036954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.036963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.036972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.036977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.036982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.036989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.036999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.037034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:56.042171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342828203975198:4974] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:56.042186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342828203975198:4974] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:56.042190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342828203975198:4974] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:56.042197Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1979 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:56.043443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:56.044458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:56.044472Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:56.044474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:56.044477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:56.044483Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1978 [2:7559342827842141889:10769]
2025-10-09T21:50:56.044494Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:56.044509Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:56.044514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:56.044516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:56.044518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:56.044532Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:56.044539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:56.044545Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:56.044589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.044597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.044601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.044604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:56.044673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:56.044680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.044688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.044694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.044704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:56.044723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975176:2432], now have 1 active actors on pipe
2025-10-09T21:50:56.044729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975237:4974], now have 1 active actors on pipe
2025-10-09T21:50:56.044735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342827842141884:10769]
2025-10-09T21:50:56.044737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342827842141884:10769]
2025-10-09T21:50:56.044738Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342827842141884:10769]
2025-10-09T21:50:56.044750Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:56.044761Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:56.044762Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:56.044768Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:56.044770Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:56.044776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:56.044779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:56.044809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342827842141884:10769]
2025-10-09T21:50:56.044823Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:56.044828Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:56.044810Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1978
2025-10-09T21:50:56.044821Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1978
2025-10-09T21:50:56.044863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.044873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.044894Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dc42ac29-1d06d87-41e3bf62-30759302_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:56.044904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:56.044910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:56.044912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:56.044920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:56.044931Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:56.044945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:56.045056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.045064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.045077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:56.045163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1979 partition: 0 MaxSeqNo: 25 sessionId: src_id|dc42ac29-1d06d87-41e3bf62-30759302_0
2025-10-09T21:50:56.045518Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5798fd1-c1b5123b-ac52ec62-124b10b0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dc42ac29-1d06d87-41e3bf62-30759302_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:56.047972Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dc42ac29-1d06d87-41e3bf62-30759302_0] Write session: try to update token
2025-10-09T21:50:56.047990Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dc42ac29-1d06d87-41e3bf62-30759302_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:56.060638Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1979 sessionId: src_id|dc42ac29-1d06d87-41e3bf62-30759302_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:56.062417Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:56.062432Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:56.062442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342827842141884:10769]
2025-10-09T21:50:56.062491Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:56.062496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:56.062671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:56.062823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:56.062871Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:56.062959Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:56.063092Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:56.063263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:56.063319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.063335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.063357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:56.063391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:56.063482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:56.069759Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:56.069774Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:56.069776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:56.069778Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:56.069787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342827842141884:10769]
2025-10-09T21:50:56.069791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342827842141884:10769]
2025-10-09T21:50:56.069794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342827842141884:10769]
2025-10-09T21:50:56.069828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:56.069835Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:56.069840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.069840Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:56.069841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:56.069843Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:56.069964Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:56.125942Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342828203975247:4041]
2025-10-09T21:50:56.126245Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 1978, pipe: [1:7559342828203975247:4041]
2025-10-09T21:50:56.216687Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:56.223716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.223739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.223880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:56.223944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:56.224011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:56.224070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:56.224112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:56.224120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:56.224182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:56.224237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:56.224291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:56.224355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:56.224399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:56.224404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:56.224458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:56.224509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:56.224564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:56.224621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:56.224661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:56.224667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:56.224724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:56.224779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:56.224845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:56.224903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:56.224953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:56.224961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:56.225024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:56.225079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:56.225134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:56.225190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:56.225237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:56.225245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:56.225300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:56.225358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:56.225432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:56.225495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:56.225540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:56.225550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:56.225613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:56.225670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:56.225730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:56.225798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:56.225841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:56.225847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:56.225907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:56.225957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:56.226011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:56.226089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:56.226135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:56.226142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:56.226205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:56.226256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:56.226310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:56.226362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:56.226396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:56.226402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:56.226455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:56.226504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:56.226560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:56.226622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:56.226668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:56.226676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:56.226724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975247:4041], now have 1 active actors on pipe
2025-10-09T21:50:56.226914Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342828203975247:4041]
2025-10-09T21:50:56.226940Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:56.226954Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:56.226958Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:56.226983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1979 reason: error at write: Disconnected sessionId: src_id|dc42ac29-1d06d87-41e3bf62-30759302_0
2025-10-09T21:50:56.227157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1979 sessionId: src_id|dc42ac29-1d06d87-41e3bf62-30759302_0 is DEAD
2025-10-09T21:50:56.227272Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc42ac29-1d06d87-41e3bf62-30759302_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:56.227280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc42ac29-1d06d87-41e3bf62-30759302_0] Write session will restart in 0.005874s
2025-10-09T21:50:56.227304Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc42ac29-1d06d87-41e3bf62-30759302_0] Write session: Do CDS request
2025-10-09T21:50:56.227311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc42ac29-1d06d87-41e3bf62-30759302_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:56.227573Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:56.234399Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dc42ac29-1d06d87-41e3bf62-30759302_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:56.234494Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:56.234510Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1980
2025-10-09T21:50:56.234644Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1980 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:56.234670Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1980 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:56.234677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1980 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:56.234681Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:56.234954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:56.235090Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1980 sessionId:  describe result for acl check
2025-10-09T21:50:56.235126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:56.235133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:56.235134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:56.235140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342828203975274:4980] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:56.235726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342828203975274:4980] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:56.236308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:56.236472Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:56.236483Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:56.236492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:56.236531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:56.236561Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342827842141948:10771]
2025-10-09T21:50:56.237095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:56.237336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:56.237377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:56.237692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:56.237742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.237744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.237745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.237746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.237747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.237749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.237750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.237752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.237753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.237758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.237767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.237772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.237776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.237780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.237784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.237788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.237792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.237796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.237803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.237811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.237822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.237833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.237837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.237847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.237855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.237860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.237864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.237869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.237874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.237883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.237891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.237896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.237904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.237910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.237914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.237923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.237953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:56.238078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342828203975274:4980] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:56.243579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342828203975274:4980] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:56.243594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342828203975274:4980] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:56.243598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342828203975274:4980] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:56.243605Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1980 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:56.243956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:56.244918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:56.244928Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:56.244930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:56.244932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:56.244938Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1979 [2:7559342827842141948:10771]
2025-10-09T21:50:56.244947Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:56.244960Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:56.244962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:56.244963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:56.244965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:56.244979Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:56.244986Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:56.244992Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:56.245044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.245049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.245050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.245052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:56.245099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:56.245105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.245112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.245121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.245136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:56.245156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975253:2432], now have 1 active actors on pipe
2025-10-09T21:50:56.245164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975314:4980], now have 1 active actors on pipe
2025-10-09T21:50:56.245169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342827842141943:10771]
2025-10-09T21:50:56.245171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342827842141943:10771]
2025-10-09T21:50:56.245172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342827842141943:10771]
2025-10-09T21:50:56.245181Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:56.245192Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:56.245201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:56.245202Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:56.245204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:56.245206Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:56.245208Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:56.245220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342827842141943:10771]
2025-10-09T21:50:56.245211Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1979
2025-10-09T21:50:56.245226Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:56.245235Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:56.245216Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1979
2025-10-09T21:50:56.245248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.245251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.245276Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|80276348-6017d494-bd4d04d-8139ef38_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:56.245285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:56.245288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:56.245289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:56.245296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:56.245303Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:56.245321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:56.245485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.245497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.245518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:56.245612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1980 partition: 0 MaxSeqNo: 25 sessionId: src_id|80276348-6017d494-bd4d04d-8139ef38_0
2025-10-09T21:50:56.245932Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dc42ac29-1d06d87-41e3bf62-30759302_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|80276348-6017d494-bd4d04d-8139ef38_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:56.248259Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80276348-6017d494-bd4d04d-8139ef38_0] Write session: try to update token
2025-10-09T21:50:56.248276Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80276348-6017d494-bd4d04d-8139ef38_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:56.258956Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:56.258976Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:56.258979Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:56.258981Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:56.258990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342827842141943:10771]
2025-10-09T21:50:56.258995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342827842141943:10771]
2025-10-09T21:50:56.258999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342827842141943:10771]
2025-10-09T21:50:56.259040Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:56.259043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:56.259046Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:56.259049Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:56.259051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.259054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:56.259188Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:56.260656Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1980 sessionId: src_id|80276348-6017d494-bd4d04d-8139ef38_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:56.263021Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:56.327158Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342828203975327:4041]
2025-10-09T21:50:56.327392Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 1979, pipe: [1:7559342828203975327:4041]
2025-10-09T21:50:56.405521Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:56.405795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.405809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.405847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975327:4041], now have 1 active actors on pipe
2025-10-09T21:50:56.406025Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:56.406055Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:56.406059Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342828203975327:4041]
2025-10-09T21:50:56.406081Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:56.406114Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1980 reason: error at write: Disconnected sessionId: src_id|80276348-6017d494-bd4d04d-8139ef38_0
2025-10-09T21:50:56.406293Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1980 sessionId: src_id|80276348-6017d494-bd4d04d-8139ef38_0 is DEAD
2025-10-09T21:50:56.406440Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80276348-6017d494-bd4d04d-8139ef38_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:56.406451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80276348-6017d494-bd4d04d-8139ef38_0] Write session will restart in 0.006955s
2025-10-09T21:50:56.406476Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80276348-6017d494-bd4d04d-8139ef38_0] Write session: Do CDS request
2025-10-09T21:50:56.406482Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80276348-6017d494-bd4d04d-8139ef38_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:56.406935Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:56.413230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:56.414465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80276348-6017d494-bd4d04d-8139ef38_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:56.414515Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:56.414530Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1981
2025-10-09T21:50:56.414595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:56.414643Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1981 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:56.414673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1981 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:56.414682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1981 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:56.414686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:56.414761Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:56.414770Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:56.414781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:56.414824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:56.414855Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342827842142047:10785]
2025-10-09T21:50:56.415064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1981 sessionId:  describe result for acl check
2025-10-09T21:50:56.415094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:56.415100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:56.415102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:56.415107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342828203975377:5007] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:56.415252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:56.415491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:56.415526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:56.415672Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342828203975377:5007] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:56.415747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:56.415788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.415794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.415795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.415796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.415797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.415798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.415800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.415801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.415802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.415806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.415815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.415818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.415822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.415825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.415828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.415831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.415834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.415838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.415843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.415851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.415856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.415861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.415865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.415870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.415873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.415878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.415882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.415887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.415891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.415896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.415901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.415905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.415907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.415910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.415913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.415916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.415932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:56.417473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342828203975377:5007] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:56.421285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342828203975377:5007] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:56.421301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342828203975377:5007] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:56.421304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342828203975377:5007] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:56.421313Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1981 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:56.421650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:56.422685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:56.422698Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:56.422701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:56.422704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:56.422712Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1980 [2:7559342827842142047:10785]
2025-10-09T21:50:56.422724Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:56.422743Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:56.422746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:56.422749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:56.422753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:56.422769Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:56.422779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:56.422789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:56.422843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.422849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.422850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.422853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:56.422915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:56.422923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.422933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.422947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.422965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:56.422991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975413:5007], now have 1 active actors on pipe
2025-10-09T21:50:56.422994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975354:2432], now have 1 active actors on pipe
2025-10-09T21:50:56.423000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342827842142042:10785]
2025-10-09T21:50:56.423001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342827842142042:10785]
2025-10-09T21:50:56.423004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342827842142042:10785]
2025-10-09T21:50:56.423016Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:56.423026Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:56.423020Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1980
2025-10-09T21:50:56.423020Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1980
2025-10-09T21:50:56.423034Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:56.423037Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:56.423039Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:56.423042Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:56.423044Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:56.423060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342827842142042:10785]
2025-10-09T21:50:56.423067Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:56.423077Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:56.423083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.423087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.423127Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8164d32d-41e4d920-fdb751eb-7a98f698_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:56.423140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:56.423148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:56.423150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:56.423159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:56.423166Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:56.423180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:56.423297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.423304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.423315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:56.423385Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1981 partition: 0 MaxSeqNo: 25 sessionId: src_id|8164d32d-41e4d920-fdb751eb-7a98f698_0
2025-10-09T21:50:56.423685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80276348-6017d494-bd4d04d-8139ef38_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8164d32d-41e4d920-fdb751eb-7a98f698_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:56.426047Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8164d32d-41e4d920-fdb751eb-7a98f698_0] Write session: try to update token
2025-10-09T21:50:56.426065Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8164d32d-41e4d920-fdb751eb-7a98f698_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:56.436583Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:56.436601Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:56.436604Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:56.436606Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:56.436617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342827842142042:10785]
2025-10-09T21:50:56.436621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342827842142042:10785]
2025-10-09T21:50:56.436624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342827842142042:10785]
2025-10-09T21:50:56.436662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:56.436663Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:56.436668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.436669Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:56.436669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:56.436673Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:56.436785Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:56.438577Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1981 sessionId: src_id|8164d32d-41e4d920-fdb751eb-7a98f698_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:56.440752Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:56.506365Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342828203975420:4041]
2025-10-09T21:50:56.506623Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 1980, pipe: [1:7559342828203975420:4041]
2025-10-09T21:50:56.581662Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:56.582013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.582054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.582101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975420:4041], now have 1 active actors on pipe
2025-10-09T21:50:56.582300Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:56.582308Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:56.582329Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:56.582390Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1981 reason: error at write: Disconnected sessionId: src_id|8164d32d-41e4d920-fdb751eb-7a98f698_0
2025-10-09T21:50:56.582398Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342828203975420:4041]
2025-10-09T21:50:56.582424Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:56.582563Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1981 sessionId: src_id|8164d32d-41e4d920-fdb751eb-7a98f698_0 is DEAD
2025-10-09T21:50:56.582725Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8164d32d-41e4d920-fdb751eb-7a98f698_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:56.582734Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8164d32d-41e4d920-fdb751eb-7a98f698_0] Write session will restart in 0.008792s
2025-10-09T21:50:56.582754Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8164d32d-41e4d920-fdb751eb-7a98f698_0] Write session: Do CDS request
2025-10-09T21:50:56.582759Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8164d32d-41e4d920-fdb751eb-7a98f698_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:56.589468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:56.590665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:56.590822Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:56.590829Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:56.590838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:56.590875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:56.590902Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342827842142108:10787]
2025-10-09T21:50:56.591294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:56.591509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:56.591539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:56.591739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:56.591776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.591782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.591782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.591783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.591784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.591786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.591786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.591787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.591787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.591791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.591797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.591799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.591801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.591803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.591805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.591807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.591809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.591810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.591815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.591819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.591823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.591827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.591829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.591832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.591834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.591837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.591840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.591843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.591845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.591848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.591851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.591854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.591856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.591858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.591861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.591863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.591878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:56.592388Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8164d32d-41e4d920-fdb751eb-7a98f698_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:56.592483Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:56.592497Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1982
2025-10-09T21:50:56.592601Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1982 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:56.592629Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1982 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:56.592634Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1982 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:56.592637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:56.593020Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1982 sessionId:  describe result for acl check
2025-10-09T21:50:56.593049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:56.593057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:56.593058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:56.593063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342828203975451:5020] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:56.593587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342828203975451:5020] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:56.595512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342828203975451:5020] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:56.597375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342828203975451:5020] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:56.597387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342828203975451:5020] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:56.597391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342828203975451:5020] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:56.597395Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1982 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:56.597739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:56.598763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:56.598779Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:56.598782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:56.598785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:56.598791Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1981 [2:7559342827842142108:10787]
2025-10-09T21:50:56.598802Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:56.598818Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:56.598821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:56.598824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:56.598826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:56.598841Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:56.598853Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:56.598860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:56.598915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.598923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.598925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.598927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:56.598987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:56.598995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.599007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.599018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.599038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:56.599061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975423:2432], now have 1 active actors on pipe
2025-10-09T21:50:56.599068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975486:5020], now have 1 active actors on pipe
2025-10-09T21:50:56.599075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342827842142103:10787]
2025-10-09T21:50:56.599082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342827842142103:10787]
2025-10-09T21:50:56.599084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342827842142103:10787]
2025-10-09T21:50:56.599070Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1981
2025-10-09T21:50:56.599092Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:56.599104Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:56.599106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:56.599075Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1981
2025-10-09T21:50:56.599108Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:56.599110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:56.599124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:56.599131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:56.599151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.599159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.599187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342827842142103:10787]
2025-10-09T21:50:56.599198Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:56.599205Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:56.599211Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3656815-879c40a7-a32d09ad-2a808537_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:56.599216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:56.599218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:56.599220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:56.599227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:56.599234Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:56.599268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:56.599382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.599389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.599412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:56.599473Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1982 partition: 0 MaxSeqNo: 25 sessionId: src_id|3656815-879c40a7-a32d09ad-2a808537_0
2025-10-09T21:50:56.599715Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8164d32d-41e4d920-fdb751eb-7a98f698_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3656815-879c40a7-a32d09ad-2a808537_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:56.601898Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3656815-879c40a7-a32d09ad-2a808537_0] Write session: try to update token
2025-10-09T21:50:56.601912Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3656815-879c40a7-a32d09ad-2a808537_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:56.612378Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:56.612395Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:56.612396Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:56.612398Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:56.612405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342827842142103:10787]
2025-10-09T21:50:56.612408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342827842142103:10787]
2025-10-09T21:50:56.612414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342827842142103:10787]
2025-10-09T21:50:56.612448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:56.612455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.612456Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:56.612457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:56.612460Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:56.612463Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:56.612583Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:56.613913Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1982 sessionId: src_id|3656815-879c40a7-a32d09ad-2a808537_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:56.616103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:56.682445Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342828203975493:4041]
2025-10-09T21:50:56.682775Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 1981, pipe: [1:7559342828203975493:4041]
2025-10-09T21:50:56.749472Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:56.749773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.749795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.749839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975493:4041], now have 1 active actors on pipe
2025-10-09T21:50:56.750007Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:56.750020Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:56.750046Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:56.750050Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:56.750090Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342828203975493:4041]
2025-10-09T21:50:56.750108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1982 reason: error at write: Disconnected sessionId: src_id|3656815-879c40a7-a32d09ad-2a808537_0
2025-10-09T21:50:56.750268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1982 sessionId: src_id|3656815-879c40a7-a32d09ad-2a808537_0 is DEAD
2025-10-09T21:50:56.750433Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3656815-879c40a7-a32d09ad-2a808537_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:56.750443Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3656815-879c40a7-a32d09ad-2a808537_0] Write session will restart in 0.006575s
2025-10-09T21:50:56.750469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3656815-879c40a7-a32d09ad-2a808537_0] Write session: Do CDS request
2025-10-09T21:50:56.750477Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3656815-879c40a7-a32d09ad-2a808537_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:56.757103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:56.758340Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3656815-879c40a7-a32d09ad-2a808537_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:56.758398Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:56.758413Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1983
2025-10-09T21:50:56.758531Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1983 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:56.758553Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1983 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:56.758556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:56.758561Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1983 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:56.758564Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:56.758753Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:56.758763Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:56.758777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:56.758838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:56.758881Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342827842142174:10789]
2025-10-09T21:50:56.758954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1983 sessionId:  describe result for acl check
2025-10-09T21:50:56.758984Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:56.758989Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:56.758991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:56.758995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342828203975523:5025] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:56.759363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:56.759432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342828203975523:5025] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:56.759649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:56.759700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:56.759958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:56.759999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.760004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.760005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.760005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.760008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.760009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.760010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.760010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.760011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.760015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.760021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.760023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.760025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.760027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.760029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.760031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.760033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.760036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.760041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.760046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.760050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.760053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.760056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.760059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.760061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.760064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.760067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.760071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.760073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.760076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.760079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.760081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.760083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.760086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.760089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.760093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.760109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:56.761039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342828203975523:5025] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:56.765896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342828203975523:5025] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:56.765912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342828203975523:5025] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:56.765916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342828203975523:5025] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:56.765925Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1983 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:56.766271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:56.767268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:56.767282Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:56.767284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:56.767287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:56.767292Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1982 [2:7559342827842142174:10789]
2025-10-09T21:50:56.767308Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:56.767326Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:56.767333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:56.767336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:56.767339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:56.767353Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:56.767357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:56.767363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:56.767411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.767417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.767418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.767420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:56.767469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:56.767474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.767483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.767494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.767515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:56.767538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975495:2432], now have 1 active actors on pipe
2025-10-09T21:50:56.767544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975562:5025], now have 1 active actors on pipe
2025-10-09T21:50:56.767549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342827842142165:10789]
2025-10-09T21:50:56.767551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342827842142165:10789]
2025-10-09T21:50:56.767554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342827842142165:10789]
2025-10-09T21:50:56.767564Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:56.767576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:56.767581Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:56.767584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:56.767586Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:56.767588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:56.767590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:56.767595Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1982
2025-10-09T21:50:56.767612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342827842142165:10789]
2025-10-09T21:50:56.767607Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1982
2025-10-09T21:50:56.767618Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:56.767623Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:56.767643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.767649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.767722Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|54cd68a1-afd6305-d3918b86-a5537493_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:56.767733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:56.767735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:56.767738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:56.767745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:56.767753Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:56.767767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:56.767897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.767903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.767913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:56.768017Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1983 partition: 0 MaxSeqNo: 25 sessionId: src_id|54cd68a1-afd6305-d3918b86-a5537493_0
2025-10-09T21:50:56.768310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3656815-879c40a7-a32d09ad-2a808537_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|54cd68a1-afd6305-d3918b86-a5537493_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:56.770697Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54cd68a1-afd6305-d3918b86-a5537493_0] Write session: try to update token
2025-10-09T21:50:56.770716Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54cd68a1-afd6305-d3918b86-a5537493_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:56.782061Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:56.782076Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:56.782079Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:56.782081Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:56.782093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342827842142165:10789]
2025-10-09T21:50:56.782100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342827842142165:10789]
2025-10-09T21:50:56.782105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342827842142165:10789]
2025-10-09T21:50:56.782151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:56.782152Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:56.782157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.782158Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:56.782160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:56.782161Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:56.782293Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:56.783413Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1983 sessionId: src_id|54cd68a1-afd6305-d3918b86-a5537493_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:56.785704Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:56.850420Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342828203975569:4041]
2025-10-09T21:50:56.850679Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 1982, pipe: [1:7559342828203975569:4041]
2025-10-09T21:50:56.939998Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:56.940265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.940276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.940313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975569:4041], now have 1 active actors on pipe
2025-10-09T21:50:56.940483Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:56.940556Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:56.940594Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342828203975569:4041]
2025-10-09T21:50:56.940620Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:56.940682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1983 reason: status is not ok: tablet will be restarted right now sessionId: src_id|54cd68a1-afd6305-d3918b86-a5537493_0
2025-10-09T21:50:56.940832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1983 sessionId: src_id|54cd68a1-afd6305-d3918b86-a5537493_0 is DEAD
2025-10-09T21:50:56.940999Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54cd68a1-afd6305-d3918b86-a5537493_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:50:56.941009Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54cd68a1-afd6305-d3918b86-a5537493_0] Write session will restart in 0.006081s
2025-10-09T21:50:56.941042Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54cd68a1-afd6305-d3918b86-a5537493_0] Write session: Do CDS request
2025-10-09T21:50:56.941048Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54cd68a1-afd6305-d3918b86-a5537493_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:56.947850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:56.948421Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54cd68a1-afd6305-d3918b86-a5537493_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:56.948490Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:56.948505Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1984
2025-10-09T21:50:56.948621Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1984 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:56.948664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1984 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:56.948672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1984 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:56.948675Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:56.949099Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1984 sessionId:  describe result for acl check
2025-10-09T21:50:56.949130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:56.949136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:56.949138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:56.949152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342828203975600:5046] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:56.949316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:56.949486Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:56.949494Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:56.949504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:56.949544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:56.949572Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342827842142237:10793]
2025-10-09T21:50:56.949692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342828203975600:5046] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:56.949956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:56.950261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:56.950315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:56.950645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:56.950718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.950725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.950727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.950728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.950730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.950731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.950732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.950734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.950735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.950741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.950750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.950755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.950758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.950762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.950766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.950769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.950772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.950776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.950783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.950791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:56.950797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.950802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:56.950807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.950813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:56.950817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.950821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:56.950825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.950829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:56.950834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.950838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:56.950851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.950856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.950860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.950869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.950882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.950891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.950917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:56.952020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342828203975600:5046] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:56.956862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342828203975600:5046] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:56.956883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342828203975600:5046] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:56.956887Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342828203975600:5046] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:56.956898Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1984 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:56.957224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:56.958216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:56.958229Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:56.958232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:56.958236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:56.958242Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1983 [2:7559342827842142237:10793]
2025-10-09T21:50:56.958255Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:56.958271Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:56.958278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:56.958281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:56.958286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:56.958299Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:56.958306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:56.958312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:56.958353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.958358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.958360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.958361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:56.958409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:56.958415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.958422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:56.958428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:56.958436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:56.958450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342827842142232:10793]
2025-10-09T21:50:56.958451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342827842142232:10793]
2025-10-09T21:50:56.958452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342827842142232:10793]
2025-10-09T21:50:56.958458Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:56.958470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975574:2432], now have 1 active actors on pipe
2025-10-09T21:50:56.958470Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:56.958473Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:56.958474Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:56.958475Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:56.958482Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:56.958485Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:56.958489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342828203975639:5046], now have 1 active actors on pipe
2025-10-09T21:50:56.958510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342827842142232:10793]
2025-10-09T21:50:56.958524Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:56.958542Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:56.958545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1983
2025-10-09T21:50:56.958547Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1983
2025-10-09T21:50:56.958611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.958618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.958636Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2b1890a2-c26834bf-c77260e3-b79a6f4d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:56.958645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:56.958648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:56.958649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:56.958658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:56.958669Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:56.958686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:56.958850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:56.958864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:56.958887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:56.958976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1984 partition: 0 MaxSeqNo: 25 sessionId: src_id|2b1890a2-c26834bf-c77260e3-b79a6f4d_0
2025-10-09T21:50:56.959274Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54cd68a1-afd6305-d3918b86-a5537493_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2b1890a2-c26834bf-c77260e3-b79a6f4d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:56.961440Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b1890a2-c26834bf-c77260e3-b79a6f4d_0] Write session: try to update token
2025-10-09T21:50:56.961457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b1890a2-c26834bf-c77260e3-b79a6f4d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:56.971647Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:56.971666Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:56.971669Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:56.971670Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:56.971681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342827842142232:10793]
2025-10-09T21:50:56.971686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342827842142232:10793]
2025-10-09T21:50:56.971689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342827842142232:10793]
2025-10-09T21:50:56.971732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:56.971738Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:56.971743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:56.971744Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:56.971746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:56.971747Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:56.971874Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:56.972900Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1984 sessionId: src_id|2b1890a2-c26834bf-c77260e3-b79a6f4d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:56.975111Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:57.040901Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342832498942942:4041]
2025-10-09T21:50:57.041149Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 1983, pipe: [1:7559342832498942942:4041]
2025-10-09T21:50:57.117512Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:57.117817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.117830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.117867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342832498942942:4041], now have 1 active actors on pipe
2025-10-09T21:50:57.118099Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:57.118117Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:57.118123Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342832498942942:4041]
2025-10-09T21:50:57.118146Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:57.118149Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:57.118208Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1984 reason: error at write: Disconnected sessionId: src_id|2b1890a2-c26834bf-c77260e3-b79a6f4d_0
2025-10-09T21:50:57.118393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1984 sessionId: src_id|2b1890a2-c26834bf-c77260e3-b79a6f4d_0 is DEAD
2025-10-09T21:50:57.118531Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b1890a2-c26834bf-c77260e3-b79a6f4d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:57.118541Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b1890a2-c26834bf-c77260e3-b79a6f4d_0] Write session will restart in 0.005202s
2025-10-09T21:50:57.118560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b1890a2-c26834bf-c77260e3-b79a6f4d_0] Write session: Do CDS request
2025-10-09T21:50:57.118564Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b1890a2-c26834bf-c77260e3-b79a6f4d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:57.124542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b1890a2-c26834bf-c77260e3-b79a6f4d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:57.124622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:57.124648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1985
2025-10-09T21:50:57.124788Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1985 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:57.124826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1985 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:57.124834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1985 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:57.124843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:57.125270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1985 sessionId:  describe result for acl check
2025-10-09T21:50:57.125313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:57.125320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:57.125322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:57.125329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342832498942970:5051] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:57.125882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:57.125905Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342832498942970:5051] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:57.127513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:57.127688Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:57.127698Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:57.127708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:57.127749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:57.127783Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342832137109598:10795]
2025-10-09T21:50:57.127805Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342832498942970:5051] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:57.128290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:57.128536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:57.128564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:57.128805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:57.128842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.128848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.128849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.128849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.128850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.128851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.128851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.128852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.128852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.128856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.128863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.128865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.128867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.128869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.128871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.128874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.128876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.128878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.128882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.128887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.128892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.128895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.128898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.128901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.128904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.128907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.128910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.128913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.128915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.128919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.128921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.128925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.128931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.128936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.128941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.128945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.128970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:57.134223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342832498942970:5051] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:57.134245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342832498942970:5051] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:57.134252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342832498942970:5051] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:57.134263Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1985 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:57.135526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:57.136502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:57.136515Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:57.136517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:57.136521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:57.136527Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1984 [2:7559342832137109598:10795]
2025-10-09T21:50:57.136537Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:57.136556Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:57.136558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:57.136560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:57.136562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:57.136576Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:57.136584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:57.136590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:57.136641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.136646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.136648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.136651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:57.136707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:57.136712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.136720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.136727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.136736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:57.136752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342832498942945:2432], now have 1 active actors on pipe
2025-10-09T21:50:57.136763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342832137109593:10795]
2025-10-09T21:50:57.136765Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342832137109593:10795]
2025-10-09T21:50:57.136767Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342832137109593:10795]
2025-10-09T21:50:57.136776Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:57.136786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:57.136795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:57.136797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:57.136799Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:57.136801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342832498943011:5051], now have 1 active actors on pipe
2025-10-09T21:50:57.136802Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:57.136804Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:57.136829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342832137109593:10795]
2025-10-09T21:50:57.136832Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:57.136842Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:57.136842Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1984
2025-10-09T21:50:57.136849Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1984
2025-10-09T21:50:57.136898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.136907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.136929Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3adb3be5-be5a41-84ac5f3c-997669b1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:57.136938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:57.136940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:57.136942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:57.136949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:57.136957Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:57.136972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:57.137103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.137106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.137116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:57.137202Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1985 partition: 0 MaxSeqNo: 25 sessionId: src_id|3adb3be5-be5a41-84ac5f3c-997669b1_0
2025-10-09T21:50:57.137566Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b1890a2-c26834bf-c77260e3-b79a6f4d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3adb3be5-be5a41-84ac5f3c-997669b1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:57.139978Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3adb3be5-be5a41-84ac5f3c-997669b1_0] Write session: try to update token
2025-10-09T21:50:57.139994Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3adb3be5-be5a41-84ac5f3c-997669b1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:57.153667Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:57.153680Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:57.153689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342832137109593:10795]
2025-10-09T21:50:57.153691Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1985 sessionId: src_id|3adb3be5-be5a41-84ac5f3c-997669b1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:57.153728Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:57.153734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:57.153892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:57.154042Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:57.154179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:57.154317Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:57.154508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:57.154553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:57.156161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:57.156337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.156352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.156388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:57.156475Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:57.161472Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:57.161486Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:57.161488Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:57.161489Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:57.161497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342832137109593:10795]
2025-10-09T21:50:57.161500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342832137109593:10795]
2025-10-09T21:50:57.161503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342832137109593:10795]
2025-10-09T21:50:57.161535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:57.161541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.161544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:57.161565Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:57.161573Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:57.161579Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:57.161676Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:57.218056Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342832498943021:4041]
2025-10-09T21:50:57.218290Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 1984, pipe: [1:7559342832498943021:4041]
2025-10-09T21:50:57.305908Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:57.312170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.312194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.312331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.312387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.312443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.312497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.312539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.312548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.312606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.312657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.312711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.312765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.312802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.312807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.312867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.312921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.312974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.313025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.313063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.313069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.313124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.313175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.313226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.313280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.313317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.313322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.313375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.313443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.313506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.313567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.313611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.313618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.313680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.313741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.313792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.313844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.313887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.313894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.313955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.314009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.314074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.314124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.314160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.314165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.314228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.314278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.314328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.314378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.314415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.314420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.314481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.314529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.314580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.314632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.314668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.314673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.314734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.314782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.314829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.314881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.314930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.314935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.314981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342832498943021:4041], now have 1 active actors on pipe
2025-10-09T21:50:57.315306Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342832498943021:4041]
2025-10-09T21:50:57.315345Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:57.315447Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:57.315462Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:57.315494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1985 reason: error at write: Disconnected sessionId: src_id|3adb3be5-be5a41-84ac5f3c-997669b1_0
2025-10-09T21:50:57.315704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1985 sessionId: src_id|3adb3be5-be5a41-84ac5f3c-997669b1_0 is DEAD
2025-10-09T21:50:57.315838Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3adb3be5-be5a41-84ac5f3c-997669b1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:57.315850Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3adb3be5-be5a41-84ac5f3c-997669b1_0] Write session will restart in 0.008637s
2025-10-09T21:50:57.315877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3adb3be5-be5a41-84ac5f3c-997669b1_0] Write session: Do CDS request
2025-10-09T21:50:57.315892Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3adb3be5-be5a41-84ac5f3c-997669b1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:57.315906Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:57.322269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:57.323721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:57.323888Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:57.323898Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:57.323913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:57.323957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:57.323999Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342832137109664:10797]
2025-10-09T21:50:57.324365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:57.324666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:57.324736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:57.325005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:57.325064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.325071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.325073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.325074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.325075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.325076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.325078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.325079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.325080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.325086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.325099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.325103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.325107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.325111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.325115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.325119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.325122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.325126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.325133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.325145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.325156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.325165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.325171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.325176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.325184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.325189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.325193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.325195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.325198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.325201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.325205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.325208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.325214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.325217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.325219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.325222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.325238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:57.325473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3adb3be5-be5a41-84ac5f3c-997669b1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:57.325574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:57.325588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1986
2025-10-09T21:50:57.325712Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1986 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:57.325739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1986 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:57.325749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1986 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:57.325752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:57.326145Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1986 sessionId:  describe result for acl check
2025-10-09T21:50:57.326177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:57.326180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:57.326182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:57.326187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342832498943070:5055] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:57.326701Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342832498943070:5055] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:57.329502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342832498943070:5055] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:57.330978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:57.331692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342832498943070:5055] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:57.331706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342832498943070:5055] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:57.331709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342832498943070:5055] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:57.331716Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1986 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:57.332019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:57.332035Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:57.332038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:57.332040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:57.332045Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1985 [2:7559342832137109664:10797]
2025-10-09T21:50:57.332054Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:57.332066Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:57.332068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:57.332070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:57.332071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:57.332082Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:57.332085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:57.332090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:57.332127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.332131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.332132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.332134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:57.332178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:57.332183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.332190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.332196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.332205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:57.332221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342832137109659:10797]
2025-10-09T21:50:57.332226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342832137109659:10797]
2025-10-09T21:50:57.332227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342832137109659:10797]
2025-10-09T21:50:57.332234Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:57.332244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342832498943103:5055], now have 1 active actors on pipe
2025-10-09T21:50:57.332243Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:57.332246Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:57.332248Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:57.332250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342832498943029:2432], now have 1 active actors on pipe
2025-10-09T21:50:57.332256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:57.332269Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:57.332271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:57.332291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342832137109659:10797]
2025-10-09T21:50:57.332304Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:57.332304Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1985
2025-10-09T21:50:57.332327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:57.332305Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1985
2025-10-09T21:50:57.332361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.332370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.332392Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f5111f45-c2ac7fec-9afc1128-4c4d2c4f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:57.332402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:57.332405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:57.332406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:57.332412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:57.332422Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:57.332436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:57.332582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.332588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.332598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:57.332666Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1986 partition: 0 MaxSeqNo: 25 sessionId: src_id|f5111f45-c2ac7fec-9afc1128-4c4d2c4f_0
2025-10-09T21:50:57.332906Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3adb3be5-be5a41-84ac5f3c-997669b1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f5111f45-c2ac7fec-9afc1128-4c4d2c4f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:57.335319Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5111f45-c2ac7fec-9afc1128-4c4d2c4f_0] Write session: try to update token
2025-10-09T21:50:57.335336Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5111f45-c2ac7fec-9afc1128-4c4d2c4f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:57.343795Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:57.343816Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:57.343826Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342832137109659:10797]
2025-10-09T21:50:57.343870Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:57.343873Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:57.344028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:57.344156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:57.344295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:57.344438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:57.344650Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:57.344700Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:57.348191Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1986 sessionId: src_id|f5111f45-c2ac7fec-9afc1128-4c4d2c4f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:57.350458Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:57.350629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.350651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.350694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:57.350788Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:57.356490Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:57.356506Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:57.356508Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:57.356510Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:57.356519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342832137109659:10797]
2025-10-09T21:50:57.356523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342832137109659:10797]
2025-10-09T21:50:57.356527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342832137109659:10797]
2025-10-09T21:50:57.356567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:57.356577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.356580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:57.356564Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:57.356587Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:57.356603Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:57.356708Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:57.415681Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342832498943125:4041]
2025-10-09T21:50:57.415970Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 1985, pipe: [1:7559342832498943125:4041]
2025-10-09T21:50:57.496982Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:57.503381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.503406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.503554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.503607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.503660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.503715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.503757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.503765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.503824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.503881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.503942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.504000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.504038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.504046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.504096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.504147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.504203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.504255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.504293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.504300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.504353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.504406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.504461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.504518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.504562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.504570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.504627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.504677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.504734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.504786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.504824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.504830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.504882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.504934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.504987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.505039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.505076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.505081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.505130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.505179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.505232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.505282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.505329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.505340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.505399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.505461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.505510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.505558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.505597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.505600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.505658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.505714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.505762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.505818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.505853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.505859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.505909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.505957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.506005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.506080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.506119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.506125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.506163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342832498943125:4041], now have 1 active actors on pipe
2025-10-09T21:50:57.506361Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:57.506380Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:57.506409Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1986 reason: error at write: Disconnected sessionId: src_id|f5111f45-c2ac7fec-9afc1128-4c4d2c4f_0
2025-10-09T21:50:57.506426Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342832498943125:4041]
2025-10-09T21:50:57.506470Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:57.506604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1986 sessionId: src_id|f5111f45-c2ac7fec-9afc1128-4c4d2c4f_0 is DEAD
2025-10-09T21:50:57.506791Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5111f45-c2ac7fec-9afc1128-4c4d2c4f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:57.506802Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5111f45-c2ac7fec-9afc1128-4c4d2c4f_0] Write session will restart in 0.009480s
2025-10-09T21:50:57.506828Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5111f45-c2ac7fec-9afc1128-4c4d2c4f_0] Write session: Do CDS request
2025-10-09T21:50:57.506835Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5111f45-c2ac7fec-9afc1128-4c4d2c4f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:57.506944Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:57.513802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:57.515295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:57.515451Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:57.515459Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:57.515467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:57.515507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:57.515534Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342832137109767:10811]
2025-10-09T21:50:57.515934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:57.516170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:57.516198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:57.516442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:57.516477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.516481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.516481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.516482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.516483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.516484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.516486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.516486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.516487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.516491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.516498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.516500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.516502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.516504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.516506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.516508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.516510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.516511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.516517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.516521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.516525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.516528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.516530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.516534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.516536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.516539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.516542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.516544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.516547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.516550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.516553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.516557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.516559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.516562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.516564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.516567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.516583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:57.517402Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f5111f45-c2ac7fec-9afc1128-4c4d2c4f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:57.517507Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:57.517522Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1987
2025-10-09T21:50:57.517644Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1987 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:57.517677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1987 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:57.517689Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1987 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:57.517693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:57.518075Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1987 sessionId:  describe result for acl check
2025-10-09T21:50:57.518107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:57.518113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:57.518115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:57.518121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342832498943170:5088] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:57.518608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342832498943170:5088] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:57.520360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342832498943170:5088] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:57.522343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:57.522733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342832498943170:5088] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:57.522745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342832498943170:5088] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:57.522748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342832498943170:5088] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:57.522753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1987 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:57.523404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:57.523417Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:57.523419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:57.523423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:57.523435Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1986 [2:7559342832137109767:10811]
2025-10-09T21:50:57.523444Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:57.523457Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:57.523463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:57.523466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:57.523469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:57.523486Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:57.523496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:57.523502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:57.523542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.523546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.523547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.523549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:57.523598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:57.523607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.523614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.523619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.523629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:57.523642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342832498943136:2432], now have 1 active actors on pipe
2025-10-09T21:50:57.523644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342832498943204:5088], now have 1 active actors on pipe
2025-10-09T21:50:57.523650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342832137109762:10811]
2025-10-09T21:50:57.523651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342832137109762:10811]
2025-10-09T21:50:57.523652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342832137109762:10811]
2025-10-09T21:50:57.523661Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:57.523671Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:57.523673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:57.523673Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:57.523674Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:57.523681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:57.523683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:57.523700Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342832137109762:10811]
2025-10-09T21:50:57.523709Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:57.523713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:57.523731Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1986
2025-10-09T21:50:57.523735Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1986
2025-10-09T21:50:57.523805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.523813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.523835Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|55fc8719-ed7c8541-162d6fcf-31def4b1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:57.523841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:57.523844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:57.523845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:57.523851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:57.523859Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:57.523869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:57.524025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.524031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.524040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:57.524127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1987 partition: 0 MaxSeqNo: 25 sessionId: src_id|55fc8719-ed7c8541-162d6fcf-31def4b1_0
2025-10-09T21:50:57.524375Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f5111f45-c2ac7fec-9afc1128-4c4d2c4f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|55fc8719-ed7c8541-162d6fcf-31def4b1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:57.526828Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55fc8719-ed7c8541-162d6fcf-31def4b1_0] Write session: try to update token
2025-10-09T21:50:57.526843Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55fc8719-ed7c8541-162d6fcf-31def4b1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:57.535757Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:57.535770Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:57.535782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342832137109762:10811]
2025-10-09T21:50:57.535819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:57.535834Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:57.535970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:57.536094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:57.536214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:57.536335Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:57.536550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:57.536583Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:57.539652Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1987 sessionId: src_id|55fc8719-ed7c8541-162d6fcf-31def4b1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:57.541808Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:57.541962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.541977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.542004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:57.542110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:57.547367Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:57.547381Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:57.547386Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:57.547388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:57.547398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342832137109762:10811]
2025-10-09T21:50:57.547403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342832137109762:10811]
2025-10-09T21:50:57.547406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342832137109762:10811]
2025-10-09T21:50:57.547441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:57.547449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.547449Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:57.547450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:57.547455Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:57.547458Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:57.547565Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:57.606646Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342832498943212:4041]
2025-10-09T21:50:57.606913Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 1986, pipe: [1:7559342832498943212:4041]
2025-10-09T21:50:57.684350Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:57.691465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.691489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.691653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.691713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.691771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.691835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.691941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.691950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.692009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.692066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.692123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.692188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.692229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.692235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.692298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.692352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.692404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.692463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.692499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.692509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.692571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.692636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.692704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.692762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.692804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.692811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.692877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.692935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.692995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.693052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.693093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.693104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.693170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.693229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.693288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.693350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.693397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.693405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.693477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.693531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.693589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.693647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.693689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.693697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.693755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.693810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.693865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.693930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.693972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.693980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.694056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.694110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.694175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.694236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.694281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.694290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.694351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:57.694407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:57.694462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:57.694524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:57.694563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:57.694570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:57.694618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342832498943212:4041], now have 1 active actors on pipe
2025-10-09T21:50:57.694903Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:57.694907Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342832498943212:4041]
2025-10-09T21:50:57.694934Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:57.694936Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:57.694967Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1987 reason: error at write: Disconnected sessionId: src_id|55fc8719-ed7c8541-162d6fcf-31def4b1_0
2025-10-09T21:50:57.695159Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1987 sessionId: src_id|55fc8719-ed7c8541-162d6fcf-31def4b1_0 is DEAD
2025-10-09T21:50:57.695319Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55fc8719-ed7c8541-162d6fcf-31def4b1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:57.695330Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55fc8719-ed7c8541-162d6fcf-31def4b1_0] Write session will restart in 0.009474s
2025-10-09T21:50:57.695356Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55fc8719-ed7c8541-162d6fcf-31def4b1_0] Write session: Do CDS request
2025-10-09T21:50:57.695362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55fc8719-ed7c8541-162d6fcf-31def4b1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:57.695554Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:57.702166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:57.703495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:57.703646Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:57.703652Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:57.703660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:57.703702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:57.703740Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342832137109834:10813]
2025-10-09T21:50:57.704121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:57.704348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:57.704376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:57.704638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:57.704676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.704680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.704681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.704681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.704682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.704683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.704683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.704684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.704685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.704688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.704694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.704697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.704699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.704701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.704703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.704707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.704711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.704715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.704724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.704731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.704737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.704742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.704746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.704751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.704755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.704759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.704763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.704768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.704772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.704777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.704781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.704786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.704790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.704795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.704799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.704804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.704833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:57.705517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55fc8719-ed7c8541-162d6fcf-31def4b1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:57.705597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:57.705613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1988
2025-10-09T21:50:57.705751Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1988 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:57.705780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1988 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:57.705785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1988 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:57.705788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:57.706246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1988 sessionId:  describe result for acl check
2025-10-09T21:50:57.706280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:57.706286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:57.706288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:57.706293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342832498943248:5098] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:57.706828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342832498943248:5098] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:57.709108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342832498943248:5098] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:57.710775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:57.711787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342832498943248:5098] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:57.711802Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342832498943248:5098] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:57.711806Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342832498943248:5098] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:57.711815Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1988 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:57.711814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:57.711829Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:57.711831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:57.711835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:57.711842Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1987 [2:7559342832137109834:10813]
2025-10-09T21:50:57.711854Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:57.711870Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:57.711878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:57.711882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:57.711884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:57.711903Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:57.711912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:57.711919Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:57.711970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.711975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.711976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.711978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:57.712034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:57.712041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.712052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.712069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.712085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:57.712108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342832498943282:5098], now have 1 active actors on pipe
2025-10-09T21:50:57.712117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342832498943214:2432], now have 1 active actors on pipe
2025-10-09T21:50:57.712124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342832137109829:10813]
2025-10-09T21:50:57.712130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342832137109829:10813]
2025-10-09T21:50:57.712132Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342832137109829:10813]
2025-10-09T21:50:57.712139Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:57.712125Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1987
2025-10-09T21:50:57.712127Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1987
2025-10-09T21:50:57.712152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:57.712153Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:57.712164Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:57.712166Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:57.712171Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:57.712174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:57.712208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.712216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.712225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342832137109829:10813]
2025-10-09T21:50:57.712230Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:57.712237Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:57.712254Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4db395ad-5c81f8a9-26a415a3-98420153_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:57.712262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:57.712266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:57.712267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:57.712274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:57.712281Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:57.712294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:57.712428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.712437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.712451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:57.712518Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1988 partition: 0 MaxSeqNo: 25 sessionId: src_id|4db395ad-5c81f8a9-26a415a3-98420153_0
2025-10-09T21:50:57.712821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55fc8719-ed7c8541-162d6fcf-31def4b1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4db395ad-5c81f8a9-26a415a3-98420153_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:57.715276Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4db395ad-5c81f8a9-26a415a3-98420153_0] Write session: try to update token
2025-10-09T21:50:57.715297Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4db395ad-5c81f8a9-26a415a3-98420153_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:57.725964Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:57.725981Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:57.725983Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:57.725984Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:57.725992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342832137109829:10813]
2025-10-09T21:50:57.725995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342832137109829:10813]
2025-10-09T21:50:57.725998Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342832137109829:10813]
2025-10-09T21:50:57.726049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:57.726054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.726056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:57.726056Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:57.726062Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:57.726066Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:57.726179Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:57.728425Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1988 sessionId: src_id|4db395ad-5c81f8a9-26a415a3-98420153_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:57.730945Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:57.795029Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342832498943291:4041]
2025-10-09T21:50:57.795334Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 1987, pipe: [1:7559342832498943291:4041]
2025-10-09T21:50:57.873760Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:57.874055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.874073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.874118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342832498943291:4041], now have 1 active actors on pipe
2025-10-09T21:50:57.874335Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:57.874341Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:57.874345Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:57.874414Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342832498943291:4041]
2025-10-09T21:50:57.874420Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1988 reason: error at write: Disconnected sessionId: src_id|4db395ad-5c81f8a9-26a415a3-98420153_0
2025-10-09T21:50:57.874596Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1988 sessionId: src_id|4db395ad-5c81f8a9-26a415a3-98420153_0 is DEAD
2025-10-09T21:50:57.874577Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:57.874750Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4db395ad-5c81f8a9-26a415a3-98420153_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:57.874757Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4db395ad-5c81f8a9-26a415a3-98420153_0] Write session will restart in 0.007374s
2025-10-09T21:50:57.874784Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4db395ad-5c81f8a9-26a415a3-98420153_0] Write session: Do CDS request
2025-10-09T21:50:57.874790Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4db395ad-5c81f8a9-26a415a3-98420153_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:57.882101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:57.883440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:57.883493Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4db395ad-5c81f8a9-26a415a3-98420153_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:57.883567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:57.883586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1989
2025-10-09T21:50:57.883615Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:57.883624Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:57.883635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:57.883687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:57.883722Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1989 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:57.883744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1989 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:57.883752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1989 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:57.883754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:57.883731Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342832137109906:10817]
2025-10-09T21:50:57.884081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1989 sessionId:  describe result for acl check
2025-10-09T21:50:57.884111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:57.884113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:57.884115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:57.884122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342832498943326:5107] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:57.884303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:57.884612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:57.884660Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342832498943326:5107] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:57.884659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:57.884968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:57.885024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.885030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.885031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.885032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.885033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.885034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.885035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.885036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.885037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.885042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.885052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.885056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.885060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.885065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.885069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.885072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.885075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.885078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.885084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.885092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:57.885097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.885102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:57.885106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.885111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:57.885115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.885122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:57.885126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.885130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:57.885135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.885139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:57.885144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.885149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.885153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.885158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.885169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.885173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.885207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:57.886583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342832498943326:5107] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:57.890996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342832498943326:5107] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:57.891019Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342832498943326:5107] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:57.891022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342832498943326:5107] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:57.891029Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1989 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:57.891387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:57.892388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:57.892405Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:57.892408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:57.892412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:57.892421Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1988 [2:7559342832137109906:10817]
2025-10-09T21:50:57.892435Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:57.892455Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:57.892462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:57.892465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:57.892467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:57.892483Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:57.892494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:57.892502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:57.892561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.892567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.892569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.892572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:57.892635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:57.892643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.892655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:57.892667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:57.892684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:57.892709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342832498943292:2432], now have 1 active actors on pipe
2025-10-09T21:50:57.892718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342832498943366:5107], now have 1 active actors on pipe
2025-10-09T21:50:57.892724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342832137109901:10817]
2025-10-09T21:50:57.892725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342832137109901:10817]
2025-10-09T21:50:57.892728Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342832137109901:10817]
2025-10-09T21:50:57.892736Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:57.892748Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1988
2025-10-09T21:50:57.892771Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1988
2025-10-09T21:50:57.892749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:57.892752Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:57.892755Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:57.892757Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:57.892761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:57.892764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:57.892785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342832137109901:10817]
2025-10-09T21:50:57.892786Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:57.892795Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:57.892798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.892801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.892841Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e9ca8c10-439b88cf-65e93b23-9519a796_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:57.892853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:57.892856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:57.892860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:57.892870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:57.892877Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:57.892894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:57.893020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:57.893028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:57.893040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:57.893130Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1989 partition: 0 MaxSeqNo: 25 sessionId: src_id|e9ca8c10-439b88cf-65e93b23-9519a796_0
2025-10-09T21:50:57.893500Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4db395ad-5c81f8a9-26a415a3-98420153_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e9ca8c10-439b88cf-65e93b23-9519a796_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:57.895953Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9ca8c10-439b88cf-65e93b23-9519a796_0] Write session: try to update token
2025-10-09T21:50:57.895967Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9ca8c10-439b88cf-65e93b23-9519a796_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:57.907071Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:57.907091Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:57.907093Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:57.907097Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:57.907104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342832137109901:10817]
2025-10-09T21:50:57.907107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342832137109901:10817]
2025-10-09T21:50:57.907110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342832137109901:10817]
2025-10-09T21:50:57.907150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:57.907150Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:57.907156Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:57.907157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:57.907158Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:57.907159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:57.907296Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:57.908173Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1989 sessionId: src_id|e9ca8c10-439b88cf-65e93b23-9519a796_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:57.910444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:57.974775Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342832498943373:4041]
2025-10-09T21:50:57.975053Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 1988, pipe: [1:7559342832498943373:4041]
2025-10-09T21:50:58.055980Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:58.056206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.056218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.056263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342832498943373:4041], now have 1 active actors on pipe
2025-10-09T21:50:58.056523Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:58.056539Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:58.056543Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342832498943373:4041]
2025-10-09T21:50:58.056563Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:58.056628Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1989 reason: error at write: Disconnected sessionId: src_id|e9ca8c10-439b88cf-65e93b23-9519a796_0
2025-10-09T21:50:58.056722Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:58.056777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1989 sessionId: src_id|e9ca8c10-439b88cf-65e93b23-9519a796_0 is DEAD
2025-10-09T21:50:58.056953Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9ca8c10-439b88cf-65e93b23-9519a796_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:58.056965Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9ca8c10-439b88cf-65e93b23-9519a796_0] Write session will restart in 0.008279s
2025-10-09T21:50:58.056993Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9ca8c10-439b88cf-65e93b23-9519a796_0] Write session: Do CDS request
2025-10-09T21:50:58.056999Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9ca8c10-439b88cf-65e93b23-9519a796_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:58.064935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:58.066524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9ca8c10-439b88cf-65e93b23-9519a796_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:58.066621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:58.066639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1990
2025-10-09T21:50:58.066772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:58.066799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1990 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:58.066833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1990 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:58.066844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1990 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:58.066847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:58.066968Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:58.066982Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:58.066995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:58.067047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:58.067088Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342836432077271:10819]
2025-10-09T21:50:58.067283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1990 sessionId:  describe result for acl check
2025-10-09T21:50:58.067325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:58.067338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:58.067340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:58.067345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342836793910704:5110] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:58.067664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:58.067905Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342836793910704:5110] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:58.067977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:58.068031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:58.068356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:58.068423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.068430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.068431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.068432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.068434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.068435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.068436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.068437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.068438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.068443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.068452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.068455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.068459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.068463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.068467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.068470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.068474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.068477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.068484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.068493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.068499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.068510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.068518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.068529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.068534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.068548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.068553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.068559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.068563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.068568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.068573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.068580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.068584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.068589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.068594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.068598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.068625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:58.069752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342836793910704:5110] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:58.074550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342836793910704:5110] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:58.074567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342836793910704:5110] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:58.074572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342836793910704:5110] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:58.074582Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1990 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:58.074925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:58.075947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:58.075962Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:58.075966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:58.075970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:58.075976Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1989 [2:7559342836432077271:10819]
2025-10-09T21:50:58.075989Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:58.076003Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:58.076008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:58.076011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:58.076012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:58.076024Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:58.076028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:58.076033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:58.076073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.076078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.076079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.076082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:58.076132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:58.076140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.076152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.076167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.076185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:58.076204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342836793910671:2432], now have 1 active actors on pipe
2025-10-09T21:50:58.076212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342836793910742:5110], now have 1 active actors on pipe
2025-10-09T21:50:58.076220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342836432077266:10819]
2025-10-09T21:50:58.076222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342836432077266:10819]
2025-10-09T21:50:58.076224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342836432077266:10819]
2025-10-09T21:50:58.076240Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:58.076258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:58.076259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:58.076263Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:58.076264Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:58.076276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:58.076279Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:58.076279Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1989
2025-10-09T21:50:58.076288Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1989
2025-10-09T21:50:58.076306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342836432077266:10819]
2025-10-09T21:50:58.076313Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:58.076324Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:58.076340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.076344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.076378Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|79bc2adb-8392bde6-769e6fd1-6348165c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:58.076390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:58.076394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:58.076396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:58.076406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:58.076413Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:58.076433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:58.076596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.076607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.076621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:58.076722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1990 partition: 0 MaxSeqNo: 25 sessionId: src_id|79bc2adb-8392bde6-769e6fd1-6348165c_0
2025-10-09T21:50:58.077015Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9ca8c10-439b88cf-65e93b23-9519a796_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|79bc2adb-8392bde6-769e6fd1-6348165c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:58.079442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|79bc2adb-8392bde6-769e6fd1-6348165c_0] Write session: try to update token
2025-10-09T21:50:58.079460Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|79bc2adb-8392bde6-769e6fd1-6348165c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:58.090646Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:58.090666Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:58.090668Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:58.090670Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:58.090684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342836432077266:10819]
2025-10-09T21:50:58.090688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342836432077266:10819]
2025-10-09T21:50:58.090691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342836432077266:10819]
2025-10-09T21:50:58.090730Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:58.090735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:58.090738Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:58.090741Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:58.090743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.090745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:58.090873Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:58.091888Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1990 sessionId: src_id|79bc2adb-8392bde6-769e6fd1-6348165c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:58.094319Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:58.156550Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342836793910750:4041]
2025-10-09T21:50:58.156851Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 1989, pipe: [1:7559342836793910750:4041]
2025-10-09T21:50:58.250925Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:58.251236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.251251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.251299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342836793910750:4041], now have 1 active actors on pipe
2025-10-09T21:50:58.251622Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342836793910750:4041]
2025-10-09T21:50:58.251646Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:58.251661Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:58.251663Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:58.251725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1990 reason: error at write: Disconnected sessionId: src_id|79bc2adb-8392bde6-769e6fd1-6348165c_0
2025-10-09T21:50:58.251915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1990 sessionId: src_id|79bc2adb-8392bde6-769e6fd1-6348165c_0 is DEAD
2025-10-09T21:50:58.252073Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79bc2adb-8392bde6-769e6fd1-6348165c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:58.252098Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79bc2adb-8392bde6-769e6fd1-6348165c_0] Write session will restart in 0.007938s
2025-10-09T21:50:58.252123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79bc2adb-8392bde6-769e6fd1-6348165c_0] Write session: Do CDS request
2025-10-09T21:50:58.252132Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79bc2adb-8392bde6-769e6fd1-6348165c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:58.252778Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:58.260380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:58.261526Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|79bc2adb-8392bde6-769e6fd1-6348165c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:58.261629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:58.261647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1991
2025-10-09T21:50:58.261801Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1991 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:58.261846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1991 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:58.261855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1991 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:58.261859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:58.262008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:58.262191Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:58.262201Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:58.262211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:58.262257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:58.262299Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342836432077342:10821]
2025-10-09T21:50:58.262343Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1991 sessionId:  describe result for acl check
2025-10-09T21:50:58.262382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:58.262388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:58.262390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:58.262396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342836793910788:5121] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:58.262907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:58.262951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342836793910788:5121] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:58.263280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:58.263327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:58.263644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:58.263695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.263704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.263705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.263707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.263709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.263712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.263713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.263715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.263716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.263722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.263733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.263738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.263742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.263747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.263752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.263756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.263760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.263765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.263771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.263780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.263786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.263792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.263797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.263803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.263807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.263813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.263818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.263824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.263830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.263838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.263844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.263850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.263855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.263860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.263865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.263877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.263908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:58.264930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342836793910788:5121] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:58.269816Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342836793910788:5121] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:58.269829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342836793910788:5121] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:58.269832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342836793910788:5121] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:58.269839Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1991 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:58.270253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:58.271252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:58.271269Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:58.271272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:58.271276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:58.271283Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1990 [2:7559342836432077342:10821]
2025-10-09T21:50:58.271297Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:58.271314Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:58.271321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:58.271324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:58.271327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:58.271346Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:58.271358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:58.271371Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:58.271427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.271436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.271439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.271443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:58.271510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:58.271517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.271529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.271543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.271568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:58.271595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342836793910827:5121], now have 1 active actors on pipe
2025-10-09T21:50:58.271604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342836793910756:2432], now have 1 active actors on pipe
2025-10-09T21:50:58.271611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342836432077337:10821]
2025-10-09T21:50:58.271614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342836432077337:10821]
2025-10-09T21:50:58.271616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342836432077337:10821]
2025-10-09T21:50:58.271628Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:58.271635Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1990
2025-10-09T21:50:58.271644Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:58.271647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:58.271661Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:58.271638Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1990
2025-10-09T21:50:58.271664Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:58.271664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:58.271667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:58.271695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342836432077337:10821]
2025-10-09T21:50:58.271706Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:58.271711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:58.271722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.271727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.271778Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7fcc5679-4d5ac09c-714cd3a0-320a4eec_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:58.271790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:58.271793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:58.271795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:58.271805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:58.271818Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:58.271838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:58.271967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.271974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.271987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:58.272057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1991 partition: 0 MaxSeqNo: 25 sessionId: src_id|7fcc5679-4d5ac09c-714cd3a0-320a4eec_0
2025-10-09T21:50:58.272385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|79bc2adb-8392bde6-769e6fd1-6348165c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7fcc5679-4d5ac09c-714cd3a0-320a4eec_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:58.274765Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7fcc5679-4d5ac09c-714cd3a0-320a4eec_0] Write session: try to update token
2025-10-09T21:50:58.274783Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7fcc5679-4d5ac09c-714cd3a0-320a4eec_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:58.286317Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:58.286339Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:58.286341Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:58.286343Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:58.286353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342836432077337:10821]
2025-10-09T21:50:58.286358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342836432077337:10821]
2025-10-09T21:50:58.286365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342836432077337:10821]
2025-10-09T21:50:58.286411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:58.286415Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:58.286422Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:58.286422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.286425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:58.286426Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:58.286544Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:58.287840Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1991 sessionId: src_id|7fcc5679-4d5ac09c-714cd3a0-320a4eec_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:58.290275Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:58.351620Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342836793910839:4041]
2025-10-09T21:50:58.351960Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 1990, pipe: [1:7559342836793910839:4041]
2025-10-09T21:50:58.434056Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:58.434434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.434450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.434494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342836793910839:4041], now have 1 active actors on pipe
2025-10-09T21:50:58.434719Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:58.434714Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342836793910839:4041]
2025-10-09T21:50:58.434738Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:58.434742Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:58.434749Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:58.434816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1991 reason: error at write: Disconnected sessionId: src_id|7fcc5679-4d5ac09c-714cd3a0-320a4eec_0
2025-10-09T21:50:58.435019Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1991 sessionId: src_id|7fcc5679-4d5ac09c-714cd3a0-320a4eec_0 is DEAD
2025-10-09T21:50:58.435129Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fcc5679-4d5ac09c-714cd3a0-320a4eec_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:58.435139Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fcc5679-4d5ac09c-714cd3a0-320a4eec_0] Write session will restart in 0.008301s
2025-10-09T21:50:58.435176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fcc5679-4d5ac09c-714cd3a0-320a4eec_0] Write session: Do CDS request
2025-10-09T21:50:58.435182Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fcc5679-4d5ac09c-714cd3a0-320a4eec_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:58.443096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:58.444518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7fcc5679-4d5ac09c-714cd3a0-320a4eec_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:58.444597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:58.444613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1992
2025-10-09T21:50:58.444684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:58.444743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1992 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:58.444776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1992 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:58.444781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1992 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:58.444785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:58.444865Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:58.444876Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:58.444887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:58.444936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:58.444976Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342836432077427:10823]
2025-10-09T21:50:58.445115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1992 sessionId:  describe result for acl check
2025-10-09T21:50:58.445151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:58.445158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:58.445161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:58.445166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342836793910889:5141] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:58.445469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:58.445605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342836793910889:5141] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:58.445779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:58.445831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:58.446173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:58.446240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.446246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.446247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.446248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.446250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.446251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.446252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.446253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.446255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.446261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.446270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.446275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.446279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.446287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.446290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.446294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.446298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.446301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.446307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.446316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.446324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.446329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.446336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.446340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.446346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.446351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.446359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.446364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.446369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.446374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.446378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.446383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.446387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.446391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.446395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.446399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.446423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:58.447575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342836793910889:5141] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:58.452289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342836793910889:5141] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:58.452306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342836793910889:5141] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:58.452309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342836793910889:5141] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:58.452317Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1992 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:58.452647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:58.453801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:58.453816Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:58.453819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:58.453823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:58.453829Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1991 [2:7559342836432077427:10823]
2025-10-09T21:50:58.453840Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:58.453857Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:58.453859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:58.453863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:58.453865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:58.453880Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:58.453893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:58.453906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:58.453961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.453967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.453970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.453972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:58.454050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:58.454058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.454070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.454082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.454103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:58.454127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342836793910842:2432], now have 1 active actors on pipe
2025-10-09T21:50:58.454136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342836793910927:5141], now have 1 active actors on pipe
2025-10-09T21:50:58.454143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342836432077412:10823]
2025-10-09T21:50:58.454145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342836432077412:10823]
2025-10-09T21:50:58.454147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342836432077412:10823]
2025-10-09T21:50:58.454160Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:58.454174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:58.454185Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:58.454165Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1991
2025-10-09T21:50:58.454189Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:58.454191Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:58.454192Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:58.454195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:58.454205Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1991
2025-10-09T21:50:58.454228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342836432077412:10823]
2025-10-09T21:50:58.454235Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:58.454237Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:58.454248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.454257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.454299Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a9d0e75d-1460bc23-bf5f9bb1-171408be_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:58.454311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:58.454315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:58.454323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:58.454334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:58.454347Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:58.454362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:58.454468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.454474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.454486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:58.454599Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1992 partition: 0 MaxSeqNo: 25 sessionId: src_id|a9d0e75d-1460bc23-bf5f9bb1-171408be_0
2025-10-09T21:50:58.454900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7fcc5679-4d5ac09c-714cd3a0-320a4eec_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a9d0e75d-1460bc23-bf5f9bb1-171408be_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:58.457340Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9d0e75d-1460bc23-bf5f9bb1-171408be_0] Write session: try to update token
2025-10-09T21:50:58.457358Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9d0e75d-1460bc23-bf5f9bb1-171408be_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:58.468362Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:58.468377Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:58.468379Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:58.468381Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:58.468389Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342836432077412:10823]
2025-10-09T21:50:58.468394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342836432077412:10823]
2025-10-09T21:50:58.468397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342836432077412:10823]
2025-10-09T21:50:58.468437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:58.468438Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:58.468444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.468445Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:58.468447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:58.468447Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:58.468586Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:58.470054Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1992 sessionId: src_id|a9d0e75d-1460bc23-bf5f9bb1-171408be_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:58.472366Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:58.534905Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342836793910944:4041]
2025-10-09T21:50:58.535188Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 1991, pipe: [1:7559342836793910944:4041]
2025-10-09T21:50:58.616839Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:58.617158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.617174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.617219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342836793910944:4041], now have 1 active actors on pipe
2025-10-09T21:50:58.617422Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:58.617471Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:58.617479Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:58.617539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1992 reason: error at write: Disconnected sessionId: src_id|a9d0e75d-1460bc23-bf5f9bb1-171408be_0
2025-10-09T21:50:58.617693Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:58.617736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1992 sessionId: src_id|a9d0e75d-1460bc23-bf5f9bb1-171408be_0 is DEAD
2025-10-09T21:50:58.617883Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342836793910944:4041]
2025-10-09T21:50:58.617908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9d0e75d-1460bc23-bf5f9bb1-171408be_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:58.617920Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9d0e75d-1460bc23-bf5f9bb1-171408be_0] Write session will restart in 0.005480s
2025-10-09T21:50:58.617943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9d0e75d-1460bc23-bf5f9bb1-171408be_0] Write session: Do CDS request
2025-10-09T21:50:58.617947Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9d0e75d-1460bc23-bf5f9bb1-171408be_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:58.624443Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a9d0e75d-1460bc23-bf5f9bb1-171408be_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:58.624538Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:58.624569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1993
2025-10-09T21:50:58.624705Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1993 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:58.624745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1993 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:58.624755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1993 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:58.624758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:58.625157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1993 sessionId:  describe result for acl check
2025-10-09T21:50:58.625189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:58.625196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:58.625197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:58.625203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342836793910980:5146] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:58.625717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342836793910980:5146] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:58.625722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:58.627248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:58.627417Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:58.627425Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:58.627435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:58.627492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:58.627536Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342836432077523:10837]
2025-10-09T21:50:58.627823Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342836793910980:5146] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:58.627985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:58.628260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:58.628298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:58.628530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:58.628567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.628571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.628572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.628573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.628573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.628574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.628575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.628575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.628576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.628579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.628586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.628588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.628590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.628592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.628594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.628595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.628597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.628599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.628603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.628608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.628612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.628617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.628622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.628627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.628631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.628635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.628639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.628643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.628647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.628651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.628655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.628659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.628664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.628668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.628672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.628676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.628705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:58.633548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342836793910980:5146] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:58.633561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342836793910980:5146] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:58.633564Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342836793910980:5146] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:58.633571Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1993 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:58.634638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:58.635647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:58.635661Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:58.635664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:58.635667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:58.635674Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1992 [2:7559342836432077523:10837]
2025-10-09T21:50:58.635686Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:58.635701Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:58.635709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:58.635712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:58.635714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:58.635732Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:58.635739Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:58.635746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:58.635801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.635809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.635810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.635813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:58.635878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:58.635886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.635898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.635910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.635927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:58.635952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342836793911021:5146], now have 1 active actors on pipe
2025-10-09T21:50:58.635959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342836793910947:2432], now have 1 active actors on pipe
2025-10-09T21:50:58.635966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342836432077518:10837]
2025-10-09T21:50:58.635969Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342836432077518:10837]
2025-10-09T21:50:58.635971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342836432077518:10837]
2025-10-09T21:50:58.635983Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:58.636000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:58.636008Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:58.636011Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:58.636013Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:58.636018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:58.636021Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:58.636018Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1992
2025-10-09T21:50:58.636021Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1992
2025-10-09T21:50:58.636044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342836432077518:10837]
2025-10-09T21:50:58.636050Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:58.636052Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:58.636068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.636075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.636118Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1b2257bc-569d7ebf-bea28a9d-13edf06a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:58.636129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:58.636133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:58.636135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:58.636145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:58.636159Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:58.636177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:58.636303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.636310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.636321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:58.636406Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1993 partition: 0 MaxSeqNo: 25 sessionId: src_id|1b2257bc-569d7ebf-bea28a9d-13edf06a_0
2025-10-09T21:50:58.636734Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a9d0e75d-1460bc23-bf5f9bb1-171408be_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1b2257bc-569d7ebf-bea28a9d-13edf06a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:58.639076Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b2257bc-569d7ebf-bea28a9d-13edf06a_0] Write session: try to update token
2025-10-09T21:50:58.639094Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b2257bc-569d7ebf-bea28a9d-13edf06a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:58.649857Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:58.649872Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:58.649874Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:58.649876Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:58.649883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342836432077518:10837]
2025-10-09T21:50:58.649887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342836432077518:10837]
2025-10-09T21:50:58.649890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342836432077518:10837]
2025-10-09T21:50:58.649929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:58.649936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.649936Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:58.649939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:58.649943Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:58.649948Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:58.650081Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:58.651353Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1993 sessionId: src_id|1b2257bc-569d7ebf-bea28a9d-13edf06a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:58.653666Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:58.718207Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342836793911028:4041]
2025-10-09T21:50:58.718485Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 1992, pipe: [1:7559342836793911028:4041]
2025-10-09T21:50:58.797276Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:58.797693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.797710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.797757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342836793911028:4041], now have 1 active actors on pipe
2025-10-09T21:50:58.798021Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:58.798053Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:58.798126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1993 reason: error at write: Disconnected sessionId: src_id|1b2257bc-569d7ebf-bea28a9d-13edf06a_0
2025-10-09T21:50:58.798297Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342836793911028:4041]
2025-10-09T21:50:58.798336Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:58.798374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1993 sessionId: src_id|1b2257bc-569d7ebf-bea28a9d-13edf06a_0 is DEAD
2025-10-09T21:50:58.798504Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b2257bc-569d7ebf-bea28a9d-13edf06a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:58.798515Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b2257bc-569d7ebf-bea28a9d-13edf06a_0] Write session will restart in 0.007665s
2025-10-09T21:50:58.798540Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b2257bc-569d7ebf-bea28a9d-13edf06a_0] Write session: Do CDS request
2025-10-09T21:50:58.798524Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:58.798550Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b2257bc-569d7ebf-bea28a9d-13edf06a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:58.806744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:58.807508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b2257bc-569d7ebf-bea28a9d-13edf06a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:58.807610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:58.807628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1994
2025-10-09T21:50:58.807769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1994 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:58.807802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1994 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:58.807806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1994 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:58.807808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:58.808202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:58.808256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1994 sessionId:  describe result for acl check
2025-10-09T21:50:58.808291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:58.808296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:58.808298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:58.808304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342836793911067:5150] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:58.808386Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:58.808397Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:58.808410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:58.808465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:58.808512Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342836432077601:10841]
2025-10-09T21:50:58.808841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342836793911067:5150] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:58.809070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:58.809369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:58.809444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:58.809754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:58.809812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.809818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.809819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.809820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.809821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.809823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.809824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.809825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.809826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.809831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.809840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.809844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.809848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.809850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.809852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.809854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.809856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.809858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.809864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.809872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:58.809879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.809884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:58.809889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.809893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:58.809896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.809901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:58.809905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.809910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:58.809914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.809919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:58.809923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.809929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.809932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.809937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.809947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.809951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.809978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:58.810537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342836793911067:5150] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:58.815614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342836793911067:5150] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:58.815629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342836793911067:5150] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:58.815632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342836793911067:5150] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:58.815641Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1994 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:58.815985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:58.816979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:58.816994Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:58.816996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:58.816999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:58.817004Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1993 [2:7559342836432077601:10841]
2025-10-09T21:50:58.817015Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:58.817027Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:58.817029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:58.817031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:58.817033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:58.817045Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:58.817048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:58.817055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:58.817109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.817117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.817120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.817122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:58.817182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:58.817188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.817195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:58.817201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:58.817217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:58.817242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342836793911031:2432], now have 1 active actors on pipe
2025-10-09T21:50:58.817250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342836793911106:5150], now have 1 active actors on pipe
2025-10-09T21:50:58.817257Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342836432077596:10841]
2025-10-09T21:50:58.817261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342836432077596:10841]
2025-10-09T21:50:58.817264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342836432077596:10841]
2025-10-09T21:50:58.817275Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:58.817289Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:58.817294Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:58.817305Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:58.817307Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:58.817307Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:58.817309Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:58.817334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342836432077596:10841]
2025-10-09T21:50:58.817347Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:58.817349Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1993
2025-10-09T21:50:58.817380Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1993
2025-10-09T21:50:58.817378Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:58.817434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.817439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.817466Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ef895703-6e5b0729-64659ac9-2cd4179b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:58.817482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:58.817486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:58.817490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:58.817498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:58.817509Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:58.817523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:58.817712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.817723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.817740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:58.817811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1994 partition: 0 MaxSeqNo: 25 sessionId: src_id|ef895703-6e5b0729-64659ac9-2cd4179b_0
2025-10-09T21:50:58.818109Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b2257bc-569d7ebf-bea28a9d-13edf06a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ef895703-6e5b0729-64659ac9-2cd4179b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:58.820687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef895703-6e5b0729-64659ac9-2cd4179b_0] Write session: try to update token
2025-10-09T21:50:58.820703Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef895703-6e5b0729-64659ac9-2cd4179b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:58.828752Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:58.828770Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:58.828779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342836432077596:10841]
2025-10-09T21:50:58.828819Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:58.828825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:58.828989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:58.829124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:58.829263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:58.829385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:58.829572Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:58.829610Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:58.833590Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1994 sessionId: src_id|ef895703-6e5b0729-64659ac9-2cd4179b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:58.835954Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:58.836163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.836184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.836222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:58.836318Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:58.841514Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:58.841531Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:58.841533Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:58.841537Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:58.841551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342836432077596:10841]
2025-10-09T21:50:58.841554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342836432077596:10841]
2025-10-09T21:50:58.841557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342836432077596:10841]
2025-10-09T21:50:58.841595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:58.841599Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:58.841605Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:58.841609Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:58.841610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:58.841612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:58.841748Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:58.898416Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342836793911116:4041]
2025-10-09T21:50:58.898691Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 1993, pipe: [1:7559342836793911116:4041]
2025-10-09T21:50:58.984185Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:58.991256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:58.991288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:58.991454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:58.991523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:58.991590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:58.991654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:58.991699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:58.991710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:58.991783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:58.991845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:58.991914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:58.991978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:58.992022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:58.992032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:58.992095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:58.992151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:58.992218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:58.992276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:58.992325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:58.992333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:58.992397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:58.992456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:58.992515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:58.992587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:58.992626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:58.992634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:58.992696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:58.992760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:58.992815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:58.992869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:58.992912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:58.992920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:58.992985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:58.993049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:58.993113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:58.993168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:58.993205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:58.993210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:58.993267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:58.993317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:58.993368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:58.993433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:58.993477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:58.993484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:58.993543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:58.993594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:58.993648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:58.993705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:58.993805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:58.993814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:58.993867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:58.993920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:58.993972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:58.994023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:58.994074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:58.994080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:58.994134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:58.994187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:58.994241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:58.994304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:58.994350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:58.994357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:58.994399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342836793911116:4041], now have 1 active actors on pipe
2025-10-09T21:50:58.994657Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:58.994694Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342836793911116:4041]
2025-10-09T21:50:58.994717Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:58.994721Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:58.994752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1994 reason: error at write: Disconnected sessionId: src_id|ef895703-6e5b0729-64659ac9-2cd4179b_0
2025-10-09T21:50:58.994936Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1994 sessionId: src_id|ef895703-6e5b0729-64659ac9-2cd4179b_0 is DEAD
2025-10-09T21:50:58.995089Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef895703-6e5b0729-64659ac9-2cd4179b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:58.995101Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef895703-6e5b0729-64659ac9-2cd4179b_0] Write session will restart in 0.009195s
2025-10-09T21:50:58.995126Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef895703-6e5b0729-64659ac9-2cd4179b_0] Write session: Do CDS request
2025-10-09T21:50:58.995132Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef895703-6e5b0729-64659ac9-2cd4179b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:58.995146Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:59.004487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:59.005616Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef895703-6e5b0729-64659ac9-2cd4179b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:59.005690Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:59.005708Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1995
2025-10-09T21:50:59.005856Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1995 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:59.005889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1995 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:59.005895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1995 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:59.005899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:59.006234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:59.006327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1995 sessionId:  describe result for acl check
2025-10-09T21:50:59.006374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:59.006381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:59.006384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:59.006390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342841088878454:5162] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:59.006391Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:59.006400Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:59.006409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:59.006459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:59.006500Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342840727044972:10843]
2025-10-09T21:50:59.006935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342841088878454:5162] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:59.006987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:59.007281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:59.007315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:59.007642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:59.007710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.007718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.007720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.007721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.007722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.007723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.007724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.007725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.007727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.007732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.007742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.007746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.007750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.007754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.007758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.007761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.007765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.007769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.007775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.007783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.007789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.007795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.007800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.007805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.007810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.007814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.007817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.007820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.007824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.007830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.007835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.007841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.007845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.007850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.007854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.007859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.007894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:59.009487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342841088878454:5162] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:59.013926Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342841088878454:5162] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:59.013946Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342841088878454:5162] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:59.013950Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342841088878454:5162] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:59.013961Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1995 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:59.014398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:59.015474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:59.015489Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:59.015491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:59.015494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:59.015500Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1994 [2:7559342840727044972:10843]
2025-10-09T21:50:59.015511Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:59.015529Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:59.015532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:59.015535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:59.015537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:59.015554Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:59.015558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:59.015570Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:59.015615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.015621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.015622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.015624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:59.015687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:59.015695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.015702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.015708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.015721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:59.015736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342840727044966:10843]
2025-10-09T21:50:59.015741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342840727044966:10843]
2025-10-09T21:50:59.015742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342840727044966:10843]
2025-10-09T21:50:59.015748Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:59.015759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342841088878489:5162], now have 1 active actors on pipe
2025-10-09T21:50:59.015761Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:59.015764Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:59.015766Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:59.015768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342836793911120:2432], now have 1 active actors on pipe
2025-10-09T21:50:59.015772Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:59.015780Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:59.015782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:59.015808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342840727044966:10843]
2025-10-09T21:50:59.015818Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:59.015854Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:59.015887Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1994
2025-10-09T21:50:59.015887Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1994
2025-10-09T21:50:59.016045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.016059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.016090Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|97db2c99-e8c772fd-3bb2cca3-6558d45e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:59.016106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:59.016115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:59.016118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:59.016128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:59.016144Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:59.016164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:59.016358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.016370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.016383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:59.016461Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1995 partition: 0 MaxSeqNo: 25 sessionId: src_id|97db2c99-e8c772fd-3bb2cca3-6558d45e_0
2025-10-09T21:50:59.016835Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef895703-6e5b0729-64659ac9-2cd4179b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|97db2c99-e8c772fd-3bb2cca3-6558d45e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:59.019417Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97db2c99-e8c772fd-3bb2cca3-6558d45e_0] Write session: try to update token
2025-10-09T21:50:59.019438Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97db2c99-e8c772fd-3bb2cca3-6558d45e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:59.032054Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:59.032080Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:59.032085Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:59.032087Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:59.032099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342840727044966:10843]
2025-10-09T21:50:59.032106Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342840727044966:10843]
2025-10-09T21:50:59.032113Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342840727044966:10843]
2025-10-09T21:50:59.032168Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:59.032174Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:59.032178Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:59.032177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:59.032188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.032202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:59.032470Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:59.033655Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1995 sessionId: src_id|97db2c99-e8c772fd-3bb2cca3-6558d45e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:59.036288Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:59.095045Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342841088878497:4041]
2025-10-09T21:50:59.095332Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 1994, pipe: [1:7559342841088878497:4041]
2025-10-09T21:50:59.189086Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:59.189439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.189458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.189505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342841088878497:4041], now have 1 active actors on pipe
2025-10-09T21:50:59.189693Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:59.189738Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:59.189752Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:59.189757Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:59.189799Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342841088878497:4041]
2025-10-09T21:50:59.189852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1995 reason: error at write: Disconnected sessionId: src_id|97db2c99-e8c772fd-3bb2cca3-6558d45e_0
2025-10-09T21:50:59.190052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1995 sessionId: src_id|97db2c99-e8c772fd-3bb2cca3-6558d45e_0 is DEAD
2025-10-09T21:50:59.190184Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97db2c99-e8c772fd-3bb2cca3-6558d45e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:59.190196Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97db2c99-e8c772fd-3bb2cca3-6558d45e_0] Write session will restart in 0.009191s
2025-10-09T21:50:59.190234Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97db2c99-e8c772fd-3bb2cca3-6558d45e_0] Write session: Do CDS request
2025-10-09T21:50:59.190240Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97db2c99-e8c772fd-3bb2cca3-6558d45e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:59.198777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:59.200450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:59.200505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97db2c99-e8c772fd-3bb2cca3-6558d45e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:59.200621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:59.200640Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1996
2025-10-09T21:50:59.200648Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:59.200662Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:59.200675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:59.200736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:59.200769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1996 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:59.200781Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342840727045046:10845]
2025-10-09T21:50:59.200812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1996 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:59.200822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1996 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:59.200826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:59.201258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1996 sessionId:  describe result for acl check
2025-10-09T21:50:59.201299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:59.201300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:59.201302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:59.201307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342841088878537:5184] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:59.201377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:59.201759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:59.201814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:59.201870Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342841088878537:5184] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:59.202158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:59.202212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.202218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.202219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.202221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.202222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.202223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.202224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.202225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.202226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.202232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.202242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.202246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.202250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.202254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.202259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.202263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.202267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.202271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.202278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.202286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.202292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.202296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.202299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.202302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.202304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.202307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.202310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.202313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.202317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.202320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.202324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.202327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.202329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.202332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.202336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.202339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.202355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:59.203886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342841088878537:5184] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:59.208170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342841088878537:5184] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:59.208188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342841088878537:5184] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:59.208192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342841088878537:5184] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:59.208201Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1996 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:59.208656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:59.209719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:59.209735Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:59.209737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:59.209742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:59.209748Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1995 [2:7559342840727045046:10845]
2025-10-09T21:50:59.209764Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:59.209782Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:59.209784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:59.209788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:59.209790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:59.209808Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:59.209813Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:59.209821Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:59.209882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.209891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.209893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.209896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:59.209964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:59.209974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.209986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.209998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.210017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:59.210062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342841088878499:2432], now have 1 active actors on pipe
2025-10-09T21:50:59.210071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342841088878579:5184], now have 1 active actors on pipe
2025-10-09T21:50:59.210078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342840727045041:10845]
2025-10-09T21:50:59.210081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342840727045041:10845]
2025-10-09T21:50:59.210083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342840727045041:10845]
2025-10-09T21:50:59.210095Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:59.210086Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1995
2025-10-09T21:50:59.210103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1995
2025-10-09T21:50:59.210114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:59.210113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:59.210117Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:59.210128Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:59.210134Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:59.210137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:59.210186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.210195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.210208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342840727045041:10845]
2025-10-09T21:50:59.210223Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:59.210238Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:59.210254Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c730a4e2-33bde758-3d929af2-282a885d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:59.210267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:59.210274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:59.210277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:59.210287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:59.210301Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:59.210324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:59.210490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.210500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.210516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:59.210633Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1996 partition: 0 MaxSeqNo: 25 sessionId: src_id|c730a4e2-33bde758-3d929af2-282a885d_0
2025-10-09T21:50:59.211011Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97db2c99-e8c772fd-3bb2cca3-6558d45e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c730a4e2-33bde758-3d929af2-282a885d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:59.213674Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c730a4e2-33bde758-3d929af2-282a885d_0] Write session: try to update token
2025-10-09T21:50:59.213690Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c730a4e2-33bde758-3d929af2-282a885d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:59.224483Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:59.224509Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:59.224511Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:59.224513Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:59.224522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342840727045041:10845]
2025-10-09T21:50:59.224529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342840727045041:10845]
2025-10-09T21:50:59.224533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342840727045041:10845]
2025-10-09T21:50:59.224574Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:59.224579Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:59.224580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:59.224581Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:59.224586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.224589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:59.224719Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:59.226475Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1996 sessionId: src_id|c730a4e2-33bde758-3d929af2-282a885d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:59.228678Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:59.290101Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342841088878592:4041]
2025-10-09T21:50:59.290380Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 1995, pipe: [1:7559342841088878592:4041]
2025-10-09T21:50:59.375976Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:59.376604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.376623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.376671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342841088878592:4041], now have 1 active actors on pipe
2025-10-09T21:50:59.376921Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:59.376932Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:59.376939Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:59.376956Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:59.376993Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342841088878592:4041]
2025-10-09T21:50:59.377041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1996 reason: error at write: Disconnected sessionId: src_id|c730a4e2-33bde758-3d929af2-282a885d_0
2025-10-09T21:50:59.377232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1996 sessionId: src_id|c730a4e2-33bde758-3d929af2-282a885d_0 is DEAD
2025-10-09T21:50:59.377373Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c730a4e2-33bde758-3d929af2-282a885d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:59.377387Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c730a4e2-33bde758-3d929af2-282a885d_0] Write session will restart in 0.007544s
2025-10-09T21:50:59.377421Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c730a4e2-33bde758-3d929af2-282a885d_0] Write session: Do CDS request
2025-10-09T21:50:59.377439Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c730a4e2-33bde758-3d929af2-282a885d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:59.386142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:59.386529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c730a4e2-33bde758-3d929af2-282a885d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:59.386575Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:59.386593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1997
2025-10-09T21:50:59.386714Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1997 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:59.386754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1997 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:59.386763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1997 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:59.386767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:59.387173Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1997 sessionId:  describe result for acl check
2025-10-09T21:50:59.387205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:59.387207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:59.387208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:59.387212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342841088878633:5173] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:59.387708Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342841088878633:5173] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:59.387829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:59.388016Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:59.388027Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:59.388038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:59.388090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:59.388132Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342840727045122:10847]
2025-10-09T21:50:59.388666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:59.388962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:59.389025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:59.389378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:59.389455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.389461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.389463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.389464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.389465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.389466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.389467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.389468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.389469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.389475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.389484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.389488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.389493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.389496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.389500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.389504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.389508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.389511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.389519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.389527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.389535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.389540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.389545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.389550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.389554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.389559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.389567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.389572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.389581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.389585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.389592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.389597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.389602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.389607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.389612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.389616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.389648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:59.389724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342841088878633:5173] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:59.395045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342841088878633:5173] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:59.395062Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342841088878633:5173] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:59.395066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342841088878633:5173] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:59.395074Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1997 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:59.395830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:59.397485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:59.397500Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:59.397502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:59.397506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:59.397513Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1996 [2:7559342840727045122:10847]
2025-10-09T21:50:59.397527Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:59.397545Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:59.397552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:59.397556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:59.397558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:59.397575Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:59.397581Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:59.397590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:59.397651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.397658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.397660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.397663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:59.397733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:59.397742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.397756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.397773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.397798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:59.397823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342841088878594:2432], now have 1 active actors on pipe
2025-10-09T21:50:59.397832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342841088878673:5173], now have 1 active actors on pipe
2025-10-09T21:50:59.397839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342840727045117:10847]
2025-10-09T21:50:59.397847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342840727045117:10847]
2025-10-09T21:50:59.397849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342840727045117:10847]
2025-10-09T21:50:59.397863Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:59.397878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:59.397857Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1996
2025-10-09T21:50:59.397861Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1996
2025-10-09T21:50:59.397896Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:59.397899Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:59.397902Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:59.397908Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:59.397910Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:59.397952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.397961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.397972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342840727045117:10847]
2025-10-09T21:50:59.397980Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:59.397995Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:59.398009Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1d774796-6b6ac520-a1459213-2b53e0e9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:59.398022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:59.398048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:59.398052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:59.398063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:59.398074Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:59.398168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:59.398395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.398408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.398425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:59.398519Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1997 partition: 0 MaxSeqNo: 25 sessionId: src_id|1d774796-6b6ac520-a1459213-2b53e0e9_0
2025-10-09T21:50:59.398843Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c730a4e2-33bde758-3d929af2-282a885d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1d774796-6b6ac520-a1459213-2b53e0e9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:59.405706Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1d774796-6b6ac520-a1459213-2b53e0e9_0] Write session: try to update token
2025-10-09T21:50:59.405727Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1d774796-6b6ac520-a1459213-2b53e0e9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:59.412026Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:59.412041Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:59.412043Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:59.412045Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:59.412052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342840727045117:10847]
2025-10-09T21:50:59.412057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342840727045117:10847]
2025-10-09T21:50:59.412062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342840727045117:10847]
2025-10-09T21:50:59.412110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:59.412117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.412120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:59.412123Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:59.412140Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:59.412145Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:59.412271Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:59.420925Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1997 sessionId: src_id|1d774796-6b6ac520-a1459213-2b53e0e9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:59.423273Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:59.476999Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342841088878680:4041]
2025-10-09T21:50:59.477275Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 1996, pipe: [1:7559342841088878680:4041]
2025-10-09T21:50:59.557250Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:59.557624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.557642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.557664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342841088878680:4041], now have 1 active actors on pipe
2025-10-09T21:50:59.557876Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:59.557875Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:59.557888Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342841088878680:4041]
2025-10-09T21:50:59.557912Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:59.557915Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:59.557985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1997 reason: error at write: Disconnected sessionId: src_id|1d774796-6b6ac520-a1459213-2b53e0e9_0
2025-10-09T21:50:59.558169Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1997 sessionId: src_id|1d774796-6b6ac520-a1459213-2b53e0e9_0 is DEAD
2025-10-09T21:50:59.558326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d774796-6b6ac520-a1459213-2b53e0e9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:59.558338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d774796-6b6ac520-a1459213-2b53e0e9_0] Write session will restart in 0.005622s
2025-10-09T21:50:59.558365Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d774796-6b6ac520-a1459213-2b53e0e9_0] Write session: Do CDS request
2025-10-09T21:50:59.558371Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d774796-6b6ac520-a1459213-2b53e0e9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:59.565510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1d774796-6b6ac520-a1459213-2b53e0e9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:59.565706Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:59.565722Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1998
2025-10-09T21:50:59.566113Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1998 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:59.566152Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1998 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:59.566163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1998 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:59.566167Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:59.566604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1998 sessionId:  describe result for acl check
2025-10-09T21:50:59.566642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:59.566644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:59.566646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:59.566652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342841088878746:5203] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:59.566968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:59.567182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342841088878746:5203] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:59.568467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:59.568648Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:59.568660Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:59.568674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:59.568750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:59.568806Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342840727045237:10861]
2025-10-09T21:50:59.568838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342841088878746:5203] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:59.569358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:59.569673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:59.569725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:59.570017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:59.570096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.570106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.570107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.570108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.570109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.570110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.570111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.570112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.570114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.570118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.570128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.570132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.570135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.570139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.570143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.570147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.570152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.570155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.570161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.570168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.570175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.570180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.570185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.570189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.570193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.570198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.570207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.570212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.570217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.570226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.570231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.570236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.570240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.570245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.570250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.570255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.570284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:59.575051Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342841088878746:5203] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:59.575067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342841088878746:5203] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:59.575071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342841088878746:5203] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:59.575080Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1998 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:59.576335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:59.577669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:59.577683Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:59.577687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:59.577691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:59.577697Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1997 [2:7559342840727045237:10861]
2025-10-09T21:50:59.577713Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:59.577729Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:59.577736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:59.577739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:59.577741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:59.577757Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:59.577769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:59.577776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:59.577833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.577841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.577843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.577846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:59.577907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:59.577918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.577929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.577938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.577955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:59.577976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342840727045232:10861]
2025-10-09T21:50:59.577983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342840727045232:10861]
2025-10-09T21:50:59.577985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342840727045232:10861]
2025-10-09T21:50:59.577992Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:59.578008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:59.578011Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:59.578014Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:59.578016Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:59.578024Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:59.578043Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:59.578059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342841088878783:5203], now have 1 active actors on pipe
2025-10-09T21:50:59.578066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342841088878706:2432], now have 1 active actors on pipe
2025-10-09T21:50:59.578088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342840727045232:10861]
2025-10-09T21:50:59.578094Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:59.578080Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1997
2025-10-09T21:50:59.578082Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1997
2025-10-09T21:50:59.578106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:59.578136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.578146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.578179Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f1e996e8-bb103df2-7e5611d1-8bcae850_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:59.578194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:59.578201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:59.578203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:59.578213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:59.578224Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:59.578239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:59.578379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.578391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.578411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:59.578500Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1998 partition: 0 MaxSeqNo: 25 sessionId: src_id|f1e996e8-bb103df2-7e5611d1-8bcae850_0
2025-10-09T21:50:59.578819Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1d774796-6b6ac520-a1459213-2b53e0e9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f1e996e8-bb103df2-7e5611d1-8bcae850_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:59.581380Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f1e996e8-bb103df2-7e5611d1-8bcae850_0] Write session: try to update token
2025-10-09T21:50:59.581392Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f1e996e8-bb103df2-7e5611d1-8bcae850_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:59.594287Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1998 sessionId: src_id|f1e996e8-bb103df2-7e5611d1-8bcae850_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:59.594495Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:59.594519Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:59.594529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342840727045232:10861]
2025-10-09T21:50:59.594575Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:59.594579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:59.594786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:59.594929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:59.595087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:59.595204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:59.595376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:59.595412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:59.596645Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:59.596842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.596859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.596897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:59.597005Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:59.602734Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:59.602750Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:59.602752Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:59.602753Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:59.602762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342840727045232:10861]
2025-10-09T21:50:59.602766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342840727045232:10861]
2025-10-09T21:50:59.602771Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342840727045232:10861]
2025-10-09T21:50:59.602812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:59.602822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.602825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:59.602824Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:59.602830Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:59.602835Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:59.602946Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:59.658235Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342841088878790:4041]
2025-10-09T21:50:59.658474Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 1997, pipe: [1:7559342841088878790:4041]
2025-10-09T21:50:59.749039Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:59.756056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.756083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.756236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:59.756298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:59.756371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:59.756428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:59.756471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:59.756480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:59.756548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:59.756614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:59.756683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:59.756756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:59.756796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:59.756803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:59.756860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:59.756917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:59.756978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:59.757034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:59.757071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:59.757077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:59.757143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:59.757205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:59.757266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:59.757325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:59.757366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:59.757373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:59.757446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:59.757505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:59.757567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:59.757624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:59.757665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:59.757671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:59.757729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:59.757782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:59.757833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:59.757898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:59.757941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:59.757949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:59.758007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:59.758078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:59.758138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:59.758200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:59.758339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:59.758351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:59.758414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:59.758467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:59.758526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:59.758590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:59.758632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:59.758639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:59.758693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:59.758744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:59.758793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:59.758844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:59.758882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:59.758888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:59.758940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:50:59.758989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:50:59.759044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:50:59.759102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:50:59.759155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:50:59.759161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:50:59.759199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342841088878790:4041], now have 1 active actors on pipe
2025-10-09T21:50:59.759358Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:59.759376Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:59.759406Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1998 reason: error at write: Disconnected sessionId: src_id|f1e996e8-bb103df2-7e5611d1-8bcae850_0
2025-10-09T21:50:59.759474Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:59.759529Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342841088878790:4041]
2025-10-09T21:50:59.759611Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1998 sessionId: src_id|f1e996e8-bb103df2-7e5611d1-8bcae850_0 is DEAD
2025-10-09T21:50:59.759760Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f1e996e8-bb103df2-7e5611d1-8bcae850_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:59.759770Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f1e996e8-bb103df2-7e5611d1-8bcae850_0] Write session will restart in 0.009688s
2025-10-09T21:50:59.759794Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f1e996e8-bb103df2-7e5611d1-8bcae850_0] Write session: Do CDS request
2025-10-09T21:50:59.759800Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f1e996e8-bb103df2-7e5611d1-8bcae850_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:59.759834Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:59.768795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:59.770422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:59.770485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f1e996e8-bb103df2-7e5611d1-8bcae850_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:59.770551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:59.770567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1999
2025-10-09T21:50:59.770601Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:59.770612Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:59.770623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:59.770673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:59.770685Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1999 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:59.770710Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342840727045319:10865]
2025-10-09T21:50:59.770725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1999 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:59.770730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1999 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:59.770735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:59.771132Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1999 sessionId:  describe result for acl check
2025-10-09T21:50:59.771166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:59.771174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:59.771176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:59.771181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342841088878834:5208] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:59.771254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:59.771533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:59.771575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:59.771695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342841088878834:5208] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:59.771869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:59.771929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.771934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.771935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.771937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.771937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.771939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.771939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.771940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.771941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.771946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.771955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.771959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.771964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.771967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.771971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.771975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.771978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.771982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.771989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.771996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.772002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.772007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.772011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.772016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.772026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.772031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.772035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.772040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.772048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.772054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.772058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.772063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.772068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.772073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.772079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.772084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.772111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:59.775731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342841088878834:5208] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:59.778123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:59.778628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342841088878834:5208] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:59.778642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342841088878834:5208] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:59.778645Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342841088878834:5208] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:59.778653Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1999 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:59.779236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:59.779254Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:59.779256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:59.779260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:59.779266Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1998 [2:7559342840727045319:10865]
2025-10-09T21:50:59.779279Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:59.779298Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:59.779306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:59.779309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:59.779311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:59.779328Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:59.779338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:59.779349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:59.779401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.779409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.779410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.779413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:59.779472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:59.779480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.779491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.779507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.779527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:59.779552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342841088878873:5208], now have 1 active actors on pipe
2025-10-09T21:50:59.779558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342841088878793:2432], now have 1 active actors on pipe
2025-10-09T21:50:59.779563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342840727045314:10865]
2025-10-09T21:50:59.779564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342840727045314:10865]
2025-10-09T21:50:59.779566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342840727045314:10865]
2025-10-09T21:50:59.779573Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:59.779561Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1998
2025-10-09T21:50:59.779568Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1998
2025-10-09T21:50:59.779588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:59.779588Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:59.779591Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:59.779593Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:59.779604Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:59.779606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:59.779620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.779626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.779661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342840727045314:10865]
2025-10-09T21:50:59.779674Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:59.779689Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9c4b3f67-153653d4-485fb8c6-adb3e649_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:59.779692Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:59.779697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:59.779702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:59.779703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:59.779711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:59.779718Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:59.779754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:59.780007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.780017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.780037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:59.780182Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1999 partition: 0 MaxSeqNo: 25 sessionId: src_id|9c4b3f67-153653d4-485fb8c6-adb3e649_0
2025-10-09T21:50:59.780469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f1e996e8-bb103df2-7e5611d1-8bcae850_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9c4b3f67-153653d4-485fb8c6-adb3e649_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:59.782800Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c4b3f67-153653d4-485fb8c6-adb3e649_0] Write session: try to update token
2025-10-09T21:50:59.782816Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c4b3f67-153653d4-485fb8c6-adb3e649_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:59.793211Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:59.793230Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:59.793233Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:59.793235Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:59.793244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342840727045314:10865]
2025-10-09T21:50:59.793248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342840727045314:10865]
2025-10-09T21:50:59.793252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342840727045314:10865]
2025-10-09T21:50:59.793290Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:59.793295Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:59.793296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:59.793298Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:59.793304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.793307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:59.793448Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:50:59.794943Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1999 sessionId: src_id|9c4b3f67-153653d4-485fb8c6-adb3e649_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:59.797175Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:59.859419Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342841088878880:4041]
2025-10-09T21:50:59.859727Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 1998, pipe: [1:7559342841088878880:4041]
2025-10-09T21:50:59.938928Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:50:59.939211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.939226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.939268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342841088878880:4041], now have 1 active actors on pipe
2025-10-09T21:50:59.939475Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:50:59.939479Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342841088878880:4041]
2025-10-09T21:50:59.939488Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:50:59.939493Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:50:59.939509Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:50:59.939557Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 1999 reason: error at write: Disconnected sessionId: src_id|9c4b3f67-153653d4-485fb8c6-adb3e649_0
2025-10-09T21:50:59.939740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1999 sessionId: src_id|9c4b3f67-153653d4-485fb8c6-adb3e649_0 is DEAD
2025-10-09T21:50:59.939880Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c4b3f67-153653d4-485fb8c6-adb3e649_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:50:59.939900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c4b3f67-153653d4-485fb8c6-adb3e649_0] Write session will restart in 0.006329s
2025-10-09T21:50:59.939926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c4b3f67-153653d4-485fb8c6-adb3e649_0] Write session: Do CDS request
2025-10-09T21:50:59.939932Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c4b3f67-153653d4-485fb8c6-adb3e649_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:50:59.947471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c4b3f67-153653d4-485fb8c6-adb3e649_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:59.947533Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:50:59.947549Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2000
2025-10-09T21:50:59.947713Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2000 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:50:59.947745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2000 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:50:59.947750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2000 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:50:59.947753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:50:59.947874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:50:59.948084Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2000 sessionId:  describe result for acl check
2025-10-09T21:50:59.948112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:50:59.948117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:50:59.948120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:50:59.948123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342841088878929:5221] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:50:59.948591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342841088878929:5221] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:50:59.949439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:50:59.949605Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:50:59.949615Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:50:59.949626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:50:59.949679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:50:59.949715Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342840727045398:10867]
2025-10-09T21:50:59.950269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:50:59.950410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342841088878929:5221] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:50:59.950640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:50:59.950713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:50:59.951029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:50:59.951099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.951107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.951108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.951109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.951111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.951112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.951113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.951114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.951115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.951122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.951132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.951135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.951138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.951141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.951145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.951149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.951152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.951167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.951173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.951181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:50:59.951187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.951192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:50:59.951197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.951201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:50:59.951205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.951212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:50:59.951223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.951228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:50:59.951232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.951236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:50:59.951240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.951244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.951248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.951252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.951257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.951262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.951289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:50:59.956494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342841088878929:5221] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:50:59.956515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342841088878929:5221] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:50:59.956519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342841088878929:5221] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:50:59.956528Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2000 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:50:59.957230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:50:59.958414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:50:59.958429Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:50:59.958433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:50:59.958437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:50:59.958443Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1999 [2:7559342840727045398:10867]
2025-10-09T21:50:59.958455Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:50:59.958470Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:50:59.958477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:59.958480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:59.958482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:59.958501Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:50:59.958511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:50:59.958518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:50:59.958575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.958581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.958584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.958586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:50:59.958648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:50:59.958655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.958667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:50:59.958681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:50:59.958702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:50:59.958726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342841088878886:2432], now have 1 active actors on pipe
2025-10-09T21:50:59.958733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342841088878967:5221], now have 1 active actors on pipe
2025-10-09T21:50:59.958740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342840727045393:10867]
2025-10-09T21:50:59.958745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342840727045393:10867]
2025-10-09T21:50:59.958747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342840727045393:10867]
2025-10-09T21:50:59.958761Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:50:59.958752Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1999
2025-10-09T21:50:59.958776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:50:59.958783Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:50:59.958787Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:50:59.958795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:50:59.958795Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:50:59.958798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:50:59.958826Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342840727045393:10867]
2025-10-09T21:50:59.958835Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1999
2025-10-09T21:50:59.958840Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:50:59.958866Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:50:59.958901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.958909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.958935Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1930bc4b-cd19515e-851d0878-592a4595_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:50:59.958948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:50:59.958952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:50:59.958958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:50:59.958968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:50:59.958979Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:50:59.958993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:59.959175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.959183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.959197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:50:59.959283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2000 partition: 0 MaxSeqNo: 25 sessionId: src_id|1930bc4b-cd19515e-851d0878-592a4595_0
2025-10-09T21:50:59.959578Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c4b3f67-153653d4-485fb8c6-adb3e649_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1930bc4b-cd19515e-851d0878-592a4595_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:50:59.961870Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1930bc4b-cd19515e-851d0878-592a4595_0] Write session: try to update token
2025-10-09T21:50:59.961884Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1930bc4b-cd19515e-851d0878-592a4595_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:50:59.973807Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2000 sessionId: src_id|1930bc4b-cd19515e-851d0878-592a4595_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:50:59.975307Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:50:59.975323Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:59.975332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342840727045393:10867]
2025-10-09T21:50:59.975378Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:50:59.975386Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:50:59.975542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:50:59.975679Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:59.975820Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:59.975932Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:50:59.975945Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:50:59.976143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:50:59.976200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:50:59.976214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:50:59.976233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:50:59.976271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:50:59.976368Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:50:59.982129Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:50:59.982143Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:50:59.982146Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:50:59.982148Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:50:59.982157Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342840727045393:10867]
2025-10-09T21:50:59.982162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342840727045393:10867]
2025-10-09T21:50:59.982166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342840727045393:10867]
2025-10-09T21:50:59.982204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:50:59.982215Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:50:59.982217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:50:59.982220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:50:59.982221Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:50:59.982226Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:50:59.982348Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:00.039693Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342845383846271:4041]
2025-10-09T21:51:00.040001Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 1999, pipe: [1:7559342845383846271:4041]
2025-10-09T21:51:00.134949Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:00.142169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.142196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.142362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.142429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.142493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.142557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.142601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.142611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.142679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.142741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.142809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.142874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.143217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.143231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.143299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.143365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.143431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.143497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.143539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.143546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.143612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.143675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.143738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.143801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.143846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.143854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.143925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.143986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.144046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.144107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.144152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.144160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.144229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.144290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.144351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.144417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.144459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.144467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.144527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.144585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.144650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.144711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.144753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.144760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.144819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.144880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.144940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.144997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.145036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.145042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.145098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.145151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.145211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.145271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.145309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.145314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.145375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.145447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.145510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.145571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.145613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.145620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.145682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342845383846271:4041], now have 1 active actors on pipe
2025-10-09T21:51:00.145910Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:00.145929Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:00.145934Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:00.145966Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2000 reason: error at write: Disconnected sessionId: src_id|1930bc4b-cd19515e-851d0878-592a4595_0
2025-10-09T21:51:00.145971Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342845383846271:4041]
2025-10-09T21:51:00.146171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2000 sessionId: src_id|1930bc4b-cd19515e-851d0878-592a4595_0 is DEAD
2025-10-09T21:51:00.146336Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1930bc4b-cd19515e-851d0878-592a4595_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:00.146347Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1930bc4b-cd19515e-851d0878-592a4595_0] Write session will restart in 0.007180s
2025-10-09T21:51:00.146383Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1930bc4b-cd19515e-851d0878-592a4595_0] Write session: Do CDS request
2025-10-09T21:51:00.146390Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1930bc4b-cd19515e-851d0878-592a4595_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:00.146521Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:00.154522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1930bc4b-cd19515e-851d0878-592a4595_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:00.154602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:00.154617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2001
2025-10-09T21:51:00.154743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2001 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:00.154780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2001 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:00.154792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2001 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:00.154798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:00.155244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2001 sessionId:  describe result for acl check
2025-10-09T21:51:00.155282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:00.155293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:00.155295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:00.155300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342845383846316:5237] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:00.155633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:00.155879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342845383846316:5237] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:00.157524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:00.157707Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:00.157722Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:00.157734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:00.157791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:00.157838Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342845022012776:10869]
2025-10-09T21:51:00.158445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:00.158604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342845383846316:5237] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:00.158731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:00.158772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:00.159005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:00.159050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.159056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.159057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.159057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.159058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.159059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.159059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.159060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.159061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.159064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.159071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.159073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.159075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.159078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.159084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.159086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.159088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.159090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.159095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.159099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.159103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.159107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.159110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.159117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.159123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.159128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.159132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.159138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.159142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.159147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.159153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.159159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.159162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.159165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.159167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.159170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.159187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:00.164058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342845383846316:5237] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:00.164072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342845383846316:5237] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:00.164076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342845383846316:5237] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:00.164084Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2001 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:00.165432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:00.166481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:00.166498Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:00.166500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:00.166505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:00.166512Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2000 [2:7559342845022012776:10869]
2025-10-09T21:51:00.166525Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:00.166542Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:00.166545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:00.166548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:00.166550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:00.166568Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:00.166577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:00.166588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:00.166652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.166660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.166662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.166666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:00.166732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:00.166739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.166754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.166772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.166797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:00.166823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342845383846274:2432], now have 1 active actors on pipe
2025-10-09T21:51:00.166831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342845383846355:5237], now have 1 active actors on pipe
2025-10-09T21:51:00.166838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342845022012770:10869]
2025-10-09T21:51:00.166841Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342845022012770:10869]
2025-10-09T21:51:00.166843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342845022012770:10869]
2025-10-09T21:51:00.166855Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:00.166869Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:00.166873Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:00.166879Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:00.166881Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:00.166888Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:00.166866Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2000
2025-10-09T21:51:00.166867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2000
2025-10-09T21:51:00.166891Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:00.166928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342845022012770:10869]
2025-10-09T21:51:00.166942Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:00.166948Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:00.166957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.166962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.167010Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|392a25fc-68a4a45f-da99015f-42440594_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:00.167023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:00.167027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:00.167029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:00.167041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:00.167055Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:00.167074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:00.167246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.167255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.167268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:00.167370Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2001 partition: 0 MaxSeqNo: 25 sessionId: src_id|392a25fc-68a4a45f-da99015f-42440594_0
2025-10-09T21:51:00.167728Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1930bc4b-cd19515e-851d0878-592a4595_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|392a25fc-68a4a45f-da99015f-42440594_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:00.170150Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|392a25fc-68a4a45f-da99015f-42440594_0] Write session: try to update token
2025-10-09T21:51:00.170168Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|392a25fc-68a4a45f-da99015f-42440594_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:00.181720Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:00.181739Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:00.181741Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:00.181743Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:00.181752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342845022012770:10869]
2025-10-09T21:51:00.181758Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342845022012770:10869]
2025-10-09T21:51:00.181762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342845022012770:10869]
2025-10-09T21:51:00.181807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:00.181814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.181818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:00.181822Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:00.181828Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:00.181832Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:00.181959Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:00.183521Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2001 sessionId: src_id|392a25fc-68a4a45f-da99015f-42440594_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:00.186078Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:00.246047Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342845383846367:4041]
2025-10-09T21:51:00.246362Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 2000, pipe: [1:7559342845383846367:4041]
2025-10-09T21:51:00.344664Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:00.345049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.345073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.345121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342845383846367:4041], now have 1 active actors on pipe
2025-10-09T21:51:00.345319Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:00.345341Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342845383846367:4041]
2025-10-09T21:51:00.345377Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:00.345381Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:00.345429Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:51:00.345474Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2001 reason: status is not ok: tablet will be restarted right now sessionId: src_id|392a25fc-68a4a45f-da99015f-42440594_0
2025-10-09T21:51:00.345550Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2001 sessionId: src_id|392a25fc-68a4a45f-da99015f-42440594_0 is DEAD
2025-10-09T21:51:00.345608Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 20 consumer shared/user session shared/user_1_20_4254230261340021184_v1 is DEAD
2025-10-09T21:51:00.345726Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342759484476839:4037] disconnected.
2025-10-09T21:51:00.345733Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342759484476839:4037] disconnected; active server actors: 1
2025-10-09T21:51:00.345736Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342759484476839:4037] client user disconnected session shared/user_1_20_4254230261340021184_v1
2025-10-09T21:51:00.345789Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:51:00.345840Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|392a25fc-68a4a45f-da99015f-42440594_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:00.345850Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|392a25fc-68a4a45f-da99015f-42440594_0] Write session will restart in 0.009580s
2025-10-09T21:51:00.345875Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|392a25fc-68a4a45f-da99015f-42440594_0] Write session: Do CDS request
2025-10-09T21:51:00.345881Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|392a25fc-68a4a45f-da99015f-42440594_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:00.345960Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:51:00.345965Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:51:00.345974Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.005840s
2025-10-09T21:51:00.352401Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:51:00.352674Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:51:00.352687Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 21
2025-10-09T21:51:00.352879Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 21 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:51:00.352938Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:51:00.353034Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 auth for : user
2025-10-09T21:51:00.353220Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 Handle describe topics response
2025-10-09T21:51:00.353256Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 auth is DEAD
2025-10-09T21:51:00.353275Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:51:00.353579Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:51:00.353689Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_21_4029661093892576224_v1
2025-10-09T21:51:00.353749Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342845383846416:5242] connected; active server actors: 1
2025-10-09T21:51:00.353768Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559342845383846416:5242] session shared/user_1_21_4029661093892576224_v1
2025-10-09T21:51:00.353779Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:51:00.353789Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:51:00.353796Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_21_4029661093892576224_v1" (Sender=[1:7559342845383846395:5242], Pipe=[1:7559342845383846416:5242], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:51:00.353801Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:51:00.353812Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:51:00.353821Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_21_4029661093892576224_v1" (Sender=[1:7559342845383846395:5242], Pipe=[1:7559342845383846416:5242], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:51:00.353834Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_21_4029661093892576224_v1" sender [1:7559342845383846395:5242] lock partition 0 for ReadingSession "shared/user_1_21_4029661093892576224_v1" (Sender=[1:7559342845383846395:5242], Pipe=[1:7559342845383846416:5242], Partitions=[], ActiveFamilyCount=1) generation 1 step 21
2025-10-09T21:51:00.353849Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:51:00.353854Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000038s
2025-10-09T21:51:00.354103Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 21 Session: "shared/user_1_21_4029661093892576224_v1" ClientId: "user" PipeClient { RawX1: 7559342845383846416 RawX2: 4503603922343034 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:51:00.354138Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:51:00.354653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:00.356522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|392a25fc-68a4a45f-da99015f-42440594_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:00.356588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:00.356600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2002
2025-10-09T21:51:00.356727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:00.356741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2002 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:00.356787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2002 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:00.356799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2002 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:00.356803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:00.356965Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:00.356976Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:00.356988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:00.357051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:00.357098Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342845022012857:10871]
2025-10-09T21:51:00.357211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2002 sessionId:  describe result for acl check
2025-10-09T21:51:00.357245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:00.357252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:00.357254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:00.357260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342845383846426:5252] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:00.357700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:00.357770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342845383846426:5252] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:00.358068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:00.358130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:00.358444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:00.358493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.358499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.358500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.358502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.358503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.358504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.358505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.358506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.358508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.358513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.358523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.358527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.358531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.358535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.358539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.358542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.358546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.358550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.358558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.358571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.358578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.358588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.358596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.358613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.358624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.358629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.358634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.358639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.358643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.358648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.358654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.358659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.358664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.358669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.358674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.358686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.358717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:00.360330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342845383846426:5252] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:00.365002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342845383846426:5252] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:00.365021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342845383846426:5252] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:00.365025Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342845383846426:5252] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:00.365035Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2002 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:00.365457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:00.366577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:00.366593Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:00.366596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:00.366600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:00.366606Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2001 [2:7559342845022012857:10871]
2025-10-09T21:51:00.366617Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:00.366636Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:00.366639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:00.366642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:00.366644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:00.366667Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:00.366673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:00.366679Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:00.366729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.366737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.366738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.366740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:00.366800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:00.366808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.366819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.366829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.366843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:00.366870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342845022012852:10871]
2025-10-09T21:51:00.366878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342845022012852:10871]
2025-10-09T21:51:00.366879Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342845022012852:10871]
2025-10-09T21:51:00.366892Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:00.366906Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:00.366912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342845383846374:2432], now have 1 active actors on pipe
2025-10-09T21:51:00.366915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342845383846464:5252], now have 1 active actors on pipe
2025-10-09T21:51:00.366916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342845383846420:5250], now have 1 active actors on pipe
2025-10-09T21:51:00.366917Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:00.366920Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:00.366923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:00.366932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:00.366934Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:00.366974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342845022012852:10871]
2025-10-09T21:51:00.366987Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:00.366975Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2001
2025-10-09T21:51:00.366980Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2001
2025-10-09T21:51:00.367015Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:00.367028Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 2001, pipe: [1:7559342845383846420:5250]
2025-10-09T21:51:00.367055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.367066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.367086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.367092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.367099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_21_4029661093892576224_v1 on pipe: [1:7559342845383846420:5250]
2025-10-09T21:51:00.367136Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c2867d30-d09ebe99-f37c6a38-31cee7ae_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:00.367140Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_21_4029661093892576224_v1:1 with generation 2001
2025-10-09T21:51:00.367150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:00.367154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:00.367156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:00.367169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:00.367180Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:00.367191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:00.367195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:00.367196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:00.367209Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_21_4029661093892576224_v1
2025-10-09T21:51:00.367214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:00.367237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:00.367255Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:51:00.367439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.367447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.367462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:00.367550Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2002 partition: 0 MaxSeqNo: 25 sessionId: src_id|c2867d30-d09ebe99-f37c6a38-31cee7ae_0
2025-10-09T21:51:00.367883Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|392a25fc-68a4a45f-da99015f-42440594_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c2867d30-d09ebe99-f37c6a38-31cee7ae_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:00.370414Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c2867d30-d09ebe99-f37c6a38-31cee7ae_0] Write session: try to update token
2025-10-09T21:51:00.370433Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c2867d30-d09ebe99-f37c6a38-31cee7ae_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:00.371938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:51:00.371983Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:51:00.372005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:00.372012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:00.372015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:00.372025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:51:00.372045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:51:00.372260Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046660366 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:51:00.372280Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:51:00.372316Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 sending to client partition status
2025-10-09T21:51:00.379527Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:00.379549Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:00.379559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342845022012852:10871]
2025-10-09T21:51:00.379614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:00.379621Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:00.379789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:00.379938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:00.380077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:00.380215Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:00.380400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:00.380446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:00.384869Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2002 sessionId: src_id|c2867d30-d09ebe99-f37c6a38-31cee7ae_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:00.387598Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:00.387848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.387872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.387921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:00.388011Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:00.393079Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:00.393099Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:00.393102Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:00.393105Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:00.393116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342845022012852:10871]
2025-10-09T21:51:00.393121Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342845022012852:10871]
2025-10-09T21:51:00.393125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342845022012852:10871]
2025-10-09T21:51:00.393173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:00.393187Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:00.393190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.393192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:00.393193Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:00.393198Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:00.393336Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:00.552216Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:00.561319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.561355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.561554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.561617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.561684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.561754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.561797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.561802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.561878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.561942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.562015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.562201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.562248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.562256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.562323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.562398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.562465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.562532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.562577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.562585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.562656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.562734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.562805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.562867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.562918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.562921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.562993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.563062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.563140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.563209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.563254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.563262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.563336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.563398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.563481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.563543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.563589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.563598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.563666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.563730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.563800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.563864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.563911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.563918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.563988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.564055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.564126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.564199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.564254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.564262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.564334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.564401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.564468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.564585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.564639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.564649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.564722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:00.564787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:00.564857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:00.564932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:00.564981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:00.564995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:00.565356Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342845383846420:5250]
2025-10-09T21:51:00.565401Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:00.565515Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:00.565523Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:00.565572Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2002 reason: error at write: Disconnected sessionId: src_id|c2867d30-d09ebe99-f37c6a38-31cee7ae_0
2025-10-09T21:51:00.565755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2002 sessionId: src_id|c2867d30-d09ebe99-f37c6a38-31cee7ae_0 is DEAD
2025-10-09T21:51:00.565905Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2867d30-d09ebe99-f37c6a38-31cee7ae_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:00.565915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2867d30-d09ebe99-f37c6a38-31cee7ae_0] Write session will restart in 0.009475s
2025-10-09T21:51:00.565941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2867d30-d09ebe99-f37c6a38-31cee7ae_0] Write session: Do CDS request
2025-10-09T21:51:00.565948Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2867d30-d09ebe99-f37c6a38-31cee7ae_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:00.566286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_21_4029661093892576224_v1
2025-10-09T21:51:00.566303Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:00.566730Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_21_4029661093892576224_v1
2025-10-09T21:51:00.576599Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c2867d30-d09ebe99-f37c6a38-31cee7ae_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:00.576663Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:00.576680Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2003
2025-10-09T21:51:00.576793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:00.576827Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2003 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:00.576870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2003 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:00.576886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2003 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:00.576889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:00.577292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2003 sessionId:  describe result for acl check
2025-10-09T21:51:00.577337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:00.577344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:00.577345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:00.577351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342845383846544:5281] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:00.578195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342845383846544:5281] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:00.578630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:00.578843Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:00.578852Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:00.578865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:00.578917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:00.578950Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342845022012980:10879]
2025-10-09T21:51:00.579471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:00.579762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:00.579803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:00.580086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:00.580132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.580139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.580140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.580141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.580142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.580143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.580144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.580145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.580146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.580150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.580159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.580162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.580166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.580169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.580172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.580175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.580178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.580181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.580188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.580194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.580200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.580204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.580210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.580214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.580217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.580222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.580225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.580230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.580233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.580237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.580241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.580245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.580248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.580252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.580256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.580259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.580283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:00.580684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342845383846544:5281] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:00.586572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342845383846544:5281] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:00.586592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342845383846544:5281] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:00.586597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342845383846544:5281] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:00.586610Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2003 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:00.587044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:00.588369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:00.588386Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:00.588388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:00.588393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:00.588400Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2002 [2:7559342845022012980:10879]
2025-10-09T21:51:00.588415Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:00.588433Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:00.588436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:00.588440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:00.588442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:00.588462Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:00.588474Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:00.588483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:00.588551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.588559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.588561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.588563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:00.588644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:00.588653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.588667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.588680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.588705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:00.588728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342845383846488:2432], now have 1 active actors on pipe
2025-10-09T21:51:00.588740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342845383846580:5281], now have 1 active actors on pipe
2025-10-09T21:51:00.588749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342845022012973:10879]
2025-10-09T21:51:00.588750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342845022012973:10879]
2025-10-09T21:51:00.588752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342845022012973:10879]
2025-10-09T21:51:00.588768Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:00.588783Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:00.588771Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2002
2025-10-09T21:51:00.588781Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2002
2025-10-09T21:51:00.588799Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:00.588798Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:00.588803Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:00.588805Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:00.588808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:00.588844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342845022012973:10879]
2025-10-09T21:51:00.588856Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:00.588863Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:00.588874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.588879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.588934Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b64065ba-53b2e588-b2db79aa-7597cd61_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:00.588949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:00.588952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:00.588955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:00.588966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:00.588982Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:00.589011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:00.589168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.589175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.589189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:00.589268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2003 partition: 0 MaxSeqNo: 25 sessionId: src_id|b64065ba-53b2e588-b2db79aa-7597cd61_0
2025-10-09T21:51:00.589622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2867d30-d09ebe99-f37c6a38-31cee7ae_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b64065ba-53b2e588-b2db79aa-7597cd61_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:00.592161Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b64065ba-53b2e588-b2db79aa-7597cd61_0] Write session: try to update token
2025-10-09T21:51:00.592182Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b64065ba-53b2e588-b2db79aa-7597cd61_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:00.604551Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:00.604573Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:00.604576Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:00.604579Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:00.604588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342845022012973:10879]
2025-10-09T21:51:00.604593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342845022012973:10879]
2025-10-09T21:51:00.604598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342845022012973:10879]
2025-10-09T21:51:00.604645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:00.604652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.604654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:00.604659Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:00.604665Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:00.604668Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:00.604800Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:00.606200Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2003 sessionId: src_id|b64065ba-53b2e588-b2db79aa-7597cd61_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:00.608818Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:00.665341Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342845383846588:5250]
2025-10-09T21:51:00.665622Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 2002, pipe: [1:7559342845383846588:5250]
2025-10-09T21:51:00.768390Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:00.768802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.768820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.768870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342845383846588:5250], now have 1 active actors on pipe
2025-10-09T21:51:00.769091Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342845383846588:5250]
2025-10-09T21:51:00.769111Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:00.769128Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:00.769131Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:00.769207Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2003 reason: error at write: Disconnected sessionId: src_id|b64065ba-53b2e588-b2db79aa-7597cd61_0
2025-10-09T21:51:00.769385Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2003 sessionId: src_id|b64065ba-53b2e588-b2db79aa-7597cd61_0 is DEAD
2025-10-09T21:51:00.769418Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:00.769541Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b64065ba-53b2e588-b2db79aa-7597cd61_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:00.769554Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b64065ba-53b2e588-b2db79aa-7597cd61_0] Write session will restart in 0.009511s
2025-10-09T21:51:00.769582Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b64065ba-53b2e588-b2db79aa-7597cd61_0] Write session: Do CDS request
2025-10-09T21:51:00.769589Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b64065ba-53b2e588-b2db79aa-7597cd61_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:00.779817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:00.780595Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b64065ba-53b2e588-b2db79aa-7597cd61_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:00.780639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:00.780656Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2004
2025-10-09T21:51:00.780844Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2004 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:00.780894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2004 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:00.780904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2004 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:00.780908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:00.781314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2004 sessionId:  describe result for acl check
2025-10-09T21:51:00.781352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:00.781359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:00.781362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:00.781368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342845383846637:5289] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:00.781649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:00.781839Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:00.781861Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:00.781873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:00.781905Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342845383846637:5289] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:00.781924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:00.781967Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342845022013069:10889]
2025-10-09T21:51:00.782555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:00.782938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:00.782988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:00.783305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:00.783370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.783377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.783379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.783380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.783381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.783383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.783384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.783385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.783387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.783392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.783402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.783407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.783410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.783414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.783417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.783423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.783426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.783430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.783436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.783444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.783450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.783455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.783460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.783465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.783469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.783474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.783479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.783485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.783497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.783502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.783512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.783518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.783527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.783532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.783537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.783546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.783574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:00.784028Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342845383846637:5289] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:00.789858Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342845383846637:5289] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:00.789877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342845383846637:5289] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:00.789881Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342845383846637:5289] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:00.789891Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2004 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:00.790106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:00.791158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:00.791173Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:00.791176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:00.791180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:00.791187Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2003 [2:7559342845022013069:10889]
2025-10-09T21:51:00.791200Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:00.791228Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:00.791231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:00.791237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:00.791239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:00.791257Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:00.791268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:00.791275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:00.791327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.791333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.791336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.791338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:00.791450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:00.791457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.791468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.791483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.791504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:00.791528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342845383846591:2432], now have 1 active actors on pipe
2025-10-09T21:51:00.791536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342845383846675:5289], now have 1 active actors on pipe
2025-10-09T21:51:00.791543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342845022013062:10889]
2025-10-09T21:51:00.791545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342845022013062:10889]
2025-10-09T21:51:00.791547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342845022013062:10889]
2025-10-09T21:51:00.791559Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:00.791576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:00.791589Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:00.791592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:00.791598Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:00.791608Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:00.791556Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2003
2025-10-09T21:51:00.791611Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:00.791576Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2003
2025-10-09T21:51:00.791624Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342845022013062:10889]
2025-10-09T21:51:00.791627Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:00.791638Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:00.791657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.791667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.791698Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|db121689-c1337f25-9f2ba88a-d1d04ea6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:00.791706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:00.791713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:00.791715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:00.791724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:00.791733Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:00.791749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:00.791898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.791904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.791916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:00.792015Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2004 partition: 0 MaxSeqNo: 25 sessionId: src_id|db121689-c1337f25-9f2ba88a-d1d04ea6_0
2025-10-09T21:51:00.792381Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b64065ba-53b2e588-b2db79aa-7597cd61_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|db121689-c1337f25-9f2ba88a-d1d04ea6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:00.794862Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db121689-c1337f25-9f2ba88a-d1d04ea6_0] Write session: try to update token
2025-10-09T21:51:00.794882Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db121689-c1337f25-9f2ba88a-d1d04ea6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:00.806432Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:00.806455Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:00.806457Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:00.806459Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:00.806470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342845022013062:10889]
2025-10-09T21:51:00.806474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342845022013062:10889]
2025-10-09T21:51:00.806477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342845022013062:10889]
2025-10-09T21:51:00.806519Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:00.806522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:00.806528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.806529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:00.806535Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:00.806540Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:00.806660Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:00.808362Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2004 sessionId: src_id|db121689-c1337f25-9f2ba88a-d1d04ea6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:00.810853Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:00.869112Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342845383846682:5250]
2025-10-09T21:51:00.869426Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 2003, pipe: [1:7559342845383846682:5250]
2025-10-09T21:51:00.963705Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:00.964044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.964058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.964104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342845383846682:5250], now have 1 active actors on pipe
2025-10-09T21:51:00.964316Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:00.964330Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:00.964359Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:00.964363Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:00.964383Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342845383846682:5250]
2025-10-09T21:51:00.964436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2004 reason: error at write: Disconnected sessionId: src_id|db121689-c1337f25-9f2ba88a-d1d04ea6_0
2025-10-09T21:51:00.964691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2004 sessionId: src_id|db121689-c1337f25-9f2ba88a-d1d04ea6_0 is DEAD
2025-10-09T21:51:00.964812Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db121689-c1337f25-9f2ba88a-d1d04ea6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:00.964824Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db121689-c1337f25-9f2ba88a-d1d04ea6_0] Write session will restart in 0.008389s
2025-10-09T21:51:00.964851Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db121689-c1337f25-9f2ba88a-d1d04ea6_0] Write session: Do CDS request
2025-10-09T21:51:00.964857Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db121689-c1337f25-9f2ba88a-d1d04ea6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:00.974323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:00.974497Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|db121689-c1337f25-9f2ba88a-d1d04ea6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:00.974552Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:00.974565Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2005
2025-10-09T21:51:00.974728Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2005 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:00.974761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2005 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:00.974772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2005 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:00.974775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:00.975235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2005 sessionId:  describe result for acl check
2025-10-09T21:51:00.975285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:00.975288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:00.975290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:00.975296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342845383846736:5296] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:00.975847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342845383846736:5296] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:00.975895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:00.976065Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:00.976073Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:00.976083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:00.976141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:00.976183Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342845022013151:10891]
2025-10-09T21:51:00.976632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:00.976925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:00.976991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:00.977317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:00.977374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.977378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.977379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.977381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.977382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.977383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.977385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.977386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.977387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.977392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.977401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.977415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.977418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.977422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.977426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.977430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.977432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.977435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.977442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.977449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:00.977456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.977461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:00.977466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.977471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:00.977475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.977480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:00.977484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.977489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:00.977495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.977500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:00.977505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.977509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.977515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.977521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.977526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.977536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.977565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:00.978109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342845383846736:5296] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:00.983675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342845383846736:5296] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:00.983688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342845383846736:5296] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:00.983692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342845383846736:5296] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:00.983703Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2005 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:00.984020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:00.985128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:00.985147Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:00.985150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:00.985154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:00.985161Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2004 [2:7559342845022013151:10891]
2025-10-09T21:51:00.985175Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:00.985193Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:00.985196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:00.985199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:00.985201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:00.985218Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:00.985229Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:00.985236Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:00.985297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.985304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.985306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.985309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:00.985376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:00.985384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.985396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:00.985416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:00.985437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:00.985464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342845383846690:2432], now have 1 active actors on pipe
2025-10-09T21:51:00.985468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342845383846772:5296], now have 1 active actors on pipe
2025-10-09T21:51:00.985475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342845022013146:10891]
2025-10-09T21:51:00.985478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342845022013146:10891]
2025-10-09T21:51:00.985481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342845022013146:10891]
2025-10-09T21:51:00.985497Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:00.985516Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:00.985526Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:00.985530Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:00.985534Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:00.985515Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2004
2025-10-09T21:51:00.985535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:00.985525Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2004
2025-10-09T21:51:00.985538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:00.985569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342845022013146:10891]
2025-10-09T21:51:00.985572Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:00.985575Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:00.985596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.985601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.985650Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|13f7e71-b849f7c5-40e2be84-8e03516c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:00.985662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:00.985665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:00.985672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:00.985683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:00.985693Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:00.985712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:00.985855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:00.985869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:00.985892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:00.985960Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2005 partition: 0 MaxSeqNo: 25 sessionId: src_id|13f7e71-b849f7c5-40e2be84-8e03516c_0
2025-10-09T21:51:00.986305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|db121689-c1337f25-9f2ba88a-d1d04ea6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|13f7e71-b849f7c5-40e2be84-8e03516c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:00.988859Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|13f7e71-b849f7c5-40e2be84-8e03516c_0] Write session: try to update token
2025-10-09T21:51:00.988878Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|13f7e71-b849f7c5-40e2be84-8e03516c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:00.999870Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:00.999887Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:00.999889Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:00.999890Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:00.999901Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342845022013146:10891]
2025-10-09T21:51:00.999904Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342845022013146:10891]
2025-10-09T21:51:00.999910Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342845022013146:10891]
2025-10-09T21:51:00.999947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:00.999952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:00.999954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:00.999967Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:00.999974Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:00.999977Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:01.000086Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:01.002495Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2005 sessionId: src_id|13f7e71-b849f7c5-40e2be84-8e03516c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:01.005069Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:01.064699Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342849678814077:5250]
2025-10-09T21:51:01.065110Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 2004, pipe: [1:7559342849678814077:5250]
2025-10-09T21:51:01.155459Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:01.155776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.155793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.155839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342849678814077:5250], now have 1 active actors on pipe
2025-10-09T21:51:01.156076Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:01.156130Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:01.156137Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342849678814077:5250]
2025-10-09T21:51:01.156160Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:01.156247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2005 reason: status is not ok: tablet will be restarted right now sessionId: src_id|13f7e71-b849f7c5-40e2be84-8e03516c_0
2025-10-09T21:51:01.156433Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2005 sessionId: src_id|13f7e71-b849f7c5-40e2be84-8e03516c_0 is DEAD
2025-10-09T21:51:01.156618Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13f7e71-b849f7c5-40e2be84-8e03516c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:01.156630Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13f7e71-b849f7c5-40e2be84-8e03516c_0] Write session will restart in 0.007928s
2025-10-09T21:51:01.156672Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13f7e71-b849f7c5-40e2be84-8e03516c_0] Write session: Do CDS request
2025-10-09T21:51:01.156678Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13f7e71-b849f7c5-40e2be84-8e03516c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:01.165541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|13f7e71-b849f7c5-40e2be84-8e03516c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:01.165642Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:01.165658Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2006
2025-10-09T21:51:01.165796Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2006 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:01.165830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2006 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:01.165841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2006 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:01.165845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:01.165929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:01.166286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2006 sessionId:  describe result for acl check
2025-10-09T21:51:01.166320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:01.166326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:01.166330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:01.166335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342849678814129:5299] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:01.166864Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342849678814129:5299] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:01.167620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:01.167808Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:01.167818Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:01.167830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:01.167883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:01.167921Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342849316980535:10893]
2025-10-09T21:51:01.168512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:01.168786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:01.168829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:01.168901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342849678814129:5299] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:01.169137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:01.169220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.169226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.169227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.169228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.169229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.169230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.169232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.169233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.169234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.169240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.169250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.169254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.169258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.169262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.169266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.169270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.169274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.169277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.169284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.169292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.169298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.169303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.169308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.169313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.169318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.169323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.169328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.169334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.169344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.169350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.169361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.169370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.169375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.169381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.169386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.169391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.169432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:01.174463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342849678814129:5299] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:01.174479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342849678814129:5299] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:01.174483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342849678814129:5299] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:01.174492Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2006 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:01.175885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:01.176877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:01.176891Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:01.176893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:01.176895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:01.176901Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2005 [2:7559342849316980535:10893]
2025-10-09T21:51:01.176911Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:01.176927Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:01.176929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:01.176931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:01.176933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:01.176946Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:01.176956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:01.176962Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:01.177008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.177013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.177015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.177016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:01.177069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:01.177074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.177082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.177092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.177107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:01.177128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342849316980529:10893]
2025-10-09T21:51:01.177136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342849316980529:10893]
2025-10-09T21:51:01.177138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342849316980529:10893]
2025-10-09T21:51:01.177150Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:01.177163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342849678814168:5299], now have 1 active actors on pipe
2025-10-09T21:51:01.177165Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:01.177169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342849678814081:2432], now have 1 active actors on pipe
2025-10-09T21:51:01.177169Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:01.177171Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:01.177174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:01.177187Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:01.177191Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:01.177222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342849316980529:10893]
2025-10-09T21:51:01.177235Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:01.177249Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:01.177259Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2005
2025-10-09T21:51:01.177261Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2005
2025-10-09T21:51:01.177390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.177400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.177437Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9ead3eee-33486412-c3d03de4-394ea6dc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:01.177443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:01.177447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:01.177449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:01.177467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:01.177480Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:01.177494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:01.177629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.177635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.177656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:01.177742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2006 partition: 0 MaxSeqNo: 25 sessionId: src_id|9ead3eee-33486412-c3d03de4-394ea6dc_0
2025-10-09T21:51:01.178088Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13f7e71-b849f7c5-40e2be84-8e03516c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9ead3eee-33486412-c3d03de4-394ea6dc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:01.180495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ead3eee-33486412-c3d03de4-394ea6dc_0] Write session: try to update token
2025-10-09T21:51:01.180514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ead3eee-33486412-c3d03de4-394ea6dc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:01.189245Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:01.189263Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:01.189272Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342849316980529:10893]
2025-10-09T21:51:01.189319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:01.189322Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:01.189502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:01.189646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:01.189813Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:01.189960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:01.190180Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:01.190228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:01.193222Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2006 sessionId: src_id|9ead3eee-33486412-c3d03de4-394ea6dc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:01.195639Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:01.195859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.195879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.195918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:01.195994Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:01.202646Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:01.202664Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:01.202667Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:01.202669Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:01.202680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342849316980529:10893]
2025-10-09T21:51:01.202685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342849316980529:10893]
2025-10-09T21:51:01.202691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342849316980529:10893]
2025-10-09T21:51:01.202731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:01.202745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.202746Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:01.202753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:01.202753Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:01.202758Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:01.202881Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:01.256190Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342849678814180:5250]
2025-10-09T21:51:01.256467Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 2005, pipe: [1:7559342849678814180:5250]
2025-10-09T21:51:01.361288Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:01.368910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.368938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.369104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.369173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.369245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.369311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.369365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.369376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.369457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.369519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.369579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.369639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.369677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.369684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.369740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.369790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.369842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.369895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.369936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.369942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.370000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.370076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.370137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.370203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.370246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.370257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.370322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.370385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.370449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.370511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.370557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.370562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.370624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.370684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.370748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.370811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.370856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.370863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.370926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.370983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.371044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.371104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.371145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.371152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.371212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.371268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.371328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.371388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.371432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.371439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.371497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.371552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.371611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.371671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.371789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.371799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.371860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.371920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.371980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.372040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.372081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.372088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.372148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342849678814180:5250], now have 1 active actors on pipe
2025-10-09T21:51:01.372381Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:01.372383Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342849678814180:5250]
2025-10-09T21:51:01.372393Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:01.372405Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:01.372424Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2006 reason: error at write: Disconnected sessionId: src_id|9ead3eee-33486412-c3d03de4-394ea6dc_0
2025-10-09T21:51:01.372609Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2006 sessionId: src_id|9ead3eee-33486412-c3d03de4-394ea6dc_0 is DEAD
2025-10-09T21:51:01.372783Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ead3eee-33486412-c3d03de4-394ea6dc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:01.372793Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ead3eee-33486412-c3d03de4-394ea6dc_0] Write session will restart in 0.008730s
2025-10-09T21:51:01.372819Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ead3eee-33486412-c3d03de4-394ea6dc_0] Write session: Do CDS request
2025-10-09T21:51:01.372824Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ead3eee-33486412-c3d03de4-394ea6dc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:01.373108Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:01.382323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:01.382531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ead3eee-33486412-c3d03de4-394ea6dc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:01.382611Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:01.382624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2007
2025-10-09T21:51:01.382756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2007 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:01.382789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2007 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:01.382794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2007 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:01.382798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:01.383255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2007 sessionId:  describe result for acl check
2025-10-09T21:51:01.383294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:01.383300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:01.383302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:01.383308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342849678814236:5320] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:01.383862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342849678814236:5320] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:01.384178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:01.384367Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:01.384379Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:01.384391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:01.384450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:01.384494Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342849316980621:10895]
2025-10-09T21:51:01.385074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:01.385425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:01.385488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:01.385808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:01.385868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.385877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.385878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.385880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.385881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.385882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.385884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.385885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.385886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.385892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.385902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.385905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.385909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.385913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.385918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.385922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.385926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.385932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.385939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.385946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.385959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.385965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.385969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.385974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.385979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.385984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.385988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.385993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.385997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.386001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.386006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.386012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.386016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.386020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.386024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.386050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.386079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:01.386713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342849678814236:5320] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:01.392198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342849678814236:5320] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:01.392209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342849678814236:5320] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:01.392212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342849678814236:5320] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:01.392219Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2007 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:01.392681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:01.393774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:01.393790Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:01.393793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:01.393797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:01.393805Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2006 [2:7559342849316980621:10895]
2025-10-09T21:51:01.393818Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:01.393835Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:01.393843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:01.393846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:01.393849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:01.393871Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:01.393882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:01.393888Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:01.393953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.393961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.394088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.394097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:01.394172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:01.394181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.394195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.394207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.394228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:01.394251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342849678814187:2432], now have 1 active actors on pipe
2025-10-09T21:51:01.394260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342849678814274:5320], now have 1 active actors on pipe
2025-10-09T21:51:01.394267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342849316980615:10895]
2025-10-09T21:51:01.394269Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342849316980615:10895]
2025-10-09T21:51:01.394272Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342849316980615:10895]
2025-10-09T21:51:01.394287Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:01.394309Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:01.394310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:01.394312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:01.394315Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:01.394324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:01.394327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:01.394366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342849316980615:10895]
2025-10-09T21:51:01.394375Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:01.394380Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:01.394391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.394400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.394460Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|36c0d883-2d46a21e-dc4a4266-7568548_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:01.394474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:01.394478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:01.394483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:01.394495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:01.394510Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:01.394539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:01.394318Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2006
2025-10-09T21:51:01.394322Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2006
2025-10-09T21:51:01.394693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.394704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.394722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:01.394811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2007 partition: 0 MaxSeqNo: 25 sessionId: src_id|36c0d883-2d46a21e-dc4a4266-7568548_0
2025-10-09T21:51:01.395177Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ead3eee-33486412-c3d03de4-394ea6dc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|36c0d883-2d46a21e-dc4a4266-7568548_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:01.397604Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36c0d883-2d46a21e-dc4a4266-7568548_0] Write session: try to update token
2025-10-09T21:51:01.397622Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36c0d883-2d46a21e-dc4a4266-7568548_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:01.409016Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:01.409034Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:01.409037Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:01.409038Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:01.409049Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342849316980615:10895]
2025-10-09T21:51:01.409057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342849316980615:10895]
2025-10-09T21:51:01.409061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342849316980615:10895]
2025-10-09T21:51:01.409107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:01.409109Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:01.409117Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:01.409119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.409120Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:01.409122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:01.409249Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:01.410535Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2007 sessionId: src_id|36c0d883-2d46a21e-dc4a4266-7568548_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:01.412963Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:01.472783Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342849678814283:5250]
2025-10-09T21:51:01.473088Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 2006, pipe: [1:7559342849678814283:5250]
2025-10-09T21:51:01.559921Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:01.560224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.560238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.560282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342849678814283:5250], now have 1 active actors on pipe
2025-10-09T21:51:01.560498Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:01.560526Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342849678814283:5250]
2025-10-09T21:51:01.560527Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:01.560561Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:01.560653Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2007 reason: status is not ok: tablet will be restarted right now sessionId: src_id|36c0d883-2d46a21e-dc4a4266-7568548_0
2025-10-09T21:51:01.560837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2007 sessionId: src_id|36c0d883-2d46a21e-dc4a4266-7568548_0 is DEAD
2025-10-09T21:51:01.561004Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36c0d883-2d46a21e-dc4a4266-7568548_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:01.561016Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36c0d883-2d46a21e-dc4a4266-7568548_0] Write session will restart in 0.006764s
2025-10-09T21:51:01.561047Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36c0d883-2d46a21e-dc4a4266-7568548_0] Write session: Do CDS request
2025-10-09T21:51:01.561052Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36c0d883-2d46a21e-dc4a4266-7568548_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:01.568521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36c0d883-2d46a21e-dc4a4266-7568548_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:01.568603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:01.568616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2008
2025-10-09T21:51:01.568743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2008 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:01.568777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2008 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:01.568782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2008 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:01.568786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:01.569163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2008 sessionId:  describe result for acl check
2025-10-09T21:51:01.569194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:01.569200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:01.569202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:01.569207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342849678814333:5330] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:01.569698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342849678814333:5330] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:01.570060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:01.571750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:01.571856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342849678814333:5330] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:01.571932Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:01.571944Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:01.571958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:01.572030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:01.572070Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342849316980707:10897]
2025-10-09T21:51:01.572632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:01.572896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:01.572938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:01.573239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:01.573296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.573302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.573303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.573304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.573306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.573308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.573309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.573310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.573311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.573316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.573326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.573330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.573334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.573338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.573342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.573346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.573349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.573353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.573359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.573368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.573374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.573380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.573386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.573391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.573395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.573400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.573417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.573424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.573428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.573433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.573438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.573445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.573449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.573455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.573464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.573469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.573497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:01.574353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342849678814333:5330] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:01.574364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342849678814333:5330] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:01.574368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342849678814333:5330] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:01.574376Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2008 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:01.579697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:01.580776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:01.580799Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:01.580801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:01.580806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:01.580813Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2007 [2:7559342849316980707:10897]
2025-10-09T21:51:01.580826Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:01.580842Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:01.580845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:01.580847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:01.580850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:01.580866Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:01.580872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:01.580887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:01.580945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.580956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.580958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.580960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:01.581023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:01.581030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.581041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.581055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.581071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:01.581091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342849678814375:5330], now have 1 active actors on pipe
2025-10-09T21:51:01.581100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342849678814285:2432], now have 1 active actors on pipe
2025-10-09T21:51:01.581107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342849316980702:10897]
2025-10-09T21:51:01.581108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342849316980702:10897]
2025-10-09T21:51:01.581110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342849316980702:10897]
2025-10-09T21:51:01.581125Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:01.581139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:01.581147Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:01.581151Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:01.581154Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:01.581157Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:01.581160Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:01.581174Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2007
2025-10-09T21:51:01.581205Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2007
2025-10-09T21:51:01.581189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342849316980702:10897]
2025-10-09T21:51:01.581205Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:01.581228Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:01.581249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.581258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.581287Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4af7f0a3-f54a281d-1bd5a1fe-40486cbf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:01.581298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:01.581304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:01.581306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:01.581316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:01.581330Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:01.581348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:01.581535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.581542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.581556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:01.581666Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2008 partition: 0 MaxSeqNo: 25 sessionId: src_id|4af7f0a3-f54a281d-1bd5a1fe-40486cbf_0
2025-10-09T21:51:01.582083Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36c0d883-2d46a21e-dc4a4266-7568548_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4af7f0a3-f54a281d-1bd5a1fe-40486cbf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:01.584458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4af7f0a3-f54a281d-1bd5a1fe-40486cbf_0] Write session: try to update token
2025-10-09T21:51:01.584474Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4af7f0a3-f54a281d-1bd5a1fe-40486cbf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:01.598388Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:01.598405Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:01.598413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342849316980702:10897]
2025-10-09T21:51:01.598463Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:01.598471Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:01.598491Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2008 sessionId: src_id|4af7f0a3-f54a281d-1bd5a1fe-40486cbf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:01.598635Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:01.598779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:01.598931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:01.599080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:01.599423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:01.599476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:01.600883Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:01.601131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.601162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.601201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:01.601293Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:01.605762Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:01.605776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:01.605778Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:01.605780Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:01.605790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342849316980702:10897]
2025-10-09T21:51:01.605794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342849316980702:10897]
2025-10-09T21:51:01.605797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342849316980702:10897]
2025-10-09T21:51:01.605839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:01.605846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.605848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:01.605868Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:01.605874Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:01.605878Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:01.605988Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:01.660679Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342849678814404:5250]
2025-10-09T21:51:01.660970Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 2007, pipe: [1:7559342849678814404:5250]
2025-10-09T21:51:01.755384Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:01.762763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.762792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.762949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.763014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.763071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.763126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.763464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.763478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.763566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.763645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.763724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.763806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.763856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.763869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.763939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.764004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.764073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.764136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.764285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.764296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.764357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.764414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.764469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.764546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.764589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.764594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.764652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.764703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.764758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.764817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.764864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.764872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.764932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.764985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.765039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.765096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.765136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.765142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.765200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.765253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.765362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.765477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.765515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.765521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.765634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.765687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.765743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.765795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.765833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.765839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.765893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.765943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.765995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.766067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.766106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.766111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.766167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:01.766217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:01.766269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:01.766324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:01.766371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:01.766379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:01.766442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342849678814404:5250], now have 1 active actors on pipe
2025-10-09T21:51:01.766721Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:01.766779Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342849678814404:5250]
2025-10-09T21:51:01.766950Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:01.767154Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:01.767201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2008 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4af7f0a3-f54a281d-1bd5a1fe-40486cbf_0
2025-10-09T21:51:01.767370Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2008 sessionId: src_id|4af7f0a3-f54a281d-1bd5a1fe-40486cbf_0 is DEAD
2025-10-09T21:51:01.767520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4af7f0a3-f54a281d-1bd5a1fe-40486cbf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:01.767532Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4af7f0a3-f54a281d-1bd5a1fe-40486cbf_0] Write session will restart in 0.009644s
2025-10-09T21:51:01.767558Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4af7f0a3-f54a281d-1bd5a1fe-40486cbf_0] Write session: Do CDS request
2025-10-09T21:51:01.767564Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4af7f0a3-f54a281d-1bd5a1fe-40486cbf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:01.776590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:01.778492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:01.778530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4af7f0a3-f54a281d-1bd5a1fe-40486cbf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:01.778602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:01.778616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2009
2025-10-09T21:51:01.778667Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:01.778678Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:01.778688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:01.778735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:01.778764Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342849316980838:10913]
2025-10-09T21:51:01.778764Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2009 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:01.778794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2009 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:01.778801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2009 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:01.778806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:01.779218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2009 sessionId:  describe result for acl check
2025-10-09T21:51:01.779232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:01.779254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:01.779262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:01.779264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:01.779271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342849678814462:10226] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:01.779497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:01.779536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:01.779786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:01.779829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.779831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.779832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.779832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.779824Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342849678814462:10226] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:01.779833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.779834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.779835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.779835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.779836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.779839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.779845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.779848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.779850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.779852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.779854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.779856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.779858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.779860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.779864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.779869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.779873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.779878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.779881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.779883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.779886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.779889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.779891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.779894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.779896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.779900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.779903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.779906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.779908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.779911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.779914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.779916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.779932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:01.781792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342849678814462:10226] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:01.785516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342849678814462:10226] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:01.785529Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342849678814462:10226] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:01.785532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342849678814462:10226] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:01.785549Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2009 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:01.785858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:01.786901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:01.786915Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:01.786917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:01.786920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:01.786924Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2008 [2:7559342849316980838:10913]
2025-10-09T21:51:01.786933Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:01.786945Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:01.786950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:01.786952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:01.786953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:01.786965Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:01.786969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:01.786974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:01.787015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.787020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.787021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.787023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:01.787074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:01.787080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.787087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.787093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.787107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:01.787123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342849678814499:10226], now have 1 active actors on pipe
2025-10-09T21:51:01.787133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342849316980832:10913]
2025-10-09T21:51:01.787134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342849316980832:10913]
2025-10-09T21:51:01.787135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342849316980832:10913]
2025-10-09T21:51:01.787143Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:01.787149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342849678814410:2432], now have 1 active actors on pipe
2025-10-09T21:51:01.787157Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:01.787158Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:01.787161Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:01.787163Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:01.787175Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:01.787178Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:01.787201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342849316980832:10913]
2025-10-09T21:51:01.787211Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:01.787217Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:01.787252Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2008
2025-10-09T21:51:01.787253Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2008
2025-10-09T21:51:01.787322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.787329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.787352Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ee49a79c-9ab776b3-9d1b726f-7a3d3a63_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:01.787361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:01.787363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:01.787367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:01.787374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:01.787381Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:01.787396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:01.787560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.787565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.787574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:01.787644Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2009 partition: 0 MaxSeqNo: 25 sessionId: src_id|ee49a79c-9ab776b3-9d1b726f-7a3d3a63_0
2025-10-09T21:51:01.787919Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4af7f0a3-f54a281d-1bd5a1fe-40486cbf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ee49a79c-9ab776b3-9d1b726f-7a3d3a63_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:01.790302Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee49a79c-9ab776b3-9d1b726f-7a3d3a63_0] Write session: try to update token
2025-10-09T21:51:01.790322Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee49a79c-9ab776b3-9d1b726f-7a3d3a63_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:01.801185Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:01.801204Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:01.801205Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:01.801207Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:01.801215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342849316980832:10913]
2025-10-09T21:51:01.801218Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342849316980832:10913]
2025-10-09T21:51:01.801221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342849316980832:10913]
2025-10-09T21:51:01.801256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:01.801262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.801263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:01.801280Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:01.801287Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:01.801291Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:01.801381Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:01.803141Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2009 sessionId: src_id|ee49a79c-9ab776b3-9d1b726f-7a3d3a63_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:01.805467Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:01.867014Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342849678814507:5250]
2025-10-09T21:51:01.867263Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 2008, pipe: [1:7559342849678814507:5250]
2025-10-09T21:51:01.948607Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:01.948935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.948949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.948986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342849678814507:5250], now have 1 active actors on pipe
2025-10-09T21:51:01.949158Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:01.949197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:01.949199Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342849678814507:5250]
2025-10-09T21:51:01.949220Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:01.949288Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2009 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ee49a79c-9ab776b3-9d1b726f-7a3d3a63_0
2025-10-09T21:51:01.949505Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2009 sessionId: src_id|ee49a79c-9ab776b3-9d1b726f-7a3d3a63_0 is DEAD
2025-10-09T21:51:01.949647Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee49a79c-9ab776b3-9d1b726f-7a3d3a63_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:01.949657Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee49a79c-9ab776b3-9d1b726f-7a3d3a63_0] Write session will restart in 0.006989s
2025-10-09T21:51:01.949677Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee49a79c-9ab776b3-9d1b726f-7a3d3a63_0] Write session: Do CDS request
2025-10-09T21:51:01.949682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee49a79c-9ab776b3-9d1b726f-7a3d3a63_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:01.957564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee49a79c-9ab776b3-9d1b726f-7a3d3a63_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:01.957637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:01.957652Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2010
2025-10-09T21:51:01.957762Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2010 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:01.957794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2010 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:01.957804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2010 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:01.957807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:01.958221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2010 sessionId:  describe result for acl check
2025-10-09T21:51:01.958257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:01.958260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:01.958262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:01.958268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342849678814564:10223] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:01.958793Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342849678814564:10223] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:01.958997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:01.960370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:01.960473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342849678814564:10223] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:01.960523Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:01.960533Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:01.960544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:01.960594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:01.960643Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342849316980926:10915]
2025-10-09T21:51:01.961162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:01.961478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:01.961521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:01.961755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:01.961794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.961798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.961799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.961800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.961801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.961801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.961802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.961803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.961805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.961808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.961814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.961816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.961818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.961820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.961822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.961824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.961826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.961828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.961832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.961837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:01.961842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.961845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:01.961848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.961851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:01.961853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.961856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:01.961858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.961861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:01.961863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.961866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:01.961869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.961872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.961874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.961877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.961880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.961885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.961913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:01.966230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342849678814564:10223] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:01.966244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342849678814564:10223] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:01.966249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342849678814564:10223] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:01.966257Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2010 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:01.968189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:01.969331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:01.969346Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:01.969349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:01.969353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:01.969358Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2009 [2:7559342849316980926:10915]
2025-10-09T21:51:01.969370Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:01.969386Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:01.969390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:01.969392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:01.969395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:01.969422Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:01.969434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:01.969441Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:01.969495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.969502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.969504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.969506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:01.969570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:01.969576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.969587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:01.969600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:01.969619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:01.969640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342849678814603:10223], now have 1 active actors on pipe
2025-10-09T21:51:01.969649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342849678814512:2432], now have 1 active actors on pipe
2025-10-09T21:51:01.969656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342849316980921:10915]
2025-10-09T21:51:01.969662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342849316980921:10915]
2025-10-09T21:51:01.969665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342849316980921:10915]
2025-10-09T21:51:01.969679Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:01.969700Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:01.969699Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:01.969703Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:01.969685Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2009
2025-10-09T21:51:01.969705Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:01.969695Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2009
2025-10-09T21:51:01.969720Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:01.969723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:01.969754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342849316980921:10915]
2025-10-09T21:51:01.969765Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:01.969772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:01.969783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.969791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.969833Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dd7aa95f-8ff49702-4c724f6b-b92ea762_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:01.969845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:01.969853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:01.969855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:01.969865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:01.969878Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:01.969898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:01.970088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:01.970100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:01.970118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:01.970196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2010 partition: 0 MaxSeqNo: 25 sessionId: src_id|dd7aa95f-8ff49702-4c724f6b-b92ea762_0
2025-10-09T21:51:01.970499Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee49a79c-9ab776b3-9d1b726f-7a3d3a63_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dd7aa95f-8ff49702-4c724f6b-b92ea762_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:01.972879Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dd7aa95f-8ff49702-4c724f6b-b92ea762_0] Write session: try to update token
2025-10-09T21:51:01.972894Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dd7aa95f-8ff49702-4c724f6b-b92ea762_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:01.984333Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:01.984352Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:01.984355Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:01.984357Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:01.984366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342849316980921:10915]
2025-10-09T21:51:01.984374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342849316980921:10915]
2025-10-09T21:51:01.984377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342849316980921:10915]
2025-10-09T21:51:01.984423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:01.984424Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:01.984429Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:01.984429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:01.984432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:01.984434Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:01.984574Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:01.985056Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2010 sessionId: src_id|dd7aa95f-8ff49702-4c724f6b-b92ea762_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:01.987385Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:02.049306Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342853973781906:5250]
2025-10-09T21:51:02.049630Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 2009, pipe: [1:7559342853973781906:5250]
2025-10-09T21:51:02.136340Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:02.136725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:02.136741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:02.136784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342853973781906:5250], now have 1 active actors on pipe
2025-10-09T21:51:02.137062Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:02.137096Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342853973781906:5250]
2025-10-09T21:51:02.137098Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:02.137105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:02.137132Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:02.137186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2010 reason: error at write: Disconnected sessionId: src_id|dd7aa95f-8ff49702-4c724f6b-b92ea762_0
2025-10-09T21:51:02.137360Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2010 sessionId: src_id|dd7aa95f-8ff49702-4c724f6b-b92ea762_0 is DEAD
2025-10-09T21:51:02.137576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd7aa95f-8ff49702-4c724f6b-b92ea762_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:02.137589Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd7aa95f-8ff49702-4c724f6b-b92ea762_0] Write session will restart in 0.009836s
2025-10-09T21:51:02.137619Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd7aa95f-8ff49702-4c724f6b-b92ea762_0] Write session: Do CDS request
2025-10-09T21:51:02.137624Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd7aa95f-8ff49702-4c724f6b-b92ea762_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:02.147093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:02.148587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:02.148647Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dd7aa95f-8ff49702-4c724f6b-b92ea762_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:02.148694Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:02.148707Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2011
2025-10-09T21:51:02.148843Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:02.148854Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:02.148869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:02.148864Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2011 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:02.148917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2011 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:02.148933Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2011 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:02.148940Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:02.148943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:02.148990Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342853611948313:10917]
2025-10-09T21:51:02.149467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2011 sessionId:  describe result for acl check
2025-10-09T21:51:02.149503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:02.149507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:02.149508Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:02.149514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342853973781965:10218] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:02.149563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:02.149911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:02.149974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:02.150107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342853973781965:10218] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:02.150355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:02.150427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.150436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.150437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.150438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.150439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.150441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.150442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.150443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.150444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.150450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.150460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.150464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.150467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.150470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.150474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.150478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.150481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.150485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.150491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.150500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.150506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.150512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.150516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.150523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.150528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.150533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.150537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.150543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.150547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.150552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.150557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.150562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.150566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.150578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.150583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.150588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.150616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:02.152203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342853973781965:10218] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:02.156818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342853973781965:10218] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:02.156832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342853973781965:10218] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:02.156836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342853973781965:10218] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:02.156845Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2011 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:02.157277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:02.158320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:02.158338Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:02.158341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:02.158345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:02.158352Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2010 [2:7559342853611948313:10917]
2025-10-09T21:51:02.158367Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:02.158385Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:02.158388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:02.158392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:02.158394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:02.158413Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:02.158424Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:02.158431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:02.158492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.158500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.158502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.158505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:02.158571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:02.158578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.158591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.158608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.158631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:02.158652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342853973782003:10218], now have 1 active actors on pipe
2025-10-09T21:51:02.158661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342853973781911:2432], now have 1 active actors on pipe
2025-10-09T21:51:02.158669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342853611948308:10917]
2025-10-09T21:51:02.158676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342853611948308:10917]
2025-10-09T21:51:02.158678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342853611948308:10917]
2025-10-09T21:51:02.158690Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:02.158705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:02.158704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:02.158708Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:02.158710Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:02.158724Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:02.158727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:02.158740Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2010
2025-10-09T21:51:02.158761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342853611948308:10917]
2025-10-09T21:51:02.158741Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2010
2025-10-09T21:51:02.158776Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:02.158796Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:02.158835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:02.158847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:02.158877Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dfdf0d32-ad3ae417-771278b1-a6de355e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:02.158889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:02.158892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:02.158895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:02.158905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:02.158914Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:02.158937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:02.159142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:02.159152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:02.159169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:02.159286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2011 partition: 0 MaxSeqNo: 25 sessionId: src_id|dfdf0d32-ad3ae417-771278b1-a6de355e_0
2025-10-09T21:51:02.159633Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd7aa95f-8ff49702-4c724f6b-b92ea762_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dfdf0d32-ad3ae417-771278b1-a6de355e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:02.164908Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfdf0d32-ad3ae417-771278b1-a6de355e_0] Write session: try to update token
2025-10-09T21:51:02.164926Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfdf0d32-ad3ae417-771278b1-a6de355e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:02.170545Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:02.170566Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:02.170575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342853611948308:10917]
2025-10-09T21:51:02.170613Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:02.170622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:02.170792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:02.170934Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:02.171067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:02.171217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:02.171402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:02.171441Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:02.179758Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2011 sessionId: src_id|dfdf0d32-ad3ae417-771278b1-a6de355e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:02.182187Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:02.182514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:02.182538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:02.182577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:02.182660Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:02.184271Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:02.184287Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:02.184289Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:02.184292Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:02.184302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342853611948308:10917]
2025-10-09T21:51:02.184306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342853611948308:10917]
2025-10-09T21:51:02.184310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342853611948308:10917]
2025-10-09T21:51:02.184351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:02.184357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.184357Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:02.184362Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:02.184365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:02.184367Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:02.184496Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:02.237197Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342853973782014:5250]
2025-10-09T21:51:02.237465Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 2010, pipe: [1:7559342853973782014:5250]
2025-10-09T21:51:02.337447Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:02.345036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:02.345067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:02.345244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:02.345308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:02.345372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:02.345450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:02.345498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:02.345509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:02.345572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:02.345638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:02.345705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:02.345773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:02.345817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:02.345825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:02.345886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:02.345955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:02.346075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:02.346135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:02.346178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:02.346184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:02.346253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:02.346319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:02.346454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:02.346527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:02.346577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:02.346586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:02.346654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:02.346718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:02.346782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:02.346845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:02.346889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:02.346896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:02.346957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:02.347018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:02.347084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:02.347143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:02.347184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:02.347193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:02.347259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:02.347322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:02.347387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:02.347446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:02.347488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:02.347501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:02.347559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:02.347613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:02.347673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:02.347734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:02.347779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:02.347790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:02.347847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:02.347902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:02.347963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:02.348023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:02.348064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:02.348074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:02.348134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:02.348189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:02.348248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:02.348309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:02.348369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:02.348378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:02.348439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342853973782014:5250], now have 1 active actors on pipe
2025-10-09T21:51:02.348708Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:02.348721Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:02.348752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2011 reason: error at write: Disconnected sessionId: src_id|dfdf0d32-ad3ae417-771278b1-a6de355e_0
2025-10-09T21:51:02.348960Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2011 sessionId: src_id|dfdf0d32-ad3ae417-771278b1-a6de355e_0 is DEAD
2025-10-09T21:51:02.348964Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342853973782014:5250]
2025-10-09T21:51:02.349005Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:02.349125Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfdf0d32-ad3ae417-771278b1-a6de355e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:02.349136Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfdf0d32-ad3ae417-771278b1-a6de355e_0] Write session will restart in 0.007615s
2025-10-09T21:51:02.349168Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfdf0d32-ad3ae417-771278b1-a6de355e_0] Write session: Do CDS request
2025-10-09T21:51:02.349174Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfdf0d32-ad3ae417-771278b1-a6de355e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:02.349476Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:02.357520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfdf0d32-ad3ae417-771278b1-a6de355e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:02.357596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:02.357611Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2012
2025-10-09T21:51:02.357831Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2012 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:02.357867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2012 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:02.357890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2012 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:02.357899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:02.358361Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2012 sessionId:  describe result for acl check
2025-10-09T21:51:02.358394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:02.358398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:02.358401Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:02.358407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342853973782061:10210] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:02.358972Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342853973782061:10210] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:02.360604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:02.361578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342853973782061:10210] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:02.362212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:02.362409Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:02.362420Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:02.362432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:02.362488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:02.362530Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342853611948405:10919]
2025-10-09T21:51:02.362924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:02.363174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:02.363206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:02.363430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:02.363476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.363481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.363482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.363482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.363483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.363483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.363484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.363484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.363485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.363489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.363495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.363498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.363500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.363502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.363504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.363506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.363508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.363510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.363515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.363520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.363525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.363528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.363531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.363534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.363537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.363539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.363542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.363545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.363547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.363550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.363553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.363556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.363558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.363561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.363564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.363567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.363583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:02.364410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342853973782061:10210] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:02.364423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342853973782061:10210] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:02.364426Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342853973782061:10210] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:02.364434Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2012 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:02.370247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:02.371404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:02.371421Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:02.371424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:02.371428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:02.371435Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2011 [2:7559342853611948405:10919]
2025-10-09T21:51:02.371449Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:02.371468Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:02.371472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:02.371475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:02.371478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:02.371495Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:02.371508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:02.371517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:02.371578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.371590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.371592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.371595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:02.371664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:02.371672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.371685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.371701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.371723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:02.371762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342853973782021:2432], now have 1 active actors on pipe
2025-10-09T21:51:02.371766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342853973782114:10210], now have 1 active actors on pipe
2025-10-09T21:51:02.371774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342853611948399:10919]
2025-10-09T21:51:02.371781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342853611948399:10919]
2025-10-09T21:51:02.371784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342853611948399:10919]
2025-10-09T21:51:02.371800Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:02.371817Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:02.371803Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2011
2025-10-09T21:51:02.371821Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:02.371805Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2011
2025-10-09T21:51:02.371840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:02.371842Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:02.371843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:02.371845Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:02.371895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:02.371905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:02.371919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342853611948399:10919]
2025-10-09T21:51:02.371931Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:02.371951Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:02.371971Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bab1614d-e8737f2b-fb1c730f-62e8a1b0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:02.371983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:02.371992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:02.371995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:02.372006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:02.372021Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:02.372048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:02.372198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:02.372207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:02.372222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:02.372296Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2012 partition: 0 MaxSeqNo: 25 sessionId: src_id|bab1614d-e8737f2b-fb1c730f-62e8a1b0_0
2025-10-09T21:51:02.372668Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfdf0d32-ad3ae417-771278b1-a6de355e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bab1614d-e8737f2b-fb1c730f-62e8a1b0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:02.375134Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bab1614d-e8737f2b-fb1c730f-62e8a1b0_0] Write session: try to update token
2025-10-09T21:51:02.375154Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bab1614d-e8737f2b-fb1c730f-62e8a1b0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:02.386707Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:02.386729Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:02.386735Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:02.386738Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:02.386747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342853611948399:10919]
2025-10-09T21:51:02.386751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342853611948399:10919]
2025-10-09T21:51:02.386754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342853611948399:10919]
2025-10-09T21:51:02.386801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:02.386803Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:02.386808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.386811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:02.386821Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:02.386826Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:02.386934Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:02.388488Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2012 sessionId: src_id|bab1614d-e8737f2b-fb1c730f-62e8a1b0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:02.390933Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:02.449145Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342853973782122:5250]
2025-10-09T21:51:02.449347Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 2011, pipe: [1:7559342853973782122:5250]
2025-10-09T21:51:02.538555Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:02.538920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:02.538933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:02.538972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342853973782122:5250], now have 1 active actors on pipe
2025-10-09T21:51:02.539185Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:02.539224Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:02.539231Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:02.539235Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:02.539282Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342853973782122:5250]
2025-10-09T21:51:02.539304Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2012 reason: error at write: Disconnected sessionId: src_id|bab1614d-e8737f2b-fb1c730f-62e8a1b0_0
2025-10-09T21:51:02.539485Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2012 sessionId: src_id|bab1614d-e8737f2b-fb1c730f-62e8a1b0_0 is DEAD
2025-10-09T21:51:02.539613Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bab1614d-e8737f2b-fb1c730f-62e8a1b0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:02.539623Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bab1614d-e8737f2b-fb1c730f-62e8a1b0_0] Write session will restart in 0.007215s
2025-10-09T21:51:02.539642Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bab1614d-e8737f2b-fb1c730f-62e8a1b0_0] Write session: Do CDS request
2025-10-09T21:51:02.539647Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bab1614d-e8737f2b-fb1c730f-62e8a1b0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:02.547490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bab1614d-e8737f2b-fb1c730f-62e8a1b0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:02.547597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:02.547610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2013
2025-10-09T21:51:02.547725Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2013 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:02.547768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2013 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:02.547778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2013 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:02.547781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:02.548254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2013 sessionId:  describe result for acl check
2025-10-09T21:51:02.548289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:02.548291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:02.548294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:02.548300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342853973782177:10205] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:02.548842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342853973782177:10205] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:02.548890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:02.550679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:02.550832Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:02.550840Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:02.550843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342853973782177:10205] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:02.550848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:02.550886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:02.550917Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342853611948496:10921]
2025-10-09T21:51:02.551295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:02.551559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:02.551598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:02.551812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:02.551848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.551852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.551853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.551854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.551854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.551855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.551856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.551856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.551857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.551860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.551866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.551869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.551871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.551873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.551876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.551878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.551879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.551881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.551887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.551891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.551896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.551899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.551901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.551904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.551907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.551910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.551912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.551915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.551919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.551922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.551925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.551928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.551930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.551933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.551936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.551940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.551955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:02.556601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342853973782177:10205] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:02.556612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342853973782177:10205] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:02.556615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342853973782177:10205] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:02.556623Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2013 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:02.558542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:02.559576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:02.559592Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:02.559593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:02.559597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:02.559603Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2012 [2:7559342853611948496:10921]
2025-10-09T21:51:02.559611Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:02.559624Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:02.559626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:02.559628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:02.559630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:02.559643Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:02.559651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:02.559657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:02.559699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.559704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.559705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.559707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:02.559757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:02.559763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.559773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.559785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.559806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:02.559826Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342853611948491:10921]
2025-10-09T21:51:02.559831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342853611948491:10921]
2025-10-09T21:51:02.559833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342853611948491:10921]
2025-10-09T21:51:02.559839Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:02.559850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342853973782218:10205], now have 1 active actors on pipe
2025-10-09T21:51:02.559850Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:02.559853Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:02.559855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:02.559856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342853973782124:2432], now have 1 active actors on pipe
2025-10-09T21:51:02.559863Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:02.559870Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:02.559872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:02.559896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342853611948491:10921]
2025-10-09T21:51:02.559904Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:02.559913Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:02.559923Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2012
2025-10-09T21:51:02.559925Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2012
2025-10-09T21:51:02.559982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:02.559989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:02.560018Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|afe25a9f-71e1dad8-ca039b06-4841d5c2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:02.560032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:02.560036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:02.560038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:02.560045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:02.560056Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:02.560070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:02.560230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:02.560235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:02.560245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:02.560348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2013 partition: 0 MaxSeqNo: 25 sessionId: src_id|afe25a9f-71e1dad8-ca039b06-4841d5c2_0
2025-10-09T21:51:02.560693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bab1614d-e8737f2b-fb1c730f-62e8a1b0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|afe25a9f-71e1dad8-ca039b06-4841d5c2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:02.563134Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|afe25a9f-71e1dad8-ca039b06-4841d5c2_0] Write session: try to update token
2025-10-09T21:51:02.563151Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|afe25a9f-71e1dad8-ca039b06-4841d5c2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:02.574301Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:02.574325Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:02.574331Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:02.574333Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:02.574342Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342853611948491:10921]
2025-10-09T21:51:02.574348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342853611948491:10921]
2025-10-09T21:51:02.574448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342853611948491:10921]
2025-10-09T21:51:02.574497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:02.574497Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:02.574504Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:02.574508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.574508Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:02.574510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:02.574638Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:02.575870Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2013 sessionId: src_id|afe25a9f-71e1dad8-ca039b06-4841d5c2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:02.578248Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:02.639606Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342853973782239:5250]
2025-10-09T21:51:02.639851Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 2012, pipe: [1:7559342853973782239:5250]
2025-10-09T21:51:02.725866Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:02.726192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:02.726209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:02.726251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342853973782239:5250], now have 1 active actors on pipe
2025-10-09T21:51:02.726500Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:02.726513Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:02.726519Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342853973782239:5250]
2025-10-09T21:51:02.726539Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:02.726593Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2013 reason: error at write: Disconnected sessionId: src_id|afe25a9f-71e1dad8-ca039b06-4841d5c2_0
2025-10-09T21:51:02.726432Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:02.726775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2013 sessionId: src_id|afe25a9f-71e1dad8-ca039b06-4841d5c2_0 is DEAD
2025-10-09T21:51:02.726932Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|afe25a9f-71e1dad8-ca039b06-4841d5c2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:02.726944Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|afe25a9f-71e1dad8-ca039b06-4841d5c2_0] Write session will restart in 0.009765s
2025-10-09T21:51:02.726972Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|afe25a9f-71e1dad8-ca039b06-4841d5c2_0] Write session: Do CDS request
2025-10-09T21:51:02.726980Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|afe25a9f-71e1dad8-ca039b06-4841d5c2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:02.735080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:02.736507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:02.736680Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:02.736690Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:02.736699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:02.736734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:02.736771Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342853611948627:10935]
2025-10-09T21:51:02.737218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:02.737489Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|afe25a9f-71e1dad8-ca039b06-4841d5c2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:02.737518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:02.737552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:02.737566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:02.737579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2014
2025-10-09T21:51:02.737719Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2014 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:02.737747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2014 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:02.737757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2014 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:02.737760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:02.737854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:02.737901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.737907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.737909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.737910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.737911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.737912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.737913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.737914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.737915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.737920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.737930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.737934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.737938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.737942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.737946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.737950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.737953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.737957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.737963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.737976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.737982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.737988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.737998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.738003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.738006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.738009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.738012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.738015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.738017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.738020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.738023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.738042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.738047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.738052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.738056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.738061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.738089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:02.738211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2014 sessionId:  describe result for acl check
2025-10-09T21:51:02.738246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:02.738252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:02.738254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:02.738260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342853973782309:10186] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:02.738785Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342853973782309:10186] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:02.741103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342853973782309:10186] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:02.743641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342853973782309:10186] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:02.743652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342853973782309:10186] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:02.743655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342853973782309:10186] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:02.743734Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2014 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:02.744040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:02.745011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:02.745031Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:02.745033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:02.745037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:02.745043Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2013 [2:7559342853611948627:10935]
2025-10-09T21:51:02.745055Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:02.745075Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:02.745083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:02.745086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:02.745088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:02.745104Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:02.745110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:02.745117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:02.745172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.745177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.745179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.745181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:02.745238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:02.745245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.745257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.745274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.745295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:02.745318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342853973782346:10186], now have 1 active actors on pipe
2025-10-09T21:51:02.745326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342853973782250:2432], now have 1 active actors on pipe
2025-10-09T21:51:02.745333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342853611948622:10935]
2025-10-09T21:51:02.745337Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342853611948622:10935]
2025-10-09T21:51:02.745340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342853611948622:10935]
2025-10-09T21:51:02.745350Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:02.745360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:02.745358Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2013
2025-10-09T21:51:02.745361Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2013
2025-10-09T21:51:02.745372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:02.745372Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:02.745375Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:02.745377Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:02.745378Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:02.745415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:02.745421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:02.745434Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342853611948622:10935]
2025-10-09T21:51:02.745440Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:02.745459Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:02.745466Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|38e1934b-4b71cd37-b75a31ee-8039fb5e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:02.745472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:02.745474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:02.745476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:02.745482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:02.745491Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:02.745515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:02.745663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:02.745672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:02.745683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:02.745772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2014 partition: 0 MaxSeqNo: 25 sessionId: src_id|38e1934b-4b71cd37-b75a31ee-8039fb5e_0
2025-10-09T21:51:02.746077Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|afe25a9f-71e1dad8-ca039b06-4841d5c2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|38e1934b-4b71cd37-b75a31ee-8039fb5e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:02.748391Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38e1934b-4b71cd37-b75a31ee-8039fb5e_0] Write session: try to update token
2025-10-09T21:51:02.748405Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38e1934b-4b71cd37-b75a31ee-8039fb5e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:02.758787Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:02.758802Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:02.758805Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:02.758807Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:02.758817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342853611948622:10935]
2025-10-09T21:51:02.758821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342853611948622:10935]
2025-10-09T21:51:02.758835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342853611948622:10935]
2025-10-09T21:51:02.758875Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:02.758880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:02.758887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.758889Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:02.758890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:02.758892Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:02.759021Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:02.760456Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2014 sessionId: src_id|38e1934b-4b71cd37-b75a31ee-8039fb5e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:02.762978Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:02.826509Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342853973782353:5250]
2025-10-09T21:51:02.826874Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 2013, pipe: [1:7559342853973782353:5250]
2025-10-09T21:51:02.907706Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:02.908000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:02.908021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:02.908063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342853973782353:5250], now have 1 active actors on pipe
2025-10-09T21:51:02.908266Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:02.908304Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:02.908320Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342853973782353:5250]
2025-10-09T21:51:02.908346Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:02.908414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2014 reason: status is not ok: tablet will be restarted right now sessionId: src_id|38e1934b-4b71cd37-b75a31ee-8039fb5e_0
2025-10-09T21:51:02.908605Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2014 sessionId: src_id|38e1934b-4b71cd37-b75a31ee-8039fb5e_0 is DEAD
2025-10-09T21:51:02.908773Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38e1934b-4b71cd37-b75a31ee-8039fb5e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:02.908784Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38e1934b-4b71cd37-b75a31ee-8039fb5e_0] Write session will restart in 0.009934s
2025-10-09T21:51:02.908813Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38e1934b-4b71cd37-b75a31ee-8039fb5e_0] Write session: Do CDS request
2025-10-09T21:51:02.908820Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38e1934b-4b71cd37-b75a31ee-8039fb5e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:02.917456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:02.919089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:02.919266Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:02.919276Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:02.919285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:02.919325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:02.919354Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342853611948725:10939]
2025-10-09T21:51:02.919521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38e1934b-4b71cd37-b75a31ee-8039fb5e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:02.919602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:02.919621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2015
2025-10-09T21:51:02.919739Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2015 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:02.919763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2015 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:02.919767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2015 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:02.919770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:02.919835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:02.920087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:02.920131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:02.920182Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2015 sessionId:  describe result for acl check
2025-10-09T21:51:02.920218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:02.920220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:02.920221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:02.920229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342853973782418:10185] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:02.920446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:02.920507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.920513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.920515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.920516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.920517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.920518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.920520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.920521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.920522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.920529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.920539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.920543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.920548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.920552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.920556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.920560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.920564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.920567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.920573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.920581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:02.920587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.920597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:02.920607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.920612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:02.920621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.920626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:02.920635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.920640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:02.920644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.920653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:02.920659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.920664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.920674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.920679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.920683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.920688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.920672Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342853973782418:10185] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:02.920716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:02.922394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342853973782418:10185] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:02.926555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342853973782418:10185] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:02.926567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342853973782418:10185] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:02.926571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342853973782418:10185] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:02.926579Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2015 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:02.926932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:02.927923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:02.927935Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:02.927936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:02.927942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:02.927946Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2014 [2:7559342853611948725:10939]
2025-10-09T21:51:02.927955Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:02.927968Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:02.927970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:02.927972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:02.927973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:02.927986Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:02.927989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:02.927995Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:02.928033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.928038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.928040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.928042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:02.928090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:02.928094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.928102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:02.928113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:02.928126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:02.928141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342853973782358:2432], now have 1 active actors on pipe
2025-10-09T21:51:02.928150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342853973782454:10185], now have 1 active actors on pipe
2025-10-09T21:51:02.928159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342853611948720:10939]
2025-10-09T21:51:02.928161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342853611948720:10939]
2025-10-09T21:51:02.928162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342853611948720:10939]
2025-10-09T21:51:02.928176Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:02.928186Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:02.928189Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:02.928197Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:02.928198Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:02.928199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:02.928200Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:02.928194Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2014
2025-10-09T21:51:02.928202Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2014
2025-10-09T21:51:02.928217Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342853611948720:10939]
2025-10-09T21:51:02.928224Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:02.928226Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:02.928236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:02.928242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:02.928277Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6e939dba-8a284cb3-47cbaf04-20d58e95_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:02.928289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:02.928291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:02.928293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:02.928300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:02.928314Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:02.928336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:02.928457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:02.928467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:02.928480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:02.928565Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2015 partition: 0 MaxSeqNo: 25 sessionId: src_id|6e939dba-8a284cb3-47cbaf04-20d58e95_0
2025-10-09T21:51:02.928839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38e1934b-4b71cd37-b75a31ee-8039fb5e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6e939dba-8a284cb3-47cbaf04-20d58e95_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:02.931061Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e939dba-8a284cb3-47cbaf04-20d58e95_0] Write session: try to update token
2025-10-09T21:51:02.931080Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e939dba-8a284cb3-47cbaf04-20d58e95_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:02.942008Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:02.942026Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:02.942027Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:02.942047Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:02.942055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342853611948720:10939]
2025-10-09T21:51:02.942058Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342853611948720:10939]
2025-10-09T21:51:02.942061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342853611948720:10939]
2025-10-09T21:51:02.942100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:02.942106Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:02.942110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:02.942113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:02.942114Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:02.942118Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:02.942241Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:02.942637Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2015 sessionId: src_id|6e939dba-8a284cb3-47cbaf04-20d58e95_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:02.944671Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:03.008321Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342858268749757:5250]
2025-10-09T21:51:03.008601Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 2014, pipe: [1:7559342858268749757:5250]
2025-10-09T21:51:03.090009Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:03.090357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:03.090372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:03.090408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342858268749757:5250], now have 1 active actors on pipe
2025-10-09T21:51:03.090603Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:03.090600Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:03.090630Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:03.090634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:03.090671Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342858268749757:5250]
2025-10-09T21:51:03.090698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2015 reason: error at write: Disconnected sessionId: src_id|6e939dba-8a284cb3-47cbaf04-20d58e95_0
2025-10-09T21:51:03.090876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2015 sessionId: src_id|6e939dba-8a284cb3-47cbaf04-20d58e95_0 is DEAD
2025-10-09T21:51:03.091043Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e939dba-8a284cb3-47cbaf04-20d58e95_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:03.091056Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e939dba-8a284cb3-47cbaf04-20d58e95_0] Write session will restart in 0.008820s
2025-10-09T21:51:03.091085Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e939dba-8a284cb3-47cbaf04-20d58e95_0] Write session: Do CDS request
2025-10-09T21:51:03.091092Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e939dba-8a284cb3-47cbaf04-20d58e95_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:03.100278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:03.101567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e939dba-8a284cb3-47cbaf04-20d58e95_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:03.101610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:03.101622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2016
2025-10-09T21:51:03.101793Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2016 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:03.101821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2016 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:03.101842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2016 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:03.101850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:03.102134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:03.102266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2016 sessionId:  describe result for acl check
2025-10-09T21:51:03.102309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:03.102313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:03.102315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:03.102322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342858268749818:10175] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:03.102342Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:03.102359Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:03.102372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:03.102437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:03.102483Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342857906916116:10941]
2025-10-09T21:51:03.102927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342858268749818:10175] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:03.103080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:03.103339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:03.103392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:03.103720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:03.103794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.103800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.103802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.103803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.103805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.103806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.103807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.103809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.103810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.103816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.103826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.103830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.103834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.103838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.103843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.103847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.103851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.103859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.103866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.103879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.103886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.103892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.103897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.103902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.103906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.103914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.103919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.103924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.103929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.103935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.103944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.103949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.103958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.103962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.103968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.103978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.104008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:03.104843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342858268749818:10175] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:03.110081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342858268749818:10175] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:03.110096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342858268749818:10175] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:03.110100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342858268749818:10175] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:03.110111Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2016 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:03.110441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:03.111526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:03.111547Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:03.111550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:03.111558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:03.111565Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2015 [2:7559342857906916116:10941]
2025-10-09T21:51:03.111579Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:03.111597Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:03.111604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:03.111607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:03.111610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:03.111627Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:03.111638Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:03.111645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:03.111714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.111721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.111723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.111727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:03.111805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:03.111814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.111827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.111840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.111863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:03.111888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342858268749857:10175], now have 1 active actors on pipe
2025-10-09T21:51:03.111896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342858268749760:2432], now have 1 active actors on pipe
2025-10-09T21:51:03.111904Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342857906916111:10941]
2025-10-09T21:51:03.111926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342857906916111:10941]
2025-10-09T21:51:03.111928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342857906916111:10941]
2025-10-09T21:51:03.111943Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:03.111963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:03.111966Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:03.111969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:03.111971Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:03.111961Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2015
2025-10-09T21:51:03.111980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:03.111978Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2015
2025-10-09T21:51:03.111987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:03.112025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342857906916111:10941]
2025-10-09T21:51:03.112038Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:03.112067Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:03.112169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:03.112179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:03.112211Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a1807997-ba040dbe-85f402dd-3ccd0233_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:03.112222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:03.112227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:03.112234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:03.112245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:03.112259Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:03.112280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:03.112455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:03.112463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:03.112477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:03.112577Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2016 partition: 0 MaxSeqNo: 25 sessionId: src_id|a1807997-ba040dbe-85f402dd-3ccd0233_0
2025-10-09T21:51:03.112923Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e939dba-8a284cb3-47cbaf04-20d58e95_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a1807997-ba040dbe-85f402dd-3ccd0233_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:03.115405Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a1807997-ba040dbe-85f402dd-3ccd0233_0] Write session: try to update token
2025-10-09T21:51:03.115427Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a1807997-ba040dbe-85f402dd-3ccd0233_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:03.126710Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:03.126727Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:03.126730Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:03.126732Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:03.126742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342857906916111:10941]
2025-10-09T21:51:03.126747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342857906916111:10941]
2025-10-09T21:51:03.126755Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342857906916111:10941]
2025-10-09T21:51:03.126804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:03.126803Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:03.126809Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:03.126813Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:03.126814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.126818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:03.126952Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:03.128154Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2016 sessionId: src_id|a1807997-ba040dbe-85f402dd-3ccd0233_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:03.130582Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:03.191041Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342858268749864:5250]
2025-10-09T21:51:03.191335Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 2015, pipe: [1:7559342858268749864:5250]
2025-10-09T21:51:03.277336Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:03.277679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:03.277695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:03.277741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342858268749864:5250], now have 1 active actors on pipe
2025-10-09T21:51:03.277958Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:03.278046Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:03.278073Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342858268749864:5250]
2025-10-09T21:51:03.278107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:03.278178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2016 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a1807997-ba040dbe-85f402dd-3ccd0233_0
2025-10-09T21:51:03.278346Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2016 sessionId: src_id|a1807997-ba040dbe-85f402dd-3ccd0233_0 is DEAD
2025-10-09T21:51:03.278489Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1807997-ba040dbe-85f402dd-3ccd0233_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:03.278501Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1807997-ba040dbe-85f402dd-3ccd0233_0] Write session will restart in 0.005779s
2025-10-09T21:51:03.278528Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1807997-ba040dbe-85f402dd-3ccd0233_0] Write session: Do CDS request
2025-10-09T21:51:03.278534Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1807997-ba040dbe-85f402dd-3ccd0233_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:03.285552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a1807997-ba040dbe-85f402dd-3ccd0233_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:03.286118Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:03.286134Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2017
2025-10-09T21:51:03.286843Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2017 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:03.286884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2017 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:03.286894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2017 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:03.286898Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:03.287278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2017 sessionId:  describe result for acl check
2025-10-09T21:51:03.287313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:03.287319Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:03.287322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:03.287329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342858268749928:10167] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:03.287914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342858268749928:10167] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:03.288850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:03.289649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342858268749928:10167] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:03.290557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:03.290756Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:03.290769Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:03.290782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:03.290840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:03.290882Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342857906916213:10943]
2025-10-09T21:51:03.291447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:03.291726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:03.291773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:03.292004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342858268749928:10167] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:03.292016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342858268749928:10167] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:03.292019Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342858268749928:10167] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:03.292026Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2017 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:03.292044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:03.292079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.292083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.292084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.292084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.292085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.292086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.292086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.292087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.292088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.292091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.292097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.292100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.292102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.292104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.292106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.292108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.292110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.292112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.292116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.292123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.292127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.292131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.292133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.292136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.292139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.292143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.292148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.292152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.292156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.292161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.292166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.292170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.292173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.292177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.292182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.292187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.292208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:03.298733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:03.299852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:03.299869Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:03.299872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:03.299877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:03.299884Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2016 [2:7559342857906916213:10943]
2025-10-09T21:51:03.299899Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:03.299920Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:03.299929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:03.299933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:03.299936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:03.299956Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:03.299969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:03.299977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:03.300043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.300050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.300052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.300055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:03.300130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:03.300138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.300150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.300160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.300183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:03.300205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342858268749971:10167], now have 1 active actors on pipe
2025-10-09T21:51:03.300214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342858268749870:2432], now have 1 active actors on pipe
2025-10-09T21:51:03.300222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342857906916207:10943]
2025-10-09T21:51:03.300224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342857906916207:10943]
2025-10-09T21:51:03.300227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342857906916207:10943]
2025-10-09T21:51:03.300240Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:03.300256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:03.300260Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:03.300263Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:03.300266Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:03.300275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:03.300278Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:03.300286Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2016
2025-10-09T21:51:03.300290Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2016
2025-10-09T21:51:03.300314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342857906916207:10943]
2025-10-09T21:51:03.300321Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:03.300329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:03.300355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:03.300364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:03.300398Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9132c95-a903797b-2d4839c2-49caba66_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:03.300410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:03.300414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:03.300416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:03.300428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:03.300442Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:03.300460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:03.300608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:03.300616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:03.300629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:03.300704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2017 partition: 0 MaxSeqNo: 25 sessionId: src_id|9132c95-a903797b-2d4839c2-49caba66_0
2025-10-09T21:51:03.301098Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1807997-ba040dbe-85f402dd-3ccd0233_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9132c95-a903797b-2d4839c2-49caba66_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:03.303585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9132c95-a903797b-2d4839c2-49caba66_0] Write session: try to update token
2025-10-09T21:51:03.303604Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9132c95-a903797b-2d4839c2-49caba66_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:03.314593Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:03.314609Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:03.314612Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:03.314614Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:03.314628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342857906916207:10943]
2025-10-09T21:51:03.314633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342857906916207:10943]
2025-10-09T21:51:03.314637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342857906916207:10943]
2025-10-09T21:51:03.314681Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:03.314682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:03.314687Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:03.314690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.314691Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:03.314693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:03.314823Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:03.316659Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2017 sessionId: src_id|9132c95-a903797b-2d4839c2-49caba66_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:03.319003Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:03.378409Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342858268749980:5250]
2025-10-09T21:51:03.378663Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 2016, pipe: [1:7559342858268749980:5250]
2025-10-09T21:51:03.464820Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:03.465186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:03.465201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:03.465247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342858268749980:5250], now have 1 active actors on pipe
2025-10-09T21:51:03.465487Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:03.465527Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:03.465540Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:03.465543Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342858268749980:5250]
2025-10-09T21:51:03.465564Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:03.465618Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2017 reason: error at write: Disconnected sessionId: src_id|9132c95-a903797b-2d4839c2-49caba66_0
2025-10-09T21:51:03.465793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2017 sessionId: src_id|9132c95-a903797b-2d4839c2-49caba66_0 is DEAD
2025-10-09T21:51:03.465956Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9132c95-a903797b-2d4839c2-49caba66_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:03.465966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9132c95-a903797b-2d4839c2-49caba66_0] Write session will restart in 0.007148s
2025-10-09T21:51:03.466003Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9132c95-a903797b-2d4839c2-49caba66_0] Write session: Do CDS request
2025-10-09T21:51:03.466009Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9132c95-a903797b-2d4839c2-49caba66_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:03.474486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9132c95-a903797b-2d4839c2-49caba66_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:03.474560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:03.474573Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2018
2025-10-09T21:51:03.474703Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2018 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:03.474736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2018 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:03.474745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2018 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:03.474749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:03.475152Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2018 sessionId:  describe result for acl check
2025-10-09T21:51:03.475185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:03.475187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:03.475188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:03.475194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342858268750042:10157] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:03.475301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:03.475735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342858268750042:10157] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:03.476743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:03.476914Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:03.476925Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:03.476936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:03.476987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:03.477025Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342857906916310:10945]
2025-10-09T21:51:03.477572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:03.477593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342858268750042:10157] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:03.477839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:03.477878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:03.478213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:03.478260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.478266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.478267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.478268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.478269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.478270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.478271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.478272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.478273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.478278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.478287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.478291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.478295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.478299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.478303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.478307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.478311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.478315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.478320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.478328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.478334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.478339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.478344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.478350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.478356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.478361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.478366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.478371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.478383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.478388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.478393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.478399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.478403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.478408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.478413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.478418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.478447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:03.483310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342858268750042:10157] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:03.483324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342858268750042:10157] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:03.483327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342858268750042:10157] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:03.483336Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2018 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:03.484599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:03.485586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:03.485600Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:03.485601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:03.485604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:03.485609Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2017 [2:7559342857906916310:10945]
2025-10-09T21:51:03.485618Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:03.485632Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:03.485634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:03.485637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:03.485638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:03.485650Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:03.485657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:03.485662Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:03.485703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.485707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.485708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.485710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:03.485759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:03.485764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.485771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.485777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.485790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:03.485812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342857906916305:10945]
2025-10-09T21:51:03.485818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342857906916305:10945]
2025-10-09T21:51:03.485820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342857906916305:10945]
2025-10-09T21:51:03.485834Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:03.485846Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:03.485852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342858268749983:2432], now have 1 active actors on pipe
2025-10-09T21:51:03.485853Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:03.485855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:03.485856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342858268750082:10157], now have 1 active actors on pipe
2025-10-09T21:51:03.485865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:03.485878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:03.485885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:03.485918Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2017
2025-10-09T21:51:03.485922Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2017
2025-10-09T21:51:03.485909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342857906916305:10945]
2025-10-09T21:51:03.485927Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:03.485951Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:03.486054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:03.486063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:03.486090Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3c3e4efa-6e494120-63a42b07-975593b1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:03.486104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:03.486108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:03.486114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:03.486124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:03.486134Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:03.486150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:03.486312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:03.486319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:03.486334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:03.486418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2018 partition: 0 MaxSeqNo: 25 sessionId: src_id|3c3e4efa-6e494120-63a42b07-975593b1_0
2025-10-09T21:51:03.486733Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9132c95-a903797b-2d4839c2-49caba66_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3c3e4efa-6e494120-63a42b07-975593b1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:03.489226Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c3e4efa-6e494120-63a42b07-975593b1_0] Write session: try to update token
2025-10-09T21:51:03.489241Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c3e4efa-6e494120-63a42b07-975593b1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:03.500157Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:03.500175Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:03.500177Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:03.500180Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:03.500190Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342857906916305:10945]
2025-10-09T21:51:03.500194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342857906916305:10945]
2025-10-09T21:51:03.500198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342857906916305:10945]
2025-10-09T21:51:03.500251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:03.500256Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:03.500259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.500263Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:03.500263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:03.500266Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:03.500403Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:03.502148Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2018 sessionId: src_id|3c3e4efa-6e494120-63a42b07-975593b1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:03.504795Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:03.565759Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342858268750090:5250]
2025-10-09T21:51:03.566015Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 2017, pipe: [1:7559342858268750090:5250]
2025-10-09T21:51:03.645638Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:03.645922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:03.645938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:03.645984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342858268750090:5250], now have 1 active actors on pipe
2025-10-09T21:51:03.646194Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:03.646232Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:03.646222Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342858268750090:5250]
2025-10-09T21:51:03.646260Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:03.646263Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:03.646325Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2018 reason: error at write: Disconnected sessionId: src_id|3c3e4efa-6e494120-63a42b07-975593b1_0
2025-10-09T21:51:03.646506Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2018 sessionId: src_id|3c3e4efa-6e494120-63a42b07-975593b1_0 is DEAD
2025-10-09T21:51:03.646667Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c3e4efa-6e494120-63a42b07-975593b1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:03.646679Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c3e4efa-6e494120-63a42b07-975593b1_0] Write session will restart in 0.009035s
2025-10-09T21:51:03.646706Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c3e4efa-6e494120-63a42b07-975593b1_0] Write session: Do CDS request
2025-10-09T21:51:03.646712Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c3e4efa-6e494120-63a42b07-975593b1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:03.655160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:03.656462Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c3e4efa-6e494120-63a42b07-975593b1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:03.656547Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:03.656568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2019
2025-10-09T21:51:03.656674Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2019 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:03.656704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2019 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:03.656710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2019 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:03.656713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:03.656745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:03.656917Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:03.656927Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:03.656938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:03.656985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:03.657022Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342857906916407:10947]
2025-10-09T21:51:03.657081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2019 sessionId:  describe result for acl check
2025-10-09T21:51:03.657115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:03.657120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:03.657122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:03.657127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342858268750153:10155] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:03.657596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:03.657675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342858268750153:10155] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:03.657897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:03.657936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:03.658249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:03.658294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.658299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.658301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.658302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.658303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.658304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.658305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.658306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.658307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.658311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.658321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.658325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.658329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.658333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.658337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.658344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.658348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.658356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.658361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.658369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.658380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.658385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.658388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.658398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.658407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.658415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.658424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.658430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.658438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.658444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.658453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.658457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.658465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.658470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.658474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.658480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.658512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:03.659752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342858268750153:10155] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:03.664034Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342858268750153:10155] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:03.664045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342858268750153:10155] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:03.664047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342858268750153:10155] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:03.664060Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2019 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:03.664396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:03.665488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:03.665499Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:03.665501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:03.665505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:03.665511Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2018 [2:7559342857906916407:10947]
2025-10-09T21:51:03.665524Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:03.665545Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:03.665551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:03.665554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:03.665556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:03.665570Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:03.665580Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:03.665591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:03.665642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.665648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.665650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.665653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:03.665710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:03.665717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.665728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.665743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.665761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:03.665784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342858268750192:10155], now have 1 active actors on pipe
2025-10-09T21:51:03.665791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342858268750092:2432], now have 1 active actors on pipe
2025-10-09T21:51:03.665799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342857906916402:10947]
2025-10-09T21:51:03.665804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342857906916402:10947]
2025-10-09T21:51:03.665806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342857906916402:10947]
2025-10-09T21:51:03.665818Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:03.665833Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:03.665834Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:03.665840Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:03.665822Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2018
2025-10-09T21:51:03.665842Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:03.665848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:03.665829Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2018
2025-10-09T21:51:03.665850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:03.665880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:03.665889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:03.665901Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342857906916402:10947]
2025-10-09T21:51:03.665913Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:03.665931Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:03.665942Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2b7f5bc3-9aeaa9c2-1eb05f5b-a0133b5b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:03.665950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:03.666004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:03.666011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:03.666020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:03.666045Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:03.666073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:03.666294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:03.666304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:03.666320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:03.666464Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2019 partition: 0 MaxSeqNo: 25 sessionId: src_id|2b7f5bc3-9aeaa9c2-1eb05f5b-a0133b5b_0
2025-10-09T21:51:03.666731Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c3e4efa-6e494120-63a42b07-975593b1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2b7f5bc3-9aeaa9c2-1eb05f5b-a0133b5b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:03.669096Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b7f5bc3-9aeaa9c2-1eb05f5b-a0133b5b_0] Write session: try to update token
2025-10-09T21:51:03.669115Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b7f5bc3-9aeaa9c2-1eb05f5b-a0133b5b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:03.679496Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:03.679511Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:03.679513Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:03.679515Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:03.679524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342857906916402:10947]
2025-10-09T21:51:03.679528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342857906916402:10947]
2025-10-09T21:51:03.679533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342857906916402:10947]
2025-10-09T21:51:03.679570Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:03.679575Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:03.679577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:03.679577Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:03.679583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.679586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:03.679726Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:03.681862Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2019 sessionId: src_id|2b7f5bc3-9aeaa9c2-1eb05f5b-a0133b5b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:03.684116Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:03.746515Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342858268750222:5250]
2025-10-09T21:51:03.746766Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 2018, pipe: [1:7559342858268750222:5250]
2025-10-09T21:51:03.826220Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:03.826509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:03.826526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:03.826568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342858268750222:5250], now have 1 active actors on pipe
2025-10-09T21:51:03.826813Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342858268750222:5250]
2025-10-09T21:51:03.826819Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:03.826827Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:03.826846Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:03.826849Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:03.826904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2019 reason: error at write: Disconnected sessionId: src_id|2b7f5bc3-9aeaa9c2-1eb05f5b-a0133b5b_0
2025-10-09T21:51:03.827081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2019 sessionId: src_id|2b7f5bc3-9aeaa9c2-1eb05f5b-a0133b5b_0 is DEAD
2025-10-09T21:51:03.827242Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b7f5bc3-9aeaa9c2-1eb05f5b-a0133b5b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:03.827254Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b7f5bc3-9aeaa9c2-1eb05f5b-a0133b5b_0] Write session will restart in 0.009055s
2025-10-09T21:51:03.827293Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b7f5bc3-9aeaa9c2-1eb05f5b-a0133b5b_0] Write session: Do CDS request
2025-10-09T21:51:03.827299Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b7f5bc3-9aeaa9c2-1eb05f5b-a0133b5b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:03.836655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:03.837538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2b7f5bc3-9aeaa9c2-1eb05f5b-a0133b5b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:03.837613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:03.837629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2020
2025-10-09T21:51:03.837757Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2020 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:03.837799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2020 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:03.837809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2020 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:03.837813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:03.838264Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2020 sessionId:  describe result for acl check
2025-10-09T21:51:03.838300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:03.838303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:03.838281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:03.838304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:03.838312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342858268750286:10136] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:03.838461Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:03.838473Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:03.838485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:03.838535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:03.838572Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342857906916549:10963]
2025-10-09T21:51:03.838819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342858268750286:10136] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:03.839117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:03.839413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:03.839458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:03.839762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:03.839811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.839813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.839814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.839815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.839816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.839817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.839819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.839820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.839821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.839825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.839834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.839838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.839842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.839845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.839849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.839853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.839857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.839862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.839868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.839876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:03.839886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.839892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:03.839896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.839901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:03.839910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.839915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:03.839922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.839927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:03.839931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.839936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:03.839941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.839946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.839950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.839961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.839966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.839971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.840002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:03.840913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342858268750286:10136] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:03.845819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342858268750286:10136] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:03.845835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342858268750286:10136] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:03.845839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342858268750286:10136] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:03.845847Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2020 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:03.846206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:03.847312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:03.847327Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:03.847332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:03.847336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:03.847342Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2019 [2:7559342857906916549:10963]
2025-10-09T21:51:03.847355Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:03.847371Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:03.847374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:03.847377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:03.847380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:03.847396Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:03.847403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:03.847412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:03.847462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.847469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.847471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.847474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:03.847540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:03.847547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.847558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:03.847575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:03.847596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:03.847620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342858268750324:10136], now have 1 active actors on pipe
2025-10-09T21:51:03.847628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342858268750224:2432], now have 1 active actors on pipe
2025-10-09T21:51:03.847635Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342857906916544:10963]
2025-10-09T21:51:03.847642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342857906916544:10963]
2025-10-09T21:51:03.847645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342857906916544:10963]
2025-10-09T21:51:03.847656Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:03.847668Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:03.847671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:03.847676Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:03.847686Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:03.847687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:03.847690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:03.847670Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2019
2025-10-09T21:51:03.847690Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2019
2025-10-09T21:51:03.847715Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342857906916544:10963]
2025-10-09T21:51:03.847727Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:03.847732Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:03.847747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:03.847760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:03.847804Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ca0e039e-4dc46f25-31b643d-34e8f01a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:03.847817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:03.847821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:03.847823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:03.847833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:03.847842Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:03.847862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:03.848011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:03.848020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:03.848036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:03.848123Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2020 partition: 0 MaxSeqNo: 25 sessionId: src_id|ca0e039e-4dc46f25-31b643d-34e8f01a_0
2025-10-09T21:51:03.848428Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2b7f5bc3-9aeaa9c2-1eb05f5b-a0133b5b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ca0e039e-4dc46f25-31b643d-34e8f01a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:03.850944Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca0e039e-4dc46f25-31b643d-34e8f01a_0] Write session: try to update token
2025-10-09T21:51:03.850960Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca0e039e-4dc46f25-31b643d-34e8f01a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:03.861793Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:03.861812Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:03.861815Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:03.861817Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:03.861827Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342857906916544:10963]
2025-10-09T21:51:03.861831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342857906916544:10963]
2025-10-09T21:51:03.861835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342857906916544:10963]
2025-10-09T21:51:03.861882Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:03.861884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:03.861891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:03.861894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:03.861908Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:03.861912Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:03.862061Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:03.865105Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2020 sessionId: src_id|ca0e039e-4dc46f25-31b643d-34e8f01a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:03.867851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:03.926780Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342858268750335:5250]
2025-10-09T21:51:03.927050Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 2019, pipe: [1:7559342858268750335:5250]
2025-10-09T21:51:04.022114Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:04.022405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.022420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.022463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342858268750335:5250], now have 1 active actors on pipe
2025-10-09T21:51:04.022701Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:04.022740Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:04.022758Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:04.022823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2020 reason: error at write: Disconnected sessionId: src_id|ca0e039e-4dc46f25-31b643d-34e8f01a_0
2025-10-09T21:51:04.023007Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2020 sessionId: src_id|ca0e039e-4dc46f25-31b643d-34e8f01a_0 is DEAD
2025-10-09T21:51:04.023072Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:04.023142Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342858268750335:5250]
2025-10-09T21:51:04.023160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca0e039e-4dc46f25-31b643d-34e8f01a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:04.023172Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca0e039e-4dc46f25-31b643d-34e8f01a_0] Write session will restart in 0.006123s
2025-10-09T21:51:04.023199Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca0e039e-4dc46f25-31b643d-34e8f01a_0] Write session: Do CDS request
2025-10-09T21:51:04.023206Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca0e039e-4dc46f25-31b643d-34e8f01a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:04.030560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ca0e039e-4dc46f25-31b643d-34e8f01a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:04.030641Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:04.030655Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2021
2025-10-09T21:51:04.030803Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2021 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:04.030844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2021 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:04.030854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2021 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:04.030858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:04.031481Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2021 sessionId:  describe result for acl check
2025-10-09T21:51:04.031520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:04.031528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:04.031530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:04.031536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342862563717681:10134] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:04.032134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342862563717681:10134] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:04.033432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:04.034412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342862563717681:10134] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:04.035093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:04.035320Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:04.035331Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:04.035345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:04.035400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:04.035441Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342862201883946:10965]
2025-10-09T21:51:04.035900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:04.036231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:04.036294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:04.036646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:04.036702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.036708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.036710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.036711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.036712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.036714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.036718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.036719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.036721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.036726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.036738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.036742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.036746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.036750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.036755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.036759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.036762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.036767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.036774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.036783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.036803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.036813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.036818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.036823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.036832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.036837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.036846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.036851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.036856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.036866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.036871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.036881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.036886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.036896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.036901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.036906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.036939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:04.037053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342862563717681:10134] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:04.037063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342862563717681:10134] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:04.037066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342862563717681:10134] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:04.037072Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2021 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:04.043787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:04.044840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:04.044855Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:04.044857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:04.044860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:04.044866Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2020 [2:7559342862201883946:10965]
2025-10-09T21:51:04.044877Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:04.044894Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:04.044897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:04.044899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:04.044900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:04.044915Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:04.044920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:04.044925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:04.044973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.044981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.044983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.044986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:04.045054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:04.045061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.045074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.045092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.045113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:04.045142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563717732:10134], now have 1 active actors on pipe
2025-10-09T21:51:04.045152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563717633:2432], now have 1 active actors on pipe
2025-10-09T21:51:04.045161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342862201883941:10965]
2025-10-09T21:51:04.045167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342862201883941:10965]
2025-10-09T21:51:04.045169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342862201883941:10965]
2025-10-09T21:51:04.045184Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:04.045201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:04.045206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:04.045209Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:04.045211Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:04.045221Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:04.045224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:04.045218Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2020
2025-10-09T21:51:04.045229Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2020
2025-10-09T21:51:04.045264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342862201883941:10965]
2025-10-09T21:51:04.045272Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:04.045287Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:04.045289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.045294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.045353Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d369ab40-8b014ac5-4967316e-44e6752a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:04.045365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:04.045369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:04.045372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:04.045382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:04.045395Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:04.045435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:04.045612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.045625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.045648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:04.045806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2021 partition: 0 MaxSeqNo: 25 sessionId: src_id|d369ab40-8b014ac5-4967316e-44e6752a_0
2025-10-09T21:51:04.046177Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ca0e039e-4dc46f25-31b643d-34e8f01a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d369ab40-8b014ac5-4967316e-44e6752a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:04.048670Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d369ab40-8b014ac5-4967316e-44e6752a_0] Write session: try to update token
2025-10-09T21:51:04.048687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d369ab40-8b014ac5-4967316e-44e6752a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:04.060106Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:04.060122Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:04.060125Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:04.060127Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:04.060137Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342862201883941:10965]
2025-10-09T21:51:04.060142Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342862201883941:10965]
2025-10-09T21:51:04.060145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342862201883941:10965]
2025-10-09T21:51:04.060191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:04.060192Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:04.060198Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:04.060199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.060202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:04.060202Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:04.060341Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:04.061561Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2021 sessionId: src_id|d369ab40-8b014ac5-4967316e-44e6752a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:04.064055Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:04.123357Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342862563717742:5250]
2025-10-09T21:51:04.123638Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 2020, pipe: [1:7559342862563717742:5250]
2025-10-09T21:51:04.217513Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:04.217809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.217824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.217871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563717742:5250], now have 1 active actors on pipe
2025-10-09T21:51:04.218079Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:04.218136Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342862563717742:5250]
2025-10-09T21:51:04.218142Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:04.218146Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:04.218172Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:04.218200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2021 reason: error at write: Disconnected sessionId: src_id|d369ab40-8b014ac5-4967316e-44e6752a_0
2025-10-09T21:51:04.218394Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2021 sessionId: src_id|d369ab40-8b014ac5-4967316e-44e6752a_0 is DEAD
2025-10-09T21:51:04.218632Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d369ab40-8b014ac5-4967316e-44e6752a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:04.218642Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d369ab40-8b014ac5-4967316e-44e6752a_0] Write session will restart in 0.007498s
2025-10-09T21:51:04.218670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d369ab40-8b014ac5-4967316e-44e6752a_0] Write session: Do CDS request
2025-10-09T21:51:04.218675Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d369ab40-8b014ac5-4967316e-44e6752a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:04.227606Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d369ab40-8b014ac5-4967316e-44e6752a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:04.227715Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:04.227732Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2022
2025-10-09T21:51:04.227874Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2022 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:04.227908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2022 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:04.227914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2022 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:04.227918Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:04.228382Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2022 sessionId:  describe result for acl check
2025-10-09T21:51:04.228422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:04.228428Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:04.228430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:04.228437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342862563717810:10126] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:04.228745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:04.229028Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342862563717810:10126] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:04.230675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:04.230877Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:04.230889Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:04.230903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:04.230968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:04.231016Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342862201884047:10967]
2025-10-09T21:51:04.231532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342862563717810:10126] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:04.231599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:04.231967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:04.232018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:04.232355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:04.232424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.232431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.232433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.232434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.232435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.232436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.232437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.232439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.232440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.232445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.232457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.232461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.232466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.232470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.232474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.232478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.232481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.232484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.232492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.232500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.232506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.232513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.232519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.232525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.232530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.232536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.232548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.232554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.232559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.232564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.232570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.232576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.232580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.232586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.232591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.232597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.232628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:04.237636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342862563717810:10126] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:04.237650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342862563717810:10126] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:04.237654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342862563717810:10126] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:04.237664Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2022 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:04.238985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:04.240122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:04.240138Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:04.240143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:04.240147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:04.240153Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2021 [2:7559342862201884047:10967]
2025-10-09T21:51:04.240165Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:04.240193Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:04.240200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:04.240203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:04.240205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:04.240223Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:04.240234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:04.240240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:04.240298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.240305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.240307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.240310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:04.240376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:04.240386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.240396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.240405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.240419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:04.240438Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342862201884041:10967]
2025-10-09T21:51:04.240444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342862201884041:10967]
2025-10-09T21:51:04.240447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342862201884041:10967]
2025-10-09T21:51:04.240462Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:04.240481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563717851:10126], now have 1 active actors on pipe
2025-10-09T21:51:04.240485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563717747:2432], now have 1 active actors on pipe
2025-10-09T21:51:04.240484Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:04.240488Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:04.240491Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:04.240494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:04.240509Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:04.240516Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:04.240517Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2021
2025-10-09T21:51:04.240533Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2021
2025-10-09T21:51:04.240551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342862201884041:10967]
2025-10-09T21:51:04.240561Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:04.240569Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:04.240586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.240590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.240625Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|23dd71d9-dd054577-98e72b55-8f4171bd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:04.240637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:04.240642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:04.240644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:04.240654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:04.240663Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:04.240686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:04.240839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.240848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.240860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:04.240942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2022 partition: 0 MaxSeqNo: 25 sessionId: src_id|23dd71d9-dd054577-98e72b55-8f4171bd_0
2025-10-09T21:51:04.241262Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d369ab40-8b014ac5-4967316e-44e6752a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|23dd71d9-dd054577-98e72b55-8f4171bd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:04.243761Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23dd71d9-dd054577-98e72b55-8f4171bd_0] Write session: try to update token
2025-10-09T21:51:04.243780Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23dd71d9-dd054577-98e72b55-8f4171bd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:04.254795Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:04.254816Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:04.254819Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:04.254821Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:04.254830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342862201884041:10967]
2025-10-09T21:51:04.254833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342862201884041:10967]
2025-10-09T21:51:04.254837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342862201884041:10967]
2025-10-09T21:51:04.254874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:04.254881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.254882Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:04.254884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:04.254889Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:04.254893Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:04.255025Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:04.256619Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2022 sessionId: src_id|23dd71d9-dd054577-98e72b55-8f4171bd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:04.259007Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:04.318460Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342862563717863:5250]
2025-10-09T21:51:04.318712Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 2021, pipe: [1:7559342862563717863:5250]
2025-10-09T21:51:04.404462Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:04.404735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.404749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.404787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563717863:5250], now have 1 active actors on pipe
2025-10-09T21:51:04.404993Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:04.405100Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342862563717863:5250]
2025-10-09T21:51:04.405244Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:04.405341Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2022 reason: status is not ok: tablet will be restarted right now sessionId: src_id|23dd71d9-dd054577-98e72b55-8f4171bd_0
2025-10-09T21:51:04.405534Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2022 sessionId: src_id|23dd71d9-dd054577-98e72b55-8f4171bd_0 is DEAD
2025-10-09T21:51:04.405639Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:04.405693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23dd71d9-dd054577-98e72b55-8f4171bd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:04.405708Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23dd71d9-dd054577-98e72b55-8f4171bd_0] Write session will restart in 0.009628s
2025-10-09T21:51:04.405748Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23dd71d9-dd054577-98e72b55-8f4171bd_0] Write session: Do CDS request
2025-10-09T21:51:04.405754Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23dd71d9-dd054577-98e72b55-8f4171bd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:04.414843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:04.416513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23dd71d9-dd054577-98e72b55-8f4171bd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:04.416615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:04.416624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:04.416638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2023
2025-10-09T21:51:04.416729Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2023 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:04.416752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2023 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:04.416762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2023 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:04.416766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:04.416817Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:04.416826Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:04.416838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:04.416888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:04.416924Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342862201884148:10969]
2025-10-09T21:51:04.417155Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2023 sessionId:  describe result for acl check
2025-10-09T21:51:04.417191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:04.417197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:04.417199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:04.417204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342862563717931:10123] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:04.417528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:04.417758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342862563717931:10123] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:04.417779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:04.417815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:04.418132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:04.418176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.418181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.418182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.418183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.418184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.418185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.418186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.418188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.418189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.418193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.418203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.418207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.418211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.418215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.418218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.418222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.418225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.418228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.418234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.418241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.418248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.418253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.418262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.418269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.418277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.418283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.418287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.418296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.418300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.418305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.418314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.418319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.418324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.418328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.418334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.418340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.418366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:04.419614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342862563717931:10123] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:04.424009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342862563717931:10123] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:04.424022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342862563717931:10123] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:04.424025Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342862563717931:10123] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:04.424034Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2023 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:04.424522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:04.425600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:04.425611Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:04.425614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:04.425618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:04.425625Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2022 [2:7559342862201884148:10969]
2025-10-09T21:51:04.425637Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:04.425655Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:04.425662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:04.425664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:04.425666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:04.425678Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:04.425682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:04.425687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:04.425730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.425737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.425739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.425742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:04.425795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:04.425800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.425810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.425825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.425844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:04.425864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563717970:10123], now have 1 active actors on pipe
2025-10-09T21:51:04.425869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563717866:2432], now have 1 active actors on pipe
2025-10-09T21:51:04.425875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342862201884143:10969]
2025-10-09T21:51:04.425876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342862201884143:10969]
2025-10-09T21:51:04.425877Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342862201884143:10969]
2025-10-09T21:51:04.425886Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:04.425896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:04.425902Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:04.425905Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:04.425907Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:04.425908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:04.425911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:04.425927Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2022
2025-10-09T21:51:04.425929Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2022
2025-10-09T21:51:04.425929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342862201884143:10969]
2025-10-09T21:51:04.425937Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:04.425944Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:04.425961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.425964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.425989Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9b1f009c-6f914b13-18f937ed-a6564436_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:04.425996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:04.426005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:04.426007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:04.426018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:04.426043Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:04.426061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:04.426168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.426174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.426184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:04.426256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2023 partition: 0 MaxSeqNo: 25 sessionId: src_id|9b1f009c-6f914b13-18f937ed-a6564436_0
2025-10-09T21:51:04.426548Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23dd71d9-dd054577-98e72b55-8f4171bd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9b1f009c-6f914b13-18f937ed-a6564436_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:04.428894Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9b1f009c-6f914b13-18f937ed-a6564436_0] Write session: try to update token
2025-10-09T21:51:04.428908Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9b1f009c-6f914b13-18f937ed-a6564436_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:04.439468Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:04.439480Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:04.439482Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:04.439484Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:04.439492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342862201884143:10969]
2025-10-09T21:51:04.439495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342862201884143:10969]
2025-10-09T21:51:04.439497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342862201884143:10969]
2025-10-09T21:51:04.439530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:04.439535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.439536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:04.439540Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:04.439545Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:04.439548Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:04.439649Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:04.440851Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2023 sessionId: src_id|9b1f009c-6f914b13-18f937ed-a6564436_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:04.443157Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:04.505331Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342862563717977:5250]
2025-10-09T21:51:04.505626Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 2022, pipe: [1:7559342862563717977:5250]
2025-10-09T21:51:04.590820Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:04.591269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.591284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.591328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563717977:5250], now have 1 active actors on pipe
2025-10-09T21:51:04.591547Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342862563717977:5250]
2025-10-09T21:51:04.591586Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:04.591587Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:04.591591Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:04.591652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2023 reason: error at write: Disconnected sessionId: src_id|9b1f009c-6f914b13-18f937ed-a6564436_0
2025-10-09T21:51:04.591675Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:04.591826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2023 sessionId: src_id|9b1f009c-6f914b13-18f937ed-a6564436_0 is DEAD
2025-10-09T21:51:04.591965Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b1f009c-6f914b13-18f937ed-a6564436_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:04.591977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b1f009c-6f914b13-18f937ed-a6564436_0] Write session will restart in 0.009517s
2025-10-09T21:51:04.592013Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b1f009c-6f914b13-18f937ed-a6564436_0] Write session: Do CDS request
2025-10-09T21:51:04.592019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b1f009c-6f914b13-18f937ed-a6564436_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:04.601227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:04.602536Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9b1f009c-6f914b13-18f937ed-a6564436_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:04.602646Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:04.602663Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2024
2025-10-09T21:51:04.602787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2024 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:04.602819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2024 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:04.602830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2024 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:04.602833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:04.603172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:04.603336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2024 sessionId:  describe result for acl check
2025-10-09T21:51:04.603362Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:04.603372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:04.603370Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:04.603379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:04.603380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:04.603381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:04.603387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342862563718047:10114] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:04.603434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:04.603475Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342862201884252:10971]
2025-10-09T21:51:04.603915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:04.603948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342862563718047:10114] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:04.604263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:04.604329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:04.604673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:04.604739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.604746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.604747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.604748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.604750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.604751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.604753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.604754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.604755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.604761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.604771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.604776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.604780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.604783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.604788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.604792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.604795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.604799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.604807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.604815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.604822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.604827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.604833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.604839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.604844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.604849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.604854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.604859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.604863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.604869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.604874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.604879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.604890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.604895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.604901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.604911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.604939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:04.606128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342862563718047:10114] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:04.610835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342862563718047:10114] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:04.610849Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342862563718047:10114] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:04.610854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342862563718047:10114] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:04.610863Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2024 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:04.611282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:04.612417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:04.612437Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:04.612440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:04.612444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:04.612578Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2023 [2:7559342862201884252:10971]
2025-10-09T21:51:04.612598Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:04.612615Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:04.612618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:04.612624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:04.612626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:04.612644Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:04.612651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:04.612657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:04.612709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.612714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.612715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.612717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:04.612771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:04.612776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.612785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.612791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.612800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:04.612815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342862201884246:10971]
2025-10-09T21:51:04.612820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342862201884246:10971]
2025-10-09T21:51:04.612822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342862201884246:10971]
2025-10-09T21:51:04.612828Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:04.612839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:04.612839Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:04.612843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:04.612845Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:04.612848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:04.612850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:04.612859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563717980:2432], now have 1 active actors on pipe
2025-10-09T21:51:04.612861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563718085:10114], now have 1 active actors on pipe
2025-10-09T21:51:04.612890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342862201884246:10971]
2025-10-09T21:51:04.612901Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:04.612914Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:04.612938Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2023
2025-10-09T21:51:04.612957Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2023
2025-10-09T21:51:04.612987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.612998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.613020Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a8791dd7-df3293b9-2533adb6-3cd9f9a2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:04.613029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:04.613034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:04.613035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:04.613042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:04.613051Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:04.613065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:04.613201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.613207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.613217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:04.613304Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2024 partition: 0 MaxSeqNo: 25 sessionId: src_id|a8791dd7-df3293b9-2533adb6-3cd9f9a2_0
2025-10-09T21:51:04.613672Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9b1f009c-6f914b13-18f937ed-a6564436_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a8791dd7-df3293b9-2533adb6-3cd9f9a2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:04.616461Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a8791dd7-df3293b9-2533adb6-3cd9f9a2_0] Write session: try to update token
2025-10-09T21:51:04.616480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a8791dd7-df3293b9-2533adb6-3cd9f9a2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:04.627226Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:04.627244Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:04.627247Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:04.627248Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:04.627258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342862201884246:10971]
2025-10-09T21:51:04.627262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342862201884246:10971]
2025-10-09T21:51:04.627267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342862201884246:10971]
2025-10-09T21:51:04.627317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:04.627324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.627327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:04.627414Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:04.627428Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:04.627432Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:04.627483Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:04.630566Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2024 sessionId: src_id|a8791dd7-df3293b9-2533adb6-3cd9f9a2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:04.632958Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:04.691719Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342862563718092:5250]
2025-10-09T21:51:04.692025Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 2023, pipe: [1:7559342862563718092:5250]
2025-10-09T21:51:04.775430Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:04.775748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.775775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.775824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563718092:5250], now have 1 active actors on pipe
2025-10-09T21:51:04.776058Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:04.776107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:04.776111Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342862563718092:5250]
2025-10-09T21:51:04.776121Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:04.776110Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:04.776184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2024 reason: error at write: Disconnected sessionId: src_id|a8791dd7-df3293b9-2533adb6-3cd9f9a2_0
2025-10-09T21:51:04.776361Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2024 sessionId: src_id|a8791dd7-df3293b9-2533adb6-3cd9f9a2_0 is DEAD
2025-10-09T21:51:04.776525Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a8791dd7-df3293b9-2533adb6-3cd9f9a2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:04.776537Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a8791dd7-df3293b9-2533adb6-3cd9f9a2_0] Write session will restart in 0.007968s
2025-10-09T21:51:04.776574Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a8791dd7-df3293b9-2533adb6-3cd9f9a2_0] Write session: Do CDS request
2025-10-09T21:51:04.776581Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a8791dd7-df3293b9-2533adb6-3cd9f9a2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:04.785470Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a8791dd7-df3293b9-2533adb6-3cd9f9a2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:04.785782Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:04.785797Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2025
2025-10-09T21:51:04.785914Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2025 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:04.785947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2025 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:04.785954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2025 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:04.785960Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:04.786076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:04.786456Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2025 sessionId:  describe result for acl check
2025-10-09T21:51:04.786484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:04.786485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:04.786486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:04.786490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342862563718187:10102] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:04.786922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342862563718187:10102] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:04.787503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:04.787674Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:04.787684Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:04.787695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:04.787747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:04.787786Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342862201884399:10987]
2025-10-09T21:51:04.788334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:04.788589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:04.788632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:04.788688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342862563718187:10102] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:04.788938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:04.788994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.789000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.789002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.789003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.789004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.789005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.789007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.789008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.789009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.789014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.789024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.789028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.789032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.789036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.789040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.789043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.789047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.789050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.789059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.789066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.789073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.789078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.789084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.789090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.789094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.789099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.789104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.789108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.789112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.789117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.789121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.789130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.789134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.789141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.789145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.789149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.789177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:04.794175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342862563718187:10102] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:04.794188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342862563718187:10102] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:04.794192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342862563718187:10102] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:04.794200Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2025 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:04.795458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:04.796566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:04.796583Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:04.796585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:04.796589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:04.796598Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2024 [2:7559342862201884399:10987]
2025-10-09T21:51:04.796610Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:04.796625Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:04.796628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:04.796631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:04.796633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:04.796651Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:04.796661Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:04.796667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:04.796719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.796726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.796728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.796731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:04.796795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:04.796804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.796817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.796829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.796847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:04.796868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563718120:2432], now have 1 active actors on pipe
2025-10-09T21:51:04.796874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563718225:10102], now have 1 active actors on pipe
2025-10-09T21:51:04.796881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342862201884394:10987]
2025-10-09T21:51:04.796883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342862201884394:10987]
2025-10-09T21:51:04.796885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342862201884394:10987]
2025-10-09T21:51:04.796900Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:04.796916Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:04.796916Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:04.796920Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:04.796922Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:04.796935Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:04.796938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:04.796942Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2024
2025-10-09T21:51:04.796963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342862201884394:10987]
2025-10-09T21:51:04.796945Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2024
2025-10-09T21:51:04.796970Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:04.796979Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:04.796998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.797007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.797041Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|827cf783-f4795845-cfbba14d-88f9c2c0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:04.797054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:04.797058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:04.797060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:04.797069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:04.797083Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:04.797104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:04.797235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.797241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.797253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:04.797313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2025 partition: 0 MaxSeqNo: 25 sessionId: src_id|827cf783-f4795845-cfbba14d-88f9c2c0_0
2025-10-09T21:51:04.797649Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a8791dd7-df3293b9-2533adb6-3cd9f9a2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|827cf783-f4795845-cfbba14d-88f9c2c0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:04.800106Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|827cf783-f4795845-cfbba14d-88f9c2c0_0] Write session: try to update token
2025-10-09T21:51:04.800125Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|827cf783-f4795845-cfbba14d-88f9c2c0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:04.811475Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:04.811492Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:04.811495Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:04.811497Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:04.811507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342862201884394:10987]
2025-10-09T21:51:04.811511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342862201884394:10987]
2025-10-09T21:51:04.811513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342862201884394:10987]
2025-10-09T21:51:04.811551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:04.811558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.811566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:04.811583Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:04.811604Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:04.811610Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:04.811715Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:04.813254Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2025 sessionId: src_id|827cf783-f4795845-cfbba14d-88f9c2c0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:04.815741Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:04.876030Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342862563718233:5250]
2025-10-09T21:51:04.876298Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 2024, pipe: [1:7559342862563718233:5250]
2025-10-09T21:51:04.962212Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:04.962513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.962526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.962565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563718233:5250], now have 1 active actors on pipe
2025-10-09T21:51:04.962816Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:04.962843Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:04.962871Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342862563718233:5250]
2025-10-09T21:51:04.962895Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:04.962989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2025 reason: status is not ok: tablet will be restarted right now sessionId: src_id|827cf783-f4795845-cfbba14d-88f9c2c0_0
2025-10-09T21:51:04.963174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2025 sessionId: src_id|827cf783-f4795845-cfbba14d-88f9c2c0_0 is DEAD
2025-10-09T21:51:04.963343Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|827cf783-f4795845-cfbba14d-88f9c2c0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:04.963356Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|827cf783-f4795845-cfbba14d-88f9c2c0_0] Write session will restart in 0.009158s
2025-10-09T21:51:04.963390Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|827cf783-f4795845-cfbba14d-88f9c2c0_0] Write session: Do CDS request
2025-10-09T21:51:04.963394Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|827cf783-f4795845-cfbba14d-88f9c2c0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:04.972792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:04.973506Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|827cf783-f4795845-cfbba14d-88f9c2c0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:04.973551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:04.973576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2026
2025-10-09T21:51:04.973733Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2026 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:04.973764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2026 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:04.973771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2026 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:04.973774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:04.974222Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2026 sessionId:  describe result for acl check
2025-10-09T21:51:04.974258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:04.974260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:04.974262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:04.974268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342862563718310:10092] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:04.974572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:04.974839Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:04.974848Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:04.974839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342862563718310:10092] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:04.974861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:04.974917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:04.974947Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342862201884503:10989]
2025-10-09T21:51:04.975404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:04.975687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:04.975725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:04.975964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:04.976000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.976004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.976004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.976007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.976007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.976008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.976008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.976009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.976009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.976012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.976018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.976020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.976022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.976024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.976026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.976028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.976030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.976032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.976036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.976040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:04.976044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.976047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:04.976050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.976053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:04.976055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.976057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:04.976060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.976062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:04.976065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.976069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:04.976071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.976074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.976076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.976078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.976081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.976084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.976101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:04.976549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342862563718310:10092] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:04.983448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342862563718310:10092] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:04.983460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342862563718310:10092] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:04.983463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342862563718310:10092] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:04.983469Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2026 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:04.983826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:04.985230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:04.985245Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:04.985247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:04.985251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:04.985257Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2025 [2:7559342862201884503:10989]
2025-10-09T21:51:04.985270Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:04.985287Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:04.985295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:04.985299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:04.985301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:04.985321Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:04.985329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:04.985336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:04.985414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.985424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.985426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.985429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:04.985501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:04.985509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:04.985522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:04.985535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:04.985555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:04.985580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563718346:10092], now have 1 active actors on pipe
2025-10-09T21:51:04.985588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342862563718242:2432], now have 1 active actors on pipe
2025-10-09T21:51:04.985595Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2025
2025-10-09T21:51:04.985602Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2025
2025-10-09T21:51:04.985594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342862201884498:10989]
2025-10-09T21:51:04.985604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342862201884498:10989]
2025-10-09T21:51:04.985606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342862201884498:10989]
2025-10-09T21:51:04.985613Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:04.985628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:04.985641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:04.985641Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:04.985644Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:04.985644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:04.985646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:04.985695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.985704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.985719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342862201884498:10989]
2025-10-09T21:51:04.985732Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:04.985743Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:04.985763Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|24370179-2f3d57cd-70daddac-72bd94e3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:04.985774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:04.985778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:04.985780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:04.985791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:04.985804Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:04.985827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:04.985980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:04.985987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:04.986000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:04.986097Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2026 partition: 0 MaxSeqNo: 25 sessionId: src_id|24370179-2f3d57cd-70daddac-72bd94e3_0
2025-10-09T21:51:04.986350Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|827cf783-f4795845-cfbba14d-88f9c2c0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|24370179-2f3d57cd-70daddac-72bd94e3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:04.988495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|24370179-2f3d57cd-70daddac-72bd94e3_0] Write session: try to update token
2025-10-09T21:51:04.988509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|24370179-2f3d57cd-70daddac-72bd94e3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:05.002301Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2026 sessionId: src_id|24370179-2f3d57cd-70daddac-72bd94e3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:05.002750Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:05.002768Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:05.002770Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:05.002774Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:05.002784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342862201884498:10989]
2025-10-09T21:51:05.002787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342862201884498:10989]
2025-10-09T21:51:05.002794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342862201884498:10989]
2025-10-09T21:51:05.002838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:05.002845Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:05.002846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.002849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:05.002862Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:05.002866Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:05.003033Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:05.004677Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:05.062924Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342866858685649:5250]
2025-10-09T21:51:05.063237Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 2025, pipe: [1:7559342866858685649:5250]
2025-10-09T21:51:05.157982Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:05.158246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:05.158257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:05.158296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342866858685649:5250], now have 1 active actors on pipe
2025-10-09T21:51:05.158457Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:05.158510Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:05.158522Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:05.158529Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:05.158566Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342866858685649:5250]
2025-10-09T21:51:05.158581Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2026 reason: error at write: Disconnected sessionId: src_id|24370179-2f3d57cd-70daddac-72bd94e3_0
2025-10-09T21:51:05.158754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2026 sessionId: src_id|24370179-2f3d57cd-70daddac-72bd94e3_0 is DEAD
2025-10-09T21:51:05.158911Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24370179-2f3d57cd-70daddac-72bd94e3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:05.158932Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24370179-2f3d57cd-70daddac-72bd94e3_0] Write session will restart in 0.005062s
2025-10-09T21:51:05.158959Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24370179-2f3d57cd-70daddac-72bd94e3_0] Write session: Do CDS request
2025-10-09T21:51:05.158966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24370179-2f3d57cd-70daddac-72bd94e3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:05.165511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|24370179-2f3d57cd-70daddac-72bd94e3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:05.165594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:05.165607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2027
2025-10-09T21:51:05.165732Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2027 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:05.165764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2027 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:05.165775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2027 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:05.165778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:05.166212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2027 sessionId:  describe result for acl check
2025-10-09T21:51:05.166248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:05.166264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:05.166270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:05.166275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342866858685677:10093] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:05.166808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342866858685677:10093] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:05.168405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:05.168630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342866858685677:10093] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:05.170204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:05.170375Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:05.170382Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:05.170392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:05.170443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:05.170484Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342866496851906:10991]
2025-10-09T21:51:05.170937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:05.170962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342866858685677:10093] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:05.170974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342866858685677:10093] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:05.170977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342866858685677:10093] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:05.170984Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2027 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:05.171251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:05.171301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:05.171573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:05.171614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.171615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.171615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.171616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.171617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.171617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.171618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.171618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.171619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.171622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.171629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.171631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.171633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.171635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.171637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.171639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.171641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.171643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.171647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.171652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.171656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.171659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.171662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.171665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.171667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.171670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.171673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.171675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.171679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.171684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.171688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.171693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.171698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.171702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.171706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.171711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.171739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:05.179468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:05.180687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:05.180699Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:05.180701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:05.180704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:05.180709Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2026 [2:7559342866496851906:10991]
2025-10-09T21:51:05.180719Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:05.180734Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:05.180738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:05.180741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:05.180743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:05.180763Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:05.180775Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:05.180783Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:05.180843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.180848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.180849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.180850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:05.180907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:05.180912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.180923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.180938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.180960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:05.180983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342866858685652:2432], now have 1 active actors on pipe
2025-10-09T21:51:05.180991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342866858685756:10093], now have 1 active actors on pipe
2025-10-09T21:51:05.180999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342866496851901:10991]
2025-10-09T21:51:05.181006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342866496851901:10991]
2025-10-09T21:51:05.181009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342866496851901:10991]
2025-10-09T21:51:05.181022Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:05.181040Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:05.181044Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:05.181047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:05.181049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:05.181059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:05.181062Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:05.181060Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2026
2025-10-09T21:51:05.181074Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2026
2025-10-09T21:51:05.181090Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342866496851901:10991]
2025-10-09T21:51:05.181096Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:05.181098Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:05.181112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:05.181116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:05.181151Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9812393a-989a90de-23e5634a-d6ee86f6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:05.181162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:05.181168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:05.181170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:05.181178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:05.181189Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:05.181209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:05.181380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:05.181391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:05.181420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:05.181530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2027 partition: 0 MaxSeqNo: 25 sessionId: src_id|9812393a-989a90de-23e5634a-d6ee86f6_0
2025-10-09T21:51:05.181827Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|24370179-2f3d57cd-70daddac-72bd94e3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9812393a-989a90de-23e5634a-d6ee86f6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:05.184136Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9812393a-989a90de-23e5634a-d6ee86f6_0] Write session: try to update token
2025-10-09T21:51:05.184154Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9812393a-989a90de-23e5634a-d6ee86f6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:05.196657Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2027 sessionId: src_id|9812393a-989a90de-23e5634a-d6ee86f6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:05.199068Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:05.199458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:05.199486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:05.199509Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:05.199512Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:05.199514Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:05.199517Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:05.199525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342866496851901:10991]
2025-10-09T21:51:05.199529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342866496851901:10991]
2025-10-09T21:51:05.199534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342866496851901:10991]
2025-10-09T21:51:05.199576Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:05.199588Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:05.199592Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:05.199604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:05.199610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.199613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:05.199747Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:05.258489Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342866858685771:5250]
2025-10-09T21:51:05.258792Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 2026, pipe: [1:7559342866858685771:5250]
2025-10-09T21:51:05.349287Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:05.349560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:05.349615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342866858685771:5250], now have 1 active actors on pipe
2025-10-09T21:51:05.349711Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:05.349774Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:05.349806Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:05.349814Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:05.349828Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342866858685771:5250]
2025-10-09T21:51:05.349876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2027 reason: error at write: Disconnected sessionId: src_id|9812393a-989a90de-23e5634a-d6ee86f6_0
2025-10-09T21:51:05.350002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2027 sessionId: src_id|9812393a-989a90de-23e5634a-d6ee86f6_0 is DEAD
2025-10-09T21:51:05.350203Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9812393a-989a90de-23e5634a-d6ee86f6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:05.350217Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9812393a-989a90de-23e5634a-d6ee86f6_0] Write session will restart in 0.006099s
2025-10-09T21:51:05.350240Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9812393a-989a90de-23e5634a-d6ee86f6_0] Write session: Do CDS request
2025-10-09T21:51:05.350250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9812393a-989a90de-23e5634a-d6ee86f6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:05.357524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9812393a-989a90de-23e5634a-d6ee86f6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:05.357624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:05.357639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2028
2025-10-09T21:51:05.357739Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2028 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:05.357784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2028 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:05.357795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2028 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:05.357799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:05.358341Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2028 sessionId:  describe result for acl check
2025-10-09T21:51:05.358379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:05.358386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:05.358388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:05.358394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342866858685833:10082] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:05.358964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342866858685833:10082] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:05.360247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:05.360717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342866858685833:10082] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:05.361917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:05.362094Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:05.362109Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:05.362123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:05.362166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:05.362198Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342866496852014:10993]
2025-10-09T21:51:05.362584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:05.362837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:05.362878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:05.362968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342866858685833:10082] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:05.362982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342866858685833:10082] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:05.362985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342866858685833:10082] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:05.362994Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2028 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:05.363203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:05.363253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.363257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.363258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.363259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.363260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.363260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.363261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.363262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.363262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.363265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.363272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.363275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.363277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.363279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.363281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.363283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.363285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.363287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.363292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.363297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.363301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.363304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.363306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.363309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.363312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.363315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.363318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.363320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.363324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.363327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.363330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.363332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.363336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.363339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.363342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.363344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.363361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:05.370954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:05.372018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:05.372029Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:05.372032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:05.372036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:05.372043Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2027 [2:7559342866496852014:10993]
2025-10-09T21:51:05.372056Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:05.372073Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:05.372082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:05.372085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:05.372087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:05.372103Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:05.372114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:05.372121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:05.372176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.372183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.372185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.372187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:05.372257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:05.372263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.372270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.372276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.372291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:05.372308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342866496852008:10993]
2025-10-09T21:51:05.372309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342866496852008:10993]
2025-10-09T21:51:05.372311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342866496852008:10993]
2025-10-09T21:51:05.372317Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:05.372327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:05.372329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342866858685885:10082], now have 1 active actors on pipe
2025-10-09T21:51:05.372329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:05.372331Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:05.372331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342866858685778:2432], now have 1 active actors on pipe
2025-10-09T21:51:05.372338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:05.372346Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:05.372348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:05.372367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342866496852008:10993]
2025-10-09T21:51:05.372378Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:05.372401Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:05.372403Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2027
2025-10-09T21:51:05.372410Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2027
2025-10-09T21:51:05.372498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:05.372508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:05.372533Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|11c27b5e-25c7a1f2-453f1c8d-e5eacd40_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:05.372539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:05.372542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:05.372544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:05.372551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:05.372559Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:05.372575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:05.372740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:05.372750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:05.372778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:05.372870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2028 partition: 0 MaxSeqNo: 25 sessionId: src_id|11c27b5e-25c7a1f2-453f1c8d-e5eacd40_0
2025-10-09T21:51:05.373192Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9812393a-989a90de-23e5634a-d6ee86f6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|11c27b5e-25c7a1f2-453f1c8d-e5eacd40_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:05.375478Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|11c27b5e-25c7a1f2-453f1c8d-e5eacd40_0] Write session: try to update token
2025-10-09T21:51:05.375493Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|11c27b5e-25c7a1f2-453f1c8d-e5eacd40_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:05.389620Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:05.389635Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:05.389637Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:05.389639Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:05.389649Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342866496852008:10993]
2025-10-09T21:51:05.389653Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342866496852008:10993]
2025-10-09T21:51:05.389660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342866496852008:10993]
2025-10-09T21:51:05.389705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:05.389712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.389715Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:05.389720Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:05.389721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:05.389723Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:05.389838Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2028 sessionId: src_id|11c27b5e-25c7a1f2-453f1c8d-e5eacd40_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:05.389871Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:05.392202Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:05.449981Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342866858685895:5250]
2025-10-09T21:51:05.450252Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 2027, pipe: [1:7559342866858685895:5250]
2025-10-09T21:51:05.535516Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:05.535863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:05.535881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:05.535926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342866858685895:5250], now have 1 active actors on pipe
2025-10-09T21:51:05.536136Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:05.536151Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:05.536158Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342866858685895:5250]
2025-10-09T21:51:05.536180Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:05.536156Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:05.536216Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2028 reason: error at write: Disconnected sessionId: src_id|11c27b5e-25c7a1f2-453f1c8d-e5eacd40_0
2025-10-09T21:51:05.536406Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2028 sessionId: src_id|11c27b5e-25c7a1f2-453f1c8d-e5eacd40_0 is DEAD
2025-10-09T21:51:05.536546Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11c27b5e-25c7a1f2-453f1c8d-e5eacd40_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:05.536560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11c27b5e-25c7a1f2-453f1c8d-e5eacd40_0] Write session will restart in 0.007575s
2025-10-09T21:51:05.536586Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11c27b5e-25c7a1f2-453f1c8d-e5eacd40_0] Write session: Do CDS request
2025-10-09T21:51:05.536593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11c27b5e-25c7a1f2-453f1c8d-e5eacd40_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:05.545571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|11c27b5e-25c7a1f2-453f1c8d-e5eacd40_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:05.545596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:05.545610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2029
2025-10-09T21:51:05.545832Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2029 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:05.545866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2029 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:05.545875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2029 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:05.545878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:05.546318Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2029 sessionId:  describe result for acl check
2025-10-09T21:51:05.546356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:05.546363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:05.546365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:05.546371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342866858685966:10077] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:05.546911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:05.546950Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342866858685966:10077] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:05.548549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:05.548733Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:05.548744Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:05.548755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:05.548810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:05.548852Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342866496852122:10995]
2025-10-09T21:51:05.549205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342866858685966:10077] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:05.549443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:05.549702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:05.549742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:05.549973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:05.550008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.550013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.550014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.550014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.550015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.550016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.550016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.550017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.550018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.550021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.550042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.550044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.550047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.550049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.550051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.550053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.550055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.550057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.550061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.550066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.550070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.550073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.550076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.550079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.550081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.550084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.550087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.550089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.550092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.550095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.550098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.550102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.550106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.550111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.550116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.550121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.550145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:05.555570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342866858685966:10077] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:05.555580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342866858685966:10077] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:05.555582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342866858685966:10077] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:05.555589Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2029 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:05.557924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:05.559173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:05.559186Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:05.559188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:05.559192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:05.559199Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2028 [2:7559342866496852122:10995]
2025-10-09T21:51:05.559211Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:05.559233Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:05.559240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:05.559244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:05.559247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:05.559264Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:05.559274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:05.559281Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:05.559337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.559344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.559346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.559348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:05.559413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:05.559420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.559431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.559443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.559463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:05.559487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342866858686006:10077], now have 1 active actors on pipe
2025-10-09T21:51:05.559492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342866858685897:2432], now have 1 active actors on pipe
2025-10-09T21:51:05.559499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342866496852116:10995]
2025-10-09T21:51:05.559505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342866496852116:10995]
2025-10-09T21:51:05.559507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342866496852116:10995]
2025-10-09T21:51:05.559518Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:05.559507Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2028
2025-10-09T21:51:05.559522Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2028
2025-10-09T21:51:05.559531Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:05.559534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:05.559539Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:05.559540Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:05.559548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:05.559551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:05.559586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:05.559595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:05.559613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342866496852116:10995]
2025-10-09T21:51:05.559625Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:05.559635Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:05.559661Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|72250fcc-d457d6fa-94c09f0c-a0acc1c6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:05.559668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:05.559672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:05.559676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:05.559685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:05.559698Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:05.559762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:05.559909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:05.559918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:05.559932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:05.560003Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2029 partition: 0 MaxSeqNo: 25 sessionId: src_id|72250fcc-d457d6fa-94c09f0c-a0acc1c6_0
2025-10-09T21:51:05.560321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11c27b5e-25c7a1f2-453f1c8d-e5eacd40_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|72250fcc-d457d6fa-94c09f0c-a0acc1c6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:05.562575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72250fcc-d457d6fa-94c09f0c-a0acc1c6_0] Write session: try to update token
2025-10-09T21:51:05.562589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72250fcc-d457d6fa-94c09f0c-a0acc1c6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:05.574017Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2029 sessionId: src_id|72250fcc-d457d6fa-94c09f0c-a0acc1c6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:05.576296Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:05.577113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:05.577136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:05.577156Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:05.577159Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:05.577162Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:05.577165Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:05.577173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342866496852116:10995]
2025-10-09T21:51:05.577177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342866496852116:10995]
2025-10-09T21:51:05.577181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342866496852116:10995]
2025-10-09T21:51:05.577222Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:05.577231Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:05.577234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:05.577235Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:05.577240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.577243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:05.577367Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:05.636356Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342866858686015:5250]
2025-10-09T21:51:05.636594Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 2028, pipe: [1:7559342866858686015:5250]
2025-10-09T21:51:05.714742Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:05.715075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:05.715127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342866858686015:5250], now have 1 active actors on pipe
2025-10-09T21:51:05.715197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:05.715276Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342866858686015:5250]
2025-10-09T21:51:05.715300Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:05.725187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:05.726454Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:05.726464Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:05.726483Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2029 reason: error at write: Disconnected sessionId: src_id|72250fcc-d457d6fa-94c09f0c-a0acc1c6_0
2025-10-09T21:51:05.726592Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2029 sessionId: src_id|72250fcc-d457d6fa-94c09f0c-a0acc1c6_0 is DEAD
2025-10-09T21:51:05.726743Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72250fcc-d457d6fa-94c09f0c-a0acc1c6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:05.726756Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72250fcc-d457d6fa-94c09f0c-a0acc1c6_0] Write session will restart in 0.007932s
2025-10-09T21:51:05.726780Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72250fcc-d457d6fa-94c09f0c-a0acc1c6_0] Write session: Do CDS request
2025-10-09T21:51:05.726785Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72250fcc-d457d6fa-94c09f0c-a0acc1c6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:05.731629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:05.731931Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:05.731942Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:05.731955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:05.732020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:05.732060Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342866496852231:10997]
2025-10-09T21:51:05.732650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:05.732967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:05.733011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:05.733328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:05.733407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.733414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.733415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.733416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.733418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.733419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.733422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.733423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.733425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.733430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.733441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.733445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.733450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.733453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.733457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.733461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.733465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.733469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.733477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.733486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.733491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.733496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.733502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.733507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.733511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.733516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.733519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.733525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.733529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.733534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.733539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.733543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.733548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.733553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.733558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.733563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.733592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:05.735487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|72250fcc-d457d6fa-94c09f0c-a0acc1c6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:05.735552Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:05.735566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2030
2025-10-09T21:51:05.735716Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2030 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:05.735749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2030 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:05.735759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2030 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:05.735765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:05.736166Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2030 sessionId:  describe result for acl check
2025-10-09T21:51:05.736199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:05.736208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:05.736210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:05.736215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342866858686095:10071] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:05.736778Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342866858686095:10071] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:05.739020Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342866858686095:10071] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:05.740462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:05.741426Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342866858686095:10071] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:05.741440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342866858686095:10071] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:05.741444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342866858686095:10071] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:05.741451Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2030 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:05.741535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:05.741548Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:05.741550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:05.741554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:05.741560Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2029 [2:7559342866496852231:10997]
2025-10-09T21:51:05.741572Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:05.741588Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:05.741597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:05.741600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:05.741602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:05.741617Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:05.741623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:05.741629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:05.741690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.741697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.741699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.741702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:05.741765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:05.741772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.741785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.741802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.741823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:05.741847Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2029
2025-10-09T21:51:05.741850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342866858686017:2432], now have 1 active actors on pipe
2025-10-09T21:51:05.741852Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2029
2025-10-09T21:51:05.741863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342866858686129:10071], now have 1 active actors on pipe
2025-10-09T21:51:05.741871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342866496852225:10997]
2025-10-09T21:51:05.741873Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342866496852225:10997]
2025-10-09T21:51:05.741878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342866496852225:10997]
2025-10-09T21:51:05.741886Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:05.741894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:05.741897Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:05.741899Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:05.741899Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:05.741913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:05.741916Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:05.741932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:05.741941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:05.741978Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342866496852225:10997]
2025-10-09T21:51:05.741991Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:05.742002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:05.742010Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cf442b95-b88ef399-a1aab03d-b4ce8bb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:05.742018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:05.742044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:05.742046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:05.742056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:05.742070Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:05.742135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:05.742275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:05.742285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:05.742300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:05.742386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2030 partition: 0 MaxSeqNo: 25 sessionId: src_id|cf442b95-b88ef399-a1aab03d-b4ce8bb_0
2025-10-09T21:51:05.742660Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|72250fcc-d457d6fa-94c09f0c-a0acc1c6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cf442b95-b88ef399-a1aab03d-b4ce8bb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:05.744985Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf442b95-b88ef399-a1aab03d-b4ce8bb_0] Write session: try to update token
2025-10-09T21:51:05.745002Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf442b95-b88ef399-a1aab03d-b4ce8bb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:05.757264Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2030 sessionId: src_id|cf442b95-b88ef399-a1aab03d-b4ce8bb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:05.757278Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:05.757294Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:05.757296Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:05.757298Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:05.757306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342866496852225:10997]
2025-10-09T21:51:05.757311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342866496852225:10997]
2025-10-09T21:51:05.757315Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342866496852225:10997]
2025-10-09T21:51:05.757349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:05.757354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.757354Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:05.757359Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:05.757360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:05.757362Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:05.757512Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:05.759614Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:05.815545Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342866858686158:5250]
2025-10-09T21:51:05.815814Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 2029, pipe: [1:7559342866858686158:5250]
2025-10-09T21:51:05.900623Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:05.901247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:05.901263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:05.901313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342866858686158:5250], now have 1 active actors on pipe
2025-10-09T21:51:05.901497Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:05.901605Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:05.901607Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342866858686158:5250]
2025-10-09T21:51:05.901626Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:05.901627Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:05.901685Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2030 reason: error at write: Disconnected sessionId: src_id|cf442b95-b88ef399-a1aab03d-b4ce8bb_0
2025-10-09T21:51:05.901881Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2030 sessionId: src_id|cf442b95-b88ef399-a1aab03d-b4ce8bb_0 is DEAD
2025-10-09T21:51:05.902019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf442b95-b88ef399-a1aab03d-b4ce8bb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:05.902041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf442b95-b88ef399-a1aab03d-b4ce8bb_0] Write session will restart in 0.007252s
2025-10-09T21:51:05.902061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf442b95-b88ef399-a1aab03d-b4ce8bb_0] Write session: Do CDS request
2025-10-09T21:51:05.902066Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf442b95-b88ef399-a1aab03d-b4ce8bb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:05.910572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf442b95-b88ef399-a1aab03d-b4ce8bb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:05.910653Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:05.910666Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2031
2025-10-09T21:51:05.910811Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2031 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:05.910851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2031 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:05.910860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2031 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:05.910864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:05.911252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2031 sessionId:  describe result for acl check
2025-10-09T21:51:05.911282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:05.911285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:05.911287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:05.911291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342866858686237:10053] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:05.911487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:05.911801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342866858686237:10053] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:05.913129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:05.913310Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:05.913319Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:05.913331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:05.913380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:05.913433Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342866496852382:11013]
2025-10-09T21:51:05.913722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342866858686237:10053] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:05.914002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:05.914314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:05.914359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:05.914665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:05.914714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.914719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.914721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.914722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.914723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.914724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.914726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.914727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.914728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.914732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.914743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.914747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.914751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.914755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.914759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.914762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.914766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.914770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.914777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.914785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:05.914790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.914796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:05.914800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.914805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:05.914809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.914814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:05.914818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.914823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:05.914827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.914832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:05.914837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.914847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.914852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.914857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.914869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.914878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.914905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:05.920155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342866858686237:10053] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:05.920167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342866858686237:10053] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:05.920170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342866858686237:10053] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:05.920176Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2031 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:05.921600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:05.922721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:05.922734Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:05.922737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:05.922741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:05.922747Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2030 [2:7559342866496852382:11013]
2025-10-09T21:51:05.922759Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:05.922777Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:05.922781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:05.922784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:05.922787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:05.922804Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:05.922815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:05.922822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:05.922875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.922883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.922885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.922888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:05.922952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:05.922959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.922971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:05.922983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:05.923006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:05.923028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342866858686277:10053], now have 1 active actors on pipe
2025-10-09T21:51:05.923036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342866858686164:2432], now have 1 active actors on pipe
2025-10-09T21:51:05.923043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342866496852377:11013]
2025-10-09T21:51:05.923045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342866496852377:11013]
2025-10-09T21:51:05.923047Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342866496852377:11013]
2025-10-09T21:51:05.923058Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:05.923078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:05.923084Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:05.923088Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:05.923064Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2030
2025-10-09T21:51:05.923090Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:05.923096Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:05.923092Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2030
2025-10-09T21:51:05.923099Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:05.923124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342866496852377:11013]
2025-10-09T21:51:05.923132Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:05.923134Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:05.923148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:05.923157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:05.923206Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d05f7c11-3014b6cc-e4e0a526-9003fec1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:05.923218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:05.923222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:05.923224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:05.923234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:05.923248Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:05.923265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:05.923432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:05.923441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:05.923455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:05.923526Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2031 partition: 0 MaxSeqNo: 25 sessionId: src_id|d05f7c11-3014b6cc-e4e0a526-9003fec1_0
2025-10-09T21:51:05.923862Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf442b95-b88ef399-a1aab03d-b4ce8bb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d05f7c11-3014b6cc-e4e0a526-9003fec1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:05.926323Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d05f7c11-3014b6cc-e4e0a526-9003fec1_0] Write session: try to update token
2025-10-09T21:51:05.926340Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d05f7c11-3014b6cc-e4e0a526-9003fec1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:05.937316Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:05.937333Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:05.937336Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:05.937338Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:05.937348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342866496852377:11013]
2025-10-09T21:51:05.937353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342866496852377:11013]
2025-10-09T21:51:05.937356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342866496852377:11013]
2025-10-09T21:51:05.937407Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:05.937410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:05.937413Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:05.937417Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:05.937418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:05.937421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:05.937552Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:05.938654Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2031 sessionId: src_id|d05f7c11-3014b6cc-e4e0a526-9003fec1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:05.940908Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:06.001898Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342871153653581:5250]
2025-10-09T21:51:06.002118Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 2030, pipe: [1:7559342871153653581:5250]
2025-10-09T21:51:06.082065Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:06.082391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:06.082405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:06.082441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342871153653581:5250], now have 1 active actors on pipe
2025-10-09T21:51:06.082613Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:06.082705Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:06.082718Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:06.082769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2031 reason: error at write: Disconnected sessionId: src_id|d05f7c11-3014b6cc-e4e0a526-9003fec1_0
2025-10-09T21:51:06.082802Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:06.082856Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342871153653581:5250]
2025-10-09T21:51:06.082947Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2031 sessionId: src_id|d05f7c11-3014b6cc-e4e0a526-9003fec1_0 is DEAD
2025-10-09T21:51:06.083343Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d05f7c11-3014b6cc-e4e0a526-9003fec1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:06.083354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d05f7c11-3014b6cc-e4e0a526-9003fec1_0] Write session will restart in 0.006484s
2025-10-09T21:51:06.083375Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d05f7c11-3014b6cc-e4e0a526-9003fec1_0] Write session: Do CDS request
2025-10-09T21:51:06.083381Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d05f7c11-3014b6cc-e4e0a526-9003fec1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:06.090475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d05f7c11-3014b6cc-e4e0a526-9003fec1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:06.090584Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:06.090597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2032
2025-10-09T21:51:06.090701Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2032 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:06.090735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2032 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:06.090746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2032 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:06.090750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:06.091167Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2032 sessionId:  describe result for acl check
2025-10-09T21:51:06.091199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:06.091207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:06.091209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:06.091215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342871153653641:10049] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:06.091749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342871153653641:10049] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:06.092658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:06.093843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342871153653641:10049] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:06.094198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:06.094368Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:06.094379Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:06.094390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:06.094440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:06.094478Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342870791819789:11015]
2025-10-09T21:51:06.095047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:06.095304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:06.095346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:06.095657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:06.095703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.095709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.095710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.095711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.095713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.095714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.095715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.095716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.095717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.095722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.095733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.095737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.095744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.095752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.095756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.095764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.095768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.095772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.095778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.095787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.095797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.095807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.095818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.095826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.095834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.095839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.095848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.095853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.095857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.095863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.095868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.095873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.095882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.095887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.095892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.095901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.095929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:06.095966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342871153653641:10049] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:06.095976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342871153653641:10049] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:06.095981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342871153653641:10049] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:06.095986Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2032 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:06.103666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:06.104716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:06.104732Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:06.104735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:06.104740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:06.104748Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2031 [2:7559342870791819789:11015]
2025-10-09T21:51:06.104903Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:06.104927Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:06.104935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:06.104938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:06.104941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:06.104959Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:06.104970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:06.104977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:06.105034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.105040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.105041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.105043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:06.105109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:06.105117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.105129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.105141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.105156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:06.105183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342871153653697:10049], now have 1 active actors on pipe
2025-10-09T21:51:06.105189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342871153653583:2432], now have 1 active actors on pipe
2025-10-09T21:51:06.105195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342870791819784:11015]
2025-10-09T21:51:06.105197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342870791819784:11015]
2025-10-09T21:51:06.105199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342870791819784:11015]
2025-10-09T21:51:06.105210Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:06.105226Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:06.105230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:06.105234Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:06.105236Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:06.105244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:06.105247Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:06.105238Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2031
2025-10-09T21:51:06.105239Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2031
2025-10-09T21:51:06.105289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342870791819784:11015]
2025-10-09T21:51:06.105300Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:06.105306Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:06.105319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:06.105324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:06.105375Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b8ea85c3-40ea8ad0-1a8510fb-cb0c945c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:06.105388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:06.105405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:06.105412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:06.105425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:06.105439Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:06.105461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:06.105602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:06.105610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:06.105625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:06.105741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2032 partition: 0 MaxSeqNo: 25 sessionId: src_id|b8ea85c3-40ea8ad0-1a8510fb-cb0c945c_0
2025-10-09T21:51:06.106112Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d05f7c11-3014b6cc-e4e0a526-9003fec1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b8ea85c3-40ea8ad0-1a8510fb-cb0c945c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:06.108496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b8ea85c3-40ea8ad0-1a8510fb-cb0c945c_0] Write session: try to update token
2025-10-09T21:51:06.108516Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b8ea85c3-40ea8ad0-1a8510fb-cb0c945c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:06.120058Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:06.120075Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:06.120077Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:06.120079Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:06.120088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342870791819784:11015]
2025-10-09T21:51:06.120092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342870791819784:11015]
2025-10-09T21:51:06.120095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342870791819784:11015]
2025-10-09T21:51:06.120131Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:06.120136Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:06.120136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:06.120139Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:06.120142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.120143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:06.120282Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:06.121902Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2032 sessionId: src_id|b8ea85c3-40ea8ad0-1a8510fb-cb0c945c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:06.124284Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:06.183165Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342871153653707:5250]
2025-10-09T21:51:06.183375Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 2031, pipe: [1:7559342871153653707:5250]
2025-10-09T21:51:06.264358Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:06.264587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:06.264599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:06.264637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342871153653707:5250], now have 1 active actors on pipe
2025-10-09T21:51:06.264800Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:06.264853Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:06.264867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:06.264932Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2032 reason: error at write: Disconnected sessionId: src_id|b8ea85c3-40ea8ad0-1a8510fb-cb0c945c_0
2025-10-09T21:51:06.264934Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342871153653707:5250]
2025-10-09T21:51:06.264955Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:06.265098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2032 sessionId: src_id|b8ea85c3-40ea8ad0-1a8510fb-cb0c945c_0 is DEAD
2025-10-09T21:51:06.265228Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8ea85c3-40ea8ad0-1a8510fb-cb0c945c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:06.265238Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8ea85c3-40ea8ad0-1a8510fb-cb0c945c_0] Write session will restart in 0.009369s
2025-10-09T21:51:06.265261Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8ea85c3-40ea8ad0-1a8510fb-cb0c945c_0] Write session: Do CDS request
2025-10-09T21:51:06.265268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8ea85c3-40ea8ad0-1a8510fb-cb0c945c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:06.275301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:06.275513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b8ea85c3-40ea8ad0-1a8510fb-cb0c945c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:06.275603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:06.275630Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2033
2025-10-09T21:51:06.275731Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2033 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:06.275763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2033 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:06.275773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2033 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:06.275779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:06.276195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2033 sessionId:  describe result for acl check
2025-10-09T21:51:06.276233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:06.276241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:06.276243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:06.276249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342871153653788:10044] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:06.276791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342871153653788:10044] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:06.277056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:06.277275Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:06.277287Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:06.277302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:06.277364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:06.277415Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342870791819902:11017]
2025-10-09T21:51:06.277982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:06.278236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:06.278280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:06.278585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:06.278638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.278645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.278646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.278647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.278648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.278651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342871153653788:10044] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:06.278649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.278650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.278651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.278652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.278657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.278667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.278671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.278674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.278677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.278680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.278683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.278687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.278691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.278696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.278704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.278710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.278716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.278720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.278725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.278729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.278734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.278743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.278748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.278752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.278757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.278762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.278767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.278771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.278776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.278780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.278784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.278812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:06.285045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342871153653788:10044] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:06.285057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342871153653788:10044] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:06.285059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342871153653788:10044] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:06.285066Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2033 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:06.285954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:06.287105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:06.287121Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:06.287123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:06.287127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:06.287134Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2032 [2:7559342870791819902:11017]
2025-10-09T21:51:06.287147Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:06.287163Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:06.287166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:06.287169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:06.287172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:06.287189Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:06.287200Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:06.287208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:06.287267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.287275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.287278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.287280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:06.287352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:06.287360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.287373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.287390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.287408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:06.287433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342871153653828:10044], now have 1 active actors on pipe
2025-10-09T21:51:06.287441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342871153653713:2432], now have 1 active actors on pipe
2025-10-09T21:51:06.287448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342870791819896:11017]
2025-10-09T21:51:06.287454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342870791819896:11017]
2025-10-09T21:51:06.287456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342870791819896:11017]
2025-10-09T21:51:06.287469Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:06.287487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:06.287496Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:06.287499Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:06.287500Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:06.287461Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2032
2025-10-09T21:51:06.287502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:06.287468Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2032
2025-10-09T21:51:06.287505Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:06.287541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:06.287546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:06.287562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342870791819896:11017]
2025-10-09T21:51:06.287575Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:06.287602Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:06.287614Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|da57a446-dfca48de-8c5202ed-577ae48_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:06.287621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:06.287624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:06.287627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:06.287636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:06.287650Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:06.287676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:06.287851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:06.287861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:06.287877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:06.288012Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2033 partition: 0 MaxSeqNo: 25 sessionId: src_id|da57a446-dfca48de-8c5202ed-577ae48_0
2025-10-09T21:51:06.288338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8ea85c3-40ea8ad0-1a8510fb-cb0c945c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|da57a446-dfca48de-8c5202ed-577ae48_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:06.290541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da57a446-dfca48de-8c5202ed-577ae48_0] Write session: try to update token
2025-10-09T21:51:06.290556Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da57a446-dfca48de-8c5202ed-577ae48_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:06.302506Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:06.302525Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:06.302528Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:06.302530Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:06.302539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342870791819896:11017]
2025-10-09T21:51:06.302544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342870791819896:11017]
2025-10-09T21:51:06.302547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342870791819896:11017]
2025-10-09T21:51:06.302586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:06.302588Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:06.302594Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:06.302596Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:06.302597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.302599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:06.302715Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:06.303251Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2033 sessionId: src_id|da57a446-dfca48de-8c5202ed-577ae48_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:06.305654Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:06.364934Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342871153653840:5250]
2025-10-09T21:51:06.365195Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 2032, pipe: [1:7559342871153653840:5250]
2025-10-09T21:51:06.447555Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:06.447911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:06.447924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:06.447961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342871153653840:5250], now have 1 active actors on pipe
2025-10-09T21:51:06.448123Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:06.448204Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342871153653840:5250]
2025-10-09T21:51:06.448209Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:06.448238Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:06.448278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2033 reason: status is not ok: tablet will be restarted right now sessionId: src_id|da57a446-dfca48de-8c5202ed-577ae48_0
2025-10-09T21:51:06.448461Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2033 sessionId: src_id|da57a446-dfca48de-8c5202ed-577ae48_0 is DEAD
2025-10-09T21:51:06.448622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da57a446-dfca48de-8c5202ed-577ae48_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:06.448631Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da57a446-dfca48de-8c5202ed-577ae48_0] Write session will restart in 0.008088s
2025-10-09T21:51:06.448650Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da57a446-dfca48de-8c5202ed-577ae48_0] Write session: Do CDS request
2025-10-09T21:51:06.448655Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da57a446-dfca48de-8c5202ed-577ae48_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:06.457512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da57a446-dfca48de-8c5202ed-577ae48_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:06.457550Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:06.457561Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2034
2025-10-09T21:51:06.457726Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2034 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:06.457769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2034 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:06.457781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2034 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:06.457785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:06.458258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2034 sessionId:  describe result for acl check
2025-10-09T21:51:06.458292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:06.458298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:06.458300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:06.458305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342871153653918:10037] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:06.458327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:06.458854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342871153653918:10037] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:06.460031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:06.460210Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:06.460221Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:06.460233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:06.460287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:06.460327Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342870791820016:11019]
2025-10-09T21:51:06.460847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:06.460991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342871153653918:10037] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:06.461130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:06.461180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:06.461514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:06.461585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.461591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.461592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.461593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.461594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.461595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.461596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.461597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.461599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.461604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.461614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.461618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.461622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.461626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.461630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.461634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.461638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.461642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.461649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.461656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.461665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.461675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.461680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.461690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.461694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.461700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.461709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.461714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.461719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.461728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.461733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.461738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.461747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.461754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.461758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.461767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.461796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:06.467305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342871153653918:10037] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:06.467328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342871153653918:10037] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:06.467332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342871153653918:10037] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:06.467340Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2034 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:06.468845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:06.470010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:06.470025Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:06.470026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:06.470043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:06.470047Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2033 [2:7559342870791820016:11019]
2025-10-09T21:51:06.470055Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:06.470068Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:06.470071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:06.470074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:06.470076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:06.470092Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:06.470097Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:06.470105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:06.470147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.470153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.470154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.470156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:06.470203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:06.470209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.470217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.470223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.470232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:06.470244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342870791820011:11019]
2025-10-09T21:51:06.470246Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342870791820011:11019]
2025-10-09T21:51:06.470248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342870791820011:11019]
2025-10-09T21:51:06.470256Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:06.470266Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:06.470270Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:06.470272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:06.470274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:06.470287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:06.470295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:06.470306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342871153653957:10037], now have 1 active actors on pipe
2025-10-09T21:51:06.470313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342871153653844:2432], now have 1 active actors on pipe
2025-10-09T21:51:06.470342Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342870791820011:11019]
2025-10-09T21:51:06.470353Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:06.470354Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2033
2025-10-09T21:51:06.470356Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2033
2025-10-09T21:51:06.470381Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:06.470447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:06.470457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:06.470484Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b8c76c4e-1c049665-dac5bdee-a978e56b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:06.470492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:06.470500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:06.470502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:06.470511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:06.470521Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:06.470534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:06.470735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:06.470757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:06.470782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:06.470876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2034 partition: 0 MaxSeqNo: 25 sessionId: src_id|b8c76c4e-1c049665-dac5bdee-a978e56b_0
2025-10-09T21:51:06.471148Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da57a446-dfca48de-8c5202ed-577ae48_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b8c76c4e-1c049665-dac5bdee-a978e56b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:06.473426Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b8c76c4e-1c049665-dac5bdee-a978e56b_0] Write session: try to update token
2025-10-09T21:51:06.473438Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b8c76c4e-1c049665-dac5bdee-a978e56b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:06.485215Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2034 sessionId: src_id|b8c76c4e-1c049665-dac5bdee-a978e56b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:06.486896Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:06.486912Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:06.486933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342870791820011:11019]
2025-10-09T21:51:06.486981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:06.487007Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:06.487167Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:06.487320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:06.487443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:06.487516Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:06.487561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:06.487729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:06.487772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:06.487784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:06.487802Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:06.487848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:06.487924Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:06.493987Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:06.493999Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:06.494000Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:06.494002Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:06.494008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342870791820011:11019]
2025-10-09T21:51:06.494011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342870791820011:11019]
2025-10-09T21:51:06.494013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342870791820011:11019]
2025-10-09T21:51:06.494064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:06.494070Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:06.494072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.494075Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:06.494075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:06.494077Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:06.494194Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:06.548248Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342871153653965:5250]
2025-10-09T21:51:06.548556Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 2033, pipe: [1:7559342871153653965:5250]
2025-10-09T21:51:06.630985Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:06.637435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:06.637454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:06.637597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:06.637647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:06.637701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:06.637757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:06.637803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:06.637811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:06.637868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:06.637919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:06.637970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:06.638038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:06.638080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:06.638085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:06.638138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:06.638184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:06.638233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:06.638286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:06.638327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:06.638334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:06.638419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:06.638497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:06.638561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:06.638613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:06.638649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:06.638655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:06.638709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:06.638757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:06.638813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:06.638863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:06.638911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:06.638916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:06.638970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:06.639018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:06.639075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:06.639125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:06.639175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:06.639180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:06.639229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:06.639277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:06.639334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:06.639383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:06.639430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:06.639436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:06.639484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:06.639530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:06.639587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:06.639637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:06.639686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:06.639690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:06.639740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:06.639788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:06.639844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:06.639891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:06.639926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:06.639931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:06.639981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:06.640031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:06.640081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:06.640135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:06.640170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:06.640175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:06.640213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342871153653965:5250], now have 1 active actors on pipe
2025-10-09T21:51:06.640444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:06.640461Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:06.640474Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342871153653965:5250]
2025-10-09T21:51:06.640489Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2034 reason: error at write: Disconnected sessionId: src_id|b8c76c4e-1c049665-dac5bdee-a978e56b_0
2025-10-09T21:51:06.640604Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:06.640670Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2034 sessionId: src_id|b8c76c4e-1c049665-dac5bdee-a978e56b_0 is DEAD
2025-10-09T21:51:06.640824Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8c76c4e-1c049665-dac5bdee-a978e56b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:06.640837Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8c76c4e-1c049665-dac5bdee-a978e56b_0] Write session will restart in 0.007735s
2025-10-09T21:51:06.640861Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8c76c4e-1c049665-dac5bdee-a978e56b_0] Write session: Do CDS request
2025-10-09T21:51:06.640867Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8c76c4e-1c049665-dac5bdee-a978e56b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:06.641042Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:06.649524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b8c76c4e-1c049665-dac5bdee-a978e56b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:06.649536Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:06.649549Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2035
2025-10-09T21:51:06.649713Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2035 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:06.649711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:06.649746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2035 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:06.649757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2035 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:06.649761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:06.650117Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2035 sessionId:  describe result for acl check
2025-10-09T21:51:06.650141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:06.650145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:06.650146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:06.650150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342871153654045:10032] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:06.650597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342871153654045:10032] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:06.651584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:06.651754Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:06.651765Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:06.651776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:06.651829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:06.651867Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342870791820130:11021]
2025-10-09T21:51:06.652273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:06.652500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:06.652542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:06.652761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:06.652808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.652816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.652817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.652818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.652831Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342871153654045:10032] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:06.652819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.652820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.652821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.652822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.652823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.652828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.652838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.652842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.652846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.652850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.652853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.652857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.652859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.652861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.652865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.652870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.652874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.652877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.652883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.652885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.652888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.652891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.652893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.652896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.652898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.652901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.652904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.652907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.652909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.652912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.652915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.652918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.652933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:06.658272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342871153654045:10032] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:06.658287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342871153654045:10032] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:06.658290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342871153654045:10032] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:06.658298Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2035 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:06.658703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:06.659841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:06.659852Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:06.659854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:06.659856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:06.659860Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2034 [2:7559342870791820130:11021]
2025-10-09T21:51:06.659871Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:06.659887Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:06.659890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:06.659893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:06.659895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:06.659909Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:06.659919Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:06.659926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:06.659979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.659983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.659984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.659987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:06.660036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:06.660042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.660049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.660055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.660064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:06.660077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342871153654084:10032], now have 1 active actors on pipe
2025-10-09T21:51:06.660083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342870791820125:11021]
2025-10-09T21:51:06.660084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342870791820125:11021]
2025-10-09T21:51:06.660085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342870791820125:11021]
2025-10-09T21:51:06.660090Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:06.660101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:06.660101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:06.660104Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:06.660106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:06.660112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:06.660114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:06.660118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342871153653968:2432], now have 1 active actors on pipe
2025-10-09T21:51:06.660133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342870791820125:11021]
2025-10-09T21:51:06.660136Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:06.660142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:06.660148Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2034
2025-10-09T21:51:06.660185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:06.660191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:06.660210Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2d1610bb-b3754685-7de9444a-a5198425_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:06.660215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:06.660218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:06.660219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:06.660226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:06.660241Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:06.660233Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2034
2025-10-09T21:51:06.660258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:06.660434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:06.660446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:06.660468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:06.660585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2035 partition: 0 MaxSeqNo: 25 sessionId: src_id|2d1610bb-b3754685-7de9444a-a5198425_0
2025-10-09T21:51:06.660877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b8c76c4e-1c049665-dac5bdee-a978e56b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2d1610bb-b3754685-7de9444a-a5198425_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:06.663217Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d1610bb-b3754685-7de9444a-a5198425_0] Write session: try to update token
2025-10-09T21:51:06.663233Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d1610bb-b3754685-7de9444a-a5198425_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:06.673408Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:06.673425Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:06.673427Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:06.673430Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:06.673443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342870791820125:11021]
2025-10-09T21:51:06.673447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342870791820125:11021]
2025-10-09T21:51:06.673450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342870791820125:11021]
2025-10-09T21:51:06.673492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:06.673493Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:06.673499Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:06.673499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.673502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:06.673503Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:06.673648Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:06.675990Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2035 sessionId: src_id|2d1610bb-b3754685-7de9444a-a5198425_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:06.678341Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:06.740692Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342871153654091:5250]
2025-10-09T21:51:06.740981Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 2034, pipe: [1:7559342871153654091:5250]
2025-10-09T21:51:06.814595Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:06.815070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:06.815088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:06.815133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342871153654091:5250], now have 1 active actors on pipe
2025-10-09T21:51:06.815270Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:06.815309Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:06.815357Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:06.815440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2035 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2d1610bb-b3754685-7de9444a-a5198425_0
2025-10-09T21:51:06.815615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2035 sessionId: src_id|2d1610bb-b3754685-7de9444a-a5198425_0 is DEAD
2025-10-09T21:51:06.815800Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d1610bb-b3754685-7de9444a-a5198425_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:06.815809Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d1610bb-b3754685-7de9444a-a5198425_0] Write session will restart in 0.009209s
2025-10-09T21:51:06.815826Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d1610bb-b3754685-7de9444a-a5198425_0] Write session: Do CDS request
2025-10-09T21:51:06.815831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d1610bb-b3754685-7de9444a-a5198425_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:06.815892Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342871153654091:5250]
2025-10-09T21:51:06.826513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d1610bb-b3754685-7de9444a-a5198425_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:06.826573Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:06.826588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2036
2025-10-09T21:51:06.826612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:06.826716Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2036 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:06.826778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2036 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:06.826789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2036 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:06.826792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:06.827245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2036 sessionId:  describe result for acl check
2025-10-09T21:51:06.827281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:06.827287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:06.827289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:06.827293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342871153654202:10014] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:06.827748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342871153654202:10014] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:06.828371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:06.828544Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:06.828554Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:06.828565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:06.828615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:06.828652Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342870791820289:11031]
2025-10-09T21:51:06.829206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:06.829300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342871153654202:10014] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:06.829504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:06.829545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:06.829827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:06.829873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.829879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.829880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.829882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.829883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.829884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.829885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.829886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.829887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.829891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.829901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.829904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.829907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.829910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.829914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.829918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.829922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.829926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.829931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.829939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:06.829945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.829950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:06.829953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.829958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:06.829961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.829967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:06.829972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.829976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:06.829981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.829985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:06.829990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.829995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.830000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.830010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.830014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.830019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.830061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:06.834976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342871153654202:10014] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:06.834990Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342871153654202:10014] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:06.834994Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342871153654202:10014] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:06.835002Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2036 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:06.836246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:06.837854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:06.837868Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:06.837871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:06.837874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:06.837881Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2035 [2:7559342870791820289:11031]
2025-10-09T21:51:06.837892Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:06.837907Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:06.837910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:06.837913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:06.837915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:06.837931Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:06.837937Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:06.837944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:06.837994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.838001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.838003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.838006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:06.838085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:06.838092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.838104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:06.838117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:06.838138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:06.838163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342871153654110:2432], now have 1 active actors on pipe
2025-10-09T21:51:06.838171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342871153654237:10014], now have 1 active actors on pipe
2025-10-09T21:51:06.838178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342870791820282:11031]
2025-10-09T21:51:06.838181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342870791820282:11031]
2025-10-09T21:51:06.838182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342870791820282:11031]
2025-10-09T21:51:06.838195Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:06.838209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:06.838197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2035
2025-10-09T21:51:06.838221Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:06.838202Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2035
2025-10-09T21:51:06.838231Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:06.838233Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:06.838237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:06.838241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:06.838281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:06.838290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:06.838302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342870791820282:11031]
2025-10-09T21:51:06.838315Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:06.838336Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:06.838345Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|32f8df60-d5282165-47dc05de-be75f5e2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:06.838353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:06.838362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:06.838364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:06.838375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:06.838384Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:06.838405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:06.838555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:06.838562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:06.838574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:06.838657Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2036 partition: 0 MaxSeqNo: 25 sessionId: src_id|32f8df60-d5282165-47dc05de-be75f5e2_0
2025-10-09T21:51:06.838955Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d1610bb-b3754685-7de9444a-a5198425_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|32f8df60-d5282165-47dc05de-be75f5e2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:06.841282Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32f8df60-d5282165-47dc05de-be75f5e2_0] Write session: try to update token
2025-10-09T21:51:06.841297Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32f8df60-d5282165-47dc05de-be75f5e2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:06.852492Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:06.852511Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:06.852514Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:06.852516Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:06.852527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342870791820282:11031]
2025-10-09T21:51:06.852532Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342870791820282:11031]
2025-10-09T21:51:06.852535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342870791820282:11031]
2025-10-09T21:51:06.852578Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:06.852580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:06.852583Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:06.852587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:06.852589Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:06.852590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:06.852734Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:06.853944Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2036 sessionId: src_id|32f8df60-d5282165-47dc05de-be75f5e2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:06.856303Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:06.916343Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342871153654246:5250]
2025-10-09T21:51:06.916605Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 2035, pipe: [1:7559342871153654246:5250]
2025-10-09T21:51:07.001811Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:07.002131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.002162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.002208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342871153654246:5250], now have 1 active actors on pipe
2025-10-09T21:51:07.002404Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342871153654246:5250]
2025-10-09T21:51:07.002425Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:07.002433Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:07.002436Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:07.002459Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:07.002488Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2036 reason: error at write: Disconnected sessionId: src_id|32f8df60-d5282165-47dc05de-be75f5e2_0
2025-10-09T21:51:07.002673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2036 sessionId: src_id|32f8df60-d5282165-47dc05de-be75f5e2_0 is DEAD
2025-10-09T21:51:07.002824Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32f8df60-d5282165-47dc05de-be75f5e2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:07.002842Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32f8df60-d5282165-47dc05de-be75f5e2_0] Write session will restart in 0.007046s
2025-10-09T21:51:07.002863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32f8df60-d5282165-47dc05de-be75f5e2_0] Write session: Do CDS request
2025-10-09T21:51:07.002868Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32f8df60-d5282165-47dc05de-be75f5e2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:07.011579Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32f8df60-d5282165-47dc05de-be75f5e2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:07.011675Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:07.011691Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2037
2025-10-09T21:51:07.011830Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2037 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:07.011863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2037 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:07.011869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2037 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:07.011872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:07.012302Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2037 sessionId:  describe result for acl check
2025-10-09T21:51:07.012341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:07.012345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:07.012347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:07.012352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342875448621623:10002] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:07.012928Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342875448621623:10002] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:07.013566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:07.015180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342875448621623:10002] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:07.015761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:07.015964Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:07.015976Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:07.015989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:07.016055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:07.016100Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342875086787701:11039]
2025-10-09T21:51:07.016533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:07.016865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:07.016928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:07.017273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:07.017339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342875448621623:10002] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:07.017357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.017353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342875448621623:10002] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:07.017363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.017364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.017357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342875448621623:10002] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:07.017366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.017366Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2037 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:07.017367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.017368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.017370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.017371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.017372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.017378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.017388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.017404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.017409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.017413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.017417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.017421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.017425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.017429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.017436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.017444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.017451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.017457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.017462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.017467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.017472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.017477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.017482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.017487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.017492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.017498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.017509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.017514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.017519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.017524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.017529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.017534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.017561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:07.024121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:07.025451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:07.025472Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:07.025475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:07.025481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:07.025489Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2036 [2:7559342875086787701:11039]
2025-10-09T21:51:07.025503Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:07.025525Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:07.025529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:07.025532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:07.025535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:07.025553Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:07.025566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:07.025573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:07.025635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.025642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.025644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.025647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:07.025719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:07.025726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.025740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.025758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.025778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:07.025806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448621547:2432], now have 1 active actors on pipe
2025-10-09T21:51:07.025815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448621660:10002], now have 1 active actors on pipe
2025-10-09T21:51:07.025821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342875086787695:11039]
2025-10-09T21:51:07.025824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342875086787695:11039]
2025-10-09T21:51:07.025825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342875086787695:11039]
2025-10-09T21:51:07.025836Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:07.025850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:07.025853Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:07.025856Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:07.025858Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:07.025850Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2036
2025-10-09T21:51:07.025856Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2036
2025-10-09T21:51:07.025868Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:07.025880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:07.025930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.025940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.025952Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342875086787695:11039]
2025-10-09T21:51:07.025965Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:07.025978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:07.025994Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|df300bb9-eda50cfd-3bc1565b-f7fa70c9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:07.026000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:07.026006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:07.026008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:07.026015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:07.026023Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:07.026061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:07.026174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.026183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.026200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:07.026278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2037 partition: 0 MaxSeqNo: 25 sessionId: src_id|df300bb9-eda50cfd-3bc1565b-f7fa70c9_0
2025-10-09T21:51:07.026653Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32f8df60-d5282165-47dc05de-be75f5e2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|df300bb9-eda50cfd-3bc1565b-f7fa70c9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:07.029206Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df300bb9-eda50cfd-3bc1565b-f7fa70c9_0] Write session: try to update token
2025-10-09T21:51:07.029224Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df300bb9-eda50cfd-3bc1565b-f7fa70c9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:07.040947Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:07.040964Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:07.040966Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:07.040968Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:07.040977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342875086787695:11039]
2025-10-09T21:51:07.040982Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342875086787695:11039]
2025-10-09T21:51:07.040988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342875086787695:11039]
2025-10-09T21:51:07.041026Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:07.041031Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:07.041031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:07.041034Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:07.041040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.041043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:07.041170Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:07.042634Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2037 sessionId: src_id|df300bb9-eda50cfd-3bc1565b-f7fa70c9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:07.044994Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:07.102692Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342875448621670:5250]
2025-10-09T21:51:07.102908Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 2036, pipe: [1:7559342875448621670:5250]
2025-10-09T21:51:07.187319Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:07.187592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.187605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.187643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448621670:5250], now have 1 active actors on pipe
2025-10-09T21:51:07.187797Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:07.187868Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342875448621670:5250]
2025-10-09T21:51:07.187874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:07.187884Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:07.187893Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:07.187937Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2037 reason: error at write: Disconnected sessionId: src_id|df300bb9-eda50cfd-3bc1565b-f7fa70c9_0
2025-10-09T21:51:07.188116Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2037 sessionId: src_id|df300bb9-eda50cfd-3bc1565b-f7fa70c9_0 is DEAD
2025-10-09T21:51:07.188240Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df300bb9-eda50cfd-3bc1565b-f7fa70c9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:07.188252Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df300bb9-eda50cfd-3bc1565b-f7fa70c9_0] Write session will restart in 0.009266s
2025-10-09T21:51:07.188280Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df300bb9-eda50cfd-3bc1565b-f7fa70c9_0] Write session: Do CDS request
2025-10-09T21:51:07.188286Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df300bb9-eda50cfd-3bc1565b-f7fa70c9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:07.197222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:07.198465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|df300bb9-eda50cfd-3bc1565b-f7fa70c9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:07.198528Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:07.198541Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2038
2025-10-09T21:51:07.198653Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2038 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:07.198677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2038 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:07.198691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:07.198685Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2038 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:07.198688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:07.198867Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:07.198876Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:07.198885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:07.198927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:07.198959Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342875086787817:11041]
2025-10-09T21:51:07.199121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2038 sessionId:  describe result for acl check
2025-10-09T21:51:07.199163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:07.199170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:07.199172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:07.199178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342875448621754:9997] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:07.199331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:07.199636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:07.199668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:07.199707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342875448621754:9997] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:07.199893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:07.199925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.199930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.199931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.199931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.199932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.199933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.199933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.199934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.199934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.199938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.199944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.199946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.199948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.199950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.199953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.199956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.199960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.199963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.199969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.199976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.199982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.199992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.200001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.200007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.200015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.200018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.200021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.200027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.200030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.200033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.200036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.200039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.200042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.200045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.200047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.200050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.200067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:07.201689Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342875448621754:9997] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:07.205779Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342875448621754:9997] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:07.205792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342875448621754:9997] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:07.205795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342875448621754:9997] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:07.205801Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2038 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:07.206282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:07.207416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:07.207431Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:07.207434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:07.207438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:07.207445Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2037 [2:7559342875086787817:11041]
2025-10-09T21:51:07.207457Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:07.207473Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:07.207479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:07.207482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:07.207485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:07.207501Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:07.207512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:07.207520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:07.207574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.207581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.207583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.207585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:07.207659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:07.207669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.207684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.207702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.207730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:07.207758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448621793:9997], now have 1 active actors on pipe
2025-10-09T21:51:07.207770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448621674:2432], now have 1 active actors on pipe
2025-10-09T21:51:07.207778Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342875086787812:11041]
2025-10-09T21:51:07.207785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342875086787812:11041]
2025-10-09T21:51:07.207788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342875086787812:11041]
2025-10-09T21:51:07.207791Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2037
2025-10-09T21:51:07.207804Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:07.207798Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2037
2025-10-09T21:51:07.207825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:07.207824Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:07.207827Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:07.207829Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:07.207841Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:07.207845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:07.207875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342875086787812:11041]
2025-10-09T21:51:07.207889Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:07.207891Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:07.207930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.207943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.207979Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e259032-7be9e679-43560bb1-b67ad19_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:07.207994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:07.207998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:07.208001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:07.208013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:07.208030Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:07.208054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:07.208245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.208259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.208279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:07.208355Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2038 partition: 0 MaxSeqNo: 25 sessionId: src_id|e259032-7be9e679-43560bb1-b67ad19_0
2025-10-09T21:51:07.208661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|df300bb9-eda50cfd-3bc1565b-f7fa70c9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e259032-7be9e679-43560bb1-b67ad19_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:07.211368Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e259032-7be9e679-43560bb1-b67ad19_0] Write session: try to update token
2025-10-09T21:51:07.211383Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e259032-7be9e679-43560bb1-b67ad19_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:07.219357Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:07.219370Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:07.219378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342875086787812:11041]
2025-10-09T21:51:07.219418Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:07.219425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:07.219620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:07.219754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:07.219889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:07.220028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:07.220208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:07.220248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:07.224086Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2038 sessionId: src_id|e259032-7be9e679-43560bb1-b67ad19_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:07.226374Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:07.226689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.226709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.226750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:07.226832Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:07.232261Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:07.232276Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:07.232279Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:07.232281Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:07.232291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342875086787812:11041]
2025-10-09T21:51:07.232296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342875086787812:11041]
2025-10-09T21:51:07.232300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342875086787812:11041]
2025-10-09T21:51:07.232341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:07.232343Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:07.232349Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:07.232353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.232354Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:07.232355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:07.232496Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:07.288041Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342875448621804:5250]
2025-10-09T21:51:07.288253Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 2037, pipe: [1:7559342875448621804:5250]
2025-10-09T21:51:07.378879Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:07.385856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.385878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.386019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:07.386095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:07.386163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:07.386220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:07.386261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:07.386274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:07.386337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:07.386396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:07.386462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:07.386528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:07.386572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:07.386577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:07.386635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:07.386697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:07.386755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:07.386810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:07.386848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:07.386854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:07.386913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:07.386978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:07.387035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:07.387091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:07.387128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:07.387134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:07.387196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:07.387259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:07.387320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:07.387373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:07.387412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:07.387420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:07.387482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:07.387538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:07.387595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:07.387652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:07.387715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:07.387724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:07.387780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:07.387833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:07.387889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:07.387944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:07.387996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:07.388001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:07.388054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:07.388106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:07.388162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:07.388254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:07.388306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:07.388311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:07.388364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:07.388416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:07.388470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:07.388529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:07.388581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:07.388586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:07.388639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:07.388690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:07.388744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:07.388799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:07.388836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:07.388841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:07.388884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448621804:5250], now have 1 active actors on pipe
2025-10-09T21:51:07.389133Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342875448621804:5250]
2025-10-09T21:51:07.389147Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:07.389151Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:07.389167Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:07.389181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2038 reason: error at write: Disconnected sessionId: src_id|e259032-7be9e679-43560bb1-b67ad19_0
2025-10-09T21:51:07.389369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2038 sessionId: src_id|e259032-7be9e679-43560bb1-b67ad19_0 is DEAD
2025-10-09T21:51:07.389507Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e259032-7be9e679-43560bb1-b67ad19_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:07.389515Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e259032-7be9e679-43560bb1-b67ad19_0] Write session will restart in 0.007955s
2025-10-09T21:51:07.389540Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e259032-7be9e679-43560bb1-b67ad19_0] Write session: Do CDS request
2025-10-09T21:51:07.389546Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e259032-7be9e679-43560bb1-b67ad19_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:07.389701Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:07.398432Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e259032-7be9e679-43560bb1-b67ad19_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:07.398543Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:07.398556Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2039
2025-10-09T21:51:07.398674Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2039 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:07.398715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2039 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:07.398728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2039 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:07.398732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:07.398963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:07.399169Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2039 sessionId:  describe result for acl check
2025-10-09T21:51:07.399201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:07.399207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:07.399209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:07.399214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342875448621892:9992] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:07.399749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342875448621892:9992] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:07.400634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:07.400806Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:07.400812Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:07.400821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:07.400861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:07.400890Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342875086787936:11043]
2025-10-09T21:51:07.401250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:07.401492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:07.401517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:07.401709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:07.401738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.401742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.401743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.401743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.401744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.401744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.401745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.401746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.401746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.401749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.401755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.401758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.401760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.401762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.401764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.401766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.401768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.401770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.401774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.401779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.401783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.401786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.401789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.401792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.401795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.401797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.401800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.401803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.401805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.401809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.401811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.401814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.401817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.401820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.401822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.401825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.401840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:07.402089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342875448621892:9992] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:07.407730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342875448621892:9992] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:07.407745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342875448621892:9992] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:07.407749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342875448621892:9992] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:07.407758Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2039 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:07.408053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:07.409043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:07.409057Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:07.409058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:07.409062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:07.409067Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2038 [2:7559342875086787936:11043]
2025-10-09T21:51:07.409076Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:07.409090Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:07.409091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:07.409094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:07.409095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:07.409109Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:07.409113Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:07.409118Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:07.409169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.409175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.409177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.409180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:07.409251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:07.409259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.409272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.409289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.409311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:07.409329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448621931:9992], now have 1 active actors on pipe
2025-10-09T21:51:07.409336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448621811:2432], now have 1 active actors on pipe
2025-10-09T21:51:07.409342Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342875086787930:11043]
2025-10-09T21:51:07.409344Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342875086787930:11043]
2025-10-09T21:51:07.409345Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342875086787930:11043]
2025-10-09T21:51:07.409355Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:07.409366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:07.409375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:07.409379Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:07.409379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:07.409383Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:07.409386Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:07.409364Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2038
2025-10-09T21:51:07.409379Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2038
2025-10-09T21:51:07.409414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342875086787930:11043]
2025-10-09T21:51:07.409422Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:07.409425Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:07.409434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.409438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.409476Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e3f6d283-b3a2c04c-fc83d658-d7f57ee2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:07.409488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:07.409497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:07.409501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:07.409511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:07.409525Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:07.409538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:07.409710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.409721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.409740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:07.409844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2039 partition: 0 MaxSeqNo: 25 sessionId: src_id|e3f6d283-b3a2c04c-fc83d658-d7f57ee2_0
2025-10-09T21:51:07.410167Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e259032-7be9e679-43560bb1-b67ad19_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e3f6d283-b3a2c04c-fc83d658-d7f57ee2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:07.412595Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3f6d283-b3a2c04c-fc83d658-d7f57ee2_0] Write session: try to update token
2025-10-09T21:51:07.412614Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3f6d283-b3a2c04c-fc83d658-d7f57ee2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:07.423337Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:07.423354Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:07.423356Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:07.423358Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:07.423366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342875086787930:11043]
2025-10-09T21:51:07.423371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342875086787930:11043]
2025-10-09T21:51:07.423375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342875086787930:11043]
2025-10-09T21:51:07.423420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:07.423427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.423429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:07.423437Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:07.423443Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:07.423446Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:07.423555Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:07.425740Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2039 sessionId: src_id|e3f6d283-b3a2c04c-fc83d658-d7f57ee2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:07.427983Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:07.489274Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342875448621939:5250]
2025-10-09T21:51:07.489549Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 2038, pipe: [1:7559342875448621939:5250]
2025-10-09T21:51:07.563848Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:07.564358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.564376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.564421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448621939:5250], now have 1 active actors on pipe
2025-10-09T21:51:07.564668Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:07.564684Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:07.564706Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:07.564718Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342875448621939:5250]
2025-10-09T21:51:07.564745Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:07.564773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2039 reason: error at write: Disconnected sessionId: src_id|e3f6d283-b3a2c04c-fc83d658-d7f57ee2_0
2025-10-09T21:51:07.564984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2039 sessionId: src_id|e3f6d283-b3a2c04c-fc83d658-d7f57ee2_0 is DEAD
2025-10-09T21:51:07.565151Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3f6d283-b3a2c04c-fc83d658-d7f57ee2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:07.565163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3f6d283-b3a2c04c-fc83d658-d7f57ee2_0] Write session will restart in 0.005894s
2025-10-09T21:51:07.565189Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3f6d283-b3a2c04c-fc83d658-d7f57ee2_0] Write session: Do CDS request
2025-10-09T21:51:07.565195Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3f6d283-b3a2c04c-fc83d658-d7f57ee2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:07.572459Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e3f6d283-b3a2c04c-fc83d658-d7f57ee2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:07.572551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:07.572563Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2040
2025-10-09T21:51:07.572700Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2040 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:07.572730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2040 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:07.572739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2040 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:07.572743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:07.573133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2040 sessionId:  describe result for acl check
2025-10-09T21:51:07.573163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:07.573170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:07.573172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:07.573177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342875448621999:9986] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:07.573701Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342875448621999:9986] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:07.574937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:07.575502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342875448621999:9986] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:07.576540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:07.576727Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:07.576735Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:07.576745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:07.576798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:07.576840Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342875086788055:11045]
2025-10-09T21:51:07.577382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:07.577667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:07.577712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:07.577854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342875448621999:9986] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:07.577864Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342875448621999:9986] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:07.577866Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342875448621999:9986] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:07.577871Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2040 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:07.578022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:07.578101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.578107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.578108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.578109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.578110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.578111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.578112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.578114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.578115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.578120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.578131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.578134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.578138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.578142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.578146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.578149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.578153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.578156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.578164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.578171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.578178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.578184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.578188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.578193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.578197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.578202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.578209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.578216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.578220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.578225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.578230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.578240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.578246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.578250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.578255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.578260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.578290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:07.584207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:07.585256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:07.585271Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:07.585273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:07.585277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:07.585283Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2039 [2:7559342875086788055:11045]
2025-10-09T21:51:07.585295Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:07.585313Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:07.585316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:07.585319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:07.585321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:07.585337Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:07.585345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:07.585352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:07.585415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.585423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.585425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.585428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:07.585598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:07.585609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.585621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.585633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.585652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:07.585639Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2039
2025-10-09T21:51:07.585645Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2039
2025-10-09T21:51:07.585676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448622060:9986], now have 1 active actors on pipe
2025-10-09T21:51:07.585683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448621940:2432], now have 1 active actors on pipe
2025-10-09T21:51:07.585690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342875086788049:11045]
2025-10-09T21:51:07.585693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342875086788049:11045]
2025-10-09T21:51:07.585695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342875086788049:11045]
2025-10-09T21:51:07.585715Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:07.585727Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:07.585730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:07.585732Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:07.585733Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:07.585748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:07.585752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:07.585772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.585777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.585805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342875086788049:11045]
2025-10-09T21:51:07.585814Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:07.585829Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9f5cfd54-451fe03c-f6fb05da-eeedc458_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:07.585834Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:07.585839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:07.585842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:07.585844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:07.585852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:07.585860Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:07.585887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:07.586068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.586074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.586084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:07.586165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2040 partition: 0 MaxSeqNo: 25 sessionId: src_id|9f5cfd54-451fe03c-f6fb05da-eeedc458_0
2025-10-09T21:51:07.586481Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e3f6d283-b3a2c04c-fc83d658-d7f57ee2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9f5cfd54-451fe03c-f6fb05da-eeedc458_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:07.588817Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f5cfd54-451fe03c-f6fb05da-eeedc458_0] Write session: try to update token
2025-10-09T21:51:07.588834Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f5cfd54-451fe03c-f6fb05da-eeedc458_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:07.599153Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:07.599168Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:07.599170Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:07.599171Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:07.599180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342875086788049:11045]
2025-10-09T21:51:07.599186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342875086788049:11045]
2025-10-09T21:51:07.599189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342875086788049:11045]
2025-10-09T21:51:07.599222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:07.599228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.599230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:07.599246Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:07.599252Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:07.599256Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:07.599358Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:07.601318Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2040 sessionId: src_id|9f5cfd54-451fe03c-f6fb05da-eeedc458_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:07.603619Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:07.664912Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342875448622070:5250]
2025-10-09T21:51:07.665232Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 2039, pipe: [1:7559342875448622070:5250]
2025-10-09T21:51:07.749142Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:07.749509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.749524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.749572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448622070:5250], now have 1 active actors on pipe
2025-10-09T21:51:07.749785Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:07.749818Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:07.749821Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:07.749843Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342875448622070:5250]
2025-10-09T21:51:07.749827Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:07.749884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2040 reason: error at write: Disconnected sessionId: src_id|9f5cfd54-451fe03c-f6fb05da-eeedc458_0
2025-10-09T21:51:07.750059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2040 sessionId: src_id|9f5cfd54-451fe03c-f6fb05da-eeedc458_0 is DEAD
2025-10-09T21:51:07.750246Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f5cfd54-451fe03c-f6fb05da-eeedc458_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:07.750258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f5cfd54-451fe03c-f6fb05da-eeedc458_0] Write session will restart in 0.005448s
2025-10-09T21:51:07.750285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f5cfd54-451fe03c-f6fb05da-eeedc458_0] Write session: Do CDS request
2025-10-09T21:51:07.750291Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f5cfd54-451fe03c-f6fb05da-eeedc458_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:07.756393Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f5cfd54-451fe03c-f6fb05da-eeedc458_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:07.756512Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:07.756525Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2041
2025-10-09T21:51:07.756676Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2041 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:07.756710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2041 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:07.756715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2041 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:07.756718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:07.757187Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2041 sessionId:  describe result for acl check
2025-10-09T21:51:07.757221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:07.757223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:07.757225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:07.757230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342875448622091:9980] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:07.757903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342875448622091:9980] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:07.760023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342875448622091:9980] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:07.760150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:07.761704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:07.761880Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:07.761890Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:07.761901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:07.761959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:07.761995Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342875086788175:11047]
2025-10-09T21:51:07.762216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342875448622091:9980] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:07.762226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342875448622091:9980] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:07.762231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342875448622091:9980] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:07.762236Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2041 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:07.762491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:07.762767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:07.762826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:07.763154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:07.763213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.763219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.763220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.763221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.763223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.763224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.763225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.763226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.763227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.763232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.763242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.763246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.763250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.763253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.763257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.763261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.763265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.763269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.763275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.763283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.763294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.763301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.763310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.763315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.763323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.763329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.763335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.763344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.763349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.763354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.763359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.763364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.763368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.763377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.763385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.763390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.763423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:07.770853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:07.771970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:07.771984Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:07.771987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:07.771991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:07.771997Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2040 [2:7559342875086788175:11047]
2025-10-09T21:51:07.772012Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:07.772029Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:07.772032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:07.772036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:07.772038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:07.772054Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:07.772063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:07.772070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:07.772124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.772132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.772133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.772137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:07.772204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:07.772212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.772223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.772235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.772250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:07.772274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448622072:2432], now have 1 active actors on pipe
2025-10-09T21:51:07.772282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448622190:9980], now have 1 active actors on pipe
2025-10-09T21:51:07.772289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342875086788169:11047]
2025-10-09T21:51:07.772292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342875086788169:11047]
2025-10-09T21:51:07.772294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342875086788169:11047]
2025-10-09T21:51:07.772311Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:07.772328Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:07.772331Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:07.772337Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:07.772339Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:07.772346Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:07.772352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:07.772323Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2040
2025-10-09T21:51:07.772323Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2040
2025-10-09T21:51:07.772378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342875086788169:11047]
2025-10-09T21:51:07.772389Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:07.772396Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:07.772410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.772419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.772466Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c518bcef-6e04c714-c7888fa4-e8814d6e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:07.772478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:07.772482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:07.772485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:07.772494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:07.772509Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:07.772526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:07.772716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.772725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.772740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:07.772834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2041 partition: 0 MaxSeqNo: 25 sessionId: src_id|c518bcef-6e04c714-c7888fa4-e8814d6e_0
2025-10-09T21:51:07.773181Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f5cfd54-451fe03c-f6fb05da-eeedc458_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c518bcef-6e04c714-c7888fa4-e8814d6e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:07.781976Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c518bcef-6e04c714-c7888fa4-e8814d6e_0] Write session: try to update token
2025-10-09T21:51:07.781993Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c518bcef-6e04c714-c7888fa4-e8814d6e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:07.789335Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:07.789349Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:07.789351Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:07.789352Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:07.789359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342875086788169:11047]
2025-10-09T21:51:07.789364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342875086788169:11047]
2025-10-09T21:51:07.789368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342875086788169:11047]
2025-10-09T21:51:07.789423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:07.789434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.789435Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:07.789437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:07.789441Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:07.789444Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:07.789601Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:07.794141Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2041 sessionId: src_id|c518bcef-6e04c714-c7888fa4-e8814d6e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:07.801856Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:07.850195Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342875448622215:5250]
2025-10-09T21:51:07.850430Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 2040, pipe: [1:7559342875448622215:5250]
2025-10-09T21:51:07.935715Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:07.936055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.936072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.936096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448622215:5250], now have 1 active actors on pipe
2025-10-09T21:51:07.936281Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:07.936308Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:07.936294Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342875448622215:5250]
2025-10-09T21:51:07.936311Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:07.936314Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:07.936371Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2041 reason: error at write: Disconnected sessionId: src_id|c518bcef-6e04c714-c7888fa4-e8814d6e_0
2025-10-09T21:51:07.936516Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2041 sessionId: src_id|c518bcef-6e04c714-c7888fa4-e8814d6e_0 is DEAD
2025-10-09T21:51:07.936685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c518bcef-6e04c714-c7888fa4-e8814d6e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:07.936697Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c518bcef-6e04c714-c7888fa4-e8814d6e_0] Write session will restart in 0.007177s
2025-10-09T21:51:07.936735Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c518bcef-6e04c714-c7888fa4-e8814d6e_0] Write session: Do CDS request
2025-10-09T21:51:07.936742Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c518bcef-6e04c714-c7888fa4-e8814d6e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:07.945455Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c518bcef-6e04c714-c7888fa4-e8814d6e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:07.945530Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:07.945543Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2042
2025-10-09T21:51:07.945657Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2042 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:07.945682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2042 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:07.945685Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2042 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:07.945687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:07.945946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:07.946007Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2042 sessionId:  describe result for acl check
2025-10-09T21:51:07.946043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:07.946047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:07.946049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:07.946052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342875448622312:9965] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:07.946485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342875448622312:9965] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:07.947663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:07.947823Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:07.947833Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:07.947843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:07.947893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:07.947929Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342875086788337:11063]
2025-10-09T21:51:07.948471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:07.948667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342875448622312:9965] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:07.948783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:07.948829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:07.949125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:07.949168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.949171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.949172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.949173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.949174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.949176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.949177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.949178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.949179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.949183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.949193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.949197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.949201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.949205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.949209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.949213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.949216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.949220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.949230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.949239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:07.949249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.949259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:07.949267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.949272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:07.949280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.949285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:07.949293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.949301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:07.949309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.949314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:07.949320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.949329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.949337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.949343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.949348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.949356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.949381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:07.954357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342875448622312:9965] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:07.954369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342875448622312:9965] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:07.954372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342875448622312:9965] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:07.954380Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2042 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:07.955381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:07.956294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:07.956307Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:07.956309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:07.956311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:07.956318Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2041 [2:7559342875086788337:11063]
2025-10-09T21:51:07.956333Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:07.956348Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:07.956354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:07.956357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:07.956360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:07.956373Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:07.956380Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:07.956386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:07.956424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.956428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.956430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.956431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:07.956481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:07.956487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.956498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:07.956513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:07.956526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:07.956543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448622229:2432], now have 1 active actors on pipe
2025-10-09T21:51:07.956548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342875448622352:9965], now have 1 active actors on pipe
2025-10-09T21:51:07.956553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342875086788332:11063]
2025-10-09T21:51:07.956555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342875086788332:11063]
2025-10-09T21:51:07.956556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342875086788332:11063]
2025-10-09T21:51:07.956564Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:07.956575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:07.956575Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:07.956577Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:07.956578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:07.956583Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:07.956585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:07.956585Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2041
2025-10-09T21:51:07.956606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342875086788332:11063]
2025-10-09T21:51:07.956585Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2041
2025-10-09T21:51:07.956611Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:07.956616Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:07.956631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.956634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.956660Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c947edf5-4ebe5fa8-a628b72e-f209762a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:07.956671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:07.956675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:07.956677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:07.956686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:07.956696Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:07.956711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:07.956823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:07.956829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:07.956842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:07.956913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2042 partition: 0 MaxSeqNo: 25 sessionId: src_id|c947edf5-4ebe5fa8-a628b72e-f209762a_0
2025-10-09T21:51:07.957190Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c518bcef-6e04c714-c7888fa4-e8814d6e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c947edf5-4ebe5fa8-a628b72e-f209762a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:07.959563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c947edf5-4ebe5fa8-a628b72e-f209762a_0] Write session: try to update token
2025-10-09T21:51:07.959581Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c947edf5-4ebe5fa8-a628b72e-f209762a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:07.970285Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:07.970303Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:07.970305Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:07.970307Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:07.970317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342875086788332:11063]
2025-10-09T21:51:07.970321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342875086788332:11063]
2025-10-09T21:51:07.970329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342875086788332:11063]
2025-10-09T21:51:07.970371Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:07.970375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:07.970378Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:07.970381Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:07.970382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:07.970384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:07.970512Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:07.971348Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2042 sessionId: src_id|c947edf5-4ebe5fa8-a628b72e-f209762a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:07.973551Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:08.036610Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342879743589655:5250]
2025-10-09T21:51:08.036901Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 2041, pipe: [1:7559342879743589655:5250]
2025-10-09T21:51:08.114671Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:08.114942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:08.114957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:08.115011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342879743589655:5250], now have 1 active actors on pipe
2025-10-09T21:51:08.115317Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:08.115330Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342879743589655:5250]
2025-10-09T21:51:08.115360Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:08.115363Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:08.115424Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2042 reason: error at write: Disconnected sessionId: src_id|c947edf5-4ebe5fa8-a628b72e-f209762a_0
2025-10-09T21:51:08.115622Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2042 sessionId: src_id|c947edf5-4ebe5fa8-a628b72e-f209762a_0 is DEAD
2025-10-09T21:51:08.115773Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c947edf5-4ebe5fa8-a628b72e-f209762a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:08.115786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c947edf5-4ebe5fa8-a628b72e-f209762a_0] Write session will restart in 0.008294s
2025-10-09T21:51:08.115810Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c947edf5-4ebe5fa8-a628b72e-f209762a_0] Write session: Do CDS request
2025-10-09T21:51:08.115821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c947edf5-4ebe5fa8-a628b72e-f209762a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:08.115825Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:08.125531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c947edf5-4ebe5fa8-a628b72e-f209762a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:08.125599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:08.125613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2043
2025-10-09T21:51:08.125733Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2043 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:08.125768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2043 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:08.125774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2043 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:08.125778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:08.126071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:08.126228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2043 sessionId:  describe result for acl check
2025-10-09T21:51:08.126265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:08.126272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:08.126274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:08.126280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342879743589745:9958] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:08.126863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342879743589745:9958] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:08.127731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:08.128015Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:08.128028Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:08.128042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:08.128107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:08.128147Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342879381755757:11065]
2025-10-09T21:51:08.128684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:08.129005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:08.129066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:08.129088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342879743589745:9958] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:08.129419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:08.129459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.129464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.129464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.129465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.129466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.129467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.129467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.129468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.129469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.129472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.129478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.129480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.129483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.129485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.129487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.129489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.129491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.129493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.129498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.129503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.129508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.129513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.129517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.129522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.129526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.129531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.129535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.129541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.129544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.129547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.129550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.129553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.129555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.129558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.129562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.129565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.129591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:08.134303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342879743589745:9958] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:08.134316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342879743589745:9958] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:08.134320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342879743589745:9958] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:08.134328Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2043 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:08.135659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:08.136631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:08.136642Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:08.136644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:08.136647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:08.136652Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2042 [2:7559342879381755757:11065]
2025-10-09T21:51:08.136660Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:08.136675Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:08.136678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:08.136681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:08.136683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:08.136699Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:08.136705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:08.136712Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:08.136755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.136761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.136762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.136764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:08.136812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:08.136818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.136826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.136832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.136842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:08.136860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342879743589660:2432], now have 1 active actors on pipe
2025-10-09T21:51:08.136863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342879743589784:9958], now have 1 active actors on pipe
2025-10-09T21:51:08.136868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342879381755750:11065]
2025-10-09T21:51:08.136870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342879381755750:11065]
2025-10-09T21:51:08.136871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342879381755750:11065]
2025-10-09T21:51:08.136879Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:08.136893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:08.136894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:08.136898Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:08.136900Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:08.136905Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:08.136907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:08.136929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342879381755750:11065]
2025-10-09T21:51:08.136937Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:08.136938Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:08.136928Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2042
2025-10-09T21:51:08.137040Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2042
2025-10-09T21:51:08.137166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:08.137179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:08.137208Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|160cb518-964d1f89-dcf6ffcc-7f9e178_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:08.137227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:08.137231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:08.137234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:08.137244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:08.137260Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:08.137274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:08.137593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:08.137602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:08.137618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:08.137698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2043 partition: 0 MaxSeqNo: 25 sessionId: src_id|160cb518-964d1f89-dcf6ffcc-7f9e178_0
2025-10-09T21:51:08.137955Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c947edf5-4ebe5fa8-a628b72e-f209762a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|160cb518-964d1f89-dcf6ffcc-7f9e178_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:08.140355Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|160cb518-964d1f89-dcf6ffcc-7f9e178_0] Write session: try to update token
2025-10-09T21:51:08.140371Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|160cb518-964d1f89-dcf6ffcc-7f9e178_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:08.150999Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:08.151020Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:08.151022Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:08.151025Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:08.151034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342879381755750:11065]
2025-10-09T21:51:08.151039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342879381755750:11065]
2025-10-09T21:51:08.151043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342879381755750:11065]
2025-10-09T21:51:08.151084Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:08.151090Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:08.151092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:08.151094Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:08.151098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.151101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:08.151230Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:08.152975Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2043 sessionId: src_id|160cb518-964d1f89-dcf6ffcc-7f9e178_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:08.155875Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:08.215308Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342879743589794:5250]
2025-10-09T21:51:08.215539Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 2042, pipe: [1:7559342879743589794:5250]
2025-10-09T21:51:08.297203Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:08.297511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:08.297532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:08.297575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342879743589794:5250], now have 1 active actors on pipe
2025-10-09T21:51:08.297801Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:08.297821Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:08.297824Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342879743589794:5250]
2025-10-09T21:51:08.297840Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:08.297904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2043 reason: status is not ok: tablet will be restarted right now sessionId: src_id|160cb518-964d1f89-dcf6ffcc-7f9e178_0
2025-10-09T21:51:08.298082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2043 sessionId: src_id|160cb518-964d1f89-dcf6ffcc-7f9e178_0 is DEAD
2025-10-09T21:51:08.298243Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|160cb518-964d1f89-dcf6ffcc-7f9e178_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:08.298255Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|160cb518-964d1f89-dcf6ffcc-7f9e178_0] Write session will restart in 0.008357s
2025-10-09T21:51:08.298287Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|160cb518-964d1f89-dcf6ffcc-7f9e178_0] Write session: Do CDS request
2025-10-09T21:51:08.298309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|160cb518-964d1f89-dcf6ffcc-7f9e178_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:08.307559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|160cb518-964d1f89-dcf6ffcc-7f9e178_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:08.307674Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:08.307693Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2044
2025-10-09T21:51:08.307824Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2044 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:08.307866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2044 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:08.307877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2044 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:08.307882Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:08.308272Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2044 sessionId:  describe result for acl check
2025-10-09T21:51:08.308303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:08.308308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:08.308310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:08.308314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342879743589885:9951] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:08.308787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342879743589885:9951] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:08.309269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:08.310926Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342879743589885:9951] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:08.311090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:08.311294Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:08.311305Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:08.311318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:08.311379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:08.311423Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342879381755879:11067]
2025-10-09T21:51:08.312023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:08.312346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:08.312397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:08.312719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:08.312769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.312774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.312776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.312777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.312778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.312779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.312780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.312782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.312783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.312788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.312799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.312803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.312807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.312810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.312815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.312818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.312823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.312827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.312834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.312842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.312852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.312857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.312862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.312869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.312874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.312879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.312889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.312904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.312917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.312923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.312928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.312933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.312938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.312943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.312948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.312953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.312984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:08.313192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342879743589885:9951] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:08.313207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342879743589885:9951] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:08.313211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342879743589885:9951] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:08.313220Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2044 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:08.319550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:08.320873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:08.320893Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:08.320896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:08.320901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:08.320909Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2043 [2:7559342879381755879:11067]
2025-10-09T21:51:08.320928Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:08.320948Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:08.320955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:08.320958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:08.320960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:08.320981Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:08.320987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:08.320997Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:08.321064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.321074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.321076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.321079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:08.321156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:08.321163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.321176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.321194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.321217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:08.321246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342879743589802:2432], now have 1 active actors on pipe
2025-10-09T21:51:08.321249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342879743589927:9951], now have 1 active actors on pipe
2025-10-09T21:51:08.321258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342879381755873:11067]
2025-10-09T21:51:08.321267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342879381755873:11067]
2025-10-09T21:51:08.321270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342879381755873:11067]
2025-10-09T21:51:08.321282Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:08.321298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:08.321318Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:08.321322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:08.321324Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:08.321297Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2043
2025-10-09T21:51:08.321328Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:08.321304Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2043
2025-10-09T21:51:08.321330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:08.321364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342879381755873:11067]
2025-10-09T21:51:08.321372Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:08.321377Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:08.321416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:08.321426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:08.321461Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|af8baa13-8c032542-42518a30-bc623376_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:08.321473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:08.321481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:08.321484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:08.321496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:08.321511Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:08.321537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:08.321693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:08.321702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:08.321718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:08.321836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2044 partition: 0 MaxSeqNo: 25 sessionId: src_id|af8baa13-8c032542-42518a30-bc623376_0
2025-10-09T21:51:08.322192Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|160cb518-964d1f89-dcf6ffcc-7f9e178_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|af8baa13-8c032542-42518a30-bc623376_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:08.324809Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af8baa13-8c032542-42518a30-bc623376_0] Write session: try to update token
2025-10-09T21:51:08.324829Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af8baa13-8c032542-42518a30-bc623376_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:08.336050Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:08.336065Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:08.336067Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:08.336069Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:08.336078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342879381755873:11067]
2025-10-09T21:51:08.336082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342879381755873:11067]
2025-10-09T21:51:08.336086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342879381755873:11067]
2025-10-09T21:51:08.336135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:08.336139Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:08.336143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.336144Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:08.336146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:08.336147Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:08.336282Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:08.338418Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2044 sessionId: src_id|af8baa13-8c032542-42518a30-bc623376_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:08.340798Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:08.398089Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342879743589934:5250]
2025-10-09T21:51:08.398362Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 2043, pipe: [1:7559342879743589934:5250]
2025-10-09T21:51:08.490519Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:08.490908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:08.490923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:08.490961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342879743589934:5250], now have 1 active actors on pipe
2025-10-09T21:51:08.491201Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:08.491196Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342879743589934:5250]
2025-10-09T21:51:08.491202Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:08.491230Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:08.491245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:08.491306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2044 reason: error at write: Disconnected sessionId: src_id|af8baa13-8c032542-42518a30-bc623376_0
2025-10-09T21:51:08.491490Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2044 sessionId: src_id|af8baa13-8c032542-42518a30-bc623376_0 is DEAD
2025-10-09T21:51:08.491662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af8baa13-8c032542-42518a30-bc623376_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:08.491670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af8baa13-8c032542-42518a30-bc623376_0] Write session will restart in 0.007502s
2025-10-09T21:51:08.491689Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af8baa13-8c032542-42518a30-bc623376_0] Write session: Do CDS request
2025-10-09T21:51:08.491694Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af8baa13-8c032542-42518a30-bc623376_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:08.500481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af8baa13-8c032542-42518a30-bc623376_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:08.500549Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:08.500562Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2045
2025-10-09T21:51:08.500688Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2045 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:08.500718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2045 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:08.500737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2045 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:08.500745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:08.501070Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2045 sessionId:  describe result for acl check
2025-10-09T21:51:08.501096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:08.501098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:08.501099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:08.501103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342879743590021:9937] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:08.501552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342879743590021:9937] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:08.502062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:08.503478Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342879743590021:9937] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:08.503670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:08.503851Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:08.503860Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:08.503872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:08.503920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:08.503959Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342879381756004:11069]
2025-10-09T21:51:08.504439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:08.504695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:08.504733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:08.504994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:08.505040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.505045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.505046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.505047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.505048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.505049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.505050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.505051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.505052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.505056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.505064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.505068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.505071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.505074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.505077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.505080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.505083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.505086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.505091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.505098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.505103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.505107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.505112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.505116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.505120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.505123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.505127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.505131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.505135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.505139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.505143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.505147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.505163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.505170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.505174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.505178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.505202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:08.510154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342879743590021:9937] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:08.510165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342879743590021:9937] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:08.510168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342879743590021:9937] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:08.510175Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2045 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:08.511547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:08.512696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:08.512707Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:08.512709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:08.512713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:08.512719Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2044 [2:7559342879381756004:11069]
2025-10-09T21:51:08.512735Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:08.512758Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:08.512761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:08.512764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:08.512766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:08.512781Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:08.512786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:08.512792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:08.512843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.512851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.512853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.512855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:08.512914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:08.512920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.512929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.512941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.512958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:08.512980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342879743590063:9937], now have 1 active actors on pipe
2025-10-09T21:51:08.512987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342879743589937:2432], now have 1 active actors on pipe
2025-10-09T21:51:08.512994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342879381755998:11069]
2025-10-09T21:51:08.512995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342879381755998:11069]
2025-10-09T21:51:08.512997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342879381755998:11069]
2025-10-09T21:51:08.513009Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:08.513022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:08.513024Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:08.513027Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:08.513029Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:08.513038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:08.513041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:08.513062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342879381755998:11069]
2025-10-09T21:51:08.513049Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2044
2025-10-09T21:51:08.513068Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:08.513076Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:08.513054Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2044
2025-10-09T21:51:08.513095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:08.513103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:08.513136Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|12a21b94-3230666a-614b5bec-b72bf0fb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:08.513147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:08.513150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:08.513152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:08.513162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:08.513174Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:08.513191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:08.513321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:08.513327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:08.513340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:08.513424Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2045 partition: 0 MaxSeqNo: 25 sessionId: src_id|12a21b94-3230666a-614b5bec-b72bf0fb_0
2025-10-09T21:51:08.513707Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af8baa13-8c032542-42518a30-bc623376_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|12a21b94-3230666a-614b5bec-b72bf0fb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:08.515938Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12a21b94-3230666a-614b5bec-b72bf0fb_0] Write session: try to update token
2025-10-09T21:51:08.515954Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12a21b94-3230666a-614b5bec-b72bf0fb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:08.526368Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:08.526386Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:08.526389Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:08.526391Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:08.526399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342879381755998:11069]
2025-10-09T21:51:08.526403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342879381755998:11069]
2025-10-09T21:51:08.526405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342879381755998:11069]
2025-10-09T21:51:08.526440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:08.526446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.526447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:08.526449Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:08.526456Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:08.526460Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:08.526563Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:08.527477Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2045 sessionId: src_id|12a21b94-3230666a-614b5bec-b72bf0fb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:08.529867Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:08.591607Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342879743590070:5250]
2025-10-09T21:51:08.591871Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 2044, pipe: [1:7559342879743590070:5250]
2025-10-09T21:51:08.670464Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:08.670723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:08.670735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:08.670770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342879743590070:5250], now have 1 active actors on pipe
2025-10-09T21:51:08.670929Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:08.670956Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:08.670969Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:08.670979Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342879743590070:5250]
2025-10-09T21:51:08.671001Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:08.671036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2045 reason: error at write: Disconnected sessionId: src_id|12a21b94-3230666a-614b5bec-b72bf0fb_0
2025-10-09T21:51:08.671206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2045 sessionId: src_id|12a21b94-3230666a-614b5bec-b72bf0fb_0 is DEAD
2025-10-09T21:51:08.671359Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12a21b94-3230666a-614b5bec-b72bf0fb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:08.671371Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12a21b94-3230666a-614b5bec-b72bf0fb_0] Write session will restart in 0.009150s
2025-10-09T21:51:08.671395Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12a21b94-3230666a-614b5bec-b72bf0fb_0] Write session: Do CDS request
2025-10-09T21:51:08.671401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12a21b94-3230666a-614b5bec-b72bf0fb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:08.680589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:08.681457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12a21b94-3230666a-614b5bec-b72bf0fb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:08.681537Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:08.681550Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2046
2025-10-09T21:51:08.681668Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2046 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:08.681702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2046 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:08.681706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2046 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:08.681709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:08.682142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2046 sessionId:  describe result for acl check
2025-10-09T21:51:08.682171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:08.682177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:08.682179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:08.682184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342879743590161:9938] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:08.682430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:08.682610Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:08.682619Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:08.682630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:08.682686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:08.682721Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342879381756128:11071]
2025-10-09T21:51:08.682729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342879743590161:9938] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:08.683303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:08.683666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:08.683706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:08.684016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:08.684065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.684072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.684073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.684074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.684075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.684076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.684077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.684079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.684080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.684084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.684093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.684098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.684101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.684105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.684109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.684113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.684117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.684120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.684126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.684133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.684139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.684145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.684154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.684160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.684166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.684171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.684175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.684180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.684189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.684194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.684203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.684207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.684215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.684220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.684230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.684235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.684267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:08.684540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342879743590161:9938] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:08.689675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342879743590161:9938] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:08.689688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342879743590161:9938] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:08.689692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342879743590161:9938] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:08.689699Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2046 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:08.690059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:08.691126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:08.691140Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:08.691143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:08.691146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:08.691152Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2045 [2:7559342879381756128:11071]
2025-10-09T21:51:08.691166Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:08.691181Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:08.691188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:08.691191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:08.691195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:08.691209Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:08.691219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:08.691230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:08.691279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.691286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.691288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.691291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:08.691362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:08.691368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.691380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.691395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.691416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:08.691438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342879743590073:2432], now have 1 active actors on pipe
2025-10-09T21:51:08.691446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342879743590200:9938], now have 1 active actors on pipe
2025-10-09T21:51:08.691452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342879381756123:11071]
2025-10-09T21:51:08.691454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342879381756123:11071]
2025-10-09T21:51:08.691456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342879381756123:11071]
2025-10-09T21:51:08.691470Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:08.691469Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2045
2025-10-09T21:51:08.691480Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:08.691483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:08.691477Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2045
2025-10-09T21:51:08.691487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:08.691492Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:08.691501Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:08.691504Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:08.691527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342879381756123:11071]
2025-10-09T21:51:08.691533Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:08.691543Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:08.691548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:08.691552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:08.691583Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ce3bd3ee-7284a07b-fe10c076-1228cd25_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:08.691594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:08.691597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:08.691600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:08.691609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:08.691617Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:08.691634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:08.691775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:08.691788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:08.691807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:08.691905Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2046 partition: 0 MaxSeqNo: 25 sessionId: src_id|ce3bd3ee-7284a07b-fe10c076-1228cd25_0
2025-10-09T21:51:08.692156Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12a21b94-3230666a-614b5bec-b72bf0fb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ce3bd3ee-7284a07b-fe10c076-1228cd25_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:08.694453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ce3bd3ee-7284a07b-fe10c076-1228cd25_0] Write session: try to update token
2025-10-09T21:51:08.694469Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ce3bd3ee-7284a07b-fe10c076-1228cd25_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:08.704937Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:08.704954Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:08.704957Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:08.704959Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:08.704969Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342879381756123:11071]
2025-10-09T21:51:08.704974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342879381756123:11071]
2025-10-09T21:51:08.704978Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342879381756123:11071]
2025-10-09T21:51:08.705023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:08.705025Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:08.705030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.705032Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:08.705033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:08.705035Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:08.705160Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:08.706828Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2046 sessionId: src_id|ce3bd3ee-7284a07b-fe10c076-1228cd25_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:08.709001Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:08.771307Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342879743590208:5250]
2025-10-09T21:51:08.771589Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 2045, pipe: [1:7559342879743590208:5250]
2025-10-09T21:51:08.841403Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: CLIENT_DEADLINE_EXCEEDED. Description: [ { <main>: Error: GRpc error: (4): Deadline Exceeded } { <main>: Error: Grpc error response on endpoint localhost:14704 } ]
2025-10-09T21:51:08.850073Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:08.850326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:08.850339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:08.850377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342879743590208:5250], now have 1 active actors on pipe
2025-10-09T21:51:08.850555Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:08.850616Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:08.850678Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:08.850681Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:08.850719Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342879743590208:5250]
2025-10-09T21:51:08.850740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2046 reason: error at write: Disconnected sessionId: src_id|ce3bd3ee-7284a07b-fe10c076-1228cd25_0
2025-10-09T21:51:08.850880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2046 sessionId: src_id|ce3bd3ee-7284a07b-fe10c076-1228cd25_0 is DEAD
2025-10-09T21:51:08.851008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce3bd3ee-7284a07b-fe10c076-1228cd25_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:08.851018Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce3bd3ee-7284a07b-fe10c076-1228cd25_0] Write session will restart in 0.009913s
2025-10-09T21:51:08.851036Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce3bd3ee-7284a07b-fe10c076-1228cd25_0] Write session: Do CDS request
2025-10-09T21:51:08.851041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce3bd3ee-7284a07b-fe10c076-1228cd25_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:08.860127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:08.861919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:08.862154Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:08.862162Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:08.862172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:08.862214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:08.862254Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342879381756258:11075]
2025-10-09T21:51:08.862511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ce3bd3ee-7284a07b-fe10c076-1228cd25_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:08.862566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:08.862577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2047
2025-10-09T21:51:08.862716Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2047 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:08.862742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2047 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:08.862746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2047 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:08.862749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:08.862736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:08.863034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:08.863099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:08.863133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2047 sessionId:  describe result for acl check
2025-10-09T21:51:08.863167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:08.863175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:08.863177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:08.863182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342879743590302:9934] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:08.863422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:08.863472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.863478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.863479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.863480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.863481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.863482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.863483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.863485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.863486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.863491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.863501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.863504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.863508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.863512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.863516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.863520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.863523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.863527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.863531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.863537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:08.863541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.863545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:08.863549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.863554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:08.863558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.863562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:08.863566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.863569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:08.863573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.863578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:08.863582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.863587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.863592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.863597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.863601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.863604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.863621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:08.863642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342879743590302:9934] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:08.865330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342879743590302:9934] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:08.869104Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342879743590302:9934] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:08.869116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342879743590302:9934] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:08.869120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342879743590302:9934] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:08.869128Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2047 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:08.869509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:08.870640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:08.870657Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:08.870659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:08.870663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:08.870669Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2046 [2:7559342879381756258:11075]
2025-10-09T21:51:08.870681Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:08.870698Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:08.870706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:08.870711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:08.870713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:08.870728Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:08.870733Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:08.870739Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:08.870791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.870799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.870801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.870804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:08.870865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:08.870876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.870887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:08.870899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:08.870914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:08.870940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342879743590337:9934], now have 1 active actors on pipe
2025-10-09T21:51:08.870944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342879743590209:2432], now have 1 active actors on pipe
2025-10-09T21:51:08.870951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342879381756253:11075]
2025-10-09T21:51:08.870953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342879381756253:11075]
2025-10-09T21:51:08.870955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342879381756253:11075]
2025-10-09T21:51:08.870950Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2046
2025-10-09T21:51:08.870967Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:08.870984Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:08.870979Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2046
2025-10-09T21:51:08.870987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:08.870987Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:08.870989Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:08.871001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:08.871005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:08.871043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:08.871053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:08.871068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342879381756253:11075]
2025-10-09T21:51:08.871078Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:08.871083Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:08.871099Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a08e91b8-664f474b-1f4c3117-977d1a91_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:08.871110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:08.871114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:08.871117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:08.871129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:08.871142Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:08.871168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:08.871258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:08.871265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:08.871277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:08.871353Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2047 partition: 0 MaxSeqNo: 25 sessionId: src_id|a08e91b8-664f474b-1f4c3117-977d1a91_0
2025-10-09T21:51:08.871641Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce3bd3ee-7284a07b-fe10c076-1228cd25_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a08e91b8-664f474b-1f4c3117-977d1a91_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:08.874117Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a08e91b8-664f474b-1f4c3117-977d1a91_0] Write session: try to update token
2025-10-09T21:51:08.874135Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a08e91b8-664f474b-1f4c3117-977d1a91_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:08.884765Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:08.884781Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:08.884782Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:08.884784Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:08.884790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342879381756253:11075]
2025-10-09T21:51:08.884793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342879381756253:11075]
2025-10-09T21:51:08.884795Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342879381756253:11075]
2025-10-09T21:51:08.884830Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:08.884835Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:08.884837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:08.884838Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:08.884843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:08.884846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:08.884978Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:08.886794Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2047 sessionId: src_id|a08e91b8-664f474b-1f4c3117-977d1a91_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:08.889070Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:08.951108Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342879743590370:5250]
2025-10-09T21:51:08.951345Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 2046, pipe: [1:7559342879743590370:5250]
2025-10-09T21:51:09.026953Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:09.027294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.027308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.027346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342879743590370:5250], now have 1 active actors on pipe
2025-10-09T21:51:09.027514Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:09.027588Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:09.027590Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342879743590370:5250]
2025-10-09T21:51:09.027603Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:09.027618Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:09.027668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2047 reason: error at write: Disconnected sessionId: src_id|a08e91b8-664f474b-1f4c3117-977d1a91_0
2025-10-09T21:51:09.027847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2047 sessionId: src_id|a08e91b8-664f474b-1f4c3117-977d1a91_0 is DEAD
2025-10-09T21:51:09.027978Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a08e91b8-664f474b-1f4c3117-977d1a91_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:09.028003Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a08e91b8-664f474b-1f4c3117-977d1a91_0] Write session will restart in 0.006686s
2025-10-09T21:51:09.028029Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a08e91b8-664f474b-1f4c3117-977d1a91_0] Write session: Do CDS request
2025-10-09T21:51:09.028036Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a08e91b8-664f474b-1f4c3117-977d1a91_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:09.035471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a08e91b8-664f474b-1f4c3117-977d1a91_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:09.035615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:09.035629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2048
2025-10-09T21:51:09.035743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2048 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:09.035776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2048 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:09.035787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2048 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:09.035791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:09.036173Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2048 sessionId:  describe result for acl check
2025-10-09T21:51:09.036205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:09.036209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:09.036212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:09.036216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342884038557755:9919] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:09.036674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342884038557755:9919] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:09.037180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:09.039026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:09.039212Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:09.039222Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:09.039234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:09.039285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:09.039325Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342883676723720:11089]
2025-10-09T21:51:09.039320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342884038557755:9919] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:09.039835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:09.040104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:09.040133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:09.040430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:09.040484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.040490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.040491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.040492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.040493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.040494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.040495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.040497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.040498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.040503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.040511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.040513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.040515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.040518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.040520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.040522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.040524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.040526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.040531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.040537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.040541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.040545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.040547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.040550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.040552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.040556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.040560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.040563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.040567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.040569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.040572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.040575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.040577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.040580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.040583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.040585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.040601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:09.045433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342884038557755:9919] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:09.045446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342884038557755:9919] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:09.045449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342884038557755:9919] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:09.045457Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2048 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:09.046839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:09.047976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:09.047990Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:09.047993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:09.047997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:09.048003Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2047 [2:7559342883676723720:11089]
2025-10-09T21:51:09.048016Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:09.048033Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:09.048039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:09.048047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:09.048050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:09.048066Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:09.048076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:09.048082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:09.048136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.048143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.048144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.048147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:09.048212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:09.048219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.048226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.048233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.048242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:09.048259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342883676723714:11089]
2025-10-09T21:51:09.048265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342883676723714:11089]
2025-10-09T21:51:09.048267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342883676723714:11089]
2025-10-09T21:51:09.048275Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:09.048285Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:09.048290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:09.048291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:09.048292Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:09.048303Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:09.048306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:09.048317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342884038557667:2432], now have 1 active actors on pipe
2025-10-09T21:51:09.048326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342884038557797:9919], now have 1 active actors on pipe
2025-10-09T21:51:09.048350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342883676723714:11089]
2025-10-09T21:51:09.048364Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:09.048384Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:09.048386Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2047
2025-10-09T21:51:09.048400Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2047
2025-10-09T21:51:09.048456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.048465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.048493Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a84a034-a3d6ee22-fab38f89-ba4b7b10_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:09.048505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:09.048509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:09.048516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:09.048525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:09.048539Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:09.048556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:09.048711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.048719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.048732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:09.048825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2048 partition: 0 MaxSeqNo: 25 sessionId: src_id|a84a034-a3d6ee22-fab38f89-ba4b7b10_0
2025-10-09T21:51:09.049199Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a08e91b8-664f474b-1f4c3117-977d1a91_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a84a034-a3d6ee22-fab38f89-ba4b7b10_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:09.051566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a84a034-a3d6ee22-fab38f89-ba4b7b10_0] Write session: try to update token
2025-10-09T21:51:09.051582Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a84a034-a3d6ee22-fab38f89-ba4b7b10_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:09.062250Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:09.062269Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:09.062274Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:09.062275Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:09.062285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342883676723714:11089]
2025-10-09T21:51:09.062292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342883676723714:11089]
2025-10-09T21:51:09.062295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342883676723714:11089]
2025-10-09T21:51:09.062341Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:09.062343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:09.062352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.062353Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:09.062355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:09.062357Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:09.062487Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:09.063712Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2048 sessionId: src_id|a84a034-a3d6ee22-fab38f89-ba4b7b10_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:09.066105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:09.127684Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342884038557805:5250]
2025-10-09T21:51:09.127908Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 2047, pipe: [1:7559342884038557805:5250]
2025-10-09T21:51:09.209365Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:09.209645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.209658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.209697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342884038557805:5250], now have 1 active actors on pipe
2025-10-09T21:51:09.209867Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:09.209935Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:09.209947Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342884038557805:5250]
2025-10-09T21:51:09.209974Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:09.210075Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2048 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a84a034-a3d6ee22-fab38f89-ba4b7b10_0
2025-10-09T21:51:09.210235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2048 sessionId: src_id|a84a034-a3d6ee22-fab38f89-ba4b7b10_0 is DEAD
2025-10-09T21:51:09.210351Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a84a034-a3d6ee22-fab38f89-ba4b7b10_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:09.210360Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a84a034-a3d6ee22-fab38f89-ba4b7b10_0] Write session will restart in 0.006120s
2025-10-09T21:51:09.210380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a84a034-a3d6ee22-fab38f89-ba4b7b10_0] Write session: Do CDS request
2025-10-09T21:51:09.210385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a84a034-a3d6ee22-fab38f89-ba4b7b10_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:09.217489Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a84a034-a3d6ee22-fab38f89-ba4b7b10_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:09.217535Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:09.217546Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2049
2025-10-09T21:51:09.217738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2049 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:09.217780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2049 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:09.217792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2049 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:09.217798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:09.218248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2049 sessionId:  describe result for acl check
2025-10-09T21:51:09.218288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:09.218295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:09.218296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:09.218302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342884038557834:9908] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:09.218856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342884038557834:9908] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:09.220958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342884038557834:9908] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:09.221844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:09.223360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:09.223487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342884038557834:9908] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:09.223502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342884038557834:9908] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:09.223507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342884038557834:9908] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:09.223514Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2049 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:09.223535Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:09.223547Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:09.223559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:09.223614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:09.223653Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342883676723848:11091]
2025-10-09T21:51:09.224221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:09.224477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:09.224521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:09.224828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:09.224882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.224888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.224889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.224890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.224891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.224892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.224893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.224894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.224895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.224900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.224910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.224914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.224918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.224922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.224926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.224930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.224934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.224937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.224944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.224951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.224958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.224963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.224967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.224972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.224977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.224982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.224986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.224991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.224999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.225003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.225008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.225016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.225020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.225025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.225030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.225035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.225066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:09.231308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:09.232421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:09.232435Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:09.232436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:09.232440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:09.232445Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2048 [2:7559342883676723848:11091]
2025-10-09T21:51:09.232454Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:09.232468Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:09.232470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:09.232472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:09.232473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:09.232486Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:09.232491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:09.232496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:09.232542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.232547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.232548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.232550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:09.232604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:09.232611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.232622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.232633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.232652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:09.232672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342884038557936:9908], now have 1 active actors on pipe
2025-10-09T21:51:09.232678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342884038557809:2432], now have 1 active actors on pipe
2025-10-09T21:51:09.232683Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342883676723843:11091]
2025-10-09T21:51:09.232685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342883676723843:11091]
2025-10-09T21:51:09.232687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342883676723843:11091]
2025-10-09T21:51:09.232700Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:09.232719Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:09.232724Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:09.232728Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:09.232731Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:09.232741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:09.232744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:09.232772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342883676723843:11091]
2025-10-09T21:51:09.232780Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:09.232746Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2048
2025-10-09T21:51:09.232783Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2048
2025-10-09T21:51:09.232791Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:09.232825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.232835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.232865Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|15335c26-afe011f6-1fb10385-511caf5a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:09.232879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:09.232883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:09.232886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:09.232895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:09.232913Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:09.232929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:09.233082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.233090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.233105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:09.233219Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2049 partition: 0 MaxSeqNo: 25 sessionId: src_id|15335c26-afe011f6-1fb10385-511caf5a_0
2025-10-09T21:51:09.233551Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a84a034-a3d6ee22-fab38f89-ba4b7b10_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|15335c26-afe011f6-1fb10385-511caf5a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:09.236004Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15335c26-afe011f6-1fb10385-511caf5a_0] Write session: try to update token
2025-10-09T21:51:09.236023Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15335c26-afe011f6-1fb10385-511caf5a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:09.248174Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2049 sessionId: src_id|15335c26-afe011f6-1fb10385-511caf5a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:09.250571Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:09.250853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.250877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.250897Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:09.250899Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:09.250905Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342883676723843:11091]
2025-10-09T21:51:09.250938Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:09.250956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:09.251108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:09.251237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:09.251364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:09.251489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:09.251659Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:09.251694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:09.251714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:09.251798Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:09.259199Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:09.259213Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:09.259214Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:09.259218Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:09.259227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342883676723843:11091]
2025-10-09T21:51:09.259229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342883676723843:11091]
2025-10-09T21:51:09.259232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342883676723843:11091]
2025-10-09T21:51:09.259264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:09.259272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.259274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:09.259285Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:09.259291Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:09.259295Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:09.259392Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:09.309933Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342884038557953:5250]
2025-10-09T21:51:09.310180Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 2048, pipe: [1:7559342884038557953:5250]
2025-10-09T21:51:09.404325Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:09.410897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.410920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.411070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.411125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.411184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.411236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.411433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.411441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.411498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.411553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.411604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.411680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.411719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.411724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.411777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.411828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.411879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.411931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.411970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.411975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.412028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.412240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.412298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.412350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.412387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.412393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.412456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.412515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.412572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.412626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.412664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.412669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.412725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.412774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.412823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.412874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.412915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.412920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.412973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.413020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.413083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.413134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.413171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.413179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.413228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.413275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.413323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.413376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.413432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.413439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.413500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.413552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.413602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.413659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.413753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.413759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.413814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.413862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.413912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.413964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.413998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.414005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.414064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342884038557953:5250], now have 1 active actors on pipe
2025-10-09T21:51:09.414312Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342884038557953:5250]
2025-10-09T21:51:09.414320Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:09.414325Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:09.414356Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2049 reason: error at write: Disconnected sessionId: src_id|15335c26-afe011f6-1fb10385-511caf5a_0
2025-10-09T21:51:09.414359Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:09.414437Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:09.414526Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2049 sessionId: src_id|15335c26-afe011f6-1fb10385-511caf5a_0 is DEAD
2025-10-09T21:51:09.414662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15335c26-afe011f6-1fb10385-511caf5a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:09.414672Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15335c26-afe011f6-1fb10385-511caf5a_0] Write session will restart in 0.005641s
2025-10-09T21:51:09.414690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15335c26-afe011f6-1fb10385-511caf5a_0] Write session: Do CDS request
2025-10-09T21:51:09.414694Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15335c26-afe011f6-1fb10385-511caf5a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:09.421426Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|15335c26-afe011f6-1fb10385-511caf5a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:09.421540Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:09.421552Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2050
2025-10-09T21:51:09.421683Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2050 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:09.421713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2050 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:09.421723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2050 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:09.421740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:09.422196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2050 sessionId:  describe result for acl check
2025-10-09T21:51:09.422228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:09.422234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:09.422236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:09.422241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342884038557994:9905] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:09.422785Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342884038557994:9905] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:09.424693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:09.425140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342884038557994:9905] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:09.426467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:09.426662Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:09.426670Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:09.426681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:09.426733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:09.426769Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342883676723977:11093]
2025-10-09T21:51:09.427306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:09.427619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:09.427663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:09.427713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342884038557994:9905] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:09.427721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342884038557994:9905] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:09.427724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342884038557994:9905] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:09.427729Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2050 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:09.427968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:09.428019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.428027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.428028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.428030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.428031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.428032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.428033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.428034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.428035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.428039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.428049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.428052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.428056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.428059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.428062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.428066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.428069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.428072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.428078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.428085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.428091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.428096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.428101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.428107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.428111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.428116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.428120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.428125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.428130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.428134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.428139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.428144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.428148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.428153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.428157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.428170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.428199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:09.434111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:09.435232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:09.435247Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:09.435249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:09.435253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:09.435258Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2049 [2:7559342883676723977:11093]
2025-10-09T21:51:09.435272Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:09.435287Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:09.435291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:09.435293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:09.435295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:09.435312Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:09.435322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:09.435329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:09.435388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.435396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.435398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.435401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:09.435463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:09.435471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.435483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.435498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.435519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:09.435542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342884038558084:9905], now have 1 active actors on pipe
2025-10-09T21:51:09.435550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342884038557955:2432], now have 1 active actors on pipe
2025-10-09T21:51:09.435558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342883676723971:11093]
2025-10-09T21:51:09.435560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342883676723971:11093]
2025-10-09T21:51:09.435563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342883676723971:11093]
2025-10-09T21:51:09.435576Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:09.435594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:09.435593Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:09.435596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:09.435598Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:09.435611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:09.435615Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:09.435610Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2049
2025-10-09T21:51:09.435621Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2049
2025-10-09T21:51:09.435638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342883676723971:11093]
2025-10-09T21:51:09.435649Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:09.435668Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:09.435677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.435683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.435713Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c5c5350-d8aad4fe-5e8e4d54-8daa5402_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:09.435725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:09.435728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:09.435730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:09.435740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:09.435753Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:09.435772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:09.435932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.435943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.435960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:09.436043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2050 partition: 0 MaxSeqNo: 25 sessionId: src_id|c5c5350-d8aad4fe-5e8e4d54-8daa5402_0
2025-10-09T21:51:09.436338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|15335c26-afe011f6-1fb10385-511caf5a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c5c5350-d8aad4fe-5e8e4d54-8daa5402_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:09.438732Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c5c5350-d8aad4fe-5e8e4d54-8daa5402_0] Write session: try to update token
2025-10-09T21:51:09.438746Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c5c5350-d8aad4fe-5e8e4d54-8daa5402_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:09.446681Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:09.446698Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:09.446707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342883676723971:11093]
2025-10-09T21:51:09.446750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:09.446762Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:09.446915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:09.447051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:09.447180Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:09.447312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:09.447491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:09.447528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:09.451143Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2050 sessionId: src_id|c5c5350-d8aad4fe-5e8e4d54-8daa5402_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:09.453322Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:09.453530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.453550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.453588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:09.453679Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:09.460064Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:09.460077Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:09.460079Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:09.460081Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:09.460092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342883676723971:11093]
2025-10-09T21:51:09.460095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342883676723971:11093]
2025-10-09T21:51:09.460102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342883676723971:11093]
2025-10-09T21:51:09.460141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:09.460148Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:09.460151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.460153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:09.460157Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:09.460161Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:09.460274Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:09.514209Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342884038558094:5250]
2025-10-09T21:51:09.514521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 2049, pipe: [1:7559342884038558094:5250]
2025-10-09T21:51:09.601374Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:09.608912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.608938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.609099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.609166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.609231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.609295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.609835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.609849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.609921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.609978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.610057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.610116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.610155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.610162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.610218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.610276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.610330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.610390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.610427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.610434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.610497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.610555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.610612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.610674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.610714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.610720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.610781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.610840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.610897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.610955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.610994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.611002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.611059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.611116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.611173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.611231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.611272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.611280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.611336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.611392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.611449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.611505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.611546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.611554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.611611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.611664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.611720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.611783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.611839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.611845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.611902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.611956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.612013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.612073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.612114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.612120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.612178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.612235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.612289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.612346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.612488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.612495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.612546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342884038558094:5250], now have 1 active actors on pipe
2025-10-09T21:51:09.612730Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:09.612751Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342884038558094:5250]
2025-10-09T21:51:09.612777Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:09.612780Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:09.612805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2050 reason: error at write: Disconnected sessionId: src_id|c5c5350-d8aad4fe-5e8e4d54-8daa5402_0
2025-10-09T21:51:09.612977Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2050 sessionId: src_id|c5c5350-d8aad4fe-5e8e4d54-8daa5402_0 is DEAD
2025-10-09T21:51:09.613121Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5c5350-d8aad4fe-5e8e4d54-8daa5402_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:09.613134Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5c5350-d8aad4fe-5e8e4d54-8daa5402_0] Write session will restart in 0.009771s
2025-10-09T21:51:09.613157Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5c5350-d8aad4fe-5e8e4d54-8daa5402_0] Write session: Do CDS request
2025-10-09T21:51:09.613164Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5c5350-d8aad4fe-5e8e4d54-8daa5402_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:09.613110Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:09.623565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:09.624485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c5c5350-d8aad4fe-5e8e4d54-8daa5402_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:09.624552Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:09.624564Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2051
2025-10-09T21:51:09.624685Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2051 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:09.624715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2051 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:09.624728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2051 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:09.624734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:09.625059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2051 sessionId:  describe result for acl check
2025-10-09T21:51:09.625084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:09.625088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:09.625090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:09.625095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342884038558191:9898] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:09.625221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:09.625432Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:09.625441Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:09.625454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:09.625508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:09.625545Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342883676724107:11095]
2025-10-09T21:51:09.625601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342884038558191:9898] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:09.626100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:09.626364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:09.626389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:09.626620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:09.626656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.626663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.626664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.626665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.626666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.626666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.626667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.626668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.626669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.626674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.626684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.626688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.626692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.626695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.626699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.626702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.626704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.626706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.626710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.626716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.626720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.626723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.626726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.626729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.626731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.626734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.626736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.626739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.626741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.626743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.626746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.626749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.626751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.626754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.626756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.626759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.626776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:09.627983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342884038558191:9898] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:09.631997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342884038558191:9898] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:09.632014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342884038558191:9898] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:09.632018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342884038558191:9898] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:09.632026Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2051 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:09.632349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:09.633282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:09.633293Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:09.633294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:09.633297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:09.633302Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2050 [2:7559342883676724107:11095]
2025-10-09T21:51:09.633317Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:09.633333Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:09.633336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:09.633339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:09.633341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:09.633354Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:09.633362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:09.633367Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:09.633419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.633425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.633426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.633428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:09.633478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:09.633483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.633494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.633500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.633513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:09.633530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342884038558229:9898], now have 1 active actors on pipe
2025-10-09T21:51:09.633532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342884038558097:2432], now have 1 active actors on pipe
2025-10-09T21:51:09.633538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342883676724101:11095]
2025-10-09T21:51:09.633543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342883676724101:11095]
2025-10-09T21:51:09.633544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342883676724101:11095]
2025-10-09T21:51:09.633553Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:09.633563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:09.633575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:09.633576Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:09.633580Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:09.633581Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:09.633582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:09.633596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342883676724101:11095]
2025-10-09T21:51:09.633601Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:09.633596Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2050
2025-10-09T21:51:09.633620Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:09.633610Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2050
2025-10-09T21:51:09.633647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.633655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.633677Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cbb55d37-202aba16-58b61b9a-b1b6c227_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:09.633687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:09.633689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:09.633691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:09.633698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:09.633709Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:09.633719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:09.633843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.633849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.633860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:09.633985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2051 partition: 0 MaxSeqNo: 25 sessionId: src_id|cbb55d37-202aba16-58b61b9a-b1b6c227_0
2025-10-09T21:51:09.634293Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c5c5350-d8aad4fe-5e8e4d54-8daa5402_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cbb55d37-202aba16-58b61b9a-b1b6c227_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:09.636642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cbb55d37-202aba16-58b61b9a-b1b6c227_0] Write session: try to update token
2025-10-09T21:51:09.636659Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cbb55d37-202aba16-58b61b9a-b1b6c227_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:09.644719Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:09.644739Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:09.644752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342883676724101:11095]
2025-10-09T21:51:09.644794Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:09.644799Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:09.644966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:09.645107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:09.645240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:09.645371Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:09.645564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:09.645606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:09.648810Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2051 sessionId: src_id|cbb55d37-202aba16-58b61b9a-b1b6c227_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:09.650856Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:09.651163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.651182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.651219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:09.651282Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:09.656535Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:09.656547Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:09.656549Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:09.656550Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:09.656557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342883676724101:11095]
2025-10-09T21:51:09.656561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342883676724101:11095]
2025-10-09T21:51:09.656563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342883676724101:11095]
2025-10-09T21:51:09.656598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:09.656605Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:09.656606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.656608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:09.656609Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:09.656613Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:09.656725Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:09.712906Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342884038558236:5250]
2025-10-09T21:51:09.713198Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 2050, pipe: [1:7559342884038558236:5250]
2025-10-09T21:51:09.796832Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:09.803978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.804003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.804160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.804223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.804283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.804353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.804400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.804410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.804474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.804535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.804596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.804658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.804697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.804703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.804760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.804812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.804866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.804923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.804964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.804969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.805042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.805100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.805157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.805213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.805313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.805319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.805381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.805451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.805509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.805565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.805605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.805610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.805680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.805745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.805805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.805865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.805914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.805919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.805982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.806050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.806112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.806176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.806220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.806227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.806286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.806334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.806391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.806445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.806488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.806494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.806549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.806604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.806669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.806736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.806776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.806781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.806841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:09.806894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:09.806952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:09.807014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:09.807058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:09.807066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:09.807110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342884038558236:5250], now have 1 active actors on pipe
2025-10-09T21:51:09.807417Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:09.807429Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:09.807440Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:09.807459Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2051 reason: error at write: Disconnected sessionId: src_id|cbb55d37-202aba16-58b61b9a-b1b6c227_0
2025-10-09T21:51:09.807496Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342884038558236:5250]
2025-10-09T21:51:09.807666Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2051 sessionId: src_id|cbb55d37-202aba16-58b61b9a-b1b6c227_0 is DEAD
2025-10-09T21:51:09.807799Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cbb55d37-202aba16-58b61b9a-b1b6c227_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:09.807811Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cbb55d37-202aba16-58b61b9a-b1b6c227_0] Write session will restart in 0.007241s
2025-10-09T21:51:09.807842Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cbb55d37-202aba16-58b61b9a-b1b6c227_0] Write session: Do CDS request
2025-10-09T21:51:09.807849Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cbb55d37-202aba16-58b61b9a-b1b6c227_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:09.807949Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:09.816537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cbb55d37-202aba16-58b61b9a-b1b6c227_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:09.816635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:09.816653Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2052
2025-10-09T21:51:09.816785Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2052 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:09.816810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2052 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:09.816826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2052 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:09.816834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:09.817399Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2052 sessionId:  describe result for acl check
2025-10-09T21:51:09.817447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:09.817448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:09.817450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:09.817455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342884038558330:9883] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:09.818121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342884038558330:9883] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:09.818816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:09.821075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:09.821135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342884038558330:9883] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:09.821294Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:09.821304Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:09.821317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:09.821375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:09.821431Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342883676724243:11099]
2025-10-09T21:51:09.822072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:09.822381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:09.822433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:09.822758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:09.822809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.822816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.822818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.822819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.822820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.822822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.822823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.822824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.822825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.822831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.822842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.822846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.822851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.822854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.822859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.822863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.822867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.822871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.822877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.822885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:09.822891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.822897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:09.822903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.822912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:09.822917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.822926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:09.822937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.822943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:09.822947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.822953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:09.822958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.822964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.822968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.822973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.822983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.822988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.823022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:09.823668Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342884038558330:9883] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:09.823684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342884038558330:9883] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:09.823688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342884038558330:9883] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:09.823698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2052 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:09.829741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:09.831118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:09.831132Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:09.831135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:09.831141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:09.831149Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2051 [2:7559342883676724243:11099]
2025-10-09T21:51:09.831165Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:09.831186Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:09.831189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:09.831193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:09.831195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:09.831215Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:09.831233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:09.831248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:09.831316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.831322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.831325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.831328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:09.831404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:09.831411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.831423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:09.831434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:09.831444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:09.831467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342883676724237:11099]
2025-10-09T21:51:09.831475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342883676724237:11099]
2025-10-09T21:51:09.831477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342883676724237:11099]
2025-10-09T21:51:09.831489Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:09.831503Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:09.831506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342884038558239:2432], now have 1 active actors on pipe
2025-10-09T21:51:09.831507Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:09.831509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342884038558372:9883], now have 1 active actors on pipe
2025-10-09T21:51:09.831509Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:09.831517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:09.831527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:09.831533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:09.831566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342883676724237:11099]
2025-10-09T21:51:09.831575Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:09.831569Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2051
2025-10-09T21:51:09.831599Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2051
2025-10-09T21:51:09.831600Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:09.831650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.831659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.831685Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e90eb067-efaa8200-7737145e-8d538461_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:09.831694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:09.831697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:09.831700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:09.831708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:09.831718Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:09.831740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:09.831878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.831885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.831897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:09.831995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2052 partition: 0 MaxSeqNo: 25 sessionId: src_id|e90eb067-efaa8200-7737145e-8d538461_0
2025-10-09T21:51:09.832412Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cbb55d37-202aba16-58b61b9a-b1b6c227_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e90eb067-efaa8200-7737145e-8d538461_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:09.834983Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e90eb067-efaa8200-7737145e-8d538461_0] Write session: try to update token
2025-10-09T21:51:09.835003Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e90eb067-efaa8200-7737145e-8d538461_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:09.843881Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:09.843899Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:09.843909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342883676724237:11099]
2025-10-09T21:51:09.843960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:09.843970Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:09.844144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:09.844294Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:09.844436Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:09.844584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:09.844794Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:09.844834Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:09.848389Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2052 sessionId: src_id|e90eb067-efaa8200-7737145e-8d538461_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:09.851080Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:09.851303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:09.851325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:09.851366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:09.851470Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:09.856715Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:09.856731Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:09.856733Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:09.856734Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:09.856743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342883676724237:11099]
2025-10-09T21:51:09.856746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342883676724237:11099]
2025-10-09T21:51:09.856749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342883676724237:11099]
2025-10-09T21:51:09.856786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:09.856792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:09.856793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:09.856794Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:09.856801Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:09.856809Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:09.856915Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:09.907475Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342884038558381:5250]
2025-10-09T21:51:09.907728Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 2051, pipe: [1:7559342884038558381:5250]
2025-10-09T21:51:10.000432Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:10.007613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.007640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.007780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.007837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.007902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.007960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.008431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.008442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.008504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.008557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.008626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.008692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.008732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.008737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.008792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.008844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.008902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.008959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.008999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.009005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.009066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.009132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.009199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.009263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.009303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.009313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.009380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.009459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.009521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.009580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.009621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.009626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.009690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.009755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.009816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.009872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.009917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.009925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.009989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.010059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.010118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.010176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.010220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.010226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.010282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.010334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.010391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.010452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.010567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.010578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.010641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.010694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.010759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.010825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.010867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.010875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.010948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.011005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.011060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.011123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.011177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.011186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.011249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342884038558381:5250], now have 1 active actors on pipe
2025-10-09T21:51:10.011467Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:10.011482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:10.011490Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:10.011515Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2052 reason: error at write: Disconnected sessionId: src_id|e90eb067-efaa8200-7737145e-8d538461_0
2025-10-09T21:51:10.011544Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342884038558381:5250]
2025-10-09T21:51:10.011711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2052 sessionId: src_id|e90eb067-efaa8200-7737145e-8d538461_0 is DEAD
2025-10-09T21:51:10.011842Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:10.011851Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e90eb067-efaa8200-7737145e-8d538461_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:10.011877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e90eb067-efaa8200-7737145e-8d538461_0] Write session will restart in 0.005583s
2025-10-09T21:51:10.011902Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e90eb067-efaa8200-7737145e-8d538461_0] Write session: Do CDS request
2025-10-09T21:51:10.011907Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e90eb067-efaa8200-7737145e-8d538461_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:10.018392Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e90eb067-efaa8200-7737145e-8d538461_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:10.018488Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:10.018504Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2053
2025-10-09T21:51:10.018613Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2053 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:10.018648Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2053 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:10.018658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2053 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:10.018662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:10.019071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2053 sessionId:  describe result for acl check
2025-10-09T21:51:10.019102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:10.019108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:10.019110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:10.019116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342888333525719:9875] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:10.019666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342888333525719:9875] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:10.021697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342888333525719:9875] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:10.022798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:10.024651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:10.024813Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:10.024821Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:10.024831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:10.024869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:10.024899Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342887971691709:11113]
2025-10-09T21:51:10.024994Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342888333525719:9875] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:10.025004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342888333525719:9875] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:10.025007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342888333525719:9875] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:10.025013Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2053 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:10.025261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:10.025508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:10.025536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:10.025732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:10.025762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.025768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.025768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.025769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.025770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.025770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.025771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.025772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.025772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.025775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.025782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.025784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.025786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.025788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.025790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.025792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.025794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.025796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.025801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.025805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.025810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.025813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.025815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.025819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.025821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.025824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.025826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.025829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.025832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.025835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.025838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.025841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.025843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.025846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.025849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.025852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.025868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:10.031927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:10.032913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:10.032927Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:10.032928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:10.032931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:10.032937Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2052 [2:7559342887971691709:11113]
2025-10-09T21:51:10.032958Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:10.032973Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:10.032978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:10.032981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:10.032983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:10.032997Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:10.033004Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:10.033010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:10.033050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.033057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.033058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.033060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:10.033115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:10.033120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.033127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.033134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.033143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:10.033161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342887971691703:11113]
2025-10-09T21:51:10.033166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342887971691703:11113]
2025-10-09T21:51:10.033167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342887971691703:11113]
2025-10-09T21:51:10.033175Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:10.033184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:10.033190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333525831:9875], now have 1 active actors on pipe
2025-10-09T21:51:10.033194Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:10.033195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:10.033196Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:10.033204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:10.033205Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:10.033210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333525704:2432], now have 1 active actors on pipe
2025-10-09T21:51:10.033231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342887971691703:11113]
2025-10-09T21:51:10.033240Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:10.033272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:10.033274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2052
2025-10-09T21:51:10.033274Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2052
2025-10-09T21:51:10.033372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.033379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.033415Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4d3d5e01-277a687-c7fd7311-9fab07c8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:10.033424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:10.033427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:10.033428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:10.033436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:10.033444Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:10.033456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:10.033609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.033616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.033630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:10.033729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2053 partition: 0 MaxSeqNo: 25 sessionId: src_id|4d3d5e01-277a687-c7fd7311-9fab07c8_0
2025-10-09T21:51:10.034059Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e90eb067-efaa8200-7737145e-8d538461_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4d3d5e01-277a687-c7fd7311-9fab07c8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:10.036594Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4d3d5e01-277a687-c7fd7311-9fab07c8_0] Write session: try to update token
2025-10-09T21:51:10.036609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4d3d5e01-277a687-c7fd7311-9fab07c8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:10.044978Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:10.044995Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:10.045005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342887971691703:11113]
2025-10-09T21:51:10.045049Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:10.045052Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:10.045211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:10.045354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:10.045496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:10.045627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:10.045930Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:10.045972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:10.049930Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2053 sessionId: src_id|4d3d5e01-277a687-c7fd7311-9fab07c8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:10.052439Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:10.052620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.052641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.052687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:10.052783Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:10.058433Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:10.058449Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:10.058451Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:10.058453Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:10.058467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342887971691703:11113]
2025-10-09T21:51:10.058471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342887971691703:11113]
2025-10-09T21:51:10.058474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342887971691703:11113]
2025-10-09T21:51:10.058512Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:10.058515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:10.058522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.058525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:10.058530Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:10.058533Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:10.058655Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:10.111737Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342888333525843:5250]
2025-10-09T21:51:10.111963Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 2052, pipe: [1:7559342888333525843:5250]
2025-10-09T21:51:10.209920Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:10.217052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.217076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.217222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.217282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.217353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.217438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.217490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.217501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.217572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.217639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.217704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.217779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.218223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.218238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.218296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.218355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.218417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.218479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.218523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.218531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.218603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.218671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.218733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.218794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.218837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.218846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.218913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.218978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.219046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.219109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.219148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.219150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.219206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.219261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.219322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.219375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.219414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.219419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.219480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.219535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.219596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.219660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.219706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.219713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.219777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.219837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.219899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.219988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.220029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.220033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.220088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.220136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.220190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.220240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.220280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.220282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.220336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:10.220389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:10.220450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:10.220510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:10.220555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:10.220562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:10.220627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333525843:5250], now have 1 active actors on pipe
2025-10-09T21:51:10.220892Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:10.220894Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342888333525843:5250]
2025-10-09T21:51:10.220903Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:10.220910Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:10.220927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2053 reason: error at write: Disconnected sessionId: src_id|4d3d5e01-277a687-c7fd7311-9fab07c8_0
2025-10-09T21:51:10.221082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2053 sessionId: src_id|4d3d5e01-277a687-c7fd7311-9fab07c8_0 is DEAD
2025-10-09T21:51:10.221188Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:10.221236Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d3d5e01-277a687-c7fd7311-9fab07c8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:10.221249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d3d5e01-277a687-c7fd7311-9fab07c8_0] Write session will restart in 0.009302s
2025-10-09T21:51:10.221272Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d3d5e01-277a687-c7fd7311-9fab07c8_0] Write session: Do CDS request
2025-10-09T21:51:10.221279Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d3d5e01-277a687-c7fd7311-9fab07c8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:10.231508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4d3d5e01-277a687-c7fd7311-9fab07c8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:10.231576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:10.231591Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2054
2025-10-09T21:51:10.231743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2054 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:10.231776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2054 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:10.231786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2054 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:10.231789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:10.232146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2054 sessionId:  describe result for acl check
2025-10-09T21:51:10.232178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:10.232184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:10.232186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:10.232190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342888333525944:9863] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:10.232207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:10.232707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342888333525944:9863] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:10.233982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:10.234195Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:10.234203Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:10.234213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:10.234259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:10.234288Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342887971691842:11115]
2025-10-09T21:51:10.234734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:10.235000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:10.235031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:10.235279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:10.235313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.235317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.235318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.235319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.235319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.235320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.235321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.235321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.235322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.235325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.235331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.235333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.235335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.235338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.235340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.235342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.235344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.235346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.235350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.235355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.235359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.235362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.235365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.235368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.235370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.235373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.235384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.235387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.235390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.235396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.235400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.235409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.235413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.235423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.235430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.235434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.235454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:10.235446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342888333525944:9863] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:10.241354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342888333525944:9863] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:10.241370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342888333525944:9863] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:10.241374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342888333525944:9863] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:10.241383Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2054 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:10.241733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:10.242710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:10.242723Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:10.242724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:10.242728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:10.242733Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2053 [2:7559342887971691842:11115]
2025-10-09T21:51:10.242743Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:10.242757Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:10.242763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:10.242765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:10.242767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:10.242780Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:10.242787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:10.242792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:10.242833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.242838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.242839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.242841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:10.242902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:10.242908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.242916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.242922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.242931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:10.242944Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342887971691837:11115]
2025-10-09T21:51:10.242951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342887971691837:11115]
2025-10-09T21:51:10.242952Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342887971691837:11115]
2025-10-09T21:51:10.242961Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:10.242971Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:10.243014Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2053
2025-10-09T21:51:10.242975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333525848:2432], now have 1 active actors on pipe
2025-10-09T21:51:10.243028Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2053
2025-10-09T21:51:10.242979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333525983:9863], now have 1 active actors on pipe
2025-10-09T21:51:10.242981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:10.242983Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:10.242986Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:10.243000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:10.243002Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:10.243027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342887971691837:11115]
2025-10-09T21:51:10.243030Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:10.243040Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:10.243061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.243076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.243104Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7c84d85d-fdc913b4-453751f3-58a28e89_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:10.243114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:10.243116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:10.243118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:10.243125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:10.243136Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:10.243153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:10.243312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.243319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.243330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:10.243401Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2054 partition: 0 MaxSeqNo: 25 sessionId: src_id|7c84d85d-fdc913b4-453751f3-58a28e89_0
2025-10-09T21:51:10.243739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d3d5e01-277a687-c7fd7311-9fab07c8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7c84d85d-fdc913b4-453751f3-58a28e89_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:10.246122Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c84d85d-fdc913b4-453751f3-58a28e89_0] Write session: try to update token
2025-10-09T21:51:10.246137Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c84d85d-fdc913b4-453751f3-58a28e89_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:10.257066Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:10.257082Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:10.257083Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:10.257085Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:10.257093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342887971691837:11115]
2025-10-09T21:51:10.257097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342887971691837:11115]
2025-10-09T21:51:10.257100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342887971691837:11115]
2025-10-09T21:51:10.257135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:10.257141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.257143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:10.257184Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:10.257206Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:10.257220Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:10.257258Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:10.258449Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2054 sessionId: src_id|7c84d85d-fdc913b4-453751f3-58a28e89_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:10.260728Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:10.321143Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342888333525996:5250]
2025-10-09T21:51:10.321373Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 2053, pipe: [1:7559342888333525996:5250]
2025-10-09T21:51:10.399309Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:10.399735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.399752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.399795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333525996:5250], now have 1 active actors on pipe
2025-10-09T21:51:10.400012Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342888333525996:5250]
2025-10-09T21:51:10.400019Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:10.400047Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:10.400050Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:10.400139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2054 reason: error at write: Disconnected sessionId: src_id|7c84d85d-fdc913b4-453751f3-58a28e89_0
2025-10-09T21:51:10.400236Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:10.400317Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2054 sessionId: src_id|7c84d85d-fdc913b4-453751f3-58a28e89_0 is DEAD
2025-10-09T21:51:10.400471Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c84d85d-fdc913b4-453751f3-58a28e89_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:10.400483Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c84d85d-fdc913b4-453751f3-58a28e89_0] Write session will restart in 0.009245s
2025-10-09T21:51:10.400508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c84d85d-fdc913b4-453751f3-58a28e89_0] Write session: Do CDS request
2025-10-09T21:51:10.400515Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c84d85d-fdc913b4-453751f3-58a28e89_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:10.410485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c84d85d-fdc913b4-453751f3-58a28e89_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:10.410523Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:10.410536Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2055
2025-10-09T21:51:10.410695Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2055 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:10.410731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2055 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:10.410741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2055 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:10.410745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:10.411131Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2055 sessionId:  describe result for acl check
2025-10-09T21:51:10.411170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:10.411176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:10.411178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:10.411184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342888333526094:9864] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:10.411460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:10.411704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342888333526094:9864] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:10.413207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:10.413371Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:10.413381Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:10.413401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:10.413454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:10.413495Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342887971691976:11117]
2025-10-09T21:51:10.413674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342888333526094:9864] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:10.414069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:10.414348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:10.414396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:10.414696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:10.414742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.414748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.414749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.414750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.414752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.414753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.414754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.414755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.414756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.414761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.414770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.414774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.414778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.414781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.414785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.414789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.414793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.414796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.414803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.414810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.414816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.414822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.414826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.414831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.414835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.414840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.414845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.414850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.414854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.414859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.414864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.414867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.414872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.414876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.414880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.414884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.414911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:10.419577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342888333526094:9864] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:10.419588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342888333526094:9864] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:10.419591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342888333526094:9864] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:10.419609Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2055 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:10.420878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:10.421925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:10.421940Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:10.421943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:10.421947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:10.421952Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2054 [2:7559342887971691976:11117]
2025-10-09T21:51:10.421963Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:10.421985Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:10.421992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:10.421995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:10.421997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:10.422013Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:10.422023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:10.422046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:10.422105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.422112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.422114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.422117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:10.422180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:10.422188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.422199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.422211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.422228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:10.422251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333526134:9864], now have 1 active actors on pipe
2025-10-09T21:51:10.422259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333525997:2432], now have 1 active actors on pipe
2025-10-09T21:51:10.422266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342887971691971:11117]
2025-10-09T21:51:10.422272Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342887971691971:11117]
2025-10-09T21:51:10.422274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342887971691971:11117]
2025-10-09T21:51:10.422288Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:10.422306Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:10.422294Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2054
2025-10-09T21:51:10.422296Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2054
2025-10-09T21:51:10.422305Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:10.422310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:10.422312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:10.422324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:10.422328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:10.422350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342887971691971:11117]
2025-10-09T21:51:10.422357Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:10.422359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:10.422376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.422385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.422427Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|85d5d18c-31e5aecb-9fa0e193-d138bdce_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:10.422441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:10.422445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:10.422448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:10.422459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:10.422474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:10.422495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:10.422674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.422685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.422699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:10.422833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2055 partition: 0 MaxSeqNo: 25 sessionId: src_id|85d5d18c-31e5aecb-9fa0e193-d138bdce_0
2025-10-09T21:51:10.423140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c84d85d-fdc913b4-453751f3-58a28e89_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|85d5d18c-31e5aecb-9fa0e193-d138bdce_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:10.425416Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85d5d18c-31e5aecb-9fa0e193-d138bdce_0] Write session: try to update token
2025-10-09T21:51:10.425428Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85d5d18c-31e5aecb-9fa0e193-d138bdce_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:10.436258Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:10.436277Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:10.436279Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:10.436282Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:10.436291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342887971691971:11117]
2025-10-09T21:51:10.436295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342887971691971:11117]
2025-10-09T21:51:10.436299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342887971691971:11117]
2025-10-09T21:51:10.436339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:10.436340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:10.436346Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:10.436347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.436349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:10.436350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:10.436481Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:10.437670Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2055 sessionId: src_id|85d5d18c-31e5aecb-9fa0e193-d138bdce_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:10.439961Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:10.500478Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342888333526142:5250]
2025-10-09T21:51:10.500733Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 2054, pipe: [1:7559342888333526142:5250]
2025-10-09T21:51:10.581138Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:10.581514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.581533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.581574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333526142:5250], now have 1 active actors on pipe
2025-10-09T21:51:10.581836Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:10.581849Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:10.581851Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:10.581885Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342888333526142:5250]
2025-10-09T21:51:10.581917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2055 reason: error at write: Disconnected sessionId: src_id|85d5d18c-31e5aecb-9fa0e193-d138bdce_0
2025-10-09T21:51:10.581941Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:10.582115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2055 sessionId: src_id|85d5d18c-31e5aecb-9fa0e193-d138bdce_0 is DEAD
2025-10-09T21:51:10.582266Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85d5d18c-31e5aecb-9fa0e193-d138bdce_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:10.582284Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85d5d18c-31e5aecb-9fa0e193-d138bdce_0] Write session will restart in 0.008621s
2025-10-09T21:51:10.582305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85d5d18c-31e5aecb-9fa0e193-d138bdce_0] Write session: Do CDS request
2025-10-09T21:51:10.582310Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85d5d18c-31e5aecb-9fa0e193-d138bdce_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:10.592498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|85d5d18c-31e5aecb-9fa0e193-d138bdce_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:10.592556Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:10.592571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2056
2025-10-09T21:51:10.592656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:10.592705Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2056 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:10.592739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2056 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:10.592749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2056 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:10.592752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:10.593154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2056 sessionId:  describe result for acl check
2025-10-09T21:51:10.593191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:10.593198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:10.593199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:10.593205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342888333526242:9861] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:10.593663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342888333526242:9861] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:10.594635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:10.594814Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:10.594825Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:10.594837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:10.594893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:10.594931Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342887971692111:11119]
2025-10-09T21:51:10.595484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:10.595778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:10.595820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:10.595808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342888333526242:9861] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:10.596115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:10.596162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.596169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.596170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.596171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.596173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.596174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.596175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.596176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.596177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.596182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.596192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.596196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.596200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.596204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.596208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.596212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.596216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.596220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.596228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.596235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.596241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.596246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.596252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.596261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.596266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.596271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.596280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.596285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.596289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.596299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.596302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.596305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.596307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.596310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.596312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.596316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.596332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:10.601121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342888333526242:9861] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:10.601137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342888333526242:9861] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:10.601141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342888333526242:9861] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:10.601149Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2056 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:10.602608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:10.603650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:10.603672Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:10.603675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:10.603679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:10.603685Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2055 [2:7559342887971692111:11119]
2025-10-09T21:51:10.603702Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:10.603722Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:10.603726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:10.603729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:10.603731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:10.603747Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:10.603754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:10.603761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:10.603824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.603836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.603838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.603840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:10.603910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:10.603918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.603930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.603949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.603965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:10.603986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333526281:9861], now have 1 active actors on pipe
2025-10-09T21:51:10.603995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333526144:2432], now have 1 active actors on pipe
2025-10-09T21:51:10.604003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342887971692105:11119]
2025-10-09T21:51:10.604008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342887971692105:11119]
2025-10-09T21:51:10.604010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342887971692105:11119]
2025-10-09T21:51:10.604024Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:10.604040Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:10.604041Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:10.604045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:10.604048Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:10.604044Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2055
2025-10-09T21:51:10.604061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:10.604049Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2055
2025-10-09T21:51:10.604065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:10.604098Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342887971692105:11119]
2025-10-09T21:51:10.604102Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:10.604104Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:10.604119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.604124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.604319Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5c9e1b13-7127866d-1ba046cc-b2415545_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:10.604336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:10.604341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:10.604343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:10.604356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:10.604368Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:10.604407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:10.604641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.604653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.604672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:10.604816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2056 partition: 0 MaxSeqNo: 25 sessionId: src_id|5c9e1b13-7127866d-1ba046cc-b2415545_0
2025-10-09T21:51:10.605153Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|85d5d18c-31e5aecb-9fa0e193-d138bdce_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5c9e1b13-7127866d-1ba046cc-b2415545_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:10.607805Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9e1b13-7127866d-1ba046cc-b2415545_0] Write session: try to update token
2025-10-09T21:51:10.607827Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9e1b13-7127866d-1ba046cc-b2415545_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:10.619081Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:10.619101Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:10.619103Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:10.619105Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:10.619115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342887971692105:11119]
2025-10-09T21:51:10.619119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342887971692105:11119]
2025-10-09T21:51:10.619124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342887971692105:11119]
2025-10-09T21:51:10.619177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:10.619190Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:10.619192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.619194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:10.619196Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:10.619201Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:10.619324Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:10.621626Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2056 sessionId: src_id|5c9e1b13-7127866d-1ba046cc-b2415545_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:10.623955Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:10.682093Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342888333526289:5250]
2025-10-09T21:51:10.682354Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 2055, pipe: [1:7559342888333526289:5250]
2025-10-09T21:51:10.768240Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:10.768553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.768571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.768612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333526289:5250], now have 1 active actors on pipe
2025-10-09T21:51:10.768813Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:10.768846Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342888333526289:5250]
2025-10-09T21:51:10.768856Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:10.768859Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:10.768874Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:10.768923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2056 reason: error at write: Disconnected sessionId: src_id|5c9e1b13-7127866d-1ba046cc-b2415545_0
2025-10-09T21:51:10.769093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2056 sessionId: src_id|5c9e1b13-7127866d-1ba046cc-b2415545_0 is DEAD
2025-10-09T21:51:10.769239Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9e1b13-7127866d-1ba046cc-b2415545_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:10.769250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9e1b13-7127866d-1ba046cc-b2415545_0] Write session will restart in 0.005438s
2025-10-09T21:51:10.769277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9e1b13-7127866d-1ba046cc-b2415545_0] Write session: Do CDS request
2025-10-09T21:51:10.769294Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9e1b13-7127866d-1ba046cc-b2415545_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:10.775390Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9e1b13-7127866d-1ba046cc-b2415545_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:10.775494Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:10.775507Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2057
2025-10-09T21:51:10.775647Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2057 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:10.775677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2057 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:10.775687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2057 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:10.775690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:10.776123Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2057 sessionId:  describe result for acl check
2025-10-09T21:51:10.776157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:10.776163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:10.776165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:10.776171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342888333526316:9848] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:10.776794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342888333526316:9848] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:10.779210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342888333526316:9848] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:10.779943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:10.781649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:10.781796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342888333526316:9848] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:10.781808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342888333526316:9848] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:10.781812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342888333526316:9848] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:10.781830Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2057 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:10.781826Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:10.781838Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:10.781851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:10.781907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:10.781945Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342887971692247:11121]
2025-10-09T21:51:10.782523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:10.782802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:10.782844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:10.783143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:10.783214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.783220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.783221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.783223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.783224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.783225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.783226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.783227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.783228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.783233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.783242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.783246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.783250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.783253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.783258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.783261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.783265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.783269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.783276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.783284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.783290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.783296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.783301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.783306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.783311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.783315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.783320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.783325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.783333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.783338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.783344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.783353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.783357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.783362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.783369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.783373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.783403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:10.789608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:10.790813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:10.790826Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:10.790829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:10.790832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:10.790839Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2056 [2:7559342887971692247:11121]
2025-10-09T21:51:10.790850Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:10.790865Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:10.790871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:10.790874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:10.790877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:10.790893Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:10.790903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:10.790909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:10.790968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.790977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.790978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.790981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:10.791041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:10.791048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.791060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.791076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.791097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:10.791121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333526426:9848], now have 1 active actors on pipe
2025-10-09T21:51:10.791128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333526291:2432], now have 1 active actors on pipe
2025-10-09T21:51:10.791135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342887971692241:11121]
2025-10-09T21:51:10.791139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342887971692241:11121]
2025-10-09T21:51:10.791141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342887971692241:11121]
2025-10-09T21:51:10.791156Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:10.791149Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2056
2025-10-09T21:51:10.791167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:10.791161Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2056
2025-10-09T21:51:10.791173Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:10.791175Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:10.791176Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:10.791188Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:10.791191Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:10.791221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.791229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.791244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342887971692241:11121]
2025-10-09T21:51:10.791257Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:10.791266Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:10.791287Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4b3f8338-f259a4a1-6708c445-73720b82_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:10.791299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:10.791304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:10.791308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:10.791317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:10.791333Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:10.791359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:10.791555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.791564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.791579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:10.791700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2057 partition: 0 MaxSeqNo: 25 sessionId: src_id|4b3f8338-f259a4a1-6708c445-73720b82_0
2025-10-09T21:51:10.791977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c9e1b13-7127866d-1ba046cc-b2415545_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4b3f8338-f259a4a1-6708c445-73720b82_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:10.794306Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b3f8338-f259a4a1-6708c445-73720b82_0] Write session: try to update token
2025-10-09T21:51:10.794321Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b3f8338-f259a4a1-6708c445-73720b82_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:10.804797Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:10.804818Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:10.804820Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:10.804822Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:10.804831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342887971692241:11121]
2025-10-09T21:51:10.804836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342887971692241:11121]
2025-10-09T21:51:10.804839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342887971692241:11121]
2025-10-09T21:51:10.804883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:10.804890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.804890Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:10.804893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:10.804897Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:10.804900Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:10.805015Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:10.806358Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2057 sessionId: src_id|4b3f8338-f259a4a1-6708c445-73720b82_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:10.808499Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:10.868988Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342888333526437:5250]
2025-10-09T21:51:10.869239Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 2056, pipe: [1:7559342888333526437:5250]
2025-10-09T21:51:10.947449Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:10.947733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.947747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.947789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333526437:5250], now have 1 active actors on pipe
2025-10-09T21:51:10.947954Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:10.947987Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:10.948004Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:10.948057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2057 reason: error at write: Disconnected sessionId: src_id|4b3f8338-f259a4a1-6708c445-73720b82_0
2025-10-09T21:51:10.948138Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:10.948188Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342888333526437:5250]
2025-10-09T21:51:10.948235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2057 sessionId: src_id|4b3f8338-f259a4a1-6708c445-73720b82_0 is DEAD
2025-10-09T21:51:10.948414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b3f8338-f259a4a1-6708c445-73720b82_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:10.948424Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b3f8338-f259a4a1-6708c445-73720b82_0] Write session will restart in 0.006200s
2025-10-09T21:51:10.948445Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b3f8338-f259a4a1-6708c445-73720b82_0] Write session: Do CDS request
2025-10-09T21:51:10.948449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b3f8338-f259a4a1-6708c445-73720b82_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:10.955543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4b3f8338-f259a4a1-6708c445-73720b82_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:10.955663Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:10.955679Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2058
2025-10-09T21:51:10.955795Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2058 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:10.955826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2058 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:10.955831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2058 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:10.955834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:10.956213Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2058 sessionId:  describe result for acl check
2025-10-09T21:51:10.956244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:10.956250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:10.956252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:10.956257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342888333526464:9846] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:10.956654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342888333526464:9846] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:10.959004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342888333526464:9846] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:10.960938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:10.962762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:10.962954Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:10.962967Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:10.962955Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342888333526464:9846] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:10.962980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:10.962967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342888333526464:9846] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:10.962970Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342888333526464:9846] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:10.962978Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2058 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:10.963043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:10.963087Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342887971692401:11125]
2025-10-09T21:51:10.963613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:10.963913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:10.963966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:10.964268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:10.964315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.964322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.964323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.964325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.964326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.964327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.964328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.964329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.964331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.964335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.964345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.964349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.964353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.964357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.964360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.964364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.964368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.964371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.964377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.964384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:10.964390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.964395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:10.964399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.964407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:10.964411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.964416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:10.964420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.964424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:10.964432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.964437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:10.964442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.964446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.964450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.964455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.964459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.964463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.964491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:10.970514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:10.971600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:10.971616Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:10.971619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:10.971622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:10.971628Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2057 [2:7559342887971692401:11125]
2025-10-09T21:51:10.971642Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:10.971658Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:10.971661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:10.971664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:10.971666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:10.971681Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:10.971693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:10.971700Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:10.971761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.971769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.971771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.971773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:10.971838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:10.971845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.971856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:10.971870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:10.971893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:10.971917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333526589:9846], now have 1 active actors on pipe
2025-10-09T21:51:10.971926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342888333526442:2432], now have 1 active actors on pipe
2025-10-09T21:51:10.971933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342887971692396:11125]
2025-10-09T21:51:10.971935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342887971692396:11125]
2025-10-09T21:51:10.971936Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342887971692396:11125]
2025-10-09T21:51:10.971950Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:10.971961Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:10.971964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:10.971975Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:10.971978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:10.971978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:10.971951Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2057
2025-10-09T21:51:10.971992Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2057
2025-10-09T21:51:10.971982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:10.972018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342887971692396:11125]
2025-10-09T21:51:10.972027Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:10.972028Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:10.972042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.972051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.972097Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|718f97c-c8cd1535-865329d6-1e2c9066_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:10.972107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:10.972110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:10.972112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:10.972122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:10.972138Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:10.972156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:10.972335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:10.972342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:10.972354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:10.972434Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2058 partition: 0 MaxSeqNo: 25 sessionId: src_id|718f97c-c8cd1535-865329d6-1e2c9066_0
2025-10-09T21:51:10.972780Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4b3f8338-f259a4a1-6708c445-73720b82_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|718f97c-c8cd1535-865329d6-1e2c9066_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:10.975106Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|718f97c-c8cd1535-865329d6-1e2c9066_0] Write session: try to update token
2025-10-09T21:51:10.975123Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|718f97c-c8cd1535-865329d6-1e2c9066_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:10.986534Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:10.986554Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:10.986557Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:10.986558Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:10.986566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342887971692396:11125]
2025-10-09T21:51:10.986571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342887971692396:11125]
2025-10-09T21:51:10.986574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342887971692396:11125]
2025-10-09T21:51:10.986616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:10.986623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:10.986622Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:10.986625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:10.986628Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:10.986632Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:10.986754Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:10.987985Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2058 sessionId: src_id|718f97c-c8cd1535-865329d6-1e2c9066_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:10.990262Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:11.048173Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342892628493906:5250]
2025-10-09T21:51:11.048445Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 2057, pipe: [1:7559342892628493906:5250]
2025-10-09T21:51:11.135688Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:11.135928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:11.135940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:11.135978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342892628493906:5250], now have 1 active actors on pipe
2025-10-09T21:51:11.136143Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:11.136223Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:11.136264Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:11.136271Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342892628493906:5250]
2025-10-09T21:51:11.136327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2058 reason: status is not ok: tablet will be restarted right now sessionId: src_id|718f97c-c8cd1535-865329d6-1e2c9066_0
2025-10-09T21:51:11.136476Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2058 sessionId: src_id|718f97c-c8cd1535-865329d6-1e2c9066_0 is DEAD
2025-10-09T21:51:11.136627Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|718f97c-c8cd1535-865329d6-1e2c9066_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:11.136637Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|718f97c-c8cd1535-865329d6-1e2c9066_0] Write session will restart in 0.007197s
2025-10-09T21:51:11.136658Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|718f97c-c8cd1535-865329d6-1e2c9066_0] Write session: Do CDS request
2025-10-09T21:51:11.136663Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|718f97c-c8cd1535-865329d6-1e2c9066_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:11.144578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|718f97c-c8cd1535-865329d6-1e2c9066_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:11.144650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:11.144667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2059
2025-10-09T21:51:11.144804Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2059 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:11.144844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2059 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:11.144852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2059 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:11.144855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:11.145293Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2059 sessionId:  describe result for acl check
2025-10-09T21:51:11.145332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:11.145340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:11.145342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:11.145349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342892628493967:9828] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:11.145956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342892628493967:9828] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:11.147992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:11.148046Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342892628493967:9828] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:11.149815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:11.150007Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:11.150017Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:11.150043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:11.150104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:11.150144Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342892266659861:11139]
2025-10-09T21:51:11.150736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:11.150922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342892628493967:9828] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:11.150935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342892628493967:9828] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:11.150938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342892628493967:9828] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:11.150947Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2059 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:11.151031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:11.151077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:11.151397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:11.151443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.151450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.151452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.151453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.151454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.151455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.151456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.151458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.151459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.151464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.151475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.151479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.151483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.151487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.151491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.151495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.151499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.151503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.151509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.151517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.151523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.151528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.151533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.151538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.151543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.151548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.151553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.151559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.151565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.151570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.151574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.151579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.151584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.151589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.151593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.151598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.151627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:11.158134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:11.159189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:11.159203Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:11.159205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:11.159209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:11.159217Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2058 [2:7559342892266659861:11139]
2025-10-09T21:51:11.159232Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:11.159250Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:11.159258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:11.159261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:11.159264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:11.159280Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:11.159290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:11.159297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:11.159356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.159366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.159368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.159370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:11.159438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:11.159445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.159457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.159475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.159489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:11.159505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342892628494047:9828], now have 1 active actors on pipe
2025-10-09T21:51:11.159513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342892628493909:2432], now have 1 active actors on pipe
2025-10-09T21:51:11.159519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342892266659855:11139]
2025-10-09T21:51:11.159520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342892266659855:11139]
2025-10-09T21:51:11.159522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342892266659855:11139]
2025-10-09T21:51:11.159533Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:11.159544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:11.159557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:11.159559Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:11.159560Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:11.159562Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:11.159565Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:11.159588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342892266659855:11139]
2025-10-09T21:51:11.159590Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:11.159594Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:11.159574Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2058
2025-10-09T21:51:11.159579Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2058
2025-10-09T21:51:11.159621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:11.159630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:11.159658Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dbc7f970-365d19dc-4a60bc6a-710412de_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:11.159668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:11.159675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:11.159677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:11.159685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:11.159693Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:11.159709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:11.159843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:11.159857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:11.159880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:11.159990Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2059 partition: 0 MaxSeqNo: 25 sessionId: src_id|dbc7f970-365d19dc-4a60bc6a-710412de_0
2025-10-09T21:51:11.160337Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|718f97c-c8cd1535-865329d6-1e2c9066_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dbc7f970-365d19dc-4a60bc6a-710412de_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:11.162902Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dbc7f970-365d19dc-4a60bc6a-710412de_0] Write session: try to update token
2025-10-09T21:51:11.162917Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dbc7f970-365d19dc-4a60bc6a-710412de_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:11.174283Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:11.174303Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:11.174305Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:11.174307Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:11.174317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342892266659855:11139]
2025-10-09T21:51:11.174324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342892266659855:11139]
2025-10-09T21:51:11.174328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342892266659855:11139]
2025-10-09T21:51:11.174377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:11.174391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.174390Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:11.174394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:11.174399Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:11.174404Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:11.174527Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:11.176199Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2059 sessionId: src_id|dbc7f970-365d19dc-4a60bc6a-710412de_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:11.178710Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:11.236557Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342892628494059:5250]
2025-10-09T21:51:11.236796Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 2058, pipe: [1:7559342892628494059:5250]
2025-10-09T21:51:11.322647Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:11.322881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:11.322893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:11.322929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342892628494059:5250], now have 1 active actors on pipe
2025-10-09T21:51:11.323145Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:11.323176Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342892628494059:5250]
2025-10-09T21:51:11.323178Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:11.323183Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:11.323216Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:11.323237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2059 reason: error at write: Disconnected sessionId: src_id|dbc7f970-365d19dc-4a60bc6a-710412de_0
2025-10-09T21:51:11.323410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2059 sessionId: src_id|dbc7f970-365d19dc-4a60bc6a-710412de_0 is DEAD
2025-10-09T21:51:11.323537Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbc7f970-365d19dc-4a60bc6a-710412de_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:11.323547Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbc7f970-365d19dc-4a60bc6a-710412de_0] Write session will restart in 0.008397s
2025-10-09T21:51:11.323566Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbc7f970-365d19dc-4a60bc6a-710412de_0] Write session: Do CDS request
2025-10-09T21:51:11.323571Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbc7f970-365d19dc-4a60bc6a-710412de_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:11.333490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dbc7f970-365d19dc-4a60bc6a-710412de_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:11.333572Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:11.333586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2060
2025-10-09T21:51:11.333734Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2060 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:11.333768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2060 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:11.333773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2060 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:11.333775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:11.334144Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2060 sessionId:  describe result for acl check
2025-10-09T21:51:11.334174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:11.334179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:11.334180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:11.334184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342892628494168:9824] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:11.334202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:11.334589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342892628494168:9824] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:11.336059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:11.336230Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:11.336243Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:11.336256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:11.336314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:11.336354Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342892266660000:11141]
2025-10-09T21:51:11.336560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342892628494168:9824] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:11.336902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:11.337185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:11.337229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:11.337539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:11.337589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.337595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.337596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.337597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.337599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.337600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.337601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.337602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.337603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.337608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.337618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.337621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.337625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.337633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.337636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.337640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.337644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.337647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.337653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.337664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.337675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.337684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.337689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.337694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.337698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.337703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.337708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.337713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.337717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.337722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.337727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.337732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.337736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.337746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.337754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.337759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.337786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:11.338412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342892628494168:9824] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:11.338421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342892628494168:9824] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:11.338423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342892628494168:9824] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:11.338429Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2060 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:11.343917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:11.344927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:11.344941Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:11.344943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:11.344945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:11.344950Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2059 [2:7559342892266660000:11141]
2025-10-09T21:51:11.344959Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:11.344971Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:11.344972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:11.344974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:11.344976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:11.344988Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:11.344996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:11.345001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:11.345049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.345057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.345058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.345061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:11.345126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:11.345133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.345140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.345147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.345161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:11.345179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342892628494067:2432], now have 1 active actors on pipe
2025-10-09T21:51:11.345187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342892628494208:9824], now have 1 active actors on pipe
2025-10-09T21:51:11.345192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342892266659994:11141]
2025-10-09T21:51:11.345197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342892266659994:11141]
2025-10-09T21:51:11.345198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342892266659994:11141]
2025-10-09T21:51:11.345208Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:11.345226Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:11.345235Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:11.345236Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:11.345238Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:11.345243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:11.345245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:11.345247Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2059
2025-10-09T21:51:11.345253Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2059
2025-10-09T21:51:11.345265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342892266659994:11141]
2025-10-09T21:51:11.345273Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:11.345275Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:11.345296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:11.345300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:11.345331Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7b23d5c5-ad533200-abf9e0b5-e06aad31_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:11.345342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:11.345346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:11.345348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:11.345356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:11.345364Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:11.345380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:11.345523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:11.345529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:11.345539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:11.345609Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2060 partition: 0 MaxSeqNo: 25 sessionId: src_id|7b23d5c5-ad533200-abf9e0b5-e06aad31_0
2025-10-09T21:51:11.345901Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbc7f970-365d19dc-4a60bc6a-710412de_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7b23d5c5-ad533200-abf9e0b5-e06aad31_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:11.348351Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b23d5c5-ad533200-abf9e0b5-e06aad31_0] Write session: try to update token
2025-10-09T21:51:11.348369Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b23d5c5-ad533200-abf9e0b5-e06aad31_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:11.359536Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:11.359556Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:11.359559Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:11.359561Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:11.359570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342892266659994:11141]
2025-10-09T21:51:11.359576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342892266659994:11141]
2025-10-09T21:51:11.359579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342892266659994:11141]
2025-10-09T21:51:11.359619Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:11.359622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:11.359629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.359632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:11.359634Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:11.359639Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:11.359766Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:11.361129Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2060 sessionId: src_id|7b23d5c5-ad533200-abf9e0b5-e06aad31_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:11.363373Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:11.423470Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342892628494215:5250]
2025-10-09T21:51:11.423764Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 2059, pipe: [1:7559342892628494215:5250]
2025-10-09T21:51:11.502885Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:11.503251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:11.503263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:11.503299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342892628494215:5250], now have 1 active actors on pipe
2025-10-09T21:51:11.503494Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:11.503508Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:11.503533Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:11.503536Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:11.503558Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342892628494215:5250]
2025-10-09T21:51:11.503601Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2060 reason: error at write: Disconnected sessionId: src_id|7b23d5c5-ad533200-abf9e0b5-e06aad31_0
2025-10-09T21:51:11.503790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2060 sessionId: src_id|7b23d5c5-ad533200-abf9e0b5-e06aad31_0 is DEAD
2025-10-09T21:51:11.503930Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b23d5c5-ad533200-abf9e0b5-e06aad31_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:11.503941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b23d5c5-ad533200-abf9e0b5-e06aad31_0] Write session will restart in 0.008000s
2025-10-09T21:51:11.503969Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b23d5c5-ad533200-abf9e0b5-e06aad31_0] Write session: Do CDS request
2025-10-09T21:51:11.503976Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b23d5c5-ad533200-abf9e0b5-e06aad31_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:11.513511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7b23d5c5-ad533200-abf9e0b5-e06aad31_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:11.513598Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:11.513611Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2061
2025-10-09T21:51:11.513714Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2061 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:11.513745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2061 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:11.513767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2061 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:11.513770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:11.514124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2061 sessionId:  describe result for acl check
2025-10-09T21:51:11.514149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:11.514154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:11.514155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:11.514159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342892628494321:9818] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:11.514361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:11.514684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342892628494321:9818] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:11.515998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:11.516162Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:11.516173Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:11.516183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:11.516230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:11.516269Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342892266660141:11143]
2025-10-09T21:51:11.516464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342892628494321:9818] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:11.516809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:11.517073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:11.517116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:11.517437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:11.517480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.517486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.517487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.517488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.517489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.517490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.517491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.517492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.517493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.517497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.517508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.517512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.517516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.517520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.517524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.517527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.517531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.517535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.517545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.517552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.517558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.517568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.517576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.517582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.517589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.517594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.517599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.517604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.517610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.517615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.517623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.517629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.517638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.517642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.517647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.517652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.517682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:11.522408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342892628494321:9818] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:11.522420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342892628494321:9818] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:11.522424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342892628494321:9818] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:11.522430Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2061 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:11.523829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:11.524873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:11.524884Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:11.524887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:11.524890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:11.524896Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2060 [2:7559342892266660141:11143]
2025-10-09T21:51:11.524997Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:11.525016Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:11.525019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:11.525023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:11.525025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:11.525040Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:11.525046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:11.525052Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:11.525102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.525109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.525111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.525114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:11.525176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:11.525183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.525197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.525213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.525235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:11.525260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342892628494361:9818], now have 1 active actors on pipe
2025-10-09T21:51:11.525268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342892628494218:2432], now have 1 active actors on pipe
2025-10-09T21:51:11.525275Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342892266660135:11143]
2025-10-09T21:51:11.525282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342892266660135:11143]
2025-10-09T21:51:11.525284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342892266660135:11143]
2025-10-09T21:51:11.525301Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:11.525282Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2060
2025-10-09T21:51:11.525306Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2060
2025-10-09T21:51:11.525312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:11.525318Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:11.525320Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:11.525322Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:11.525333Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:11.525336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:11.525373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:11.525382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:11.525408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342892266660135:11143]
2025-10-09T21:51:11.525420Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:11.525424Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:11.525445Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e9b8ab08-7af1e263-8bf72472-3b2baf2c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:11.525458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:11.525466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:11.525469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:11.525478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:11.525491Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:11.525518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:11.525640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:11.525650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:11.525663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:11.525764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2061 partition: 0 MaxSeqNo: 25 sessionId: src_id|e9b8ab08-7af1e263-8bf72472-3b2baf2c_0
2025-10-09T21:51:11.526063Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7b23d5c5-ad533200-abf9e0b5-e06aad31_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e9b8ab08-7af1e263-8bf72472-3b2baf2c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:11.528399Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9b8ab08-7af1e263-8bf72472-3b2baf2c_0] Write session: try to update token
2025-10-09T21:51:11.528415Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9b8ab08-7af1e263-8bf72472-3b2baf2c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:11.539251Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:11.539270Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:11.539273Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:11.539275Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:11.539284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342892266660135:11143]
2025-10-09T21:51:11.539289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342892266660135:11143]
2025-10-09T21:51:11.539293Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342892266660135:11143]
2025-10-09T21:51:11.539338Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:11.539341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:11.539348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.539350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:11.539351Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:11.539354Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:11.539478Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:11.540615Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2061 sessionId: src_id|e9b8ab08-7af1e263-8bf72472-3b2baf2c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:11.542913Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:11.603833Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342892628494368:5250]
2025-10-09T21:51:11.604141Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 2060, pipe: [1:7559342892628494368:5250]
2025-10-09T21:51:11.679988Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:11.680200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:11.680211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:11.680247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342892628494368:5250], now have 1 active actors on pipe
2025-10-09T21:51:11.680396Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:11.680445Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:11.680449Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:11.680459Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:11.680492Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342892628494368:5250]
2025-10-09T21:51:11.680533Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2061 reason: error at write: Disconnected sessionId: src_id|e9b8ab08-7af1e263-8bf72472-3b2baf2c_0
2025-10-09T21:51:11.680707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2061 sessionId: src_id|e9b8ab08-7af1e263-8bf72472-3b2baf2c_0 is DEAD
2025-10-09T21:51:11.680850Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9b8ab08-7af1e263-8bf72472-3b2baf2c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:11.680862Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9b8ab08-7af1e263-8bf72472-3b2baf2c_0] Write session will restart in 0.009008s
2025-10-09T21:51:11.680885Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9b8ab08-7af1e263-8bf72472-3b2baf2c_0] Write session: Do CDS request
2025-10-09T21:51:11.680890Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9b8ab08-7af1e263-8bf72472-3b2baf2c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:11.691359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:11.691510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9b8ab08-7af1e263-8bf72472-3b2baf2c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:11.691571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:11.691581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2062
2025-10-09T21:51:11.691710Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2062 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:11.691742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2062 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:11.691748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2062 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:11.691752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:11.692114Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2062 sessionId:  describe result for acl check
2025-10-09T21:51:11.692142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:11.692144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:11.692145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:11.692148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342892628494475:9809] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:11.692566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342892628494475:9809] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:11.693171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:11.693303Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:11.693316Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:11.693327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:11.693371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:11.693422Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342892266660282:11145]
2025-10-09T21:51:11.693897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:11.694175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:11.694217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:11.694452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:11.694484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.694489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.694490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.694491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.694491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.694492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.694492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.694493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.694493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.694497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.694503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.694505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.694507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.694509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.694511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.694513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.694515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.694517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.694522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.694526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.694530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.694534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.694536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.694539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.694541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.694544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.694549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.694552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.694555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.694558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.694551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342892628494475:9809] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:11.694561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.694566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.694570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.694575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.694579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.694584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.694611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:11.700121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342892628494475:9809] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:11.700133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342892628494475:9809] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:11.700137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342892628494475:9809] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:11.700144Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2062 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:11.700608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:11.701664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:11.701679Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:11.701681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:11.701685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:11.701691Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2061 [2:7559342892266660282:11145]
2025-10-09T21:51:11.701703Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:11.701721Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:11.701728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:11.701732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:11.701734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:11.701748Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:11.701757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:11.701764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:11.701810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.701816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.701817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.701819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:11.701879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:11.701885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.701892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.701902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.701915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:11.701932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342892628494514:9809], now have 1 active actors on pipe
2025-10-09T21:51:11.701937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342892628494370:2432], now have 1 active actors on pipe
2025-10-09T21:51:11.701944Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342892266660276:11145]
2025-10-09T21:51:11.701945Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342892266660276:11145]
2025-10-09T21:51:11.701947Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342892266660276:11145]
2025-10-09T21:51:11.701956Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:11.701966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:11.701979Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:11.701978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:11.701981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:11.701981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:11.701974Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2061
2025-10-09T21:51:11.701984Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:11.701989Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2061
2025-10-09T21:51:11.702002Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342892266660276:11145]
2025-10-09T21:51:11.702004Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:11.702015Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:11.702071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:11.702079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:11.702108Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b21b3036-5065d262-af83b43b-74eba27a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:11.702117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:11.702124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:11.702125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:11.702133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:11.702141Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:11.702161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:11.702289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:11.702300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:11.702321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:11.702375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2062 partition: 0 MaxSeqNo: 25 sessionId: src_id|b21b3036-5065d262-af83b43b-74eba27a_0
2025-10-09T21:51:11.702626Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9b8ab08-7af1e263-8bf72472-3b2baf2c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b21b3036-5065d262-af83b43b-74eba27a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:11.704992Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b21b3036-5065d262-af83b43b-74eba27a_0] Write session: try to update token
2025-10-09T21:51:11.705008Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b21b3036-5065d262-af83b43b-74eba27a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:11.715632Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:11.715650Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:11.715652Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:11.715654Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:11.715663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342892266660276:11145]
2025-10-09T21:51:11.715667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342892266660276:11145]
2025-10-09T21:51:11.715671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342892266660276:11145]
2025-10-09T21:51:11.715714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:11.715718Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:11.715722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.715723Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:11.715724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:11.715727Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:11.715855Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:11.717554Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2062 sessionId: src_id|b21b3036-5065d262-af83b43b-74eba27a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:11.719881Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:11.780855Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342892628494525:5250]
2025-10-09T21:51:11.781124Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 2061, pipe: [1:7559342892628494525:5250]
2025-10-09T21:51:11.859651Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:11.860028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:11.860045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:11.860083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342892628494525:5250], now have 1 active actors on pipe
2025-10-09T21:51:11.860236Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:11.860353Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342892628494525:5250]
2025-10-09T21:51:11.860358Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:11.860379Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:11.860438Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2062 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b21b3036-5065d262-af83b43b-74eba27a_0
2025-10-09T21:51:11.860597Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2062 sessionId: src_id|b21b3036-5065d262-af83b43b-74eba27a_0 is DEAD
2025-10-09T21:51:11.860755Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b21b3036-5065d262-af83b43b-74eba27a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:11.860763Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b21b3036-5065d262-af83b43b-74eba27a_0] Write session will restart in 0.005964s
2025-10-09T21:51:11.860793Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b21b3036-5065d262-af83b43b-74eba27a_0] Write session: Do CDS request
2025-10-09T21:51:11.860800Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b21b3036-5065d262-af83b43b-74eba27a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:11.867442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b21b3036-5065d262-af83b43b-74eba27a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:11.867521Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:11.867534Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2063
2025-10-09T21:51:11.867705Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2063 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:11.867757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2063 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:11.867768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2063 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:11.867772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:11.868182Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2063 sessionId:  describe result for acl check
2025-10-09T21:51:11.868227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:11.868234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:11.868236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:11.868242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342892628494561:9808] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:11.868786Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342892628494561:9808] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:11.870828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342892628494561:9808] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:11.871746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:11.873256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:11.873431Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:11.873443Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:11.873454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:11.873503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:11.873538Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342892266660428:11149]
2025-10-09T21:51:11.873539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342892628494561:9808] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:11.873552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342892628494561:9808] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:11.873556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342892628494561:9808] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:11.873572Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2063 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:11.874014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:11.874334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:11.874388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:11.874623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:11.874662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.874667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.874667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.874668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.874669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.874669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.874670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.874670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.874671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.874674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.874681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.874685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.874689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.874692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.874696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.874700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.874703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.874706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.874713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.874721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:11.874727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.874732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:11.874737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.874741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:11.874743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.874746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:11.874748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.874751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:11.874754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.874757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:11.874760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.874763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.874765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.874768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.874771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.874774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.874792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:11.881004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:11.881957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:11.881967Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:11.881969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:11.881971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:11.881976Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2062 [2:7559342892266660428:11149]
2025-10-09T21:51:11.881984Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:11.881998Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:11.881999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:11.882001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:11.882003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:11.882015Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:11.882021Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:11.882026Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:11.882078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.882084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.882085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.882087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:11.882137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:11.882144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.882151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:11.882157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:11.882167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:11.882181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342892628494667:9808], now have 1 active actors on pipe
2025-10-09T21:51:11.882187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342892266660422:11149]
2025-10-09T21:51:11.882189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342892266660422:11149]
2025-10-09T21:51:11.882190Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342892266660422:11149]
2025-10-09T21:51:11.882196Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:11.882202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342892628494526:2432], now have 1 active actors on pipe
2025-10-09T21:51:11.882204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:11.882207Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:11.882208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:11.882208Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:11.882216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:11.882218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:11.882236Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342892266660422:11149]
2025-10-09T21:51:11.882248Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:11.882276Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:11.882282Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2062
2025-10-09T21:51:11.882282Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2062
2025-10-09T21:51:11.882357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:11.882365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:11.882391Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3b936dfc-cf5c3f23-c0f8a986-9f471aaa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:11.882403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:11.882407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:11.882408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:11.882415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:11.882423Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:11.882433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:11.882566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:11.882572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:11.882584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:11.882649Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2063 partition: 0 MaxSeqNo: 25 sessionId: src_id|3b936dfc-cf5c3f23-c0f8a986-9f471aaa_0
2025-10-09T21:51:11.882931Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b21b3036-5065d262-af83b43b-74eba27a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3b936dfc-cf5c3f23-c0f8a986-9f471aaa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:11.885288Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3b936dfc-cf5c3f23-c0f8a986-9f471aaa_0] Write session: try to update token
2025-10-09T21:51:11.885303Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3b936dfc-cf5c3f23-c0f8a986-9f471aaa_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:11.896417Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:11.896434Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:11.896437Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:11.896439Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:11.896449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342892266660422:11149]
2025-10-09T21:51:11.896453Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342892266660422:11149]
2025-10-09T21:51:11.896457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342892266660422:11149]
2025-10-09T21:51:11.896498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:11.896498Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:11.896504Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:11.896507Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:11.896511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:11.896514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:11.896642Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:11.897500Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2063 sessionId: src_id|3b936dfc-cf5c3f23-c0f8a986-9f471aaa_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:11.899769Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:11.960526Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342892628494681:5250]
2025-10-09T21:51:11.960817Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 2062, pipe: [1:7559342892628494681:5250]
2025-10-09T21:51:12.046812Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:12.047255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:12.047272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:12.047318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342892628494681:5250], now have 1 active actors on pipe
2025-10-09T21:51:12.047569Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342892628494681:5250]
2025-10-09T21:51:12.047573Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:12.047576Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:12.047600Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:12.047638Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2063 reason: error at write: Disconnected sessionId: src_id|3b936dfc-cf5c3f23-c0f8a986-9f471aaa_0
2025-10-09T21:51:12.047519Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:12.047821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2063 sessionId: src_id|3b936dfc-cf5c3f23-c0f8a986-9f471aaa_0 is DEAD
2025-10-09T21:51:12.047978Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b936dfc-cf5c3f23-c0f8a986-9f471aaa_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:12.047990Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b936dfc-cf5c3f23-c0f8a986-9f471aaa_0] Write session will restart in 0.005501s
2025-10-09T21:51:12.048014Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b936dfc-cf5c3f23-c0f8a986-9f471aaa_0] Write session: Do CDS request
2025-10-09T21:51:12.048019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b936dfc-cf5c3f23-c0f8a986-9f471aaa_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:12.054504Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3b936dfc-cf5c3f23-c0f8a986-9f471aaa_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:12.054638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:12.054653Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2064
2025-10-09T21:51:12.054802Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2064 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:12.054837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2064 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:12.054849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2064 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:12.054853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:12.055306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2064 sessionId:  describe result for acl check
2025-10-09T21:51:12.055338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:12.055340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:12.055342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:12.055347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342896923462022:9788] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:12.055910Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342896923462022:9788] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:12.058044Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342896923462022:9788] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:12.059792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:12.061630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:12.061813Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:12.061822Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:12.061834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:12.061869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342896923462022:9788] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:12.061882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342896923462022:9788] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:12.061886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:12.061885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342896923462022:9788] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:12.061891Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2064 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:12.061921Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342896561627905:11163]
2025-10-09T21:51:12.062512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:12.062820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:12.062861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:12.063172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:12.063216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.063222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.063223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.063224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.063225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.063226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.063227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.063229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.063230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.063235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.063246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.063250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.063254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.063258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.063263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.063266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.063270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.063274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.063280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.063288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.063295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.063300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.063304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.063310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.063319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.063324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.063329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.063334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.063340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.063347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.063352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.063362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.063366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.063371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.063376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.063381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.063410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:12.069472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:12.070619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:12.070629Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:12.070632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:12.070636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:12.070642Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2063 [2:7559342896561627905:11163]
2025-10-09T21:51:12.070654Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:12.070673Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:12.070676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:12.070679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:12.070682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:12.070698Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:12.070705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:12.070715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:12.070769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.070779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.070781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.070784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:12.070848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:12.070856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.070867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.070877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.070896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:12.070920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462003:2432], now have 1 active actors on pipe
2025-10-09T21:51:12.070927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462141:9788], now have 1 active actors on pipe
2025-10-09T21:51:12.070934Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342896561627900:11163]
2025-10-09T21:51:12.070936Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342896561627900:11163]
2025-10-09T21:51:12.070938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342896561627900:11163]
2025-10-09T21:51:12.070950Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:12.070969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:12.070977Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:12.070980Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:12.070981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:12.070987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:12.070990Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:12.071014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342896561627900:11163]
2025-10-09T21:51:12.071002Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2063
2025-10-09T21:51:12.071023Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:12.071002Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2063
2025-10-09T21:51:12.071052Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:12.071062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:12.071067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:12.071093Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1430fd19-7d1541d8-7e636d3e-6b9d029e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:12.071106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:12.071112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:12.071116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:12.071127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:12.071142Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:12.071159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:12.071309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:12.071321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:12.071347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:12.071440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2064 partition: 0 MaxSeqNo: 25 sessionId: src_id|1430fd19-7d1541d8-7e636d3e-6b9d029e_0
2025-10-09T21:51:12.071750Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b936dfc-cf5c3f23-c0f8a986-9f471aaa_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1430fd19-7d1541d8-7e636d3e-6b9d029e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:12.073919Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1430fd19-7d1541d8-7e636d3e-6b9d029e_0] Write session: try to update token
2025-10-09T21:51:12.073933Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1430fd19-7d1541d8-7e636d3e-6b9d029e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:12.084837Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:12.084854Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:12.084856Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:12.084858Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:12.084865Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342896561627900:11163]
2025-10-09T21:51:12.084869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342896561627900:11163]
2025-10-09T21:51:12.084872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342896561627900:11163]
2025-10-09T21:51:12.084917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:12.084918Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:12.084922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.084924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:12.084925Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:12.084930Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:12.085047Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:12.086217Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2064 sessionId: src_id|1430fd19-7d1541d8-7e636d3e-6b9d029e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:12.088514Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:12.147938Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342896923462153:5250]
2025-10-09T21:51:12.148208Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 2063, pipe: [1:7559342896923462153:5250]
2025-10-09T21:51:12.233741Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:12.234181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:12.234194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:12.234233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462153:5250], now have 1 active actors on pipe
2025-10-09T21:51:12.234403Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:12.234551Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342896923462153:5250]
2025-10-09T21:51:12.234574Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:12.234584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:12.234654Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2064 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1430fd19-7d1541d8-7e636d3e-6b9d029e_0
2025-10-09T21:51:12.234836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2064 sessionId: src_id|1430fd19-7d1541d8-7e636d3e-6b9d029e_0 is DEAD
2025-10-09T21:51:12.234983Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1430fd19-7d1541d8-7e636d3e-6b9d029e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:12.234995Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1430fd19-7d1541d8-7e636d3e-6b9d029e_0] Write session will restart in 0.008534s
2025-10-09T21:51:12.235019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1430fd19-7d1541d8-7e636d3e-6b9d029e_0] Write session: Do CDS request
2025-10-09T21:51:12.235026Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1430fd19-7d1541d8-7e636d3e-6b9d029e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:12.244476Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1430fd19-7d1541d8-7e636d3e-6b9d029e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:12.244612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:12.244628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2065
2025-10-09T21:51:12.244761Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2065 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:12.244792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2065 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:12.244803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2065 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:12.244807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:12.245168Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2065 sessionId:  describe result for acl check
2025-10-09T21:51:12.245197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:12.245198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:12.245199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:12.245204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342896923462262:9782] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:12.245767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342896923462262:9782] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:12.246222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:12.247705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342896923462262:9782] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:12.247981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:12.248154Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:12.248161Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:12.248170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:12.248218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:12.248247Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342896561628049:11165]
2025-10-09T21:51:12.248661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:12.248980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:12.249025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:12.249284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:12.249322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.249326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.249327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.249328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.249328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.249329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.249330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.249330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.249331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.249334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.249340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.249343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.249345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.249347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.249349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.249351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.249353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.249355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.249359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.249365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.249369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.249372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.249376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.249381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.249396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.249401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.249406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.249411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.249419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.249422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.249425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.249428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.249430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.249433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.249436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.249439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.249455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:12.249935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342896923462262:9782] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:12.249948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342896923462262:9782] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:12.249951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342896923462262:9782] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:12.249958Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2065 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:12.255565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:12.256557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:12.256572Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:12.256573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:12.256576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:12.256581Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2064 [2:7559342896561628049:11165]
2025-10-09T21:51:12.256592Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:12.256605Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:12.256607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:12.256609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:12.256611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:12.256623Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:12.256628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:12.256633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:12.256845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.256859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.256861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.256863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:12.256926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:12.256936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.256948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.256960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.256978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:12.256994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462158:2432], now have 1 active actors on pipe
2025-10-09T21:51:12.257000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462304:9782], now have 1 active actors on pipe
2025-10-09T21:51:12.257006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342896561628043:11165]
2025-10-09T21:51:12.257009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342896561628043:11165]
2025-10-09T21:51:12.257010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342896561628043:11165]
2025-10-09T21:51:12.257020Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:12.257031Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:12.257041Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:12.257044Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:12.257046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:12.257046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:12.257048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:12.257071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342896561628043:11165]
2025-10-09T21:51:12.257075Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:12.257083Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:12.257075Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2064
2025-10-09T21:51:12.257083Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2064
2025-10-09T21:51:12.257107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:12.257119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:12.257150Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c4c7f9ba-6cca75c3-d1548e84-de947596_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:12.257156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:12.257159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:12.257161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:12.257168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:12.257176Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:12.257188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:12.257300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:12.257305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:12.257315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:12.257401Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2065 partition: 0 MaxSeqNo: 25 sessionId: src_id|c4c7f9ba-6cca75c3-d1548e84-de947596_0
2025-10-09T21:51:12.257731Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1430fd19-7d1541d8-7e636d3e-6b9d029e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c4c7f9ba-6cca75c3-d1548e84-de947596_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:12.260076Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4c7f9ba-6cca75c3-d1548e84-de947596_0] Write session: try to update token
2025-10-09T21:51:12.260091Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4c7f9ba-6cca75c3-d1548e84-de947596_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:12.271237Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:12.271253Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:12.271256Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:12.271258Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:12.271268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342896561628043:11165]
2025-10-09T21:51:12.271272Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342896561628043:11165]
2025-10-09T21:51:12.271277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342896561628043:11165]
2025-10-09T21:51:12.271326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:12.271333Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:12.271335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.271338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:12.271340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:12.271345Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:12.271473Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:12.272721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2065 sessionId: src_id|c4c7f9ba-6cca75c3-d1548e84-de947596_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:12.275137Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:12.334855Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342896923462317:5250]
2025-10-09T21:51:12.335115Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 2064, pipe: [1:7559342896923462317:5250]
2025-10-09T21:51:12.420295Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:12.420537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:12.420554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:12.420597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462317:5250], now have 1 active actors on pipe
2025-10-09T21:51:12.420770Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:12.420808Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342896923462317:5250]
2025-10-09T21:51:12.420820Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:12.420824Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:12.420840Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:12.420888Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2065 reason: error at write: Disconnected sessionId: src_id|c4c7f9ba-6cca75c3-d1548e84-de947596_0
2025-10-09T21:51:12.421068Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2065 sessionId: src_id|c4c7f9ba-6cca75c3-d1548e84-de947596_0 is DEAD
2025-10-09T21:51:12.421216Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4c7f9ba-6cca75c3-d1548e84-de947596_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:12.421222Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4c7f9ba-6cca75c3-d1548e84-de947596_0] Write session will restart in 0.009690s
2025-10-09T21:51:12.421243Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4c7f9ba-6cca75c3-d1548e84-de947596_0] Write session: Do CDS request
2025-10-09T21:51:12.421248Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4c7f9ba-6cca75c3-d1548e84-de947596_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:12.432521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4c7f9ba-6cca75c3-d1548e84-de947596_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:12.432616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:12.432635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2066
2025-10-09T21:51:12.432754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2066 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:12.432786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2066 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:12.432796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2066 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:12.432800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:12.432954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:12.433244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2066 sessionId:  describe result for acl check
2025-10-09T21:51:12.433285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:12.433291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:12.433292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:12.433298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342896923462428:9777] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:12.433879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342896923462428:9777] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:12.435016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:12.435233Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:12.435244Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:12.435257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:12.435311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:12.435351Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342896561628194:11167]
2025-10-09T21:51:12.435898Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342896923462428:9777] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:12.435928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:12.436217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:12.436262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:12.436517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:12.436566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.436569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.436569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.436570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.436571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.436571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.436572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.436573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.436573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.436577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.436584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.436586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.436588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.436590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.436593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.436595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.436597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.436599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.436603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.436608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.436613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.436616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.436618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.436622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.436626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.436629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.436633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.436636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.436638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.436642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.436644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.436647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.436650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.436653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.436656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.436659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.436675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:12.441503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342896923462428:9777] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:12.441516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342896923462428:9777] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:12.441519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342896923462428:9777] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:12.441526Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2066 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:12.442984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:12.444259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:12.444275Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:12.444277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:12.444282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:12.444289Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2065 [2:7559342896561628194:11167]
2025-10-09T21:51:12.444304Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:12.444321Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:12.444324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:12.444327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:12.444329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:12.444347Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:12.444358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:12.444366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:12.444433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.444440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.444443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.444446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:12.444517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:12.444525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.444537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.444550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.444572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:12.444596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462467:9777], now have 1 active actors on pipe
2025-10-09T21:51:12.444604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462319:2432], now have 1 active actors on pipe
2025-10-09T21:51:12.444612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342896561628189:11167]
2025-10-09T21:51:12.444617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342896561628189:11167]
2025-10-09T21:51:12.444619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342896561628189:11167]
2025-10-09T21:51:12.444632Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:12.444649Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:12.444651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:12.444641Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2065
2025-10-09T21:51:12.444660Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:12.444662Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:12.444643Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2065
2025-10-09T21:51:12.444664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:12.444668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:12.444703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342896561628189:11167]
2025-10-09T21:51:12.444715Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:12.444721Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:12.444734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:12.444739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:12.444793Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d7ed6bdc-4623bf65-7459b509-cda95e2e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:12.444806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:12.444811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:12.444813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:12.444824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:12.444839Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:12.444860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:12.445124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:12.445136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:12.445151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:12.445228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2066 partition: 0 MaxSeqNo: 25 sessionId: src_id|d7ed6bdc-4623bf65-7459b509-cda95e2e_0
2025-10-09T21:51:12.445590Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4c7f9ba-6cca75c3-d1548e84-de947596_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d7ed6bdc-4623bf65-7459b509-cda95e2e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:12.448130Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ed6bdc-4623bf65-7459b509-cda95e2e_0] Write session: try to update token
2025-10-09T21:51:12.448149Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ed6bdc-4623bf65-7459b509-cda95e2e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:12.459176Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:12.459197Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:12.459200Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:12.459201Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:12.459211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342896561628189:11167]
2025-10-09T21:51:12.459216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342896561628189:11167]
2025-10-09T21:51:12.459220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342896561628189:11167]
2025-10-09T21:51:12.459264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:12.459272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.459275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:12.459276Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:12.459283Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:12.459289Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:12.459427Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:12.460937Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2066 sessionId: src_id|d7ed6bdc-4623bf65-7459b509-cda95e2e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:12.463409Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:12.521162Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342896923462474:5250]
2025-10-09T21:51:12.521417Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 2065, pipe: [1:7559342896923462474:5250]
2025-10-09T21:51:12.610883Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:12.611136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:12.611150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:12.611187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462474:5250], now have 1 active actors on pipe
2025-10-09T21:51:12.611354Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:12.611402Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342896923462474:5250]
2025-10-09T21:51:12.611431Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:12.611433Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:12.611513Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2066 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d7ed6bdc-4623bf65-7459b509-cda95e2e_0
2025-10-09T21:51:12.611696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2066 sessionId: src_id|d7ed6bdc-4623bf65-7459b509-cda95e2e_0 is DEAD
2025-10-09T21:51:12.611823Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ed6bdc-4623bf65-7459b509-cda95e2e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:12.611842Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ed6bdc-4623bf65-7459b509-cda95e2e_0] Write session will restart in 0.009878s
2025-10-09T21:51:12.611863Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ed6bdc-4623bf65-7459b509-cda95e2e_0] Write session: Do CDS request
2025-10-09T21:51:12.611868Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ed6bdc-4623bf65-7459b509-cda95e2e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:12.622458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ed6bdc-4623bf65-7459b509-cda95e2e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:12.622523Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:12.622537Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2067
2025-10-09T21:51:12.622685Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2067 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:12.622716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2067 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:12.622722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2067 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:12.622726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:12.623033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:12.623147Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2067 sessionId:  describe result for acl check
2025-10-09T21:51:12.623184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:12.623190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:12.623192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:12.623198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342896923462586:9772] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:12.623682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342896923462586:9772] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:12.624784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:12.624942Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:12.624949Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:12.624958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:12.624996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:12.625026Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342896561628340:11169]
2025-10-09T21:51:12.625419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:12.625448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342896923462586:9772] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:12.625672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:12.625702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:12.625902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:12.625938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.625944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.625946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.625947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.625948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.625949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.625949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.625950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.625951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.625956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.625965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.625969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.625973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.625976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.625980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.625984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.625987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.625990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.625996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.626004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.626010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.626016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.626023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.626042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.626045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.626050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.626055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.626063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.626067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.626072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.626081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.626095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.626101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.626104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.626106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.626109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.626125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:12.630704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342896923462586:9772] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:12.630720Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342896923462586:9772] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:12.630724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342896923462586:9772] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:12.630740Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2067 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:12.632070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:12.633104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:12.633118Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:12.633121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:12.633125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:12.633131Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2066 [2:7559342896561628340:11169]
2025-10-09T21:51:12.633145Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:12.633161Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:12.633171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:12.633174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:12.633176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:12.633192Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:12.633202Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:12.633209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:12.633261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.633268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.633270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.633273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:12.633340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:12.633349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.633361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.633372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.633405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:12.633440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462625:9772], now have 1 active actors on pipe
2025-10-09T21:51:12.633452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462477:2432], now have 1 active actors on pipe
2025-10-09T21:51:12.633459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342896561628335:11169]
2025-10-09T21:51:12.633461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342896561628335:11169]
2025-10-09T21:51:12.633464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342896561628335:11169]
2025-10-09T21:51:12.633476Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:12.633492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:12.633496Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:12.633500Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:12.633502Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:12.633506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:12.633510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:12.633488Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2066
2025-10-09T21:51:12.633490Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2066
2025-10-09T21:51:12.633536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342896561628335:11169]
2025-10-09T21:51:12.633545Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:12.633547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:12.633561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:12.633571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:12.633636Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5de55270-5fb90907-d9db26d7-a53a11f7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:12.633649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:12.633653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:12.633655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:12.633665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:12.633675Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:12.633699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:12.633920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:12.633929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:12.633947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:12.634093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2067 partition: 0 MaxSeqNo: 25 sessionId: src_id|5de55270-5fb90907-d9db26d7-a53a11f7_0
2025-10-09T21:51:12.634451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ed6bdc-4623bf65-7459b509-cda95e2e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5de55270-5fb90907-d9db26d7-a53a11f7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:12.636992Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5de55270-5fb90907-d9db26d7-a53a11f7_0] Write session: try to update token
2025-10-09T21:51:12.637012Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5de55270-5fb90907-d9db26d7-a53a11f7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:12.648430Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:12.648448Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:12.648451Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:12.648453Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:12.648462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342896561628335:11169]
2025-10-09T21:51:12.648467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342896561628335:11169]
2025-10-09T21:51:12.648472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342896561628335:11169]
2025-10-09T21:51:12.648517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:12.648524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.648527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:12.648530Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:12.648537Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:12.648541Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:12.648661Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:12.650940Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2067 sessionId: src_id|5de55270-5fb90907-d9db26d7-a53a11f7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:12.653240Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:12.711676Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342896923462632:5250]
2025-10-09T21:51:12.711916Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 2066, pipe: [1:7559342896923462632:5250]
2025-10-09T21:51:12.794881Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:12.795093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:12.795109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:12.795152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462632:5250], now have 1 active actors on pipe
2025-10-09T21:51:12.795311Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:12.795374Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342896923462632:5250]
2025-10-09T21:51:12.795391Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:12.795394Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:12.795415Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:12.795467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2067 reason: error at write: Disconnected sessionId: src_id|5de55270-5fb90907-d9db26d7-a53a11f7_0
2025-10-09T21:51:12.795640Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2067 sessionId: src_id|5de55270-5fb90907-d9db26d7-a53a11f7_0 is DEAD
2025-10-09T21:51:12.795801Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5de55270-5fb90907-d9db26d7-a53a11f7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:12.795810Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5de55270-5fb90907-d9db26d7-a53a11f7_0] Write session will restart in 0.007902s
2025-10-09T21:51:12.795853Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5de55270-5fb90907-d9db26d7-a53a11f7_0] Write session: Do CDS request
2025-10-09T21:51:12.795859Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5de55270-5fb90907-d9db26d7-a53a11f7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:12.804512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5de55270-5fb90907-d9db26d7-a53a11f7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:12.804592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:12.804619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2068
2025-10-09T21:51:12.804730Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2068 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:12.804772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2068 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:12.804777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2068 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:12.804780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:12.805221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2068 sessionId:  describe result for acl check
2025-10-09T21:51:12.805257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:12.805263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:12.805264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:12.805268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342896923462743:9760] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:12.805643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:12.805863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342896923462743:9760] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:12.807450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:12.807610Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:12.807620Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:12.807630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:12.807671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:12.807709Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342896561628492:11173]
2025-10-09T21:51:12.807717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342896923462743:9760] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:12.808099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:12.808361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:12.808416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:12.808670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:12.808722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.808728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.808729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.808730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.808732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.808733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.808734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.808735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.808736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.808741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.808747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.808750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.808752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.808754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.808756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.808758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.808760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.808762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.808767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.808772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.808776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.808780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.808782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.808785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.808788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.808791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.808793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.808797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.808799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.808802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.808805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.808809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.808811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.808814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.808817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.808820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.808837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:12.813782Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342896923462743:9760] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:12.813797Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342896923462743:9760] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:12.813801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342896923462743:9760] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:12.813809Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2068 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:12.815087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:12.816285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:12.816299Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:12.816300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:12.816303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:12.816307Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2067 [2:7559342896561628492:11173]
2025-10-09T21:51:12.816318Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:12.816331Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:12.816333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:12.816334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:12.816336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:12.816347Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:12.816352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:12.816356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:12.816396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.816401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.816402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.816404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:12.816460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:12.816466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.816473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.816479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.816488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:12.816503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342896561628486:11173]
2025-10-09T21:51:12.816507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342896561628486:11173]
2025-10-09T21:51:12.816508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342896561628486:11173]
2025-10-09T21:51:12.816514Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:12.816525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462784:9760], now have 1 active actors on pipe
2025-10-09T21:51:12.816525Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:12.816528Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:12.816530Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:12.816534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:12.816543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:12.816546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:12.816551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462635:2432], now have 1 active actors on pipe
2025-10-09T21:51:12.816573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342896561628486:11173]
2025-10-09T21:51:12.816584Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:12.816597Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:12.816593Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2067
2025-10-09T21:51:12.816607Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2067
2025-10-09T21:51:12.816656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:12.816663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:12.816692Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bafb67b7-7521b571-f6c3a9f7-4758516a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:12.816704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:12.816707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:12.816709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:12.816718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:12.816727Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:12.816743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:12.816851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:12.816856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:12.816866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:12.816964Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2068 partition: 0 MaxSeqNo: 25 sessionId: src_id|bafb67b7-7521b571-f6c3a9f7-4758516a_0
2025-10-09T21:51:12.817271Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5de55270-5fb90907-d9db26d7-a53a11f7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bafb67b7-7521b571-f6c3a9f7-4758516a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:12.819763Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bafb67b7-7521b571-f6c3a9f7-4758516a_0] Write session: try to update token
2025-10-09T21:51:12.819779Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bafb67b7-7521b571-f6c3a9f7-4758516a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:12.832111Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:12.832129Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:12.832133Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:12.832135Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:12.832144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342896561628486:11173]
2025-10-09T21:51:12.832149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342896561628486:11173]
2025-10-09T21:51:12.832153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342896561628486:11173]
2025-10-09T21:51:12.832200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:12.832204Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:12.832210Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:12.832211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.832213Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:12.832214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:12.832347Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:12.833065Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2068 sessionId: src_id|bafb67b7-7521b571-f6c3a9f7-4758516a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:12.835382Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:12.895490Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342896923462793:5250]
2025-10-09T21:51:12.895790Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 2067, pipe: [1:7559342896923462793:5250]
2025-10-09T21:51:12.981715Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:12.981968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:12.981981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:12.982021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462793:5250], now have 1 active actors on pipe
2025-10-09T21:51:12.982256Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:12.982256Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:12.982274Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:12.982281Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342896923462793:5250]
2025-10-09T21:51:12.982300Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:12.982326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2068 reason: error at write: Disconnected sessionId: src_id|bafb67b7-7521b571-f6c3a9f7-4758516a_0
2025-10-09T21:51:12.982472Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2068 sessionId: src_id|bafb67b7-7521b571-f6c3a9f7-4758516a_0 is DEAD
2025-10-09T21:51:12.982656Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bafb67b7-7521b571-f6c3a9f7-4758516a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:12.982665Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bafb67b7-7521b571-f6c3a9f7-4758516a_0] Write session will restart in 0.006353s
2025-10-09T21:51:12.982693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bafb67b7-7521b571-f6c3a9f7-4758516a_0] Write session: Do CDS request
2025-10-09T21:51:12.982699Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bafb67b7-7521b571-f6c3a9f7-4758516a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:12.990456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bafb67b7-7521b571-f6c3a9f7-4758516a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:12.990527Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:12.990542Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2069
2025-10-09T21:51:12.990667Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2069 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:12.990715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2069 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:12.990727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2069 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:12.990730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:12.991056Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2069 sessionId:  describe result for acl check
2025-10-09T21:51:12.991089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:12.991095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:12.991097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:12.991103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342896923462841:9756] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:12.991645Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342896923462841:9756] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:12.993861Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342896923462841:9756] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:12.994582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:12.996373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:12.996556Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:12.996566Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:12.996577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:12.996627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:12.996670Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342896561628639:11175]
2025-10-09T21:51:12.996731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342896923462841:9756] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:12.996742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342896923462841:9756] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:12.996746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342896923462841:9756] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:12.996753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2069 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:12.997213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:12.997569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:12.997617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:12.997935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:12.997988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.997994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.997995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.997996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.997997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.997998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.997999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.998000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.998001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.998006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.998016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.998019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.998021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.998023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.998025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.998060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.998062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.998064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.998069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.998074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:12.998078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.998082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:12.998084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.998087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:12.998091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.998094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:12.998097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.998099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:12.998102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.998104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:12.998107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.998110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:12.998112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.998115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:12.998118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.998122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:12.998137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:13.004396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:13.005469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:13.005483Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:13.005485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:13.005489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:13.005495Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2068 [2:7559342896561628639:11175]
2025-10-09T21:51:13.005505Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:13.005520Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:13.005522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:13.005524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:13.005526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:13.005539Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:13.005544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:13.005549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:13.005606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.005614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.005616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.005619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:13.005818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:13.005822Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2068
2025-10-09T21:51:13.005830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.005841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.005837Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2068
2025-10-09T21:51:13.005852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.005869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:13.005894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462941:9756], now have 1 active actors on pipe
2025-10-09T21:51:13.005902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342896923462798:2432], now have 1 active actors on pipe
2025-10-09T21:51:13.005921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.005930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.005947Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342896561628634:11175]
2025-10-09T21:51:13.005953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342896561628634:11175]
2025-10-09T21:51:13.005955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342896561628634:11175]
2025-10-09T21:51:13.005969Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:13.005979Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:13.005981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:13.005988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:13.005996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:13.006005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:13.006013Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:13.006050Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b9f5d93b-31694e06-bf2d0302-b366b4c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:13.006064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:13.006068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:13.006071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:13.006082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:13.006093Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:13.006121Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342896561628634:11175]
2025-10-09T21:51:13.006131Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:13.006139Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:13.006140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:13.006273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.006284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.006300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:13.006419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2069 partition: 0 MaxSeqNo: 25 sessionId: src_id|b9f5d93b-31694e06-bf2d0302-b366b4c_0
2025-10-09T21:51:13.006761Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bafb67b7-7521b571-f6c3a9f7-4758516a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b9f5d93b-31694e06-bf2d0302-b366b4c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:13.009172Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b9f5d93b-31694e06-bf2d0302-b366b4c_0] Write session: try to update token
2025-10-09T21:51:13.009190Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b9f5d93b-31694e06-bf2d0302-b366b4c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:13.022192Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2069 sessionId: src_id|b9f5d93b-31694e06-bf2d0302-b366b4c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:13.023369Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:13.023388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:13.023397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342896561628634:11175]
2025-10-09T21:51:13.023445Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:13.023451Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:13.023607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:13.023749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:13.023882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:13.024024Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:13.024213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:13.024255Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:13.024657Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:13.024877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.024896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.024929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:13.025025Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:13.031044Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:13.031064Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:13.031067Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:13.031069Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:13.031083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342896561628634:11175]
2025-10-09T21:51:13.031087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342896561628634:11175]
2025-10-09T21:51:13.031091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342896561628634:11175]
2025-10-09T21:51:13.031131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:13.031145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.031145Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:13.031147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:13.031152Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:13.031157Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:13.031284Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:13.082353Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342901218430273:5250]
2025-10-09T21:51:13.082592Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 2068, pipe: [1:7559342901218430273:5250]
2025-10-09T21:51:13.177987Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:13.184815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.184841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.184983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.185037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.185094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.185157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.185411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.185426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.185502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.185568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.185625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.185846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.185885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.185887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.185944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.186006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.186077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.186133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.186172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.186180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.186271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.186340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.186404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.186460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.186506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.186514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.186583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.186644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.186701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.186754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.186809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.186815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.186869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.186926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.186980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.187035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.187074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.187079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.187131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.187182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.187235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.187288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.187326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.187331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.187383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.187436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.187501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.187566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.187607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.187620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.187684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.187740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.187792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.187849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.187889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.187897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.187952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.188002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.188055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.188106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.188145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.188146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.188187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342901218430273:5250], now have 1 active actors on pipe
2025-10-09T21:51:13.188429Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:13.188442Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:13.188444Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342901218430273:5250]
2025-10-09T21:51:13.188469Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2069 reason: error at write: Disconnected sessionId: src_id|b9f5d93b-31694e06-bf2d0302-b366b4c_0
2025-10-09T21:51:13.188471Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:13.188637Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:13.188961Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2069 sessionId: src_id|b9f5d93b-31694e06-bf2d0302-b366b4c_0 is DEAD
2025-10-09T21:51:13.189106Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9f5d93b-31694e06-bf2d0302-b366b4c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:13.189116Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9f5d93b-31694e06-bf2d0302-b366b4c_0] Write session will restart in 0.008823s
2025-10-09T21:51:13.189142Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9f5d93b-31694e06-bf2d0302-b366b4c_0] Write session: Do CDS request
2025-10-09T21:51:13.189148Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9f5d93b-31694e06-bf2d0302-b366b4c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:13.199585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b9f5d93b-31694e06-bf2d0302-b366b4c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:13.199644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:13.199664Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2070
2025-10-09T21:51:13.199805Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2070 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:13.199839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2070 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:13.199849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2070 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:13.199854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:13.200276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2070 sessionId:  describe result for acl check
2025-10-09T21:51:13.200313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:13.200320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:13.200322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:13.200328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342901218430379:9144] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:13.200864Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342901218430379:9144] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:13.202054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:13.203195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342901218430379:9144] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:13.203867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:13.204059Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:13.204068Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:13.204080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:13.204138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:13.204184Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342900856596123:11189]
2025-10-09T21:51:13.204762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:13.205045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342901218430379:9144] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:13.205061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:13.205072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342901218430379:9144] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:13.205076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342901218430379:9144] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:13.205085Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2070 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:13.205108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:13.205426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:13.205495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.205503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.205504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.205505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.205506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.205507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.205508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.205510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.205511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.205516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.205526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.205530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.205534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.205538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.205542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.205546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.205549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.205553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.205560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.205569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.205575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.205580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.205583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.205589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.205595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.205601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.205605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.205616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.205620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.205626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.205631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.205636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.205645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.205649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.205654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.205659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.205690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:13.212046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:13.213106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:13.213123Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:13.213126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:13.213130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:13.213137Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2069 [2:7559342900856596123:11189]
2025-10-09T21:51:13.213151Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:13.213169Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:13.213177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:13.213180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:13.213183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:13.213199Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:13.213208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:13.213215Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:13.213268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.213275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.213277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.213280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:13.213345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:13.213352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.213363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.213379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.213410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:13.213436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342901218430427:9144], now have 1 active actors on pipe
2025-10-09T21:51:13.213444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342901218430277:2432], now have 1 active actors on pipe
2025-10-09T21:51:13.213451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342900856596118:11189]
2025-10-09T21:51:13.213453Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342900856596118:11189]
2025-10-09T21:51:13.213455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342900856596118:11189]
2025-10-09T21:51:13.213469Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:13.213489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:13.213497Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:13.213499Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:13.213501Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:13.213515Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:13.213522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:13.213487Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2069
2025-10-09T21:51:13.213499Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2069
2025-10-09T21:51:13.213553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342900856596118:11189]
2025-10-09T21:51:13.213565Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:13.213568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:13.213585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.213595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.213643Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2ea69444-7d0960d5-bf564afe-179f003e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:13.213654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:13.213660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:13.213663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:13.213674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:13.213688Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:13.213708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:13.213820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.213829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.213843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:13.213934Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2070 partition: 0 MaxSeqNo: 25 sessionId: src_id|2ea69444-7d0960d5-bf564afe-179f003e_0
2025-10-09T21:51:13.214293Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b9f5d93b-31694e06-bf2d0302-b366b4c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2ea69444-7d0960d5-bf564afe-179f003e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:13.216664Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ea69444-7d0960d5-bf564afe-179f003e_0] Write session: try to update token
2025-10-09T21:51:13.216682Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ea69444-7d0960d5-bf564afe-179f003e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:13.227799Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:13.227817Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:13.227819Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:13.227821Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:13.227831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342900856596118:11189]
2025-10-09T21:51:13.227836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342900856596118:11189]
2025-10-09T21:51:13.227839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342900856596118:11189]
2025-10-09T21:51:13.227883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:13.227891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.227890Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:13.227897Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:13.227899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:13.227901Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:13.228027Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:13.229133Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2070 sessionId: src_id|2ea69444-7d0960d5-bf564afe-179f003e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:13.231433Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:13.288699Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342901218430445:5250]
2025-10-09T21:51:13.288945Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 2069, pipe: [1:7559342901218430445:5250]
2025-10-09T21:51:13.374728Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:13.374939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.374951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.374987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342901218430445:5250], now have 1 active actors on pipe
2025-10-09T21:51:13.375178Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:13.375189Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:13.375215Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342901218430445:5250]
2025-10-09T21:51:13.375239Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:13.375282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2070 reason: error at write: Disconnected sessionId: src_id|2ea69444-7d0960d5-bf564afe-179f003e_0
2025-10-09T21:51:13.375431Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2070 sessionId: src_id|2ea69444-7d0960d5-bf564afe-179f003e_0 is DEAD
2025-10-09T21:51:13.375570Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:13.375583Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ea69444-7d0960d5-bf564afe-179f003e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:13.375621Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ea69444-7d0960d5-bf564afe-179f003e_0] Write session will restart in 0.007884s
2025-10-09T21:51:13.375650Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ea69444-7d0960d5-bf564afe-179f003e_0] Write session: Do CDS request
2025-10-09T21:51:13.375656Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ea69444-7d0960d5-bf564afe-179f003e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:13.384518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ea69444-7d0960d5-bf564afe-179f003e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:13.384596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:13.384607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2071
2025-10-09T21:51:13.384706Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2071 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:13.384740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2071 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:13.384745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2071 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:13.384749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:13.385203Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2071 sessionId:  describe result for acl check
2025-10-09T21:51:13.385238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:13.385240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:13.385243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:13.385249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342901218430561:9739] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:13.385731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342901218430561:9739] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:13.386726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:13.387996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342901218430561:9739] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:13.388681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:13.388877Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:13.388888Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:13.388899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:13.388950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:13.388996Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342900856596273:11191]
2025-10-09T21:51:13.389553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:13.389865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:13.389910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:13.390148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342901218430561:9739] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:13.390162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342901218430561:9739] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:13.390166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342901218430561:9739] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:13.390174Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2071 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:13.390231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:13.390287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.390293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.390294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.390295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.390296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.390298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.390299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.390300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.390301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.390306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.390317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.390321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.390325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.390329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.390333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.390337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.390341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.390345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.390351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.390359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.390365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.390371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.390375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.390385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.390390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.390395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.390404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.390409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.390414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.390418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.390426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.390430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.390432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.390435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.390439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.390442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.390458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:13.396766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:13.397857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:13.397873Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:13.397876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:13.397880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:13.397887Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2070 [2:7559342900856596273:11191]
2025-10-09T21:51:13.397900Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:13.397920Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:13.397928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:13.397932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:13.397934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:13.397954Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:13.397966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:13.397973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:13.398046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.398055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.398057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.398060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:13.398129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:13.398139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.398152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.398164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.398181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:13.398203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342901218430602:9739], now have 1 active actors on pipe
2025-10-09T21:51:13.398214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342901218430452:2432], now have 1 active actors on pipe
2025-10-09T21:51:13.398222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342900856596268:11191]
2025-10-09T21:51:13.398226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342900856596268:11191]
2025-10-09T21:51:13.398228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342900856596268:11191]
2025-10-09T21:51:13.398242Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:13.398255Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:13.398256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:13.398259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:13.398261Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:13.398271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:13.398274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:13.398256Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2070
2025-10-09T21:51:13.398273Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2070
2025-10-09T21:51:13.398306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342900856596268:11191]
2025-10-09T21:51:13.398315Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:13.398317Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:13.398335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.398344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.398391Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5c74a852-d387945f-c3185cd-6373df5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:13.398403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:13.398407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:13.398409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:13.398419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:13.398428Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:13.398453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:13.398550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.398558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.398571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:13.398669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2071 partition: 0 MaxSeqNo: 25 sessionId: src_id|5c74a852-d387945f-c3185cd-6373df5_0
2025-10-09T21:51:13.399041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ea69444-7d0960d5-bf564afe-179f003e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5c74a852-d387945f-c3185cd-6373df5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:13.401450Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c74a852-d387945f-c3185cd-6373df5_0] Write session: try to update token
2025-10-09T21:51:13.401470Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c74a852-d387945f-c3185cd-6373df5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:13.414087Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:13.414113Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:13.414117Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:13.414121Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:13.414134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342900856596268:11191]
2025-10-09T21:51:13.414142Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342900856596268:11191]
2025-10-09T21:51:13.414148Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342900856596268:11191]
2025-10-09T21:51:13.414205Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:13.414213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:13.414220Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:13.414222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.414224Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:13.414225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:13.414498Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:13.415233Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2071 sessionId: src_id|5c74a852-d387945f-c3185cd-6373df5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:13.417551Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:13.475566Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342901218430611:5250]
2025-10-09T21:51:13.475795Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 2070, pipe: [1:7559342901218430611:5250]
2025-10-09T21:51:13.559073Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:13.559501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.559516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.559556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342901218430611:5250], now have 1 active actors on pipe
2025-10-09T21:51:13.559738Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:13.559807Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:13.559820Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:13.559827Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342901218430611:5250]
2025-10-09T21:51:13.559845Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:13.559877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2071 reason: error at write: Disconnected sessionId: src_id|5c74a852-d387945f-c3185cd-6373df5_0
2025-10-09T21:51:13.560045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2071 sessionId: src_id|5c74a852-d387945f-c3185cd-6373df5_0 is DEAD
2025-10-09T21:51:13.560212Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c74a852-d387945f-c3185cd-6373df5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:13.560222Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c74a852-d387945f-c3185cd-6373df5_0] Write session will restart in 0.008591s
2025-10-09T21:51:13.560247Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c74a852-d387945f-c3185cd-6373df5_0] Write session: Do CDS request
2025-10-09T21:51:13.560254Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c74a852-d387945f-c3185cd-6373df5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:13.569528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c74a852-d387945f-c3185cd-6373df5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:13.569611Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:13.569622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2072
2025-10-09T21:51:13.569736Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2072 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:13.569760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2072 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:13.569766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2072 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:13.569769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:13.570182Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2072 sessionId:  describe result for acl check
2025-10-09T21:51:13.570219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:13.570221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:13.570223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:13.570229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342901218430721:9731] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:13.570730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342901218430721:9731] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:13.571666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:13.572710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342901218430721:9731] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:13.573367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:13.573536Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:13.573544Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:13.573553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:13.573594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:13.573620Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342900856596424:11193]
2025-10-09T21:51:13.574087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:13.574394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:13.574449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:13.574753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:13.574807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.574813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.574814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.574815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.574816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.574817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.574818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.574820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.574821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.574825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.574835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.574839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.574842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.574846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.574850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.574854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.574858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.574861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.574866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.574871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.574876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.574879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.574881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.574884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.574887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.574890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.574892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.574895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.574897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.574900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.574903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.574906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.574908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.574911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.574913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.574917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.574933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:13.574965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342901218430721:9731] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:13.574977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342901218430721:9731] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:13.574981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342901218430721:9731] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:13.574987Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2072 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:13.581125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:13.582388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:13.582404Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:13.582407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:13.582410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:13.582417Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2071 [2:7559342900856596424:11193]
2025-10-09T21:51:13.582428Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:13.582441Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:13.582443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:13.582445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:13.582446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:13.582459Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:13.582462Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:13.582467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:13.582507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.582513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.582514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.582516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:13.582561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:13.582566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.582574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.582580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.582588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:13.582604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342900856596419:11193]
2025-10-09T21:51:13.582609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342900856596419:11193]
2025-10-09T21:51:13.582611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342900856596419:11193]
2025-10-09T21:51:13.582617Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:13.582628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342901218430614:2432], now have 1 active actors on pipe
2025-10-09T21:51:13.582628Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:13.582631Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:13.582633Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:13.582635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342901218430764:9731], now have 1 active actors on pipe
2025-10-09T21:51:13.582643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:13.582650Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:13.582652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:13.582674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342900856596419:11193]
2025-10-09T21:51:13.582693Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2071
2025-10-09T21:51:13.582687Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:13.582717Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2071
2025-10-09T21:51:13.582693Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:13.582779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.582789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.582813Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c9862e57-f71990a0-35a803f2-30b0a000_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:13.582824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:13.582827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:13.582829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:13.582836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:13.582844Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:13.582854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:13.583000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.583012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.583032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:13.583151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2072 partition: 0 MaxSeqNo: 25 sessionId: src_id|c9862e57-f71990a0-35a803f2-30b0a000_0
2025-10-09T21:51:13.583485Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c74a852-d387945f-c3185cd-6373df5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c9862e57-f71990a0-35a803f2-30b0a000_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:13.586122Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c9862e57-f71990a0-35a803f2-30b0a000_0] Write session: try to update token
2025-10-09T21:51:13.586139Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c9862e57-f71990a0-35a803f2-30b0a000_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:13.593527Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:13.593543Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:13.593553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342900856596419:11193]
2025-10-09T21:51:13.593595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:13.593600Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:13.593752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:13.593888Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:13.594021Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:13.594168Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:13.594528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:13.594568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:13.598599Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2072 sessionId: src_id|c9862e57-f71990a0-35a803f2-30b0a000_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:13.600819Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:13.601093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.601112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.601147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:13.601240Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:13.605434Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:13.605448Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:13.605450Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:13.605451Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:13.605460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342900856596419:11193]
2025-10-09T21:51:13.605462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342900856596419:11193]
2025-10-09T21:51:13.605464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342900856596419:11193]
2025-10-09T21:51:13.605497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:13.605503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.605505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:13.605544Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:13.605571Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:13.605577Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:13.605624Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:13.659905Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342901218430774:5250]
2025-10-09T21:51:13.660235Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 2071, pipe: [1:7559342901218430774:5250]
2025-10-09T21:51:13.749045Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:13.755615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.755641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.755780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.755832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.755888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.755942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.756204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.756215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.756274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.756336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.756396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.756457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.756498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.756508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.756563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.756611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.756665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.756715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.756754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.756759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.756813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.756867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.756934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.756995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.757031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.757036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.757090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.757177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.757235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.757292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.757329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.757334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.757399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.757450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.757517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.757577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.757615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.757620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.757672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.757720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.757771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.757826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.757863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.757868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.757918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.757967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.758016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.758088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.758125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.758130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.758180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.758226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.758276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.758326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.758363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.758368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.758420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:13.758468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:13.758519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:13.758570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:13.758605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:13.758610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:13.758647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342901218430774:5250], now have 1 active actors on pipe
2025-10-09T21:51:13.758893Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342901218430774:5250]
2025-10-09T21:51:13.758914Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:13.758933Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:13.758935Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:13.758962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2072 reason: error at write: Disconnected sessionId: src_id|c9862e57-f71990a0-35a803f2-30b0a000_0
2025-10-09T21:51:13.759157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2072 sessionId: src_id|c9862e57-f71990a0-35a803f2-30b0a000_0 is DEAD
2025-10-09T21:51:13.759184Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:13.759323Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9862e57-f71990a0-35a803f2-30b0a000_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:13.759334Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9862e57-f71990a0-35a803f2-30b0a000_0] Write session will restart in 0.007433s
2025-10-09T21:51:13.759357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9862e57-f71990a0-35a803f2-30b0a000_0] Write session: Do CDS request
2025-10-09T21:51:13.759373Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9862e57-f71990a0-35a803f2-30b0a000_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:13.767477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c9862e57-f71990a0-35a803f2-30b0a000_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:13.767533Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:13.767547Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2073
2025-10-09T21:51:13.767678Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2073 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:13.767717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2073 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:13.767723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2073 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:13.767728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:13.768154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2073 sessionId:  describe result for acl check
2025-10-09T21:51:13.768187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:13.768193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:13.768195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:13.768200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342901218430804:9723] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:13.768767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342901218430804:9723] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:13.772090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342901218430804:9723] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:13.773311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:13.775176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:13.775368Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:13.775379Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:13.775391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:13.775454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:13.775458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342901218430804:9723] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:13.775471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342901218430804:9723] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:13.775474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342901218430804:9723] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:13.775493Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342900856596576:11195]
2025-10-09T21:51:13.775482Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2073 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:13.776052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:13.776345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:13.776393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:13.776717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:13.776773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.776780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.776781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.776782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.776783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.776785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.776786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.776787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.776788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.776793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.776803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.776807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.776811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.776816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.776820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.776824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.776828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.776832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.776839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.776846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.776853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.776859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.776863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.776869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.776873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.776878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.776888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.776893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.776898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.776903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.776913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.776918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.776922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.776932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.776936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.776942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.776969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:13.783505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:13.784506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:13.784521Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:13.784523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:13.784526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:13.784530Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2072 [2:7559342900856596576:11195]
2025-10-09T21:51:13.784540Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:13.784556Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:13.784559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:13.784562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:13.784564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:13.784580Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:13.784586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:13.784591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:13.784632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.784638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.784639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.784641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:13.784693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:13.784698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.784705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.784711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.784721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:13.784735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342900856596571:11195]
2025-10-09T21:51:13.784740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342900856596571:11195]
2025-10-09T21:51:13.784742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342900856596571:11195]
2025-10-09T21:51:13.784748Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:13.784757Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:13.784759Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:13.784760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342901218430927:9723], now have 1 active actors on pipe
2025-10-09T21:51:13.784761Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:13.784766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:13.784773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:13.784776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:13.784780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342901218430777:2432], now have 1 active actors on pipe
2025-10-09T21:51:13.784801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342900856596571:11195]
2025-10-09T21:51:13.784816Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:13.784837Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:13.784841Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2072
2025-10-09T21:51:13.784876Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2072
2025-10-09T21:51:13.784920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.784937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.784974Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|da849d0d-1fc2778-eaeb0d55-653c5269_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:13.784988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:13.784998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:13.785001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:13.785012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:13.785028Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:13.785049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:13.785230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.785239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.785256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:13.785378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2073 partition: 0 MaxSeqNo: 25 sessionId: src_id|da849d0d-1fc2778-eaeb0d55-653c5269_0
2025-10-09T21:51:13.785680Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9862e57-f71990a0-35a803f2-30b0a000_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|da849d0d-1fc2778-eaeb0d55-653c5269_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:13.788161Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da849d0d-1fc2778-eaeb0d55-653c5269_0] Write session: try to update token
2025-10-09T21:51:13.788176Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da849d0d-1fc2778-eaeb0d55-653c5269_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:13.799352Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:13.799371Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:13.799373Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:13.799375Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:13.799382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342900856596571:11195]
2025-10-09T21:51:13.799385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342900856596571:11195]
2025-10-09T21:51:13.799388Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342900856596571:11195]
2025-10-09T21:51:13.799433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:13.799441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.799440Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:13.799444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:13.799445Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:13.799448Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:13.799565Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:13.800267Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2073 sessionId: src_id|da849d0d-1fc2778-eaeb0d55-653c5269_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:13.802874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:13.859066Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342901218430938:5250]
2025-10-09T21:51:13.859403Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 2072, pipe: [1:7559342901218430938:5250]
2025-10-09T21:51:13.949116Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:13.949526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.949539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.949579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342901218430938:5250], now have 1 active actors on pipe
2025-10-09T21:51:13.949747Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:13.949807Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:13.949809Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342901218430938:5250]
2025-10-09T21:51:13.949822Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:13.949834Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:13.949886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2073 reason: error at write: Disconnected sessionId: src_id|da849d0d-1fc2778-eaeb0d55-653c5269_0
2025-10-09T21:51:13.950099Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2073 sessionId: src_id|da849d0d-1fc2778-eaeb0d55-653c5269_0 is DEAD
2025-10-09T21:51:13.950242Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da849d0d-1fc2778-eaeb0d55-653c5269_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:13.950252Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da849d0d-1fc2778-eaeb0d55-653c5269_0] Write session will restart in 0.006087s
2025-10-09T21:51:13.950281Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da849d0d-1fc2778-eaeb0d55-653c5269_0] Write session: Do CDS request
2025-10-09T21:51:13.950286Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da849d0d-1fc2778-eaeb0d55-653c5269_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:13.957521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|da849d0d-1fc2778-eaeb0d55-653c5269_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:13.957602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:13.957616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2074
2025-10-09T21:51:13.957745Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2074 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:13.957778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2074 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:13.957787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2074 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:13.957790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:13.958281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2074 sessionId:  describe result for acl check
2025-10-09T21:51:13.958316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:13.958321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:13.958324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:13.958331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342901218430970:9722] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:13.958775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342901218430970:9722] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:13.961048Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342901218430970:9722] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:13.962611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:13.964189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:13.964359Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:13.964369Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:13.964381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:13.964426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:13.964465Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342900856596733:11199]
2025-10-09T21:51:13.964501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342901218430970:9722] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:13.964513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342901218430970:9722] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:13.964516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342901218430970:9722] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:13.964524Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2074 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:13.965023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:13.965332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:13.965375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:13.965676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:13.965719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.965724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.965725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.965726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.965727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.965728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.965729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.965730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.965731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.965736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.965745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.965749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.965753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.965756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.965760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.965764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.965767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.965771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.965777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.965784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:13.965790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.965795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:13.965803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.965808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:13.965812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.965816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:13.965821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.965825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:13.965830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.965835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:13.965839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.965844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.965848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.965853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.965862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.965866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.965895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:13.972048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:13.973054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:13.973071Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:13.973073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:13.973077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:13.973083Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2073 [2:7559342900856596733:11199]
2025-10-09T21:51:13.973096Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:13.973112Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:13.973120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:13.973123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:13.973125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:13.973141Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:13.973152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:13.973160Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:13.973213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.973221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.973223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.973226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:13.973290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:13.973298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.973308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:13.973319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:13.973340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:13.973359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342901218431094:9722], now have 1 active actors on pipe
2025-10-09T21:51:13.973367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342901218430943:2432], now have 1 active actors on pipe
2025-10-09T21:51:13.973374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342900856596728:11199]
2025-10-09T21:51:13.973377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342900856596728:11199]
2025-10-09T21:51:13.973389Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342900856596728:11199]
2025-10-09T21:51:13.973401Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:13.973416Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:13.973420Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:13.973425Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:13.973427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:13.973437Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:13.973440Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:13.973416Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2073
2025-10-09T21:51:13.973424Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2073
2025-10-09T21:51:13.973460Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342900856596728:11199]
2025-10-09T21:51:13.973467Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:13.973468Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:13.973482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.973491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.973548Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d712b4bf-4f70c138-5d76b57f-7a39c70f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:13.973560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:13.973568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:13.973570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:13.973579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:13.973593Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:13.973617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:13.973761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:13.973770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:13.973784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:13.973880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2074 partition: 0 MaxSeqNo: 25 sessionId: src_id|d712b4bf-4f70c138-5d76b57f-7a39c70f_0
2025-10-09T21:51:13.974165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|da849d0d-1fc2778-eaeb0d55-653c5269_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d712b4bf-4f70c138-5d76b57f-7a39c70f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:13.976474Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d712b4bf-4f70c138-5d76b57f-7a39c70f_0] Write session: try to update token
2025-10-09T21:51:13.976491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d712b4bf-4f70c138-5d76b57f-7a39c70f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:13.986952Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:13.986970Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:13.986972Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:13.986973Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:13.986982Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342900856596728:11199]
2025-10-09T21:51:13.986987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342900856596728:11199]
2025-10-09T21:51:13.986991Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342900856596728:11199]
2025-10-09T21:51:13.987033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:13.987039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:13.987039Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:13.987040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:13.987044Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:13.987049Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:13.987163Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:13.988736Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2074 sessionId: src_id|d712b4bf-4f70c138-5d76b57f-7a39c70f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:13.990993Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:14.050064Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342905513398402:5250]
2025-10-09T21:51:14.050326Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 2073, pipe: [1:7559342905513398402:5250]
2025-10-09T21:51:14.134335Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:14.134718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:14.134735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:14.134777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342905513398402:5250], now have 1 active actors on pipe
2025-10-09T21:51:14.134949Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:14.135033Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342905513398402:5250]
2025-10-09T21:51:14.135040Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:14.135064Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:14.135115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2074 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d712b4bf-4f70c138-5d76b57f-7a39c70f_0
2025-10-09T21:51:14.135299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2074 sessionId: src_id|d712b4bf-4f70c138-5d76b57f-7a39c70f_0 is DEAD
2025-10-09T21:51:14.135470Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d712b4bf-4f70c138-5d76b57f-7a39c70f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:14.135481Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d712b4bf-4f70c138-5d76b57f-7a39c70f_0] Write session will restart in 0.008185s
2025-10-09T21:51:14.135507Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d712b4bf-4f70c138-5d76b57f-7a39c70f_0] Write session: Do CDS request
2025-10-09T21:51:14.135513Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d712b4bf-4f70c138-5d76b57f-7a39c70f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:14.144543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d712b4bf-4f70c138-5d76b57f-7a39c70f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:14.144645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:14.144663Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2075
2025-10-09T21:51:14.144800Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2075 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:14.144839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2075 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:14.144845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2075 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:14.144849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:14.145279Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2075 sessionId:  describe result for acl check
2025-10-09T21:51:14.145312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:14.145316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:14.145318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:14.145324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342905513398525:9703] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:14.145854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342905513398525:9703] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:14.147268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:14.147534Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342905513398525:9703] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:14.150424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:14.150633Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:14.150645Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:14.150658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:14.150708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:14.150750Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342905151564222:11213]
2025-10-09T21:51:14.150872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342905513398525:9703] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:14.150885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342905513398525:9703] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:14.150888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342905513398525:9703] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:14.150895Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2075 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:14.151355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:14.151751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:14.151794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:14.152107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:14.152156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.152162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.152163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.152164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.152166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.152167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.152168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.152169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.152171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.152176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.152187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.152191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.152196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.152200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.152205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.152209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.152213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.152217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.152224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.152233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.152239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.152245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.152250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.152255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.152259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.152264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.152268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.152273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.152277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.152283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.152288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.152294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.152299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.152305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.152309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.152315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.152345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:14.159151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:14.160624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:14.160639Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:14.160642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:14.160646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:14.160654Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2074 [2:7559342905151564222:11213]
2025-10-09T21:51:14.160672Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:14.160693Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:14.160701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:14.160704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:14.160707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:14.160728Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:14.160739Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:14.160750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:14.160822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.160830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.160833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.160836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:14.160913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:14.160922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.160937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.160950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.160971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:14.161001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342905513398430:2432], now have 1 active actors on pipe
2025-10-09T21:51:14.161015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342905151564217:11213]
2025-10-09T21:51:14.161016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342905151564217:11213]
2025-10-09T21:51:14.161018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342905151564217:11213]
2025-10-09T21:51:14.161031Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:14.161038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342905513398578:9703], now have 1 active actors on pipe
2025-10-09T21:51:14.161046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:14.161055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:14.161058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:14.161068Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:14.161072Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:14.161075Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:14.161094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342905151564217:11213]
2025-10-09T21:51:14.161097Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:14.161100Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:14.161124Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2074
2025-10-09T21:51:14.161166Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2074
2025-10-09T21:51:14.161231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:14.161242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:14.161267Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ff0d3c0d-e49e25b4-f77694fb-5d23e6f6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:14.161277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:14.161281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:14.161282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:14.161292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:14.161305Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:14.161319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:14.161514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:14.161520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:14.161533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:14.161632Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2075 partition: 0 MaxSeqNo: 25 sessionId: src_id|ff0d3c0d-e49e25b4-f77694fb-5d23e6f6_0
2025-10-09T21:51:14.162080Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d712b4bf-4f70c138-5d76b57f-7a39c70f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ff0d3c0d-e49e25b4-f77694fb-5d23e6f6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:14.165291Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff0d3c0d-e49e25b4-f77694fb-5d23e6f6_0] Write session: try to update token
2025-10-09T21:51:14.165315Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff0d3c0d-e49e25b4-f77694fb-5d23e6f6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:14.175607Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:14.175630Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:14.175633Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:14.175636Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:14.175645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342905151564217:11213]
2025-10-09T21:51:14.175651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342905151564217:11213]
2025-10-09T21:51:14.175655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342905151564217:11213]
2025-10-09T21:51:14.175702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:14.175709Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:14.175715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.175718Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:14.175718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:14.175739Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:14.175862Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:14.180332Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2075 sessionId: src_id|ff0d3c0d-e49e25b4-f77694fb-5d23e6f6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:14.183373Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:14.235396Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342905513398594:5250]
2025-10-09T21:51:14.235813Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 2074, pipe: [1:7559342905513398594:5250]
2025-10-09T21:51:14.331861Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:14.332233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:14.332249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:14.332291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342905513398594:5250], now have 1 active actors on pipe
2025-10-09T21:51:14.332558Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:14.332636Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:14.332686Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342905513398594:5250]
2025-10-09T21:51:14.332692Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:14.332773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2075 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ff0d3c0d-e49e25b4-f77694fb-5d23e6f6_0
2025-10-09T21:51:14.332948Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2075 sessionId: src_id|ff0d3c0d-e49e25b4-f77694fb-5d23e6f6_0 is DEAD
2025-10-09T21:51:14.333091Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff0d3c0d-e49e25b4-f77694fb-5d23e6f6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:14.333103Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff0d3c0d-e49e25b4-f77694fb-5d23e6f6_0] Write session will restart in 0.007415s
2025-10-09T21:51:14.333130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff0d3c0d-e49e25b4-f77694fb-5d23e6f6_0] Write session: Do CDS request
2025-10-09T21:51:14.333144Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff0d3c0d-e49e25b4-f77694fb-5d23e6f6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:14.341448Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff0d3c0d-e49e25b4-f77694fb-5d23e6f6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:14.341524Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:14.341538Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2076
2025-10-09T21:51:14.341640Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2076 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:14.341671Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2076 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:14.341683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2076 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:14.341687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:14.342135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2076 sessionId:  describe result for acl check
2025-10-09T21:51:14.342167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:14.342172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:14.342173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:14.342177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342905513398671:9698] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:14.342653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342905513398671:9698] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:14.344701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:14.344765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342905513398671:9698] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:14.346438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:14.346640Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:14.346655Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:14.346668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:14.346723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:14.346760Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342905151564376:11215]
2025-10-09T21:51:14.346976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342905513398671:9698] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:14.346985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342905513398671:9698] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:14.346988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342905513398671:9698] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:14.346994Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2076 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:14.347310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:14.347603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:14.347645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:14.347951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:14.347997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.348006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.348007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.348008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.348009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.348010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.348011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.348012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.348013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.348019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.348029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.348033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.348037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.348041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.348044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.348048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.348051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.348054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.348060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.348067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.348074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.348079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.348083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.348089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.348095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.348100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.348105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.348110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.348115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.348120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.348125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.348130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.348134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.348139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.348143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.348148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.348177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:14.354452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:14.355744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:14.355761Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:14.355763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:14.355766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:14.355774Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2075 [2:7559342905151564376:11215]
2025-10-09T21:51:14.355787Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:14.355804Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:14.355807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:14.355810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:14.355812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:14.355828Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:14.355833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:14.355840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:14.355897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.355909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.355911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.355914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:14.355980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:14.355988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.356001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.356014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.356036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:14.356061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342905513398601:2432], now have 1 active actors on pipe
2025-10-09T21:51:14.356069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342905513398754:9698], now have 1 active actors on pipe
2025-10-09T21:51:14.356077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342905151564371:11215]
2025-10-09T21:51:14.356079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342905151564371:11215]
2025-10-09T21:51:14.356081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342905151564371:11215]
2025-10-09T21:51:14.356093Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:14.356112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:14.356121Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:14.356125Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:14.356130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:14.356132Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:14.356136Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:14.356111Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2075
2025-10-09T21:51:14.356113Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2075
2025-10-09T21:51:14.356163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342905151564371:11215]
2025-10-09T21:51:14.356171Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:14.356172Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:14.356191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:14.356200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:14.356236Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9eddf01a-f5c52c11-256a1be5-401d0ae_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:14.356245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:14.356248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:14.356250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:14.356257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:14.356264Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:14.356283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:14.356415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:14.356423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:14.356436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:14.356514Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2076 partition: 0 MaxSeqNo: 25 sessionId: src_id|9eddf01a-f5c52c11-256a1be5-401d0ae_0
2025-10-09T21:51:14.356828Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff0d3c0d-e49e25b4-f77694fb-5d23e6f6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9eddf01a-f5c52c11-256a1be5-401d0ae_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:14.359537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9eddf01a-f5c52c11-256a1be5-401d0ae_0] Write session: try to update token
2025-10-09T21:51:14.359558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9eddf01a-f5c52c11-256a1be5-401d0ae_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:14.370237Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:14.370259Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:14.370261Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:14.370263Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:14.370273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342905151564371:11215]
2025-10-09T21:51:14.370277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342905151564371:11215]
2025-10-09T21:51:14.370280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342905151564371:11215]
2025-10-09T21:51:14.370319Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:14.370323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:14.370324Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:14.370327Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:14.370329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.370331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:14.370454Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:14.371767Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2076 sessionId: src_id|9eddf01a-f5c52c11-256a1be5-401d0ae_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:14.374051Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:14.433009Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342905513398766:5250]
2025-10-09T21:51:14.433303Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 2075, pipe: [1:7559342905513398766:5250]
2025-10-09T21:51:14.510576Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:14.510844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:14.510858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:14.510895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342905513398766:5250], now have 1 active actors on pipe
2025-10-09T21:51:14.511067Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:14.511087Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:14.511097Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342905513398766:5250]
2025-10-09T21:51:14.511117Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:14.511119Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:14.511207Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2076 reason: error at write: Disconnected sessionId: src_id|9eddf01a-f5c52c11-256a1be5-401d0ae_0
2025-10-09T21:51:14.511372Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2076 sessionId: src_id|9eddf01a-f5c52c11-256a1be5-401d0ae_0 is DEAD
2025-10-09T21:51:14.511522Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9eddf01a-f5c52c11-256a1be5-401d0ae_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:14.511534Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9eddf01a-f5c52c11-256a1be5-401d0ae_0] Write session will restart in 0.009040s
2025-10-09T21:51:14.511562Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9eddf01a-f5c52c11-256a1be5-401d0ae_0] Write session: Do CDS request
2025-10-09T21:51:14.511569Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9eddf01a-f5c52c11-256a1be5-401d0ae_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:14.521477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9eddf01a-f5c52c11-256a1be5-401d0ae_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:14.521546Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:14.521560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2077
2025-10-09T21:51:14.521670Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2077 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:14.521701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2077 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:14.521712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2077 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:14.521715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:14.522108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2077 sessionId:  describe result for acl check
2025-10-09T21:51:14.522150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:14.522156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:14.522158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:14.522164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342905513398886:9695] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:14.522335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:14.522672Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342905513398886:9695] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:14.524204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:14.524373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342905513398886:9695] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:14.524408Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:14.524420Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:14.524432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:14.524491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:14.524534Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342905151564533:11217]
2025-10-09T21:51:14.525093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:14.525369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:14.525416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:14.525716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:14.525771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.525777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.525778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.525779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.525780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.525781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.525782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.525783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.525784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.525789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.525799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.525804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.525808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.525811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.525815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.525819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.525823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.525827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.525833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.525841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.525846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.525852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.525856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.525861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.525865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.525871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.525875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.525882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.525891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.525896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.525900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.525905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.525909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.525913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.525922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.525926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.525955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:14.528883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342905513398886:9695] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:14.528894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342905513398886:9695] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:14.528897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342905513398886:9695] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:14.528903Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2077 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:14.532022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:14.533323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:14.533339Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:14.533342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:14.533345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:14.533351Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2076 [2:7559342905151564533:11217]
2025-10-09T21:51:14.533362Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:14.533378Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:14.533392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:14.533398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:14.533400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:14.533417Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:14.533427Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:14.533434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:14.533485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.533492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.533494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.533497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:14.533558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:14.533565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.533577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.533593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.533610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:14.533635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342905513398925:9695], now have 1 active actors on pipe
2025-10-09T21:51:14.533637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342905513398768:2432], now have 1 active actors on pipe
2025-10-09T21:51:14.533642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342905151564527:11217]
2025-10-09T21:51:14.533643Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342905151564527:11217]
2025-10-09T21:51:14.533645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342905151564527:11217]
2025-10-09T21:51:14.533657Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:14.533670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:14.533682Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:14.533687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:14.533686Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:14.533690Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:14.533691Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:14.533707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342905151564527:11217]
2025-10-09T21:51:14.533702Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2076
2025-10-09T21:51:14.533718Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:14.533725Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:14.533710Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2076
2025-10-09T21:51:14.533748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:14.533757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:14.533782Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7d5d91b-61abfedc-a7c44a11-db21dab3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:14.533790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:14.533793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:14.533797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:14.533804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:14.533816Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:14.533835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:14.533981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:14.533990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:14.534001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:14.534093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2077 partition: 0 MaxSeqNo: 25 sessionId: src_id|7d5d91b-61abfedc-a7c44a11-db21dab3_0
2025-10-09T21:51:14.534396Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9eddf01a-f5c52c11-256a1be5-401d0ae_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7d5d91b-61abfedc-a7c44a11-db21dab3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:14.537003Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7d5d91b-61abfedc-a7c44a11-db21dab3_0] Write session: try to update token
2025-10-09T21:51:14.537018Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7d5d91b-61abfedc-a7c44a11-db21dab3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:14.550900Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:14.550924Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:14.550936Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342905151564527:11217]
2025-10-09T21:51:14.550982Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:14.550989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:14.551165Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:14.551297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:14.551433Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:14.551564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:14.551740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:14.551779Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:14.552200Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2077 sessionId: src_id|7d5d91b-61abfedc-a7c44a11-db21dab3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:14.554306Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:14.554483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:14.554501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:14.554536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:14.554605Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:14.557530Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:14.557544Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:14.557545Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:14.557546Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:14.557553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342905151564527:11217]
2025-10-09T21:51:14.557556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342905151564527:11217]
2025-10-09T21:51:14.557558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342905151564527:11217]
2025-10-09T21:51:14.557594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:14.557595Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:14.557599Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:14.557602Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:14.557609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.557612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:14.557807Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:14.611185Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342905513398934:5250]
2025-10-09T21:51:14.611542Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 2076, pipe: [1:7559342905513398934:5250]
2025-10-09T21:51:14.701564Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:14.708225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:14.708252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:14.708408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:14.708471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:14.708527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:14.708589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:14.708706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:14.708715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:14.708783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:14.708853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:14.708915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:14.708979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:14.709027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:14.709038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:14.709095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:14.709150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:14.709220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:14.709283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:14.709324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:14.709330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:14.709394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:14.709717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:14.709782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:14.709844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:14.709887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:14.709894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:14.709955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:14.710014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:14.710094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:14.710159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:14.710201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:14.710208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:14.710272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:14.710328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:14.710384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:14.710450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:14.710489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:14.710495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:14.710549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:14.710602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:14.710658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:14.710712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:14.710757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:14.710764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:14.710831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:14.710895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:14.710961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:14.711023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:14.711068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:14.711076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:14.711138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:14.711191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:14.711247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:14.711302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:14.711348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:14.711358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:14.711424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:14.711479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:14.711535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:14.711594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:14.711649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:14.711660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:14.711714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342905513398934:5250], now have 1 active actors on pipe
2025-10-09T21:51:14.711982Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342905513398934:5250]
2025-10-09T21:51:14.712022Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:14.712169Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:14.712179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:14.712206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2077 reason: error at write: Disconnected sessionId: src_id|7d5d91b-61abfedc-a7c44a11-db21dab3_0
2025-10-09T21:51:14.712333Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:14.712384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2077 sessionId: src_id|7d5d91b-61abfedc-a7c44a11-db21dab3_0 is DEAD
2025-10-09T21:51:14.712544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d5d91b-61abfedc-a7c44a11-db21dab3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:14.712556Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d5d91b-61abfedc-a7c44a11-db21dab3_0] Write session will restart in 0.005893s
2025-10-09T21:51:14.712581Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d5d91b-61abfedc-a7c44a11-db21dab3_0] Write session: Do CDS request
2025-10-09T21:51:14.712587Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d5d91b-61abfedc-a7c44a11-db21dab3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:14.719454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7d5d91b-61abfedc-a7c44a11-db21dab3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:14.719525Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:14.719541Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2078
2025-10-09T21:51:14.719698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2078 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:14.719757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2078 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:14.719766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2078 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:14.719770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:14.720269Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2078 sessionId:  describe result for acl check
2025-10-09T21:51:14.720306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:14.720307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:14.720309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:14.720313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342905513398967:9689] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:14.720859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342905513398967:9689] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:14.724205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342905513398967:9689] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:14.724800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:14.726814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:14.727001Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:14.727012Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:14.727023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:14.727072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:14.727071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342905513398967:9689] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:14.727084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342905513398967:9689] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:14.727088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342905513398967:9689] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:14.727097Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2078 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:14.727110Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342905151564691:11219]
2025-10-09T21:51:14.727676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:14.727978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:14.728018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:14.728318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:14.728375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.728382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.728384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.728385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.728386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.728387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.728388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.728389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.728390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.728395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.728406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.728410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.728414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.728418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.728422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.728426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.728430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.728433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.728440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.728447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.728452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.728458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.728467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.728472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.728476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.728481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.728486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.728491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.728495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.728500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.728505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.728510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.728515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.728524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.728529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.728534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.728563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:14.734530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:14.735756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:14.735769Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:14.735772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:14.735776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:14.735782Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2077 [2:7559342905151564691:11219]
2025-10-09T21:51:14.735795Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:14.735813Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:14.735818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:14.735821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:14.735823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:14.735840Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:14.735847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:14.735854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:14.735915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.735923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.735925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.735928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:14.736008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:14.736015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.736026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.736038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.736059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:14.736085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342905513398937:2432], now have 1 active actors on pipe
2025-10-09T21:51:14.736093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342905513399093:9689], now have 1 active actors on pipe
2025-10-09T21:51:14.736101Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342905151564686:11219]
2025-10-09T21:51:14.736107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342905151564686:11219]
2025-10-09T21:51:14.736110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342905151564686:11219]
2025-10-09T21:51:14.736122Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:14.736140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:14.736140Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:14.736143Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:14.736145Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:14.736160Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:14.736163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:14.736170Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2077
2025-10-09T21:51:14.736190Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342905151564686:11219]
2025-10-09T21:51:14.736204Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:14.736205Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:14.736238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:14.736247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:14.736259Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2077
2025-10-09T21:51:14.736276Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f30c6887-8e070c6a-8e3aef6a-439c8da6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:14.736289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:14.736293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:14.736296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:14.736306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:14.736320Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:14.736341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:14.736483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:14.736491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:14.736505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:14.736612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2078 partition: 0 MaxSeqNo: 25 sessionId: src_id|f30c6887-8e070c6a-8e3aef6a-439c8da6_0
2025-10-09T21:51:14.736936Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7d5d91b-61abfedc-a7c44a11-db21dab3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f30c6887-8e070c6a-8e3aef6a-439c8da6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:14.739251Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f30c6887-8e070c6a-8e3aef6a-439c8da6_0] Write session: try to update token
2025-10-09T21:51:14.739271Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f30c6887-8e070c6a-8e3aef6a-439c8da6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:14.749775Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:14.749789Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:14.749791Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:14.749793Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:14.749803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342905151564686:11219]
2025-10-09T21:51:14.749807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342905151564686:11219]
2025-10-09T21:51:14.749812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342905151564686:11219]
2025-10-09T21:51:14.749852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:14.749857Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:14.749861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.749863Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:14.749864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:14.749869Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:14.749989Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:14.751721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2078 sessionId: src_id|f30c6887-8e070c6a-8e3aef6a-439c8da6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:14.754020Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:14.811880Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342905513399106:5250]
2025-10-09T21:51:14.812120Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 2077, pipe: [1:7559342905513399106:5250]
2025-10-09T21:51:14.889554Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:14.889819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:14.889834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:14.889877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342905513399106:5250], now have 1 active actors on pipe
2025-10-09T21:51:14.890080Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:14.890112Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342905513399106:5250]
2025-10-09T21:51:14.890125Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:14.890130Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:14.890140Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:14.890201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2078 reason: error at write: Disconnected sessionId: src_id|f30c6887-8e070c6a-8e3aef6a-439c8da6_0
2025-10-09T21:51:14.890378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2078 sessionId: src_id|f30c6887-8e070c6a-8e3aef6a-439c8da6_0 is DEAD
2025-10-09T21:51:14.890511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f30c6887-8e070c6a-8e3aef6a-439c8da6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:14.890520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f30c6887-8e070c6a-8e3aef6a-439c8da6_0] Write session will restart in 0.006603s
2025-10-09T21:51:14.890546Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f30c6887-8e070c6a-8e3aef6a-439c8da6_0] Write session: Do CDS request
2025-10-09T21:51:14.890552Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f30c6887-8e070c6a-8e3aef6a-439c8da6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:14.898448Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f30c6887-8e070c6a-8e3aef6a-439c8da6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:14.898488Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:14.898500Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2079
2025-10-09T21:51:14.898628Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2079 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:14.898650Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2079 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:14.898659Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2079 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:14.898662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:14.899060Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2079 sessionId:  describe result for acl check
2025-10-09T21:51:14.899095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:14.899101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:14.899103Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:14.899109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342905513399159:9686] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:14.899679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342905513399159:9686] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:14.902062Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342905513399159:9686] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:14.902629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:14.904459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:14.904626Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:14.904635Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:14.904647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:14.904703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:14.904738Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342905151564853:11223]
2025-10-09T21:51:14.904829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342905513399159:9686] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:14.904840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342905513399159:9686] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:14.904844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342905513399159:9686] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:14.904852Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2079 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:14.905293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:14.905594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:14.905635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:14.905937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:14.905979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.905984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.905985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.905986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.905988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.905989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.905990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.905991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.905992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.905997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.906007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.906011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.906015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.906019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.906023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.906045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.906049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.906052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.906059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.906066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:14.906072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.906077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:14.906082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.906087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:14.906091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.906096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:14.906104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.906109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:14.906119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.906124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:14.906129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.906139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.906143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.906148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.906152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.906157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.906185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:14.912203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:14.913193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:14.913208Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:14.913210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:14.913213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:14.913220Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2078 [2:7559342905151564853:11223]
2025-10-09T21:51:14.913232Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:14.913248Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:14.913256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:14.913259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:14.913261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:14.913275Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:14.913283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:14.913288Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:14.913336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.913344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.913346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.913349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:14.913415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:14.913421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.913428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:14.913434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:14.913453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:14.913477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342905513399112:2432], now have 1 active actors on pipe
2025-10-09T21:51:14.913485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342905513399269:9686], now have 1 active actors on pipe
2025-10-09T21:51:14.913493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342905151564848:11223]
2025-10-09T21:51:14.913499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342905151564848:11223]
2025-10-09T21:51:14.913501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342905151564848:11223]
2025-10-09T21:51:14.913512Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:14.913524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:14.913534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:14.913534Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:14.913538Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:14.913539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:14.913540Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:14.913556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342905151564848:11223]
2025-10-09T21:51:14.913561Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:14.913538Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2078
2025-10-09T21:51:14.913561Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2078
2025-10-09T21:51:14.913575Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:14.913593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:14.913598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:14.913628Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6b7f6aa8-72482d7e-ab02400a-ca3755b4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:14.913640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:14.913647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:14.913650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:14.913659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:14.913672Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:14.913688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:14.913834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:14.913841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:14.913852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:14.913920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2079 partition: 0 MaxSeqNo: 25 sessionId: src_id|6b7f6aa8-72482d7e-ab02400a-ca3755b4_0
2025-10-09T21:51:14.914235Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f30c6887-8e070c6a-8e3aef6a-439c8da6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6b7f6aa8-72482d7e-ab02400a-ca3755b4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:14.916492Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b7f6aa8-72482d7e-ab02400a-ca3755b4_0] Write session: try to update token
2025-10-09T21:51:14.916508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b7f6aa8-72482d7e-ab02400a-ca3755b4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:14.926445Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:14.926461Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:14.926462Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:14.926464Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:14.926471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342905151564848:11223]
2025-10-09T21:51:14.926474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342905151564848:11223]
2025-10-09T21:51:14.926477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342905151564848:11223]
2025-10-09T21:51:14.926510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:14.926517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:14.926519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:14.926523Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:14.926528Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:14.926532Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:14.926639Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:14.928607Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2079 sessionId: src_id|6b7f6aa8-72482d7e-ab02400a-ca3755b4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:14.930979Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:14.990095Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342905513399282:5250]
2025-10-09T21:51:14.990388Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 2078, pipe: [1:7559342905513399282:5250]
2025-10-09T21:51:15.070843Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:15.071256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:15.071274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:15.071318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342905513399282:5250], now have 1 active actors on pipe
2025-10-09T21:51:15.071535Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:15.071522Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:15.071530Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342905513399282:5250]
2025-10-09T21:51:15.071556Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:15.071558Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:15.071627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2079 reason: error at write: Disconnected sessionId: src_id|6b7f6aa8-72482d7e-ab02400a-ca3755b4_0
2025-10-09T21:51:15.071826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2079 sessionId: src_id|6b7f6aa8-72482d7e-ab02400a-ca3755b4_0 is DEAD
2025-10-09T21:51:15.071962Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b7f6aa8-72482d7e-ab02400a-ca3755b4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:15.071974Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b7f6aa8-72482d7e-ab02400a-ca3755b4_0] Write session will restart in 0.005239s
2025-10-09T21:51:15.071999Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b7f6aa8-72482d7e-ab02400a-ca3755b4_0] Write session: Do CDS request
2025-10-09T21:51:15.072015Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b7f6aa8-72482d7e-ab02400a-ca3755b4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:15.078451Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b7f6aa8-72482d7e-ab02400a-ca3755b4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:15.078502Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:15.078513Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2080
2025-10-09T21:51:15.078683Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2080 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:15.078709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2080 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:15.078713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2080 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:15.078717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:15.079102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2080 sessionId:  describe result for acl check
2025-10-09T21:51:15.079139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:15.079145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:15.079146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:15.079151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342909808366605:9679] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:15.079702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342909808366605:9679] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:15.081836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342909808366605:9679] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:15.083498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:15.085337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:15.085519Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:15.085525Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:15.085534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:15.085571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:15.085615Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342909446532308:11225]
2025-10-09T21:51:15.085658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342909808366605:9679] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:15.085674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342909808366605:9679] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:15.085678Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342909808366605:9679] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:15.085686Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2080 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:15.086067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:15.086366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:15.086409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:15.086763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:15.086811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.086816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.086817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.086818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.086818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.086819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.086819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.086820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.086821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.086824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.086831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.086833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.086835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.086837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.086839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.086841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.086843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.086845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.086850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.086856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.086859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.086863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.086865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.086869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.086871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.086874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.086877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.086880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.086883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.086886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.086889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.086892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.086894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.086897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.086900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.086903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.086919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:15.093092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:15.094200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:15.094213Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:15.094216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:15.094221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:15.094228Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2079 [2:7559342909446532308:11225]
2025-10-09T21:51:15.094241Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:15.094259Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:15.094270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:15.094274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:15.094276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:15.094294Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:15.094300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:15.094308Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:15.094365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.094374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.094376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.094379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:15.094553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:15.094563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.094576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.094593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.094617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:15.094590Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2079
2025-10-09T21:51:15.094607Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2079
2025-10-09T21:51:15.094644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342909808366581:2432], now have 1 active actors on pipe
2025-10-09T21:51:15.094653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342909808366737:9679], now have 1 active actors on pipe
2025-10-09T21:51:15.094669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:15.094679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:15.094697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342909446532303:11225]
2025-10-09T21:51:15.094707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342909446532303:11225]
2025-10-09T21:51:15.094709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342909446532303:11225]
2025-10-09T21:51:15.094723Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:15.094738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:15.094738Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:15.094741Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:15.094743Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:15.094752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:15.094756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:15.094776Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9868f3d4-3f6f5c22-c2c50c54-9b368df4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:15.094790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:15.094794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:15.094797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:15.094807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:15.094818Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:15.094848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342909446532303:11225]
2025-10-09T21:51:15.094862Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:15.094876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:15.094883Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:15.095039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:15.095052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:15.095074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:15.095160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2080 partition: 0 MaxSeqNo: 25 sessionId: src_id|9868f3d4-3f6f5c22-c2c50c54-9b368df4_0
2025-10-09T21:51:15.095514Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b7f6aa8-72482d7e-ab02400a-ca3755b4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9868f3d4-3f6f5c22-c2c50c54-9b368df4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:15.098087Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9868f3d4-3f6f5c22-c2c50c54-9b368df4_0] Write session: try to update token
2025-10-09T21:51:15.098105Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9868f3d4-3f6f5c22-c2c50c54-9b368df4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:15.109097Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:15.109113Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:15.109115Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:15.109117Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:15.109128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342909446532303:11225]
2025-10-09T21:51:15.109132Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342909446532303:11225]
2025-10-09T21:51:15.109136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342909446532303:11225]
2025-10-09T21:51:15.109179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:15.109186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.109188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:15.109200Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:15.109213Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:15.109216Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:15.109322Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:15.111175Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2080 sessionId: src_id|9868f3d4-3f6f5c22-c2c50c54-9b368df4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:15.113534Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:15.171834Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342909808366771:5250]
2025-10-09T21:51:15.172073Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 2079, pipe: [1:7559342909808366771:5250]
2025-10-09T21:51:15.256729Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:15.257264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:15.257277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:15.257316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342909808366771:5250], now have 1 active actors on pipe
2025-10-09T21:51:15.257540Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:15.257551Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342909808366771:5250]
2025-10-09T21:51:15.257576Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:15.257579Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:15.257645Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2080 reason: error at write: Disconnected sessionId: src_id|9868f3d4-3f6f5c22-c2c50c54-9b368df4_0
2025-10-09T21:51:15.257803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2080 sessionId: src_id|9868f3d4-3f6f5c22-c2c50c54-9b368df4_0 is DEAD
2025-10-09T21:51:15.257951Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9868f3d4-3f6f5c22-c2c50c54-9b368df4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:15.257965Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9868f3d4-3f6f5c22-c2c50c54-9b368df4_0] Write session will restart in 0.009346s
2025-10-09T21:51:15.257983Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9868f3d4-3f6f5c22-c2c50c54-9b368df4_0] Write session: Do CDS request
2025-10-09T21:51:15.257991Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9868f3d4-3f6f5c22-c2c50c54-9b368df4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:15.258292Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:15.268527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9868f3d4-3f6f5c22-c2c50c54-9b368df4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:15.268561Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:15.268578Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2081
2025-10-09T21:51:15.268760Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2081 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:15.268786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2081 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:15.268794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2081 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:15.268797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:15.269232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2081 sessionId:  describe result for acl check
2025-10-09T21:51:15.269272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:15.269280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:15.269281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:15.269287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342909808366899:9661] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:15.269607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:15.269855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342909808366899:9661] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:15.271455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:15.271644Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:15.271654Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:15.271666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:15.271716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:15.271754Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342909446532506:11239]
2025-10-09T21:51:15.272258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342909808366899:9661] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:15.272324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:15.272603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:15.272642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:15.272937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:15.272990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.272995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.272996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.272997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.272998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.273000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.273001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.273001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.273002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.273006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.273017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.273022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.273027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.273035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.273040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.273047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.273051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.273059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.273065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.273074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.273084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.273089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.273098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.273104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.273112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.273118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.273122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.273131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.273136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.273142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.273146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.273156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.273164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.273169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.273177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.273182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.273214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:15.278165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342909808366899:9661] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:15.278177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342909808366899:9661] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:15.278181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342909808366899:9661] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:15.278187Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2081 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:15.279556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:15.280807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:15.280822Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:15.280824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:15.280829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:15.280836Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2080 [2:7559342909446532506:11239]
2025-10-09T21:51:15.280849Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:15.280865Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:15.280874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:15.280877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:15.280880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:15.280898Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:15.280910Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:15.280917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:15.280972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.280977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.280980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.280983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:15.281055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:15.281062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.281076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.281095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.281112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:15.281137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342909808366939:9661], now have 1 active actors on pipe
2025-10-09T21:51:15.281147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342909808366777:2432], now have 1 active actors on pipe
2025-10-09T21:51:15.281155Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342909446532501:11239]
2025-10-09T21:51:15.281158Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342909446532501:11239]
2025-10-09T21:51:15.281160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342909446532501:11239]
2025-10-09T21:51:15.281175Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:15.281190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:15.281195Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:15.281198Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:15.281201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:15.281211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:15.281219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:15.281194Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2080
2025-10-09T21:51:15.281195Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2080
2025-10-09T21:51:15.281250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342909446532501:11239]
2025-10-09T21:51:15.281264Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:15.281278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:15.281281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:15.281285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:15.281336Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e30b99a8-2b958ce9-bfae0c23-f31a3e57_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:15.281348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:15.281352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:15.281359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:15.281370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:15.281397Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:15.281421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:15.281565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:15.281572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:15.281584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:15.281662Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2081 partition: 0 MaxSeqNo: 25 sessionId: src_id|e30b99a8-2b958ce9-bfae0c23-f31a3e57_0
2025-10-09T21:51:15.282008Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9868f3d4-3f6f5c22-c2c50c54-9b368df4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e30b99a8-2b958ce9-bfae0c23-f31a3e57_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:15.284270Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e30b99a8-2b958ce9-bfae0c23-f31a3e57_0] Write session: try to update token
2025-10-09T21:51:15.284287Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e30b99a8-2b958ce9-bfae0c23-f31a3e57_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:15.295527Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:15.295540Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:15.295542Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:15.295544Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:15.295551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342909446532501:11239]
2025-10-09T21:51:15.295554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342909446532501:11239]
2025-10-09T21:51:15.295556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342909446532501:11239]
2025-10-09T21:51:15.295591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:15.295597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.295599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:15.295604Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:15.295611Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:15.295614Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:15.295725Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:15.296642Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2081 sessionId: src_id|e30b99a8-2b958ce9-bfae0c23-f31a3e57_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:15.298903Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:15.357703Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342909808366951:5250]
2025-10-09T21:51:15.357992Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 2080, pipe: [1:7559342909808366951:5250]
2025-10-09T21:51:15.446817Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:15.447139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:15.447151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:15.447200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342909808366951:5250], now have 1 active actors on pipe
2025-10-09T21:51:15.447353Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:15.447390Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342909808366951:5250]
2025-10-09T21:51:15.447392Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:15.447395Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:15.447431Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:15.447450Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2081 reason: error at write: Disconnected sessionId: src_id|e30b99a8-2b958ce9-bfae0c23-f31a3e57_0
2025-10-09T21:51:15.447640Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2081 sessionId: src_id|e30b99a8-2b958ce9-bfae0c23-f31a3e57_0 is DEAD
2025-10-09T21:51:15.447804Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e30b99a8-2b958ce9-bfae0c23-f31a3e57_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:15.447813Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e30b99a8-2b958ce9-bfae0c23-f31a3e57_0] Write session will restart in 0.005693s
2025-10-09T21:51:15.447842Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e30b99a8-2b958ce9-bfae0c23-f31a3e57_0] Write session: Do CDS request
2025-10-09T21:51:15.447849Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e30b99a8-2b958ce9-bfae0c23-f31a3e57_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:15.454368Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e30b99a8-2b958ce9-bfae0c23-f31a3e57_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:15.454500Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:15.454513Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2082
2025-10-09T21:51:15.454626Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2082 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:15.454655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2082 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:15.454665Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2082 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:15.454669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:15.455095Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2082 sessionId:  describe result for acl check
2025-10-09T21:51:15.455128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:15.455135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:15.455136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:15.455142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342909808366991:9651] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:15.455669Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342909808366991:9651] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:15.458379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342909808366991:9651] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:15.459572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:15.461511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:15.461636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342909808366991:9651] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:15.461650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342909808366991:9651] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:15.461654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342909808366991:9651] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:15.461663Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2082 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:15.461696Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:15.461709Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:15.461724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:15.461793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:15.461831Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342909446532667:11241]
2025-10-09T21:51:15.462400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:15.462739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:15.462788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:15.463096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:15.463154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.463161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.463162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.463163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.463164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.463165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.463166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.463167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.463168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.463173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.463185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.463189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.463193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.463197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.463201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.463205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.463208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.463212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.463220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.463227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.463234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.463239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.463244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.463249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.463253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.463258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.463262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.463267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.463271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.463276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.463280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.463285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.463289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.463293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.463305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.463311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.463339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:15.469397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:15.470430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:15.470443Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:15.470445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:15.470448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:15.470452Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2081 [2:7559342909446532667:11241]
2025-10-09T21:51:15.470462Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:15.470477Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:15.470480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:15.470482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:15.470483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:15.470496Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:15.470500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:15.470505Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:15.470560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.470567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.470570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.470572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:15.470637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:15.470645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.470653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.470659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.470670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:15.470684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342909446532662:11241]
2025-10-09T21:51:15.470689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342909446532662:11241]
2025-10-09T21:51:15.470690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342909446532662:11241]
2025-10-09T21:51:15.470697Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:15.470707Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:15.470709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342909808366954:2432], now have 1 active actors on pipe
2025-10-09T21:51:15.470712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342909808367113:9651], now have 1 active actors on pipe
2025-10-09T21:51:15.470713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:15.470716Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:15.470719Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:15.470727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:15.470729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:15.470753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342909446532662:11241]
2025-10-09T21:51:15.470762Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:15.470779Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:15.470794Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2081
2025-10-09T21:51:15.470801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2081
2025-10-09T21:51:15.470850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:15.470861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:15.470891Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8c3a3dcd-16857c96-d7dc90ca-f747699f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:15.470900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:15.470904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:15.470906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:15.470916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:15.470931Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:15.470951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:15.471086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:15.471092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:15.471103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:15.471197Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2082 partition: 0 MaxSeqNo: 25 sessionId: src_id|8c3a3dcd-16857c96-d7dc90ca-f747699f_0
2025-10-09T21:51:15.471569Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e30b99a8-2b958ce9-bfae0c23-f31a3e57_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8c3a3dcd-16857c96-d7dc90ca-f747699f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:15.473982Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c3a3dcd-16857c96-d7dc90ca-f747699f_0] Write session: try to update token
2025-10-09T21:51:15.473998Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c3a3dcd-16857c96-d7dc90ca-f747699f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:15.485032Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:15.485048Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:15.485051Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:15.485053Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:15.485062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342909446532662:11241]
2025-10-09T21:51:15.485067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342909446532662:11241]
2025-10-09T21:51:15.485071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342909446532662:11241]
2025-10-09T21:51:15.485108Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:15.485113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:15.485117Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:15.485120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.485121Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:15.485123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:15.485253Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:15.486505Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2082 sessionId: src_id|8c3a3dcd-16857c96-d7dc90ca-f747699f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:15.488826Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:15.547638Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342909808367125:5250]
2025-10-09T21:51:15.547890Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 2081, pipe: [1:7559342909808367125:5250]
2025-10-09T21:51:15.625864Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:15.626087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:15.626099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:15.626135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342909808367125:5250], now have 1 active actors on pipe
2025-10-09T21:51:15.626300Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:15.626333Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:15.626346Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342909808367125:5250]
2025-10-09T21:51:15.626373Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:15.626443Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2082 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8c3a3dcd-16857c96-d7dc90ca-f747699f_0
2025-10-09T21:51:15.626615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2082 sessionId: src_id|8c3a3dcd-16857c96-d7dc90ca-f747699f_0 is DEAD
2025-10-09T21:51:15.626774Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c3a3dcd-16857c96-d7dc90ca-f747699f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:15.626784Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c3a3dcd-16857c96-d7dc90ca-f747699f_0] Write session will restart in 0.009471s
2025-10-09T21:51:15.626811Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c3a3dcd-16857c96-d7dc90ca-f747699f_0] Write session: Do CDS request
2025-10-09T21:51:15.626817Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c3a3dcd-16857c96-d7dc90ca-f747699f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:15.637532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c3a3dcd-16857c96-d7dc90ca-f747699f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:15.637612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:15.637626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2083
2025-10-09T21:51:15.637752Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2083 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:15.637782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2083 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:15.637792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2083 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:15.637795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:15.638011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:15.638194Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2083 sessionId:  describe result for acl check
2025-10-09T21:51:15.638231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:15.638235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:15.638237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:15.638242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342909808367253:9639] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:15.638754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342909808367253:9639] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:15.639812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:15.639990Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:15.639999Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:15.640009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:15.640052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:15.640086Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342909446532829:11243]
2025-10-09T21:51:15.640444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:15.640467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342909808367253:9639] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:15.640699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:15.640733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:15.640980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:15.641027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.641034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.641035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.641036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.641037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.641038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.641039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.641040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.641041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.641046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.641056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.641060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.641064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.641067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.641071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.641075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.641078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.641082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.641088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.641095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.641102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.641107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.641111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.641117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.641120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.641125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.641129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.641139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.641143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.641148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.641153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.641157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.641163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.641168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.641172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.641178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.641208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:15.645585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342909808367253:9639] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:15.645599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342909808367253:9639] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:15.645603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342909808367253:9639] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:15.645610Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2083 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:15.646911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:15.648088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:15.648100Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:15.648102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:15.648104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:15.648109Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2082 [2:7559342909446532829:11243]
2025-10-09T21:51:15.648118Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:15.648130Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:15.648132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:15.648134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:15.648135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:15.648146Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:15.648150Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:15.648155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:15.648192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.648197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.648198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.648200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:15.648246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:15.648253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.648263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.648272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.648289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:15.648312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342909808367127:2432], now have 1 active actors on pipe
2025-10-09T21:51:15.648317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342909808367291:9639], now have 1 active actors on pipe
2025-10-09T21:51:15.648322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342909446532823:11243]
2025-10-09T21:51:15.648324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342909446532823:11243]
2025-10-09T21:51:15.648325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342909446532823:11243]
2025-10-09T21:51:15.648334Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:15.648349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:15.648353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:15.648358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:15.648359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:15.648360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:15.648361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:15.648354Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2082
2025-10-09T21:51:15.648365Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2082
2025-10-09T21:51:15.648380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342909446532823:11243]
2025-10-09T21:51:15.648385Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:15.648409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:15.648410Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:15.648415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:15.648437Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f341fa90-7b22294-e6215f7e-c3b45904_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:15.648445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:15.648448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:15.648450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:15.648458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:15.648467Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:15.648484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:15.648629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:15.648635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:15.648645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:15.648737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2083 partition: 0 MaxSeqNo: 25 sessionId: src_id|f341fa90-7b22294-e6215f7e-c3b45904_0
2025-10-09T21:51:15.649057Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c3a3dcd-16857c96-d7dc90ca-f747699f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f341fa90-7b22294-e6215f7e-c3b45904_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:15.651431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f341fa90-7b22294-e6215f7e-c3b45904_0] Write session: try to update token
2025-10-09T21:51:15.651447Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f341fa90-7b22294-e6215f7e-c3b45904_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:15.661691Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:15.661713Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:15.661715Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:15.661717Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:15.661727Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342909446532823:11243]
2025-10-09T21:51:15.661732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342909446532823:11243]
2025-10-09T21:51:15.661735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342909446532823:11243]
2025-10-09T21:51:15.661778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:15.661785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.661788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:15.661787Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:15.661793Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:15.661797Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:15.661917Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:15.663709Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2083 sessionId: src_id|f341fa90-7b22294-e6215f7e-c3b45904_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:15.666008Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:15.726323Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342909808367299:5250]
2025-10-09T21:51:15.726575Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 2082, pipe: [1:7559342909808367299:5250]
2025-10-09T21:51:15.810371Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:15.810595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:15.810610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:15.810649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342909808367299:5250], now have 1 active actors on pipe
2025-10-09T21:51:15.810823Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:15.810878Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:15.810917Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:15.810919Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342909808367299:5250]
2025-10-09T21:51:15.810988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2083 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f341fa90-7b22294-e6215f7e-c3b45904_0
2025-10-09T21:51:15.811174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2083 sessionId: src_id|f341fa90-7b22294-e6215f7e-c3b45904_0 is DEAD
2025-10-09T21:51:15.811343Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f341fa90-7b22294-e6215f7e-c3b45904_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:15.811354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f341fa90-7b22294-e6215f7e-c3b45904_0] Write session will restart in 0.009510s
2025-10-09T21:51:15.811382Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f341fa90-7b22294-e6215f7e-c3b45904_0] Write session: Do CDS request
2025-10-09T21:51:15.811388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f341fa90-7b22294-e6215f7e-c3b45904_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:15.822517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f341fa90-7b22294-e6215f7e-c3b45904_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:15.822533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:15.822580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:15.822594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2084
2025-10-09T21:51:15.822744Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2084 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:15.822787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2084 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:15.822797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2084 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:15.822801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:15.823236Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2084 sessionId:  describe result for acl check
2025-10-09T21:51:15.823274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:15.823279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:15.823280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:15.823285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342909808367428:9631] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:15.823759Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342909808367428:9631] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:15.824602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:15.824778Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:15.824786Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:15.824797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:15.824860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:15.824915Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342909446532997:11247]
2025-10-09T21:51:15.825520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:15.825803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:15.825853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:15.825984Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342909808367428:9631] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:15.826196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:15.826251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.826256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.826256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.826257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.826258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.826259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.826259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.826260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.826261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.826264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.826270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.826273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.826275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.826277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.826279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.826281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.826283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.826284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.826289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.826294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:15.826298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.826302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:15.826305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.826308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:15.826311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.826314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:15.826317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.826320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:15.826322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.826325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:15.826328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.826331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.826334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.826337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.826340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.826343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.826359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:15.831085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342909808367428:9631] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:15.831098Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342909808367428:9631] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:15.831101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342909808367428:9631] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:15.831108Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2084 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:15.832437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:15.833419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:15.833430Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:15.833432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:15.833435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:15.833440Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2083 [2:7559342909446532997:11247]
2025-10-09T21:51:15.833452Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:15.833472Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:15.833479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:15.833483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:15.833485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:15.833502Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:15.833513Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:15.833520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:15.833572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.833576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.833578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.833580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:15.833642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:15.833651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.833663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:15.833675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:15.833690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:15.833707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342909808367301:2432], now have 1 active actors on pipe
2025-10-09T21:51:15.833713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342909808367466:9631], now have 1 active actors on pipe
2025-10-09T21:51:15.833718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342909446532991:11247]
2025-10-09T21:51:15.833721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342909446532991:11247]
2025-10-09T21:51:15.833723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342909446532991:11247]
2025-10-09T21:51:15.833731Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:15.833742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:15.833749Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:15.833753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:15.833755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:15.833756Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:15.833757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:15.833778Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342909446532991:11247]
2025-10-09T21:51:15.833780Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:15.833791Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:15.833765Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2083
2025-10-09T21:51:15.833813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:15.833767Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2083
2025-10-09T21:51:15.833822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:15.833855Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ec0c1bb0-d15c7d50-bcf9cee0-b640215_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:15.833866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:15.833874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:15.833876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:15.833891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:15.833904Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:15.833928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:15.834068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:15.834075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:15.834088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:15.834182Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2084 partition: 0 MaxSeqNo: 25 sessionId: src_id|ec0c1bb0-d15c7d50-bcf9cee0-b640215_0
2025-10-09T21:51:15.834505Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f341fa90-7b22294-e6215f7e-c3b45904_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ec0c1bb0-d15c7d50-bcf9cee0-b640215_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:15.836776Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ec0c1bb0-d15c7d50-bcf9cee0-b640215_0] Write session: try to update token
2025-10-09T21:51:15.836791Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ec0c1bb0-d15c7d50-bcf9cee0-b640215_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:15.847623Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:15.847639Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:15.847641Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:15.847642Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:15.847650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342909446532991:11247]
2025-10-09T21:51:15.847654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342909446532991:11247]
2025-10-09T21:51:15.847657Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342909446532991:11247]
2025-10-09T21:51:15.847691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:15.847696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:15.847695Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:15.847697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:15.847699Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:15.847702Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:15.847820Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:15.848718Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2084 sessionId: src_id|ec0c1bb0-d15c7d50-bcf9cee0-b640215_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:15.851193Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:15.911129Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342909808367477:5250]
2025-10-09T21:51:15.911403Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 2083, pipe: [1:7559342909808367477:5250]
2025-10-09T21:51:15.997716Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:15.998149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:15.998161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:15.998203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342909808367477:5250], now have 1 active actors on pipe
2025-10-09T21:51:15.998367Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:15.998416Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:15.998421Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342909808367477:5250]
2025-10-09T21:51:15.998452Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:15.998541Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2084 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ec0c1bb0-d15c7d50-bcf9cee0-b640215_0
2025-10-09T21:51:15.998714Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2084 sessionId: src_id|ec0c1bb0-d15c7d50-bcf9cee0-b640215_0 is DEAD
2025-10-09T21:51:15.998859Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec0c1bb0-d15c7d50-bcf9cee0-b640215_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:15.998873Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec0c1bb0-d15c7d50-bcf9cee0-b640215_0] Write session will restart in 0.007915s
2025-10-09T21:51:15.998899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec0c1bb0-d15c7d50-bcf9cee0-b640215_0] Write session: Do CDS request
2025-10-09T21:51:15.998905Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec0c1bb0-d15c7d50-bcf9cee0-b640215_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:16.007537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ec0c1bb0-d15c7d50-bcf9cee0-b640215_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:16.007634Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:16.007649Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2085
2025-10-09T21:51:16.007772Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2085 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:16.007814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2085 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:16.007826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2085 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:16.007830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:16.008311Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2085 sessionId:  describe result for acl check
2025-10-09T21:51:16.008351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:16.008360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:16.008361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:16.008367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342914103334855:9628] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:16.008871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342914103334855:9628] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:16.010792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342914103334855:9628] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:16.011031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:16.013103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:16.013274Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:16.013283Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:16.013291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:16.013335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:16.013364Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342913741500456:11249]
2025-10-09T21:51:16.013447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342914103334855:9628] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:16.013458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342914103334855:9628] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:16.013461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342914103334855:9628] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:16.013468Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2085 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:16.013801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:16.014100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:16.014134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:16.014401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:16.014444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.014448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.014449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.014450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.014452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.014453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.014455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.014456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.014457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.014462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.014472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.014476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.014479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.014483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.014487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.014491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.014495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.014499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.014505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.014514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.014520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.014526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.014531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.014536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.014540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.014545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.014550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.014555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.014559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.014565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.014576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.014581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.014586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.014595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.014600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.014605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.014633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:16.021156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:16.022316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:16.022328Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:16.022331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:16.022335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:16.022342Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2084 [2:7559342913741500456:11249]
2025-10-09T21:51:16.022357Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:16.022378Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:16.022386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:16.022389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:16.022392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:16.022412Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:16.022426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:16.022434Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:16.022500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.022507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.022509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.022512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:16.022587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:16.022598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.022611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.022624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.022642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:16.022674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342909808367481:2432], now have 1 active actors on pipe
2025-10-09T21:51:16.022678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342914103334936:9628], now have 1 active actors on pipe
2025-10-09T21:51:16.022688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342913741500451:11249]
2025-10-09T21:51:16.022690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342913741500451:11249]
2025-10-09T21:51:16.022693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342913741500451:11249]
2025-10-09T21:51:16.022703Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:16.022723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:16.022731Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:16.022736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:16.022738Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:16.022706Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2084
2025-10-09T21:51:16.022744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:16.022724Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2084
2025-10-09T21:51:16.022747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:16.022781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342913741500451:11249]
2025-10-09T21:51:16.022785Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:16.022787Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:16.022803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.022812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.022863Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3b13f070-eecb5575-e15d9292-13cbb4a4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:16.022876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:16.022880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:16.022888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:16.022899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:16.022910Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:16.022931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:16.023110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.023121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.023148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:16.023233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2085 partition: 0 MaxSeqNo: 25 sessionId: src_id|3b13f070-eecb5575-e15d9292-13cbb4a4_0
2025-10-09T21:51:16.023604Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec0c1bb0-d15c7d50-bcf9cee0-b640215_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3b13f070-eecb5575-e15d9292-13cbb4a4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:16.025938Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3b13f070-eecb5575-e15d9292-13cbb4a4_0] Write session: try to update token
2025-10-09T21:51:16.025964Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3b13f070-eecb5575-e15d9292-13cbb4a4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:16.037683Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:16.037701Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:16.037704Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:16.037706Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:16.037717Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342913741500451:11249]
2025-10-09T21:51:16.037722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342913741500451:11249]
2025-10-09T21:51:16.037725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342913741500451:11249]
2025-10-09T21:51:16.037772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:16.037773Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:16.037780Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:16.037780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.037783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:16.037784Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:16.037934Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:16.039468Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2085 sessionId: src_id|3b13f070-eecb5575-e15d9292-13cbb4a4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:16.041955Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:16.098570Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342914103334948:5250]
2025-10-09T21:51:16.098896Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 2084, pipe: [1:7559342914103334948:5250]
2025-10-09T21:51:16.186991Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:16.187355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.187371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.187413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342914103334948:5250], now have 1 active actors on pipe
2025-10-09T21:51:16.187601Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:16.187588Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:16.187629Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:16.187632Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:16.187700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2085 reason: error at write: Disconnected sessionId: src_id|3b13f070-eecb5575-e15d9292-13cbb4a4_0
2025-10-09T21:51:16.187890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2085 sessionId: src_id|3b13f070-eecb5575-e15d9292-13cbb4a4_0 is DEAD
2025-10-09T21:51:16.188023Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b13f070-eecb5575-e15d9292-13cbb4a4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:16.188034Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b13f070-eecb5575-e15d9292-13cbb4a4_0] Write session will restart in 0.009042s
2025-10-09T21:51:16.188058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b13f070-eecb5575-e15d9292-13cbb4a4_0] Write session: Do CDS request
2025-10-09T21:51:16.188065Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b13f070-eecb5575-e15d9292-13cbb4a4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:16.188053Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342914103334948:5250]
2025-10-09T21:51:16.198523Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3b13f070-eecb5575-e15d9292-13cbb4a4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:16.198912Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:16.198935Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2086
2025-10-09T21:51:16.199073Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2086 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:16.199120Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2086 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:16.199130Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2086 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:16.199134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:16.199457Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2086 sessionId:  describe result for acl check
2025-10-09T21:51:16.199484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:16.199489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:16.199490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:16.199494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342914103335103:9606] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:16.199914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342914103335103:9606] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:16.200022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:16.201646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342914103335103:9606] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:16.202390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:16.202594Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:16.202606Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:16.202619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:16.202673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:16.202713Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342913741500660:11259]
2025-10-09T21:51:16.203250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:16.203593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:16.203633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:16.203927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:16.203977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.203982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.203983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.203984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.203986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.203987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.203988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.203989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.203990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.203994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.204005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.204009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.204012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.204016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.204020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.204024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.204028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.204032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.204037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.204046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.204052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.204057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.204061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.204066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.204070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.204074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.204081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.204092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.204095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.204100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.204106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.204111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.204115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.204120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.204124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.204129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.204156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:16.204159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342914103335103:9606] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:16.204172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342914103335103:9606] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:16.204176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342914103335103:9606] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:16.204183Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2086 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:16.210245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:16.211247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:16.211262Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:16.211264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:16.211268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:16.211273Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2085 [2:7559342913741500660:11259]
2025-10-09T21:51:16.211283Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:16.211297Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:16.211304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:16.211306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:16.211307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:16.211320Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:16.211324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:16.211330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:16.211380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.211386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.211387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.211389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:16.211553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:16.211560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.211568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.211574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.211580Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2085
2025-10-09T21:51:16.211592Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2085
2025-10-09T21:51:16.211585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:16.211617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342914103334968:2432], now have 1 active actors on pipe
2025-10-09T21:51:16.211620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342914103335137:9606], now have 1 active actors on pipe
2025-10-09T21:51:16.211625Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342913741500655:11259]
2025-10-09T21:51:16.211627Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342913741500655:11259]
2025-10-09T21:51:16.211629Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342913741500655:11259]
2025-10-09T21:51:16.211635Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:16.211647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:16.211647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:16.211650Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:16.211651Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:16.211660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:16.211663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:16.211682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.211687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.211725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342913741500655:11259]
2025-10-09T21:51:16.211736Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:16.211747Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:16.211752Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bf565ebf-65b5570a-cced9b25-d4e4eba1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:16.211760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:16.211764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:16.211766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:16.211776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:16.211786Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:16.211818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:16.211931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.211936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.211948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:16.212044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2086 partition: 0 MaxSeqNo: 25 sessionId: src_id|bf565ebf-65b5570a-cced9b25-d4e4eba1_0
2025-10-09T21:51:16.212333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3b13f070-eecb5575-e15d9292-13cbb4a4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bf565ebf-65b5570a-cced9b25-d4e4eba1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:16.214614Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf565ebf-65b5570a-cced9b25-d4e4eba1_0] Write session: try to update token
2025-10-09T21:51:16.214630Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf565ebf-65b5570a-cced9b25-d4e4eba1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:16.227019Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2086 sessionId: src_id|bf565ebf-65b5570a-cced9b25-d4e4eba1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:16.227653Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:16.227668Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:16.227669Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:16.227671Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:16.227678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342913741500655:11259]
2025-10-09T21:51:16.227680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342913741500655:11259]
2025-10-09T21:51:16.227684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342913741500655:11259]
2025-10-09T21:51:16.227720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:16.227725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.227727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:16.227734Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:16.227740Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:16.227744Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:16.227854Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:16.229416Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:16.287964Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342914103335151:5250]
2025-10-09T21:51:16.288230Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 2085, pipe: [1:7559342914103335151:5250]
2025-10-09T21:51:16.374108Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:16.374377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.374389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.374426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342914103335151:5250], now have 1 active actors on pipe
2025-10-09T21:51:16.374613Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:16.374614Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:16.374647Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342914103335151:5250]
2025-10-09T21:51:16.374674Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:16.374748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2086 reason: status is not ok: tablet will be restarted right now sessionId: src_id|bf565ebf-65b5570a-cced9b25-d4e4eba1_0
2025-10-09T21:51:16.374936Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2086 sessionId: src_id|bf565ebf-65b5570a-cced9b25-d4e4eba1_0 is DEAD
2025-10-09T21:51:16.375080Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf565ebf-65b5570a-cced9b25-d4e4eba1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:16.375091Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf565ebf-65b5570a-cced9b25-d4e4eba1_0] Write session will restart in 0.005833s
2025-10-09T21:51:16.375128Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf565ebf-65b5570a-cced9b25-d4e4eba1_0] Write session: Do CDS request
2025-10-09T21:51:16.375134Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf565ebf-65b5570a-cced9b25-d4e4eba1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:16.382512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bf565ebf-65b5570a-cced9b25-d4e4eba1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:16.382574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:16.382590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2087
2025-10-09T21:51:16.382733Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2087 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:16.382782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2087 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:16.382795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2087 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:16.382798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:16.383531Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2087 sessionId:  describe result for acl check
2025-10-09T21:51:16.383569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:16.383575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:16.383577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:16.383585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342914103335195:9598] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:16.384141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342914103335195:9598] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:16.386588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342914103335195:9598] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:16.388060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:16.390011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:16.390133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342914103335195:9598] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:16.390145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342914103335195:9598] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:16.390163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342914103335195:9598] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:16.390171Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2087 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:16.390203Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:16.390212Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:16.390223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:16.390271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:16.390302Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342913741500826:11265]
2025-10-09T21:51:16.390720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:16.391015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:16.391044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:16.391335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:16.391381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.391386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.391387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.391388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.391390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.391391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.391392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.391397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.391398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.391402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.391409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.391411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.391414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.391416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.391418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.391420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.391422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.391424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.391429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.391437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.391444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.391450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.391454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.391459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.391463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.391469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.391473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.391477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.391481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.391485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.391490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.391495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.391498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.391503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.391507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.391512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.391541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:16.397738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:16.398740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:16.398753Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:16.398755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:16.398758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:16.398764Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2086 [2:7559342913741500826:11265]
2025-10-09T21:51:16.398774Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:16.398788Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:16.398790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:16.398792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:16.398793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:16.398806Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:16.398812Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:16.398818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:16.398858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.398863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.398864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.398866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:16.398932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:16.398940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.398951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.398963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.398979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:16.398999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342914103335323:9598], now have 1 active actors on pipe
2025-10-09T21:51:16.399007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342914103335158:2432], now have 1 active actors on pipe
2025-10-09T21:51:16.399015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342913741500821:11265]
2025-10-09T21:51:16.399016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342913741500821:11265]
2025-10-09T21:51:16.399018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342913741500821:11265]
2025-10-09T21:51:16.399028Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:16.399046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:16.399061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:16.399065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:16.399067Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:16.399068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:16.399070Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:16.399046Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2086
2025-10-09T21:51:16.399056Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2086
2025-10-09T21:51:16.399106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.399116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.399128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342913741500821:11265]
2025-10-09T21:51:16.399136Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:16.399161Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:16.399187Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|627c42df-edd81759-99264db8-81add485_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:16.399198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:16.399202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:16.399204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:16.399215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:16.399227Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:16.399256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:16.399400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.399408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.399422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:16.399521Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2087 partition: 0 MaxSeqNo: 25 sessionId: src_id|627c42df-edd81759-99264db8-81add485_0
2025-10-09T21:51:16.399854Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bf565ebf-65b5570a-cced9b25-d4e4eba1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|627c42df-edd81759-99264db8-81add485_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:16.402237Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|627c42df-edd81759-99264db8-81add485_0] Write session: try to update token
2025-10-09T21:51:16.402254Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|627c42df-edd81759-99264db8-81add485_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:16.413362Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:16.413375Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:16.413390Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:16.413392Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:16.413399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342913741500821:11265]
2025-10-09T21:51:16.413402Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342913741500821:11265]
2025-10-09T21:51:16.413405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342913741500821:11265]
2025-10-09T21:51:16.413446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:16.413453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.413456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:16.413457Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:16.413461Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:16.413465Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:16.413579Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:16.415166Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2087 sessionId: src_id|627c42df-edd81759-99264db8-81add485_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:16.417510Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:16.474868Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342914103335334:5250]
2025-10-09T21:51:16.475127Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 2086, pipe: [1:7559342914103335334:5250]
2025-10-09T21:51:16.556078Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:16.556527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.556539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.556575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342914103335334:5250], now have 1 active actors on pipe
2025-10-09T21:51:16.556756Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:16.556752Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:16.556766Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:16.556772Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:16.556814Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342914103335334:5250]
2025-10-09T21:51:16.556836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2087 reason: error at write: Disconnected sessionId: src_id|627c42df-edd81759-99264db8-81add485_0
2025-10-09T21:51:16.556996Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2087 sessionId: src_id|627c42df-edd81759-99264db8-81add485_0 is DEAD
2025-10-09T21:51:16.557158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|627c42df-edd81759-99264db8-81add485_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:16.557169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|627c42df-edd81759-99264db8-81add485_0] Write session will restart in 0.005724s
2025-10-09T21:51:16.557194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|627c42df-edd81759-99264db8-81add485_0] Write session: Do CDS request
2025-10-09T21:51:16.557200Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|627c42df-edd81759-99264db8-81add485_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:16.564519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|627c42df-edd81759-99264db8-81add485_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:16.564594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:16.564610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2088
2025-10-09T21:51:16.564721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2088 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:16.564753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2088 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:16.564758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2088 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:16.564762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:16.565186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2088 sessionId:  describe result for acl check
2025-10-09T21:51:16.565221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:16.565229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:16.565231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:16.565237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342914103335364:9595] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:16.565775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342914103335364:9595] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:16.568266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342914103335364:9595] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:16.570060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:16.571996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:16.572128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342914103335364:9595] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:16.572141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342914103335364:9595] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:16.572144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342914103335364:9595] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:16.572152Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2088 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:16.572167Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:16.572177Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:16.572190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:16.572244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:16.572279Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342913741500993:11267]
2025-10-09T21:51:16.572641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:16.572963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:16.573003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:16.573254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:16.573289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.573293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.573294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.573294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.573295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.573295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.573296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.573297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.573297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.573300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.573306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.573308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.573310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.573312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.573314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.573316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.573317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.573319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.573323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.573328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.573332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.573335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.573337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.573340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.573342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.573345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.573347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.573350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.573354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.573359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.573363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.573373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.573392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.573395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.573399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.573402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.573418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:16.579373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:16.580441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:16.580454Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:16.580456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:16.580461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:16.580467Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2087 [2:7559342913741500993:11267]
2025-10-09T21:51:16.580482Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:16.580497Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:16.580504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:16.580507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:16.580509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:16.580525Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:16.580535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:16.580542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:16.580598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.580605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.580607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.580610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:16.580676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:16.580684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.580695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.580711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.580731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:16.580757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342914103335502:9595], now have 1 active actors on pipe
2025-10-09T21:51:16.580760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342914103335337:2432], now have 1 active actors on pipe
2025-10-09T21:51:16.580768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342913741500988:11267]
2025-10-09T21:51:16.580771Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342913741500988:11267]
2025-10-09T21:51:16.580773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342913741500988:11267]
2025-10-09T21:51:16.580787Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:16.580807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:16.580809Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:16.580812Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:16.580799Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2087
2025-10-09T21:51:16.580814Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:16.580802Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2087
2025-10-09T21:51:16.580822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:16.580825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:16.580850Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342913741500988:11267]
2025-10-09T21:51:16.580853Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:16.580855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:16.580879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.580888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.580937Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ea2231e7-4f066fd9-48880aa2-353ebed1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:16.580951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:16.580956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:16.580959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:16.580968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:16.580982Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:16.581002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:16.581160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.581171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.581195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:16.581292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2088 partition: 0 MaxSeqNo: 25 sessionId: src_id|ea2231e7-4f066fd9-48880aa2-353ebed1_0
2025-10-09T21:51:16.581623Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|627c42df-edd81759-99264db8-81add485_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ea2231e7-4f066fd9-48880aa2-353ebed1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:16.583984Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea2231e7-4f066fd9-48880aa2-353ebed1_0] Write session: try to update token
2025-10-09T21:51:16.583999Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea2231e7-4f066fd9-48880aa2-353ebed1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:16.594788Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:16.594800Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:16.594802Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:16.594804Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:16.594814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342913741500988:11267]
2025-10-09T21:51:16.594818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342913741500988:11267]
2025-10-09T21:51:16.594821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342913741500988:11267]
2025-10-09T21:51:16.594865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:16.594872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.594876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:16.594878Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:16.594885Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:16.594893Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:16.594999Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:16.596504Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2088 sessionId: src_id|ea2231e7-4f066fd9-48880aa2-353ebed1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:16.598638Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:16.657150Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342914103335513:5250]
2025-10-09T21:51:16.657402Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 2087, pipe: [1:7559342914103335513:5250]
2025-10-09T21:51:16.742692Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:16.742965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.742980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.743027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342914103335513:5250], now have 1 active actors on pipe
2025-10-09T21:51:16.743239Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:16.743272Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:16.743288Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342914103335513:5250]
2025-10-09T21:51:16.743289Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:16.743323Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:16.743355Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2088 reason: error at write: Disconnected sessionId: src_id|ea2231e7-4f066fd9-48880aa2-353ebed1_0
2025-10-09T21:51:16.743612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2088 sessionId: src_id|ea2231e7-4f066fd9-48880aa2-353ebed1_0 is DEAD
2025-10-09T21:51:16.743698Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea2231e7-4f066fd9-48880aa2-353ebed1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:16.743710Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea2231e7-4f066fd9-48880aa2-353ebed1_0] Write session will restart in 0.006343s
2025-10-09T21:51:16.743738Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea2231e7-4f066fd9-48880aa2-353ebed1_0] Write session: Do CDS request
2025-10-09T21:51:16.743744Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea2231e7-4f066fd9-48880aa2-353ebed1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:16.751488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea2231e7-4f066fd9-48880aa2-353ebed1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:16.751580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:16.751591Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2089
2025-10-09T21:51:16.751695Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2089 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:16.751730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2089 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:16.751738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2089 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:16.751741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:16.752109Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2089 sessionId:  describe result for acl check
2025-10-09T21:51:16.752137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:16.752142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:16.752143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:16.752147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342914103335553:9591] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:16.752580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342914103335553:9591] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:16.755126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342914103335553:9591] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:16.756288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:16.758045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:16.758207Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:16.758216Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:16.758227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:16.758273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:16.758309Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342913741501161:11269]
2025-10-09T21:51:16.758381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342914103335553:9591] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:16.758394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342914103335553:9591] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:16.758397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342914103335553:9591] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:16.758405Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2089 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:16.758882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:16.759168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:16.759207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:16.759509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:16.759558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.759559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.759561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.759562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.759563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.759564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.759565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.759566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.759567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.759572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.759582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.759587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.759591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.759595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.759599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.759603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.759606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.759610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.759616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.759623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.759628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.759634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.759640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.759644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.759649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.759654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.759659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.759664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.759668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.759673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.759678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.759683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.759687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.759692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.759701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.759707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.759738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:16.765746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:16.766985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:16.766996Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:16.766998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:16.767000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:16.767005Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2088 [2:7559342913741501161:11269]
2025-10-09T21:51:16.767013Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:16.767026Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:16.767028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:16.767030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:16.767032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:16.767045Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:16.767048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:16.767054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:16.767107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.767115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.767116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.767118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:16.767180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:16.767186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.767193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.767199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.767214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:16.767229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342914103335516:2432], now have 1 active actors on pipe
2025-10-09T21:51:16.767237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342914103335685:9591], now have 1 active actors on pipe
2025-10-09T21:51:16.767242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342913741501156:11269]
2025-10-09T21:51:16.767245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342913741501156:11269]
2025-10-09T21:51:16.767247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342913741501156:11269]
2025-10-09T21:51:16.767260Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:16.767275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:16.767289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:16.767289Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:16.767292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:16.767293Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:16.767295Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:16.767316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342913741501156:11269]
2025-10-09T21:51:16.767324Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:16.767332Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:16.767310Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2088
2025-10-09T21:51:16.767311Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2088
2025-10-09T21:51:16.767359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.767369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.767398Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dbc00da3-1122f7e3-448ebda4-affe0202_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:16.767405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:16.767409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:16.767412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:16.767421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:16.767431Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:16.767445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:16.767596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.767611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.767634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:16.767724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2089 partition: 0 MaxSeqNo: 25 sessionId: src_id|dbc00da3-1122f7e3-448ebda4-affe0202_0
2025-10-09T21:51:16.768004Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea2231e7-4f066fd9-48880aa2-353ebed1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dbc00da3-1122f7e3-448ebda4-affe0202_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:16.770477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dbc00da3-1122f7e3-448ebda4-affe0202_0] Write session: try to update token
2025-10-09T21:51:16.770493Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dbc00da3-1122f7e3-448ebda4-affe0202_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:16.782336Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:16.782352Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:16.782354Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:16.782356Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:16.782366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342913741501156:11269]
2025-10-09T21:51:16.782371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342913741501156:11269]
2025-10-09T21:51:16.782375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342913741501156:11269]
2025-10-09T21:51:16.782421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:16.782421Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:16.782427Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:16.782429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.782430Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:16.782431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:16.782569Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:16.783858Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2089 sessionId: src_id|dbc00da3-1122f7e3-448ebda4-affe0202_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:16.786161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:16.843528Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342914103335696:5250]
2025-10-09T21:51:16.843892Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 2088, pipe: [1:7559342914103335696:5250]
2025-10-09T21:51:16.931337Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:16.931615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.931628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.931671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342914103335696:5250], now have 1 active actors on pipe
2025-10-09T21:51:16.931883Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:16.931915Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342914103335696:5250]
2025-10-09T21:51:16.931916Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:16.931920Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:16.931948Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:16.931983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2089 reason: error at write: Disconnected sessionId: src_id|dbc00da3-1122f7e3-448ebda4-affe0202_0
2025-10-09T21:51:16.932174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2089 sessionId: src_id|dbc00da3-1122f7e3-448ebda4-affe0202_0 is DEAD
2025-10-09T21:51:16.932324Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbc00da3-1122f7e3-448ebda4-affe0202_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:16.932334Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbc00da3-1122f7e3-448ebda4-affe0202_0] Write session will restart in 0.008720s
2025-10-09T21:51:16.932354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbc00da3-1122f7e3-448ebda4-affe0202_0] Write session: Do CDS request
2025-10-09T21:51:16.932359Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbc00da3-1122f7e3-448ebda4-affe0202_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:16.942531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dbc00da3-1122f7e3-448ebda4-affe0202_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:16.942581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:16.942597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2090
2025-10-09T21:51:16.942760Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2090 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:16.942792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2090 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:16.942798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2090 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:16.942802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:16.943227Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2090 sessionId:  describe result for acl check
2025-10-09T21:51:16.943264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:16.943271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:16.943273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:16.943281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342914103335830:9573] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:16.943759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:16.943851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342914103335830:9573] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:16.945584Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342914103335830:9573] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:16.945864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:16.946022Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:16.946046Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:16.946061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:16.946110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:16.946145Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342913741501335:11273]
2025-10-09T21:51:16.946672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:16.946961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:16.947003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:16.947307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:16.947353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.947358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.947359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.947360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.947361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.947362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.947363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.947364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.947365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.947370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.947379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.947382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.947385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.947388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.947391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.947394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.947397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.947400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.947405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.947413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:16.947419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.947424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:16.947428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.947432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:16.947436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.947440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:16.947443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.947447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:16.947451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.947455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:16.947459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.947464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.947467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.947471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.947481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.947486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.947513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:16.947990Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342914103335830:9573] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:16.948002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342914103335830:9573] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:16.948005Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342914103335830:9573] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:16.948012Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2090 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:16.953555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:16.954594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:16.954609Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:16.954611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:16.954615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:16.954622Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2089 [2:7559342913741501335:11273]
2025-10-09T21:51:16.954636Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:16.954653Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:16.954661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:16.954664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:16.954666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:16.954682Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:16.954693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:16.954704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:16.954891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.954900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.954902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.954905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:16.954967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:16.954973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.954984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:16.955000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:16.955021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:16.955046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342914103335871:9573], now have 1 active actors on pipe
2025-10-09T21:51:16.955049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342914103335701:2432], now have 1 active actors on pipe
2025-10-09T21:51:16.955055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342913741501330:11273]
2025-10-09T21:51:16.955058Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342913741501330:11273]
2025-10-09T21:51:16.955060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342913741501330:11273]
2025-10-09T21:51:16.955072Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:16.955081Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:16.955085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:16.955092Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:16.955094Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:16.955098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:16.955105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:16.955092Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2089
2025-10-09T21:51:16.955134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342913741501330:11273]
2025-10-09T21:51:16.955093Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2089
2025-10-09T21:51:16.955146Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:16.955152Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:16.955168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.955177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.955221Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6e2edbec-38f81923-fa4eac32-8d1b0b83_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:16.955233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:16.955236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:16.955239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:16.955248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:16.955258Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:16.955278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:16.955431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:16.955439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:16.955453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:16.955537Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2090 partition: 0 MaxSeqNo: 25 sessionId: src_id|6e2edbec-38f81923-fa4eac32-8d1b0b83_0
2025-10-09T21:51:16.955834Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dbc00da3-1122f7e3-448ebda4-affe0202_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6e2edbec-38f81923-fa4eac32-8d1b0b83_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:16.958142Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e2edbec-38f81923-fa4eac32-8d1b0b83_0] Write session: try to update token
2025-10-09T21:51:16.958155Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e2edbec-38f81923-fa4eac32-8d1b0b83_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:16.968864Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:16.968879Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:16.968881Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:16.968882Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:16.968890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342913741501330:11273]
2025-10-09T21:51:16.968893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342913741501330:11273]
2025-10-09T21:51:16.968896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342913741501330:11273]
2025-10-09T21:51:16.968936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:16.968941Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:16.968944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:16.968945Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:16.968946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:16.968949Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:16.969070Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:16.970328Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2090 sessionId: src_id|6e2edbec-38f81923-fa4eac32-8d1b0b83_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:16.972595Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:17.031963Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342918398303174:5250]
2025-10-09T21:51:17.032239Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 2089, pipe: [1:7559342918398303174:5250]
2025-10-09T21:51:17.119750Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:17.120016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.120030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.120072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342918398303174:5250], now have 1 active actors on pipe
2025-10-09T21:51:17.120318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342918398303174:5250]
2025-10-09T21:51:17.120320Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:17.120324Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:17.120355Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:17.120364Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:17.120388Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2090 reason: error at write: Disconnected sessionId: src_id|6e2edbec-38f81923-fa4eac32-8d1b0b83_0
2025-10-09T21:51:17.120576Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2090 sessionId: src_id|6e2edbec-38f81923-fa4eac32-8d1b0b83_0 is DEAD
2025-10-09T21:51:17.120740Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e2edbec-38f81923-fa4eac32-8d1b0b83_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:17.120752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e2edbec-38f81923-fa4eac32-8d1b0b83_0] Write session will restart in 0.008231s
2025-10-09T21:51:17.120776Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e2edbec-38f81923-fa4eac32-8d1b0b83_0] Write session: Do CDS request
2025-10-09T21:51:17.120782Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e2edbec-38f81923-fa4eac32-8d1b0b83_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:17.130470Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e2edbec-38f81923-fa4eac32-8d1b0b83_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:17.130568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:17.130582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2091
2025-10-09T21:51:17.130712Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2091 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:17.130744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2091 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:17.130755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2091 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:17.130763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:17.131250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2091 sessionId:  describe result for acl check
2025-10-09T21:51:17.131289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:17.131297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:17.131299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:17.131305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342918398303270:9576] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:17.131818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342918398303270:9576] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:17.133865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:17.134213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342918398303270:9576] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:17.135638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:17.135822Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:17.135834Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:17.135847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:17.135925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:17.135965Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342918036468801:11275]
2025-10-09T21:51:17.136402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342918398303270:9576] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:17.136418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342918398303270:9576] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:17.136420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342918398303270:9576] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:17.136426Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2091 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:17.136553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:17.136889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:17.136943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:17.137273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:17.137335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.137341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.137342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.137343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.137345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.137346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.137347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.137348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.137349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.137355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.137364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.137369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.137373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.137389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.137393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.137397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.137401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.137405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.137412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.137420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.137426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.137432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.137438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.137443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.137447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.137453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.137462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.137467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.137471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.137476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.137481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.137487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.137495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.137500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.137505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.137509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.137534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:17.143886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:17.145112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:17.145127Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:17.145130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:17.145133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:17.145140Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2090 [2:7559342918036468801:11275]
2025-10-09T21:51:17.145157Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:17.145174Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:17.145177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:17.145180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:17.145183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:17.145201Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:17.145212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:17.145219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:17.145284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.145293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.145294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.145296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:17.145348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:17.145350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.145358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.145364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.145374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:17.145407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342918036468796:11275]
2025-10-09T21:51:17.145409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342918036468796:11275]
2025-10-09T21:51:17.145410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342918036468796:11275]
2025-10-09T21:51:17.145416Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:17.145427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:17.145428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342918398303345:9576], now have 1 active actors on pipe
2025-10-09T21:51:17.145430Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:17.145431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342918398303177:2432], now have 1 active actors on pipe
2025-10-09T21:51:17.145432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:17.145437Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:17.145446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:17.145448Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:17.145471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342918036468796:11275]
2025-10-09T21:51:17.145477Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:17.145485Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:17.145510Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2090
2025-10-09T21:51:17.145512Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2090
2025-10-09T21:51:17.145576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.145585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.145610Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|54cfff83-1449cfbd-7fcd9cf5-b8af5743_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:17.145619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:17.145622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:17.145624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:17.145632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:17.145643Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:17.145653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:17.145789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.145797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.145811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:17.145916Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2091 partition: 0 MaxSeqNo: 25 sessionId: src_id|54cfff83-1449cfbd-7fcd9cf5-b8af5743_0
2025-10-09T21:51:17.146253Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e2edbec-38f81923-fa4eac32-8d1b0b83_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|54cfff83-1449cfbd-7fcd9cf5-b8af5743_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:17.148680Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54cfff83-1449cfbd-7fcd9cf5-b8af5743_0] Write session: try to update token
2025-10-09T21:51:17.148698Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54cfff83-1449cfbd-7fcd9cf5-b8af5743_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:17.159876Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:17.159895Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:17.159897Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:17.159898Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:17.159907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342918036468796:11275]
2025-10-09T21:51:17.159911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342918036468796:11275]
2025-10-09T21:51:17.159913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342918036468796:11275]
2025-10-09T21:51:17.159951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:17.159958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.159961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:17.159960Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:17.159965Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:17.159969Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:17.160096Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:17.161772Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2091 sessionId: src_id|54cfff83-1449cfbd-7fcd9cf5-b8af5743_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:17.164231Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:17.220295Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342918398303373:5250]
2025-10-09T21:51:17.220524Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 2090, pipe: [1:7559342918398303373:5250]
2025-10-09T21:51:17.315114Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:17.315439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.315454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.315499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342918398303373:5250], now have 1 active actors on pipe
2025-10-09T21:51:17.315667Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:17.315706Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:17.315708Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:17.315720Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:17.315750Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342918398303373:5250]
2025-10-09T21:51:17.315774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2091 reason: error at write: Disconnected sessionId: src_id|54cfff83-1449cfbd-7fcd9cf5-b8af5743_0
2025-10-09T21:51:17.315956Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2091 sessionId: src_id|54cfff83-1449cfbd-7fcd9cf5-b8af5743_0 is DEAD
2025-10-09T21:51:17.316107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54cfff83-1449cfbd-7fcd9cf5-b8af5743_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:17.316119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54cfff83-1449cfbd-7fcd9cf5-b8af5743_0] Write session will restart in 0.007889s
2025-10-09T21:51:17.316148Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54cfff83-1449cfbd-7fcd9cf5-b8af5743_0] Write session: Do CDS request
2025-10-09T21:51:17.316154Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54cfff83-1449cfbd-7fcd9cf5-b8af5743_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:17.325457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54cfff83-1449cfbd-7fcd9cf5-b8af5743_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:17.325573Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:17.325588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2092
2025-10-09T21:51:17.325721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2092 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:17.325753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2092 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:17.325760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2092 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:17.325764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:17.326250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2092 sessionId:  describe result for acl check
2025-10-09T21:51:17.326284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:17.326290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:17.326292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:17.326298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342918398303490:9548] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:17.326853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342918398303490:9548] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:17.328439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:17.328840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342918398303490:9548] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:17.330473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:17.330663Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:17.330673Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:17.330685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:17.330741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:17.330782Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342918036469010:11289]
2025-10-09T21:51:17.331349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:17.331570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342918398303490:9548] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:17.331592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342918398303490:9548] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:17.331596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342918398303490:9548] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:17.331601Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2092 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:17.331672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:17.331715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:17.331970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:17.332130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.332137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.332138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.332139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.332139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.332140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.332140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.332141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.332142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.332146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.332153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.332156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.332160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.332164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.332167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.332170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.332174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.332178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.332186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.332194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.332201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.332207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.332211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.332217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.332220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.332226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.332230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.332236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.332240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.332245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.332249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.332254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.332258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.332264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.332268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.332273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.332301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:17.338586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:17.339650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:17.339666Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:17.339669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:17.339673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:17.339679Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2091 [2:7559342918036469010:11289]
2025-10-09T21:51:17.339695Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:17.339713Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:17.339718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:17.339721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:17.339723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:17.339741Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:17.339748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:17.339755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:17.339824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.339832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.339834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.339837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:17.339896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:17.339903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.339915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.339926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.339942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:17.339964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342918398303560:9548], now have 1 active actors on pipe
2025-10-09T21:51:17.339972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342918398303389:2432], now have 1 active actors on pipe
2025-10-09T21:51:17.339980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342918036469005:11289]
2025-10-09T21:51:17.339986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342918036469005:11289]
2025-10-09T21:51:17.339989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342918036469005:11289]
2025-10-09T21:51:17.340003Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:17.340020Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:17.340027Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:17.340031Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:17.340034Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:17.340040Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:17.340043Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:17.340051Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2091
2025-10-09T21:51:17.340056Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2091
2025-10-09T21:51:17.340075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342918036469005:11289]
2025-10-09T21:51:17.340084Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:17.340087Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:17.340114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.340124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.340157Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e02ba17f-c5036bb8-88e2d1eb-82a48b57_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:17.340169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:17.340173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:17.340176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:17.340187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:17.340202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:17.340226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:17.340355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.340362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.340376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:17.340444Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2092 partition: 0 MaxSeqNo: 25 sessionId: src_id|e02ba17f-c5036bb8-88e2d1eb-82a48b57_0
2025-10-09T21:51:17.340781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54cfff83-1449cfbd-7fcd9cf5-b8af5743_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e02ba17f-c5036bb8-88e2d1eb-82a48b57_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:17.343246Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e02ba17f-c5036bb8-88e2d1eb-82a48b57_0] Write session: try to update token
2025-10-09T21:51:17.343265Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e02ba17f-c5036bb8-88e2d1eb-82a48b57_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:17.354579Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:17.354595Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:17.354597Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:17.354599Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:17.354608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342918036469005:11289]
2025-10-09T21:51:17.354616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342918036469005:11289]
2025-10-09T21:51:17.354620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342918036469005:11289]
2025-10-09T21:51:17.354665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:17.354674Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:17.354676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.354680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:17.354681Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:17.354685Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:17.354819Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:17.356567Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2092 sessionId: src_id|e02ba17f-c5036bb8-88e2d1eb-82a48b57_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:17.359103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:17.415900Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342918398303570:5250]
2025-10-09T21:51:17.416199Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 2091, pipe: [1:7559342918398303570:5250]
2025-10-09T21:51:17.500451Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:17.500761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.500774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.500817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342918398303570:5250], now have 1 active actors on pipe
2025-10-09T21:51:17.500994Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:17.501070Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342918398303570:5250]
2025-10-09T21:51:17.501075Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:17.501079Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:17.501096Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:17.501143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2092 reason: error at write: Disconnected sessionId: src_id|e02ba17f-c5036bb8-88e2d1eb-82a48b57_0
2025-10-09T21:51:17.501310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2092 sessionId: src_id|e02ba17f-c5036bb8-88e2d1eb-82a48b57_0 is DEAD
2025-10-09T21:51:17.501489Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e02ba17f-c5036bb8-88e2d1eb-82a48b57_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:17.501504Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e02ba17f-c5036bb8-88e2d1eb-82a48b57_0] Write session will restart in 0.005700s
2025-10-09T21:51:17.501531Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e02ba17f-c5036bb8-88e2d1eb-82a48b57_0] Write session: Do CDS request
2025-10-09T21:51:17.501538Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e02ba17f-c5036bb8-88e2d1eb-82a48b57_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:17.508501Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e02ba17f-c5036bb8-88e2d1eb-82a48b57_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:17.508564Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:17.508577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2093
2025-10-09T21:51:17.508696Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2093 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:17.508729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2093 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:17.508739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2093 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:17.508743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:17.509151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2093 sessionId:  describe result for acl check
2025-10-09T21:51:17.509186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:17.509194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:17.509196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:17.509201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342918398303611:9547] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:17.509641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342918398303611:9547] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:17.511769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342918398303611:9547] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:17.513519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:17.515421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:17.515585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342918398303611:9547] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:17.515599Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:17.515598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342918398303611:9547] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:17.515609Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:17.515602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342918398303611:9547] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:17.515621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:17.515610Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2093 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:17.515673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:17.515712Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342918036469183:11291]
2025-10-09T21:51:17.516225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:17.516493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:17.516536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:17.516827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:17.516874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.516880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.516882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.516883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.516884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.516885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.516886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.516887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.516888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.516894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.516905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.516910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.516914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.516919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.516923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.516927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.516931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.516935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.516942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.516951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.516959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.516964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.516971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.516976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.516981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.516987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.516991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.516997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.517002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.517008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.517015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.517022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.517026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.517032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.517041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.517046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.517074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:17.523400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:17.524477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:17.524492Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:17.524494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:17.524497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:17.524504Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2092 [2:7559342918036469183:11291]
2025-10-09T21:51:17.524515Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:17.524531Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:17.524538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:17.524541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:17.524543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:17.524558Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:17.524566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:17.524577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:17.524636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.524644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.524646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.524648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:17.524850Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2092
2025-10-09T21:51:17.524855Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2092
2025-10-09T21:51:17.524873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:17.524882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.524894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.524912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.524933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:17.524960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342918398303742:9547], now have 1 active actors on pipe
2025-10-09T21:51:17.524965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342918398303572:2432], now have 1 active actors on pipe
2025-10-09T21:51:17.524988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.524997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.525023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342918036469178:11291]
2025-10-09T21:51:17.525026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342918036469178:11291]
2025-10-09T21:51:17.525029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342918036469178:11291]
2025-10-09T21:51:17.525041Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:17.525053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:17.525058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:17.525060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:17.525062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:17.525074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:17.525081Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:17.525103Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8930dfb6-27f2330c-5b72c1f6-a38715f9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:17.525115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:17.525123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:17.525126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:17.525136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:17.525151Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:17.525181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342918036469178:11291]
2025-10-09T21:51:17.525195Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:17.525209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:17.525216Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:17.525422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.525430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.525447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:17.525525Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2093 partition: 0 MaxSeqNo: 25 sessionId: src_id|8930dfb6-27f2330c-5b72c1f6-a38715f9_0
2025-10-09T21:51:17.525884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e02ba17f-c5036bb8-88e2d1eb-82a48b57_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8930dfb6-27f2330c-5b72c1f6-a38715f9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:17.528368Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8930dfb6-27f2330c-5b72c1f6-a38715f9_0] Write session: try to update token
2025-10-09T21:51:17.528386Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8930dfb6-27f2330c-5b72c1f6-a38715f9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:17.537125Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:17.537141Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:17.537150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342918036469178:11291]
2025-10-09T21:51:17.537193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:17.537216Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:17.537367Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:17.537523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:17.537663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:17.537813Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:17.538003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:17.538065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:17.541241Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2093 sessionId: src_id|8930dfb6-27f2330c-5b72c1f6-a38715f9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:17.543808Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:17.544214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.544231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.544263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:17.544394Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:17.553196Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:17.553211Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:17.553214Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:17.553216Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:17.553227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342918036469178:11291]
2025-10-09T21:51:17.553230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342918036469178:11291]
2025-10-09T21:51:17.553234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342918036469178:11291]
2025-10-09T21:51:17.553274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:17.553276Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:17.553284Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:17.553287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.553287Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:17.553289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:17.553426Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:17.601226Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342918398303754:5250]
2025-10-09T21:51:17.601525Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 2092, pipe: [1:7559342918398303754:5250]
2025-10-09T21:51:17.691920Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:17.698759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.698780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.698945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.699006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.699069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.699131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.699178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.699186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.699245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.699309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.699370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.699476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.699518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.699526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.699594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.699651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.699713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.699770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.699810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.699817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.699887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.700174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.700237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.700292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.700334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.700341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.700395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.700467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.700536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.700593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.700632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.700636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.700699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.700754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.700814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.700869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.700910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.700918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.700971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.701022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.701089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.701145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.701188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.701193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.701244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.701295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.701348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.701419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.701459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.701465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.701516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.701575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.701635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.701693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.701737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.701742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.701797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.701847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.701908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.701971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.702014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.702019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.702073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342918398303754:5250], now have 1 active actors on pipe
2025-10-09T21:51:17.702318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342918398303754:5250]
2025-10-09T21:51:17.702320Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:17.702357Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:17.702364Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:17.702393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2093 reason: error at write: Disconnected sessionId: src_id|8930dfb6-27f2330c-5b72c1f6-a38715f9_0
2025-10-09T21:51:17.702588Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2093 sessionId: src_id|8930dfb6-27f2330c-5b72c1f6-a38715f9_0 is DEAD
2025-10-09T21:51:17.702741Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8930dfb6-27f2330c-5b72c1f6-a38715f9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:17.702753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8930dfb6-27f2330c-5b72c1f6-a38715f9_0] Write session will restart in 0.009913s
2025-10-09T21:51:17.702777Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8930dfb6-27f2330c-5b72c1f6-a38715f9_0] Write session: Do CDS request
2025-10-09T21:51:17.702783Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8930dfb6-27f2330c-5b72c1f6-a38715f9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:17.702729Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:17.713576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8930dfb6-27f2330c-5b72c1f6-a38715f9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:17.713624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:17.713642Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2094
2025-10-09T21:51:17.713827Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2094 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:17.713860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2094 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:17.713870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2094 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:17.713876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:17.714291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2094 sessionId:  describe result for acl check
2025-10-09T21:51:17.714324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:17.714326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:17.714327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:17.714331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342918398303811:9541] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:17.714856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342918398303811:9541] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:17.717865Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342918398303811:9541] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:17.718918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:17.720875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:17.721068Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:17.721078Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:17.721089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:17.721154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:17.721169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342918398303811:9541] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:17.721182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342918398303811:9541] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:17.721186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342918398303811:9541] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:17.721194Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2094 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:17.721196Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342918036469356:11293]
2025-10-09T21:51:17.721769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:17.722125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:17.722197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:17.722500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:17.722558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.722564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.722566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.722567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.722568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.722569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.722569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.722570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.722571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.722574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.722581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.722583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.722585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.722588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.722590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.722592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.722594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.722596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.722601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.722606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.722611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.722614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.722616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.722620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.722622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.722625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.722628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.722631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.722634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.722637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.722640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.722643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.722647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.722652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.722657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.722661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.722689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:17.728872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:17.729856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:17.729870Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:17.729873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:17.729877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:17.729882Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2093 [2:7559342918036469356:11293]
2025-10-09T21:51:17.729895Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:17.729910Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:17.729912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:17.729914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:17.729916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:17.729927Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:17.729933Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:17.729938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:17.729982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.729989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.729990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.729992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:17.730055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:17.730062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.730069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.730075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.730085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:17.730099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342918036469351:11293]
2025-10-09T21:51:17.730104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342918036469351:11293]
2025-10-09T21:51:17.730106Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342918036469351:11293]
2025-10-09T21:51:17.730111Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:17.730121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:17.730130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:17.730129Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:17.730133Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:17.730135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:17.730136Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:17.730161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342918398303928:9541], now have 1 active actors on pipe
2025-10-09T21:51:17.730169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342918398303756:2432], now have 1 active actors on pipe
2025-10-09T21:51:17.730171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342918036469351:11293]
2025-10-09T21:51:17.730177Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:17.730208Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:17.730259Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2093
2025-10-09T21:51:17.730266Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2093
2025-10-09T21:51:17.730521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.730536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.730573Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b68b3739-552c6294-c255abd4-8a97a093_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:17.730587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:17.730596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:17.730599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:17.730611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:17.730636Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:17.730657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:17.730883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.730893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.730913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:17.731002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2094 partition: 0 MaxSeqNo: 25 sessionId: src_id|b68b3739-552c6294-c255abd4-8a97a093_0
2025-10-09T21:51:17.731317Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8930dfb6-27f2330c-5b72c1f6-a38715f9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b68b3739-552c6294-c255abd4-8a97a093_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:17.733839Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b68b3739-552c6294-c255abd4-8a97a093_0] Write session: try to update token
2025-10-09T21:51:17.733856Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b68b3739-552c6294-c255abd4-8a97a093_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:17.742125Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:17.742138Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:17.742147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342918036469351:11293]
2025-10-09T21:51:17.742191Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:17.742194Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:17.742345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:17.742476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:17.742614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:17.742742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:17.742916Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:17.742956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:17.746838Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2094 sessionId: src_id|b68b3739-552c6294-c255abd4-8a97a093_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:17.749231Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:17.749437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.749453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.749486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:17.749579Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:17.755345Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:17.755359Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:17.755361Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:17.755362Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:17.755370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342918036469351:11293]
2025-10-09T21:51:17.755373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342918036469351:11293]
2025-10-09T21:51:17.755375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342918036469351:11293]
2025-10-09T21:51:17.755408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:17.755416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.755419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:17.755421Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:17.755427Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:17.755431Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:17.755557Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:17.802476Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342918398303939:5250]
2025-10-09T21:51:17.802747Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 2093, pipe: [1:7559342918398303939:5250]
2025-10-09T21:51:17.898965Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:17.905720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.905742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.905878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.905929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.905983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.906054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.906409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.906420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.906478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.906531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.906584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.906649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.906688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.906701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.906764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.906822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.906884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.906941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.906982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.906989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.907051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.907113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.907173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.907234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.907276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.907283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.907342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.907401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.907461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.907527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.907569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.907578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.907636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.907695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.907752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.907808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.907849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.907855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.907909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.907957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.908011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.908064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.908101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.908106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.908156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.908204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.908254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.908308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.908345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.908350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.908399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.908452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.908508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.908558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.908594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.908599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.908684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:17.908735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:17.908790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:17.908840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:17.908876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:17.908882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:17.908918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342918398303939:5250], now have 1 active actors on pipe
2025-10-09T21:51:17.909193Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342918398303939:5250]
2025-10-09T21:51:17.909210Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:17.909213Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:17.909226Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:17.909244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2094 reason: error at write: Disconnected sessionId: src_id|b68b3739-552c6294-c255abd4-8a97a093_0
2025-10-09T21:51:17.909261Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:17.909459Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2094 sessionId: src_id|b68b3739-552c6294-c255abd4-8a97a093_0 is DEAD
2025-10-09T21:51:17.909588Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b68b3739-552c6294-c255abd4-8a97a093_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:17.909600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b68b3739-552c6294-c255abd4-8a97a093_0] Write session will restart in 0.009316s
2025-10-09T21:51:17.909626Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b68b3739-552c6294-c255abd4-8a97a093_0] Write session: Do CDS request
2025-10-09T21:51:17.909632Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b68b3739-552c6294-c255abd4-8a97a093_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:17.920468Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b68b3739-552c6294-c255abd4-8a97a093_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:17.920562Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:17.920573Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2095
2025-10-09T21:51:17.920679Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2095 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:17.920702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2095 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:17.920710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2095 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:17.920712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:17.921104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2095 sessionId:  describe result for acl check
2025-10-09T21:51:17.921140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:17.921146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:17.921148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:17.921154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342918398304081:9538] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:17.921324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:17.921675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342918398304081:9538] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:17.923508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:17.923660Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:17.923668Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:17.923677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:17.923717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:17.923747Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342918036469534:11297]
2025-10-09T21:51:17.923892Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342918398304081:9538] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:17.924146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:17.924404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:17.924436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:17.924675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:17.924710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.924715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.924716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.924717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.924718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.924718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.924719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.924719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.924720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.924723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.924729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.924731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.924733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.924735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.924737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.924739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.924741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.924743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.924747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.924751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:17.924755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.924759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:17.924761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.924764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:17.924766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.924769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:17.924772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.924774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:17.924777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.924780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:17.924783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.924793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.924796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.924799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.924801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.924804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.924821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:17.929505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342918398304081:9538] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:17.929521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342918398304081:9538] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:17.929525Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342918398304081:9538] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:17.929533Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2095 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:17.930705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:17.931676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:17.931690Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:17.931691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:17.931695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:17.931699Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2094 [2:7559342918036469534:11297]
2025-10-09T21:51:17.931708Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:17.931721Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:17.931723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:17.931725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:17.931726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:17.931738Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:17.931741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:17.931747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:17.931786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.931791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.931792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.931794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:17.931853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:17.931858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.931866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:17.931872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:17.931884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:17.931900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342918398304119:9538], now have 1 active actors on pipe
2025-10-09T21:51:17.931906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342918398303944:2432], now have 1 active actors on pipe
2025-10-09T21:51:17.931911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342918036469529:11297]
2025-10-09T21:51:17.931912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342918036469529:11297]
2025-10-09T21:51:17.931913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342918036469529:11297]
2025-10-09T21:51:17.931919Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:17.931929Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:17.931931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:17.931938Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:17.931939Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:17.931940Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:17.931941Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:17.931959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342918036469529:11297]
2025-10-09T21:51:17.931965Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:17.931967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:17.931986Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2094
2025-10-09T21:51:17.931991Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2094
2025-10-09T21:51:17.932042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.932050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.932073Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3408454-36e6b8b0-4b87b7c1-eb9165ae_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:17.932081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:17.932084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:17.932085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:17.932092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:17.932102Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:17.932122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:17.932286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:17.932297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:17.932317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:17.932433Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2095 partition: 0 MaxSeqNo: 25 sessionId: src_id|3408454-36e6b8b0-4b87b7c1-eb9165ae_0
2025-10-09T21:51:17.932729Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b68b3739-552c6294-c255abd4-8a97a093_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3408454-36e6b8b0-4b87b7c1-eb9165ae_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:17.935045Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3408454-36e6b8b0-4b87b7c1-eb9165ae_0] Write session: try to update token
2025-10-09T21:51:17.935061Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3408454-36e6b8b0-4b87b7c1-eb9165ae_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:17.945446Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:17.945460Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:17.945463Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:17.945465Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:17.945474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342918036469529:11297]
2025-10-09T21:51:17.945479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342918036469529:11297]
2025-10-09T21:51:17.945481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342918036469529:11297]
2025-10-09T21:51:17.945518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:17.945523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:17.945525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:17.945544Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:17.945550Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:17.945554Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:17.945651Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:17.946830Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2095 sessionId: src_id|3408454-36e6b8b0-4b87b7c1-eb9165ae_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:17.949238Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:18.009354Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342922693271423:5250]
2025-10-09T21:51:18.009660Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 2094, pipe: [1:7559342922693271423:5250]
2025-10-09T21:51:18.097662Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:18.097951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.097965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.098013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342922693271423:5250], now have 1 active actors on pipe
2025-10-09T21:51:18.098219Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:18.098232Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:18.098240Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:18.098298Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2095 reason: error at write: Disconnected sessionId: src_id|3408454-36e6b8b0-4b87b7c1-eb9165ae_0
2025-10-09T21:51:18.098305Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342922693271423:5250]
2025-10-09T21:51:18.098465Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2095 sessionId: src_id|3408454-36e6b8b0-4b87b7c1-eb9165ae_0 is DEAD
2025-10-09T21:51:18.098541Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:18.098608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3408454-36e6b8b0-4b87b7c1-eb9165ae_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:18.098618Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3408454-36e6b8b0-4b87b7c1-eb9165ae_0] Write session will restart in 0.007260s
2025-10-09T21:51:18.098638Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3408454-36e6b8b0-4b87b7c1-eb9165ae_0] Write session: Do CDS request
2025-10-09T21:51:18.098643Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3408454-36e6b8b0-4b87b7c1-eb9165ae_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:18.107496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3408454-36e6b8b0-4b87b7c1-eb9165ae_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:18.107573Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:18.107592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2096
2025-10-09T21:51:18.107727Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2096 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:18.107754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2096 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:18.107758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2096 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:18.107761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:18.108182Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2096 sessionId:  describe result for acl check
2025-10-09T21:51:18.108223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:18.108231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:18.108233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:18.108238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342922693271476:9532] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:18.108789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342922693271476:9532] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:18.111512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342922693271476:9532] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:18.112477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:18.114330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:18.114529Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:18.114539Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:18.114552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:18.114607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:18.114646Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342922331437005:11299]
2025-10-09T21:51:18.114654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342922693271476:9532] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:18.114666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342922693271476:9532] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:18.114670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342922693271476:9532] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:18.114678Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2096 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:18.115231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:18.115551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:18.115593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:18.115907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:18.115955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.115960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.115962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.115963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.115964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.115965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.115966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.115967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.115969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.115973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.115984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.115988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.115993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.115996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.116001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.116004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.116008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.116012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.116019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.116027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.116033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.116041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.116046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.116056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.116060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.116065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.116074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.116078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.116082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.116087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.116094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.116099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.116105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.116110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.116115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.116121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.116155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:18.122501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:18.123631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:18.123644Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:18.123645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:18.123648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:18.123653Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2095 [2:7559342922331437005:11299]
2025-10-09T21:51:18.123664Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:18.123679Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:18.123682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:18.123685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:18.123687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:18.123703Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:18.123709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:18.123716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:18.123772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.123783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.123785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.123788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:18.123840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:18.123846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.123854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.123860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.123870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:18.123887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342922331437000:11299]
2025-10-09T21:51:18.123891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342922331437000:11299]
2025-10-09T21:51:18.123893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342922331437000:11299]
2025-10-09T21:51:18.123899Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:18.123912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342922693271426:2432], now have 1 active actors on pipe
2025-10-09T21:51:18.123912Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:18.123915Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:18.123917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:18.123917Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:18.123925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:18.123929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:18.123933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342922693271599:9532], now have 1 active actors on pipe
2025-10-09T21:51:18.123953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342922331437000:11299]
2025-10-09T21:51:18.123962Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:18.123973Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:18.123986Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2095
2025-10-09T21:51:18.124005Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2095
2025-10-09T21:51:18.124055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.124063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.124086Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4fd7e6b2-73695025-16efa1b-29077e16_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:18.124092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:18.124099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:18.124101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:18.124108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:18.124116Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:18.124126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:18.124331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.124343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.124366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:18.124457Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2096 partition: 0 MaxSeqNo: 25 sessionId: src_id|4fd7e6b2-73695025-16efa1b-29077e16_0
2025-10-09T21:51:18.124758Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3408454-36e6b8b0-4b87b7c1-eb9165ae_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4fd7e6b2-73695025-16efa1b-29077e16_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:18.127213Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4fd7e6b2-73695025-16efa1b-29077e16_0] Write session: try to update token
2025-10-09T21:51:18.127233Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4fd7e6b2-73695025-16efa1b-29077e16_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:18.135814Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:18.135830Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:18.135840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342922331437000:11299]
2025-10-09T21:51:18.135888Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:18.135899Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:18.136058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:18.136202Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:18.136341Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:18.136484Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:18.136673Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:18.136717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:18.139516Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2096 sessionId: src_id|4fd7e6b2-73695025-16efa1b-29077e16_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:18.141804Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:18.142046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.142066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.142103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:18.142198Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:18.148929Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:18.148944Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:18.148947Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:18.148948Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:18.148959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342922331437000:11299]
2025-10-09T21:51:18.148962Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342922331437000:11299]
2025-10-09T21:51:18.148966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342922331437000:11299]
2025-10-09T21:51:18.149007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:18.149012Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:18.149019Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:18.149020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.149023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:18.149023Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:18.149155Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:18.198265Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342922693271610:5250]
2025-10-09T21:51:18.198575Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 2095, pipe: [1:7559342922693271610:5250]
2025-10-09T21:51:18.296606Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:18.303034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.303053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.303191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.303248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.303310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.303383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.303426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.303430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.303492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.303548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.303611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.303676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.303714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.303720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.303775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.303824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.303887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.303943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.303987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.303996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.304058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.304118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.304179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.304235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.304274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.304279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.304336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.304400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.304472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.304531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.304570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.304576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.304639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.304697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.304752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.304806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.304850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.304860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.304922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.304983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.305295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.305347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.305397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.305405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.305458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.305509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.305568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.305626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.305672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.305679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.305747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.305798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.305851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.305921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.305962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.305969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.306024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.306085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.306137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.306201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.306247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.306252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.306292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342922693271610:5250], now have 1 active actors on pipe
2025-10-09T21:51:18.306510Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342922693271610:5250]
2025-10-09T21:51:18.306533Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:18.306538Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:18.306542Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:18.306570Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2096 reason: error at write: Disconnected sessionId: src_id|4fd7e6b2-73695025-16efa1b-29077e16_0
2025-10-09T21:51:18.306787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2096 sessionId: src_id|4fd7e6b2-73695025-16efa1b-29077e16_0 is DEAD
2025-10-09T21:51:18.306833Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:18.306912Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4fd7e6b2-73695025-16efa1b-29077e16_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:18.306924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4fd7e6b2-73695025-16efa1b-29077e16_0] Write session will restart in 0.006348s
2025-10-09T21:51:18.306948Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4fd7e6b2-73695025-16efa1b-29077e16_0] Write session: Do CDS request
2025-10-09T21:51:18.306954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4fd7e6b2-73695025-16efa1b-29077e16_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:18.314370Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4fd7e6b2-73695025-16efa1b-29077e16_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:18.314558Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:18.314571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2097
2025-10-09T21:51:18.315122Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2097 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:18.315159Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2097 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:18.315170Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2097 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:18.315184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:18.315577Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2097 sessionId:  describe result for acl check
2025-10-09T21:51:18.315608Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:18.315613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:18.315614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:18.315618Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342922693271689:9512] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:18.316053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342922693271689:9512] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:18.318180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342922693271689:9512] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:18.319698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:18.321942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:18.322126Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:18.322137Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:18.322150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:18.322211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:18.322250Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342922331437219:11313]
2025-10-09T21:51:18.322505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342922693271689:9512] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:18.322520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342922693271689:9512] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:18.322523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342922693271689:9512] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:18.322531Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2097 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:18.322845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:18.323191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:18.323238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:18.323544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:18.323590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.323595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.323597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.323598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.323599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.323600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.323601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.323602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.323604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.323608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.323618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.323622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.323626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.323629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.323632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.323635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.323639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.323642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.323648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.323656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.323662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.323667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.323671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.323676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.323680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.323685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.323689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.323694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.323698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.323703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.323707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.323712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.323716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.323726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.323731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.323736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.323767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:18.329607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:18.330782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:18.330798Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:18.330800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:18.330804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:18.330810Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2096 [2:7559342922331437219:11313]
2025-10-09T21:51:18.330822Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:18.330837Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:18.330839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:18.330841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:18.330842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:18.330855Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:18.330865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:18.330872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:18.330910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.330917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.330918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.330920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:18.330970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:18.330977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.330983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.330989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.330998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:18.331012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342922693271817:9512], now have 1 active actors on pipe
2025-10-09T21:51:18.331017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342922331437214:11313]
2025-10-09T21:51:18.331019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342922331437214:11313]
2025-10-09T21:51:18.331021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342922331437214:11313]
2025-10-09T21:51:18.331034Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:18.331043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342922693271646:2432], now have 1 active actors on pipe
2025-10-09T21:51:18.331042Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:18.331045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:18.331046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:18.331050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:18.331059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:18.331061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:18.331078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342922331437214:11313]
2025-10-09T21:51:18.331088Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:18.331092Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:18.331128Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2096
2025-10-09T21:51:18.331139Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2096
2025-10-09T21:51:18.331209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.331217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.331239Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bc76d183-c4a759c2-4e4b3389-b8ede69f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:18.331248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:18.331251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:18.331252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:18.331260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:18.331271Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:18.331285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:18.331460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.331469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.331483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:18.331586Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2097 partition: 0 MaxSeqNo: 25 sessionId: src_id|bc76d183-c4a759c2-4e4b3389-b8ede69f_0
2025-10-09T21:51:18.331910Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4fd7e6b2-73695025-16efa1b-29077e16_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bc76d183-c4a759c2-4e4b3389-b8ede69f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:18.334559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc76d183-c4a759c2-4e4b3389-b8ede69f_0] Write session: try to update token
2025-10-09T21:51:18.334574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc76d183-c4a759c2-4e4b3389-b8ede69f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:18.341986Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:18.342001Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:18.342012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342922331437214:11313]
2025-10-09T21:51:18.342064Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:18.342077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:18.342243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:18.342402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:18.342536Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:18.342690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:18.342876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:18.342922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:18.347125Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2097 sessionId: src_id|bc76d183-c4a759c2-4e4b3389-b8ede69f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:18.349363Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:18.349575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.349594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.349639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:18.349718Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:18.355076Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:18.355087Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:18.355089Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:18.355090Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:18.355100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342922331437214:11313]
2025-10-09T21:51:18.355103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342922331437214:11313]
2025-10-09T21:51:18.355106Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342922331437214:11313]
2025-10-09T21:51:18.355138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:18.355144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.355143Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:18.355145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:18.355148Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:18.355151Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:18.355264Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:18.406635Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342922693271829:5250]
2025-10-09T21:51:18.406882Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 2096, pipe: [1:7559342922693271829:5250]
2025-10-09T21:51:18.499499Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:18.505840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.505860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.506013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.506090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.506142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.506196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.506237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.506244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.506300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.506350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.506401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.506459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.506503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.506510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.506571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.506618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.506668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.506721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.506759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.506764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.506817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.506869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.506922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.506975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.507012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.507018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.507070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.507122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.507173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.507223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.507258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.507263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.507312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.507361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.507410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.507464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.507501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.507507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.507556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.507604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.507651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.507698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.507734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.507741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.507790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.507846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.507905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.507963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.507999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.508004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.508060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.508111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.508160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.508210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.508246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.508251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.508303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.508351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.508401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.508462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.508497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.508503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.508539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342922693271829:5250], now have 1 active actors on pipe
2025-10-09T21:51:18.508771Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:18.508785Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:18.508817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2097 reason: error at write: Disconnected sessionId: src_id|bc76d183-c4a759c2-4e4b3389-b8ede69f_0
2025-10-09T21:51:18.508915Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:18.508970Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342922693271829:5250]
2025-10-09T21:51:18.508991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2097 sessionId: src_id|bc76d183-c4a759c2-4e4b3389-b8ede69f_0 is DEAD
2025-10-09T21:51:18.509140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc76d183-c4a759c2-4e4b3389-b8ede69f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:18.509151Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc76d183-c4a759c2-4e4b3389-b8ede69f_0] Write session will restart in 0.006367s
2025-10-09T21:51:18.509176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc76d183-c4a759c2-4e4b3389-b8ede69f_0] Write session: Do CDS request
2025-10-09T21:51:18.509181Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc76d183-c4a759c2-4e4b3389-b8ede69f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:18.509328Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:18.516491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc76d183-c4a759c2-4e4b3389-b8ede69f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:18.516571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:18.516584Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2098
2025-10-09T21:51:18.516723Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2098 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:18.516770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2098 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:18.516781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2098 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:18.516785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:18.517191Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2098 sessionId:  describe result for acl check
2025-10-09T21:51:18.517221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:18.517226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:18.517227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:18.517231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342922693271870:9509] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:18.517762Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342922693271870:9509] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:18.520060Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342922693271870:9509] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:18.522006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:18.523731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342922693271870:9509] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:18.523743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342922693271870:9509] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:18.523747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342922693271870:9509] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:18.523754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2098 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:18.523772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:18.523910Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:18.523918Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:18.523925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:18.523960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:18.523988Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342922331437398:11315]
2025-10-09T21:51:18.524335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:18.524558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:18.524588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:18.524799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:18.524827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.524831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.524832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.524833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.524833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.524834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.524835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.524835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.524836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.524839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.524846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.524848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.524850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.524852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.524854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.524856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.524858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.524860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.524864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.524868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.524872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.524875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.524878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.524881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.524883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.524886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.524890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.524892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.524895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.524898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.524901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.524904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.524906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.524909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.524911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.524914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.524929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:18.530731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:18.531898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:18.531911Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:18.531914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:18.531917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:18.531924Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2097 [2:7559342922331437398:11315]
2025-10-09T21:51:18.531937Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:18.531954Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:18.531957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:18.531960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:18.531962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:18.531979Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:18.531985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:18.531995Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:18.532047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.532055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.532057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.532059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:18.532120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:18.532126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.532139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.532154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.532171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:18.532196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342922693272006:9509], now have 1 active actors on pipe
2025-10-09T21:51:18.532200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342922693271831:2432], now have 1 active actors on pipe
2025-10-09T21:51:18.532207Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342922331437392:11315]
2025-10-09T21:51:18.532213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342922331437392:11315]
2025-10-09T21:51:18.532215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342922331437392:11315]
2025-10-09T21:51:18.532229Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:18.532246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:18.532247Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:18.532250Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:18.532251Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:18.532261Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:18.532264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:18.532287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342922331437392:11315]
2025-10-09T21:51:18.532290Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:18.532256Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2097
2025-10-09T21:51:18.532290Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:18.532346Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2097
2025-10-09T21:51:18.532384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.532394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.532430Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|eea705cc-bab27fab-4d12ae85-54f9a108_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:18.532441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:18.532446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:18.532449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:18.532459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:18.532472Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:18.532489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:18.532677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.532683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.532697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:18.532779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2098 partition: 0 MaxSeqNo: 25 sessionId: src_id|eea705cc-bab27fab-4d12ae85-54f9a108_0
2025-10-09T21:51:18.533087Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc76d183-c4a759c2-4e4b3389-b8ede69f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|eea705cc-bab27fab-4d12ae85-54f9a108_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:18.535494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eea705cc-bab27fab-4d12ae85-54f9a108_0] Write session: try to update token
2025-10-09T21:51:18.535510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eea705cc-bab27fab-4d12ae85-54f9a108_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:18.543316Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:18.543331Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:18.543340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342922331437392:11315]
2025-10-09T21:51:18.543387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:18.543386Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:18.543556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:18.543688Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:18.543815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:18.543952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:18.544130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:18.544166Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:18.548260Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2098 sessionId: src_id|eea705cc-bab27fab-4d12ae85-54f9a108_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:18.550497Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:18.550699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.550717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.550754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:18.550844Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:18.556354Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:18.556365Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:18.556366Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:18.556368Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:18.556375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342922331437392:11315]
2025-10-09T21:51:18.556378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342922331437392:11315]
2025-10-09T21:51:18.556381Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342922331437392:11315]
2025-10-09T21:51:18.556419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:18.556422Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:18.556425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.556426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:18.556427Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:18.556430Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:18.556550Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:18.609402Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342922693272018:5250]
2025-10-09T21:51:18.609686Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 2097, pipe: [1:7559342922693272018:5250]
2025-10-09T21:51:18.705091Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:18.712544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.712566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.712719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.712776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.712837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.712896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.712940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.712949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.713011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.713070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.713130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.713457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.713501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.713508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.713565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.713620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.713676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.713727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.713765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.713773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.713834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.713893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.713946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.714003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.714066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.714073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.714133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.714187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.714244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.714301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.714338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.714345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.714402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.714460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.714517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.714579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.714622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.714628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.714687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.714739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.714795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.714854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.714892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.714899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.714952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.715004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.715063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.715116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.715155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.715162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.715219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.715274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.715331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.715387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.715428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.715435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.715488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:18.715540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:18.715599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:18.715654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:18.715694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:18.715700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:18.715755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342922693272018:5250], now have 1 active actors on pipe
2025-10-09T21:51:18.715994Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:18.716026Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342922693272018:5250]
2025-10-09T21:51:18.716051Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:18.716053Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:18.716085Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2098 reason: error at write: Disconnected sessionId: src_id|eea705cc-bab27fab-4d12ae85-54f9a108_0
2025-10-09T21:51:18.716269Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2098 sessionId: src_id|eea705cc-bab27fab-4d12ae85-54f9a108_0 is DEAD
2025-10-09T21:51:18.716405Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eea705cc-bab27fab-4d12ae85-54f9a108_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:18.716417Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eea705cc-bab27fab-4d12ae85-54f9a108_0] Write session will restart in 0.008621s
2025-10-09T21:51:18.716408Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:18.716441Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eea705cc-bab27fab-4d12ae85-54f9a108_0] Write session: Do CDS request
2025-10-09T21:51:18.716446Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eea705cc-bab27fab-4d12ae85-54f9a108_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:18.726526Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eea705cc-bab27fab-4d12ae85-54f9a108_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:18.726623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:18.726638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2099
2025-10-09T21:51:18.726779Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2099 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:18.726810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2099 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:18.726821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2099 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:18.726824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:18.727248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2099 sessionId:  describe result for acl check
2025-10-09T21:51:18.727290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:18.727296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:18.727298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:18.727304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342922693272082:9503] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:18.727835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342922693272082:9503] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:18.730936Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342922693272082:9503] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:18.731300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:18.733111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:18.733231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342922693272082:9503] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:18.733244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342922693272082:9503] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:18.733247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342922693272082:9503] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:18.733255Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2099 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:18.733267Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:18.733278Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:18.733290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:18.733356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:18.733405Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342922331437575:11317]
2025-10-09T21:51:18.733883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:18.734096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:18.734124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:18.734343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:18.734376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.734381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.734381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.734382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.734382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.734383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.734383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.734384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.734384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.734387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.734394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.734396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.734398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.734400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.734402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.734404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.734405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.734408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.734412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.734417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.734421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.734424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.734426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.734429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.734431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.734433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.734436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.734438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.734441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.734444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.734447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.734450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.734452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.734455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.734457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.734460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.734474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:18.740323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:18.741427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:18.741439Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:18.741441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:18.741445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:18.741452Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2098 [2:7559342922331437575:11317]
2025-10-09T21:51:18.741460Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:18.741473Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:18.741479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:18.741481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:18.741482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:18.741494Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:18.741502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:18.741507Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:18.741545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.741554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.741555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.741558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:18.741620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:18.741625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.741632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.741641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.741654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:18.741672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342922693272021:2432], now have 1 active actors on pipe
2025-10-09T21:51:18.741678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342922693272197:9503], now have 1 active actors on pipe
2025-10-09T21:51:18.741683Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342922331437570:11317]
2025-10-09T21:51:18.741685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342922331437570:11317]
2025-10-09T21:51:18.741686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342922331437570:11317]
2025-10-09T21:51:18.741695Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:18.741706Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:18.741707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:18.741712Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:18.741714Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:18.741715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:18.741717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:18.741736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342922331437570:11317]
2025-10-09T21:51:18.741741Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:18.741743Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:18.741725Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2098
2025-10-09T21:51:18.741733Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2098
2025-10-09T21:51:18.741769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.741777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.741801Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b1d81c01-2324a97f-83d2958a-adf02996_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:18.741807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:18.741810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:18.741812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:18.741820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:18.741828Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:18.741837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:18.741962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.741969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.741979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:18.742077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2099 partition: 0 MaxSeqNo: 25 sessionId: src_id|b1d81c01-2324a97f-83d2958a-adf02996_0
2025-10-09T21:51:18.742401Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eea705cc-bab27fab-4d12ae85-54f9a108_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b1d81c01-2324a97f-83d2958a-adf02996_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:18.744770Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1d81c01-2324a97f-83d2958a-adf02996_0] Write session: try to update token
2025-10-09T21:51:18.744791Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1d81c01-2324a97f-83d2958a-adf02996_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:18.755204Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:18.755232Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:18.755235Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:18.755237Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:18.755246Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342922331437570:11317]
2025-10-09T21:51:18.755251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342922331437570:11317]
2025-10-09T21:51:18.755254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342922331437570:11317]
2025-10-09T21:51:18.755294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:18.755300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.755302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:18.755302Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:18.755307Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:18.755310Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:18.755421Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:18.757636Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2099 sessionId: src_id|b1d81c01-2324a97f-83d2958a-adf02996_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:18.759709Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:18.816244Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342922693272208:5250]
2025-10-09T21:51:18.816454Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 2098, pipe: [1:7559342922693272208:5250]
2025-10-09T21:51:18.899828Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:18.900154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.900165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.900201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342922693272208:5250], now have 1 active actors on pipe
2025-10-09T21:51:18.900399Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:18.900484Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:18.900514Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342922693272208:5250]
2025-10-09T21:51:18.900532Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:18.900603Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2099 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b1d81c01-2324a97f-83d2958a-adf02996_0
2025-10-09T21:51:18.900785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2099 sessionId: src_id|b1d81c01-2324a97f-83d2958a-adf02996_0 is DEAD
2025-10-09T21:51:18.900949Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1d81c01-2324a97f-83d2958a-adf02996_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:18.900959Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1d81c01-2324a97f-83d2958a-adf02996_0] Write session will restart in 0.007476s
2025-10-09T21:51:18.900979Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1d81c01-2324a97f-83d2958a-adf02996_0] Write session: Do CDS request
2025-10-09T21:51:18.900984Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1d81c01-2324a97f-83d2958a-adf02996_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:18.909437Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1d81c01-2324a97f-83d2958a-adf02996_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:18.909563Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:18.909579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2100
2025-10-09T21:51:18.909719Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2100 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:18.909751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2100 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:18.909761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2100 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:18.909765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:18.910261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2100 sessionId:  describe result for acl check
2025-10-09T21:51:18.910307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:18.910314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:18.910316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:18.910321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342922693272267:9497] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:18.910832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342922693272267:9497] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:18.913502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342922693272267:9497] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:18.915286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:18.917212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:18.917392Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:18.917404Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:18.917418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:18.917430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342922693272267:9497] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:18.917443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342922693272267:9497] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:18.917446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342922693272267:9497] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:18.917454Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2100 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:18.917472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:18.917511Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342922331437758:11321]
2025-10-09T21:51:18.918071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:18.918322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:18.918362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:18.918652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:18.918712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.918718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.918719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.918721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.918722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.918723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.918724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.918725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.918726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.918731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.918741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.918749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.918753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.918756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.918764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.918767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.918771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.918775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.918785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.918793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:18.918803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.918813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:18.918822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.918827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:18.918835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.918840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:18.918849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.918857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:18.918866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.918870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:18.918880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.918889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.918897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.918902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.918910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.918916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.918949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:18.924702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:18.925682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:18.925694Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:18.925695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:18.925699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:18.925704Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2099 [2:7559342922331437758:11321]
2025-10-09T21:51:18.925713Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:18.925727Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:18.925735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:18.925738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:18.925740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:18.925756Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:18.925764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:18.925771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:18.925814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.925820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.925821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.925823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:18.925875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:18.925877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.925884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:18.925891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:18.925901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:18.925918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342922331437753:11321]
2025-10-09T21:51:18.925923Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342922331437753:11321]
2025-10-09T21:51:18.925924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342922331437753:11321]
2025-10-09T21:51:18.925931Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:18.925939Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:18.925941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342922693272390:9497], now have 1 active actors on pipe
2025-10-09T21:51:18.925946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342922693272213:2432], now have 1 active actors on pipe
2025-10-09T21:51:18.925946Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:18.925948Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:18.925949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:18.925958Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:18.925960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:18.925976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342922331437753:11321]
2025-10-09T21:51:18.925987Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:18.926009Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:18.926011Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2099
2025-10-09T21:51:18.926024Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2099
2025-10-09T21:51:18.926124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.926133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.926157Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|463005d8-3c562af2-7fd6a909-e53d89ef_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:18.926166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:18.926171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:18.926173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:18.926180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:18.926191Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:18.926207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:18.926333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:18.926341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:18.926356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:18.926495Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2100 partition: 0 MaxSeqNo: 25 sessionId: src_id|463005d8-3c562af2-7fd6a909-e53d89ef_0
2025-10-09T21:51:18.926816Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1d81c01-2324a97f-83d2958a-adf02996_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|463005d8-3c562af2-7fd6a909-e53d89ef_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:18.929088Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|463005d8-3c562af2-7fd6a909-e53d89ef_0] Write session: try to update token
2025-10-09T21:51:18.929101Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|463005d8-3c562af2-7fd6a909-e53d89ef_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:18.938990Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:18.939007Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:18.939010Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:18.939012Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:18.939022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342922331437753:11321]
2025-10-09T21:51:18.939025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342922331437753:11321]
2025-10-09T21:51:18.939029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342922331437753:11321]
2025-10-09T21:51:18.939071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:18.939071Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:18.939076Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:18.939083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:18.939086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:18.939088Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:18.939210Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:18.941379Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2100 sessionId: src_id|463005d8-3c562af2-7fd6a909-e53d89ef_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:18.943647Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:19.000571Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342926988239697:5250]
2025-10-09T21:51:19.000801Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 2099, pipe: [1:7559342926988239697:5250]
2025-10-09T21:51:19.084726Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:19.085280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.085295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.085337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342926988239697:5250], now have 1 active actors on pipe
2025-10-09T21:51:19.085598Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:19.085605Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:19.085612Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:19.085659Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342926988239697:5250]
2025-10-09T21:51:19.085672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2100 reason: error at write: Disconnected sessionId: src_id|463005d8-3c562af2-7fd6a909-e53d89ef_0
2025-10-09T21:51:19.085669Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:19.085876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2100 sessionId: src_id|463005d8-3c562af2-7fd6a909-e53d89ef_0 is DEAD
2025-10-09T21:51:19.086049Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|463005d8-3c562af2-7fd6a909-e53d89ef_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:19.086060Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|463005d8-3c562af2-7fd6a909-e53d89ef_0] Write session will restart in 0.006467s
2025-10-09T21:51:19.086096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|463005d8-3c562af2-7fd6a909-e53d89ef_0] Write session: Do CDS request
2025-10-09T21:51:19.086102Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|463005d8-3c562af2-7fd6a909-e53d89ef_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:19.093571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|463005d8-3c562af2-7fd6a909-e53d89ef_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:19.093698Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:19.093714Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2101
2025-10-09T21:51:19.093855Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2101 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:19.093892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2101 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:19.093902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2101 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:19.093906Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:19.094362Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2101 sessionId:  describe result for acl check
2025-10-09T21:51:19.094399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:19.094405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:19.094407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:19.094412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342926988239751:9485] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:19.095029Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342926988239751:9485] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:19.097596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342926988239751:9485] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:19.098401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:19.100345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:19.100554Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:19.100565Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:19.100577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:19.100639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:19.100683Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342926626405234:11323]
2025-10-09T21:51:19.100756Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342926988239751:9485] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:19.100772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342926988239751:9485] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:19.100776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342926988239751:9485] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:19.100785Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2101 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:19.101218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:19.101519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:19.101562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:19.101817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:19.101853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.101855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.101856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.101857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.101858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.101859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.101859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.101860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.101861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.101864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.101871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.101873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.101877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.101881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.101885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.101888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.101892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.101896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.101905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.101919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.101929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.101935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.101941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.101947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.101951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.101956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.101964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.101968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.101970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.101974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.101977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.101980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.101982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.101985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.101988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.101991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.102010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:19.108385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:19.109736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:19.109750Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:19.109751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:19.109755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:19.109761Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2100 [2:7559342926626405234:11323]
2025-10-09T21:51:19.109771Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:19.109787Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:19.109791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:19.109793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:19.109795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:19.109809Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:19.109815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:19.109821Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:19.109869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.109874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.109876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.109878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:19.109930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:19.109937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.109945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.109951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.109962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:19.109979Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342926626405229:11323]
2025-10-09T21:51:19.109988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342926626405229:11323]
2025-10-09T21:51:19.109989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342926626405229:11323]
2025-10-09T21:51:19.109996Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:19.110007Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:19.110007Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:19.110011Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:19.110013Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:19.110016Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:19.110018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:19.110045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342926988239878:9485], now have 1 active actors on pipe
2025-10-09T21:51:19.110048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342926988239700:2432], now have 1 active actors on pipe
2025-10-09T21:51:19.110081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342926626405229:11323]
2025-10-09T21:51:19.110091Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:19.110100Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2100
2025-10-09T21:51:19.110112Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:19.110127Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2100
2025-10-09T21:51:19.110169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.110181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.110212Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7a91a504-d9261b4e-9fc9a8f8-707a9f7c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:19.110227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:19.110237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:19.110243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:19.110251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:19.110260Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:19.110276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:19.110473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.110485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.110509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:19.110604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2101 partition: 0 MaxSeqNo: 25 sessionId: src_id|7a91a504-d9261b4e-9fc9a8f8-707a9f7c_0
2025-10-09T21:51:19.110954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|463005d8-3c562af2-7fd6a909-e53d89ef_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7a91a504-d9261b4e-9fc9a8f8-707a9f7c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:19.113426Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a91a504-d9261b4e-9fc9a8f8-707a9f7c_0] Write session: try to update token
2025-10-09T21:51:19.113445Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a91a504-d9261b4e-9fc9a8f8-707a9f7c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:19.126593Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2101 sessionId: src_id|7a91a504-d9261b4e-9fc9a8f8-707a9f7c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:19.128476Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:19.128497Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:19.128508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342926626405229:11323]
2025-10-09T21:51:19.128568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:19.128575Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:19.128744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:19.128881Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:19.128914Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:19.129026Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:19.129203Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:19.129394Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:19.129440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.129452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.129470Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:19.129506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:19.129600Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:19.136367Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:19.136388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:19.136391Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:19.136394Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:19.136403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342926626405229:11323]
2025-10-09T21:51:19.136406Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342926626405229:11323]
2025-10-09T21:51:19.136409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342926626405229:11323]
2025-10-09T21:51:19.136451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:19.136459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.136462Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:19.136467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:19.136468Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:19.136473Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:19.136596Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:19.185875Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342926988239891:5250]
2025-10-09T21:51:19.186165Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 2100, pipe: [1:7559342926988239891:5250]
2025-10-09T21:51:19.284301Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:19.290941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.290963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.291102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:19.291160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:19.291215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:19.291273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:19.291313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:19.291325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:19.291398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:19.291452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:19.291505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:19.291563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:19.291602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:19.291610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:19.291662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:19.291712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:19.291761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:19.291815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:19.291853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:19.291859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:19.291915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:19.291970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:19.292031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:19.292085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:19.292123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:19.292131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:19.292186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:19.292237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:19.292290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:19.292349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:19.292393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:19.292401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:19.292452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:19.292504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:19.292568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:19.292628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:19.292667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:19.292672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:19.292723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:19.292773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:19.292828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:19.292882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:19.292922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:19.292927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:19.292983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:19.293046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:19.293101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:19.293148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:19.293189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:19.293194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:19.293247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:19.293295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:19.293346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:19.293413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:19.293458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:19.293463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:19.293523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:19.293578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:19.293631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:19.293687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:19.293724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:19.293726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:19.293762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342926988239891:5250], now have 1 active actors on pipe
2025-10-09T21:51:19.294015Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:19.294042Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:19.294072Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2101 reason: error at write: Disconnected sessionId: src_id|7a91a504-d9261b4e-9fc9a8f8-707a9f7c_0
2025-10-09T21:51:19.294273Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2101 sessionId: src_id|7a91a504-d9261b4e-9fc9a8f8-707a9f7c_0 is DEAD
2025-10-09T21:51:19.294419Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a91a504-d9261b4e-9fc9a8f8-707a9f7c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:19.294429Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a91a504-d9261b4e-9fc9a8f8-707a9f7c_0] Write session will restart in 0.007004s
2025-10-09T21:51:19.294455Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a91a504-d9261b4e-9fc9a8f8-707a9f7c_0] Write session: Do CDS request
2025-10-09T21:51:19.294462Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a91a504-d9261b4e-9fc9a8f8-707a9f7c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:19.294448Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:19.294486Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342926988239891:5250]
2025-10-09T21:51:19.294507Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:51:19.294537Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:19.294612Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 21 consumer shared/user session shared/user_1_21_4029661093892576224_v1 is DEAD
2025-10-09T21:51:19.294724Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:51:19.294836Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:51:19.294841Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:51:19.294859Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.007288s
2025-10-09T21:51:19.294894Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342845383846416:5242] disconnected.
2025-10-09T21:51:19.294905Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342845383846416:5242] disconnected; active server actors: 1
2025-10-09T21:51:19.294908Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342845383846416:5242] client user disconnected session shared/user_1_21_4029661093892576224_v1
2025-10-09T21:51:19.302480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a91a504-d9261b4e-9fc9a8f8-707a9f7c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:19.302589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:19.302604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2102
2025-10-09T21:51:19.302741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2102 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:19.302774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2102 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:19.302784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2102 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:19.302788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:19.303142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2102 sessionId:  describe result for acl check
2025-10-09T21:51:19.303177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:19.303183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:19.303185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:19.303191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342926988239943:9477] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:19.303289Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:51:19.303726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342926988239943:9477] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:19.303767Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:51:19.303773Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 22
2025-10-09T21:51:19.304312Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 22 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:51:19.304359Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:51:19.304526Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 auth for : user
2025-10-09T21:51:19.304637Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 Handle describe topics response
2025-10-09T21:51:19.304661Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 auth is DEAD
2025-10-09T21:51:19.304687Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:51:19.304930Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:51:19.305004Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_22_15461964503436223557_v1
2025-10-09T21:51:19.305114Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342926988239998:9473] connected; active server actors: 1
2025-10-09T21:51:19.305144Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559342926988239998:9473] session shared/user_1_22_15461964503436223557_v1
2025-10-09T21:51:19.305157Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:51:19.305167Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:51:19.305177Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_22_15461964503436223557_v1" (Sender=[1:7559342926988239958:9473], Pipe=[1:7559342926988239998:9473], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:51:19.305181Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:51:19.305191Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:51:19.305204Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_22_15461964503436223557_v1" (Sender=[1:7559342926988239958:9473], Pipe=[1:7559342926988239998:9473], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:51:19.305214Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_22_15461964503436223557_v1" sender [1:7559342926988239958:9473] lock partition 0 for ReadingSession "shared/user_1_22_15461964503436223557_v1" (Sender=[1:7559342926988239958:9473], Pipe=[1:7559342926988239998:9473], Partitions=[], ActiveFamilyCount=1) generation 1 step 22
2025-10-09T21:51:19.305227Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:51:19.305237Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000036s
2025-10-09T21:51:19.305442Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 22 Session: "shared/user_1_22_15461964503436223557_v1" ClientId: "user" PipeClient { RawX1: 7559342926988239998 RawX2: 4503603922347265 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:51:19.305533Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:51:19.307202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342926988239943:9477] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:19.308602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:19.310524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:19.310695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342926988239943:9477] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:19.310707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342926988239943:9477] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:19.310713Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342926988239943:9477] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:19.310721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2102 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:19.310829Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:19.310846Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:19.310861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:19.310925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:19.310969Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342926626405429:11327]
2025-10-09T21:51:19.311583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:19.311874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:19.311926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:19.312255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:19.312314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.312324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.312325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.312326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.312327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.312328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.312329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.312330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.312331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.312337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.312347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.312351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.312356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.312360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.312364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.312371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.312375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.312379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.312385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.312399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.312416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.312422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.312426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.312431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.312436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.312441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.312446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.312451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.312456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.312461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.312466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.312471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.312478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.312483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.312488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.312492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.312525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:19.318734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:19.319843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:19.319858Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:19.319859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:19.319862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:19.319866Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2101 [2:7559342926626405429:11327]
2025-10-09T21:51:19.319876Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:19.319888Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:19.319891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:19.319893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:19.319894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:19.319907Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:19.319914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:19.319919Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:19.319964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.319968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.319970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.319971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:19.320020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:19.320026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.320033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.320039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.320054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:19.320077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342926626405424:11327]
2025-10-09T21:51:19.320087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342926626405424:11327]
2025-10-09T21:51:19.320089Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342926626405424:11327]
2025-10-09T21:51:19.320097Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:19.320107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:19.320109Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:19.320111Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:19.320113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342926988240099:9477], now have 1 active actors on pipe
2025-10-09T21:51:19.320120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:19.320134Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:19.320137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:19.320143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342926988240022:9464], now have 1 active actors on pipe
2025-10-09T21:51:19.320146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342926988239903:2432], now have 1 active actors on pipe
2025-10-09T21:51:19.320161Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2101
2025-10-09T21:51:19.320171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342926626405424:11327]
2025-10-09T21:51:19.320203Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2101
2025-10-09T21:51:19.320186Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:19.320189Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:19.320232Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 2101, pipe: [1:7559342926988240022:9464]
2025-10-09T21:51:19.320302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.320316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.320331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.320333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.320339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_22_15461964503436223557_v1 on pipe: [1:7559342926988240022:9464]
2025-10-09T21:51:19.320365Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|54439ce8-61ffa21-fcfc492f-d0069773_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:19.320375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:19.320379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:19.320380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:19.320395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:19.320408Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:19.320424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:19.320432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:19.320433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:19.320444Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_22_15461964503436223557_v1
2025-10-09T21:51:19.320449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:19.320473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:19.320496Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:51:19.320604Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_22_15461964503436223557_v1:1 with generation 2101
2025-10-09T21:51:19.320637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.320645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.320659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:19.320758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2102 partition: 0 MaxSeqNo: 25 sessionId: src_id|54439ce8-61ffa21-fcfc492f-d0069773_0
2025-10-09T21:51:19.321133Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a91a504-d9261b4e-9fc9a8f8-707a9f7c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|54439ce8-61ffa21-fcfc492f-d0069773_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:19.323574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54439ce8-61ffa21-fcfc492f-d0069773_0] Write session: try to update token
2025-10-09T21:51:19.323591Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54439ce8-61ffa21-fcfc492f-d0069773_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:19.324761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:51:19.324792Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:51:19.324803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:19.324811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:19.324813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:19.324822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:51:19.324835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:51:19.325006Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046679320 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:51:19.325026Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:51:19.325065Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 sending to client partition status
2025-10-09T21:51:19.334564Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:19.334581Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:19.334583Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:19.334585Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:19.334593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342926626405424:11327]
2025-10-09T21:51:19.334597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342926626405424:11327]
2025-10-09T21:51:19.334600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342926626405424:11327]
2025-10-09T21:51:19.334639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:19.334644Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:19.334645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.334648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:19.334650Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:19.334653Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:19.334775Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:19.336378Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2102 sessionId: src_id|54439ce8-61ffa21-fcfc492f-d0069773_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:19.338709Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:19.480675Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:19.481170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.481184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.481483Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:19.481498Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:19.481498Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342926988240022:9464]
2025-10-09T21:51:19.481522Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:19.481558Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2102 reason: error at write: Disconnected sessionId: src_id|54439ce8-61ffa21-fcfc492f-d0069773_0
2025-10-09T21:51:19.481605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_22_15461964503436223557_v1
2025-10-09T21:51:19.481623Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:19.481662Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_22_15461964503436223557_v1
2025-10-09T21:51:19.481742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2102 sessionId: src_id|54439ce8-61ffa21-fcfc492f-d0069773_0 is DEAD
2025-10-09T21:51:19.481904Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54439ce8-61ffa21-fcfc492f-d0069773_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:19.481915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54439ce8-61ffa21-fcfc492f-d0069773_0] Write session will restart in 0.005772s
2025-10-09T21:51:19.481939Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54439ce8-61ffa21-fcfc492f-d0069773_0] Write session: Do CDS request
2025-10-09T21:51:19.481945Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54439ce8-61ffa21-fcfc492f-d0069773_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:19.488408Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|54439ce8-61ffa21-fcfc492f-d0069773_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:19.488483Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:19.488497Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2103
2025-10-09T21:51:19.488591Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2103 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:19.488624Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2103 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:19.488634Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2103 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:19.488637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:19.489052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2103 sessionId:  describe result for acl check
2025-10-09T21:51:19.489082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:19.489088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:19.489090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:19.489095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342926988240147:9463] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:19.489629Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342926988240147:9463] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:19.492075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342926988240147:9463] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:19.494322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:19.496126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:19.496297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342926988240147:9463] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:19.496309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342926988240147:9463] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:19.496312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342926988240147:9463] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:19.496314Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:19.496322Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:19.496319Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2103 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:19.496331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:19.496371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:19.496403Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342926626405639:11339]
2025-10-09T21:51:19.496758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:19.496988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:19.497029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:19.497279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:19.497314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.497318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.497318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.497319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.497320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.497320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.497321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.497321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.497322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.497325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.497331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.497333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.497335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.497337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.497339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.497341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.497343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.497345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.497349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.497354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.497360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.497365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.497369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.497385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.497389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.497393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.497395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.497398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.497401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.497404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.497406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.497409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.497411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.497414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.497417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.497420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.497435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:19.502980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:19.503897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:19.503913Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:19.503916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:19.503919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:19.503926Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2102 [2:7559342926626405639:11339]
2025-10-09T21:51:19.503935Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:19.503947Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:19.503952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:19.503954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:19.503955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:19.503967Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:19.503970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:19.503976Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:19.504165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.504171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.504173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.504175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:19.504220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:19.504226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.504233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.504239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.504255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:19.504278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342926988240123:2432], now have 1 active actors on pipe
2025-10-09T21:51:19.504286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342926988240303:9463], now have 1 active actors on pipe
2025-10-09T21:51:19.504293Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342926626405634:11339]
2025-10-09T21:51:19.504295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342926626405634:11339]
2025-10-09T21:51:19.504296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342926626405634:11339]
2025-10-09T21:51:19.504308Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:19.504321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:19.504321Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:19.504323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:19.504324Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:19.504333Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:19.504336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:19.504355Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342926626405634:11339]
2025-10-09T21:51:19.504362Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:19.504328Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2102
2025-10-09T21:51:19.504330Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2102
2025-10-09T21:51:19.504363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:19.504380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.504385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.504424Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|820501cb-bc61e533-18d5db67-696dd99_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:19.504436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:19.504442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:19.504443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:19.504451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:19.504463Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:19.504475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:19.504616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.504628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.504642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:19.504728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2103 partition: 0 MaxSeqNo: 25 sessionId: src_id|820501cb-bc61e533-18d5db67-696dd99_0
2025-10-09T21:51:19.505030Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|54439ce8-61ffa21-fcfc492f-d0069773_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|820501cb-bc61e533-18d5db67-696dd99_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:19.507471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|820501cb-bc61e533-18d5db67-696dd99_0] Write session: try to update token
2025-10-09T21:51:19.507487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|820501cb-bc61e533-18d5db67-696dd99_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:19.517465Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:19.517482Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:19.517484Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:19.517486Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:19.517495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342926626405634:11339]
2025-10-09T21:51:19.517500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342926626405634:11339]
2025-10-09T21:51:19.517504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342926626405634:11339]
2025-10-09T21:51:19.517545Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:19.517547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:19.517550Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:19.517554Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:19.517555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.517557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:19.517686Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:19.519534Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2103 sessionId: src_id|820501cb-bc61e533-18d5db67-696dd99_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:19.521665Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:19.581682Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342926988240315:9464]
2025-10-09T21:51:19.581930Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 2102, pipe: [1:7559342926988240315:9464]
2025-10-09T21:51:19.656894Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:19.657269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.657282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.657321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342926988240315:9464], now have 1 active actors on pipe
2025-10-09T21:51:19.657531Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:19.657530Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:19.657552Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:19.657604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2103 reason: error at write: Disconnected sessionId: src_id|820501cb-bc61e533-18d5db67-696dd99_0
2025-10-09T21:51:19.657762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2103 sessionId: src_id|820501cb-bc61e533-18d5db67-696dd99_0 is DEAD
2025-10-09T21:51:19.657851Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:19.657896Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342926988240315:9464]
2025-10-09T21:51:19.657924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|820501cb-bc61e533-18d5db67-696dd99_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:19.657934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|820501cb-bc61e533-18d5db67-696dd99_0] Write session will restart in 0.009842s
2025-10-09T21:51:19.657961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|820501cb-bc61e533-18d5db67-696dd99_0] Write session: Do CDS request
2025-10-09T21:51:19.657966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|820501cb-bc61e533-18d5db67-696dd99_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:19.668458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|820501cb-bc61e533-18d5db67-696dd99_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:19.668522Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:19.668536Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2104
2025-10-09T21:51:19.668693Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2104 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:19.668737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2104 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:19.668748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2104 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:19.668751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:19.669138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2104 sessionId:  describe result for acl check
2025-10-09T21:51:19.669168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:19.669174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:19.669176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:19.669181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342926988240459:9457] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:19.669726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342926988240459:9457] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:19.670659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:19.671697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342926988240459:9457] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:19.672450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:19.672608Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:19.672617Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:19.672628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:19.672672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:19.672707Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342926626405821:11341]
2025-10-09T21:51:19.673256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:19.673502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:19.673541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:19.673825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:19.673809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342926988240459:9457] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:19.673821Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342926988240459:9457] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:19.673824Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342926988240459:9457] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:19.673831Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2104 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:19.673866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.673873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.673875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.673876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.673877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.673878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.673879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.673880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.673880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.673885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.673896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.673899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.673903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.673907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.673911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.673915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.673918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.673922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.673928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.673935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.673941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.673948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.673952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.673959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.673963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.673972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.673976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.673985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.673989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.673998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.674003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.674012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.674016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.674023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.674051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.674060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.674088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:19.679857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:19.680853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:19.680865Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:19.680867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:19.680871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:19.680877Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2103 [2:7559342926626405821:11341]
2025-10-09T21:51:19.680893Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:19.680908Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:19.680917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:19.680919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:19.680921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:19.680939Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:19.680948Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:19.680957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:19.681013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.681019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.681021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.681024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:19.681086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:19.681093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.681105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.681122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.681139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:19.681162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342926988240317:2432], now have 1 active actors on pipe
2025-10-09T21:51:19.681169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342926988240499:9457], now have 1 active actors on pipe
2025-10-09T21:51:19.681176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342926626405816:11341]
2025-10-09T21:51:19.681178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342926626405816:11341]
2025-10-09T21:51:19.681180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342926626405816:11341]
2025-10-09T21:51:19.681192Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:19.681206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:19.681205Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:19.681208Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:19.681209Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:19.681217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:19.681219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:19.681238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342926626405816:11341]
2025-10-09T21:51:19.681246Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:19.681250Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:19.681252Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2103
2025-10-09T21:51:19.681260Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2103
2025-10-09T21:51:19.681326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.681335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.681360Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d36f958d-51c942e-e4da4953-303ba749_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:19.681382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:19.681391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:19.681393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:19.681400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:19.681409Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:19.681567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:19.681892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.681904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.681936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:19.682058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2104 partition: 0 MaxSeqNo: 25 sessionId: src_id|d36f958d-51c942e-e4da4953-303ba749_0
2025-10-09T21:51:19.682362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|820501cb-bc61e533-18d5db67-696dd99_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d36f958d-51c942e-e4da4953-303ba749_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:19.684648Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d36f958d-51c942e-e4da4953-303ba749_0] Write session: try to update token
2025-10-09T21:51:19.684665Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d36f958d-51c942e-e4da4953-303ba749_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:19.694588Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:19.694602Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:19.694604Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:19.694605Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:19.694612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342926626405816:11341]
2025-10-09T21:51:19.694615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342926626405816:11341]
2025-10-09T21:51:19.694618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342926626405816:11341]
2025-10-09T21:51:19.694652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:19.694657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.694660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:19.694667Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:19.694673Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:19.694676Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:19.694777Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:19.697647Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2104 sessionId: src_id|d36f958d-51c942e-e4da4953-303ba749_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:19.699902Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:19.757834Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342926988240509:9464]
2025-10-09T21:51:19.758114Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 2103, pipe: [1:7559342926988240509:9464]
2025-10-09T21:51:19.838326Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:19.838531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.838543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.838579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342926988240509:9464], now have 1 active actors on pipe
2025-10-09T21:51:19.838734Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:19.838802Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:19.838826Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342926988240509:9464]
2025-10-09T21:51:19.838848Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:19.838921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2104 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d36f958d-51c942e-e4da4953-303ba749_0
2025-10-09T21:51:19.839129Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2104 sessionId: src_id|d36f958d-51c942e-e4da4953-303ba749_0 is DEAD
2025-10-09T21:51:19.839240Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d36f958d-51c942e-e4da4953-303ba749_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:19.839250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d36f958d-51c942e-e4da4953-303ba749_0] Write session will restart in 0.005681s
2025-10-09T21:51:19.839278Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d36f958d-51c942e-e4da4953-303ba749_0] Write session: Do CDS request
2025-10-09T21:51:19.839283Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d36f958d-51c942e-e4da4953-303ba749_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:19.846482Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d36f958d-51c942e-e4da4953-303ba749_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:19.846492Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:19.846506Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2105
2025-10-09T21:51:19.846685Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2105 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:19.846716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2105 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:19.846727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2105 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:19.846730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:19.847162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2105 sessionId:  describe result for acl check
2025-10-09T21:51:19.847200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:19.847313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:19.847321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:19.847326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342926988240533:9453] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:19.847861Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342926988240533:9453] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:19.850065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342926988240533:9453] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:19.852516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342926988240533:9453] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:19.852528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342926988240533:9453] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:19.852531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342926988240533:9453] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:19.852536Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2105 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:19.853546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:19.855459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:19.855661Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:19.855671Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:19.855684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:19.855758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:19.855801Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342926626406011:11345]
2025-10-09T21:51:19.856389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:19.856699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:19.856742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:19.856971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:19.857007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.857012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.857012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.857013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.857014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.857015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.857015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.857016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.857017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.857020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.857028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.857030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.857033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.857035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.857037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.857039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.857041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.857043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.857050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.857055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:19.857060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.857063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:19.857065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.857069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:19.857072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.857078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:19.857082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.857087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:19.857091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.857096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:19.857103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.857106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.857109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.857112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.857115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.857121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.857147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:19.863625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:19.865001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:19.865019Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:19.865022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:19.865026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:19.865034Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2104 [2:7559342926626406011:11345]
2025-10-09T21:51:19.865050Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:19.865067Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:19.865072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:19.865076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:19.865078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:19.865145Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:19.865156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:19.865163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:19.865229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.865236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.865240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.865243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:19.865315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:19.865322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.865335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:19.865347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:19.865367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:19.865407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342926988240511:2432], now have 1 active actors on pipe
2025-10-09T21:51:19.865415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342926988240697:9453], now have 1 active actors on pipe
2025-10-09T21:51:19.865422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342926626406006:11345]
2025-10-09T21:51:19.865425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342926626406006:11345]
2025-10-09T21:51:19.865427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342926626406006:11345]
2025-10-09T21:51:19.865439Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:19.865457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:19.865466Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:19.865469Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:19.865471Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:19.865477Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:19.865441Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2104
2025-10-09T21:51:19.865483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:19.865482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2104
2025-10-09T21:51:19.865528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342926626406006:11345]
2025-10-09T21:51:19.865542Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:19.865557Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:19.865558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.865563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.865608Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c8c01bd8-c6115147-1ed4e13-d32677eb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:19.865620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:19.865624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:19.865627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:19.865638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:19.865653Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:19.865673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:19.865873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:19.865883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:19.865899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:19.865979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2105 partition: 0 MaxSeqNo: 25 sessionId: src_id|c8c01bd8-c6115147-1ed4e13-d32677eb_0
2025-10-09T21:51:19.866341Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d36f958d-51c942e-e4da4953-303ba749_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c8c01bd8-c6115147-1ed4e13-d32677eb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:19.868762Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8c01bd8-c6115147-1ed4e13-d32677eb_0] Write session: try to update token
2025-10-09T21:51:19.868783Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8c01bd8-c6115147-1ed4e13-d32677eb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:19.879764Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:19.879782Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:19.879784Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:19.879786Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:19.879795Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342926626406006:11345]
2025-10-09T21:51:19.879799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342926626406006:11345]
2025-10-09T21:51:19.879802Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342926626406006:11345]
2025-10-09T21:51:19.879839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:19.879847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:19.879849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:19.879860Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:19.879882Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:19.879890Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:19.879968Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:19.881645Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2105 sessionId: src_id|c8c01bd8-c6115147-1ed4e13-d32677eb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:19.883941Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:19.939066Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342926988240714:9464]
2025-10-09T21:51:19.939335Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 2104, pipe: [1:7559342926988240714:9464]
2025-10-09T21:51:20.025841Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:20.026235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.026250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.026299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342926988240714:9464], now have 1 active actors on pipe
2025-10-09T21:51:20.026481Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:20.026508Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342926988240714:9464]
2025-10-09T21:51:20.026531Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:20.026542Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:20.026625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2105 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c8c01bd8-c6115147-1ed4e13-d32677eb_0
2025-10-09T21:51:20.026805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2105 sessionId: src_id|c8c01bd8-c6115147-1ed4e13-d32677eb_0 is DEAD
2025-10-09T21:51:20.026964Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8c01bd8-c6115147-1ed4e13-d32677eb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:20.026975Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8c01bd8-c6115147-1ed4e13-d32677eb_0] Write session will restart in 0.006593s
2025-10-09T21:51:20.026996Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8c01bd8-c6115147-1ed4e13-d32677eb_0] Write session: Do CDS request
2025-10-09T21:51:20.027001Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8c01bd8-c6115147-1ed4e13-d32677eb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:20.034500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8c01bd8-c6115147-1ed4e13-d32677eb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:20.034600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:20.034617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2106
2025-10-09T21:51:20.034758Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2106 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:20.034798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2106 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:20.034808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2106 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:20.034811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:20.035237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2106 sessionId:  describe result for acl check
2025-10-09T21:51:20.035283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:20.035290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:20.035292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:20.035297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342931283208047:9445] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:20.035851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342931283208047:9445] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:20.038084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342931283208047:9445] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:20.040609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342931283208047:9445] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:20.040621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342931283208047:9445] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:20.040624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342931283208047:9445] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:20.040631Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2106 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:20.040642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:20.043083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:20.043279Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:20.043287Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:20.043298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:20.043351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:20.043382Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342930921373492:11347]
2025-10-09T21:51:20.043779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:20.044079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:20.044128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:20.044497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:20.044555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.044561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.044562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.044563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.044564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.044566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.044567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.044568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.044569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.044574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.044584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.044587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.044591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.044594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.044597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.044600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.044603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.044606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.044613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.044621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.044626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.044631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.044635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.044639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.044643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.044647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.044651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.044656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.044659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.044664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.044668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.044672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.044676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.044681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.044686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.044699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.044727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:20.051137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:20.052445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:20.052460Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:20.052463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:20.052467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:20.052475Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2105 [2:7559342930921373492:11347]
2025-10-09T21:51:20.052490Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:20.052512Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:20.052522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:20.052525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:20.052527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:20.052545Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:20.052551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:20.052558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:20.052613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.052620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.052623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.052625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:20.052697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:20.052705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.052717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.052734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.052750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:20.052776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342931283208194:9445], now have 1 active actors on pipe
2025-10-09T21:51:20.052779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342931283208012:2432], now have 1 active actors on pipe
2025-10-09T21:51:20.052786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342930921373485:11347]
2025-10-09T21:51:20.052788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342930921373485:11347]
2025-10-09T21:51:20.052790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342930921373485:11347]
2025-10-09T21:51:20.052803Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:20.052816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:20.052824Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:20.052826Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:20.052828Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:20.052834Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:20.052838Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:20.052870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342930921373485:11347]
2025-10-09T21:51:20.052878Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:20.052879Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:20.052852Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2105
2025-10-09T21:51:20.052893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.052853Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2105
2025-10-09T21:51:20.052899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.052953Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9931d4de-a1570f2d-f8052548-a461bcb2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:20.052970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:20.052974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:20.052983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:20.052993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:20.053008Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:20.053029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:20.053185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.053194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.053211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:20.053314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2106 partition: 0 MaxSeqNo: 25 sessionId: src_id|9931d4de-a1570f2d-f8052548-a461bcb2_0
2025-10-09T21:51:20.053662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8c01bd8-c6115147-1ed4e13-d32677eb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9931d4de-a1570f2d-f8052548-a461bcb2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:20.055967Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9931d4de-a1570f2d-f8052548-a461bcb2_0] Write session: try to update token
2025-10-09T21:51:20.055983Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9931d4de-a1570f2d-f8052548-a461bcb2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:20.067254Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:20.067274Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:20.067276Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:20.067279Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:20.067288Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342930921373485:11347]
2025-10-09T21:51:20.067292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342930921373485:11347]
2025-10-09T21:51:20.067296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342930921373485:11347]
2025-10-09T21:51:20.067338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:20.067346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.067349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:20.067366Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:20.067399Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:20.067412Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:20.067483Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:20.068252Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2106 sessionId: src_id|9931d4de-a1570f2d-f8052548-a461bcb2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:20.070720Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:20.126459Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342931283208209:9464]
2025-10-09T21:51:20.126692Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 2105, pipe: [1:7559342931283208209:9464]
2025-10-09T21:51:20.227661Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:20.227949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.227963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.228007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342931283208209:9464], now have 1 active actors on pipe
2025-10-09T21:51:20.228222Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342931283208209:9464]
2025-10-09T21:51:20.228320Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:20.228397Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:20.228404Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:20.228432Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:20.228464Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2106 reason: error at write: Disconnected sessionId: src_id|9931d4de-a1570f2d-f8052548-a461bcb2_0
2025-10-09T21:51:20.228659Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2106 sessionId: src_id|9931d4de-a1570f2d-f8052548-a461bcb2_0 is DEAD
2025-10-09T21:51:20.228798Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9931d4de-a1570f2d-f8052548-a461bcb2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:20.228804Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9931d4de-a1570f2d-f8052548-a461bcb2_0] Write session will restart in 0.006884s
2025-10-09T21:51:20.228825Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9931d4de-a1570f2d-f8052548-a461bcb2_0] Write session: Do CDS request
2025-10-09T21:51:20.228830Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9931d4de-a1570f2d-f8052548-a461bcb2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:20.236517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9931d4de-a1570f2d-f8052548-a461bcb2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:20.236561Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:20.236575Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2107
2025-10-09T21:51:20.236761Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2107 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:20.236813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2107 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:20.236825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2107 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:20.236829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:20.237292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2107 sessionId:  describe result for acl check
2025-10-09T21:51:20.237325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:20.237332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:20.237333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:20.237339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342931283208249:9441] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:20.237806Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342931283208249:9441] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:20.240434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342931283208249:9441] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:20.242826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342931283208249:9441] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:20.242843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342931283208249:9441] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:20.242847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342931283208249:9441] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:20.242866Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2107 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:20.242900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:20.244944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:20.245268Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:20.245280Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:20.245293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:20.245348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:20.245398Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342930921373678:11349]
2025-10-09T21:51:20.246014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:20.246328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:20.246373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:20.246694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:20.246735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.246741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.246742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.246743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.246745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.246746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.246747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.246748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.246749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.246754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.246765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.246769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.246773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.246777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.246782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.246786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.246789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.246793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.246800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.246808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.246814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.246820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.246824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.246829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.246839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.246844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.246854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.246863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.246872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.246878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.246882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.246891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.246900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.246906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.246915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.246920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.246950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:20.253018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:20.254013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:20.254044Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:20.254047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:20.254051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:20.254058Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2106 [2:7559342930921373678:11349]
2025-10-09T21:51:20.254071Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:20.254088Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:20.254093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:20.254101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:20.254105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:20.254120Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:20.254133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:20.254140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:20.254194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.254202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.254204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.254207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:20.254270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:20.254279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.254291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.254304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.254321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:20.254345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342931283208215:2432], now have 1 active actors on pipe
2025-10-09T21:51:20.254357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342931283208398:9441], now have 1 active actors on pipe
2025-10-09T21:51:20.254365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342930921373672:11349]
2025-10-09T21:51:20.254367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342930921373672:11349]
2025-10-09T21:51:20.254369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342930921373672:11349]
2025-10-09T21:51:20.254382Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:20.254398Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:20.254399Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:20.254402Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:20.254404Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:20.254413Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:20.254417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:20.254438Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342930921373672:11349]
2025-10-09T21:51:20.254446Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:20.254452Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:20.254431Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2106
2025-10-09T21:51:20.254433Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2106
2025-10-09T21:51:20.254572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.254581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.254603Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|be077537-7b51c833-78993011-1c5ea5ac_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:20.254614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:20.254617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:20.254619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:20.254626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:20.254635Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:20.254645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:20.254800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.254812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.254834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:20.254938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2107 partition: 0 MaxSeqNo: 25 sessionId: src_id|be077537-7b51c833-78993011-1c5ea5ac_0
2025-10-09T21:51:20.255267Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9931d4de-a1570f2d-f8052548-a461bcb2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|be077537-7b51c833-78993011-1c5ea5ac_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:20.257641Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be077537-7b51c833-78993011-1c5ea5ac_0] Write session: try to update token
2025-10-09T21:51:20.257657Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be077537-7b51c833-78993011-1c5ea5ac_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:20.265876Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:20.265892Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:20.265902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342930921373672:11349]
2025-10-09T21:51:20.265947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:20.265957Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:20.266141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:20.266284Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:20.266417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:20.266554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:20.266726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:20.266764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:20.270389Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2107 sessionId: src_id|be077537-7b51c833-78993011-1c5ea5ac_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:20.272653Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:20.272971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.272989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.273026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:20.273127Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:20.278275Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:20.278288Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:20.278291Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:20.278292Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:20.278303Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342930921373672:11349]
2025-10-09T21:51:20.278306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342930921373672:11349]
2025-10-09T21:51:20.278309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342930921373672:11349]
2025-10-09T21:51:20.278348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:20.278354Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:20.278359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.278359Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:20.278361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:20.278363Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:20.278486Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:20.328147Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342931283208415:9464]
2025-10-09T21:51:20.328403Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 2106, pipe: [1:7559342931283208415:9464]
2025-10-09T21:51:20.423801Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:20.430330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.430351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.430499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.430551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.430605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.430663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.431026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.431043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.431108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.431160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.431218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.431272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.431315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.431323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.431378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.431436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.431495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.431549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.431586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.431592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.431651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.431719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.431783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.431838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.431881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.431890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.431949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.431999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.432050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.432100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.432138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.432145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.432200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.432250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.432310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.432367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.432403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.432409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.432460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.432507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.432561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.432614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.432659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.432669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.432726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.432782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.432839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.432892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.432928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.432934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.432983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.433031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.433081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.433143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.433185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.433190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.433241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.433289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.433339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.433401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.433439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.433444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.433483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342931283208415:9464], now have 1 active actors on pipe
2025-10-09T21:51:20.433748Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:20.433763Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:20.433794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2107 reason: error at write: Disconnected sessionId: src_id|be077537-7b51c833-78993011-1c5ea5ac_0
2025-10-09T21:51:20.433894Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:20.433921Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:20.433938Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342931283208415:9464]
2025-10-09T21:51:20.434009Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2107 sessionId: src_id|be077537-7b51c833-78993011-1c5ea5ac_0 is DEAD
2025-10-09T21:51:20.434119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be077537-7b51c833-78993011-1c5ea5ac_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:20.434129Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be077537-7b51c833-78993011-1c5ea5ac_0] Write session will restart in 0.009660s
2025-10-09T21:51:20.434158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be077537-7b51c833-78993011-1c5ea5ac_0] Write session: Do CDS request
2025-10-09T21:51:20.434163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be077537-7b51c833-78993011-1c5ea5ac_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:20.444507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|be077537-7b51c833-78993011-1c5ea5ac_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:20.444579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:20.444593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2108
2025-10-09T21:51:20.444685Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2108 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:20.444716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2108 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:20.444722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2108 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:20.444724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:20.445099Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2108 sessionId:  describe result for acl check
2025-10-09T21:51:20.445139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:20.445145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:20.445147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:20.445153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342931283208590:9422] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:20.445681Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342931283208590:9422] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:20.445821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:20.447668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:20.447809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342931283208590:9422] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:20.447821Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:20.447832Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:20.447843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:20.447885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:20.447922Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342930921373903:11363]
2025-10-09T21:51:20.448444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:20.448708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:20.448746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:20.449040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:20.449080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.449086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.449087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.449088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.449089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.449090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.449091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.449092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.449093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.449101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.449111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.449115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.449118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.449122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.449126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.449129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.449133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.449136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.449143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.449154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.449160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.449171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.449175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.449180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.449184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.449189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.449193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.449198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.449202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.449209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.449217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.449227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.449231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.449240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.449246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.449254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.449282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:20.452240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342931283208590:9422] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:20.452255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342931283208590:9422] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:20.452258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342931283208590:9422] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:20.452267Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2108 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:20.455194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:20.456383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:20.456396Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:20.456398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:20.456401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:20.456407Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2107 [2:7559342930921373903:11363]
2025-10-09T21:51:20.456420Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:20.456437Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:20.456441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:20.456444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:20.456446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:20.456462Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:20.456475Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:20.456486Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:20.456542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.456549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.456551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.456553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:20.456629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:20.456635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.456645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.456655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.456672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:20.456695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342931283208442:2432], now have 1 active actors on pipe
2025-10-09T21:51:20.456703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342931283208626:9422], now have 1 active actors on pipe
2025-10-09T21:51:20.456705Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2107
2025-10-09T21:51:20.456719Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2107
2025-10-09T21:51:20.456710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342930921373897:11363]
2025-10-09T21:51:20.456716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342930921373897:11363]
2025-10-09T21:51:20.456718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342930921373897:11363]
2025-10-09T21:51:20.456731Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:20.456745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:20.456753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:20.456757Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:20.456758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:20.456759Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:20.456761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:20.456801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.456809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.456825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342930921373897:11363]
2025-10-09T21:51:20.456836Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:20.456849Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:20.456869Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b713893c-46af6e66-d2acb2ba-24d8d4a0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:20.456880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:20.456888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:20.456891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:20.456903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:20.456917Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:20.456938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:20.457108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.457116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.457133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:20.457246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2108 partition: 0 MaxSeqNo: 25 sessionId: src_id|b713893c-46af6e66-d2acb2ba-24d8d4a0_0
2025-10-09T21:51:20.457567Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|be077537-7b51c833-78993011-1c5ea5ac_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b713893c-46af6e66-d2acb2ba-24d8d4a0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:20.459809Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b713893c-46af6e66-d2acb2ba-24d8d4a0_0] Write session: try to update token
2025-10-09T21:51:20.459823Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b713893c-46af6e66-d2acb2ba-24d8d4a0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:20.470227Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:20.470241Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:20.470242Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:20.470244Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:20.470251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342930921373897:11363]
2025-10-09T21:51:20.470255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342930921373897:11363]
2025-10-09T21:51:20.470257Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342930921373897:11363]
2025-10-09T21:51:20.470288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:20.470294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.470295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:20.470309Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:20.470323Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:20.470326Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:20.470420Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:20.472659Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2108 sessionId: src_id|b713893c-46af6e66-d2acb2ba-24d8d4a0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:20.474868Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:20.534004Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342931283208634:9464]
2025-10-09T21:51:20.534219Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 2107, pipe: [1:7559342931283208634:9464]
2025-10-09T21:51:20.604440Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:20.604834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.604847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.604886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342931283208634:9464], now have 1 active actors on pipe
2025-10-09T21:51:20.605133Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:20.605147Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:20.605157Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342931283208634:9464]
2025-10-09T21:51:20.605182Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:20.605208Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2108 reason: error at write: Disconnected sessionId: src_id|b713893c-46af6e66-d2acb2ba-24d8d4a0_0
2025-10-09T21:51:20.605233Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:20.605398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2108 sessionId: src_id|b713893c-46af6e66-d2acb2ba-24d8d4a0_0 is DEAD
2025-10-09T21:51:20.605547Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b713893c-46af6e66-d2acb2ba-24d8d4a0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:20.605558Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b713893c-46af6e66-d2acb2ba-24d8d4a0_0] Write session will restart in 0.006762s
2025-10-09T21:51:20.605578Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b713893c-46af6e66-d2acb2ba-24d8d4a0_0] Write session: Do CDS request
2025-10-09T21:51:20.605583Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b713893c-46af6e66-d2acb2ba-24d8d4a0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:20.613464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b713893c-46af6e66-d2acb2ba-24d8d4a0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:20.613520Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:20.613532Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2109
2025-10-09T21:51:20.613641Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2109 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:20.613671Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2109 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:20.613680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2109 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:20.613684Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:20.614082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2109 sessionId:  describe result for acl check
2025-10-09T21:51:20.614116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:20.614122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:20.614124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:20.614130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342931283208760:9418] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:20.614679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342931283208760:9418] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:20.615975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:20.616614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342931283208760:9418] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:20.617699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:20.617846Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:20.617855Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:20.617866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:20.617911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:20.617948Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342930921374091:11365]
2025-10-09T21:51:20.618506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:20.618718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:20.618754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:20.618896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342931283208760:9418] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:20.618909Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342931283208760:9418] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:20.618915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342931283208760:9418] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:20.618922Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2109 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:20.619051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:20.619098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.619104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.619105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.619106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.619107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.619108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.619109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.619110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.619111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.619115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.619125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.619129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.619133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.619136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.619140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.619143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.619147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.619155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.619161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.619172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.619181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.619186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.619190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.619199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.619203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.619212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.619218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.619224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.619232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.619237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.619245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.619252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.619260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.619266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.619272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.619279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.619310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:20.625067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:20.626150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:20.626161Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:20.626163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:20.626167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:20.626172Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2108 [2:7559342930921374091:11365]
2025-10-09T21:51:20.626183Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:20.626197Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:20.626204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:20.626207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:20.626209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:20.626223Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:20.626233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:20.626240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:20.626292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.626299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.626301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.626303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:20.626364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:20.626371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.626381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.626397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.626417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:20.626440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342931283208825:9418], now have 1 active actors on pipe
2025-10-09T21:51:20.626448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342931283208637:2432], now have 1 active actors on pipe
2025-10-09T21:51:20.626455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342930921374085:11365]
2025-10-09T21:51:20.626462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342930921374085:11365]
2025-10-09T21:51:20.626464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342930921374085:11365]
2025-10-09T21:51:20.626476Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:20.626486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:20.626470Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2108
2025-10-09T21:51:20.626493Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:20.626475Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2108
2025-10-09T21:51:20.626494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:20.626494Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:20.626508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:20.626511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:20.626542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.626550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.626566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342930921374085:11365]
2025-10-09T21:51:20.626577Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:20.626580Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:20.626598Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4238de70-52e88f96-7e5e1318-a50da9e7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:20.626609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:20.626616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:20.626619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:20.626628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:20.626640Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:20.626663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:20.626788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.626796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.626809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:20.626892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2109 partition: 0 MaxSeqNo: 25 sessionId: src_id|4238de70-52e88f96-7e5e1318-a50da9e7_0
2025-10-09T21:51:20.627147Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b713893c-46af6e66-d2acb2ba-24d8d4a0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4238de70-52e88f96-7e5e1318-a50da9e7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:20.629257Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4238de70-52e88f96-7e5e1318-a50da9e7_0] Write session: try to update token
2025-10-09T21:51:20.629269Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4238de70-52e88f96-7e5e1318-a50da9e7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:20.639216Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:20.639235Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:20.639237Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:20.639239Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:20.639249Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342930921374085:11365]
2025-10-09T21:51:20.639254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342930921374085:11365]
2025-10-09T21:51:20.639258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342930921374085:11365]
2025-10-09T21:51:20.639289Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:20.639299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:20.639305Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:20.639306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.639309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:20.639310Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:20.639446Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:20.641408Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2109 sessionId: src_id|4238de70-52e88f96-7e5e1318-a50da9e7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:20.643551Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:20.705526Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342931283208834:9464]
2025-10-09T21:51:20.705773Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 2108, pipe: [1:7559342931283208834:9464]
2025-10-09T21:51:20.784953Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:20.785184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.785195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.785231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342931283208834:9464], now have 1 active actors on pipe
2025-10-09T21:51:20.785432Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:20.785435Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342931283208834:9464]
2025-10-09T21:51:20.785512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:20.785517Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:20.785540Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:20.785568Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2109 reason: error at write: Disconnected sessionId: src_id|4238de70-52e88f96-7e5e1318-a50da9e7_0
2025-10-09T21:51:20.785746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2109 sessionId: src_id|4238de70-52e88f96-7e5e1318-a50da9e7_0 is DEAD
2025-10-09T21:51:20.785869Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4238de70-52e88f96-7e5e1318-a50da9e7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:20.785877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4238de70-52e88f96-7e5e1318-a50da9e7_0] Write session will restart in 0.007602s
2025-10-09T21:51:20.785894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4238de70-52e88f96-7e5e1318-a50da9e7_0] Write session: Do CDS request
2025-10-09T21:51:20.785899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4238de70-52e88f96-7e5e1318-a50da9e7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:20.794366Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4238de70-52e88f96-7e5e1318-a50da9e7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:20.794467Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:20.794479Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2110
2025-10-09T21:51:20.794591Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2110 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:20.794620Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2110 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:20.794631Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2110 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:20.794635Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:20.794950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2110 sessionId:  describe result for acl check
2025-10-09T21:51:20.794979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:20.794985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:20.794986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:20.794989Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342931283208928:9412] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:20.795386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342931283208928:9412] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:20.797297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342931283208928:9412] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:20.797719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:20.801916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:20.802046Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342931283208928:9412] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:20.802059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342931283208928:9412] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:20.802063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342931283208928:9412] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:20.802071Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2110 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:20.802098Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:20.802109Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:20.802121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:20.802171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:20.802214Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342930921374284:11367]
2025-10-09T21:51:20.802794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:20.803027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:20.803061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:20.803348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:20.803393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.803398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.803400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.803401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.803402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.803403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.803404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.803405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.803406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.803411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.803421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.803425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.803432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.803440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.803444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.803452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.803456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.803462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.803468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.803475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:20.803486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.803494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:20.803499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.803508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:20.803513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.803519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:20.803528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.803533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:20.803537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.803546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:20.803555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.803564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.803568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.803577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.803584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.803589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.803621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:20.809621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:20.811064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:20.811076Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:20.811078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:20.811081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:20.811085Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2109 [2:7559342930921374284:11367]
2025-10-09T21:51:20.811093Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:20.811107Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:20.811112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:20.811114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:20.811116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:20.811128Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:20.811135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:20.811140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:20.811190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.811195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.811196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.811198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:20.811246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:20.811251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.811258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:20.811264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:20.811274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:20.811290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342931283209024:9412], now have 1 active actors on pipe
2025-10-09T21:51:20.811298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342931283208837:2432], now have 1 active actors on pipe
2025-10-09T21:51:20.811305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342930921374279:11367]
2025-10-09T21:51:20.811312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342930921374279:11367]
2025-10-09T21:51:20.811314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342930921374279:11367]
2025-10-09T21:51:20.811324Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:20.811336Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:20.811338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:20.811343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:20.811345Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:20.811347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:20.811349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:20.811366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342930921374279:11367]
2025-10-09T21:51:20.811379Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:20.811381Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:20.811377Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2109
2025-10-09T21:51:20.811386Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2109
2025-10-09T21:51:20.811410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.811417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.811437Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b5289d49-8cf17914-6df9ded5-4c6d5e31_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:20.811448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:20.811452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:20.811454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:20.811461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:20.811468Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:20.811478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:20.811630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.811640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.811655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:20.811734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2110 partition: 0 MaxSeqNo: 25 sessionId: src_id|b5289d49-8cf17914-6df9ded5-4c6d5e31_0
2025-10-09T21:51:20.812064Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4238de70-52e88f96-7e5e1318-a50da9e7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b5289d49-8cf17914-6df9ded5-4c6d5e31_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:20.814439Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5289d49-8cf17914-6df9ded5-4c6d5e31_0] Write session: try to update token
2025-10-09T21:51:20.814454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5289d49-8cf17914-6df9ded5-4c6d5e31_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:20.825306Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:20.825322Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:20.825332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342930921374279:11367]
2025-10-09T21:51:20.825386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:20.825386Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:20.825552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:20.825687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:20.826014Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:20.826344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:20.826614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:20.826653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:20.829167Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2110 sessionId: src_id|b5289d49-8cf17914-6df9ded5-4c6d5e31_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:20.831361Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:20.831522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.831537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.831571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:20.831640Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:20.834588Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:20.834600Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:20.834603Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:20.834605Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:20.834614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342930921374279:11367]
2025-10-09T21:51:20.834617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342930921374279:11367]
2025-10-09T21:51:20.834621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342930921374279:11367]
2025-10-09T21:51:20.834651Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:20.834656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:20.834661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:20.834663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:20.834664Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:20.834666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:20.834791Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:20.883698Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:51:20.883726Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 17
2025-10-09T21:51:20.885746Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342931283209038:9464]
2025-10-09T21:51:20.885931Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 2109, pipe: [1:7559342931283209038:9464]
2025-10-09T21:51:20.987670Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:20.994896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:20.994921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:20.995068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.995122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.995178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.995238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.995281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.995290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.995352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.995407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.995466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.995525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.995568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.995571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.995659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.995755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.995810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.995867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.995912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.995919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.995980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.996037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.996100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.996158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.996272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.996278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.996336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.996392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.996450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.996505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.996545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.996550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.996607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.996663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.996723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.996831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.996883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.996889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.996975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.997060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.997118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.997175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.997219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.997225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.997284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.997338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.997411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.997481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.997520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.997525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.997583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.997635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.997691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.997751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.997791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.997796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.997860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:20.997926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:20.997986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:20.998061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:20.998177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:20.998184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:20.998273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342931283209038:9464], now have 1 active actors on pipe
2025-10-09T21:51:20.998462Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:20.998475Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:20.998479Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:20.998482Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 17
2025-10-09T21:51:20.998511Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2110 reason: error at write: Disconnected sessionId: src_id|b5289d49-8cf17914-6df9ded5-4c6d5e31_0
2025-10-09T21:51:20.998511Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342931283209038:9464]
2025-10-09T21:51:20.998720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2110 sessionId: src_id|b5289d49-8cf17914-6df9ded5-4c6d5e31_0 is DEAD
2025-10-09T21:51:20.998861Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5289d49-8cf17914-6df9ded5-4c6d5e31_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:20.998874Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5289d49-8cf17914-6df9ded5-4c6d5e31_0] Write session will restart in 0.009664s
2025-10-09T21:51:20.998899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5289d49-8cf17914-6df9ded5-4c6d5e31_0] Write session: Do CDS request
2025-10-09T21:51:20.998919Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5289d49-8cf17914-6df9ded5-4c6d5e31_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:20.999194Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:21.009504Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5289d49-8cf17914-6df9ded5-4c6d5e31_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:21.009647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:21.009662Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2111
2025-10-09T21:51:21.009806Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2111 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:21.009840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2111 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:21.009850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2111 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:21.009854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:21.010276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2111 sessionId:  describe result for acl check
2025-10-09T21:51:21.010309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:21.010314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:21.010315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:21.010320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342935578176475:9405] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:21.010762Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342935578176475:9405] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:21.012338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:21.012858Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342935578176475:9405] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:21.014614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:21.014788Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:21.014799Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:21.014810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:21.014858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:21.014897Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342935216341771:11372]
2025-10-09T21:51:21.015393Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342935578176475:9405] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:21.015404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342935578176475:9405] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:21.015407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342935578176475:9405] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:21.015413Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2111 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:21.015492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:21.015751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:21.015788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:21.016050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:21.016093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.016098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.016099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.016100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.016101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.016101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.016102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.016102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.016103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.016106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.016117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.016120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.016125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.016129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.016133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.016137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.016141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.016143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.016148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.016154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.016160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.016166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.016171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.016177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.016186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.016191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.016196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.016206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.016214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.016219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.016229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.016239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.016247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.016252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.016261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.016270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.016290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:21.022588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:21.023619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:21.023633Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:21.023636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:21.023641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:21.023647Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2110 [2:7559342935216341771:11372]
2025-10-09T21:51:21.023666Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:21.023688Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:21.023695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:21.023699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:21.023701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:21.023718Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:21.023728Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:21.023739Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:21.023797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.023803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.023806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.023809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:21.023989Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2110
2025-10-09T21:51:21.023992Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2110
2025-10-09T21:51:21.024102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:21.024111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.024122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.024139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.024163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:21.024190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342935578176530:9405], now have 1 active actors on pipe
2025-10-09T21:51:21.024198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342931283209049:2432], now have 1 active actors on pipe
2025-10-09T21:51:21.024216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.024225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.024259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342935216341766:11372]
2025-10-09T21:51:21.024265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342935216341766:11372]
2025-10-09T21:51:21.024267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342935216341766:11372]
2025-10-09T21:51:21.024280Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:21.024298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:21.024301Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:21.024304Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:21.024306Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:21.024313Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:21.024320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:21.024341Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ab6daf2c-8876b91b-88f7511c-7ff6b8b2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:21.024353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:21.024357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:21.024359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:21.024369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:21.024383Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:21.024443Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:51:21.024490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342935216341766:11372]
2025-10-09T21:51:21.024502Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:21.024515Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:21.024518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:21.024672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.024680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.024673Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 22 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:51:21.024694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:21.024703Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:51:21.024817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2111 partition: 0 MaxSeqNo: 25 sessionId: src_id|ab6daf2c-8876b91b-88f7511c-7ff6b8b2_0
2025-10-09T21:51:21.025192Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5289d49-8cf17914-6df9ded5-4c6d5e31_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ab6daf2c-8876b91b-88f7511c-7ff6b8b2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:21.027534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ab6daf2c-8876b91b-88f7511c-7ff6b8b2_0] Write session: try to update token
2025-10-09T21:51:21.027547Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ab6daf2c-8876b91b-88f7511c-7ff6b8b2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:21.037776Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:21.037796Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:21.037806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342935216341766:11372]
2025-10-09T21:51:21.037854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:21.037863Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:21.038023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:21.038186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:21.038322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:21.038461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:21.038645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:21.038685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:21.041173Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2111 sessionId: src_id|ab6daf2c-8876b91b-88f7511c-7ff6b8b2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:21.043707Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:21.043912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.043931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.043966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:21.044045Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:21.049973Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:21.049989Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:21.049991Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:21.049993Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:21.050003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342935216341766:11372]
2025-10-09T21:51:21.050007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342935216341766:11372]
2025-10-09T21:51:21.050011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342935216341766:11372]
2025-10-09T21:51:21.050055Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:21.050063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:21.050065Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:21.050068Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:21.050070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.050073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:21.050201Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:21.098760Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342935578176544:9464]
2025-10-09T21:51:21.099014Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 2110, pipe: [1:7559342935578176544:9464]
2025-10-09T21:51:21.196240Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:21.203278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.203302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.203459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.203517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.203572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.203626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.203668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.203676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.203732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.203787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.203852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.203908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.203947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.203954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.204009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.204058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.204112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.204167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.204206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.204211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.204296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.204377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.204432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.204482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.204522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.204528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.204579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.204629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.204684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.204737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.204776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.204782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.204835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.204883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.204933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.204984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.205022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.205027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.205079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.205129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.205180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.205231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.205268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.205273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.205324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.205385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.205434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.205481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.205517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.205522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.205573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.205635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.205696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.205756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.205806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.205813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.205870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.205918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.205970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.206022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.206078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.206086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.206132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342935578176544:9464], now have 1 active actors on pipe
2025-10-09T21:51:21.206405Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342935578176544:9464]
2025-10-09T21:51:21.206417Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:21.206421Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:21.206442Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:21.206450Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2111 reason: error at write: Disconnected sessionId: src_id|ab6daf2c-8876b91b-88f7511c-7ff6b8b2_0
2025-10-09T21:51:21.206625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2111 sessionId: src_id|ab6daf2c-8876b91b-88f7511c-7ff6b8b2_0 is DEAD
2025-10-09T21:51:21.206765Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab6daf2c-8876b91b-88f7511c-7ff6b8b2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:21.206775Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab6daf2c-8876b91b-88f7511c-7ff6b8b2_0] Write session will restart in 0.005552s
2025-10-09T21:51:21.206757Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:21.206794Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab6daf2c-8876b91b-88f7511c-7ff6b8b2_0] Write session: Do CDS request
2025-10-09T21:51:21.206799Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab6daf2c-8876b91b-88f7511c-7ff6b8b2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:21.213461Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ab6daf2c-8876b91b-88f7511c-7ff6b8b2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:21.213555Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:21.213566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2112
2025-10-09T21:51:21.213680Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2112 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:21.213718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2112 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:21.213728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2112 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:21.213732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:21.214162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2112 sessionId:  describe result for acl check
2025-10-09T21:51:21.214195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:21.214203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:21.214205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:21.214211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342935578176583:9390] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:21.214748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342935578176583:9390] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:21.217815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342935578176583:9390] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:21.219681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:21.221382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:21.221535Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:21.221544Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:21.221555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:21.221547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342935578176583:9390] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:21.221558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342935578176583:9390] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:21.221561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342935578176583:9390] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:21.221566Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2112 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:21.221606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:21.221643Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342935216341964:11375]
2025-10-09T21:51:21.222224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:21.222483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:21.222521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:21.222824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:21.222866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.222873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.222874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.222875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.222876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.222877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.222879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.222880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.222881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.222885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.222896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.222900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.222904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.222908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.222912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.222916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.222920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.222923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.222929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.222937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.222942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.222947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.222952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.222956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.222965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.222970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.222975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.222980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.222988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.222994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.222998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.223005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.223010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.223019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.223023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.223029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.223057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:21.229334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:21.230583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:21.230597Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:21.230599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:21.230602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:21.230606Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2111 [2:7559342935216341964:11375]
2025-10-09T21:51:21.230614Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:21.230626Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:21.230630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:21.230633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:21.230634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:21.230647Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:21.230656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:21.230661Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:21.230702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.230707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.230708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.230710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:21.230770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:21.230775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.230782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.230788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.230798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:21.230814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342935578176550:2432], now have 1 active actors on pipe
2025-10-09T21:51:21.230824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342935216341959:11375]
2025-10-09T21:51:21.230825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342935216341959:11375]
2025-10-09T21:51:21.230827Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342935216341959:11375]
2025-10-09T21:51:21.230833Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:21.230838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342935578176743:9390], now have 1 active actors on pipe
2025-10-09T21:51:21.230844Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:21.230845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:21.230847Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:21.230849Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:21.230856Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:21.230859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:21.230881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342935216341959:11375]
2025-10-09T21:51:21.230891Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:21.230898Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2111
2025-10-09T21:51:21.230906Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:21.230900Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2111
2025-10-09T21:51:21.230944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.230952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.230980Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cd260142-cf10170e-f75f96d1-67ac5d86_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:21.230992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:21.230995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:21.230997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:21.231004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:21.231012Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:21.231025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:21.231167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.231175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.231187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:21.231268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2112 partition: 0 MaxSeqNo: 25 sessionId: src_id|cd260142-cf10170e-f75f96d1-67ac5d86_0
2025-10-09T21:51:21.231583Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ab6daf2c-8876b91b-88f7511c-7ff6b8b2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cd260142-cf10170e-f75f96d1-67ac5d86_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:21.233871Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd260142-cf10170e-f75f96d1-67ac5d86_0] Write session: try to update token
2025-10-09T21:51:21.233885Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd260142-cf10170e-f75f96d1-67ac5d86_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:21.246329Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2112 sessionId: src_id|cd260142-cf10170e-f75f96d1-67ac5d86_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:21.247101Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:21.247120Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:21.247128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342935216341959:11375]
2025-10-09T21:51:21.247169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:21.247179Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:21.247323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:21.247446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:21.247567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:21.247690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:21.247859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:21.247892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:21.248645Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:21.248834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.248853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.248883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:21.248963Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:21.255712Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:21.255731Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:21.255733Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:21.255735Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:21.255744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342935216341959:11375]
2025-10-09T21:51:21.255748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342935216341959:11375]
2025-10-09T21:51:21.255751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342935216341959:11375]
2025-10-09T21:51:21.255784Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:21.255787Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:21.255790Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:21.255791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:21.255798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.255800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:21.255925Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:21.306605Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342935578176759:9464]
2025-10-09T21:51:21.306876Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 2111, pipe: [1:7559342935578176759:9464]
2025-10-09T21:51:21.398529Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:21.404793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.404813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.404944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.404997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.405049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.405102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.405141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.405149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.405208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.405259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.405314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.405366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.405416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.405423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.405476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.405526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.405579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.405631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.405666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.405672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.405727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.405778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.405837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.405896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.405933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.405938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.405993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.406069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.406131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.406194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.406236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.406246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.406308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.406366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.406415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.406467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.406506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.406511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.406566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.406613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.406665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.406717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.406755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.406761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.406812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.406862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.406913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.406966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.407003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.407007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.407058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.407105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.407154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.407204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.407240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.407245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.407294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.407347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.407398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.407451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.407488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.407494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.407534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342935578176759:9464], now have 1 active actors on pipe
2025-10-09T21:51:21.407681Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:21.407683Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342935578176759:9464]
2025-10-09T21:51:21.407962Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:21.407969Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:21.407989Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2112 reason: error at write: Disconnected sessionId: src_id|cd260142-cf10170e-f75f96d1-67ac5d86_0
2025-10-09T21:51:21.408150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2112 sessionId: src_id|cd260142-cf10170e-f75f96d1-67ac5d86_0 is DEAD
2025-10-09T21:51:21.408202Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:21.408278Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd260142-cf10170e-f75f96d1-67ac5d86_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:21.408285Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd260142-cf10170e-f75f96d1-67ac5d86_0] Write session will restart in 0.007961s
2025-10-09T21:51:21.408307Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd260142-cf10170e-f75f96d1-67ac5d86_0] Write session: Do CDS request
2025-10-09T21:51:21.408312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd260142-cf10170e-f75f96d1-67ac5d86_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:21.417502Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd260142-cf10170e-f75f96d1-67ac5d86_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:21.417586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:21.417602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2113
2025-10-09T21:51:21.417722Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2113 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:21.417751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2113 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:21.417767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2113 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:21.417774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:21.418232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2113 sessionId:  describe result for acl check
2025-10-09T21:51:21.418268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:21.418273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:21.418275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:21.418279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342935578176939:9377] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:21.418688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342935578176939:9377] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:21.419555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:21.420081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342935578176939:9377] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:21.421506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:21.421660Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:21.421669Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:21.421680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:21.421727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:21.421765Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342935216342194:11385]
2025-10-09T21:51:21.422233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:21.422467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:21.422503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:21.422494Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342935578176939:9377] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:21.422507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342935578176939:9377] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:21.422510Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342935578176939:9377] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:21.422516Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2113 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:21.422776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:21.422821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.422826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.422827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.422828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.422829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.422830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.422831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.422833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.422834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.422838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.422847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.422851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.422854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.422858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.422862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.422866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.422869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.422873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.422878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.422885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.422895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.422900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.422904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.422913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.422921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.422926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.422934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.422939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.422942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.422947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.422957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.422964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.422971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.422975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.422985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.422993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.423023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:21.428895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:21.429930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:21.429946Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:21.429949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:21.429953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:21.429959Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2112 [2:7559342935216342194:11385]
2025-10-09T21:51:21.429971Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:21.429986Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:21.429993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:21.429996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:21.429998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:21.430012Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:21.430023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:21.430044Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:21.430094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.430101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.430103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.430105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:21.430292Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2112
2025-10-09T21:51:21.430297Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2112
2025-10-09T21:51:21.430354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:21.430362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.430374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.430391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.430412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:21.430437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342935578176786:2432], now have 1 active actors on pipe
2025-10-09T21:51:21.430441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342935578176976:9377], now have 1 active actors on pipe
2025-10-09T21:51:21.430456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.430461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.430481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342935216342189:11385]
2025-10-09T21:51:21.430487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342935216342189:11385]
2025-10-09T21:51:21.430489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342935216342189:11385]
2025-10-09T21:51:21.430501Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:21.430512Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:21.430516Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:21.430520Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:21.430522Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:21.430529Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:21.430532Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:21.430551Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9af79022-a28d5028-1d25799e-2a2efa74_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:21.430564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:21.430568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:21.430570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:21.430579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:21.430594Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:21.430618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342935216342189:11385]
2025-10-09T21:51:21.430629Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:21.430633Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:21.430644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:21.430808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.430815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.430829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:21.430908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2113 partition: 0 MaxSeqNo: 25 sessionId: src_id|9af79022-a28d5028-1d25799e-2a2efa74_0
2025-10-09T21:51:21.431189Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd260142-cf10170e-f75f96d1-67ac5d86_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9af79022-a28d5028-1d25799e-2a2efa74_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:21.433459Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9af79022-a28d5028-1d25799e-2a2efa74_0] Write session: try to update token
2025-10-09T21:51:21.433470Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9af79022-a28d5028-1d25799e-2a2efa74_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:21.441509Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:21.441523Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:21.441530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342935216342189:11385]
2025-10-09T21:51:21.441565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:21.441571Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:21.441717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:21.441836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:21.441956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:21.442114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:21.442283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:21.442314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:21.444652Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2113 sessionId: src_id|9af79022-a28d5028-1d25799e-2a2efa74_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:21.446866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:21.447010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.447025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.447056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:21.447133Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:21.453762Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:21.453779Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:21.453781Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:21.453783Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:21.453792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342935216342189:11385]
2025-10-09T21:51:21.453796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342935216342189:11385]
2025-10-09T21:51:21.453800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342935216342189:11385]
2025-10-09T21:51:21.453834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:21.453837Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:21.453843Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:21.453844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.453846Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:21.453847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:21.453970Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:21.507837Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342935578176987:9464]
2025-10-09T21:51:21.508082Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 2112, pipe: [1:7559342935578176987:9464]
2025-10-09T21:51:21.594703Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:21.600886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.600910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.601050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.601106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.601159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.601211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.601250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.601258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.601311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.601363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.601426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.601482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.601777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.601783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.601838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.601895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.601950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.602010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.602069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.602077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.602147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.602212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.602274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.602328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.602365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.602370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.602424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.602477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.602529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.602580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.602617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.602623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.602673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.602732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.602787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.602838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.602877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.602883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.602934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.602980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.603031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.603079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.603114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.603120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.603171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.603219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.603267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.603320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.603357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.603362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.603413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.603462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.603515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.603564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.603601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.603605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.603656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.603705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.603754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.603803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.603838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.603846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.603883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342935578176987:9464], now have 1 active actors on pipe
2025-10-09T21:51:21.604097Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:21.604100Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342935578176987:9464]
2025-10-09T21:51:21.604140Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:21.604142Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:21.604169Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2113 reason: error at write: Disconnected sessionId: src_id|9af79022-a28d5028-1d25799e-2a2efa74_0
2025-10-09T21:51:21.604373Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:21.604364Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2113 sessionId: src_id|9af79022-a28d5028-1d25799e-2a2efa74_0 is DEAD
2025-10-09T21:51:21.604484Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9af79022-a28d5028-1d25799e-2a2efa74_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:21.604503Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9af79022-a28d5028-1d25799e-2a2efa74_0] Write session will restart in 0.005224s
2025-10-09T21:51:21.604521Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9af79022-a28d5028-1d25799e-2a2efa74_0] Write session: Do CDS request
2025-10-09T21:51:21.604525Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9af79022-a28d5028-1d25799e-2a2efa74_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:21.610393Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9af79022-a28d5028-1d25799e-2a2efa74_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:21.610474Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:21.610487Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2114
2025-10-09T21:51:21.610576Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2114 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:21.610605Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2114 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:21.610616Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2114 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:21.610619Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:21.610952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2114 sessionId:  describe result for acl check
2025-10-09T21:51:21.610982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:21.610985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:21.610987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:21.610991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342935578177006:9373] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:21.611406Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342935578177006:9373] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:21.614279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342935578177006:9373] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:21.616680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:21.616887Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342935578177006:9373] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:21.616897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342935578177006:9373] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:21.616900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342935578177006:9373] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:21.616905Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2114 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:21.618610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:21.618775Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:21.618784Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:21.618795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:21.618846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:21.618882Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342935216342388:11391]
2025-10-09T21:51:21.619468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:21.619722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:21.619764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:21.620071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:21.620112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.620118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.620119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.620120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.620122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.620123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.620124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.620125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.620126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.620131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.620141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.620145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.620150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.620154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.620162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.620166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.620170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.620174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.620180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.620192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.620198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.620207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.620217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.620228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.620233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.620238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.620243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.620248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.620252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.620257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.620263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.620267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.620271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.620276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.620280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.620285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.620316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:21.626738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:21.627881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:21.627894Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:21.627895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:21.627898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:21.627903Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2113 [2:7559342935216342388:11391]
2025-10-09T21:51:21.627911Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:21.627924Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:21.627926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:21.627928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:21.627930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:21.627942Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:21.627947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:21.627953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:21.628003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.628010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.628011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.628013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:21.628063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:21.628068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.628075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.628081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.628091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:21.628109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342935578176989:2432], now have 1 active actors on pipe
2025-10-09T21:51:21.628117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342935578177180:9373], now have 1 active actors on pipe
2025-10-09T21:51:21.628124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342935216342382:11391]
2025-10-09T21:51:21.628126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342935216342382:11391]
2025-10-09T21:51:21.628128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342935216342382:11391]
2025-10-09T21:51:21.628141Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:21.628151Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:21.628153Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:21.628155Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:21.628158Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:21.628172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:21.628180Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:21.628201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342935216342382:11391]
2025-10-09T21:51:21.628210Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:21.628214Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:21.628216Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2113
2025-10-09T21:51:21.628237Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2113
2025-10-09T21:51:21.628254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.628260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.628282Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|73ab4079-36a0cbb9-834ec0b4-28275c73_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:21.628290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:21.628292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:21.628294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:21.628301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:21.628308Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:21.628322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:21.628500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.628510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.628526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:21.628633Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2114 partition: 0 MaxSeqNo: 25 sessionId: src_id|73ab4079-36a0cbb9-834ec0b4-28275c73_0
2025-10-09T21:51:21.628941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9af79022-a28d5028-1d25799e-2a2efa74_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|73ab4079-36a0cbb9-834ec0b4-28275c73_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:21.631346Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73ab4079-36a0cbb9-834ec0b4-28275c73_0] Write session: try to update token
2025-10-09T21:51:21.631362Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73ab4079-36a0cbb9-834ec0b4-28275c73_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:21.640017Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:21.640034Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:21.640044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342935216342382:11391]
2025-10-09T21:51:21.640090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:21.640086Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:21.640396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:21.640644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:21.640877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:21.641114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:21.641488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:21.641570Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:21.644862Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2114 sessionId: src_id|73ab4079-36a0cbb9-834ec0b4-28275c73_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:21.647374Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:21.647611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.647631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.647680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:21.647764Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:21.653310Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:21.653336Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:21.653339Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:21.653342Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:21.653355Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342935216342382:11391]
2025-10-09T21:51:21.653362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342935216342382:11391]
2025-10-09T21:51:21.653382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342935216342382:11391]
2025-10-09T21:51:21.653438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:21.653455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.653458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:21.653628Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:21.653647Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:21.653654Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:21.653658Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:21.704434Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342935578177192:9464]
2025-10-09T21:51:21.704681Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 2113, pipe: [1:7559342935578177192:9464]
2025-10-09T21:51:21.800884Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:21.807878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.807906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.808064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.808127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.808194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.808258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.808306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.808317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.808380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.808440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.808501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.808571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.808615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.808624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.808683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.808742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.808807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.808873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.808914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.808921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.808980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.809033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.809087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.809145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.809184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.809189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.809248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.809301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.809356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.809442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.809483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.809488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.809544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.809597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.809655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.809708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.809748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.809753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.809807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.809867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.809932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.809995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.810055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.810063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.810125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.810187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.810243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.810295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.810334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.810340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.810393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.810444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.810498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.810565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.810612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.810619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.810682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:21.810733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:21.810787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:21.810852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:21.810895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:21.810901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:21.810941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342935578177192:9464], now have 1 active actors on pipe
2025-10-09T21:51:21.811163Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342935578177192:9464]
2025-10-09T21:51:21.811176Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:21.811215Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:21.811225Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:21.811257Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2114 reason: error at write: Disconnected sessionId: src_id|73ab4079-36a0cbb9-834ec0b4-28275c73_0
2025-10-09T21:51:21.811467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2114 sessionId: src_id|73ab4079-36a0cbb9-834ec0b4-28275c73_0 is DEAD
2025-10-09T21:51:21.811600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73ab4079-36a0cbb9-834ec0b4-28275c73_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:21.811609Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73ab4079-36a0cbb9-834ec0b4-28275c73_0] Write session will restart in 0.009286s
2025-10-09T21:51:21.811645Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73ab4079-36a0cbb9-834ec0b4-28275c73_0] Write session: Do CDS request
2025-10-09T21:51:21.811651Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73ab4079-36a0cbb9-834ec0b4-28275c73_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:21.811734Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:21.822503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|73ab4079-36a0cbb9-834ec0b4-28275c73_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:21.822556Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:21.822570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2115
2025-10-09T21:51:21.822694Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2115 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:21.822730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2115 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:21.822734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2115 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:21.822737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:21.823133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2115 sessionId:  describe result for acl check
2025-10-09T21:51:21.823171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:21.823173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:21.823175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:21.823181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342935578177332:9370] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:21.823718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342935578177332:9370] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:21.825086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:21.825982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342935578177332:9370] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:21.827132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:21.827290Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:21.827298Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:21.827308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:21.827360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:21.827410Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342935216342585:11395]
2025-10-09T21:51:21.828013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:21.828316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:21.828358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:21.828374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342935578177332:9370] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:21.828389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342935578177332:9370] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:21.828393Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342935578177332:9370] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:21.828400Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2115 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:21.828689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:21.828734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.828740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.828741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.828742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.828744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.828745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.828746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.828747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.828749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.828753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.828765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.828770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.828774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.828778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.828782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.828786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.828790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.828794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.828800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.828807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:21.828812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.828819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:21.828823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.828828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:21.828832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.828837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:21.828842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.828847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:21.828852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.828858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:21.828863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.828873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.828877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.828882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.828888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.828897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.828927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:21.835253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:21.836272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:21.836287Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:21.836289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:21.836293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:21.836299Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2114 [2:7559342935216342585:11395]
2025-10-09T21:51:21.836311Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:21.836328Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:21.836331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:21.836333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:21.836336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:21.836352Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:21.836362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:21.836370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:21.836422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.836430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.836432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.836435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:21.836498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:21.836507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.836517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:21.836528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:21.836544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:21.836570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342935578177197:2432], now have 1 active actors on pipe
2025-10-09T21:51:21.836579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342935578177391:9370], now have 1 active actors on pipe
2025-10-09T21:51:21.836586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342935216342580:11395]
2025-10-09T21:51:21.836588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342935216342580:11395]
2025-10-09T21:51:21.836590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342935216342580:11395]
2025-10-09T21:51:21.836602Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:21.836613Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:21.836615Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:21.836622Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:21.836623Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:21.836631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:21.836634Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:21.836616Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2114
2025-10-09T21:51:21.836616Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2114
2025-10-09T21:51:21.836658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342935216342580:11395]
2025-10-09T21:51:21.836666Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:21.836667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:21.836685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.836693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.836737Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5cf0650f-4a00a8bd-5c3d43be-7878595e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:21.836749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:21.836753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:21.836755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:21.836765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:21.836774Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:21.836794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:21.836925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.836932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.836946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:21.837026Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2115 partition: 0 MaxSeqNo: 25 sessionId: src_id|5cf0650f-4a00a8bd-5c3d43be-7878595e_0
2025-10-09T21:51:21.837344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|73ab4079-36a0cbb9-834ec0b4-28275c73_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5cf0650f-4a00a8bd-5c3d43be-7878595e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:21.839620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5cf0650f-4a00a8bd-5c3d43be-7878595e_0] Write session: try to update token
2025-10-09T21:51:21.839635Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5cf0650f-4a00a8bd-5c3d43be-7878595e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:21.850560Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:21.850583Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:21.850586Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:21.850589Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:21.850599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342935216342580:11395]
2025-10-09T21:51:21.850605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342935216342580:11395]
2025-10-09T21:51:21.850610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342935216342580:11395]
2025-10-09T21:51:21.850660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:21.850675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:21.850674Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:21.850678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:21.850681Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:21.850684Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:21.850838Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:21.852375Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2115 sessionId: src_id|5cf0650f-4a00a8bd-5c3d43be-7878595e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:21.854602Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:21.911277Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342935578177403:9464]
2025-10-09T21:51:21.911582Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 2114, pipe: [1:7559342935578177403:9464]
2025-10-09T21:51:21.997415Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:21.998566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:21.998589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:21.998635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342935578177403:9464], now have 1 active actors on pipe
2025-10-09T21:51:21.998854Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:21.998893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:21.998897Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:21.998913Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342935578177403:9464]
2025-10-09T21:51:21.998978Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2115 reason: error at write: Disconnected sessionId: src_id|5cf0650f-4a00a8bd-5c3d43be-7878595e_0
2025-10-09T21:51:21.999016Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:21.999170Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2115 sessionId: src_id|5cf0650f-4a00a8bd-5c3d43be-7878595e_0 is DEAD
2025-10-09T21:51:21.999321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5cf0650f-4a00a8bd-5c3d43be-7878595e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:21.999330Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5cf0650f-4a00a8bd-5c3d43be-7878595e_0] Write session will restart in 0.008846s
2025-10-09T21:51:21.999368Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5cf0650f-4a00a8bd-5c3d43be-7878595e_0] Write session: Do CDS request
2025-10-09T21:51:21.999375Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5cf0650f-4a00a8bd-5c3d43be-7878595e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:22.009457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5cf0650f-4a00a8bd-5c3d43be-7878595e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:22.009562Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:22.009579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2116
2025-10-09T21:51:22.009707Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2116 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:22.009743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2116 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:22.009753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2116 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:22.009757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:22.010201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2116 sessionId:  describe result for acl check
2025-10-09T21:51:22.010240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:22.010246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:22.010248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:22.010254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342939873144841:9357] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:22.010811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342939873144841:9357] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:22.012034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:22.012860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342939873144841:9357] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:22.014444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:22.014643Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:22.014653Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:22.014663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:22.014714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:22.014756Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342939511310077:11397]
2025-10-09T21:51:22.015224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342939873144841:9357] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:22.015236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342939873144841:9357] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:22.015239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342939873144841:9357] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:22.015247Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2116 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:22.015333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:22.015579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:22.015625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:22.015949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:22.016002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.016008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.016009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.016010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.016012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.016013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.016014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.016015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.016016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.016021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.016032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.016036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.016041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.016045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.016050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.016055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.016059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.016063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.016069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.016077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.016083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.016089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.016108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.016114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.016118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.016124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.016128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.016134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.016138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.016143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.016149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.016154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.016158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.016164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.016171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.016176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.016205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:22.022717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:22.024338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:22.024356Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:22.024359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:22.024363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:22.024371Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2115 [2:7559342939511310077:11397]
2025-10-09T21:51:22.024387Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:22.024402Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:22.024404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:22.024407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:22.024409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:22.024422Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:22.024426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:22.024432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:22.024489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.024498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.024501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.024503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:22.024577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:22.024584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.024595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.024607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.024630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:22.024657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342935578177407:2432], now have 1 active actors on pipe
2025-10-09T21:51:22.024665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342939873144894:9357], now have 1 active actors on pipe
2025-10-09T21:51:22.024673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342939511310072:11397]
2025-10-09T21:51:22.024679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342939511310072:11397]
2025-10-09T21:51:22.024681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342939511310072:11397]
2025-10-09T21:51:22.024703Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:22.024690Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2115
2025-10-09T21:51:22.024720Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:22.024703Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2115
2025-10-09T21:51:22.024723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:22.024723Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:22.024725Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:22.024738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:22.024742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:22.024796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.024804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.024815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342939511310072:11397]
2025-10-09T21:51:22.024824Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:22.024836Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:22.024854Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b27b3475-c6323e5c-d0e215a0-5dcb8036_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:22.024859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:22.024863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:22.024865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:22.024873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:22.024882Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:22.024903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:22.025053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.025062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.025078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:22.025154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2116 partition: 0 MaxSeqNo: 25 sessionId: src_id|b27b3475-c6323e5c-d0e215a0-5dcb8036_0
2025-10-09T21:51:22.025542Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5cf0650f-4a00a8bd-5c3d43be-7878595e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b27b3475-c6323e5c-d0e215a0-5dcb8036_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:22.027903Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b27b3475-c6323e5c-d0e215a0-5dcb8036_0] Write session: try to update token
2025-10-09T21:51:22.027918Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b27b3475-c6323e5c-d0e215a0-5dcb8036_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:22.039517Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:22.039539Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:22.039541Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:22.039543Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:22.039555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342939511310072:11397]
2025-10-09T21:51:22.039560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342939511310072:11397]
2025-10-09T21:51:22.039565Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342939511310072:11397]
2025-10-09T21:51:22.039609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:22.039612Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:22.039618Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:22.039621Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:22.039622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.039625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:22.039766Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:22.043471Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2116 sessionId: src_id|b27b3475-c6323e5c-d0e215a0-5dcb8036_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:22.045825Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:22.099150Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342939873144905:9464]
2025-10-09T21:51:22.099430Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 2115, pipe: [1:7559342939873144905:9464]
2025-10-09T21:51:22.186977Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:22.187288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.187307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.187352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342939873144905:9464], now have 1 active actors on pipe
2025-10-09T21:51:22.187563Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:22.187601Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:22.187620Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:22.187621Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342939873144905:9464]
2025-10-09T21:51:22.187628Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:22.187708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2116 reason: error at write: Disconnected sessionId: src_id|b27b3475-c6323e5c-d0e215a0-5dcb8036_0
2025-10-09T21:51:22.187902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2116 sessionId: src_id|b27b3475-c6323e5c-d0e215a0-5dcb8036_0 is DEAD
2025-10-09T21:51:22.188081Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b27b3475-c6323e5c-d0e215a0-5dcb8036_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:22.188093Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b27b3475-c6323e5c-d0e215a0-5dcb8036_0] Write session will restart in 0.009041s
2025-10-09T21:51:22.188123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b27b3475-c6323e5c-d0e215a0-5dcb8036_0] Write session: Do CDS request
2025-10-09T21:51:22.188129Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b27b3475-c6323e5c-d0e215a0-5dcb8036_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:22.198567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b27b3475-c6323e5c-d0e215a0-5dcb8036_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:22.198669Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:22.198691Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2117
2025-10-09T21:51:22.198805Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2117 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:22.198843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2117 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:22.198854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2117 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:22.198858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:22.199379Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2117 sessionId:  describe result for acl check
2025-10-09T21:51:22.199429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:22.199436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:22.199439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:22.199444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342939873145015:9355] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:22.200036Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342939873145015:9355] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:22.202187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342939873145015:9355] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:22.202771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:22.204887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:22.205086Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:22.205097Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:22.205108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:22.205164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:22.205206Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342939511310274:11399]
2025-10-09T21:51:22.205297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342939873145015:9355] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:22.205311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342939873145015:9355] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:22.205314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342939873145015:9355] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:22.205333Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2117 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:22.205789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:22.206051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:22.206101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:22.206410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:22.206459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.206466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.206467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.206469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.206470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.206471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.206473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.206474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.206475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.206480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.206492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.206496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.206500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.206504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.206508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.206513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.206517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.206521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.206528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.206536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.206542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.206548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.206552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.206558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.206562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.206573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.206578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.206583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.206593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.206598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.206608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.206613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.206617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.206628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.206632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.206637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.206657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:22.213281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:22.214541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:22.214556Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:22.214560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:22.214565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:22.214574Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2116 [2:7559342939511310274:11399]
2025-10-09T21:51:22.214590Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:22.214611Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:22.214618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:22.214622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:22.214624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:22.214645Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:22.214658Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:22.214665Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:22.214738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.214745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.214748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.214751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:22.214830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:22.214839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.214853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.214866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.214888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:22.214915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342939873144908:2432], now have 1 active actors on pipe
2025-10-09T21:51:22.214929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342939873145102:9355], now have 1 active actors on pipe
2025-10-09T21:51:22.214938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342939511310269:11399]
2025-10-09T21:51:22.214941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342939511310269:11399]
2025-10-09T21:51:22.214943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342939511310269:11399]
2025-10-09T21:51:22.214959Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:22.214973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:22.214978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:22.214982Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:22.214985Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:22.214989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:22.214993Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:22.214952Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2116
2025-10-09T21:51:22.214958Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2116
2025-10-09T21:51:22.215048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.215058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.215070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342939511310269:11399]
2025-10-09T21:51:22.215082Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:22.215100Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:22.215120Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|21e25769-f29b67bc-3fcb80db-a36afc06_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:22.215132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:22.215136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:22.215143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:22.215155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:22.215168Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:22.215196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:22.215364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.215374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.215388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:22.215502Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2117 partition: 0 MaxSeqNo: 25 sessionId: src_id|21e25769-f29b67bc-3fcb80db-a36afc06_0
2025-10-09T21:51:22.215894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b27b3475-c6323e5c-d0e215a0-5dcb8036_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|21e25769-f29b67bc-3fcb80db-a36afc06_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:22.218527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21e25769-f29b67bc-3fcb80db-a36afc06_0] Write session: try to update token
2025-10-09T21:51:22.218547Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21e25769-f29b67bc-3fcb80db-a36afc06_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:22.229962Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:22.229983Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:22.229985Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:22.229987Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:22.229996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342939511310269:11399]
2025-10-09T21:51:22.230000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342939511310269:11399]
2025-10-09T21:51:22.230003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342939511310269:11399]
2025-10-09T21:51:22.230064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:22.230070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.230072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:22.230078Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:22.230085Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:22.230088Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:22.230206Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:22.234572Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2117 sessionId: src_id|21e25769-f29b67bc-3fcb80db-a36afc06_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:22.237040Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:22.287616Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342939873145116:9464]
2025-10-09T21:51:22.287860Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 2116, pipe: [1:7559342939873145116:9464]
2025-10-09T21:51:22.374760Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:22.375240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.375257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.375307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342939873145116:9464], now have 1 active actors on pipe
2025-10-09T21:51:22.375494Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:22.375567Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:22.375581Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:22.375603Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:22.375637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2117 reason: error at write: Disconnected sessionId: src_id|21e25769-f29b67bc-3fcb80db-a36afc06_0
2025-10-09T21:51:22.375649Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342939873145116:9464]
2025-10-09T21:51:22.375818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2117 sessionId: src_id|21e25769-f29b67bc-3fcb80db-a36afc06_0 is DEAD
2025-10-09T21:51:22.375930Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21e25769-f29b67bc-3fcb80db-a36afc06_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:22.375940Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21e25769-f29b67bc-3fcb80db-a36afc06_0] Write session will restart in 0.009472s
2025-10-09T21:51:22.375961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21e25769-f29b67bc-3fcb80db-a36afc06_0] Write session: Do CDS request
2025-10-09T21:51:22.375979Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21e25769-f29b67bc-3fcb80db-a36afc06_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:22.386484Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21e25769-f29b67bc-3fcb80db-a36afc06_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:22.386569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:22.386584Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2118
2025-10-09T21:51:22.386723Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2118 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:22.386756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2118 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:22.386762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2118 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:22.386766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:22.387140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2118 sessionId:  describe result for acl check
2025-10-09T21:51:22.387175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:22.387176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:22.387177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:22.387181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342939873145281:9351] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:22.387636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342939873145281:9351] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:22.388577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:22.389983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342939873145281:9351] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:22.390656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:22.390804Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:22.390813Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:22.390821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:22.390859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:22.390897Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342939511310470:11401]
2025-10-09T21:51:22.391296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:22.391539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:22.391579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:22.391886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:22.391933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.391939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.391941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.391942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.391943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.391944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.391945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.391947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.391948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.391952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.391962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.391965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.391969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.391972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.391976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.391984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.391987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.391990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.391997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.392004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.392015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.392024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.392029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.392039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.392043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.392052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.392056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.392065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.392070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.392079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.392084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.392088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.392097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.392101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.392110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.392114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.392116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342939873145281:9351] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:22.392129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342939873145281:9351] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:22.392133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342939873145281:9351] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:22.392140Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2118 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:22.392141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:22.398384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:22.399780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:22.399793Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:22.399796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:22.399800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:22.399806Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2117 [2:7559342939511310470:11401]
2025-10-09T21:51:22.399819Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:22.399836Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:22.399843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:22.399846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:22.399848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:22.399866Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:22.399876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:22.399883Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:22.399942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.399950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.399952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.399955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:22.400024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:22.400035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.400047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.400060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.400078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:22.400103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342939873145321:9351], now have 1 active actors on pipe
2025-10-09T21:51:22.400110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342939873145124:2432], now have 1 active actors on pipe
2025-10-09T21:51:22.400117Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342939511310465:11401]
2025-10-09T21:51:22.400124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342939511310465:11401]
2025-10-09T21:51:22.400126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342939511310465:11401]
2025-10-09T21:51:22.400138Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:22.400156Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:22.400163Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:22.400166Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:22.400169Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:22.400175Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:22.400179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:22.400163Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2117
2025-10-09T21:51:22.400206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342939511310465:11401]
2025-10-09T21:51:22.400214Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:22.400170Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2117
2025-10-09T21:51:22.400221Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:22.400232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.400237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.400280Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|eb58fa56-ff39585b-5f42bf83-743fa604_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:22.400294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:22.400297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:22.400300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:22.400310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:22.400321Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:22.400343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:22.400478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.400485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.400498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:22.400586Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2118 partition: 0 MaxSeqNo: 25 sessionId: src_id|eb58fa56-ff39585b-5f42bf83-743fa604_0
2025-10-09T21:51:22.400973Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21e25769-f29b67bc-3fcb80db-a36afc06_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|eb58fa56-ff39585b-5f42bf83-743fa604_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:22.403364Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eb58fa56-ff39585b-5f42bf83-743fa604_0] Write session: try to update token
2025-10-09T21:51:22.403381Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eb58fa56-ff39585b-5f42bf83-743fa604_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:22.414518Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:22.414536Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:22.414539Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:22.414541Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:22.414550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342939511310465:11401]
2025-10-09T21:51:22.414554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342939511310465:11401]
2025-10-09T21:51:22.414558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342939511310465:11401]
2025-10-09T21:51:22.414603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:22.414608Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:22.414614Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:22.414615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.414617Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:22.414617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:22.414753Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:22.416129Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2118 sessionId: src_id|eb58fa56-ff39585b-5f42bf83-743fa604_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:22.418697Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:22.476022Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342939873145352:9464]
2025-10-09T21:51:22.476278Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 2117, pipe: [1:7559342939873145352:9464]
2025-10-09T21:51:22.557965Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:22.558211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.558223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.558259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342939873145352:9464], now have 1 active actors on pipe
2025-10-09T21:51:22.558417Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:22.558476Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:22.558484Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342939873145352:9464]
2025-10-09T21:51:22.558492Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:22.558505Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:22.558555Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2118 reason: error at write: Disconnected sessionId: src_id|eb58fa56-ff39585b-5f42bf83-743fa604_0
2025-10-09T21:51:22.558743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2118 sessionId: src_id|eb58fa56-ff39585b-5f42bf83-743fa604_0 is DEAD
2025-10-09T21:51:22.558908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb58fa56-ff39585b-5f42bf83-743fa604_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:22.558926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb58fa56-ff39585b-5f42bf83-743fa604_0] Write session will restart in 0.009411s
2025-10-09T21:51:22.558947Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb58fa56-ff39585b-5f42bf83-743fa604_0] Write session: Do CDS request
2025-10-09T21:51:22.558952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb58fa56-ff39585b-5f42bf83-743fa604_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:22.569464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eb58fa56-ff39585b-5f42bf83-743fa604_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:22.569551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:22.569567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2119
2025-10-09T21:51:22.569685Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2119 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:22.569716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2119 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:22.569727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2119 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:22.569732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:22.570118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:22.570135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2119 sessionId:  describe result for acl check
2025-10-09T21:51:22.570169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:22.570175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:22.570176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:22.570182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342939873145515:9330] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:22.570696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342939873145515:9330] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:22.571985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:22.572127Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:22.572135Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:22.572145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:22.572183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:22.572217Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342939511310707:11415]
2025-10-09T21:51:22.572617Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342939873145515:9330] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:22.572684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:22.572875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:22.572906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:22.573149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:22.573185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.573191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.573191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.573192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.573193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.573194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.573195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.573196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.573196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.573200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.573207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.573210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.573213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.573216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.573219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.573221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.573224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.573226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.573231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.573238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.573242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.573247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.573251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.573256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.573263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.573267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.573274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.573278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.573285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.573289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.573297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.573300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.573308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.573312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.573321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.573325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.573347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:22.577807Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342939873145515:9330] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:22.577818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342939873145515:9330] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:22.577822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342939873145515:9330] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:22.577827Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2119 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:22.579113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:22.580158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:22.580171Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:22.580173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:22.580176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:22.580181Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2118 [2:7559342939511310707:11415]
2025-10-09T21:51:22.580356Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:22.580373Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:22.580381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:22.580384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:22.580385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:22.580399Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:22.580410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:22.580416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:22.580460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.580468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.580469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.580472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:22.580525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:22.580534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.580543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.580551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.580568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:22.580589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342939873145555:9330], now have 1 active actors on pipe
2025-10-09T21:51:22.580592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342939873145355:2432], now have 1 active actors on pipe
2025-10-09T21:51:22.580598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342939511310701:11415]
2025-10-09T21:51:22.580605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342939511310701:11415]
2025-10-09T21:51:22.580606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342939511310701:11415]
2025-10-09T21:51:22.580594Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2118
2025-10-09T21:51:22.580613Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:22.580594Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2118
2025-10-09T21:51:22.580621Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:22.580627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:22.580629Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:22.580630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:22.580639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:22.580645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:22.580660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.580668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.580695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342939511310701:11415]
2025-10-09T21:51:22.580706Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:22.580709Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:22.580723Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|21a3dd2a-fb994ffa-a3f221e8-48dfffc7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:22.580735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:22.580738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:22.580740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:22.580749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:22.580757Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:22.580782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:22.580891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.580896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.580907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:22.580963Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2119 partition: 0 MaxSeqNo: 25 sessionId: src_id|21a3dd2a-fb994ffa-a3f221e8-48dfffc7_0
2025-10-09T21:51:22.581219Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb58fa56-ff39585b-5f42bf83-743fa604_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|21a3dd2a-fb994ffa-a3f221e8-48dfffc7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:22.583624Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21a3dd2a-fb994ffa-a3f221e8-48dfffc7_0] Write session: try to update token
2025-10-09T21:51:22.583637Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21a3dd2a-fb994ffa-a3f221e8-48dfffc7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:22.593809Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:22.593827Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:22.593829Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:22.593831Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:22.593839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342939511310701:11415]
2025-10-09T21:51:22.593843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342939511310701:11415]
2025-10-09T21:51:22.593846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342939511310701:11415]
2025-10-09T21:51:22.593873Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:22.593877Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:22.593879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:22.593881Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:22.593884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.593886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:22.594023Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:22.596427Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2119 sessionId: src_id|21a3dd2a-fb994ffa-a3f221e8-48dfffc7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:22.598606Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:22.658830Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342939873145562:9464]
2025-10-09T21:51:22.659095Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 2118, pipe: [1:7559342939873145562:9464]
2025-10-09T21:51:22.732891Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:22.733176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.733198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.733246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342939873145562:9464], now have 1 active actors on pipe
2025-10-09T21:51:22.733437Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342939873145562:9464]
2025-10-09T21:51:22.733450Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:22.733454Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:22.733477Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:22.733518Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2119 reason: error at write: Disconnected sessionId: src_id|21a3dd2a-fb994ffa-a3f221e8-48dfffc7_0
2025-10-09T21:51:22.733722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2119 sessionId: src_id|21a3dd2a-fb994ffa-a3f221e8-48dfffc7_0 is DEAD
2025-10-09T21:51:22.733883Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21a3dd2a-fb994ffa-a3f221e8-48dfffc7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:22.733893Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21a3dd2a-fb994ffa-a3f221e8-48dfffc7_0] Write session will restart in 0.008273s
2025-10-09T21:51:22.733911Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21a3dd2a-fb994ffa-a3f221e8-48dfffc7_0] Write session: Do CDS request
2025-10-09T21:51:22.733916Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21a3dd2a-fb994ffa-a3f221e8-48dfffc7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:22.734418Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:22.743409Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21a3dd2a-fb994ffa-a3f221e8-48dfffc7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:22.743480Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:22.743494Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2120
2025-10-09T21:51:22.743587Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2120 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:22.743619Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2120 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:22.743624Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2120 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:22.743627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:22.744009Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2120 sessionId:  describe result for acl check
2025-10-09T21:51:22.744039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:22.744045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:22.744048Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:22.744053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342939873145710:9326] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:22.744567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342939873145710:9326] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:22.745603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:22.746613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342939873145710:9326] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:22.747340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:22.747479Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:22.747488Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:22.747498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:22.747537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:22.747566Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342939511310906:11417]
2025-10-09T21:51:22.748015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:22.748227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:22.748258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:22.748493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:22.748531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.748536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.748537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.748538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.748539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.748540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.748541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.748542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.748543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.748546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.748554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.748557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.748560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.748563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.748566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.748569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.748572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.748574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.748579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.748590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.748599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.748607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.748611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.748614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.748617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.748620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.748624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.748628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.748631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.748636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.748644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.748647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.748650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.748653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.748656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.748659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.748680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:22.748784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342939873145710:9326] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:22.748795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342939873145710:9326] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:22.748798Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342939873145710:9326] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:22.748803Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2120 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:22.754475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:22.755655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:22.755669Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:22.755671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:22.755673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:22.755679Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2119 [2:7559342939511310906:11417]
2025-10-09T21:51:22.755689Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:22.755702Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:22.755704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:22.755706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:22.755708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:22.755722Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:22.755731Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:22.755737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:22.755780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.755789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.755790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.755792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:22.755845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:22.755852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.755861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.755873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.755889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:22.755910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342939873145564:2432], now have 1 active actors on pipe
2025-10-09T21:51:22.755916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342939873145766:9326], now have 1 active actors on pipe
2025-10-09T21:51:22.755922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342939511310901:11417]
2025-10-09T21:51:22.755923Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342939511310901:11417]
2025-10-09T21:51:22.755925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342939511310901:11417]
2025-10-09T21:51:22.755936Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:22.755935Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2119
2025-10-09T21:51:22.755939Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2119
2025-10-09T21:51:22.755951Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:22.755951Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:22.755953Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:22.755955Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:22.755966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:22.755969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:22.756001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.756009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.756020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342939511310901:11417]
2025-10-09T21:51:22.756029Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:22.756040Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:22.756053Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|48b41279-7dbe0a56-4cb6af55-dc12f4af_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:22.756063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:22.756070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:22.756072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:22.756080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:22.756092Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:22.756113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:22.756234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.756240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.756251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:22.756314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2120 partition: 0 MaxSeqNo: 25 sessionId: src_id|48b41279-7dbe0a56-4cb6af55-dc12f4af_0
2025-10-09T21:51:22.756533Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21a3dd2a-fb994ffa-a3f221e8-48dfffc7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|48b41279-7dbe0a56-4cb6af55-dc12f4af_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:22.758779Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48b41279-7dbe0a56-4cb6af55-dc12f4af_0] Write session: try to update token
2025-10-09T21:51:22.758791Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48b41279-7dbe0a56-4cb6af55-dc12f4af_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:22.769379Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:22.769395Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:22.769397Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:22.769398Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:22.769405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342939511310901:11417]
2025-10-09T21:51:22.769409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342939511310901:11417]
2025-10-09T21:51:22.769413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342939511310901:11417]
2025-10-09T21:51:22.769447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:22.769449Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:22.769453Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:22.769456Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:22.769456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.769459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:22.769625Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:22.771871Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2120 sessionId: src_id|48b41279-7dbe0a56-4cb6af55-dc12f4af_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:22.774148Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:22.833400Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342939873145773:9464]
2025-10-09T21:51:22.833577Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 2119, pipe: [1:7559342939873145773:9464]
2025-10-09T21:51:22.907697Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:22.908008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.908026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.908065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342939873145773:9464], now have 1 active actors on pipe
2025-10-09T21:51:22.908233Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:22.908248Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:22.908253Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:22.908303Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342939873145773:9464]
2025-10-09T21:51:22.908329Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2120 reason: error at write: Disconnected sessionId: src_id|48b41279-7dbe0a56-4cb6af55-dc12f4af_0
2025-10-09T21:51:22.908509Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2120 sessionId: src_id|48b41279-7dbe0a56-4cb6af55-dc12f4af_0 is DEAD
2025-10-09T21:51:22.908643Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48b41279-7dbe0a56-4cb6af55-dc12f4af_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:22.908652Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48b41279-7dbe0a56-4cb6af55-dc12f4af_0] Write session will restart in 0.006699s
2025-10-09T21:51:22.908670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48b41279-7dbe0a56-4cb6af55-dc12f4af_0] Write session: Do CDS request
2025-10-09T21:51:22.908678Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48b41279-7dbe0a56-4cb6af55-dc12f4af_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:22.909872Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:22.916537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48b41279-7dbe0a56-4cb6af55-dc12f4af_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:22.916622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:22.916635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2121
2025-10-09T21:51:22.916738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2121 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:22.916766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2121 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:22.916778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2121 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:22.916781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:22.917264Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2121 sessionId:  describe result for acl check
2025-10-09T21:51:22.917298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:22.917304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:22.917306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:22.917311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342939873145869:9322] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:22.917838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342939873145869:9322] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:22.919801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342939873145869:9322] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:22.920094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:22.921759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:22.921862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342939873145869:9322] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:22.921876Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342939873145869:9322] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:22.921897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342939873145869:9322] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:22.921904Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2121 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:22.921916Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:22.921924Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:22.921935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:22.921981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:22.922018Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342939511311110:11421]
2025-10-09T21:51:22.922593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:22.922819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:22.922854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:22.923146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:22.923190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.923195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.923196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.923198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.923199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.923200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.923201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.923202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.923203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.923208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.923218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.923222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.923226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.923229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.923234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.923237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.923241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.923245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.923250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.923262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:22.923272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.923281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:22.923289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.923294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:22.923303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.923312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:22.923320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.923327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:22.923335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.923340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:22.923349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.923358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.923366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.923372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.923380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.923385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.923418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:22.929275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:22.930354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:22.930368Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:22.930370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:22.930372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:22.930377Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2120 [2:7559342939511311110:11421]
2025-10-09T21:51:22.930387Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:22.930398Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:22.930400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:22.930402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:22.930403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:22.930415Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:22.930418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:22.930424Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:22.930468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.930473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.930474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.930476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:22.930531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:22.930538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.930546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:22.930555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:22.930568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:22.930585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342939873145777:2432], now have 1 active actors on pipe
2025-10-09T21:51:22.930593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342939873145976:9322], now have 1 active actors on pipe
2025-10-09T21:51:22.930601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342939511311105:11421]
2025-10-09T21:51:22.930603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342939511311105:11421]
2025-10-09T21:51:22.930605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342939511311105:11421]
2025-10-09T21:51:22.930618Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:22.930631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:22.930631Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:22.930636Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:22.930637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:22.930649Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:22.930652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:22.930628Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2120
2025-10-09T21:51:22.930640Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2120
2025-10-09T21:51:22.930686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.930694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.930707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342939511311105:11421]
2025-10-09T21:51:22.930719Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:22.930730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:22.930752Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|61ea71ab-32413bdb-bea89167-46a347c4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:22.930764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:22.930770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:22.930772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:22.930778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:22.930789Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:22.930807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:22.930949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:22.930959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:22.930975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:22.931047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2121 partition: 0 MaxSeqNo: 25 sessionId: src_id|61ea71ab-32413bdb-bea89167-46a347c4_0
2025-10-09T21:51:22.931340Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48b41279-7dbe0a56-4cb6af55-dc12f4af_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|61ea71ab-32413bdb-bea89167-46a347c4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:22.933624Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|61ea71ab-32413bdb-bea89167-46a347c4_0] Write session: try to update token
2025-10-09T21:51:22.933637Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|61ea71ab-32413bdb-bea89167-46a347c4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:22.944363Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:22.944382Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:22.944384Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:22.944386Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:22.944395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342939511311105:11421]
2025-10-09T21:51:22.944401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342939511311105:11421]
2025-10-09T21:51:22.944405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342939511311105:11421]
2025-10-09T21:51:22.944441Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:22.944443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:22.944445Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:22.944447Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:22.944450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:22.944453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:22.944585Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:22.945530Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2121 sessionId: src_id|61ea71ab-32413bdb-bea89167-46a347c4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:22.947764Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:23.008508Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342944168113284:9464]
2025-10-09T21:51:23.008768Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 2120, pipe: [1:7559342944168113284:9464]
2025-10-09T21:51:23.090109Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:23.090702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.090734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.090792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168113284:9464], now have 1 active actors on pipe
2025-10-09T21:51:23.090982Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:23.090987Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:23.091000Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:23.091007Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:23.091053Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342944168113284:9464]
2025-10-09T21:51:23.091059Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2121 reason: error at write: Disconnected sessionId: src_id|61ea71ab-32413bdb-bea89167-46a347c4_0
2025-10-09T21:51:23.091232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2121 sessionId: src_id|61ea71ab-32413bdb-bea89167-46a347c4_0 is DEAD
2025-10-09T21:51:23.091412Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61ea71ab-32413bdb-bea89167-46a347c4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:23.091422Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61ea71ab-32413bdb-bea89167-46a347c4_0] Write session will restart in 0.006426s
2025-10-09T21:51:23.091456Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61ea71ab-32413bdb-bea89167-46a347c4_0] Write session: Do CDS request
2025-10-09T21:51:23.091461Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61ea71ab-32413bdb-bea89167-46a347c4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:23.098461Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|61ea71ab-32413bdb-bea89167-46a347c4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:23.098527Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:23.098545Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2122
2025-10-09T21:51:23.098684Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2122 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:23.098730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2122 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:23.098742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2122 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:23.098747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:23.099143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2122 sessionId:  describe result for acl check
2025-10-09T21:51:23.099180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:23.099189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:23.099190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:23.099195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342944168113322:9303] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:23.099724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342944168113322:9303] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:23.101906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342944168113322:9303] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:23.104112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342944168113322:9303] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:23.104126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342944168113322:9303] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:23.104135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342944168113322:9303] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:23.104142Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2122 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:23.104531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:23.106695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:23.106868Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:23.106878Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:23.106890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:23.106940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:23.106978Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342943806278608:11423]
2025-10-09T21:51:23.107562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:23.107812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:23.107854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:23.108168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:23.108216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.108222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.108223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.108224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.108226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.108227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.108228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.108229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.108230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.108235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.108246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.108250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.108255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.108259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.108263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.108267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.108270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.108274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.108280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.108288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.108298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.108304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.108313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.108319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.108328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.108333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.108342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.108348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.108356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.108361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.108368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.108377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.108382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.108387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.108391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.108396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.108423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:23.114721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:23.115758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:23.115772Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:23.115774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:23.115777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:23.115782Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2121 [2:7559342943806278608:11423]
2025-10-09T21:51:23.115793Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:23.115808Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:23.115810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:23.115813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:23.115814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:23.115828Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:23.115836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:23.115843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:23.115889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.115894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.115896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.115898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:23.116117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:23.116126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.116135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.116094Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2121
2025-10-09T21:51:23.116143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.116101Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2121
2025-10-09T21:51:23.116153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:23.116171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168113286:2432], now have 1 active actors on pipe
2025-10-09T21:51:23.116173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168113484:9303], now have 1 active actors on pipe
2025-10-09T21:51:23.116187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.116194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.116212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342943806278603:11423]
2025-10-09T21:51:23.116217Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342943806278603:11423]
2025-10-09T21:51:23.116219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342943806278603:11423]
2025-10-09T21:51:23.116229Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:23.116244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:23.116253Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:23.116257Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:23.116260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:23.116264Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:23.116267Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:23.116274Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e86e0021-ab041b5d-3f7b7905-bac835d1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:23.116281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:23.116288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:23.116289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:23.116297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:23.116305Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:23.116332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342943806278603:11423]
2025-10-09T21:51:23.116353Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:23.116362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:23.116371Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:23.116479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.116485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.116498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:23.116595Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2122 partition: 0 MaxSeqNo: 25 sessionId: src_id|e86e0021-ab041b5d-3f7b7905-bac835d1_0
2025-10-09T21:51:23.117010Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61ea71ab-32413bdb-bea89167-46a347c4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e86e0021-ab041b5d-3f7b7905-bac835d1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:23.119673Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e86e0021-ab041b5d-3f7b7905-bac835d1_0] Write session: try to update token
2025-10-09T21:51:23.119691Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e86e0021-ab041b5d-3f7b7905-bac835d1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:23.128145Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:23.128164Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:23.128174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342943806278603:11423]
2025-10-09T21:51:23.128210Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:23.128223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:23.128468Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:23.128611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:23.128786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:23.128926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:23.129118Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:23.129166Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:23.136076Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2122 sessionId: src_id|e86e0021-ab041b5d-3f7b7905-bac835d1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:23.138325Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:23.138554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.138578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.138621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:23.138712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:23.141268Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:23.141282Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:23.141285Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:23.141286Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:23.141296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342943806278603:11423]
2025-10-09T21:51:23.141301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342943806278603:11423]
2025-10-09T21:51:23.141310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342943806278603:11423]
2025-10-09T21:51:23.141347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:23.141349Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:23.141353Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:23.141357Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:23.141361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.141377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:23.141547Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:23.191281Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342944168113497:9464]
2025-10-09T21:51:23.191535Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 2121, pipe: [1:7559342944168113497:9464]
2025-10-09T21:51:23.283248Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:23.290182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.290205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.290339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:23.290390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:23.290446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:23.290502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:23.290626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:23.290636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:23.290819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:23.291022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:23.291084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:23.291174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:23.291210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:23.291217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:23.291291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:23.291362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:23.291415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:23.291470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:23.291507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:23.291513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:23.291572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:23.291633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:23.291714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:23.291767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:23.291806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:23.291811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:23.291867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:23.291950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:23.292032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:23.292085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:23.292143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:23.292149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:23.292207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:23.292286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:23.292341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:23.292399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:23.292447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:23.292453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:23.292505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:23.292555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:23.292609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:23.292664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:23.292772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:23.292778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:23.292826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:23.292874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:23.292925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:23.292988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:23.293025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:23.293030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:23.293080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:23.293133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:23.293187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:23.293238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:23.293275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:23.293281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:23.293331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:23.293389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:23.293440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:23.293490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:23.293527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:23.293532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:23.293578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168113497:9464], now have 1 active actors on pipe
2025-10-09T21:51:23.293772Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342944168113497:9464]
2025-10-09T21:51:23.293791Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:23.293814Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:23.293817Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:23.293852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2122 reason: error at write: Disconnected sessionId: src_id|e86e0021-ab041b5d-3f7b7905-bac835d1_0
2025-10-09T21:51:23.294104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2122 sessionId: src_id|e86e0021-ab041b5d-3f7b7905-bac835d1_0 is DEAD
2025-10-09T21:51:23.294267Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e86e0021-ab041b5d-3f7b7905-bac835d1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:23.294279Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e86e0021-ab041b5d-3f7b7905-bac835d1_0] Write session will restart in 0.009434s
2025-10-09T21:51:23.294301Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e86e0021-ab041b5d-3f7b7905-bac835d1_0] Write session: Do CDS request
2025-10-09T21:51:23.294306Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e86e0021-ab041b5d-3f7b7905-bac835d1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:23.294432Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:23.304456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e86e0021-ab041b5d-3f7b7905-bac835d1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:23.304676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:23.304698Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2123
2025-10-09T21:51:23.304792Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2123 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:23.304815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2123 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:23.304823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2123 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:23.304827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:23.305211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2123 sessionId:  describe result for acl check
2025-10-09T21:51:23.305247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:23.305258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:23.305259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:23.305265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342944168113670:9304] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:23.305823Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342944168113670:9304] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:23.305874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:23.308122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:23.308124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342944168113670:9304] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:23.308324Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:23.308332Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:23.308342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:23.308387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:23.308416Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342943806278809:11425]
2025-10-09T21:51:23.308918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:23.309128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:23.309168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:23.309471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:23.309516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.309525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.309526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.309527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.309528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.309529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.309530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.309531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.309532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.309537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.309547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.309552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.309556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.309559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.309563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.309567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.309571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.309574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.309580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.309587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.309593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.309598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.309602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.309607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.309611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.309616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.309625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.309630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.309638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.309643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.309653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.309663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.309671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.309676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.309681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.309686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.309718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:23.309922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342944168113670:9304] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:23.309931Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342944168113670:9304] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:23.309934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342944168113670:9304] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:23.309939Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2123 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:23.315507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:23.316533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:23.316548Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:23.316550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:23.316554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:23.316559Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2122 [2:7559342943806278809:11425]
2025-10-09T21:51:23.316570Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:23.316589Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:23.316596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:23.316600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:23.316602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:23.316617Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:23.316627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:23.316638Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:23.316690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.316697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.316698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.316701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:23.316761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:23.316769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.316780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.316796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.316818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:23.316842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168113502:2432], now have 1 active actors on pipe
2025-10-09T21:51:23.316849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168113712:9304], now have 1 active actors on pipe
2025-10-09T21:51:23.316856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342943806278804:11425]
2025-10-09T21:51:23.316858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342943806278804:11425]
2025-10-09T21:51:23.316861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342943806278804:11425]
2025-10-09T21:51:23.316874Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:23.316844Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2122
2025-10-09T21:51:23.316846Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2122
2025-10-09T21:51:23.316884Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:23.316890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:23.316894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:23.316896Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:23.316904Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:23.316911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:23.316930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.316939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.316965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342943806278804:11425]
2025-10-09T21:51:23.316977Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:23.316983Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:23.316998Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|663076f6-59b9d0e6-5ad33c16-647b96d4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:23.317009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:23.317017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:23.317019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:23.317029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:23.317037Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:23.317073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:23.317204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.317213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.317228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:23.317279Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2123 partition: 0 MaxSeqNo: 25 sessionId: src_id|663076f6-59b9d0e6-5ad33c16-647b96d4_0
2025-10-09T21:51:23.317546Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e86e0021-ab041b5d-3f7b7905-bac835d1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|663076f6-59b9d0e6-5ad33c16-647b96d4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:23.319804Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|663076f6-59b9d0e6-5ad33c16-647b96d4_0] Write session: try to update token
2025-10-09T21:51:23.319817Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|663076f6-59b9d0e6-5ad33c16-647b96d4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:23.330685Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:23.330702Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:23.330704Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:23.330706Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:23.330717Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342943806278804:11425]
2025-10-09T21:51:23.330721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342943806278804:11425]
2025-10-09T21:51:23.330724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342943806278804:11425]
2025-10-09T21:51:23.330767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:23.330775Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:23.330780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.330780Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:23.330782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:23.330783Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:23.330909Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:23.332087Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2123 sessionId: src_id|663076f6-59b9d0e6-5ad33c16-647b96d4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:23.341327Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:23.394055Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342944168113720:9464]
2025-10-09T21:51:23.394275Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 2122, pipe: [1:7559342944168113720:9464]
2025-10-09T21:51:23.467108Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:23.467461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.467474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.467492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168113720:9464], now have 1 active actors on pipe
2025-10-09T21:51:23.467729Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:23.467779Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342944168113720:9464]
2025-10-09T21:51:23.467799Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:23.467801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:23.467856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2123 reason: error at write: Disconnected sessionId: src_id|663076f6-59b9d0e6-5ad33c16-647b96d4_0
2025-10-09T21:51:23.468027Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2123 sessionId: src_id|663076f6-59b9d0e6-5ad33c16-647b96d4_0 is DEAD
2025-10-09T21:51:23.468166Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|663076f6-59b9d0e6-5ad33c16-647b96d4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:23.468172Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|663076f6-59b9d0e6-5ad33c16-647b96d4_0] Write session will restart in 0.006347s
2025-10-09T21:51:23.468189Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|663076f6-59b9d0e6-5ad33c16-647b96d4_0] Write session: Do CDS request
2025-10-09T21:51:23.468196Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|663076f6-59b9d0e6-5ad33c16-647b96d4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:23.468661Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:23.475429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|663076f6-59b9d0e6-5ad33c16-647b96d4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:23.475516Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:23.475529Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2124
2025-10-09T21:51:23.475671Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2124 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:23.475701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2124 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:23.475710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2124 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:23.475713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:23.476034Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2124 sessionId:  describe result for acl check
2025-10-09T21:51:23.476080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:23.476088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:23.476090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:23.476095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342944168113938:9293] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:23.476568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342944168113938:9293] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:23.478210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342944168113938:9293] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:23.478883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:23.479290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:23.479433Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:23.479442Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:23.479451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:23.479494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:23.479523Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342943806279024:11433]
2025-10-09T21:51:23.479935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:23.480170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:23.480206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:23.480414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342944168113938:9293] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:23.480423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342944168113938:9293] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:23.480426Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342944168113938:9293] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:23.480430Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2124 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:23.480496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:23.480538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.480543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.480544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.480545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.480546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.480547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.480548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.480549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.480551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.480555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.480565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.480569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.480572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.480576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.480580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.480583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.480587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.480590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.480594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.480599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.480606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.480609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.480612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.480616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.480619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.480622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.480624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.480628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.480631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.480634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.480637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.480639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.480642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.480644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.480648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.480653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.480678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:23.486471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:23.487534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:23.487545Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:23.487547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:23.487549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:23.487554Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2123 [2:7559342943806279024:11433]
2025-10-09T21:51:23.487563Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:23.487577Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:23.487579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:23.487582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:23.487583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:23.487595Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:23.487601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:23.487606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:23.487644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.487648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.487649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.487650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:23.487692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:23.487697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.487703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.487712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.487725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:23.487741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168114089:9293], now have 1 active actors on pipe
2025-10-09T21:51:23.487747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168113736:2432], now have 1 active actors on pipe
2025-10-09T21:51:23.487751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342943806279019:11433]
2025-10-09T21:51:23.487752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342943806279019:11433]
2025-10-09T21:51:23.487753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342943806279019:11433]
2025-10-09T21:51:23.487762Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:23.487775Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:23.487782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:23.487785Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:23.487786Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:23.487788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:23.487790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:23.487807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342943806279019:11433]
2025-10-09T21:51:23.487810Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:23.487811Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:23.487821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.487824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.487786Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2123
2025-10-09T21:51:23.487787Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2123
2025-10-09T21:51:23.487857Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c8a3db90-b453426a-26cadcc4-ec3e5a8a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:23.487866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:23.487869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:23.487870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:23.487876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:23.487883Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:23.487899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:23.488042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.488056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.488076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:23.488158Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2124 partition: 0 MaxSeqNo: 25 sessionId: src_id|c8a3db90-b453426a-26cadcc4-ec3e5a8a_0
2025-10-09T21:51:23.488395Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|663076f6-59b9d0e6-5ad33c16-647b96d4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c8a3db90-b453426a-26cadcc4-ec3e5a8a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:23.490570Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8a3db90-b453426a-26cadcc4-ec3e5a8a_0] Write session: try to update token
2025-10-09T21:51:23.490583Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8a3db90-b453426a-26cadcc4-ec3e5a8a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:23.501054Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:23.501077Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:23.501080Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:23.501081Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:23.501091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342943806279019:11433]
2025-10-09T21:51:23.501094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342943806279019:11433]
2025-10-09T21:51:23.501097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342943806279019:11433]
2025-10-09T21:51:23.501137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:23.501140Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:23.501143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.501145Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:23.501145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:23.501150Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:23.501276Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:23.501838Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2124 sessionId: src_id|c8a3db90-b453426a-26cadcc4-ec3e5a8a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:23.504013Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:23.568130Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342944168114109:9464]
2025-10-09T21:51:23.568381Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 2123, pipe: [1:7559342944168114109:9464]
2025-10-09T21:51:23.631159Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:23.631669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.631684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.631721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168114109:9464], now have 1 active actors on pipe
2025-10-09T21:51:23.631937Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:23.631941Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342944168114109:9464]
2025-10-09T21:51:23.631952Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:23.631964Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:23.632012Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2124 reason: error at write: Disconnected sessionId: src_id|c8a3db90-b453426a-26cadcc4-ec3e5a8a_0
2025-10-09T21:51:23.632187Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2124 sessionId: src_id|c8a3db90-b453426a-26cadcc4-ec3e5a8a_0 is DEAD
2025-10-09T21:51:23.632200Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:23.632326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8a3db90-b453426a-26cadcc4-ec3e5a8a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:23.632333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8a3db90-b453426a-26cadcc4-ec3e5a8a_0] Write session will restart in 0.006966s
2025-10-09T21:51:23.632370Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8a3db90-b453426a-26cadcc4-ec3e5a8a_0] Write session: Do CDS request
2025-10-09T21:51:23.632377Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8a3db90-b453426a-26cadcc4-ec3e5a8a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:23.640425Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c8a3db90-b453426a-26cadcc4-ec3e5a8a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:23.640481Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:23.640496Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2125
2025-10-09T21:51:23.640630Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2125 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:23.640665Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2125 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:23.640675Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2125 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:23.640679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:23.641539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2125 sessionId:  describe result for acl check
2025-10-09T21:51:23.641570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:23.641572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:23.641574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:23.641579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342944168114423:9286] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:23.642119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342944168114423:9286] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:23.642163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:23.642816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:23.642962Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:23.642972Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:23.642983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:23.643034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:23.643068Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342943806279253:11441]
2025-10-09T21:51:23.643498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:23.643696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:23.643734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:23.643899Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342944168114423:9286] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:23.644012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:23.644063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.644069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.644070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.644071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.644072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.644073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.644075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.644076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.644077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.644081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.644091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.644095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.644099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.644108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.644111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.644116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.644122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.644125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.644133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.644145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.644155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.644165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.644169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.644175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.644183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.644188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.644197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.644202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.644208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.644211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.644214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.644221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.644223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.644226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.644229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.644232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.644247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:23.648626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342944168114423:9286] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:23.648636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342944168114423:9286] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:23.648638Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342944168114423:9286] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:23.648644Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2125 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:23.649950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:23.651018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:23.651031Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:23.651033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:23.651037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:23.651042Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2124 [2:7559342943806279253:11441]
2025-10-09T21:51:23.651053Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:23.651072Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:23.651082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:23.651084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:23.651087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:23.651102Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:23.651107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:23.651114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:23.651165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.651172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.651174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.651177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:23.651234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:23.651241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.651252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.651267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.651287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:23.651310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168114110:2432], now have 1 active actors on pipe
2025-10-09T21:51:23.651317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168114464:9286], now have 1 active actors on pipe
2025-10-09T21:51:23.651311Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2124
2025-10-09T21:51:23.651314Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2124
2025-10-09T21:51:23.651325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342943806279248:11441]
2025-10-09T21:51:23.651333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342943806279248:11441]
2025-10-09T21:51:23.651335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342943806279248:11441]
2025-10-09T21:51:23.651342Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:23.651355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:23.651355Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:23.651358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:23.651360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:23.651368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:23.651371Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:23.651385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.651394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.651425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342943806279248:11441]
2025-10-09T21:51:23.651436Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:23.651446Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:23.651452Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e949ca90-1424ce45-88548c66-bc0d9992_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:23.651459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:23.651463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:23.651470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:23.651480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:23.651492Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:23.651529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:23.651637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.651646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.651659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:23.651740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2125 partition: 0 MaxSeqNo: 25 sessionId: src_id|e949ca90-1424ce45-88548c66-bc0d9992_0
2025-10-09T21:51:23.652004Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c8a3db90-b453426a-26cadcc4-ec3e5a8a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e949ca90-1424ce45-88548c66-bc0d9992_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:23.654273Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e949ca90-1424ce45-88548c66-bc0d9992_0] Write session: try to update token
2025-10-09T21:51:23.654286Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e949ca90-1424ce45-88548c66-bc0d9992_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:23.665004Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:23.665022Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:23.665024Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:23.665026Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:23.665035Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342943806279248:11441]
2025-10-09T21:51:23.665039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342943806279248:11441]
2025-10-09T21:51:23.665043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342943806279248:11441]
2025-10-09T21:51:23.665077Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:23.665082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:23.665088Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:23.665089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.665090Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:23.665091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:23.665225Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:23.666045Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2125 sessionId: src_id|e949ca90-1424ce45-88548c66-bc0d9992_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:23.668197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:23.731911Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342944168114472:9464]
2025-10-09T21:51:23.732157Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 2124, pipe: [1:7559342944168114472:9464]
2025-10-09T21:51:23.798239Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:23.798679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.798691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.798730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168114472:9464], now have 1 active actors on pipe
2025-10-09T21:51:23.798928Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:23.798932Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:23.798944Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:23.798947Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:23.798978Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342944168114472:9464]
2025-10-09T21:51:23.799010Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2125 reason: error at write: Disconnected sessionId: src_id|e949ca90-1424ce45-88548c66-bc0d9992_0
2025-10-09T21:51:23.799181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2125 sessionId: src_id|e949ca90-1424ce45-88548c66-bc0d9992_0 is DEAD
2025-10-09T21:51:23.799326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e949ca90-1424ce45-88548c66-bc0d9992_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:23.799335Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e949ca90-1424ce45-88548c66-bc0d9992_0] Write session will restart in 0.006265s
2025-10-09T21:51:23.799369Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e949ca90-1424ce45-88548c66-bc0d9992_0] Write session: Do CDS request
2025-10-09T21:51:23.799374Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e949ca90-1424ce45-88548c66-bc0d9992_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:23.806397Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e949ca90-1424ce45-88548c66-bc0d9992_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:23.806496Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:23.806511Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2126
2025-10-09T21:51:23.806604Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2126 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:23.806634Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2126 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:23.806643Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2126 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:23.806647Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:23.807070Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2126 sessionId:  describe result for acl check
2025-10-09T21:51:23.807104Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:23.807119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:23.807121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:23.807126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342944168114699:9280] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:23.807620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342944168114699:9280] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:23.809301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342944168114699:9280] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:23.809642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:23.810278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:23.810429Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:23.810438Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:23.810448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:23.810498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:23.810531Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342943806279463:11445]
2025-10-09T21:51:23.811051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:23.811215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342944168114699:9280] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:23.811226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342944168114699:9280] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:23.811228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342944168114699:9280] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:23.811233Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2126 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:23.811301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:23.811338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:23.811628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:23.811661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.811665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.811666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.811666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.811667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.811668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.811668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.811669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.811669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.811672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.811678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.811680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.811682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.811684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.811686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.811688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.811689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.811691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.811696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.811701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.811704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.811708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.811711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.811715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.811717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.811720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.811722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.811725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.811727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.811730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.811733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.811735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.811738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.811741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.811744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.811747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.811761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:23.817440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:23.818562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:23.818573Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:23.818575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:23.818577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:23.818581Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2125 [2:7559342943806279463:11445]
2025-10-09T21:51:23.818588Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:23.818599Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:23.818604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:23.818606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:23.818607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:23.818618Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:23.818624Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:23.818630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:23.818668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.818673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.818674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.818676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:23.818732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:23.818739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.818745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.818754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.818767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:23.818784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168114826:9280], now have 1 active actors on pipe
2025-10-09T21:51:23.818791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168114473:2432], now have 1 active actors on pipe
2025-10-09T21:51:23.818798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342943806279457:11445]
2025-10-09T21:51:23.818805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342943806279457:11445]
2025-10-09T21:51:23.818807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342943806279457:11445]
2025-10-09T21:51:23.818819Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:23.818833Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:23.818836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:23.818842Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:23.818844Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:23.818818Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2125
2025-10-09T21:51:23.818845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:23.818847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:23.818822Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2125
2025-10-09T21:51:23.818862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342943806279457:11445]
2025-10-09T21:51:23.818864Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:23.818873Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:23.818877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.818879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.818910Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f17e068-e07d0689-a05696f8-28275ed2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:23.818919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:23.818922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:23.818923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:23.818929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:23.818939Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:23.818955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:23.819093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.819101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.819112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:23.819187Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2126 partition: 0 MaxSeqNo: 25 sessionId: src_id|f17e068-e07d0689-a05696f8-28275ed2_0
2025-10-09T21:51:23.819437Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e949ca90-1424ce45-88548c66-bc0d9992_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f17e068-e07d0689-a05696f8-28275ed2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:23.821552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f17e068-e07d0689-a05696f8-28275ed2_0] Write session: try to update token
2025-10-09T21:51:23.821564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f17e068-e07d0689-a05696f8-28275ed2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:23.832091Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:23.832112Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:23.832114Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:23.832116Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:23.832125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342943806279457:11445]
2025-10-09T21:51:23.832129Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342943806279457:11445]
2025-10-09T21:51:23.832133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342943806279457:11445]
2025-10-09T21:51:23.832168Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:23.832174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:23.832178Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:23.832180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.832181Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:23.832183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:23.832319Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:23.832862Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2126 sessionId: src_id|f17e068-e07d0689-a05696f8-28275ed2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:23.835021Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:23.899390Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342944168114838:9464]
2025-10-09T21:51:23.899645Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 2125, pipe: [1:7559342944168114838:9464]
2025-10-09T21:51:23.973941Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:23.974195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.974212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.974253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168114838:9464], now have 1 active actors on pipe
2025-10-09T21:51:23.974402Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:23.974495Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:23.974507Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:23.974510Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342944168114838:9464]
2025-10-09T21:51:23.974531Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:23.974560Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2126 reason: error at write: Disconnected sessionId: src_id|f17e068-e07d0689-a05696f8-28275ed2_0
2025-10-09T21:51:23.974726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2126 sessionId: src_id|f17e068-e07d0689-a05696f8-28275ed2_0 is DEAD
2025-10-09T21:51:23.974856Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f17e068-e07d0689-a05696f8-28275ed2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:23.974866Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f17e068-e07d0689-a05696f8-28275ed2_0] Write session will restart in 0.009147s
2025-10-09T21:51:23.974886Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f17e068-e07d0689-a05696f8-28275ed2_0] Write session: Do CDS request
2025-10-09T21:51:23.974892Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f17e068-e07d0689-a05696f8-28275ed2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:23.985512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f17e068-e07d0689-a05696f8-28275ed2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:23.985574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:23.985585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:23.985588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2127
2025-10-09T21:51:23.985707Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2127 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:23.985739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2127 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:23.985749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2127 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:23.985752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:23.986176Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2127 sessionId:  describe result for acl check
2025-10-09T21:51:23.986208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:23.986215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:23.986217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:23.986222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342944168115159:9270] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:23.986333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:23.986484Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:23.986494Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:23.986505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:23.986554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:23.986588Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342943806279668:11447]
2025-10-09T21:51:23.986754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342944168115159:9270] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:23.987168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:23.987411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:23.987452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:23.987749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:23.987790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.987796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.987797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.987798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.987799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.987800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.987802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.987803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.987804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.987808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.987819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.987823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.987827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.987831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.987839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.987843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.987852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.987856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.987867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.987875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:23.987885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.987891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:23.987899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.987904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:23.987913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.987918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:23.987926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.987936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:23.987946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.987956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:23.987961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.987967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.987971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.987980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.987987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.987993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.988027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:23.988586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342944168115159:9270] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:23.993539Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342944168115159:9270] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:23.993550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342944168115159:9270] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:23.993552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342944168115159:9270] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:23.993559Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2127 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:23.993985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:23.995039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:23.995056Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:23.995059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:23.995062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:23.995068Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2126 [2:7559342943806279668:11447]
2025-10-09T21:51:23.995080Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:23.995099Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:23.995107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:23.995110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:23.995112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:23.995128Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:23.995139Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:23.995146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:23.995196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.995203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.995205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.995209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:23.995272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:23.995279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:23.995291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:23.995307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:23.995329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:23.995352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168115195:9270], now have 1 active actors on pipe
2025-10-09T21:51:23.995361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342944168114842:2432], now have 1 active actors on pipe
2025-10-09T21:51:23.995352Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2126
2025-10-09T21:51:23.995355Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2126
2025-10-09T21:51:23.995368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342943806279663:11447]
2025-10-09T21:51:23.995370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342943806279663:11447]
2025-10-09T21:51:23.995373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342943806279663:11447]
2025-10-09T21:51:23.995381Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:23.995394Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:23.995399Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:23.995402Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:23.995405Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:23.995409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:23.995417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:23.995436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.995445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.995472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342943806279663:11447]
2025-10-09T21:51:23.995484Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:23.995499Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:23.995505Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4e96f357-da32363d-c1ba5d4a-9cd9a9f0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:23.995511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:23.995515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:23.995517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:23.995526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:23.995540Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:23.995570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:23.995718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:23.995731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:23.995745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:23.995830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2127 partition: 0 MaxSeqNo: 25 sessionId: src_id|4e96f357-da32363d-c1ba5d4a-9cd9a9f0_0
2025-10-09T21:51:23.996110Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f17e068-e07d0689-a05696f8-28275ed2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4e96f357-da32363d-c1ba5d4a-9cd9a9f0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:23.998429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e96f357-da32363d-c1ba5d4a-9cd9a9f0_0] Write session: try to update token
2025-10-09T21:51:23.998446Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e96f357-da32363d-c1ba5d4a-9cd9a9f0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:24.009545Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:24.009564Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:24.009566Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:24.009568Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:24.009577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342943806279663:11447]
2025-10-09T21:51:24.009584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342943806279663:11447]
2025-10-09T21:51:24.009588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342943806279663:11447]
2025-10-09T21:51:24.009627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:24.009632Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:24.009636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.009637Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:24.009638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:24.009640Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:24.009761Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:24.010285Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2127 sessionId: src_id|4e96f357-da32363d-c1ba5d4a-9cd9a9f0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:24.012429Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:24.074492Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342948463082499:9464]
2025-10-09T21:51:24.074721Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 2126, pipe: [1:7559342948463082499:9464]
2025-10-09T21:51:24.150781Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:24.151028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:24.151046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:24.151095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463082499:9464], now have 1 active actors on pipe
2025-10-09T21:51:24.151257Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:24.151333Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342948463082499:9464]
2025-10-09T21:51:24.151411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:24.151421Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:24.151427Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:24.151478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2127 reason: error at write: Disconnected sessionId: src_id|4e96f357-da32363d-c1ba5d4a-9cd9a9f0_0
2025-10-09T21:51:24.151649Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2127 sessionId: src_id|4e96f357-da32363d-c1ba5d4a-9cd9a9f0_0 is DEAD
2025-10-09T21:51:24.151789Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e96f357-da32363d-c1ba5d4a-9cd9a9f0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:24.151799Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e96f357-da32363d-c1ba5d4a-9cd9a9f0_0] Write session will restart in 0.008496s
2025-10-09T21:51:24.151817Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e96f357-da32363d-c1ba5d4a-9cd9a9f0_0] Write session: Do CDS request
2025-10-09T21:51:24.151822Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e96f357-da32363d-c1ba5d4a-9cd9a9f0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:24.161494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4e96f357-da32363d-c1ba5d4a-9cd9a9f0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:24.161613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:24.161624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2128
2025-10-09T21:51:24.161754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2128 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:24.161779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2128 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:24.161790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2128 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:24.161793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:24.162114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:24.162207Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2128 sessionId:  describe result for acl check
2025-10-09T21:51:24.162245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:24.162251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:24.162253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:24.162259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342948463082818:9267] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:24.162816Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342948463082818:9267] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:24.162993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:24.163112Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:24.163120Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:24.163128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:24.163161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:24.163188Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342948101247172:11449]
2025-10-09T21:51:24.163542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:24.163800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:24.163827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:24.164020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:24.164050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.164054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.164055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.164056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.164056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.164057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.164057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.164058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.164059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.164061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.164067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.164070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.164072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.164074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.164076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.164078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.164080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.164082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.164085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.164090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.164095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.164098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.164100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.164104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.164106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.164109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.164112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.164115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.164117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.164121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.164123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.164127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.164129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.164132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.164135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.164138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.164153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:24.164727Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342948463082818:9267] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:24.169731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342948463082818:9267] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:24.169744Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342948463082818:9267] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:24.169747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342948463082818:9267] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:24.169755Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2128 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:24.170142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:24.171145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:24.171157Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:24.171159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:24.171162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:24.171166Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2127 [2:7559342948101247172:11449]
2025-10-09T21:51:24.171174Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:24.171187Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:24.171192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:24.171194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:24.171195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:24.171207Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:24.171214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:24.171219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:24.171255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.171259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.171261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.171262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:24.171309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:24.171314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.171321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.171331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.171344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:24.171361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463082859:9267], now have 1 active actors on pipe
2025-10-09T21:51:24.171367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463082501:2432], now have 1 active actors on pipe
2025-10-09T21:51:24.171371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342948101247166:11449]
2025-10-09T21:51:24.171373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342948101247166:11449]
2025-10-09T21:51:24.171374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342948101247166:11449]
2025-10-09T21:51:24.171382Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:24.171393Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:24.171396Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:24.171403Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:24.171403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:24.171404Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:24.171405Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:24.171423Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342948101247166:11449]
2025-10-09T21:51:24.171429Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:24.171440Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:24.171421Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2127
2025-10-09T21:51:24.171447Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2127
2025-10-09T21:51:24.171616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:24.171626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:24.171658Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|12a19c86-25ed23d2-5d283674-70d7d428_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:24.171667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:24.171670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:24.171675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:24.171683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:24.171694Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:24.171708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:24.171848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:24.171859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:24.171877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:24.171944Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2128 partition: 0 MaxSeqNo: 25 sessionId: src_id|12a19c86-25ed23d2-5d283674-70d7d428_0
2025-10-09T21:51:24.172207Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4e96f357-da32363d-c1ba5d4a-9cd9a9f0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|12a19c86-25ed23d2-5d283674-70d7d428_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:24.174657Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12a19c86-25ed23d2-5d283674-70d7d428_0] Write session: try to update token
2025-10-09T21:51:24.174676Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12a19c86-25ed23d2-5d283674-70d7d428_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:24.185277Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:24.185295Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:24.185296Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:24.185297Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:24.185305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342948101247166:11449]
2025-10-09T21:51:24.185308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342948101247166:11449]
2025-10-09T21:51:24.185311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342948101247166:11449]
2025-10-09T21:51:24.185344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:24.185351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.185353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:24.185353Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:24.185357Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:24.185360Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:24.185484Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:24.187987Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2128 sessionId: src_id|12a19c86-25ed23d2-5d283674-70d7d428_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:24.190201Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:24.251641Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342948463082869:9464]
2025-10-09T21:51:24.251888Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 2127, pipe: [1:7559342948463082869:9464]
2025-10-09T21:51:24.320796Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:24.321150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:24.321164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:24.321202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463082869:9464], now have 1 active actors on pipe
2025-10-09T21:51:24.321351Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:24.321393Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:24.321395Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342948463082869:9464]
2025-10-09T21:51:24.321406Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:24.321416Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:24.321463Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2128 reason: error at write: Disconnected sessionId: src_id|12a19c86-25ed23d2-5d283674-70d7d428_0
2025-10-09T21:51:24.321617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2128 sessionId: src_id|12a19c86-25ed23d2-5d283674-70d7d428_0 is DEAD
2025-10-09T21:51:24.321781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12a19c86-25ed23d2-5d283674-70d7d428_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:24.321794Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12a19c86-25ed23d2-5d283674-70d7d428_0] Write session will restart in 0.007858s
2025-10-09T21:51:24.321820Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12a19c86-25ed23d2-5d283674-70d7d428_0] Write session: Do CDS request
2025-10-09T21:51:24.321827Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12a19c86-25ed23d2-5d283674-70d7d428_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:24.330342Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|12a19c86-25ed23d2-5d283674-70d7d428_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:24.330448Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:24.330462Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2129
2025-10-09T21:51:24.330544Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2129 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:24.330575Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2129 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:24.330584Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2129 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:24.330588Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:24.331002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2129 sessionId:  describe result for acl check
2025-10-09T21:51:24.331033Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:24.331040Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:24.331041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:24.331046Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342948463083192:9261] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:24.331446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342948463083192:9261] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:24.332300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:24.332906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:24.333047Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:24.333056Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:24.333066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:24.333116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:24.333150Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342948101247379:11451]
2025-10-09T21:51:24.333190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342948463083192:9261] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:24.333711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:24.333928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:24.333962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:24.334271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:24.334311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.334317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.334318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.334319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.334320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.334321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.334322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.334323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.334324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.334329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.334339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.334343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.334351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.334354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.334358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.334362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.334366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.334369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.334378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.334385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.334396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.334405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.334413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.334418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.334426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.334431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.334439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.334448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.334455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.334460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.334469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.334478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.334486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.334495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.334506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.334514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.334546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:24.335045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342948463083192:9261] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:24.335059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342948463083192:9261] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:24.335061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342948463083192:9261] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:24.335067Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2129 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:24.340243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:24.341318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:24.341329Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:24.341331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:24.341333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:24.341338Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2128 [2:7559342948101247379:11451]
2025-10-09T21:51:24.341345Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:24.341356Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:24.341358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:24.341359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:24.341361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:24.341385Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:24.341392Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:24.341398Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:24.341438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.341443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.341444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.341446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:24.341503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:24.341510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.341517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.341523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.341534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:24.341551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463083234:9261], now have 1 active actors on pipe
2025-10-09T21:51:24.341556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463082875:2432], now have 1 active actors on pipe
2025-10-09T21:51:24.341562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342948101247374:11451]
2025-10-09T21:51:24.341567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342948101247374:11451]
2025-10-09T21:51:24.341568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342948101247374:11451]
2025-10-09T21:51:24.341580Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:24.341594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:24.341601Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:24.341604Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:24.341605Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:24.341605Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:24.341607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:24.341626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342948101247374:11451]
2025-10-09T21:51:24.341631Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:24.341607Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2128
2025-10-09T21:51:24.341632Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:24.341619Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2128
2025-10-09T21:51:24.341648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:24.341654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:24.341681Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ebb9f106-e6cd094e-dd8dfc30-e9f3e09a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:24.341690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:24.341692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:24.341697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:24.341704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:24.341714Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:24.341739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:24.341888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:24.341893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:24.341903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:24.341990Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2129 partition: 0 MaxSeqNo: 25 sessionId: src_id|ebb9f106-e6cd094e-dd8dfc30-e9f3e09a_0
2025-10-09T21:51:24.342244Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|12a19c86-25ed23d2-5d283674-70d7d428_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ebb9f106-e6cd094e-dd8dfc30-e9f3e09a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:24.344458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb9f106-e6cd094e-dd8dfc30-e9f3e09a_0] Write session: try to update token
2025-10-09T21:51:24.344471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb9f106-e6cd094e-dd8dfc30-e9f3e09a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:24.354741Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:24.354759Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:24.354760Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:24.354762Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:24.354769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342948101247374:11451]
2025-10-09T21:51:24.354773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342948101247374:11451]
2025-10-09T21:51:24.354776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342948101247374:11451]
2025-10-09T21:51:24.354807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:24.354811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.354812Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:24.354814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:24.354817Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:24.354820Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:24.354925Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:24.355551Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2129 sessionId: src_id|ebb9f106-e6cd094e-dd8dfc30-e9f3e09a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:24.357800Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:24.421628Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342948463083242:9464]
2025-10-09T21:51:24.421858Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 2128, pipe: [1:7559342948463083242:9464]
2025-10-09T21:51:24.488301Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:24.488499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:24.488514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:24.488550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463083242:9464], now have 1 active actors on pipe
2025-10-09T21:51:24.488693Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:24.488753Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:24.488804Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342948463083242:9464]
2025-10-09T21:51:24.488825Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:24.488897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2129 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ebb9f106-e6cd094e-dd8dfc30-e9f3e09a_0
2025-10-09T21:51:24.489098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2129 sessionId: src_id|ebb9f106-e6cd094e-dd8dfc30-e9f3e09a_0 is DEAD
2025-10-09T21:51:24.489212Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb9f106-e6cd094e-dd8dfc30-e9f3e09a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:24.489229Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb9f106-e6cd094e-dd8dfc30-e9f3e09a_0] Write session will restart in 0.009687s
2025-10-09T21:51:24.489249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb9f106-e6cd094e-dd8dfc30-e9f3e09a_0] Write session: Do CDS request
2025-10-09T21:51:24.489254Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb9f106-e6cd094e-dd8dfc30-e9f3e09a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:24.500435Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb9f106-e6cd094e-dd8dfc30-e9f3e09a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:24.500445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:24.500556Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:24.500573Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2130
2025-10-09T21:51:24.500667Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2130 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:24.500697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2130 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:24.500707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2130 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:24.500716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:24.501045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2130 sessionId:  describe result for acl check
2025-10-09T21:51:24.501076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:24.501082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:24.501084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:24.501090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342948463083578:9246] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:24.501431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:24.501586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342948463083578:9246] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:24.501631Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:24.501641Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:24.501649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:24.501700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:24.501726Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342948101247603:11459]
2025-10-09T21:51:24.502120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:24.502338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:24.502373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:24.502579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:24.502612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.502616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.502617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.502617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.502618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.502619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.502620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.502620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.502621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.502624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.502630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.502632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.502634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.502636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.502638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.502640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.502641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.502643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.502647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.502652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.502656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.502659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.502662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.502665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.502667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.502670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.502672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.502675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.502677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.502680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.502683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.502686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.502688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.502692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.502695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.502698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.502715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:24.503381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342948463083578:9246] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:24.508022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342948463083578:9246] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:24.508036Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342948463083578:9246] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:24.508039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342948463083578:9246] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:24.508047Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2130 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:24.508280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:24.509385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:24.509397Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:24.509399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:24.509402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:24.509407Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2129 [2:7559342948101247603:11459]
2025-10-09T21:51:24.509418Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:24.509434Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:24.509436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:24.509439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:24.509441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:24.509455Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:24.509462Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:24.509467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:24.509501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.509506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.509507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.509509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:24.509552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:24.509558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.509565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.509571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.509579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:24.509594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463083247:2432], now have 1 active actors on pipe
2025-10-09T21:51:24.509600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463083616:9246], now have 1 active actors on pipe
2025-10-09T21:51:24.509604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342948101247598:11459]
2025-10-09T21:51:24.509609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342948101247598:11459]
2025-10-09T21:51:24.509610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342948101247598:11459]
2025-10-09T21:51:24.509619Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:24.509628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:24.509629Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:24.509631Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:24.509633Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:24.509639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:24.509640Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:24.509656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342948101247598:11459]
2025-10-09T21:51:24.509664Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:24.509667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:24.509691Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2129
2025-10-09T21:51:24.509724Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2129
2025-10-09T21:51:24.509758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:24.509764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:24.509788Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1b63a568-81f04fde-7ad40be7-d8e07b8b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:24.509799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:24.509806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:24.509807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:24.509814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:24.509823Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:24.509837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:24.509980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:24.509985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:24.509994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:24.510076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2130 partition: 0 MaxSeqNo: 25 sessionId: src_id|1b63a568-81f04fde-7ad40be7-d8e07b8b_0
2025-10-09T21:51:24.510364Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebb9f106-e6cd094e-dd8dfc30-e9f3e09a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1b63a568-81f04fde-7ad40be7-d8e07b8b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:24.512513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b63a568-81f04fde-7ad40be7-d8e07b8b_0] Write session: try to update token
2025-10-09T21:51:24.512524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b63a568-81f04fde-7ad40be7-d8e07b8b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:24.522591Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:24.522601Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:24.522603Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:24.522604Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:24.522610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342948101247598:11459]
2025-10-09T21:51:24.522612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342948101247598:11459]
2025-10-09T21:51:24.522615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342948101247598:11459]
2025-10-09T21:51:24.522646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:24.522650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.522649Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:24.522651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:24.522653Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:24.522655Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:24.522770Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:24.523952Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2130 sessionId: src_id|1b63a568-81f04fde-7ad40be7-d8e07b8b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:24.525986Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:24.589052Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342948463083633:9464]
2025-10-09T21:51:24.589309Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 2129, pipe: [1:7559342948463083633:9464]
2025-10-09T21:51:24.653712Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:24.653966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:24.653977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:24.654015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463083633:9464], now have 1 active actors on pipe
2025-10-09T21:51:24.654181Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:24.654243Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:24.654249Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342948463083633:9464]
2025-10-09T21:51:24.654271Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:24.654346Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2130 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1b63a568-81f04fde-7ad40be7-d8e07b8b_0
2025-10-09T21:51:24.654526Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2130 sessionId: src_id|1b63a568-81f04fde-7ad40be7-d8e07b8b_0 is DEAD
2025-10-09T21:51:24.654707Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b63a568-81f04fde-7ad40be7-d8e07b8b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:24.654720Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b63a568-81f04fde-7ad40be7-d8e07b8b_0] Write session will restart in 0.008718s
2025-10-09T21:51:24.654744Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b63a568-81f04fde-7ad40be7-d8e07b8b_0] Write session: Do CDS request
2025-10-09T21:51:24.654750Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b63a568-81f04fde-7ad40be7-d8e07b8b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:24.664372Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b63a568-81f04fde-7ad40be7-d8e07b8b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:24.664459Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:24.664472Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2131
2025-10-09T21:51:24.664567Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2131 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:24.664591Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2131 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:24.664598Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2131 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:24.664602Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:24.664939Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2131 sessionId:  describe result for acl check
2025-10-09T21:51:24.664970Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:24.664977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:24.664978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:24.664983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342948463083954:9236] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:24.665212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:24.665545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342948463083954:9236] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:24.665946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:24.666111Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:24.666125Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:24.666133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:24.666171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:24.666198Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342948101247837:11467]
2025-10-09T21:51:24.666562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:24.666748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:24.666774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:24.666971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:24.667003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.667007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.667007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.667008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.667009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.667009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.667010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.667010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.667011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.667014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.667019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.667022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.667024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.667025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.667027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.667029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.667031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.667033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.667037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.667042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.667046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.667049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.667052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.667055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.667058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.667061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.667064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.667067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.667070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.667075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.667080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.667085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.667089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.667094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.667099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.667102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.667122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:24.667821Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342948463083954:9236] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:24.672398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342948463083954:9236] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:24.672410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342948463083954:9236] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:24.672414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342948463083954:9236] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:24.672420Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2131 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:24.672701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:24.673760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:24.673771Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:24.673772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:24.673774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:24.673779Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2130 [2:7559342948101247837:11467]
2025-10-09T21:51:24.673786Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:24.673798Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:24.673799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:24.673801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:24.673803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:24.673814Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:24.673821Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:24.673825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:24.673860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.673865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.673866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.673867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:24.673911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:24.673916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.673922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.673931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.673940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:24.673957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463083995:9236], now have 1 active actors on pipe
2025-10-09T21:51:24.673962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463083634:2432], now have 1 active actors on pipe
2025-10-09T21:51:24.673967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342948101247831:11467]
2025-10-09T21:51:24.673968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342948101247831:11467]
2025-10-09T21:51:24.673970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342948101247831:11467]
2025-10-09T21:51:24.673979Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:24.673989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:24.674000Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:24.674001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:24.674004Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:24.674007Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:24.674009Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:24.674021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342948101247831:11467]
2025-10-09T21:51:24.674023Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:24.674021Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2130
2025-10-09T21:51:24.674023Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2130
2025-10-09T21:51:24.674024Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:24.674080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:24.674087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:24.674108Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e414b220-89731c07-c8c57244-9c62c88f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:24.674118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:24.674120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:24.674122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:24.674128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:24.674135Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:24.674145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:24.674274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:24.674279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:24.674288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:24.674375Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2131 partition: 0 MaxSeqNo: 25 sessionId: src_id|e414b220-89731c07-c8c57244-9c62c88f_0
2025-10-09T21:51:24.674617Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b63a568-81f04fde-7ad40be7-d8e07b8b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e414b220-89731c07-c8c57244-9c62c88f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:24.677013Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e414b220-89731c07-c8c57244-9c62c88f_0] Write session: try to update token
2025-10-09T21:51:24.677028Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e414b220-89731c07-c8c57244-9c62c88f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:24.686994Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:24.687004Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:24.687006Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:24.687007Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:24.687014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342948101247831:11467]
2025-10-09T21:51:24.687018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342948101247831:11467]
2025-10-09T21:51:24.687020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342948101247831:11467]
2025-10-09T21:51:24.687051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:24.687056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.687055Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:24.687058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:24.687059Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:24.687062Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:24.687177Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:24.689638Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2131 sessionId: src_id|e414b220-89731c07-c8c57244-9c62c88f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:24.691825Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:24.754389Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342948463084003:9464]
2025-10-09T21:51:24.754593Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 2130, pipe: [1:7559342948463084003:9464]
2025-10-09T21:51:24.818608Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:24.818940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:24.818952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:24.818989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463084003:9464], now have 1 active actors on pipe
2025-10-09T21:51:24.819124Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:24.819141Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:24.819190Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:24.819267Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2131 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e414b220-89731c07-c8c57244-9c62c88f_0
2025-10-09T21:51:24.819431Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342948463084003:9464]
2025-10-09T21:51:24.819449Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2131 sessionId: src_id|e414b220-89731c07-c8c57244-9c62c88f_0 is DEAD
2025-10-09T21:51:24.819600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e414b220-89731c07-c8c57244-9c62c88f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:24.819613Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e414b220-89731c07-c8c57244-9c62c88f_0] Write session will restart in 0.008908s
2025-10-09T21:51:24.819640Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e414b220-89731c07-c8c57244-9c62c88f_0] Write session: Do CDS request
2025-10-09T21:51:24.819647Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e414b220-89731c07-c8c57244-9c62c88f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:24.829487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e414b220-89731c07-c8c57244-9c62c88f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:24.829554Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:24.829565Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2132
2025-10-09T21:51:24.829700Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2132 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:24.829729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2132 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:24.829738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2132 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:24.829743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:24.830110Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2132 sessionId:  describe result for acl check
2025-10-09T21:51:24.830138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:24.830144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:24.830146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:24.830153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342948463084326:9230] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:24.830669Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342948463084326:9230] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:24.830787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:24.831656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:24.831803Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:24.831811Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:24.831822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:24.831869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:24.831902Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342948101248054:11471]
2025-10-09T21:51:24.832177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342948463084326:9230] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:24.832316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:24.832524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:24.832551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:24.832762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:24.832795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.832799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.832800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.832801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.832801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.832802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.832802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.832803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.832803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.832806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.832812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.832814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.832816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.832818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.832820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.832822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.832824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.832826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.832830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.832834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:24.832838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.832841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:24.832844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.832847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:24.832849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.832851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:24.832854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.832857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:24.832859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.832862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:24.832864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.832867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.832869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.832872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.832875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.832881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.832897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:24.837123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342948463084326:9230] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:24.837134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342948463084326:9230] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:24.837136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342948463084326:9230] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:24.837141Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2132 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:24.838410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:24.839397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:24.839407Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:24.839408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:24.839410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:24.839415Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2131 [2:7559342948101248054:11471]
2025-10-09T21:51:24.839423Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:24.839435Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:24.839440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:24.839441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:24.839443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:24.839454Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:24.839461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:24.839466Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:24.839504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.839509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.839510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.839512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:24.839555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:24.839560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.839567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:24.839572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:24.839589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:24.839611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463084369:9230], now have 1 active actors on pipe
2025-10-09T21:51:24.839620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463084007:2432], now have 1 active actors on pipe
2025-10-09T21:51:24.839627Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342948101248048:11471]
2025-10-09T21:51:24.839629Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342948101248048:11471]
2025-10-09T21:51:24.839630Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342948101248048:11471]
2025-10-09T21:51:24.839644Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:24.839660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:24.839659Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:24.839647Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2131
2025-10-09T21:51:24.839662Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:24.839648Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2131
2025-10-09T21:51:24.839663Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:24.839672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:24.839675Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:24.839703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:24.839709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:24.839719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342948101248048:11471]
2025-10-09T21:51:24.839727Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:24.839732Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:24.839748Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7e7f2bb7-91b49814-7368a148-c56e966_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:24.839757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:24.839759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:24.839761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:24.839767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:24.839777Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:24.839796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:24.840039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:24.840047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:24.840057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:24.840149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2132 partition: 0 MaxSeqNo: 25 sessionId: src_id|7e7f2bb7-91b49814-7368a148-c56e966_0
2025-10-09T21:51:24.840427Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e414b220-89731c07-c8c57244-9c62c88f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7e7f2bb7-91b49814-7368a148-c56e966_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:24.842872Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e7f2bb7-91b49814-7368a148-c56e966_0] Write session: try to update token
2025-10-09T21:51:24.842888Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e7f2bb7-91b49814-7368a148-c56e966_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:24.852138Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:24.852149Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:24.852151Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:24.852152Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:24.852159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342948101248048:11471]
2025-10-09T21:51:24.852163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342948101248048:11471]
2025-10-09T21:51:24.852165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342948101248048:11471]
2025-10-09T21:51:24.852196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:24.852202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:24.852202Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:24.852204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:24.852206Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:24.852209Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:24.852331Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:24.855870Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2132 sessionId: src_id|7e7f2bb7-91b49814-7368a148-c56e966_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:24.858133Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:24.919805Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342948463084379:9464]
2025-10-09T21:51:24.920081Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 2131, pipe: [1:7559342948463084379:9464]
2025-10-09T21:51:24.990019Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:24.990299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:24.990313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:24.990355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463084379:9464], now have 1 active actors on pipe
2025-10-09T21:51:24.990559Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:24.990546Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:24.990551Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:24.990556Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:24.990601Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342948463084379:9464]
2025-10-09T21:51:24.990637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2132 reason: error at write: Disconnected sessionId: src_id|7e7f2bb7-91b49814-7368a148-c56e966_0
2025-10-09T21:51:24.990852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2132 sessionId: src_id|7e7f2bb7-91b49814-7368a148-c56e966_0 is DEAD
2025-10-09T21:51:24.990964Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e7f2bb7-91b49814-7368a148-c56e966_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:24.990974Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e7f2bb7-91b49814-7368a148-c56e966_0] Write session will restart in 0.006351s
2025-10-09T21:51:24.990993Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e7f2bb7-91b49814-7368a148-c56e966_0] Write session: Do CDS request
2025-10-09T21:51:24.990998Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e7f2bb7-91b49814-7368a148-c56e966_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:24.998454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7e7f2bb7-91b49814-7368a148-c56e966_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:24.998529Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:24.998545Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2133
2025-10-09T21:51:24.998672Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2133 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:24.998707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2133 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:24.998718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2133 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:24.998721Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:24.999102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2133 sessionId:  describe result for acl check
2025-10-09T21:51:24.999142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:24.999148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:24.999150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:24.999155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342948463084631:9225] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:24.999737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342948463084631:9225] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:25.001509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342948463084631:9225] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:25.001948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:25.002700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:25.002859Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:25.002866Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:25.002874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:25.002908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:25.002935Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342952396215561:11473]
2025-10-09T21:51:25.003287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:25.003489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:25.003530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:25.003743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:25.003739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342948463084631:9225] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:25.003750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342948463084631:9225] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:25.003752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342948463084631:9225] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:25.003758Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2133 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:25.003789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.003794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.003795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.003796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.003798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.003799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.003800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.003801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.003802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.003806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.003816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.003820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.003824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.003828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.003832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.003836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.003840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.003843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.003849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.003854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.003861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.003864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.003867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.003872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.003874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.003877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.003883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.003886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.003889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.003892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.003895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.003901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.003904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.003907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.003910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.003913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.003933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:25.009818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:25.011059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:25.011071Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:25.011073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:25.011076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:25.011083Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2132 [2:7559342952396215561:11473]
2025-10-09T21:51:25.011092Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:25.011106Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:25.011109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:25.011111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:25.011113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:25.011125Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:25.011129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:25.011135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:25.011180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.011185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.011186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.011188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:25.011235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:25.011242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.011249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.011256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.011265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:25.011280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758052038:9225], now have 1 active actors on pipe
2025-10-09T21:51:25.011282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342948463084380:2432], now have 1 active actors on pipe
2025-10-09T21:51:25.011288Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342952396215555:11473]
2025-10-09T21:51:25.011290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342952396215555:11473]
2025-10-09T21:51:25.011291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342952396215555:11473]
2025-10-09T21:51:25.011302Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:25.011315Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:25.011322Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:25.011328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:25.011328Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:25.011329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:25.011330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:25.011359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342952396215555:11473]
2025-10-09T21:51:25.011372Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:25.011383Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:25.011378Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2132
2025-10-09T21:51:25.011384Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2132
2025-10-09T21:51:25.011417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.011424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.011448Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a89f816e-55e04a25-e771d3c3-78cf3b76_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:25.011457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:25.011460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:25.011461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:25.011468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:25.011482Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:25.011499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:25.011602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.011609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.011622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:25.011706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2133 partition: 0 MaxSeqNo: 25 sessionId: src_id|a89f816e-55e04a25-e771d3c3-78cf3b76_0
2025-10-09T21:51:25.012050Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7e7f2bb7-91b49814-7368a148-c56e966_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a89f816e-55e04a25-e771d3c3-78cf3b76_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:25.014653Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a89f816e-55e04a25-e771d3c3-78cf3b76_0] Write session: try to update token
2025-10-09T21:51:25.014668Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a89f816e-55e04a25-e771d3c3-78cf3b76_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:25.025426Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:25.025439Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:25.025441Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:25.025442Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:25.025451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342952396215555:11473]
2025-10-09T21:51:25.025455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342952396215555:11473]
2025-10-09T21:51:25.025457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342952396215555:11473]
2025-10-09T21:51:25.025491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:25.025496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.025496Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:25.025498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:25.025501Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:25.025506Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:25.025617Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:25.027577Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2133 sessionId: src_id|a89f816e-55e04a25-e771d3c3-78cf3b76_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:25.029790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:25.090821Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342952758052047:9464]
2025-10-09T21:51:25.091084Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 2132, pipe: [1:7559342952758052047:9464]
2025-10-09T21:51:25.168586Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:25.168800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.168812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.168849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758052047:9464], now have 1 active actors on pipe
2025-10-09T21:51:25.169062Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:25.169064Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342952758052047:9464]
2025-10-09T21:51:25.169079Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:25.169092Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:25.169150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2133 reason: error at write: Disconnected sessionId: src_id|a89f816e-55e04a25-e771d3c3-78cf3b76_0
2025-10-09T21:51:25.169160Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:25.169346Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2133 sessionId: src_id|a89f816e-55e04a25-e771d3c3-78cf3b76_0 is DEAD
2025-10-09T21:51:25.169521Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a89f816e-55e04a25-e771d3c3-78cf3b76_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:25.169532Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a89f816e-55e04a25-e771d3c3-78cf3b76_0] Write session will restart in 0.005296s
2025-10-09T21:51:25.169567Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a89f816e-55e04a25-e771d3c3-78cf3b76_0] Write session: Do CDS request
2025-10-09T21:51:25.169574Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a89f816e-55e04a25-e771d3c3-78cf3b76_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:25.175504Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a89f816e-55e04a25-e771d3c3-78cf3b76_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:25.175567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:25.175582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2134
2025-10-09T21:51:25.175743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2134 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:25.175777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2134 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:25.175787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2134 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:25.175790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:25.176176Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2134 sessionId:  describe result for acl check
2025-10-09T21:51:25.176206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:25.176212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:25.176213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:25.176218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342952758052132:9217] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:25.176730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342952758052132:9217] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:25.178784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342952758052132:9217] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:25.180941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342952758052132:9217] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:25.180955Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342952758052132:9217] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:25.180959Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342952758052132:9217] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:25.180966Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2134 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:25.181652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:25.182623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:25.182796Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:25.182806Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:25.182818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:25.182865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:25.182902Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342952396215776:11475]
2025-10-09T21:51:25.183482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:25.183703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:25.183736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:25.184042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:25.184088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.184093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.184095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.184096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.184097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.184098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.184099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.184100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.184102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.184106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.184116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.184120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.184124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.184132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.184136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.184140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.184144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.184148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.184154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.184166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.184176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.184185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.184189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.184195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.184203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.184209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.184214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.184219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.184223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.184232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.184240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.184249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.184255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.184261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.184270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.184279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.184311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:25.190585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:25.191783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:25.191794Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:25.191796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:25.191799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:25.191804Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2133 [2:7559342952396215776:11475]
2025-10-09T21:51:25.191815Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:25.191829Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:25.191834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:25.191837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:25.191839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:25.191851Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:25.191858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:25.191864Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:25.191903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.191908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.191909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.191911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:25.191968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:25.191976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.191987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.191996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.192005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:25.192021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758052051:2432], now have 1 active actors on pipe
2025-10-09T21:51:25.192023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758052409:9217], now have 1 active actors on pipe
2025-10-09T21:51:25.192029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342952396215771:11475]
2025-10-09T21:51:25.192030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342952396215771:11475]
2025-10-09T21:51:25.192032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342952396215771:11475]
2025-10-09T21:51:25.192042Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:25.192054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:25.192066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:25.192066Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:25.192068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:25.192069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:25.192071Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:25.192095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342952396215771:11475]
2025-10-09T21:51:25.192102Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:25.192110Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:25.192137Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2133
2025-10-09T21:51:25.192136Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2133
2025-10-09T21:51:25.192211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.192225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.192265Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|826791a6-d46bed9d-b2edad34-258c3dd0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:25.192279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:25.192288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:25.192291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:25.192302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:25.192317Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:25.192336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:25.192474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.192481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.192494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:25.192594Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2134 partition: 0 MaxSeqNo: 25 sessionId: src_id|826791a6-d46bed9d-b2edad34-258c3dd0_0
2025-10-09T21:51:25.192940Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a89f816e-55e04a25-e771d3c3-78cf3b76_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|826791a6-d46bed9d-b2edad34-258c3dd0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:25.195353Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|826791a6-d46bed9d-b2edad34-258c3dd0_0] Write session: try to update token
2025-10-09T21:51:25.195368Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|826791a6-d46bed9d-b2edad34-258c3dd0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:25.206495Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:25.206514Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:25.206517Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:25.206519Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:25.206529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342952396215771:11475]
2025-10-09T21:51:25.206535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342952396215771:11475]
2025-10-09T21:51:25.206539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342952396215771:11475]
2025-10-09T21:51:25.206581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:25.206584Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:25.206592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.206594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:25.206599Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:25.206601Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:25.206724Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:25.208667Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2134 sessionId: src_id|826791a6-d46bed9d-b2edad34-258c3dd0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:25.211204Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:25.269004Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342952758052425:9464]
2025-10-09T21:51:25.269286Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 2133, pipe: [1:7559342952758052425:9464]
2025-10-09T21:51:25.349414Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:25.349779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.349791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.349827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758052425:9464], now have 1 active actors on pipe
2025-10-09T21:51:25.349988Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:25.350022Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:25.350044Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:25.350047Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:25.350088Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342952758052425:9464]
2025-10-09T21:51:25.350107Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2134 reason: error at write: Disconnected sessionId: src_id|826791a6-d46bed9d-b2edad34-258c3dd0_0
2025-10-09T21:51:25.350273Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2134 sessionId: src_id|826791a6-d46bed9d-b2edad34-258c3dd0_0 is DEAD
2025-10-09T21:51:25.350411Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|826791a6-d46bed9d-b2edad34-258c3dd0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:25.350421Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|826791a6-d46bed9d-b2edad34-258c3dd0_0] Write session will restart in 0.009860s
2025-10-09T21:51:25.350451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|826791a6-d46bed9d-b2edad34-258c3dd0_0] Write session: Do CDS request
2025-10-09T21:51:25.350456Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|826791a6-d46bed9d-b2edad34-258c3dd0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:25.361471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|826791a6-d46bed9d-b2edad34-258c3dd0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:25.361475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:25.361574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:25.361590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2135
2025-10-09T21:51:25.361708Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2135 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:25.361754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2135 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:25.361765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2135 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:25.361773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:25.362165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2135 sessionId:  describe result for acl check
2025-10-09T21:51:25.362201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:25.362208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:25.362210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:25.362215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342952758052759:9215] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:25.362511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:25.362634Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:25.362641Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:25.362649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:25.362683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:25.362710Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342952396215988:11477]
2025-10-09T21:51:25.362766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342952758052759:9215] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:25.363047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:25.363287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:25.363338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:25.363573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:25.363608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.363612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.363613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.363614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.363614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.363615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.363615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.363616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.363617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.363620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.363626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.363628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.363630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.363631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.363634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.363636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.363637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.363639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.363643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.363648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.363653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.363656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.363658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.363661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.363663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.363666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.363668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.363671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.363673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.363676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.363679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.363682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.363684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.363687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.363689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.363693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.363708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:25.364245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342952758052759:9215] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:25.368764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342952758052759:9215] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:25.368775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342952758052759:9215] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:25.368777Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342952758052759:9215] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:25.368783Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2135 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:25.369159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:25.370291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:25.370301Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:25.370302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:25.370305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:25.370310Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2134 [2:7559342952396215988:11477]
2025-10-09T21:51:25.370317Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:25.370330Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:25.370335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:25.370337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:25.370338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:25.370350Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:25.370357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:25.370362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:25.370416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.370422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.370423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.370425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:25.370469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:25.370474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.370480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.370486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.370494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:25.370511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758052433:2432], now have 1 active actors on pipe
2025-10-09T21:51:25.370517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758052797:9215], now have 1 active actors on pipe
2025-10-09T21:51:25.370521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342952396215982:11477]
2025-10-09T21:51:25.370526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342952396215982:11477]
2025-10-09T21:51:25.370527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342952396215982:11477]
2025-10-09T21:51:25.370536Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:25.370545Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:25.370549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:25.370551Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:25.370552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:25.370557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:25.370559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:25.370545Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2134
2025-10-09T21:51:25.370568Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2134
2025-10-09T21:51:25.370579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342952396215982:11477]
2025-10-09T21:51:25.370585Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:25.370586Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:25.370603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.370607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.370633Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7329fb08-9ace919a-852bb4fb-9a2457e8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:25.370643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:25.370651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:25.370653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:25.370661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:25.370671Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:25.370683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:25.370812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.370817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.370825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:25.370904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2135 partition: 0 MaxSeqNo: 25 sessionId: src_id|7329fb08-9ace919a-852bb4fb-9a2457e8_0
2025-10-09T21:51:25.371150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|826791a6-d46bed9d-b2edad34-258c3dd0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7329fb08-9ace919a-852bb4fb-9a2457e8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:25.373478Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7329fb08-9ace919a-852bb4fb-9a2457e8_0] Write session: try to update token
2025-10-09T21:51:25.373491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7329fb08-9ace919a-852bb4fb-9a2457e8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:25.383952Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:25.383969Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:25.383972Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:25.383974Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:25.383982Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342952396215982:11477]
2025-10-09T21:51:25.383985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342952396215982:11477]
2025-10-09T21:51:25.383988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342952396215982:11477]
2025-10-09T21:51:25.384022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:25.384028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.384030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:25.384065Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:25.384075Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:25.384077Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:25.384152Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:25.385442Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2135 sessionId: src_id|7329fb08-9ace919a-852bb4fb-9a2457e8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:25.387686Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:25.450214Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342952758052804:9464]
2025-10-09T21:51:25.450444Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 2134, pipe: [1:7559342952758052804:9464]
2025-10-09T21:51:25.539191Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:25.539600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.539613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.539650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758052804:9464], now have 1 active actors on pipe
2025-10-09T21:51:25.539808Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:25.539891Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342952758052804:9464]
2025-10-09T21:51:25.539922Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:25.539996Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2135 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7329fb08-9ace919a-852bb4fb-9a2457e8_0
2025-10-09T21:51:25.540173Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2135 sessionId: src_id|7329fb08-9ace919a-852bb4fb-9a2457e8_0 is DEAD
2025-10-09T21:51:25.540302Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7329fb08-9ace919a-852bb4fb-9a2457e8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:25.540322Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7329fb08-9ace919a-852bb4fb-9a2457e8_0] Write session will restart in 0.005746s
2025-10-09T21:51:25.540343Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7329fb08-9ace919a-852bb4fb-9a2457e8_0] Write session: Do CDS request
2025-10-09T21:51:25.540348Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7329fb08-9ace919a-852bb4fb-9a2457e8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:25.540333Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:25.547478Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7329fb08-9ace919a-852bb4fb-9a2457e8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:25.547568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:25.547581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2136
2025-10-09T21:51:25.547683Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2136 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:25.547708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2136 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:25.547711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2136 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:25.547714Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:25.548006Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2136 sessionId:  describe result for acl check
2025-10-09T21:51:25.548036Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:25.548037Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:25.548039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:25.548044Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342952758053018:9201] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:25.548447Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342952758053018:9201] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:25.550533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342952758053018:9201] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:25.551711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:25.552667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:25.552836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342952758053018:9201] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:25.552847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342952758053018:9201] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:25.552849Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342952758053018:9201] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:25.552855Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2136 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:25.552897Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:25.552908Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:25.552920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:25.552974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:25.553011Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342952396216242:11485]
2025-10-09T21:51:25.553601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:25.553831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:25.553866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:25.554182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:25.554225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.554231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.554232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.554233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.554234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.554236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.554237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.554238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.554239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.554244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.554253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.554258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.554262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.554265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.554270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.554274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.554277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.554281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.554287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.554295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.554301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.554306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.554311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.554316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.554320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.554325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.554329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.554334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.554339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.554344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.554349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.554354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.554364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.554369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.554373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.554378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.554406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:25.560675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:25.561936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:25.561949Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:25.561952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:25.561957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:25.561963Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2135 [2:7559342952396216242:11485]
2025-10-09T21:51:25.561976Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:25.561997Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:25.562004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:25.562007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:25.562009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:25.562043Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:25.562053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:25.562061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:25.562118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.562124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.562126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.562128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:25.562195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:25.562201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.562212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.562228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.562248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:25.562276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758053187:9201], now have 1 active actors on pipe
2025-10-09T21:51:25.562280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758052820:2432], now have 1 active actors on pipe
2025-10-09T21:51:25.562287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342952396216237:11485]
2025-10-09T21:51:25.562289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342952396216237:11485]
2025-10-09T21:51:25.562291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342952396216237:11485]
2025-10-09T21:51:25.562304Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:25.562319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:25.562307Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2135
2025-10-09T21:51:25.562330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:25.562313Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2135
2025-10-09T21:51:25.562337Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:25.562340Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:25.562340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:25.562342Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:25.562377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.562386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.562398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342952396216237:11485]
2025-10-09T21:51:25.562407Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:25.562412Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:25.562438Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fd4d16d5-25797aa7-83c71e2-dad3c31_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:25.562449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:25.562453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:25.562456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:25.562466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:25.562479Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:25.562502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:25.562627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.562634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.562647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:25.562702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2136 partition: 0 MaxSeqNo: 25 sessionId: src_id|fd4d16d5-25797aa7-83c71e2-dad3c31_0
2025-10-09T21:51:25.563004Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7329fb08-9ace919a-852bb4fb-9a2457e8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fd4d16d5-25797aa7-83c71e2-dad3c31_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:25.565293Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fd4d16d5-25797aa7-83c71e2-dad3c31_0] Write session: try to update token
2025-10-09T21:51:25.565308Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fd4d16d5-25797aa7-83c71e2-dad3c31_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:25.575548Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:25.575563Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:25.575565Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:25.575567Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:25.575577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342952396216237:11485]
2025-10-09T21:51:25.575584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342952396216237:11485]
2025-10-09T21:51:25.575588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342952396216237:11485]
2025-10-09T21:51:25.575630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:25.575642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.575645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:25.575646Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:25.575651Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:25.575655Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:25.575779Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:25.577975Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2136 sessionId: src_id|fd4d16d5-25797aa7-83c71e2-dad3c31_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:25.580081Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:25.640188Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342952758053198:9464]
2025-10-09T21:51:25.640445Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 2135, pipe: [1:7559342952758053198:9464]
2025-10-09T21:51:25.711081Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:25.711395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.711410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.711462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758053198:9464], now have 1 active actors on pipe
2025-10-09T21:51:25.711661Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:25.711674Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:25.711727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2136 reason: error at write: Disconnected sessionId: src_id|fd4d16d5-25797aa7-83c71e2-dad3c31_0
2025-10-09T21:51:25.711909Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:25.711916Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2136 sessionId: src_id|fd4d16d5-25797aa7-83c71e2-dad3c31_0 is DEAD
2025-10-09T21:51:25.711987Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342952758053198:9464]
2025-10-09T21:51:25.712018Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:25.712065Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd4d16d5-25797aa7-83c71e2-dad3c31_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:25.712074Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd4d16d5-25797aa7-83c71e2-dad3c31_0] Write session will restart in 0.008316s
2025-10-09T21:51:25.712101Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd4d16d5-25797aa7-83c71e2-dad3c31_0] Write session: Do CDS request
2025-10-09T21:51:25.712108Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd4d16d5-25797aa7-83c71e2-dad3c31_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:25.721560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fd4d16d5-25797aa7-83c71e2-dad3c31_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:25.721589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:25.721601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2137
2025-10-09T21:51:25.721776Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2137 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:25.721831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2137 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:25.721840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2137 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:25.721844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:25.722232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2137 sessionId:  describe result for acl check
2025-10-09T21:51:25.722260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:25.722263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:25.722264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:25.722269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342952758053498:9196] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:25.722694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342952758053498:9196] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:25.724143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:25.724296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342952758053498:9196] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:25.724994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:25.725150Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:25.725159Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:25.725169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:25.725217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:25.725254Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342952396216457:11493]
2025-10-09T21:51:25.725815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:25.726050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:25.726086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:25.726383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:25.726428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.726434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.726435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.726436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.726437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.726438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.726440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.726441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.726442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.726447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.726457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.726460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.726464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.726468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.726472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.726476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.726480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.726484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.726495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.726503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.726513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.726522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.726530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.726535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.726544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.726553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.726557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.726562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.726565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.726570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.726578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.726582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.726590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.726594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.726603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.726608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.726633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:25.726650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342952758053498:9196] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:25.726663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342952758053498:9196] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:25.726667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342952758053498:9196] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:25.726675Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2137 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:25.732460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:25.733764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:25.733777Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:25.733779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:25.733782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:25.733788Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2136 [2:7559342952396216457:11493]
2025-10-09T21:51:25.733801Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:25.733816Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:25.733819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:25.733822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:25.733824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:25.733840Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:25.733849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:25.733860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:25.733911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.733917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.733919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.733921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:25.733982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:25.733989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.734000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.734011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.734042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:25.734062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758053200:2432], now have 1 active actors on pipe
2025-10-09T21:51:25.734065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758053563:9196], now have 1 active actors on pipe
2025-10-09T21:51:25.734072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342952396216452:11493]
2025-10-09T21:51:25.734074Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342952396216452:11493]
2025-10-09T21:51:25.734076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342952396216452:11493]
2025-10-09T21:51:25.734088Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:25.734097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:25.734099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:25.734101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:25.734107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:25.734120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:25.734126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:25.734105Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2136
2025-10-09T21:51:25.734114Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2136
2025-10-09T21:51:25.734160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.734168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.734185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342952396216452:11493]
2025-10-09T21:51:25.734196Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:25.734200Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:25.734222Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|37422451-ef6e4e23-3dc2835f-7ccde19_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:25.734233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:25.734242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:25.734248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:25.734257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:25.734266Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:25.734290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:25.734423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.734431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.734444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:25.734530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2137 partition: 0 MaxSeqNo: 25 sessionId: src_id|37422451-ef6e4e23-3dc2835f-7ccde19_0
2025-10-09T21:51:25.734836Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd4d16d5-25797aa7-83c71e2-dad3c31_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|37422451-ef6e4e23-3dc2835f-7ccde19_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:25.737203Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|37422451-ef6e4e23-3dc2835f-7ccde19_0] Write session: try to update token
2025-10-09T21:51:25.737220Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|37422451-ef6e4e23-3dc2835f-7ccde19_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:25.747606Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:25.747619Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:25.747621Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:25.747622Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:25.747630Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342952396216452:11493]
2025-10-09T21:51:25.747633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342952396216452:11493]
2025-10-09T21:51:25.747635Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342952396216452:11493]
2025-10-09T21:51:25.747669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:25.747674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.747676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:25.747679Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:25.747684Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:25.747687Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:25.747797Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:25.750438Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2137 sessionId: src_id|37422451-ef6e4e23-3dc2835f-7ccde19_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:25.752696Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:25.812208Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342952758053573:9464]
2025-10-09T21:51:25.812455Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 2136, pipe: [1:7559342952758053573:9464]
2025-10-09T21:51:25.893490Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:25.893828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.893842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.893883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758053573:9464], now have 1 active actors on pipe
2025-10-09T21:51:25.894083Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:25.894144Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:25.894177Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342952758053573:9464]
2025-10-09T21:51:25.894202Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:25.894381Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2137 reason: status is not ok: tablet will be restarted right now sessionId: src_id|37422451-ef6e4e23-3dc2835f-7ccde19_0
2025-10-09T21:51:25.894567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2137 sessionId: src_id|37422451-ef6e4e23-3dc2835f-7ccde19_0 is DEAD
2025-10-09T21:51:25.894725Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37422451-ef6e4e23-3dc2835f-7ccde19_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:25.894736Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37422451-ef6e4e23-3dc2835f-7ccde19_0] Write session will restart in 0.009122s
2025-10-09T21:51:25.894756Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37422451-ef6e4e23-3dc2835f-7ccde19_0] Write session: Do CDS request
2025-10-09T21:51:25.894761Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37422451-ef6e4e23-3dc2835f-7ccde19_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:25.904550Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|37422451-ef6e4e23-3dc2835f-7ccde19_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:25.904609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:25.904626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2138
2025-10-09T21:51:25.904743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2138 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:25.904776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2138 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:25.904781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2138 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:25.904785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:25.905154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2138 sessionId:  describe result for acl check
2025-10-09T21:51:25.905187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:25.905192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:25.905194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:25.905198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342952758053903:9185] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:25.905660Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342952758053903:9185] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:25.906391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:25.907448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:25.907661Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:25.907671Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:25.907683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:25.907740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:25.907779Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342952396216679:11497]
2025-10-09T21:51:25.907838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342952758053903:9185] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:25.908338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:25.908577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:25.908606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:25.908828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:25.908864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.908868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.908869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.908870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.908870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.908871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.908872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.908873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.908873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.908876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.908883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.908885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.908887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.908889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.908891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.908893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.908895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.908897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.908901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.908907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:25.908911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.908914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:25.908918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.908921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:25.908924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.908927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:25.908931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.908934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:25.908936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.908939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:25.908942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.908945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.908948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.908951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.908953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.908956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.908974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:25.913723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342952758053903:9185] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:25.913736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342952758053903:9185] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:25.913738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342952758053903:9185] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:25.913746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2138 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:25.915056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:25.916208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:25.916220Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:25.916221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:25.916225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:25.916230Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2137 [2:7559342952396216679:11497]
2025-10-09T21:51:25.916244Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:25.916261Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:25.916267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:25.916269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:25.916270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:25.916284Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:25.916291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:25.916297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:25.916342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.916347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.916348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.916350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:25.916403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:25.916408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.916415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:25.916421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:25.916431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:25.916453Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342952396216674:11497]
2025-10-09T21:51:25.916461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342952396216674:11497]
2025-10-09T21:51:25.916463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342952396216674:11497]
2025-10-09T21:51:25.916476Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:25.916493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758053578:2432], now have 1 active actors on pipe
2025-10-09T21:51:25.916499Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:25.916501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758053945:9185], now have 1 active actors on pipe
2025-10-09T21:51:25.916502Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:25.916505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:25.916511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:25.916524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:25.916531Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:25.916567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342952396216674:11497]
2025-10-09T21:51:25.916575Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:25.916543Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2137
2025-10-09T21:51:25.916578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:25.916544Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2137
2025-10-09T21:51:25.916598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.916606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.916643Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ddd22bdf-3221085a-379214f7-22b5c496_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:25.916657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:25.916665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:25.916668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:25.916678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:25.916694Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:25.916717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:25.916860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:25.916867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:25.916879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:25.916952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2138 partition: 0 MaxSeqNo: 25 sessionId: src_id|ddd22bdf-3221085a-379214f7-22b5c496_0
2025-10-09T21:51:25.917282Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|37422451-ef6e4e23-3dc2835f-7ccde19_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ddd22bdf-3221085a-379214f7-22b5c496_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:25.919666Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ddd22bdf-3221085a-379214f7-22b5c496_0] Write session: try to update token
2025-10-09T21:51:25.919684Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ddd22bdf-3221085a-379214f7-22b5c496_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:25.930626Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:25.930642Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:25.930644Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:25.930646Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:25.930658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342952396216674:11497]
2025-10-09T21:51:25.930663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342952396216674:11497]
2025-10-09T21:51:25.930667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342952396216674:11497]
2025-10-09T21:51:25.930711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:25.930720Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:25.930723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:25.930726Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:25.930726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:25.930730Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:25.930859Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:25.932314Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2138 sessionId: src_id|ddd22bdf-3221085a-379214f7-22b5c496_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:25.934558Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:25.994499Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342952758053952:9464]
2025-10-09T21:51:25.994770Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 2137, pipe: [1:7559342952758053952:9464]
2025-10-09T21:51:26.075595Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:26.075808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:26.075822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:26.075866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342952758053952:9464], now have 1 active actors on pipe
2025-10-09T21:51:26.076026Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:26.076051Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:26.076063Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342952758053952:9464]
2025-10-09T21:51:26.076087Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:26.076089Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:26.076150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2138 reason: error at write: Disconnected sessionId: src_id|ddd22bdf-3221085a-379214f7-22b5c496_0
2025-10-09T21:51:26.076342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2138 sessionId: src_id|ddd22bdf-3221085a-379214f7-22b5c496_0 is DEAD
2025-10-09T21:51:26.076487Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddd22bdf-3221085a-379214f7-22b5c496_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:26.076496Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddd22bdf-3221085a-379214f7-22b5c496_0] Write session will restart in 0.008442s
2025-10-09T21:51:26.076520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddd22bdf-3221085a-379214f7-22b5c496_0] Write session: Do CDS request
2025-10-09T21:51:26.076525Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddd22bdf-3221085a-379214f7-22b5c496_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:26.086553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ddd22bdf-3221085a-379214f7-22b5c496_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:26.086611Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:26.086629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2139
2025-10-09T21:51:26.086786Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2139 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:26.086828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2139 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:26.086839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2139 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:26.086843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:26.087291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2139 sessionId:  describe result for acl check
2025-10-09T21:51:26.087333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:26.087342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:26.087344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:26.087350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342957053021536:9174] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:26.087922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342957053021536:9174] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:26.089713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:26.090140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342957053021536:9174] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:26.090814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:26.091007Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:26.091016Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:26.091028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:26.091092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:26.091131Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342956691184193:11499]
2025-10-09T21:51:26.091746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:26.092015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:26.092064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:26.092377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:26.092427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.092433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.092434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.092435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.092437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.092438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.092439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.092440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.092442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.092446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.092458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.092462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.092466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.092470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.092474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.092478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.092482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.092486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.092493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.092501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.092507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.092512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.092516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.092522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.092525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.092529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.092533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.092537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.092542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.092546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.092551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.092561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.092565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.092569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.092576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.092580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.092608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:26.092647Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342957053021536:9174] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:26.092660Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342957053021536:9174] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:26.092664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342957053021536:9174] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:26.092673Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2139 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:26.098995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:26.100300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:26.100311Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:26.100313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:26.100316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:26.100321Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2138 [2:7559342956691184193:11499]
2025-10-09T21:51:26.100332Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:26.100347Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:26.100352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:26.100354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:26.100356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:26.100369Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:26.100378Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:26.100386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:26.100442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.100448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.100449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.100451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:26.100506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:26.100516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.100526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.100539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.100554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:26.100574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342957053021251:2432], now have 1 active actors on pipe
2025-10-09T21:51:26.100591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342956691184188:11499]
2025-10-09T21:51:26.100594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342956691184188:11499]
2025-10-09T21:51:26.100596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342956691184188:11499]
2025-10-09T21:51:26.100608Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:26.100620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:26.100622Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:26.100627Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:26.100630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:26.100633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:26.100636Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:26.100646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342957053021617:9174], now have 1 active actors on pipe
2025-10-09T21:51:26.100637Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2138
2025-10-09T21:51:26.100654Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2138
2025-10-09T21:51:26.100677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342956691184188:11499]
2025-10-09T21:51:26.100690Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:26.100704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:26.100707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:26.100712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:26.100762Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|41ed46d3-bc0751e0-c4588509-d69a3a97_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:26.100776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:26.100780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:26.100783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:26.100793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:26.100803Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:26.100822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:26.100966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:26.100975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:26.100990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:26.101070Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2139 partition: 0 MaxSeqNo: 25 sessionId: src_id|41ed46d3-bc0751e0-c4588509-d69a3a97_0
2025-10-09T21:51:26.101455Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddd22bdf-3221085a-379214f7-22b5c496_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|41ed46d3-bc0751e0-c4588509-d69a3a97_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:26.103978Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|41ed46d3-bc0751e0-c4588509-d69a3a97_0] Write session: try to update token
2025-10-09T21:51:26.103997Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|41ed46d3-bc0751e0-c4588509-d69a3a97_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:26.115449Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:26.115466Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:26.115469Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:26.115471Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:26.115482Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342956691184188:11499]
2025-10-09T21:51:26.115486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342956691184188:11499]
2025-10-09T21:51:26.115490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342956691184188:11499]
2025-10-09T21:51:26.115534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:26.115546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.115551Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:26.115553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:26.115557Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:26.115561Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:26.115694Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:26.117177Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2139 sessionId: src_id|41ed46d3-bc0751e0-c4588509-d69a3a97_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:26.119693Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:26.176239Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342957053021629:9464]
2025-10-09T21:51:26.176461Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 2138, pipe: [1:7559342957053021629:9464]
2025-10-09T21:51:26.259306Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:26.259683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:26.259696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:26.259735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342957053021629:9464], now have 1 active actors on pipe
2025-10-09T21:51:26.259951Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:26.259992Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:26.259965Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:26.259998Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342957053021629:9464]
2025-10-09T21:51:26.260024Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2139 reason: error at write: Disconnected sessionId: src_id|41ed46d3-bc0751e0-c4588509-d69a3a97_0
2025-10-09T21:51:26.260108Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:26.260162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2139 sessionId: src_id|41ed46d3-bc0751e0-c4588509-d69a3a97_0 is DEAD
2025-10-09T21:51:26.260332Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41ed46d3-bc0751e0-c4588509-d69a3a97_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:26.260342Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41ed46d3-bc0751e0-c4588509-d69a3a97_0] Write session will restart in 0.007883s
2025-10-09T21:51:26.260362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41ed46d3-bc0751e0-c4588509-d69a3a97_0] Write session: Do CDS request
2025-10-09T21:51:26.260367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41ed46d3-bc0751e0-c4588509-d69a3a97_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:26.269510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|41ed46d3-bc0751e0-c4588509-d69a3a97_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:26.269558Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:26.269580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2140
2025-10-09T21:51:26.269706Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2140 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:26.269730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2140 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:26.269737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2140 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:26.269740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:26.270177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2140 sessionId:  describe result for acl check
2025-10-09T21:51:26.270217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:26.270223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:26.270225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:26.270231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342957053021916:9170] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:26.270781Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342957053021916:9170] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:26.272278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:26.272626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342957053021916:9170] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:26.273400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:26.273547Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:26.273556Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:26.273565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:26.273614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:26.273648Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342956691184413:11501]
2025-10-09T21:51:26.274190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:26.274441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:26.274479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:26.274667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342957053021916:9170] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:26.274681Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342957053021916:9170] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:26.274687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342957053021916:9170] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:26.274693Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2140 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:26.274760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:26.274806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.274812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.274813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.274814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.274815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.274816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.274817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.274818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.274820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.274824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.274834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.274841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.274845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.274849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.274853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.274856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.274860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.274863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.274868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.274880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.274890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.274899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.274907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.274910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.274916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.274918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.274921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.274923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.274926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.274932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.274934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.274937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.274940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.274946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.274949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.274952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.274968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:26.280820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:26.282117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:26.282129Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:26.282131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:26.282135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:26.282141Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2139 [2:7559342956691184413:11501]
2025-10-09T21:51:26.282152Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:26.282168Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:26.282175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:26.282179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:26.282182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:26.282197Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:26.282204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:26.282214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:26.282267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.282273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.282275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.282277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:26.282337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:26.282345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.282357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.282372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.282394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:26.282415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342957053021634:2432], now have 1 active actors on pipe
2025-10-09T21:51:26.282422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342957053022000:9170], now have 1 active actors on pipe
2025-10-09T21:51:26.282429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342956691184408:11501]
2025-10-09T21:51:26.282436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342956691184408:11501]
2025-10-09T21:51:26.282438Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342956691184408:11501]
2025-10-09T21:51:26.282452Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:26.282429Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2139
2025-10-09T21:51:26.282466Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:26.282469Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2139
2025-10-09T21:51:26.282466Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:26.282468Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:26.282469Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:26.282484Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:26.282487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:26.282521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:26.282530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:26.282547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342956691184408:11501]
2025-10-09T21:51:26.282560Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:26.282586Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|709f3599-8e9f11c8-cf004cc1-9ef6f328_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:26.282596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:26.282598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:26.282602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:26.282605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:26.282614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:26.282623Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:26.282647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:26.282778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:26.282785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:26.282797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:26.282892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2140 partition: 0 MaxSeqNo: 25 sessionId: src_id|709f3599-8e9f11c8-cf004cc1-9ef6f328_0
2025-10-09T21:51:26.283184Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41ed46d3-bc0751e0-c4588509-d69a3a97_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|709f3599-8e9f11c8-cf004cc1-9ef6f328_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:26.285373Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|709f3599-8e9f11c8-cf004cc1-9ef6f328_0] Write session: try to update token
2025-10-09T21:51:26.285385Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|709f3599-8e9f11c8-cf004cc1-9ef6f328_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:26.295875Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:26.295893Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:26.295895Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:26.295897Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:26.295906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342956691184408:11501]
2025-10-09T21:51:26.295911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342956691184408:11501]
2025-10-09T21:51:26.295915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342956691184408:11501]
2025-10-09T21:51:26.295956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:26.295959Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:26.295964Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:26.295964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.295967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:26.295968Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:26.296104Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:26.297035Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2140 sessionId: src_id|709f3599-8e9f11c8-cf004cc1-9ef6f328_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:26.299256Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:26.360037Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342957053022015:9464]
2025-10-09T21:51:26.360258Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 2139, pipe: [1:7559342957053022015:9464]
2025-10-09T21:51:26.436706Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:26.436948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:26.436963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:26.437006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342957053022015:9464], now have 1 active actors on pipe
2025-10-09T21:51:26.437264Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342957053022015:9464]
2025-10-09T21:51:26.437294Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:26.437296Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:26.437295Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:26.437371Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2140 reason: error at write: Disconnected sessionId: src_id|709f3599-8e9f11c8-cf004cc1-9ef6f328_0
2025-10-09T21:51:26.437550Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2140 sessionId: src_id|709f3599-8e9f11c8-cf004cc1-9ef6f328_0 is DEAD
2025-10-09T21:51:26.437688Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|709f3599-8e9f11c8-cf004cc1-9ef6f328_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:26.437699Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|709f3599-8e9f11c8-cf004cc1-9ef6f328_0] Write session will restart in 0.009713s
2025-10-09T21:51:26.437679Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:26.437723Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|709f3599-8e9f11c8-cf004cc1-9ef6f328_0] Write session: Do CDS request
2025-10-09T21:51:26.437730Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|709f3599-8e9f11c8-cf004cc1-9ef6f328_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:26.448485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|709f3599-8e9f11c8-cf004cc1-9ef6f328_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:26.448583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:26.448598Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2141
2025-10-09T21:51:26.448724Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2141 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:26.448755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2141 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:26.448775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2141 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:26.448782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:26.449185Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2141 sessionId:  describe result for acl check
2025-10-09T21:51:26.449223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:26.449229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:26.449231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:26.449237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342957053022337:9169] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:26.449771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342957053022337:9169] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:26.451133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:26.451328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342957053022337:9169] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:26.452177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:26.452338Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:26.452359Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:26.452373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:26.452423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:26.452476Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342956691184633:11503]
2025-10-09T21:51:26.453051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:26.453342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:26.453392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:26.453691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:26.453706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342957053022337:9169] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:26.453718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342957053022337:9169] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:26.453722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342957053022337:9169] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:26.453733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.453739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.453740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.453738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2141 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:26.453741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.453742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.453743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.453744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.453746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.453747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.453752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.453762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.453766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.453770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.453774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.453779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.453783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.453786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.453790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.453796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.453804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.453810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.453817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.453826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.453831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.453840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.453846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.453850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.453860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.453865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.453874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.453880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.453885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.453894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.453900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.453909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.453914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.453944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:26.459706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:26.461006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:26.461019Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:26.461022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:26.461026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:26.461034Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2140 [2:7559342956691184633:11503]
2025-10-09T21:51:26.461046Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:26.461063Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:26.461070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:26.461074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:26.461076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:26.461091Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:26.461101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:26.461112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:26.461167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.461173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.461176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.461178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:26.461241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:26.461248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.461259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.461275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.461297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:26.461321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342957053022016:2432], now have 1 active actors on pipe
2025-10-09T21:51:26.461328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342957053022385:9169], now have 1 active actors on pipe
2025-10-09T21:51:26.461335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342956691184628:11503]
2025-10-09T21:51:26.461340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342956691184628:11503]
2025-10-09T21:51:26.461343Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342956691184628:11503]
2025-10-09T21:51:26.461355Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:26.461343Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2140
2025-10-09T21:51:26.461378Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2140
2025-10-09T21:51:26.461382Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:26.461387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:26.461391Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:26.461396Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:26.461402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:26.461406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:26.461443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:26.461451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:26.461464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342956691184628:11503]
2025-10-09T21:51:26.461475Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:26.461498Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:26.461505Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6ed35118-c9dc62b2-bae64bb-1cce6056_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:26.461512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:26.461516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:26.461524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:26.461534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:26.461547Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:26.461574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:26.461694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:26.461701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:26.461715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:26.461796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2141 partition: 0 MaxSeqNo: 25 sessionId: src_id|6ed35118-c9dc62b2-bae64bb-1cce6056_0
2025-10-09T21:51:26.462088Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|709f3599-8e9f11c8-cf004cc1-9ef6f328_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6ed35118-c9dc62b2-bae64bb-1cce6056_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:26.464463Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ed35118-c9dc62b2-bae64bb-1cce6056_0] Write session: try to update token
2025-10-09T21:51:26.464481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ed35118-c9dc62b2-bae64bb-1cce6056_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:26.475177Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:26.475194Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:26.475197Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:26.475199Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:26.475210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342956691184628:11503]
2025-10-09T21:51:26.475215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342956691184628:11503]
2025-10-09T21:51:26.475220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342956691184628:11503]
2025-10-09T21:51:26.475261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:26.475268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.475267Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:26.475271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:26.475273Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:26.475277Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:26.475407Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:26.476864Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2141 sessionId: src_id|6ed35118-c9dc62b2-bae64bb-1cce6056_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:26.479116Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:26.537217Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342957053022395:9464]
2025-10-09T21:51:26.537517Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 2140, pipe: [1:7559342957053022395:9464]
2025-10-09T21:51:26.616815Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:26.617210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:26.617227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:26.617272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342957053022395:9464], now have 1 active actors on pipe
2025-10-09T21:51:26.617501Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:26.617525Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:26.617535Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:26.617539Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:26.617578Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342957053022395:9464]
2025-10-09T21:51:26.617614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2141 reason: error at write: Disconnected sessionId: src_id|6ed35118-c9dc62b2-bae64bb-1cce6056_0
2025-10-09T21:51:26.617786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2141 sessionId: src_id|6ed35118-c9dc62b2-bae64bb-1cce6056_0 is DEAD
2025-10-09T21:51:26.617926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ed35118-c9dc62b2-bae64bb-1cce6056_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:26.617944Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ed35118-c9dc62b2-bae64bb-1cce6056_0] Write session will restart in 0.006904s
2025-10-09T21:51:26.617965Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ed35118-c9dc62b2-bae64bb-1cce6056_0] Write session: Do CDS request
2025-10-09T21:51:26.617971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ed35118-c9dc62b2-bae64bb-1cce6056_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:26.626437Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ed35118-c9dc62b2-bae64bb-1cce6056_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:26.626541Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:26.626555Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2142
2025-10-09T21:51:26.626694Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2142 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:26.626725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2142 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:26.626734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2142 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:26.626737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:26.627173Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2142 sessionId:  describe result for acl check
2025-10-09T21:51:26.627207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:26.627209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:26.627211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:26.627216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342957053022632:9149] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:26.627763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342957053022632:9149] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:26.629651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342957053022632:9149] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:26.630914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:26.631890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:26.632049Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:26.632058Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:26.632068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:26.632113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:26.632147Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342956691184892:11517]
2025-10-09T21:51:26.632558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342957053022632:9149] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:26.632572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342957053022632:9149] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:26.632575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342957053022632:9149] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:26.632583Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2142 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:26.632685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:26.632926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:26.632962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:26.633249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:26.633293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.633298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.633299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.633300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.633302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.633303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.633304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.633305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.633306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.633311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.633321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.633325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.633329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.633333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.633337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.633340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.633344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.633347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.633354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.633372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.633378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.633387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.633391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.633396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.633400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.633405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.633413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.633418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.633422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.633431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.633435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.633440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.633448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.633454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.633458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.633463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.633491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:26.639517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:26.640714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:26.640725Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:26.640728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:26.640731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:26.640738Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2141 [2:7559342956691184892:11517]
2025-10-09T21:51:26.640752Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:26.640771Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:26.640778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:26.640781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:26.640784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:26.640799Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:26.640809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:26.640818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:26.640873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.640879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.640881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.640883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:26.640948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:26.640954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.640965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.640974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.640995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:26.641018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342957053022785:9149], now have 1 active actors on pipe
2025-10-09T21:51:26.641025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342957053022421:2432], now have 1 active actors on pipe
2025-10-09T21:51:26.641032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342956691184887:11517]
2025-10-09T21:51:26.641036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342956691184887:11517]
2025-10-09T21:51:26.641040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342956691184887:11517]
2025-10-09T21:51:26.641052Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:26.641065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:26.641070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:26.641072Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:26.641074Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:26.641083Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:26.641086Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:26.641107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342956691184887:11517]
2025-10-09T21:51:26.641114Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:26.641116Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:26.641071Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2141
2025-10-09T21:51:26.641083Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2141
2025-10-09T21:51:26.641138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:26.641148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:26.641304Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|189ffa0b-a2933434-f2f0d2e6-60bb08df_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:26.641318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:26.641322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:26.641324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:26.641334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:26.641343Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:26.641378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:26.641561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:26.641570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:26.641584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:26.641665Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2142 partition: 0 MaxSeqNo: 25 sessionId: src_id|189ffa0b-a2933434-f2f0d2e6-60bb08df_0
2025-10-09T21:51:26.641947Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ed35118-c9dc62b2-bae64bb-1cce6056_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|189ffa0b-a2933434-f2f0d2e6-60bb08df_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:26.644384Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|189ffa0b-a2933434-f2f0d2e6-60bb08df_0] Write session: try to update token
2025-10-09T21:51:26.644404Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|189ffa0b-a2933434-f2f0d2e6-60bb08df_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:26.654470Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:26.654484Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:26.654486Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:26.654487Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:26.654495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342956691184887:11517]
2025-10-09T21:51:26.654498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342956691184887:11517]
2025-10-09T21:51:26.654504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342956691184887:11517]
2025-10-09T21:51:26.654537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:26.654543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.654545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:26.654553Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:26.654559Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:26.654563Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:26.654663Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:26.657213Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2142 sessionId: src_id|189ffa0b-a2933434-f2f0d2e6-60bb08df_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:26.659488Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:26.717920Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342957053022798:9464]
2025-10-09T21:51:26.718292Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 2141, pipe: [1:7559342957053022798:9464]
2025-10-09T21:51:26.806814Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:26.807179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:26.807195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:26.807234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342957053022798:9464], now have 1 active actors on pipe
2025-10-09T21:51:26.807462Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:26.807532Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:26.807539Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:26.807583Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342957053022798:9464]
2025-10-09T21:51:26.807626Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2142 reason: status is not ok: tablet will be restarted right now sessionId: src_id|189ffa0b-a2933434-f2f0d2e6-60bb08df_0
2025-10-09T21:51:26.807786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2142 sessionId: src_id|189ffa0b-a2933434-f2f0d2e6-60bb08df_0 is DEAD
2025-10-09T21:51:26.807925Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|189ffa0b-a2933434-f2f0d2e6-60bb08df_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:26.807934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|189ffa0b-a2933434-f2f0d2e6-60bb08df_0] Write session will restart in 0.009521s
2025-10-09T21:51:26.807959Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|189ffa0b-a2933434-f2f0d2e6-60bb08df_0] Write session: Do CDS request
2025-10-09T21:51:26.807966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|189ffa0b-a2933434-f2f0d2e6-60bb08df_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:26.818518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|189ffa0b-a2933434-f2f0d2e6-60bb08df_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:26.818582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:26.818596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2143
2025-10-09T21:51:26.818723Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2143 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:26.818753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2143 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:26.818763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2143 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:26.818768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:26.819202Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2143 sessionId:  describe result for acl check
2025-10-09T21:51:26.819237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:26.819489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:26.819498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:26.819505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342957053023060:9139] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:26.820058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342957053023060:9139] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:26.822264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342957053023060:9139] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:26.823901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:26.824326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:26.824441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342957053023060:9139] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:26.824452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342957053023060:9139] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:26.824455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342957053023060:9139] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:26.824460Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2143 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:26.824618Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:26.824630Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:26.824642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:26.824693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:26.824740Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342956691185119:11521]
2025-10-09T21:51:26.825413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:26.825683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:26.825721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:26.826071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:26.826124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.826134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.826135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.826137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.826139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.826140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.826142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.826143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.826145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.826150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.826160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.826165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.826169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.826173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.826178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.826182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.826186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.826190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.826197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.826204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:26.826210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.826217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:26.826222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.826227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:26.826232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.826238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:26.826244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.826249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:26.826254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.826260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:26.826265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.826271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.826276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.826282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.826287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.826292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.826327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:26.832865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:26.834369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:26.834385Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:26.834387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:26.834391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:26.834398Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2142 [2:7559342956691185119:11521]
2025-10-09T21:51:26.834413Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:26.834435Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:26.834448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:26.834451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:26.834454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:26.834469Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:26.834481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:26.834487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:26.834543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.834551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.834554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.834557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:26.834631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:26.834640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.834652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:26.834665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:26.834689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:26.834710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342956691185113:11521]
2025-10-09T21:51:26.834718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342956691185113:11521]
2025-10-09T21:51:26.834721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342956691185113:11521]
2025-10-09T21:51:26.834731Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:26.834751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:26.834758Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:26.834762Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:26.834764Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:26.834772Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:26.834775Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:26.834785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342957053022803:2432], now have 1 active actors on pipe
2025-10-09T21:51:26.834789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342957053023171:9139], now have 1 active actors on pipe
2025-10-09T21:51:26.834793Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2142
2025-10-09T21:51:26.834820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342956691185113:11521]
2025-10-09T21:51:26.834797Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2142
2025-10-09T21:51:26.834831Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:26.834832Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:26.834859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:26.834869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:26.834900Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3ab1f4c0-a4301e34-36a3cf97-b128419a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:26.834912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:26.834917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:26.834925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:26.834933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:26.834947Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:26.834968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:26.835103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:26.835112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:26.835124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:26.835222Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2143 partition: 0 MaxSeqNo: 25 sessionId: src_id|3ab1f4c0-a4301e34-36a3cf97-b128419a_0
2025-10-09T21:51:26.835582Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|189ffa0b-a2933434-f2f0d2e6-60bb08df_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3ab1f4c0-a4301e34-36a3cf97-b128419a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:26.837846Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3ab1f4c0-a4301e34-36a3cf97-b128419a_0] Write session: try to update token
2025-10-09T21:51:26.837857Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3ab1f4c0-a4301e34-36a3cf97-b128419a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:26.849373Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2143 sessionId: src_id|3ab1f4c0-a4301e34-36a3cf97-b128419a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:26.849830Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:26.849847Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:26.849850Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:26.849853Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:26.849863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342956691185113:11521]
2025-10-09T21:51:26.849869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342956691185113:11521]
2025-10-09T21:51:26.849874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342956691185113:11521]
2025-10-09T21:51:26.849917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:26.849926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:26.849927Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:26.849928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:26.849934Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:26.849938Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:26.850117Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:26.851752Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:26.907893Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342957053023185:9464]
2025-10-09T21:51:26.908174Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 2142, pipe: [1:7559342957053023185:9464]
2025-10-09T21:51:26.999085Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:26.999501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:26.999514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:26.999548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342957053023185:9464], now have 1 active actors on pipe
2025-10-09T21:51:26.999764Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:26.999771Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342957053023185:9464]
2025-10-09T21:51:26.999795Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:26.999797Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:26.999866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2143 reason: error at write: Disconnected sessionId: src_id|3ab1f4c0-a4301e34-36a3cf97-b128419a_0
2025-10-09T21:51:26.999885Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:27.000055Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2143 sessionId: src_id|3ab1f4c0-a4301e34-36a3cf97-b128419a_0 is DEAD
2025-10-09T21:51:27.000231Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ab1f4c0-a4301e34-36a3cf97-b128419a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:27.000243Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ab1f4c0-a4301e34-36a3cf97-b128419a_0] Write session will restart in 0.006619s
2025-10-09T21:51:27.000274Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ab1f4c0-a4301e34-36a3cf97-b128419a_0] Write session: Do CDS request
2025-10-09T21:51:27.000281Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ab1f4c0-a4301e34-36a3cf97-b128419a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:27.008548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3ab1f4c0-a4301e34-36a3cf97-b128419a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:27.008578Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:27.008596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2144
2025-10-09T21:51:27.008791Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2144 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:27.008826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2144 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:27.008837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2144 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:27.008841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:27.009314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2144 sessionId:  describe result for acl check
2025-10-09T21:51:27.009348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:27.009356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:27.009367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:27.009372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342961347990602:9132] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:27.009923Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342961347990602:9132] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:27.012397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342961347990602:9132] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:27.015324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342961347990602:9132] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:27.015337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342961347990602:9132] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:27.015341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342961347990602:9132] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:27.015352Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2144 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:27.017316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:27.018912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:27.019074Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:27.019083Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:27.019092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:27.019131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:27.019165Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342960986152638:11523]
2025-10-09T21:51:27.019578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:27.019844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:27.019877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:27.020099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:27.020151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.020157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.020157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.020158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.020159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.020160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.020160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.020161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.020163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.020166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.020174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.020176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.020178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.020180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.020183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.020185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.020187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.020190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.020195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.020200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.020205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.020209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.020212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.020215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.020218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.020221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.020224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.020227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.020230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.020234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.020237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.020240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.020243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.020246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.020250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.020253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.020271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:27.027196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:27.028577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:27.028591Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:27.028593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:27.028596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:27.028603Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2143 [2:7559342960986152638:11523]
2025-10-09T21:51:27.028615Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:27.028636Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:27.028639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:27.028642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:27.028643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:27.028657Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:27.028662Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:27.028669Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:27.028716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.028723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.028726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.028728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:27.028785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:27.028795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.028801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.028808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.028818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:27.028833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342960986152633:11523]
2025-10-09T21:51:27.028835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342960986152633:11523]
2025-10-09T21:51:27.028837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342960986152633:11523]
2025-10-09T21:51:27.028843Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:27.028853Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:27.028855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342957053023189:2432], now have 1 active actors on pipe
2025-10-09T21:51:27.028856Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:27.028858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342961347990849:9132], now have 1 active actors on pipe
2025-10-09T21:51:27.028858Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:27.028865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:27.028879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:27.028883Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:27.028915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342960986152633:11523]
2025-10-09T21:51:27.028927Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:27.028934Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:27.028940Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2143
2025-10-09T21:51:27.028985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.028994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.028973Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2143
2025-10-09T21:51:27.029016Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3af43ab8-3e82edf5-26d21b8e-bb6f09a2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:27.029027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:27.029031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:27.029033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:27.029041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:27.029049Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:27.029062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:27.029206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.029213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.029223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:27.029293Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2144 partition: 0 MaxSeqNo: 25 sessionId: src_id|3af43ab8-3e82edf5-26d21b8e-bb6f09a2_0
2025-10-09T21:51:27.029677Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3ab1f4c0-a4301e34-36a3cf97-b128419a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3af43ab8-3e82edf5-26d21b8e-bb6f09a2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:27.032298Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3af43ab8-3e82edf5-26d21b8e-bb6f09a2_0] Write session: try to update token
2025-10-09T21:51:27.032316Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3af43ab8-3e82edf5-26d21b8e-bb6f09a2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:27.044046Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:27.044062Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:27.044064Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:27.044065Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:27.044075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342960986152633:11523]
2025-10-09T21:51:27.044078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342960986152633:11523]
2025-10-09T21:51:27.044082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342960986152633:11523]
2025-10-09T21:51:27.044120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:27.044126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.044128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:27.044130Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:27.044136Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:27.044140Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:27.044266Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:27.045255Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2144 sessionId: src_id|3af43ab8-3e82edf5-26d21b8e-bb6f09a2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:27.047565Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:27.099882Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342961347990863:9464]
2025-10-09T21:51:27.100244Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 2143, pipe: [1:7559342961347990863:9464]
2025-10-09T21:51:27.191063Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:27.191532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.191548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.191591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342961347990863:9464], now have 1 active actors on pipe
2025-10-09T21:51:27.191782Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:27.191823Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342961347990863:9464]
2025-10-09T21:51:27.191854Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:27.191869Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:27.191952Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2144 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3af43ab8-3e82edf5-26d21b8e-bb6f09a2_0
2025-10-09T21:51:27.192177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2144 sessionId: src_id|3af43ab8-3e82edf5-26d21b8e-bb6f09a2_0 is DEAD
2025-10-09T21:51:27.192326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3af43ab8-3e82edf5-26d21b8e-bb6f09a2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:27.192340Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3af43ab8-3e82edf5-26d21b8e-bb6f09a2_0] Write session will restart in 0.009735s
2025-10-09T21:51:27.192370Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3af43ab8-3e82edf5-26d21b8e-bb6f09a2_0] Write session: Do CDS request
2025-10-09T21:51:27.192377Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3af43ab8-3e82edf5-26d21b8e-bb6f09a2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:27.203691Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3af43ab8-3e82edf5-26d21b8e-bb6f09a2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:27.203711Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:27.203732Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2145
2025-10-09T21:51:27.203949Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2145 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:27.203988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2145 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:27.204000Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2145 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:27.204004Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:27.204400Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2145 sessionId:  describe result for acl check
2025-10-09T21:51:27.204438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:27.204446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:27.204448Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:27.204452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342961347991068:9129] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:27.204931Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342961347991068:9129] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:27.207697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342961347991068:9129] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:27.209897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:27.211721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342961347991068:9129] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:27.211736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:27.211738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342961347991068:9129] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:27.211743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342961347991068:9129] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:27.211756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2145 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:27.211927Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:27.211939Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:27.211949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:27.212012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:27.212052Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342960986152862:11525]
2025-10-09T21:51:27.212593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:27.212895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:27.212954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:27.213283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:27.213338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.213343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.213344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.213345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.213346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.213347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.213348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.213348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.213349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.213353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.213376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.213378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.213381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.213383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.213385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.213388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.213390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.213393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.213399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.213405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.213409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.213413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.213416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.213419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.213422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.213425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.213430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.213433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.213436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.213439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.213443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.213446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.213449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.213452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.213464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.213468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.213487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:27.220437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:27.221705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:27.221719Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:27.221722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:27.221726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:27.221732Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2144 [2:7559342960986152862:11525]
2025-10-09T21:51:27.221746Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:27.221760Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:27.221762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:27.221764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:27.221766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:27.221779Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:27.221783Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:27.221789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:27.221854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.221861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.221864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.221866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:27.221922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:27.221928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.221937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.221947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.221960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:27.221977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342960986152857:11525]
2025-10-09T21:51:27.221983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342960986152857:11525]
2025-10-09T21:51:27.221985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342960986152857:11525]
2025-10-09T21:51:27.221992Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:27.222003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342961347991236:9129], now have 1 active actors on pipe
2025-10-09T21:51:27.222004Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:27.222009Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:27.222012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342961347990866:2432], now have 1 active actors on pipe
2025-10-09T21:51:27.222012Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:27.222016Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:27.222026Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:27.222050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:27.222092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342960986152857:11525]
2025-10-09T21:51:27.222106Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:27.222117Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:27.222106Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2144
2025-10-09T21:51:27.222105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2144
2025-10-09T21:51:27.222175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.222183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.222201Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|51f0a1b0-a315dd2e-3cd39a6b-e5a9b494_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:27.222210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:27.222213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:27.222219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:27.222226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:27.222237Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:27.222252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:27.222410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.222417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.222427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:27.222573Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2145 partition: 0 MaxSeqNo: 25 sessionId: src_id|51f0a1b0-a315dd2e-3cd39a6b-e5a9b494_0
2025-10-09T21:51:27.222968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3af43ab8-3e82edf5-26d21b8e-bb6f09a2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|51f0a1b0-a315dd2e-3cd39a6b-e5a9b494_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:27.225496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51f0a1b0-a315dd2e-3cd39a6b-e5a9b494_0] Write session: try to update token
2025-10-09T21:51:27.225514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51f0a1b0-a315dd2e-3cd39a6b-e5a9b494_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:27.237235Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:27.237250Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:27.237254Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:27.237256Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:27.237269Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342960986152857:11525]
2025-10-09T21:51:27.237275Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342960986152857:11525]
2025-10-09T21:51:27.237279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342960986152857:11525]
2025-10-09T21:51:27.237322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:27.237330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.237331Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:27.237332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:27.237338Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:27.237371Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:27.237507Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:27.238506Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2145 sessionId: src_id|51f0a1b0-a315dd2e-3cd39a6b-e5a9b494_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:27.240677Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:27.291883Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342961347991250:9464]
2025-10-09T21:51:27.292176Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 2144, pipe: [1:7559342961347991250:9464]
2025-10-09T21:51:27.394110Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:27.394766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.394779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.394820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342961347991250:9464], now have 1 active actors on pipe
2025-10-09T21:51:27.395041Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:27.395166Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:27.395169Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342961347991250:9464]
2025-10-09T21:51:27.395181Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:27.395189Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:27.395247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2145 reason: error at write: Disconnected sessionId: src_id|51f0a1b0-a315dd2e-3cd39a6b-e5a9b494_0
2025-10-09T21:51:27.395413Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2145 sessionId: src_id|51f0a1b0-a315dd2e-3cd39a6b-e5a9b494_0 is DEAD
2025-10-09T21:51:27.395569Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51f0a1b0-a315dd2e-3cd39a6b-e5a9b494_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:27.395579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51f0a1b0-a315dd2e-3cd39a6b-e5a9b494_0] Write session will restart in 0.006505s
2025-10-09T21:51:27.395617Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51f0a1b0-a315dd2e-3cd39a6b-e5a9b494_0] Write session: Do CDS request
2025-10-09T21:51:27.395624Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51f0a1b0-a315dd2e-3cd39a6b-e5a9b494_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:27.403539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|51f0a1b0-a315dd2e-3cd39a6b-e5a9b494_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:27.403645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:27.403663Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2146
2025-10-09T21:51:27.403858Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2146 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:27.403895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2146 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:27.403907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2146 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:27.403911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:27.404379Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2146 sessionId:  describe result for acl check
2025-10-09T21:51:27.404416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:27.404422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:27.404425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:27.404431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342961347991324:9117] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:27.404991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342961347991324:9117] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:27.407367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342961347991324:9117] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:27.410558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342961347991324:9117] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:27.410578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342961347991324:9117] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:27.410582Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342961347991324:9117] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:27.410591Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2146 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:27.414237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:27.417990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:27.418225Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:27.418235Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:27.418246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:27.418295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:27.418341Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342960986153088:11527]
2025-10-09T21:51:27.418957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:27.419352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:27.419403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:27.419776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:27.419830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.419836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.419837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.419838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.419840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.419841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.419842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.419844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.419845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.419850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.419860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.419863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.419867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.419870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.419874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.419877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.419881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.419884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.419891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.419898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.419904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.419909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.419914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.419918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.419923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.419928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.419933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.419937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.419941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.419946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.419951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.419956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.419961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.419966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.419971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.419977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.420007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:27.426843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:27.428669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:27.428685Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:27.428688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:27.428692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:27.428701Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2145 [2:7559342960986153088:11527]
2025-10-09T21:51:27.428722Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:27.428744Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:27.428751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:27.428755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:27.428757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:27.428772Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:27.428778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:27.428785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:27.428852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.428859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.428862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.428864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:27.428945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:27.428953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.428963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.428975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.428996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:27.429018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342961347991258:2432], now have 1 active actors on pipe
2025-10-09T21:51:27.429026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342961347991574:9117], now have 1 active actors on pipe
2025-10-09T21:51:27.429032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342960986153083:11527]
2025-10-09T21:51:27.429034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342960986153083:11527]
2025-10-09T21:51:27.429037Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342960986153083:11527]
2025-10-09T21:51:27.429056Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:27.429074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:27.429093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:27.429092Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:27.429097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:27.429100Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:27.429101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:27.429077Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2145
2025-10-09T21:51:27.429133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342960986153083:11527]
2025-10-09T21:51:27.429141Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:27.429082Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2145
2025-10-09T21:51:27.429143Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:27.429157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.429166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.429216Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|666543d1-cebe1190-96db9b9c-ffe99e9f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:27.429227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:27.429230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:27.429233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:27.429242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:27.429250Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:27.429270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:27.429473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.429484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.429500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:27.429603Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2146 partition: 0 MaxSeqNo: 25 sessionId: src_id|666543d1-cebe1190-96db9b9c-ffe99e9f_0
2025-10-09T21:51:27.430024Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|51f0a1b0-a315dd2e-3cd39a6b-e5a9b494_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|666543d1-cebe1190-96db9b9c-ffe99e9f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:27.432496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|666543d1-cebe1190-96db9b9c-ffe99e9f_0] Write session: try to update token
2025-10-09T21:51:27.432516Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|666543d1-cebe1190-96db9b9c-ffe99e9f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:27.445907Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2146 sessionId: src_id|666543d1-cebe1190-96db9b9c-ffe99e9f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:27.448551Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:27.448573Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:27.448582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342960986153083:11527]
2025-10-09T21:51:27.448630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:27.448639Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:27.448822Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:27.448988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:27.448973Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:27.449149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:27.449310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:27.449551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:27.449600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.449616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.449632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:27.449838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:27.449928Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:27.456134Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:27.456148Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:27.456151Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:27.456153Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:27.456161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342960986153083:11527]
2025-10-09T21:51:27.456166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342960986153083:11527]
2025-10-09T21:51:27.456170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342960986153083:11527]
2025-10-09T21:51:27.456206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:27.456206Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:27.456212Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:27.456213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.456215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:27.456217Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:27.456371Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:27.495174Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342961347991640:9464]
2025-10-09T21:51:27.495533Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 2145, pipe: [1:7559342961347991640:9464]
2025-10-09T21:51:27.615682Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:27.624592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.624619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.624794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.624870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.624942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.625015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.625327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.625343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.625437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.625509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.625587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.625654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.625795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.625806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.625873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.625939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.626003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.626111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.626158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.626174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.626248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.626329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.626405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.626480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.626538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.626549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.626624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.626691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.626760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.626830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.626878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.626887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.626955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.627023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.627090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.627160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.627236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.627245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.627317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.627384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.627455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.627521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.627592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.627601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.627664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.627728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.627798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.627867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.627936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.627945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.628008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.628076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.628148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.628222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.628272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.628280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.628351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.628418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.628483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.628548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.628599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.628607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.628672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342961347991640:9464], now have 1 active actors on pipe
2025-10-09T21:51:27.629530Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:27.629835Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:27.629892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2146 reason: status is not ok: tablet will be restarted right now sessionId: src_id|666543d1-cebe1190-96db9b9c-ffe99e9f_0
2025-10-09T21:51:27.630137Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:27.630192Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342961347991640:9464]
2025-10-09T21:51:27.630261Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|666543d1-cebe1190-96db9b9c-ffe99e9f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:27.630274Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|666543d1-cebe1190-96db9b9c-ffe99e9f_0] Write session will restart in 0.006965s
2025-10-09T21:51:27.630316Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|666543d1-cebe1190-96db9b9c-ffe99e9f_0] Write session: Do CDS request
2025-10-09T21:51:27.630323Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|666543d1-cebe1190-96db9b9c-ffe99e9f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:27.630400Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2146 sessionId: src_id|666543d1-cebe1190-96db9b9c-ffe99e9f_0 is DEAD
2025-10-09T21:51:27.638485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|666543d1-cebe1190-96db9b9c-ffe99e9f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:27.638537Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:27.638558Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2147
2025-10-09T21:51:27.638705Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2147 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:27.638739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2147 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:27.638742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2147 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:27.638745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:27.639231Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2147 sessionId:  describe result for acl check
2025-10-09T21:51:27.639273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:27.639275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:27.639277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:27.639283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342961347991717:9101] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:27.639859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342961347991717:9101] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:27.642707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342961347991717:9101] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:27.645804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342961347991717:9101] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:27.645830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342961347991717:9101] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:27.645835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342961347991717:9101] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:27.645849Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2147 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:27.651393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:27.652831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:27.653094Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:27.653109Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:27.653120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:27.653183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:27.653242Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342960986153352:11535]
2025-10-09T21:51:27.653944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:27.654361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:27.654419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:27.654806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:27.654878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.654887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.654888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.654891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.654893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.654894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.654896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.654898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.654899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.654906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.654916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.654920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.654925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.654929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.654932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.654936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.654940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.654944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.654951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.654959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.654966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.654972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.654979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.654986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.654991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.654997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.655002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.655008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.655013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.655021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.655030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.655035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.655039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.655045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.655050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.655056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.655085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:27.662620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:27.664724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:27.664740Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:27.664744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:27.664748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:27.664756Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2146 [2:7559342960986153352:11535]
2025-10-09T21:51:27.664778Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:27.664796Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:27.664799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:27.664802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:27.664804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:27.664821Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:27.664826Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:27.664833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:27.664903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.664913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.664916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.664918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:27.664995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:27.665003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.665016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.665026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.665042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:27.665057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342960986153347:11535]
2025-10-09T21:51:27.665060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342960986153347:11535]
2025-10-09T21:51:27.665064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342960986153347:11535]
2025-10-09T21:51:27.665078Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:27.665091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342961347991927:9101], now have 1 active actors on pipe
2025-10-09T21:51:27.665091Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:27.665095Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:27.665097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:27.665106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342961347991658:2432], now have 1 active actors on pipe
2025-10-09T21:51:27.665114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:27.665127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:27.665131Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:27.665167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342960986153347:11535]
2025-10-09T21:51:27.665174Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2146
2025-10-09T21:51:27.665180Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:27.665208Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2146
2025-10-09T21:51:27.665196Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:27.665220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.665224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.665248Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|56fe2f5e-eee2b95c-a440ef03-a53bd4d2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:27.665262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:27.665268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:27.665277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:27.665288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:27.665297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:27.665317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:27.665472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.665481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.665495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:27.665568Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2147 partition: 0 MaxSeqNo: 25 sessionId: src_id|56fe2f5e-eee2b95c-a440ef03-a53bd4d2_0
2025-10-09T21:51:27.666004Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|666543d1-cebe1190-96db9b9c-ffe99e9f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|56fe2f5e-eee2b95c-a440ef03-a53bd4d2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:27.668537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56fe2f5e-eee2b95c-a440ef03-a53bd4d2_0] Write session: try to update token
2025-10-09T21:51:27.668558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56fe2f5e-eee2b95c-a440ef03-a53bd4d2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:27.678728Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:27.678744Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:27.678755Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342960986153347:11535]
2025-10-09T21:51:27.678808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:27.678822Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:27.678990Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:27.679138Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:27.679291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:27.679447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:27.679649Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:27.679692Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:27.682621Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2147 sessionId: src_id|56fe2f5e-eee2b95c-a440ef03-a53bd4d2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:27.685473Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:27.685710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.685730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.685766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:27.685861Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:27.692994Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:27.693009Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:27.693013Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:27.693015Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:27.693027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342960986153347:11535]
2025-10-09T21:51:27.693032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342960986153347:11535]
2025-10-09T21:51:27.693038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342960986153347:11535]
2025-10-09T21:51:27.693085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:27.693084Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:27.693091Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:27.693097Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:27.693099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.693102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:27.693268Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:27.730203Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342961347992048:9464]
2025-10-09T21:51:27.730525Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 2146, pipe: [1:7559342961347992048:9464]
2025-10-09T21:51:27.861015Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:27.872273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.872307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.872514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.872611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.872690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.872768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.872828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.872839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.872920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.872990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.873068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.873159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.873218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.873227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.873310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.873516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.873601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.873662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.873706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.873719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.873784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.873884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.873941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.874059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.874105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.874115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.874180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.874245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.874306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.874386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.874436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.874446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.874509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.874575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.874641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.874702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.874768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.874778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.874838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.874927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.874991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.875053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.875103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.875111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.875181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.875236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.875297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.875364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.875411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.875417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.875478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.875544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.875608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.875674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.875733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.875741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.875804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:27.875865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:27.875930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:27.875993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:27.876035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:27.876044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:27.876115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342961347992048:9464], now have 1 active actors on pipe
2025-10-09T21:51:27.876450Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:27.876479Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:27.876509Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342961347992048:9464]
2025-10-09T21:51:27.876512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2147 reason: error at write: Disconnected sessionId: src_id|56fe2f5e-eee2b95c-a440ef03-a53bd4d2_0
2025-10-09T21:51:27.876543Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:27.876829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2147 sessionId: src_id|56fe2f5e-eee2b95c-a440ef03-a53bd4d2_0 is DEAD
2025-10-09T21:51:27.876942Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56fe2f5e-eee2b95c-a440ef03-a53bd4d2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:27.876957Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56fe2f5e-eee2b95c-a440ef03-a53bd4d2_0] Write session will restart in 0.008562s
2025-10-09T21:51:27.876983Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56fe2f5e-eee2b95c-a440ef03-a53bd4d2_0] Write session: Do CDS request
2025-10-09T21:51:27.876990Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56fe2f5e-eee2b95c-a440ef03-a53bd4d2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:27.877408Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:27.886455Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|56fe2f5e-eee2b95c-a440ef03-a53bd4d2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:27.886557Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:27.886575Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2148
2025-10-09T21:51:27.886754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2148 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:27.886788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2148 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:27.886805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2148 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:27.886809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:27.887329Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2148 sessionId:  describe result for acl check
2025-10-09T21:51:27.887378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:27.887387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:27.887389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:27.887397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342961347992156:9099] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:27.888030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342961347992156:9099] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:27.891076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342961347992156:9099] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:27.894296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342961347992156:9099] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:27.894312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342961347992156:9099] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:27.894315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342961347992156:9099] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:27.894325Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2148 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:27.895094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:27.896646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:27.896843Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:27.896855Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:27.896865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:27.896916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:27.896959Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342960986153584:11545]
2025-10-09T21:51:27.897632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:27.897949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:27.897990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:27.898339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:27.898386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.898391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.898392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.898394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.898396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.898397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.898399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.898400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.898401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.898406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.898416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.898420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.898424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.898428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.898431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.898433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.898435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.898437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.898442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.898446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:27.898450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.898454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:27.898456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.898459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:27.898462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.898465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:27.898469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.898474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:27.898479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.898484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:27.898490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.898495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.898501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.898505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.898508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.898512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.898528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:27.905531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:27.907316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:27.907329Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:27.907332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:27.907334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:27.907340Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2147 [2:7559342960986153584:11545]
2025-10-09T21:51:27.907352Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:27.907373Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:27.907376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:27.907380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:27.907383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:27.907397Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:27.907403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:27.907409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:27.907474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.907482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.907484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.907487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:27.907572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:27.907580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.907591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:27.907606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:27.907624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:27.907645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342961347992424:9099], now have 1 active actors on pipe
2025-10-09T21:51:27.907653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342961347992051:2432], now have 1 active actors on pipe
2025-10-09T21:51:27.907660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342960986153579:11545]
2025-10-09T21:51:27.907663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342960986153579:11545]
2025-10-09T21:51:27.907665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342960986153579:11545]
2025-10-09T21:51:27.907678Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:27.907697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:27.907699Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:27.907702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:27.907704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:27.907715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:27.907718Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:27.907741Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2147
2025-10-09T21:51:27.907752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342960986153579:11545]
2025-10-09T21:51:27.907757Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2147
2025-10-09T21:51:27.907762Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:27.907765Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:27.907794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.907798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.907829Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|29614f0d-e89d7598-a41fa4d5-9c1f2485_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:27.907843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:27.907847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:27.907850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:27.907860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:27.907873Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:27.907889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:27.908066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.908078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.908111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:27.908205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2148 partition: 0 MaxSeqNo: 25 sessionId: src_id|29614f0d-e89d7598-a41fa4d5-9c1f2485_0
2025-10-09T21:51:27.908638Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|56fe2f5e-eee2b95c-a440ef03-a53bd4d2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|29614f0d-e89d7598-a41fa4d5-9c1f2485_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:27.911889Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29614f0d-e89d7598-a41fa4d5-9c1f2485_0] Write session: try to update token
2025-10-09T21:51:27.911909Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29614f0d-e89d7598-a41fa4d5-9c1f2485_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:27.920952Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:27.920971Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:27.920981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342960986153579:11545]
2025-10-09T21:51:27.921026Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:27.921027Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:27.921223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:27.921395Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:27.921550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:27.921706Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:27.922054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:27.922099Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:27.929272Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2148 sessionId: src_id|29614f0d-e89d7598-a41fa4d5-9c1f2485_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:27.932174Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:27.932391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:27.932413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:27.932449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:27.932544Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:27.935465Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:27.935479Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:27.935481Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:27.935484Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:27.935495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342960986153579:11545]
2025-10-09T21:51:27.935500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342960986153579:11545]
2025-10-09T21:51:27.935506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342960986153579:11545]
2025-10-09T21:51:27.935547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:27.935548Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:27.935555Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:27.935558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:27.935560Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:27.935561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:27.935732Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:27.976992Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342961347992437:9464]
2025-10-09T21:51:27.977291Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 2147, pipe: [1:7559342961347992437:9464]
2025-10-09T21:51:28.096090Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:28.106866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:28.106893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:28.107060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:28.107124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:28.107194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:28.107264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:28.107319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:28.107327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:28.107402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:28.107473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:28.107543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:28.107614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:28.107668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:28.107675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:28.107742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:28.107807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:28.107886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:28.108071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:28.108121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:28.108130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:28.108219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:28.108294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:28.108356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:28.108414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:28.108469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:28.108475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:28.108549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:28.108607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:28.108664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:28.108720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:28.108772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:28.108781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:28.108858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:28.108917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:28.108977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:28.109038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:28.109103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:28.109109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:28.109167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:28.109216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:28.109271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:28.109334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:28.109391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:28.109399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:28.109456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:28.109508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:28.109570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:28.109630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:28.109669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:28.109676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:28.109736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:28.109795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:28.109857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:28.109919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:28.109967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:28.109970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:28.110047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:28.110109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:28.110169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:28.110228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:28.110268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:28.110274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:28.110331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342961347992437:9464], now have 1 active actors on pipe
2025-10-09T21:51:28.110614Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:28.110625Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342961347992437:9464]
2025-10-09T21:51:28.110662Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:28.110665Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:28.110709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2148 reason: error at write: Disconnected sessionId: src_id|29614f0d-e89d7598-a41fa4d5-9c1f2485_0
2025-10-09T21:51:28.110885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2148 sessionId: src_id|29614f0d-e89d7598-a41fa4d5-9c1f2485_0 is DEAD
2025-10-09T21:51:28.111068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29614f0d-e89d7598-a41fa4d5-9c1f2485_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:28.111077Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29614f0d-e89d7598-a41fa4d5-9c1f2485_0] Write session will restart in 0.008600s
2025-10-09T21:51:28.111103Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29614f0d-e89d7598-a41fa4d5-9c1f2485_0] Write session: Do CDS request
2025-10-09T21:51:28.111109Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29614f0d-e89d7598-a41fa4d5-9c1f2485_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:28.111260Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:28.120513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29614f0d-e89d7598-a41fa4d5-9c1f2485_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:28.120736Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:28.120756Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2149
2025-10-09T21:51:28.120933Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2149 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:28.120971Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2149 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:28.120978Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2149 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:28.120988Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:28.121498Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2149 sessionId:  describe result for acl check
2025-10-09T21:51:28.121542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:28.121549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:28.121551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:28.121557Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342965642959875:9094] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:28.122214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342965642959875:9094] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:28.125768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342965642959875:9094] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:28.128871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342965642959875:9094] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:28.128885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342965642959875:9094] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:28.128888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342965642959875:9094] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:28.128909Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2149 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:28.128897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:28.130445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:28.130691Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:28.130701Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:28.130711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:28.130755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:28.130787Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342965281121109:11547]
2025-10-09T21:51:28.131294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:28.131551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:28.131583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:28.131832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:28.131867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.131871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.131872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.131873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.131873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.131874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.131875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.131875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.131876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.131879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.131886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.131888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.131891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.131893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.131895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.131897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.131899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.131901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.131906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.131911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.131918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.131927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.131936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.131941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.131949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.131952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.131959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.131962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.131966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.131969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.131972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.131976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.131978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.131981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.131984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.131987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.132004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:28.138758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:28.139723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:28.139737Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:28.139739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:28.139743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:28.139749Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2148 [2:7559342965281121109:11547]
2025-10-09T21:51:28.139765Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:28.139780Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:28.139786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:28.139789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:28.139791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:28.139806Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:28.139815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:28.139820Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:28.140091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.140102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.140104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.140107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:28.140172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:28.140178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.140190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.140204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.140229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:28.140250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342965642960108:9094], now have 1 active actors on pipe
2025-10-09T21:51:28.140259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342965642959736:2432], now have 1 active actors on pipe
2025-10-09T21:51:28.140266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342965281121103:11547]
2025-10-09T21:51:28.140268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342965281121103:11547]
2025-10-09T21:51:28.140271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342965281121103:11547]
2025-10-09T21:51:28.140286Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:28.140301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:28.140308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:28.140311Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:28.140313Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:28.140320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:28.140323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:28.140296Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2148
2025-10-09T21:51:28.140346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342965281121103:11547]
2025-10-09T21:51:28.140318Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2148
2025-10-09T21:51:28.140358Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:28.140360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:28.140380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:28.140389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:28.140443Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a7c82e6e-e059951a-2e2f9be8-3eadd6dd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:28.140455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:28.140462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:28.140464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:28.140475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:28.140489Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:28.140506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:28.140646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:28.140654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:28.140669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:28.140758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2149 partition: 0 MaxSeqNo: 25 sessionId: src_id|a7c82e6e-e059951a-2e2f9be8-3eadd6dd_0
2025-10-09T21:51:28.141125Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29614f0d-e89d7598-a41fa4d5-9c1f2485_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a7c82e6e-e059951a-2e2f9be8-3eadd6dd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:28.144334Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c82e6e-e059951a-2e2f9be8-3eadd6dd_0] Write session: try to update token
2025-10-09T21:51:28.144357Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c82e6e-e059951a-2e2f9be8-3eadd6dd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:28.155526Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:28.155541Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:28.155544Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:28.155546Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:28.155555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342965281121103:11547]
2025-10-09T21:51:28.155560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342965281121103:11547]
2025-10-09T21:51:28.155564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342965281121103:11547]
2025-10-09T21:51:28.155601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:28.155608Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:28.155612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.155614Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:28.155615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:28.155618Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:28.155750Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:28.160906Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2149 sessionId: src_id|a7c82e6e-e059951a-2e2f9be8-3eadd6dd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:28.163814Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:28.210956Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342965642960121:9464]
2025-10-09T21:51:28.211266Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 2148, pipe: [1:7559342965642960121:9464]
2025-10-09T21:51:28.311104Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:28.311372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:28.311397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:28.311439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342965642960121:9464], now have 1 active actors on pipe
2025-10-09T21:51:28.311659Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:28.311665Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:28.311684Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:28.311690Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:28.311731Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342965642960121:9464]
2025-10-09T21:51:28.311769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2149 reason: error at write: Disconnected sessionId: src_id|a7c82e6e-e059951a-2e2f9be8-3eadd6dd_0
2025-10-09T21:51:28.311959Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2149 sessionId: src_id|a7c82e6e-e059951a-2e2f9be8-3eadd6dd_0 is DEAD
2025-10-09T21:51:28.312097Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c82e6e-e059951a-2e2f9be8-3eadd6dd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:28.312105Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c82e6e-e059951a-2e2f9be8-3eadd6dd_0] Write session will restart in 0.009641s
2025-10-09T21:51:28.312132Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c82e6e-e059951a-2e2f9be8-3eadd6dd_0] Write session: Do CDS request
2025-10-09T21:51:28.312138Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c82e6e-e059951a-2e2f9be8-3eadd6dd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:28.322601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c82e6e-e059951a-2e2f9be8-3eadd6dd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:28.322688Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:28.322716Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2150
2025-10-09T21:51:28.322860Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2150 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:28.322895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2150 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:28.322905Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2150 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:28.322909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:28.323378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2150 sessionId:  describe result for acl check
2025-10-09T21:51:28.323418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:28.323424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:28.323425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:28.323432Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342965642960369:9085] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:28.323992Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342965642960369:9085] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:28.326143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342965642960369:9085] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:28.327365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:28.328373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:28.328549Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:28.328558Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:28.328570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:28.328566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342965642960369:9085] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:28.328576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342965642960369:9085] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:28.328579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342965642960369:9085] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:28.328585Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2150 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:28.328624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:28.328660Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342965281121337:11549]
2025-10-09T21:51:28.329250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:28.329506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:28.329545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:28.329854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:28.329900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.329906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.329907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.329908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.329910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.329911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.329912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.329913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.329915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.329920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.329931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.329936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.329940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.329944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.329949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.329953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.329957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.329961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.329968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.329976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.329988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.329993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.329999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.330004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.330008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.330013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.330018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.330024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.330054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.330062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.330067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.330073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.330082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.330087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.330097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.330104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.330139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:28.336059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:28.337128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:28.337141Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:28.337143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:28.337148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:28.337154Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2149 [2:7559342965281121337:11549]
2025-10-09T21:51:28.337167Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:28.337187Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:28.337194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:28.337197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:28.337199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:28.337214Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:28.337224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:28.337231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:28.337288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.337292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.337294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.337297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:28.337378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:28.337387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.337399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.337412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.337431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:28.337454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342965642960131:2432], now have 1 active actors on pipe
2025-10-09T21:51:28.337462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342965642960506:9085], now have 1 active actors on pipe
2025-10-09T21:51:28.337470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342965281121332:11549]
2025-10-09T21:51:28.337473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342965281121332:11549]
2025-10-09T21:51:28.337475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342965281121332:11549]
2025-10-09T21:51:28.337487Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:28.337502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:28.337506Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:28.337509Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:28.337511Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:28.337521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:28.337524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:28.337513Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2149
2025-10-09T21:51:28.337515Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2149
2025-10-09T21:51:28.337546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342965281121332:11549]
2025-10-09T21:51:28.337556Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:28.337559Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:28.337572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:28.337577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:28.337625Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ddb3d941-92a94cb3-e6cf244-6c273976_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:28.337639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:28.337643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:28.337645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:28.337655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:28.337664Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:28.337681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:28.337842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:28.337849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:28.337863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:28.337954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2150 partition: 0 MaxSeqNo: 25 sessionId: src_id|ddb3d941-92a94cb3-e6cf244-6c273976_0
2025-10-09T21:51:28.338351Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a7c82e6e-e059951a-2e2f9be8-3eadd6dd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ddb3d941-92a94cb3-e6cf244-6c273976_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:28.341240Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ddb3d941-92a94cb3-e6cf244-6c273976_0] Write session: try to update token
2025-10-09T21:51:28.341259Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ddb3d941-92a94cb3-e6cf244-6c273976_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:28.351130Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:28.351147Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:28.351149Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:28.351151Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:28.351161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342965281121332:11549]
2025-10-09T21:51:28.351166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342965281121332:11549]
2025-10-09T21:51:28.351170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342965281121332:11549]
2025-10-09T21:51:28.351212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:28.351214Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:28.351220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.351223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:28.351231Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:28.351235Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:28.351369Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:28.355684Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2150 sessionId: src_id|ddb3d941-92a94cb3-e6cf244-6c273976_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:28.357899Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:28.411653Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342965642960518:9464]
2025-10-09T21:51:28.411877Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 2149, pipe: [1:7559342965642960518:9464]
2025-10-09T21:51:28.511179Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:28.511449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:28.511466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:28.511509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342965642960518:9464], now have 1 active actors on pipe
2025-10-09T21:51:28.511705Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342965642960518:9464]
2025-10-09T21:51:28.511710Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:28.511753Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:28.511755Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:28.511819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2150 reason: error at write: Disconnected sessionId: src_id|ddb3d941-92a94cb3-e6cf244-6c273976_0
2025-10-09T21:51:28.511983Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:28.512011Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2150 sessionId: src_id|ddb3d941-92a94cb3-e6cf244-6c273976_0 is DEAD
2025-10-09T21:51:28.512161Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddb3d941-92a94cb3-e6cf244-6c273976_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:28.512169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddb3d941-92a94cb3-e6cf244-6c273976_0] Write session will restart in 0.009542s
2025-10-09T21:51:28.512206Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddb3d941-92a94cb3-e6cf244-6c273976_0] Write session: Do CDS request
2025-10-09T21:51:28.512213Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddb3d941-92a94cb3-e6cf244-6c273976_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:28.522548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ddb3d941-92a94cb3-e6cf244-6c273976_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:28.522626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:28.522643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2151
2025-10-09T21:51:28.522843Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2151 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:28.522878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2151 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:28.522891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2151 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:28.522894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:28.523330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2151 sessionId:  describe result for acl check
2025-10-09T21:51:28.523367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:28.523373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:28.523376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:28.523381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342965642960774:9079] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:28.523912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342965642960774:9079] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:28.526348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342965642960774:9079] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:28.527107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:28.528156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:28.528332Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:28.528342Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:28.528354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:28.528418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:28.528463Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342965281121568:11551]
2025-10-09T21:51:28.528934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342965642960774:9079] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:28.528957Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342965642960774:9079] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:28.528960Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342965642960774:9079] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:28.528972Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2151 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:28.529120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:28.529337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:28.529382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:28.529595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:28.529629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.529634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.529635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.529636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.529637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.529637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.529638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.529638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.529639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.529642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.529649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.529651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.529653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.529655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.529657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.529659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.529661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.529663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.529668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.529672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.529677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.529680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.529682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.529685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.529687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.529690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.529693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.529695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.529697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.529700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.529703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.529706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.529708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.529711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.529714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.529717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.529731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:28.536165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:28.537184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:28.537199Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:28.537201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:28.537204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:28.537210Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2150 [2:7559342965281121568:11551]
2025-10-09T21:51:28.537221Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:28.537238Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:28.537240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:28.537242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:28.537243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:28.537257Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:28.537261Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:28.537267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:28.537324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.537331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.537333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.537335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:28.537407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:28.537412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.537420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.537426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.537438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:28.537455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342965281121563:11551]
2025-10-09T21:51:28.537463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342965281121563:11551]
2025-10-09T21:51:28.537465Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342965281121563:11551]
2025-10-09T21:51:28.537470Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:28.537479Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:28.537481Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:28.537482Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:28.537483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:28.537491Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:28.537493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:28.537503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342965642960897:9079], now have 1 active actors on pipe
2025-10-09T21:51:28.537505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342965642960521:2432], now have 1 active actors on pipe
2025-10-09T21:51:28.537531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342965281121563:11551]
2025-10-09T21:51:28.537541Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:28.537554Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:28.537584Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2150
2025-10-09T21:51:28.537607Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2150
2025-10-09T21:51:28.537656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:28.537663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:28.537684Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|48a8e9c6-e7b21c16-7b9c3d07-82a5e468_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:28.537694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:28.537701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:28.537702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:28.537710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:28.537722Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:28.537734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:28.537875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:28.537880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:28.537890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:28.537961Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2151 partition: 0 MaxSeqNo: 25 sessionId: src_id|48a8e9c6-e7b21c16-7b9c3d07-82a5e468_0
2025-10-09T21:51:28.538359Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ddb3d941-92a94cb3-e6cf244-6c273976_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|48a8e9c6-e7b21c16-7b9c3d07-82a5e468_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:28.540950Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48a8e9c6-e7b21c16-7b9c3d07-82a5e468_0] Write session: try to update token
2025-10-09T21:51:28.540964Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48a8e9c6-e7b21c16-7b9c3d07-82a5e468_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:28.551122Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:28.551142Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:28.551144Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:28.551146Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:28.551155Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342965281121563:11551]
2025-10-09T21:51:28.551160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342965281121563:11551]
2025-10-09T21:51:28.551164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342965281121563:11551]
2025-10-09T21:51:28.551205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:28.551216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.551222Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:28.551224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:28.551228Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:28.551232Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:28.551350Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:28.552745Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2151 sessionId: src_id|48a8e9c6-e7b21c16-7b9c3d07-82a5e468_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:28.555033Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:28.611870Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342965642960908:9464]
2025-10-09T21:51:28.612116Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 2150, pipe: [1:7559342965642960908:9464]
2025-10-09T21:51:28.710053Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:28.710709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:28.710724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:28.710769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342965642960908:9464], now have 1 active actors on pipe
2025-10-09T21:51:28.710968Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:28.710976Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342965642960908:9464]
2025-10-09T21:51:28.710984Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:28.710997Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:28.711052Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2151 reason: error at write: Disconnected sessionId: src_id|48a8e9c6-e7b21c16-7b9c3d07-82a5e468_0
2025-10-09T21:51:28.711233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2151 sessionId: src_id|48a8e9c6-e7b21c16-7b9c3d07-82a5e468_0 is DEAD
2025-10-09T21:51:28.711393Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48a8e9c6-e7b21c16-7b9c3d07-82a5e468_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:28.711402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48a8e9c6-e7b21c16-7b9c3d07-82a5e468_0] Write session will restart in 0.008131s
2025-10-09T21:51:28.711427Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48a8e9c6-e7b21c16-7b9c3d07-82a5e468_0] Write session: Do CDS request
2025-10-09T21:51:28.711436Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48a8e9c6-e7b21c16-7b9c3d07-82a5e468_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:28.711930Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:28.720524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48a8e9c6-e7b21c16-7b9c3d07-82a5e468_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:28.720609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:28.720628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2152
2025-10-09T21:51:28.720787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2152 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:28.720822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2152 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:28.720835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2152 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:28.720839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:28.721339Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2152 sessionId:  describe result for acl check
2025-10-09T21:51:28.721389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:28.721395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:28.721397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:28.721403Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342965642961084:9063] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:28.721951Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342965642961084:9063] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:28.723824Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342965642961084:9063] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:28.726790Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342965642961084:9063] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:28.726804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342965642961084:9063] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:28.726808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342965642961084:9063] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:28.726816Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2152 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:28.728051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:28.729417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:28.729582Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:28.729589Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:28.729599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:28.729658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:28.729688Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342965281121837:11565]
2025-10-09T21:51:28.730207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:28.730552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:28.730620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:28.730976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:28.731031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.731035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.731036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.731037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.731038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.731039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.731041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.731042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.731043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.731048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.731059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.731063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.731068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.731072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.731076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.731080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.731083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.731087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.731095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.731103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.731109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.731115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.731119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.731124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.731129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.731134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.731139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.731144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.731148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.731153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.731159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.731164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.731168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.731174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.731178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.731187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.731217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:28.737952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:28.739550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:28.739565Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:28.739568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:28.739572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:28.739578Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2151 [2:7559342965281121837:11565]
2025-10-09T21:51:28.739592Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:28.739611Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:28.739614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:28.739617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:28.739619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:28.739637Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:28.739648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:28.739656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:28.739722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.739727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.739729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.739732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:28.739809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:28.739818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.739831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.739843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.739860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:28.739887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342965281121832:11565]
2025-10-09T21:51:28.739890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342965281121832:11565]
2025-10-09T21:51:28.739892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342965281121832:11565]
2025-10-09T21:51:28.739901Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:28.739911Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:28.739914Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:28.739916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342965642961307:9063], now have 1 active actors on pipe
2025-10-09T21:51:28.739917Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:28.739920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342965642960935:2432], now have 1 active actors on pipe
2025-10-09T21:51:28.739929Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:28.739944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:28.739947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:28.739985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342965281121832:11565]
2025-10-09T21:51:28.739977Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2151
2025-10-09T21:51:28.739979Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2151
2025-10-09T21:51:28.739999Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:28.740009Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:28.740036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:28.740041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:28.740075Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d11282b-4b6c001f-9190f780-787de625_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:28.740090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:28.740094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:28.740096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:28.740109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:28.740121Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:28.740136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:28.740280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:28.740291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:28.740314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:28.740427Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2152 partition: 0 MaxSeqNo: 25 sessionId: src_id|d11282b-4b6c001f-9190f780-787de625_0
2025-10-09T21:51:28.740818Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48a8e9c6-e7b21c16-7b9c3d07-82a5e468_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d11282b-4b6c001f-9190f780-787de625_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:28.743290Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d11282b-4b6c001f-9190f780-787de625_0] Write session: try to update token
2025-10-09T21:51:28.743306Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d11282b-4b6c001f-9190f780-787de625_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:28.755054Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:28.755071Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:28.755073Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:28.755076Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:28.755086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342965281121832:11565]
2025-10-09T21:51:28.755090Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342965281121832:11565]
2025-10-09T21:51:28.755094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342965281121832:11565]
2025-10-09T21:51:28.755135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:28.755140Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:28.755146Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:28.755149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.755149Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:28.755152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:28.755291Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:28.756390Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2152 sessionId: src_id|d11282b-4b6c001f-9190f780-787de625_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:28.758826Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:28.810990Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342965642961320:9464]
2025-10-09T21:51:28.811269Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 2151, pipe: [1:7559342965642961320:9464]
2025-10-09T21:51:28.912131Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:28.912362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:28.912374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:28.912414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342965642961320:9464], now have 1 active actors on pipe
2025-10-09T21:51:28.912594Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:28.912655Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342965642961320:9464]
2025-10-09T21:51:28.912681Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:28.912689Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:28.912775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2152 reason: status is not ok: tablet will be restarted right now sessionId: src_id|d11282b-4b6c001f-9190f780-787de625_0
2025-10-09T21:51:28.912958Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2152 sessionId: src_id|d11282b-4b6c001f-9190f780-787de625_0 is DEAD
2025-10-09T21:51:28.913119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d11282b-4b6c001f-9190f780-787de625_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:28.913128Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d11282b-4b6c001f-9190f780-787de625_0] Write session will restart in 0.005027s
2025-10-09T21:51:28.913150Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d11282b-4b6c001f-9190f780-787de625_0] Write session: Do CDS request
2025-10-09T21:51:28.913155Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d11282b-4b6c001f-9190f780-787de625_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:28.919495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d11282b-4b6c001f-9190f780-787de625_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:28.919680Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:28.919697Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2153
2025-10-09T21:51:28.919807Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2153 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:28.919839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2153 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:28.919852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2153 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:28.919855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:28.920296Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2153 sessionId:  describe result for acl check
2025-10-09T21:51:28.920329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:28.920336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:28.920338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:28.920344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342965642961419:9056] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:28.920891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342965642961419:9056] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:28.923267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342965642961419:9056] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:28.925947Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342965642961419:9056] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:28.925962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342965642961419:9056] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:28.925964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342965642961419:9056] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:28.925972Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2153 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:28.928019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:28.929109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:28.929285Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:28.929294Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:28.929305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:28.929380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:28.929421Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342965281122074:11569]
2025-10-09T21:51:28.929939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:28.930242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:28.930305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:28.930639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:28.930684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.930689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.930690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.930692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.930693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.930694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.930695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.930696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.930697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.930702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.930712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.930716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.930720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.930724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.930728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.930731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.930734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.930738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.930744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.930752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:28.930758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.930764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:28.930768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.930773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:28.930777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.930782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:28.930787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.930792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:28.930796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.930801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:28.930806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.930811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.930815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.930825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.930830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.930835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.930864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:28.936808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:28.937847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:28.937859Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:28.937861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:28.937865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:28.937873Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2152 [2:7559342965281122074:11569]
2025-10-09T21:51:28.937886Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:28.937902Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:28.937905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:28.937908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:28.937910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:28.937927Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:28.937938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:28.937944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:28.938003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.938009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.938012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.938015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:28.938089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:28.938098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.938108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:28.938118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:28.938134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:28.938155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342965642961703:9056], now have 1 active actors on pipe
2025-10-09T21:51:28.938164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342965642961325:2432], now have 1 active actors on pipe
2025-10-09T21:51:28.938172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342965281122069:11569]
2025-10-09T21:51:28.938174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342965281122069:11569]
2025-10-09T21:51:28.938176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342965281122069:11569]
2025-10-09T21:51:28.938188Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:28.938202Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:28.938205Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:28.938207Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:28.938209Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:28.938215Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:28.938218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:28.938202Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2152
2025-10-09T21:51:28.938242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342965281122069:11569]
2025-10-09T21:51:28.938204Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2152
2025-10-09T21:51:28.938247Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:28.938249Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:28.938273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:28.938282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:28.938315Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bc6640d4-c57f3e-dbfe6252-948033b9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:28.938326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:28.938334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:28.938337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:28.938346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:28.938360Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:28.938381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:28.938489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:28.938496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:28.938509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:28.938585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2153 partition: 0 MaxSeqNo: 25 sessionId: src_id|bc6640d4-c57f3e-dbfe6252-948033b9_0
2025-10-09T21:51:28.938916Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d11282b-4b6c001f-9190f780-787de625_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bc6640d4-c57f3e-dbfe6252-948033b9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:28.941217Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc6640d4-c57f3e-dbfe6252-948033b9_0] Write session: try to update token
2025-10-09T21:51:28.941233Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc6640d4-c57f3e-dbfe6252-948033b9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:28.951502Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:28.951520Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:28.951523Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:28.951525Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:28.951536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342965281122069:11569]
2025-10-09T21:51:28.951540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342965281122069:11569]
2025-10-09T21:51:28.951544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342965281122069:11569]
2025-10-09T21:51:28.951593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:28.951596Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:28.951601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:28.951603Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:28.951605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:28.951606Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:28.951738Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:28.954369Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2153 sessionId: src_id|bc6640d4-c57f3e-dbfe6252-948033b9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:28.956744Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:29.012704Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342969937929022:9464]
2025-10-09T21:51:29.012978Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 2152, pipe: [1:7559342969937929022:9464]
2025-10-09T21:51:29.107698Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:29.108480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.108496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.108543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342969937929022:9464], now have 1 active actors on pipe
2025-10-09T21:51:29.108755Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:29.108761Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:29.108767Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342969937929022:9464]
2025-10-09T21:51:29.108785Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:29.108790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:29.108853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2153 reason: error at write: Disconnected sessionId: src_id|bc6640d4-c57f3e-dbfe6252-948033b9_0
2025-10-09T21:51:29.109033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2153 sessionId: src_id|bc6640d4-c57f3e-dbfe6252-948033b9_0 is DEAD
2025-10-09T21:51:29.109195Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc6640d4-c57f3e-dbfe6252-948033b9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:29.109208Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc6640d4-c57f3e-dbfe6252-948033b9_0] Write session will restart in 0.006859s
2025-10-09T21:51:29.109236Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc6640d4-c57f3e-dbfe6252-948033b9_0] Write session: Do CDS request
2025-10-09T21:51:29.109243Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc6640d4-c57f3e-dbfe6252-948033b9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:29.117633Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc6640d4-c57f3e-dbfe6252-948033b9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:29.117735Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:29.117756Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2154
2025-10-09T21:51:29.117907Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2154 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:29.117953Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2154 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:29.117964Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2154 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:29.117969Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:29.118460Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2154 sessionId:  describe result for acl check
2025-10-09T21:51:29.118501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:29.118507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:29.118509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:29.118517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342969937929137:9050] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:29.118987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342969937929137:9050] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:29.121414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342969937929137:9050] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:29.124464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342969937929137:9050] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:29.124480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342969937929137:9050] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:29.124484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342969937929137:9050] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:29.124492Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2154 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:29.126389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:29.128383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:29.128596Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:29.128606Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:29.128620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:29.128683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:29.128726Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342969576089603:11571]
2025-10-09T21:51:29.129351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:29.129670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:29.129717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:29.130058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:29.130120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.130127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.130128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.130130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.130131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.130132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.130133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.130135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.130136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.130141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.130154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.130158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.130163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.130167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.130171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.130175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.130179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.130183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.130190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.130199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.130206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.130212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.130217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.130222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.130227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.130232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.130237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.130242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.130248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.130253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.130280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.130291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.130296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.130302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.130307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.130315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.130346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:29.136941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:29.138235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:29.138249Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:29.138252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:29.138257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:29.138263Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2153 [2:7559342969576089603:11571]
2025-10-09T21:51:29.138566Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:29.138591Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:29.138598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:29.138601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:29.138603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:29.138622Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:29.138633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:29.138641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:29.138719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.138727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.138729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.138732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:29.138804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:29.138811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.138825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.138841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.138864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:29.138888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342969576089598:11571]
2025-10-09T21:51:29.138893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342969576089598:11571]
2025-10-09T21:51:29.138895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342969576089598:11571]
2025-10-09T21:51:29.138907Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:29.138922Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:29.138928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342969937929400:9050], now have 1 active actors on pipe
2025-10-09T21:51:29.138929Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:29.138931Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:29.138931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342969937929024:2432], now have 1 active actors on pipe
2025-10-09T21:51:29.138940Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:29.138956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:29.138959Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:29.138997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342969576089598:11571]
2025-10-09T21:51:29.139008Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:29.139014Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:29.139014Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2153
2025-10-09T21:51:29.139120Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2153
2025-10-09T21:51:29.139181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.139190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.139216Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|52f4a60d-e2588121-4662f0ef-d82d0cc2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:29.139229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:29.139238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:29.139241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:29.139252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:29.139267Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:29.139286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:29.139464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.139476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.139505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:29.139609Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2154 partition: 0 MaxSeqNo: 25 sessionId: src_id|52f4a60d-e2588121-4662f0ef-d82d0cc2_0
2025-10-09T21:51:29.139964Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc6640d4-c57f3e-dbfe6252-948033b9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|52f4a60d-e2588121-4662f0ef-d82d0cc2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:29.142456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52f4a60d-e2588121-4662f0ef-d82d0cc2_0] Write session: try to update token
2025-10-09T21:51:29.142474Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52f4a60d-e2588121-4662f0ef-d82d0cc2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:29.151369Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:29.151387Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:29.151398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342969576089598:11571]
2025-10-09T21:51:29.151441Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:29.151449Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:29.151597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:29.151737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:29.151865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:29.152008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:29.152196Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:29.152231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:29.156228Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2154 sessionId: src_id|52f4a60d-e2588121-4662f0ef-d82d0cc2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:29.158580Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:29.158854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.158882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.158921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:29.159006Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:29.164703Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:29.164716Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:29.164718Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:29.164720Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:29.164730Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342969576089598:11571]
2025-10-09T21:51:29.164734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342969576089598:11571]
2025-10-09T21:51:29.164737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342969576089598:11571]
2025-10-09T21:51:29.164781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:29.164790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.164790Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:29.164792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:29.164796Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:29.164801Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:29.164920Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:29.208824Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342969937929415:9464]
2025-10-09T21:51:29.209093Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 2153, pipe: [1:7559342969937929415:9464]
2025-10-09T21:51:29.311687Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:29.318327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.318346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.318506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.318561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.318834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.318897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.319070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.319079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.319140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.319194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.319250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.319308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.319345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.319351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.319413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.319466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.319534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.319585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.319623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.319629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.319691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.319748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.319806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.319862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.319927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.319933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.319990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.320045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.320099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.320156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.320201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.320207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.320264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.320316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.320370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.320430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.320558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.320564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.320616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.320669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.320728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.320780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.320816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.320821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.320872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.320936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.320990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.321044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.321081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.321088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.321139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.321193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.321248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.321302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.321343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.321348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.321419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.321475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.321527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.321578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.321614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.321619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.321657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342969937929415:9464], now have 1 active actors on pipe
2025-10-09T21:51:29.321860Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:29.321886Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342969937929415:9464]
2025-10-09T21:51:29.321908Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:29.321910Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:29.321943Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2154 reason: error at write: Disconnected sessionId: src_id|52f4a60d-e2588121-4662f0ef-d82d0cc2_0
2025-10-09T21:51:29.322019Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:29.322115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2154 sessionId: src_id|52f4a60d-e2588121-4662f0ef-d82d0cc2_0 is DEAD
2025-10-09T21:51:29.322247Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52f4a60d-e2588121-4662f0ef-d82d0cc2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:29.322258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52f4a60d-e2588121-4662f0ef-d82d0cc2_0] Write session will restart in 0.006448s
2025-10-09T21:51:29.322276Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52f4a60d-e2588121-4662f0ef-d82d0cc2_0] Write session: Do CDS request
2025-10-09T21:51:29.322282Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52f4a60d-e2588121-4662f0ef-d82d0cc2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:29.329471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52f4a60d-e2588121-4662f0ef-d82d0cc2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:29.329584Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:29.329601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2155
2025-10-09T21:51:29.329707Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2155 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:29.329740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2155 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:29.329750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2155 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:29.329754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:29.330175Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2155 sessionId:  describe result for acl check
2025-10-09T21:51:29.330211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:29.330219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:29.330221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:29.330226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342969937929587:9045] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:29.330762Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342969937929587:9045] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:29.332905Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342969937929587:9045] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:29.335238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:29.335316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342969937929587:9045] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:29.335330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342969937929587:9045] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:29.335335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342969937929587:9045] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:29.335347Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2155 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:29.336079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:29.336265Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:29.336275Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:29.336286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:29.336334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:29.336368Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342969576089837:11573]
2025-10-09T21:51:29.336820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:29.337054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:29.337081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:29.337299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:29.337335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.337340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.337340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.337341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.337341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.337342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.337343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.337343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.337344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.337347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.337362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.337364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.337366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.337368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.337371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.337387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.337390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.337394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.337400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.337406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.337412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.337417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.337420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.337423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.337426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.337428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.337431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.337433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.337437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.337442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.337446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.337450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.337455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.337459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.337463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.337470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.337495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:29.343150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:29.344351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:29.344362Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:29.344364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:29.344366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:29.344370Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2154 [2:7559342969576089837:11573]
2025-10-09T21:51:29.344381Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:29.344393Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:29.344394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:29.344397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:29.344398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:29.344410Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:29.344414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:29.344419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:29.344467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.344475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.344476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.344478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:29.344523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:29.344528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.344535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.344542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.344557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:29.344578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342969576089831:11573]
2025-10-09T21:51:29.344585Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342969576089831:11573]
2025-10-09T21:51:29.344586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342969576089831:11573]
2025-10-09T21:51:29.344597Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:29.344608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342969937929804:9045], now have 1 active actors on pipe
2025-10-09T21:51:29.344614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342969937929425:2432], now have 1 active actors on pipe
2025-10-09T21:51:29.344614Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:29.344617Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:29.344619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:29.344620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:29.344628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:29.344630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:29.344646Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342969576089831:11573]
2025-10-09T21:51:29.344652Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:29.344663Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:29.344642Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2154
2025-10-09T21:51:29.344673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.344657Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2154
2025-10-09T21:51:29.344676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.344699Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1bd123a8-bcaabe16-6a785dc5-9bde5963_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:29.344708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:29.344711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:29.344713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:29.344720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:29.344729Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:29.344743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:29.344845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.344851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.344862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:29.344935Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2155 partition: 0 MaxSeqNo: 25 sessionId: src_id|1bd123a8-bcaabe16-6a785dc5-9bde5963_0
2025-10-09T21:51:29.345204Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52f4a60d-e2588121-4662f0ef-d82d0cc2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1bd123a8-bcaabe16-6a785dc5-9bde5963_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:29.347496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1bd123a8-bcaabe16-6a785dc5-9bde5963_0] Write session: try to update token
2025-10-09T21:51:29.347508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1bd123a8-bcaabe16-6a785dc5-9bde5963_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:29.355061Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:29.355075Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:29.355083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342969576089831:11573]
2025-10-09T21:51:29.355115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:29.355127Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:29.355275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:29.355412Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:29.355545Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:29.355677Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:29.355862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:29.355899Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:29.358739Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2155 sessionId: src_id|1bd123a8-bcaabe16-6a785dc5-9bde5963_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:29.360700Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:29.360850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.360869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.360898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:29.360960Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:29.366626Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:29.366640Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:29.366643Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:29.366645Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:29.366655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342969576089831:11573]
2025-10-09T21:51:29.366658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342969576089831:11573]
2025-10-09T21:51:29.366662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342969576089831:11573]
2025-10-09T21:51:29.366701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:29.366702Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:29.366707Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:29.366708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.366711Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:29.366711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:29.366868Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:29.421810Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342969937929817:9464]
2025-10-09T21:51:29.422012Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 2154, pipe: [1:7559342969937929817:9464]
2025-10-09T21:51:29.503976Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:29.510806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.510826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.510970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.511025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.511081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.511135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.511567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.511579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.511637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.511689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.511752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.511813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.511851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.511857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.511909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.511960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.512024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.512074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.512120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.512125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.512181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.512239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.512297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.512350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.512388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.512393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.512448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.512509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.512563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.512613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.512651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.512660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.512714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.512806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.512862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.512923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.512961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.512966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.513025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.513085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.513165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.513215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.513258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.513267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.513323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.513382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.513431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.513480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.513522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.513529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.513591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.513641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.513694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.513751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.513810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.513817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.513869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:29.513931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:29.513982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:29.514052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:29.514103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:29.514108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:29.514147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342969937929817:9464], now have 1 active actors on pipe
2025-10-09T21:51:29.514390Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342969937929817:9464]
2025-10-09T21:51:29.514401Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:29.514405Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:29.514427Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:29.514436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2155 reason: error at write: Disconnected sessionId: src_id|1bd123a8-bcaabe16-6a785dc5-9bde5963_0
2025-10-09T21:51:29.514632Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2155 sessionId: src_id|1bd123a8-bcaabe16-6a785dc5-9bde5963_0 is DEAD
2025-10-09T21:51:29.514650Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:29.514814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bd123a8-bcaabe16-6a785dc5-9bde5963_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:29.514827Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bd123a8-bcaabe16-6a785dc5-9bde5963_0] Write session will restart in 0.008512s
2025-10-09T21:51:29.514852Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bd123a8-bcaabe16-6a785dc5-9bde5963_0] Write session: Do CDS request
2025-10-09T21:51:29.514858Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bd123a8-bcaabe16-6a785dc5-9bde5963_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:29.524530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1bd123a8-bcaabe16-6a785dc5-9bde5963_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:29.524629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:29.524643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2156
2025-10-09T21:51:29.524745Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2156 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:29.524769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2156 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:29.524777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2156 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:29.524780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:29.525154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2156 sessionId:  describe result for acl check
2025-10-09T21:51:29.525187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:29.525193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:29.525194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:29.525199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342969937930112:9042] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:29.525691Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342969937930112:9042] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:29.527761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342969937930112:9042] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:29.528736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:29.529115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:29.529268Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:29.529278Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:29.529289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:29.529340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:29.529390Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342969576090072:11575]
2025-10-09T21:51:29.529818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342969937930112:9042] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:29.529831Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342969937930112:9042] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:29.529834Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342969937930112:9042] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:29.529843Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2156 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:29.529883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:29.530134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:29.530176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:29.530450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:29.530513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.530522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.530523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.530524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.530525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.530526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.530527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.530528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.530528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.530533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.530542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.530545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.530548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.530551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.530554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.530556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.530559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.530562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.530571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.530577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.530582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.530586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.530590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.530594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.530597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.530601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.530605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.530609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.530612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.530616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.530621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.530625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.530628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.530632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.530639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.530643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.530670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:29.536333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:29.537464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:29.537477Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:29.537479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:29.537483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:29.537489Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2155 [2:7559342969576090072:11575]
2025-10-09T21:51:29.537501Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:29.537517Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:29.537520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:29.537523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:29.537525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:29.537540Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:29.537546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:29.537559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:29.537607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.537613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.537614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.537617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:29.537682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:29.537689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.537702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.537717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.537739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:29.537763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342969937929820:2432], now have 1 active actors on pipe
2025-10-09T21:51:29.537771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342969937930200:9042], now have 1 active actors on pipe
2025-10-09T21:51:29.537779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342969576090067:11575]
2025-10-09T21:51:29.537784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342969576090067:11575]
2025-10-09T21:51:29.537787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342969576090067:11575]
2025-10-09T21:51:29.537802Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:29.537819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:29.537825Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:29.537829Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:29.537830Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:29.537839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:29.537838Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2155
2025-10-09T21:51:29.537843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:29.537866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342969576090067:11575]
2025-10-09T21:51:29.537838Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2155
2025-10-09T21:51:29.537869Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:29.537878Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:29.537898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.537907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.537942Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f60de468-ade3afd4-15002371-f366098a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:29.537954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:29.537957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:29.537960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:29.537970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:29.537983Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:29.538000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:29.538191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.538200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.538214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:29.538319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2156 partition: 0 MaxSeqNo: 25 sessionId: src_id|f60de468-ade3afd4-15002371-f366098a_0
2025-10-09T21:51:29.538588Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1bd123a8-bcaabe16-6a785dc5-9bde5963_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f60de468-ade3afd4-15002371-f366098a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:29.540827Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f60de468-ade3afd4-15002371-f366098a_0] Write session: try to update token
2025-10-09T21:51:29.540844Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f60de468-ade3afd4-15002371-f366098a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:29.550749Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:29.550767Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:29.550769Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:29.550771Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:29.550781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342969576090067:11575]
2025-10-09T21:51:29.550786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342969576090067:11575]
2025-10-09T21:51:29.550790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342969576090067:11575]
2025-10-09T21:51:29.550833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:29.550839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.550839Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:29.550841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:29.550844Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:29.550848Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:29.551003Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:29.552358Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2156 sessionId: src_id|f60de468-ade3afd4-15002371-f366098a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:29.554475Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:29.614320Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342969937930212:9464]
2025-10-09T21:51:29.614577Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 2155, pipe: [1:7559342969937930212:9464]
2025-10-09T21:51:29.691775Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:29.692224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.692236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.692272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342969937930212:9464], now have 1 active actors on pipe
2025-10-09T21:51:29.692466Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:29.692507Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:29.692550Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:29.692623Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2156 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f60de468-ade3afd4-15002371-f366098a_0
2025-10-09T21:51:29.692796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2156 sessionId: src_id|f60de468-ade3afd4-15002371-f366098a_0 is DEAD
2025-10-09T21:51:29.692947Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f60de468-ade3afd4-15002371-f366098a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:29.692956Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f60de468-ade3afd4-15002371-f366098a_0] Write session will restart in 0.006660s
2025-10-09T21:51:29.692952Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342969937930212:9464]
2025-10-09T21:51:29.692973Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f60de468-ade3afd4-15002371-f366098a_0] Write session: Do CDS request
2025-10-09T21:51:29.692979Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f60de468-ade3afd4-15002371-f366098a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:29.700476Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f60de468-ade3afd4-15002371-f366098a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:29.700583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:29.700600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2157
2025-10-09T21:51:29.700721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2157 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:29.700752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2157 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:29.700761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2157 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:29.700765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:29.701178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2157 sessionId:  describe result for acl check
2025-10-09T21:51:29.701208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:29.701214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:29.701216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:29.701221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342969937930335:9025] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:29.701743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342969937930335:9025] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:29.704021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342969937930335:9025] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:29.706925Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342969937930335:9025] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:29.706952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342969937930335:9025] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:29.706956Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342969937930335:9025] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:29.706964Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2157 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:29.708558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:29.709885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:29.710011Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:29.710018Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:29.710040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:29.710074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:29.710103Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342969576090346:11583]
2025-10-09T21:51:29.710462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:29.710705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:29.710735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:29.710927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:29.710955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.710959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.710960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.710961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.710961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.710962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.710962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.710963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.710963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.710966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.710972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.710974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.710976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.710979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.710981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.710982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.710984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.710986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.710990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.710995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.710999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.711002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.711004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.711007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.711009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.711012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.711015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.711017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.711020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.711023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.711026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.711029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.711031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.711034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.711036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.711039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.711055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:29.716730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:29.717830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:29.717841Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:29.717842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:29.717845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:29.717850Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2156 [2:7559342969576090346:11583]
2025-10-09T21:51:29.717858Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:29.717871Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:29.717876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:29.717878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:29.717880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:29.717892Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:29.717899Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:29.717904Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:29.717941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.717946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.717948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.717950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:29.717996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:29.718001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.718007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.718013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.718024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:29.718055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342969937930227:2432], now have 1 active actors on pipe
2025-10-09T21:51:29.718064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342969576090341:11583]
2025-10-09T21:51:29.718069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342969576090341:11583]
2025-10-09T21:51:29.718071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342969576090341:11583]
2025-10-09T21:51:29.718079Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:29.718088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342969937930615:9025], now have 1 active actors on pipe
2025-10-09T21:51:29.718092Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:29.718094Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:29.718095Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:29.718096Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:29.718103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:29.718105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:29.718121Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342969576090341:11583]
2025-10-09T21:51:29.718126Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:29.718127Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:29.718110Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2156
2025-10-09T21:51:29.718156Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2156
2025-10-09T21:51:29.718168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.718174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.718196Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f6e01964-5ef0dbf-9f49534b-a83f177e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:29.718204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:29.718206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:29.718207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:29.718214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:29.718225Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:29.718240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:29.718419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.718440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.718466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:29.718567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2157 partition: 0 MaxSeqNo: 25 sessionId: src_id|f6e01964-5ef0dbf-9f49534b-a83f177e_0
2025-10-09T21:51:29.718874Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f60de468-ade3afd4-15002371-f366098a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f6e01964-5ef0dbf-9f49534b-a83f177e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:29.721216Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f6e01964-5ef0dbf-9f49534b-a83f177e_0] Write session: try to update token
2025-10-09T21:51:29.721231Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f6e01964-5ef0dbf-9f49534b-a83f177e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:29.730891Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:29.730905Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:29.730906Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:29.730908Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:29.730915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342969576090341:11583]
2025-10-09T21:51:29.730919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342969576090341:11583]
2025-10-09T21:51:29.730922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342969576090341:11583]
2025-10-09T21:51:29.730955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:29.730960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.730962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:29.730963Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:29.730967Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:29.730971Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:29.731077Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:29.732956Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2157 sessionId: src_id|f6e01964-5ef0dbf-9f49534b-a83f177e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:29.735064Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:29.793015Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342969937930631:9464]
2025-10-09T21:51:29.793277Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 2156, pipe: [1:7559342969937930631:9464]
2025-10-09T21:51:29.866418Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:29.866774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.866788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.866842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342969937930631:9464], now have 1 active actors on pipe
2025-10-09T21:51:29.867026Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:29.867062Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:29.867071Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342969937930631:9464]
2025-10-09T21:51:29.867075Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:29.867100Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:29.867129Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2157 reason: error at write: Disconnected sessionId: src_id|f6e01964-5ef0dbf-9f49534b-a83f177e_0
2025-10-09T21:51:29.867299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2157 sessionId: src_id|f6e01964-5ef0dbf-9f49534b-a83f177e_0 is DEAD
2025-10-09T21:51:29.867468Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6e01964-5ef0dbf-9f49534b-a83f177e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:29.867479Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6e01964-5ef0dbf-9f49534b-a83f177e_0] Write session will restart in 0.005322s
2025-10-09T21:51:29.867504Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6e01964-5ef0dbf-9f49534b-a83f177e_0] Write session: Do CDS request
2025-10-09T21:51:29.867512Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6e01964-5ef0dbf-9f49534b-a83f177e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:29.873497Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f6e01964-5ef0dbf-9f49534b-a83f177e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:29.873580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:29.873593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2158
2025-10-09T21:51:29.873700Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2158 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:29.873724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2158 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:29.873738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2158 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:29.873744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:29.874145Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2158 sessionId:  describe result for acl check
2025-10-09T21:51:29.874178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:29.874184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:29.874186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:29.874191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342969937930729:9020] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:29.874683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342969937930729:9020] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:29.876772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342969937930729:9020] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:29.879118Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342969937930729:9020] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:29.879127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342969937930729:9020] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:29.879130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342969937930729:9020] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:29.879134Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2158 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:29.880048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:29.881472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:29.881623Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:29.881629Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:29.881638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:29.881675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:29.881703Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342969576090588:11593]
2025-10-09T21:51:29.882056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:29.882299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:29.882329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:29.882521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:29.882551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.882555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.882556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.882557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.882557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.882558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.882559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.882559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.882560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.882562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.882569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.882571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.882573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.882575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.882577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.882579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.882580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.882582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.882586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.882591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:29.882595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.882598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:29.882600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.882603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:29.882605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.882608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:29.882612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.882615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:29.882617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.882620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:29.882622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.882625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.882627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.882630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.882633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.882636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.882651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:29.888325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:29.889441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:29.889451Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:29.889452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:29.889455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:29.889460Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2157 [2:7559342969576090588:11593]
2025-10-09T21:51:29.889467Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:29.889480Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:29.889481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:29.889483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:29.889484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:29.889495Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:29.889499Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:29.889505Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:29.889541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.889546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.889547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.889550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:29.889593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:29.889600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.889607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:29.889612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:29.889621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:29.889634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342969576090583:11593]
2025-10-09T21:51:29.889635Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342969576090583:11593]
2025-10-09T21:51:29.889637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342969576090583:11593]
2025-10-09T21:51:29.889642Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:29.889652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342969937930633:2432], now have 1 active actors on pipe
2025-10-09T21:51:29.889655Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:29.889658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342969937931013:9020], now have 1 active actors on pipe
2025-10-09T21:51:29.889659Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:29.889661Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:29.889665Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:29.889672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:29.889674Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:29.889690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342969576090583:11593]
2025-10-09T21:51:29.889698Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:29.889711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:29.889698Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2157
2025-10-09T21:51:29.889709Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2157
2025-10-09T21:51:29.889752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.889759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.889778Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9d53044b-9ae8495c-6b74afda-ac416e48_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:29.889786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:29.889792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:29.889794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:29.889801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:29.889811Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:29.889821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:29.889941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.889946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.889956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:29.890042Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2158 partition: 0 MaxSeqNo: 25 sessionId: src_id|9d53044b-9ae8495c-6b74afda-ac416e48_0
2025-10-09T21:51:29.890339Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6e01964-5ef0dbf-9f49534b-a83f177e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9d53044b-9ae8495c-6b74afda-ac416e48_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:29.892750Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d53044b-9ae8495c-6b74afda-ac416e48_0] Write session: try to update token
2025-10-09T21:51:29.892765Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d53044b-9ae8495c-6b74afda-ac416e48_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:29.901198Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:29.901212Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:29.901220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342969576090583:11593]
2025-10-09T21:51:29.901256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:29.901277Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:29.901430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:29.901561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:29.901696Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:29.901827Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:29.901991Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:29.902024Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:29.905622Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2158 sessionId: src_id|9d53044b-9ae8495c-6b74afda-ac416e48_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:29.907916Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:29.908108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:29.908125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:29.908166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:29.908236Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:29.913077Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:29.913088Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:29.913090Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:29.913091Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:29.913099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342969576090583:11593]
2025-10-09T21:51:29.913102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342969576090583:11593]
2025-10-09T21:51:29.913105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342969576090583:11593]
2025-10-09T21:51:29.913134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:29.913145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:29.913147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:29.913152Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:29.913157Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:29.913161Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:29.913263Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:29.967300Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342969937931031:9464]
2025-10-09T21:51:29.967541Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 2157, pipe: [1:7559342969937931031:9464]
2025-10-09T21:51:30.054450Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:30.061101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.061122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.061272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.061324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.061397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.061469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.061513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.061520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.061583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.061643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.061702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.061765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.061808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.061817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.061870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.061922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.061978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.062051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.062089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.062095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.062185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.062250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.062306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.062361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.062399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.062404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.062462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.062521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.062577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.062634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.062677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.062682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.062754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.062808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.062861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.062914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.062952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.062957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.063017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.063069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.063121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.063175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.063213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.063219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.063278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.063328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.063380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.063434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.063473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.063478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.063534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.063591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.063645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.063698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.063745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.063752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.063806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.063856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.063910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.063966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.064018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.064025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.064067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342969937931031:9464], now have 1 active actors on pipe
2025-10-09T21:51:30.064300Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:30.064329Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342969937931031:9464]
2025-10-09T21:51:30.064358Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:30.064361Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:30.064393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2158 reason: error at write: Disconnected sessionId: src_id|9d53044b-9ae8495c-6b74afda-ac416e48_0
2025-10-09T21:51:30.064570Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2158 sessionId: src_id|9d53044b-9ae8495c-6b74afda-ac416e48_0 is DEAD
2025-10-09T21:51:30.064789Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d53044b-9ae8495c-6b74afda-ac416e48_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:30.064799Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d53044b-9ae8495c-6b74afda-ac416e48_0] Write session will restart in 0.008206s
2025-10-09T21:51:30.064796Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:30.064824Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d53044b-9ae8495c-6b74afda-ac416e48_0] Write session: Do CDS request
2025-10-09T21:51:30.064831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d53044b-9ae8495c-6b74afda-ac416e48_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:30.074464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d53044b-9ae8495c-6b74afda-ac416e48_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:30.074604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:30.074618Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2159
2025-10-09T21:51:30.074774Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2159 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:30.074809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2159 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:30.074817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2159 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:30.074821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:30.075131Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2159 sessionId:  describe result for acl check
2025-10-09T21:51:30.075156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:30.075161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:30.075162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:30.075166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342974232898619:9015] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:30.075597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342974232898619:9015] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:30.077765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:30.077788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342974232898619:9015] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:30.078952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:30.079145Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:30.079154Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:30.079165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:30.079223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:30.079260Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342973871058121:11595]
2025-10-09T21:51:30.079839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:30.080140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:30.080181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:30.080246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342974232898619:9015] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:30.080257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342974232898619:9015] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:30.080259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342974232898619:9015] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:30.080265Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2159 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:30.080505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:30.080554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.080561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.080562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.080564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.080565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.080566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.080568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.080569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.080570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.080575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.080585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.080589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.080593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.080597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.080601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.080605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.080609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.080613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.080619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.080627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.080633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.080638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.080643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.080648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.080652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.080657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.080662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.080668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.080672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.080677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.080681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.080686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.080690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.080695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.080700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.080705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.080733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:30.086829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:30.087917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:30.087928Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:30.087929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:30.087933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:30.087939Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2158 [2:7559342973871058121:11595]
2025-10-09T21:51:30.087947Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:30.087961Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:30.087963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:30.087965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:30.087966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:30.087983Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:30.087994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:30.088001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:30.088053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.088059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.088060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.088062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:30.088113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:30.088118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.088126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.088132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.088145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:30.088169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342974232898716:9015], now have 1 active actors on pipe
2025-10-09T21:51:30.088181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342973871058115:11595]
2025-10-09T21:51:30.088184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342973871058115:11595]
2025-10-09T21:51:30.088186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342973871058115:11595]
2025-10-09T21:51:30.088198Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:30.088212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342974232898330:2432], now have 1 active actors on pipe
2025-10-09T21:51:30.088213Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:30.088216Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:30.088218Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:30.088219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:30.088232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:30.088237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:30.088246Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2158
2025-10-09T21:51:30.088266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342973871058115:11595]
2025-10-09T21:51:30.088273Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:30.088277Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2158
2025-10-09T21:51:30.088282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:30.088287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.088291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.088338Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|90d75131-aa53857e-8a030b53-f3e8e5f5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:30.088348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:30.088351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:30.088353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:30.088361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:30.088369Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:30.088386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:30.088530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.088537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.088552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:30.088627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2159 partition: 0 MaxSeqNo: 25 sessionId: src_id|90d75131-aa53857e-8a030b53-f3e8e5f5_0
2025-10-09T21:51:30.088950Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d53044b-9ae8495c-6b74afda-ac416e48_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|90d75131-aa53857e-8a030b53-f3e8e5f5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:30.091239Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90d75131-aa53857e-8a030b53-f3e8e5f5_0] Write session: try to update token
2025-10-09T21:51:30.091255Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90d75131-aa53857e-8a030b53-f3e8e5f5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:30.102362Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:30.102379Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:30.102381Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:30.102383Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:30.102392Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342973871058115:11595]
2025-10-09T21:51:30.102397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342973871058115:11595]
2025-10-09T21:51:30.102401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342973871058115:11595]
2025-10-09T21:51:30.102444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:30.102444Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:30.102449Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:30.102450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.102452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:30.102454Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:30.102574Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:30.104150Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2159 sessionId: src_id|90d75131-aa53857e-8a030b53-f3e8e5f5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:30.106684Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:30.164715Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342974232898725:9464]
2025-10-09T21:51:30.164985Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 2158, pipe: [1:7559342974232898725:9464]
2025-10-09T21:51:30.249516Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:30.250000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.250013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.250067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342974232898725:9464], now have 1 active actors on pipe
2025-10-09T21:51:30.250274Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:30.250331Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342974232898725:9464]
2025-10-09T21:51:30.250351Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:30.250356Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:30.250359Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:30.250494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2159 reason: error at write: Disconnected sessionId: src_id|90d75131-aa53857e-8a030b53-f3e8e5f5_0
2025-10-09T21:51:30.250663Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2159 sessionId: src_id|90d75131-aa53857e-8a030b53-f3e8e5f5_0 is DEAD
2025-10-09T21:51:30.250822Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90d75131-aa53857e-8a030b53-f3e8e5f5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:30.250830Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90d75131-aa53857e-8a030b53-f3e8e5f5_0] Write session will restart in 0.008272s
2025-10-09T21:51:30.250858Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90d75131-aa53857e-8a030b53-f3e8e5f5_0] Write session: Do CDS request
2025-10-09T21:51:30.250864Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90d75131-aa53857e-8a030b53-f3e8e5f5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:30.260559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90d75131-aa53857e-8a030b53-f3e8e5f5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:30.260609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:30.260623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2160
2025-10-09T21:51:30.260766Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2160 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:30.260791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2160 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:30.260795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2160 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:30.260798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:30.261227Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2160 sessionId:  describe result for acl check
2025-10-09T21:51:30.261263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:30.261269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:30.261271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:30.261277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342974232899037:9003] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:30.261868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342974232899037:9003] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:30.263650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342974232899037:9003] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:30.263819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:30.265007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:30.265167Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:30.265178Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:30.265189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:30.265237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:30.265275Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342973871058360:11597]
2025-10-09T21:51:30.265492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342974232899037:9003] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:30.265505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342974232899037:9003] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:30.265507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342974232899037:9003] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:30.265512Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2160 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:30.265746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:30.266006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:30.266051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:30.266313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:30.266356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.266363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.266364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.266365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.266366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.266367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.266367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.266368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.266369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.266374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.266385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.266389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.266394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.266398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.266403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.266407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.266411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.266414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.266421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.266429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.266435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.266441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.266446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.266450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.266453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.266457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.266460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.266464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.266466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.266470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.266473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.266477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.266480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.266485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.266488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.266491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.266513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:30.272822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:30.273982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:30.273995Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:30.273997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:30.274000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:30.274006Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2159 [2:7559342973871058360:11597]
2025-10-09T21:51:30.274016Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:30.274046Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:30.274055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:30.274059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:30.274062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:30.274080Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:30.274087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:30.274095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:30.274149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.274154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.274156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.274158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:30.274214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:30.274219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.274229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.274236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.274251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:30.274270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342974232899116:9003], now have 1 active actors on pipe
2025-10-09T21:51:30.274276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342974232898731:2432], now have 1 active actors on pipe
2025-10-09T21:51:30.274282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342973871058355:11597]
2025-10-09T21:51:30.274284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342973871058355:11597]
2025-10-09T21:51:30.274285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342973871058355:11597]
2025-10-09T21:51:30.274296Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:30.274311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:30.274317Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:30.274321Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:30.274323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:30.274324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:30.274326Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:30.274325Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2159
2025-10-09T21:51:30.274326Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2159
2025-10-09T21:51:30.274362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342973871058355:11597]
2025-10-09T21:51:30.274381Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:30.274390Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:30.274402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.274411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.274468Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7479c5d-28a92d50-c5ad5264-6bccac09_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:30.274480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:30.274483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:30.274485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:30.274492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:30.274500Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:30.274517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:30.274628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.274635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.274645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:30.274699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2160 partition: 0 MaxSeqNo: 25 sessionId: src_id|7479c5d-28a92d50-c5ad5264-6bccac09_0
2025-10-09T21:51:30.275010Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90d75131-aa53857e-8a030b53-f3e8e5f5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7479c5d-28a92d50-c5ad5264-6bccac09_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:30.277447Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7479c5d-28a92d50-c5ad5264-6bccac09_0] Write session: try to update token
2025-10-09T21:51:30.277463Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7479c5d-28a92d50-c5ad5264-6bccac09_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:30.288615Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:30.288633Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:30.288636Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:30.288638Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:30.288648Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342973871058355:11597]
2025-10-09T21:51:30.288654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342973871058355:11597]
2025-10-09T21:51:30.288659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342973871058355:11597]
2025-10-09T21:51:30.288704Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:30.288708Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:30.288711Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:30.288715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:30.288722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.288725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:30.288845Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:30.289961Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2160 sessionId: src_id|7479c5d-28a92d50-c5ad5264-6bccac09_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:30.292082Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:30.350539Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342974232899133:9464]
2025-10-09T21:51:30.350769Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 2159, pipe: [1:7559342974232899133:9464]
2025-10-09T21:51:30.435095Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:30.435712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.435727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.435768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342974232899133:9464], now have 1 active actors on pipe
2025-10-09T21:51:30.435986Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:30.435991Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342974232899133:9464]
2025-10-09T21:51:30.435996Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:30.436028Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:30.436031Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:30.436090Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2160 reason: error at write: Disconnected sessionId: src_id|7479c5d-28a92d50-c5ad5264-6bccac09_0
2025-10-09T21:51:30.436267Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2160 sessionId: src_id|7479c5d-28a92d50-c5ad5264-6bccac09_0 is DEAD
2025-10-09T21:51:30.436436Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7479c5d-28a92d50-c5ad5264-6bccac09_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:30.436448Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7479c5d-28a92d50-c5ad5264-6bccac09_0] Write session will restart in 0.009834s
2025-10-09T21:51:30.436476Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7479c5d-28a92d50-c5ad5264-6bccac09_0] Write session: Do CDS request
2025-10-09T21:51:30.436497Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7479c5d-28a92d50-c5ad5264-6bccac09_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:30.447513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7479c5d-28a92d50-c5ad5264-6bccac09_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:30.447542Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:30.447554Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2161
2025-10-09T21:51:30.447720Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2161 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:30.447752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2161 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:30.447760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2161 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:30.447763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:30.448106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2161 sessionId:  describe result for acl check
2025-10-09T21:51:30.448133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:30.448137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:30.448139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:30.448143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342974232899472:8999] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:30.448586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342974232899472:8999] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:30.449642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:30.450799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342974232899472:8999] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:30.450874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:30.451071Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:30.451080Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:30.451091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:30.451141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:30.451181Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342973871058600:11599]
2025-10-09T21:51:30.451632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:30.451885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:30.451913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:30.452116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:30.452147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.452152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.452152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.452153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.452154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.452154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.452155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.452156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.452156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.452159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.452165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.452168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.452170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.452173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.452177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.452181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.452185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.452188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.452194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.452202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.452208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.452214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.452217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.452220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.452222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.452225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.452227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.452230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.452234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.452239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.452244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.452248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.452252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.452256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.452260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.452266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.452292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:30.454069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342974232899472:8999] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:30.454080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342974232899472:8999] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:30.454093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342974232899472:8999] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:30.454100Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2161 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:30.458489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:30.459702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:30.459714Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:30.459717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:30.459721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:30.459726Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2160 [2:7559342973871058600:11599]
2025-10-09T21:51:30.459738Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:30.459755Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:30.459762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:30.459765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:30.459768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:30.459784Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:30.459794Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:30.459802Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:30.459850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.459857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.459859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.459861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:30.459922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:30.459928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.459940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.459953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.459975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:30.459999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342974232899525:8999], now have 1 active actors on pipe
2025-10-09T21:51:30.460006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342974232899135:2432], now have 1 active actors on pipe
2025-10-09T21:51:30.460013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342973871058595:11599]
2025-10-09T21:51:30.460001Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2160
2025-10-09T21:51:30.460018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342973871058595:11599]
2025-10-09T21:51:30.460006Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2160
2025-10-09T21:51:30.460021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342973871058595:11599]
2025-10-09T21:51:30.460029Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:30.460043Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:30.460049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:30.460052Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:30.460054Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:30.460058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:30.460061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:30.460098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.460106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.460123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342973871058595:11599]
2025-10-09T21:51:30.460135Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:30.460150Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:30.460164Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ac253530-d02a528a-1bba75eb-272c273_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:30.460171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:30.460175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:30.460178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:30.460189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:30.460202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:30.460228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:30.460338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.460346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.460360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:30.460434Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2161 partition: 0 MaxSeqNo: 25 sessionId: src_id|ac253530-d02a528a-1bba75eb-272c273_0
2025-10-09T21:51:30.460723Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7479c5d-28a92d50-c5ad5264-6bccac09_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ac253530-d02a528a-1bba75eb-272c273_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:30.463345Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac253530-d02a528a-1bba75eb-272c273_0] Write session: try to update token
2025-10-09T21:51:30.463361Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac253530-d02a528a-1bba75eb-272c273_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:30.475329Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:30.475345Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:30.475348Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:30.475350Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:30.475359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342973871058595:11599]
2025-10-09T21:51:30.475364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342973871058595:11599]
2025-10-09T21:51:30.475368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342973871058595:11599]
2025-10-09T21:51:30.475412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:30.475419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.475422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:30.475424Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:30.475430Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:30.475434Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:30.475561Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:30.476717Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2161 sessionId: src_id|ac253530-d02a528a-1bba75eb-272c273_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:30.478893Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:30.536441Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342974232899533:9464]
2025-10-09T21:51:30.536707Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 2160, pipe: [1:7559342974232899533:9464]
2025-10-09T21:51:30.622439Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:30.622669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.622680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.622719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342974232899533:9464], now have 1 active actors on pipe
2025-10-09T21:51:30.622956Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342974232899533:9464]
2025-10-09T21:51:30.622968Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:30.622972Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:30.622984Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:30.623048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2161 reason: error at write: Disconnected sessionId: src_id|ac253530-d02a528a-1bba75eb-272c273_0
2025-10-09T21:51:30.623074Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:30.623243Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2161 sessionId: src_id|ac253530-d02a528a-1bba75eb-272c273_0 is DEAD
2025-10-09T21:51:30.623396Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac253530-d02a528a-1bba75eb-272c273_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:30.623408Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac253530-d02a528a-1bba75eb-272c273_0] Write session will restart in 0.005556s
2025-10-09T21:51:30.623437Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac253530-d02a528a-1bba75eb-272c273_0] Write session: Do CDS request
2025-10-09T21:51:30.623444Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac253530-d02a528a-1bba75eb-272c273_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:30.630588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ac253530-d02a528a-1bba75eb-272c273_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:30.630594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:30.630613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2162
2025-10-09T21:51:30.630783Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2162 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:30.630817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2162 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:30.630823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2162 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:30.630834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:30.631273Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2162 sessionId:  describe result for acl check
2025-10-09T21:51:30.631313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:30.631321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:30.631323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:30.631329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342974232899641:8994] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:30.631947Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342974232899641:8994] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:30.634326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342974232899641:8994] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:30.636769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342974232899641:8994] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:30.636781Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342974232899641:8994] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:30.636784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342974232899641:8994] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:30.636790Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2162 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:30.637601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:30.638817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:30.638995Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:30.639006Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:30.639021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:30.639069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:30.639109Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342973871058842:11601]
2025-10-09T21:51:30.639681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:30.639913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:30.639962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:30.640259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:30.640302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.640308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.640310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.640311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.640312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.640313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.640314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.640315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.640317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.640321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.640332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.640337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.640341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.640345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.640349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.640353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.640356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.640360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.640367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.640374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.640380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.640386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.640391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.640396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.640401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.640406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.640410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.640428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.640436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.640441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.640447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.640452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.640461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.640467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.640471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.640476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.640509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:30.646585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:30.647594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:30.647606Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:30.647608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:30.647612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:30.647619Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2161 [2:7559342973871058842:11601]
2025-10-09T21:51:30.647737Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:30.647758Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:30.647761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:30.647763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:30.647766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:30.647781Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:30.647791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:30.647797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:30.647847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.647853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.647855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.647858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:30.647920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:30.647926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.647938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.647952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.647974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:30.647998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342974232899535:2432], now have 1 active actors on pipe
2025-10-09T21:51:30.648005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342974232899920:8994], now have 1 active actors on pipe
2025-10-09T21:51:30.648012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342973871058837:11601]
2025-10-09T21:51:30.648014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342973871058837:11601]
2025-10-09T21:51:30.648016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342973871058837:11601]
2025-10-09T21:51:30.648029Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:30.648047Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:30.648033Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2161
2025-10-09T21:51:30.648049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:30.648052Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:30.648054Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:30.648065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:30.648068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:30.648045Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2161
2025-10-09T21:51:30.648104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.648113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.648137Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342973871058837:11601]
2025-10-09T21:51:30.648149Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:30.648166Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:30.648179Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e8065b0-ebc74553-1b6eeacc-374010ce_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:30.648186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:30.648193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:30.648196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:30.648205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:30.648218Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:30.648240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:30.648378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.648385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.648398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:30.648474Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2162 partition: 0 MaxSeqNo: 25 sessionId: src_id|e8065b0-ebc74553-1b6eeacc-374010ce_0
2025-10-09T21:51:30.648814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ac253530-d02a528a-1bba75eb-272c273_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e8065b0-ebc74553-1b6eeacc-374010ce_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:30.651076Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8065b0-ebc74553-1b6eeacc-374010ce_0] Write session: try to update token
2025-10-09T21:51:30.651092Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8065b0-ebc74553-1b6eeacc-374010ce_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:30.661964Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:30.661980Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:30.661981Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:30.661983Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:30.661992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342973871058837:11601]
2025-10-09T21:51:30.661995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342973871058837:11601]
2025-10-09T21:51:30.661998Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342973871058837:11601]
2025-10-09T21:51:30.662050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:30.662058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.662059Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:30.662062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:30.662064Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:30.662068Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:30.662183Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:30.662784Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2162 sessionId: src_id|e8065b0-ebc74553-1b6eeacc-374010ce_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:30.664965Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:30.723309Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342974232899947:9464]
2025-10-09T21:51:30.723542Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 2161, pipe: [1:7559342974232899947:9464]
2025-10-09T21:51:30.802808Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:30.803035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.803050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.803098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342974232899947:9464], now have 1 active actors on pipe
2025-10-09T21:51:30.803363Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:30.803377Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:30.803380Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:30.803437Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342974232899947:9464]
2025-10-09T21:51:30.803440Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:30.803448Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2162 reason: error at write: Disconnected sessionId: src_id|e8065b0-ebc74553-1b6eeacc-374010ce_0
2025-10-09T21:51:30.803621Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2162 sessionId: src_id|e8065b0-ebc74553-1b6eeacc-374010ce_0 is DEAD
2025-10-09T21:51:30.803772Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8065b0-ebc74553-1b6eeacc-374010ce_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:30.803783Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8065b0-ebc74553-1b6eeacc-374010ce_0] Write session will restart in 0.008218s
2025-10-09T21:51:30.803803Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8065b0-ebc74553-1b6eeacc-374010ce_0] Write session: Do CDS request
2025-10-09T21:51:30.803808Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8065b0-ebc74553-1b6eeacc-374010ce_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:30.813465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8065b0-ebc74553-1b6eeacc-374010ce_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:30.813513Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:30.813528Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2163
2025-10-09T21:51:30.813658Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2163 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:30.813691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2163 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:30.813699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2163 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:30.813702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:30.814115Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2163 sessionId:  describe result for acl check
2025-10-09T21:51:30.814150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:30.814156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:30.814158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:30.814164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342974232900224:8981] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:30.814633Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342974232900224:8981] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:30.817122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342974232900224:8981] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:30.817326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:30.818606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:30.818741Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:30.818748Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:30.818755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:30.818787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:30.818825Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342973871059125:11617]
2025-10-09T21:51:30.819002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342974232900224:8981] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:30.819024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342974232900224:8981] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:30.819027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342974232900224:8981] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:30.819034Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2163 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:30.819187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:30.819422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:30.819449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:30.819653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:30.819688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.819692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.819693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.819694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.819694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.819695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.819695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.819696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.819697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.819700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.819706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.819708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.819710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.819712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.819714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.819716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.819718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.819720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.819724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.819729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:30.819734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.819737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:30.819739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.819742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:30.819745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.819747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:30.819750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.819753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:30.819756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.819759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:30.819762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.819765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.819767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.819770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.819773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.819776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.819791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:30.825679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:30.826708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:30.826720Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:30.826722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:30.826726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:30.826732Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2162 [2:7559342973871059125:11617]
2025-10-09T21:51:30.826744Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:30.826758Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:30.826763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:30.826765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:30.826767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:30.826777Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:30.826781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:30.826786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:30.826822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.826827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.826829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.826831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:30.826876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:30.826881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.826889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:30.826895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:30.826904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:30.826916Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342973871059120:11617]
2025-10-09T21:51:30.826920Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342973871059120:11617]
2025-10-09T21:51:30.826921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342973871059120:11617]
2025-10-09T21:51:30.826927Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:30.826937Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:30.826937Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:30.826940Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:30.826942Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:30.826946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:30.826948Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:30.826956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342974232899958:2432], now have 1 active actors on pipe
2025-10-09T21:51:30.826959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342974232900346:8981], now have 1 active actors on pipe
2025-10-09T21:51:30.826972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342973871059120:11617]
2025-10-09T21:51:30.826983Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:30.826999Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:30.827008Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2162
2025-10-09T21:51:30.827013Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2162
2025-10-09T21:51:30.827093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.827101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.827122Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4d8e77db-646639c8-def676de-f2d6cbc0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:30.827131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:30.827134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:30.827135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:30.827142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:30.827150Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:30.827162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:30.827309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.827314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.827323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:30.827415Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2163 partition: 0 MaxSeqNo: 25 sessionId: src_id|4d8e77db-646639c8-def676de-f2d6cbc0_0
2025-10-09T21:51:30.827697Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8065b0-ebc74553-1b6eeacc-374010ce_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4d8e77db-646639c8-def676de-f2d6cbc0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:30.830008Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4d8e77db-646639c8-def676de-f2d6cbc0_0] Write session: try to update token
2025-10-09T21:51:30.830019Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4d8e77db-646639c8-def676de-f2d6cbc0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:30.839556Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:30.839578Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:30.839592Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342973871059120:11617]
2025-10-09T21:51:30.839641Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:30.839651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:30.840036Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:30.840317Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:30.840586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:30.840880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:30.841235Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:30.841297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:30.842586Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2163 sessionId: src_id|4d8e77db-646639c8-def676de-f2d6cbc0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:30.844714Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:30.844950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.844972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.845002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:30.845056Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:30.850976Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:30.850987Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:30.850989Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:30.850990Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:30.850998Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342973871059120:11617]
2025-10-09T21:51:30.851001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342973871059120:11617]
2025-10-09T21:51:30.851003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342973871059120:11617]
2025-10-09T21:51:30.851032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:30.851037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:30.851039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:30.851049Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:30.851053Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:30.851057Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:30.851162Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:30.903520Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342974232900360:9464]
2025-10-09T21:51:30.903772Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 2162, pipe: [1:7559342974232900360:9464]
2025-10-09T21:51:30.987989Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:30.995340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:30.995367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:30.995528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.995585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.995642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.995706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.995749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.995757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.995821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.995876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.995937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.996003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.996045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.996052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.996111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.996164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.996216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.996274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.996318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.996325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.996385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.996438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.996490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.996542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.996583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.996589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.996651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.996704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.996766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.996841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.996881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.996890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.996953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.997009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.997063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.997118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.997158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.997167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.997224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.997276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.997330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.997397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.997458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.997466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.997527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.997583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.997635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.997693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.997736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.997743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.997797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.997852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.997907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.997963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.998005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.998012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.998086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:30.998140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:30.998195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:30.998254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:30.998292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:30.998298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:30.998371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342974232900360:9464], now have 1 active actors on pipe
2025-10-09T21:51:30.998536Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:30.998545Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342974232900360:9464]
2025-10-09T21:51:30.998575Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:30.998578Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:30.998606Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2163 reason: error at write: Disconnected sessionId: src_id|4d8e77db-646639c8-def676de-f2d6cbc0_0
2025-10-09T21:51:30.998767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2163 sessionId: src_id|4d8e77db-646639c8-def676de-f2d6cbc0_0 is DEAD
2025-10-09T21:51:30.998780Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:30.998943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d8e77db-646639c8-def676de-f2d6cbc0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:30.998952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d8e77db-646639c8-def676de-f2d6cbc0_0] Write session will restart in 0.009145s
2025-10-09T21:51:30.998975Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d8e77db-646639c8-def676de-f2d6cbc0_0] Write session: Do CDS request
2025-10-09T21:51:30.998981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d8e77db-646639c8-def676de-f2d6cbc0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:31.009541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4d8e77db-646639c8-def676de-f2d6cbc0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:31.009655Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:31.009671Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2164
2025-10-09T21:51:31.009804Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2164 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:31.009838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2164 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:31.009846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2164 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:31.009849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:31.010241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2164 sessionId:  describe result for acl check
2025-10-09T21:51:31.010275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:31.010278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:31.010279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:31.010283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342978527867938:8972] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:31.010714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342978527867938:8972] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:31.013205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:31.013460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342978527867938:8972] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:31.014957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:31.015183Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:31.015193Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:31.015205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:31.015261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:31.015300Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342978166026665:11619]
2025-10-09T21:51:31.015679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342978527867938:8972] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:31.015695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342978527867938:8972] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:31.015699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342978527867938:8972] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:31.015708Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2164 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:31.015760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:31.016015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:31.016055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:31.016350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:31.016401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.016407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.016408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.016410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.016411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.016412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.016414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.016415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.016417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.016422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.016432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.016437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.016441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.016445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.016455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.016459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.016463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.016467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.016478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.016490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.016501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.016511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.016520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.016526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.016535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.016540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.016545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.016555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.016559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.016568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.016578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.016587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.016595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.016601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.016609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.016619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.016655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:31.023198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:31.024832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:31.024847Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:31.024850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:31.024854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:31.024861Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2163 [2:7559342978166026665:11619]
2025-10-09T21:51:31.024875Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:31.024898Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:31.024906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:31.024909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:31.024911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:31.024927Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:31.024931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:31.024937Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:31.024994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.025001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.025003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.025006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:31.025077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:31.025084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.025097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.025118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.025136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:31.025157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342974232900364:2432], now have 1 active actors on pipe
2025-10-09T21:51:31.025165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342978527868047:8972], now have 1 active actors on pipe
2025-10-09T21:51:31.025172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342978166026660:11619]
2025-10-09T21:51:31.025179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342978166026660:11619]
2025-10-09T21:51:31.025182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342978166026660:11619]
2025-10-09T21:51:31.025196Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:31.025210Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:31.025215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:31.025219Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:31.025221Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:31.025229Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:31.025232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:31.025204Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2163
2025-10-09T21:51:31.025222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2163
2025-10-09T21:51:31.025271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342978166026660:11619]
2025-10-09T21:51:31.025282Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:31.025289Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:31.025297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:31.025306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:31.025365Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c59248a4-595acd7e-fdd22479-60b78d3f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:31.025377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:31.025381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:31.025383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:31.025394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:31.025408Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:31.025428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:31.025579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:31.025587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:31.025601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:31.025667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2164 partition: 0 MaxSeqNo: 25 sessionId: src_id|c59248a4-595acd7e-fdd22479-60b78d3f_0
2025-10-09T21:51:31.026019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d8e77db-646639c8-def676de-f2d6cbc0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c59248a4-595acd7e-fdd22479-60b78d3f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:31.028556Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c59248a4-595acd7e-fdd22479-60b78d3f_0] Write session: try to update token
2025-10-09T21:51:31.028576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c59248a4-595acd7e-fdd22479-60b78d3f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:31.039371Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:31.039388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:31.039391Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:31.039393Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:31.039402Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342978166026660:11619]
2025-10-09T21:51:31.039407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342978166026660:11619]
2025-10-09T21:51:31.039411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342978166026660:11619]
2025-10-09T21:51:31.039453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:31.039465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.039468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:31.039467Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:31.039472Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:31.039478Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:31.039609Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:31.042412Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2164 sessionId: src_id|c59248a4-595acd7e-fdd22479-60b78d3f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:31.044843Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:31.098616Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342978527868058:9464]
2025-10-09T21:51:31.098859Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 2163, pipe: [1:7559342978527868058:9464]
2025-10-09T21:51:31.185984Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:31.186233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:31.186246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:31.186282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342978527868058:9464], now have 1 active actors on pipe
2025-10-09T21:51:31.186459Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:31.186484Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:31.186501Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:31.186520Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342978527868058:9464]
2025-10-09T21:51:31.186552Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:31.186576Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2164 reason: error at write: Disconnected sessionId: src_id|c59248a4-595acd7e-fdd22479-60b78d3f_0
2025-10-09T21:51:31.186761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2164 sessionId: src_id|c59248a4-595acd7e-fdd22479-60b78d3f_0 is DEAD
2025-10-09T21:51:31.186934Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c59248a4-595acd7e-fdd22479-60b78d3f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:31.186943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c59248a4-595acd7e-fdd22479-60b78d3f_0] Write session will restart in 0.005877s
2025-10-09T21:51:31.186970Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c59248a4-595acd7e-fdd22479-60b78d3f_0] Write session: Do CDS request
2025-10-09T21:51:31.186976Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c59248a4-595acd7e-fdd22479-60b78d3f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:31.194504Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c59248a4-595acd7e-fdd22479-60b78d3f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:31.194590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:31.194607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2165
2025-10-09T21:51:31.194731Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2165 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:31.194756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2165 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:31.194764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2165 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:31.194767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:31.195201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2165 sessionId:  describe result for acl check
2025-10-09T21:51:31.195241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:31.195243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:31.195245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:31.195250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342978527868168:8967] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:31.195848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342978527868168:8967] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:31.197859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342978527868168:8967] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:31.200597Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342978527868168:8967] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:31.200610Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342978527868168:8967] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:31.200613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342978527868168:8967] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:31.200620Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2165 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:31.203523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:31.203995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:31.204184Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:31.204196Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:31.204207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:31.204256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:31.204306Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342978166026911:11621]
2025-10-09T21:51:31.204955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:31.205234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:31.205286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:31.205649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:31.205710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.205718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.205720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.205721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.205723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.205724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.205726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.205727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.205729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.205734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.205746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.205751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.205756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.205760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.205765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.205768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.205772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.205775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.205782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.205790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.205796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.205802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.205807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.205813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.205817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.205824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.205829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.205835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.205840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.205845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.205853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.205859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.205868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.205874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.205880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.205889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.205920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:31.212807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:31.214506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:31.214523Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:31.214526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:31.214531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:31.214538Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2164 [2:7559342978166026911:11621]
2025-10-09T21:51:31.214553Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:31.214574Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:31.214582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:31.214585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:31.214588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:31.214607Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:31.214619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:31.214633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:31.214701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.214708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.214710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.214713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:31.214788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:31.214796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.214811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.214829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.214848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:31.214874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342978527868063:2432], now have 1 active actors on pipe
2025-10-09T21:51:31.214882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342978527868450:8967], now have 1 active actors on pipe
2025-10-09T21:51:31.214891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342978166026906:11621]
2025-10-09T21:51:31.214893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342978166026906:11621]
2025-10-09T21:51:31.214895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342978166026906:11621]
2025-10-09T21:51:31.214910Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:31.214924Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:31.214937Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:31.214944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:31.214947Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:31.214949Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:31.214949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:31.214991Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342978166026906:11621]
2025-10-09T21:51:31.214964Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2164
2025-10-09T21:51:31.215001Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:31.215009Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:31.215021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:31.215026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:31.215076Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a8e0ce30-a1d1cd6f-c5abf7b0-d8beb1da_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:31.215072Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2164
2025-10-09T21:51:31.215089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:31.215093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:31.215095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:31.215105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:31.215117Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:31.215144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:31.215281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:31.215290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:31.215304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:31.215405Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2165 partition: 0 MaxSeqNo: 25 sessionId: src_id|a8e0ce30-a1d1cd6f-c5abf7b0-d8beb1da_0
2025-10-09T21:51:31.215831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c59248a4-595acd7e-fdd22479-60b78d3f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a8e0ce30-a1d1cd6f-c5abf7b0-d8beb1da_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:31.218383Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a8e0ce30-a1d1cd6f-c5abf7b0-d8beb1da_0] Write session: try to update token
2025-10-09T21:51:31.218403Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a8e0ce30-a1d1cd6f-c5abf7b0-d8beb1da_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:31.229928Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:31.229946Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:31.229948Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:31.229950Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:31.229960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342978166026906:11621]
2025-10-09T21:51:31.229966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342978166026906:11621]
2025-10-09T21:51:31.229970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342978166026906:11621]
2025-10-09T21:51:31.230016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:31.230042Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:31.230047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.230049Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:31.230050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:31.230053Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:31.230194Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:31.232244Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2165 sessionId: src_id|a8e0ce30-a1d1cd6f-c5abf7b0-d8beb1da_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:31.234806Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:31.286537Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342978527868467:9464]
2025-10-09T21:51:31.286724Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 2164, pipe: [1:7559342978527868467:9464]
2025-10-09T21:51:31.376083Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:31.376349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:31.376365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:31.376414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342978527868467:9464], now have 1 active actors on pipe
2025-10-09T21:51:31.376604Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:31.376697Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:31.376712Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342978527868467:9464]
2025-10-09T21:51:31.376741Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:31.376743Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:31.376801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2165 reason: error at write: Disconnected sessionId: src_id|a8e0ce30-a1d1cd6f-c5abf7b0-d8beb1da_0
2025-10-09T21:51:31.376998Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2165 sessionId: src_id|a8e0ce30-a1d1cd6f-c5abf7b0-d8beb1da_0 is DEAD
2025-10-09T21:51:31.377153Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a8e0ce30-a1d1cd6f-c5abf7b0-d8beb1da_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:31.377167Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a8e0ce30-a1d1cd6f-c5abf7b0-d8beb1da_0] Write session will restart in 0.008332s
2025-10-09T21:51:31.377207Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a8e0ce30-a1d1cd6f-c5abf7b0-d8beb1da_0] Write session: Do CDS request
2025-10-09T21:51:31.377214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a8e0ce30-a1d1cd6f-c5abf7b0-d8beb1da_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:31.386560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a8e0ce30-a1d1cd6f-c5abf7b0-d8beb1da_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:31.386659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:31.386677Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2166
2025-10-09T21:51:31.386803Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2166 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:31.386841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2166 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:31.386853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2166 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:31.386857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:31.387323Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2166 sessionId:  describe result for acl check
2025-10-09T21:51:31.387365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:31.387371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:31.387372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:31.387378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342978527868628:8960] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:31.387975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342978527868628:8960] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:31.390772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342978527868628:8960] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:31.393700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342978527868628:8960] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:31.393715Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342978527868628:8960] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:31.393719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342978527868628:8960] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:31.393726Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2166 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:31.394762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:31.396269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:31.396444Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:31.396454Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:31.396465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:31.396528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:31.396566Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342978166027155:11623]
2025-10-09T21:51:31.397157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:31.397474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:31.397511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:31.397826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:31.397877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.397885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.397886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.397887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.397889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.397890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.397891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.397892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.397893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.397899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.397911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.397915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.397920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.397924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.397928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.397932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.397936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.397940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.397946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.397954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.397960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.397965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.397970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.397975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.397982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.397987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.397992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.397998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.398003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.398009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.398013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.398019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.398025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.398045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.398052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.398063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.398093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:31.404919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:31.406150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:31.406164Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:31.406167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:31.406172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:31.406181Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2165 [2:7559342978166027155:11623]
2025-10-09T21:51:31.406198Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:31.406218Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:31.406224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:31.406227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:31.406229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:31.406244Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:31.406252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:31.406258Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:31.406310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.406316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.406318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.406320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:31.406393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:31.406400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.406409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.406415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.406428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:31.406452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342978166027150:11623]
2025-10-09T21:51:31.406464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342978166027150:11623]
2025-10-09T21:51:31.406466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342978166027150:11623]
2025-10-09T21:51:31.406482Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:31.406499Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:31.406500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342978527868864:8960], now have 1 active actors on pipe
2025-10-09T21:51:31.406502Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:31.406503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342978527868475:2432], now have 1 active actors on pipe
2025-10-09T21:51:31.406506Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:31.406512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:31.406526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:31.406531Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:31.406572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342978166027150:11623]
2025-10-09T21:51:31.406582Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:31.406563Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2165
2025-10-09T21:51:31.406561Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2165
2025-10-09T21:51:31.406619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:31.406646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:31.406657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:31.406686Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ecb1a1ea-1de1e25d-dc02b40a-3c5e93cf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:31.406701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:31.406705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:31.406708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:31.406720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:31.406735Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:31.406756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:31.406911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:31.406921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:31.406938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:31.407044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2166 partition: 0 MaxSeqNo: 25 sessionId: src_id|ecb1a1ea-1de1e25d-dc02b40a-3c5e93cf_0
2025-10-09T21:51:31.407466Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a8e0ce30-a1d1cd6f-c5abf7b0-d8beb1da_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ecb1a1ea-1de1e25d-dc02b40a-3c5e93cf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:31.410088Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecb1a1ea-1de1e25d-dc02b40a-3c5e93cf_0] Write session: try to update token
2025-10-09T21:51:31.410108Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecb1a1ea-1de1e25d-dc02b40a-3c5e93cf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:31.424451Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2166 sessionId: src_id|ecb1a1ea-1de1e25d-dc02b40a-3c5e93cf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:31.424972Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:31.424991Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:31.425002Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342978166027150:11623]
2025-10-09T21:51:31.425050Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:31.425054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:31.425221Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:31.425397Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:31.425535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:31.425667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:31.425898Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:31.425942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:31.427105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:31.427294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:31.427325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:31.427363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:31.427438Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:31.432388Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:31.432405Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:31.432408Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:31.432410Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:31.432419Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342978166027150:11623]
2025-10-09T21:51:31.432423Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342978166027150:11623]
2025-10-09T21:51:31.432427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342978166027150:11623]
2025-10-09T21:51:31.432462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:31.432469Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:31.432474Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:31.432475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.432477Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:31.432478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:31.432609Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:31.476989Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342978527868877:9464]
2025-10-09T21:51:31.477257Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 2165, pipe: [1:7559342978527868877:9464]
2025-10-09T21:51:31.581583Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:31.587851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:31.587872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:31.588010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:31.588070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:31.588133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:31.588196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:31.588235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:31.588241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:31.588297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:31.588347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:31.588400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:31.588453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:31.588490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:31.588495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:31.588545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:31.588592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:31.588641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:31.588692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:31.588728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:31.588734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:31.588789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:31.588845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:31.588896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:31.588949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:31.588986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:31.588990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:31.589049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:31.589102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:31.589164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:31.589218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:31.589259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:31.589264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:31.589318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:31.589379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:31.589430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:31.589479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:31.589517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:31.589522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:31.589572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:31.589617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:31.589670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:31.589722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:31.589763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:31.589768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:31.589818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:31.589868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:31.589919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:31.589971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:31.590012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:31.590018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:31.590095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:31.590141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:31.590190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:31.590241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:31.590275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:31.590281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:31.590330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:31.590377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:31.590428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:31.590477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:31.590512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:31.590516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:31.590552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342978527868877:9464], now have 1 active actors on pipe
2025-10-09T21:51:31.590801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:31.590804Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342978527868877:9464]
2025-10-09T21:51:31.590816Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:31.590828Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:31.590848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2166 reason: error at write: Disconnected sessionId: src_id|ecb1a1ea-1de1e25d-dc02b40a-3c5e93cf_0
2025-10-09T21:51:31.591025Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2166 sessionId: src_id|ecb1a1ea-1de1e25d-dc02b40a-3c5e93cf_0 is DEAD
2025-10-09T21:51:31.591077Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:31.591135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecb1a1ea-1de1e25d-dc02b40a-3c5e93cf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:31.591146Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecb1a1ea-1de1e25d-dc02b40a-3c5e93cf_0] Write session will restart in 0.007798s
2025-10-09T21:51:31.591164Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecb1a1ea-1de1e25d-dc02b40a-3c5e93cf_0] Write session: Do CDS request
2025-10-09T21:51:31.591169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecb1a1ea-1de1e25d-dc02b40a-3c5e93cf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:31.600508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecb1a1ea-1de1e25d-dc02b40a-3c5e93cf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:31.600576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:31.600594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2167
2025-10-09T21:51:31.600693Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2167 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:31.600726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2167 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:31.600738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2167 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:31.600742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:31.601205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2167 sessionId:  describe result for acl check
2025-10-09T21:51:31.601242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:31.601245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:31.601247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:31.601251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342978527869051:8957] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:31.602113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342978527869051:8957] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:31.604654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342978527869051:8957] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:31.606971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:31.607092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342978527869051:8957] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:31.607107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342978527869051:8957] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:31.607111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342978527869051:8957] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:31.607118Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2167 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:31.608348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:31.608577Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:31.608588Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:31.608600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:31.608651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:31.608693Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342978166027402:11625]
2025-10-09T21:51:31.609261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:31.609500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:31.609539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:31.609845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:31.609887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.609893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.609894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.609896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.609897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.609898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.609899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.609900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.609901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.609906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.609916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.609920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.609924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.609927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.609931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.609934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.609941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.609945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.609956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.609964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.609973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.609979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.609986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.609995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.609999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.610008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.610013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.610018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.610025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.610047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.610056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.610062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.610070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.610076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.610084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.610089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.610121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:31.616065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:31.617112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:31.617126Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:31.617129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:31.617133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:31.617139Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2166 [2:7559342978166027402:11625]
2025-10-09T21:51:31.617153Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:31.617169Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:31.617176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:31.617180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:31.617182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:31.617197Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:31.617207Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:31.617214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:31.617266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.617271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.617273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.617274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:31.617339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:31.617346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.617368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.617384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.617406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:31.617430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342978527869271:8957], now have 1 active actors on pipe
2025-10-09T21:51:31.617439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342978527868880:2432], now have 1 active actors on pipe
2025-10-09T21:51:31.617447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342978166027396:11625]
2025-10-09T21:51:31.617449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342978166027396:11625]
2025-10-09T21:51:31.617451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342978166027396:11625]
2025-10-09T21:51:31.617452Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2166
2025-10-09T21:51:31.617459Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:31.617452Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2166
2025-10-09T21:51:31.617473Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:31.617472Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:31.617474Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:31.617476Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:31.617487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:31.617489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:31.617525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:31.617532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:31.617544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342978166027396:11625]
2025-10-09T21:51:31.617553Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:31.617559Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:31.617581Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fe76f469-35f8c882-59a969a-bbcfbf93_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:31.617588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:31.617591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:31.617593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:31.617601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:31.617611Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:31.617634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:31.617768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:31.617776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:31.617788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:31.617863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2167 partition: 0 MaxSeqNo: 25 sessionId: src_id|fe76f469-35f8c882-59a969a-bbcfbf93_0
2025-10-09T21:51:31.618214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecb1a1ea-1de1e25d-dc02b40a-3c5e93cf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fe76f469-35f8c882-59a969a-bbcfbf93_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:31.620597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe76f469-35f8c882-59a969a-bbcfbf93_0] Write session: try to update token
2025-10-09T21:51:31.620612Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe76f469-35f8c882-59a969a-bbcfbf93_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:31.631472Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:31.631489Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:31.631492Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:31.631494Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:31.631503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342978166027396:11625]
2025-10-09T21:51:31.631508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342978166027396:11625]
2025-10-09T21:51:31.631512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342978166027396:11625]
2025-10-09T21:51:31.631544Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:31.631547Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:31.631550Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:31.631552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:31.631559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.631561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:31.631705Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:31.633822Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2167 sessionId: src_id|fe76f469-35f8c882-59a969a-bbcfbf93_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:31.635936Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:31.691001Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342978527869284:9464]
2025-10-09T21:51:31.691267Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 2166, pipe: [1:7559342978527869284:9464]
2025-10-09T21:51:31.784571Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:31.785097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:31.785118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:31.785160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342978527869284:9464], now have 1 active actors on pipe
2025-10-09T21:51:31.785344Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:31.785338Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342978527869284:9464]
2025-10-09T21:51:31.785384Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:31.785386Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:31.785436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2167 reason: error at write: Disconnected sessionId: src_id|fe76f469-35f8c882-59a969a-bbcfbf93_0
2025-10-09T21:51:31.785467Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:31.785622Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2167 sessionId: src_id|fe76f469-35f8c882-59a969a-bbcfbf93_0 is DEAD
2025-10-09T21:51:31.785783Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe76f469-35f8c882-59a969a-bbcfbf93_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:31.785794Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe76f469-35f8c882-59a969a-bbcfbf93_0] Write session will restart in 0.006425s
2025-10-09T21:51:31.785822Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe76f469-35f8c882-59a969a-bbcfbf93_0] Write session: Do CDS request
2025-10-09T21:51:31.785828Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe76f469-35f8c882-59a969a-bbcfbf93_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:31.793587Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe76f469-35f8c882-59a969a-bbcfbf93_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:31.794015Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:31.794050Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2168
2025-10-09T21:51:31.794190Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2168 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:31.794237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2168 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:31.794249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2168 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:31.794253Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:31.794894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2168 sessionId:  describe result for acl check
2025-10-09T21:51:31.794935Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:31.794943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:31.794945Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:31.794952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342978527869367:8936] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:31.795545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342978527869367:8936] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:31.797671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342978527869367:8936] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:31.800577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342978527869367:8936] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:31.800594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342978527869367:8936] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:31.800598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342978527869367:8936] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:31.800607Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2168 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:31.806850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:31.808608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:31.808831Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:31.808840Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:31.808852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:31.808908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:31.808954Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342978166027691:11639]
2025-10-09T21:51:31.809571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:31.809908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:31.809978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:31.810329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:31.810400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.810406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.810407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.810409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.810410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.810412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.810413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.810414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.810415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.810421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.810433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.810437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.810441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.810445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.810449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.810453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.810457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.810460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.810468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.810476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:31.810482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.810490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:31.810495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.810500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:31.810509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.810514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:31.810518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.810522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:31.810526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.810530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:31.810535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.810541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.810545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.810550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.810556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.810561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.810590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:31.817059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:31.818827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:31.818846Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:31.818849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:31.818853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:31.818861Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2167 [2:7559342978166027691:11639]
2025-10-09T21:51:31.818877Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:31.818898Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:31.818906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:31.818910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:31.818912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:31.818931Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:31.818942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:31.818950Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:31.819020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.819029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.819031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.819034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:31.819112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:31.819121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.819134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:31.819151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:31.819174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:31.819201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342978527869581:8936], now have 1 active actors on pipe
2025-10-09T21:51:31.819210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342978527869313:2432], now have 1 active actors on pipe
2025-10-09T21:51:31.819219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342978166027686:11639]
2025-10-09T21:51:31.819225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342978166027686:11639]
2025-10-09T21:51:31.819228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342978166027686:11639]
2025-10-09T21:51:31.819248Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:31.819272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:31.819271Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:31.819276Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:31.819279Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:31.819288Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:31.819293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:31.819263Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2167
2025-10-09T21:51:31.819334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342978166027686:11639]
2025-10-09T21:51:31.819269Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2167
2025-10-09T21:51:31.819346Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:31.819371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:31.819372Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:31.819377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:31.819432Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|70e3c2fb-1249218a-28350cf6-9485bd86_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:31.819446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:31.819451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:31.819458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:31.819473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:31.819484Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:31.819509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:31.819689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:31.819698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:31.819716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:31.819850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2168 partition: 0 MaxSeqNo: 25 sessionId: src_id|70e3c2fb-1249218a-28350cf6-9485bd86_0
2025-10-09T21:51:31.820213Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe76f469-35f8c882-59a969a-bbcfbf93_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|70e3c2fb-1249218a-28350cf6-9485bd86_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:31.822851Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70e3c2fb-1249218a-28350cf6-9485bd86_0] Write session: try to update token
2025-10-09T21:51:31.822871Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70e3c2fb-1249218a-28350cf6-9485bd86_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:31.834263Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:31.834287Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:31.834290Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:31.834292Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:31.834302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342978166027686:11639]
2025-10-09T21:51:31.834307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342978166027686:11639]
2025-10-09T21:51:31.834311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342978166027686:11639]
2025-10-09T21:51:31.834362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:31.834370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:31.834372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:31.834381Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:31.834391Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:31.834395Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:31.834507Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:31.838110Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2168 sessionId: src_id|70e3c2fb-1249218a-28350cf6-9485bd86_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:31.840890Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:31.885568Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342978527869716:9464]
2025-10-09T21:51:31.885845Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 2167, pipe: [1:7559342978527869716:9464]
2025-10-09T21:51:31.985129Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:31.985370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:31.985386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:31.985429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342978527869716:9464], now have 1 active actors on pipe
2025-10-09T21:51:31.985656Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:31.985675Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342978527869716:9464]
2025-10-09T21:51:31.985702Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:31.985704Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:31.985704Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:31.985776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2168 reason: error at write: Disconnected sessionId: src_id|70e3c2fb-1249218a-28350cf6-9485bd86_0
2025-10-09T21:51:31.985957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2168 sessionId: src_id|70e3c2fb-1249218a-28350cf6-9485bd86_0 is DEAD
2025-10-09T21:51:31.986105Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70e3c2fb-1249218a-28350cf6-9485bd86_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:31.986129Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70e3c2fb-1249218a-28350cf6-9485bd86_0] Write session will restart in 0.008714s
2025-10-09T21:51:31.986156Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70e3c2fb-1249218a-28350cf6-9485bd86_0] Write session: Do CDS request
2025-10-09T21:51:31.986163Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70e3c2fb-1249218a-28350cf6-9485bd86_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:31.996597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70e3c2fb-1249218a-28350cf6-9485bd86_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:31.996662Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:31.996682Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2169
2025-10-09T21:51:31.996858Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2169 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:31.996901Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2169 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:31.996911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2169 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:31.996915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:31.997411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2169 sessionId:  describe result for acl check
2025-10-09T21:51:31.997455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:31.997462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:31.997464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:31.997470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342978527869951:8931] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:31.998018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342978527869951:8931] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:32.000286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342978527869951:8931] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:32.003187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342978527869951:8931] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:32.003205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342978527869951:8931] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:32.003209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342978527869951:8931] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:32.003219Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2169 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:32.003736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:32.005342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:32.005703Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:32.005715Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:32.005731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:32.005804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:32.005838Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342982460995235:11643]
2025-10-09T21:51:32.006375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:32.006710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:32.006766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:32.007123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:32.007178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.007184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.007184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.007185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.007186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.007200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.007201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.007202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.007203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.007209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.007220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.007224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.007227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.007230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.007234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.007237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.007240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.007244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.007252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.007262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.007273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.007278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.007282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.007287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.007296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.007301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.007306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.007311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.007315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.007320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.007325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.007330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.007335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.007340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.007345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.007350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.007370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:32.014175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:32.015228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:32.015244Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:32.015246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:32.015250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:32.015257Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2168 [2:7559342982460995235:11643]
2025-10-09T21:51:32.015268Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:32.015285Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:32.015291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:32.015293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:32.015295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:32.015312Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:32.015324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:32.015337Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:32.015393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.015398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.015400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.015402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:32.015463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:32.015469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.015477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.015484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.015494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:32.015513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342982460995230:11643]
2025-10-09T21:51:32.015518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342982460995230:11643]
2025-10-09T21:51:32.015520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342982460995230:11643]
2025-10-09T21:51:32.015527Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:32.015536Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:32.015539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342982822837409:8931], now have 1 active actors on pipe
2025-10-09T21:51:32.015546Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:32.015546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:32.015548Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:32.015555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:32.015557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:32.015561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342978527869722:2432], now have 1 active actors on pipe
2025-10-09T21:51:32.015591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342982460995230:11643]
2025-10-09T21:51:32.015602Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:32.015607Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:32.015632Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2168
2025-10-09T21:51:32.015638Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2168
2025-10-09T21:51:32.015765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.015778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.015810Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|de635304-9697359a-d6b06d42-3ac048ac_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:32.015825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:32.015829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:32.015837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:32.015849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:32.015858Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:32.015872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:32.016044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.016053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.016067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:32.016200Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2169 partition: 0 MaxSeqNo: 25 sessionId: src_id|de635304-9697359a-d6b06d42-3ac048ac_0
2025-10-09T21:51:32.016591Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70e3c2fb-1249218a-28350cf6-9485bd86_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|de635304-9697359a-d6b06d42-3ac048ac_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:32.019161Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|de635304-9697359a-d6b06d42-3ac048ac_0] Write session: try to update token
2025-10-09T21:51:32.019182Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|de635304-9697359a-d6b06d42-3ac048ac_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:32.027934Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:32.027951Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:32.027961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342982460995230:11643]
2025-10-09T21:51:32.028012Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:32.028017Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:32.028183Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:32.028340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:32.028539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:32.028800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:32.029150Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:32.029220Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:32.032979Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2169 sessionId: src_id|de635304-9697359a-d6b06d42-3ac048ac_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:32.035378Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:32.035582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.035604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.035647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:32.035732Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:32.040982Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:32.041005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:32.041007Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:32.041010Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:32.041022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342982460995230:11643]
2025-10-09T21:51:32.041027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342982460995230:11643]
2025-10-09T21:51:32.041033Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342982460995230:11643]
2025-10-09T21:51:32.041086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:32.041094Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:32.041101Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:32.041103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.041106Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:32.041106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:32.041335Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:32.085775Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342982822837422:9464]
2025-10-09T21:51:32.086191Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 2168, pipe: [1:7559342982822837422:9464]
2025-10-09T21:51:32.193770Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:32.202836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.202871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.203040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.203116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.203189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.203262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.203324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.203334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.203424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.203498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.203572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.203643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.203695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.203703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.203773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.203838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.203906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.203971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.204023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.204031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.204105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.204174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.204246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.204316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.204372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.204380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.204453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.204522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.204589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.204658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.204706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.204714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.204783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.204852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.204921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.204989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.205046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.205053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.205121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.205183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.205248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.205314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.205377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.205387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.205455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.205516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.205581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.205647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.205704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.205715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.205791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.205853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.205918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.205986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.206055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.206065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.206134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.206196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.206262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.206328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.206382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.206389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.206456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342982822837422:9464], now have 1 active actors on pipe
2025-10-09T21:51:32.206683Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:32.206685Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:32.206700Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:32.206725Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342982822837422:9464]
2025-10-09T21:51:32.206734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2169 reason: error at write: Disconnected sessionId: src_id|de635304-9697359a-d6b06d42-3ac048ac_0
2025-10-09T21:51:32.206942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2169 sessionId: src_id|de635304-9697359a-d6b06d42-3ac048ac_0 is DEAD
2025-10-09T21:51:32.207096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de635304-9697359a-d6b06d42-3ac048ac_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:32.207107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de635304-9697359a-d6b06d42-3ac048ac_0] Write session will restart in 0.005948s
2025-10-09T21:51:32.207135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de635304-9697359a-d6b06d42-3ac048ac_0] Write session: Do CDS request
2025-10-09T21:51:32.207142Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de635304-9697359a-d6b06d42-3ac048ac_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:32.207908Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:32.214506Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|de635304-9697359a-d6b06d42-3ac048ac_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:32.214634Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:32.214651Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2170
2025-10-09T21:51:32.214787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2170 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:32.214823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2170 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:32.214829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2170 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:32.214850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:32.215275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2170 sessionId:  describe result for acl check
2025-10-09T21:51:32.215315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:32.215321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:32.215324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:32.215329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342982822837492:8926] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:32.215888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342982822837492:8926] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:32.218609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342982822837492:8926] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:32.221173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342982822837492:8926] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:32.221183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342982822837492:8926] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:32.221186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342982822837492:8926] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:32.221192Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2170 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:32.224459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:32.224841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:32.225015Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:32.225026Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:32.225036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:32.225079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:32.225129Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342982460995485:11645]
2025-10-09T21:51:32.225765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:32.226081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:32.226134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:32.226466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:32.226525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.226533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.226535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.226536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.226538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.226539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.226541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.226542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.226544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.226549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.226559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.226563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.226568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.226572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.226577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.226581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.226585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.226589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.226597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.226611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.226621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.226628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.226640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.226646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.226651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.226657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.226662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.226673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.226678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.226689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.226695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.226707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.226712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.226724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.226731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.226744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.226773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:32.233158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:32.234731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:32.234749Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:32.234752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:32.234756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:32.234763Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2169 [2:7559342982460995485:11645]
2025-10-09T21:51:32.234780Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:32.234794Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:32.234797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:32.234801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:32.234804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:32.234819Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:32.234825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:32.234832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:32.234894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.234907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.234909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.234911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:32.234984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:32.234998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.235009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.235020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.235037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:32.235056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342982822837822:8926], now have 1 active actors on pipe
2025-10-09T21:51:32.235066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342982822837427:2432], now have 1 active actors on pipe
2025-10-09T21:51:32.235073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342982460995480:11645]
2025-10-09T21:51:32.235075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342982460995480:11645]
2025-10-09T21:51:32.235078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342982460995480:11645]
2025-10-09T21:51:32.235100Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:32.235121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:32.235136Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:32.235142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:32.235137Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2169
2025-10-09T21:51:32.235143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:32.235145Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:32.235146Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2169
2025-10-09T21:51:32.235146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:32.235173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342982460995480:11645]
2025-10-09T21:51:32.235177Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:32.235181Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:32.235195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.235199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.235251Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a63c2714-d0dc132e-6524d096-e12ff04e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:32.235265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:32.235269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:32.235273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:32.235280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:32.235289Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:32.235312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:32.235614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.235632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.235651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:32.235757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2170 partition: 0 MaxSeqNo: 25 sessionId: src_id|a63c2714-d0dc132e-6524d096-e12ff04e_0
2025-10-09T21:51:32.236147Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de635304-9697359a-d6b06d42-3ac048ac_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a63c2714-d0dc132e-6524d096-e12ff04e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:32.239164Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a63c2714-d0dc132e-6524d096-e12ff04e_0] Write session: try to update token
2025-10-09T21:51:32.239182Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a63c2714-d0dc132e-6524d096-e12ff04e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:32.249305Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:32.249328Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:32.249331Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:32.249333Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:32.249343Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342982460995480:11645]
2025-10-09T21:51:32.249360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342982460995480:11645]
2025-10-09T21:51:32.249365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342982460995480:11645]
2025-10-09T21:51:32.249411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:32.249419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.249423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:32.249435Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:32.249442Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:32.249447Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:32.249577Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:32.254990Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2170 sessionId: src_id|a63c2714-d0dc132e-6524d096-e12ff04e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:32.257558Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:32.306950Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342982822837841:9464]
2025-10-09T21:51:32.307258Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 2169, pipe: [1:7559342982822837841:9464]
2025-10-09T21:51:32.399207Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:32.399852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.399871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.399921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342982822837841:9464], now have 1 active actors on pipe
2025-10-09T21:51:32.400172Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342982822837841:9464]
2025-10-09T21:51:32.400170Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:32.400181Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:32.400215Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:32.400238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2170 reason: error at write: Disconnected sessionId: src_id|a63c2714-d0dc132e-6524d096-e12ff04e_0
2025-10-09T21:51:32.400396Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2170 sessionId: src_id|a63c2714-d0dc132e-6524d096-e12ff04e_0 is DEAD
2025-10-09T21:51:32.400525Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:32.400545Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a63c2714-d0dc132e-6524d096-e12ff04e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:32.400556Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a63c2714-d0dc132e-6524d096-e12ff04e_0] Write session will restart in 0.009683s
2025-10-09T21:51:32.400579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a63c2714-d0dc132e-6524d096-e12ff04e_0] Write session: Do CDS request
2025-10-09T21:51:32.400584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a63c2714-d0dc132e-6524d096-e12ff04e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:32.411636Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a63c2714-d0dc132e-6524d096-e12ff04e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:32.411816Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:32.411836Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2171
2025-10-09T21:51:32.411987Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2171 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:32.412022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2171 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:32.412033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2171 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:32.412037Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:32.412454Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2171 sessionId:  describe result for acl check
2025-10-09T21:51:32.412496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:32.412503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:32.412505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:32.412511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342982822838062:8916] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:32.413089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342982822838062:8916] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:32.415446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342982822838062:8916] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:32.417961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:32.419108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:32.419134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342982822838062:8916] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:32.419147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342982822838062:8916] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:32.419151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342982822838062:8916] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:32.419160Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2171 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:32.419291Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:32.419303Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:32.419317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:32.419384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:32.419429Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342982460995733:11647]
2025-10-09T21:51:32.420008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:32.420327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:32.420384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:32.420716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:32.420790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.420795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.420796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.420797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.420797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.420798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.420799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.420800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.420800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.420804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.420811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.420813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.420815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.420817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.420819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.420821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.420823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.420825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.420830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.420836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.420840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.420843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.420847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.420850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.420852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.420855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.420858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.420861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.420863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.420866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.420869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.420872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.420875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.420878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.420881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.420884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.420901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:32.426927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:32.427969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:32.427986Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:32.427988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:32.427993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:32.427999Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2170 [2:7559342982460995733:11647]
2025-10-09T21:51:32.428010Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:32.428026Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:32.428033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:32.428036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:32.428038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:32.428054Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:32.428068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:32.428075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:32.428129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.428136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.428139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.428142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:32.428208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:32.428217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.428230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.428243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.428262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:32.428287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342982822838240:8916], now have 1 active actors on pipe
2025-10-09T21:51:32.428299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342982460995728:11647]
2025-10-09T21:51:32.428305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342982460995728:11647]
2025-10-09T21:51:32.428308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342982460995728:11647]
2025-10-09T21:51:32.428320Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:32.428370Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2170
2025-10-09T21:51:32.428373Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2170
2025-10-09T21:51:32.428332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342982822837844:2432], now have 1 active actors on pipe
2025-10-09T21:51:32.428345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:32.428350Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:32.428353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:32.428355Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:32.428360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:32.428363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:32.428383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342982460995728:11647]
2025-10-09T21:51:32.428386Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:32.428415Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:32.428419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.428424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.428450Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f98fff9c-4e432512-c2ad573-13695c75_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:32.428462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:32.428470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:32.428473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:32.428486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:32.428501Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:32.428516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:32.428676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.428686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.428700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:32.428806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2171 partition: 0 MaxSeqNo: 25 sessionId: src_id|f98fff9c-4e432512-c2ad573-13695c75_0
2025-10-09T21:51:32.429166Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a63c2714-d0dc132e-6524d096-e12ff04e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f98fff9c-4e432512-c2ad573-13695c75_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:32.432105Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f98fff9c-4e432512-c2ad573-13695c75_0] Write session: try to update token
2025-10-09T21:51:32.432127Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f98fff9c-4e432512-c2ad573-13695c75_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:32.439272Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:32.439287Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:32.439295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342982460995728:11647]
2025-10-09T21:51:32.439329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:32.439340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:32.439478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:32.439633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:32.439767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:32.439898Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:32.440078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:32.440111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:32.447689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2171 sessionId: src_id|f98fff9c-4e432512-c2ad573-13695c75_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:32.450051Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:32.450243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.450264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.450303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:32.450386Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:32.451519Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:32.451534Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:32.451536Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:32.451538Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:32.451547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342982460995728:11647]
2025-10-09T21:51:32.451550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342982460995728:11647]
2025-10-09T21:51:32.451555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342982460995728:11647]
2025-10-09T21:51:32.451590Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:32.451592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:32.451599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.451601Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:32.451602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:32.451604Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:32.451730Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:32.500478Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342982822838251:9464]
2025-10-09T21:51:32.500773Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 2170, pipe: [1:7559342982822838251:9464]
2025-10-09T21:51:32.592825Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:32.599680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.599708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.599850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.599908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.599969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.600027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.600070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.600079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.600149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.600201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.600256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.600322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.600368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.600380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.600439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.600494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.600552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.600620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.600668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.600675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.600738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.600798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.600863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.600923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.600963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.600968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.601028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.601086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.601147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.601204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.601248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.601254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.601309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.601375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.601432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.601486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.601530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.601537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.601602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.601660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.601716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.601772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.601813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.601820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.601878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.601932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.601985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.602055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.602098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.602107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.602175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.602232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.602288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.602343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.602383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.602388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.602449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:32.602512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:32.602566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:32.602627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:32.602669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:32.602675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:32.602717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342982822838251:9464], now have 1 active actors on pipe
2025-10-09T21:51:32.603060Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:32.603061Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342982822838251:9464]
2025-10-09T21:51:32.603080Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:32.603084Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:32.603110Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2171 reason: error at write: Disconnected sessionId: src_id|f98fff9c-4e432512-c2ad573-13695c75_0
2025-10-09T21:51:32.603309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2171 sessionId: src_id|f98fff9c-4e432512-c2ad573-13695c75_0 is DEAD
2025-10-09T21:51:32.603486Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f98fff9c-4e432512-c2ad573-13695c75_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:32.603499Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f98fff9c-4e432512-c2ad573-13695c75_0] Write session will restart in 0.007568s
2025-10-09T21:51:32.603600Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f98fff9c-4e432512-c2ad573-13695c75_0] Write session: Do CDS request
2025-10-09T21:51:32.603615Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f98fff9c-4e432512-c2ad573-13695c75_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:32.603800Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:32.612563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f98fff9c-4e432512-c2ad573-13695c75_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:32.612682Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:32.612702Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2172
2025-10-09T21:51:32.612835Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2172 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:32.612869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2172 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:32.612876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2172 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:32.612880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:32.613297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2172 sessionId:  describe result for acl check
2025-10-09T21:51:32.613330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:32.613332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:32.613333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:32.613339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342982822838388:8911] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:32.613838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342982822838388:8911] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:32.616796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342982822838388:8911] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:32.619782Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342982822838388:8911] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:32.619795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342982822838388:8911] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:32.619797Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342982822838388:8911] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:32.619829Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2172 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:32.621846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:32.623159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:32.623311Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:32.623320Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:32.623331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:32.623368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:32.623400Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342982460995986:11649]
2025-10-09T21:51:32.623873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:32.624125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:32.624161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:32.624461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:32.624523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.624529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.624530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.624531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.624532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.624534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.624535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.624536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.624537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.624542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.624552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.624555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.624559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.624562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.624565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.624568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.624571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.624575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.624587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.624594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.624605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.624610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.624619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.624624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.624628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.624634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.624640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.624647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.624655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.624660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.624668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.624674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.624678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.624687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.624696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.624706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.624734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:32.630747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:32.631813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:32.631830Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:32.631832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:32.631838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:32.631845Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2171 [2:7559342982460995986:11649]
2025-10-09T21:51:32.631858Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:32.631876Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:32.631879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:32.631882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:32.631884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:32.631899Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:32.631908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:32.631914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:32.631961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.631966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.631967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.631969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:32.632033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:32.632039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.632046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.632053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.632066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:32.632083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342982460995981:11649]
2025-10-09T21:51:32.632084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342982460995981:11649]
2025-10-09T21:51:32.632086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342982460995981:11649]
2025-10-09T21:51:32.632095Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:32.632104Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:32.632106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342982822838254:2432], now have 1 active actors on pipe
2025-10-09T21:51:32.632107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:32.632108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342982822838643:8911], now have 1 active actors on pipe
2025-10-09T21:51:32.632109Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:32.632115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:32.632124Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:32.632126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:32.632149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342982460995981:11649]
2025-10-09T21:51:32.632156Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:32.632194Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:32.632175Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2171
2025-10-09T21:51:32.632176Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2171
2025-10-09T21:51:32.632308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.632322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.632350Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|218f44d6-bf5f800f-e58864f5-5ddae068_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:32.632362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:32.632365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:32.632367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:32.632376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:32.632386Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:32.632405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:32.632599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.632609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.632624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:32.632737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2172 partition: 0 MaxSeqNo: 25 sessionId: src_id|218f44d6-bf5f800f-e58864f5-5ddae068_0
2025-10-09T21:51:32.633100Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f98fff9c-4e432512-c2ad573-13695c75_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|218f44d6-bf5f800f-e58864f5-5ddae068_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:32.635717Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|218f44d6-bf5f800f-e58864f5-5ddae068_0] Write session: try to update token
2025-10-09T21:51:32.635733Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|218f44d6-bf5f800f-e58864f5-5ddae068_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:32.645529Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:32.645545Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:32.645548Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:32.645550Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:32.645558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342982460995981:11649]
2025-10-09T21:51:32.645563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342982460995981:11649]
2025-10-09T21:51:32.645567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342982460995981:11649]
2025-10-09T21:51:32.645607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:32.645609Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:32.645614Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:32.645618Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:32.645618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.645621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:32.645761Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:32.649211Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2172 sessionId: src_id|218f44d6-bf5f800f-e58864f5-5ddae068_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:32.651761Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:32.703233Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342982822838663:9464]
2025-10-09T21:51:32.703525Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 2171, pipe: [1:7559342982822838663:9464]
2025-10-09T21:51:32.794099Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:32.794945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.794966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.795011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342982822838663:9464], now have 1 active actors on pipe
2025-10-09T21:51:32.795194Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342982822838663:9464]
2025-10-09T21:51:32.795228Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:32.795260Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:32.795268Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:32.795320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2172 reason: error at write: Disconnected sessionId: src_id|218f44d6-bf5f800f-e58864f5-5ddae068_0
2025-10-09T21:51:32.795404Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:32.795498Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2172 sessionId: src_id|218f44d6-bf5f800f-e58864f5-5ddae068_0 is DEAD
2025-10-09T21:51:32.795671Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|218f44d6-bf5f800f-e58864f5-5ddae068_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:32.795682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|218f44d6-bf5f800f-e58864f5-5ddae068_0] Write session will restart in 0.009284s
2025-10-09T21:51:32.795710Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|218f44d6-bf5f800f-e58864f5-5ddae068_0] Write session: Do CDS request
2025-10-09T21:51:32.795715Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|218f44d6-bf5f800f-e58864f5-5ddae068_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:32.806492Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|218f44d6-bf5f800f-e58864f5-5ddae068_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:32.806587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:32.806604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2173
2025-10-09T21:51:32.806745Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2173 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:32.806776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2173 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:32.806786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2173 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:32.806790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:32.807174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2173 sessionId:  describe result for acl check
2025-10-09T21:51:32.807208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:32.807217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:32.807221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:32.807228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342982822838934:8899] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:32.807768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342982822838934:8899] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:32.810142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342982822838934:8899] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:32.811091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:32.812560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:32.812706Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:32.812715Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:32.812723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:32.812759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:32.812783Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342982460996254:11659]
2025-10-09T21:51:32.812890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342982822838934:8899] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:32.812907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342982822838934:8899] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:32.812911Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342982822838934:8899] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:32.812919Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2173 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:32.813244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:32.813513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:32.813567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:32.813894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:32.813942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.813948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.813949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.813950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.813951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.813953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.813954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.813955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.813956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.813961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.813971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.813979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.813984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.813987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.813991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.813994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.813997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.814001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.814008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.814021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.814047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.814057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.814066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.814076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.814084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.814090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.814099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.814104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.814108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.814117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.814122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.814133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.814139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.814149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.814158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.814163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.814196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:32.820340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:32.821290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:32.821304Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:32.821306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:32.821309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:32.821314Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2172 [2:7559342982460996254:11659]
2025-10-09T21:51:32.821324Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:32.821342Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:32.821359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:32.821362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:32.821363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:32.821377Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:32.821385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:32.821390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:32.821435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.821441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.821442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.821444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:32.821498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:32.821500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.821507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.821515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.821528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:32.821544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342982460996249:11659]
2025-10-09T21:51:32.821549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342982460996249:11659]
2025-10-09T21:51:32.821550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342982460996249:11659]
2025-10-09T21:51:32.821556Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:32.821566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:32.821568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:32.821571Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:32.821574Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:32.821574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:32.821577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:32.821585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342982822838678:2432], now have 1 active actors on pipe
2025-10-09T21:51:32.821607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342982822839075:8899], now have 1 active actors on pipe
2025-10-09T21:51:32.821613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342982460996249:11659]
2025-10-09T21:51:32.821618Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:32.821638Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:32.821662Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2172
2025-10-09T21:51:32.821707Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2172
2025-10-09T21:51:32.821738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.821746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.821765Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8b2a3d6b-ef897159-f9327f19-35c57191_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:32.821771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:32.821774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:32.821775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:32.821782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:32.821792Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:32.821806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:32.821929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.821936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.821945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:32.822040Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2173 partition: 0 MaxSeqNo: 25 sessionId: src_id|8b2a3d6b-ef897159-f9327f19-35c57191_0
2025-10-09T21:51:32.822388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|218f44d6-bf5f800f-e58864f5-5ddae068_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8b2a3d6b-ef897159-f9327f19-35c57191_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:32.824716Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2a3d6b-ef897159-f9327f19-35c57191_0] Write session: try to update token
2025-10-09T21:51:32.824729Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2a3d6b-ef897159-f9327f19-35c57191_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:32.835502Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:32.835518Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:32.835519Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:32.835520Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:32.835526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342982460996249:11659]
2025-10-09T21:51:32.835529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342982460996249:11659]
2025-10-09T21:51:32.835531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342982460996249:11659]
2025-10-09T21:51:32.835563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:32.835566Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:32.835567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.835569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:32.835570Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:32.835573Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:32.835693Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:32.837061Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2173 sessionId: src_id|8b2a3d6b-ef897159-f9327f19-35c57191_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:32.839438Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:32.895229Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342982822839097:9464]
2025-10-09T21:51:32.895520Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 2172, pipe: [1:7559342982822839097:9464]
2025-10-09T21:51:32.974758Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:32.975173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:32.975198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:32.975248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342982822839097:9464], now have 1 active actors on pipe
2025-10-09T21:51:32.975456Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:32.975480Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:32.975485Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:32.975499Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:32.975548Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342982822839097:9464]
2025-10-09T21:51:32.975561Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2173 reason: error at write: Disconnected sessionId: src_id|8b2a3d6b-ef897159-f9327f19-35c57191_0
2025-10-09T21:51:32.975754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2173 sessionId: src_id|8b2a3d6b-ef897159-f9327f19-35c57191_0 is DEAD
2025-10-09T21:51:32.975871Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2a3d6b-ef897159-f9327f19-35c57191_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:32.975884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2a3d6b-ef897159-f9327f19-35c57191_0] Write session will restart in 0.009541s
2025-10-09T21:51:32.975911Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2a3d6b-ef897159-f9327f19-35c57191_0] Write session: Do CDS request
2025-10-09T21:51:32.975916Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2a3d6b-ef897159-f9327f19-35c57191_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:32.986407Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2a3d6b-ef897159-f9327f19-35c57191_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:32.986512Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:32.986527Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2174
2025-10-09T21:51:32.986634Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2174 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:32.986680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2174 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:32.986691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2174 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:32.986695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:32.987123Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2174 sessionId:  describe result for acl check
2025-10-09T21:51:32.987167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:32.987172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:32.987174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:32.987181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342982822839348:8889] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:32.987741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342982822839348:8889] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:32.989944Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342982822839348:8889] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:32.991760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:32.993209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:32.993408Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:32.993418Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:32.993431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:32.993498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:32.993542Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342982460996532:11667]
2025-10-09T21:51:32.993576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342982822839348:8889] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:32.993587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342982822839348:8889] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:32.993590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342982822839348:8889] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:32.993595Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2174 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:32.994145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:32.994389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:32.994428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:32.994748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:32.994799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.994805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.994807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.994808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.994809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.994810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.994811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.994813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.994814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.994818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.994829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.994838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.994847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.994851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.994855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.994863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.994867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.994875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.994880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.994892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:32.994903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.994912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:32.994920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.994925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:32.994934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.994943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:32.994952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.994961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:32.994967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.994976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:32.994986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.994995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:32.995003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.995009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:32.995013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.995022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:32.995050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:33.001100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:33.002556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:33.002569Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:33.002572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:33.002576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:33.002583Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2173 [2:7559342982460996532:11667]
2025-10-09T21:51:33.002597Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:33.002613Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:33.002619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:33.002621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:33.002622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:33.002636Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:33.002644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:33.002651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:33.002711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.002717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.002719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.002720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:33.002773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:33.002779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.002786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.002796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.002809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:33.002828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342982460996527:11667]
2025-10-09T21:51:33.002836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342982460996527:11667]
2025-10-09T21:51:33.002837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342982460996527:11667]
2025-10-09T21:51:33.002844Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:33.002855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:33.002858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:33.002858Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:33.002861Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:33.002872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:33.002876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:33.002886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342982822839103:2432], now have 1 active actors on pipe
2025-10-09T21:51:33.002890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342982822839499:8889], now have 1 active actors on pipe
2025-10-09T21:51:33.002915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342982460996527:11667]
2025-10-09T21:51:33.002925Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:33.002944Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:33.002925Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2173
2025-10-09T21:51:33.002947Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2173
2025-10-09T21:51:33.002990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.002997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.003018Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b78640d0-3fc0fe31-9d047c01-61dde213_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:33.003027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:33.003034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:33.003035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:33.003043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:33.003053Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:33.003068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:33.003226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.003241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.003269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:33.003374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2174 partition: 0 MaxSeqNo: 25 sessionId: src_id|b78640d0-3fc0fe31-9d047c01-61dde213_0
2025-10-09T21:51:33.003709Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b2a3d6b-ef897159-f9327f19-35c57191_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b78640d0-3fc0fe31-9d047c01-61dde213_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:33.006105Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b78640d0-3fc0fe31-9d047c01-61dde213_0] Write session: try to update token
2025-10-09T21:51:33.006120Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b78640d0-3fc0fe31-9d047c01-61dde213_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:33.014550Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:33.014572Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:33.014582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342982460996527:11667]
2025-10-09T21:51:33.014624Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:33.014630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:33.014805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:33.014952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:33.015089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:33.015231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:33.015415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:33.015459Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:33.019284Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2174 sessionId: src_id|b78640d0-3fc0fe31-9d047c01-61dde213_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:33.021669Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:33.021841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.021861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.021897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:33.022010Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:33.027731Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:33.027746Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:33.027748Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:33.027750Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:33.027761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342982460996527:11667]
2025-10-09T21:51:33.027765Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342982460996527:11667]
2025-10-09T21:51:33.027769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342982460996527:11667]
2025-10-09T21:51:33.027812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:33.027815Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:33.027817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.027819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:33.027820Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:33.027822Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:33.027950Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:33.075970Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342987117806807:9464]
2025-10-09T21:51:33.076280Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 2173, pipe: [1:7559342987117806807:9464]
2025-10-09T21:51:33.178277Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:33.186049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.186081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.186237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.186305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.186374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.186435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.186476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.186484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.186546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.186603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.186665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.186725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.186763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.186770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.186826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.186883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.186943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.186999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.187042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.187049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.187112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.187171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.187242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.187302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.187347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.187354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.187426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.187492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.187551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.187611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.187651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.187653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.187721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.187783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.187845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.187902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.187944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.187950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.188007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.188063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.188122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.188177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.188217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.188223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.188277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.188332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.188386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.188440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.188479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.188484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.188551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.188607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.188660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.188715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.188755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.188761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.188817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.188869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.188929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.188985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.189027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.189030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.189083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342987117806807:9464], now have 1 active actors on pipe
2025-10-09T21:51:33.189380Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:33.189395Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:33.189398Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:33.189433Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2174 reason: error at write: Disconnected sessionId: src_id|b78640d0-3fc0fe31-9d047c01-61dde213_0
2025-10-09T21:51:33.189450Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342987117806807:9464]
2025-10-09T21:51:33.189617Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2174 sessionId: src_id|b78640d0-3fc0fe31-9d047c01-61dde213_0 is DEAD
2025-10-09T21:51:33.189781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b78640d0-3fc0fe31-9d047c01-61dde213_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:33.189791Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b78640d0-3fc0fe31-9d047c01-61dde213_0] Write session will restart in 0.009828s
2025-10-09T21:51:33.189810Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b78640d0-3fc0fe31-9d047c01-61dde213_0] Write session: Do CDS request
2025-10-09T21:51:33.189815Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b78640d0-3fc0fe31-9d047c01-61dde213_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:33.190071Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:33.200615Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b78640d0-3fc0fe31-9d047c01-61dde213_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:33.200759Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:33.200777Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2175
2025-10-09T21:51:33.200913Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2175 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:33.200956Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2175 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:33.200968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2175 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:33.200974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:33.201462Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2175 sessionId:  describe result for acl check
2025-10-09T21:51:33.201505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:33.201511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:33.201513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:33.201520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342987117806969:8886] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:33.202146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342987117806969:8886] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:33.205276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342987117806969:8886] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:33.208151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342987117806969:8886] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:33.208167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342987117806969:8886] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:33.208171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342987117806969:8886] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:33.208181Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2175 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:33.208173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:33.209832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:33.209999Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:33.210006Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:33.210015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:33.210077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:33.210109Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342986755964084:11669]
2025-10-09T21:51:33.210650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:33.210918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:33.210958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:33.211176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:33.211211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.211217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.211218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.211219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.211220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.211221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.211222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.211222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.211223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.211226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.211234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.211236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.211239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.211241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.211243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.211246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.211249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.211253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.211261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.211268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.211275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.211281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.211284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.211288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.211291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.211294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.211297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.211302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.211306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.211311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.211316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.211320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.211324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.211329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.211335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.211339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.211366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:33.217796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:33.218839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:33.218855Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:33.218858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:33.218863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:33.218870Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2174 [2:7559342986755964084:11669]
2025-10-09T21:51:33.218885Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:33.218904Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:33.218907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:33.218910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:33.218913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:33.218929Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:33.218941Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:33.218948Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:33.219003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.219012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.219014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.219017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:33.219080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:33.219087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.219094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.219106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.219116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:33.219136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342986755964078:11669]
2025-10-09T21:51:33.219143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342986755964078:11669]
2025-10-09T21:51:33.219146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342986755964078:11669]
2025-10-09T21:51:33.219158Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:33.219172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342987117807209:8886], now have 1 active actors on pipe
2025-10-09T21:51:33.219173Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:33.219176Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:33.219177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342987117806810:2432], now have 1 active actors on pipe
2025-10-09T21:51:33.219178Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:33.219185Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:33.219198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:33.219200Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:33.219226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342986755964078:11669]
2025-10-09T21:51:33.219234Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:33.219246Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:33.219216Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2174
2025-10-09T21:51:33.219242Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2174
2025-10-09T21:51:33.219276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.219287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.219321Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6926256d-af43f87c-3ecd4ff1-dadc1a11_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:33.219333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:33.219336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:33.219338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:33.219345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:33.219353Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:33.219370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:33.219499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.219505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.219515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:33.219612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2175 partition: 0 MaxSeqNo: 25 sessionId: src_id|6926256d-af43f87c-3ecd4ff1-dadc1a11_0
2025-10-09T21:51:33.219943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b78640d0-3fc0fe31-9d047c01-61dde213_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6926256d-af43f87c-3ecd4ff1-dadc1a11_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:33.222263Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6926256d-af43f87c-3ecd4ff1-dadc1a11_0] Write session: try to update token
2025-10-09T21:51:33.222278Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6926256d-af43f87c-3ecd4ff1-dadc1a11_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:33.233508Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:33.233529Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:33.233531Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:33.233533Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:33.233542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342986755964078:11669]
2025-10-09T21:51:33.233547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342986755964078:11669]
2025-10-09T21:51:33.233550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342986755964078:11669]
2025-10-09T21:51:33.233596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:33.233597Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:33.233602Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:33.233605Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:33.233608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.233611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:33.233744Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:33.235029Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2175 sessionId: src_id|6926256d-af43f87c-3ecd4ff1-dadc1a11_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:33.237374Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:33.289445Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342987117807224:9464]
2025-10-09T21:51:33.289713Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 2174, pipe: [1:7559342987117807224:9464]
2025-10-09T21:51:33.383700Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:33.384168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.384181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.384217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342987117807224:9464], now have 1 active actors on pipe
2025-10-09T21:51:33.384402Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:33.384445Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342987117807224:9464]
2025-10-09T21:51:33.384449Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:33.384453Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:33.384478Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:33.384513Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2175 reason: error at write: Disconnected sessionId: src_id|6926256d-af43f87c-3ecd4ff1-dadc1a11_0
2025-10-09T21:51:33.384695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2175 sessionId: src_id|6926256d-af43f87c-3ecd4ff1-dadc1a11_0 is DEAD
2025-10-09T21:51:33.384839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6926256d-af43f87c-3ecd4ff1-dadc1a11_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:33.384851Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6926256d-af43f87c-3ecd4ff1-dadc1a11_0] Write session will restart in 0.008682s
2025-10-09T21:51:33.384877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6926256d-af43f87c-3ecd4ff1-dadc1a11_0] Write session: Do CDS request
2025-10-09T21:51:33.384883Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6926256d-af43f87c-3ecd4ff1-dadc1a11_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:33.394476Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6926256d-af43f87c-3ecd4ff1-dadc1a11_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:33.394555Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:33.394571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2176
2025-10-09T21:51:33.394698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2176 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:33.394728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2176 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:33.394739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2176 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:33.394748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:33.395275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2176 sessionId:  describe result for acl check
2025-10-09T21:51:33.395314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:33.395322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:33.395324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:33.395331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342987117807429:8881] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:33.395884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342987117807429:8881] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:33.398185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342987117807429:8881] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:33.400626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342987117807429:8881] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:33.400639Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342987117807429:8881] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:33.400642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:33.400642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342987117807429:8881] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:33.400654Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2176 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:33.402748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:33.402946Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:33.402957Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:33.402970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:33.403024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:33.403055Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342986755964338:11671]
2025-10-09T21:51:33.403526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:33.403827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:33.403877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:33.404169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:33.404208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.404214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.404215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.404216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.404216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.404217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.404218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.404219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.404219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.404223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.404229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.404232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.404234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.404236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.404238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.404240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.404243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.404245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.404250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.404255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.404260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.404263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.404266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.404269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.404272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.404275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.404278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.404281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.404286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.404289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.404292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.404295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.404298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.404301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.404303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.404306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.404324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:33.410461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:33.411453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:33.411465Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:33.411467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:33.411470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:33.411476Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2175 [2:7559342986755964338:11671]
2025-10-09T21:51:33.411486Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:33.411501Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:33.411506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:33.411509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:33.411510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:33.411523Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:33.411530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:33.411535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:33.411591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.411597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.411599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.411601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:33.411655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:33.411660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.411667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.411677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.411689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:33.411709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342987117807632:8881], now have 1 active actors on pipe
2025-10-09T21:51:33.411715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342987117807232:2432], now have 1 active actors on pipe
2025-10-09T21:51:33.411721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342986755964332:11671]
2025-10-09T21:51:33.411726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342986755964332:11671]
2025-10-09T21:51:33.411728Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342986755964332:11671]
2025-10-09T21:51:33.411737Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:33.411749Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:33.411751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:33.411760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:33.411762Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:33.411767Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:33.411768Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:33.411787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342986755964332:11671]
2025-10-09T21:51:33.411789Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:33.411790Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:33.411810Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2175
2025-10-09T21:51:33.411811Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2175
2025-10-09T21:51:33.411879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.411892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.411927Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d28b3fd9-8db82cda-96848f88-65d27051_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:33.411938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:33.411942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:33.411947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:33.411958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:33.411968Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:33.411986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:33.412153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.412164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.412179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:33.412280Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2176 partition: 0 MaxSeqNo: 25 sessionId: src_id|d28b3fd9-8db82cda-96848f88-65d27051_0
2025-10-09T21:51:33.412661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6926256d-af43f87c-3ecd4ff1-dadc1a11_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d28b3fd9-8db82cda-96848f88-65d27051_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:33.415462Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d28b3fd9-8db82cda-96848f88-65d27051_0] Write session: try to update token
2025-10-09T21:51:33.415481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d28b3fd9-8db82cda-96848f88-65d27051_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:33.425558Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:33.425576Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:33.425578Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:33.425579Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:33.425586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342986755964332:11671]
2025-10-09T21:51:33.425589Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342986755964332:11671]
2025-10-09T21:51:33.425594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342986755964332:11671]
2025-10-09T21:51:33.425629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:33.425635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.425642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:33.425643Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:33.425649Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:33.425653Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:33.425776Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:33.428381Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2176 sessionId: src_id|d28b3fd9-8db82cda-96848f88-65d27051_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:33.430694Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:33.484563Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342987117807644:9464]
2025-10-09T21:51:33.484865Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 2175, pipe: [1:7559342987117807644:9464]
2025-10-09T21:51:33.568648Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:33.569272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.569287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.569326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342987117807644:9464], now have 1 active actors on pipe
2025-10-09T21:51:33.569492Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:33.569571Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:33.569586Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:33.569590Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:33.569616Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342987117807644:9464]
2025-10-09T21:51:33.569655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2176 reason: error at write: Disconnected sessionId: src_id|d28b3fd9-8db82cda-96848f88-65d27051_0
2025-10-09T21:51:33.569845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2176 sessionId: src_id|d28b3fd9-8db82cda-96848f88-65d27051_0 is DEAD
2025-10-09T21:51:33.569952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d28b3fd9-8db82cda-96848f88-65d27051_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:33.569963Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d28b3fd9-8db82cda-96848f88-65d27051_0] Write session will restart in 0.007411s
2025-10-09T21:51:33.569983Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d28b3fd9-8db82cda-96848f88-65d27051_0] Write session: Do CDS request
2025-10-09T21:51:33.569988Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d28b3fd9-8db82cda-96848f88-65d27051_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:33.578387Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d28b3fd9-8db82cda-96848f88-65d27051_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:33.578460Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:33.578477Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2177
2025-10-09T21:51:33.578585Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2177 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:33.578615Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2177 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:33.578633Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2177 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:33.578641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:33.579102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2177 sessionId:  describe result for acl check
2025-10-09T21:51:33.579132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:33.579134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:33.579137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:33.579143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342987117807843:8877] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:33.579656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342987117807843:8877] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:33.581869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342987117807843:8877] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:33.583972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:33.584407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342987117807843:8877] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:33.584423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342987117807843:8877] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:33.584427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342987117807843:8877] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:33.584434Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2177 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:33.585680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:33.585839Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:33.585847Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:33.585855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:33.585900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:33.585937Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342986755964595:11673]
2025-10-09T21:51:33.586338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:33.586583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:33.586610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:33.586803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:33.586833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.586839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.586840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.586841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.586842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.586843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.586843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.586844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.586845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.586850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.586859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.586863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.586866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.586870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.586873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.586877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.586880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.586884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.586888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.586893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.586897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.586900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.586903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.586908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.586912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.586916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.586919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.586924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.586933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.586938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.586945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.586949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.586952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.586955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.586957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.586960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.586976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:33.592576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:33.593631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:33.593643Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:33.593644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:33.593647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:33.593659Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2176 [2:7559342986755964595:11673]
2025-10-09T21:51:33.593669Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:33.593684Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:33.593689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:33.593690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:33.593692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:33.593707Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:33.593716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:33.593727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:33.593769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.593773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.593775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.593777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:33.593823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:33.593827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.593834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.593840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.593850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:33.593867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342987117807647:2432], now have 1 active actors on pipe
2025-10-09T21:51:33.593875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342986755964589:11673]
2025-10-09T21:51:33.593877Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342986755964589:11673]
2025-10-09T21:51:33.593878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342986755964589:11673]
2025-10-09T21:51:33.593887Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:33.593893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342987117808049:8877], now have 1 active actors on pipe
2025-10-09T21:51:33.593900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:33.593906Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:33.593908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:33.593910Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:33.593915Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:33.593917Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:33.593923Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342986755964589:11673]
2025-10-09T21:51:33.593925Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:33.593927Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:33.593950Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2176
2025-10-09T21:51:33.593957Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2176
2025-10-09T21:51:33.594003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.594010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.594042Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4c8129dd-116f83e1-4f33a1c7-42f0b779_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:33.594051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:33.594055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:33.594056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:33.594063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:33.594070Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:33.594084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:33.594200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.594211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.594238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:33.594316Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2177 partition: 0 MaxSeqNo: 25 sessionId: src_id|4c8129dd-116f83e1-4f33a1c7-42f0b779_0
2025-10-09T21:51:33.594620Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d28b3fd9-8db82cda-96848f88-65d27051_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4c8129dd-116f83e1-4f33a1c7-42f0b779_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:33.596849Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4c8129dd-116f83e1-4f33a1c7-42f0b779_0] Write session: try to update token
2025-10-09T21:51:33.596860Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4c8129dd-116f83e1-4f33a1c7-42f0b779_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:33.607446Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:33.607464Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:33.607466Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:33.607468Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:33.607477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342986755964589:11673]
2025-10-09T21:51:33.607482Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342986755964589:11673]
2025-10-09T21:51:33.607486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342986755964589:11673]
2025-10-09T21:51:33.607531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:33.607538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.607541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:33.607573Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:33.607586Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:33.607589Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:33.607683Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:33.608913Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2177 sessionId: src_id|4c8129dd-116f83e1-4f33a1c7-42f0b779_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:33.611294Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:33.669890Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342987117808060:9464]
2025-10-09T21:51:33.670140Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 2176, pipe: [1:7559342987117808060:9464]
2025-10-09T21:51:33.756174Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:33.756677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.756693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.756731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342987117808060:9464], now have 1 active actors on pipe
2025-10-09T21:51:33.756937Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:33.757020Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342987117808060:9464]
2025-10-09T21:51:33.757029Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:33.757063Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:33.757106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2177 reason: status is not ok: tablet will be restarted right now sessionId: src_id|4c8129dd-116f83e1-4f33a1c7-42f0b779_0
2025-10-09T21:51:33.757283Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2177 sessionId: src_id|4c8129dd-116f83e1-4f33a1c7-42f0b779_0 is DEAD
2025-10-09T21:51:33.757438Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c8129dd-116f83e1-4f33a1c7-42f0b779_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:33.757447Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c8129dd-116f83e1-4f33a1c7-42f0b779_0] Write session will restart in 0.008968s
2025-10-09T21:51:33.757473Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c8129dd-116f83e1-4f33a1c7-42f0b779_0] Write session: Do CDS request
2025-10-09T21:51:33.757478Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c8129dd-116f83e1-4f33a1c7-42f0b779_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:33.767519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4c8129dd-116f83e1-4f33a1c7-42f0b779_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:33.767613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:33.767633Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2178
2025-10-09T21:51:33.767732Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2178 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:33.767765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2178 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:33.767770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2178 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:33.767774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:33.768126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2178 sessionId:  describe result for acl check
2025-10-09T21:51:33.768159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:33.768160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:33.768162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:33.768167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342987117808270:8869] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:33.768737Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342987117808270:8869] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:33.770612Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342987117808270:8869] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:33.773294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342987117808270:8869] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:33.773318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342987117808270:8869] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:33.773324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342987117808270:8869] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:33.773332Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2178 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:33.773500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:33.775015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:33.775151Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:33.775158Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:33.775167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:33.775213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:33.775242Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342986755964851:11675]
2025-10-09T21:51:33.775749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:33.776031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:33.776074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:33.776321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:33.776358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.776363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.776364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.776365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.776365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.776366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.776366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.776367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.776368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.776371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.776378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.776380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.776382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.776384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.776386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.776388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.776390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.776392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.776397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.776402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.776406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.776409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.776411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.776414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.776417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.776420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.776423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.776428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.776432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.776437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.776442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.776452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.776454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.776457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.776459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.776462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.776478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:33.782439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:33.783429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:33.783444Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:33.783446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:33.783450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:33.783457Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2177 [2:7559342986755964851:11675]
2025-10-09T21:51:33.783473Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:33.783494Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:33.783496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:33.783498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:33.783499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:33.783512Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:33.783515Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:33.783521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:33.783569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.783575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.783577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.783578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:33.783629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:33.783634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.783641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.783647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.783658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:33.783675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342986755964845:11675]
2025-10-09T21:51:33.783678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342986755964845:11675]
2025-10-09T21:51:33.783680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342986755964845:11675]
2025-10-09T21:51:33.783688Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:33.783699Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:33.783702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:33.783703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:33.783704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:33.783716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:33.783719Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:33.783729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342987117808465:8869], now have 1 active actors on pipe
2025-10-09T21:51:33.783731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342987117808063:2432], now have 1 active actors on pipe
2025-10-09T21:51:33.783752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342986755964845:11675]
2025-10-09T21:51:33.783762Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:33.783779Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:33.783799Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2177
2025-10-09T21:51:33.783811Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2177
2025-10-09T21:51:33.783875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.783882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.783906Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a13137c7-1df94fa2-b0e5311-2b5efa0f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:33.783915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:33.783918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:33.783920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:33.783927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:33.783937Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:33.783955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:33.784125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.784131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.784142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:33.784212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2178 partition: 0 MaxSeqNo: 25 sessionId: src_id|a13137c7-1df94fa2-b0e5311-2b5efa0f_0
2025-10-09T21:51:33.784534Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c8129dd-116f83e1-4f33a1c7-42f0b779_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a13137c7-1df94fa2-b0e5311-2b5efa0f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:33.786998Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a13137c7-1df94fa2-b0e5311-2b5efa0f_0] Write session: try to update token
2025-10-09T21:51:33.787017Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a13137c7-1df94fa2-b0e5311-2b5efa0f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:33.799665Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2178 sessionId: src_id|a13137c7-1df94fa2-b0e5311-2b5efa0f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:33.801064Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:33.801085Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:33.801094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342986755964845:11675]
2025-10-09T21:51:33.801138Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:33.801145Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:33.801326Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:33.801487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:33.801626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:33.801764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:33.801945Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:33.802059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:33.802101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:33.802146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.802160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.802194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:33.802290Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:33.808612Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:33.808629Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:33.808631Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:33.808633Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:33.808642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342986755964845:11675]
2025-10-09T21:51:33.808646Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342986755964845:11675]
2025-10-09T21:51:33.808650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342986755964845:11675]
2025-10-09T21:51:33.808684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:33.808689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.808691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:33.808699Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:33.808704Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:33.808708Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:33.808804Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:33.857262Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342987117808491:9464]
2025-10-09T21:51:33.857544Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 2177, pipe: [1:7559342987117808491:9464]
2025-10-09T21:51:33.956916Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:33.963989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.964013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.964148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.964200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.964254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.964322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.964381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.964393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.964462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.964516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.964575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.964636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.964681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.964689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.964748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.964803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.964857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.964914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.964952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.964957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.965016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.965067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.965123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.965177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.965219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.965224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.965283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.965343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.965416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.965470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.965508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.965513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.965568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.965619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.965673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.965725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.965765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.965774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.965835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.965895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.965955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.966014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.966063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.966069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.966121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.966176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.966241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.966306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.966344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.966350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.966403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.966452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.966511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.966576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.966614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.966620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.966674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:33.966723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:33.966779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:33.966830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:33.966868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:33.966875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:33.966919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342987117808491:9464], now have 1 active actors on pipe
2025-10-09T21:51:33.967178Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342987117808491:9464]
2025-10-09T21:51:33.967181Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:33.967185Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:33.967214Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2178 reason: error at write: Disconnected sessionId: src_id|a13137c7-1df94fa2-b0e5311-2b5efa0f_0
2025-10-09T21:51:33.967215Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:33.967386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2178 sessionId: src_id|a13137c7-1df94fa2-b0e5311-2b5efa0f_0 is DEAD
2025-10-09T21:51:33.967533Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a13137c7-1df94fa2-b0e5311-2b5efa0f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:33.967542Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a13137c7-1df94fa2-b0e5311-2b5efa0f_0] Write session will restart in 0.005074s
2025-10-09T21:51:33.967569Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a13137c7-1df94fa2-b0e5311-2b5efa0f_0] Write session: Do CDS request
2025-10-09T21:51:33.967576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a13137c7-1df94fa2-b0e5311-2b5efa0f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:33.967734Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:33.973417Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a13137c7-1df94fa2-b0e5311-2b5efa0f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:33.973536Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:33.973551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2179
2025-10-09T21:51:33.973680Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2179 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:33.973717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2179 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:33.973725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2179 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:33.973728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:33.974098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2179 sessionId:  describe result for acl check
2025-10-09T21:51:33.974127Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:33.974132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:33.974134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:33.974145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342987117808540:8854] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:33.974581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342987117808540:8854] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:33.976568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342987117808540:8854] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:33.979450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342987117808540:8854] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:33.979463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342987117808540:8854] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:33.979467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342987117808540:8854] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:33.979474Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2179 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:33.983950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:33.985297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:33.985466Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:33.985475Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:33.985484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:33.985529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:33.985574Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342986755965152:11691]
2025-10-09T21:51:33.986025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:33.986325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:33.986357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:33.986623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:33.986681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.986688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.986689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.986689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.986690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.986691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.986691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.986692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.986693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.986696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.986703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.986706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.986708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.986710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.986712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.986714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.986716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.986718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.986723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.986729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:33.986733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.986737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:33.986740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.986743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:33.986746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.986749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:33.986752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.986755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:33.986757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.986760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:33.986763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.986766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.986768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.986771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.986775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.986778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.986793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:33.992945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:33.993976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:33.993994Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:33.993997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:33.994002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:33.994006Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2178 [2:7559342986755965152:11691]
2025-10-09T21:51:33.994018Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:33.994049Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:33.994052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:33.994055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:33.994058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:33.994072Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:33.994081Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:33.994087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:33.994133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.994140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.994142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.994145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:33.994201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:33.994207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:33.994214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:33.994220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:33.994232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:33.994256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342986755965147:11691]
2025-10-09T21:51:33.994263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342986755965147:11691]
2025-10-09T21:51:33.994266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342986755965147:11691]
2025-10-09T21:51:33.994279Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:33.994294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342987117808780:8854], now have 1 active actors on pipe
2025-10-09T21:51:33.994293Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:33.994297Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:33.994299Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:33.994302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342987117808507:2432], now have 1 active actors on pipe
2025-10-09T21:51:33.994309Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:33.994318Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:33.994320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:33.994339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342986755965147:11691]
2025-10-09T21:51:33.994349Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:33.994350Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2178
2025-10-09T21:51:33.994357Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:33.994352Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2178
2025-10-09T21:51:33.994420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.994430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.994461Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ff3e8cf3-213f2cf2-ceaf1abe-6cbba5f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:33.994473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:33.994477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:33.994484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:33.994495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:33.994510Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:33.994529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:33.994646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:33.994654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:33.994667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:33.994768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2179 partition: 0 MaxSeqNo: 25 sessionId: src_id|ff3e8cf3-213f2cf2-ceaf1abe-6cbba5f_0
2025-10-09T21:51:33.995102Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a13137c7-1df94fa2-b0e5311-2b5efa0f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ff3e8cf3-213f2cf2-ceaf1abe-6cbba5f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:33.997569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff3e8cf3-213f2cf2-ceaf1abe-6cbba5f_0] Write session: try to update token
2025-10-09T21:51:33.997588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff3e8cf3-213f2cf2-ceaf1abe-6cbba5f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:34.006140Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:34.006157Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:34.006167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342986755965147:11691]
2025-10-09T21:51:34.006218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:34.006220Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:34.006394Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:34.006552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:34.006685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:34.006816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:34.006998Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:34.007033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:34.010631Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2179 sessionId: src_id|ff3e8cf3-213f2cf2-ceaf1abe-6cbba5f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:34.012965Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:34.013498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:34.013518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:34.013559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:34.013655Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:34.019459Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:34.019479Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:34.019481Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:34.019483Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:34.019495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342986755965147:11691]
2025-10-09T21:51:34.019498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342986755965147:11691]
2025-10-09T21:51:34.019502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342986755965147:11691]
2025-10-09T21:51:34.019545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:34.019549Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:34.019553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.019556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:34.019565Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:34.019569Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:34.019703Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:34.067157Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342991412776216:9464]
2025-10-09T21:51:34.067433Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 2178, pipe: [1:7559342991412776216:9464]
2025-10-09T21:51:34.166283Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:34.173024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:34.173049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:34.173192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:34.173249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:34.173323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:34.173407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:34.173454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:34.173463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:34.173533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:34.173593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:34.173649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:34.173709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:34.173751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:34.173757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:34.173814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:34.173870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:34.173925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:34.173980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:34.174019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:34.174025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:34.174099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:34.174156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:34.174214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:34.174274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:34.174312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:34.174318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:34.174386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:34.174442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:34.174511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:34.174571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:34.174609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:34.174614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:34.174671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:34.174727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:34.174786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:34.174848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:34.174886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:34.174894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:34.174949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:34.175006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:34.175067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:34.175125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:34.175168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:34.175173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:34.175225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:34.175280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:34.175334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:34.175389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:34.175426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:34.175433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:34.175486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:34.175538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:34.175592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:34.175647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:34.175685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:34.175691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:34.175745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:34.175798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:34.175849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:34.175904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:34.175941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:34.175946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:34.175984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342991412776216:9464], now have 1 active actors on pipe
2025-10-09T21:51:34.176241Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:34.176287Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:34.176298Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342991412776216:9464]
2025-10-09T21:51:34.176300Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:34.176329Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2179 reason: error at write: Disconnected sessionId: src_id|ff3e8cf3-213f2cf2-ceaf1abe-6cbba5f_0
2025-10-09T21:51:34.176499Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2179 sessionId: src_id|ff3e8cf3-213f2cf2-ceaf1abe-6cbba5f_0 is DEAD
2025-10-09T21:51:34.176609Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff3e8cf3-213f2cf2-ceaf1abe-6cbba5f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:34.176630Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff3e8cf3-213f2cf2-ceaf1abe-6cbba5f_0] Write session will restart in 0.005748s
2025-10-09T21:51:34.176655Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff3e8cf3-213f2cf2-ceaf1abe-6cbba5f_0] Write session: Do CDS request
2025-10-09T21:51:34.176661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff3e8cf3-213f2cf2-ceaf1abe-6cbba5f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:34.176728Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:34.183464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff3e8cf3-213f2cf2-ceaf1abe-6cbba5f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:34.183512Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:34.183525Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2180
2025-10-09T21:51:34.183648Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2180 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:34.183675Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2180 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:34.183680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2180 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:34.183688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:34.184081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2180 sessionId:  describe result for acl check
2025-10-09T21:51:34.184116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:34.184122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:34.184124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:34.184130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342991412776252:8846] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:34.184642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342991412776252:8846] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:34.187304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342991412776252:8846] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:34.190374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342991412776252:8846] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:34.190386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342991412776252:8846] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:34.190389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342991412776252:8846] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:34.190395Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2180 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:34.193938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:34.195579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:34.195760Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:34.195770Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:34.195781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:34.195830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:34.195869Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342991050932707:11693]
2025-10-09T21:51:34.196279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:34.196548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:34.196591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:34.196811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:34.196856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.196862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.196863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.196865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.196866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.196867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.196868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.196870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.196871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.196875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.196886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.196890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.196894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.196897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.196901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.196905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.196908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.196911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.196918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.196926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.196932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.196938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.196946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.196949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.196951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.196955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.196957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.196960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.196962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.196965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.196969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.196972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.196974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.196977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.196980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.196982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.196998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:34.203170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:34.204292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:34.204310Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:34.204313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:34.204318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:34.204325Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2179 [2:7559342991050932707:11693]
2025-10-09T21:51:34.204342Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:34.204361Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:34.204364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:34.204368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:34.204370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:34.204389Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:34.204403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:34.204415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:34.204480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.204487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.204489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.204492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:34.204566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:34.204587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.204605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.204621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.204641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:34.204670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342991412776552:8846], now have 1 active actors on pipe
2025-10-09T21:51:34.204678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342991412776218:2432], now have 1 active actors on pipe
2025-10-09T21:51:34.204686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342991050932702:11693]
2025-10-09T21:51:34.204689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342991050932702:11693]
2025-10-09T21:51:34.204691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342991050932702:11693]
2025-10-09T21:51:34.204705Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:34.204723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:34.204733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:34.204737Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:34.204739Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:34.204742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:34.204746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:34.204731Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2179
2025-10-09T21:51:34.204744Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2179
2025-10-09T21:51:34.204782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342991050932702:11693]
2025-10-09T21:51:34.204793Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:34.204801Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:34.204810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:34.204816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:34.204864Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f401d7c6-c0aff02e-f83c8d4f-49400145_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:34.204877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:34.204882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:34.204885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:34.204896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:34.204911Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:34.204928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:34.205068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:34.205076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:34.205091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:34.205177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2180 partition: 0 MaxSeqNo: 25 sessionId: src_id|f401d7c6-c0aff02e-f83c8d4f-49400145_0
2025-10-09T21:51:34.205508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff3e8cf3-213f2cf2-ceaf1abe-6cbba5f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f401d7c6-c0aff02e-f83c8d4f-49400145_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:34.208050Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f401d7c6-c0aff02e-f83c8d4f-49400145_0] Write session: try to update token
2025-10-09T21:51:34.208070Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f401d7c6-c0aff02e-f83c8d4f-49400145_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:34.218512Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:34.218534Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:34.218536Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:34.218539Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:34.218546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342991050932702:11693]
2025-10-09T21:51:34.218549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342991050932702:11693]
2025-10-09T21:51:34.218552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342991050932702:11693]
2025-10-09T21:51:34.218587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:34.218593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.218595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:34.218597Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:34.218602Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:34.218606Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:34.218711Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:34.221601Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2180 sessionId: src_id|f401d7c6-c0aff02e-f83c8d4f-49400145_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:34.223996Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:34.276770Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342991412776638:9464]
2025-10-09T21:51:34.277010Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 2179, pipe: [1:7559342991412776638:9464]
2025-10-09T21:51:34.358625Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:34.359091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:34.359106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:34.359149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342991412776638:9464], now have 1 active actors on pipe
2025-10-09T21:51:34.359358Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342991412776638:9464]
2025-10-09T21:51:34.359362Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:34.359360Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:34.359366Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:34.359390Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:34.359421Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2180 reason: error at write: Disconnected sessionId: src_id|f401d7c6-c0aff02e-f83c8d4f-49400145_0
2025-10-09T21:51:34.359584Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2180 sessionId: src_id|f401d7c6-c0aff02e-f83c8d4f-49400145_0 is DEAD
2025-10-09T21:51:34.359711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f401d7c6-c0aff02e-f83c8d4f-49400145_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:34.359722Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f401d7c6-c0aff02e-f83c8d4f-49400145_0] Write session will restart in 0.005470s
2025-10-09T21:51:34.359757Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f401d7c6-c0aff02e-f83c8d4f-49400145_0] Write session: Do CDS request
2025-10-09T21:51:34.359764Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f401d7c6-c0aff02e-f83c8d4f-49400145_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:34.366463Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f401d7c6-c0aff02e-f83c8d4f-49400145_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:34.366496Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:34.366509Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2181
2025-10-09T21:51:34.366639Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2181 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:34.366678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2181 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:34.366688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2181 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:34.366692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:34.367102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2181 sessionId:  describe result for acl check
2025-10-09T21:51:34.367133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:34.367134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:34.367139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:34.367144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342991412776711:8838] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:34.367703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342991412776711:8838] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:34.369838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342991412776711:8838] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:34.372692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342991412776711:8838] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:34.372703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342991412776711:8838] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:34.372707Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342991412776711:8838] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:34.372714Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2181 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:34.375251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:34.376684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:34.376847Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:34.376858Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:34.376870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:34.376918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:34.376954Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342991050932967:11695]
2025-10-09T21:51:34.377502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:34.377747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:34.377783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:34.378088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:34.378140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.378145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.378147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.378148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.378149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.378150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.378151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.378152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.378153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.378158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.378169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.378173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.378177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.378180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.378184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.378188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.378191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.378195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.378201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.378208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.378214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.378219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.378222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.378226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.378229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.378234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.378239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.378243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.378252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.378257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.378261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.378266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.378271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.378275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.378280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.378290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.378318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:34.384092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:34.385078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:34.385094Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:34.385097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:34.385100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:34.385106Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2180 [2:7559342991050932967:11695]
2025-10-09T21:51:34.385117Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:34.385134Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:34.385141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:34.385144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:34.385146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:34.385161Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:34.385166Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:34.385179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:34.385230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.385237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.385239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.385242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:34.385297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:34.385304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.385316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.385328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.385359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:34.385383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342991412777007:8838], now have 1 active actors on pipe
2025-10-09T21:51:34.385393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342991412776646:2432], now have 1 active actors on pipe
2025-10-09T21:51:34.385399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342991050932962:11695]
2025-10-09T21:51:34.385402Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342991050932962:11695]
2025-10-09T21:51:34.385404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342991050932962:11695]
2025-10-09T21:51:34.385416Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:34.385424Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:34.385427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:34.385428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:34.385428Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:34.385423Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2180
2025-10-09T21:51:34.385440Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:34.385425Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2180
2025-10-09T21:51:34.385443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:34.385461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342991050932962:11695]
2025-10-09T21:51:34.385464Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:34.385465Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:34.385479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:34.385486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:34.385530Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f211894e-66276c31-316b616e-25f440ba_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:34.385542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:34.385546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:34.385548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:34.385557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:34.385566Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:34.385583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:34.385724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:34.385729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:34.385741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:34.385847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2181 partition: 0 MaxSeqNo: 25 sessionId: src_id|f211894e-66276c31-316b616e-25f440ba_0
2025-10-09T21:51:34.386192Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f401d7c6-c0aff02e-f83c8d4f-49400145_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f211894e-66276c31-316b616e-25f440ba_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:34.388575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f211894e-66276c31-316b616e-25f440ba_0] Write session: try to update token
2025-10-09T21:51:34.388592Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f211894e-66276c31-316b616e-25f440ba_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:34.399328Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:34.399348Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:34.399350Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:34.399352Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:34.399362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342991050932962:11695]
2025-10-09T21:51:34.399367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342991050932962:11695]
2025-10-09T21:51:34.399371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342991050932962:11695]
2025-10-09T21:51:34.399413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:34.399414Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:34.399420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.399421Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:34.399423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:34.399425Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:34.399564Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:34.400888Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2181 sessionId: src_id|f211894e-66276c31-316b616e-25f440ba_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:34.403063Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:34.459373Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342991412777063:9464]
2025-10-09T21:51:34.459572Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 2180, pipe: [1:7559342991412777063:9464]
2025-10-09T21:51:34.541682Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:34.542178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:34.542192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:34.542229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342991412777063:9464], now have 1 active actors on pipe
2025-10-09T21:51:34.542410Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342991412777063:9464]
2025-10-09T21:51:34.542441Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:34.542444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:34.542448Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:34.542497Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:34.542494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2181 reason: error at write: Disconnected sessionId: src_id|f211894e-66276c31-316b616e-25f440ba_0
2025-10-09T21:51:34.542652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2181 sessionId: src_id|f211894e-66276c31-316b616e-25f440ba_0 is DEAD
2025-10-09T21:51:34.542809Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f211894e-66276c31-316b616e-25f440ba_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:34.542820Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f211894e-66276c31-316b616e-25f440ba_0] Write session will restart in 0.007664s
2025-10-09T21:51:34.542841Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f211894e-66276c31-316b616e-25f440ba_0] Write session: Do CDS request
2025-10-09T21:51:34.542845Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f211894e-66276c31-316b616e-25f440ba_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:34.551504Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f211894e-66276c31-316b616e-25f440ba_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:34.551586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:34.551602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2182
2025-10-09T21:51:34.551717Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2182 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:34.551741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2182 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:34.551749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2182 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:34.551752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:34.552154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2182 sessionId:  describe result for acl check
2025-10-09T21:51:34.552184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:34.552190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:34.552192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:34.552198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342991412777219:8826] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:34.552747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342991412777219:8826] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:34.555249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342991412777219:8826] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:34.557671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342991412777219:8826] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:34.557684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342991412777219:8826] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:34.557688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342991412777219:8826] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:34.557694Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2182 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:34.558605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:34.560023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:34.560206Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:34.560216Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:34.560227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:34.560278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:34.560317Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342991050933229:11697]
2025-10-09T21:51:34.560872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:34.561115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:34.561155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:34.561462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:34.561523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.561529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.561530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.561531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.561532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.561534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.561535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.561536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.561537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.561542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.561552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.561556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.561561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.561564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.561568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.561572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.561576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.561580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.561588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.561598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.561607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.561614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.561619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.561628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.561633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.561642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.561647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.561656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.561660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.561670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.561675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.561684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.561688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.561693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.561700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.561705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.561738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:34.567871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:34.569051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:34.569068Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:34.569070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:34.569073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:34.569079Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2181 [2:7559342991050933229:11697]
2025-10-09T21:51:34.569092Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:34.569108Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:34.569115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:34.569118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:34.569120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:34.569138Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:34.569148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:34.569155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:34.569200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.569206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.569207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.569209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:34.569271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:34.569279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.569288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.569295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.569304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:34.569320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342991412777470:8826], now have 1 active actors on pipe
2025-10-09T21:51:34.569328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342991412777066:2432], now have 1 active actors on pipe
2025-10-09T21:51:34.569334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342991050933224:11697]
2025-10-09T21:51:34.569335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342991050933224:11697]
2025-10-09T21:51:34.569336Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342991050933224:11697]
2025-10-09T21:51:34.569359Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:34.569373Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:34.569380Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:34.569383Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:34.569384Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:34.569388Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:34.569391Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:34.569374Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2181
2025-10-09T21:51:34.569378Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2181
2025-10-09T21:51:34.569412Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342991050933224:11697]
2025-10-09T21:51:34.569422Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:34.569422Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:34.569445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:34.569452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:34.569491Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|99131fe3-f72b48fc-1f6d815f-e3b794cb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:34.569503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:34.569507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:34.569509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:34.569519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:34.569530Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:34.569556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:34.569682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:34.569693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:34.569709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:34.569795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2182 partition: 0 MaxSeqNo: 25 sessionId: src_id|99131fe3-f72b48fc-1f6d815f-e3b794cb_0
2025-10-09T21:51:34.570086Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f211894e-66276c31-316b616e-25f440ba_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|99131fe3-f72b48fc-1f6d815f-e3b794cb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:34.572483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99131fe3-f72b48fc-1f6d815f-e3b794cb_0] Write session: try to update token
2025-10-09T21:51:34.572499Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99131fe3-f72b48fc-1f6d815f-e3b794cb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:34.582925Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:34.582946Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:34.582949Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:34.582951Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:34.582959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342991050933224:11697]
2025-10-09T21:51:34.582964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342991050933224:11697]
2025-10-09T21:51:34.582969Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342991050933224:11697]
2025-10-09T21:51:34.583008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:34.583007Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:34.583011Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:34.583014Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:34.583015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.583017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:34.583149Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:34.584698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2182 sessionId: src_id|99131fe3-f72b48fc-1f6d815f-e3b794cb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:34.586790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:34.642676Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342991412777484:9464]
2025-10-09T21:51:34.642975Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 2181, pipe: [1:7559342991412777484:9464]
2025-10-09T21:51:34.724408Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:34.724952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:34.724967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:34.725005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342991412777484:9464], now have 1 active actors on pipe
2025-10-09T21:51:34.725160Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:34.725226Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:34.725241Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:34.725253Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:34.725288Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342991412777484:9464]
2025-10-09T21:51:34.725307Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2182 reason: error at write: Disconnected sessionId: src_id|99131fe3-f72b48fc-1f6d815f-e3b794cb_0
2025-10-09T21:51:34.725504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2182 sessionId: src_id|99131fe3-f72b48fc-1f6d815f-e3b794cb_0 is DEAD
2025-10-09T21:51:34.725652Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99131fe3-f72b48fc-1f6d815f-e3b794cb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:34.725661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99131fe3-f72b48fc-1f6d815f-e3b794cb_0] Write session will restart in 0.008339s
2025-10-09T21:51:34.725694Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99131fe3-f72b48fc-1f6d815f-e3b794cb_0] Write session: Do CDS request
2025-10-09T21:51:34.725699Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99131fe3-f72b48fc-1f6d815f-e3b794cb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:34.735477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|99131fe3-f72b48fc-1f6d815f-e3b794cb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:34.735574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:34.735586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2183
2025-10-09T21:51:34.735692Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2183 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:34.735718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2183 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:34.735722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2183 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:34.735728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:34.736190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2183 sessionId:  describe result for acl check
2025-10-09T21:51:34.736224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:34.736231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:34.736232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:34.736239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342991412777718:8827] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:34.736877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342991412777718:8827] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:34.738931Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342991412777718:8827] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:34.740602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:34.741972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:34.742059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342991412777718:8827] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:34.742070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342991412777718:8827] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:34.742072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342991412777718:8827] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:34.742078Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2183 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:34.742159Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:34.742169Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:34.742181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:34.742230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:34.742273Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342991050933490:11699]
2025-10-09T21:51:34.742858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:34.743141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:34.743184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:34.743496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:34.743540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.743546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.743547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.743548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.743550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.743551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.743552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.743553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.743554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.743559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.743569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.743573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.743577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.743581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.743585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.743589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.743594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.743598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.743604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.743612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.743618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.743623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.743628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.743633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.743638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.743643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.743647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.743653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.743657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.743664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.743675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.743680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.743689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.743694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.743699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.743704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.743738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:34.749822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:34.750900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:34.750917Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:34.750919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:34.750923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:34.750929Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2182 [2:7559342991050933490:11699]
2025-10-09T21:51:34.750942Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:34.750958Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:34.750966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:34.750969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:34.750972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:34.750991Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:34.751002Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:34.751009Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:34.751064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.751071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.751073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.751076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:34.751143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:34.751151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.751162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.751172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.751192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:34.751217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342991412777896:8827], now have 1 active actors on pipe
2025-10-09T21:51:34.751226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342991412777488:2432], now have 1 active actors on pipe
2025-10-09T21:51:34.751234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342991050933485:11699]
2025-10-09T21:51:34.751241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342991050933485:11699]
2025-10-09T21:51:34.751243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342991050933485:11699]
2025-10-09T21:51:34.751257Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:34.751276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:34.751246Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2182
2025-10-09T21:51:34.751278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:34.751254Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2182
2025-10-09T21:51:34.751282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:34.751283Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:34.751295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:34.751299Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:34.751335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:34.751345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:34.751362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342991050933485:11699]
2025-10-09T21:51:34.751374Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:34.751386Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:34.751409Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1b048665-d231092c-335278da-caf98f9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:34.751421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:34.751425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:34.751428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:34.751438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:34.751448Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:34.751477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:34.751638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:34.751649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:34.751664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:34.751767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2183 partition: 0 MaxSeqNo: 25 sessionId: src_id|1b048665-d231092c-335278da-caf98f9_0
2025-10-09T21:51:34.752086Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|99131fe3-f72b48fc-1f6d815f-e3b794cb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1b048665-d231092c-335278da-caf98f9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:34.754535Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b048665-d231092c-335278da-caf98f9_0] Write session: try to update token
2025-10-09T21:51:34.754551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b048665-d231092c-335278da-caf98f9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:34.765416Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:34.765438Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:34.765440Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:34.765443Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:34.765452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342991050933485:11699]
2025-10-09T21:51:34.765457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342991050933485:11699]
2025-10-09T21:51:34.765461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342991050933485:11699]
2025-10-09T21:51:34.765505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:34.765505Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:34.765509Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:34.765513Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:34.765520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.765524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:34.765677Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:34.767588Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2183 sessionId: src_id|1b048665-d231092c-335278da-caf98f9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:34.769907Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:34.825451Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342991412777910:9464]
2025-10-09T21:51:34.825740Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 2182, pipe: [1:7559342991412777910:9464]
2025-10-09T21:51:34.910815Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:34.911024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:34.911039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:34.911073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342991412777910:9464], now have 1 active actors on pipe
2025-10-09T21:51:34.911208Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:34.911239Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342991412777910:9464]
2025-10-09T21:51:34.911262Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:34.911271Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:34.911338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2183 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1b048665-d231092c-335278da-caf98f9_0
2025-10-09T21:51:34.911478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2183 sessionId: src_id|1b048665-d231092c-335278da-caf98f9_0 is DEAD
2025-10-09T21:51:34.911646Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b048665-d231092c-335278da-caf98f9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:34.911656Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b048665-d231092c-335278da-caf98f9_0] Write session will restart in 0.005432s
2025-10-09T21:51:34.911682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b048665-d231092c-335278da-caf98f9_0] Write session: Do CDS request
2025-10-09T21:51:34.911686Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b048665-d231092c-335278da-caf98f9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:34.918461Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1b048665-d231092c-335278da-caf98f9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:34.918534Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:34.918548Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2184
2025-10-09T21:51:34.918685Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2184 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:34.918719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2184 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:34.918723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2184 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:34.918726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:34.919126Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2184 sessionId:  describe result for acl check
2025-10-09T21:51:34.919160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:34.919162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:34.919164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:34.919169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342991412778062:8809] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:34.919656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342991412778062:8809] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:34.921664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342991412778062:8809] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:34.924013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342991412778062:8809] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:34.924039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342991412778062:8809] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:34.924042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342991412778062:8809] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:34.924048Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2184 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:34.925711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:34.927000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:34.927167Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:34.927178Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:34.927189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:34.927237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:34.927276Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342991050933797:11715]
2025-10-09T21:51:34.927686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:34.927951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:34.927989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:34.928261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:34.928309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.928317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.928318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.928319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.928320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.928321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.928322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.928323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.928324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.928330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.928340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.928344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.928348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.928352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.928355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.928359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.928363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.928366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.928372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.928379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:34.928385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.928389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:34.928396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.928400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:34.928408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.928412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:34.928420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.928424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:34.928428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.928436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:34.928440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.928445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.928453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.928458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.928466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.928473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.928501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:34.934544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:34.935607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:34.935623Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:34.935625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:34.935630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:34.935637Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2183 [2:7559342991050933797:11715]
2025-10-09T21:51:34.935650Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:34.935670Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:34.935673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:34.935676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:34.935679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:34.935694Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:34.935705Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:34.935716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:34.935774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.935780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.935782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.935785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:34.935850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:34.935858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.935873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:34.935891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:34.935907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:34.935934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342991412777940:2432], now have 1 active actors on pipe
2025-10-09T21:51:34.935943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342991412778335:8809], now have 1 active actors on pipe
2025-10-09T21:51:34.935950Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342991050933792:11715]
2025-10-09T21:51:34.935956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342991050933792:11715]
2025-10-09T21:51:34.935958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342991050933792:11715]
2025-10-09T21:51:34.935967Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:34.935980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:34.935980Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:34.935982Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:34.935984Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:34.935946Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2183
2025-10-09T21:51:34.936000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:34.936008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:34.935953Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2183
2025-10-09T21:51:34.936051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:34.936060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:34.936072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342991050933792:11715]
2025-10-09T21:51:34.936083Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:34.936086Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:34.936114Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e1abfc7d-4b0efbbd-77a4ac4a-6822c819_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:34.936128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:34.936132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:34.936134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:34.936144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:34.936155Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:34.936179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:34.936343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:34.936354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:34.936370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:34.936455Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2184 partition: 0 MaxSeqNo: 25 sessionId: src_id|e1abfc7d-4b0efbbd-77a4ac4a-6822c819_0
2025-10-09T21:51:34.936729Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1b048665-d231092c-335278da-caf98f9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e1abfc7d-4b0efbbd-77a4ac4a-6822c819_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:34.939143Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1abfc7d-4b0efbbd-77a4ac4a-6822c819_0] Write session: try to update token
2025-10-09T21:51:34.939160Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1abfc7d-4b0efbbd-77a4ac4a-6822c819_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:34.949163Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:34.949183Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:34.949186Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:34.949188Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:34.949197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342991050933792:11715]
2025-10-09T21:51:34.949202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342991050933792:11715]
2025-10-09T21:51:34.949205Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342991050933792:11715]
2025-10-09T21:51:34.949241Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:34.949246Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:34.949246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:34.949248Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:34.949253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:34.949257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:34.949434Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:34.951167Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2184 sessionId: src_id|e1abfc7d-4b0efbbd-77a4ac4a-6822c819_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:34.953421Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:35.011304Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342995707745653:9464]
2025-10-09T21:51:35.011593Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 2183, pipe: [1:7559342995707745653:9464]
2025-10-09T21:51:35.093938Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:35.094676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:35.094692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:35.094731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342995707745653:9464], now have 1 active actors on pipe
2025-10-09T21:51:35.094913Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:35.094975Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:35.094988Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342995707745653:9464]
2025-10-09T21:51:35.095016Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:35.095079Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2184 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e1abfc7d-4b0efbbd-77a4ac4a-6822c819_0
2025-10-09T21:51:35.095256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2184 sessionId: src_id|e1abfc7d-4b0efbbd-77a4ac4a-6822c819_0 is DEAD
2025-10-09T21:51:35.095389Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1abfc7d-4b0efbbd-77a4ac4a-6822c819_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:35.095397Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1abfc7d-4b0efbbd-77a4ac4a-6822c819_0] Write session will restart in 0.005016s
2025-10-09T21:51:35.095437Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1abfc7d-4b0efbbd-77a4ac4a-6822c819_0] Write session: Do CDS request
2025-10-09T21:51:35.095444Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1abfc7d-4b0efbbd-77a4ac4a-6822c819_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:35.101505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1abfc7d-4b0efbbd-77a4ac4a-6822c819_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:35.101627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:35.101647Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2185
2025-10-09T21:51:35.101793Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2185 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:35.101820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2185 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:35.101829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2185 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:35.101832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:35.102297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2185 sessionId:  describe result for acl check
2025-10-09T21:51:35.102333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:35.102335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:35.102336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:35.102342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342995707745701:8804] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:35.102918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342995707745701:8804] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:35.105137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342995707745701:8804] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:35.107825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342995707745701:8804] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:35.107841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342995707745701:8804] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:35.107845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342995707745701:8804] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:35.107854Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2185 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:35.110921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:35.112859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:35.113071Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:35.113082Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:35.113093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:35.113143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:35.113190Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342995345901356:11717]
2025-10-09T21:51:35.113778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:35.114083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:35.114121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:35.114370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:35.114412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.114419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.114420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.114421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.114422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.114423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.114424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.114425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.114426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.114430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.114441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.114445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.114449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.114453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.114457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.114461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.114464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.114467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.114475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.114483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.114489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.114493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.114498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.114503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.114507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.114513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.114518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.114522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.114527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.114530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.114534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.114541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.114544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.114547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.114549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.114554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.114571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:35.120982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:35.122671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:35.122689Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:35.122692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:35.122697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:35.122704Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2184 [2:7559342995345901356:11717]
2025-10-09T21:51:35.122719Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:35.122737Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:35.122745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:35.122749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:35.122751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:35.122771Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:35.122782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:35.122792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:35.122861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.122868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.122871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.122874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:35.122949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:35.122957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.122971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.122984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.123005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:35.123031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342995707745656:2432], now have 1 active actors on pipe
2025-10-09T21:51:35.123040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342995707746002:8804], now have 1 active actors on pipe
2025-10-09T21:51:35.123048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342995345901351:11717]
2025-10-09T21:51:35.123055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342995345901351:11717]
2025-10-09T21:51:35.123057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342995345901351:11717]
2025-10-09T21:51:35.123074Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:35.123094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:35.123099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:35.123103Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:35.123105Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:35.123110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:35.123115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:35.123113Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2184
2025-10-09T21:51:35.123154Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342995345901351:11717]
2025-10-09T21:51:35.123162Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:35.123164Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:35.123117Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2184
2025-10-09T21:51:35.123194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:35.123204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:35.123241Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dd91a987-81c2c34b-b94b50a0-e7caac66_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:35.123254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:35.123258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:35.123261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:35.123272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:35.123286Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:35.123309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:35.123463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:35.123474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:35.123490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:35.123582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2185 partition: 0 MaxSeqNo: 25 sessionId: src_id|dd91a987-81c2c34b-b94b50a0-e7caac66_0
2025-10-09T21:51:35.123951Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1abfc7d-4b0efbbd-77a4ac4a-6822c819_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dd91a987-81c2c34b-b94b50a0-e7caac66_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:35.126436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dd91a987-81c2c34b-b94b50a0-e7caac66_0] Write session: try to update token
2025-10-09T21:51:35.126452Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dd91a987-81c2c34b-b94b50a0-e7caac66_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:35.138966Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:35.138999Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:35.139002Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:35.139004Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:35.139019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342995345901351:11717]
2025-10-09T21:51:35.139027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342995345901351:11717]
2025-10-09T21:51:35.139031Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342995345901351:11717]
2025-10-09T21:51:35.139092Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:35.139105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:35.139106Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:35.139111Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:35.139116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.139119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:35.139413Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:35.140971Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2185 sessionId: src_id|dd91a987-81c2c34b-b94b50a0-e7caac66_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:35.143321Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:35.195136Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342995707746078:9464]
2025-10-09T21:51:35.195395Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 2184, pipe: [1:7559342995707746078:9464]
2025-10-09T21:51:35.287347Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:35.287727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:35.287741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:35.287780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342995707746078:9464], now have 1 active actors on pipe
2025-10-09T21:51:35.287958Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:35.288008Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:35.288011Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342995707746078:9464]
2025-10-09T21:51:35.288016Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:35.288078Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2185 reason: error at write: Disconnected sessionId: src_id|dd91a987-81c2c34b-b94b50a0-e7caac66_0
2025-10-09T21:51:35.288098Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:35.288241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2185 sessionId: src_id|dd91a987-81c2c34b-b94b50a0-e7caac66_0 is DEAD
2025-10-09T21:51:35.288389Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd91a987-81c2c34b-b94b50a0-e7caac66_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:35.288397Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd91a987-81c2c34b-b94b50a0-e7caac66_0] Write session will restart in 0.007638s
2025-10-09T21:51:35.288418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd91a987-81c2c34b-b94b50a0-e7caac66_0] Write session: Do CDS request
2025-10-09T21:51:35.288423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd91a987-81c2c34b-b94b50a0-e7caac66_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:35.297502Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dd91a987-81c2c34b-b94b50a0-e7caac66_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:35.297602Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:35.297616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2186
2025-10-09T21:51:35.297731Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2186 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:35.297758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2186 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:35.297765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2186 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:35.297769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:35.298201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2186 sessionId:  describe result for acl check
2025-10-09T21:51:35.298239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:35.298244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:35.298245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:35.298250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342995707746262:8796] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:35.298766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342995707746262:8796] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:35.301508Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342995707746262:8796] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:35.304085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342995707746262:8796] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:35.304097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342995707746262:8796] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:35.304100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342995707746262:8796] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:35.304107Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2186 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:35.305734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:35.307741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:35.307893Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:35.307901Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:35.307910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:35.307945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:35.307981Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342995345901622:11719]
2025-10-09T21:51:35.308398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:35.308726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:35.308773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:35.309057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:35.309105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.309112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.309113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.309114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.309116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.309117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.309118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.309119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.309121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.309126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.309138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.309142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.309147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.309151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.309155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.309159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.309163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.309167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.309173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.309181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.309187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.309198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.309202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.309208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.309217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.309223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.309228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.309233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.309238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.309242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.309248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.309253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.309258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.309264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.309276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.309286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.309316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:35.315906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:35.317024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:35.317041Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:35.317044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:35.317049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:35.317056Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2185 [2:7559342995345901622:11719]
2025-10-09T21:51:35.317074Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:35.317098Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:35.317109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:35.317112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:35.317115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:35.317134Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:35.317141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:35.317149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:35.317214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.317225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.317227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.317230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:35.317305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:35.317313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.317324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.317342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.317377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:35.317405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342995707746496:8796], now have 1 active actors on pipe
2025-10-09T21:51:35.317414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342995707746083:2432], now have 1 active actors on pipe
2025-10-09T21:51:35.317422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342995345901617:11719]
2025-10-09T21:51:35.317428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342995345901617:11719]
2025-10-09T21:51:35.317431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342995345901617:11719]
2025-10-09T21:51:35.317449Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:35.317468Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:35.317481Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:35.317483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:35.317488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:35.317471Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2185
2025-10-09T21:51:35.317500Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:35.317503Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:35.317524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342995345901617:11719]
2025-10-09T21:51:35.317474Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2185
2025-10-09T21:51:35.317528Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:35.317536Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:35.317544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:35.317550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:35.317607Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2e364856-df1b586d-ef3acbef-aa20aacc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:35.317998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:35.318017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:35.318022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:35.318053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:35.318070Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:35.318101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:35.318296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:35.318307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:35.318324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:35.318425Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2186 partition: 0 MaxSeqNo: 25 sessionId: src_id|2e364856-df1b586d-ef3acbef-aa20aacc_0
2025-10-09T21:51:35.318761Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd91a987-81c2c34b-b94b50a0-e7caac66_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2e364856-df1b586d-ef3acbef-aa20aacc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:35.321213Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e364856-df1b586d-ef3acbef-aa20aacc_0] Write session: try to update token
2025-10-09T21:51:35.321230Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e364856-df1b586d-ef3acbef-aa20aacc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:35.333099Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:35.333117Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:35.333119Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:35.333121Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:35.333131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342995345901617:11719]
2025-10-09T21:51:35.333135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342995345901617:11719]
2025-10-09T21:51:35.333139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342995345901617:11719]
2025-10-09T21:51:35.333187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:35.333199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.333199Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:35.333206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:35.333208Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:35.333213Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:35.333339Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:35.334938Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2186 sessionId: src_id|2e364856-df1b586d-ef3acbef-aa20aacc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:35.337498Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:35.388156Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342995707746511:9464]
2025-10-09T21:51:35.388533Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 2185, pipe: [1:7559342995707746511:9464]
2025-10-09T21:51:35.478488Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:35.478732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:35.478752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:35.478798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342995707746511:9464], now have 1 active actors on pipe
2025-10-09T21:51:35.479035Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:35.479102Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:35.479118Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:35.479131Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342995707746511:9464]
2025-10-09T21:51:35.479149Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:35.479176Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2186 reason: error at write: Disconnected sessionId: src_id|2e364856-df1b586d-ef3acbef-aa20aacc_0
2025-10-09T21:51:35.479327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2186 sessionId: src_id|2e364856-df1b586d-ef3acbef-aa20aacc_0 is DEAD
2025-10-09T21:51:35.479455Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e364856-df1b586d-ef3acbef-aa20aacc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:35.479464Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e364856-df1b586d-ef3acbef-aa20aacc_0] Write session will restart in 0.006375s
2025-10-09T21:51:35.479483Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e364856-df1b586d-ef3acbef-aa20aacc_0] Write session: Do CDS request
2025-10-09T21:51:35.479488Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e364856-df1b586d-ef3acbef-aa20aacc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:35.486355Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e364856-df1b586d-ef3acbef-aa20aacc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:35.486488Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:35.486499Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2187
2025-10-09T21:51:35.486641Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2187 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:35.486687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2187 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:35.486697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2187 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:35.486705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:35.487106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2187 sessionId:  describe result for acl check
2025-10-09T21:51:35.487143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:35.487151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:35.487153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:35.487159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342995707746547:8788] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:35.487684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342995707746547:8788] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:35.490225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342995707746547:8788] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:35.492890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342995707746547:8788] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:35.492906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342995707746547:8788] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:35.492910Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342995707746547:8788] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:35.492918Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2187 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:35.498330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:35.500051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:35.500242Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:35.500250Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:35.500260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:35.500313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:35.500352Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342995345901888:11721]
2025-10-09T21:51:35.500760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:35.501021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:35.501052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:35.501270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:35.501309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.501314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.501315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.501315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.501316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.501317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.501318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.501318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.501319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.501322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.501329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.501331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.501333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.501335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.501337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.501339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.501341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.501354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.501359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.501364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.501369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.501372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.501376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.501381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.501386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.501391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.501396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.501401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.501408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.501412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.501415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.501420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.501422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.501425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.501431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.501436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.501455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:35.508313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:35.509512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:35.509527Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:35.509529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:35.509533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:35.509538Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2186 [2:7559342995345901888:11721]
2025-10-09T21:51:35.509548Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:35.509565Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:35.509570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:35.509573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:35.509574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:35.509589Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:35.509601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:35.509607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:35.509670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.509679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.509681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.509684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:35.509754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:35.509762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.509770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.509777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.509787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:35.509802Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342995345901883:11721]
2025-10-09T21:51:35.509808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342995345901883:11721]
2025-10-09T21:51:35.509809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342995345901883:11721]
2025-10-09T21:51:35.509817Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:35.509829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:35.509835Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:35.509840Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:35.509842Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:35.509843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:35.509844Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:35.509854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342995707746795:8788], now have 1 active actors on pipe
2025-10-09T21:51:35.509861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342995707746513:2432], now have 1 active actors on pipe
2025-10-09T21:51:35.509886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342995345901883:11721]
2025-10-09T21:51:35.509896Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:35.509924Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:35.509938Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2186
2025-10-09T21:51:35.509940Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2186
2025-10-09T21:51:35.510166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:35.510184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:35.510217Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f2a4846a-1b7efed7-6856def5-8613ed8b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:35.510235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:35.510240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:35.510244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:35.510256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:35.510273Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:35.510297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:35.510523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:35.510549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:35.510582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:35.510701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2187 partition: 0 MaxSeqNo: 25 sessionId: src_id|f2a4846a-1b7efed7-6856def5-8613ed8b_0
2025-10-09T21:51:35.511055Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e364856-df1b586d-ef3acbef-aa20aacc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f2a4846a-1b7efed7-6856def5-8613ed8b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:35.513572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a4846a-1b7efed7-6856def5-8613ed8b_0] Write session: try to update token
2025-10-09T21:51:35.513593Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a4846a-1b7efed7-6856def5-8613ed8b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:35.525284Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:35.525309Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:35.525312Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:35.525313Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:35.525321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342995345901883:11721]
2025-10-09T21:51:35.525325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342995345901883:11721]
2025-10-09T21:51:35.525328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342995345901883:11721]
2025-10-09T21:51:35.525378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:35.525385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.525387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:35.525396Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:35.525402Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:35.525406Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:35.525507Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:35.526818Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2187 sessionId: src_id|f2a4846a-1b7efed7-6856def5-8613ed8b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:35.529263Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:35.579096Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342995707746937:9464]
2025-10-09T21:51:35.579404Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 2186, pipe: [1:7559342995707746937:9464]
2025-10-09T21:51:35.663910Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:35.664402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:35.664417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:35.664454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342995707746937:9464], now have 1 active actors on pipe
2025-10-09T21:51:35.664616Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:35.664718Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:35.664757Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342995707746937:9464]
2025-10-09T21:51:35.664773Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:35.664814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2187 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f2a4846a-1b7efed7-6856def5-8613ed8b_0
2025-10-09T21:51:35.664984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2187 sessionId: src_id|f2a4846a-1b7efed7-6856def5-8613ed8b_0 is DEAD
2025-10-09T21:51:35.665117Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a4846a-1b7efed7-6856def5-8613ed8b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:35.665128Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a4846a-1b7efed7-6856def5-8613ed8b_0] Write session will restart in 0.009006s
2025-10-09T21:51:35.665153Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a4846a-1b7efed7-6856def5-8613ed8b_0] Write session: Do CDS request
2025-10-09T21:51:35.665159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a4846a-1b7efed7-6856def5-8613ed8b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:35.675437Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a4846a-1b7efed7-6856def5-8613ed8b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:35.675542Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:35.675559Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2188
2025-10-09T21:51:35.675683Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2188 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:35.675715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2188 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:35.675727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2188 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:35.675731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:35.676121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2188 sessionId:  describe result for acl check
2025-10-09T21:51:35.676158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:35.676164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:35.676166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:35.676181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342995707747163:8780] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:35.676718Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342995707747163:8780] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:35.678930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342995707747163:8780] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:35.680906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342995707747163:8780] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:35.680921Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342995707747163:8780] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:35.680925Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342995707747163:8780] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:35.680931Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2188 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:35.681130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:35.682752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:35.682921Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:35.682929Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:35.682938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:35.682986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:35.683014Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342995345902155:11723]
2025-10-09T21:51:35.683376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:35.683664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:35.683693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:35.683882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:35.683923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.683929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.683930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.683931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.683931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.683932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.683932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.683933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.683934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.683937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.683943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.683945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.683947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.683949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.683951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.683953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.683955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.683957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.683962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.683967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.683972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.683975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.683977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.683980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.683982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.683985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.683988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.683990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.683993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.683996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.683998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.684001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.684004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.684006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.684009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.684011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.684026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:35.690080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:35.691088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:35.691104Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:35.691106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:35.691110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:35.691116Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2187 [2:7559342995345902155:11723]
2025-10-09T21:51:35.691127Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:35.691139Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:35.691140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:35.691142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:35.691143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:35.691154Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:35.691162Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:35.691167Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:35.691205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.691210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.691211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.691213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:35.691259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:35.691264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.691272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.691278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.691287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:35.691311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342995345902150:11723]
2025-10-09T21:51:35.691317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342995345902150:11723]
2025-10-09T21:51:35.691319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342995345902150:11723]
2025-10-09T21:51:35.691328Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:35.691342Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:35.691344Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:35.691347Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:35.691348Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:35.691357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:35.691359Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:35.691367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342995707747351:8780], now have 1 active actors on pipe
2025-10-09T21:51:35.691370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342995707746939:2432], now have 1 active actors on pipe
2025-10-09T21:51:35.691384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342995345902150:11723]
2025-10-09T21:51:35.691394Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:35.691413Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:35.691418Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2187
2025-10-09T21:51:35.691425Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2187
2025-10-09T21:51:35.691514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:35.691521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:35.691542Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d3f51ed8-b3db0941-38b5bf71-5e12ebc6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:35.691550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:35.691553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:35.691554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:35.691561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:35.691568Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:35.691585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:35.691699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:35.691704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:35.691714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:35.691777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2188 partition: 0 MaxSeqNo: 25 sessionId: src_id|d3f51ed8-b3db0941-38b5bf71-5e12ebc6_0
2025-10-09T21:51:35.692077Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f2a4846a-1b7efed7-6856def5-8613ed8b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d3f51ed8-b3db0941-38b5bf71-5e12ebc6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:35.694478Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d3f51ed8-b3db0941-38b5bf71-5e12ebc6_0] Write session: try to update token
2025-10-09T21:51:35.694496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d3f51ed8-b3db0941-38b5bf71-5e12ebc6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:35.705754Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:35.705773Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:35.705775Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:35.705777Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:35.705785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342995345902150:11723]
2025-10-09T21:51:35.705799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342995345902150:11723]
2025-10-09T21:51:35.705802Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342995345902150:11723]
2025-10-09T21:51:35.705838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:35.705843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.705845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:35.705852Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:35.705867Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:35.705879Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:35.705966Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:35.707149Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2188 sessionId: src_id|d3f51ed8-b3db0941-38b5bf71-5e12ebc6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:35.709385Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:35.764986Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342995707747364:9464]
2025-10-09T21:51:35.765327Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 2187, pipe: [1:7559342995707747364:9464]
2025-10-09T21:51:35.846059Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:35.846284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:35.846296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:35.846331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342995707747364:9464], now have 1 active actors on pipe
2025-10-09T21:51:35.846501Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:35.846607Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:35.846621Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:35.846638Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342995707747364:9464]
2025-10-09T21:51:35.846654Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:35.846690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2188 reason: error at write: Disconnected sessionId: src_id|d3f51ed8-b3db0941-38b5bf71-5e12ebc6_0
2025-10-09T21:51:35.846844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2188 sessionId: src_id|d3f51ed8-b3db0941-38b5bf71-5e12ebc6_0 is DEAD
2025-10-09T21:51:35.846975Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3f51ed8-b3db0941-38b5bf71-5e12ebc6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:35.846982Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3f51ed8-b3db0941-38b5bf71-5e12ebc6_0] Write session will restart in 0.008122s
2025-10-09T21:51:35.847001Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3f51ed8-b3db0941-38b5bf71-5e12ebc6_0] Write session: Do CDS request
2025-10-09T21:51:35.847006Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3f51ed8-b3db0941-38b5bf71-5e12ebc6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:35.856558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d3f51ed8-b3db0941-38b5bf71-5e12ebc6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:35.856608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:35.856625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2189
2025-10-09T21:51:35.856753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2189 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:35.856788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2189 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:35.856795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2189 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:35.856798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:35.857120Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2189 sessionId:  describe result for acl check
2025-10-09T21:51:35.857146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:35.857150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:35.857151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:35.857155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559342995707747593:8772] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:35.857593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559342995707747593:8772] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:35.859745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559342995707747593:8772] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:35.862354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559342995707747593:8772] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:35.862367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559342995707747593:8772] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:35.862369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559342995707747593:8772] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:35.862376Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2189 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:35.862927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:35.864603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:35.864764Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:35.864772Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:35.864781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:35.864818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:35.864850Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342995345902427:11727]
2025-10-09T21:51:35.865208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:35.865488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:35.865520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:35.865730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:35.865763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.865764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.865764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.865765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.865766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.865766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.865767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.865767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.865768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.865771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.865778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.865780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.865783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.865785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.865787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.865789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.865790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.865792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.865797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.865802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:35.865806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.865809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:35.865812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.865815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:35.865817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.865820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:35.865823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.865826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:35.865828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.865831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:35.865834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.865837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.865839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.865842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.865845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.865847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.865864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:35.871949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:35.873054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:35.873064Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:35.873066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:35.873069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:35.873074Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2188 [2:7559342995345902427:11727]
2025-10-09T21:51:35.873084Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:35.873099Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:35.873100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:35.873103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:35.873104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:35.873117Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:35.873120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:35.873125Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:35.873164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.873170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.873171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.873173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:35.873223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:35.873228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.873234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:35.873240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:35.873249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:35.873267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342995345902422:11727]
2025-10-09T21:51:35.873271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342995345902422:11727]
2025-10-09T21:51:35.873272Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342995345902422:11727]
2025-10-09T21:51:35.873280Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:35.873290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:35.873293Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:35.873296Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:35.873299Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:35.873305Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:35.873307Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:35.873318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342995707747781:8772], now have 1 active actors on pipe
2025-10-09T21:51:35.873323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342995707747369:2432], now have 1 active actors on pipe
2025-10-09T21:51:35.873360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342995345902422:11727]
2025-10-09T21:51:35.873370Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:35.873390Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:35.873403Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2188
2025-10-09T21:51:35.873406Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2188
2025-10-09T21:51:35.873482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:35.873488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:35.873513Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|688789b0-e039c485-c775a719-76f41ed5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:35.873521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:35.873524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:35.873526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:35.873535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:35.873545Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:35.873560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:35.873719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:35.873729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:35.873752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:35.873853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2189 partition: 0 MaxSeqNo: 25 sessionId: src_id|688789b0-e039c485-c775a719-76f41ed5_0
2025-10-09T21:51:35.874212Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3f51ed8-b3db0941-38b5bf71-5e12ebc6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|688789b0-e039c485-c775a719-76f41ed5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:35.876528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|688789b0-e039c485-c775a719-76f41ed5_0] Write session: try to update token
2025-10-09T21:51:35.876543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|688789b0-e039c485-c775a719-76f41ed5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:35.883904Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:35.883917Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:35.883924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342995345902422:11727]
2025-10-09T21:51:35.883961Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:35.883972Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:35.884113Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:35.884241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:35.884362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:35.884479Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:35.884641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:35.884671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:35.888363Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2189 sessionId: src_id|688789b0-e039c485-c775a719-76f41ed5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:35.890828Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:35.891018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:35.891032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:35.891062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:35.891151Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:35.896087Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:35.896098Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:35.896099Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:35.896100Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:35.896107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342995345902422:11727]
2025-10-09T21:51:35.896110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342995345902422:11727]
2025-10-09T21:51:35.896112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342995345902422:11727]
2025-10-09T21:51:35.896144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:35.896149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:35.896151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:35.896155Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:35.896160Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:35.896164Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:35.896276Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:35.946741Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559342995707747818:9464]
2025-10-09T21:51:35.946987Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 2188, pipe: [1:7559342995707747818:9464]
2025-10-09T21:51:36.040832Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:36.048756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.048778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.048939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.049000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.049066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.049129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.049470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.049481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.049547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.049607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.049668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.049727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.049769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.049777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.049835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.049893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.049954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.050010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.050070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.050077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.050141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.050202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.050267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.050328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.050368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.050375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.050441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.050499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.050559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.050626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.050666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.050673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.050733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.050794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.050853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.050914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.050954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.050961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.051021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.051074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.051130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.051188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.051229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.051237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.051298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.051354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.051413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.051470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.051513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.051519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.051578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.051632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.051694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.051753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.051797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.051805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.051862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.051917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.051973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.052033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.052075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.052082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.052140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559342995707747818:9464], now have 1 active actors on pipe
2025-10-09T21:51:36.052387Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:36.052400Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:36.052404Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:36.052436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2189 reason: error at write: Disconnected sessionId: src_id|688789b0-e039c485-c775a719-76f41ed5_0
2025-10-09T21:51:36.052449Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559342995707747818:9464]
2025-10-09T21:51:36.052624Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2189 sessionId: src_id|688789b0-e039c485-c775a719-76f41ed5_0 is DEAD
2025-10-09T21:51:36.052762Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|688789b0-e039c485-c775a719-76f41ed5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:36.052773Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|688789b0-e039c485-c775a719-76f41ed5_0] Write session will restart in 0.006232s
2025-10-09T21:51:36.052791Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|688789b0-e039c485-c775a719-76f41ed5_0] Write session: Do CDS request
2025-10-09T21:51:36.052796Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|688789b0-e039c485-c775a719-76f41ed5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:36.052792Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:36.060445Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|688789b0-e039c485-c775a719-76f41ed5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:36.060505Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:36.060521Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2190
2025-10-09T21:51:36.060633Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2190 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:36.060655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2190 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:36.060668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2190 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:36.060671Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:36.061082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2190 sessionId:  describe result for acl check
2025-10-09T21:51:36.061111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:36.061114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:36.061116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:36.061121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343000002715211:8761] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:36.061671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343000002715211:8761] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:36.064348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343000002715211:8761] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:36.066644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343000002715211:8761] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:36.066659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343000002715211:8761] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:36.066663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343000002715211:8761] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:36.066670Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2190 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:36.070151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:36.070533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:36.070736Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:36.070743Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:36.070752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:36.070791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:36.070822Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342999640870030:11741]
2025-10-09T21:51:36.071221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:36.071446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:36.071475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:36.071709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:36.071748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.071753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.071754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.071755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.071755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.071756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.071757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.071757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.071758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.071761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.071768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.071770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.071773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.071775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.071777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.071779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.071781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.071783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.071788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.071793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.071797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.071801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.071803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.071807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.071809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.071812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.071814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.071817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.071820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.071823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.071825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.071828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.071831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.071833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.071836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.071839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.071856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:36.077874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:36.078874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:36.078887Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:36.078889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:36.078892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:36.078899Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2189 [2:7559342999640870030:11741]
2025-10-09T21:51:36.078907Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:36.078921Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:36.078923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:36.078925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:36.078927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:36.078939Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:36.078943Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:36.078949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:36.079208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.079216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.079218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.079219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:36.079272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:36.079277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.079285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.079291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.079301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:36.079316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342999640870025:11741]
2025-10-09T21:51:36.079320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342999640870025:11741]
2025-10-09T21:51:36.079322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342999640870025:11741]
2025-10-09T21:51:36.079328Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:36.079336Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:36.079338Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:36.079339Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:36.079342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343000002715118:2432], now have 1 active actors on pipe
2025-10-09T21:51:36.079348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:36.079356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:36.079362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:36.079367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343000002715501:8761], now have 1 active actors on pipe
2025-10-09T21:51:36.079390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342999640870025:11741]
2025-10-09T21:51:36.079402Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:36.079413Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:36.079420Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2189
2025-10-09T21:51:36.079431Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2189
2025-10-09T21:51:36.079455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.079462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.079484Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6947c8de-9039fc96-2a758412-f24e0a15_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:36.079493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:36.079496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:36.079498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:36.079505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:36.079516Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:36.079528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:36.079701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.079715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.079738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:36.079850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2190 partition: 0 MaxSeqNo: 25 sessionId: src_id|6947c8de-9039fc96-2a758412-f24e0a15_0
2025-10-09T21:51:36.080145Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|688789b0-e039c485-c775a719-76f41ed5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6947c8de-9039fc96-2a758412-f24e0a15_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:36.082527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6947c8de-9039fc96-2a758412-f24e0a15_0] Write session: try to update token
2025-10-09T21:51:36.082541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6947c8de-9039fc96-2a758412-f24e0a15_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:36.090405Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:36.090419Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:36.090426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342999640870025:11741]
2025-10-09T21:51:36.090463Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:36.090466Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:36.090623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:36.090751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:36.090867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:36.090994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:36.091174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:36.091206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:36.094406Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2190 sessionId: src_id|6947c8de-9039fc96-2a758412-f24e0a15_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:36.096839Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:36.097022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.097040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.097077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:36.097170Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:36.103262Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:36.103274Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:36.103276Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:36.103277Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:36.103285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342999640870025:11741]
2025-10-09T21:51:36.103288Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342999640870025:11741]
2025-10-09T21:51:36.103291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342999640870025:11741]
2025-10-09T21:51:36.103322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:36.103328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.103330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:36.103329Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:36.103334Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:36.103338Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:36.103457Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:36.152589Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343000002715544:9464]
2025-10-09T21:51:36.152916Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 2189, pipe: [1:7559343000002715544:9464]
2025-10-09T21:51:36.251680Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:36.258493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.258515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.258669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.258733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.258790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.258856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.258900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.258910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.258978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.259045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.259112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.259183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.259221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.259229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.259289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.259346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.259412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.259484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.259524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.259530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.259600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.259672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.259741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.259797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.259838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.259844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.259903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.259966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.260027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.260088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.260147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.260154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.260213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.260271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.260335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.260395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.260436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.260445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.260502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.260557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.260612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.260668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.260708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.260714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.260771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.260826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.260886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.260940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.260982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.260991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.261050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.261106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.261160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.261223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.261264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.261274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.261335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:36.261406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:36.261463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:36.261521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:36.261562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:36.261568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:36.261614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343000002715544:9464], now have 1 active actors on pipe
2025-10-09T21:51:36.261858Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:36.261868Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:36.261873Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:36.261899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2190 reason: error at write: Disconnected sessionId: src_id|6947c8de-9039fc96-2a758412-f24e0a15_0
2025-10-09T21:51:36.261910Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343000002715544:9464]
2025-10-09T21:51:36.262114Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2190 sessionId: src_id|6947c8de-9039fc96-2a758412-f24e0a15_0 is DEAD
2025-10-09T21:51:36.262225Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6947c8de-9039fc96-2a758412-f24e0a15_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:36.262235Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6947c8de-9039fc96-2a758412-f24e0a15_0] Write session will restart in 0.009045s
2025-10-09T21:51:36.262253Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6947c8de-9039fc96-2a758412-f24e0a15_0] Write session: Do CDS request
2025-10-09T21:51:36.262258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6947c8de-9039fc96-2a758412-f24e0a15_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:36.262440Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:36.272494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6947c8de-9039fc96-2a758412-f24e0a15_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:36.272749Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:36.272766Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2191
2025-10-09T21:51:36.272892Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2191 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:36.272928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2191 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:36.272941Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2191 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:36.272950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:36.273301Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2191 sessionId:  describe result for acl check
2025-10-09T21:51:36.273333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:36.273338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:36.273339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:36.273356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343000002715807:8753] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:36.273793Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343000002715807:8753] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:36.276320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343000002715807:8753] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:36.277781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:36.280310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:36.280459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343000002715807:8753] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:36.280473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343000002715807:8753] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:36.280479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343000002715807:8753] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:36.280487Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2191 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:36.280487Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:36.280497Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:36.280516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:36.280569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:36.280605Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342999640870301:11743]
2025-10-09T21:51:36.281192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:36.281483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:36.281522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:36.281811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:36.281849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.281854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.281855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.281856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.281856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.281857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.281858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.281858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.281859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.281862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.281868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.281870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.281872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.281874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.281876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.281878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.281880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.281882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.281886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.281891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.281895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.281898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.281900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.281903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.281906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.281908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.281911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.281914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.281916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.281920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.281923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.281926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.281928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.281931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.281934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.281937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.281953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:36.287796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:36.288719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:36.288730Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:36.288731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:36.288734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:36.288738Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2190 [2:7559342999640870301:11743]
2025-10-09T21:51:36.288746Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:36.288759Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:36.288761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:36.288763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:36.288764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:36.288776Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:36.288780Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:36.288785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:36.288823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.288828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.288829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.288830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:36.288884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:36.288891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.288901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.288912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.288921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:36.288939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343000002715966:8753], now have 1 active actors on pipe
2025-10-09T21:51:36.288948Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342999640870296:11743]
2025-10-09T21:51:36.288953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342999640870296:11743]
2025-10-09T21:51:36.288955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342999640870296:11743]
2025-10-09T21:51:36.288965Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:36.288974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343000002715550:2432], now have 1 active actors on pipe
2025-10-09T21:51:36.288981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:36.288982Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:36.288985Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:36.288987Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:36.288992Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:36.288994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:36.289010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342999640870296:11743]
2025-10-09T21:51:36.289013Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:36.289015Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:36.289028Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2190
2025-10-09T21:51:36.289036Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2190
2025-10-09T21:51:36.289122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.289128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.289148Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d4fa5d24-ce248471-2ad8b6da-3b9f8371_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:36.289157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:36.289163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:36.289164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:36.289171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:36.289181Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:36.289194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:36.289374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.289381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.289394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:36.289492Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2191 partition: 0 MaxSeqNo: 25 sessionId: src_id|d4fa5d24-ce248471-2ad8b6da-3b9f8371_0
2025-10-09T21:51:36.289817Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6947c8de-9039fc96-2a758412-f24e0a15_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d4fa5d24-ce248471-2ad8b6da-3b9f8371_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:36.292192Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d4fa5d24-ce248471-2ad8b6da-3b9f8371_0] Write session: try to update token
2025-10-09T21:51:36.292211Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d4fa5d24-ce248471-2ad8b6da-3b9f8371_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:36.302062Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:36.302073Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:36.302075Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:36.302076Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:36.302084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342999640870296:11743]
2025-10-09T21:51:36.302087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342999640870296:11743]
2025-10-09T21:51:36.302089Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342999640870296:11743]
2025-10-09T21:51:36.302122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:36.302127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.302129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:36.302136Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:36.302142Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:36.302146Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:36.302251Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:36.305115Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2191 sessionId: src_id|d4fa5d24-ce248471-2ad8b6da-3b9f8371_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:36.307598Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:36.361962Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343000002715983:9464]
2025-10-09T21:51:36.362316Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 2190, pipe: [1:7559343000002715983:9464]
2025-10-09T21:51:36.440125Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:36.440796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.440808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.440846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343000002715983:9464], now have 1 active actors on pipe
2025-10-09T21:51:36.441015Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:36.441103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:36.441119Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:36.441125Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343000002715983:9464]
2025-10-09T21:51:36.441148Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:36.441178Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2191 reason: error at write: Disconnected sessionId: src_id|d4fa5d24-ce248471-2ad8b6da-3b9f8371_0
2025-10-09T21:51:36.441406Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2191 sessionId: src_id|d4fa5d24-ce248471-2ad8b6da-3b9f8371_0 is DEAD
2025-10-09T21:51:36.441510Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4fa5d24-ce248471-2ad8b6da-3b9f8371_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:36.441523Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4fa5d24-ce248471-2ad8b6da-3b9f8371_0] Write session will restart in 0.006719s
2025-10-09T21:51:36.441561Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4fa5d24-ce248471-2ad8b6da-3b9f8371_0] Write session: Do CDS request
2025-10-09T21:51:36.441567Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4fa5d24-ce248471-2ad8b6da-3b9f8371_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:36.449493Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d4fa5d24-ce248471-2ad8b6da-3b9f8371_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:36.449564Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:36.449580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2192
2025-10-09T21:51:36.449714Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2192 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:36.449745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2192 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:36.449757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2192 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:36.449760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:36.450180Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2192 sessionId:  describe result for acl check
2025-10-09T21:51:36.450213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:36.450215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:36.450216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:36.450221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343000002716157:8749] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:36.450747Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343000002716157:8749] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:36.453012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343000002716157:8749] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:36.455220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343000002716157:8749] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:36.455236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343000002716157:8749] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:36.455239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343000002716157:8749] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:36.455248Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2192 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:36.456635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:36.458323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:36.458490Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:36.458504Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:36.458513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:36.458553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:36.458577Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342999640870573:11745]
2025-10-09T21:51:36.458962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:36.459223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:36.459251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:36.459463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:36.459498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.459502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.459502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.459503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.459504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.459504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.459505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.459505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.459506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.459509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.459516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.459518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.459520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.459522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.459524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.459526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.459528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.459530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.459534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.459539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.459543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.459546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.459548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.459551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.459554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.459559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.459563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.459568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.459572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.459578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.459583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.459587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.459590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.459592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.459596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.459599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.459614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:36.465164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:36.466151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:36.466160Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:36.466162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:36.466165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:36.466169Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2191 [2:7559342999640870573:11745]
2025-10-09T21:51:36.466178Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:36.466191Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:36.466192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:36.466194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:36.466195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:36.466206Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:36.466213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:36.466219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:36.466262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.466266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.466267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.466269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:36.466318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:36.466323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.466330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.466338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.466347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:36.466367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342999640870567:11745]
2025-10-09T21:51:36.466374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342999640870567:11745]
2025-10-09T21:51:36.466376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342999640870567:11745]
2025-10-09T21:51:36.466384Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:36.466393Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:36.466396Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:36.466397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343000002716399:8749], now have 1 active actors on pipe
2025-10-09T21:51:36.466398Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:36.466401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343000002715984:2432], now have 1 active actors on pipe
2025-10-09T21:51:36.466409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:36.466419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:36.466425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:36.466442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342999640870567:11745]
2025-10-09T21:51:36.466452Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:36.466455Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2191
2025-10-09T21:51:36.466465Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2191
2025-10-09T21:51:36.466506Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:36.466571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.466585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.466619Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|587b3db-1ba3b0dc-1eae715b-8fda9cae_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:36.466628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:36.466634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:36.466636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:36.466663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:36.466675Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:36.466692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:36.466853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.466863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.466881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:36.466995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2192 partition: 0 MaxSeqNo: 25 sessionId: src_id|587b3db-1ba3b0dc-1eae715b-8fda9cae_0
2025-10-09T21:51:36.467279Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d4fa5d24-ce248471-2ad8b6da-3b9f8371_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|587b3db-1ba3b0dc-1eae715b-8fda9cae_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:36.469576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|587b3db-1ba3b0dc-1eae715b-8fda9cae_0] Write session: try to update token
2025-10-09T21:51:36.469590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|587b3db-1ba3b0dc-1eae715b-8fda9cae_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:36.479792Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:36.479801Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:36.479803Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:36.479805Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:36.479812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342999640870567:11745]
2025-10-09T21:51:36.479815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342999640870567:11745]
2025-10-09T21:51:36.479817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342999640870567:11745]
2025-10-09T21:51:36.479849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:36.479857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.479857Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:36.479858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:36.479862Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:36.479866Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:36.479970Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:36.481413Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2192 sessionId: src_id|587b3db-1ba3b0dc-1eae715b-8fda9cae_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:36.483450Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:36.541191Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343000002716414:9464]
2025-10-09T21:51:36.541434Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 2191, pipe: [1:7559343000002716414:9464]
2025-10-09T21:51:36.617108Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:36.617833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.617846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.617884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343000002716414:9464], now have 1 active actors on pipe
2025-10-09T21:51:36.618026Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:36.618183Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:36.618200Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343000002716414:9464]
2025-10-09T21:51:36.618206Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:36.618222Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:36.618288Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2192 reason: error at write: Disconnected sessionId: src_id|587b3db-1ba3b0dc-1eae715b-8fda9cae_0
2025-10-09T21:51:36.618460Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2192 sessionId: src_id|587b3db-1ba3b0dc-1eae715b-8fda9cae_0 is DEAD
2025-10-09T21:51:36.618621Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|587b3db-1ba3b0dc-1eae715b-8fda9cae_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:36.618632Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|587b3db-1ba3b0dc-1eae715b-8fda9cae_0] Write session will restart in 0.008055s
2025-10-09T21:51:36.618654Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|587b3db-1ba3b0dc-1eae715b-8fda9cae_0] Write session: Do CDS request
2025-10-09T21:51:36.618658Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|587b3db-1ba3b0dc-1eae715b-8fda9cae_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:36.627490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|587b3db-1ba3b0dc-1eae715b-8fda9cae_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:36.627590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:36.627608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2193
2025-10-09T21:51:36.627764Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2193 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:36.627806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2193 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:36.627815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2193 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:36.627819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:36.628257Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2193 sessionId:  describe result for acl check
2025-10-09T21:51:36.628294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:36.628301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:36.628303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:36.628308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343000002716597:8735] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:36.628893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343000002716597:8735] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:36.631310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343000002716597:8735] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:36.633952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343000002716597:8735] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:36.633966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343000002716597:8735] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:36.633969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343000002716597:8735] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:36.633976Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2193 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:36.634763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:36.636392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:36.636558Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:36.636567Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:36.636576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:36.636614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:36.636644Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342999640870845:11747]
2025-10-09T21:51:36.637009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:36.637303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:36.637352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:36.637557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:36.637585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.637589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.637590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.637591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.637591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.637592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.637592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.637593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.637594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.637597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.637604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.637606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.637608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.637610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.637612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.637614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.637616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.637618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.637622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.637627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.637631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.637635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.637637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.637640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.637642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.637645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.637647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.637653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.637656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.637659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.637661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.637670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.637673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.637676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.637678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.637681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.637698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:36.643708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:36.645012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:36.645025Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:36.645027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:36.645031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:36.645037Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2192 [2:7559342999640870845:11747]
2025-10-09T21:51:36.645051Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:36.645066Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:36.645070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:36.645072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:36.645075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:36.645090Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:36.645102Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:36.645112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:36.645162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.645169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.645171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.645173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:36.645234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:36.645242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.645253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.645265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.645287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:36.645311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343000002716832:8735], now have 1 active actors on pipe
2025-10-09T21:51:36.645314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343000002716415:2432], now have 1 active actors on pipe
2025-10-09T21:51:36.645321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342999640870840:11747]
2025-10-09T21:51:36.645328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342999640870840:11747]
2025-10-09T21:51:36.645330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342999640870840:11747]
2025-10-09T21:51:36.645354Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:36.645372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:36.645383Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:36.645369Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2192
2025-10-09T21:51:36.645390Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:36.645378Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2192
2025-10-09T21:51:36.645391Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:36.645392Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:36.645394Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:36.645415Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342999640870840:11747]
2025-10-09T21:51:36.645423Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:36.645438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.645439Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:36.645447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.645499Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b5d9aabb-9cdea078-4a85af1c-eb8b60d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:36.645512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:36.645520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:36.645523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:36.645535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:36.645549Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:36.645571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:36.645717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.645727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.645743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:36.645860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2193 partition: 0 MaxSeqNo: 25 sessionId: src_id|b5d9aabb-9cdea078-4a85af1c-eb8b60d_0
2025-10-09T21:51:36.646127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|587b3db-1ba3b0dc-1eae715b-8fda9cae_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b5d9aabb-9cdea078-4a85af1c-eb8b60d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:36.648328Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5d9aabb-9cdea078-4a85af1c-eb8b60d_0] Write session: try to update token
2025-10-09T21:51:36.648340Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5d9aabb-9cdea078-4a85af1c-eb8b60d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:36.659387Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:36.659406Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:36.659408Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:36.659409Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:36.659416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342999640870840:11747]
2025-10-09T21:51:36.659420Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342999640870840:11747]
2025-10-09T21:51:36.659423Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342999640870840:11747]
2025-10-09T21:51:36.659462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:36.659468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.659468Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:36.659470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:36.659473Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:36.659476Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:36.659594Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:36.660869Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2193 sessionId: src_id|b5d9aabb-9cdea078-4a85af1c-eb8b60d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:36.663002Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:36.718336Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343000002716846:9464]
2025-10-09T21:51:36.718641Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 2192, pipe: [1:7559343000002716846:9464]
2025-10-09T21:51:36.801076Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:36.801284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.801298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.801334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343000002716846:9464], now have 1 active actors on pipe
2025-10-09T21:51:36.801475Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:36.801576Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:36.801758Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:36.801766Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:36.801786Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343000002716846:9464]
2025-10-09T21:51:36.801840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2193 reason: error at write: Disconnected sessionId: src_id|b5d9aabb-9cdea078-4a85af1c-eb8b60d_0
2025-10-09T21:51:36.801999Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2193 sessionId: src_id|b5d9aabb-9cdea078-4a85af1c-eb8b60d_0 is DEAD
2025-10-09T21:51:36.802154Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5d9aabb-9cdea078-4a85af1c-eb8b60d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:36.802165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5d9aabb-9cdea078-4a85af1c-eb8b60d_0] Write session will restart in 0.008636s
2025-10-09T21:51:36.802183Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5d9aabb-9cdea078-4a85af1c-eb8b60d_0] Write session: Do CDS request
2025-10-09T21:51:36.802188Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5d9aabb-9cdea078-4a85af1c-eb8b60d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:36.811488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5d9aabb-9cdea078-4a85af1c-eb8b60d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:36.811514Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:36.811532Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2194
2025-10-09T21:51:36.811735Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2194 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:36.811769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2194 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:36.811774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2194 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:36.811778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:36.812332Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2194 sessionId:  describe result for acl check
2025-10-09T21:51:36.812372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:36.812378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:36.812379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:36.812384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343000002717006:8727] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:36.812837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343000002717006:8727] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:36.814958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343000002717006:8727] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:36.817723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343000002717006:8727] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:36.817733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343000002717006:8727] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:36.817736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343000002717006:8727] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:36.817825Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2194 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:36.820520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:36.822213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:36.822373Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:36.822382Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:36.822394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:36.822439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:36.822476Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559342999640871122:11751]
2025-10-09T21:51:36.823027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:36.823290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:36.823329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:36.823621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:36.823670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.823675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.823676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.823678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.823679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.823680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.823681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.823682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.823683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.823688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.823699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.823703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.823707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.823710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.823714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.823718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.823722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.823726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.823737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.823749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:36.823758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.823767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:36.823776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.823781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:36.823785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.823794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:36.823803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.823808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:36.823818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.823823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:36.823828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.823833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.823837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.823842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.823846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.823851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.823881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:36.829700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:36.830797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:36.830810Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:36.830812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:36.830815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:36.830821Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2193 [2:7559342999640871122:11751]
2025-10-09T21:51:36.830833Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:36.830848Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:36.830850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:36.830853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:36.830856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:36.830870Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:36.830875Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:36.830882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:36.830930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.830935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.830936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.830938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:36.831142Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2193
2025-10-09T21:51:36.831172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:36.831179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.831187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:36.831196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:36.831145Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2193
2025-10-09T21:51:36.831212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:36.831236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343000002717252:8727], now have 1 active actors on pipe
2025-10-09T21:51:36.831244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343000002716851:2432], now have 1 active actors on pipe
2025-10-09T21:51:36.831261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.831269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.831290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559342999640871117:11751]
2025-10-09T21:51:36.831298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559342999640871117:11751]
2025-10-09T21:51:36.831300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559342999640871117:11751]
2025-10-09T21:51:36.831312Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:36.831325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:36.831330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:36.831333Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:36.831335Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:36.831345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:36.831348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:36.831367Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|25f48ddc-ae16cc9b-46ef7989-9409841e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:36.831379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:36.831382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:36.831385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:36.831394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:36.831407Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:36.831428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559342999640871117:11751]
2025-10-09T21:51:36.831440Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:36.831455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:36.831462Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:36.831589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.831598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.831612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:36.831678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2194 partition: 0 MaxSeqNo: 25 sessionId: src_id|25f48ddc-ae16cc9b-46ef7989-9409841e_0
2025-10-09T21:51:36.832051Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5d9aabb-9cdea078-4a85af1c-eb8b60d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|25f48ddc-ae16cc9b-46ef7989-9409841e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:36.834435Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25f48ddc-ae16cc9b-46ef7989-9409841e_0] Write session: try to update token
2025-10-09T21:51:36.834448Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25f48ddc-ae16cc9b-46ef7989-9409841e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:36.842447Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:36.842465Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:36.842474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559342999640871117:11751]
2025-10-09T21:51:36.842516Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:36.842522Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:36.842674Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:36.842805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:36.842935Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:36.843064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:36.843251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:36.843291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:36.846400Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2194 sessionId: src_id|25f48ddc-ae16cc9b-46ef7989-9409841e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:36.848910Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:36.849128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:36.849143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:36.849177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:36.849265Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:36.855044Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:36.855059Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:36.855062Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:36.855064Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:36.855072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559342999640871117:11751]
2025-10-09T21:51:36.855076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559342999640871117:11751]
2025-10-09T21:51:36.855079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559342999640871117:11751]
2025-10-09T21:51:36.855116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:36.855115Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:36.855120Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:36.855123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:36.855124Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:36.855126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:36.855261Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:36.902102Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343000002717284:9464]
2025-10-09T21:51:36.902412Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 2193, pipe: [1:7559343000002717284:9464]
2025-10-09T21:51:36.996758Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:37.003322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.003341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.003476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.003530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.003588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.003648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.003690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.003698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.003756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.003808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.003862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.003921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.003961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.003967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.004019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.004068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.004122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.004175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.004214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.004219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.004275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.004331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.004386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.004441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.004479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.004488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.004542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.004594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.004656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.004710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.004747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.004752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.004804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.004855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.004907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.004959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.004997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.005002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.005053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.005101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.005157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.005209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.005253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.005258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.005312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.005377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.005430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.005481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.005518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.005523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.005575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.005623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.005674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.005731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.005770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.005781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.005834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.005889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.005944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.006000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.006057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.006063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.006102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343000002717284:9464], now have 1 active actors on pipe
2025-10-09T21:51:37.006322Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343000002717284:9464]
2025-10-09T21:51:37.006329Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:37.006335Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:37.006365Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2194 reason: error at write: Disconnected sessionId: src_id|25f48ddc-ae16cc9b-46ef7989-9409841e_0
2025-10-09T21:51:37.006367Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:37.006564Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2194 sessionId: src_id|25f48ddc-ae16cc9b-46ef7989-9409841e_0 is DEAD
2025-10-09T21:51:37.006680Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25f48ddc-ae16cc9b-46ef7989-9409841e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:37.006704Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25f48ddc-ae16cc9b-46ef7989-9409841e_0] Write session will restart in 0.006262s
2025-10-09T21:51:37.006729Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25f48ddc-ae16cc9b-46ef7989-9409841e_0] Write session: Do CDS request
2025-10-09T21:51:37.006736Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25f48ddc-ae16cc9b-46ef7989-9409841e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:37.006801Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:37.014533Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25f48ddc-ae16cc9b-46ef7989-9409841e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:37.014605Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:37.014623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2195
2025-10-09T21:51:37.014780Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2195 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:37.014813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2195 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:37.014819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2195 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:37.014829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:37.015246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2195 sessionId:  describe result for acl check
2025-10-09T21:51:37.015292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:37.015300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:37.015302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:37.015307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343004297684681:8714] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:37.015854Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343004297684681:8714] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:37.018568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343004297684681:8714] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:37.021656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343004297684681:8714] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:37.021671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343004297684681:8714] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:37.021675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343004297684681:8714] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:37.021684Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2195 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:37.027141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:37.028843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:37.028995Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:37.029003Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:37.029011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:37.029056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:37.029090Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343003935838730:11765]
2025-10-09T21:51:37.029501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:37.029791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:37.029823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:37.030109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:37.030153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.030158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.030159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.030159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.030160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.030161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.030161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.030162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.030163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.030166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.030173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.030176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.030178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.030180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.030182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.030184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.030186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.030188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.030193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.030199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.030203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.030206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.030209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.030212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.030214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.030217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.030220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.030223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.030226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.030229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.030232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.030235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.030238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.030241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.030243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.030246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.030263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:37.036513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:37.037626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:37.037639Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:37.037641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:37.037644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:37.037650Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2194 [2:7559343003935838730:11765]
2025-10-09T21:51:37.037660Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:37.037675Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:37.037677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:37.037679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:37.037680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:37.037693Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:37.037702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:37.037709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:37.037760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.037767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.037768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.037770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:37.037823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:37.037829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.037836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.037843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.037852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:37.037868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343003935838725:11765]
2025-10-09T21:51:37.037870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343003935838725:11765]
2025-10-09T21:51:37.037872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343003935838725:11765]
2025-10-09T21:51:37.037896Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:37.037904Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:37.037907Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:37.037909Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:37.037917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:37.037926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:37.037928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:37.037939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297684918:8714], now have 1 active actors on pipe
2025-10-09T21:51:37.037949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297684607:2432], now have 1 active actors on pipe
2025-10-09T21:51:37.037978Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343003935838725:11765]
2025-10-09T21:51:37.037990Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:37.037994Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:37.038020Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2194
2025-10-09T21:51:37.038024Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2194
2025-10-09T21:51:37.038113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.038123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.038149Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|23406345-c6357de4-4eba65b6-840b1051_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:37.038159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:37.038163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:37.038164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:37.038172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:37.038180Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:37.038194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:37.038692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.038704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.038724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:37.038827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2195 partition: 0 MaxSeqNo: 25 sessionId: src_id|23406345-c6357de4-4eba65b6-840b1051_0
2025-10-09T21:51:37.039199Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25f48ddc-ae16cc9b-46ef7989-9409841e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|23406345-c6357de4-4eba65b6-840b1051_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:37.041669Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23406345-c6357de4-4eba65b6-840b1051_0] Write session: try to update token
2025-10-09T21:51:37.041687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23406345-c6357de4-4eba65b6-840b1051_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:37.052168Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:37.052184Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:37.052185Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:37.052187Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:37.052194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343003935838725:11765]
2025-10-09T21:51:37.052197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343003935838725:11765]
2025-10-09T21:51:37.052200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343003935838725:11765]
2025-10-09T21:51:37.052235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:37.052240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.052239Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:37.052242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:37.052244Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:37.052247Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:37.052359Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:37.054355Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2195 sessionId: src_id|23406345-c6357de4-4eba65b6-840b1051_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:37.056660Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:37.106439Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343004297685035:9464]
2025-10-09T21:51:37.106736Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 2194, pipe: [1:7559343004297685035:9464]
2025-10-09T21:51:37.195148Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:37.195409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.195422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.195465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297685035:9464], now have 1 active actors on pipe
2025-10-09T21:51:37.195649Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:37.195720Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:37.195722Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:37.195737Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:37.195759Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343004297685035:9464]
2025-10-09T21:51:37.195805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2195 reason: error at write: Disconnected sessionId: src_id|23406345-c6357de4-4eba65b6-840b1051_0
2025-10-09T21:51:37.195991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2195 sessionId: src_id|23406345-c6357de4-4eba65b6-840b1051_0 is DEAD
2025-10-09T21:51:37.196166Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23406345-c6357de4-4eba65b6-840b1051_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:37.196177Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23406345-c6357de4-4eba65b6-840b1051_0] Write session will restart in 0.007398s
2025-10-09T21:51:37.196202Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23406345-c6357de4-4eba65b6-840b1051_0] Write session: Do CDS request
2025-10-09T21:51:37.196207Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23406345-c6357de4-4eba65b6-840b1051_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:37.204597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|23406345-c6357de4-4eba65b6-840b1051_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:37.204665Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:37.204681Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2196
2025-10-09T21:51:37.204836Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2196 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:37.204876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2196 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:37.204885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2196 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:37.204888Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:37.205220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2196 sessionId:  describe result for acl check
2025-10-09T21:51:37.205247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:37.205248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:37.205250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:37.205256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343004297685135:8703] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:37.205842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343004297685135:8703] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:37.207952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343004297685135:8703] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:37.211076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343004297685135:8703] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:37.211091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343004297685135:8703] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:37.211096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343004297685135:8703] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:37.211106Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2196 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:37.214533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:37.216675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:37.216847Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:37.216855Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:37.216864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:37.216902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:37.216939Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343003935839006:11767]
2025-10-09T21:51:37.217324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:37.217602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:37.217629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:37.217835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:37.217869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.217874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.217875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.217876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.217877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.217877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.217878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.217879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.217880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.217883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.217890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.217893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.217895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.217897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.217900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.217902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.217904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.217906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.217911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.217916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.217920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.217923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.217926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.217929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.217932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.217935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.217938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.217941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.217943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.217946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.217951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.217954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.217957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.217960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.217963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.217967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.217985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:37.224504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:37.225497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:37.225511Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:37.225513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:37.225516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:37.225523Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2195 [2:7559343003935839006:11767]
2025-10-09T21:51:37.225534Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:37.225552Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:37.225553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:37.225556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:37.225558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:37.225572Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:37.225576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:37.225582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:37.225627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.225632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.225634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.225635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:37.225694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:37.225701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.225708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.225714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.225724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:37.225740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343003935839001:11767]
2025-10-09T21:51:37.225741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343003935839001:11767]
2025-10-09T21:51:37.225744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343003935839001:11767]
2025-10-09T21:51:37.225751Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:37.225763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297685399:8703], now have 1 active actors on pipe
2025-10-09T21:51:37.225763Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:37.225766Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:37.225769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:37.225769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297685040:2432], now have 1 active actors on pipe
2025-10-09T21:51:37.225776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:37.225785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:37.225788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:37.225818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343003935839001:11767]
2025-10-09T21:51:37.225829Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:37.225850Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:37.225838Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2195
2025-10-09T21:51:37.225884Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2195
2025-10-09T21:51:37.225936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.225947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.225967Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b4cd8fdf-3c290f78-641d64cc-678781a2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:37.225975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:37.225978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:37.225980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:37.225988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:37.225996Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:37.226011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:37.226124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.226129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.226139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:37.226202Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2196 partition: 0 MaxSeqNo: 25 sessionId: src_id|b4cd8fdf-3c290f78-641d64cc-678781a2_0
2025-10-09T21:51:37.226526Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|23406345-c6357de4-4eba65b6-840b1051_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b4cd8fdf-3c290f78-641d64cc-678781a2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:37.228790Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4cd8fdf-3c290f78-641d64cc-678781a2_0] Write session: try to update token
2025-10-09T21:51:37.228803Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4cd8fdf-3c290f78-641d64cc-678781a2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:37.239980Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:37.239995Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:37.239998Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:37.240000Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:37.240007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343003935839001:11767]
2025-10-09T21:51:37.240011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343003935839001:11767]
2025-10-09T21:51:37.240014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343003935839001:11767]
2025-10-09T21:51:37.240048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:37.240054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.240056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:37.240057Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:37.240064Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:37.240068Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:37.240183Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:37.241142Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2196 sessionId: src_id|b4cd8fdf-3c290f78-641d64cc-678781a2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:37.243304Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:37.295869Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343004297685475:9464]
2025-10-09T21:51:37.296102Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 2195, pipe: [1:7559343004297685475:9464]
2025-10-09T21:51:37.382129Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:37.382819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.382832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.382873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297685475:9464], now have 1 active actors on pipe
2025-10-09T21:51:37.383053Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:37.383070Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343004297685475:9464]
2025-10-09T21:51:37.383077Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:37.383081Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:37.383096Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:37.383137Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2196 reason: error at write: Disconnected sessionId: src_id|b4cd8fdf-3c290f78-641d64cc-678781a2_0
2025-10-09T21:51:37.383319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2196 sessionId: src_id|b4cd8fdf-3c290f78-641d64cc-678781a2_0 is DEAD
2025-10-09T21:51:37.383460Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4cd8fdf-3c290f78-641d64cc-678781a2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:37.383473Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4cd8fdf-3c290f78-641d64cc-678781a2_0] Write session will restart in 0.009750s
2025-10-09T21:51:37.383499Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4cd8fdf-3c290f78-641d64cc-678781a2_0] Write session: Do CDS request
2025-10-09T21:51:37.383505Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4cd8fdf-3c290f78-641d64cc-678781a2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:37.394536Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b4cd8fdf-3c290f78-641d64cc-678781a2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:37.394608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:37.394624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2197
2025-10-09T21:51:37.394759Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2197 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:37.394785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2197 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:37.394794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2197 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:37.394797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:37.395245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2197 sessionId:  describe result for acl check
2025-10-09T21:51:37.395276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:37.395279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:37.395281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:37.395286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343004297685737:8699] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:37.395762Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343004297685737:8699] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:37.398007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343004297685737:8699] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:37.400335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343004297685737:8699] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:37.400349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343004297685737:8699] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:37.400353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343004297685737:8699] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:37.400361Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2197 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:37.400663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:37.402494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:37.402671Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:37.402683Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:37.402695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:37.402755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:37.402799Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343003935839281:11769]
2025-10-09T21:51:37.403385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:37.403674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:37.403717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:37.404016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:37.404078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.404085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.404086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.404087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.404089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.404090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.404091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.404092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.404093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.404098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.404109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.404113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.404117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.404121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.404126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.404130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.404133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.404137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.404145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.404153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.404159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.404163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.404168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.404173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.404177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.404182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.404185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.404190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.404193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.404198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.404202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.404206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.404210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.404220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.404224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.404229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.404255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:37.410477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:37.411783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:37.411796Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:37.411798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:37.411801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:37.411806Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2196 [2:7559343003935839281:11769]
2025-10-09T21:51:37.411815Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:37.411830Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:37.411831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:37.411833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:37.411835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:37.411847Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:37.411851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:37.411857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:37.411905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.411911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.411912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.411914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:37.411963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:37.411969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.411975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.411981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.411990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:37.412004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343003935839276:11769]
2025-10-09T21:51:37.412009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343003935839276:11769]
2025-10-09T21:51:37.412012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343003935839276:11769]
2025-10-09T21:51:37.412018Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:37.412025Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:37.412029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297685904:8699], now have 1 active actors on pipe
2025-10-09T21:51:37.412029Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:37.412031Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:37.412032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297685483:2432], now have 1 active actors on pipe
2025-10-09T21:51:37.412038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:37.412046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:37.412047Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:37.412070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343003935839276:11769]
2025-10-09T21:51:37.412080Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:37.412087Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:37.412104Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2196
2025-10-09T21:51:37.412117Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2196
2025-10-09T21:51:37.412200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.412212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.412237Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cd34de26-46a462ca-34f28a2d-de91a9f9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:37.412247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:37.412251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:37.412252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:37.412268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:37.412281Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:37.412292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:37.412415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.412421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.412431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:37.412588Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2197 partition: 0 MaxSeqNo: 25 sessionId: src_id|cd34de26-46a462ca-34f28a2d-de91a9f9_0
2025-10-09T21:51:37.412887Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b4cd8fdf-3c290f78-641d64cc-678781a2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cd34de26-46a462ca-34f28a2d-de91a9f9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:37.415186Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd34de26-46a462ca-34f28a2d-de91a9f9_0] Write session: try to update token
2025-10-09T21:51:37.415202Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd34de26-46a462ca-34f28a2d-de91a9f9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:37.423295Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:37.423309Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:37.423326Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343003935839276:11769]
2025-10-09T21:51:37.423361Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:37.423363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:37.423508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:37.423634Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:37.423760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:37.423872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:37.424034Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:37.424064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:37.427371Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2197 sessionId: src_id|cd34de26-46a462ca-34f28a2d-de91a9f9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:37.429785Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:37.429949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.429968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.430007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:37.430079Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:37.435989Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:37.436001Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:37.436003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:37.436004Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:37.436013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343003935839276:11769]
2025-10-09T21:51:37.436016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343003935839276:11769]
2025-10-09T21:51:37.436019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343003935839276:11769]
2025-10-09T21:51:37.436049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:37.436055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.436057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:37.436057Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:37.436062Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:37.436065Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:37.436175Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:37.483269Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343004297685916:9464]
2025-10-09T21:51:37.483537Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 2196, pipe: [1:7559343004297685916:9464]
2025-10-09T21:51:37.582021Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:37.588519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.588546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.588694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.588751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.588806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.588860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.588899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.588906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.588964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.589014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.589070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.589127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.589165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.589171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.589222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.589271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.589320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.589381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.589422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.589427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.589481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.589533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.589585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.589637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.589674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.589679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.589731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.589781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.589833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.589884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.589921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.589927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.589977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.590052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.590116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.590173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.590212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.590219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.590277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.590326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.590402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.590451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.590487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.590493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.590543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.590590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.590641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.590691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.590727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.590732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.590779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.590831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.590881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.590930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.590966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.590971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.591019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.591067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.591117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.591167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.591203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.591208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.591246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297685916:9464], now have 1 active actors on pipe
2025-10-09T21:51:37.591523Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:37.591529Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:37.591533Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:37.591561Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2197 reason: error at write: Disconnected sessionId: src_id|cd34de26-46a462ca-34f28a2d-de91a9f9_0
2025-10-09T21:51:37.591568Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343004297685916:9464]
2025-10-09T21:51:37.591749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2197 sessionId: src_id|cd34de26-46a462ca-34f28a2d-de91a9f9_0 is DEAD
2025-10-09T21:51:37.591899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd34de26-46a462ca-34f28a2d-de91a9f9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:37.591909Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd34de26-46a462ca-34f28a2d-de91a9f9_0] Write session will restart in 0.008996s
2025-10-09T21:51:37.591928Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd34de26-46a462ca-34f28a2d-de91a9f9_0] Write session: Do CDS request
2025-10-09T21:51:37.591933Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd34de26-46a462ca-34f28a2d-de91a9f9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:37.592015Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:37.602489Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd34de26-46a462ca-34f28a2d-de91a9f9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:37.602575Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:37.602590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2198
2025-10-09T21:51:37.602717Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2198 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:37.602749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2198 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:37.602762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2198 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:37.602772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:37.603149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2198 sessionId:  describe result for acl check
2025-10-09T21:51:37.603185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:37.603191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:37.603193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:37.603199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343004297686111:8696] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:37.603745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343004297686111:8696] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:37.606729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343004297686111:8696] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:37.609244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343004297686111:8696] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:37.609254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343004297686111:8696] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:37.609257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343004297686111:8696] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:37.609263Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2198 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:37.610297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:37.612021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:37.612166Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:37.612175Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:37.612184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:37.612227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:37.612268Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343003935839559:11771]
2025-10-09T21:51:37.612794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:37.613117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:37.613160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:37.613476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:37.613527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.613533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.613534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.613535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.613536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.613537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.613539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.613540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.613541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.613545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.613555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.613559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.613562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.613566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.613569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.613573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.613577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.613582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.613588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.613597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.613603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.613608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.613613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.613618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.613623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.613628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.613633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.613638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.613642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.613647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.613651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.613661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.613665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.613669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.613674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.613679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.613711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:37.620064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:37.621309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:37.621324Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:37.621326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:37.621330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:37.621351Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2197 [2:7559343003935839559:11771]
2025-10-09T21:51:37.621366Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:37.621383Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:37.621391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:37.621395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:37.621397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:37.621413Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:37.621424Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:37.621431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:37.621486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.621492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.621494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.621497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:37.621563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:37.621572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.621583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.621594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.621609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:37.621630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297686339:8696], now have 1 active actors on pipe
2025-10-09T21:51:37.621639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297685919:2432], now have 1 active actors on pipe
2025-10-09T21:51:37.621645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343003935839554:11771]
2025-10-09T21:51:37.621648Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343003935839554:11771]
2025-10-09T21:51:37.621650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343003935839554:11771]
2025-10-09T21:51:37.621663Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:37.621681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:37.621685Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:37.621688Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:37.621691Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:37.621701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:37.621704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:37.621683Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2197
2025-10-09T21:51:37.621690Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2197
2025-10-09T21:51:37.621731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343003935839554:11771]
2025-10-09T21:51:37.621742Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:37.621751Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:37.621759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.621763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.621810Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fd567b64-96f255bf-c8c3da7e-96ba993b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:37.621822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:37.621826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:37.621829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:37.621839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:37.621852Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:37.621876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:37.622050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.622055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.622068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:37.622160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2198 partition: 0 MaxSeqNo: 25 sessionId: src_id|fd567b64-96f255bf-c8c3da7e-96ba993b_0
2025-10-09T21:51:37.622469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd34de26-46a462ca-34f28a2d-de91a9f9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fd567b64-96f255bf-c8c3da7e-96ba993b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:37.624673Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fd567b64-96f255bf-c8c3da7e-96ba993b_0] Write session: try to update token
2025-10-09T21:51:37.624689Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fd567b64-96f255bf-c8c3da7e-96ba993b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:37.636021Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:37.636039Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:37.636041Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:37.636042Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:37.636050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343003935839554:11771]
2025-10-09T21:51:37.636054Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343003935839554:11771]
2025-10-09T21:51:37.636057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343003935839554:11771]
2025-10-09T21:51:37.636091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:37.636097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.636099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:37.636104Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:37.636111Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:37.636115Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:37.636225Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:37.637181Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2198 sessionId: src_id|fd567b64-96f255bf-c8c3da7e-96ba993b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:37.639673Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:37.691642Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343004297686354:9464]
2025-10-09T21:51:37.691920Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 2197, pipe: [1:7559343004297686354:9464]
2025-10-09T21:51:37.780474Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:37.781013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.781029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.781071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297686354:9464], now have 1 active actors on pipe
2025-10-09T21:51:37.781288Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:37.781289Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343004297686354:9464]
2025-10-09T21:51:37.781315Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:37.781319Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:37.781326Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:37.781386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2198 reason: error at write: Disconnected sessionId: src_id|fd567b64-96f255bf-c8c3da7e-96ba993b_0
2025-10-09T21:51:37.781536Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2198 sessionId: src_id|fd567b64-96f255bf-c8c3da7e-96ba993b_0 is DEAD
2025-10-09T21:51:37.781672Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd567b64-96f255bf-c8c3da7e-96ba993b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:37.781679Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd567b64-96f255bf-c8c3da7e-96ba993b_0] Write session will restart in 0.007014s
2025-10-09T21:51:37.781698Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd567b64-96f255bf-c8c3da7e-96ba993b_0] Write session: Do CDS request
2025-10-09T21:51:37.781703Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd567b64-96f255bf-c8c3da7e-96ba993b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:37.789457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fd567b64-96f255bf-c8c3da7e-96ba993b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:37.789522Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:37.789536Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2199
2025-10-09T21:51:37.789633Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2199 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:37.789664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2199 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:37.789674Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2199 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:37.789677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:37.790190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2199 sessionId:  describe result for acl check
2025-10-09T21:51:37.790224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:37.790230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:37.790232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:37.790237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343004297686522:8693] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:37.790765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343004297686522:8693] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:37.792794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343004297686522:8693] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:37.795390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343004297686522:8693] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:37.795402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343004297686522:8693] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:37.795405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343004297686522:8693] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:37.795410Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2199 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:37.797427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:37.799039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:37.799197Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:37.799204Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:37.799213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:37.799249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:37.799278Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343003935839841:11773]
2025-10-09T21:51:37.799656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:37.799906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:37.799935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:37.800137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:37.800166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.800171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.800172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.800172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.800173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.800173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.800174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.800175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.800176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.800179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.800185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.800188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.800190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.800192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.800195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.800197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.800198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.800200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.800213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.800218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:37.800222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.800225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:37.800228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.800231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:37.800235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.800238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:37.800241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.800244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:37.800246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.800250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:37.800252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.800255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.800258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.800262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.800265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.800268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.800286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:37.806246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:37.807295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:37.807309Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:37.807310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:37.807314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:37.807318Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2198 [2:7559343003935839841:11773]
2025-10-09T21:51:37.807327Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:37.807343Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:37.807346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:37.807349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:37.807351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:37.807368Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:37.807378Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:37.807385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:37.807427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.807432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.807433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.807435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:37.807486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:37.807491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.807498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:37.807504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:37.807513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:37.807530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343003935839832:11773]
2025-10-09T21:51:37.807535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343003935839832:11773]
2025-10-09T21:51:37.807537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343003935839832:11773]
2025-10-09T21:51:37.807545Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:37.807555Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:37.807559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297686760:8693], now have 1 active actors on pipe
2025-10-09T21:51:37.807562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297686356:2432], now have 1 active actors on pipe
2025-10-09T21:51:37.807564Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:37.807566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:37.807568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:37.807576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:37.807579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:37.807598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343003935839832:11773]
2025-10-09T21:51:37.807608Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:37.807616Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:37.807648Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2198
2025-10-09T21:51:37.807709Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2198
2025-10-09T21:51:37.807732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.807740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.807761Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|28c967d3-b6171278-9ef913c1-189d8a6c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:37.807771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:37.807774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:37.807776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:37.807783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:37.807795Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:37.807810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:37.807989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.808000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.808019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:37.808129Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2199 partition: 0 MaxSeqNo: 25 sessionId: src_id|28c967d3-b6171278-9ef913c1-189d8a6c_0
2025-10-09T21:51:37.808440Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fd567b64-96f255bf-c8c3da7e-96ba993b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|28c967d3-b6171278-9ef913c1-189d8a6c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:37.810740Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28c967d3-b6171278-9ef913c1-189d8a6c_0] Write session: try to update token
2025-10-09T21:51:37.810755Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28c967d3-b6171278-9ef913c1-189d8a6c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:37.818418Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:37.818433Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:37.818440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343003935839832:11773]
2025-10-09T21:51:37.818477Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:37.818497Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:37.818633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:37.818760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:37.818880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:37.819028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:37.819196Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:37.819231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:37.822712Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2199 sessionId: src_id|28c967d3-b6171278-9ef913c1-189d8a6c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:37.824777Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:37.824941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.824957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.824989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:37.825051Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:37.830400Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:37.830416Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:37.830419Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:37.830421Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:37.830430Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343003935839832:11773]
2025-10-09T21:51:37.830433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343003935839832:11773]
2025-10-09T21:51:37.830437Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343003935839832:11773]
2025-10-09T21:51:37.830476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:37.830483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:37.830483Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:37.830485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:37.830489Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:37.830492Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:37.830610Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:37.881594Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343004297686792:9464]
2025-10-09T21:51:37.881858Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 2198, pipe: [1:7559343004297686792:9464]
2025-10-09T21:51:37.972604Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:37.979062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:37.979084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:37.979229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.979286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.979340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.979404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.979447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.979456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.979513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.979564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.979617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.979680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.979718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.979726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.979776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.979830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.979885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.979948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.979990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.980000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.980063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.980125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.980188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.980245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.980290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.980298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.980356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.980412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.980471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.980530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.980573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.980580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.980634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.980692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.980745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.980796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.980832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.980839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.980890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.980943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.981001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.981058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.981108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.981116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.981169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.981225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.981284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.981333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.981378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.981384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.981433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.981481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.981530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.981579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.981618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.981624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.981682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:37.981736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:37.981790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:37.981843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:37.981885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:37.981890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:37.981928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297686792:9464], now have 1 active actors on pipe
2025-10-09T21:51:37.982196Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343004297686792:9464]
2025-10-09T21:51:37.982205Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:37.982210Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:37.982240Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2199 reason: error at write: Disconnected sessionId: src_id|28c967d3-b6171278-9ef913c1-189d8a6c_0
2025-10-09T21:51:37.982241Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:37.982440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2199 sessionId: src_id|28c967d3-b6171278-9ef913c1-189d8a6c_0 is DEAD
2025-10-09T21:51:37.982593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28c967d3-b6171278-9ef913c1-189d8a6c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:37.982601Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28c967d3-b6171278-9ef913c1-189d8a6c_0] Write session will restart in 0.008341s
2025-10-09T21:51:37.982620Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28c967d3-b6171278-9ef913c1-189d8a6c_0] Write session: Do CDS request
2025-10-09T21:51:37.982630Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28c967d3-b6171278-9ef913c1-189d8a6c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:37.982847Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:37.992473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28c967d3-b6171278-9ef913c1-189d8a6c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:37.992499Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:37.992513Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2200
2025-10-09T21:51:37.992681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2200 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:37.992710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2200 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:37.992724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2200 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:37.992727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:37.993047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2200 sessionId:  describe result for acl check
2025-10-09T21:51:37.993075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:37.993081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:37.993082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:37.993088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343004297686992:8671] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:37.993511Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343004297686992:8671] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:37.995697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343004297686992:8671] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:37.998239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343004297686992:8671] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:37.998249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343004297686992:8671] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:37.998251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343004297686992:8671] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:37.998256Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2200 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:37.999578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:38.001792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:38.001978Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:38.001990Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:38.001999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:38.002087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:38.002120Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343008230807429:11779]
2025-10-09T21:51:38.002547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:38.002863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:38.002897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:38.003109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:38.003142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.003146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.003147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.003148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.003148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.003149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.003151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.003152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.003152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.003156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.003162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.003164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.003166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.003168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.003170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.003172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.003174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.003176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.003181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.003186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.003190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.003193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.003196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.003199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.003202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.003205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.003207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.003210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.003213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.003216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.003218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.003222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.003224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.003228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.003230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.003233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.003249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:38.009585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:38.010749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:38.010760Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:38.010762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:38.010765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:38.010769Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2199 [2:7559343008230807429:11779]
2025-10-09T21:51:38.010779Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:38.010793Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:38.010795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:38.010797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:38.010798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:38.010811Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:38.010818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:38.010824Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:38.010878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.010885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.010887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.010890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:38.010956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:38.010969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.010980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.010991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.011008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:38.011034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297686798:2432], now have 1 active actors on pipe
2025-10-09T21:51:38.011041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343004297687229:8671], now have 1 active actors on pipe
2025-10-09T21:51:38.011048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343003935840128:11779]
2025-10-09T21:51:38.011050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343003935840128:11779]
2025-10-09T21:51:38.011051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343003935840128:11779]
2025-10-09T21:51:38.011063Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:38.011077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:38.011085Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:38.011088Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:38.011090Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:38.011107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:38.011111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:38.011104Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2199
2025-10-09T21:51:38.011142Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343003935840128:11779]
2025-10-09T21:51:38.011154Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:38.011154Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2199
2025-10-09T21:51:38.011162Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:38.011205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.011216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.011242Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|661982b3-5d233240-32be5152-c4a3e8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:38.011252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:38.011255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:38.011256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:38.011263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:38.011275Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:38.011289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:38.011407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.011410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.011419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:38.011484Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2200 partition: 0 MaxSeqNo: 25 sessionId: src_id|661982b3-5d233240-32be5152-c4a3e8_0
2025-10-09T21:51:38.011843Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28c967d3-b6171278-9ef913c1-189d8a6c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|661982b3-5d233240-32be5152-c4a3e8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:38.014352Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|661982b3-5d233240-32be5152-c4a3e8_0] Write session: try to update token
2025-10-09T21:51:38.014369Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|661982b3-5d233240-32be5152-c4a3e8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:38.023193Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:38.023207Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:38.023214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343003935840128:11779]
2025-10-09T21:51:38.023249Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:38.023253Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:38.023397Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:38.023521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:38.023641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:38.023768Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:38.024361Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:38.024400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:38.028480Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2200 sessionId: src_id|661982b3-5d233240-32be5152-c4a3e8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:38.030897Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:38.031095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.031111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.031145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:38.031242Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:38.036644Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:38.036658Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:38.036660Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:38.036661Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:38.036670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343003935840128:11779]
2025-10-09T21:51:38.036672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343003935840128:11779]
2025-10-09T21:51:38.036676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343003935840128:11779]
2025-10-09T21:51:38.036716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:38.036724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.036724Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:38.036726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:38.036732Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:38.036736Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:38.036865Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:38.082298Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343008592654550:9464]
2025-10-09T21:51:38.082588Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 2199, pipe: [1:7559343008592654550:9464]
2025-10-09T21:51:38.182781Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:38.189708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.189727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.189874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.189933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.189987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.190056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.190096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.190104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.190176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.190234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.190286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.190340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.190379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.190385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.190439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.190491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.190549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.190601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.190644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.190649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.190711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.190774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.190830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.190883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.190920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.190925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.190978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.191027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.191081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.191132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.191176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.191181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.191232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.191281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.191332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.191384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.191421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.191427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.191479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.191528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.191583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.191635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.191674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.191679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.191730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.191784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.191843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.191894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.191933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.191938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.191990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.192040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.192090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.192142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.192183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.192188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.192241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.192291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.192349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.192407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.192447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.192452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.192493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343008592654550:9464], now have 1 active actors on pipe
2025-10-09T21:51:38.192728Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:38.192744Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:38.192748Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343008592654550:9464]
2025-10-09T21:51:38.192764Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:38.192776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2200 reason: error at write: Disconnected sessionId: src_id|661982b3-5d233240-32be5152-c4a3e8_0
2025-10-09T21:51:38.192967Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2200 sessionId: src_id|661982b3-5d233240-32be5152-c4a3e8_0 is DEAD
2025-10-09T21:51:38.193090Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|661982b3-5d233240-32be5152-c4a3e8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:38.193110Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|661982b3-5d233240-32be5152-c4a3e8_0] Write session will restart in 0.006181s
2025-10-09T21:51:38.193135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|661982b3-5d233240-32be5152-c4a3e8_0] Write session: Do CDS request
2025-10-09T21:51:38.193141Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|661982b3-5d233240-32be5152-c4a3e8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:38.193206Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:38.200524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|661982b3-5d233240-32be5152-c4a3e8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:38.200642Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:38.200657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2201
2025-10-09T21:51:38.200794Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2201 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:38.200818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2201 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:38.200825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2201 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:38.200829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:38.201213Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2201 sessionId:  describe result for acl check
2025-10-09T21:51:38.201244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:38.201246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:38.201247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:38.201251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343008592654617:8667] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:38.201726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343008592654617:8667] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:38.204698Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343008592654617:8667] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:38.207748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343008592654617:8667] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:38.207764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343008592654617:8667] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:38.207768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343008592654617:8667] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:38.207777Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2201 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:38.210718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:38.212761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:38.212971Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:38.212983Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:38.212996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:38.213069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:38.213118Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343008230807735:11791]
2025-10-09T21:51:38.213637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:38.213952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:38.214000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:38.214328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:38.214402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.214409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.214410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.214411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.214413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.214414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.214415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.214416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.214417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.214424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.214435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.214440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.214445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.214453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.214457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.214461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.214464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.214468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.214476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.214489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.214500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.214510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.214522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.214527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.214532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.214537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.214542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.214553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.214562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.214568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.214577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.214587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.214596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.214605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.214615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.214624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.214658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:38.220823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:38.222128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:38.222142Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:38.222144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:38.222148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:38.222156Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2200 [2:7559343008230807735:11791]
2025-10-09T21:51:38.222166Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:38.222182Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:38.222187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:38.222190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:38.222191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:38.222206Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:38.222214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:38.222230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:38.222284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.222289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.222291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.222292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:38.222348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:38.222353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.222360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.222366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.222375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:38.222394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343008230807730:11791]
2025-10-09T21:51:38.222398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343008230807730:11791]
2025-10-09T21:51:38.222400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343008230807730:11791]
2025-10-09T21:51:38.222406Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:38.222418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:38.222420Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:38.222423Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:38.222425Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:38.222430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:38.222432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:38.222442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343008592654916:8667], now have 1 active actors on pipe
2025-10-09T21:51:38.222448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343008592654555:2432], now have 1 active actors on pipe
2025-10-09T21:51:38.222476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343008230807730:11791]
2025-10-09T21:51:38.222488Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:38.222506Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:38.222519Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2200
2025-10-09T21:51:38.222519Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2200
2025-10-09T21:51:38.222619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.222630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.222659Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|40c78f45-6f0ef88b-a8018b53-fc1e3b6b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:38.222669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:38.222672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:38.222674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:38.222682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:38.222695Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:38.222710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:38.222867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.222872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.222883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:38.222967Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2201 partition: 0 MaxSeqNo: 25 sessionId: src_id|40c78f45-6f0ef88b-a8018b53-fc1e3b6b_0
2025-10-09T21:51:38.223293Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|661982b3-5d233240-32be5152-c4a3e8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|40c78f45-6f0ef88b-a8018b53-fc1e3b6b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:38.225745Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|40c78f45-6f0ef88b-a8018b53-fc1e3b6b_0] Write session: try to update token
2025-10-09T21:51:38.225764Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|40c78f45-6f0ef88b-a8018b53-fc1e3b6b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:38.234337Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:38.234356Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:38.234366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343008230807730:11791]
2025-10-09T21:51:38.234414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:38.234418Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:38.234600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:38.234746Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:38.234886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:38.235019Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:38.235208Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:38.235250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:38.238407Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2201 sessionId: src_id|40c78f45-6f0ef88b-a8018b53-fc1e3b6b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:38.240727Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:38.240938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.240959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.241000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:38.241077Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:38.247474Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:38.247492Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:38.247494Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:38.247496Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:38.247507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343008230807730:11791]
2025-10-09T21:51:38.247511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343008230807730:11791]
2025-10-09T21:51:38.247516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343008230807730:11791]
2025-10-09T21:51:38.247559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:38.247565Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:38.247571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.247572Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:38.247574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:38.247576Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:38.247719Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:38.292736Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343008592654995:9464]
2025-10-09T21:51:38.293001Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 2200, pipe: [1:7559343008592654995:9464]
2025-10-09T21:51:38.391422Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:38.398440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.398461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.398597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.398654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.398713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.398776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.398827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.398838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.398904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.398958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.399010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.399066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.399104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.399109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.399171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.399226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.399279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.399330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.399370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.399373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.399436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.399493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.399546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.399600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.399639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.399644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.399700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.399758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.399821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.399884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.399922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.399927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.399983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.400035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.400092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.400146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.400183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.400188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.400240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.400289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.400347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.400397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.400441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.400447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.400502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.400554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.400606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.400657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.400697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.400702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.400754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.400810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.400872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.400931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.400969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.400974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.401035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.401090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.401143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.401194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.401232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.401237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.401278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343008592654995:9464], now have 1 active actors on pipe
2025-10-09T21:51:38.401519Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:38.401535Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:38.401539Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:38.401567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2201 reason: error at write: Disconnected sessionId: src_id|40c78f45-6f0ef88b-a8018b53-fc1e3b6b_0
2025-10-09T21:51:38.401583Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343008592654995:9464]
2025-10-09T21:51:38.401597Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:51:38.401667Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 22 consumer shared/user session shared/user_1_22_15461964503436223557_v1 is DEAD
2025-10-09T21:51:38.401752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2201 sessionId: src_id|40c78f45-6f0ef88b-a8018b53-fc1e3b6b_0 is DEAD
2025-10-09T21:51:38.401962Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|40c78f45-6f0ef88b-a8018b53-fc1e3b6b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:38.401968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|40c78f45-6f0ef88b-a8018b53-fc1e3b6b_0] Write session will restart in 0.009399s
2025-10-09T21:51:38.401985Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:51:38.401993Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|40c78f45-6f0ef88b-a8018b53-fc1e3b6b_0] Write session: Do CDS request
2025-10-09T21:51:38.401998Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|40c78f45-6f0ef88b-a8018b53-fc1e3b6b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:38.402085Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:51:38.402090Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:51:38.402097Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.009997s
2025-10-09T21:51:38.402153Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342926988239998:9473] disconnected.
2025-10-09T21:51:38.402168Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342926988239998:9473] disconnected; active server actors: 1
2025-10-09T21:51:38.402171Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559342926988239998:9473] client user disconnected session shared/user_1_22_15461964503436223557_v1
2025-10-09T21:51:38.402174Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:38.412543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|40c78f45-6f0ef88b-a8018b53-fc1e3b6b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:38.412591Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:38.412608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2202
2025-10-09T21:51:38.412760Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2202 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:38.412791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2202 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:38.412804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2202 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:38.412808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:38.413251Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2202 sessionId:  describe result for acl check
2025-10-09T21:51:38.413292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:38.413298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:38.413316Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:51:38.413300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:38.413306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343008592655216:8657] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:38.413425Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:51:38.413433Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 23
2025-10-09T21:51:38.413560Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 23 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:51:38.413605Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:51:38.413665Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 auth for : user
2025-10-09T21:51:38.413767Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 Handle describe topics response
2025-10-09T21:51:38.413784Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 auth is DEAD
2025-10-09T21:51:38.413797Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:51:38.413879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343008592655216:8657] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:38.413970Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:51:38.414044Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559343008592655251:8656] connected; active server actors: 1
2025-10-09T21:51:38.414056Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559343008592655251:8656] session shared/user_1_23_18068588973188758792_v1
2025-10-09T21:51:38.414063Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:51:38.414071Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:51:38.414075Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_23_18068588973188758792_v1" (Sender=[1:7559343008592655224:8656], Pipe=[1:7559343008592655251:8656], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:51:38.414078Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:51:38.414083Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_23_18068588973188758792_v1
2025-10-09T21:51:38.414085Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:51:38.414095Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_23_18068588973188758792_v1" (Sender=[1:7559343008592655224:8656], Pipe=[1:7559343008592655251:8656], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:51:38.414102Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_23_18068588973188758792_v1" sender [1:7559343008592655224:8656] lock partition 0 for ReadingSession "shared/user_1_23_18068588973188758792_v1" (Sender=[1:7559343008592655224:8656], Pipe=[1:7559343008592655251:8656], Partitions=[], ActiveFamilyCount=1) generation 1 step 23
2025-10-09T21:51:38.414115Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:51:38.414125Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000033s
2025-10-09T21:51:38.414297Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 23 Session: "shared/user_1_23_18068588973188758792_v1" ClientId: "user" PipeClient { RawX1: 7559343008592655251 RawX2: 4503603922346448 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:51:38.414327Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:51:38.416314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343008592655216:8657] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:38.419236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343008592655216:8657] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:38.419250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343008592655216:8657] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:38.419254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343008592655216:8657] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:38.419262Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2202 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:38.419405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:38.421150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:38.421310Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:38.421318Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:38.421327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:38.421380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:38.421408Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343008230808016:11793]
2025-10-09T21:51:38.421787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:38.422049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:38.422091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:38.422342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:38.422387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.422393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.422394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.422395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.422396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.422397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.422398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.422399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.422400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.422405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.422416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.422421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.422425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.422429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.422433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.422436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.422439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.422443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.422449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.422457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.422463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.422469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.422473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.422478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.422483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.422488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.422492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.422497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.422502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.422512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.422518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.422526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.422531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.422540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.422545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.422554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.422579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:38.428595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:38.429873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:38.429888Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:38.429890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:38.429894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:38.429900Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2201 [2:7559343008230808016:11793]
2025-10-09T21:51:38.429914Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:38.429931Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:38.429939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:38.429943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:38.429945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:38.429962Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:38.429970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:38.429982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:38.430059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.430065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.430067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.430069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:38.430138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:38.430143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.430155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.430171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.430192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:38.430216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343008592655435:8657], now have 1 active actors on pipe
2025-10-09T21:51:38.430219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343008592655270:8652], now have 1 active actors on pipe
2025-10-09T21:51:38.430222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343008592655003:2432], now have 1 active actors on pipe
2025-10-09T21:51:38.430229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343008230808010:11793]
2025-10-09T21:51:38.430231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343008230808010:11793]
2025-10-09T21:51:38.430233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343008230808010:11793]
2025-10-09T21:51:38.430245Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:38.430259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:38.430270Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:38.430272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:38.430274Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:38.430276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:38.430279Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:38.430259Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2201
2025-10-09T21:51:38.430304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343008230808010:11793]
2025-10-09T21:51:38.430288Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 2201, pipe: [1:7559343008592655270:8652]
2025-10-09T21:51:38.430311Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:38.430296Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2201
2025-10-09T21:51:38.430317Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:38.430326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.430330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.430346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.430352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.430358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_23_18068588973188758792_v1 on pipe: [1:7559343008592655270:8652]
2025-10-09T21:51:38.430400Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_23_18068588973188758792_v1:1 with generation 2201
2025-10-09T21:51:38.430405Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|523e38b-54d1c572-bf883e26-f5fa4528_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:38.430412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:38.430416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:38.430418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:38.430429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:38.430439Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:38.430455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:38.430465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:38.430467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:38.430481Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_23_18068588973188758792_v1
2025-10-09T21:51:38.430488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:38.430517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:38.430534Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:51:38.430625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.430634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.430647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:38.430718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2202 partition: 0 MaxSeqNo: 25 sessionId: src_id|523e38b-54d1c572-bf883e26-f5fa4528_0
2025-10-09T21:51:38.431035Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|40c78f45-6f0ef88b-a8018b53-fc1e3b6b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|523e38b-54d1c572-bf883e26-f5fa4528_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:38.433362Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|523e38b-54d1c572-bf883e26-f5fa4528_0] Write session: try to update token
2025-10-09T21:51:38.433377Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|523e38b-54d1c572-bf883e26-f5fa4528_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:38.433926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:51:38.433954Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:51:38.433964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:38.433966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:38.433968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:38.433976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:51:38.433989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:51:38.434136Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046698430 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:51:38.434156Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:51:38.434202Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 sending to client partition status
2025-10-09T21:51:38.443396Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:38.443412Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:38.443414Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:38.443416Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:38.443426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343008230808010:11793]
2025-10-09T21:51:38.443431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343008230808010:11793]
2025-10-09T21:51:38.443435Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343008230808010:11793]
2025-10-09T21:51:38.443474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:38.443482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.443488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:38.443488Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:38.443495Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:38.443498Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:38.443609Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:38.445852Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2202 sessionId: src_id|523e38b-54d1c572-bf883e26-f5fa4528_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:38.447965Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:38.588384Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:38.588914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.588929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.589131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_23_18068588973188758792_v1
2025-10-09T21:51:38.589144Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:38.589191Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:38.589208Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:38.589211Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343008592655270:8652]
2025-10-09T21:51:38.589230Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:38.589238Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_23_18068588973188758792_v1
2025-10-09T21:51:38.589268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2202 reason: error at write: Disconnected sessionId: src_id|523e38b-54d1c572-bf883e26-f5fa4528_0
2025-10-09T21:51:38.589461Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2202 sessionId: src_id|523e38b-54d1c572-bf883e26-f5fa4528_0 is DEAD
2025-10-09T21:51:38.589597Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|523e38b-54d1c572-bf883e26-f5fa4528_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:38.589624Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|523e38b-54d1c572-bf883e26-f5fa4528_0] Write session will restart in 0.005012s
2025-10-09T21:51:38.589648Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|523e38b-54d1c572-bf883e26-f5fa4528_0] Write session: Do CDS request
2025-10-09T21:51:38.589656Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|523e38b-54d1c572-bf883e26-f5fa4528_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:38.595475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|523e38b-54d1c572-bf883e26-f5fa4528_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:38.595583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:38.595599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2203
2025-10-09T21:51:38.595721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2203 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:38.595752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2203 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:38.595760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2203 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:38.595763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:38.596124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2203 sessionId:  describe result for acl check
2025-10-09T21:51:38.596153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:38.596158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:38.596160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:38.596164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343008592655490:8648] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:38.596628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343008592655490:8648] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:38.598986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343008592655490:8648] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:38.601825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343008592655490:8648] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:38.601841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343008592655490:8648] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:38.601845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343008592655490:8648] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:38.601854Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2203 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:38.605674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:38.607571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:38.607750Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:38.607761Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:38.607770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:38.607810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:38.607853Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343008230808301:11795]
2025-10-09T21:51:38.608246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:38.608523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:38.608562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:38.608782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:38.608816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.608821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.608821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.608822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.608823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.608824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.608825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.608825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.608826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.608829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.608836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.608838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.608841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.608843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.608845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.608847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.608849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.608851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.608856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.608861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.608867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.608872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.608876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.608881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.608885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.608889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.608894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.608898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.608903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.608907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.608912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.608919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.608923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.608927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.608931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.608936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.608962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:38.615280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:38.616222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:38.616236Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:38.616238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:38.616242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:38.616247Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2202 [2:7559343008230808301:11795]
2025-10-09T21:51:38.616259Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:38.616280Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:38.616285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:38.616289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:38.616291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:38.616310Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:38.616321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:38.616329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:38.616384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.616391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.616392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.616394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:38.616453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:38.616458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.616465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.616471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.616481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:38.616501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343008230808296:11795]
2025-10-09T21:51:38.616507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343008230808296:11795]
2025-10-09T21:51:38.616510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343008230808296:11795]
2025-10-09T21:51:38.616520Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:38.616527Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:38.616534Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:38.616536Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:38.616538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343008592655761:8648], now have 1 active actors on pipe
2025-10-09T21:51:38.616548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:38.616563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:38.616574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:38.616580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343008592655451:2432], now have 1 active actors on pipe
2025-10-09T21:51:38.616606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343008230808296:11795]
2025-10-09T21:51:38.616612Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:38.616634Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:38.616625Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2202
2025-10-09T21:51:38.616626Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2202
2025-10-09T21:51:38.616691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.616698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.616724Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f35fc982-12f25090-39291407-b37267d2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:38.616733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:38.616735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:38.616737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:38.616746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:38.616757Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:38.616774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:38.616886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.616891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.616902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:38.616987Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2203 partition: 0 MaxSeqNo: 25 sessionId: src_id|f35fc982-12f25090-39291407-b37267d2_0
2025-10-09T21:51:38.617353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|523e38b-54d1c572-bf883e26-f5fa4528_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f35fc982-12f25090-39291407-b37267d2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:38.619735Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f35fc982-12f25090-39291407-b37267d2_0] Write session: try to update token
2025-10-09T21:51:38.619751Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f35fc982-12f25090-39291407-b37267d2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:38.628032Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:38.628049Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:38.628060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343008230808296:11795]
2025-10-09T21:51:38.628098Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:38.628118Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:38.628282Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:38.628428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:38.628563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:38.628694Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:38.628866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:38.628904Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:38.632471Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2203 sessionId: src_id|f35fc982-12f25090-39291407-b37267d2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:38.634823Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:38.635025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.635046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.635083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:38.635176Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:38.640472Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:38.640485Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:38.640488Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:38.640490Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:38.640500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343008230808296:11795]
2025-10-09T21:51:38.640505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343008230808296:11795]
2025-10-09T21:51:38.640508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343008230808296:11795]
2025-10-09T21:51:38.640549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:38.640549Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:38.640553Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:38.640557Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:38.640557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.640560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:38.640684Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:38.689525Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343008592655891:8652]
2025-10-09T21:51:38.689741Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 2202, pipe: [1:7559343008592655891:8652]
2025-10-09T21:51:38.793094Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:38.801069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.801092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.801237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.801294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.801365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.801427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.801474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.801486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.801546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.801599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.801652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.801720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.801758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.801767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.801821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.802006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.802080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.802134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.802247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.802254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.802312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.802378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.802432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.802652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.802893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.802901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.803102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.803165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.803225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.803289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.803332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.803339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.803392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.803451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.803665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.803719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.803830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.803838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.803893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.803939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.803992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.804053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.804093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.804100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.804150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.804201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.804255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.804312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.804351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.804356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.804408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.804455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.804506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.804567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.804605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.804610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.804673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:38.804726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:38.804781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:38.804835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:38.804874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:38.804879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:38.804928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343008592655891:8652], now have 1 active actors on pipe
2025-10-09T21:51:38.805257Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343008592655891:8652]
2025-10-09T21:51:38.805276Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:38.805280Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:38.805283Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:38.805309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2203 reason: error at write: Disconnected sessionId: src_id|f35fc982-12f25090-39291407-b37267d2_0
2025-10-09T21:51:38.805505Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2203 sessionId: src_id|f35fc982-12f25090-39291407-b37267d2_0 is DEAD
2025-10-09T21:51:38.805675Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f35fc982-12f25090-39291407-b37267d2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:38.805685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f35fc982-12f25090-39291407-b37267d2_0] Write session will restart in 0.009210s
2025-10-09T21:51:38.805710Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f35fc982-12f25090-39291407-b37267d2_0] Write session: Do CDS request
2025-10-09T21:51:38.805716Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f35fc982-12f25090-39291407-b37267d2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:38.806067Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:38.816558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f35fc982-12f25090-39291407-b37267d2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:38.816615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:38.816634Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2204
2025-10-09T21:51:38.817263Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2204 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:38.817304Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2204 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:38.817310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2204 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:38.817313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:38.817740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2204 sessionId:  describe result for acl check
2025-10-09T21:51:38.817780Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:38.817786Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:38.817788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:38.817794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343008592656094:8643] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:38.818346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343008592656094:8643] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:38.821076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343008592656094:8643] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:38.823968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343008592656094:8643] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:38.823981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343008592656094:8643] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:38.823992Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343008592656094:8643] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:38.823998Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2204 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:38.825974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:38.828292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:38.828462Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:38.828469Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:38.828478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:38.828521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:38.828553Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343008230808588:11799]
2025-10-09T21:51:38.828982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:38.829355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:38.829402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:38.829673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:38.829733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.829738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.829740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.829744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.829746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.829747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.829748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.829750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.829751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.829757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.829768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.829770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.829773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.829777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.829781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.829783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.829785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.829788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.829793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.829799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:38.829807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.829810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:38.829813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.829816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:38.829819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.829821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:38.829824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.829829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:38.829832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.829837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:38.829841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.829848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.829852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.829858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.829862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.829865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.829894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:38.836050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:38.836994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:38.837007Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:38.837010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:38.837014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:38.837020Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2203 [2:7559343008230808588:11799]
2025-10-09T21:51:38.837032Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:38.837046Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:38.837048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:38.837050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:38.837051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:38.837064Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:38.837068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:38.837073Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:38.837118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.837123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.837124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.837126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:38.837173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:38.837178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.837185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:38.837195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:38.837208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:38.837224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343008230808583:11799]
2025-10-09T21:51:38.837229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343008230808583:11799]
2025-10-09T21:51:38.837231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343008230808583:11799]
2025-10-09T21:51:38.837238Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:38.837248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:38.837248Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:38.837251Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:38.837253Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:38.837260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:38.837262Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:38.837271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343008592655893:2432], now have 1 active actors on pipe
2025-10-09T21:51:38.837273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343008592656305:8643], now have 1 active actors on pipe
2025-10-09T21:51:38.837294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343008230808583:11799]
2025-10-09T21:51:38.837302Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:38.837311Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:38.837328Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2203
2025-10-09T21:51:38.837367Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2203
2025-10-09T21:51:38.837418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.837425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.837453Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3c08ce39-da65fa47-5834bb9d-8ee13834_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:38.837465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:38.837468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:38.837474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:38.837485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:38.837498Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:38.837515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:38.837771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:38.837783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:38.837803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:38.837920Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2204 partition: 0 MaxSeqNo: 25 sessionId: src_id|3c08ce39-da65fa47-5834bb9d-8ee13834_0
2025-10-09T21:51:38.838276Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f35fc982-12f25090-39291407-b37267d2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3c08ce39-da65fa47-5834bb9d-8ee13834_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:38.845379Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c08ce39-da65fa47-5834bb9d-8ee13834_0] Write session: try to update token
2025-10-09T21:51:38.845400Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c08ce39-da65fa47-5834bb9d-8ee13834_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:38.848871Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:38.848883Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:38.848890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343008230808583:11799]
2025-10-09T21:51:38.848929Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:38.848930Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:38.849092Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:38.849243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:38.849386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:38.849510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:38.849692Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:38.849730Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:38.860682Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2204 sessionId: src_id|3c08ce39-da65fa47-5834bb9d-8ee13834_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:38.861543Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:38.861558Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:38.861560Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:38.861561Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:38.861569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343008230808583:11799]
2025-10-09T21:51:38.861572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343008230808583:11799]
2025-10-09T21:51:38.861575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343008230808583:11799]
2025-10-09T21:51:38.861609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:38.861614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:38.861616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:38.861624Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:38.861629Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:38.861633Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:38.861798Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:38.863107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:38.905579Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343008592656336:8652]
2025-10-09T21:51:38.905931Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 2203, pipe: [1:7559343008592656336:8652]
2025-10-09T21:51:39.014204Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:39.014769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.014784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.014805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343008592656336:8652], now have 1 active actors on pipe
2025-10-09T21:51:39.015060Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:39.015089Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:39.015062Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:39.015075Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:39.015097Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343008592656336:8652]
2025-10-09T21:51:39.015130Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2204 reason: error at write: Disconnected sessionId: src_id|3c08ce39-da65fa47-5834bb9d-8ee13834_0
2025-10-09T21:51:39.015300Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2204 sessionId: src_id|3c08ce39-da65fa47-5834bb9d-8ee13834_0 is DEAD
2025-10-09T21:51:39.015432Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c08ce39-da65fa47-5834bb9d-8ee13834_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:39.015445Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c08ce39-da65fa47-5834bb9d-8ee13834_0] Write session will restart in 0.006388s
2025-10-09T21:51:39.015474Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c08ce39-da65fa47-5834bb9d-8ee13834_0] Write session: Do CDS request
2025-10-09T21:51:39.015480Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c08ce39-da65fa47-5834bb9d-8ee13834_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:39.022564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3c08ce39-da65fa47-5834bb9d-8ee13834_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:39.022601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:39.022625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2205
2025-10-09T21:51:39.022768Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2205 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:39.022799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2205 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:39.022808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2205 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:39.022812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:39.023136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2205 sessionId:  describe result for acl check
2025-10-09T21:51:39.023170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:39.023176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:39.023178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:39.023183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343012887623672:8639] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:39.023656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343012887623672:8639] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:39.026065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343012887623672:8639] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:39.028789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343012887623672:8639] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:39.028803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343012887623672:8639] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:39.028807Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343012887623672:8639] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:39.028815Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2205 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:39.034000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:39.036009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:39.036215Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:39.036226Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:39.036238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:39.036302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:39.036340Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343012525776181:11801]
2025-10-09T21:51:39.036918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:39.037192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:39.037231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:39.037555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:39.037625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.037632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.037633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.037634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.037635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.037637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.037638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.037639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.037640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.037645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.037656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.037660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.037664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.037668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.037672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.037676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.037685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.037689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.037699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.037711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.037722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.037732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.037741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.037751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.037759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.037764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.037773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.037780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.037789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.037794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.037804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.037813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.037821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.037830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.037835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.037840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.037874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:39.044125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:39.045206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:39.045221Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:39.045224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:39.045229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:39.045235Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2204 [2:7559343012525776181:11801]
2025-10-09T21:51:39.045250Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:39.045268Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:39.045276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:39.045279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:39.045281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:39.045299Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:39.045310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:39.045317Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:39.045386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.045393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.045395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.045398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:39.045466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:39.045475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.045487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.045504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.045527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:39.045553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343012887623942:8639], now have 1 active actors on pipe
2025-10-09T21:51:39.045561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343012887623637:2432], now have 1 active actors on pipe
2025-10-09T21:51:39.045569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343012525776176:11801]
2025-10-09T21:51:39.045575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343012525776176:11801]
2025-10-09T21:51:39.045578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343012525776176:11801]
2025-10-09T21:51:39.045590Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:39.045607Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:39.045608Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:39.045617Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:39.045619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:39.045623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:39.045626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:39.045625Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2204
2025-10-09T21:51:39.045659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343012525776176:11801]
2025-10-09T21:51:39.045668Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:39.045679Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:39.045677Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2204
2025-10-09T21:51:39.045706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.045715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.045745Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fc1f6ca1-8dc9e6a5-ffd20c64-ab61e511_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:39.045756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:39.045765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:39.045767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:39.045777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:39.045791Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:39.045812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:39.045994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.046005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.046022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:39.046151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2205 partition: 0 MaxSeqNo: 25 sessionId: src_id|fc1f6ca1-8dc9e6a5-ffd20c64-ab61e511_0
2025-10-09T21:51:39.046487Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3c08ce39-da65fa47-5834bb9d-8ee13834_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fc1f6ca1-8dc9e6a5-ffd20c64-ab61e511_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:39.049056Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc1f6ca1-8dc9e6a5-ffd20c64-ab61e511_0] Write session: try to update token
2025-10-09T21:51:39.049069Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc1f6ca1-8dc9e6a5-ffd20c64-ab61e511_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:39.060009Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:39.060029Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:39.060031Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:39.060033Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:39.060042Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343012525776176:11801]
2025-10-09T21:51:39.060047Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343012525776176:11801]
2025-10-09T21:51:39.060050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343012525776176:11801]
2025-10-09T21:51:39.060098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:39.060110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.060109Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:39.060113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:39.060115Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:39.060119Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:39.060252Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:39.062538Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2205 sessionId: src_id|fc1f6ca1-8dc9e6a5-ffd20c64-ab61e511_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:39.064878Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:39.114963Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343012887624097:8652]
2025-10-09T21:51:39.115245Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 2204, pipe: [1:7559343012887624097:8652]
2025-10-09T21:51:39.210110Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:39.210364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.210391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.210439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343012887624097:8652], now have 1 active actors on pipe
2025-10-09T21:51:39.210642Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:39.210760Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:39.210760Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343012887624097:8652]
2025-10-09T21:51:39.210786Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:39.210876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2205 reason: status is not ok: tablet will be restarted right now sessionId: src_id|fc1f6ca1-8dc9e6a5-ffd20c64-ab61e511_0
2025-10-09T21:51:39.211058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2205 sessionId: src_id|fc1f6ca1-8dc9e6a5-ffd20c64-ab61e511_0 is DEAD
2025-10-09T21:51:39.211214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc1f6ca1-8dc9e6a5-ffd20c64-ab61e511_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:39.211225Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc1f6ca1-8dc9e6a5-ffd20c64-ab61e511_0] Write session will restart in 0.006000s
2025-10-09T21:51:39.211253Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc1f6ca1-8dc9e6a5-ffd20c64-ab61e511_0] Write session: Do CDS request
2025-10-09T21:51:39.211260Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc1f6ca1-8dc9e6a5-ffd20c64-ab61e511_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:39.218564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc1f6ca1-8dc9e6a5-ffd20c64-ab61e511_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:39.218637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:39.218654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2206
2025-10-09T21:51:39.218769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2206 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:39.218804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2206 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:39.218832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2206 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:39.218836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:39.219252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2206 sessionId:  describe result for acl check
2025-10-09T21:51:39.219288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:39.219294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:39.219296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:39.219304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343012887624136:8618] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:39.219895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343012887624136:8618] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:39.222628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343012887624136:8618] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:39.225275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343012887624136:8618] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:39.225295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343012887624136:8618] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:39.225298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343012887624136:8618] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:39.225305Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2206 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:39.230709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:39.232758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:39.232965Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:39.232975Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:39.232987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:39.233035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:39.233073Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343012525776492:11815]
2025-10-09T21:51:39.233692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:39.233970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:39.234016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:39.234356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:39.234389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.234394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.234395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.234395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.234396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.234397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.234398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.234398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.234399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.234402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.234410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.234412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.234414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.234416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.234418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.234420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.234422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.234424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.234430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.234435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.234439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.234442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.234445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.234449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.234451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.234454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.234457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.234459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.234463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.234466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.234469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.234472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.234474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.234477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.234487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.234490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.234506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:39.240595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:39.241591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:39.241603Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:39.241604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:39.241608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:39.241614Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2205 [2:7559343012525776492:11815]
2025-10-09T21:51:39.241624Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:39.241639Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:39.241641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:39.241643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:39.241645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:39.241658Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:39.241666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:39.241671Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:39.241712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.241716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.241718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.241720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:39.241773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:39.241780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.241787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.241795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.241807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:39.241829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343012525776487:11815]
2025-10-09T21:51:39.241835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343012525776487:11815]
2025-10-09T21:51:39.241837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343012525776487:11815]
2025-10-09T21:51:39.241850Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:39.241869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343012887624100:2432], now have 1 active actors on pipe
2025-10-09T21:51:39.241869Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:39.241872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:39.241874Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:39.241876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343012887624394:8618], now have 1 active actors on pipe
2025-10-09T21:51:39.241885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:39.241900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:39.241908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:39.241938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343012525776487:11815]
2025-10-09T21:51:39.241950Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:39.241972Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:39.242047Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2205
2025-10-09T21:51:39.242113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.242126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.242157Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e41dac34-29be20a3-14bbc872-c6f02a04_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:39.242168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:39.242175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:39.242176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:39.242186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:39.242199Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:39.242221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:39.242219Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2205
2025-10-09T21:51:39.242348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.242360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.242371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:39.242477Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2206 partition: 0 MaxSeqNo: 25 sessionId: src_id|e41dac34-29be20a3-14bbc872-c6f02a04_0
2025-10-09T21:51:39.242793Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc1f6ca1-8dc9e6a5-ffd20c64-ab61e511_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e41dac34-29be20a3-14bbc872-c6f02a04_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:39.245090Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e41dac34-29be20a3-14bbc872-c6f02a04_0] Write session: try to update token
2025-10-09T21:51:39.245104Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e41dac34-29be20a3-14bbc872-c6f02a04_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:39.256073Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:39.256090Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:39.256092Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:39.256094Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:39.256103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343012525776487:11815]
2025-10-09T21:51:39.256108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343012525776487:11815]
2025-10-09T21:51:39.256114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343012525776487:11815]
2025-10-09T21:51:39.256153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:39.256158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.256160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:39.256161Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:39.256168Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:39.256172Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:39.256274Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:39.257656Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2206 sessionId: src_id|e41dac34-29be20a3-14bbc872-c6f02a04_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:39.260068Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:39.311100Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343012887624550:8652]
2025-10-09T21:51:39.311407Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 2205, pipe: [1:7559343012887624550:8652]
2025-10-09T21:51:39.399524Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:39.400109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.400123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.400163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343012887624550:8652], now have 1 active actors on pipe
2025-10-09T21:51:39.400341Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:39.400428Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:39.400434Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:39.400457Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:39.400480Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343012887624550:8652]
2025-10-09T21:51:39.400509Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2206 reason: error at write: Disconnected sessionId: src_id|e41dac34-29be20a3-14bbc872-c6f02a04_0
2025-10-09T21:51:39.400690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2206 sessionId: src_id|e41dac34-29be20a3-14bbc872-c6f02a04_0 is DEAD
2025-10-09T21:51:39.400875Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e41dac34-29be20a3-14bbc872-c6f02a04_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:39.400885Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e41dac34-29be20a3-14bbc872-c6f02a04_0] Write session will restart in 0.007743s
2025-10-09T21:51:39.400905Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e41dac34-29be20a3-14bbc872-c6f02a04_0] Write session: Do CDS request
2025-10-09T21:51:39.400910Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e41dac34-29be20a3-14bbc872-c6f02a04_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:39.409570Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e41dac34-29be20a3-14bbc872-c6f02a04_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:39.409631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:39.409664Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2207
2025-10-09T21:51:39.409778Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2207 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:39.409811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2207 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:39.409819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2207 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:39.409822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:39.410230Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2207 sessionId:  describe result for acl check
2025-10-09T21:51:39.410264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:39.410271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:39.410273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:39.410278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343012887624611:8607] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:39.410712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343012887624611:8607] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:39.413128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343012887624611:8607] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:39.415857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343012887624611:8607] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:39.415868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343012887624611:8607] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:39.415871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343012887624611:8607] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:39.415878Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2207 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:39.422018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:39.424752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:39.424952Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:39.424962Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:39.424974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:39.425028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:39.425070Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343012525776777:11817]
2025-10-09T21:51:39.425673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:39.426065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:39.426106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:39.426413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:39.426458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.426464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.426465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.426466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.426468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.426469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.426470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.426472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.426473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.426477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.426489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.426497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.426502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.426505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.426510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.426514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.426517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.426521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.426533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.426541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.426552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.426562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.426571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.426580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.426586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.426595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.426603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.426608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.426613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.426618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.426627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.426637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.426642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.426647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.426656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.426666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.426697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:39.433376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:39.434626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:39.434642Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:39.434644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:39.434648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:39.434655Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2206 [2:7559343012525776777:11817]
2025-10-09T21:51:39.434669Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:39.434687Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:39.434690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:39.434693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:39.434695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:39.434712Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:39.434726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:39.434735Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:39.434798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.434806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.434808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.434811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:39.434886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:39.434893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.434904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.434920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.434940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:39.434961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343012887624553:2432], now have 1 active actors on pipe
2025-10-09T21:51:39.434970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343012887624825:8607], now have 1 active actors on pipe
2025-10-09T21:51:39.434978Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343012525776772:11817]
2025-10-09T21:51:39.434981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343012525776772:11817]
2025-10-09T21:51:39.434983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343012525776772:11817]
2025-10-09T21:51:39.434995Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:39.435006Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:39.435008Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:39.435010Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:39.435015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:39.435028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:39.435031Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:39.435065Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343012525776772:11817]
2025-10-09T21:51:39.435076Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:39.435084Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:39.435061Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2206
2025-10-09T21:51:39.435086Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2206
2025-10-09T21:51:39.435132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.435142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.435170Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4ac49d-97fb26de-34423cf9-66e539b3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:39.435184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:39.435188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:39.435191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:39.435201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:39.435211Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:39.435225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:39.435398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.435407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.435423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:39.435524Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2207 partition: 0 MaxSeqNo: 25 sessionId: src_id|4ac49d-97fb26de-34423cf9-66e539b3_0
2025-10-09T21:51:39.435924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e41dac34-29be20a3-14bbc872-c6f02a04_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4ac49d-97fb26de-34423cf9-66e539b3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:39.438309Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ac49d-97fb26de-34423cf9-66e539b3_0] Write session: try to update token
2025-10-09T21:51:39.438327Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ac49d-97fb26de-34423cf9-66e539b3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:39.447438Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:39.447457Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:39.447466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343012525776772:11817]
2025-10-09T21:51:39.447510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:39.447511Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:39.447669Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:39.447806Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:39.447941Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:39.448072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:39.448246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:39.448283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:39.451183Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2207 sessionId: src_id|4ac49d-97fb26de-34423cf9-66e539b3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:39.453873Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:39.454082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.454098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.454133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:39.454270Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:39.460759Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:39.460773Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:39.460775Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:39.460778Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:39.460785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343012525776772:11817]
2025-10-09T21:51:39.460790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343012525776772:11817]
2025-10-09T21:51:39.460794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343012525776772:11817]
2025-10-09T21:51:39.460828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:39.460831Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:39.460836Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:39.460839Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:39.460839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.460842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:39.460965Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:39.500564Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343012887624996:8652]
2025-10-09T21:51:39.500950Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 2206, pipe: [1:7559343012887624996:8652]
2025-10-09T21:51:39.602019Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:39.609301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.609321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.609471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.609525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.609582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.609639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.609681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.609689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.609755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.609812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.609868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.609925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.609973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.609981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.610050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.610101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.610158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.610210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.610254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.610260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.610317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.610371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.610425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.610478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.610515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.610521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.610576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.610631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.610683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.610738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.610775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.610784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.610844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.610895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.610950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.611035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.611073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.611078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.611132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.611185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.611240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.611301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.611346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.611352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.611404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.611453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.611503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.611553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.611603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.611609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.611665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.611725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.611776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.611830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.611924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.611930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.611987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.612052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.612105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.612163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.612285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.612294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.612335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343012887624996:8652], now have 1 active actors on pipe
2025-10-09T21:51:39.612587Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:39.612605Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:39.612608Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:39.612639Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2207 reason: error at write: Disconnected sessionId: src_id|4ac49d-97fb26de-34423cf9-66e539b3_0
2025-10-09T21:51:39.612650Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343012887624996:8652]
2025-10-09T21:51:39.612815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2207 sessionId: src_id|4ac49d-97fb26de-34423cf9-66e539b3_0 is DEAD
2025-10-09T21:51:39.612964Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ac49d-97fb26de-34423cf9-66e539b3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:39.612977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ac49d-97fb26de-34423cf9-66e539b3_0] Write session will restart in 0.007439s
2025-10-09T21:51:39.613013Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ac49d-97fb26de-34423cf9-66e539b3_0] Write session: Do CDS request
2025-10-09T21:51:39.613019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ac49d-97fb26de-34423cf9-66e539b3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:39.613363Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:39.621575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ac49d-97fb26de-34423cf9-66e539b3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:39.621639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:39.621663Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2208
2025-10-09T21:51:39.621816Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2208 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:39.621858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2208 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:39.621871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2208 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:39.621876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:39.622319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2208 sessionId:  describe result for acl check
2025-10-09T21:51:39.622352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:39.622353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:39.622355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:39.622358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343012887625097:8603] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:39.622803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343012887625097:8603] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:39.625359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343012887625097:8603] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:39.628375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343012887625097:8603] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:39.628388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343012887625097:8603] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:39.628391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343012887625097:8603] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:39.628398Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2208 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:39.631978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:39.634087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:39.634273Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:39.634284Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:39.634296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:39.634346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:39.634385Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343012525777065:11819]
2025-10-09T21:51:39.634933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:39.635228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:39.635273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:39.635577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:39.635627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.635633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.635634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.635635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.635636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.635638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.635639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.635640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.635641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.635646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.635657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.635661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.635665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.635669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.635673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.635677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.635681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.635685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.635691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.635699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.635705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.635710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.635714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.635718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.635720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.635723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.635725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.635728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.635730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.635733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.635736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.635739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.635742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.635745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.635747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.635750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.635765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:39.641882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:39.643182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:39.643195Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:39.643197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:39.643201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:39.643208Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2207 [2:7559343012525777065:11819]
2025-10-09T21:51:39.643221Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:39.643238Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:39.643245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:39.643248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:39.643251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:39.643268Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:39.643281Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:39.643292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:39.643354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.643362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.643364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.643367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:39.643434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:39.643440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.643453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.643469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.643489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:39.643512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343012887625373:8603], now have 1 active actors on pipe
2025-10-09T21:51:39.643518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343012887624999:2432], now have 1 active actors on pipe
2025-10-09T21:51:39.643526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343012525777060:11819]
2025-10-09T21:51:39.643528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343012525777060:11819]
2025-10-09T21:51:39.643530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343012525777060:11819]
2025-10-09T21:51:39.643542Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:39.643551Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:39.643555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:39.643558Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:39.643560Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:39.643568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:39.643571Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:39.643600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343012525777060:11819]
2025-10-09T21:51:39.643611Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:39.643597Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2207
2025-10-09T21:51:39.643625Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:39.643660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.643670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.643661Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2207
2025-10-09T21:51:39.643697Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|53c2e919-33018fd6-cb86fba2-cfab1c8a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:39.643708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:39.643715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:39.643718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:39.643728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:39.643737Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:39.643759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:39.643906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.643913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.643925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:39.644012Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2208 partition: 0 MaxSeqNo: 25 sessionId: src_id|53c2e919-33018fd6-cb86fba2-cfab1c8a_0
2025-10-09T21:51:39.644347Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ac49d-97fb26de-34423cf9-66e539b3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|53c2e919-33018fd6-cb86fba2-cfab1c8a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:39.646754Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53c2e919-33018fd6-cb86fba2-cfab1c8a_0] Write session: try to update token
2025-10-09T21:51:39.646773Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53c2e919-33018fd6-cb86fba2-cfab1c8a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:39.655052Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:39.655076Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:39.655087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343012525777060:11819]
2025-10-09T21:51:39.655143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:39.655151Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:39.655343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:39.655498Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:39.655664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:39.655808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:39.656008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:39.656051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:39.659407Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2208 sessionId: src_id|53c2e919-33018fd6-cb86fba2-cfab1c8a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:39.661505Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:39.661702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.661726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.661768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:39.661837Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:39.667224Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:39.667239Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:39.667242Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:39.667244Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:39.667252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343012525777060:11819]
2025-10-09T21:51:39.667257Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343012525777060:11819]
2025-10-09T21:51:39.667261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343012525777060:11819]
2025-10-09T21:51:39.667303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:39.667303Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:39.667310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.667311Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:39.667313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:39.667315Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:39.667436Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:39.712975Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343012887625444:8652]
2025-10-09T21:51:39.713241Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 2207, pipe: [1:7559343012887625444:8652]
2025-10-09T21:51:39.811671Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:39.818201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.818221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.818352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.818402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.818453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.818506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.818544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.818553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.818612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.818665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.818717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.818770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.818807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.818812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.818862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.818909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.818959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.819011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.819047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.819052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.819105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.819157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.819214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.819285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.819322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.819327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.819380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.819433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.819484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.819534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.819570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.819575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.819646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.819701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.819755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.819810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.819861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.819866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.819915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.819961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.820008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.820059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.820094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.820101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.820152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.820200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.820251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.820301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.820341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.820346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.820395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.820441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.820491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.820541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.820576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.820584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.820633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:39.820680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:39.820729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:39.820780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:39.820830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:39.820835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:39.820872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343012887625444:8652], now have 1 active actors on pipe
2025-10-09T21:51:39.821147Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343012887625444:8652]
2025-10-09T21:51:39.821157Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:39.821165Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:39.821190Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:39.821194Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2208 reason: error at write: Disconnected sessionId: src_id|53c2e919-33018fd6-cb86fba2-cfab1c8a_0
2025-10-09T21:51:39.821363Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2208 sessionId: src_id|53c2e919-33018fd6-cb86fba2-cfab1c8a_0 is DEAD
2025-10-09T21:51:39.821522Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53c2e919-33018fd6-cb86fba2-cfab1c8a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:39.821534Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53c2e919-33018fd6-cb86fba2-cfab1c8a_0] Write session will restart in 0.007388s
2025-10-09T21:51:39.821558Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53c2e919-33018fd6-cb86fba2-cfab1c8a_0] Write session: Do CDS request
2025-10-09T21:51:39.821563Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53c2e919-33018fd6-cb86fba2-cfab1c8a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:39.821617Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:39.830418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|53c2e919-33018fd6-cb86fba2-cfab1c8a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:39.830524Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:39.830540Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2209
2025-10-09T21:51:39.830632Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2209 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:39.830666Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2209 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:39.830677Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2209 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:39.830680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:39.831061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2209 sessionId:  describe result for acl check
2025-10-09T21:51:39.831091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:39.831093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:39.831094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:39.831099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343012887625595:8600] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:39.831598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343012887625595:8600] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:39.834272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343012887625595:8600] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:39.836891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343012887625595:8600] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:39.836901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343012887625595:8600] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:39.836904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343012887625595:8600] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:39.836909Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2209 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:39.839016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:39.840611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:39.840776Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:39.840785Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:39.840796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:39.840840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:39.840876Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343012525777357:11823]
2025-10-09T21:51:39.841415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:39.841723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:39.841756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:39.842044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:39.842094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.842100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.842101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.842102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.842104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.842105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.842106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.842107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.842108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.842113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.842123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.842127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.842131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.842134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.842138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.842141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.842145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.842147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.842154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.842167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:39.842173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.842178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:39.842182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.842186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:39.842190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.842195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:39.842199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.842204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:39.842209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.842214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:39.842225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.842233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.842241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.842246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.842251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.842260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.842285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:39.847849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:39.848777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:39.848789Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:39.848790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:39.848794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:39.848799Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2208 [2:7559343012525777357:11823]
2025-10-09T21:51:39.848807Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:39.848820Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:39.848825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:39.848827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:39.848828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:39.848839Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:39.848846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:39.848851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:39.848893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.848897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.848898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.848900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:39.848955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:39.848961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.848967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:39.848973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:39.848985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:39.849003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343012887625446:2432], now have 1 active actors on pipe
2025-10-09T21:51:39.849012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343012525777352:11823]
2025-10-09T21:51:39.849014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343012525777352:11823]
2025-10-09T21:51:39.849015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343012525777352:11823]
2025-10-09T21:51:39.849024Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:39.849033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:39.849040Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:39.849043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:39.849044Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:39.849045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:39.849046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:39.849050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343012887625867:8600], now have 1 active actors on pipe
2025-10-09T21:51:39.849064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343012525777352:11823]
2025-10-09T21:51:39.849070Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:39.849072Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:39.849092Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2208
2025-10-09T21:51:39.849097Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2208
2025-10-09T21:51:39.849155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.849162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.849186Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|58d12eb6-98db7b8-9d143c26-f0eb7c20_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:39.849197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:39.849203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:39.849204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:39.849212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:39.849223Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:39.849233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:39.849429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:39.849441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:39.849455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:39.849530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2209 partition: 0 MaxSeqNo: 25 sessionId: src_id|58d12eb6-98db7b8-9d143c26-f0eb7c20_0
2025-10-09T21:51:39.849822Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|53c2e919-33018fd6-cb86fba2-cfab1c8a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|58d12eb6-98db7b8-9d143c26-f0eb7c20_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:39.852224Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|58d12eb6-98db7b8-9d143c26-f0eb7c20_0] Write session: try to update token
2025-10-09T21:51:39.852243Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|58d12eb6-98db7b8-9d143c26-f0eb7c20_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:39.862365Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:39.862385Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:39.862387Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:39.862389Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:39.862399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343012525777352:11823]
2025-10-09T21:51:39.862403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343012525777352:11823]
2025-10-09T21:51:39.862407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343012525777352:11823]
2025-10-09T21:51:39.862450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:39.862453Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:39.862457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:39.862460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:39.862466Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:39.862473Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:39.862606Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:39.864444Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2209 sessionId: src_id|58d12eb6-98db7b8-9d143c26-f0eb7c20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:39.866694Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:39.921373Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343012887625894:8652]
2025-10-09T21:51:39.921683Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 2208, pipe: [1:7559343012887625894:8652]
2025-10-09T21:51:40.009149Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:40.010518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.010540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.010590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343012887625894:8652], now have 1 active actors on pipe
2025-10-09T21:51:40.010852Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:40.010858Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343012887625894:8652]
2025-10-09T21:51:40.010881Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:40.010884Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:40.010895Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:40.010962Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2209 reason: error at write: Disconnected sessionId: src_id|58d12eb6-98db7b8-9d143c26-f0eb7c20_0
2025-10-09T21:51:40.011185Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2209 sessionId: src_id|58d12eb6-98db7b8-9d143c26-f0eb7c20_0 is DEAD
2025-10-09T21:51:40.011404Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|58d12eb6-98db7b8-9d143c26-f0eb7c20_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:40.011419Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|58d12eb6-98db7b8-9d143c26-f0eb7c20_0] Write session will restart in 0.005816s
2025-10-09T21:51:40.011457Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|58d12eb6-98db7b8-9d143c26-f0eb7c20_0] Write session: Do CDS request
2025-10-09T21:51:40.011464Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|58d12eb6-98db7b8-9d143c26-f0eb7c20_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:40.018609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|58d12eb6-98db7b8-9d143c26-f0eb7c20_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:40.018700Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:40.018715Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2210
2025-10-09T21:51:40.018821Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2210 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:40.018854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2210 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:40.018863Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2210 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:40.018867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:40.019516Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2210 sessionId:  describe result for acl check
2025-10-09T21:51:40.019549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:40.019552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:40.019553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:40.019558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343017182593236:8593] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:40.020065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343017182593236:8593] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:40.022385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343017182593236:8593] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:40.025316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343017182593236:8593] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:40.025332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343017182593236:8593] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:40.025358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343017182593236:8593] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:40.025369Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2210 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:40.031614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:40.033741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:40.033997Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:40.034006Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:40.034020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:40.034099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:40.034144Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343016820744942:11825]
2025-10-09T21:51:40.034804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:40.035112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:40.035154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:40.035485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:40.035540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.035548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.035550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.035552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.035553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.035554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.035555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.035557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.035558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.035563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.035575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.035579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.035584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.035588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.035593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.035598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.035606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.035610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.035618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.035627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.035633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.035638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.035643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.035648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.035657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.035662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.035670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.035675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.035683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.035691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.035700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.035710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.035719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.035724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.035733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.035739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.035773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:40.042598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:40.044124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:40.044145Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:40.044148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:40.044154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:40.044162Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2209 [2:7559343016820744942:11825]
2025-10-09T21:51:40.044177Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:40.044198Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:40.044207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:40.044212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:40.044215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:40.044234Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:40.044246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:40.044259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:40.044329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.044337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.044339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.044342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:40.044424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:40.044432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.044445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.044464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.044489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:40.044516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182593193:2432], now have 1 active actors on pipe
2025-10-09T21:51:40.044524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182593462:8593], now have 1 active actors on pipe
2025-10-09T21:51:40.044532Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343016820744937:11825]
2025-10-09T21:51:40.044535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343016820744937:11825]
2025-10-09T21:51:40.044537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343016820744937:11825]
2025-10-09T21:51:40.044551Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:40.044559Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:40.044562Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:40.044565Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:40.044571Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:40.044591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:40.044598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:40.044558Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2209
2025-10-09T21:51:40.044575Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2209
2025-10-09T21:51:40.044661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.044670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.044685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343016820744937:11825]
2025-10-09T21:51:40.044697Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:40.044704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:40.044737Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f89007dd-4c3e85ef-5d577f2b-e67c3daa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:40.044750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:40.044759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:40.044762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:40.044773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:40.044790Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:40.044823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:40.044993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.045004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.045021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:40.045156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2210 partition: 0 MaxSeqNo: 25 sessionId: src_id|f89007dd-4c3e85ef-5d577f2b-e67c3daa_0
2025-10-09T21:51:40.045603Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|58d12eb6-98db7b8-9d143c26-f0eb7c20_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f89007dd-4c3e85ef-5d577f2b-e67c3daa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:40.051319Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f89007dd-4c3e85ef-5d577f2b-e67c3daa_0] Write session: try to update token
2025-10-09T21:51:40.051348Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f89007dd-4c3e85ef-5d577f2b-e67c3daa_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:40.059527Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:40.059547Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:40.059549Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:40.059551Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:40.059560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343016820744937:11825]
2025-10-09T21:51:40.059564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343016820744937:11825]
2025-10-09T21:51:40.059567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343016820744937:11825]
2025-10-09T21:51:40.059612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:40.059617Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:40.059623Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:40.059625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.059627Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:40.059627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:40.059767Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:40.066377Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2210 sessionId: src_id|f89007dd-4c3e85ef-5d577f2b-e67c3daa_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:40.068768Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:40.110804Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343017182593661:8652]
2025-10-09T21:51:40.111050Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 2209, pipe: [1:7559343017182593661:8652]
2025-10-09T21:51:40.208812Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:40.209371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.209391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.209435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182593661:8652], now have 1 active actors on pipe
2025-10-09T21:51:40.209574Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:40.209604Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:40.209617Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343017182593661:8652]
2025-10-09T21:51:40.209620Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:40.209636Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:40.209683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2210 reason: error at write: Disconnected sessionId: src_id|f89007dd-4c3e85ef-5d577f2b-e67c3daa_0
2025-10-09T21:51:40.209844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2210 sessionId: src_id|f89007dd-4c3e85ef-5d577f2b-e67c3daa_0 is DEAD
2025-10-09T21:51:40.210048Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f89007dd-4c3e85ef-5d577f2b-e67c3daa_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:40.210061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f89007dd-4c3e85ef-5d577f2b-e67c3daa_0] Write session will restart in 0.009638s
2025-10-09T21:51:40.210088Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f89007dd-4c3e85ef-5d577f2b-e67c3daa_0] Write session: Do CDS request
2025-10-09T21:51:40.210094Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f89007dd-4c3e85ef-5d577f2b-e67c3daa_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:40.220558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f89007dd-4c3e85ef-5d577f2b-e67c3daa_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:40.220633Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:40.220651Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2211
2025-10-09T21:51:40.220783Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2211 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:40.220816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2211 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:40.220827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2211 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:40.220830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:40.221191Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2211 sessionId:  describe result for acl check
2025-10-09T21:51:40.221224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:40.221229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:40.221230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:40.221235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343017182593883:8571] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:40.221653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343017182593883:8571] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:40.224429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343017182593883:8571] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:40.227109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343017182593883:8571] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:40.227120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343017182593883:8571] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:40.227123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343017182593883:8571] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:40.227165Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2211 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:40.227853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:40.229704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:40.229924Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:40.229934Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:40.229947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:40.230003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:40.230057Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343016820745271:11839]
2025-10-09T21:51:40.230582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:40.230815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:40.230845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:40.231063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:40.231095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.231100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.231101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.231102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.231102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.231103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.231104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.231104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.231105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.231108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.231115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.231117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.231119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.231121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.231123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.231125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.231127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.231129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.231134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.231139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.231143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.231146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.231148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.231152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.231154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.231157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.231160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.231163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.231165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.231169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.231171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.231175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.231177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.231180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.231183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.231186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.231201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:40.237395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:40.238450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:40.238468Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:40.238470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:40.238474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:40.238480Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2210 [2:7559343016820745271:11839]
2025-10-09T21:51:40.238497Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:40.238515Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:40.238517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:40.238520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:40.238522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:40.238539Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:40.238548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:40.238556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:40.238610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.238618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.238621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.238623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:40.238689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:40.238697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.238709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.238721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.238738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:40.238763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182594100:8571], now have 1 active actors on pipe
2025-10-09T21:51:40.238772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182593664:2432], now have 1 active actors on pipe
2025-10-09T21:51:40.238781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343016820745266:11839]
2025-10-09T21:51:40.238783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343016820745266:11839]
2025-10-09T21:51:40.238785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343016820745266:11839]
2025-10-09T21:51:40.238798Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:40.238812Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:40.238822Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:40.238824Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:40.238825Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:40.238827Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:40.238827Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:40.238814Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2210
2025-10-09T21:51:40.238849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343016820745266:11839]
2025-10-09T21:51:40.238819Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2210
2025-10-09T21:51:40.238855Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:40.238857Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:40.238880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.238890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.238926Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|92867a60-1fcb5a6-c47853a1-26a023ea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:40.238938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:40.238942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:40.238944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:40.238954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:40.238967Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:40.238994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:40.239131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.239139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.239152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:40.239239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2211 partition: 0 MaxSeqNo: 25 sessionId: src_id|92867a60-1fcb5a6-c47853a1-26a023ea_0
2025-10-09T21:51:40.239547Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f89007dd-4c3e85ef-5d577f2b-e67c3daa_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|92867a60-1fcb5a6-c47853a1-26a023ea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:40.241849Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92867a60-1fcb5a6-c47853a1-26a023ea_0] Write session: try to update token
2025-10-09T21:51:40.241863Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92867a60-1fcb5a6-c47853a1-26a023ea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:40.253852Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:40.253986Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:40.253993Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:40.253994Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:40.254001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343016820745266:11839]
2025-10-09T21:51:40.254004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343016820745266:11839]
2025-10-09T21:51:40.254007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343016820745266:11839]
2025-10-09T21:51:40.254059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:40.254068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.254070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:40.254072Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:40.254078Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:40.254081Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:40.254098Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2211 sessionId: src_id|92867a60-1fcb5a6-c47853a1-26a023ea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:40.254182Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:40.256402Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:40.309997Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343017182594119:8652]
2025-10-09T21:51:40.310263Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 2210, pipe: [1:7559343017182594119:8652]
2025-10-09T21:51:40.393202Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:40.393878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.393894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.393940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182594119:8652], now have 1 active actors on pipe
2025-10-09T21:51:40.394108Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:40.394179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:40.394184Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343017182594119:8652]
2025-10-09T21:51:40.394199Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:40.394202Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:40.394249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2211 reason: error at write: Disconnected sessionId: src_id|92867a60-1fcb5a6-c47853a1-26a023ea_0
2025-10-09T21:51:40.394384Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2211 sessionId: src_id|92867a60-1fcb5a6-c47853a1-26a023ea_0 is DEAD
2025-10-09T21:51:40.394522Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92867a60-1fcb5a6-c47853a1-26a023ea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:40.394533Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92867a60-1fcb5a6-c47853a1-26a023ea_0] Write session will restart in 0.007043s
2025-10-09T21:51:40.394568Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92867a60-1fcb5a6-c47853a1-26a023ea_0] Write session: Do CDS request
2025-10-09T21:51:40.394574Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92867a60-1fcb5a6-c47853a1-26a023ea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:40.402516Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92867a60-1fcb5a6-c47853a1-26a023ea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:40.402578Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:40.402596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2212
2025-10-09T21:51:40.402727Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2212 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:40.402755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2212 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:40.402764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2212 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:40.402768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:40.403214Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2212 sessionId:  describe result for acl check
2025-10-09T21:51:40.403253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:40.403259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:40.403261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:40.403268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343017182594241:8566] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:40.403819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343017182594241:8566] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:40.406309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343017182594241:8566] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:40.409067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343017182594241:8566] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:40.409080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343017182594241:8566] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:40.409086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343017182594241:8566] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:40.409106Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2212 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:40.411787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:40.413782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:40.413999Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:40.414011Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:40.414025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:40.414126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:40.414167Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343016820745561:11841]
2025-10-09T21:51:40.414743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:40.415049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:40.415093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:40.415388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:40.415450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.415456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.415457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.415458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.415460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.415461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.415462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.415463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.415465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.415470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.415481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.415484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.415489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.415493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.415497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.415501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.415504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.415508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.415520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.415532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.415543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.415552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.415556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.415561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.415564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.415569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.415573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.415578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.415582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.415587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.415591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.415601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.415606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.415615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.415623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.415632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.415663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:40.421932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:40.422980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:40.423001Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:40.423004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:40.423009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:40.423016Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2211 [2:7559343016820745561:11841]
2025-10-09T21:51:40.423029Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:40.423049Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:40.423057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:40.423060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:40.423063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:40.423081Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:40.423093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:40.423099Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:40.423163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.423170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.423172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.423175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:40.423392Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2211
2025-10-09T21:51:40.423392Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2211
2025-10-09T21:51:40.423571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:40.423586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.423600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.423618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.423641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:40.423672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182594517:8566], now have 1 active actors on pipe
2025-10-09T21:51:40.423680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182594122:2432], now have 1 active actors on pipe
2025-10-09T21:51:40.423699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.423708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.423734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343016820745556:11841]
2025-10-09T21:51:40.423740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343016820745556:11841]
2025-10-09T21:51:40.423742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343016820745556:11841]
2025-10-09T21:51:40.423756Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:40.423775Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:40.423776Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:40.423780Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:40.423782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:40.423794Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:40.423797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:40.423814Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bbdc08e0-e7bd2091-8d4435ae-1b3f96bf_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:40.423826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:40.423833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:40.423836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:40.423845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:40.423859Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:40.423891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343016820745556:11841]
2025-10-09T21:51:40.423900Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:40.423909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:40.423912Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:40.424053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.424061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.424073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:40.424211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2212 partition: 0 MaxSeqNo: 25 sessionId: src_id|bbdc08e0-e7bd2091-8d4435ae-1b3f96bf_0
2025-10-09T21:51:40.424580Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92867a60-1fcb5a6-c47853a1-26a023ea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bbdc08e0-e7bd2091-8d4435ae-1b3f96bf_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:40.426879Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bbdc08e0-e7bd2091-8d4435ae-1b3f96bf_0] Write session: try to update token
2025-10-09T21:51:40.426896Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bbdc08e0-e7bd2091-8d4435ae-1b3f96bf_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:40.440987Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:40.441003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:40.441013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343016820745556:11841]
2025-10-09T21:51:40.441065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:40.441069Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:40.441122Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2212 sessionId: src_id|bbdc08e0-e7bd2091-8d4435ae-1b3f96bf_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:40.441245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:40.441408Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:40.441555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:40.441702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:40.441900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:40.441942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:40.443412Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:40.443617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.443634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.443671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:40.443752Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:40.448093Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:40.448108Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:40.448110Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:40.448112Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:40.448120Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343016820745556:11841]
2025-10-09T21:51:40.448124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343016820745556:11841]
2025-10-09T21:51:40.448128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343016820745556:11841]
2025-10-09T21:51:40.448163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:40.448176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.448183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:40.448180Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:40.448204Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:40.448208Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:40.448313Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:40.494328Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343017182594570:8652]
2025-10-09T21:51:40.494598Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 2211, pipe: [1:7559343017182594570:8652]
2025-10-09T21:51:40.589850Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:40.596819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.596841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.596995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:40.597061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:40.597126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:40.597190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:40.597234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:40.597242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:40.597308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:40.597386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:40.597449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:40.597509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:40.597551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:40.597558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:40.597620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:40.597677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:40.597732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:40.597790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:40.597833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:40.597839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:40.597901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:40.597957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:40.598013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:40.598092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:40.598132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:40.598139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:40.598195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:40.598252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:40.598310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:40.598434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:40.598478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:40.598483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:40.598602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:40.598663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:40.598717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:40.598779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:40.598829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:40.598839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:40.598901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:40.598953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:40.599006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:40.599060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:40.599102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:40.599112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:40.599175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:40.599225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:40.599283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:40.599339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:40.599433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:40.599438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:40.599489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:40.599539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:40.599597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:40.599657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:40.599701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:40.599708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:40.599767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:40.599818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:40.599877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:40.599935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:40.599978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:40.599986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:40.600032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182594570:8652], now have 1 active actors on pipe
2025-10-09T21:51:40.600309Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343017182594570:8652]
2025-10-09T21:51:40.600341Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:40.600605Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:40.600616Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:40.600646Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2212 reason: error at write: Disconnected sessionId: src_id|bbdc08e0-e7bd2091-8d4435ae-1b3f96bf_0
2025-10-09T21:51:40.600836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2212 sessionId: src_id|bbdc08e0-e7bd2091-8d4435ae-1b3f96bf_0 is DEAD
2025-10-09T21:51:40.600849Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:40.600984Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbdc08e0-e7bd2091-8d4435ae-1b3f96bf_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:40.600998Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbdc08e0-e7bd2091-8d4435ae-1b3f96bf_0] Write session will restart in 0.005648s
2025-10-09T21:51:40.601022Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbdc08e0-e7bd2091-8d4435ae-1b3f96bf_0] Write session: Do CDS request
2025-10-09T21:51:40.601028Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbdc08e0-e7bd2091-8d4435ae-1b3f96bf_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:40.607446Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bbdc08e0-e7bd2091-8d4435ae-1b3f96bf_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:40.607518Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:40.607530Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2213
2025-10-09T21:51:40.607640Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2213 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:40.607668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2213 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:40.607678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2213 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:40.607683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:40.608057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2213 sessionId:  describe result for acl check
2025-10-09T21:51:40.608085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:40.608090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:40.608091Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:40.608095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343017182594638:8561] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:40.608517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343017182594638:8561] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:40.611076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343017182594638:8561] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:40.613234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343017182594638:8561] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:40.613249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343017182594638:8561] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:40.613252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343017182594638:8561] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:40.613259Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2213 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:40.617591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:40.619593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:40.619777Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:40.619788Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:40.619800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:40.619855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:40.619894Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343016820745854:11843]
2025-10-09T21:51:40.620466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:40.620733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:40.620767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:40.621013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:40.621050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.621054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.621055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.621056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.621056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.621057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.621058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.621058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.621059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.621062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.621069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.621071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.621073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.621075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.621077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.621079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.621081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.621083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.621088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.621095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.621101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.621106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.621110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.621116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.621120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.621125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.621129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.621134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.621138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.621143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.621148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.621153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.621157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.621161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.621164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.621167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.621182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:40.626999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:40.628018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:40.628031Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:40.628034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:40.628038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:40.628044Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2212 [2:7559343016820745854:11843]
2025-10-09T21:51:40.628058Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:40.628075Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:40.628078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:40.628081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:40.628088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:40.628103Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:40.628115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:40.628122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:40.628171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.628178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.628179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.628182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:40.628245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:40.628253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.628264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.628273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.628287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:40.628308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182594903:8561], now have 1 active actors on pipe
2025-10-09T21:51:40.628321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343016820745849:11843]
2025-10-09T21:51:40.628323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343016820745849:11843]
2025-10-09T21:51:40.628325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343016820745849:11843]
2025-10-09T21:51:40.628337Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:40.628346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182594573:2432], now have 1 active actors on pipe
2025-10-09T21:51:40.628354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:40.628355Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:40.628358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:40.628360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:40.628366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:40.628369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:40.628360Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2212
2025-10-09T21:51:40.628391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343016820745849:11843]
2025-10-09T21:51:40.628399Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:40.628367Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2212
2025-10-09T21:51:40.628401Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:40.628414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.628423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.628469Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4c315f9b-93e6931d-19b8cd2b-1d44a8d1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:40.628481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:40.628487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:40.628493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:40.628503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:40.628517Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:40.628536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:40.628648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.628658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.628673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:40.628758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2213 partition: 0 MaxSeqNo: 25 sessionId: src_id|4c315f9b-93e6931d-19b8cd2b-1d44a8d1_0
2025-10-09T21:51:40.629106Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbdc08e0-e7bd2091-8d4435ae-1b3f96bf_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4c315f9b-93e6931d-19b8cd2b-1d44a8d1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:40.631401Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4c315f9b-93e6931d-19b8cd2b-1d44a8d1_0] Write session: try to update token
2025-10-09T21:51:40.631418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4c315f9b-93e6931d-19b8cd2b-1d44a8d1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:40.641751Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:40.641771Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:40.641773Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:40.641776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:40.641785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343016820745849:11843]
2025-10-09T21:51:40.641789Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343016820745849:11843]
2025-10-09T21:51:40.641794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343016820745849:11843]
2025-10-09T21:51:40.641835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:40.641844Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:40.641847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.641849Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:40.641850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:40.641853Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:40.641982Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:40.644769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2213 sessionId: src_id|4c315f9b-93e6931d-19b8cd2b-1d44a8d1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:40.647020Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:40.700659Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343017182595023:8652]
2025-10-09T21:51:40.700924Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 2212, pipe: [1:7559343017182595023:8652]
2025-10-09T21:51:40.783145Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:40.783855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.783868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.783906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182595023:8652], now have 1 active actors on pipe
2025-10-09T21:51:40.784076Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:40.784120Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:40.784135Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:40.784142Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:40.784191Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343017182595023:8652]
2025-10-09T21:51:40.784195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2213 reason: error at write: Disconnected sessionId: src_id|4c315f9b-93e6931d-19b8cd2b-1d44a8d1_0
2025-10-09T21:51:40.784373Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2213 sessionId: src_id|4c315f9b-93e6931d-19b8cd2b-1d44a8d1_0 is DEAD
2025-10-09T21:51:40.784510Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c315f9b-93e6931d-19b8cd2b-1d44a8d1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:40.784522Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c315f9b-93e6931d-19b8cd2b-1d44a8d1_0] Write session will restart in 0.008317s
2025-10-09T21:51:40.784546Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c315f9b-93e6931d-19b8cd2b-1d44a8d1_0] Write session: Do CDS request
2025-10-09T21:51:40.784553Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c315f9b-93e6931d-19b8cd2b-1d44a8d1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:40.794487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4c315f9b-93e6931d-19b8cd2b-1d44a8d1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:40.794571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:40.794592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2214
2025-10-09T21:51:40.794701Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2214 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:40.794728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2214 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:40.794736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2214 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:40.794738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:40.795182Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2214 sessionId:  describe result for acl check
2025-10-09T21:51:40.795225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:40.795232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:40.795234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:40.795242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343017182595206:8549] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:40.795828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343017182595206:8549] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:40.797967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343017182595206:8549] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:40.800637Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343017182595206:8549] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:40.800653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343017182595206:8549] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:40.800657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343017182595206:8549] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:40.800666Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2214 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:40.802377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:40.804097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:40.804287Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:40.804298Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:40.804309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:40.804356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:40.804406Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343016820746151:11845]
2025-10-09T21:51:40.805029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:40.805285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:40.805321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:40.805648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:40.805695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.805705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.805706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.805707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.805709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.805710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.805711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.805713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.805714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.805719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.805730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.805735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.805739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.805743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.805747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.805751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.805754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.805758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.805767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.805774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:40.805781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.805786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:40.805791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.805797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:40.805801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.805806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:40.805810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.805815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:40.805824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.805829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:40.805836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.805845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.805850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.805854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.805864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.805869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.805898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:40.812282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:40.813371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:40.813390Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:40.813393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:40.813398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:40.813405Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2213 [2:7559343016820746151:11845]
2025-10-09T21:51:40.813420Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:40.813442Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:40.813451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:40.813454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:40.813456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:40.813476Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:40.813483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:40.813494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:40.813558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.813566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.813569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.813572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:40.813651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:40.813660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.813671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:40.813682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:40.813701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:40.813728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182595461:8549], now have 1 active actors on pipe
2025-10-09T21:51:40.813736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182595025:2432], now have 1 active actors on pipe
2025-10-09T21:51:40.813744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343016820746146:11845]
2025-10-09T21:51:40.813747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343016820746146:11845]
2025-10-09T21:51:40.813749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343016820746146:11845]
2025-10-09T21:51:40.813764Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:40.813783Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:40.813784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:40.813795Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:40.813798Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:40.813800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:40.813804Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:40.813843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343016820746146:11845]
2025-10-09T21:51:40.813851Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:40.813805Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2213
2025-10-09T21:51:40.813824Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2213
2025-10-09T21:51:40.813853Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:40.813869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.813875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.813929Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4bc49946-5f96c5cb-b277f9ab-bb4feb97_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:40.813943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:40.813947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:40.813950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:40.813961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:40.813971Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:40.813999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:40.814153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.814161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.814178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:40.814255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2214 partition: 0 MaxSeqNo: 25 sessionId: src_id|4bc49946-5f96c5cb-b277f9ab-bb4feb97_0
2025-10-09T21:51:40.814588Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c315f9b-93e6931d-19b8cd2b-1d44a8d1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4bc49946-5f96c5cb-b277f9ab-bb4feb97_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:40.817183Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4bc49946-5f96c5cb-b277f9ab-bb4feb97_0] Write session: try to update token
2025-10-09T21:51:40.817202Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4bc49946-5f96c5cb-b277f9ab-bb4feb97_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:40.828472Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:40.828489Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:40.828492Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:40.828493Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:40.828501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343016820746146:11845]
2025-10-09T21:51:40.828505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343016820746146:11845]
2025-10-09T21:51:40.828508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343016820746146:11845]
2025-10-09T21:51:40.828542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:40.828548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:40.828549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:40.828552Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:40.828558Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:40.828564Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:40.828681Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:40.830846Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2214 sessionId: src_id|4bc49946-5f96c5cb-b277f9ab-bb4feb97_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:40.833088Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:40.884469Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343017182595476:8652]
2025-10-09T21:51:40.884749Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 2213, pipe: [1:7559343017182595476:8652]
2025-10-09T21:51:40.978751Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:40.978991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:40.979008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:40.979053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182595476:8652], now have 1 active actors on pipe
2025-10-09T21:51:40.979245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:40.979261Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:40.979277Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:40.979328Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343017182595476:8652]
2025-10-09T21:51:40.979365Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2214 reason: error at write: Disconnected sessionId: src_id|4bc49946-5f96c5cb-b277f9ab-bb4feb97_0
2025-10-09T21:51:40.979549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2214 sessionId: src_id|4bc49946-5f96c5cb-b277f9ab-bb4feb97_0 is DEAD
2025-10-09T21:51:40.979692Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4bc49946-5f96c5cb-b277f9ab-bb4feb97_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:40.979705Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4bc49946-5f96c5cb-b277f9ab-bb4feb97_0] Write session will restart in 0.006011s
2025-10-09T21:51:40.979728Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4bc49946-5f96c5cb-b277f9ab-bb4feb97_0] Write session: Do CDS request
2025-10-09T21:51:40.979739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4bc49946-5f96c5cb-b277f9ab-bb4feb97_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:40.979975Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:40.986489Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4bc49946-5f96c5cb-b277f9ab-bb4feb97_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:40.986589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:40.986604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2215
2025-10-09T21:51:40.986734Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2215 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:40.986778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2215 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:40.986791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2215 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:40.986796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:40.987165Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2215 sessionId:  describe result for acl check
2025-10-09T21:51:40.987198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:40.987204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:40.987206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:40.987212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343017182595519:8544] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:40.987699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343017182595519:8544] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:40.989883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343017182595519:8544] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:40.992300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343017182595519:8544] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:40.992313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343017182595519:8544] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:40.992316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343017182595519:8544] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:40.992323Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2215 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:40.997408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:40.999580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:40.999777Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:40.999787Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:40.999799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:40.999850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:40.999887Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343016820746445:11849]
2025-10-09T21:51:41.000466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:41.000720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:41.000776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:41.001068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:41.001123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.001129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.001130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.001132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.001133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.001134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.001136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.001137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.001138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.001143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.001155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.001160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.001164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.001168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.001172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.001176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.001182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.001186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.001198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.001206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.001217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.001228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.001237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.001244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.001252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.001258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.001268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.001273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.001277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.001283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.001288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.001293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.001298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.001303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.001307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.001315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.001357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:41.007486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:41.008666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:41.008686Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:41.008689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:41.008694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:41.008701Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2214 [2:7559343016820746445:11849]
2025-10-09T21:51:41.008716Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:41.008734Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:41.008742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:41.008745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:41.008748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:41.008765Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:41.008776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:41.008783Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:41.008841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.008848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.008850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.008853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:41.008922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:41.008930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.008943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.008956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.008978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:41.009002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182595482:2432], now have 1 active actors on pipe
2025-10-09T21:51:41.009010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343017182595779:8544], now have 1 active actors on pipe
2025-10-09T21:51:41.009018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343016820746440:11849]
2025-10-09T21:51:41.009025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343016820746440:11849]
2025-10-09T21:51:41.009028Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343016820746440:11849]
2025-10-09T21:51:41.009041Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:41.009055Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:41.009059Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:41.009059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:41.009061Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:41.009074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:41.009081Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:41.009070Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2214
2025-10-09T21:51:41.009110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343016820746440:11849]
2025-10-09T21:51:41.009085Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2214
2025-10-09T21:51:41.009121Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:41.009128Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:41.009139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.009144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.009201Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3d1063e8-49a13ed1-f113d033-e3541cda_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:41.009214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:41.009219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:41.009226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:41.009238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:41.009250Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:41.009278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:41.009454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.009463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.009479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:41.009561Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2215 partition: 0 MaxSeqNo: 25 sessionId: src_id|3d1063e8-49a13ed1-f113d033-e3541cda_0
2025-10-09T21:51:41.009926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4bc49946-5f96c5cb-b277f9ab-bb4feb97_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3d1063e8-49a13ed1-f113d033-e3541cda_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:41.012388Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d1063e8-49a13ed1-f113d033-e3541cda_0] Write session: try to update token
2025-10-09T21:51:41.012407Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d1063e8-49a13ed1-f113d033-e3541cda_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:41.023262Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:41.023281Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:41.023284Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:41.023286Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:41.023296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343016820746440:11849]
2025-10-09T21:51:41.023300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343016820746440:11849]
2025-10-09T21:51:41.023304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343016820746440:11849]
2025-10-09T21:51:41.023351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:41.023359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.023359Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:41.023362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:41.023365Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:41.023369Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:41.023487Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:41.026062Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2215 sessionId: src_id|3d1063e8-49a13ed1-f113d033-e3541cda_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:41.028603Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:41.079554Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343021477563227:8652]
2025-10-09T21:51:41.079829Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 2214, pipe: [1:7559343021477563227:8652]
2025-10-09T21:51:41.171525Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:41.171965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.171978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.172017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343021477563227:8652], now have 1 active actors on pipe
2025-10-09T21:51:41.172179Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:41.172261Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343021477563227:8652]
2025-10-09T21:51:41.172263Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:41.172297Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:41.172380Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2215 reason: status is not ok: tablet will be restarted right now sessionId: src_id|3d1063e8-49a13ed1-f113d033-e3541cda_0
2025-10-09T21:51:41.172575Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2215 sessionId: src_id|3d1063e8-49a13ed1-f113d033-e3541cda_0 is DEAD
2025-10-09T21:51:41.172696Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d1063e8-49a13ed1-f113d033-e3541cda_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:41.172708Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d1063e8-49a13ed1-f113d033-e3541cda_0] Write session will restart in 0.008312s
2025-10-09T21:51:41.172736Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d1063e8-49a13ed1-f113d033-e3541cda_0] Write session: Do CDS request
2025-10-09T21:51:41.172743Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d1063e8-49a13ed1-f113d033-e3541cda_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:41.182578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3d1063e8-49a13ed1-f113d033-e3541cda_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:41.182681Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:41.182700Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2216
2025-10-09T21:51:41.182812Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2216 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:41.182848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2216 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:41.182859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2216 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:41.182864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:41.183336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2216 sessionId:  describe result for acl check
2025-10-09T21:51:41.183373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:41.183379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:41.183381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:41.183387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343021477563419:8532] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:41.183835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343021477563419:8532] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:41.186072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343021477563419:8532] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:41.189614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343021477563419:8532] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:41.189628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343021477563419:8532] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:41.189631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343021477563419:8532] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:41.189639Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2216 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:41.192489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:41.194634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:41.194821Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:41.194833Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:41.194845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:41.194900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:41.194943Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343021115714075:11863]
2025-10-09T21:51:41.195552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:41.195900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:41.195949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:41.196269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:41.196321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.196327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.196328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.196330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.196331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.196332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.196333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.196335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.196336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.196341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.196353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.196358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.196362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.196366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.196370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.196374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.196378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.196382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.196390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.196398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.196405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.196411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.196421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.196426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.196433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.196437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.196442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.196451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.196455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.196460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.196468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.196473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.196482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.196487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.196494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.196501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.196531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:41.203126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:41.204300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:41.204317Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:41.204320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:41.204325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:41.204333Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2215 [2:7559343021115714075:11863]
2025-10-09T21:51:41.204350Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:41.204369Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:41.204372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:41.204376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:41.204379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:41.204397Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:41.204409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:41.204417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:41.204482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.204489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.204491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.204495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:41.204571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:41.204579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.204591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.204604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.204626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:41.204653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343021477563255:2432], now have 1 active actors on pipe
2025-10-09T21:51:41.204661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343021477563661:8532], now have 1 active actors on pipe
2025-10-09T21:51:41.204669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343021115714070:11863]
2025-10-09T21:51:41.204672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343021115714070:11863]
2025-10-09T21:51:41.204674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343021115714070:11863]
2025-10-09T21:51:41.204689Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:41.204709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:41.204711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:41.204714Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:41.204716Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:41.204730Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:41.204733Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:41.204718Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2215
2025-10-09T21:51:41.204718Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2215
2025-10-09T21:51:41.204768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343021115714070:11863]
2025-10-09T21:51:41.204776Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:41.204783Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:41.204794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.204803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.204856Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2c79b1c9-78e6bda-96064ab1-bab018cc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:41.204868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:41.204872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:41.204875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:41.204887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:41.204901Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:41.204925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:41.205121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.205129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.205146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:41.205241Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2216 partition: 0 MaxSeqNo: 25 sessionId: src_id|2c79b1c9-78e6bda-96064ab1-bab018cc_0
2025-10-09T21:51:41.205644Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3d1063e8-49a13ed1-f113d033-e3541cda_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2c79b1c9-78e6bda-96064ab1-bab018cc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:41.208032Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c79b1c9-78e6bda-96064ab1-bab018cc_0] Write session: try to update token
2025-10-09T21:51:41.208054Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c79b1c9-78e6bda-96064ab1-bab018cc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:41.219919Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:41.219943Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:41.219946Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:41.219948Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:41.219957Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343021115714070:11863]
2025-10-09T21:51:41.219963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343021115714070:11863]
2025-10-09T21:51:41.219967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343021115714070:11863]
2025-10-09T21:51:41.220011Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:41.220013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:41.220020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.220023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:41.220025Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:41.220029Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:41.220171Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:41.222001Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2216 sessionId: src_id|2c79b1c9-78e6bda-96064ab1-bab018cc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:41.224441Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:41.272570Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343021477563708:8652]
2025-10-09T21:51:41.272812Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 2215, pipe: [1:7559343021477563708:8652]
2025-10-09T21:51:41.365493Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:41.365745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.365757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.365795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343021477563708:8652], now have 1 active actors on pipe
2025-10-09T21:51:41.366003Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:41.366007Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343021477563708:8652]
2025-10-09T21:51:41.366018Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:41.366039Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:41.366085Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2216 reason: error at write: Disconnected sessionId: src_id|2c79b1c9-78e6bda-96064ab1-bab018cc_0
2025-10-09T21:51:41.366134Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:41.366250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2216 sessionId: src_id|2c79b1c9-78e6bda-96064ab1-bab018cc_0 is DEAD
2025-10-09T21:51:41.366396Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c79b1c9-78e6bda-96064ab1-bab018cc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:41.366407Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c79b1c9-78e6bda-96064ab1-bab018cc_0] Write session will restart in 0.005074s
2025-10-09T21:51:41.366428Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c79b1c9-78e6bda-96064ab1-bab018cc_0] Write session: Do CDS request
2025-10-09T21:51:41.366434Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c79b1c9-78e6bda-96064ab1-bab018cc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:41.372515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c79b1c9-78e6bda-96064ab1-bab018cc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:41.372563Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:41.372582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2217
2025-10-09T21:51:41.372711Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2217 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:41.372756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2217 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:41.372767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2217 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:41.372770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:41.373210Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2217 sessionId:  describe result for acl check
2025-10-09T21:51:41.373246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:41.373253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:41.373255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:41.373261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343021477563764:8519] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:41.373856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343021477563764:8519] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:41.376080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343021477563764:8519] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:41.378187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343021477563764:8519] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:41.378201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343021477563764:8519] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:41.378204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343021477563764:8519] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:41.378210Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2217 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:41.384773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:41.386640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:41.386810Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:41.386820Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:41.386831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:41.386879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:41.386914Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343021115714371:11865]
2025-10-09T21:51:41.387480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:41.387784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:41.387828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:41.388191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:41.388252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.388258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.388260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.388261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.388262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.388263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.388264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.388266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.388267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.388272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.388283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.388287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.388291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.388295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.388299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.388303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.388307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.388311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.388317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.388326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.388331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.388337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.388341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.388345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.388349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.388357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.388361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.388365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.388369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.388374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.388379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.388384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.388388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.388398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.388403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.388408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.388438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:41.394564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:41.395924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:41.395939Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:41.395940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:41.395944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:41.395951Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2216 [2:7559343021115714371:11865]
2025-10-09T21:51:41.395964Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:41.395981Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:41.395983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:41.395987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:41.395989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:41.396005Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:41.396017Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:41.396026Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:41.396076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.396083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.396085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.396088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:41.396152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:41.396162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.396175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.396186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.396205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:41.396225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343021477563956:8519], now have 1 active actors on pipe
2025-10-09T21:51:41.396234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343021477563716:2432], now have 1 active actors on pipe
2025-10-09T21:51:41.396241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343021115714366:11865]
2025-10-09T21:51:41.396243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343021115714366:11865]
2025-10-09T21:51:41.396246Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343021115714366:11865]
2025-10-09T21:51:41.396259Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:41.396274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:41.396290Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:41.396293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:41.396296Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:41.396301Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:41.396303Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:41.396319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343021115714366:11865]
2025-10-09T21:51:41.396323Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:41.396325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:41.396309Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2216
2025-10-09T21:51:41.396312Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2216
2025-10-09T21:51:41.396359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.396369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.396403Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fbf24e2-805c257a-473efdc4-d80cc3f6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:41.396415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:41.396418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:41.396421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:41.396432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:41.396445Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:41.396462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:41.396630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.396640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.396654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:41.396747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2217 partition: 0 MaxSeqNo: 25 sessionId: src_id|fbf24e2-805c257a-473efdc4-d80cc3f6_0
2025-10-09T21:51:41.397061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c79b1c9-78e6bda-96064ab1-bab018cc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fbf24e2-805c257a-473efdc4-d80cc3f6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:41.399438Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fbf24e2-805c257a-473efdc4-d80cc3f6_0] Write session: try to update token
2025-10-09T21:51:41.399459Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fbf24e2-805c257a-473efdc4-d80cc3f6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:41.409966Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:41.409986Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:41.409989Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:41.409991Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:41.410001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343021115714366:11865]
2025-10-09T21:51:41.410005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343021115714366:11865]
2025-10-09T21:51:41.410010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343021115714366:11865]
2025-10-09T21:51:41.410063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:41.410063Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:41.410069Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:41.410070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.410073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:41.410077Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:41.410214Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:41.412244Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2217 sessionId: src_id|fbf24e2-805c257a-473efdc4-d80cc3f6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:41.414635Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:41.466108Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343021477564169:8652]
2025-10-09T21:51:41.466349Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 2216, pipe: [1:7559343021477564169:8652]
2025-10-09T21:51:41.555103Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:41.555356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.555375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.555427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343021477564169:8652], now have 1 active actors on pipe
2025-10-09T21:51:41.555601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:41.555616Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:41.555671Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2217 reason: error at write: Disconnected sessionId: src_id|fbf24e2-805c257a-473efdc4-d80cc3f6_0
2025-10-09T21:51:41.555842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2217 sessionId: src_id|fbf24e2-805c257a-473efdc4-d80cc3f6_0 is DEAD
2025-10-09T21:51:41.555977Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:41.555991Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343021477564169:8652]
2025-10-09T21:51:41.556014Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fbf24e2-805c257a-473efdc4-d80cc3f6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:41.556027Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fbf24e2-805c257a-473efdc4-d80cc3f6_0] Write session will restart in 0.006770s
2025-10-09T21:51:41.556046Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fbf24e2-805c257a-473efdc4-d80cc3f6_0] Write session: Do CDS request
2025-10-09T21:51:41.556053Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fbf24e2-805c257a-473efdc4-d80cc3f6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:41.556703Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:41.563562Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fbf24e2-805c257a-473efdc4-d80cc3f6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:41.563644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:41.563659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2218
2025-10-09T21:51:41.563771Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2218 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:41.563804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2218 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:41.563811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2218 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:41.563814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:41.564212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2218 sessionId:  describe result for acl check
2025-10-09T21:51:41.564249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:41.564255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:41.564257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:41.564264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343021477564287:8515] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:41.564792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343021477564287:8515] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:41.566840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343021477564287:8515] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:41.569181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343021477564287:8515] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:41.569194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343021477564287:8515] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:41.569198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343021477564287:8515] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:41.569205Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2218 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:41.573723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:41.574110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:41.574276Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:41.574286Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:41.574298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:41.574343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:41.574383Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343021115714669:11867]
2025-10-09T21:51:41.574950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:41.575224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:41.575268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:41.575585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:41.575634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.575640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.575642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.575643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.575644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.575645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.575647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.575648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.575649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.575654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.575665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.575670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.575674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.575678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.575683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.575687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.575691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.575695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.575702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.575710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.575722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.575728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.575737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.575742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.575752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.575758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.575763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.575770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.575774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.575784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.575791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.575801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.575806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.575813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.575817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.575822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.575856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:41.581964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:41.583002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:41.583017Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:41.583019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:41.583023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:41.583029Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2217 [2:7559343021115714669:11867]
2025-10-09T21:51:41.583041Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:41.583071Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:41.583073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:41.583077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:41.583079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:41.583095Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:41.583108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:41.583119Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:41.583174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.583181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.583183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.583186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:41.583336Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2217
2025-10-09T21:51:41.583469Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2217
2025-10-09T21:51:41.583612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:41.583623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.583634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.583647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.583668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:41.583694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343021477564561:8515], now have 1 active actors on pipe
2025-10-09T21:51:41.583702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343021477564172:2432], now have 1 active actors on pipe
2025-10-09T21:51:41.583719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.583727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.583748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343021115714664:11867]
2025-10-09T21:51:41.583755Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343021115714664:11867]
2025-10-09T21:51:41.583758Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343021115714664:11867]
2025-10-09T21:51:41.583770Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:41.583784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:41.583786Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:41.583788Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:41.583797Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:41.583800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:41.583803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:41.583821Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|28bff147-7b1088fa-771a4248-6cb19232_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:41.583828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:41.583834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:41.583836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:41.583847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:41.583859Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:41.583889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343021115714664:11867]
2025-10-09T21:51:41.583900Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:41.583910Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:41.583912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:41.584049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.584058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.584073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:41.584152Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2218 partition: 0 MaxSeqNo: 25 sessionId: src_id|28bff147-7b1088fa-771a4248-6cb19232_0
2025-10-09T21:51:41.584530Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fbf24e2-805c257a-473efdc4-d80cc3f6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|28bff147-7b1088fa-771a4248-6cb19232_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:41.586813Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28bff147-7b1088fa-771a4248-6cb19232_0] Write session: try to update token
2025-10-09T21:51:41.586827Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28bff147-7b1088fa-771a4248-6cb19232_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:41.598799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2218 sessionId: src_id|28bff147-7b1088fa-771a4248-6cb19232_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:41.600756Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:41.600771Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:41.600777Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343021115714664:11867]
2025-10-09T21:51:41.600821Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:41.600828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:41.600990Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:41.601060Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:41.601123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:41.601250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:41.601393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:41.601575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:41.601636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.601650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.601670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:41.601711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:41.601791Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:41.607658Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:41.607680Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:41.607684Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:41.607686Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:41.607695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343021115714664:11867]
2025-10-09T21:51:41.607699Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343021115714664:11867]
2025-10-09T21:51:41.607703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343021115714664:11867]
2025-10-09T21:51:41.607744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:41.607749Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:41.607752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.607755Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:41.607756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:41.607759Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:41.607892Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:41.656064Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343021477564626:8652]
2025-10-09T21:51:41.656328Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 2217, pipe: [1:7559343021477564626:8652]
2025-10-09T21:51:41.749765Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:41.756863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.756886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.757020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.757073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.757127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.757182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.757222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.757234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.757290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.757356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.757413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.757476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.757522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.757529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.757580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.757627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.757681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.757729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.757767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.757773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.757828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.757895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.757950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.758002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.758058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.758068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.758132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.758183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.758236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.758286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.758321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.758327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.758379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.758430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.758484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.758537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.758591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.758599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.758649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.758699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.758752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.758802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.758891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.758897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.758952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.759011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.759066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.759119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.759157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.759164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.759225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.759273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.759323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.759379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.759506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.759513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.759565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.759614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.759664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.759714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.759751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.759756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.759794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343021477564626:8652], now have 1 active actors on pipe
2025-10-09T21:51:41.759980Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:41.759992Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:41.759994Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:41.760020Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2218 reason: error at write: Disconnected sessionId: src_id|28bff147-7b1088fa-771a4248-6cb19232_0
2025-10-09T21:51:41.760034Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343021477564626:8652]
2025-10-09T21:51:41.760171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2218 sessionId: src_id|28bff147-7b1088fa-771a4248-6cb19232_0 is DEAD
2025-10-09T21:51:41.760312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28bff147-7b1088fa-771a4248-6cb19232_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:41.760323Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28bff147-7b1088fa-771a4248-6cb19232_0] Write session will restart in 0.008299s
2025-10-09T21:51:41.760342Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28bff147-7b1088fa-771a4248-6cb19232_0] Write session: Do CDS request
2025-10-09T21:51:41.760347Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28bff147-7b1088fa-771a4248-6cb19232_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:41.760543Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:41.769475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28bff147-7b1088fa-771a4248-6cb19232_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:41.769496Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:41.769509Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2219
2025-10-09T21:51:41.769645Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2219 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:41.769689Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2219 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:41.769699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2219 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:41.769703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:41.770112Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2219 sessionId:  describe result for acl check
2025-10-09T21:51:41.770148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:41.770149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:41.770151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:41.770156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343021477564789:8505] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:41.770693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343021477564789:8505] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:41.773143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343021477564789:8505] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:41.776202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343021477564789:8505] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:41.776216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343021477564789:8505] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:41.776219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343021477564789:8505] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:41.776226Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2219 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:41.778761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:41.780586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:41.780757Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:41.780768Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:41.780779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:41.780845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:41.780884Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343021115714966:11869]
2025-10-09T21:51:41.781241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:41.781625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:41.781659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:41.781879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:41.781911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.781915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.781915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.781916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.781917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.781917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.781918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.781919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.781919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.781922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.781929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.781931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.781933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.781935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.781937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.781939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.781941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.781943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.781947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.781952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.781956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.781960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.781962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.781965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.781967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.781970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.781972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.781975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.781980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.781982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.781985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.781988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.781990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.781993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.781995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.781998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.782012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:41.788605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:41.789785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:41.789797Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:41.789798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:41.789801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:41.789805Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2218 [2:7559343021115714966:11869]
2025-10-09T21:51:41.789814Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:41.789827Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:41.789829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:41.789831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:41.789832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:41.789843Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:41.789847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:41.789852Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:41.789889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.789895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.789896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.789898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:41.789944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:41.789950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.789956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.789965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.789974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:41.789992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343021477564629:2432], now have 1 active actors on pipe
2025-10-09T21:51:41.789997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343021477565043:8505], now have 1 active actors on pipe
2025-10-09T21:51:41.790004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343021115714961:11869]
2025-10-09T21:51:41.790005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343021115714961:11869]
2025-10-09T21:51:41.790007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343021115714961:11869]
2025-10-09T21:51:41.790017Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:41.790042Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:41.790051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:41.790056Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:41.790062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:41.790066Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:41.790068Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:41.790073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343021115714961:11869]
2025-10-09T21:51:41.790075Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:41.790077Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:41.790089Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2218
2025-10-09T21:51:41.790127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.790134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.790118Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2218
2025-10-09T21:51:41.790155Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c38caeb6-cb92560d-a71a62a4-eea05a0f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:41.790163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:41.790165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:41.790170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:41.790177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:41.790184Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:41.790198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:41.790315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.790322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.790335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:41.790416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2219 partition: 0 MaxSeqNo: 25 sessionId: src_id|c38caeb6-cb92560d-a71a62a4-eea05a0f_0
2025-10-09T21:51:41.790698Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28bff147-7b1088fa-771a4248-6cb19232_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c38caeb6-cb92560d-a71a62a4-eea05a0f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:41.793066Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c38caeb6-cb92560d-a71a62a4-eea05a0f_0] Write session: try to update token
2025-10-09T21:51:41.793083Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c38caeb6-cb92560d-a71a62a4-eea05a0f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:41.802299Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:41.802316Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:41.802325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343021115714961:11869]
2025-10-09T21:51:41.802362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:41.802368Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:41.802518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:41.802646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:41.802777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:41.802894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:41.803055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:41.803098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:41.806514Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2219 sessionId: src_id|c38caeb6-cb92560d-a71a62a4-eea05a0f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:41.808626Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:41.808809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.808827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.808859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:41.808940Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:41.815974Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:41.815989Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:41.815990Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:41.815992Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:41.816000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343021115714961:11869]
2025-10-09T21:51:41.816004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343021115714961:11869]
2025-10-09T21:51:41.816007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343021115714961:11869]
2025-10-09T21:51:41.816040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:41.816046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.816047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:41.816060Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:41.816069Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:41.816074Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:41.816208Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:41.860362Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343021477565088:8652]
2025-10-09T21:51:41.860626Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 2218, pipe: [1:7559343021477565088:8652]
2025-10-09T21:51:41.958953Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:41.966593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:41.966618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:41.966772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.966831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.966891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.966953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.966995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.967005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.967071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.967130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.967189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.967251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.967291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.967300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.967355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.967410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.967466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.967523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.967632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.967639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.967697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.967758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.967817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.967875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.967913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.967921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.967982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.968036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.968086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.968138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.968173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.968182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.968242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.968298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.968353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.968412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.968455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.968465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.968522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.968571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.968625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.968675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.968712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.968717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.968777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.968826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.968875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.968923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.968959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.968964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.969018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.969064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.969111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.969161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.969286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.969293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.969355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:41.969404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:41.969453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:41.969505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:41.969545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:41.969550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:41.969597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343021477565088:8652], now have 1 active actors on pipe
2025-10-09T21:51:41.969834Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343021477565088:8652]
2025-10-09T21:51:41.969975Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:41.970042Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:41.970055Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:41.970083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2219 reason: error at write: Disconnected sessionId: src_id|c38caeb6-cb92560d-a71a62a4-eea05a0f_0
2025-10-09T21:51:41.970282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2219 sessionId: src_id|c38caeb6-cb92560d-a71a62a4-eea05a0f_0 is DEAD
2025-10-09T21:51:41.970418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c38caeb6-cb92560d-a71a62a4-eea05a0f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:41.970429Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c38caeb6-cb92560d-a71a62a4-eea05a0f_0] Write session will restart in 0.006473s
2025-10-09T21:51:41.970454Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c38caeb6-cb92560d-a71a62a4-eea05a0f_0] Write session: Do CDS request
2025-10-09T21:51:41.970460Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c38caeb6-cb92560d-a71a62a4-eea05a0f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:41.970462Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:41.978450Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c38caeb6-cb92560d-a71a62a4-eea05a0f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:41.978506Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:41.978521Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2220
2025-10-09T21:51:41.978636Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2220 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:41.978674Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2220 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:41.978689Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2220 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:41.978697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:41.979090Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2220 sessionId:  describe result for acl check
2025-10-09T21:51:41.979129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:41.979136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:41.979138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:41.979144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343021477565161:8503] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:41.979672Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343021477565161:8503] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:41.981861Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343021477565161:8503] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:41.984803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343021477565161:8503] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:41.984818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343021477565161:8503] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:41.984822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343021477565161:8503] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:41.984830Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2220 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:41.990930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:41.993653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:41.993845Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:41.993857Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:41.993867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:41.994044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:41.994106Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343021115715270:11873]
2025-10-09T21:51:41.994595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:41.994894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:41.994935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:41.995225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:41.995292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.995299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.995300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.995302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.995303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.995304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.995306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.995307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.995309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.995315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.995326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.995330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.995334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.995343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.995348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.995357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.995361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.995369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.995376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.995385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:41.995393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.995396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:41.995402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.995406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:41.995410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.995417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:41.995420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.995424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:41.995431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.995434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:41.995437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.995444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:41.995446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.995449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:41.995456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.995459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:41.995476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:42.001943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:42.003003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:42.003020Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:42.003023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:42.003028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:42.003034Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2219 [2:7559343021115715270:11873]
2025-10-09T21:51:42.003051Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:42.003069Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:42.003077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:42.003080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:42.003082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:42.003100Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:42.003111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:42.003122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:42.003184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.003191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.003193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.003196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:42.003266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:42.003275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.003287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.003300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.003320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:42.003345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343021477565393:8503], now have 1 active actors on pipe
2025-10-09T21:51:42.003356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343021477565095:2432], now have 1 active actors on pipe
2025-10-09T21:51:42.003364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343021115715265:11873]
2025-10-09T21:51:42.003366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343021115715265:11873]
2025-10-09T21:51:42.003368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343021115715265:11873]
2025-10-09T21:51:42.003385Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:42.003404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:42.003423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:42.003426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:42.003427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:42.003433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:42.003438Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:42.003461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343021115715265:11873]
2025-10-09T21:51:42.003470Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:42.003436Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2219
2025-10-09T21:51:42.003436Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2219
2025-10-09T21:51:42.003483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:42.003506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:42.003511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:42.003563Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9963a831-bee62720-dd7ecbd0-42d169cb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:42.003576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:42.003581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:42.003584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:42.003594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:42.003604Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:42.003622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:42.003810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:42.003818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:42.003833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:42.003928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2220 partition: 0 MaxSeqNo: 25 sessionId: src_id|9963a831-bee62720-dd7ecbd0-42d169cb_0
2025-10-09T21:51:42.004300Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c38caeb6-cb92560d-a71a62a4-eea05a0f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9963a831-bee62720-dd7ecbd0-42d169cb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:42.006657Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9963a831-bee62720-dd7ecbd0-42d169cb_0] Write session: try to update token
2025-10-09T21:51:42.006673Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9963a831-bee62720-dd7ecbd0-42d169cb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:42.018523Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:42.018544Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:42.018545Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:42.018547Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:42.018555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343021115715265:11873]
2025-10-09T21:51:42.018558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343021115715265:11873]
2025-10-09T21:51:42.018561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343021115715265:11873]
2025-10-09T21:51:42.018598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:42.018605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.018607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:42.018613Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:42.018621Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:42.018625Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:42.018735Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:42.019577Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2220 sessionId: src_id|9963a831-bee62720-dd7ecbd0-42d169cb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:42.021997Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:42.069724Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343025772532845:8652]
2025-10-09T21:51:42.070086Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 2219, pipe: [1:7559343025772532845:8652]
2025-10-09T21:51:42.185294Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:42.185719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:42.185733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:42.185768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343025772532845:8652], now have 1 active actors on pipe
2025-10-09T21:51:42.185915Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:42.185988Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343025772532845:8652]
2025-10-09T21:51:42.186013Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:42.186322Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:42.186396Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2220 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9963a831-bee62720-dd7ecbd0-42d169cb_0
2025-10-09T21:51:42.186569Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2220 sessionId: src_id|9963a831-bee62720-dd7ecbd0-42d169cb_0 is DEAD
2025-10-09T21:51:42.186710Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9963a831-bee62720-dd7ecbd0-42d169cb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:42.186720Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9963a831-bee62720-dd7ecbd0-42d169cb_0] Write session will restart in 0.007514s
2025-10-09T21:51:42.186741Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9963a831-bee62720-dd7ecbd0-42d169cb_0] Write session: Do CDS request
2025-10-09T21:51:42.186749Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9963a831-bee62720-dd7ecbd0-42d169cb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:42.195517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9963a831-bee62720-dd7ecbd0-42d169cb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:42.195587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:42.195600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2221
2025-10-09T21:51:42.195743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2221 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:42.195767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2221 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:42.195771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2221 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:42.195774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:42.196164Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2221 sessionId:  describe result for acl check
2025-10-09T21:51:42.196203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:42.196206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:42.196207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:42.196213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343025772532974:8487] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:42.196736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343025772532974:8487] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:42.198613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343025772532974:8487] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:42.201472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343025772532974:8487] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:42.201488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343025772532974:8487] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:42.201491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343025772532974:8487] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:42.201501Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2221 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:42.209289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:42.209880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:42.210065Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:42.210077Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:42.210091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:42.210140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:42.210174Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343025410682904:11887]
2025-10-09T21:51:42.210748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:42.212053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:42.212106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:42.212431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:42.212486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:42.212491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:42.212492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:42.212493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:42.212494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:42.212494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:42.212495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.212496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.212496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.212500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:42.212511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:42.212515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:42.212519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:42.212522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:42.212526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:42.212529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.212532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.212535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.212543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:42.212552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:42.212558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:42.212564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:42.212568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:42.212573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:42.212578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:42.212583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:42.212588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:42.212597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:42.212602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:42.212608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:42.212614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.212619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.212623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.212628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.212633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.212637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.212662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:42.219500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:42.220781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:42.220798Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:42.220802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:42.220806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:42.220815Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2220 [2:7559343025410682904:11887]
2025-10-09T21:51:42.220832Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:42.220859Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:42.220867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:42.220870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:42.220872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:42.220892Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:42.220903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:42.220912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:42.220985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.220992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.220995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.220998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:42.221078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:42.221086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.221099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.221119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.221137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:42.221159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343025772532873:2432], now have 1 active actors on pipe
2025-10-09T21:51:42.221168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343025772533158:8487], now have 1 active actors on pipe
2025-10-09T21:51:42.221176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343025410682899:11887]
2025-10-09T21:51:42.221183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343025410682899:11887]
2025-10-09T21:51:42.221185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343025410682899:11887]
2025-10-09T21:51:42.221198Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:42.221213Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:42.221218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:42.221226Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:42.221228Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:42.221234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:42.221237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:42.221279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343025410682899:11887]
2025-10-09T21:51:42.221287Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:42.221298Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:42.221268Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2220
2025-10-09T21:51:42.221328Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2220
2025-10-09T21:51:42.221330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:42.221354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:42.221400Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6e0fc4e1-b6c19ba6-b7712b7-74b41e9b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:42.221414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:42.221418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:42.221421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:42.221434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:42.221449Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:42.221473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:42.221690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:42.221702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:42.221720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:42.221818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2221 partition: 0 MaxSeqNo: 25 sessionId: src_id|6e0fc4e1-b6c19ba6-b7712b7-74b41e9b_0
2025-10-09T21:51:42.222199Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9963a831-bee62720-dd7ecbd0-42d169cb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6e0fc4e1-b6c19ba6-b7712b7-74b41e9b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:42.224593Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e0fc4e1-b6c19ba6-b7712b7-74b41e9b_0] Write session: try to update token
2025-10-09T21:51:42.224613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e0fc4e1-b6c19ba6-b7712b7-74b41e9b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:42.236243Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:42.236262Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:42.236265Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:42.236268Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:42.236277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343025410682899:11887]
2025-10-09T21:51:42.236282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343025410682899:11887]
2025-10-09T21:51:42.236286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343025410682899:11887]
2025-10-09T21:51:42.236333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:42.236334Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:42.236340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:42.236343Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:42.236346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.236348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:42.236481Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:42.238286Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2221 sessionId: src_id|6e0fc4e1-b6c19ba6-b7712b7-74b41e9b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:42.240701Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:42.286319Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343025772533330:8652]
2025-10-09T21:51:42.286546Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 2220, pipe: [1:7559343025772533330:8652]
2025-10-09T21:51:42.383121Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:42.383350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:42.383363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:42.383411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343025772533330:8652], now have 1 active actors on pipe
2025-10-09T21:51:42.383620Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:42.383634Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:42.383646Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343025772533330:8652]
2025-10-09T21:51:42.383674Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:42.383746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2221 reason: status is not ok: tablet will be restarted right now sessionId: src_id|6e0fc4e1-b6c19ba6-b7712b7-74b41e9b_0
2025-10-09T21:51:42.383921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2221 sessionId: src_id|6e0fc4e1-b6c19ba6-b7712b7-74b41e9b_0 is DEAD
2025-10-09T21:51:42.384060Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e0fc4e1-b6c19ba6-b7712b7-74b41e9b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:42.384071Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e0fc4e1-b6c19ba6-b7712b7-74b41e9b_0] Write session will restart in 0.009850s
2025-10-09T21:51:42.384091Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e0fc4e1-b6c19ba6-b7712b7-74b41e9b_0] Write session: Do CDS request
2025-10-09T21:51:42.384096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e0fc4e1-b6c19ba6-b7712b7-74b41e9b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:42.395475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6e0fc4e1-b6c19ba6-b7712b7-74b41e9b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:42.395553Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:42.395570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2222
2025-10-09T21:51:42.395679Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2222 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:42.395713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2222 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:42.395725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2222 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:42.395734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:42.396137Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2222 sessionId:  describe result for acl check
2025-10-09T21:51:42.396176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:42.396182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:42.396185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:42.396190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343025772533550:8480] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:42.396734Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343025772533550:8480] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:42.399411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343025772533550:8480] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:42.402420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343025772533550:8480] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:42.402435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343025772533550:8480] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:42.402438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343025772533550:8480] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:42.402448Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2222 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:42.404190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:42.406344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:42.406571Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:42.406583Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:42.406597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:42.406656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:42.406699Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343025410683205:11889]
2025-10-09T21:51:42.407267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:42.407621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:42.407673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:42.407904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:42.407946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:42.407952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:42.407953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:42.407954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:42.407955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:42.407957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:42.407958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.407959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.407961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.407966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:42.407976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:42.407978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:42.407980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:42.407982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:42.407984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:42.407986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.407988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.407990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.407995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:42.408000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:42.408005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:42.408009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:42.408011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:42.408014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:42.408017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:42.408020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:42.408024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:42.408027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:42.408030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:42.408034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:42.408036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.408039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.408042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.408045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.408048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.408051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.408071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:42.414702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:42.415866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:42.415883Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:42.415885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:42.415890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:42.415898Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2221 [2:7559343025410683205:11889]
2025-10-09T21:51:42.415914Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:42.415935Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:42.415943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:42.415946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:42.415949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:42.415968Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:42.415978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:42.415986Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:42.416048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.416056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.416058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.416061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:42.416139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:42.416148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.416162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.416181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.416201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:42.416226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343025772533782:8480], now have 1 active actors on pipe
2025-10-09T21:51:42.416236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343025772533338:2432], now have 1 active actors on pipe
2025-10-09T21:51:42.416245Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343025410683200:11889]
2025-10-09T21:51:42.416250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343025410683200:11889]
2025-10-09T21:51:42.416252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343025410683200:11889]
2025-10-09T21:51:42.416266Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:42.416287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:42.416289Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:42.416292Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:42.416295Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:42.416307Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:42.416311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:42.416312Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2221
2025-10-09T21:51:42.416350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343025410683200:11889]
2025-10-09T21:51:42.416366Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2221
2025-10-09T21:51:42.416362Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:42.416370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:42.416407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:42.416417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:42.416450Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|16a9e27c-14687043-e18c3377-51c07ad_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:42.416466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:42.416470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:42.416477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:42.416488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:42.416498Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:42.416512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:42.416692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:42.416702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:42.416717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:42.416818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2222 partition: 0 MaxSeqNo: 25 sessionId: src_id|16a9e27c-14687043-e18c3377-51c07ad_0
2025-10-09T21:51:42.417251Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6e0fc4e1-b6c19ba6-b7712b7-74b41e9b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|16a9e27c-14687043-e18c3377-51c07ad_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:42.419592Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16a9e27c-14687043-e18c3377-51c07ad_0] Write session: try to update token
2025-10-09T21:51:42.419610Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16a9e27c-14687043-e18c3377-51c07ad_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:42.431058Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:42.431077Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:42.431079Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:42.431080Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:42.431091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343025410683200:11889]
2025-10-09T21:51:42.431095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343025410683200:11889]
2025-10-09T21:51:42.431097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343025410683200:11889]
2025-10-09T21:51:42.431132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:42.431137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.431139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:42.431151Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:42.431157Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:42.431161Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:42.431262Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:42.433553Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2222 sessionId: src_id|16a9e27c-14687043-e18c3377-51c07ad_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:42.436214Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:42.483957Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343025772533796:8652]
2025-10-09T21:51:42.484207Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 2221, pipe: [1:7559343025772533796:8652]
2025-10-09T21:51:42.576979Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:42.577643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:42.577657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:42.577697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343025772533796:8652], now have 1 active actors on pipe
2025-10-09T21:51:42.577876Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:42.577948Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343025772533796:8652]
2025-10-09T21:51:42.577958Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:42.577963Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:42.577985Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:42.578022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2222 reason: error at write: Disconnected sessionId: src_id|16a9e27c-14687043-e18c3377-51c07ad_0
2025-10-09T21:51:42.578204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2222 sessionId: src_id|16a9e27c-14687043-e18c3377-51c07ad_0 is DEAD
2025-10-09T21:51:42.578351Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16a9e27c-14687043-e18c3377-51c07ad_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:42.578363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16a9e27c-14687043-e18c3377-51c07ad_0] Write session will restart in 0.009948s
2025-10-09T21:51:42.578383Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16a9e27c-14687043-e18c3377-51c07ad_0] Write session: Do CDS request
2025-10-09T21:51:42.578388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16a9e27c-14687043-e18c3377-51c07ad_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:42.589488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|16a9e27c-14687043-e18c3377-51c07ad_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:42.589560Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:42.589575Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2223
2025-10-09T21:51:42.589695Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2223 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:42.589736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2223 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:42.589747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2223 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:42.589752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:42.590197Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2223 sessionId:  describe result for acl check
2025-10-09T21:51:42.590232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:42.590241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:42.590243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:42.590248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343025772533995:8476] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:42.590752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343025772533995:8476] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:42.593545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343025772533995:8476] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:42.596435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343025772533995:8476] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:42.596449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343025772533995:8476] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:42.596453Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343025772533995:8476] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:42.596460Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2223 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:42.599162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:42.601553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:42.601751Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:42.601761Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:42.601775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:42.601836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:42.601880Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343025410683508:11891]
2025-10-09T21:51:42.602488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:42.602774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:42.602813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:42.603088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:42.603136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:42.603142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:42.603143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:42.603143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:42.603144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:42.603145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:42.603145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.603146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.603147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.603150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:42.603158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:42.603160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:42.603162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:42.603164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:42.603166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:42.603168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.603171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.603173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.603178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:42.603183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:42.603188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:42.603191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:42.603193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:42.603197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:42.603199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:42.603202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:42.603205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:42.603209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:42.603213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:42.603218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:42.603222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.603227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.603231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.603237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.603242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.603247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.603281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:42.610187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:42.611408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:42.611422Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:42.611423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:42.611427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:42.611432Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2222 [2:7559343025410683508:11891]
2025-10-09T21:51:42.611442Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:42.611458Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:42.611460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:42.611463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:42.611466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:42.611483Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:42.611489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:42.611497Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:42.611566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.611574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.611576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.611579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:42.611652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:42.611660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.611674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.611686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.611703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:42.611727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343025772533799:2432], now have 1 active actors on pipe
2025-10-09T21:51:42.611734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343025772534209:8476], now have 1 active actors on pipe
2025-10-09T21:51:42.611741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343025410683503:11891]
2025-10-09T21:51:42.611743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343025410683503:11891]
2025-10-09T21:51:42.611745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343025410683503:11891]
2025-10-09T21:51:42.611726Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2222
2025-10-09T21:51:42.611755Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:42.611748Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2222
2025-10-09T21:51:42.611762Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:42.611765Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:42.611766Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:42.611774Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:42.611790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:42.611795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:42.611813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:42.611822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:42.611859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343025410683503:11891]
2025-10-09T21:51:42.611869Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:42.611877Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:42.611883Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bb25794b-c2db4624-38c2af4f-f2e2a91f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:42.611889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:42.611892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:42.611894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:42.611901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:42.611910Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:42.611935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:42.612053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:42.612062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:42.612075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:42.612153Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2223 partition: 0 MaxSeqNo: 25 sessionId: src_id|bb25794b-c2db4624-38c2af4f-f2e2a91f_0
2025-10-09T21:51:42.612517Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|16a9e27c-14687043-e18c3377-51c07ad_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bb25794b-c2db4624-38c2af4f-f2e2a91f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:42.615002Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb25794b-c2db4624-38c2af4f-f2e2a91f_0] Write session: try to update token
2025-10-09T21:51:42.615023Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb25794b-c2db4624-38c2af4f-f2e2a91f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:42.625038Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:42.625056Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:42.625066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343025410683503:11891]
2025-10-09T21:51:42.625112Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:42.625116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:42.625289Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:42.625437Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:42.625562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:42.625694Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:42.625867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:42.625903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:42.628703Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2223 sessionId: src_id|bb25794b-c2db4624-38c2af4f-f2e2a91f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:42.631149Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:42.631342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:42.631363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:42.631403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:42.631480Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:42.639267Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:42.639284Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:42.639287Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:42.639290Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:42.639301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343025410683503:11891]
2025-10-09T21:51:42.639305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343025410683503:11891]
2025-10-09T21:51:42.639309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343025410683503:11891]
2025-10-09T21:51:42.639351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:42.639352Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:42.639358Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:42.639359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.639361Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:42.639363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:42.639507Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:42.677966Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343025772534259:8652]
2025-10-09T21:51:42.678221Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 2222, pipe: [1:7559343025772534259:8652]
2025-10-09T21:51:42.783734Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:42.790874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:42.790899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:42.791045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:42.791098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:42.791155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:42.791213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:42.791257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:42.791268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:42.791338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:42.791394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:42.791452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:42.791518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:42.791561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:42.791566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:42.791629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:42.791684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:42.791756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:42.791823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:42.791865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:42.791873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:42.791934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:42.791990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:42.792051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:42.792110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:42.792149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:42.792155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:42.792210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:42.792261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:42.792315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:42.792379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:42.792425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:42.792430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:42.792484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:42.792537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:42.792590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:42.792643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:42.792746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:42.792754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:42.792812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:42.792864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:42.792917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:42.792970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:42.793009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:42.793014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:42.793070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:42.793120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:42.793171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:42.793225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:42.793265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:42.793270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:42.793324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:42.793389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:42.793450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:42.793505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:42.793545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:42.793550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:42.793603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:42.793665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:42.793718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:42.793775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:42.793815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:42.793821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:42.793868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343025772534259:8652], now have 1 active actors on pipe
2025-10-09T21:51:42.794129Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343025772534259:8652]
2025-10-09T21:51:42.794131Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:42.794170Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:42.794173Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:42.794214Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2223 reason: error at write: Disconnected sessionId: src_id|bb25794b-c2db4624-38c2af4f-f2e2a91f_0
2025-10-09T21:51:42.794410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2223 sessionId: src_id|bb25794b-c2db4624-38c2af4f-f2e2a91f_0 is DEAD
2025-10-09T21:51:42.794572Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb25794b-c2db4624-38c2af4f-f2e2a91f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:42.794584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb25794b-c2db4624-38c2af4f-f2e2a91f_0] Write session will restart in 0.005748s
2025-10-09T21:51:42.794608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb25794b-c2db4624-38c2af4f-f2e2a91f_0] Write session: Do CDS request
2025-10-09T21:51:42.794614Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb25794b-c2db4624-38c2af4f-f2e2a91f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:42.794736Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:42.801556Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb25794b-c2db4624-38c2af4f-f2e2a91f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:42.801649Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:42.801665Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2224
2025-10-09T21:51:42.801803Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2224 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:42.801835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2224 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:42.801847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2224 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:42.801851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:42.802237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2224 sessionId:  describe result for acl check
2025-10-09T21:51:42.802269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:42.802276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:42.802278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:42.802284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343025772534315:8462] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:42.802758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343025772534315:8462] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:42.805540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343025772534315:8462] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:42.808466Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343025772534315:8462] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:42.808479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343025772534315:8462] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:42.808482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343025772534315:8462] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:42.808490Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2224 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:42.813500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:42.815631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:42.815915Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:42.815926Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:42.815938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:42.815991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:42.816025Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343025410683815:11893]
2025-10-09T21:51:42.816458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:42.816724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:42.816762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:42.817048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:42.817090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:42.817097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:42.817098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:42.817099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:42.817100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:42.817100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:42.817101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.817102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.817102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.817106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:42.817113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:42.817116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:42.817118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:42.817120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:42.817122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:42.817124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.817126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.817128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.817133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:42.817138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:42.817142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:42.817146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:42.817149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:42.817152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:42.817154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:42.817157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:42.817160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:42.817163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:42.817165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:42.817169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:42.817173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.817176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.817178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.817181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.817184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.817192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.817218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:42.823392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:42.824405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:42.824420Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:42.824422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:42.824427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:42.824432Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2223 [2:7559343025410683815:11893]
2025-10-09T21:51:42.824440Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:42.824455Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:42.824457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:42.824460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:42.824461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:42.824474Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:42.824483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:42.824490Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:42.824534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.824539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.824540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.824542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:42.824593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:42.824599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.824607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:42.824612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:42.824627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:42.824646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343025772534261:2432], now have 1 active actors on pipe
2025-10-09T21:51:42.824652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343025772534589:8462], now have 1 active actors on pipe
2025-10-09T21:51:42.824658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343025410683810:11893]
2025-10-09T21:51:42.824659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343025410683810:11893]
2025-10-09T21:51:42.824660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343025410683810:11893]
2025-10-09T21:51:42.824667Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:42.824678Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:42.824679Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:42.824682Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:42.824690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:42.824692Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:42.824697Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:42.824712Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343025410683810:11893]
2025-10-09T21:51:42.824723Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:42.824736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:42.824723Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2223
2025-10-09T21:51:42.824742Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2223
2025-10-09T21:51:42.824762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:42.824772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:42.824801Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|87000573-3a73e540-c7825814-676aed69_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:42.824814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:42.824818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:42.824822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:42.824832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:42.824841Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:42.824852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:42.824955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:42.824961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:42.824973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:42.825061Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2224 partition: 0 MaxSeqNo: 25 sessionId: src_id|87000573-3a73e540-c7825814-676aed69_0
2025-10-09T21:51:42.825411Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb25794b-c2db4624-38c2af4f-f2e2a91f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|87000573-3a73e540-c7825814-676aed69_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:42.827904Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87000573-3a73e540-c7825814-676aed69_0] Write session: try to update token
2025-10-09T21:51:42.827921Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87000573-3a73e540-c7825814-676aed69_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:42.840752Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2224 sessionId: src_id|87000573-3a73e540-c7825814-676aed69_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:42.841776Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:42.841793Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:42.841801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343025410683810:11893]
2025-10-09T21:51:42.841845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:42.841918Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:42.842011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:42.842177Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:42.842316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:42.842449Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:42.842639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:42.842674Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:42.843194Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:42.843357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:42.843369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:42.843396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:42.843478Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:42.849933Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:42.849950Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:42.849952Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:42.849953Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:42.849961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343025410683810:11893]
2025-10-09T21:51:42.849964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343025410683810:11893]
2025-10-09T21:51:42.849968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343025410683810:11893]
2025-10-09T21:51:42.850000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:42.850007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:42.850009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:42.850011Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:42.850016Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:42.850019Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:42.850148Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:42.894588Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343025772534722:8652]
2025-10-09T21:51:42.894860Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 2223, pipe: [1:7559343025772534722:8652]
2025-10-09T21:51:42.998909Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:43.006335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.006359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.006512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:43.006571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:43.006633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:43.006695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:43.006739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:43.006750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:43.006824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:43.006883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:43.006949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:43.007013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:43.007060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:43.007068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:43.007125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:43.007179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:43.007233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:43.007300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:43.007355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:43.007361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:43.007426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:43.007488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:43.007551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:43.007617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:43.007662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:43.007671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:43.007735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:43.007800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:43.007864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:43.007926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:43.007967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:43.007972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:43.008033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:43.008093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:43.008152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:43.008210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:43.008251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:43.008258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:43.008324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:43.008377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:43.008433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:43.008489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:43.008532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:43.008540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:43.008602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:43.008656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:43.008717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:43.008775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:43.008823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:43.008832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:43.008889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:43.008953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:43.009011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:43.009076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:43.009124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:43.009130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:43.009188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:43.009241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:43.009302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:43.009380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:43.009422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:43.009427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:43.009469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343025772534722:8652], now have 1 active actors on pipe
2025-10-09T21:51:43.009660Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:43.009687Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343025772534722:8652]
2025-10-09T21:51:43.009712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:43.009714Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:43.009742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2224 reason: error at write: Disconnected sessionId: src_id|87000573-3a73e540-c7825814-676aed69_0
2025-10-09T21:51:43.009924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2224 sessionId: src_id|87000573-3a73e540-c7825814-676aed69_0 is DEAD
2025-10-09T21:51:43.010096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87000573-3a73e540-c7825814-676aed69_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:43.010109Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87000573-3a73e540-c7825814-676aed69_0] Write session will restart in 0.009179s
2025-10-09T21:51:43.010134Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87000573-3a73e540-c7825814-676aed69_0] Write session: Do CDS request
2025-10-09T21:51:43.010140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87000573-3a73e540-c7825814-676aed69_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:43.010392Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:43.020492Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87000573-3a73e540-c7825814-676aed69_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:43.020552Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:43.020568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2225
2025-10-09T21:51:43.020698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2225 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:43.020725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2225 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:43.020734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2225 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:43.020740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:43.021090Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2225 sessionId:  describe result for acl check
2025-10-09T21:51:43.021116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:43.021120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:43.021121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:43.021126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343030067502154:8460] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:43.021651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343030067502154:8460] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:43.023927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343030067502154:8460] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:43.030223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343030067502154:8460] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:43.030243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343030067502154:8460] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:43.030248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343030067502154:8460] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:43.030259Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2225 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:43.051732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:43.054381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:43.054639Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:43.054657Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:43.054706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:43.054768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:43.054820Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343029705651424:11897]
2025-10-09T21:51:43.055548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:43.055858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:43.055904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:43.056253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:43.056321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.056330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.056331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.056333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.056334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.056335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.056337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.056338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.056339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.056345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.056355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.056359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.056363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.056367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.056371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.056374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.056378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.056382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.056390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.056397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.056403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.056408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.056413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.056418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.056422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.056427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.056433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.056438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.056444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.056449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.056455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.056460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.056465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.056470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.056476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.056489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.056519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:43.062875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:43.064756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:43.064779Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:43.064783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:43.064786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:43.064794Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2224 [2:7559343029705651424:11897]
2025-10-09T21:51:43.064815Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:43.064833Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:43.064844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:43.064848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:43.064850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:43.064867Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:43.064880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:43.064889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:43.064963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.064973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.064976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.064979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:43.065062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:43.065073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.065088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.065102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.065126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:43.065151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502204:8460], now have 1 active actors on pipe
2025-10-09T21:51:43.065161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502025:2432], now have 1 active actors on pipe
2025-10-09T21:51:43.065170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343029705651414:11897]
2025-10-09T21:51:43.065173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343029705651414:11897]
2025-10-09T21:51:43.065176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343029705651414:11897]
2025-10-09T21:51:43.065193Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:43.065187Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2224
2025-10-09T21:51:43.065210Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:43.065213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:43.065225Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:43.065227Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:43.065187Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2224
2025-10-09T21:51:43.065229Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:43.065232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:43.065281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.065293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.065309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343029705651414:11897]
2025-10-09T21:51:43.065326Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:43.065349Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:43.065373Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6ad71559-33b98547-e08cb3bd-47f16936_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:43.065387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:43.065391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:43.065394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:43.065403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:43.065419Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:43.065446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:43.065664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.065678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.065695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:43.065812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2225 partition: 0 MaxSeqNo: 25 sessionId: src_id|6ad71559-33b98547-e08cb3bd-47f16936_0
2025-10-09T21:51:43.066149Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87000573-3a73e540-c7825814-676aed69_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6ad71559-33b98547-e08cb3bd-47f16936_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:43.068488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ad71559-33b98547-e08cb3bd-47f16936_0] Write session: try to update token
2025-10-09T21:51:43.068504Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ad71559-33b98547-e08cb3bd-47f16936_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:43.079689Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:43.079718Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:43.079721Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:43.079723Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:43.079733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343029705651414:11897]
2025-10-09T21:51:43.079739Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343029705651414:11897]
2025-10-09T21:51:43.079744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343029705651414:11897]
2025-10-09T21:51:43.079791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:43.079799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.079802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:43.079801Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:43.079821Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:43.079825Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:43.079958Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:43.082973Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2225 sessionId: src_id|6ad71559-33b98547-e08cb3bd-47f16936_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:43.085325Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:43.109630Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343030067502488:8652]
2025-10-09T21:51:43.109907Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 2224, pipe: [1:7559343030067502488:8652]
2025-10-09T21:51:43.236735Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:43.237667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.237691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.237733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502488:8652], now have 1 active actors on pipe
2025-10-09T21:51:43.237992Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:43.237998Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343030067502488:8652]
2025-10-09T21:51:43.238012Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:43.238017Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:43.238045Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:43.238110Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2225 reason: error at write: Disconnected sessionId: src_id|6ad71559-33b98547-e08cb3bd-47f16936_0
2025-10-09T21:51:43.238304Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2225 sessionId: src_id|6ad71559-33b98547-e08cb3bd-47f16936_0 is DEAD
2025-10-09T21:51:43.238492Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ad71559-33b98547-e08cb3bd-47f16936_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:43.238502Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ad71559-33b98547-e08cb3bd-47f16936_0] Write session will restart in 0.009310s
2025-10-09T21:51:43.238524Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ad71559-33b98547-e08cb3bd-47f16936_0] Write session: Do CDS request
2025-10-09T21:51:43.238529Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ad71559-33b98547-e08cb3bd-47f16936_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:43.245612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:43.246615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:43.246808Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:43.246817Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:43.246826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:43.246871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:43.246904Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343029705651519:11911]
2025-10-09T21:51:43.247363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:43.247585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:43.247615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:43.247857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:43.247903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.247909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.247910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.247911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.247913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.247914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.247915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.247916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.247917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.247922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.247934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.247938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.247941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.247945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.247948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.247952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.247956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.247959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.247967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.247975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.247981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.247997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.248007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.248013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.248020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.248030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.248034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.248037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.248043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.248047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.248050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.248054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.248056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.248059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.248062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.248066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.248084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:43.248483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6ad71559-33b98547-e08cb3bd-47f16936_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:43.248568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:43.248585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2226
2025-10-09T21:51:43.248708Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2226 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:43.248739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2226 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:43.248750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2226 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:43.248758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:43.249187Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2226 sessionId:  describe result for acl check
2025-10-09T21:51:43.249225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:43.249232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:43.249234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:43.249239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343030067502545:8448] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:43.249794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343030067502545:8448] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:43.252167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343030067502545:8448] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:43.254215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:43.254621Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343030067502545:8448] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:43.254631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343030067502545:8448] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:43.254634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343030067502545:8448] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:43.254642Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2226 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:43.255193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:43.255207Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:43.255209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:43.255212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:43.255217Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2225 [2:7559343029705651519:11911]
2025-10-09T21:51:43.255226Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:43.255240Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:43.255242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:43.255244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:43.255245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:43.255261Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:43.255266Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:43.255274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:43.255320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.255327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.255328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.255330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:43.255381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:43.255386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.255394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.255400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.255410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:43.255429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502577:8448], now have 1 active actors on pipe
2025-10-09T21:51:43.255432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502521:2432], now have 1 active actors on pipe
2025-10-09T21:51:43.255437Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343029705651514:11911]
2025-10-09T21:51:43.255439Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343029705651514:11911]
2025-10-09T21:51:43.255441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343029705651514:11911]
2025-10-09T21:51:43.255449Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:43.255459Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:43.255459Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:43.255462Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:43.255463Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:43.255470Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:43.255473Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:43.255497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343029705651514:11911]
2025-10-09T21:51:43.255479Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2225
2025-10-09T21:51:43.255503Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:43.255504Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:43.255524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.255529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.255552Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2225
2025-10-09T21:51:43.255557Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|459d2735-10d49a67-4775a02d-d8e10c9d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:43.255567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:43.255570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:43.255572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:43.255579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:43.255590Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:43.255609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:43.255718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.255724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.255734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:43.255856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2226 partition: 0 MaxSeqNo: 25 sessionId: src_id|459d2735-10d49a67-4775a02d-d8e10c9d_0
2025-10-09T21:51:43.256123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6ad71559-33b98547-e08cb3bd-47f16936_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|459d2735-10d49a67-4775a02d-d8e10c9d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:43.258347Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|459d2735-10d49a67-4775a02d-d8e10c9d_0] Write session: try to update token
2025-10-09T21:51:43.258362Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|459d2735-10d49a67-4775a02d-d8e10c9d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:43.269739Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:43.269760Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:43.269762Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:43.269764Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:43.269775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343029705651514:11911]
2025-10-09T21:51:43.269780Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343029705651514:11911]
2025-10-09T21:51:43.269785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343029705651514:11911]
2025-10-09T21:51:43.269827Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:43.269831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:43.269832Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:43.269835Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:43.269839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.269841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:43.269977Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:43.270759Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2226 sessionId: src_id|459d2735-10d49a67-4775a02d-d8e10c9d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:43.273034Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:43.337970Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343030067502589:8652]
2025-10-09T21:51:43.338220Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 2225, pipe: [1:7559343030067502589:8652]
2025-10-09T21:51:43.413898Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:43.414156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.414169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.414212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502589:8652], now have 1 active actors on pipe
2025-10-09T21:51:43.414403Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:43.414478Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:43.414479Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:43.414482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:43.414528Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343030067502589:8652]
2025-10-09T21:51:43.414537Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2226 reason: error at write: Disconnected sessionId: src_id|459d2735-10d49a67-4775a02d-d8e10c9d_0
2025-10-09T21:51:43.414707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2226 sessionId: src_id|459d2735-10d49a67-4775a02d-d8e10c9d_0 is DEAD
2025-10-09T21:51:43.414854Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|459d2735-10d49a67-4775a02d-d8e10c9d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:43.414865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|459d2735-10d49a67-4775a02d-d8e10c9d_0] Write session will restart in 0.007464s
2025-10-09T21:51:43.414893Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|459d2735-10d49a67-4775a02d-d8e10c9d_0] Write session: Do CDS request
2025-10-09T21:51:43.414899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|459d2735-10d49a67-4775a02d-d8e10c9d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:43.420407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:43.421477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:43.421627Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:43.421634Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:43.421642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:43.421685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:43.421719Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343029705651575:11913]
2025-10-09T21:51:43.422174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:43.422500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:43.422534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:43.422798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:43.422856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.422862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.422863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.422865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.422866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.422868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.422869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.422870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.422872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.422877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.422888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.422895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.422897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.422899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.422903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.422905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.422907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.422909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.422914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.422919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.422923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.422926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.422929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.422932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.422934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.422937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.422940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.422943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.422946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.422949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.422952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.422955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.422959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.422962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.422965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.422968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.422985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:43.423461Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|459d2735-10d49a67-4775a02d-d8e10c9d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:43.423556Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:43.423574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2227
2025-10-09T21:51:43.423722Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2227 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:43.423753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2227 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:43.423764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2227 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:43.423768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:43.424143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2227 sessionId:  describe result for acl check
2025-10-09T21:51:43.424174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:43.424179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:43.424180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:43.424185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343030067502616:8428] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:43.424616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343030067502616:8428] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:43.426212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343030067502616:8428] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:43.428556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343030067502616:8428] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:43.428567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343030067502616:8428] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:43.428569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343030067502616:8428] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:43.428575Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2227 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:43.429208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:43.430496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:43.430513Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:43.430517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:43.430522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:43.430529Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2226 [2:7559343029705651575:11913]
2025-10-09T21:51:43.430551Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:43.430567Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:43.430577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:43.430580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:43.430582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:43.430597Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:43.430608Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:43.430614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:43.430681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.430688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.430689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.430691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:43.430755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:43.430762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.430770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.430777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.430791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:43.430809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502651:8428], now have 1 active actors on pipe
2025-10-09T21:51:43.430817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502592:2432], now have 1 active actors on pipe
2025-10-09T21:51:43.430822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343029705651570:11913]
2025-10-09T21:51:43.430823Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343029705651570:11913]
2025-10-09T21:51:43.430825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343029705651570:11913]
2025-10-09T21:51:43.430838Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:43.430854Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:43.430855Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:43.430865Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:43.430867Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:43.430870Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:43.430875Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:43.430876Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2226
2025-10-09T21:51:43.430877Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2226
2025-10-09T21:51:43.430910Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343029705651570:11913]
2025-10-09T21:51:43.430927Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:43.430933Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:43.430954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.430963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.431003Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7105902e-b7dbf691-85195bc1-484c6705_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:43.431018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:43.431021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:43.431024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:43.431032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:43.431040Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:43.431056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:43.431184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.431205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.431228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:43.431316Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2227 partition: 0 MaxSeqNo: 25 sessionId: src_id|7105902e-b7dbf691-85195bc1-484c6705_0
2025-10-09T21:51:43.431549Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|459d2735-10d49a67-4775a02d-d8e10c9d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7105902e-b7dbf691-85195bc1-484c6705_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:43.433889Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7105902e-b7dbf691-85195bc1-484c6705_0] Write session: try to update token
2025-10-09T21:51:43.433903Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7105902e-b7dbf691-85195bc1-484c6705_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:43.444748Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:43.444768Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:43.444771Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:43.444776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:43.444785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343029705651570:11913]
2025-10-09T21:51:43.444790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343029705651570:11913]
2025-10-09T21:51:43.444794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343029705651570:11913]
2025-10-09T21:51:43.444838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:43.444845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.444848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:43.444859Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:43.444878Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:43.444883Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:43.445015Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:43.446707Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2227 sessionId: src_id|7105902e-b7dbf691-85195bc1-484c6705_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:43.448970Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:43.514587Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343030067502659:8652]
2025-10-09T21:51:43.514864Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 2226, pipe: [1:7559343030067502659:8652]
2025-10-09T21:51:43.589547Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:43.589803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.589816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.589853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502659:8652], now have 1 active actors on pipe
2025-10-09T21:51:43.590057Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:43.590104Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:43.590117Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:43.590142Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:43.590179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2227 reason: error at write: Disconnected sessionId: src_id|7105902e-b7dbf691-85195bc1-484c6705_0
2025-10-09T21:51:43.590188Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343030067502659:8652]
2025-10-09T21:51:43.590320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2227 sessionId: src_id|7105902e-b7dbf691-85195bc1-484c6705_0 is DEAD
2025-10-09T21:51:43.590498Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7105902e-b7dbf691-85195bc1-484c6705_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:43.590510Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7105902e-b7dbf691-85195bc1-484c6705_0] Write session will restart in 0.005075s
2025-10-09T21:51:43.590537Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7105902e-b7dbf691-85195bc1-484c6705_0] Write session: Do CDS request
2025-10-09T21:51:43.590544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7105902e-b7dbf691-85195bc1-484c6705_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:43.596369Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7105902e-b7dbf691-85195bc1-484c6705_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:43.596479Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:43.596491Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2228
2025-10-09T21:51:43.596581Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2228 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:43.596621Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2228 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:43.596633Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2228 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:43.596636Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:43.597029Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2228 sessionId:  describe result for acl check
2025-10-09T21:51:43.597068Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:43.597076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:43.597078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:43.597084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343030067502681:8429] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:43.597131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:43.597670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343030067502681:8429] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:43.598475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:43.598725Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:43.598743Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:43.598766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:43.598884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:43.598930Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343029705651634:11915]
2025-10-09T21:51:43.599361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343030067502681:8429] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:43.599430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:43.599690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:43.599728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:43.600020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:43.600073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.600078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.600078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.600079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.600080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.600081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.600081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.600082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.600082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.600086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.600092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.600095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.600097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.600099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.600101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.600103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.600105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.600107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.600111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.600116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.600122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.600128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.600132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.600137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.600141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.600147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.600154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.600159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.600163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.600168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.600173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.600178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.600182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.600193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.600196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.600199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.600215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:43.604786Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343030067502681:8429] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:43.604799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343030067502681:8429] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:43.604802Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343030067502681:8429] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:43.604807Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2228 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:43.606145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:43.607103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:43.607119Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:43.607122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:43.607126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:43.607131Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2227 [2:7559343029705651634:11915]
2025-10-09T21:51:43.607140Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:43.607153Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:43.607155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:43.607157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:43.607159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:43.607171Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:43.607177Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:43.607183Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:43.607225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.607231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.607233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.607234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:43.607284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:43.607292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.607299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.607305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.607314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:43.607332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343029705651629:11915]
2025-10-09T21:51:43.607334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343029705651629:11915]
2025-10-09T21:51:43.607335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343029705651629:11915]
2025-10-09T21:51:43.607341Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:43.607353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:43.607361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:43.607366Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:43.607366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:43.607368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:43.607369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:43.607381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502660:2432], now have 1 active actors on pipe
2025-10-09T21:51:43.607402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502721:8429], now have 1 active actors on pipe
2025-10-09T21:51:43.607408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343029705651629:11915]
2025-10-09T21:51:43.607413Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:43.607429Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:43.607447Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2227
2025-10-09T21:51:43.607449Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2227
2025-10-09T21:51:43.607504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.607511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.607532Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ef7b35f-6faeb0c2-feb365da-c0f64e29_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:43.607542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:43.607545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:43.607547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:43.607554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:43.607563Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:43.607574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:43.607693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.607699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.607710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:43.607794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2228 partition: 0 MaxSeqNo: 25 sessionId: src_id|ef7b35f-6faeb0c2-feb365da-c0f64e29_0
2025-10-09T21:51:43.608068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7105902e-b7dbf691-85195bc1-484c6705_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ef7b35f-6faeb0c2-feb365da-c0f64e29_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:43.610496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef7b35f-6faeb0c2-feb365da-c0f64e29_0] Write session: try to update token
2025-10-09T21:51:43.610511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef7b35f-6faeb0c2-feb365da-c0f64e29_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:43.621027Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:43.621046Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:43.621048Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:43.621050Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:43.621058Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343029705651629:11915]
2025-10-09T21:51:43.621066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343029705651629:11915]
2025-10-09T21:51:43.621070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343029705651629:11915]
2025-10-09T21:51:43.621110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:43.621121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.621123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:43.621128Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:43.621135Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:43.621138Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:43.621268Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:43.623411Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2228 sessionId: src_id|ef7b35f-6faeb0c2-feb365da-c0f64e29_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:43.625579Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:43.690229Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343030067502729:8652]
2025-10-09T21:51:43.690513Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 2227, pipe: [1:7559343030067502729:8652]
2025-10-09T21:51:43.770943Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:43.771358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.771372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.771404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502729:8652], now have 1 active actors on pipe
2025-10-09T21:51:43.771603Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:43.771613Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:43.771614Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343030067502729:8652]
2025-10-09T21:51:43.771784Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:43.771795Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:43.771865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2228 reason: error at write: Disconnected sessionId: src_id|ef7b35f-6faeb0c2-feb365da-c0f64e29_0
2025-10-09T21:51:43.772020Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2228 sessionId: src_id|ef7b35f-6faeb0c2-feb365da-c0f64e29_0 is DEAD
2025-10-09T21:51:43.772151Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef7b35f-6faeb0c2-feb365da-c0f64e29_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:43.772160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef7b35f-6faeb0c2-feb365da-c0f64e29_0] Write session will restart in 0.005221s
2025-10-09T21:51:43.772183Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef7b35f-6faeb0c2-feb365da-c0f64e29_0] Write session: Do CDS request
2025-10-09T21:51:43.772189Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef7b35f-6faeb0c2-feb365da-c0f64e29_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:43.778387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:43.778455Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:43.778480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ef7b35f-6faeb0c2-feb365da-c0f64e29_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:43.778469Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2229
2025-10-09T21:51:43.778657Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2229 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:43.778692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2229 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:43.778702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2229 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:43.778706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:43.779097Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2229 sessionId:  describe result for acl check
2025-10-09T21:51:43.779132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:43.779139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:43.779141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:43.779147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343030067502753:8419] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:43.779636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343030067502753:8419] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:43.779745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:43.779933Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:43.779943Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:43.779951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:43.779999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:43.780036Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343029705651692:11917]
2025-10-09T21:51:43.780492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:43.780768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:43.780813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:43.781049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:43.781090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.781098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.781099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.781101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.781102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.781104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.781105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.781106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.781108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.781113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.781122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.781127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.781131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.781133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.781135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.781137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.781139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.781141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.781147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.781152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.781156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.781159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.781162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.781167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.781170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.781174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.781177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.781180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.781182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.781186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.781189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.781192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.781197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.781200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.781203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.781206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.781222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:43.781412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343030067502753:8419] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:43.786576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343030067502753:8419] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:43.786590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343030067502753:8419] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:43.786594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343030067502753:8419] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:43.786602Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2229 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:43.787149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:43.788370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:43.788390Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:43.788394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:43.788400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:43.788407Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2228 [2:7559343029705651692:11917]
2025-10-09T21:51:43.788426Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:43.788439Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:43.788441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:43.788443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:43.788445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:43.788456Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:43.788460Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:43.788465Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:43.788535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.788544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.788546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.788547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:43.788607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:43.788617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.788628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.788634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.788643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:43.788657Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343029705651687:11917]
2025-10-09T21:51:43.788658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343029705651687:11917]
2025-10-09T21:51:43.788660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343029705651687:11917]
2025-10-09T21:51:43.788666Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:43.788675Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:43.788678Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:43.788678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502792:8419], now have 1 active actors on pipe
2025-10-09T21:51:43.788680Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:43.788683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502731:2432], now have 1 active actors on pipe
2025-10-09T21:51:43.788687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:43.788698Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:43.788702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:43.788725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343029705651687:11917]
2025-10-09T21:51:43.788737Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:43.788753Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2228
2025-10-09T21:51:43.788769Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2228
2025-10-09T21:51:43.788771Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:43.788899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.788915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.788942Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6b7e3b0a-6467530-fac45cf3-8c26d13_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:43.788949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:43.788952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:43.788954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:43.788963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:43.788973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:43.788989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:43.789172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.789186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.789205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:43.789294Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2229 partition: 0 MaxSeqNo: 25 sessionId: src_id|6b7e3b0a-6467530-fac45cf3-8c26d13_0
2025-10-09T21:51:43.789567Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ef7b35f-6faeb0c2-feb365da-c0f64e29_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6b7e3b0a-6467530-fac45cf3-8c26d13_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:43.791928Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b7e3b0a-6467530-fac45cf3-8c26d13_0] Write session: try to update token
2025-10-09T21:51:43.791943Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b7e3b0a-6467530-fac45cf3-8c26d13_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:43.802687Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:43.802710Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:43.802714Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:43.802717Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:43.802724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343029705651687:11917]
2025-10-09T21:51:43.802729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343029705651687:11917]
2025-10-09T21:51:43.802732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343029705651687:11917]
2025-10-09T21:51:43.802765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:43.802771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.802772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:43.802771Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:43.802777Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:43.802781Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:43.802943Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:43.804213Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2229 sessionId: src_id|6b7e3b0a-6467530-fac45cf3-8c26d13_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:43.806472Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:43.871996Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343030067502800:8652]
2025-10-09T21:51:43.872284Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 2228, pipe: [1:7559343030067502800:8652]
2025-10-09T21:51:43.942799Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:43.943077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.943091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.943137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502800:8652], now have 1 active actors on pipe
2025-10-09T21:51:43.943408Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343030067502800:8652]
2025-10-09T21:51:43.943411Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:43.943566Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:43.943572Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:43.943624Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2229 reason: error at write: Disconnected sessionId: src_id|6b7e3b0a-6467530-fac45cf3-8c26d13_0
2025-10-09T21:51:43.943758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2229 sessionId: src_id|6b7e3b0a-6467530-fac45cf3-8c26d13_0 is DEAD
2025-10-09T21:51:43.943827Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:43.943919Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b7e3b0a-6467530-fac45cf3-8c26d13_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:43.943939Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b7e3b0a-6467530-fac45cf3-8c26d13_0] Write session will restart in 0.005755s
2025-10-09T21:51:43.943964Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b7e3b0a-6467530-fac45cf3-8c26d13_0] Write session: Do CDS request
2025-10-09T21:51:43.943970Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b7e3b0a-6467530-fac45cf3-8c26d13_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:43.949082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:43.950393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:43.950554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b7e3b0a-6467530-fac45cf3-8c26d13_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:43.950626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:43.950657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2230
2025-10-09T21:51:43.950613Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:43.950626Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:43.950640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:43.950710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:43.950767Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343029705651757:11921]
2025-10-09T21:51:43.951059Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2230 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:43.951100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2230 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:43.951106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2230 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:43.951112Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:43.951373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:43.951501Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2230 sessionId:  describe result for acl check
2025-10-09T21:51:43.951537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:43.951544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:43.951546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:43.951551Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343030067502830:8412] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:43.951699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:43.951745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:43.952056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343030067502830:8412] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:43.952070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:43.952119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.952126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.952127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.952128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.952130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.952131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.952132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.952134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.952135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.952140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.952151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.952155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.952159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.952163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.952167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.952171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.952175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.952179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.952185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.952193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:43.952200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.952206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:43.952210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.952215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:43.952221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.952226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:43.952230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.952235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:43.952245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.952249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:43.952254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.952259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.952268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.952273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.952281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.952286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.952313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:43.954112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343030067502830:8412] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:43.957985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343030067502830:8412] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:43.957998Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343030067502830:8412] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:43.958000Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343030067502830:8412] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:43.958007Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2230 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:43.958468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:43.959484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:43.959500Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:43.959503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:43.959507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:43.959514Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2229 [2:7559343029705651757:11921]
2025-10-09T21:51:43.959529Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:43.959546Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:43.959553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:43.959556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:43.959558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:43.959574Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:43.959583Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:43.959590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:43.959647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.959653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.959656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.959658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:43.959720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:43.959728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.959738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:43.959753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:43.959773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:43.959797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502803:2432], now have 1 active actors on pipe
2025-10-09T21:51:43.959807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343030067502866:8412], now have 1 active actors on pipe
2025-10-09T21:51:43.959818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343029705651752:11921]
2025-10-09T21:51:43.959800Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2229
2025-10-09T21:51:43.959825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343029705651752:11921]
2025-10-09T21:51:43.959812Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2229
2025-10-09T21:51:43.959826Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343029705651752:11921]
2025-10-09T21:51:43.959834Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:43.959841Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:43.959842Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:43.959844Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:43.959847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:43.959862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:43.959865Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:43.959880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.959888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.959920Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343029705651752:11921]
2025-10-09T21:51:43.959931Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:43.959935Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:43.959953Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9d32f36f-1223365a-dac7421f-f476287f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:43.959965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:43.959969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:43.959971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:43.959981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:43.959990Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:43.960018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:43.960197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:43.960204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:43.960216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:43.960292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2230 partition: 0 MaxSeqNo: 25 sessionId: src_id|9d32f36f-1223365a-dac7421f-f476287f_0
2025-10-09T21:51:43.960570Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b7e3b0a-6467530-fac45cf3-8c26d13_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9d32f36f-1223365a-dac7421f-f476287f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:43.963040Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d32f36f-1223365a-dac7421f-f476287f_0] Write session: try to update token
2025-10-09T21:51:43.963058Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d32f36f-1223365a-dac7421f-f476287f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:43.973572Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:43.973590Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:43.973592Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:43.973594Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:43.973603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343029705651752:11921]
2025-10-09T21:51:43.973608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343029705651752:11921]
2025-10-09T21:51:43.973612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343029705651752:11921]
2025-10-09T21:51:43.973654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:43.973657Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:43.973662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:43.973662Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:43.973665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:43.973665Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:43.973794Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:43.975494Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2230 sessionId: src_id|9d32f36f-1223365a-dac7421f-f476287f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:43.977840Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:44.043535Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343034362470170:8652]
2025-10-09T21:51:44.043827Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 2229, pipe: [1:7559343034362470170:8652]
2025-10-09T21:51:44.122454Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:44.122711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:44.122728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:44.122769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343034362470170:8652], now have 1 active actors on pipe
2025-10-09T21:51:44.122998Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:44.123054Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:44.123068Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:44.123070Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:44.123114Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343034362470170:8652]
2025-10-09T21:51:44.123136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2230 reason: error at write: Disconnected sessionId: src_id|9d32f36f-1223365a-dac7421f-f476287f_0
2025-10-09T21:51:44.123315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2230 sessionId: src_id|9d32f36f-1223365a-dac7421f-f476287f_0 is DEAD
2025-10-09T21:51:44.123454Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d32f36f-1223365a-dac7421f-f476287f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:44.123463Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d32f36f-1223365a-dac7421f-f476287f_0] Write session will restart in 0.005828s
2025-10-09T21:51:44.123506Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d32f36f-1223365a-dac7421f-f476287f_0] Write session: Do CDS request
2025-10-09T21:51:44.123512Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d32f36f-1223365a-dac7421f-f476287f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:44.130350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:44.130465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d32f36f-1223365a-dac7421f-f476287f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:44.130507Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:44.130523Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2231
2025-10-09T21:51:44.130663Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2231 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:44.130698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2231 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:44.130708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2231 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:44.130711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:44.131068Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2231 sessionId:  describe result for acl check
2025-10-09T21:51:44.131099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:44.131100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:44.131101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:44.131106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343034362470198:8415] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:44.131556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343034362470198:8415] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:44.132444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:44.132613Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:44.132622Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:44.132634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:44.132679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:44.132714Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343034000619113:11923]
2025-10-09T21:51:44.133133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:44.133442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:44.133491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:44.133549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343034362470198:8415] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:44.133718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:44.133754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.133759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.133760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.133760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.133761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.133762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.133763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.133764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.133766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.133770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.133776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.133779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.133781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.133783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.133785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.133788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.133791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.133793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.133797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.133803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.133807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.133811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.133814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.133817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.133822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.133826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.133831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.133836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.133840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.133844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.133852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.133862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.133874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.133879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.133889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.133892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.133908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:44.139954Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343034362470198:8415] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:44.139973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343034362470198:8415] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:44.139976Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343034362470198:8415] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:44.139984Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2231 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:44.140408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:44.141441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:44.141457Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:44.141463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:44.141466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:44.141472Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2230 [2:7559343034000619113:11923]
2025-10-09T21:51:44.141483Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:44.141500Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:44.141502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:44.141504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:44.141505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:44.141519Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:44.141527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:44.141537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:44.141587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.141592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.141593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.141595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:44.141653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:44.141659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.141666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.141672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.141682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:44.141698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343034000619108:11923]
2025-10-09T21:51:44.141703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343034000619108:11923]
2025-10-09T21:51:44.141704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343034000619108:11923]
2025-10-09T21:51:44.141712Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:44.141724Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:44.141731Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:44.141734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:44.141736Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:44.141745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343034362470237:8415], now have 1 active actors on pipe
2025-10-09T21:51:44.141747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343034362470172:2432], now have 1 active actors on pipe
2025-10-09T21:51:44.141749Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:44.141752Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:44.141777Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343034000619108:11923]
2025-10-09T21:51:44.141783Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:44.141806Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:44.141802Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2230
2025-10-09T21:51:44.141805Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2230
2025-10-09T21:51:44.141912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:44.141920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:44.141941Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|66057243-3f3916d1-4e7caa64-93a435da_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:44.141952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:44.141959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:44.141963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:44.141973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:44.141987Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:44.142002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:44.142216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:44.142224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:44.142236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:44.142341Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2231 partition: 0 MaxSeqNo: 25 sessionId: src_id|66057243-3f3916d1-4e7caa64-93a435da_0
2025-10-09T21:51:44.142729Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d32f36f-1223365a-dac7421f-f476287f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|66057243-3f3916d1-4e7caa64-93a435da_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:44.145243Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66057243-3f3916d1-4e7caa64-93a435da_0] Write session: try to update token
2025-10-09T21:51:44.145263Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66057243-3f3916d1-4e7caa64-93a435da_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:44.155923Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:44.155945Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:44.155948Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:44.155950Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:44.155960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343034000619108:11923]
2025-10-09T21:51:44.155965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343034000619108:11923]
2025-10-09T21:51:44.155969Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343034000619108:11923]
2025-10-09T21:51:44.156017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:44.156019Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:44.156026Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:44.156029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.156031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:44.156040Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:44.156153Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:44.158753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2231 sessionId: src_id|66057243-3f3916d1-4e7caa64-93a435da_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:44.161234Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:44.223222Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343034362470249:8652]
2025-10-09T21:51:44.223442Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 2230, pipe: [1:7559343034362470249:8652]
2025-10-09T21:51:44.301134Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:44.301405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:44.301421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:44.301467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343034362470249:8652], now have 1 active actors on pipe
2025-10-09T21:51:44.301708Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:44.301708Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:44.301717Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343034362470249:8652]
2025-10-09T21:51:44.301723Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:44.301739Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:44.301786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2231 reason: error at write: Disconnected sessionId: src_id|66057243-3f3916d1-4e7caa64-93a435da_0
2025-10-09T21:51:44.301934Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2231 sessionId: src_id|66057243-3f3916d1-4e7caa64-93a435da_0 is DEAD
2025-10-09T21:51:44.302100Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66057243-3f3916d1-4e7caa64-93a435da_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:44.302111Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66057243-3f3916d1-4e7caa64-93a435da_0] Write session will restart in 0.008281s
2025-10-09T21:51:44.302134Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66057243-3f3916d1-4e7caa64-93a435da_0] Write session: Do CDS request
2025-10-09T21:51:44.302138Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66057243-3f3916d1-4e7caa64-93a435da_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:44.308103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:44.309546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:44.309690Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:44.309698Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:44.309707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:44.309744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:44.309777Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343034000619214:11937]
2025-10-09T21:51:44.310205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:44.310505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:44.310550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:44.310787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:44.310828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.310833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.310834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.310834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.310837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.310837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.310838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.310838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.310839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.310842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.310849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.310852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.310854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.310856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.310858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.310860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.310862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.310865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.310870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.310875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.310879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.310883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.310885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.310888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.310892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.310895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.310897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.310901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.310904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.310907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.310910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.310913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.310917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.310920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.310922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.310926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.310943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:44.311487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66057243-3f3916d1-4e7caa64-93a435da_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:44.311606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:44.311631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2232
2025-10-09T21:51:44.311785Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2232 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:44.311822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2232 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:44.311834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2232 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:44.311838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:44.312234Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2232 sessionId:  describe result for acl check
2025-10-09T21:51:44.312269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:44.312276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:44.312277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:44.312282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343034362470308:8393] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:44.312703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343034362470308:8393] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:44.315110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343034362470308:8393] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:44.316982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:44.317552Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343034362470308:8393] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:44.317566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343034362470308:8393] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:44.317569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343034362470308:8393] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:44.317576Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2232 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:44.318073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:44.318087Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:44.318089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:44.318092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:44.318097Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2231 [2:7559343034000619214:11937]
2025-10-09T21:51:44.318106Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:44.318120Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:44.318122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:44.318125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:44.318126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:44.318138Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:44.318146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:44.318153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:44.318198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.318203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.318204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.318206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:44.318253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:44.318259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.318266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.318272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.318285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:44.318303Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343034000619209:11937]
2025-10-09T21:51:44.318307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343034000619209:11937]
2025-10-09T21:51:44.318309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343034000619209:11937]
2025-10-09T21:51:44.318314Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:44.318325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343034362470339:8393], now have 1 active actors on pipe
2025-10-09T21:51:44.318325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:44.318328Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:44.318330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:44.318333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343034362470274:2432], now have 1 active actors on pipe
2025-10-09T21:51:44.318339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:44.318348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:44.318353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:44.318378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343034000619209:11937]
2025-10-09T21:51:44.318384Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:44.318371Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2231
2025-10-09T21:51:44.318390Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:44.318376Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2231
2025-10-09T21:51:44.318418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:44.318426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:44.318446Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ec008689-e53eb11d-15540f49-4ec24ee6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:44.318453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:44.318459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:44.318462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:44.318468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:44.318478Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:44.318492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:44.318634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:44.318645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:44.318661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:44.318749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2232 partition: 0 MaxSeqNo: 25 sessionId: src_id|ec008689-e53eb11d-15540f49-4ec24ee6_0
2025-10-09T21:51:44.319080Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66057243-3f3916d1-4e7caa64-93a435da_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ec008689-e53eb11d-15540f49-4ec24ee6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:44.321408Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ec008689-e53eb11d-15540f49-4ec24ee6_0] Write session: try to update token
2025-10-09T21:51:44.321426Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ec008689-e53eb11d-15540f49-4ec24ee6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:44.333440Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2232 sessionId: src_id|ec008689-e53eb11d-15540f49-4ec24ee6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:44.334517Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:44.334533Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:44.334540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343034000619209:11937]
2025-10-09T21:51:44.334587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:44.334592Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:44.334747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:44.334889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:44.335019Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:44.335152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:44.335333Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:44.335374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:44.335849Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:44.336029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:44.336042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:44.336067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:44.336144Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:44.341886Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:44.341901Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:44.341904Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:44.341906Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:44.341913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343034000619209:11937]
2025-10-09T21:51:44.341918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343034000619209:11937]
2025-10-09T21:51:44.341921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343034000619209:11937]
2025-10-09T21:51:44.341959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:44.341965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.341966Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:44.341968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:44.341971Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:44.341974Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:44.342109Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:44.401932Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343034362470348:8652]
2025-10-09T21:51:44.402264Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 2231, pipe: [1:7559343034362470348:8652]
2025-10-09T21:51:44.501061Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:44.508015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:44.508042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:44.508192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:44.508251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:44.508304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:44.508359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:44.508398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:44.508412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:44.508482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:44.508535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:44.508589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:44.508650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:44.509000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:44.509009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:44.509063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:44.509111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:44.509161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:44.509224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:44.509263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:44.509268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:44.509357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:44.509411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:44.509472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:44.509526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:44.509566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:44.509571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:44.509630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:44.509684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:44.509748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:44.509804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:44.509840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:44.509845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:44.509898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:44.509955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:44.510009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:44.510076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:44.510112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:44.510117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:44.510168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:44.510220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:44.510275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:44.510324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:44.510365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:44.510373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:44.510431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:44.510486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:44.510539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:44.510589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:44.510624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:44.510629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:44.510681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:44.510736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:44.510790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:44.510843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:44.510884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:44.510890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:44.510942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:44.510997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:44.511051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:44.511101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:44.511138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:44.511143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:44.511184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343034362470348:8652], now have 1 active actors on pipe
2025-10-09T21:51:44.511347Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343034362470348:8652]
2025-10-09T21:51:44.511350Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:44.511387Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:44.511390Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:44.511414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2232 reason: error at write: Disconnected sessionId: src_id|ec008689-e53eb11d-15540f49-4ec24ee6_0
2025-10-09T21:51:44.511563Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2232 sessionId: src_id|ec008689-e53eb11d-15540f49-4ec24ee6_0 is DEAD
2025-10-09T21:51:44.511647Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:44.511756Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec008689-e53eb11d-15540f49-4ec24ee6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:44.511776Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec008689-e53eb11d-15540f49-4ec24ee6_0] Write session will restart in 0.007808s
2025-10-09T21:51:44.511802Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec008689-e53eb11d-15540f49-4ec24ee6_0] Write session: Do CDS request
2025-10-09T21:51:44.511808Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec008689-e53eb11d-15540f49-4ec24ee6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:44.517697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:44.518977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:44.519130Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:44.519137Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:44.519145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:44.519182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:44.519212Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343034000619277:11939]
2025-10-09T21:51:44.519584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:44.519801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:44.519826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:44.520024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:44.520054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.520058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.520059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.520060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.520061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.520061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.520062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.520063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.520065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.520068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.520076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.520078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.520080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.520082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.520084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.520086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.520088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.520090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.520095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.520099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.520103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.520110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.520112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.520115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.520118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.520121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.520123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.520126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.520128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.520131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.520134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.520139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.520141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.520149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.520151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.520154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.520170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:44.520432Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ec008689-e53eb11d-15540f49-4ec24ee6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:44.520541Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:44.520557Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2233
2025-10-09T21:51:44.520665Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2233 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:44.520690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2233 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:44.520697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2233 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:44.520700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:44.521005Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2233 sessionId:  describe result for acl check
2025-10-09T21:51:44.521032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:44.521038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:44.521039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:44.521043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343034362470381:8389] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:44.521496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343034362470381:8389] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:44.523623Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343034362470381:8389] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:44.526510Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343034362470381:8389] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:44.526524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343034362470381:8389] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:44.526526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343034362470381:8389] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:44.526539Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2233 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:44.526673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:44.527796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:44.527808Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:44.527814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:44.527818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:44.527824Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2232 [2:7559343034000619277:11939]
2025-10-09T21:51:44.527837Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:44.527856Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:44.527865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:44.527869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:44.527871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:44.527887Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:44.527893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:44.527902Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:44.527952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.527958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.527960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.527962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:44.528021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:44.528028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.528040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.528056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.528077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:44.528103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343034362470351:2432], now have 1 active actors on pipe
2025-10-09T21:51:44.528107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343034362470416:8389], now have 1 active actors on pipe
2025-10-09T21:51:44.528114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343034000619272:11939]
2025-10-09T21:51:44.528121Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343034000619272:11939]
2025-10-09T21:51:44.528122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343034000619272:11939]
2025-10-09T21:51:44.528134Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2232
2025-10-09T21:51:44.528145Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2232
2025-10-09T21:51:44.528136Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:44.528151Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:44.528153Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:44.528155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:44.528155Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:44.528169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:44.528172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:44.528212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:44.528221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:44.528232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343034000619272:11939]
2025-10-09T21:51:44.528243Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:44.528249Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:44.528271Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e39eaebe-57634321-5625a626-14693153_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:44.528282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:44.528286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:44.528293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:44.528304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:44.528314Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:44.528338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:44.528482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:44.528488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:44.528501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:44.528579Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2233 partition: 0 MaxSeqNo: 25 sessionId: src_id|e39eaebe-57634321-5625a626-14693153_0
2025-10-09T21:51:44.528886Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ec008689-e53eb11d-15540f49-4ec24ee6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e39eaebe-57634321-5625a626-14693153_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:44.531401Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e39eaebe-57634321-5625a626-14693153_0] Write session: try to update token
2025-10-09T21:51:44.531419Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e39eaebe-57634321-5625a626-14693153_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:44.543402Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:44.543424Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:44.543426Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:44.543427Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:44.543434Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343034000619272:11939]
2025-10-09T21:51:44.543438Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343034000619272:11939]
2025-10-09T21:51:44.543440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343034000619272:11939]
2025-10-09T21:51:44.543478Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:44.543486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:44.543491Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:44.543492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.543494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:44.543496Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:44.543623Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:44.543968Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2233 sessionId: src_id|e39eaebe-57634321-5625a626-14693153_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:44.546234Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:44.588431Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: CLIENT_DEADLINE_EXCEEDED. Description: [ { <main>: Error: GRpc error: (4): Deadline Exceeded } { <main>: Error: Grpc error response on endpoint localhost:14704 } ]
2025-10-09T21:51:44.611346Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343034362470423:8652]
2025-10-09T21:51:44.611631Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 2232, pipe: [1:7559343034362470423:8652]
2025-10-09T21:51:44.690142Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:44.690472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:44.690487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:44.690527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343034362470423:8652], now have 1 active actors on pipe
2025-10-09T21:51:44.690753Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:44.690778Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343034362470423:8652]
2025-10-09T21:51:44.690793Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:44.690798Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:44.690816Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:44.690862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2233 reason: error at write: Disconnected sessionId: src_id|e39eaebe-57634321-5625a626-14693153_0
2025-10-09T21:51:44.691028Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2233 sessionId: src_id|e39eaebe-57634321-5625a626-14693153_0 is DEAD
2025-10-09T21:51:44.691180Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e39eaebe-57634321-5625a626-14693153_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:44.691191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e39eaebe-57634321-5625a626-14693153_0] Write session will restart in 0.005999s
2025-10-09T21:51:44.691211Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e39eaebe-57634321-5625a626-14693153_0] Write session: Do CDS request
2025-10-09T21:51:44.691215Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e39eaebe-57634321-5625a626-14693153_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:44.697503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:44.698408Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e39eaebe-57634321-5625a626-14693153_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:44.698522Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:44.698542Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2234
2025-10-09T21:51:44.698672Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2234 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:44.698678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:44.698706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2234 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:44.698714Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2234 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:44.698718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:44.698875Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:44.698885Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:44.698894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:44.698937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:44.698972Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343034000619341:11941]
2025-10-09T21:51:44.699081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2234 sessionId:  describe result for acl check
2025-10-09T21:51:44.699123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:44.699132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:44.699134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:44.699139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343034362470453:8380] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:44.699422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:44.699581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343034362470453:8380] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:44.699669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:44.699701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:44.699972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:44.700021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.700029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.700030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.700031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.700032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.700033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.700033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.700034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.700035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.700038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.700048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.700052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.700056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.700060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.700064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.700068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.700071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.700075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.700091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.700105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.700112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.700124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.700129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.700134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.700139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.700151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.700156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.700163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.700174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.700177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.700180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.700183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.700186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.700190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.700193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.700197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.700216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:44.701473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343034362470453:8380] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:44.706362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343034362470453:8380] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:44.706376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343034362470453:8380] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:44.706379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343034362470453:8380] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:44.706393Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2234 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:44.706991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:44.708246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:44.708266Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:44.708270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:44.708273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:44.708278Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2233 [2:7559343034000619341:11941]
2025-10-09T21:51:44.708290Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:44.708305Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:44.708307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:44.708309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:44.708311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:44.708326Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:44.708332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:44.708340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:44.708386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.708396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.708398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.708400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:44.708457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:44.708464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.708470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.708478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.708488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:44.708505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343034000619336:11941]
2025-10-09T21:51:44.708506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343034000619336:11941]
2025-10-09T21:51:44.708507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343034000619336:11941]
2025-10-09T21:51:44.708514Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:44.708525Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:44.708533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:44.708535Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:44.708540Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:44.708544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343034362470426:2432], now have 1 active actors on pipe
2025-10-09T21:51:44.708544Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:44.708547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343034362470491:8380], now have 1 active actors on pipe
2025-10-09T21:51:44.708547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:44.708570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343034000619336:11941]
2025-10-09T21:51:44.708585Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:44.708614Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:44.708609Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2233
2025-10-09T21:51:44.708628Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2233
2025-10-09T21:51:44.708702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:44.708710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:44.708730Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d1937881-32f36d59-4efd9cd5-4f7d67b8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:44.708740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:44.708742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:44.708744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:44.708752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:44.708759Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:44.708769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:44.708875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:44.708882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:44.708889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:44.708976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2234 partition: 0 MaxSeqNo: 25 sessionId: src_id|d1937881-32f36d59-4efd9cd5-4f7d67b8_0
2025-10-09T21:51:44.709255Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e39eaebe-57634321-5625a626-14693153_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d1937881-32f36d59-4efd9cd5-4f7d67b8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:44.711642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1937881-32f36d59-4efd9cd5-4f7d67b8_0] Write session: try to update token
2025-10-09T21:51:44.711660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1937881-32f36d59-4efd9cd5-4f7d67b8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:44.723835Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:44.723850Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:44.723853Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:44.723856Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:44.723866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343034000619336:11941]
2025-10-09T21:51:44.723872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343034000619336:11941]
2025-10-09T21:51:44.723877Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343034000619336:11941]
2025-10-09T21:51:44.723920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:44.723934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.723937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:44.723950Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:44.723965Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:44.723970Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:44.724087Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:44.724515Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2234 sessionId: src_id|d1937881-32f36d59-4efd9cd5-4f7d67b8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:44.726679Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:44.791039Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343034362470502:8652]
2025-10-09T21:51:44.791242Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 2233, pipe: [1:7559343034362470502:8652]
2025-10-09T21:51:44.873475Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:44.873864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:44.873878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:44.873911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343034362470502:8652], now have 1 active actors on pipe
2025-10-09T21:51:44.874170Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:44.874170Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343034362470502:8652]
2025-10-09T21:51:44.874195Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:44.874197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:44.874243Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:44.874258Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2234 reason: error at write: Disconnected sessionId: src_id|d1937881-32f36d59-4efd9cd5-4f7d67b8_0
2025-10-09T21:51:44.874410Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2234 sessionId: src_id|d1937881-32f36d59-4efd9cd5-4f7d67b8_0 is DEAD
2025-10-09T21:51:44.874571Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1937881-32f36d59-4efd9cd5-4f7d67b8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:44.874586Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1937881-32f36d59-4efd9cd5-4f7d67b8_0] Write session will restart in 0.006128s
2025-10-09T21:51:44.874613Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1937881-32f36d59-4efd9cd5-4f7d67b8_0] Write session: Do CDS request
2025-10-09T21:51:44.874619Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1937881-32f36d59-4efd9cd5-4f7d67b8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:44.880391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:44.881459Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1937881-32f36d59-4efd9cd5-4f7d67b8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:44.881536Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:44.881551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2235
2025-10-09T21:51:44.881666Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2235 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:44.881700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2235 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:44.881711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2235 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:44.881715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:44.881938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:44.882075Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2235 sessionId:  describe result for acl check
2025-10-09T21:51:44.882111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:44.882113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:44.882114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:44.882118Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343034362470534:8377] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:44.882137Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:44.882146Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:44.882154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:44.882195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:44.882232Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343034000619411:11945]
2025-10-09T21:51:44.882571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343034362470534:8377] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:44.882668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:44.882939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:44.882974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:44.883221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:44.883264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.883265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.883266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.883266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.883267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.883268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.883269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.883269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.883270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.883273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.883280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.883283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.883285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.883287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.883290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.883292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.883294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.883296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.883303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.883308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:44.883313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.883316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:44.883319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.883324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:44.883327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.883331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:44.883334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.883338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:44.883340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.883346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:44.883348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.883351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.883354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.883357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.883360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.883363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.883383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:44.884381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343034362470534:8377] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:44.888573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343034362470534:8377] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:44.888587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343034362470534:8377] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:44.888590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343034362470534:8377] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:44.888598Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2235 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:44.889098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:44.890285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:44.890297Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:44.890300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:44.890302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:44.890307Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2234 [2:7559343034000619411:11945]
2025-10-09T21:51:44.890319Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:44.890334Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:44.890336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:44.890338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:44.890341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:44.890352Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:44.890356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:44.890361Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:44.890407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.890414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.890416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.890418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:44.890471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:44.890478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.890485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:44.890492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:44.890501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:44.890518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343034000619406:11945]
2025-10-09T21:51:44.890519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343034000619406:11945]
2025-10-09T21:51:44.890521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343034000619406:11945]
2025-10-09T21:51:44.890527Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:44.890536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343034362470506:2432], now have 1 active actors on pipe
2025-10-09T21:51:44.890541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343034362470572:8377], now have 1 active actors on pipe
2025-10-09T21:51:44.890541Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:44.890545Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:44.890545Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:44.890547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:44.890553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:44.890555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:44.890575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343034000619406:11945]
2025-10-09T21:51:44.890586Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:44.890597Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:44.890599Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2234
2025-10-09T21:51:44.890605Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2234
2025-10-09T21:51:44.890672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:44.890681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:44.890698Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8087c854-8856bb9b-587cf54c-39c10a84_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:44.890708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:44.890711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:44.890713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:44.890718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:44.890725Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:44.890735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:44.890928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:44.890959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:44.890985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:44.891103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2235 partition: 0 MaxSeqNo: 25 sessionId: src_id|8087c854-8856bb9b-587cf54c-39c10a84_0
2025-10-09T21:51:44.891389Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1937881-32f36d59-4efd9cd5-4f7d67b8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8087c854-8856bb9b-587cf54c-39c10a84_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:44.893773Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8087c854-8856bb9b-587cf54c-39c10a84_0] Write session: try to update token
2025-10-09T21:51:44.893791Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8087c854-8856bb9b-587cf54c-39c10a84_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:44.905761Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:44.905781Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:44.905785Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:44.905787Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:44.905797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343034000619406:11945]
2025-10-09T21:51:44.905801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343034000619406:11945]
2025-10-09T21:51:44.905806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343034000619406:11945]
2025-10-09T21:51:44.905847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:44.905857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:44.905859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:44.905883Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:44.905890Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:44.905894Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:44.906009Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:44.906929Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2235 sessionId: src_id|8087c854-8856bb9b-587cf54c-39c10a84_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:44.909132Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:44.974345Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343034362470585:8652]
2025-10-09T21:51:44.974627Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 2234, pipe: [1:7559343034362470585:8652]
2025-10-09T21:51:45.054733Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:45.055072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.055086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.055129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343034362470585:8652], now have 1 active actors on pipe
2025-10-09T21:51:45.055320Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:45.055367Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:45.055374Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343034362470585:8652]
2025-10-09T21:51:45.055399Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:45.055482Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2235 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8087c854-8856bb9b-587cf54c-39c10a84_0
2025-10-09T21:51:45.055709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2235 sessionId: src_id|8087c854-8856bb9b-587cf54c-39c10a84_0 is DEAD
2025-10-09T21:51:45.055856Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8087c854-8856bb9b-587cf54c-39c10a84_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:45.055878Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8087c854-8856bb9b-587cf54c-39c10a84_0] Write session will restart in 0.005429s
2025-10-09T21:51:45.055897Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8087c854-8856bb9b-587cf54c-39c10a84_0] Write session: Do CDS request
2025-10-09T21:51:45.055902Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8087c854-8856bb9b-587cf54c-39c10a84_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:45.062502Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8087c854-8856bb9b-587cf54c-39c10a84_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:45.062607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:45.062627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2236
2025-10-09T21:51:45.062744Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2236 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:45.062778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2236 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:45.062789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2236 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:45.062794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:45.063223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2236 sessionId:  describe result for acl check
2025-10-09T21:51:45.063262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:45.063264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:45.063266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:45.063271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343038657437912:8370] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:45.063768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343038657437912:8370] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:45.065927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343038657437912:8370] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:45.067930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:45.069494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:45.069726Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:45.069739Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:45.069752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:45.069830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:45.069729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343038657437912:8370] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:45.069754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343038657437912:8370] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:45.069757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343038657437912:8370] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:45.069769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2236 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:45.069890Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343038295586774:11947]
2025-10-09T21:51:45.070469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:45.070818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:45.070871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:45.071115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:45.071158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.071159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.071159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.071160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.071161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.071162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.071163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.071163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.071164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.071168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.071175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.071177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.071180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.071182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.071185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.071187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.071189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.071191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.071196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.071201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.071208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.071211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.071214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.071217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.071222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.071225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.071229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.071232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.071235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.071238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.071243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.071247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.071251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.071258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.071261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.071264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.071282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:45.079051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:45.080543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:45.080559Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:45.080562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:45.080567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:45.080575Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2235 [2:7559343038295586774:11947]
2025-10-09T21:51:45.080591Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:45.080612Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:45.080615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:45.080619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:45.080622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:45.080642Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:45.080650Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:45.080660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:45.080765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.080773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.080776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.080778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:45.080890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:45.080898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.080911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.080923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.080944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:45.080971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343038657437948:8370], now have 1 active actors on pipe
2025-10-09T21:51:45.080979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343038657437884:2432], now have 1 active actors on pipe
2025-10-09T21:51:45.080987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343038295586769:11947]
2025-10-09T21:51:45.080990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343038295586769:11947]
2025-10-09T21:51:45.080992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343038295586769:11947]
2025-10-09T21:51:45.081007Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:45.080994Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2235
2025-10-09T21:51:45.081007Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2235
2025-10-09T21:51:45.081036Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:45.081038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:45.081043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:45.081049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:45.081053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:45.081056Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:45.081074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.081085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.081146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343038295586769:11947]
2025-10-09T21:51:45.081158Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:45.081174Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:45.081182Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e65e8cb3-ddecf37-f186dbea-89cc84d7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:45.081194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:45.081199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:45.081201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:45.081212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:45.081227Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:45.081273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:45.081422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.081433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.081455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:45.081548Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2236 partition: 0 MaxSeqNo: 25 sessionId: src_id|e65e8cb3-ddecf37-f186dbea-89cc84d7_0
2025-10-09T21:51:45.081922Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8087c854-8856bb9b-587cf54c-39c10a84_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e65e8cb3-ddecf37-f186dbea-89cc84d7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:45.084358Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e65e8cb3-ddecf37-f186dbea-89cc84d7_0] Write session: try to update token
2025-10-09T21:51:45.084379Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e65e8cb3-ddecf37-f186dbea-89cc84d7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:45.098575Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2236 sessionId: src_id|e65e8cb3-ddecf37-f186dbea-89cc84d7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:45.099102Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:45.099121Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:45.099124Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:45.099125Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:45.099133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343038295586769:11947]
2025-10-09T21:51:45.099138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343038295586769:11947]
2025-10-09T21:51:45.099142Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343038295586769:11947]
2025-10-09T21:51:45.099187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:45.099194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.099197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:45.099222Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:45.099242Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:45.099247Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:45.099363Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:45.101120Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:45.155585Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343038657437959:8652]
2025-10-09T21:51:45.155916Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 2235, pipe: [1:7559343038657437959:8652]
2025-10-09T21:51:45.249737Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:45.250068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.250082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.250122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343038657437959:8652], now have 1 active actors on pipe
2025-10-09T21:51:45.250349Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:45.250422Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343038657437959:8652]
2025-10-09T21:51:45.250448Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:45.250453Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:45.250476Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:45.250515Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2236 reason: error at write: Disconnected sessionId: src_id|e65e8cb3-ddecf37-f186dbea-89cc84d7_0
2025-10-09T21:51:45.250687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2236 sessionId: src_id|e65e8cb3-ddecf37-f186dbea-89cc84d7_0 is DEAD
2025-10-09T21:51:45.250832Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e65e8cb3-ddecf37-f186dbea-89cc84d7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:45.250844Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e65e8cb3-ddecf37-f186dbea-89cc84d7_0] Write session will restart in 0.009865s
2025-10-09T21:51:45.250871Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e65e8cb3-ddecf37-f186dbea-89cc84d7_0] Write session: Do CDS request
2025-10-09T21:51:45.250877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e65e8cb3-ddecf37-f186dbea-89cc84d7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:45.257356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:45.258618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:45.258791Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:45.258799Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:45.258807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:45.258855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:45.258888Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343038295586847:11949]
2025-10-09T21:51:45.259293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:45.259575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:45.259618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:45.259876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:45.259917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.259922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.259922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.259923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.259924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.259924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.259925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.259926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.259927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.259930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.259938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.259941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.259943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.259946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.259948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.259950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.259952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.259954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.259968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.259974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.259978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.259982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.259984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.259987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.259990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.259992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.259995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.259998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.260001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.260004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.260007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.260010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.260012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.260015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.260018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.260021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.260037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:45.261531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e65e8cb3-ddecf37-f186dbea-89cc84d7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:45.261621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:45.261636Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2237
2025-10-09T21:51:45.261729Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2237 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:45.261753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2237 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:45.261757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2237 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:45.261764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:45.262023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2237 sessionId:  describe result for acl check
2025-10-09T21:51:45.262062Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:45.262067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:45.262071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:45.262075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343038657438003:8357] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:45.262426Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343038657438003:8357] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:45.264399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343038657438003:8357] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:45.267475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343038657438003:8357] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:45.267486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343038657438003:8357] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:45.267490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343038657438003:8357] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:45.267498Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2237 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:45.267843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:45.268838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:45.268852Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:45.268856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:45.268859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:45.268865Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2236 [2:7559343038295586847:11949]
2025-10-09T21:51:45.268876Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:45.268892Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:45.268897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:45.268899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:45.268900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:45.268914Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:45.268923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:45.268932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:45.268973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.268978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.268979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.268981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:45.269039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:45.269049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.269059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.269065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.269074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:45.269088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343038295586840:11949]
2025-10-09T21:51:45.269093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343038295586840:11949]
2025-10-09T21:51:45.269095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343038295586840:11949]
2025-10-09T21:51:45.269100Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:45.269111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:45.269118Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:45.269119Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:45.269121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:45.269127Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:45.269129Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:45.269153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343038657437965:2432], now have 1 active actors on pipe
2025-10-09T21:51:45.269159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343038657438046:8357], now have 1 active actors on pipe
2025-10-09T21:51:45.269161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343038295586840:11949]
2025-10-09T21:51:45.269166Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:45.269175Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:45.269185Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2236
2025-10-09T21:51:45.269229Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2236
2025-10-09T21:51:45.269301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.269312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.269345Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3dd794e5-8c795ba3-4bf5e9f4-10b96b47_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:45.269351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:45.269356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:45.269358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:45.269365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:45.269372Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:45.269385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:45.269540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.269552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.269573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:45.269667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2237 partition: 0 MaxSeqNo: 25 sessionId: src_id|3dd794e5-8c795ba3-4bf5e9f4-10b96b47_0
2025-10-09T21:51:45.269998Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e65e8cb3-ddecf37-f186dbea-89cc84d7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3dd794e5-8c795ba3-4bf5e9f4-10b96b47_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:45.272373Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3dd794e5-8c795ba3-4bf5e9f4-10b96b47_0] Write session: try to update token
2025-10-09T21:51:45.272388Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3dd794e5-8c795ba3-4bf5e9f4-10b96b47_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:45.284360Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:45.284378Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:45.284395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343038295586840:11949]
2025-10-09T21:51:45.284435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:45.284439Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:45.284584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:45.284719Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:45.284848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:45.284970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:45.285135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:45.285168Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:45.285291Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2237 sessionId: src_id|3dd794e5-8c795ba3-4bf5e9f4-10b96b47_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:45.287677Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:45.287847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.287862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.287892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:45.287982Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:45.303075Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:45.303089Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:45.303090Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:45.303091Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:45.303099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343038295586840:11949]
2025-10-09T21:51:45.303102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343038295586840:11949]
2025-10-09T21:51:45.303104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343038295586840:11949]
2025-10-09T21:51:45.303138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:45.303143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.303142Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:45.303145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:45.303150Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:45.303156Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:45.303259Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:45.350632Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343038657438069:8652]
2025-10-09T21:51:45.350927Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 2236, pipe: [1:7559343038657438069:8652]
2025-10-09T21:51:45.450053Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:45.457605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.457627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.457774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.457829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.457885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.457940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.457991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.458001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.458089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.458153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.458216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.458281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.458322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.458330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.458398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.458455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.458515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.458570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.458610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.458617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.458683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.459078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.459140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.459200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.459240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.459246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.459321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.459389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.459455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.459522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.459567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.459574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.459634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.459685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.459742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.459799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.459840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.459845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.459902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.459954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.460010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.460066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.460110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.460119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.460174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.460230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.460293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.460360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.460399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.460404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.460460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.460522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.460585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.460647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.460687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.460692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.460748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.460801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.460861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.460921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.460965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.460971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.461029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343038657438069:8652], now have 1 active actors on pipe
2025-10-09T21:51:45.461265Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343038657438069:8652]
2025-10-09T21:51:45.461278Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:45.461293Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:45.461300Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:45.461334Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2237 reason: error at write: Disconnected sessionId: src_id|3dd794e5-8c795ba3-4bf5e9f4-10b96b47_0
2025-10-09T21:51:45.461536Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2237 sessionId: src_id|3dd794e5-8c795ba3-4bf5e9f4-10b96b47_0 is DEAD
2025-10-09T21:51:45.461684Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3dd794e5-8c795ba3-4bf5e9f4-10b96b47_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:45.461694Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3dd794e5-8c795ba3-4bf5e9f4-10b96b47_0] Write session will restart in 0.006479s
2025-10-09T21:51:45.461716Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3dd794e5-8c795ba3-4bf5e9f4-10b96b47_0] Write session: Do CDS request
2025-10-09T21:51:45.461720Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3dd794e5-8c795ba3-4bf5e9f4-10b96b47_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:45.461749Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:45.469416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:45.469526Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3dd794e5-8c795ba3-4bf5e9f4-10b96b47_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:45.469652Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:45.469668Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2238
2025-10-09T21:51:45.469799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2238 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:45.469829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2238 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:45.469840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2238 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:45.469844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:45.470286Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2238 sessionId:  describe result for acl check
2025-10-09T21:51:45.470316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:45.470320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:45.470322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:45.470328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343038657438102:8679] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:45.470981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343038657438102:8679] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:45.471083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:45.471268Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:45.471277Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:45.471289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:45.471348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:45.471387Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343038295586946:11963]
2025-10-09T21:51:45.471936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:45.472163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:45.472199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:45.472502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:45.472550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.472556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.472557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.472558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.472559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.472560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.472561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.472562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.472563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.472568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.472577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.472581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.472585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.472589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.472592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.472595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.472599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.472605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.472611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.472622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.472632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.472641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.472645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.472651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.472654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.472659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.472663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.472668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.472677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.472682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.472690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.472695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.472699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.472708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.472713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.472717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.472745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:45.473013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343038657438102:8679] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:45.479656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343038657438102:8679] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:45.479670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343038657438102:8679] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:45.479674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343038657438102:8679] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:45.479683Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2238 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:45.480011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:45.481189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:45.481202Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:45.481204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:45.481206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:45.481213Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2237 [2:7559343038295586946:11963]
2025-10-09T21:51:45.481222Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:45.481237Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:45.481239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:45.481241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:45.481242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:45.481258Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:45.481265Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:45.481270Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:45.481322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.481335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.481337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.481339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:45.481390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:45.481394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.481401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.481407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.481416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:45.481431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343038657438071:2432], now have 1 active actors on pipe
2025-10-09T21:51:45.481437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343038657438141:8679], now have 1 active actors on pipe
2025-10-09T21:51:45.481443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343038295586941:11963]
2025-10-09T21:51:45.481444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343038295586941:11963]
2025-10-09T21:51:45.481445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343038295586941:11963]
2025-10-09T21:51:45.481453Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:45.481463Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:45.481468Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:45.481471Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:45.481473Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:45.481473Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:45.481475Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:45.481498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343038295586941:11963]
2025-10-09T21:51:45.481484Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2237
2025-10-09T21:51:45.481503Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:45.481504Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2237
2025-10-09T21:51:45.481525Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:45.481544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.481553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.481575Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5733205e-7ee11e09-48b0ce74-d985b19a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:45.481585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:45.481587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:45.481588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:45.481596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:45.481606Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:45.481623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:45.481759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.481773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.481796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:45.481868Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2238 partition: 0 MaxSeqNo: 25 sessionId: src_id|5733205e-7ee11e09-48b0ce74-d985b19a_0
2025-10-09T21:51:45.482194Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3dd794e5-8c795ba3-4bf5e9f4-10b96b47_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5733205e-7ee11e09-48b0ce74-d985b19a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:45.484549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5733205e-7ee11e09-48b0ce74-d985b19a_0] Write session: try to update token
2025-10-09T21:51:45.484563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5733205e-7ee11e09-48b0ce74-d985b19a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:45.494468Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:45.494483Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:45.494493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343038295586941:11963]
2025-10-09T21:51:45.494527Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:45.494539Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:45.494714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:45.494864Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:45.495004Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:45.495142Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:45.495326Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:45.495372Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:45.496818Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2238 sessionId: src_id|5733205e-7ee11e09-48b0ce74-d985b19a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:45.499207Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:45.499400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.499428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.499463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:45.499538Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:45.509604Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:45.509618Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:45.509619Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:45.509622Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:45.509631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343038295586941:11963]
2025-10-09T21:51:45.509635Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343038295586941:11963]
2025-10-09T21:51:45.509638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343038295586941:11963]
2025-10-09T21:51:45.509670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:45.509674Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:45.509677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.509678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:45.509679Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:45.509682Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:45.509805Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:45.561598Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343038657438149:8652]
2025-10-09T21:51:45.561886Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 2237, pipe: [1:7559343038657438149:8652]
2025-10-09T21:51:45.651945Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:45.659233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.659259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.659431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.659493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.659560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.659625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.659674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.659685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.659759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.659821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.659888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.659964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.660011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.660020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.660084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.660141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.660211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.660326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.660367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.660376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.660445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.660511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.660573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.660639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.660687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.660690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.660754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.660806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.660868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.660974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.661018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.661021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.661091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.661147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.661209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.661275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.661322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.661339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.661411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.661475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.661546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.661616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.661682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.661690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.661756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.661861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.661969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.662105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.662155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.662168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.662239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.662300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.662370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.662436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.662482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.662485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.662554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:45.662625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:45.662697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:45.662768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:45.662819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:45.662829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:45.662902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343038657438149:8652], now have 1 active actors on pipe
2025-10-09T21:51:45.664156Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:45.664171Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343038657438149:8652]
2025-10-09T21:51:45.664219Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:45.664511Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:45.664567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2238 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5733205e-7ee11e09-48b0ce74-d985b19a_0
2025-10-09T21:51:45.664761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2238 sessionId: src_id|5733205e-7ee11e09-48b0ce74-d985b19a_0 is DEAD
2025-10-09T21:51:45.664908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5733205e-7ee11e09-48b0ce74-d985b19a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:45.664918Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5733205e-7ee11e09-48b0ce74-d985b19a_0] Write session will restart in 0.008373s
2025-10-09T21:51:45.664944Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5733205e-7ee11e09-48b0ce74-d985b19a_0] Write session: Do CDS request
2025-10-09T21:51:45.664951Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5733205e-7ee11e09-48b0ce74-d985b19a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:45.672477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:45.673937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:45.674175Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:45.674186Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:45.674202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:45.674267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:45.674316Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343038295587016:11965]
2025-10-09T21:51:45.674588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5733205e-7ee11e09-48b0ce74-d985b19a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:45.674661Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:45.674684Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2239
2025-10-09T21:51:45.674819Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2239 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:45.674851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2239 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:45.674862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2239 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:45.674866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:45.674872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:45.675170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:45.675224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:45.675280Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2239 sessionId:  describe result for acl check
2025-10-09T21:51:45.675326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:45.675334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:45.675336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:45.675343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343038657438185:8663] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:45.675542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:45.675610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.675617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.675619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.675620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.675621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.675622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.675623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.675624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.675625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.675631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.675642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.675646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.675650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.675654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.675658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.675662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.675666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.675670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.675677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.675686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.675692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.675697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.675700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.675707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.675711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.675715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.675719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.675724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.675728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.675732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.675737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.675741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.675745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.675750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.675754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.675759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.675786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:45.675903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343038657438185:8663] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:45.679002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343038657438185:8663] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:45.682813Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343038657438185:8663] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:45.682827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343038657438185:8663] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:45.682830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343038657438185:8663] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:45.682838Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2239 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:45.683243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:45.684240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:45.684261Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:45.684263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:45.684280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:45.684287Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2238 [2:7559343038295587016:11965]
2025-10-09T21:51:45.684300Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:45.684323Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:45.684328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:45.684330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:45.684333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:45.684346Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:45.684355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:45.684360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:45.684413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.684420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.684422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.684425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:45.684498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:45.684507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.684520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.684536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.684551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:45.684573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343038657438151:2432], now have 1 active actors on pipe
2025-10-09T21:51:45.684583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343038657438223:8663], now have 1 active actors on pipe
2025-10-09T21:51:45.684590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343038295587011:11965]
2025-10-09T21:51:45.684592Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343038295587011:11965]
2025-10-09T21:51:45.684594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343038295587011:11965]
2025-10-09T21:51:45.684606Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:45.684619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:45.684625Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:45.684628Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:45.684629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:45.684631Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:45.684605Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2238
2025-10-09T21:51:45.684631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:45.684622Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2238
2025-10-09T21:51:45.684658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343038295587011:11965]
2025-10-09T21:51:45.684664Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:45.684667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:45.684680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.684687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.684723Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fa86d655-1eac538-fd6d02f7-c2510e3d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:45.684733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:45.684736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:45.684738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:45.684746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:45.684753Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:45.684772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:45.684922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.684934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.684956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:45.685022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2239 partition: 0 MaxSeqNo: 25 sessionId: src_id|fa86d655-1eac538-fd6d02f7-c2510e3d_0
2025-10-09T21:51:45.685378Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5733205e-7ee11e09-48b0ce74-d985b19a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fa86d655-1eac538-fd6d02f7-c2510e3d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:45.687728Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fa86d655-1eac538-fd6d02f7-c2510e3d_0] Write session: try to update token
2025-10-09T21:51:45.687747Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fa86d655-1eac538-fd6d02f7-c2510e3d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:45.700334Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2239 sessionId: src_id|fa86d655-1eac538-fd6d02f7-c2510e3d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:45.702163Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:45.702182Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:45.702186Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:45.702188Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:45.702199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343038295587011:11965]
2025-10-09T21:51:45.702204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343038295587011:11965]
2025-10-09T21:51:45.702209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343038295587011:11965]
2025-10-09T21:51:45.702263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:45.702264Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:45.702269Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:45.702272Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:45.702280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.702284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:45.702479Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:45.703581Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:45.764363Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343038657438230:8652]
2025-10-09T21:51:45.764678Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 2238, pipe: [1:7559343038657438230:8652]
2025-10-09T21:51:45.849907Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:45.850249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.850262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.850302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343038657438230:8652], now have 1 active actors on pipe
2025-10-09T21:51:45.850491Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:45.850517Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343038657438230:8652]
2025-10-09T21:51:45.850520Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:45.850554Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:45.850620Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2239 reason: status is not ok: tablet will be restarted right now sessionId: src_id|fa86d655-1eac538-fd6d02f7-c2510e3d_0
2025-10-09T21:51:45.850806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2239 sessionId: src_id|fa86d655-1eac538-fd6d02f7-c2510e3d_0 is DEAD
2025-10-09T21:51:45.850961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa86d655-1eac538-fd6d02f7-c2510e3d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:45.850973Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa86d655-1eac538-fd6d02f7-c2510e3d_0] Write session will restart in 0.009369s
2025-10-09T21:51:45.851011Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa86d655-1eac538-fd6d02f7-c2510e3d_0] Write session: Do CDS request
2025-10-09T21:51:45.851017Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa86d655-1eac538-fd6d02f7-c2510e3d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:45.857560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:45.858953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:45.859146Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:45.859157Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:45.859169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:45.859223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:45.859267Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343038295587090:11969]
2025-10-09T21:51:45.859882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:45.860177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:45.860220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:45.860543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:45.860588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.860594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.860595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.860596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.860598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.860599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.860600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.860602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.860604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.860609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.860620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.860624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.860629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.860633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.860637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.860641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.860646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.860650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.860658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.860673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:45.860679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.860684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:45.860688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.860694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:45.860699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.860705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:45.860709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.860721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:45.860725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.860735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:45.860741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.860747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.860751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.860756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.860765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.860770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.860801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:45.861567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fa86d655-1eac538-fd6d02f7-c2510e3d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:45.861712Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:45.861732Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2240
2025-10-09T21:51:45.861860Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2240 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:45.861886Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2240 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:45.861894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2240 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:45.861897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:45.862344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2240 sessionId:  describe result for acl check
2025-10-09T21:51:45.862385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:45.862390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:45.862392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:45.862398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343038657438271:8644] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:45.862844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343038657438271:8644] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:45.864986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343038657438271:8644] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:45.868117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343038657438271:8644] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:45.868135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343038657438271:8644] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:45.868139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343038657438271:8644] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:45.868149Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2240 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:45.868585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:45.869691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:45.869705Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:45.869708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:45.869712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:45.869719Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2239 [2:7559343038295587090:11969]
2025-10-09T21:51:45.869736Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:45.869757Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:45.869763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:45.869766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:45.869769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:45.869788Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:45.869800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:45.869809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:45.869878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.869884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.869886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.869889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:45.869966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:45.869973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.869985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:45.870000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:45.870020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:45.870064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343038657438305:8644], now have 1 active actors on pipe
2025-10-09T21:51:45.870074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343038657438233:2432], now have 1 active actors on pipe
2025-10-09T21:51:45.870082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343038295587085:11969]
2025-10-09T21:51:45.870084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343038295587085:11969]
2025-10-09T21:51:45.870087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343038295587085:11969]
2025-10-09T21:51:45.870100Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:45.870114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:45.870117Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2239
2025-10-09T21:51:45.870128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:45.870118Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2239
2025-10-09T21:51:45.870136Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:45.870138Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:45.870142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:45.870144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:45.870190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.870200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.870212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343038295587085:11969]
2025-10-09T21:51:45.870224Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:45.870238Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:45.870262Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|92c3b987-469b57c-87be5e0a-3126f574_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:45.870274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:45.870282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:45.870285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:45.870299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:45.870313Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:45.870340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:45.870543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:45.870552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:45.870567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:45.870655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2240 partition: 0 MaxSeqNo: 25 sessionId: src_id|92c3b987-469b57c-87be5e0a-3126f574_0
2025-10-09T21:51:45.871019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fa86d655-1eac538-fd6d02f7-c2510e3d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|92c3b987-469b57c-87be5e0a-3126f574_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:45.873608Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92c3b987-469b57c-87be5e0a-3126f574_0] Write session: try to update token
2025-10-09T21:51:45.873626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92c3b987-469b57c-87be5e0a-3126f574_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:45.887141Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:45.887161Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:45.887164Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:45.887166Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:45.887179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343038295587085:11969]
2025-10-09T21:51:45.887184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343038295587085:11969]
2025-10-09T21:51:45.887188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343038295587085:11969]
2025-10-09T21:51:45.887228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:45.887237Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:45.887239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:45.887241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:45.887250Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:45.887254Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:45.887384Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:45.887609Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2240 sessionId: src_id|92c3b987-469b57c-87be5e0a-3126f574_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:45.890090Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:45.950727Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343038657438314:8652]
2025-10-09T21:51:45.951097Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 2239, pipe: [1:7559343038657438314:8652]
2025-10-09T21:51:46.040021Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:46.040319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.040334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.040375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343038657438314:8652], now have 1 active actors on pipe
2025-10-09T21:51:46.040647Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343038657438314:8652]
2025-10-09T21:51:46.040661Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:46.040692Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:46.040695Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:46.040760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2240 reason: error at write: Disconnected sessionId: src_id|92c3b987-469b57c-87be5e0a-3126f574_0
2025-10-09T21:51:46.040786Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:46.040940Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2240 sessionId: src_id|92c3b987-469b57c-87be5e0a-3126f574_0 is DEAD
2025-10-09T21:51:46.041125Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92c3b987-469b57c-87be5e0a-3126f574_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:46.041137Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92c3b987-469b57c-87be5e0a-3126f574_0] Write session will restart in 0.008439s
2025-10-09T21:51:46.041168Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92c3b987-469b57c-87be5e0a-3126f574_0] Write session: Do CDS request
2025-10-09T21:51:46.041174Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92c3b987-469b57c-87be5e0a-3126f574_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:46.048745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:46.050306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:46.050537Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:46.050548Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:46.050577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92c3b987-469b57c-87be5e0a-3126f574_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:46.050563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:46.050627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:46.050659Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:46.050691Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343042590554458:11971]
2025-10-09T21:51:46.050678Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2241
2025-10-09T21:51:46.050800Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2241 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:46.050825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2241 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:46.050832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2241 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:46.050836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:46.051228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:46.051243Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2241 sessionId:  describe result for acl check
2025-10-09T21:51:46.051277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:46.051282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:46.051284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:46.051288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343042952405647:8630] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:46.051596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:46.051675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:46.051785Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343042952405647:8630] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:46.052032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:46.052089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.052097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.052098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.052099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.052100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.052102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.052103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.052104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.052105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.052112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.052122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.052126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.052130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.052134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.052139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.052143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.052147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.052151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.052158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.052167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.052174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.052180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.052185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.052191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.052196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.052207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.052212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.052221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.052226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.052236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.052248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.052259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.052264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.052269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.052276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.052285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.052326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:46.053981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343042952405647:8630] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:46.059026Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343042952405647:8630] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:46.059043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343042952405647:8630] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:46.059047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343042952405647:8630] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:46.059057Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2241 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:46.059466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:46.060599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:46.060619Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:46.060622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:46.060627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:46.060637Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2240 [2:7559343042590554458:11971]
2025-10-09T21:51:46.060652Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:46.060680Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:46.060687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:46.060691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:46.060693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:46.060715Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:46.060725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:46.060736Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:46.060803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.060810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.060812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.060817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:46.060888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:46.060895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.060909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.060926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.060948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:46.060974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343042952405613:2432], now have 1 active actors on pipe
2025-10-09T21:51:46.060982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343042952405686:8630], now have 1 active actors on pipe
2025-10-09T21:51:46.060990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343042590554453:11971]
2025-10-09T21:51:46.060997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343042590554453:11971]
2025-10-09T21:51:46.060999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343042590554453:11971]
2025-10-09T21:51:46.061013Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:46.061033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:46.061039Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:46.061043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:46.061045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:46.061021Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2240
2025-10-09T21:51:46.061038Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2240
2025-10-09T21:51:46.061053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:46.061065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:46.061125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.061136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.061150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343042590554453:11971]
2025-10-09T21:51:46.061160Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:46.061166Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:46.061193Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5119d486-d641e658-cc19f655-d4e15778_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:46.061210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:46.061216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:46.061221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:46.061231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:46.061245Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:46.061273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:46.061412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.061422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.061438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:46.061512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2241 partition: 0 MaxSeqNo: 25 sessionId: src_id|5119d486-d641e658-cc19f655-d4e15778_0
2025-10-09T21:51:46.061880Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92c3b987-469b57c-87be5e0a-3126f574_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5119d486-d641e658-cc19f655-d4e15778_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:46.064365Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5119d486-d641e658-cc19f655-d4e15778_0] Write session: try to update token
2025-10-09T21:51:46.064388Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5119d486-d641e658-cc19f655-d4e15778_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:46.076225Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:46.076241Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:46.076244Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:46.076248Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:46.076261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343042590554453:11971]
2025-10-09T21:51:46.076265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343042590554453:11971]
2025-10-09T21:51:46.076269Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343042590554453:11971]
2025-10-09T21:51:46.076313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:46.076317Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:46.076320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.076322Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:46.076323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:46.076325Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:46.076457Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:46.077669Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2241 sessionId: src_id|5119d486-d641e658-cc19f655-d4e15778_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:46.080099Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:46.140680Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343042952405693:8652]
2025-10-09T21:51:46.140976Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 2240, pipe: [1:7559343042952405693:8652]
2025-10-09T21:51:46.235949Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:46.236349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.236366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.236415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343042952405693:8652], now have 1 active actors on pipe
2025-10-09T21:51:46.236660Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:46.236740Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:46.236740Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343042952405693:8652]
2025-10-09T21:51:46.236767Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:46.236845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2241 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5119d486-d641e658-cc19f655-d4e15778_0
2025-10-09T21:51:46.237047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2241 sessionId: src_id|5119d486-d641e658-cc19f655-d4e15778_0 is DEAD
2025-10-09T21:51:46.237252Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5119d486-d641e658-cc19f655-d4e15778_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:46.237265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5119d486-d641e658-cc19f655-d4e15778_0] Write session will restart in 0.005424s
2025-10-09T21:51:46.237296Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5119d486-d641e658-cc19f655-d4e15778_0] Write session: Do CDS request
2025-10-09T21:51:46.237303Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5119d486-d641e658-cc19f655-d4e15778_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:46.243579Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5119d486-d641e658-cc19f655-d4e15778_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:46.243694Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:46.243714Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2242
2025-10-09T21:51:46.243853Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2242 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:46.243895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2242 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:46.243907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2242 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:46.243910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:46.244117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:46.244284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2242 sessionId:  describe result for acl check
2025-10-09T21:51:46.244315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:46.244320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:46.244321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:46.244326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343042952405734:8622] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:46.244783Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343042952405734:8622] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:46.245528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:46.245714Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:46.245723Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:46.245734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:46.245787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:46.245827Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343042590554531:11973]
2025-10-09T21:51:46.246269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:46.246542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:46.246574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:46.246813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:46.246854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.246858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.246859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.246860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.246860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.246861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.246861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.246862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.246863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.246867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.246873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.246875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.246880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.246882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.246885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.246887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.246889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.246891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.246896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.246902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.246906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.246910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.246913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.246910Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343042952405734:8622] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:46.246918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.246921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.246924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.246927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.246930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.246934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.246937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.246940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.246944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.246946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.246950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.246953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.246956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.246972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:46.254358Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343042952405734:8622] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:46.254379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343042952405734:8622] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:46.254383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343042952405734:8622] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:46.254393Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2242 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:46.254570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:46.255787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:46.255802Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:46.255805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:46.255809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:46.255817Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2241 [2:7559343042590554531:11973]
2025-10-09T21:51:46.255832Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:46.255853Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:46.255860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:46.255866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:46.255869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:46.255887Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:46.255897Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:46.255904Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:46.255971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.255978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.255980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.255984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:46.256054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:46.256062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.256074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.256089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.256109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:46.256135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343042952405773:8622], now have 1 active actors on pipe
2025-10-09T21:51:46.256144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343042952405699:2432], now have 1 active actors on pipe
2025-10-09T21:51:46.256153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343042590554526:11973]
2025-10-09T21:51:46.256156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343042590554526:11973]
2025-10-09T21:51:46.256158Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343042590554526:11973]
2025-10-09T21:51:46.256169Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:46.256179Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:46.256182Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:46.256183Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:46.256184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:46.256198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:46.256203Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:46.256194Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2241
2025-10-09T21:51:46.256224Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2241
2025-10-09T21:51:46.256255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.256264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.256276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343042590554526:11973]
2025-10-09T21:51:46.256289Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:46.256295Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:46.256330Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e38ac1f1-192bd1c0-39875b0e-5124549a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:46.256341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:46.256347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:46.256349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:46.256358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:46.256370Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:46.256405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:46.256588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.256608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.256635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:46.256728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2242 partition: 0 MaxSeqNo: 25 sessionId: src_id|e38ac1f1-192bd1c0-39875b0e-5124549a_0
2025-10-09T21:51:46.257127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5119d486-d641e658-cc19f655-d4e15778_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e38ac1f1-192bd1c0-39875b0e-5124549a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:46.259657Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e38ac1f1-192bd1c0-39875b0e-5124549a_0] Write session: try to update token
2025-10-09T21:51:46.259680Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e38ac1f1-192bd1c0-39875b0e-5124549a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:46.272329Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:46.272349Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:46.272353Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:46.272355Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:46.272363Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343042590554526:11973]
2025-10-09T21:51:46.272367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343042590554526:11973]
2025-10-09T21:51:46.272371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343042590554526:11973]
2025-10-09T21:51:46.272411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:46.272421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.272423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:46.272432Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:46.272438Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:46.272443Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:46.272573Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:46.273182Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2242 sessionId: src_id|e38ac1f1-192bd1c0-39875b0e-5124549a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:46.275751Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:46.336758Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343042952405798:8652]
2025-10-09T21:51:46.336932Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 2241, pipe: [1:7559343042952405798:8652]
2025-10-09T21:51:46.419853Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:46.420112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.420124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.420169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343042952405798:8652], now have 1 active actors on pipe
2025-10-09T21:51:46.420405Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:46.420398Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343042952405798:8652]
2025-10-09T21:51:46.420426Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:46.420430Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:46.420441Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:46.420491Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2242 reason: error at write: Disconnected sessionId: src_id|e38ac1f1-192bd1c0-39875b0e-5124549a_0
2025-10-09T21:51:46.420791Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e38ac1f1-192bd1c0-39875b0e-5124549a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:46.420803Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e38ac1f1-192bd1c0-39875b0e-5124549a_0] Write session will restart in 0.006056s
2025-10-09T21:51:46.420827Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e38ac1f1-192bd1c0-39875b0e-5124549a_0] Write session: Do CDS request
2025-10-09T21:51:46.420836Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e38ac1f1-192bd1c0-39875b0e-5124549a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:46.421082Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2242 sessionId: src_id|e38ac1f1-192bd1c0-39875b0e-5124549a_0 is DEAD
2025-10-09T21:51:46.427538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e38ac1f1-192bd1c0-39875b0e-5124549a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:46.427646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:46.427644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:46.427662Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2243
2025-10-09T21:51:46.427789Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2243 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:46.427823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2243 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:46.427833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2243 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:46.427836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:46.428261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2243 sessionId:  describe result for acl check
2025-10-09T21:51:46.428299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:46.428303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:46.428305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:46.428311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343042952405846:8582] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:46.428915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343042952405846:8582] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:46.429156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:46.429385Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:46.429394Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:46.429403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:46.429448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:46.429489Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343042590554641:11987]
2025-10-09T21:51:46.429883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:46.430167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:46.430217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:46.430452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:46.430491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.430496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.430497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.430497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.430498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.430499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.430499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.430500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.430501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.430505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.430511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.430513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.430516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.430518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.430519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.430523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.430525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.430528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.430534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.430541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.430547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.430553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.430557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.430561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.430563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.430566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.430569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.430572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.430574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.430577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.430582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.430585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.430588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.430591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.430593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.430596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.430611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:46.430655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343042952405846:8582] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:46.437522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:46.437524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343042952405846:8582] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:46.437537Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343042952405846:8582] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:46.437541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343042952405846:8582] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:46.437549Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2243 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:46.438666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:46.438677Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:46.438679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:46.438682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:46.438689Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2242 [2:7559343042590554641:11987]
2025-10-09T21:51:46.438699Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:46.438715Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:46.438716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:46.438720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:46.438722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:46.438739Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:46.438744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:46.438751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:46.438800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.438806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.438807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.438817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:46.438872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:46.438877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.438884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.438890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.438899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:46.438915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343042952405810:2432], now have 1 active actors on pipe
2025-10-09T21:51:46.438925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343042590554636:11987]
2025-10-09T21:51:46.438926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343042590554636:11987]
2025-10-09T21:51:46.438927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343042590554636:11987]
2025-10-09T21:51:46.438938Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:46.438949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343042952405885:8582], now have 1 active actors on pipe
2025-10-09T21:51:46.438950Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:46.438952Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:46.438953Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:46.438956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:46.438964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:46.438966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:46.438984Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2242
2025-10-09T21:51:46.439000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343042590554636:11987]
2025-10-09T21:51:46.439007Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:46.438986Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2242
2025-10-09T21:51:46.439008Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:46.439023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.439029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.439066Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|11d2a8dc-a161649b-aeb17cdd-22765959_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:46.439074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:46.439078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:46.439079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:46.439087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:46.439098Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:46.439114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:46.439232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.439239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.439251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:46.439315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2243 partition: 0 MaxSeqNo: 25 sessionId: src_id|11d2a8dc-a161649b-aeb17cdd-22765959_0
2025-10-09T21:51:46.439662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e38ac1f1-192bd1c0-39875b0e-5124549a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|11d2a8dc-a161649b-aeb17cdd-22765959_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:46.442104Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|11d2a8dc-a161649b-aeb17cdd-22765959_0] Write session: try to update token
2025-10-09T21:51:46.442121Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|11d2a8dc-a161649b-aeb17cdd-22765959_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:46.453508Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:46.453523Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:46.453525Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:46.453529Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:46.453539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343042590554636:11987]
2025-10-09T21:51:46.453546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343042590554636:11987]
2025-10-09T21:51:46.453549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343042590554636:11987]
2025-10-09T21:51:46.453591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:46.453593Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:46.453598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.453599Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:46.453601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:46.453603Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:46.453737Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:46.454558Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2243 sessionId: src_id|11d2a8dc-a161649b-aeb17cdd-22765959_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:46.456985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:46.520570Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343042952405893:8652]
2025-10-09T21:51:46.520896Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 2242, pipe: [1:7559343042952405893:8652]
2025-10-09T21:51:46.597455Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:46.597767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.597782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.597822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343042952405893:8652], now have 1 active actors on pipe
2025-10-09T21:51:46.598020Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:46.598094Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:46.598094Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:46.598110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:46.598143Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343042952405893:8652]
2025-10-09T21:51:46.598177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2243 reason: error at write: Disconnected sessionId: src_id|11d2a8dc-a161649b-aeb17cdd-22765959_0
2025-10-09T21:51:46.598362Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2243 sessionId: src_id|11d2a8dc-a161649b-aeb17cdd-22765959_0 is DEAD
2025-10-09T21:51:46.598542Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11d2a8dc-a161649b-aeb17cdd-22765959_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:46.598555Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11d2a8dc-a161649b-aeb17cdd-22765959_0] Write session will restart in 0.008715s
2025-10-09T21:51:46.598587Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11d2a8dc-a161649b-aeb17cdd-22765959_0] Write session: Do CDS request
2025-10-09T21:51:46.598593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11d2a8dc-a161649b-aeb17cdd-22765959_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:46.604456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:46.605851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:46.606022Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:46.606052Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:46.606064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:46.606114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:46.606142Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343042590554716:11989]
2025-10-09T21:51:46.606615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:46.606896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:46.606945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:46.607248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:46.607296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.607301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.607302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.607303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.607304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.607305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.607306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.607307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.607309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.607314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.607324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.607329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.607333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.607336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.607340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.607345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.607351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.607355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.607363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.607371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.607377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.607383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.607390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.607395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.607400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.607405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.607412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.607417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.607422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.607430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.607437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.607441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.607446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.607455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.607459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.607465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.607495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:46.608567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|11d2a8dc-a161649b-aeb17cdd-22765959_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:46.608651Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:46.608669Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2244
2025-10-09T21:51:46.608767Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2244 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:46.608796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2244 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:46.608807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2244 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:46.608811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:46.609213Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2244 sessionId:  describe result for acl check
2025-10-09T21:51:46.609247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:46.609254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:46.609258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:46.609263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343042952405937:8583] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:46.609675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343042952405937:8583] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:46.611489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343042952405937:8583] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:46.613783Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343042952405937:8583] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:46.613796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343042952405937:8583] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:46.613798Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343042952405937:8583] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:46.613805Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2244 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:46.614203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:46.615210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:46.615223Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:46.615225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:46.615230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:46.615238Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2243 [2:7559343042590554716:11989]
2025-10-09T21:51:46.615255Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:46.615272Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:46.615280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:46.615283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:46.615288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:46.615305Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:46.615315Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:46.615323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:46.615378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.615384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.615385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.615388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:46.615453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:46.615460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.615471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.615484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.615504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:46.615528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343042952405972:8583], now have 1 active actors on pipe
2025-10-09T21:51:46.615535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343042952405896:2432], now have 1 active actors on pipe
2025-10-09T21:51:46.615543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343042590554711:11989]
2025-10-09T21:51:46.615547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343042590554711:11989]
2025-10-09T21:51:46.615549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343042590554711:11989]
2025-10-09T21:51:46.615561Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:46.615571Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:46.615578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:46.615562Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2243
2025-10-09T21:51:46.615588Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:46.615565Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2243
2025-10-09T21:51:46.615592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:46.615593Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:46.615595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:46.615637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.615646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.615659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343042590554711:11989]
2025-10-09T21:51:46.615671Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:46.615671Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:46.615703Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|78adb22a-5b3f115c-27d83b29-447621ba_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:46.615714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:46.615721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:46.615723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:46.615735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:46.615748Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:46.615774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:46.615871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.615879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.615892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:46.615954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2244 partition: 0 MaxSeqNo: 25 sessionId: src_id|78adb22a-5b3f115c-27d83b29-447621ba_0
2025-10-09T21:51:46.616313Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|11d2a8dc-a161649b-aeb17cdd-22765959_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|78adb22a-5b3f115c-27d83b29-447621ba_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:46.618796Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78adb22a-5b3f115c-27d83b29-447621ba_0] Write session: try to update token
2025-10-09T21:51:46.618815Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78adb22a-5b3f115c-27d83b29-447621ba_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:46.631396Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:46.631418Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:46.631421Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:46.631427Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:46.631438Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343042590554711:11989]
2025-10-09T21:51:46.631444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343042590554711:11989]
2025-10-09T21:51:46.631450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343042590554711:11989]
2025-10-09T21:51:46.631494Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:46.631497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:46.631500Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:46.631503Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:46.631506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.631509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:46.631679Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:46.631939Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2244 sessionId: src_id|78adb22a-5b3f115c-27d83b29-447621ba_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:46.634273Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:46.698237Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343042952405980:8652]
2025-10-09T21:51:46.698551Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 2243, pipe: [1:7559343042952405980:8652]
2025-10-09T21:51:46.781402Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:46.781670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.781684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.781715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343042952405980:8652], now have 1 active actors on pipe
2025-10-09T21:51:46.781965Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:46.781997Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:46.782002Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343042952405980:8652]
2025-10-09T21:51:46.782012Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:46.782019Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:46.782097Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2244 reason: error at write: Disconnected sessionId: src_id|78adb22a-5b3f115c-27d83b29-447621ba_0
2025-10-09T21:51:46.782271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2244 sessionId: src_id|78adb22a-5b3f115c-27d83b29-447621ba_0 is DEAD
2025-10-09T21:51:46.782414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78adb22a-5b3f115c-27d83b29-447621ba_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:46.782426Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78adb22a-5b3f115c-27d83b29-447621ba_0] Write session will restart in 0.005000s
2025-10-09T21:51:46.782455Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78adb22a-5b3f115c-27d83b29-447621ba_0] Write session: Do CDS request
2025-10-09T21:51:46.782461Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78adb22a-5b3f115c-27d83b29-447621ba_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:46.788485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78adb22a-5b3f115c-27d83b29-447621ba_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:46.788590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:46.788608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2245
2025-10-09T21:51:46.788736Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2245 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:46.788779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2245 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:46.788790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2245 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:46.788794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:46.789213Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2245 sessionId:  describe result for acl check
2025-10-09T21:51:46.789248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:46.789254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:46.789256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:46.789260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343042952406021:8556] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:46.789729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343042952406021:8556] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:46.789753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:46.791242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:46.791445Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:46.791455Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:46.791465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:46.791516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:46.791561Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343042590554790:11991]
2025-10-09T21:51:46.791763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343042952406021:8556] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:46.792218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:46.792569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:46.792629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:46.792960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:46.793012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.793018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.793019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.793019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.793020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.793021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.793022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.793023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.793023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.793027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.793034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.793036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.793039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.793041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.793043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.793048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.793051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.793053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.793058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.793063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.793067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.793071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.793074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.793077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.793080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.793083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.793086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.793089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.793092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.793095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.793099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.793103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.793113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.793118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.793122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.793125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.793144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:46.796477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343042952406021:8556] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:46.796493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343042952406021:8556] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:46.796496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343042952406021:8556] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:46.796505Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2245 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:46.799811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:46.800999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:46.801014Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:46.801017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:46.801020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:46.801026Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2244 [2:7559343042590554790:11991]
2025-10-09T21:51:46.801039Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:46.801053Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:46.801060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:46.801067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:46.801068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:46.801082Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:46.801090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:46.801095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:46.801143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.801149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.801151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.801153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:46.801212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:46.801220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.801227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.801237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.801246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:46.801258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343042590554785:11991]
2025-10-09T21:51:46.801260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343042590554785:11991]
2025-10-09T21:51:46.801262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343042590554785:11991]
2025-10-09T21:51:46.801269Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:46.801284Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:46.801286Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:46.801291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:46.801294Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:46.801298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:46.801300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:46.801309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343042952406061:8556], now have 1 active actors on pipe
2025-10-09T21:51:46.801312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343042952405985:2432], now have 1 active actors on pipe
2025-10-09T21:51:46.801352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343042590554785:11991]
2025-10-09T21:51:46.801366Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:46.801383Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:46.801397Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2244
2025-10-09T21:51:46.801401Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2244
2025-10-09T21:51:46.801449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.801463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.801492Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|92551b3b-53cf65b0-557e1744-ac673af5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:46.801503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:46.801506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:46.801508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:46.801515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:46.801523Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:46.801534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:46.801710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.801725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.801750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:46.801861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2245 partition: 0 MaxSeqNo: 25 sessionId: src_id|92551b3b-53cf65b0-557e1744-ac673af5_0
2025-10-09T21:51:46.802224Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78adb22a-5b3f115c-27d83b29-447621ba_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|92551b3b-53cf65b0-557e1744-ac673af5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:46.804595Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92551b3b-53cf65b0-557e1744-ac673af5_0] Write session: try to update token
2025-10-09T21:51:46.804612Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92551b3b-53cf65b0-557e1744-ac673af5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:46.813243Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:46.813259Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:46.813269Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343042590554785:11991]
2025-10-09T21:51:46.813315Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:46.813343Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:46.813521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:46.813668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:46.813828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:46.813974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:46.814232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:46.814276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:46.817887Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2245 sessionId: src_id|92551b3b-53cf65b0-557e1744-ac673af5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:46.820250Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:46.820473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.820494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.820529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:46.820613Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:46.826087Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:46.826098Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:46.826100Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:46.826102Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:46.826111Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343042590554785:11991]
2025-10-09T21:51:46.826117Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343042590554785:11991]
2025-10-09T21:51:46.826128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343042590554785:11991]
2025-10-09T21:51:46.826172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:46.826174Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:46.826180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.826181Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:46.826182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:46.826183Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:46.826337Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:46.882059Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343042952406070:8652]
2025-10-09T21:51:46.882370Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 2244, pipe: [1:7559343042952406070:8652]
2025-10-09T21:51:46.975212Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:46.982800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:46.982834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:46.983007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:46.983073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:46.983142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:46.983205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:46.983253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:46.983267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:46.983336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:46.983397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:46.983459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:46.983537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:46.983579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:46.983588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:46.983649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:46.983708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:46.983769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:46.983830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:46.983874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:46.983884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:46.983946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:46.984011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:46.984072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:46.984138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:46.984191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:46.984202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:46.984270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:46.984329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:46.984389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:46.984441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:46.984482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:46.984489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:46.984543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:46.984594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:46.984646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:46.984699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:46.984740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:46.984747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:46.984809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:46.984873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:46.984938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:46.984995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:46.985036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:46.985048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:46.985102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:46.985154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:46.985204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:46.985255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:46.985293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:46.985299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:46.985363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:46.985413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:46.985473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:46.985525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:46.985593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:46.985599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:46.985653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:46.985700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:46.985752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:46.985808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:46.985846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:46.985853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:46.985910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343042952406070:8652], now have 1 active actors on pipe
2025-10-09T21:51:46.986185Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343042952406070:8652]
2025-10-09T21:51:46.986229Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:46.986421Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:46.986434Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:46.986459Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2245 reason: error at write: Disconnected sessionId: src_id|92551b3b-53cf65b0-557e1744-ac673af5_0
2025-10-09T21:51:46.986612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2245 sessionId: src_id|92551b3b-53cf65b0-557e1744-ac673af5_0 is DEAD
2025-10-09T21:51:46.986766Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92551b3b-53cf65b0-557e1744-ac673af5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:46.986777Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92551b3b-53cf65b0-557e1744-ac673af5_0] Write session will restart in 0.008728s
2025-10-09T21:51:46.986801Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92551b3b-53cf65b0-557e1744-ac673af5_0] Write session: Do CDS request
2025-10-09T21:51:46.986807Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92551b3b-53cf65b0-557e1744-ac673af5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:46.986786Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:46.995368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:46.996560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|92551b3b-53cf65b0-557e1744-ac673af5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:46.996599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:46.996620Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2246
2025-10-09T21:51:46.996702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:46.996742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2246 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:46.996767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2246 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:46.996775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2246 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:46.996780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:46.996913Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:46.996923Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:46.996935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:46.996984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:46.997029Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343042590554871:11995]
2025-10-09T21:51:46.997127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2246 sessionId:  describe result for acl check
2025-10-09T21:51:46.997166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:46.997171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:46.997172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:46.997176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343042952406115:8553] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:46.997617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:46.997668Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343042952406115:8553] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:46.997952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:46.997996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:46.998327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:46.998388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.998392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.998393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.998396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.998398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.998399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.998399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.998400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.998402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.998406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.998416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.998419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.998423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.998427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.998430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.998434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.998438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.998443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.998450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.998458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:46.998469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.998477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:46.998485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.998491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:46.998499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.998504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:46.998512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.998518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:46.998526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.998532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:46.998538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.998542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:46.998547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.998568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:46.998578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.998584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:46.998613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:47.000081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343042952406115:8553] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:47.004380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343042952406115:8553] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:47.004396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343042952406115:8553] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:47.004400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343042952406115:8553] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:47.004409Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2246 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:47.004801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:47.005838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:47.005855Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:47.005858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:47.005862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:47.005869Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2245 [2:7559343042590554871:11995]
2025-10-09T21:51:47.005881Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:47.005898Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:47.005906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:47.005910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:47.005913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:47.005932Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:47.005946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:47.005954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:47.006008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.006016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.006017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.006020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:47.006109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:47.006115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.006128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.006144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.006166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:47.006193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343042952406076:2432], now have 1 active actors on pipe
2025-10-09T21:51:47.006201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343047247373447:8553], now have 1 active actors on pipe
2025-10-09T21:51:47.006208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343042590554866:11995]
2025-10-09T21:51:47.006211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343042590554866:11995]
2025-10-09T21:51:47.006214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343042590554866:11995]
2025-10-09T21:51:47.006228Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:47.006245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:47.006259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:47.006264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:47.006268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:47.006270Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:47.006273Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:47.006279Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2245
2025-10-09T21:51:47.006296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343042590554866:11995]
2025-10-09T21:51:47.006286Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2245
2025-10-09T21:51:47.006304Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:47.006325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:47.006339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.006344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.006374Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|36d546cd-492c717a-2e72f3ce-b6575f7b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:47.006388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:47.006396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:47.006398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:47.006407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:47.006421Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:47.006439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:47.006589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.006598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.006612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:47.006705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2246 partition: 0 MaxSeqNo: 25 sessionId: src_id|36d546cd-492c717a-2e72f3ce-b6575f7b_0
2025-10-09T21:51:47.007032Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|92551b3b-53cf65b0-557e1744-ac673af5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|36d546cd-492c717a-2e72f3ce-b6575f7b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:47.009429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36d546cd-492c717a-2e72f3ce-b6575f7b_0] Write session: try to update token
2025-10-09T21:51:47.009445Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36d546cd-492c717a-2e72f3ce-b6575f7b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:47.020545Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:47.020565Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:47.020569Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:47.020571Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:47.020581Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343042590554866:11995]
2025-10-09T21:51:47.020587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343042590554866:11995]
2025-10-09T21:51:47.020595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343042590554866:11995]
2025-10-09T21:51:47.020644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:47.020653Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:47.020657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.020660Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:47.020660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:47.020665Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:47.020838Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:47.022781Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2246 sessionId: src_id|36d546cd-492c717a-2e72f3ce-b6575f7b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:47.025157Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:47.086336Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343047247373455:8652]
2025-10-09T21:51:47.086649Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 2245, pipe: [1:7559343047247373455:8652]
2025-10-09T21:51:47.173318Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:47.173671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.173687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.173737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343047247373455:8652], now have 1 active actors on pipe
2025-10-09T21:51:47.173970Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:47.174087Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:47.174102Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343047247373455:8652]
2025-10-09T21:51:47.174177Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:47.174254Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2246 reason: status is not ok: tablet will be restarted right now sessionId: src_id|36d546cd-492c717a-2e72f3ce-b6575f7b_0
2025-10-09T21:51:47.174426Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2246 sessionId: src_id|36d546cd-492c717a-2e72f3ce-b6575f7b_0 is DEAD
2025-10-09T21:51:47.174581Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36d546cd-492c717a-2e72f3ce-b6575f7b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:47.174591Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36d546cd-492c717a-2e72f3ce-b6575f7b_0] Write session will restart in 0.008428s
2025-10-09T21:51:47.174616Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36d546cd-492c717a-2e72f3ce-b6575f7b_0] Write session: Do CDS request
2025-10-09T21:51:47.174623Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36d546cd-492c717a-2e72f3ce-b6575f7b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:47.182099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:47.183510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:47.183725Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:47.183737Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:47.183749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:47.183815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:47.183873Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343046885522244:11997]
2025-10-09T21:51:47.184480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36d546cd-492c717a-2e72f3ce-b6575f7b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:47.184542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:47.184601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:47.184617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2247
2025-10-09T21:51:47.184732Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2247 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:47.184758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2247 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:47.184763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2247 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:47.184765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:47.184869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:47.184913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:47.185112Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2247 sessionId:  describe result for acl check
2025-10-09T21:51:47.185151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:47.185158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:47.185160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:47.185164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343047247373500:8527] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:47.185241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:47.185289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.185296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.185297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.185299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.185300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.185301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.185303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.185304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.185305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.185310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.185321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.185335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.185340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.185344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.185349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.185353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.185357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.185361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.185367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.185376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.185382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.185387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.185393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.185398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.185407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.185412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.185422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.185433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.185438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.185443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.185452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.185461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.185471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.185478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.185486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.185491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.185524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:47.185756Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343047247373500:8527] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:47.187891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343047247373500:8527] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:47.191341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343047247373500:8527] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:47.191357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343047247373500:8527] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:47.191361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343047247373500:8527] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:47.191370Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2247 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:47.193006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:47.193960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:47.193974Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:47.193977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:47.193981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:47.193990Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2246 [2:7559343046885522244:11997]
2025-10-09T21:51:47.194005Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:47.194023Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:47.194025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:47.194027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:47.194044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:47.194057Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:47.194064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:47.194070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:47.194116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.194121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.194123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.194124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:47.194180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:47.194185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.194193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.194199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.194213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:47.194239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343047247373457:2432], now have 1 active actors on pipe
2025-10-09T21:51:47.194252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343046885522239:11997]
2025-10-09T21:51:47.194258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343046885522239:11997]
2025-10-09T21:51:47.194260Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343046885522239:11997]
2025-10-09T21:51:47.194278Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:47.194288Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:47.194291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:47.194292Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:47.194298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:47.194283Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2246
2025-10-09T21:51:47.194314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:47.194320Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:47.194325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343047247373537:8527], now have 1 active actors on pipe
2025-10-09T21:51:47.194338Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2246
2025-10-09T21:51:47.194358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343046885522239:11997]
2025-10-09T21:51:47.194370Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:47.194379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:47.194399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.194408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.194442Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|66f64add-44f86b83-62414418-6bdd59c8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:47.194457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:47.194461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:47.194464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:47.194474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:47.194488Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:47.194508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:47.194667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.194680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.194703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:47.194796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2247 partition: 0 MaxSeqNo: 25 sessionId: src_id|66f64add-44f86b83-62414418-6bdd59c8_0
2025-10-09T21:51:47.195156Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36d546cd-492c717a-2e72f3ce-b6575f7b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|66f64add-44f86b83-62414418-6bdd59c8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:47.203585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66f64add-44f86b83-62414418-6bdd59c8_0] Write session: try to update token
2025-10-09T21:51:47.203614Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66f64add-44f86b83-62414418-6bdd59c8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:47.211967Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:47.211983Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:47.211986Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:47.211990Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:47.212002Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343046885522239:11997]
2025-10-09T21:51:47.212008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343046885522239:11997]
2025-10-09T21:51:47.212011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343046885522239:11997]
2025-10-09T21:51:47.212059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:47.212065Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:47.212067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.212069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:47.212072Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:47.212076Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:47.212207Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:47.216587Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2247 sessionId: src_id|66f64add-44f86b83-62414418-6bdd59c8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:47.223638Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:47.274210Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343047247373547:8652]
2025-10-09T21:51:47.274458Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 2246, pipe: [1:7559343047247373547:8652]
2025-10-09T21:51:47.360063Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:47.360370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.360382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.360405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343047247373547:8652], now have 1 active actors on pipe
2025-10-09T21:51:47.360561Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:47.360630Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:47.360646Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:47.360700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2247 reason: error at write: Disconnected sessionId: src_id|66f64add-44f86b83-62414418-6bdd59c8_0
2025-10-09T21:51:47.360870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2247 sessionId: src_id|66f64add-44f86b83-62414418-6bdd59c8_0 is DEAD
2025-10-09T21:51:47.361013Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66f64add-44f86b83-62414418-6bdd59c8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:47.361023Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66f64add-44f86b83-62414418-6bdd59c8_0] Write session will restart in 0.008078s
2025-10-09T21:51:47.361052Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66f64add-44f86b83-62414418-6bdd59c8_0] Write session: Do CDS request
2025-10-09T21:51:47.361056Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66f64add-44f86b83-62414418-6bdd59c8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:47.361105Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:47.361135Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343047247373547:8652]
2025-10-09T21:51:47.368640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:47.370566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:47.370603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66f64add-44f86b83-62414418-6bdd59c8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:47.370656Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:47.370672Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2248
2025-10-09T21:51:47.370762Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:47.370773Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:47.370788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:47.370817Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2248 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:47.370842Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2248 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:47.370846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2248 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:47.370852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:47.370840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:47.370881Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343046885522335:12005]
2025-10-09T21:51:47.371168Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2248 sessionId:  describe result for acl check
2025-10-09T21:51:47.371195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:47.371196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:47.371197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:47.371203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343047247373609:8497] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:47.371288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:47.371560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:47.371597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:47.371638Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343047247373609:8497] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:47.371859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:47.371913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.371918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.371920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.371921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.371922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.371924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.371925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.371926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.371928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.371932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.371942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.371951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.371961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.371967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.371975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.371978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.371986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.371989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.372001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.372013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.372019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.372029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.372038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.372043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.372052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.372062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.372068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.372075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.372084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.372090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.372099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.372104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.372108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.372114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.372120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.372129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.372162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:47.373371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343047247373609:8497] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:47.378402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343047247373609:8497] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:47.378419Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343047247373609:8497] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:47.378422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343047247373609:8497] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:47.378429Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2248 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:47.378907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:47.379965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:47.379983Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:47.379986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:47.379990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:47.379996Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2247 [2:7559343046885522335:12005]
2025-10-09T21:51:47.380010Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:47.380028Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:47.380031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:47.380034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:47.380037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:47.380055Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:47.380067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:47.380078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:47.380137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.380144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.380146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.380149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:47.380222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:47.380232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.380244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.380256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.380271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:47.380292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343047247373568:2432], now have 1 active actors on pipe
2025-10-09T21:51:47.380301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343047247373647:8497], now have 1 active actors on pipe
2025-10-09T21:51:47.380309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343046885522330:12005]
2025-10-09T21:51:47.380311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343046885522330:12005]
2025-10-09T21:51:47.380314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343046885522330:12005]
2025-10-09T21:51:47.380330Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:47.380351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:47.380350Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:47.380354Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:47.380355Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:47.380366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:47.380369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:47.380381Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2247
2025-10-09T21:51:47.380402Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343046885522330:12005]
2025-10-09T21:51:47.380413Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:47.380414Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:47.380422Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2247
2025-10-09T21:51:47.380525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.380536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.380563Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bef55b44-d31f74c8-b32ced18-46e895be_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:47.380575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:47.380584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:47.380586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:47.380597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:47.380611Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:47.380629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:47.380952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.380962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.380975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:47.381066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2248 partition: 0 MaxSeqNo: 25 sessionId: src_id|bef55b44-d31f74c8-b32ced18-46e895be_0
2025-10-09T21:51:47.381417Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66f64add-44f86b83-62414418-6bdd59c8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bef55b44-d31f74c8-b32ced18-46e895be_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:47.383747Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bef55b44-d31f74c8-b32ced18-46e895be_0] Write session: try to update token
2025-10-09T21:51:47.383761Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bef55b44-d31f74c8-b32ced18-46e895be_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:47.395583Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:47.395601Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:47.395603Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:47.395604Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:47.395611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343046885522330:12005]
2025-10-09T21:51:47.395614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343046885522330:12005]
2025-10-09T21:51:47.395618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343046885522330:12005]
2025-10-09T21:51:47.395654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:47.395663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.395668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:47.395694Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:47.395707Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:47.395710Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:47.395835Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:47.396359Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2248 sessionId: src_id|bef55b44-d31f74c8-b32ced18-46e895be_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:47.398594Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:47.461136Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343047247373668:8652]
2025-10-09T21:51:47.461451Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 2247, pipe: [1:7559343047247373668:8652]
2025-10-09T21:51:47.544824Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:47.545363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.545378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.545413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343047247373668:8652], now have 1 active actors on pipe
2025-10-09T21:51:47.545701Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343047247373668:8652]
2025-10-09T21:51:47.545746Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:47.545728Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:47.545790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:47.545801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:47.545884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2248 reason: error at write: Disconnected sessionId: src_id|bef55b44-d31f74c8-b32ced18-46e895be_0
2025-10-09T21:51:47.546064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2248 sessionId: src_id|bef55b44-d31f74c8-b32ced18-46e895be_0 is DEAD
2025-10-09T21:51:47.546217Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bef55b44-d31f74c8-b32ced18-46e895be_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:47.546227Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bef55b44-d31f74c8-b32ced18-46e895be_0] Write session will restart in 0.006885s
2025-10-09T21:51:47.546249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bef55b44-d31f74c8-b32ced18-46e895be_0] Write session: Do CDS request
2025-10-09T21:51:47.546254Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bef55b44-d31f74c8-b32ced18-46e895be_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:47.553804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:47.554495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bef55b44-d31f74c8-b32ced18-46e895be_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:47.554552Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:47.554567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2249
2025-10-09T21:51:47.554940Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2249 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:47.554968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2249 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:47.554976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2249 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:47.554979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:47.555327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:47.555369Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2249 sessionId:  describe result for acl check
2025-10-09T21:51:47.555408Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:47.555414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:47.555416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:47.555423Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343047247373714:8471] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:47.555522Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:47.555533Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:47.555546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:47.555615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:47.555650Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343046885522440:12013]
2025-10-09T21:51:47.555987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343047247373714:8471] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:47.556198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:47.556470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:47.556512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:47.556742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:47.556778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.556783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.556784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.556785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.556785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.556786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.556787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.556787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.556788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.556792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.556799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.556802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.556807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.556810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.556814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.556818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.556821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.556824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.556832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.556840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.556846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.556851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.556855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.556860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.556864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.556869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.556875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.556879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.556883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.556888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.556892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.556897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.556901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.556905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.556910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.556916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.556943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:47.558298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343047247373714:8471] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:47.562996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343047247373714:8471] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:47.563012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343047247373714:8471] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:47.563017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343047247373714:8471] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:47.563026Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2249 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:47.563346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:47.564476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:47.564491Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:47.564493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:47.564498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:47.564505Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2248 [2:7559343046885522440:12013]
2025-10-09T21:51:47.564518Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:47.564535Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:47.564538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:47.564541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:47.564543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:47.564563Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:47.564574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:47.564581Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:47.564646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.564654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.564656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.564659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:47.564731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:47.564740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.564818Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2248
2025-10-09T21:51:47.564829Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2248
2025-10-09T21:51:47.564851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.564870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.564895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:47.564922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343047247373671:2432], now have 1 active actors on pipe
2025-10-09T21:51:47.564932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343047247373753:8471], now have 1 active actors on pipe
2025-10-09T21:51:47.564946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.564957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.564980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343046885522434:12013]
2025-10-09T21:51:47.564988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343046885522434:12013]
2025-10-09T21:51:47.564990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343046885522434:12013]
2025-10-09T21:51:47.565003Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:47.565023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:47.565025Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:47.565028Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:47.565030Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:47.565044Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:47.565047Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:47.565066Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b1274053-b0b9cd0a-771adf58-bd33739_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:47.565074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:47.565078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:47.565081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:47.565092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:47.565108Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:47.565144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343046885522434:12013]
2025-10-09T21:51:47.565159Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:47.565171Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:47.565177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:47.565299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.565307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.565332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:47.565429Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2249 partition: 0 MaxSeqNo: 25 sessionId: src_id|b1274053-b0b9cd0a-771adf58-bd33739_0
2025-10-09T21:51:47.565805Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bef55b44-d31f74c8-b32ced18-46e895be_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b1274053-b0b9cd0a-771adf58-bd33739_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:47.568247Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1274053-b0b9cd0a-771adf58-bd33739_0] Write session: try to update token
2025-10-09T21:51:47.568267Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1274053-b0b9cd0a-771adf58-bd33739_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:47.579749Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:47.579767Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:47.579769Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:47.579771Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:47.579781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343046885522434:12013]
2025-10-09T21:51:47.579786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343046885522434:12013]
2025-10-09T21:51:47.579790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343046885522434:12013]
2025-10-09T21:51:47.579837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:47.579845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.579848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:47.579871Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:47.579878Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:47.579882Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:47.579989Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:47.580998Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2249 sessionId: src_id|b1274053-b0b9cd0a-771adf58-bd33739_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:47.583513Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:47.645963Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343047247373760:8652]
2025-10-09T21:51:47.646300Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 2248, pipe: [1:7559343047247373760:8652]
2025-10-09T21:51:47.729983Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:47.730293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.730308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.730355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343047247373760:8652], now have 1 active actors on pipe
2025-10-09T21:51:47.730565Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:47.730623Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:47.730635Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343047247373760:8652]
2025-10-09T21:51:47.730670Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:47.730745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2249 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b1274053-b0b9cd0a-771adf58-bd33739_0
2025-10-09T21:51:47.730942Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2249 sessionId: src_id|b1274053-b0b9cd0a-771adf58-bd33739_0 is DEAD
2025-10-09T21:51:47.731084Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1274053-b0b9cd0a-771adf58-bd33739_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:47.731095Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1274053-b0b9cd0a-771adf58-bd33739_0] Write session will restart in 0.009091s
2025-10-09T21:51:47.731121Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1274053-b0b9cd0a-771adf58-bd33739_0] Write session: Do CDS request
2025-10-09T21:51:47.731128Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1274053-b0b9cd0a-771adf58-bd33739_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:47.738867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:47.740417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:47.740611Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:47.740621Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:47.740632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:47.740681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:47.740727Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343046885522519:12015]
2025-10-09T21:51:47.741365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:47.741538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1274053-b0b9cd0a-771adf58-bd33739_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:47.741657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:47.741673Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2250
2025-10-09T21:51:47.741695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:47.741739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:47.741790Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2250 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:47.741830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2250 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:47.741843Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2250 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:47.741847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:47.742141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:47.742210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.742218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.742219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.742221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.742222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.742223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.742224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.742235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2250 sessionId:  describe result for acl check
2025-10-09T21:51:47.742226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.742265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:47.742266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:47.742267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:47.742272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343047247373809:8464] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:47.742227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.742232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.742242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.742246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.742252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.742255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.742259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.742263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.742267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.742270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.742277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.742285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.742290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.742296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.742300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.742306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.742312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.742317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.742322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.742327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.742332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.742337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.742342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.742348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.742352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.742358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.742363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.742369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.742400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:47.742845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343047247373809:8464] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:47.744986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343047247373809:8464] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:47.748069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343047247373809:8464] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:47.748082Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343047247373809:8464] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:47.748088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343047247373809:8464] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:47.748114Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2250 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:47.748700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:47.749961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:47.749977Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:47.749980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:47.749983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:47.749989Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2249 [2:7559343046885522519:12015]
2025-10-09T21:51:47.750003Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:47.750018Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:47.750044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:47.750047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:47.750049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:47.750063Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:47.750069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:47.750080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:47.750142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.750149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.750150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.750152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:47.750222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:47.750231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.750241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.750252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.750266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:47.750286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343047247373763:2432], now have 1 active actors on pipe
2025-10-09T21:51:47.750294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343047247373845:8464], now have 1 active actors on pipe
2025-10-09T21:51:47.750300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343046885522514:12015]
2025-10-09T21:51:47.750308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343046885522514:12015]
2025-10-09T21:51:47.750310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343046885522514:12015]
2025-10-09T21:51:47.750323Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:47.750340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:47.750354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:47.750356Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:47.750357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:47.750359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:47.750362Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:47.750380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343046885522514:12015]
2025-10-09T21:51:47.750387Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:47.750390Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:47.750414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.750418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.750357Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2249
2025-10-09T21:51:47.750372Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2249
2025-10-09T21:51:47.750451Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|33974e01-771d5ff3-1ab1c6c3-fc28a10f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:47.750462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:47.750465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:47.750468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:47.750476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:47.750485Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:47.750497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:47.750693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.750700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.750713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:47.750808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2250 partition: 0 MaxSeqNo: 25 sessionId: src_id|33974e01-771d5ff3-1ab1c6c3-fc28a10f_0
2025-10-09T21:51:47.751230Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1274053-b0b9cd0a-771adf58-bd33739_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|33974e01-771d5ff3-1ab1c6c3-fc28a10f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:47.753648Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33974e01-771d5ff3-1ab1c6c3-fc28a10f_0] Write session: try to update token
2025-10-09T21:51:47.753662Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33974e01-771d5ff3-1ab1c6c3-fc28a10f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:47.764528Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:47.764541Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:47.764544Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:47.764545Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:47.764554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343046885522514:12015]
2025-10-09T21:51:47.764558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343046885522514:12015]
2025-10-09T21:51:47.764561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343046885522514:12015]
2025-10-09T21:51:47.764599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:47.764606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.764613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:47.764613Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:47.764620Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:47.764625Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:47.764746Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:47.766397Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2250 sessionId: src_id|33974e01-771d5ff3-1ab1c6c3-fc28a10f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:47.769438Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:47.830776Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343047247373852:8652]
2025-10-09T21:51:47.831100Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 2249, pipe: [1:7559343047247373852:8652]
2025-10-09T21:51:47.909961Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:47.910344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.910366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.910405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343047247373852:8652], now have 1 active actors on pipe
2025-10-09T21:51:47.910670Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:47.910671Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:47.910688Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:47.910729Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343047247373852:8652]
2025-10-09T21:51:47.910747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2250 reason: error at write: Disconnected sessionId: src_id|33974e01-771d5ff3-1ab1c6c3-fc28a10f_0
2025-10-09T21:51:47.910902Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2250 sessionId: src_id|33974e01-771d5ff3-1ab1c6c3-fc28a10f_0 is DEAD
2025-10-09T21:51:47.910957Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:47.911061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33974e01-771d5ff3-1ab1c6c3-fc28a10f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:47.911073Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33974e01-771d5ff3-1ab1c6c3-fc28a10f_0] Write session will restart in 0.005478s
2025-10-09T21:51:47.911099Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33974e01-771d5ff3-1ab1c6c3-fc28a10f_0] Write session: Do CDS request
2025-10-09T21:51:47.911107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33974e01-771d5ff3-1ab1c6c3-fc28a10f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:47.917488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|33974e01-771d5ff3-1ab1c6c3-fc28a10f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:47.917541Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:47.917555Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2251
2025-10-09T21:51:47.917684Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2251 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:47.917716Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2251 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:47.917725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2251 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:47.917728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:47.918129Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2251 sessionId:  describe result for acl check
2025-10-09T21:51:47.918162Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:47.918163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:47.918164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:47.918169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343047247373898:8433] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:47.918679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343047247373898:8433] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:47.919041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:47.920453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:47.920630Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:47.920642Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:47.920650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:47.920691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:47.920699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343047247373898:8433] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:47.920726Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343046885522605:12019]
2025-10-09T21:51:47.921181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:47.921528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:47.921579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:47.921868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:47.921922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.921930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.921931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.921932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.921933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.921933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.921934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.921935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.921936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.921940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.921946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.921948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.921951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.921953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.921955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.921957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.921959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.921961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.921967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.921972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:47.921976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.921979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:47.921985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.921988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:47.921991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.921996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:47.922001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.922006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:47.922010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.922016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:47.922022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.922027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.922046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.922052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.922057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.922063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.922082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:47.926496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343047247373898:8433] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:47.926515Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343047247373898:8433] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:47.926519Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343047247373898:8433] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:47.926529Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2251 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:47.928617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:47.929983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:47.929994Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:47.929997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:47.930000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:47.930006Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2250 [2:7559343046885522605:12019]
2025-10-09T21:51:47.930017Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:47.930050Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:47.930057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:47.930062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:47.930063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:47.930075Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:47.930079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:47.930084Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:47.930132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.930141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.930142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.930145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:47.930202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:47.930209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.930216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:47.930223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:47.930232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:47.930244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343046885522600:12019]
2025-10-09T21:51:47.930249Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343046885522600:12019]
2025-10-09T21:51:47.930250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343046885522600:12019]
2025-10-09T21:51:47.930257Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:47.930267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:47.930275Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:47.930278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:47.930279Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:47.930281Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:47.930281Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:47.930288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343047247373856:2432], now have 1 active actors on pipe
2025-10-09T21:51:47.930316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343047247373940:8433], now have 1 active actors on pipe
2025-10-09T21:51:47.930317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343046885522600:12019]
2025-10-09T21:51:47.930323Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:47.930332Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:47.930398Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2250
2025-10-09T21:51:47.930403Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2250
2025-10-09T21:51:47.930457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.930464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.930481Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dfee440-8ca740ca-3aa0298c-88f23729_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:47.930488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:47.930490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:47.930492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:47.930499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:47.930510Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:47.930521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:47.930647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.930652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.930661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:47.930752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2251 partition: 0 MaxSeqNo: 25 sessionId: src_id|dfee440-8ca740ca-3aa0298c-88f23729_0
2025-10-09T21:51:47.931028Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|33974e01-771d5ff3-1ab1c6c3-fc28a10f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dfee440-8ca740ca-3aa0298c-88f23729_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:47.933589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfee440-8ca740ca-3aa0298c-88f23729_0] Write session: try to update token
2025-10-09T21:51:47.933606Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfee440-8ca740ca-3aa0298c-88f23729_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:47.942209Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:47.942222Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:47.942230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343046885522600:12019]
2025-10-09T21:51:47.942262Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:47.942270Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:47.942436Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:47.942587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:47.942750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:47.942900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:47.943142Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:47.943173Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:47.946630Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2251 sessionId: src_id|dfee440-8ca740ca-3aa0298c-88f23729_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:47.948832Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:47.949037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:47.949056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:47.949095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:47.949191Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:47.964130Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:47.964141Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:47.964143Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:47.964144Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:47.964155Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343046885522600:12019]
2025-10-09T21:51:47.964157Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343046885522600:12019]
2025-10-09T21:51:47.964160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343046885522600:12019]
2025-10-09T21:51:47.964195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:47.964204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:47.964203Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:47.964205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:47.964210Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:47.964215Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:47.964367Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:48.011009Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343051542341243:8652]
2025-10-09T21:51:48.011377Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 2250, pipe: [1:7559343051542341243:8652]
2025-10-09T21:51:48.124472Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:48.134146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:48.134171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:48.134337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:48.134410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:48.134480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:48.134547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:48.134872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:48.134882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:48.134966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:48.135033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:48.135117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:48.135199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:48.135258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:48.135266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:48.135339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:48.135406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:48.135474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:48.135545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:48.135597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:48.135604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:48.135689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:48.135764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:48.135834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:48.135908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:48.135962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:48.135968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:48.136042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:48.136108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:48.136195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:48.136266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:48.136386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:48.136394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:48.136473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:48.136539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:48.136607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:48.136687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:48.136744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:48.136750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:48.136817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:48.136880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:48.136951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:48.137033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:48.137086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:48.137093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:48.137165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:48.137233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:48.137300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:48.137381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:48.137435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:48.137442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:48.137514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:48.137580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:48.137648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:48.137718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:48.137776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:48.137783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:48.137860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:48.137926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:48.137996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:48.138072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:48.138119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:48.138126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:48.138187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343051542341243:8652], now have 1 active actors on pipe
2025-10-09T21:51:48.138414Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:48.138432Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:48.138436Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:48.138477Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2251 reason: error at write: Disconnected sessionId: src_id|dfee440-8ca740ca-3aa0298c-88f23729_0
2025-10-09T21:51:48.138489Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343051542341243:8652]
2025-10-09T21:51:48.138639Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2251 sessionId: src_id|dfee440-8ca740ca-3aa0298c-88f23729_0 is DEAD
2025-10-09T21:51:48.138792Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfee440-8ca740ca-3aa0298c-88f23729_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:48.138803Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfee440-8ca740ca-3aa0298c-88f23729_0] Write session will restart in 0.006729s
2025-10-09T21:51:48.138831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfee440-8ca740ca-3aa0298c-88f23729_0] Write session: Do CDS request
2025-10-09T21:51:48.138836Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfee440-8ca740ca-3aa0298c-88f23729_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:48.138885Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:48.146482Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfee440-8ca740ca-3aa0298c-88f23729_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:48.146543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:48.146587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:48.146606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2252
2025-10-09T21:51:48.146753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2252 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:48.146787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2252 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:48.146799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2252 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:48.146807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:48.147216Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2252 sessionId:  describe result for acl check
2025-10-09T21:51:48.147259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:48.147265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:48.147267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:48.147273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343051542341290:8440] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:48.147819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343051542341290:8440] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:48.148088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:48.148278Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:48.148293Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:48.148302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:48.148346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:48.148379Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343051180489983:12021]
2025-10-09T21:51:48.148822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:48.149054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:48.149085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:48.149355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:48.149405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.149410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.149411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.149412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.149413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.149413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.149414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.149415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.149415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.149418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.149427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.149431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.149435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.149439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.149443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.149446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.149449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.149453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.149459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.149468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.149474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.149478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.149484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.149487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.149491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.149494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.149497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.149500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.149503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.149506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.149510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.149514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.149517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.149520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.149523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.149526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.149544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:48.149898Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343051542341290:8440] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:48.155583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343051542341290:8440] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:48.155600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343051542341290:8440] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:48.155604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343051542341290:8440] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:48.155614Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2252 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:48.156100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:48.157380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:48.157395Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:48.157399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:48.157403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:48.157410Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2251 [2:7559343051180489983:12021]
2025-10-09T21:51:48.157421Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:48.157436Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:48.157437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:48.157440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:48.157442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:48.157454Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:48.157458Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:48.157463Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:48.157515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.157521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.157523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.157528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:48.157588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:48.157595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.157604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.157610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.157624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:48.157647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343051542341330:8440], now have 1 active actors on pipe
2025-10-09T21:51:48.157659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343051180489978:12021]
2025-10-09T21:51:48.157666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343051180489978:12021]
2025-10-09T21:51:48.157668Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343051180489978:12021]
2025-10-09T21:51:48.157683Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:48.157702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:48.157717Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:48.157720Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:48.157712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2251
2025-10-09T21:51:48.157736Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2251
2025-10-09T21:51:48.157807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343051542341246:2432], now have 1 active actors on pipe
2025-10-09T21:51:48.157817Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:48.157827Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:48.157833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:48.157873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:48.157881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:48.157898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343051180489978:12021]
2025-10-09T21:51:48.157910Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:48.157926Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:48.157933Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|84fca9b4-5eb28dd2-467fe651-97949658_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:48.157938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:48.157944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:48.157946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:48.157952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:48.157959Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:48.157979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:48.158153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:48.158164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:48.158174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:48.158281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2252 partition: 0 MaxSeqNo: 25 sessionId: src_id|84fca9b4-5eb28dd2-467fe651-97949658_0
2025-10-09T21:51:48.158624Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfee440-8ca740ca-3aa0298c-88f23729_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|84fca9b4-5eb28dd2-467fe651-97949658_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:48.161079Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|84fca9b4-5eb28dd2-467fe651-97949658_0] Write session: try to update token
2025-10-09T21:51:48.161094Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|84fca9b4-5eb28dd2-467fe651-97949658_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:48.172353Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:48.172370Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:48.172372Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:48.172375Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:48.172390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343051180489978:12021]
2025-10-09T21:51:48.172393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343051180489978:12021]
2025-10-09T21:51:48.172395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343051180489978:12021]
2025-10-09T21:51:48.172432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:48.172431Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:48.172436Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:48.172438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.172439Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:48.172440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:48.172573Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:48.174307Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2252 sessionId: src_id|84fca9b4-5eb28dd2-467fe651-97949658_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:48.176767Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:48.238804Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343051542341341:8652]
2025-10-09T21:51:48.239094Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 2251, pipe: [1:7559343051542341341:8652]
2025-10-09T21:51:48.325751Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:48.326135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:48.326147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:48.326181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343051542341341:8652], now have 1 active actors on pipe
2025-10-09T21:51:48.326408Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:48.326435Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:48.326445Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:48.326451Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:48.326493Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343051542341341:8652]
2025-10-09T21:51:48.326514Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2252 reason: error at write: Disconnected sessionId: src_id|84fca9b4-5eb28dd2-467fe651-97949658_0
2025-10-09T21:51:48.326670Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2252 sessionId: src_id|84fca9b4-5eb28dd2-467fe651-97949658_0 is DEAD
2025-10-09T21:51:48.326819Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84fca9b4-5eb28dd2-467fe651-97949658_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:48.326832Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84fca9b4-5eb28dd2-467fe651-97949658_0] Write session will restart in 0.008631s
2025-10-09T21:51:48.326856Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84fca9b4-5eb28dd2-467fe651-97949658_0] Write session: Do CDS request
2025-10-09T21:51:48.326862Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84fca9b4-5eb28dd2-467fe651-97949658_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:48.334071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:48.335308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:48.335472Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:48.335482Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:48.335490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:48.335528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:48.335561Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343051180490066:12023]
2025-10-09T21:51:48.335955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:48.336196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:48.336224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:48.336422Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|84fca9b4-5eb28dd2-467fe651-97949658_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:48.336449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:48.336486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.336492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.336493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.336494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.336494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.336495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.336495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.336496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.336496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.336500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.336507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.336509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.336511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.336514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.336516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.336518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.336519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.336521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.336527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.336532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.336536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.336539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.336546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.336549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.336552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.336555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.336558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.336561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.336564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.336566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.336570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.336573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.336576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.336579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.336583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.336586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.336603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:48.336630Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:48.336643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2253
2025-10-09T21:51:48.336755Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2253 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:48.336777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2253 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:48.336784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2253 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:48.336788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:48.337103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2253 sessionId:  describe result for acl check
2025-10-09T21:51:48.337133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:48.337138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:48.337141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:48.337146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343051542341398:8420] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:48.337563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343051542341398:8420] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:48.339245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343051542341398:8420] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:48.342357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343051542341398:8420] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:48.342369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343051542341398:8420] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:48.342372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343051542341398:8420] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:48.342378Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2253 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:48.342739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:48.343725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:48.343741Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:48.343743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:48.343746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:48.343751Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2252 [2:7559343051180490066:12023]
2025-10-09T21:51:48.343762Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:48.343777Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:48.343779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:48.343782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:48.343783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:48.343798Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:48.343805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:48.343811Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:48.343855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.343863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.343865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.343868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:48.343998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:48.344008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.344016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.344022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.344033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:48.344040Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2252
2025-10-09T21:51:48.344048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343051542341348:2432], now have 1 active actors on pipe
2025-10-09T21:51:48.344056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343051542341433:8420], now have 1 active actors on pipe
2025-10-09T21:51:48.344049Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2252
2025-10-09T21:51:48.344062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343051180490061:12023]
2025-10-09T21:51:48.344063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343051180490061:12023]
2025-10-09T21:51:48.344065Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343051180490061:12023]
2025-10-09T21:51:48.344072Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:48.344083Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:48.344096Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:48.344099Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:48.344099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:48.344101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:48.344102Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:48.344135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343051180490061:12023]
2025-10-09T21:51:48.344146Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:48.344152Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:48.344157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:48.344160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:48.344197Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1e1de9ff-41444fb1-56fd114-e8d090ed_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:48.344208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:48.344211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:48.344214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:48.344222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:48.344236Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:48.344259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:48.344393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:48.344400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:48.344410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:48.344469Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2253 partition: 0 MaxSeqNo: 25 sessionId: src_id|1e1de9ff-41444fb1-56fd114-e8d090ed_0
2025-10-09T21:51:48.344773Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|84fca9b4-5eb28dd2-467fe651-97949658_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1e1de9ff-41444fb1-56fd114-e8d090ed_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:48.347220Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1e1de9ff-41444fb1-56fd114-e8d090ed_0] Write session: try to update token
2025-10-09T21:51:48.347240Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1e1de9ff-41444fb1-56fd114-e8d090ed_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:48.358356Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:48.358376Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:48.358382Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:48.358384Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:48.358393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343051180490061:12023]
2025-10-09T21:51:48.358398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343051180490061:12023]
2025-10-09T21:51:48.358402Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343051180490061:12023]
2025-10-09T21:51:48.358441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:48.358447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.358449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:48.358456Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:48.358465Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:48.358469Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:48.358573Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:48.360666Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2253 sessionId: src_id|1e1de9ff-41444fb1-56fd114-e8d090ed_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:48.362969Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:48.426742Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343051542341454:8652]
2025-10-09T21:51:48.427070Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 2252, pipe: [1:7559343051542341454:8652]
2025-10-09T21:51:48.505147Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:48.505543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:48.505565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:48.505609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343051542341454:8652], now have 1 active actors on pipe
2025-10-09T21:51:48.505837Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:48.505884Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:48.505902Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343051542341454:8652]
2025-10-09T21:51:48.505938Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:48.506021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2253 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1e1de9ff-41444fb1-56fd114-e8d090ed_0
2025-10-09T21:51:48.506184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2253 sessionId: src_id|1e1de9ff-41444fb1-56fd114-e8d090ed_0 is DEAD
2025-10-09T21:51:48.506321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1e1de9ff-41444fb1-56fd114-e8d090ed_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:48.506344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1e1de9ff-41444fb1-56fd114-e8d090ed_0] Write session will restart in 0.005359s
2025-10-09T21:51:48.506371Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1e1de9ff-41444fb1-56fd114-e8d090ed_0] Write session: Do CDS request
2025-10-09T21:51:48.506378Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1e1de9ff-41444fb1-56fd114-e8d090ed_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:48.512509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1e1de9ff-41444fb1-56fd114-e8d090ed_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:48.512597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:48.512621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2254
2025-10-09T21:51:48.512745Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2254 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:48.512780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2254 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:48.512788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2254 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:48.512791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:48.513144Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2254 sessionId:  describe result for acl check
2025-10-09T21:51:48.513174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:48.513180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:48.513181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:48.513185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343051542341509:8392] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:48.513628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343051542341509:8392] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:48.514306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:48.516220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343051542341509:8392] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:48.517165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:48.517382Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:48.517393Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:48.517402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:48.517459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:48.517503Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343051180490188:12037]
2025-10-09T21:51:48.517958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:48.518295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:48.518331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:48.518577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:48.518627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.518635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.518636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.518638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.518640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.518641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.518643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.518644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.518645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.518648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.518655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.518657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.518660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.518662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.518664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.518666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.518668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.518670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.518675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.518680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.518683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.518687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.518690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.518694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.518696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.518699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.518703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.518706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.518721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.518724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.518728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.518732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.518735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.518739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.518744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.518747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.518764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:48.519057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343051542341509:8392] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:48.519069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343051542341509:8392] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:48.519072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343051542341509:8392] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:48.519079Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2254 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:48.524999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:48.526513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:48.526528Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:48.526531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:48.526534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:48.526539Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2253 [2:7559343051180490188:12037]
2025-10-09T21:51:48.526558Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:48.526577Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:48.526581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:48.526584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:48.526586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:48.526601Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:48.526607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:48.526625Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:48.526685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.526694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.526696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.526698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:48.526755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:48.526762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.526769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.526776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.526792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:48.526809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343051180490183:12037]
2025-10-09T21:51:48.526818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343051180490183:12037]
2025-10-09T21:51:48.526820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343051180490183:12037]
2025-10-09T21:51:48.526832Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:48.526842Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:48.526845Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:48.526852Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:48.526861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:48.526848Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:48.526871Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:48.526878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343051542341465:2432], now have 1 active actors on pipe
2025-10-09T21:51:48.526882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343051542341552:8392], now have 1 active actors on pipe
2025-10-09T21:51:48.526907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343051180490183:12037]
2025-10-09T21:51:48.526919Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:48.526929Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:48.526941Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2253
2025-10-09T21:51:48.526963Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2253
2025-10-09T21:51:48.527059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:48.527069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:48.527093Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|216a94d6-c51fa752-15890b72-d3a8197b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:48.527103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:48.527107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:48.527109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:48.527115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:48.527123Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:48.527136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:48.527333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:48.527342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:48.527355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:48.527467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2254 partition: 0 MaxSeqNo: 25 sessionId: src_id|216a94d6-c51fa752-15890b72-d3a8197b_0
2025-10-09T21:51:48.527822Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1e1de9ff-41444fb1-56fd114-e8d090ed_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|216a94d6-c51fa752-15890b72-d3a8197b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:48.530265Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|216a94d6-c51fa752-15890b72-d3a8197b_0] Write session: try to update token
2025-10-09T21:51:48.530287Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|216a94d6-c51fa752-15890b72-d3a8197b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:48.553760Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:48.553774Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:48.553790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343051180490183:12037]
2025-10-09T21:51:48.553840Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:48.556004Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:48.556218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:48.556393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:48.556557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:48.556729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:48.557750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:48.557795Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:48.557799Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:48.557800Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:48.557802Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:48.557809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343051180490183:12037]
2025-10-09T21:51:48.557812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343051180490183:12037]
2025-10-09T21:51:48.557817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343051180490183:12037]
2025-10-09T21:51:48.557854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:48.557867Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:48.557874Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:48.557877Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:48.557885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:48.557891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.557893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:48.558081Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:48.559368Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2254 sessionId: src_id|216a94d6-c51fa752-15890b72-d3a8197b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:48.561938Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:48.605962Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343051542341559:8652]
2025-10-09T21:51:48.606328Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 2253, pipe: [1:7559343051542341559:8652]
2025-10-09T21:51:48.711470Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:48.711893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:48.711904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:48.711920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343051542341559:8652], now have 1 active actors on pipe
2025-10-09T21:51:48.712196Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:48.712225Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343051542341559:8652]
2025-10-09T21:51:48.712235Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:48.712262Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:48.712346Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2254 reason: status is not ok: tablet will be restarted right now sessionId: src_id|216a94d6-c51fa752-15890b72-d3a8197b_0
2025-10-09T21:51:48.712522Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2254 sessionId: src_id|216a94d6-c51fa752-15890b72-d3a8197b_0 is DEAD
2025-10-09T21:51:48.712707Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|216a94d6-c51fa752-15890b72-d3a8197b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:48.712719Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|216a94d6-c51fa752-15890b72-d3a8197b_0] Write session will restart in 0.006948s
2025-10-09T21:51:48.712747Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|216a94d6-c51fa752-15890b72-d3a8197b_0] Write session: Do CDS request
2025-10-09T21:51:48.712753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|216a94d6-c51fa752-15890b72-d3a8197b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:48.720539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|216a94d6-c51fa752-15890b72-d3a8197b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:48.720598Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:48.720612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2255
2025-10-09T21:51:48.720779Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2255 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:48.720818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2255 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:48.720825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2255 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:48.720829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:48.721051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:48.721234Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2255 sessionId:  describe result for acl check
2025-10-09T21:51:48.721270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:48.721276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:48.721277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:48.721282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343051542341609:8374] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:48.721769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343051542341609:8374] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:48.722900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:48.723066Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:48.723076Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:48.723086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:48.723126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:48.723160Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343051180490273:12039]
2025-10-09T21:51:48.723541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:48.723811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:48.723842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:48.723943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343051542341609:8374] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:48.724049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:48.724082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.724088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.724089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.724091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.724092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.724092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.724093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.724093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.724094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.724097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.724104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.724107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.724109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.724111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.724113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.724115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.724117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.724119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.724123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.724128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.724132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.724136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.724138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.724142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.724144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.724148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.724150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.724153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.724155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.724158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.724164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.724167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.724170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.724173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.724177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.724181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.724198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:48.729967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343051542341609:8374] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:48.729985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343051542341609:8374] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:48.729990Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343051542341609:8374] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:48.730001Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2255 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:48.730405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:48.731758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:48.731776Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:48.731779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:48.731783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:48.731790Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2254 [2:7559343051180490273:12039]
2025-10-09T21:51:48.731804Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:48.731820Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:48.731822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:48.731825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:48.731827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:48.731840Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:48.731844Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:48.731850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:48.731912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.731919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.731922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.731925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:48.732027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:48.732036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.732049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.732060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.732073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:48.732092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343051542341649:8374], now have 1 active actors on pipe
2025-10-09T21:51:48.732100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343051542341562:2432], now have 1 active actors on pipe
2025-10-09T21:51:48.732108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343051180490268:12039]
2025-10-09T21:51:48.732110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343051180490268:12039]
2025-10-09T21:51:48.732112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343051180490268:12039]
2025-10-09T21:51:48.732125Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:48.732140Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:48.732143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:48.732148Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:48.732151Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:48.732157Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:48.732161Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:48.732161Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2254
2025-10-09T21:51:48.732172Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2254
2025-10-09T21:51:48.732200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343051180490268:12039]
2025-10-09T21:51:48.732209Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:48.732211Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:48.732243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:48.732251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:48.732388Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|21b6139-78fe20e8-c4035a24-d8d48b93_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:48.732398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:48.732401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:48.732403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:48.732409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:48.732417Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:48.732449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:48.732604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:48.732612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:48.732623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:48.732723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2255 partition: 0 MaxSeqNo: 25 sessionId: src_id|21b6139-78fe20e8-c4035a24-d8d48b93_0
2025-10-09T21:51:48.733085Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|216a94d6-c51fa752-15890b72-d3a8197b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|21b6139-78fe20e8-c4035a24-d8d48b93_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:48.735599Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21b6139-78fe20e8-c4035a24-d8d48b93_0] Write session: try to update token
2025-10-09T21:51:48.735616Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21b6139-78fe20e8-c4035a24-d8d48b93_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:48.746830Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:48.746850Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:48.746852Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:48.746854Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:48.746863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343051180490268:12039]
2025-10-09T21:51:48.746870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343051180490268:12039]
2025-10-09T21:51:48.746876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343051180490268:12039]
2025-10-09T21:51:48.746915Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:48.746918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:48.746921Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:48.746924Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:48.746926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.746928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:48.747086Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:48.752624Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2255 sessionId: src_id|21b6139-78fe20e8-c4035a24-d8d48b93_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:48.755025Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:48.812285Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343051542341656:8652]
2025-10-09T21:51:48.812587Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 2254, pipe: [1:7559343051542341656:8652]
2025-10-09T21:51:48.897865Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:48.898235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:48.898249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:48.898283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343051542341656:8652], now have 1 active actors on pipe
2025-10-09T21:51:48.898535Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:48.898583Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343051542341656:8652]
2025-10-09T21:51:48.898615Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:48.898621Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:48.898628Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:48.898707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2255 reason: error at write: Disconnected sessionId: src_id|21b6139-78fe20e8-c4035a24-d8d48b93_0
2025-10-09T21:51:48.898862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2255 sessionId: src_id|21b6139-78fe20e8-c4035a24-d8d48b93_0 is DEAD
2025-10-09T21:51:48.899083Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21b6139-78fe20e8-c4035a24-d8d48b93_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:48.899099Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21b6139-78fe20e8-c4035a24-d8d48b93_0] Write session will restart in 0.006220s
2025-10-09T21:51:48.899128Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21b6139-78fe20e8-c4035a24-d8d48b93_0] Write session: Do CDS request
2025-10-09T21:51:48.899135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21b6139-78fe20e8-c4035a24-d8d48b93_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:48.906487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21b6139-78fe20e8-c4035a24-d8d48b93_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:48.906613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:48.906631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2256
2025-10-09T21:51:48.906738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2256 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:48.906769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2256 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:48.906780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2256 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:48.906783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:48.907195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2256 sessionId:  describe result for acl check
2025-10-09T21:51:48.907236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:48.907245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:48.907247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:48.907253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343051542341708:8363] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:48.907739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:48.907882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343051542341708:8363] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:48.909487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:48.909693Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:48.909703Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:48.909713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:48.909768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:48.909815Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343051180490364:12043]
2025-10-09T21:51:48.909988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343051542341708:8363] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:48.910468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:48.910819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:48.910862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:48.911192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:48.911246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.911254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.911256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.911257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.911259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.911260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.911261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.911263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.911264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.911268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.911279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.911284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.911288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.911292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.911296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.911300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.911304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.911309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.911315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.911323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:48.911328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.911334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:48.911342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.911347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:48.911352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.911357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:48.911364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.911370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:48.911375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.911380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:48.911387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.911393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.911398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.911403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.911408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.911413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.911445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:48.912201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343051542341708:8363] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:48.912217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343051542341708:8363] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:48.912220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343051542341708:8363] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:48.912230Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2256 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:48.917822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:48.919060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:48.919076Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:48.919079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:48.919083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:48.919090Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2255 [2:7559343051180490364:12043]
2025-10-09T21:51:48.919107Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:48.919124Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:48.919128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:48.919131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:48.919133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:48.919147Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:48.919153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:48.919165Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:48.919230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.919238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.919241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.919243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:48.919315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:48.919322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.919330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:48.919336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:48.919346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:48.919361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343051542341659:2432], now have 1 active actors on pipe
2025-10-09T21:51:48.919368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343051542341749:8363], now have 1 active actors on pipe
2025-10-09T21:51:48.919374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343051180490359:12043]
2025-10-09T21:51:48.919377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343051180490359:12043]
2025-10-09T21:51:48.919380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343051180490359:12043]
2025-10-09T21:51:48.919393Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:48.919404Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:48.919407Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:48.919408Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:48.919410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:48.919419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:48.919423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:48.919444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343051180490359:12043]
2025-10-09T21:51:48.919450Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:48.919436Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2255
2025-10-09T21:51:48.919468Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:48.919436Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2255
2025-10-09T21:51:48.919501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:48.919508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:48.919529Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b99b68dc-f27ca397-91d9a793-d00b1685_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:48.919539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:48.919543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:48.919545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:48.919551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:48.919565Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:48.919578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:48.919717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:48.919725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:48.919736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:48.919823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2256 partition: 0 MaxSeqNo: 25 sessionId: src_id|b99b68dc-f27ca397-91d9a793-d00b1685_0
2025-10-09T21:51:48.920159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21b6139-78fe20e8-c4035a24-d8d48b93_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b99b68dc-f27ca397-91d9a793-d00b1685_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:48.922715Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b99b68dc-f27ca397-91d9a793-d00b1685_0] Write session: try to update token
2025-10-09T21:51:48.922741Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b99b68dc-f27ca397-91d9a793-d00b1685_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:48.934486Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:48.934505Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:48.934508Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:48.934514Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:48.934526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343051180490359:12043]
2025-10-09T21:51:48.934531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343051180490359:12043]
2025-10-09T21:51:48.934537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343051180490359:12043]
2025-10-09T21:51:48.934583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:48.934586Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:48.934589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:48.934591Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:48.934592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:48.934600Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:48.934921Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:48.936100Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2256 sessionId: src_id|b99b68dc-f27ca397-91d9a793-d00b1685_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:48.938511Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:48.998662Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343051542341756:8652]
2025-10-09T21:51:48.998917Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 2255, pipe: [1:7559343051542341756:8652]
2025-10-09T21:51:49.082576Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:49.083021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.083041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.083085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343051542341756:8652], now have 1 active actors on pipe
2025-10-09T21:51:49.083343Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:49.083333Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:49.083352Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:49.083356Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:49.083392Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343051542341756:8652]
2025-10-09T21:51:49.083437Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2256 reason: error at write: Disconnected sessionId: src_id|b99b68dc-f27ca397-91d9a793-d00b1685_0
2025-10-09T21:51:49.083622Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2256 sessionId: src_id|b99b68dc-f27ca397-91d9a793-d00b1685_0 is DEAD
2025-10-09T21:51:49.083792Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b99b68dc-f27ca397-91d9a793-d00b1685_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:49.083804Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b99b68dc-f27ca397-91d9a793-d00b1685_0] Write session will restart in 0.005146s
2025-10-09T21:51:49.083826Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b99b68dc-f27ca397-91d9a793-d00b1685_0] Write session: Do CDS request
2025-10-09T21:51:49.083831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b99b68dc-f27ca397-91d9a793-d00b1685_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:49.090492Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b99b68dc-f27ca397-91d9a793-d00b1685_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:49.090552Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:49.090571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2257
2025-10-09T21:51:49.090723Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2257 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:49.090750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2257 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:49.090754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2257 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:49.090756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:49.091154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2257 sessionId:  describe result for acl check
2025-10-09T21:51:49.091186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:49.091193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:49.091195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:49.091201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343055837309103:8337] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:49.091746Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343055837309103:8337] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:49.091934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:49.093340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:49.093540Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:49.093550Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:49.093561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:49.093609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:49.093651Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343055475457746:12045]
2025-10-09T21:51:49.093655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343055837309103:8337] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:49.094114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:49.094389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:49.094421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:49.094697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:49.094750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.094756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.094757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.094758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.094758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.094759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.094760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.094761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.094762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.094768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.094775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.094778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.094780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.094782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.094785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.094787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.094789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.094791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.094796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.094800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.094804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.094808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.094813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.094816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.094819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.094822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.094825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.094828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.094831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.094835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.094838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.094843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.094847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.094853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.094857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.094862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.094891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:49.099593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343055837309103:8337] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:49.099607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343055837309103:8337] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:49.099610Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343055837309103:8337] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:49.099618Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2257 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:49.101130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:49.102536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:49.102550Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:49.102552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:49.102559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:49.102565Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2256 [2:7559343055475457746:12045]
2025-10-09T21:51:49.102578Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:49.102593Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:49.102595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:49.102598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:49.102599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:49.102611Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:49.102615Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:49.102620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:49.102689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.102697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.102699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.102700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:49.102767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:49.102773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.102783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.102791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.102802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:49.102821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343055475457741:12045]
2025-10-09T21:51:49.102823Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343055475457741:12045]
2025-10-09T21:51:49.102824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343055475457741:12045]
2025-10-09T21:51:49.102836Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:49.102848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:49.102847Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:49.102852Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:49.102855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:49.102858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:49.102860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:49.102867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343055837309055:2432], now have 1 active actors on pipe
2025-10-09T21:51:49.102871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343055837309144:8337], now have 1 active actors on pipe
2025-10-09T21:51:49.102896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343055475457741:12045]
2025-10-09T21:51:49.102914Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:49.102934Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:49.102937Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2256
2025-10-09T21:51:49.102958Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2256
2025-10-09T21:51:49.103008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.103016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.103036Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bb45b883-ea62692a-a4d89824-144f59c3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:49.103046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:49.103050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:49.103053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:49.103061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:49.103078Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:49.103095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:49.103310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.103316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.103327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:49.103419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2257 partition: 0 MaxSeqNo: 25 sessionId: src_id|bb45b883-ea62692a-a4d89824-144f59c3_0
2025-10-09T21:51:49.103782Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b99b68dc-f27ca397-91d9a793-d00b1685_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bb45b883-ea62692a-a4d89824-144f59c3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:49.106316Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb45b883-ea62692a-a4d89824-144f59c3_0] Write session: try to update token
2025-10-09T21:51:49.106336Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb45b883-ea62692a-a4d89824-144f59c3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:49.119638Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:49.119659Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:49.119672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343055475457741:12045]
2025-10-09T21:51:49.119726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:49.119758Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:49.119926Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:49.120109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:49.120263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:49.120404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:49.120636Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:49.120674Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:49.120838Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2257 sessionId: src_id|bb45b883-ea62692a-a4d89824-144f59c3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:49.123293Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:49.123538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.123552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.123582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:49.123678Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:49.127560Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:49.127577Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:49.127579Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:49.127581Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:49.127590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343055475457741:12045]
2025-10-09T21:51:49.127593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343055475457741:12045]
2025-10-09T21:51:49.127596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343055475457741:12045]
2025-10-09T21:51:49.127632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:49.127639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.127641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:49.127653Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:49.127661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:49.127666Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:49.127799Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:49.183473Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343055837309153:8652]
2025-10-09T21:51:49.183727Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 2256, pipe: [1:7559343055837309153:8652]
2025-10-09T21:51:49.277314Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:49.287288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.287313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.287506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.287579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.287674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.287748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.288302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.288315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.288394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.288464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.288538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.288620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.288673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.288681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.288759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.288877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.288950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.289100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.289154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.289162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.289232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.289356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.289465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.289533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.289608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.289617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.289690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.289760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.289833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.289954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.290009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.290017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.290108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.290177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.290245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.290311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.290385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.290393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.290466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.290530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.290598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.290666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.290738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.290745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.290814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.290879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.290949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.291020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.291177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.291186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.291258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.291322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.291388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.291460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.291517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.291524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.291600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.291666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.291736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.291806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.291859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.291866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.291934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343055837309153:8652], now have 1 active actors on pipe
2025-10-09T21:51:49.292144Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:49.292165Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343055837309153:8652]
2025-10-09T21:51:49.292200Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:49.292207Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:49.292236Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2257 reason: error at write: Disconnected sessionId: src_id|bb45b883-ea62692a-a4d89824-144f59c3_0
2025-10-09T21:51:49.292429Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2257 sessionId: src_id|bb45b883-ea62692a-a4d89824-144f59c3_0 is DEAD
2025-10-09T21:51:49.292558Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb45b883-ea62692a-a4d89824-144f59c3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:49.292565Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb45b883-ea62692a-a4d89824-144f59c3_0] Write session will restart in 0.005244s
2025-10-09T21:51:49.292586Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb45b883-ea62692a-a4d89824-144f59c3_0] Write session: Do CDS request
2025-10-09T21:51:49.292592Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb45b883-ea62692a-a4d89824-144f59c3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:49.292666Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:49.298526Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb45b883-ea62692a-a4d89824-144f59c3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:49.298520Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:49.298541Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2258
2025-10-09T21:51:49.298713Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2258 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:49.298739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2258 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:49.298742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2258 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:49.298745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:49.299180Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2258 sessionId:  describe result for acl check
2025-10-09T21:51:49.299222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:49.299229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:49.299231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:49.299237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343055837309207:8333] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:49.299790Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343055837309207:8333] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:49.300407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:49.301734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:49.301922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343055837309207:8333] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:49.301920Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:49.301931Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:49.301941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:49.302000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:49.302068Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343055475457835:12047]
2025-10-09T21:51:49.302550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:49.302842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:49.302884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:49.303141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:49.303189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.303195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.303196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.303197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.303199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.303200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.303201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.303202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.303203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.303209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.303218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.303222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.303226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.303230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.303235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.303240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.303244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.303247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.303254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.303264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.303270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.303276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.303284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.303289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.303293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.303299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.303304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.303310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.303314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.303318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.303322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.303327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.303331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.303337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.303342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.303348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.303378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:49.308254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343055837309207:8333] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:49.308272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343055837309207:8333] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:49.308276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343055837309207:8333] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:49.308286Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2258 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:49.309757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:49.311178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:49.311193Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:49.311196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:49.311199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:49.311205Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2257 [2:7559343055475457835:12047]
2025-10-09T21:51:49.311218Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:49.311237Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:49.311241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:49.311245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:49.311248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:49.311265Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:49.311271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:49.311278Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:49.311353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.311362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.311366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.311370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:49.311442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:49.311448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.311456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.311463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.311473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:49.311494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343055837309159:2432], now have 1 active actors on pipe
2025-10-09T21:51:49.311501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343055475457830:12047]
2025-10-09T21:51:49.311503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343055475457830:12047]
2025-10-09T21:51:49.311504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343055475457830:12047]
2025-10-09T21:51:49.311516Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:49.311523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343055837309251:8333], now have 1 active actors on pipe
2025-10-09T21:51:49.311530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:49.311536Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:49.311540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:49.311540Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:49.311542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:49.311543Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:49.311571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343055475457830:12047]
2025-10-09T21:51:49.311578Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:49.311578Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2257
2025-10-09T21:51:49.311603Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:49.311585Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2257
2025-10-09T21:51:49.311695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.311705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.311728Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9e67aaab-685ee9cf-f3964906-d2513646_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:49.311740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:49.311747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:49.311752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:49.311762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:49.311776Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:49.311789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:49.311968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.311978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.311991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:49.312101Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2258 partition: 0 MaxSeqNo: 25 sessionId: src_id|9e67aaab-685ee9cf-f3964906-d2513646_0
2025-10-09T21:51:49.312438Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb45b883-ea62692a-a4d89824-144f59c3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9e67aaab-685ee9cf-f3964906-d2513646_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:49.314960Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e67aaab-685ee9cf-f3964906-d2513646_0] Write session: try to update token
2025-10-09T21:51:49.314983Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e67aaab-685ee9cf-f3964906-d2513646_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:49.323930Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:49.323945Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:49.323953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343055475457830:12047]
2025-10-09T21:51:49.323989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:49.324012Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:49.324176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:49.324376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:49.324538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:49.324686Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:49.324899Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:49.324932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:49.328258Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2258 sessionId: src_id|9e67aaab-685ee9cf-f3964906-d2513646_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:49.330530Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:49.330724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.330739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.330767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:49.330866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:49.337955Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:49.337970Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:49.337972Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:49.337976Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:49.337985Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343055475457830:12047]
2025-10-09T21:51:49.337988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343055475457830:12047]
2025-10-09T21:51:49.337992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343055475457830:12047]
2025-10-09T21:51:49.338025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:49.338045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.338047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:49.338066Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:49.338073Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:49.338079Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:49.338209Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:49.392361Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343055837309262:8652]
2025-10-09T21:51:49.392662Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 2257, pipe: [1:7559343055837309262:8652]
2025-10-09T21:51:49.486717Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:49.496541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.496569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.496746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.496814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.496887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.496958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.497299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.497309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.497413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.497485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.497558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.497643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.497692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.497698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.497772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.497927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.498152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.498349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.498413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.498422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.498505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.499046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.499118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.499191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.499246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.499252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.499331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.499405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.499480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.499548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.499606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.499612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.499706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.499780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.500006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.500201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.500300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.500306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.500383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.500580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.500773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.500847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.500900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.500906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.500991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.501070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.501146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.501218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.501465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.501473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.501545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.501607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.501676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.501749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.501952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.501960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.502053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:49.502137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:49.502213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:49.502286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:49.502343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:49.502350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:49.502426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343055837309262:8652], now have 1 active actors on pipe
2025-10-09T21:51:49.502733Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343055837309262:8652]
2025-10-09T21:51:49.502744Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:49.502750Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:49.502788Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:49.502789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2258 reason: error at write: Disconnected sessionId: src_id|9e67aaab-685ee9cf-f3964906-d2513646_0
2025-10-09T21:51:49.503014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2258 sessionId: src_id|9e67aaab-685ee9cf-f3964906-d2513646_0 is DEAD
2025-10-09T21:51:49.503153Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e67aaab-685ee9cf-f3964906-d2513646_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:49.503164Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e67aaab-685ee9cf-f3964906-d2513646_0] Write session will restart in 0.007525s
2025-10-09T21:51:49.503185Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e67aaab-685ee9cf-f3964906-d2513646_0] Write session: Do CDS request
2025-10-09T21:51:49.503190Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e67aaab-685ee9cf-f3964906-d2513646_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:49.503213Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:49.511589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e67aaab-685ee9cf-f3964906-d2513646_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:49.511734Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:49.511753Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2259
2025-10-09T21:51:49.511919Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2259 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:49.511971Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2259 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:49.511994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2259 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:49.512003Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:49.512469Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2259 sessionId:  describe result for acl check
2025-10-09T21:51:49.512512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:49.512521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:49.512523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:49.512530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343055837309339:8299] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:49.512646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:49.513047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343055837309339:8299] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:49.514365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:49.514561Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:49.514571Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:49.514582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:49.514627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:49.514661Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343055475457963:12061]
2025-10-09T21:51:49.515095Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343055837309339:8299] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:49.515186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:49.515438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:49.515469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:49.515707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:49.515747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.515751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.515752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.515753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.515753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.515754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.515755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.515756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.515757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.515760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.515766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.515768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.515771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.515773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.515776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.515778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.515780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.515784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.515788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.515793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.515797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.515800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.515803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.515806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.515809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.515812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.515815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.515818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.515824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.515829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.515834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.515839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.515842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.515847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.515852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.515858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.515880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:49.526369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343055837309339:8299] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:49.526397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343055837309339:8299] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:49.526401Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343055837309339:8299] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:49.526412Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2259 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:49.526834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:49.530292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:49.530310Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:49.530316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:49.530332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:49.530350Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2258 [2:7559343055475457963:12061]
2025-10-09T21:51:49.530375Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:49.530466Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:49.530477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:49.530481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:49.530492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:49.530522Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:49.530536Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:49.530543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:49.530745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.530764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.530766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.530769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:49.530895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:49.530906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.530919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.530932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.530950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:49.530971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343055837309377:8299], now have 1 active actors on pipe
2025-10-09T21:51:49.530981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343055837309288:2432], now have 1 active actors on pipe
2025-10-09T21:51:49.530988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343055475457957:12061]
2025-10-09T21:51:49.530991Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343055475457957:12061]
2025-10-09T21:51:49.530995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343055475457957:12061]
2025-10-09T21:51:49.531008Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:49.531022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:49.531035Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:49.531037Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:49.531038Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:49.531040Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:49.531042Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:49.531014Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2258
2025-10-09T21:51:49.531024Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2258
2025-10-09T21:51:49.531108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.531119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.531129Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343055475457957:12061]
2025-10-09T21:51:49.531140Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:49.531172Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|106d2ff3-529f303c-565bc970-213a0d8e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:49.531174Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:49.531190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:49.531195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:49.531197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:49.531206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:49.531216Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:49.531234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:49.531410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.531418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.531430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:49.531519Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2259 partition: 0 MaxSeqNo: 25 sessionId: src_id|106d2ff3-529f303c-565bc970-213a0d8e_0
2025-10-09T21:51:49.531873Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e67aaab-685ee9cf-f3964906-d2513646_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|106d2ff3-529f303c-565bc970-213a0d8e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:49.534371Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|106d2ff3-529f303c-565bc970-213a0d8e_0] Write session: try to update token
2025-10-09T21:51:49.534388Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|106d2ff3-529f303c-565bc970-213a0d8e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:49.546014Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:49.546047Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:49.546049Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:49.546051Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:49.546062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343055475457957:12061]
2025-10-09T21:51:49.546066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343055475457957:12061]
2025-10-09T21:51:49.546071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343055475457957:12061]
2025-10-09T21:51:49.546111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:49.546117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.546118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:49.546136Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:49.546144Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:49.546149Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:49.546273Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:49.549817Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2259 sessionId: src_id|106d2ff3-529f303c-565bc970-213a0d8e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:49.552107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:49.602827Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343055837309384:8652]
2025-10-09T21:51:49.603119Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 2258, pipe: [1:7559343055837309384:8652]
2025-10-09T21:51:49.692147Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:49.692719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.692743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.692792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343055837309384:8652], now have 1 active actors on pipe
2025-10-09T21:51:49.693039Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:49.693054Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:49.693079Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343055837309384:8652]
2025-10-09T21:51:49.693105Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:49.693235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2259 reason: error at write: Disconnected sessionId: src_id|106d2ff3-529f303c-565bc970-213a0d8e_0
2025-10-09T21:51:49.693416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2259 sessionId: src_id|106d2ff3-529f303c-565bc970-213a0d8e_0 is DEAD
2025-10-09T21:51:49.693622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|106d2ff3-529f303c-565bc970-213a0d8e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:49.693632Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|106d2ff3-529f303c-565bc970-213a0d8e_0] Write session will restart in 0.009826s
2025-10-09T21:51:49.693654Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|106d2ff3-529f303c-565bc970-213a0d8e_0] Write session: Do CDS request
2025-10-09T21:51:49.693662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|106d2ff3-529f303c-565bc970-213a0d8e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:49.693825Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:49.700990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:49.702465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:49.702630Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:49.702639Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:49.702646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:49.702681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:49.702738Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343055475458052:12063]
2025-10-09T21:51:49.703194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:49.703532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:49.703564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:49.703823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:49.703865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.703871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.703872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.703873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.703874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.703875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.703875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.703876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.703877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.703881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.703888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.703891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.703893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.703895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.703898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.703900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.703902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.703904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.703909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.703913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.703917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.703924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.703927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.703930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.703933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.703936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.703939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.703942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.703945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.703948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.703951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.703955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.703957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.703960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.703966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.703969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.703996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:49.704495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|106d2ff3-529f303c-565bc970-213a0d8e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:49.704594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:49.704612Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2260
2025-10-09T21:51:49.704746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2260 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:49.704789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2260 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:49.704799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2260 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:49.704802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:49.705236Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2260 sessionId:  describe result for acl check
2025-10-09T21:51:49.705273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:49.705279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:49.705281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:49.705288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343055837309445:8286] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:49.705863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343055837309445:8286] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:49.707966Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343055837309445:8286] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:49.710679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343055837309445:8286] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:49.710691Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343055837309445:8286] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:49.710694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343055837309445:8286] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:49.710701Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2260 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:49.710689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:49.713345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:49.713364Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:49.713367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:49.713371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:49.713379Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2259 [2:7559343055475458052:12063]
2025-10-09T21:51:49.713396Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:49.713416Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:49.713419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:49.713423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:49.713425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:49.713439Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:49.713443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:49.713449Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:49.713523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.713539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.713541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.713544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:49.713626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:49.713637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.713651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.713673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.713691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:49.713717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343055837309479:8286], now have 1 active actors on pipe
2025-10-09T21:51:49.713728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343055837309387:2432], now have 1 active actors on pipe
2025-10-09T21:51:49.713736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343055475458047:12063]
2025-10-09T21:51:49.713741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343055475458047:12063]
2025-10-09T21:51:49.713743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343055475458047:12063]
2025-10-09T21:51:49.713758Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:49.713778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:49.713780Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:49.713785Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:49.713789Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:49.713800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:49.713803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:49.713809Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2259
2025-10-09T21:51:49.713811Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2259
2025-10-09T21:51:49.713839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343055475458047:12063]
2025-10-09T21:51:49.713856Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:49.713892Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:49.714003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.714017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.714059Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2ed6e686-1a2ce37f-5bafc726-620bb000_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:49.714066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:49.714072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:49.714075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:49.714085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:49.714097Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:49.714119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:49.714393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.714409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.714429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:49.714553Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2260 partition: 0 MaxSeqNo: 25 sessionId: src_id|2ed6e686-1a2ce37f-5bafc726-620bb000_0
2025-10-09T21:51:49.714936Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|106d2ff3-529f303c-565bc970-213a0d8e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2ed6e686-1a2ce37f-5bafc726-620bb000_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:49.717649Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed6e686-1a2ce37f-5bafc726-620bb000_0] Write session: try to update token
2025-10-09T21:51:49.717673Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed6e686-1a2ce37f-5bafc726-620bb000_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:49.728912Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:49.728931Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:49.728933Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:49.728936Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:49.728948Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343055475458047:12063]
2025-10-09T21:51:49.728953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343055475458047:12063]
2025-10-09T21:51:49.728957Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343055475458047:12063]
2025-10-09T21:51:49.729006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:49.729007Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:49.729013Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:49.729014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.729017Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:49.729020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:49.729161Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:49.734026Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2260 sessionId: src_id|2ed6e686-1a2ce37f-5bafc726-620bb000_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:49.736401Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:49.793278Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343055837309486:8652]
2025-10-09T21:51:49.793568Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 2259, pipe: [1:7559343055837309486:8652]
2025-10-09T21:51:49.884244Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:49.884518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.884533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.884564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343055837309486:8652], now have 1 active actors on pipe
2025-10-09T21:51:49.884743Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:49.884768Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:49.884784Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:49.884785Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:49.884827Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343055837309486:8652]
2025-10-09T21:51:49.884848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2260 reason: error at write: Disconnected sessionId: src_id|2ed6e686-1a2ce37f-5bafc726-620bb000_0
2025-10-09T21:51:49.885005Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2260 sessionId: src_id|2ed6e686-1a2ce37f-5bafc726-620bb000_0 is DEAD
2025-10-09T21:51:49.885167Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed6e686-1a2ce37f-5bafc726-620bb000_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:49.885187Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed6e686-1a2ce37f-5bafc726-620bb000_0] Write session will restart in 0.008769s
2025-10-09T21:51:49.885214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed6e686-1a2ce37f-5bafc726-620bb000_0] Write session: Do CDS request
2025-10-09T21:51:49.885221Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed6e686-1a2ce37f-5bafc726-620bb000_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:49.892384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:49.893580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:49.893758Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:49.893768Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:49.893781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:49.893833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:49.893869Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343055475458148:12067]
2025-10-09T21:51:49.894341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:49.894589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:49.894627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:49.894866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:49.894902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.894906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.894907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.894907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.894908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.894909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.894910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.894910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.894913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.894916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.894923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.894926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.894928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.894930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.894932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.894934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.894936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.894939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.894944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.894949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:49.894953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.894957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:49.894960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.894963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:49.894965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.894969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:49.894972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.894975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:49.894979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.894982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:49.894985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.894989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.894994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.894997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.895000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.895003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.895019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:49.895434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed6e686-1a2ce37f-5bafc726-620bb000_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:49.895514Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:49.895527Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2261
2025-10-09T21:51:49.895634Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2261 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:49.895657Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2261 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:49.895664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2261 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:49.895667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:49.895981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2261 sessionId:  describe result for acl check
2025-10-09T21:51:49.896005Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:49.896010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:49.896013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:49.896017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343055837309547:8274] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:49.896451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343055837309547:8274] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:49.898591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343055837309547:8274] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:49.900890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343055837309547:8274] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:49.900910Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343055837309547:8274] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:49.900914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343055837309547:8274] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:49.900920Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2261 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:49.901451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:49.902723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:49.902740Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:49.902742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:49.902745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:49.902750Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2260 [2:7559343055475458148:12067]
2025-10-09T21:51:49.902764Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:49.902779Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:49.902780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:49.902783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:49.902784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:49.902799Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:49.902807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:49.902815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:49.902869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.902875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.902876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.902878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:49.902945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:49.902953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.902961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:49.902967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:49.902977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:49.902993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343055837309489:2432], now have 1 active actors on pipe
2025-10-09T21:51:49.903000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343055837309582:8274], now have 1 active actors on pipe
2025-10-09T21:51:49.903005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343055475458143:12067]
2025-10-09T21:51:49.903007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343055475458143:12067]
2025-10-09T21:51:49.903008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343055475458143:12067]
2025-10-09T21:51:49.903019Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:49.903033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:49.903034Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:49.903037Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:49.903038Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:49.903045Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:49.903048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:49.903063Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2260
2025-10-09T21:51:49.903071Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2260
2025-10-09T21:51:49.903084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343055475458143:12067]
2025-10-09T21:51:49.903092Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:49.903094Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:49.903116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.903122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.903147Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|21d27f9e-bae0496a-98bf9090-3f71687c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:49.903158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:49.903161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:49.903163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:49.903170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:49.903177Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:49.903193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:49.903304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:49.903311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:49.903324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:49.903374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2261 partition: 0 MaxSeqNo: 25 sessionId: src_id|21d27f9e-bae0496a-98bf9090-3f71687c_0
2025-10-09T21:51:49.903707Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2ed6e686-1a2ce37f-5bafc726-620bb000_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|21d27f9e-bae0496a-98bf9090-3f71687c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:49.906189Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21d27f9e-bae0496a-98bf9090-3f71687c_0] Write session: try to update token
2025-10-09T21:51:49.906207Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21d27f9e-bae0496a-98bf9090-3f71687c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:49.916825Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:49.916839Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:49.916841Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:49.916843Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:49.916850Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343055475458143:12067]
2025-10-09T21:51:49.916854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343055475458143:12067]
2025-10-09T21:51:49.916857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343055475458143:12067]
2025-10-09T21:51:49.916897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:49.916896Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:49.916902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:49.916903Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:49.916904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:49.916907Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:49.917028Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:49.919672Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2261 sessionId: src_id|21d27f9e-bae0496a-98bf9090-3f71687c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:49.921976Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:49.984767Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343055837309591:8652]
2025-10-09T21:51:49.985026Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 2260, pipe: [1:7559343055837309591:8652]
2025-10-09T21:51:50.063171Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:50.063558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:50.063576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:50.063617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343055837309591:8652], now have 1 active actors on pipe
2025-10-09T21:51:50.063901Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343055837309591:8652]
2025-10-09T21:51:50.063937Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:50.063939Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:50.063940Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:50.064002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2261 reason: error at write: Disconnected sessionId: src_id|21d27f9e-bae0496a-98bf9090-3f71687c_0
2025-10-09T21:51:50.064149Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:50.064194Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2261 sessionId: src_id|21d27f9e-bae0496a-98bf9090-3f71687c_0 is DEAD
2025-10-09T21:51:50.064333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21d27f9e-bae0496a-98bf9090-3f71687c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:50.064344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21d27f9e-bae0496a-98bf9090-3f71687c_0] Write session will restart in 0.006287s
2025-10-09T21:51:50.064383Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21d27f9e-bae0496a-98bf9090-3f71687c_0] Write session: Do CDS request
2025-10-09T21:51:50.064390Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21d27f9e-bae0496a-98bf9090-3f71687c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:50.071496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|21d27f9e-bae0496a-98bf9090-3f71687c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:50.071571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:50.071587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2262
2025-10-09T21:51:50.071702Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2262 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:50.071727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2262 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:50.071736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2262 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:50.071739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:50.071793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:50.072190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2262 sessionId:  describe result for acl check
2025-10-09T21:51:50.072225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:50.072233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:50.072235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:50.072241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343060132276944:8246] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:50.072800Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343060132276944:8246] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:50.073176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:50.073359Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:50.073371Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:50.073379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:50.073419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:50.073457Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343059770425536:12069]
2025-10-09T21:51:50.073902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:50.074188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:50.074217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:50.074466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:50.074474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343060132276944:8246] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:50.074521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.074529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.074530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.074531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.074532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.074532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.074534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.074535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.074535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.074539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.074546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.074548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.074551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.074553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.074556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.074558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.074560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.074563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.074567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.074573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.074577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.074582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.074587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.074593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.074597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.074602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.074607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.074612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.074619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.074625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.074638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.074644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.074658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.074667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.074671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.074675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.074693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:50.080514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343060132276944:8246] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:50.080527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343060132276944:8246] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:50.080530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343060132276944:8246] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:50.080537Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2262 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:50.081019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:50.082615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:50.082629Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:50.082632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:50.082635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:50.082640Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2261 [2:7559343059770425536:12069]
2025-10-09T21:51:50.082654Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:50.082671Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:50.082681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:50.082684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:50.082686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:50.082700Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:50.082717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:50.082723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:50.082769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.082775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.082776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.082778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:50.082839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:50.082846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.082859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.082878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.082896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:50.082916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343060132276983:8246], now have 1 active actors on pipe
2025-10-09T21:51:50.082924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343060132276889:2432], now have 1 active actors on pipe
2025-10-09T21:51:50.082929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343059770425531:12069]
2025-10-09T21:51:50.082931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343059770425531:12069]
2025-10-09T21:51:50.082932Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343059770425531:12069]
2025-10-09T21:51:50.082944Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:50.082955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:50.082967Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:50.082970Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:50.082973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:50.082974Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:50.082977Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:50.082992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343059770425531:12069]
2025-10-09T21:51:50.082996Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:50.082999Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:50.082979Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2261
2025-10-09T21:51:50.083002Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2261
2025-10-09T21:51:50.083027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:50.083036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:50.083072Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f59ec4a7-a44e8dde-50be46d7-ae2f9980_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:50.083083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:50.083087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:50.083093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:50.083102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:50.083114Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:50.083134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:50.083319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:50.083325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:50.083335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:50.083452Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2262 partition: 0 MaxSeqNo: 25 sessionId: src_id|f59ec4a7-a44e8dde-50be46d7-ae2f9980_0
2025-10-09T21:51:50.083801Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|21d27f9e-bae0496a-98bf9090-3f71687c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f59ec4a7-a44e8dde-50be46d7-ae2f9980_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:50.086229Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f59ec4a7-a44e8dde-50be46d7-ae2f9980_0] Write session: try to update token
2025-10-09T21:51:50.086248Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f59ec4a7-a44e8dde-50be46d7-ae2f9980_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:50.097561Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:50.097581Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:50.097583Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:50.097585Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:50.097598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343059770425531:12069]
2025-10-09T21:51:50.097603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343059770425531:12069]
2025-10-09T21:51:50.097606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343059770425531:12069]
2025-10-09T21:51:50.097649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:50.097658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.097661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:50.097671Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:50.097678Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:50.097681Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:50.097834Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:50.099209Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2262 sessionId: src_id|f59ec4a7-a44e8dde-50be46d7-ae2f9980_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:50.101572Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:50.163968Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343060132276992:8652]
2025-10-09T21:51:50.164193Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 2261, pipe: [1:7559343060132276992:8652]
2025-10-09T21:51:50.244427Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:50.244875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:50.244884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:50.244916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343060132276992:8652], now have 1 active actors on pipe
2025-10-09T21:51:50.245232Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:50.245218Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343060132276992:8652]
2025-10-09T21:51:50.245235Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:50.245239Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:50.245270Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:50.245310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2262 reason: error at write: Disconnected sessionId: src_id|f59ec4a7-a44e8dde-50be46d7-ae2f9980_0
2025-10-09T21:51:50.245509Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2262 sessionId: src_id|f59ec4a7-a44e8dde-50be46d7-ae2f9980_0 is DEAD
2025-10-09T21:51:50.245652Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f59ec4a7-a44e8dde-50be46d7-ae2f9980_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:50.245666Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f59ec4a7-a44e8dde-50be46d7-ae2f9980_0] Write session will restart in 0.007998s
2025-10-09T21:51:50.245693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f59ec4a7-a44e8dde-50be46d7-ae2f9980_0] Write session: Do CDS request
2025-10-09T21:51:50.245700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f59ec4a7-a44e8dde-50be46d7-ae2f9980_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:50.253716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:50.254497Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f59ec4a7-a44e8dde-50be46d7-ae2f9980_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:50.254592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:50.254608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2263
2025-10-09T21:51:50.254745Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2263 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:50.254777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2263 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:50.254787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2263 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:50.254791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:50.255082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:50.255184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2263 sessionId:  describe result for acl check
2025-10-09T21:51:50.255215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:50.255222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:50.255224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:50.255228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343060132277054:8240] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:50.255244Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:50.255252Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:50.255260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:50.255298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:50.255330Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343059770425629:12071]
2025-10-09T21:51:50.255745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343060132277054:8240] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:50.255770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:50.256089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:50.256131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:50.256395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:50.256438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.256442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.256444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.256446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.256447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.256448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.256449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.256451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.256452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.256457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.256471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.256479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.256483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.256491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.256495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.256504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.256508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.256512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.256519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.256526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.256532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.256537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.256543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.256548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.256557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.256562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.256572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.256577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.256586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.256592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.256596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.256603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.256606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.256609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.256613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.256616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.256647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:50.257463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343060132277054:8240] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:50.262242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343060132277054:8240] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:50.262253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343060132277054:8240] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:50.262256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343060132277054:8240] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:50.262261Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2263 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:50.262759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:50.264031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:50.264050Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:50.264053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:50.264058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:50.264064Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2262 [2:7559343059770425629:12071]
2025-10-09T21:51:50.264083Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:50.264104Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:50.264111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:50.264114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:50.264116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:50.264132Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:50.264143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:50.264154Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:50.264210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.264218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.264221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.264223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:50.264292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:50.264299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.264311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.264321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.264344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:50.264365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343060132277093:8240], now have 1 active actors on pipe
2025-10-09T21:51:50.264374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343060132276998:2432], now have 1 active actors on pipe
2025-10-09T21:51:50.264381Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343059770425624:12071]
2025-10-09T21:51:50.264387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343059770425624:12071]
2025-10-09T21:51:50.264389Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343059770425624:12071]
2025-10-09T21:51:50.264405Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:50.264423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:50.264418Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2262
2025-10-09T21:51:50.264422Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2262
2025-10-09T21:51:50.264439Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:50.264442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:50.264447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:50.264451Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:50.264452Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:50.264475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343059770425624:12071]
2025-10-09T21:51:50.264480Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:50.264482Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:50.264500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:50.264511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:50.264561Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c6eeb204-a58c6391-daf29c1f-12d5c9af_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:50.264573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:50.264577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:50.264580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:50.264587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:50.264602Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:50.264626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:50.264891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:50.264901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:50.264916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:50.265020Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2263 partition: 0 MaxSeqNo: 25 sessionId: src_id|c6eeb204-a58c6391-daf29c1f-12d5c9af_0
2025-10-09T21:51:50.265333Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f59ec4a7-a44e8dde-50be46d7-ae2f9980_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c6eeb204-a58c6391-daf29c1f-12d5c9af_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:50.267723Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c6eeb204-a58c6391-daf29c1f-12d5c9af_0] Write session: try to update token
2025-10-09T21:51:50.267740Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c6eeb204-a58c6391-daf29c1f-12d5c9af_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:50.278542Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:50.278561Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:50.278563Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:50.278566Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:50.278581Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343059770425624:12071]
2025-10-09T21:51:50.278585Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343059770425624:12071]
2025-10-09T21:51:50.278589Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343059770425624:12071]
2025-10-09T21:51:50.278634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:50.278638Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:50.278645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.278648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:50.278655Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:50.278658Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:50.278975Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:50.280035Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2263 sessionId: src_id|c6eeb204-a58c6391-daf29c1f-12d5c9af_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:50.282391Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:50.345198Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343060132277105:8652]
2025-10-09T21:51:50.345500Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 2262, pipe: [1:7559343060132277105:8652]
2025-10-09T21:51:50.425614Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:50.425915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:50.425929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:50.425962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343060132277105:8652], now have 1 active actors on pipe
2025-10-09T21:51:50.426164Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:50.426342Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:50.426343Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343060132277105:8652]
2025-10-09T21:51:50.426360Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:50.426465Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2263 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c6eeb204-a58c6391-daf29c1f-12d5c9af_0
2025-10-09T21:51:50.426621Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2263 sessionId: src_id|c6eeb204-a58c6391-daf29c1f-12d5c9af_0 is DEAD
2025-10-09T21:51:50.426772Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6eeb204-a58c6391-daf29c1f-12d5c9af_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:50.426786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6eeb204-a58c6391-daf29c1f-12d5c9af_0] Write session will restart in 0.006948s
2025-10-09T21:51:50.426814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6eeb204-a58c6391-daf29c1f-12d5c9af_0] Write session: Do CDS request
2025-10-09T21:51:50.426820Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6eeb204-a58c6391-daf29c1f-12d5c9af_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:50.434518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c6eeb204-a58c6391-daf29c1f-12d5c9af_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:50.434597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:50.434618Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2264
2025-10-09T21:51:50.434675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:50.434729Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2264 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:50.434754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2264 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:50.434762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2264 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:50.434765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:50.435180Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2264 sessionId:  describe result for acl check
2025-10-09T21:51:50.435221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:50.435228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:50.435230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:50.435236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343060132277165:8226] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:50.435750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343060132277165:8226] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:50.436119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:50.436294Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:50.436304Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:50.436314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:50.436361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:50.436404Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343059770425722:12073]
2025-10-09T21:51:50.437026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:50.437284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:50.437334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:50.437633Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343060132277165:8226] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:50.437661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:50.437726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.437733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.437735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.437736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.437738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.437739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.437740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.437741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.437743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.437748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.437757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.437760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.437764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.437768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.437773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.437777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.437780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.437784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.437791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.437799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.437807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.437815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.437821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.437826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.437831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.437837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.437842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.437847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.437852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.437857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.437862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.437867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.437871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.437877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.437882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.437886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.437917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:50.442968Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343060132277165:8226] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:50.442981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343060132277165:8226] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:50.442984Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343060132277165:8226] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:50.442991Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2264 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:50.444471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:50.445892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:50.445909Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:50.445912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:50.445916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:50.445923Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2263 [2:7559343059770425722:12073]
2025-10-09T21:51:50.445937Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:50.445955Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:50.445966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:50.445970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:50.445972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:50.445987Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:50.445997Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:50.446004Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:50.446082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.446090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.446093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.446095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:50.446191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:50.446199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.446211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.446227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.446245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:50.446266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343060132277108:2432], now have 1 active actors on pipe
2025-10-09T21:51:50.446274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343060132277204:8226], now have 1 active actors on pipe
2025-10-09T21:51:50.446281Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343059770425717:12073]
2025-10-09T21:51:50.446283Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343059770425717:12073]
2025-10-09T21:51:50.446285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343059770425717:12073]
2025-10-09T21:51:50.446298Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:50.446313Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:50.446314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:50.446324Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:50.446327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:50.446330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:50.446332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:50.446366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343059770425717:12073]
2025-10-09T21:51:50.446371Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:50.446388Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:50.446404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:50.446408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:50.446355Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2263
2025-10-09T21:51:50.446356Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2263
2025-10-09T21:51:50.446437Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ce5de984-5f99dc71-4bd59808-3993c7ff_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:50.446450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:50.446453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:50.446456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:50.446463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:50.446478Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:50.446490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:50.446668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:50.446676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:50.446689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:50.446787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2264 partition: 0 MaxSeqNo: 25 sessionId: src_id|ce5de984-5f99dc71-4bd59808-3993c7ff_0
2025-10-09T21:51:50.447140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6eeb204-a58c6391-daf29c1f-12d5c9af_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ce5de984-5f99dc71-4bd59808-3993c7ff_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:50.449481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ce5de984-5f99dc71-4bd59808-3993c7ff_0] Write session: try to update token
2025-10-09T21:51:50.449498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ce5de984-5f99dc71-4bd59808-3993c7ff_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:50.459455Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:50.459476Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:50.459488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343059770425717:12073]
2025-10-09T21:51:50.459526Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:50.459534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:50.459706Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:50.459852Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:50.459980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:50.460115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:50.460315Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:50.460355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:50.461935Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2264 sessionId: src_id|ce5de984-5f99dc71-4bd59808-3993c7ff_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:50.464255Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:50.464460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:50.464480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:50.464512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:50.464605Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:50.473412Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:50.473427Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:50.473430Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:50.473433Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:50.473442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343059770425717:12073]
2025-10-09T21:51:50.473446Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343059770425717:12073]
2025-10-09T21:51:50.473450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343059770425717:12073]
2025-10-09T21:51:50.473496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:50.473499Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:50.473506Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:50.473510Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:50.473510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.473513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:50.473654Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:50.526436Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343060132277234:8652]
2025-10-09T21:51:50.526640Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 2263, pipe: [1:7559343060132277234:8652]
2025-10-09T21:51:50.637311Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:50.647522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:50.647547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:50.647730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:50.647798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:50.647870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:50.647942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:50.648303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:50.648315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:50.648395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:50.648464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:50.648538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:50.648618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:50.648666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:50.648673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:50.648746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:50.648814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:50.648933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:50.649007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:50.649063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:50.649066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:50.649139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:50.649208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:50.649273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:50.649352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:50.649405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:50.649412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:50.649520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:50.649591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:50.649665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:50.649735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:50.649814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:50.649823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:50.649902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:50.649966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:50.650051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:50.650122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:50.650190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:50.650199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:50.650273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:50.650339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:50.650416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:50.650484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:50.650544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:50.650554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:50.650628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:50.650695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:50.650765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:50.650833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:50.650884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:50.650892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:50.650966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:50.651033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:50.651112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:50.651186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:50.651262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:50.651272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:50.651350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:50.651419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:50.651493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:50.651564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:50.651641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:50.651649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:50.651716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343060132277234:8652], now have 1 active actors on pipe
2025-10-09T21:51:50.651998Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343060132277234:8652]
2025-10-09T21:51:50.651999Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:50.652045Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:50.652048Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:50.652074Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2264 reason: error at write: Disconnected sessionId: src_id|ce5de984-5f99dc71-4bd59808-3993c7ff_0
2025-10-09T21:51:50.652218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2264 sessionId: src_id|ce5de984-5f99dc71-4bd59808-3993c7ff_0 is DEAD
2025-10-09T21:51:50.652384Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce5de984-5f99dc71-4bd59808-3993c7ff_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:50.652397Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce5de984-5f99dc71-4bd59808-3993c7ff_0] Write session will restart in 0.007623s
2025-10-09T21:51:50.652416Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce5de984-5f99dc71-4bd59808-3993c7ff_0] Write session: Do CDS request
2025-10-09T21:51:50.652421Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce5de984-5f99dc71-4bd59808-3993c7ff_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:50.652545Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:50.660765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:50.661468Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ce5de984-5f99dc71-4bd59808-3993c7ff_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:50.661552Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:50.661568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2265
2025-10-09T21:51:50.661678Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2265 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:50.661717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2265 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:50.661728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2265 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:50.661731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:50.662139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2265 sessionId:  describe result for acl check
2025-10-09T21:51:50.662170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:50.662173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:50.662174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:50.662181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343060132277294:8202] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:50.662240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:50.662436Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:50.662445Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:50.662454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:50.662504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:50.662542Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343059770425856:12087]
2025-10-09T21:51:50.662845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343060132277294:8202] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:50.663117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:50.663393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:50.663434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:50.663787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:50.663862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.663868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.663869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.663871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.663872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.663873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.663874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.663875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.663877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.663882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.663894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.663898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.663902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.663906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.663910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.663914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.663919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.663923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.663931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.663939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.663945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.663956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.663961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.663966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.663973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.663978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.663986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.663996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.664002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.664008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.664018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.664024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.664031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.664037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.664042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.664052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.664084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:50.664799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343060132277294:8202] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:50.669969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343060132277294:8202] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:50.669981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343060132277294:8202] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:50.669984Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343060132277294:8202] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:50.669992Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2265 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:50.670262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:50.671918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:50.671933Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:50.671936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:50.671940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:50.671951Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2264 [2:7559343059770425856:12087]
2025-10-09T21:51:50.671966Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:50.671982Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:50.671992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:50.671995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:50.671997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:50.672011Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:50.672017Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:50.672023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:50.672083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.672091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.672094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.672096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:50.672176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:50.672184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.672197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.672212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.672232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:50.672254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343060132277333:8202], now have 1 active actors on pipe
2025-10-09T21:51:50.672262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343060132277236:2432], now have 1 active actors on pipe
2025-10-09T21:51:50.672269Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343059770425851:12087]
2025-10-09T21:51:50.672276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343059770425851:12087]
2025-10-09T21:51:50.672278Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343059770425851:12087]
2025-10-09T21:51:50.672272Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2264
2025-10-09T21:51:50.672276Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2264
2025-10-09T21:51:50.672295Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:50.672311Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:50.672313Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:50.672324Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:50.672326Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:50.672327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:50.672331Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:50.672375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:50.672385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:50.672399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343059770425851:12087]
2025-10-09T21:51:50.672412Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:50.672427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:50.672448Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a31556ad-2309f4e9-8d52bb1b-b10474b1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:50.672459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:50.672467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:50.672469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:50.672482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:50.672495Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:50.672518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:50.672667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:50.672674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:50.672686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:50.672790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2265 partition: 0 MaxSeqNo: 25 sessionId: src_id|a31556ad-2309f4e9-8d52bb1b-b10474b1_0
2025-10-09T21:51:50.673062Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce5de984-5f99dc71-4bd59808-3993c7ff_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a31556ad-2309f4e9-8d52bb1b-b10474b1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:50.675323Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a31556ad-2309f4e9-8d52bb1b-b10474b1_0] Write session: try to update token
2025-10-09T21:51:50.675335Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a31556ad-2309f4e9-8d52bb1b-b10474b1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:50.686103Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:50.686119Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:50.686121Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:50.686123Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:50.686131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343059770425851:12087]
2025-10-09T21:51:50.686136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343059770425851:12087]
2025-10-09T21:51:50.686141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343059770425851:12087]
2025-10-09T21:51:50.686184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:50.686190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.686192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:50.686196Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:50.686203Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:50.686208Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:50.686340Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:50.686693Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2265 sessionId: src_id|a31556ad-2309f4e9-8d52bb1b-b10474b1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:50.688935Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:50.752250Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343060132277341:8652]
2025-10-09T21:51:50.752560Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 2264, pipe: [1:7559343060132277341:8652]
2025-10-09T21:51:50.841176Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:50.841528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:50.841546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:50.841584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343060132277341:8652], now have 1 active actors on pipe
2025-10-09T21:51:50.841803Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:50.841826Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:50.841839Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:50.841849Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:50.841887Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343060132277341:8652]
2025-10-09T21:51:50.841912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2265 reason: error at write: Disconnected sessionId: src_id|a31556ad-2309f4e9-8d52bb1b-b10474b1_0
2025-10-09T21:51:50.842129Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2265 sessionId: src_id|a31556ad-2309f4e9-8d52bb1b-b10474b1_0 is DEAD
2025-10-09T21:51:50.842225Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a31556ad-2309f4e9-8d52bb1b-b10474b1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:50.842235Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a31556ad-2309f4e9-8d52bb1b-b10474b1_0] Write session will restart in 0.009579s
2025-10-09T21:51:50.842261Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a31556ad-2309f4e9-8d52bb1b-b10474b1_0] Write session: Do CDS request
2025-10-09T21:51:50.842266Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a31556ad-2309f4e9-8d52bb1b-b10474b1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:50.849607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:50.850929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:50.851103Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:50.851112Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:50.851120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:50.851159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:50.851188Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343059770425956:12091]
2025-10-09T21:51:50.851668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:50.851966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:50.852007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:50.852368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:50.852425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.852435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.852437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.852438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.852439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.852441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.852442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.852443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.852444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.852450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.852460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.852464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.852468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.852472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.852477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.852481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.852484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.852489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.852495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.852502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:50.852534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a31556ad-2309f4e9-8d52bb1b-b10474b1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:50.852508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.852514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:50.852523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.852529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:50.852534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.852540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:50.852546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.852552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:50.852557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.852563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:50.852569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.852576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.852581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.852586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.852592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.852597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.852596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:50.852608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2266
2025-10-09T21:51:50.852626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:50.852751Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2266 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:50.852776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2266 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:50.852780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2266 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:50.852783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:50.853156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2266 sessionId:  describe result for acl check
2025-10-09T21:51:50.853188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:50.853195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:50.853196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:50.853201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343060132277409:8191] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:50.853649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343060132277409:8191] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:50.855272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343060132277409:8191] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:50.858138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343060132277409:8191] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:50.858150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343060132277409:8191] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:50.858153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343060132277409:8191] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:50.858162Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2266 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:50.858883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:50.863191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:50.863212Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:50.863215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:50.863220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:50.863229Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2265 [2:7559343059770425956:12091]
2025-10-09T21:51:50.863258Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:50.863303Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:50.863335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:50.863376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:50.863394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:50.863426Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:50.863442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:50.863449Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:50.863523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.863531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.863532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.863534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:50.863653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:50.863665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.863675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:50.863682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:50.863692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:50.863708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343060132277443:8191], now have 1 active actors on pipe
2025-10-09T21:51:50.863717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343060132277346:2432], now have 1 active actors on pipe
2025-10-09T21:51:50.863723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343059770425951:12091]
2025-10-09T21:51:50.863724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343059770425951:12091]
2025-10-09T21:51:50.863726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343059770425951:12091]
2025-10-09T21:51:50.863739Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:50.863750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:50.863767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:50.863771Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:50.863774Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:50.863786Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:50.863755Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2265
2025-10-09T21:51:50.863789Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:50.863755Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2265
2025-10-09T21:51:50.863943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343059770425951:12091]
2025-10-09T21:51:50.863960Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:50.863981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:50.863992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:50.863997Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:50.864048Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e991f74f-c4933097-560537d9-dfefcc6b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:50.864062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:50.864067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:50.864070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:50.864079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:50.864091Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:50.864121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:50.864332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:50.864340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:50.864350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:50.864432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2266 partition: 0 MaxSeqNo: 25 sessionId: src_id|e991f74f-c4933097-560537d9-dfefcc6b_0
2025-10-09T21:51:50.864760Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a31556ad-2309f4e9-8d52bb1b-b10474b1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e991f74f-c4933097-560537d9-dfefcc6b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:50.867053Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e991f74f-c4933097-560537d9-dfefcc6b_0] Write session: try to update token
2025-10-09T21:51:50.867071Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e991f74f-c4933097-560537d9-dfefcc6b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:50.878395Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:50.878414Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:50.878416Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:50.878418Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:50.878426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343059770425951:12091]
2025-10-09T21:51:50.878430Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343059770425951:12091]
2025-10-09T21:51:50.878434Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343059770425951:12091]
2025-10-09T21:51:50.878469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:50.878476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:50.878478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:50.878481Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:50.878489Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:50.878493Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:50.878622Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:50.880399Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2266 sessionId: src_id|e991f74f-c4933097-560537d9-dfefcc6b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:50.882641Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:50.883816Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:51:50.883842Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 18
2025-10-09T21:51:50.942223Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343060132277453:8652]
2025-10-09T21:51:50.942566Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 2265, pipe: [1:7559343060132277453:8652]
2025-10-09T21:51:51.023025Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:51.023569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.023591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.023640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343060132277453:8652], now have 1 active actors on pipe
2025-10-09T21:51:51.023919Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:51.023928Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:51.023928Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343060132277453:8652]
2025-10-09T21:51:51.023956Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:51.023959Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:51.023960Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 18
2025-10-09T21:51:51.024043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2266 reason: error at write: Disconnected sessionId: src_id|e991f74f-c4933097-560537d9-dfefcc6b_0
2025-10-09T21:51:51.024197Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2266 sessionId: src_id|e991f74f-c4933097-560537d9-dfefcc6b_0 is DEAD
2025-10-09T21:51:51.024358Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e991f74f-c4933097-560537d9-dfefcc6b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:51.024373Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e991f74f-c4933097-560537d9-dfefcc6b_0] Write session will restart in 0.008764s
2025-10-09T21:51:51.024398Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e991f74f-c4933097-560537d9-dfefcc6b_0] Write session: Do CDS request
2025-10-09T21:51:51.024404Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e991f74f-c4933097-560537d9-dfefcc6b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:51.033215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:51.034532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e991f74f-c4933097-560537d9-dfefcc6b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:51.034619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:51.034639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2267
2025-10-09T21:51:51.034752Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2267 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:51.034787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2267 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:51.034798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2267 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:51.034804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:51.034865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:51.035016Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:51.035034Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:51.035048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:51.035089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:51.035126Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343064065393351:12094]
2025-10-09T21:51:51.035206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2267 sessionId:  describe result for acl check
2025-10-09T21:51:51.035245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:51.035252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:51.035254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:51.035260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343064427244813:8176] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:51.035571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:51.035833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:51.035864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:51.035864Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343064427244813:8176] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:51.036112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:51.036160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.036167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.036167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.036168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.036169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.036169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.036170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.036171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.036172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.036175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.036182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.036185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.036187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.036189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.036192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.036194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.036196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.036199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.036206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.036214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.036221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.036228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.036233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.036237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.036240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.036243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.036247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.036250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.036253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.036257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.036260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.036265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.036268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.036272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.036275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.036279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.036297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:51.037789Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343064427244813:8176] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:51.041959Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343064427244813:8176] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:51.041974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343064427244813:8176] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:51.041982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343064427244813:8176] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:51.041989Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2267 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:51.042578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:51.044050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:51.044064Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:51.044067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:51.044070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:51.044078Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2266 [2:7559343064065393351:12094]
2025-10-09T21:51:51.044089Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:51.044113Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:51.044120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:51.044122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:51.044125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:51.044139Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:51.044144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:51.044149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:51.044195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.044202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.044204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.044206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:51.044271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:51.044280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.044288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.044294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.044304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:51.044317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427244752:2432], now have 1 active actors on pipe
2025-10-09T21:51:51.044321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427244850:8176], now have 1 active actors on pipe
2025-10-09T21:51:51.044326Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343064065393345:12094]
2025-10-09T21:51:51.044333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343064065393345:12094]
2025-10-09T21:51:51.044335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343064065393345:12094]
2025-10-09T21:51:51.044346Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:51.044362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:51.044374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:51.044376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:51.044378Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:51.044381Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:51.044384Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:51.044403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343064065393345:12094]
2025-10-09T21:51:51.044396Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2266
2025-10-09T21:51:51.044415Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:51.044414Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2266
2025-10-09T21:51:51.044420Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:51.044444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.044447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.044506Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4d638cec-8e27b86e-fc2ded6a-82d90e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:51.044520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:51.044524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:51.044528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:51.044535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:51.044544Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:51.044624Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:51:51.044649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:51.044834Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 23 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:51:51.044845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.044872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.044866Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:51:51.044892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:51.044993Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2267 partition: 0 MaxSeqNo: 25 sessionId: src_id|4d638cec-8e27b86e-fc2ded6a-82d90e_0
2025-10-09T21:51:51.045299Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e991f74f-c4933097-560537d9-dfefcc6b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4d638cec-8e27b86e-fc2ded6a-82d90e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:51.047722Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4d638cec-8e27b86e-fc2ded6a-82d90e_0] Write session: try to update token
2025-10-09T21:51:51.047744Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4d638cec-8e27b86e-fc2ded6a-82d90e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:51.058720Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:51.058743Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:51.058746Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:51.058749Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:51.058761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343064065393345:12094]
2025-10-09T21:51:51.058766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343064065393345:12094]
2025-10-09T21:51:51.058772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343064065393345:12094]
2025-10-09T21:51:51.058820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:51.058830Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:51.058836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.058838Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:51.058841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:51.058842Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:51.058997Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:51.060664Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2267 sessionId: src_id|4d638cec-8e27b86e-fc2ded6a-82d90e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:51.063107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:51.123950Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343064427244861:8652]
2025-10-09T21:51:51.124251Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 2266, pipe: [1:7559343064427244861:8652]
2025-10-09T21:51:51.207817Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:51.208344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.208356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.208389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427244861:8652], now have 1 active actors on pipe
2025-10-09T21:51:51.208601Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:51.208620Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:51.208601Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343064427244861:8652]
2025-10-09T21:51:51.208629Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:51.208632Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:51.208708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2267 reason: error at write: Disconnected sessionId: src_id|4d638cec-8e27b86e-fc2ded6a-82d90e_0
2025-10-09T21:51:51.208914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2267 sessionId: src_id|4d638cec-8e27b86e-fc2ded6a-82d90e_0 is DEAD
2025-10-09T21:51:51.209069Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d638cec-8e27b86e-fc2ded6a-82d90e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:51.209079Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d638cec-8e27b86e-fc2ded6a-82d90e_0] Write session will restart in 0.005565s
2025-10-09T21:51:51.209104Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d638cec-8e27b86e-fc2ded6a-82d90e_0] Write session: Do CDS request
2025-10-09T21:51:51.209117Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d638cec-8e27b86e-fc2ded6a-82d90e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:51.215601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4d638cec-8e27b86e-fc2ded6a-82d90e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:51.215685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:51.215702Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2268
2025-10-09T21:51:51.215844Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2268 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:51.215880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2268 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:51.215893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2268 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:51.215901Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:51.216379Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2268 sessionId:  describe result for acl check
2025-10-09T21:51:51.216424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:51.216431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:51.216433Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:51.216443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343064427244925:8163] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:51.217009Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343064427244925:8163] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:51.217390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:51.219330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343064427244925:8163] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:51.219359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:51.219536Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:51.219553Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:51.219566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:51.219630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:51.219664Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343064065393449:12097]
2025-10-09T21:51:51.220191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:51.220514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:51.220584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:51.220939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:51.221001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.221010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.221011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.221012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.221013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.221014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.221016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.221017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.221019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.221024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.221036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.221041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.221045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.221049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.221054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.221058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.221062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.221067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.221075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.221084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.221090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.221096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.221100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.221106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.221110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.221116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.221121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.221126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.221131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.221136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.221142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.221147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.221151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.221156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.221161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.221167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.221186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:51.222146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343064427244925:8163] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:51.222164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343064427244925:8163] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:51.222168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343064427244925:8163] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:51.222177Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2268 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:51.227944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:51.229122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:51.229137Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:51.229143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:51.229147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:51.229153Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2267 [2:7559343064065393449:12097]
2025-10-09T21:51:51.229165Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:51.229184Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:51.229195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:51.229198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:51.229201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:51.229219Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:51.229224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:51.229230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:51.229387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.229397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.229399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.229401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:51.229469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:51.229478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.229488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.229495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.229506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:51.229523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343064065393444:12097]
2025-10-09T21:51:51.229525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343064065393444:12097]
2025-10-09T21:51:51.229526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343064065393444:12097]
2025-10-09T21:51:51.229535Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:51.229546Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:51.229548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427244967:8163], now have 1 active actors on pipe
2025-10-09T21:51:51.229553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427244867:2432], now have 1 active actors on pipe
2025-10-09T21:51:51.229558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:51.229558Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:51.229561Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:51.229567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:51.229569Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:51.229608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343064065393444:12097]
2025-10-09T21:51:51.229623Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:51.229630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:51.229634Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2267
2025-10-09T21:51:51.229634Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2267
2025-10-09T21:51:51.229687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.229698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.229728Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a79f2c1f-22d5e2ff-6c06e188-31b77f3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:51.229741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:51.229744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:51.229747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:51.229755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:51.229769Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:51.229786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:51.229940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.229948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.229963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:51.230066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2268 partition: 0 MaxSeqNo: 25 sessionId: src_id|a79f2c1f-22d5e2ff-6c06e188-31b77f3_0
2025-10-09T21:51:51.230461Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d638cec-8e27b86e-fc2ded6a-82d90e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a79f2c1f-22d5e2ff-6c06e188-31b77f3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:51.233177Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a79f2c1f-22d5e2ff-6c06e188-31b77f3_0] Write session: try to update token
2025-10-09T21:51:51.233203Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a79f2c1f-22d5e2ff-6c06e188-31b77f3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:51.247317Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2268 sessionId: src_id|a79f2c1f-22d5e2ff-6c06e188-31b77f3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:51.247805Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:51.247818Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:51.247825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343064065393444:12097]
2025-10-09T21:51:51.247873Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:51.247881Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:51.248093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:51.248247Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:51.248392Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:51.248530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:51.248788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:51.248834Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:51.249837Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:51.250081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.250100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.250148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:51.250245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:51.255587Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:51.255604Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:51.255606Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:51.255607Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:51.255617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343064065393444:12097]
2025-10-09T21:51:51.255620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343064065393444:12097]
2025-10-09T21:51:51.255623Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343064065393444:12097]
2025-10-09T21:51:51.255658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:51.255664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.255666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:51.255668Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:51.255675Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:51.255680Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:51.255814Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:51.308777Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343064427244979:8652]
2025-10-09T21:51:51.309018Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 2267, pipe: [1:7559343064427244979:8652]
2025-10-09T21:51:51.399012Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:51.405656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.405687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.405852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.405908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.405964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.406045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.406102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.406114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.406178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.406231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.406293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.406350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.406391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.406398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.406458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.406515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.406569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.406624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.406664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.406672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.406727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.407031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.407087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.407143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.407186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.407193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.407370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.407452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.407532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.407613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.407653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.407663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.407719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.407775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.407831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.407884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.407939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.407947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.408016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.408077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.408144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.408229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.408269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.408276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.408356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.408409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.408511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.408566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.408618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.408624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.408678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.408731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.408786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.408847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.408901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.408908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.408964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.409017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.409071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.409127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.409167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.409176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.409219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427244979:8652], now have 1 active actors on pipe
2025-10-09T21:51:51.409474Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:51.409487Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:51.409486Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:51.409515Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2268 reason: error at write: Disconnected sessionId: src_id|a79f2c1f-22d5e2ff-6c06e188-31b77f3_0
2025-10-09T21:51:51.409542Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343064427244979:8652]
2025-10-09T21:51:51.409752Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:51.409912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2268 sessionId: src_id|a79f2c1f-22d5e2ff-6c06e188-31b77f3_0 is DEAD
2025-10-09T21:51:51.410089Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a79f2c1f-22d5e2ff-6c06e188-31b77f3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:51.410099Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a79f2c1f-22d5e2ff-6c06e188-31b77f3_0] Write session will restart in 0.006705s
2025-10-09T21:51:51.410118Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a79f2c1f-22d5e2ff-6c06e188-31b77f3_0] Write session: Do CDS request
2025-10-09T21:51:51.410123Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a79f2c1f-22d5e2ff-6c06e188-31b77f3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:51.417573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a79f2c1f-22d5e2ff-6c06e188-31b77f3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:51.417660Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:51.417677Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2269
2025-10-09T21:51:51.417710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:51.417813Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2269 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:51.417846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2269 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:51.417857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2269 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:51.417866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:51.418307Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2269 sessionId:  describe result for acl check
2025-10-09T21:51:51.418340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:51.418342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:51.418346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:51.418351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343064427245047:8140] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:51.418916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343064427245047:8140] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:51.419337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:51.419528Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:51.419541Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:51.419553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:51.419603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:51.419647Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343064065393548:12099]
2025-10-09T21:51:51.420247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:51.420573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:51.420616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:51.420872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:51.420914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.420918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.420919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.420920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.420920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.420921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.420922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.420922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.420923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.420926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.420933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.420936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.420938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.420940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.420942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.420944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.420946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.420948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.420953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.420958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.420962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.420966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.420969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.420972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.420975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.420978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.420981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.420984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.420987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.420990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.420994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.420997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.420999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.421002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.421005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.421008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.421024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:51.421123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343064427245047:8140] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:51.426766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343064427245047:8140] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:51.426784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343064427245047:8140] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:51.426788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343064427245047:8140] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:51.426797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2269 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:51.427125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:51.428177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:51.428193Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:51.428196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:51.428201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:51.428209Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2268 [2:7559343064065393548:12099]
2025-10-09T21:51:51.428226Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:51.428248Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:51.428255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:51.428258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:51.428259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:51.428275Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:51.428282Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:51.428293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:51.428361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.428368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.428370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.428373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:51.428441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:51.428449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.428463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.428477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.428499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:51.428525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427244985:2432], now have 1 active actors on pipe
2025-10-09T21:51:51.428533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427245086:8140], now have 1 active actors on pipe
2025-10-09T21:51:51.428540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343064065393543:12099]
2025-10-09T21:51:51.428546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343064065393543:12099]
2025-10-09T21:51:51.428548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343064065393543:12099]
2025-10-09T21:51:51.428541Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2268
2025-10-09T21:51:51.428558Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:51.428568Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2268
2025-10-09T21:51:51.428576Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:51.428578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:51.428584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:51.428586Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:51.428592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:51.428595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:51.428616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.428627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.428671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343064065393543:12099]
2025-10-09T21:51:51.428684Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:51.428694Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:51.428707Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|45594795-606dd511-ae0e79d7-2899be9e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:51.428719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:51.428727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:51.428734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:51.428742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:51.428753Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:51.428788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:51.428912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.428921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.428935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:51.429003Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2269 partition: 0 MaxSeqNo: 25 sessionId: src_id|45594795-606dd511-ae0e79d7-2899be9e_0
2025-10-09T21:51:51.429332Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a79f2c1f-22d5e2ff-6c06e188-31b77f3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|45594795-606dd511-ae0e79d7-2899be9e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:51.431727Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45594795-606dd511-ae0e79d7-2899be9e_0] Write session: try to update token
2025-10-09T21:51:51.431743Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45594795-606dd511-ae0e79d7-2899be9e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:51.440252Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:51.440272Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:51.440282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343064065393543:12099]
2025-10-09T21:51:51.440318Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:51.440329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:51.440519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:51.440701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:51.440872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:51.440995Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:51.441174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:51.441214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:51.444583Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2269 sessionId: src_id|45594795-606dd511-ae0e79d7-2899be9e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:51.446810Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:51.446991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.447013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.447057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:51.447113Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:51.453376Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:51.453392Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:51.453395Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:51.453397Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:51.453407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343064065393543:12099]
2025-10-09T21:51:51.453414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343064065393543:12099]
2025-10-09T21:51:51.453418Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343064065393543:12099]
2025-10-09T21:51:51.453455Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:51.453461Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:51.453461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:51.453463Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:51.453468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.453472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:51.453606Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:51.509498Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343064427245094:8652]
2025-10-09T21:51:51.509709Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 2268, pipe: [1:7559343064427245094:8652]
2025-10-09T21:51:51.606116Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:51.612984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.613009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.613159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.613215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.613275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.613350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.613733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.613743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.613810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.613863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.613921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.613984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.614022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.614042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.614098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.614154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.614208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.614297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.614339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.614348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.614410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.614469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.614530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.614598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.614647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.614655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.614725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.614780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.614842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.614900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.614940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.614947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.615006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.615061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.615118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.615203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.615249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.615255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.615341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.615417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.615474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.615530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.615568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.615573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.615629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.615684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.615740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.615792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.615833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.615839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.615893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.615945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.616075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.616134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.616174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.616179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.616253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:51.616308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:51.616364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:51.616420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:51.616472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:51.616478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:51.616545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427245094:8652], now have 1 active actors on pipe
2025-10-09T21:51:51.616775Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:51.616783Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343064427245094:8652]
2025-10-09T21:51:51.616813Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:51.616817Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:51.616848Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2269 reason: error at write: Disconnected sessionId: src_id|45594795-606dd511-ae0e79d7-2899be9e_0
2025-10-09T21:51:51.617047Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:51.617084Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2269 sessionId: src_id|45594795-606dd511-ae0e79d7-2899be9e_0 is DEAD
2025-10-09T21:51:51.617211Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45594795-606dd511-ae0e79d7-2899be9e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:51.617222Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45594795-606dd511-ae0e79d7-2899be9e_0] Write session will restart in 0.008721s
2025-10-09T21:51:51.617250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45594795-606dd511-ae0e79d7-2899be9e_0] Write session: Do CDS request
2025-10-09T21:51:51.617257Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45594795-606dd511-ae0e79d7-2899be9e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:51.625704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:51.627137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:51.627287Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:51.627295Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:51.627303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:51.627339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:51.627370Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343064065393687:12113]
2025-10-09T21:51:51.627459Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45594795-606dd511-ae0e79d7-2899be9e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:51.627567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:51.627587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2270
2025-10-09T21:51:51.627739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:51.627989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:51.628023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:51.628033Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2270 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:51.628065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2270 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:51.628077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2270 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:51.628081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:51.628238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:51.628272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.628277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.628278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.628278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.628279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.628279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.628280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.628281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.628281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.628285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.628292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.628294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.628297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.628299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.628301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.628305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.628309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.628312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.628320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.628327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.628334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.628338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.628342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.628347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.628351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.628355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.628360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.628365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.628369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.628375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.628380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.628383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.628386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.628389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.628392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.628395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.628412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:51.628494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2270 sessionId:  describe result for acl check
2025-10-09T21:51:51.628531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:51.628538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:51.628540Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:51.628546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343064427245186:8122] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:51.629115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343064427245186:8122] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:51.631270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343064427245186:8122] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:51.634273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343064427245186:8122] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:51.634288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343064427245186:8122] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:51.634291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343064427245186:8122] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:51.634300Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2270 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:51.634619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:51.635683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:51.635699Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:51.635703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:51.635707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:51.635713Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2269 [2:7559343064065393687:12113]
2025-10-09T21:51:51.635729Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:51.635746Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:51.635753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:51.635756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:51.635758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:51.635774Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:51.635785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:51.635792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:51.635858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.635866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.635867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.635870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:51.635943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:51.635950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.635962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.635975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.635996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:51.636020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427245219:8122], now have 1 active actors on pipe
2025-10-09T21:51:51.636028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427245120:2432], now have 1 active actors on pipe
2025-10-09T21:51:51.636035Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343064065393682:12113]
2025-10-09T21:51:51.636038Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343064065393682:12113]
2025-10-09T21:51:51.636040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343064065393682:12113]
2025-10-09T21:51:51.636054Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:51.636056Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2269
2025-10-09T21:51:51.636068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:51.636071Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:51.636074Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:51.636080Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2269
2025-10-09T21:51:51.636076Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:51.636089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:51.636093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:51.636122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343064065393682:12113]
2025-10-09T21:51:51.636139Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:51.636145Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:51.636158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.636163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.636209Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|80c324cb-55add5e7-ca74c6e4-e7253941_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:51.636223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:51.636227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:51.636230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:51.636240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:51.636251Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:51.636273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:51.636529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.636542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.636564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:51.636664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2270 partition: 0 MaxSeqNo: 25 sessionId: src_id|80c324cb-55add5e7-ca74c6e4-e7253941_0
2025-10-09T21:51:51.637000Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45594795-606dd511-ae0e79d7-2899be9e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|80c324cb-55add5e7-ca74c6e4-e7253941_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:51.639548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80c324cb-55add5e7-ca74c6e4-e7253941_0] Write session: try to update token
2025-10-09T21:51:51.639567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80c324cb-55add5e7-ca74c6e4-e7253941_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:51.650959Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:51.650980Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:51.650982Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:51.650985Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:51.650993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343064065393682:12113]
2025-10-09T21:51:51.650998Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343064065393682:12113]
2025-10-09T21:51:51.651002Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343064065393682:12113]
2025-10-09T21:51:51.651047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:51.651054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.651055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:51.651064Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:51.651070Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:51.651075Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:51.651205Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:51.652785Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2270 sessionId: src_id|80c324cb-55add5e7-ca74c6e4-e7253941_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:51.655204Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:51.716853Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343064427245235:8652]
2025-10-09T21:51:51.717090Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 2269, pipe: [1:7559343064427245235:8652]
2025-10-09T21:51:51.799134Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:51.799543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.799560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.799606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427245235:8652], now have 1 active actors on pipe
2025-10-09T21:51:51.799849Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:51.799854Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343064427245235:8652]
2025-10-09T21:51:51.799863Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:51.799866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:51.799894Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:51.799928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2270 reason: error at write: Disconnected sessionId: src_id|80c324cb-55add5e7-ca74c6e4-e7253941_0
2025-10-09T21:51:51.800058Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2270 sessionId: src_id|80c324cb-55add5e7-ca74c6e4-e7253941_0 is DEAD
2025-10-09T21:51:51.800202Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80c324cb-55add5e7-ca74c6e4-e7253941_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:51.800212Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80c324cb-55add5e7-ca74c6e4-e7253941_0] Write session will restart in 0.009252s
2025-10-09T21:51:51.800233Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80c324cb-55add5e7-ca74c6e4-e7253941_0] Write session: Do CDS request
2025-10-09T21:51:51.800238Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80c324cb-55add5e7-ca74c6e4-e7253941_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:51.808352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:51.809796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:51.809954Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:51.809964Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:51.809972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:51.810016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:51.810063Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343064065393792:12115]
2025-10-09T21:51:51.810467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:51.810493Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:51.810515Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2271
2025-10-09T21:51:51.810526Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|80c324cb-55add5e7-ca74c6e4-e7253941_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:51.810717Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2271 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:51.810728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:51.810752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2271 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:51.810760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2271 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:51.810764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:51.810765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:51.810972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:51.811007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.811012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.811013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.811014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.811015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.811015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.811016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.811017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.811017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.811021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.811028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.811030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.811032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.811034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.811037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.811039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.811041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.811043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.811048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.811053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.811057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.811060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.811063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.811066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.811069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.811072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.811074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.811078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.811080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.811083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.811086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.811125Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2271 sessionId:  describe result for acl check
2025-10-09T21:51:51.811091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.811093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.811155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:51.811096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.811156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:51.811100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.811104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.811120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:51.811160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:51.811164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343064427245308:8072] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:51.811571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343064427245308:8072] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:51.813378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343064427245308:8072] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:51.816521Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343064427245308:8072] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:51.816533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343064427245308:8072] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:51.816536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343064427245308:8072] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:51.816542Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2271 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:51.816970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:51.818021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:51.818048Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:51.818049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:51.818052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:51.818057Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2270 [2:7559343064065393792:12115]
2025-10-09T21:51:51.818066Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:51.818080Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:51.818082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:51.818084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:51.818085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:51.818098Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:51.818101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:51.818107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:51.818149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.818156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.818158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.818161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:51.818224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:51.818233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.818244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.818261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.818282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:51.818304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427245242:2432], now have 1 active actors on pipe
2025-10-09T21:51:51.818311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427245345:8072], now have 1 active actors on pipe
2025-10-09T21:51:51.818317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343064065393787:12115]
2025-10-09T21:51:51.818318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343064065393787:12115]
2025-10-09T21:51:51.818320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343064065393787:12115]
2025-10-09T21:51:51.818331Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:51.818346Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:51.818349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:51.818356Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:51.818358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:51.818363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:51.818366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:51.818392Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343064065393787:12115]
2025-10-09T21:51:51.818378Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2270
2025-10-09T21:51:51.818400Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:51.818402Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:51.818414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.818380Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2270
2025-10-09T21:51:51.818418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.818465Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7c93be0b-d22c3688-c819f398-c4ad4c72_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:51.818478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:51.818483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:51.818485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:51.818495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:51.818511Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:51.818533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:51.818667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.818678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.818694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:51.818774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2271 partition: 0 MaxSeqNo: 25 sessionId: src_id|7c93be0b-d22c3688-c819f398-c4ad4c72_0
2025-10-09T21:51:51.819080Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|80c324cb-55add5e7-ca74c6e4-e7253941_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7c93be0b-d22c3688-c819f398-c4ad4c72_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:51.821377Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c93be0b-d22c3688-c819f398-c4ad4c72_0] Write session: try to update token
2025-10-09T21:51:51.821391Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c93be0b-d22c3688-c819f398-c4ad4c72_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:51.832791Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:51.832810Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:51.832813Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:51.832815Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:51.832828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343064065393787:12115]
2025-10-09T21:51:51.832832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343064065393787:12115]
2025-10-09T21:51:51.832836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343064065393787:12115]
2025-10-09T21:51:51.832875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:51.832880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.832881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:51.832889Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:51.832896Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:51.832901Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:51.833028Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:51.835248Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2271 sessionId: src_id|7c93be0b-d22c3688-c819f398-c4ad4c72_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:51.837578Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:51.900154Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343064427245353:8652]
2025-10-09T21:51:51.900436Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 2270, pipe: [1:7559343064427245353:8652]
2025-10-09T21:51:51.979283Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:51.979636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.979649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.979688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427245353:8652], now have 1 active actors on pipe
2025-10-09T21:51:51.979880Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:51.979887Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:51.979895Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:51.979924Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343064427245353:8652]
2025-10-09T21:51:51.979983Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2271 reason: error at write: Disconnected sessionId: src_id|7c93be0b-d22c3688-c819f398-c4ad4c72_0
2025-10-09T21:51:51.980055Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:51.980160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2271 sessionId: src_id|7c93be0b-d22c3688-c819f398-c4ad4c72_0 is DEAD
2025-10-09T21:51:51.980312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c93be0b-d22c3688-c819f398-c4ad4c72_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:51.980323Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c93be0b-d22c3688-c819f398-c4ad4c72_0] Write session will restart in 0.006106s
2025-10-09T21:51:51.980347Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c93be0b-d22c3688-c819f398-c4ad4c72_0] Write session: Do CDS request
2025-10-09T21:51:51.980354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c93be0b-d22c3688-c819f398-c4ad4c72_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:51.987502Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c93be0b-d22c3688-c819f398-c4ad4c72_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:51.987571Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:51.987590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2272
2025-10-09T21:51:51.987697Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2272 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:51.987729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2272 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:51.987739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2272 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:51.987743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:51.988087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:51.988146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2272 sessionId:  describe result for acl check
2025-10-09T21:51:51.988187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:51.988197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:51.988199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:51.988204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343064427245423:8047] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:51.988750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343064427245423:8047] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:51.989671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:51.989852Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:51.989861Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:51.989873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:51.989922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:51.989957Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343064065393895:12119]
2025-10-09T21:51:51.990511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:51.990555Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343064427245423:8047] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:51.990823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:51.990864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:51.991153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:51.991200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.991205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.991207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.991208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.991209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.991210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.991211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.991212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.991214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.991218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.991226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.991229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.991238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.991241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.991249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.991252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.991255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.991262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.991268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.991281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:51.991287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.991292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:51.991296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.991300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:51.991305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.991309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:51.991319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.991324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:51.991333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.991340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:51.991348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.991352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.991357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.991366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.991370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.991375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.991408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:51.996250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343064427245423:8047] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:51.996266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343064427245423:8047] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:51.996269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343064427245423:8047] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:51.996279Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2272 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:51.997582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:51.998810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:51.998827Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:51.998829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:51.998834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:51.998841Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2271 [2:7559343064065393895:12119]
2025-10-09T21:51:51.998855Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:51.998870Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:51.998879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:51.998883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:51.998885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:51.998901Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:51.998912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:51.998919Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:51.998985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.998994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.998996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.998999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:51.999069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:51.999077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:51.999089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:51.999102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:51.999124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:51.999147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427245359:2432], now have 1 active actors on pipe
2025-10-09T21:51:51.999155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343064427245460:8047], now have 1 active actors on pipe
2025-10-09T21:51:51.999163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343064065393890:12119]
2025-10-09T21:51:51.999170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343064065393890:12119]
2025-10-09T21:51:51.999172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343064065393890:12119]
2025-10-09T21:51:51.999181Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:51.999169Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2271
2025-10-09T21:51:51.999197Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:51.999197Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:51.999198Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2271
2025-10-09T21:51:51.999201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:51.999203Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:51.999216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:51.999219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:51.999261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.999270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.999283Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343064065393890:12119]
2025-10-09T21:51:51.999295Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:51.999307Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:51.999328Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a249f895-8c348705-664906d-e0f9475b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:51.999339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:51.999344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:51.999346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:51.999356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:51.999371Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:51.999397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:51.999551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:51.999564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:51.999579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:51.999660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2272 partition: 0 MaxSeqNo: 25 sessionId: src_id|a249f895-8c348705-664906d-e0f9475b_0
2025-10-09T21:51:51.999982Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c93be0b-d22c3688-c819f398-c4ad4c72_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a249f895-8c348705-664906d-e0f9475b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:52.002381Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a249f895-8c348705-664906d-e0f9475b_0] Write session: try to update token
2025-10-09T21:51:52.002400Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a249f895-8c348705-664906d-e0f9475b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:52.013766Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:52.013788Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:52.013791Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:52.013793Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:52.013803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343064065393890:12119]
2025-10-09T21:51:52.013808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343064065393890:12119]
2025-10-09T21:51:52.013812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343064065393890:12119]
2025-10-09T21:51:52.013860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:52.013878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.013886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:52.013903Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:52.013911Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:52.013917Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:52.014045Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:52.015362Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2272 sessionId: src_id|a249f895-8c348705-664906d-e0f9475b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:52.017663Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:52.079870Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343068722212764:8652]
2025-10-09T21:51:52.080236Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 2271, pipe: [1:7559343068722212764:8652]
2025-10-09T21:51:52.162159Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:52.162403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:52.162415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:52.162454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343068722212764:8652], now have 1 active actors on pipe
2025-10-09T21:51:52.162630Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:52.162659Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:52.162703Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343068722212764:8652]
2025-10-09T21:51:52.162706Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:52.162780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2272 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a249f895-8c348705-664906d-e0f9475b_0
2025-10-09T21:51:52.162931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2272 sessionId: src_id|a249f895-8c348705-664906d-e0f9475b_0 is DEAD
2025-10-09T21:51:52.163074Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a249f895-8c348705-664906d-e0f9475b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:52.163086Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a249f895-8c348705-664906d-e0f9475b_0] Write session will restart in 0.006671s
2025-10-09T21:51:52.163107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a249f895-8c348705-664906d-e0f9475b_0] Write session: Do CDS request
2025-10-09T21:51:52.163112Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a249f895-8c348705-664906d-e0f9475b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:52.170552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a249f895-8c348705-664906d-e0f9475b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:52.170664Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:52.170683Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2273
2025-10-09T21:51:52.170884Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2273 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:52.170917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2273 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:52.170927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2273 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:52.170931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:52.171289Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2273 sessionId:  describe result for acl check
2025-10-09T21:51:52.171322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:52.171327Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:52.171328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:52.171332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343068722212832:8034] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:52.171588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:52.171796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343068722212832:8034] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:52.173083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:52.173237Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:52.173244Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:52.173254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:52.173299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:52.173349Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343068360361294:12121]
2025-10-09T21:51:52.173862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343068722212832:8034] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:52.173920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:52.174229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:52.174269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:52.174562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:52.174611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.174619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.174621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.174622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.174623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.174624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.174626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.174627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.174628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.174633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.174643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.174646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.174651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.174655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.174659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.174663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.174667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.174671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.174676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.174684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.174691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.174697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.174703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.174708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.174712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.174717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.174722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.174727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.174731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.174737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.174741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.174746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.174752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.174758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.174768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.174773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.174800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:52.179548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343068722212832:8034] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:52.179565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343068722212832:8034] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:52.179569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343068722212832:8034] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:52.179578Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2273 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:52.180850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:52.181905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:52.181920Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:52.181923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:52.181927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:52.181934Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2272 [2:7559343068360361294:12121]
2025-10-09T21:51:52.181952Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:52.181972Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:52.181977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:52.181980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:52.181982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:52.181994Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:52.181999Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:52.182009Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:52.182089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.182096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.182097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.182101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:52.182169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:52.182176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.182187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.182202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.182221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:52.182246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343068722212872:8034], now have 1 active actors on pipe
2025-10-09T21:51:52.182254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343068722212766:2432], now have 1 active actors on pipe
2025-10-09T21:51:52.182262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343068360361288:12121]
2025-10-09T21:51:52.182264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343068360361288:12121]
2025-10-09T21:51:52.182266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343068360361288:12121]
2025-10-09T21:51:52.182279Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:52.182295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:52.182278Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2272
2025-10-09T21:51:52.182294Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:52.182284Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2272
2025-10-09T21:51:52.182297Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:52.182300Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:52.182308Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:52.182310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:52.182347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:52.182354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:52.182364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343068360361288:12121]
2025-10-09T21:51:52.182373Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:52.182383Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:52.182401Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e72a0f09-119bd5f5-f4f7bf8e-ffdb302c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:52.182410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:52.182417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:52.182419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:52.182427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:52.182439Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:52.182460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:52.182629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:52.182637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:52.182649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:52.182728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2273 partition: 0 MaxSeqNo: 25 sessionId: src_id|e72a0f09-119bd5f5-f4f7bf8e-ffdb302c_0
2025-10-09T21:51:52.183058Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a249f895-8c348705-664906d-e0f9475b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e72a0f09-119bd5f5-f4f7bf8e-ffdb302c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:52.185332Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e72a0f09-119bd5f5-f4f7bf8e-ffdb302c_0] Write session: try to update token
2025-10-09T21:51:52.185347Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e72a0f09-119bd5f5-f4f7bf8e-ffdb302c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:52.196318Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:52.196338Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:52.196341Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:52.196342Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:52.196351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343068360361288:12121]
2025-10-09T21:51:52.196356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343068360361288:12121]
2025-10-09T21:51:52.196361Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343068360361288:12121]
2025-10-09T21:51:52.196406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:52.196412Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:52.196414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.196417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:52.196417Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:52.196422Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:52.196558Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:52.198585Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2273 sessionId: src_id|e72a0f09-119bd5f5-f4f7bf8e-ffdb302c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:52.201167Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:52.262644Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343068722212882:8652]
2025-10-09T21:51:52.262940Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 2272, pipe: [1:7559343068722212882:8652]
2025-10-09T21:51:52.339517Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:52.339844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:52.339857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:52.339896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343068722212882:8652], now have 1 active actors on pipe
2025-10-09T21:51:52.340065Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:52.340108Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:52.340120Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343068722212882:8652]
2025-10-09T21:51:52.340124Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:52.340148Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:52.340189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2273 reason: error at write: Disconnected sessionId: src_id|e72a0f09-119bd5f5-f4f7bf8e-ffdb302c_0
2025-10-09T21:51:52.340378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2273 sessionId: src_id|e72a0f09-119bd5f5-f4f7bf8e-ffdb302c_0 is DEAD
2025-10-09T21:51:52.340519Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e72a0f09-119bd5f5-f4f7bf8e-ffdb302c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:52.340529Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e72a0f09-119bd5f5-f4f7bf8e-ffdb302c_0] Write session will restart in 0.006544s
2025-10-09T21:51:52.340564Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e72a0f09-119bd5f5-f4f7bf8e-ffdb302c_0] Write session: Do CDS request
2025-10-09T21:51:52.340570Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e72a0f09-119bd5f5-f4f7bf8e-ffdb302c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:52.348167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:52.348525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e72a0f09-119bd5f5-f4f7bf8e-ffdb302c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:52.348610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:52.348627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2274
2025-10-09T21:51:52.348757Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2274 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:52.348789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2274 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:52.348797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2274 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:52.348800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:52.349148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2274 sessionId:  describe result for acl check
2025-10-09T21:51:52.349174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:52.349178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:52.349180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:52.349184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343068722212956:8022] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:52.349616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343068722212956:8022] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:52.349744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:52.349911Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:52.349922Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:52.349934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:52.349988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:52.350056Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343068360361398:12123]
2025-10-09T21:51:52.350614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:52.350848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:52.350885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:52.351173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:52.351223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.351230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.351231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.351232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.351233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.351235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.351236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.351237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.351238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.351244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.351254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.351258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.351261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.351266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.351270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.351274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.351278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.351287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.351293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.351305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.351312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.351322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.351327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.351336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.351340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.351345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.351354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.351359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.351363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.351372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.351355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343068722212956:8022] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:52.351377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.351387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.351390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.351395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.351400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.351405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.351434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:52.356187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343068722212956:8022] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:52.356202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343068722212956:8022] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:52.356206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343068722212956:8022] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:52.356215Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2274 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:52.357513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:52.358653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:52.358675Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:52.358677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:52.358681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:52.358688Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2273 [2:7559343068360361398:12123]
2025-10-09T21:51:52.358702Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:52.358721Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:52.358729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:52.358732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:52.358734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:52.358752Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:52.358758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:52.358766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:52.358835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.358844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.358846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.358849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:52.358919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:52.358927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.358939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.358951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.358969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:52.358996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343068722212996:8022], now have 1 active actors on pipe
2025-10-09T21:51:52.359004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343068722212890:2432], now have 1 active actors on pipe
2025-10-09T21:51:52.359012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343068360361393:12123]
2025-10-09T21:51:52.359018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343068360361393:12123]
2025-10-09T21:51:52.359020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343068360361393:12123]
2025-10-09T21:51:52.359033Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:52.359051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:52.359049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:52.359061Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:52.359064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:52.359066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:52.359069Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:52.359045Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2273
2025-10-09T21:51:52.359071Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2273
2025-10-09T21:51:52.359111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:52.359120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:52.359138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343068360361393:12123]
2025-10-09T21:51:52.359148Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:52.359152Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:52.359180Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|64a1c846-67f15b07-74ec5e69-73aec824_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:52.359191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:52.359200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:52.359202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:52.359211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:52.359225Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:52.359250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:52.359387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:52.359399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:52.359413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:52.359471Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2274 partition: 0 MaxSeqNo: 25 sessionId: src_id|64a1c846-67f15b07-74ec5e69-73aec824_0
2025-10-09T21:51:52.359771Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e72a0f09-119bd5f5-f4f7bf8e-ffdb302c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|64a1c846-67f15b07-74ec5e69-73aec824_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:52.362108Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|64a1c846-67f15b07-74ec5e69-73aec824_0] Write session: try to update token
2025-10-09T21:51:52.362123Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|64a1c846-67f15b07-74ec5e69-73aec824_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:52.372964Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:52.372981Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:52.372983Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:52.372984Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:52.372992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343068360361393:12123]
2025-10-09T21:51:52.372996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343068360361393:12123]
2025-10-09T21:51:52.372999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343068360361393:12123]
2025-10-09T21:51:52.373033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:52.373039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.373042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:52.373051Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:52.373074Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:52.373078Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:52.373163Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:52.374873Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2274 sessionId: src_id|64a1c846-67f15b07-74ec5e69-73aec824_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:52.377141Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:52.440279Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343068722213003:8652]
2025-10-09T21:51:52.440552Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 2273, pipe: [1:7559343068722213003:8652]
2025-10-09T21:51:52.518393Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:52.518767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:52.518781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:52.518824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343068722213003:8652], now have 1 active actors on pipe
2025-10-09T21:51:52.519019Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:52.519069Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343068722213003:8652]
2025-10-09T21:51:52.519090Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:52.519094Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:52.519099Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:52.519167Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2274 reason: error at write: Disconnected sessionId: src_id|64a1c846-67f15b07-74ec5e69-73aec824_0
2025-10-09T21:51:52.519387Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2274 sessionId: src_id|64a1c846-67f15b07-74ec5e69-73aec824_0 is DEAD
2025-10-09T21:51:52.519547Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|64a1c846-67f15b07-74ec5e69-73aec824_0] Counters: { Errors: 0 CurrentSessionLifetimeMs: 159 BytesWritten: 56320040 MessagesWritten: 25 BytesWrittenCompressed: 0 BytesInflightUncompressed: 22528020 BytesInflightCompressed: 0 BytesInflightTotal: 18022416 MessagesInflight: 10 }
2025-10-09T21:51:52.519575Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|64a1c846-67f15b07-74ec5e69-73aec824_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:52.519579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|64a1c846-67f15b07-74ec5e69-73aec824_0] Write session will restart in 0.006737s
2025-10-09T21:51:52.519598Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|64a1c846-67f15b07-74ec5e69-73aec824_0] Write session: Do CDS request
2025-10-09T21:51:52.519603Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|64a1c846-67f15b07-74ec5e69-73aec824_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:52.527626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|64a1c846-67f15b07-74ec5e69-73aec824_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:52.527631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:52.527649Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2275
2025-10-09T21:51:52.527824Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2275 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:52.527857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2275 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:52.527866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2275 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:52.527869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:52.528291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2275 sessionId:  describe result for acl check
2025-10-09T21:51:52.528323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:52.528332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:52.528333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:52.528338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343068722213072:8006] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:52.528837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343068722213072:8006] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:52.529452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:52.530971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343068722213072:8006] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:52.530998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:52.531196Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:52.531207Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:52.531216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:52.531270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:52.531310Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343068360361504:12125]
2025-10-09T21:51:52.531718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:52.531994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:52.532037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:52.532265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:52.532313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.532319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.532320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.532321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.532323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.532324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.532326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.532327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.532329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.532334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.532345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.532349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.532353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.532357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.532360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.532364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.532368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.532372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.532379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.532386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.532392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.532397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.532401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.532406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.532409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.532414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.532419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.532423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.532428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.532433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.532439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.532445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.532450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.532457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.532463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.532474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.532506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:52.537916Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343068722213072:8006] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:52.537940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343068722213072:8006] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:52.537945Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343068722213072:8006] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:52.537955Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2275 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:52.539107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:52.540341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:52.540360Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:52.540363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:52.540368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:52.540377Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2274 [2:7559343068360361504:12125]
2025-10-09T21:51:52.540400Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:52.540422Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:52.540431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:52.540435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:52.540437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:52.540458Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:52.540466Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:52.540472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:52.540533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.540539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.540541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.540543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:52.540608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:52.540614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.540623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.540634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.540644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:52.540664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343068360361498:12125]
2025-10-09T21:51:52.540669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343068360361498:12125]
2025-10-09T21:51:52.540671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343068360361498:12125]
2025-10-09T21:51:52.540680Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:52.540693Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:52.540697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343068722213006:2432], now have 1 active actors on pipe
2025-10-09T21:51:52.540700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343068722213113:8006], now have 1 active actors on pipe
2025-10-09T21:51:52.540702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:52.540704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:52.540716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:52.540727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:52.540730Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:52.540771Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343068360361498:12125]
2025-10-09T21:51:52.540777Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:52.540779Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:52.540748Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2274
2025-10-09T21:51:52.540761Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2274
2025-10-09T21:51:52.540811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:52.540818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:52.540850Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|537baedc-7758d387-8692fe9d-57ebd2ab_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:52.540861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:52.540864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:52.540866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:52.540875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:52.540886Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:52.540898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:52.541019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:52.541026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:52.541038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:52.541100Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2275 partition: 0 MaxSeqNo: 25 sessionId: src_id|537baedc-7758d387-8692fe9d-57ebd2ab_0
2025-10-09T21:51:52.541459Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|64a1c846-67f15b07-74ec5e69-73aec824_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|537baedc-7758d387-8692fe9d-57ebd2ab_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:52.543995Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|537baedc-7758d387-8692fe9d-57ebd2ab_0] Write session: try to update token
2025-10-09T21:51:52.544017Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|537baedc-7758d387-8692fe9d-57ebd2ab_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:52.556004Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:52.556024Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:52.556026Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:52.556027Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:52.556035Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343068360361498:12125]
2025-10-09T21:51:52.556040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343068360361498:12125]
2025-10-09T21:51:52.556044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343068360361498:12125]
2025-10-09T21:51:52.556083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:52.556091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.556093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:52.556100Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:52.556107Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:52.556111Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:52.556241Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:52.557638Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2275 sessionId: src_id|537baedc-7758d387-8692fe9d-57ebd2ab_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:52.559939Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:52.619019Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343068722213142:8652]
2025-10-09T21:51:52.619294Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 2274, pipe: [1:7559343068722213142:8652]
2025-10-09T21:51:52.704197Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:52.704595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:52.704610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:52.704648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343068722213142:8652], now have 1 active actors on pipe
2025-10-09T21:51:52.704884Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:52.704916Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:52.704991Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343068722213142:8652]
2025-10-09T21:51:52.705045Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:52.705146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2275 reason: status is not ok: tablet will be restarted right now sessionId: src_id|537baedc-7758d387-8692fe9d-57ebd2ab_0
2025-10-09T21:51:52.705327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2275 sessionId: src_id|537baedc-7758d387-8692fe9d-57ebd2ab_0 is DEAD
2025-10-09T21:51:52.705482Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|537baedc-7758d387-8692fe9d-57ebd2ab_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:52.705493Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|537baedc-7758d387-8692fe9d-57ebd2ab_0] Write session will restart in 0.009193s
2025-10-09T21:51:52.705513Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|537baedc-7758d387-8692fe9d-57ebd2ab_0] Write session: Do CDS request
2025-10-09T21:51:52.705518Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|537baedc-7758d387-8692fe9d-57ebd2ab_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:52.713532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:52.715135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:52.715287Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:52.715297Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:52.715304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:52.715338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:52.715372Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343068360361647:12139]
2025-10-09T21:51:52.715488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|537baedc-7758d387-8692fe9d-57ebd2ab_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:52.715604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:52.715634Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2276
2025-10-09T21:51:52.715774Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2276 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:52.715822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2276 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:52.715829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2276 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:52.715833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:52.715816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:52.716088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:52.716122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:52.716210Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2276 sessionId:  describe result for acl check
2025-10-09T21:51:52.716240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:52.716243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:52.716245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:52.716249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343068722213216:7974] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:52.716357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:52.716409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.716417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.716418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.716418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.716420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.716421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.716421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.716422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.716423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.716434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.716444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.716449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.716454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.716458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.716463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.716466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.716468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.716471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.716476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.716482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.716486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.716493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.716505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.716511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.716516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.716521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.716527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.716536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.716540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.716544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.716547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.716553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.716556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.716559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.716563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.716569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.716602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:52.716740Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343068722213216:7974] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:52.718846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343068722213216:7974] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:52.722286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343068722213216:7974] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:52.722301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343068722213216:7974] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:52.722304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343068722213216:7974] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:52.722312Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2276 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:52.723021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:52.724612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:52.724630Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:52.724633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:52.724638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:52.724646Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2275 [2:7559343068360361647:12139]
2025-10-09T21:51:52.724663Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:52.724680Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:52.724688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:52.724690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:52.724692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:52.724705Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:52.724708Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:52.724713Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:52.724768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.724776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.724778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.724779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:52.724833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:52.724841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.724850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.724857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.724881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:52.724902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343068722213145:2432], now have 1 active actors on pipe
2025-10-09T21:51:52.724910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343068722213253:7974], now have 1 active actors on pipe
2025-10-09T21:51:52.724915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343068360361642:12139]
2025-10-09T21:51:52.724918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343068360361642:12139]
2025-10-09T21:51:52.724920Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343068360361642:12139]
2025-10-09T21:51:52.724934Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:52.724947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:52.724959Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:52.724963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:52.724964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:52.724965Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:52.724967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:52.724992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343068360361642:12139]
2025-10-09T21:51:52.724978Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2275
2025-10-09T21:51:52.725003Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:52.724997Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2275
2025-10-09T21:51:52.725010Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:52.725092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:52.725100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:52.725119Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6dd10311-7632252a-133b3187-4f925732_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:52.725130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:52.725133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:52.725136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:52.725142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:52.725149Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:52.725168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:52.725310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:52.725333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:52.725345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:52.725444Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2276 partition: 0 MaxSeqNo: 25 sessionId: src_id|6dd10311-7632252a-133b3187-4f925732_0
2025-10-09T21:51:52.725762Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|537baedc-7758d387-8692fe9d-57ebd2ab_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6dd10311-7632252a-133b3187-4f925732_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:52.728168Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6dd10311-7632252a-133b3187-4f925732_0] Write session: try to update token
2025-10-09T21:51:52.728185Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6dd10311-7632252a-133b3187-4f925732_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:52.739288Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:52.739308Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:52.739310Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:52.739313Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:52.739324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343068360361642:12139]
2025-10-09T21:51:52.739330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343068360361642:12139]
2025-10-09T21:51:52.739335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343068360361642:12139]
2025-10-09T21:51:52.739377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:52.739386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.739387Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:52.739389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:52.739394Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:52.739399Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:52.739548Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:52.741269Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2276 sessionId: src_id|6dd10311-7632252a-133b3187-4f925732_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:52.743466Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:52.805369Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343068722213260:8652]
2025-10-09T21:51:52.805621Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 2275, pipe: [1:7559343068722213260:8652]
2025-10-09T21:51:52.888503Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:52.888859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:52.888882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:52.888934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343068722213260:8652], now have 1 active actors on pipe
2025-10-09T21:51:52.889151Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:52.889185Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343068722213260:8652]
2025-10-09T21:51:52.889200Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:52.889203Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:52.889184Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:52.889285Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2276 reason: error at write: Disconnected sessionId: src_id|6dd10311-7632252a-133b3187-4f925732_0
2025-10-09T21:51:52.889453Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2276 sessionId: src_id|6dd10311-7632252a-133b3187-4f925732_0 is DEAD
2025-10-09T21:51:52.889616Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6dd10311-7632252a-133b3187-4f925732_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:52.889628Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6dd10311-7632252a-133b3187-4f925732_0] Write session will restart in 0.008326s
2025-10-09T21:51:52.889656Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6dd10311-7632252a-133b3187-4f925732_0] Write session: Do CDS request
2025-10-09T21:51:52.889661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6dd10311-7632252a-133b3187-4f925732_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:52.897863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:52.899277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:52.899551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6dd10311-7632252a-133b3187-4f925732_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:52.899603Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:52.899618Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:52.899631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:52.899636Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:52.899654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2277
2025-10-09T21:51:52.899694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:52.899734Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343068360361758:12143]
2025-10-09T21:51:52.899775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2277 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:52.899801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2277 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:52.899809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2277 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:52.899813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:52.900199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:52.900213Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2277 sessionId:  describe result for acl check
2025-10-09T21:51:52.900251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:52.900258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:52.900260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:52.900267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343068722213333:7964] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:52.900484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:52.900521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:52.900745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343068722213333:7964] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:52.900764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:52.900803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.900810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.900811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.900812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.900813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.900814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.900815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.900815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.900816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.900820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.900827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.900829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.900831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.900834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.900836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.900839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.900841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.900843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.900848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.900853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:52.900857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.900862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:52.900865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.900869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:52.900872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.900875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:52.900879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.900883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:52.900886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.900890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:52.900893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.900899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.900905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.900911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.900916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.900922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.900950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:52.902338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343068722213333:7964] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:52.906728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343068722213333:7964] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:52.906749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343068722213333:7964] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:52.906754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343068722213333:7964] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:52.906763Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2277 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:52.907311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:52.908537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:52.908558Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:52.908562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:52.908566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:52.908574Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2276 [2:7559343068360361758:12143]
2025-10-09T21:51:52.908593Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:52.908608Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:52.908610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:52.908613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:52.908615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:52.908628Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:52.908639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:52.908645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:52.908698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.908708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.908710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.908712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:52.908779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:52.908787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.908796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:52.908802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:52.908813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:52.908830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343068722213374:7964], now have 1 active actors on pipe
2025-10-09T21:51:52.908849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343068360361753:12143]
2025-10-09T21:51:52.908856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343068360361753:12143]
2025-10-09T21:51:52.908858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343068360361753:12143]
2025-10-09T21:51:52.908874Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:52.908885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343068722213263:2432], now have 1 active actors on pipe
2025-10-09T21:51:52.908893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:52.908895Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:52.908899Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:52.908902Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:52.908914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:52.908917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:52.908912Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2276
2025-10-09T21:51:52.908928Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2276
2025-10-09T21:51:52.908959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343068360361753:12143]
2025-10-09T21:51:52.908972Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:52.908982Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:52.908994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:52.908998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:52.909033Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|46828226-3d8ee894-9177f60b-934aacc6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:52.909045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:52.909048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:52.909050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:52.909058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:52.909065Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:52.909085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:52.909247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:52.909257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:52.909269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:52.909353Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2277 partition: 0 MaxSeqNo: 25 sessionId: src_id|46828226-3d8ee894-9177f60b-934aacc6_0
2025-10-09T21:51:52.909713Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6dd10311-7632252a-133b3187-4f925732_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|46828226-3d8ee894-9177f60b-934aacc6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:52.912225Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46828226-3d8ee894-9177f60b-934aacc6_0] Write session: try to update token
2025-10-09T21:51:52.912244Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46828226-3d8ee894-9177f60b-934aacc6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:52.923977Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:52.923995Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:52.923997Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:52.923999Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:52.924007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343068360361753:12143]
2025-10-09T21:51:52.924011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343068360361753:12143]
2025-10-09T21:51:52.924016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343068360361753:12143]
2025-10-09T21:51:52.924051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:52.924057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:52.924059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:52.924095Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:52.924116Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:52.924120Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:52.924211Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:52.926238Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2277 sessionId: src_id|46828226-3d8ee894-9177f60b-934aacc6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:52.928468Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:52.989191Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343068722213381:8652]
2025-10-09T21:51:52.989517Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 2276, pipe: [1:7559343068722213381:8652]
2025-10-09T21:51:53.071555Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:53.072044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:53.072079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:53.072125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343068722213381:8652], now have 1 active actors on pipe
2025-10-09T21:51:53.072448Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343068722213381:8652]
2025-10-09T21:51:53.072448Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:53.072448Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:53.072453Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:53.072481Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:53.072530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2277 reason: error at write: Disconnected sessionId: src_id|46828226-3d8ee894-9177f60b-934aacc6_0
2025-10-09T21:51:53.072711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2277 sessionId: src_id|46828226-3d8ee894-9177f60b-934aacc6_0 is DEAD
2025-10-09T21:51:53.072879Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46828226-3d8ee894-9177f60b-934aacc6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:53.072891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46828226-3d8ee894-9177f60b-934aacc6_0] Write session will restart in 0.009857s
2025-10-09T21:51:53.072918Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46828226-3d8ee894-9177f60b-934aacc6_0] Write session: Do CDS request
2025-10-09T21:51:53.072925Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46828226-3d8ee894-9177f60b-934aacc6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:53.081927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:53.083464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|46828226-3d8ee894-9177f60b-934aacc6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:53.083599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:53.083615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2278
2025-10-09T21:51:53.083656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:53.083715Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2278 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:53.083748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2278 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:53.083759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2278 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:53.083768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:53.083857Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:53.083870Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:53.083881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:53.083931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:53.083973Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343072655329162:12145]
2025-10-09T21:51:53.084235Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2278 sessionId:  describe result for acl check
2025-10-09T21:51:53.084272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:53.084278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:53.084280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:53.084286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343073017180751:7945] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:53.084584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:53.084816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:53.084821Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343073017180751:7945] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:53.084860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:53.085187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:53.085236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.085242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.085243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.085244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.085246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.085247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.085248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.085249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.085250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.085255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.085266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.085271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.085275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.085284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.085288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.085292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.085296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.085300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.085309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.085327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.085335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.085340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.085347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.085352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.085361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.085366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.085373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.085378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.085387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.085393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.085402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.085407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.085416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.085422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.085430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.085437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.085468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:53.086848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343073017180751:7945] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:53.091154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343073017180751:7945] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:53.091170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343073017180751:7945] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:53.091174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343073017180751:7945] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:53.091182Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2278 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:53.091562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:53.092793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:53.092808Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:53.092810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:53.092815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:53.092822Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2277 [2:7559343072655329162:12145]
2025-10-09T21:51:53.092837Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:53.092855Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:53.092865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:53.092868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:53.092871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:53.092889Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:53.092899Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:53.092906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:53.092957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.092964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.092966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.092969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:53.093036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:53.093043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.093055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.093068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.093091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:53.093116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343073017180790:7945], now have 1 active actors on pipe
2025-10-09T21:51:53.093123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343073017180680:2432], now have 1 active actors on pipe
2025-10-09T21:51:53.093131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343072655329157:12145]
2025-10-09T21:51:53.093135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343072655329157:12145]
2025-10-09T21:51:53.093138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343072655329157:12145]
2025-10-09T21:51:53.093152Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:53.093134Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2277
2025-10-09T21:51:53.093149Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2277
2025-10-09T21:51:53.093166Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:53.093169Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:53.093176Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:53.093178Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:53.093184Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:53.093187Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:53.093232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:53.093242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:53.093259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343072655329157:12145]
2025-10-09T21:51:53.093270Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:53.093282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:53.093304Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5c761102-d9cac0ac-e3a2b961-4c3735da_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:53.093329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:53.093337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:53.093339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:53.093351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:53.093364Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:53.093392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:53.093524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:53.093531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:53.093545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:53.093622Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2278 partition: 0 MaxSeqNo: 25 sessionId: src_id|5c761102-d9cac0ac-e3a2b961-4c3735da_0
2025-10-09T21:51:53.093973Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|46828226-3d8ee894-9177f60b-934aacc6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5c761102-d9cac0ac-e3a2b961-4c3735da_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:53.096372Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c761102-d9cac0ac-e3a2b961-4c3735da_0] Write session: try to update token
2025-10-09T21:51:53.096391Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c761102-d9cac0ac-e3a2b961-4c3735da_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:53.107898Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:53.107922Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:53.107925Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:53.107928Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:53.107939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343072655329157:12145]
2025-10-09T21:51:53.107948Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343072655329157:12145]
2025-10-09T21:51:53.107954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343072655329157:12145]
2025-10-09T21:51:53.108008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:53.108013Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:53.108020Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:53.108024Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:53.108025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.108028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:53.108206Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:53.108714Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2278 sessionId: src_id|5c761102-d9cac0ac-e3a2b961-4c3735da_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:53.111156Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:53.172421Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343073017180798:8652]
2025-10-09T21:51:53.172680Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 2277, pipe: [1:7559343073017180798:8652]
2025-10-09T21:51:53.259691Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:53.259984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:53.259999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:53.260031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343073017180798:8652], now have 1 active actors on pipe
2025-10-09T21:51:53.260235Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:53.260259Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:53.260254Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:53.260257Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:53.260289Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343073017180798:8652]
2025-10-09T21:51:53.260401Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2278 reason: error at write: Disconnected sessionId: src_id|5c761102-d9cac0ac-e3a2b961-4c3735da_0
2025-10-09T21:51:53.260549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2278 sessionId: src_id|5c761102-d9cac0ac-e3a2b961-4c3735da_0 is DEAD
2025-10-09T21:51:53.260693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c761102-d9cac0ac-e3a2b961-4c3735da_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:53.260701Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c761102-d9cac0ac-e3a2b961-4c3735da_0] Write session will restart in 0.005167s
2025-10-09T21:51:53.260725Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c761102-d9cac0ac-e3a2b961-4c3735da_0] Write session: Do CDS request
2025-10-09T21:51:53.260730Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c761102-d9cac0ac-e3a2b961-4c3735da_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:53.267428Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c761102-d9cac0ac-e3a2b961-4c3735da_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:53.267535Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:53.267551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2279
2025-10-09T21:51:53.267662Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2279 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:53.267696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2279 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:53.267704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2279 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:53.267707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:53.268034Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2279 sessionId:  describe result for acl check
2025-10-09T21:51:53.268066Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:53.268071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:53.268072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:53.268077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343073017180872:7940] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:53.268514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343073017180872:7940] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:53.269239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:53.270585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343073017180872:7940] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:53.270620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:53.270815Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:53.270828Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:53.270839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:53.270891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:53.270937Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343072655329272:12147]
2025-10-09T21:51:53.271579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:53.271856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:53.271909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:53.272250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:53.272298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.272306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.272307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.272309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.272310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.272312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.272313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.272315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.272316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.272322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.272333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.272338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.272342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.272346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.272351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.272355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.272359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.272363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.272369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.272377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.272382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.272388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.272396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.272402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.272407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.272413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.272425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.272437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.272442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.272454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.272459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.272474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.272479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.272485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.272490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.272496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.272526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:53.272818Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343073017180872:7940] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:53.272829Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343073017180872:7940] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:53.272832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343073017180872:7940] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:53.272838Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2279 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:53.278799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:53.280161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:53.280178Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:53.280182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:53.280186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:53.280194Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2278 [2:7559343072655329272:12147]
2025-10-09T21:51:53.280211Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:53.280227Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:53.280239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:53.280243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:53.280246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:53.280262Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:53.280268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:53.280283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:53.280350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.280359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.280361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.280364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:53.280433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:53.280442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.280453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.280467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.280491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:53.280518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343073017180914:7940], now have 1 active actors on pipe
2025-10-09T21:51:53.280522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343073017180803:2432], now have 1 active actors on pipe
2025-10-09T21:51:53.280529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343072655329267:12147]
2025-10-09T21:51:53.280531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343072655329267:12147]
2025-10-09T21:51:53.280533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343072655329267:12147]
2025-10-09T21:51:53.280549Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:53.280562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:53.280576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:53.280578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:53.280562Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2278
2025-10-09T21:51:53.280582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:53.280584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:53.280585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:53.280585Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2278
2025-10-09T21:51:53.280612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343072655329267:12147]
2025-10-09T21:51:53.280616Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:53.280619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:53.280639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:53.280650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:53.280694Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e1935ad9-57365ba3-af4cc924-a72e59b2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:53.280707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:53.280711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:53.280714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:53.280722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:53.280730Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:53.280748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:53.280964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:53.280977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:53.280994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:53.281103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2279 partition: 0 MaxSeqNo: 25 sessionId: src_id|e1935ad9-57365ba3-af4cc924-a72e59b2_0
2025-10-09T21:51:53.281426Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c761102-d9cac0ac-e3a2b961-4c3735da_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e1935ad9-57365ba3-af4cc924-a72e59b2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:53.283791Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1935ad9-57365ba3-af4cc924-a72e59b2_0] Write session: try to update token
2025-10-09T21:51:53.283806Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1935ad9-57365ba3-af4cc924-a72e59b2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:53.294807Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:53.294830Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:53.294833Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:53.294836Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:53.294845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343072655329267:12147]
2025-10-09T21:51:53.294850Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343072655329267:12147]
2025-10-09T21:51:53.294855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343072655329267:12147]
2025-10-09T21:51:53.294900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:53.294909Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:53.294913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.294916Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:53.294917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:53.294920Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:53.295073Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:53.298214Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2279 sessionId: src_id|e1935ad9-57365ba3-af4cc924-a72e59b2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:53.300478Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:53.325155Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] Counters: { Errors: 22 CurrentSessionLifetimeMs: 541776 BytesRead: 24780812 MessagesRead: 11 BytesReadCompressed: 24780812 BytesInflightUncompressed: 2252802 BytesInflightCompressed: 0 BytesInflightTotal: 2252802 MessagesInflight: 1 }
2025-10-09T21:51:53.360364Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343073017180927:8652]
2025-10-09T21:51:53.360591Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 2278, pipe: [1:7559343073017180927:8652]
2025-10-09T21:51:53.444937Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:53.445474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:53.445493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:53.445538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343073017180927:8652], now have 1 active actors on pipe
2025-10-09T21:51:53.445793Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343073017180927:8652]
2025-10-09T21:51:53.445830Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:53.445818Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:53.445842Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:53.445845Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:53.445924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2279 reason: error at write: Disconnected sessionId: src_id|e1935ad9-57365ba3-af4cc924-a72e59b2_0
2025-10-09T21:51:53.446188Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2279 sessionId: src_id|e1935ad9-57365ba3-af4cc924-a72e59b2_0 is DEAD
2025-10-09T21:51:53.446308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1935ad9-57365ba3-af4cc924-a72e59b2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:53.446328Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1935ad9-57365ba3-af4cc924-a72e59b2_0] Write session will restart in 0.005263s
2025-10-09T21:51:53.446354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1935ad9-57365ba3-af4cc924-a72e59b2_0] Write session: Do CDS request
2025-10-09T21:51:53.446361Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1935ad9-57365ba3-af4cc924-a72e59b2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:53.452477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1935ad9-57365ba3-af4cc924-a72e59b2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:53.452580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:53.452597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2280
2025-10-09T21:51:53.452722Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2280 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:53.452756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2280 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:53.452762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2280 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:53.452765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:53.453270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2280 sessionId:  describe result for acl check
2025-10-09T21:51:53.453301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:53.453304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:53.453305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:53.453309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343073017180969:7927] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:53.453784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343073017180969:7927] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:53.455273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:53.456023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343073017180969:7927] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:53.456872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:53.457062Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:53.457078Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:53.457090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:53.457153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:53.457203Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343072655329381:12149]
2025-10-09T21:51:53.457796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:53.458141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:53.458186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:53.458278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343073017180969:7927] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:53.458291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343073017180969:7927] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:53.458294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343073017180969:7927] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:53.458300Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2280 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:53.458450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:53.458491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.458499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.458500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.458501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.458503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.458504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.458505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.458507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.458509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.458514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.458523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.458525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.458528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.458530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.458533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.458535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.458537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.458540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.458546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.458552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.458558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.458563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.458568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.458573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.458579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.458585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.458591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.458604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.458607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.458611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.458614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.458617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.458622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.458631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.458634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.458637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.458655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:53.465403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:53.466677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:53.466697Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:53.466700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:53.466705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:53.466713Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2279 [2:7559343072655329381:12149]
2025-10-09T21:51:53.466731Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:53.466747Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:53.466750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:53.466754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:53.466756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:53.466772Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:53.466789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:53.466796Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:53.466863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.466873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.466876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.466878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:53.466955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:53.466965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.466978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.466991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.467010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:53.467033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343073017180929:2432], now have 1 active actors on pipe
2025-10-09T21:51:53.467043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343073017181039:7927], now have 1 active actors on pipe
2025-10-09T21:51:53.467050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343072655329376:12149]
2025-10-09T21:51:53.467053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343072655329376:12149]
2025-10-09T21:51:53.467056Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343072655329376:12149]
2025-10-09T21:51:53.467071Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:53.467085Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:53.467087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:53.467097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:53.467099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:53.467103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:53.467107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:53.467107Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2279
2025-10-09T21:51:53.467136Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2279
2025-10-09T21:51:53.467138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343072655329376:12149]
2025-10-09T21:51:53.467155Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:53.467157Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:53.467186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:53.467196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:53.467230Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ff3e473d-97f03ab-c8c4c968-b1a0dc2d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:53.467243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:53.467247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:53.467250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:53.467259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:53.467268Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:53.467291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:53.467440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:53.467454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:53.467468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:53.467558Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2280 partition: 0 MaxSeqNo: 25 sessionId: src_id|ff3e473d-97f03ab-c8c4c968-b1a0dc2d_0
2025-10-09T21:51:53.467899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1935ad9-57365ba3-af4cc924-a72e59b2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ff3e473d-97f03ab-c8c4c968-b1a0dc2d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:53.470334Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff3e473d-97f03ab-c8c4c968-b1a0dc2d_0] Write session: try to update token
2025-10-09T21:51:53.470349Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff3e473d-97f03ab-c8c4c968-b1a0dc2d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:53.482279Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:53.482301Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:53.482304Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:53.482307Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:53.482318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343072655329376:12149]
2025-10-09T21:51:53.482324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343072655329376:12149]
2025-10-09T21:51:53.482328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343072655329376:12149]
2025-10-09T21:51:53.482376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:53.482390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.482392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:53.482394Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:53.482403Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:53.482407Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:53.482544Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:53.483341Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2280 sessionId: src_id|ff3e473d-97f03ab-c8c4c968-b1a0dc2d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:53.485522Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:53.546246Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343073017181049:8652]
2025-10-09T21:51:53.546563Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 2279, pipe: [1:7559343073017181049:8652]
2025-10-09T21:51:53.630209Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:53.630466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:53.630478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:53.630508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343073017181049:8652], now have 1 active actors on pipe
2025-10-09T21:51:53.630698Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343073017181049:8652]
2025-10-09T21:51:53.630726Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:53.630757Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:53.630765Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:53.630739Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:53.630827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2280 reason: error at write: Disconnected sessionId: src_id|ff3e473d-97f03ab-c8c4c968-b1a0dc2d_0
2025-10-09T21:51:53.630979Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2280 sessionId: src_id|ff3e473d-97f03ab-c8c4c968-b1a0dc2d_0 is DEAD
2025-10-09T21:51:53.631093Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff3e473d-97f03ab-c8c4c968-b1a0dc2d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:53.631103Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff3e473d-97f03ab-c8c4c968-b1a0dc2d_0] Write session will restart in 0.005897s
2025-10-09T21:51:53.631129Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff3e473d-97f03ab-c8c4c968-b1a0dc2d_0] Write session: Do CDS request
2025-10-09T21:51:53.631135Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff3e473d-97f03ab-c8c4c968-b1a0dc2d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:53.638491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ff3e473d-97f03ab-c8c4c968-b1a0dc2d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:53.638582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:53.638604Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2281
2025-10-09T21:51:53.638728Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2281 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:53.638753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2281 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:53.638756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2281 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:53.638759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:53.639173Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2281 sessionId:  describe result for acl check
2025-10-09T21:51:53.639213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:53.639219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:53.639220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:53.639226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343073017181149:7879] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:53.639471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:53.639729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343073017181149:7879] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:53.641110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:53.641339Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:53.641349Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:53.641360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:53.641421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:53.641462Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343072655329531:12163]
2025-10-09T21:51:53.641623Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343073017181149:7879] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:53.642106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:53.642436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:53.642485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:53.642822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:53.642878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.642885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.642887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.642888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.642890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.642891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.642892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.642893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.642894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.642900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.642910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.642915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.642919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.642923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.642928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.642932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.642936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.642940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.642946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.642953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.642959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.642965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.642969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.642975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.642980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.642986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.642991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.642997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.643001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.643007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.643012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.643018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.643023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.643028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.643035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.643042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.643071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:53.648487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343073017181149:7879] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:53.648501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343073017181149:7879] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:53.648504Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343073017181149:7879] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:53.648513Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2281 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:53.649725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:53.651195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:53.651210Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:53.651212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:53.651215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:53.651222Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2280 [2:7559343072655329531:12163]
2025-10-09T21:51:53.651237Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:53.651255Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:53.651259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:53.651262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:53.651264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:53.651279Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:53.651291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:53.651297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:53.651345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.651350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.651352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.651353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:53.651426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:53.651435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.651446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.651459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.651476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:53.651493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343073017181186:7879], now have 1 active actors on pipe
2025-10-09T21:51:53.651497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343073017181067:2432], now have 1 active actors on pipe
2025-10-09T21:51:53.651503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343072655329526:12163]
2025-10-09T21:51:53.651510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343072655329526:12163]
2025-10-09T21:51:53.651513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343072655329526:12163]
2025-10-09T21:51:53.651528Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:53.651543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:53.651553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:53.651555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:53.651563Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:53.651567Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2280
2025-10-09T21:51:53.651571Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2280
2025-10-09T21:51:53.651579Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:53.651582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:53.651583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343072655329526:12163]
2025-10-09T21:51:53.651586Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:53.651590Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:53.651626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:53.651638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:53.651670Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|af6783e9-98d463ce-7e53d586-97b406de_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:53.651683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:53.651688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:53.651691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:53.651699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:53.651709Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:53.651727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:53.651938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:53.651948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:53.651961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:53.652076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2281 partition: 0 MaxSeqNo: 25 sessionId: src_id|af6783e9-98d463ce-7e53d586-97b406de_0
2025-10-09T21:51:53.652425Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ff3e473d-97f03ab-c8c4c968-b1a0dc2d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|af6783e9-98d463ce-7e53d586-97b406de_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:53.654852Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af6783e9-98d463ce-7e53d586-97b406de_0] Write session: try to update token
2025-10-09T21:51:53.654869Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af6783e9-98d463ce-7e53d586-97b406de_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:53.666517Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:53.666536Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:53.666539Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:53.666541Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:53.666552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343072655329526:12163]
2025-10-09T21:51:53.666557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343072655329526:12163]
2025-10-09T21:51:53.666563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343072655329526:12163]
2025-10-09T21:51:53.666607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:53.666616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.666616Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:53.666622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:53.666623Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:53.666628Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:53.666786Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:53.667447Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2281 sessionId: src_id|af6783e9-98d463ce-7e53d586-97b406de_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:53.669815Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:53.731078Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343073017181194:8652]
2025-10-09T21:51:53.731396Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 2280, pipe: [1:7559343073017181194:8652]
2025-10-09T21:51:53.816230Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:53.816572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:53.816585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:53.816615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343073017181194:8652], now have 1 active actors on pipe
2025-10-09T21:51:53.816911Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:53.816928Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:53.816991Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343073017181194:8652]
2025-10-09T21:51:53.817002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2281 reason: error at write: Disconnected sessionId: src_id|af6783e9-98d463ce-7e53d586-97b406de_0
2025-10-09T21:51:53.817046Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:53.817196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2281 sessionId: src_id|af6783e9-98d463ce-7e53d586-97b406de_0 is DEAD
2025-10-09T21:51:53.817303Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af6783e9-98d463ce-7e53d586-97b406de_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:53.817326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af6783e9-98d463ce-7e53d586-97b406de_0] Write session will restart in 0.007503s
2025-10-09T21:51:53.817345Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af6783e9-98d463ce-7e53d586-97b406de_0] Write session: Do CDS request
2025-10-09T21:51:53.817361Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af6783e9-98d463ce-7e53d586-97b406de_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:53.817445Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:53.825550Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af6783e9-98d463ce-7e53d586-97b406de_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:53.825609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:53.825628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2282
2025-10-09T21:51:53.825784Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2282 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:53.825822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2282 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:53.825833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2282 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:53.825837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:53.826311Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2282 sessionId:  describe result for acl check
2025-10-09T21:51:53.826351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:53.826357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:53.826359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:53.826365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343073017181270:7860] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:53.826419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:53.826978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343073017181270:7860] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:53.827883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:53.828063Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:53.828073Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:53.828083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:53.828128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:53.828171Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343072655329647:12167]
2025-10-09T21:51:53.828749Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343073017181270:7860] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:53.828799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:53.829079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:53.829123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:53.829459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:53.829513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.829519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.829520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.829521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.829523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.829524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.829526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.829527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.829528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.829533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.829543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.829547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.829552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.829556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.829560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.829564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.829567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.829571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.829578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.829585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:53.829591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.829598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:53.829604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.829609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:53.829614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.829620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:53.829628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.829633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:53.829639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.829647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:53.829657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.829663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.829675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.829680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.829685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.829691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.829727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:53.834858Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343073017181270:7860] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:53.834871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343073017181270:7860] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:53.834874Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343073017181270:7860] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:53.834880Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2282 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:53.836347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:53.837605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:53.837621Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:53.837624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:53.837628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:53.837637Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2281 [2:7559343072655329647:12167]
2025-10-09T21:51:53.837653Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:53.837669Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:53.837674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:53.837677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:53.837680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:53.837695Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:53.837701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:53.837707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:53.837768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.837776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.837779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.837782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:53.837854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:53.837862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.837875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:53.837892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:53.837909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:53.837932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343073017181196:2432], now have 1 active actors on pipe
2025-10-09T21:51:53.837941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343073017181309:7860], now have 1 active actors on pipe
2025-10-09T21:51:53.837949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343072655329641:12167]
2025-10-09T21:51:53.837956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343072655329641:12167]
2025-10-09T21:51:53.837959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343072655329641:12167]
2025-10-09T21:51:53.837976Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:53.837989Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:53.838005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:53.837985Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2281
2025-10-09T21:51:53.837997Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2281
2025-10-09T21:51:53.838006Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:53.838009Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:53.838011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:53.838011Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:53.838060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343072655329641:12167]
2025-10-09T21:51:53.838068Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:53.838093Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:53.838096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:53.838100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:53.838140Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|af6c704c-9b58bffd-d40afa02-2c0dbea_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:53.838152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:53.838157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:53.838160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:53.838171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:53.838180Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:53.838196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:53.838421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:53.838433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:53.838448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:53.838575Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2282 partition: 0 MaxSeqNo: 25 sessionId: src_id|af6c704c-9b58bffd-d40afa02-2c0dbea_0
2025-10-09T21:51:53.838904Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af6783e9-98d463ce-7e53d586-97b406de_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|af6c704c-9b58bffd-d40afa02-2c0dbea_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:53.841200Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af6c704c-9b58bffd-d40afa02-2c0dbea_0] Write session: try to update token
2025-10-09T21:51:53.841218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af6c704c-9b58bffd-d40afa02-2c0dbea_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:53.853309Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:53.853344Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:53.853347Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:53.853350Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:53.853360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343072655329641:12167]
2025-10-09T21:51:53.853366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343072655329641:12167]
2025-10-09T21:51:53.853371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343072655329641:12167]
2025-10-09T21:51:53.853419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:53.853435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:53.853438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:53.853449Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:53.853457Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:53.853462Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:53.853597Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:53.856068Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2282 sessionId: src_id|af6c704c-9b58bffd-d40afa02-2c0dbea_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:53.858523Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:53.917444Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343073017181319:8652]
2025-10-09T21:51:53.917738Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 2281, pipe: [1:7559343073017181319:8652]
2025-10-09T21:51:54.001141Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:54.001723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.001740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.001772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343073017181319:8652], now have 1 active actors on pipe
2025-10-09T21:51:54.001998Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:54.002072Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:54.002076Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343073017181319:8652]
2025-10-09T21:51:54.002107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:54.002180Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2282 reason: status is not ok: tablet will be restarted right now sessionId: src_id|af6c704c-9b58bffd-d40afa02-2c0dbea_0
2025-10-09T21:51:54.002338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2282 sessionId: src_id|af6c704c-9b58bffd-d40afa02-2c0dbea_0 is DEAD
2025-10-09T21:51:54.002483Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af6c704c-9b58bffd-d40afa02-2c0dbea_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:54.002494Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af6c704c-9b58bffd-d40afa02-2c0dbea_0] Write session will restart in 0.006106s
2025-10-09T21:51:54.002515Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af6c704c-9b58bffd-d40afa02-2c0dbea_0] Write session: Do CDS request
2025-10-09T21:51:54.002521Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af6c704c-9b58bffd-d40afa02-2c0dbea_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:54.009520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af6c704c-9b58bffd-d40afa02-2c0dbea_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:54.009633Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:54.009658Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2283
2025-10-09T21:51:54.009770Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2283 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:54.009797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2283 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:54.009804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2283 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:54.009808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:54.010157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2283 sessionId:  describe result for acl check
2025-10-09T21:51:54.010185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:54.010189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:54.010191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:54.010194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343077312148694:7850] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:54.010651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343077312148694:7850] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:54.010822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:54.012399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343077312148694:7850] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:54.012426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:54.012622Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:54.012631Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:54.012641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:54.012688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:54.012732Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343076950297057:12169]
2025-10-09T21:51:54.013392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:54.013763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:54.013833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:54.014220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:54.014288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.014294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.014296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.014297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.014298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.014300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.014301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.014303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.014304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.014310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.014321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.014325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.014329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.014334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.014338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.014342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.014346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.014350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.014358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.014366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.014372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.014378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.014383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.014389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.014394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.014400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.014405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.014410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.014416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.014422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.014425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.014429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.014432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.014435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.014438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.014442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.014459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:54.014904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343077312148694:7850] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:54.014919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343077312148694:7850] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:54.014923Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343077312148694:7850] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:54.014931Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2283 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:54.021021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:54.022922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:54.022942Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:54.022947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:54.022952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:54.022960Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2282 [2:7559343076950297057:12169]
2025-10-09T21:51:54.022976Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:54.022990Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:54.022993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:54.022996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:54.022998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:54.023010Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:54.023018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:54.023023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:54.023084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.023090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.023092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.023094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:54.023159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:54.023171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.023184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.023196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.023209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:54.023229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343077312148619:2432], now have 1 active actors on pipe
2025-10-09T21:51:54.023236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343077312148732:7850], now have 1 active actors on pipe
2025-10-09T21:51:54.023242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343076950297052:12169]
2025-10-09T21:51:54.023248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343076950297052:12169]
2025-10-09T21:51:54.023249Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343076950297052:12169]
2025-10-09T21:51:54.023262Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:54.023274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:54.023283Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:54.023287Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:54.023289Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:54.023258Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2282
2025-10-09T21:51:54.023293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:54.023272Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2282
2025-10-09T21:51:54.023302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:54.023364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.023375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.023386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343076950297052:12169]
2025-10-09T21:51:54.023395Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:54.023411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:54.023426Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fff3817d-78ef7c14-f4c26640-1abb7af2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:54.023432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:54.023434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:54.023437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:54.023443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:54.023450Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:54.023468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:54.023625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.023636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.023648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:54.023741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2283 partition: 0 MaxSeqNo: 25 sessionId: src_id|fff3817d-78ef7c14-f4c26640-1abb7af2_0
2025-10-09T21:51:54.024091Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af6c704c-9b58bffd-d40afa02-2c0dbea_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fff3817d-78ef7c14-f4c26640-1abb7af2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:54.026678Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fff3817d-78ef7c14-f4c26640-1abb7af2_0] Write session: try to update token
2025-10-09T21:51:54.026695Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fff3817d-78ef7c14-f4c26640-1abb7af2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:54.037720Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:54.037742Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:54.037744Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:54.037746Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:54.037754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343076950297052:12169]
2025-10-09T21:51:54.037759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343076950297052:12169]
2025-10-09T21:51:54.037763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343076950297052:12169]
2025-10-09T21:51:54.037803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:54.037809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.037811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:54.037819Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:54.037835Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:54.037839Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:54.037960Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:54.040559Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2283 sessionId: src_id|fff3817d-78ef7c14-f4c26640-1abb7af2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:54.042951Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:54.102274Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343077312148739:8652]
2025-10-09T21:51:54.102594Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 2282, pipe: [1:7559343077312148739:8652]
2025-10-09T21:51:54.190663Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:54.190961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.190974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.191008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343077312148739:8652], now have 1 active actors on pipe
2025-10-09T21:51:54.191212Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:54.191274Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:54.191278Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:54.191282Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343077312148739:8652]
2025-10-09T21:51:54.191303Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:54.191340Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2283 reason: error at write: Disconnected sessionId: src_id|fff3817d-78ef7c14-f4c26640-1abb7af2_0
2025-10-09T21:51:54.191490Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2283 sessionId: src_id|fff3817d-78ef7c14-f4c26640-1abb7af2_0 is DEAD
2025-10-09T21:51:54.191635Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fff3817d-78ef7c14-f4c26640-1abb7af2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:54.191660Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fff3817d-78ef7c14-f4c26640-1abb7af2_0] Write session will restart in 0.008940s
2025-10-09T21:51:54.191689Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fff3817d-78ef7c14-f4c26640-1abb7af2_0] Write session: Do CDS request
2025-10-09T21:51:54.191695Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fff3817d-78ef7c14-f4c26640-1abb7af2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:54.201419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:54.201564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fff3817d-78ef7c14-f4c26640-1abb7af2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:54.201628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:54.201650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2284
2025-10-09T21:51:54.201803Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2284 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:54.201831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2284 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:54.201835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2284 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:54.201839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:54.202238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2284 sessionId:  describe result for acl check
2025-10-09T21:51:54.202274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:54.202279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:54.202281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:54.202285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343077312148819:7837] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:54.202783Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343077312148819:7837] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:54.203150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:54.203366Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:54.203376Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:54.203387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:54.203440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:54.203486Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343076950297172:12171]
2025-10-09T21:51:54.204141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:54.204438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:54.204488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:54.204829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:54.204819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343077312148819:7837] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:54.204889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.204897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.204899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.204900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.204902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.204903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.204905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.204906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.204907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.204914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.204925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.204929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.204933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.204937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.204941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.204946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.204950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.204955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.204963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.204971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.204977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.204983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.204988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.204994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.204999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.205004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.205009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.205015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.205020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.205026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.205031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.205040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.205044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.205050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.205055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.205061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.205092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:54.211078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343077312148819:7837] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:54.211108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343077312148819:7837] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:54.211113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343077312148819:7837] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:54.211123Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2284 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:54.211655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:54.213264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:54.213286Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:54.213291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:54.213295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:54.213303Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2283 [2:7559343076950297172:12171]
2025-10-09T21:51:54.213333Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:54.213353Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:54.213356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:54.213360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:54.213362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:54.213379Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:54.213390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:54.213399Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:54.213475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.213482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.213484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.213487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:54.213570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:54.213580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.213592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.213603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.213627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:54.213649Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343076950297166:12171]
2025-10-09T21:51:54.213656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343076950297166:12171]
2025-10-09T21:51:54.213659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343076950297166:12171]
2025-10-09T21:51:54.213672Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:54.213691Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:54.213695Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:54.213700Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:54.213703Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:54.213711Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:54.213714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:54.213725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343077312148742:2432], now have 1 active actors on pipe
2025-10-09T21:51:54.213729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343077312148858:7837], now have 1 active actors on pipe
2025-10-09T21:51:54.213765Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343076950297166:12171]
2025-10-09T21:51:54.213764Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2283
2025-10-09T21:51:54.213781Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:54.213805Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:54.213771Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2283
2025-10-09T21:51:54.213830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.213837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.213863Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c1ccf59b-880dbc5-e7f50c36-c9ced79_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:54.213875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:54.213879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:54.213882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:54.213891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:54.213899Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:54.213916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:54.214099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.214111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.214125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:54.214238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2284 partition: 0 MaxSeqNo: 25 sessionId: src_id|c1ccf59b-880dbc5-e7f50c36-c9ced79_0
2025-10-09T21:51:54.214580Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fff3817d-78ef7c14-f4c26640-1abb7af2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c1ccf59b-880dbc5-e7f50c36-c9ced79_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:54.217282Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1ccf59b-880dbc5-e7f50c36-c9ced79_0] Write session: try to update token
2025-10-09T21:51:54.217303Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1ccf59b-880dbc5-e7f50c36-c9ced79_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:54.231614Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2284 sessionId: src_id|c1ccf59b-880dbc5-e7f50c36-c9ced79_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:54.231658Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:54.231679Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:54.231688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343076950297166:12171]
2025-10-09T21:51:54.231738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:54.231747Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:54.231910Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:54.232053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:54.232214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:54.232429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:54.232667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:54.232713Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:54.234159Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:54.234345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.234361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.234397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:54.234483Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:54.240038Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:54.240056Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:54.240060Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:54.240063Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:54.240072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343076950297166:12171]
2025-10-09T21:51:54.240077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343076950297166:12171]
2025-10-09T21:51:54.240082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343076950297166:12171]
2025-10-09T21:51:54.240123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:54.240133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.240136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:54.240137Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:54.240147Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:54.240153Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:54.240303Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:54.291221Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343077312148868:8652]
2025-10-09T21:51:54.291543Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 2283, pipe: [1:7559343077312148868:8652]
2025-10-09T21:51:54.389070Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:54.397842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.397869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.398052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.398119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.398188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.398275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.398336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.398344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.398419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.398491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.398575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.398659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.398722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.398732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.398803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.398862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.398929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.398996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.399331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.399340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.399410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.399477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.399546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.399613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.399665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.399671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.399743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.399816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.399888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.399976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.400037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.400047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.400126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.400189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.400259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.400334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.400386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.400394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.400461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.400532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.400598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.400659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.400714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.400721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.400790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.400852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.400914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.400978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.401034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.401042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.401109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.401168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.401231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.401295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.401357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.401364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.401432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.401493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.401560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.401627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.401682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.401690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.401750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343077312148868:8652], now have 1 active actors on pipe
2025-10-09T21:51:54.402022Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343077312148868:8652]
2025-10-09T21:51:54.402073Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:54.402078Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:54.402080Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:54.402124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2284 reason: error at write: Disconnected sessionId: src_id|c1ccf59b-880dbc5-e7f50c36-c9ced79_0
2025-10-09T21:51:54.402301Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2284 sessionId: src_id|c1ccf59b-880dbc5-e7f50c36-c9ced79_0 is DEAD
2025-10-09T21:51:54.402458Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1ccf59b-880dbc5-e7f50c36-c9ced79_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:54.402470Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1ccf59b-880dbc5-e7f50c36-c9ced79_0] Write session will restart in 0.006709s
2025-10-09T21:51:54.402489Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1ccf59b-880dbc5-e7f50c36-c9ced79_0] Write session: Do CDS request
2025-10-09T21:51:54.402494Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1ccf59b-880dbc5-e7f50c36-c9ced79_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:54.402483Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:54.410453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c1ccf59b-880dbc5-e7f50c36-c9ced79_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:54.410577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:54.410595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2285
2025-10-09T21:51:54.410702Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2285 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:54.410729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2285 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:54.410736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2285 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:54.410740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:54.411171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2285 sessionId:  describe result for acl check
2025-10-09T21:51:54.411206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:54.411211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:54.411212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:54.411217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343077312148926:7810] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:54.411674Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343077312148926:7810] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:54.413266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:54.414285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343077312148926:7810] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:54.414811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:54.415004Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:54.415015Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:54.415024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:54.415072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:54.415131Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343076950297287:12173]
2025-10-09T21:51:54.415763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:54.416093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:54.416146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:54.416482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:54.416538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.416545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.416546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.416548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.416549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.416551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.416552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.416554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.416555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.416560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.416571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.416575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.416580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.416584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.416588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.416592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.416597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.416601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.416608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.416622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.416629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.416635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.416646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.416651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.416663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.416668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.416675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.416680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.416685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.416694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.416700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.416711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.416716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.416729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.416711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343077312148926:7810] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:54.416735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.416747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.416725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343077312148926:7810] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:54.416728Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343077312148926:7810] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:54.416737Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2285 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:54.416782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:54.423282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:54.424680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:54.424698Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:54.424701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:54.424705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:54.424714Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2284 [2:7559343076950297287:12173]
2025-10-09T21:51:54.424732Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:54.424748Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:54.424755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:54.424759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:54.424762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:54.424778Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:54.424790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:54.424797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:54.424857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.424866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.424869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.424871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:54.424944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:54.424956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.424967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.424979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.424995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:54.425012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343077312148991:7810], now have 1 active actors on pipe
2025-10-09T21:51:54.425024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343077312148876:2432], now have 1 active actors on pipe
2025-10-09T21:51:54.425031Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343076950297281:12173]
2025-10-09T21:51:54.425034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343076950297281:12173]
2025-10-09T21:51:54.425036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343076950297281:12173]
2025-10-09T21:51:54.425053Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:54.425067Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:54.425073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:54.425074Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:54.425079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:54.425094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:54.425102Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:54.425079Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2284
2025-10-09T21:51:54.425079Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2284
2025-10-09T21:51:54.425148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.425160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.425171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343076950297281:12173]
2025-10-09T21:51:54.425186Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:54.425191Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:54.425216Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fdf6c8b4-eef91171-84dabe09-f7670f51_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:54.425228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:54.425232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:54.425234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:54.425247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:54.425261Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:54.425283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:54.425457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.425469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.425486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:54.425612Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2285 partition: 0 MaxSeqNo: 25 sessionId: src_id|fdf6c8b4-eef91171-84dabe09-f7670f51_0
2025-10-09T21:51:54.425959Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c1ccf59b-880dbc5-e7f50c36-c9ced79_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fdf6c8b4-eef91171-84dabe09-f7670f51_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:54.428349Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf6c8b4-eef91171-84dabe09-f7670f51_0] Write session: try to update token
2025-10-09T21:51:54.428369Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf6c8b4-eef91171-84dabe09-f7670f51_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:54.439585Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:54.439607Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:54.439611Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:54.439613Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:54.439622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343076950297281:12173]
2025-10-09T21:51:54.439627Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343076950297281:12173]
2025-10-09T21:51:54.439631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343076950297281:12173]
2025-10-09T21:51:54.439677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:54.439684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.439693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:54.439704Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:54.439724Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:54.439729Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:54.439830Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:54.441040Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2285 sessionId: src_id|fdf6c8b4-eef91171-84dabe09-f7670f51_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:54.443251Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:54.502127Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343077312149000:8652]
2025-10-09T21:51:54.502397Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 2284, pipe: [1:7559343077312149000:8652]
2025-10-09T21:51:54.586886Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:54.587181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.587195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.587233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343077312149000:8652], now have 1 active actors on pipe
2025-10-09T21:51:54.587380Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:54.587403Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:54.587409Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343077312149000:8652]
2025-10-09T21:51:54.587420Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:54.587434Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:54.587483Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2285 reason: error at write: Disconnected sessionId: src_id|fdf6c8b4-eef91171-84dabe09-f7670f51_0
2025-10-09T21:51:54.587682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2285 sessionId: src_id|fdf6c8b4-eef91171-84dabe09-f7670f51_0 is DEAD
2025-10-09T21:51:54.587817Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf6c8b4-eef91171-84dabe09-f7670f51_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:54.587827Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf6c8b4-eef91171-84dabe09-f7670f51_0] Write session will restart in 0.009525s
2025-10-09T21:51:54.587846Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf6c8b4-eef91171-84dabe09-f7670f51_0] Write session: Do CDS request
2025-10-09T21:51:54.587851Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf6c8b4-eef91171-84dabe09-f7670f51_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:54.596269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:54.597632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:54.597816Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:54.597824Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:54.597833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:54.597876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:54.597909Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343076950297403:12175]
2025-10-09T21:51:54.598334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:54.598442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf6c8b4-eef91171-84dabe09-f7670f51_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:54.598523Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:54.598540Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2286
2025-10-09T21:51:54.598625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:54.598655Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2286 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:54.598664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:54.598684Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2286 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:54.598691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2286 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:54.598694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:54.598903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:54.598949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.598955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.598956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.598957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.598959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.598960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.598961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.598962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.598963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.598968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.598975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.598977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.598979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.598981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.598983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.598985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.598987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.598989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.598995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.598999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.599004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.599007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.599010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.599013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2286 sessionId:  describe result for acl check
2025-10-09T21:51:54.599013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.599016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.599019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.599021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.599024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.599027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.599030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.599033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.599047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:54.599036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.599052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:54.599039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.599054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:54.599060Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343077312149085:7798] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:54.599043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.599046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.599049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.599065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:54.599632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343077312149085:7798] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:54.601688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343077312149085:7798] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:54.604902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343077312149085:7798] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:54.604917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343077312149085:7798] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:54.604920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343077312149085:7798] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:54.604929Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2286 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:54.605370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:54.606672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:54.606690Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:54.606692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:54.606696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:54.606703Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2285 [2:7559343076950297403:12175]
2025-10-09T21:51:54.606717Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:54.606733Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:54.606740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:54.606742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:54.606744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:54.606757Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:54.606766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:54.606771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:54.606835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.606842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.606843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.606845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:54.606917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:54.606926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.606935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.606942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.606955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:54.606972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343077312149003:2432], now have 1 active actors on pipe
2025-10-09T21:51:54.606981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343077312149121:7798], now have 1 active actors on pipe
2025-10-09T21:51:54.606986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343076950297398:12175]
2025-10-09T21:51:54.606988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343076950297398:12175]
2025-10-09T21:51:54.606990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343076950297398:12175]
2025-10-09T21:51:54.607000Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:54.607011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:54.607019Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:54.607022Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:54.607024Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:54.607025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:54.607028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:54.607059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343076950297398:12175]
2025-10-09T21:51:54.607070Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:54.607079Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:54.607080Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2285
2025-10-09T21:51:54.607086Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2285
2025-10-09T21:51:54.607135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.607144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.607170Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|adb83386-d48eed12-d7a3e782-7c29bef1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:54.607183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:54.607186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:54.607188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:54.607195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:54.607202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:54.607218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:54.607401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.607418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.607441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:54.607534Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2286 partition: 0 MaxSeqNo: 25 sessionId: src_id|adb83386-d48eed12-d7a3e782-7c29bef1_0
2025-10-09T21:51:54.607812Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fdf6c8b4-eef91171-84dabe09-f7670f51_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|adb83386-d48eed12-d7a3e782-7c29bef1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:54.610244Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|adb83386-d48eed12-d7a3e782-7c29bef1_0] Write session: try to update token
2025-10-09T21:51:54.610263Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|adb83386-d48eed12-d7a3e782-7c29bef1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:54.619240Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:54.619258Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:54.619269Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343076950297398:12175]
2025-10-09T21:51:54.619314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:54.619319Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:54.619512Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:54.619667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:54.619815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:54.619958Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:54.620174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:54.620223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:54.623327Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2286 sessionId: src_id|adb83386-d48eed12-d7a3e782-7c29bef1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:54.626072Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:54.626445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.626464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.626491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:54.626592Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:54.632339Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:54.632354Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:54.632356Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:54.632357Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:54.632365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343076950297398:12175]
2025-10-09T21:51:54.632367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343076950297398:12175]
2025-10-09T21:51:54.632370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343076950297398:12175]
2025-10-09T21:51:54.632401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:54.632407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.632409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:54.632414Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:54.632422Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:54.632425Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:54.632553Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:54.687467Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343077312149151:8652]
2025-10-09T21:51:54.687831Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 2285, pipe: [1:7559343077312149151:8652]
2025-10-09T21:51:54.781034Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:54.791668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.791699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.791866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.791934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.792006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.792075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.792131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.792140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.792220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.792288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.792368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.792452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.792510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.792520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.792590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.792652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.792724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.792881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.792939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.792947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.793019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.793082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.793302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.793402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.793464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.793475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.793584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.793660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.793734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.793806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.793863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.793870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.793945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.794012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.794089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.794155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.794206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.794213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.794278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.794345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.794421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.794499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.794554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.794563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.794630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.794695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.794760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.794830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.794898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.794905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.794971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.795036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.795104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.795168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.795213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.795220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.795292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:54.795367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:54.795438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:54.795504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:54.796056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:54.796072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:54.796131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343077312149151:8652], now have 1 active actors on pipe
2025-10-09T21:51:54.796348Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:54.796375Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343077312149151:8652]
2025-10-09T21:51:54.796516Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:54.796527Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:54.796557Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2286 reason: error at write: Disconnected sessionId: src_id|adb83386-d48eed12-d7a3e782-7c29bef1_0
2025-10-09T21:51:54.796722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2286 sessionId: src_id|adb83386-d48eed12-d7a3e782-7c29bef1_0 is DEAD
2025-10-09T21:51:54.796825Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adb83386-d48eed12-d7a3e782-7c29bef1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:54.796844Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adb83386-d48eed12-d7a3e782-7c29bef1_0] Write session will restart in 0.008083s
2025-10-09T21:51:54.796868Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adb83386-d48eed12-d7a3e782-7c29bef1_0] Write session: Do CDS request
2025-10-09T21:51:54.796873Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adb83386-d48eed12-d7a3e782-7c29bef1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:54.797538Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:54.805660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:54.806496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|adb83386-d48eed12-d7a3e782-7c29bef1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:54.806569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:54.806581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2287
2025-10-09T21:51:54.806687Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2287 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:54.806711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2287 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:54.806718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2287 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:54.806721Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:54.807172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:54.807390Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:54.807401Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:54.807415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:54.807480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:54.807524Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343076950297562:12189]
2025-10-09T21:51:54.807597Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2287 sessionId:  describe result for acl check
2025-10-09T21:51:54.807631Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:54.807633Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:54.807635Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:54.807640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343077312149236:7769] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:54.808101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:54.808149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343077312149236:7769] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:54.808355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:54.808399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:54.808616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:54.808647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.808651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.808652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.808653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.808653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.808654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.808655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.808655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.808656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.808659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.808665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.808667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.808669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.808671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.808674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.808676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.808677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.808679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.808684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.808690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.808694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.808697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.808700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.808703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.808705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.808708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.808711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.808714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.808717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.808720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.808722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.808725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.808728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.808731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.808735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.808739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.808766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:54.810116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343077312149236:7769] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:54.814550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343077312149236:7769] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:54.814566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343077312149236:7769] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:54.814570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343077312149236:7769] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:54.814578Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2287 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:54.815160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:54.816358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:54.816376Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:54.816380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:54.816384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:54.816392Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2286 [2:7559343076950297562:12189]
2025-10-09T21:51:54.816409Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:54.816427Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:54.816437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:54.816442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:54.816444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:54.816460Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:54.816467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:54.816474Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:54.816539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.816551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.816553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.816556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:54.816633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:54.816642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.816654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.816668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.816691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:54.816714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343077312149156:2432], now have 1 active actors on pipe
2025-10-09T21:51:54.816724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343077312149275:7769], now have 1 active actors on pipe
2025-10-09T21:51:54.816731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343076950297553:12189]
2025-10-09T21:51:54.816734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343076950297553:12189]
2025-10-09T21:51:54.816737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343076950297553:12189]
2025-10-09T21:51:54.816754Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:54.816747Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2286
2025-10-09T21:51:54.816772Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:54.816775Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:54.816749Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2286
2025-10-09T21:51:54.816778Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:54.816780Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:54.816792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:54.816795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:54.816851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.816862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.816873Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343076950297553:12189]
2025-10-09T21:51:54.816884Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:54.816898Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:54.816920Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fc08c137-1b7ccec9-f98861c7-88c4d81b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:54.816935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:54.816939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:54.816941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:54.816950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:54.816959Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:54.816982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:54.817140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.817149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.817161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:54.817247Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2287 partition: 0 MaxSeqNo: 25 sessionId: src_id|fc08c137-1b7ccec9-f98861c7-88c4d81b_0
2025-10-09T21:51:54.817542Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|adb83386-d48eed12-d7a3e782-7c29bef1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fc08c137-1b7ccec9-f98861c7-88c4d81b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:54.819886Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc08c137-1b7ccec9-f98861c7-88c4d81b_0] Write session: try to update token
2025-10-09T21:51:54.819906Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc08c137-1b7ccec9-f98861c7-88c4d81b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:54.831044Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:54.831060Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:54.831062Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:54.831064Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:54.831071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343076950297553:12189]
2025-10-09T21:51:54.831076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343076950297553:12189]
2025-10-09T21:51:54.831081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343076950297553:12189]
2025-10-09T21:51:54.831118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:54.831125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.831127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:54.831131Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:54.831138Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:54.831141Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:54.831282Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:54.832838Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2287 sessionId: src_id|fc08c137-1b7ccec9-f98861c7-88c4d81b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:54.835153Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:54.896390Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343077312149284:8652]
2025-10-09T21:51:54.896683Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 2286, pipe: [1:7559343077312149284:8652]
2025-10-09T21:51:54.982630Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:54.983066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:54.983080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:54.983128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343077312149284:8652], now have 1 active actors on pipe
2025-10-09T21:51:54.983333Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:54.983325Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:54.983383Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:54.983391Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:54.983450Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2287 reason: error at write: Disconnected sessionId: src_id|fc08c137-1b7ccec9-f98861c7-88c4d81b_0
2025-10-09T21:51:54.983625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2287 sessionId: src_id|fc08c137-1b7ccec9-f98861c7-88c4d81b_0 is DEAD
2025-10-09T21:51:54.983711Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343077312149284:8652]
2025-10-09T21:51:54.983793Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc08c137-1b7ccec9-f98861c7-88c4d81b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:54.983802Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc08c137-1b7ccec9-f98861c7-88c4d81b_0] Write session will restart in 0.007070s
2025-10-09T21:51:54.983828Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc08c137-1b7ccec9-f98861c7-88c4d81b_0] Write session: Do CDS request
2025-10-09T21:51:54.983835Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc08c137-1b7ccec9-f98861c7-88c4d81b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:54.992458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc08c137-1b7ccec9-f98861c7-88c4d81b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:54.992537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:54.992566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:54.992581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2288
2025-10-09T21:51:54.992692Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2288 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:54.992723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2288 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:54.992732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2288 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:54.992736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:54.993153Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2288 sessionId:  describe result for acl check
2025-10-09T21:51:54.993190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:54.993199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:54.993201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:54.993207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343077312149371:7752] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:54.993753Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343077312149371:7752] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:54.994219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:54.994371Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:54.994384Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:54.994395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:54.994449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:54.994490Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343076950297682:12193]
2025-10-09T21:51:54.994921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:54.995221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:54.995267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:54.995485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343077312149371:7752] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:54.995573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:54.995634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.995640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.995642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.995643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.995644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.995645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.995646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.995648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.995649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.995654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.995664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.995668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.995672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.995676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.995681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.995684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.995688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.995691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.995698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.995706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:54.995712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.995717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:54.995722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.995728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:54.995733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.995739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:54.995743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.995748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:54.995752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.995756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:54.995761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.995766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:54.995771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.995783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:54.995788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.995794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:54.995824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:55.000646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343077312149371:7752] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:55.000660Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343077312149371:7752] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:55.000664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343077312149371:7752] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:55.000673Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2288 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:55.002002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:55.003094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:55.003109Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:55.003112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:55.003116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:55.003123Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2287 [2:7559343076950297682:12193]
2025-10-09T21:51:55.003135Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:55.003153Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:55.003161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:55.003164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:55.003166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:55.003184Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:55.003197Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:55.003203Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:55.003262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.003269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.003271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.003274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:55.003344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:55.003351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.003362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.003379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.003400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:55.003425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343077312149291:2432], now have 1 active actors on pipe
2025-10-09T21:51:55.003433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607116703:7752], now have 1 active actors on pipe
2025-10-09T21:51:55.003441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343076950297676:12193]
2025-10-09T21:51:55.003443Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343076950297676:12193]
2025-10-09T21:51:55.003446Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343076950297676:12193]
2025-10-09T21:51:55.003459Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:55.003480Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:55.003486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:55.003489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:55.003491Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:55.003500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:55.003503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:55.003479Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2287
2025-10-09T21:51:55.003493Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2287
2025-10-09T21:51:55.003530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343076950297676:12193]
2025-10-09T21:51:55.003542Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:55.003554Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:55.003560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.003565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.003599Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9c96fede-ebafd153-f9d5ae0d-4b32768b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:55.003612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:55.003620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:55.003623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:55.003633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:55.003645Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:55.003663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:55.003845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.003852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.003866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:55.003954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2288 partition: 0 MaxSeqNo: 25 sessionId: src_id|9c96fede-ebafd153-f9d5ae0d-4b32768b_0
2025-10-09T21:51:55.004264Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc08c137-1b7ccec9-f98861c7-88c4d81b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9c96fede-ebafd153-f9d5ae0d-4b32768b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:55.006684Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c96fede-ebafd153-f9d5ae0d-4b32768b_0] Write session: try to update token
2025-10-09T21:51:55.006702Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c96fede-ebafd153-f9d5ae0d-4b32768b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:55.018185Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:55.018202Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:55.018204Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:55.018207Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:55.018217Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343076950297676:12193]
2025-10-09T21:51:55.018222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343076950297676:12193]
2025-10-09T21:51:55.018227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343076950297676:12193]
2025-10-09T21:51:55.018266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:55.018272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.018274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:55.018281Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:55.018294Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:55.018301Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:55.018395Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:55.019982Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2288 sessionId: src_id|9c96fede-ebafd153-f9d5ae0d-4b32768b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:55.022384Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:55.083787Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343081607116711:8652]
2025-10-09T21:51:55.084094Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 2287, pipe: [1:7559343081607116711:8652]
2025-10-09T21:51:55.166305Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:55.166550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.166566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.166609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607116711:8652], now have 1 active actors on pipe
2025-10-09T21:51:55.166850Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343081607116711:8652]
2025-10-09T21:51:55.166889Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:55.166923Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:55.167076Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:55.167160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2288 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9c96fede-ebafd153-f9d5ae0d-4b32768b_0
2025-10-09T21:51:55.167348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2288 sessionId: src_id|9c96fede-ebafd153-f9d5ae0d-4b32768b_0 is DEAD
2025-10-09T21:51:55.167486Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c96fede-ebafd153-f9d5ae0d-4b32768b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:55.167501Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c96fede-ebafd153-f9d5ae0d-4b32768b_0] Write session will restart in 0.007670s
2025-10-09T21:51:55.167521Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c96fede-ebafd153-f9d5ae0d-4b32768b_0] Write session: Do CDS request
2025-10-09T21:51:55.167526Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c96fede-ebafd153-f9d5ae0d-4b32768b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:55.175670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:55.176458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c96fede-ebafd153-f9d5ae0d-4b32768b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:55.176555Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:55.176573Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2289
2025-10-09T21:51:55.176696Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2289 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:55.176735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2289 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:55.176745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2289 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:55.176749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:55.177175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:55.177197Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2289 sessionId:  describe result for acl check
2025-10-09T21:51:55.177235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:55.177243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:55.177247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:55.177253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343081607116798:7740] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:55.177344Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:55.177352Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:55.177362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:55.177411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:55.177440Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343081245265096:12195]
2025-10-09T21:51:55.177802Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343081607116798:7740] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:55.177853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:55.178129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:55.178159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:55.178372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:55.178418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.178422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.178423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.178424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.178425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.178425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.178426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.178426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.178427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.178430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.178436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.178439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.178441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.178443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.178445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.178447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.178449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.178451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.178455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.178461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.178465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.178468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.178471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.178474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.178477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.178480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.178483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.178486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.178488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.178492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.178495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.178498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.178500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.178503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.178510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.178513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.178536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:55.179801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343081607116798:7740] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:55.184431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343081607116798:7740] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:55.184446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343081607116798:7740] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:55.184450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343081607116798:7740] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:55.184460Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2289 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:55.184775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:55.185838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:55.185850Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:55.185852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:55.185855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:55.185861Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2288 [2:7559343081245265096:12195]
2025-10-09T21:51:55.185869Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:55.185882Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:55.185884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:55.185885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:55.185887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:55.185899Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:55.185903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:55.185909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:55.185952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.185958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.185959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.185961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:55.186049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:55.186058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.186071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.186086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.186102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:55.186122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607116837:7740], now have 1 active actors on pipe
2025-10-09T21:51:55.186128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607116715:2432], now have 1 active actors on pipe
2025-10-09T21:51:55.186134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343081245265091:12195]
2025-10-09T21:51:55.186135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343081245265091:12195]
2025-10-09T21:51:55.186136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343081245265091:12195]
2025-10-09T21:51:55.186149Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:55.186164Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:55.186166Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:55.186170Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:55.186172Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:55.186176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:55.186178Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:55.186202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343081245265091:12195]
2025-10-09T21:51:55.186209Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:55.186183Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2288
2025-10-09T21:51:55.186184Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2288
2025-10-09T21:51:55.186217Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:55.186237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.186247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.186284Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|690a21b6-8c10bbba-fa081f72-a78bb2b1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:55.186295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:55.186301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:55.186302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:55.186309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:55.186320Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:55.186351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:55.186492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.186503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.186530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:55.186634Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2289 partition: 0 MaxSeqNo: 25 sessionId: src_id|690a21b6-8c10bbba-fa081f72-a78bb2b1_0
2025-10-09T21:51:55.186972Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c96fede-ebafd153-f9d5ae0d-4b32768b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|690a21b6-8c10bbba-fa081f72-a78bb2b1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:55.189362Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|690a21b6-8c10bbba-fa081f72-a78bb2b1_0] Write session: try to update token
2025-10-09T21:51:55.189376Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|690a21b6-8c10bbba-fa081f72-a78bb2b1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:55.200775Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:55.200793Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:55.200796Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:55.200798Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:55.200807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343081245265091:12195]
2025-10-09T21:51:55.200813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343081245265091:12195]
2025-10-09T21:51:55.200819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343081245265091:12195]
2025-10-09T21:51:55.200862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:55.200869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.200868Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:55.200871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:55.200875Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:55.200878Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:55.201002Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:55.202533Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2289 sessionId: src_id|690a21b6-8c10bbba-fa081f72-a78bb2b1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:55.205077Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:55.267083Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343081607116847:8652]
2025-10-09T21:51:55.267337Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 2288, pipe: [1:7559343081607116847:8652]
2025-10-09T21:51:55.349180Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:55.349590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.349602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.349640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607116847:8652], now have 1 active actors on pipe
2025-10-09T21:51:55.349836Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:55.349889Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:55.349921Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343081607116847:8652]
2025-10-09T21:51:55.349947Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:55.350022Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2289 reason: status is not ok: tablet will be restarted right now sessionId: src_id|690a21b6-8c10bbba-fa081f72-a78bb2b1_0
2025-10-09T21:51:55.350208Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2289 sessionId: src_id|690a21b6-8c10bbba-fa081f72-a78bb2b1_0 is DEAD
2025-10-09T21:51:55.350380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|690a21b6-8c10bbba-fa081f72-a78bb2b1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:55.350388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|690a21b6-8c10bbba-fa081f72-a78bb2b1_0] Write session will restart in 0.005323s
2025-10-09T21:51:55.350413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|690a21b6-8c10bbba-fa081f72-a78bb2b1_0] Write session: Do CDS request
2025-10-09T21:51:55.350418Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|690a21b6-8c10bbba-fa081f72-a78bb2b1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:55.356464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|690a21b6-8c10bbba-fa081f72-a78bb2b1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:55.356564Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:55.356577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2290
2025-10-09T21:51:55.356667Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2290 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:55.356710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2290 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:55.356721Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2290 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:55.356725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:55.357167Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2290 sessionId:  describe result for acl check
2025-10-09T21:51:55.357200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:55.357207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:55.357208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:55.357214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343081607116906:7729] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:55.357782Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343081607116906:7729] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:55.359390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:55.359502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343081607116906:7729] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:55.360782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:55.360952Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:55.360961Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:55.360972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:55.361020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:55.361063Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343081245265217:12197]
2025-10-09T21:51:55.361646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:55.361724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343081607116906:7729] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:55.361735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343081607116906:7729] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:55.361738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343081607116906:7729] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:55.361743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2290 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:55.361890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:55.361933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:55.362215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:55.362265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.362270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.362272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.362273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.362274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.362275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.362276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.362277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.362279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.362283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.362293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.362297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.362301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.362305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.362307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.362309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.362311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.362313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.362318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.362323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.362327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.362331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.362333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.362336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.362339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.362342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.362344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.362347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.362349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.362352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.362355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.362358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.362361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.362363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.362367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.362370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.362392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:55.368182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:55.369210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:55.369220Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:55.369222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:55.369224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:55.369229Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2289 [2:7559343081245265217:12197]
2025-10-09T21:51:55.369237Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:55.369251Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:55.369253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:55.369255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:55.369256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:55.369268Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:55.369272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:55.369277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:55.369325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.369331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.369332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.369334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:55.369380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:55.369385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.369392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.369398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.369407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:55.369421Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343081245265212:12197]
2025-10-09T21:51:55.369426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343081245265212:12197]
2025-10-09T21:51:55.369427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343081245265212:12197]
2025-10-09T21:51:55.369433Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:55.369443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:55.369451Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:55.369452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:55.369463Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:55.369468Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:55.369470Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:55.369471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343081245265212:12197]
2025-10-09T21:51:55.369472Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:55.369476Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:55.369488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607116855:2432], now have 1 active actors on pipe
2025-10-09T21:51:55.369490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607116974:7729], now have 1 active actors on pipe
2025-10-09T21:51:55.369571Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2289
2025-10-09T21:51:55.369596Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2289
2025-10-09T21:51:55.369609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.369616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.369637Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a74fbd0b-4095e23-efe0f953-b56c78cd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:55.369646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:55.369648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:55.369650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:55.369656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:55.369667Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:55.369678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:55.369849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.369853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.369862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:55.369939Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2290 partition: 0 MaxSeqNo: 25 sessionId: src_id|a74fbd0b-4095e23-efe0f953-b56c78cd_0
2025-10-09T21:51:55.370268Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|690a21b6-8c10bbba-fa081f72-a78bb2b1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a74fbd0b-4095e23-efe0f953-b56c78cd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:55.372566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a74fbd0b-4095e23-efe0f953-b56c78cd_0] Write session: try to update token
2025-10-09T21:51:55.372581Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a74fbd0b-4095e23-efe0f953-b56c78cd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:55.383183Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:55.383199Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:55.383202Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:55.383204Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:55.383214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343081245265212:12197]
2025-10-09T21:51:55.383218Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343081245265212:12197]
2025-10-09T21:51:55.383223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343081245265212:12197]
2025-10-09T21:51:55.383270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:55.383277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.383276Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:55.383279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:55.383282Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:55.383286Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:55.383407Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:55.385040Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2290 sessionId: src_id|a74fbd0b-4095e23-efe0f953-b56c78cd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:55.387314Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:55.450368Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343081607116984:8652]
2025-10-09T21:51:55.450641Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 2289, pipe: [1:7559343081607116984:8652]
2025-10-09T21:51:55.528277Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:55.528549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.528560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.528597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607116984:8652], now have 1 active actors on pipe
2025-10-09T21:51:55.528794Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:55.528810Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:55.528832Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:55.528814Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:55.528842Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343081607116984:8652]
2025-10-09T21:51:55.528869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2290 reason: error at write: Disconnected sessionId: src_id|a74fbd0b-4095e23-efe0f953-b56c78cd_0
2025-10-09T21:51:55.529036Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2290 sessionId: src_id|a74fbd0b-4095e23-efe0f953-b56c78cd_0 is DEAD
2025-10-09T21:51:55.529198Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a74fbd0b-4095e23-efe0f953-b56c78cd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:55.529211Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a74fbd0b-4095e23-efe0f953-b56c78cd_0] Write session will restart in 0.006820s
2025-10-09T21:51:55.529247Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a74fbd0b-4095e23-efe0f953-b56c78cd_0] Write session: Do CDS request
2025-10-09T21:51:55.529254Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a74fbd0b-4095e23-efe0f953-b56c78cd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:55.537345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:55.537489Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a74fbd0b-4095e23-efe0f953-b56c78cd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:55.537577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:55.537597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2291
2025-10-09T21:51:55.537776Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2291 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:55.537809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2291 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:55.537819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2291 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:55.537823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:55.538233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2291 sessionId:  describe result for acl check
2025-10-09T21:51:55.538267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:55.538272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:55.538274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:55.538279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343081607117071:7716] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:55.538817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343081607117071:7716] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:55.539016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:55.539197Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:55.539209Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:55.539220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:55.539272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:55.539313Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343081245265338:12199]
2025-10-09T21:51:55.539832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:55.540128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:55.540182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:55.540449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:55.540485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.540490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.540490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.540491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.540492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.540492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.540493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.540493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.540494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.540497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.540503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.540505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.540508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.540510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.540512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.540514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.540515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.540518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.540525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.540544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.540550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.540555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.540560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.540565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.540573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.540576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.540579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.540581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.540588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.540591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.540594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.540600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.540603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.540606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.540609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.540618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.540652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:55.540738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343081607117071:7716] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:55.546160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343081607117071:7716] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:55.546177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343081607117071:7716] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:55.546181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343081607117071:7716] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:55.546192Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2291 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:55.546516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:55.547687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:55.547703Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:55.547705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:55.547709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:55.547716Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2290 [2:7559343081245265338:12199]
2025-10-09T21:51:55.547730Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:55.547747Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:55.547756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:55.547759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:55.547761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:55.547780Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:55.547788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:55.547794Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:55.547837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.547844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.547845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.547848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:55.547915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:55.547923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.547935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.547952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.547973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:55.547998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607116987:2432], now have 1 active actors on pipe
2025-10-09T21:51:55.548001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607117110:7716], now have 1 active actors on pipe
2025-10-09T21:51:55.548011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343081245265333:12199]
2025-10-09T21:51:55.548017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343081245265333:12199]
2025-10-09T21:51:55.548020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343081245265333:12199]
2025-10-09T21:51:55.548033Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:55.548049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:55.548050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:55.548049Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2290
2025-10-09T21:51:55.548057Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:55.548059Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:55.548053Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2290
2025-10-09T21:51:55.548063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:55.548066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:55.548092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343081245265333:12199]
2025-10-09T21:51:55.548095Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:55.548097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:55.548120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.548129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.548250Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|aec947c0-63d8e66f-cc836b38-1aadf79a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:55.548268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:55.548272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:55.548275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:55.548285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:55.548296Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:55.548318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:55.548498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.548507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.548523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:55.548623Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2291 partition: 0 MaxSeqNo: 25 sessionId: src_id|aec947c0-63d8e66f-cc836b38-1aadf79a_0
2025-10-09T21:51:55.548913Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a74fbd0b-4095e23-efe0f953-b56c78cd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|aec947c0-63d8e66f-cc836b38-1aadf79a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:55.551364Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aec947c0-63d8e66f-cc836b38-1aadf79a_0] Write session: try to update token
2025-10-09T21:51:55.551383Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aec947c0-63d8e66f-cc836b38-1aadf79a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:55.561431Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:55.561445Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:55.561447Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:55.561449Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:55.561462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343081245265333:12199]
2025-10-09T21:51:55.561466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343081245265333:12199]
2025-10-09T21:51:55.561469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343081245265333:12199]
2025-10-09T21:51:55.561506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:55.561512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.561513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:55.561521Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:55.561534Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:55.561538Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:55.561646Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:55.563913Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2291 sessionId: src_id|aec947c0-63d8e66f-cc836b38-1aadf79a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:55.566180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:55.629036Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343081607117117:8652]
2025-10-09T21:51:55.629273Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 2290, pipe: [1:7559343081607117117:8652]
2025-10-09T21:51:55.702171Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:55.702637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.702651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.702687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607117117:8652], now have 1 active actors on pipe
2025-10-09T21:51:55.702911Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:55.702953Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343081607117117:8652]
2025-10-09T21:51:55.702986Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:55.703019Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:55.703101Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2291 reason: status is not ok: tablet will be restarted right now sessionId: src_id|aec947c0-63d8e66f-cc836b38-1aadf79a_0
2025-10-09T21:51:55.703275Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2291 sessionId: src_id|aec947c0-63d8e66f-cc836b38-1aadf79a_0 is DEAD
2025-10-09T21:51:55.703388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aec947c0-63d8e66f-cc836b38-1aadf79a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:55.703394Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aec947c0-63d8e66f-cc836b38-1aadf79a_0] Write session will restart in 0.005046s
2025-10-09T21:51:55.703410Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aec947c0-63d8e66f-cc836b38-1aadf79a_0] Write session: Do CDS request
2025-10-09T21:51:55.703415Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aec947c0-63d8e66f-cc836b38-1aadf79a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:55.709481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aec947c0-63d8e66f-cc836b38-1aadf79a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:55.709553Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:55.709568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2292
2025-10-09T21:51:55.709704Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2292 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:55.709757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2292 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:55.709767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2292 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:55.709770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:55.710138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2292 sessionId:  describe result for acl check
2025-10-09T21:51:55.710167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:55.710179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:55.710181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:55.710185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343081607117194:7682] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:55.710676Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343081607117194:7682] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:55.712093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343081607117194:7682] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:55.712343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:55.713779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:55.713985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343081607117194:7682] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:55.713996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343081607117194:7682] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:55.713999Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343081607117194:7682] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:55.714006Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2292 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:55.714153Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:55.714163Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:55.714177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:55.714240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:55.714283Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343081245265498:12207]
2025-10-09T21:51:55.714842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:55.715092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:55.715134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:55.715344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:55.715374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.715379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.715380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.715380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.715381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.715382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.715382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.715383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.715384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.715388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.715398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.715401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.715406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.715410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.715414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.715418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.715421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.715430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.715436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.715444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.715454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.715461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.715465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.715469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.715473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.715478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.715482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.715491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.715495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.715506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.715515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.715520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.715524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.715530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.715534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.715539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.715563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:55.721579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:55.722940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:55.722954Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:55.722956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:55.722960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:55.722967Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2291 [2:7559343081245265498:12207]
2025-10-09T21:51:55.722979Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:55.722998Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:55.723002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:55.723004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:55.723007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:55.723023Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:55.723030Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:55.723039Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:55.723098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.723107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.723109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.723112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:55.723179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:55.723186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.723197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.723207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.723225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:55.723247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607117261:7682], now have 1 active actors on pipe
2025-10-09T21:51:55.723256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607117132:2432], now have 1 active actors on pipe
2025-10-09T21:51:55.723263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343081245265493:12207]
2025-10-09T21:51:55.723265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343081245265493:12207]
2025-10-09T21:51:55.723267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343081245265493:12207]
2025-10-09T21:51:55.723280Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:55.723297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:55.723299Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:55.723303Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:55.723305Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:55.723310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:55.723313Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:55.723337Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343081245265493:12207]
2025-10-09T21:51:55.723313Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2291
2025-10-09T21:51:55.723322Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2291
2025-10-09T21:51:55.723345Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:55.723347Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:55.723361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.723365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.723419Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c6e39d9a-86ab8e43-f18ab7a7-6d40ca82_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:55.723435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:55.723440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:55.723443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:55.723452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:55.723462Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:55.723484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:55.723609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.723619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.723634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:55.723770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2292 partition: 0 MaxSeqNo: 25 sessionId: src_id|c6e39d9a-86ab8e43-f18ab7a7-6d40ca82_0
2025-10-09T21:51:55.724063Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aec947c0-63d8e66f-cc836b38-1aadf79a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c6e39d9a-86ab8e43-f18ab7a7-6d40ca82_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:55.726440Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c6e39d9a-86ab8e43-f18ab7a7-6d40ca82_0] Write session: try to update token
2025-10-09T21:51:55.726457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c6e39d9a-86ab8e43-f18ab7a7-6d40ca82_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:55.737456Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:55.737472Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:55.737475Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:55.737477Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:55.737487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343081245265493:12207]
2025-10-09T21:51:55.737491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343081245265493:12207]
2025-10-09T21:51:55.737495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343081245265493:12207]
2025-10-09T21:51:55.737541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:55.737550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.737557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:55.737562Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:55.737569Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:55.737589Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:55.737699Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:55.738623Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2292 sessionId: src_id|c6e39d9a-86ab8e43-f18ab7a7-6d40ca82_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:55.740899Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:55.803146Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343081607117273:8652]
2025-10-09T21:51:55.803524Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 2291, pipe: [1:7559343081607117273:8652]
2025-10-09T21:51:55.884365Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:55.884752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.884769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.884816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607117273:8652], now have 1 active actors on pipe
2025-10-09T21:51:55.885037Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:55.885028Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:55.885044Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:55.885104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2292 reason: error at write: Disconnected sessionId: src_id|c6e39d9a-86ab8e43-f18ab7a7-6d40ca82_0
2025-10-09T21:51:55.885279Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2292 sessionId: src_id|c6e39d9a-86ab8e43-f18ab7a7-6d40ca82_0 is DEAD
2025-10-09T21:51:55.885411Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343081607117273:8652]
2025-10-09T21:51:55.885444Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6e39d9a-86ab8e43-f18ab7a7-6d40ca82_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:55.885454Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6e39d9a-86ab8e43-f18ab7a7-6d40ca82_0] Write session will restart in 0.005167s
2025-10-09T21:51:55.885447Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:55.885478Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6e39d9a-86ab8e43-f18ab7a7-6d40ca82_0] Write session: Do CDS request
2025-10-09T21:51:55.885499Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6e39d9a-86ab8e43-f18ab7a7-6d40ca82_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:55.891533Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c6e39d9a-86ab8e43-f18ab7a7-6d40ca82_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:55.891606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:55.891624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2293
2025-10-09T21:51:55.891773Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2293 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:55.891810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2293 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:55.891821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2293 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:55.891826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:55.892276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2293 sessionId:  describe result for acl check
2025-10-09T21:51:55.892314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:55.892321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:55.892323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:55.892329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343081607117324:7679] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:55.892867Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343081607117324:7679] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:55.895124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:55.895236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343081607117324:7679] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:55.896733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:55.896918Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:55.896928Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:55.896941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:55.896987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:55.897017Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343081245265626:12217]
2025-10-09T21:51:55.897520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:55.897879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:55.897946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:55.897954Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343081607117324:7679] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:55.897971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343081607117324:7679] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:55.897974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343081607117324:7679] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:55.897983Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2293 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:55.898272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:55.898359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.898365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.898366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.898367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.898369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.898370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.898371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.898372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.898374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.898379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.898390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.898394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.898398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.898401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.898405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.898409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.898412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.898415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.898422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.898429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:55.898435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.898440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:55.898444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.898449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:55.898453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.898458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:55.898463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.898468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:55.898472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.898477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:55.898481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.898486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.898491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.898496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.898506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.898512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.898540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:55.904736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:55.905764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:55.905778Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:55.905780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:55.905785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:55.905790Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2292 [2:7559343081245265626:12217]
2025-10-09T21:51:55.905799Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:55.905815Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:55.905817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:55.905819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:55.905821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:55.905835Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:55.905842Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:55.905848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:55.905895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.905903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.905905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.905908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:55.905977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:55.905984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.905992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:55.905998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:55.906007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:55.906025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343081245265621:12217]
2025-10-09T21:51:55.906046Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343081245265621:12217]
2025-10-09T21:51:55.906049Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343081245265621:12217]
2025-10-09T21:51:55.906065Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:55.906085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607117276:2432], now have 1 active actors on pipe
2025-10-09T21:51:55.906089Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:55.906092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607117398:7679], now have 1 active actors on pipe
2025-10-09T21:51:55.906093Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:55.906095Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:55.906100Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:55.906114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:55.906122Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:55.906157Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343081245265621:12217]
2025-10-09T21:51:55.906165Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:55.906167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:55.906139Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2292
2025-10-09T21:51:55.906181Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2292
2025-10-09T21:51:55.906217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.906226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.906253Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ebc90294-43918bf2-f470ece6-35b96c04_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:55.906265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:55.906273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:55.906276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:55.906286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:55.906299Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:55.906316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:55.906465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.906478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.906502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:55.906587Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2293 partition: 0 MaxSeqNo: 25 sessionId: src_id|ebc90294-43918bf2-f470ece6-35b96c04_0
2025-10-09T21:51:55.906919Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c6e39d9a-86ab8e43-f18ab7a7-6d40ca82_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ebc90294-43918bf2-f470ece6-35b96c04_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:55.909464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ebc90294-43918bf2-f470ece6-35b96c04_0] Write session: try to update token
2025-10-09T21:51:55.909482Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ebc90294-43918bf2-f470ece6-35b96c04_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:55.922932Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2293 sessionId: src_id|ebc90294-43918bf2-f470ece6-35b96c04_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:55.923284Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:55.923298Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:55.923308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343081245265621:12217]
2025-10-09T21:51:55.923356Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:55.923363Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:55.923534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:55.923680Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:55.923816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:55.923969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:55.924161Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:55.924203Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:55.925348Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:55.925534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:55.925548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:55.925584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:55.925686Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:55.930633Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:55.930644Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:55.930645Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:55.930647Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:55.930655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343081245265621:12217]
2025-10-09T21:51:55.930658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343081245265621:12217]
2025-10-09T21:51:55.930661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343081245265621:12217]
2025-10-09T21:51:55.930701Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:55.930706Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:55.930709Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:55.930709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:55.930720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:55.930723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:55.930859Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:55.985436Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343081607117409:8652]
2025-10-09T21:51:55.985706Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 2292, pipe: [1:7559343081607117409:8652]
2025-10-09T21:51:56.080911Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:56.088882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:56.088907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:56.089062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:56.089125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:56.089191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:56.089263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:56.089323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:56.089333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:56.089400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:56.089457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:56.089531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:56.089607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:56.089661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:56.089670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:56.089739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:56.089793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:56.089860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:56.089922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:56.089963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:56.089969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:56.090053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:56.090122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:56.090185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:56.090251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:56.090570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:56.090581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:56.090647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:56.090716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:56.090783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:56.090840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:56.090881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:56.090887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:56.090955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:56.091021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:56.091088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:56.091192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:56.091249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:56.091259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:56.091333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:56.091404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:56.091476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:56.091546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:56.091614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:56.091619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:56.091685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:56.091761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:56.091832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:56.091889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:56.091933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:56.091941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:56.092011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:56.092079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:56.092142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:56.092212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:56.092269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:56.092276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:56.092343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:56.092403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:56.092465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:56.092535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:56.092591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:56.092599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:56.092660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343081607117409:8652], now have 1 active actors on pipe
2025-10-09T21:51:56.092869Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:56.092891Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:56.092895Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:56.092928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2293 reason: error at write: Disconnected sessionId: src_id|ebc90294-43918bf2-f470ece6-35b96c04_0
2025-10-09T21:51:56.092931Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343081607117409:8652]
2025-10-09T21:51:56.093117Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2293 sessionId: src_id|ebc90294-43918bf2-f470ece6-35b96c04_0 is DEAD
2025-10-09T21:51:56.093293Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebc90294-43918bf2-f470ece6-35b96c04_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:56.093305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebc90294-43918bf2-f470ece6-35b96c04_0] Write session will restart in 0.006394s
2025-10-09T21:51:56.093338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebc90294-43918bf2-f470ece6-35b96c04_0] Write session: Do CDS request
2025-10-09T21:51:56.093345Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebc90294-43918bf2-f470ece6-35b96c04_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:56.093466Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:56.100537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ebc90294-43918bf2-f470ece6-35b96c04_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:56.100616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:56.100633Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2294
2025-10-09T21:51:56.100808Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2294 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:56.100841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2294 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:56.100851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2294 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:56.100854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:56.101387Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2294 sessionId:  describe result for acl check
2025-10-09T21:51:56.101422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:56.101429Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:56.101431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:56.101436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343085902084773:7661] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:56.101893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343085902084773:7661] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:56.103474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:56.104371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343085902084773:7661] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:56.105262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:56.105454Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:56.105464Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:56.105473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:56.105516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:56.105549Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343085540233046:12219]
2025-10-09T21:51:56.105967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:56.106259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:56.106303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:56.106565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:56.106627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.106641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.106642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.106644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.106645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.106646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.106648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.106649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.106651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.106659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.106670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.106675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.106679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.106682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.106686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.106689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.106693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.106697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.106706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.106715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.106723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.106739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.106753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.106772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.106778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.106784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.106791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.106798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.106803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.106809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.106814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.106820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.106824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.106830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.106835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.106865Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343085902084773:7661] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:56.106840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.106870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:56.106897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343085902084773:7661] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:56.106900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343085902084773:7661] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:56.106909Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2294 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:56.113567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:56.114678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:56.114696Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:56.114699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:56.114704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:56.114712Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2293 [2:7559343085540233046:12219]
2025-10-09T21:51:56.114730Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:56.114754Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:56.114758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:56.114762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:56.114765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:56.114785Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:56.114800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:56.114808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:56.114891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.114902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.114905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.114907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:56.114994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:56.115002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.115016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.115034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.115058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:56.115086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343085902084708:2432], now have 1 active actors on pipe
2025-10-09T21:51:56.115094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343085902084832:7661], now have 1 active actors on pipe
2025-10-09T21:51:56.115104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343085540233040:12219]
2025-10-09T21:51:56.115106Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343085540233040:12219]
2025-10-09T21:51:56.115108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343085540233040:12219]
2025-10-09T21:51:56.115120Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:56.115135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:56.115141Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:56.115144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:56.115147Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:56.115149Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:56.115152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:56.115150Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2293
2025-10-09T21:51:56.115188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343085540233040:12219]
2025-10-09T21:51:56.115161Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2293
2025-10-09T21:51:56.115194Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:56.115197Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:56.115216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:56.115220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:56.115272Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|10a0b900-19ed2e18-9ac01d38-ed82683f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:56.115285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:56.115290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:56.115293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:56.115305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:56.115320Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:56.115351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:56.115580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:56.115606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:56.115630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:56.115726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2294 partition: 0 MaxSeqNo: 25 sessionId: src_id|10a0b900-19ed2e18-9ac01d38-ed82683f_0
2025-10-09T21:51:56.116114Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ebc90294-43918bf2-f470ece6-35b96c04_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|10a0b900-19ed2e18-9ac01d38-ed82683f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:56.118508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10a0b900-19ed2e18-9ac01d38-ed82683f_0] Write session: try to update token
2025-10-09T21:51:56.118524Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10a0b900-19ed2e18-9ac01d38-ed82683f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:56.130059Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:56.130073Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:56.130075Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:56.130077Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:56.130087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343085540233040:12219]
2025-10-09T21:51:56.130091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343085540233040:12219]
2025-10-09T21:51:56.130093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343085540233040:12219]
2025-10-09T21:51:56.130131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:56.130143Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:56.130147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.130148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:56.130149Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:56.130153Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:56.130285Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:56.132146Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2294 sessionId: src_id|10a0b900-19ed2e18-9ac01d38-ed82683f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:56.134584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:56.193303Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343085902084844:8652]
2025-10-09T21:51:56.193609Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 2293, pipe: [1:7559343085902084844:8652]
2025-10-09T21:51:56.278347Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:56.278812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:56.278827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:56.278870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343085902084844:8652], now have 1 active actors on pipe
2025-10-09T21:51:56.279116Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343085902084844:8652]
2025-10-09T21:51:56.279131Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:56.279148Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:56.279150Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:56.279205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2294 reason: error at write: Disconnected sessionId: src_id|10a0b900-19ed2e18-9ac01d38-ed82683f_0
2025-10-09T21:51:56.279350Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2294 sessionId: src_id|10a0b900-19ed2e18-9ac01d38-ed82683f_0 is DEAD
2025-10-09T21:51:56.279482Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:56.279510Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10a0b900-19ed2e18-9ac01d38-ed82683f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:56.279523Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10a0b900-19ed2e18-9ac01d38-ed82683f_0] Write session will restart in 0.005140s
2025-10-09T21:51:56.279552Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10a0b900-19ed2e18-9ac01d38-ed82683f_0] Write session: Do CDS request
2025-10-09T21:51:56.279559Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10a0b900-19ed2e18-9ac01d38-ed82683f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:56.285525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10a0b900-19ed2e18-9ac01d38-ed82683f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:56.285590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:56.285607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2295
2025-10-09T21:51:56.285762Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2295 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:56.285802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2295 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:56.285808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2295 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:56.285812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:56.286343Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2295 sessionId:  describe result for acl check
2025-10-09T21:51:56.286375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:56.286381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:56.286384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:56.286389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343085902084892:7643] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:56.286932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343085902084892:7643] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:56.288894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343085902084892:7643] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:56.288934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:56.290568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:56.290726Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:56.290733Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:56.290743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:56.290785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:56.290813Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343085540233172:12221]
2025-10-09T21:51:56.291120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343085902084892:7643] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:56.291137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343085902084892:7643] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:56.291140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343085902084892:7643] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:56.291153Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2295 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:56.291279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:56.291552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:56.291605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:56.291928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:56.291964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.291969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.291969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.291970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.291971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.291971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.291972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.291972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.291973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.291976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.291982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.291984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.291986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.291988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.291990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.291992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.291994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.291996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.292001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.292006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.292012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.292015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.292018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.292021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.292024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.292027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.292030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.292033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.292035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.292038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.292042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.292044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.292048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.292051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.292053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.292056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.292072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:56.298147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:56.299169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:56.299182Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:56.299184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:56.299189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:56.299196Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2294 [2:7559343085540233172:12221]
2025-10-09T21:51:56.299208Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:56.299224Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:56.299228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:56.299231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:56.299233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:56.299251Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:56.299261Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:56.299268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:56.299325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.299334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.299336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.299339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:56.299404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:56.299411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.299423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.299435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.299457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:56.299483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343085902084849:2432], now have 1 active actors on pipe
2025-10-09T21:51:56.299491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343085902084972:7643], now have 1 active actors on pipe
2025-10-09T21:51:56.299498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343085540233166:12221]
2025-10-09T21:51:56.299503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343085540233166:12221]
2025-10-09T21:51:56.299506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343085540233166:12221]
2025-10-09T21:51:56.299518Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:56.299535Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:56.299536Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:56.299543Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:56.299546Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:56.299550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:56.299553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:56.299532Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2294
2025-10-09T21:51:56.299552Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2294
2025-10-09T21:51:56.299584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343085540233166:12221]
2025-10-09T21:51:56.299595Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:56.299601Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:56.299613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:56.299618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:56.299666Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7c695b59-eeaabc74-813c3c73-4931ba68_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:56.299679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:56.299682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:56.299684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:56.299695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:56.299708Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:56.299723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:56.299833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:56.299840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:56.299856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:56.299930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2295 partition: 0 MaxSeqNo: 25 sessionId: src_id|7c695b59-eeaabc74-813c3c73-4931ba68_0
2025-10-09T21:51:56.300311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10a0b900-19ed2e18-9ac01d38-ed82683f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7c695b59-eeaabc74-813c3c73-4931ba68_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:56.302792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c695b59-eeaabc74-813c3c73-4931ba68_0] Write session: try to update token
2025-10-09T21:51:56.302812Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c695b59-eeaabc74-813c3c73-4931ba68_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:56.313605Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:56.313619Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:56.313622Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:56.313624Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:56.313633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343085540233166:12221]
2025-10-09T21:51:56.313637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343085540233166:12221]
2025-10-09T21:51:56.313641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343085540233166:12221]
2025-10-09T21:51:56.313682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:56.313688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.313691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:56.313707Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:56.313717Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:56.313721Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:56.313837Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:56.315832Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2295 sessionId: src_id|7c695b59-eeaabc74-813c3c73-4931ba68_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:56.318245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:56.379106Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343085902084989:8652]
2025-10-09T21:51:56.379362Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 2294, pipe: [1:7559343085902084989:8652]
2025-10-09T21:51:56.458521Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:56.458863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:56.458877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:56.458914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343085902084989:8652], now have 1 active actors on pipe
2025-10-09T21:51:56.459082Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:56.459112Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:56.459120Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:56.459124Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:56.459176Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343085902084989:8652]
2025-10-09T21:51:56.459278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2295 reason: error at write: Disconnected sessionId: src_id|7c695b59-eeaabc74-813c3c73-4931ba68_0
2025-10-09T21:51:56.459461Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2295 sessionId: src_id|7c695b59-eeaabc74-813c3c73-4931ba68_0 is DEAD
2025-10-09T21:51:56.459630Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c695b59-eeaabc74-813c3c73-4931ba68_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:56.459640Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c695b59-eeaabc74-813c3c73-4931ba68_0] Write session will restart in 0.006612s
2025-10-09T21:51:56.459667Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c695b59-eeaabc74-813c3c73-4931ba68_0] Write session: Do CDS request
2025-10-09T21:51:56.459672Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c695b59-eeaabc74-813c3c73-4931ba68_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:56.467465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c695b59-eeaabc74-813c3c73-4931ba68_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:56.467569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:56.467586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2296
2025-10-09T21:51:56.467699Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2296 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:56.467736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2296 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:56.467747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2296 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:56.467763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:56.468163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:56.468218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2296 sessionId:  describe result for acl check
2025-10-09T21:51:56.468258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:56.468265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:56.468266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:56.468272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343085902085079:7624] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:56.468824Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343085902085079:7624] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:56.469725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:56.469891Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:56.469904Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:56.469918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:56.469971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:56.470010Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343085540233297:12223]
2025-10-09T21:51:56.470547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:56.470587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343085902085079:7624] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:56.470816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:56.470859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:56.471082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:56.471127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.471133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.471134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.471135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.471136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.471137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.471139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.471140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.471141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.471144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.471151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.471153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.471155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.471157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.471159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.471161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.471163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.471165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.471170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.471174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.471178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.471181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.471184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.471187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.471189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.471192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.471198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.471203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.471207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.471212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.471217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.471222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.471226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.471231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.471236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.471239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.471255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:56.475980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343085902085079:7624] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:56.475997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343085902085079:7624] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:56.476002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343085902085079:7624] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:56.476010Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2296 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:56.477264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:56.478438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:56.478455Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:56.478458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:56.478462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:56.478469Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2295 [2:7559343085540233297:12223]
2025-10-09T21:51:56.478483Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:56.478500Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:56.478511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:56.478514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:56.478517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:56.478533Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:56.478538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:56.478546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:56.478604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.478611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.478613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.478616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:56.478694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:56.478702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.478715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.478733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.478756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:56.478782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343085902085119:7624], now have 1 active actors on pipe
2025-10-09T21:51:56.478790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343085902084991:2432], now have 1 active actors on pipe
2025-10-09T21:51:56.478798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343085540233292:12223]
2025-10-09T21:51:56.478800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343085540233292:12223]
2025-10-09T21:51:56.478803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343085540233292:12223]
2025-10-09T21:51:56.478820Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:56.478821Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2295
2025-10-09T21:51:56.478839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:56.478834Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2295
2025-10-09T21:51:56.478838Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:56.478842Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:56.478843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:56.478859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:56.478862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:56.478906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:56.478915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:56.478927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343085540233292:12223]
2025-10-09T21:51:56.478938Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:56.478942Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:56.478974Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|66ba1ff8-68b5f75f-b78ca75e-dc89bc62_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:56.478986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:56.478995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:56.479002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:56.479013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:56.479023Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:56.479181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:56.479423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:56.479432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:56.479448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:56.479515Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2296 partition: 0 MaxSeqNo: 25 sessionId: src_id|66ba1ff8-68b5f75f-b78ca75e-dc89bc62_0
2025-10-09T21:51:56.479821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c695b59-eeaabc74-813c3c73-4931ba68_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|66ba1ff8-68b5f75f-b78ca75e-dc89bc62_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:56.482209Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66ba1ff8-68b5f75f-b78ca75e-dc89bc62_0] Write session: try to update token
2025-10-09T21:51:56.482229Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66ba1ff8-68b5f75f-b78ca75e-dc89bc62_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:56.492757Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:56.492778Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:56.492781Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:56.492783Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:56.492796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343085540233292:12223]
2025-10-09T21:51:56.492801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343085540233292:12223]
2025-10-09T21:51:56.492805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343085540233292:12223]
2025-10-09T21:51:56.492854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:56.492861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.492864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:56.492863Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:56.492868Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:56.492871Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:56.493000Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:56.494951Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2296 sessionId: src_id|66ba1ff8-68b5f75f-b78ca75e-dc89bc62_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:56.497245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:56.559349Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343085902085127:8652]
2025-10-09T21:51:56.559604Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 2295, pipe: [1:7559343085902085127:8652]
2025-10-09T21:51:56.639273Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:56.639612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:56.639630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:56.639678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343085902085127:8652], now have 1 active actors on pipe
2025-10-09T21:51:56.639888Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:56.639912Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:56.639913Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:56.639977Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2296 reason: error at write: Disconnected sessionId: src_id|66ba1ff8-68b5f75f-b78ca75e-dc89bc62_0
2025-10-09T21:51:56.639981Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343085902085127:8652]
2025-10-09T21:51:56.640120Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2296 sessionId: src_id|66ba1ff8-68b5f75f-b78ca75e-dc89bc62_0 is DEAD
2025-10-09T21:51:56.640146Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:56.640249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66ba1ff8-68b5f75f-b78ca75e-dc89bc62_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:56.640257Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66ba1ff8-68b5f75f-b78ca75e-dc89bc62_0] Write session will restart in 0.008934s
2025-10-09T21:51:56.640284Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66ba1ff8-68b5f75f-b78ca75e-dc89bc62_0] Write session: Do CDS request
2025-10-09T21:51:56.640291Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66ba1ff8-68b5f75f-b78ca75e-dc89bc62_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:56.649784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:56.650403Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|66ba1ff8-68b5f75f-b78ca75e-dc89bc62_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:56.650487Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:56.650506Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2297
2025-10-09T21:51:56.650629Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2297 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:56.650665Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2297 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:56.650678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2297 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:56.650681Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:56.651092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2297 sessionId:  describe result for acl check
2025-10-09T21:51:56.651126Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:56.651131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:56.651132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:56.651138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343085902085219:7619] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:56.651452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:56.651603Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:56.651610Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:56.651618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:56.651616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343085902085219:7619] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:56.651657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:56.651689Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343085540233425:12225]
2025-10-09T21:51:56.652087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:56.652307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:56.652334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:56.652572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:56.652609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.652613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.652614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.652614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.652615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.652616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.652616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.652617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.652617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.652620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.652626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.652628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.652630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.652632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.652634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.652636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.652638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.652640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.652644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.652649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.652653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.652656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.652659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.652662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.652664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.652667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.652669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.652672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.652674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.652678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.652680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.652683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.652686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.652689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.652691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.652696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.652723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:56.653243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343085902085219:7619] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:56.658269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343085902085219:7619] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:56.658284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343085902085219:7619] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:56.658287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343085902085219:7619] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:56.658294Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2297 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:56.658666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:56.659691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:56.659707Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:56.659709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:56.659713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:56.659720Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2296 [2:7559343085540233425:12225]
2025-10-09T21:51:56.659733Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:56.659750Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:56.659753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:56.659756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:56.659758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:56.659774Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:56.659783Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:56.659790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:56.659845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.659852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.659855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.659857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:56.659926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:56.659933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.659945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.659956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.659971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:56.659996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343085902085129:2432], now have 1 active actors on pipe
2025-10-09T21:51:56.660004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343085902085258:7619], now have 1 active actors on pipe
2025-10-09T21:51:56.660011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343085540233420:12225]
2025-10-09T21:51:56.660013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343085540233420:12225]
2025-10-09T21:51:56.660015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343085540233420:12225]
2025-10-09T21:51:56.660027Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:56.660042Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:56.660042Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:56.660045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:56.660047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:56.660059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:56.660062Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:56.660066Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2296
2025-10-09T21:51:56.660069Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2296
2025-10-09T21:51:56.660091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343085540233420:12225]
2025-10-09T21:51:56.660099Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:56.660101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:56.660132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:56.660137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:56.660184Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9c09f69-37b6cf04-8afa7bbf-2173393d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:56.660199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:56.660203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:56.660206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:56.660215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:56.660225Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:56.660244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:56.660383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:56.660401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:56.660419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:56.660497Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2297 partition: 0 MaxSeqNo: 25 sessionId: src_id|9c09f69-37b6cf04-8afa7bbf-2173393d_0
2025-10-09T21:51:56.660810Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|66ba1ff8-68b5f75f-b78ca75e-dc89bc62_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9c09f69-37b6cf04-8afa7bbf-2173393d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:56.663019Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c09f69-37b6cf04-8afa7bbf-2173393d_0] Write session: try to update token
2025-10-09T21:51:56.663034Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c09f69-37b6cf04-8afa7bbf-2173393d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:56.673305Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:56.673333Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:56.673335Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:56.673337Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:56.673347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343085540233420:12225]
2025-10-09T21:51:56.673352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343085540233420:12225]
2025-10-09T21:51:56.673356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343085540233420:12225]
2025-10-09T21:51:56.673399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:56.673406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.673409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:56.673420Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:56.673426Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:56.673432Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:56.673532Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:56.675234Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2297 sessionId: src_id|9c09f69-37b6cf04-8afa7bbf-2173393d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:56.677464Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:56.740087Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343085902085279:8652]
2025-10-09T21:51:56.740444Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 2296, pipe: [1:7559343085902085279:8652]
2025-10-09T21:51:56.824413Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:56.824788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:56.824802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:56.824842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343085902085279:8652], now have 1 active actors on pipe
2025-10-09T21:51:56.825048Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:56.825136Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:56.825139Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:56.825155Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:56.825184Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343085902085279:8652]
2025-10-09T21:51:56.825220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2297 reason: error at write: Disconnected sessionId: src_id|9c09f69-37b6cf04-8afa7bbf-2173393d_0
2025-10-09T21:51:56.825414Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2297 sessionId: src_id|9c09f69-37b6cf04-8afa7bbf-2173393d_0 is DEAD
2025-10-09T21:51:56.825572Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c09f69-37b6cf04-8afa7bbf-2173393d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:56.825584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c09f69-37b6cf04-8afa7bbf-2173393d_0] Write session will restart in 0.006569s
2025-10-09T21:51:56.825614Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c09f69-37b6cf04-8afa7bbf-2173393d_0] Write session: Do CDS request
2025-10-09T21:51:56.825620Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c09f69-37b6cf04-8afa7bbf-2173393d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:56.833565Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c09f69-37b6cf04-8afa7bbf-2173393d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:56.833654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:56.833669Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2298
2025-10-09T21:51:56.833771Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2298 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:56.833804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2298 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:56.833809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2298 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:56.833813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:56.834321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2298 sessionId:  describe result for acl check
2025-10-09T21:51:56.834360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:56.834366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:56.834368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:56.834374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343085902085381:7562] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:56.834930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343085902085381:7562] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:56.835491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:56.836791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343085902085381:7562] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:56.837087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:56.837369Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:56.837379Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:56.837392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:56.837453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:56.837490Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343085540233595:12241]
2025-10-09T21:51:56.837906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:56.838173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:56.838219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:56.838429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:56.838474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.838480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.838481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.838482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.838484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.838485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.838486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.838486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.838487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.838491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.838497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.838499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.838501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.838503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.838506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.838508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.838510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.838512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.838516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.838521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:56.838525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.838529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:56.838531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.838534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:56.838536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.838540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:56.838542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.838545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:56.838549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.838553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:56.838556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.838558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.838562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.838565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.838567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.838570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.838586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:56.843236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343085902085381:7562] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:56.843250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343085902085381:7562] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:56.843253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343085902085381:7562] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:56.843260Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2298 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:56.844598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:56.845693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:56.845710Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:56.845713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:56.845716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:56.845722Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2297 [2:7559343085540233595:12241]
2025-10-09T21:51:56.845735Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:56.845751Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:56.845755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:56.845757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:56.845760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:56.845777Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:56.845790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:56.845797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:56.845850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.845857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.845859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.845861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:56.845924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:56.845930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.845941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:56.845952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:56.845973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:56.845998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343085902085293:2432], now have 1 active actors on pipe
2025-10-09T21:51:56.846001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343085902085424:7562], now have 1 active actors on pipe
2025-10-09T21:51:56.846009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343085540233589:12241]
2025-10-09T21:51:56.846011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343085540233589:12241]
2025-10-09T21:51:56.846013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343085540233589:12241]
2025-10-09T21:51:56.846044Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:56.846060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:56.846063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:56.846071Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:56.846060Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2297
2025-10-09T21:51:56.846073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:56.846077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:56.846080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:56.846103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343085540233589:12241]
2025-10-09T21:51:56.846108Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:56.846110Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:56.846143Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2297
2025-10-09T21:51:56.846215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:56.846224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:56.846251Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1c8b9eef-147f2b74-28802dfe-d4921aff_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:56.846263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:56.846273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:56.846279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:56.846289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:56.846299Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:56.846317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:56.846454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:56.846464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:56.846482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:56.846592Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2298 partition: 0 MaxSeqNo: 25 sessionId: src_id|1c8b9eef-147f2b74-28802dfe-d4921aff_0
2025-10-09T21:51:56.846896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c09f69-37b6cf04-8afa7bbf-2173393d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1c8b9eef-147f2b74-28802dfe-d4921aff_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:56.849287Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c8b9eef-147f2b74-28802dfe-d4921aff_0] Write session: try to update token
2025-10-09T21:51:56.849305Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c8b9eef-147f2b74-28802dfe-d4921aff_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:56.857818Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:56.857836Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:56.857846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343085540233589:12241]
2025-10-09T21:51:56.857897Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:56.857905Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:56.858121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:56.858272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:56.858408Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:56.858549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:56.858739Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:56.858788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:56.862817Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2298 sessionId: src_id|1c8b9eef-147f2b74-28802dfe-d4921aff_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:56.865274Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:56.865494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:56.865513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:56.865550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:56.865657Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:56.870679Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:56.870694Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:56.870697Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:56.870699Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:56.870709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343085540233589:12241]
2025-10-09T21:51:56.870713Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343085540233589:12241]
2025-10-09T21:51:56.870721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343085540233589:12241]
2025-10-09T21:51:56.870778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:56.870779Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:56.870785Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:56.870789Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:56.870791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:56.870794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:56.870929Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:56.925444Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343085902085433:8652]
2025-10-09T21:51:56.925845Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 2297, pipe: [1:7559343085902085433:8652]
2025-10-09T21:51:57.025724Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:57.033155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:57.033182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:57.033364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:57.033432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:57.033504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:57.033572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:57.033810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:57.033825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:57.033898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:57.033967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:57.034052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:57.034118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:57.034167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:57.034175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:57.034235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:57.034296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:57.034358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:57.034414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:57.034454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:57.034462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:57.034533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:57.034794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:57.034852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:57.034922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:57.034972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:57.034980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:57.035060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:57.035146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:57.035203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:57.035268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:57.035327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:57.035335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:57.035398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:57.035473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:57.035539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:57.035605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:57.035651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:57.035660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:57.035727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:57.035786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:57.035844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:57.035902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:57.036040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:57.036048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:57.036110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:57.036166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:57.036222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:57.036284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:57.036338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:57.036345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:57.036404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:57.036461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:57.036524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:57.036592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:57.036638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:57.036649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:57.036710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:57.036766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:57.036827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:57.036895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:57.036937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:57.036945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:57.037012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343085902085433:8652], now have 1 active actors on pipe
2025-10-09T21:51:57.037221Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:57.037242Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343085902085433:8652]
2025-10-09T21:51:57.037273Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:57.037287Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:57.037328Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2298 reason: error at write: Disconnected sessionId: src_id|1c8b9eef-147f2b74-28802dfe-d4921aff_0
2025-10-09T21:51:57.037544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2298 sessionId: src_id|1c8b9eef-147f2b74-28802dfe-d4921aff_0 is DEAD
2025-10-09T21:51:57.037544Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:57.037683Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c8b9eef-147f2b74-28802dfe-d4921aff_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:57.037694Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c8b9eef-147f2b74-28802dfe-d4921aff_0] Write session will restart in 0.006331s
2025-10-09T21:51:57.037721Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c8b9eef-147f2b74-28802dfe-d4921aff_0] Write session: Do CDS request
2025-10-09T21:51:57.037727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c8b9eef-147f2b74-28802dfe-d4921aff_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:57.045500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c8b9eef-147f2b74-28802dfe-d4921aff_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:57.045607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:57.045625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2299
2025-10-09T21:51:57.045777Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2299 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:57.045812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2299 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:57.045821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2299 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:57.045827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:57.046312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2299 sessionId:  describe result for acl check
2025-10-09T21:51:57.046356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:57.046359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:57.046361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:57.046367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343090197052790:7553] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:57.046962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343090197052790:7553] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:57.048957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:57.049513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343090197052790:7553] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:57.050919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:57.051156Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:57.051167Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:57.051180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:57.051246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:57.051290Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343089835201021:12243]
2025-10-09T21:51:57.051907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:57.052242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:57.052295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:57.052650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343090197052790:7553] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:57.052668Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343090197052790:7553] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:57.052672Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343090197052790:7553] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:57.052683Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2299 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:57.052901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:57.052978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.052985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.052987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.052989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.052990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.052991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.052992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.052994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.052995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.053000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.053017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.053021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.053025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.053029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.053033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.053037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.053040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.053047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.053054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.053061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.053067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.053072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.053078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.053085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.053089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.053093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.053098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.053102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.053110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.053116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.053121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.053127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.053131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.053136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.053141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.053146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.053178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:57.060302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:57.061490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:57.061504Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:57.061513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:57.061519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:57.061530Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2298 [2:7559343089835201021:12243]
2025-10-09T21:51:57.061546Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:57.061572Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:57.061579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:57.061583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:57.061585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:57.061603Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:57.061616Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:57.061624Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:57.061683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.061689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.061692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.061695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:57.061756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:57.061761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.061770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.061776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.061786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:57.061806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343089835201015:12243]
2025-10-09T21:51:57.061810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343089835201015:12243]
2025-10-09T21:51:57.061812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343089835201015:12243]
2025-10-09T21:51:57.061822Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:57.061832Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:57.061842Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:57.061845Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:57.061870Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:57.061892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:57.061900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:57.061913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343090197052862:7553], now have 1 active actors on pipe
2025-10-09T21:51:57.061921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343090197052733:2432], now have 1 active actors on pipe
2025-10-09T21:51:57.061958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343089835201015:12243]
2025-10-09T21:51:57.061971Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:57.061948Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2298
2025-10-09T21:51:57.061953Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2298
2025-10-09T21:51:57.061993Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:57.062056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:57.062087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:57.062155Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c0eba07c-102c99e1-5b2a644-70b799b5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:57.062170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:57.062174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:57.062177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:57.062190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:57.062202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:57.062218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:57.062367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:57.062375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:57.062391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:57.062473Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2299 partition: 0 MaxSeqNo: 25 sessionId: src_id|c0eba07c-102c99e1-5b2a644-70b799b5_0
2025-10-09T21:51:57.062874Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c8b9eef-147f2b74-28802dfe-d4921aff_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c0eba07c-102c99e1-5b2a644-70b799b5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:57.065483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c0eba07c-102c99e1-5b2a644-70b799b5_0] Write session: try to update token
2025-10-09T21:51:57.065500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c0eba07c-102c99e1-5b2a644-70b799b5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:57.077721Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:57.077740Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:57.077743Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:57.077745Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:57.077756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343089835201015:12243]
2025-10-09T21:51:57.077761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343089835201015:12243]
2025-10-09T21:51:57.077764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343089835201015:12243]
2025-10-09T21:51:57.077802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:57.077809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.077810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:57.077827Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:57.077834Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:57.077838Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:57.077960Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:57.079371Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2299 sessionId: src_id|c0eba07c-102c99e1-5b2a644-70b799b5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:57.081901Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:57.137427Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343090197052871:8652]
2025-10-09T21:51:57.137711Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 2298, pipe: [1:7559343090197052871:8652]
2025-10-09T21:51:57.240642Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:57.241100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:57.241117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:57.241163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343090197052871:8652], now have 1 active actors on pipe
2025-10-09T21:51:57.241445Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:57.241458Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343090197052871:8652]
2025-10-09T21:51:57.241467Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:57.241493Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:57.241591Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2299 reason: error at write: Disconnected sessionId: src_id|c0eba07c-102c99e1-5b2a644-70b799b5_0
2025-10-09T21:51:57.241787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2299 sessionId: src_id|c0eba07c-102c99e1-5b2a644-70b799b5_0 is DEAD
2025-10-09T21:51:57.241952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0eba07c-102c99e1-5b2a644-70b799b5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:57.241964Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0eba07c-102c99e1-5b2a644-70b799b5_0] Write session will restart in 0.007495s
2025-10-09T21:51:57.241990Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0eba07c-102c99e1-5b2a644-70b799b5_0] Write session: Do CDS request
2025-10-09T21:51:57.241999Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0eba07c-102c99e1-5b2a644-70b799b5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:57.241698Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:57.250593Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c0eba07c-102c99e1-5b2a644-70b799b5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:57.250660Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:57.250679Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2300
2025-10-09T21:51:57.250861Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2300 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:57.250917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2300 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:57.250923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2300 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:57.250927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:57.251427Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2300 sessionId:  describe result for acl check
2025-10-09T21:51:57.251505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:57.251512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:57.251516Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:57.251523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343090197052955:7541] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:57.252116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343090197052955:7541] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:57.253924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:57.254214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343090197052955:7541] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:57.255990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:57.256246Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:57.256258Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:57.256272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:57.256331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:57.256378Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343089835201151:12245]
2025-10-09T21:51:57.256997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:57.257371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:57.257429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:57.257767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:57.257820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.257821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.257822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.257824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.257825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.257827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.257828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.257829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.257831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.257837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.257848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.257853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.257858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.257862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.257866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.257871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.257875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.257880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.257888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.257897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.257904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.257910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.257918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.257924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.257934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.257940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.257945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.257951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.257955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.257961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.257968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.257974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.257978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.257984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.257989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.257995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.258027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:57.259008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343090197052955:7541] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:57.259026Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343090197052955:7541] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:57.259030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343090197052955:7541] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:57.259040Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2300 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:57.264740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:57.265861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:57.265875Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:57.265879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:57.265884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:57.265892Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2299 [2:7559343089835201151:12245]
2025-10-09T21:51:57.265906Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:57.265926Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:57.265935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:57.265939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:57.265942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:57.265962Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:57.265972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:57.265979Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:57.266080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.266089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.266091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.266094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:57.266321Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2299
2025-10-09T21:51:57.266359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:57.266368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.266382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.266355Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2299
2025-10-09T21:51:57.266394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.266420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:57.266448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343090197052877:2432], now have 1 active actors on pipe
2025-10-09T21:51:57.266452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343090197053009:7541], now have 1 active actors on pipe
2025-10-09T21:51:57.266468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:57.266473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:57.266497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343089835201146:12245]
2025-10-09T21:51:57.266504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343089835201146:12245]
2025-10-09T21:51:57.266506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343089835201146:12245]
2025-10-09T21:51:57.266520Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:57.266533Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:57.266538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:57.266552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:57.266555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:57.266558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:57.266560Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:57.266578Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|93bac993-24c07ca0-ff417096-62525c74_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:57.266586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:57.266594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:57.266597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:57.266609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:57.266625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:57.266672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343089835201146:12245]
2025-10-09T21:51:57.266687Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:57.266700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:57.266706Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:57.266878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:57.266891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:57.266917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:57.267011Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2300 partition: 0 MaxSeqNo: 25 sessionId: src_id|93bac993-24c07ca0-ff417096-62525c74_0
2025-10-09T21:51:57.267425Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c0eba07c-102c99e1-5b2a644-70b799b5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|93bac993-24c07ca0-ff417096-62525c74_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:57.269958Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|93bac993-24c07ca0-ff417096-62525c74_0] Write session: try to update token
2025-10-09T21:51:57.269978Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|93bac993-24c07ca0-ff417096-62525c74_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:57.281225Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:57.281245Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:57.281247Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:57.281249Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:57.281263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343089835201146:12245]
2025-10-09T21:51:57.281268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343089835201146:12245]
2025-10-09T21:51:57.281271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343089835201146:12245]
2025-10-09T21:51:57.281327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:57.281327Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:57.281333Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:57.281334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.281336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:57.281337Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:57.281465Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:57.283961Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2300 sessionId: src_id|93bac993-24c07ca0-ff417096-62525c74_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:57.286409Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:57.341650Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343090197053022:8652]
2025-10-09T21:51:57.341926Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 2299, pipe: [1:7559343090197053022:8652]
2025-10-09T21:51:57.447789Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:57.448086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:57.448101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:57.448147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343090197053022:8652], now have 1 active actors on pipe
2025-10-09T21:51:57.448379Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:57.448415Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:57.448428Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343090197053022:8652]
2025-10-09T21:51:57.448455Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:57.448545Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2300 reason: status is not ok: tablet will be restarted right now sessionId: src_id|93bac993-24c07ca0-ff417096-62525c74_0
2025-10-09T21:51:57.448733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2300 sessionId: src_id|93bac993-24c07ca0-ff417096-62525c74_0 is DEAD
2025-10-09T21:51:57.448888Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93bac993-24c07ca0-ff417096-62525c74_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:57.448898Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93bac993-24c07ca0-ff417096-62525c74_0] Write session will restart in 0.008418s
2025-10-09T21:51:57.448926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93bac993-24c07ca0-ff417096-62525c74_0] Write session: Do CDS request
2025-10-09T21:51:57.448933Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93bac993-24c07ca0-ff417096-62525c74_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:57.458515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|93bac993-24c07ca0-ff417096-62525c74_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:57.458615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:57.458635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2301
2025-10-09T21:51:57.458843Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2301 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:57.458877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2301 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:57.458890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2301 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:57.458893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:57.459362Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2301 sessionId:  describe result for acl check
2025-10-09T21:51:57.459404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:57.459406Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:57.459407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:57.459414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343090197053117:7520] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:57.459752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:57.460018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343090197053117:7520] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:57.461817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:57.462011Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:57.462024Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:57.462048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:57.462101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:57.462112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343090197053117:7520] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:57.462154Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343089835201282:12247]
2025-10-09T21:51:57.462763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:57.463124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:57.463191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:57.463569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:57.463629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.463636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.463637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.463638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.463639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.463641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.463642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.463643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.463644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.463650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.463661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.463665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.463669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.463673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.463677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.463682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.463686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.463690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.463697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.463705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.463712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.463717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.463730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.463735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.463739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.463744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.463748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.463752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.463756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.463760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.463766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.463771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.463776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.463781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.463786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.463792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.463819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:57.464752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343090197053117:7520] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:57.464767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343090197053117:7520] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:57.464771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343090197053117:7520] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:57.464779Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2301 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:57.470191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:57.471207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:57.471221Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:57.471223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:57.471227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:57.471234Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2300 [2:7559343089835201282:12247]
2025-10-09T21:51:57.471248Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:57.471266Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:57.471269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:57.471272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:57.471275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:57.471291Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:57.471300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:57.471311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:57.471367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.471374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.471376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.471379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:57.471445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:57.471452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.471464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.471483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.471501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:57.471521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343090197053024:2432], now have 1 active actors on pipe
2025-10-09T21:51:57.471526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343090197053157:7520], now have 1 active actors on pipe
2025-10-09T21:51:57.471534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343089835201276:12247]
2025-10-09T21:51:57.471541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343089835201276:12247]
2025-10-09T21:51:57.471543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343089835201276:12247]
2025-10-09T21:51:57.471554Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:57.471568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:57.471569Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:57.471573Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:57.471575Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:57.471588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:57.471591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:57.471620Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343089835201276:12247]
2025-10-09T21:51:57.471605Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2300
2025-10-09T21:51:57.471629Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:57.471633Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:57.471608Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2300
2025-10-09T21:51:57.471665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:57.471675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:57.471709Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|29eb6f92-6088fcb7-dfb9b208-4bf2f932_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:57.471716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:57.471725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:57.471728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:57.471739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:57.471751Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:57.471766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:57.471903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:57.471911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:57.471924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:57.471994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2301 partition: 0 MaxSeqNo: 25 sessionId: src_id|29eb6f92-6088fcb7-dfb9b208-4bf2f932_0
2025-10-09T21:51:57.472353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|93bac993-24c07ca0-ff417096-62525c74_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|29eb6f92-6088fcb7-dfb9b208-4bf2f932_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:57.474763Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29eb6f92-6088fcb7-dfb9b208-4bf2f932_0] Write session: try to update token
2025-10-09T21:51:57.474781Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29eb6f92-6088fcb7-dfb9b208-4bf2f932_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:57.485631Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:57.485648Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:57.485650Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:57.485652Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:57.485661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343089835201276:12247]
2025-10-09T21:51:57.485667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343089835201276:12247]
2025-10-09T21:51:57.485670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343089835201276:12247]
2025-10-09T21:51:57.485711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:57.485721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.485725Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:57.485728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:57.485731Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:57.485736Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:57.485854Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:57.487969Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2301 sessionId: src_id|29eb6f92-6088fcb7-dfb9b208-4bf2f932_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:57.490379Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:57.548561Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343090197053165:8652]
2025-10-09T21:51:57.548856Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 2300, pipe: [1:7559343090197053165:8652]
2025-10-09T21:51:57.639903Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:57.640193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:57.640209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:57.640255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343090197053165:8652], now have 1 active actors on pipe
2025-10-09T21:51:57.640502Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:57.640547Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343090197053165:8652]
2025-10-09T21:51:57.640552Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:57.640554Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:57.640568Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:51:57.640671Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2301 reason: error at write: Disconnected sessionId: src_id|29eb6f92-6088fcb7-dfb9b208-4bf2f932_0
2025-10-09T21:51:57.640760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2301 sessionId: src_id|29eb6f92-6088fcb7-dfb9b208-4bf2f932_0 is DEAD
2025-10-09T21:51:57.640826Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 23 consumer shared/user session shared/user_1_23_18068588973188758792_v1 is DEAD
2025-10-09T21:51:57.640932Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:51:57.640971Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559343008592655251:8656] disconnected.
2025-10-09T21:51:57.640983Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559343008592655251:8656] disconnected; active server actors: 1
2025-10-09T21:51:57.640986Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559343008592655251:8656] client user disconnected session shared/user_1_23_18068588973188758792_v1
2025-10-09T21:51:57.641023Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:57.641097Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:51:57.641103Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:51:57.641110Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.008054s
2025-10-09T21:51:57.641186Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29eb6f92-6088fcb7-dfb9b208-4bf2f932_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:57.641202Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29eb6f92-6088fcb7-dfb9b208-4bf2f932_0] Write session will restart in 0.006469s
2025-10-09T21:51:57.641227Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29eb6f92-6088fcb7-dfb9b208-4bf2f932_0] Write session: Do CDS request
2025-10-09T21:51:57.641233Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29eb6f92-6088fcb7-dfb9b208-4bf2f932_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:57.648488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|29eb6f92-6088fcb7-dfb9b208-4bf2f932_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:57.648543Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:57.648559Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2302
2025-10-09T21:51:57.648666Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2302 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:57.648701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2302 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:57.648710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2302 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:57.648714Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:57.649319Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2302 sessionId:  describe result for acl check
2025-10-09T21:51:57.649357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:57.649364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:57.649368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:57.649373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343090197053218:7512] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:57.649911Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343090197053218:7512] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:57.650310Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:51:57.650420Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:51:57.650444Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 24
2025-10-09T21:51:57.650585Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 24 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:51:57.650649Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:51:57.650754Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 auth for : user
2025-10-09T21:51:57.650842Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 Handle describe topics response
2025-10-09T21:51:57.650869Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 auth is DEAD
2025-10-09T21:51:57.650883Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:51:57.651127Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:51:57.651195Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_24_13326786123235704269_v1
2025-10-09T21:51:57.651189Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559343090197053282:7487] connected; active server actors: 1
2025-10-09T21:51:57.651206Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559343090197053282:7487] session shared/user_1_24_13326786123235704269_v1
2025-10-09T21:51:57.651216Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:51:57.651227Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:51:57.651234Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_24_13326786123235704269_v1" (Sender=[1:7559343090197053267:7487], Pipe=[1:7559343090197053282:7487], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:51:57.651237Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:51:57.651247Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:51:57.651262Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_24_13326786123235704269_v1" (Sender=[1:7559343090197053267:7487], Pipe=[1:7559343090197053282:7487], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:51:57.651272Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_24_13326786123235704269_v1" sender [1:7559343090197053267:7487] lock partition 0 for ReadingSession "shared/user_1_24_13326786123235704269_v1" (Sender=[1:7559343090197053267:7487], Pipe=[1:7559343090197053282:7487], Partitions=[], ActiveFamilyCount=1) generation 1 step 24
2025-10-09T21:51:57.651284Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:51:57.651289Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000034s
2025-10-09T21:51:57.651480Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 24 Session: "shared/user_1_24_13326786123235704269_v1" ClientId: "user" PipeClient { RawX1: 7559343090197053282 RawX2: 4503603922345279 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:51:57.651515Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:51:57.651689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:57.651794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343090197053218:7512] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:57.653464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:57.653671Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:57.653681Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:57.653694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:57.653752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:57.653792Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343089835201416:12249]
2025-10-09T21:51:57.654079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343090197053218:7512] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:57.654090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343090197053218:7512] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:57.654093Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343090197053218:7512] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:57.654101Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2302 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:57.654362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:57.654655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:57.654709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:57.655058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:57.655110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.655117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.655118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.655119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.655120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.655122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.655123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.655124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.655125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.655131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.655141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.655145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.655149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.655153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.655157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.655161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.655165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.655168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.655175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.655183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.655189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.655194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.655199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.655204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.655208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.655213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.655217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.655222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.655226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.655232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.655237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.655254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.655258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.655264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.655268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.655272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.655301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:57.661588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:57.662677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:57.662692Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:57.662695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:57.662700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:57.662706Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2301 [2:7559343089835201416:12249]
2025-10-09T21:51:57.662719Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:57.662742Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:57.662750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:57.662753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:57.662755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:57.662772Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:57.662782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:57.662791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:57.662849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.662857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.662859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.662862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:57.662927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:57.662933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.662945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.662960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.662978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:57.663007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343090197053304:7512], now have 1 active actors on pipe
2025-10-09T21:51:57.663015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343090197053167:2432], now have 1 active actors on pipe
2025-10-09T21:51:57.663017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343090197053286:7496], now have 1 active actors on pipe
2025-10-09T21:51:57.663026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343089835201409:12249]
2025-10-09T21:51:57.663032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343089835201409:12249]
2025-10-09T21:51:57.663034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343089835201409:12249]
2025-10-09T21:51:57.663046Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:57.663059Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:57.663064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:57.663066Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:57.663047Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2301
2025-10-09T21:51:57.663068Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:57.663058Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2301
2025-10-09T21:51:57.663077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:57.663080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:57.663091Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 2301, pipe: [1:7559343090197053286:7496]
2025-10-09T21:51:57.663128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:57.663137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:57.663150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343089835201409:12249]
2025-10-09T21:51:57.663160Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:57.663169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:57.663171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:57.663173Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:57.663177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_24_13326786123235704269_v1 on pipe: [1:7559343090197053286:7496]
2025-10-09T21:51:57.663223Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|89c20908-eeaf45f6-6c2cb89a-43820499_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:57.663231Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_24_13326786123235704269_v1:1 with generation 2301
2025-10-09T21:51:57.663235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:57.663238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:57.663241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:57.663252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:57.663264Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:57.663287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:57.663295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:57.663297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:57.663311Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_24_13326786123235704269_v1
2025-10-09T21:51:57.663318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:57.663343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:57.663368Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:51:57.663463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:57.663471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:57.663483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:57.663571Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2302 partition: 0 MaxSeqNo: 25 sessionId: src_id|89c20908-eeaf45f6-6c2cb89a-43820499_0
2025-10-09T21:51:57.663917Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|29eb6f92-6088fcb7-dfb9b208-4bf2f932_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|89c20908-eeaf45f6-6c2cb89a-43820499_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:57.666331Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|89c20908-eeaf45f6-6c2cb89a-43820499_0] Write session: try to update token
2025-10-09T21:51:57.666350Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|89c20908-eeaf45f6-6c2cb89a-43820499_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:57.667656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:51:57.667687Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:51:57.667698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:57.667700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:57.667703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:57.667711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:51:57.667729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:51:57.667863Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046717663 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:51:57.667887Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:51:57.667921Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 sending to client partition status
2025-10-09T21:51:57.677141Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:57.677159Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:57.677162Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:57.677164Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:57.677173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343089835201409:12249]
2025-10-09T21:51:57.677178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343089835201409:12249]
2025-10-09T21:51:57.677182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343089835201409:12249]
2025-10-09T21:51:57.677223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:57.677229Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:57.677235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.677236Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:57.677238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:57.677240Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:57.677393Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:57.679513Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2302 sessionId: src_id|89c20908-eeaf45f6-6c2cb89a-43820499_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:57.681854Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:57.842837Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:57.843126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:57.843141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:57.843449Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:57.843451Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:57.843466Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:57.843500Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343090197053286:7496]
2025-10-09T21:51:57.843541Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2302 reason: error at write: Disconnected sessionId: src_id|89c20908-eeaf45f6-6c2cb89a-43820499_0
2025-10-09T21:51:57.843722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2302 sessionId: src_id|89c20908-eeaf45f6-6c2cb89a-43820499_0 is DEAD
2025-10-09T21:51:57.843750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_24_13326786123235704269_v1
2025-10-09T21:51:57.843774Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:57.843790Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_24_13326786123235704269_v1
2025-10-09T21:51:57.843850Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89c20908-eeaf45f6-6c2cb89a-43820499_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:57.843873Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89c20908-eeaf45f6-6c2cb89a-43820499_0] Write session will restart in 0.007998s
2025-10-09T21:51:57.843899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89c20908-eeaf45f6-6c2cb89a-43820499_0] Write session: Do CDS request
2025-10-09T21:51:57.843905Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89c20908-eeaf45f6-6c2cb89a-43820499_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:57.853611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|89c20908-eeaf45f6-6c2cb89a-43820499_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:57.853654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:57.853673Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2303
2025-10-09T21:51:57.853863Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2303 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:57.853927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2303 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:57.853933Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2303 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:57.853938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:57.854419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2303 sessionId:  describe result for acl check
2025-10-09T21:51:57.854462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:57.854469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:57.854471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:57.854477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343090197053437:7480] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:57.854871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:57.855080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343090197053437:7480] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:57.856732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:57.856958Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:57.856968Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:57.856981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:57.857035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:57.857077Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343089835201593:12265]
2025-10-09T21:51:57.857355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343090197053437:7480] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:57.857691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:57.857950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:57.857988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:57.858307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:57.858473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.858480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.858482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.858483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.858485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.858486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.858487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.858488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.858490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.858494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.858505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.858509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.858514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.858518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.858522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.858526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.858530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.858534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.858540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.858549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:57.858555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.858561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:57.858571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.858576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:57.858585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.858590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:57.858599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.858604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:57.858614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.858618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:57.858623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.858628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.858632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.858637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.858642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.858647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.858676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:57.864114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343090197053437:7480] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:57.864137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343090197053437:7480] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:57.864141Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343090197053437:7480] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:57.864148Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2303 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:57.865106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:57.866228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:57.866242Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:57.866246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:57.866250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:57.866257Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2302 [2:7559343089835201593:12265]
2025-10-09T21:51:57.866270Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:57.866287Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:57.866290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:57.866294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:57.866297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:57.866316Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:57.866324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:57.866331Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:57.866388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.866398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.866400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.866402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:57.866471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:57.866477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.866489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:57.866505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:57.866520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:57.866537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343090197053343:2432], now have 1 active actors on pipe
2025-10-09T21:51:57.866541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343090197053475:7480], now have 1 active actors on pipe
2025-10-09T21:51:57.866547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343089835201587:12265]
2025-10-09T21:51:57.866555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343089835201587:12265]
2025-10-09T21:51:57.866557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343089835201587:12265]
2025-10-09T21:51:57.866569Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:57.866583Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:57.866587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:57.866592Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:57.866596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:57.866600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:57.866603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:57.866634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343089835201587:12265]
2025-10-09T21:51:57.866611Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2302
2025-10-09T21:51:57.866640Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:57.866643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:57.866620Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2302
2025-10-09T21:51:57.866660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:57.866668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:57.866697Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7f8f427d-84de5deb-c7a5baad-8564010d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:57.866706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:57.866710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:57.866712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:57.866720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:57.866728Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:57.866741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:57.866862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:57.866868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:57.866881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:57.866976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2303 partition: 0 MaxSeqNo: 25 sessionId: src_id|7f8f427d-84de5deb-c7a5baad-8564010d_0
2025-10-09T21:51:57.867371Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89c20908-eeaf45f6-6c2cb89a-43820499_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7f8f427d-84de5deb-c7a5baad-8564010d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:57.869885Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7f8f427d-84de5deb-c7a5baad-8564010d_0] Write session: try to update token
2025-10-09T21:51:57.869905Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7f8f427d-84de5deb-c7a5baad-8564010d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:57.881058Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:57.881076Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:57.881078Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:57.881080Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:57.881091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343089835201587:12265]
2025-10-09T21:51:57.881096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343089835201587:12265]
2025-10-09T21:51:57.881103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343089835201587:12265]
2025-10-09T21:51:57.881145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:57.881152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:57.881155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:57.881158Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:57.881165Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:57.881169Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:57.881294Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:57.883507Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2303 sessionId: src_id|7f8f427d-84de5deb-c7a5baad-8564010d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:57.885985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:57.943814Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343090197053484:7496]
2025-10-09T21:51:57.944127Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 2302, pipe: [1:7559343090197053484:7496]
2025-10-09T21:51:58.030756Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:58.031054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:58.031065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:58.031103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343090197053484:7496], now have 1 active actors on pipe
2025-10-09T21:51:58.031337Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:58.031339Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:58.031345Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:58.031348Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:58.031410Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343090197053484:7496]
2025-10-09T21:51:58.031413Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2303 reason: error at write: Disconnected sessionId: src_id|7f8f427d-84de5deb-c7a5baad-8564010d_0
2025-10-09T21:51:58.031593Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2303 sessionId: src_id|7f8f427d-84de5deb-c7a5baad-8564010d_0 is DEAD
2025-10-09T21:51:58.031729Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f8f427d-84de5deb-c7a5baad-8564010d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:58.031739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f8f427d-84de5deb-c7a5baad-8564010d_0] Write session will restart in 0.006816s
2025-10-09T21:51:58.031760Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f8f427d-84de5deb-c7a5baad-8564010d_0] Write session: Do CDS request
2025-10-09T21:51:58.031765Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f8f427d-84de5deb-c7a5baad-8564010d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:58.039605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7f8f427d-84de5deb-c7a5baad-8564010d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:58.039650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:58.039670Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2304
2025-10-09T21:51:58.039892Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2304 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:58.039940Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2304 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:58.039953Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2304 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:58.039957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:58.040512Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2304 sessionId:  describe result for acl check
2025-10-09T21:51:58.040558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:58.040561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:58.040563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:58.040569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343094492020856:7461] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:58.041166Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343094492020856:7461] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:58.042967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:58.043592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343094492020856:7461] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:58.045019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:58.045242Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:58.045254Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:58.045267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:58.045336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:58.045382Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343094130169023:12267]
2025-10-09T21:51:58.045995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:58.046374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:58.046438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:58.046517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343094492020856:7461] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:58.046532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343094492020856:7461] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:58.046536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343094492020856:7461] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:58.046548Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2304 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:58.046792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:58.046866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.046874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.046875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.046876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.046878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.046879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.046880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.046881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.046882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.046888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.046899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.046903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.046907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.046910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.046914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.046918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.046922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.046925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.046933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.046941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.046953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.046959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.046964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.046970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.046974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.046979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.046984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.046990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.046994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.047000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.047005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.047011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.047016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.047021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.047026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.047031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.047058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:58.053819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:58.055208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:58.055222Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:58.055224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:58.055228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:58.055236Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2303 [2:7559343094130169023:12267]
2025-10-09T21:51:58.055247Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:58.055267Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:58.055275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:58.055279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:58.055281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:58.055303Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:58.055312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:58.055319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:58.055372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.055378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.055379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.055382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:58.055437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:58.055444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.055451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.055458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.055468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:58.055486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343094130169017:12267]
2025-10-09T21:51:58.055489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343094130169017:12267]
2025-10-09T21:51:58.055491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343094130169017:12267]
2025-10-09T21:51:58.055499Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:58.055515Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:58.055516Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:58.055525Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:58.055527Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:58.055531Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:58.055534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:58.055543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343094492020916:7461], now have 1 active actors on pipe
2025-10-09T21:51:58.055546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343094492020785:2432], now have 1 active actors on pipe
2025-10-09T21:51:58.055575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343094130169017:12267]
2025-10-09T21:51:58.055585Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:58.055594Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:58.055595Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2303
2025-10-09T21:51:58.055595Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2303
2025-10-09T21:51:58.055657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:58.055668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:58.055692Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|af2fbe50-9e8270dd-caf1c9d1-15ada3e3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:58.055698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:58.055706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:58.055708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:58.055716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:58.055724Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:58.055744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:58.055904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:58.055909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:58.055919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:58.056014Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2304 partition: 0 MaxSeqNo: 25 sessionId: src_id|af2fbe50-9e8270dd-caf1c9d1-15ada3e3_0
2025-10-09T21:51:58.056411Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f8f427d-84de5deb-c7a5baad-8564010d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|af2fbe50-9e8270dd-caf1c9d1-15ada3e3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:58.058888Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af2fbe50-9e8270dd-caf1c9d1-15ada3e3_0] Write session: try to update token
2025-10-09T21:51:58.058904Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af2fbe50-9e8270dd-caf1c9d1-15ada3e3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:58.067826Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:58.067848Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:58.067859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343094130169017:12267]
2025-10-09T21:51:58.067900Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:58.067913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:58.068142Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:58.068311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:58.068467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:58.068613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:58.068832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:58.068878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:58.072575Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2304 sessionId: src_id|af2fbe50-9e8270dd-caf1c9d1-15ada3e3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:58.074987Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:58.075196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:58.075219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:58.075261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:58.075354Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:58.080839Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:58.080856Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:58.080859Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:58.080861Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:58.080872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343094130169017:12267]
2025-10-09T21:51:58.080876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343094130169017:12267]
2025-10-09T21:51:58.080880Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343094130169017:12267]
2025-10-09T21:51:58.080916Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:58.080921Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:58.080923Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:58.080924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:58.080932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.080935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:58.081081Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:58.131722Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343094492020926:7496]
2025-10-09T21:51:58.132024Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 2303, pipe: [1:7559343094492020926:7496]
2025-10-09T21:51:58.232690Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:58.240146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:58.240171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:58.240327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:58.240385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:58.240438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:58.240498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:58.240541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:58.240552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:58.240615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:58.240679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:58.240748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:58.240806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:58.240846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:58.240855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:58.240911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:58.240961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:58.241017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:58.241081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:58.241127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:58.241136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:58.241192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:58.241244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:58.241322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:58.241383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:58.241430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:58.241437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:58.241510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:58.241568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:58.241620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:58.241681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:58.241727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:58.241732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:58.241789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:58.241842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:58.241899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:58.241968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:58.242047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:58.242054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:58.242114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:58.242172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:58.242225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:58.242279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:58.242320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:58.242326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:58.242378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:58.242431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:58.242489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:58.242574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:58.242623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:58.242630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:58.242688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:58.242741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:58.242803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:58.242873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:58.242923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:58.242931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:58.242990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:58.243043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:58.243095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:58.243148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:58.243185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:58.243191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:58.243242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343094492020926:7496], now have 1 active actors on pipe
2025-10-09T21:51:58.243597Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343094492020926:7496]
2025-10-09T21:51:58.243601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:58.243606Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:58.243637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2304 reason: error at write: Disconnected sessionId: src_id|af2fbe50-9e8270dd-caf1c9d1-15ada3e3_0
2025-10-09T21:51:58.243818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2304 sessionId: src_id|af2fbe50-9e8270dd-caf1c9d1-15ada3e3_0 is DEAD
2025-10-09T21:51:58.243858Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:58.243958Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af2fbe50-9e8270dd-caf1c9d1-15ada3e3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:58.243990Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af2fbe50-9e8270dd-caf1c9d1-15ada3e3_0] Write session will restart in 0.006878s
2025-10-09T21:51:58.244016Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af2fbe50-9e8270dd-caf1c9d1-15ada3e3_0] Write session: Do CDS request
2025-10-09T21:51:58.244022Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af2fbe50-9e8270dd-caf1c9d1-15ada3e3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:58.244148Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:58.252522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af2fbe50-9e8270dd-caf1c9d1-15ada3e3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:58.252638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:58.252657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2305
2025-10-09T21:51:58.252757Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2305 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:58.252805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2305 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:58.252815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2305 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:58.252818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:58.253263Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2305 sessionId:  describe result for acl check
2025-10-09T21:51:58.253302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:58.253321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:58.253323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:58.253329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343094492021026:7457] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:58.253910Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343094492021026:7457] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:58.254564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:58.256512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:58.256588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343094492021026:7457] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:58.256719Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:58.256729Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:58.256741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:58.256795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:58.256834Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343094130169158:12269]
2025-10-09T21:51:58.257478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:58.257807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:58.257872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:58.258250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:58.258294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.258300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.258300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.258301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.258302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.258302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.258303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.258304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.258304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.258308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.258315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.258317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.258319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.258321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.258323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.258325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.258327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.258329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.258334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.258339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.258343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.258347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.258349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.258352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.258356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.258359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.258362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.258365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.258367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.258371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.258373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.258376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.258379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.258382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.258385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.258388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.258405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:58.263215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343094492021026:7457] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:58.263232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343094492021026:7457] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:58.263236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343094492021026:7457] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:58.263244Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2305 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:58.264538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:58.265700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:58.265715Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:58.265717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:58.265721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:58.265728Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2304 [2:7559343094130169158:12269]
2025-10-09T21:51:58.265743Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:58.265759Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:58.265767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:58.265770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:58.265772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:58.265789Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:58.265799Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:58.265807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:58.265860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.265869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.265870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.265873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:58.265943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:58.265946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.265957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.265973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.265994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:58.266014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343094492020931:2432], now have 1 active actors on pipe
2025-10-09T21:51:58.266021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343094492021067:7457], now have 1 active actors on pipe
2025-10-09T21:51:58.266047Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343094130169152:12269]
2025-10-09T21:51:58.266050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343094130169152:12269]
2025-10-09T21:51:58.266053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343094130169152:12269]
2025-10-09T21:51:58.266066Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:58.266088Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:58.266090Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:58.266085Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2304
2025-10-09T21:51:58.266094Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:58.266096Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:58.266112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:58.266115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:58.266150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343094130169152:12269]
2025-10-09T21:51:58.266132Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2304
2025-10-09T21:51:58.266165Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:58.266166Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:58.266196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:58.266206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:58.266238Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|25afae5-cef2c93-fff43043-30ab38fe_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:58.266250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:58.266259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:58.266262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:58.266274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:58.266289Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:58.266308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:58.266464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:58.266474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:58.266490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:58.266588Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2305 partition: 0 MaxSeqNo: 25 sessionId: src_id|25afae5-cef2c93-fff43043-30ab38fe_0
2025-10-09T21:51:58.266937Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af2fbe50-9e8270dd-caf1c9d1-15ada3e3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|25afae5-cef2c93-fff43043-30ab38fe_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:58.269401Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25afae5-cef2c93-fff43043-30ab38fe_0] Write session: try to update token
2025-10-09T21:51:58.269418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25afae5-cef2c93-fff43043-30ab38fe_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:58.280909Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:58.280926Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:58.280929Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:58.280931Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:58.280943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343094130169152:12269]
2025-10-09T21:51:58.280948Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343094130169152:12269]
2025-10-09T21:51:58.280952Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343094130169152:12269]
2025-10-09T21:51:58.280997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:58.281003Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:58.281005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.281008Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:58.281008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:58.281013Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:58.281147Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:58.282080Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2305 sessionId: src_id|25afae5-cef2c93-fff43043-30ab38fe_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:58.284373Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:58.343673Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343094492021081:7496]
2025-10-09T21:51:58.343915Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 2304, pipe: [1:7559343094492021081:7496]
2025-10-09T21:51:58.445857Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:58.446170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:58.446188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:58.446234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343094492021081:7496], now have 1 active actors on pipe
2025-10-09T21:51:58.446467Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343094492021081:7496]
2025-10-09T21:51:58.446484Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:58.446513Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:58.446516Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:58.446653Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2305 reason: error at write: Disconnected sessionId: src_id|25afae5-cef2c93-fff43043-30ab38fe_0
2025-10-09T21:51:58.446714Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:58.446822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2305 sessionId: src_id|25afae5-cef2c93-fff43043-30ab38fe_0 is DEAD
2025-10-09T21:51:58.446998Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25afae5-cef2c93-fff43043-30ab38fe_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:58.447007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25afae5-cef2c93-fff43043-30ab38fe_0] Write session will restart in 0.007721s
2025-10-09T21:51:58.447035Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25afae5-cef2c93-fff43043-30ab38fe_0] Write session: Do CDS request
2025-10-09T21:51:58.447041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25afae5-cef2c93-fff43043-30ab38fe_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:58.455565Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25afae5-cef2c93-fff43043-30ab38fe_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:58.455624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:58.455655Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2306
2025-10-09T21:51:58.455772Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2306 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:58.455805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2306 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:58.455815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2306 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:58.455819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:58.456261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2306 sessionId:  describe result for acl check
2025-10-09T21:51:58.456302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:58.456310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:58.456312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:58.456318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343094492021180:7445] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:58.456882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343094492021180:7445] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:58.457083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:58.458941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:58.458983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343094492021180:7445] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:58.459135Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:58.459144Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:58.459156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:58.459209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:58.459247Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343094130169294:12271]
2025-10-09T21:51:58.459803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:58.460217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:58.460264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:58.460564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:58.460626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.460634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.460635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.460636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.460637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.460638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.460639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.460640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.460641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.460646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.460656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.460659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.460663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.460666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.460670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.460674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.460677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.460681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.460687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.460695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.460701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.460705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.460711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.460716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.460720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.460724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.460729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.460733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.460737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.460741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.460746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.460750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.460754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.460758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.460763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.460768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.460794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:58.461575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343094492021180:7445] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:58.461592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343094492021180:7445] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:58.461595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343094492021180:7445] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:58.461603Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2306 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:58.467043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:58.468348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:58.468362Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:58.468365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:58.468370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:58.468378Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2305 [2:7559343094130169294:12271]
2025-10-09T21:51:58.468391Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:58.468409Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:58.468418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:58.468421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:58.468423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:58.468440Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:58.468452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:58.468463Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:58.468529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.468536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.468538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.468541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:58.468608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:58.468612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.468622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.468633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.468643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:58.468663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343094492021221:7445], now have 1 active actors on pipe
2025-10-09T21:51:58.468673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343094130169288:12271]
2025-10-09T21:51:58.468674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343094130169288:12271]
2025-10-09T21:51:58.468676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343094130169288:12271]
2025-10-09T21:51:58.468685Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:58.468696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343094492021083:2432], now have 1 active actors on pipe
2025-10-09T21:51:58.468702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:58.468706Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:58.468707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:58.468709Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:58.468721Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:58.468729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:58.468761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343094130169288:12271]
2025-10-09T21:51:58.468769Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:58.468780Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:58.468753Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2305
2025-10-09T21:51:58.468777Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2305
2025-10-09T21:51:58.468802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:58.468811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:58.468850Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5daa46a2-6a9073d8-5cf8fb41-9c2b089_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:58.468862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:58.468870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:58.468873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:58.468883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:58.468897Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:58.468914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:58.469110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:58.469123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:58.469146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:58.469279Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2306 partition: 0 MaxSeqNo: 25 sessionId: src_id|5daa46a2-6a9073d8-5cf8fb41-9c2b089_0
2025-10-09T21:51:58.469682Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25afae5-cef2c93-fff43043-30ab38fe_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5daa46a2-6a9073d8-5cf8fb41-9c2b089_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:58.472032Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5daa46a2-6a9073d8-5cf8fb41-9c2b089_0] Write session: try to update token
2025-10-09T21:51:58.472052Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5daa46a2-6a9073d8-5cf8fb41-9c2b089_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:58.483113Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:58.483128Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:58.483132Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:58.483134Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:58.483145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343094130169288:12271]
2025-10-09T21:51:58.483149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343094130169288:12271]
2025-10-09T21:51:58.483154Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343094130169288:12271]
2025-10-09T21:51:58.483198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:58.483201Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:58.483207Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:58.483214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.483215Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:58.483217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:58.483345Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:58.484819Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2306 sessionId: src_id|5daa46a2-6a9073d8-5cf8fb41-9c2b089_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:58.487323Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:58.546441Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343094492021229:7496]
2025-10-09T21:51:58.546709Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 2305, pipe: [1:7559343094492021229:7496]
2025-10-09T21:51:58.639333Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:58.639633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:58.639651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:58.639699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343094492021229:7496], now have 1 active actors on pipe
2025-10-09T21:51:58.639876Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:58.639896Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343094492021229:7496]
2025-10-09T21:51:58.639924Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:58.639926Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:58.639976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2306 reason: error at write: Disconnected sessionId: src_id|5daa46a2-6a9073d8-5cf8fb41-9c2b089_0
2025-10-09T21:51:58.640127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2306 sessionId: src_id|5daa46a2-6a9073d8-5cf8fb41-9c2b089_0 is DEAD
2025-10-09T21:51:58.640284Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5daa46a2-6a9073d8-5cf8fb41-9c2b089_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:58.640297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5daa46a2-6a9073d8-5cf8fb41-9c2b089_0] Write session will restart in 0.009050s
2025-10-09T21:51:58.640325Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5daa46a2-6a9073d8-5cf8fb41-9c2b089_0] Write session: Do CDS request
2025-10-09T21:51:58.640335Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5daa46a2-6a9073d8-5cf8fb41-9c2b089_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:58.641668Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:58.650503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5daa46a2-6a9073d8-5cf8fb41-9c2b089_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:58.650580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:58.650599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2307
2025-10-09T21:51:58.650762Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2307 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:58.650798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2307 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:58.650809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2307 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:58.650812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:58.651263Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2307 sessionId:  describe result for acl check
2025-10-09T21:51:58.651305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:58.651306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:58.651308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:58.651314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343094492021330:7431] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:58.651422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:58.651848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343094492021330:7431] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:58.653084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:58.653257Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:58.653268Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:58.653279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:58.653347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:58.653400Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343094130169432:12273]
2025-10-09T21:51:58.653794Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343094492021330:7431] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:58.654072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:58.654363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:58.654413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:58.654687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:58.654738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.654744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.654746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.654747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.654748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.654749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.654750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.654751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.654752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.654757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.654767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.654771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.654775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.654779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.654783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.654787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.654790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.654794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.654800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.654807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.654813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.654819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.654824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.654829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.654833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.654840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.654845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.654850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.654854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.654859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.654864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.654869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.654873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.654877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.654881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.654886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.654916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:58.660036Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343094492021330:7431] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:58.660053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343094492021330:7431] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:58.660057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343094492021330:7431] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:58.660072Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2307 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:58.661387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:58.662586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:58.662601Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:58.662604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:58.662608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:58.662616Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2306 [2:7559343094130169432:12273]
2025-10-09T21:51:58.662630Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:58.662648Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:58.662651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:58.662654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:58.662657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:58.662673Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:58.662684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:58.662691Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:58.662745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.662754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.662756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.662759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:58.662890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:58.662899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.662912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.662923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.662948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:58.662970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343094492021370:7431], now have 1 active actors on pipe
2025-10-09T21:51:58.662977Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2306
2025-10-09T21:51:58.662977Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2306
2025-10-09T21:51:58.662984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343094492021231:2432], now have 1 active actors on pipe
2025-10-09T21:51:58.662991Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343094130169426:12273]
2025-10-09T21:51:58.662994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343094130169426:12273]
2025-10-09T21:51:58.662996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343094130169426:12273]
2025-10-09T21:51:58.663011Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:58.663025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:58.663037Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:58.663041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:58.663042Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:58.663044Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:58.663045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:58.663088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:58.663099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:58.663112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343094130169426:12273]
2025-10-09T21:51:58.663120Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:58.663134Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:58.663159Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a16e382-c14ac875-ff7de77b-21df9ab6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:58.663172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:58.663176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:58.663179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:58.663190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:58.663204Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:58.663229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:58.663390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:58.663398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:58.663413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:58.663518Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2307 partition: 0 MaxSeqNo: 25 sessionId: src_id|a16e382-c14ac875-ff7de77b-21df9ab6_0
2025-10-09T21:51:58.663864Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5daa46a2-6a9073d8-5cf8fb41-9c2b089_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a16e382-c14ac875-ff7de77b-21df9ab6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:58.666216Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a16e382-c14ac875-ff7de77b-21df9ab6_0] Write session: try to update token
2025-10-09T21:51:58.666233Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a16e382-c14ac875-ff7de77b-21df9ab6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:58.677648Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:58.677667Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:58.677670Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:58.677672Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:58.677681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343094130169426:12273]
2025-10-09T21:51:58.677687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343094130169426:12273]
2025-10-09T21:51:58.677691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343094130169426:12273]
2025-10-09T21:51:58.677734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:58.677735Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:58.677742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.677745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:58.677745Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:58.677749Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:58.677878Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:58.679267Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2307 sessionId: src_id|a16e382-c14ac875-ff7de77b-21df9ab6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:58.681587Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:58.739935Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343094492021378:7496]
2025-10-09T21:51:58.740237Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 2306, pipe: [1:7559343094492021378:7496]
2025-10-09T21:51:58.832457Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:58.832690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:58.832707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:58.832750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343094492021378:7496], now have 1 active actors on pipe
2025-10-09T21:51:58.832905Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:58.832977Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:58.832998Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:58.833060Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2307 reason: error at write: Disconnected sessionId: src_id|a16e382-c14ac875-ff7de77b-21df9ab6_0
2025-10-09T21:51:58.833242Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2307 sessionId: src_id|a16e382-c14ac875-ff7de77b-21df9ab6_0 is DEAD
2025-10-09T21:51:58.833423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a16e382-c14ac875-ff7de77b-21df9ab6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:58.833430Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a16e382-c14ac875-ff7de77b-21df9ab6_0] Write session will restart in 0.006892s
2025-10-09T21:51:58.833454Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a16e382-c14ac875-ff7de77b-21df9ab6_0] Write session: Do CDS request
2025-10-09T21:51:58.833460Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a16e382-c14ac875-ff7de77b-21df9ab6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:58.833433Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:58.833511Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343094492021378:7496]
2025-10-09T21:51:58.841558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a16e382-c14ac875-ff7de77b-21df9ab6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:58.841713Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:58.841736Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2308
2025-10-09T21:51:58.841872Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2308 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:58.841910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2308 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:58.841916Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2308 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:58.841927Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:58.842372Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2308 sessionId:  describe result for acl check
2025-10-09T21:51:58.842411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:58.842417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:58.842421Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:58.842426Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343094492021446:7389] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:58.842977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343094492021446:7389] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:58.845434Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343094492021446:7389] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:58.845842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:58.847714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:58.847929Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:58.847941Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:58.847954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:58.848017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:58.848007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343094492021446:7389] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:58.848021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343094492021446:7389] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:58.848025Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343094492021446:7389] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:58.848033Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2308 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:58.848057Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343094130169586:12283]
2025-10-09T21:51:58.848640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:58.848973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:58.849037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:58.849361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:58.849423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.849430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.849432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.849433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.849434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.849435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.849437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.849438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.849439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.849444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.849455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.849459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.849463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.849467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.849471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.849475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.849479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.849483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.849490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.849497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:58.849504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.849509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:58.849514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.849519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:58.849524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.849530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:58.849534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.849539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:58.849544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.849549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:58.849554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.849559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.849563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.849568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.849573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.849578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.849608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:58.857070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:58.858426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:58.858450Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:58.858452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:58.858457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:58.858465Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2307 [2:7559343094130169586:12283]
2025-10-09T21:51:58.858479Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:58.858498Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:58.858501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:58.858505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:58.858508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:58.858527Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:58.858540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:58.858549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:58.858611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.858624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.858626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.858629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:58.858698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:58.858708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.858721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:58.858733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:58.858748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:58.858770Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343094130169581:12283]
2025-10-09T21:51:58.858772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343094130169581:12283]
2025-10-09T21:51:58.858774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343094130169581:12283]
2025-10-09T21:51:58.858783Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:58.858793Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:58.858796Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:58.858797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:58.858798Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:58.858812Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:58.858816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:58.858829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343094492021393:2432], now have 1 active actors on pipe
2025-10-09T21:51:58.858832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343094492021529:7389], now have 1 active actors on pipe
2025-10-09T21:51:58.858870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343094130169581:12283]
2025-10-09T21:51:58.858887Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:58.858900Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:58.858888Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2307
2025-10-09T21:51:58.858899Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2307
2025-10-09T21:51:58.859082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:58.859100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:58.859130Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|984e5ecb-f5fecb19-22780d1c-e7da8b6b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:58.859146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:58.859150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:58.859152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:58.859162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:58.859174Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:58.859193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:58.859546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:58.859558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:58.859578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:58.859696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2308 partition: 0 MaxSeqNo: 25 sessionId: src_id|984e5ecb-f5fecb19-22780d1c-e7da8b6b_0
2025-10-09T21:51:58.860131Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a16e382-c14ac875-ff7de77b-21df9ab6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|984e5ecb-f5fecb19-22780d1c-e7da8b6b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:58.862731Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|984e5ecb-f5fecb19-22780d1c-e7da8b6b_0] Write session: try to update token
2025-10-09T21:51:58.862752Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|984e5ecb-f5fecb19-22780d1c-e7da8b6b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:58.875028Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:58.875048Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:58.875051Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:58.875053Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:58.875063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343094130169581:12283]
2025-10-09T21:51:58.875068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343094130169581:12283]
2025-10-09T21:51:58.875073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343094130169581:12283]
2025-10-09T21:51:58.875121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:58.875129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:58.875128Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:58.875132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:58.875135Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:58.875138Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:58.875273Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:58.877210Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2308 sessionId: src_id|984e5ecb-f5fecb19-22780d1c-e7da8b6b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:58.879734Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:58.933531Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343094492021552:7496]
2025-10-09T21:51:58.933829Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 2307, pipe: [1:7559343094492021552:7496]
2025-10-09T21:51:59.032485Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:59.032927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.032944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.032990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343094492021552:7496], now have 1 active actors on pipe
2025-10-09T21:51:59.033228Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:59.033217Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:59.033235Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343094492021552:7496]
2025-10-09T21:51:59.033264Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:59.033268Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:59.033772Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|984e5ecb-f5fecb19-22780d1c-e7da8b6b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:59.033784Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|984e5ecb-f5fecb19-22780d1c-e7da8b6b_0] Write session will restart in 0.009917s
2025-10-09T21:51:59.033808Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|984e5ecb-f5fecb19-22780d1c-e7da8b6b_0] Write session: Do CDS request
2025-10-09T21:51:59.033378Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2308 reason: error at write: Disconnected sessionId: src_id|984e5ecb-f5fecb19-22780d1c-e7da8b6b_0
2025-10-09T21:51:59.033817Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|984e5ecb-f5fecb19-22780d1c-e7da8b6b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:59.033584Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2308 sessionId: src_id|984e5ecb-f5fecb19-22780d1c-e7da8b6b_0 is DEAD
2025-10-09T21:51:59.044609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|984e5ecb-f5fecb19-22780d1c-e7da8b6b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:59.044714Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:59.044734Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2309
2025-10-09T21:51:59.044892Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2309 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:59.044939Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2309 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:59.044953Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2309 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:59.044957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:59.045446Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2309 sessionId:  describe result for acl check
2025-10-09T21:51:59.045493Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:59.045501Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:59.045503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:59.045509Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343098786988952:7374] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:59.046181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343098786988952:7374] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:59.046196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:59.048375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:59.048632Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:59.048668Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343098786988952:7374] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:59.048644Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:59.048657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:59.048722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:59.048784Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343098425137047:9421]
2025-10-09T21:51:59.049392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:59.049877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:59.049919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:59.050276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:59.050332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.050337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.050338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.050339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.050339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.050340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.050341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.050341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.050342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.050346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.050353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.050355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.050358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.050360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.050362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.050364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.050366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.050368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.050373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.050378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.050383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.050386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.050389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.050393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.050395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.050398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.050401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.050404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.050406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.050410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.050415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.050418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.050421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.050424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.050427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.050430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.050448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:59.055822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343098786988952:7374] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:59.055847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343098786988952:7374] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:59.055851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343098786988952:7374] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:59.055861Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2309 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:59.057267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:59.058818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:59.058832Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:59.058835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:59.058839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:59.058846Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2308 [2:7559343098425137047:9421]
2025-10-09T21:51:59.058859Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:59.058877Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:59.058880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:59.058884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:59.058886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:59.058904Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:59.058910Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:59.058918Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:59.058975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.058983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.058985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.058988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:59.059061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:59.059069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.059082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.059097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.059119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:59.059145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786988851:2432], now have 1 active actors on pipe
2025-10-09T21:51:59.059153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786988992:7374], now have 1 active actors on pipe
2025-10-09T21:51:59.059161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343098425137042:9421]
2025-10-09T21:51:59.059167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343098425137042:9421]
2025-10-09T21:51:59.059169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343098425137042:9421]
2025-10-09T21:51:59.059179Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:59.059190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:59.059202Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:59.059207Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:59.059210Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:59.059213Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:59.059216Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:59.059237Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343098425137042:9421]
2025-10-09T21:51:59.059245Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:59.059255Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:59.059235Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2308
2025-10-09T21:51:59.059258Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2308
2025-10-09T21:51:59.059281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.059289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.059317Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5c5b8874-1ab41474-b1125752-8030a3c0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:59.059327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:59.059330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:59.059332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:59.059340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:59.059351Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:59.059364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:59.059518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.059531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.059554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:59.059643Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2309 partition: 0 MaxSeqNo: 25 sessionId: src_id|5c5b8874-1ab41474-b1125752-8030a3c0_0
2025-10-09T21:51:59.060014Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|984e5ecb-f5fecb19-22780d1c-e7da8b6b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5c5b8874-1ab41474-b1125752-8030a3c0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:59.066996Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c5b8874-1ab41474-b1125752-8030a3c0_0] Write session: try to update token
2025-10-09T21:51:59.067014Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c5b8874-1ab41474-b1125752-8030a3c0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:59.073995Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:59.074013Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:59.074016Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:59.074018Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:59.074045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343098425137042:9421]
2025-10-09T21:51:59.074050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343098425137042:9421]
2025-10-09T21:51:59.074054Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343098425137042:9421]
2025-10-09T21:51:59.074099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:59.074102Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:59.074106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.074109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:59.074110Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:59.074114Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:59.074247Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:59.083093Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2309 sessionId: src_id|5c5b8874-1ab41474-b1125752-8030a3c0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:59.085625Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:59.133163Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343098786988999:7496]
2025-10-09T21:51:59.133422Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 2308, pipe: [1:7559343098786988999:7496]
2025-10-09T21:51:59.223040Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:59.223280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.223292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.223312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786988999:7496], now have 1 active actors on pipe
2025-10-09T21:51:59.223484Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:59.223550Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343098786988999:7496]
2025-10-09T21:51:59.223558Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:59.223598Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:59.223672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2309 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5c5b8874-1ab41474-b1125752-8030a3c0_0
2025-10-09T21:51:59.223855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2309 sessionId: src_id|5c5b8874-1ab41474-b1125752-8030a3c0_0 is DEAD
2025-10-09T21:51:59.224005Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c5b8874-1ab41474-b1125752-8030a3c0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:51:59.224017Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c5b8874-1ab41474-b1125752-8030a3c0_0] Write session will restart in 0.008980s
2025-10-09T21:51:59.224045Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c5b8874-1ab41474-b1125752-8030a3c0_0] Write session: Do CDS request
2025-10-09T21:51:59.224052Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c5b8874-1ab41474-b1125752-8030a3c0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:59.233245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:59.234471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c5b8874-1ab41474-b1125752-8030a3c0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:59.234596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:59.234613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2310
2025-10-09T21:51:59.234706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:59.234734Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2310 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:59.234774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2310 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:59.234785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2310 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:59.234788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:59.234906Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:59.234916Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:59.234929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:59.234994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:59.235034Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343098425137186:9411]
2025-10-09T21:51:59.235185Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2310 sessionId:  describe result for acl check
2025-10-09T21:51:59.235221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:59.235227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:59.235229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:59.235234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343098786989109:7352] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:59.235504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:59.235717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343098786989109:7352] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:59.235750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:59.235793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:59.236059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:59.236097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.236101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.236102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.236103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.236104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.236104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.236105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.236105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.236106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.236109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.236116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.236118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.236120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.236122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.236126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.236130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.236134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.236138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.236144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.236153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.236158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.236161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.236164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.236167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.236169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.236172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.236175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.236178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.236181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.236184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.236186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.236190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.236192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.236195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.236200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.236203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.236219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:59.237364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343098786989109:7352] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:59.242018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343098786989109:7352] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:59.242046Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343098786989109:7352] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:59.242049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343098786989109:7352] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:59.242056Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2310 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:59.242407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:59.243519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:59.243534Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:59.243537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:59.243541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:59.243548Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2309 [2:7559343098425137186:9411]
2025-10-09T21:51:59.243561Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:59.243578Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:59.243582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:59.243586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:59.243589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:59.243605Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:59.243615Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:59.243623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:59.243681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.243688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.243690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.243692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:59.243767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:59.243774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.243784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.243795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.243814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:59.243836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343098425137181:9411]
2025-10-09T21:51:59.243842Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343098425137181:9411]
2025-10-09T21:51:59.243844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343098425137181:9411]
2025-10-09T21:51:59.243853Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:59.243861Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:59.243863Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:59.243865Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:59.243871Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:59.243887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:59.243893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:59.243906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786989147:7352], now have 1 active actors on pipe
2025-10-09T21:51:59.243913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786989002:2432], now have 1 active actors on pipe
2025-10-09T21:51:59.243896Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2309
2025-10-09T21:51:59.243902Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2309
2025-10-09T21:51:59.243939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343098425137181:9411]
2025-10-09T21:51:59.243950Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:59.243954Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:59.243973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.243981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.244025Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d32a8a8-aaabdd54-645ba985-f9448bd8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:59.244035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:59.244039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:59.244043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:59.244054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:59.244068Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:59.244089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:59.244273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.244283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.244302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:59.244387Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2310 partition: 0 MaxSeqNo: 25 sessionId: src_id|d32a8a8-aaabdd54-645ba985-f9448bd8_0
2025-10-09T21:51:59.244672Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c5b8874-1ab41474-b1125752-8030a3c0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d32a8a8-aaabdd54-645ba985-f9448bd8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:59.247010Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d32a8a8-aaabdd54-645ba985-f9448bd8_0] Write session: try to update token
2025-10-09T21:51:59.247028Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d32a8a8-aaabdd54-645ba985-f9448bd8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:59.259319Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:59.259338Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:59.259340Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:59.259342Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:59.259351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343098425137181:9411]
2025-10-09T21:51:59.259356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343098425137181:9411]
2025-10-09T21:51:59.259360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343098425137181:9411]
2025-10-09T21:51:59.259402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:59.259410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.259408Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:59.259413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:59.259416Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:59.259421Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:59.259547Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:59.260659Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2310 sessionId: src_id|d32a8a8-aaabdd54-645ba985-f9448bd8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:59.263054Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:59.323647Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343098786989159:7496]
2025-10-09T21:51:59.323855Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 2309, pipe: [1:7559343098786989159:7496]
2025-10-09T21:51:59.403048Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:59.403332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.403347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.403392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786989159:7496], now have 1 active actors on pipe
2025-10-09T21:51:59.403580Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:59.403584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:59.403604Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:59.403619Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:59.403678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2310 reason: error at write: Disconnected sessionId: src_id|d32a8a8-aaabdd54-645ba985-f9448bd8_0
2025-10-09T21:51:59.403681Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343098786989159:7496]
2025-10-09T21:51:59.403859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2310 sessionId: src_id|d32a8a8-aaabdd54-645ba985-f9448bd8_0 is DEAD
2025-10-09T21:51:59.404018Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d32a8a8-aaabdd54-645ba985-f9448bd8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:59.404030Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d32a8a8-aaabdd54-645ba985-f9448bd8_0] Write session will restart in 0.006008s
2025-10-09T21:51:59.404054Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d32a8a8-aaabdd54-645ba985-f9448bd8_0] Write session: Do CDS request
2025-10-09T21:51:59.404061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d32a8a8-aaabdd54-645ba985-f9448bd8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:59.411539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d32a8a8-aaabdd54-645ba985-f9448bd8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:59.411638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:59.411665Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2311
2025-10-09T21:51:59.411862Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2311 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:59.411899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2311 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:59.411909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2311 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:59.411915Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:59.412345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2311 sessionId:  describe result for acl check
2025-10-09T21:51:59.412387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:59.412389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:59.412391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:59.412399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343098786989256:7344] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:59.412900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343098786989256:7344] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:59.414027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:59.415245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343098786989256:7344] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:59.415711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:59.415874Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:59.415882Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:59.415890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:59.415936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:59.415978Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343098425137328:9415]
2025-10-09T21:51:59.416398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:59.416642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:59.416684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:59.416948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:59.416993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.416999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.417000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.417001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.417002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.417003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.417005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.417006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.417008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.417012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.417022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.417026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.417030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.417034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.417038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.417041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.417045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.417050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.417056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.417064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.417070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.417076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.417081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.417086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.417091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.417096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.417101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.417106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.417110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.417115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.417120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.417125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.417135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.417140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.417146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.417154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.417185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:59.417477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343098786989256:7344] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:59.417489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343098786989256:7344] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:59.417492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343098786989256:7344] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:59.417497Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2311 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:59.423770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:59.424965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:59.424978Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:59.424980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:59.424984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:59.424990Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2310 [2:7559343098425137328:9415]
2025-10-09T21:51:59.425001Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:59.425016Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:59.425018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:59.425021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:59.425023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:59.425038Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:59.425046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:59.425053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:59.425105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.425113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.425115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.425119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:59.425181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:59.425188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.425200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.425213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.425235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:59.425254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786989304:7344], now have 1 active actors on pipe
2025-10-09T21:51:59.425261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786989162:2432], now have 1 active actors on pipe
2025-10-09T21:51:59.425267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343098425137322:9415]
2025-10-09T21:51:59.425268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343098425137322:9415]
2025-10-09T21:51:59.425270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343098425137322:9415]
2025-10-09T21:51:59.425282Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:59.425296Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:59.425321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:59.425325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:59.425327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:59.425329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:59.425332Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:59.425359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343098425137322:9415]
2025-10-09T21:51:59.425368Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:59.425380Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:59.425360Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2310
2025-10-09T21:51:59.425363Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2310
2025-10-09T21:51:59.425554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.425562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.425588Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c36ae69b-a85dbf87-d6ee7c34-f015989f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:59.425597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:59.425601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:59.425603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:59.425610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:59.425621Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:59.425638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:59.425791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.425797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.425807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:59.425896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2311 partition: 0 MaxSeqNo: 25 sessionId: src_id|c36ae69b-a85dbf87-d6ee7c34-f015989f_0
2025-10-09T21:51:59.426250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d32a8a8-aaabdd54-645ba985-f9448bd8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c36ae69b-a85dbf87-d6ee7c34-f015989f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:59.428469Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c36ae69b-a85dbf87-d6ee7c34-f015989f_0] Write session: try to update token
2025-10-09T21:51:59.428485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c36ae69b-a85dbf87-d6ee7c34-f015989f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:59.439804Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:59.439823Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:59.439825Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:59.439828Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:59.439838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343098425137322:9415]
2025-10-09T21:51:59.439842Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343098425137322:9415]
2025-10-09T21:51:59.439847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343098425137322:9415]
2025-10-09T21:51:59.439890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:59.439897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.439900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:59.439900Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:59.439906Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:59.439910Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:59.440033Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:59.441209Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2311 sessionId: src_id|c36ae69b-a85dbf87-d6ee7c34-f015989f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:59.443910Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:59.503904Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343098786989312:7496]
2025-10-09T21:51:59.504264Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 2310, pipe: [1:7559343098786989312:7496]
2025-10-09T21:51:59.584266Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:59.584537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.584555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.584604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786989312:7496], now have 1 active actors on pipe
2025-10-09T21:51:59.584807Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:59.584828Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:59.584847Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343098786989312:7496]
2025-10-09T21:51:59.584829Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:59.584883Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:59.584892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2311 reason: error at write: Disconnected sessionId: src_id|c36ae69b-a85dbf87-d6ee7c34-f015989f_0
2025-10-09T21:51:59.585090Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2311 sessionId: src_id|c36ae69b-a85dbf87-d6ee7c34-f015989f_0 is DEAD
2025-10-09T21:51:59.585250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c36ae69b-a85dbf87-d6ee7c34-f015989f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:59.585262Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c36ae69b-a85dbf87-d6ee7c34-f015989f_0] Write session will restart in 0.008589s
2025-10-09T21:51:59.585289Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c36ae69b-a85dbf87-d6ee7c34-f015989f_0] Write session: Do CDS request
2025-10-09T21:51:59.585295Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c36ae69b-a85dbf87-d6ee7c34-f015989f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:59.594618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:59.595513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c36ae69b-a85dbf87-d6ee7c34-f015989f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:59.595619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:59.595644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2312
2025-10-09T21:51:59.595764Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2312 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:59.595794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2312 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:59.595803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2312 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:59.595807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:59.596270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2312 sessionId:  describe result for acl check
2025-10-09T21:51:59.596306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:59.596311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:59.596312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:59.596316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343098786989420:7338] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:59.596386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:59.596544Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:59.596555Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:59.596567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:59.596638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:59.596683Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343098425137470:9413]
2025-10-09T21:51:59.596852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343098786989420:7338] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:59.597120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:59.597383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:59.597417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:59.597629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:59.597674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.597679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.597679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.597680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.597681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.597681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.597682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.597683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.597683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.597687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.597693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.597695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.597697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.597699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.597701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.597703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.597705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.597707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.597711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.597716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.597721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.597724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.597728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.597731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.597734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.597737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.597739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.597742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.597745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.597748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.597750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.597753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.597756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.597758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.597761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.597764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.597779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:59.598978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343098786989420:7338] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:59.603606Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343098786989420:7338] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:59.603623Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343098786989420:7338] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:59.603626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343098786989420:7338] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:59.603635Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2312 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:59.604084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:59.605039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:59.605052Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:59.605054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:59.605057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:59.605062Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2311 [2:7559343098425137470:9413]
2025-10-09T21:51:59.605071Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:59.605086Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:59.605089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:59.605092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:59.605095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:59.605112Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:59.605125Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:59.605133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:59.605173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.605179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.605181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.605182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:59.605238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:59.605247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.605260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.605273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.605287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:59.605317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786989472:7338], now have 1 active actors on pipe
2025-10-09T21:51:59.605323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786989315:2432], now have 1 active actors on pipe
2025-10-09T21:51:59.605328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343098425137465:9413]
2025-10-09T21:51:59.605330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343098425137465:9413]
2025-10-09T21:51:59.605331Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343098425137465:9413]
2025-10-09T21:51:59.605339Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:59.605350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:59.605356Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:59.605359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:59.605361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:59.605362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:59.605364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:59.605386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343098425137465:9413]
2025-10-09T21:51:59.605394Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:59.605402Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:59.605461Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2311
2025-10-09T21:51:59.605461Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2311
2025-10-09T21:51:59.605509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.605516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.605537Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c2041d2b-65c5afdd-b4f38d4d-f6e0985b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:59.605546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:59.605549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:59.605550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:59.605557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:59.605565Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:59.605577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:59.605712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.605717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.605727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:59.605826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2312 partition: 0 MaxSeqNo: 25 sessionId: src_id|c2041d2b-65c5afdd-b4f38d4d-f6e0985b_0
2025-10-09T21:51:59.606139Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c36ae69b-a85dbf87-d6ee7c34-f015989f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c2041d2b-65c5afdd-b4f38d4d-f6e0985b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:59.608410Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c2041d2b-65c5afdd-b4f38d4d-f6e0985b_0] Write session: try to update token
2025-10-09T21:51:59.608427Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c2041d2b-65c5afdd-b4f38d4d-f6e0985b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:59.620727Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2312 sessionId: src_id|c2041d2b-65c5afdd-b4f38d4d-f6e0985b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:59.622507Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:51:59.622522Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:59.622530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343098425137465:9413]
2025-10-09T21:51:59.622570Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:51:59.622681Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:51:59.622742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:51:59.622908Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:59.622922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:59.623053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:59.623184Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:51:59.623379Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:51:59.623420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.623430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.623448Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:51:59.623480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:51:59.623573Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:51:59.629934Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:59.629948Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:59.629949Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:59.629951Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:59.629958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343098425137465:9413]
2025-10-09T21:51:59.629960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343098425137465:9413]
2025-10-09T21:51:59.629963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343098425137465:9413]
2025-10-09T21:51:59.629994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:59.630000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.630002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:59.630004Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:59.630009Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:59.630012Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:59.630135Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:59.685108Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343098786989480:7496]
2025-10-09T21:51:59.685383Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 2311, pipe: [1:7559343098786989480:7496]
2025-10-09T21:51:59.774438Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:59.781050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.781072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.781216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:59.781269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:59.781331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:59.781390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:59.781428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:59.781436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:59.781495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:59.781554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:59.781619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:59.781826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:59.781921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:59.781926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:59.781977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:59.782025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:59.782099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:59.782149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:59.782189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:59.782195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:59.782250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:59.782302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:59.782368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:59.782420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:59.782460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:59.782465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:59.782518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:59.782568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:59.782620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:59.782674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:59.782715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:59.782719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:59.782770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:59.782821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:59.782882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:59.782934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:59.782970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:59.782977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:59.783032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:59.783088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:59.783140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:59.783187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:59.783222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:59.783229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:59.783279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:59.783326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:59.783375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:59.783425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:59.783462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:59.783463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:59.783514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:59.783565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:59.783613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:59.783666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:59.783707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:59.783712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:59.783761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:51:59.783812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:51:59.783865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:51:59.783916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:51:59.783954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:51:59.783960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:51:59.783996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786989480:7496], now have 1 active actors on pipe
2025-10-09T21:51:59.784222Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343098786989480:7496]
2025-10-09T21:51:59.784245Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:59.784249Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:59.784250Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:59.784281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2312 reason: error at write: Disconnected sessionId: src_id|c2041d2b-65c5afdd-b4f38d4d-f6e0985b_0
2025-10-09T21:51:59.784456Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2312 sessionId: src_id|c2041d2b-65c5afdd-b4f38d4d-f6e0985b_0 is DEAD
2025-10-09T21:51:59.784554Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:59.784621Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2041d2b-65c5afdd-b4f38d4d-f6e0985b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:59.784633Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2041d2b-65c5afdd-b4f38d4d-f6e0985b_0] Write session will restart in 0.006361s
2025-10-09T21:51:59.784657Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2041d2b-65c5afdd-b4f38d4d-f6e0985b_0] Write session: Do CDS request
2025-10-09T21:51:59.784663Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2041d2b-65c5afdd-b4f38d4d-f6e0985b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:59.792496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c2041d2b-65c5afdd-b4f38d4d-f6e0985b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:59.792575Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:59.792590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2313
2025-10-09T21:51:59.792967Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2313 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:59.793012Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2313 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:59.793017Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2313 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:59.793019Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:59.793448Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2313 sessionId:  describe result for acl check
2025-10-09T21:51:59.793482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:59.793489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:59.793490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:59.793496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343098786989584:7327] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:59.793936Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343098786989584:7327] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:59.794659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:59.796637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:59.796792Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:59.796800Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:59.796810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:59.796848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:59.796878Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343098425137613:9406]
2025-10-09T21:51:59.796941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343098786989584:7327] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:59.797253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:59.797582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:59.797655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:59.797983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:59.798047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.798053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.798055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.798056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.798057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.798058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.798059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.798060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.798061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.798066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.798077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.798081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.798085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.798089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.798093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.798097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.798105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.798109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.798115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.798123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.798133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.798138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.798140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.798143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.798146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.798149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.798155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.798158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.798160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.798163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.798166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.798169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.798172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.798175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.798177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.798181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.798206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:59.803198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343098786989584:7327] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:59.803215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343098786989584:7327] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:59.803219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343098786989584:7327] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:59.803227Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2313 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:59.804506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:59.805626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:59.805639Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:59.805641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:59.805644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:59.805649Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2312 [2:7559343098425137613:9406]
2025-10-09T21:51:59.805661Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:59.805675Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:59.805677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:59.805679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:59.805681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:59.805693Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:59.805697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:59.805702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:59.805745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.805753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.805755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.805758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:59.805826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:59.805836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.805846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.805857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.805879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:59.805902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786989482:2432], now have 1 active actors on pipe
2025-10-09T21:51:59.805905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786989627:7327], now have 1 active actors on pipe
2025-10-09T21:51:59.805913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343098425137607:9406]
2025-10-09T21:51:59.805915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343098425137607:9406]
2025-10-09T21:51:59.805917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343098425137607:9406]
2025-10-09T21:51:59.805930Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:59.805943Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:59.805949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:59.805952Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:59.805954Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:59.805964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:59.805967Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:59.805955Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2312
2025-10-09T21:51:59.805992Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2312
2025-10-09T21:51:59.805996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343098425137607:9406]
2025-10-09T21:51:59.806006Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:59.806007Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:59.806051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.806056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.806083Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e9b4efc6-c4c393e3-24088ab5-65694552_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:59.806093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:59.806096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:59.806098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:59.806105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:59.806114Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:59.806133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:59.806292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.806303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.806322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:59.806395Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2313 partition: 0 MaxSeqNo: 25 sessionId: src_id|e9b4efc6-c4c393e3-24088ab5-65694552_0
2025-10-09T21:51:59.806709Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c2041d2b-65c5afdd-b4f38d4d-f6e0985b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e9b4efc6-c4c393e3-24088ab5-65694552_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:59.808989Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9b4efc6-c4c393e3-24088ab5-65694552_0] Write session: try to update token
2025-10-09T21:51:59.809006Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9b4efc6-c4c393e3-24088ab5-65694552_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:51:59.820163Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:51:59.820178Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:51:59.820179Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:51:59.820181Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:51:59.820188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343098425137607:9406]
2025-10-09T21:51:59.820193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343098425137607:9406]
2025-10-09T21:51:59.820196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343098425137607:9406]
2025-10-09T21:51:59.820230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:51:59.820235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.820237Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:51:59.820239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:51:59.820244Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:51:59.820247Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:51:59.820353Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:51:59.821687Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2313 sessionId: src_id|e9b4efc6-c4c393e3-24088ab5-65694552_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:51:59.823883Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:51:59.884246Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343098786989649:7496]
2025-10-09T21:51:59.884548Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 2312, pipe: [1:7559343098786989649:7496]
2025-10-09T21:51:59.972819Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:51:59.973061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.973073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.973111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786989649:7496], now have 1 active actors on pipe
2025-10-09T21:51:59.973322Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:51:59.973333Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:51:59.973344Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:51:59.973348Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:51:59.973386Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343098786989649:7496]
2025-10-09T21:51:59.973412Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2313 reason: error at write: Disconnected sessionId: src_id|e9b4efc6-c4c393e3-24088ab5-65694552_0
2025-10-09T21:51:59.973577Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2313 sessionId: src_id|e9b4efc6-c4c393e3-24088ab5-65694552_0 is DEAD
2025-10-09T21:51:59.973727Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9b4efc6-c4c393e3-24088ab5-65694552_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:51:59.973735Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9b4efc6-c4c393e3-24088ab5-65694552_0] Write session will restart in 0.008926s
2025-10-09T21:51:59.973755Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9b4efc6-c4c393e3-24088ab5-65694552_0] Write session: Do CDS request
2025-10-09T21:51:59.973760Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9b4efc6-c4c393e3-24088ab5-65694552_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:51:59.983509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9b4efc6-c4c393e3-24088ab5-65694552_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:59.983621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:51:59.983639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2314
2025-10-09T21:51:59.983756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2314 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:51:59.983788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2314 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:51:59.983794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2314 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:51:59.983797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:51:59.983779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:51:59.984263Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2314 sessionId:  describe result for acl check
2025-10-09T21:51:59.984302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:51:59.984309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:51:59.984311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:51:59.984317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343098786989773:7300] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:51:59.984859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343098786989773:7300] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:51:59.985551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:51:59.985722Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:51:59.985734Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:51:59.985746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:51:59.985799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:51:59.985830Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343098425137800:9390]
2025-10-09T21:51:59.986423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:51:59.986699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:51:59.986746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:51:59.986958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343098786989773:7300] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:51:59.987066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:51:59.987118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.987124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.987125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.987126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.987128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.987129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.987130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.987131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.987132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.987137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.987148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.987152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.987156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.987160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.987164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.987168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.987171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.987175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.987181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.987189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:51:59.987195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.987200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:51:59.987205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.987210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:51:59.987214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.987219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:51:59.987223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.987228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:51:59.987235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.987240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:51:59.987245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.987250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.987254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.987259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.987263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.987273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.987302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:51:59.992300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343098786989773:7300] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:51:59.992317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343098786989773:7300] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:51:59.992321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343098786989773:7300] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:51:59.992330Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2314 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:51:59.993536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:51:59.994527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:51:59.994541Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:51:59.994544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:51:59.994546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:51:59.994551Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2313 [2:7559343098425137800:9390]
2025-10-09T21:51:59.994560Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:51:59.994573Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:51:59.994575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:59.994577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:59.994579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:59.994592Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:51:59.994596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:51:59.994601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:51:59.994654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.994664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.994666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.994668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:51:59.994734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:51:59.994741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:51:59.994749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:51:59.994756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:51:59.994765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:51:59.994785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786989809:7300], now have 1 active actors on pipe
2025-10-09T21:51:59.994791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343098786989666:2432], now have 1 active actors on pipe
2025-10-09T21:51:59.994797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343098425137794:9390]
2025-10-09T21:51:59.994798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343098425137794:9390]
2025-10-09T21:51:59.994799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343098425137794:9390]
2025-10-09T21:51:59.994808Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:51:59.994819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:51:59.994823Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:51:59.994825Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:51:59.994827Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:51:59.994828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:51:59.994830Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:51:59.994819Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2313
2025-10-09T21:51:59.994853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343098425137794:9390]
2025-10-09T21:51:59.994858Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:51:59.994861Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:51:59.994822Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2313
2025-10-09T21:51:59.994877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.994886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.994931Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8292bdc0-347a22d2-6ce03da2-d6d42ed5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:51:59.994942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:51:59.994946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:51:59.994954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:51:59.994963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:51:59.994976Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:51:59.994991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:59.995108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:51:59.995113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:51:59.995124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:51:59.995213Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2314 partition: 0 MaxSeqNo: 25 sessionId: src_id|8292bdc0-347a22d2-6ce03da2-d6d42ed5_0
2025-10-09T21:51:59.995516Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9b4efc6-c4c393e3-24088ab5-65694552_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8292bdc0-347a22d2-6ce03da2-d6d42ed5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:51:59.997904Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8292bdc0-347a22d2-6ce03da2-d6d42ed5_0] Write session: try to update token
2025-10-09T21:51:59.997919Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8292bdc0-347a22d2-6ce03da2-d6d42ed5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:00.009343Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:00.009360Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:00.009362Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:00.009364Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:00.009375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343098425137794:9390]
2025-10-09T21:52:00.009380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343098425137794:9390]
2025-10-09T21:52:00.009384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343098425137794:9390]
2025-10-09T21:52:00.009432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:00.009439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.009442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:00.009444Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:00.009450Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:00.009453Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:00.009582Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:00.011416Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2314 sessionId: src_id|8292bdc0-347a22d2-6ce03da2-d6d42ed5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:00.013927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:00.073736Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343103081957113:7496]
2025-10-09T21:52:00.073973Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 2313, pipe: [1:7559343103081957113:7496]
2025-10-09T21:52:00.160423Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:00.160905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.160922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.160963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343103081957113:7496], now have 1 active actors on pipe
2025-10-09T21:52:00.161177Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:00.161202Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:00.161220Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:00.161226Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:00.161277Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343103081957113:7496]
2025-10-09T21:52:00.161297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2314 reason: error at write: Disconnected sessionId: src_id|8292bdc0-347a22d2-6ce03da2-d6d42ed5_0
2025-10-09T21:52:00.161481Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2314 sessionId: src_id|8292bdc0-347a22d2-6ce03da2-d6d42ed5_0 is DEAD
2025-10-09T21:52:00.161633Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8292bdc0-347a22d2-6ce03da2-d6d42ed5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:00.161644Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8292bdc0-347a22d2-6ce03da2-d6d42ed5_0] Write session will restart in 0.009974s
2025-10-09T21:52:00.161665Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8292bdc0-347a22d2-6ce03da2-d6d42ed5_0] Write session: Do CDS request
2025-10-09T21:52:00.161670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8292bdc0-347a22d2-6ce03da2-d6d42ed5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:00.172299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:00.172561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8292bdc0-347a22d2-6ce03da2-d6d42ed5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:00.172576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:00.172597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2315
2025-10-09T21:52:00.172780Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2315 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:00.172812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2315 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:00.172821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2315 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:00.172826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:00.173236Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2315 sessionId:  describe result for acl check
2025-10-09T21:52:00.173276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:00.173283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:00.173284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:00.173292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343103081957224:7278] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:00.173910Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343103081957224:7278] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:00.174417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:00.174587Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:00.174597Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:00.174607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:00.174648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:00.174682Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343102720105240:9387]
2025-10-09T21:52:00.175119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:00.175344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:00.175374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:00.175590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:00.175629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.175634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.175635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.175635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.175636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.175637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.175637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.175638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.175639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.175642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.175649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.175651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.175654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.175656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.175658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.175660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.175662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.175664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.175668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.175674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.175678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.175682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.175684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.175687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.175690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.175693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.175695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.175698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.175701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.175704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.175707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.175710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.175713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.175716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.175719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.175722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.175740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:00.175903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343103081957224:7278] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:00.181286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343103081957224:7278] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:00.181315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343103081957224:7278] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:00.181321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343103081957224:7278] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:00.181332Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2315 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:00.185666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:00.188720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:00.188748Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:00.188752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:00.188764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:00.188774Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2314 [2:7559343102720105240:9387]
2025-10-09T21:52:00.188814Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:00.188844Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:00.188855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:00.188859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:00.188865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:00.188884Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:00.188896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:00.188903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:00.189016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.189030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.189033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.189035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:00.189160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:00.189169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.189187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.189206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.189250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:00.189276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343103081957115:2432], now have 1 active actors on pipe
2025-10-09T21:52:00.189290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343103081957262:7278], now have 1 active actors on pipe
2025-10-09T21:52:00.189318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343102720105234:9387]
2025-10-09T21:52:00.189324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343102720105234:9387]
2025-10-09T21:52:00.189327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343102720105234:9387]
2025-10-09T21:52:00.189276Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2314
2025-10-09T21:52:00.189333Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2314
2025-10-09T21:52:00.189353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:00.189363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:00.189366Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:00.189372Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:00.189393Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:00.189435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:00.189446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:00.189480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.189502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.189588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343102720105234:9387]
2025-10-09T21:52:00.189619Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:00.189627Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:00.189644Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ae5a7d8a-17e41683-2c98ac12-5eb1074c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:00.189666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:00.189677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:00.189680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:00.189707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:00.189723Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:00.189802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:00.190014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.190039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.190080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:00.190202Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2315 partition: 0 MaxSeqNo: 25 sessionId: src_id|ae5a7d8a-17e41683-2c98ac12-5eb1074c_0
2025-10-09T21:52:00.190660Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8292bdc0-347a22d2-6ce03da2-d6d42ed5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ae5a7d8a-17e41683-2c98ac12-5eb1074c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:00.196308Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae5a7d8a-17e41683-2c98ac12-5eb1074c_0] Write session: try to update token
2025-10-09T21:52:00.196337Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae5a7d8a-17e41683-2c98ac12-5eb1074c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:00.205503Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:00.205540Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:00.205542Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:00.205544Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:00.205552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343102720105234:9387]
2025-10-09T21:52:00.205557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343102720105234:9387]
2025-10-09T21:52:00.205562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343102720105234:9387]
2025-10-09T21:52:00.205605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:00.205614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.205618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:00.205625Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:00.205635Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:00.205639Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:00.205800Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:00.211222Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2315 sessionId: src_id|ae5a7d8a-17e41683-2c98ac12-5eb1074c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:00.213861Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:00.261658Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343103081957273:7496]
2025-10-09T21:52:00.261895Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 2314, pipe: [1:7559343103081957273:7496]
2025-10-09T21:52:00.362089Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:00.362749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.362767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.362818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343103081957273:7496], now have 1 active actors on pipe
2025-10-09T21:52:00.363095Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:00.363092Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:00.363097Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343103081957273:7496]
2025-10-09T21:52:00.363110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:00.363123Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:00.363187Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2315 reason: error at write: Disconnected sessionId: src_id|ae5a7d8a-17e41683-2c98ac12-5eb1074c_0
2025-10-09T21:52:00.363370Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2315 sessionId: src_id|ae5a7d8a-17e41683-2c98ac12-5eb1074c_0 is DEAD
2025-10-09T21:52:00.363509Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae5a7d8a-17e41683-2c98ac12-5eb1074c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:00.363521Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae5a7d8a-17e41683-2c98ac12-5eb1074c_0] Write session will restart in 0.006093s
2025-10-09T21:52:00.363552Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae5a7d8a-17e41683-2c98ac12-5eb1074c_0] Write session: Do CDS request
2025-10-09T21:52:00.363559Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae5a7d8a-17e41683-2c98ac12-5eb1074c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:00.370505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ae5a7d8a-17e41683-2c98ac12-5eb1074c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:00.370650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:00.370684Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2316
2025-10-09T21:52:00.371333Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2316 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:00.371380Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2316 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:00.371397Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2316 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:00.371403Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:00.371853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2316 sessionId:  describe result for acl check
2025-10-09T21:52:00.371896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:00.371899Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:00.371901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:00.371906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343103081957360:7248] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:00.372413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343103081957360:7248] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:00.373924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:00.374828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343103081957360:7248] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:00.375834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:00.376025Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:00.376038Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:00.376050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:00.376099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:00.376130Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343102720105387:9380]
2025-10-09T21:52:00.376584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:00.376879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343103081957360:7248] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:00.376884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:00.376893Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343103081957360:7248] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:00.376896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343103081957360:7248] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:00.376920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:00.376904Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2316 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:00.377225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:00.377273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.377279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.377281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.377282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.377283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.377284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.377286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.377287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.377288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.377293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.377314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.377319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.377323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.377327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.377331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.377335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.377339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.377343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.377349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.377357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.377368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.377378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.377387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.377397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.377406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.377411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.377416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.377421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.377425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.377435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.377445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.377454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.377463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.377472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.377481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.377486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.377520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:00.384103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:00.385326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:00.385341Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:00.385344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:00.385348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:00.385355Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2315 [2:7559343102720105387:9380]
2025-10-09T21:52:00.385369Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:00.385386Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:00.385390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:00.385393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:00.385396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:00.385414Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:00.385425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:00.385431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:00.385479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.385487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.385489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.385492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:00.385557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:00.385563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.385570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.385577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.385591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:00.385608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343103081957426:7248], now have 1 active actors on pipe
2025-10-09T21:52:00.385616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343103081957281:2432], now have 1 active actors on pipe
2025-10-09T21:52:00.385621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343102720105381:9380]
2025-10-09T21:52:00.385623Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343102720105381:9380]
2025-10-09T21:52:00.385625Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343102720105381:9380]
2025-10-09T21:52:00.385635Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:00.385644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:00.385651Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:00.385653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:00.385655Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:00.385655Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:00.385657Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:00.385677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343102720105381:9380]
2025-10-09T21:52:00.385687Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:00.385679Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2315
2025-10-09T21:52:00.385702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:00.385690Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2315
2025-10-09T21:52:00.385711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.385715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.385744Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|457898af-faad9cae-d4a3cc65-bff08977_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:00.385759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:00.385769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:00.385772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:00.385780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:00.385792Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:00.385808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:00.385936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.385944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.385954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:00.386054Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2316 partition: 0 MaxSeqNo: 25 sessionId: src_id|457898af-faad9cae-d4a3cc65-bff08977_0
2025-10-09T21:52:00.386383Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ae5a7d8a-17e41683-2c98ac12-5eb1074c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|457898af-faad9cae-d4a3cc65-bff08977_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:00.388832Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|457898af-faad9cae-d4a3cc65-bff08977_0] Write session: try to update token
2025-10-09T21:52:00.388847Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|457898af-faad9cae-d4a3cc65-bff08977_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:00.402181Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2316 sessionId: src_id|457898af-faad9cae-d4a3cc65-bff08977_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:00.403475Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:00.403501Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:00.403511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343102720105381:9380]
2025-10-09T21:52:00.403564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:00.403562Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:00.403752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:00.403962Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:00.404114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:00.404263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:00.404467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:00.404516Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:00.404713Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:00.404907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.404925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.404960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:00.405048Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:00.411470Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:00.411496Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:00.411499Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:00.411501Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:00.411513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343102720105381:9380]
2025-10-09T21:52:00.411517Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343102720105381:9380]
2025-10-09T21:52:00.411521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343102720105381:9380]
2025-10-09T21:52:00.411569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:00.411573Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:00.411577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.411579Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:00.411580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:00.411583Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:00.411761Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:00.463382Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343103081957436:7496]
2025-10-09T21:52:00.463696Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 2315, pipe: [1:7559343103081957436:7496]
2025-10-09T21:52:00.553452Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:00.559880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.559902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.560069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.560122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.560177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.560234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.560502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.560511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.560571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.560630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.560686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.560746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.560782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.560788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.560840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.560898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.560951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.561005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.561048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.561054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.561115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.561177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.561231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.561287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.561336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.561342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.561403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.561455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.561510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.561563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.561600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.561605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.561668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.561718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.561770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.561822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.561859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.561865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.561924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.561972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.562042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.562097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.562135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.562142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.562197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.562246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.562307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.562360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.562401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.562410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.562468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.562515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.562564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.562616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.562663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.562670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.562722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.562770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.562820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.562871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.562987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.562993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.563032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343103081957436:7496], now have 1 active actors on pipe
2025-10-09T21:52:00.563344Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343103081957436:7496]
2025-10-09T21:52:00.563360Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:00.563365Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:00.563377Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:00.563404Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2316 reason: error at write: Disconnected sessionId: src_id|457898af-faad9cae-d4a3cc65-bff08977_0
2025-10-09T21:52:00.563513Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:00.563597Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2316 sessionId: src_id|457898af-faad9cae-d4a3cc65-bff08977_0 is DEAD
2025-10-09T21:52:00.563724Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|457898af-faad9cae-d4a3cc65-bff08977_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:00.563735Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|457898af-faad9cae-d4a3cc65-bff08977_0] Write session will restart in 0.006339s
2025-10-09T21:52:00.563770Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|457898af-faad9cae-d4a3cc65-bff08977_0] Write session: Do CDS request
2025-10-09T21:52:00.563776Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|457898af-faad9cae-d4a3cc65-bff08977_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:00.571518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|457898af-faad9cae-d4a3cc65-bff08977_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:00.571637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:00.571655Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2317
2025-10-09T21:52:00.571784Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2317 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:00.572262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2317 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:00.572274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2317 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:00.572279Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:00.572664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2317 sessionId:  describe result for acl check
2025-10-09T21:52:00.572709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:00.572710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:00.572711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:00.572716Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343103081957521:7245] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:00.573215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343103081957521:7245] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:00.574698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:00.575505Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343103081957521:7245] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:00.576650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:00.576841Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:00.576852Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:00.576864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:00.576918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:00.576963Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343102720105534:9376]
2025-10-09T21:52:00.577556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:00.577844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:00.577884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:00.578120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343103081957521:7245] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:00.578132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343103081957521:7245] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:00.578135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343103081957521:7245] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:00.578146Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2317 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:00.578197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:00.578245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.578254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.578255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.578256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.578257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.578258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.578259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.578260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.578262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.578267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.578277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.578281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.578285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.578289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.578293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.578297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.578301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.578304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.578310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.578319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.578324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.578329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.578333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.578339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.578343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.578348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.578353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.578358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.578363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.578368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.578373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.578383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.578387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.578393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.578397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.578402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.578435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:00.584602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:00.585853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:00.585867Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:00.585870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:00.585874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:00.585880Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2316 [2:7559343102720105534:9376]
2025-10-09T21:52:00.585894Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:00.585912Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:00.585916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:00.585919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:00.585921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:00.585939Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:00.585954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:00.585961Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:00.586015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.586021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.586023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.586026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:00.586111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:00.586120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.586132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.586143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.586160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:00.586181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343103081957439:2432], now have 1 active actors on pipe
2025-10-09T21:52:00.586194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343103081957586:7245], now have 1 active actors on pipe
2025-10-09T21:52:00.586201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343102720105528:9376]
2025-10-09T21:52:00.586204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343102720105528:9376]
2025-10-09T21:52:00.586206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343102720105528:9376]
2025-10-09T21:52:00.586219Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:00.586232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:00.586234Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:00.586237Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:00.586238Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:00.586250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:00.586253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:00.586281Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343102720105528:9376]
2025-10-09T21:52:00.586252Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2316
2025-10-09T21:52:00.586289Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:00.586290Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:00.586297Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2316
2025-10-09T21:52:00.586336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.586346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.586373Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|809169b7-928558e-57ee5b73-a9cbd0c9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:00.586385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:00.586394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:00.586397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:00.586410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:00.586424Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:00.586443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:00.586598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.586608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.586624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:00.586723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2317 partition: 0 MaxSeqNo: 25 sessionId: src_id|809169b7-928558e-57ee5b73-a9cbd0c9_0
2025-10-09T21:52:00.587071Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|457898af-faad9cae-d4a3cc65-bff08977_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|809169b7-928558e-57ee5b73-a9cbd0c9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:00.589546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|809169b7-928558e-57ee5b73-a9cbd0c9_0] Write session: try to update token
2025-10-09T21:52:00.589564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|809169b7-928558e-57ee5b73-a9cbd0c9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:00.597338Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:00.597356Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:00.597367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343102720105528:9376]
2025-10-09T21:52:00.597413Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:00.597423Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:00.597594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:00.597745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:00.597893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:00.598057Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:00.598246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:00.598290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:00.602998Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2317 sessionId: src_id|809169b7-928558e-57ee5b73-a9cbd0c9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:00.605354Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:00.605571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.605593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.605636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:00.605744Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:00.610391Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:00.610408Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:00.610411Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:00.610413Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:00.610424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343102720105528:9376]
2025-10-09T21:52:00.610428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343102720105528:9376]
2025-10-09T21:52:00.610433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343102720105528:9376]
2025-10-09T21:52:00.610477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:00.610490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.610500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:00.610500Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:00.610505Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:00.610508Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:00.610651Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:00.663567Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343103081957595:7496]
2025-10-09T21:52:00.663888Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 2316, pipe: [1:7559343103081957595:7496]
2025-10-09T21:52:00.767064Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:00.775098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.775126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.775291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.775354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.775429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.775497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.775540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.775550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.775615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.775677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.775741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.775808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.775850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.775858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.775918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.775978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.776037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.776097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.776147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.776155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.776219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.776282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.776341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.776403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.776512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.776519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.776582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.776643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.776702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.776760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.776803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.776810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.776870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.776928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.776990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.777048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.777095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.777103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.777159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.777217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.777283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.777356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.777398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.777404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.777466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.777522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.777577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.777641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.777684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.777691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.777748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.777803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.777863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.777919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.777966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.777972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.778045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:00.778101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:00.778158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:00.778218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:00.778258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:00.778264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:00.778327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343103081957595:7496], now have 1 active actors on pipe
2025-10-09T21:52:00.778584Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:00.778595Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343103081957595:7496]
2025-10-09T21:52:00.778626Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:00.778629Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:00.778661Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2317 reason: error at write: Disconnected sessionId: src_id|809169b7-928558e-57ee5b73-a9cbd0c9_0
2025-10-09T21:52:00.778846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2317 sessionId: src_id|809169b7-928558e-57ee5b73-a9cbd0c9_0 is DEAD
2025-10-09T21:52:00.779007Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|809169b7-928558e-57ee5b73-a9cbd0c9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:00.779017Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|809169b7-928558e-57ee5b73-a9cbd0c9_0] Write session will restart in 0.008104s
2025-10-09T21:52:00.779041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|809169b7-928558e-57ee5b73-a9cbd0c9_0] Write session: Do CDS request
2025-10-09T21:52:00.779047Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|809169b7-928558e-57ee5b73-a9cbd0c9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:00.779398Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:00.788562Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|809169b7-928558e-57ee5b73-a9cbd0c9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:00.788663Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:00.788692Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2318
2025-10-09T21:52:00.788800Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2318 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:00.788826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2318 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:00.788833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2318 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:00.788836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:00.789226Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2318 sessionId:  describe result for acl check
2025-10-09T21:52:00.789263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:00.789269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:00.789271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:00.789277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343103081957706:7238] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:00.789809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343103081957706:7238] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:00.790123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:00.791915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:00.792115Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:00.792126Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:00.792138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:00.792200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:00.792239Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343102720105682:9374]
2025-10-09T21:52:00.792381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343103081957706:7238] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:00.792821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:00.793107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:00.793147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:00.793470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:00.793522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.793528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.793529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.793530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.793532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.793533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.793534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.793535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.793536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.793541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.793552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.793556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.793560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.793564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.793568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.793572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.793576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.793580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.793586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.793599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.793609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.793614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.793624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.793629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.793634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.793639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.793643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.793648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.793653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.793658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.793667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.793672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.793681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.793686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.793691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.793701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.793730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:00.798926Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343103081957706:7238] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:00.798943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343103081957706:7238] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:00.798945Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343103081957706:7238] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:00.798957Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2318 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:00.800295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:00.801445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:00.801462Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:00.801465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:00.801469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:00.801479Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2317 [2:7559343102720105682:9374]
2025-10-09T21:52:00.801492Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:00.801510Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:00.801519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:00.801522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:00.801525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:00.801541Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:00.801547Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:00.801554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:00.801616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.801623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.801626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.801628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:00.801693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:00.801701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.801711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.801720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.801741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:00.801766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343103081957748:7238], now have 1 active actors on pipe
2025-10-09T21:52:00.801773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343103081957598:2432], now have 1 active actors on pipe
2025-10-09T21:52:00.801781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343102720105676:9374]
2025-10-09T21:52:00.801783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343102720105676:9374]
2025-10-09T21:52:00.801785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343102720105676:9374]
2025-10-09T21:52:00.801799Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:00.801818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:00.801812Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2317
2025-10-09T21:52:00.801819Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:00.801823Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:00.801825Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:00.801837Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:00.801840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:00.801817Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2317
2025-10-09T21:52:00.801873Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343102720105676:9374]
2025-10-09T21:52:00.801885Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:00.801891Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:00.801902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.801912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.801971Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cf8aa71b-ed2af873-3c5e8df9-d2b5ddb6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:00.801984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:00.801989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:00.801992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:00.802006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:00.802019Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:00.802060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:00.802318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.802328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.802345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:00.802451Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2318 partition: 0 MaxSeqNo: 25 sessionId: src_id|cf8aa71b-ed2af873-3c5e8df9-d2b5ddb6_0
2025-10-09T21:52:00.802797Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|809169b7-928558e-57ee5b73-a9cbd0c9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cf8aa71b-ed2af873-3c5e8df9-d2b5ddb6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:00.805218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf8aa71b-ed2af873-3c5e8df9-d2b5ddb6_0] Write session: try to update token
2025-10-09T21:52:00.805235Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf8aa71b-ed2af873-3c5e8df9-d2b5ddb6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:00.816186Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:00.816210Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:00.816212Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:00.816214Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:00.816224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343102720105676:9374]
2025-10-09T21:52:00.816228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343102720105676:9374]
2025-10-09T21:52:00.816232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343102720105676:9374]
2025-10-09T21:52:00.816276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:00.816281Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:00.816286Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:00.816287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.816289Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:00.816290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:00.816447Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:00.818213Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2318 sessionId: src_id|cf8aa71b-ed2af873-3c5e8df9-d2b5ddb6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:00.820412Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:00.878583Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343103081957765:7496]
2025-10-09T21:52:00.878842Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 2317, pipe: [1:7559343103081957765:7496]
2025-10-09T21:52:00.966960Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:00.967259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.967273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.967317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343103081957765:7496], now have 1 active actors on pipe
2025-10-09T21:52:00.967545Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:00.967547Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343103081957765:7496]
2025-10-09T21:52:00.967561Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:00.967564Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:00.967624Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2318 reason: error at write: Disconnected sessionId: src_id|cf8aa71b-ed2af873-3c5e8df9-d2b5ddb6_0
2025-10-09T21:52:00.967801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2318 sessionId: src_id|cf8aa71b-ed2af873-3c5e8df9-d2b5ddb6_0 is DEAD
2025-10-09T21:52:00.967945Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf8aa71b-ed2af873-3c5e8df9-d2b5ddb6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:00.967955Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf8aa71b-ed2af873-3c5e8df9-d2b5ddb6_0] Write session will restart in 0.007881s
2025-10-09T21:52:00.967973Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf8aa71b-ed2af873-3c5e8df9-d2b5ddb6_0] Write session: Do CDS request
2025-10-09T21:52:00.967980Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf8aa71b-ed2af873-3c5e8df9-d2b5ddb6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:00.968685Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:00.976522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cf8aa71b-ed2af873-3c5e8df9-d2b5ddb6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:00.976568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:00.976590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2319
2025-10-09T21:52:00.976746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2319 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:00.976779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2319 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:00.976789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2319 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:00.976792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:00.977149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2319 sessionId:  describe result for acl check
2025-10-09T21:52:00.977176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:00.977177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:00.977178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:00.977182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343103081957902:7209] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:00.977653Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343103081957902:7209] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:00.977999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:00.979628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343103081957902:7209] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:00.979985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:00.980151Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:00.980163Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:00.980174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:00.980223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:00.980262Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343102720105874:9354]
2025-10-09T21:52:00.980697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:00.980974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:00.981014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:00.981322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:00.981368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.981374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.981375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.981376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.981377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.981378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.981379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.981380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.981381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.981386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.981396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.981399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.981403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.981411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.981415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.981419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.981422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.981426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.981432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.981439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:00.981445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.981450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:00.981458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.981463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:00.981471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.981475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:00.981483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.981485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:00.981488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.981491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:00.981495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.981500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.981504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.981509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.981513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.981518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.981546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:00.982046Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343103081957902:7209] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:00.982061Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343103081957902:7209] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:00.982065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343103081957902:7209] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:00.982078Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2319 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:00.987542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:00.988570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:00.988584Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:00.988587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:00.988592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:00.988598Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2318 [2:7559343102720105874:9354]
2025-10-09T21:52:00.988610Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:00.988628Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:00.988635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:00.988638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:00.988640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:00.988656Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:00.988667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:00.988678Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:00.988732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.988741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.988742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.988745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:00.988810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:00.988819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:00.988829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:00.988843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:00.988864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:00.988888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343103081957793:2432], now have 1 active actors on pipe
2025-10-09T21:52:00.988898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343103081957941:7209], now have 1 active actors on pipe
2025-10-09T21:52:00.988906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343102720105868:9354]
2025-10-09T21:52:00.988908Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343102720105868:9354]
2025-10-09T21:52:00.988911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343102720105868:9354]
2025-10-09T21:52:00.988924Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:00.988938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:00.988940Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:00.988943Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:00.988946Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:00.988957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:00.988941Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2318
2025-10-09T21:52:00.988966Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2318
2025-10-09T21:52:00.988960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:00.988984Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343102720105868:9354]
2025-10-09T21:52:00.988992Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:00.988994Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:00.989019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.989027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.989062Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|17bd2834-f2b934cc-b3f8dc3b-a93cda92_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:00.989075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:00.989079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:00.989082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:00.989093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:00.989106Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:00.989131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:00.989289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:00.989320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:00.989347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:00.989449Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2319 partition: 0 MaxSeqNo: 25 sessionId: src_id|17bd2834-f2b934cc-b3f8dc3b-a93cda92_0
2025-10-09T21:52:00.989749Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cf8aa71b-ed2af873-3c5e8df9-d2b5ddb6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|17bd2834-f2b934cc-b3f8dc3b-a93cda92_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:00.992040Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|17bd2834-f2b934cc-b3f8dc3b-a93cda92_0] Write session: try to update token
2025-10-09T21:52:00.992057Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|17bd2834-f2b934cc-b3f8dc3b-a93cda92_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:01.003549Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:01.003567Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:01.003570Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:01.003572Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:01.003581Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343102720105868:9354]
2025-10-09T21:52:01.003587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343102720105868:9354]
2025-10-09T21:52:01.003591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343102720105868:9354]
2025-10-09T21:52:01.003631Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:01.003637Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:01.003638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:01.003645Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:01.003645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.003648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:01.003787Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:01.005281Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2319 sessionId: src_id|17bd2834-f2b934cc-b3f8dc3b-a93cda92_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:01.007717Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:01.067517Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343107376925245:7496]
2025-10-09T21:52:01.067806Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 2318, pipe: [1:7559343107376925245:7496]
2025-10-09T21:52:01.152864Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:01.153383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.153399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.153441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343107376925245:7496], now have 1 active actors on pipe
2025-10-09T21:52:01.153621Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:01.153664Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:01.153692Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:01.153698Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:01.153735Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343107376925245:7496]
2025-10-09T21:52:01.153763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2319 reason: error at write: Disconnected sessionId: src_id|17bd2834-f2b934cc-b3f8dc3b-a93cda92_0
2025-10-09T21:52:01.153938Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2319 sessionId: src_id|17bd2834-f2b934cc-b3f8dc3b-a93cda92_0 is DEAD
2025-10-09T21:52:01.154096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|17bd2834-f2b934cc-b3f8dc3b-a93cda92_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:01.154109Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|17bd2834-f2b934cc-b3f8dc3b-a93cda92_0] Write session will restart in 0.007812s
2025-10-09T21:52:01.154146Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|17bd2834-f2b934cc-b3f8dc3b-a93cda92_0] Write session: Do CDS request
2025-10-09T21:52:01.154154Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|17bd2834-f2b934cc-b3f8dc3b-a93cda92_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:01.163451Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|17bd2834-f2b934cc-b3f8dc3b-a93cda92_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:01.163599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:01.163617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2320
2025-10-09T21:52:01.163759Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2320 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:01.163796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2320 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:01.163804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2320 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:01.163807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:01.164147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:01.164198Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2320 sessionId:  describe result for acl check
2025-10-09T21:52:01.164241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:01.164247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:01.164249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:01.164255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343107376925361:7195] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:01.164805Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343107376925361:7195] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:01.165891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:01.166102Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:01.166112Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:01.166124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:01.166194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:01.166239Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343107015073320:9346]
2025-10-09T21:52:01.166766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343107376925361:7195] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:01.166826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:01.167095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:01.167139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:01.167468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:01.167511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.167517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.167518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.167520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.167521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.167523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.167524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.167525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.167526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.167531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.167542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.167553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.167557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.167561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.167565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.167569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.167574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.167578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.167584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.167593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.167600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.167605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.167611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.167615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.167620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.167625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.167629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.167634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.167638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.167643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.167648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.167659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.167663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.167668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.167672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.167678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.167707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:01.172558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343107376925361:7195] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:01.172575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343107376925361:7195] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:01.172579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343107376925361:7195] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:01.172588Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2320 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:01.174123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:01.175341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:01.175358Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:01.175361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:01.175366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:01.175374Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2319 [2:7559343107015073320:9346]
2025-10-09T21:52:01.175387Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:01.175410Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:01.175418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:01.175421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:01.175424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:01.175441Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:01.175452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:01.175464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:01.175523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.175530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.175532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.175536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:01.175605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:01.175614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.175626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.175642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.175663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:01.175689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343107376925400:7195], now have 1 active actors on pipe
2025-10-09T21:52:01.175693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343107376925247:2432], now have 1 active actors on pipe
2025-10-09T21:52:01.175702Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343107015073314:9346]
2025-10-09T21:52:01.175709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343107015073314:9346]
2025-10-09T21:52:01.175711Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343107015073314:9346]
2025-10-09T21:52:01.175725Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:01.175711Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2319
2025-10-09T21:52:01.175739Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:01.175726Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2319
2025-10-09T21:52:01.175744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:01.175748Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:01.175750Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:01.175759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:01.175762Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:01.175804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.175814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.175832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343107015073314:9346]
2025-10-09T21:52:01.175843Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:01.175852Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:01.175874Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ee1fe691-602d15b9-edcf6b58-f49ad49_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:01.175886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:01.175895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:01.175898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:01.175909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:01.175923Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:01.175950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:01.176072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.176082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.176095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:01.176160Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2320 partition: 0 MaxSeqNo: 25 sessionId: src_id|ee1fe691-602d15b9-edcf6b58-f49ad49_0
2025-10-09T21:52:01.176473Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|17bd2834-f2b934cc-b3f8dc3b-a93cda92_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ee1fe691-602d15b9-edcf6b58-f49ad49_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:01.178783Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee1fe691-602d15b9-edcf6b58-f49ad49_0] Write session: try to update token
2025-10-09T21:52:01.178799Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee1fe691-602d15b9-edcf6b58-f49ad49_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:01.190134Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:01.190153Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:01.190155Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:01.190156Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:01.190168Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343107015073314:9346]
2025-10-09T21:52:01.190172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343107015073314:9346]
2025-10-09T21:52:01.190175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343107015073314:9346]
2025-10-09T21:52:01.190211Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:01.190216Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:01.190219Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:01.190221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:01.190229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.190230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:01.190360Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:01.192288Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2320 sessionId: src_id|ee1fe691-602d15b9-edcf6b58-f49ad49_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:01.194860Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:01.253872Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343107376925412:7496]
2025-10-09T21:52:01.254159Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 2319, pipe: [1:7559343107376925412:7496]
2025-10-09T21:52:01.350805Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:01.351179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.351193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.351232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343107376925412:7496], now have 1 active actors on pipe
2025-10-09T21:52:01.351400Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:01.351492Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:01.351501Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:01.351509Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:01.351547Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343107376925412:7496]
2025-10-09T21:52:01.351574Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2320 reason: error at write: Disconnected sessionId: src_id|ee1fe691-602d15b9-edcf6b58-f49ad49_0
2025-10-09T21:52:01.351751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2320 sessionId: src_id|ee1fe691-602d15b9-edcf6b58-f49ad49_0 is DEAD
2025-10-09T21:52:01.351884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee1fe691-602d15b9-edcf6b58-f49ad49_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:01.351890Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee1fe691-602d15b9-edcf6b58-f49ad49_0] Write session will restart in 0.009947s
2025-10-09T21:52:01.351920Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee1fe691-602d15b9-edcf6b58-f49ad49_0] Write session: Do CDS request
2025-10-09T21:52:01.351925Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee1fe691-602d15b9-edcf6b58-f49ad49_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:01.362518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ee1fe691-602d15b9-edcf6b58-f49ad49_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:01.362583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:01.362600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2321
2025-10-09T21:52:01.362752Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2321 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:01.362807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2321 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:01.362820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2321 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:01.362823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:01.362848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:01.363246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2321 sessionId:  describe result for acl check
2025-10-09T21:52:01.363282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:01.363288Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:01.363290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:01.363296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343107376925534:7185] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:01.363862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343107376925534:7185] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:01.364634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:01.364812Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:01.364819Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:01.364828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:01.364868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:01.364898Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343107015073472:9349]
2025-10-09T21:52:01.365375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:01.365633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:01.365668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:01.365788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343107376925534:7185] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:01.365896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:01.365933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.365939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.365940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.365941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.365941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.365942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.365943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.365943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.365944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.365948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.365955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.365957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.365959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.365962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.365964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.365966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.365968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.365970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.365975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.365980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.365985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.365990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.365994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.366000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.366003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.366008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.366012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.366016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.366020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.366025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.366043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.366048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.366050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.366053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.366058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.366062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.366080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:01.371041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343107376925534:7185] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:01.371059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343107376925534:7185] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:01.371063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343107376925534:7185] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:01.371073Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2321 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:01.372417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:01.373659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:01.373671Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:01.373673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:01.373675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:01.373681Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2320 [2:7559343107015073472:9349]
2025-10-09T21:52:01.373690Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:01.373703Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:01.373705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:01.373708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:01.373709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:01.373722Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:01.373727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:01.373733Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:01.373772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.373777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.373778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.373780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:01.373836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:01.373842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.373849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.373855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.373864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:01.373878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343107015073466:9349]
2025-10-09T21:52:01.373883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343107015073466:9349]
2025-10-09T21:52:01.373885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343107015073466:9349]
2025-10-09T21:52:01.373891Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:01.373902Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:01.373903Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:01.373905Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:01.373907Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:01.373910Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:01.373912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:01.373921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343107376925573:7185], now have 1 active actors on pipe
2025-10-09T21:52:01.373924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343107376925420:2432], now have 1 active actors on pipe
2025-10-09T21:52:01.373946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343107015073466:9349]
2025-10-09T21:52:01.373957Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:01.373976Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:01.373982Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2320
2025-10-09T21:52:01.373988Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2320
2025-10-09T21:52:01.374070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.374078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.374099Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|28934265-d3a4266e-ad1973e2-e39e8d91_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:01.374109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:01.374112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:01.374114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:01.374122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:01.374133Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:01.374151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:01.374294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.374304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.374321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:01.374394Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2321 partition: 0 MaxSeqNo: 25 sessionId: src_id|28934265-d3a4266e-ad1973e2-e39e8d91_0
2025-10-09T21:52:01.374705Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ee1fe691-602d15b9-edcf6b58-f49ad49_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|28934265-d3a4266e-ad1973e2-e39e8d91_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:01.377235Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28934265-d3a4266e-ad1973e2-e39e8d91_0] Write session: try to update token
2025-10-09T21:52:01.377257Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28934265-d3a4266e-ad1973e2-e39e8d91_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:01.390364Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2321 sessionId: src_id|28934265-d3a4266e-ad1973e2-e39e8d91_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:01.392844Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:01.394449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.394470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.394491Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:01.394493Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:01.394499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343107015073466:9349]
2025-10-09T21:52:01.394538Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:01.394548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:01.394722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:01.394863Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:01.394983Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:01.395116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:01.395286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:01.395330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:01.395349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:01.395446Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:01.400276Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:01.400295Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:01.400299Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:01.400301Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:01.400312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343107015073466:9349]
2025-10-09T21:52:01.400323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343107015073466:9349]
2025-10-09T21:52:01.400328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343107015073466:9349]
2025-10-09T21:52:01.400367Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:01.400376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:01.400379Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:01.400383Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:01.400384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.400386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:01.400576Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:01.451583Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343107376925580:7496]
2025-10-09T21:52:01.451886Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 2320, pipe: [1:7559343107376925580:7496]
2025-10-09T21:52:01.551729Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:01.558568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.558590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.558743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.558801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.558866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.558923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.559090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.559099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.559162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.559221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.559277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.559337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.559376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.559387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.559440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.559492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.559549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.559602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.559642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.559647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.559702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.559771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.559828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.559883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.559922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.559928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.559988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.560042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.560097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.560152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.560192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.560200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.560256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.560466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.560524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.560579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.560664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.560670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.560727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.560780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.561022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.561078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.561117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.561128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.561182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.561235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.561290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.561358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.561406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.561414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.561471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.561524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.561580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.561640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.561684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.561690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.561743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.561793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.561853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.562078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.562127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.562134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.562188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343107376925580:7496], now have 1 active actors on pipe
2025-10-09T21:52:01.562407Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:01.562407Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:01.562425Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:01.562459Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2321 reason: error at write: Disconnected sessionId: src_id|28934265-d3a4266e-ad1973e2-e39e8d91_0
2025-10-09T21:52:01.562467Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343107376925580:7496]
2025-10-09T21:52:01.562656Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2321 sessionId: src_id|28934265-d3a4266e-ad1973e2-e39e8d91_0 is DEAD
2025-10-09T21:52:01.562723Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:01.562828Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28934265-d3a4266e-ad1973e2-e39e8d91_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:01.562843Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28934265-d3a4266e-ad1973e2-e39e8d91_0] Write session will restart in 0.009301s
2025-10-09T21:52:01.562868Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28934265-d3a4266e-ad1973e2-e39e8d91_0] Write session: Do CDS request
2025-10-09T21:52:01.562877Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28934265-d3a4266e-ad1973e2-e39e8d91_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:01.572811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:01.573551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|28934265-d3a4266e-ad1973e2-e39e8d91_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:01.573655Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:01.573687Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2322
2025-10-09T21:52:01.573855Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2322 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:01.573888Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2322 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:01.573900Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2322 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:01.573908Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:01.574351Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2322 sessionId:  describe result for acl check
2025-10-09T21:52:01.574392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:01.574400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:01.574405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:01.574411Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343107376925700:7178] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:01.574604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:01.574796Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:01.574807Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:01.574820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:01.574880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:01.574924Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343107015073624:9347]
2025-10-09T21:52:01.575002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343107376925700:7178] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:01.575504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:01.575752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:01.575791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:01.576083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:01.576130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.576138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.576139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.576141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.576142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.576143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.576144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.576145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.576145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.576150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.576159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.576162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.576166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.576169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.576172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.576175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.576179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.576182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.576188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.576197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.576203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.576207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.576210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.576215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.576218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.576223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.576228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.576232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.576237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.576246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.576251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.576260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.576266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.576270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.576278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.576282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.576307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:01.577268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343107376925700:7178] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:01.583373Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343107376925700:7178] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:01.583390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343107376925700:7178] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:01.583395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343107376925700:7178] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:01.583405Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2322 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:01.583689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:01.584906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:01.584919Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:01.584921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:01.584925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:01.584933Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2321 [2:7559343107015073624:9347]
2025-10-09T21:52:01.584950Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:01.584968Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:01.584971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:01.584975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:01.584977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:01.584996Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:01.584999Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:01.585005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:01.585063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.585070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.585072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.585075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:01.585142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:01.585150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.585162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.585178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.585202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:01.585230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343107376925737:7178], now have 1 active actors on pipe
2025-10-09T21:52:01.585233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343107376925584:2432], now have 1 active actors on pipe
2025-10-09T21:52:01.585241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343107015073618:9347]
2025-10-09T21:52:01.585248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343107015073618:9347]
2025-10-09T21:52:01.585251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343107015073618:9347]
2025-10-09T21:52:01.585245Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2321
2025-10-09T21:52:01.585266Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:01.585258Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2321
2025-10-09T21:52:01.585277Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:01.585284Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:01.585290Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:01.585292Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:01.585311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:01.585319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:01.585368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.585377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.585387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343107015073618:9347]
2025-10-09T21:52:01.585394Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:01.585401Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:01.585417Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f91959aa-96a7e0a0-a02fafe5-8151b31d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:01.585427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:01.585430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:01.585432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:01.585439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:01.585451Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:01.585471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:01.585606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.585615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.585628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:01.585720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2322 partition: 0 MaxSeqNo: 25 sessionId: src_id|f91959aa-96a7e0a0-a02fafe5-8151b31d_0
2025-10-09T21:52:01.586079Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|28934265-d3a4266e-ad1973e2-e39e8d91_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f91959aa-96a7e0a0-a02fafe5-8151b31d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:01.588430Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f91959aa-96a7e0a0-a02fafe5-8151b31d_0] Write session: try to update token
2025-10-09T21:52:01.588447Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f91959aa-96a7e0a0-a02fafe5-8151b31d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:01.602068Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:01.602086Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:01.602088Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:01.602090Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:01.602100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343107015073618:9347]
2025-10-09T21:52:01.602105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343107015073618:9347]
2025-10-09T21:52:01.602110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343107015073618:9347]
2025-10-09T21:52:01.602156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:01.602160Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:01.602165Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:01.602168Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:01.602169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.602171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:01.602308Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:01.603278Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2322 sessionId: src_id|f91959aa-96a7e0a0-a02fafe5-8151b31d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:01.605832Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:01.662456Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343107376925745:7496]
2025-10-09T21:52:01.662745Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 2321, pipe: [1:7559343107376925745:7496]
2025-10-09T21:52:01.748501Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:01.748815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.748832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.748873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343107376925745:7496], now have 1 active actors on pipe
2025-10-09T21:52:01.749032Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:01.749120Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:01.749131Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:01.749178Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343107376925745:7496]
2025-10-09T21:52:01.749231Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2322 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f91959aa-96a7e0a0-a02fafe5-8151b31d_0
2025-10-09T21:52:01.749440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2322 sessionId: src_id|f91959aa-96a7e0a0-a02fafe5-8151b31d_0 is DEAD
2025-10-09T21:52:01.749587Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f91959aa-96a7e0a0-a02fafe5-8151b31d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:01.749598Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f91959aa-96a7e0a0-a02fafe5-8151b31d_0] Write session will restart in 0.006040s
2025-10-09T21:52:01.749623Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f91959aa-96a7e0a0-a02fafe5-8151b31d_0] Write session: Do CDS request
2025-10-09T21:52:01.749629Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f91959aa-96a7e0a0-a02fafe5-8151b31d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:01.756470Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f91959aa-96a7e0a0-a02fafe5-8151b31d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:01.756516Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:01.756530Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2323
2025-10-09T21:52:01.756700Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2323 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:01.756725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2323 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:01.756733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2323 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:01.756735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:01.757151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2323 sessionId:  describe result for acl check
2025-10-09T21:52:01.757188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:01.757195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:01.757196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:01.757202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343107376925870:7165] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:01.757778Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343107376925870:7165] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:01.758521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:01.759512Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343107376925870:7165] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:01.760072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:01.760250Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:01.760261Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:01.760274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:01.760333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:01.760377Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343107015073777:9335]
2025-10-09T21:52:01.760774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:01.761032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:01.761061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:01.761313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:01.761359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.761364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.761365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.761366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.761366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.761367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.761368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.761368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.761369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.761372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.761378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.761381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.761383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.761385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.761387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.761391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.761395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.761399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.761404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.761411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.761417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.761422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.761426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.761431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.761435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.761441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.761445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.761450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.761454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.761457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.761462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.761465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.761468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.761470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.761474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.761477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.761493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:01.761828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343107376925870:7165] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:01.761841Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343107376925870:7165] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:01.761846Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343107376925870:7165] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:01.761854Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2323 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:01.768825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:01.769827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:01.769840Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:01.769842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:01.769845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:01.769850Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2322 [2:7559343107015073777:9335]
2025-10-09T21:52:01.769860Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:01.769874Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:01.769876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:01.769879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:01.769880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:01.769893Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:01.769897Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:01.769903Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:01.769951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.769957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.769958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.769960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:01.770013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:01.770019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.770027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.770046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.770057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:01.770074Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343107015073772:9335]
2025-10-09T21:52:01.770080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343107015073772:9335]
2025-10-09T21:52:01.770081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343107015073772:9335]
2025-10-09T21:52:01.770087Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:01.770098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:01.770101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:01.770104Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:01.770106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:01.770111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:01.770113Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:01.770122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343107376925748:2432], now have 1 active actors on pipe
2025-10-09T21:52:01.770124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343107376925911:7165], now have 1 active actors on pipe
2025-10-09T21:52:01.770144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343107015073772:9335]
2025-10-09T21:52:01.770153Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:01.770174Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:01.770170Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2322
2025-10-09T21:52:01.770189Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2322
2025-10-09T21:52:01.770222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.770229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.770251Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2e333681-8d041a22-ba67c85b-5aa2d398_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:01.770261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:01.770264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:01.770265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:01.770272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:01.770282Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:01.770296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:01.770432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.770437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.770447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:01.770530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2323 partition: 0 MaxSeqNo: 25 sessionId: src_id|2e333681-8d041a22-ba67c85b-5aa2d398_0
2025-10-09T21:52:01.770802Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f91959aa-96a7e0a0-a02fafe5-8151b31d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2e333681-8d041a22-ba67c85b-5aa2d398_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:01.773129Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e333681-8d041a22-ba67c85b-5aa2d398_0] Write session: try to update token
2025-10-09T21:52:01.773147Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e333681-8d041a22-ba67c85b-5aa2d398_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:01.787975Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2323 sessionId: src_id|2e333681-8d041a22-ba67c85b-5aa2d398_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:01.788179Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:01.788194Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:01.788201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343107015073772:9335]
2025-10-09T21:52:01.788242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:01.788250Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:01.788406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:01.788540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:01.788665Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:01.788785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:01.789047Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:01.789082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:01.790317Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:01.790483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.790498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.790526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:01.790609Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:01.799122Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:01.799137Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:01.799139Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:01.799140Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:01.799149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343107015073772:9335]
2025-10-09T21:52:01.799152Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343107015073772:9335]
2025-10-09T21:52:01.799155Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343107015073772:9335]
2025-10-09T21:52:01.799187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:01.799193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.799195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:01.799201Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:01.799206Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:01.799210Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:01.799315Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:01.849312Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343107376925922:7496]
2025-10-09T21:52:01.849557Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 2322, pipe: [1:7559343107376925922:7496]
2025-10-09T21:52:01.942334Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:01.948980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.949006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.949158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.949217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.949275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.949349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.949394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.949402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.949468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.949525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.949591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.949658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.949699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.949705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.949769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.949830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.949894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.949949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.949989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.949994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.950067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.950176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.950237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.950299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.950341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.950352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.950418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.950485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.950547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.950616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.950740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.950756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.950823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.950894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.950961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.951024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.951063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.951066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.951142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.951206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.951278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.951635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.951700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.951708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.951771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.951876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.951929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.951989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.952039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.952045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.952112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.952164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.952222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.952277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.952323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.952335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.952400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:01.952456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:01.952517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:01.952581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:01.952645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:01.952652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:01.952711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343107376925922:7496], now have 1 active actors on pipe
2025-10-09T21:52:01.952992Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343107376925922:7496]
2025-10-09T21:52:01.953028Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:01.953066Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:01.953071Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:01.953093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2323 reason: error at write: Disconnected sessionId: src_id|2e333681-8d041a22-ba67c85b-5aa2d398_0
2025-10-09T21:52:01.953261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2323 sessionId: src_id|2e333681-8d041a22-ba67c85b-5aa2d398_0 is DEAD
2025-10-09T21:52:01.953578Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e333681-8d041a22-ba67c85b-5aa2d398_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:01.953587Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e333681-8d041a22-ba67c85b-5aa2d398_0] Write session will restart in 0.009640s
2025-10-09T21:52:01.953638Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e333681-8d041a22-ba67c85b-5aa2d398_0] Write session: Do CDS request
2025-10-09T21:52:01.953644Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e333681-8d041a22-ba67c85b-5aa2d398_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:01.953654Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:01.964483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2e333681-8d041a22-ba67c85b-5aa2d398_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:01.964554Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:01.964570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2324
2025-10-09T21:52:01.964589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:01.964684Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2324 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:01.964707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2324 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:01.964714Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2324 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:01.964717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:01.964999Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2324 sessionId:  describe result for acl check
2025-10-09T21:52:01.965025Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:01.965027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:01.965028Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:01.965032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343107376926060:7132] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:01.965471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343107376926060:7132] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:01.966339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:01.966608Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:01.966622Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:01.966638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:01.966713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:01.966755Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343107015073949:9332]
2025-10-09T21:52:01.967356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:01.967646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:01.967691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:01.967885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343107376926060:7132] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:01.968013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:01.968059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.968065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.968066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.968067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.968068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.968070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.968071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.968072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.968073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.968077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.968089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.968092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.968096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.968101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.968104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.968108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.968111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.968114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.968121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.968129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:01.968135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.968140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:01.968147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.968158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:01.968162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.968167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:01.968176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.968182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:01.968186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.968191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:01.968196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.968200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.968204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.968209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.968213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.968217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.968246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:01.974469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343107376926060:7132] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:01.974486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343107376926060:7132] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:01.974488Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343107376926060:7132] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:01.974495Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2324 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:01.975778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:01.976956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:01.976972Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:01.976975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:01.976979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:01.976987Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2323 [2:7559343107015073949:9332]
2025-10-09T21:52:01.977000Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:01.977016Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:01.977024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:01.977027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:01.977030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:01.977046Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:01.977052Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:01.977063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:01.977117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.977124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.977126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.977129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:01.977196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:01.977206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:01.977217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:01.977230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:01.977246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:01.977267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343107376925933:2432], now have 1 active actors on pipe
2025-10-09T21:52:01.977276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343107376926094:7132], now have 1 active actors on pipe
2025-10-09T21:52:01.977284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343107015073943:9332]
2025-10-09T21:52:01.977287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343107015073943:9332]
2025-10-09T21:52:01.977289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343107015073943:9332]
2025-10-09T21:52:01.977312Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:01.977327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:01.977331Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:01.977337Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:01.977340Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:01.977344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:01.977347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:01.977359Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2323
2025-10-09T21:52:01.977383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343107015073943:9332]
2025-10-09T21:52:01.977392Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:01.977394Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:01.977412Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2323
2025-10-09T21:52:01.977430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.977440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.977467Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|eb14b464-b64fdff4-a8de1eb1-d788d81d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:01.977479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:01.977483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:01.977485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:01.977494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:01.977509Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:01.977524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:01.977666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.977674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.977687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:01.977781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2324 partition: 0 MaxSeqNo: 25 sessionId: src_id|eb14b464-b64fdff4-a8de1eb1-d788d81d_0
2025-10-09T21:52:01.978121Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2e333681-8d041a22-ba67c85b-5aa2d398_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|eb14b464-b64fdff4-a8de1eb1-d788d81d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:01.980396Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eb14b464-b64fdff4-a8de1eb1-d788d81d_0] Write session: try to update token
2025-10-09T21:52:01.980411Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eb14b464-b64fdff4-a8de1eb1-d788d81d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:01.991306Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:01.991325Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:01.991335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343107015073943:9332]
2025-10-09T21:52:01.991377Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:01.991381Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:01.991541Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:01.991694Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:01.991827Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:01.991948Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:01.992386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:01.992429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:01.993059Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2324 sessionId: src_id|eb14b464-b64fdff4-a8de1eb1-d788d81d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:01.995253Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:01.995406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:01.995422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:01.995452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:01.995505Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:02.008617Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:02.008634Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:02.008636Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:02.008638Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:02.008648Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343107015073943:9332]
2025-10-09T21:52:02.008651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343107015073943:9332]
2025-10-09T21:52:02.008656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343107015073943:9332]
2025-10-09T21:52:02.008691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:02.008701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.008703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:02.008702Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:02.008708Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:02.008711Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:02.008825Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:02.053120Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343111671893408:7496]
2025-10-09T21:52:02.053383Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 2323, pipe: [1:7559343111671893408:7496]
2025-10-09T21:52:02.157635Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:02.165315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:02.165339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:02.165501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:02.165566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:02.165625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:02.165687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:02.165732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:02.165741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:02.165803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:02.165858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:02.165917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:02.165984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:02.166044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:02.166050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:02.166110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:02.166165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:02.166224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:02.166282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:02.166327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:02.166333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:02.166393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:02.166452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:02.166506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:02.166563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:02.166611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:02.166619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:02.166682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:02.166736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:02.166793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:02.166862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:02.166934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:02.166945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:02.167007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:02.167066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:02.167135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:02.167195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:02.167237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:02.167243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:02.167330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:02.167409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:02.167472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:02.167532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:02.167578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:02.167585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:02.167645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:02.167712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:02.167772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:02.167830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:02.167878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:02.167884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:02.167952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:02.168008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:02.168069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:02.168165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:02.168206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:02.168212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:02.168303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:02.168362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:02.168426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:02.168485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:02.168539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:02.168547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:02.168603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343111671893408:7496], now have 1 active actors on pipe
2025-10-09T21:52:02.168907Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343111671893408:7496]
2025-10-09T21:52:02.168965Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:02.168967Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:02.169007Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2324 reason: error at write: Disconnected sessionId: src_id|eb14b464-b64fdff4-a8de1eb1-d788d81d_0
2025-10-09T21:52:02.169183Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2324 sessionId: src_id|eb14b464-b64fdff4-a8de1eb1-d788d81d_0 is DEAD
2025-10-09T21:52:02.169385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb14b464-b64fdff4-a8de1eb1-d788d81d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:02.169405Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb14b464-b64fdff4-a8de1eb1-d788d81d_0] Write session will restart in 0.005326s
2025-10-09T21:52:02.169428Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb14b464-b64fdff4-a8de1eb1-d788d81d_0] Write session: Do CDS request
2025-10-09T21:52:02.169436Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb14b464-b64fdff4-a8de1eb1-d788d81d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:02.169690Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:02.169775Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:02.175496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|eb14b464-b64fdff4-a8de1eb1-d788d81d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:02.175596Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:02.175616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2325
2025-10-09T21:52:02.175805Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2325 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:02.175841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2325 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:02.175853Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2325 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:02.175858Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:02.176313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2325 sessionId:  describe result for acl check
2025-10-09T21:52:02.176350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:02.176351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:02.176353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:02.176357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343111671893427:7104] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:02.176861Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343111671893427:7104] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:02.180225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343111671893427:7104] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:02.182655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:02.182730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343111671893427:7104] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:02.182750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343111671893427:7104] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:02.182755Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343111671893427:7104] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:02.182764Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2325 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:02.185082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:02.185322Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:02.185337Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:02.185348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:02.185400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:02.185444Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343111310041426:9311]
2025-10-09T21:52:02.186132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:02.186478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:02.186524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:02.186863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:02.186925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.186936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.186938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.186939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.186940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.186942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.186943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.186945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.186946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.186951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.186962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.186967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.186971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.186976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.186981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.186984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.186988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.186992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.186998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.187006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.187012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.187018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.187022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.187028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.187033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.187038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.187043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.187049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.187057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.187062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.187068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.187073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.187078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.187084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.187090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.187096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.187128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:02.196474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:02.198354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:02.198376Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:02.198379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:02.198382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:02.198392Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2324 [2:7559343111310041426:9311]
2025-10-09T21:52:02.198404Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:02.198420Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:02.198422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:02.198424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:02.198426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:02.198438Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:02.198442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:02.198448Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:02.198513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.198524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.198526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.198529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:02.198610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:02.198628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.198645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.198658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.198676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:02.198710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343111671893412:2432], now have 1 active actors on pipe
2025-10-09T21:52:02.198724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343111671893564:7104], now have 1 active actors on pipe
2025-10-09T21:52:02.198732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343111310041419:9311]
2025-10-09T21:52:02.198742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343111310041419:9311]
2025-10-09T21:52:02.198747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343111310041419:9311]
2025-10-09T21:52:02.198768Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:02.198777Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:02.198780Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:02.198782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:02.198800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:02.198790Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2324
2025-10-09T21:52:02.198817Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:02.198790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2324
2025-10-09T21:52:02.198828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:02.198890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:02.198906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:02.198929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343111310041419:9311]
2025-10-09T21:52:02.198950Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:02.198952Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:02.198982Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|449613f0-687d350f-39bc6713-acc901a1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:02.198996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:02.199002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:02.199005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:02.199021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:02.199041Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:02.199076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:02.199353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:02.199370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:02.199390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:02.199502Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2325 partition: 0 MaxSeqNo: 25 sessionId: src_id|449613f0-687d350f-39bc6713-acc901a1_0
2025-10-09T21:52:02.199888Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|eb14b464-b64fdff4-a8de1eb1-d788d81d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|449613f0-687d350f-39bc6713-acc901a1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:02.202543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|449613f0-687d350f-39bc6713-acc901a1_0] Write session: try to update token
2025-10-09T21:52:02.202567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|449613f0-687d350f-39bc6713-acc901a1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:02.217329Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:02.217353Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:02.217357Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:02.217359Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:02.217370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343111310041419:9311]
2025-10-09T21:52:02.217375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343111310041419:9311]
2025-10-09T21:52:02.217380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343111310041419:9311]
2025-10-09T21:52:02.217428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:02.217431Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:02.217435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.217438Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:02.217438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:02.217441Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:02.217786Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:02.217880Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2325 sessionId: src_id|449613f0-687d350f-39bc6713-acc901a1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:02.220270Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:02.269140Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343111671893578:7496]
2025-10-09T21:52:02.269496Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 2324, pipe: [1:7559343111671893578:7496]
2025-10-09T21:52:02.363816Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:02.364054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:02.364065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:02.364105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343111671893578:7496], now have 1 active actors on pipe
2025-10-09T21:52:02.364292Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:02.364388Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:02.364409Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343111671893578:7496]
2025-10-09T21:52:02.364418Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:02.364428Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:02.364481Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2325 reason: error at write: Disconnected sessionId: src_id|449613f0-687d350f-39bc6713-acc901a1_0
2025-10-09T21:52:02.364694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2325 sessionId: src_id|449613f0-687d350f-39bc6713-acc901a1_0 is DEAD
2025-10-09T21:52:02.364824Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|449613f0-687d350f-39bc6713-acc901a1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:02.364838Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|449613f0-687d350f-39bc6713-acc901a1_0] Write session will restart in 0.005387s
2025-10-09T21:52:02.364862Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|449613f0-687d350f-39bc6713-acc901a1_0] Write session: Do CDS request
2025-10-09T21:52:02.364869Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|449613f0-687d350f-39bc6713-acc901a1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:02.371413Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|449613f0-687d350f-39bc6713-acc901a1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:02.371478Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:02.371494Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2326
2025-10-09T21:52:02.371606Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2326 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:02.371631Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2326 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:02.371641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2326 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:02.371657Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:02.372078Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2326 sessionId:  describe result for acl check
2025-10-09T21:52:02.372102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:02.372108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:02.372110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:02.372114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343111671893623:7099] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:02.372531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343111671893623:7099] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:02.374936Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343111671893623:7099] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:02.376247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:02.379134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:02.379269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343111671893623:7099] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:02.379284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343111671893623:7099] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:02.379287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343111671893623:7099] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:02.379294Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2326 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:02.379364Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:02.379373Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:02.379385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:02.379442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:02.379477Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343111310041582:9319]
2025-10-09T21:52:02.380080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:02.380385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:02.380425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:02.380745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:02.380793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.380799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.380800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.380802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.380803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.380804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.380806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.380807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.380808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.380813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.380824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.380828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.380832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.380837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.380841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.380845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.380849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.380853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.380861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.380869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.380875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.380883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.380887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.380895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.380899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.380909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.380914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.380919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.380929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.380935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.380940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.380949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.380954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.380959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.380969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.380973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.381005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:02.391112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:02.392454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:02.392471Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:02.392474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:02.392478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:02.392485Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2325 [2:7559343111310041582:9319]
2025-10-09T21:52:02.392501Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:02.392525Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:02.392537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:02.392540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:02.392543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:02.392561Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:02.392567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:02.392575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:02.392635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.392643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.392646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.392649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:02.392725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:02.392733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.392746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.392758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.392781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:02.392807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343111671893740:7099], now have 1 active actors on pipe
2025-10-09T21:52:02.392817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343111671893586:2432], now have 1 active actors on pipe
2025-10-09T21:52:02.392826Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343111310041576:9319]
2025-10-09T21:52:02.392828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343111310041576:9319]
2025-10-09T21:52:02.392831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343111310041576:9319]
2025-10-09T21:52:02.392845Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:02.392860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:02.392869Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:02.392879Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2325
2025-10-09T21:52:02.392875Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:02.392882Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2325
2025-10-09T21:52:02.392875Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:02.392878Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:02.392878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:02.392912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343111310041576:9319]
2025-10-09T21:52:02.392917Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:02.392920Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:02.392937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:02.392943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:02.392992Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|97e348fe-8fe20f48-29fa8785-9304f3d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:02.393005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:02.393009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:02.393012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:02.393023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:02.393037Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:02.393058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:02.393204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:02.393212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:02.393225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:02.393310Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2326 partition: 0 MaxSeqNo: 25 sessionId: src_id|97e348fe-8fe20f48-29fa8785-9304f3d_0
2025-10-09T21:52:02.393662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|449613f0-687d350f-39bc6713-acc901a1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|97e348fe-8fe20f48-29fa8785-9304f3d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:02.396056Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97e348fe-8fe20f48-29fa8785-9304f3d_0] Write session: try to update token
2025-10-09T21:52:02.396075Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97e348fe-8fe20f48-29fa8785-9304f3d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:02.409643Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2326 sessionId: src_id|97e348fe-8fe20f48-29fa8785-9304f3d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:02.410980Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:02.411000Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:02.411003Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:02.411005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:02.411015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343111310041576:9319]
2025-10-09T21:52:02.411019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343111310041576:9319]
2025-10-09T21:52:02.411024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343111310041576:9319]
2025-10-09T21:52:02.411070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:02.411084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.411084Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:02.411087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:02.411097Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:02.411103Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:02.411221Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:02.412031Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:02.464756Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343111671893751:7496]
2025-10-09T21:52:02.465097Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 2325, pipe: [1:7559343111671893751:7496]
2025-10-09T21:52:02.553024Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:02.553450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:02.553464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:02.553503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343111671893751:7496], now have 1 active actors on pipe
2025-10-09T21:52:02.553683Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:02.553797Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:02.553797Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343111671893751:7496]
2025-10-09T21:52:02.553825Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:02.553832Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:02.554097Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2326 reason: error at write: Disconnected sessionId: src_id|97e348fe-8fe20f48-29fa8785-9304f3d_0
2025-10-09T21:52:02.554246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2326 sessionId: src_id|97e348fe-8fe20f48-29fa8785-9304f3d_0 is DEAD
2025-10-09T21:52:02.554387Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97e348fe-8fe20f48-29fa8785-9304f3d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:02.554396Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97e348fe-8fe20f48-29fa8785-9304f3d_0] Write session will restart in 0.008245s
2025-10-09T21:52:02.554423Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97e348fe-8fe20f48-29fa8785-9304f3d_0] Write session: Do CDS request
2025-10-09T21:52:02.554430Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97e348fe-8fe20f48-29fa8785-9304f3d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:02.563485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|97e348fe-8fe20f48-29fa8785-9304f3d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:02.563566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:02.563589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2327
2025-10-09T21:52:02.563838Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2327 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:02.563889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2327 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:02.563900Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2327 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:02.563904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:02.564291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2327 sessionId:  describe result for acl check
2025-10-09T21:52:02.564326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:02.564333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:02.564335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:02.564340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343111671893871:7097] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:02.564860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343111671893871:7097] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:02.565387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:02.567105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343111671893871:7097] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:02.567236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:02.567405Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:02.567413Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:02.567422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:02.567463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:02.567501Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343111310041739:9317]
2025-10-09T21:52:02.567945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:02.568232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:02.568265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:02.568477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:02.568510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.568511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.568512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.568512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.568513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.568514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.568514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.568515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.568515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.568519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.568525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.568527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.568529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.568531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.568533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.568535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.568537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.568539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.568543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.568548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.568552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.568555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.568558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.568561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.568563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.568566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.568568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.568571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.568574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.568577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.568580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.568583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.568585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.568589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.568592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.568595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.568610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:02.569398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343111671893871:7097] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:02.569412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343111671893871:7097] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:02.569414Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343111671893871:7097] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:02.569420Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2327 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:02.574903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:02.575904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:02.575917Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:02.575919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:02.575921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:02.575926Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2326 [2:7559343111310041739:9317]
2025-10-09T21:52:02.575934Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:02.575946Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:02.575948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:02.575950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:02.575951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:02.575962Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:02.575966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:02.575971Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:02.576006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.576014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.576015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.576017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:02.576068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:02.576074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.576081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.576087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.576100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:02.576114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343111671893916:7097], now have 1 active actors on pipe
2025-10-09T21:52:02.576116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343111671893754:2432], now have 1 active actors on pipe
2025-10-09T21:52:02.576121Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343111310041733:9317]
2025-10-09T21:52:02.576123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343111310041733:9317]
2025-10-09T21:52:02.576124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343111310041733:9317]
2025-10-09T21:52:02.576133Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:02.576144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:02.576155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:02.576163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:02.576180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343111310041733:9317]
2025-10-09T21:52:02.576182Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:02.576191Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:02.576195Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:02.576171Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2326
2025-10-09T21:52:02.576171Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2326
2025-10-09T21:52:02.576197Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:02.576203Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:02.576210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:02.576214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:02.576235Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|55d6ff75-ab5393e4-2b01620c-29577c70_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:02.576240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:02.576245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:02.576247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:02.576253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:02.576260Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:02.576270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:02.576386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:02.576391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:02.576401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:02.576493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2327 partition: 0 MaxSeqNo: 25 sessionId: src_id|55d6ff75-ab5393e4-2b01620c-29577c70_0
2025-10-09T21:52:02.576782Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|97e348fe-8fe20f48-29fa8785-9304f3d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|55d6ff75-ab5393e4-2b01620c-29577c70_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:02.579145Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55d6ff75-ab5393e4-2b01620c-29577c70_0] Write session: try to update token
2025-10-09T21:52:02.579165Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55d6ff75-ab5393e4-2b01620c-29577c70_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:02.591099Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:02.591115Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:02.591118Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:02.591120Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:02.591130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343111310041733:9317]
2025-10-09T21:52:02.591134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343111310041733:9317]
2025-10-09T21:52:02.591139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343111310041733:9317]
2025-10-09T21:52:02.591176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:02.591179Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:02.591181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.591183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:02.591191Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:02.591194Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:02.591308Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:02.592200Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2327 sessionId: src_id|55d6ff75-ab5393e4-2b01620c-29577c70_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:02.594652Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:02.654207Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343111671893924:7496]
2025-10-09T21:52:02.654499Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 2326, pipe: [1:7559343111671893924:7496]
2025-10-09T21:52:02.741119Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:02.741667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:02.741682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:02.741720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343111671893924:7496], now have 1 active actors on pipe
2025-10-09T21:52:02.741901Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:02.741945Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:02.741948Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343111671893924:7496]
2025-10-09T21:52:02.741961Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:02.741971Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:02.742047Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2327 reason: error at write: Disconnected sessionId: src_id|55d6ff75-ab5393e4-2b01620c-29577c70_0
2025-10-09T21:52:02.742230Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2327 sessionId: src_id|55d6ff75-ab5393e4-2b01620c-29577c70_0 is DEAD
2025-10-09T21:52:02.742394Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55d6ff75-ab5393e4-2b01620c-29577c70_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:02.742406Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55d6ff75-ab5393e4-2b01620c-29577c70_0] Write session will restart in 0.007671s
2025-10-09T21:52:02.742436Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55d6ff75-ab5393e4-2b01620c-29577c70_0] Write session: Do CDS request
2025-10-09T21:52:02.742443Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55d6ff75-ab5393e4-2b01620c-29577c70_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:02.751527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|55d6ff75-ab5393e4-2b01620c-29577c70_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:02.751636Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:02.751652Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2328
2025-10-09T21:52:02.751774Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2328 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:02.751811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2328 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:02.751822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2328 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:02.751826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:02.752042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:02.752209Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2328 sessionId:  describe result for acl check
2025-10-09T21:52:02.752240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:02.752252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:02.752253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:02.752257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343111671894048:7087] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:02.752844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343111671894048:7087] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:02.753617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:02.753751Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:02.753759Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:02.753768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:02.753804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:02.753840Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343111310041897:9315]
2025-10-09T21:52:02.754218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:02.754463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:02.754501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:02.754744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:02.754788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.754792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.754764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343111671894048:7087] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:02.754793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.754793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.754794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.754795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.754795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.754796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.754796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.754799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.754806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.754809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.754813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.754817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.754821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.754824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.754827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.754830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.754836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.754843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.754849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.754854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.754858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.754863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.754869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.754874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.754879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.754884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.754888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.754894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.754904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.754907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.754909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.754912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.754915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.754918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.754935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:02.761266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343111671894048:7087] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:02.761282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343111671894048:7087] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:02.761284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343111671894048:7087] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:02.761291Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2328 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:02.762920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:02.763917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:02.763928Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:02.763930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:02.763933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:02.763939Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2327 [2:7559343111310041897:9315]
2025-10-09T21:52:02.763947Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:02.763961Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:02.763963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:02.763964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:02.763966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:02.763978Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:02.763982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:02.763988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:02.764026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.764031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.764032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.764034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:02.764087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:02.764095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.764106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.764115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.764129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:02.764148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343111671894087:7087], now have 1 active actors on pipe
2025-10-09T21:52:02.764154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343111671893927:2432], now have 1 active actors on pipe
2025-10-09T21:52:02.764159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343111310041891:9315]
2025-10-09T21:52:02.764161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343111310041891:9315]
2025-10-09T21:52:02.764162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343111310041891:9315]
2025-10-09T21:52:02.764175Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:02.764188Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:02.764190Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:02.764200Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:02.764201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:02.764201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:02.764203Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:02.764222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343111310041891:9315]
2025-10-09T21:52:02.764231Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:02.764219Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2327
2025-10-09T21:52:02.764236Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2327
2025-10-09T21:52:02.764240Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:02.764271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:02.764278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:02.764300Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c3f6bce4-fa6cd4a7-bbc02b9d-6fb4691d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:02.764308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:02.764311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:02.764313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:02.764319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:02.764332Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:02.764345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:02.764533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:02.764545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:02.764561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:02.764714Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2328 partition: 0 MaxSeqNo: 25 sessionId: src_id|c3f6bce4-fa6cd4a7-bbc02b9d-6fb4691d_0
2025-10-09T21:52:02.765054Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|55d6ff75-ab5393e4-2b01620c-29577c70_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c3f6bce4-fa6cd4a7-bbc02b9d-6fb4691d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:02.771658Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f6bce4-fa6cd4a7-bbc02b9d-6fb4691d_0] Write session: try to update token
2025-10-09T21:52:02.771673Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f6bce4-fa6cd4a7-bbc02b9d-6fb4691d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:02.776657Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:02.776680Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:02.776689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343111310041891:9315]
2025-10-09T21:52:02.776743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:02.776746Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:02.776968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:02.777157Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:02.777370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:02.777564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:02.777864Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:02.777909Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:02.784614Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2328 sessionId: src_id|c3f6bce4-fa6cd4a7-bbc02b9d-6fb4691d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:02.791249Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:02.791266Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:02.791267Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:02.791268Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:02.791276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343111310041891:9315]
2025-10-09T21:52:02.791280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343111310041891:9315]
2025-10-09T21:52:02.791282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343111310041891:9315]
2025-10-09T21:52:02.791314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:02.791321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.791323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:02.791324Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:02.791328Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:02.791332Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:02.791449Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:02.792183Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:02.842109Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343111671894095:7496]
2025-10-09T21:52:02.842340Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 2327, pipe: [1:7559343111671894095:7496]
2025-10-09T21:52:02.930686Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:02.931179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:02.931199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:02.931223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343111671894095:7496], now have 1 active actors on pipe
2025-10-09T21:52:02.931404Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:02.931445Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:02.931456Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:02.931456Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343111671894095:7496]
2025-10-09T21:52:02.931464Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:02.931527Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2328 reason: error at write: Disconnected sessionId: src_id|c3f6bce4-fa6cd4a7-bbc02b9d-6fb4691d_0
2025-10-09T21:52:02.931709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2328 sessionId: src_id|c3f6bce4-fa6cd4a7-bbc02b9d-6fb4691d_0 is DEAD
2025-10-09T21:52:02.931856Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f6bce4-fa6cd4a7-bbc02b9d-6fb4691d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:02.931867Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f6bce4-fa6cd4a7-bbc02b9d-6fb4691d_0] Write session will restart in 0.006067s
2025-10-09T21:52:02.931894Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f6bce4-fa6cd4a7-bbc02b9d-6fb4691d_0] Write session: Do CDS request
2025-10-09T21:52:02.931900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f6bce4-fa6cd4a7-bbc02b9d-6fb4691d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:02.939533Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f6bce4-fa6cd4a7-bbc02b9d-6fb4691d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:02.939620Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:02.939638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2329
2025-10-09T21:52:02.939753Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2329 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:02.939805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2329 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:02.939816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2329 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:02.939820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:02.940295Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2329 sessionId:  describe result for acl check
2025-10-09T21:52:02.940334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:02.940341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:02.940342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:02.940348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343111671894164:7073] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:02.940895Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343111671894164:7073] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:02.942786Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343111671894164:7073] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:02.942981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:02.944531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:02.944710Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:02.944720Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:02.944730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:02.944778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:02.944815Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343111310042060:9308]
2025-10-09T21:52:02.945016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343111671894164:7073] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:02.945028Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343111671894164:7073] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:02.945032Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343111671894164:7073] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:02.945038Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2329 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:02.945196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:02.945431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:02.945458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:02.945653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:02.945681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.945685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.945686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.945687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.945687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.945688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.945688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.945689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.945689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.945692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.945698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.945700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.945703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.945704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.945706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.945708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.945710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.945712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.945716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.945721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:02.945725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.945728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:02.945730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.945733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:02.945736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.945739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:02.945741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.945744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:02.945746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.945749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:02.945752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.945755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.945758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.945761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.945764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.945767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.945783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:02.952969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:02.954100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:02.954113Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:02.954116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:02.954121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:02.954127Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2328 [2:7559343111310042060:9308]
2025-10-09T21:52:02.954141Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:02.954162Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:02.954169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:02.954172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:02.954175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:02.954191Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:02.954202Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:02.954209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:02.954409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.954417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.954421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.954423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:02.954492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:02.954499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.954510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:02.954526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:02.954544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:02.954570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343111671894100:2432], now have 1 active actors on pipe
2025-10-09T21:52:02.954579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343111671894257:7073], now have 1 active actors on pipe
2025-10-09T21:52:02.954586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343111310042054:9308]
2025-10-09T21:52:02.954588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343111310042054:9308]
2025-10-09T21:52:02.954590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343111310042054:9308]
2025-10-09T21:52:02.954603Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:02.954593Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2328
2025-10-09T21:52:02.954617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:02.954633Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2328
2025-10-09T21:52:02.954624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:02.954634Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:02.954637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:02.954670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343111310042054:9308]
2025-10-09T21:52:02.954673Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:02.954675Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:02.954678Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:02.954682Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:02.954698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:02.954703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:02.954752Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b312fc42-a6af2198-952aac4d-960ccfa1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:02.954765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:02.954773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:02.954776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:02.954786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:02.954800Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:02.954820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:02.954958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:02.954965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:02.954979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:02.955046Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2329 partition: 0 MaxSeqNo: 25 sessionId: src_id|b312fc42-a6af2198-952aac4d-960ccfa1_0
2025-10-09T21:52:02.955377Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c3f6bce4-fa6cd4a7-bbc02b9d-6fb4691d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b312fc42-a6af2198-952aac4d-960ccfa1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:02.957885Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b312fc42-a6af2198-952aac4d-960ccfa1_0] Write session: try to update token
2025-10-09T21:52:02.957898Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b312fc42-a6af2198-952aac4d-960ccfa1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:02.969805Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:02.969822Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:02.969825Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:02.969827Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:02.969839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343111310042054:9308]
2025-10-09T21:52:02.969844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343111310042054:9308]
2025-10-09T21:52:02.969848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343111310042054:9308]
2025-10-09T21:52:02.969888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:02.969888Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:02.969894Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:02.969895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:02.969897Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:02.969898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:02.970056Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:02.970297Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2329 sessionId: src_id|b312fc42-a6af2198-952aac4d-960ccfa1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:02.972611Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:03.031549Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343115966861586:7496]
2025-10-09T21:52:03.031765Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 2328, pipe: [1:7559343115966861586:7496]
2025-10-09T21:52:03.112598Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:03.112838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:03.112850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:03.112888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343115966861586:7496], now have 1 active actors on pipe
2025-10-09T21:52:03.113065Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:03.113087Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343115966861586:7496]
2025-10-09T21:52:03.113104Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:03.113109Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:03.113124Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:03.113172Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2329 reason: error at write: Disconnected sessionId: src_id|b312fc42-a6af2198-952aac4d-960ccfa1_0
2025-10-09T21:52:03.113380Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2329 sessionId: src_id|b312fc42-a6af2198-952aac4d-960ccfa1_0 is DEAD
2025-10-09T21:52:03.113552Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b312fc42-a6af2198-952aac4d-960ccfa1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:03.113565Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b312fc42-a6af2198-952aac4d-960ccfa1_0] Write session will restart in 0.007826s
2025-10-09T21:52:03.113593Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b312fc42-a6af2198-952aac4d-960ccfa1_0] Write session: Do CDS request
2025-10-09T21:52:03.113599Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b312fc42-a6af2198-952aac4d-960ccfa1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:03.122515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b312fc42-a6af2198-952aac4d-960ccfa1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:03.122590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:03.122607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2330
2025-10-09T21:52:03.122769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2330 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:03.122803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2330 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:03.122809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2330 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:03.122813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:03.123710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2330 sessionId:  describe result for acl check
2025-10-09T21:52:03.123752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:03.123760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:03.123762Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:03.123768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343115966861710:7019] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:03.124126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:03.124328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343115966861710:7019] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:03.125970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:03.126180Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:03.126191Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:03.126203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:03.126262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:03.126301Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343115605009554:9291]
2025-10-09T21:52:03.126871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:03.126919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343115966861710:7019] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:03.127182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:03.127228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:03.127543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:03.127589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.127597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.127598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.127599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.127600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.127601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.127603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.127604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.127605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.127610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.127621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.127625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.127629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.127633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.127637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.127641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.127645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.127649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.127655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.127663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.127670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.127676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.127681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.127686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.127690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.127695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.127699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.127704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.127709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.127714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.127724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.127729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.127733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.127744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.127749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.127754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.127780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:03.133129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343115966861710:7019] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:03.133142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343115966861710:7019] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:03.133147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343115966861710:7019] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:03.133155Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2330 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:03.134620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:03.135732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:03.135748Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:03.135749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:03.135752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:03.135757Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2329 [2:7559343115605009554:9291]
2025-10-09T21:52:03.135772Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:03.135790Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:03.135798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:03.135801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:03.135803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:03.135816Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:03.135824Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:03.135829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:03.135873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.135878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.135880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.135881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:03.135931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:03.135936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.135943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.135950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.135959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:03.135973Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343115605009548:9291]
2025-10-09T21:52:03.135978Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343115605009548:9291]
2025-10-09T21:52:03.135980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343115605009548:9291]
2025-10-09T21:52:03.135986Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:03.135997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343115966861751:7019], now have 1 active actors on pipe
2025-10-09T21:52:03.135997Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:03.136000Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:03.136002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:03.136004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343115966861589:2432], now have 1 active actors on pipe
2025-10-09T21:52:03.136008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:03.136016Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:03.136018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:03.136036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343115605009548:9291]
2025-10-09T21:52:03.136042Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:03.136047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:03.136058Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2329
2025-10-09T21:52:03.136084Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2329
2025-10-09T21:52:03.136116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:03.136124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:03.136144Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1878690-c94ceb8b-9db6825e-93c2a76_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:03.136154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:03.136162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:03.136163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:03.136170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:03.136181Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:03.136195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:03.136364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:03.136375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:03.136389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:03.136486Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2330 partition: 0 MaxSeqNo: 25 sessionId: src_id|1878690-c94ceb8b-9db6825e-93c2a76_0
2025-10-09T21:52:03.136799Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b312fc42-a6af2198-952aac4d-960ccfa1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1878690-c94ceb8b-9db6825e-93c2a76_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:03.139243Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1878690-c94ceb8b-9db6825e-93c2a76_0] Write session: try to update token
2025-10-09T21:52:03.139262Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1878690-c94ceb8b-9db6825e-93c2a76_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:03.150699Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:03.150718Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:03.150720Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:03.150722Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:03.150732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343115605009548:9291]
2025-10-09T21:52:03.150736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343115605009548:9291]
2025-10-09T21:52:03.150740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343115605009548:9291]
2025-10-09T21:52:03.150784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:03.150796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.150797Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:03.150803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:03.150804Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:03.150808Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:03.150939Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:03.152157Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2330 sessionId: src_id|1878690-c94ceb8b-9db6825e-93c2a76_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:03.154499Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:03.213305Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343115966861758:7496]
2025-10-09T21:52:03.213527Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 2329, pipe: [1:7559343115966861758:7496]
2025-10-09T21:52:03.297185Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:03.297612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:03.297625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:03.297663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343115966861758:7496], now have 1 active actors on pipe
2025-10-09T21:52:03.297822Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:03.297875Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343115966861758:7496]
2025-10-09T21:52:03.297900Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:03.297904Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:03.297976Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2330 reason: status is not ok: tablet will be restarted right now sessionId: src_id|1878690-c94ceb8b-9db6825e-93c2a76_0
2025-10-09T21:52:03.298168Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2330 sessionId: src_id|1878690-c94ceb8b-9db6825e-93c2a76_0 is DEAD
2025-10-09T21:52:03.298295Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1878690-c94ceb8b-9db6825e-93c2a76_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:03.298305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1878690-c94ceb8b-9db6825e-93c2a76_0] Write session will restart in 0.009899s
2025-10-09T21:52:03.298325Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1878690-c94ceb8b-9db6825e-93c2a76_0] Write session: Do CDS request
2025-10-09T21:52:03.298330Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1878690-c94ceb8b-9db6825e-93c2a76_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:03.308495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:03.309569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1878690-c94ceb8b-9db6825e-93c2a76_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:03.309627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:03.309645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2331
2025-10-09T21:52:03.309799Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2331 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:03.309825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2331 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:03.309829Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2331 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:03.309831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:03.310223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:03.310238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2331 sessionId:  describe result for acl check
2025-10-09T21:52:03.310278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:03.310284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:03.310286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:03.310292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343115966861889:7025] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:03.310380Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:03.310387Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:03.310395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:03.310435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:03.310474Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343115605009716:9289]
2025-10-09T21:52:03.310856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:03.310843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343115966861889:7025] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:03.311149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:03.311177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:03.311380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:03.311414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.311418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.311419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.311420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.311420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.311421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.311422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.311423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.311423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.311427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.311432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.311435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.311437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.311439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.311442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.311444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.311446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.311448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.311452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.311457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.311461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.311464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.311467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.311470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.311472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.311475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.311478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.311481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.311483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.311487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.311490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.311493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.311495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.311498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.311501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.311504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.311520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:03.312897Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343115966861889:7025] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:03.317234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343115966861889:7025] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:03.317249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343115966861889:7025] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:03.317256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343115966861889:7025] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:03.317265Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2331 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:03.317596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:03.318585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:03.318600Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:03.318603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:03.318606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:03.318611Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2330 [2:7559343115605009716:9289]
2025-10-09T21:52:03.318621Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:03.318634Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:03.318640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:03.318643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:03.318644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:03.318657Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:03.318661Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:03.318666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:03.318704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.318710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.318711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.318713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:03.318765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:03.318771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.318780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.318790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.318803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:03.318820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343115605009710:9289]
2025-10-09T21:52:03.318829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343115605009710:9289]
2025-10-09T21:52:03.318830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343115605009710:9289]
2025-10-09T21:52:03.318840Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:03.318853Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:03.318855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343115966861764:2432], now have 1 active actors on pipe
2025-10-09T21:52:03.318855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:03.318857Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:03.318858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343115966861932:7025], now have 1 active actors on pipe
2025-10-09T21:52:03.318866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:03.318879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:03.318882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:03.318901Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2330
2025-10-09T21:52:03.318923Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2330
2025-10-09T21:52:03.318907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343115605009710:9289]
2025-10-09T21:52:03.318917Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:03.318935Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:03.318942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:03.318947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:03.318974Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8e577b35-9a84c87d-3a8887a4-de95bc29_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:03.318984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:03.318987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:03.318989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:03.318996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:03.319004Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:03.319016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:03.319195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:03.319200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:03.319211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:03.319308Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2331 partition: 0 MaxSeqNo: 25 sessionId: src_id|8e577b35-9a84c87d-3a8887a4-de95bc29_0
2025-10-09T21:52:03.319613Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1878690-c94ceb8b-9db6825e-93c2a76_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8e577b35-9a84c87d-3a8887a4-de95bc29_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:03.322085Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e577b35-9a84c87d-3a8887a4-de95bc29_0] Write session: try to update token
2025-10-09T21:52:03.322102Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e577b35-9a84c87d-3a8887a4-de95bc29_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:03.334520Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:03.334537Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:03.334538Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:03.334540Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:03.334548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343115605009710:9289]
2025-10-09T21:52:03.334551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343115605009710:9289]
2025-10-09T21:52:03.334554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343115605009710:9289]
2025-10-09T21:52:03.334588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:03.334595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.334597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:03.334600Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:03.334606Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:03.334610Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:03.334719Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:03.335260Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2331 sessionId: src_id|8e577b35-9a84c87d-3a8887a4-de95bc29_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:03.337528Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:03.398112Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343115966861939:7496]
2025-10-09T21:52:03.398436Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 2330, pipe: [1:7559343115966861939:7496]
2025-10-09T21:52:03.471070Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:03.471563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:03.471576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:03.471612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343115966861939:7496], now have 1 active actors on pipe
2025-10-09T21:52:03.471806Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:03.471856Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343115966861939:7496]
2025-10-09T21:52:03.471872Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:03.471886Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:03.471957Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2331 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8e577b35-9a84c87d-3a8887a4-de95bc29_0
2025-10-09T21:52:03.472130Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2331 sessionId: src_id|8e577b35-9a84c87d-3a8887a4-de95bc29_0 is DEAD
2025-10-09T21:52:03.472304Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e577b35-9a84c87d-3a8887a4-de95bc29_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:03.472327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e577b35-9a84c87d-3a8887a4-de95bc29_0] Write session will restart in 0.008888s
2025-10-09T21:52:03.472354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e577b35-9a84c87d-3a8887a4-de95bc29_0] Write session: Do CDS request
2025-10-09T21:52:03.472361Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e577b35-9a84c87d-3a8887a4-de95bc29_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:03.481564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:03.482420Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e577b35-9a84c87d-3a8887a4-de95bc29_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:03.482458Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:03.482473Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2332
2025-10-09T21:52:03.482617Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2332 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:03.482660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2332 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:03.482670Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2332 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:03.482673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:03.483051Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2332 sessionId:  describe result for acl check
2025-10-09T21:52:03.483086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:03.483092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:03.483094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:03.483100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343115966862067:7011] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:03.483357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:03.483502Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:03.483509Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:03.483517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:03.483560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:03.483587Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343115605009876:9284]
2025-10-09T21:52:03.483602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343115966862067:7011] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:03.483947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:03.484157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:03.484183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:03.484379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:03.484412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.484417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.484417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.484418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.484419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.484420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.484420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.484420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.484421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.484424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.484430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.484432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.484434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.484436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.484438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.484439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.484441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.484443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.484447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.484452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.484456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.484459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.484461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.484464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.484466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.484469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.484472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.484475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.484477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.484480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.484483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.484486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.484488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.484492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.484495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.484498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.484512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:03.485214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343115966862067:7011] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:03.489662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343115966862067:7011] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:03.489680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343115966862067:7011] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:03.489683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343115966862067:7011] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:03.489688Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2332 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:03.490114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:03.491062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:03.491074Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:03.491075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:03.491078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:03.491082Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2331 [2:7559343115605009876:9284]
2025-10-09T21:52:03.491090Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:03.491101Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:03.491107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:03.491108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:03.491110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:03.491121Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:03.491129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:03.491134Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:03.491174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.491179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.491181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.491184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:03.491239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:03.491245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.491251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.491257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.491276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:03.491298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343115966862106:7011], now have 1 active actors on pipe
2025-10-09T21:52:03.491305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343115966861941:2432], now have 1 active actors on pipe
2025-10-09T21:52:03.491311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343115605009871:9284]
2025-10-09T21:52:03.491317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343115605009871:9284]
2025-10-09T21:52:03.491319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343115605009871:9284]
2025-10-09T21:52:03.491317Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2331
2025-10-09T21:52:03.491331Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:03.491336Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2331
2025-10-09T21:52:03.491340Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:03.491343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:03.491344Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:03.491349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:03.491362Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:03.491365Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:03.491399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:03.491408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:03.491419Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343115605009871:9284]
2025-10-09T21:52:03.491431Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:03.491453Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:03.491458Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8743e4e8-148cbc8-3e8e8933-4e849d1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:03.491464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:03.491466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:03.491467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:03.491474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:03.491481Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:03.491515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:03.491609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:03.491616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:03.491630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:03.491694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2332 partition: 0 MaxSeqNo: 25 sessionId: src_id|8743e4e8-148cbc8-3e8e8933-4e849d1_0
2025-10-09T21:52:03.491938Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e577b35-9a84c87d-3a8887a4-de95bc29_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8743e4e8-148cbc8-3e8e8933-4e849d1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:03.494359Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8743e4e8-148cbc8-3e8e8933-4e849d1_0] Write session: try to update token
2025-10-09T21:52:03.494375Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8743e4e8-148cbc8-3e8e8933-4e849d1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:03.506477Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:03.506496Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:03.506498Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:03.506500Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:03.506509Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343115605009871:9284]
2025-10-09T21:52:03.506514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343115605009871:9284]
2025-10-09T21:52:03.506518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343115605009871:9284]
2025-10-09T21:52:03.506561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:03.506565Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:03.506567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.506570Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:03.506570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:03.506573Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:03.506698Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:03.507405Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2332 sessionId: src_id|8743e4e8-148cbc8-3e8e8933-4e849d1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:03.509548Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:03.572172Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343115966862114:7496]
2025-10-09T21:52:03.572430Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 2331, pipe: [1:7559343115966862114:7496]
2025-10-09T21:52:03.646497Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:03.646724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:03.646736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:03.646773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343115966862114:7496], now have 1 active actors on pipe
2025-10-09T21:52:03.646932Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:03.647011Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343115966862114:7496]
2025-10-09T21:52:03.647021Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:03.647052Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:03.647099Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2332 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8743e4e8-148cbc8-3e8e8933-4e849d1_0
2025-10-09T21:52:03.647291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2332 sessionId: src_id|8743e4e8-148cbc8-3e8e8933-4e849d1_0 is DEAD
2025-10-09T21:52:03.647442Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8743e4e8-148cbc8-3e8e8933-4e849d1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:03.647452Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8743e4e8-148cbc8-3e8e8933-4e849d1_0] Write session will restart in 0.005133s
2025-10-09T21:52:03.647494Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8743e4e8-148cbc8-3e8e8933-4e849d1_0] Write session: Do CDS request
2025-10-09T21:52:03.647500Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8743e4e8-148cbc8-3e8e8933-4e849d1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:03.653472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8743e4e8-148cbc8-3e8e8933-4e849d1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:03.653566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:03.653592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2333
2025-10-09T21:52:03.653726Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2333 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:03.653782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2333 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:03.653793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2333 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:03.653803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:03.654228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2333 sessionId:  describe result for acl check
2025-10-09T21:52:03.654265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:03.654271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:03.654272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:03.654278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343115966862163:6997] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:03.654819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343115966862163:6997] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:03.657276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343115966862163:6997] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:03.658394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:03.660020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:03.660174Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:03.660181Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:03.660190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:03.660227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:03.660265Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343115605010041:9280]
2025-10-09T21:52:03.660330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343115966862163:6997] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:03.660344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343115966862163:6997] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:03.660348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343115966862163:6997] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:03.660355Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2333 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:03.660639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:03.660860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:03.660887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:03.661099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:03.661140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.661146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.661147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.661149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.661150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.661151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.661152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.661154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.661155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.661159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.661168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.661170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.661172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.661174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.661176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.661178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.661180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.661182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.661186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.661191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.661195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.661198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.661201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.661203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.661206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.661208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.661211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.661214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.661216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.661219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.661223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.661226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.661228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.661231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.661233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.661236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.661254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:03.667168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:03.668311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:03.668324Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:03.668326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:03.668328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:03.668333Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2332 [2:7559343115605010041:9280]
2025-10-09T21:52:03.668343Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:03.668357Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:03.668360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:03.668362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:03.668365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:03.668381Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:03.668385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:03.668390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:03.668428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.668434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.668435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.668437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:03.668494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:03.668501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.668510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.668520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.668534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:03.668547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343115605010035:9280]
2025-10-09T21:52:03.668548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343115605010035:9280]
2025-10-09T21:52:03.668550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343115605010035:9280]
2025-10-09T21:52:03.668557Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:03.668568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343115966862277:6997], now have 1 active actors on pipe
2025-10-09T21:52:03.668568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:03.668571Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:03.668573Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:03.668578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:03.668586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:03.668587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:03.668591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343115966862115:2432], now have 1 active actors on pipe
2025-10-09T21:52:03.668613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343115605010035:9280]
2025-10-09T21:52:03.668620Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:03.668624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:03.668663Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2332
2025-10-09T21:52:03.668667Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2332
2025-10-09T21:52:03.668721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:03.668731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:03.668760Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|af129d53-ea68556b-75eef486-4931e350_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:03.668769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:03.668775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:03.668778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:03.668789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:03.668800Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:03.668818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:03.668959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:03.668967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:03.668981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:03.669064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2333 partition: 0 MaxSeqNo: 25 sessionId: src_id|af129d53-ea68556b-75eef486-4931e350_0
2025-10-09T21:52:03.669368Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8743e4e8-148cbc8-3e8e8933-4e849d1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|af129d53-ea68556b-75eef486-4931e350_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:03.671852Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af129d53-ea68556b-75eef486-4931e350_0] Write session: try to update token
2025-10-09T21:52:03.671870Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af129d53-ea68556b-75eef486-4931e350_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:03.679923Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:03.679943Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:03.679955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343115605010035:9280]
2025-10-09T21:52:03.680010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:03.680014Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:03.680273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:03.680446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:03.680592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:03.680722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:03.680891Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:03.680928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:03.685012Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2333 sessionId: src_id|af129d53-ea68556b-75eef486-4931e350_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:03.687329Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:03.687531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:03.687553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:03.687594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:03.687677Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:03.700635Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:03.700651Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:03.700653Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:03.700654Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:03.700663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343115605010035:9280]
2025-10-09T21:52:03.700667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343115605010035:9280]
2025-10-09T21:52:03.700670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343115605010035:9280]
2025-10-09T21:52:03.700715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:03.700715Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:03.700721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.700722Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:03.700723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:03.700726Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:03.700865Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:03.747324Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343115966862289:7496]
2025-10-09T21:52:03.747632Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 2332, pipe: [1:7559343115966862289:7496]
2025-10-09T21:52:03.850354Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:03.858201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:03.858228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:03.858389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:03.858451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:03.858517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:03.858579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:03.859126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:03.859140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:03.859204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:03.859264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:03.859331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:03.859394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:03.859434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:03.859441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:03.859507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:03.859564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:03.859621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:03.859685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:03.859725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:03.859732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:03.859796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:03.859854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:03.859914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:03.859967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:03.860011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:03.860019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:03.860084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:03.860144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:03.860212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:03.860274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:03.860316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:03.860325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:03.860385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:03.860450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:03.860509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:03.860566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:03.860613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:03.860621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:03.860677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:03.860735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:03.860807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:03.860862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:03.860909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:03.860916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:03.860969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:03.861020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:03.861077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:03.861135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:03.861175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:03.861180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:03.861234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:03.861284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:03.861358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:03.861419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:03.861458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:03.861466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:03.861528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:03.861585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:03.861643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:03.861709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:03.861844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:03.861853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:03.861917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343115966862289:7496], now have 1 active actors on pipe
2025-10-09T21:52:03.862128Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:03.862137Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343115966862289:7496]
2025-10-09T21:52:03.862144Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:03.862159Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:03.862170Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2333 reason: error at write: Disconnected sessionId: src_id|af129d53-ea68556b-75eef486-4931e350_0
2025-10-09T21:52:03.862345Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2333 sessionId: src_id|af129d53-ea68556b-75eef486-4931e350_0 is DEAD
2025-10-09T21:52:03.862469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af129d53-ea68556b-75eef486-4931e350_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:03.862482Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af129d53-ea68556b-75eef486-4931e350_0] Write session will restart in 0.009157s
2025-10-09T21:52:03.862508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af129d53-ea68556b-75eef486-4931e350_0] Write session: Do CDS request
2025-10-09T21:52:03.862516Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af129d53-ea68556b-75eef486-4931e350_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:03.862538Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:03.872520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af129d53-ea68556b-75eef486-4931e350_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:03.872622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:03.872637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2334
2025-10-09T21:52:03.872736Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2334 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:03.872760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2334 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:03.872764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2334 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:03.872767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:03.873193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2334 sessionId:  describe result for acl check
2025-10-09T21:52:03.873227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:03.873233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:03.873234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:03.873245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343115966862415:6988] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:03.873687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343115966862415:6988] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:03.874457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:03.876174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:03.876366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343115966862415:6988] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:03.876396Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:03.876405Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:03.876417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:03.876472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:03.876507Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343115605010209:9262]
2025-10-09T21:52:03.877081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:03.877358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:03.877396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:03.877689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:03.877735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.877742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.877743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.877744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.877745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.877747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.877748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.877749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.877750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.877755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.877765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.877769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.877773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.877777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.877781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.877785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.877789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.877793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.877799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.877806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:03.877812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.877817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:03.877821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.877826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:03.877830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.877835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:03.877839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.877844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:03.877849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.877856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:03.877862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.877869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.877873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.877878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.877882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.877892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.877920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:03.882580Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343115966862415:6988] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:03.882593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343115966862415:6988] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:03.882596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343115966862415:6988] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:03.882603Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2334 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:03.883920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:03.885089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:03.885101Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:03.885103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:03.885106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:03.885111Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2333 [2:7559343115605010209:9262]
2025-10-09T21:52:03.885120Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:03.885134Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:03.885136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:03.885138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:03.885139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:03.885156Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:03.885161Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:03.885168Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:03.885225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.885232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.885233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.885235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:03.885289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:03.885310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.885322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:03.885338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:03.885350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:03.885368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343115966862291:2432], now have 1 active actors on pipe
2025-10-09T21:52:03.885374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343115966862456:6988], now have 1 active actors on pipe
2025-10-09T21:52:03.885379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343115605010204:9262]
2025-10-09T21:52:03.885380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343115605010204:9262]
2025-10-09T21:52:03.885381Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343115605010204:9262]
2025-10-09T21:52:03.885391Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:03.885402Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:03.885406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:03.885411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:03.885412Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:03.885414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:03.885416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:03.885441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343115605010204:9262]
2025-10-09T21:52:03.885447Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:03.885454Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:03.885434Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2333
2025-10-09T21:52:03.885469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:03.885475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:03.885448Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2333
2025-10-09T21:52:03.885499Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|777f4225-1ead28c6-f68b6c13-ae56d650_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:03.885509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:03.885516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:03.885517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:03.885524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:03.885536Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:03.885557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:03.885666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:03.885672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:03.885681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:03.885755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2334 partition: 0 MaxSeqNo: 25 sessionId: src_id|777f4225-1ead28c6-f68b6c13-ae56d650_0
2025-10-09T21:52:03.886046Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af129d53-ea68556b-75eef486-4931e350_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|777f4225-1ead28c6-f68b6c13-ae56d650_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:03.888292Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|777f4225-1ead28c6-f68b6c13-ae56d650_0] Write session: try to update token
2025-10-09T21:52:03.888306Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|777f4225-1ead28c6-f68b6c13-ae56d650_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:03.899301Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:03.899318Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:03.899320Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:03.899321Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:03.899329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343115605010204:9262]
2025-10-09T21:52:03.899333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343115605010204:9262]
2025-10-09T21:52:03.899336Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343115605010204:9262]
2025-10-09T21:52:03.899377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:03.899384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:03.899384Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:03.899385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:03.899388Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:03.899391Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:03.899514Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:03.901101Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2334 sessionId: src_id|777f4225-1ead28c6-f68b6c13-ae56d650_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:03.903518Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:03.962365Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343115966862467:7496]
2025-10-09T21:52:03.962713Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 2333, pipe: [1:7559343115966862467:7496]
2025-10-09T21:52:04.049952Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:04.050292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.050308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.050344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343115966862467:7496], now have 1 active actors on pipe
2025-10-09T21:52:04.050497Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:04.050540Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:04.050559Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:04.050621Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2334 reason: error at write: Disconnected sessionId: src_id|777f4225-1ead28c6-f68b6c13-ae56d650_0
2025-10-09T21:52:04.050799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2334 sessionId: src_id|777f4225-1ead28c6-f68b6c13-ae56d650_0 is DEAD
2025-10-09T21:52:04.050959Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|777f4225-1ead28c6-f68b6c13-ae56d650_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:04.050968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|777f4225-1ead28c6-f68b6c13-ae56d650_0] Write session will restart in 0.005965s
2025-10-09T21:52:04.050985Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|777f4225-1ead28c6-f68b6c13-ae56d650_0] Write session: Do CDS request
2025-10-09T21:52:04.050990Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|777f4225-1ead28c6-f68b6c13-ae56d650_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:04.050978Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:04.051029Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343115966862467:7496]
2025-10-09T21:52:04.058521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|777f4225-1ead28c6-f68b6c13-ae56d650_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:04.058570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:04.058589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2335
2025-10-09T21:52:04.058730Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2335 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:04.058765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2335 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:04.058772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2335 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:04.058781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:04.059224Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2335 sessionId:  describe result for acl check
2025-10-09T21:52:04.059261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:04.059268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:04.059270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:04.059275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343120261829826:6948] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:04.059799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343120261829826:6948] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:04.061128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343120261829826:6948] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:04.063881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:04.064115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343120261829826:6948] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:04.064131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343120261829826:6948] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:04.064135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343120261829826:6948] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:04.064143Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2335 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:04.067509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:04.067805Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:04.067816Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:04.067829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:04.067886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:04.067928Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343119899977710:9268]
2025-10-09T21:52:04.068532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:04.068827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:04.068866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:04.069171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:04.069214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.069220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.069221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.069222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.069224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.069225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.069226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.069227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.069228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.069233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.069244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.069248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.069253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.069257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.069261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.069265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.069269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.069273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.069278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.069286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.069292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.069309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.069313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.069319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.069323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.069328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.069332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.069337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.069345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.069350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.069355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.069360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.069364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.069371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.069376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.069381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.069410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:04.075919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:04.077458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:04.077472Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:04.077474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:04.077479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:04.077485Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2334 [2:7559343119899977710:9268]
2025-10-09T21:52:04.077498Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:04.077517Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:04.077520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:04.077523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:04.077525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:04.077543Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:04.077549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:04.077561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:04.077620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.077629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.077631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.077633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:04.077703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:04.077713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.077725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.077737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.077754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:04.077781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343120261829946:6948], now have 1 active actors on pipe
2025-10-09T21:52:04.077784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343120261829778:2432], now have 1 active actors on pipe
2025-10-09T21:52:04.077791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343119899977696:9268]
2025-10-09T21:52:04.077794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343119899977696:9268]
2025-10-09T21:52:04.077796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343119899977696:9268]
2025-10-09T21:52:04.077811Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:04.077829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:04.077830Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:04.077833Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:04.077835Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:04.077844Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:04.077852Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:04.077842Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2334
2025-10-09T21:52:04.077862Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2334
2025-10-09T21:52:04.077884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343119899977696:9268]
2025-10-09T21:52:04.077895Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:04.077895Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:04.077915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.077921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.077973Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7ad3ef64-eb5f63c4-ffc5d8d3-42d4a514_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:04.077985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:04.077995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:04.078001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:04.078012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:04.078027Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:04.078059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:04.078220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.078229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.078244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:04.078330Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2335 partition: 0 MaxSeqNo: 25 sessionId: src_id|7ad3ef64-eb5f63c4-ffc5d8d3-42d4a514_0
2025-10-09T21:52:04.078679Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|777f4225-1ead28c6-f68b6c13-ae56d650_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7ad3ef64-eb5f63c4-ffc5d8d3-42d4a514_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:04.081472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ad3ef64-eb5f63c4-ffc5d8d3-42d4a514_0] Write session: try to update token
2025-10-09T21:52:04.081492Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ad3ef64-eb5f63c4-ffc5d8d3-42d4a514_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:04.093694Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:04.093713Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:04.093714Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:04.093716Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:04.093725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343119899977696:9268]
2025-10-09T21:52:04.093730Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343119899977696:9268]
2025-10-09T21:52:04.093734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343119899977696:9268]
2025-10-09T21:52:04.093775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:04.093781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.093787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:04.093792Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:04.093798Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:04.093802Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:04.094108Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:04.100393Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2335 sessionId: src_id|7ad3ef64-eb5f63c4-ffc5d8d3-42d4a514_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:04.102875Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:04.151306Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343120261829962:7496]
2025-10-09T21:52:04.151671Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 2334, pipe: [1:7559343120261829962:7496]
2025-10-09T21:52:04.244750Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:04.245000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.245013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.245034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343120261829962:7496], now have 1 active actors on pipe
2025-10-09T21:52:04.245240Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:04.245258Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:04.245266Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:04.245306Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343120261829962:7496]
2025-10-09T21:52:04.245365Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2335 reason: error at write: Disconnected sessionId: src_id|7ad3ef64-eb5f63c4-ffc5d8d3-42d4a514_0
2025-10-09T21:52:04.245557Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2335 sessionId: src_id|7ad3ef64-eb5f63c4-ffc5d8d3-42d4a514_0 is DEAD
2025-10-09T21:52:04.245709Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ad3ef64-eb5f63c4-ffc5d8d3-42d4a514_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:04.245722Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ad3ef64-eb5f63c4-ffc5d8d3-42d4a514_0] Write session will restart in 0.007447s
2025-10-09T21:52:04.245701Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:04.245748Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ad3ef64-eb5f63c4-ffc5d8d3-42d4a514_0] Write session: Do CDS request
2025-10-09T21:52:04.245755Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ad3ef64-eb5f63c4-ffc5d8d3-42d4a514_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:04.254496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7ad3ef64-eb5f63c4-ffc5d8d3-42d4a514_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:04.254568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:04.254584Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2336
2025-10-09T21:52:04.254700Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2336 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:04.254731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2336 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:04.254742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2336 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:04.254746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:04.255171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2336 sessionId:  describe result for acl check
2025-10-09T21:52:04.255205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:04.255207Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:04.255211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:04.255216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343120261830094:6936] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:04.255769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343120261830094:6936] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:04.255758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:04.257760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343120261830094:6936] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:04.257864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:04.258022Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:04.258049Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:04.258061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:04.258113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:04.258152Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343119899977875:9238]
2025-10-09T21:52:04.258607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:04.258836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:04.258866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:04.259074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:04.259107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.259111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.259112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.259112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.259113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.259114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.259114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.259115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.259115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.259119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.259125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.259127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.259129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.259131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.259133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.259135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.259137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.259139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.259143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.259148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.259153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.259156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.259158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.259161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.259167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.259170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.259173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.259176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.259178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.259185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.259187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.259190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.259193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.259196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.259199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.259202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.259219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:04.264092Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343120261830094:6936] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:04.264107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343120261830094:6936] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:04.264111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343120261830094:6936] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:04.264118Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2336 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:04.265294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:04.266480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:04.266493Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:04.266494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:04.266496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:04.266501Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2335 [2:7559343119899977875:9238]
2025-10-09T21:52:04.266510Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:04.266522Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:04.266528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:04.266530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:04.266531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:04.266543Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:04.266547Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:04.266552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:04.266588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.266593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.266594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.266596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:04.266643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:04.266650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.266657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.266663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.266672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:04.266687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343120261829968:2432], now have 1 active actors on pipe
2025-10-09T21:52:04.266689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343120261830136:6936], now have 1 active actors on pipe
2025-10-09T21:52:04.266695Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343119899977869:9238]
2025-10-09T21:52:04.266696Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343119899977869:9238]
2025-10-09T21:52:04.266697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343119899977869:9238]
2025-10-09T21:52:04.266705Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:04.266715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:04.266715Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:04.266717Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:04.266718Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:04.266723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:04.266725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:04.266741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343119899977869:9238]
2025-10-09T21:52:04.266750Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:04.266747Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2335
2025-10-09T21:52:04.266760Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2335
2025-10-09T21:52:04.266766Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:04.266774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.266777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.266799Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7f700dac-a43360df-c2248fd9-562b0300_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:04.266806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:04.266810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:04.266812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:04.266821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:04.266830Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:04.266843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:04.266967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.266973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.266982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:04.267048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2336 partition: 0 MaxSeqNo: 25 sessionId: src_id|7f700dac-a43360df-c2248fd9-562b0300_0
2025-10-09T21:52:04.267365Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7ad3ef64-eb5f63c4-ffc5d8d3-42d4a514_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7f700dac-a43360df-c2248fd9-562b0300_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:04.269749Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7f700dac-a43360df-c2248fd9-562b0300_0] Write session: try to update token
2025-10-09T21:52:04.269766Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7f700dac-a43360df-c2248fd9-562b0300_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:04.280802Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:04.280817Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:04.280819Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:04.280820Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:04.280828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343119899977869:9238]
2025-10-09T21:52:04.280831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343119899977869:9238]
2025-10-09T21:52:04.280834Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343119899977869:9238]
2025-10-09T21:52:04.280877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:04.280885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.280885Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:04.280888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:04.280890Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:04.280893Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:04.281006Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:04.285286Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2336 sessionId: src_id|7f700dac-a43360df-c2248fd9-562b0300_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:04.287546Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:04.345350Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343120261830148:7496]
2025-10-09T21:52:04.345596Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 2335, pipe: [1:7559343120261830148:7496]
2025-10-09T21:52:04.420927Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:04.421263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.421274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.421323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343120261830148:7496], now have 1 active actors on pipe
2025-10-09T21:52:04.421482Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:04.421546Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:04.421564Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:04.421569Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:04.421606Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343120261830148:7496]
2025-10-09T21:52:04.421625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2336 reason: error at write: Disconnected sessionId: src_id|7f700dac-a43360df-c2248fd9-562b0300_0
2025-10-09T21:52:04.421808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2336 sessionId: src_id|7f700dac-a43360df-c2248fd9-562b0300_0 is DEAD
2025-10-09T21:52:04.421942Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f700dac-a43360df-c2248fd9-562b0300_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:04.421954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f700dac-a43360df-c2248fd9-562b0300_0] Write session will restart in 0.008230s
2025-10-09T21:52:04.421979Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f700dac-a43360df-c2248fd9-562b0300_0] Write session: Do CDS request
2025-10-09T21:52:04.421986Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f700dac-a43360df-c2248fd9-562b0300_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:04.431519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7f700dac-a43360df-c2248fd9-562b0300_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:04.431600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:04.431619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2337
2025-10-09T21:52:04.431725Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2337 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:04.431755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2337 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:04.431765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2337 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:04.431768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:04.431789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:04.432149Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2337 sessionId:  describe result for acl check
2025-10-09T21:52:04.432191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:04.432197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:04.432199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:04.432204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343120261830281:6926] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:04.432630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343120261830281:6926] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:04.433637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:04.433791Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:04.433800Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:04.433809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:04.433847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:04.433884Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343119899978042:9248]
2025-10-09T21:52:04.434312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:04.434547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:04.434579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:04.434789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:04.434805Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343120261830281:6926] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:04.434822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.434827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.434828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.434828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.434829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.434829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.434830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.434830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.434831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.434834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.434840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.434842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.434844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.434846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.434848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.434850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.434852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.434854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.434858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.434863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.434867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.434870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.434872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.434876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.434878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.434881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.434884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.434887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.434891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.434894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.434897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.434900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.434902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.434905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.434907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.434910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.434925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:04.440356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343120261830281:6926] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:04.440374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343120261830281:6926] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:04.440377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343120261830281:6926] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:04.440388Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2337 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:04.440685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:04.441829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:04.441842Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:04.441845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:04.441849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:04.441856Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2336 [2:7559343119899978042:9248]
2025-10-09T21:52:04.441869Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:04.441889Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:04.441894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:04.441896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:04.441897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:04.441910Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:04.441918Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:04.441924Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:04.441966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.441972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.441973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.441975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:04.442154Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2336
2025-10-09T21:52:04.442192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:04.442197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2336
2025-10-09T21:52:04.442200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.442207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.442213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.442224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:04.442240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343120261830320:6926], now have 1 active actors on pipe
2025-10-09T21:52:04.442246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343120261830150:2432], now have 1 active actors on pipe
2025-10-09T21:52:04.442260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.442267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.442282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343119899978036:9248]
2025-10-09T21:52:04.442287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343119899978036:9248]
2025-10-09T21:52:04.442288Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343119899978036:9248]
2025-10-09T21:52:04.442298Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:04.442311Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:04.442314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:04.442319Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:04.442321Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:04.442322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:04.442324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:04.442339Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2f564415-98570fa2-8b9f0045-1cb5c0cb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:04.442348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:04.442355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:04.442358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:04.442371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:04.442385Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:04.442418Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343119899978036:9248]
2025-10-09T21:52:04.442427Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:04.442434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:04.442438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:04.442554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.442560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.442571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:04.442660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2337 partition: 0 MaxSeqNo: 25 sessionId: src_id|2f564415-98570fa2-8b9f0045-1cb5c0cb_0
2025-10-09T21:52:04.442982Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f700dac-a43360df-c2248fd9-562b0300_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2f564415-98570fa2-8b9f0045-1cb5c0cb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:04.445278Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2f564415-98570fa2-8b9f0045-1cb5c0cb_0] Write session: try to update token
2025-10-09T21:52:04.445293Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2f564415-98570fa2-8b9f0045-1cb5c0cb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:04.455512Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:04.455526Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:04.455528Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:04.455529Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:04.455537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343119899978036:9248]
2025-10-09T21:52:04.455540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343119899978036:9248]
2025-10-09T21:52:04.455543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343119899978036:9248]
2025-10-09T21:52:04.455579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:04.455587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.455590Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:04.455599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:04.455599Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:04.455603Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:04.455722Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:04.458119Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2337 sessionId: src_id|2f564415-98570fa2-8b9f0045-1cb5c0cb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:04.460482Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:04.521929Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343120261830327:7496]
2025-10-09T21:52:04.522165Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 2336, pipe: [1:7559343120261830327:7496]
2025-10-09T21:52:04.595212Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:04.595513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.595528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.595574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343120261830327:7496], now have 1 active actors on pipe
2025-10-09T21:52:04.595730Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:04.595774Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:04.595816Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:04.595832Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343120261830327:7496]
2025-10-09T21:52:04.595895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2337 reason: status is not ok: tablet will be restarted right now sessionId: src_id|2f564415-98570fa2-8b9f0045-1cb5c0cb_0
2025-10-09T21:52:04.596078Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2337 sessionId: src_id|2f564415-98570fa2-8b9f0045-1cb5c0cb_0 is DEAD
2025-10-09T21:52:04.596232Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f564415-98570fa2-8b9f0045-1cb5c0cb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:04.596245Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f564415-98570fa2-8b9f0045-1cb5c0cb_0] Write session will restart in 0.009985s
2025-10-09T21:52:04.596269Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f564415-98570fa2-8b9f0045-1cb5c0cb_0] Write session: Do CDS request
2025-10-09T21:52:04.596275Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f564415-98570fa2-8b9f0045-1cb5c0cb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:04.606321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:04.607491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2f564415-98570fa2-8b9f0045-1cb5c0cb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:04.607549Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:04.607569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2338
2025-10-09T21:52:04.607685Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2338 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:04.607711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2338 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:04.607717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2338 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:04.607720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:04.608162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2338 sessionId:  describe result for acl check
2025-10-09T21:52:04.608198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:04.608205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:04.608206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:04.608212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343120261830462:6912] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:04.608385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:04.608546Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:04.608555Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:04.608564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:04.608617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:04.608650Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343119899978210:9246]
2025-10-09T21:52:04.608803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343120261830462:6912] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:04.609034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:04.609333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:04.609392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:04.609643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:04.609691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.609695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.609696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.609697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.609697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.609698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.609699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.609700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.609700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.609703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.609709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.609711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.609714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.609716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.609718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.609720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.609722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.609724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.609729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.609733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.609738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.609741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.609744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.609747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.609749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.609752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.609755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.609758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.609761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.609764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.609767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.609771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.609773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.609776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.609779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.609782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.609797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:04.610697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343120261830462:6912] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:04.615404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343120261830462:6912] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:04.615427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343120261830462:6912] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:04.615431Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343120261830462:6912] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:04.615438Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2338 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:04.615835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:04.617108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:04.617122Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:04.617125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:04.617129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:04.617135Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2337 [2:7559343119899978210:9246]
2025-10-09T21:52:04.617151Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:04.617171Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:04.617179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:04.617183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:04.617185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:04.617201Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:04.617212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:04.617224Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:04.617286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.617293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.617305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.617308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:04.617376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:04.617384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.617397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.617414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.617436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:04.617463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343120261830500:6912], now have 1 active actors on pipe
2025-10-09T21:52:04.617471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343120261830329:2432], now have 1 active actors on pipe
2025-10-09T21:52:04.617455Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2337
2025-10-09T21:52:04.617480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343119899978204:9246]
2025-10-09T21:52:04.617467Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2337
2025-10-09T21:52:04.617486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343119899978204:9246]
2025-10-09T21:52:04.617489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343119899978204:9246]
2025-10-09T21:52:04.617503Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:04.617509Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:04.617512Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:04.617513Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:04.617522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:04.617538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:04.617546Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:04.617568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.617578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.617616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343119899978204:9246]
2025-10-09T21:52:04.617629Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:04.617632Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:04.617650Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|580a5354-368fa0c2-c195025e-2b6bcd95_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:04.617661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:04.617669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:04.617676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:04.617687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:04.617701Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:04.617738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:04.617862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.617870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.617882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:04.617950Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2338 partition: 0 MaxSeqNo: 25 sessionId: src_id|580a5354-368fa0c2-c195025e-2b6bcd95_0
2025-10-09T21:52:04.618300Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2f564415-98570fa2-8b9f0045-1cb5c0cb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|580a5354-368fa0c2-c195025e-2b6bcd95_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:04.620645Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|580a5354-368fa0c2-c195025e-2b6bcd95_0] Write session: try to update token
2025-10-09T21:52:04.620661Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|580a5354-368fa0c2-c195025e-2b6bcd95_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:04.631277Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:04.631295Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:04.631298Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:04.631300Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:04.631310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343119899978204:9246]
2025-10-09T21:52:04.631315Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343119899978204:9246]
2025-10-09T21:52:04.631319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343119899978204:9246]
2025-10-09T21:52:04.631356Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:04.631364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:04.631369Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:04.631371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.631372Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:04.631374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:04.631550Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:04.633741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2338 sessionId: src_id|580a5354-368fa0c2-c195025e-2b6bcd95_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:04.636010Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:04.696061Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343120261830507:7496]
2025-10-09T21:52:04.696338Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 2337, pipe: [1:7559343120261830507:7496]
2025-10-09T21:52:04.788311Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:04.788613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.788625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.788662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343120261830507:7496], now have 1 active actors on pipe
2025-10-09T21:52:04.788815Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:04.788872Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343120261830507:7496]
2025-10-09T21:52:04.788886Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:04.788901Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:04.788970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2338 reason: status is not ok: tablet will be restarted right now sessionId: src_id|580a5354-368fa0c2-c195025e-2b6bcd95_0
2025-10-09T21:52:04.789179Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2338 sessionId: src_id|580a5354-368fa0c2-c195025e-2b6bcd95_0 is DEAD
2025-10-09T21:52:04.789339Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|580a5354-368fa0c2-c195025e-2b6bcd95_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:04.789353Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|580a5354-368fa0c2-c195025e-2b6bcd95_0] Write session will restart in 0.008974s
2025-10-09T21:52:04.789382Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|580a5354-368fa0c2-c195025e-2b6bcd95_0] Write session: Do CDS request
2025-10-09T21:52:04.789388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|580a5354-368fa0c2-c195025e-2b6bcd95_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:04.799237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:04.799508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|580a5354-368fa0c2-c195025e-2b6bcd95_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:04.799632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:04.799645Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2339
2025-10-09T21:52:04.799773Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2339 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:04.799804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2339 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:04.799814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2339 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:04.799819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:04.800238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2339 sessionId:  describe result for acl check
2025-10-09T21:52:04.800269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:04.800275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:04.800277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:04.800284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343120261830645:6898] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:04.800821Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343120261830645:6898] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:04.801158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:04.801343Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:04.801352Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:04.801361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:04.801411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:04.801441Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343119899978380:9240]
2025-10-09T21:52:04.801913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:04.802136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:04.802178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:04.802421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:04.802459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.802464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.802465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.802465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.802466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.802467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.802467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.802468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.802469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.802472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.802478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.802480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.802482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.802494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.802496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.802499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.802501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.802503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.802508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.802513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:04.802517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.802520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:04.802523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.802526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:04.802528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.802531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:04.802534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.802537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:04.802540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.802543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:04.802546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.802551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.802554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.802557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.802563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.802566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.802581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:04.803006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343120261830645:6898] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:04.808322Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343120261830645:6898] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:04.808337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343120261830645:6898] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:04.808341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343120261830645:6898] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:04.808352Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2339 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:04.808697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:04.809905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:04.809921Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:04.809923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:04.809927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:04.809934Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2338 [2:7559343119899978380:9240]
2025-10-09T21:52:04.809945Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:04.809961Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:04.809969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:04.809973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:04.809975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:04.809988Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:04.809998Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:04.810004Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:04.810061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.810062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.810063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.810065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:04.810118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:04.810125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.810133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:04.810139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:04.810148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:04.810163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343120261830511:2432], now have 1 active actors on pipe
2025-10-09T21:52:04.810170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343120261830684:6898], now have 1 active actors on pipe
2025-10-09T21:52:04.810175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343119899978374:9240]
2025-10-09T21:52:04.810176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343119899978374:9240]
2025-10-09T21:52:04.810178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343119899978374:9240]
2025-10-09T21:52:04.810187Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:04.810201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:04.810213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:04.810215Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:04.810215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:04.810219Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:04.810220Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:04.810237Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343119899978374:9240]
2025-10-09T21:52:04.810244Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:04.810245Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:04.810285Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2338
2025-10-09T21:52:04.810287Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2338
2025-10-09T21:52:04.810371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.810382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.810407Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|57d7f909-6be4941a-ce56b2eb-841af76b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:04.810420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:04.810429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:04.810431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:04.810439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:04.810451Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:04.810466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:04.810620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.810633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.810661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:04.810789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2339 partition: 0 MaxSeqNo: 25 sessionId: src_id|57d7f909-6be4941a-ce56b2eb-841af76b_0
2025-10-09T21:52:04.811165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|580a5354-368fa0c2-c195025e-2b6bcd95_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|57d7f909-6be4941a-ce56b2eb-841af76b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:04.813671Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57d7f909-6be4941a-ce56b2eb-841af76b_0] Write session: try to update token
2025-10-09T21:52:04.813688Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57d7f909-6be4941a-ce56b2eb-841af76b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:04.823815Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:04.823831Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:04.823838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343119899978374:9240]
2025-10-09T21:52:04.823879Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:04.823879Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:04.824061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:04.824192Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:04.824312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:04.824437Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:04.824617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:04.824653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:04.826633Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2339 sessionId: src_id|57d7f909-6be4941a-ce56b2eb-841af76b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:04.828863Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:04.829034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.829050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.829082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:04.829154Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:04.834104Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:04.834117Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:04.834120Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:04.834122Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:04.834131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343119899978374:9240]
2025-10-09T21:52:04.834135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343119899978374:9240]
2025-10-09T21:52:04.834139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343119899978374:9240]
2025-10-09T21:52:04.834178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:04.834178Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:04.834183Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:04.834186Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:04.834190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:04.834193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:04.834322Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:04.889086Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343120261830692:7496]
2025-10-09T21:52:04.889364Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 2338, pipe: [1:7559343120261830692:7496]
2025-10-09T21:52:04.982721Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:04.989540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:04.989568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:04.989717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:04.989780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:04.989851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:04.989915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:04.989959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:04.989969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:04.990046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:04.990105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:04.990166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:04.990236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:04.990277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:04.990285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:04.990344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:04.990401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:04.990461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:04.990518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:04.990557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:04.990564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:04.990626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:04.990686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:04.990744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:04.990801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:04.990846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:04.990853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:04.990914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:04.991012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:04.991113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:04.991172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:04.991212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:04.991219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:04.991314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:04.991368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:04.991427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:04.991488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:04.991529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:04.991535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:04.991594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:04.991650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:04.991707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:04.991917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:04.991962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:04.991969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:04.992166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:04.992236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:04.992298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:04.992356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:04.992397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:04.992404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:04.992461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:04.992515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:04.992576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:04.992631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:04.992687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:04.992697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:04.992754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:04.992811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:04.992867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:04.992923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:04.993015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:04.993024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:04.993070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343120261830692:7496], now have 1 active actors on pipe
2025-10-09T21:52:04.993319Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:04.993353Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:04.993357Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:04.993370Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343120261830692:7496]
2025-10-09T21:52:04.993386Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2339 reason: error at write: Disconnected sessionId: src_id|57d7f909-6be4941a-ce56b2eb-841af76b_0
2025-10-09T21:52:04.993587Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2339 sessionId: src_id|57d7f909-6be4941a-ce56b2eb-841af76b_0 is DEAD
2025-10-09T21:52:04.993694Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:04.993732Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57d7f909-6be4941a-ce56b2eb-841af76b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:04.993744Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57d7f909-6be4941a-ce56b2eb-841af76b_0] Write session will restart in 0.005635s
2025-10-09T21:52:04.993768Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57d7f909-6be4941a-ce56b2eb-841af76b_0] Write session: Do CDS request
2025-10-09T21:52:04.993774Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57d7f909-6be4941a-ce56b2eb-841af76b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:05.000496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57d7f909-6be4941a-ce56b2eb-841af76b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:05.000576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:05.000592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2340
2025-10-09T21:52:05.000752Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2340 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:05.000795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2340 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:05.000804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2340 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:05.000818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:05.001224Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2340 sessionId:  describe result for acl check
2025-10-09T21:52:05.001254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:05.001259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:05.001260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:05.001265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343124556798043:6900] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:05.001752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343124556798043:6900] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:05.004641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343124556798043:6900] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:05.004878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:05.006590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:05.006936Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:05.006949Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:05.006965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:05.007049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:05.007089Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343124194945851:9242]
2025-10-09T21:52:05.007417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343124556798043:6900] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:05.007430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343124556798043:6900] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:05.007436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343124556798043:6900] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:05.007444Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2340 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:05.007678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:05.007972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:05.008020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:05.008335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:05.008406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.008413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.008415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.008416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.008417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.008419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.008420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.008421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.008423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.008428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.008440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.008445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.008450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.008454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.008458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.008463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.008466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.008470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.008478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.008486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.008492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.008497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.008504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.008509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.008514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.008519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.008524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.008529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.008538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.008543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.008548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.008553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.008558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.008563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.008571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.008576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.008608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:05.015021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:05.016381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:05.016400Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:05.016403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:05.016407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:05.016416Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2339 [2:7559343124194945851:9242]
2025-10-09T21:52:05.016431Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:05.016451Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:05.016460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:05.016464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:05.016466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:05.016485Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:05.016497Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:05.016508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:05.016581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.016590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.016592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.016596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:05.016676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:05.016685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.016699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.016711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.016730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:05.016761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343124556798161:6900], now have 1 active actors on pipe
2025-10-09T21:52:05.016772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343120261830700:2432], now have 1 active actors on pipe
2025-10-09T21:52:05.016779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343124194945845:9242]
2025-10-09T21:52:05.016782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343124194945845:9242]
2025-10-09T21:52:05.016784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343124194945845:9242]
2025-10-09T21:52:05.016799Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:05.016809Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:05.016811Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:05.016813Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:05.016816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:05.016796Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2339
2025-10-09T21:52:05.016833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:05.016797Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2339
2025-10-09T21:52:05.016843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:05.016896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.016906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.016923Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343124194945845:9242]
2025-10-09T21:52:05.016937Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:05.016939Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:05.016972Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|38adc5ef-bba9a9a8-3ac2b970-107b8fcd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:05.016985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:05.016990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:05.016994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:05.017005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:05.017021Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:05.017053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:05.017214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.017228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.017247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:05.017490Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2340 partition: 0 MaxSeqNo: 25 sessionId: src_id|38adc5ef-bba9a9a8-3ac2b970-107b8fcd_0
2025-10-09T21:52:05.017847Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57d7f909-6be4941a-ce56b2eb-841af76b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|38adc5ef-bba9a9a8-3ac2b970-107b8fcd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:05.020252Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38adc5ef-bba9a9a8-3ac2b970-107b8fcd_0] Write session: try to update token
2025-10-09T21:52:05.020270Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38adc5ef-bba9a9a8-3ac2b970-107b8fcd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:05.032837Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:05.032870Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:05.032874Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:05.032878Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:05.032894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343124194945845:9242]
2025-10-09T21:52:05.032902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343124194945845:9242]
2025-10-09T21:52:05.032909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343124194945845:9242]
2025-10-09T21:52:05.032960Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:05.032973Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:05.032977Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:05.032980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:05.032989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.032993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:05.033268Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:05.034133Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2340 sessionId: src_id|38adc5ef-bba9a9a8-3ac2b970-107b8fcd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:05.036604Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:05.093389Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343124556798186:7496]
2025-10-09T21:52:05.093693Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 2339, pipe: [1:7559343124556798186:7496]
2025-10-09T21:52:05.181277Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:05.181728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.181741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.181779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343124556798186:7496], now have 1 active actors on pipe
2025-10-09T21:52:05.181972Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:05.182003Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:05.182008Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:05.182014Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:05.182079Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343124556798186:7496]
2025-10-09T21:52:05.182092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2340 reason: error at write: Disconnected sessionId: src_id|38adc5ef-bba9a9a8-3ac2b970-107b8fcd_0
2025-10-09T21:52:05.182271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2340 sessionId: src_id|38adc5ef-bba9a9a8-3ac2b970-107b8fcd_0 is DEAD
2025-10-09T21:52:05.182431Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38adc5ef-bba9a9a8-3ac2b970-107b8fcd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:05.182449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38adc5ef-bba9a9a8-3ac2b970-107b8fcd_0] Write session will restart in 0.009041s
2025-10-09T21:52:05.182468Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38adc5ef-bba9a9a8-3ac2b970-107b8fcd_0] Write session: Do CDS request
2025-10-09T21:52:05.182473Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38adc5ef-bba9a9a8-3ac2b970-107b8fcd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:05.192534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|38adc5ef-bba9a9a8-3ac2b970-107b8fcd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:05.192628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:05.192644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2341
2025-10-09T21:52:05.192748Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2341 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:05.192779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2341 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:05.192790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2341 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:05.192797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:05.193201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2341 sessionId:  describe result for acl check
2025-10-09T21:52:05.193240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:05.193244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:05.193245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:05.193250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343124556798331:6863] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:05.193784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343124556798331:6863] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:05.193943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:05.195390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343124556798331:6863] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:05.195748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:05.195929Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:05.195937Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:05.195948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:05.196001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:05.196040Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343124194946060:9221]
2025-10-09T21:52:05.196419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:05.196666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:05.196703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:05.196994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:05.197045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.197051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.197052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.197053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.197054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.197056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.197057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.197058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.197059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.197064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.197075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.197080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.197084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.197088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.197092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.197096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.197100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.197105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.197111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.197118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.197125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.197131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.197140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.197145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.197149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.197155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.197164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.197169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.197173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.197179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.197184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.197191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.197196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.197202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.197208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.197214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.197241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:05.197993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343124556798331:6863] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:05.198005Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343124556798331:6863] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:05.198008Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343124556798331:6863] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:05.198016Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2341 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:05.203400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:05.205057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:05.205076Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:05.205079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:05.205083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:05.205091Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2340 [2:7559343124194946060:9221]
2025-10-09T21:52:05.205106Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:05.205125Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:05.205133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:05.205137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:05.205144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:05.205163Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:05.205175Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:05.205182Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:05.205244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.205251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.205253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.205256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:05.205451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:05.205459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.205470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.205480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.205504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:05.205502Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2340
2025-10-09T21:52:05.205530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343124556798369:6863], now have 1 active actors on pipe
2025-10-09T21:52:05.205538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343124556798199:2432], now have 1 active actors on pipe
2025-10-09T21:52:05.205546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343124194946055:9221]
2025-10-09T21:52:05.205550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343124194946055:9221]
2025-10-09T21:52:05.205553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343124194946055:9221]
2025-10-09T21:52:05.205562Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:05.205572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:05.205575Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:05.205577Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:05.205579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:05.205595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:05.205603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:05.205613Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2340
2025-10-09T21:52:05.205644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343124194946055:9221]
2025-10-09T21:52:05.205653Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:05.205660Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:05.205669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.205678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.205724Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f3072a30-1c608eb5-82969666-e551ab6d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:05.205735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:05.205740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:05.205744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:05.205755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:05.205769Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:05.205790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:05.205937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.205945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.205959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:05.206038Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2341 partition: 0 MaxSeqNo: 25 sessionId: src_id|f3072a30-1c608eb5-82969666-e551ab6d_0
2025-10-09T21:52:05.206326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|38adc5ef-bba9a9a8-3ac2b970-107b8fcd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f3072a30-1c608eb5-82969666-e551ab6d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:05.208611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3072a30-1c608eb5-82969666-e551ab6d_0] Write session: try to update token
2025-10-09T21:52:05.208626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3072a30-1c608eb5-82969666-e551ab6d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:05.219675Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:05.219696Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:05.219698Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:05.219700Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:05.219709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343124194946055:9221]
2025-10-09T21:52:05.219714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343124194946055:9221]
2025-10-09T21:52:05.219718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343124194946055:9221]
2025-10-09T21:52:05.219761Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:05.219764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:05.219771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.219774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:05.219780Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:05.219784Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:05.220394Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:05.228145Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2341 sessionId: src_id|f3072a30-1c608eb5-82969666-e551ab6d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:05.230343Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:05.282303Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343124556798381:7496]
2025-10-09T21:52:05.282590Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 2340, pipe: [1:7559343124556798381:7496]
2025-10-09T21:52:05.369321Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:05.369622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.369635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.369656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343124556798381:7496], now have 1 active actors on pipe
2025-10-09T21:52:05.369847Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:05.369915Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343124556798381:7496]
2025-10-09T21:52:05.369923Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:05.369943Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:05.370004Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2341 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f3072a30-1c608eb5-82969666-e551ab6d_0
2025-10-09T21:52:05.370203Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2341 sessionId: src_id|f3072a30-1c608eb5-82969666-e551ab6d_0 is DEAD
2025-10-09T21:52:05.370357Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3072a30-1c608eb5-82969666-e551ab6d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:05.370367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3072a30-1c608eb5-82969666-e551ab6d_0] Write session will restart in 0.008242s
2025-10-09T21:52:05.370386Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3072a30-1c608eb5-82969666-e551ab6d_0] Write session: Do CDS request
2025-10-09T21:52:05.370391Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3072a30-1c608eb5-82969666-e551ab6d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:05.379416Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f3072a30-1c608eb5-82969666-e551ab6d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:05.379532Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:05.379542Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2342
2025-10-09T21:52:05.379657Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2342 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:05.379695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2342 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:05.379704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2342 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:05.379707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:05.380104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2342 sessionId:  describe result for acl check
2025-10-09T21:52:05.380139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:05.380143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:05.380144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:05.380150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343124556798519:6854] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:05.380704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343124556798519:6854] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:05.381025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:05.382480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343124556798519:6854] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:05.383048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:05.383230Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:05.383242Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:05.383253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:05.383309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:05.383362Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343124194946232:9219]
2025-10-09T21:52:05.383951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:05.384253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:05.384301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:05.384608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:05.384653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.384657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.384658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.384659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.384659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.384660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.384661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.384661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.384662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.384665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.384671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.384674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.384676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.384678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.384680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.384682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.384684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.384686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.384691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.384696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.384680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343124556798519:6854] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:05.384701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.384692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343124556798519:6854] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:05.384695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343124556798519:6854] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:05.384704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.384706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.384710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.384714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.384702Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2342 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:05.384717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.384719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.384722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.384724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.384727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.384730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.384732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.384735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.384737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.384740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.384743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.384758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:05.390630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:05.391771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:05.391783Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:05.391784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:05.391787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:05.391791Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2341 [2:7559343124194946232:9219]
2025-10-09T21:52:05.391799Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:05.391812Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:05.391813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:05.391816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:05.391817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:05.391828Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:05.391832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:05.391837Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:05.391874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.391880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.391881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.391883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:05.391930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:05.391935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.391943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.391949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.391962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:05.391984Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343124194946226:9219]
2025-10-09T21:52:05.391990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343124194946226:9219]
2025-10-09T21:52:05.391992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343124194946226:9219]
2025-10-09T21:52:05.392001Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:05.392012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343124556798559:6854], now have 1 active actors on pipe
2025-10-09T21:52:05.392016Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:05.392018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343124556798387:2432], now have 1 active actors on pipe
2025-10-09T21:52:05.392019Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:05.392023Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:05.392025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:05.392035Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:05.392037Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:05.392056Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343124194946226:9219]
2025-10-09T21:52:05.392068Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:05.392073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:05.392056Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2341
2025-10-09T21:52:05.392096Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2341
2025-10-09T21:52:05.392102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.392111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.392134Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dd44379-96496de3-ded39985-26ca8357_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:05.392143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:05.392145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:05.392147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:05.392153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:05.392161Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:05.392173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:05.392280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.392286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.392296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:05.392373Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2342 partition: 0 MaxSeqNo: 25 sessionId: src_id|dd44379-96496de3-ded39985-26ca8357_0
2025-10-09T21:52:05.392602Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f3072a30-1c608eb5-82969666-e551ab6d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dd44379-96496de3-ded39985-26ca8357_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:05.394886Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dd44379-96496de3-ded39985-26ca8357_0] Write session: try to update token
2025-10-09T21:52:05.394900Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dd44379-96496de3-ded39985-26ca8357_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:05.405763Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:05.405782Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:05.405784Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:05.405785Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:05.405793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343124194946226:9219]
2025-10-09T21:52:05.405797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343124194946226:9219]
2025-10-09T21:52:05.405799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343124194946226:9219]
2025-10-09T21:52:05.405835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:05.405841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.405845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:05.405848Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:05.405854Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:05.405869Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:05.405966Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:05.408668Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2342 sessionId: src_id|dd44379-96496de3-ded39985-26ca8357_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:05.410971Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:05.470197Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343124556798569:7496]
2025-10-09T21:52:05.470443Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 2341, pipe: [1:7559343124556798569:7496]
2025-10-09T21:52:05.549966Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:05.550299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.550311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.550347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343124556798569:7496], now have 1 active actors on pipe
2025-10-09T21:52:05.550565Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:05.550592Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:05.550603Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:05.550641Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343124556798569:7496]
2025-10-09T21:52:05.550666Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2342 reason: error at write: Disconnected sessionId: src_id|dd44379-96496de3-ded39985-26ca8357_0
2025-10-09T21:52:05.550669Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:05.550836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2342 sessionId: src_id|dd44379-96496de3-ded39985-26ca8357_0 is DEAD
2025-10-09T21:52:05.550981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd44379-96496de3-ded39985-26ca8357_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:05.550992Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd44379-96496de3-ded39985-26ca8357_0] Write session will restart in 0.006206s
2025-10-09T21:52:05.551021Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd44379-96496de3-ded39985-26ca8357_0] Write session: Do CDS request
2025-10-09T21:52:05.551027Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd44379-96496de3-ded39985-26ca8357_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:05.558497Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dd44379-96496de3-ded39985-26ca8357_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:05.558606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:05.558619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2343
2025-10-09T21:52:05.558749Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2343 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:05.558781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2343 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:05.558792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2343 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:05.558795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:05.559301Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2343 sessionId:  describe result for acl check
2025-10-09T21:52:05.559339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:05.559369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:05.559375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:05.559381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343124556798639:6823] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:05.559967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343124556798639:6823] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:05.562216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343124556798639:6823] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:05.562391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:05.564034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:05.564196Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:05.564204Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:05.564214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:05.564254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:05.564282Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343124194946405:9217]
2025-10-09T21:52:05.564360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343124556798639:6823] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:05.564372Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343124556798639:6823] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:05.564375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343124556798639:6823] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:05.564382Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2343 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:05.564703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:05.564948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:05.564988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:05.565195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:05.565226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.565231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.565232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.565232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.565233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.565233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.565234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.565235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.565236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.565239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.565245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.565247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.565249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.565251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.565253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.565255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.565257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.565259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.565263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.565268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.565273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.565276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.565279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.565282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.565284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.565288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.565290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.565308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.565311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.565314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.565317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.565321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.565323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.565326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.565329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.565332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.565349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:05.571669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:05.572945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:05.572962Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:05.572964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:05.572967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:05.572972Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2342 [2:7559343124194946405:9217]
2025-10-09T21:52:05.573140Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:05.573164Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:05.573168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:05.573170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:05.573172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:05.573184Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:05.573192Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:05.573198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:05.573240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.573245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.573247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.573249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:05.573318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:05.573326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.573338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.573350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.573368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:05.573385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343124556798742:6823], now have 1 active actors on pipe
2025-10-09T21:52:05.573392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343124556798571:2432], now have 1 active actors on pipe
2025-10-09T21:52:05.573397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343124194946399:9217]
2025-10-09T21:52:05.573399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343124194946399:9217]
2025-10-09T21:52:05.573400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343124194946399:9217]
2025-10-09T21:52:05.573412Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:05.573423Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:05.573436Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:05.573438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:05.573448Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:05.573458Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:05.573460Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:05.573463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343124194946399:9217]
2025-10-09T21:52:05.573465Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:05.573467Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:05.573476Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2342
2025-10-09T21:52:05.573481Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2342
2025-10-09T21:52:05.573577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.573586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.573612Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bb788691-b4ea266b-708c8a65-11ae8197_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:05.573622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:05.573626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:05.573627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:05.573634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:05.573646Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:05.573658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:05.573793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.573801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.573813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:05.573909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2343 partition: 0 MaxSeqNo: 25 sessionId: src_id|bb788691-b4ea266b-708c8a65-11ae8197_0
2025-10-09T21:52:05.574211Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd44379-96496de3-ded39985-26ca8357_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bb788691-b4ea266b-708c8a65-11ae8197_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:05.576535Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb788691-b4ea266b-708c8a65-11ae8197_0] Write session: try to update token
2025-10-09T21:52:05.576549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb788691-b4ea266b-708c8a65-11ae8197_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:05.585111Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:05.585136Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:05.585143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343124194946399:9217]
2025-10-09T21:52:05.585182Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:05.585185Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:05.585378Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:05.585500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:05.585619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:05.585736Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:05.586051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:05.586090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:05.589021Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2343 sessionId: src_id|bb788691-b4ea266b-708c8a65-11ae8197_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:05.591252Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:05.591419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.591442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.591472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:05.591588Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:05.599011Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:05.599026Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:05.599029Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:05.599031Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:05.599041Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343124194946399:9217]
2025-10-09T21:52:05.599046Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343124194946399:9217]
2025-10-09T21:52:05.599050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343124194946399:9217]
2025-10-09T21:52:05.599092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:05.599097Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:05.599104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.599104Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:05.599107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:05.599108Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:05.599256Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:05.650937Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343124556798754:7496]
2025-10-09T21:52:05.651162Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 2342, pipe: [1:7559343124556798754:7496]
2025-10-09T21:52:05.745284Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:05.751727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.751753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.751887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:05.751940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:05.751999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:05.752059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:05.752099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:05.752110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:05.752172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:05.752223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:05.752276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:05.752329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:05.752366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:05.752368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:05.752422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:05.752472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:05.752531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:05.752625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:05.752665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:05.752671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:05.752725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:05.752774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:05.752826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:05.752879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:05.752918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:05.752923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:05.753018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:05.753073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:05.753157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:05.753210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:05.753249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:05.753256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:05.753373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:05.753453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:05.753574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:05.753625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:05.753702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:05.753711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:05.753770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:05.753822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:05.753877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:05.753927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:05.753980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:05.753985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:05.754054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:05.754101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:05.754157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:05.754218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:05.754258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:05.754266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:05.754326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:05.754374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:05.754435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:05.754496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:05.754535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:05.754541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:05.754593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:05.754641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:05.754688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:05.754737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:05.754917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:05.754925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:05.754965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343124556798754:7496], now have 1 active actors on pipe
2025-10-09T21:52:05.755186Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343124556798754:7496]
2025-10-09T21:52:05.755191Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:05.755195Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:05.755219Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:05.755227Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2343 reason: error at write: Disconnected sessionId: src_id|bb788691-b4ea266b-708c8a65-11ae8197_0
2025-10-09T21:52:05.755407Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2343 sessionId: src_id|bb788691-b4ea266b-708c8a65-11ae8197_0 is DEAD
2025-10-09T21:52:05.755579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb788691-b4ea266b-708c8a65-11ae8197_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:05.755591Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb788691-b4ea266b-708c8a65-11ae8197_0] Write session will restart in 0.006332s
2025-10-09T21:52:05.755620Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb788691-b4ea266b-708c8a65-11ae8197_0] Write session: Do CDS request
2025-10-09T21:52:05.755625Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb788691-b4ea266b-708c8a65-11ae8197_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:05.755679Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:05.763426Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bb788691-b4ea266b-708c8a65-11ae8197_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:05.763960Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:05.763972Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2344
2025-10-09T21:52:05.764091Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2344 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:05.764123Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2344 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:05.764133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2344 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:05.764136Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:05.764601Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2344 sessionId:  describe result for acl check
2025-10-09T21:52:05.764634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:05.764636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:05.764637Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:05.764642Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343124556798851:6831] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:05.765176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343124556798851:6831] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:05.767131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343124556798851:6831] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:05.767154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:05.768648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:05.768808Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:05.768818Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:05.768830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:05.768877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:05.768914Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343124194946579:9212]
2025-10-09T21:52:05.769146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343124556798851:6831] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:05.769159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343124556798851:6831] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:05.769163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343124556798851:6831] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:05.769170Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2344 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:05.769518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:05.769816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:05.769856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:05.770179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:05.770228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.770234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.770235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.770236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.770237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.770238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.770240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.770241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.770242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.770247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.770257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.770262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.770266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.770269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.770273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.770276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.770279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.770287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.770293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.770301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.770307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.770318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.770327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.770332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.770341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.770346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.770355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.770360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.770365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.770374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.770383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.770388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.770396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.770401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.770410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.770415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.770448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:05.776611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:05.777701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:05.777716Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:05.777718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:05.777722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:05.777727Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2343 [2:7559343124194946579:9212]
2025-10-09T21:52:05.777742Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:05.777760Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:05.777762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:05.777765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:05.777768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:05.777784Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:05.777795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:05.777802Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:05.777853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.777860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.777862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.777865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:05.777928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:05.777935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.777947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.777959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.777977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:05.778001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343124556798757:2432], now have 1 active actors on pipe
2025-10-09T21:52:05.778009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343124556798928:6831], now have 1 active actors on pipe
2025-10-09T21:52:05.778017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343124194946574:9212]
2025-10-09T21:52:05.778019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343124194946574:9212]
2025-10-09T21:52:05.778021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343124194946574:9212]
2025-10-09T21:52:05.778049Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:05.778058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:05.778062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:05.778064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:05.778067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:05.778082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:05.778090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:05.778113Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343124194946574:9212]
2025-10-09T21:52:05.778090Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2343
2025-10-09T21:52:05.778120Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:05.778091Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2343
2025-10-09T21:52:05.778122Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:05.778151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.778160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.778198Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5405d616-3eb12f71-6546efe6-cd5587c6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:05.778209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:05.778213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:05.778215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:05.778227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:05.778240Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:05.778255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:05.778432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.778442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.778456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:05.778541Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2344 partition: 0 MaxSeqNo: 25 sessionId: src_id|5405d616-3eb12f71-6546efe6-cd5587c6_0
2025-10-09T21:52:05.778847Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bb788691-b4ea266b-708c8a65-11ae8197_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5405d616-3eb12f71-6546efe6-cd5587c6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:05.781265Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5405d616-3eb12f71-6546efe6-cd5587c6_0] Write session: try to update token
2025-10-09T21:52:05.781279Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5405d616-3eb12f71-6546efe6-cd5587c6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:05.792692Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:05.792715Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:05.792717Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:05.792719Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:05.792729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343124194946574:9212]
2025-10-09T21:52:05.792735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343124194946574:9212]
2025-10-09T21:52:05.792738Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343124194946574:9212]
2025-10-09T21:52:05.792781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:05.792787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.792789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:05.792797Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:05.792803Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:05.792807Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:05.792914Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:05.793498Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2344 sessionId: src_id|5405d616-3eb12f71-6546efe6-cd5587c6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:05.795753Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:05.855219Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343124556798943:7496]
2025-10-09T21:52:05.855468Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 2343, pipe: [1:7559343124556798943:7496]
2025-10-09T21:52:05.935964Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:05.936410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.936430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.936474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343124556798943:7496], now have 1 active actors on pipe
2025-10-09T21:52:05.936674Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:05.936702Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343124556798943:7496]
2025-10-09T21:52:05.936707Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:05.936712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:05.936730Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:05.936776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2344 reason: error at write: Disconnected sessionId: src_id|5405d616-3eb12f71-6546efe6-cd5587c6_0
2025-10-09T21:52:05.936941Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2344 sessionId: src_id|5405d616-3eb12f71-6546efe6-cd5587c6_0 is DEAD
2025-10-09T21:52:05.937074Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5405d616-3eb12f71-6546efe6-cd5587c6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:05.937084Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5405d616-3eb12f71-6546efe6-cd5587c6_0] Write session will restart in 0.009789s
2025-10-09T21:52:05.937104Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5405d616-3eb12f71-6546efe6-cd5587c6_0] Write session: Do CDS request
2025-10-09T21:52:05.937109Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5405d616-3eb12f71-6546efe6-cd5587c6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:05.947482Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5405d616-3eb12f71-6546efe6-cd5587c6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:05.947577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:05.947598Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2345
2025-10-09T21:52:05.947715Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2345 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:05.947748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2345 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:05.947768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2345 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:05.947775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:05.948077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:05.948195Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2345 sessionId:  describe result for acl check
2025-10-09T21:52:05.948231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:05.948238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:05.948239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:05.948246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343124556799084:6814] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:05.948848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343124556799084:6814] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:05.949805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:05.949956Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:05.949964Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:05.949972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:05.950020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:05.950063Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343124194946758:9207]
2025-10-09T21:52:05.950556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:05.950654Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343124556799084:6814] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:05.950942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:05.950997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:05.951322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:05.951386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.951391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.951392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.951392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.951393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.951394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.951394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.951395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.951395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.951399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.951405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.951407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.951410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.951412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.951414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.951416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.951418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.951420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.951426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.951433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:05.951439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.951444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:05.951448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.951452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:05.951456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.951460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:05.951464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.951469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:05.951473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.951478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:05.951484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.951488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.951492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.951497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.951502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.951507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.951536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:05.957021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343124556799084:6814] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:05.957034Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343124556799084:6814] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:05.957038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343124556799084:6814] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:05.957048Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2345 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:05.957740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:05.958754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:05.958766Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:05.958768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:05.958772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:05.958776Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2344 [2:7559343124194946758:9207]
2025-10-09T21:52:05.958785Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:05.958798Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:05.958800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:05.958802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:05.958803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:05.958815Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:05.958819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:05.958825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:05.958865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.958874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.958875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.958877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:05.958926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:05.958928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.958935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:05.958941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:05.958950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:05.958965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343124194946752:9207]
2025-10-09T21:52:05.958971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343124194946752:9207]
2025-10-09T21:52:05.958974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343124194946752:9207]
2025-10-09T21:52:05.958980Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:05.958989Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:05.958991Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:05.958999Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:05.959000Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:05.959001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:05.959002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:05.959010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343124556798947:2432], now have 1 active actors on pipe
2025-10-09T21:52:05.959012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343124556799124:6814], now have 1 active actors on pipe
2025-10-09T21:52:05.959030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343124194946752:9207]
2025-10-09T21:52:05.959041Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:05.959048Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:05.959081Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2344
2025-10-09T21:52:05.959090Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2344
2025-10-09T21:52:05.959145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.959154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.959175Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|953b580a-fe781dd0-fefb66d8-f80a0208_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:05.959184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:05.959187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:05.959189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:05.959196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:05.959208Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:05.959219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:05.959332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.959337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.959346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:05.959408Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2345 partition: 0 MaxSeqNo: 25 sessionId: src_id|953b580a-fe781dd0-fefb66d8-f80a0208_0
2025-10-09T21:52:05.959693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5405d616-3eb12f71-6546efe6-cd5587c6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|953b580a-fe781dd0-fefb66d8-f80a0208_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:05.962179Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|953b580a-fe781dd0-fefb66d8-f80a0208_0] Write session: try to update token
2025-10-09T21:52:05.962194Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|953b580a-fe781dd0-fefb66d8-f80a0208_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:05.974129Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2345 sessionId: src_id|953b580a-fe781dd0-fefb66d8-f80a0208_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:05.976435Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:05.976643Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:05.976666Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:05.976675Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343124194946752:9207]
2025-10-09T21:52:05.976724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:05.976725Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:05.976738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:05.976767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:05.976941Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:05.977108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:05.977259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:05.977416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:05.977626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:05.977690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:05.977723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:05.977842Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:05.983741Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:05.983762Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:05.983765Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:05.983767Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:05.983778Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343124194946752:9207]
2025-10-09T21:52:05.983782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343124194946752:9207]
2025-10-09T21:52:05.983786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343124194946752:9207]
2025-10-09T21:52:05.983827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:05.983832Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:05.983838Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:05.983840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:05.983843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:05.983843Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:05.983981Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:06.036963Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343128851766428:7496]
2025-10-09T21:52:06.037261Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 2344, pipe: [1:7559343128851766428:7496]
2025-10-09T21:52:06.133925Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:06.141109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.141136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.141279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.141365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.141425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.141486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.141531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.141539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.141810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.141869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.141927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.141990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.142051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.142062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.142135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.142190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.142256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.142319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.142362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.142368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.142431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.142490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.142549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.142637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.142679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.142685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.142772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.142964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.143024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.143082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.143125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.143130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.143747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.143919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.143977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.144036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.144083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.144089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.144145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.144199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.144468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.144592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.144631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.144638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.144722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.144776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.144833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.144887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.144929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.144936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.145076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.145280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.145506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.145568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.145625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.145630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.145692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.145746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.145802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.145859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.145899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.145905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.145953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343128851766428:7496], now have 1 active actors on pipe
2025-10-09T21:52:06.146173Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:06.146190Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343128851766428:7496]
2025-10-09T21:52:06.146221Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:06.146223Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:06.146243Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2345 reason: error at write: Disconnected sessionId: src_id|953b580a-fe781dd0-fefb66d8-f80a0208_0
2025-10-09T21:52:06.146270Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:06.146425Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2345 sessionId: src_id|953b580a-fe781dd0-fefb66d8-f80a0208_0 is DEAD
2025-10-09T21:52:06.146549Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|953b580a-fe781dd0-fefb66d8-f80a0208_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:06.146559Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|953b580a-fe781dd0-fefb66d8-f80a0208_0] Write session will restart in 0.007223s
2025-10-09T21:52:06.146577Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|953b580a-fe781dd0-fefb66d8-f80a0208_0] Write session: Do CDS request
2025-10-09T21:52:06.146582Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|953b580a-fe781dd0-fefb66d8-f80a0208_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:06.154490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|953b580a-fe781dd0-fefb66d8-f80a0208_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:06.154588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:06.154603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2346
2025-10-09T21:52:06.154714Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2346 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:06.154747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2346 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:06.154753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2346 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:06.154757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:06.155274Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2346 sessionId:  describe result for acl check
2025-10-09T21:52:06.155313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:06.155316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:06.155317Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:06.155324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343128851766536:6783] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:06.155963Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343128851766536:6783] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:06.158250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:06.158347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343128851766536:6783] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:06.160091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:06.160281Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:06.160294Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:06.160305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:06.160355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:06.160402Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343128489914268:9194]
2025-10-09T21:52:06.160558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343128851766536:6783] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:06.160569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343128851766536:6783] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:06.160572Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343128851766536:6783] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:06.160579Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2346 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:06.161005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:06.161282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:06.161344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:06.161674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:06.161724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.161730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.161732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.161733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.161734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.161735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.161737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.161738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.161739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.161744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.161755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.161760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.161764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.161768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.161772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.161776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.161780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.161789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.161796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.161810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.161816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.161822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.161826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.161832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.161836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.161842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.161846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.161851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.161856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.161862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.161867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.161872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.161878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.161886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.161892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.161900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.161930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:06.168556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:06.169893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:06.169909Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:06.169911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:06.169914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:06.169920Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2345 [2:7559343128489914268:9194]
2025-10-09T21:52:06.169932Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:06.169948Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:06.169951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:06.169954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:06.169956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:06.169969Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:06.169973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:06.169979Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:06.170060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.170069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.170071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.170074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:06.170136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:06.170145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.170154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.170160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.170170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:06.170188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343128851766626:6783], now have 1 active actors on pipe
2025-10-09T21:52:06.170195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343128489914263:9194]
2025-10-09T21:52:06.170196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343128489914263:9194]
2025-10-09T21:52:06.170198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343128489914263:9194]
2025-10-09T21:52:06.170208Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:06.170214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343128851766455:2432], now have 1 active actors on pipe
2025-10-09T21:52:06.170221Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:06.170230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:06.170232Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:06.170236Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:06.170240Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:06.170243Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:06.170265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343128489914263:9194]
2025-10-09T21:52:06.170267Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:06.170270Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:06.170293Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2345
2025-10-09T21:52:06.170299Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2345
2025-10-09T21:52:06.170396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.170410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.170437Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ecec229f-f3065fe-8ce6dc0b-8b689785_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:06.170448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:06.170451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:06.170452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:06.170460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:06.170469Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:06.170481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:06.170636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.170645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.170657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:06.170749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2346 partition: 0 MaxSeqNo: 25 sessionId: src_id|ecec229f-f3065fe-8ce6dc0b-8b689785_0
2025-10-09T21:52:06.171106Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|953b580a-fe781dd0-fefb66d8-f80a0208_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ecec229f-f3065fe-8ce6dc0b-8b689785_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:06.173601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecec229f-f3065fe-8ce6dc0b-8b689785_0] Write session: try to update token
2025-10-09T21:52:06.173616Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecec229f-f3065fe-8ce6dc0b-8b689785_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:06.182283Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:06.182303Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:06.182312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343128489914263:9194]
2025-10-09T21:52:06.182354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:06.182362Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:06.182640Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:06.182784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:06.182937Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:06.183089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:06.183275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:06.183321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:06.186425Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2346 sessionId: src_id|ecec229f-f3065fe-8ce6dc0b-8b689785_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:06.188718Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:06.188927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.188949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.188985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:06.189077Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:06.195291Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:06.195312Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:06.195314Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:06.195316Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:06.195327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343128489914263:9194]
2025-10-09T21:52:06.195335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343128489914263:9194]
2025-10-09T21:52:06.195339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343128489914263:9194]
2025-10-09T21:52:06.195375Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:06.195377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:06.195384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.195385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:06.195389Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:06.195393Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:06.195535Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:06.246407Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343128851766641:7496]
2025-10-09T21:52:06.246640Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 2345, pipe: [1:7559343128851766641:7496]
2025-10-09T21:52:06.346610Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:06.353401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.353427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.353568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.353621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.353677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.353733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.353988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.353998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.354073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.354129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.354184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.354265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.354310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.354318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.354379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.354449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.354505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.354558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.354596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.354601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.354659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.354715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.354769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.354823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.354861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.354867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.354961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.355015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.355066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.355118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.355204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.355210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.355264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.355316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.355371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.355423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.355461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.355467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.355523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.355573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.355624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.355680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.355733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.355741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.355795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.355869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.355924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.355975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.356013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.356020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.356074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.356122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.356173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.356228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.356279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.356284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.356337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.356385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.356438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.356489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.356530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.356534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.356577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343128851766641:7496], now have 1 active actors on pipe
2025-10-09T21:52:06.356846Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:06.356855Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343128851766641:7496]
2025-10-09T21:52:06.356898Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:06.356901Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:06.356930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2346 reason: error at write: Disconnected sessionId: src_id|ecec229f-f3065fe-8ce6dc0b-8b689785_0
2025-10-09T21:52:06.357127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2346 sessionId: src_id|ecec229f-f3065fe-8ce6dc0b-8b689785_0 is DEAD
2025-10-09T21:52:06.357283Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecec229f-f3065fe-8ce6dc0b-8b689785_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:06.357309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecec229f-f3065fe-8ce6dc0b-8b689785_0] Write session will restart in 0.007783s
2025-10-09T21:52:06.357335Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecec229f-f3065fe-8ce6dc0b-8b689785_0] Write session: Do CDS request
2025-10-09T21:52:06.357343Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecec229f-f3065fe-8ce6dc0b-8b689785_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:06.357369Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:06.366425Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ecec229f-f3065fe-8ce6dc0b-8b689785_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:06.366509Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:06.366524Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2347
2025-10-09T21:52:06.366659Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2347 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:06.366694Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2347 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:06.366707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2347 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:06.366711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:06.367125Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2347 sessionId:  describe result for acl check
2025-10-09T21:52:06.367159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:06.367163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:06.367164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:06.367172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343128851766775:6775] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:06.367739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343128851766775:6775] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:06.369087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:06.370100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343128851766775:6775] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:06.370710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:06.370923Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:06.370934Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:06.370947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:06.371003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:06.371041Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343128489914446:9186]
2025-10-09T21:52:06.371628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:06.371896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:06.371938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:06.372254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:06.372303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.372309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.372310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.372312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.372313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.372314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.372315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.372316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.372318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.372323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.372334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.372338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.372339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343128851766775:6775] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:06.372342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.372346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.372352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343128851766775:6775] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:06.372350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.372356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343128851766775:6775] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:06.372354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.372358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.372364Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2347 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:06.372362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.372368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.372376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.372382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.372388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.372393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.372398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.372402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.372407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.372414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.372419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.372426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.372432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.372442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.372447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.372457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.372463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.372472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.372477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.372511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:06.378669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:06.379763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:06.379779Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:06.379782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:06.379786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:06.379792Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2346 [2:7559343128489914446:9186]
2025-10-09T21:52:06.379807Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:06.379843Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:06.379846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:06.379850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:06.379852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:06.379869Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:06.379874Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:06.379881Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:06.379941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.379947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.379949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.379952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:06.380047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:06.380056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.380068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.380086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.380108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:06.380134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343128851766650:2432], now have 1 active actors on pipe
2025-10-09T21:52:06.380142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343128851766827:6775], now have 1 active actors on pipe
2025-10-09T21:52:06.380149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343128489914441:9186]
2025-10-09T21:52:06.380155Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343128489914441:9186]
2025-10-09T21:52:06.380134Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2346
2025-10-09T21:52:06.380157Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343128489914441:9186]
2025-10-09T21:52:06.380144Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2346
2025-10-09T21:52:06.380166Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:06.380180Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:06.380184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:06.380186Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:06.380189Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:06.380195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:06.380201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:06.380245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.380255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.380268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343128489914441:9186]
2025-10-09T21:52:06.380279Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:06.380292Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:06.380317Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a1b83084-497fa11e-361dcb9d-3fba8ad3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:06.380331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:06.380336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:06.380338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:06.380349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:06.380359Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:06.380381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:06.380564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.380573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.380589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:06.380689Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2347 partition: 0 MaxSeqNo: 25 sessionId: src_id|a1b83084-497fa11e-361dcb9d-3fba8ad3_0
2025-10-09T21:52:06.381040Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ecec229f-f3065fe-8ce6dc0b-8b689785_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a1b83084-497fa11e-361dcb9d-3fba8ad3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:06.383574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a1b83084-497fa11e-361dcb9d-3fba8ad3_0] Write session: try to update token
2025-10-09T21:52:06.383593Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a1b83084-497fa11e-361dcb9d-3fba8ad3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:06.394316Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:06.394336Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:06.394338Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:06.394342Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:06.394350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343128489914441:9186]
2025-10-09T21:52:06.394353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343128489914441:9186]
2025-10-09T21:52:06.394356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343128489914441:9186]
2025-10-09T21:52:06.394392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:06.394398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.394399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:06.394405Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:06.394410Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:06.394415Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:06.394524Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:06.395705Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2347 sessionId: src_id|a1b83084-497fa11e-361dcb9d-3fba8ad3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:06.398155Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:06.456851Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343128851766836:7496]
2025-10-09T21:52:06.457136Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 2346, pipe: [1:7559343128851766836:7496]
2025-10-09T21:52:06.536157Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:06.536434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.536448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.536490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343128851766836:7496], now have 1 active actors on pipe
2025-10-09T21:52:06.536660Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:06.536664Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:06.536668Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:06.536718Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343128851766836:7496]
2025-10-09T21:52:06.536734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2347 reason: error at write: Disconnected sessionId: src_id|a1b83084-497fa11e-361dcb9d-3fba8ad3_0
2025-10-09T21:52:06.536740Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:06.536897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2347 sessionId: src_id|a1b83084-497fa11e-361dcb9d-3fba8ad3_0 is DEAD
2025-10-09T21:52:06.537068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1b83084-497fa11e-361dcb9d-3fba8ad3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:06.537082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1b83084-497fa11e-361dcb9d-3fba8ad3_0] Write session will restart in 0.009544s
2025-10-09T21:52:06.537112Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1b83084-497fa11e-361dcb9d-3fba8ad3_0] Write session: Do CDS request
2025-10-09T21:52:06.537119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1b83084-497fa11e-361dcb9d-3fba8ad3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:06.547570Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a1b83084-497fa11e-361dcb9d-3fba8ad3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:06.547629Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:06.547643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2348
2025-10-09T21:52:06.547780Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2348 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:06.547810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2348 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:06.547814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2348 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:06.547817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:06.548000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:06.548230Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2348 sessionId:  describe result for acl check
2025-10-09T21:52:06.548266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:06.548274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:06.548276Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:06.548282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343128851766980:6760] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:06.548851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343128851766980:6760] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:06.549946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:06.550193Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:06.550204Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:06.550216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:06.550274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:06.550336Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343128489914625:9184]
2025-10-09T21:52:06.550709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:06.550896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343128851766980:6760] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:06.550998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:06.551039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:06.551299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:06.551359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.551365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.551366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.551367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.551369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.551370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.551371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.551372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.551373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.551378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.551388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.551392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.551396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.551400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.551404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.551407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.551411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.551414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.551421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.551429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.551440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.551444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.551455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.551460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.551468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.551473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.551478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.551487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.551491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.551496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.551501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.551510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.551514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.551518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.551526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.551531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.551558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:06.556007Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343128851766980:6760] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:06.556019Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343128851766980:6760] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:06.556022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343128851766980:6760] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:06.556030Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2348 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:06.557285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:06.558548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:06.558561Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:06.558562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:06.558565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:06.558570Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2347 [2:7559343128489914625:9184]
2025-10-09T21:52:06.558579Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:06.558592Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:06.558594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:06.558596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:06.558597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:06.558609Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:06.558616Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:06.558621Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:06.558663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.558669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.558671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.558672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:06.558719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:06.558725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.558736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.558746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.558759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:06.558778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343128851767019:6760], now have 1 active actors on pipe
2025-10-09T21:52:06.558787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343128489914619:9184]
2025-10-09T21:52:06.558792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343128489914619:9184]
2025-10-09T21:52:06.558793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343128489914619:9184]
2025-10-09T21:52:06.558800Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:06.558808Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:06.558809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:06.558817Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:06.558818Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:06.558819Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:06.558819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:06.558823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343128851766839:2432], now have 1 active actors on pipe
2025-10-09T21:52:06.558823Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2347
2025-10-09T21:52:06.558838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343128489914619:9184]
2025-10-09T21:52:06.558840Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:06.558855Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2347
2025-10-09T21:52:06.558855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:06.558862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.558864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.558888Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|aacfc299-f4f22061-1d3d123-da340120_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:06.558898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:06.558902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:06.558903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:06.558910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:06.558919Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:06.558930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:06.559027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.559033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.559043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:06.559138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2348 partition: 0 MaxSeqNo: 25 sessionId: src_id|aacfc299-f4f22061-1d3d123-da340120_0
2025-10-09T21:52:06.559451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1b83084-497fa11e-361dcb9d-3fba8ad3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|aacfc299-f4f22061-1d3d123-da340120_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:06.562008Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aacfc299-f4f22061-1d3d123-da340120_0] Write session: try to update token
2025-10-09T21:52:06.562025Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aacfc299-f4f22061-1d3d123-da340120_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:06.570333Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:06.570358Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:06.570370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343128489914619:9184]
2025-10-09T21:52:06.570418Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:06.570422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:06.570866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:06.571034Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:06.571187Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:06.571357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:06.571587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:06.571641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:06.576730Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2348 sessionId: src_id|aacfc299-f4f22061-1d3d123-da340120_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:06.579055Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:06.579264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.579283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.579322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:06.579412Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:06.582947Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:06.582961Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:06.582964Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:06.582966Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:06.582975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343128489914619:9184]
2025-10-09T21:52:06.582979Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343128489914619:9184]
2025-10-09T21:52:06.582983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343128489914619:9184]
2025-10-09T21:52:06.583023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:06.583034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.583039Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:06.583041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:06.583045Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:06.583049Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:06.583174Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:06.637042Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343128851767026:7496]
2025-10-09T21:52:06.637312Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 2347, pipe: [1:7559343128851767026:7496]
2025-10-09T21:52:06.728213Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:06.735204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.735232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.735377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.735432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.735488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.735542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.735587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.735596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.735659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.735727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.735791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.735850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.735889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.735895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.735951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.736138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.736195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.736249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.736289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.736294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.736349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.736409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.736465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.736519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.736558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.736563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.736616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.736673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.736737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.736799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.736844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.736853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.736908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.736967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.737022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.737078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.737120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.737128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.737184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.737258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.737329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.737386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.737434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.737443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.737495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.737550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.737610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.737665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.737970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.737978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.738047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.738102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.738159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.738221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.738260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.738266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.738319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:06.738376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:06.738432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:06.738491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:06.738573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:06.738579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:06.738635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343128851767026:7496], now have 1 active actors on pipe
2025-10-09T21:52:06.738880Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:06.738902Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343128851767026:7496]
2025-10-09T21:52:06.738931Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:06.738936Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:06.738959Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2348 reason: error at write: Disconnected sessionId: src_id|aacfc299-f4f22061-1d3d123-da340120_0
2025-10-09T21:52:06.739114Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2348 sessionId: src_id|aacfc299-f4f22061-1d3d123-da340120_0 is DEAD
2025-10-09T21:52:06.739139Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:06.739276Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aacfc299-f4f22061-1d3d123-da340120_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:06.739289Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aacfc299-f4f22061-1d3d123-da340120_0] Write session will restart in 0.006165s
2025-10-09T21:52:06.739313Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aacfc299-f4f22061-1d3d123-da340120_0] Write session: Do CDS request
2025-10-09T21:52:06.739320Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aacfc299-f4f22061-1d3d123-da340120_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:06.746446Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|aacfc299-f4f22061-1d3d123-da340120_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:06.746546Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:06.746572Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2349
2025-10-09T21:52:06.746826Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2349 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:06.746861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2349 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:06.746872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2349 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:06.746876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:06.747269Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2349 sessionId:  describe result for acl check
2025-10-09T21:52:06.747309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:06.747314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:06.747316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:06.747320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343128851767071:6746] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:06.747768Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343128851767071:6746] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:06.750894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343128851767071:6746] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:06.752263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:06.753965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:06.754155Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:06.754165Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:06.754177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:06.754172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343128851767071:6746] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:06.754188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343128851767071:6746] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:06.754191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343128851767071:6746] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:06.754197Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2349 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:06.754226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:06.754265Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343128489914804:9178]
2025-10-09T21:52:06.754841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:06.755176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:06.755222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:06.755542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:06.755589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.755595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.755596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.755597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.755598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.755600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.755601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.755602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.755603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.755608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.755619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.755624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.755628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.755632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.755636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.755640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.755644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.755648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.755654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.755662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.755672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.755677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.755683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.755693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.755700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.755705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.755714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.755718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.755728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.755733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.755741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.755746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.755754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.755760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.755765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.755775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.755811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:06.761951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:06.763115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:06.763135Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:06.763137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:06.763141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:06.763148Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2348 [2:7559343128489914804:9178]
2025-10-09T21:52:06.763161Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:06.763179Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:06.763182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:06.763185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:06.763188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:06.763202Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:06.763214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:06.763222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:06.763270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.763277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.763280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.763283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:06.763349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:06.763356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.763368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.763382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.763403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:06.763427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343128851767029:2432], now have 1 active actors on pipe
2025-10-09T21:52:06.763435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343128851767206:6746], now have 1 active actors on pipe
2025-10-09T21:52:06.763442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343128489914799:9178]
2025-10-09T21:52:06.763447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343128489914799:9178]
2025-10-09T21:52:06.763449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343128489914799:9178]
2025-10-09T21:52:06.763463Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:06.763444Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2348
2025-10-09T21:52:06.763455Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2348
2025-10-09T21:52:06.763481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:06.763483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:06.763486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:06.763488Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:06.763499Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:06.763502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:06.763542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.763553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.763566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343128489914799:9178]
2025-10-09T21:52:06.763577Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:06.763593Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:06.763606Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e5746e26-f9a567a1-c744b5cf-cd99b2c5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:06.763614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:06.763618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:06.763621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:06.763631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:06.763645Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:06.763669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:06.763805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.763816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.763832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:06.763910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2349 partition: 0 MaxSeqNo: 25 sessionId: src_id|e5746e26-f9a567a1-c744b5cf-cd99b2c5_0
2025-10-09T21:52:06.764244Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|aacfc299-f4f22061-1d3d123-da340120_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e5746e26-f9a567a1-c744b5cf-cd99b2c5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:06.766521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5746e26-f9a567a1-c744b5cf-cd99b2c5_0] Write session: try to update token
2025-10-09T21:52:06.766540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5746e26-f9a567a1-c744b5cf-cd99b2c5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:06.777760Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:06.777780Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:06.777782Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:06.777784Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:06.777792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343128489914799:9178]
2025-10-09T21:52:06.777796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343128489914799:9178]
2025-10-09T21:52:06.777799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343128489914799:9178]
2025-10-09T21:52:06.777834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:06.777840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.777842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:06.777865Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:06.777881Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:06.777890Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:06.777959Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:06.778847Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2349 sessionId: src_id|e5746e26-f9a567a1-c744b5cf-cd99b2c5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:06.781154Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:06.839284Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343128851767221:7496]
2025-10-09T21:52:06.839543Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 2348, pipe: [1:7559343128851767221:7496]
2025-10-09T21:52:06.920752Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:06.921127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.921143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.921186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343128851767221:7496], now have 1 active actors on pipe
2025-10-09T21:52:06.921410Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:06.921434Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:06.921436Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343128851767221:7496]
2025-10-09T21:52:06.921451Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:06.921457Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:06.921620Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2349 reason: error at write: Disconnected sessionId: src_id|e5746e26-f9a567a1-c744b5cf-cd99b2c5_0
2025-10-09T21:52:06.921789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2349 sessionId: src_id|e5746e26-f9a567a1-c744b5cf-cd99b2c5_0 is DEAD
2025-10-09T21:52:06.921952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5746e26-f9a567a1-c744b5cf-cd99b2c5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:06.921961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5746e26-f9a567a1-c744b5cf-cd99b2c5_0] Write session will restart in 0.007643s
2025-10-09T21:52:06.921981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5746e26-f9a567a1-c744b5cf-cd99b2c5_0] Write session: Do CDS request
2025-10-09T21:52:06.921986Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5746e26-f9a567a1-c744b5cf-cd99b2c5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:06.930450Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e5746e26-f9a567a1-c744b5cf-cd99b2c5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:06.930490Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:06.930502Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2350
2025-10-09T21:52:06.930617Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2350 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:06.930648Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2350 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:06.930656Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2350 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:06.930659Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:06.931039Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2350 sessionId:  describe result for acl check
2025-10-09T21:52:06.931065Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:06.931069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:06.931070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:06.931075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343128851767360:6737] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:06.931520Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343128851767360:6737] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:06.932530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:06.933359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343128851767360:6737] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:06.934193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:06.934355Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:06.934366Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:06.934377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:06.934425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:06.934462Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343128489914988:9167]
2025-10-09T21:52:06.935044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:06.935339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:06.935383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:06.935450Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343128851767360:6737] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:06.935474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343128851767360:6737] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:06.935477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343128851767360:6737] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:06.935482Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2350 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:06.935680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:06.935736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.935742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.935743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.935744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.935745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.935747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.935748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.935749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.935750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.935755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.935764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.935768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.935772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.935775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.935779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.935783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.935786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.935789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.935795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.935803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:06.935809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.935815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:06.935819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.935824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:06.935828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.935839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:06.935843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.935849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:06.935854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.935860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:06.935870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.935875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.935882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.935886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.935891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.935896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.935927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:06.941987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:06.943176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:06.943190Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:06.943193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:06.943196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:06.943202Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2349 [2:7559343128489914988:9167]
2025-10-09T21:52:06.943215Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:06.943235Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:06.943238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:06.943241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:06.943243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:06.943258Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:06.943268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:06.943279Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:06.943337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.943343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.943345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.943348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:06.943409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:06.943416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.943429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:06.943441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:06.943462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:06.943489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343128851767404:6737], now have 1 active actors on pipe
2025-10-09T21:52:06.943497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343128851767225:2432], now have 1 active actors on pipe
2025-10-09T21:52:06.943504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343128489914982:9167]
2025-10-09T21:52:06.943506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343128489914982:9167]
2025-10-09T21:52:06.943508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343128489914982:9167]
2025-10-09T21:52:06.943520Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:06.943538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:06.943546Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:06.943549Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:06.943552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:06.943555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:06.943558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:06.943551Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2349
2025-10-09T21:52:06.943570Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2349
2025-10-09T21:52:06.943599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343128489914982:9167]
2025-10-09T21:52:06.943611Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:06.943623Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:06.943632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.943636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.943676Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e8cfda91-f271c418-c05f08e8-7adcfc48_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:06.943690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:06.943694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:06.943696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:06.943706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:06.943716Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:06.943732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:06.943859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:06.943867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:06.943880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:06.943956Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2350 partition: 0 MaxSeqNo: 25 sessionId: src_id|e8cfda91-f271c418-c05f08e8-7adcfc48_0
2025-10-09T21:52:06.944239Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e5746e26-f9a567a1-c744b5cf-cd99b2c5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e8cfda91-f271c418-c05f08e8-7adcfc48_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:06.946606Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8cfda91-f271c418-c05f08e8-7adcfc48_0] Write session: try to update token
2025-10-09T21:52:06.946621Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8cfda91-f271c418-c05f08e8-7adcfc48_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:06.957249Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:06.957270Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:06.957273Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:06.957275Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:06.957285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343128489914982:9167]
2025-10-09T21:52:06.957289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343128489914982:9167]
2025-10-09T21:52:06.957301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343128489914982:9167]
2025-10-09T21:52:06.957342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:06.957354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:06.957357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:06.957357Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:06.957363Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:06.957367Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:06.957478Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:06.959239Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2350 sessionId: src_id|e8cfda91-f271c418-c05f08e8-7adcfc48_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:06.961810Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:07.021560Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343133146734711:7496]
2025-10-09T21:52:07.021821Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 2349, pipe: [1:7559343133146734711:7496]
2025-10-09T21:52:07.107159Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:07.107464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:07.107478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:07.107517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343133146734711:7496], now have 1 active actors on pipe
2025-10-09T21:52:07.107703Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:07.107782Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:07.107792Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343133146734711:7496]
2025-10-09T21:52:07.107812Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:07.107883Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2350 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e8cfda91-f271c418-c05f08e8-7adcfc48_0
2025-10-09T21:52:07.108070Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2350 sessionId: src_id|e8cfda91-f271c418-c05f08e8-7adcfc48_0 is DEAD
2025-10-09T21:52:07.108231Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8cfda91-f271c418-c05f08e8-7adcfc48_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:07.108240Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8cfda91-f271c418-c05f08e8-7adcfc48_0] Write session will restart in 0.008392s
2025-10-09T21:52:07.108269Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8cfda91-f271c418-c05f08e8-7adcfc48_0] Write session: Do CDS request
2025-10-09T21:52:07.108275Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8cfda91-f271c418-c05f08e8-7adcfc48_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:07.117548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e8cfda91-f271c418-c05f08e8-7adcfc48_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:07.117664Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:07.117676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2351
2025-10-09T21:52:07.117860Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2351 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:07.117907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2351 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:07.117918Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2351 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:07.117922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:07.118347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2351 sessionId:  describe result for acl check
2025-10-09T21:52:07.118380Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:07.118386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:07.118388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:07.118394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343133146734854:6722] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:07.118940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343133146734854:6722] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:07.119248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:07.120553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343133146734854:6722] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:07.121085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:07.121277Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:07.121287Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:07.121311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:07.121367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:07.121410Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343132784882465:9175]
2025-10-09T21:52:07.121800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:07.122138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:07.122186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:07.122498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:07.122542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.122547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.122548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.122549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.122549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.122550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.122550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.122551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.122552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.122555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.122562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.122564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.122568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.122572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.122576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.122580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.122583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.122586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.122592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.122600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.122607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.122613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.122617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.122622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.122626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.122631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.122636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.122641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.122645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.122651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.122655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.122660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.122664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.122669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.122680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.122685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.122714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:07.122964Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343133146734854:6722] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:07.122975Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343133146734854:6722] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:07.122977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343133146734854:6722] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:07.122983Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2351 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:07.128808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:07.130074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:07.130088Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:07.130090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:07.130095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:07.130101Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2350 [2:7559343132784882465:9175]
2025-10-09T21:52:07.130113Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:07.130129Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:07.130136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:07.130140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:07.130142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:07.130159Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:07.130167Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:07.130174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:07.130231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.130238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.130240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.130243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:07.130306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:07.130314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.130327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.130341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.130365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:07.130390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343133146734896:6722], now have 1 active actors on pipe
2025-10-09T21:52:07.130398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343133146734713:2432], now have 1 active actors on pipe
2025-10-09T21:52:07.130405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343132784882459:9175]
2025-10-09T21:52:07.130408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343132784882459:9175]
2025-10-09T21:52:07.130410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343132784882459:9175]
2025-10-09T21:52:07.130424Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:07.130409Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2350
2025-10-09T21:52:07.130410Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2350
2025-10-09T21:52:07.130442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:07.130448Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:07.130450Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:07.130455Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:07.130461Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:07.130464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:07.130504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:07.130517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:07.130534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343132784882459:9175]
2025-10-09T21:52:07.130542Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:07.130552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:07.130571Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d7e4ae49-c769dfc5-8edb92b4-dedeedc8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:07.130583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:07.130587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:07.130594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:07.130605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:07.130614Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:07.130642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:07.130795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:07.130807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:07.130823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:07.130912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2351 partition: 0 MaxSeqNo: 25 sessionId: src_id|d7e4ae49-c769dfc5-8edb92b4-dedeedc8_0
2025-10-09T21:52:07.131249Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e8cfda91-f271c418-c05f08e8-7adcfc48_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d7e4ae49-c769dfc5-8edb92b4-dedeedc8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:07.133827Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7e4ae49-c769dfc5-8edb92b4-dedeedc8_0] Write session: try to update token
2025-10-09T21:52:07.133843Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7e4ae49-c769dfc5-8edb92b4-dedeedc8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:07.144473Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:07.144489Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:07.144491Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:07.144492Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:07.144500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343132784882459:9175]
2025-10-09T21:52:07.144504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343132784882459:9175]
2025-10-09T21:52:07.144506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343132784882459:9175]
2025-10-09T21:52:07.144543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:07.144543Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:07.144549Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:07.144550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.144552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:07.144565Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:07.144679Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:07.147073Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2351 sessionId: src_id|d7e4ae49-c769dfc5-8edb92b4-dedeedc8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:07.149326Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:07.207934Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343133146734928:7496]
2025-10-09T21:52:07.208179Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 2350, pipe: [1:7559343133146734928:7496]
2025-10-09T21:52:07.290156Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:07.290366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:07.290379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:07.290420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343133146734928:7496], now have 1 active actors on pipe
2025-10-09T21:52:07.290588Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:07.290591Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:07.290603Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:07.290618Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:07.290666Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2351 reason: error at write: Disconnected sessionId: src_id|d7e4ae49-c769dfc5-8edb92b4-dedeedc8_0
2025-10-09T21:52:07.290670Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343133146734928:7496]
2025-10-09T21:52:07.290864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2351 sessionId: src_id|d7e4ae49-c769dfc5-8edb92b4-dedeedc8_0 is DEAD
2025-10-09T21:52:07.291023Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7e4ae49-c769dfc5-8edb92b4-dedeedc8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:07.291030Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7e4ae49-c769dfc5-8edb92b4-dedeedc8_0] Write session will restart in 0.008285s
2025-10-09T21:52:07.291051Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7e4ae49-c769dfc5-8edb92b4-dedeedc8_0] Write session: Do CDS request
2025-10-09T21:52:07.291055Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7e4ae49-c769dfc5-8edb92b4-dedeedc8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:07.300517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7e4ae49-c769dfc5-8edb92b4-dedeedc8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:07.300582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:07.300593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2352
2025-10-09T21:52:07.300735Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2352 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:07.300775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2352 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:07.300782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2352 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:07.300786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:07.301172Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2352 sessionId:  describe result for acl check
2025-10-09T21:52:07.301204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:07.301211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:07.301213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:07.301219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343133146735075:6687] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:07.301733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343133146735075:6687] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:07.302370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:07.303451Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343133146735075:6687] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:07.303902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:07.304071Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:07.304081Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:07.304093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:07.304139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:07.304179Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343132784882687:9155]
2025-10-09T21:52:07.304724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:07.304996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:07.305041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:07.305353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:07.305404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.305409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.305410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.305412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.305413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.305414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.305415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.305416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.305417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.305421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.305431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.305440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.305443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.305451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.305456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.305459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.305463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.305467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.305480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.305488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.305494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.305499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.305503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.305509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.305518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.305525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.305530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.305536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.305544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.305554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.305562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.305572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.305576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.305581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.305587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.305596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.305630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:07.306011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343133146735075:6687] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:07.306021Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343133146735075:6687] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:07.306023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343133146735075:6687] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:07.306038Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2352 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:07.311617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:07.313025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:07.313042Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:07.313044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:07.313049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:07.313055Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2351 [2:7559343132784882687:9155]
2025-10-09T21:52:07.313070Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:07.313086Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:07.313094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:07.313097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:07.313100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:07.313116Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:07.313127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:07.313136Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:07.313195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.313201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.313203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.313206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:07.313269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:07.313276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.313289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.313316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.313336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:07.313362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343133146735116:6687], now have 1 active actors on pipe
2025-10-09T21:52:07.313370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343133146734933:2432], now have 1 active actors on pipe
2025-10-09T21:52:07.313377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343132784882682:9155]
2025-10-09T21:52:07.313383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343132784882682:9155]
2025-10-09T21:52:07.313385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343132784882682:9155]
2025-10-09T21:52:07.313394Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:07.313380Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2351
2025-10-09T21:52:07.313418Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2351
2025-10-09T21:52:07.313414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:07.313414Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:07.313418Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:07.313419Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:07.313435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:07.313438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:07.313475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:07.313480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:07.313495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343132784882682:9155]
2025-10-09T21:52:07.313508Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:07.313521Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:07.313542Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a35cf186-23c566b3-163e8d84-a37acd13_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:07.313553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:07.313557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:07.313559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:07.313570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:07.313583Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:07.313609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:07.313766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:07.313775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:07.313793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:07.313872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2352 partition: 0 MaxSeqNo: 25 sessionId: src_id|a35cf186-23c566b3-163e8d84-a37acd13_0
2025-10-09T21:52:07.314169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7e4ae49-c769dfc5-8edb92b4-dedeedc8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a35cf186-23c566b3-163e8d84-a37acd13_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:07.316498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a35cf186-23c566b3-163e8d84-a37acd13_0] Write session: try to update token
2025-10-09T21:52:07.316513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a35cf186-23c566b3-163e8d84-a37acd13_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:07.327121Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:07.327141Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:07.327142Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:07.327143Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:07.327152Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343132784882682:9155]
2025-10-09T21:52:07.327158Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343132784882682:9155]
2025-10-09T21:52:07.327160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343132784882682:9155]
2025-10-09T21:52:07.327197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:07.327203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.327204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:07.327205Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:07.327210Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:07.327213Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:07.327354Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:07.329310Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2352 sessionId: src_id|a35cf186-23c566b3-163e8d84-a37acd13_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:07.331391Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:07.390711Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343133146735129:7496]
2025-10-09T21:52:07.390958Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 2351, pipe: [1:7559343133146735129:7496]
2025-10-09T21:52:07.468418Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:07.468672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:07.468695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:07.468763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343133146735129:7496], now have 1 active actors on pipe
2025-10-09T21:52:07.468973Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343133146735129:7496]
2025-10-09T21:52:07.468997Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:07.469000Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:07.469004Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:07.469006Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:07.469074Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2352 reason: error at write: Disconnected sessionId: src_id|a35cf186-23c566b3-163e8d84-a37acd13_0
2025-10-09T21:52:07.469259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2352 sessionId: src_id|a35cf186-23c566b3-163e8d84-a37acd13_0 is DEAD
2025-10-09T21:52:07.469425Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a35cf186-23c566b3-163e8d84-a37acd13_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:07.469437Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a35cf186-23c566b3-163e8d84-a37acd13_0] Write session will restart in 0.005410s
2025-10-09T21:52:07.469461Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a35cf186-23c566b3-163e8d84-a37acd13_0] Write session: Do CDS request
2025-10-09T21:52:07.469466Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a35cf186-23c566b3-163e8d84-a37acd13_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:07.475444Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a35cf186-23c566b3-163e8d84-a37acd13_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:07.475529Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:07.475541Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2353
2025-10-09T21:52:07.475659Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2353 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:07.475688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2353 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:07.475699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2353 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:07.475703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:07.476142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2353 sessionId:  describe result for acl check
2025-10-09T21:52:07.476172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:07.476179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:07.476182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:07.476188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343133146735174:6676] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:07.476733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343133146735174:6676] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:07.478874Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343133146735174:6676] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:07.480304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:07.482136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:07.482301Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:07.482312Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:07.482324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:07.482378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:07.482413Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343132784882869:9143]
2025-10-09T21:52:07.482430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343133146735174:6676] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:07.482441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343133146735174:6676] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:07.482445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343133146735174:6676] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:07.482451Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2353 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:07.482966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:07.483216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:07.483256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:07.483542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:07.483576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.483580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.483581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.483582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.483582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.483583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.483584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.483585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.483586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.483591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.483600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.483605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.483609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.483612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.483614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.483616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.483617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.483619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.483624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.483630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.483634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.483637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.483640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.483643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.483645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.483648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.483651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.483653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.483656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.483659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.483662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.483665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.483667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.483670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.483674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.483677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.483691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:07.489497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:07.490618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:07.490636Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:07.490638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:07.490641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:07.490648Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2352 [2:7559343132784882869:9143]
2025-10-09T21:52:07.490659Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:07.490675Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:07.490683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:07.490686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:07.490688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:07.490703Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:07.490709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:07.490718Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:07.490770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.490776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.490778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.490781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:07.490836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:07.490844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.490854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.490864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.490879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:07.490903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343133146735312:6676], now have 1 active actors on pipe
2025-10-09T21:52:07.490910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343133146735130:2432], now have 1 active actors on pipe
2025-10-09T21:52:07.490917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343132784882863:9143]
2025-10-09T21:52:07.490919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343132784882863:9143]
2025-10-09T21:52:07.490921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343132784882863:9143]
2025-10-09T21:52:07.490935Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:07.490938Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2352
2025-10-09T21:52:07.490947Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2352
2025-10-09T21:52:07.490952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:07.490958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:07.490962Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:07.490965Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:07.490970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:07.490973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:07.491001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:07.491009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:07.491021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343132784882863:9143]
2025-10-09T21:52:07.491033Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:07.491040Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:07.491064Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c9e1e7dd-f79226ee-56319c52-242045e7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:07.491076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:07.491079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:07.491081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:07.491093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:07.491107Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:07.491129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:07.491271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:07.491282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:07.491297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:07.491395Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2353 partition: 0 MaxSeqNo: 25 sessionId: src_id|c9e1e7dd-f79226ee-56319c52-242045e7_0
2025-10-09T21:52:07.491652Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a35cf186-23c566b3-163e8d84-a37acd13_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c9e1e7dd-f79226ee-56319c52-242045e7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:07.493971Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c9e1e7dd-f79226ee-56319c52-242045e7_0] Write session: try to update token
2025-10-09T21:52:07.493987Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c9e1e7dd-f79226ee-56319c52-242045e7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:07.504274Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:07.504299Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:07.504301Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:07.504303Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:07.504313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343132784882863:9143]
2025-10-09T21:52:07.504317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343132784882863:9143]
2025-10-09T21:52:07.504321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343132784882863:9143]
2025-10-09T21:52:07.504366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:07.504368Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:07.504373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.504374Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:07.504376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:07.504378Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:07.504508Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:07.505735Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2353 sessionId: src_id|c9e1e7dd-f79226ee-56319c52-242045e7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:07.507919Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:07.568886Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343133146735324:7496]
2025-10-09T21:52:07.569146Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 2352, pipe: [1:7559343133146735324:7496]
2025-10-09T21:52:07.651565Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:07.651854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:07.651870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:07.651914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343133146735324:7496], now have 1 active actors on pipe
2025-10-09T21:52:07.652097Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343133146735324:7496]
2025-10-09T21:52:07.652110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:07.652114Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:07.652140Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:07.652123Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:07.652174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2353 reason: error at write: Disconnected sessionId: src_id|c9e1e7dd-f79226ee-56319c52-242045e7_0
2025-10-09T21:52:07.652320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2353 sessionId: src_id|c9e1e7dd-f79226ee-56319c52-242045e7_0 is DEAD
2025-10-09T21:52:07.652482Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9e1e7dd-f79226ee-56319c52-242045e7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:07.652494Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9e1e7dd-f79226ee-56319c52-242045e7_0] Write session will restart in 0.007783s
2025-10-09T21:52:07.652514Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9e1e7dd-f79226ee-56319c52-242045e7_0] Write session: Do CDS request
2025-10-09T21:52:07.652519Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9e1e7dd-f79226ee-56319c52-242045e7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:07.661487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c9e1e7dd-f79226ee-56319c52-242045e7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:07.661589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:07.661603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2354
2025-10-09T21:52:07.661725Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2354 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:07.661758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2354 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:07.661768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2354 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:07.661772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:07.662171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2354 sessionId:  describe result for acl check
2025-10-09T21:52:07.662204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:07.662209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:07.662211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:07.662217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343133146735470:6655] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:07.662748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343133146735470:6655] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:07.663068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:07.664936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:07.664970Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343133146735470:6655] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:07.665071Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:07.665081Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:07.665091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:07.665140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:07.665181Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343132784883054:9151]
2025-10-09T21:52:07.665652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:07.665932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:07.665974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:07.666302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:07.666347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.666356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.666357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.666358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.666359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.666361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.666362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.666363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.666364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.666368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.666378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.666382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.666386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.666390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.666394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.666398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.666401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.666405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.666411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.666418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.666424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.666429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.666434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.666439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.666443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.666447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.666452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.666457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.666461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.666466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.666472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.666477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.666482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.666487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.666491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.666496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.666524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:07.668244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343133146735470:6655] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:07.668254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343133146735470:6655] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:07.668258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343133146735470:6655] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:07.668265Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2354 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:07.672284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:07.673629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:07.673646Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:07.673648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:07.673652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:07.673658Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2353 [2:7559343132784883054:9151]
2025-10-09T21:52:07.673672Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:07.673690Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:07.673698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:07.673701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:07.673704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:07.673720Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:07.673731Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:07.673737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:07.673794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.673802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.673805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.673808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:07.673980Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2353
2025-10-09T21:52:07.673982Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2353
2025-10-09T21:52:07.674053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:07.674063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.674076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.674092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.674115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:07.674138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343133146735513:6655], now have 1 active actors on pipe
2025-10-09T21:52:07.674148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343133146735326:2432], now have 1 active actors on pipe
2025-10-09T21:52:07.674163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:07.674168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:07.674186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343132784883048:9151]
2025-10-09T21:52:07.674192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343132784883048:9151]
2025-10-09T21:52:07.674194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343132784883048:9151]
2025-10-09T21:52:07.674207Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:07.674225Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:07.674230Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:07.674233Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:07.674235Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:07.674245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:07.674248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:07.674267Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b636c643-b34ab3b2-f931898a-85cccf5e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:07.674278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:07.674287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:07.674289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:07.674299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:07.674309Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:07.674336Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343132784883048:9151]
2025-10-09T21:52:07.674347Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:07.674363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:07.674364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:07.674512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:07.674522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:07.674536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:07.674606Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2354 partition: 0 MaxSeqNo: 25 sessionId: src_id|b636c643-b34ab3b2-f931898a-85cccf5e_0
2025-10-09T21:52:07.674900Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c9e1e7dd-f79226ee-56319c52-242045e7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b636c643-b34ab3b2-f931898a-85cccf5e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:07.677247Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b636c643-b34ab3b2-f931898a-85cccf5e_0] Write session: try to update token
2025-10-09T21:52:07.677260Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b636c643-b34ab3b2-f931898a-85cccf5e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:07.689257Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2354 sessionId: src_id|b636c643-b34ab3b2-f931898a-85cccf5e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:07.690259Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:07.690275Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:07.690284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343132784883048:9151]
2025-10-09T21:52:07.690333Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:07.690342Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:07.690528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:07.690687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:07.690830Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:07.691022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:07.691231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:07.691275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:07.691673Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:07.691862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:07.691882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:07.691913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:07.692006Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:07.696992Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:07.697006Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:07.697008Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:07.697010Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:07.697016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343132784883048:9151]
2025-10-09T21:52:07.697019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343132784883048:9151]
2025-10-09T21:52:07.697022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343132784883048:9151]
2025-10-09T21:52:07.697050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:07.697059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.697061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:07.697063Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:07.697070Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:07.697072Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:07.697188Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:07.752165Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343133146735520:7496]
2025-10-09T21:52:07.752460Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 2353, pipe: [1:7559343133146735520:7496]
2025-10-09T21:52:07.841651Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:07.848332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:07.848364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:07.848526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:07.848590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:07.848645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:07.848705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:07.848749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:07.848757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:07.848830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:07.848889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:07.848949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:07.849003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:07.849042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:07.849048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:07.849103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:07.849154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:07.849211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:07.849265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:07.849324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:07.849336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:07.849395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:07.849450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:07.849505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:07.849560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:07.849600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:07.849605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:07.849661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:07.849717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:07.849776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:07.849835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:07.849882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:07.849890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:07.849946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:07.850001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:07.850071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:07.850127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:07.850165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:07.850172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:07.850225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:07.850277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:07.850332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:07.850386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:07.850425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:07.850430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:07.850491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:07.850541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:07.850595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:07.850648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:07.850734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:07.850740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:07.850791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:07.850842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:07.850894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:07.850959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:07.851003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:07.851011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:07.851067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:07.851120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:07.851170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:07.851223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:07.851260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:07.851266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:07.851306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343133146735520:7496], now have 1 active actors on pipe
2025-10-09T21:52:07.851583Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343133146735520:7496]
2025-10-09T21:52:07.851651Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:07.851685Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:07.851692Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:07.851719Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2354 reason: error at write: Disconnected sessionId: src_id|b636c643-b34ab3b2-f931898a-85cccf5e_0
2025-10-09T21:52:07.851904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2354 sessionId: src_id|b636c643-b34ab3b2-f931898a-85cccf5e_0 is DEAD
2025-10-09T21:52:07.852014Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b636c643-b34ab3b2-f931898a-85cccf5e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:07.852023Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b636c643-b34ab3b2-f931898a-85cccf5e_0] Write session will restart in 0.009646s
2025-10-09T21:52:07.852041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b636c643-b34ab3b2-f931898a-85cccf5e_0] Write session: Do CDS request
2025-10-09T21:52:07.852046Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b636c643-b34ab3b2-f931898a-85cccf5e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:07.852200Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:07.862479Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b636c643-b34ab3b2-f931898a-85cccf5e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:07.862786Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:07.862799Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2355
2025-10-09T21:52:07.862930Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2355 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:07.862963Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2355 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:07.862970Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2355 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:07.862973Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:07.863413Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2355 sessionId:  describe result for acl check
2025-10-09T21:52:07.863456Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:07.863462Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:07.863464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:07.863470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343133146735670:6651] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:07.863929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:07.863946Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343133146735670:6651] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:07.865945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:07.866142Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:07.866152Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:07.866164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:07.866216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:07.866256Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343132784883243:9147]
2025-10-09T21:52:07.866395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343133146735670:6651] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:07.866836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:07.867101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:07.867141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:07.867439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:07.867500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.867507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.867509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.867510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.867511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.867512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.867513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.867514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.867515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.867521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.867531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.867535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.867539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.867544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.867548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.867551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.867554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.867558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.867565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.867573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:07.867579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.867585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:07.867589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.867595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:07.867599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.867604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:07.867609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.867614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:07.867619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.867627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:07.867632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.867638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.867644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.867651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.867655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.867660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.867691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:07.872465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343133146735670:6651] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:07.872479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343133146735670:6651] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:07.872482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343133146735670:6651] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:07.872492Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2355 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:07.873796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:07.874848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:07.874866Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:07.874868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:07.874872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:07.874881Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2354 [2:7559343132784883243:9147]
2025-10-09T21:52:07.874895Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:07.874912Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:07.874917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:07.874920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:07.874923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:07.874938Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:07.874949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:07.874956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:07.875010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.875017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.875019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.875022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:07.875091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:07.875099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.875110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:07.875121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:07.875136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:07.875163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343133146735523:2432], now have 1 active actors on pipe
2025-10-09T21:52:07.875170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343133146735709:6651], now have 1 active actors on pipe
2025-10-09T21:52:07.875177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343132784883238:9147]
2025-10-09T21:52:07.875180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343132784883238:9147]
2025-10-09T21:52:07.875182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343132784883238:9147]
2025-10-09T21:52:07.875194Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:07.875205Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:07.875207Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:07.875209Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:07.875213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:07.875226Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:07.875234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:07.875219Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2354
2025-10-09T21:52:07.875236Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2354
2025-10-09T21:52:07.875261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343132784883238:9147]
2025-10-09T21:52:07.875276Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:07.875301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:07.875299Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:07.875309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:07.875351Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8b5b63d2-dff9eda1-73b75a09-4347ba21_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:07.875363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:07.875367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:07.875369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:07.875379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:07.875394Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:07.875418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:07.875585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:07.875596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:07.875610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:07.875698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2355 partition: 0 MaxSeqNo: 25 sessionId: src_id|8b5b63d2-dff9eda1-73b75a09-4347ba21_0
2025-10-09T21:52:07.876013Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b636c643-b34ab3b2-f931898a-85cccf5e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8b5b63d2-dff9eda1-73b75a09-4347ba21_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:07.878293Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b5b63d2-dff9eda1-73b75a09-4347ba21_0] Write session: try to update token
2025-10-09T21:52:07.878307Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b5b63d2-dff9eda1-73b75a09-4347ba21_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:07.889205Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:07.889227Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:07.889230Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:07.889232Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:07.889241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343132784883238:9147]
2025-10-09T21:52:07.889244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343132784883238:9147]
2025-10-09T21:52:07.889247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343132784883238:9147]
2025-10-09T21:52:07.889280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:07.889283Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:07.889287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:07.889287Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:07.889300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:07.889301Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:07.889437Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:07.890231Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2355 sessionId: src_id|8b5b63d2-dff9eda1-73b75a09-4347ba21_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:07.892444Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:07.951862Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343133146735719:7496]
2025-10-09T21:52:07.952133Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 2354, pipe: [1:7559343133146735719:7496]
2025-10-09T21:52:08.036371Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:08.036832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.036848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.036896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343133146735719:7496], now have 1 active actors on pipe
2025-10-09T21:52:08.037124Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343133146735719:7496]
2025-10-09T21:52:08.037140Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:08.037145Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:08.037164Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:08.037214Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2355 reason: error at write: Disconnected sessionId: src_id|8b5b63d2-dff9eda1-73b75a09-4347ba21_0
2025-10-09T21:52:08.037325Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:08.037411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2355 sessionId: src_id|8b5b63d2-dff9eda1-73b75a09-4347ba21_0 is DEAD
2025-10-09T21:52:08.037561Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b5b63d2-dff9eda1-73b75a09-4347ba21_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:08.037571Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b5b63d2-dff9eda1-73b75a09-4347ba21_0] Write session will restart in 0.006264s
2025-10-09T21:52:08.037592Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b5b63d2-dff9eda1-73b75a09-4347ba21_0] Write session: Do CDS request
2025-10-09T21:52:08.037598Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b5b63d2-dff9eda1-73b75a09-4347ba21_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:08.044506Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b5b63d2-dff9eda1-73b75a09-4347ba21_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:08.044592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:08.044607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2356
2025-10-09T21:52:08.044755Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2356 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:08.044790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2356 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:08.044800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2356 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:08.044804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:08.045272Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2356 sessionId:  describe result for acl check
2025-10-09T21:52:08.045325Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:08.045331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:08.045333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:08.045339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343137441703081:6639] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:08.045795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343137441703081:6639] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:08.048416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343137441703081:6639] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:08.049406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:08.051240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:08.051434Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:08.051444Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:08.051455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:08.051501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:08.051548Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343137079850725:9146]
2025-10-09T21:52:08.051615Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343137441703081:6639] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:08.051627Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343137441703081:6639] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:08.051630Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343137441703081:6639] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:08.051637Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2356 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:08.052057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:08.052278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:08.052308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:08.052514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:08.052548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.052552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.052553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.052554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.052555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.052555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.052556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.052556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.052557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.052560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.052567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.052570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.052572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.052574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.052576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.052579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.052580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.052582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.052588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.052596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.052603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.052608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.052612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.052618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.052623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.052627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.052630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.052633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.052635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.052639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.052642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.052645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.052648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.052650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.052653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.052656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.052672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:08.058838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:08.060029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:08.060048Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:08.060050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:08.060055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:08.060062Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2355 [2:7559343137079850725:9146]
2025-10-09T21:52:08.060077Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:08.060101Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:08.060107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:08.060110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:08.060113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:08.060130Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:08.060146Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:08.060158Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:08.060219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.060227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.060229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.060232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:08.060299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:08.060307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.060319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.060331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.060353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:08.060379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441703202:6639], now have 1 active actors on pipe
2025-10-09T21:52:08.060387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441703018:2432], now have 1 active actors on pipe
2025-10-09T21:52:08.060394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343137079850720:9146]
2025-10-09T21:52:08.060396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343137079850720:9146]
2025-10-09T21:52:08.060398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343137079850720:9146]
2025-10-09T21:52:08.060411Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:08.060423Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:08.060426Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:08.060427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:08.060433Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:08.060446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:08.060453Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:08.060449Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2355
2025-10-09T21:52:08.060449Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2355
2025-10-09T21:52:08.060482Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343137079850720:9146]
2025-10-09T21:52:08.060492Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:08.060495Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:08.060514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.060523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.060571Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7841ab2b-3d55556c-c5548421-2f7519dd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:08.060584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:08.060589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:08.060592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:08.060602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:08.060614Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:08.060641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:08.060812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.060820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.060835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:08.060910Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2356 partition: 0 MaxSeqNo: 25 sessionId: src_id|7841ab2b-3d55556c-c5548421-2f7519dd_0
2025-10-09T21:52:08.061257Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b5b63d2-dff9eda1-73b75a09-4347ba21_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7841ab2b-3d55556c-c5548421-2f7519dd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:08.063525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7841ab2b-3d55556c-c5548421-2f7519dd_0] Write session: try to update token
2025-10-09T21:52:08.063544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7841ab2b-3d55556c-c5548421-2f7519dd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:08.074768Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:08.074791Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:08.074794Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:08.074797Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:08.074808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343137079850720:9146]
2025-10-09T21:52:08.074813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343137079850720:9146]
2025-10-09T21:52:08.074817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343137079850720:9146]
2025-10-09T21:52:08.074861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:08.074866Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:08.074869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.074871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:08.074872Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:08.074887Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:08.075003Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:08.075774Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2356 sessionId: src_id|7841ab2b-3d55556c-c5548421-2f7519dd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:08.078083Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:08.137212Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343137441703213:7496]
2025-10-09T21:52:08.137421Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 2355, pipe: [1:7559343137441703213:7496]
2025-10-09T21:52:08.220759Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:08.221170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.221183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.221219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441703213:7496], now have 1 active actors on pipe
2025-10-09T21:52:08.221348Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:08.221440Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:08.221483Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:08.221564Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2356 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7841ab2b-3d55556c-c5548421-2f7519dd_0
2025-10-09T21:52:08.221745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2356 sessionId: src_id|7841ab2b-3d55556c-c5548421-2f7519dd_0 is DEAD
2025-10-09T21:52:08.221892Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7841ab2b-3d55556c-c5548421-2f7519dd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:08.221899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7841ab2b-3d55556c-c5548421-2f7519dd_0] Write session will restart in 0.007942s
2025-10-09T21:52:08.221923Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7841ab2b-3d55556c-c5548421-2f7519dd_0] Write session: Do CDS request
2025-10-09T21:52:08.221930Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7841ab2b-3d55556c-c5548421-2f7519dd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:08.221956Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343137441703213:7496]
2025-10-09T21:52:08.231414Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7841ab2b-3d55556c-c5548421-2f7519dd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:08.231524Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:08.231536Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2357
2025-10-09T21:52:08.231876Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2357 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:08.231904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2357 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:08.231912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2357 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:08.231914Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:08.232261Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2357 sessionId:  describe result for acl check
2025-10-09T21:52:08.232294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:08.232296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:08.232297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:08.232302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343137441703390:6602] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:08.232770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343137441703390:6602] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:08.232972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:08.234216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343137441703390:6602] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:08.234814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:08.235049Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:08.235060Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:08.235071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:08.235122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:08.235158Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343137079850950:9124]
2025-10-09T21:52:08.235715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:08.235981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:08.236023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:08.236305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:08.236355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.236361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.236362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.236363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.236364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.236365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.236366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.236367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.236369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.236373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.236381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.236384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.236387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.236390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.236394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.236397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.236401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.236404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.236409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.236416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.236422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.236427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.236431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.236435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.236438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.236443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.236451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.236455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.236462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.236467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.236476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.236481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.236485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.236489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.236493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.236497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.236523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:08.236696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343137441703390:6602] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:08.236708Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343137441703390:6602] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:08.236711Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343137441703390:6602] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:08.236717Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2357 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:08.242363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:08.243476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:08.243496Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:08.243498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:08.243502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:08.243509Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2356 [2:7559343137079850950:9124]
2025-10-09T21:52:08.243522Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:08.243538Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:08.243547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:08.243550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:08.243553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:08.243568Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:08.243578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:08.243585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:08.243634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.243641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.243643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.243646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:08.243704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:08.243712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.243725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.243737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.243754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:08.243779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441703432:6602], now have 1 active actors on pipe
2025-10-09T21:52:08.243788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441703231:2432], now have 1 active actors on pipe
2025-10-09T21:52:08.243795Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343137079850944:9124]
2025-10-09T21:52:08.243798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343137079850944:9124]
2025-10-09T21:52:08.243800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343137079850944:9124]
2025-10-09T21:52:08.243813Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:08.243831Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:08.243838Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:08.243841Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:08.243843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:08.243851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:08.243854Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:08.243833Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2356
2025-10-09T21:52:08.243837Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2356
2025-10-09T21:52:08.243876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343137079850944:9124]
2025-10-09T21:52:08.243883Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:08.243884Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:08.243898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.243905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.244145Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|745e678e-b0c128b9-c4e2a19-4398aad5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:08.244162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:08.244166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:08.244168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:08.244179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:08.244189Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:08.244216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:08.244391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.244401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.244415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:08.244506Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2357 partition: 0 MaxSeqNo: 25 sessionId: src_id|745e678e-b0c128b9-c4e2a19-4398aad5_0
2025-10-09T21:52:08.244776Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7841ab2b-3d55556c-c5548421-2f7519dd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|745e678e-b0c128b9-c4e2a19-4398aad5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:08.246970Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|745e678e-b0c128b9-c4e2a19-4398aad5_0] Write session: try to update token
2025-10-09T21:52:08.246983Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|745e678e-b0c128b9-c4e2a19-4398aad5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:08.257039Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:08.257063Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:08.257065Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:08.257067Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:08.257077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343137079850944:9124]
2025-10-09T21:52:08.257082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343137079850944:9124]
2025-10-09T21:52:08.257086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343137079850944:9124]
2025-10-09T21:52:08.257129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:08.257133Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:08.257138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.257139Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:08.257141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:08.257142Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:08.257287Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:08.258678Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2357 sessionId: src_id|745e678e-b0c128b9-c4e2a19-4398aad5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:08.260749Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:08.322053Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343137441703444:7496]
2025-10-09T21:52:08.322304Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 2356, pipe: [1:7559343137441703444:7496]
2025-10-09T21:52:08.392849Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:08.393215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.393229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.393268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441703444:7496], now have 1 active actors on pipe
2025-10-09T21:52:08.393475Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:08.393506Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343137441703444:7496]
2025-10-09T21:52:08.393530Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:08.393533Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:08.393535Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:08.393589Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2357 reason: error at write: Disconnected sessionId: src_id|745e678e-b0c128b9-c4e2a19-4398aad5_0
2025-10-09T21:52:08.393774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2357 sessionId: src_id|745e678e-b0c128b9-c4e2a19-4398aad5_0 is DEAD
2025-10-09T21:52:08.393946Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|745e678e-b0c128b9-c4e2a19-4398aad5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:08.393952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|745e678e-b0c128b9-c4e2a19-4398aad5_0] Write session will restart in 0.006107s
2025-10-09T21:52:08.393971Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|745e678e-b0c128b9-c4e2a19-4398aad5_0] Write session: Do CDS request
2025-10-09T21:52:08.393976Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|745e678e-b0c128b9-c4e2a19-4398aad5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:08.401479Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|745e678e-b0c128b9-c4e2a19-4398aad5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:08.401598Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:08.401609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2358
2025-10-09T21:52:08.401720Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2358 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:08.401744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2358 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:08.401751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2358 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:08.401754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:08.402199Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2358 sessionId:  describe result for acl check
2025-10-09T21:52:08.402236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:08.402240Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:08.402241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:08.402247Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343137441703523:6595] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:08.402795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343137441703523:6595] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:08.404914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343137441703523:6595] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:08.405631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:08.407211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:08.407383Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:08.407394Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:08.407406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:08.407458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:08.407472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343137441703523:6595] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:08.407494Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343137079851138:9111]
2025-10-09T21:52:08.407486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343137441703523:6595] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:08.407489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343137441703523:6595] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:08.407497Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2358 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:08.408063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:08.408284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:08.408322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:08.408614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:08.408658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.408664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.408665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.408667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.408668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.408669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.408670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.408671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.408672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.408676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.408687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.408695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.408699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.408703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.408707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.408715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.408718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.408726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.408732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.408744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.408754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.408763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.408768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.408773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.408781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.408790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.408799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.408808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.408814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.408823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.408828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.408835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.408839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.408844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.408853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.408858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.408891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:08.414492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:08.415511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:08.415526Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:08.415529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:08.415533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:08.415539Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2357 [2:7559343137079851138:9111]
2025-10-09T21:52:08.415551Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:08.415567Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:08.415576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:08.415579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:08.415582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:08.415601Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:08.415614Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:08.415624Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:08.415674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.415679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.415684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.415686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:08.415747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:08.415756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.415768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.415778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.415795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:08.415822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441703446:2432], now have 1 active actors on pipe
2025-10-09T21:52:08.415832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441703633:6595], now have 1 active actors on pipe
2025-10-09T21:52:08.415839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343137079851133:9111]
2025-10-09T21:52:08.415841Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343137079851133:9111]
2025-10-09T21:52:08.415843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343137079851133:9111]
2025-10-09T21:52:08.415857Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:08.415864Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:08.415844Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2357
2025-10-09T21:52:08.415867Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:08.415866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2357
2025-10-09T21:52:08.415869Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:08.415875Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:08.415890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:08.415893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:08.415919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343137079851133:9111]
2025-10-09T21:52:08.415930Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:08.415934Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:08.415953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.415962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.415993Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8c04767d-9f2fe440-9bcab29f-d2ad94e9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:08.416005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:08.416008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:08.416011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:08.416020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:08.416033Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:08.416054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:08.416195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.416205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.416221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:08.416295Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2358 partition: 0 MaxSeqNo: 25 sessionId: src_id|8c04767d-9f2fe440-9bcab29f-d2ad94e9_0
2025-10-09T21:52:08.416601Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|745e678e-b0c128b9-c4e2a19-4398aad5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8c04767d-9f2fe440-9bcab29f-d2ad94e9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:08.418832Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c04767d-9f2fe440-9bcab29f-d2ad94e9_0] Write session: try to update token
2025-10-09T21:52:08.418845Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c04767d-9f2fe440-9bcab29f-d2ad94e9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:08.429536Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:08.429551Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:08.429554Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:08.429556Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:08.429565Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343137079851133:9111]
2025-10-09T21:52:08.429569Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343137079851133:9111]
2025-10-09T21:52:08.429573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343137079851133:9111]
2025-10-09T21:52:08.429602Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:08.429607Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:08.429610Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:08.429613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:08.429621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.429625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:08.429755Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:08.430188Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2358 sessionId: src_id|8c04767d-9f2fe440-9bcab29f-d2ad94e9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:08.432392Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:08.493567Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343137441703644:7496]
2025-10-09T21:52:08.493791Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 2357, pipe: [1:7559343137441703644:7496]
2025-10-09T21:52:08.566433Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:08.566873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.566890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.566937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441703644:7496], now have 1 active actors on pipe
2025-10-09T21:52:08.567115Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:08.567158Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:08.567161Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:08.567170Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:08.567210Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343137441703644:7496]
2025-10-09T21:52:08.567232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2358 reason: error at write: Disconnected sessionId: src_id|8c04767d-9f2fe440-9bcab29f-d2ad94e9_0
2025-10-09T21:52:08.567390Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2358 sessionId: src_id|8c04767d-9f2fe440-9bcab29f-d2ad94e9_0 is DEAD
2025-10-09T21:52:08.567542Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c04767d-9f2fe440-9bcab29f-d2ad94e9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:08.567554Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c04767d-9f2fe440-9bcab29f-d2ad94e9_0] Write session will restart in 0.007406s
2025-10-09T21:52:08.567578Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c04767d-9f2fe440-9bcab29f-d2ad94e9_0] Write session: Do CDS request
2025-10-09T21:52:08.567595Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c04767d-9f2fe440-9bcab29f-d2ad94e9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:08.576436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c04767d-9f2fe440-9bcab29f-d2ad94e9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:08.576521Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:08.576531Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2359
2025-10-09T21:52:08.576633Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2359 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:08.576659Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2359 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:08.576665Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2359 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:08.576668Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:08.577102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2359 sessionId:  describe result for acl check
2025-10-09T21:52:08.577139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:08.577145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:08.577147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:08.577152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343137441703774:6585] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:08.577686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343137441703774:6585] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:08.579032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:08.579315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343137441703774:6585] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:08.580878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:08.581044Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:08.581055Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:08.581066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:08.581117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:08.581152Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343137079851327:9100]
2025-10-09T21:52:08.581614Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343137441703774:6585] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:08.581624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343137441703774:6585] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:08.581626Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343137441703774:6585] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:08.581632Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2359 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:08.581731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:08.581955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:08.581997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:08.582298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:08.582355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.582361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.582362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.582363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.582364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.582365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.582366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.582367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.582368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.582373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.582383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.582387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.582391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.582395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.582399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.582402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.582406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.582410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.582417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.582424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.582432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.582441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.582446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.582455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.582459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.582469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.582478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.582483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.582492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.582499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.582508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.582513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.582521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.582526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.582531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.582536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.582565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:08.588395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:08.589471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:08.589481Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:08.589483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:08.589485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:08.589489Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2358 [2:7559343137079851327:9100]
2025-10-09T21:52:08.589502Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:08.589515Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:08.589524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:08.589526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:08.589529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:08.589544Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:08.589549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:08.589555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:08.589605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.589613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.589615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.589618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:08.589669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:08.589676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.589687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.589700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.589712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:08.589730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441703834:6585], now have 1 active actors on pipe
2025-10-09T21:52:08.589737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441703646:2432], now have 1 active actors on pipe
2025-10-09T21:52:08.589747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343137079851322:9100]
2025-10-09T21:52:08.589749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343137079851322:9100]
2025-10-09T21:52:08.589751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343137079851322:9100]
2025-10-09T21:52:08.589765Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:08.589758Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2358
2025-10-09T21:52:08.589781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:08.589770Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2358
2025-10-09T21:52:08.589790Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:08.589798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:08.589801Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:08.589801Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:08.589803Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:08.589831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.589836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.589851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343137079851322:9100]
2025-10-09T21:52:08.589861Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:08.589873Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:08.589887Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8a0ecd2a-585e332e-ddd288ae-dd88812d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:08.589898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:08.589902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:08.589907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:08.589916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:08.589930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:08.589953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:08.590095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.590109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.590132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:08.590306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2359 partition: 0 MaxSeqNo: 25 sessionId: src_id|8a0ecd2a-585e332e-ddd288ae-dd88812d_0
2025-10-09T21:52:08.590650Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c04767d-9f2fe440-9bcab29f-d2ad94e9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8a0ecd2a-585e332e-ddd288ae-dd88812d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:08.593030Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a0ecd2a-585e332e-ddd288ae-dd88812d_0] Write session: try to update token
2025-10-09T21:52:08.593044Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a0ecd2a-585e332e-ddd288ae-dd88812d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:08.603419Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:08.603435Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:08.603437Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:08.603439Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:08.603450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343137079851322:9100]
2025-10-09T21:52:08.603455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343137079851322:9100]
2025-10-09T21:52:08.603459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343137079851322:9100]
2025-10-09T21:52:08.603495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:08.603497Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:08.603501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.603502Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:08.603503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:08.603504Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:08.603616Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:08.605077Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2359 sessionId: src_id|8a0ecd2a-585e332e-ddd288ae-dd88812d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:08.607540Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:08.667607Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343137441703845:7496]
2025-10-09T21:52:08.667845Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 2358, pipe: [1:7559343137441703845:7496]
2025-10-09T21:52:08.741779Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:08.742219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.742231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.742270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441703845:7496], now have 1 active actors on pipe
2025-10-09T21:52:08.742434Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:08.742517Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343137441703845:7496]
2025-10-09T21:52:08.742530Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:08.742548Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:08.742605Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2359 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8a0ecd2a-585e332e-ddd288ae-dd88812d_0
2025-10-09T21:52:08.742778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2359 sessionId: src_id|8a0ecd2a-585e332e-ddd288ae-dd88812d_0 is DEAD
2025-10-09T21:52:08.742920Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a0ecd2a-585e332e-ddd288ae-dd88812d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:08.742931Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a0ecd2a-585e332e-ddd288ae-dd88812d_0] Write session will restart in 0.005739s
2025-10-09T21:52:08.742951Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a0ecd2a-585e332e-ddd288ae-dd88812d_0] Write session: Do CDS request
2025-10-09T21:52:08.742956Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a0ecd2a-585e332e-ddd288ae-dd88812d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:08.749436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a0ecd2a-585e332e-ddd288ae-dd88812d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:08.749540Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:08.749553Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2360
2025-10-09T21:52:08.749681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2360 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:08.749713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2360 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:08.749727Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2360 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:08.749731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:08.750101Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2360 sessionId:  describe result for acl check
2025-10-09T21:52:08.750130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:08.750131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:08.750132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:08.750136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343137441703883:6573] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:08.750578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343137441703883:6573] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:08.752962Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343137441703883:6573] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:08.755143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:08.755198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343137441703883:6573] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:08.755211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343137441703883:6573] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:08.755214Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343137441703883:6573] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:08.755221Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2360 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:08.756821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:08.756990Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:08.757000Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:08.757011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:08.757064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:08.757102Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343137079851518:9105]
2025-10-09T21:52:08.757536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:08.757776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:08.757820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:08.758111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:08.758173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.758179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.758180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.758181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.758182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.758184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.758185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.758186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.758187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.758192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.758202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.758205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.758207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.758209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.758211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.758213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.758215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.758217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.758223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.758229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.758234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.758237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.758239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.758242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.758245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.758248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.758251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.758253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.758260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.758263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.758266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.758273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.758276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.758279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.758282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.758285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.758300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:08.764370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:08.765413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:08.765425Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:08.765427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:08.765431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:08.765438Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2359 [2:7559343137079851518:9105]
2025-10-09T21:52:08.765450Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:08.765462Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:08.765464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:08.765466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:08.765467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:08.765478Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:08.765481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:08.765487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:08.765582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.765590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.765591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.765593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:08.765652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:08.765659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.765671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.765687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.765702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:08.765716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441703847:2432], now have 1 active actors on pipe
2025-10-09T21:52:08.765722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441704035:6573], now have 1 active actors on pipe
2025-10-09T21:52:08.765727Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343137079851512:9105]
2025-10-09T21:52:08.765728Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343137079851512:9105]
2025-10-09T21:52:08.765729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343137079851512:9105]
2025-10-09T21:52:08.765740Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:08.765752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:08.765742Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2359
2025-10-09T21:52:08.765764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:08.765768Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2359
2025-10-09T21:52:08.765763Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:08.765766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:08.765766Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:08.765768Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:08.765783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343137079851512:9105]
2025-10-09T21:52:08.765785Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:08.765787Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:08.765800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.765804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.765837Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8b3ff1f1-210abb21-fafda573-6f1cac49_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:08.765848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:08.765857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:08.765859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:08.765868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:08.765880Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:08.765896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:08.766052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.766060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.766073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:08.766189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2360 partition: 0 MaxSeqNo: 25 sessionId: src_id|8b3ff1f1-210abb21-fafda573-6f1cac49_0
2025-10-09T21:52:08.766498Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a0ecd2a-585e332e-ddd288ae-dd88812d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8b3ff1f1-210abb21-fafda573-6f1cac49_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:08.768847Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b3ff1f1-210abb21-fafda573-6f1cac49_0] Write session: try to update token
2025-10-09T21:52:08.768862Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b3ff1f1-210abb21-fafda573-6f1cac49_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:08.780437Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:08.780460Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:08.780463Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:08.780467Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:08.780479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343137079851512:9105]
2025-10-09T21:52:08.780485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343137079851512:9105]
2025-10-09T21:52:08.780493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343137079851512:9105]
2025-10-09T21:52:08.780536Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:08.780548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:08.780551Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:08.780554Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:08.780557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.780561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:08.780821Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:08.781563Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2360 sessionId: src_id|8b3ff1f1-210abb21-fafda573-6f1cac49_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:08.784009Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:08.842763Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343137441704048:7496]
2025-10-09T21:52:08.842985Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 2359, pipe: [1:7559343137441704048:7496]
2025-10-09T21:52:08.918609Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:08.918836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.918848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.918885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441704048:7496], now have 1 active actors on pipe
2025-10-09T21:52:08.919046Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:08.919048Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:08.919062Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:08.919098Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343137441704048:7496]
2025-10-09T21:52:08.919130Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:08.919140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2360 reason: error at write: Disconnected sessionId: src_id|8b3ff1f1-210abb21-fafda573-6f1cac49_0
2025-10-09T21:52:08.919314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2360 sessionId: src_id|8b3ff1f1-210abb21-fafda573-6f1cac49_0 is DEAD
2025-10-09T21:52:08.919422Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b3ff1f1-210abb21-fafda573-6f1cac49_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:08.919428Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b3ff1f1-210abb21-fafda573-6f1cac49_0] Write session will restart in 0.005768s
2025-10-09T21:52:08.919447Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b3ff1f1-210abb21-fafda573-6f1cac49_0] Write session: Do CDS request
2025-10-09T21:52:08.919452Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b3ff1f1-210abb21-fafda573-6f1cac49_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:08.926447Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b3ff1f1-210abb21-fafda573-6f1cac49_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:08.926521Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:08.926534Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2361
2025-10-09T21:52:08.926662Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2361 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:08.926695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2361 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:08.926705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2361 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:08.926708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:08.927171Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2361 sessionId:  describe result for acl check
2025-10-09T21:52:08.927206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:08.927213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:08.927215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:08.927220Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343137441704103:6554] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:08.927750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343137441704103:6554] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:08.929723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343137441704103:6554] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:08.931444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:08.932967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:08.933027Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343137441704103:6554] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:08.933038Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343137441704103:6554] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:08.933041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343137441704103:6554] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:08.933046Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2361 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:08.933132Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:08.933141Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:08.933152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:08.933203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:08.933239Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343137079851712:9098]
2025-10-09T21:52:08.933794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:08.934059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:08.934103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:08.934364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:08.934412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.934418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.934419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.934419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.934421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.934422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.934423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.934424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.934424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.934429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.934438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.934441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.934449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.934452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.934456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.934460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.934463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.934467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.934473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.934482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:08.934491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.934496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:08.934505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.934515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:08.934523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.934528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:08.934536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.934541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:08.934546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.934555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:08.934559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.934570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.934573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.934578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.934583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.934587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.934613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:08.940374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:08.941733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:08.941747Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:08.941750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:08.941754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:08.941760Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2360 [2:7559343137079851712:9098]
2025-10-09T21:52:08.941772Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:08.941788Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:08.941796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:08.941799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:08.941801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:08.941817Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:08.941823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:08.941830Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:08.941888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.941896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.941898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.941901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:08.941967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:08.941976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.941989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:08.942002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:08.942019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:08.942063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441704052:2432], now have 1 active actors on pipe
2025-10-09T21:52:08.942070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343137441704241:6554], now have 1 active actors on pipe
2025-10-09T21:52:08.942078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343137079851707:9098]
2025-10-09T21:52:08.942080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343137079851707:9098]
2025-10-09T21:52:08.942082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343137079851707:9098]
2025-10-09T21:52:08.942099Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:08.942086Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2360
2025-10-09T21:52:08.942103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2360
2025-10-09T21:52:08.942115Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:08.942118Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:08.942122Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:08.942124Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:08.942132Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:08.942135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:08.942174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.942184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.942201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343137079851707:9098]
2025-10-09T21:52:08.942213Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:08.942236Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:08.942245Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|90548cc3-16f7ab36-5a0b140b-f9ab8a8d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:08.942252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:08.942260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:08.942262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:08.942272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:08.942286Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:08.942309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:08.942479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:08.942486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:08.942500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:08.942585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2361 partition: 0 MaxSeqNo: 25 sessionId: src_id|90548cc3-16f7ab36-5a0b140b-f9ab8a8d_0
2025-10-09T21:52:08.942926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b3ff1f1-210abb21-fafda573-6f1cac49_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|90548cc3-16f7ab36-5a0b140b-f9ab8a8d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:08.945193Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90548cc3-16f7ab36-5a0b140b-f9ab8a8d_0] Write session: try to update token
2025-10-09T21:52:08.945209Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90548cc3-16f7ab36-5a0b140b-f9ab8a8d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:08.955541Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:08.955558Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:08.955561Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:08.955563Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:08.955573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343137079851707:9098]
2025-10-09T21:52:08.955579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343137079851707:9098]
2025-10-09T21:52:08.955583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343137079851707:9098]
2025-10-09T21:52:08.955625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:08.955624Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:08.955629Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:08.955635Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:08.955635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:08.955638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:08.955772Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:08.957498Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2361 sessionId: src_id|90548cc3-16f7ab36-5a0b140b-f9ab8a8d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:08.959905Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:09.019396Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343141736671549:7496]
2025-10-09T21:52:09.019770Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 2360, pipe: [1:7559343141736671549:7496]
2025-10-09T21:52:09.099100Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:09.099358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.099374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.099422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343141736671549:7496], now have 1 active actors on pipe
2025-10-09T21:52:09.099603Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:09.099683Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343141736671549:7496]
2025-10-09T21:52:09.099697Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:09.099722Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:09.099787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2361 reason: status is not ok: tablet will be restarted right now sessionId: src_id|90548cc3-16f7ab36-5a0b140b-f9ab8a8d_0
2025-10-09T21:52:09.099926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2361 sessionId: src_id|90548cc3-16f7ab36-5a0b140b-f9ab8a8d_0 is DEAD
2025-10-09T21:52:09.100098Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90548cc3-16f7ab36-5a0b140b-f9ab8a8d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:09.100109Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90548cc3-16f7ab36-5a0b140b-f9ab8a8d_0] Write session will restart in 0.006171s
2025-10-09T21:52:09.100130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90548cc3-16f7ab36-5a0b140b-f9ab8a8d_0] Write session: Do CDS request
2025-10-09T21:52:09.100143Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90548cc3-16f7ab36-5a0b140b-f9ab8a8d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:09.107515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90548cc3-16f7ab36-5a0b140b-f9ab8a8d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:09.107671Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:09.107685Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2362
2025-10-09T21:52:09.107811Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2362 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:09.107849Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2362 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:09.107869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2362 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:09.107874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:09.108349Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2362 sessionId:  describe result for acl check
2025-10-09T21:52:09.108378Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:09.108379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:09.108381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:09.108385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343141736671609:6529] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:09.108892Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343141736671609:6529] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:09.111507Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343141736671609:6529] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:09.112801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:09.114698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:09.114850Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:09.114858Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:09.114867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:09.114886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343141736671609:6529] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:09.114900Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343141736671609:6529] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:09.114904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343141736671609:6529] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:09.114907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:09.114913Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2362 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:09.114938Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343141374819200:9096]
2025-10-09T21:52:09.115383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:09.115633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:09.115673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:09.115982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:09.116024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.116029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.116030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.116030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.116031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.116032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.116032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.116033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.116034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.116037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.116044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.116047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.116049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.116050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.116053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.116055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.116057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.116059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.116064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.116072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.116078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.116084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.116089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.116094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.116098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.116103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.116106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.116111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.116115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.116121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.116128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.116134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.116138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.116141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.116144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.116149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.116172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:09.122612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:09.123777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:09.123792Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:09.123794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:09.123799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:09.123805Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2361 [2:7559343141374819200:9096]
2025-10-09T21:52:09.123819Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:09.123837Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:09.123845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:09.123849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:09.123851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:09.123868Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:09.123878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:09.123886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:09.123939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.123946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.123948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.123951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:09.124023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:09.124031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.124040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.124048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.124058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:09.124082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343141736671741:6529], now have 1 active actors on pipe
2025-10-09T21:52:09.124090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343141736671551:2432], now have 1 active actors on pipe
2025-10-09T21:52:09.124098Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343141374819195:9096]
2025-10-09T21:52:09.124100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343141374819195:9096]
2025-10-09T21:52:09.124103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343141374819195:9096]
2025-10-09T21:52:09.124114Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:09.124133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:09.124133Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:09.124136Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:09.124137Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:09.124152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:09.124155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:09.124180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343141374819195:9096]
2025-10-09T21:52:09.124191Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:09.124193Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:09.124169Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2361
2025-10-09T21:52:09.124175Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2361
2025-10-09T21:52:09.124224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.124232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.124256Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d3507675-d3096c13-3e78f4b3-f922b608_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:09.124265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:09.124268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:09.124270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:09.124278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:09.124286Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:09.124297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:09.124406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.124412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.124423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:09.124546Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2362 partition: 0 MaxSeqNo: 25 sessionId: src_id|d3507675-d3096c13-3e78f4b3-f922b608_0
2025-10-09T21:52:09.124887Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90548cc3-16f7ab36-5a0b140b-f9ab8a8d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d3507675-d3096c13-3e78f4b3-f922b608_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:09.127124Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d3507675-d3096c13-3e78f4b3-f922b608_0] Write session: try to update token
2025-10-09T21:52:09.127138Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d3507675-d3096c13-3e78f4b3-f922b608_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:09.139117Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2362 sessionId: src_id|d3507675-d3096c13-3e78f4b3-f922b608_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:09.141273Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:09.141299Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:09.141307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343141374819195:9096]
2025-10-09T21:52:09.141351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:09.141359Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:09.141548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:09.141645Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:09.141684Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:09.141824Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:09.141963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:09.142163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:09.142209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.142225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.142246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:09.142411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:09.142525Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:09.148345Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:09.148360Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:09.148362Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:09.148365Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:09.148373Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343141374819195:9096]
2025-10-09T21:52:09.148377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343141374819195:9096]
2025-10-09T21:52:09.148382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343141374819195:9096]
2025-10-09T21:52:09.148418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:09.148423Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:09.148429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.148429Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:09.148431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:09.148433Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:09.148574Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:09.199616Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343141736671753:7496]
2025-10-09T21:52:09.199861Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 2361, pipe: [1:7559343141736671753:7496]
2025-10-09T21:52:09.294669Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:09.301062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.301086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.301230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.301286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.301353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.301409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.301717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.301727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.301788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.301849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.301913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.301984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.302024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.302046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.302106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.302163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.302220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.302280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.302321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.302329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.302392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.302446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.302498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.302554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.302592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.302597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.302657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.302796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.302851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.302903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.302941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.302946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.303000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.303050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.303101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.303154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.303199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.303204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.303256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.303307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.303362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.303413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.303503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.303510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.303562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.303610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.303661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.303715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.303751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.303755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.303808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.303902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.303954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.304008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.304045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.304050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.304135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.304190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.304249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.304302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.304338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.304347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.304389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343141736671753:7496], now have 1 active actors on pipe
2025-10-09T21:52:09.304604Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:09.304639Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:09.304643Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:09.304660Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343141736671753:7496]
2025-10-09T21:52:09.304672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2362 reason: error at write: Disconnected sessionId: src_id|d3507675-d3096c13-3e78f4b3-f922b608_0
2025-10-09T21:52:09.304827Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:09.304865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2362 sessionId: src_id|d3507675-d3096c13-3e78f4b3-f922b608_0 is DEAD
2025-10-09T21:52:09.305016Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3507675-d3096c13-3e78f4b3-f922b608_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:09.305026Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3507675-d3096c13-3e78f4b3-f922b608_0] Write session will restart in 0.007495s
2025-10-09T21:52:09.305046Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3507675-d3096c13-3e78f4b3-f922b608_0] Write session: Do CDS request
2025-10-09T21:52:09.305052Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3507675-d3096c13-3e78f4b3-f922b608_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:09.313499Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d3507675-d3096c13-3e78f4b3-f922b608_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:09.313990Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:09.314005Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2363
2025-10-09T21:52:09.314152Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2363 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:09.314184Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2363 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:09.314190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2363 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:09.314193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:09.314616Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2363 sessionId:  describe result for acl check
2025-10-09T21:52:09.314652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:09.314657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:09.314658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:09.314662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343141736671870:6496] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:09.315149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343141736671870:6496] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:09.317138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343141736671870:6496] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:09.317800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:09.319659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:09.319830Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:09.319839Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:09.319851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:09.319906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:09.319961Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343141374819432:9047]
2025-10-09T21:52:09.319967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343141736671870:6496] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:09.319979Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343141736671870:6496] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:09.319983Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343141736671870:6496] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:09.319990Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2363 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:09.320519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:09.320783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:09.320820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:09.321109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:09.321150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.321155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.321156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.321157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.321158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.321159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.321160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.321161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.321162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.321167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.321177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.321181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.321184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.321188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.321192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.321195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.321199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.321203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.321209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.321216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.321222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.321227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.321231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.321240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.321244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.321250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.321254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.321258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.321262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.321267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.321272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.321280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.321285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.321301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.321306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.321310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.321337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:09.327273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:09.328440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:09.328453Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:09.328455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:09.328459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:09.328466Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2362 [2:7559343141374819432:9047]
2025-10-09T21:52:09.328479Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:09.328496Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:09.328500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:09.328503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:09.328505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:09.328521Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:09.328528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:09.328536Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:09.328598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.328604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.328606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.328609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:09.328670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:09.328677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.328687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.328696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.328710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:09.328732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343141374819427:9047]
2025-10-09T21:52:09.328738Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343141374819427:9047]
2025-10-09T21:52:09.328740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343141374819427:9047]
2025-10-09T21:52:09.328753Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:09.328762Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:09.328767Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:09.328768Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:09.328769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343141736671975:6496], now have 1 active actors on pipe
2025-10-09T21:52:09.328772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343141736671782:2432], now have 1 active actors on pipe
2025-10-09T21:52:09.328781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:09.328798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:09.328805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:09.328828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343141374819427:9047]
2025-10-09T21:52:09.328838Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:09.328823Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2362
2025-10-09T21:52:09.328824Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2362
2025-10-09T21:52:09.328856Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:09.328887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.328896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.328925Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|930a2d6-89403e9-4e579413-ae001ba7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:09.328936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:09.328942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:09.328944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:09.328956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:09.328970Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:09.328988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:09.329304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.329313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.329332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:09.329438Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2363 partition: 0 MaxSeqNo: 25 sessionId: src_id|930a2d6-89403e9-4e579413-ae001ba7_0
2025-10-09T21:52:09.329768Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3507675-d3096c13-3e78f4b3-f922b608_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|930a2d6-89403e9-4e579413-ae001ba7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:09.332205Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|930a2d6-89403e9-4e579413-ae001ba7_0] Write session: try to update token
2025-10-09T21:52:09.332222Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|930a2d6-89403e9-4e579413-ae001ba7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:09.340948Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:09.340962Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:09.340972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343141374819427:9047]
2025-10-09T21:52:09.341018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:09.341021Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:09.341202Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:09.341357Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:09.341494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:09.341635Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:09.341828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:09.341868Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:09.345532Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2363 sessionId: src_id|930a2d6-89403e9-4e579413-ae001ba7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:09.348092Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:09.348292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.348311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.348349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:09.348439Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:09.353578Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:09.353592Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:09.353595Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:09.353597Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:09.353607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343141374819427:9047]
2025-10-09T21:52:09.353611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343141374819427:9047]
2025-10-09T21:52:09.353615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343141374819427:9047]
2025-10-09T21:52:09.353654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:09.353657Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:09.353661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.353662Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:09.353665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:09.353666Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:09.353804Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:09.404932Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343141736671987:7496]
2025-10-09T21:52:09.405219Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 2362, pipe: [1:7559343141736671987:7496]
2025-10-09T21:52:09.504498Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:09.511477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.511498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.511639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.511698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.511753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.511809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.511846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.511856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.511928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.511982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.512033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.512090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.512128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.512133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.512192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.512244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.512306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.512355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.512391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.512396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.512459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.512513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.512565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.512618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.512663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.512670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.512760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.512814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.512865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.512914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.512956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.512963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.513018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.513066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.513120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.513174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.513215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.513223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.513272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.513335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.513385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.513436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.513490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.513495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.513545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.513593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.513642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.513696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.513733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.513739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.513795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.513845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.513893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.513946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.513988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.513995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.514061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:09.514113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:09.514164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:09.514217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:09.514258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:09.514265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:09.514312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343141736671987:7496], now have 1 active actors on pipe
2025-10-09T21:52:09.514589Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:09.514592Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:09.514596Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:09.514627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2363 reason: error at write: Disconnected sessionId: src_id|930a2d6-89403e9-4e579413-ae001ba7_0
2025-10-09T21:52:09.514636Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343141736671987:7496]
2025-10-09T21:52:09.514816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2363 sessionId: src_id|930a2d6-89403e9-4e579413-ae001ba7_0 is DEAD
2025-10-09T21:52:09.514927Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|930a2d6-89403e9-4e579413-ae001ba7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:09.514937Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|930a2d6-89403e9-4e579413-ae001ba7_0] Write session will restart in 0.007966s
2025-10-09T21:52:09.514955Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|930a2d6-89403e9-4e579413-ae001ba7_0] Write session: Do CDS request
2025-10-09T21:52:09.514960Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|930a2d6-89403e9-4e579413-ae001ba7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:09.515057Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:09.524496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|930a2d6-89403e9-4e579413-ae001ba7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:09.524555Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:09.524566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2364
2025-10-09T21:52:09.524685Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2364 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:09.524730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2364 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:09.524738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2364 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:09.524741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:09.525081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2364 sessionId:  describe result for acl check
2025-10-09T21:52:09.525109Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:09.525113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:09.525115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:09.525119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343141736672111:6481] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:09.525569Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343141736672111:6481] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:09.527264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:09.527927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343141736672111:6481] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:09.529121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:09.529283Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:09.529300Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:09.529311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:09.529362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:09.529456Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343141374819625:9055]
2025-10-09T21:52:09.530005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:09.530319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:09.530356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:09.530461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343141736672111:6481] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:09.530470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343141736672111:6481] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:09.530473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343141736672111:6481] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:09.530478Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2364 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:09.530637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:09.530685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.530690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.530692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.530693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.530694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.530695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.530696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.530697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.530698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.530702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.530713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.530720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.530724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.530732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.530736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.530743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.530747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.530754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.530760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.530771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.530777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.530787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.530795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.530804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.530808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.530815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.530824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.530829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.530837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.530842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.530851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.530860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.530864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.530872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.530881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.530886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.530916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:09.536638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:09.537612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:09.537625Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:09.537627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:09.537630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:09.537635Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2363 [2:7559343141374819625:9055]
2025-10-09T21:52:09.537649Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:09.537668Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:09.537675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:09.537678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:09.537680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:09.537694Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:09.537703Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:09.537710Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:09.537760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.537767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.537769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.537771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:09.537828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:09.537834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.537845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.537861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.537881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:09.537903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343141736671990:2432], now have 1 active actors on pipe
2025-10-09T21:52:09.537912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343141736672183:6481], now have 1 active actors on pipe
2025-10-09T21:52:09.537919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343141374819620:9055]
2025-10-09T21:52:09.537921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343141374819620:9055]
2025-10-09T21:52:09.537923Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343141374819620:9055]
2025-10-09T21:52:09.537934Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2363
2025-10-09T21:52:09.537935Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:09.537957Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2363
2025-10-09T21:52:09.537948Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:09.537949Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:09.537954Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:09.537956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:09.537963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:09.537966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:09.537984Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343141374819620:9055]
2025-10-09T21:52:09.537987Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:09.537989Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:09.538002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.538011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.538076Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d8d0f964-80a19861-5a78200-fa6e09a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:09.538088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:09.538091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:09.538093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:09.538103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:09.538117Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:09.538138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:09.538264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.538272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.538285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:09.538362Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2364 partition: 0 MaxSeqNo: 25 sessionId: src_id|d8d0f964-80a19861-5a78200-fa6e09a_0
2025-10-09T21:52:09.538644Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|930a2d6-89403e9-4e579413-ae001ba7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d8d0f964-80a19861-5a78200-fa6e09a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:09.541071Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d8d0f964-80a19861-5a78200-fa6e09a_0] Write session: try to update token
2025-10-09T21:52:09.541085Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d8d0f964-80a19861-5a78200-fa6e09a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:09.551057Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:09.551073Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:09.551075Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:09.551077Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:09.551087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343141374819620:9055]
2025-10-09T21:52:09.551092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343141374819620:9055]
2025-10-09T21:52:09.551096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343141374819620:9055]
2025-10-09T21:52:09.551138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:09.551150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.551153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:09.551152Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:09.551158Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:09.551161Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:09.551277Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:09.552723Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2364 sessionId: src_id|d8d0f964-80a19861-5a78200-fa6e09a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:09.554857Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:09.614844Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343141736672193:7496]
2025-10-09T21:52:09.615088Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 2363, pipe: [1:7559343141736672193:7496]
2025-10-09T21:52:09.692026Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:09.692596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.692614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.692657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343141736672193:7496], now have 1 active actors on pipe
2025-10-09T21:52:09.692851Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343141736672193:7496]
2025-10-09T21:52:09.692851Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:09.692886Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:09.692889Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:09.692948Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2364 reason: error at write: Disconnected sessionId: src_id|d8d0f964-80a19861-5a78200-fa6e09a_0
2025-10-09T21:52:09.693043Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:09.693134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2364 sessionId: src_id|d8d0f964-80a19861-5a78200-fa6e09a_0 is DEAD
2025-10-09T21:52:09.693326Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d8d0f964-80a19861-5a78200-fa6e09a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:09.693338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d8d0f964-80a19861-5a78200-fa6e09a_0] Write session will restart in 0.006953s
2025-10-09T21:52:09.693361Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d8d0f964-80a19861-5a78200-fa6e09a_0] Write session: Do CDS request
2025-10-09T21:52:09.693368Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d8d0f964-80a19861-5a78200-fa6e09a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:09.701539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d8d0f964-80a19861-5a78200-fa6e09a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:09.701613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:09.701626Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2365
2025-10-09T21:52:09.701743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2365 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:09.701784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2365 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:09.701800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2365 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:09.701807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:09.702233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2365 sessionId:  describe result for acl check
2025-10-09T21:52:09.702266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:09.702272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:09.702273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:09.702279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343141736672287:6474] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:09.702827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343141736672287:6474] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:09.704943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343141736672287:6474] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:09.705557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:09.707476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:09.707663Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:09.707673Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:09.707686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:09.707753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:09.707753Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343141736672287:6474] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:09.707767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343141736672287:6474] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:09.707770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343141736672287:6474] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:09.707778Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2365 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:09.707797Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343141374819821:9053]
2025-10-09T21:52:09.708249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:09.708561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:09.708596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:09.708885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:09.708939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.708945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.708946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.708947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.708948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.708950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.708951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.708952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.708954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.708958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.708969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.708973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.708977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.708981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.708985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.708989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.708993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.708996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.709003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.709011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.709017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.709024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.709029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.709035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.709043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.709048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.709053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.709058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.709063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.709067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.709072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.709075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.709078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.709081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.709083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.709086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.709102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:09.715150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:09.716194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:09.716208Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:09.716210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:09.716216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:09.716221Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2364 [2:7559343141374819821:9053]
2025-10-09T21:52:09.716231Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:09.716247Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:09.716251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:09.716254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:09.716256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:09.716273Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:09.716279Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:09.716286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:09.716347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.716354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.716356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.716359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:09.716421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:09.716428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.716440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.716452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.716480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:09.716504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343141736672389:6474], now have 1 active actors on pipe
2025-10-09T21:52:09.716512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343141736672195:2432], now have 1 active actors on pipe
2025-10-09T21:52:09.716520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343141374819816:9053]
2025-10-09T21:52:09.716527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343141374819816:9053]
2025-10-09T21:52:09.716528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343141374819816:9053]
2025-10-09T21:52:09.716544Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:09.716563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:09.716562Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:09.716565Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:09.716567Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:09.716579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:09.716583Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:09.716609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343141374819816:9053]
2025-10-09T21:52:09.716581Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2364
2025-10-09T21:52:09.716618Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:09.716619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:09.716590Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2364
2025-10-09T21:52:09.716634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.716639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.716689Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|995f34ed-76542a65-bfe3202e-5543f988_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:09.716700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:09.716705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:09.716711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:09.716722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:09.716738Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:09.716761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:09.716904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.716911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.716926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:09.717004Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2365 partition: 0 MaxSeqNo: 25 sessionId: src_id|995f34ed-76542a65-bfe3202e-5543f988_0
2025-10-09T21:52:09.717348Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d8d0f964-80a19861-5a78200-fa6e09a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|995f34ed-76542a65-bfe3202e-5543f988_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:09.719906Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|995f34ed-76542a65-bfe3202e-5543f988_0] Write session: try to update token
2025-10-09T21:52:09.719923Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|995f34ed-76542a65-bfe3202e-5543f988_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:09.731076Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:09.731093Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:09.731095Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:09.731096Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:09.731104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343141374819816:9053]
2025-10-09T21:52:09.731108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343141374819816:9053]
2025-10-09T21:52:09.731111Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343141374819816:9053]
2025-10-09T21:52:09.731146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:09.731152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.731154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:09.731155Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:09.731161Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:09.731166Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:09.731282Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:09.733817Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2365 sessionId: src_id|995f34ed-76542a65-bfe3202e-5543f988_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:09.735963Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:09.793041Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343141736672400:7496]
2025-10-09T21:52:09.793313Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 2364, pipe: [1:7559343141736672400:7496]
2025-10-09T21:52:09.878552Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:09.879040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.879055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.879093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343141736672400:7496], now have 1 active actors on pipe
2025-10-09T21:52:09.879279Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:09.879323Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:09.879332Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343141736672400:7496]
2025-10-09T21:52:09.879357Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:09.879433Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2365 reason: status is not ok: tablet will be restarted right now sessionId: src_id|995f34ed-76542a65-bfe3202e-5543f988_0
2025-10-09T21:52:09.879611Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2365 sessionId: src_id|995f34ed-76542a65-bfe3202e-5543f988_0 is DEAD
2025-10-09T21:52:09.879753Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|995f34ed-76542a65-bfe3202e-5543f988_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:09.879765Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|995f34ed-76542a65-bfe3202e-5543f988_0] Write session will restart in 0.005733s
2025-10-09T21:52:09.879792Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|995f34ed-76542a65-bfe3202e-5543f988_0] Write session: Do CDS request
2025-10-09T21:52:09.879798Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|995f34ed-76542a65-bfe3202e-5543f988_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:09.886437Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|995f34ed-76542a65-bfe3202e-5543f988_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:09.886529Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:09.886541Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2366
2025-10-09T21:52:09.886639Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2366 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:09.886664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2366 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:09.886686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2366 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:09.886695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:09.887145Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2366 sessionId:  describe result for acl check
2025-10-09T21:52:09.887175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:09.887177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:09.887178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:09.887183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343141736672456:6464] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:09.887638Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343141736672456:6464] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:09.889913Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343141736672456:6464] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:09.891817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:09.893708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:09.893843Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343141736672456:6464] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:09.893855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343141736672456:6464] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:09.893857Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343141736672456:6464] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:09.893864Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2366 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:09.893886Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:09.893895Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:09.893907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:09.893957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:09.893995Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343141374820021:9049]
2025-10-09T21:52:09.894580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:09.894897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:09.894938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:09.895170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:09.895210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.895215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.895216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.895217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.895217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.895218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.895219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.895219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.895220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.895223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.895230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.895232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.895234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.895237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.895241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.895245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.895248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.895252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.895259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.895267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:09.895273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.895276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:09.895279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.895282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:09.895284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.895287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:09.895291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.895293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:09.895296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.895299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:09.895302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.895305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.895307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.895309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.895312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.895315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.895332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:09.901550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:09.902888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:09.902901Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:09.902904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:09.902907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:09.902913Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2365 [2:7559343141374820021:9049]
2025-10-09T21:52:09.902926Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:09.902946Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:09.902949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:09.902953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:09.902955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:09.902972Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:09.902993Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:09.903005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:09.903066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.903076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.903078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.903081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:09.903147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:09.903155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.903168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:09.903185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:09.903204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:09.903229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343141736672597:6464], now have 1 active actors on pipe
2025-10-09T21:52:09.903239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343141736672403:2432], now have 1 active actors on pipe
2025-10-09T21:52:09.903247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343141374820016:9049]
2025-10-09T21:52:09.903258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343141374820016:9049]
2025-10-09T21:52:09.903265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343141374820016:9049]
2025-10-09T21:52:09.903280Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:09.903298Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:09.903304Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:09.903307Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:09.903310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:09.903288Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2365
2025-10-09T21:52:09.903316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:09.903324Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:09.903335Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2365
2025-10-09T21:52:09.903363Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343141374820016:9049]
2025-10-09T21:52:09.903371Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:09.903373Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:09.903385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.903394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.903443Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e4932932-a8e44649-4e5b7f71-20fbee8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:09.903454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:09.903462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:09.903465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:09.903480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:09.903494Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:09.903516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:09.903656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:09.903664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:09.903679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:09.903753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2366 partition: 0 MaxSeqNo: 25 sessionId: src_id|e4932932-a8e44649-4e5b7f71-20fbee8_0
2025-10-09T21:52:09.904062Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|995f34ed-76542a65-bfe3202e-5543f988_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e4932932-a8e44649-4e5b7f71-20fbee8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:09.906431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4932932-a8e44649-4e5b7f71-20fbee8_0] Write session: try to update token
2025-10-09T21:52:09.906447Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4932932-a8e44649-4e5b7f71-20fbee8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:09.917813Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:09.917834Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:09.917836Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:09.917838Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:09.917848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343141374820016:9049]
2025-10-09T21:52:09.917852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343141374820016:9049]
2025-10-09T21:52:09.917856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343141374820016:9049]
2025-10-09T21:52:09.917899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:09.917908Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:09.917912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:09.917915Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:09.917915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:09.917921Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:09.918068Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:09.918358Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2366 sessionId: src_id|e4932932-a8e44649-4e5b7f71-20fbee8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:09.920591Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:09.979408Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343141736672610:7496]
2025-10-09T21:52:09.979642Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 2365, pipe: [1:7559343141736672610:7496]
2025-10-09T21:52:10.066889Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:10.067132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:10.067144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:10.067182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343141736672610:7496], now have 1 active actors on pipe
2025-10-09T21:52:10.067406Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343141736672610:7496]
2025-10-09T21:52:10.067436Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:10.067439Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:10.067491Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2366 reason: error at write: Disconnected sessionId: src_id|e4932932-a8e44649-4e5b7f71-20fbee8_0
2025-10-09T21:52:10.067650Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2366 sessionId: src_id|e4932932-a8e44649-4e5b7f71-20fbee8_0 is DEAD
2025-10-09T21:52:10.067773Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:10.067792Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:10.067822Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4932932-a8e44649-4e5b7f71-20fbee8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:10.067834Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4932932-a8e44649-4e5b7f71-20fbee8_0] Write session will restart in 0.009693s
2025-10-09T21:52:10.067870Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4932932-a8e44649-4e5b7f71-20fbee8_0] Write session: Do CDS request
2025-10-09T21:52:10.067876Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4932932-a8e44649-4e5b7f71-20fbee8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:10.078387Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e4932932-a8e44649-4e5b7f71-20fbee8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:10.078495Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:10.078509Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2367
2025-10-09T21:52:10.078680Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2367 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:10.078711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2367 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:10.078723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2367 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:10.078729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:10.079139Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2367 sessionId:  describe result for acl check
2025-10-09T21:52:10.079170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:10.079177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:10.079179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:10.079185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343146031640069:6442] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:10.079567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:10.079725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343146031640069:6442] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:10.081479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:10.081647Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:10.081662Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:10.081675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:10.081694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343146031640069:6442] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:10.081732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:10.081772Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343145669787514:9044]
2025-10-09T21:52:10.082378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:10.082628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:10.082666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:10.082977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:10.083020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.083026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.083028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.083029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.083030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.083031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.083033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.083034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.083034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.083039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.083050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.083054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.083059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.083062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.083066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.083070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.083074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.083082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.083088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.083095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.083106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.083111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.083115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.083120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.083125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.083130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.083134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.083144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.083147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.083157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.083166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.083174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.083182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.083187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.083192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.083202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.083235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:10.088183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343146031640069:6442] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:10.088203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343146031640069:6442] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:10.088208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343146031640069:6442] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:10.088217Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2367 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:10.089457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:10.090823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:10.090838Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:10.090840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:10.090844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:10.090851Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2366 [2:7559343145669787514:9044]
2025-10-09T21:52:10.090864Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:10.090882Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:10.090886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:10.090890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:10.090892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:10.090909Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:10.090921Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:10.090931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:10.090995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.091003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.091005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.091007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:10.091075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:10.091083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.091095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.091108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.091125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:10.091148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343146031639909:2432], now have 1 active actors on pipe
2025-10-09T21:52:10.091156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343146031640108:6442], now have 1 active actors on pipe
2025-10-09T21:52:10.091164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343145669787509:9044]
2025-10-09T21:52:10.091166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343145669787509:9044]
2025-10-09T21:52:10.091169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343145669787509:9044]
2025-10-09T21:52:10.091181Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:10.091191Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:10.091194Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:10.091195Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:10.091199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:10.091214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:10.091217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:10.091206Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2366
2025-10-09T21:52:10.091251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343145669787509:9044]
2025-10-09T21:52:10.091260Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:10.091278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:10.091275Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2366
2025-10-09T21:52:10.091320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:10.091328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:10.091353Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|48cddf2-12f6eebc-4db4da5b-c01b78af_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:10.091366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:10.091373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:10.091375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:10.091383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:10.091394Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:10.091410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:10.091625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:10.091634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:10.091650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:10.091760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2367 partition: 0 MaxSeqNo: 25 sessionId: src_id|48cddf2-12f6eebc-4db4da5b-c01b78af_0
2025-10-09T21:52:10.092098Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e4932932-a8e44649-4e5b7f71-20fbee8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|48cddf2-12f6eebc-4db4da5b-c01b78af_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:10.094609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48cddf2-12f6eebc-4db4da5b-c01b78af_0] Write session: try to update token
2025-10-09T21:52:10.094626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48cddf2-12f6eebc-4db4da5b-c01b78af_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:10.105185Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:10.105202Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:10.105204Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:10.105205Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:10.105214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343145669787509:9044]
2025-10-09T21:52:10.105220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343145669787509:9044]
2025-10-09T21:52:10.105223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343145669787509:9044]
2025-10-09T21:52:10.105267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:10.105266Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:10.105272Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:10.105275Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:10.105280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.105282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:10.105422Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:10.107885Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2367 sessionId: src_id|48cddf2-12f6eebc-4db4da5b-c01b78af_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:10.110204Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:10.167771Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343146031640115:7496]
2025-10-09T21:52:10.168023Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 2366, pipe: [1:7559343146031640115:7496]
2025-10-09T21:52:10.254208Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:10.254453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:10.254465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:10.254507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343146031640115:7496], now have 1 active actors on pipe
2025-10-09T21:52:10.254772Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343146031640115:7496]
2025-10-09T21:52:10.254802Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:10.254852Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:10.254869Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:10.254972Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:10.254963Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2367 reason: error at write: Disconnected sessionId: src_id|48cddf2-12f6eebc-4db4da5b-c01b78af_0
2025-10-09T21:52:10.255212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2367 sessionId: src_id|48cddf2-12f6eebc-4db4da5b-c01b78af_0 is DEAD
2025-10-09T21:52:10.255360Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48cddf2-12f6eebc-4db4da5b-c01b78af_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:10.255374Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48cddf2-12f6eebc-4db4da5b-c01b78af_0] Write session will restart in 0.007267s
2025-10-09T21:52:10.255404Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48cddf2-12f6eebc-4db4da5b-c01b78af_0] Write session: Do CDS request
2025-10-09T21:52:10.255412Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48cddf2-12f6eebc-4db4da5b-c01b78af_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:10.263528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48cddf2-12f6eebc-4db4da5b-c01b78af_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:10.263587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:10.263600Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2368
2025-10-09T21:52:10.263730Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2368 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:10.263772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2368 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:10.263776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2368 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:10.263780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:10.264188Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2368 sessionId:  describe result for acl check
2025-10-09T21:52:10.264216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:10.264223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:10.264224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:10.264228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343146031640217:6422] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:10.264650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343146031640217:6422] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:10.266882Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343146031640217:6422] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:10.267381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:10.269249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:10.269365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343146031640217:6422] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:10.269375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343146031640217:6422] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:10.269387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343146031640217:6422] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:10.269392Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2368 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:10.269434Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:10.269446Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:10.269459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:10.269523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:10.269560Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343145669787712:9042]
2025-10-09T21:52:10.270161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:10.270476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:10.270520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:10.270771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:10.270812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.270818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.270819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.270820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.270821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.270821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.270822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.270822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.270823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.270826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.270835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.270839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.270842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.270845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.270849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.270852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.270855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.270858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.270864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.270872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.270878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.270884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.270888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.270900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.270909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.270925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.270930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.270935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.270945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.270948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.270951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.270954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.270957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.270964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.270967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.270971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.270988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:10.277120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:10.278278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:10.278291Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:10.278293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:10.278297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:10.278302Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2367 [2:7559343145669787712:9042]
2025-10-09T21:52:10.278314Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:10.278331Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:10.278339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:10.278343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:10.278345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:10.278361Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:10.278366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:10.278373Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:10.278423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.278429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.278432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.278435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:10.278494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:10.278502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.278514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.278529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.278548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:10.278575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343146031640317:6422], now have 1 active actors on pipe
2025-10-09T21:52:10.278578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343146031640121:2432], now have 1 active actors on pipe
2025-10-09T21:52:10.278584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343145669787707:9042]
2025-10-09T21:52:10.278586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343145669787707:9042]
2025-10-09T21:52:10.278588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343145669787707:9042]
2025-10-09T21:52:10.278599Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:10.278616Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:10.278617Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:10.278619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:10.278622Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:10.278630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:10.278633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:10.278631Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2367
2025-10-09T21:52:10.278663Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2367
2025-10-09T21:52:10.278656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343145669787707:9042]
2025-10-09T21:52:10.278667Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:10.278677Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:10.278683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:10.278687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:10.278736Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|444b58de-d9e66bf1-11428dd2-e7bad460_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:10.278748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:10.278755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:10.278757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:10.278766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:10.278780Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:10.278802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:10.278968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:10.278976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:10.278993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:10.279226Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2368 partition: 0 MaxSeqNo: 25 sessionId: src_id|444b58de-d9e66bf1-11428dd2-e7bad460_0
2025-10-09T21:52:10.279599Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48cddf2-12f6eebc-4db4da5b-c01b78af_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|444b58de-d9e66bf1-11428dd2-e7bad460_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:10.282008Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|444b58de-d9e66bf1-11428dd2-e7bad460_0] Write session: try to update token
2025-10-09T21:52:10.282024Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|444b58de-d9e66bf1-11428dd2-e7bad460_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:10.292194Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:10.292212Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:10.292214Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:10.292216Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:10.292225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343145669787707:9042]
2025-10-09T21:52:10.292230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343145669787707:9042]
2025-10-09T21:52:10.292234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343145669787707:9042]
2025-10-09T21:52:10.292278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:10.292280Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:10.292285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.292286Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:10.292288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:10.292290Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:10.292424Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:10.295189Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2368 sessionId: src_id|444b58de-d9e66bf1-11428dd2-e7bad460_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:10.298058Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:10.355193Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343146031640356:7496]
2025-10-09T21:52:10.355500Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 2367, pipe: [1:7559343146031640356:7496]
2025-10-09T21:52:10.437473Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:10.437792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:10.437809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:10.437850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343146031640356:7496], now have 1 active actors on pipe
2025-10-09T21:52:10.438052Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:10.438101Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343146031640356:7496]
2025-10-09T21:52:10.438106Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:10.438110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:10.438202Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2368 reason: error at write: Disconnected sessionId: src_id|444b58de-d9e66bf1-11428dd2-e7bad460_0
2025-10-09T21:52:10.438311Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:10.438373Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2368 sessionId: src_id|444b58de-d9e66bf1-11428dd2-e7bad460_0 is DEAD
2025-10-09T21:52:10.438513Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|444b58de-d9e66bf1-11428dd2-e7bad460_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:10.438522Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|444b58de-d9e66bf1-11428dd2-e7bad460_0] Write session will restart in 0.006049s
2025-10-09T21:52:10.438548Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|444b58de-d9e66bf1-11428dd2-e7bad460_0] Write session: Do CDS request
2025-10-09T21:52:10.438555Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|444b58de-d9e66bf1-11428dd2-e7bad460_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:10.445456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|444b58de-d9e66bf1-11428dd2-e7bad460_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:10.445543Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:10.445557Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2369
2025-10-09T21:52:10.445706Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2369 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:10.445747Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2369 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:10.445753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2369 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:10.445756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:10.446306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2369 sessionId:  describe result for acl check
2025-10-09T21:52:10.446336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:10.446343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:10.446345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:10.446350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343146031640403:6395] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:10.446914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343146031640403:6395] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:10.449724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343146031640403:6395] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:10.451017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:10.452750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:10.452947Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:10.452957Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:10.452969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:10.453025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:10.453060Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343145669787949:9028]
2025-10-09T21:52:10.453417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343146031640403:6395] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:10.453439Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343146031640403:6395] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:10.453444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343146031640403:6395] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:10.453450Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2369 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:10.453487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:10.453734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:10.453772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:10.454058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:10.454119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.454126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.454128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.454129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.454130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.454131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.454133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.454134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.454135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.454140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.454151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.454155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.454159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.454163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.454167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.454171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.454175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.454179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.454186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.454193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.454202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.454209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.454218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.454223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.454227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.454232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.454237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.454242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.454247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.454257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.454266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.454272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.454281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.454285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.454293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.454298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.454331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:10.460450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:10.461578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:10.461591Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:10.461593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:10.461596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:10.461600Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2368 [2:7559343145669787949:9028]
2025-10-09T21:52:10.461609Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:10.461622Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:10.461624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:10.461625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:10.461627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:10.461639Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:10.461646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:10.461652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:10.461696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.461701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.461702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.461704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:10.461753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:10.461759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.461766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.461773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.461782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:10.461798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343146031640555:6395], now have 1 active actors on pipe
2025-10-09T21:52:10.461801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343146031640358:2432], now have 1 active actors on pipe
2025-10-09T21:52:10.461806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343145669787944:9028]
2025-10-09T21:52:10.461811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343145669787944:9028]
2025-10-09T21:52:10.461813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343145669787944:9028]
2025-10-09T21:52:10.461823Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:10.461836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:10.461840Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:10.461843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:10.461844Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:10.461844Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:10.461847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:10.461866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343145669787944:9028]
2025-10-09T21:52:10.461877Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:10.461878Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:10.461908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:10.461913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:10.461873Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2368
2025-10-09T21:52:10.461902Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2368
2025-10-09T21:52:10.461945Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a1fb308-db901eef-57dae473-490e7fc6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:10.461955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:10.461959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:10.461961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:10.461967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:10.461978Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:10.461989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:10.462175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:10.462183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:10.462194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:10.462316Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2369 partition: 0 MaxSeqNo: 25 sessionId: src_id|a1fb308-db901eef-57dae473-490e7fc6_0
2025-10-09T21:52:10.462631Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|444b58de-d9e66bf1-11428dd2-e7bad460_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a1fb308-db901eef-57dae473-490e7fc6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:10.465073Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a1fb308-db901eef-57dae473-490e7fc6_0] Write session: try to update token
2025-10-09T21:52:10.465091Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a1fb308-db901eef-57dae473-490e7fc6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:10.475779Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:10.475793Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:10.475795Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:10.475797Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:10.475804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343145669787944:9028]
2025-10-09T21:52:10.475809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343145669787944:9028]
2025-10-09T21:52:10.475813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343145669787944:9028]
2025-10-09T21:52:10.475857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:10.475856Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:10.475860Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:10.475862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.475862Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:10.475864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:10.475980Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:10.477765Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2369 sessionId: src_id|a1fb308-db901eef-57dae473-490e7fc6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:10.480055Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:10.538499Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343146031640567:7496]
2025-10-09T21:52:10.538764Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 2368, pipe: [1:7559343146031640567:7496]
2025-10-09T21:52:10.620005Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:10.620472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:10.620487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:10.620527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343146031640567:7496], now have 1 active actors on pipe
2025-10-09T21:52:10.620735Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:10.620772Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:10.620774Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:10.620762Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:10.620826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2369 reason: error at write: Disconnected sessionId: src_id|a1fb308-db901eef-57dae473-490e7fc6_0
2025-10-09T21:52:10.620834Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343146031640567:7496]
2025-10-09T21:52:10.620974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2369 sessionId: src_id|a1fb308-db901eef-57dae473-490e7fc6_0 is DEAD
2025-10-09T21:52:10.621120Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1fb308-db901eef-57dae473-490e7fc6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:10.621132Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1fb308-db901eef-57dae473-490e7fc6_0] Write session will restart in 0.006421s
2025-10-09T21:52:10.621159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1fb308-db901eef-57dae473-490e7fc6_0] Write session: Do CDS request
2025-10-09T21:52:10.621166Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1fb308-db901eef-57dae473-490e7fc6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:10.628543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a1fb308-db901eef-57dae473-490e7fc6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:10.628565Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:10.628577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2370
2025-10-09T21:52:10.628741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2370 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:10.628766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2370 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:10.628774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2370 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:10.628777Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:10.629272Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2370 sessionId:  describe result for acl check
2025-10-09T21:52:10.629369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:10.629377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:10.629379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:10.629384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343146031640614:6391] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:10.629879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343146031640614:6391] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:10.632132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343146031640614:6391] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:10.634216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343146031640614:6391] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:10.634229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343146031640614:6391] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:10.634232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343146031640614:6391] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:10.634241Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2370 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:10.634354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:10.636123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:10.636287Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:10.636296Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:10.636307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:10.636357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:10.636396Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343145669788150:9026]
2025-10-09T21:52:10.636993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:10.637226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:10.637265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:10.637556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:10.637592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.637597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.637599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.637600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.637601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.637602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.637603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.637603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.637604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.637609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.637620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.637624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.637628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.637632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.637636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.637640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.637643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.637647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.637653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.637661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.637667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.637672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.637677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.637683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.637693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.637698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.637707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.637713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.637722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.637727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.637734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.637737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.637739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.637742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.637745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.637751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.637769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:10.644195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:10.645370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:10.645385Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:10.645387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:10.645391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:10.645397Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2369 [2:7559343145669788150:9026]
2025-10-09T21:52:10.645414Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:10.645431Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:10.645439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:10.645442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:10.645444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:10.645461Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:10.645471Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:10.645484Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:10.645539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.645548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.645550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.645552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:10.645617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:10.645625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.645638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.645650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.645672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:10.645697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343146031640766:6391], now have 1 active actors on pipe
2025-10-09T21:52:10.645704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343146031640569:2432], now have 1 active actors on pipe
2025-10-09T21:52:10.645711Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343145669788144:9026]
2025-10-09T21:52:10.645718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343145669788144:9026]
2025-10-09T21:52:10.645720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343145669788144:9026]
2025-10-09T21:52:10.645732Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:10.645749Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:10.645752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:10.645753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:10.645755Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:10.645766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:10.645769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:10.645793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343145669788144:9026]
2025-10-09T21:52:10.645801Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:10.645790Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2369
2025-10-09T21:52:10.645825Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:10.645835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:10.645838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:10.645790Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2369
2025-10-09T21:52:10.645862Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|30f80456-df528f8b-d8900387-fd719b37_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:10.645872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:10.645875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:10.645877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:10.645884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:10.645896Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:10.645906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:10.646075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:10.646082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:10.646094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:10.646173Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2370 partition: 0 MaxSeqNo: 25 sessionId: src_id|30f80456-df528f8b-d8900387-fd719b37_0
2025-10-09T21:52:10.646569Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a1fb308-db901eef-57dae473-490e7fc6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|30f80456-df528f8b-d8900387-fd719b37_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:10.649040Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|30f80456-df528f8b-d8900387-fd719b37_0] Write session: try to update token
2025-10-09T21:52:10.649055Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|30f80456-df528f8b-d8900387-fd719b37_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:10.660294Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:10.660313Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:10.660315Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:10.660318Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:10.660328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343145669788144:9026]
2025-10-09T21:52:10.660336Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343145669788144:9026]
2025-10-09T21:52:10.660340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343145669788144:9026]
2025-10-09T21:52:10.660388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:10.660398Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:10.660400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.660403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:10.660405Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:10.660409Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:10.660537Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:10.663198Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2370 sessionId: src_id|30f80456-df528f8b-d8900387-fd719b37_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:10.665834Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:10.720800Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343146031640779:7496]
2025-10-09T21:52:10.721102Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 2369, pipe: [1:7559343146031640779:7496]
2025-10-09T21:52:10.804655Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:10.805134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:10.805150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:10.805194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343146031640779:7496], now have 1 active actors on pipe
2025-10-09T21:52:10.805439Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343146031640779:7496]
2025-10-09T21:52:10.805466Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:10.805468Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:10.805485Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:10.805520Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2370 reason: error at write: Disconnected sessionId: src_id|30f80456-df528f8b-d8900387-fd719b37_0
2025-10-09T21:52:10.805673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2370 sessionId: src_id|30f80456-df528f8b-d8900387-fd719b37_0 is DEAD
2025-10-09T21:52:10.805829Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30f80456-df528f8b-d8900387-fd719b37_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:10.805841Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30f80456-df528f8b-d8900387-fd719b37_0] Write session will restart in 0.008130s
2025-10-09T21:52:10.805867Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30f80456-df528f8b-d8900387-fd719b37_0] Write session: Do CDS request
2025-10-09T21:52:10.805874Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30f80456-df528f8b-d8900387-fd719b37_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:10.806019Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:10.815523Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|30f80456-df528f8b-d8900387-fd719b37_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:10.815628Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:10.815643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2371
2025-10-09T21:52:10.815829Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2371 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:10.815867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2371 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:10.815873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2371 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:10.815877Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:10.816336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2371 sessionId:  describe result for acl check
2025-10-09T21:52:10.816375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:10.816381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:10.816383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:10.816389Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343146031640935:6373] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:10.816971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343146031640935:6373] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:10.817752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:10.819376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343146031640935:6373] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:10.819508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:10.819704Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:10.819714Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:10.819726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:10.819782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:10.819817Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343145669788354:9020]
2025-10-09T21:52:10.820202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:10.820438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:10.820465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:10.820693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:10.820731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.820735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.820736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.820737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.820737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.820738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.820738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.820739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.820740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.820743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.820750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.820752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.820754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.820756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.820758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.820760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.820762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.820764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.820769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.820775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:10.820781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.820787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:10.820792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.820798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:10.820802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.820807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:10.820812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.820817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:10.820819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.820822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:10.820825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.820828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.820831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.820833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.820836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.820840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.820858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:10.825687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343146031640935:6373] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:10.825699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343146031640935:6373] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:10.825702Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343146031640935:6373] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:10.825709Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2371 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:10.826986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:10.828109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:10.828120Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:10.828122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:10.828124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:10.828129Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2370 [2:7559343145669788354:9020]
2025-10-09T21:52:10.828138Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:10.828150Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:10.828160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:10.828163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:10.828164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:10.828177Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:10.828184Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:10.828190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:10.828234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.828238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.828239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.828241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:10.828295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:10.828303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.828313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:10.828336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:10.828352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:10.828370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343146031640781:2432], now have 1 active actors on pipe
2025-10-09T21:52:10.828377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343146031640984:6373], now have 1 active actors on pipe
2025-10-09T21:52:10.828382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343145669788349:9020]
2025-10-09T21:52:10.828384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343145669788349:9020]
2025-10-09T21:52:10.828386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343145669788349:9020]
2025-10-09T21:52:10.828395Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:10.828409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:10.828421Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:10.828426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:10.828427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:10.828407Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2370
2025-10-09T21:52:10.828411Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2370
2025-10-09T21:52:10.828431Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:10.828434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:10.828445Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343145669788349:9020]
2025-10-09T21:52:10.828447Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:10.828457Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:10.828463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:10.828471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:10.828506Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d7c3cbfd-7fff1498-e83f984b-f7a9edf6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:10.828515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:10.828518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:10.828519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:10.828527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:10.828534Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:10.828551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:10.828697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:10.828703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:10.828715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:10.828796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2371 partition: 0 MaxSeqNo: 25 sessionId: src_id|d7c3cbfd-7fff1498-e83f984b-f7a9edf6_0
2025-10-09T21:52:10.829078Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30f80456-df528f8b-d8900387-fd719b37_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d7c3cbfd-7fff1498-e83f984b-f7a9edf6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:10.831410Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7c3cbfd-7fff1498-e83f984b-f7a9edf6_0] Write session: try to update token
2025-10-09T21:52:10.831426Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7c3cbfd-7fff1498-e83f984b-f7a9edf6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:10.843691Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:10.843707Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:10.843709Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:10.843710Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:10.843718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343145669788349:9020]
2025-10-09T21:52:10.843688Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2371 sessionId: src_id|d7c3cbfd-7fff1498-e83f984b-f7a9edf6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:10.843721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343145669788349:9020]
2025-10-09T21:52:10.843724Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343145669788349:9020]
2025-10-09T21:52:10.843755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:10.843760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:10.843762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:10.843765Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:10.843772Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:10.843787Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:10.843921Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:10.845998Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:10.905940Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343146031640992:7496]
2025-10-09T21:52:10.906214Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 2370, pipe: [1:7559343146031640992:7496]
2025-10-09T21:52:10.990751Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:10.991215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:10.991233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:10.991282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343146031640992:7496], now have 1 active actors on pipe
2025-10-09T21:52:10.991458Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:10.991488Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:10.991512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:10.991536Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:10.991574Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2371 reason: error at write: Disconnected sessionId: src_id|d7c3cbfd-7fff1498-e83f984b-f7a9edf6_0
2025-10-09T21:52:10.991582Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343146031640992:7496]
2025-10-09T21:52:10.991724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2371 sessionId: src_id|d7c3cbfd-7fff1498-e83f984b-f7a9edf6_0 is DEAD
2025-10-09T21:52:10.991884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7c3cbfd-7fff1498-e83f984b-f7a9edf6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:10.991896Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7c3cbfd-7fff1498-e83f984b-f7a9edf6_0] Write session will restart in 0.008635s
2025-10-09T21:52:10.991924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7c3cbfd-7fff1498-e83f984b-f7a9edf6_0] Write session: Do CDS request
2025-10-09T21:52:10.991930Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7c3cbfd-7fff1498-e83f984b-f7a9edf6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:11.001599Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7c3cbfd-7fff1498-e83f984b-f7a9edf6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:11.001657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:11.001674Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2372
2025-10-09T21:52:11.001862Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2372 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:11.001900Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2372 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:11.001913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2372 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:11.001917Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:11.002344Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2372 sessionId:  describe result for acl check
2025-10-09T21:52:11.002383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:11.002392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:11.002394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:11.002398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343150326608388:6350] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:11.002859Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343150326608388:6350] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:11.004974Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343150326608388:6350] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:11.006252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:11.008259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:11.008440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343150326608388:6350] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:11.008455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343150326608388:6350] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:11.008460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343150326608388:6350] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:11.008470Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2372 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:11.008526Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:11.008538Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:11.008555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:11.008621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:11.008662Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343149964755853:9012]
2025-10-09T21:52:11.009363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:11.009684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:11.009729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:11.010085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:11.010146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.010152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.010153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.010155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.010156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.010157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.010181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.010182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.010183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.010190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.010201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.010206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.010216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.010221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.010225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.010232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.010240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.010245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.010257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.010266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.010278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.010288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.010298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.010305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.010315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.010325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.010329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.010335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.010341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.010347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.010357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.010364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.010369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.010379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.010389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.010396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.010432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:11.016801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:11.017929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:11.017948Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:11.017951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:11.017956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:11.017964Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2371 [2:7559343149964755853:9012]
2025-10-09T21:52:11.017983Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:11.018009Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:11.018018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:11.018022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:11.018024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:11.018062Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:11.018068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:11.018077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:11.018141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.018150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.018153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.018156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:11.018236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:11.018244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.018257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.018271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.018296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:11.018323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343150326608492:6350], now have 1 active actors on pipe
2025-10-09T21:52:11.018332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343146031640998:2432], now have 1 active actors on pipe
2025-10-09T21:52:11.018341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343149964755848:9012]
2025-10-09T21:52:11.018348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343149964755848:9012]
2025-10-09T21:52:11.018352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343149964755848:9012]
2025-10-09T21:52:11.018368Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:11.018387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:11.018386Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:11.018389Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:11.018392Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:11.018407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:11.018367Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2371
2025-10-09T21:52:11.018416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:11.018380Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2371
2025-10-09T21:52:11.018464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.018474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.018491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343149964755848:9012]
2025-10-09T21:52:11.018503Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:11.018513Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:11.018535Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8eefac68-1796699b-e8c25ac8-c0067c7e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:11.018548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:11.018559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:11.018562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:11.018574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:11.018586Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:11.018617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:11.018805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.018816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.018833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:11.018933Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2372 partition: 0 MaxSeqNo: 25 sessionId: src_id|8eefac68-1796699b-e8c25ac8-c0067c7e_0
2025-10-09T21:52:11.019351Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7c3cbfd-7fff1498-e83f984b-f7a9edf6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8eefac68-1796699b-e8c25ac8-c0067c7e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:11.021838Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8eefac68-1796699b-e8c25ac8-c0067c7e_0] Write session: try to update token
2025-10-09T21:52:11.021858Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8eefac68-1796699b-e8c25ac8-c0067c7e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:11.032663Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:11.032683Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:11.032686Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:11.032688Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:11.032698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343149964755848:9012]
2025-10-09T21:52:11.032702Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343149964755848:9012]
2025-10-09T21:52:11.032706Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343149964755848:9012]
2025-10-09T21:52:11.032752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:11.032760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.032763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:11.032787Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:11.032799Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:11.032806Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:11.032912Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:11.034862Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2372 sessionId: src_id|8eefac68-1796699b-e8c25ac8-c0067c7e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:11.037283Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:11.091970Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343150326608503:7496]
2025-10-09T21:52:11.092230Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 2371, pipe: [1:7559343150326608503:7496]
2025-10-09T21:52:11.201177Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:11.201440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.201453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.201495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343150326608503:7496], now have 1 active actors on pipe
2025-10-09T21:52:11.201660Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:11.201690Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:11.201701Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343150326608503:7496]
2025-10-09T21:52:11.201719Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:11.201722Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:11.201783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2372 reason: error at write: Disconnected sessionId: src_id|8eefac68-1796699b-e8c25ac8-c0067c7e_0
2025-10-09T21:52:11.201959Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2372 sessionId: src_id|8eefac68-1796699b-e8c25ac8-c0067c7e_0 is DEAD
2025-10-09T21:52:11.202146Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8eefac68-1796699b-e8c25ac8-c0067c7e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:11.202161Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8eefac68-1796699b-e8c25ac8-c0067c7e_0] Write session will restart in 0.006962s
2025-10-09T21:52:11.202188Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8eefac68-1796699b-e8c25ac8-c0067c7e_0] Write session: Do CDS request
2025-10-09T21:52:11.202195Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8eefac68-1796699b-e8c25ac8-c0067c7e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:11.210476Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8eefac68-1796699b-e8c25ac8-c0067c7e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:11.210569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:11.210585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2373
2025-10-09T21:52:11.210735Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2373 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:11.210783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2373 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:11.210796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2373 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:11.210800Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:11.211318Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2373 sessionId:  describe result for acl check
2025-10-09T21:52:11.211361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:11.211367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:11.211368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:11.211374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343150326608591:6323] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:11.211887Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343150326608591:6323] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:11.214460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343150326608591:6323] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:11.215248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:11.217458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:11.217589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343150326608591:6323] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:11.217601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343150326608591:6323] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:11.217604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343150326608591:6323] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:11.217612Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2373 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:11.217646Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:11.217655Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:11.217667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:11.217717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:11.217759Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343149964756055:8998]
2025-10-09T21:52:11.218255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:11.218541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:11.218584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:11.218838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:11.218889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.218895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.218896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.218897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.218899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.218900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.218902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.218903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.218904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.218909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.218920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.218924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.218929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.218933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.218937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.218943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.218947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.218954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.218960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.218968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.218978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.218982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.218991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.218997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.219001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.219006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.219011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.219016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.219020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.219030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.219035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.219044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.219048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.219052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.219056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.219061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.219087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:11.225422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:11.226487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:11.226498Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:11.226500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:11.226503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:11.226508Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2372 [2:7559343149964756055:8998]
2025-10-09T21:52:11.226519Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:11.226536Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:11.226538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:11.226540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:11.226542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:11.226555Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:11.226563Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:11.226569Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:11.226619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.226625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.226626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.226628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:11.226684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:11.226690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.226699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.226707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.226720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:11.226746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343150326608707:6323], now have 1 active actors on pipe
2025-10-09T21:52:11.226757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343150326608506:2432], now have 1 active actors on pipe
2025-10-09T21:52:11.226765Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343149964756050:8998]
2025-10-09T21:52:11.226767Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343149964756050:8998]
2025-10-09T21:52:11.226776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343149964756050:8998]
2025-10-09T21:52:11.226786Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:11.226799Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:11.226806Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:11.226810Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:11.226811Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:11.226812Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:11.226814Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:11.226839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343149964756050:8998]
2025-10-09T21:52:11.226845Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:11.226848Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:11.226812Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2372
2025-10-09T21:52:11.226868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.226875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.226826Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2372
2025-10-09T21:52:11.226906Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c4a4ef3f-5a07363c-5b9baea7-fdbfb1a0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:11.226915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:11.226919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:11.226922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:11.226929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:11.226941Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:11.226958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:11.227118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.227127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.227139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:11.227214Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2373 partition: 0 MaxSeqNo: 25 sessionId: src_id|c4a4ef3f-5a07363c-5b9baea7-fdbfb1a0_0
2025-10-09T21:52:11.227568Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8eefac68-1796699b-e8c25ac8-c0067c7e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c4a4ef3f-5a07363c-5b9baea7-fdbfb1a0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:11.229964Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4a4ef3f-5a07363c-5b9baea7-fdbfb1a0_0] Write session: try to update token
2025-10-09T21:52:11.229981Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4a4ef3f-5a07363c-5b9baea7-fdbfb1a0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:11.240909Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:11.240927Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:11.240929Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:11.240931Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:11.240941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343149964756050:8998]
2025-10-09T21:52:11.240946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343149964756050:8998]
2025-10-09T21:52:11.240950Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343149964756050:8998]
2025-10-09T21:52:11.240993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:11.241000Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:11.241005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.241006Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:11.241008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:11.241010Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:11.241140Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:11.242777Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2373 sessionId: src_id|c4a4ef3f-5a07363c-5b9baea7-fdbfb1a0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:11.245043Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:11.301904Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343150326608721:7496]
2025-10-09T21:52:11.302181Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 2372, pipe: [1:7559343150326608721:7496]
2025-10-09T21:52:11.385927Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:11.386321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.386336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.386377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343150326608721:7496], now have 1 active actors on pipe
2025-10-09T21:52:11.386582Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343150326608721:7496]
2025-10-09T21:52:11.386591Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:11.386595Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:11.386609Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:11.386649Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2373 reason: error at write: Disconnected sessionId: src_id|c4a4ef3f-5a07363c-5b9baea7-fdbfb1a0_0
2025-10-09T21:52:11.386690Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:11.386844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2373 sessionId: src_id|c4a4ef3f-5a07363c-5b9baea7-fdbfb1a0_0 is DEAD
2025-10-09T21:52:11.386988Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4a4ef3f-5a07363c-5b9baea7-fdbfb1a0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:11.387000Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4a4ef3f-5a07363c-5b9baea7-fdbfb1a0_0] Write session will restart in 0.008682s
2025-10-09T21:52:11.387028Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4a4ef3f-5a07363c-5b9baea7-fdbfb1a0_0] Write session: Do CDS request
2025-10-09T21:52:11.387036Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4a4ef3f-5a07363c-5b9baea7-fdbfb1a0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:11.396436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c4a4ef3f-5a07363c-5b9baea7-fdbfb1a0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:11.396482Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:11.396494Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2374
2025-10-09T21:52:11.396629Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2374 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:11.396663Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2374 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:11.396681Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2374 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:11.396689Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:11.397063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2374 sessionId:  describe result for acl check
2025-10-09T21:52:11.397096Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:11.397101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:11.397102Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:11.397106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343150326608881:6296] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:11.397570Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343150326608881:6296] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:11.399328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:11.399833Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343150326608881:6296] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:11.401314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:11.401560Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:11.401574Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:11.401586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:11.401643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:11.401680Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343149964756298:8988]
2025-10-09T21:52:11.402001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343150326608881:6296] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:11.402013Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343150326608881:6296] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:11.402017Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343150326608881:6296] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:11.402024Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2374 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:11.402279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:11.402548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:11.402586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:11.402886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:11.402931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.402936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.402937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.402939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.402940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.402941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.402942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.402943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.402944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.402949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.402959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.402964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.402972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.402976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.402980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.402986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.402990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.402994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.403000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.403011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.403022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.403032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.403039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.403044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.403051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.403056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.403063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.403068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.403074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.403080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.403089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.403094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.403098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.403109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.403113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.403118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.403148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:11.409130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:11.410371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:11.410384Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:11.410386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:11.410389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:11.410395Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2373 [2:7559343149964756298:8988]
2025-10-09T21:52:11.410409Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:11.410426Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:11.410433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:11.410436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:11.410439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:11.410453Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:11.410463Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:11.410474Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:11.410529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.410535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.410538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.410541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:11.410607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:11.410614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.410625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.410641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.410663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:11.410687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343150326608953:6296], now have 1 active actors on pipe
2025-10-09T21:52:11.410695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343150326608754:2432], now have 1 active actors on pipe
2025-10-09T21:52:11.410703Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343149964756293:8988]
2025-10-09T21:52:11.410709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343149964756293:8988]
2025-10-09T21:52:11.410711Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343149964756293:8988]
2025-10-09T21:52:11.410723Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:11.410737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:11.410757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:11.410756Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:11.410759Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:11.410760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:11.410764Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:11.410785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343149964756293:8988]
2025-10-09T21:52:11.410760Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2373
2025-10-09T21:52:11.410793Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:11.410761Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2373
2025-10-09T21:52:11.410794Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:11.410821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.410831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.410863Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4c47c6b1-b21ddf96-f7266fe9-a0fa43b0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:11.410876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:11.410880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:11.410883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:11.410893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:11.410907Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:11.410932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:11.411067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.411075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.411088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:11.411170Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2374 partition: 0 MaxSeqNo: 25 sessionId: src_id|4c47c6b1-b21ddf96-f7266fe9-a0fa43b0_0
2025-10-09T21:52:11.411451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c4a4ef3f-5a07363c-5b9baea7-fdbfb1a0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4c47c6b1-b21ddf96-f7266fe9-a0fa43b0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:11.413802Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4c47c6b1-b21ddf96-f7266fe9-a0fa43b0_0] Write session: try to update token
2025-10-09T21:52:11.413815Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4c47c6b1-b21ddf96-f7266fe9-a0fa43b0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:11.425755Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:11.425778Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:11.425781Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:11.425783Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:11.425793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343149964756293:8988]
2025-10-09T21:52:11.425798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343149964756293:8988]
2025-10-09T21:52:11.425802Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343149964756293:8988]
2025-10-09T21:52:11.425847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:11.425856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.425859Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:11.425864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:11.425865Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:11.425869Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:11.425997Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:11.426160Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2374 sessionId: src_id|4c47c6b1-b21ddf96-f7266fe9-a0fa43b0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:11.428410Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:11.486708Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343150326608964:7496]
2025-10-09T21:52:11.486971Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 2373, pipe: [1:7559343150326608964:7496]
2025-10-09T21:52:11.570837Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:11.571183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.571197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.571240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343150326608964:7496], now have 1 active actors on pipe
2025-10-09T21:52:11.571423Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:11.571436Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:11.571456Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:11.571504Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2374 reason: error at write: Disconnected sessionId: src_id|4c47c6b1-b21ddf96-f7266fe9-a0fa43b0_0
2025-10-09T21:52:11.571506Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:11.571565Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343150326608964:7496]
2025-10-09T21:52:11.571678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2374 sessionId: src_id|4c47c6b1-b21ddf96-f7266fe9-a0fa43b0_0 is DEAD
2025-10-09T21:52:11.571826Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c47c6b1-b21ddf96-f7266fe9-a0fa43b0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:11.571839Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c47c6b1-b21ddf96-f7266fe9-a0fa43b0_0] Write session will restart in 0.008418s
2025-10-09T21:52:11.571868Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c47c6b1-b21ddf96-f7266fe9-a0fa43b0_0] Write session: Do CDS request
2025-10-09T21:52:11.571875Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c47c6b1-b21ddf96-f7266fe9-a0fa43b0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:11.581491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4c47c6b1-b21ddf96-f7266fe9-a0fa43b0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:11.581572Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:11.581585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2375
2025-10-09T21:52:11.581680Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2375 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:11.581712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2375 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:11.581718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2375 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:11.581721Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:11.582087Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2375 sessionId:  describe result for acl check
2025-10-09T21:52:11.582114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:11.582120Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:11.582121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:11.582128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343150326609077:6274] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:11.582550Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343150326609077:6274] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:11.584761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343150326609077:6274] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:11.585211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:11.586884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:11.586948Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343150326609077:6274] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:11.586963Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343150326609077:6274] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:11.586967Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343150326609077:6274] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:11.586975Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2375 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:11.587063Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:11.587073Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:11.587085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:11.587132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:11.587171Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343149964756503:8986]
2025-10-09T21:52:11.587759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:11.588016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:11.588060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:11.588380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:11.588429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.588437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.588438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.588439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.588440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.588441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.588442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.588443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.588444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.588449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.588459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.588464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.588468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.588472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.588476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.588480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.588484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.588488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.588494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.588502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.588509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.588514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.588519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.588524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.588529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.588534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.588538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.588543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.588548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.588553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.588558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.588563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.588572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.588577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.588586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.588592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.588626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:11.594630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:11.595881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:11.595893Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:11.595896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:11.595900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:11.595906Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2374 [2:7559343149964756503:8986]
2025-10-09T21:52:11.595918Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:11.595933Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:11.595942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:11.595945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:11.595948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:11.595964Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:11.595970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:11.595977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:11.596030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.596038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.596040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.596042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:11.596104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:11.596112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.596124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.596142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.596163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:11.596188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343150326609170:6274], now have 1 active actors on pipe
2025-10-09T21:52:11.596199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343150326608967:2432], now have 1 active actors on pipe
2025-10-09T21:52:11.596206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343149964756498:8986]
2025-10-09T21:52:11.596209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343149964756498:8986]
2025-10-09T21:52:11.596211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343149964756498:8986]
2025-10-09T21:52:11.596223Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:11.596237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:11.596242Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:11.596245Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:11.596247Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:11.596223Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2374
2025-10-09T21:52:11.596251Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:11.596254Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:11.596231Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2374
2025-10-09T21:52:11.596277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343149964756498:8986]
2025-10-09T21:52:11.596289Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:11.596296Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:11.596313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.596321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.596367Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|916dd255-79b5781b-1e79f86-5dd0514f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:11.596379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:11.596383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:11.596389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:11.596398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:11.596407Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:11.596445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:11.596668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.596677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.596694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:11.596833Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2375 partition: 0 MaxSeqNo: 25 sessionId: src_id|916dd255-79b5781b-1e79f86-5dd0514f_0
2025-10-09T21:52:11.597144Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4c47c6b1-b21ddf96-f7266fe9-a0fa43b0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|916dd255-79b5781b-1e79f86-5dd0514f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:11.599571Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|916dd255-79b5781b-1e79f86-5dd0514f_0] Write session: try to update token
2025-10-09T21:52:11.599588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|916dd255-79b5781b-1e79f86-5dd0514f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:11.611426Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:11.611453Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:11.611457Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:11.611460Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:11.611473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343149964756498:8986]
2025-10-09T21:52:11.611480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343149964756498:8986]
2025-10-09T21:52:11.611487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343149964756498:8986]
2025-10-09T21:52:11.611554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:11.611569Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:11.611573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.611576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:11.611588Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:11.611592Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:11.611810Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:11.613255Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2375 sessionId: src_id|916dd255-79b5781b-1e79f86-5dd0514f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:11.615563Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:11.671538Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343150326609181:7496]
2025-10-09T21:52:11.671809Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 2374, pipe: [1:7559343150326609181:7496]
2025-10-09T21:52:11.759966Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:11.760316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.760331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.760369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343150326609181:7496], now have 1 active actors on pipe
2025-10-09T21:52:11.760563Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:11.760611Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:11.760623Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:11.760646Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343150326609181:7496]
2025-10-09T21:52:11.760668Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:11.760697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2375 reason: error at write: Disconnected sessionId: src_id|916dd255-79b5781b-1e79f86-5dd0514f_0
2025-10-09T21:52:11.760861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2375 sessionId: src_id|916dd255-79b5781b-1e79f86-5dd0514f_0 is DEAD
2025-10-09T21:52:11.761031Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|916dd255-79b5781b-1e79f86-5dd0514f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:11.761051Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|916dd255-79b5781b-1e79f86-5dd0514f_0] Write session will restart in 0.008164s
2025-10-09T21:52:11.761082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|916dd255-79b5781b-1e79f86-5dd0514f_0] Write session: Do CDS request
2025-10-09T21:52:11.761089Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|916dd255-79b5781b-1e79f86-5dd0514f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:11.770401Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|916dd255-79b5781b-1e79f86-5dd0514f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:11.770458Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:11.770470Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2376
2025-10-09T21:52:11.770625Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2376 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:11.770659Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2376 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:11.770673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2376 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:11.770680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:11.771127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2376 sessionId:  describe result for acl check
2025-10-09T21:52:11.771176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:11.771418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:11.771430Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:11.771438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343150326609338:6267] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:11.771923Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343150326609338:6267] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:11.772930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:11.773616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343150326609338:6267] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:11.774626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:11.774788Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:11.774799Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:11.774809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:11.774855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:11.774886Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343149964756709:8970]
2025-10-09T21:52:11.775398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:11.775648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:11.775684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:11.775985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:11.776004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343150326609338:6267] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:11.776015Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343150326609338:6267] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:11.776018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343150326609338:6267] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:11.776042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.776025Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2376 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:11.776048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.776049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.776050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.776051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.776053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.776054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.776055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.776056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.776062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.776073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.776077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.776081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.776085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.776089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.776093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.776097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.776101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.776108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.776116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.776126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.776132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.776142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.776148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.776152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.776157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.776162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.776166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.776171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.776176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.776181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.776187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.776191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.776196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.776201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.776206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.776235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:11.782386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:11.783690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:11.783703Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:11.783705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:11.783709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:11.783715Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2375 [2:7559343149964756709:8970]
2025-10-09T21:52:11.783726Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:11.783739Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:11.783741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:11.783743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:11.783744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:11.783756Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:11.783761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:11.783766Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:11.783808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.783815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.783816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.783818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:11.783873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:11.783881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.783892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.783901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.783921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:11.783944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343150326609392:6267], now have 1 active actors on pipe
2025-10-09T21:52:11.783950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343150326609184:2432], now have 1 active actors on pipe
2025-10-09T21:52:11.783955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343149964756704:8970]
2025-10-09T21:52:11.783960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343149964756704:8970]
2025-10-09T21:52:11.783961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343149964756704:8970]
2025-10-09T21:52:11.783973Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:11.783987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:11.783991Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:11.783994Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:11.783996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:11.784004Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:11.784007Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:11.784036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343149964756704:8970]
2025-10-09T21:52:11.784045Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:11.784061Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:11.784047Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2375
2025-10-09T21:52:11.784052Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2375
2025-10-09T21:52:11.784118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.784128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.784158Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|20986afa-1f26a96b-9d147042-73867711_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:11.784171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:11.784179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:11.784182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:11.784192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:11.784207Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:11.784226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:11.784389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.784395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.784406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:11.784489Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2376 partition: 0 MaxSeqNo: 25 sessionId: src_id|20986afa-1f26a96b-9d147042-73867711_0
2025-10-09T21:52:11.784890Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|916dd255-79b5781b-1e79f86-5dd0514f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|20986afa-1f26a96b-9d147042-73867711_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:11.787295Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|20986afa-1f26a96b-9d147042-73867711_0] Write session: try to update token
2025-10-09T21:52:11.787314Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|20986afa-1f26a96b-9d147042-73867711_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:11.800641Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:11.800658Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:11.800667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343149964756704:8970]
2025-10-09T21:52:11.800708Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:11.800714Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:11.800905Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:11.801005Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2376 sessionId: src_id|20986afa-1f26a96b-9d147042-73867711_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:11.801068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:11.801197Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:11.801374Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:11.801665Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:11.801701Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:11.803964Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:11.804170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.804186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.804222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:11.804319Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:11.808150Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:11.808163Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:11.808166Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:11.808168Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:11.808177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343149964756704:8970]
2025-10-09T21:52:11.808181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343149964756704:8970]
2025-10-09T21:52:11.808185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343149964756704:8970]
2025-10-09T21:52:11.808221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:11.808233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.808234Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:11.808238Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:11.808242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:11.808246Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:11.808373Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:11.861016Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343150326609403:7496]
2025-10-09T21:52:11.861345Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 2375, pipe: [1:7559343150326609403:7496]
2025-10-09T21:52:11.955572Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:11.962885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.962908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.963059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:11.963128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:11.963204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:11.963276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:11.963368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:11.963376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:11.963436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:11.963487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:11.963543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:11.963602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:11.963642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:11.963650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:11.963705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:11.963754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:11.963806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:11.963858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:11.963899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:11.963904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:11.963989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:11.964044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:11.964108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:11.964181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:11.964225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:11.964232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:11.964400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:11.964455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:11.964507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:11.964571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:11.964620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:11.964629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:11.964688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:11.964740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:11.964796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:11.964862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:11.964916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:11.964922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:11.965086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:11.965138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:11.965189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:11.965240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:11.965279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:11.965298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:11.965357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:11.965409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:11.965457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:11.965508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:11.965550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:11.965557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:11.965617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:11.965664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:11.965721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:11.965777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:11.965819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:11.965826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:11.965892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:11.965956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:11.966018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:11.966091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:11.966130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:11.966138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:11.966185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343150326609403:7496], now have 1 active actors on pipe
2025-10-09T21:52:11.966512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:11.966518Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343150326609403:7496]
2025-10-09T21:52:11.966530Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:11.966568Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2376 reason: error at write: Disconnected sessionId: src_id|20986afa-1f26a96b-9d147042-73867711_0
2025-10-09T21:52:11.966675Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:11.966806Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2376 sessionId: src_id|20986afa-1f26a96b-9d147042-73867711_0 is DEAD
2025-10-09T21:52:11.966965Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20986afa-1f26a96b-9d147042-73867711_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:11.966989Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20986afa-1f26a96b-9d147042-73867711_0] Write session will restart in 0.006675s
2025-10-09T21:52:11.967016Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20986afa-1f26a96b-9d147042-73867711_0] Write session: Do CDS request
2025-10-09T21:52:11.967023Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20986afa-1f26a96b-9d147042-73867711_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:11.967018Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:11.974489Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|20986afa-1f26a96b-9d147042-73867711_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:11.974586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:11.974605Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2377
2025-10-09T21:52:11.974797Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2377 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:11.974834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2377 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:11.974845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2377 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:11.974850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:11.975357Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2377 sessionId:  describe result for acl check
2025-10-09T21:52:11.975399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:11.975406Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:11.975409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:11.975416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343150326609466:6256] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:11.976025Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343150326609466:6256] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:11.979097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343150326609466:6256] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:11.980555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:11.982869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:11.983127Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:11.983139Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:11.983154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:11.983217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:11.983258Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343149964756921:8980]
2025-10-09T21:52:11.983238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343150326609466:6256] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:11.983251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343150326609466:6256] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:11.983253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343150326609466:6256] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:11.983260Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2377 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:11.983824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:11.984143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:11.984187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:11.984489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:11.984552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.984559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.984560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.984561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.984562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.984564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.984565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.984566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.984567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.984571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.984583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.984587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.984591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.984595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.984599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.984608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.984612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.984619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.984627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.984638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:11.984649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.984659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:11.984665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.984672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:11.984676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.984681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:11.984690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.984695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:11.984704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.984709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:11.984718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.984727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.984731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.984739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.984744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.984753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.984787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:11.990928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:11.992192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:11.992207Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:11.992209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:11.992213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:11.992219Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2376 [2:7559343149964756921:8980]
2025-10-09T21:52:11.992235Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:11.992251Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:11.992259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:11.992262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:11.992264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:11.992280Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:11.992285Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:11.992293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:11.992346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.992356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.992358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.992361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:11.992422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:11.992429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:11.992442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:11.992459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:11.992481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:11.992506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343150326609410:2432], now have 1 active actors on pipe
2025-10-09T21:52:11.992513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343150326609612:6256], now have 1 active actors on pipe
2025-10-09T21:52:11.992521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343149964756916:8980]
2025-10-09T21:52:11.992525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343149964756916:8980]
2025-10-09T21:52:11.992528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343149964756916:8980]
2025-10-09T21:52:11.992542Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:11.992560Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:11.992566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:11.992569Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:11.992572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:11.992579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:11.992582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:11.992608Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343149964756916:8980]
2025-10-09T21:52:11.992619Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:11.992661Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:11.992685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.992631Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2376
2025-10-09T21:52:11.992691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.992640Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2376
2025-10-09T21:52:11.992722Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bbb6d8e-8abf081-65407b60-7f72757d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:11.992734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:11.992738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:11.992741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:11.992751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:11.992764Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:11.992783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:11.992944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:11.992953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:11.992966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:11.993045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2377 partition: 0 MaxSeqNo: 25 sessionId: src_id|bbb6d8e-8abf081-65407b60-7f72757d_0
2025-10-09T21:52:11.993380Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|20986afa-1f26a96b-9d147042-73867711_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bbb6d8e-8abf081-65407b60-7f72757d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:11.995624Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bbb6d8e-8abf081-65407b60-7f72757d_0] Write session: try to update token
2025-10-09T21:52:11.995642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bbb6d8e-8abf081-65407b60-7f72757d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:12.004729Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:12.004744Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:12.004752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343149964756916:8980]
2025-10-09T21:52:12.004790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:12.004809Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:12.004942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:12.005082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:12.005216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:12.005351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:12.005523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:12.005557Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:12.007974Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2377 sessionId: src_id|bbb6d8e-8abf081-65407b60-7f72757d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:12.010307Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:12.010490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.010509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.010548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:12.010626Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:12.017916Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:12.017935Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:12.017938Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:12.017939Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:12.017949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343149964756916:8980]
2025-10-09T21:52:12.017953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343149964756916:8980]
2025-10-09T21:52:12.017956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343149964756916:8980]
2025-10-09T21:52:12.017996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:12.018002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.018006Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:12.018010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:12.018012Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:12.018015Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:12.018139Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:12.066835Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343154621576920:7496]
2025-10-09T21:52:12.067144Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 2376, pipe: [1:7559343154621576920:7496]
2025-10-09T21:52:12.167536Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:12.174308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.174331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.174481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.174537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.174599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.174680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.174723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.174731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.174792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.174842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.174896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.174956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.175000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.175006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.175072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.175121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.175175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.175235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.175274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.175279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.175339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.175405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.175464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.175522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.175560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.175565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.175620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.175672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.175726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.175779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.175870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.175876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.175930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.175978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.176030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.176087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.176123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.176129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.176214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.176268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.176327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.176378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.176415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.176421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.176474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.176523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.176575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.176629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.176673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.176680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.176744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.176798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.176854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.176911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.176948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.176953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.177013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.177072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.177128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.177179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.177219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.177225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.177264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343154621576920:7496], now have 1 active actors on pipe
2025-10-09T21:52:12.177492Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:12.177495Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343154621576920:7496]
2025-10-09T21:52:12.177524Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:12.177526Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:12.177561Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2377 reason: error at write: Disconnected sessionId: src_id|bbb6d8e-8abf081-65407b60-7f72757d_0
2025-10-09T21:52:12.177762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2377 sessionId: src_id|bbb6d8e-8abf081-65407b60-7f72757d_0 is DEAD
2025-10-09T21:52:12.177880Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbb6d8e-8abf081-65407b60-7f72757d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:12.177891Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbb6d8e-8abf081-65407b60-7f72757d_0] Write session will restart in 0.007409s
2025-10-09T21:52:12.177911Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbb6d8e-8abf081-65407b60-7f72757d_0] Write session: Do CDS request
2025-10-09T21:52:12.177916Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbb6d8e-8abf081-65407b60-7f72757d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:12.178059Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:12.186431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bbb6d8e-8abf081-65407b60-7f72757d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:12.186519Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:12.186534Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2378
2025-10-09T21:52:12.186697Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2378 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:12.186735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2378 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:12.186748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2378 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:12.186752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:12.187104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2378 sessionId:  describe result for acl check
2025-10-09T21:52:12.187134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:12.187139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:12.187140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:12.187144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343154621577003:6250] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:12.187556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343154621577003:6250] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:12.190199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343154621577003:6250] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:12.191298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:12.193305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:12.193566Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:12.193577Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:12.193589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:12.193578Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343154621577003:6250] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:12.193591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343154621577003:6250] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:12.193594Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343154621577003:6250] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:12.193600Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2378 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:12.193644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:12.193684Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343154259724425:8974]
2025-10-09T21:52:12.194278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:12.194611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:12.194652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:12.194971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:12.195023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.195029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.195030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.195031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.195032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.195033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.195035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.195036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.195038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.195043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.195054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.195058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.195063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.195067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.195071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.195075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.195079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.195083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.195089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.195099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.195104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.195110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.195119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.195125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.195129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.195134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.195139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.195143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.195147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.195152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.195162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.195167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.195171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.195181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.195187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.195196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.195228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:12.201645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:12.202715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:12.202733Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:12.202736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:12.202741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:12.202748Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2377 [2:7559343154259724425:8974]
2025-10-09T21:52:12.202765Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:12.202786Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:12.202793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:12.202796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:12.202799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:12.202819Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:12.202830Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:12.202838Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:12.203123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.203131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.203133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.203136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:12.203211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:12.203219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.203231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.203243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.203263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:12.203294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343154621577129:6250], now have 1 active actors on pipe
2025-10-09T21:52:12.203297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343154621576923:2432], now have 1 active actors on pipe
2025-10-09T21:52:12.203305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343154259724420:8974]
2025-10-09T21:52:12.203308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343154259724420:8974]
2025-10-09T21:52:12.203310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343154259724420:8974]
2025-10-09T21:52:12.203324Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:12.203343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:12.203344Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:12.203347Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:12.203350Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:12.203363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:12.203370Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:12.203343Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2377
2025-10-09T21:52:12.203408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343154259724420:8974]
2025-10-09T21:52:12.203419Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:12.203423Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:12.203462Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2377
2025-10-09T21:52:12.203504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.203511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.203534Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6c81ee64-538d36b0-1a9920c9-c42e66e4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:12.203544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:12.203550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:12.203552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:12.203560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:12.203569Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:12.203589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:12.203717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.203725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.203737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:12.203803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2378 partition: 0 MaxSeqNo: 25 sessionId: src_id|6c81ee64-538d36b0-1a9920c9-c42e66e4_0
2025-10-09T21:52:12.204140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bbb6d8e-8abf081-65407b60-7f72757d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6c81ee64-538d36b0-1a9920c9-c42e66e4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:12.206472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6c81ee64-538d36b0-1a9920c9-c42e66e4_0] Write session: try to update token
2025-10-09T21:52:12.206486Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6c81ee64-538d36b0-1a9920c9-c42e66e4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:12.219667Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2378 sessionId: src_id|6c81ee64-538d36b0-1a9920c9-c42e66e4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:12.220082Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:12.220100Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:12.220110Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343154259724420:8974]
2025-10-09T21:52:12.220158Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:12.220163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:12.220342Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:12.220496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:12.220644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:12.220791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:12.221101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:12.221144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:12.221937Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:12.222139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.222156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.222192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:12.222369Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:12.227874Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:12.227889Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:12.227891Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:12.227894Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:12.227906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343154259724420:8974]
2025-10-09T21:52:12.227910Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343154259724420:8974]
2025-10-09T21:52:12.227914Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343154259724420:8974]
2025-10-09T21:52:12.227950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:12.227962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.227963Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:12.227968Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:12.227969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:12.227972Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:12.228095Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:12.277746Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343154621577143:7496]
2025-10-09T21:52:12.278026Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 2377, pipe: [1:7559343154621577143:7496]
2025-10-09T21:52:12.375589Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:12.382271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.382299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.382442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.382506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.382568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.382624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.382663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.382670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.382728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.382780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.382835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.382892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.382932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.382937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.382991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.383043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.383096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.383147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.383376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.383382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.383436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.383494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.383557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.383611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.383648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.383654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.383709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.383760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.383815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.383892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.383931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.383936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.383991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.384042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.384092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.384141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.384182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.384190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.384240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.384288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.384343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.384410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.384452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.384459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.384516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.384572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.384624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.384677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.384713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.384719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.384769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.384817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.384868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.384923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.384964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.384970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.385020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:12.385070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:12.385122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:12.385179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:12.385214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:12.385221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:12.385256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343154621577143:7496], now have 1 active actors on pipe
2025-10-09T21:52:12.385472Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:12.385568Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:12.385576Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:12.385602Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2378 reason: error at write: Disconnected sessionId: src_id|6c81ee64-538d36b0-1a9920c9-c42e66e4_0
2025-10-09T21:52:12.385721Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:12.385804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2378 sessionId: src_id|6c81ee64-538d36b0-1a9920c9-c42e66e4_0 is DEAD
2025-10-09T21:52:12.385844Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343154621577143:7496]
2025-10-09T21:52:12.385925Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c81ee64-538d36b0-1a9920c9-c42e66e4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:12.385936Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c81ee64-538d36b0-1a9920c9-c42e66e4_0] Write session will restart in 0.005137s
2025-10-09T21:52:12.385961Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c81ee64-538d36b0-1a9920c9-c42e66e4_0] Write session: Do CDS request
2025-10-09T21:52:12.385967Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c81ee64-538d36b0-1a9920c9-c42e66e4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:12.392410Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6c81ee64-538d36b0-1a9920c9-c42e66e4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:12.392470Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:12.392479Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2379
2025-10-09T21:52:12.392591Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2379 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:12.392625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2379 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:12.392632Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2379 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:12.392635Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:12.393013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2379 sessionId:  describe result for acl check
2025-10-09T21:52:12.393043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:12.393049Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:12.393051Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:12.393056Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343154621577207:6208] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:12.393601Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343154621577207:6208] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:12.395435Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343154621577207:6208] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:12.397724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343154621577207:6208] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:12.397735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343154621577207:6208] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:12.397739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343154621577207:6208] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:12.397745Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2379 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:12.398830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:12.400721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:12.400887Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:12.400896Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:12.400906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:12.400959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:12.400996Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343154259724674:8952]
2025-10-09T21:52:12.401581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:12.401881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:12.401919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:12.402228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:12.402280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.402287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.402288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.402289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.402291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.402292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.402293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.402294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.402295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.402300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.402310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.402313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.402317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.402319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.402323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.402326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.402330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.402333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.402339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.402346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.402352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.402356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.402360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.402364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.402368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.402372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.402383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.402387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.402392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.402396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.402400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.402404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.402408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.402412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.402416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.402425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.402452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:12.408503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:12.409639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:12.409653Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:12.409655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:12.409659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:12.409664Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2378 [2:7559343154259724674:8952]
2025-10-09T21:52:12.409678Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:12.409692Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:12.409695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:12.409698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:12.409705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:12.409722Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:12.409732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:12.409740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:12.409793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.409805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.409807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.409810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:12.409869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:12.409876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.409887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.409897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.409915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:12.409940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343154621577176:2432], now have 1 active actors on pipe
2025-10-09T21:52:12.409947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343154621577377:6208], now have 1 active actors on pipe
2025-10-09T21:52:12.409954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343154259724669:8952]
2025-10-09T21:52:12.409957Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343154259724669:8952]
2025-10-09T21:52:12.409959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343154259724669:8952]
2025-10-09T21:52:12.409973Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:12.409993Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:12.409993Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:12.409996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:12.409998Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:12.410006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:12.410010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:12.410045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343154259724669:8952]
2025-10-09T21:52:12.410049Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:12.410002Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2378
2025-10-09T21:52:12.410051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:12.410023Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2378
2025-10-09T21:52:12.410075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.410080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.410111Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a4f77722-5abd4d5d-9745cbcc-c1e67591_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:12.410125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:12.410128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:12.410131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:12.410142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:12.410152Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:12.410173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:12.410320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.410329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.410343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:12.410420Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2379 partition: 0 MaxSeqNo: 25 sessionId: src_id|a4f77722-5abd4d5d-9745cbcc-c1e67591_0
2025-10-09T21:52:12.410739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6c81ee64-538d36b0-1a9920c9-c42e66e4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a4f77722-5abd4d5d-9745cbcc-c1e67591_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:12.413047Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a4f77722-5abd4d5d-9745cbcc-c1e67591_0] Write session: try to update token
2025-10-09T21:52:12.413066Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a4f77722-5abd4d5d-9745cbcc-c1e67591_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:12.424241Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:12.424261Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:12.424264Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:12.424266Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:12.424276Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343154259724669:8952]
2025-10-09T21:52:12.424281Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343154259724669:8952]
2025-10-09T21:52:12.424285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343154259724669:8952]
2025-10-09T21:52:12.424329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:12.424336Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:12.424342Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:12.424342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.424345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:12.424345Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:12.424503Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:12.427074Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2379 sessionId: src_id|a4f77722-5abd4d5d-9745cbcc-c1e67591_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:12.429369Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:12.486142Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343154621577391:7496]
2025-10-09T21:52:12.486372Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 2378, pipe: [1:7559343154621577391:7496]
2025-10-09T21:52:12.570167Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:12.570404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.570420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.570461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343154621577391:7496], now have 1 active actors on pipe
2025-10-09T21:52:12.570688Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:12.570724Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:12.570728Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:12.570752Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343154621577391:7496]
2025-10-09T21:52:12.570793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2379 reason: error at write: Disconnected sessionId: src_id|a4f77722-5abd4d5d-9745cbcc-c1e67591_0
2025-10-09T21:52:12.570991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2379 sessionId: src_id|a4f77722-5abd4d5d-9745cbcc-c1e67591_0 is DEAD
2025-10-09T21:52:12.571149Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4f77722-5abd4d5d-9745cbcc-c1e67591_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:12.571159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4f77722-5abd4d5d-9745cbcc-c1e67591_0] Write session will restart in 0.009585s
2025-10-09T21:52:12.571180Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4f77722-5abd4d5d-9745cbcc-c1e67591_0] Write session: Do CDS request
2025-10-09T21:52:12.571188Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4f77722-5abd4d5d-9745cbcc-c1e67591_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:12.571199Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:12.581539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a4f77722-5abd4d5d-9745cbcc-c1e67591_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:12.581608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:12.581622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2380
2025-10-09T21:52:12.581828Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2380 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:12.581865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2380 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:12.581875Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2380 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:12.581879Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:12.582314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2380 sessionId:  describe result for acl check
2025-10-09T21:52:12.582352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:12.582360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:12.582362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:12.582367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343154621577564:6206] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:12.582909Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343154621577564:6206] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:12.583287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:12.584838Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343154621577564:6206] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:12.585001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:12.585134Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:12.585141Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:12.585151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:12.585196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:12.585233Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343154259724883:8956]
2025-10-09T21:52:12.585815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:12.586110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:12.586150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:12.586455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:12.586499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.586504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.586505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.586506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.586507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.586509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.586510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.586511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.586512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.586517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.586527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.586531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.586540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.586544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.586548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.586552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.586560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.586564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.586574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.586582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.586592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.586602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.586608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.586615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.586623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.586630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.586637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.586642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.586646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.586651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.586657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.586662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.586668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.586678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.586682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.586691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.586721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:12.586925Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343154621577564:6206] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:12.586938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343154621577564:6206] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:12.586942Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343154621577564:6206] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:12.586949Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2380 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:12.592603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:12.593632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:12.593646Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:12.593648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:12.593651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:12.593657Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2379 [2:7559343154259724883:8956]
2025-10-09T21:52:12.593676Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:12.593695Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:12.593705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:12.593708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:12.593710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:12.593724Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:12.593729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:12.593736Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:12.593784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.593790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.593792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.593795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:12.593854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:12.593862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.593872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.593885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.593907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:12.593928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343154621577606:6206], now have 1 active actors on pipe
2025-10-09T21:52:12.593936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343154621577394:2432], now have 1 active actors on pipe
2025-10-09T21:52:12.593944Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343154259724878:8956]
2025-10-09T21:52:12.593950Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343154259724878:8956]
2025-10-09T21:52:12.593952Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343154259724878:8956]
2025-10-09T21:52:12.593964Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:12.593981Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:12.593983Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:12.593986Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:12.593989Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:12.593971Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2379
2025-10-09T21:52:12.593999Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:12.594002Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:12.593998Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2379
2025-10-09T21:52:12.594048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.594056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.594068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343154259724878:8956]
2025-10-09T21:52:12.594079Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:12.594089Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:12.594114Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ccca7d55-c18c7288-2441bfd7-73ce7c6d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:12.594124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:12.594128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:12.594130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:12.594140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:12.594153Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:12.594172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:12.594284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.594293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.594305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:12.594372Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2380 partition: 0 MaxSeqNo: 25 sessionId: src_id|ccca7d55-c18c7288-2441bfd7-73ce7c6d_0
2025-10-09T21:52:12.594631Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a4f77722-5abd4d5d-9745cbcc-c1e67591_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ccca7d55-c18c7288-2441bfd7-73ce7c6d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:12.596870Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ccca7d55-c18c7288-2441bfd7-73ce7c6d_0] Write session: try to update token
2025-10-09T21:52:12.596883Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ccca7d55-c18c7288-2441bfd7-73ce7c6d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:12.607724Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:12.607745Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:12.607747Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:12.607749Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:12.607760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343154259724878:8956]
2025-10-09T21:52:12.607765Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343154259724878:8956]
2025-10-09T21:52:12.607769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343154259724878:8956]
2025-10-09T21:52:12.607812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:12.607819Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:12.607824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.607825Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:12.607826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:12.607829Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:12.607962Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:12.608970Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2380 sessionId: src_id|ccca7d55-c18c7288-2441bfd7-73ce7c6d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:12.611172Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:12.670960Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343154621577613:7496]
2025-10-09T21:52:12.671227Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 2379, pipe: [1:7559343154621577613:7496]
2025-10-09T21:52:12.758894Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:12.759374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.759387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.759424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343154621577613:7496], now have 1 active actors on pipe
2025-10-09T21:52:12.759582Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:12.759668Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:12.759745Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:12.759752Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:12.759787Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343154621577613:7496]
2025-10-09T21:52:12.759807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2380 reason: error at write: Disconnected sessionId: src_id|ccca7d55-c18c7288-2441bfd7-73ce7c6d_0
2025-10-09T21:52:12.759974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2380 sessionId: src_id|ccca7d55-c18c7288-2441bfd7-73ce7c6d_0 is DEAD
2025-10-09T21:52:12.760119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ccca7d55-c18c7288-2441bfd7-73ce7c6d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:12.760129Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ccca7d55-c18c7288-2441bfd7-73ce7c6d_0] Write session will restart in 0.006946s
2025-10-09T21:52:12.760158Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ccca7d55-c18c7288-2441bfd7-73ce7c6d_0] Write session: Do CDS request
2025-10-09T21:52:12.760164Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ccca7d55-c18c7288-2441bfd7-73ce7c6d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:12.768441Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ccca7d55-c18c7288-2441bfd7-73ce7c6d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:12.768504Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:12.768517Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2381
2025-10-09T21:52:12.768626Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2381 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:12.768652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2381 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:12.768660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2381 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:12.768664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:12.769044Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2381 sessionId:  describe result for acl check
2025-10-09T21:52:12.769070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:12.769074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:12.769076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:12.769080Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343154621577695:6182] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:12.769499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343154621577695:6182] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:12.771930Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343154621577695:6182] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:12.773362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:12.775214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:12.775456Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:12.775468Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:12.775481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:12.775464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343154621577695:6182] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:12.775473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343154621577695:6182] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:12.775476Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343154621577695:6182] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:12.775481Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2381 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:12.775540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:12.775577Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343154259725094:8953]
2025-10-09T21:52:12.776142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:12.776421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:12.776461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:12.776771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:12.776819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.776825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.776826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.776828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.776829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.776830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.776831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.776832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.776833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.776838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.776850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.776854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.776858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.776862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.776866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.776870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.776874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.776878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.776884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.776892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.776898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.776903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.776907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.776913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.776917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.776922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.776926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.776931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.776935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.776945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.776950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.776955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.776964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.776969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.776979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.776984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.777012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:12.783006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:12.784005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:12.784019Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:12.784021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:12.784025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:12.784031Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2380 [2:7559343154259725094:8953]
2025-10-09T21:52:12.784043Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:12.784063Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:12.784070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:12.784073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:12.784075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:12.784091Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:12.784100Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:12.784107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:12.784160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.784166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.784168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.784171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:12.784237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:12.784244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.784255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.784270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.784291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:12.784315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343154621577616:2432], now have 1 active actors on pipe
2025-10-09T21:52:12.784323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343154621577825:6182], now have 1 active actors on pipe
2025-10-09T21:52:12.784330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343154259725089:8953]
2025-10-09T21:52:12.784332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343154259725089:8953]
2025-10-09T21:52:12.784334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343154259725089:8953]
2025-10-09T21:52:12.784346Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:12.784360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:12.784360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:12.784363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:12.784365Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:12.784379Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:12.784382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:12.784370Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2380
2025-10-09T21:52:12.784404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343154259725089:8953]
2025-10-09T21:52:12.784373Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2380
2025-10-09T21:52:12.784412Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:12.784417Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:12.784431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.784435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.784481Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8a649a07-79592832-fb1529bb-55e0382c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:12.784494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:12.784497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:12.784500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:12.784510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:12.784520Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:12.784535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:12.784657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.784665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.784680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:12.784774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2381 partition: 0 MaxSeqNo: 25 sessionId: src_id|8a649a07-79592832-fb1529bb-55e0382c_0
2025-10-09T21:52:12.785102Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ccca7d55-c18c7288-2441bfd7-73ce7c6d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8a649a07-79592832-fb1529bb-55e0382c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:12.787659Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a649a07-79592832-fb1529bb-55e0382c_0] Write session: try to update token
2025-10-09T21:52:12.787675Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a649a07-79592832-fb1529bb-55e0382c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:12.798004Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:12.798020Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:12.798023Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:12.798025Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:12.798050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343154259725089:8953]
2025-10-09T21:52:12.798054Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343154259725089:8953]
2025-10-09T21:52:12.798058Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343154259725089:8953]
2025-10-09T21:52:12.798100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:12.798100Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:12.798106Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:12.798111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.798112Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:12.798114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:12.798238Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:12.800547Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2381 sessionId: src_id|8a649a07-79592832-fb1529bb-55e0382c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:12.803036Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:12.859903Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343154621577837:7496]
2025-10-09T21:52:12.860166Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 2380, pipe: [1:7559343154621577837:7496]
2025-10-09T21:52:12.943298Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:12.943752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.943765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.943802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343154621577837:7496], now have 1 active actors on pipe
2025-10-09T21:52:12.944064Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:12.944095Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:12.944099Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:12.944124Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343154621577837:7496]
2025-10-09T21:52:12.944161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2381 reason: error at write: Disconnected sessionId: src_id|8a649a07-79592832-fb1529bb-55e0382c_0
2025-10-09T21:52:12.944342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2381 sessionId: src_id|8a649a07-79592832-fb1529bb-55e0382c_0 is DEAD
2025-10-09T21:52:12.944400Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:12.944476Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a649a07-79592832-fb1529bb-55e0382c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:12.944483Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a649a07-79592832-fb1529bb-55e0382c_0] Write session will restart in 0.007501s
2025-10-09T21:52:12.944501Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a649a07-79592832-fb1529bb-55e0382c_0] Write session: Do CDS request
2025-10-09T21:52:12.944506Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a649a07-79592832-fb1529bb-55e0382c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:12.953477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8a649a07-79592832-fb1529bb-55e0382c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:12.953542Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:12.953569Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2382
2025-10-09T21:52:12.953689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2382 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:12.953714Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2382 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:12.953718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2382 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:12.953721Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:12.954121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2382 sessionId:  describe result for acl check
2025-10-09T21:52:12.954160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:12.954163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:12.954164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:12.954170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343154621577960:6161] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:12.954697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343154621577960:6161] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:12.956559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343154621577960:6161] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:12.956858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:12.958636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:12.958798Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:12.958808Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:12.958818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:12.958863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:12.958900Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343154259725310:8944]
2025-10-09T21:52:12.958904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343154621577960:6161] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:12.958918Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343154621577960:6161] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:12.958922Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343154621577960:6161] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:12.958929Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2382 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:12.959454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:12.959704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:12.959738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:12.960005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:12.960060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.960067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.960068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.960069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.960070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.960071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.960072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.960073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.960074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.960079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.960089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.960092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.960096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.960099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.960103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.960107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.960110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.960114Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.960119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.960127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:12.960133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.960137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:12.960141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.960145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:12.960149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.960153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:12.960158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.960163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:12.960168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.960173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:12.960178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.960182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.960186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.960191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.960195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.960199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.960227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:12.966237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:12.967242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:12.967256Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:12.967258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:12.967262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:12.967268Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2381 [2:7559343154259725310:8944]
2025-10-09T21:52:12.967283Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:12.967299Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:12.967304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:12.967306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:12.967308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:12.967319Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:12.967323Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:12.967329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:12.967368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.967373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.967374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.967376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:12.967423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:12.967428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.967434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:12.967440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:12.967449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:12.967464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343154259725305:8944]
2025-10-09T21:52:12.967466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343154259725305:8944]
2025-10-09T21:52:12.967467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343154259725305:8944]
2025-10-09T21:52:12.967473Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:12.967482Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:12.967485Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:12.967487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:12.967489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:12.967490Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:12.967492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:12.967500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343154621577841:2432], now have 1 active actors on pipe
2025-10-09T21:52:12.967502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343154621578051:6161], now have 1 active actors on pipe
2025-10-09T21:52:12.967519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343154259725305:8944]
2025-10-09T21:52:12.967526Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:12.967535Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:12.967561Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2381
2025-10-09T21:52:12.967575Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2381
2025-10-09T21:52:12.967625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.967633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.967653Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a43508d9-96142091-b70764e5-c68ae978_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:12.967663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:12.967666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:12.967667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:12.967674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:12.967681Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:12.967692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:12.967831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.967836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.967846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:12.967924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2382 partition: 0 MaxSeqNo: 25 sessionId: src_id|a43508d9-96142091-b70764e5-c68ae978_0
2025-10-09T21:52:12.968232Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8a649a07-79592832-fb1529bb-55e0382c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a43508d9-96142091-b70764e5-c68ae978_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:12.970792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a43508d9-96142091-b70764e5-c68ae978_0] Write session: try to update token
2025-10-09T21:52:12.970808Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a43508d9-96142091-b70764e5-c68ae978_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:12.978250Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:12.978264Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:12.978273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343154259725305:8944]
2025-10-09T21:52:12.978319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:12.978324Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:12.978486Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:12.978630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:12.978760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:12.978891Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:12.979068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:12.979110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:12.982442Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2382 sessionId: src_id|a43508d9-96142091-b70764e5-c68ae978_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:12.984641Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:12.984908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:12.984925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:12.984971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:12.985053Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:12.989865Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:12.989876Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:12.989877Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:12.989878Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:12.989886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343154259725305:8944]
2025-10-09T21:52:12.989889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343154259725305:8944]
2025-10-09T21:52:12.989893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343154259725305:8944]
2025-10-09T21:52:12.989928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:12.989935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:12.989942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:12.989961Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:12.989977Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:12.989981Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:12.990073Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:13.044187Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343158916545359:7496]
2025-10-09T21:52:13.044445Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 2381, pipe: [1:7559343158916545359:7496]
2025-10-09T21:52:13.137172Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:13.144302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.144328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.144469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.144523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.144580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.144642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.144683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.144691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.144755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.144809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.144882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.144954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.144996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.145004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.145068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.145124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.145177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.145230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.145273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.145279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.145360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.145414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.145474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.145534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.145573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.145575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.145633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.145695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.145756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.145812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.145850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.145856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.145911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.145963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.146020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.146093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.146138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.146145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.146210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.146270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.146323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.146386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.146428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.146435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.146490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.146538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.146589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.146642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.146681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.146688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.146742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.146796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.146860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.146923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.146968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.146974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.147030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.147092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.147152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.147204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.147250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.147258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.147306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343158916545359:7496], now have 1 active actors on pipe
2025-10-09T21:52:13.147584Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:13.147614Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343158916545359:7496]
2025-10-09T21:52:13.147635Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:13.147646Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:13.147681Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2382 reason: error at write: Disconnected sessionId: src_id|a43508d9-96142091-b70764e5-c68ae978_0
2025-10-09T21:52:13.147872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2382 sessionId: src_id|a43508d9-96142091-b70764e5-c68ae978_0 is DEAD
2025-10-09T21:52:13.148041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a43508d9-96142091-b70764e5-c68ae978_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:13.148053Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a43508d9-96142091-b70764e5-c68ae978_0] Write session will restart in 0.008819s
2025-10-09T21:52:13.148091Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a43508d9-96142091-b70764e5-c68ae978_0] Write session: Do CDS request
2025-10-09T21:52:13.148097Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a43508d9-96142091-b70764e5-c68ae978_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:13.148252Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:13.158548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a43508d9-96142091-b70764e5-c68ae978_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:13.158616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:13.158631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2383
2025-10-09T21:52:13.158781Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2383 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:13.158816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2383 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:13.158824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2383 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:13.158827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:13.159366Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2383 sessionId:  describe result for acl check
2025-10-09T21:52:13.159402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:13.159407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:13.159409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:13.159416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343158916545415:6148] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:13.159987Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343158916545415:6148] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:13.163441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343158916545415:6148] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:13.166164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343158916545415:6148] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:13.166180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343158916545415:6148] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:13.166183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343158916545415:6148] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:13.166191Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2383 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:13.166461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:13.168623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:13.168826Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:13.168836Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:13.168846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:13.168895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:13.168934Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343158554692820:8938]
2025-10-09T21:52:13.169443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:13.169764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:13.169799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:13.170088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:13.170138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.170147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.170149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.170150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.170151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.170152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.170154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.170155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.170156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.170163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.170174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.170179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.170183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.170188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.170192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.170196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.170200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.170204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.170212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.170217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.170221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.170225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.170227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.170231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.170233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.170237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.170240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.170243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.170246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.170250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.170253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.170257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.170260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.170263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.170267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.170270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.170288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:13.176593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:13.177804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:13.177819Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:13.177821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:13.177824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:13.177831Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2382 [2:7559343158554692820:8938]
2025-10-09T21:52:13.177850Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:13.177866Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:13.177877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:13.177881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:13.177883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:13.177900Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:13.177906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:13.177912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:13.177965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.177972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.177974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.177976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:13.178060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:13.178067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.178075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.178082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.178098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:13.178122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343158916545571:6148], now have 1 active actors on pipe
2025-10-09T21:52:13.178132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343158916545361:2432], now have 1 active actors on pipe
2025-10-09T21:52:13.178139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343158554692815:8938]
2025-10-09T21:52:13.178147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343158554692815:8938]
2025-10-09T21:52:13.178149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343158554692815:8938]
2025-10-09T21:52:13.178165Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:13.178181Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:13.178184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:13.178188Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:13.178190Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:13.178195Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:13.178197Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:13.178220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343158554692815:8938]
2025-10-09T21:52:13.178222Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:13.178204Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2382
2025-10-09T21:52:13.178205Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2382
2025-10-09T21:52:13.178248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.178252Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:13.178258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.178290Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8e9f9f5a-e2d73641-541451da-e871076f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:13.178302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:13.178306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:13.178309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:13.178324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:13.178333Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:13.178346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:13.178478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.178484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.178494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:13.178593Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2383 partition: 0 MaxSeqNo: 25 sessionId: src_id|8e9f9f5a-e2d73641-541451da-e871076f_0
2025-10-09T21:52:13.178890Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a43508d9-96142091-b70764e5-c68ae978_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8e9f9f5a-e2d73641-541451da-e871076f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:13.181238Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e9f9f5a-e2d73641-541451da-e871076f_0] Write session: try to update token
2025-10-09T21:52:13.181253Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e9f9f5a-e2d73641-541451da-e871076f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:13.192593Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:13.192615Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:13.192618Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:13.192621Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:13.192631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343158554692815:8938]
2025-10-09T21:52:13.192636Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343158554692815:8938]
2025-10-09T21:52:13.192640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343158554692815:8938]
2025-10-09T21:52:13.192677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:13.192683Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:13.192685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.192687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:13.192690Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:13.192694Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:13.192831Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:13.193425Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2383 sessionId: src_id|8e9f9f5a-e2d73641-541451da-e871076f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:13.195810Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:13.247993Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343158916545589:7496]
2025-10-09T21:52:13.248277Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 2382, pipe: [1:7559343158916545589:7496]
2025-10-09T21:52:13.340829Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:13.341374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.341390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.341429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343158916545589:7496], now have 1 active actors on pipe
2025-10-09T21:52:13.341644Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:13.341746Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343158916545589:7496]
2025-10-09T21:52:13.341757Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:13.341774Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:13.341822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2383 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8e9f9f5a-e2d73641-541451da-e871076f_0
2025-10-09T21:52:13.341997Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2383 sessionId: src_id|8e9f9f5a-e2d73641-541451da-e871076f_0 is DEAD
2025-10-09T21:52:13.342128Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e9f9f5a-e2d73641-541451da-e871076f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:13.342139Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e9f9f5a-e2d73641-541451da-e871076f_0] Write session will restart in 0.007892s
2025-10-09T21:52:13.342160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e9f9f5a-e2d73641-541451da-e871076f_0] Write session: Do CDS request
2025-10-09T21:52:13.342165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e9f9f5a-e2d73641-541451da-e871076f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:13.351463Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e9f9f5a-e2d73641-541451da-e871076f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:13.351553Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:13.351566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2384
2025-10-09T21:52:13.351689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2384 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:13.351722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2384 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:13.351734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2384 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:13.351740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:13.352162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2384 sessionId:  describe result for acl check
2025-10-09T21:52:13.352198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:13.352200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:13.352201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:13.352208Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343158916545716:6130] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:13.352744Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343158916545716:6130] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:13.354709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343158916545716:6130] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:13.355024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:13.356729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:13.356894Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:13.356903Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:13.356915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:13.356961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:13.356980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343158916545716:6130] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:13.357001Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343158554693034:8934]
2025-10-09T21:52:13.356991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343158916545716:6130] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:13.356995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343158916545716:6130] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:13.357001Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2384 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:13.357558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:13.357821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:13.357861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:13.358120Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:13.358169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.358177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.358178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.358179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.358180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.358181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.358182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.358183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.358184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.358189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.358199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.358204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.358207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.358212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.358216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.358219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.358223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.358227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.358233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.358241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.358247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.358250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.358253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.358256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.358258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.358261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.358264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.358267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.358270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.358273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.358277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.358280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.358282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.358285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.358288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.358290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.358306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:13.364138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:13.365133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:13.365147Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:13.365149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:13.365151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:13.365156Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2383 [2:7559343158554693034:8934]
2025-10-09T21:52:13.365165Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:13.365178Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:13.365184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:13.365186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:13.365187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:13.365198Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:13.365206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:13.365212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:13.365252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.365255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.365256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.365258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:13.365317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:13.365326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.365337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.365347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.365362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:13.365380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343158554693029:8934]
2025-10-09T21:52:13.365387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343158554693029:8934]
2025-10-09T21:52:13.365389Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343158554693029:8934]
2025-10-09T21:52:13.365402Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:13.365411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:13.365416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343158916545596:2432], now have 1 active actors on pipe
2025-10-09T21:52:13.365419Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:13.365420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343158916545809:6130], now have 1 active actors on pipe
2025-10-09T21:52:13.365420Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:13.365428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:13.365441Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:13.365449Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:13.365469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343158554693029:8934]
2025-10-09T21:52:13.365483Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:13.365481Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2383
2025-10-09T21:52:13.365488Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2383
2025-10-09T21:52:13.365505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:13.365524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.365528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.365553Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|87d7688c-7650c939-fc2cfc24-b9a26726_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:13.365564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:13.365568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:13.365574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:13.365584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:13.365596Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:13.365612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:13.365755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.365762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.365775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:13.365865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2384 partition: 0 MaxSeqNo: 25 sessionId: src_id|87d7688c-7650c939-fc2cfc24-b9a26726_0
2025-10-09T21:52:13.366198Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e9f9f5a-e2d73641-541451da-e871076f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|87d7688c-7650c939-fc2cfc24-b9a26726_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:13.368355Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87d7688c-7650c939-fc2cfc24-b9a26726_0] Write session: try to update token
2025-10-09T21:52:13.368367Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87d7688c-7650c939-fc2cfc24-b9a26726_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:13.379281Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:13.379309Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:13.379312Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:13.379314Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:13.379323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343158554693029:8934]
2025-10-09T21:52:13.379329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343158554693029:8934]
2025-10-09T21:52:13.379334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343158554693029:8934]
2025-10-09T21:52:13.379384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:13.379390Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:13.379395Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:13.379400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.379401Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:13.379403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:13.379548Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:13.379774Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2384 sessionId: src_id|87d7688c-7650c939-fc2cfc24-b9a26726_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:13.382006Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:13.442104Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343158916545842:7496]
2025-10-09T21:52:13.442378Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 2383, pipe: [1:7559343158916545842:7496]
2025-10-09T21:52:13.522285Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:13.522715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.522730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.522769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343158916545842:7496], now have 1 active actors on pipe
2025-10-09T21:52:13.522942Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:13.522955Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:13.522958Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343158916545842:7496]
2025-10-09T21:52:13.522980Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:13.522984Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:13.523028Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2384 reason: error at write: Disconnected sessionId: src_id|87d7688c-7650c939-fc2cfc24-b9a26726_0
2025-10-09T21:52:13.523211Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2384 sessionId: src_id|87d7688c-7650c939-fc2cfc24-b9a26726_0 is DEAD
2025-10-09T21:52:13.523348Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87d7688c-7650c939-fc2cfc24-b9a26726_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:13.523356Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87d7688c-7650c939-fc2cfc24-b9a26726_0] Write session will restart in 0.008711s
2025-10-09T21:52:13.523382Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87d7688c-7650c939-fc2cfc24-b9a26726_0] Write session: Do CDS request
2025-10-09T21:52:13.523388Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87d7688c-7650c939-fc2cfc24-b9a26726_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:13.533417Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87d7688c-7650c939-fc2cfc24-b9a26726_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:13.533522Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:13.533534Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2385
2025-10-09T21:52:13.533636Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2385 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:13.533667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2385 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:13.533676Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2385 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:13.533679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:13.534081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2385 sessionId:  describe result for acl check
2025-10-09T21:52:13.534115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:13.534123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:13.534125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:13.534130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343158916545976:6102] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:13.534677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343158916545976:6102] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:13.536764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:13.536940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343158916545976:6102] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:13.538493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:13.538649Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:13.538656Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:13.538666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:13.538708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:13.538743Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343158554693286:8915]
2025-10-09T21:52:13.539143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:13.539234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343158916545976:6102] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:13.539245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343158916545976:6102] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:13.539249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343158916545976:6102] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:13.539256Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2385 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:13.539423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:13.539469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:13.539733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:13.539799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.539804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.539805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.539806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.539808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.539809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.539810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.539811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.539812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.539817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.539827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.539831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.539834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.539836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.539838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.539840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.539842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.539844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.539849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.539855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.539859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.539862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.539864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.539867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.539870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.539873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.539875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.539878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.539880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.539883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.539885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.539888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.539891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.539895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.539900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.539904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.539930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:13.545556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:13.546646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:13.546660Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:13.546662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:13.546666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:13.546672Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2384 [2:7559343158554693286:8915]
2025-10-09T21:52:13.546685Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:13.546701Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:13.546708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:13.546710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:13.546713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:13.546729Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:13.546738Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:13.546747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:13.546801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.546811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.546813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.546815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:13.546876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:13.546883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.546894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.546909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.546929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:13.546952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343158916545845:2432], now have 1 active actors on pipe
2025-10-09T21:52:13.546961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343158916546060:6102], now have 1 active actors on pipe
2025-10-09T21:52:13.546968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343158554693281:8915]
2025-10-09T21:52:13.546971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343158554693281:8915]
2025-10-09T21:52:13.546972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343158554693281:8915]
2025-10-09T21:52:13.546985Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:13.547003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:13.547010Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:13.547013Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:13.547015Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:13.547020Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:13.547023Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:13.547034Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2384
2025-10-09T21:52:13.547040Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2384
2025-10-09T21:52:13.547046Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343158554693281:8915]
2025-10-09T21:52:13.547053Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:13.547062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:13.547089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.547098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.547123Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1fac8e8a-5a60031f-abd3ccdb-8bb3759_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:13.547136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:13.547143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:13.547145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:13.547155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:13.547168Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:13.547186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:13.547323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.547331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.547345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:13.547449Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2385 partition: 0 MaxSeqNo: 25 sessionId: src_id|1fac8e8a-5a60031f-abd3ccdb-8bb3759_0
2025-10-09T21:52:13.547744Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87d7688c-7650c939-fc2cfc24-b9a26726_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1fac8e8a-5a60031f-abd3ccdb-8bb3759_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:13.550338Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1fac8e8a-5a60031f-abd3ccdb-8bb3759_0] Write session: try to update token
2025-10-09T21:52:13.550356Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1fac8e8a-5a60031f-abd3ccdb-8bb3759_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:13.562809Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2385 sessionId: src_id|1fac8e8a-5a60031f-abd3ccdb-8bb3759_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:13.563498Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:13.563517Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:13.563526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343158554693281:8915]
2025-10-09T21:52:13.563567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:13.563582Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:13.563733Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:13.563878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:13.564019Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:13.564155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:13.564346Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:13.564386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:13.565086Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:13.565273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.565295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.565327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:13.565417Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:13.570647Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:13.570661Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:13.570663Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:13.570665Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:13.570674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343158554693281:8915]
2025-10-09T21:52:13.570677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343158554693281:8915]
2025-10-09T21:52:13.570681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343158554693281:8915]
2025-10-09T21:52:13.570716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:13.570727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.570734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:13.570743Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:13.570749Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:13.570752Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:13.570858Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:13.623269Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343158916546069:7496]
2025-10-09T21:52:13.623589Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 2384, pipe: [1:7559343158916546069:7496]
2025-10-09T21:52:13.713326Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:13.720343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.720372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.720524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.720590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.720657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.720722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.720767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.720776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.720840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.720905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.720967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.721032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.721074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.721078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.721140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.721195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.721253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.721322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.721365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.721374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.721435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.721497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.721551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.721605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.721649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.721659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.721724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.721786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.721848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.721909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.721952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.721958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.722019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.722095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.722161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.722222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.722260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.722265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.722330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.722388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.722448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.722511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.722621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.722630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.722692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.722749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.722812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.722871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.722915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.722922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.722982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.723039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.723095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.723155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.723215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.723221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.723282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.723338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.723397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.723455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.723496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.723504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.723551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343158916546069:7496], now have 1 active actors on pipe
2025-10-09T21:52:13.724096Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:13.724109Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:13.724135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2385 reason: error at write: Disconnected sessionId: src_id|1fac8e8a-5a60031f-abd3ccdb-8bb3759_0
2025-10-09T21:52:13.724174Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343158916546069:7496]
2025-10-09T21:52:13.724207Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:13.724333Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2385 sessionId: src_id|1fac8e8a-5a60031f-abd3ccdb-8bb3759_0 is DEAD
2025-10-09T21:52:13.724477Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fac8e8a-5a60031f-abd3ccdb-8bb3759_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:13.724464Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:13.724494Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fac8e8a-5a60031f-abd3ccdb-8bb3759_0] Write session will restart in 0.009030s
2025-10-09T21:52:13.724522Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fac8e8a-5a60031f-abd3ccdb-8bb3759_0] Write session: Do CDS request
2025-10-09T21:52:13.724528Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fac8e8a-5a60031f-abd3ccdb-8bb3759_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:13.734431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1fac8e8a-5a60031f-abd3ccdb-8bb3759_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:13.734495Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:13.734506Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2386
2025-10-09T21:52:13.734642Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2386 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:13.734675Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2386 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:13.734685Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2386 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:13.734689Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:13.735096Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2386 sessionId:  describe result for acl check
2025-10-09T21:52:13.735128Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:13.735134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:13.735136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:13.735142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343158916546231:6074] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:13.735760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343158916546231:6074] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:13.737052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:13.738112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343158916546231:6074] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:13.738673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:13.738834Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:13.738842Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:13.738853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:13.738905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:13.739026Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343158554693503:8902]
2025-10-09T21:52:13.739580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:13.739876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:13.739917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:13.740206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:13.740258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.740266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.740267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.740268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.740269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.740270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.740271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.740272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.740273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.740278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.740287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.740291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.740295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.740299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.740303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.740308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.740312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.740316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.740321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.740330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.740336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.740342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.740347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.740352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.740356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.740361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.740366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.740375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.740380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.740386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.740395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.740401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.740407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.740412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.740418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.740424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.740455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:13.740513Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343158916546231:6074] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:13.740524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343158916546231:6074] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:13.740528Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343158916546231:6074] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:13.740535Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2386 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:13.746649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:13.747659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:13.747672Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:13.747674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:13.747678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:13.747684Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2385 [2:7559343158554693503:8902]
2025-10-09T21:52:13.747697Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:13.747713Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:13.747715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:13.747718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:13.747721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:13.747736Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:13.747747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:13.747754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:13.747807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.747813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.747815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.747817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:13.747878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:13.747996Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2385
2025-10-09T21:52:13.748005Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2385
2025-10-09T21:52:13.748073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.748089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.748105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.748124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:13.748149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343158916546287:6074], now have 1 active actors on pipe
2025-10-09T21:52:13.748156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343158916546072:2432], now have 1 active actors on pipe
2025-10-09T21:52:13.748171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.748179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.748200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343158554693498:8902]
2025-10-09T21:52:13.748209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343158554693498:8902]
2025-10-09T21:52:13.748211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343158554693498:8902]
2025-10-09T21:52:13.748224Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:13.748238Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:13.748241Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:13.748245Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:13.748247Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:13.748257Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:13.748263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:13.748281Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|61bf6260-340da27a-f556b695-7b3f6f69_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:13.748293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:13.748296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:13.748299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:13.748308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:13.748317Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:13.748347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343158554693498:8902]
2025-10-09T21:52:13.748358Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:13.748373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:13.748376Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:13.748513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.748520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.748536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:13.748618Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2386 partition: 0 MaxSeqNo: 25 sessionId: src_id|61bf6260-340da27a-f556b695-7b3f6f69_0
2025-10-09T21:52:13.748911Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1fac8e8a-5a60031f-abd3ccdb-8bb3759_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|61bf6260-340da27a-f556b695-7b3f6f69_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:13.751235Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|61bf6260-340da27a-f556b695-7b3f6f69_0] Write session: try to update token
2025-10-09T21:52:13.751248Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|61bf6260-340da27a-f556b695-7b3f6f69_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:13.762922Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2386 sessionId: src_id|61bf6260-340da27a-f556b695-7b3f6f69_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:13.765065Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:13.765297Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:13.765315Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:13.765324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343158554693498:8902]
2025-10-09T21:52:13.765371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.765373Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:13.765385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.765418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:13.765585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:13.765721Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:13.765858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:13.765995Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:13.766190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:13.766242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:13.766272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:13.766391Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:13.772750Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:13.772765Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:13.772767Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:13.772769Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:13.772777Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343158554693498:8902]
2025-10-09T21:52:13.772781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343158554693498:8902]
2025-10-09T21:52:13.772784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343158554693498:8902]
2025-10-09T21:52:13.772819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:13.772830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.772832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:13.772841Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:13.772849Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:13.772861Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:13.772957Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:13.824493Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343158916546296:7496]
2025-10-09T21:52:13.824756Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 2385, pipe: [1:7559343158916546296:7496]
2025-10-09T21:52:13.911339Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:13.917766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.917792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.917943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.917996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.918077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.918141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.918183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.918192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.918258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.918316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.918378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.918443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.918485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.918492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.918549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.918604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.918659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.918721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.918762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.918769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.918827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.918921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.918976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.919033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.919069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.919074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.919129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.919179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.919235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.919302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.919349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.919356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.919413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.919471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.919528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.919585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.919626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.919633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.919692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.919746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.919805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.919859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.919991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.920000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.920056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.920109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.920165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.920218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.920256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.920262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.920317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.920375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.920434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.920491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.920529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.920536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.920593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:13.920642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:13.920694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:13.920743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:13.920779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:13.920785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:13.920822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343158916546296:7496], now have 1 active actors on pipe
2025-10-09T21:52:13.921045Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343158916546296:7496]
2025-10-09T21:52:13.921045Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:13.921082Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:13.921085Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:13.921114Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2386 reason: error at write: Disconnected sessionId: src_id|61bf6260-340da27a-f556b695-7b3f6f69_0
2025-10-09T21:52:13.921292Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2386 sessionId: src_id|61bf6260-340da27a-f556b695-7b3f6f69_0 is DEAD
2025-10-09T21:52:13.921422Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61bf6260-340da27a-f556b695-7b3f6f69_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:13.921433Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61bf6260-340da27a-f556b695-7b3f6f69_0] Write session will restart in 0.009094s
2025-10-09T21:52:13.921451Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61bf6260-340da27a-f556b695-7b3f6f69_0] Write session: Do CDS request
2025-10-09T21:52:13.921456Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61bf6260-340da27a-f556b695-7b3f6f69_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:13.921596Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:13.931400Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|61bf6260-340da27a-f556b695-7b3f6f69_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:13.931513Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:13.931525Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2387
2025-10-09T21:52:13.931639Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2387 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:13.931691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2387 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:13.931700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2387 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:13.931703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:13.932081Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2387 sessionId:  describe result for acl check
2025-10-09T21:52:13.932114Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:13.932116Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:13.932117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:13.932123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343158916546459:6060] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:13.932645Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343158916546459:6060] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:13.934014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:13.934911Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343158916546459:6060] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:13.935998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:13.936165Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:13.936174Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:13.936185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:13.936234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:13.936268Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343158554693724:8909]
2025-10-09T21:52:13.936822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:13.937074Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343158916546459:6060] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:13.937084Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343158916546459:6060] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:13.937086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343158916546459:6060] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:13.937091Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2387 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:13.937091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:13.937132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:13.937438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:13.937484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.937491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.937492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.937493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.937494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.937495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.937496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.937497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.937498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.937503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.937513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.937517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.937520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.937524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.937528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.937531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.937535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.937539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.937544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.937552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:13.937563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.937572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:13.937578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.937587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:13.937591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.937596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:13.937600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.937609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:13.937617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.937622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:13.937629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.937634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.937642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.937647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.937655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.937660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.937689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:13.943673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:13.944808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:13.944824Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:13.944825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:13.944828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:13.944833Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2386 [2:7559343158554693724:8909]
2025-10-09T21:52:13.944843Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:13.944855Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:13.944857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:13.944859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:13.944861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:13.944872Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:13.944877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:13.944885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:13.944925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.944930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.944931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.944933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:13.944976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:13.944982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.944989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:13.945001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:13.945010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:13.945026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343158916546518:6060], now have 1 active actors on pipe
2025-10-09T21:52:13.945033Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343158554693719:8909]
2025-10-09T21:52:13.945035Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343158554693719:8909]
2025-10-09T21:52:13.945037Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343158554693719:8909]
2025-10-09T21:52:13.945050Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:13.945056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343158916546301:2432], now have 1 active actors on pipe
2025-10-09T21:52:13.945062Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:13.945064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:13.945068Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:13.945070Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:13.945070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:13.945072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:13.945090Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343158554693719:8909]
2025-10-09T21:52:13.945098Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:13.945105Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2386
2025-10-09T21:52:13.945107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:13.945129Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2386
2025-10-09T21:52:13.945169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.945173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.945192Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b78ccddc-88cd2aa2-758b6c17-935def55_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:13.945204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:13.945206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:13.945208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:13.945214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:13.945222Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:13.945235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:13.945370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:13.945375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:13.945385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:13.945481Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2387 partition: 0 MaxSeqNo: 25 sessionId: src_id|b78ccddc-88cd2aa2-758b6c17-935def55_0
2025-10-09T21:52:13.945781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|61bf6260-340da27a-f556b695-7b3f6f69_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b78ccddc-88cd2aa2-758b6c17-935def55_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:13.948051Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b78ccddc-88cd2aa2-758b6c17-935def55_0] Write session: try to update token
2025-10-09T21:52:13.948066Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b78ccddc-88cd2aa2-758b6c17-935def55_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:13.958535Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:13.958551Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:13.958553Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:13.958555Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:13.958562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343158554693719:8909]
2025-10-09T21:52:13.958565Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343158554693719:8909]
2025-10-09T21:52:13.958567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343158554693719:8909]
2025-10-09T21:52:13.958600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:13.958604Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:13.958606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:13.958608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:13.958613Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:13.958615Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:13.958732Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:13.960644Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2387 sessionId: src_id|b78ccddc-88cd2aa2-758b6c17-935def55_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:13.962822Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:14.021078Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343163211513824:7496]
2025-10-09T21:52:14.021310Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 2386, pipe: [1:7559343163211513824:7496]
2025-10-09T21:52:14.104392Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:14.104965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.104982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.105027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343163211513824:7496], now have 1 active actors on pipe
2025-10-09T21:52:14.105226Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:14.105249Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343163211513824:7496]
2025-10-09T21:52:14.105292Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:14.105298Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:14.105303Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:14.105374Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2387 reason: error at write: Disconnected sessionId: src_id|b78ccddc-88cd2aa2-758b6c17-935def55_0
2025-10-09T21:52:14.105560Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2387 sessionId: src_id|b78ccddc-88cd2aa2-758b6c17-935def55_0 is DEAD
2025-10-09T21:52:14.105721Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b78ccddc-88cd2aa2-758b6c17-935def55_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:14.105733Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b78ccddc-88cd2aa2-758b6c17-935def55_0] Write session will restart in 0.007044s
2025-10-09T21:52:14.105761Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b78ccddc-88cd2aa2-758b6c17-935def55_0] Write session: Do CDS request
2025-10-09T21:52:14.105767Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b78ccddc-88cd2aa2-758b6c17-935def55_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:14.113531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b78ccddc-88cd2aa2-758b6c17-935def55_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:14.113599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:14.113608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2388
2025-10-09T21:52:14.113724Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2388 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:14.113748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2388 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:14.113757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2388 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:14.113773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:14.114199Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2388 sessionId:  describe result for acl check
2025-10-09T21:52:14.114236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:14.114243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:14.114244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:14.114250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343163211513881:6044] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:14.114812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343163211513881:6044] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:14.117639Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343163211513881:6044] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:14.120215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:14.120425Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343163211513881:6044] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:14.120438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343163211513881:6044] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:14.120442Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343163211513881:6044] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:14.120449Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2388 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:14.122199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:14.122373Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:14.122381Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:14.122391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:14.122432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:14.122465Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343162849661239:8907]
2025-10-09T21:52:14.122900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:14.123128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:14.123164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:14.123400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:14.123440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.123444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.123445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.123446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.123446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.123447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.123448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.123449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.123449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.123453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.123460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.123462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.123464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.123466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.123469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.123471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.123473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.123475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.123480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.123485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.123489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.123493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.123496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.123499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.123501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.123504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.123507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.123511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.123513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.123516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.123520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.123523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.123525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.123529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.123531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.123534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.123554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:14.129684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:14.130755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:14.130771Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:14.130774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:14.130777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:14.130785Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2387 [2:7559343162849661239:8907]
2025-10-09T21:52:14.130800Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:14.130819Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:14.130827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:14.130830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:14.130833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:14.130849Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:14.130860Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:14.130867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:14.130910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.130915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.130916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.130918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:14.130982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:14.130990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.130999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.131005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.131014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:14.131030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343163211513826:2432], now have 1 active actors on pipe
2025-10-09T21:52:14.131036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343163211514042:6044], now have 1 active actors on pipe
2025-10-09T21:52:14.131041Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343162849661233:8907]
2025-10-09T21:52:14.131043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343162849661233:8907]
2025-10-09T21:52:14.131045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343162849661233:8907]
2025-10-09T21:52:14.131054Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:14.131068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:14.131068Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:14.131071Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:14.131072Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:14.131085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:14.131089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:14.131090Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2387
2025-10-09T21:52:14.131114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343162849661233:8907]
2025-10-09T21:52:14.131092Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2387
2025-10-09T21:52:14.131122Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:14.131123Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:14.131144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.131151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.131179Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e613680c-132e8e04-9689983b-4368b258_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:14.131189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:14.131191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:14.131193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:14.131202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:14.131212Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:14.131231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:14.131332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.131338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.131351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:14.131422Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2388 partition: 0 MaxSeqNo: 25 sessionId: src_id|e613680c-132e8e04-9689983b-4368b258_0
2025-10-09T21:52:14.131752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b78ccddc-88cd2aa2-758b6c17-935def55_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e613680c-132e8e04-9689983b-4368b258_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:14.134234Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e613680c-132e8e04-9689983b-4368b258_0] Write session: try to update token
2025-10-09T21:52:14.134249Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e613680c-132e8e04-9689983b-4368b258_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:14.145076Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:14.145095Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:14.145097Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:14.145099Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:14.145108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343162849661233:8907]
2025-10-09T21:52:14.145112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343162849661233:8907]
2025-10-09T21:52:14.145114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343162849661233:8907]
2025-10-09T21:52:14.145155Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:14.145158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:14.145166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.145166Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:14.145169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:14.145169Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:14.145306Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:14.146872Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2388 sessionId: src_id|e613680c-132e8e04-9689983b-4368b258_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:14.148981Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:14.205439Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343163211514056:7496]
2025-10-09T21:52:14.205700Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 2387, pipe: [1:7559343163211514056:7496]
2025-10-09T21:52:14.287790Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:14.288280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.288293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.288337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343163211514056:7496], now have 1 active actors on pipe
2025-10-09T21:52:14.288524Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:14.288593Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:14.288622Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343163211514056:7496]
2025-10-09T21:52:14.288633Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:14.288706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2388 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e613680c-132e8e04-9689983b-4368b258_0
2025-10-09T21:52:14.288867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2388 sessionId: src_id|e613680c-132e8e04-9689983b-4368b258_0 is DEAD
2025-10-09T21:52:14.289026Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e613680c-132e8e04-9689983b-4368b258_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:14.289039Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e613680c-132e8e04-9689983b-4368b258_0] Write session will restart in 0.007576s
2025-10-09T21:52:14.289064Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e613680c-132e8e04-9689983b-4368b258_0] Write session: Do CDS request
2025-10-09T21:52:14.289070Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e613680c-132e8e04-9689983b-4368b258_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:14.297493Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e613680c-132e8e04-9689983b-4368b258_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:14.297529Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:14.297539Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2389
2025-10-09T21:52:14.297722Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2389 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:14.297761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2389 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:14.297770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2389 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:14.297773Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:14.298175Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2389 sessionId:  describe result for acl check
2025-10-09T21:52:14.298210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:14.298216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:14.298217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:14.298223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343163211514158:6029] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:14.298741Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343163211514158:6029] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:14.300699Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343163211514158:6029] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:14.301793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:14.304000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:14.304139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343163211514158:6029] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:14.304153Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343163211514158:6029] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:14.304157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343163211514158:6029] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:14.304154Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:14.304164Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2389 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:14.304162Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:14.304170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:14.304210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:14.304236Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343162849661457:8905]
2025-10-09T21:52:14.304582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:14.304816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:14.304842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:14.305046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:14.305084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.305091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.305091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.305092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.305092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.305093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.305094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.305094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.305095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.305098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.305104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.305106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.305108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.305110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.305113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.305115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.305116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.305118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.305122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.305127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.305132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.305135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.305137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.305140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.305142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.305145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.305148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.305151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.305153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.305156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.305160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.305163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.305165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.305168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.305170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.305173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.305189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:14.311031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:14.311925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:14.311936Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:14.311937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:14.311940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:14.311944Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2388 [2:7559343162849661457:8905]
2025-10-09T21:52:14.311953Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:14.311970Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:14.311977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:14.311979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:14.311980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:14.311991Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:14.311995Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:14.312000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:14.312037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.312042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.312044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.312046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:14.312094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:14.312099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.312105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.312111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.312123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:14.312139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343162849661452:8905]
2025-10-09T21:52:14.312143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343162849661452:8905]
2025-10-09T21:52:14.312145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343162849661452:8905]
2025-10-09T21:52:14.312151Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:14.312161Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:14.312162Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:14.312165Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:14.312168Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:14.312172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:14.312174Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:14.312184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343163211514061:2432], now have 1 active actors on pipe
2025-10-09T21:52:14.312187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343163211514278:6029], now have 1 active actors on pipe
2025-10-09T21:52:14.312206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343162849661452:8905]
2025-10-09T21:52:14.312218Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:14.312237Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:14.312259Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2388
2025-10-09T21:52:14.312262Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2388
2025-10-09T21:52:14.312318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.312325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.312346Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b05e9976-ca6ecec8-25440ee1-9c9a2512_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:14.312355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:14.312358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:14.312359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:14.312366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:14.312372Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:14.312389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:14.312525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.312537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.312558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:14.312643Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2389 partition: 0 MaxSeqNo: 25 sessionId: src_id|b05e9976-ca6ecec8-25440ee1-9c9a2512_0
2025-10-09T21:52:14.312908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e613680c-132e8e04-9689983b-4368b258_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b05e9976-ca6ecec8-25440ee1-9c9a2512_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:14.315240Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b05e9976-ca6ecec8-25440ee1-9c9a2512_0] Write session: try to update token
2025-10-09T21:52:14.315258Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b05e9976-ca6ecec8-25440ee1-9c9a2512_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:14.325497Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:14.325511Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:14.325513Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:14.325515Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:14.325523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343162849661452:8905]
2025-10-09T21:52:14.325526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343162849661452:8905]
2025-10-09T21:52:14.325529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343162849661452:8905]
2025-10-09T21:52:14.325561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:14.325566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.325566Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:14.325568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:14.325571Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:14.325576Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:14.325692Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:14.327317Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2389 sessionId: src_id|b05e9976-ca6ecec8-25440ee1-9c9a2512_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:14.329498Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:14.388713Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343163211514295:7496]
2025-10-09T21:52:14.389043Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 2388, pipe: [1:7559343163211514295:7496]
2025-10-09T21:52:14.467845Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:14.468193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.468217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.468264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343163211514295:7496], now have 1 active actors on pipe
2025-10-09T21:52:14.468516Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:14.468557Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:14.468562Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343163211514295:7496]
2025-10-09T21:52:14.468562Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:14.468707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2389 reason: error at write: Disconnected sessionId: src_id|b05e9976-ca6ecec8-25440ee1-9c9a2512_0
2025-10-09T21:52:14.468721Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:14.468890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2389 sessionId: src_id|b05e9976-ca6ecec8-25440ee1-9c9a2512_0 is DEAD
2025-10-09T21:52:14.469107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b05e9976-ca6ecec8-25440ee1-9c9a2512_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:14.469116Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b05e9976-ca6ecec8-25440ee1-9c9a2512_0] Write session will restart in 0.007735s
2025-10-09T21:52:14.469139Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b05e9976-ca6ecec8-25440ee1-9c9a2512_0] Write session: Do CDS request
2025-10-09T21:52:14.469144Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b05e9976-ca6ecec8-25440ee1-9c9a2512_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:14.478419Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b05e9976-ca6ecec8-25440ee1-9c9a2512_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:14.478589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:14.478601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2390
2025-10-09T21:52:14.478790Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2390 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:14.478839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2390 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:14.478846Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2390 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:14.478851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:14.479277Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2390 sessionId:  describe result for acl check
2025-10-09T21:52:14.479321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:14.479331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:14.479333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:14.479340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343163211514435:5998] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:14.479772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343163211514435:5998] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:14.482043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343163211514435:5998] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:14.482815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:14.484580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:14.484760Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:14.484773Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:14.484786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:14.484845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:14.484889Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343162849661715:8882]
2025-10-09T21:52:14.485001Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343163211514435:5998] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:14.485011Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343163211514435:5998] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:14.485014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343163211514435:5998] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:14.485021Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2390 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:14.485454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:14.485745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:14.485796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:14.486015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:14.486058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.486063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.486064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.486065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.486065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.486066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.486066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.486067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.486068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.486071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.486078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.486080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.486082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.486084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.486086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.486088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.486090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.486092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.486097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.486102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.486106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.486109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.486113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.486116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.486119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.486122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.486125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.486127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.486130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.486133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.486136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.486139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.486141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.486144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.486147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.486150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.486166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:14.492282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:14.493558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:14.493573Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:14.493575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:14.493577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:14.493582Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2389 [2:7559343162849661715:8882]
2025-10-09T21:52:14.493594Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:14.493610Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:14.493619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:14.493621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:14.493623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:14.493638Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:14.493645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:14.493652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:14.493702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.493707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.493708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.493710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:14.493775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:14.493783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.493794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.493805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.493821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:14.493842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343163211514321:2432], now have 1 active actors on pipe
2025-10-09T21:52:14.493850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343163211514535:5998], now have 1 active actors on pipe
2025-10-09T21:52:14.493855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343162849661710:8882]
2025-10-09T21:52:14.493859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343162849661710:8882]
2025-10-09T21:52:14.493860Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343162849661710:8882]
2025-10-09T21:52:14.493869Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:14.493882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:14.493894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:14.493893Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:14.493896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:14.493897Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:14.493899Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:14.493914Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343162849661710:8882]
2025-10-09T21:52:14.493920Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:14.493923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:14.493907Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2389
2025-10-09T21:52:14.493946Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2389
2025-10-09T21:52:14.493954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.493962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.493984Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|604345d2-d7f39fd5-98e9a74a-412d73c8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:14.493992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:14.493995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:14.493997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:14.494004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:14.494016Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:14.494045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:14.494168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.494177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.494188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:14.494271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2390 partition: 0 MaxSeqNo: 25 sessionId: src_id|604345d2-d7f39fd5-98e9a74a-412d73c8_0
2025-10-09T21:52:14.494650Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b05e9976-ca6ecec8-25440ee1-9c9a2512_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|604345d2-d7f39fd5-98e9a74a-412d73c8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:14.497048Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|604345d2-d7f39fd5-98e9a74a-412d73c8_0] Write session: try to update token
2025-10-09T21:52:14.497064Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|604345d2-d7f39fd5-98e9a74a-412d73c8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:14.505473Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:14.505488Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:14.505496Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343162849661710:8882]
2025-10-09T21:52:14.505532Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:14.505535Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:14.505686Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:14.505832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:14.505966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:14.506108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:14.506279Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:14.506318Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:14.510526Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2390 sessionId: src_id|604345d2-d7f39fd5-98e9a74a-412d73c8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:14.513142Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:14.513304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.513318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.513348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:14.513415Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:14.517511Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:14.517524Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:14.517526Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:14.517527Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:14.517534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343162849661710:8882]
2025-10-09T21:52:14.517537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343162849661710:8882]
2025-10-09T21:52:14.517539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343162849661710:8882]
2025-10-09T21:52:14.517567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:14.517575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.517577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:14.517584Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:14.517601Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:14.517611Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:14.517693Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:14.568945Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343163211514548:7496]
2025-10-09T21:52:14.569182Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 2389, pipe: [1:7559343163211514548:7496]
2025-10-09T21:52:14.658498Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:14.665489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.665513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.665658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:14.665722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:14.665783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:14.665841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:14.665883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:14.665890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:14.665952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:14.666009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:14.666093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:14.666157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:14.666196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:14.666202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:14.666253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:14.666304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:14.666429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:14.666491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:14.666533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:14.666541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:14.666610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:14.666667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:14.666721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:14.666774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:14.666813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:14.666822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:14.666878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:14.666932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:14.666994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:14.667051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:14.667088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:14.667093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:14.667149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:14.667205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:14.667271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:14.667330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:14.667372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:14.667380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:14.667438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:14.667487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:14.667539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:14.667589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:14.667628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:14.667633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:14.667683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:14.667754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:14.667804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:14.667858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:14.667898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:14.667903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:14.667953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:14.668000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:14.668056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:14.668121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:14.668180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:14.668185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:14.668238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:14.668285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:14.668337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:14.668388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:14.668427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:14.668433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:14.668471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343163211514548:7496], now have 1 active actors on pipe
2025-10-09T21:52:14.668681Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343163211514548:7496]
2025-10-09T21:52:14.668688Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:14.668692Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:14.668704Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:14.668718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2390 reason: error at write: Disconnected sessionId: src_id|604345d2-d7f39fd5-98e9a74a-412d73c8_0
2025-10-09T21:52:14.668909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2390 sessionId: src_id|604345d2-d7f39fd5-98e9a74a-412d73c8_0 is DEAD
2025-10-09T21:52:14.669035Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|604345d2-d7f39fd5-98e9a74a-412d73c8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:14.669046Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|604345d2-d7f39fd5-98e9a74a-412d73c8_0] Write session will restart in 0.007137s
2025-10-09T21:52:14.669065Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|604345d2-d7f39fd5-98e9a74a-412d73c8_0] Write session: Do CDS request
2025-10-09T21:52:14.669070Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|604345d2-d7f39fd5-98e9a74a-412d73c8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:14.669216Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:14.677478Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|604345d2-d7f39fd5-98e9a74a-412d73c8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:14.677672Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:14.677687Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2391
2025-10-09T21:52:14.677838Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2391 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:14.677869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2391 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:14.677881Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2391 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:14.677888Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:14.678243Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2391 sessionId:  describe result for acl check
2025-10-09T21:52:14.678272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:14.678278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:14.678279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:14.678291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343163211514623:5988] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:14.678708Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343163211514623:5988] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:14.681817Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343163211514623:5988] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:14.682875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:14.684745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:14.684884Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:14.684891Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:14.684900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:14.684936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:14.684965Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343162849661937:8880]
2025-10-09T21:52:14.684993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343163211514623:5988] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:14.685004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343163211514623:5988] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:14.685006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343163211514623:5988] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:14.685013Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2391 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:14.685346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:14.685606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:14.685661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:14.685895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:14.685934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.685941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.685942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.685943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.685944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.685946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.685947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.685947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.685948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.685951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.685958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.685960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.685962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.685964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.685975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.685977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.685979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.685981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.685987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.685992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.685996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.685999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.686002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.686005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.686008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.686011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.686013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.686016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.686018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.686021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.686024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.686042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.686046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.686051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.686055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.686060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.686087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:14.692193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:14.693277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:14.693300Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:14.693301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:14.693304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:14.693310Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2390 [2:7559343162849661937:8880]
2025-10-09T21:52:14.693320Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:14.693341Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:14.693348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:14.693351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:14.693354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:14.693371Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:14.693381Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:14.693392Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:14.693444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.693449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.693450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.693452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:14.693501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:14.693506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.693513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.693523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.693536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:14.693552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343162849661932:8880]
2025-10-09T21:52:14.693557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343162849661932:8880]
2025-10-09T21:52:14.693559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343162849661932:8880]
2025-10-09T21:52:14.693565Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:14.693572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:14.693577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343163211514550:2432], now have 1 active actors on pipe
2025-10-09T21:52:14.693579Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:14.693581Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:14.693585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:14.693599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:14.693607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:14.693613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343163211514770:5988], now have 1 active actors on pipe
2025-10-09T21:52:14.693637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343162849661932:8880]
2025-10-09T21:52:14.693646Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:14.693651Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:14.693661Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2390
2025-10-09T21:52:14.693684Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2390
2025-10-09T21:52:14.693740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.693746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.693768Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fc7921a3-d57ec2c8-fe342368-fcb446b9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:14.693779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:14.693782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:14.693783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:14.693792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:14.693799Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:14.693810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:14.693990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.693996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.694005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:14.694087Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2391 partition: 0 MaxSeqNo: 25 sessionId: src_id|fc7921a3-d57ec2c8-fe342368-fcb446b9_0
2025-10-09T21:52:14.694416Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|604345d2-d7f39fd5-98e9a74a-412d73c8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fc7921a3-d57ec2c8-fe342368-fcb446b9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:14.696680Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc7921a3-d57ec2c8-fe342368-fcb446b9_0] Write session: try to update token
2025-10-09T21:52:14.696695Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc7921a3-d57ec2c8-fe342368-fcb446b9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:14.707763Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:14.707779Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:14.707781Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:14.707782Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:14.707790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343162849661932:8880]
2025-10-09T21:52:14.707793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343162849661932:8880]
2025-10-09T21:52:14.707796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343162849661932:8880]
2025-10-09T21:52:14.707830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:14.707834Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:14.707836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.707838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:14.707839Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:14.707842Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:14.707953Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:14.711461Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2391 sessionId: src_id|fc7921a3-d57ec2c8-fe342368-fcb446b9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:14.713889Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:14.768642Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343163211514786:7496]
2025-10-09T21:52:14.769153Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 2390, pipe: [1:7559343163211514786:7496]
2025-10-09T21:52:14.851796Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:14.852028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.852042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.852088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343163211514786:7496], now have 1 active actors on pipe
2025-10-09T21:52:14.852266Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:14.852330Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:14.852340Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:14.852345Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:14.852382Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343163211514786:7496]
2025-10-09T21:52:14.852402Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2391 reason: error at write: Disconnected sessionId: src_id|fc7921a3-d57ec2c8-fe342368-fcb446b9_0
2025-10-09T21:52:14.852563Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2391 sessionId: src_id|fc7921a3-d57ec2c8-fe342368-fcb446b9_0 is DEAD
2025-10-09T21:52:14.852709Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc7921a3-d57ec2c8-fe342368-fcb446b9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:14.852720Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc7921a3-d57ec2c8-fe342368-fcb446b9_0] Write session will restart in 0.007467s
2025-10-09T21:52:14.852741Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc7921a3-d57ec2c8-fe342368-fcb446b9_0] Write session: Do CDS request
2025-10-09T21:52:14.852746Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc7921a3-d57ec2c8-fe342368-fcb446b9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:14.861490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fc7921a3-d57ec2c8-fe342368-fcb446b9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:14.861635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:14.861648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2392
2025-10-09T21:52:14.861791Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2392 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:14.861827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2392 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:14.861832Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2392 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:14.861836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:14.862314Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2392 sessionId:  describe result for acl check
2025-10-09T21:52:14.862346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:14.862353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:14.862354Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:14.862360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343163211514867:5964] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:14.862894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343163211514867:5964] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:14.865067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343163211514867:5964] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:14.867386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343163211514867:5964] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:14.867396Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343163211514867:5964] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:14.867399Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343163211514867:5964] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:14.867404Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2392 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:14.867506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:14.869615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:14.869803Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:14.869814Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:14.869821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:14.869864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:14.869906Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343162849662164:8866]
2025-10-09T21:52:14.870502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:14.870821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:14.870861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:14.871134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:14.871177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.871183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.871184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.871185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.871186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.871186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.871187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.871188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.871188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.871192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.871199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.871201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.871203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.871205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.871208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.871211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.871213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.871215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.871220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.871225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:14.871230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.871234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:14.871246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.871249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:14.871252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.871256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:14.871260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.871263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:14.871266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.871270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:14.871273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.871277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.871280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.871286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.871289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.871293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.871310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:14.877827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:14.879049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:14.879065Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:14.879067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:14.879070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:14.879076Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2391 [2:7559343162849662164:8866]
2025-10-09T21:52:14.879089Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:14.879103Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:14.879105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:14.879107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:14.879108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:14.879121Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:14.879125Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:14.879130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:14.879179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.879186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.879188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.879191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:14.879259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:14.879270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.879279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:14.879286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:14.879296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:14.879309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343162849662158:8866]
2025-10-09T21:52:14.879316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343162849662158:8866]
2025-10-09T21:52:14.879319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343162849662158:8866]
2025-10-09T21:52:14.879328Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:14.879340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:14.879341Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:14.879344Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:14.879346Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:14.879355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:14.879358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:14.879368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343163211515012:5964], now have 1 active actors on pipe
2025-10-09T21:52:14.879371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343163211514792:2432], now have 1 active actors on pipe
2025-10-09T21:52:14.879396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343162849662158:8866]
2025-10-09T21:52:14.879407Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:14.879418Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2391
2025-10-09T21:52:14.879427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:14.879419Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2391
2025-10-09T21:52:14.879526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.879535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.879552Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|14b48d11-da4bb0e2-e80d8ea2-9d5974be_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:14.879562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:14.879566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:14.879568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:14.879574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:14.879587Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:14.879597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:14.879761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.879774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.879784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:14.879881Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2392 partition: 0 MaxSeqNo: 25 sessionId: src_id|14b48d11-da4bb0e2-e80d8ea2-9d5974be_0
2025-10-09T21:52:14.880173Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fc7921a3-d57ec2c8-fe342368-fcb446b9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|14b48d11-da4bb0e2-e80d8ea2-9d5974be_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:14.882448Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14b48d11-da4bb0e2-e80d8ea2-9d5974be_0] Write session: try to update token
2025-10-09T21:52:14.882461Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14b48d11-da4bb0e2-e80d8ea2-9d5974be_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:14.890887Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:14.890904Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:14.890913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343162849662158:8866]
2025-10-09T21:52:14.890950Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:14.890959Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:14.891135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:14.891291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:14.891440Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:14.891584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:14.891808Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:14.891841Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:14.894861Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2392 sessionId: src_id|14b48d11-da4bb0e2-e80d8ea2-9d5974be_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:14.897118Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:14.897315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:14.897338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:14.897365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:14.897462Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:14.903604Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:14.903621Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:14.903625Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:14.903627Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:14.903638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343162849662158:8866]
2025-10-09T21:52:14.903643Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343162849662158:8866]
2025-10-09T21:52:14.903649Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343162849662158:8866]
2025-10-09T21:52:14.903683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:14.903690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:14.903691Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:14.903692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:14.903697Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:14.903702Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:14.903873Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:14.952442Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343163211515029:7496]
2025-10-09T21:52:14.952703Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 2391, pipe: [1:7559343163211515029:7496]
2025-10-09T21:52:15.055848Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:15.063619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:15.063645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:15.063799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.063856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.063916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.063981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.064026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.064037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.064102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.064162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.064223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.064291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.064339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.064347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.064415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.064520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.064577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.064634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.064674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.064680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.064743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.064813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.064883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.064942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.064983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.064989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.065052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.065109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.065167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.065227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.065267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.065273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.065355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.065427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.065495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.065560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.065611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.065619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.065677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.065732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.065792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.065849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.065892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.065897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.065955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.066007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.066085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.066141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.066183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.066191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.066252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.066308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.066375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.066440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.066480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.066486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.066545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.066599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.066659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.066724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.066773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.066779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.066836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343163211515029:7496], now have 1 active actors on pipe
2025-10-09T21:52:15.067109Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343163211515029:7496]
2025-10-09T21:52:15.067111Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:15.067114Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:15.067134Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:15.067146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2392 reason: error at write: Disconnected sessionId: src_id|14b48d11-da4bb0e2-e80d8ea2-9d5974be_0
2025-10-09T21:52:15.067324Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2392 sessionId: src_id|14b48d11-da4bb0e2-e80d8ea2-9d5974be_0 is DEAD
2025-10-09T21:52:15.067479Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14b48d11-da4bb0e2-e80d8ea2-9d5974be_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:15.067487Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14b48d11-da4bb0e2-e80d8ea2-9d5974be_0] Write session will restart in 0.008283s
2025-10-09T21:52:15.067507Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14b48d11-da4bb0e2-e80d8ea2-9d5974be_0] Write session: Do CDS request
2025-10-09T21:52:15.067511Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14b48d11-da4bb0e2-e80d8ea2-9d5974be_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:15.067858Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:15.076494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|14b48d11-da4bb0e2-e80d8ea2-9d5974be_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:15.076641Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:15.076654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2393
2025-10-09T21:52:15.076800Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2393 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:15.076834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2393 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:15.076845Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2393 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:15.076851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:15.077306Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2393 sessionId:  describe result for acl check
2025-10-09T21:52:15.077355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:15.077362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:15.077364Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:15.077371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343167506482403:5939] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:15.078012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343167506482403:5939] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:15.081369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343167506482403:5939] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:15.082378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:15.084955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:15.085200Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:15.085209Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:15.085218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:15.085260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:15.085266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343167506482403:5939] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:15.085292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343167506482403:5939] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:15.085296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343167506482403:5939] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:15.085306Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2393 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:15.085318Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343167144629683:8855]
2025-10-09T21:52:15.085804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:15.086053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:15.086086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:15.086334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:15.086391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.086398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.086399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.086400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.086401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.086401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.086402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.086403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.086403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.086407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.086414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.086416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.086419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.086421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.086423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.086425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.086427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.086429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.086435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.086440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.086445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.086448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.086450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.086454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.086456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.086459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.086461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.086464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.086467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.086470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.086473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.086476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.086479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.086482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.086485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.086488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.086504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:15.092966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:15.094011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:15.094045Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:15.094049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:15.094054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:15.094067Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2392 [2:7559343167144629683:8855]
2025-10-09T21:52:15.094086Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:15.094109Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:15.094115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:15.094117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:15.094119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:15.094135Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:15.094145Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:15.094154Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:15.094228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.094237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.094239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.094242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:15.094312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:15.094321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.094336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.094349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.094363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:15.094384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343167506482328:2432], now have 1 active actors on pipe
2025-10-09T21:52:15.094390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343167506482549:5939], now have 1 active actors on pipe
2025-10-09T21:52:15.094397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343167144629678:8855]
2025-10-09T21:52:15.094398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343167144629678:8855]
2025-10-09T21:52:15.094400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343167144629678:8855]
2025-10-09T21:52:15.094410Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:15.094422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:15.094430Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:15.094432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:15.094433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:15.094435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:15.094437Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:15.094448Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2392
2025-10-09T21:52:15.094471Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2392
2025-10-09T21:52:15.094470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343167144629678:8855]
2025-10-09T21:52:15.094477Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:15.094479Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:15.094503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:15.094511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:15.094539Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|98b95ee4-e5247a13-dce80d25-be6b8294_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:15.094548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:15.094551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:15.094555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:15.094562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:15.094570Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:15.094583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:15.094783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:15.094790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:15.094801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:15.094924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2393 partition: 0 MaxSeqNo: 25 sessionId: src_id|98b95ee4-e5247a13-dce80d25-be6b8294_0
2025-10-09T21:52:15.095288Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|14b48d11-da4bb0e2-e80d8ea2-9d5974be_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|98b95ee4-e5247a13-dce80d25-be6b8294_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:15.097838Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98b95ee4-e5247a13-dce80d25-be6b8294_0] Write session: try to update token
2025-10-09T21:52:15.097856Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98b95ee4-e5247a13-dce80d25-be6b8294_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:15.109870Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:15.109896Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:15.109898Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:15.109900Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:15.109909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343167144629678:8855]
2025-10-09T21:52:15.109913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343167144629678:8855]
2025-10-09T21:52:15.109916Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343167144629678:8855]
2025-10-09T21:52:15.109957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:15.109963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.109966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:15.109985Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:15.109994Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:15.109999Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:15.110161Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:15.112126Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2393 sessionId: src_id|98b95ee4-e5247a13-dce80d25-be6b8294_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:15.114584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:15.167481Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343167506482560:7496]
2025-10-09T21:52:15.167773Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 2392, pipe: [1:7559343167506482560:7496]
2025-10-09T21:52:15.257935Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:15.258227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:15.258241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:15.258288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343167506482560:7496], now have 1 active actors on pipe
2025-10-09T21:52:15.258454Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:15.258524Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:15.258534Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343167506482560:7496]
2025-10-09T21:52:15.258558Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:15.258637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2393 reason: status is not ok: tablet will be restarted right now sessionId: src_id|98b95ee4-e5247a13-dce80d25-be6b8294_0
2025-10-09T21:52:15.258818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2393 sessionId: src_id|98b95ee4-e5247a13-dce80d25-be6b8294_0 is DEAD
2025-10-09T21:52:15.258963Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98b95ee4-e5247a13-dce80d25-be6b8294_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:15.258974Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98b95ee4-e5247a13-dce80d25-be6b8294_0] Write session will restart in 0.007171s
2025-10-09T21:52:15.258994Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98b95ee4-e5247a13-dce80d25-be6b8294_0] Write session: Do CDS request
2025-10-09T21:52:15.258999Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98b95ee4-e5247a13-dce80d25-be6b8294_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:15.267522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98b95ee4-e5247a13-dce80d25-be6b8294_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:15.267593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:15.267608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2394
2025-10-09T21:52:15.267736Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2394 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:15.267769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2394 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:15.267779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2394 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:15.267791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:15.268221Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2394 sessionId:  describe result for acl check
2025-10-09T21:52:15.268253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:15.268258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:15.268260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:15.268264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343167506482646:5921] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:15.268697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343167506482646:5921] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:15.271139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343167506482646:5921] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:15.272660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:15.274678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:15.274844Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:15.274854Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:15.274863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:15.274904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:15.274934Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343167144629907:8863]
2025-10-09T21:52:15.275041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343167506482646:5921] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:15.275055Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343167506482646:5921] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:15.275059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343167506482646:5921] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:15.275067Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2394 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:15.275321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:15.275562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:15.275595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:15.275806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:15.275838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.275844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.275845Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.275846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.275847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.275848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.275850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.275851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.275852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.275857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.275868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.275875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.275878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.275880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.275882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.275884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.275886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.275888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.275893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.275898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.275902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.275906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.275909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.275912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.275914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.275917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.275920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.275923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.275926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.275929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.275932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.275935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.275939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.275942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.275945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.275948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.275964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:15.281850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:15.282897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:15.282911Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:15.282913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:15.282916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:15.282921Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2393 [2:7559343167144629907:8863]
2025-10-09T21:52:15.282932Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:15.282946Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:15.282952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:15.282954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:15.282956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:15.282969Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:15.282977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:15.282983Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:15.283038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.283045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.283048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.283051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:15.283126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:15.283134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.283141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.283148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.283159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:15.283176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343167506482788:5921], now have 1 active actors on pipe
2025-10-09T21:52:15.283178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343167506482566:2432], now have 1 active actors on pipe
2025-10-09T21:52:15.283184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343167144629902:8863]
2025-10-09T21:52:15.283185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343167144629902:8863]
2025-10-09T21:52:15.283186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343167144629902:8863]
2025-10-09T21:52:15.283196Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:15.283222Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:15.283230Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:15.283233Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:15.283234Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:15.283236Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:15.283236Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:15.283227Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2393
2025-10-09T21:52:15.283231Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2393
2025-10-09T21:52:15.283262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343167144629902:8863]
2025-10-09T21:52:15.283269Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:15.283272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:15.283293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:15.283303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:15.283344Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e69a2ee3-9495d0c4-be2e2c11-51552a88_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:15.283353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:15.283357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:15.283362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:15.283371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:15.283378Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:15.283396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:15.283517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:15.283523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:15.283536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:15.283608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2394 partition: 0 MaxSeqNo: 25 sessionId: src_id|e69a2ee3-9495d0c4-be2e2c11-51552a88_0
2025-10-09T21:52:15.283948Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98b95ee4-e5247a13-dce80d25-be6b8294_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e69a2ee3-9495d0c4-be2e2c11-51552a88_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:15.286205Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e69a2ee3-9495d0c4-be2e2c11-51552a88_0] Write session: try to update token
2025-10-09T21:52:15.286223Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e69a2ee3-9495d0c4-be2e2c11-51552a88_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:15.296733Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:15.296748Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:15.296750Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:15.296751Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:15.296759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343167144629902:8863]
2025-10-09T21:52:15.296763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343167144629902:8863]
2025-10-09T21:52:15.296766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343167144629902:8863]
2025-10-09T21:52:15.296800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:15.296805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.296807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:15.296813Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:15.296818Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:15.296824Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:15.296929Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:15.298181Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2394 sessionId: src_id|e69a2ee3-9495d0c4-be2e2c11-51552a88_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:15.300302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:15.358428Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343167506482808:7496]
2025-10-09T21:52:15.358656Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 2393, pipe: [1:7559343167506482808:7496]
2025-10-09T21:52:15.440428Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:15.440949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:15.440962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:15.440998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343167506482808:7496], now have 1 active actors on pipe
2025-10-09T21:52:15.441132Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:15.441203Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343167506482808:7496]
2025-10-09T21:52:15.441212Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:15.441231Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:15.441297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2394 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e69a2ee3-9495d0c4-be2e2c11-51552a88_0
2025-10-09T21:52:15.441472Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2394 sessionId: src_id|e69a2ee3-9495d0c4-be2e2c11-51552a88_0 is DEAD
2025-10-09T21:52:15.441631Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e69a2ee3-9495d0c4-be2e2c11-51552a88_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:15.441640Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e69a2ee3-9495d0c4-be2e2c11-51552a88_0] Write session will restart in 0.005776s
2025-10-09T21:52:15.441661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e69a2ee3-9495d0c4-be2e2c11-51552a88_0] Write session: Do CDS request
2025-10-09T21:52:15.441666Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e69a2ee3-9495d0c4-be2e2c11-51552a88_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:15.448379Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e69a2ee3-9495d0c4-be2e2c11-51552a88_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:15.448412Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:15.448422Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2395
2025-10-09T21:52:15.448560Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2395 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:15.448598Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2395 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:15.448605Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2395 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:15.448609Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:15.449023Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2395 sessionId:  describe result for acl check
2025-10-09T21:52:15.449052Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:15.449057Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:15.449059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:15.449063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343167506482846:5898] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:15.449531Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343167506482846:5898] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:15.451726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343167506482846:5898] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:15.453809Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343167506482846:5898] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:15.453822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343167506482846:5898] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:15.453826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343167506482846:5898] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:15.453832Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2395 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:15.455289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:15.457115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:15.457309Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:15.457319Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:15.457333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:15.457386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:15.457423Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343167144630133:8861]
2025-10-09T21:52:15.457962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:15.458252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:15.458282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:15.458531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:15.458582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.458587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.458589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.458590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.458591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.458592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.458593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.458594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.458595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.458600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.458610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.458614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.458618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.458622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.458626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.458628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.458630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.458632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.458636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.458641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.458646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.458649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.458652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.458654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.458657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.458660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.458662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.458665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.458668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.458671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.458674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.458677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.458680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.458683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.458685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.458688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.458704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:15.464617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:15.465791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:15.465803Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:15.465804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:15.465807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:15.465812Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2394 [2:7559343167144630133:8861]
2025-10-09T21:52:15.465821Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:15.465832Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:15.465834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:15.465836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:15.465838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:15.465849Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:15.465861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:15.465866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:15.465910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.465917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.465918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.465920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:15.465971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:15.465978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.465989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.465999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.466020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:15.466054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343167506483027:5898], now have 1 active actors on pipe
2025-10-09T21:52:15.466060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343167144630128:8861]
2025-10-09T21:52:15.466062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343167144630128:8861]
2025-10-09T21:52:15.466064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343167144630128:8861]
2025-10-09T21:52:15.466074Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:15.466080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343167506482810:2432], now have 1 active actors on pipe
2025-10-09T21:52:15.466088Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:15.466096Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:15.466099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:15.466102Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:15.466102Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:15.466103Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:15.466122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343167144630128:8861]
2025-10-09T21:52:15.466124Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:15.466127Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:15.466184Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2394
2025-10-09T21:52:15.466219Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2394
2025-10-09T21:52:15.466248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:15.466262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:15.466294Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6afae79c-3a131ff2-efae729c-6ab12d5f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:15.466308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:15.466314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:15.466317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:15.466327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:15.466343Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:15.466362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:15.466592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:15.466601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:15.466616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:15.466738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2395 partition: 0 MaxSeqNo: 25 sessionId: src_id|6afae79c-3a131ff2-efae729c-6ab12d5f_0
2025-10-09T21:52:15.467055Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e69a2ee3-9495d0c4-be2e2c11-51552a88_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6afae79c-3a131ff2-efae729c-6ab12d5f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:15.469350Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6afae79c-3a131ff2-efae729c-6ab12d5f_0] Write session: try to update token
2025-10-09T21:52:15.469364Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6afae79c-3a131ff2-efae729c-6ab12d5f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:15.481615Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2395 sessionId: src_id|6afae79c-3a131ff2-efae729c-6ab12d5f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:15.483341Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:15.483358Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:15.483366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343167144630128:8861]
2025-10-09T21:52:15.483412Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:15.483415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:15.483587Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:15.483729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:15.483858Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:15.483863Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:15.483984Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:15.484157Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:15.484210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:15.484227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:15.484246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:15.484278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:15.484368Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:15.490267Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:15.490282Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:15.490284Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:15.490286Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:15.490295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343167144630128:8861]
2025-10-09T21:52:15.490298Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343167144630128:8861]
2025-10-09T21:52:15.490301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343167144630128:8861]
2025-10-09T21:52:15.490332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:15.490339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.490346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:15.490358Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:15.490380Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:15.490387Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:15.490480Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:15.541264Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343167506483067:7496]
2025-10-09T21:52:15.541562Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 2394, pipe: [1:7559343167506483067:7496]
2025-10-09T21:52:15.628519Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:15.636012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:15.636039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:15.636198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.636253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.636319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.636386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.636433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.636445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.636512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.636578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.636637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.636698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.636742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.636755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.636820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.636882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.636945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.637007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.637051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.637058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.637124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.637185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.637245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.637322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.637363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.637371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.637433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.637497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.637641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.637705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.637753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.637761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.637826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.637884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.637946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.638011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.638071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.638079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.638140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.638195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.638260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.638322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.638363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.638370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.638433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.638489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.638546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.638609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.638653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.638661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.638723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.638784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.638840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.638894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.638937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.638945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.639003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:15.639060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:15.639119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:15.639182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:15.639225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:15.639232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:15.639286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343167506483067:7496], now have 1 active actors on pipe
2025-10-09T21:52:15.639523Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:15.639527Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:15.639538Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:15.639574Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343167506483067:7496]
2025-10-09T21:52:15.639574Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2395 reason: error at write: Disconnected sessionId: src_id|6afae79c-3a131ff2-efae729c-6ab12d5f_0
2025-10-09T21:52:15.639810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2395 sessionId: src_id|6afae79c-3a131ff2-efae729c-6ab12d5f_0 is DEAD
2025-10-09T21:52:15.639920Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6afae79c-3a131ff2-efae729c-6ab12d5f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:15.639930Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6afae79c-3a131ff2-efae729c-6ab12d5f_0] Write session will restart in 0.009409s
2025-10-09T21:52:15.639954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6afae79c-3a131ff2-efae729c-6ab12d5f_0] Write session: Do CDS request
2025-10-09T21:52:15.639959Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6afae79c-3a131ff2-efae729c-6ab12d5f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:15.640271Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:15.650480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6afae79c-3a131ff2-efae729c-6ab12d5f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:15.650574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:15.650588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2396
2025-10-09T21:52:15.650696Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2396 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:15.650740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2396 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:15.650751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2396 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:15.650755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:15.651199Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2396 sessionId:  describe result for acl check
2025-10-09T21:52:15.651234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:15.651241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:15.651243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:15.651249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343167506483205:5872] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:15.651770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343167506483205:5872] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:15.653643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:15.653815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343167506483205:5872] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:15.655537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:15.655685Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:15.655692Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:15.655701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:15.655751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:15.655790Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343167144630400:8838]
2025-10-09T21:52:15.656075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343167506483205:5872] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:15.656086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343167506483205:5872] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:15.656090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343167506483205:5872] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:15.656097Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2396 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:15.656334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:15.656605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:15.656637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:15.656866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:15.656902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.656907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.656908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.656909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.656909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.656910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.656910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.656911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.656911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.656914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.656921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.656923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.656925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.656927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.656930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.656932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.656933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.656935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.656940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.656946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.656950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.656953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.656956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.656959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.656961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.656964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.656966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.656970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.656972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.656975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.656978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.656981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.656983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.656986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.656990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.656995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.657020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:15.662947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:15.663987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:15.663999Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:15.664001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:15.664004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:15.664009Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2395 [2:7559343167144630400:8838]
2025-10-09T21:52:15.664019Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:15.664033Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:15.664035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:15.664037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:15.664038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:15.664051Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:15.664055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:15.664060Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:15.664101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.664107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.664108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.664110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:15.664302Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2395
2025-10-09T21:52:15.664404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:15.664411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.664423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.664435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.664448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:15.664468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343167506483069:2432], now have 1 active actors on pipe
2025-10-09T21:52:15.664475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343167506483293:5872], now have 1 active actors on pipe
2025-10-09T21:52:15.664474Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2395
2025-10-09T21:52:15.664485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343167144630395:8838]
2025-10-09T21:52:15.664487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343167144630395:8838]
2025-10-09T21:52:15.664488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343167144630395:8838]
2025-10-09T21:52:15.664498Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:15.664505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:15.664507Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:15.664509Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:15.664510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:15.664518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:15.664520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:15.664555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:15.664562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:15.664573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343167144630395:8838]
2025-10-09T21:52:15.664582Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:15.664590Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:15.664604Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d1e4c286-71f23b1a-99a75890-1d65d9d4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:15.664610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:15.664617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:15.664618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:15.664627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:15.664634Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:15.664656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:15.664820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:15.664835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:15.664858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:15.665035Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2396 partition: 0 MaxSeqNo: 25 sessionId: src_id|d1e4c286-71f23b1a-99a75890-1d65d9d4_0
2025-10-09T21:52:15.665404Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6afae79c-3a131ff2-efae729c-6ab12d5f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d1e4c286-71f23b1a-99a75890-1d65d9d4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:15.667608Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1e4c286-71f23b1a-99a75890-1d65d9d4_0] Write session: try to update token
2025-10-09T21:52:15.667628Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1e4c286-71f23b1a-99a75890-1d65d9d4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:15.678667Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:15.678682Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:15.678684Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:15.678685Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:15.678694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343167144630395:8838]
2025-10-09T21:52:15.678697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343167144630395:8838]
2025-10-09T21:52:15.678701Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343167144630395:8838]
2025-10-09T21:52:15.678734Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:15.678738Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:15.678741Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:15.678743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:15.678749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.678751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:15.678870Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:15.679663Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2396 sessionId: src_id|d1e4c286-71f23b1a-99a75890-1d65d9d4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:15.682457Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:15.739939Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343167506483305:7496]
2025-10-09T21:52:15.740242Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 2395, pipe: [1:7559343167506483305:7496]
2025-10-09T21:52:15.827097Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:15.827514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:15.827526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:15.827565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343167506483305:7496], now have 1 active actors on pipe
2025-10-09T21:52:15.827738Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:15.827777Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:15.827793Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:15.827821Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343167506483305:7496]
2025-10-09T21:52:15.827854Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:15.827855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2396 reason: error at write: Disconnected sessionId: src_id|d1e4c286-71f23b1a-99a75890-1d65d9d4_0
2025-10-09T21:52:15.828040Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2396 sessionId: src_id|d1e4c286-71f23b1a-99a75890-1d65d9d4_0 is DEAD
2025-10-09T21:52:15.828192Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1e4c286-71f23b1a-99a75890-1d65d9d4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:15.828202Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1e4c286-71f23b1a-99a75890-1d65d9d4_0] Write session will restart in 0.005045s
2025-10-09T21:52:15.828223Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1e4c286-71f23b1a-99a75890-1d65d9d4_0] Write session: Do CDS request
2025-10-09T21:52:15.828228Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1e4c286-71f23b1a-99a75890-1d65d9d4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:15.834441Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d1e4c286-71f23b1a-99a75890-1d65d9d4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:15.834534Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:15.834553Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2397
2025-10-09T21:52:15.834711Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2397 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:15.834750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2397 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:15.834757Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2397 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:15.834766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:15.835318Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2397 sessionId:  describe result for acl check
2025-10-09T21:52:15.835359Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:15.835366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:15.835368Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:15.835374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343167506483357:5851] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:15.836123Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343167506483357:5851] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:15.837946Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343167506483357:5851] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:15.840230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343167506483357:5851] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:15.840246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343167506483357:5851] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:15.840249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343167506483357:5851] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:15.840257Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2397 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:15.840749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:15.842615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:15.842750Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:15.842758Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:15.842767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:15.842811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:15.842841Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343167144630631:8836]
2025-10-09T21:52:15.843262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:15.843479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:15.843508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:15.843745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:15.843791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.843797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.843798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.843799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.843800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.843801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.843802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.843803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.843804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.843808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.843818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.843822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.843826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.843829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.843833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.843836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.843839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.843842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.843849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.843857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:15.843863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.843868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:15.843873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.843878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:15.843884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.843889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:15.843893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.843898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:15.843902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.843907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:15.843913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.843918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.843920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.843923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.843926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.843928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.843943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:15.849852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:15.850914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:15.850928Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:15.850931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:15.850935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:15.850942Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2396 [2:7559343167144630631:8836]
2025-10-09T21:52:15.850954Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:15.850970Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:15.850978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:15.850981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:15.850984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:15.851002Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:15.851012Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:15.851019Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:15.851071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.851077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.851079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.851082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:15.851144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:15.851151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.851162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:15.851171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:15.851191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:15.851215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343167506483307:2432], now have 1 active actors on pipe
2025-10-09T21:52:15.851222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343167506483530:5851], now have 1 active actors on pipe
2025-10-09T21:52:15.851230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343167144630626:8836]
2025-10-09T21:52:15.851236Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343167144630626:8836]
2025-10-09T21:52:15.851238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343167144630626:8836]
2025-10-09T21:52:15.851250Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:15.851269Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:15.851270Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:15.851274Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:15.851276Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:15.851288Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:15.851292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:15.851276Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2396
2025-10-09T21:52:15.851278Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2396
2025-10-09T21:52:15.851321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343167144630626:8836]
2025-10-09T21:52:15.851328Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:15.851330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:15.851355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:15.851363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:15.851396Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|64416439-d6fcf270-522426ba-7221fcf4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:15.851407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:15.851411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:15.851414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:15.851424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:15.851439Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:15.851463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:15.851577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:15.851584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:15.851596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:15.851673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2397 partition: 0 MaxSeqNo: 25 sessionId: src_id|64416439-d6fcf270-522426ba-7221fcf4_0
2025-10-09T21:52:15.851991Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d1e4c286-71f23b1a-99a75890-1d65d9d4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|64416439-d6fcf270-522426ba-7221fcf4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:15.854322Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|64416439-d6fcf270-522426ba-7221fcf4_0] Write session: try to update token
2025-10-09T21:52:15.854338Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|64416439-d6fcf270-522426ba-7221fcf4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:15.865225Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:15.865251Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:15.865253Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:15.865255Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:15.865266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343167144630626:8836]
2025-10-09T21:52:15.865270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343167144630626:8836]
2025-10-09T21:52:15.865274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343167144630626:8836]
2025-10-09T21:52:15.865337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:15.865350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:15.865353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:15.865356Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:15.865362Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:15.865365Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:15.865494Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:15.866958Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2397 sessionId: src_id|64416439-d6fcf270-522426ba-7221fcf4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:15.869216Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:15.927807Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343167506483545:7496]
2025-10-09T21:52:15.928075Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 2396, pipe: [1:7559343167506483545:7496]
2025-10-09T21:52:16.011870Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:16.012515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.012538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.012580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343167506483545:7496], now have 1 active actors on pipe
2025-10-09T21:52:16.012756Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:16.012818Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:16.012822Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343167506483545:7496]
2025-10-09T21:52:16.012849Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:16.012926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2397 reason: status is not ok: tablet will be restarted right now sessionId: src_id|64416439-d6fcf270-522426ba-7221fcf4_0
2025-10-09T21:52:16.013103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2397 sessionId: src_id|64416439-d6fcf270-522426ba-7221fcf4_0 is DEAD
2025-10-09T21:52:16.013258Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|64416439-d6fcf270-522426ba-7221fcf4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:16.013271Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|64416439-d6fcf270-522426ba-7221fcf4_0] Write session will restart in 0.009521s
2025-10-09T21:52:16.013305Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|64416439-d6fcf270-522426ba-7221fcf4_0] Write session: Do CDS request
2025-10-09T21:52:16.013311Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|64416439-d6fcf270-522426ba-7221fcf4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:16.023466Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|64416439-d6fcf270-522426ba-7221fcf4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:16.023554Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:16.023565Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2398
2025-10-09T21:52:16.023684Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2398 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:16.023709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2398 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:16.023717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2398 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:16.023720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:16.024154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2398 sessionId:  describe result for acl check
2025-10-09T21:52:16.024188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:16.024195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:16.024197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:16.024202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343171801450973:5844] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:16.024753Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343171801450973:5844] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:16.027264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343171801450973:5844] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:16.027770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:16.029798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:16.029921Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343171801450973:5844] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:16.029934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343171801450973:5844] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:16.029938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343171801450973:5844] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:16.029946Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2398 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:16.029973Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:16.029983Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:16.029994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:16.030054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:16.030094Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343171439598155:8833]
2025-10-09T21:52:16.030675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:16.030993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:16.031037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:16.031354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:16.031404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.031410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.031411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.031412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.031414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.031415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.031417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.031418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.031419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.031424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.031435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.031444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.031448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.031452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.031455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.031458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.031461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.031465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.031471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.031479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.031490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.031496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.031505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.031512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.031520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.031526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.031532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.031541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.031545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.031550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.031557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.031562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.031571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.031576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.031581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.031590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.031619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:16.037906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:16.039172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:16.039186Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:16.039188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:16.039193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:16.039203Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2397 [2:7559343171439598155:8833]
2025-10-09T21:52:16.039218Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:16.039236Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:16.039246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:16.039249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:16.039251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:16.039269Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:16.039275Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:16.039283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:16.039343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.039352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.039354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.039357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:16.039426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:16.039435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.039448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.039461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.039479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:16.039505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801451070:5844], now have 1 active actors on pipe
2025-10-09T21:52:16.039512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801450843:2432], now have 1 active actors on pipe
2025-10-09T21:52:16.039520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343171439598150:8833]
2025-10-09T21:52:16.039523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343171439598150:8833]
2025-10-09T21:52:16.039525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343171439598150:8833]
2025-10-09T21:52:16.039540Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:16.039554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:16.039574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:16.039577Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:16.039582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:16.039582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:16.039584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:16.039573Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2397
2025-10-09T21:52:16.039578Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2397
2025-10-09T21:52:16.039613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343171439598150:8833]
2025-10-09T21:52:16.039621Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:16.039625Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:16.039646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.039656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.039697Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e47667fa-a81a3235-ded32d9b-259ece03_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:16.039709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:16.039713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:16.039716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:16.039725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:16.039734Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:16.039756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:16.039889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.039899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.039914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:16.040006Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2398 partition: 0 MaxSeqNo: 25 sessionId: src_id|e47667fa-a81a3235-ded32d9b-259ece03_0
2025-10-09T21:52:16.040339Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|64416439-d6fcf270-522426ba-7221fcf4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e47667fa-a81a3235-ded32d9b-259ece03_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:16.042692Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e47667fa-a81a3235-ded32d9b-259ece03_0] Write session: try to update token
2025-10-09T21:52:16.042708Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e47667fa-a81a3235-ded32d9b-259ece03_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:16.054285Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:16.054305Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:16.054308Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:16.054310Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:16.054323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343171439598150:8833]
2025-10-09T21:52:16.054328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343171439598150:8833]
2025-10-09T21:52:16.054333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343171439598150:8833]
2025-10-09T21:52:16.054376Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:16.054378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:16.054380Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:16.054385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.054386Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:16.054388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:16.054539Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:16.055485Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2398 sessionId: src_id|e47667fa-a81a3235-ded32d9b-259ece03_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:16.057846Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:16.113151Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343171801451081:7496]
2025-10-09T21:52:16.113458Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 2397, pipe: [1:7559343171801451081:7496]
2025-10-09T21:52:16.204474Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:16.204984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.205002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.205047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801451081:7496], now have 1 active actors on pipe
2025-10-09T21:52:16.205258Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:16.205324Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:16.205335Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:16.205379Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:16.205400Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2398 reason: error at write: Disconnected sessionId: src_id|e47667fa-a81a3235-ded32d9b-259ece03_0
2025-10-09T21:52:16.205421Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343171801451081:7496]
2025-10-09T21:52:16.205569Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2398 sessionId: src_id|e47667fa-a81a3235-ded32d9b-259ece03_0 is DEAD
2025-10-09T21:52:16.205708Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e47667fa-a81a3235-ded32d9b-259ece03_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:16.205718Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e47667fa-a81a3235-ded32d9b-259ece03_0] Write session will restart in 0.009137s
2025-10-09T21:52:16.205739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e47667fa-a81a3235-ded32d9b-259ece03_0] Write session: Do CDS request
2025-10-09T21:52:16.205744Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e47667fa-a81a3235-ded32d9b-259ece03_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:16.215447Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e47667fa-a81a3235-ded32d9b-259ece03_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:16.215575Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:16.215588Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2399
2025-10-09T21:52:16.215699Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2399 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:16.215734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2399 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:16.215739Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2399 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:16.215743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:16.216205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2399 sessionId:  describe result for acl check
2025-10-09T21:52:16.216239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:16.216246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:16.216248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:16.216254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343171801451182:5831] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:16.216725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343171801451182:5831] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:16.219234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343171801451182:5831] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:16.220613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:16.222553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:16.222672Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343171801451182:5831] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:16.222685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343171801451182:5831] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:16.222690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343171801451182:5831] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:16.222698Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2399 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:16.222736Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:16.222748Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:16.222763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:16.222823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:16.222866Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343171439598384:8824]
2025-10-09T21:52:16.223460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:16.223747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:16.223790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:16.224103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:16.224149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.224156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.224157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.224158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.224159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.224161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.224162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.224163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.224164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.224170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.224180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.224185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.224189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.224193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.224197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.224201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.224204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.224208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.224215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.224223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.224229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.224234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.224239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.224244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.224249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.224259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.224263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.224268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.224277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.224283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.224288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.224295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.224299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.224304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.224311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.224316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.224346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:16.230626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:16.231784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:16.231801Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:16.231803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:16.231807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:16.231814Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2398 [2:7559343171439598384:8824]
2025-10-09T21:52:16.231830Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:16.231846Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:16.231854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:16.231856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:16.231859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:16.231876Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:16.231887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:16.231899Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:16.231965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.231974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.231977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.231980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:16.232044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:16.232052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.232061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.232067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.232077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:16.232095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343171439598379:8824]
2025-10-09T21:52:16.232102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343171439598379:8824]
2025-10-09T21:52:16.232104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343171439598379:8824]
2025-10-09T21:52:16.232113Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:16.232120Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:16.232122Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:16.232124Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:16.232129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801451084:2432], now have 1 active actors on pipe
2025-10-09T21:52:16.232136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801451311:5831], now have 1 active actors on pipe
2025-10-09T21:52:16.232144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:16.232158Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:16.232166Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:16.232192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343171439598379:8824]
2025-10-09T21:52:16.232176Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2398
2025-10-09T21:52:16.232200Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:16.232201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:16.232218Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2398
2025-10-09T21:52:16.232237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.232246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.232273Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|90481bc8-4c9edaba-76e83432-e3d16d6a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:16.232282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:16.232289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:16.232291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:16.232301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:16.232312Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:16.232331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:16.232504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.232513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.232525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:16.232614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2399 partition: 0 MaxSeqNo: 25 sessionId: src_id|90481bc8-4c9edaba-76e83432-e3d16d6a_0
2025-10-09T21:52:16.232941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e47667fa-a81a3235-ded32d9b-259ece03_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|90481bc8-4c9edaba-76e83432-e3d16d6a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:16.235259Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90481bc8-4c9edaba-76e83432-e3d16d6a_0] Write session: try to update token
2025-10-09T21:52:16.235275Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90481bc8-4c9edaba-76e83432-e3d16d6a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:16.246805Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:16.246826Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:16.246829Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:16.246831Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:16.246841Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343171439598379:8824]
2025-10-09T21:52:16.246845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343171439598379:8824]
2025-10-09T21:52:16.246850Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343171439598379:8824]
2025-10-09T21:52:16.246892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:16.246900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.246899Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:16.246904Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:16.246905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:16.246907Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:16.247045Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:16.247265Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2399 sessionId: src_id|90481bc8-4c9edaba-76e83432-e3d16d6a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:16.249494Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:16.305674Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343171801451325:7496]
2025-10-09T21:52:16.305970Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 2398, pipe: [1:7559343171801451325:7496]
2025-10-09T21:52:16.397170Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:16.397649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.397663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.397702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801451325:7496], now have 1 active actors on pipe
2025-10-09T21:52:16.397865Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:16.397909Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:16.397922Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:16.397974Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2399 reason: error at write: Disconnected sessionId: src_id|90481bc8-4c9edaba-76e83432-e3d16d6a_0
2025-10-09T21:52:16.397980Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343171801451325:7496]
2025-10-09T21:52:16.398010Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:16.398169Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2399 sessionId: src_id|90481bc8-4c9edaba-76e83432-e3d16d6a_0 is DEAD
2025-10-09T21:52:16.398298Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90481bc8-4c9edaba-76e83432-e3d16d6a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:16.398308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90481bc8-4c9edaba-76e83432-e3d16d6a_0] Write session will restart in 0.009351s
2025-10-09T21:52:16.398327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90481bc8-4c9edaba-76e83432-e3d16d6a_0] Write session: Do CDS request
2025-10-09T21:52:16.398332Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90481bc8-4c9edaba-76e83432-e3d16d6a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:16.408463Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90481bc8-4c9edaba-76e83432-e3d16d6a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:16.408567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:16.408580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2400
2025-10-09T21:52:16.408691Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2400 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:16.408723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2400 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:16.408733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2400 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:16.408737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:16.409227Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2400 sessionId:  describe result for acl check
2025-10-09T21:52:16.409265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:16.409274Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:16.409286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:16.409292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343171801451466:5819] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:16.409810Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343171801451466:5819] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:16.412044Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343171801451466:5819] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:16.412335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:16.414269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:16.414446Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:16.414458Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:16.414470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:16.414526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:16.414567Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343171439598615:8822]
2025-10-09T21:52:16.414559Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343171801451466:5819] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:16.414571Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343171801451466:5819] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:16.414575Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343171801451466:5819] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:16.414583Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2400 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:16.415158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:16.415412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:16.415455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:16.415775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:16.415821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.415829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.415830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.415831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.415833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.415834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.415835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.415836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.415837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.415842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.415853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.415857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.415861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.415864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.415868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.415872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.415875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.415878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.415884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.415892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.415898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.415904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.415914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.415919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.415928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.415933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.415942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.415948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.415957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.415962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.415972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.415978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.415987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.415991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.415996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.416001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.416031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:16.422258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:16.423317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:16.423333Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:16.423335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:16.423338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:16.423343Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2399 [2:7559343171439598615:8822]
2025-10-09T21:52:16.423353Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:16.423367Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:16.423372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:16.423374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:16.423376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:16.423390Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:16.423397Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:16.423403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:16.423451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.423456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.423457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.423459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:16.423511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:16.423516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.423523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.423529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.423539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:16.423556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343171439598610:8822]
2025-10-09T21:52:16.423561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343171439598610:8822]
2025-10-09T21:52:16.423562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343171439598610:8822]
2025-10-09T21:52:16.423569Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:16.423577Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:16.423582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801451561:5819], now have 1 active actors on pipe
2025-10-09T21:52:16.423586Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:16.423587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801451333:2432], now have 1 active actors on pipe
2025-10-09T21:52:16.423588Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:16.423592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:16.423600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:16.423603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:16.423633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343171439598610:8822]
2025-10-09T21:52:16.423648Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:16.423667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:16.423650Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2399
2025-10-09T21:52:16.423660Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2399
2025-10-09T21:52:16.423720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.423728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.423753Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|41989ea8-4738a4f6-a3b4ebfa-4b7519c9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:16.423762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:16.423765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:16.423767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:16.423775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:16.423783Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:16.423798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:16.424035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.424049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.424072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:16.424191Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2400 partition: 0 MaxSeqNo: 25 sessionId: src_id|41989ea8-4738a4f6-a3b4ebfa-4b7519c9_0
2025-10-09T21:52:16.424534Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90481bc8-4c9edaba-76e83432-e3d16d6a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|41989ea8-4738a4f6-a3b4ebfa-4b7519c9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:16.426798Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|41989ea8-4738a4f6-a3b4ebfa-4b7519c9_0] Write session: try to update token
2025-10-09T21:52:16.426813Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|41989ea8-4738a4f6-a3b4ebfa-4b7519c9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:16.435374Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:16.435394Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:16.435402Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343171439598610:8822]
2025-10-09T21:52:16.435445Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:16.435440Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:16.435600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:16.435740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:16.435896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:16.436031Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:16.436199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:16.436236Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:16.438720Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2400 sessionId: src_id|41989ea8-4738a4f6-a3b4ebfa-4b7519c9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:16.440964Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:16.441455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.441474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.441506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:16.441589Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:16.448281Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:16.448300Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:16.448303Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:16.448305Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:16.448316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343171439598610:8822]
2025-10-09T21:52:16.448320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343171439598610:8822]
2025-10-09T21:52:16.448325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343171439598610:8822]
2025-10-09T21:52:16.448358Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:16.448365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:16.448372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.448375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:16.448377Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:16.448380Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:16.448516Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:16.498164Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343171801451572:7496]
2025-10-09T21:52:16.498413Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 2399, pipe: [1:7559343171801451572:7496]
2025-10-09T21:52:16.587317Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:16.594085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.594108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.594255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:16.594312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:16.594371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:16.594433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:16.594475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:16.594483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:16.594542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:16.594597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:16.594656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:16.594714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:16.594753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:16.594759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:16.594815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:16.594869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:16.594922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:16.594975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:16.595016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:16.595023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:16.595082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:16.595141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:16.595199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:16.595257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:16.595294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:16.595299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:16.595354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:16.595409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:16.595466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:16.595524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:16.595563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:16.595568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:16.595625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:16.595684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:16.595754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:16.595822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:16.595872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:16.595878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:16.595931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:16.595983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:16.596039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:16.596095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:16.596133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:16.596139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:16.596195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:16.596247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:16.596302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:16.596356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:16.596395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:16.596401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:16.596457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:16.596510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:16.596563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:16.596620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:16.596749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:16.596756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:16.596810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:16.596871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:16.596924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:16.596976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:16.597013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:16.597018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:16.597059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801451572:7496], now have 1 active actors on pipe
2025-10-09T21:52:16.597339Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:16.597349Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343171801451572:7496]
2025-10-09T21:52:16.597353Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:16.597380Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:16.597385Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2400 reason: error at write: Disconnected sessionId: src_id|41989ea8-4738a4f6-a3b4ebfa-4b7519c9_0
2025-10-09T21:52:16.597604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2400 sessionId: src_id|41989ea8-4738a4f6-a3b4ebfa-4b7519c9_0 is DEAD
2025-10-09T21:52:16.597713Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41989ea8-4738a4f6-a3b4ebfa-4b7519c9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:16.597722Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41989ea8-4738a4f6-a3b4ebfa-4b7519c9_0] Write session will restart in 0.007454s
2025-10-09T21:52:16.597741Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41989ea8-4738a4f6-a3b4ebfa-4b7519c9_0] Write session: Do CDS request
2025-10-09T21:52:16.597746Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41989ea8-4738a4f6-a3b4ebfa-4b7519c9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:16.597818Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:16.606418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|41989ea8-4738a4f6-a3b4ebfa-4b7519c9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:16.606500Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:16.606517Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2401
2025-10-09T21:52:16.606652Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2401 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:16.606687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2401 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:16.606696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2401 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:16.606700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:16.607246Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2401 sessionId:  describe result for acl check
2025-10-09T21:52:16.607282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:16.607290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:16.607292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:16.607297Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343171801451666:5785] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:16.607812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343171801451666:5785] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:16.609826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343171801451666:5785] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:16.612408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:16.612457Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343171801451666:5785] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:16.612471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343171801451666:5785] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:16.612475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343171801451666:5785] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:16.612482Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2401 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:16.614574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:16.614767Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:16.614779Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:16.614792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:16.614848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:16.614890Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343171439598885:8810]
2025-10-09T21:52:16.615378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:16.615654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:16.615687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:16.615959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:16.616008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.616013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.616015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.616016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.616017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.616018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.616019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.616020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.616022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.616026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.616037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.616042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.616046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.616050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.616053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.616057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.616061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.616068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.616074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.616082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.616088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.616093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.616097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.616107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.616112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.616117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.616121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.616126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.616130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.616135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.616143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.616149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.616158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.616162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.616171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.616176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.616203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:16.622320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:16.623377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:16.623395Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:16.623397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:16.623401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:16.623409Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2400 [2:7559343171439598885:8810]
2025-10-09T21:52:16.623421Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:16.623444Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:16.623451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:16.623455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:16.623458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:16.623479Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:16.623492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:16.623499Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:16.623556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.623562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.623564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.623567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:16.623750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:16.623760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.623772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.623782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.623773Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2400
2025-10-09T21:52:16.623774Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2400
2025-10-09T21:52:16.623805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:16.623829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801451599:2432], now have 1 active actors on pipe
2025-10-09T21:52:16.623836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801451824:5785], now have 1 active actors on pipe
2025-10-09T21:52:16.623842Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343171439598879:8810]
2025-10-09T21:52:16.623847Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343171439598879:8810]
2025-10-09T21:52:16.623848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343171439598879:8810]
2025-10-09T21:52:16.623858Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:16.623870Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:16.623875Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:16.623879Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:16.623880Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:16.623882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:16.623885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:16.623905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.623917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.623948Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343171439598879:8810]
2025-10-09T21:52:16.623960Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:16.623978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:16.623984Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|697dcc42-79e84785-66bd9399-e93b43e9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:16.623991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:16.623996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:16.623998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:16.624010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:16.624020Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:16.624059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:16.624228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.624239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.624254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:16.624364Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2401 partition: 0 MaxSeqNo: 25 sessionId: src_id|697dcc42-79e84785-66bd9399-e93b43e9_0
2025-10-09T21:52:16.624678Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|41989ea8-4738a4f6-a3b4ebfa-4b7519c9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|697dcc42-79e84785-66bd9399-e93b43e9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:16.627037Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|697dcc42-79e84785-66bd9399-e93b43e9_0] Write session: try to update token
2025-10-09T21:52:16.627057Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|697dcc42-79e84785-66bd9399-e93b43e9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:16.638156Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:16.638174Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:16.638177Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:16.638179Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:16.638188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343171439598879:8810]
2025-10-09T21:52:16.638193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343171439598879:8810]
2025-10-09T21:52:16.638198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343171439598879:8810]
2025-10-09T21:52:16.638241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:16.638255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.638258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:16.638280Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:16.638289Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:16.638310Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:16.638393Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:16.638845Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2401 sessionId: src_id|697dcc42-79e84785-66bd9399-e93b43e9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:16.640984Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:16.697348Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343171801451837:7496]
2025-10-09T21:52:16.697604Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 2400, pipe: [1:7559343171801451837:7496]
2025-10-09T21:52:16.784868Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:16.785405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.785418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.785458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801451837:7496], now have 1 active actors on pipe
2025-10-09T21:52:16.785625Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:16.785635Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343171801451837:7496]
2025-10-09T21:52:16.785664Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:16.785666Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:16.785674Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:52:16.785725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2401 reason: error at write: Disconnected sessionId: src_id|697dcc42-79e84785-66bd9399-e93b43e9_0
2025-10-09T21:52:16.785786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2401 sessionId: src_id|697dcc42-79e84785-66bd9399-e93b43e9_0 is DEAD
2025-10-09T21:52:16.785801Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 24 consumer shared/user session shared/user_1_24_13326786123235704269_v1 is DEAD
2025-10-09T21:52:16.785914Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559343090197053282:7487] disconnected.
2025-10-09T21:52:16.785925Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559343090197053282:7487] disconnected; active server actors: 1
2025-10-09T21:52:16.785929Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559343090197053282:7487] client user disconnected session shared/user_1_24_13326786123235704269_v1
2025-10-09T21:52:16.786078Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:52:16.786096Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|697dcc42-79e84785-66bd9399-e93b43e9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:16.786107Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|697dcc42-79e84785-66bd9399-e93b43e9_0] Write session will restart in 0.006689s
2025-10-09T21:52:16.786106Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:16.786140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|697dcc42-79e84785-66bd9399-e93b43e9_0] Write session: Do CDS request
2025-10-09T21:52:16.786146Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|697dcc42-79e84785-66bd9399-e93b43e9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:16.786203Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:52:16.786208Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:52:16.786215Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.008614s
2025-10-09T21:52:16.793494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|697dcc42-79e84785-66bd9399-e93b43e9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:16.793605Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:16.793618Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2402
2025-10-09T21:52:16.793739Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2402 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:16.793772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2402 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:16.793782Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2402 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:16.793785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:16.794267Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2402 sessionId:  describe result for acl check
2025-10-09T21:52:16.794306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:16.794312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:16.794314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:16.794320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343171801451893:5770] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:16.794873Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343171801451893:5770] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:16.795284Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:52:16.795404Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:52:16.795411Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 25
2025-10-09T21:52:16.795520Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 25 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:52:16.795556Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:52:16.795602Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 auth for : user
2025-10-09T21:52:16.795714Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 Handle describe topics response
2025-10-09T21:52:16.795747Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 auth is DEAD
2025-10-09T21:52:16.795750Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:52:16.795918Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:52:16.796004Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_25_16632666203476474409_v1
2025-10-09T21:52:16.796140Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559343171801451938:5764] connected; active server actors: 1
2025-10-09T21:52:16.796275Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559343171801451938:5764] session shared/user_1_25_16632666203476474409_v1
2025-10-09T21:52:16.796290Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:52:16.796298Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:52:16.796302Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_25_16632666203476474409_v1" (Sender=[1:7559343171801451923:5764], Pipe=[1:7559343171801451938:5764], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:52:16.796306Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:52:16.796316Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:52:16.796327Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_25_16632666203476474409_v1" (Sender=[1:7559343171801451923:5764], Pipe=[1:7559343171801451938:5764], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:52:16.796336Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_25_16632666203476474409_v1" sender [1:7559343171801451923:5764] lock partition 0 for ReadingSession "shared/user_1_25_16632666203476474409_v1" (Sender=[1:7559343171801451923:5764], Pipe=[1:7559343171801451938:5764], Partitions=[], ActiveFamilyCount=1) generation 1 step 25
2025-10-09T21:52:16.796346Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:52:16.796352Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000029s
2025-10-09T21:52:16.796522Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 25 Session: "shared/user_1_25_16632666203476474409_v1" ClientId: "user" PipeClient { RawX1: 7559343171801451938 RawX2: 4503603922343556 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:52:16.796538Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:52:16.797473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343171801451893:5770] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:16.799822Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343171801451893:5770] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:16.799836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343171801451893:5770] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:16.799839Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343171801451893:5770] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:16.799846Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2402 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:16.800891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:16.803094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:16.803273Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:16.803284Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:16.803296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:16.803352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:16.803394Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343171439599118:8800]
2025-10-09T21:52:16.803868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:16.804166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:16.804211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:16.804489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:16.804536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.804542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.804543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.804545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.804546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.804547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.804549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.804550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.804551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.804556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.804567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.804571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.804575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.804579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.804583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.804592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.804596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.804600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.804608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.804621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:16.804632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.804642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:16.804647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.804653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:16.804657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.804667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:16.804676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.804685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:16.804691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.804700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:16.804709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.804716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.804720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.804724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.804729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.804738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.804767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:16.810838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:16.812201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:16.812217Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:16.812219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:16.812224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:16.812230Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2401 [2:7559343171439599118:8800]
2025-10-09T21:52:16.812242Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:16.812258Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:16.812265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:16.812268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:16.812271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:16.812287Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:16.812297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:16.812304Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:16.812357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.812364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.812366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.812369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:16.812430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:16.812440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.812452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:16.812464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:16.812481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:16.812506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801452076:5770], now have 1 active actors on pipe
2025-10-09T21:52:16.812514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801451962:5758], now have 1 active actors on pipe
2025-10-09T21:52:16.812517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801451842:2432], now have 1 active actors on pipe
2025-10-09T21:52:16.812526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343171439599113:8800]
2025-10-09T21:52:16.812528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343171439599113:8800]
2025-10-09T21:52:16.812530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343171439599113:8800]
2025-10-09T21:52:16.812543Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:16.812560Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:16.812560Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:16.812564Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:16.812566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:16.812575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:16.812578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:16.812589Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2401
2025-10-09T21:52:16.812600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343171439599113:8800]
2025-10-09T21:52:16.812590Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 2401, pipe: [1:7559343171801451962:5758]
2025-10-09T21:52:16.812613Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:16.812628Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:16.812616Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2401
2025-10-09T21:52:16.812654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.812664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.812677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_25_16632666203476474409_v1 on pipe: [1:7559343171801451962:5758]
2025-10-09T21:52:16.812703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.812709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.812709Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_25_16632666203476474409_v1:1 with generation 2401
2025-10-09T21:52:16.812720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:16.812725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:16.812732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:16.812752Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_25_16632666203476474409_v1
2025-10-09T21:52:16.812755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:16.812790Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|30d5f41b-dc2eea8f-3e1ff34b-4900d64f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:16.812802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:52:16.812806Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:52:16.817400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:52:16.817431Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:52:16.817439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:16.817449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:16.817452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:16.817463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:16.817474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:16.817483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:52:16.817495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:52:16.817515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:16.817653Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046736812 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:52:16.817669Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:52:16.817690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.817704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.817724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:16.817702Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 sending to client partition status
2025-10-09T21:52:16.817814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2402 partition: 0 MaxSeqNo: 25 sessionId: src_id|30d5f41b-dc2eea8f-3e1ff34b-4900d64f_0
2025-10-09T21:52:16.818086Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|697dcc42-79e84785-66bd9399-e93b43e9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|30d5f41b-dc2eea8f-3e1ff34b-4900d64f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:16.820427Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|30d5f41b-dc2eea8f-3e1ff34b-4900d64f_0] Write session: try to update token
2025-10-09T21:52:16.820441Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|30d5f41b-dc2eea8f-3e1ff34b-4900d64f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:16.823685Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:16.823701Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:16.823709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343171439599113:8800]
2025-10-09T21:52:16.823746Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:16.823750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:16.823894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:16.824021Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:16.824140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:16.824264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:16.824609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:16.824643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:16.834150Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2402 sessionId: src_id|30d5f41b-dc2eea8f-3e1ff34b-4900d64f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:16.836423Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:16.836446Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:16.836449Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:16.836451Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:16.836462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343171439599113:8800]
2025-10-09T21:52:16.836467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343171439599113:8800]
2025-10-09T21:52:16.836469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343171439599113:8800]
2025-10-09T21:52:16.836506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:16.836513Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:16.836515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:16.836517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:16.836518Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:16.836525Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:16.836605Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:16.836686Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:16.981443Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:16.981687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:16.981700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:16.981888Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:16.981907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_25_16632666203476474409_v1
2025-10-09T21:52:16.981890Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:16.981901Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:16.981935Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343171801451962:5758]
2025-10-09T21:52:16.981924Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:16.981950Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_25_16632666203476474409_v1
2025-10-09T21:52:16.981961Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2402 reason: error at write: Disconnected sessionId: src_id|30d5f41b-dc2eea8f-3e1ff34b-4900d64f_0
2025-10-09T21:52:16.982156Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2402 sessionId: src_id|30d5f41b-dc2eea8f-3e1ff34b-4900d64f_0 is DEAD
2025-10-09T21:52:16.982325Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30d5f41b-dc2eea8f-3e1ff34b-4900d64f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:16.982338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30d5f41b-dc2eea8f-3e1ff34b-4900d64f_0] Write session will restart in 0.006052s
2025-10-09T21:52:16.982362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30d5f41b-dc2eea8f-3e1ff34b-4900d64f_0] Write session: Do CDS request
2025-10-09T21:52:16.982367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30d5f41b-dc2eea8f-3e1ff34b-4900d64f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:16.989496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|30d5f41b-dc2eea8f-3e1ff34b-4900d64f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:16.989581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:16.989595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2403
2025-10-09T21:52:16.989728Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2403 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:16.989760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2403 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:16.989771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2403 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:16.989775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:16.990440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2403 sessionId:  describe result for acl check
2025-10-09T21:52:16.990480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:16.990482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:16.990484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:16.990489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343171801452136:5745] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:16.991015Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343171801452136:5745] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:16.993565Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343171801452136:5745] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:16.996168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343171801452136:5745] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:16.996180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343171801452136:5745] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:16.996187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343171801452136:5745] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:16.996217Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2403 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:16.998164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:17.000096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:17.000273Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:17.000281Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:17.000290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:17.000337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:17.000369Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343171439599358:8796]
2025-10-09T21:52:17.000773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:17.000977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:17.001006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:17.001207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:17.001253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.001257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.001258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.001259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.001259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.001260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.001261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.001261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.001262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.001265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.001272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.001287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.001290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.001293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.001295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.001297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.001299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.001301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.001306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.001311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.001315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.001319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.001321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.001324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.001327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.001329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.001334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.001337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.001340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.001343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.001346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.001349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.001351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.001354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.001356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.001359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.001374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:17.007764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:17.008949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:17.008967Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:17.008970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:17.008974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:17.008983Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2402 [2:7559343171439599358:8796]
2025-10-09T21:52:17.008997Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:17.009015Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:17.009023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:17.009026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:17.009028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:17.009046Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:17.009056Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:17.009063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:17.009372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.009386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.009388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.009391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:17.009470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:17.009479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.009492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.009504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.009524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:17.009551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801452318:5745], now have 1 active actors on pipe
2025-10-09T21:52:17.009573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343171801452098:2432], now have 1 active actors on pipe
2025-10-09T21:52:17.009585Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343171439599353:8796]
2025-10-09T21:52:17.009592Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343171439599353:8796]
2025-10-09T21:52:17.009594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343171439599353:8796]
2025-10-09T21:52:17.009606Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:17.009622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:17.009629Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:17.009631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:17.009632Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:17.009634Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:17.009634Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:17.009619Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2402
2025-10-09T21:52:17.009659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343171439599353:8796]
2025-10-09T21:52:17.009665Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:17.009620Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2402
2025-10-09T21:52:17.009667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:17.009681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.009686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.009728Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d392de3c-855e3048-360ca0af-781af60a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:17.009738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:17.009741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:17.009743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:17.009751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:17.009759Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:17.009778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:17.009924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.009930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.009941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:17.010043Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2403 partition: 0 MaxSeqNo: 25 sessionId: src_id|d392de3c-855e3048-360ca0af-781af60a_0
2025-10-09T21:52:17.010409Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|30d5f41b-dc2eea8f-3e1ff34b-4900d64f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d392de3c-855e3048-360ca0af-781af60a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:17.012852Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d392de3c-855e3048-360ca0af-781af60a_0] Write session: try to update token
2025-10-09T21:52:17.012872Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d392de3c-855e3048-360ca0af-781af60a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:17.024126Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:17.024145Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:17.024147Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:17.024148Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:17.024156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343171439599353:8796]
2025-10-09T21:52:17.024160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343171439599353:8796]
2025-10-09T21:52:17.024164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343171439599353:8796]
2025-10-09T21:52:17.024199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:17.024205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.024207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:17.024215Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:17.024221Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:17.024224Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:17.024338Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:17.025694Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2403 sessionId: src_id|d392de3c-855e3048-360ca0af-781af60a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:17.028013Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:17.081891Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343176096419634:5758]
2025-10-09T21:52:17.082154Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 2402, pipe: [1:7559343176096419634:5758]
2025-10-09T21:52:17.175957Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:17.176203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.176216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.176255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343176096419634:5758], now have 1 active actors on pipe
2025-10-09T21:52:17.176427Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:17.176462Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:17.176464Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:17.176479Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:17.176511Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343176096419634:5758]
2025-10-09T21:52:17.176544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2403 reason: error at write: Disconnected sessionId: src_id|d392de3c-855e3048-360ca0af-781af60a_0
2025-10-09T21:52:17.176717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2403 sessionId: src_id|d392de3c-855e3048-360ca0af-781af60a_0 is DEAD
2025-10-09T21:52:17.176887Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d392de3c-855e3048-360ca0af-781af60a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:17.176899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d392de3c-855e3048-360ca0af-781af60a_0] Write session will restart in 0.006916s
2025-10-09T21:52:17.176926Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d392de3c-855e3048-360ca0af-781af60a_0] Write session: Do CDS request
2025-10-09T21:52:17.176932Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d392de3c-855e3048-360ca0af-781af60a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:17.184429Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d392de3c-855e3048-360ca0af-781af60a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:17.184532Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:17.184545Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2404
2025-10-09T21:52:17.184662Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2404 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:17.184695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2404 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:17.184707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2404 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:17.184711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:17.185308Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2404 sessionId:  describe result for acl check
2025-10-09T21:52:17.185340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:17.185345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:17.185346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:17.185351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343176096419691:5740] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:17.185772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343176096419691:5740] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:17.187894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343176096419691:5740] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:17.190646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343176096419691:5740] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:17.190661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343176096419691:5740] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:17.190667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343176096419691:5740] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:17.190675Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2404 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:17.191953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:17.194420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:17.194599Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:17.194606Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:17.194615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:17.194666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:17.194698Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343175734566887:8782]
2025-10-09T21:52:17.195099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:17.195543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:17.195583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:17.195806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:17.195851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.195856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.195857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.195858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.195860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.195861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.195862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.195864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.195865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.195870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.195883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.195887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.195890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.195894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.195899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.195904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.195908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.195913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.195922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.195932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.195943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.195949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.195958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.195964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.195971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.195975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.195980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.195986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.195991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.196000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.196006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.196015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.196020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.196025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.196030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.196035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.196065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:17.202345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:17.203412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:17.203428Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:17.203431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:17.203434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:17.203439Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2403 [2:7559343175734566887:8782]
2025-10-09T21:52:17.203454Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:17.203475Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:17.203481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:17.203484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:17.203485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:17.203499Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:17.203503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:17.203510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:17.203558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.203563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.203565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.203567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:17.203629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:17.203638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.203651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.203660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.203672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:17.203690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343176096419866:5740], now have 1 active actors on pipe
2025-10-09T21:52:17.203696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343176096419636:2432], now have 1 active actors on pipe
2025-10-09T21:52:17.203704Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343175734566882:8782]
2025-10-09T21:52:17.203706Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343175734566882:8782]
2025-10-09T21:52:17.203709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343175734566882:8782]
2025-10-09T21:52:17.203722Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:17.203740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:17.203746Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:17.203750Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:17.203752Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:17.203755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:17.203757Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:17.203785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343175734566882:8782]
2025-10-09T21:52:17.203796Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:17.203818Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:17.203801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2403
2025-10-09T21:52:17.203804Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2403
2025-10-09T21:52:17.203886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.203898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.203928Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|547ea4fe-cae86c68-1e917f39-e0cde766_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:17.203937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:17.203941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:17.203943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:17.203953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:17.203974Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:17.203993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:17.204157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.204170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.204181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:17.204290Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2404 partition: 0 MaxSeqNo: 25 sessionId: src_id|547ea4fe-cae86c68-1e917f39-e0cde766_0
2025-10-09T21:52:17.204681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d392de3c-855e3048-360ca0af-781af60a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|547ea4fe-cae86c68-1e917f39-e0cde766_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:17.207187Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|547ea4fe-cae86c68-1e917f39-e0cde766_0] Write session: try to update token
2025-10-09T21:52:17.207204Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|547ea4fe-cae86c68-1e917f39-e0cde766_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:17.215142Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:17.215158Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:17.215169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343175734566882:8782]
2025-10-09T21:52:17.215211Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:17.215218Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:17.215386Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:17.215526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:17.215663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:17.215797Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:17.215976Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:17.216018Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:17.220175Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2404 sessionId: src_id|547ea4fe-cae86c68-1e917f39-e0cde766_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:17.222501Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:17.222687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.222702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.222746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:17.222820Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:17.227924Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:17.227936Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:17.227938Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:17.227939Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:17.227948Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343175734566882:8782]
2025-10-09T21:52:17.227951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343175734566882:8782]
2025-10-09T21:52:17.227954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343175734566882:8782]
2025-10-09T21:52:17.227986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:17.227986Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:17.227990Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:17.227991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.227993Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:17.227993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:17.228114Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:17.276923Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343176096419883:5758]
2025-10-09T21:52:17.277145Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 2403, pipe: [1:7559343176096419883:5758]
2025-10-09T21:52:17.370468Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:17.377040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.377062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.377203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.377255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.377320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.377377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.377414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.377422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.377478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.377530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.377583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.377639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.377684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.377692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.377749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.377799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.377903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.377954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.377990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.377996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.378063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.378114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.378165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.378219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.378257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.378262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.378317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.378372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.378425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.378488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.378530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.378539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.378602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.378653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.378705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.378760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.378801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.378807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.378857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.378902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.378953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.379007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.379043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.379048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.379106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.379157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.379204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.379254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.379289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.379295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.379343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.379391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.379441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.379494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.379530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.379536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.379586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.379634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.379685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.379734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.379770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.379775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.379815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343176096419883:5758], now have 1 active actors on pipe
2025-10-09T21:52:17.380071Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:17.380072Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343176096419883:5758]
2025-10-09T21:52:17.380112Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:17.380114Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:17.380143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2404 reason: error at write: Disconnected sessionId: src_id|547ea4fe-cae86c68-1e917f39-e0cde766_0
2025-10-09T21:52:17.380329Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2404 sessionId: src_id|547ea4fe-cae86c68-1e917f39-e0cde766_0 is DEAD
2025-10-09T21:52:17.380460Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|547ea4fe-cae86c68-1e917f39-e0cde766_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:17.380472Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|547ea4fe-cae86c68-1e917f39-e0cde766_0] Write session will restart in 0.007942s
2025-10-09T21:52:17.380508Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|547ea4fe-cae86c68-1e917f39-e0cde766_0] Write session: Do CDS request
2025-10-09T21:52:17.380515Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|547ea4fe-cae86c68-1e917f39-e0cde766_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:17.380536Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:17.389420Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|547ea4fe-cae86c68-1e917f39-e0cde766_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:17.389511Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:17.389524Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2405
2025-10-09T21:52:17.389630Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2405 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:17.389663Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2405 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:17.389673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2405 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:17.389678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:17.390080Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2405 sessionId:  describe result for acl check
2025-10-09T21:52:17.390107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:17.390112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:17.390113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:17.390117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343176096419984:5706] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:17.390546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343176096419984:5706] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:17.392704Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343176096419984:5706] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:17.394628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:17.396708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:17.396770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343176096419984:5706] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:17.396784Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343176096419984:5706] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:17.396787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343176096419984:5706] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:17.396794Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2405 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:17.396892Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:17.396901Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:17.396913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:17.396973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:17.397013Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343175734567122:8792]
2025-10-09T21:52:17.397608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:17.397884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:17.397919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:17.398185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:17.398251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.398255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.398256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.398257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.398257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.398258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.398259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.398259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.398260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.398265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.398275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.398279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.398282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.398286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.398291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.398294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.398298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.398301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.398308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.398316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.398322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.398325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.398328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.398331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.398333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.398336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.398338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.398341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.398344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.398347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.398350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.398353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.398355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.398358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.398361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.398363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.398379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:17.404578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:17.405759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:17.405773Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:17.405776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:17.405779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:17.405787Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2404 [2:7559343175734567122:8792]
2025-10-09T21:52:17.405800Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:17.405815Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:17.405820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:17.405822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:17.405823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:17.405835Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:17.405840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:17.405846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:17.405886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.405891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.405893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.405895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:17.405961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:17.405968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.405980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.405992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.406009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:17.406053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343176096419891:2432], now have 1 active actors on pipe
2025-10-09T21:52:17.406061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343176096420124:5706], now have 1 active actors on pipe
2025-10-09T21:52:17.406069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343175734567117:8792]
2025-10-09T21:52:17.406072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343175734567117:8792]
2025-10-09T21:52:17.406074Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343175734567117:8792]
2025-10-09T21:52:17.406056Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2404
2025-10-09T21:52:17.406082Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:17.406076Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2404
2025-10-09T21:52:17.406098Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:17.406097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:17.406100Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:17.406102Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:17.406112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:17.406115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:17.406135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.406145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.406178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343175734567117:8792]
2025-10-09T21:52:17.406187Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:17.406196Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:17.406201Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|13543f00-b4bc4c3c-55de2471-8d9a6e04_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:17.406208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:17.406212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:17.406214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:17.406224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:17.406239Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:17.406277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:17.406437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.406447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.406466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:17.406561Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2405 partition: 0 MaxSeqNo: 25 sessionId: src_id|13543f00-b4bc4c3c-55de2471-8d9a6e04_0
2025-10-09T21:52:17.406893Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|547ea4fe-cae86c68-1e917f39-e0cde766_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|13543f00-b4bc4c3c-55de2471-8d9a6e04_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:17.409088Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|13543f00-b4bc4c3c-55de2471-8d9a6e04_0] Write session: try to update token
2025-10-09T21:52:17.409103Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|13543f00-b4bc4c3c-55de2471-8d9a6e04_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:17.419962Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:17.419977Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:17.419978Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:17.419979Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:17.419987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343175734567117:8792]
2025-10-09T21:52:17.419991Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343175734567117:8792]
2025-10-09T21:52:17.419994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343175734567117:8792]
2025-10-09T21:52:17.420028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:17.420034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.420036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:17.420036Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:17.420041Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:17.420043Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:17.420155Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:17.421043Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2405 sessionId: src_id|13543f00-b4bc4c3c-55de2471-8d9a6e04_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:17.423356Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:17.480183Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343176096420135:5758]
2025-10-09T21:52:17.480479Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 2404, pipe: [1:7559343176096420135:5758]
2025-10-09T21:52:17.561820Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:17.562297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.562310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.562345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343176096420135:5758], now have 1 active actors on pipe
2025-10-09T21:52:17.562496Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:17.562550Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:17.562562Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:17.562608Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2405 reason: error at write: Disconnected sessionId: src_id|13543f00-b4bc4c3c-55de2471-8d9a6e04_0
2025-10-09T21:52:17.562786Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2405 sessionId: src_id|13543f00-b4bc4c3c-55de2471-8d9a6e04_0 is DEAD
2025-10-09T21:52:17.562936Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13543f00-b4bc4c3c-55de2471-8d9a6e04_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:17.562945Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13543f00-b4bc4c3c-55de2471-8d9a6e04_0] Write session will restart in 0.007120s
2025-10-09T21:52:17.562963Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13543f00-b4bc4c3c-55de2471-8d9a6e04_0] Write session: Do CDS request
2025-10-09T21:52:17.562967Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13543f00-b4bc4c3c-55de2471-8d9a6e04_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:17.562978Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:17.563008Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343176096420135:5758]
2025-10-09T21:52:17.571453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|13543f00-b4bc4c3c-55de2471-8d9a6e04_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:17.571549Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:17.571561Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2406
2025-10-09T21:52:17.571677Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2406 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:17.571709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2406 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:17.571715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2406 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:17.571718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:17.572145Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2406 sessionId:  describe result for acl check
2025-10-09T21:52:17.572172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:17.572173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:17.572174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:17.572178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343176096420245:5657] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:17.572556Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343176096420245:5657] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:17.574184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343176096420245:5657] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:17.576386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343176096420245:5657] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:17.576395Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343176096420245:5657] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:17.576398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343176096420245:5657] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:17.576404Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2406 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:17.577155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:17.579080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:17.579219Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:17.579226Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:17.579234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:17.579268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:17.579297Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343175734567397:8786]
2025-10-09T21:52:17.579709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:17.579976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:17.580010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:17.580262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:17.580319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.580324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.580324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.580325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.580325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.580326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.580327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.580327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.580328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.580331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.580337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.580339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.580341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.580343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.580345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.580347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.580349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.580351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.580355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.580360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.580365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.580368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.580370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.580373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.580376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.580379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.580381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.580384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.580386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.580389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.580392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.580395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.580399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.580401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.580404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.580407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.580422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:17.586274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:17.587286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:17.587300Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:17.587303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:17.587306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:17.587313Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2405 [2:7559343175734567397:8786]
2025-10-09T21:52:17.587325Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:17.587337Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:17.587339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:17.587341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:17.587342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:17.587355Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:17.587359Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:17.587364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:17.587403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.587409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.587411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.587413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:17.587530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:17.587538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.587549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.587558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.587572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:17.587591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343176096420391:5657], now have 1 active actors on pipe
2025-10-09T21:52:17.587597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343176096420151:2432], now have 1 active actors on pipe
2025-10-09T21:52:17.587602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343175734567392:8786]
2025-10-09T21:52:17.587604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343175734567392:8786]
2025-10-09T21:52:17.587605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343175734567392:8786]
2025-10-09T21:52:17.587593Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2405
2025-10-09T21:52:17.587612Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:17.587598Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2405
2025-10-09T21:52:17.587623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:17.587631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:17.587630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:17.587633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:17.587633Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:17.587635Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:17.587673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.587676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.587687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343175734567392:8786]
2025-10-09T21:52:17.587695Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:17.587711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:17.587719Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|82ec6945-d515c7f8-41b21634-90d43cac_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:17.587726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:17.587730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:17.587732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:17.587742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:17.587753Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:17.587782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:17.587899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.587909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.587928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:17.588048Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2406 partition: 0 MaxSeqNo: 25 sessionId: src_id|82ec6945-d515c7f8-41b21634-90d43cac_0
2025-10-09T21:52:17.588382Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|13543f00-b4bc4c3c-55de2471-8d9a6e04_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|82ec6945-d515c7f8-41b21634-90d43cac_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:17.590793Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|82ec6945-d515c7f8-41b21634-90d43cac_0] Write session: try to update token
2025-10-09T21:52:17.590812Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|82ec6945-d515c7f8-41b21634-90d43cac_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:17.601333Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:17.601345Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:17.601347Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:17.601349Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:17.601356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343175734567392:8786]
2025-10-09T21:52:17.601360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343175734567392:8786]
2025-10-09T21:52:17.601362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343175734567392:8786]
2025-10-09T21:52:17.601397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:17.601403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.601405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:17.601407Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:17.601413Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:17.601417Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:17.601520Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:17.602842Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2406 sessionId: src_id|82ec6945-d515c7f8-41b21634-90d43cac_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:17.605053Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:17.662979Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343176096420405:5758]
2025-10-09T21:52:17.663255Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 2405, pipe: [1:7559343176096420405:5758]
2025-10-09T21:52:17.741212Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:17.741686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.741700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.741740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343176096420405:5758], now have 1 active actors on pipe
2025-10-09T21:52:17.741914Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:17.741975Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:17.741992Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343176096420405:5758]
2025-10-09T21:52:17.742016Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:17.742074Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2406 reason: status is not ok: tablet will be restarted right now sessionId: src_id|82ec6945-d515c7f8-41b21634-90d43cac_0
2025-10-09T21:52:17.742269Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2406 sessionId: src_id|82ec6945-d515c7f8-41b21634-90d43cac_0 is DEAD
2025-10-09T21:52:17.742429Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82ec6945-d515c7f8-41b21634-90d43cac_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:17.742441Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82ec6945-d515c7f8-41b21634-90d43cac_0] Write session will restart in 0.009289s
2025-10-09T21:52:17.742463Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82ec6945-d515c7f8-41b21634-90d43cac_0] Write session: Do CDS request
2025-10-09T21:52:17.742468Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82ec6945-d515c7f8-41b21634-90d43cac_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:17.752497Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|82ec6945-d515c7f8-41b21634-90d43cac_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:17.752574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:17.752586Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2407
2025-10-09T21:52:17.752720Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2407 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:17.752756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2407 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:17.752764Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2407 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:17.752767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:17.753215Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2407 sessionId:  describe result for acl check
2025-10-09T21:52:17.753251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:17.753258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:17.753260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:17.753266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343176096420520:5638] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:17.753824Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343176096420520:5638] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:17.756151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343176096420520:5638] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:17.757239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:17.759480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:17.759650Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:17.759659Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:17.759668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:17.759717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:17.759746Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343175734567633:8759]
2025-10-09T21:52:17.759804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343176096420520:5638] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:17.759815Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343176096420520:5638] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:17.759819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343176096420520:5638] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:17.759826Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2407 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:17.760167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:17.760457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:17.760489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:17.760707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:17.760736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.760740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.760741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.760742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.760743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.760743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.760744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.760745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.760745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.760748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.760755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.760757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.760760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.760762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.760764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.760766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.760768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.760770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.760774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.760779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.760783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.760787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.760789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.760792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.760795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.760798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.760801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.760804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.760806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.760809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.760812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.760815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.760817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.760820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.760823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.760826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.760842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:17.767081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:17.768312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:17.768323Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:17.768324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:17.768327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:17.768332Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2406 [2:7559343175734567633:8759]
2025-10-09T21:52:17.768341Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:17.768355Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:17.768357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:17.768359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:17.768360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:17.768372Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:17.768376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:17.768382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:17.768424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.768431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.768432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.768434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:17.768481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:17.768487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.768493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.768510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.768520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:17.768535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343175734567628:8759]
2025-10-09T21:52:17.768540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343175734567628:8759]
2025-10-09T21:52:17.768542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343175734567628:8759]
2025-10-09T21:52:17.768550Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:17.768559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343176096420408:2432], now have 1 active actors on pipe
2025-10-09T21:52:17.768559Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:17.768562Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:17.768564Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:17.768568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:17.768576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:17.768577Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:17.768581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343176096420643:5638], now have 1 active actors on pipe
2025-10-09T21:52:17.768598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343175734567628:8759]
2025-10-09T21:52:17.768606Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:17.768614Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:17.768649Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2406
2025-10-09T21:52:17.768680Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2406
2025-10-09T21:52:17.768728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.768735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.768755Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|10f83be-5570ff89-3a6183b6-397de007_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:17.768761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:17.768764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:17.768765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:17.768772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:17.768780Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:17.768790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:17.768990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.769001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.769019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:17.769104Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2407 partition: 0 MaxSeqNo: 25 sessionId: src_id|10f83be-5570ff89-3a6183b6-397de007_0
2025-10-09T21:52:17.769415Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82ec6945-d515c7f8-41b21634-90d43cac_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|10f83be-5570ff89-3a6183b6-397de007_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:17.771927Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10f83be-5570ff89-3a6183b6-397de007_0] Write session: try to update token
2025-10-09T21:52:17.771947Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10f83be-5570ff89-3a6183b6-397de007_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:17.780056Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:17.780072Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:17.780083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343175734567628:8759]
2025-10-09T21:52:17.780126Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:17.780130Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:17.780321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:17.780467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:17.780606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:17.780750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:17.780940Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:17.780977Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:17.784478Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2407 sessionId: src_id|10f83be-5570ff89-3a6183b6-397de007_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:17.786929Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:17.787120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.787136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.787171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:17.787253Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:17.793766Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:17.793780Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:17.793782Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:17.793783Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:17.793791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343175734567628:8759]
2025-10-09T21:52:17.793794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343175734567628:8759]
2025-10-09T21:52:17.793797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343175734567628:8759]
2025-10-09T21:52:17.793828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:17.793835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.793838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:17.793843Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:17.793848Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:17.793851Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:17.793963Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:17.842160Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343176096420654:5758]
2025-10-09T21:52:17.842417Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 2406, pipe: [1:7559343176096420654:5758]
2025-10-09T21:52:17.933422Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:17.939724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.939744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.939881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.939939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.940000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.940054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.940093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.940097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.940154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.940216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.940271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.940335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.940374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.940380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.940431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.940486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.940539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.940590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.940625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.940631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.940686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.940750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.940804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.940855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.940894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.940899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.940957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.941015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.941067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.941119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.941157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.941164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.941217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.941265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.941325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.941376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.941415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.941417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.941468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.941517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.941567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.941619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.941657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.941662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.941717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.941771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.941822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.941873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.941910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.941916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.941970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.942040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.942092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.942145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.942185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.942191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.942245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:17.942293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:17.942343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:17.942397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:17.942433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:17.942442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:17.942480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343176096420654:5758], now have 1 active actors on pipe
2025-10-09T21:52:17.942672Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:17.942698Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343176096420654:5758]
2025-10-09T21:52:17.942725Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:17.942727Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:17.942753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2407 reason: error at write: Disconnected sessionId: src_id|10f83be-5570ff89-3a6183b6-397de007_0
2025-10-09T21:52:17.942926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2407 sessionId: src_id|10f83be-5570ff89-3a6183b6-397de007_0 is DEAD
2025-10-09T21:52:17.943076Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10f83be-5570ff89-3a6183b6-397de007_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:17.943086Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10f83be-5570ff89-3a6183b6-397de007_0] Write session will restart in 0.008465s
2025-10-09T21:52:17.943104Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10f83be-5570ff89-3a6183b6-397de007_0] Write session: Do CDS request
2025-10-09T21:52:17.943119Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10f83be-5570ff89-3a6183b6-397de007_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:17.943183Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:17.952441Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|10f83be-5570ff89-3a6183b6-397de007_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:17.952533Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:17.952546Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2408
2025-10-09T21:52:17.952659Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2408 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:17.952690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2408 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:17.952699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2408 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:17.952703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:17.953146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2408 sessionId:  describe result for acl check
2025-10-09T21:52:17.953193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:17.953201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:17.953202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:17.953209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343176096420762:5631] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:17.953661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343176096420762:5631] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:17.956706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343176096420762:5631] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:17.957729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:17.959796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:17.959981Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:17.959994Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:17.960007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:17.960060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:17.960064Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343176096420762:5631] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:17.960075Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343176096420762:5631] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:17.960079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343176096420762:5631] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:17.960087Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2408 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:17.960099Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343175734567875:8765]
2025-10-09T21:52:17.960660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:17.960911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:17.960952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:17.961241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:17.961299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.961300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.961301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.961303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.961304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.961305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.961306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.961307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.961309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.961314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.961323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.961327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.961331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.961335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.961339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.961343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.961347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.961350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.961357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.961364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:17.961370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.961375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:17.961380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.961385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:17.961390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.961395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:17.961400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.961404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:17.961408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.961413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:17.961418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.961423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.961426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.961430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.961435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.961440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.961466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:17.967329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:17.968569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:17.968584Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:17.968586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:17.968589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:17.968595Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2407 [2:7559343175734567875:8765]
2025-10-09T21:52:17.968607Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:17.968623Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:17.968626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:17.968629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:17.968631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:17.968647Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:17.968652Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:17.968658Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:17.968715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.968724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.968726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.968729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:17.968791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:17.968799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.968810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:17.968823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:17.968839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:17.968864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343176096420895:5631], now have 1 active actors on pipe
2025-10-09T21:52:17.968874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343176096420659:2432], now have 1 active actors on pipe
2025-10-09T21:52:17.968881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343175734567870:8765]
2025-10-09T21:52:17.968883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343175734567870:8765]
2025-10-09T21:52:17.968885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343175734567870:8765]
2025-10-09T21:52:17.968899Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:17.968914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:17.968923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:17.968927Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:17.968931Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:17.968935Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:17.968938Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:17.968934Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2407
2025-10-09T21:52:17.968962Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343175734567870:8765]
2025-10-09T21:52:17.968971Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:17.968973Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:17.968987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.968981Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2407
2025-10-09T21:52:17.968991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.969039Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8b296fe8-ffedf82a-b7359aa2-f7ff2012_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:17.969059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:17.969062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:17.969065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:17.969075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:17.969086Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:17.969103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:17.969250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:17.969260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:17.969288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:17.969382Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2408 partition: 0 MaxSeqNo: 25 sessionId: src_id|8b296fe8-ffedf82a-b7359aa2-f7ff2012_0
2025-10-09T21:52:17.969673Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|10f83be-5570ff89-3a6183b6-397de007_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8b296fe8-ffedf82a-b7359aa2-f7ff2012_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:17.972037Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b296fe8-ffedf82a-b7359aa2-f7ff2012_0] Write session: try to update token
2025-10-09T21:52:17.972052Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b296fe8-ffedf82a-b7359aa2-f7ff2012_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:17.982172Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:17.982186Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:17.982188Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:17.982191Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:17.982200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343175734567870:8765]
2025-10-09T21:52:17.982204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343175734567870:8765]
2025-10-09T21:52:17.982208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343175734567870:8765]
2025-10-09T21:52:17.982248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:17.982254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:17.982257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:17.982265Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:17.982272Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:17.982275Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:17.982392Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:17.984083Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2408 sessionId: src_id|8b296fe8-ffedf82a-b7359aa2-f7ff2012_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:17.986299Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:18.042870Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343180391388203:5758]
2025-10-09T21:52:18.043109Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 2407, pipe: [1:7559343180391388203:5758]
2025-10-09T21:52:18.126334Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:18.126574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.126586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.126625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343180391388203:5758], now have 1 active actors on pipe
2025-10-09T21:52:18.126856Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343180391388203:5758]
2025-10-09T21:52:18.126862Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:18.126866Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:18.126885Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:18.126887Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:18.126941Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2408 reason: error at write: Disconnected sessionId: src_id|8b296fe8-ffedf82a-b7359aa2-f7ff2012_0
2025-10-09T21:52:18.127103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2408 sessionId: src_id|8b296fe8-ffedf82a-b7359aa2-f7ff2012_0 is DEAD
2025-10-09T21:52:18.127263Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b296fe8-ffedf82a-b7359aa2-f7ff2012_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:18.127274Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b296fe8-ffedf82a-b7359aa2-f7ff2012_0] Write session will restart in 0.007571s
2025-10-09T21:52:18.127294Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b296fe8-ffedf82a-b7359aa2-f7ff2012_0] Write session: Do CDS request
2025-10-09T21:52:18.127299Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b296fe8-ffedf82a-b7359aa2-f7ff2012_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:18.136490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8b296fe8-ffedf82a-b7359aa2-f7ff2012_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:18.136601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:18.136610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2409
2025-10-09T21:52:18.136741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2409 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:18.136779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2409 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:18.136787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2409 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:18.136791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:18.137220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2409 sessionId:  describe result for acl check
2025-10-09T21:52:18.137250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:18.137259Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:18.137261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:18.137266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343180391388297:5599] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:18.137812Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343180391388297:5599] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:18.140121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343180391388297:5599] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:18.142097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:18.144464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343180391388297:5599] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:18.144479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343180391388297:5599] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:18.144483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343180391388297:5599] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:18.144492Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2409 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:18.144524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:18.144734Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:18.144747Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:18.144760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:18.144826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:18.144876Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343180029535410:8763]
2025-10-09T21:52:18.145484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:18.146161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:18.146223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:18.146580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:18.146654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.146660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.146661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.146663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.146664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.146665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.146666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.146667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.146669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.146676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.146687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.146691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.146695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.146699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.146703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.146707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.146711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.146715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.146723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.146732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.146738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.146743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.146750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.146756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.146761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.146771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.146777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.146782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.146786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.146792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.146796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.146802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.146807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.146812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.146819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.146825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.146857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:18.153308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:18.154388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:18.154402Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:18.154405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:18.154409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:18.154416Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2408 [2:7559343180029535410:8763]
2025-10-09T21:52:18.154429Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:18.154447Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:18.154454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:18.154458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:18.154460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:18.154476Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:18.154487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:18.154494Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:18.154548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.154554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.154557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.154559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:18.154784Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2408
2025-10-09T21:52:18.154787Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2408
2025-10-09T21:52:18.154942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:18.154952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.154965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.154978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.154998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:18.155024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343180391388442:5599], now have 1 active actors on pipe
2025-10-09T21:52:18.155032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343180391388205:2432], now have 1 active actors on pipe
2025-10-09T21:52:18.155051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.155060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.155078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343180029535405:8763]
2025-10-09T21:52:18.155085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343180029535405:8763]
2025-10-09T21:52:18.155087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343180029535405:8763]
2025-10-09T21:52:18.155100Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:18.155118Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:18.155123Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:18.155126Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:18.155128Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:18.155132Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:18.155135Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:18.155154Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|60a0fa36-54cfe257-de7cd8b1-71eaf447_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:18.155166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:18.155174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:18.155177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:18.155187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:18.155201Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:18.155236Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343180029535405:8763]
2025-10-09T21:52:18.155249Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:18.155260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:18.155265Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:18.155400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.155408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.155421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:18.155582Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2409 partition: 0 MaxSeqNo: 25 sessionId: src_id|60a0fa36-54cfe257-de7cd8b1-71eaf447_0
2025-10-09T21:52:18.155941Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8b296fe8-ffedf82a-b7359aa2-f7ff2012_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|60a0fa36-54cfe257-de7cd8b1-71eaf447_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:18.158378Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|60a0fa36-54cfe257-de7cd8b1-71eaf447_0] Write session: try to update token
2025-10-09T21:52:18.158393Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|60a0fa36-54cfe257-de7cd8b1-71eaf447_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:18.166778Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:18.166796Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:18.166805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343180029535405:8763]
2025-10-09T21:52:18.166851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:18.166884Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:18.167035Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:18.167179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:18.167322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:18.167456Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:18.167638Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:18.167681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:18.170834Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2409 sessionId: src_id|60a0fa36-54cfe257-de7cd8b1-71eaf447_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:18.173188Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:18.173460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.173488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.173524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:18.173614Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:18.179531Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:18.179545Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:18.179548Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:18.179551Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:18.179560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343180029535405:8763]
2025-10-09T21:52:18.179563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343180029535405:8763]
2025-10-09T21:52:18.179567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343180029535405:8763]
2025-10-09T21:52:18.179605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:18.179612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.179612Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:18.179619Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:18.179621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:18.179622Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:18.179746Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:18.227153Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343180391388454:5758]
2025-10-09T21:52:18.227429Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 2408, pipe: [1:7559343180391388454:5758]
2025-10-09T21:52:18.329245Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:18.336381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.336402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.336540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.336600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.336658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.336717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.336763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.336771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.336827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.336881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.336934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.337014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.337053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.337061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.337116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.337171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.337230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.337299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.337340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.337349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.337415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.337472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.337525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.337578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.337617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.337622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.337679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.337733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.337785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.337839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.337877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.337883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.337935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.337986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.338053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.338105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.338144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.338149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.338203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.338262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.338320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.338379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.338422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.338430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.338485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.338534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.338588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.338648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.338688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.338695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.338748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.338796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.338849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.338899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.338938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.338945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.339007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.339063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.339119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.339179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.339219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.339224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.339263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343180391388454:5758], now have 1 active actors on pipe
2025-10-09T21:52:18.339521Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:18.339521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343180391388454:5758]
2025-10-09T21:52:18.339531Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:18.339555Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:18.339556Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2409 reason: error at write: Disconnected sessionId: src_id|60a0fa36-54cfe257-de7cd8b1-71eaf447_0
2025-10-09T21:52:18.339731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2409 sessionId: src_id|60a0fa36-54cfe257-de7cd8b1-71eaf447_0 is DEAD
2025-10-09T21:52:18.339861Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60a0fa36-54cfe257-de7cd8b1-71eaf447_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:18.339870Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60a0fa36-54cfe257-de7cd8b1-71eaf447_0] Write session will restart in 0.007274s
2025-10-09T21:52:18.339888Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60a0fa36-54cfe257-de7cd8b1-71eaf447_0] Write session: Do CDS request
2025-10-09T21:52:18.339893Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60a0fa36-54cfe257-de7cd8b1-71eaf447_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:18.339989Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:18.348410Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|60a0fa36-54cfe257-de7cd8b1-71eaf447_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:18.348500Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:18.348511Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2410
2025-10-09T21:52:18.348674Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2410 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:18.348705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2410 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:18.348715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2410 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:18.348718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:18.349198Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2410 sessionId:  describe result for acl check
2025-10-09T21:52:18.349228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:18.349234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:18.349235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:18.349241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343180391388537:5606] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:18.349770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343180391388537:5606] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:18.352766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343180391388537:5606] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:18.354791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:18.355176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343180391388537:5606] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:18.355186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343180391388537:5606] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:18.355189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343180391388537:5606] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:18.355195Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2410 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:18.356827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:18.356999Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:18.357009Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:18.357022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:18.357088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:18.357126Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343180029535652:8762]
2025-10-09T21:52:18.357691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:18.357985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:18.358052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:18.358375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:18.358433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.358439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.358440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.358441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.358442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.358444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.358445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.358446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.358447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.358452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.358462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.358466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.358470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.358473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.358477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.358480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.358484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.358487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.358493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.358501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.358507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.358516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.358524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.358530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.358538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.358542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.358547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.358556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.358562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.358568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.358577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.358582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.358590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.358595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.358599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.358603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.358631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:18.364596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:18.365800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:18.365812Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:18.365814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:18.365818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:18.365825Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2409 [2:7559343180029535652:8762]
2025-10-09T21:52:18.365838Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:18.365857Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:18.365864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:18.365866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:18.365868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:18.365880Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:18.365889Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:18.365894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:18.365939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.365944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.365945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.365947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:18.366005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:18.366010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.366017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.366040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.366054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:18.366069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343180391388703:5606], now have 1 active actors on pipe
2025-10-09T21:52:18.366071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343180391388464:2432], now have 1 active actors on pipe
2025-10-09T21:52:18.366076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343180029535646:8762]
2025-10-09T21:52:18.366077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343180029535646:8762]
2025-10-09T21:52:18.366078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343180029535646:8762]
2025-10-09T21:52:18.366088Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:18.366099Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:18.366099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:18.366101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:18.366103Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:18.366110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:18.366113Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:18.366133Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343180029535646:8762]
2025-10-09T21:52:18.366142Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:18.366136Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2409
2025-10-09T21:52:18.366138Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2409
2025-10-09T21:52:18.366158Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:18.366206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.366215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.366243Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|266d3483-3386377a-a6a5bf9-41fda27d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:18.366255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:18.366264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:18.366270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:18.366280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:18.366293Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:18.366308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:18.366461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.366470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.366484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:18.366580Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2410 partition: 0 MaxSeqNo: 25 sessionId: src_id|266d3483-3386377a-a6a5bf9-41fda27d_0
2025-10-09T21:52:18.366849Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|60a0fa36-54cfe257-de7cd8b1-71eaf447_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|266d3483-3386377a-a6a5bf9-41fda27d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:18.369067Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|266d3483-3386377a-a6a5bf9-41fda27d_0] Write session: try to update token
2025-10-09T21:52:18.369080Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|266d3483-3386377a-a6a5bf9-41fda27d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:18.379731Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:18.379746Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:18.379748Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:18.379750Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:18.379760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343180029535646:8762]
2025-10-09T21:52:18.379764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343180029535646:8762]
2025-10-09T21:52:18.379768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343180029535646:8762]
2025-10-09T21:52:18.379809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:18.379817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.379819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:18.379823Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:18.379835Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:18.379840Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:18.379944Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:18.380714Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2410 sessionId: src_id|266d3483-3386377a-a6a5bf9-41fda27d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:18.382918Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:18.439677Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343180391388714:5758]
2025-10-09T21:52:18.439928Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 2409, pipe: [1:7559343180391388714:5758]
2025-10-09T21:52:18.518344Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:18.518767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.518780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.518816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343180391388714:5758], now have 1 active actors on pipe
2025-10-09T21:52:18.519024Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:18.519044Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:18.519060Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343180391388714:5758]
2025-10-09T21:52:18.519086Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:18.519111Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2410 reason: error at write: Disconnected sessionId: src_id|266d3483-3386377a-a6a5bf9-41fda27d_0
2025-10-09T21:52:18.519291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2410 sessionId: src_id|266d3483-3386377a-a6a5bf9-41fda27d_0 is DEAD
2025-10-09T21:52:18.519446Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|266d3483-3386377a-a6a5bf9-41fda27d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:18.519457Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|266d3483-3386377a-a6a5bf9-41fda27d_0] Write session will restart in 0.009439s
2025-10-09T21:52:18.519482Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|266d3483-3386377a-a6a5bf9-41fda27d_0] Write session: Do CDS request
2025-10-09T21:52:18.519490Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|266d3483-3386377a-a6a5bf9-41fda27d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:18.519640Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:18.530488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|266d3483-3386377a-a6a5bf9-41fda27d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:18.530570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:18.530584Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2411
2025-10-09T21:52:18.530710Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2411 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:18.530742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2411 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:18.530752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2411 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:18.530755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:18.531199Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2411 sessionId:  describe result for acl check
2025-10-09T21:52:18.531229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:18.531235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:18.531236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:18.531242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343180391388896:5598] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:18.531780Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343180391388896:5598] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:18.533461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:18.533632Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343180391388896:5598] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:18.535593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:18.535765Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:18.535777Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:18.535789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:18.535836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:18.535874Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343180029535891:8756]
2025-10-09T21:52:18.535879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343180391388896:5598] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:18.535889Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343180391388896:5598] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:18.535891Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343180391388896:5598] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:18.535897Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2411 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:18.536457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:18.536733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:18.536784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:18.537105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:18.537167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.537173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.537174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.537175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.537177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.537178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.537179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.537180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.537181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.537185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.537196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.537200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.537203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.537207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.537211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.537214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.537217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.537221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.537227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.537236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.537241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.537247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.537251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.537257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.537261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.537266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.537271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.537286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.537298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.537304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.537309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.537314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.537318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.537324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.537328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.537333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.537362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:18.543292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:18.544293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:18.544305Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:18.544306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:18.544309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:18.544314Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2410 [2:7559343180029535891:8756]
2025-10-09T21:52:18.544323Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:18.544336Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:18.544337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:18.544339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:18.544341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:18.544356Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:18.544368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:18.544375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:18.544429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.544439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.544441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.544444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:18.544497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:18.544503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.544510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.544520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.544533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:18.544552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343180391388956:5598], now have 1 active actors on pipe
2025-10-09T21:52:18.544559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343180391388717:2432], now have 1 active actors on pipe
2025-10-09T21:52:18.544567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343180029535886:8756]
2025-10-09T21:52:18.544573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343180029535886:8756]
2025-10-09T21:52:18.544575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343180029535886:8756]
2025-10-09T21:52:18.544588Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:18.544598Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:18.544600Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:18.544602Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:18.544607Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:18.544623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:18.544630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:18.544649Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343180029535886:8756]
2025-10-09T21:52:18.544658Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:18.544639Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2410
2025-10-09T21:52:18.544640Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2410
2025-10-09T21:52:18.544671Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:18.544697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.544704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.544725Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bd5323ea-c2113c7a-a8690499-88230d8f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:18.544734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:18.544737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:18.544738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:18.544745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:18.544754Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:18.544767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:18.544887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.544893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.544903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:18.545057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2411 partition: 0 MaxSeqNo: 25 sessionId: src_id|bd5323ea-c2113c7a-a8690499-88230d8f_0
2025-10-09T21:52:18.545422Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|266d3483-3386377a-a6a5bf9-41fda27d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bd5323ea-c2113c7a-a8690499-88230d8f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:18.547860Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd5323ea-c2113c7a-a8690499-88230d8f_0] Write session: try to update token
2025-10-09T21:52:18.547879Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd5323ea-c2113c7a-a8690499-88230d8f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:18.556003Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:18.556021Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:18.556030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343180029535886:8756]
2025-10-09T21:52:18.556069Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:18.556080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:18.556417Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:18.556660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:18.556911Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:18.557147Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:18.557515Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:18.557572Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:18.560332Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2411 sessionId: src_id|bd5323ea-c2113c7a-a8690499-88230d8f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:18.562575Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:18.562748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.562766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.562803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:18.562894Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:18.568341Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:18.568353Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:18.568356Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:18.568358Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:18.568368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343180029535886:8756]
2025-10-09T21:52:18.568372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343180029535886:8756]
2025-10-09T21:52:18.568375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343180029535886:8756]
2025-10-09T21:52:18.568407Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:18.568410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:18.568412Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:18.568415Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:18.568416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.568418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:18.568539Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:18.619354Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343180391388989:5758]
2025-10-09T21:52:18.619554Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 2410, pipe: [1:7559343180391388989:5758]
2025-10-09T21:52:18.704449Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:18.711220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.711241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.711382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.711436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.711493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.711547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.711588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.711593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.711650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.711705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.711760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.711819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.711857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.711864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.711918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.711966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.712017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.712072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.712111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.712116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.712172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.712227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.712281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.712336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.712374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.712379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.712434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.712488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.712543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.712596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.712635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.712641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.712694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.712748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.712802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.712856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.712897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.712902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.712954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.713003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.713060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.713113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.713150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.713155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.713206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.713255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.713320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.713371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.713408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.713413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.713463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.713512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.713563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.713615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.713652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.713657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.713709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.713758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.713811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.713863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.713906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.713912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.713951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343180391388989:5758], now have 1 active actors on pipe
2025-10-09T21:52:18.714222Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:18.714224Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343180391388989:5758]
2025-10-09T21:52:18.714237Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:18.714246Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:18.714276Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2411 reason: error at write: Disconnected sessionId: src_id|bd5323ea-c2113c7a-a8690499-88230d8f_0
2025-10-09T21:52:18.714467Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2411 sessionId: src_id|bd5323ea-c2113c7a-a8690499-88230d8f_0 is DEAD
2025-10-09T21:52:18.714623Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd5323ea-c2113c7a-a8690499-88230d8f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:18.714636Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd5323ea-c2113c7a-a8690499-88230d8f_0] Write session will restart in 0.007216s
2025-10-09T21:52:18.714663Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd5323ea-c2113c7a-a8690499-88230d8f_0] Write session: Do CDS request
2025-10-09T21:52:18.714669Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd5323ea-c2113c7a-a8690499-88230d8f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:18.714794Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:18.722464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bd5323ea-c2113c7a-a8690499-88230d8f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:18.722536Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:18.722546Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2412
2025-10-09T21:52:18.722681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2412 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:18.722712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2412 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:18.722723Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2412 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:18.722726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:18.723478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2412 sessionId:  describe result for acl check
2025-10-09T21:52:18.723517Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:18.723526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:18.723527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:18.723535Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343180391389029:5546] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:18.724137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343180391389029:5546] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:18.727245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343180391389029:5546] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:18.730107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343180391389029:5546] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:18.730122Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343180391389029:5546] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:18.730125Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343180391389029:5546] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:18.730133Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2412 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:18.731610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:18.733901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:18.734083Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:18.734091Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:18.734100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:18.734164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:18.734205Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343180029536173:8739]
2025-10-09T21:52:18.734630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:18.734950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:18.734993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:18.735258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:18.735311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.735317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.735317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.735318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.735319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.735319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.735320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.735320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.735321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.735333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.735340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.735342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.735344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.735345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.735348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.735349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.735351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.735353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.735358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.735363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.735367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.735371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.735373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.735376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.735378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.735381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.735383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.735386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.735388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.735391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.735395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.735398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.735400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.735403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.735406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.735408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.735423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:18.741479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:18.742510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:18.742523Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:18.742524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:18.742527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:18.742532Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2411 [2:7559343180029536173:8739]
2025-10-09T21:52:18.742541Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:18.742554Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:18.742556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:18.742559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:18.742560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:18.742574Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:18.742578Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:18.742583Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:18.742622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.742628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.742629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.742631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:18.742684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:18.742691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.742705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.742719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.742728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:18.742743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343180391389223:5546], now have 1 active actors on pipe
2025-10-09T21:52:18.742749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343180029536168:8739]
2025-10-09T21:52:18.742750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343180029536168:8739]
2025-10-09T21:52:18.742752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343180029536168:8739]
2025-10-09T21:52:18.742761Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:18.742768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343180391388992:2432], now have 1 active actors on pipe
2025-10-09T21:52:18.742777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:18.742777Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:18.742780Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:18.742782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:18.742789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:18.742791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:18.742816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343180029536168:8739]
2025-10-09T21:52:18.742827Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:18.742833Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:18.742846Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2411
2025-10-09T21:52:18.743067Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2411
2025-10-09T21:52:18.743123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.743134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.743165Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c7886b58-95155a26-3ba96b30-fab3433d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:18.743179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:18.743187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:18.743190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:18.743204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:18.743231Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:18.743248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:18.743423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.743432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.743451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:18.743538Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2412 partition: 0 MaxSeqNo: 25 sessionId: src_id|c7886b58-95155a26-3ba96b30-fab3433d_0
2025-10-09T21:52:18.743868Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bd5323ea-c2113c7a-a8690499-88230d8f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c7886b58-95155a26-3ba96b30-fab3433d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:18.746366Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c7886b58-95155a26-3ba96b30-fab3433d_0] Write session: try to update token
2025-10-09T21:52:18.746384Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c7886b58-95155a26-3ba96b30-fab3433d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:18.753546Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:18.753560Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:18.753571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343180029536168:8739]
2025-10-09T21:52:18.753617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:18.753616Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:18.753776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:18.753912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:18.754051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:18.754183Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:18.754347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:18.754388Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:18.759057Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2412 sessionId: src_id|c7886b58-95155a26-3ba96b30-fab3433d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:18.761256Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:18.761445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.761463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.761497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:18.761593Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:18.765445Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:18.765455Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:18.765456Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:18.765458Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:18.765466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343180029536168:8739]
2025-10-09T21:52:18.765469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343180029536168:8739]
2025-10-09T21:52:18.765472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343180029536168:8739]
2025-10-09T21:52:18.765509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:18.765515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.765518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:18.765526Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:18.765539Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:18.765544Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:18.765639Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:18.814459Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343180391389243:5758]
2025-10-09T21:52:18.814746Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 2411, pipe: [1:7559343180391389243:5758]
2025-10-09T21:52:18.905061Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:18.912367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.912388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.912526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.912578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.912630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.912682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.912902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.912915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.912987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.913044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.913101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.913155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.913202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.913208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.913263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.913332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.913392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.913450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.913488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.913495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.913549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.913603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.913656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.913708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.913753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.913762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.913826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.913887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.913943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.913996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.914048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.914055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.914110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.914159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.914223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.914277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.914314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.914319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.914373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.914430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.914493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.914547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.914584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.914593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.914647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.914698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.914747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.914795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.914831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.914839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.914888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.914935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.914984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.915047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.915089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.915096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.915154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:18.915211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:18.915270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:18.915325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:18.915363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:18.915370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:18.915406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343180391389243:5758], now have 1 active actors on pipe
2025-10-09T21:52:18.915614Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343180391389243:5758]
2025-10-09T21:52:18.915622Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:18.915626Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:18.915646Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:18.915651Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2412 reason: error at write: Disconnected sessionId: src_id|c7886b58-95155a26-3ba96b30-fab3433d_0
2025-10-09T21:52:18.915803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2412 sessionId: src_id|c7886b58-95155a26-3ba96b30-fab3433d_0 is DEAD
2025-10-09T21:52:18.915952Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7886b58-95155a26-3ba96b30-fab3433d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:18.915962Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7886b58-95155a26-3ba96b30-fab3433d_0] Write session will restart in 0.005274s
2025-10-09T21:52:18.915981Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7886b58-95155a26-3ba96b30-fab3433d_0] Write session: Do CDS request
2025-10-09T21:52:18.915986Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7886b58-95155a26-3ba96b30-fab3433d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:18.916156Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:18.922454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c7886b58-95155a26-3ba96b30-fab3433d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:18.922521Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:18.922530Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2413
2025-10-09T21:52:18.922642Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2413 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:18.922671Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2413 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:18.922679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2413 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:18.922683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:18.923013Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2413 sessionId:  describe result for acl check
2025-10-09T21:52:18.923040Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:18.923042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:18.923043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:18.923047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343180391389270:5530] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:18.923489Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343180391389270:5530] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:18.926210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343180391389270:5530] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:18.929361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343180391389270:5530] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:18.929374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343180391389270:5530] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:18.929376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343180391389270:5530] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:18.929381Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2413 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:18.931324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:18.933495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:18.933665Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:18.933674Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:18.933683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:18.933719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:18.933753Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343180029536420:8730]
2025-10-09T21:52:18.934208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:18.934509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:18.934555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:18.934787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:18.934824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.934830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.934831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.934832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.934833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.934834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.934835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.934835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.934836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.934841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.934850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.934853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.934857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.934861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.934865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.934868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.934870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.934872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.934876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.934881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:18.934885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.934888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:18.934890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.934893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:18.934895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.934897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:18.934900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.934902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:18.934904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.934907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:18.934910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.934913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.934915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.934918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.934921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.934924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.934940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:18.940690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:18.941739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:18.941751Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:18.941752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:18.941754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:18.941759Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2412 [2:7559343180029536420:8730]
2025-10-09T21:52:18.941767Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:18.941779Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:18.941780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:18.941782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:18.941783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:18.941795Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:18.941799Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:18.941804Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:18.941839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.941844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.941845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.941847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:18.941890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:18.941895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.941902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:18.941908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:18.941916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:18.941932Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343180029536415:8730]
2025-10-09T21:52:18.941937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343180029536415:8730]
2025-10-09T21:52:18.941939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343180029536415:8730]
2025-10-09T21:52:18.941944Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:18.941953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:18.941955Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:18.941957Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:18.941959Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:18.941964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:18.941966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:18.941974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343180391389461:5530], now have 1 active actors on pipe
2025-10-09T21:52:18.941979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343180391389246:2432], now have 1 active actors on pipe
2025-10-09T21:52:18.941995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343180029536415:8730]
2025-10-09T21:52:18.942003Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:18.942047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:18.942066Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2412
2025-10-09T21:52:18.942131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.942139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.942150Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2412
2025-10-09T21:52:18.942158Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|82ebac39-6beaf0f1-ccc6adb5-efdb6b20_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:18.942168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:18.942170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:18.942172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:18.942178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:18.942185Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:18.942200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:18.942356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.942362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.942372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:18.942469Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2413 partition: 0 MaxSeqNo: 25 sessionId: src_id|82ebac39-6beaf0f1-ccc6adb5-efdb6b20_0
2025-10-09T21:52:18.942777Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c7886b58-95155a26-3ba96b30-fab3433d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|82ebac39-6beaf0f1-ccc6adb5-efdb6b20_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:18.945375Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|82ebac39-6beaf0f1-ccc6adb5-efdb6b20_0] Write session: try to update token
2025-10-09T21:52:18.945394Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|82ebac39-6beaf0f1-ccc6adb5-efdb6b20_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:18.958054Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:18.958067Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:18.958076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343180029536415:8730]
2025-10-09T21:52:18.958121Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:18.958125Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:18.958290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:18.958459Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:18.958538Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2413 sessionId: src_id|82ebac39-6beaf0f1-ccc6adb5-efdb6b20_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:18.958594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:18.958735Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:18.958922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:18.958964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:18.960811Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:18.961014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:18.961028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:18.961062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:18.961141Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:18.965135Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:18.965149Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:18.965151Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:18.965153Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:18.965162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343180029536415:8730]
2025-10-09T21:52:18.965166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343180029536415:8730]
2025-10-09T21:52:18.965169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343180029536415:8730]
2025-10-09T21:52:18.965206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:18.965214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:18.965222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:18.965227Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:18.965235Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:18.965246Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:18.965353Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:19.015696Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343184686356796:5758]
2025-10-09T21:52:19.015992Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 2412, pipe: [1:7559343184686356796:5758]
2025-10-09T21:52:19.112333Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:19.120063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:19.120084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:19.120242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:19.120306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:19.120371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:19.120431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:19.120476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:19.120485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:19.120562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:19.120623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:19.120695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:19.120766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:19.120813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:19.120821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:19.120881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:19.120938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:19.120998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:19.121065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:19.121107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:19.121113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:19.121178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:19.121237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:19.121312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:19.121378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:19.121424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:19.121429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:19.121492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:19.121556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:19.121626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:19.121687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:19.121727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:19.121732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:19.121798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:19.121861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:19.121925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:19.121993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:19.122056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:19.122065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:19.122126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:19.122181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:19.122243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:19.122310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:19.122354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:19.122363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:19.122425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:19.122496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:19.122567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:19.122631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:19.122673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:19.122680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:19.122745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:19.122805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:19.122867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:19.122932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:19.122979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:19.122984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:19.123049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:19.123109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:19.123172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:19.123241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:19.123284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:19.123292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:19.123362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343184686356796:5758], now have 1 active actors on pipe
2025-10-09T21:52:19.123617Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:19.123618Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343184686356796:5758]
2025-10-09T21:52:19.123650Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:19.123655Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:19.123685Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2413 reason: error at write: Disconnected sessionId: src_id|82ebac39-6beaf0f1-ccc6adb5-efdb6b20_0
2025-10-09T21:52:19.123889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2413 sessionId: src_id|82ebac39-6beaf0f1-ccc6adb5-efdb6b20_0 is DEAD
2025-10-09T21:52:19.124041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82ebac39-6beaf0f1-ccc6adb5-efdb6b20_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:19.124054Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82ebac39-6beaf0f1-ccc6adb5-efdb6b20_0] Write session will restart in 0.006071s
2025-10-09T21:52:19.124075Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82ebac39-6beaf0f1-ccc6adb5-efdb6b20_0] Write session: Do CDS request
2025-10-09T21:52:19.124080Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82ebac39-6beaf0f1-ccc6adb5-efdb6b20_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:19.124694Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:19.131552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|82ebac39-6beaf0f1-ccc6adb5-efdb6b20_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:19.131622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:19.131640Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2414
2025-10-09T21:52:19.131781Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2414 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:19.131817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2414 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:19.131830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2414 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:19.131834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:19.132268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2414 sessionId:  describe result for acl check
2025-10-09T21:52:19.132307Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:19.132314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:19.132316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:19.132323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343184686356828:5515] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:19.132892Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343184686356828:5515] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:19.136675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343184686356828:5515] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:19.139324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343184686356828:5515] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:19.139334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343184686356828:5515] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:19.139337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343184686356828:5515] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:19.139344Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2414 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:19.140141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:19.142819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:19.143035Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:19.143045Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:19.143058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:19.143122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:19.143166Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343184324503960:8728]
2025-10-09T21:52:19.143787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:19.144063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:19.144102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:19.144412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:19.144473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.144478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.144480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.144481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.144482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.144484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.144485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.144486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.144488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.144493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.144505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.144509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.144513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.144517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.144521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.144525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.144529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.144533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.144541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.144549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.144560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.144567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.144575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.144581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.144586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.144593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.144600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.144605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.144612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.144617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.144624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.144628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.144630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.144635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.144638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.144646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.144671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:19.151426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:19.152448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:19.152461Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:19.152463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:19.152467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:19.152475Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2413 [2:7559343184324503960:8728]
2025-10-09T21:52:19.152486Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:19.152502Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:19.152505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:19.152507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:19.152508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:19.152523Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:19.152531Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:19.152538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:19.152584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.152588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.152590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.152592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:19.152921Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2413
2025-10-09T21:52:19.152971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:19.152979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.152987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.152994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.153004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:19.152998Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2413
2025-10-09T21:52:19.153021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343184686357038:5515], now have 1 active actors on pipe
2025-10-09T21:52:19.153028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343184686356799:2432], now have 1 active actors on pipe
2025-10-09T21:52:19.153039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343184324503955:8728]
2025-10-09T21:52:19.153040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343184324503955:8728]
2025-10-09T21:52:19.153041Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343184324503955:8728]
2025-10-09T21:52:19.153048Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:19.153061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:19.153069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:19.153072Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:19.153074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:19.153075Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:19.153076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:19.153094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:19.153101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:19.153139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343184324503955:8728]
2025-10-09T21:52:19.153152Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:19.153170Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|71690196-534f650f-84933fd1-4faafd33_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:19.153178Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:19.153183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:19.153188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:19.153190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:19.153202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:19.153213Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:19.153252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:19.153430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:19.153436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:19.153447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:19.153549Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2414 partition: 0 MaxSeqNo: 25 sessionId: src_id|71690196-534f650f-84933fd1-4faafd33_0
2025-10-09T21:52:19.153916Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82ebac39-6beaf0f1-ccc6adb5-efdb6b20_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|71690196-534f650f-84933fd1-4faafd33_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:19.156440Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|71690196-534f650f-84933fd1-4faafd33_0] Write session: try to update token
2025-10-09T21:52:19.156463Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|71690196-534f650f-84933fd1-4faafd33_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:19.168142Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:19.168157Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:19.168160Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:19.168162Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:19.168171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343184324503955:8728]
2025-10-09T21:52:19.168176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343184324503955:8728]
2025-10-09T21:52:19.168181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343184324503955:8728]
2025-10-09T21:52:19.168223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:19.168231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.168232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:19.168262Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:19.168284Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:19.168290Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:19.168373Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:19.170806Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2414 sessionId: src_id|71690196-534f650f-84933fd1-4faafd33_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:19.173251Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:19.223554Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343184686357055:5758]
2025-10-09T21:52:19.223845Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 2413, pipe: [1:7559343184686357055:5758]
2025-10-09T21:52:19.312297Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:19.312719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:19.312735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:19.312786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343184686357055:5758], now have 1 active actors on pipe
2025-10-09T21:52:19.312961Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:19.312999Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:19.313016Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343184686357055:5758]
2025-10-09T21:52:19.313047Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:19.313061Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:19.313159Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2414 reason: error at write: Disconnected sessionId: src_id|71690196-534f650f-84933fd1-4faafd33_0
2025-10-09T21:52:19.313359Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2414 sessionId: src_id|71690196-534f650f-84933fd1-4faafd33_0 is DEAD
2025-10-09T21:52:19.313590Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71690196-534f650f-84933fd1-4faafd33_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:19.313603Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71690196-534f650f-84933fd1-4faafd33_0] Write session will restart in 0.006713s
2025-10-09T21:52:19.313631Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71690196-534f650f-84933fd1-4faafd33_0] Write session: Do CDS request
2025-10-09T21:52:19.313638Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71690196-534f650f-84933fd1-4faafd33_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:19.321514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|71690196-534f650f-84933fd1-4faafd33_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:19.321633Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:19.321648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2415
2025-10-09T21:52:19.321778Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2415 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:19.321811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2415 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:19.321823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2415 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:19.321828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:19.322297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2415 sessionId:  describe result for acl check
2025-10-09T21:52:19.322334Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:19.322336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:19.322338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:19.322345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343184686357112:5501] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:19.322842Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343184686357112:5501] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:19.325543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343184686357112:5501] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:19.328241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343184686357112:5501] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:19.328258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343184686357112:5501] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:19.328262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343184686357112:5501] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:19.328270Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2415 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:19.329540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:19.331690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:19.331824Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:19.331831Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:19.331840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:19.331876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:19.331905Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343184324504206:8724]
2025-10-09T21:52:19.332263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:19.332655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:19.332697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:19.332911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:19.332939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.332943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.332944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.332944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.332945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.332946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.332946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.332947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.332948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.332951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.332958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.332960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.332963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.332965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.332967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.332969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.332971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.332973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.332977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.332982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.332986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.332990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.332992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.332995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.332999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.333002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.333005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.333008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.333010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.333013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.333016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.333021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.333027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.333030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.333032Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.333035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.333051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:19.338883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:19.339878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:19.339893Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:19.339895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:19.339899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:19.339905Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2414 [2:7559343184324504206:8724]
2025-10-09T21:52:19.339917Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:19.339931Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:19.339934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:19.339937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:19.339940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:19.339955Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:19.339960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:19.339972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:19.340023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.340029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.340031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.340034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:19.340098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:19.340106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.340117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.340129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.340150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:19.340174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343184686357060:2432], now have 1 active actors on pipe
2025-10-09T21:52:19.340182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343184686357305:5501], now have 1 active actors on pipe
2025-10-09T21:52:19.340189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343184324504201:8724]
2025-10-09T21:52:19.340197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343184324504201:8724]
2025-10-09T21:52:19.340199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343184324504201:8724]
2025-10-09T21:52:19.340211Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:19.340225Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:19.340236Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:19.340239Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:19.340241Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:19.340243Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:19.340246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:19.340267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343184324504201:8724]
2025-10-09T21:52:19.340272Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:19.340246Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2414
2025-10-09T21:52:19.340258Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2414
2025-10-09T21:52:19.340283Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:19.340299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:19.340308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:19.340341Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d089b754-6a163883-23e1de2b-aedbbe04_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:19.340352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:19.340356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:19.340360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:19.340370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:19.340384Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:19.340407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:19.340550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:19.340557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:19.340569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:19.340660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2415 partition: 0 MaxSeqNo: 25 sessionId: src_id|d089b754-6a163883-23e1de2b-aedbbe04_0
2025-10-09T21:52:19.340972Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|71690196-534f650f-84933fd1-4faafd33_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d089b754-6a163883-23e1de2b-aedbbe04_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:19.343313Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d089b754-6a163883-23e1de2b-aedbbe04_0] Write session: try to update token
2025-10-09T21:52:19.343332Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d089b754-6a163883-23e1de2b-aedbbe04_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:19.353268Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:19.353297Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:19.353299Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:19.353301Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:19.353310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343184324504201:8724]
2025-10-09T21:52:19.353313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343184324504201:8724]
2025-10-09T21:52:19.353317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343184324504201:8724]
2025-10-09T21:52:19.353358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:19.353370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.353373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:19.353400Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:19.353421Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:19.353428Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:19.353500Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:19.355302Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2415 sessionId: src_id|d089b754-6a163883-23e1de2b-aedbbe04_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:19.357597Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:19.413018Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343184686357320:5758]
2025-10-09T21:52:19.413341Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 2414, pipe: [1:7559343184686357320:5758]
2025-10-09T21:52:19.499291Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:19.499928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:19.499941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:19.499978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343184686357320:5758], now have 1 active actors on pipe
2025-10-09T21:52:19.500200Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:19.500219Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343184686357320:5758]
2025-10-09T21:52:19.500253Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:19.500255Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:19.500243Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:19.500312Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2415 reason: error at write: Disconnected sessionId: src_id|d089b754-6a163883-23e1de2b-aedbbe04_0
2025-10-09T21:52:19.500470Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2415 sessionId: src_id|d089b754-6a163883-23e1de2b-aedbbe04_0 is DEAD
2025-10-09T21:52:19.500611Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d089b754-6a163883-23e1de2b-aedbbe04_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:19.500622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d089b754-6a163883-23e1de2b-aedbbe04_0] Write session will restart in 0.008406s
2025-10-09T21:52:19.500650Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d089b754-6a163883-23e1de2b-aedbbe04_0] Write session: Do CDS request
2025-10-09T21:52:19.500656Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d089b754-6a163883-23e1de2b-aedbbe04_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:19.510578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d089b754-6a163883-23e1de2b-aedbbe04_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:19.510880Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:19.510895Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2416
2025-10-09T21:52:19.511029Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2416 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:19.511062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2416 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:19.511083Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2416 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:19.511087Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:19.511545Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2416 sessionId:  describe result for acl check
2025-10-09T21:52:19.511584Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:19.511590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:19.511592Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:19.511598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343184686357422:5480] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:19.512168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343184686357422:5480] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:19.514604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343184686357422:5480] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:19.516352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:19.518853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:19.518921Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343184686357422:5480] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:19.518934Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343184686357422:5480] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:19.518938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343184686357422:5480] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:19.518945Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2416 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:19.519033Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:19.519042Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:19.519054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:19.519105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:19.519146Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343184324504450:8710]
2025-10-09T21:52:19.519693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:19.519964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:19.519995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:19.520201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:19.520239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.520244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.520246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.520247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.520248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.520249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.520250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.520251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.520252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.520255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.520262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.520264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.520266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.520268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.520270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.520272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.520274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.520276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.520281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.520286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.520290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.520293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.520295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.520298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.520300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.520304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.520307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.520310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.520312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.520315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.520318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.520321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.520323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.520326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.520329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.520331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.520349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:19.526102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:19.527208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:19.527225Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:19.527227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:19.527231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:19.527238Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2415 [2:7559343184324504450:8710]
2025-10-09T21:52:19.527252Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:19.527268Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:19.527277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:19.527280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:19.527282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:19.527299Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:19.527309Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:19.527321Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:19.527379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.527385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.527387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.527390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:19.527456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:19.527463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.527475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.527491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.527513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:19.527534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343184686357567:5480], now have 1 active actors on pipe
2025-10-09T21:52:19.527546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343184324504445:8710]
2025-10-09T21:52:19.527548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343184324504445:8710]
2025-10-09T21:52:19.527551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343184324504445:8710]
2025-10-09T21:52:19.527586Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:19.527597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343184686357322:2432], now have 1 active actors on pipe
2025-10-09T21:52:19.527603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:19.527608Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:19.527611Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:19.527613Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:19.527615Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:19.527618Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:19.527639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343184324504445:8710]
2025-10-09T21:52:19.527645Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:19.527633Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2415
2025-10-09T21:52:19.527635Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2415
2025-10-09T21:52:19.527677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:19.527684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:19.527684Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:19.527956Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|91bbe775-190024f7-33d628fd-29ce262d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:19.527973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:19.527977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:19.527980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:19.527991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:19.527999Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:19.528013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:19.528209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:19.528218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:19.528234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:19.528335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2416 partition: 0 MaxSeqNo: 25 sessionId: src_id|91bbe775-190024f7-33d628fd-29ce262d_0
2025-10-09T21:52:19.528681Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d089b754-6a163883-23e1de2b-aedbbe04_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|91bbe775-190024f7-33d628fd-29ce262d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:19.531240Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91bbe775-190024f7-33d628fd-29ce262d_0] Write session: try to update token
2025-10-09T21:52:19.531256Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91bbe775-190024f7-33d628fd-29ce262d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:19.541129Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:19.541147Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:19.541149Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:19.541151Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:19.541161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343184324504445:8710]
2025-10-09T21:52:19.541165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343184324504445:8710]
2025-10-09T21:52:19.541170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343184324504445:8710]
2025-10-09T21:52:19.541212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:19.541223Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:19.541224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.541228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:19.541229Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:19.541234Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:19.541368Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:19.543754Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2416 sessionId: src_id|91bbe775-190024f7-33d628fd-29ce262d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:19.545926Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:19.600367Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343184686357578:5758]
2025-10-09T21:52:19.600648Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 2415, pipe: [1:7559343184686357578:5758]
2025-10-09T21:52:19.681359Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:19.681586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:19.681600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:19.681638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343184686357578:5758], now have 1 active actors on pipe
2025-10-09T21:52:19.681843Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:19.681854Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:19.681860Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343184686357578:5758]
2025-10-09T21:52:19.681880Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:19.681935Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2416 reason: error at write: Disconnected sessionId: src_id|91bbe775-190024f7-33d628fd-29ce262d_0
2025-10-09T21:52:19.682014Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:19.682122Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2416 sessionId: src_id|91bbe775-190024f7-33d628fd-29ce262d_0 is DEAD
2025-10-09T21:52:19.682260Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91bbe775-190024f7-33d628fd-29ce262d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:19.682272Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91bbe775-190024f7-33d628fd-29ce262d_0] Write session will restart in 0.006991s
2025-10-09T21:52:19.682301Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91bbe775-190024f7-33d628fd-29ce262d_0] Write session: Do CDS request
2025-10-09T21:52:19.682308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91bbe775-190024f7-33d628fd-29ce262d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:19.690388Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|91bbe775-190024f7-33d628fd-29ce262d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:19.690820Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:19.690831Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2417
2025-10-09T21:52:19.690947Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2417 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:19.690982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2417 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:19.690991Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2417 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:19.690996Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:19.691347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2417 sessionId:  describe result for acl check
2025-10-09T21:52:19.691377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:19.691382Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:19.691384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:19.691390Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343184686357675:5440] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:19.691801Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343184686357675:5440] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:19.693919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343184686357675:5440] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:19.696661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:19.696721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343184686357675:5440] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:19.696733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343184686357675:5440] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:19.696736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343184686357675:5440] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:19.696741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2417 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:19.698774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:19.698924Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:19.698934Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:19.698946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:19.698999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:19.699033Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343184324504737:8704]
2025-10-09T21:52:19.699450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:19.699685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:19.699714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:19.699906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:19.699933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.699938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.699938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.699939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.699939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.699940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.699941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.699941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.699942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.699945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.699951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.699953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.699955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.699957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.699959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.699961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.699963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.699964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.699968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.699973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.699977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.699980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.699982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.699986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.699988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.699991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.699997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.699999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.700001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.700005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.700010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.700013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.700016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.700019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.700021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.700024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.700040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:19.705970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:19.707271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:19.707287Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:19.707290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:19.707293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:19.707301Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2416 [2:7559343184324504737:8704]
2025-10-09T21:52:19.707316Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:19.707334Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:19.707338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:19.707341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:19.707344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:19.707363Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:19.707373Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:19.707380Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:19.707436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.707443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.707445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.707448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:19.707512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:19.707519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.707531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.707547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.707569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:19.707595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343184686357604:2432], now have 1 active actors on pipe
2025-10-09T21:52:19.707598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343184686357846:5440], now have 1 active actors on pipe
2025-10-09T21:52:19.707606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343184324504731:8704]
2025-10-09T21:52:19.707609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343184324504731:8704]
2025-10-09T21:52:19.707611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343184324504731:8704]
2025-10-09T21:52:19.707624Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:19.707636Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:19.707638Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:19.707639Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:19.707640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:19.707653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:19.707656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:19.707682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343184324504731:8704]
2025-10-09T21:52:19.707691Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:19.707662Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2416
2025-10-09T21:52:19.707692Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:19.707671Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2416
2025-10-09T21:52:19.707720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:19.707725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:19.707759Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|57d802a1-b2299de3-78f3b0b9-6c093f3e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:19.707771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:19.707775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:19.707777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:19.707787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:19.707801Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:19.707827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:19.707980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:19.707989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:19.708003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:19.708089Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2417 partition: 0 MaxSeqNo: 25 sessionId: src_id|57d802a1-b2299de3-78f3b0b9-6c093f3e_0
2025-10-09T21:52:19.708382Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|91bbe775-190024f7-33d628fd-29ce262d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|57d802a1-b2299de3-78f3b0b9-6c093f3e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:19.710772Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57d802a1-b2299de3-78f3b0b9-6c093f3e_0] Write session: try to update token
2025-10-09T21:52:19.710788Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57d802a1-b2299de3-78f3b0b9-6c093f3e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:19.721740Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:19.721755Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:19.721758Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:19.721760Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:19.721769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343184324504731:8704]
2025-10-09T21:52:19.721773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343184324504731:8704]
2025-10-09T21:52:19.721776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343184324504731:8704]
2025-10-09T21:52:19.721807Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:19.721816Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:19.721819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:19.721821Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:19.721827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.721830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:19.721959Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:19.723133Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2417 sessionId: src_id|57d802a1-b2299de3-78f3b0b9-6c093f3e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:19.725396Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:19.782143Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343184686357858:5758]
2025-10-09T21:52:19.782400Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 2416, pipe: [1:7559343184686357858:5758]
2025-10-09T21:52:19.862611Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:19.863042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:19.863059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:19.863105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343184686357858:5758], now have 1 active actors on pipe
2025-10-09T21:52:19.863272Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:19.863299Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:19.863327Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343184686357858:5758]
2025-10-09T21:52:19.863338Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:19.863403Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2417 reason: status is not ok: tablet will be restarted right now sessionId: src_id|57d802a1-b2299de3-78f3b0b9-6c093f3e_0
2025-10-09T21:52:19.863556Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2417 sessionId: src_id|57d802a1-b2299de3-78f3b0b9-6c093f3e_0 is DEAD
2025-10-09T21:52:19.863729Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57d802a1-b2299de3-78f3b0b9-6c093f3e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:19.863741Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57d802a1-b2299de3-78f3b0b9-6c093f3e_0] Write session will restart in 0.005746s
2025-10-09T21:52:19.863770Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57d802a1-b2299de3-78f3b0b9-6c093f3e_0] Write session: Do CDS request
2025-10-09T21:52:19.863777Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57d802a1-b2299de3-78f3b0b9-6c093f3e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:19.870456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57d802a1-b2299de3-78f3b0b9-6c093f3e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:19.870590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:19.870605Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2418
2025-10-09T21:52:19.870769Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2418 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:19.870803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2418 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:19.870814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2418 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:19.870818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:19.871423Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2418 sessionId:  describe result for acl check
2025-10-09T21:52:19.871461Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:19.871467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:19.871468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:19.871474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343184686357885:5429] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:19.872076Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343184686357885:5429] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:19.875191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343184686357885:5429] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:19.878213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343184686357885:5429] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:19.878226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343184686357885:5429] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:19.878230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343184686357885:5429] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:19.878239Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2418 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:19.880934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:19.883080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:19.883269Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:19.883278Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:19.883300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:19.883355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:19.883395Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343184324504988:8699]
2025-10-09T21:52:19.883984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:19.884282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:19.884336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:19.884637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:19.884680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.884687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.884688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.884689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.884690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.884691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.884692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.884694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.884695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.884700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.884711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.884715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.884718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.884722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.884725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.884728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.884732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.884738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.884745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.884753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:19.884760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.884770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:19.884779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.884784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:19.884789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.884794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:19.884797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.884800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:19.884802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.884805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:19.884808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.884815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.884818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.884823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.884827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.884832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.884864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:19.891172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:19.892244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:19.892256Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:19.892258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:19.892261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:19.892266Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2417 [2:7559343184324504988:8699]
2025-10-09T21:52:19.892278Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:19.892296Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:19.892299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:19.892302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:19.892305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:19.892323Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:19.892329Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:19.892342Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:19.892393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.892398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.892399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.892401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:19.892451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:19.892458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.892464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:19.892471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:19.892480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:19.892495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343184324504983:8699]
2025-10-09T21:52:19.892500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343184324504983:8699]
2025-10-09T21:52:19.892501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343184324504983:8699]
2025-10-09T21:52:19.892507Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:19.892517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:19.892519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:19.892527Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:19.892528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:19.892529Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:19.892529Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:19.892538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343184686357861:2432], now have 1 active actors on pipe
2025-10-09T21:52:19.892540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343184686358077:5429], now have 1 active actors on pipe
2025-10-09T21:52:19.892563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343184324504983:8699]
2025-10-09T21:52:19.892572Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:19.892588Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:19.892621Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2417
2025-10-09T21:52:19.892638Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2417
2025-10-09T21:52:19.892690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:19.892698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:19.892719Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d3a5d708-a92f74e9-e4a6597f-28bb2a14_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:19.892725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:19.892732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:19.892733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:19.892740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:19.892750Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:19.892766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:19.892892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:19.892897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:19.892907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:19.892981Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2418 partition: 0 MaxSeqNo: 25 sessionId: src_id|d3a5d708-a92f74e9-e4a6597f-28bb2a14_0
2025-10-09T21:52:19.893315Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57d802a1-b2299de3-78f3b0b9-6c093f3e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d3a5d708-a92f74e9-e4a6597f-28bb2a14_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:19.895708Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d3a5d708-a92f74e9-e4a6597f-28bb2a14_0] Write session: try to update token
2025-10-09T21:52:19.895725Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d3a5d708-a92f74e9-e4a6597f-28bb2a14_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:19.904531Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:19.904548Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:19.904556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343184324504983:8699]
2025-10-09T21:52:19.904594Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:19.904596Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:19.904745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:19.904869Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:19.904985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:19.905104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:19.905282Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:19.905319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:19.908684Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2418 sessionId: src_id|d3a5d708-a92f74e9-e4a6597f-28bb2a14_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:19.911035Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:19.911472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:19.911488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:19.911522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:19.911627Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:19.917262Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:19.917283Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:19.917285Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:19.917286Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:19.917297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343184324504983:8699]
2025-10-09T21:52:19.917299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343184324504983:8699]
2025-10-09T21:52:19.917303Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343184324504983:8699]
2025-10-09T21:52:19.917338Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:19.917342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:19.917347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:19.917349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:19.917350Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:19.917353Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:19.917472Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:19.963355Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343184686358120:5758]
2025-10-09T21:52:19.963633Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 2417, pipe: [1:7559343184686358120:5758]
2025-10-09T21:52:20.065415Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:20.072038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.072061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.072207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.072258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.072314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.072366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.072404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.072419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.072475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.072527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.072585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.072645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.072684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.072690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.072750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.072802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.072856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.072912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.072950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.072955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.073009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.073062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.073114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.073166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.073204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.073209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.073265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.073332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.073387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.073443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.073482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.073487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.073543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.073595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.073647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.073701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.073740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.073746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.073801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.073854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.073910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.073974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.074021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.074044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.074101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.074150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.074202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.074254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.074294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.074299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.074352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.074403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.074456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.074507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.074544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.074549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.074602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.074654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.074706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.074759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.074796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.074805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.074848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343184686358120:5758], now have 1 active actors on pipe
2025-10-09T21:52:20.075046Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:20.075065Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:20.075070Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:20.075106Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343184686358120:5758]
2025-10-09T21:52:20.075114Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2418 reason: error at write: Disconnected sessionId: src_id|d3a5d708-a92f74e9-e4a6597f-28bb2a14_0
2025-10-09T21:52:20.075295Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2418 sessionId: src_id|d3a5d708-a92f74e9-e4a6597f-28bb2a14_0 is DEAD
2025-10-09T21:52:20.075425Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3a5d708-a92f74e9-e4a6597f-28bb2a14_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:20.075436Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3a5d708-a92f74e9-e4a6597f-28bb2a14_0] Write session will restart in 0.008263s
2025-10-09T21:52:20.075462Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3a5d708-a92f74e9-e4a6597f-28bb2a14_0] Write session: Do CDS request
2025-10-09T21:52:20.075469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3a5d708-a92f74e9-e4a6597f-28bb2a14_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:20.075710Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:20.084488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d3a5d708-a92f74e9-e4a6597f-28bb2a14_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:20.084595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:20.084609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2419
2025-10-09T21:52:20.084751Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2419 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:20.084785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2419 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:20.084795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2419 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:20.084799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:20.085340Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2419 sessionId:  describe result for acl check
2025-10-09T21:52:20.085377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:20.085384Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:20.085386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:20.085392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343188981325504:5413] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:20.085944Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343188981325504:5413] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:20.088969Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343188981325504:5413] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:20.091014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:20.093154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:20.093159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343188981325504:5413] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:20.093172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343188981325504:5413] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:20.093176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343188981325504:5413] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:20.093185Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2419 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:20.093350Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:20.093360Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:20.093372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:20.093422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:20.093461Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343188619472533:8687]
2025-10-09T21:52:20.094076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:20.094352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:20.094390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:20.094706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:20.094750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.094755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.094757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.094757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.094759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.094760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.094761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.094762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.094763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.094767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.094778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.094782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.094786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.094790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.094794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.094798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.094802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.094806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.094813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.094821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.094829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.094834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.094838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.094844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.094848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.094853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.094862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.094868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.094876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.094881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.094890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.094899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.094907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.094912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.094921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.094930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.094959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:20.101107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:20.102453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:20.102464Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:20.102466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:20.102469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:20.102474Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2418 [2:7559343188619472533:8687]
2025-10-09T21:52:20.102486Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:20.102501Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:20.102506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:20.102509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:20.102510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:20.102523Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:20.102527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:20.102533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:20.102576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.102581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.102583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.102585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:20.102638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:20.102644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.102651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.102657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.102667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:20.102685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343188619472528:8687]
2025-10-09T21:52:20.102688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343188619472528:8687]
2025-10-09T21:52:20.102690Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343188619472528:8687]
2025-10-09T21:52:20.102699Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:20.102708Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:20.102711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:20.102712Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:20.102713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:20.102720Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:20.102722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:20.102731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343188981325419:2432], now have 1 active actors on pipe
2025-10-09T21:52:20.102733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343188981325666:5413], now have 1 active actors on pipe
2025-10-09T21:52:20.102754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343188619472528:8687]
2025-10-09T21:52:20.102764Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:20.102772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:20.102784Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2418
2025-10-09T21:52:20.102795Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2418
2025-10-09T21:52:20.102897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.102910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.102954Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|35386aaa-f69e589c-52ebac3b-1f682e81_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:20.102968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:20.102972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:20.102975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:20.102986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:20.103003Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:20.103022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:20.103188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.103197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.103211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:20.103277Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2419 partition: 0 MaxSeqNo: 25 sessionId: src_id|35386aaa-f69e589c-52ebac3b-1f682e81_0
2025-10-09T21:52:20.103626Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d3a5d708-a92f74e9-e4a6597f-28bb2a14_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|35386aaa-f69e589c-52ebac3b-1f682e81_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:20.106145Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|35386aaa-f69e589c-52ebac3b-1f682e81_0] Write session: try to update token
2025-10-09T21:52:20.106161Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|35386aaa-f69e589c-52ebac3b-1f682e81_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:20.114400Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:20.114416Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:20.114427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343188619472528:8687]
2025-10-09T21:52:20.114462Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:20.114477Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:20.114656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:20.114806Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:20.114959Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:20.115090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:20.115261Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:20.115297Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:20.119092Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2419 sessionId: src_id|35386aaa-f69e589c-52ebac3b-1f682e81_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:20.121318Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:20.121507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.121524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.121569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:20.121651Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:20.127331Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:20.127344Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:20.127347Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:20.127349Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:20.127359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343188619472528:8687]
2025-10-09T21:52:20.127363Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343188619472528:8687]
2025-10-09T21:52:20.127367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343188619472528:8687]
2025-10-09T21:52:20.127407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:20.127407Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:20.127412Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:20.127415Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:20.127418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.127420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:20.127561Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:20.175367Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343188981325679:5758]
2025-10-09T21:52:20.175939Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 2418, pipe: [1:7559343188981325679:5758]
2025-10-09T21:52:20.274662Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:20.281435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.281460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.281612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.281667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.281722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.281779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.281817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.281824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.281880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.281933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.281988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.282060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.282097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.282103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.282154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.282215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.282274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.282325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.282362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.282367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.282421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.282475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.282529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.282581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.282618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.282624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.282678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.282728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.282779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.282830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.282867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.282873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.282923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.282973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.283027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.283078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.283113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.283120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.283172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.283221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.283280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.283350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.283393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.283399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.283448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.283504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.283558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.283611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.283648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.283653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.283704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.283752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.283801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.283852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.283892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.283895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.283950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.283997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.284049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.284100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.284139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.284145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.284184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343188981325679:5758], now have 1 active actors on pipe
2025-10-09T21:52:20.284390Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343188981325679:5758]
2025-10-09T21:52:20.284390Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:20.284394Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:20.284420Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:20.284436Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2419 reason: error at write: Disconnected sessionId: src_id|35386aaa-f69e589c-52ebac3b-1f682e81_0
2025-10-09T21:52:20.284649Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2419 sessionId: src_id|35386aaa-f69e589c-52ebac3b-1f682e81_0 is DEAD
2025-10-09T21:52:20.284757Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35386aaa-f69e589c-52ebac3b-1f682e81_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:20.284767Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35386aaa-f69e589c-52ebac3b-1f682e81_0] Write session will restart in 0.005416s
2025-10-09T21:52:20.284786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35386aaa-f69e589c-52ebac3b-1f682e81_0] Write session: Do CDS request
2025-10-09T21:52:20.284792Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35386aaa-f69e589c-52ebac3b-1f682e81_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:20.284983Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:20.291451Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|35386aaa-f69e589c-52ebac3b-1f682e81_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:20.291532Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:20.291543Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2420
2025-10-09T21:52:20.291649Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2420 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:20.291679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2420 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:20.291692Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2420 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:20.291696Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:20.292215Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2420 sessionId:  describe result for acl check
2025-10-09T21:52:20.292248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:20.292250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:20.292254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:20.292258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343188981325720:5407] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:20.292684Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343188981325720:5407] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:20.295773Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343188981325720:5407] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:20.298738Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343188981325720:5407] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:20.298750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343188981325720:5407] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:20.298760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343188981325720:5407] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:20.298766Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2420 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:20.299741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:20.301735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:20.301888Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:20.301898Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:20.301908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:20.301956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:20.301986Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343188619472784:8695]
2025-10-09T21:52:20.302400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:20.302673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:20.302715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:20.302941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:20.302986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.302991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.302992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.302993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.302993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.302994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.302995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.302995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.302996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.302999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.303006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.303008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.303010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.303012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.303014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.303016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.303018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.303019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.303024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.303029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.303033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.303036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.303038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.303041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.303051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.303054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.303056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.303059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.303062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.303065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.303068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.303070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.303075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.303078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.303081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.303084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.303099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:20.308913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:20.309920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:20.309935Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:20.309937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:20.309941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:20.309947Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2419 [2:7559343188619472784:8695]
2025-10-09T21:52:20.309960Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:20.309974Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:20.309981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:20.309984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:20.309986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:20.309998Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:20.310003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:20.310009Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:20.310064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.310070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.310071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.310073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:20.310135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:20.310143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.310155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.310173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.310183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:20.310199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343188981325930:5407], now have 1 active actors on pipe
2025-10-09T21:52:20.310205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343188981325685:2432], now have 1 active actors on pipe
2025-10-09T21:52:20.310211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343188619472779:8695]
2025-10-09T21:52:20.310212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343188619472779:8695]
2025-10-09T21:52:20.310214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343188619472779:8695]
2025-10-09T21:52:20.310224Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:20.310239Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:20.310250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:20.310253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:20.310262Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:20.310272Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343188619472779:8695]
2025-10-09T21:52:20.310272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:20.310273Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:20.310275Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:20.310282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:20.310310Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2419
2025-10-09T21:52:20.310318Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2419
2025-10-09T21:52:20.310362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.310369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.310391Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|943e2012-21ca2516-407b26c8-37864ba4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:20.310401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:20.310404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:20.310405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:20.310413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:20.310421Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:20.310435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:20.310551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.310557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.310566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:20.310635Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2420 partition: 0 MaxSeqNo: 25 sessionId: src_id|943e2012-21ca2516-407b26c8-37864ba4_0
2025-10-09T21:52:20.310960Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|35386aaa-f69e589c-52ebac3b-1f682e81_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|943e2012-21ca2516-407b26c8-37864ba4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:20.313357Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|943e2012-21ca2516-407b26c8-37864ba4_0] Write session: try to update token
2025-10-09T21:52:20.313372Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|943e2012-21ca2516-407b26c8-37864ba4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:20.323612Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:20.323627Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:20.323629Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:20.323630Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:20.323638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343188619472779:8695]
2025-10-09T21:52:20.323641Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343188619472779:8695]
2025-10-09T21:52:20.323644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343188619472779:8695]
2025-10-09T21:52:20.323677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:20.323683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.323683Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:20.323684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:20.323689Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:20.323693Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:20.323813Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:20.326085Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2420 sessionId: src_id|943e2012-21ca2516-407b26c8-37864ba4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:20.328335Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:20.384754Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343188981325950:5758]
2025-10-09T21:52:20.385021Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 2419, pipe: [1:7559343188981325950:5758]
2025-10-09T21:52:20.466591Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:20.467374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.467395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.467447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343188981325950:5758], now have 1 active actors on pipe
2025-10-09T21:52:20.467613Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:20.467698Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:20.467708Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:20.467712Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:20.467754Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343188981325950:5758]
2025-10-09T21:52:20.467770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2420 reason: error at write: Disconnected sessionId: src_id|943e2012-21ca2516-407b26c8-37864ba4_0
2025-10-09T21:52:20.467923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2420 sessionId: src_id|943e2012-21ca2516-407b26c8-37864ba4_0 is DEAD
2025-10-09T21:52:20.468088Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|943e2012-21ca2516-407b26c8-37864ba4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:20.468099Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|943e2012-21ca2516-407b26c8-37864ba4_0] Write session will restart in 0.007410s
2025-10-09T21:52:20.468126Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|943e2012-21ca2516-407b26c8-37864ba4_0] Write session: Do CDS request
2025-10-09T21:52:20.468132Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|943e2012-21ca2516-407b26c8-37864ba4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:20.476455Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|943e2012-21ca2516-407b26c8-37864ba4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:20.476564Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:20.476575Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2421
2025-10-09T21:52:20.476712Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2421 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:20.476746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2421 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:20.476753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2421 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:20.476758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:20.477313Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2421 sessionId:  describe result for acl check
2025-10-09T21:52:20.477342Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:20.477344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:20.477345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:20.477350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343188981326026:5386] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:20.477871Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343188981326026:5386] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:20.480862Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343188981326026:5386] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:20.483574Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343188981326026:5386] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:20.483585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343188981326026:5386] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:20.483589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343188981326026:5386] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:20.483595Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2421 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:20.484493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:20.486477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:20.486644Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:20.486654Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:20.486666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:20.486721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:20.486755Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343188619473035:8693]
2025-10-09T21:52:20.487318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:20.487576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:20.487617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:20.487894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:20.487929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.487933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.487934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.487935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.487935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.487936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.487936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.487937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.487938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.487941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.487947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.487949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.487951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.487953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.487955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.487957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.487959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.487961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.487966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.487970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.487975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.487978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.487980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.487983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.487985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.487988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.487990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.487993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.487995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.487998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.488000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.488003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.488007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.488010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.488012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.488014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.488031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:20.493584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:20.494633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:20.494647Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:20.494650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:20.494653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:20.494660Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2420 [2:7559343188619473035:8693]
2025-10-09T21:52:20.494671Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:20.494687Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:20.494690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:20.494693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:20.494695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:20.494709Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:20.494717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:20.494723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:20.494769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.494777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.494779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.494782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:20.494838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:20.494846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.494856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.494867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.494882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:20.494906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343188981326199:5386], now have 1 active actors on pipe
2025-10-09T21:52:20.494913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343188981325952:2432], now have 1 active actors on pipe
2025-10-09T21:52:20.494918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343188619473030:8693]
2025-10-09T21:52:20.494920Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343188619473030:8693]
2025-10-09T21:52:20.494921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343188619473030:8693]
2025-10-09T21:52:20.494930Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:20.494942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:20.494950Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:20.494953Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:20.494956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:20.494959Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:20.494962Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:20.494943Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2420
2025-10-09T21:52:20.494946Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2420
2025-10-09T21:52:20.494983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343188619473030:8693]
2025-10-09T21:52:20.494990Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:20.494991Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:20.495004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.495010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.495055Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b81d98d4-73b617b4-7d978bc6-d93bb9e2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:20.495067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:20.495072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:20.495074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:20.495083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:20.495093Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:20.495112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:20.495233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.495241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.495255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:20.495329Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2421 partition: 0 MaxSeqNo: 25 sessionId: src_id|b81d98d4-73b617b4-7d978bc6-d93bb9e2_0
2025-10-09T21:52:20.495689Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|943e2012-21ca2516-407b26c8-37864ba4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b81d98d4-73b617b4-7d978bc6-d93bb9e2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:20.498115Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b81d98d4-73b617b4-7d978bc6-d93bb9e2_0] Write session: try to update token
2025-10-09T21:52:20.498132Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b81d98d4-73b617b4-7d978bc6-d93bb9e2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:20.510207Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:20.510224Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:20.510226Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:20.510228Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:20.510235Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343188619473030:8693]
2025-10-09T21:52:20.510239Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343188619473030:8693]
2025-10-09T21:52:20.510242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343188619473030:8693]
2025-10-09T21:52:20.510275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:20.510281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.510283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:20.510294Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:20.510300Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:20.510303Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:20.510440Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2421 sessionId: src_id|b81d98d4-73b617b4-7d978bc6-d93bb9e2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:20.510783Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:20.512667Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:20.568028Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343188981326213:5758]
2025-10-09T21:52:20.568287Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 2420, pipe: [1:7559343188981326213:5758]
2025-10-09T21:52:20.650888Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:20.651078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.651094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.651139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343188981326213:5758], now have 1 active actors on pipe
2025-10-09T21:52:20.651354Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:20.651365Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:20.651423Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2421 reason: error at write: Disconnected sessionId: src_id|b81d98d4-73b617b4-7d978bc6-d93bb9e2_0
2025-10-09T21:52:20.651487Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:20.651583Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2421 sessionId: src_id|b81d98d4-73b617b4-7d978bc6-d93bb9e2_0 is DEAD
2025-10-09T21:52:20.651751Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b81d98d4-73b617b4-7d978bc6-d93bb9e2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:20.651762Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b81d98d4-73b617b4-7d978bc6-d93bb9e2_0] Write session will restart in 0.005874s
2025-10-09T21:52:20.651744Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:20.651786Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b81d98d4-73b617b4-7d978bc6-d93bb9e2_0] Write session: Do CDS request
2025-10-09T21:52:20.651793Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b81d98d4-73b617b4-7d978bc6-d93bb9e2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:20.651785Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343188981326213:5758]
2025-10-09T21:52:20.658452Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b81d98d4-73b617b4-7d978bc6-d93bb9e2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:20.658530Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:20.658541Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2422
2025-10-09T21:52:20.658643Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2422 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:20.658667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2422 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:20.658672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2422 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:20.658675Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:20.658978Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2422 sessionId:  describe result for acl check
2025-10-09T21:52:20.659010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:20.659012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:20.659014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:20.659018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343188981326243:5340] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:20.659471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343188981326243:5340] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:20.662266Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343188981326243:5340] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:20.665941Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343188981326243:5340] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:20.665958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343188981326243:5340] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:20.665961Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343188981326243:5340] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:20.665969Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2422 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:20.669297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:20.671306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:20.671461Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:20.671470Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:20.671482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:20.671534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:20.671571Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343188619473301:8678]
2025-10-09T21:52:20.671946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:20.672241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:20.672282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:20.672614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:20.672674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.672680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.672681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.672682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.672683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.672685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.672686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.672687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.672689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.672694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.672704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.672708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.672712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.672716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.672720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.672723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.672727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.672731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.672737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.672744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.672750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.672755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.672759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.672763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.672767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.672772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.672777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.672781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.672785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.672790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.672799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.672803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.672811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.672816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.672820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.672825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.672856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:20.679558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:20.680579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:20.680588Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:20.680590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:20.680593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:20.680597Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2421 [2:7559343188619473301:8678]
2025-10-09T21:52:20.680608Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:20.680621Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:20.680623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:20.680625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:20.680626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:20.680639Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:20.680648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:20.680653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:20.680696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.680702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.680703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.680705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:20.680754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:20.680760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.680767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.680773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.680782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:20.680794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343188619473296:8678]
2025-10-09T21:52:20.680800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343188619473296:8678]
2025-10-09T21:52:20.680802Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343188619473296:8678]
2025-10-09T21:52:20.680807Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:20.680817Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:20.680817Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:20.680820Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:20.680822Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:20.680828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:20.680830Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:20.680838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343188981326219:2432], now have 1 active actors on pipe
2025-10-09T21:52:20.680841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343188981326411:5340], now have 1 active actors on pipe
2025-10-09T21:52:20.680858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343188619473296:8678]
2025-10-09T21:52:20.680866Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:20.680874Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:20.680887Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2421
2025-10-09T21:52:20.680920Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2421
2025-10-09T21:52:20.680977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.680984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.681003Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b5c5a247-3daf2500-66253be3-bef80655_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:20.681011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:20.681014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:20.681015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:20.681023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:20.681030Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:20.681042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:20.681206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.681220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.681232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:20.681308Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2422 partition: 0 MaxSeqNo: 25 sessionId: src_id|b5c5a247-3daf2500-66253be3-bef80655_0
2025-10-09T21:52:20.681638Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b81d98d4-73b617b4-7d978bc6-d93bb9e2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b5c5a247-3daf2500-66253be3-bef80655_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:20.683996Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5c5a247-3daf2500-66253be3-bef80655_0] Write session: try to update token
2025-10-09T21:52:20.684015Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5c5a247-3daf2500-66253be3-bef80655_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:20.693407Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:20.693422Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:20.693430Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343188619473296:8678]
2025-10-09T21:52:20.693478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:20.693485Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:20.693641Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:20.693778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:20.693912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:20.694070Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:20.694978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:20.695020Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:20.696085Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2422 sessionId: src_id|b5c5a247-3daf2500-66253be3-bef80655_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:20.698285Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:20.698476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.698494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.698532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:20.698611Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:20.705785Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:20.705800Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:20.705802Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:20.705804Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:20.705814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343188619473296:8678]
2025-10-09T21:52:20.705818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343188619473296:8678]
2025-10-09T21:52:20.705822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343188619473296:8678]
2025-10-09T21:52:20.705856Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:20.705861Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:20.705862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:20.705864Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:20.705868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.705871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:20.705988Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:20.751844Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343188981326499:5758]
2025-10-09T21:52:20.752071Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 2421, pipe: [1:7559343188981326499:5758]
2025-10-09T21:52:20.845321Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:20.852455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.852478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.852622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.852681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.852740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.852806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.853056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.853064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.853119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.853170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.853229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.853301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.853344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.853351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.853409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.853464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.853524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.853589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.853627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.853632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.853691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.853746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.853808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.853875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.853916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.853922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.853980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.854048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.854102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.854154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.854191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.854199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.854252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.854300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.854350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.854399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.854453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.854458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.854508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.854556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.854611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.854660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.854712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.854717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.854768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.854818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.854879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.854935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.854971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.854977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.855028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.855090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.855147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.855198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.855235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.855240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.855294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:20.855344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:20.855393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:20.855445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:20.855710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:20.855717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:20.855755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343188981326499:5758], now have 1 active actors on pipe
2025-10-09T21:52:20.856007Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:20.856010Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343188981326499:5758]
2025-10-09T21:52:20.856022Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:20.856049Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:20.856055Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2422 reason: error at write: Disconnected sessionId: src_id|b5c5a247-3daf2500-66253be3-bef80655_0
2025-10-09T21:52:20.856223Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:20.856237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2422 sessionId: src_id|b5c5a247-3daf2500-66253be3-bef80655_0 is DEAD
2025-10-09T21:52:20.856405Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5c5a247-3daf2500-66253be3-bef80655_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:20.856414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5c5a247-3daf2500-66253be3-bef80655_0] Write session will restart in 0.007453s
2025-10-09T21:52:20.856439Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5c5a247-3daf2500-66253be3-bef80655_0] Write session: Do CDS request
2025-10-09T21:52:20.856445Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5c5a247-3daf2500-66253be3-bef80655_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:20.865417Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b5c5a247-3daf2500-66253be3-bef80655_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:20.865509Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:20.865522Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2423
2025-10-09T21:52:20.865657Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2423 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:20.865690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2423 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:20.865700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2423 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:20.865704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:20.866304Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2423 sessionId:  describe result for acl check
2025-10-09T21:52:20.866339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:20.866345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:20.866347Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:20.866352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343188981326606:5328] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:20.866892Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343188981326606:5328] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:20.869736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343188981326606:5328] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:20.871038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:20.873151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:20.873347Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:20.873356Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:20.873368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:20.873419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:20.873422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343188981326606:5328] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:20.873437Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343188981326606:5328] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:20.873440Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343188981326606:5328] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:20.873447Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2423 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:20.873458Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343188619473581:8670]
2025-10-09T21:52:20.873949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:20.874348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:20.874380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:20.874583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:20.874616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.874622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.874623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.874624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.874624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.874625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.874625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.874626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.874627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.874630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.874636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.874638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.874640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.874642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.874644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.874646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.874648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.874650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.874654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.874659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:20.874663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.874666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:20.874669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.874672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:20.874674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.874677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:20.874679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.874682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:20.874685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.874688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:20.874690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.874695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.874699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.874705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.874709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.874714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.874740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:20.880683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:20.881978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:20.881992Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:20.881994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:20.881998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:20.882004Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2422 [2:7559343188619473581:8670]
2025-10-09T21:52:20.882017Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:20.882050Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:20.882054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:20.882056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:20.882059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:20.882076Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:20.882082Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:20.882089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:20.882143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.882150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.882152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.882155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:20.882220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:20.882230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.882242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:20.882253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:20.882267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:20.882295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343188981326755:5328], now have 1 active actors on pipe
2025-10-09T21:52:20.882303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343188981326504:2432], now have 1 active actors on pipe
2025-10-09T21:52:20.882310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343188619473576:8670]
2025-10-09T21:52:20.882312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343188619473576:8670]
2025-10-09T21:52:20.882314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343188619473576:8670]
2025-10-09T21:52:20.882327Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:20.882345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:20.882350Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:20.882353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:20.882355Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:20.882358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:20.882361Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:20.882354Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2422
2025-10-09T21:52:20.882386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343188619473576:8670]
2025-10-09T21:52:20.882362Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2422
2025-10-09T21:52:20.882394Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:20.882395Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:20.882411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.882420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.882472Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|48239707-371b8a82-8ae90dcb-d35b7014_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:20.882483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:20.882488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:20.882491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:20.882501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:20.882513Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:20.882535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:20.882689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:20.882700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:20.882714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:20.882813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2423 partition: 0 MaxSeqNo: 25 sessionId: src_id|48239707-371b8a82-8ae90dcb-d35b7014_0
2025-10-09T21:52:20.883140Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b5c5a247-3daf2500-66253be3-bef80655_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|48239707-371b8a82-8ae90dcb-d35b7014_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:20.883894Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:52:20.883912Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 19
2025-10-09T21:52:20.884125Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:52:20.884327Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 24 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:52:20.884361Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:52:20.885478Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48239707-371b8a82-8ae90dcb-d35b7014_0] Write session: try to update token
2025-10-09T21:52:20.885494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48239707-371b8a82-8ae90dcb-d35b7014_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:20.897305Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:20.897330Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:20.897334Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:20.897336Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:20.897350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343188619473576:8670]
2025-10-09T21:52:20.897358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343188619473576:8670]
2025-10-09T21:52:20.897363Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343188619473576:8670]
2025-10-09T21:52:20.897413Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:20.897420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:20.897425Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:20.897427Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:20.897429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:20.897432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:20.897691Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:20.898681Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2423 sessionId: src_id|48239707-371b8a82-8ae90dcb-d35b7014_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:20.901011Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:20.956167Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343188981326771:5758]
2025-10-09T21:52:20.956394Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 2422, pipe: [1:7559343188981326771:5758]
2025-10-09T21:52:21.037035Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:21.037415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.037428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.037464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343188981326771:5758], now have 1 active actors on pipe
2025-10-09T21:52:21.037624Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:21.037693Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:21.037705Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:21.037706Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343188981326771:5758]
2025-10-09T21:52:21.037725Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:21.037739Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 0
2025-10-09T21:52:21.037770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2423 reason: error at write: Disconnected sessionId: src_id|48239707-371b8a82-8ae90dcb-d35b7014_0
2025-10-09T21:52:21.037931Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2423 sessionId: src_id|48239707-371b8a82-8ae90dcb-d35b7014_0 is DEAD
2025-10-09T21:52:21.038115Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48239707-371b8a82-8ae90dcb-d35b7014_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:21.038130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48239707-371b8a82-8ae90dcb-d35b7014_0] Write session will restart in 0.007667s
2025-10-09T21:52:21.038156Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48239707-371b8a82-8ae90dcb-d35b7014_0] Write session: Do CDS request
2025-10-09T21:52:21.038162Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48239707-371b8a82-8ae90dcb-d35b7014_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:21.046513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|48239707-371b8a82-8ae90dcb-d35b7014_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:21.046617Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:21.046631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2424
2025-10-09T21:52:21.046762Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2424 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:21.046798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2424 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:21.046809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2424 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:21.046813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:21.047377Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2424 sessionId:  describe result for acl check
2025-10-09T21:52:21.047413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:21.047420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:21.047422Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:21.047427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343193276294126:5305] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:21.047949Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343193276294126:5305] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:21.050710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343193276294126:5305] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:21.053675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343193276294126:5305] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:21.053690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343193276294126:5305] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:21.053694Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343193276294126:5305] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:21.053704Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2424 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:21.054838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:21.056995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:21.057224Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:21.057241Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:21.057256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:21.057337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:21.057382Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343192914441134:8667]
2025-10-09T21:52:21.057909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:21.058194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:21.058240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:21.058490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:21.058530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.058536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.058537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.058538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.058540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.058541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.058542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.058543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.058544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.058550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.058560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.058567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.058572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.058574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.058576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.058579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.058581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.058583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.058588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.058593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.058599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.058602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.058605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.058608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.058610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.058614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.058617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.058619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.058622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.058625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.058628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.058631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.058634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.058637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.058640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.058643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.058659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:21.064825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:21.065986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:21.065999Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:21.066001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:21.066004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:21.066010Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2423 [2:7559343192914441134:8667]
2025-10-09T21:52:21.066019Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:21.066053Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:21.066061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:21.066064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:21.066066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:21.066084Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:21.066096Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:21.066108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:21.066173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.066179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.066182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.066184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:21.066253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:21.066262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.066269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.066275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.066284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:21.066301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343193276294073:2432], now have 1 active actors on pipe
2025-10-09T21:52:21.066308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343193276294322:5305], now have 1 active actors on pipe
2025-10-09T21:52:21.066315Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343192914441129:8667]
2025-10-09T21:52:21.066316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343192914441129:8667]
2025-10-09T21:52:21.066317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343192914441129:8667]
2025-10-09T21:52:21.066329Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:21.066340Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:21.066349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:21.066353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:21.066355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:21.066357Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:21.066360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:21.066376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343192914441129:8667]
2025-10-09T21:52:21.066386Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:21.066413Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:21.066397Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2423
2025-10-09T21:52:21.066396Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2423
2025-10-09T21:52:21.066725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.066738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.066797Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 47522338 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:52:21.066838Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5ddceb54-623fb33c-c054657d-561d6ffb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:21.066852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:21.066860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:21.066863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:21.066874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:21.066889Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:21.066907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:21.067043Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 25 DataSize: 47522338 UsedReserveSize: 0
2025-10-09T21:52:21.067076Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:52:21.067092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.067099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.067114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:21.067203Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2424 partition: 0 MaxSeqNo: 25 sessionId: src_id|5ddceb54-623fb33c-c054657d-561d6ffb_0
2025-10-09T21:52:21.067586Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|48239707-371b8a82-8ae90dcb-d35b7014_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5ddceb54-623fb33c-c054657d-561d6ffb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:21.070107Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ddceb54-623fb33c-c054657d-561d6ffb_0] Write session: try to update token
2025-10-09T21:52:21.070126Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ddceb54-623fb33c-c054657d-561d6ffb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:21.082804Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:21.082835Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:21.082840Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:21.082843Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:21.082856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343192914441129:8667]
2025-10-09T21:52:21.082863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343192914441129:8667]
2025-10-09T21:52:21.082870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343192914441129:8667]
2025-10-09T21:52:21.082919Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:21.082932Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:21.082932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:21.082935Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:21.082942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.082946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:21.083217Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:21.083854Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2424 sessionId: src_id|5ddceb54-623fb33c-c054657d-561d6ffb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:21.086218Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:21.137864Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343193276294339:5758]
2025-10-09T21:52:21.138150Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 2423, pipe: [1:7559343193276294339:5758]
2025-10-09T21:52:21.231483Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:21.231914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.231926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.231968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343193276294339:5758], now have 1 active actors on pipe
2025-10-09T21:52:21.232155Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:21.232238Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343193276294339:5758]
2025-10-09T21:52:21.232238Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:21.232282Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:21.232359Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2424 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5ddceb54-623fb33c-c054657d-561d6ffb_0
2025-10-09T21:52:21.232530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2424 sessionId: src_id|5ddceb54-623fb33c-c054657d-561d6ffb_0 is DEAD
2025-10-09T21:52:21.232679Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ddceb54-623fb33c-c054657d-561d6ffb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:21.232702Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ddceb54-623fb33c-c054657d-561d6ffb_0] Write session will restart in 0.007517s
2025-10-09T21:52:21.232730Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ddceb54-623fb33c-c054657d-561d6ffb_0] Write session: Do CDS request
2025-10-09T21:52:21.232737Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ddceb54-623fb33c-c054657d-561d6ffb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:21.241499Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ddceb54-623fb33c-c054657d-561d6ffb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:21.241552Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:21.241565Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2425
2025-10-09T21:52:21.241689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2425 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:21.241721Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2425 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:21.241726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2425 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:21.241731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:21.242164Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2425 sessionId:  describe result for acl check
2025-10-09T21:52:21.242192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:21.242195Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:21.242196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:21.242200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343193276294398:5294] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:21.242685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343193276294398:5294] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:21.245154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343193276294398:5294] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:21.247610Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343193276294398:5294] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:21.247623Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343193276294398:5294] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:21.247627Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343193276294398:5294] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:21.247635Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2425 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:21.249829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:21.251871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:21.252021Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:21.252029Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:21.252039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:21.252077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:21.252105Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343192914441390:8665]
2025-10-09T21:52:21.252512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:21.252736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:21.252793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:21.253093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:21.253136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.253141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.253142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.253143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.253143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.253144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.253144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.253145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.253146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.253150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.253157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.253159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.253161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.253163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.253165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.253167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.253169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.253171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.253177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.253181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.253186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.253189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.253192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.253195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.253197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.253200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.253203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.253206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.253208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.253211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.253215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.253218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.253221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.253223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.253226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.253229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.253245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:21.259534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:21.260574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:21.260586Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:21.260588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:21.260591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:21.260596Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2424 [2:7559343192914441390:8665]
2025-10-09T21:52:21.260605Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:21.260619Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:21.260620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:21.260623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:21.260624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:21.260636Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:21.260640Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:21.260647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:21.260687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.260692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.260693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.260695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:21.260744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:21.260751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.260757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.260763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.260773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:21.260790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343192914441385:8665]
2025-10-09T21:52:21.260797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343192914441385:8665]
2025-10-09T21:52:21.260799Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343192914441385:8665]
2025-10-09T21:52:21.260808Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:21.260818Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:21.260827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343193276294345:2432], now have 1 active actors on pipe
2025-10-09T21:52:21.260830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343193276294579:5294], now have 1 active actors on pipe
2025-10-09T21:52:21.260835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:21.260838Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:21.260839Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:21.260848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:21.260851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:21.260878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343192914441385:8665]
2025-10-09T21:52:21.260890Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:21.260907Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:21.260891Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2424
2025-10-09T21:52:21.260896Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2424
2025-10-09T21:52:21.260936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.260943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.260965Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e41ebdfc-6e138786-f6e79eb0-f49436d0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:21.260973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:21.260976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:21.260977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:21.260985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:21.260991Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:21.261005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:21.261117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.261123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.261133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:21.261224Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2425 partition: 0 MaxSeqNo: 25 sessionId: src_id|e41ebdfc-6e138786-f6e79eb0-f49436d0_0
2025-10-09T21:52:21.261569Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ddceb54-623fb33c-c054657d-561d6ffb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e41ebdfc-6e138786-f6e79eb0-f49436d0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:21.263989Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e41ebdfc-6e138786-f6e79eb0-f49436d0_0] Write session: try to update token
2025-10-09T21:52:21.264006Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e41ebdfc-6e138786-f6e79eb0-f49436d0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:21.272099Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:21.272112Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:21.272121Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343192914441385:8665]
2025-10-09T21:52:21.272163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:21.272163Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:21.272333Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:21.272474Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:21.272603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:21.272737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:21.273004Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:21.273050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:21.276736Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2425 sessionId: src_id|e41ebdfc-6e138786-f6e79eb0-f49436d0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:21.279093Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:21.279426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.279444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.279484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:21.279590Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:21.286140Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:21.286156Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:21.286159Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:21.286161Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:21.286173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343192914441385:8665]
2025-10-09T21:52:21.286182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343192914441385:8665]
2025-10-09T21:52:21.286187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343192914441385:8665]
2025-10-09T21:52:21.286229Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:21.286235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:21.286239Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:21.286242Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:21.286244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.286247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:21.286447Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:21.332487Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343193276294617:5758]
2025-10-09T21:52:21.332753Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 2424, pipe: [1:7559343193276294617:5758]
2025-10-09T21:52:21.431148Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:21.437924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.437944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.438093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.438143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.438197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.438260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.438310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.438320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.438384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.438436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.438490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.438548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.438589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.438599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.438662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.438717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.438774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.438832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.438878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.438885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.438948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.439233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.439288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.439341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.439384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.439390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.439450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.439514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.439571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.439631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.439670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.439675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.439735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.439798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.439856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.439909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.439947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.439953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.440007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.440054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.440108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.440170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.440216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.440223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.440275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.440322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.440369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.440418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.440459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.440464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.440514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.440572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.440627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.440679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.440715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.440722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.440782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.440842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.440898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.440948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.440985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.440990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.441029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343193276294617:5758], now have 1 active actors on pipe
2025-10-09T21:52:21.441241Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:21.441266Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343193276294617:5758]
2025-10-09T21:52:21.441302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:21.441306Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:21.441336Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2425 reason: error at write: Disconnected sessionId: src_id|e41ebdfc-6e138786-f6e79eb0-f49436d0_0
2025-10-09T21:52:21.441523Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2425 sessionId: src_id|e41ebdfc-6e138786-f6e79eb0-f49436d0_0 is DEAD
2025-10-09T21:52:21.441581Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:21.441703Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e41ebdfc-6e138786-f6e79eb0-f49436d0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:21.441713Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e41ebdfc-6e138786-f6e79eb0-f49436d0_0] Write session will restart in 0.005204s
2025-10-09T21:52:21.441737Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e41ebdfc-6e138786-f6e79eb0-f49436d0_0] Write session: Do CDS request
2025-10-09T21:52:21.441743Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e41ebdfc-6e138786-f6e79eb0-f49436d0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:21.448409Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e41ebdfc-6e138786-f6e79eb0-f49436d0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:21.448526Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:21.448539Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2426
2025-10-09T21:52:21.448641Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2426 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:21.448672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2426 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:21.448682Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2426 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:21.448688Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:21.449103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2426 sessionId:  describe result for acl check
2025-10-09T21:52:21.449136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:21.449143Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:21.449145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:21.449151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343193276294655:5286] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:21.449729Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343193276294655:5286] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:21.452495Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343193276294655:5286] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:21.455477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343193276294655:5286] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:21.455487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343193276294655:5286] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:21.455490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343193276294655:5286] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:21.455505Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2426 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:21.458123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:21.460391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:21.460534Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:21.460542Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:21.460550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:21.460587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:21.460621Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343192914441646:8658]
2025-10-09T21:52:21.460972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:21.461236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:21.461276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:21.461482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:21.461513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.461517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.461518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.461519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.461520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.461520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.461521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.461521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.461522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.461525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.461532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.461534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.461537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.461538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.461541Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.461542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.461544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.461546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.461551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.461556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.461560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.461563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.461566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.461569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.461571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.461574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.461577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.461580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.461582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.461585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.461588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.461591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.461593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.461596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.461599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.461602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.461618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:21.467562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:21.468711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:21.468723Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:21.468724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:21.468728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:21.468732Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2425 [2:7559343192914441646:8658]
2025-10-09T21:52:21.468745Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:21.468759Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:21.468761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:21.468763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:21.468765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:21.468778Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:21.468785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:21.468791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:21.468834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.468839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.468840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.468842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:21.468899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:21.468907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.468918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.468924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.468938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:21.468955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343192914441641:8658]
2025-10-09T21:52:21.468961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343192914441641:8658]
2025-10-09T21:52:21.468962Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343192914441641:8658]
2025-10-09T21:52:21.468969Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:21.468979Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:21.468980Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:21.468983Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:21.468985Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:21.468991Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:21.468994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:21.469003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343193276294621:2432], now have 1 active actors on pipe
2025-10-09T21:52:21.469006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343193276294831:5286], now have 1 active actors on pipe
2025-10-09T21:52:21.469026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343192914441641:8658]
2025-10-09T21:52:21.469035Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:21.469050Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:21.469052Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2425
2025-10-09T21:52:21.469057Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2425
2025-10-09T21:52:21.469110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.469117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.469139Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5f963989-7a2a8f35-7bd78890-d786936c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:21.469151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:21.469160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:21.469163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:21.469174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:21.469183Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:21.469192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:21.469350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.469356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.469365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:21.469459Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2426 partition: 0 MaxSeqNo: 25 sessionId: src_id|5f963989-7a2a8f35-7bd78890-d786936c_0
2025-10-09T21:52:21.469807Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e41ebdfc-6e138786-f6e79eb0-f49436d0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5f963989-7a2a8f35-7bd78890-d786936c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:21.472451Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f963989-7a2a8f35-7bd78890-d786936c_0] Write session: try to update token
2025-10-09T21:52:21.472469Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f963989-7a2a8f35-7bd78890-d786936c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:21.479816Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:21.479835Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:21.479846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343192914441641:8658]
2025-10-09T21:52:21.479894Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:21.479913Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:21.480061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:21.480196Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:21.480328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:21.480464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:21.480643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:21.480685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:21.484996Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2426 sessionId: src_id|5f963989-7a2a8f35-7bd78890-d786936c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:21.487336Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:21.487632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.487650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.487693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:21.487834Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:21.492444Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:21.492459Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:21.492462Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:21.492464Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:21.492474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343192914441641:8658]
2025-10-09T21:52:21.492479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343192914441641:8658]
2025-10-09T21:52:21.492482Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343192914441641:8658]
2025-10-09T21:52:21.492522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:21.492532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.492534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:21.492551Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:21.492569Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:21.492579Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:21.492652Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:21.541394Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343193276294887:5758]
2025-10-09T21:52:21.541772Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 2425, pipe: [1:7559343193276294887:5758]
2025-10-09T21:52:21.585394Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: CLIENT_DEADLINE_EXCEEDED. Description: [ { <main>: Error: GRpc error: (4): Deadline Exceeded } { <main>: Error: Grpc error response on endpoint localhost:14704 } ]
2025-10-09T21:52:21.635963Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:21.642441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.642463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.642616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.642674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.642731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.642790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.642828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.642836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.642892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.642947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.643000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.643056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.643093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.643099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.643149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.643200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.643252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.643303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.643351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.643363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.643428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.643479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.643532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.643587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.643624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.643631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.643694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.643751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.643814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.643871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.643908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.643915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.643969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.644020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.644075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.644128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.644166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.644172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.644225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.644273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.644326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.644382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.644426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.644435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.644489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.644539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.644604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.644665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.644706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.644711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.644765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.644824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.644878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.644928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.644964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.644970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.645025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:21.645074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:21.645121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:21.645170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:21.645220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:21.645225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:21.645264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343193276294887:5758], now have 1 active actors on pipe
2025-10-09T21:52:21.645577Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:21.645585Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:21.645588Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:21.645614Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2426 reason: error at write: Disconnected sessionId: src_id|5f963989-7a2a8f35-7bd78890-d786936c_0
2025-10-09T21:52:21.645743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2426 sessionId: src_id|5f963989-7a2a8f35-7bd78890-d786936c_0 is DEAD
2025-10-09T21:52:21.645835Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f963989-7a2a8f35-7bd78890-d786936c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:21.645847Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f963989-7a2a8f35-7bd78890-d786936c_0] Write session will restart in 0.007263s
2025-10-09T21:52:21.645873Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f963989-7a2a8f35-7bd78890-d786936c_0] Write session: Do CDS request
2025-10-09T21:52:21.645880Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f963989-7a2a8f35-7bd78890-d786936c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:21.645895Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343193276294887:5758]
2025-10-09T21:52:21.646163Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:21.654375Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f963989-7a2a8f35-7bd78890-d786936c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:21.654469Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:21.654482Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2427
2025-10-09T21:52:21.654622Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2427 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:21.654656Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2427 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:21.654666Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2427 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:21.654673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:21.655174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2427 sessionId:  describe result for acl check
2025-10-09T21:52:21.655211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:21.655217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:21.655219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:21.655224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343193276294946:5283] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:21.655772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343193276294946:5283] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:21.659295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343193276294946:5283] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:21.661758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343193276294946:5283] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:21.661769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343193276294946:5283] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:21.661772Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343193276294946:5283] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:21.661786Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2427 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:21.662281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:21.664321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:21.664502Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:21.664513Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:21.664524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:21.664576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:21.664620Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343192914441904:8656]
2025-10-09T21:52:21.665209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:21.665551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:21.665595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:21.665901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:21.665947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.665952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.665953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.665955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.665956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.665957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.665959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.665960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.665961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.665966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.665977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.665982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.665986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.665989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.665993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.665997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.666001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.666005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.666017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.666026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.666059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.666065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.666071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.666078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.666082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.666087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.666095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.666100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.666104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.666109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.666118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.666122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.666126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.666131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.666142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.666147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.666174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:21.672360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:21.673511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:21.673524Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:21.673526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:21.673531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:21.673538Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2426 [2:7559343192914441904:8656]
2025-10-09T21:52:21.673551Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:21.673568Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:21.673575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:21.673579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:21.673581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:21.673597Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:21.673606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:21.673612Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:21.673666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.673671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.673674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.673676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:21.673740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:21.673750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.673761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.673772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.673788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:21.673813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343193276295142:5283], now have 1 active actors on pipe
2025-10-09T21:52:21.673822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343193276294889:2432], now have 1 active actors on pipe
2025-10-09T21:52:21.673830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343192914441899:8656]
2025-10-09T21:52:21.673832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343192914441899:8656]
2025-10-09T21:52:21.673835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343192914441899:8656]
2025-10-09T21:52:21.673846Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:21.673859Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:21.673859Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:21.673863Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:21.673865Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:21.673873Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:21.673877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:21.673850Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2426
2025-10-09T21:52:21.673852Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2426
2025-10-09T21:52:21.673903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343192914441899:8656]
2025-10-09T21:52:21.673913Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:21.673919Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:21.673936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.673945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.673989Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2a516016-64a40f60-8f3b9ad9-6765ca8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:21.673999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:21.674003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:21.674005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:21.674015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:21.674049Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:21.674071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:21.674253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.674265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.674287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:21.674408Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2427 partition: 0 MaxSeqNo: 25 sessionId: src_id|2a516016-64a40f60-8f3b9ad9-6765ca8_0
2025-10-09T21:52:21.674714Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f963989-7a2a8f35-7bd78890-d786936c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2a516016-64a40f60-8f3b9ad9-6765ca8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:21.677309Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a516016-64a40f60-8f3b9ad9-6765ca8_0] Write session: try to update token
2025-10-09T21:52:21.677324Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a516016-64a40f60-8f3b9ad9-6765ca8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:21.687502Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:21.687522Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:21.687526Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:21.687528Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:21.687537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343192914441899:8656]
2025-10-09T21:52:21.687540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343192914441899:8656]
2025-10-09T21:52:21.687543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343192914441899:8656]
2025-10-09T21:52:21.687586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:21.687593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.687592Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:21.687597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:21.687614Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:21.687622Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:21.687720Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:21.689840Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2427 sessionId: src_id|2a516016-64a40f60-8f3b9ad9-6765ca8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:21.692033Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:21.746179Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343193276295178:5758]
2025-10-09T21:52:21.746482Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 2426, pipe: [1:7559343193276295178:5758]
2025-10-09T21:52:21.829592Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:21.829910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.829925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.829970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343193276295178:5758], now have 1 active actors on pipe
2025-10-09T21:52:21.830211Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:21.830173Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:21.830176Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343193276295178:5758]
2025-10-09T21:52:21.830192Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:21.830195Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:21.830252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2427 reason: error at write: Disconnected sessionId: src_id|2a516016-64a40f60-8f3b9ad9-6765ca8_0
2025-10-09T21:52:21.830429Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2427 sessionId: src_id|2a516016-64a40f60-8f3b9ad9-6765ca8_0 is DEAD
2025-10-09T21:52:21.830562Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a516016-64a40f60-8f3b9ad9-6765ca8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:21.830572Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a516016-64a40f60-8f3b9ad9-6765ca8_0] Write session will restart in 0.006309s
2025-10-09T21:52:21.830592Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a516016-64a40f60-8f3b9ad9-6765ca8_0] Write session: Do CDS request
2025-10-09T21:52:21.830604Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a516016-64a40f60-8f3b9ad9-6765ca8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:21.838456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2a516016-64a40f60-8f3b9ad9-6765ca8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:21.838534Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:21.838556Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2428
2025-10-09T21:52:21.838655Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2428 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:21.838679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2428 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:21.838683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2428 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:21.838685Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:21.839028Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2428 sessionId:  describe result for acl check
2025-10-09T21:52:21.839054Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:21.839058Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:21.839059Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:21.839063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343193276295239:5226] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:21.839491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343193276295239:5226] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:21.841868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343193276295239:5226] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:21.844757Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343193276295239:5226] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:21.844767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343193276295239:5226] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:21.844769Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343193276295239:5226] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:21.844775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2428 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:21.846115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:21.848181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:21.848354Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:21.848365Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:21.848377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:21.848425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:21.848463Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343192914442203:8636]
2025-10-09T21:52:21.849039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:21.849320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:21.849359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:21.849650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:21.849696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.849702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.849703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.849704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.849705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.849706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.849707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.849709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.849710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.849714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.849725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.849729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.849733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.849737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.849741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.849745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.849748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.849752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.849759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.849766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:21.849772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.849778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:21.849782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.849787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:21.849791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.849796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:21.849801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.849806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:21.849810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.849815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:21.849821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.849828Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.849835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.849843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.849848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.849856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.849881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:21.855824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:21.857016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:21.857028Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:21.857031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:21.857036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:21.857044Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2427 [2:7559343192914442203:8636]
2025-10-09T21:52:21.857053Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:21.857074Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:21.857081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:21.857084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:21.857087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:21.857104Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:21.857114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:21.857121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:21.857180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.857187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.857189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.857192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:21.857260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:21.857281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.857292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:21.857302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:21.857318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:21.857339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343193276295437:5226], now have 1 active actors on pipe
2025-10-09T21:52:21.857347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343193276295183:2432], now have 1 active actors on pipe
2025-10-09T21:52:21.857354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343192914442198:8636]
2025-10-09T21:52:21.857357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343192914442198:8636]
2025-10-09T21:52:21.857359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343192914442198:8636]
2025-10-09T21:52:21.857368Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:21.857350Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2427
2025-10-09T21:52:21.857381Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2427
2025-10-09T21:52:21.857384Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:21.857397Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:21.857399Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:21.857401Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:21.857403Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:21.857406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:21.857450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.857455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.857467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343192914442198:8636]
2025-10-09T21:52:21.857478Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:21.857485Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:21.857509Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4d0680f4-6b734f4d-44d6e2f5-ac8fdbba_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:21.857521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:21.857526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:21.857528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:21.857538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:21.857548Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:21.857574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:21.857709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:21.857717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:21.857731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:21.857827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2428 partition: 0 MaxSeqNo: 25 sessionId: src_id|4d0680f4-6b734f4d-44d6e2f5-ac8fdbba_0
2025-10-09T21:52:21.858177Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2a516016-64a40f60-8f3b9ad9-6765ca8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4d0680f4-6b734f4d-44d6e2f5-ac8fdbba_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:21.860432Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4d0680f4-6b734f4d-44d6e2f5-ac8fdbba_0] Write session: try to update token
2025-10-09T21:52:21.860450Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4d0680f4-6b734f4d-44d6e2f5-ac8fdbba_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:21.871078Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:21.871094Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:21.871097Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:21.871099Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:21.871108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343192914442198:8636]
2025-10-09T21:52:21.871115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343192914442198:8636]
2025-10-09T21:52:21.871119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343192914442198:8636]
2025-10-09T21:52:21.871162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:21.871164Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:21.871168Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:21.871171Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:21.871177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:21.871180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:21.871308Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:21.872757Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2428 sessionId: src_id|4d0680f4-6b734f4d-44d6e2f5-ac8fdbba_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:21.875068Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:21.930488Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343193276295453:5758]
2025-10-09T21:52:21.930767Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 2427, pipe: [1:7559343193276295453:5758]
2025-10-09T21:52:22.023744Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:22.024311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.024323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.024363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343193276295453:5758], now have 1 active actors on pipe
2025-10-09T21:52:22.024544Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:22.024596Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:22.024610Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:22.024642Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343193276295453:5758]
2025-10-09T21:52:22.024659Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:22.024674Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2428 reason: error at write: Disconnected sessionId: src_id|4d0680f4-6b734f4d-44d6e2f5-ac8fdbba_0
2025-10-09T21:52:22.024889Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2428 sessionId: src_id|4d0680f4-6b734f4d-44d6e2f5-ac8fdbba_0 is DEAD
2025-10-09T21:52:22.025074Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d0680f4-6b734f4d-44d6e2f5-ac8fdbba_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:22.025086Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d0680f4-6b734f4d-44d6e2f5-ac8fdbba_0] Write session will restart in 0.006000s
2025-10-09T21:52:22.025117Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d0680f4-6b734f4d-44d6e2f5-ac8fdbba_0] Write session: Do CDS request
2025-10-09T21:52:22.025122Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d0680f4-6b734f4d-44d6e2f5-ac8fdbba_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:22.032563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4d0680f4-6b734f4d-44d6e2f5-ac8fdbba_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:22.032631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:22.032657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2429
2025-10-09T21:52:22.032790Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2429 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:22.032823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2429 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:22.032834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2429 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:22.032838Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:22.033255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2429 sessionId:  describe result for acl check
2025-10-09T21:52:22.033303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:22.033310Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:22.033312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:22.033318Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343197571262792:5223] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:22.033905Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343197571262792:5223] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:22.036545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343197571262792:5223] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:22.039243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343197571262792:5223] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:22.039255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343197571262792:5223] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:22.039258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343197571262792:5223] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:22.039275Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2429 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:22.041078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:22.043521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:22.043694Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:22.043701Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:22.043712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:22.043763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:22.043805Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343197209409760:8634]
2025-10-09T21:52:22.044197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:22.044478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:22.044514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:22.044728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:22.044758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.044763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.044763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.044764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.044765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.044765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.044766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.044767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.044767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.044770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.044778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.044780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.044782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.044784Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.044786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.044788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.044790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.044792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.044797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.044802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.044806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.044810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.044812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.044815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.044818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.044821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.044826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.044829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.044831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.044835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.044838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.044841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.044847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.044851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.044853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.044856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.044872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:22.051252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:22.052388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:22.052400Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:22.052401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:22.052405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:22.052411Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2428 [2:7559343197209409760:8634]
2025-10-09T21:52:22.052420Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:22.052437Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:22.052444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:22.052446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:22.052448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:22.052461Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:22.052469Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:22.052475Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:22.052526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.052531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.052533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.052535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:22.052740Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2428
2025-10-09T21:52:22.052747Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2428
2025-10-09T21:52:22.052889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:22.052902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.052917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.052936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.052959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:22.052993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343197571262752:2432], now have 1 active actors on pipe
2025-10-09T21:52:22.053002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343197571263005:5223], now have 1 active actors on pipe
2025-10-09T21:52:22.053028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.053038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.053059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343197209409755:8634]
2025-10-09T21:52:22.053064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343197209409755:8634]
2025-10-09T21:52:22.053066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343197209409755:8634]
2025-10-09T21:52:22.053076Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:22.053089Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:22.053094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:22.053096Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:22.053097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:22.053104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:22.053107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:22.053122Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d7ad9d7a-d8f1c655-b9b673ce-8c0b12f9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:22.053132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:22.053138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:22.053141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:22.053158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:22.053173Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:22.053213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343197209409755:8634]
2025-10-09T21:52:22.053222Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:22.053228Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:22.053233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:22.053386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.053392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.053405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:22.053496Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2429 partition: 0 MaxSeqNo: 25 sessionId: src_id|d7ad9d7a-d8f1c655-b9b673ce-8c0b12f9_0
2025-10-09T21:52:22.053842Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4d0680f4-6b734f4d-44d6e2f5-ac8fdbba_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d7ad9d7a-d8f1c655-b9b673ce-8c0b12f9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:22.056235Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ad9d7a-d8f1c655-b9b673ce-8c0b12f9_0] Write session: try to update token
2025-10-09T21:52:22.056253Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ad9d7a-d8f1c655-b9b673ce-8c0b12f9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:22.067854Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:22.067871Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:22.067873Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:22.067876Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:22.067885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343197209409755:8634]
2025-10-09T21:52:22.067891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343197209409755:8634]
2025-10-09T21:52:22.067895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343197209409755:8634]
2025-10-09T21:52:22.067937Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:22.067940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:22.067943Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:22.067945Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:22.067946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.067949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:22.068082Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:22.068804Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2429 sessionId: src_id|d7ad9d7a-d8f1c655-b9b673ce-8c0b12f9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:22.071173Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:22.125049Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343197571263020:5758]
2025-10-09T21:52:22.125321Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 2428, pipe: [1:7559343197571263020:5758]
2025-10-09T21:52:22.216824Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:22.217087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.217100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.217139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343197571263020:5758], now have 1 active actors on pipe
2025-10-09T21:52:22.217331Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:22.217342Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:22.217342Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:22.217347Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:22.217403Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343197571263020:5758]
2025-10-09T21:52:22.217416Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2429 reason: error at write: Disconnected sessionId: src_id|d7ad9d7a-d8f1c655-b9b673ce-8c0b12f9_0
2025-10-09T21:52:22.217603Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2429 sessionId: src_id|d7ad9d7a-d8f1c655-b9b673ce-8c0b12f9_0 is DEAD
2025-10-09T21:52:22.217778Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ad9d7a-d8f1c655-b9b673ce-8c0b12f9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:22.217788Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ad9d7a-d8f1c655-b9b673ce-8c0b12f9_0] Write session will restart in 0.005395s
2025-10-09T21:52:22.217809Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ad9d7a-d8f1c655-b9b673ce-8c0b12f9_0] Write session: Do CDS request
2025-10-09T21:52:22.217815Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ad9d7a-d8f1c655-b9b673ce-8c0b12f9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:22.224497Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ad9d7a-d8f1c655-b9b673ce-8c0b12f9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:22.224593Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:22.224607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2430
2025-10-09T21:52:22.224743Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2430 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:22.224789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2430 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:22.224803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2430 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:22.224807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:22.225260Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2430 sessionId:  describe result for acl check
2025-10-09T21:52:22.225309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:22.225311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:22.225314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:22.225321Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343197571263044:5216] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:22.225884Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343197571263044:5216] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:22.229324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343197571263044:5216] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:22.232445Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343197571263044:5216] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:22.232459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343197571263044:5216] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:22.232463Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343197571263044:5216] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:22.232474Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2430 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:22.235167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:22.237503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:22.237676Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:22.237685Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:22.237697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:22.237747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:22.237782Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343197209410019:8628]
2025-10-09T21:52:22.238270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:22.238544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:22.238584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:22.238892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:22.238935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.238940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.238941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.238941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.238942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.238943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.238945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.238946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.238947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.238952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.238961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.238965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.238969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.238972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.238977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.238981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.238985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.238989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.238996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.239005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.239017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.239022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.239028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.239033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.239038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.239043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.239047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.239053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.239062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.239067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.239071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.239077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.239085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.239090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.239098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.239107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.239132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:22.245257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:22.246360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:22.246371Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:22.246373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:22.246376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:22.246382Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2429 [2:7559343197209410019:8628]
2025-10-09T21:52:22.246391Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:22.246404Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:22.246407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:22.246410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:22.246412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:22.246429Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:22.246435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:22.246441Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:22.246481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.246487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.246488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.246489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:22.246542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:22.246547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.246554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.246560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.246572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:22.246590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343197209410014:8628]
2025-10-09T21:52:22.246595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343197209410014:8628]
2025-10-09T21:52:22.246596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343197209410014:8628]
2025-10-09T21:52:22.246602Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:22.246611Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:22.246614Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:22.246615Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:22.246615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343197571263229:5216], now have 1 active actors on pipe
2025-10-09T21:52:22.246619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:22.246627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:22.246629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:22.246633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343197571263023:2432], now have 1 active actors on pipe
2025-10-09T21:52:22.246651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343197209410014:8628]
2025-10-09T21:52:22.246659Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:22.246668Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:22.246697Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2429
2025-10-09T21:52:22.246702Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2429
2025-10-09T21:52:22.246899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.246907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.246933Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ce38f0c1-1157a1b9-6c64cc07-bc2c82e6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:22.246946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:22.246952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:22.246954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:22.246962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:22.246973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:22.246988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:22.247120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.247127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.247137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:22.247218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2430 partition: 0 MaxSeqNo: 25 sessionId: src_id|ce38f0c1-1157a1b9-6c64cc07-bc2c82e6_0
2025-10-09T21:52:22.247584Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d7ad9d7a-d8f1c655-b9b673ce-8c0b12f9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ce38f0c1-1157a1b9-6c64cc07-bc2c82e6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:22.250001Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ce38f0c1-1157a1b9-6c64cc07-bc2c82e6_0] Write session: try to update token
2025-10-09T21:52:22.250020Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ce38f0c1-1157a1b9-6c64cc07-bc2c82e6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:22.260983Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:22.260998Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:22.261000Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:22.261001Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:22.261010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343197209410014:8628]
2025-10-09T21:52:22.261014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343197209410014:8628]
2025-10-09T21:52:22.261017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343197209410014:8628]
2025-10-09T21:52:22.261053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:22.261058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.261060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:22.261075Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:22.261081Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:22.261086Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:22.261193Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:22.262964Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2430 sessionId: src_id|ce38f0c1-1157a1b9-6c64cc07-bc2c82e6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:22.265240Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:22.317577Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343197571263295:5758]
2025-10-09T21:52:22.317872Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 2429, pipe: [1:7559343197571263295:5758]
2025-10-09T21:52:22.403605Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:22.403832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.403843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.403882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343197571263295:5758], now have 1 active actors on pipe
2025-10-09T21:52:22.404047Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:22.404132Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343197571263295:5758]
2025-10-09T21:52:22.404131Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:22.404162Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:22.404317Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2430 reason: status is not ok: tablet will be restarted right now sessionId: src_id|ce38f0c1-1157a1b9-6c64cc07-bc2c82e6_0
2025-10-09T21:52:22.404478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2430 sessionId: src_id|ce38f0c1-1157a1b9-6c64cc07-bc2c82e6_0 is DEAD
2025-10-09T21:52:22.404653Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce38f0c1-1157a1b9-6c64cc07-bc2c82e6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:22.404665Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce38f0c1-1157a1b9-6c64cc07-bc2c82e6_0] Write session will restart in 0.008008s
2025-10-09T21:52:22.404692Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce38f0c1-1157a1b9-6c64cc07-bc2c82e6_0] Write session: Do CDS request
2025-10-09T21:52:22.404699Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce38f0c1-1157a1b9-6c64cc07-bc2c82e6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:22.413526Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ce38f0c1-1157a1b9-6c64cc07-bc2c82e6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:22.413616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:22.413631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2431
2025-10-09T21:52:22.413768Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2431 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:22.413802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2431 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:22.413807Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2431 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:22.413811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:22.414252Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2431 sessionId:  describe result for acl check
2025-10-09T21:52:22.414284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:22.414291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:22.414293Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:22.414299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343197571263381:5202] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:22.414847Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343197571263381:5202] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:22.416936Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343197571263381:5202] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:22.418973Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343197571263381:5202] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:22.418986Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343197571263381:5202] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:22.418991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343197571263381:5202] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:22.418998Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2431 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:22.420245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:22.422288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:22.422472Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:22.422481Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:22.422493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:22.422547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:22.422585Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343197209410281:8615]
2025-10-09T21:52:22.423193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:22.423500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:22.423553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:22.423795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:22.423830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.423834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.423835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.423835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.423836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.423837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.423837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.423838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.423838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.423842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.423849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.423851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.423853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.423855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.423857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.423858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.423860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.423862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.423868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.423873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.423877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.423880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.423883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.423885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.423888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.423891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.423893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.423896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.423898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.423901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.423904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.423906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.423909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.423912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.423914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.423917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.423943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:22.429942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:22.430925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:22.430936Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:22.430938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:22.430941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:22.430946Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2430 [2:7559343197209410281:8615]
2025-10-09T21:52:22.430956Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:22.430971Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:22.430973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:22.430975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:22.430977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:22.430990Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:22.430999Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:22.431004Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:22.431046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.431050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.431052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.431054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:22.431209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:22.431219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.431231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.431240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.431249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:22.431269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343197571263559:5202], now have 1 active actors on pipe
2025-10-09T21:52:22.431268Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2430
2025-10-09T21:52:22.431272Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2430
2025-10-09T21:52:22.431278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343197571263303:2432], now have 1 active actors on pipe
2025-10-09T21:52:22.431285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343197209410276:8615]
2025-10-09T21:52:22.431286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343197209410276:8615]
2025-10-09T21:52:22.431288Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343197209410276:8615]
2025-10-09T21:52:22.431298Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:22.431309Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:22.431312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:22.431315Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:22.431317Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:22.431317Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:22.431319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:22.431344Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343197209410276:8615]
2025-10-09T21:52:22.431347Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:22.431349Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:22.431360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.431367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.431405Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5f1ca40a-f55194cd-db923c30-681f349a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:22.431417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:22.431420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:22.431421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:22.431429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:22.431437Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:22.431452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:22.431600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.431608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.431620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:22.431697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2431 partition: 0 MaxSeqNo: 25 sessionId: src_id|5f1ca40a-f55194cd-db923c30-681f349a_0
2025-10-09T21:52:22.432017Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ce38f0c1-1157a1b9-6c64cc07-bc2c82e6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5f1ca40a-f55194cd-db923c30-681f349a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:22.434620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f1ca40a-f55194cd-db923c30-681f349a_0] Write session: try to update token
2025-10-09T21:52:22.434638Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f1ca40a-f55194cd-db923c30-681f349a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:22.446633Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:22.446647Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:22.446649Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:22.446651Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:22.446658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343197209410276:8615]
2025-10-09T21:52:22.446662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343197209410276:8615]
2025-10-09T21:52:22.446665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343197209410276:8615]
2025-10-09T21:52:22.446701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:22.446707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.446709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:22.446713Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:22.446719Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:22.446723Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:22.446832Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:22.448114Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2431 sessionId: src_id|5f1ca40a-f55194cd-db923c30-681f349a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:22.450462Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:22.504422Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343197571263574:5758]
2025-10-09T21:52:22.504639Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 2430, pipe: [1:7559343197571263574:5758]
2025-10-09T21:52:22.585946Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:22.586502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.586515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.586550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343197571263574:5758], now have 1 active actors on pipe
2025-10-09T21:52:22.586726Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:22.586785Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:22.586798Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343197571263574:5758]
2025-10-09T21:52:22.586820Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:22.586878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2431 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5f1ca40a-f55194cd-db923c30-681f349a_0
2025-10-09T21:52:22.587062Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2431 sessionId: src_id|5f1ca40a-f55194cd-db923c30-681f349a_0 is DEAD
2025-10-09T21:52:22.587236Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f1ca40a-f55194cd-db923c30-681f349a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:22.587248Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f1ca40a-f55194cd-db923c30-681f349a_0] Write session will restart in 0.006422s
2025-10-09T21:52:22.587286Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f1ca40a-f55194cd-db923c30-681f349a_0] Write session: Do CDS request
2025-10-09T21:52:22.587292Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f1ca40a-f55194cd-db923c30-681f349a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:22.594414Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f1ca40a-f55194cd-db923c30-681f349a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:22.594499Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:22.594513Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2432
2025-10-09T21:52:22.594654Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2432 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:22.594687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2432 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:22.594698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2432 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:22.594705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:22.595135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2432 sessionId:  describe result for acl check
2025-10-09T21:52:22.595169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:22.595171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:22.595173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:22.595178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343197571263600:5183] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:22.595697Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343197571263600:5183] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:22.599202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343197571263600:5183] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:22.602375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343197571263600:5183] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:22.602387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343197571263600:5183] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:22.602391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343197571263600:5183] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:22.602398Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2432 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:22.605906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:22.609462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:22.609665Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:22.609679Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:22.609689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:22.609735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:22.609787Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343197209410543:8623]
2025-10-09T21:52:22.610475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:22.610827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:22.610868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:22.611208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:22.611268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.611275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.611277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.611278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.611280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.611281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.611283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.611284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.611286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.611292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.611302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.611307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.611312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.611316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.611321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.611331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.611335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.611339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.611347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.611360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.611372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.611379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.611383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.611389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.611394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.611406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.611412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.611423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.611428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.611440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.611446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.611456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.611461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.611467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.611472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.611485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.611516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:22.618608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:22.620422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:22.620436Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:22.620438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:22.620440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:22.620449Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2431 [2:7559343197209410543:8623]
2025-10-09T21:52:22.620465Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:22.620479Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:22.620481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:22.620484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:22.620486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:22.620500Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:22.620509Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:22.620514Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:22.620572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.620578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.620580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.620581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:22.620638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:22.620646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.620654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.620661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.620670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:22.620684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343197209410538:8623]
2025-10-09T21:52:22.620685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343197209410538:8623]
2025-10-09T21:52:22.620687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343197209410538:8623]
2025-10-09T21:52:22.620693Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:22.620704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343197571263576:2432], now have 1 active actors on pipe
2025-10-09T21:52:22.620705Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:22.620709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343197571263780:5183], now have 1 active actors on pipe
2025-10-09T21:52:22.620710Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:22.620713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:22.620714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:22.620722Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:22.620724Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:22.620757Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343197209410538:8623]
2025-10-09T21:52:22.620768Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:22.620776Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:22.620792Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2431
2025-10-09T21:52:22.620802Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2431
2025-10-09T21:52:22.620912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.620927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.620960Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b1189743-76949471-4fd1ae0c-6fef32b0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:22.620973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:22.620976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:22.620978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:22.620991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:22.621006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:22.621023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:22.621203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.621228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.621249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:22.621405Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2432 partition: 0 MaxSeqNo: 25 sessionId: src_id|b1189743-76949471-4fd1ae0c-6fef32b0_0
2025-10-09T21:52:22.621737Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f1ca40a-f55194cd-db923c30-681f349a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b1189743-76949471-4fd1ae0c-6fef32b0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:22.624038Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1189743-76949471-4fd1ae0c-6fef32b0_0] Write session: try to update token
2025-10-09T21:52:22.624057Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1189743-76949471-4fd1ae0c-6fef32b0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:22.632932Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:22.632950Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:22.632958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343197209410538:8623]
2025-10-09T21:52:22.632996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:22.633008Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:22.633170Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:22.633341Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:22.633487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:22.633637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:22.633846Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:22.633882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:22.636143Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2432 sessionId: src_id|b1189743-76949471-4fd1ae0c-6fef32b0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:22.638385Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:22.638796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.638822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.638850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:22.638941Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:22.646294Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:22.646313Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:22.646315Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:22.646317Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:22.646328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343197209410538:8623]
2025-10-09T21:52:22.646332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343197209410538:8623]
2025-10-09T21:52:22.646336Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343197209410538:8623]
2025-10-09T21:52:22.646372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:22.646379Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:22.646382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.646384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:22.646385Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:22.646388Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:22.646524Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:22.687247Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343197571263848:5758]
2025-10-09T21:52:22.687599Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 2431, pipe: [1:7559343197571263848:5758]
2025-10-09T21:52:22.789261Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:22.795699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.795725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.795872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:22.795930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:22.795985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:22.796041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:22.796369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:22.796385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:22.796439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:22.796489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:22.796541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:22.796596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:22.796634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:22.796638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:22.796690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:22.796742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:22.796795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:22.796846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:22.796888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:22.796895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:22.796950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:22.797009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:22.797062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:22.797115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:22.797152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:22.797159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:22.797212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:22.797264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:22.797327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:22.797378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:22.797419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:22.797427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:22.797480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:22.797529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:22.797581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:22.797633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:22.797671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:22.797678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:22.797730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:22.797782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:22.797836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:22.797886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:22.797923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:22.797929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:22.797978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:22.798043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:22.798097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:22.798147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:22.798189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:22.798196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:22.798246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:22.798297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:22.798350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:22.798406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:22.798445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:22.798451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:22.798504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:22.798554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:22.798605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:22.798660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:22.798697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:22.798703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:22.798741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343197571263848:5758], now have 1 active actors on pipe
2025-10-09T21:52:22.798989Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343197571263848:5758]
2025-10-09T21:52:22.799022Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:22.799024Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:22.799038Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:22.799066Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2432 reason: error at write: Disconnected sessionId: src_id|b1189743-76949471-4fd1ae0c-6fef32b0_0
2025-10-09T21:52:22.799164Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:22.799237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2432 sessionId: src_id|b1189743-76949471-4fd1ae0c-6fef32b0_0 is DEAD
2025-10-09T21:52:22.799470Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1189743-76949471-4fd1ae0c-6fef32b0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:22.799486Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1189743-76949471-4fd1ae0c-6fef32b0_0] Write session will restart in 0.008460s
2025-10-09T21:52:22.799512Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1189743-76949471-4fd1ae0c-6fef32b0_0] Write session: Do CDS request
2025-10-09T21:52:22.799519Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1189743-76949471-4fd1ae0c-6fef32b0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:22.809511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b1189743-76949471-4fd1ae0c-6fef32b0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:22.809581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:22.809597Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2433
2025-10-09T21:52:22.809785Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2433 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:22.809828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2433 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:22.809835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2433 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:22.809840Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:22.810544Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2433 sessionId:  describe result for acl check
2025-10-09T21:52:22.810590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:22.810598Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:22.810600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:22.810607Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343197571263934:5157] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:22.811163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343197571263934:5157] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:22.814100Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343197571263934:5157] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:22.816458Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343197571263934:5157] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:22.816470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343197571263934:5157] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:22.816473Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343197571263934:5157] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:22.816481Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2433 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:22.818706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:22.821967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:22.822172Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:22.822182Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:22.822192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:22.822241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:22.822280Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343197209410848:8590]
2025-10-09T21:52:22.822921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:22.823250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:22.823292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:22.823618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:22.823666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.823672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.823673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.823675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.823676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.823677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.823678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.823680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.823681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.823686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.823697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.823702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.823706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.823710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.823714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.823718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.823722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.823726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.823738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.823750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:22.823761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.823772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:22.823781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.823791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:22.823797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.823808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:22.823813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.823825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:22.823830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.823836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:22.823843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.823848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.823853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.823859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.823864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.823869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.823898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:22.830511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:22.831957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:22.831970Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:22.831973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:22.831977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:22.831984Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2432 [2:7559343197209410848:8590]
2025-10-09T21:52:22.832002Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:22.832022Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:22.832029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:22.832032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:22.832034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:22.832050Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:22.832061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:22.832072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:22.832137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.832144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.832147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.832149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:22.832222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:22.832232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.832244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:22.832258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:22.832276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:22.832297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343197571263874:2432], now have 1 active actors on pipe
2025-10-09T21:52:22.832306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343197571264113:5157], now have 1 active actors on pipe
2025-10-09T21:52:22.832314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343197209410839:8590]
2025-10-09T21:52:22.832317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343197209410839:8590]
2025-10-09T21:52:22.832320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343197209410839:8590]
2025-10-09T21:52:22.832334Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:22.832354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:22.832353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:22.832357Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:22.832359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:22.832368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:22.832371Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:22.832379Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2432
2025-10-09T21:52:22.832399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343197209410839:8590]
2025-10-09T21:52:22.832377Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2432
2025-10-09T21:52:22.832410Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:22.832438Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:22.832446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.832452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.832495Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dd059542-158cf45a-8e8aec31-a0f21dcd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:22.832507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:22.832512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:22.832514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:22.832523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:22.832534Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:22.832546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:22.832699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.832708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.832721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:22.832811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2433 partition: 0 MaxSeqNo: 25 sessionId: src_id|dd059542-158cf45a-8e8aec31-a0f21dcd_0
2025-10-09T21:52:22.833216Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b1189743-76949471-4fd1ae0c-6fef32b0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dd059542-158cf45a-8e8aec31-a0f21dcd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:22.835653Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dd059542-158cf45a-8e8aec31-a0f21dcd_0] Write session: try to update token
2025-10-09T21:52:22.835674Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dd059542-158cf45a-8e8aec31-a0f21dcd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:22.848139Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2433 sessionId: src_id|dd059542-158cf45a-8e8aec31-a0f21dcd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:22.848882Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:22.848902Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:22.848904Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:22.848906Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:22.848915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343197209410839:8590]
2025-10-09T21:52:22.848919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343197209410839:8590]
2025-10-09T21:52:22.848925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343197209410839:8590]
2025-10-09T21:52:22.848970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:22.848976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:22.848979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:22.849137Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:22.849146Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:22.849169Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:22.849175Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:22.850815Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:22.899234Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343197571264144:5758]
2025-10-09T21:52:22.899543Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 2432, pipe: [1:7559343197571264144:5758]
2025-10-09T21:52:22.994569Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:22.994794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:22.994806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:22.994846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343197571264144:5758], now have 1 active actors on pipe
2025-10-09T21:52:22.995044Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:22.995113Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:22.995130Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:22.995240Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2433 reason: error at write: Disconnected sessionId: src_id|dd059542-158cf45a-8e8aec31-a0f21dcd_0
2025-10-09T21:52:22.995458Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2433 sessionId: src_id|dd059542-158cf45a-8e8aec31-a0f21dcd_0 is DEAD
2025-10-09T21:52:22.995521Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343197571264144:5758]
2025-10-09T21:52:22.995556Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:22.995625Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd059542-158cf45a-8e8aec31-a0f21dcd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:22.995636Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd059542-158cf45a-8e8aec31-a0f21dcd_0] Write session will restart in 0.008657s
2025-10-09T21:52:22.995658Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd059542-158cf45a-8e8aec31-a0f21dcd_0] Write session: Do CDS request
2025-10-09T21:52:22.995663Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd059542-158cf45a-8e8aec31-a0f21dcd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:23.005568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dd059542-158cf45a-8e8aec31-a0f21dcd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:23.005648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:23.005664Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2434
2025-10-09T21:52:23.005811Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2434 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:23.005857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2434 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:23.005865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2434 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:23.005869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:23.006422Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2434 sessionId:  describe result for acl check
2025-10-09T21:52:23.006465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:23.006468Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:23.006469Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:23.006475Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343201866231502:5144] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:23.007110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343201866231502:5144] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:23.009599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343201866231502:5144] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:23.012726Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343201866231502:5144] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:23.012739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343201866231502:5144] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:23.012743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343201866231502:5144] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:23.012751Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2434 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:23.014962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:23.017697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:23.017867Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:23.017878Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:23.017886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:23.017925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:23.017963Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343201504378409:8585]
2025-10-09T21:52:23.018531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:23.018893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:23.018955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:23.019260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:23.019321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.019328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.019330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.019331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.019332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.019334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.019335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.019336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.019338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.019344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.019354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.019359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.019363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.019365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.019368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.019370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.019372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.019375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.019380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.019385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.019389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.019392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.019396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.019400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.019403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.019406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.019409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.019412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.019415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.019418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.019422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.019426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.019428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.019432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.019435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.019438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.019466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:23.026314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:23.027723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:23.027740Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:23.027743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:23.027747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:23.027753Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2433 [2:7559343201504378409:8585]
2025-10-09T21:52:23.027766Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:23.027781Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:23.027783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:23.027786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:23.027788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:23.027800Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:23.027804Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:23.027809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:23.027869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.027876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.027877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.027879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:23.027936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:23.027943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.027954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.027961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.027975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:23.027998Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343201504378404:8585]
2025-10-09T21:52:23.028004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343201504378404:8585]
2025-10-09T21:52:23.028006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343201504378404:8585]
2025-10-09T21:52:23.028013Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:23.028025Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:23.028028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343197571264151:2432], now have 1 active actors on pipe
2025-10-09T21:52:23.028032Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:23.028035Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:23.028038Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:23.028041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:23.028043Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:23.028048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343201866231701:5144], now have 1 active actors on pipe
2025-10-09T21:52:23.028075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343201504378404:8585]
2025-10-09T21:52:23.028085Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:23.028097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:23.028136Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2433
2025-10-09T21:52:23.028138Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2433
2025-10-09T21:52:23.028211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.028221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.028243Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c48e9b06-3b53b8fc-b6776224-749b0e3a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:23.028251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:23.028255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:23.028258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:23.028268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:23.028276Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:23.028286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:23.028459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.028467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.028480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:23.028620Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2434 partition: 0 MaxSeqNo: 25 sessionId: src_id|c48e9b06-3b53b8fc-b6776224-749b0e3a_0
2025-10-09T21:52:23.028978Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dd059542-158cf45a-8e8aec31-a0f21dcd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c48e9b06-3b53b8fc-b6776224-749b0e3a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:23.031458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c48e9b06-3b53b8fc-b6776224-749b0e3a_0] Write session: try to update token
2025-10-09T21:52:23.031480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c48e9b06-3b53b8fc-b6776224-749b0e3a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:23.042916Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:23.042933Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:23.042935Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:23.042937Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:23.042945Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343201504378404:8585]
2025-10-09T21:52:23.042949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343201504378404:8585]
2025-10-09T21:52:23.042953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343201504378404:8585]
2025-10-09T21:52:23.042988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:23.042995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.042997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:23.043005Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:23.043012Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:23.043016Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:23.043164Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:23.044146Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2434 sessionId: src_id|c48e9b06-3b53b8fc-b6776224-749b0e3a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:23.046578Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:23.095806Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343201866231718:5758]
2025-10-09T21:52:23.096178Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 2433, pipe: [1:7559343201866231718:5758]
2025-10-09T21:52:23.192067Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:23.192326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.192342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.192383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343201866231718:5758], now have 1 active actors on pipe
2025-10-09T21:52:23.192591Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:23.192687Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:23.192746Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343201866231718:5758]
2025-10-09T21:52:23.192747Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:23.192830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2434 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c48e9b06-3b53b8fc-b6776224-749b0e3a_0
2025-10-09T21:52:23.192995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2434 sessionId: src_id|c48e9b06-3b53b8fc-b6776224-749b0e3a_0 is DEAD
2025-10-09T21:52:23.193137Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c48e9b06-3b53b8fc-b6776224-749b0e3a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:23.193148Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c48e9b06-3b53b8fc-b6776224-749b0e3a_0] Write session will restart in 0.009838s
2025-10-09T21:52:23.193168Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c48e9b06-3b53b8fc-b6776224-749b0e3a_0] Write session: Do CDS request
2025-10-09T21:52:23.193173Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c48e9b06-3b53b8fc-b6776224-749b0e3a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:23.204585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c48e9b06-3b53b8fc-b6776224-749b0e3a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:23.204623Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:23.204639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2435
2025-10-09T21:52:23.204803Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2435 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:23.204836Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2435 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:23.204847Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2435 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:23.204851Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:23.205260Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2435 sessionId:  describe result for acl check
2025-10-09T21:52:23.205303Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:23.205308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:23.205309Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:23.205314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343201866231797:5113] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:23.205770Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343201866231797:5113] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:23.208176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343201866231797:5113] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:23.210940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343201866231797:5113] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:23.210954Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343201866231797:5113] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:23.210958Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343201866231797:5113] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:23.210968Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2435 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:23.213422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:23.216220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:23.216442Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:23.216457Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:23.216471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:23.216555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:23.216608Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343201504378673:8578]
2025-10-09T21:52:23.217208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:23.217668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:23.217735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:23.218376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:23.218454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.218465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.218466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.218467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.218468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.218470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.218471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.218473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.218474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.218480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.218492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.218496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.218500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.218505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.218509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.218513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.218517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.218520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.218529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.218539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.218546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.218552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.218558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.218563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.218568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.218574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.218579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.218584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.218589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.218595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.218600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.218606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.218610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.218616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.218622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.218626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.218656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:23.226292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:23.227411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:23.227432Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:23.227435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:23.227440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:23.227449Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2434 [2:7559343201504378673:8578]
2025-10-09T21:52:23.227466Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:23.227486Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:23.227490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:23.227493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:23.227496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:23.227515Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:23.227529Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:23.227537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:23.227601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.227611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.227614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.227617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:23.227677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:23.227684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.227695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.227707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.227725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:23.227744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343201504378668:8578]
2025-10-09T21:52:23.227751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343201504378668:8578]
2025-10-09T21:52:23.227753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343201504378668:8578]
2025-10-09T21:52:23.227762Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:23.227775Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:23.227776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343201866231973:5113], now have 1 active actors on pipe
2025-10-09T21:52:23.227779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343201866231721:2432], now have 1 active actors on pipe
2025-10-09T21:52:23.227787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:23.227792Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:23.227794Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:23.227801Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:23.227805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:23.227842Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343201504378668:8578]
2025-10-09T21:52:23.227855Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:23.227862Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2434
2025-10-09T21:52:23.227893Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:23.227873Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2434
2025-10-09T21:52:23.228394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.228410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.228439Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|417d48fa-95c48c78-43c60cb7-f928316b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:23.228456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:23.228460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:23.228462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:23.228475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:23.228491Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:23.228506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:23.228666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.228675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.228687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:23.228808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2435 partition: 0 MaxSeqNo: 25 sessionId: src_id|417d48fa-95c48c78-43c60cb7-f928316b_0
2025-10-09T21:52:23.229132Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c48e9b06-3b53b8fc-b6776224-749b0e3a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|417d48fa-95c48c78-43c60cb7-f928316b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:23.231454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|417d48fa-95c48c78-43c60cb7-f928316b_0] Write session: try to update token
2025-10-09T21:52:23.231470Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|417d48fa-95c48c78-43c60cb7-f928316b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:23.244103Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2435 sessionId: src_id|417d48fa-95c48c78-43c60cb7-f928316b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:23.246459Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:23.246788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.246809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.246832Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:23.246835Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:23.246844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343201504378668:8578]
2025-10-09T21:52:23.246876Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:23.246891Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:23.247037Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:23.247171Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:23.247302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:23.247436Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:23.247611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:23.247657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:23.247683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:23.247750Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:23.254903Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:23.254919Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:23.254922Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:23.254924Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:23.254933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343201504378668:8578]
2025-10-09T21:52:23.254937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343201504378668:8578]
2025-10-09T21:52:23.254940Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343201504378668:8578]
2025-10-09T21:52:23.254979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:23.254982Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:23.254988Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:23.254992Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:23.254992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.254996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:23.255122Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:23.292921Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343201866231998:5758]
2025-10-09T21:52:23.293184Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 2434, pipe: [1:7559343201866231998:5758]
2025-10-09T21:52:23.397127Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:23.403809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.403835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.403987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:23.404042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:23.404094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:23.404156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:23.404194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:23.404201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:23.404256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:23.404320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:23.404381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:23.404444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:23.404488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:23.404496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:23.404550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:23.404602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:23.404661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:23.404720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:23.404756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:23.404763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:23.404816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:23.404871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:23.404926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:23.404983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:23.405021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:23.405027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:23.405082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:23.405138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:23.405192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:23.405252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:23.405301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:23.405307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:23.405363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:23.405420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:23.405469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:23.405520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:23.405556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:23.405561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:23.405612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:23.405670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:23.405722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:23.405771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:23.405807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:23.405812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:23.405863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:23.405918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:23.405968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:23.406019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:23.406072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:23.406078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:23.406134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:23.406190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:23.406243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:23.406294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:23.406329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:23.406336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:23.406389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:23.406442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:23.406493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:23.406543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:23.406583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:23.406588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:23.406628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343201866231998:5758], now have 1 active actors on pipe
2025-10-09T21:52:23.406940Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343201866231998:5758]
2025-10-09T21:52:23.406969Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:23.407080Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:23.407090Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:23.407119Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2435 reason: error at write: Disconnected sessionId: src_id|417d48fa-95c48c78-43c60cb7-f928316b_0
2025-10-09T21:52:23.407309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2435 sessionId: src_id|417d48fa-95c48c78-43c60cb7-f928316b_0 is DEAD
2025-10-09T21:52:23.407387Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:23.407430Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|417d48fa-95c48c78-43c60cb7-f928316b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:23.407440Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|417d48fa-95c48c78-43c60cb7-f928316b_0] Write session will restart in 0.008956s
2025-10-09T21:52:23.407459Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|417d48fa-95c48c78-43c60cb7-f928316b_0] Write session: Do CDS request
2025-10-09T21:52:23.407463Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|417d48fa-95c48c78-43c60cb7-f928316b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:23.417512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|417d48fa-95c48c78-43c60cb7-f928316b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:23.417592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:23.417607Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2436
2025-10-09T21:52:23.417724Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2436 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:23.417759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2436 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:23.417767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2436 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:23.417770Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:23.418266Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2436 sessionId:  describe result for acl check
2025-10-09T21:52:23.418305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:23.418312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:23.418314Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:23.418320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343201866232066:5100] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:23.418863Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343201866232066:5100] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:23.422514Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343201866232066:5100] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:23.426072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343201866232066:5100] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:23.426085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343201866232066:5100] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:23.426088Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343201866232066:5100] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:23.426101Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2436 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:23.427939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:23.431544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:23.431727Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:23.431737Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:23.431748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:23.431794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:23.431834Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343201504378940:8572]
2025-10-09T21:52:23.432411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:23.432775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:23.432818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:23.433125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:23.433169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.433174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.433175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.433177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.433178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.433179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.433181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.433182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.433183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.433187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.433199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.433203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.433207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.433210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.433214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.433218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.433222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.433226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.433233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.433240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.433251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.433256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.433261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.433281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.433286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.433291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.433296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.433301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.433305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.433310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.433315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.433323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.433327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.433332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.433337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.433343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.433374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:23.440098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:23.441239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:23.441254Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:23.441255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:23.441259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:23.441278Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2435 [2:7559343201504378940:8572]
2025-10-09T21:52:23.441288Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:23.441307Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:23.441310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:23.441314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:23.441316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:23.441332Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:23.441339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:23.441346Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:23.441405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.441414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.441416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.441419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:23.441481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:23.441487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.441494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.441501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.441514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:23.441537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343201504378935:8572]
2025-10-09T21:52:23.441545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343201504378935:8572]
2025-10-09T21:52:23.441547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343201504378935:8572]
2025-10-09T21:52:23.441562Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:23.441577Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:23.441579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343201866232267:5100], now have 1 active actors on pipe
2025-10-09T21:52:23.441582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343201866232006:2432], now have 1 active actors on pipe
2025-10-09T21:52:23.441585Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:23.441587Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:23.441591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:23.441606Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:23.441609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:23.441608Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2435
2025-10-09T21:52:23.441639Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343201504378935:8572]
2025-10-09T21:52:23.441648Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:23.441618Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2435
2025-10-09T21:52:23.441651Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:23.441665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.441674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.441726Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|82fd3335-ce48bc0a-c4ed6458-2fb7f061_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:23.441738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:23.441743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:23.441745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:23.441756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:23.441767Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:23.441790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:23.442018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.442056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.442080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:23.442190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2436 partition: 0 MaxSeqNo: 25 sessionId: src_id|82fd3335-ce48bc0a-c4ed6458-2fb7f061_0
2025-10-09T21:52:23.442513Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|417d48fa-95c48c78-43c60cb7-f928316b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|82fd3335-ce48bc0a-c4ed6458-2fb7f061_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:23.444870Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|82fd3335-ce48bc0a-c4ed6458-2fb7f061_0] Write session: try to update token
2025-10-09T21:52:23.444884Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|82fd3335-ce48bc0a-c4ed6458-2fb7f061_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:23.456861Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2436 sessionId: src_id|82fd3335-ce48bc0a-c4ed6458-2fb7f061_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:23.457118Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:23.457141Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:23.457142Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:23.457144Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:23.457150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343201504378935:8572]
2025-10-09T21:52:23.457153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343201504378935:8572]
2025-10-09T21:52:23.457156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343201504378935:8572]
2025-10-09T21:52:23.457190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:23.457191Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:23.457196Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:23.457196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.457198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:23.457198Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:23.457346Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:23.459065Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:23.506892Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343201866232282:5758]
2025-10-09T21:52:23.507165Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 2435, pipe: [1:7559343201866232282:5758]
2025-10-09T21:52:23.595640Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:23.595891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.595905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.595945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343201866232282:5758], now have 1 active actors on pipe
2025-10-09T21:52:23.596113Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:23.596148Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343201866232282:5758]
2025-10-09T21:52:23.596163Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:23.596178Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:23.596229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2436 reason: status is not ok: tablet will be restarted right now sessionId: src_id|82fd3335-ce48bc0a-c4ed6458-2fb7f061_0
2025-10-09T21:52:23.596385Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2436 sessionId: src_id|82fd3335-ce48bc0a-c4ed6458-2fb7f061_0 is DEAD
2025-10-09T21:52:23.596549Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82fd3335-ce48bc0a-c4ed6458-2fb7f061_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:23.596560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82fd3335-ce48bc0a-c4ed6458-2fb7f061_0] Write session will restart in 0.007016s
2025-10-09T21:52:23.596596Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82fd3335-ce48bc0a-c4ed6458-2fb7f061_0] Write session: Do CDS request
2025-10-09T21:52:23.596604Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82fd3335-ce48bc0a-c4ed6458-2fb7f061_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:23.604499Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:23.604532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|82fd3335-ce48bc0a-c4ed6458-2fb7f061_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:23.604510Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2437
2025-10-09T21:52:23.604764Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2437 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:23.604792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2437 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:23.604804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2437 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:23.604808Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:23.605323Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2437 sessionId:  describe result for acl check
2025-10-09T21:52:23.605356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:23.605362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:23.605363Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:23.605369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343201866232339:5089] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:23.605837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343201866232339:5089] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:23.608869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343201866232339:5089] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:23.611165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343201866232339:5089] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:23.611174Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343201866232339:5089] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:23.611178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343201866232339:5089] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:23.611192Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2437 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:23.613635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:23.616358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:23.616548Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:23.616557Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:23.616568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:23.616624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:23.616667Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343201504379207:8569]
2025-10-09T21:52:23.617224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:23.617521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:23.617565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:23.617863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:23.617913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.617918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.617919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.617920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.617921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.617922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.617924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.617925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.617926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.617930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.617942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.617946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.617950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.617953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.617957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.617961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.617964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.617968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.617975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.617983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.617993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.617998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.618003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.618008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.618014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.618019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.618023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.618043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.618047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.618053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.618059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.618064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.618070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.618075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.618080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.618084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.618111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:23.624487Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:23.625482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:23.625495Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:23.625497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:23.625499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:23.625504Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2436 [2:7559343201504379207:8569]
2025-10-09T21:52:23.625513Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:23.625527Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:23.625529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:23.625531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:23.625533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:23.625544Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:23.625548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:23.625554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:23.625592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.625598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.625600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.625602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:23.625657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:23.625665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.625676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.625687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.625703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:23.625724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343201866232284:2432], now have 1 active actors on pipe
2025-10-09T21:52:23.625727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343201866232505:5089], now have 1 active actors on pipe
2025-10-09T21:52:23.625735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343201504379202:8569]
2025-10-09T21:52:23.625742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343201504379202:8569]
2025-10-09T21:52:23.625744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343201504379202:8569]
2025-10-09T21:52:23.625759Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:23.625776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:23.625782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:23.625785Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:23.625787Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:23.625787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:23.625789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:23.625772Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2436
2025-10-09T21:52:23.625789Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2436
2025-10-09T21:52:23.625808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343201504379202:8569]
2025-10-09T21:52:23.625815Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:23.625818Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:23.625840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.625849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.625881Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ea270abb-8893e534-e17bea2a-5b2826c2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:23.625891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:23.625893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:23.625896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:23.625905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:23.625919Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:23.625941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:23.626067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.626075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.626088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:23.626168Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2437 partition: 0 MaxSeqNo: 25 sessionId: src_id|ea270abb-8893e534-e17bea2a-5b2826c2_0
2025-10-09T21:52:23.626504Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|82fd3335-ce48bc0a-c4ed6458-2fb7f061_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ea270abb-8893e534-e17bea2a-5b2826c2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:23.628835Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea270abb-8893e534-e17bea2a-5b2826c2_0] Write session: try to update token
2025-10-09T21:52:23.628853Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea270abb-8893e534-e17bea2a-5b2826c2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:23.641352Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2437 sessionId: src_id|ea270abb-8893e534-e17bea2a-5b2826c2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:23.641813Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:23.641830Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:23.641831Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:23.641833Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:23.641840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343201504379202:8569]
2025-10-09T21:52:23.641844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343201504379202:8569]
2025-10-09T21:52:23.641848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343201504379202:8569]
2025-10-09T21:52:23.641882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:23.641887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.641889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:23.641987Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:23.641997Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:23.642000Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:23.642020Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:23.643635Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:23.696383Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343201866232561:5758]
2025-10-09T21:52:23.696622Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 2436, pipe: [1:7559343201866232561:5758]
2025-10-09T21:52:23.778570Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:23.779005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.779019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.779056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343201866232561:5758], now have 1 active actors on pipe
2025-10-09T21:52:23.779260Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:23.779307Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:23.779313Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:23.779372Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2437 reason: error at write: Disconnected sessionId: src_id|ea270abb-8893e534-e17bea2a-5b2826c2_0
2025-10-09T21:52:23.779437Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:23.779539Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2437 sessionId: src_id|ea270abb-8893e534-e17bea2a-5b2826c2_0 is DEAD
2025-10-09T21:52:23.779673Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea270abb-8893e534-e17bea2a-5b2826c2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:23.779683Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea270abb-8893e534-e17bea2a-5b2826c2_0] Write session will restart in 0.006222s
2025-10-09T21:52:23.779700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea270abb-8893e534-e17bea2a-5b2826c2_0] Write session: Do CDS request
2025-10-09T21:52:23.779706Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea270abb-8893e534-e17bea2a-5b2826c2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:23.779693Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 35 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343201866232561:5758]
2025-10-09T21:52:23.787434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ea270abb-8893e534-e17bea2a-5b2826c2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:23.787495Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:23.787507Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2438
2025-10-09T21:52:23.787627Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2438 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:23.787656Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2438 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:23.787665Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2438 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:23.787669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:23.788108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2438 sessionId:  describe result for acl check
2025-10-09T21:52:23.788140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:23.788146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:23.788148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:23.788154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343201866232621:5064] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:23.788655Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343201866232621:5064] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:23.791308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343201866232621:5064] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:23.794014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343201866232621:5064] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:23.794026Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343201866232621:5064] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:23.794042Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343201866232621:5064] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:23.794047Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2438 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:23.796028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:23.798304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:23.798445Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:23.798453Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:23.798461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:23.798498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:23.798670Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343201504379488:8547]
2025-10-09T21:52:23.799072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:23.799331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:23.799360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:23.799572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:23.799603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.799607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.799608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.799609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.799609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.799610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.799611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.799611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.799612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.799615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.799620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.799622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.799624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.799626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.799629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.799630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.799632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.799634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.799638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.799643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.799648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.799651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.799654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.799656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.799658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.799661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.799663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.799666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.799668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.799671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.799673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.799676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.799678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.799681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.799683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.799686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.799701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:23.805655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:23.806712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:23.806725Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:23.806726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:23.806729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:23.806734Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2437 [2:7559343201504379488:8547]
2025-10-09T21:52:23.806742Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:23.806755Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:23.806756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:23.806759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:23.806760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:23.806772Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:23.806776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:23.806781Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:23.806829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.806835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.806836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.806837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:23.806883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:23.806889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.806896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.806902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.806914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:23.806930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343201504379483:8547]
2025-10-09T21:52:23.806932Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343201504379483:8547]
2025-10-09T21:52:23.806933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343201504379483:8547]
2025-10-09T21:52:23.806940Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:23.806949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:23.806954Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:23.806956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:23.806960Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:23.806962Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:23.806964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:23.806973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343201866232805:5064], now have 1 active actors on pipe
2025-10-09T21:52:23.806975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343201866232576:2432], now have 1 active actors on pipe
2025-10-09T21:52:23.806991Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343201504379483:8547]
2025-10-09T21:52:23.806999Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:23.807009Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:23.807040Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2437
2025-10-09T21:52:23.807058Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2437
2025-10-09T21:52:23.807104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.807112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.807135Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|af526c93-4505b3df-2d494c04-2721e9d3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:23.807143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:23.807147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:23.807148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:23.807155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:23.807163Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:23.807173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:23.807328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.807337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.807354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:23.807456Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2438 partition: 0 MaxSeqNo: 25 sessionId: src_id|af526c93-4505b3df-2d494c04-2721e9d3_0
2025-10-09T21:52:23.807752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ea270abb-8893e534-e17bea2a-5b2826c2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|af526c93-4505b3df-2d494c04-2721e9d3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:23.809907Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af526c93-4505b3df-2d494c04-2721e9d3_0] Write session: try to update token
2025-10-09T21:52:23.809920Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af526c93-4505b3df-2d494c04-2721e9d3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:23.821143Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:23.821157Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:23.821159Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:23.821160Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:23.821167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343201504379483:8547]
2025-10-09T21:52:23.821171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343201504379483:8547]
2025-10-09T21:52:23.821174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343201504379483:8547]
2025-10-09T21:52:23.821207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:23.821212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.821214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:23.821229Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:23.821235Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:23.821240Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:23.821353Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:23.821472Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2438 sessionId: src_id|af526c93-4505b3df-2d494c04-2721e9d3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:23.823694Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:23.879674Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343201866232863:5758]
2025-10-09T21:52:23.879925Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 36 pipe creation result: OK TabletId: 72075186224037892 Generation: 2437, pipe: [1:7559343201866232863:5758]
2025-10-09T21:52:23.960416Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:23.960655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.960668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.960708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343201866232863:5758], now have 1 active actors on pipe
2025-10-09T21:52:23.960869Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:23.960937Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 36 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343201866232863:5758]
2025-10-09T21:52:23.960942Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:23.960946Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:23.960969Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:23.961019Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2438 reason: error at write: Disconnected sessionId: src_id|af526c93-4505b3df-2d494c04-2721e9d3_0
2025-10-09T21:52:23.961177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2438 sessionId: src_id|af526c93-4505b3df-2d494c04-2721e9d3_0 is DEAD
2025-10-09T21:52:23.961327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af526c93-4505b3df-2d494c04-2721e9d3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:23.961338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af526c93-4505b3df-2d494c04-2721e9d3_0] Write session will restart in 0.008502s
2025-10-09T21:52:23.961359Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af526c93-4505b3df-2d494c04-2721e9d3_0] Write session: Do CDS request
2025-10-09T21:52:23.961366Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af526c93-4505b3df-2d494c04-2721e9d3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:23.970469Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|af526c93-4505b3df-2d494c04-2721e9d3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:23.970509Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:23.970521Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2439
2025-10-09T21:52:23.970682Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2439 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:23.970715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2439 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:23.970725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2439 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:23.970728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:23.971152Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2439 sessionId:  describe result for acl check
2025-10-09T21:52:23.971188Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:23.971194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:23.971196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:23.971202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343201866232932:5038] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:23.971683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343201866232932:5038] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:23.973886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343201866232932:5038] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:23.976791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343201866232932:5038] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:23.976802Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343201866232932:5038] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:23.976806Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343201866232932:5038] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:23.976816Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2439 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:23.978661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:23.980889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:23.981057Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:23.981066Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:23.981077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:23.981123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:23.981160Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343201504379786:8528]
2025-10-09T21:52:23.981724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:23.982013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:23.982072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:23.982373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:23.982418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.982423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.982424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.982425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.982427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.982428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.982429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.982430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.982431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.982435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.982446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.982449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.982454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.982458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.982462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.982465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.982469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.982472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.982478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.982485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:23.982491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.982496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:23.982500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.982504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:23.982508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.982513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:23.982519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.982523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:23.982527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.982532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:23.982538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.982543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.982547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.982551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.982556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.982560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.982588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:23.988170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:23.989506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:23.989521Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:23.989522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:23.989526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:23.989532Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2438 [2:7559343201504379786:8528]
2025-10-09T21:52:23.989546Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:23.989562Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:23.989565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:23.989568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:23.989570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:23.989586Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:23.989592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:23.989601Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:23.989654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.989662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.989664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.989666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:23.989728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:23.989736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:23.989746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:23.989757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:23.989775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:23.989799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343201866233121:5038], now have 1 active actors on pipe
2025-10-09T21:52:23.989807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343201866232869:2432], now have 1 active actors on pipe
2025-10-09T21:52:23.989814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343201504379781:8528]
2025-10-09T21:52:23.989816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343201504379781:8528]
2025-10-09T21:52:23.989818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343201504379781:8528]
2025-10-09T21:52:23.989826Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:23.989836Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:23.989839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:23.989846Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:23.989848Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:23.989853Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:23.989856Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:23.989880Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343201504379781:8528]
2025-10-09T21:52:23.989889Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:23.989871Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2438
2025-10-09T21:52:23.989880Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2438
2025-10-09T21:52:23.989904Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:23.989919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.989928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.989955Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ace07bd6-3eb82fb0-d14d10d7-62fdace4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:23.989966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:23.989970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:23.989972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:23.989981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:23.989994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:23.990007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:23.990182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:23.990191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:23.990205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:23.990282Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2439 partition: 0 MaxSeqNo: 25 sessionId: src_id|ace07bd6-3eb82fb0-d14d10d7-62fdace4_0
2025-10-09T21:52:23.990594Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|af526c93-4505b3df-2d494c04-2721e9d3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ace07bd6-3eb82fb0-d14d10d7-62fdace4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:23.992844Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ace07bd6-3eb82fb0-d14d10d7-62fdace4_0] Write session: try to update token
2025-10-09T21:52:23.992860Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ace07bd6-3eb82fb0-d14d10d7-62fdace4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:24.003140Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:24.003157Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:24.003160Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:24.003162Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:24.003172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343201504379781:8528]
2025-10-09T21:52:24.003177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343201504379781:8528]
2025-10-09T21:52:24.003181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343201504379781:8528]
2025-10-09T21:52:24.003224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:24.003231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.003234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:24.003235Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:24.003244Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:24.003248Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:24.003363Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:24.005504Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2439 sessionId: src_id|ace07bd6-3eb82fb0-d14d10d7-62fdace4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:24.007804Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:24.060937Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343206161200441:5758]
2025-10-09T21:52:24.061213Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 37 pipe creation result: OK TabletId: 72075186224037892 Generation: 2438, pipe: [1:7559343206161200441:5758]
2025-10-09T21:52:24.158627Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:24.158864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:24.158876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:24.158915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343206161200441:5758], now have 1 active actors on pipe
2025-10-09T21:52:24.159084Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:24.159150Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:24.159164Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:24.159178Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:24.159232Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 37 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343206161200441:5758]
2025-10-09T21:52:24.159238Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2439 reason: error at write: Disconnected sessionId: src_id|ace07bd6-3eb82fb0-d14d10d7-62fdace4_0
2025-10-09T21:52:24.159398Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2439 sessionId: src_id|ace07bd6-3eb82fb0-d14d10d7-62fdace4_0 is DEAD
2025-10-09T21:52:24.159548Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ace07bd6-3eb82fb0-d14d10d7-62fdace4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:24.159568Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ace07bd6-3eb82fb0-d14d10d7-62fdace4_0] Write session will restart in 0.005178s
2025-10-09T21:52:24.159603Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ace07bd6-3eb82fb0-d14d10d7-62fdace4_0] Write session: Do CDS request
2025-10-09T21:52:24.159609Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ace07bd6-3eb82fb0-d14d10d7-62fdace4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:24.165423Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ace07bd6-3eb82fb0-d14d10d7-62fdace4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:24.165540Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:24.165562Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2440
2025-10-09T21:52:24.165691Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2440 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:24.165722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2440 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:24.165733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2440 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:24.165741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:24.166151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2440 sessionId:  describe result for acl check
2025-10-09T21:52:24.166177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:24.166183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:24.166184Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:24.166189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343206161200458:5032] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:24.166593Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343206161200458:5032] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:24.169320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343206161200458:5032] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:24.172486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343206161200458:5032] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:24.172499Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343206161200458:5032] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:24.172503Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343206161200458:5032] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:24.172511Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2440 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:24.176561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:24.178741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:24.178923Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:24.178933Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:24.178944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:24.178999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:24.179037Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343205799347352:8526]
2025-10-09T21:52:24.179601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:24.179958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:24.180002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:24.180314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:24.180357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.180364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.180365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.180366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.180367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.180368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.180369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.180371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.180372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.180377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.180389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.180393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.180397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.180401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.180405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.180409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.180413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.180417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.180424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.180432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.180443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.180452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.180456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.180465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.180472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.180476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.180488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.180493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.180498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.180503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.180507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.180512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.180517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.180523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.180527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.180534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.180564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:24.186803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:24.187903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:24.187917Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:24.187920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:24.187924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:24.187930Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2439 [2:7559343205799347352:8526]
2025-10-09T21:52:24.187944Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:24.187962Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:24.187969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:24.187972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:24.187975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:24.187990Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:24.188001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:24.188008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:24.188068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.188074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.188077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.188079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:24.188250Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2439
2025-10-09T21:52:24.188253Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2439
2025-10-09T21:52:24.188428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:24.188439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.188451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.188465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.188485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:24.188514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343206161200617:5032], now have 1 active actors on pipe
2025-10-09T21:52:24.188522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343206161200443:2432], now have 1 active actors on pipe
2025-10-09T21:52:24.188541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:24.188550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:24.188568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343205799347347:8526]
2025-10-09T21:52:24.188575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343205799347347:8526]
2025-10-09T21:52:24.188577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343205799347347:8526]
2025-10-09T21:52:24.188590Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:24.188603Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:24.188609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:24.188611Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:24.188612Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:24.188624Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:24.188631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:24.188653Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1c437c7e-305ac3c0-1a5bb921-7755e116_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:24.188664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:24.188668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:24.188670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:24.188681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:24.188696Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:24.188720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343205799347347:8526]
2025-10-09T21:52:24.188731Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:24.188734Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:24.188748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:24.188913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:24.188922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:24.188940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:24.189063Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2440 partition: 0 MaxSeqNo: 25 sessionId: src_id|1c437c7e-305ac3c0-1a5bb921-7755e116_0
2025-10-09T21:52:24.189424Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ace07bd6-3eb82fb0-d14d10d7-62fdace4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1c437c7e-305ac3c0-1a5bb921-7755e116_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:24.191829Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c437c7e-305ac3c0-1a5bb921-7755e116_0] Write session: try to update token
2025-10-09T21:52:24.191848Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c437c7e-305ac3c0-1a5bb921-7755e116_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:24.204333Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:24.204360Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:24.204364Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:24.204366Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:24.204378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343205799347347:8526]
2025-10-09T21:52:24.204386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343205799347347:8526]
2025-10-09T21:52:24.204393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343205799347347:8526]
2025-10-09T21:52:24.204443Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:24.204450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:24.204454Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:24.204457Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:24.204459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.204462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:24.204697Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:24.205021Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2440 sessionId: src_id|1c437c7e-305ac3c0-1a5bb921-7755e116_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:24.207409Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:24.259577Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343206161200726:5758]
2025-10-09T21:52:24.259827Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 38 pipe creation result: OK TabletId: 72075186224037892 Generation: 2439, pipe: [1:7559343206161200726:5758]
2025-10-09T21:52:24.351118Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:24.351367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:24.351378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:24.351417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343206161200726:5758], now have 1 active actors on pipe
2025-10-09T21:52:24.351596Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:24.351651Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:24.351663Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:24.351665Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 38 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343206161200726:5758]
2025-10-09T21:52:24.351687Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:24.351733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2440 reason: error at write: Disconnected sessionId: src_id|1c437c7e-305ac3c0-1a5bb921-7755e116_0
2025-10-09T21:52:24.351907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2440 sessionId: src_id|1c437c7e-305ac3c0-1a5bb921-7755e116_0 is DEAD
2025-10-09T21:52:24.352075Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c437c7e-305ac3c0-1a5bb921-7755e116_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:24.352087Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c437c7e-305ac3c0-1a5bb921-7755e116_0] Write session will restart in 0.009658s
2025-10-09T21:52:24.352114Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c437c7e-305ac3c0-1a5bb921-7755e116_0] Write session: Do CDS request
2025-10-09T21:52:24.352120Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c437c7e-305ac3c0-1a5bb921-7755e116_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:24.362442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1c437c7e-305ac3c0-1a5bb921-7755e116_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:24.362491Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:24.362505Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2441
2025-10-09T21:52:24.362654Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2441 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:24.362686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2441 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:24.362697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2441 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:24.362704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:24.363098Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2441 sessionId:  describe result for acl check
2025-10-09T21:52:24.363134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:24.363140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:24.363142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:24.363146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343206161200832:5016] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:24.363696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343206161200832:5016] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:24.366717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343206161200832:5016] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:24.369130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343206161200832:5016] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:24.369157Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343206161200832:5016] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:24.369161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343206161200832:5016] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:24.369173Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2441 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:24.369446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:24.371845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:24.372017Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:24.372026Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:24.372037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:24.372088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:24.372124Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343205799347624:8521]
2025-10-09T21:52:24.372677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:24.372951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:24.372994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:24.373303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:24.373351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.373357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.373358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.373359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.373360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.373361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.373362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.373363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.373364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.373369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.373379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.373383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.373387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.373390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.373393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.373397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.373401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.373405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.373412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.373419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.373425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.373431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.373435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.373440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.373450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.373455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.373460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.373465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.373469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.373474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.373479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.373484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.373488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.373498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.373502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.373507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.373537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:24.379334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:24.380434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:24.380448Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:24.380450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:24.380454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:24.380460Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2440 [2:7559343205799347624:8521]
2025-10-09T21:52:24.380471Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:24.380487Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:24.380491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:24.380494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:24.380496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:24.380511Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:24.380521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:24.380528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:24.380583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.380590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.380592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.380595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:24.380653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:24.380660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.380671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.380694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.380716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:24.380742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343206161200734:2432], now have 1 active actors on pipe
2025-10-09T21:52:24.380745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343206161201002:5016], now have 1 active actors on pipe
2025-10-09T21:52:24.380752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343205799347619:8521]
2025-10-09T21:52:24.380754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343205799347619:8521]
2025-10-09T21:52:24.380756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343205799347619:8521]
2025-10-09T21:52:24.380769Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:24.380786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:24.380792Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:24.380776Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2440
2025-10-09T21:52:24.380794Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:24.380787Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2440
2025-10-09T21:52:24.380796Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:24.380804Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:24.380807Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:24.380827Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343205799347619:8521]
2025-10-09T21:52:24.380831Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:24.380841Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:24.380849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:24.380853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:24.380893Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6b1501b5-9c6858a7-eb59c84f-9e8edab9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:24.380905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:24.380913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:24.380916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:24.380925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:24.380934Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:24.380955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:24.381084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:24.381091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:24.381103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:24.381206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2441 partition: 0 MaxSeqNo: 25 sessionId: src_id|6b1501b5-9c6858a7-eb59c84f-9e8edab9_0
2025-10-09T21:52:24.381509Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1c437c7e-305ac3c0-1a5bb921-7755e116_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6b1501b5-9c6858a7-eb59c84f-9e8edab9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:24.383761Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b1501b5-9c6858a7-eb59c84f-9e8edab9_0] Write session: try to update token
2025-10-09T21:52:24.383776Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b1501b5-9c6858a7-eb59c84f-9e8edab9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:24.394195Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:24.394211Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:24.394213Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:24.394214Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:24.394223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343205799347619:8521]
2025-10-09T21:52:24.394226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343205799347619:8521]
2025-10-09T21:52:24.394229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343205799347619:8521]
2025-10-09T21:52:24.394263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:24.394268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.394270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:24.394272Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:24.394279Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:24.394283Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:24.394388Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:24.395573Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2441 sessionId: src_id|6b1501b5-9c6858a7-eb59c84f-9e8edab9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:24.397847Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:24.451604Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343206161201014:5758]
2025-10-09T21:52:24.451912Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 39 pipe creation result: OK TabletId: 72075186224037892 Generation: 2440, pipe: [1:7559343206161201014:5758]
2025-10-09T21:52:24.533961Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:24.534286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:24.534299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:24.534337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343206161201014:5758], now have 1 active actors on pipe
2025-10-09T21:52:24.534499Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:24.534554Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:24.534575Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 39 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343206161201014:5758]
2025-10-09T21:52:24.534610Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:24.534674Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2441 reason: status is not ok: tablet will be restarted right now sessionId: src_id|6b1501b5-9c6858a7-eb59c84f-9e8edab9_0
2025-10-09T21:52:24.534826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2441 sessionId: src_id|6b1501b5-9c6858a7-eb59c84f-9e8edab9_0 is DEAD
2025-10-09T21:52:24.534966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b1501b5-9c6858a7-eb59c84f-9e8edab9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:24.534977Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b1501b5-9c6858a7-eb59c84f-9e8edab9_0] Write session will restart in 0.008616s
2025-10-09T21:52:24.534998Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b1501b5-9c6858a7-eb59c84f-9e8edab9_0] Write session: Do CDS request
2025-10-09T21:52:24.535003Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b1501b5-9c6858a7-eb59c84f-9e8edab9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:24.544465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6b1501b5-9c6858a7-eb59c84f-9e8edab9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:24.544543Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:24.544559Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2442
2025-10-09T21:52:24.544649Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2442 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:24.544678Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2442 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:24.544689Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2442 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:24.544693Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:24.545234Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2442 sessionId:  describe result for acl check
2025-10-09T21:52:24.545277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:24.545279Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:24.545281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:24.545287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343206161201101:4999] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:24.545853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343206161201101:4999] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:24.548526Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343206161201101:4999] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:24.550904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343206161201101:4999] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:24.550915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343206161201101:4999] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:24.550919Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343206161201101:4999] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:24.550926Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2442 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:24.553631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:24.554093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:24.554222Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:24.554228Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:24.554237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:24.554280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:24.554308Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343205799347895:8519]
2025-10-09T21:52:24.554646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:24.554907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:24.554946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:24.555168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:24.555207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.555212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.555213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.555213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.555214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.555214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.555215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.555215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.555216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.555219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.555225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.555227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.555229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.555231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.555233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.555235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.555237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.555239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.555243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.555249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.555253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.555256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.555258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.555261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.555263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.555265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.555268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.555271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.555273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.555276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.555278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.555281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.555284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.555286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.555289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.555291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.555308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:24.561423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:24.562486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:24.562500Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:24.562501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:24.562504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:24.562509Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2441 [2:7559343205799347895:8519]
2025-10-09T21:52:24.562517Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:24.562528Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:24.562530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:24.562532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:24.562533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:24.562545Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:24.562548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:24.562555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:24.562592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.562597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.562598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.562600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:24.562653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:24.562658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.562664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.562675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.562686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:24.562701Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343205799347890:8519]
2025-10-09T21:52:24.562705Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343205799347890:8519]
2025-10-09T21:52:24.562707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343205799347890:8519]
2025-10-09T21:52:24.562712Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:24.562722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343206161201285:4999], now have 1 active actors on pipe
2025-10-09T21:52:24.562722Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:24.562725Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:24.562728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343206161201017:2432], now have 1 active actors on pipe
2025-10-09T21:52:24.562729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:24.562732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:24.562740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:24.562742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:24.562757Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343205799347890:8519]
2025-10-09T21:52:24.562766Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:24.562773Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:24.562798Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2441
2025-10-09T21:52:24.562801Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2441
2025-10-09T21:52:24.562839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:24.562849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:24.562875Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bfdaca14-c7a67f0c-41cb94f8-b6549807_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:24.562884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:24.562888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:24.562890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:24.562897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:24.562907Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:24.562919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:24.563071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:24.563077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:24.563086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:24.563186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2442 partition: 0 MaxSeqNo: 25 sessionId: src_id|bfdaca14-c7a67f0c-41cb94f8-b6549807_0
2025-10-09T21:52:24.563466Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6b1501b5-9c6858a7-eb59c84f-9e8edab9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bfdaca14-c7a67f0c-41cb94f8-b6549807_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:24.565791Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bfdaca14-c7a67f0c-41cb94f8-b6549807_0] Write session: try to update token
2025-10-09T21:52:24.565809Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bfdaca14-c7a67f0c-41cb94f8-b6549807_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:24.576708Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:24.576725Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:24.576727Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:24.576730Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:24.576739Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343205799347890:8519]
2025-10-09T21:52:24.576744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343205799347890:8519]
2025-10-09T21:52:24.576748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343205799347890:8519]
2025-10-09T21:52:24.576790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:24.576796Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:24.576801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.576801Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:24.576804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:24.576806Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:24.576933Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:24.577353Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2442 sessionId: src_id|bfdaca14-c7a67f0c-41cb94f8-b6549807_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:24.579650Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:24.634893Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343206161201298:5758]
2025-10-09T21:52:24.635106Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 40 pipe creation result: OK TabletId: 72075186224037892 Generation: 2441, pipe: [1:7559343206161201298:5758]
2025-10-09T21:52:24.728689Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:24.729223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:24.729239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:24.729294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343206161201298:5758], now have 1 active actors on pipe
2025-10-09T21:52:24.729469Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:24.729488Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:24.729492Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 40 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343206161201298:5758]
2025-10-09T21:52:24.729502Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:24.729512Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:24.729567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2442 reason: error at write: Disconnected sessionId: src_id|bfdaca14-c7a67f0c-41cb94f8-b6549807_0
2025-10-09T21:52:24.729766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2442 sessionId: src_id|bfdaca14-c7a67f0c-41cb94f8-b6549807_0 is DEAD
2025-10-09T21:52:24.729920Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfdaca14-c7a67f0c-41cb94f8-b6549807_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:24.729929Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfdaca14-c7a67f0c-41cb94f8-b6549807_0] Write session will restart in 0.009635s
2025-10-09T21:52:24.729958Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfdaca14-c7a67f0c-41cb94f8-b6549807_0] Write session: Do CDS request
2025-10-09T21:52:24.729963Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfdaca14-c7a67f0c-41cb94f8-b6549807_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:24.740474Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bfdaca14-c7a67f0c-41cb94f8-b6549807_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:24.740584Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:24.740598Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2443
2025-10-09T21:52:24.740705Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2443 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:24.740737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2443 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:24.740748Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2443 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:24.740752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:24.741237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2443 sessionId:  describe result for acl check
2025-10-09T21:52:24.741283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:24.741290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:24.741291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:24.741298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343206161201405:4989] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:24.741776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343206161201405:4989] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:24.743819Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343206161201405:4989] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:24.745933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343206161201405:4989] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:24.745943Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343206161201405:4989] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:24.745946Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343206161201405:4989] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:24.745956Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2443 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:24.747036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:24.749283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:24.749442Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:24.749450Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:24.749461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:24.749509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:24.749550Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343205799348170:8515]
2025-10-09T21:52:24.750054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:24.750346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:24.750389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:24.750687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:24.750737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.750745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.750747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.750748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.750749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.750750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.750752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.750753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.750754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.750758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.750769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.750774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.750778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.750782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.750786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.750790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.750793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.750797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.750806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.750814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.750820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.750825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.750829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.750840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.750848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.750853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.750862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.750867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.750872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.750877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.750882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.750888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.750898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.750907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.750912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.750916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.750950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:24.757179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:24.758360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:24.758376Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:24.758379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:24.758383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:24.758390Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2442 [2:7559343205799348170:8515]
2025-10-09T21:52:24.758402Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:24.758415Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:24.758417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:24.758419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:24.758420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:24.758433Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:24.758437Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:24.758442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:24.758482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.758488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.758489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.758491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:24.758541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:24.758548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.758559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.758572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.758594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:24.758622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343206161201302:2432], now have 1 active actors on pipe
2025-10-09T21:52:24.758625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343206161201570:4989], now have 1 active actors on pipe
2025-10-09T21:52:24.758631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343205799348165:8515]
2025-10-09T21:52:24.758632Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343205799348165:8515]
2025-10-09T21:52:24.758634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343205799348165:8515]
2025-10-09T21:52:24.758645Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:24.758656Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:24.758668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:24.758669Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:24.758670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:24.758672Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:24.758674Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:24.758660Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2442
2025-10-09T21:52:24.758696Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343205799348165:8515]
2025-10-09T21:52:24.758680Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2442
2025-10-09T21:52:24.758704Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:24.758714Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:24.758737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:24.758744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:24.758769Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|ad4bc1ec-f296ca29-cc216ce7-5f4fa13e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:24.758778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:24.758780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:24.758782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:24.758789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:24.758797Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:24.758808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:24.758942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:24.758950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:24.758963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:24.759045Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2443 partition: 0 MaxSeqNo: 25 sessionId: src_id|ad4bc1ec-f296ca29-cc216ce7-5f4fa13e_0
2025-10-09T21:52:24.759364Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfdaca14-c7a67f0c-41cb94f8-b6549807_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|ad4bc1ec-f296ca29-cc216ce7-5f4fa13e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:24.761663Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad4bc1ec-f296ca29-cc216ce7-5f4fa13e_0] Write session: try to update token
2025-10-09T21:52:24.761678Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad4bc1ec-f296ca29-cc216ce7-5f4fa13e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:24.772749Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:24.772770Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:24.772773Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:24.772775Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:24.772783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343205799348165:8515]
2025-10-09T21:52:24.772788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343205799348165:8515]
2025-10-09T21:52:24.772792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343205799348165:8515]
2025-10-09T21:52:24.772828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:24.772834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.772836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:24.772839Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:24.772845Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:24.772849Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:24.772970Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:24.774741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2443 sessionId: src_id|ad4bc1ec-f296ca29-cc216ce7-5f4fa13e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:24.777116Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:24.829435Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343206161201600:5758]
2025-10-09T21:52:24.829653Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 41 pipe creation result: OK TabletId: 72075186224037892 Generation: 2442, pipe: [1:7559343206161201600:5758]
2025-10-09T21:52:24.918206Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:24.918406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:24.918418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:24.918463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343206161201600:5758], now have 1 active actors on pipe
2025-10-09T21:52:24.918640Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:24.918655Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:24.918671Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:24.918675Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:24.918708Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 41 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343206161201600:5758]
2025-10-09T21:52:24.918737Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2443 reason: error at write: Disconnected sessionId: src_id|ad4bc1ec-f296ca29-cc216ce7-5f4fa13e_0
2025-10-09T21:52:24.918904Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2443 sessionId: src_id|ad4bc1ec-f296ca29-cc216ce7-5f4fa13e_0 is DEAD
2025-10-09T21:52:24.919060Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad4bc1ec-f296ca29-cc216ce7-5f4fa13e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:24.919072Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad4bc1ec-f296ca29-cc216ce7-5f4fa13e_0] Write session will restart in 0.009284s
2025-10-09T21:52:24.919100Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad4bc1ec-f296ca29-cc216ce7-5f4fa13e_0] Write session: Do CDS request
2025-10-09T21:52:24.919106Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad4bc1ec-f296ca29-cc216ce7-5f4fa13e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:24.929484Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|ad4bc1ec-f296ca29-cc216ce7-5f4fa13e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:24.929570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:24.929583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2444
2025-10-09T21:52:24.929694Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2444 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:24.929725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2444 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:24.929734Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2444 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:24.929740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:24.930193Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2444 sessionId:  describe result for acl check
2025-10-09T21:52:24.930229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:24.930237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:24.930238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:24.930244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343206161201740:4933] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:24.931181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343206161201740:4933] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:24.933344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343206161201740:4933] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:24.934716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:24.936561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:24.936729Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:24.936738Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:24.936747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:24.936792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:24.936820Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343205799348485:8482]
2025-10-09T21:52:24.936868Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343206161201740:4933] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:24.936879Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343206161201740:4933] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:24.936881Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343206161201740:4933] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:24.936887Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2444 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:24.937171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:24.937368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:24.937392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:24.937588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:24.937617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.937621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.937622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.937622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.937623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.937624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.937624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.937625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.937625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.937628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.937634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.937636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.937638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.937640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.937642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.937644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.937646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.937648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.937652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.937657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:24.937661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.937664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:24.937666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.937669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:24.937672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.937675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:24.937678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.937681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:24.937683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.937686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:24.937689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.937692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.937694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.937697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.937699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.937703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.937719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:24.943421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:24.944525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:24.944538Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:24.944539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:24.944542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:24.944547Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2443 [2:7559343205799348485:8482]
2025-10-09T21:52:24.944555Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:24.944568Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:24.944573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:24.944575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:24.944576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:24.944588Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:24.944592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:24.944597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:24.944633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.944638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.944640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.944642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:24.944688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:24.944693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.944699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:24.944705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:24.944714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:24.944726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343205799348480:8482]
2025-10-09T21:52:24.944731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343205799348480:8482]
2025-10-09T21:52:24.944733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343205799348480:8482]
2025-10-09T21:52:24.944739Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:24.944747Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:24.944748Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:24.944755Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:24.944756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:24.944757Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:24.944759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:24.944767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343206161201887:4933], now have 1 active actors on pipe
2025-10-09T21:52:24.944770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343206161201613:2432], now have 1 active actors on pipe
2025-10-09T21:52:24.944783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343205799348480:8482]
2025-10-09T21:52:24.944791Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:24.944796Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:24.944821Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2443
2025-10-09T21:52:24.944827Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2443
2025-10-09T21:52:24.944887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:24.944902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:24.944931Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a8610929-e0034525-41f9a76c-f99daa96_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:24.944945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:24.944953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:24.944955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:24.944967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:24.944980Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:24.944998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:24.945141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:24.945151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:24.945181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:24.945285Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2444 partition: 0 MaxSeqNo: 25 sessionId: src_id|a8610929-e0034525-41f9a76c-f99daa96_0
2025-10-09T21:52:24.945601Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|ad4bc1ec-f296ca29-cc216ce7-5f4fa13e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a8610929-e0034525-41f9a76c-f99daa96_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:24.948053Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a8610929-e0034525-41f9a76c-f99daa96_0] Write session: try to update token
2025-10-09T21:52:24.948069Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a8610929-e0034525-41f9a76c-f99daa96_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:24.957198Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:24.957219Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:24.957228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343205799348480:8482]
2025-10-09T21:52:24.957279Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:24.957282Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:24.957446Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:24.957584Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:24.957704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:24.957826Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:24.958006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:24.958064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:24.961061Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2444 sessionId: src_id|a8610929-e0034525-41f9a76c-f99daa96_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:24.963347Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:24.963686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:24.963702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:24.963732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:24.963813Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:24.969837Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:24.969851Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:24.969852Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:24.969854Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:24.969862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343205799348480:8482]
2025-10-09T21:52:24.969864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343205799348480:8482]
2025-10-09T21:52:24.969866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343205799348480:8482]
2025-10-09T21:52:24.969897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:24.969906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:24.969908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:24.969912Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:24.969918Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:24.969922Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:24.970020Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:25.018822Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343210456169195:5758]
2025-10-09T21:52:25.019165Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 42 pipe creation result: OK TabletId: 72075186224037892 Generation: 2443, pipe: [1:7559343210456169195:5758]
2025-10-09T21:52:25.114235Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:25.121946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:25.121974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:25.122159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.122225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.122297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.122367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.122656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.122666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.122730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.122786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.122852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.122921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.122962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.122968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.123031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.123094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.123151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.123206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.123251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.123271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.123345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.123411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.123469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.123530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.123577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.123585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.123654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.123714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.123772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.123837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.123884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.123904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.123967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.124028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.124093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.124155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.124201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.124210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.124271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.124327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.124394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.124463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.124522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.124531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.124596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.124657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.124719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.124779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.124837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.124845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.124913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.124970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.125030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.125096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.125137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.125142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.125197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.125256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.125326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.125387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.125429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.125434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.125490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343210456169195:5758], now have 1 active actors on pipe
2025-10-09T21:52:25.125699Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 42 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343210456169195:5758]
2025-10-09T21:52:25.125714Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:25.125719Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:25.125731Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:25.125751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2444 reason: error at write: Disconnected sessionId: src_id|a8610929-e0034525-41f9a76c-f99daa96_0
2025-10-09T21:52:25.125954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2444 sessionId: src_id|a8610929-e0034525-41f9a76c-f99daa96_0 is DEAD
2025-10-09T21:52:25.126111Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a8610929-e0034525-41f9a76c-f99daa96_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:25.126125Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a8610929-e0034525-41f9a76c-f99daa96_0] Write session will restart in 0.005918s
2025-10-09T21:52:25.126152Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a8610929-e0034525-41f9a76c-f99daa96_0] Write session: Do CDS request
2025-10-09T21:52:25.126159Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a8610929-e0034525-41f9a76c-f99daa96_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:25.126250Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:25.133539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a8610929-e0034525-41f9a76c-f99daa96_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:25.133621Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:25.133638Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2445
2025-10-09T21:52:25.133837Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2445 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:25.133871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2445 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:25.133883Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2445 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:25.133894Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:25.134296Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2445 sessionId:  describe result for acl check
2025-10-09T21:52:25.134333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:25.134335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:25.134338Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:25.134344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343210456169221:4936] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:25.134851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343210456169221:4936] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:25.138099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343210456169221:4936] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:25.141651Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343210456169221:4936] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:25.141666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343210456169221:4936] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:25.141670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343210456169221:4936] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:25.141684Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2445 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:25.145431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:25.147894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:25.148105Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:25.148119Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:25.148134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:25.148194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:25.148241Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343210094316058:8466]
2025-10-09T21:52:25.148837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:25.149152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:25.149197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:25.149531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:25.149594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.149600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.149602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.149603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.149605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.149606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.149607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.149609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.149610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.149615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.149626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.149630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.149635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.149639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.149647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.149651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.149656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.149664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.149672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.149684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.149695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.149705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.149714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.149722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.149730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.149736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.149741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.149746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.149752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.149758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.149767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.149773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.149778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.149783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.149791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.149797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.149829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:25.156371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:25.157628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:25.157644Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:25.157647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:25.157652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:25.157659Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2444 [2:7559343210094316058:8466]
2025-10-09T21:52:25.157676Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:25.157694Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:25.157702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:25.157705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:25.157708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:25.157726Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:25.157732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:25.157744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:25.157811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.157819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.157821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.157824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:25.157906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:25.157913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.157925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.157943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.157963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:25.157989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343210456169391:4936], now have 1 active actors on pipe
2025-10-09T21:52:25.157997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343210456169197:2432], now have 1 active actors on pipe
2025-10-09T21:52:25.158005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343210094316052:8466]
2025-10-09T21:52:25.158008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343210094316052:8466]
2025-10-09T21:52:25.158010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343210094316052:8466]
2025-10-09T21:52:25.158023Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:25.158053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:25.158059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:25.158064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:25.158066Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:25.158075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:25.158078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:25.158075Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2444
2025-10-09T21:52:25.158078Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2444
2025-10-09T21:52:25.158114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343210094316052:8466]
2025-10-09T21:52:25.158122Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:25.158124Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:25.158143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:25.158153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:25.158206Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|78bf15ec-765de45d-ebb0a954-f3f4394e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:25.158218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:25.158226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:25.158229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:25.158241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:25.158255Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:25.158277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:25.158457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:25.158469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:25.158486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:25.158591Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2445 partition: 0 MaxSeqNo: 25 sessionId: src_id|78bf15ec-765de45d-ebb0a954-f3f4394e_0
2025-10-09T21:52:25.158975Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a8610929-e0034525-41f9a76c-f99daa96_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|78bf15ec-765de45d-ebb0a954-f3f4394e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:25.161553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78bf15ec-765de45d-ebb0a954-f3f4394e_0] Write session: try to update token
2025-10-09T21:52:25.161576Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78bf15ec-765de45d-ebb0a954-f3f4394e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:25.172812Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:25.172828Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:25.172831Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:25.172833Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:25.172844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343210094316052:8466]
2025-10-09T21:52:25.172849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343210094316052:8466]
2025-10-09T21:52:25.172853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343210094316052:8466]
2025-10-09T21:52:25.172899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:25.172904Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:25.172906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.172908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:25.172909Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:25.172913Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:25.173043Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:25.174800Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2445 sessionId: src_id|78bf15ec-765de45d-ebb0a954-f3f4394e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:25.177380Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:25.225723Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343210456169483:5758]
2025-10-09T21:52:25.225992Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 43 pipe creation result: OK TabletId: 72075186224037892 Generation: 2444, pipe: [1:7559343210456169483:5758]
2025-10-09T21:52:25.320292Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:25.320872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:25.320890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:25.320941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343210456169483:5758], now have 1 active actors on pipe
2025-10-09T21:52:25.321118Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:25.321187Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 43 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343210456169483:5758]
2025-10-09T21:52:25.321197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:25.321218Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:25.321296Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2445 reason: status is not ok: tablet will be restarted right now sessionId: src_id|78bf15ec-765de45d-ebb0a954-f3f4394e_0
2025-10-09T21:52:25.321482Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2445 sessionId: src_id|78bf15ec-765de45d-ebb0a954-f3f4394e_0 is DEAD
2025-10-09T21:52:25.321623Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78bf15ec-765de45d-ebb0a954-f3f4394e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:25.321636Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78bf15ec-765de45d-ebb0a954-f3f4394e_0] Write session will restart in 0.009580s
2025-10-09T21:52:25.321658Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78bf15ec-765de45d-ebb0a954-f3f4394e_0] Write session: Do CDS request
2025-10-09T21:52:25.321663Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78bf15ec-765de45d-ebb0a954-f3f4394e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:25.332573Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78bf15ec-765de45d-ebb0a954-f3f4394e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:25.332579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:25.332594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2446
2025-10-09T21:52:25.332782Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2446 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:25.332818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2446 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:25.332831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2446 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:25.332835Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:25.333342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2446 sessionId:  describe result for acl check
2025-10-09T21:52:25.333376Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:25.333383Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:25.333385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:25.333391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343210456169610:4904] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:25.333938Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343210456169610:4904] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:25.336602Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343210456169610:4904] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:25.338457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:25.340554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:25.340700Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:25.340679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343210456169610:4904] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:25.340692Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343210456169610:4904] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:25.340709Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:25.340696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343210456169610:4904] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:25.340703Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2446 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:25.340720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:25.340770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:25.340805Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343210094316334:8478]
2025-10-09T21:52:25.341273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:25.341562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:25.341601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:25.341839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:25.341889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.341896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.341897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.341899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.341900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.341901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.341902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.341903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.341904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.341910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.341920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.341924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.341928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.341932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.341936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.341940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.341947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.341949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.341954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.341958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.341966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.341970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.341976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.341980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.341988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.341994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.342003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.342012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.342022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.342046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.342055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.342060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.342069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.342074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.342082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.342087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.342107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:25.348421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:25.349478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:25.349489Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:25.349490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:25.349493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:25.349498Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2445 [2:7559343210094316334:8478]
2025-10-09T21:52:25.349506Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:25.349518Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:25.349519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:25.349521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:25.349522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:25.349534Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:25.349538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:25.349543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:25.349582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.349588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.349589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.349591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:25.349764Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2445
2025-10-09T21:52:25.349772Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2445
2025-10-09T21:52:25.349817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:25.349824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.349832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.349838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.349847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:25.349866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343210456169768:4904], now have 1 active actors on pipe
2025-10-09T21:52:25.349871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343210456169488:2432], now have 1 active actors on pipe
2025-10-09T21:52:25.349882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:25.349886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:25.349903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343210094316329:8478]
2025-10-09T21:52:25.349908Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343210094316329:8478]
2025-10-09T21:52:25.349909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343210094316329:8478]
2025-10-09T21:52:25.349921Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:25.349936Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:25.349946Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:25.349950Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:25.349952Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:25.349954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:25.349956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:25.349970Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bfcc574a-e2baed94-3b315598-9cf43ad6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:25.349976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:25.349979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:25.349980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:25.349993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:25.350001Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:25.350018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343210094316329:8478]
2025-10-09T21:52:25.350040Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:25.350048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:25.350048Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:25.350183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:25.350188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:25.350201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:25.350277Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2446 partition: 0 MaxSeqNo: 25 sessionId: src_id|bfcc574a-e2baed94-3b315598-9cf43ad6_0
2025-10-09T21:52:25.350586Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78bf15ec-765de45d-ebb0a954-f3f4394e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bfcc574a-e2baed94-3b315598-9cf43ad6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:25.353008Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bfcc574a-e2baed94-3b315598-9cf43ad6_0] Write session: try to update token
2025-10-09T21:52:25.353022Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bfcc574a-e2baed94-3b315598-9cf43ad6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:25.361796Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:25.361814Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:25.361823Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343210094316329:8478]
2025-10-09T21:52:25.361866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:25.361874Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:25.362046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:25.362175Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:25.362300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:25.362431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:25.362609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:25.362649Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:25.365406Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2446 sessionId: src_id|bfcc574a-e2baed94-3b315598-9cf43ad6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:25.367674Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:25.367866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:25.367886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:25.367921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:25.367985Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:25.374688Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:25.374702Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:25.374705Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:25.374706Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:25.374716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343210094316329:8478]
2025-10-09T21:52:25.374721Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343210094316329:8478]
2025-10-09T21:52:25.374725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343210094316329:8478]
2025-10-09T21:52:25.374763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:25.374769Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:25.374775Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:25.374777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.374779Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:25.374779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:25.374900Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:25.421323Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343210456169779:5758]
2025-10-09T21:52:25.421575Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 44 pipe creation result: OK TabletId: 72075186224037892 Generation: 2445, pipe: [1:7559343210456169779:5758]
2025-10-09T21:52:25.518088Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:25.524416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:25.524443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:25.524592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.524646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.524698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.524751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.524791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.524799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.524856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.524905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.524959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.525013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.525051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.525056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.525105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.525152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.525200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.525248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.525300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.525308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.525378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.525433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.525483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.525536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.525572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.525578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.525632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.525681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.525731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.525781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.525818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.525823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.525875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.525925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.525974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.526021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.526091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.526097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.526145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.526191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.526239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.526287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.526322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.526330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.526379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.526424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.526473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.526521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.526556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.526561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.526607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.526654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.526703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.526750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.526787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.526792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.526841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:25.526891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:25.526936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:25.526986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:25.527025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:25.527032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:25.527068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343210456169779:5758], now have 1 active actors on pipe
2025-10-09T21:52:25.527275Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 44 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343210456169779:5758]
2025-10-09T21:52:25.527288Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:25.527293Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:25.527317Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:25.527325Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2446 reason: error at write: Disconnected sessionId: src_id|bfcc574a-e2baed94-3b315598-9cf43ad6_0
2025-10-09T21:52:25.527499Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2446 sessionId: src_id|bfcc574a-e2baed94-3b315598-9cf43ad6_0 is DEAD
2025-10-09T21:52:25.527665Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfcc574a-e2baed94-3b315598-9cf43ad6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:25.527676Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfcc574a-e2baed94-3b315598-9cf43ad6_0] Write session will restart in 0.005904s
2025-10-09T21:52:25.527700Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfcc574a-e2baed94-3b315598-9cf43ad6_0] Write session: Do CDS request
2025-10-09T21:52:25.527706Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfcc574a-e2baed94-3b315598-9cf43ad6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:25.527755Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:25.534446Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bfcc574a-e2baed94-3b315598-9cf43ad6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:25.534515Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:25.534528Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2447
2025-10-09T21:52:25.534627Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2447 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:25.534659Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2447 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:25.534670Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2447 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:25.534679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:25.535067Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2447 sessionId:  describe result for acl check
2025-10-09T21:52:25.535097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:25.535099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:25.535104Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:25.535110Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343210456169808:4902] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:25.535624Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343210456169808:4902] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:25.538904Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343210456169808:4902] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:25.541965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343210456169808:4902] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:25.541977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343210456169808:4902] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:25.541981Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343210456169808:4902] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:25.541988Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2447 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:25.544616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:25.546902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:25.547078Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:25.547088Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:25.547099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:25.547158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:25.547198Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343210094316611:8476]
2025-10-09T21:52:25.547736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:25.547989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:25.548029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:25.548331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:25.548383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.548388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.548390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.548391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.548392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.548393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.548394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.548395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.548396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.548400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.548411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.548415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.548419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.548423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.548427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.548435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.548439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.548447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.548453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.548461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.548472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.548477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.548483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.548488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.548493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.548502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.548507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.548511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.548515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.548520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.548529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.548534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.548542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.548547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.548556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.548565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.548597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:25.554536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:25.555621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:25.555634Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:25.555636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:25.555640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:25.555645Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2446 [2:7559343210094316611:8476]
2025-10-09T21:52:25.555865Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:25.555889Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:25.555892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:25.555895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:25.555897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:25.555912Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:25.555923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:25.555930Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:25.555987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.555994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.555996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.555999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:25.556060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:25.556067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.556078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.556094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.556110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:25.556130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343210456169782:2432], now have 1 active actors on pipe
2025-10-09T21:52:25.556138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343210456170000:4902], now have 1 active actors on pipe
2025-10-09T21:52:25.556145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343210094316606:8476]
2025-10-09T21:52:25.556151Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343210094316606:8476]
2025-10-09T21:52:25.556153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343210094316606:8476]
2025-10-09T21:52:25.556165Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:25.556182Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:25.556184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:25.556187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:25.556188Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:25.556200Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:25.556207Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:25.556192Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2446
2025-10-09T21:52:25.556200Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2446
2025-10-09T21:52:25.556231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343210094316606:8476]
2025-10-09T21:52:25.556241Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:25.556248Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:25.556258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:25.556262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:25.556301Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c41520fc-625b1bfa-40a9ab08-50c1db83_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:25.556312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:25.556320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:25.556322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:25.556331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:25.556344Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:25.556363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:25.556507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:25.556516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:25.556530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:25.556620Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2447 partition: 0 MaxSeqNo: 25 sessionId: src_id|c41520fc-625b1bfa-40a9ab08-50c1db83_0
2025-10-09T21:52:25.556963Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bfcc574a-e2baed94-3b315598-9cf43ad6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c41520fc-625b1bfa-40a9ab08-50c1db83_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:25.559336Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c41520fc-625b1bfa-40a9ab08-50c1db83_0] Write session: try to update token
2025-10-09T21:52:25.559353Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c41520fc-625b1bfa-40a9ab08-50c1db83_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:25.569095Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:25.569112Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:25.569114Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:25.569116Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:25.569125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343210094316606:8476]
2025-10-09T21:52:25.569130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343210094316606:8476]
2025-10-09T21:52:25.569134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343210094316606:8476]
2025-10-09T21:52:25.569174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:25.569181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.569184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:25.569188Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:25.569194Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:25.569197Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:25.569333Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:25.571678Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2447 sessionId: src_id|c41520fc-625b1bfa-40a9ab08-50c1db83_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:25.574144Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:25.627651Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343210456170068:5758]
2025-10-09T21:52:25.627932Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 45 pipe creation result: OK TabletId: 72075186224037892 Generation: 2446, pipe: [1:7559343210456170068:5758]
2025-10-09T21:52:25.707166Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:25.707395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:25.707407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:25.707443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343210456170068:5758], now have 1 active actors on pipe
2025-10-09T21:52:25.707635Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:25.707684Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 45 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343210456170068:5758]
2025-10-09T21:52:25.707709Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:25.707711Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:25.707780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2447 reason: status is not ok: tablet will be restarted right now sessionId: src_id|c41520fc-625b1bfa-40a9ab08-50c1db83_0
2025-10-09T21:52:25.707925Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2447 sessionId: src_id|c41520fc-625b1bfa-40a9ab08-50c1db83_0 is DEAD
2025-10-09T21:52:25.708092Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c41520fc-625b1bfa-40a9ab08-50c1db83_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:25.708103Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c41520fc-625b1bfa-40a9ab08-50c1db83_0] Write session will restart in 0.005474s
2025-10-09T21:52:25.708124Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c41520fc-625b1bfa-40a9ab08-50c1db83_0] Write session: Do CDS request
2025-10-09T21:52:25.708129Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c41520fc-625b1bfa-40a9ab08-50c1db83_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:25.714476Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c41520fc-625b1bfa-40a9ab08-50c1db83_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:25.714551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:25.714562Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2448
2025-10-09T21:52:25.714699Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2448 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:25.714722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2448 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:25.714729Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2448 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:25.714733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:25.715203Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2448 sessionId:  describe result for acl check
2025-10-09T21:52:25.715241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:25.715245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:25.715246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:25.715252Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343210456170095:4877] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:25.715790Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343210456170095:4877] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:25.718124Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343210456170095:4877] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:25.720965Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343210456170095:4877] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:25.720978Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343210456170095:4877] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:25.720982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343210456170095:4877] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:25.720988Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2448 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:25.725585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:25.728187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:25.728376Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:25.728386Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:25.728398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:25.728450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:25.728488Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343210094316890:8470]
2025-10-09T21:52:25.728968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:25.729258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:25.729309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:25.729615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:25.729666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.729672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.729673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.729675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.729676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.729677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.729678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.729679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.729681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.729685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.729697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.729702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.729706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.729710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.729714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.729720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.729726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.729730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.729737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.729745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.729756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.729762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.729768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.729773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.729777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.729783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.729787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.729790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.729793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.729796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.729798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.729802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.729804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.729807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.729810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.729812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.729832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:25.735925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:25.736959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:25.736971Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:25.736973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:25.736977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:25.736984Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2447 [2:7559343210094316890:8470]
2025-10-09T21:52:25.736998Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:25.737014Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:25.737024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:25.737027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:25.737029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:25.737046Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:25.737051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:25.737062Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:25.737117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.737124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.737126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.737129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:25.737201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:25.737209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.737221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.737236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.737258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:25.737294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343210456170071:2432], now have 1 active actors on pipe
2025-10-09T21:52:25.737303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343210456170228:4877], now have 1 active actors on pipe
2025-10-09T21:52:25.737312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343210094316885:8470]
2025-10-09T21:52:25.737314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343210094316885:8470]
2025-10-09T21:52:25.737316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343210094316885:8470]
2025-10-09T21:52:25.737329Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:25.737347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:25.737353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:25.737356Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:25.737358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:25.737366Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:25.737369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:25.737353Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2447
2025-10-09T21:52:25.737379Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2447
2025-10-09T21:52:25.737394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343210094316885:8470]
2025-10-09T21:52:25.737403Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:25.737405Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:25.737421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:25.737430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:25.737473Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9c016967-3e5de3fc-ad80ccb2-1eef6cfc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:25.737484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:25.737488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:25.737490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:25.737501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:25.737514Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:25.737533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:25.737682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:25.737693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:25.737709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:25.737788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2448 partition: 0 MaxSeqNo: 25 sessionId: src_id|9c016967-3e5de3fc-ad80ccb2-1eef6cfc_0
2025-10-09T21:52:25.738130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c41520fc-625b1bfa-40a9ab08-50c1db83_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9c016967-3e5de3fc-ad80ccb2-1eef6cfc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:25.740495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c016967-3e5de3fc-ad80ccb2-1eef6cfc_0] Write session: try to update token
2025-10-09T21:52:25.740508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c016967-3e5de3fc-ad80ccb2-1eef6cfc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:25.751002Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:25.751020Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:25.751022Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:25.751024Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:25.751034Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343210094316885:8470]
2025-10-09T21:52:25.751039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343210094316885:8470]
2025-10-09T21:52:25.751044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343210094316885:8470]
2025-10-09T21:52:25.751088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:25.751098Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:25.751100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.751103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:25.751103Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:25.751107Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:25.751230Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:25.752666Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2448 sessionId: src_id|9c016967-3e5de3fc-ad80ccb2-1eef6cfc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:25.755364Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:25.807885Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343210456170358:5758]
2025-10-09T21:52:25.808180Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 46 pipe creation result: OK TabletId: 72075186224037892 Generation: 2447, pipe: [1:7559343210456170358:5758]
2025-10-09T21:52:25.896930Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:25.897512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:25.897525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:25.897563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343210456170358:5758], now have 1 active actors on pipe
2025-10-09T21:52:25.897828Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:25.897840Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 46 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343210456170358:5758]
2025-10-09T21:52:25.897866Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:25.897868Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:25.897928Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:25.897940Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2448 reason: error at write: Disconnected sessionId: src_id|9c016967-3e5de3fc-ad80ccb2-1eef6cfc_0
2025-10-09T21:52:25.898138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2448 sessionId: src_id|9c016967-3e5de3fc-ad80ccb2-1eef6cfc_0 is DEAD
2025-10-09T21:52:25.898301Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c016967-3e5de3fc-ad80ccb2-1eef6cfc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:25.898312Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c016967-3e5de3fc-ad80ccb2-1eef6cfc_0] Write session will restart in 0.009434s
2025-10-09T21:52:25.898330Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c016967-3e5de3fc-ad80ccb2-1eef6cfc_0] Write session: Do CDS request
2025-10-09T21:52:25.898335Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c016967-3e5de3fc-ad80ccb2-1eef6cfc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:25.908527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9c016967-3e5de3fc-ad80ccb2-1eef6cfc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:25.908664Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:25.908679Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2449
2025-10-09T21:52:25.908828Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2449 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:25.908873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2449 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:25.908885Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2449 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:25.908890Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:25.909381Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2449 sessionId:  describe result for acl check
2025-10-09T21:52:25.909427Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:25.909436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:25.909438Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:25.909444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343210456170462:4860] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:25.909988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343210456170462:4860] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:25.912736Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343210456170462:4860] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:25.915530Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343210456170462:4860] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:25.915545Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343210456170462:4860] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:25.915549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343210456170462:4860] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:25.915557Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2449 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:25.916767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:25.919078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:25.919234Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:25.919242Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:25.919251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:25.919288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:25.919316Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343210094317211:8453]
2025-10-09T21:52:25.919692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:25.919939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:25.919978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:25.920203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:25.920241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.920245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.920246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.920247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.920247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.920248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.920249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.920249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.920250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.920253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.920260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.920262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.920265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.920267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.920269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.920271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.920273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.920275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.920281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.920286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:25.920290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.920293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:25.920296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.920299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:25.920302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.920305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:25.920308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.920311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:25.920313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.920317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:25.920322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.920327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.920331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.920336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.920340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.920345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.920373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:25.926621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:25.927681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:25.927689Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:25.927690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:25.927694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:25.927699Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2448 [2:7559343210094317211:8453]
2025-10-09T21:52:25.927709Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:25.927723Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:25.927725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:25.927727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:25.927728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:25.927750Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:25.927754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:25.927760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:25.927805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.927814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.927816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.927818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:25.927880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:25.927888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.927900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:25.927911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:25.927934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:25.927960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343210456170656:4860], now have 1 active actors on pipe
2025-10-09T21:52:25.927969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343210456170385:2432], now have 1 active actors on pipe
2025-10-09T21:52:25.927976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343210094317206:8453]
2025-10-09T21:52:25.927978Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343210094317206:8453]
2025-10-09T21:52:25.927981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343210094317206:8453]
2025-10-09T21:52:25.927994Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:25.928010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:25.928015Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:25.928018Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:25.928020Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:25.928022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:25.928024Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:25.928015Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2448
2025-10-09T21:52:25.928042Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343210094317206:8453]
2025-10-09T21:52:25.928048Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:25.928032Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2448
2025-10-09T21:52:25.928059Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:25.928067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:25.928070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:25.928098Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|fe21e4a7-ac2747dd-5a6c20a6-c0bc9a1a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:25.928108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:25.928111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:25.928112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:25.928119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:25.928126Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:25.928139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:25.928270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:25.928274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:25.928284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:25.928352Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2449 partition: 0 MaxSeqNo: 25 sessionId: src_id|fe21e4a7-ac2747dd-5a6c20a6-c0bc9a1a_0
2025-10-09T21:52:25.928666Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9c016967-3e5de3fc-ad80ccb2-1eef6cfc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|fe21e4a7-ac2747dd-5a6c20a6-c0bc9a1a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:25.931010Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe21e4a7-ac2747dd-5a6c20a6-c0bc9a1a_0] Write session: try to update token
2025-10-09T21:52:25.931027Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe21e4a7-ac2747dd-5a6c20a6-c0bc9a1a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:25.941981Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:25.942005Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:25.942007Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:25.942008Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:25.942016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343210094317206:8453]
2025-10-09T21:52:25.942019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343210094317206:8453]
2025-10-09T21:52:25.942022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343210094317206:8453]
2025-10-09T21:52:25.942061Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:25.942066Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:25.942069Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:25.942075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:25.942082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:25.942085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:25.942200Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:25.942957Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2449 sessionId: src_id|fe21e4a7-ac2747dd-5a6c20a6-c0bc9a1a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:25.945122Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:25.997791Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343210456170672:5758]
2025-10-09T21:52:25.998019Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 47 pipe creation result: OK TabletId: 72075186224037892 Generation: 2448, pipe: [1:7559343210456170672:5758]
2025-10-09T21:52:26.089069Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:26.089580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:26.089616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:26.089686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343210456170672:5758], now have 1 active actors on pipe
2025-10-09T21:52:26.089909Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:26.089927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:26.089944Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:26.090016Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2449 reason: error at write: Disconnected sessionId: src_id|fe21e4a7-ac2747dd-5a6c20a6-c0bc9a1a_0
2025-10-09T21:52:26.090052Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 47 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343210456170672:5758]
2025-10-09T21:52:26.090079Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:26.090208Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2449 sessionId: src_id|fe21e4a7-ac2747dd-5a6c20a6-c0bc9a1a_0 is DEAD
2025-10-09T21:52:26.090350Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe21e4a7-ac2747dd-5a6c20a6-c0bc9a1a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:26.090362Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe21e4a7-ac2747dd-5a6c20a6-c0bc9a1a_0] Write session will restart in 0.006631s
2025-10-09T21:52:26.090389Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe21e4a7-ac2747dd-5a6c20a6-c0bc9a1a_0] Write session: Do CDS request
2025-10-09T21:52:26.090396Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe21e4a7-ac2747dd-5a6c20a6-c0bc9a1a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:26.098495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|fe21e4a7-ac2747dd-5a6c20a6-c0bc9a1a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:26.098576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:26.098587Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2450
2025-10-09T21:52:26.098679Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2450 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:26.098703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2450 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:26.098706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2450 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:26.098709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:26.099150Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2450 sessionId:  describe result for acl check
2025-10-09T21:52:26.099194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:26.099200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:26.099203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:26.099209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343214751138011:4843] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:26.099670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343214751138011:4843] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:26.101652Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343214751138011:4843] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:26.104336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343214751138011:4843] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:26.104350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343214751138011:4843] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:26.104353Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343214751138011:4843] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:26.104361Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2450 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:26.108569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:26.110827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:26.111071Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:26.111080Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:26.111091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:26.111140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:26.111171Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343214389284787:8451]
2025-10-09T21:52:26.111593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:26.111870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:26.111917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:26.112193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:26.112246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.112253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.112253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.112254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.112255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.112256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.112256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.112257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.112258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.112262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.112269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.112272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.112274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.112276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.112278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.112280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.112282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.112285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.112291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.112300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.112306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.112312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.112316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.112322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.112326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.112332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.112336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.112340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.112342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.112346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.112349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.112352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.112355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.112358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.112361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.112365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.112387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:26.118697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:26.119781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:26.119796Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:26.119798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:26.119802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:26.119809Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2449 [2:7559343214389284787:8451]
2025-10-09T21:52:26.119820Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:26.119836Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:26.119839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:26.119841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:26.119843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:26.119859Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:26.119866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:26.119874Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:26.119926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.119934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.119936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.119938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:26.119997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:26.120007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.120020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.120031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.120046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:26.120061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343214389284782:8451]
2025-10-09T21:52:26.120069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343214389284782:8451]
2025-10-09T21:52:26.120070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343214389284782:8451]
2025-10-09T21:52:26.120080Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:26.120097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343214751137971:2432], now have 1 active actors on pipe
2025-10-09T21:52:26.120097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:26.120100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343214751138157:4843], now have 1 active actors on pipe
2025-10-09T21:52:26.120100Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:26.120103Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:26.120108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:26.120116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:26.120118Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:26.120145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343214389284782:8451]
2025-10-09T21:52:26.120157Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:26.120161Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2449
2025-10-09T21:52:26.120201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:26.120180Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2449
2025-10-09T21:52:26.120502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:26.120515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:26.120540Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b73c07e4-43aaccaf-bfdc76-50c7efd6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:26.120551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:26.120554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:26.120556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:26.120564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:26.120574Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:26.120591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:26.120742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:26.120753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:26.120767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:26.120850Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2450 partition: 0 MaxSeqNo: 25 sessionId: src_id|b73c07e4-43aaccaf-bfdc76-50c7efd6_0
2025-10-09T21:52:26.121173Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|fe21e4a7-ac2747dd-5a6c20a6-c0bc9a1a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b73c07e4-43aaccaf-bfdc76-50c7efd6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:26.123679Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b73c07e4-43aaccaf-bfdc76-50c7efd6_0] Write session: try to update token
2025-10-09T21:52:26.123701Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b73c07e4-43aaccaf-bfdc76-50c7efd6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:26.134208Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:26.134225Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:26.134227Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:26.134228Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:26.134236Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343214389284782:8451]
2025-10-09T21:52:26.134239Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343214389284782:8451]
2025-10-09T21:52:26.134243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343214389284782:8451]
2025-10-09T21:52:26.134282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:26.134291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.134294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:26.134299Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:26.134305Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:26.134309Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:26.134426Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:26.136322Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2450 sessionId: src_id|b73c07e4-43aaccaf-bfdc76-50c7efd6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:26.138678Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:26.190286Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343214751138263:5758]
2025-10-09T21:52:26.190578Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 48 pipe creation result: OK TabletId: 72075186224037892 Generation: 2449, pipe: [1:7559343214751138263:5758]
2025-10-09T21:52:26.282227Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:26.282658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:26.282673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:26.282711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343214751138263:5758], now have 1 active actors on pipe
2025-10-09T21:52:26.282894Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:26.282920Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:26.282930Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 48 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343214751138263:5758]
2025-10-09T21:52:26.282951Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:26.282953Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:26.283015Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2450 reason: error at write: Disconnected sessionId: src_id|b73c07e4-43aaccaf-bfdc76-50c7efd6_0
2025-10-09T21:52:26.283192Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2450 sessionId: src_id|b73c07e4-43aaccaf-bfdc76-50c7efd6_0 is DEAD
2025-10-09T21:52:26.283345Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b73c07e4-43aaccaf-bfdc76-50c7efd6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:26.283355Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b73c07e4-43aaccaf-bfdc76-50c7efd6_0] Write session will restart in 0.009038s
2025-10-09T21:52:26.283376Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b73c07e4-43aaccaf-bfdc76-50c7efd6_0] Write session: Do CDS request
2025-10-09T21:52:26.283381Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b73c07e4-43aaccaf-bfdc76-50c7efd6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:26.293505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b73c07e4-43aaccaf-bfdc76-50c7efd6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:26.293603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:26.293615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2451
2025-10-09T21:52:26.293709Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2451 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:26.293733Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2451 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:26.293740Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2451 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:26.293743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:26.294127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2451 sessionId:  describe result for acl check
2025-10-09T21:52:26.294155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:26.294161Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:26.294163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:26.294169Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343214751138354:4828] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:26.294579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343214751138354:4828] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:26.296906Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343214751138354:4828] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:26.299366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343214751138354:4828] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:26.299377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343214751138354:4828] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:26.299379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343214751138354:4828] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:26.299384Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2451 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:26.300333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:26.303107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:26.303283Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:26.303296Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:26.303308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:26.303357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:26.303390Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343214389285068:8449]
2025-10-09T21:52:26.303807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:26.304107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:26.304136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:26.304361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:26.304402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.304408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.304408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.304409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.304410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.304411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.304411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.304412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.304413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.304416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.304423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.304425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.304428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.304430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.304432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.304434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.304436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.304438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.304442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.304447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.304452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.304455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.304458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.304461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.304463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.304466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.304469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.304472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.304474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.304477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.304480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.304483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.304486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.304489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.304493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.304497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.304526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:26.310620Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:26.311710Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:26.311723Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:26.311724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:26.311727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:26.311733Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2450 [2:7559343214389285068:8449]
2025-10-09T21:52:26.311741Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:26.311754Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:26.311756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:26.311758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:26.311759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:26.311775Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:26.311780Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:26.311788Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:26.311831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.311841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.311842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.311844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:26.311892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:26.311898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.311907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.311914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.311929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:26.311942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343214389285063:8449]
2025-10-09T21:52:26.311949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343214389285063:8449]
2025-10-09T21:52:26.311950Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343214389285063:8449]
2025-10-09T21:52:26.311957Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:26.311967Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:26.311967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:26.311970Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:26.311971Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:26.311980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:26.311982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:26.311991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343214751138544:4828], now have 1 active actors on pipe
2025-10-09T21:52:26.311994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343214751138268:2432], now have 1 active actors on pipe
2025-10-09T21:52:26.312012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343214389285063:8449]
2025-10-09T21:52:26.312016Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:26.312023Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:26.312048Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2450
2025-10-09T21:52:26.312053Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2450
2025-10-09T21:52:26.312147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:26.312161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:26.312186Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2c42fa9a-fb4bb297-72a5ecf-b3a7c1cc_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:26.312200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:26.312208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:26.312210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:26.312218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:26.312232Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:26.312248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:26.312392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:26.312399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:26.312408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:26.312485Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2451 partition: 0 MaxSeqNo: 25 sessionId: src_id|2c42fa9a-fb4bb297-72a5ecf-b3a7c1cc_0
2025-10-09T21:52:26.312822Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b73c07e4-43aaccaf-bfdc76-50c7efd6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2c42fa9a-fb4bb297-72a5ecf-b3a7c1cc_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:26.315337Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c42fa9a-fb4bb297-72a5ecf-b3a7c1cc_0] Write session: try to update token
2025-10-09T21:52:26.315360Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c42fa9a-fb4bb297-72a5ecf-b3a7c1cc_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:26.325576Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:26.325595Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:26.325597Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:26.325598Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:26.325606Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343214389285063:8449]
2025-10-09T21:52:26.325609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343214389285063:8449]
2025-10-09T21:52:26.325612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343214389285063:8449]
2025-10-09T21:52:26.325645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:26.325651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.325652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:26.325669Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:26.325675Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:26.325679Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:26.325766Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:26.327170Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2451 sessionId: src_id|2c42fa9a-fb4bb297-72a5ecf-b3a7c1cc_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:26.329178Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:26.383362Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343214751138565:5758]
2025-10-09T21:52:26.383610Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 49 pipe creation result: OK TabletId: 72075186224037892 Generation: 2450, pipe: [1:7559343214751138565:5758]
2025-10-09T21:52:26.463742Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:26.463969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:26.463983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:26.464021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343214751138565:5758], now have 1 active actors on pipe
2025-10-09T21:52:26.464243Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 49 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343214751138565:5758]
2025-10-09T21:52:26.464255Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:26.464259Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:26.464269Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:26.464353Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2451 reason: error at write: Disconnected sessionId: src_id|2c42fa9a-fb4bb297-72a5ecf-b3a7c1cc_0
2025-10-09T21:52:26.464498Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2451 sessionId: src_id|2c42fa9a-fb4bb297-72a5ecf-b3a7c1cc_0 is DEAD
2025-10-09T21:52:26.464674Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c42fa9a-fb4bb297-72a5ecf-b3a7c1cc_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:26.464685Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c42fa9a-fb4bb297-72a5ecf-b3a7c1cc_0] Write session will restart in 0.008235s
2025-10-09T21:52:26.464703Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c42fa9a-fb4bb297-72a5ecf-b3a7c1cc_0] Write session: Do CDS request
2025-10-09T21:52:26.464711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c42fa9a-fb4bb297-72a5ecf-b3a7c1cc_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:26.465056Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:26.474459Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2c42fa9a-fb4bb297-72a5ecf-b3a7c1cc_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:26.474543Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:26.474556Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2452
2025-10-09T21:52:26.474662Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2452 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:26.474699Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2452 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:26.474711Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2452 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:26.474715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:26.475148Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2452 sessionId:  describe result for acl check
2025-10-09T21:52:26.475176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:26.475181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:26.475182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:26.475186Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343214751138658:4809] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:26.475613Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343214751138658:4809] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:26.477725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343214751138658:4809] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:26.480465Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343214751138658:4809] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:26.480477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343214751138658:4809] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:26.480480Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343214751138658:4809] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:26.480487Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2452 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:26.481924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:26.483862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:26.484001Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:26.484008Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:26.484016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:26.484050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:26.484078Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343214389285350:8447]
2025-10-09T21:52:26.484421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:26.484670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:26.484701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:26.484902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:26.484931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.484936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.484937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.484937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.484938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.484938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.484939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.484939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.484940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.484943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.484949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.484951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.484953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.484955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.484957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.484959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.484961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.484963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.484967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.484972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.484976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.484979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.484981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.484984Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.484986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.484990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.484992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.484995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.484998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.485001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.485004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.485010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.485012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.485015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.485018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.485020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.485035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:26.490883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:26.492219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:26.492235Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:26.492237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:26.492240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:26.492246Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2451 [2:7559343214389285350:8447]
2025-10-09T21:52:26.492258Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:26.492271Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:26.492273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:26.492276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:26.492277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:26.492295Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:26.492299Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:26.492304Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:26.492348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.492355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.492357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.492359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:26.492417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:26.492424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.492433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.492439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.492456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:26.492473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343214751138567:2432], now have 1 active actors on pipe
2025-10-09T21:52:26.492482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343214751138844:4809], now have 1 active actors on pipe
2025-10-09T21:52:26.492487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343214389285345:8447]
2025-10-09T21:52:26.492488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343214389285345:8447]
2025-10-09T21:52:26.492490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343214389285345:8447]
2025-10-09T21:52:26.492502Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:26.492511Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:26.492524Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:26.492524Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:26.492526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:26.492527Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:26.492529Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:26.492553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343214389285345:8447]
2025-10-09T21:52:26.492561Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:26.492567Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:26.492544Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2451
2025-10-09T21:52:26.492549Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2451
2025-10-09T21:52:26.492595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:26.492603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:26.492624Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|45c52908-dae25b6-d7394bdd-65d65f6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:26.492634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:26.492636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:26.492638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:26.492644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:26.492651Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:26.492662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:26.492818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:26.492824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:26.492833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:26.492907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2452 partition: 0 MaxSeqNo: 25 sessionId: src_id|45c52908-dae25b6-d7394bdd-65d65f6_0
2025-10-09T21:52:26.493152Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2c42fa9a-fb4bb297-72a5ecf-b3a7c1cc_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|45c52908-dae25b6-d7394bdd-65d65f6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:26.495380Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45c52908-dae25b6-d7394bdd-65d65f6_0] Write session: try to update token
2025-10-09T21:52:26.495395Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45c52908-dae25b6-d7394bdd-65d65f6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:26.505956Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:26.505978Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:26.505981Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:26.505983Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:26.505992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343214389285345:8447]
2025-10-09T21:52:26.505996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343214389285345:8447]
2025-10-09T21:52:26.506000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343214389285345:8447]
2025-10-09T21:52:26.506055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:26.506066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.506068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:26.506071Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:26.506078Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:26.506086Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:26.506214Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:26.507329Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2452 sessionId: src_id|45c52908-dae25b6-d7394bdd-65d65f6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:26.509455Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:26.564605Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343214751138859:5758]
2025-10-09T21:52:26.564894Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 50 pipe creation result: OK TabletId: 72075186224037892 Generation: 2451, pipe: [1:7559343214751138859:5758]
2025-10-09T21:52:26.644568Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:26.645085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:26.645100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:26.645140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343214751138859:5758], now have 1 active actors on pipe
2025-10-09T21:52:26.645340Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:26.645345Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:26.645376Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:26.645380Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:26.645390Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 50 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343214751138859:5758]
2025-10-09T21:52:26.645444Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2452 reason: error at write: Disconnected sessionId: src_id|45c52908-dae25b6-d7394bdd-65d65f6_0
2025-10-09T21:52:26.645566Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2452 sessionId: src_id|45c52908-dae25b6-d7394bdd-65d65f6_0 is DEAD
2025-10-09T21:52:26.645661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45c52908-dae25b6-d7394bdd-65d65f6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:26.645669Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45c52908-dae25b6-d7394bdd-65d65f6_0] Write session will restart in 0.005911s
2025-10-09T21:52:26.645690Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45c52908-dae25b6-d7394bdd-65d65f6_0] Write session: Do CDS request
2025-10-09T21:52:26.645694Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45c52908-dae25b6-d7394bdd-65d65f6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:26.652375Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|45c52908-dae25b6-d7394bdd-65d65f6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:26.652465Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:26.652477Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2453
2025-10-09T21:52:26.652600Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2453 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:26.652631Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2453 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:26.652641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2453 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:26.652645Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:26.653133Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2453 sessionId:  describe result for acl check
2025-10-09T21:52:26.653173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:26.653179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:26.653181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:26.653189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343214751138890:4786] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:26.653787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343214751138890:4786] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:26.656589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343214751138890:4786] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:26.659700Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343214751138890:4786] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:26.659723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343214751138890:4786] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:26.659727Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343214751138890:4786] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:26.659734Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2453 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:26.665578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:26.665991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:26.666152Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:26.666158Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:26.666166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:26.666201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:26.666229Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343214389285635:8444]
2025-10-09T21:52:26.666647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:26.666911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:26.666943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:26.667145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:26.667175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.667180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.667181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.667182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.667182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.667183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.667184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.667184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.667185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.667188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.667195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.667197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.667199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.667201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.667203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.667205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.667207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.667209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.667213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.667218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.667222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.667225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.667228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.667230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.667233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.667236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.667238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.667241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.667244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.667246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.667249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.667260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.667268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.667270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.667273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.667275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.667294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:26.673037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:26.674159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:26.674171Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:26.674173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:26.674176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:26.674183Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2452 [2:7559343214389285635:8444]
2025-10-09T21:52:26.674195Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:26.674212Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:26.674220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:26.674223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:26.674225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:26.674249Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:26.674257Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:26.674263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:26.674304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.674310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.674311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.674313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:26.674360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:26.674365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.674372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.674381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.674393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:26.674407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343214389285630:8444]
2025-10-09T21:52:26.674408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343214389285630:8444]
2025-10-09T21:52:26.674410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343214389285630:8444]
2025-10-09T21:52:26.674417Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:26.674427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:26.674430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:26.674430Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:26.674432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:26.674442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:26.674445Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:26.674461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343214751138861:2432], now have 1 active actors on pipe
2025-10-09T21:52:26.674467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343214751139060:4786], now have 1 active actors on pipe
2025-10-09T21:52:26.674483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343214389285630:8444]
2025-10-09T21:52:26.674491Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:26.674514Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:26.674501Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2452
2025-10-09T21:52:26.674512Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2452
2025-10-09T21:52:26.674592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:26.674599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:26.674620Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|149b6caa-2218a4b8-42774bed-7d0feffa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:26.674629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:26.674632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:26.674635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:26.674644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:26.674658Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:26.674833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:26.675021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:26.675033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:26.675054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:26.675166Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2453 partition: 0 MaxSeqNo: 25 sessionId: src_id|149b6caa-2218a4b8-42774bed-7d0feffa_0
2025-10-09T21:52:26.675521Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|45c52908-dae25b6-d7394bdd-65d65f6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|149b6caa-2218a4b8-42774bed-7d0feffa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:26.677898Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|149b6caa-2218a4b8-42774bed-7d0feffa_0] Write session: try to update token
2025-10-09T21:52:26.677914Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|149b6caa-2218a4b8-42774bed-7d0feffa_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:26.685326Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:26.685343Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:26.685353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343214389285630:8444]
2025-10-09T21:52:26.685399Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:26.685400Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:26.685598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:26.685759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:26.685918Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:26.686102Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:26.686336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:26.686379Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:26.690825Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2453 sessionId: src_id|149b6caa-2218a4b8-42774bed-7d0feffa_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:26.693021Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:26.693361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:26.693378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:26.693410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:26.693506Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:26.696912Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:26.696926Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:26.696929Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:26.696931Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:26.696939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343214389285630:8444]
2025-10-09T21:52:26.696942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343214389285630:8444]
2025-10-09T21:52:26.696947Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343214389285630:8444]
2025-10-09T21:52:26.696980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:26.696986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.696990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:26.697066Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:26.697080Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:26.697084Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:26.697120Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:26.745349Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343214751139154:5758]
2025-10-09T21:52:26.745635Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 51 pipe creation result: OK TabletId: 72075186224037892 Generation: 2452, pipe: [1:7559343214751139154:5758]
2025-10-09T21:52:26.844295Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:26.851111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:26.851140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:26.851304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:26.851374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:26.851443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:26.851504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:26.851544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:26.851558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:26.851630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:26.851688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:26.851750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:26.851811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:26.851850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:26.851856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:26.851912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:26.851962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:26.852019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:26.852073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:26.852113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:26.852119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:26.852176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:26.852231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:26.852290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:26.852344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:26.852383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:26.852391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:26.852449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:26.852503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:26.852556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:26.852613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:26.852652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:26.852658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:26.852710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:26.852763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:26.852825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:26.852890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:26.852936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:26.852944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:26.852998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:26.853047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:26.853114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:26.853169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:26.853215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:26.853223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:26.853298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:26.853359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:26.853418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:26.853473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:26.853509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:26.853514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:26.853573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:26.853632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:26.853687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:26.853741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:26.853778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:26.853784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:26.853837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:26.853889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:26.853942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:26.853995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:26.854052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:26.854059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:26.854097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343214751139154:5758], now have 1 active actors on pipe
2025-10-09T21:52:26.854373Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:26.854392Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:26.854396Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:26.854423Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2453 reason: error at write: Disconnected sessionId: src_id|149b6caa-2218a4b8-42774bed-7d0feffa_0
2025-10-09T21:52:26.854457Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 51 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343214751139154:5758]
2025-10-09T21:52:26.854604Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2453 sessionId: src_id|149b6caa-2218a4b8-42774bed-7d0feffa_0 is DEAD
2025-10-09T21:52:26.854725Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|149b6caa-2218a4b8-42774bed-7d0feffa_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:26.854737Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|149b6caa-2218a4b8-42774bed-7d0feffa_0] Write session will restart in 0.007613s
2025-10-09T21:52:26.854762Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|149b6caa-2218a4b8-42774bed-7d0feffa_0] Write session: Do CDS request
2025-10-09T21:52:26.854768Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|149b6caa-2218a4b8-42774bed-7d0feffa_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:26.854954Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:26.863419Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|149b6caa-2218a4b8-42774bed-7d0feffa_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:26.863509Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:26.863520Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2454
2025-10-09T21:52:26.863787Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2454 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:26.863819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2454 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:26.863830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2454 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:26.863834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:26.864190Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2454 sessionId:  describe result for acl check
2025-10-09T21:52:26.864232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:26.864237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:26.864239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:26.864244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343214751139201:4774] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:26.864658Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343214751139201:4774] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:26.867869Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343214751139201:4774] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:26.871387Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343214751139201:4774] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:26.871398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343214751139201:4774] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:26.871400Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343214751139201:4774] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:26.871405Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2454 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:26.875080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:26.877563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:26.877733Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:26.877741Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:26.877750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:26.877794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:26.877825Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343214389285922:8435]
2025-10-09T21:52:26.878254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:26.878565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:26.878599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:26.878813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:26.878848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.878854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.878855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.878855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.878856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.878857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.878857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.878858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.878859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.878862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.878869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.878871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.878874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.878876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.878878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.878880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.878882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.878884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.878889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.878894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:26.878898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.878902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:26.878905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.878908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:26.878910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.878914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:26.878917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.878920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:26.878923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.878926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:26.878929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.878932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.878935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.878938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.878941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.878944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.878959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:26.885655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:26.886983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:26.887008Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:26.887011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:26.887015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:26.887022Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2453 [2:7559343214389285922:8435]
2025-10-09T21:52:26.887037Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:26.887055Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:26.887058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:26.887061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:26.887064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:26.887080Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:26.887091Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:26.887099Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:26.887170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.887179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.887182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.887185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:26.887283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:26.887297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.887311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:26.887324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:26.887344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:26.887367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343214751139160:2432], now have 1 active actors on pipe
2025-10-09T21:52:26.887354Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2453
2025-10-09T21:52:26.887376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343214751139357:4774], now have 1 active actors on pipe
2025-10-09T21:52:26.887382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343214389285917:8435]
2025-10-09T21:52:26.887384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343214389285917:8435]
2025-10-09T21:52:26.887387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343214389285917:8435]
2025-10-09T21:52:26.887396Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2453
2025-10-09T21:52:26.887396Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:26.887406Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:26.887409Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:26.887410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:26.887411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:26.887426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:26.887429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:26.887485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:26.887489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:26.887500Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343214389285917:8435]
2025-10-09T21:52:26.887516Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:26.887527Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:26.887552Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7639fe8c-56747f85-cb6897fc-818b13b2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:26.887566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:26.887570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:26.887572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:26.887580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:26.887589Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:26.887614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:26.887749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:26.887760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:26.887774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:26.887839Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2454 partition: 0 MaxSeqNo: 25 sessionId: src_id|7639fe8c-56747f85-cb6897fc-818b13b2_0
2025-10-09T21:52:26.888169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|149b6caa-2218a4b8-42774bed-7d0feffa_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7639fe8c-56747f85-cb6897fc-818b13b2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:26.890468Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7639fe8c-56747f85-cb6897fc-818b13b2_0] Write session: try to update token
2025-10-09T21:52:26.890482Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7639fe8c-56747f85-cb6897fc-818b13b2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:26.902018Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:26.902060Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:26.902064Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:26.902066Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:26.902075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343214389285917:8435]
2025-10-09T21:52:26.902081Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343214389285917:8435]
2025-10-09T21:52:26.902086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343214389285917:8435]
2025-10-09T21:52:26.902133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:26.902136Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:26.902140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:26.902143Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:26.902145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:26.902147Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:26.902302Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:26.902742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2454 sessionId: src_id|7639fe8c-56747f85-cb6897fc-818b13b2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:26.905179Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:26.954741Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343214751139477:5758]
2025-10-09T21:52:26.954961Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 52 pipe creation result: OK TabletId: 72075186224037892 Generation: 2453, pipe: [1:7559343214751139477:5758]
2025-10-09T21:52:27.049768Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:27.050069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:27.050087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:27.050128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343214751139477:5758], now have 1 active actors on pipe
2025-10-09T21:52:27.050425Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 52 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343214751139477:5758]
2025-10-09T21:52:27.050433Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:27.050463Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:27.050465Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:27.050494Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:27.050532Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2454 reason: error at write: Disconnected sessionId: src_id|7639fe8c-56747f85-cb6897fc-818b13b2_0
2025-10-09T21:52:27.050736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2454 sessionId: src_id|7639fe8c-56747f85-cb6897fc-818b13b2_0 is DEAD
2025-10-09T21:52:27.050885Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7639fe8c-56747f85-cb6897fc-818b13b2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:27.050895Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7639fe8c-56747f85-cb6897fc-818b13b2_0] Write session will restart in 0.006315s
2025-10-09T21:52:27.050936Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7639fe8c-56747f85-cb6897fc-818b13b2_0] Write session: Do CDS request
2025-10-09T21:52:27.050943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7639fe8c-56747f85-cb6897fc-818b13b2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:27.058540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7639fe8c-56747f85-cb6897fc-818b13b2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:27.058635Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:27.058648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2455
2025-10-09T21:52:27.058798Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2455 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:27.058826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2455 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:27.058834Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2455 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:27.058837Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:27.059229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2455 sessionId:  describe result for acl check
2025-10-09T21:52:27.059258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:27.059260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:27.059262Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:27.059268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343219046106803:4728] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:27.059752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343219046106803:4728] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:27.063012Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343219046106803:4728] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:27.066388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343219046106803:4728] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:27.066402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343219046106803:4728] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:27.066407Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343219046106803:4728] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:27.066417Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2455 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:27.070027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:27.072437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:27.072591Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:27.072600Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:27.072608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:27.072653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:27.072693Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343218684253542:8404]
2025-10-09T21:52:27.073084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:27.073386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:27.073427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:27.073670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:27.073706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.073712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.073713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.073714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.073715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.073717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.073718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.073719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.073720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.073726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.073738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.073742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.073746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.073750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.073754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.073758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.073762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.073766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.073773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.073783Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.073790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.073798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.073802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.073808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.073814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.073819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.073826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.073831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.073841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.073847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.073856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.073861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.073870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.073875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.073882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.073892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.073921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:27.080346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:27.081631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:27.081648Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:27.081651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:27.081655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:27.081663Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2454 [2:7559343218684253542:8404]
2025-10-09T21:52:27.081678Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:27.081698Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:27.081705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:27.081708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:27.081710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:27.081730Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:27.081737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:27.081743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:27.081818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.081828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.081830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.081833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:27.081907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:27.081915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.081924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.081931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.081952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:27.081978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343219046106776:2432], now have 1 active actors on pipe
2025-10-09T21:52:27.081982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343219046106995:4728], now have 1 active actors on pipe
2025-10-09T21:52:27.081992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343218684253537:8404]
2025-10-09T21:52:27.081999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343218684253537:8404]
2025-10-09T21:52:27.082001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343218684253537:8404]
2025-10-09T21:52:27.082016Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:27.082043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:27.082046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:27.082054Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:27.082056Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:27.082060Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:27.082064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:27.082056Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2454
2025-10-09T21:52:27.082097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343218684253537:8404]
2025-10-09T21:52:27.082076Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2454
2025-10-09T21:52:27.082106Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:27.082107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:27.082122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:27.082127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:27.082178Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|90fb3bd2-6d6c1e6c-687df273-29da1bfa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:27.082191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:27.082197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:27.082199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:27.082211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:27.082226Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:27.082252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:27.082377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:27.082390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:27.082406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:27.082469Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2455 partition: 0 MaxSeqNo: 25 sessionId: src_id|90fb3bd2-6d6c1e6c-687df273-29da1bfa_0
2025-10-09T21:52:27.082830Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7639fe8c-56747f85-cb6897fc-818b13b2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|90fb3bd2-6d6c1e6c-687df273-29da1bfa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:27.085075Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90fb3bd2-6d6c1e6c-687df273-29da1bfa_0] Write session: try to update token
2025-10-09T21:52:27.085090Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90fb3bd2-6d6c1e6c-687df273-29da1bfa_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:27.096252Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:27.096272Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:27.096274Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:27.096275Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:27.096283Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343218684253537:8404]
2025-10-09T21:52:27.096287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343218684253537:8404]
2025-10-09T21:52:27.096291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343218684253537:8404]
2025-10-09T21:52:27.096331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:27.096338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.096340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:27.096342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:27.096345Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:27.096351Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:27.096486Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:27.097663Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2455 sessionId: src_id|90fb3bd2-6d6c1e6c-687df273-29da1bfa_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:27.100216Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:27.150808Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343219046107070:5758]
2025-10-09T21:52:27.151033Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 53 pipe creation result: OK TabletId: 72075186224037892 Generation: 2454, pipe: [1:7559343219046107070:5758]
2025-10-09T21:52:27.244815Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:27.245319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:27.245338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:27.245380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343219046107070:5758], now have 1 active actors on pipe
2025-10-09T21:52:27.245587Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:27.245609Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:27.245646Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 53 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343219046107070:5758]
2025-10-09T21:52:27.245675Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:27.245781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2455 reason: status is not ok: tablet will be restarted right now sessionId: src_id|90fb3bd2-6d6c1e6c-687df273-29da1bfa_0
2025-10-09T21:52:27.245966Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2455 sessionId: src_id|90fb3bd2-6d6c1e6c-687df273-29da1bfa_0 is DEAD
2025-10-09T21:52:27.246099Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90fb3bd2-6d6c1e6c-687df273-29da1bfa_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:27.246110Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90fb3bd2-6d6c1e6c-687df273-29da1bfa_0] Write session will restart in 0.009756s
2025-10-09T21:52:27.246131Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90fb3bd2-6d6c1e6c-687df273-29da1bfa_0] Write session: Do CDS request
2025-10-09T21:52:27.246136Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90fb3bd2-6d6c1e6c-687df273-29da1bfa_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:27.256440Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90fb3bd2-6d6c1e6c-687df273-29da1bfa_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:27.256552Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:27.256565Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2456
2025-10-09T21:52:27.256685Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2456 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:27.256712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2456 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:27.256717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2456 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:27.256720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:27.257161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2456 sessionId:  describe result for acl check
2025-10-09T21:52:27.257190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:27.257191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:27.257193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:27.257197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343219046107172:4727] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:27.257640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343219046107172:4727] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:27.260300Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343219046107172:4727] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:27.262483Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343219046107172:4727] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:27.262497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343219046107172:4727] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:27.262500Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343219046107172:4727] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:27.262508Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2456 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:27.264258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:27.266641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:27.266842Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:27.266851Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:27.266862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:27.266915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:27.266955Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343218684253828:8405]
2025-10-09T21:52:27.267475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:27.267896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:27.267950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:27.268276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:27.268348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.268352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.268353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.268355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.268356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.268357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.268358Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.268359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.268361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.268366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.268376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.268380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.268384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.268389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.268392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.268395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.268398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.268402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.268410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.268418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.268425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.268431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.268436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.268440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.268449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.268453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.268457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.268462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.268469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.268474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.268481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.268486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.268492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.268497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.268502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.268506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.268535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:27.275262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:27.276548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:27.276567Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:27.276570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:27.276575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:27.276581Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2455 [2:7559343218684253828:8405]
2025-10-09T21:52:27.276596Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:27.276615Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:27.276623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:27.276627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:27.276629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:27.276647Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:27.276658Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:27.276665Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:27.276729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.276736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.276739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.276742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:27.276815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:27.276823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.276836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.276848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.276869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:27.276895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343219046107355:4727], now have 1 active actors on pipe
2025-10-09T21:52:27.276905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343219046107076:2432], now have 1 active actors on pipe
2025-10-09T21:52:27.276913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343218684253823:8405]
2025-10-09T21:52:27.276915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343218684253823:8405]
2025-10-09T21:52:27.276917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343218684253823:8405]
2025-10-09T21:52:27.276930Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:27.276944Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:27.276955Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:27.276960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:27.276964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:27.276964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:27.276966Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:27.277000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343218684253823:8405]
2025-10-09T21:52:27.276970Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2455
2025-10-09T21:52:27.277009Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:27.276975Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2455
2025-10-09T21:52:27.277018Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:27.277045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:27.277055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:27.277090Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9f23b952-73b0efea-774f1552-1601aef6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:27.277101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:27.277105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:27.277108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:27.277120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:27.277133Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:27.277153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:27.277282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:27.277290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:27.277305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:27.277390Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2456 partition: 0 MaxSeqNo: 25 sessionId: src_id|9f23b952-73b0efea-774f1552-1601aef6_0
2025-10-09T21:52:27.277731Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90fb3bd2-6d6c1e6c-687df273-29da1bfa_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9f23b952-73b0efea-774f1552-1601aef6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:27.280037Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f23b952-73b0efea-774f1552-1601aef6_0] Write session: try to update token
2025-10-09T21:52:27.280054Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f23b952-73b0efea-774f1552-1601aef6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:27.291598Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:27.291621Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:27.291624Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:27.291625Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:27.291633Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343218684253823:8405]
2025-10-09T21:52:27.291637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343218684253823:8405]
2025-10-09T21:52:27.291640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343218684253823:8405]
2025-10-09T21:52:27.291678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:27.291687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.291689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:27.291700Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:27.291713Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:27.291717Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:27.291835Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:27.292764Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2456 sessionId: src_id|9f23b952-73b0efea-774f1552-1601aef6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:27.295238Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:27.345922Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343219046107385:5758]
2025-10-09T21:52:27.346180Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 54 pipe creation result: OK TabletId: 72075186224037892 Generation: 2455, pipe: [1:7559343219046107385:5758]
2025-10-09T21:52:27.434172Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:27.434596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:27.434609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:27.434646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343219046107385:5758], now have 1 active actors on pipe
2025-10-09T21:52:27.434827Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:27.434843Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 54 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343219046107385:5758]
2025-10-09T21:52:27.434844Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:27.434849Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:27.434874Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:27.434913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2456 reason: error at write: Disconnected sessionId: src_id|9f23b952-73b0efea-774f1552-1601aef6_0
2025-10-09T21:52:27.435069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2456 sessionId: src_id|9f23b952-73b0efea-774f1552-1601aef6_0 is DEAD
2025-10-09T21:52:27.435191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f23b952-73b0efea-774f1552-1601aef6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:27.435200Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f23b952-73b0efea-774f1552-1601aef6_0] Write session will restart in 0.009412s
2025-10-09T21:52:27.435218Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f23b952-73b0efea-774f1552-1601aef6_0] Write session: Do CDS request
2025-10-09T21:52:27.435223Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f23b952-73b0efea-774f1552-1601aef6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:27.445500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9f23b952-73b0efea-774f1552-1601aef6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:27.445578Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:27.445589Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2457
2025-10-09T21:52:27.445676Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2457 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:27.445700Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2457 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:27.445717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2457 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:27.445720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:27.446220Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2457 sessionId:  describe result for acl check
2025-10-09T21:52:27.446253Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:27.446257Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:27.446260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:27.446264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343219046107478:4717] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:27.446721Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343219046107478:4717] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:27.449086Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343219046107478:4717] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:27.451677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343219046107478:4717] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:27.451690Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343219046107478:4717] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:27.451693Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343219046107478:4717] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:27.451700Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2457 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:27.452738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:27.455195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:27.455383Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:27.455393Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:27.455405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:27.455452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:27.455482Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343218684254115:8402]
2025-10-09T21:52:27.455883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:27.456138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:27.456169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:27.456368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:27.456399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.456405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.456406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.456407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.456407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.456408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.456409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.456409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.456410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.456413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.456420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.456422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.456424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.456426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.456428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.456430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.456432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.456434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.456438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.456443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.456447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.456450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.456453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.456456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.456458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.456461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.456464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.456467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.456470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.456473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.456475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.456478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.456481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.456484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.456486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.456489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.456504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:27.462464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:27.463442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:27.463455Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:27.463457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:27.463461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:27.463467Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2456 [2:7559343218684254115:8402]
2025-10-09T21:52:27.463481Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:27.463496Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:27.463502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:27.463505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:27.463507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:27.463522Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:27.463533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:27.463540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:27.463756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.463764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.463766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.463769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:27.463822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:27.463833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.463843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.463854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.463870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:27.463889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343219046107388:2432], now have 1 active actors on pipe
2025-10-09T21:52:27.463897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343219046107670:4717], now have 1 active actors on pipe
2025-10-09T21:52:27.463903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343218684254110:8402]
2025-10-09T21:52:27.463905Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343218684254110:8402]
2025-10-09T21:52:27.463906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343218684254110:8402]
2025-10-09T21:52:27.463915Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:27.463925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:27.463930Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:27.463932Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:27.463934Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:27.463937Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:27.463939Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:27.463955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343218684254110:8402]
2025-10-09T21:52:27.463960Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:27.463946Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2456
2025-10-09T21:52:27.463954Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2456
2025-10-09T21:52:27.463983Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:27.464009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:27.464014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:27.464039Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5ed0bf80-eece017-855e0536-8feffec2_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:27.464052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:27.464060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:27.464062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:27.464073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:27.464085Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:27.464102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:27.464224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:27.464231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:27.464245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:27.464321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2457 partition: 0 MaxSeqNo: 25 sessionId: src_id|5ed0bf80-eece017-855e0536-8feffec2_0
2025-10-09T21:52:27.464603Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9f23b952-73b0efea-774f1552-1601aef6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5ed0bf80-eece017-855e0536-8feffec2_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:27.466813Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ed0bf80-eece017-855e0536-8feffec2_0] Write session: try to update token
2025-10-09T21:52:27.466826Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ed0bf80-eece017-855e0536-8feffec2_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:27.478506Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2457 sessionId: src_id|5ed0bf80-eece017-855e0536-8feffec2_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:27.479778Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:27.479795Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:27.479804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343218684254110:8402]
2025-10-09T21:52:27.479848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:27.479866Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:27.480019Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:27.480168Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:27.480299Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:27.480426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:27.480612Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:27.480646Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:27.480748Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:27.480879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:27.480889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:27.480913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:27.480978Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:27.487972Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:27.487985Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:27.487986Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:27.487988Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:27.487995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343218684254110:8402]
2025-10-09T21:52:27.487998Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343218684254110:8402]
2025-10-09T21:52:27.488000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343218684254110:8402]
2025-10-09T21:52:27.488027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:27.488036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.488038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:27.488039Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:27.488045Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:27.488060Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:27.488147Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:27.534853Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343219046107684:5758]
2025-10-09T21:52:27.535130Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 55 pipe creation result: OK TabletId: 72075186224037892 Generation: 2456, pipe: [1:7559343219046107684:5758]
2025-10-09T21:52:27.627473Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:27.634023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:27.634062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:27.634221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:27.634278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:27.634331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:27.634384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:27.634425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:27.634436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:27.634499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:27.634552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:27.634604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:27.634658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:27.634706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:27.634714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:27.634769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:27.634821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:27.634873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:27.634926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:27.634970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:27.634979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:27.635037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:27.635093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:27.635147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:27.635204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:27.635252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:27.635259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:27.635312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:27.635362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:27.635412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:27.635468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:27.635512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:27.635518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:27.635570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:27.635616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:27.635668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:27.635718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:27.635763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:27.635769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:27.635820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:27.635871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:27.635922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:27.635969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:27.636012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:27.636019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:27.636071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:27.636119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:27.636169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:27.636218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:27.636262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:27.636269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:27.636322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:27.636370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:27.636425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:27.636477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:27.636521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:27.636530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:27.636588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:27.636635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:27.636685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:27.636733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:27.636782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:27.636790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:27.636828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343219046107684:5758], now have 1 active actors on pipe
2025-10-09T21:52:27.637037Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:27.637086Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 55 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343219046107684:5758]
2025-10-09T21:52:27.637087Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:27.637089Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:27.637122Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2457 reason: error at write: Disconnected sessionId: src_id|5ed0bf80-eece017-855e0536-8feffec2_0
2025-10-09T21:52:27.637338Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2457 sessionId: src_id|5ed0bf80-eece017-855e0536-8feffec2_0 is DEAD
2025-10-09T21:52:27.637471Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ed0bf80-eece017-855e0536-8feffec2_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:27.637480Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ed0bf80-eece017-855e0536-8feffec2_0] Write session will restart in 0.008106s
2025-10-09T21:52:27.637513Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ed0bf80-eece017-855e0536-8feffec2_0] Write session: Do CDS request
2025-10-09T21:52:27.637520Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ed0bf80-eece017-855e0536-8feffec2_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:27.637649Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:27.646442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ed0bf80-eece017-855e0536-8feffec2_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:27.646814Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:27.646826Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2458
2025-10-09T21:52:27.646950Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2458 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:27.646975Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2458 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:27.646982Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2458 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:27.646985Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:27.647305Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2458 sessionId:  describe result for acl check
2025-10-09T21:52:27.647331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:27.647336Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:27.647337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:27.647341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343219046107739:4701] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:27.647781Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343219046107739:4701] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:27.650724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343219046107739:4701] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:27.653344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343219046107739:4701] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:27.653357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343219046107739:4701] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:27.653361Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343219046107739:4701] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:27.653368Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2458 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:27.656479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:27.658603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:27.658775Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:27.658784Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:27.658796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:27.658846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:27.658885Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343218684254403:8391]
2025-10-09T21:52:27.659448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:27.659743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:27.659785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:27.660084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:27.660126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.660133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.660134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.660135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.660136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.660137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.660138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.660139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.660140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.660145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.660156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.660160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.660164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.660168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.660171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.660175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.660179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.660183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.660189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.660201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.660211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.660216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.660220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.660229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.660238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.660243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.660251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.660255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.660264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.660268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.660277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.660286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.660292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.660301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.660310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.660315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.660342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:27.666360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:27.667681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:27.667695Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:27.667696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:27.667699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:27.667705Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2457 [2:7559343218684254403:8391]
2025-10-09T21:52:27.667713Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:27.667727Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:27.667730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:27.667734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:27.667736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:27.667754Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:27.667765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:27.667770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:27.667813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.667820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.667822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.667825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:27.667884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:27.667890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.667898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.667914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.667938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:27.667955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343219046107912:4701], now have 1 active actors on pipe
2025-10-09T21:52:27.667957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343219046107687:2432], now have 1 active actors on pipe
2025-10-09T21:52:27.667963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343218684254398:8391]
2025-10-09T21:52:27.667965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343218684254398:8391]
2025-10-09T21:52:27.667967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343218684254398:8391]
2025-10-09T21:52:27.667980Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:27.667994Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:27.667996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:27.667999Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:27.668001Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:27.668012Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:27.668020Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:27.668044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343218684254398:8391]
2025-10-09T21:52:27.668051Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:27.668055Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:27.668030Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2457
2025-10-09T21:52:27.668032Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2457
2025-10-09T21:52:27.668082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:27.668089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:27.668115Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|88caf58e-14510765-8645aaa4-f96b6612_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:27.668126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:27.668132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:27.668134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:27.668141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:27.668152Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:27.668166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:27.668304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:27.668313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:27.668325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:27.668391Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2458 partition: 0 MaxSeqNo: 25 sessionId: src_id|88caf58e-14510765-8645aaa4-f96b6612_0
2025-10-09T21:52:27.668677Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ed0bf80-eece017-855e0536-8feffec2_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|88caf58e-14510765-8645aaa4-f96b6612_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:27.670949Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88caf58e-14510765-8645aaa4-f96b6612_0] Write session: try to update token
2025-10-09T21:52:27.670965Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88caf58e-14510765-8645aaa4-f96b6612_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:27.681926Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:27.681944Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:27.681947Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:27.681949Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:27.681960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343218684254398:8391]
2025-10-09T21:52:27.681965Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343218684254398:8391]
2025-10-09T21:52:27.681969Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343218684254398:8391]
2025-10-09T21:52:27.682011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:27.682013Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:27.682018Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:27.682022Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:27.682023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.682043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:27.682164Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:27.682510Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2458 sessionId: src_id|88caf58e-14510765-8645aaa4-f96b6612_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:27.684693Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:27.737093Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343219046107984:5758]
2025-10-09T21:52:27.737366Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 56 pipe creation result: OK TabletId: 72075186224037892 Generation: 2457, pipe: [1:7559343219046107984:5758]
2025-10-09T21:52:27.821402Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:27.821648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:27.821665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:27.821706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343219046107984:5758], now have 1 active actors on pipe
2025-10-09T21:52:27.821901Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:27.821964Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:27.821968Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:27.822011Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 56 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343219046107984:5758]
2025-10-09T21:52:27.822065Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2458 reason: status is not ok: tablet will be restarted right now sessionId: src_id|88caf58e-14510765-8645aaa4-f96b6612_0
2025-10-09T21:52:27.822255Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2458 sessionId: src_id|88caf58e-14510765-8645aaa4-f96b6612_0 is DEAD
2025-10-09T21:52:27.822413Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88caf58e-14510765-8645aaa4-f96b6612_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:27.822424Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88caf58e-14510765-8645aaa4-f96b6612_0] Write session will restart in 0.006427s
2025-10-09T21:52:27.822445Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88caf58e-14510765-8645aaa4-f96b6612_0] Write session: Do CDS request
2025-10-09T21:52:27.822449Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88caf58e-14510765-8645aaa4-f96b6612_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:27.829464Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88caf58e-14510765-8645aaa4-f96b6612_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:27.829567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:27.829580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2459
2025-10-09T21:52:27.829700Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2459 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:27.829735Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2459 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:27.829741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2459 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:27.829744Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:27.830181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2459 sessionId:  describe result for acl check
2025-10-09T21:52:27.830212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:27.830219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:27.830221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:27.830226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343219046108020:4690] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:27.830767Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343219046108020:4690] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:27.833459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343219046108020:4690] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:27.836649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343219046108020:4690] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:27.836660Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343219046108020:4690] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:27.836664Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343219046108020:4690] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:27.836671Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2459 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:27.840564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:27.842741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:27.842936Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:27.842947Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:27.842960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:27.843015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:27.843059Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343218684254692:8399]
2025-10-09T21:52:27.843652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:27.843933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:27.843972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:27.844281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:27.844325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.844331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.844332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.844334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.844335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.844336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.844337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.844338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.844340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.844344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.844356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.844360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.844364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.844368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.844372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.844376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.844380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.844384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.844392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.844400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:27.844406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.844411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:27.844416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.844421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:27.844426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.844431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:27.844435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.844441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:27.844445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.844450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:27.844462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.844467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.844472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.844479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.844483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.844488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.844519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:27.850942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:27.852003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:27.852018Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:27.852021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:27.852025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:27.852032Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2458 [2:7559343218684254692:8399]
2025-10-09T21:52:27.852046Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:27.852066Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:27.852070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:27.852073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:27.852076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:27.852093Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:27.852104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:27.852110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:27.852169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.852177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.852179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.852183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:27.852249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:27.852256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.852269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:27.852286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:27.852309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:27.852334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343219046107987:2432], now have 1 active actors on pipe
2025-10-09T21:52:27.852343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343219046108186:4690], now have 1 active actors on pipe
2025-10-09T21:52:27.852392Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2458
2025-10-09T21:52:27.852408Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2458
2025-10-09T21:52:27.852350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343218684254687:8399]
2025-10-09T21:52:27.852357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343218684254687:8399]
2025-10-09T21:52:27.852359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343218684254687:8399]
2025-10-09T21:52:27.852372Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:27.852387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:27.852390Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:27.852393Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:27.852395Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:27.852400Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:27.852404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:27.852434Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343218684254687:8399]
2025-10-09T21:52:27.852437Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:27.852440Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:27.852456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:27.852462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:27.852511Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bc8ef463-7427b8f5-5ec33af9-407b6f81_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:27.852518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:27.852522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:27.852525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:27.852537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:27.852547Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:27.852569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:27.852734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:27.852742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:27.852755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:27.852867Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2459 partition: 0 MaxSeqNo: 25 sessionId: src_id|bc8ef463-7427b8f5-5ec33af9-407b6f81_0
2025-10-09T21:52:27.853221Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88caf58e-14510765-8645aaa4-f96b6612_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bc8ef463-7427b8f5-5ec33af9-407b6f81_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:27.855598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc8ef463-7427b8f5-5ec33af9-407b6f81_0] Write session: try to update token
2025-10-09T21:52:27.855613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc8ef463-7427b8f5-5ec33af9-407b6f81_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:27.866786Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:27.866806Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:27.866809Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:27.866811Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:27.866821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343218684254687:8399]
2025-10-09T21:52:27.866826Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343218684254687:8399]
2025-10-09T21:52:27.866830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343218684254687:8399]
2025-10-09T21:52:27.866872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:27.866879Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:27.866881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:27.866885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:27.866892Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:27.866896Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:27.867013Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:27.867468Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2459 sessionId: src_id|bc8ef463-7427b8f5-5ec33af9-407b6f81_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:27.869717Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:27.922408Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343219046108291:5758]
2025-10-09T21:52:27.922600Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 57 pipe creation result: OK TabletId: 72075186224037892 Generation: 2458, pipe: [1:7559343219046108291:5758]
2025-10-09T21:52:28.012119Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:28.012381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.012394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.012433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343219046108291:5758], now have 1 active actors on pipe
2025-10-09T21:52:28.012611Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:28.012676Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:28.012690Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:28.012708Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:28.012754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2459 reason: error at write: Disconnected sessionId: src_id|bc8ef463-7427b8f5-5ec33af9-407b6f81_0
2025-10-09T21:52:28.012754Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 57 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343219046108291:5758]
2025-10-09T21:52:28.012959Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2459 sessionId: src_id|bc8ef463-7427b8f5-5ec33af9-407b6f81_0 is DEAD
2025-10-09T21:52:28.013127Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc8ef463-7427b8f5-5ec33af9-407b6f81_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:28.013138Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc8ef463-7427b8f5-5ec33af9-407b6f81_0] Write session will restart in 0.008929s
2025-10-09T21:52:28.013169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc8ef463-7427b8f5-5ec33af9-407b6f81_0] Write session: Do CDS request
2025-10-09T21:52:28.013176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc8ef463-7427b8f5-5ec33af9-407b6f81_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:28.023498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc8ef463-7427b8f5-5ec33af9-407b6f81_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:28.023525Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:28.023540Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2460
2025-10-09T21:52:28.023713Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2460 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:28.023750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2460 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:28.023761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2460 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:28.023765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:28.024295Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2460 sessionId:  describe result for acl check
2025-10-09T21:52:28.024335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:28.024341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:28.024343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:28.024349Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343223341075682:4647] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:28.024977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343223341075682:4647] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:28.028119Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343223341075682:4647] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:28.031053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343223341075682:4647] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:28.031067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343223341075682:4647] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:28.031070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343223341075682:4647] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:28.031076Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2460 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:28.032886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:28.035467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:28.035641Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:28.035650Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:28.035662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:28.035711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:28.035750Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343222979222321:8367]
2025-10-09T21:52:28.036341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:28.036623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:28.036667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:28.036964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:28.037012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.037018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.037019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.037020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.037022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.037023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.037024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.037025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.037026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.037031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.037043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.037047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.037051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.037054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.037058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.037061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.037065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.037069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.037076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.037084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.037091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.037096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.037100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.037104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.037108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.037113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.037117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.037122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.037126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.037132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.037142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.037148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.037154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.037158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.037162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.037168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.037199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:28.043605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:28.044844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:28.044857Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:28.044860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:28.044863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:28.044870Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2459 [2:7559343222979222321:8367]
2025-10-09T21:52:28.044882Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:28.044903Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:28.044910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:28.044913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:28.044916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:28.044931Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:28.044942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:28.044950Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:28.045004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.045009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.045010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.045012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:28.045068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:28.045074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.045081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.045087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.045099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:28.045123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341075610:2432], now have 1 active actors on pipe
2025-10-09T21:52:28.045132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341075892:4647], now have 1 active actors on pipe
2025-10-09T21:52:28.045139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343222979222316:8367]
2025-10-09T21:52:28.045143Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343222979222316:8367]
2025-10-09T21:52:28.045146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343222979222316:8367]
2025-10-09T21:52:28.045158Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:28.045171Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:28.045172Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:28.045175Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:28.045176Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:28.045184Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:28.045186Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:28.045182Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2459
2025-10-09T21:52:28.045206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343222979222316:8367]
2025-10-09T21:52:28.045208Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2459
2025-10-09T21:52:28.045212Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:28.045218Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:28.045237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.045240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.045282Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4a9b1f50-f2d3b51b-dbc06821-d3c94b5f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:28.045295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:28.045304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:28.045307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:28.045318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:28.045333Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:28.045348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:28.045531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.045541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.045557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:28.045655Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2460 partition: 0 MaxSeqNo: 25 sessionId: src_id|4a9b1f50-f2d3b51b-dbc06821-d3c94b5f_0
2025-10-09T21:52:28.045954Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc8ef463-7427b8f5-5ec33af9-407b6f81_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4a9b1f50-f2d3b51b-dbc06821-d3c94b5f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:28.048446Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a9b1f50-f2d3b51b-dbc06821-d3c94b5f_0] Write session: try to update token
2025-10-09T21:52:28.048462Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a9b1f50-f2d3b51b-dbc06821-d3c94b5f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:28.059084Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:28.059102Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:28.059104Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:28.059105Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:28.059114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343222979222316:8367]
2025-10-09T21:52:28.059117Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343222979222316:8367]
2025-10-09T21:52:28.059120Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343222979222316:8367]
2025-10-09T21:52:28.059159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:28.059164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.059165Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:28.059169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:28.059172Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:28.059176Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:28.059295Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:28.061170Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2460 sessionId: src_id|4a9b1f50-f2d3b51b-dbc06821-d3c94b5f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:28.063400Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:28.112913Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343223341075907:5758]
2025-10-09T21:52:28.113216Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 58 pipe creation result: OK TabletId: 72075186224037892 Generation: 2459, pipe: [1:7559343223341075907:5758]
2025-10-09T21:52:28.205092Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:28.205370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.205384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.205423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341075907:5758], now have 1 active actors on pipe
2025-10-09T21:52:28.205599Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:28.205659Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:28.205677Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 58 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343223341075907:5758]
2025-10-09T21:52:28.205682Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:28.205703Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:28.205743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2460 reason: error at write: Disconnected sessionId: src_id|4a9b1f50-f2d3b51b-dbc06821-d3c94b5f_0
2025-10-09T21:52:28.205929Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2460 sessionId: src_id|4a9b1f50-f2d3b51b-dbc06821-d3c94b5f_0 is DEAD
2025-10-09T21:52:28.206103Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a9b1f50-f2d3b51b-dbc06821-d3c94b5f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:28.206115Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a9b1f50-f2d3b51b-dbc06821-d3c94b5f_0] Write session will restart in 0.009638s
2025-10-09T21:52:28.206154Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a9b1f50-f2d3b51b-dbc06821-d3c94b5f_0] Write session: Do CDS request
2025-10-09T21:52:28.206161Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a9b1f50-f2d3b51b-dbc06821-d3c94b5f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:28.216510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4a9b1f50-f2d3b51b-dbc06821-d3c94b5f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:28.216594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:28.216606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2461
2025-10-09T21:52:28.216722Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2461 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:28.216752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2461 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:28.216760Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2461 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:28.216763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:28.217119Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2461 sessionId:  describe result for acl check
2025-10-09T21:52:28.217145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:28.217150Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:28.217151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:28.217155Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343223341075990:4637] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:28.217588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343223341075990:4637] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:28.220200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343223341075990:4637] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:28.222650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343223341075990:4637] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:28.222662Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343223341075990:4637] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:28.222665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343223341075990:4637] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:28.222671Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2461 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:28.224960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:28.227298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:28.227475Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:28.227487Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:28.227498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:28.227547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:28.227586Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343222979222611:8375]
2025-10-09T21:52:28.228145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:28.228426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:28.228468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:28.228777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:28.228824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.228832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.228834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.228835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.228836Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.228837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.228839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.228840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.228841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.228846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.228857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.228861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.228865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.228869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.228874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.228878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.228881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.228886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.228896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.228909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.228919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.228929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.228938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.228943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.228952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.228959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.228967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.228972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.228981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.228991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.229000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.229010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.229014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.229018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.229024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.229034Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.229067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:28.235086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:28.236203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:28.236218Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:28.236220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:28.236225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:28.236231Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2460 [2:7559343222979222611:8375]
2025-10-09T21:52:28.236246Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:28.236261Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:28.236267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:28.236269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:28.236271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:28.236286Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:28.236296Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:28.236304Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:28.236361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.236368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.236370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.236373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:28.236571Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2460
2025-10-09T21:52:28.236579Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2460
2025-10-09T21:52:28.236682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:28.236693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.236706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.236718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.236736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:28.236762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341076168:4637], now have 1 active actors on pipe
2025-10-09T21:52:28.236771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341075909:2432], now have 1 active actors on pipe
2025-10-09T21:52:28.236786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.236791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.236809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343222979222606:8375]
2025-10-09T21:52:28.236817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343222979222606:8375]
2025-10-09T21:52:28.236819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343222979222606:8375]
2025-10-09T21:52:28.236833Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:28.236847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:28.236854Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:28.236857Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:28.236859Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:28.236869Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:28.236873Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:28.236893Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|de7f8560-f182de2f-7fec305a-637a37c9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:28.236899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:28.236903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:28.236906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:28.236916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:28.236929Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:28.236960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343222979222606:8375]
2025-10-09T21:52:28.236972Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:28.236982Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:28.236984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:28.237172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.237180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.237196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:28.237297Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2461 partition: 0 MaxSeqNo: 25 sessionId: src_id|de7f8560-f182de2f-7fec305a-637a37c9_0
2025-10-09T21:52:28.237616Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4a9b1f50-f2d3b51b-dbc06821-d3c94b5f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|de7f8560-f182de2f-7fec305a-637a37c9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:28.239916Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|de7f8560-f182de2f-7fec305a-637a37c9_0] Write session: try to update token
2025-10-09T21:52:28.239931Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|de7f8560-f182de2f-7fec305a-637a37c9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:28.247918Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:28.247941Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:28.247951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343222979222606:8375]
2025-10-09T21:52:28.247992Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:28.247995Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:28.248159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:28.248292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:28.248436Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:28.248573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:28.248754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:28.248793Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:28.251796Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2461 sessionId: src_id|de7f8560-f182de2f-7fec305a-637a37c9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:28.254018Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:28.254205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.254230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.254271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:28.254386Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:28.260276Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:28.260291Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:28.260294Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:28.260296Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:28.260306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343222979222606:8375]
2025-10-09T21:52:28.260309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343222979222606:8375]
2025-10-09T21:52:28.260313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343222979222606:8375]
2025-10-09T21:52:28.260350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:28.260353Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:28.260358Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:28.260361Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:28.260362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.260366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:28.260488Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:28.305921Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343223341076213:5758]
2025-10-09T21:52:28.306187Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 59 pipe creation result: OK TabletId: 72075186224037892 Generation: 2460, pipe: [1:7559343223341076213:5758]
2025-10-09T21:52:28.401712Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:28.408363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.408388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.408527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:28.408578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:28.408645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:28.408707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:28.408755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:28.408767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:28.408835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:28.408886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:28.408940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:28.408993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:28.409197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:28.409205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:28.409268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:28.409327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:28.409389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:28.409443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:28.409479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:28.409485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:28.409541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:28.409596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:28.409657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:28.409727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:28.409766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:28.409772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:28.409827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:28.409878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:28.409941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:28.410004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:28.410054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:28.410059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:28.410112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:28.410163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:28.410216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:28.410269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:28.410307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:28.410314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:28.410365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:28.410425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:28.410483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:28.410534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:28.410571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:28.410577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:28.410630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:28.410677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:28.410728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:28.410778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:28.410815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:28.410820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:28.410870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:28.410918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:28.410969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:28.411021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:28.411059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:28.411064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:28.411117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:28.411167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:28.411218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:28.411269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:28.411308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:28.411314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:28.411353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341076213:5758], now have 1 active actors on pipe
2025-10-09T21:52:28.411590Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 59 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343223341076213:5758]
2025-10-09T21:52:28.411591Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:28.411595Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:28.411626Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:28.411642Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2461 reason: error at write: Disconnected sessionId: src_id|de7f8560-f182de2f-7fec305a-637a37c9_0
2025-10-09T21:52:28.411821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2461 sessionId: src_id|de7f8560-f182de2f-7fec305a-637a37c9_0 is DEAD
2025-10-09T21:52:28.411862Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:28.411983Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de7f8560-f182de2f-7fec305a-637a37c9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:28.411995Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de7f8560-f182de2f-7fec305a-637a37c9_0] Write session will restart in 0.008561s
2025-10-09T21:52:28.412020Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de7f8560-f182de2f-7fec305a-637a37c9_0] Write session: Do CDS request
2025-10-09T21:52:28.412026Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de7f8560-f182de2f-7fec305a-637a37c9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:28.421462Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|de7f8560-f182de2f-7fec305a-637a37c9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:28.421547Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:28.421559Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2462
2025-10-09T21:52:28.421715Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2462 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:28.421753Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2462 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:28.421759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2462 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:28.421762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:28.422239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2462 sessionId:  describe result for acl check
2025-10-09T21:52:28.422283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:28.422289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:28.422291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:28.422298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343223341076291:4622] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:28.422844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343223341076291:4622] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:28.426023Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343223341076291:4622] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:28.429010Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343223341076291:4622] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:28.429022Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343223341076291:4622] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:28.429025Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343223341076291:4622] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:28.429033Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2462 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:28.430150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:28.432542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:28.432719Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:28.432728Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:28.432740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:28.432807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:28.432852Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343222979222904:8373]
2025-10-09T21:52:28.433409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:28.433721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:28.433766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:28.434090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:28.434144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.434149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.434151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.434152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.434153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.434154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.434156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.434157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.434158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.434163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.434173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.434177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.434180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.434184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.434187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.434190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.434194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.434197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.434204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.434212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.434221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.434231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.434240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.434245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.434249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.434259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.434268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.434275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.434283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.434289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.434298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.434306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.434314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.434319Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.434327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.434334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.434363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:28.440287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:28.441385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:28.441401Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:28.441404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:28.441408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:28.441413Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2461 [2:7559343222979222904:8373]
2025-10-09T21:52:28.441427Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:28.441442Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:28.441445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:28.441448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:28.441450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:28.441464Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:28.441476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:28.441483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:28.441540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.441548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.441549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.441552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:28.441616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:28.441624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.441635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.441650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.441671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:28.441696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341076220:2432], now have 1 active actors on pipe
2025-10-09T21:52:28.441704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341076507:4622], now have 1 active actors on pipe
2025-10-09T21:52:28.441712Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343222979222899:8373]
2025-10-09T21:52:28.441714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343222979222899:8373]
2025-10-09T21:52:28.441716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343222979222899:8373]
2025-10-09T21:52:28.441729Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:28.441743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:28.441748Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:28.441751Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:28.441753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:28.441761Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:28.441765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:28.441785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343222979222899:8373]
2025-10-09T21:52:28.441793Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:28.441794Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:28.441765Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2461
2025-10-09T21:52:28.441772Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2461
2025-10-09T21:52:28.441820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.441830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.441863Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5de7a252-53631b0c-d02719e5-2485b81e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:28.441876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:28.441879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:28.441882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:28.441892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:28.441902Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:28.441927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:28.442082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.442092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.442107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:28.442218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2462 partition: 0 MaxSeqNo: 25 sessionId: src_id|5de7a252-53631b0c-d02719e5-2485b81e_0
2025-10-09T21:52:28.442576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|de7f8560-f182de2f-7fec305a-637a37c9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5de7a252-53631b0c-d02719e5-2485b81e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:28.444742Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5de7a252-53631b0c-d02719e5-2485b81e_0] Write session: try to update token
2025-10-09T21:52:28.444755Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5de7a252-53631b0c-d02719e5-2485b81e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:28.455513Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:28.455537Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:28.455540Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:28.455542Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:28.455551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343222979222899:8373]
2025-10-09T21:52:28.455555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343222979222899:8373]
2025-10-09T21:52:28.455559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343222979222899:8373]
2025-10-09T21:52:28.455601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:28.455607Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:28.455612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.455615Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:28.455615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:28.455619Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:28.455740Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:28.456744Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2462 sessionId: src_id|5de7a252-53631b0c-d02719e5-2485b81e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:28.459097Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:28.511755Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343223341076522:5758]
2025-10-09T21:52:28.512035Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 60 pipe creation result: OK TabletId: 72075186224037892 Generation: 2461, pipe: [1:7559343223341076522:5758]
2025-10-09T21:52:28.597843Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:28.598397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.598410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.598446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341076522:5758], now have 1 active actors on pipe
2025-10-09T21:52:28.598615Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:28.598624Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:28.598666Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:28.598749Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2462 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5de7a252-53631b0c-d02719e5-2485b81e_0
2025-10-09T21:52:28.598752Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 60 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343223341076522:5758]
2025-10-09T21:52:28.598928Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2462 sessionId: src_id|5de7a252-53631b0c-d02719e5-2485b81e_0 is DEAD
2025-10-09T21:52:28.599038Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5de7a252-53631b0c-d02719e5-2485b81e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:28.599048Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5de7a252-53631b0c-d02719e5-2485b81e_0] Write session will restart in 0.009081s
2025-10-09T21:52:28.599068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5de7a252-53631b0c-d02719e5-2485b81e_0] Write session: Do CDS request
2025-10-09T21:52:28.599083Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5de7a252-53631b0c-d02719e5-2485b81e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:28.609446Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5de7a252-53631b0c-d02719e5-2485b81e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:28.609532Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:28.609546Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2463
2025-10-09T21:52:28.609665Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2463 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:28.609698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2463 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:28.609708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2463 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:28.609720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:28.610135Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2463 sessionId:  describe result for acl check
2025-10-09T21:52:28.610168Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:28.610175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:28.610177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:28.610183Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343223341076620:4608] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:28.610730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343223341076620:4608] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:28.613682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343223341076620:4608] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:28.616228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343223341076620:4608] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:28.616242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343223341076620:4608] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:28.616246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343223341076620:4608] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:28.616255Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2463 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:28.616952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:28.619225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:28.619403Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:28.619412Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:28.619424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:28.619472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:28.619512Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343222979223197:8371]
2025-10-09T21:52:28.620103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:28.620390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:28.620431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:28.620746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:28.620790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.620796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.620797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.620798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.620800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.620801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.620802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.620804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.620805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.620810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.620822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.620826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.620831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.620835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.620839Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.620843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.620847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.620850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.620857Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.620864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.620870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.620875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.620879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.620884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.620889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.620894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.620898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.620904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.620908Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.620914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.620922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.620927Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.620931Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.620936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.620941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.620948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.620979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:28.627174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:28.628175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:28.628191Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:28.628193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:28.628197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:28.628203Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2462 [2:7559343222979223197:8371]
2025-10-09T21:52:28.628216Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:28.628231Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:28.628234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:28.628237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:28.628240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:28.628256Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:28.628261Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:28.628268Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:28.628323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.628330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.628333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.628335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:28.628520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:28.628532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.628545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.628557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.628546Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2462
2025-10-09T21:52:28.628567Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2462
2025-10-09T21:52:28.628575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:28.628599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341076525:2432], now have 1 active actors on pipe
2025-10-09T21:52:28.628607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341076813:4608], now have 1 active actors on pipe
2025-10-09T21:52:28.628615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343222979223192:8371]
2025-10-09T21:52:28.628617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343222979223192:8371]
2025-10-09T21:52:28.628619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343222979223192:8371]
2025-10-09T21:52:28.628633Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:28.628645Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:28.628650Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:28.628663Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:28.628666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:28.628667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:28.628668Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:28.628689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.628699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.628732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343222979223192:8371]
2025-10-09T21:52:28.628743Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:28.628755Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:28.628760Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|87cd7146-c9e662fd-b43d8546-408c0023_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:28.628768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:28.628772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:28.628775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:28.628786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:28.628795Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:28.628832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:28.628985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.628994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.629008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:28.629092Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2463 partition: 0 MaxSeqNo: 25 sessionId: src_id|87cd7146-c9e662fd-b43d8546-408c0023_0
2025-10-09T21:52:28.629410Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5de7a252-53631b0c-d02719e5-2485b81e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|87cd7146-c9e662fd-b43d8546-408c0023_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:28.631650Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87cd7146-c9e662fd-b43d8546-408c0023_0] Write session: try to update token
2025-10-09T21:52:28.631663Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87cd7146-c9e662fd-b43d8546-408c0023_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:28.642899Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:28.642922Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:28.642925Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:28.642927Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:28.642937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343222979223192:8371]
2025-10-09T21:52:28.642942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343222979223192:8371]
2025-10-09T21:52:28.642946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343222979223192:8371]
2025-10-09T21:52:28.642991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:28.642992Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:28.642998Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:28.642998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.643001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:28.643001Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:28.643129Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:28.643343Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2463 sessionId: src_id|87cd7146-c9e662fd-b43d8546-408c0023_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:28.645698Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:28.699110Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343223341076827:5758]
2025-10-09T21:52:28.699357Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 61 pipe creation result: OK TabletId: 72075186224037892 Generation: 2462, pipe: [1:7559343223341076827:5758]
2025-10-09T21:52:28.787166Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:28.787742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.787758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.787797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341076827:5758], now have 1 active actors on pipe
2025-10-09T21:52:28.787989Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:28.787994Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 61 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343223341076827:5758]
2025-10-09T21:52:28.788029Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:28.788032Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:28.788093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2463 reason: error at write: Disconnected sessionId: src_id|87cd7146-c9e662fd-b43d8546-408c0023_0
2025-10-09T21:52:28.788284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2463 sessionId: src_id|87cd7146-c9e662fd-b43d8546-408c0023_0 is DEAD
2025-10-09T21:52:28.788456Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87cd7146-c9e662fd-b43d8546-408c0023_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:28.788467Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87cd7146-c9e662fd-b43d8546-408c0023_0] Write session will restart in 0.009437s
2025-10-09T21:52:28.788486Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87cd7146-c9e662fd-b43d8546-408c0023_0] Write session: Do CDS request
2025-10-09T21:52:28.788493Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87cd7146-c9e662fd-b43d8546-408c0023_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:28.788809Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:28.799527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|87cd7146-c9e662fd-b43d8546-408c0023_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:28.799567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:28.799579Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2464
2025-10-09T21:52:28.799756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2464 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:28.799810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2464 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:28.799819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2464 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:28.799823Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:28.800299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2464 sessionId:  describe result for acl check
2025-10-09T21:52:28.800335Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:28.800341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:28.800344Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:28.800357Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343223341076941:4596] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:28.800944Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343223341076941:4596] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:28.803497Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343223341076941:4596] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:28.805649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343223341076941:4596] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:28.805660Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343223341076941:4596] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:28.805663Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343223341076941:4596] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:28.805669Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2464 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:28.806605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:28.809091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:28.809270Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:28.809280Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:28.809292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:28.809340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:28.809379Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343222979223491:8369]
2025-10-09T21:52:28.809954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:28.810258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:28.810304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:28.810622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:28.810665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.810672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.810673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.810675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.810676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.810677Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.810678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.810679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.810680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.810685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.810696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.810701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.810705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.810708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.810712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.810716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.810720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.810724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.810732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.810740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:28.810751Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.810756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:28.810761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.810766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:28.810770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.810775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:28.810780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.810785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:28.810789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.810795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:28.810800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.810805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.810812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.810817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.810821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.810826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.810854Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:28.817049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:28.818265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:28.818280Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:28.818283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:28.818288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:28.818293Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2463 [2:7559343222979223491:8369]
2025-10-09T21:52:28.818306Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:28.818325Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:28.818333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:28.818336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:28.818339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:28.818355Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:28.818365Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:28.818377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:28.818704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.818712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.818715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.818718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:28.818787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:28.818795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.818809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:28.818827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:28.818846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:28.818867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341077119:4596], now have 1 active actors on pipe
2025-10-09T21:52:28.818875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341076830:2432], now have 1 active actors on pipe
2025-10-09T21:52:28.818883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343222979223486:8369]
2025-10-09T21:52:28.818885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343222979223486:8369]
2025-10-09T21:52:28.818887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343222979223486:8369]
2025-10-09T21:52:28.818900Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:28.818910Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:28.818918Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:28.818919Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:28.818920Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:28.818931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:28.818934Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:28.818959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343222979223486:8369]
2025-10-09T21:52:28.818966Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:28.818933Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2463
2025-10-09T21:52:28.818968Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:28.818934Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2463
2025-10-09T21:52:28.818991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.818997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.819030Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9d5512b5-dd80805a-296ef792-f6b246a0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:28.819043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:28.819048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:28.819050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:28.819062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:28.819076Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:28.819101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:28.819237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.819247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.819261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:28.819353Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2464 partition: 0 MaxSeqNo: 25 sessionId: src_id|9d5512b5-dd80805a-296ef792-f6b246a0_0
2025-10-09T21:52:28.819664Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|87cd7146-c9e662fd-b43d8546-408c0023_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9d5512b5-dd80805a-296ef792-f6b246a0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:28.821846Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d5512b5-dd80805a-296ef792-f6b246a0_0] Write session: try to update token
2025-10-09T21:52:28.821859Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d5512b5-dd80805a-296ef792-f6b246a0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:28.833233Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:28.833253Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:28.833255Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:28.833269Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:28.833278Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343222979223486:8369]
2025-10-09T21:52:28.833283Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343222979223486:8369]
2025-10-09T21:52:28.833287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343222979223486:8369]
2025-10-09T21:52:28.833328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:28.833335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:28.833335Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:28.833339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:28.833340Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:28.833345Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:28.833335Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2464 sessionId: src_id|9d5512b5-dd80805a-296ef792-f6b246a0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:28.833466Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:28.835751Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:28.888057Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343223341077133:5758]
2025-10-09T21:52:28.888323Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 62 pipe creation result: OK TabletId: 72075186224037892 Generation: 2463, pipe: [1:7559343223341077133:5758]
2025-10-09T21:52:28.979445Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:28.979668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:28.979685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:28.979726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341077133:5758], now have 1 active actors on pipe
2025-10-09T21:52:28.979904Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:28.979976Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:28.979988Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:28.980041Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2464 reason: error at write: Disconnected sessionId: src_id|9d5512b5-dd80805a-296ef792-f6b246a0_0
2025-10-09T21:52:28.980233Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2464 sessionId: src_id|9d5512b5-dd80805a-296ef792-f6b246a0_0 is DEAD
2025-10-09T21:52:28.980397Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d5512b5-dd80805a-296ef792-f6b246a0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:28.980408Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d5512b5-dd80805a-296ef792-f6b246a0_0] Write session will restart in 0.005728s
2025-10-09T21:52:28.980393Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:28.980434Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d5512b5-dd80805a-296ef792-f6b246a0_0] Write session: Do CDS request
2025-10-09T21:52:28.980441Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d5512b5-dd80805a-296ef792-f6b246a0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:28.980429Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 62 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343223341077133:5758]
2025-10-09T21:52:28.987512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9d5512b5-dd80805a-296ef792-f6b246a0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:28.987565Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:28.987578Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2465
2025-10-09T21:52:28.987730Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2465 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:28.987762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2465 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:28.987771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2465 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:28.987783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:28.988166Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2465 sessionId:  describe result for acl check
2025-10-09T21:52:28.988194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:28.988199Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:28.988200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:28.988204Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343223341077169:4554] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:28.988667Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343223341077169:4554] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:28.991490Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343223341077169:4554] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:28.994742Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343223341077169:4554] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:28.994754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343223341077169:4554] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:28.994758Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343223341077169:4554] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:28.994766Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2465 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:28.999616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:29.002653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:29.002843Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:29.002852Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:29.002862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:29.002906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:29.002939Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343227274191125:8352]
2025-10-09T21:52:29.003384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:29.003634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:29.003663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:29.003902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:29.003939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.003944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.003944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.003945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.003946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.003946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.003947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.003948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.003948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.003952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.003958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.003960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.003963Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.003965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.003967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.003969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.003971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.003973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.003978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.003983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.003986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.003990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.003992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.003995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.003998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.004001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.004005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.004008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.004011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.004015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.004018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.004021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.004023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.004026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.004029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.004031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.004047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:29.010461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:29.011728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:29.011744Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:29.011746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:29.011749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:29.011755Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2464 [2:7559343227274191125:8352]
2025-10-09T21:52:29.011764Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:29.011779Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:29.011781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:29.011784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:29.011785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:29.011798Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:29.011801Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:29.011806Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:29.011849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.011863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.011865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.011867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:29.011922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:29.011929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.011940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.011954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.011975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:29.011996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341077333:4554], now have 1 active actors on pipe
2025-10-09T21:52:29.012008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343222979223816:8352]
2025-10-09T21:52:29.012015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343222979223816:8352]
2025-10-09T21:52:29.012017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343222979223816:8352]
2025-10-09T21:52:29.012030Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:29.012045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343223341077153:2432], now have 1 active actors on pipe
2025-10-09T21:52:29.012056Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:29.012059Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:29.012063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:29.012065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:29.012065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:29.012068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:29.012064Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2464
2025-10-09T21:52:29.012095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343222979223816:8352]
2025-10-09T21:52:29.012069Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2464
2025-10-09T21:52:29.012103Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:29.012105Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:29.012131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:29.012141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:29.012180Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7640706f-79388480-7b84d6be-81f1424b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:29.012192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:29.012200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:29.012203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:29.012214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:29.012228Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:29.012254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:29.012405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:29.012415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:29.012435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:29.012551Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2465 partition: 0 MaxSeqNo: 25 sessionId: src_id|7640706f-79388480-7b84d6be-81f1424b_0
2025-10-09T21:52:29.012897Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9d5512b5-dd80805a-296ef792-f6b246a0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7640706f-79388480-7b84d6be-81f1424b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:29.015284Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7640706f-79388480-7b84d6be-81f1424b_0] Write session: try to update token
2025-10-09T21:52:29.015300Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7640706f-79388480-7b84d6be-81f1424b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:29.026172Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:29.026189Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:29.026191Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:29.026193Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:29.026203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343222979223816:8352]
2025-10-09T21:52:29.026206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343222979223816:8352]
2025-10-09T21:52:29.026209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343222979223816:8352]
2025-10-09T21:52:29.026244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:29.026252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.026253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:29.026254Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:29.026260Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:29.026264Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:29.026377Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:29.027306Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2465 sessionId: src_id|7640706f-79388480-7b84d6be-81f1424b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:29.029660Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:29.080580Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343227636044759:5758]
2025-10-09T21:52:29.080810Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 63 pipe creation result: OK TabletId: 72075186224037892 Generation: 2464, pipe: [1:7559343227636044759:5758]
2025-10-09T21:52:29.176467Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:29.176932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:29.176960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:29.177012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343227636044759:5758], now have 1 active actors on pipe
2025-10-09T21:52:29.177376Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:29.177379Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 63 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343227636044759:5758]
2025-10-09T21:52:29.177390Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:29.177454Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2465 reason: error at write: Disconnected sessionId: src_id|7640706f-79388480-7b84d6be-81f1424b_0
2025-10-09T21:52:29.177465Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:29.177613Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2465 sessionId: src_id|7640706f-79388480-7b84d6be-81f1424b_0 is DEAD
2025-10-09T21:52:29.177702Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:29.177770Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7640706f-79388480-7b84d6be-81f1424b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:29.177781Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7640706f-79388480-7b84d6be-81f1424b_0] Write session will restart in 0.009181s
2025-10-09T21:52:29.177801Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7640706f-79388480-7b84d6be-81f1424b_0] Write session: Do CDS request
2025-10-09T21:52:29.177818Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7640706f-79388480-7b84d6be-81f1424b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:29.188475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7640706f-79388480-7b84d6be-81f1424b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:29.188577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:29.188592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2466
2025-10-09T21:52:29.188708Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2466 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:29.188741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2466 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:29.188759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2466 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:29.188767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:29.189393Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2466 sessionId:  describe result for acl check
2025-10-09T21:52:29.189436Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:29.189446Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:29.189449Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:29.189455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343227636044854:4548] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:29.189997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343227636044854:4548] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:29.192547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343227636044854:4548] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:29.195665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343227636044854:4548] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:29.195677Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343227636044854:4548] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:29.195680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343227636044854:4548] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:29.195688Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2466 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:29.197355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:29.200504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:29.200747Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:29.200760Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:29.200771Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:29.200835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:29.200880Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343227274191420:8346]
2025-10-09T21:52:29.201575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:29.201851Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:29.201890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:29.202267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:29.202329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.202338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.202339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.202341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.202343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.202344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.202346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.202347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.202350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.202355Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.202367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.202371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.202375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.202379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.202383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.202388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.202392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.202397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.202403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.202413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.202425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.202432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.202443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.202449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.202460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.202465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.202470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.202476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.202481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.202486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.202493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.202498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.202515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.202521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.202528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.202534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.202567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:29.209303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:29.210994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:29.211011Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:29.211013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:29.211016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:29.211022Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2465 [2:7559343227274191420:8346]
2025-10-09T21:52:29.211032Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:29.211049Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:29.211051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:29.211053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:29.211055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:29.211069Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:29.211073Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:29.211079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:29.211130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.211136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.211138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.211140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:29.211200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:29.211206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.211215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.211221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.211230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:29.211248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343227274191415:8346]
2025-10-09T21:52:29.211252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343227274191415:8346]
2025-10-09T21:52:29.211254Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343227274191415:8346]
2025-10-09T21:52:29.211261Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:29.211273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:29.211279Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:29.211284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:29.211287Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:29.211290Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:29.211294Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:29.211306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343227636044762:2432], now have 1 active actors on pipe
2025-10-09T21:52:29.211313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343227636045052:4548], now have 1 active actors on pipe
2025-10-09T21:52:29.211348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343227274191415:8346]
2025-10-09T21:52:29.211359Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:29.211370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:29.211390Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2465
2025-10-09T21:52:29.211392Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2465
2025-10-09T21:52:29.211537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:29.211552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:29.211581Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cafc27df-7a5fd5dd-2675892-d4391cd6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:29.211593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:29.211598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:29.211606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:29.211618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:29.211631Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:29.211652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:29.211829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:29.211838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:29.211851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:29.211984Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2466 partition: 0 MaxSeqNo: 25 sessionId: src_id|cafc27df-7a5fd5dd-2675892-d4391cd6_0
2025-10-09T21:52:29.212325Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7640706f-79388480-7b84d6be-81f1424b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cafc27df-7a5fd5dd-2675892-d4391cd6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:29.214680Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cafc27df-7a5fd5dd-2675892-d4391cd6_0] Write session: try to update token
2025-10-09T21:52:29.214698Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cafc27df-7a5fd5dd-2675892-d4391cd6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:29.223283Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:29.223306Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:29.223315Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343227274191415:8346]
2025-10-09T21:52:29.223353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:29.223372Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:29.223544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:29.223702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:29.223853Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:29.224011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:29.224230Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:29.224283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:29.227619Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2466 sessionId: src_id|cafc27df-7a5fd5dd-2675892-d4391cd6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:29.230193Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:29.230398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:29.230420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:29.230449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:29.230526Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:29.236385Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:29.236409Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:29.236412Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:29.236415Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:29.236425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343227274191415:8346]
2025-10-09T21:52:29.236431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343227274191415:8346]
2025-10-09T21:52:29.236435Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343227274191415:8346]
2025-10-09T21:52:29.236470Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:29.236475Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:29.236476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:29.236479Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:29.236482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.236484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:29.236635Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:29.277636Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343227636045071:5758]
2025-10-09T21:52:29.277985Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 64 pipe creation result: OK TabletId: 72075186224037892 Generation: 2465, pipe: [1:7559343227636045071:5758]
2025-10-09T21:52:29.386047Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:29.393096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:29.393125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:29.393285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:29.393344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:29.393404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:29.393471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:29.393514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:29.393524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:29.393585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:29.393643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:29.393711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:29.393775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:29.393816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:29.393823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:29.393878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:29.393931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:29.393987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:29.394057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:29.394099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:29.394105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:29.394167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:29.394223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:29.394286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:29.394349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:29.394395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:29.394402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:29.394463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:29.394528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:29.394612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:29.394670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:29.394710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:29.394717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:29.394779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:29.394842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:29.394898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:29.394956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:29.394997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:29.395004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:29.395060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:29.395114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:29.395176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:29.395236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:29.395283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:29.395291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:29.395346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:29.395396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:29.395447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:29.395508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:29.395551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:29.395558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:29.395611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:29.395661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:29.395714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:29.395784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:29.395832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:29.395843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:29.395899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:29.395954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:29.396018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:29.396078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:29.396120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:29.396129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:29.396187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343227636045071:5758], now have 1 active actors on pipe
2025-10-09T21:52:29.396465Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:29.396513Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 64 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343227636045071:5758]
2025-10-09T21:52:29.396515Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:29.396519Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:29.396560Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2466 reason: error at write: Disconnected sessionId: src_id|cafc27df-7a5fd5dd-2675892-d4391cd6_0
2025-10-09T21:52:29.396731Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2466 sessionId: src_id|cafc27df-7a5fd5dd-2675892-d4391cd6_0 is DEAD
2025-10-09T21:52:29.396884Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cafc27df-7a5fd5dd-2675892-d4391cd6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:29.396897Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cafc27df-7a5fd5dd-2675892-d4391cd6_0] Write session will restart in 0.007237s
2025-10-09T21:52:29.396917Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cafc27df-7a5fd5dd-2675892-d4391cd6_0] Write session: Do CDS request
2025-10-09T21:52:29.396922Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cafc27df-7a5fd5dd-2675892-d4391cd6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:29.397209Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:29.405546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cafc27df-7a5fd5dd-2675892-d4391cd6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:29.405583Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:29.405595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2467
2025-10-09T21:52:29.405881Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2467 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:29.405926Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2467 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:29.405946Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2467 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:29.405956Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:29.406437Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2467 sessionId:  describe result for acl check
2025-10-09T21:52:29.406478Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:29.406484Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:29.406486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:29.406492Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343227636045113:4525] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:29.407073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343227636045113:4525] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:29.410278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343227636045113:4525] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:29.413118Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343227636045113:4525] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:29.413131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343227636045113:4525] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:29.413136Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343227636045113:4525] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:29.413144Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2467 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:29.422171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:29.422694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:29.422864Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:29.422872Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:29.422881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:29.422916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:29.422950Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343227274191718:8336]
2025-10-09T21:52:29.423463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:29.423769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:29.423817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:29.424100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:29.424148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.424154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.424155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.424156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.424157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.424158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.424159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.424160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.424161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.424164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.424171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.424173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.424176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.424178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.424180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.424182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.424184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.424186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.424192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.424197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.424201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.424204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.424207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.424211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.424216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.424220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.424223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.424226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.424229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.424233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.424236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.424240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.424243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.424246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.424249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.424252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.424270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:29.430758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:29.431991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:29.432007Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:29.432011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:29.432014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:29.432022Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2466 [2:7559343227274191718:8336]
2025-10-09T21:52:29.432038Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:29.432056Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:29.432063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:29.432067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:29.432070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:29.432087Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:29.432095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:29.432101Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:29.432161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.432169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.432171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.432174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:29.432249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:29.432257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.432266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.432277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.432292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:29.432310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343227636045078:2432], now have 1 active actors on pipe
2025-10-09T21:52:29.432319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343227636045252:4525], now have 1 active actors on pipe
2025-10-09T21:52:29.432326Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343227274191713:8336]
2025-10-09T21:52:29.432328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343227274191713:8336]
2025-10-09T21:52:29.432330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343227274191713:8336]
2025-10-09T21:52:29.432343Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:29.432354Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:29.432361Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:29.432373Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:29.432375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:29.432375Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:29.432379Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:29.432407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343227274191713:8336]
2025-10-09T21:52:29.432422Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:29.432406Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2466
2025-10-09T21:52:29.432450Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:29.432540Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2466
2025-10-09T21:52:29.432623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:29.432637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:29.432660Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|57a1a0c8-5ecf6475-16a35440-40623c0_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:29.432667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:29.432672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:29.432674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:29.432685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:29.432698Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:29.432718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:29.432870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:29.432883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:29.432912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:29.433015Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2467 partition: 0 MaxSeqNo: 25 sessionId: src_id|57a1a0c8-5ecf6475-16a35440-40623c0_0
2025-10-09T21:52:29.433385Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cafc27df-7a5fd5dd-2675892-d4391cd6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|57a1a0c8-5ecf6475-16a35440-40623c0_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:29.435835Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57a1a0c8-5ecf6475-16a35440-40623c0_0] Write session: try to update token
2025-10-09T21:52:29.435859Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57a1a0c8-5ecf6475-16a35440-40623c0_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:29.446847Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:29.446869Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:29.446871Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:29.446874Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:29.446882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343227274191713:8336]
2025-10-09T21:52:29.446886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343227274191713:8336]
2025-10-09T21:52:29.446889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343227274191713:8336]
2025-10-09T21:52:29.446922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:29.446928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.446929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:29.446928Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:29.446934Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:29.446941Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:29.447104Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:29.449043Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2467 sessionId: src_id|57a1a0c8-5ecf6475-16a35440-40623c0_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:29.451544Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:29.496843Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343227636045385:5758]
2025-10-09T21:52:29.497170Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 65 pipe creation result: OK TabletId: 72075186224037892 Generation: 2466, pipe: [1:7559343227636045385:5758]
2025-10-09T21:52:29.596676Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:29.597175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:29.597192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:29.597231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343227636045385:5758], now have 1 active actors on pipe
2025-10-09T21:52:29.597427Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:29.597517Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:29.597528Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:29.597538Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:29.597575Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 65 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343227636045385:5758]
2025-10-09T21:52:29.597590Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2467 reason: error at write: Disconnected sessionId: src_id|57a1a0c8-5ecf6475-16a35440-40623c0_0
2025-10-09T21:52:29.597759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2467 sessionId: src_id|57a1a0c8-5ecf6475-16a35440-40623c0_0 is DEAD
2025-10-09T21:52:29.597905Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57a1a0c8-5ecf6475-16a35440-40623c0_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:29.597915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57a1a0c8-5ecf6475-16a35440-40623c0_0] Write session will restart in 0.006822s
2025-10-09T21:52:29.597943Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57a1a0c8-5ecf6475-16a35440-40623c0_0] Write session: Do CDS request
2025-10-09T21:52:29.597948Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57a1a0c8-5ecf6475-16a35440-40623c0_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:29.605547Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|57a1a0c8-5ecf6475-16a35440-40623c0_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:29.605622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:29.605633Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2468
2025-10-09T21:52:29.605760Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2468 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:29.605792Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2468 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:29.605796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2468 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:29.605799Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:29.606216Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2468 sessionId:  describe result for acl check
2025-10-09T21:52:29.606255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:29.606261Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:29.606263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:29.606269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343227636045416:4523] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:29.606827Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343227636045416:4523] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:29.609939Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343227636045416:4523] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:29.613159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343227636045416:4523] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:29.613172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343227636045416:4523] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:29.613176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343227636045416:4523] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:29.613186Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2468 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:29.618098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:29.620644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:29.620819Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:29.620826Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:29.620834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:29.620877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:29.620909Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343227274192016:8334]
2025-10-09T21:52:29.621365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:29.621707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:29.621773Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:29.622075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:29.622121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.622126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.622127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.622127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.622128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.622129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.622130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.622131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.622131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.622135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.622142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.622144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.622147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.622149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.622151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.622154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.622156Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.622158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.622163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.622168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.622172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.622176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.622179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.622183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.622186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.622189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.622195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.622198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.622201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.622204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.622211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.622217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.622231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.622242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.622251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.622258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.622283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:29.628514Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:29.629737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:29.629759Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:29.629761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:29.629765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:29.629772Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2467 [2:7559343227274192016:8334]
2025-10-09T21:52:29.629783Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:29.629801Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:29.629808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:29.629810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:29.629812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:29.629830Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:29.629837Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:29.629844Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:29.629906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.629915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.629917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.629919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:29.629978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:29.629985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.629996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.630002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.630012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:29.630049Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343227274192011:8334]
2025-10-09T21:52:29.630059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343227274192011:8334]
2025-10-09T21:52:29.630062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343227274192011:8334]
2025-10-09T21:52:29.630072Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:29.630089Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:29.630095Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:29.630098Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:29.630100Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:29.630112Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:29.630115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:29.630124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343227636045387:2432], now have 1 active actors on pipe
2025-10-09T21:52:29.630133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343227636045573:4523], now have 1 active actors on pipe
2025-10-09T21:52:29.630151Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2467
2025-10-09T21:52:29.630174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343227274192011:8334]
2025-10-09T21:52:29.630170Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2467
2025-10-09T21:52:29.630186Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:29.630187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:29.630219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:29.630224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:29.630254Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3cc867a5-ced1860f-ae7902a8-aa0418c6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:29.630266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:29.630269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:29.630272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:29.630281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:29.630295Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:29.630314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:29.630449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:29.630461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:29.630473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:29.630578Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2468 partition: 0 MaxSeqNo: 25 sessionId: src_id|3cc867a5-ced1860f-ae7902a8-aa0418c6_0
2025-10-09T21:52:29.630915Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|57a1a0c8-5ecf6475-16a35440-40623c0_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3cc867a5-ced1860f-ae7902a8-aa0418c6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:29.633292Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3cc867a5-ced1860f-ae7902a8-aa0418c6_0] Write session: try to update token
2025-10-09T21:52:29.633309Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3cc867a5-ced1860f-ae7902a8-aa0418c6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:29.644282Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:29.644306Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:29.644308Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:29.644309Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:29.644316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343227274192011:8334]
2025-10-09T21:52:29.644319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343227274192011:8334]
2025-10-09T21:52:29.644322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343227274192011:8334]
2025-10-09T21:52:29.644358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:29.644358Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:29.644364Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:29.644364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.644367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:29.644367Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:29.644510Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:29.645238Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2468 sessionId: src_id|3cc867a5-ced1860f-ae7902a8-aa0418c6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:29.647399Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:29.697540Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343227636045695:5758]
2025-10-09T21:52:29.697866Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 66 pipe creation result: OK TabletId: 72075186224037892 Generation: 2467, pipe: [1:7559343227636045695:5758]
2025-10-09T21:52:29.804554Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:29.805175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:29.805191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:29.805235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343227636045695:5758], now have 1 active actors on pipe
2025-10-09T21:52:29.805457Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:29.805468Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:29.805494Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 66 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343227636045695:5758]
2025-10-09T21:52:29.805514Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:29.805578Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2468 reason: error at write: Disconnected sessionId: src_id|3cc867a5-ced1860f-ae7902a8-aa0418c6_0
2025-10-09T21:52:29.805614Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:29.805778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2468 sessionId: src_id|3cc867a5-ced1860f-ae7902a8-aa0418c6_0 is DEAD
2025-10-09T21:52:29.805925Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cc867a5-ced1860f-ae7902a8-aa0418c6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:29.805937Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cc867a5-ced1860f-ae7902a8-aa0418c6_0] Write session will restart in 0.006637s
2025-10-09T21:52:29.805966Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cc867a5-ced1860f-ae7902a8-aa0418c6_0] Write session: Do CDS request
2025-10-09T21:52:29.805973Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cc867a5-ced1860f-ae7902a8-aa0418c6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:29.813452Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3cc867a5-ced1860f-ae7902a8-aa0418c6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:29.813519Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:29.813533Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2469
2025-10-09T21:52:29.813669Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2469 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:29.813697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2469 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:29.813701Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2469 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:29.813704Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:29.814232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2469 sessionId:  describe result for acl check
2025-10-09T21:52:29.814272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:29.814281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:29.814283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:29.814289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343227636045722:4511] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:29.814855Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343227636045722:4511] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:29.817579Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343227636045722:4511] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:29.820777Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343227636045722:4511] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:29.820792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343227636045722:4511] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:29.820795Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343227636045722:4511] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:29.820802Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2469 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:29.825834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:29.828417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:29.828600Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:29.828612Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:29.828625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:29.828679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:29.828719Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343227274192315:8332]
2025-10-09T21:52:29.829322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:29.829658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:29.829703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:29.830021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:29.830079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.830085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.830087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.830088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.830089Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.830090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.830091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.830092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.830093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.830098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.830107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.830109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.830113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.830117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.830121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.830125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.830128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.830131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.830138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.830152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:29.830163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.830167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:29.830170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.830173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:29.830176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.830179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:29.830184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.830189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:29.830193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.830201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:29.830206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.830211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.830215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.830219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.830222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.830226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.830252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:29.836268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:29.837432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:29.837448Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:29.837451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:29.837455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:29.837461Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2468 [2:7559343227274192315:8332]
2025-10-09T21:52:29.837474Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:29.837488Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:29.837491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:29.837494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:29.837496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:29.837513Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:29.837523Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:29.837530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:29.837584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.837591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.837593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.837596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:29.837657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:29.837664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.837675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:29.837688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:29.837711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:29.837735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343227636045698:2432], now have 1 active actors on pipe
2025-10-09T21:52:29.837743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343227636045869:4511], now have 1 active actors on pipe
2025-10-09T21:52:29.837750Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343227274192310:8332]
2025-10-09T21:52:29.837757Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343227274192310:8332]
2025-10-09T21:52:29.837759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343227274192310:8332]
2025-10-09T21:52:29.837770Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:29.837782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:29.837785Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:29.837786Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:29.837787Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:29.837801Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:29.837809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:29.837828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343227274192310:8332]
2025-10-09T21:52:29.837795Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2468
2025-10-09T21:52:29.837836Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:29.837837Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:29.837796Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2468
2025-10-09T21:52:29.837851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:29.837855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:29.837909Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b3758e40-8f78dc20-f17435cb-f79a32f8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:29.837922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:29.837930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:29.837932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:29.837942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:29.837957Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:29.837979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:29.838157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:29.838167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:29.838182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:29.838268Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2469 partition: 0 MaxSeqNo: 25 sessionId: src_id|b3758e40-8f78dc20-f17435cb-f79a32f8_0
2025-10-09T21:52:29.838549Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3cc867a5-ced1860f-ae7902a8-aa0418c6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b3758e40-8f78dc20-f17435cb-f79a32f8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:29.840872Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3758e40-8f78dc20-f17435cb-f79a32f8_0] Write session: try to update token
2025-10-09T21:52:29.840890Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3758e40-8f78dc20-f17435cb-f79a32f8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:29.851685Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:29.851703Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:29.851706Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:29.851708Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:29.851718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343227274192310:8332]
2025-10-09T21:52:29.851722Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343227274192310:8332]
2025-10-09T21:52:29.851727Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343227274192310:8332]
2025-10-09T21:52:29.851764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:29.851769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:29.851771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:29.851770Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:29.851777Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:29.851781Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:29.851905Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:29.853119Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2469 sessionId: src_id|b3758e40-8f78dc20-f17435cb-f79a32f8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:29.855617Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:29.905877Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343227636046006:5758]
2025-10-09T21:52:29.906216Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 67 pipe creation result: OK TabletId: 72075186224037892 Generation: 2468, pipe: [1:7559343227636046006:5758]
2025-10-09T21:52:29.999493Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:29.999692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:29.999703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:29.999748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343227636046006:5758], now have 1 active actors on pipe
2025-10-09T21:52:29.999880Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:29.999882Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:29.999924Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 67 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343227636046006:5758]
2025-10-09T21:52:29.999927Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:29.999993Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2469 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b3758e40-8f78dc20-f17435cb-f79a32f8_0
2025-10-09T21:52:30.000155Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2469 sessionId: src_id|b3758e40-8f78dc20-f17435cb-f79a32f8_0 is DEAD
2025-10-09T21:52:30.000328Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3758e40-8f78dc20-f17435cb-f79a32f8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:30.000338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3758e40-8f78dc20-f17435cb-f79a32f8_0] Write session will restart in 0.005350s
2025-10-09T21:52:30.000360Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3758e40-8f78dc20-f17435cb-f79a32f8_0] Write session: Do CDS request
2025-10-09T21:52:30.000365Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3758e40-8f78dc20-f17435cb-f79a32f8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:30.006500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3758e40-8f78dc20-f17435cb-f79a32f8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:30.006594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:30.006609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2470
2025-10-09T21:52:30.006725Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2470 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:30.006769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2470 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:30.006779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2470 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:30.006783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:30.007151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2470 sessionId:  describe result for acl check
2025-10-09T21:52:30.007190Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:30.007192Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:30.007196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:30.007202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343231931013339:4489] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:30.007775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343231931013339:4489] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:30.009836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343231931013339:4489] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:30.013073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343231931013339:4489] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:30.013090Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343231931013339:4489] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:30.013094Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343231931013339:4489] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:30.013104Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2470 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:30.020492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:30.023200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:30.023415Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:30.023427Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:30.023441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:30.023499Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:30.023540Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343231569159929:8326]
2025-10-09T21:52:30.024165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:30.024443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:30.024492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:30.024802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:30.024858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.024864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.024865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.024865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.024867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.024868Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.024869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.024869Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.024870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.024876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.024885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.024889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.024893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.024896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.024900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.024903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.024906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.024910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.024916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.024923Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.024930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.024939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.024943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.024948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.024951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.024956Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.024960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.024964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.024973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.024977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.024982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.024987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.024995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.024999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.025008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.025013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.025038Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:30.031489Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:30.032571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:30.032591Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:30.032594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:30.032599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:30.032606Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2469 [2:7559343231569159929:8326]
2025-10-09T21:52:30.032620Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:30.032639Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:30.032647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:30.032651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:30.032654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:30.032673Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:30.032686Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:30.032693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:30.032751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.032759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.032762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.032765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:30.032838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:30.032847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.032862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.032880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.032904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:30.032933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343227636046015:2432], now have 1 active actors on pipe
2025-10-09T21:52:30.032942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343231931013453:4489], now have 1 active actors on pipe
2025-10-09T21:52:30.032949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343231569159924:8326]
2025-10-09T21:52:30.032952Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343231569159924:8326]
2025-10-09T21:52:30.032954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343231569159924:8326]
2025-10-09T21:52:30.032969Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:30.032988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:30.032990Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:30.032993Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:30.032996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:30.033007Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:30.033012Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:30.033007Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2469
2025-10-09T21:52:30.033056Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2469
2025-10-09T21:52:30.033052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343231569159924:8326]
2025-10-09T21:52:30.033066Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:30.033075Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:30.033205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:30.033218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:30.033247Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|579451b5-d53e97-71f8844-14dff671_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:30.033274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:30.033283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:30.033286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:30.033296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:30.033307Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:30.033327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:30.033518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:30.033528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:30.033544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:30.033632Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2470 partition: 0 MaxSeqNo: 25 sessionId: src_id|579451b5-d53e97-71f8844-14dff671_0
2025-10-09T21:52:30.033994Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3758e40-8f78dc20-f17435cb-f79a32f8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|579451b5-d53e97-71f8844-14dff671_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:30.036434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|579451b5-d53e97-71f8844-14dff671_0] Write session: try to update token
2025-10-09T21:52:30.036454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|579451b5-d53e97-71f8844-14dff671_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:30.044968Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:30.044985Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:30.044995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343231569159924:8326]
2025-10-09T21:52:30.045044Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:30.045049Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:30.045241Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:30.045402Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:30.045545Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:30.045685Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:30.045880Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:30.045920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:30.049513Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2470 sessionId: src_id|579451b5-d53e97-71f8844-14dff671_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:30.051833Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:30.052031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:30.052055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:30.052097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:30.052206Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:30.057897Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:30.057914Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:30.057917Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:30.057919Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:30.057933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343231569159924:8326]
2025-10-09T21:52:30.057937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343231569159924:8326]
2025-10-09T21:52:30.057942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343231569159924:8326]
2025-10-09T21:52:30.057985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:30.057998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.057998Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:30.058004Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:30.058006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:30.058008Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:30.058164Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:30.099973Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343231931013639:5758]
2025-10-09T21:52:30.100281Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 68 pipe creation result: OK TabletId: 72075186224037892 Generation: 2469, pipe: [1:7559343231931013639:5758]
2025-10-09T21:52:30.207694Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:30.215005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:30.215029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:30.215188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:30.215246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:30.215306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:30.215365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:30.215472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:30.215486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:30.215550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:30.215606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:30.215662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:30.215721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:30.215765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:30.215775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:30.215844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:30.215909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:30.215966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:30.216022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:30.216066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:30.216072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:30.216135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:30.216192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:30.216253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:30.216311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:30.216350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:30.216357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:30.216414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:30.216469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:30.216529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:30.216586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:30.216627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:30.216634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:30.216690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:30.216749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:30.216817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:30.216875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:30.216915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:30.216920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:30.216981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:30.217044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:30.217105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:30.217159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:30.217276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:30.217284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:30.217350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:30.217408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:30.217462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:30.217512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:30.217552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:30.217558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:30.217614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:30.217671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:30.217745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:30.217802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:30.217846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:30.217853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:30.217909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:30.217963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:30.218020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:30.218105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:30.218152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:30.218161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:30.218220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343231931013639:5758], now have 1 active actors on pipe
2025-10-09T21:52:30.218449Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:30.218454Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 68 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343231931013639:5758]
2025-10-09T21:52:30.218464Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:30.218475Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:30.218494Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2470 reason: error at write: Disconnected sessionId: src_id|579451b5-d53e97-71f8844-14dff671_0
2025-10-09T21:52:30.218690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2470 sessionId: src_id|579451b5-d53e97-71f8844-14dff671_0 is DEAD
2025-10-09T21:52:30.218821Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|579451b5-d53e97-71f8844-14dff671_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:30.218831Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|579451b5-d53e97-71f8844-14dff671_0] Write session will restart in 0.007858s
2025-10-09T21:52:30.218851Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|579451b5-d53e97-71f8844-14dff671_0] Write session: Do CDS request
2025-10-09T21:52:30.218856Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|579451b5-d53e97-71f8844-14dff671_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:30.219091Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:30.227503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|579451b5-d53e97-71f8844-14dff671_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:30.227615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:30.227631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2471
2025-10-09T21:52:30.227763Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2471 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:30.227811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2471 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:30.227822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2471 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:30.227826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:30.228309Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2471 sessionId:  describe result for acl check
2025-10-09T21:52:30.228346Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:30.228355Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:30.228356Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:30.228362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343231931013709:4471] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:30.228907Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343231931013709:4471] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:30.232079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343231931013709:4471] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:30.235522Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343231931013709:4471] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:30.235538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343231931013709:4471] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:30.235541Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343231931013709:4471] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:30.235550Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2471 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:30.237787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:30.240516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:30.240715Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:30.240725Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:30.240740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:30.240800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:30.240845Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343231569160254:8309]
2025-10-09T21:52:30.241456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:30.241746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:30.241795Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:30.242132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:30.242180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.242186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.242187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.242188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.242190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.242191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.242192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.242193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.242194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.242199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.242211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.242215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.242219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.242223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.242226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.242230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.242233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.242238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.242244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.242252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.242259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.242263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.242267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.242274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.242279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.242283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.242288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.242292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.242301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.242311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.242316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.242326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.242331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.242336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.242345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.242350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.242380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:30.248615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:30.249672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:30.249687Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:30.249690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:30.249694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:30.249700Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2470 [2:7559343231569160254:8309]
2025-10-09T21:52:30.249713Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:30.249736Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:30.249744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:30.249747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:30.249750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:30.249766Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:30.249774Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:30.249786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:30.249845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.249852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.249854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.249857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:30.249921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:30.249929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.249940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.249953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.249975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:30.249999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343231931013642:2432], now have 1 active actors on pipe
2025-10-09T21:52:30.250004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343231931013917:4471], now have 1 active actors on pipe
2025-10-09T21:52:30.250011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343231569160249:8309]
2025-10-09T21:52:30.250013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343231569160249:8309]
2025-10-09T21:52:30.250015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343231569160249:8309]
2025-10-09T21:52:30.250047Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:30.250052Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:30.250054Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:30.250056Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:30.250061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:30.250077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:30.250085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:30.250105Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343231569160249:8309]
2025-10-09T21:52:30.250114Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:30.250120Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:30.250100Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2470
2025-10-09T21:52:30.250138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:30.250147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:30.250178Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|83bc11f9-327f5a9c-189bb8eb-8a5d8ee7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:30.250189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:30.250197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:30.250200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:30.250211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:30.250225Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:30.250199Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2470
2025-10-09T21:52:30.250253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:30.250376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:30.250385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:30.250399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:30.250471Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2471 partition: 0 MaxSeqNo: 25 sessionId: src_id|83bc11f9-327f5a9c-189bb8eb-8a5d8ee7_0
2025-10-09T21:52:30.250802Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|579451b5-d53e97-71f8844-14dff671_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|83bc11f9-327f5a9c-189bb8eb-8a5d8ee7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:30.252971Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|83bc11f9-327f5a9c-189bb8eb-8a5d8ee7_0] Write session: try to update token
2025-10-09T21:52:30.252986Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|83bc11f9-327f5a9c-189bb8eb-8a5d8ee7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:30.263916Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:30.263932Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:30.263935Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:30.263937Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:30.263946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343231569160249:8309]
2025-10-09T21:52:30.263951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343231569160249:8309]
2025-10-09T21:52:30.263955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343231569160249:8309]
2025-10-09T21:52:30.263995Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:30.263997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:30.264004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.264006Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:30.264008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:30.264009Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:30.264134Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:30.264570Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2471 sessionId: src_id|83bc11f9-327f5a9c-189bb8eb-8a5d8ee7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:30.266850Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:30.318603Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343231931013956:5758]
2025-10-09T21:52:30.318893Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 69 pipe creation result: OK TabletId: 72075186224037892 Generation: 2470, pipe: [1:7559343231931013956:5758]
2025-10-09T21:52:30.413777Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:30.414418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:30.414431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:30.414471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343231931013956:5758], now have 1 active actors on pipe
2025-10-09T21:52:30.414664Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:30.414743Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:30.414761Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:30.414765Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:30.414798Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 69 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343231931013956:5758]
2025-10-09T21:52:30.414825Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2471 reason: error at write: Disconnected sessionId: src_id|83bc11f9-327f5a9c-189bb8eb-8a5d8ee7_0
2025-10-09T21:52:30.415021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2471 sessionId: src_id|83bc11f9-327f5a9c-189bb8eb-8a5d8ee7_0 is DEAD
2025-10-09T21:52:30.415154Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83bc11f9-327f5a9c-189bb8eb-8a5d8ee7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:30.415164Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83bc11f9-327f5a9c-189bb8eb-8a5d8ee7_0] Write session will restart in 0.005871s
2025-10-09T21:52:30.415186Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83bc11f9-327f5a9c-189bb8eb-8a5d8ee7_0] Write session: Do CDS request
2025-10-09T21:52:30.415191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83bc11f9-327f5a9c-189bb8eb-8a5d8ee7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:30.422419Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|83bc11f9-327f5a9c-189bb8eb-8a5d8ee7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:30.422515Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:30.422530Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2472
2025-10-09T21:52:30.422630Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2472 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:30.422661Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2472 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:30.422667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2472 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:30.422679Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:30.423102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2472 sessionId:  describe result for acl check
2025-10-09T21:52:30.423135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:30.423142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:30.423145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:30.423151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343231931013987:4454] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:30.423717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343231931013987:4454] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:30.426502Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343231931013987:4454] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:30.429618Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343231931013987:4454] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:30.429628Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343231931013987:4454] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:30.429633Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343231931013987:4454] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:30.429649Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2472 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:30.436203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:30.436649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:30.436821Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:30.436830Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:30.436842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:30.436896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:30.436934Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343231569160557:8307]
2025-10-09T21:52:30.437503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:30.437781Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:30.437819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:30.438122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:30.438168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.438173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.438173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.438174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.438175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.438176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.438176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.438177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.438177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.438181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.438188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.438190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.438192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.438194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.438197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.438199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.438201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.438203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.438208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.438213Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.438218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.438221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.438224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.438227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.438231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.438237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.438246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.438254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.438263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.438268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.438278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.438283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.438292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.438296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.438305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.438311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.438342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:30.444689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:30.445717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:30.445731Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:30.445733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:30.445737Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:30.445743Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2471 [2:7559343231569160557:8307]
2025-10-09T21:52:30.445757Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:30.445778Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:30.445786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:30.445789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:30.445791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:30.445807Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:30.445818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:30.445825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:30.445884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.445891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.445893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.445896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:30.445963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:30.445970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.445982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.445997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.446021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:30.446063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343231931014168:4454], now have 1 active actors on pipe
2025-10-09T21:52:30.446071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343231931013963:2432], now have 1 active actors on pipe
2025-10-09T21:52:30.446078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343231569160552:8307]
2025-10-09T21:52:30.446080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343231569160552:8307]
2025-10-09T21:52:30.446082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343231569160552:8307]
2025-10-09T21:52:30.446096Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:30.446111Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:30.446103Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2471
2025-10-09T21:52:30.446114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:30.446121Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:30.446105Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2471
2025-10-09T21:52:30.446123Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:30.446130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:30.446133Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:30.446164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343231569160552:8307]
2025-10-09T21:52:30.446171Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:30.446176Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:30.446188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:30.446198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:30.446241Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|52bae2f0-684f1698-cbc9d729-cc1a242d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:30.446253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:30.446262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:30.446265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:30.446277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:30.446291Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:30.446311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:30.446468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:30.446477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:30.446491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:30.446589Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2472 partition: 0 MaxSeqNo: 25 sessionId: src_id|52bae2f0-684f1698-cbc9d729-cc1a242d_0
2025-10-09T21:52:30.446924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|83bc11f9-327f5a9c-189bb8eb-8a5d8ee7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|52bae2f0-684f1698-cbc9d729-cc1a242d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:30.449185Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52bae2f0-684f1698-cbc9d729-cc1a242d_0] Write session: try to update token
2025-10-09T21:52:30.449198Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52bae2f0-684f1698-cbc9d729-cc1a242d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:30.460813Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:30.460831Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:30.460834Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:30.460837Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:30.460846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343231569160552:8307]
2025-10-09T21:52:30.460851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343231569160552:8307]
2025-10-09T21:52:30.460854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343231569160552:8307]
2025-10-09T21:52:30.460893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:30.460897Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:30.460901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.460902Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:30.460902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:30.460905Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:30.461049Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:30.461458Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2472 sessionId: src_id|52bae2f0-684f1698-cbc9d729-cc1a242d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:30.463584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:30.515189Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343231931014275:5758]
2025-10-09T21:52:30.515516Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 70 pipe creation result: OK TabletId: 72075186224037892 Generation: 2471, pipe: [1:7559343231931014275:5758]
2025-10-09T21:52:30.603421Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:30.603678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:30.603692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:30.603737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343231931014275:5758], now have 1 active actors on pipe
2025-10-09T21:52:30.603934Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 70 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343231931014275:5758]
2025-10-09T21:52:30.603951Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:30.603966Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:30.603969Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:30.603990Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:30.604033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2472 reason: error at write: Disconnected sessionId: src_id|52bae2f0-684f1698-cbc9d729-cc1a242d_0
2025-10-09T21:52:30.604231Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2472 sessionId: src_id|52bae2f0-684f1698-cbc9d729-cc1a242d_0 is DEAD
2025-10-09T21:52:30.604384Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52bae2f0-684f1698-cbc9d729-cc1a242d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:30.604394Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52bae2f0-684f1698-cbc9d729-cc1a242d_0] Write session will restart in 0.007080s
2025-10-09T21:52:30.604414Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52bae2f0-684f1698-cbc9d729-cc1a242d_0] Write session: Do CDS request
2025-10-09T21:52:30.604419Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52bae2f0-684f1698-cbc9d729-cc1a242d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:30.612538Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52bae2f0-684f1698-cbc9d729-cc1a242d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:30.612613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:30.612627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2473
2025-10-09T21:52:30.612776Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2473 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:30.612809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2473 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:30.612821Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2473 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:30.612824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:30.613377Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2473 sessionId:  describe result for acl check
2025-10-09T21:52:30.613412Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:30.613416Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:30.613417Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:30.613424Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343231931014322:4451] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:30.613952Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343231931014322:4451] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:30.616804Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343231931014322:4451] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:30.619603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343231931014322:4451] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:30.619616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343231931014322:4451] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:30.619620Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343231931014322:4451] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:30.619628Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2473 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:30.623513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:30.625684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:30.625858Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:30.625868Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:30.625880Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:30.625929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:30.625967Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343231569160860:8295]
2025-10-09T21:52:30.626572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:30.626843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:30.626885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:30.627182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:30.627230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.627237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.627239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.627240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.627241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.627242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.627244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.627245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.627246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.627251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.627262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.627266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.627271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.627274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.627279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.627282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.627286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.627290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.627297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.627305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.627311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.627316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.627321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.627326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.627336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.627341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.627346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.627351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.627356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.627366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.627371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.627376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.627384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.627390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.627398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.627404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.627441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:30.633478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:30.634649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:30.634663Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:30.634665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:30.634669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:30.634677Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2472 [2:7559343231569160860:8295]
2025-10-09T21:52:30.634690Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:30.634711Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:30.634719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:30.634722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:30.634724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:30.634740Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:30.634751Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:30.634758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:30.634813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.634820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.634822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.634825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:30.634892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:30.634899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.634911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.634926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.634951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:30.634970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343231931014278:2432], now have 1 active actors on pipe
2025-10-09T21:52:30.634978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343231931014497:4451], now have 1 active actors on pipe
2025-10-09T21:52:30.634986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343231569160855:8295]
2025-10-09T21:52:30.634992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343231569160855:8295]
2025-10-09T21:52:30.634994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343231569160855:8295]
2025-10-09T21:52:30.634984Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2472
2025-10-09T21:52:30.634985Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2472
2025-10-09T21:52:30.635003Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:30.635022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:30.635035Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:30.635036Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:30.635038Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:30.635039Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:30.635040Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:30.635075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:30.635085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:30.635102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343231569160855:8295]
2025-10-09T21:52:30.635115Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:30.635139Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:30.635145Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|49ad9f85-f1036c39-c9fafea9-eed706c6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:30.635153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:30.635156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:30.635159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:30.635169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:30.635180Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:30.635568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:30.635771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:30.635780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:30.635796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:30.635880Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2473 partition: 0 MaxSeqNo: 25 sessionId: src_id|49ad9f85-f1036c39-c9fafea9-eed706c6_0
2025-10-09T21:52:30.636201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52bae2f0-684f1698-cbc9d729-cc1a242d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|49ad9f85-f1036c39-c9fafea9-eed706c6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:30.638512Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|49ad9f85-f1036c39-c9fafea9-eed706c6_0] Write session: try to update token
2025-10-09T21:52:30.638527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|49ad9f85-f1036c39-c9fafea9-eed706c6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:30.649030Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:30.649042Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:30.649044Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:30.649048Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:30.649055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343231569160855:8295]
2025-10-09T21:52:30.649058Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343231569160855:8295]
2025-10-09T21:52:30.649061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343231569160855:8295]
2025-10-09T21:52:30.649097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:30.649106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.649112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:30.649117Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:30.649123Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:30.649133Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:30.649234Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:30.651080Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2473 sessionId: src_id|49ad9f85-f1036c39-c9fafea9-eed706c6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:30.653357Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:30.704156Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343231931014590:5758]
2025-10-09T21:52:30.704503Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 71 pipe creation result: OK TabletId: 72075186224037892 Generation: 2472, pipe: [1:7559343231931014590:5758]
2025-10-09T21:52:30.795560Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:30.795847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:30.795861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:30.795900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343231931014590:5758], now have 1 active actors on pipe
2025-10-09T21:52:30.796156Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 71 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343231931014590:5758]
2025-10-09T21:52:30.796169Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:30.796174Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:30.796198Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:30.796259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2473 reason: error at write: Disconnected sessionId: src_id|49ad9f85-f1036c39-c9fafea9-eed706c6_0
2025-10-09T21:52:30.796389Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:30.796442Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2473 sessionId: src_id|49ad9f85-f1036c39-c9fafea9-eed706c6_0 is DEAD
2025-10-09T21:52:30.796606Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49ad9f85-f1036c39-c9fafea9-eed706c6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:30.796618Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49ad9f85-f1036c39-c9fafea9-eed706c6_0] Write session will restart in 0.009143s
2025-10-09T21:52:30.796646Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49ad9f85-f1036c39-c9fafea9-eed706c6_0] Write session: Do CDS request
2025-10-09T21:52:30.796655Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49ad9f85-f1036c39-c9fafea9-eed706c6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:30.806608Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|49ad9f85-f1036c39-c9fafea9-eed706c6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:30.806690Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:30.806706Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2474
2025-10-09T21:52:30.806812Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2474 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:30.806844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2474 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:30.806857Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2474 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:30.806861Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:30.807320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2474 sessionId:  describe result for acl check
2025-10-09T21:52:30.807360Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:30.807367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:30.807369Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:30.807375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343231931014652:4433] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:30.807852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343231931014652:4433] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:30.810755Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343231931014652:4433] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:30.813605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343231931014652:4433] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:30.813618Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343231931014652:4433] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:30.813622Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343231931014652:4433] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:30.813631Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2474 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:30.816895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:30.819412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:30.819862Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:30.819872Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:30.819883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:30.819935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:30.819986Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343231569161173:8303]
2025-10-09T21:52:30.820535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:30.820811Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:30.820856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:30.821189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:30.821264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.821272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.821273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.821274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.821276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.821277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.821278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.821279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.821280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.821285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.821296Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.821301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.821305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.821309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.821313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.821317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.821321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.821325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.821332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.821340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:30.821350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.821356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:30.821364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.821369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:30.821378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.821383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:30.821393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.821399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:30.821403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.821413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:30.821419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.821424Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.821433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.821438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.821445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.821454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.821483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:30.827695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:30.829434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:30.829443Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:30.829444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:30.829447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:30.829452Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2473 [2:7559343231569161173:8303]
2025-10-09T21:52:30.829468Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:30.829481Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:30.829488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:30.829491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:30.829493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:30.829510Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:30.829521Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:30.829527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:30.829582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.829587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.829588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.829590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:30.829641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:30.829647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.829654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:30.829660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:30.829669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:30.829682Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343231569161163:8303]
2025-10-09T21:52:30.829684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343231569161163:8303]
2025-10-09T21:52:30.829685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343231569161163:8303]
2025-10-09T21:52:30.829692Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:30.829700Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:30.829702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:30.829702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:30.829704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:30.829710Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:30.829713Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:30.829721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343231931014841:4433], now have 1 active actors on pipe
2025-10-09T21:52:30.829724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343231931014593:2432], now have 1 active actors on pipe
2025-10-09T21:52:30.829743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343231569161163:8303]
2025-10-09T21:52:30.829751Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:30.829766Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:30.829785Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2473
2025-10-09T21:52:30.829790Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2473
2025-10-09T21:52:30.829889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:30.829901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:30.829933Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|c952230c-f6cd542a-c64e0aa9-27679bfa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:30.829947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:30.829951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:30.829954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:30.829965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:30.829979Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:30.829993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:30.830191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:30.830200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:30.830215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:30.830289Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2474 partition: 0 MaxSeqNo: 25 sessionId: src_id|c952230c-f6cd542a-c64e0aa9-27679bfa_0
2025-10-09T21:52:30.830615Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|49ad9f85-f1036c39-c9fafea9-eed706c6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|c952230c-f6cd542a-c64e0aa9-27679bfa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:30.837561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c952230c-f6cd542a-c64e0aa9-27679bfa_0] Write session: try to update token
2025-10-09T21:52:30.837579Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c952230c-f6cd542a-c64e0aa9-27679bfa_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:30.841331Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:30.841341Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:30.841348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343231569161163:8303]
2025-10-09T21:52:30.841383Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:30.841396Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:30.841527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:30.841657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:30.841777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:30.841907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:30.842103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:30.842141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:30.851695Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2474 sessionId: src_id|c952230c-f6cd542a-c64e0aa9-27679bfa_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:30.853640Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:30.853653Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:30.853655Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:30.853657Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:30.853666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343231569161163:8303]
2025-10-09T21:52:30.853670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343231569161163:8303]
2025-10-09T21:52:30.853673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343231569161163:8303]
2025-10-09T21:52:30.853715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:30.853711Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:30.853725Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:30.853725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:30.853727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:30.853731Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:30.853904Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:30.853957Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:30.896129Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343231931014909:5758]
2025-10-09T21:52:30.896392Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 72 pipe creation result: OK TabletId: 72075186224037892 Generation: 2473, pipe: [1:7559343231931014909:5758]
2025-10-09T21:52:31.002734Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:31.003250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.003264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.003289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343231931014909:5758], now have 1 active actors on pipe
2025-10-09T21:52:31.003497Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:31.003484Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:31.003499Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:31.003513Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 72 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343231931014909:5758]
2025-10-09T21:52:31.003579Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2474 reason: error at write: Disconnected sessionId: src_id|c952230c-f6cd542a-c64e0aa9-27679bfa_0
2025-10-09T21:52:31.003610Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:31.003761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2474 sessionId: src_id|c952230c-f6cd542a-c64e0aa9-27679bfa_0 is DEAD
2025-10-09T21:52:31.003899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c952230c-f6cd542a-c64e0aa9-27679bfa_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:31.003908Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c952230c-f6cd542a-c64e0aa9-27679bfa_0] Write session will restart in 0.005037s
2025-10-09T21:52:31.003940Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c952230c-f6cd542a-c64e0aa9-27679bfa_0] Write session: Do CDS request
2025-10-09T21:52:31.003945Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c952230c-f6cd542a-c64e0aa9-27679bfa_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:31.010496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|c952230c-f6cd542a-c64e0aa9-27679bfa_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:31.010499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:31.010603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:31.010616Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2475
2025-10-09T21:52:31.010730Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2475 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:31.010755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2475 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:31.010762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2475 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:31.010766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:31.011199Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2475 sessionId:  describe result for acl check
2025-10-09T21:52:31.011241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:31.011248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:31.011249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:31.011255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343236225982232:4429] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:31.011799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343236225982232:4429] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:31.012063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:31.012217Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:31.012224Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:31.012233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:31.012273Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:31.012305Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343235864128530:8299]
2025-10-09T21:52:31.012808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:31.013071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:31.013105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:31.013337Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:31.013371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.013375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.013376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.013377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.013378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.013378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.013379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.013380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.013381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.013384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.013391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.013393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.013396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.013398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.013400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.013404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.013406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.013408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.013413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.013418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.013423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.013427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.013429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.013433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.013435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.013438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.013441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.013444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.013448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.013454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.013458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.013463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.013467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.013472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.013478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.013484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.013508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:31.013705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343236225982232:4429] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:31.019352Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343236225982232:4429] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:31.019371Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343236225982232:4429] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:31.019375Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343236225982232:4429] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:31.019387Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2475 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:31.019656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:31.020607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:31.020620Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:31.020623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:31.020627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:31.020643Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2474 [2:7559343235864128530:8299]
2025-10-09T21:52:31.020654Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:31.020669Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:31.020674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:31.020679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:31.020681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:31.020694Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:31.020702Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:31.020708Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:31.020752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.020757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.020758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.020760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:31.020812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:31.020818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.020825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.020831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.020841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:31.020856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982268:4429], now have 1 active actors on pipe
2025-10-09T21:52:31.020863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982212:2432], now have 1 active actors on pipe
2025-10-09T21:52:31.020869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343235864128525:8299]
2025-10-09T21:52:31.020870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343235864128525:8299]
2025-10-09T21:52:31.020872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343235864128525:8299]
2025-10-09T21:52:31.020882Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:31.020898Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:31.020908Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:31.020911Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:31.020913Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:31.020916Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:31.020919Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:31.020931Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2474
2025-10-09T21:52:31.020946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343235864128525:8299]
2025-10-09T21:52:31.020943Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2474
2025-10-09T21:52:31.020954Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:31.020955Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:31.020978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.020986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.021024Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|860cf82c-c439c784-684b825-ca8024eb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:31.021034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:31.021037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:31.021038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:31.021046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:31.021057Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:31.021074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:31.021211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.021217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.021226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:31.021315Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2475 partition: 0 MaxSeqNo: 25 sessionId: src_id|860cf82c-c439c784-684b825-ca8024eb_0
2025-10-09T21:52:31.021636Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|c952230c-f6cd542a-c64e0aa9-27679bfa_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|860cf82c-c439c784-684b825-ca8024eb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:31.024125Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|860cf82c-c439c784-684b825-ca8024eb_0] Write session: try to update token
2025-10-09T21:52:31.024145Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|860cf82c-c439c784-684b825-ca8024eb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:31.036315Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:31.036331Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:31.036333Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:31.036334Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:31.036343Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343235864128525:8299]
2025-10-09T21:52:31.036349Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343235864128525:8299]
2025-10-09T21:52:31.036352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343235864128525:8299]
2025-10-09T21:52:31.036388Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:31.036392Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:31.036395Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:31.036396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:31.036404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.036407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:31.036535Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:31.038228Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2475 sessionId: src_id|860cf82c-c439c784-684b825-ca8024eb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:31.040579Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:31.103460Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343236225982301:5758]
2025-10-09T21:52:31.103728Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 73 pipe creation result: OK TabletId: 72075186224037892 Generation: 2474, pipe: [1:7559343236225982301:5758]
2025-10-09T21:52:31.185659Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:31.185928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.185945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.185990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982301:5758], now have 1 active actors on pipe
2025-10-09T21:52:31.186211Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:31.186234Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 73 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343236225982301:5758]
2025-10-09T21:52:31.186235Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:31.186240Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:31.186299Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:31.186303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2475 reason: error at write: Disconnected sessionId: src_id|860cf82c-c439c784-684b825-ca8024eb_0
2025-10-09T21:52:31.186466Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2475 sessionId: src_id|860cf82c-c439c784-684b825-ca8024eb_0 is DEAD
2025-10-09T21:52:31.186612Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|860cf82c-c439c784-684b825-ca8024eb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:31.186623Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|860cf82c-c439c784-684b825-ca8024eb_0] Write session will restart in 0.005596s
2025-10-09T21:52:31.186651Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|860cf82c-c439c784-684b825-ca8024eb_0] Write session: Do CDS request
2025-10-09T21:52:31.186657Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|860cf82c-c439c784-684b825-ca8024eb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:31.193513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|860cf82c-c439c784-684b825-ca8024eb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:31.193614Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:31.193627Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2476
2025-10-09T21:52:31.193771Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2476 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:31.193814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2476 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:31.193824Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2476 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:31.193827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:31.193818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:31.194248Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2476 sessionId:  describe result for acl check
2025-10-09T21:52:31.194277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:31.194284Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:31.194291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:31.194296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343236225982324:4403] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:31.194837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343236225982324:4403] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:31.195260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:31.195493Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:31.195505Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:31.195519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:31.195605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:31.195656Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343235864128624:8279]
2025-10-09T21:52:31.196248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:31.196526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:31.196575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:31.196884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:31.196946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.196952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.196953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.196954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.196955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.196957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.196958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.196959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.196960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.196966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.196978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.196982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.196986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.196990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.196994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.196998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.197002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.197007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.197014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.197023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.197030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.197035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.197040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.197045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.197049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.197054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.197058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.197063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.197070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.197075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.197080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.197095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.197106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.197115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.197122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.197129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.197164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:31.197472Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343236225982324:4403] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:31.203392Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343236225982324:4403] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:31.203409Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343236225982324:4403] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:31.203413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343236225982324:4403] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:31.203439Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2476 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:31.203738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:31.204763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:31.204774Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:31.204776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:31.204779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:31.204785Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2475 [2:7559343235864128624:8279]
2025-10-09T21:52:31.204798Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:31.204813Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:31.204821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:31.204824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:31.204825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:31.204840Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:31.204844Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:31.204851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:31.204895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.204900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.204902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.204904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:31.204964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:31.204970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.204978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.204992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.205013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:31.205037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982360:4403], now have 1 active actors on pipe
2025-10-09T21:52:31.205046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982304:2432], now have 1 active actors on pipe
2025-10-09T21:52:31.205054Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343235864128619:8279]
2025-10-09T21:52:31.205061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343235864128619:8279]
2025-10-09T21:52:31.205063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343235864128619:8279]
2025-10-09T21:52:31.205075Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:31.205096Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:31.205102Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:31.205093Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2475
2025-10-09T21:52:31.205105Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:31.205108Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:31.205111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:31.205114Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:31.205103Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2475
2025-10-09T21:52:31.205147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343235864128619:8279]
2025-10-09T21:52:31.205154Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:31.205157Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:31.205169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.205171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.205225Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f7a3cfad-30a77b4b-1e03eac-8a1b7e08_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:31.205237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:31.205239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:31.205241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:31.205248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:31.205272Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:31.205292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:31.205436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.205443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.205454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:31.205527Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2476 partition: 0 MaxSeqNo: 25 sessionId: src_id|f7a3cfad-30a77b4b-1e03eac-8a1b7e08_0
2025-10-09T21:52:31.205865Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|860cf82c-c439c784-684b825-ca8024eb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f7a3cfad-30a77b4b-1e03eac-8a1b7e08_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:31.208474Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7a3cfad-30a77b4b-1e03eac-8a1b7e08_0] Write session: try to update token
2025-10-09T21:52:31.208503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7a3cfad-30a77b4b-1e03eac-8a1b7e08_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:31.219135Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:31.219148Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:31.219150Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:31.219151Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:31.219161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343235864128619:8279]
2025-10-09T21:52:31.219164Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343235864128619:8279]
2025-10-09T21:52:31.219170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343235864128619:8279]
2025-10-09T21:52:31.219204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:31.219210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.219212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:31.219212Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:31.219260Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:31.219264Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:31.219337Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:31.225052Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2476 sessionId: src_id|f7a3cfad-30a77b4b-1e03eac-8a1b7e08_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:31.227583Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:31.286279Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343236225982373:5758]
2025-10-09T21:52:31.286611Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 74 pipe creation result: OK TabletId: 72075186224037892 Generation: 2475, pipe: [1:7559343236225982373:5758]
2025-10-09T21:52:31.366561Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:31.366893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.366908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.366953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982373:5758], now have 1 active actors on pipe
2025-10-09T21:52:31.367145Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:31.367174Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:31.367204Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 74 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343236225982373:5758]
2025-10-09T21:52:31.367225Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:31.367307Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2476 reason: status is not ok: tablet will be restarted right now sessionId: src_id|f7a3cfad-30a77b4b-1e03eac-8a1b7e08_0
2025-10-09T21:52:31.367478Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2476 sessionId: src_id|f7a3cfad-30a77b4b-1e03eac-8a1b7e08_0 is DEAD
2025-10-09T21:52:31.367662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7a3cfad-30a77b4b-1e03eac-8a1b7e08_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:31.367679Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7a3cfad-30a77b4b-1e03eac-8a1b7e08_0] Write session will restart in 0.005614s
2025-10-09T21:52:31.367711Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7a3cfad-30a77b4b-1e03eac-8a1b7e08_0] Write session: Do CDS request
2025-10-09T21:52:31.367717Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7a3cfad-30a77b4b-1e03eac-8a1b7e08_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:31.374472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f7a3cfad-30a77b4b-1e03eac-8a1b7e08_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:31.374533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:31.374567Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:31.374581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2477
2025-10-09T21:52:31.374764Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2477 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:31.374801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2477 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:31.374811Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2477 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:31.374816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:31.375181Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2477 sessionId:  describe result for acl check
2025-10-09T21:52:31.375215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:31.375222Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:31.375224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:31.375230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343236225982400:4395] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:31.375719Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343236225982400:4395] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:31.375856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:31.376011Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:31.376019Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:31.376027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:31.376066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:31.376095Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343235864128684:8277]
2025-10-09T21:52:31.376463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:31.376716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:31.376746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:31.376997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:31.377051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.377057Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.377059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.377060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.377061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.377063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.377064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.377065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.377068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.377073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.377082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.377086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.377090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.377094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.377098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.377100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.377102Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.377104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.377108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.377113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.377118Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.377121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.377125Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.377128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.377130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.377133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.377136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.377141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.377145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.377150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.377157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.377162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.377166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.377171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.377176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.377181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.377209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:31.377748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343236225982400:4395] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:31.382722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343236225982400:4395] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:31.382733Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343236225982400:4395] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:31.382735Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343236225982400:4395] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:31.382742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2477 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:31.383093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:31.384073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:31.384086Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:31.384091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:31.384095Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:31.384101Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2476 [2:7559343235864128684:8277]
2025-10-09T21:52:31.384114Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:31.384129Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:31.384132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:31.384134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:31.384137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:31.384153Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:31.384159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:31.384165Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:31.384221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.384227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.384229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.384232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:31.384293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:31.384300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.384311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.384326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.384346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:31.384369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982439:4395], now have 1 active actors on pipe
2025-10-09T21:52:31.384377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982379:2432], now have 1 active actors on pipe
2025-10-09T21:52:31.384384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343235864128679:8277]
2025-10-09T21:52:31.384386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343235864128679:8277]
2025-10-09T21:52:31.384388Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343235864128679:8277]
2025-10-09T21:52:31.384402Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:31.384416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:31.384416Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:31.384418Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:31.384420Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:31.384435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:31.384406Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2476
2025-10-09T21:52:31.384438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:31.384416Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2476
2025-10-09T21:52:31.384473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.384481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.384497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343235864128679:8277]
2025-10-09T21:52:31.384507Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:31.384525Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:31.384537Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|88856d0e-6a3eccae-94348888-a34592e9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:31.384543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:31.384547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:31.384549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:31.384558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:31.384567Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:31.384592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:31.384754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.384761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.384773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:31.384852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2477 partition: 0 MaxSeqNo: 25 sessionId: src_id|88856d0e-6a3eccae-94348888-a34592e9_0
2025-10-09T21:52:31.385169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f7a3cfad-30a77b4b-1e03eac-8a1b7e08_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|88856d0e-6a3eccae-94348888-a34592e9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:31.387583Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88856d0e-6a3eccae-94348888-a34592e9_0] Write session: try to update token
2025-10-09T21:52:31.387601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88856d0e-6a3eccae-94348888-a34592e9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:31.397917Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:31.397929Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:31.397931Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:31.397932Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:31.397943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343235864128679:8277]
2025-10-09T21:52:31.397946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343235864128679:8277]
2025-10-09T21:52:31.397950Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343235864128679:8277]
2025-10-09T21:52:31.397983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:31.397991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.397995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:31.398000Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:31.398007Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:31.398014Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:31.398139Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:31.400176Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2477 sessionId: src_id|88856d0e-6a3eccae-94348888-a34592e9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:31.402632Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:31.467506Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343236225982447:5758]
2025-10-09T21:52:31.467862Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 75 pipe creation result: OK TabletId: 72075186224037892 Generation: 2476, pipe: [1:7559343236225982447:5758]
2025-10-09T21:52:31.551676Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:31.551955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.551970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.552021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982447:5758], now have 1 active actors on pipe
2025-10-09T21:52:31.552193Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:31.552242Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:31.552253Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:31.552253Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 75 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343236225982447:5758]
2025-10-09T21:52:31.552274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:31.552307Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2477 reason: error at write: Disconnected sessionId: src_id|88856d0e-6a3eccae-94348888-a34592e9_0
2025-10-09T21:52:31.552460Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2477 sessionId: src_id|88856d0e-6a3eccae-94348888-a34592e9_0 is DEAD
2025-10-09T21:52:31.552565Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88856d0e-6a3eccae-94348888-a34592e9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:31.552574Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88856d0e-6a3eccae-94348888-a34592e9_0] Write session will restart in 0.006858s
2025-10-09T21:52:31.552602Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88856d0e-6a3eccae-94348888-a34592e9_0] Write session: Do CDS request
2025-10-09T21:52:31.552607Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88856d0e-6a3eccae-94348888-a34592e9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:31.558924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:31.560354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:31.560539Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:31.560547Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:31.560556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:31.560602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:31.560625Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|88856d0e-6a3eccae-94348888-a34592e9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:31.560633Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343235864128742:8275]
2025-10-09T21:52:31.560728Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:31.560743Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2478
2025-10-09T21:52:31.560873Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2478 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:31.560903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2478 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:31.560909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2478 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:31.560913Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:31.561029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:31.561287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:31.561317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:31.561348Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2478 sessionId:  describe result for acl check
2025-10-09T21:52:31.561381Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:31.561386Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:31.561388Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:31.561394Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343236225982473:4368] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:31.561533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:31.561570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.561574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.561575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.561576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.561577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.561577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.561578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.561579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.561580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.561583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.561589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.561592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.561594Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.561596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.561599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.561601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.561603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.561605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.561609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.561614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.561618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.561623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.561626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.561629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.561632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.561635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.561638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.561641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.561643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.561646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.561649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.561652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.561655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.561659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.561662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.561665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.561685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:31.561946Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343236225982473:4368] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:31.563939Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343236225982473:4368] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:31.567566Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343236225982473:4368] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:31.567577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343236225982473:4368] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:31.567581Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343236225982473:4368] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:31.567588Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2478 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:31.567877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:31.568829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:31.568845Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:31.568848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:31.568853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:31.568860Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2477 [2:7559343235864128742:8275]
2025-10-09T21:52:31.568873Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:31.568891Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:31.568901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:31.568904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:31.568906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:31.568919Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:31.568923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:31.568928Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:31.568969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.568975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.568976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.568978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:31.569024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:31.569030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.569038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.569044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.569054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:31.569072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982449:2432], now have 1 active actors on pipe
2025-10-09T21:52:31.569078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982509:4368], now have 1 active actors on pipe
2025-10-09T21:52:31.569083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343235864128737:8275]
2025-10-09T21:52:31.569084Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343235864128737:8275]
2025-10-09T21:52:31.569085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343235864128737:8275]
2025-10-09T21:52:31.569094Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:31.569104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:31.569109Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:31.569111Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:31.569113Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:31.569113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:31.569115Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:31.569134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343235864128737:8275]
2025-10-09T21:52:31.569136Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:31.569129Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2477
2025-10-09T21:52:31.569151Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:31.569159Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2477
2025-10-09T21:52:31.569196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.569204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.569229Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|98b5da02-a047a04-3130821-b820844_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:31.569240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:31.569243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:31.569245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:31.569264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:31.569275Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:31.569287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:31.569429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.569441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.569461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:31.569554Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2478 partition: 0 MaxSeqNo: 25 sessionId: src_id|98b5da02-a047a04-3130821-b820844_0
2025-10-09T21:52:31.569843Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|88856d0e-6a3eccae-94348888-a34592e9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|98b5da02-a047a04-3130821-b820844_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:31.572092Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98b5da02-a047a04-3130821-b820844_0] Write session: try to update token
2025-10-09T21:52:31.572108Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98b5da02-a047a04-3130821-b820844_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:31.582907Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:31.582922Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:31.582926Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:31.582927Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:31.582935Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343235864128737:8275]
2025-10-09T21:52:31.582939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343235864128737:8275]
2025-10-09T21:52:31.582942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343235864128737:8275]
2025-10-09T21:52:31.582984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:31.582994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.582998Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:31.583002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:31.583002Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:31.583006Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:31.583128Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:31.584381Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2478 sessionId: src_id|98b5da02-a047a04-3130821-b820844_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:31.586938Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:31.652295Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343236225982518:5758]
2025-10-09T21:52:31.652562Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 76 pipe creation result: OK TabletId: 72075186224037892 Generation: 2477, pipe: [1:7559343236225982518:5758]
2025-10-09T21:52:31.724209Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:31.724513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.724526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.724566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982518:5758], now have 1 active actors on pipe
2025-10-09T21:52:31.724801Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:31.724816Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:31.724846Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:31.724893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2478 reason: error at write: Disconnected sessionId: src_id|98b5da02-a047a04-3130821-b820844_0
2025-10-09T21:52:31.724913Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 76 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343236225982518:5758]
2025-10-09T21:52:31.724946Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:31.725071Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2478 sessionId: src_id|98b5da02-a047a04-3130821-b820844_0 is DEAD
2025-10-09T21:52:31.725250Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98b5da02-a047a04-3130821-b820844_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:31.725273Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98b5da02-a047a04-3130821-b820844_0] Write session will restart in 0.007925s
2025-10-09T21:52:31.725297Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98b5da02-a047a04-3130821-b820844_0] Write session: Do CDS request
2025-10-09T21:52:31.725303Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98b5da02-a047a04-3130821-b820844_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:31.731868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:31.733075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:31.733233Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:31.733241Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:31.733250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:31.733303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:31.733346Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343235864128804:8273]
2025-10-09T21:52:31.733720Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:31.734020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:31.734091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:31.734450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:31.734490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|98b5da02-a047a04-3130821-b820844_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:31.734521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.734527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.734529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.734530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.734533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.734534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.734535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.734536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.734538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.734543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.734554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.734558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.734563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.734567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.734571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.734575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.734579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.734583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.734590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.734597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.734604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.734614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.734581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:31.734619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.734594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2479
2025-10-09T21:52:31.734624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.734629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.734635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.734640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.734645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.734651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.734661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.734666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.734671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.734679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.734684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.734689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.734699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.734699Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2479 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:31.734732Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2479 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:31.734729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:31.734738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2479 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:31.734741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:31.735185Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2479 sessionId:  describe result for acl check
2025-10-09T21:52:31.735224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:31.735230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:31.735233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:31.735238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343236225982547:4370] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:31.735773Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343236225982547:4370] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:31.737538Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343236225982547:4370] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:31.740391Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343236225982547:4370] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:31.740402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343236225982547:4370] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:31.740404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343236225982547:4370] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:31.740411Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2479 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:31.740804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:31.741832Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:31.741842Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:31.741844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:31.741848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:31.741855Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2478 [2:7559343235864128804:8273]
2025-10-09T21:52:31.741868Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:31.741892Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:31.741896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:31.741899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:31.741902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:31.741919Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:31.741925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:31.741932Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:31.741984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.741991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.741994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.741997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:31.742080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:31.742088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.742099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.742111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.742133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:31.742160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982519:2432], now have 1 active actors on pipe
2025-10-09T21:52:31.742168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982581:4370], now have 1 active actors on pipe
2025-10-09T21:52:31.742175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343235864128799:8273]
2025-10-09T21:52:31.742177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343235864128799:8273]
2025-10-09T21:52:31.742179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343235864128799:8273]
2025-10-09T21:52:31.742160Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2478
2025-10-09T21:52:31.742189Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:31.742162Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2478
2025-10-09T21:52:31.742203Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:31.742208Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:31.742211Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:31.742213Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:31.742217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:31.742220Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:31.742263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.742272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.742285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343235864128799:8273]
2025-10-09T21:52:31.742296Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:31.742315Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:31.742326Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8e3a0557-342245a1-962133a3-bc219910_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:31.742334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:31.742338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:31.742340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:31.742349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:31.742364Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:31.742387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:31.742546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.742551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.742642Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2479 partition: 0 MaxSeqNo: 25 sessionId: src_id|8e3a0557-342245a1-962133a3-bc219910_0
2025-10-09T21:52:31.742562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:31.742924Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|98b5da02-a047a04-3130821-b820844_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8e3a0557-342245a1-962133a3-bc219910_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:31.745205Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3a0557-342245a1-962133a3-bc219910_0] Write session: try to update token
2025-10-09T21:52:31.745219Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3a0557-342245a1-962133a3-bc219910_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:31.756100Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:31.756114Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:31.756116Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:31.756117Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:31.756125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343235864128799:8273]
2025-10-09T21:52:31.756131Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343235864128799:8273]
2025-10-09T21:52:31.756138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343235864128799:8273]
2025-10-09T21:52:31.756172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:31.756177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.756179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:31.756191Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:31.756196Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:31.756199Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:31.756309Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:31.757882Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2479 sessionId: src_id|8e3a0557-342245a1-962133a3-bc219910_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:31.760143Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:31.824900Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343236225982592:5758]
2025-10-09T21:52:31.825171Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 77 pipe creation result: OK TabletId: 72075186224037892 Generation: 2478, pipe: [1:7559343236225982592:5758]
2025-10-09T21:52:31.909015Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:31.909362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.909377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.909419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982592:5758], now have 1 active actors on pipe
2025-10-09T21:52:31.909598Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:31.909633Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:31.909645Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:31.909647Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 77 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343236225982592:5758]
2025-10-09T21:52:31.909668Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:31.909725Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2479 reason: error at write: Disconnected sessionId: src_id|8e3a0557-342245a1-962133a3-bc219910_0
2025-10-09T21:52:31.909892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2479 sessionId: src_id|8e3a0557-342245a1-962133a3-bc219910_0 is DEAD
2025-10-09T21:52:31.910068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3a0557-342245a1-962133a3-bc219910_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:31.910080Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3a0557-342245a1-962133a3-bc219910_0] Write session will restart in 0.005778s
2025-10-09T21:52:31.910109Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3a0557-342245a1-962133a3-bc219910_0] Write session: Do CDS request
2025-10-09T21:52:31.910116Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3a0557-342245a1-962133a3-bc219910_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:31.916206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:31.917530Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3a0557-342245a1-962133a3-bc219910_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:31.917634Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:31.917650Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2480
2025-10-09T21:52:31.917645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:31.917757Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2480 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:31.917787Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2480 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:31.917797Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2480 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:31.917801Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:31.917800Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:31.917809Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:31.917818Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:31.917855Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:31.917890Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343235864128867:8264]
2025-10-09T21:52:31.918244Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2480 sessionId:  describe result for acl check
2025-10-09T21:52:31.918280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:31.918287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:31.918289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:31.918295Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343236225982618:4342] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:31.918482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:31.918765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:31.918807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:31.918844Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343236225982618:4342] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:31.919065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:31.919121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.919127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.919128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.919129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.919131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.919132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.919135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.919136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.919137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.919142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.919151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.919154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.919158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.919162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.919166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.919169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.919172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.919175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.919181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.919191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:31.919202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.919211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:31.919220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.919225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:31.919235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.919238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:31.919241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.919244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:31.919247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.919250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:31.919253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.919256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.919259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.919262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.919264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.919267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.919285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:31.920705Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343236225982618:4342] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:31.924980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343236225982618:4342] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:31.924993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343236225982618:4342] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:31.924996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343236225982618:4342] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:31.925002Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2480 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:31.925330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:31.926284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:31.926296Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:31.926297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:31.926301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:31.926308Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2479 [2:7559343235864128867:8264]
2025-10-09T21:52:31.926316Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:31.926329Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:31.926331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:31.926334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:31.926335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:31.926347Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:31.926354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:31.926359Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:31.926398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.926404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.926405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.926407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:31.926451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:31.926458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.926465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:31.926471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:31.926481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:31.926495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982594:2432], now have 1 active actors on pipe
2025-10-09T21:52:31.926501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343236225982659:4342], now have 1 active actors on pipe
2025-10-09T21:52:31.926507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343235864128862:8264]
2025-10-09T21:52:31.926512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343235864128862:8264]
2025-10-09T21:52:31.926514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343235864128862:8264]
2025-10-09T21:52:31.926523Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:31.926532Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:31.926539Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:31.926540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:31.926543Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:31.926547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:31.926550Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:31.926560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343235864128862:8264]
2025-10-09T21:52:31.926561Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:31.926556Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2479
2025-10-09T21:52:31.926578Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2479
2025-10-09T21:52:31.926572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:31.926640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.926647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.926666Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9ead9345-4dd06544-636b3716-715b2319_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:31.926675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:31.926678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:31.926679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:31.926686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:31.926696Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:31.926709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:31.926868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.926879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.926897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:31.926990Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2480 partition: 0 MaxSeqNo: 25 sessionId: src_id|9ead9345-4dd06544-636b3716-715b2319_0
2025-10-09T21:52:31.927289Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e3a0557-342245a1-962133a3-bc219910_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9ead9345-4dd06544-636b3716-715b2319_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:31.929541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ead9345-4dd06544-636b3716-715b2319_0] Write session: try to update token
2025-10-09T21:52:31.929554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ead9345-4dd06544-636b3716-715b2319_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:31.937661Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:31.937676Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:31.937686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343235864128862:8264]
2025-10-09T21:52:31.937725Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:31.937725Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:31.937885Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:31.938013Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:31.938198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:31.938337Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:31.938517Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:31.938548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:31.941165Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2480 sessionId: src_id|9ead9345-4dd06544-636b3716-715b2319_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:31.943243Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:31.943492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:31.943509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:31.943543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:31.943607Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:31.949899Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:31.949912Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:31.949914Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:31.949915Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:31.949926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343235864128862:8264]
2025-10-09T21:52:31.949928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343235864128862:8264]
2025-10-09T21:52:31.949932Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343235864128862:8264]
2025-10-09T21:52:31.949964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:31.949969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:31.949971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:31.949976Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:31.949982Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:31.949986Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:31.950114Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:32.009717Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343240520949963:5758]
2025-10-09T21:52:32.010005Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 78 pipe creation result: OK TabletId: 72075186224037892 Generation: 2479, pipe: [1:7559343240520949963:5758]
2025-10-09T21:52:32.096379Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:32.103083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.103105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.103251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.103304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.103361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.103420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.103460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.103468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.103533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.103595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.103670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.103735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.103773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.103779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.103836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.103888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.103949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.104018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.104110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.104116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.104174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.104231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.104288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.104345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.104384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.104390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.104452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.104522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.104589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.104649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.104695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.104702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.104764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.104816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.104870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.104923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.104976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.104982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.105035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.105087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.105141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.105194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.105234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.105242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.105318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.105381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.105440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.105493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.105547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.105554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.105610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.105662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.105719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.105776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.105814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.105820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.105877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.105928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.105984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.106055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.106097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.106102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.106147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343240520949963:5758], now have 1 active actors on pipe
2025-10-09T21:52:32.106376Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:32.106425Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:32.106430Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:32.106461Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2480 reason: error at write: Disconnected sessionId: src_id|9ead9345-4dd06544-636b3716-715b2319_0
2025-10-09T21:52:32.106643Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2480 sessionId: src_id|9ead9345-4dd06544-636b3716-715b2319_0 is DEAD
2025-10-09T21:52:32.106793Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ead9345-4dd06544-636b3716-715b2319_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:32.106815Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ead9345-4dd06544-636b3716-715b2319_0] Write session will restart in 0.009492s
2025-10-09T21:52:32.106833Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ead9345-4dd06544-636b3716-715b2319_0] Write session: Do CDS request
2025-10-09T21:52:32.106822Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 78 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343240520949963:5758]
2025-10-09T21:52:32.106838Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ead9345-4dd06544-636b3716-715b2319_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:32.106901Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:32.114207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:32.115660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:32.115818Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:32.115827Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:32.115849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:32.115894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:32.115921Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343240159096247:8240]
2025-10-09T21:52:32.116328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:32.116629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:32.116674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:32.117072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:32.117122Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.117128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.117129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.117130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.117132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.117133Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.117134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.117135Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.117136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.117141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.117151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.117154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.117157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.117161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.117166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.117170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.117173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.117177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.117184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.117192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.117197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.117200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.117202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.117207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.117212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.117216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.117220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.117224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.117228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.117233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.117239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.117245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.117262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.117268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.117278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.117283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.117301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:32.117525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9ead9345-4dd06544-636b3716-715b2319_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:32.117643Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:32.117656Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2481
2025-10-09T21:52:32.117789Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2481 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:32.117818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2481 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:32.117826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2481 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:32.117830Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:32.118177Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2481 sessionId:  describe result for acl check
2025-10-09T21:52:32.118205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:32.118211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:32.118213Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:32.118218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343240520950020:4315] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:32.118688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343240520950020:4315] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:32.120181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343240520950020:4315] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:32.123167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343240520950020:4315] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:32.123177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343240520950020:4315] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:32.123180Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343240520950020:4315] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:32.123195Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2481 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:32.123634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:32.124886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:32.124898Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:32.124900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:32.124903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:32.124909Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2480 [2:7559343240159096247:8240]
2025-10-09T21:52:32.124921Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:32.124936Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:32.124942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:32.124944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:32.124945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:32.124960Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:32.124967Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:32.124973Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:32.125015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.125020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.125021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.125023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:32.125076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:32.125082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.125089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.125099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.125112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:32.125131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343240520949979:2432], now have 1 active actors on pipe
2025-10-09T21:52:32.125141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343240159096234:8240]
2025-10-09T21:52:32.125142Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343240159096234:8240]
2025-10-09T21:52:32.125144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343240159096234:8240]
2025-10-09T21:52:32.125153Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:32.125164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343240520950052:4315], now have 1 active actors on pipe
2025-10-09T21:52:32.125167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:32.125170Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:32.125172Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:32.125176Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:32.125190Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:32.125197Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:32.125231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343240159096234:8240]
2025-10-09T21:52:32.125243Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:32.125223Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2480
2025-10-09T21:52:32.125272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:32.125228Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2480
2025-10-09T21:52:32.125286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.125295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.125329Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|32524a96-f88db721-2f8d6e5b-81d25bae_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:32.125338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:32.125345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:32.125347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:32.125354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:32.125366Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:32.125381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:32.125532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.125538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.125549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:32.125636Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2481 partition: 0 MaxSeqNo: 25 sessionId: src_id|32524a96-f88db721-2f8d6e5b-81d25bae_0
2025-10-09T21:52:32.125964Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9ead9345-4dd06544-636b3716-715b2319_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|32524a96-f88db721-2f8d6e5b-81d25bae_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:32.128286Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32524a96-f88db721-2f8d6e5b-81d25bae_0] Write session: try to update token
2025-10-09T21:52:32.128302Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32524a96-f88db721-2f8d6e5b-81d25bae_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:32.139088Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:32.139104Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:32.139106Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:32.139107Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:32.139115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343240159096234:8240]
2025-10-09T21:52:32.139119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343240159096234:8240]
2025-10-09T21:52:32.139122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343240159096234:8240]
2025-10-09T21:52:32.139158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:32.139163Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:32.139165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.139167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:32.139168Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:32.139174Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:32.139288Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:32.140679Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2481 sessionId: src_id|32524a96-f88db721-2f8d6e5b-81d25bae_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:32.143156Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:32.206843Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343240520950061:5758]
2025-10-09T21:52:32.207154Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 79 pipe creation result: OK TabletId: 72075186224037892 Generation: 2480, pipe: [1:7559343240520950061:5758]
2025-10-09T21:52:32.286929Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:32.287703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.287729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.287783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343240520950061:5758], now have 1 active actors on pipe
2025-10-09T21:52:32.287974Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:32.288004Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 79 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343240520950061:5758]
2025-10-09T21:52:32.288015Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:32.288017Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:32.288035Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:32.288067Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2481 reason: error at write: Disconnected sessionId: src_id|32524a96-f88db721-2f8d6e5b-81d25bae_0
2025-10-09T21:52:32.288212Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2481 sessionId: src_id|32524a96-f88db721-2f8d6e5b-81d25bae_0 is DEAD
2025-10-09T21:52:32.288345Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32524a96-f88db721-2f8d6e5b-81d25bae_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:32.288354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32524a96-f88db721-2f8d6e5b-81d25bae_0] Write session will restart in 0.007149s
2025-10-09T21:52:32.288373Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32524a96-f88db721-2f8d6e5b-81d25bae_0] Write session: Do CDS request
2025-10-09T21:52:32.288378Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32524a96-f88db721-2f8d6e5b-81d25bae_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:32.295617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:32.296439Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|32524a96-f88db721-2f8d6e5b-81d25bae_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:32.296530Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:32.296540Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2482
2025-10-09T21:52:32.296665Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2482 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:32.296698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2482 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:32.296702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2482 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:32.296710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:32.297033Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2482 sessionId:  describe result for acl check
2025-10-09T21:52:32.297067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:32.297071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:32.297073Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:32.297079Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343240520950093:4300] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:32.297148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:32.297370Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:32.297383Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:32.297395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:32.297453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:32.297495Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343240159096327:8223]
2025-10-09T21:52:32.297543Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343240520950093:4300] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:32.298002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:32.298305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:32.298335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:32.298586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:32.298638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.298645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.298647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.298648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.298649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.298649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.298650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.298651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.298652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.298655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.298661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.298664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.298666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.298668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.298671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.298673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.298676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.298678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.298683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.298688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.298692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.298697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.298699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.298702Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.298705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.298708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.298711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.298714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.298716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.298719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.298722Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.298725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.298728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.298730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.298733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.298736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.298765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:32.299189Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343240520950093:4300] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:32.304703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343240520950093:4300] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:32.304717Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343240520950093:4300] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:32.304720Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343240520950093:4300] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:32.304729Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2482 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:32.305090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:32.306066Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:32.306078Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:32.306079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:32.306082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:32.306088Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2481 [2:7559343240159096327:8223]
2025-10-09T21:52:32.306098Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:32.306114Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:32.306116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:32.306118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:32.306120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:32.306133Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:32.306137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:32.306142Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:32.306186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.306193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.306194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.306196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:32.306262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:32.306271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.306282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.306293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.306313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:32.306337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343240520950067:2432], now have 1 active actors on pipe
2025-10-09T21:52:32.306344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343240520950132:4300], now have 1 active actors on pipe
2025-10-09T21:52:32.306350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343240159096322:8223]
2025-10-09T21:52:32.306352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343240159096322:8223]
2025-10-09T21:52:32.306354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343240159096322:8223]
2025-10-09T21:52:32.306365Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:32.306379Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:32.306381Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:32.306384Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:32.306386Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:32.306398Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:32.306401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:32.306393Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2481
2025-10-09T21:52:32.306426Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2481
2025-10-09T21:52:32.306431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343240159096322:8223]
2025-10-09T21:52:32.306440Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:32.306466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.306466Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:32.306475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.306509Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|419da22c-c29b10a6-64acb36f-b728ebe8_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:32.306521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:32.306525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:32.306527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:32.306537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:32.306546Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:32.306566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:32.306801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.306809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.306824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:32.306884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2482 partition: 0 MaxSeqNo: 25 sessionId: src_id|419da22c-c29b10a6-64acb36f-b728ebe8_0
2025-10-09T21:52:32.307201Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|32524a96-f88db721-2f8d6e5b-81d25bae_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|419da22c-c29b10a6-64acb36f-b728ebe8_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:32.309431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|419da22c-c29b10a6-64acb36f-b728ebe8_0] Write session: try to update token
2025-10-09T21:52:32.309444Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|419da22c-c29b10a6-64acb36f-b728ebe8_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:32.320668Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:32.320686Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:32.320688Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:32.320691Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:32.320700Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343240159096322:8223]
2025-10-09T21:52:32.320709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343240159096322:8223]
2025-10-09T21:52:32.320713Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343240159096322:8223]
2025-10-09T21:52:32.320762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:32.320767Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:32.320770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.320772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:32.320773Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:32.320776Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:32.320900Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:32.321843Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2482 sessionId: src_id|419da22c-c29b10a6-64acb36f-b728ebe8_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:32.324110Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:32.388082Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343240520950144:5758]
2025-10-09T21:52:32.388365Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 80 pipe creation result: OK TabletId: 72075186224037892 Generation: 2481, pipe: [1:7559343240520950144:5758]
2025-10-09T21:52:32.466023Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:32.466594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.466608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.466646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343240520950144:5758], now have 1 active actors on pipe
2025-10-09T21:52:32.466796Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:32.466838Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 80 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343240520950144:5758]
2025-10-09T21:52:32.466852Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:32.466855Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:32.466867Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:32.466909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2482 reason: error at write: Disconnected sessionId: src_id|419da22c-c29b10a6-64acb36f-b728ebe8_0
2025-10-09T21:52:32.467096Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2482 sessionId: src_id|419da22c-c29b10a6-64acb36f-b728ebe8_0 is DEAD
2025-10-09T21:52:32.467232Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|419da22c-c29b10a6-64acb36f-b728ebe8_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:32.467246Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|419da22c-c29b10a6-64acb36f-b728ebe8_0] Write session will restart in 0.008948s
2025-10-09T21:52:32.467271Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|419da22c-c29b10a6-64acb36f-b728ebe8_0] Write session: Do CDS request
2025-10-09T21:52:32.467277Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|419da22c-c29b10a6-64acb36f-b728ebe8_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:32.473904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:32.475426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:32.475580Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:32.475588Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:32.475596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:32.475634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:32.475664Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343240159096391:8218]
2025-10-09T21:52:32.476090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:32.476342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:32.476376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:32.476589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:32.476621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.476626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.476626Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.476627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.476627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.476628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.476629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.476629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.476630Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.476633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.476641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.476643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.476645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.476647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.476650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.476652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.476654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.476656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.476662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.476667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.476671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.476674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.476676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.476679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.476682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.476684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.476687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.476691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.476694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.476697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.476700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.476703Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.476705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.476708Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.476711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.476714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.476730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:32.477505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|419da22c-c29b10a6-64acb36f-b728ebe8_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:32.477581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:32.477591Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2483
2025-10-09T21:52:32.477684Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2483 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:32.477705Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2483 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:32.477709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2483 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:32.477712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:32.478078Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2483 sessionId:  describe result for acl check
2025-10-09T21:52:32.478105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:32.478106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:32.478107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:32.478111Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343240520950178:4286] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:32.478524Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343240520950178:4286] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:32.480420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343240520950178:4286] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:32.482685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:32.482816Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343240520950178:4286] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:32.482826Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343240520950178:4286] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:32.482828Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343240520950178:4286] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:32.482835Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2483 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:32.483666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:32.483676Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:32.483678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:32.483681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:32.483685Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2482 [2:7559343240159096391:8218]
2025-10-09T21:52:32.483695Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:32.483708Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:32.483714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:32.483716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:32.483717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:32.483731Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:32.483737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:32.483742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:32.483779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.483784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.483786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.483788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:32.483834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:32.483841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.483848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.483854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.483862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:32.483877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343240520950213:4286], now have 1 active actors on pipe
2025-10-09T21:52:32.483879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343240520950147:2432], now have 1 active actors on pipe
2025-10-09T21:52:32.483885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343240159096386:8218]
2025-10-09T21:52:32.483886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343240159096386:8218]
2025-10-09T21:52:32.483887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343240159096386:8218]
2025-10-09T21:52:32.483897Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:32.483907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:32.483917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:32.483920Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:32.483920Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:32.483922Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:32.483924Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:32.483939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343240159096386:8218]
2025-10-09T21:52:32.483944Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:32.483946Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:32.483966Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2482
2025-10-09T21:52:32.483966Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2482
2025-10-09T21:52:32.484036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.484044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.484064Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5449fde7-f8c69fe2-c7d78a7b-d4f5e4bd_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:32.484073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:32.484076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:32.484077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:32.484085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:32.484092Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:32.484106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:32.484243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.484248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.484259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:32.484326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2483 partition: 0 MaxSeqNo: 25 sessionId: src_id|5449fde7-f8c69fe2-c7d78a7b-d4f5e4bd_0
2025-10-09T21:52:32.484581Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|419da22c-c29b10a6-64acb36f-b728ebe8_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5449fde7-f8c69fe2-c7d78a7b-d4f5e4bd_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:32.486965Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5449fde7-f8c69fe2-c7d78a7b-d4f5e4bd_0] Write session: try to update token
2025-10-09T21:52:32.486981Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5449fde7-f8c69fe2-c7d78a7b-d4f5e4bd_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:32.499197Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2483 sessionId: src_id|5449fde7-f8c69fe2-c7d78a7b-d4f5e4bd_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:32.500321Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:32.500339Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:32.500347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343240159096386:8218]
2025-10-09T21:52:32.500392Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:32.500398Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:32.500585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:32.500747Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:32.500891Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:32.501033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:32.501223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:32.501293Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:32.501672Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:32.501850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.501865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.501897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:32.501982Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:32.507660Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:32.507674Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:32.507676Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:32.507678Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:32.507687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343240159096386:8218]
2025-10-09T21:52:32.507694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343240159096386:8218]
2025-10-09T21:52:32.507698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343240159096386:8218]
2025-10-09T21:52:32.507740Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:32.507743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:32.507750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.507751Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:32.507751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:32.507754Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:32.507879Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:32.566747Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343240520950220:5758]
2025-10-09T21:52:32.567085Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 81 pipe creation result: OK TabletId: 72075186224037892 Generation: 2482, pipe: [1:7559343240520950220:5758]
2025-10-09T21:52:32.655070Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:32.663524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.663548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.663742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.663796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.664247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.664315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.664355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.664370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.664440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.664495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.664555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.664614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.664677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.664685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.664885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.664942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.664999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.665053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.665092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.665097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.665156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.665210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.665277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.665344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.665384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.665386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.665442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.665653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.665710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.665765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.665807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.665812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.665866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.665919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.665974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.666042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.666080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.666088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.666142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.666195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.666254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.666305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.666345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.666350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.666403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.666453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.666507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.666559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.666596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.666603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.666655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.666712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.666775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.666844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.666886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.666892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.666944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.666992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.667044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.667098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.667138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.667143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.667193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343240520950220:5758], now have 1 active actors on pipe
2025-10-09T21:52:32.667541Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:32.667544Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:32.667553Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:32.667585Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2483 reason: error at write: Disconnected sessionId: src_id|5449fde7-f8c69fe2-c7d78a7b-d4f5e4bd_0
2025-10-09T21:52:32.667602Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 81 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343240520950220:5758]
2025-10-09T21:52:32.667804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2483 sessionId: src_id|5449fde7-f8c69fe2-c7d78a7b-d4f5e4bd_0 is DEAD
2025-10-09T21:52:32.667982Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5449fde7-f8c69fe2-c7d78a7b-d4f5e4bd_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:32.668012Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5449fde7-f8c69fe2-c7d78a7b-d4f5e4bd_0] Write session will restart in 0.006834s
2025-10-09T21:52:32.668052Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5449fde7-f8c69fe2-c7d78a7b-d4f5e4bd_0] Write session: Do CDS request
2025-10-09T21:52:32.668059Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5449fde7-f8c69fe2-c7d78a7b-d4f5e4bd_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:32.668292Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:32.676295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:32.676481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5449fde7-f8c69fe2-c7d78a7b-d4f5e4bd_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:32.676570Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:32.676580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2484
2025-10-09T21:52:32.676714Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2484 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:32.676754Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2484 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:32.676762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2484 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:32.676765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:32.677271Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2484 sessionId:  describe result for acl check
2025-10-09T21:52:32.677299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:32.677304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:32.677305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:32.677312Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343240520950250:4279] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:32.677647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:32.677819Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:32.677826Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:32.677835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:32.677776Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343240520950250:4279] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:32.677875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:32.677913Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343240159096457:8212]
2025-10-09T21:52:32.678336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:32.678610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:32.678676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:32.678998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:32.679041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.679046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.679046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.679047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.679048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.679048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.679049Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.679052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.679052Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.679056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.679067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.679070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.679072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.679074Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.679076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.679078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.679080Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.679082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.679086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.679096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.679100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.679104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.679106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.679110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.679113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.679116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.679119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.679126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.679128Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.679132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.679134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.679138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.679140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.679143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.679147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.679150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.679167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:32.679988Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343240520950250:4279] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:32.685118Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343240520950250:4279] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:32.685132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343240520950250:4279] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:32.685138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343240520950250:4279] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:32.685146Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2484 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:32.685449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:32.686478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:32.686490Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:32.686493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:32.686496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:32.686500Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2483 [2:7559343240159096457:8212]
2025-10-09T21:52:32.686508Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:32.686521Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:32.686523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:32.686525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:32.686526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:32.686538Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:32.686545Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:32.686551Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:32.686593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.686596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.686597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.686598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:32.686658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:32.686669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.686676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.686682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.686692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:32.686706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343240520950223:2432], now have 1 active actors on pipe
2025-10-09T21:52:32.686711Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343240159096452:8212]
2025-10-09T21:52:32.686713Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343240159096452:8212]
2025-10-09T21:52:32.686714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343240159096452:8212]
2025-10-09T21:52:32.686721Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:32.686728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343240520950290:4279], now have 1 active actors on pipe
2025-10-09T21:52:32.686734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:32.686735Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:32.686738Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:32.686740Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:32.686741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:32.686743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:32.686769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343240159096452:8212]
2025-10-09T21:52:32.686775Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:32.686797Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:32.686848Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2483
2025-10-09T21:52:32.686887Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2483
2025-10-09T21:52:32.686915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.686929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.686957Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|78a91a04-251413dd-75455ac6-2a0be5c5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:32.686967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:32.686970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:32.686973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:32.686985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:32.687000Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:32.687019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:32.687164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.687170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.687182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:32.687281Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2484 partition: 0 MaxSeqNo: 25 sessionId: src_id|78a91a04-251413dd-75455ac6-2a0be5c5_0
2025-10-09T21:52:32.687571Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5449fde7-f8c69fe2-c7d78a7b-d4f5e4bd_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|78a91a04-251413dd-75455ac6-2a0be5c5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:32.692756Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78a91a04-251413dd-75455ac6-2a0be5c5_0] Write session: try to update token
2025-10-09T21:52:32.692773Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78a91a04-251413dd-75455ac6-2a0be5c5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:32.698114Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:32.698126Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:32.698135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343240159096452:8212]
2025-10-09T21:52:32.698175Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:32.698179Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:32.698327Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:32.698469Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:32.698612Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:32.698755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:32.698964Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:32.699011Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:32.705953Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2484 sessionId: src_id|78a91a04-251413dd-75455ac6-2a0be5c5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:32.708230Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:32.708448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.708467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.708505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:32.708569Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:32.710891Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:32.710907Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:32.710910Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:32.710912Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:32.710922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343240159096452:8212]
2025-10-09T21:52:32.710927Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343240159096452:8212]
2025-10-09T21:52:32.710933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343240159096452:8212]
2025-10-09T21:52:32.710971Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:32.710977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:32.710982Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:32.710985Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:32.710986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.710989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:32.711139Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:32.767948Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343240520950297:5758]
2025-10-09T21:52:32.768207Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 82 pipe creation result: OK TabletId: 72075186224037892 Generation: 2483, pipe: [1:7559343240520950297:5758]
2025-10-09T21:52:32.872586Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:32.880133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.880162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.880328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.880387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.880449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.880512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.880549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.880559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.880627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.880688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.880748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.880809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.880850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.880860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.880917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.880975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.881073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.881132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.881174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.881182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.881246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.881318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.881379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.881439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.881480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.881488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.881550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.881605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.881667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.881728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.881768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.881775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.881833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.881891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.881949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.882010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.882063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.882071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.882129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.882184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.882241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.882300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.882340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.882349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.882409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.882462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.882517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.882573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.882611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.882621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.882681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.882736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.882790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.882847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.882887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.882895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.882953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:32.883009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:32.883067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:32.883124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:32.883185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:32.883194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:32.883251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343240520950297:5758], now have 1 active actors on pipe
2025-10-09T21:52:32.883455Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:32.883461Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:32.883465Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:32.883492Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2484 reason: error at write: Disconnected sessionId: src_id|78a91a04-251413dd-75455ac6-2a0be5c5_0
2025-10-09T21:52:32.883514Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 82 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343240520950297:5758]
2025-10-09T21:52:32.883672Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2484 sessionId: src_id|78a91a04-251413dd-75455ac6-2a0be5c5_0 is DEAD
2025-10-09T21:52:32.883799Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78a91a04-251413dd-75455ac6-2a0be5c5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:32.883808Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78a91a04-251413dd-75455ac6-2a0be5c5_0] Write session will restart in 0.007586s
2025-10-09T21:52:32.883832Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78a91a04-251413dd-75455ac6-2a0be5c5_0] Write session: Do CDS request
2025-10-09T21:52:32.883837Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78a91a04-251413dd-75455ac6-2a0be5c5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:32.884292Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:32.890633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:32.891925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:32.892096Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:32.892103Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:32.892111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:32.892148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:32.892179Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343240159096526:8210]
2025-10-09T21:52:32.892506Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|78a91a04-251413dd-75455ac6-2a0be5c5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:32.892551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:32.892553Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:32.892577Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2485
2025-10-09T21:52:32.892701Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2485 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:32.892730Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2485 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:32.892738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2485 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:32.892742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:32.892887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:32.892933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:32.893154Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2485 sessionId:  describe result for acl check
2025-10-09T21:52:32.893182Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:32.893193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:32.893194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:32.893198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343240520950340:4273] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:32.893237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:32.893303Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.893309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.893311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.893312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.893313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.893314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.893316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.893317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.893318Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.893323Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.893334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.893338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.893341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.893346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.893350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.893354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.893357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.893361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.893367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.893375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:32.893381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.893390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:32.893395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.893404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:32.893412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.893417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:32.893422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.893426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:32.893430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.893437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:32.893441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.893443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.893447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.893449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.893453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.893456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.893471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:32.893659Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343240520950340:4273] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:32.896015Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343240520950340:4273] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:32.899179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343240520950340:4273] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:32.899194Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343240520950340:4273] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:32.899198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343240520950340:4273] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:32.899207Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2485 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:32.899581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:32.900599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:32.900616Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:32.900618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:32.900621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:32.900628Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2484 [2:7559343240159096526:8210]
2025-10-09T21:52:32.900638Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:32.900652Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:32.900654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:32.900657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:32.900658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:32.900672Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:32.900676Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:32.900681Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:32.900723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.900726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.900727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.900729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:32.900788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:32.900799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.900810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:32.900822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:32.900837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:32.900859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343240520950376:4273], now have 1 active actors on pipe
2025-10-09T21:52:32.900869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343240520950300:2432], now have 1 active actors on pipe
2025-10-09T21:52:32.900876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343240159096521:8210]
2025-10-09T21:52:32.900878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343240159096521:8210]
2025-10-09T21:52:32.900879Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343240159096521:8210]
2025-10-09T21:52:32.900889Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:32.900900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:32.900914Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:32.900922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:32.900923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:32.900926Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:32.900929Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:32.900925Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2484
2025-10-09T21:52:32.900934Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2484
2025-10-09T21:52:32.900953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343240159096521:8210]
2025-10-09T21:52:32.900960Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:32.900971Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:32.901059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.901067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.901090Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b607adb5-6fac8c0b-52021085-4662ef3b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:32.901096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:32.901099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:32.901100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:32.901107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:32.901116Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:32.901131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:32.901292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:32.901297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:32.901308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:32.901370Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2485 partition: 0 MaxSeqNo: 25 sessionId: src_id|b607adb5-6fac8c0b-52021085-4662ef3b_0
2025-10-09T21:52:32.901705Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|78a91a04-251413dd-75455ac6-2a0be5c5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b607adb5-6fac8c0b-52021085-4662ef3b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:32.904060Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b607adb5-6fac8c0b-52021085-4662ef3b_0] Write session: try to update token
2025-10-09T21:52:32.904078Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b607adb5-6fac8c0b-52021085-4662ef3b_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:32.915005Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:32.915026Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:32.915028Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:32.915033Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:32.915043Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343240159096521:8210]
2025-10-09T21:52:32.915048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343240159096521:8210]
2025-10-09T21:52:32.915051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343240159096521:8210]
2025-10-09T21:52:32.915094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:32.915099Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:32.915100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:32.915102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:32.915104Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:32.915112Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:32.915228Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:32.917299Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2485 sessionId: src_id|b607adb5-6fac8c0b-52021085-4662ef3b_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:32.919709Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:32.983504Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343240520950387:5758]
2025-10-09T21:52:32.983751Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 83 pipe creation result: OK TabletId: 72075186224037892 Generation: 2484, pipe: [1:7559343240520950387:5758]
2025-10-09T21:52:33.060704Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:33.061051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:33.061068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:33.061119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343240520950387:5758], now have 1 active actors on pipe
2025-10-09T21:52:33.061357Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:33.061395Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:33.061409Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:33.061434Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 83 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343240520950387:5758]
2025-10-09T21:52:33.061470Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:33.061487Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2485 reason: error at write: Disconnected sessionId: src_id|b607adb5-6fac8c0b-52021085-4662ef3b_0
2025-10-09T21:52:33.061720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2485 sessionId: src_id|b607adb5-6fac8c0b-52021085-4662ef3b_0 is DEAD
2025-10-09T21:52:33.061827Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b607adb5-6fac8c0b-52021085-4662ef3b_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:33.061837Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b607adb5-6fac8c0b-52021085-4662ef3b_0] Write session will restart in 0.005077s
2025-10-09T21:52:33.061868Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b607adb5-6fac8c0b-52021085-4662ef3b_0] Write session: Do CDS request
2025-10-09T21:52:33.061873Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b607adb5-6fac8c0b-52021085-4662ef3b_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:33.068546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b607adb5-6fac8c0b-52021085-4662ef3b_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:33.068636Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:33.068654Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2486
2025-10-09T21:52:33.068775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2486 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:33.068802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2486 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:33.068815Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2486 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:33.068818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:33.069215Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2486 sessionId:  describe result for acl check
2025-10-09T21:52:33.069267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:33.069272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:33.069277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:33.069281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343244815917714:4249] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:33.069573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:33.069797Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343244815917714:4249] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:33.071373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:33.071569Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:33.071581Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:33.071595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:33.071660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:33.071714Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343244454063890:8196]
2025-10-09T21:52:33.071933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343244815917714:4249] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:33.072217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:33.072479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:33.072534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:33.072816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:33.072866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.072870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.072871Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.072872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.072872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.072873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.072874Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.072875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.072875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.072879Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.072886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.072888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.072891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.072893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.072895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.072897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.072899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.072901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.072906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.072911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.072916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.072919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.072922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.072925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.072928Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.072939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.072942Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.072945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.072949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.072952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.072955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.072958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.072961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.072964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.072967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.072969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.072986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:33.077991Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343244815917714:4249] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:33.078003Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343244815917714:4249] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:33.078006Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343244815917714:4249] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:33.078014Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2486 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:33.079428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:33.080482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:33.080492Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:33.080494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:33.080498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:33.080503Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2485 [2:7559343244454063890:8196]
2025-10-09T21:52:33.080514Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:33.080528Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:33.080534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:33.080537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:33.080538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:33.080551Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:33.080554Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:33.080562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:33.080608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.080613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.080614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.080618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:33.080667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:33.080673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.080679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.080688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.080700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:33.080718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343244815917754:4249], now have 1 active actors on pipe
2025-10-09T21:52:33.080724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343244815917685:2432], now have 1 active actors on pipe
2025-10-09T21:52:33.080731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343244454063885:8196]
2025-10-09T21:52:33.080736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343244454063885:8196]
2025-10-09T21:52:33.080737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343244454063885:8196]
2025-10-09T21:52:33.080748Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:33.080758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:33.080760Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:33.080762Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:33.080764Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:33.080768Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:33.080770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:33.080794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343244454063885:8196]
2025-10-09T21:52:33.080800Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:33.080804Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:33.080807Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2485
2025-10-09T21:52:33.080882Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2485
2025-10-09T21:52:33.080925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:33.080938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:33.080965Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|131130b5-4e8581e5-dce812c1-ebf4e7ab_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:33.080974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:33.080979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:33.080984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:33.081006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:33.081022Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:33.081034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:33.081284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:33.081296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:33.081313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:33.081391Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2486 partition: 0 MaxSeqNo: 25 sessionId: src_id|131130b5-4e8581e5-dce812c1-ebf4e7ab_0
2025-10-09T21:52:33.081730Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b607adb5-6fac8c0b-52021085-4662ef3b_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|131130b5-4e8581e5-dce812c1-ebf4e7ab_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:33.084431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|131130b5-4e8581e5-dce812c1-ebf4e7ab_0] Write session: try to update token
2025-10-09T21:52:33.084450Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|131130b5-4e8581e5-dce812c1-ebf4e7ab_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:33.094883Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:33.094902Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:33.094911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343244454063885:8196]
2025-10-09T21:52:33.094958Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:33.094959Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:33.095129Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:33.095271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:33.095410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:33.095545Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:33.095736Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:33.095776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:33.098676Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2486 sessionId: src_id|131130b5-4e8581e5-dce812c1-ebf4e7ab_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:33.101167Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:33.101388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:33.101409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:33.101462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:33.101562Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:33.107990Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:33.108007Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:33.108009Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:33.108011Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:33.108020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343244454063885:8196]
2025-10-09T21:52:33.108023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343244454063885:8196]
2025-10-09T21:52:33.108029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343244454063885:8196]
2025-10-09T21:52:33.108066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:33.108072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.108074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:33.108073Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:33.108079Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:33.108082Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:33.108218Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:33.161707Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343244815917783:5758]
2025-10-09T21:52:33.161917Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 84 pipe creation result: OK TabletId: 72075186224037892 Generation: 2485, pipe: [1:7559343244815917783:5758]
2025-10-09T21:52:33.257674Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:33.264263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:33.264281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:33.264427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:33.264484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:33.264546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:33.264607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:33.264657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:33.264666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:33.264724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:33.264783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:33.264842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:33.264915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:33.264958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:33.264962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:33.265022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:33.265080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:33.265145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:33.265195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:33.265235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:33.265240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:33.265309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:33.265375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:33.265436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:33.265490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:33.265528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:33.265530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:33.265584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:33.265635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:33.265696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:33.265754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:33.265807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:33.265814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:33.265871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:33.265933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:33.265994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:33.266069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:33.266115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:33.266121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:33.266182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:33.266242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:33.266302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:33.266359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:33.266400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:33.266406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:33.266467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:33.266523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:33.266580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:33.266635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:33.266676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:33.266682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:33.266745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:33.266807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:33.266868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:33.266931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:33.266993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:33.267000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:33.267063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:33.267121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:33.267180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:33.267238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:33.267280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:33.267286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:33.267335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343244815917783:5758], now have 1 active actors on pipe
2025-10-09T21:52:33.267543Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:33.267594Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:33.267601Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:33.267603Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 84 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343244815917783:5758]
2025-10-09T21:52:33.267646Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2486 reason: error at write: Disconnected sessionId: src_id|131130b5-4e8581e5-dce812c1-ebf4e7ab_0
2025-10-09T21:52:33.267855Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2486 sessionId: src_id|131130b5-4e8581e5-dce812c1-ebf4e7ab_0 is DEAD
2025-10-09T21:52:33.268006Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|131130b5-4e8581e5-dce812c1-ebf4e7ab_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:33.268019Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|131130b5-4e8581e5-dce812c1-ebf4e7ab_0] Write session will restart in 0.007272s
2025-10-09T21:52:33.268045Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|131130b5-4e8581e5-dce812c1-ebf4e7ab_0] Write session: Do CDS request
2025-10-09T21:52:33.268051Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|131130b5-4e8581e5-dce812c1-ebf4e7ab_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:33.268168Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:33.275745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:33.276510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|131130b5-4e8581e5-dce812c1-ebf4e7ab_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:33.276591Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:33.276605Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2487
2025-10-09T21:52:33.276718Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2487 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:33.276761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2487 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:33.276771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2487 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:33.276774Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:33.277186Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2487 sessionId:  describe result for acl check
2025-10-09T21:52:33.277218Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:33.277219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:33.277221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:33.277228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343244815917821:4211] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:33.277427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:33.277634Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:33.277646Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:33.277660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:33.277729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:33.277772Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343244454063996:8190]
2025-10-09T21:52:33.277765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343244815917821:4211] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:33.278344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:33.278663Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:33.278709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:33.279024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:33.279077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.279083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.279084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.279085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.279087Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.279088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.279091Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.279092Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.279093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.279098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.279108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.279111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.279115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.279119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.279123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.279126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.279131Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.279134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.279140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.279148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.279159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.279164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.279173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.279178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.279184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.279193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.279198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.279203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.279211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.279215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.279220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.279224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.279233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.279236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.279241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.279246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.279272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:33.279763Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343244815917821:4211] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:33.284903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343244815917821:4211] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:33.284917Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343244815917821:4211] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:33.284921Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343244815917821:4211] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:33.284929Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2487 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:33.285158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:33.286191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:33.286204Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:33.286207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:33.286210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:33.286217Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2486 [2:7559343244454063996:8190]
2025-10-09T21:52:33.286231Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:33.286246Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:33.286253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:33.286256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:33.286258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:33.286275Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:33.286287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:33.286294Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:33.286343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.286349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.286351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.286354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:33.286416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:33.286423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.286436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.286448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.286469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:33.286492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343244815917789:2432], now have 1 active actors on pipe
2025-10-09T21:52:33.286499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343244815917860:4211], now have 1 active actors on pipe
2025-10-09T21:52:33.286506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343244454063991:8190]
2025-10-09T21:52:33.286511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343244454063991:8190]
2025-10-09T21:52:33.286513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343244454063991:8190]
2025-10-09T21:52:33.286526Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:33.286540Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:33.286542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:33.286551Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:33.286556Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:33.286535Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2486
2025-10-09T21:52:33.286556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:33.286542Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2486
2025-10-09T21:52:33.286559Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:33.286585Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343244454063991:8190]
2025-10-09T21:52:33.286589Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:33.286590Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:33.286606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:33.286610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:33.286654Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3e77970b-b2763ba6-6dd40190-97076a39_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:33.286666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:33.286670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:33.286672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:33.286679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:33.286688Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:33.286704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:33.286823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:33.286829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:33.286841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:33.286911Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2487 partition: 0 MaxSeqNo: 25 sessionId: src_id|3e77970b-b2763ba6-6dd40190-97076a39_0
2025-10-09T21:52:33.287172Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|131130b5-4e8581e5-dce812c1-ebf4e7ab_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3e77970b-b2763ba6-6dd40190-97076a39_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:33.289606Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3e77970b-b2763ba6-6dd40190-97076a39_0] Write session: try to update token
2025-10-09T21:52:33.289621Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3e77970b-b2763ba6-6dd40190-97076a39_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:33.301526Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:33.301543Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:33.301545Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:33.301547Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:33.301564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343244454063991:8190]
2025-10-09T21:52:33.301571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343244454063991:8190]
2025-10-09T21:52:33.301577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343244454063991:8190]
2025-10-09T21:52:33.301620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:33.301621Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:33.301630Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:33.301634Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:33.301637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.301640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:33.301771Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:33.302849Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2487 sessionId: src_id|3e77970b-b2763ba6-6dd40190-97076a39_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:33.305357Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:33.367557Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343244815917872:5758]
2025-10-09T21:52:33.367854Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 85 pipe creation result: OK TabletId: 72075186224037892 Generation: 2486, pipe: [1:7559343244815917872:5758]
2025-10-09T21:52:33.459282Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:33.459525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:33.459538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:33.459583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343244815917872:5758], now have 1 active actors on pipe
2025-10-09T21:52:33.459776Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:33.459824Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:33.459839Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:33.459897Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 85 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343244815917872:5758]
2025-10-09T21:52:33.459903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2487 reason: error at write: Disconnected sessionId: src_id|3e77970b-b2763ba6-6dd40190-97076a39_0
2025-10-09T21:52:33.459927Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:33.460093Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2487 sessionId: src_id|3e77970b-b2763ba6-6dd40190-97076a39_0 is DEAD
2025-10-09T21:52:33.460234Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e77970b-b2763ba6-6dd40190-97076a39_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:33.460246Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e77970b-b2763ba6-6dd40190-97076a39_0] Write session will restart in 0.008461s
2025-10-09T21:52:33.460270Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e77970b-b2763ba6-6dd40190-97076a39_0] Write session: Do CDS request
2025-10-09T21:52:33.460276Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e77970b-b2763ba6-6dd40190-97076a39_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:33.468009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:33.469626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3e77970b-b2763ba6-6dd40190-97076a39_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:33.469652Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:33.469669Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2488
2025-10-09T21:52:33.469771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:33.469863Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2488 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:33.469897Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2488 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:33.469903Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2488 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:33.469907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:33.470005Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:33.470015Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:33.470041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:33.470104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:33.470161Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343244454064065:8184]
2025-10-09T21:52:33.470355Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2488 sessionId:  describe result for acl check
2025-10-09T21:52:33.470397Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:33.470402Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:33.470404Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:33.470410Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343244815917907:4197] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:33.470715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:33.470989Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343244815917907:4197] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:33.471015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:33.471056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:33.471402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:33.471454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.471460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.471462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.471463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.471464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.471466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.471467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.471468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.471469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.471474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.471484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.471491Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.471496Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.471500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.471505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.471509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.471513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.471518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.471524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.471532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.471546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.471553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.471561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.471567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.471571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.471576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.471582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.471589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.471595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.471605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.471611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.471621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.471625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.471634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.471643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.471649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.471681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:33.472905Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343244815917907:4197] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:33.477544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343244815917907:4197] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:33.477563Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343244815917907:4197] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:33.477568Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343244815917907:4197] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:33.477578Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2488 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:33.478078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:33.479176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:33.479193Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:33.479196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:33.479200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:33.479207Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2487 [2:7559343244454064065:8184]
2025-10-09T21:52:33.479224Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:33.479242Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:33.479250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:33.479253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:33.479256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:33.479273Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:33.479285Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:33.479295Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:33.479356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.479362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.479364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.479366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:33.479440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:33.479450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.479462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.479475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.479493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:33.479519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343244815917946:4197], now have 1 active actors on pipe
2025-10-09T21:52:33.479527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343244815917875:2432], now have 1 active actors on pipe
2025-10-09T21:52:33.479535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343244454064060:8184]
2025-10-09T21:52:33.479541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343244454064060:8184]
2025-10-09T21:52:33.479543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343244454064060:8184]
2025-10-09T21:52:33.479556Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:33.479575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:33.479574Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:33.479578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:33.479580Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:33.479592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:33.479597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:33.479578Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2487
2025-10-09T21:52:33.479585Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2487
2025-10-09T21:52:33.479629Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343244454064060:8184]
2025-10-09T21:52:33.479640Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:33.479646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:33.479660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:33.479666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:33.479715Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|89733f3-b4a1d17-ed5f5893-d728056f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:33.479726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:33.479730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:33.479733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:33.479744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:33.479758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:33.479777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:33.479921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:33.479932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:33.479951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:33.480029Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2488 partition: 0 MaxSeqNo: 25 sessionId: src_id|89733f3-b4a1d17-ed5f5893-d728056f_0
2025-10-09T21:52:33.480367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e77970b-b2763ba6-6dd40190-97076a39_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|89733f3-b4a1d17-ed5f5893-d728056f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:33.482747Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|89733f3-b4a1d17-ed5f5893-d728056f_0] Write session: try to update token
2025-10-09T21:52:33.482767Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|89733f3-b4a1d17-ed5f5893-d728056f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:33.494248Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:33.494267Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:33.494269Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:33.494271Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:33.494282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343244454064060:8184]
2025-10-09T21:52:33.494287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343244454064060:8184]
2025-10-09T21:52:33.494291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343244454064060:8184]
2025-10-09T21:52:33.494336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:33.494335Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:33.494342Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:33.494344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.494346Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:33.494347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:33.494487Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:33.495889Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2488 sessionId: src_id|89733f3-b4a1d17-ed5f5893-d728056f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:33.498398Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:33.560078Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343244815917953:5758]
2025-10-09T21:52:33.560351Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 86 pipe creation result: OK TabletId: 72075186224037892 Generation: 2487, pipe: [1:7559343244815917953:5758]
2025-10-09T21:52:33.638054Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:33.638389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:33.638402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:33.638442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343244815917953:5758], now have 1 active actors on pipe
2025-10-09T21:52:33.638614Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:33.638672Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 86 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343244815917953:5758]
2025-10-09T21:52:33.638674Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:33.638679Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:33.638701Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:33.638743Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2488 reason: error at write: Disconnected sessionId: src_id|89733f3-b4a1d17-ed5f5893-d728056f_0
2025-10-09T21:52:33.638921Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2488 sessionId: src_id|89733f3-b4a1d17-ed5f5893-d728056f_0 is DEAD
2025-10-09T21:52:33.639066Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89733f3-b4a1d17-ed5f5893-d728056f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:33.639073Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89733f3-b4a1d17-ed5f5893-d728056f_0] Write session will restart in 0.005084s
2025-10-09T21:52:33.639100Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89733f3-b4a1d17-ed5f5893-d728056f_0] Write session: Do CDS request
2025-10-09T21:52:33.639104Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89733f3-b4a1d17-ed5f5893-d728056f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:33.645436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|89733f3-b4a1d17-ed5f5893-d728056f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:33.645510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:33.645545Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:33.645557Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2489
2025-10-09T21:52:33.645664Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2489 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:33.645698Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2489 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:33.645709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2489 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:33.645713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:33.646124Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2489 sessionId:  describe result for acl check
2025-10-09T21:52:33.646158Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:33.646163Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:33.646167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:33.646172Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343244815917989:4191] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:33.646691Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343244815917989:4191] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:33.646916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:33.647074Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:33.647083Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:33.647094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:33.647147Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:33.647188Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343244454064135:8185]
2025-10-09T21:52:33.647747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:33.648029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:33.648070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:33.648364Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:33.648374Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343244815917989:4191] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:33.648407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.648414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.648416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.648417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.648418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.648419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.648420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.648421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.648422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.648427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.648436Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.648439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.648444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.648448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.648452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.648456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.648459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.648463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.648469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.648477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.648483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.648488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.648493Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.648498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.648502Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.648507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.648511Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.648516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.648524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.648529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.648533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.648543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.648549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.648554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.648558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.648563Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.648589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:33.654089Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343244815917989:4191] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:33.654104Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343244815917989:4191] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:33.654108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343244815917989:4191] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:33.654117Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2489 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:33.654458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:33.655462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:33.655478Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:33.655480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:33.655483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:33.655489Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2488 [2:7559343244454064135:8185]
2025-10-09T21:52:33.655502Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:33.655519Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:33.655527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:33.655530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:33.655532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:33.655547Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:33.655558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:33.655564Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:33.655619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.655627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.655629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.655632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:33.655700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:33.655708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.655719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.655731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.655751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:33.655775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343244815917956:2432], now have 1 active actors on pipe
2025-10-09T21:52:33.655783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343244815918028:4191], now have 1 active actors on pipe
2025-10-09T21:52:33.655790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343244454064130:8185]
2025-10-09T21:52:33.655792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343244454064130:8185]
2025-10-09T21:52:33.655794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343244454064130:8185]
2025-10-09T21:52:33.655807Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:33.655819Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:33.655826Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:33.655829Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:33.655831Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:33.655838Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:33.655841Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:33.655818Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2488
2025-10-09T21:52:33.655820Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2488
2025-10-09T21:52:33.655862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343244454064130:8185]
2025-10-09T21:52:33.655869Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:33.655870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:33.655887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:33.655895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:33.655936Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|2d28263e-3b6078c7-385e507c-7207a59f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:33.655951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:33.655955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:33.655956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:33.655966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:33.655976Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:33.655991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:33.656146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:33.656152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:33.656165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:33.656253Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2489 partition: 0 MaxSeqNo: 25 sessionId: src_id|2d28263e-3b6078c7-385e507c-7207a59f_0
2025-10-09T21:52:33.656504Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|89733f3-b4a1d17-ed5f5893-d728056f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|2d28263e-3b6078c7-385e507c-7207a59f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:33.658701Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d28263e-3b6078c7-385e507c-7207a59f_0] Write session: try to update token
2025-10-09T21:52:33.658715Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d28263e-3b6078c7-385e507c-7207a59f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:33.669160Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:33.669177Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:33.669178Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:33.669180Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:33.669191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343244454064130:8185]
2025-10-09T21:52:33.669195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343244454064130:8185]
2025-10-09T21:52:33.669197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343244454064130:8185]
2025-10-09T21:52:33.669236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:33.669259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.669264Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:33.669267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:33.669270Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:33.669277Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:33.669400Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:33.670254Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2489 sessionId: src_id|2d28263e-3b6078c7-385e507c-7207a59f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:33.672504Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:33.738741Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343244815918035:5758]
2025-10-09T21:52:33.739062Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 87 pipe creation result: OK TabletId: 72075186224037892 Generation: 2488, pipe: [1:7559343244815918035:5758]
2025-10-09T21:52:33.818332Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:33.818642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:33.818656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:33.818701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343244815918035:5758], now have 1 active actors on pipe
2025-10-09T21:52:33.818853Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:33.818886Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:33.818904Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:33.818918Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:33.818961Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 87 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343244815918035:5758]
2025-10-09T21:52:33.818972Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2489 reason: error at write: Disconnected sessionId: src_id|2d28263e-3b6078c7-385e507c-7207a59f_0
2025-10-09T21:52:33.819152Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2489 sessionId: src_id|2d28263e-3b6078c7-385e507c-7207a59f_0 is DEAD
2025-10-09T21:52:33.819307Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d28263e-3b6078c7-385e507c-7207a59f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:33.819317Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d28263e-3b6078c7-385e507c-7207a59f_0] Write session will restart in 0.009556s
2025-10-09T21:52:33.819338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d28263e-3b6078c7-385e507c-7207a59f_0] Write session: Do CDS request
2025-10-09T21:52:33.819344Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d28263e-3b6078c7-385e507c-7207a59f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:33.826011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:33.827555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:33.827700Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:33.827707Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:33.827715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:33.827750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:33.827781Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343244454064206:8173]
2025-10-09T21:52:33.828157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:33.828423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:33.828454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:33.828657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:33.828689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.828690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.828691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.828691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.828692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.828693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.828693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.828694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.828695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.828698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.828705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.828707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.828709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.828711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.828713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.828715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.828716Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.828718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.828723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.828728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:33.828732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.828736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:33.828738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.828742Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:33.828745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.828750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:33.828752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.828755Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:33.828757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.828760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:33.828763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.828765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.828767Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.828770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.828772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.828775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.828793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:33.830375Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|2d28263e-3b6078c7-385e507c-7207a59f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:33.830473Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:33.830486Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2490
2025-10-09T21:52:33.830594Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2490 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:33.830625Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2490 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:33.830634Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2490 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:33.830637Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:33.831035Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2490 sessionId:  describe result for acl check
2025-10-09T21:52:33.831067Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:33.831069Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:33.831072Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:33.831078Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343244815918077:4171] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:33.831589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343244815918077:4171] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:33.833706Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343244815918077:4171] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:33.834900Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:33.835995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:33.836013Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:33.836015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:33.836019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:33.836025Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2489 [2:7559343244454064206:8173]
2025-10-09T21:52:33.836037Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:33.836054Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:33.836061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:33.836064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:33.836068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:33.836083Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:33.836093Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:33.836105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:33.836117Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343244815918077:4171] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:33.836129Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343244815918077:4171] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:33.836133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343244815918077:4171] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:33.836140Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2490 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:33.836163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.836172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.836174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.836177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:33.836240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:33.836249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.836263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:33.836276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:33.836293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:33.836318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343244815918038:2432], now have 1 active actors on pipe
2025-10-09T21:52:33.836326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343244815918111:4171], now have 1 active actors on pipe
2025-10-09T21:52:33.836333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343244454064201:8173]
2025-10-09T21:52:33.836335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343244454064201:8173]
2025-10-09T21:52:33.836337Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343244454064201:8173]
2025-10-09T21:52:33.836326Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2489
2025-10-09T21:52:33.836345Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:33.836342Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2489
2025-10-09T21:52:33.836361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:33.836364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:33.836364Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:33.836366Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:33.836378Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:33.836385Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:33.836430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:33.836439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:33.836452Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343244454064201:8173]
2025-10-09T21:52:33.836467Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:33.836494Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:33.836498Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|bc2fffcb-1f71c98-bb0bad35-64930ce_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:33.836506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:33.836510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:33.836512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:33.836522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:33.836532Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:33.836555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:33.836696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:33.836707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:33.836726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:33.836814Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2490 partition: 0 MaxSeqNo: 25 sessionId: src_id|bc2fffcb-1f71c98-bb0bad35-64930ce_0
2025-10-09T21:52:33.837088Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|2d28263e-3b6078c7-385e507c-7207a59f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|bc2fffcb-1f71c98-bb0bad35-64930ce_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:33.839525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2fffcb-1f71c98-bb0bad35-64930ce_0] Write session: try to update token
2025-10-09T21:52:33.839541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2fffcb-1f71c98-bb0bad35-64930ce_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:33.850192Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:33.850207Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:33.850209Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:33.850210Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:33.850219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343244454064201:8173]
2025-10-09T21:52:33.850223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343244454064201:8173]
2025-10-09T21:52:33.850226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343244454064201:8173]
2025-10-09T21:52:33.850259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:33.850265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:33.850268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:33.850268Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:33.850273Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:33.850276Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:33.850409Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:33.852665Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2490 sessionId: src_id|bc2fffcb-1f71c98-bb0bad35-64930ce_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:33.855075Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:33.918922Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343244815918118:5758]
2025-10-09T21:52:33.919151Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 88 pipe creation result: OK TabletId: 72075186224037892 Generation: 2489, pipe: [1:7559343244815918118:5758]
2025-10-09T21:52:34.001462Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:34.001786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.001805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.001850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343244815918118:5758], now have 1 active actors on pipe
2025-10-09T21:52:34.002055Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:34.002148Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:34.002168Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:34.002168Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:34.002234Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 88 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343244815918118:5758]
2025-10-09T21:52:34.002243Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2490 reason: error at write: Disconnected sessionId: src_id|bc2fffcb-1f71c98-bb0bad35-64930ce_0
2025-10-09T21:52:34.002421Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2490 sessionId: src_id|bc2fffcb-1f71c98-bb0bad35-64930ce_0 is DEAD
2025-10-09T21:52:34.002565Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2fffcb-1f71c98-bb0bad35-64930ce_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:34.002576Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2fffcb-1f71c98-bb0bad35-64930ce_0] Write session will restart in 0.009496s
2025-10-09T21:52:34.002599Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2fffcb-1f71c98-bb0bad35-64930ce_0] Write session: Do CDS request
2025-10-09T21:52:34.002604Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2fffcb-1f71c98-bb0bad35-64930ce_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:34.010541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:34.012055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:34.012264Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:34.012275Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:34.012287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:34.012338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:34.012369Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343248749031579:8179]
2025-10-09T21:52:34.012891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:34.013215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:34.013281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:34.013595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:34.013610Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2fffcb-1f71c98-bb0bad35-64930ce_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:34.013640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.013645Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.013646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.013647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.013647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.013648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.013649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.013649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.013650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.013654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.013662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.013667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.013671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.013675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.013679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.013685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.013688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.013692Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.013699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.013707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.013714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.013719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.013723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.013728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.013732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.013738Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.013743Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.013708Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:34.013722Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2491
2025-10-09T21:52:34.013748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.013753Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.013759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.013764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.013770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.013774Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.013780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.013785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.013790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.013821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:34.013847Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2491 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:34.013876Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2491 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:34.013887Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2491 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:34.013891Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:34.014267Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2491 sessionId:  describe result for acl check
2025-10-09T21:52:34.014299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:34.014306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:34.014308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:34.014315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343249110885459:4173] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:34.014830Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343249110885459:4173] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:34.016561Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343249110885459:4173] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:34.019791Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343249110885459:4173] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:34.019807Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343249110885459:4173] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:34.019811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343249110885459:4173] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:34.019820Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2491 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:34.020256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:34.021420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:34.021437Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:34.021440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:34.021444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:34.021452Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2490 [2:7559343248749031579:8179]
2025-10-09T21:52:34.021466Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:34.021484Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:34.021493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:34.021501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:34.021504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:34.021521Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:34.021527Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:34.021534Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:34.021588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.021594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.021596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.021599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:34.021653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:34.021659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.021668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.021678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.021699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:34.021725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885419:2432], now have 1 active actors on pipe
2025-10-09T21:52:34.021733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885493:4173], now have 1 active actors on pipe
2025-10-09T21:52:34.021740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343248749031574:8179]
2025-10-09T21:52:34.021742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343248749031574:8179]
2025-10-09T21:52:34.021744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343248749031574:8179]
2025-10-09T21:52:34.021757Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:34.021777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:34.021785Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:34.021788Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:34.021791Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:34.021795Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:34.021798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:34.021804Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2490
2025-10-09T21:52:34.021808Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2490
2025-10-09T21:52:34.021835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343248749031574:8179]
2025-10-09T21:52:34.021844Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:34.021846Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:34.021868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.021877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.021925Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dfcad899-997e985a-d5b0fc5f-8e67e533_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:34.021941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:34.021947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:34.021950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:34.021960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:34.021973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:34.021997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:34.022185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.022195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.022212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:34.022303Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2491 partition: 0 MaxSeqNo: 25 sessionId: src_id|dfcad899-997e985a-d5b0fc5f-8e67e533_0
2025-10-09T21:52:34.022647Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|bc2fffcb-1f71c98-bb0bad35-64930ce_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dfcad899-997e985a-d5b0fc5f-8e67e533_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:34.025014Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfcad899-997e985a-d5b0fc5f-8e67e533_0] Write session: try to update token
2025-10-09T21:52:34.025031Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfcad899-997e985a-d5b0fc5f-8e67e533_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:34.036493Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:34.036514Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:34.036517Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:34.036519Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:34.036529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343248749031574:8179]
2025-10-09T21:52:34.036534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343248749031574:8179]
2025-10-09T21:52:34.036537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343248749031574:8179]
2025-10-09T21:52:34.036572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:34.036579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.036581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:34.036582Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:34.036589Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:34.036592Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:34.036710Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:34.037964Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2491 sessionId: src_id|dfcad899-997e985a-d5b0fc5f-8e67e533_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:34.040472Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:34.102266Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343249110885500:5758]
2025-10-09T21:52:34.102544Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 89 pipe creation result: OK TabletId: 72075186224037892 Generation: 2490, pipe: [1:7559343249110885500:5758]
2025-10-09T21:52:34.191125Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:34.191379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.191393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.191437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885500:5758], now have 1 active actors on pipe
2025-10-09T21:52:34.191595Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:34.191633Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 89 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343249110885500:5758]
2025-10-09T21:52:34.191666Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:34.191694Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:34.191759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2491 reason: status is not ok: tablet will be restarted right now sessionId: src_id|dfcad899-997e985a-d5b0fc5f-8e67e533_0
2025-10-09T21:52:34.191922Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2491 sessionId: src_id|dfcad899-997e985a-d5b0fc5f-8e67e533_0 is DEAD
2025-10-09T21:52:34.192093Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfcad899-997e985a-d5b0fc5f-8e67e533_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:34.192106Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfcad899-997e985a-d5b0fc5f-8e67e533_0] Write session will restart in 0.009716s
2025-10-09T21:52:34.192133Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfcad899-997e985a-d5b0fc5f-8e67e533_0] Write session: Do CDS request
2025-10-09T21:52:34.192139Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfcad899-997e985a-d5b0fc5f-8e67e533_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:34.199538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:34.201577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:34.201804Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:34.201814Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:34.201827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:34.201873Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:34.201911Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343248749031666:8166]
2025-10-09T21:52:34.202368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:34.202541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dfcad899-997e985a-d5b0fc5f-8e67e533_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:34.202614Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:34.202631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2492
2025-10-09T21:52:34.202748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:34.202787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:34.202814Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2492 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:34.202856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2492 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:34.202865Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2492 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:34.202869Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:34.203088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:34.203165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.203170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.203171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.203171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.203172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.203173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.203173Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.203174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.203175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.203178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.203188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.203191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.203195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.203201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.203204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.203207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.203211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.203214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.203221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.203228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.203235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.203240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.203245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.203249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.203254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.203259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.203263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.203270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2492 sessionId:  describe result for acl check
2025-10-09T21:52:34.203268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.203272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.203278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.203285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.203291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.203295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.203300Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.203305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.203306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:34.203313Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:34.203311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.203315Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:34.203320Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343249110885566:4144] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:34.203338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:34.203777Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343249110885566:4144] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:34.205754Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343249110885566:4144] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:34.209792Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343249110885566:4144] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:34.209807Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343249110885566:4144] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:34.209810Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343249110885566:4144] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:34.209818Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2492 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:34.210003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:34.211322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:34.211341Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:34.211344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:34.211349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:34.211356Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2491 [2:7559343248749031666:8166]
2025-10-09T21:52:34.211373Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:34.211395Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:34.211405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:34.211408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:34.211410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:34.211431Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:34.211443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:34.211452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:34.211535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.211546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.211548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.211551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:34.211624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:34.211632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.211646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.211664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.211682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:34.211713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885600:4144], now have 1 active actors on pipe
2025-10-09T21:52:34.211716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885516:2432], now have 1 active actors on pipe
2025-10-09T21:52:34.211725Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343248749031659:8166]
2025-10-09T21:52:34.211732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343248749031659:8166]
2025-10-09T21:52:34.211734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343248749031659:8166]
2025-10-09T21:52:34.211750Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:34.211772Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:34.211780Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:34.211784Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:34.211787Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:34.211792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:34.211796Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:34.211771Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2491
2025-10-09T21:52:34.211780Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2491
2025-10-09T21:52:34.211836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343248749031659:8166]
2025-10-09T21:52:34.211844Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:34.211855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:34.211862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.211872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.211925Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7c927d14-aac1f467-91d8cec5-8308dc1c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:34.211937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:34.211941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:34.211948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:34.211959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:34.211972Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:34.211990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:34.212143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.212151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.212161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:34.212250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2492 partition: 0 MaxSeqNo: 25 sessionId: src_id|7c927d14-aac1f467-91d8cec5-8308dc1c_0
2025-10-09T21:52:34.212634Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dfcad899-997e985a-d5b0fc5f-8e67e533_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7c927d14-aac1f467-91d8cec5-8308dc1c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:34.215147Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c927d14-aac1f467-91d8cec5-8308dc1c_0] Write session: try to update token
2025-10-09T21:52:34.215166Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c927d14-aac1f467-91d8cec5-8308dc1c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:34.227539Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:34.227564Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:34.227566Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:34.227568Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:34.227578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343248749031659:8166]
2025-10-09T21:52:34.227583Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343248749031659:8166]
2025-10-09T21:52:34.227587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343248749031659:8166]
2025-10-09T21:52:34.227632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:34.227644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.227653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:34.227653Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:34.227660Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:34.227667Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:34.227799Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:34.228766Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2492 sessionId: src_id|7c927d14-aac1f467-91d8cec5-8308dc1c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:34.231197Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:34.291694Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343249110885611:5758]
2025-10-09T21:52:34.292011Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 90 pipe creation result: OK TabletId: 72075186224037892 Generation: 2491, pipe: [1:7559343249110885611:5758]
2025-10-09T21:52:34.372122Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:34.372392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.372405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.372453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885611:5758], now have 1 active actors on pipe
2025-10-09T21:52:34.372661Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:34.372702Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:34.372713Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:34.372746Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 90 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343249110885611:5758]
2025-10-09T21:52:34.372762Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:34.372766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2492 reason: error at write: Disconnected sessionId: src_id|7c927d14-aac1f467-91d8cec5-8308dc1c_0
2025-10-09T21:52:34.372909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2492 sessionId: src_id|7c927d14-aac1f467-91d8cec5-8308dc1c_0 is DEAD
2025-10-09T21:52:34.373043Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c927d14-aac1f467-91d8cec5-8308dc1c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:34.373053Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c927d14-aac1f467-91d8cec5-8308dc1c_0] Write session will restart in 0.006874s
2025-10-09T21:52:34.373069Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c927d14-aac1f467-91d8cec5-8308dc1c_0] Write session: Do CDS request
2025-10-09T21:52:34.373074Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c927d14-aac1f467-91d8cec5-8308dc1c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:34.380256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:34.381456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7c927d14-aac1f467-91d8cec5-8308dc1c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:34.381507Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:34.381521Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2493
2025-10-09T21:52:34.381641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:34.381663Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2493 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:34.381707Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2493 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:34.381715Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2493 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:34.381718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:34.381823Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:34.381836Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:34.381849Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:34.381920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:34.381961Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343248749031764:8146]
2025-10-09T21:52:34.382073Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2493 sessionId:  describe result for acl check
2025-10-09T21:52:34.382104Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:34.382105Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:34.382107Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:34.382112Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343249110885655:4133] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:34.382537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:34.382585Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343249110885655:4133] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:34.382843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:34.382886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:34.383190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:34.383244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.383251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.383255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.383256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.383257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.383258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.383259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.383260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.383262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.383267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.383277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.383281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.383285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.383288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.383291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.383294Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.383297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.383301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.383307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.383315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.383321Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.383326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.383331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.383336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.383340Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.383345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.383350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.383356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.383363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.383368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.383373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.383378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.383383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.383389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.383395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.383400Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.383427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:34.384710Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343249110885655:4133] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:34.388971Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343249110885655:4133] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:34.388982Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343249110885655:4133] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:34.388985Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343249110885655:4133] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:34.388991Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2493 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:34.389414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:34.390504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:34.390519Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:34.390521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:34.390525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:34.390531Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2492 [2:7559343248749031764:8146]
2025-10-09T21:52:34.390544Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:34.390560Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:34.390568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:34.390573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:34.390575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:34.390590Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:34.390595Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:34.390602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:34.390656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.390662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.390664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.390667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:34.390730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:34.390738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.390749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.390758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.390776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:34.390800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885618:2432], now have 1 active actors on pipe
2025-10-09T21:52:34.390808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885694:4133], now have 1 active actors on pipe
2025-10-09T21:52:34.390815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343248749031759:8146]
2025-10-09T21:52:34.390818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343248749031759:8146]
2025-10-09T21:52:34.390820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343248749031759:8146]
2025-10-09T21:52:34.390810Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2492
2025-10-09T21:52:34.390831Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2492
2025-10-09T21:52:34.390833Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:34.390850Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:34.390853Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:34.390854Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:34.390856Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:34.390868Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:34.390870Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:34.390907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.390916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.390928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343248749031759:8146]
2025-10-09T21:52:34.390940Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:34.390964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:34.390968Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7266f2db-fda758a2-78d089d3-b08af456_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:34.390976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:34.390978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:34.390979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:34.390987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:34.390994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:34.391010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:34.391139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.391149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.391167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:34.391270Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2493 partition: 0 MaxSeqNo: 25 sessionId: src_id|7266f2db-fda758a2-78d089d3-b08af456_0
2025-10-09T21:52:34.391525Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7c927d14-aac1f467-91d8cec5-8308dc1c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7266f2db-fda758a2-78d089d3-b08af456_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:34.393802Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7266f2db-fda758a2-78d089d3-b08af456_0] Write session: try to update token
2025-10-09T21:52:34.393818Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7266f2db-fda758a2-78d089d3-b08af456_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:34.405014Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:34.405032Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:34.405035Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:34.405037Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:34.405047Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343248749031759:8146]
2025-10-09T21:52:34.405052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343248749031759:8146]
2025-10-09T21:52:34.405059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343248749031759:8146]
2025-10-09T21:52:34.405106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:34.405112Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:34.405117Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:34.405120Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:34.405122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.405125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:34.405281Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:34.406558Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2493 sessionId: src_id|7266f2db-fda758a2-78d089d3-b08af456_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:34.408784Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:34.472948Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343249110885702:5758]
2025-10-09T21:52:34.473235Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 91 pipe creation result: OK TabletId: 72075186224037892 Generation: 2492, pipe: [1:7559343249110885702:5758]
2025-10-09T21:52:34.550667Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:34.550915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.550928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.550966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885702:5758], now have 1 active actors on pipe
2025-10-09T21:52:34.551126Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:34.551210Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:34.551223Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:34.551224Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:34.551262Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 91 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343249110885702:5758]
2025-10-09T21:52:34.551284Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2493 reason: error at write: Disconnected sessionId: src_id|7266f2db-fda758a2-78d089d3-b08af456_0
2025-10-09T21:52:34.551442Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2493 sessionId: src_id|7266f2db-fda758a2-78d089d3-b08af456_0 is DEAD
2025-10-09T21:52:34.551602Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7266f2db-fda758a2-78d089d3-b08af456_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:34.551612Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7266f2db-fda758a2-78d089d3-b08af456_0] Write session will restart in 0.008687s
2025-10-09T21:52:34.551632Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7266f2db-fda758a2-78d089d3-b08af456_0] Write session: Do CDS request
2025-10-09T21:52:34.551637Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7266f2db-fda758a2-78d089d3-b08af456_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:34.559040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:34.560559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:34.560749Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:34.560758Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:34.560769Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:34.560817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:34.560863Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343248749031838:8136]
2025-10-09T21:52:34.561505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7266f2db-fda758a2-78d089d3-b08af456_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:34.561495Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:34.561592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:34.561606Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2494
2025-10-09T21:52:34.561734Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2494 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:34.561763Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2494 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:34.561767Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2494 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:34.561771Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:34.561860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:34.561921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:34.562199Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2494 sessionId:  describe result for acl check
2025-10-09T21:52:34.562237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:34.562243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:34.562245Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:34.562251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343249110885744:4122] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:34.562276Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:34.562333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.562339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.562341Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.562342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.562343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.562344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.562346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.562347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.562348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.562354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.562366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.562374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.562378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.562387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.562391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.562394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.562399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.562407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.562415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.562429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.562440Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.562450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.562458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.562464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.562472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.562478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.562483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.562492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.562497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.562503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.562513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.562523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.562529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.562534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.562542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.562547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.562580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:34.562803Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343249110885744:4122] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:34.564788Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343249110885744:4122] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:34.568536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343249110885744:4122] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:34.568549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343249110885744:4122] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:34.568553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343249110885744:4122] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:34.568562Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2494 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:34.568964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:34.570307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:34.570325Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:34.570327Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:34.570332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:34.570338Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2493 [2:7559343248749031838:8136]
2025-10-09T21:52:34.570352Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:34.570370Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:34.570378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:34.570381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:34.570384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:34.570403Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:34.570413Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:34.570421Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:34.570479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.570487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.570489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.570491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:34.570556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:34.570565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.570577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.570589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.570606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:34.570626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885781:4122], now have 1 active actors on pipe
2025-10-09T21:52:34.570634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885704:2432], now have 1 active actors on pipe
2025-10-09T21:52:34.570642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343248749031833:8136]
2025-10-09T21:52:34.570648Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343248749031833:8136]
2025-10-09T21:52:34.570650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343248749031833:8136]
2025-10-09T21:52:34.570664Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:34.570683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:34.570701Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:34.570707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:34.570711Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:34.570713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:34.570716Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:34.570718Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2493
2025-10-09T21:52:34.570719Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2493
2025-10-09T21:52:34.570759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343248749031833:8136]
2025-10-09T21:52:34.570782Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:34.570804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.570808Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:34.570810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.570885Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|70d01953-31a7a525-60931a96-c456c0a9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:34.570906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:34.570914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:34.570916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:34.570927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:34.570943Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:34.570972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:34.571147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.571161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.571183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:34.571278Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2494 partition: 0 MaxSeqNo: 25 sessionId: src_id|70d01953-31a7a525-60931a96-c456c0a9_0
2025-10-09T21:52:34.571598Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7266f2db-fda758a2-78d089d3-b08af456_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|70d01953-31a7a525-60931a96-c456c0a9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:34.573934Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70d01953-31a7a525-60931a96-c456c0a9_0] Write session: try to update token
2025-10-09T21:52:34.573948Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70d01953-31a7a525-60931a96-c456c0a9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:34.585113Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:34.585129Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:34.585131Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:34.585132Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:34.585141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343248749031833:8136]
2025-10-09T21:52:34.585145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343248749031833:8136]
2025-10-09T21:52:34.585147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343248749031833:8136]
2025-10-09T21:52:34.585181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:34.585183Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:34.585187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.585188Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:34.585188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:34.585190Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:34.585322Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:34.586194Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2494 sessionId: src_id|70d01953-31a7a525-60931a96-c456c0a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:34.588555Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:34.651659Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343249110885789:5758]
2025-10-09T21:52:34.651885Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 92 pipe creation result: OK TabletId: 72075186224037892 Generation: 2493, pipe: [1:7559343249110885789:5758]
2025-10-09T21:52:34.728233Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:34.728462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.728484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.728531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885789:5758], now have 1 active actors on pipe
2025-10-09T21:52:34.728696Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:34.728791Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:34.728799Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:34.728803Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:34.728843Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 92 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343249110885789:5758]
2025-10-09T21:52:34.728859Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2494 reason: error at write: Disconnected sessionId: src_id|70d01953-31a7a525-60931a96-c456c0a9_0
2025-10-09T21:52:34.729012Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2494 sessionId: src_id|70d01953-31a7a525-60931a96-c456c0a9_0 is DEAD
2025-10-09T21:52:34.729169Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70d01953-31a7a525-60931a96-c456c0a9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:34.729181Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70d01953-31a7a525-60931a96-c456c0a9_0] Write session will restart in 0.008149s
2025-10-09T21:52:34.729209Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70d01953-31a7a525-60931a96-c456c0a9_0] Write session: Do CDS request
2025-10-09T21:52:34.729214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70d01953-31a7a525-60931a96-c456c0a9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:34.737316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:34.738546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|70d01953-31a7a525-60931a96-c456c0a9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:34.738601Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:34.738615Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2495
2025-10-09T21:52:34.738751Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2495 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:34.738751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:34.738785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2495 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:34.738796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2495 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:34.738804Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:34.738963Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:34.738976Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:34.738986Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:34.739045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:34.739089Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343248749031916:8137]
2025-10-09T21:52:34.739182Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2495 sessionId:  describe result for acl check
2025-10-09T21:52:34.739215Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:34.739221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:34.739224Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:34.739229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343249110885831:4105] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:34.739661Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:34.739793Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343249110885831:4105] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:34.739966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:34.740014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:34.740284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:34.740343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.740348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.740349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.740350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.740351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.740352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.740352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.740353Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.740354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.740357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.740363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.740365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.740367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.740370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.740372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.740374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.740376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.740378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.740384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.740389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.740393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.740397Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.740399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.740403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.740405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.740408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.740411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.740414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.740416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.740419Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.740422Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.740425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.740429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.740433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.740435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.740438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.740454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:34.741523Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343249110885831:4105] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:34.746308Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343249110885831:4105] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:34.746324Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343249110885831:4105] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:34.746328Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343249110885831:4105] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:34.746334Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2495 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:34.746680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:34.747718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:34.747737Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:34.747740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:34.747744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:34.747750Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2494 [2:7559343248749031916:8137]
2025-10-09T21:52:34.747766Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:34.747787Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:34.747795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:34.747799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:34.747801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:34.747819Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:34.747829Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:34.747840Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:34.747900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.747909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.747911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.747913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:34.747985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:34.747994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.748005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.748017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.748038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:34.748065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885870:4105], now have 1 active actors on pipe
2025-10-09T21:52:34.748076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885791:2432], now have 1 active actors on pipe
2025-10-09T21:52:34.748083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343248749031911:8137]
2025-10-09T21:52:34.748086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343248749031911:8137]
2025-10-09T21:52:34.748087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343248749031911:8137]
2025-10-09T21:52:34.748079Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2494
2025-10-09T21:52:34.748101Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:34.748099Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2494
2025-10-09T21:52:34.748116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:34.748121Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:34.748124Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:34.748126Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:34.748130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:34.748134Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:34.748163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343248749031911:8137]
2025-10-09T21:52:34.748166Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:34.748175Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:34.748182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.748187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.748242Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5310650a-47fc9080-fecbb334-f58e3e0d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:34.748255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:34.748259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:34.748262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:34.748272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:34.748285Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:34.748310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:34.748509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.748521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.748550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:34.748667Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2495 partition: 0 MaxSeqNo: 25 sessionId: src_id|5310650a-47fc9080-fecbb334-f58e3e0d_0
2025-10-09T21:52:34.748968Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|70d01953-31a7a525-60931a96-c456c0a9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5310650a-47fc9080-fecbb334-f58e3e0d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:34.751351Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5310650a-47fc9080-fecbb334-f58e3e0d_0] Write session: try to update token
2025-10-09T21:52:34.751367Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5310650a-47fc9080-fecbb334-f58e3e0d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:34.762153Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:34.762169Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:34.762171Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:34.762174Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:34.762186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343248749031911:8137]
2025-10-09T21:52:34.762189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343248749031911:8137]
2025-10-09T21:52:34.762192Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343248749031911:8137]
2025-10-09T21:52:34.762226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:34.762232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.762234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:34.762234Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:34.762240Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:34.762244Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:34.762353Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:34.763215Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2495 sessionId: src_id|5310650a-47fc9080-fecbb334-f58e3e0d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:34.765401Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:34.828791Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343249110885881:5758]
2025-10-09T21:52:34.828987Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 93 pipe creation result: OK TabletId: 72075186224037892 Generation: 2494, pipe: [1:7559343249110885881:5758]
2025-10-09T21:52:34.910962Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:34.911333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.911347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.911390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885881:5758], now have 1 active actors on pipe
2025-10-09T21:52:34.911567Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:34.911603Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 93 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343249110885881:5758]
2025-10-09T21:52:34.911631Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:34.911710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2495 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5310650a-47fc9080-fecbb334-f58e3e0d_0
2025-10-09T21:52:34.911710Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:34.911899Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2495 sessionId: src_id|5310650a-47fc9080-fecbb334-f58e3e0d_0 is DEAD
2025-10-09T21:52:34.912047Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5310650a-47fc9080-fecbb334-f58e3e0d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:34.912056Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5310650a-47fc9080-fecbb334-f58e3e0d_0] Write session will restart in 0.009724s
2025-10-09T21:52:34.912079Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5310650a-47fc9080-fecbb334-f58e3e0d_0] Write session: Do CDS request
2025-10-09T21:52:34.912085Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5310650a-47fc9080-fecbb334-f58e3e0d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:34.920010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:34.921524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:34.921712Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:34.921721Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:34.921731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:34.921775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:34.921808Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343248749031996:8133]
2025-10-09T21:52:34.922279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:34.922490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5310650a-47fc9080-fecbb334-f58e3e0d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:34.922559Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:34.922595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:34.922632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:34.922655Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2496
2025-10-09T21:52:34.922832Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2496 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:34.922841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:34.922870Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2496 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:34.922884Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2496 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:34.922892Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:34.922904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.922911Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.922913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.922914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.922915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.922916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.922917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.922918Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.922919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.922924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.922934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.922938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.922943Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.922947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.922952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.922955Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.922959Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.922962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.922971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.922979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:34.922987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.922992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:34.922997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.923001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:34.923004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.923007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:34.923010Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.923013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:34.923016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.923018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:34.923021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.923024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.923028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.923031Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.923033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.923036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.923053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:34.923428Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2496 sessionId:  describe result for acl check
2025-10-09T21:52:34.923474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:34.923477Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:34.923479Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:34.923486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343249110885928:4106] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:34.924134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343249110885928:4106] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:34.926420Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343249110885928:4106] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:34.928825Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343249110885928:4106] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:34.928837Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343249110885928:4106] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:34.928840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343249110885928:4106] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:34.928846Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2496 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:34.929284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:34.930290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:34.930305Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:34.930307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:34.930310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:34.930316Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2495 [2:7559343248749031996:8133]
2025-10-09T21:52:34.930340Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:34.930353Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:34.930355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:34.930357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:34.930358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:34.930370Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:34.930377Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:34.930384Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:34.930440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.930447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.930448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.930450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:34.930542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:34.930550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.930558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:34.930564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:34.930576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:34.930592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885884:2432], now have 1 active actors on pipe
2025-10-09T21:52:34.930595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343249110885965:4106], now have 1 active actors on pipe
2025-10-09T21:52:34.930600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343248749031991:8133]
2025-10-09T21:52:34.930602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343248749031991:8133]
2025-10-09T21:52:34.930603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343248749031991:8133]
2025-10-09T21:52:34.930613Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:34.930626Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:34.930640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:34.930644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:34.930647Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:34.930650Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:34.930652Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:34.930674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343248749031991:8133]
2025-10-09T21:52:34.930659Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2495
2025-10-09T21:52:34.930682Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:34.930692Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:34.930663Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2495
2025-10-09T21:52:34.930720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.930727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.930755Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b3b3c109-676c1b32-4b2f655a-205f9eb1_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:34.930763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:34.930770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:34.930771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:34.930780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:34.930788Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:34.930802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:34.930962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:34.930971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:34.930983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:34.931077Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2496 partition: 0 MaxSeqNo: 25 sessionId: src_id|b3b3c109-676c1b32-4b2f655a-205f9eb1_0
2025-10-09T21:52:34.931457Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5310650a-47fc9080-fecbb334-f58e3e0d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b3b3c109-676c1b32-4b2f655a-205f9eb1_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:34.933681Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3b3c109-676c1b32-4b2f655a-205f9eb1_0] Write session: try to update token
2025-10-09T21:52:34.933699Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3b3c109-676c1b32-4b2f655a-205f9eb1_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:34.943990Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:34.944010Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:34.944012Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:34.944013Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:34.944020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343248749031991:8133]
2025-10-09T21:52:34.944024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343248749031991:8133]
2025-10-09T21:52:34.944026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343248749031991:8133]
2025-10-09T21:52:34.944059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:34.944061Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:34.944064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:34.944066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:34.944067Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:34.944071Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:34.944184Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:34.945908Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2496 sessionId: src_id|b3b3c109-676c1b32-4b2f655a-205f9eb1_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:34.948786Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:35.011630Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343253405853269:5758]
2025-10-09T21:52:35.011993Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 94 pipe creation result: OK TabletId: 72075186224037892 Generation: 2495, pipe: [1:7559343253405853269:5758]
2025-10-09T21:52:35.093908Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:35.094185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:35.094198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:35.094239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343253405853269:5758], now have 1 active actors on pipe
2025-10-09T21:52:35.094433Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:35.094552Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:35.094560Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 94 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343253405853269:5758]
2025-10-09T21:52:35.094587Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:35.094687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2496 reason: status is not ok: tablet will be restarted right now sessionId: src_id|b3b3c109-676c1b32-4b2f655a-205f9eb1_0
2025-10-09T21:52:35.094871Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2496 sessionId: src_id|b3b3c109-676c1b32-4b2f655a-205f9eb1_0 is DEAD
2025-10-09T21:52:35.095041Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3b3c109-676c1b32-4b2f655a-205f9eb1_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:35.095054Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3b3c109-676c1b32-4b2f655a-205f9eb1_0] Write session will restart in 0.009927s
2025-10-09T21:52:35.095088Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3b3c109-676c1b32-4b2f655a-205f9eb1_0] Write session: Do CDS request
2025-10-09T21:52:35.095094Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3b3c109-676c1b32-4b2f655a-205f9eb1_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:35.102801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:35.104405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:35.104636Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:35.104655Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:35.104668Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:35.104724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:35.104773Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343253043999371:8131]
2025-10-09T21:52:35.105415Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:35.105760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:35.105819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:35.106117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:35.106161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.106166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.106166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.106167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.106168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.106168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.106169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.106170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.106171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.106175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.106183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.106185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.106187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.106189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.106192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.106194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.106196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.106198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.106206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.106211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.106217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.106220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.106223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.106227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.106230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.106233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.106243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.106246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.106249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.106252Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.106256Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.106259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.106262Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.106265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.106268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.106272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.106290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:35.106633Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b3b3c109-676c1b32-4b2f655a-205f9eb1_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:35.106657Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:35.106678Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2497
2025-10-09T21:52:35.106846Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2497 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:35.106895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2497 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:35.106905Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2497 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:35.106909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:35.107291Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2497 sessionId:  describe result for acl check
2025-10-09T21:52:35.107332Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:35.107341Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:35.107343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:35.107350Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343253405853317:4089] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:35.107902Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343253405853317:4089] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:35.109766Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343253405853317:4089] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:35.113004Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343253405853317:4089] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:35.113024Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343253405853317:4089] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:35.113030Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343253405853317:4089] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:35.113039Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2497 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:35.113427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:35.114608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:35.114630Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:35.114634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:35.114640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:35.114650Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2496 [2:7559343253043999371:8131]
2025-10-09T21:52:35.114667Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:35.114689Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:35.114697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:35.114701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:35.114704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:35.114729Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:35.114742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:35.114750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:35.114814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.114821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.114825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.114829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:35.114912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:35.114916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.114931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.114944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.114967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:35.114996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343253405853271:2432], now have 1 active actors on pipe
2025-10-09T21:52:35.115004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343253405853352:4089], now have 1 active actors on pipe
2025-10-09T21:52:35.115012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343253043999366:8131]
2025-10-09T21:52:35.115016Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343253043999366:8131]
2025-10-09T21:52:35.115020Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343253043999366:8131]
2025-10-09T21:52:35.115032Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:35.115053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:35.115059Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:35.115062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:35.115064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:35.115074Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:35.115077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:35.115078Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2496
2025-10-09T21:52:35.115122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343253043999366:8131]
2025-10-09T21:52:35.115106Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2496
2025-10-09T21:52:35.115131Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:35.115133Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:35.115169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:35.115178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:35.115243Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|6140057c-598ac205-d5fa766b-b53d1c61_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:35.115256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:35.115267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:35.115274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:35.115285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:35.115300Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:35.115334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:35.115510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:35.115526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:35.115553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:35.115710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2497 partition: 0 MaxSeqNo: 25 sessionId: src_id|6140057c-598ac205-d5fa766b-b53d1c61_0
2025-10-09T21:52:35.116092Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b3b3c109-676c1b32-4b2f655a-205f9eb1_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|6140057c-598ac205-d5fa766b-b53d1c61_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:35.119319Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6140057c-598ac205-d5fa766b-b53d1c61_0] Write session: try to update token
2025-10-09T21:52:35.119343Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6140057c-598ac205-d5fa766b-b53d1c61_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:35.129812Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:35.129832Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:35.129834Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:35.129835Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:35.129843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343253043999366:8131]
2025-10-09T21:52:35.129846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343253043999366:8131]
2025-10-09T21:52:35.129852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343253043999366:8131]
2025-10-09T21:52:35.129893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:35.129899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.129901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:35.129908Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:35.129916Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:35.129920Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:35.130066Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:35.136932Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2497 sessionId: src_id|6140057c-598ac205-d5fa766b-b53d1c61_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:35.140081Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:35.194870Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343253405853359:5758]
2025-10-09T21:52:35.195186Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 95 pipe creation result: OK TabletId: 72075186224037892 Generation: 2496, pipe: [1:7559343253405853359:5758]
2025-10-09T21:52:35.284559Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:35.284917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:35.284934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:35.284958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343253405853359:5758], now have 1 active actors on pipe
2025-10-09T21:52:35.285262Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:35.285270Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 95 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343253405853359:5758]
2025-10-09T21:52:35.285274Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:35.285315Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:35.285317Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:35.285407Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2497 reason: error at write: Disconnected sessionId: src_id|6140057c-598ac205-d5fa766b-b53d1c61_0
2025-10-09T21:52:35.285598Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2497 sessionId: src_id|6140057c-598ac205-d5fa766b-b53d1c61_0 is DEAD
2025-10-09T21:52:35.285764Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6140057c-598ac205-d5fa766b-b53d1c61_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:35.285773Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6140057c-598ac205-d5fa766b-b53d1c61_0] Write session will restart in 0.007115s
2025-10-09T21:52:35.285794Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6140057c-598ac205-d5fa766b-b53d1c61_0] Write session: Do CDS request
2025-10-09T21:52:35.285799Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6140057c-598ac205-d5fa766b-b53d1c61_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:35.293804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:35.294541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|6140057c-598ac205-d5fa766b-b53d1c61_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:35.294637Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:35.294651Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2498
2025-10-09T21:52:35.294773Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2498 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:35.294798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2498 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:35.294805Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2498 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:35.294809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:35.295245Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2498 sessionId:  describe result for acl check
2025-10-09T21:52:35.295289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:35.295296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:35.295299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:35.295306Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343253405853430:4058] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:35.295896Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343253405853430:4058] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:35.296129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:35.296332Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:35.296345Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:35.296356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:35.296414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:35.296462Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343253043999488:8101]
2025-10-09T21:52:35.297081Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:35.297483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:35.297530Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:35.297775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:35.297814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.297820Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.297821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.297822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.297823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.297824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.297824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.297825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.297826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.297829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.297835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.297838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.297841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.297844Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.297846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.297848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.297850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.297852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.297859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.297864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.297861Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343253405853430:4058] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:35.297867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.297872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.297875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.297878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.297881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.297884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.297887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.297890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.297893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.297897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.297901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.297904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.297907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.297910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.297913Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.297916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.297933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:35.304385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343253405853430:4058] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:35.304405Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343253405853430:4058] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:35.304413Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343253405853430:4058] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:35.304424Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2498 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:35.304731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:35.305948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:35.305962Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:35.305964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:35.305967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:35.305973Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2497 [2:7559343253043999488:8101]
2025-10-09T21:52:35.305988Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:35.306004Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:35.306006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:35.306008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:35.306009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:35.306024Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:35.306042Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:35.306048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:35.306106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.306114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.306117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.306120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:35.306176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:35.306182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.306190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.306196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.306205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:35.306224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343253043999483:8101]
2025-10-09T21:52:35.306229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343253043999483:8101]
2025-10-09T21:52:35.306230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343253043999483:8101]
2025-10-09T21:52:35.306238Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:35.306250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:35.306250Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:35.306254Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:35.306256Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:35.306262Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:35.306267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:35.306276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343253405853467:4058], now have 1 active actors on pipe
2025-10-09T21:52:35.306279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343253405853389:2432], now have 1 active actors on pipe
2025-10-09T21:52:35.306307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343253043999483:8101]
2025-10-09T21:52:35.306316Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:35.306329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:35.306327Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2497
2025-10-09T21:52:35.306337Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2497
2025-10-09T21:52:35.306411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:35.306419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:35.306446Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|dac58014-6d715531-cf77dad1-a6f39c1f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:35.306455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:35.306458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:35.306459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:35.306467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:35.306479Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:35.306495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:35.306638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:35.306644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:35.306656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:35.306796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2498 partition: 0 MaxSeqNo: 25 sessionId: src_id|dac58014-6d715531-cf77dad1-a6f39c1f_0
2025-10-09T21:52:35.307165Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|6140057c-598ac205-d5fa766b-b53d1c61_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|dac58014-6d715531-cf77dad1-a6f39c1f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:35.309835Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dac58014-6d715531-cf77dad1-a6f39c1f_0] Write session: try to update token
2025-10-09T21:52:35.309854Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dac58014-6d715531-cf77dad1-a6f39c1f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:35.320767Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:35.320786Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:35.320788Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:35.320790Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:35.320800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343253043999483:8101]
2025-10-09T21:52:35.320804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343253043999483:8101]
2025-10-09T21:52:35.320808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343253043999483:8101]
2025-10-09T21:52:35.320852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:35.320859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.320862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:35.320862Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:35.320869Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:35.320873Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:35.321000Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:35.322582Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2498 sessionId: src_id|dac58014-6d715531-cf77dad1-a6f39c1f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:35.324906Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:35.385351Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343253405853479:5758]
2025-10-09T21:52:35.385661Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 96 pipe creation result: OK TabletId: 72075186224037892 Generation: 2497, pipe: [1:7559343253405853479:5758]
2025-10-09T21:52:35.469873Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:35.470130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:35.470142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:35.470181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343253405853479:5758], now have 1 active actors on pipe
2025-10-09T21:52:35.470324Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:35.470424Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:35.470424Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:35.470470Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 96 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343253405853479:5758]
2025-10-09T21:52:35.470503Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2498 reason: status is not ok: tablet will be restarted right now sessionId: src_id|dac58014-6d715531-cf77dad1-a6f39c1f_0
2025-10-09T21:52:35.470673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2498 sessionId: src_id|dac58014-6d715531-cf77dad1-a6f39c1f_0 is DEAD
2025-10-09T21:52:35.470832Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dac58014-6d715531-cf77dad1-a6f39c1f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:35.470844Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dac58014-6d715531-cf77dad1-a6f39c1f_0] Write session will restart in 0.007849s
2025-10-09T21:52:35.470871Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dac58014-6d715531-cf77dad1-a6f39c1f_0] Write session: Do CDS request
2025-10-09T21:52:35.470878Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dac58014-6d715531-cf77dad1-a6f39c1f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:35.477801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:35.479349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:35.479465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|dac58014-6d715531-cf77dad1-a6f39c1f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:35.479522Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:35.479532Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:35.479543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:35.479515Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:35.479526Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2499
2025-10-09T21:52:35.479584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:35.479612Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343253043999568:8108]
2025-10-09T21:52:35.479629Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2499 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:35.479654Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2499 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:35.479658Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2499 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:35.479660Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:35.479976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:35.479968Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2499 sessionId:  describe result for acl check
2025-10-09T21:52:35.479996Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:35.479997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:35.479998Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:35.480002Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343253405853525:4050] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:35.480239Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:35.480271Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:35.480443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343253405853525:4050] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:35.480481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:35.480520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.480526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.480527Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.480528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.480531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.480532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.480533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.480534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.480535Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.480539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.480549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.480553Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.480557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.480560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.480562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.480564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.480566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.480568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.480573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.480577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.480582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.480585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.480588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.480591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.480593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.480596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.480598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.480601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.480604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.480607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.480610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.480613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.480615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.480618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.480622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.480625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.480641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:35.482311Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343253405853525:4050] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:35.486583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343253405853525:4050] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:35.486596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343253405853525:4050] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:35.486599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343253405853525:4050] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:35.486604Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2499 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:35.486999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:35.487983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:35.487999Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:35.488001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:35.488005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:35.488011Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2498 [2:7559343253043999568:8108]
2025-10-09T21:52:35.488022Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:35.488034Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:35.488038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:35.488040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:35.488042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:35.488054Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:35.488058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:35.488063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:35.488102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.488108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.488109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.488111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:35.488162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:35.488168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.488176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.488182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.488191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:35.488206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343253405853564:4050], now have 1 active actors on pipe
2025-10-09T21:52:35.488213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343253405853482:2432], now have 1 active actors on pipe
2025-10-09T21:52:35.488219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343253043999563:8108]
2025-10-09T21:52:35.488226Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343253043999563:8108]
2025-10-09T21:52:35.488230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343253043999563:8108]
2025-10-09T21:52:35.488242Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:35.488255Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:35.488260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:35.488265Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:35.488267Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:35.488274Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:35.488277Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:35.488266Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2498
2025-10-09T21:52:35.488277Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2498
2025-10-09T21:52:35.488300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343253043999563:8108]
2025-10-09T21:52:35.488307Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:35.488310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:35.488321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:35.488330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:35.488379Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|75d9df72-94e468d1-c8a1fc36-11c7dbb5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:35.488391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:35.488397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:35.488399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:35.488406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:35.488418Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:35.488434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:35.488547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:35.488555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:35.488566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:35.488641Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2499 partition: 0 MaxSeqNo: 25 sessionId: src_id|75d9df72-94e468d1-c8a1fc36-11c7dbb5_0
2025-10-09T21:52:35.488929Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|dac58014-6d715531-cf77dad1-a6f39c1f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|75d9df72-94e468d1-c8a1fc36-11c7dbb5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:35.491275Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|75d9df72-94e468d1-c8a1fc36-11c7dbb5_0] Write session: try to update token
2025-10-09T21:52:35.491292Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|75d9df72-94e468d1-c8a1fc36-11c7dbb5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:35.502509Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:35.502528Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:35.502530Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:35.502531Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:35.502538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343253043999563:8108]
2025-10-09T21:52:35.502541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343253043999563:8108]
2025-10-09T21:52:35.502547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343253043999563:8108]
2025-10-09T21:52:35.502584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:35.502589Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:35.502593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.502595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:35.502606Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:35.502609Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:35.502717Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:35.503488Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2499 sessionId: src_id|75d9df72-94e468d1-c8a1fc36-11c7dbb5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:35.505885Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:35.570683Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343253405853571:5758]
2025-10-09T21:52:35.571003Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 97 pipe creation result: OK TabletId: 72075186224037892 Generation: 2498, pipe: [1:7559343253405853571:5758]
2025-10-09T21:52:35.646987Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:35.647272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:35.647285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:35.647327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343253405853571:5758], now have 1 active actors on pipe
2025-10-09T21:52:35.647505Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:35.647576Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:35.647644Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 97 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343253405853571:5758]
2025-10-09T21:52:35.647678Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:35.647687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2499 reason: status is not ok: tablet will be restarted right now sessionId: src_id|75d9df72-94e468d1-c8a1fc36-11c7dbb5_0
2025-10-09T21:52:35.647873Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2499 sessionId: src_id|75d9df72-94e468d1-c8a1fc36-11c7dbb5_0 is DEAD
2025-10-09T21:52:35.648028Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|75d9df72-94e468d1-c8a1fc36-11c7dbb5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:35.648039Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|75d9df72-94e468d1-c8a1fc36-11c7dbb5_0] Write session will restart in 0.008831s
2025-10-09T21:52:35.648068Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|75d9df72-94e468d1-c8a1fc36-11c7dbb5_0] Write session: Do CDS request
2025-10-09T21:52:35.648075Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|75d9df72-94e468d1-c8a1fc36-11c7dbb5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:35.655617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:35.657393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:35.657590Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:35.657601Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:35.657613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:35.657669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:35.657707Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343253043999649:8105]
2025-10-09T21:52:35.658326Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:35.658484Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|75d9df72-94e468d1-c8a1fc36-11c7dbb5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:35.658568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:35.658581Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2500
2025-10-09T21:52:35.658634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:35.658686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:35.658721Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2500 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:35.658751Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2500 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:35.658762Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2500 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:35.658766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:35.658912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:35.658945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.658949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.658950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.658951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.658951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.658952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.658952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.658953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.658954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.658957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.658965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.658967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.658969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.658971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.658973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.658975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.658977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.658979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.658985Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.658991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.658996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.659000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.659002Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.659005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.659008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.659011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.659014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.659017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.659019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.659022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.659025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.659028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.659039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.659042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.659045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.659047Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.659063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:35.659111Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2500 sessionId:  describe result for acl check
2025-10-09T21:52:35.659142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:35.659146Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:35.659147Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:35.659152Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343253405853621:4029] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:35.659634Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343253405853621:4029] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:35.661544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343253405853621:4029] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:35.664576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343253405853621:4029] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:35.664588Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343253405853621:4029] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:35.664591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343253405853621:4029] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:35.664598Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2500 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:35.664987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:35.665977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:35.665992Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:35.665994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:35.665997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:35.666002Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2499 [2:7559343253043999649:8105]
2025-10-09T21:52:35.666011Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:35.666024Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:35.666026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:35.666044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:35.666050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:35.666065Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:35.666073Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:35.666078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:35.666123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.666128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.666130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.666133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:35.666199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:35.666207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.666214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.666220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.666230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:35.666246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343253405853573:2432], now have 1 active actors on pipe
2025-10-09T21:52:35.666252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343253405853657:4029], now have 1 active actors on pipe
2025-10-09T21:52:35.666258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343253043999644:8105]
2025-10-09T21:52:35.666262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343253043999644:8105]
2025-10-09T21:52:35.666264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343253043999644:8105]
2025-10-09T21:52:35.666271Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:35.666284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:35.666287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:35.666293Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:35.666294Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:35.666296Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:35.666299Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:35.666326Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343253043999644:8105]
2025-10-09T21:52:35.666331Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:35.666327Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2499
2025-10-09T21:52:35.666345Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:35.666329Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2499
2025-10-09T21:52:35.666367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:35.666374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:35.666396Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7a8d3eef-d12d82db-85bf9c32-957ad6e4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:35.666409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:35.666417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:35.666418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:35.666427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:35.666436Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:35.666454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:35.666620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:35.666634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:35.666651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:35.666755Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2500 partition: 0 MaxSeqNo: 25 sessionId: src_id|7a8d3eef-d12d82db-85bf9c32-957ad6e4_0
2025-10-09T21:52:35.667038Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|75d9df72-94e468d1-c8a1fc36-11c7dbb5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7a8d3eef-d12d82db-85bf9c32-957ad6e4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:35.669436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a8d3eef-d12d82db-85bf9c32-957ad6e4_0] Write session: try to update token
2025-10-09T21:52:35.669451Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a8d3eef-d12d82db-85bf9c32-957ad6e4_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:35.677478Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:35.677495Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:35.677504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343253043999644:8105]
2025-10-09T21:52:35.677552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:35.677557Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:35.677731Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:35.677875Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:35.678016Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:35.678173Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:35.678358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:35.678399Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:35.682052Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2500 sessionId: src_id|7a8d3eef-d12d82db-85bf9c32-957ad6e4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:35.684496Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:35.684684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:35.684699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:35.684741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:35.684840Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:35.690307Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:35.690321Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:35.690323Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:35.690326Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:35.690335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343253043999644:8105]
2025-10-09T21:52:35.690338Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343253043999644:8105]
2025-10-09T21:52:35.690340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343253043999644:8105]
2025-10-09T21:52:35.690375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:35.690380Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:35.690389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.690390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:35.690391Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:35.690394Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:35.690530Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:35.747857Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343253405853664:5758]
2025-10-09T21:52:35.748129Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 98 pipe creation result: OK TabletId: 72075186224037892 Generation: 2499, pipe: [1:7559343253405853664:5758]
2025-10-09T21:52:35.844529Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:35.851662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:35.851688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:35.851846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:35.851917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:35.851976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:35.852032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:35.852722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:35.852738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:35.852798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:35.852848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:35.852900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:35.852962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:35.853007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:35.853014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:35.853072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:35.853128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:35.853193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:35.853267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:35.853312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:35.853319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:35.853383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:35.853451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:35.853514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:35.853580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:35.853619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:35.853624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:35.853678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:35.853729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:35.853790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:35.853848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:35.853884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:35.853892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:35.853949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:35.854005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:35.854086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:35.854149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:35.854195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:35.854204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:35.854262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:35.854322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:35.854386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:35.854448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:35.854492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:35.854500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:35.854560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:35.854615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:35.854670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:35.854724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:35.854764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:35.854772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:35.854822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:35.854870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:35.854929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:35.854996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:35.855041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:35.855049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:35.855112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:35.855172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:35.855233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:35.855296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:35.855338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:35.855345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:35.855401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343253405853664:5758], now have 1 active actors on pipe
2025-10-09T21:52:35.855595Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:35.855611Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 98 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343253405853664:5758]
2025-10-09T21:52:35.855640Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:35.855644Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:35.855675Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2500 reason: error at write: Disconnected sessionId: src_id|7a8d3eef-d12d82db-85bf9c32-957ad6e4_0
2025-10-09T21:52:35.855825Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:35.855862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2500 sessionId: src_id|7a8d3eef-d12d82db-85bf9c32-957ad6e4_0 is DEAD
2025-10-09T21:52:35.856022Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a8d3eef-d12d82db-85bf9c32-957ad6e4_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:35.856031Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a8d3eef-d12d82db-85bf9c32-957ad6e4_0] Write session will restart in 0.005437s
2025-10-09T21:52:35.856055Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a8d3eef-d12d82db-85bf9c32-957ad6e4_0] Write session: Do CDS request
2025-10-09T21:52:35.856061Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a8d3eef-d12d82db-85bf9c32-957ad6e4_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:35.862432Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7a8d3eef-d12d82db-85bf9c32-957ad6e4_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:35.862632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:35.862646Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2501
2025-10-09T21:52:35.862751Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2501 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:35.862784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2501 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:35.862790Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2501 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:35.862798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:35.863223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2501 sessionId:  describe result for acl check
2025-10-09T21:52:35.863267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:35.863278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:35.863281Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:35.863287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343253405853742:4020] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:35.863467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:35.863808Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343253405853742:4020] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:35.865001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:35.865191Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:35.865200Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:35.865210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:35.865282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:35.865321Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343253043999735:8098]
2025-10-09T21:52:35.865687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:35.865929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:35.865958Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:35.866192Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:35.866231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.866240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.866240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.866242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.866243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.866244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.866245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.866245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.866246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.866251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.866261Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.866264Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.866268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.866272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.866275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.866279Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.866282Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.866286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.866292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.866299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:35.866306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.866312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:35.866316Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.866322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:35.866328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.866334Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:35.866338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.866347Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:35.866350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.866361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:35.866366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.866371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.866351Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343253405853742:4020] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:35.866376Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.866386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.866390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.866395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.866417Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:35.872216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343253405853742:4020] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:35.872233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343253405853742:4020] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:35.872237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343253405853742:4020] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:35.872246Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2501 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:35.872598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:35.873713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:35.873727Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:35.873730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:35.873734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:35.873741Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2500 [2:7559343253043999735:8098]
2025-10-09T21:52:35.873757Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:35.873773Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:35.873782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:35.873784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:35.873785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:35.873800Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:35.873809Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:35.873815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:35.873870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.873880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.873882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.873885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:35.873959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:35.873968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.873978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:35.873984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:35.873994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:35.874010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343253405853782:4020], now have 1 active actors on pipe
2025-10-09T21:52:35.874016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343253405853697:2432], now have 1 active actors on pipe
2025-10-09T21:52:35.874023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343253043999730:8098]
2025-10-09T21:52:35.874024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343253043999730:8098]
2025-10-09T21:52:35.874026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343253043999730:8098]
2025-10-09T21:52:35.874051Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:35.874064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:35.874069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:35.874073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:35.874074Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:35.874076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:35.874079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:35.874104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343253043999730:8098]
2025-10-09T21:52:35.874088Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2500
2025-10-09T21:52:35.874110Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:35.874115Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:35.874100Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2500
2025-10-09T21:52:35.874139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:35.874146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:35.874171Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4fdcc094-160488e6-90941a86-aa8e6dc6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:35.874182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:35.874185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:35.874186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:35.874193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:35.874204Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:35.874219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:35.874342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:35.874347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:35.874361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:35.874442Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2501 partition: 0 MaxSeqNo: 25 sessionId: src_id|4fdcc094-160488e6-90941a86-aa8e6dc6_0
2025-10-09T21:52:35.874800Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7a8d3eef-d12d82db-85bf9c32-957ad6e4_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4fdcc094-160488e6-90941a86-aa8e6dc6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:35.877170Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4fdcc094-160488e6-90941a86-aa8e6dc6_0] Write session: try to update token
2025-10-09T21:52:35.877188Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4fdcc094-160488e6-90941a86-aa8e6dc6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:35.887801Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:35.887818Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:35.887820Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:35.887821Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:35.887830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343253043999730:8098]
2025-10-09T21:52:35.887835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343253043999730:8098]
2025-10-09T21:52:35.887839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343253043999730:8098]
2025-10-09T21:52:35.887882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:35.887890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:35.887889Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:35.887895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:35.887896Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:35.887900Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:35.888024Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:35.890673Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2501 sessionId: src_id|4fdcc094-160488e6-90941a86-aa8e6dc6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:35.892925Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:35.955715Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343253405853791:5758]
2025-10-09T21:52:35.955965Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 99 pipe creation result: OK TabletId: 72075186224037892 Generation: 2500, pipe: [1:7559343253405853791:5758]
2025-10-09T21:52:36.035348Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:36.035632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.035653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.035702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343253405853791:5758], now have 1 active actors on pipe
2025-10-09T21:52:36.035879Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:36.035922Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:36.035935Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:36.035942Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:36.035987Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2501 reason: error at write: Disconnected sessionId: src_id|4fdcc094-160488e6-90941a86-aa8e6dc6_0
2025-10-09T21:52:36.035991Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 99 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343253405853791:5758]
2025-10-09T21:52:36.036003Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1661: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 closed with error: reason# too much attempts to restart pipe
2025-10-09T21:52:36.036082Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 25 consumer shared/user session shared/user_1_25_16632666203476474409_v1 is DEAD
2025-10-09T21:52:36.036095Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2501 sessionId: src_id|4fdcc094-160488e6-90941a86-aa8e6dc6_0 is DEAD
2025-10-09T21:52:36.036185Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559343171801451938:5764] disconnected.
2025-10-09T21:52:36.036194Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559343171801451938:5764] disconnected; active server actors: 1
2025-10-09T21:52:36.036197Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559343171801451938:5764] client user disconnected session shared/user_1_25_16632666203476474409_v1
2025-10-09T21:52:36.036361Z :ERROR: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got error. Status: UNAVAILABLE. Description: <main>: Error: too much attempts to restart pipe, code: 500027 
2025-10-09T21:52:36.036367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4fdcc094-160488e6-90941a86-aa8e6dc6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:36.036379Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4fdcc094-160488e6-90941a86-aa8e6dc6_0] Write session will restart in 0.007813s
2025-10-09T21:52:36.036404Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4fdcc094-160488e6-90941a86-aa8e6dc6_0] Write session: Do CDS request
2025-10-09T21:52:36.036410Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4fdcc094-160488e6-90941a86-aa8e6dc6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:36.036447Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:52:36.036453Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:52:36.036459Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Reconnecting session to cluster  in 0.007050s
2025-10-09T21:52:36.044241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:36.044345Z :DEBUG: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Successfully connected. Initializing session
2025-10-09T21:52:36.044562Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:52:36.044572Z node 1 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 26
2025-10-09T21:52:36.044714Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 26 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:52:36.044754Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 read init: from# ipv6:[::1]:57708, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:52:36.044819Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 auth for : user
2025-10-09T21:52:36.045035Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 Handle describe topics response
2025-10-09T21:52:36.045067Z node 1 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 auth is DEAD
2025-10-09T21:52:36.045076Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:52:36.045356Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:52:36.045391Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4fdcc094-160488e6-90941a86-aa8e6dc6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:36.045431Z :INFO: [/Root] [/Root] [9ceec5df-385e9549-a9d7823c-d25374c8] [] Got InitResponse. ReadSessionId: shared/user_1_26_14331931971907184625_v1
2025-10-09T21:52:36.045437Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559343257700821137:4007] connected; active server actors: 1
2025-10-09T21:52:36.045449Z node 1 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [1:7559343257700821137:4007] session shared/user_1_26_14331931971907184625_v1
2025-10-09T21:52:36.045456Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:52:36.045465Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:52:36.045469Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_1_26_14331931971907184625_v1" (Sender=[1:7559343257700821134:4007], Pipe=[1:7559343257700821137:4007], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:52:36.045476Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:52:36.045484Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:52:36.045489Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:36.045490Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_1_26_14331931971907184625_v1" (Sender=[1:7559343257700821134:4007], Pipe=[1:7559343257700821137:4007], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:52:36.045493Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2502
2025-10-09T21:52:36.045497Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_1_26_14331931971907184625_v1" sender [1:7559343257700821134:4007] lock partition 0 for ReadingSession "shared/user_1_26_14331931971907184625_v1" (Sender=[1:7559343257700821134:4007], Pipe=[1:7559343257700821137:4007], Partitions=[], ActiveFamilyCount=1) generation 1 step 26
2025-10-09T21:52:36.045503Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:52:36.045506Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000020s
2025-10-09T21:52:36.045603Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2502 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:36.045630Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2502 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:36.045639Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2502 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:36.045642Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:36.045657Z node 1 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 26 Session: "shared/user_1_26_14331931971907184625_v1" ClientId: "user" PipeClient { RawX1: 7559343257700821137 RawX2: 4503603922341799 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:52:36.045672Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:52:36.045774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:36.045895Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2502 sessionId:  describe result for acl check
2025-10-09T21:52:36.045927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:36.045932Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:36.045933Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:36.045940Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343257700821144:3999] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:36.045975Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:36.045985Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:36.045996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:36.046073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:36.046115Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343257338967116:8094]
2025-10-09T21:52:36.046485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343257700821144:3999] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:36.046568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:36.046840Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:36.046875Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:36.047107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:36.047144Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.047149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.047150Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.047151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.047151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.047152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.047152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.047153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.047154Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.047157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.047163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.047165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.047168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.047170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.047172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.047174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.047177Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.047180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.047184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.047188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.047193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.047197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.047200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.047203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.047206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.047209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.047212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.047215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.047217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.047220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.047223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.047230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.047232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.047235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.047238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.047241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.047257Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:36.048191Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343257700821144:3999] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:36.053287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343257700821144:3999] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:36.053301Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343257700821144:3999] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:36.053305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343257700821144:3999] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:36.053314Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2502 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:36.053609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:36.054761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:36.054777Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:36.054779Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:36.054782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:36.054788Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2501 [2:7559343257338967116:8094]
2025-10-09T21:52:36.054800Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:36.054815Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:36.054817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:36.054820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:36.054821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:36.054835Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:36.054839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:36.054845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:36.054904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.054910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.054911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.054913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:36.054965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:36.054971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.054978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.054985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.054997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:36.055021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343257338967110:8094]
2025-10-09T21:52:36.055023Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343257338967110:8094]
2025-10-09T21:52:36.055025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343257338967110:8094]
2025-10-09T21:52:36.055033Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:36.055049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821141:3994], now have 1 active actors on pipe
2025-10-09T21:52:36.055051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:36.055054Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:36.055057Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:36.055063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:36.055078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:36.055088Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:36.055099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821182:3999], now have 1 active actors on pipe
2025-10-09T21:52:36.055091Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2501
2025-10-09T21:52:36.055102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821090:2432], now have 1 active actors on pipe
2025-10-09T21:52:36.055109Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 2501, pipe: [1:7559343257700821141:3994]
2025-10-09T21:52:36.055118Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2501
2025-10-09T21:52:36.055132Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343257338967110:8094]
2025-10-09T21:52:36.055139Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:36.055140Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:36.055158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.055165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.055181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.055186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.055191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_1_26_14331931971907184625_v1 on pipe: [1:7559343257700821141:3994]
2025-10-09T21:52:36.055215Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_1_26_14331931971907184625_v1:1 with generation 2501
2025-10-09T21:52:36.055217Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b331597d-c1843fd-5f10146f-de6e2857_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:36.055224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:36.055226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:36.055228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:36.055235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:36.055242Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:36.055256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:36.055263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:36.055264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:36.055273Z node 2 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_1_26_14331931971907184625_v1
2025-10-09T21:52:36.055278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:36.055302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:36.055318Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:52:36.055408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.055414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.055424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:36.055483Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2502 partition: 0 MaxSeqNo: 25 sessionId: src_id|b331597d-c1843fd-5f10146f-de6e2857_0
2025-10-09T21:52:36.055774Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4fdcc094-160488e6-90941a86-aa8e6dc6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b331597d-c1843fd-5f10146f-de6e2857_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:36.058149Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b331597d-c1843fd-5f10146f-de6e2857_0] Write session: try to update token
2025-10-09T21:52:36.058165Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b331597d-c1843fd-5f10146f-de6e2857_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:36.058582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:52:36.058614Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:52:36.058627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:36.058635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:36.058637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:36.058645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:167: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-10-09T21:52:36.058661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:52:36.058837Z node 1 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 25 WriteTimestampMS: 1760046171000 SizeLag: 5735598 WriteTimestampEstimateMS: 1760046756054 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:52:36.058855Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 25 readOffset 0 committedOffset 0
2025-10-09T21:52:36.058895Z node 1 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 sending to client partition status
2025-10-09T21:52:36.069214Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:36.069234Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:36.069236Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:36.069238Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:36.069259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343257338967110:8094]
2025-10-09T21:52:36.069264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343257338967110:8094]
2025-10-09T21:52:36.069267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343257338967110:8094]
2025-10-09T21:52:36.069301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:36.069307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.069311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:36.069327Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:36.069344Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:36.069348Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:36.069426Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:36.070238Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2502 sessionId: src_id|b331597d-c1843fd-5f10146f-de6e2857_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:36.072572Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:36.215520Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:36.215794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.215809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.215992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_1_26_14331931971907184625_v1
2025-10-09T21:52:36.216008Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:36.216041Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_1_26_14331931971907184625_v1
2025-10-09T21:52:36.216107Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:36.216112Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:36.216121Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:36.216155Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 0 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343257700821141:3994]
2025-10-09T21:52:36.216234Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2502 reason: error at write: Disconnected sessionId: src_id|b331597d-c1843fd-5f10146f-de6e2857_0
2025-10-09T21:52:36.216424Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2502 sessionId: src_id|b331597d-c1843fd-5f10146f-de6e2857_0 is DEAD
2025-10-09T21:52:36.216571Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b331597d-c1843fd-5f10146f-de6e2857_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:36.216585Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b331597d-c1843fd-5f10146f-de6e2857_0] Write session will restart in 0.007090s
2025-10-09T21:52:36.216610Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b331597d-c1843fd-5f10146f-de6e2857_0] Write session: Do CDS request
2025-10-09T21:52:36.216616Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b331597d-c1843fd-5f10146f-de6e2857_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:36.224549Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b331597d-c1843fd-5f10146f-de6e2857_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:36.224566Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:36.224585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2503
2025-10-09T21:52:36.224776Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2503 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:36.224816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2503 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:36.224826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2503 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:36.224831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:36.224847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:36.225228Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2503 sessionId:  describe result for acl check
2025-10-09T21:52:36.225270Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:36.225271Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:36.225273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:36.225277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343257700821239:3986] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:36.225731Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343257700821239:3986] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:36.226484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:36.226656Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:36.226667Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:36.226678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:36.226729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:36.226769Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343257338967201:8092]
2025-10-09T21:52:36.227351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:36.227638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:36.227685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:36.227929Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343257700821239:3986] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:36.227980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:36.228029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.228039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.228040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.228041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.228042Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.228043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.228044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.228045Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.228046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.228051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.228061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.228065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.228069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.228075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.228079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.228083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.228086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.228090Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.228096Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.228103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.228109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.228113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.228117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.228123Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.228127Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.228132Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.228137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.228142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.228146Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.228151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.228157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.228162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.228166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.228172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.228176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.228182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.228211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:36.233203Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343257700821239:3986] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:36.233219Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343257700821239:3986] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:36.233223Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343257700821239:3986] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:36.233233Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2503 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:36.234581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:36.235653Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:36.235670Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:36.235673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:36.235676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:36.235683Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2502 [2:7559343257338967201:8092]
2025-10-09T21:52:36.235698Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:36.235719Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:36.235727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:36.235730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:36.235733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:36.235748Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:36.235758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:36.235769Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:36.235824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.235830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.235832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.235835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:36.235901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:36.235908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.235920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.235935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.235957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:36.235983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821278:3986], now have 1 active actors on pipe
2025-10-09T21:52:36.235991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821192:2432], now have 1 active actors on pipe
2025-10-09T21:52:36.235998Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343257338967196:8092]
2025-10-09T21:52:36.236004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343257338967196:8092]
2025-10-09T21:52:36.236006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343257338967196:8092]
2025-10-09T21:52:36.236019Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:36.236033Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:36.236037Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2502
2025-10-09T21:52:36.236039Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2502
2025-10-09T21:52:36.236038Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:36.236043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:36.236046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:36.236053Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:36.236057Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:36.236088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343257338967196:8092]
2025-10-09T21:52:36.236098Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:36.236104Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:36.236118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.236122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.236168Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|1045d9b2-bf0484f4-f142b3fb-2c492d68_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:36.236183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:36.236187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:36.236190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:36.236200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:36.236217Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:36.236240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:36.236416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.236428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.236443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:36.236522Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2503 partition: 0 MaxSeqNo: 25 sessionId: src_id|1045d9b2-bf0484f4-f142b3fb-2c492d68_0
2025-10-09T21:52:36.236828Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b331597d-c1843fd-5f10146f-de6e2857_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|1045d9b2-bf0484f4-f142b3fb-2c492d68_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:36.239182Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1045d9b2-bf0484f4-f142b3fb-2c492d68_0] Write session: try to update token
2025-10-09T21:52:36.239203Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1045d9b2-bf0484f4-f142b3fb-2c492d68_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:36.250624Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:36.250642Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:36.250644Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:36.250645Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:36.250652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343257338967196:8092]
2025-10-09T21:52:36.250655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343257338967196:8092]
2025-10-09T21:52:36.250658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343257338967196:8092]
2025-10-09T21:52:36.250694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:36.250698Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:36.250701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.250703Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:36.250703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:36.250709Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:36.250851Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:36.251927Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2503 sessionId: src_id|1045d9b2-bf0484f4-f142b3fb-2c492d68_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:36.254417Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:36.316155Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343257700821310:3994]
2025-10-09T21:52:36.316439Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 1 pipe creation result: OK TabletId: 72075186224037892 Generation: 2502, pipe: [1:7559343257700821310:3994]
2025-10-09T21:52:36.400448Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:36.400773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.400786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.400824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821310:3994], now have 1 active actors on pipe
2025-10-09T21:52:36.401050Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:36.401078Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:36.401065Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:36.401087Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 1 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343257700821310:3994]
2025-10-09T21:52:36.401151Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2503 reason: error at write: Disconnected sessionId: src_id|1045d9b2-bf0484f4-f142b3fb-2c492d68_0
2025-10-09T21:52:36.401321Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2503 sessionId: src_id|1045d9b2-bf0484f4-f142b3fb-2c492d68_0 is DEAD
2025-10-09T21:52:36.401457Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1045d9b2-bf0484f4-f142b3fb-2c492d68_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:36.401469Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1045d9b2-bf0484f4-f142b3fb-2c492d68_0] Write session will restart in 0.008181s
2025-10-09T21:52:36.401493Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1045d9b2-bf0484f4-f142b3fb-2c492d68_0] Write session: Do CDS request
2025-10-09T21:52:36.401499Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1045d9b2-bf0484f4-f142b3fb-2c492d68_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:36.401670Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:36.409527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:36.410574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|1045d9b2-bf0484f4-f142b3fb-2c492d68_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:36.410661Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:36.410676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2504
2025-10-09T21:52:36.410839Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2504 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:36.410874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2504 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:36.410893Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2504 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:36.410900Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:36.411083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:36.411296Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:36.411308Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:36.411322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:36.411333Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2504 sessionId:  describe result for acl check
2025-10-09T21:52:36.411367Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:36.411380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:36.411377Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:36.411379Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:36.411385Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343257700821366:3942] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:36.411426Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343257338967326:8072]
2025-10-09T21:52:36.411927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343257700821366:3942] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:36.412014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:36.412265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:36.412309Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:36.412634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:36.412685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.412691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.412693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.412694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.412695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.412696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.412697Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.412699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.412700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.412705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.412715Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.412719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.412723Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.412727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.412732Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.412735Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.412739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.412744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.412752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.412760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.412766Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.412772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.412777Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.412782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.412791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.412797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.412807Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.412812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.412817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.412822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.412827Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.412834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.412838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.412843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.412848Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.412853Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.412883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:36.413793Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343257700821366:3942] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:36.419212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343257700821366:3942] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:36.419228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343257700821366:3942] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:36.419232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343257700821366:3942] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:36.419242Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2504 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:36.419352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:36.420525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:36.420542Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:36.420546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:36.420550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:36.420557Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2503 [2:7559343257338967326:8072]
2025-10-09T21:52:36.420570Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:36.420588Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:36.420591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:36.420594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:36.420596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:36.420614Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:36.420622Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:36.420628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:36.420688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.420695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.420697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.420701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:36.420776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:36.420784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.420796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.420812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.420835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:36.420859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821404:3942], now have 1 active actors on pipe
2025-10-09T21:52:36.420866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821318:2432], now have 1 active actors on pipe
2025-10-09T21:52:36.420874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343257338967321:8072]
2025-10-09T21:52:36.420879Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343257338967321:8072]
2025-10-09T21:52:36.420881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343257338967321:8072]
2025-10-09T21:52:36.420895Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:36.420913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:36.420902Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2503
2025-10-09T21:52:36.420922Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:36.420924Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2503
2025-10-09T21:52:36.420926Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:36.420927Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:36.420928Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:36.420930Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:36.420957Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343257338967321:8072]
2025-10-09T21:52:36.420962Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:36.420964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:36.420992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.421002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.421035Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e75eaacc-aece8c05-115ecd5e-4cffecff_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:36.421049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:36.421057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:36.421059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:36.421069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:36.421084Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:36.421100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:36.421256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.421264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.421278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:36.421358Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2504 partition: 0 MaxSeqNo: 25 sessionId: src_id|e75eaacc-aece8c05-115ecd5e-4cffecff_0
2025-10-09T21:52:36.421683Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|1045d9b2-bf0484f4-f142b3fb-2c492d68_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e75eaacc-aece8c05-115ecd5e-4cffecff_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:36.424064Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e75eaacc-aece8c05-115ecd5e-4cffecff_0] Write session: try to update token
2025-10-09T21:52:36.424081Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e75eaacc-aece8c05-115ecd5e-4cffecff_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:36.434847Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:36.434865Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:36.434868Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:36.434870Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:36.434880Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343257338967321:8072]
2025-10-09T21:52:36.434884Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343257338967321:8072]
2025-10-09T21:52:36.434888Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343257338967321:8072]
2025-10-09T21:52:36.434922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:36.434926Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:36.434928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.434930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:36.434931Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:36.434934Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:36.435050Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:36.436580Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2504 sessionId: src_id|e75eaacc-aece8c05-115ecd5e-4cffecff_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:36.438960Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:36.501500Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343257700821411:3994]
2025-10-09T21:52:36.501752Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 2 pipe creation result: OK TabletId: 72075186224037892 Generation: 2503, pipe: [1:7559343257700821411:3994]
2025-10-09T21:52:36.577575Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:36.577842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.577854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.577894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821411:3994], now have 1 active actors on pipe
2025-10-09T21:52:36.578073Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:36.578183Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 2 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343257700821411:3994]
2025-10-09T21:52:36.578200Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:36.578220Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:36.578269Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2504 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e75eaacc-aece8c05-115ecd5e-4cffecff_0
2025-10-09T21:52:36.578453Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2504 sessionId: src_id|e75eaacc-aece8c05-115ecd5e-4cffecff_0 is DEAD
2025-10-09T21:52:36.578630Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e75eaacc-aece8c05-115ecd5e-4cffecff_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:36.578643Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e75eaacc-aece8c05-115ecd5e-4cffecff_0] Write session will restart in 0.007647s
2025-10-09T21:52:36.578670Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e75eaacc-aece8c05-115ecd5e-4cffecff_0] Write session: Do CDS request
2025-10-09T21:52:36.578676Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e75eaacc-aece8c05-115ecd5e-4cffecff_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:36.586749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:36.587462Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e75eaacc-aece8c05-115ecd5e-4cffecff_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:36.587524Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:36.587538Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2505
2025-10-09T21:52:36.587656Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2505 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:36.587680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2505 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:36.587687Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2505 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:36.587690Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:36.588094Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2505 sessionId:  describe result for acl check
2025-10-09T21:52:36.588132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:36.588137Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:36.588145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:36.588151Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343257700821463:3919] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:36.588436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:36.588600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343257700821463:3919] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:36.588600Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:36.588612Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:36.588624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:36.588673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:36.588712Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343257338967410:8062]
2025-10-09T21:52:36.589278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:36.589538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:36.589580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:36.589888Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:36.589938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.589945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.589946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.589947Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.589948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.589949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.589950Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.589951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.589952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.589957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.589967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.589971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.589975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.589979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.589983Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.589987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.589990Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.589994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.590000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.590007Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.590012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.590017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.590021Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.590025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.590043Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.590053Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.590058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.590063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.590068Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.590073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.590078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.590083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.590088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.590093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.590098Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.590108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.590139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:36.590217Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343257700821463:3919] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:36.595671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343257700821463:3919] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:36.595683Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343257700821463:3919] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:36.595686Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343257700821463:3919] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:36.595692Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2505 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:36.596036Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:36.597037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:36.597053Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:36.597055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:36.597059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:36.597065Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2504 [2:7559343257338967410:8062]
2025-10-09T21:52:36.597078Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:36.597098Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:36.597105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:36.597108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:36.597111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:36.597126Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:36.597136Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:36.597144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:36.597201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.597210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.597212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.597214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:36.597297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:36.597305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.597317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.597334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.597352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:36.597376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821413:2432], now have 1 active actors on pipe
2025-10-09T21:52:36.597385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821502:3919], now have 1 active actors on pipe
2025-10-09T21:52:36.597391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343257338967405:8062]
2025-10-09T21:52:36.597398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343257338967405:8062]
2025-10-09T21:52:36.597400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343257338967405:8062]
2025-10-09T21:52:36.597413Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:36.597430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:36.597431Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:36.597434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:36.597437Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:36.597427Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2504
2025-10-09T21:52:36.597434Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2504
2025-10-09T21:52:36.597449Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:36.597460Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:36.597493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.597500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.597513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343257338967405:8062]
2025-10-09T21:52:36.597524Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:36.597532Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:36.597556Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|74e351a0-72babf28-d565dd4b-5b052817_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:36.597568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:36.597573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:36.597581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:36.597592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:36.597603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:36.597632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:36.597779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.597787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.597800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:36.597896Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2505 partition: 0 MaxSeqNo: 25 sessionId: src_id|74e351a0-72babf28-d565dd4b-5b052817_0
2025-10-09T21:52:36.598210Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e75eaacc-aece8c05-115ecd5e-4cffecff_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|74e351a0-72babf28-d565dd4b-5b052817_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:36.600619Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74e351a0-72babf28-d565dd4b-5b052817_0] Write session: try to update token
2025-10-09T21:52:36.600635Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74e351a0-72babf28-d565dd4b-5b052817_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:36.611517Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:36.611547Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:36.611551Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:36.611552Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:36.611563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343257338967405:8062]
2025-10-09T21:52:36.611568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343257338967405:8062]
2025-10-09T21:52:36.611573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343257338967405:8062]
2025-10-09T21:52:36.611616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:36.611615Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:36.611620Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:36.611623Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:36.611627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.611630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:36.611769Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:36.613526Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2505 sessionId: src_id|74e351a0-72babf28-d565dd4b-5b052817_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:36.615810Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:36.678128Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343257700821510:3994]
2025-10-09T21:52:36.678306Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 3 pipe creation result: OK TabletId: 72075186224037892 Generation: 2504, pipe: [1:7559343257700821510:3994]
2025-10-09T21:52:36.757564Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:36.757877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.757891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.757931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821510:3994], now have 1 active actors on pipe
2025-10-09T21:52:36.758151Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:36.758151Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:36.758165Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:36.758175Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:36.758227Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 3 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343257700821510:3994]
2025-10-09T21:52:36.758232Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2505 reason: error at write: Disconnected sessionId: src_id|74e351a0-72babf28-d565dd4b-5b052817_0
2025-10-09T21:52:36.758402Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2505 sessionId: src_id|74e351a0-72babf28-d565dd4b-5b052817_0 is DEAD
2025-10-09T21:52:36.758557Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74e351a0-72babf28-d565dd4b-5b052817_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:36.758578Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74e351a0-72babf28-d565dd4b-5b052817_0] Write session will restart in 0.009970s
2025-10-09T21:52:36.758605Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74e351a0-72babf28-d565dd4b-5b052817_0] Write session: Do CDS request
2025-10-09T21:52:36.758611Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74e351a0-72babf28-d565dd4b-5b052817_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:36.766354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:36.767948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:36.768135Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:36.768146Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:36.768157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:36.768207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:36.768251Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343257338967498:8061]
2025-10-09T21:52:36.768841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:36.769124Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:36.769165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:36.769484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:36.769536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.769542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.769554Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74e351a0-72babf28-d565dd4b-5b052817_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:36.769543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.769545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.769546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.769547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.769548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.769549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.769550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.769555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.769566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.769573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.769577Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.769581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.769585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.769589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.769593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.769597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.769604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.769611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.769618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.769623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.769627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.769632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.769636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.769642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.769646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.769652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.769656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.769651Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:36.769662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.769667Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2506
2025-10-09T21:52:36.769666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.769671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.769676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.769681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.769685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.769690Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.769719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:36.769785Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2506 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:36.769816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2506 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:36.769827Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2506 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:36.769831Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:36.770205Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2506 sessionId:  describe result for acl check
2025-10-09T21:52:36.770236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:36.770242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:36.770244Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:36.770249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343257700821567:3906] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:36.770813Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343257700821567:3906] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:36.772696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343257700821567:3906] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:36.775560Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343257700821567:3906] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:36.775573Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343257700821567:3906] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:36.775576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343257700821567:3906] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:36.775584Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2506 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:36.775937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:36.776991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:36.777002Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:36.777004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:36.777009Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:36.777014Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2505 [2:7559343257338967498:8061]
2025-10-09T21:52:36.777025Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:36.777038Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:36.777040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:36.777042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:36.777043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:36.777057Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:36.777061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:36.777066Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:36.777109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.777115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.777116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.777118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:36.777166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:36.777172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.777180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.777186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.777195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:36.777209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343257338967493:8061]
2025-10-09T21:52:36.777211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343257338967493:8061]
2025-10-09T21:52:36.777212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343257338967493:8061]
2025-10-09T21:52:36.777220Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:36.777229Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:36.777232Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:36.777234Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:36.777238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821512:2432], now have 1 active actors on pipe
2025-10-09T21:52:36.777256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:36.777266Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:36.777273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:36.777277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821604:3906], now have 1 active actors on pipe
2025-10-09T21:52:36.777299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343257338967493:8061]
2025-10-09T21:52:36.777307Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:36.777342Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2505
2025-10-09T21:52:36.777343Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2505
2025-10-09T21:52:36.777329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:36.777460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.777474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.777502Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5f714f91-ba2108c5-b60d9d8f-b1ad19d5_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:36.777509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:36.777512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:36.777513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:36.777520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:36.777528Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:36.777539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:36.777697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.777703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.777714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:36.777820Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2506 partition: 0 MaxSeqNo: 25 sessionId: src_id|5f714f91-ba2108c5-b60d9d8f-b1ad19d5_0
2025-10-09T21:52:36.778112Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|74e351a0-72babf28-d565dd4b-5b052817_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5f714f91-ba2108c5-b60d9d8f-b1ad19d5_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:36.780535Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f714f91-ba2108c5-b60d9d8f-b1ad19d5_0] Write session: try to update token
2025-10-09T21:52:36.780548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f714f91-ba2108c5-b60d9d8f-b1ad19d5_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:36.788847Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:36.788870Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:36.788880Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343257338967493:8061]
2025-10-09T21:52:36.788919Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:36.788926Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:36.789080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:36.789217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:36.789359Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:36.789495Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:36.789670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:36.789707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:36.792738Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2506 sessionId: src_id|5f714f91-ba2108c5-b60d9d8f-b1ad19d5_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:36.794830Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:36.794995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.795012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.795045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:36.795141Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:36.801225Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:36.801257Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:36.801259Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:36.801261Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:36.801272Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343257338967493:8061]
2025-10-09T21:52:36.801277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343257338967493:8061]
2025-10-09T21:52:36.801281Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343257338967493:8061]
2025-10-09T21:52:36.801319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:36.801338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.801340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:36.801341Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:36.801349Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:36.801354Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:36.801474Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:36.858358Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343257700821612:3994]
2025-10-09T21:52:36.858595Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 4 pipe creation result: OK TabletId: 72075186224037892 Generation: 2505, pipe: [1:7559343257700821612:3994]
2025-10-09T21:52:36.945651Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:36.952656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.952686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.952842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:36.952909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:36.952976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:36.953035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:36.953077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:36.953087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:36.953160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:36.953220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:36.953293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:36.953357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:36.953396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:36.953404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:36.953460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:36.953514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:36.953571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:36.953630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:36.953671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:36.953680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:36.953741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:36.953810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:36.953873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:36.953939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:36.953981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:36.953990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:36.954065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:36.954124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:36.954189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:36.954250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:36.954289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:36.954296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:36.954355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:36.954411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:36.954470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:36.954529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:36.954575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:36.954582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:36.954640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:36.954696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:36.954753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:36.954812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:36.954850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:36.954859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:36.954917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:36.954968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:36.955022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:36.955079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:36.955123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:36.955133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:36.955194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:36.955256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:36.955319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:36.955381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:36.955424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:36.955431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:36.955493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:36.955559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:36.955626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:36.955687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:36.955730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:36.955737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:36.955785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821612:3994], now have 1 active actors on pipe
2025-10-09T21:52:36.955969Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:36.955980Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 4 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343257700821612:3994]
2025-10-09T21:52:36.956009Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:36.956011Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:36.956034Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2506 reason: error at write: Disconnected sessionId: src_id|5f714f91-ba2108c5-b60d9d8f-b1ad19d5_0
2025-10-09T21:52:36.956259Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2506 sessionId: src_id|5f714f91-ba2108c5-b60d9d8f-b1ad19d5_0 is DEAD
2025-10-09T21:52:36.956367Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f714f91-ba2108c5-b60d9d8f-b1ad19d5_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:36.956377Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f714f91-ba2108c5-b60d9d8f-b1ad19d5_0] Write session will restart in 0.009202s
2025-10-09T21:52:36.956402Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f714f91-ba2108c5-b60d9d8f-b1ad19d5_0] Write session: Do CDS request
2025-10-09T21:52:36.956407Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f714f91-ba2108c5-b60d9d8f-b1ad19d5_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:36.956794Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:36.964628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:36.966408Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5f714f91-ba2108c5-b60d9d8f-b1ad19d5_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:36.966404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:36.966498Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:36.966508Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2507
2025-10-09T21:52:36.966578Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:36.966590Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:36.966603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:36.966659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:36.966699Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343257338967591:8044]
2025-10-09T21:52:36.966745Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2507 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:36.966781Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2507 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:36.966791Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2507 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:36.966795Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:36.967108Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2507 sessionId:  describe result for acl check
2025-10-09T21:52:36.967131Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:36.967133Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:36.967135Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:36.967139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343257700821671:3881] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:36.967232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:36.967558Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343257700821671:3881] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:36.967598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:36.967643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:36.967941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:36.967987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.967993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.967994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.967995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.967996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.967997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.967998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.967999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.968000Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.968005Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.968014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.968018Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.968022Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.968025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.968029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.968033Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.968037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.968041Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.968048Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.968056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:36.968062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.968067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:36.968071Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.968075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:36.968079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.968084Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:36.968088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.968093Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:36.968097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.968107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:36.968112Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.968117Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.968121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.968126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.968134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.968139Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.968166Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:36.969527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343257700821671:3881] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:36.973739Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343257700821671:3881] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:36.973750Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343257700821671:3881] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:36.973752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343257700821671:3881] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:36.973758Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2507 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:36.974269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:36.975382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:36.975398Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:36.975401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:36.975404Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:36.975410Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2506 [2:7559343257338967591:8044]
2025-10-09T21:52:36.975423Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:36.975438Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:36.975441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:36.975444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:36.975446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:36.975461Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:36.975470Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:36.975476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:36.975512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.975517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.975518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.975520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:36.975577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:36.975583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.975591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:36.975600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:36.975614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:36.975639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821707:3881], now have 1 active actors on pipe
2025-10-09T21:52:36.975646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343257700821619:2432], now have 1 active actors on pipe
2025-10-09T21:52:36.975652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343257338967586:8044]
2025-10-09T21:52:36.975653Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343257338967586:8044]
2025-10-09T21:52:36.975655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343257338967586:8044]
2025-10-09T21:52:36.975666Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:36.975678Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:36.975686Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:36.975691Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:36.975696Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:36.975700Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:36.975703Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:36.975709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343257338967586:8044]
2025-10-09T21:52:36.975713Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:36.975708Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2506
2025-10-09T21:52:36.975713Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2506
2025-10-09T21:52:36.975733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:36.975805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.975813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.975840Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5ade9abf-b98b6a88-7dfbb745-9f116b2f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:36.975853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:36.975857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:36.975859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:36.975871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:36.975884Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:36.975900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:36.976064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:36.976074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:36.976090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:36.976201Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2507 partition: 0 MaxSeqNo: 25 sessionId: src_id|5ade9abf-b98b6a88-7dfbb745-9f116b2f_0
2025-10-09T21:52:36.976475Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5f714f91-ba2108c5-b60d9d8f-b1ad19d5_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5ade9abf-b98b6a88-7dfbb745-9f116b2f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:36.978688Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ade9abf-b98b6a88-7dfbb745-9f116b2f_0] Write session: try to update token
2025-10-09T21:52:36.978701Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ade9abf-b98b6a88-7dfbb745-9f116b2f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:36.989027Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:36.989049Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:36.989051Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:36.989054Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:36.989064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343257338967586:8044]
2025-10-09T21:52:36.989068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343257338967586:8044]
2025-10-09T21:52:36.989072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343257338967586:8044]
2025-10-09T21:52:36.989115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:36.989114Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:36.989119Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:36.989121Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:36.989133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:36.989136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:36.989279Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:36.990481Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2507 sessionId: src_id|5ade9abf-b98b6a88-7dfbb745-9f116b2f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:36.992716Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:37.056004Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343261995789011:3994]
2025-10-09T21:52:37.056276Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 5 pipe creation result: OK TabletId: 72075186224037892 Generation: 2506, pipe: [1:7559343261995789011:3994]
2025-10-09T21:52:37.139424Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:37.139734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.139748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.139790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343261995789011:3994], now have 1 active actors on pipe
2025-10-09T21:52:37.139997Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:37.140033Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 5 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343261995789011:3994]
2025-10-09T21:52:37.140048Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:37.140067Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:37.140157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2507 reason: status is not ok: tablet will be restarted right now sessionId: src_id|5ade9abf-b98b6a88-7dfbb745-9f116b2f_0
2025-10-09T21:52:37.140342Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2507 sessionId: src_id|5ade9abf-b98b6a88-7dfbb745-9f116b2f_0 is DEAD
2025-10-09T21:52:37.140518Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ade9abf-b98b6a88-7dfbb745-9f116b2f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:37.140528Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ade9abf-b98b6a88-7dfbb745-9f116b2f_0] Write session will restart in 0.007988s
2025-10-09T21:52:37.140553Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ade9abf-b98b6a88-7dfbb745-9f116b2f_0] Write session: Do CDS request
2025-10-09T21:52:37.140557Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ade9abf-b98b6a88-7dfbb745-9f116b2f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:37.149429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:37.149532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5ade9abf-b98b6a88-7dfbb745-9f116b2f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:37.149639Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:37.149655Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2508
2025-10-09T21:52:37.149763Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2508 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:37.149788Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2508 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:37.149796Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2508 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:37.149798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:37.150191Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2508 sessionId:  describe result for acl check
2025-10-09T21:52:37.150225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:37.150231Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:37.150235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:37.150239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343261995789065:3875] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:37.150745Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343261995789065:3875] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:37.151170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:37.151353Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:37.151367Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:37.151378Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:37.151432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:37.151473Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343261633934976:8045]
2025-10-09T21:52:37.152054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:37.152383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:37.152430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:37.152633Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343261995789065:3875] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:37.152725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:37.152790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.152796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.152797Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.152798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.152799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.152800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.152801Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.152802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.152803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.152808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.152819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.152822Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.152826Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.152830Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.152834Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.152838Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.152842Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.152846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.152852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.152859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.152865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.152870Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.152876Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.152881Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.152885Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.152891Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.152896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.152903Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.152909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.152914Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.152919Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.152924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.152929Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.152934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.152944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.152949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.152979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:37.157836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343261995789065:3875] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:37.157849Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343261995789065:3875] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:37.157853Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343261995789065:3875] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:37.157860Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2508 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:37.159220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:37.160289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:37.160305Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:37.160307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:37.160310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:37.160316Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2507 [2:7559343261633934976:8045]
2025-10-09T21:52:37.160328Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:37.160344Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:37.160350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:37.160353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:37.160354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:37.160368Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:37.160376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:37.160382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:37.160434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.160439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.160440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.160443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:37.160605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:37.160615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.160633Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2507
2025-10-09T21:52:37.160640Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2507
2025-10-09T21:52:37.160627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.160644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.160662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:37.160683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343261995789104:3875], now have 1 active actors on pipe
2025-10-09T21:52:37.160687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343261995789013:2432], now have 1 active actors on pipe
2025-10-09T21:52:37.160702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.160710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.160729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343261633934971:8045]
2025-10-09T21:52:37.160735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343261633934971:8045]
2025-10-09T21:52:37.160737Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343261633934971:8045]
2025-10-09T21:52:37.160745Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:37.160758Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:37.160759Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:37.160762Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:37.160763Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:37.160768Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:37.160771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:37.160786Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|834b33f-1b821a81-771e5bd3-cc21192a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:37.160794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:37.160797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:37.160799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:37.160807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:37.160815Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:37.160848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343261633934971:8045]
2025-10-09T21:52:37.160857Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:37.160866Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:37.160866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:37.160997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.161011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.161036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:37.161110Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2508 partition: 0 MaxSeqNo: 25 sessionId: src_id|834b33f-1b821a81-771e5bd3-cc21192a_0
2025-10-09T21:52:37.161447Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5ade9abf-b98b6a88-7dfbb745-9f116b2f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|834b33f-1b821a81-771e5bd3-cc21192a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:37.164023Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|834b33f-1b821a81-771e5bd3-cc21192a_0] Write session: try to update token
2025-10-09T21:52:37.164042Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|834b33f-1b821a81-771e5bd3-cc21192a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:37.175207Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:37.175229Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:37.175231Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:37.175233Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:37.175243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343261633934971:8045]
2025-10-09T21:52:37.175248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343261633934971:8045]
2025-10-09T21:52:37.175252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343261633934971:8045]
2025-10-09T21:52:37.175295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:37.175295Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:37.175301Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:37.175302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.175304Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:37.175305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:37.175446Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:37.177887Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2508 sessionId: src_id|834b33f-1b821a81-771e5bd3-cc21192a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:37.180596Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:37.240318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343261995789114:3994]
2025-10-09T21:52:37.240586Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 6 pipe creation result: OK TabletId: 72075186224037892 Generation: 2507, pipe: [1:7559343261995789114:3994]
2025-10-09T21:52:37.322278Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:37.322465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.322477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.322509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343261995789114:3994], now have 1 active actors on pipe
2025-10-09T21:52:37.322614Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:37.322642Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 6 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343261995789114:3994]
2025-10-09T21:52:37.322682Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:37.322685Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:37.322736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2508 reason: error at write: Disconnected sessionId: src_id|834b33f-1b821a81-771e5bd3-cc21192a_0
2025-10-09T21:52:37.322923Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2508 sessionId: src_id|834b33f-1b821a81-771e5bd3-cc21192a_0 is DEAD
2025-10-09T21:52:37.323072Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|834b33f-1b821a81-771e5bd3-cc21192a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:37.323070Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:37.323084Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|834b33f-1b821a81-771e5bd3-cc21192a_0] Write session will restart in 0.008769s
2025-10-09T21:52:37.323110Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|834b33f-1b821a81-771e5bd3-cc21192a_0] Write session: Do CDS request
2025-10-09T21:52:37.323130Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|834b33f-1b821a81-771e5bd3-cc21192a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:37.330772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:37.332488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:37.332663Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:37.332674Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:37.332685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:37.332744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:37.332783Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343261633935104:8029]
2025-10-09T21:52:37.333165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:37.333450Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|834b33f-1b821a81-771e5bd3-cc21192a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:37.333443Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:37.333484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:37.333495Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:37.333508Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2509
2025-10-09T21:52:37.333676Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2509 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:37.333708Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2509 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:37.333718Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2509 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:37.333722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:37.333704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:37.333749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.333756Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.333757Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.333758Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.333760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.333761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.333762Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.333763Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.333764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.333768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.333778Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.333785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.333789Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.333794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.333798Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.333802Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.333806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.333810Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.333816Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.333824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.333831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.333837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.333841Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.333847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.333852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.333862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.333867Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.333872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.333877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.333882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.333892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.333897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.333902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.333907Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.333912Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.333921Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.333951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:37.334095Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2509 sessionId:  describe result for acl check
2025-10-09T21:52:37.334130Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:37.334138Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:37.334140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:37.334145Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343261995789204:3835] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:37.334600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343261995789204:3835] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:37.336228Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343261995789204:3835] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:37.339536Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343261995789204:3835] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:37.339549Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343261995789204:3835] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:37.339553Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343261995789204:3835] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:37.339560Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2509 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:37.340014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:37.341351Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:37.341366Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:37.341368Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:37.341373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:37.341379Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2508 [2:7559343261633935104:8029]
2025-10-09T21:52:37.341391Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:37.341406Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:37.341409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:37.341412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:37.341415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:37.341431Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:37.341442Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:37.341449Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:37.341505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.341514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.341516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.341519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:37.341586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:37.341594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.341607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.341620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.341642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:37.341667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343261995789143:2432], now have 1 active actors on pipe
2025-10-09T21:52:37.341675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343261995789238:3835], now have 1 active actors on pipe
2025-10-09T21:52:37.341681Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343261633935099:8029]
2025-10-09T21:52:37.341683Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343261633935099:8029]
2025-10-09T21:52:37.341685Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343261633935099:8029]
2025-10-09T21:52:37.341671Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2508
2025-10-09T21:52:37.341696Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:37.341713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:37.341715Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:37.341715Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:37.341717Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:37.341729Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:37.341732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:37.341752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.341761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.341791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343261633935099:8029]
2025-10-09T21:52:37.341803Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:37.341805Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2508
2025-10-09T21:52:37.341817Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:37.341820Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|90970692-a09d1ce1-3e5709ee-49094dfb_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:37.341827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:37.341831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:37.341833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:37.341843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:37.341853Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:37.341890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:37.341987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.341995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.342007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:37.342084Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2509 partition: 0 MaxSeqNo: 25 sessionId: src_id|90970692-a09d1ce1-3e5709ee-49094dfb_0
2025-10-09T21:52:37.342363Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|834b33f-1b821a81-771e5bd3-cc21192a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|90970692-a09d1ce1-3e5709ee-49094dfb_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:37.344759Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90970692-a09d1ce1-3e5709ee-49094dfb_0] Write session: try to update token
2025-10-09T21:52:37.344772Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90970692-a09d1ce1-3e5709ee-49094dfb_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:37.355783Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:37.355804Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:37.355807Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:37.355812Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:37.355823Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343261633935099:8029]
2025-10-09T21:52:37.355828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343261633935099:8029]
2025-10-09T21:52:37.355833Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343261633935099:8029]
2025-10-09T21:52:37.355879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:37.355883Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:37.355889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.355889Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:37.355891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:37.355894Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:37.356028Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:37.356945Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2509 sessionId: src_id|90970692-a09d1ce1-3e5709ee-49094dfb_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:37.359240Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:37.422564Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343261995789246:3994]
2025-10-09T21:52:37.422834Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 7 pipe creation result: OK TabletId: 72075186224037892 Generation: 2508, pipe: [1:7559343261995789246:3994]
2025-10-09T21:52:37.505138Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:37.505389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.505401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.505440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343261995789246:3994], now have 1 active actors on pipe
2025-10-09T21:52:37.505622Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:37.505666Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:37.505680Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:37.505689Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 7 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343261995789246:3994]
2025-10-09T21:52:37.505706Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:37.505742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2509 reason: error at write: Disconnected sessionId: src_id|90970692-a09d1ce1-3e5709ee-49094dfb_0
2025-10-09T21:52:37.505909Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2509 sessionId: src_id|90970692-a09d1ce1-3e5709ee-49094dfb_0 is DEAD
2025-10-09T21:52:37.506069Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90970692-a09d1ce1-3e5709ee-49094dfb_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:37.506082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90970692-a09d1ce1-3e5709ee-49094dfb_0] Write session will restart in 0.006511s
2025-10-09T21:52:37.506111Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90970692-a09d1ce1-3e5709ee-49094dfb_0] Write session: Do CDS request
2025-10-09T21:52:37.506117Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90970692-a09d1ce1-3e5709ee-49094dfb_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:37.513458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|90970692-a09d1ce1-3e5709ee-49094dfb_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:37.513499Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:37.513513Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2510
2025-10-09T21:52:37.513587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:37.513659Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2510 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:37.513695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2510 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:37.513710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2510 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:37.513713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:37.514162Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2510 sessionId:  describe result for acl check
2025-10-09T21:52:37.514201Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:37.514209Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:37.514211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:37.514216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343261995789303:3823] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:37.514688Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343261995789303:3823] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:37.514988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:37.515152Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:37.515161Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:37.515172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:37.515224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:37.515261Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343261633935195:8015]
2025-10-09T21:52:37.515643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:37.515865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:37.515898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:37.516116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:37.516153Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.516157Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.516158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.516158Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.516159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.516159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.516160Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.516161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.516161Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.516164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.516170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.516172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.516174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.516176Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.516178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.516180Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.516182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.516184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.516188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.516193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.516197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.516200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.516203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.516207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.516209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.516212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.516214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.516217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.516219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.516222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.516226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.516228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.516231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.516233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.516236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.516238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.516253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:37.516459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343261995789303:3823] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:37.521748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343261995789303:3823] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:37.521760Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343261995789303:3823] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:37.521764Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343261995789303:3823] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:37.521771Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2510 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:37.522113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:37.523148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:37.523161Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:37.523163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:37.523165Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:37.523170Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2509 [2:7559343261633935195:8015]
2025-10-09T21:52:37.523179Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:37.523192Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:37.523194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:37.523197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:37.523199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:37.523211Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:37.523219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:37.523225Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:37.523264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.523270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.523271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.523273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:37.523323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:37.523328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.523338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.523352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.523370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:37.523390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343261995789342:3823], now have 1 active actors on pipe
2025-10-09T21:52:37.523396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343261995789248:2432], now have 1 active actors on pipe
2025-10-09T21:52:37.523401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343261633935190:8015]
2025-10-09T21:52:37.523404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343261633935190:8015]
2025-10-09T21:52:37.523405Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343261633935190:8015]
2025-10-09T21:52:37.523413Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:37.523427Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:37.523438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:37.523438Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:37.523440Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:37.523441Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:37.523443Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:37.523461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343261633935190:8015]
2025-10-09T21:52:37.523467Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:37.523461Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2509
2025-10-09T21:52:37.523485Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:37.523473Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2509
2025-10-09T21:52:37.523495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.523498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.523520Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|5c23a0bb-464bbfb7-491413ab-144b779a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:37.523530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:37.523533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:37.523535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:37.523541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:37.523551Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:37.523567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:37.523678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.523682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.523691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:37.523803Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2510 partition: 0 MaxSeqNo: 25 sessionId: src_id|5c23a0bb-464bbfb7-491413ab-144b779a_0
2025-10-09T21:52:37.524085Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|90970692-a09d1ce1-3e5709ee-49094dfb_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|5c23a0bb-464bbfb7-491413ab-144b779a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:37.526442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c23a0bb-464bbfb7-491413ab-144b779a_0] Write session: try to update token
2025-10-09T21:52:37.526458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c23a0bb-464bbfb7-491413ab-144b779a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:37.534731Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:37.534750Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:37.534758Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343261633935190:8015]
2025-10-09T21:52:37.534806Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:37.534812Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:37.535017Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:37.535130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:37.535248Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:37.535373Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:37.535555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:37.535597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:37.539789Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2510 sessionId: src_id|5c23a0bb-464bbfb7-491413ab-144b779a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:37.542005Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:37.542203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.542233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.542270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:37.542357Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:37.546479Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:37.546495Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:37.546498Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:37.546500Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:37.546512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343261633935190:8015]
2025-10-09T21:52:37.546516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343261633935190:8015]
2025-10-09T21:52:37.546520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343261633935190:8015]
2025-10-09T21:52:37.546551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:37.546559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.546561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:37.546564Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:37.546569Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:37.546582Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:37.546674Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:37.605894Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343261995789349:3994]
2025-10-09T21:52:37.606176Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 8 pipe creation result: OK TabletId: 72075186224037892 Generation: 2509, pipe: [1:7559343261995789349:3994]
2025-10-09T21:52:37.686232Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:37.692733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.692757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.692895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.692947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.693000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.693054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.693094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.693107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.693179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.693230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.693303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.693357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.693397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.693404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.693466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.693528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.693595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.693653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.693695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.693703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.693767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.693821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.693876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.693929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.693969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.693975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.694049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.694108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.694166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.694229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.694269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.694275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.694333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.694382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.694436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.694485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.694522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.694527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.694580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.694633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.694696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.694755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.694792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.694799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.694853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.694910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.694962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.695010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.695049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.695054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.695103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.695149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.695199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.695251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.695300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.695309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.695360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.695412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.695462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.695516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.695552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.695559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.695606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343261995789349:3994], now have 1 active actors on pipe
2025-10-09T21:52:37.695814Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:37.695815Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:37.695820Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:37.695852Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2510 reason: error at write: Disconnected sessionId: src_id|5c23a0bb-464bbfb7-491413ab-144b779a_0
2025-10-09T21:52:37.695878Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 8 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343261995789349:3994]
2025-10-09T21:52:37.696027Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2510 sessionId: src_id|5c23a0bb-464bbfb7-491413ab-144b779a_0 is DEAD
2025-10-09T21:52:37.696157Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c23a0bb-464bbfb7-491413ab-144b779a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:37.696168Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c23a0bb-464bbfb7-491413ab-144b779a_0] Write session will restart in 0.009336s
2025-10-09T21:52:37.696186Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c23a0bb-464bbfb7-491413ab-144b779a_0] Write session: Do CDS request
2025-10-09T21:52:37.696190Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c23a0bb-464bbfb7-491413ab-144b779a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:37.696406Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:37.704621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:37.706288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:37.706477Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:37.706487Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:37.706500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:37.706546Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|5c23a0bb-464bbfb7-491413ab-144b779a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:37.706543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:37.706576Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343261633935288:8006]
2025-10-09T21:52:37.706664Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:37.706676Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2511
2025-10-09T21:52:37.706807Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2511 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:37.706844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2511 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:37.706856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2511 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:37.706860Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:37.706979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:37.707212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:37.707248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:37.707249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2511 sessionId:  describe result for acl check
2025-10-09T21:52:37.707280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:37.707287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:37.707289Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:37.707294Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343261995789410:3809] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:37.707516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:37.707569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.707576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.707578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.707579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.707580Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.707581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.707582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.707583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.707584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.707589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.707599Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.707604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.707608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.707611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.707615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.707619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.707623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.707627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.707634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.707642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.707646Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.707650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.707654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.707656Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.707659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.707662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.707664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.707667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.707669Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.707673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.707675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.707678Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.707681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.707684Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.707686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.707689Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.707706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:37.707851Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343261995789410:3809] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:37.710272Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343261995789410:3809] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:37.713348Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343261995789410:3809] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:37.713362Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343261995789410:3809] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:37.713366Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343261995789410:3809] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:37.713375Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2511 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:37.713813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:37.714800Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:37.714817Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:37.714819Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:37.714823Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:37.714830Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2510 [2:7559343261633935288:8006]
2025-10-09T21:52:37.714845Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:37.714860Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:37.714861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:37.714864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:37.714865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:37.714877Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:37.714887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:37.714893Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:37.714933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.714938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.714939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.714941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:37.714991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:37.714996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.715002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.715009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.715018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:37.715035Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343261633935283:8006]
2025-10-09T21:52:37.715040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343261633935283:8006]
2025-10-09T21:52:37.715041Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343261633935283:8006]
2025-10-09T21:52:37.715047Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:37.715059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:37.715059Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:37.715062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:37.715064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:37.715067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:37.715071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:37.715079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343261995789446:3809], now have 1 active actors on pipe
2025-10-09T21:52:37.715082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343261995789352:2432], now have 1 active actors on pipe
2025-10-09T21:52:37.715101Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343261633935283:8006]
2025-10-09T21:52:37.715109Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:37.715125Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:37.715132Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2510
2025-10-09T21:52:37.715141Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2510
2025-10-09T21:52:37.715219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.715226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.715248Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8e1bd142-977146f6-8728879f-cd70da9c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:37.715257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:37.715260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:37.715261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:37.715268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:37.715276Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:37.715287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:37.715438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.715448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.715474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:37.715565Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2511 partition: 0 MaxSeqNo: 25 sessionId: src_id|8e1bd142-977146f6-8728879f-cd70da9c_0
2025-10-09T21:52:37.715897Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|5c23a0bb-464bbfb7-491413ab-144b779a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8e1bd142-977146f6-8728879f-cd70da9c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:37.718473Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e1bd142-977146f6-8728879f-cd70da9c_0] Write session: try to update token
2025-10-09T21:52:37.718494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e1bd142-977146f6-8728879f-cd70da9c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:37.726363Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:37.726384Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:37.726395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343261633935283:8006]
2025-10-09T21:52:37.726440Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:37.726443Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:37.726591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:37.726743Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:37.726868Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:37.727008Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:37.727183Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:37.727219Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:37.731321Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2511 sessionId: src_id|8e1bd142-977146f6-8728879f-cd70da9c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:37.733563Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:37.733737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.733758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.733798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:37.733876Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:37.738975Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:37.738992Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:37.738994Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:37.738996Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:37.739006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343261633935283:8006]
2025-10-09T21:52:37.739010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343261633935283:8006]
2025-10-09T21:52:37.739014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343261633935283:8006]
2025-10-09T21:52:37.739048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:37.739047Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:37.739051Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:37.739054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.739054Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:37.739055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:37.739173Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:37.795847Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343261995789453:3994]
2025-10-09T21:52:37.796287Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 9 pipe creation result: OK TabletId: 72075186224037892 Generation: 2510, pipe: [1:7559343261995789453:3994]
2025-10-09T21:52:37.884906Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:37.891592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.891618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.891768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.891824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.891879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.891933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.891972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.891980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.892037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.892099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.892163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.892220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.892260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.892270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.892326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.892378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.892439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.892494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.892533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.892538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.892595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.892661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.892717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.892771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.892810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.892815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.892873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.892925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.892977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.893038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.893085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.893091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.893143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.893195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.893258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.893319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.893361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.893367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.893420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.893469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.893521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.893576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.893614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.893619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.893672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.893718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.893771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.893819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.893856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.893861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.893910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.893957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.894006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.894072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.894109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.894114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.894168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:37.894217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:37.894274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:37.894338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:37.894377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:37.894382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:37.894420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343261995789453:3994], now have 1 active actors on pipe
2025-10-09T21:52:37.894633Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:37.894689Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 9 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343261995789453:3994]
2025-10-09T21:52:37.894806Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:37.894816Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:37.894844Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2511 reason: error at write: Disconnected sessionId: src_id|8e1bd142-977146f6-8728879f-cd70da9c_0
2025-10-09T21:52:37.895027Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2511 sessionId: src_id|8e1bd142-977146f6-8728879f-cd70da9c_0 is DEAD
2025-10-09T21:52:37.895181Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e1bd142-977146f6-8728879f-cd70da9c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:37.895191Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e1bd142-977146f6-8728879f-cd70da9c_0] Write session will restart in 0.009050s
2025-10-09T21:52:37.895167Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:37.895213Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e1bd142-977146f6-8728879f-cd70da9c_0] Write session: Do CDS request
2025-10-09T21:52:37.895219Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e1bd142-977146f6-8728879f-cd70da9c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:37.903562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:37.905291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:37.905480Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:37.905498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8e1bd142-977146f6-8728879f-cd70da9c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:37.905491Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:37.905505Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:37.905560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:37.905572Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:37.905585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2512
2025-10-09T21:52:37.905618Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343261633935384:8002]
2025-10-09T21:52:37.905728Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2512 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:37.905758Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2512 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:37.905768Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2512 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:37.905772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:37.906027Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:37.906196Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2512 sessionId:  describe result for acl check
2025-10-09T21:52:37.906230Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:37.906236Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:37.906237Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:37.906242Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343261995789514:3795] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:37.906325Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:37.906405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:37.906673Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:37.906719Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.906724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.906725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.906725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.906726Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.906727Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.906728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.906728Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.906729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.906733Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.906743Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343261995789514:3795] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:37.906739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.906741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.906744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.906746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.906748Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.906750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.906752Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.906754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.906759Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.906764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:37.906768Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.906772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:37.906776Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.906780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:37.906782Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.906785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:37.906788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.906791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:37.906793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.906796Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:37.906799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.906803Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.906805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.906808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.906812Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.906815Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.906831Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:37.909418Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343261995789514:3795] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:37.912329Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343261995789514:3795] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:37.912340Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343261995789514:3795] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:37.912343Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343261995789514:3795] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:37.912351Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2512 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:37.912808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:37.913878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:37.913893Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:37.913894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:37.913897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:37.913902Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2511 [2:7559343261633935384:8002]
2025-10-09T21:52:37.913911Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:37.913924Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:37.913926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:37.913928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:37.913929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:37.913942Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:37.913946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:37.913951Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:37.913997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.914005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.914007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.914010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:37.914092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:37.914100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.914111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:37.914121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:37.914145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:37.914171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343261995789456:2432], now have 1 active actors on pipe
2025-10-09T21:52:37.914179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343261995789551:3795], now have 1 active actors on pipe
2025-10-09T21:52:37.914186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343261633935379:8002]
2025-10-09T21:52:37.914189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343261633935379:8002]
2025-10-09T21:52:37.914191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343261633935379:8002]
2025-10-09T21:52:37.914203Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:37.914203Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2511
2025-10-09T21:52:37.914205Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2511
2025-10-09T21:52:37.914215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:37.914217Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:37.914225Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:37.914227Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:37.914231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:37.914235Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:37.914286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.914295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.914308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343261633935379:8002]
2025-10-09T21:52:37.914322Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:37.914337Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:37.914358Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|745b64d8-e4f5305a-c3bf7fdf-30d7d216_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:37.914370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:37.914374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:37.914379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:37.914386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:37.914398Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:37.914414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:37.914567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:37.914578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:37.914592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:37.914673Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2512 partition: 0 MaxSeqNo: 25 sessionId: src_id|745b64d8-e4f5305a-c3bf7fdf-30d7d216_0
2025-10-09T21:52:37.914988Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8e1bd142-977146f6-8728879f-cd70da9c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|745b64d8-e4f5305a-c3bf7fdf-30d7d216_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:37.917362Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|745b64d8-e4f5305a-c3bf7fdf-30d7d216_0] Write session: try to update token
2025-10-09T21:52:37.917380Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|745b64d8-e4f5305a-c3bf7fdf-30d7d216_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:37.928739Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:37.928759Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:37.928762Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:37.928764Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:37.928773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343261633935379:8002]
2025-10-09T21:52:37.928781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343261633935379:8002]
2025-10-09T21:52:37.928784Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343261633935379:8002]
2025-10-09T21:52:37.928818Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:37.928823Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:37.928825Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:37.928827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:37.928834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:37.928837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:37.928969Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:37.929875Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2512 sessionId: src_id|745b64d8-e4f5305a-c3bf7fdf-30d7d216_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:37.932125Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:37.995009Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343261995789560:3994]
2025-10-09T21:52:37.995270Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 10 pipe creation result: OK TabletId: 72075186224037892 Generation: 2511, pipe: [1:7559343261995789560:3994]
2025-10-09T21:52:38.074372Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:38.074674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.074687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.074723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343261995789560:3994], now have 1 active actors on pipe
2025-10-09T21:52:38.074989Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:38.075006Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 10 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343261995789560:3994]
2025-10-09T21:52:38.075030Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:38.075032Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:38.075091Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2512 reason: error at write: Disconnected sessionId: src_id|745b64d8-e4f5305a-c3bf7fdf-30d7d216_0
2025-10-09T21:52:38.075136Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:38.075263Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2512 sessionId: src_id|745b64d8-e4f5305a-c3bf7fdf-30d7d216_0 is DEAD
2025-10-09T21:52:38.075446Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|745b64d8-e4f5305a-c3bf7fdf-30d7d216_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:38.075459Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|745b64d8-e4f5305a-c3bf7fdf-30d7d216_0] Write session will restart in 0.009725s
2025-10-09T21:52:38.075487Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|745b64d8-e4f5305a-c3bf7fdf-30d7d216_0] Write session: Do CDS request
2025-10-09T21:52:38.075493Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|745b64d8-e4f5305a-c3bf7fdf-30d7d216_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:38.083914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:38.085448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:38.085668Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:38.085680Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:38.085694Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:38.085764Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:38.085803Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343265928902775:7988]
2025-10-09T21:52:38.086407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:38.086507Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|745b64d8-e4f5305a-c3bf7fdf-30d7d216_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:38.086618Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:38.086633Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2513
2025-10-09T21:52:38.086699Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:38.086741Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2513 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:38.086747Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:38.086772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2513 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:38.086784Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2513 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:38.086789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:38.087040Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:38.087097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.087103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.087104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.087105Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.087106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.087108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.087109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.087110Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.087111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.087115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.087126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.087130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.087134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.087138Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.087142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.087145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.087151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.087155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.087163Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2513 sessionId:  describe result for acl check
2025-10-09T21:52:38.087163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.087169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.087174Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.087179Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.087183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.087187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.087191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.087196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.087201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.087205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.087210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.087214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.087196Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:38.087219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.087223Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.087202Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:38.087227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.087205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:38.087210Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343266290756919:3788] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:38.087231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.087236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.087240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.087265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:38.087666Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343266290756919:3788] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:38.089567Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343266290756919:3788] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:38.092865Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343266290756919:3788] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:38.092877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343266290756919:3788] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:38.092880Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343266290756919:3788] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:38.092887Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2513 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:38.093431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:38.094591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:38.094607Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:38.094609Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:38.094613Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:38.094623Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2512 [2:7559343265928902775:7988]
2025-10-09T21:52:38.094640Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:38.094663Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:38.094667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:38.094670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:38.094675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:38.094691Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:38.094697Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:38.094704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:38.094764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.094771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.094773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.094776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:38.094847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:38.094855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.094866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.094885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.094907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:38.094934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343266290756955:3788], now have 1 active actors on pipe
2025-10-09T21:52:38.094942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343266290756859:2432], now have 1 active actors on pipe
2025-10-09T21:52:38.094950Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343265928902770:7988]
2025-10-09T21:52:38.094928Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2512
2025-10-09T21:52:38.094956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343265928902770:7988]
2025-10-09T21:52:38.094958Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343265928902770:7988]
2025-10-09T21:52:38.094970Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:38.094933Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2512
2025-10-09T21:52:38.094985Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:38.094986Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:38.094990Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:38.094995Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:38.095000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:38.095003Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:38.095020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.095024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.095064Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343265928902770:7988]
2025-10-09T21:52:38.095081Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:38.095103Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|36860e2-fbeb7e7f-39549ddb-65f947ad_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:38.095109Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:38.095116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:38.095120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:38.095123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:38.095132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:38.095146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:38.095188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:38.095342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.095349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.095363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:38.095452Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2513 partition: 0 MaxSeqNo: 25 sessionId: src_id|36860e2-fbeb7e7f-39549ddb-65f947ad_0
2025-10-09T21:52:38.095762Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|745b64d8-e4f5305a-c3bf7fdf-30d7d216_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|36860e2-fbeb7e7f-39549ddb-65f947ad_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:38.098170Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36860e2-fbeb7e7f-39549ddb-65f947ad_0] Write session: try to update token
2025-10-09T21:52:38.098185Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36860e2-fbeb7e7f-39549ddb-65f947ad_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:38.109262Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:38.109282Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:38.109284Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:38.109286Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:38.109296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343265928902770:7988]
2025-10-09T21:52:38.109301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343265928902770:7988]
2025-10-09T21:52:38.109305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343265928902770:7988]
2025-10-09T21:52:38.109347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:38.109351Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:38.109354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.109359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:38.109366Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:38.109373Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:38.109492Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:38.110993Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2513 sessionId: src_id|36860e2-fbeb7e7f-39549ddb-65f947ad_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:38.113282Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:38.175218Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343266290756964:3994]
2025-10-09T21:52:38.175465Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 11 pipe creation result: OK TabletId: 72075186224037892 Generation: 2512, pipe: [1:7559343266290756964:3994]
2025-10-09T21:52:38.260278Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:38.260535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.260547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.260583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343266290756964:3994], now have 1 active actors on pipe
2025-10-09T21:52:38.260820Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 11 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343266290756964:3994]
2025-10-09T21:52:38.260845Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:38.260849Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:38.260851Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:38.260912Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2513 reason: error at write: Disconnected sessionId: src_id|36860e2-fbeb7e7f-39549ddb-65f947ad_0
2025-10-09T21:52:38.261082Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:38.261106Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2513 sessionId: src_id|36860e2-fbeb7e7f-39549ddb-65f947ad_0 is DEAD
2025-10-09T21:52:38.261281Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36860e2-fbeb7e7f-39549ddb-65f947ad_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:38.261293Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36860e2-fbeb7e7f-39549ddb-65f947ad_0] Write session will restart in 0.006454s
2025-10-09T21:52:38.261321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36860e2-fbeb7e7f-39549ddb-65f947ad_0] Write session: Do CDS request
2025-10-09T21:52:38.261327Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36860e2-fbeb7e7f-39549ddb-65f947ad_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:38.268477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|36860e2-fbeb7e7f-39549ddb-65f947ad_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:38.268496Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:38.268507Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2514
2025-10-09T21:52:38.268677Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2514 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:38.268702Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2514 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:38.268709Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2514 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:38.268712Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:38.269067Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2514 sessionId:  describe result for acl check
2025-10-09T21:52:38.269097Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:38.269099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:38.269101Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:38.269106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343266290757027:3770] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:38.269387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:38.269577Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343266290757027:3770] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:38.270872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:38.271034Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:38.271044Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:38.271054Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:38.271113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:38.271155Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343265928902869:7978]
2025-10-09T21:52:38.271197Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343266290757027:3770] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:38.271632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:38.271909Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:38.271948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:38.272242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:38.272301Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.272307Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.272308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.272310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.272311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.272312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.272313Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.272314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.272315Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.272320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.272329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.272333Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.272336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.272339Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.272342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.272345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.272349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.272352Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.272359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.272366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.272372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.272377Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.272383Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.272388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.272393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.272398Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.272403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.272408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.272413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.272421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.272427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.272434Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.272438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.272447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.272456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.272461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.272492Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:38.277229Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343266290757027:3770] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:38.277255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343266290757027:3770] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:38.277258Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343266290757027:3770] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:38.277265Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2514 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:38.278619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:38.279654Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:38.279670Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:38.279671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:38.279674Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:38.279679Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2513 [2:7559343265928902869:7978]
2025-10-09T21:52:38.279689Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:38.279702Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:38.279704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:38.279708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:38.279710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:38.279727Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:38.279734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:38.279741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:38.279803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.279813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.279816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.279819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:38.279871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:38.279877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.279885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.279891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.279905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:38.279927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343266290756970:2432], now have 1 active actors on pipe
2025-10-09T21:52:38.279942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343265928902864:7978]
2025-10-09T21:52:38.279944Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343265928902864:7978]
2025-10-09T21:52:38.279946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343265928902864:7978]
2025-10-09T21:52:38.279982Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:38.279993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343266290757067:3770], now have 1 active actors on pipe
2025-10-09T21:52:38.280006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:38.280009Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:38.280012Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:38.280014Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:38.279997Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2513
2025-10-09T21:52:38.280012Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2513
2025-10-09T21:52:38.280026Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:38.280029Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:38.280059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343265928902864:7978]
2025-10-09T21:52:38.280063Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:38.280064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:38.280078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.280087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.280144Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|3e285458-25d87bfb-6f8aee56-244b9537_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:38.280156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:38.280160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:38.280163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:38.280175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:38.280185Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:38.280213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:38.280379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.280387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.280401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:38.280474Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2514 partition: 0 MaxSeqNo: 25 sessionId: src_id|3e285458-25d87bfb-6f8aee56-244b9537_0
2025-10-09T21:52:38.280770Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|36860e2-fbeb7e7f-39549ddb-65f947ad_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|3e285458-25d87bfb-6f8aee56-244b9537_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:38.283101Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3e285458-25d87bfb-6f8aee56-244b9537_0] Write session: try to update token
2025-10-09T21:52:38.283116Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3e285458-25d87bfb-6f8aee56-244b9537_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:38.294160Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:38.294179Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:38.294182Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:38.294184Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:38.294193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343265928902864:7978]
2025-10-09T21:52:38.294197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343265928902864:7978]
2025-10-09T21:52:38.294201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343265928902864:7978]
2025-10-09T21:52:38.294243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:38.294242Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:38.294248Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:38.294250Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:38.294250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.294253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:38.294398Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:38.295488Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2514 sessionId: src_id|3e285458-25d87bfb-6f8aee56-244b9537_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:38.297730Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:38.361070Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343266290757093:3994]
2025-10-09T21:52:38.361314Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 12 pipe creation result: OK TabletId: 72075186224037892 Generation: 2513, pipe: [1:7559343266290757093:3994]
2025-10-09T21:52:38.440711Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:38.440937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.440951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.440987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343266290757093:3994], now have 1 active actors on pipe
2025-10-09T21:52:38.441160Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:38.441182Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:38.441185Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 12 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343266290757093:3994]
2025-10-09T21:52:38.441212Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:38.441228Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:38.441305Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2514 reason: error at write: Disconnected sessionId: src_id|3e285458-25d87bfb-6f8aee56-244b9537_0
2025-10-09T21:52:38.441468Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2514 sessionId: src_id|3e285458-25d87bfb-6f8aee56-244b9537_0 is DEAD
2025-10-09T21:52:38.441597Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e285458-25d87bfb-6f8aee56-244b9537_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:38.441608Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e285458-25d87bfb-6f8aee56-244b9537_0] Write session will restart in 0.009339s
2025-10-09T21:52:38.441625Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e285458-25d87bfb-6f8aee56-244b9537_0] Write session: Do CDS request
2025-10-09T21:52:38.441630Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e285458-25d87bfb-6f8aee56-244b9537_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:38.449234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:38.450901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:38.451079Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:38.451089Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:38.451100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:38.451155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:38.451196Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343265928903004:7953]
2025-10-09T21:52:38.451754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:38.452056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:38.452099Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:38.452412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:38.452466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.452473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.452474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.452475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.452476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.452477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.452478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.452479Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.452480Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.452484Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.452494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.452497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.452500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.452503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.452507Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.452510Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.452513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.452547Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|3e285458-25d87bfb-6f8aee56-244b9537_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:38.452516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.452525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.452532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.452538Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.452543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.452547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.452551Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.452555Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.452560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.452564Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.452569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.452573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.452578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.452584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.452588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.452592Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.452597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.452601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.452606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.452590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:38.452613Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2515
2025-10-09T21:52:38.452634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:38.452821Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2515 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:38.452864Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2515 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:38.452874Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2515 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:38.452878Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:38.453290Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2515 sessionId:  describe result for acl check
2025-10-09T21:52:38.453330Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:38.453337Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:38.453339Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:38.453345Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343266290757168:3730] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:38.453799Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343266290757168:3730] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:38.455441Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343266290757168:3730] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:38.458576Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343266290757168:3730] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:38.458587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343266290757168:3730] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:38.458589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343266290757168:3730] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:38.458595Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2515 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:38.458917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:38.459915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:38.459931Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:38.459933Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:38.459936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:38.459940Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2514 [2:7559343265928903004:7953]
2025-10-09T21:52:38.459949Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:38.459962Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:38.459968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:38.459970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:38.459971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:38.459984Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:38.459990Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:38.459996Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:38.460038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.460043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.460046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.460048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:38.460162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:38.460167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.460176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.460182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.460192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:38.460210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343266290757202:3730], now have 1 active actors on pipe
2025-10-09T21:52:38.460216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343266290757104:2432], now have 1 active actors on pipe
2025-10-09T21:52:38.460222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343265928902999:7953]
2025-10-09T21:52:38.460223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343265928902999:7953]
2025-10-09T21:52:38.460224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343265928902999:7953]
2025-10-09T21:52:38.460233Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:38.460227Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2514
2025-10-09T21:52:38.460246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:38.460233Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2514
2025-10-09T21:52:38.460254Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:38.460260Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:38.460269Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:38.460278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:38.460279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343265928902999:7953]
2025-10-09T21:52:38.460279Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:38.460280Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:38.460284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:38.460306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.460309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.460334Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|50941287-4e3691d2-9a9fc884-4fda46c9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:38.460345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:38.460348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:38.460350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:38.460357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:38.460369Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:38.460380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:38.460502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.460507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.460516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:38.460598Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2515 partition: 0 MaxSeqNo: 25 sessionId: src_id|50941287-4e3691d2-9a9fc884-4fda46c9_0
2025-10-09T21:52:38.460854Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|3e285458-25d87bfb-6f8aee56-244b9537_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|50941287-4e3691d2-9a9fc884-4fda46c9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:38.463312Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|50941287-4e3691d2-9a9fc884-4fda46c9_0] Write session: try to update token
2025-10-09T21:52:38.463330Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|50941287-4e3691d2-9a9fc884-4fda46c9_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:38.471709Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:38.471724Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:38.471734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343265928902999:7953]
2025-10-09T21:52:38.471775Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:38.471777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:38.471952Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:38.472097Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:38.472240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:38.472382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:38.472576Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:38.472621Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:38.475506Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2515 sessionId: src_id|50941287-4e3691d2-9a9fc884-4fda46c9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:38.477845Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:38.478003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.478021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.478074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:38.478168Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:38.484563Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:38.484583Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:38.484586Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:38.484587Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:38.484597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343265928902999:7953]
2025-10-09T21:52:38.484601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343265928902999:7953]
2025-10-09T21:52:38.484605Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343265928902999:7953]
2025-10-09T21:52:38.484650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:38.484650Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:38.484657Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:38.484661Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:38.484663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.484666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:38.484807Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:38.541318Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343266290757210:3994]
2025-10-09T21:52:38.541602Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 13 pipe creation result: OK TabletId: 72075186224037892 Generation: 2514, pipe: [1:7559343266290757210:3994]
2025-10-09T21:52:38.629495Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:38.636094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.636125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.636270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:38.636325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:38.636384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:38.636442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:38.636483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:38.636492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:38.636559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:38.636628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:38.636685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:38.636743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:38.636784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:38.636790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:38.636843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:38.636897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:38.636951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:38.637003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:38.637040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:38.637048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:38.637105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:38.637162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:38.637218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:38.637288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:38.637328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:38.637333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:38.637392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:38.637446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:38.637501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:38.637554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:38.637593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:38.637598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:38.637651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:38.637710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:38.637776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:38.637831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:38.637886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:38.637891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:38.637945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:38.637994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:38.638069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:38.638125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:38.638163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:38.638168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:38.638219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:38.638272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:38.638325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:38.638376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:38.638415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:38.638420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:38.638472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:38.638524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:38.638577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:38.638628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:38.638672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:38.638680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:38.638745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:38.638794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:38.638848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:38.638900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:38.638953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:38.638958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:38.638998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343266290757210:3994], now have 1 active actors on pipe
2025-10-09T21:52:38.639260Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:38.639272Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:38.639288Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 13 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343266290757210:3994]
2025-10-09T21:52:38.639299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2515 reason: error at write: Disconnected sessionId: src_id|50941287-4e3691d2-9a9fc884-4fda46c9_0
2025-10-09T21:52:38.639309Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:38.639474Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2515 sessionId: src_id|50941287-4e3691d2-9a9fc884-4fda46c9_0 is DEAD
2025-10-09T21:52:38.639617Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50941287-4e3691d2-9a9fc884-4fda46c9_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:38.639627Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50941287-4e3691d2-9a9fc884-4fda46c9_0] Write session will restart in 0.007026s
2025-10-09T21:52:38.639647Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50941287-4e3691d2-9a9fc884-4fda46c9_0] Write session: Do CDS request
2025-10-09T21:52:38.639651Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50941287-4e3691d2-9a9fc884-4fda46c9_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:38.639786Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:38.647525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|50941287-4e3691d2-9a9fc884-4fda46c9_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:38.647608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:38.647625Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2516
2025-10-09T21:52:38.647775Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2516 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:38.647802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2516 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:38.647809Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2516 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:38.647812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:38.648208Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2516 sessionId:  describe result for acl check
2025-10-09T21:52:38.648246Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:38.648248Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:38.648250Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:38.648255Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343266290757269:3706] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:38.648771Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343266290757269:3706] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:38.648949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:38.650813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:38.651006Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:38.651018Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:38.651029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:38.651083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:38.651122Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343265928903102:7950]
2025-10-09T21:52:38.651234Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343266290757269:3706] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:38.651705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:38.652025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:38.652075Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:38.652386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:38.652442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.652449Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.652451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.652452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.652453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.652454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.652455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.652457Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.652458Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.652463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.652473Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.652477Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.652482Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.652485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.652490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.652494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.652497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.652501Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.652508Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.652516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.652522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.652528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.652537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.652543Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.652547Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.652557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.652561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.652567Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.652571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.652576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.652581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.652591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.652595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.652600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.652610Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.652616Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.652648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:38.656454Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343266290757269:3706] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:38.656470Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343266290757269:3706] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:38.656474Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343266290757269:3706] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:38.656483Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2516 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:38.658993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:38.660086Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:38.660106Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:38.660109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:38.660113Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:38.660123Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2515 [2:7559343265928903102:7950]
2025-10-09T21:52:38.660136Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:38.660153Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:38.660157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:38.660161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:38.660164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:38.660180Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:38.660192Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:38.660199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:38.660259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.660266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.660268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.660271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:38.660337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:38.660345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.660357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.660367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.660385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:38.660410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343266290757311:3706], now have 1 active actors on pipe
2025-10-09T21:52:38.660420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343266290757212:2432], now have 1 active actors on pipe
2025-10-09T21:52:38.660427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343265928903096:7950]
2025-10-09T21:52:38.660429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343265928903096:7950]
2025-10-09T21:52:38.660431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343265928903096:7950]
2025-10-09T21:52:38.660443Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:38.660454Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:38.660456Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:38.660458Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:38.660466Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:38.660453Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2515
2025-10-09T21:52:38.660470Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2515
2025-10-09T21:52:38.660478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:38.660486Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:38.660527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.660537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.660553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343265928903096:7950]
2025-10-09T21:52:38.660565Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:38.660583Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:38.660596Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d08d49bf-8cb30a25-961a0580-8c896883_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:38.660603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:38.660608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:38.660610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:38.660621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:38.660636Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:38.660663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:38.660790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.660799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.660813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:38.660888Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2516 partition: 0 MaxSeqNo: 25 sessionId: src_id|d08d49bf-8cb30a25-961a0580-8c896883_0
2025-10-09T21:52:38.661228Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|50941287-4e3691d2-9a9fc884-4fda46c9_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d08d49bf-8cb30a25-961a0580-8c896883_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:38.663630Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d08d49bf-8cb30a25-961a0580-8c896883_0] Write session: try to update token
2025-10-09T21:52:38.663650Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d08d49bf-8cb30a25-961a0580-8c896883_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:38.675443Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:38.675465Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:38.675467Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:38.675470Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:38.675480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343265928903096:7950]
2025-10-09T21:52:38.675484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343265928903096:7950]
2025-10-09T21:52:38.675488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343265928903096:7950]
2025-10-09T21:52:38.675534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:38.675542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.675549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:38.675560Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:38.675574Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:38.675577Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:38.675683Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:38.675921Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2516 sessionId: src_id|d08d49bf-8cb30a25-961a0580-8c896883_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:38.678130Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:38.739414Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343266290757319:3994]
2025-10-09T21:52:38.739665Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 14 pipe creation result: OK TabletId: 72075186224037892 Generation: 2515, pipe: [1:7559343266290757319:3994]
2025-10-09T21:52:38.822249Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:38.822510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.822528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.822572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343266290757319:3994], now have 1 active actors on pipe
2025-10-09T21:52:38.822776Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:38.822800Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:38.822801Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:38.822840Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 14 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343266290757319:3994]
2025-10-09T21:52:38.822856Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2516 reason: error at write: Disconnected sessionId: src_id|d08d49bf-8cb30a25-961a0580-8c896883_0
2025-10-09T21:52:38.823030Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2516 sessionId: src_id|d08d49bf-8cb30a25-961a0580-8c896883_0 is DEAD
2025-10-09T21:52:38.823067Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:38.823176Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d08d49bf-8cb30a25-961a0580-8c896883_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:38.823187Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d08d49bf-8cb30a25-961a0580-8c896883_0] Write session will restart in 0.007950s
2025-10-09T21:52:38.823214Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d08d49bf-8cb30a25-961a0580-8c896883_0] Write session: Do CDS request
2025-10-09T21:52:38.823220Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d08d49bf-8cb30a25-961a0580-8c896883_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:38.832265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:38.832544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d08d49bf-8cb30a25-961a0580-8c896883_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:38.832592Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:38.832605Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2517
2025-10-09T21:52:38.832750Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2517 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:38.832802Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2517 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:38.832813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2517 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:38.832817Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:38.833217Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2517 sessionId:  describe result for acl check
2025-10-09T21:52:38.833264Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:38.833265Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:38.833267Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:38.833273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343266290757381:3693] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:38.833695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343266290757381:3693] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:38.833922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:38.834085Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:38.834094Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:38.834104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:38.834155Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:38.834196Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343265928903200:7948]
2025-10-09T21:52:38.834632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:38.834846Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:38.834883Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:38.835151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:38.835193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.835199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.835200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.835202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.835203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.835204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.835205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.835206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.835207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.835212Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.835222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.835226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.835230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.835234Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.835238Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.835242Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.835246Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.835249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.835255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.835263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:38.835269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.835275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:38.835260Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343266290757381:3693] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:38.835281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.835287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:38.835291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.835297Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:38.835299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.835302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:38.835305Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.835308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:38.835311Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.835314Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.835317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.835320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.835324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.835331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.835359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:38.841144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343266290757381:3693] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:38.841160Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343266290757381:3693] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:38.841164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343266290757381:3693] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:38.841171Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2517 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:38.841486Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:38.842516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:38.842530Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:38.842531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:38.842534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:38.842539Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2516 [2:7559343265928903200:7948]
2025-10-09T21:52:38.842548Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:38.842560Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:38.842563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:38.842564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:38.842566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:38.842578Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:38.842582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:38.842589Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:38.842629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.842638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.842640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.842643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:38.842699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:38.842705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.842713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:38.842719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:38.842728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:38.842744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343266290757321:2432], now have 1 active actors on pipe
2025-10-09T21:52:38.842754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343265928903194:7948]
2025-10-09T21:52:38.842755Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343265928903194:7948]
2025-10-09T21:52:38.842757Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343265928903194:7948]
2025-10-09T21:52:38.842765Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:38.842771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343266290757420:3693], now have 1 active actors on pipe
2025-10-09T21:52:38.842778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:38.842790Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:38.842792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:38.842794Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:38.842797Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:38.842799Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:38.842811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343265928903194:7948]
2025-10-09T21:52:38.842810Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2516
2025-10-09T21:52:38.842817Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:38.842817Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2516
2025-10-09T21:52:38.842839Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:38.842845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.842848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.842870Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|25a650f4-147ad63a-5986dd58-84a2f90f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:38.842882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:38.842888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:38.842894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:38.842904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:38.842913Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:38.842928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:38.843081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.843095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.843115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:38.843214Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2517 partition: 0 MaxSeqNo: 25 sessionId: src_id|25a650f4-147ad63a-5986dd58-84a2f90f_0
2025-10-09T21:52:38.843512Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d08d49bf-8cb30a25-961a0580-8c896883_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|25a650f4-147ad63a-5986dd58-84a2f90f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:38.845867Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25a650f4-147ad63a-5986dd58-84a2f90f_0] Write session: try to update token
2025-10-09T21:52:38.845882Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25a650f4-147ad63a-5986dd58-84a2f90f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:38.854195Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:38.854212Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:38.854230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343265928903194:7948]
2025-10-09T21:52:38.854267Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:38.854276Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:38.854433Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:38.854570Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:38.854694Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:38.854845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:38.855041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:38.855076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:38.858479Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2517 sessionId: src_id|25a650f4-147ad63a-5986dd58-84a2f90f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:38.860882Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:38.861130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:38.861150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:38.861193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:38.861281Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:38.867423Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:38.867441Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:38.867443Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:38.867445Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:38.867453Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343265928903194:7948]
2025-10-09T21:52:38.867456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343265928903194:7948]
2025-10-09T21:52:38.867462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343265928903194:7948]
2025-10-09T21:52:38.867493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:38.867499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:38.867500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:38.867500Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:38.867504Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:38.867507Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:38.867620Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:38.922746Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343266290757428:3994]
2025-10-09T21:52:38.922993Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 15 pipe creation result: OK TabletId: 72075186224037892 Generation: 2516, pipe: [1:7559343266290757428:3994]
2025-10-09T21:52:39.013073Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:39.019941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.019967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.020121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:39.020179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:39.020234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:39.020309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:39.020358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:39.020366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:39.020426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:39.020478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:39.020550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:39.020616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:39.020657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:39.020663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:39.020717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:39.020776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:39.020839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:39.020894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:39.020936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:39.020942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:39.021001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:39.021065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:39.021130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:39.021186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:39.021227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:39.021232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:39.021310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:39.021367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:39.021430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:39.021485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:39.021526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:39.021536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:39.021599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:39.021653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:39.021707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:39.021760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:39.021803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:39.021812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:39.021869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:39.021919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:39.021971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:39.022022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:39.022081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:39.022088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:39.022149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:39.022205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:39.022258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:39.022317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:39.022358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:39.022364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:39.022426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:39.022482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:39.022536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:39.022596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:39.022639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:39.022647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:39.022702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:39.022753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:39.022810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:39.022868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:39.022959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:39.022964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:39.023004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343266290757428:3994], now have 1 active actors on pipe
2025-10-09T21:52:39.023212Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:39.023223Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:39.023227Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:39.023251Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2517 reason: error at write: Disconnected sessionId: src_id|25a650f4-147ad63a-5986dd58-84a2f90f_0
2025-10-09T21:52:39.023287Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 15 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343266290757428:3994]
2025-10-09T21:52:39.023451Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2517 sessionId: src_id|25a650f4-147ad63a-5986dd58-84a2f90f_0 is DEAD
2025-10-09T21:52:39.023579Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25a650f4-147ad63a-5986dd58-84a2f90f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:39.023599Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25a650f4-147ad63a-5986dd58-84a2f90f_0] Write session will restart in 0.006220s
2025-10-09T21:52:39.023618Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25a650f4-147ad63a-5986dd58-84a2f90f_0] Write session: Do CDS request
2025-10-09T21:52:39.023622Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25a650f4-147ad63a-5986dd58-84a2f90f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:39.023850Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:39.030531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25a650f4-147ad63a-5986dd58-84a2f90f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:39.030648Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:39.030662Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2518
2025-10-09T21:52:39.030795Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2518 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:39.030822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2518 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:39.030826Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2518 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:39.030828Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:39.031249Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2518 sessionId:  describe result for acl check
2025-10-09T21:52:39.031280Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:39.031285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:39.031286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:39.031290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343270585724788:3668] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:39.031775Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343270585724788:3668] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:39.032375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:39.033866Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343270585724788:3668] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:39.034129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:39.034301Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:39.034310Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:39.034330Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:39.034387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:39.034430Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343270223870600:7946]
2025-10-09T21:52:39.034866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:39.035134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:39.035191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:39.035516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:39.035574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.035581Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.035582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.035583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.035584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.035586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.035587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.035588Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.035590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.035595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.035607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.035611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.035615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.035619Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.035623Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.035627Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.035631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.035635Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.035642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.035650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.035657Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.035662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.035666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.035672Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.035681Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.035686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.035691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.035696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.035701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.035707Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.035711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.035717Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.035724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.035729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.035736Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.035741Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.035772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:39.036533Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343270585724788:3668] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:39.036544Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343270585724788:3668] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:39.036547Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343270585724788:3668] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:39.036554Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2518 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:39.042289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:39.043452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:39.043466Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:39.043469Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:39.043474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:39.043482Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2517 [2:7559343270223870600:7946]
2025-10-09T21:52:39.043497Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:39.043516Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:39.043524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:39.043527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:39.043530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:39.043549Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:39.043561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:39.043568Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:39.043626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.043634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.043636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.043639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:39.043711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:39.043718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.043730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.043747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.043767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:39.043794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585724729:2432], now have 1 active actors on pipe
2025-10-09T21:52:39.043802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585724830:3668], now have 1 active actors on pipe
2025-10-09T21:52:39.043810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343270223870595:7946]
2025-10-09T21:52:39.043812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343270223870595:7946]
2025-10-09T21:52:39.043814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343270223870595:7946]
2025-10-09T21:52:39.043804Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2517
2025-10-09T21:52:39.043830Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2517
2025-10-09T21:52:39.043825Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:39.043842Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:39.043844Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:39.043848Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:39.043850Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:39.043857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:39.043861Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:39.043915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.043924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.043937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343270223870595:7946]
2025-10-09T21:52:39.043946Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:39.043956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:39.043980Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|a2059126-dfa60907-2d675e02-2f5b0b7_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:39.043992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:39.043996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:39.043998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:39.044008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:39.044022Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:39.044047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:39.044186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.044196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.044218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:39.044328Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2518 partition: 0 MaxSeqNo: 25 sessionId: src_id|a2059126-dfa60907-2d675e02-2f5b0b7_0
2025-10-09T21:52:39.044710Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25a650f4-147ad63a-5986dd58-84a2f90f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|a2059126-dfa60907-2d675e02-2f5b0b7_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:39.047099Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a2059126-dfa60907-2d675e02-2f5b0b7_0] Write session: try to update token
2025-10-09T21:52:39.047118Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a2059126-dfa60907-2d675e02-2f5b0b7_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:39.058724Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:39.058739Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:39.058742Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:39.058745Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:39.058756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343270223870595:7946]
2025-10-09T21:52:39.058761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343270223870595:7946]
2025-10-09T21:52:39.058768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343270223870595:7946]
2025-10-09T21:52:39.058817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:39.058825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.058826Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:39.058829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:39.058832Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:39.058836Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:39.058969Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:39.060444Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2518 sessionId: src_id|a2059126-dfa60907-2d675e02-2f5b0b7_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:39.062939Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:39.123419Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343270585724837:3994]
2025-10-09T21:52:39.123665Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 16 pipe creation result: OK TabletId: 72075186224037892 Generation: 2517, pipe: [1:7559343270585724837:3994]
2025-10-09T21:52:39.206326Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:39.206570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.206588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.206635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585724837:3994], now have 1 active actors on pipe
2025-10-09T21:52:39.206809Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:39.206903Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 16 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343270585724837:3994]
2025-10-09T21:52:39.206912Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:39.206934Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:39.206995Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2518 reason: status is not ok: tablet will be restarted right now sessionId: src_id|a2059126-dfa60907-2d675e02-2f5b0b7_0
2025-10-09T21:52:39.207161Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2518 sessionId: src_id|a2059126-dfa60907-2d675e02-2f5b0b7_0 is DEAD
2025-10-09T21:52:39.207308Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2059126-dfa60907-2d675e02-2f5b0b7_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:39.207321Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2059126-dfa60907-2d675e02-2f5b0b7_0] Write session will restart in 0.005761s
2025-10-09T21:52:39.207348Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2059126-dfa60907-2d675e02-2f5b0b7_0] Write session: Do CDS request
2025-10-09T21:52:39.207354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2059126-dfa60907-2d675e02-2f5b0b7_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:39.214520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|a2059126-dfa60907-2d675e02-2f5b0b7_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:39.214619Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:39.214644Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2519
2025-10-09T21:52:39.214779Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2519 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:39.214810Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2519 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:39.214816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2519 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:39.214819Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:39.215187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:39.215250Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2519 sessionId:  describe result for acl check
2025-10-09T21:52:39.215290Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:39.215296Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:39.215298Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:39.215304Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343270585724902:3642] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:39.215852Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343270585724902:3642] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:39.216713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:39.216876Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:39.216886Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:39.216897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:39.216949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:39.216987Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343270223870699:7944]
2025-10-09T21:52:39.217409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:39.217705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:39.217740Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:39.217885Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343270585724902:3642] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:39.217937Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:39.217969Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.217973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.217973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.217974Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.217975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.217975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.217976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.217976Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.217977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.217980Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.217987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.217989Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.217991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.217993Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.217995Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.217997Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.217999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.218001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.218006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.218014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.218020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.218025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.218044Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.218050Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.218055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.218059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.218061Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.218064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.218067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.218070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.218073Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.218076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.218079Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.218082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.218085Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.218088Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.218104Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:39.223999Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343270585724902:3642] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:39.224014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343270585724902:3642] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:39.224016Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343270585724902:3642] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:39.224024Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2519 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:39.224382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:39.225354Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:39.225365Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:39.225366Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:39.225370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:39.225375Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2518 [2:7559343270223870699:7944]
2025-10-09T21:52:39.225384Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:39.225412Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:39.225413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:39.225415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:39.225417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:39.225429Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:39.225433Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:39.225438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:39.225483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.225489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.225491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.225494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:39.225565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:39.225573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.225584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.225594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.225616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:39.225639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585724942:3642], now have 1 active actors on pipe
2025-10-09T21:52:39.225646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585724840:2432], now have 1 active actors on pipe
2025-10-09T21:52:39.225654Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343270223870694:7944]
2025-10-09T21:52:39.225656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343270223870694:7944]
2025-10-09T21:52:39.225658Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343270223870694:7944]
2025-10-09T21:52:39.225673Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:39.225668Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2518
2025-10-09T21:52:39.225691Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:39.225694Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:39.225678Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2518
2025-10-09T21:52:39.225697Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:39.225699Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:39.225704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:39.225707Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:39.225731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343270223870694:7944]
2025-10-09T21:52:39.225736Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:39.225745Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:39.225748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.225751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.225784Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e1e36bdb-b739e454-4b6ca0e6-58562297_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:39.225794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:39.225797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:39.225799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:39.225805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:39.225813Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:39.225825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:39.225989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.225996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.226009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:39.226103Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2519 partition: 0 MaxSeqNo: 25 sessionId: src_id|e1e36bdb-b739e454-4b6ca0e6-58562297_0
2025-10-09T21:52:39.226436Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|a2059126-dfa60907-2d675e02-2f5b0b7_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e1e36bdb-b739e454-4b6ca0e6-58562297_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:39.228921Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1e36bdb-b739e454-4b6ca0e6-58562297_0] Write session: try to update token
2025-10-09T21:52:39.228937Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1e36bdb-b739e454-4b6ca0e6-58562297_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:39.241089Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:39.241100Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:39.241102Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:39.241103Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:39.241111Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343270223870694:7944]
2025-10-09T21:52:39.241115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343270223870694:7944]
2025-10-09T21:52:39.241117Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343270223870694:7944]
2025-10-09T21:52:39.241152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:39.241157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.241159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:39.241164Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:39.241170Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:39.241173Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:39.241220Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2519 sessionId: src_id|e1e36bdb-b739e454-4b6ca0e6-58562297_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:39.241307Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:39.243677Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:39.307276Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343270585724952:3994]
2025-10-09T21:52:39.307542Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 17 pipe creation result: OK TabletId: 72075186224037892 Generation: 2518, pipe: [1:7559343270585724952:3994]
2025-10-09T21:52:39.389524Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:39.390114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.390133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.390177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585724952:3994], now have 1 active actors on pipe
2025-10-09T21:52:39.390387Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:39.390389Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:39.390434Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 17 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343270585724952:3994]
2025-10-09T21:52:39.390450Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:39.390514Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2519 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e1e36bdb-b739e454-4b6ca0e6-58562297_0
2025-10-09T21:52:39.390666Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2519 sessionId: src_id|e1e36bdb-b739e454-4b6ca0e6-58562297_0 is DEAD
2025-10-09T21:52:39.390810Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1e36bdb-b739e454-4b6ca0e6-58562297_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:39.390819Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1e36bdb-b739e454-4b6ca0e6-58562297_0] Write session will restart in 0.008722s
2025-10-09T21:52:39.390838Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1e36bdb-b739e454-4b6ca0e6-58562297_0] Write session: Do CDS request
2025-10-09T21:52:39.390843Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1e36bdb-b739e454-4b6ca0e6-58562297_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:39.399906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:39.400491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e1e36bdb-b739e454-4b6ca0e6-58562297_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:39.400618Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:39.400631Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2520
2025-10-09T21:52:39.400777Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2520 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:39.400812Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2520 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:39.400818Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2520 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:39.400822Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:39.401174Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2520 sessionId:  describe result for acl check
2025-10-09T21:52:39.401206Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:39.401211Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:39.401212Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:39.401216Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343270585725040:3629] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:39.401553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:39.401623Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343270585725040:3629] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:39.401722Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:39.401732Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:39.401744Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:39.401804Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:39.401844Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343270223870831:7926]
2025-10-09T21:52:39.402468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:39.402765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:39.402814Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:39.403134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:39.403187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.403194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.403195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.403196Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.403197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.403199Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.403200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.403201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.403203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.403208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.403219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.403224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.403228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.403232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.403236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.403243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.403249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.403258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.403265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.403277Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.403288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.403298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.403302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.403308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.403317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.403324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.403329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.403336Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.403344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.403350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.403360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.403370Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.403374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.403379Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.403384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.403389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.403418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:39.403496Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343270585725040:3629] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:39.409532Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343270585725040:3629] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:39.409546Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343270585725040:3629] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:39.409548Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343270585725040:3629] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:39.409554Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2520 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:39.409761Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:39.410952Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:39.410962Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:39.410964Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:39.410967Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:39.410972Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2519 [2:7559343270223870831:7926]
2025-10-09T21:52:39.410981Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:39.410996Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:39.410998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:39.411000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:39.411001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:39.411015Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:39.411022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:39.411028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:39.411076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.411081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.411083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.411085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:39.411154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:39.411162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.411175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.411192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.411239Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2519
2025-10-09T21:52:39.411256Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2519
2025-10-09T21:52:39.411214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:39.411240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585725085:3629], now have 1 active actors on pipe
2025-10-09T21:52:39.411244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585724973:2432], now have 1 active actors on pipe
2025-10-09T21:52:39.411252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343270223870826:7926]
2025-10-09T21:52:39.411255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343270223870826:7926]
2025-10-09T21:52:39.411257Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343270223870826:7926]
2025-10-09T21:52:39.411266Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:39.411285Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:39.411290Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:39.411294Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:39.411296Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:39.411300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:39.411302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:39.411347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.411352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.411369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343270223870826:7926]
2025-10-09T21:52:39.411380Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:39.411395Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:39.411407Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|52a8a977-304b2c60-8bbe750d-da9d269c_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:39.411412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:39.411416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:39.411417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:39.411425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:39.411439Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:39.411457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:39.411603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.411613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.411631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:39.411722Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2520 partition: 0 MaxSeqNo: 25 sessionId: src_id|52a8a977-304b2c60-8bbe750d-da9d269c_0
2025-10-09T21:52:39.412082Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e1e36bdb-b739e454-4b6ca0e6-58562297_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|52a8a977-304b2c60-8bbe750d-da9d269c_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:39.414791Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52a8a977-304b2c60-8bbe750d-da9d269c_0] Write session: try to update token
2025-10-09T21:52:39.414812Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52a8a977-304b2c60-8bbe750d-da9d269c_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:39.426344Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:39.426355Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:39.426358Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:39.426360Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:39.426371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343270223870826:7926]
2025-10-09T21:52:39.426376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343270223870826:7926]
2025-10-09T21:52:39.426381Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343270223870826:7926]
2025-10-09T21:52:39.426421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:39.426427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.426426Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:39.426429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:39.426432Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:39.426436Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:39.426560Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:39.427409Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2520 sessionId: src_id|52a8a977-304b2c60-8bbe750d-da9d269c_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:39.429624Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:39.490545Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343270585725092:3994]
2025-10-09T21:52:39.490798Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 18 pipe creation result: OK TabletId: 72075186224037892 Generation: 2519, pipe: [1:7559343270585725092:3994]
2025-10-09T21:52:39.571671Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:39.572077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.572095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.572154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585725092:3994], now have 1 active actors on pipe
2025-10-09T21:52:39.572339Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:39.572416Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:39.572424Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 18 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343270585725092:3994]
2025-10-09T21:52:39.572431Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:39.572441Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:39.572489Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2520 reason: error at write: Disconnected sessionId: src_id|52a8a977-304b2c60-8bbe750d-da9d269c_0
2025-10-09T21:52:39.572652Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2520 sessionId: src_id|52a8a977-304b2c60-8bbe750d-da9d269c_0 is DEAD
2025-10-09T21:52:39.572745Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52a8a977-304b2c60-8bbe750d-da9d269c_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:39.572752Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52a8a977-304b2c60-8bbe750d-da9d269c_0] Write session will restart in 0.007865s
2025-10-09T21:52:39.572772Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52a8a977-304b2c60-8bbe750d-da9d269c_0] Write session: Do CDS request
2025-10-09T21:52:39.572778Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52a8a977-304b2c60-8bbe750d-da9d269c_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:39.581314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:39.581460Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|52a8a977-304b2c60-8bbe750d-da9d269c_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:39.581576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:39.581590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2521
2025-10-09T21:52:39.581711Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2521 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:39.581742Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2521 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:39.581752Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2521 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:39.581756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:39.582187Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2521 sessionId:  describe result for acl check
2025-10-09T21:52:39.582221Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:39.582225Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:39.582227Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:39.582232Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343270585725160:3620] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:39.582756Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343270585725160:3620] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:39.583122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:39.583282Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:39.583297Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:39.583308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:39.583362Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:39.583399Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343270223870940:7914]
2025-10-09T21:52:39.583961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:39.584235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:39.584267Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:39.584483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:39.584513Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.584517Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.584518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.584518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.584519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.584520Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.584521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.584521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.584529Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.584532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.584537Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.584540Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.584542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.584544Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.584546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.584548Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.584550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.584552Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.584556Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.584561Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.584565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.584568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.584570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.584573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.584576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.584579Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.584582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.584585Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.584587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.584590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.584593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.584596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.584598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.584601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.584604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.584607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.584622Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:39.584644Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343270585725160:3620] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:39.590121Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343270585725160:3620] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:39.590132Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343270585725160:3620] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:39.590144Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343270585725160:3620] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:39.590150Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2521 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:39.590497Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:39.591463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:39.591474Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:39.591476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:39.591478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:39.591486Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2520 [2:7559343270223870940:7914]
2025-10-09T21:52:39.591495Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:39.591507Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:39.591513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:39.591515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:39.591516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:39.591527Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:39.591531Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:39.591536Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:39.591574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.591580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.591583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.591585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:39.591704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:39.591709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.591716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.591724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.591744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:39.591767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585725199:3620], now have 1 active actors on pipe
2025-10-09T21:52:39.591774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585725095:2432], now have 1 active actors on pipe
2025-10-09T21:52:39.591779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343270223870935:7914]
2025-10-09T21:52:39.591781Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343270223870935:7914]
2025-10-09T21:52:39.591787Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2520
2025-10-09T21:52:39.591782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343270223870935:7914]
2025-10-09T21:52:39.591793Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:39.591805Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:39.591814Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:39.591816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:39.591819Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:39.591821Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:39.591823Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:39.591836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343270223870935:7914]
2025-10-09T21:52:39.591835Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2520
2025-10-09T21:52:39.591838Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:39.591840Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:39.591886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.591895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.591915Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9a773be4-6cd55914-ce4b2d03-2c233750_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:39.591925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:39.591928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:39.591929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:39.591936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:39.591944Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:39.591953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:39.592080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.592085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.592095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:39.592164Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2521 partition: 0 MaxSeqNo: 25 sessionId: src_id|9a773be4-6cd55914-ce4b2d03-2c233750_0
2025-10-09T21:52:39.592471Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|52a8a977-304b2c60-8bbe750d-da9d269c_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9a773be4-6cd55914-ce4b2d03-2c233750_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:39.594911Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9a773be4-6cd55914-ce4b2d03-2c233750_0] Write session: try to update token
2025-10-09T21:52:39.594924Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9a773be4-6cd55914-ce4b2d03-2c233750_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:39.605828Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:39.605842Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:39.605843Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:39.605844Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:39.605854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343270223870935:7914]
2025-10-09T21:52:39.605857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343270223870935:7914]
2025-10-09T21:52:39.605860Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343270223870935:7914]
2025-10-09T21:52:39.605892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:39.605898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.605903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:39.605903Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:39.605908Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:39.605912Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:39.606018Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:39.607200Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2521 sessionId: src_id|9a773be4-6cd55914-ce4b2d03-2c233750_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:39.609572Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:39.672823Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343270585725206:3994]
2025-10-09T21:52:39.673091Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 19 pipe creation result: OK TabletId: 72075186224037892 Generation: 2520, pipe: [1:7559343270585725206:3994]
2025-10-09T21:52:39.750678Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:39.750936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.750948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.750984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585725206:3994], now have 1 active actors on pipe
2025-10-09T21:52:39.751156Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:39.751247Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:39.751262Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:39.751326Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2521 reason: error at write: Disconnected sessionId: src_id|9a773be4-6cd55914-ce4b2d03-2c233750_0
2025-10-09T21:52:39.751358Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 19 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343270585725206:3994]
2025-10-09T21:52:39.751380Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:39.751506Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2521 sessionId: src_id|9a773be4-6cd55914-ce4b2d03-2c233750_0 is DEAD
2025-10-09T21:52:39.751653Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a773be4-6cd55914-ce4b2d03-2c233750_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:39.751666Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a773be4-6cd55914-ce4b2d03-2c233750_0] Write session will restart in 0.009660s
2025-10-09T21:52:39.751693Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a773be4-6cd55914-ce4b2d03-2c233750_0] Write session: Do CDS request
2025-10-09T21:52:39.751699Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a773be4-6cd55914-ce4b2d03-2c233750_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:39.759278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:39.760991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:39.761134Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:39.761149Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:39.761162Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:39.761209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:39.761238Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343270223871044:7912]
2025-10-09T21:52:39.761642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:39.761899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:39.761932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:39.762164Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:39.762195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.762200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.762201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.762201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.762202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.762203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.762203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.762204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.762204Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.762208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.762215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.762217Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.762220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.762222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.762224Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.762226Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.762228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.762230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.762235Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.762240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.762244Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.762247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.762250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.762253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.762255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.762258Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.762260Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.762263Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.762266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.762269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.762275Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.762278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.762281Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.762285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.762287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.762290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.762306Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:39.762516Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9a773be4-6cd55914-ce4b2d03-2c233750_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:39.762609Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:39.762622Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2522
2025-10-09T21:52:39.762756Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2522 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:39.762785Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2522 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:39.762794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2522 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:39.762798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:39.763121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2522 sessionId:  describe result for acl check
2025-10-09T21:52:39.763149Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:39.763154Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:39.763156Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:39.763159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343270585725280:3598] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:39.763590Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343270585725280:3598] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:39.765761Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343270585725280:3598] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:39.767980Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343270585725280:3598] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:39.767990Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343270585725280:3598] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:39.767993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343270585725280:3598] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:39.767999Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2522 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:39.768328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:39.769285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:39.769296Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:39.769299Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:39.769302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:39.769308Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2521 [2:7559343270223871044:7912]
2025-10-09T21:52:39.769316Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:39.769331Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:39.769337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:39.769339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:39.769340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:39.769352Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:39.769355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:39.769360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:39.769402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.769407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.769408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.769410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:39.769458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:39.769463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.769471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.769480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.769491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:39.769508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585725209:2432], now have 1 active actors on pipe
2025-10-09T21:52:39.769514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585725314:3598], now have 1 active actors on pipe
2025-10-09T21:52:39.769519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343270223871039:7912]
2025-10-09T21:52:39.769521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343270223871039:7912]
2025-10-09T21:52:39.769522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343270223871039:7912]
2025-10-09T21:52:39.769530Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:39.769540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:39.769548Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:39.769551Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:39.769552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:39.769553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:39.769555Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:39.769574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343270223871039:7912]
2025-10-09T21:52:39.769580Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:39.769555Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2521
2025-10-09T21:52:39.769566Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2521
2025-10-09T21:52:39.769582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:39.769598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.769602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.769629Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8356299d-d17ea650-ddca5583-8c705215_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:39.769637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:39.769640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:39.769643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:39.769650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:39.769661Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:39.769677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:39.769779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.769785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.769795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:39.769862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2522 partition: 0 MaxSeqNo: 25 sessionId: src_id|8356299d-d17ea650-ddca5583-8c705215_0
2025-10-09T21:52:39.770178Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9a773be4-6cd55914-ce4b2d03-2c233750_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8356299d-d17ea650-ddca5583-8c705215_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:39.772367Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8356299d-d17ea650-ddca5583-8c705215_0] Write session: try to update token
2025-10-09T21:52:39.772381Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8356299d-d17ea650-ddca5583-8c705215_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:39.783391Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:39.783405Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:39.783407Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:39.783408Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:39.783417Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343270223871039:7912]
2025-10-09T21:52:39.783421Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343270223871039:7912]
2025-10-09T21:52:39.783424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343270223871039:7912]
2025-10-09T21:52:39.783458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:39.783464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.783466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:39.783471Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:39.783477Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:39.783481Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:39.783599Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:39.784137Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2522 sessionId: src_id|8356299d-d17ea650-ddca5583-8c705215_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:39.786583Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:39.851473Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343270585725322:3994]
2025-10-09T21:52:39.851713Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 20 pipe creation result: OK TabletId: 72075186224037892 Generation: 2521, pipe: [1:7559343270585725322:3994]
2025-10-09T21:52:39.926375Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:39.926691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.926705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.926746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585725322:3994], now have 1 active actors on pipe
2025-10-09T21:52:39.926936Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:39.926989Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 20 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343270585725322:3994]
2025-10-09T21:52:39.927009Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:39.927016Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:39.927076Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2522 reason: error at write: Disconnected sessionId: src_id|8356299d-d17ea650-ddca5583-8c705215_0
2025-10-09T21:52:39.927111Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:39.927256Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2522 sessionId: src_id|8356299d-d17ea650-ddca5583-8c705215_0 is DEAD
2025-10-09T21:52:39.927424Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8356299d-d17ea650-ddca5583-8c705215_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:39.927435Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8356299d-d17ea650-ddca5583-8c705215_0] Write session will restart in 0.005542s
2025-10-09T21:52:39.927459Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8356299d-d17ea650-ddca5583-8c705215_0] Write session: Do CDS request
2025-10-09T21:52:39.927464Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8356299d-d17ea650-ddca5583-8c705215_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:39.934445Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8356299d-d17ea650-ddca5583-8c705215_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:39.934516Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:39.934531Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2523
2025-10-09T21:52:39.934650Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2523 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:39.934686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2523 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:39.934691Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2523 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:39.934695Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:39.935143Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2523 sessionId:  describe result for acl check
2025-10-09T21:52:39.935178Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:39.935185Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:39.935187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:39.935193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343270585725391:3571] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:39.935490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:39.935765Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343270585725391:3571] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:39.937205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:39.937391Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:39.937400Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:39.937413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:39.937470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:39.937502Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343270223871151:7905]
2025-10-09T21:52:39.937872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343270585725391:3571] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:39.937968Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:39.938229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:39.938265Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:39.938533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:39.938587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.938596Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.938597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.938598Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.938600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.938601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.938602Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.938603Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.938604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.938608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.938618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.938621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.938625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.938629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.938632Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.938636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.938640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.938643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.938650Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.938659Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:39.938665Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.938670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:39.938675Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.938679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:39.938682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.938685Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:39.938688Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.938691Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:39.938695Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.938698Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:39.938701Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.938704Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.938706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.938709Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.938711Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.938714Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.938730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:39.943452Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343270585725391:3571] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:39.943464Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343270585725391:3571] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:39.943467Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343270585725391:3571] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:39.943473Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2523 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:39.944788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:39.945792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:39.945805Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:39.945806Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:39.945809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:39.945815Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2522 [2:7559343270223871151:7905]
2025-10-09T21:52:39.945824Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:39.945840Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:39.945843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:39.945846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:39.945849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:39.945881Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:39.945892Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:39.945898Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:39.945941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.945946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.945947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.945949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:39.946000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:39.946006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.946013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:39.946020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:39.946042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:39.946057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343270223871146:7905]
2025-10-09T21:52:39.946060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343270223871146:7905]
2025-10-09T21:52:39.946062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343270223871146:7905]
2025-10-09T21:52:39.946068Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:39.946079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:39.946080Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:39.946083Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:39.946085Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:39.946087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:39.946090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:39.946098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585725325:2432], now have 1 active actors on pipe
2025-10-09T21:52:39.946101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343270585725432:3571], now have 1 active actors on pipe
2025-10-09T21:52:39.946122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343270223871146:7905]
2025-10-09T21:52:39.946127Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:39.946146Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:39.946229Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2522
2025-10-09T21:52:39.946247Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2522
2025-10-09T21:52:39.946281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.946290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.946316Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|4ca06cde-5f77d0e1-c67be93b-b905bf94_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:39.946327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:39.946330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:39.946331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:39.946338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:39.946348Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:39.946360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:39.946520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:39.946526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:39.946535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:39.946605Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2523 partition: 0 MaxSeqNo: 25 sessionId: src_id|4ca06cde-5f77d0e1-c67be93b-b905bf94_0
2025-10-09T21:52:39.946958Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8356299d-d17ea650-ddca5583-8c705215_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|4ca06cde-5f77d0e1-c67be93b-b905bf94_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:39.949388Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ca06cde-5f77d0e1-c67be93b-b905bf94_0] Write session: try to update token
2025-10-09T21:52:39.949405Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ca06cde-5f77d0e1-c67be93b-b905bf94_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:39.961373Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:39.961385Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:39.961387Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:39.961388Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:39.961396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343270223871146:7905]
2025-10-09T21:52:39.961399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343270223871146:7905]
2025-10-09T21:52:39.961404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343270223871146:7905]
2025-10-09T21:52:39.961438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:39.961444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:39.961446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:39.961460Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:39.961469Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:39.961472Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:39.961587Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:39.962591Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2523 sessionId: src_id|4ca06cde-5f77d0e1-c67be93b-b905bf94_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:39.965036Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:40.027140Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343274880692736:3994]
2025-10-09T21:52:40.027420Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 21 pipe creation result: OK TabletId: 72075186224037892 Generation: 2522, pipe: [1:7559343274880692736:3994]
2025-10-09T21:52:40.106269Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:40.106630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:40.106643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:40.106680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343274880692736:3994], now have 1 active actors on pipe
2025-10-09T21:52:40.106843Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:40.106930Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:40.106932Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:40.106942Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:40.106974Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 21 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343274880692736:3994]
2025-10-09T21:52:40.107002Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2523 reason: error at write: Disconnected sessionId: src_id|4ca06cde-5f77d0e1-c67be93b-b905bf94_0
2025-10-09T21:52:40.107142Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2523 sessionId: src_id|4ca06cde-5f77d0e1-c67be93b-b905bf94_0 is DEAD
2025-10-09T21:52:40.107292Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ca06cde-5f77d0e1-c67be93b-b905bf94_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:40.107303Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ca06cde-5f77d0e1-c67be93b-b905bf94_0] Write session will restart in 0.006375s
2025-10-09T21:52:40.107331Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ca06cde-5f77d0e1-c67be93b-b905bf94_0] Write session: Do CDS request
2025-10-09T21:52:40.107338Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ca06cde-5f77d0e1-c67be93b-b905bf94_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:40.114537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|4ca06cde-5f77d0e1-c67be93b-b905bf94_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:40.114568Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:40.114582Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2524
2025-10-09T21:52:40.114736Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2524 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:40.114766Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2524 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:40.114776Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2524 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:40.114780Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:40.115140Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2524 sessionId:  describe result for acl check
2025-10-09T21:52:40.115173Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:40.115175Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:40.115177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:40.115181Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343274880692803:3552] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:40.115669Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343274880692803:3552] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:40.116312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:40.117591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343274880692803:3552] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:40.118272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:40.118422Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:40.118428Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:40.118444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:40.118483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:40.118515Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343274518838553:7903]
2025-10-09T21:52:40.118882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:40.119082Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:40.119116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:40.119345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:40.119380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.119384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.119385Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.119386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.119386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.119387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.119388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.119388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.119389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.119392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.119399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.119401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.119403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.119405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.119408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.119410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.119412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.119414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.119418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.119423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.119427Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.119431Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.119433Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.119437Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.119439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.119442Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.119445Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.119448Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.119450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.119454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.119456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.119460Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.119462Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.119465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.119468Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.119471Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.119488Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:40.119796Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343274880692803:3552] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:40.119811Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343274880692803:3552] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:40.119814Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343274880692803:3552] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:40.119820Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2524 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:40.125515Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:40.126647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:40.126661Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:40.126664Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:40.126667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:40.126672Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2523 [2:7559343274518838553:7903]
2025-10-09T21:52:40.126681Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:40.126696Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:40.126698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:40.126700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:40.126701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:40.126714Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:40.126718Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:40.126723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:40.126763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.126769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.126770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.126773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:40.126846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:40.126853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.126865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.126881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.126898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:40.126918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343274880692845:3552], now have 1 active actors on pipe
2025-10-09T21:52:40.126924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343274880692738:2432], now have 1 active actors on pipe
2025-10-09T21:52:40.126929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343274518838548:7903]
2025-10-09T21:52:40.126931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343274518838548:7903]
2025-10-09T21:52:40.126932Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343274518838548:7903]
2025-10-09T21:52:40.126941Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:40.126955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:40.126967Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:40.126967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:40.126969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:40.126969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:40.126971Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:40.126991Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343274518838548:7903]
2025-10-09T21:52:40.126998Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:40.127001Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:40.126974Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2523
2025-10-09T21:52:40.126974Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2523
2025-10-09T21:52:40.127020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:40.127027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:40.127055Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e87cc672-4066144e-aca8228d-f40c12c3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:40.127064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:40.127069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:40.127071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:40.127078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:40.127089Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:40.127106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:40.127237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:40.127245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:40.127260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:40.127347Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2524 partition: 0 MaxSeqNo: 25 sessionId: src_id|e87cc672-4066144e-aca8228d-f40c12c3_0
2025-10-09T21:52:40.127662Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|4ca06cde-5f77d0e1-c67be93b-b905bf94_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e87cc672-4066144e-aca8228d-f40c12c3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:40.130013Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e87cc672-4066144e-aca8228d-f40c12c3_0] Write session: try to update token
2025-10-09T21:52:40.130047Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e87cc672-4066144e-aca8228d-f40c12c3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:40.140993Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:40.141011Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:40.141013Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:40.141014Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:40.141022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343274518838548:7903]
2025-10-09T21:52:40.141025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343274518838548:7903]
2025-10-09T21:52:40.141027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343274518838548:7903]
2025-10-09T21:52:40.141061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:40.141066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.141068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:40.141076Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:40.141082Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:40.141086Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:40.141184Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:40.142368Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2524 sessionId: src_id|e87cc672-4066144e-aca8228d-f40c12c3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:40.144507Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:40.207361Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343274880692853:3994]
2025-10-09T21:52:40.207599Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 22 pipe creation result: OK TabletId: 72075186224037892 Generation: 2523, pipe: [1:7559343274880692853:3994]
2025-10-09T21:52:40.290595Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:40.290848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:40.290863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:40.290902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343274880692853:3994], now have 1 active actors on pipe
2025-10-09T21:52:40.291042Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:40.291103Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:40.291119Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 22 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343274880692853:3994]
2025-10-09T21:52:40.291147Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:40.291229Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2524 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e87cc672-4066144e-aca8228d-f40c12c3_0
2025-10-09T21:52:40.291418Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2524 sessionId: src_id|e87cc672-4066144e-aca8228d-f40c12c3_0 is DEAD
2025-10-09T21:52:40.291560Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e87cc672-4066144e-aca8228d-f40c12c3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:40.291572Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e87cc672-4066144e-aca8228d-f40c12c3_0] Write session will restart in 0.008864s
2025-10-09T21:52:40.291599Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e87cc672-4066144e-aca8228d-f40c12c3_0] Write session: Do CDS request
2025-10-09T21:52:40.291606Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e87cc672-4066144e-aca8228d-f40c12c3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:40.299124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:40.300848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:40.300995Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:40.301003Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:40.301011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:40.301051Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:40.301078Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343274518838659:7902]
2025-10-09T21:52:40.301453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e87cc672-4066144e-aca8228d-f40c12c3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:40.301483Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:40.301558Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:40.301572Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2525
2025-10-09T21:52:40.301674Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2525 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:40.301703Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2525 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:40.301713Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2525 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:40.301717Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:40.301746Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:40.301772Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:40.301977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:40.302008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.302012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.302012Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.302013Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.302014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.302014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.302015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.302016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.302016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.302019Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.302026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.302058Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.302060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.302062Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.302065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.302067Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.302069Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.302070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.302076Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.302083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.302094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.302100Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.302103Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.302107Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.302111Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.302116Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.302121Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.302126Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.302130Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.302134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.302137Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.302140Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.302142Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.302145Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.302138Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2525 sessionId:  describe result for acl check
2025-10-09T21:52:40.302149Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.302152Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.302169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:40.302170Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:40.302177Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:40.302179Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:40.302187Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343274880692930:3554] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:40.302723Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343274880692930:3554] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:40.304888Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343274880692930:3554] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:40.307780Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343274880692930:3554] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:40.307793Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343274880692930:3554] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:40.307797Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343274880692930:3554] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:40.307805Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2525 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:40.308185Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:40.309178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:40.309189Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:40.309191Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:40.309194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:40.309200Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2524 [2:7559343274518838659:7902]
2025-10-09T21:52:40.309211Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:40.309229Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:40.309236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:40.309249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:40.309251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:40.309265Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:40.309273Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:40.309279Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:40.309319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.309324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.309325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.309327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:40.309374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:40.309379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.309386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.309396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.309408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:40.309426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343274880692966:3554], now have 1 active actors on pipe
2025-10-09T21:52:40.309433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343274880692858:2432], now have 1 active actors on pipe
2025-10-09T21:52:40.309438Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343274518838654:7902]
2025-10-09T21:52:40.309441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343274518838654:7902]
2025-10-09T21:52:40.309442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343274518838654:7902]
2025-10-09T21:52:40.309451Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:40.309462Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:40.309464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:40.309470Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:40.309472Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:40.309473Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:40.309475Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:40.309496Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343274518838654:7902]
2025-10-09T21:52:40.309508Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:40.309514Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:40.309524Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2524
2025-10-09T21:52:40.309529Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2524
2025-10-09T21:52:40.309626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:40.309633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:40.309654Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|864b7119-abc9ab3-3e79fe29-28c8f9d_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:40.309659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:40.309662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:40.309664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:40.309671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:40.309679Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:40.309696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:40.309875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:40.309884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:40.309913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:40.310049Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2525 partition: 0 MaxSeqNo: 25 sessionId: src_id|864b7119-abc9ab3-3e79fe29-28c8f9d_0
2025-10-09T21:52:40.310354Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e87cc672-4066144e-aca8228d-f40c12c3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|864b7119-abc9ab3-3e79fe29-28c8f9d_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:40.312807Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|864b7119-abc9ab3-3e79fe29-28c8f9d_0] Write session: try to update token
2025-10-09T21:52:40.312821Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|864b7119-abc9ab3-3e79fe29-28c8f9d_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:40.321136Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:40.321148Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:40.321156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343274518838654:7902]
2025-10-09T21:52:40.321194Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:40.321196Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:40.321355Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:40.321479Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:40.321602Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:40.321717Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:40.321900Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:40.321935Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:40.325385Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2525 sessionId: src_id|864b7119-abc9ab3-3e79fe29-28c8f9d_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:40.327727Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:40.327890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:40.327903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:40.327933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:40.328025Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:40.333860Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:40.333871Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:40.333873Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:40.333874Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:40.333885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343274518838654:7902]
2025-10-09T21:52:40.333887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343274518838654:7902]
2025-10-09T21:52:40.333890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343274518838654:7902]
2025-10-09T21:52:40.333922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:40.333925Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:40.333930Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:40.333931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.333932Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:40.333933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:40.334071Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:40.391156Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343274880692979:3994]
2025-10-09T21:52:40.391432Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 23 pipe creation result: OK TabletId: 72075186224037892 Generation: 2524, pipe: [1:7559343274880692979:3994]
2025-10-09T21:52:40.480962Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:40.487504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:40.487528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:40.487680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:40.487735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:40.487788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:40.487843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:40.488037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:40.488047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:40.488101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:40.488148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:40.488199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:40.488253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:40.488289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:40.488294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:40.488342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:40.488391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:40.488451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:40.488504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:40.488539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:40.488545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:40.488598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:40.488810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:40.488863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:40.488913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:40.488948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:40.488953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:40.489006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:40.489067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:40.489123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:40.489182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:40.489224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:40.489232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:40.489291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:40.489340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:40.489389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:40.489437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:40.489473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:40.489479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:40.489533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:40.489582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:40.489632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:40.489682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:40.489786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:40.489792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:40.489845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:40.489899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:40.489958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:40.490017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:40.490091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:40.490098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:40.490155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:40.490209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:40.490261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:40.490309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:40.490345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:40.490350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:40.490400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:40.490452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:40.490511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:40.490562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:40.490597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:40.490604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:40.490636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343274880692979:3994], now have 1 active actors on pipe
2025-10-09T21:52:40.490874Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:40.490888Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 23 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343274880692979:3994]
2025-10-09T21:52:40.490889Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:40.490904Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:40.490919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2525 reason: error at write: Disconnected sessionId: src_id|864b7119-abc9ab3-3e79fe29-28c8f9d_0
2025-10-09T21:52:40.491013Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:40.491134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2525 sessionId: src_id|864b7119-abc9ab3-3e79fe29-28c8f9d_0 is DEAD
2025-10-09T21:52:40.491251Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|864b7119-abc9ab3-3e79fe29-28c8f9d_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:40.491261Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|864b7119-abc9ab3-3e79fe29-28c8f9d_0] Write session will restart in 0.005159s
2025-10-09T21:52:40.491284Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|864b7119-abc9ab3-3e79fe29-28c8f9d_0] Write session: Do CDS request
2025-10-09T21:52:40.491288Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|864b7119-abc9ab3-3e79fe29-28c8f9d_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:40.497456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|864b7119-abc9ab3-3e79fe29-28c8f9d_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:40.497521Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:40.497534Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2526
2025-10-09T21:52:40.497638Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2526 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:40.497669Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2526 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:40.497680Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2526 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:40.497683Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:40.498239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2526 sessionId:  describe result for acl check
2025-10-09T21:52:40.498273Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:40.498285Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:40.498287Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:40.498292Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343274880693045:3504] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:40.498835Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343274880693045:3504] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:40.500351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:40.501106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343274880693045:3504] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:40.502182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:40.502341Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:40.502347Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:40.502356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:40.502392Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:40.502430Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343274518838804:7876]
2025-10-09T21:52:40.502824Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:40.503072Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:40.503115Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:40.503346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:40.503380Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.503386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.503387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.503387Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.503388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.503389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.503389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.503390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.503391Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.503394Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.503401Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.503403Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.503406Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.503408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.503410Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.503412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.503414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.503416Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.503420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.503425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.503429Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.503432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.503435Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.503438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.503441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.503444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.503447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.503450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.503452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.503455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.503466Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.503470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.503472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.503475Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.503478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.503481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.503498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:40.503872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343274880693045:3504] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:40.503883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343274880693045:3504] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:40.503887Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343274880693045:3504] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:40.503894Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2526 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:40.509558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:40.510549Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:40.510562Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:40.510565Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:40.510568Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:40.510576Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2525 [2:7559343274518838804:7876]
2025-10-09T21:52:40.510589Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:40.510604Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:40.510606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:40.510608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:40.510609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:40.510622Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:40.510630Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:40.510635Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:40.510676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.510680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.510682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.510683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:40.510733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:40.510739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.510745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.510752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.510761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:40.510774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343274518838799:7876]
2025-10-09T21:52:40.510779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343274518838799:7876]
2025-10-09T21:52:40.510780Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343274518838799:7876]
2025-10-09T21:52:40.510785Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:40.510794Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:40.510795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343274880693005:2432], now have 1 active actors on pipe
2025-10-09T21:52:40.510797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343274880693111:3504], now have 1 active actors on pipe
2025-10-09T21:52:40.510800Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:40.510802Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:40.510803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:40.510812Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:40.510813Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:40.510839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343274518838799:7876]
2025-10-09T21:52:40.510850Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:40.510854Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:40.510876Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2525
2025-10-09T21:52:40.510888Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2525
2025-10-09T21:52:40.510982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:40.510995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:40.511033Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|d49489d2-fed41c96-23f5c00e-730d760e_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:40.511048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:40.511057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:40.511059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:40.511070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:40.511086Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:40.511106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:40.511307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:40.511317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:40.511337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:40.511432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2526 partition: 0 MaxSeqNo: 25 sessionId: src_id|d49489d2-fed41c96-23f5c00e-730d760e_0
2025-10-09T21:52:40.511746Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|864b7119-abc9ab3-3e79fe29-28c8f9d_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|d49489d2-fed41c96-23f5c00e-730d760e_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:40.514217Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d49489d2-fed41c96-23f5c00e-730d760e_0] Write session: try to update token
2025-10-09T21:52:40.514235Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d49489d2-fed41c96-23f5c00e-730d760e_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:40.524558Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:40.524575Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:40.524577Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:40.524579Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:40.524589Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343274518838799:7876]
2025-10-09T21:52:40.524594Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343274518838799:7876]
2025-10-09T21:52:40.524598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343274518838799:7876]
2025-10-09T21:52:40.524637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:40.524642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.524643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:40.524666Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:40.524672Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:40.524676Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:40.524761Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:40.526845Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2526 sessionId: src_id|d49489d2-fed41c96-23f5c00e-730d760e_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:40.529301Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:40.590856Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343274880693119:3994]
2025-10-09T21:52:40.591082Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 24 pipe creation result: OK TabletId: 72075186224037892 Generation: 2525, pipe: [1:7559343274880693119:3994]
2025-10-09T21:52:40.668451Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:40.668869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:40.668883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:40.668924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343274880693119:3994], now have 1 active actors on pipe
2025-10-09T21:52:40.669090Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:40.669177Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:40.669191Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:40.669191Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 24 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343274880693119:3994]
2025-10-09T21:52:40.669212Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:40.669262Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2526 reason: error at write: Disconnected sessionId: src_id|d49489d2-fed41c96-23f5c00e-730d760e_0
2025-10-09T21:52:40.669440Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2526 sessionId: src_id|d49489d2-fed41c96-23f5c00e-730d760e_0 is DEAD
2025-10-09T21:52:40.669575Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d49489d2-fed41c96-23f5c00e-730d760e_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:40.669592Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d49489d2-fed41c96-23f5c00e-730d760e_0] Write session will restart in 0.009021s
2025-10-09T21:52:40.669611Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d49489d2-fed41c96-23f5c00e-730d760e_0] Write session: Do CDS request
2025-10-09T21:52:40.669615Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d49489d2-fed41c96-23f5c00e-730d760e_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:40.677508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:40.678895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:40.679042Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:40.679050Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:40.679060Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:40.679101Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:40.679133Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343274518838912:7880]
2025-10-09T21:52:40.679396Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|d49489d2-fed41c96-23f5c00e-730d760e_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:40.679491Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:40.679503Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2527
2025-10-09T21:52:40.679611Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2527 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:40.679608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:40.679636Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2527 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:40.679643Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2527 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:40.679646Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:40.679878Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:40.679917Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:40.680030Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2527 sessionId:  describe result for acl check
2025-10-09T21:52:40.680063Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:40.680070Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:40.680071Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:40.680077Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343274880693195:3493] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:40.680129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:40.680163Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.680168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.680168Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.680169Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.680170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.680170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.680171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.680171Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.680172Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.680175Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.680182Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.680184Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.680186Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.680188Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.680190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.680193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.680195Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.680197Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.680202Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.680207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.680211Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.680214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.680216Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.680219Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.680222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.680225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.680227Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.680230Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.680233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.680236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.680240Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.680243Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.680245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.680248Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.680251Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.680254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.680270Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:40.680600Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343274880693195:3493] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:40.682115Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343274880693195:3493] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:40.685832Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343274880693195:3493] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:40.685845Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343274880693195:3493] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:40.685848Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343274880693195:3493] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:40.685856Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2527 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:40.686247Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:40.687268Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:40.687281Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:40.687283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:40.687287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:40.687293Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2526 [2:7559343274518838912:7880]
2025-10-09T21:52:40.687305Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:40.687321Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:40.687328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:40.687331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:40.687333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:40.687348Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:40.687354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:40.687360Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:40.687413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.687420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.687422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.687425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:40.687486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:40.687493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.687504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.687519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.687542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:40.687565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343274880693232:3493], now have 1 active actors on pipe
2025-10-09T21:52:40.687573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343274880693121:2432], now have 1 active actors on pipe
2025-10-09T21:52:40.687579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343274518838907:7880]
2025-10-09T21:52:40.687582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343274518838907:7880]
2025-10-09T21:52:40.687584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343274518838907:7880]
2025-10-09T21:52:40.687563Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2526
2025-10-09T21:52:40.687575Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2526
2025-10-09T21:52:40.687596Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:40.687610Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:40.687613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:40.687619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:40.687621Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:40.687629Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:40.687632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:40.687662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:40.687670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:40.687686Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343274518838907:7880]
2025-10-09T21:52:40.687699Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:40.687710Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:40.687728Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|25eea548-50019fc6-d841bfec-c380c087_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:40.687736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:40.687740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:40.687742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:40.687751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:40.687764Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:40.687788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:40.687916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:40.687923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:40.687936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:40.687996Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2527 partition: 0 MaxSeqNo: 25 sessionId: src_id|25eea548-50019fc6-d841bfec-c380c087_0
2025-10-09T21:52:40.688237Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|d49489d2-fed41c96-23f5c00e-730d760e_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|25eea548-50019fc6-d841bfec-c380c087_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:40.690461Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25eea548-50019fc6-d841bfec-c380c087_0] Write session: try to update token
2025-10-09T21:52:40.690475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25eea548-50019fc6-d841bfec-c380c087_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:40.701358Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:40.701371Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:40.701373Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:40.701374Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:40.701382Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343274518838907:7880]
2025-10-09T21:52:40.701385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343274518838907:7880]
2025-10-09T21:52:40.701388Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343274518838907:7880]
2025-10-09T21:52:40.701419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:40.701423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.701425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:40.701427Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:40.701432Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:40.701435Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:40.701538Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:40.702140Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2527 sessionId: src_id|25eea548-50019fc6-d841bfec-c380c087_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:40.704363Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:40.769519Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343274880693239:3994]
2025-10-09T21:52:40.769742Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 25 pipe creation result: OK TabletId: 72075186224037892 Generation: 2526, pipe: [1:7559343274880693239:3994]
2025-10-09T21:52:40.842662Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:40.843040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:40.843051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:40.843089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343274880693239:3994], now have 1 active actors on pipe
2025-10-09T21:52:40.843318Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:40.843344Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:40.843348Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:40.843361Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 25 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343274880693239:3994]
2025-10-09T21:52:40.843427Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2527 reason: error at write: Disconnected sessionId: src_id|25eea548-50019fc6-d841bfec-c380c087_0
2025-10-09T21:52:40.843468Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:40.843595Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2527 sessionId: src_id|25eea548-50019fc6-d841bfec-c380c087_0 is DEAD
2025-10-09T21:52:40.843739Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25eea548-50019fc6-d841bfec-c380c087_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:40.843748Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25eea548-50019fc6-d841bfec-c380c087_0] Write session will restart in 0.006799s
2025-10-09T21:52:40.843768Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25eea548-50019fc6-d841bfec-c380c087_0] Write session: Do CDS request
2025-10-09T21:52:40.843776Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25eea548-50019fc6-d841bfec-c380c087_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:40.851434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|25eea548-50019fc6-d841bfec-c380c087_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:40.851493Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:40.851504Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2528
2025-10-09T21:52:40.851632Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2528 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:40.851664Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2528 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:40.851681Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2528 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:40.851689Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:40.852069Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2528 sessionId:  describe result for acl check
2025-10-09T21:52:40.852104Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:40.852106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:40.852108Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:40.852113Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343274880693313:3471] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:40.852166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:40.852670Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343274880693313:3471] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:40.853908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:40.854074Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:40.854083Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:40.854094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:40.854141Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:40.854171Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343274518839025:7877]
2025-10-09T21:52:40.854624Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:40.854752Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343274880693313:3471] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:40.854902Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:40.854941Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:40.855229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:40.855278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.855284Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.855285Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.855286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.855287Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.855288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.855289Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.855290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.855291Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.855295Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.855304Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.855308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.855312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.855320Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.855324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.855331Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.855335Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.855343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.855350Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.855361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:40.855367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.855374Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:40.855382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.855389Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:40.855395Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.855405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:40.855413Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.855418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:40.855426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.855430Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:40.855439Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.855447Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.855455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.855459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.855467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.855476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.855504Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:40.859878Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343274880693313:3471] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:40.859894Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343274880693313:3471] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:40.859898Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343274880693313:3471] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:40.859905Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2528 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:40.861148Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:40.862218Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:40.862229Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:40.862232Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:40.862236Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:40.862242Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2527 [2:7559343274518839025:7877]
2025-10-09T21:52:40.862256Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:40.862270Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:40.862278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:40.862281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:40.862284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:40.862299Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:40.862308Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:40.862316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:40.862373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.862380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.862383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.862385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:40.862447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:40.862456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.862467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:40.862479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:40.862496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:40.862519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343274880693353:3471], now have 1 active actors on pipe
2025-10-09T21:52:40.862527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343274880693240:2432], now have 1 active actors on pipe
2025-10-09T21:52:40.862534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343274518839020:7877]
2025-10-09T21:52:40.862541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343274518839020:7877]
2025-10-09T21:52:40.862543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343274518839020:7877]
2025-10-09T21:52:40.862554Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:40.862568Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:40.862571Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:40.862576Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:40.862578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:40.862560Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2527
2025-10-09T21:52:40.862570Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2527
2025-10-09T21:52:40.862585Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:40.862592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:40.862627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:40.862636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:40.862651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343274518839020:7877]
2025-10-09T21:52:40.862661Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:40.862666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:40.862690Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|b35ef5ba-aa1f8091-c17f463c-9777feca_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:40.862702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:40.862706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:40.862708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:40.862716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:40.862727Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:40.862752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:40.862864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:40.862873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:40.862890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:40.862960Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2528 partition: 0 MaxSeqNo: 25 sessionId: src_id|b35ef5ba-aa1f8091-c17f463c-9777feca_0
2025-10-09T21:52:40.863239Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|25eea548-50019fc6-d841bfec-c380c087_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|b35ef5ba-aa1f8091-c17f463c-9777feca_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:40.865444Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b35ef5ba-aa1f8091-c17f463c-9777feca_0] Write session: try to update token
2025-10-09T21:52:40.865457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b35ef5ba-aa1f8091-c17f463c-9777feca_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:40.875728Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:40.875748Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:40.875750Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:40.875752Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:40.875761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343274518839020:7877]
2025-10-09T21:52:40.875766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343274518839020:7877]
2025-10-09T21:52:40.875770Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343274518839020:7877]
2025-10-09T21:52:40.875810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:40.875811Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:40.875816Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:40.875818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:40.875819Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:40.875821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:40.875961Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:40.877230Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2528 sessionId: src_id|b35ef5ba-aa1f8091-c17f463c-9777feca_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:40.879416Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:40.943604Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343274880693362:3994]
2025-10-09T21:52:40.943830Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 26 pipe creation result: OK TabletId: 72075186224037892 Generation: 2527, pipe: [1:7559343274880693362:3994]
2025-10-09T21:52:41.022263Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:41.022621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.022636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.022674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343274880693362:3994], now have 1 active actors on pipe
2025-10-09T21:52:41.022845Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:41.022892Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:41.022905Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:41.022904Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 26 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343274880693362:3994]
2025-10-09T21:52:41.022925Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:41.022967Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2528 reason: error at write: Disconnected sessionId: src_id|b35ef5ba-aa1f8091-c17f463c-9777feca_0
2025-10-09T21:52:41.023121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2528 sessionId: src_id|b35ef5ba-aa1f8091-c17f463c-9777feca_0 is DEAD
2025-10-09T21:52:41.023286Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b35ef5ba-aa1f8091-c17f463c-9777feca_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:41.023309Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b35ef5ba-aa1f8091-c17f463c-9777feca_0] Write session will restart in 0.006214s
2025-10-09T21:52:41.023330Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b35ef5ba-aa1f8091-c17f463c-9777feca_0] Write session: Do CDS request
2025-10-09T21:52:41.023335Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b35ef5ba-aa1f8091-c17f463c-9777feca_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:41.030508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|b35ef5ba-aa1f8091-c17f463c-9777feca_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:41.030551Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:41.030564Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2529
2025-10-09T21:52:41.030742Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2529 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:41.030783Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2529 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:41.030794Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2529 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:41.030798Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:41.031279Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2529 sessionId:  describe result for acl check
2025-10-09T21:52:41.031316Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:41.031323Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:41.031326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:41.031331Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343279175660720:3467] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:41.031836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343279175660720:3467] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:41.032905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:41.033977Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343279175660720:3467] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:41.034697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:41.034878Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:41.034887Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:41.034898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:41.034951Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:41.034993Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343278813806434:7866]
2025-10-09T21:52:41.035569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:41.035794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:41.035835Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:41.036143Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:41.036193Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.036200Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.036201Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.036203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.036205Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.036206Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.036207Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.036208Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.036209Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.036214Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.036225Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.036229Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.036233Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.036237Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.036268Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343279175660720:3467] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:41.036241Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.036282Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343279175660720:3467] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:41.036245Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.036249Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.036253Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.036259Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.036266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.036272Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.036278Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.036283Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.036288Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.036292Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.036286Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343279175660720:3467] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:41.036298Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.036302Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.036308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.036312Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.036317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.036322Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.036328Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.036332Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.036305Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2529 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:41.036338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.036342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.036348Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.036371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:41.042862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:41.043996Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:41.044012Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:41.044015Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:41.044020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:41.044027Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2528 [2:7559343278813806434:7866]
2025-10-09T21:52:41.044044Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:41.044063Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:41.044070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:41.044073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:41.044076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:41.044093Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:41.044103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:41.044113Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:41.044176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.044184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.044187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.044190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:41.044259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:41.044266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.044279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.044296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.044317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:41.044344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175660661:2432], now have 1 active actors on pipe
2025-10-09T21:52:41.044357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343278813806428:7866]
2025-10-09T21:52:41.044364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343278813806428:7866]
2025-10-09T21:52:41.044366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343278813806428:7866]
2025-10-09T21:52:41.044375Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:41.044385Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:41.044392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175660771:3467], now have 1 active actors on pipe
2025-10-09T21:52:41.044393Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:41.044395Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:41.044398Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:41.044381Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2528
2025-10-09T21:52:41.044411Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:41.044405Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2528
2025-10-09T21:52:41.044414Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:41.044457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343278813806428:7866]
2025-10-09T21:52:41.044470Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:41.044482Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:41.044486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.044491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.044657Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8c02adbf-7ade069a-5015cadc-9040fa06_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:41.044670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:41.044676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:41.044678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:41.044689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:41.044706Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:41.044740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:41.044908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.044917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.044935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:41.045046Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2529 partition: 0 MaxSeqNo: 25 sessionId: src_id|8c02adbf-7ade069a-5015cadc-9040fa06_0
2025-10-09T21:52:41.045379Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|b35ef5ba-aa1f8091-c17f463c-9777feca_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|8c02adbf-7ade069a-5015cadc-9040fa06_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:41.047834Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c02adbf-7ade069a-5015cadc-9040fa06_0] Write session: try to update token
2025-10-09T21:52:41.047853Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c02adbf-7ade069a-5015cadc-9040fa06_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:41.058745Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:41.058764Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:41.058767Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:41.058769Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:41.058778Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343278813806428:7866]
2025-10-09T21:52:41.058785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343278813806428:7866]
2025-10-09T21:52:41.058788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343278813806428:7866]
2025-10-09T21:52:41.058827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:41.058833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.058834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:41.058834Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:41.058841Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:41.058845Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:41.058971Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:41.060719Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2529 sessionId: src_id|8c02adbf-7ade069a-5015cadc-9040fa06_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:41.063263Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:41.122810Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343279175660780:3994]
2025-10-09T21:52:41.123104Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 27 pipe creation result: OK TabletId: 72075186224037892 Generation: 2528, pipe: [1:7559343279175660780:3994]
2025-10-09T21:52:41.206397Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:41.206757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.206771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.206822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175660780:3994], now have 1 active actors on pipe
2025-10-09T21:52:41.207046Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:41.207125Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:41.207149Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 27 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343279175660780:3994]
2025-10-09T21:52:41.207176Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:41.207239Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2529 reason: status is not ok: tablet will be restarted right now sessionId: src_id|8c02adbf-7ade069a-5015cadc-9040fa06_0
2025-10-09T21:52:41.207411Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2529 sessionId: src_id|8c02adbf-7ade069a-5015cadc-9040fa06_0 is DEAD
2025-10-09T21:52:41.207544Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c02adbf-7ade069a-5015cadc-9040fa06_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:41.207555Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c02adbf-7ade069a-5015cadc-9040fa06_0] Write session will restart in 0.005267s
2025-10-09T21:52:41.207575Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c02adbf-7ade069a-5015cadc-9040fa06_0] Write session: Do CDS request
2025-10-09T21:52:41.207580Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c02adbf-7ade069a-5015cadc-9040fa06_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:41.213450Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|8c02adbf-7ade069a-5015cadc-9040fa06_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:41.213595Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:41.213608Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2530
2025-10-09T21:52:41.213725Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2530 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:41.213759Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2530 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:41.213765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2530 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:41.213769Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:41.214206Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2530 sessionId:  describe result for acl check
2025-10-09T21:52:41.214243Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:41.214249Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:41.214251Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:41.214256Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343279175660803:3445] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:41.214703Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343279175660803:3445] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:41.217018Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343279175660803:3445] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:41.217339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:41.219251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:41.219408Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:41.219417Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:41.219426Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:41.219464Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:41.219471Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343279175660803:3445] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:41.219496Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343278813806545:7860]
2025-10-09T21:52:41.219482Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343279175660803:3445] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:41.219485Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343279175660803:3445] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:41.219490Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2530 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:41.219856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:41.220083Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:41.220109Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:41.220308Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:41.220338Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.220342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.220342Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.220343Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.220344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.220344Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.220345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.220345Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.220346Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.220349Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.220356Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.220359Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.220361Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.220363Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.220365Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.220367Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.220369Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.220371Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.220375Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.220381Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.220384Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.220388Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.220390Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.220393Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.220396Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.220399Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.220402Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.220405Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.220407Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.220411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.220414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.220418Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.220420Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.220423Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.220425Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.220428Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.220444Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:41.226808Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:41.228006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:41.228021Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:41.228024Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:41.228028Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:41.228036Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2529 [2:7559343278813806545:7860]
2025-10-09T21:52:41.228049Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:41.228069Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:41.228077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:41.228081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:41.228083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:41.228100Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:41.228105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:41.228110Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:41.228151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.228157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.228158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.228160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:41.228212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:41.228218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.228226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.228244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.228259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:41.228281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175660891:3445], now have 1 active actors on pipe
2025-10-09T21:52:41.228293Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343278813806539:7860]
2025-10-09T21:52:41.228295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343278813806539:7860]
2025-10-09T21:52:41.228299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343278813806539:7860]
2025-10-09T21:52:41.228311Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:41.228325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:41.228327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175660783:2432], now have 1 active actors on pipe
2025-10-09T21:52:41.228329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:41.228332Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:41.228335Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:41.228350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:41.228354Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:41.228368Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2529
2025-10-09T21:52:41.228386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343278813806539:7860]
2025-10-09T21:52:41.228373Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2529
2025-10-09T21:52:41.228409Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:41.228410Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:41.228441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.228451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.228490Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e499e055-988d7ad9-5f58d80a-26faddba_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:41.228505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:41.228510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:41.228513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:41.228524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:41.228534Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:41.228560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:41.228798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.228810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.228826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:41.228924Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2530 partition: 0 MaxSeqNo: 25 sessionId: src_id|e499e055-988d7ad9-5f58d80a-26faddba_0
2025-10-09T21:52:41.229276Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|8c02adbf-7ade069a-5015cadc-9040fa06_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e499e055-988d7ad9-5f58d80a-26faddba_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:41.231558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e499e055-988d7ad9-5f58d80a-26faddba_0] Write session: try to update token
2025-10-09T21:52:41.231574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e499e055-988d7ad9-5f58d80a-26faddba_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:41.243127Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:41.243142Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:41.243145Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:41.243147Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:41.243158Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343278813806539:7860]
2025-10-09T21:52:41.243163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343278813806539:7860]
2025-10-09T21:52:41.243167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343278813806539:7860]
2025-10-09T21:52:41.243211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:41.243214Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:41.243219Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:41.243224Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:41.243224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.243227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:41.243360Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:41.243675Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2530 sessionId: src_id|e499e055-988d7ad9-5f58d80a-26faddba_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:41.245934Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:41.307136Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343279175660905:3994]
2025-10-09T21:52:41.307391Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 28 pipe creation result: OK TabletId: 72075186224037892 Generation: 2529, pipe: [1:7559343279175660905:3994]
2025-10-09T21:52:41.387877Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:41.388139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.388150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.388186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175660905:3994], now have 1 active actors on pipe
2025-10-09T21:52:41.388335Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:41.388429Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:41.388431Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:41.388473Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 28 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343279175660905:3994]
2025-10-09T21:52:41.388516Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2530 reason: status is not ok: tablet will be restarted right now sessionId: src_id|e499e055-988d7ad9-5f58d80a-26faddba_0
2025-10-09T21:52:41.388686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2530 sessionId: src_id|e499e055-988d7ad9-5f58d80a-26faddba_0 is DEAD
2025-10-09T21:52:41.388830Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e499e055-988d7ad9-5f58d80a-26faddba_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:41.388843Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e499e055-988d7ad9-5f58d80a-26faddba_0] Write session will restart in 0.009087s
2025-10-09T21:52:41.388867Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e499e055-988d7ad9-5f58d80a-26faddba_0] Write session: Do CDS request
2025-10-09T21:52:41.388873Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e499e055-988d7ad9-5f58d80a-26faddba_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:41.397859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:41.399387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:41.399420Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e499e055-988d7ad9-5f58d80a-26faddba_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:41.399542Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:41.399572Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:41.399555Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2531
2025-10-09T21:52:41.399579Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:41.399591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:41.399651Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:41.399669Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2531 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:41.399697Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2531 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:41.399706Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2531 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:41.399691Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343278813806657:7858]
2025-10-09T21:52:41.399710Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:41.400127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2531 sessionId:  describe result for acl check
2025-10-09T21:52:41.400159Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:41.400165Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:41.400167Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:41.400176Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343279175660988:3423] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:41.400280Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:41.400545Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:41.400614Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:41.400712Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343279175660988:3423] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:41.400847Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:41.400887Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.400893Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.400894Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.400895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.400896Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.400897Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.400898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.400899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.400901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.400906Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.400916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.400920Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.400924Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.400932Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.400934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.400936Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.400938Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.400940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.400946Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.400954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.400965Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.400973Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.400982Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.400988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.400991Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.400994Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.400998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.401001Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.401008Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.401011Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.401014Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.401017Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.401020Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.401023Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.401025Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.401029Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.401046Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:41.402682Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343279175660988:3423] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:41.406909Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343279175660988:3423] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:41.406923Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343279175660988:3423] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:41.406927Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343279175660988:3423] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:41.406936Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2531 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:41.407357Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:41.408421Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:41.408438Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:41.408441Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:41.408446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:41.408453Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2530 [2:7559343278813806657:7858]
2025-10-09T21:52:41.408467Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:41.408486Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:41.408489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:41.408492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:41.408495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:41.408514Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:41.408525Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:41.408532Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:41.408593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.408600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.408602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.408605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:41.408661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:41.408669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.408680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.408691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.408707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:41.408732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175661027:3423], now have 1 active actors on pipe
2025-10-09T21:52:41.408741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175660913:2432], now have 1 active actors on pipe
2025-10-09T21:52:41.408748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343278813806652:7858]
2025-10-09T21:52:41.408751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343278813806652:7858]
2025-10-09T21:52:41.408753Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343278813806652:7858]
2025-10-09T21:52:41.408769Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:41.408784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:41.408792Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:41.408798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:41.408801Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:41.408783Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2530
2025-10-09T21:52:41.408811Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:41.408791Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2530
2025-10-09T21:52:41.408819Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:41.408832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343278813806652:7858]
2025-10-09T21:52:41.408837Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:41.408848Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:41.408877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.408887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.408918Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|f6c101-ad6be188-14083c4b-ed98fc3a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:41.408930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:41.408938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:41.408940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:41.408952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:41.408966Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:41.408987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:41.409117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.409122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.409134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:41.409204Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2531 partition: 0 MaxSeqNo: 25 sessionId: src_id|f6c101-ad6be188-14083c4b-ed98fc3a_0
2025-10-09T21:52:41.409526Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e499e055-988d7ad9-5f58d80a-26faddba_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|f6c101-ad6be188-14083c4b-ed98fc3a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:41.412120Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f6c101-ad6be188-14083c4b-ed98fc3a_0] Write session: try to update token
2025-10-09T21:52:41.412137Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f6c101-ad6be188-14083c4b-ed98fc3a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:41.424766Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2531 sessionId: src_id|f6c101-ad6be188-14083c4b-ed98fc3a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:41.425695Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 1 from KV. Status 1
2025-10-09T21:52:41.425710Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:41.425719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 3 size 8295635 actorID [2:7559343278813806652:7858]
2025-10-09T21:52:41.425765Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 3 parts 15 suffix '0' size 8295635
2025-10-09T21:52:41.425773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:52:41.425942Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512009 from pos 0 cbcount 1
2025-10-09T21:52:41.426108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:41.426231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:41.426358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 0 size 512005 from pos 0 cbcount 1
2025-10-09T21:52:41.426526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 3 count 1 size 717094 from pos 0 cbcount 2
2025-10-09T21:52:41.426561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171624 queuesize 0 startOffset 0
2025-10-09T21:52:41.427078Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:41.427261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.427275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.427306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:52:41.427388Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:41.433508Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:41.433523Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:41.433526Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:41.433528Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:41.433541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343278813806652:7858]
2025-10-09T21:52:41.433545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343278813806652:7858]
2025-10-09T21:52:41.433549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343278813806652:7858]
2025-10-09T21:52:41.433586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:41.433594Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:41.433598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.433600Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:41.433601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:41.433603Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:41.433732Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:41.488367Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343279175661055:3994]
2025-10-09T21:52:41.488503Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 29 pipe creation result: OK TabletId: 72075186224037892 Generation: 2530, pipe: [1:7559343279175661055:3994]
2025-10-09T21:52:41.579254Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:41.587073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.587099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.587270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:41.587341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:41.587406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:41.587486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:41.587710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:41.587725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:41.587796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:41.587861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:41.587924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:41.588302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:41.588557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:41.588566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:41.588628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:41.588696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:41.588759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:41.588818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:41.588865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:41.588868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:41.588932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:41.588999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:41.589059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:41.589122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:41.589168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:41.589174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:41.589253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:41.589316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:41.589375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:41.589435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:41.589476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:41.589483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:41.589554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:41.589616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:41.589679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:41.589737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:41.589782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:41.589789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:41.589850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:41.589904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:41.589965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:41.590023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:41.590079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:41.590086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:41.590147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:41.590202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:41.590259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:41.590315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:41.590355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:41.590362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:41.590424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:41.590476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:41.590534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:41.590597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:41.590710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:41.590719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:41.590781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 1 size: 511959
2025-10-09T21:52:41.590836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 1 messageNo: 1 size: 511959
2025-10-09T21:52:41.590896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 2 messageNo: 1 size: 511959
2025-10-09T21:52:41.590955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 3 messageNo: 1 size: 511959
2025-10-09T21:52:41.590994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1973: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size: 205040
2025-10-09T21:52:41.590997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 4 messageNo: 1 size 205040 offset: -1
2025-10-09T21:52:41.591055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175661055:3994], now have 1 active actors on pipe
2025-10-09T21:52:41.591277Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:41.591291Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:41.591320Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2531 reason: error at write: Disconnected sessionId: src_id|f6c101-ad6be188-14083c4b-ed98fc3a_0
2025-10-09T21:52:41.591446Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:41.591466Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:41.591495Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 29 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343279175661055:3994]
2025-10-09T21:52:41.591511Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2531 sessionId: src_id|f6c101-ad6be188-14083c4b-ed98fc3a_0 is DEAD
2025-10-09T21:52:41.591619Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6c101-ad6be188-14083c4b-ed98fc3a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:41.591631Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6c101-ad6be188-14083c4b-ed98fc3a_0] Write session will restart in 0.007118s
2025-10-09T21:52:41.591656Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6c101-ad6be188-14083c4b-ed98fc3a_0] Write session: Do CDS request
2025-10-09T21:52:41.591661Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6c101-ad6be188-14083c4b-ed98fc3a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:41.599481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|f6c101-ad6be188-14083c4b-ed98fc3a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:41.599580Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:41.599594Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2532
2025-10-09T21:52:41.599696Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2532 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:41.599726Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2532 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:41.599738Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2532 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:41.599741Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:41.600157Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2532 sessionId:  describe result for acl check
2025-10-09T21:52:41.600193Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:41.600198Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:41.600200Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:41.600205Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343279175661134:3408] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:41.600258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:41.600748Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343279175661134:3408] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:41.602211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:41.602387Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:41.602397Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:41.602409Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:41.602461Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:41.602498Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343278813806807:7832]
2025-10-09T21:52:41.603063Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:41.603139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343279175661134:3408] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:41.603451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:41.603498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:41.603794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:41.603852Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.603858Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.603859Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.603860Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.603861Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.603863Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.603864Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.603865Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.603866Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.603872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.603882Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.603886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.603889Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.603892Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.603895Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.603898Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.603901Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.603904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.603910Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.603916Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.603922Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.603926Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.603930Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.603935Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.603939Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.603944Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.603948Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.603953Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.603957Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.603962Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.603970Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.603975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.603979Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.603988Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.603992Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.603998Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.604030Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:41.608695Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343279175661134:3408] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:41.608709Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343279175661134:3408] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:41.608714Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343279175661134:3408] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:41.608722Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2532 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:41.609915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:41.610954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:41.610969Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:41.610971Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:41.610975Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:41.610981Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2531 [2:7559343278813806807:7832]
2025-10-09T21:52:41.610994Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:41.611012Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:41.611020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:41.611022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:41.611024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:41.611040Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:41.611048Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:41.611055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:41.611110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.611116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.611118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.611121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:41.611187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:41.611195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.611207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.611219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.611236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:41.611256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175661173:3408], now have 1 active actors on pipe
2025-10-09T21:52:41.611263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175661059:2432], now have 1 active actors on pipe
2025-10-09T21:52:41.611271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343278813806802:7832]
2025-10-09T21:52:41.611273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343278813806802:7832]
2025-10-09T21:52:41.611275Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343278813806802:7832]
2025-10-09T21:52:41.611288Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:41.611301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:41.611312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:41.611297Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2531
2025-10-09T21:52:41.611298Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2531
2025-10-09T21:52:41.611315Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:41.611319Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:41.611320Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:41.611322Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:41.611343Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343278813806802:7832]
2025-10-09T21:52:41.611346Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:41.611347Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:41.611365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.611370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.611418Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|7f914938-7d6ac0d0-e9d8012-8b1706a3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:41.611429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:41.611437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:41.611439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:41.611449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:41.611462Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:41.611484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:41.611633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.611642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.611657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:41.611745Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2532 partition: 0 MaxSeqNo: 25 sessionId: src_id|7f914938-7d6ac0d0-e9d8012-8b1706a3_0
2025-10-09T21:52:41.612054Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|f6c101-ad6be188-14083c4b-ed98fc3a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|7f914938-7d6ac0d0-e9d8012-8b1706a3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:41.614423Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7f914938-7d6ac0d0-e9d8012-8b1706a3_0] Write session: try to update token
2025-10-09T21:52:41.614441Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7f914938-7d6ac0d0-e9d8012-8b1706a3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:41.625048Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:41.625068Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:41.625070Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:41.625072Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:41.625083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343278813806802:7832]
2025-10-09T21:52:41.625088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343278813806802:7832]
2025-10-09T21:52:41.625093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343278813806802:7832]
2025-10-09T21:52:41.625123Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:41.625128Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:41.625130Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:41.625136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:41.625144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.625147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:41.625299Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:41.627060Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2532 sessionId: src_id|7f914938-7d6ac0d0-e9d8012-8b1706a3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:41.629286Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:41.691646Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343279175661182:3994]
2025-10-09T21:52:41.691925Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 30 pipe creation result: OK TabletId: 72075186224037892 Generation: 2531, pipe: [1:7559343279175661182:3994]
2025-10-09T21:52:41.769644Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:41.769991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.770005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.770062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175661182:3994], now have 1 active actors on pipe
2025-10-09T21:52:41.770215Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:41.770300Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:41.770311Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 30 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343279175661182:3994]
2025-10-09T21:52:41.770335Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:41.770419Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2532 reason: status is not ok: tablet will be restarted right now sessionId: src_id|7f914938-7d6ac0d0-e9d8012-8b1706a3_0
2025-10-09T21:52:41.770599Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2532 sessionId: src_id|7f914938-7d6ac0d0-e9d8012-8b1706a3_0 is DEAD
2025-10-09T21:52:41.770760Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f914938-7d6ac0d0-e9d8012-8b1706a3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:41.770772Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f914938-7d6ac0d0-e9d8012-8b1706a3_0] Write session will restart in 0.008663s
2025-10-09T21:52:41.770801Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f914938-7d6ac0d0-e9d8012-8b1706a3_0] Write session: Do CDS request
2025-10-09T21:52:41.770807Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f914938-7d6ac0d0-e9d8012-8b1706a3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:41.779571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:41.780418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|7f914938-7d6ac0d0-e9d8012-8b1706a3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:41.780445Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:41.780457Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2533
2025-10-09T21:52:41.780581Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2533 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:41.780616Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2533 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:41.780627Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2533 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:41.780634Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:41.781011Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2533 sessionId:  describe result for acl check
2025-10-09T21:52:41.781039Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:41.781040Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:41.781041Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:41.781045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343279175661264:3390] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:41.781128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:41.781308Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:41.781317Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:41.781329Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:41.781382Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:41.781420Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343278813806922:7836]
2025-10-09T21:52:41.781583Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343279175661264:3390] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:41.781972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:41.782228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:41.782274Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:41.782582Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:41.782631Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.782636Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.782637Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.782639Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.782640Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.782641Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.782642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.782643Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.782644Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.782649Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.782658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.782662Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.782667Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.782671Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.782676Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.782679Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.782683Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.782687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.782693Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.782700Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.782706Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.782712Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.782718Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.782725Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.782729Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.782734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.782739Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.782745Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.782749Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.782754Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.782760Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.782765Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.782770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.782775Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.782780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.782785Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.782817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:41.783365Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343279175661264:3390] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:41.788263Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343279175661264:3390] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:41.788275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343279175661264:3390] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:41.788278Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343279175661264:3390] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:41.788285Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2533 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:41.788615Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:41.789770Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:41.789785Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:41.789787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:41.789791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:41.789797Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2532 [2:7559343278813806922:7836]
2025-10-09T21:52:41.789809Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:41.789826Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:41.789833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:41.789836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:41.789839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:41.789854Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:41.789864Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:41.789872Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:41.789925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.789931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.789933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.789936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:41.789998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:41.790005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.790016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.790044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.790066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:41.790089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175661184:2432], now have 1 active actors on pipe
2025-10-09T21:52:41.790095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175661303:3390], now have 1 active actors on pipe
2025-10-09T21:52:41.790103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343278813806917:7836]
2025-10-09T21:52:41.790106Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343278813806917:7836]
2025-10-09T21:52:41.790108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343278813806917:7836]
2025-10-09T21:52:41.790121Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:41.790130Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:41.790133Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:41.790135Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:41.790137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:41.790152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:41.790158Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:41.790140Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2532
2025-10-09T21:52:41.790154Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2532
2025-10-09T21:52:41.790181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343278813806917:7836]
2025-10-09T21:52:41.790191Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:41.790196Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:41.790207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.790212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.790260Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9417bae5-b585506a-fdcbf60d-9f6ff68f_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:41.790271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:41.790275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:41.790281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:41.790291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:41.790304Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:41.790327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:41.790468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.790476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.790491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:41.790563Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2533 partition: 0 MaxSeqNo: 25 sessionId: src_id|9417bae5-b585506a-fdcbf60d-9f6ff68f_0
2025-10-09T21:52:41.790802Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|7f914938-7d6ac0d0-e9d8012-8b1706a3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9417bae5-b585506a-fdcbf60d-9f6ff68f_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:41.793115Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9417bae5-b585506a-fdcbf60d-9f6ff68f_0] Write session: try to update token
2025-10-09T21:52:41.793128Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9417bae5-b585506a-fdcbf60d-9f6ff68f_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:41.804072Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:41.804088Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:41.804090Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:41.804091Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:41.804100Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343278813806917:7836]
2025-10-09T21:52:41.804104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343278813806917:7836]
2025-10-09T21:52:41.804107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343278813806917:7836]
2025-10-09T21:52:41.804141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:41.804148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.804150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:41.804156Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:41.804167Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:41.804170Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:41.804287Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:41.805315Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2533 sessionId: src_id|9417bae5-b585506a-fdcbf60d-9f6ff68f_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:41.807831Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:41.870329Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343279175661311:3994]
2025-10-09T21:52:41.870562Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 31 pipe creation result: OK TabletId: 72075186224037892 Generation: 2532, pipe: [1:7559343279175661311:3994]
2025-10-09T21:52:41.951130Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:41.951429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.951449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.951495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175661311:3994], now have 1 active actors on pipe
2025-10-09T21:52:41.951654Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:41.951678Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:41.951701Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 31 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343279175661311:3994]
2025-10-09T21:52:41.951727Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:41.951813Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2533 reason: status is not ok: tablet will be restarted right now sessionId: src_id|9417bae5-b585506a-fdcbf60d-9f6ff68f_0
2025-10-09T21:52:41.951998Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2533 sessionId: src_id|9417bae5-b585506a-fdcbf60d-9f6ff68f_0 is DEAD
2025-10-09T21:52:41.952151Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9417bae5-b585506a-fdcbf60d-9f6ff68f_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:41.952160Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9417bae5-b585506a-fdcbf60d-9f6ff68f_0] Write session will restart in 0.009155s
2025-10-09T21:52:41.952181Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9417bae5-b585506a-fdcbf60d-9f6ff68f_0] Write session: Do CDS request
2025-10-09T21:52:41.952186Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9417bae5-b585506a-fdcbf60d-9f6ff68f_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:41.961437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:41.962483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9417bae5-b585506a-fdcbf60d-9f6ff68f_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:41.962585Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:41.962603Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2534
2025-10-09T21:52:41.962746Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2534 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:41.962778Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2534 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:41.962789Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2534 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:41.962793Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:41.963237Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2534 sessionId:  describe result for acl check
2025-10-09T21:52:41.963269Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:41.963275Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:41.963277Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:41.963283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343279175661396:3367] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:41.963286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:41.963458Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:41.963467Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:41.963478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:41.963526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:41.963568Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343278813807041:7826]
2025-10-09T21:52:41.963836Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343279175661396:3367] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:41.964065Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:41.964293Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:41.964324Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:41.964534Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:41.964569Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.964570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.964571Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.964572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.964572Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.964573Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.964574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.964574Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.964575Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.964578Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.964584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.964586Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.964589Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.964591Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.964593Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.964597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.964601Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.964605Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.964611Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.964618Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:41.964628Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.964633Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:41.964638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.964648Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:41.964655Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.964660Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:41.964670Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.964680Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:41.964686Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.964696Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:41.964705Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.964713Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.964721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.964724Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.964731Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.964734Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.964750Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:41.965732Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343279175661396:3367] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:41.970595Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343279175661396:3367] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:41.970605Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343279175661396:3367] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:41.970609Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343279175661396:3367] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:41.970615Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2534 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:41.971003Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:41.972039Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:41.972053Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:41.972055Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:41.972059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:41.972065Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2533 [2:7559343278813807041:7826]
2025-10-09T21:52:41.972080Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:41.972099Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:41.972107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:41.972110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:41.972112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:41.972129Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:41.972140Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:41.972153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:41.972204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.972210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.972212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.972215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:41.972280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:41.972288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.972299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:41.972315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:41.972337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:41.972362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175661432:3367], now have 1 active actors on pipe
2025-10-09T21:52:41.972365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343279175661318:2432], now have 1 active actors on pipe
2025-10-09T21:52:41.972372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343278813807036:7826]
2025-10-09T21:52:41.972375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343278813807036:7826]
2025-10-09T21:52:41.972377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343278813807036:7826]
2025-10-09T21:52:41.972391Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:41.972378Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2533
2025-10-09T21:52:41.972379Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2533
2025-10-09T21:52:41.972409Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:41.972411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:41.972414Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:41.972417Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:41.972428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:41.972435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:41.972467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.972476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.972492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343278813807036:7826]
2025-10-09T21:52:41.972505Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:41.972519Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:41.972532Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e9cd47ea-3fc1926f-4eb2f744-a14cfac6_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:41.972544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:41.972552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:41.972554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:41.972568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:41.972582Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:41.972718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:41.972867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:41.972875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:41.972891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:41.972964Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2534 partition: 0 MaxSeqNo: 25 sessionId: src_id|e9cd47ea-3fc1926f-4eb2f744-a14cfac6_0
2025-10-09T21:52:41.973266Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9417bae5-b585506a-fdcbf60d-9f6ff68f_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|e9cd47ea-3fc1926f-4eb2f744-a14cfac6_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:41.975560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9cd47ea-3fc1926f-4eb2f744-a14cfac6_0] Write session: try to update token
2025-10-09T21:52:41.975577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9cd47ea-3fc1926f-4eb2f744-a14cfac6_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:41.986921Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:41.986937Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:41.986940Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:41.986942Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:41.986951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343278813807036:7826]
2025-10-09T21:52:41.986956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343278813807036:7826]
2025-10-09T21:52:41.986961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343278813807036:7826]
2025-10-09T21:52:41.987003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:41.987007Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:41.987014Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:41.987015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:41.987017Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:41.987018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:41.987145Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:41.987429Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2534 sessionId: src_id|e9cd47ea-3fc1926f-4eb2f744-a14cfac6_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:41.989775Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:42.052087Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343283470628737:3994]
2025-10-09T21:52:42.052338Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 32 pipe creation result: OK TabletId: 72075186224037892 Generation: 2533, pipe: [1:7559343283470628737:3994]
2025-10-09T21:52:42.133757Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:42.133998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:42.134010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:42.134062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343283470628737:3994], now have 1 active actors on pipe
2025-10-09T21:52:42.134312Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:42.134298Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:42.134298Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:42.134302Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:42.134358Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 32 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343283470628737:3994]
2025-10-09T21:52:42.134365Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2534 reason: error at write: Disconnected sessionId: src_id|e9cd47ea-3fc1926f-4eb2f744-a14cfac6_0
2025-10-09T21:52:42.134540Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2534 sessionId: src_id|e9cd47ea-3fc1926f-4eb2f744-a14cfac6_0 is DEAD
2025-10-09T21:52:42.134664Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9cd47ea-3fc1926f-4eb2f744-a14cfac6_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:42.134673Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9cd47ea-3fc1926f-4eb2f744-a14cfac6_0] Write session will restart in 0.006361s
2025-10-09T21:52:42.134695Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9cd47ea-3fc1926f-4eb2f744-a14cfac6_0] Write session: Do CDS request
2025-10-09T21:52:42.134699Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9cd47ea-3fc1926f-4eb2f744-a14cfac6_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:42.142495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|e9cd47ea-3fc1926f-4eb2f744-a14cfac6_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:42.142624Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:42.142634Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2535
2025-10-09T21:52:42.142747Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2535 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:42.142772Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2535 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:42.142775Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2535 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:42.142779Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:42.143107Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2535 sessionId:  describe result for acl check
2025-10-09T21:52:42.143139Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:42.143140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:42.143142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:42.143148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343283470628816:3355] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:42.143572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:42.143671Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343283470628816:3355] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:42.145155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:42.145339Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:42.145348Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:42.145360Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:42.145411Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:42.145446Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343283108774454:7824]
2025-10-09T21:52:42.145486Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343283470628816:3355] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:42.146056Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:42.146372Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:42.146414Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:42.146730Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:42.146780Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:42.146786Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:42.146787Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:42.146788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:42.146790Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:42.146791Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:42.146792Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.146793Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.146794Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.146799Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:42.146809Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:42.146813Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:42.146817Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:42.146821Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:42.146825Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:42.146829Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.146833Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.146837Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.146843Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:42.146850Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:42.146856Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:42.146862Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:42.146872Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:42.146877Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:42.146886Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:42.146890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:42.146899Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:42.146905Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:42.146915Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:42.146925Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:42.146934Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.146940Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.146945Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.146954Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.146961Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.146972Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.147006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:42.151870Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343283470628816:3355] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:42.151886Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343283470628816:3355] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:42.151890Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343283470628816:3355] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:42.151899Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2535 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:42.153187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:42.154269Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:42.154283Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:42.154286Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:42.154290Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:42.154297Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2534 [2:7559343283108774454:7824]
2025-10-09T21:52:42.154311Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:42.154327Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:42.154334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:42.154338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:42.154341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:42.154357Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:42.154369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:42.154375Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:42.154430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.154437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.154439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.154443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:42.154515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:42.154523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.154536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.154553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.154570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:42.154595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343283470628739:2432], now have 1 active actors on pipe
2025-10-09T21:52:42.154603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343283470628856:3355], now have 1 active actors on pipe
2025-10-09T21:52:42.154611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343283108774449:7824]
2025-10-09T21:52:42.154614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343283108774449:7824]
2025-10-09T21:52:42.154617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343283108774449:7824]
2025-10-09T21:52:42.154630Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:42.154646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:42.154634Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2534
2025-10-09T21:52:42.154649Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:42.154655Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:42.154643Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2534
2025-10-09T21:52:42.154658Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:42.154664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:42.154667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:42.154710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:42.154720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:42.154733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343283108774449:7824]
2025-10-09T21:52:42.154743Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:42.154753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:42.154775Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|9e82757a-46351a2d-2d1d4934-80ac63c3_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:42.154788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:42.154792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:42.154794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:42.154805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:42.154817Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:42.154844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:42.155016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:42.155025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:42.155040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:42.155123Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2535 partition: 0 MaxSeqNo: 25 sessionId: src_id|9e82757a-46351a2d-2d1d4934-80ac63c3_0
2025-10-09T21:52:42.155432Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|e9cd47ea-3fc1926f-4eb2f744-a14cfac6_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|9e82757a-46351a2d-2d1d4934-80ac63c3_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:42.157832Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e82757a-46351a2d-2d1d4934-80ac63c3_0] Write session: try to update token
2025-10-09T21:52:42.157848Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e82757a-46351a2d-2d1d4934-80ac63c3_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:42.168754Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:42.168774Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:42.168776Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:42.168778Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:42.168787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343283108774449:7824]
2025-10-09T21:52:42.168792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343283108774449:7824]
2025-10-09T21:52:42.168796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343283108774449:7824]
2025-10-09T21:52:42.168829Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:42.168834Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:42.168838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:42.168841Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:42.168844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.168847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:42.168976Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:42.170841Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2535 sessionId: src_id|9e82757a-46351a2d-2d1d4934-80ac63c3_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:42.173003Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:42.234369Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343283470628864:3994]
2025-10-09T21:52:42.234712Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 33 pipe creation result: OK TabletId: 72075186224037892 Generation: 2534, pipe: [1:7559343283470628864:3994]
2025-10-09T21:52:42.315151Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:42.315513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:42.315527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:42.315563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343283470628864:3994], now have 1 active actors on pipe
2025-10-09T21:52:42.315791Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:42.315812Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:42.315850Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvTabletPipe::TEvClientDestroyed
2025-10-09T21:52:42.315854Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:852: TPartitionWriter 72075186224037892 (partition=0) received TEvClientDestroyed
2025-10-09T21:52:42.315882Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 33 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343283470628864:3994]
2025-10-09T21:52:42.315919Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2535 reason: error at write: Disconnected sessionId: src_id|9e82757a-46351a2d-2d1d4934-80ac63c3_0
2025-10-09T21:52:42.316102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2535 sessionId: src_id|9e82757a-46351a2d-2d1d4934-80ac63c3_0 is DEAD
2025-10-09T21:52:42.316253Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e82757a-46351a2d-2d1d4934-80ac63c3_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: error at write: Disconnected, code: 500100 
2025-10-09T21:52:42.316265Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e82757a-46351a2d-2d1d4934-80ac63c3_0] Write session will restart in 0.008914s
2025-10-09T21:52:42.316289Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e82757a-46351a2d-2d1d4934-80ac63c3_0] Write session: Do CDS request
2025-10-09T21:52:42.316296Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e82757a-46351a2d-2d1d4934-80ac63c3_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:42.325372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:42.326457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|9e82757a-46351a2d-2d1d4934-80ac63c3_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:42.326576Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:42.326591Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2536
2025-10-09T21:52:42.326704Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2536 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:42.326736Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2536 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:42.326746Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2536 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:42.326750Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:42.327189Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2536 sessionId:  describe result for acl check
2025-10-09T21:52:42.327226Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:42.327233Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:42.327235Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:42.327241Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343283470628950:3339] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:42.327240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:42.327390Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:42.327399Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:42.327408Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:42.327456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:42.327488Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343283108774571:7821]
2025-10-09T21:52:42.327787Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343283470628950:3339] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:42.327904Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:42.328159Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:42.328190Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:42.328412Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:42.328446Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:42.328451Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:42.328452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:42.328452Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:42.328453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:42.328453Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:42.328454Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.328455Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.328456Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.328459Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:42.328465Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:42.328467Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:42.328470Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:42.328472Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:42.328474Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:42.328476Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.328478Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.328481Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.328485Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:42.328490Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:42.328494Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:42.328498Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:42.328500Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:42.328503Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:42.328506Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:42.328509Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:42.328512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:42.328516Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:42.328522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:42.328525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:42.328528Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.328531Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.328533Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.328536Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.328539Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.328542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.328560Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:42.329903Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343283470628950:3339] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:42.334660Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343283470628950:3339] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:42.334675Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343283470628950:3339] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:42.334680Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343283470628950:3339] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:42.334689Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2536 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:42.335064Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:42.336228Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:42.336247Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:42.336250Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:42.336255Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:42.336262Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2535 [2:7559343283108774571:7821]
2025-10-09T21:52:42.336274Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:42.336293Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:42.336304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:42.336307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:42.336309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:42.336325Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:42.336332Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:42.336339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:42.336395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.336404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.336406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.336408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:42.336473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:42.336481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.336493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.336509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.336528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:42.336554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343283470628869:2432], now have 1 active actors on pipe
2025-10-09T21:52:42.336563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343283470628992:3339], now have 1 active actors on pipe
2025-10-09T21:52:42.336571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343283108774565:7821]
2025-10-09T21:52:42.336578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343283108774565:7821]
2025-10-09T21:52:42.336580Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343283108774565:7821]
2025-10-09T21:52:42.336592Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:42.336609Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:42.336611Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:42.336617Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:42.336598Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2535
2025-10-09T21:52:42.336620Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:42.336610Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2535
2025-10-09T21:52:42.336624Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:42.336627Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:42.336673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:42.336681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:42.336694Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343283108774565:7821]
2025-10-09T21:52:42.336706Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:42.336717Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:42.336735Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cd773197-8d6aeee2-84fc8a22-b343b28a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:42.336746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:42.336752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:42.336754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:42.336764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:42.336778Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:42.336802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:42.337017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:42.337031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:42.337055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:42.337146Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2536 partition: 0 MaxSeqNo: 25 sessionId: src_id|cd773197-8d6aeee2-84fc8a22-b343b28a_0
2025-10-09T21:52:42.337458Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|9e82757a-46351a2d-2d1d4934-80ac63c3_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|cd773197-8d6aeee2-84fc8a22-b343b28a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:42.339697Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd773197-8d6aeee2-84fc8a22-b343b28a_0] Write session: try to update token
2025-10-09T21:52:42.339725Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd773197-8d6aeee2-84fc8a22-b343b28a_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:42.351442Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2536 sessionId: src_id|cd773197-8d6aeee2-84fc8a22-b343b28a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:42.351961Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:42.351979Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:42.351981Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:42.351983Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:42.351993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343283108774565:7821]
2025-10-09T21:52:42.351997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343283108774565:7821]
2025-10-09T21:52:42.352004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343283108774565:7821]
2025-10-09T21:52:42.352050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:42.352061Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:42.352063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.352066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:42.352067Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:42.352071Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:42.352198Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:42.353697Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:42.416136Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343283470629005:3994]
2025-10-09T21:52:42.416429Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 34 pipe creation result: OK TabletId: 72075186224037892 Generation: 2535, pipe: [1:7559343283470629005:3994]
2025-10-09T21:52:42.502241Z node 2 :PERSQUEUE CRIT: actor.h:45: [72075186224037892][Partition][0][StateIdle] unhandled exception yexception: ydb/library/actors/core/log.cpp:881: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
??+0 (0x148F3C3F)
??+0 (0x148F3BE7)
NActors::TEnsureFormattedRecordWriter::~TEnsureFormattedRecordWriter()+429 (0x1565871D)
NKikimr::NPQ::TPartitionedBlob::TPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, NKikimr::NPQ::THead&, NKikimr::NPQ::THead&, bool, bool, unsigned int, unsigned short, bool)+1074 (0x24435612)
NKikimr::NPQ::TPartitionBlobEncoder::NewPartitionedBlob(NKikimr::NPQ::TPartitionId const&, unsigned long, TBasicString<char, std::__y1::char_traits<char>> const&, unsigned long, unsigned short, unsigned int, bool, bool, unsigned int, unsigned short)+94 (0x2442F82E)
NKikimr::NPQ::TPartition::CompactRequestedBlob(NKikimr::NPQ::TRequestedBlob const&, NKikimr::NPQ::TPartition::TProcessParametersBase&, bool, NKikimr::TEvKeyValue::TEvRequest*, TInstant&, bool)+330 (0x2452642A)
NKikimr::NPQ::TPartition::BlobsForCompactionWereRead(TVector<NKikimr::NPQ::TRequestedBlob, std::__y1::allocator<NKikimr::NPQ::TRequestedBlob>> const&)+3341 (0x245279DD)
NKikimr::NPQ::TPartition::StateIdle(TAutoPtr<NActors::IEventHandle, TDelete>&)+7753 (0x243D47C9)
NActors::IActor::Receive(TAutoPtr<NActors::IEventHandle, TDelete>&)+216 (0x15604778)
??+0 (0x148F3C3F)
??+0 (0x148F3E14)
std::rethrow_exception(std::exception_ptr)+12 (0x1490518C)
NActors::IActorExceptionHandler::OnUnhandledException(std::exception_ptr const&)+33 (0x1740A1E1)
2025-10-09T21:52:42.502457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:42.502472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:42.502507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343283470629005:3994], now have 1 active actors on pipe
2025-10-09T21:52:42.502700Z node 2 :PERSQUEUE ERROR: pq_impl.cpp:200: Answer error topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0 requestId: <none> error: tablet will be restarted right now
2025-10-09T21:52:42.502741Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:310: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) schedule pipe restart attempt 34 reason: pipe to tablet is dead 72075186224037892, current pipe: [1:7559343283470629005:3994]
2025-10-09T21:52:42.502773Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:52:42.502841Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:815: session v1 error cookie: 2536 reason: status is not ok: tablet will be restarted right now sessionId: src_id|cd773197-8d6aeee2-84fc8a22-b343b28a_0
2025-10-09T21:52:42.503021Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2536 sessionId: src_id|cd773197-8d6aeee2-84fc8a22-b343b28a_0 is DEAD
2025-10-09T21:52:42.503162Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd773197-8d6aeee2-84fc8a22-b343b28a_0] Got error. Status: UNAVAILABLE, Description: <main>: Error: status is not ok: tablet will be restarted right now, code: 500001 
2025-10-09T21:52:42.503171Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd773197-8d6aeee2-84fc8a22-b343b28a_0] Write session will restart in 0.006415s
2025-10-09T21:52:42.503169Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:374: [72075186224037893][rt3.dc1--test-topic] TEvClientDestroyed 72075186224037892
2025-10-09T21:52:42.503187Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd773197-8d6aeee2-84fc8a22-b343b28a_0] Write session: Do CDS request
2025-10-09T21:52:42.503193Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd773197-8d6aeee2-84fc8a22-b343b28a_0] Start write session. Will connect to endpoint: localhost:11460
2025-10-09T21:52:42.510448Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cd773197-8d6aeee2-84fc8a22-b343b28a_0] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:42.510505Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:52:42.510520Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2537
2025-10-09T21:52:42.510669Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2537 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:52:42.510714Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2537 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:57708
2025-10-09T21:52:42.510724Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2537 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57708 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:52:42.510728Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:52:42.511102Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2537 sessionId:  describe result for acl check
2025-10-09T21:52:42.511134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:52:42.511140Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:52:42.511142Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:52:42.511148Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559343283470629081:3304] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:52:42.511665Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559343283470629081:3304] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:52:42.513277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:52:42.514014Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559343283470629081:3304] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:52:42.514933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:52:42.515113Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:52:42.515122Z node 2 :PERSQUEUE INFO: pq_impl.cpp:596: [PQ: 72075186224037892] has a tx writes info
2025-10-09T21:52:42.515134Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:52:42.515183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:52:42.515219Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559343283108774701:7813]
2025-10-09T21:52:42.515805Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDiskStatusStep
2025-10-09T21:52:42.516108Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitMetaStep
2025-10-09T21:52:42.516151Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInfoRangeStep
2025-10-09T21:52:42.516463Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataRangeStep
2025-10-09T21:52:42.516512Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[0]: d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:42.516518Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[1]: d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:42.516519Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[2]: d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:42.516521Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[3]: d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:42.516522Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[4]: d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:42.516523Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[5]: d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:42.516524Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[6]: d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.516525Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[7]: d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.516526Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:556: key[8]: d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.516532Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:564: add key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:42.516542Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:42.516546Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:42.516550Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:42.516554Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:42.516558Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:42.516562Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.516566Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.516570Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:606: add key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.516576Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:42.516584Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 0 count 3 size 8295635 so 0 eo 3 d0000000000_00000000000000000000_00000_0000000003_00015
2025-10-09T21:52:42.516590Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:42.516595Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 3 count 4 size 7988778 so 0 eo 7 d0000000000_00000000000000000003_00003_0000000004_00014
2025-10-09T21:52:42.516600Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:42.516586Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559343283470629081:3304] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:52:42.516599Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559343283470629081:3304] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:52:42.516604Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 7 count 3 size 8295601 so 0 eo 10 d0000000000_00000000000000000007_00001_0000000003_00015
2025-10-09T21:52:42.516608Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:42.516612Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 10 count 4 size 7988760 so 0 eo 14 d0000000000_00000000000000000010_00004_0000000004_00014
2025-10-09T21:52:42.516617Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:42.516603Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559343283470629081:3304] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:52:42.516611Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2537 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:52:42.516621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 14 count 4 size 7988756 so 0 eo 18 d0000000000_00000000000000000014_00002_0000000004_00014
2025-10-09T21:52:42.516625Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:42.516629Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 18 count 2 size 4506418 so 0 eo 20 d0000000000_00000000000000000018_00000_0000000002_00008|
2025-10-09T21:52:42.516634Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.516638Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 20 count 1 size 1741189 so 0 eo 21 d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.516642Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.516647Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 21 count 1 size 2253220 so 0 eo 22 d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.516652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:662: [rt3.dc1--test-topic:0:TInitDataRangeStep] check key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.516658Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:692: [rt3.dc1--test-topic:0:TInitDataRangeStep] Got data offset 22 count 3 size 6759616 so 0 eo 25 d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.516687Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitDataStep
2025-10-09T21:52:42.522890Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:914: [rt3.dc1--test-topic:0:TInitDataStep] read res partition offset 18 endOffset 25 key 18,2 valuesize 4506418 expected 4506418
2025-10-09T21:52:42.524194Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitEndWriteTimestampStep
2025-10-09T21:52:42.524208Z node 2 :PERSQUEUE INFO: partition_init.cpp:963: [rt3.dc1--test-topic:0:TInitEndWriteTimestampStep] Initializing EndWriteTimestamp skipped because already initialized.
2025-10-09T21:52:42.524210Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:52:42.524215Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:52:42.524222Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 2536 [2:7559343283108774701:7813]
2025-10-09T21:52:42.524236Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 25 Head Offset 25 PartNo 0 PackedSize 0 count 0 nextOffset 25 batches 0
SYNC INIT sourceId  src_id seqNo 25 offset 24
SYNC INIT DATA KEY: d0000000000_00000000000000000020_00001_0000000001_00003? size 1741189
SYNC INIT DATA KEY: d0000000000_00000000000000000021_00000_0000000001_00004? size 2253220
SYNC INIT DATA KEY: d0000000000_00000000000000000022_00000_0000000003_00012? size 6759616
2025-10-09T21:52:42.524253Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 0
2025-10-09T21:52:42.524256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:42.524260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:42.524263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:42.524279Z node 2 :PERSQUEUE DEBUG: partition.cpp:736: [72075186224037892][Partition][0][StateIdle] Init complete for topic 'rt3.dc1--test-topic' Partition: 0 SourceId:  src_id SeqNo: 25 offset: 24 MaxOffset: 25
2025-10-09T21:52:42.524286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:52:42.524294Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:52:42.524354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.524362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.524364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:197: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.524366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:202: [72075186224037892][Partition][0][StateIdle] 3 keys were taken away. Let's read 10754025 bytes
2025-10-09T21:52:42.524443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:213: [72075186224037892][Partition][0][StateIdle] Begin compaction for 3 blobs
2025-10-09T21:52:42.524451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.524465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00004?
2025-10-09T21:52:42.524475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:226: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000003_00012?
2025-10-09T21:52:42.524493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:250: [72075186224037892][Partition][0][StateIdle] Request 3 blobs for compaction
2025-10-09T21:52:42.524519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343283470629020:2432], now have 1 active actors on pipe
2025-10-09T21:52:42.524523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559343283470629139:3304], now have 1 active actors on pipe
2025-10-09T21:52:42.524531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 20 partno 1 count 1 parts_count 3 actorID [2:7559343283108774696:7813]
2025-10-09T21:52:42.524540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 21 partno 0 count 1 parts_count 4 actorID [2:7559343283108774696:7813]
2025-10-09T21:52:42.524547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 22 partno 0 count 3 parts_count 12 actorID [2:7559343283108774696:7813]
2025-10-09T21:52:42.524540Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 2536
2025-10-09T21:52:42.524562Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 0. Have to read 3 of 3 from KV
2025-10-09T21:52:42.524574Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 20 partno 1 count 1 parts_count 3
2025-10-09T21:52:42.524578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 21 partno 0 count 1 parts_count 4
2025-10-09T21:52:42.524580Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 22 partno 0 count 3 parts_count 12
2025-10-09T21:52:42.524581Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 25 max time lag 0ms effective offset 0
2025-10-09T21:52:42.524596Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 8295635 count 3 last offset 0, current partition end offset: 25
2025-10-09T21:52:42.524599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:52:42.524584Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 2536
2025-10-09T21:52:42.524649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:42.524659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:42.524671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 3 parts_count 15 actorID [2:7559343283108774696:7813]
2025-10-09T21:52:42.524684Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 3. Have to read 1 of 1 from KV
2025-10-09T21:52:42.524711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 3 parts_count 15
2025-10-09T21:52:42.524721Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|74b2f83-307779ca-769f2182-d191b725_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:52:42.524729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:52:42.524733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:52:42.524736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:52:42.524747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:52:42.524758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:52:42.524784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:42.525277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:52:42.525288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:52:42.525307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:52:42.525391Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2537 partition: 0 MaxSeqNo: 25 sessionId: src_id|74b2f83-307779ca-769f2182-d191b725_0
2025-10-09T21:52:42.525706Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cd773197-8d6aeee2-84fc8a22-b343b28a_0] Write session established. Init response: last_sequence_number: 25 session_id: "src_id|74b2f83-307779ca-769f2182-d191b725_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:52:42.528052Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74b2f83-307779ca-769f2182-d191b725_0] Write session: try to update token
2025-10-09T21:52:42.528067Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|74b2f83-307779ca-769f2182-d191b725_0] Send 10 message(s) (0 left), first sequence number is 26
2025-10-09T21:52:42.538893Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 3 of 3 from KV. Status 1
2025-10-09T21:52:42.538912Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-10-09T21:52:42.538914Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 1 from KV. Status 0
2025-10-09T21:52:42.538916Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 2 from KV. Status 0
2025-10-09T21:52:42.538925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 20 count 1 size 1741189 actorID [2:7559343283108774696:7813]
2025-10-09T21:52:42.538929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 21 count 1 size 2253220 actorID [2:7559343283108774696:7813]
2025-10-09T21:52:42.538933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 22 count 3 size 6759616 actorID [2:7559343283108774696:7813]
2025-10-09T21:52:42.538976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-10-09T21:52:42.538986Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 20 partno 1 count 1 parts 3 suffix '63' size 1741189
2025-10-09T21:52:42.538988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:426: [72075186224037892][Partition][0][StateIdle] Append blob for key d0000000000_00000000000000000020_00001_0000000001_00003?
2025-10-09T21:52:42.538991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:427: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-10-09T21:52:42.538999Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 4 suffix '63' size 2253220
2025-10-09T21:52:42.539007Z node 2 :PERSQUEUE WARN: pq_l2_cache.cpp:94: PQ Cache (L2). Same blob insertion. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 3 parts 12 suffix '63' size 6759616
2025-10-09T21:52:42.539145Z node 2 :GLOBAL ERROR: log.cpp:875: verification=NewHead.Offset == Head.GetNextOffset() && NewHead.PartNo == 0 || headCleared || needCompactHead || Head.PackedSize == 0;fline=blob.cpp:442;Head.NextOffset=20;Head.PackedSize=4506418;NewHead.Offset=20;NewHead.PartNo=1;headCleared=0;needCompactHead=0;
2025-10-09T21:52:42.540528Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2537 sessionId: src_id|74b2f83-307779ca-769f2182-d191b725_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:52:42.542811Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:52:42.603002Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:379: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 RequestInfly 0 ReadOffset 0 EndOffset 25 InitDone 1 WaitForData 0, pipe: [1:7559343283470629158:3994]
2025-10-09T21:52:42.603320Z node 1 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 26 consumer shared/user session shared/user_1_26_14331931971907184625_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 35 pipe creation result: OK TabletId: 72075186224037892 Generation: 2536, pipe: [1:7559343283470629158:3994]

Traceback (most recent call last):
  File "library/python/testing/yatest_common/yatest/common/process.py", line 384, in wait
    wait_for(
  File "library/python/testing/yatest_common/yatest/common/process.py", line 765, in wait_for
    raise TimeoutError(truncate(message, MAX_MESSAGE_LEN))
yatest.common.process.TimeoutError: 600 second(s) wait timeout has expired: Command '['/home/runner/.ya/tools/v4/9920199096/test_tool', 'run_ut', '@/home/runner/.ya/build/build_root/owmb/000361/ydb/public/sdk/cpp/src/client/federated_topic/ut/test-results/unittest/testing_out_stuff/chunk3/testing_out_stuff/test_tool.args']' stopped by 600 seconds timeout

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "devtools/ya/test/programs/test_tool/run_test/run_test.py", line 1770, in main
    res.wait(check_exit_code=False, timeout=current_run_test_timeout, on_timeout=timeout_callback)
  File "library/python/testing/yatest_common/yatest/common/process.py", line 398, in wait
    raise ExecutionTimeoutError(self, str(e))
yatest.common.process.ExecutionTimeoutError: (("600 second(s) wait timeout has expired: Command '['/home/runner/.ya/tools/v4/9920199096/test_tool', 'run_ut', '@/home/runner/.ya/build/build_root/owmb/000361/ydb/public/sdk/cpp/src/client/federated_topic/ut/test-results/unittest/testing_out_stuff/chunk3/testing_out_stuff/test_tool.args']' stopped by 600 seconds timeout",), {})
2025-10-09T21:42:51.547631Z :RetryDiscoveryWithCancel INFO: Random seed for debugging is 1760046171547623
2025-10-09T21:42:51.709760Z node 1 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7559340744120011733:2089];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:51.709781Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-10-09T21:42:51.720012Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:123: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, got undelivered to scheme cache: ActorUnknown
2025-10-09T21:42:51.730237Z node 2 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[2:7559340744717724456:2151];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:51.730316Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-10-09T21:42:51.730892Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/owmb/0001ab/r3tmp/tmp2r7AU9/pdisk_1.dat
2025-10-09T21:42:51.737035Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:51.832682Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:51.832706Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:51.833253Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:51.833264Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:51.838681Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-10-09T21:42:51.838742Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:51.838994Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:51.842964Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:51.846929Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:42:51.850388Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
TServer::EnableGrpc on GrpcPort 3994, node 1
2025-10-09T21:42:51.878733Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/owmb/0001ab/r3tmp/yandex1aCf2l.tmp
2025-10-09T21:42:51.878743Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/owmb/0001ab/r3tmp/yandex1aCf2l.tmp
2025-10-09T21:42:51.880468Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/owmb/0001ab/r3tmp/yandex1aCf2l.tmp
2025-10-09T21:42:51.880530Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-10-09T21:42:51.891532Z INFO: TTestServer started on Port 11588 GrpcPort 3994
TClient is connected to server localhost:11588
PQClient connected to localhost:3994
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-09T21:42:51.950495Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
waiting...
waiting...
waiting...
2025-10-09T21:42:52.317176Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340748414979949:2325], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.317208Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.322353Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340748414979976:2328], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.322383Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340748414979977:2329], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.322438Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:52.323643Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976720661:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-10-09T21:42:52.334981Z node 1 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [1:7559340748414979983:2331], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720661 completed, doublechecking }
2025-10-09T21:42:52.386952Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:52.426942Z node 1 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [1:7559340748414980116:2694] txid# 281474976720663, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:52.435568Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [2:7559340749012691997:2301], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:52.436588Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=2&id=M2I4MWU3YjQtMWZmYzMzOTAtNDFlNjU3NTMtODFlOTQ5NWU=, ActorId: [2:7559340749012691952:2292], ActorState: ExecuteState, TraceId: 01k75etj7gak0ynekykxz2eckz, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:52.437799Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:52.437941Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [1:7559340748414980133:2340], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:52.438705Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=1&id=ODZiNTgzYWYtN2VkYjc2NzktMjdhZjY4ZTMtMWMxOTQ0ZjQ=, ActorId: [1:7559340748414979945:2322], ActorState: ExecuteState, TraceId: 01k75etj4t7g2tm5tj10nv1gfd, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:52.438822Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:52.473547Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:52.516763Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720665:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:3994", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, false, 1000);
2025-10-09T21:42:52.635625Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720666. Ctx: { TraceId: 01k75etjcy70th8jw90f5e3sa1, Database: , SessionId: ydb://session/3?node_id=1&id=NjJjZmJmZmEtODljMGVkNGYtYTE3NWQ3YTYtY2JjNDM2NTQ=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [1:7559340748414980467:2955] 
2025-10-09T21:42:52.711338Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:52.734314Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:56.710069Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7559340744120011733:2089];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:56.710121Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
2025-10-09T21:42:56.730612Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[2:7559340744717724456:2151];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:56.730655Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
2025-10-09T21:42:57.753049Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720672:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
waiting...
PQ Client: create topic: rt3.dc1--test-topic with 1 partitions
CallPersQueueGRPC request to localhost:3994
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:57.782592Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:3994
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720673
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-09T21:42:57.804282Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][] pipe [1:7559340769889817374:3224] connected; active server actors: 1
2025-10-09T21:42:57.804343Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1517: [72075186224037893][rt3.dc1--test-topic] updating configuration. Deleted partitions []. Added partitions [0]
2025-10-09T21:42:57.805326Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:940: [72075186224037893][rt3.dc1--test-topic] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-09T21:42:57.805856Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:138: [72075186224037893][rt3.dc1--test-topic] BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) 
2025-10-09T21:42:57.811303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:57.812402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:57.812474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:784: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-09T21:42:57.812486Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:57.812497Z node 2 :PERSQUEUE INFO: pq_impl.cpp:609: [PQ: 72075186224037892] doesn't have tx writes info
2025-10-09T21:42:57.812625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340769889817401:2436], now have 1 active actors on pipe
2025-10-09T21:42:57.812634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340769889817373:3223], now have 1 active actors on pipe
2025-10-09T21:42:57.812914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.814235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.810891Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720674. Ctx: { TraceId: 01k75etqfz53jrfkh1jaxzjhtt, Database: , SessionId: ydb://session/3?node_id=1&id=YTliYTQ3YmMtZGM1NDkxNzEtOWI1NjY3MzEtNGUzNTdkZWQ=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-10-09T21:42:57.812763Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
GetTopicVersionFromPath:  record Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
 name rt3.dc1--test-topic version0
2025-10-09T21:42:57.827513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340769889817455:3279], now have 1 active actors on pipe
2025-10-09T21:42:57.827676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.829051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.829310Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:57.829331Z node 2 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:57.829386Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:57.829455Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340770487528957:2383]
2025-10-09T21:42:57.830220Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:57.830231Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:57.830235Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [2:7559340770487528957:2383]
2025-10-09T21:42:57.830242Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:57.830565Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-10-09T21:42:57.830570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.830595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.830598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:57.830601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:57.830603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.830605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.830606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:57.830613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:57.830744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.830775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:57.832103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.832145Z node 2 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046177876, TxId 281474976720673
2025-10-09T21:42:57.832150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.832152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.832175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.832227Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:57.832898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:57.834673Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-10-09T21:42:57.834747Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:57.834754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.834756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.834758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.834766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:57.834885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:395: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-10-09T21:42:57.834907Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:57.834957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.836777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.836802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.837387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.932302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.932318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.932321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.032574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.032589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.032593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.132900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.132928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.132931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.233297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.233315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.233318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046177876
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046177876
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:3994
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:58.334658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.334675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.334679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.335459Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
2025-10-09T21:42:58.342331Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-09T21:42:58.342620Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-09T21:42:58.342629Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:3994
2025-10-09T21:42:58.344919Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:58.346269Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:58.346287Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1
2025-10-09T21:42:58.350158Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:58.350213Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:59476
2025-10-09T21:42:58.350219Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:59476 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:58.350224Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:58.350816Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-09T21:42:58.350872Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:58.350875Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:58.350877Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:58.350883Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340774184784805:2450] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:58.351443Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340774184784805:2450] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:58.384527Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [1:7559340774184784805:2450] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:58.384642Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340774184784839:2450] connected; active server actors: 1
2025-10-09T21:42:58.384656Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [1:7559340774184784805:2450] (SourceId=src, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src
2025-10-09T21:42:58.384661Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340774184784805:2450] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:58.384742Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340774184784839:2450] disconnected.
2025-10-09T21:42:58.384749Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340774184784839:2450] disconnected; active server actors: 1
2025-10-09T21:42:58.384752Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340774184784839:2450] disconnected no session
2025-10-09T21:42:58.409460Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340774184784805:2450] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:58.409487Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340774184784805:2450] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:42:58.409491Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340774184784805:2450] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:58.409499Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:58.412235Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1
2025-10-09T21:42:58.413781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340774184784862:2450], now have 1 active actors on pipe
2025-10-09T21:42:58.413925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.413937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.413988Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src|80d2191a-f5c144ca-131ee55e-e71f5e7b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src
2025-10-09T21:42:58.414002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.414009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.414012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:58.414044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.414064Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:58.414087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:58.414548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:58.414554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:58.414575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:58.418456Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1 partition: 0 MaxSeqNo: 0 sessionId: src|80d2191a-f5c144ca-131ee55e-e71f5e7b_0
2025-10-09T21:42:58.418957Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046178418 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:58.418999Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|80d2191a-f5c144ca-131ee55e-e71f5e7b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:58.419200Z :INFO: [] MessageGroupId [src] SessionId [src|80d2191a-f5c144ca-131ee55e-e71f5e7b_0] Write session: close. Timeout = 0 ms
2025-10-09T21:42:58.419212Z :INFO: [] MessageGroupId [src] SessionId [src|80d2191a-f5c144ca-131ee55e-e71f5e7b_0] Write session will now close
2025-10-09T21:42:58.419217Z :DEBUG: [] MessageGroupId [src] SessionId [src|80d2191a-f5c144ca-131ee55e-e71f5e7b_0] Write session: aborting
2025-10-09T21:42:58.419385Z :INFO: [] MessageGroupId [src] SessionId [src|80d2191a-f5c144ca-131ee55e-e71f5e7b_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:42:58.419393Z :DEBUG: [] MessageGroupId [src] SessionId [src|80d2191a-f5c144ca-131ee55e-e71f5e7b_0] Write session: destroy
2025-10-09T21:42:58.421044Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId: src|80d2191a-f5c144ca-131ee55e-e71f5e7b_0 grpc read done: success: 0 data: 
2025-10-09T21:42:58.421057Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 1 sessionId: src|80d2191a-f5c144ca-131ee55e-e71f5e7b_0 grpc read failed
2025-10-09T21:42:58.421064Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 1 sessionId: src|80d2191a-f5c144ca-131ee55e-e71f5e7b_0 grpc closed
2025-10-09T21:42:58.421072Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1 sessionId: src|80d2191a-f5c144ca-131ee55e-e71f5e7b_0 is DEAD
2025-10-09T21:42:58.421254Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:42:58.421914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340774184784862:2450] destroyed
2025-10-09T21:42:58.421937Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:42:58.421948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.421951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.421954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.438099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.438121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.438126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: err
2025-10-09T21:42:58.477962Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: TRANSPORT_UNAVAILABLE. Description: [ { <main>: Error: GRpc error: (14): mock 'unavailable' } { <main>: Error: Grpc error response on endpoint localhost:17666 } ]
2025-10-09T21:42:58.540940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.540956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.540960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.641963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.641981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.641984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.742370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.742385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.742389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.843234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.843252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.843255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.942936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.942954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.942957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.046123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.046141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.046144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.148881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.148897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.148900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.250239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.250257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.250260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.350486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.350504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.350507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.450821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.450846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.450849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.551212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.551239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.551243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.651579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.651594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.651598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.751855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.751871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.751874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.852257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.852271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.852274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.952589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.952604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.952607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.053073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.053088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.053091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.153326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.153347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.153350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.253619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.253634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.253636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.354020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.354056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.354058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.454358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.454388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.454391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.554740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.554758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.554761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.655069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.655097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.655101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.756377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.756395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.756399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.862117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.862134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.862136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.959038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.959051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.959053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.062305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.062322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.062325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.162504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.162525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.162528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.263326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.263344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.263348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.363681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.363708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.363711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.464128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.464148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.464151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.565177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.565196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.565199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.665252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.665268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.665284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.765623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.765639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.765642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.865935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.865952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.865954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.966257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.966275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.966277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.066723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.066741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.066745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.169366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.169386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.169389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.270367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.270385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.270388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.371687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.371706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.371711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.471639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.471656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.471659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.571968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.571987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.571990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.672382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.672400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.672404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.774188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.774204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.774207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.830732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:02.878766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.878782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.878785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.976265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.976279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.976282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.078750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.078766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.078770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.178729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.178750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.178753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.279160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.279176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.279189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.379405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.379420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.379422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.479969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.479987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.479991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
E1009 21:43:03.538141444 3625214 chttp2_transport.cc:1143]             ipv6:%5B::1%5D:17666: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 1250ms
2025-10-09T21:43:03.580120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.580136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.580140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.680483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.680501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.680504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.782623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.782639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.782642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.882727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.882742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.882745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.986187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.986203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.986206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.087889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.087908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.087911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.187856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.187880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.187884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.290303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.290321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.290324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.391651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.391672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.391675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.494520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.494537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.494541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.595059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.595076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.595079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.695266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.695281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.695284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.795619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.795636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.795639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.895974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.895992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.895995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.998342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.998359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.998362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.098685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.098705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.098709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.199478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.199495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.199497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.299419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.299447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.299449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.400311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.400326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.400329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.501081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.501100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.501103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.601640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.601658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.601661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.701852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.701871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.701874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.802280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.802297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.802301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.902556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.902583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.902587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.002914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.002933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.002937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.103244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.103265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.103268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.203567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.203586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.203588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.303986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.304003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.304006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.406261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.406286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.406288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.506703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.506720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.506723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.607105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.607123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.607126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.707418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.707437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.707442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: ok
2025-10-09T21:43:06.788782Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard_impl.cpp:7721: Cannot get console configs
2025-10-09T21:43:06.788797Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:43:06.810317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.810336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.810339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.913776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.913797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.913800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.013587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.013612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.013616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.114054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.114071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.114075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.214304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.214318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.214321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.314647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.314662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.314665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.414984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.414998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.415001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.515346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.515361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.515365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.615969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.615983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.615986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.716097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.716114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.716117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.816399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.816414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.816418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.831594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:07.916781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.916799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.916807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.017058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.017075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.017079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.118232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.118246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.118249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.218118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.218138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.218142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.318601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.318617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.318621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.418505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.418523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.418526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.518860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.518885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.518888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.622010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.622027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.622051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.722125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.722141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.722144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.826338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.826364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.826366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.930539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.930557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.930560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.031055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.031071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.031074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.131190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.131209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.131212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.236376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.236395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.236398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.336040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.336055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.336057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.439414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.439432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.439436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.538733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.538750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.538763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.639064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.639095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.639098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.739427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.739453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.739457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.841616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.841634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.841637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.949199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.949221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.949224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.047181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.047200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.047203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.147698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.147715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.147718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.247956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.247975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.247978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.352097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.352111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.352114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.452153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.452168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.452171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.552546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.552560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.552563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.652848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.652863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.652866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.754114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.754130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.754134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.854212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.854227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.854231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.958185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.958200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.958203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.058386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.058402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.058404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.158756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.158771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.158775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.262129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.262144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.262147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.362152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.362170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.362174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.462359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.462377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.462381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.563147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.563164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.563168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.663209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.663229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.663232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.769634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.769652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.769655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.870021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.870054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.870058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.970444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.970459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.970462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.070574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.070600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.070603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.171041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.171059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.171062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.271674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.271692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.271696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.371651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.371673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.371676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.472021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.472048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.472051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.572337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.572352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.572356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.672963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.672979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.672983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.773103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.773117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.773120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.832107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:12.873415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.873432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.873436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.973815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.973835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.973851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.074098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.074115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.074118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.174484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.174498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.174502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.274835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.274850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.274854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.375225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.375242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.375246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
E1009 21:43:13.450638025 3625509 chttp2_transport.cc:1143]             ipv4:127.0.0.1:17666: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 1250ms
2025-10-09T21:43:13.475533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.475549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.475553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.575895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.575909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.575913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.676918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.676936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.676940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.777108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.777124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.777127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.878267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.878283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.878286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.978864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.978880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.978884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.079159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.079175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.079178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.179534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.179550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.179553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.279867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.279881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.279884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.380236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.380261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.380265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.483330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.483348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.483351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.583785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.583800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.583804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.684320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.684348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.684352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.784690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.784703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.784706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.885057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.885081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.885084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.985416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.985432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.985436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.085762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.085777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.085780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.186111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.186127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.186131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.286479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.286496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.286499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.386833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.386849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.386852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.490156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.490179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.490182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.590099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.590119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.590122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.694513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.694529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.694532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.794406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.794434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.794437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.894723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.894737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.894740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.995184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.995198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.995202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.095425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.095439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.095442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.195767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.195784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.195787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.296147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.296163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.296166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.396509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.396523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.396526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.496817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.496842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.496847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.597213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.597227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.597229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.697628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.697646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.697649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.800958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.800975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.800979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.901304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.901318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.901320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.001679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.001693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.001696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.102197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.102228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.102231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.202397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.202424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.202428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.302688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.302713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.302717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.403092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.403110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.403114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.503434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.503450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.503453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.604014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.604030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.604033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.704153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.704168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.704171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.804490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.804503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.804507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.832455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:17.904841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.904858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.904862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.005205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.005229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.005232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.105539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.105562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.105565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.208065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.208083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.208086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.308305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.308320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.308323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.408659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.408680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.408683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.509077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.509096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.509099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.609346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.609361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.609364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.709735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.709756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.709759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.810081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.810097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.810100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.910426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.910454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.910458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.010848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.010867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.010870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.111140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.111167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.111170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.214091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.214108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.214111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.314385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.314402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.314406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.414686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.414703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.414706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.518095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.518113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.518117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.617989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.618006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.618010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.718787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.718800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.718804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.822178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.822196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.822199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.922127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.922144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.922148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.022491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.022508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.022511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.122791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.122805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.122808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.223191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.223222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.223226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.324605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.324623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.324626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.424883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.424909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.424913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.526021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.526045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.526049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.629416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.629433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.629438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.729500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.729515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.729518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.830359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.830375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.830378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.930277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.930294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.930297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.031321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.031338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.031341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.131452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.131468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.131472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.235124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.235145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.235148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.335245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.335265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.335269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.436012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.436031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.436035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.536644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.536672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.536676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.637152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.637169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.637172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.737680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.737697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.737700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.838085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.838114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.838117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.938431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.938449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.938453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.038754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.038785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.038790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.139490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.139510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.139514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.239918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.239946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.239949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.340307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.340325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.340329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.441129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.441147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.441150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.541530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.541557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.541560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.641883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.641914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.641918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.742449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.742468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.742472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.832511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:22.846115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.846133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.846136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.946038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.946058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.946062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.046378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.046396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.046399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.146749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.146767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.146770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.257589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.257610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.257614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.358814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.358836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.358842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.462439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.462461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.462464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.565578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.565596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.565600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.669580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.669596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.669599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.770243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.770260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.770264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.870171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.870190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.870194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.970531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.970548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.970551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.070851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.070868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.070884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.171178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.171200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.171203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.271587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.271607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.271610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.375760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.375773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.375776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.478103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.478123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.478125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.582092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.582112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.582115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.682205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.682222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.682225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.782602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.782617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.782620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.886125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.886149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.886153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.986355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.986371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.986374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.086626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.086644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.086647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.187030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.187046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.187049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.287319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.287332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.287335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.388039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.388053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.388056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.491748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.491764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.491767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.592008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.592024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.592027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.692358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.692384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.692388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.792956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.792970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.792973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.895101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.895118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.895121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.996663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.996680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.996684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.101037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.101063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.101066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.201284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.201305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.201308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.301674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.301691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.301695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.402381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.402398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.402401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.502636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.502652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.502655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.603209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.603225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.603228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.705577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.705590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.705593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.806226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.806242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.806246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.907648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.907673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.907677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.014654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.014672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.014676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.114662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.114682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.114684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.218114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.218132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.218135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.319581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.319595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.319598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.422242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.422257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.422260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.523685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.523702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.523705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.623095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.623112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.623115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.726336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.726351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.726354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.807342Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 0 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:43:27.806466Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:43:27.806502Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 1
2025-10-09T21:43:27.810373Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 1 DataSize: 0 UsedReserveSize: 0
2025-10-09T21:43:27.810616Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:43:27.811375Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72057594046644480, NodeId 1, Generation 2
2025-10-09T21:43:27.829169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.829185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.829188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.832561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:27.929263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.929283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.929286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.029711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.029725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.029728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.129967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.129981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.129983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.230595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.230612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.230615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.330791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.330806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.330809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.431090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.431104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.431106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
E1009 21:43:28.450769002 3625484 chttp2_transport.cc:1143]             ipv6:%5B::1%5D:17666: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 2500ms
2025-10-09T21:43:28.532060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.532083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.532086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.634120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.634137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.634140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.734111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.734127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.734130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.834448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.834470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.834472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.934857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.934874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.934877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.035188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.035204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.035208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.135540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.135556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.135559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.238119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.238136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.238139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.339713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.339733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.339737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.440657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.440674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.440677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.546279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.546297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.546300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.646836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.646851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.646855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.750357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.750374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.750377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.850326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.850339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.850342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.950710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.950724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.950727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.051684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.051702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.051706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.151897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.151913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.151916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.254457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.254475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.254479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.354724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.354739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.354742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.455025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.455038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.455041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.558107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.558120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.558122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.657793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.657808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.657811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.758078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.758092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.758095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.859887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.859904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.859907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.959816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.959832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.959835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.061830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.061846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.061848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.162228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.162247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.162250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.262481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.262496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.262499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.362726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.362741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.362744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.465424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.465439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.465442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.565521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.565534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.565538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.668089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.668105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.668109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.768282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.768296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.768298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.873556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.873573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.873576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.975892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.975907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.975910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.075967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.075982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.075986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.176315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.176331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.176345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.276845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.276861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.276864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.376997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.377011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.377014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.477427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.477448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.477451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.577740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.577752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.577756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.678113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.678143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.678146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.778480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.778496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.778499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.834225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:32.878815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.878832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.878836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.979145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.979160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.979163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.080818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.080841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.080845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.186219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.186235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.186238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.289227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.289243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.289246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.389779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.389801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.389805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.490562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.490581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.490584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.594278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.594297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.594301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.693349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.693363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.693366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.793727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.793746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.793760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.894107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.894129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.894133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.994549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.994564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.994567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.094830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.094846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.094849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.195171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.195184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.195186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.298389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.298406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.298409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.398893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.398914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.398917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.499344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.499362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.499366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.599936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.599955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.599959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.702163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.702183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.702187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.802412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.802443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.802446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.902722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.902739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.902742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.005482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.005501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.005504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.110153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.110174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.110178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.210647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.210667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.210671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.314169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.314184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.314187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.418180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.418197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.418201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.522215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.522241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.522245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.622244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.622259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.622262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.726110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.726131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.726135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.824755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.824770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.824773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.925381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.925398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.925402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.025579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.025590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.025593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.125902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.125926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.125928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.230094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.230110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.230113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.330241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.330267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.330271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.430617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.430635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.430639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.530871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.530885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.530887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.631249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.631264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.631266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.731594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.731610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.731613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: err
2025-10-09T21:43:36.798643Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: TRANSPORT_UNAVAILABLE. Description: [ { <main>: Error: GRpc error: (14): mock 'unavailable' } { <main>: Error: Grpc error response on endpoint localhost:17666 } ]
2025-10-09T21:43:36.831949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.831962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.831965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.932353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.932370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.932373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.032701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.032719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.032723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.133190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.133207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.133210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.233547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.233566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.233569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.333723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.333742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.333746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.434083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.434099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.434102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.534440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.534457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.534461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.634791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.634808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.634811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.735110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.735123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.735127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.834554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:37.835479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.835492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.835495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.938101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.938119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.938122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.038251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.038268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.038270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.138609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.138642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.138646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.238967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.238989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.238991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.339248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.339264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.339268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.439648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.439666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.439669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.539987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.540002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.540004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.642167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.642184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.642187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.742226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.742242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.742246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.842577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.842608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.842612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.942927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.942944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.942947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.043276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.043294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.043297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.143647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.143663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.143667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.244036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.244056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.244059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.344299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.344314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.344317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.444687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.444702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.444705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.545045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.545061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.545064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.645435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.645451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.645454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.745821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.745836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.745839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.846100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.846115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.846118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.946755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.946770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.946774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.050105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.050124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.050127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.150234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.150259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.150264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.253863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.253882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.253886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.354137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.354155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.354158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.454357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.454374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.454377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.554743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.554762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.554765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.655077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.655095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.655098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.756420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.756437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.756440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.863286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.863303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.863307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.966140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.966158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.966161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.066218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.066236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.066239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.169328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.169353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.169357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.269394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.269410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.269414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.369781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.369800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.369803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.470284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.470301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.470304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.571521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.571542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.571545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.673718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.673735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.673738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.773972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.773998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.774001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.874246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.874260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.874263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.974500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.974514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.974518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.075128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.075145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.075148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.175695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.175725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.175729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.278465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.278483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.278486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.378468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.378486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.378489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.478992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.479009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.479013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.582159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.582174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.582176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.682617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.682633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.682641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.790374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.790389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.790392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.834697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:42.890480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.890497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.890500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.990754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.990769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.990773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.091241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.091259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.091262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.191547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.191565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.191568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.291864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.291882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.291886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.392219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.392238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.392241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.495420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.495438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.495441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.595003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.595018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.595021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.696354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.696370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.696373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.798086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.798104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.798107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.901866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.901880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.901899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.002010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.002044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.002048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.104417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.104440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.104444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.204283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.204300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.204303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.310464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.310479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.310483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.410595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.410611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.410614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.510947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.510965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.510968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.611385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.611402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.611405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: ok
2025-10-09T21:43:44.711592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.711618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.711621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.812121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.812158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.812162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.912441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.912457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.912460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.012690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.012708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.012711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.113373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.113390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.113392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.214371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.214388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.214391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.314309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.314329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.314332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.414560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.414574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.414585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.514978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.514996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.515009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.615305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.615324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.615328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.717601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.717615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.717619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.818444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.818463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.818467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.918387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.918403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.918406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.018854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.018872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.018875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.119117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.119138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.119141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.219477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.219495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.219498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.322405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.322435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.322438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.423783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.423798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.423801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.529502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.529518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.529521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.631258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.631274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.631277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.735683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.735699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.735702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.835980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.835998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.836001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.936407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.936424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.936427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.037131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.037149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.037152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.138702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.138720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.138723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.238853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.238869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.238872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.339199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.339229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.339232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.439649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.439676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.439680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.540017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.540030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.540033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.640500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.640515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.640518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.740599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.740616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.740619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.838227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:47.844613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.844628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.844631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.945220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.945236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.945240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.045752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.045767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.045771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.146178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.146194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.146197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.246935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.246949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.246952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.346836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.346852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.346855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.447841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.447857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.447861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.548074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.548091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.548094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.648415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.648431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.648434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.750161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.750183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.750186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.850107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.850132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.850135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.950539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.950561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.950564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.050846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.050863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.050872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.151617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.151633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.151636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.252823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.252839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.252845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.352969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.353001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.353004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.453301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.453317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.453321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.553637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.553654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.553656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.655492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.655510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.655514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.756568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.756583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.756586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.856687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.856703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.856707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.961975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.961990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.961993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.066349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.066366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.066369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.167005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.167021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.167024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.267969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.267986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.267989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.368483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.368506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.368510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.469062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.469079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.469082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.569902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.569919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.569922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.670409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.670433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.670436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.770622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.770641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.770644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.871281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.871296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.871300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.971334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.971349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.971352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.071861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.071880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.071883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.172128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.172144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.172147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.272425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.272440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.272443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.372880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.372898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.372901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.473074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.473088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.473091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.573617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.573634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.573637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.673945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.673973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.673976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.775209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.775224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.775227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.878110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.878129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.878132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.977962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.977980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.977983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.078346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.078362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.078365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.178644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.178660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.178663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
E1009 21:43:52.206648816 3625483 chttp2_transport.cc:1143]             ipv4:127.0.0.1:17666: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 2500ms
2025-10-09T21:43:52.282158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.282174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.282177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.384325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.384345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.384348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.485061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.485078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.485081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.585153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.585169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.585172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.685509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.685524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.685527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.785971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.785988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.785991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.841640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:52.887160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.887175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.887178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.987624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.987651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.987654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.088094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.088110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.088113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.188309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.188325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.188328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.289460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.289477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.289480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.389583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.389600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.389603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.489833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.489853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.489857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.590298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.590319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.590322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.690664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.690681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.690684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.791399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.791418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.791422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.891793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.891811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.891814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.992154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.992174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.992177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.094232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.094252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.094255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.194405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.194447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.194450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.294752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.294770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.294773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.397639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.397657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.397660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.498367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.498386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.498390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.598361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.598375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.598378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.698745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.698764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.698767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.799100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.799115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.799119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.899425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.899442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.899445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.000255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.000270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.000274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.100636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.100651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.100654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.202216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.202234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.202236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.302366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.302385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.302388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.403072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.403086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.403089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.505708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.505725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.505728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.605976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.605996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.606000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.709569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.709591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.709594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.809699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.809716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.809728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.910116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.910132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.910135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.014197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.014215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.014218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.114485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.114501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.114504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.214742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.214759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.214762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.315889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.315907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.315910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.421016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.421034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.421037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.526167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.526186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.526189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.622937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.622952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.622963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.723393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.723407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.723411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.823720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.823733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.823736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.924354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.924371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.924374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.025614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.025631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.025635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.126271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.126287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.126290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.226482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.226497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.226500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.330101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.330117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.330120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.430232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.430250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.430253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.533676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.533691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.533695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.634113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.634132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.634135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.734305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.734321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.734325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.810311Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:43:57.810340Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 2
2025-10-09T21:43:57.816072Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 0 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:43:57.816690Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 2 DataSize: 0 UsedReserveSize: 0
2025-10-09T21:43:57.816736Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:43:57.834778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.834792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.834796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.841904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:57.935071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.935088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.935091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.035581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.035597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.035600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.135712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.135726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.135729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.236180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.236198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.236202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.337681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.337697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.337700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.437860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.437876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.437892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.539651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.539677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.539679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.641536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.641552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.641555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.741454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.741472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.741475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.846175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.846193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.846196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.944182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.944199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.944203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.044569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.044585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.044587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.148686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.148704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.148708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.247784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.247812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.247815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.352089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.352106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.352109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.452130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.452146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.452149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.552476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.552493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.552496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.652822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.652837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.652840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.753147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.753163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.753166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.857394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.857410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.857413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.957507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.957532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.957536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.057879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.057897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.057900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.158293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.158318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.158321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.258501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.258526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.258530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.358993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.359009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.359012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.459763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.459777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.459780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.560088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.560112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.560115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.662216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.662233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.662237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.766394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.766413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.766431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.870495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.870512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.870515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.974074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.974095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.974097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.072115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.072132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.072134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.172750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.172765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.172777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.272851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.272867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.272870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.374590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.374607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.374610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.474532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.474549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.474551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.574875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.574892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.574894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.675195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.675212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.675214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.775566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.775581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.775583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.875942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.875965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.875969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.976462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.976482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.976489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.076630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.076649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.076654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.176917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.176933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.176935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.277388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.277403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.277407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.377708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.377724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.377735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.478632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.478647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.478650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.578912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.578927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.578930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.679416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.679437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.679440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.779600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.779616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.779619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.842194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:02.880436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.880449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.880452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.982797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.982818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.982821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.082839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.082856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.082859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.183082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.183101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.183103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.289474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.289498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.289501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.389389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.389404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.389407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.489747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.489761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.489764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.590122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.590150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.590153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.693280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.693295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.693298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.795631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.795644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.795647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.895345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.895376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.895379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.995615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.995630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.995633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.097670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.097685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.097688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.198196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.198211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.198214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.298435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.298463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.298466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.398559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.398576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.398580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.498837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.498853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.498856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.600110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.600125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.600128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.702099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.702119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.702122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.802102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.802118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.802121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.902547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.902562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.902564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.003318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.003344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.003348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.103525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.103549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.103553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.204167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.204182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.204185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.308291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.308306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.308310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.408202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.408218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.408221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.508576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.508590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.508593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.608916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.608932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.608936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.709267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.709284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.709287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.810176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.810192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.810195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.910519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.910535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.910538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.010877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.010891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.010904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.111187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.111202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.111205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.211697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.211722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.211725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.314464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.314480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.314483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.418309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.418327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.418330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.524257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.524274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.524277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.622361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.622376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.622387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.728111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.728127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.728130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.830632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.830648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.830651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.930779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.930795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.930798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.035471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.035486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.035490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.135311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.135327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.135330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.235631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.235649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.235652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.337124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.337140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.337143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.441898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.441917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.441920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.542650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.542666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.542670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.644581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.644599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.644602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.748507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.748522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.748524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.843794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:07.848921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.848940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.848944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.950081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.950095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.950098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.051849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.051866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.051869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.152749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.152764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.152768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.253041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.253056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.253059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.354229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.354246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.354249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.457860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.457877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.457880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.559121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.559138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.559140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.659539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.659556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.659559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.759736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.759752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.759756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.860146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.860160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.860162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.960472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.960497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.960500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.060825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.060843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.060847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.161173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.161190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.161193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.262127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.262144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.262147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.363057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.363073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.363077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.466438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.466457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.466460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.566687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.566704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.566707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.667398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.667414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.667418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.767972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.767988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.767992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.881416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.881432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.881435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.978852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.978869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.978871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.079231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.079247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.079250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.179499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.179515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.179518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.279909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.279927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.279930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.380388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.380405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.380408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.481997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.482015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.482018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.582555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.582572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.582576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.685325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.685342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.685345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.785135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.785149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.785153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.885564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.885580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.885593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.987481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.987495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.987497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.087466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.087482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.087485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.188048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.188063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.188066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.289187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.289204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.289208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.389615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.389630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.389633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.489855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.489870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.489873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.593298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.593312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.593315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.693644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.693661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.693664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.794022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.794051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.794055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.894234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.894251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.894254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.994673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.994690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.994693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.094975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.094990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.094993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.195335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.195351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.195355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.295622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.295645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.295648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.398184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.398202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.398206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.498462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.498480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.498484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.598800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.598816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.598819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.699151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.699169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.699172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.799653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.799666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.799669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.846879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:12.899874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.899889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.899892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.000249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.000263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.000265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.105196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.105211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.105214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.205717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.205734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.205737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.306152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.306166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.306169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.406166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.406182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.406184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.507795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.507816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.507819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.608498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.608516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.608519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.708812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.708829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.708832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.809365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.809381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.809384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.909461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.909476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.909479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.009857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.009876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.009880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.110207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.110222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.110226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.210503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.210520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.210523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.310858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.310876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.310880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.411217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.411234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.411238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.511598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.511611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.511614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.612000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.612030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.612033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.715373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.715387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.715390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: err
2025-10-09T21:44:14.719275Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: TRANSPORT_UNAVAILABLE. Description: [ { <main>: Error: GRpc error: (14): mock 'unavailable' } { <main>: Error: Grpc error response on endpoint localhost:17666 } ]
2025-10-09T21:44:14.815700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.815713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.815717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.916085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.916100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.916103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.016400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.016415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.016418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.117125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.117139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.117143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.218272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.218289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.218292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.318906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.318934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.318937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.419475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.419488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.419490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.519737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.519753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.519756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.620266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.620281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.620284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.720366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.720382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.720385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.820841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.820857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.820861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.921064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.921078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.921081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.021455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.021468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.021471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.121770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.121785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.121789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.224877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.224893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.224896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.325096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.325113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.325117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.425370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.425386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.425389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.531369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.531385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.531388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.631734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.631751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.631755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.732289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.732311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.732315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.832503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.832519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.832522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.933285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.933302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.933305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.033170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.033189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.033192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.133536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.133553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.133556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.234156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.234173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.234176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.334291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.334308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.334311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.434713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.434728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.434731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.534966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.534981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.534984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.638011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.638045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.638047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.738192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.738207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.738210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.838524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.838539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.838543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.847278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:17.939022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.939038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.939041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.041533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.041550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.041552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.141643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.141675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.141678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.242197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.242217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.242220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.342385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.342414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.342417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.442687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.442713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.442716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.543155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.543169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.543173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.644553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.644570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.644573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.745129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.745156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.745159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.845591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.845615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.845618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.946553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.946570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.946573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.047820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.047839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.047842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.148227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.148242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.148245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.248828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.248847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.248851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.348947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.348963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.348967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.450149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.450168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.450171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.550198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.550216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.550219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.650600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.650620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.650624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.750998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.751015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.751018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.854085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.854101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.854104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.956651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.956677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.956679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.058145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.058164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.058168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.158421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.158452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.158456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.258793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.258823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.258828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.359394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.359412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.359415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.459506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.459523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.459526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.562379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.562408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.562412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.662741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.662769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.662772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.763112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.763129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.763132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.865567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.865586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.865589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.965835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.965853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.965855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.067192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.067209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.067212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.167593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.167608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.167611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.267960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.267976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.267979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.370022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.370058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.370061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.470233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.470250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.470253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.570541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.570559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.570561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.670872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.670890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.670893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.771197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.771211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.771215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.871813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.871830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.871833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.973299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.973316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.973319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.073348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.073364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.073367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.173684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.173702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.173706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.274024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.274056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.274060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.374443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.374468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.374478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.474744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.474763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.474766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.575108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.575129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.575131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.675391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.675412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.675417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.775828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.775851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.775856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.850736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:22.877521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.877537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.877540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.978024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.978064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.978068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.078417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.078438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.078441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: ok
2025-10-09T21:44:23.179969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.179984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.179988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.282341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.282363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.282366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.029307Z :RecreateObserver INFO: Random seed for debugging is 1760046264029296
2025-10-09T21:44:24.132667Z node 3 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[3:7559341144803785703:2088];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:44:24.133044Z node 3 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-10-09T21:44:24.135242Z node 3 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:44:24.135583Z node 4 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[4:7559341145259733899:2084];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:44:24.135604Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-10-09T21:44:24.137464Z node 3 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/owmb/0001ab/r3tmp/tmp8xcVoJ/pdisk_1.dat
2025-10-09T21:44:24.141838Z node 4 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:44:24.146228Z node 4 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:44:24.188931Z node 3 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:44:24.194223Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:44:24.194251Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:44:24.195621Z node 3 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 4 Cookie 4
2025-10-09T21:44:24.195911Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Connecting -> Connected
TServer::EnableGrpc on GrpcPort 28619, node 3
2025-10-09T21:44:24.204073Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/owmb/0001ab/r3tmp/yandexLpiqp7.tmp
2025-10-09T21:44:24.204087Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/owmb/0001ab/r3tmp/yandexLpiqp7.tmp
2025-10-09T21:44:24.205196Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/owmb/0001ab/r3tmp/yandexLpiqp7.tmp
2025-10-09T21:44:24.205279Z node 3 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-10-09T21:44:24.207608Z INFO: TTestServer started on Port 26751 GrpcPort 28619
TClient is connected to server localhost:26751
PQClient connected to localhost:28619
WaitRootIsUp 'Root'...
TClient::Ls request: Root
2025-10-09T21:44:24.236676Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:44:24.236720Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Disconnected -> Connecting
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-09T21:44:24.242848Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:44:24.243239Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
2025-10-09T21:44:24.247560Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976720657, at schemeshard: 72057594046644480
waiting...
waiting...
waiting...
2025-10-09T21:44:24.291640Z node 3 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:44:24.368723Z node 4 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:44:24.655179Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559341144803786642:2325], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:44:24.655201Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559341144803786667:2328], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:44:24.655210Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:44:24.655409Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559341144803786672:2330], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:44:24.655421Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:44:24.656009Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976720661:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-10-09T21:44:24.659490Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559341144803786705:2332], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:44:24.659615Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:44:24.659936Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559341144803786715:2334], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:44:24.659954Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:44:24.665598Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:44:24.668120Z node 3 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [3:7559341144803786671:2329], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720661 completed, doublechecking }
2025-10-09T21:44:24.698826Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:44:24.728375Z node 3 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [3:7559341144803786953:2821] txid# 281474976720664, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:44:24.737984Z node 3 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [3:7559341144803786986:2349], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:5:17: Error: At function: KiReadTable!
        <main>:5:17: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Versions]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:44:24.739635Z node 3 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=3&id=YTNhNGUyODYtMWQxNTFhMjMtYTBhNTE5NDgtMTY5MTJjZmQ=, ActorId: [3:7559341144803786638:2322], ActorState: ExecuteState, TraceId: 01k75excae4t0tz5e6av6tn6pa, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:44:24.739845Z node 3 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:44:24.745168Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720665:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:28619", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, false, 1000);
2025-10-09T21:44:24.801231Z node 3 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720666. Ctx: { TraceId: 01k75exce6e2hv12he5mjwwxar, Database: , SessionId: ydb://session/3?node_id=3&id=MWYzNDZjYzQtODM2MDdjM2EtNWEyYTlhZjAtNzNkNjQ0YjA=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [3:7559341144803787203:2996] 
2025-10-09T21:44:25.135068Z node 3 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:44:25.137631Z node 4 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:44:29.133612Z node 3 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[3:7559341144803785703:2088];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:44:29.133661Z node 3 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
2025-10-09T21:44:29.135874Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[4:7559341145259733899:2084];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:44:29.135909Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
2025-10-09T21:44:30.039789Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720672:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
waiting...
PQ Client: create topic: rt3.dc1--test-topic with 1 partitions
CallPersQueueGRPC request to localhost:28619
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:44:30.073661Z node 3 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:28619
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720673
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-09T21:44:30.090823Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][] pipe [3:7559341170573591491:3310] connected; active server actors: 1
2025-10-09T21:44:30.091571Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:44:30.091696Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1517: [72075186224037893][rt3.dc1--test-topic] updating configuration. Deleted partitions []. Added partitions [0]
2025-10-09T21:44:30.092442Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:940: [72075186224037893][rt3.dc1--test-topic] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-09T21:44:30.092484Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:44:30.092564Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:784: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-09T21:44:30.092576Z node 3 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:30.092589Z node 3 :PERSQUEUE INFO: pq_impl.cpp:609: [PQ: 72075186224037892] doesn't have tx writes info
2025-10-09T21:44:30.092724Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559341170573591490:3309], now have 1 active actors on pipe
2025-10-09T21:44:30.092839Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:44:30.093047Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:138: [72075186224037893][rt3.dc1--test-topic] BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) 
2025-10-09T21:44:30.093152Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 3, Generation 1
2025-10-09T21:44:30.093410Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [4:7559341171029538298:2352], now have 1 active actors on pipe
2025-10-09T21:44:30.094115Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:44:30.096691Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559341170573591561:3365], now have 1 active actors on pipe
2025-10-09T21:44:30.096759Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:44:30.097699Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:44:30.098009Z node 3 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:44:30.098068Z node 3 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:44:30.098136Z node 3 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:44:30.098173Z node 3 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [3:7559341170573591573:2462]
2025-10-09T21:44:30.098871Z node 3 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:44:30.098875Z node 3 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:44:30.098880Z node 3 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [3:7559341170573591573:2462]
2025-10-09T21:44:30.098890Z node 3 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:44:30.099179Z node 3 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-10-09T21:44:30.099187Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.099204Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.099207Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:30.099210Z node 3 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:44:30.099212Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.099214Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.099216Z node 3 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:44:30.099224Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:44:30.099299Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:44:30.099321Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:30.100275Z node 3 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720674. Ctx: { TraceId: 01k75exhm2457h7p99a8j6q0h0, Database: , SessionId: ydb://session/3?node_id=3&id=Nzk0ODM5NjktYjMyODVlOGYtMzA4MjU4ZTItYjQ4MWVmODU=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-10-09T21:44:30.100732Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:44:30.100775Z node 3 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046270143, TxId 281474976720673
2025-10-09T21:44:30.100786Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.100788Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.100812Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.100848Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:44:30.101804Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:44:30.101862Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:44:30.101874Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.101877Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.101878Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.101885Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:30.102016Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:395: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-10-09T21:44:30.102060Z node 3 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:44:30.102108Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:44:30.105692Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:44:30.105736Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:44:30.106496Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046270143
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046270143
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:28619
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:44:30.112677Z node 3 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
2025-10-09T21:44:30.114260Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-09T21:44:30.114543Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-09T21:44:30.114551Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:28619
2025-10-09T21:44:30.114927Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:44:30.115166Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:30.115184Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1
2025-10-09T21:44:30.115354Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:44:30.115395Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:48402
2025-10-09T21:44:30.115400Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:48402 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:44:30.115403Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:30.115839Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-09T21:44:30.115872Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:30.115874Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:30.115876Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:30.115882Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [3:7559341170573591669:2480] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:30.116488Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [3:7559341170573591669:2480] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:30.150567Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [3:7559341170573591669:2480] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:44:30.151329Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559341170573591705:2480] connected; active server actors: 1
2025-10-09T21:44:30.151641Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [3:7559341170573591669:2480] (SourceId=src, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src
2025-10-09T21:44:30.151654Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [3:7559341170573591669:2480] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:30.155961Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559341170573591705:2480] disconnected.
2025-10-09T21:44:30.155977Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559341170573591705:2480] disconnected; active server actors: 1
2025-10-09T21:44:30.155981Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559341170573591705:2480] disconnected no session
2025-10-09T21:44:30.175221Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [3:7559341170573591669:2480] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:30.175243Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [3:7559341170573591669:2480] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:44:30.175247Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [3:7559341170573591669:2480] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:30.175257Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:30.176938Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 3, Generation: 1
2025-10-09T21:44:30.176977Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559341170573591727:2480], now have 1 active actors on pipe
2025-10-09T21:44:30.177169Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:30.177182Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:30.177223Z node 3 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src|75ff86c3-126f734b-3170420a-5dcaa07a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src
2025-10-09T21:44:30.177236Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.177241Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.177244Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:30.177270Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.177285Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:30.177305Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:30.177988Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046270177 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:44:30.178054Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|75ff86c3-126f734b-3170420a-5dcaa07a_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:44:30.178318Z :INFO: [] MessageGroupId [src] SessionId [src|75ff86c3-126f734b-3170420a-5dcaa07a_0] Write session: close. Timeout = 0 ms
2025-10-09T21:44:30.178329Z :INFO: [] MessageGroupId [src] SessionId [src|75ff86c3-126f734b-3170420a-5dcaa07a_0] Write session will now close
2025-10-09T21:44:30.178336Z :DEBUG: [] MessageGroupId [src] SessionId [src|75ff86c3-126f734b-3170420a-5dcaa07a_0] Write session: aborting
2025-10-09T21:44:30.177522Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:30.177529Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:30.177546Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:30.177579Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1 partition: 0 MaxSeqNo: 0 sessionId: src|75ff86c3-126f734b-3170420a-5dcaa07a_0
2025-10-09T21:44:30.178618Z :INFO: [] MessageGroupId [src] SessionId [src|75ff86c3-126f734b-3170420a-5dcaa07a_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:44:30.178635Z :DEBUG: [] MessageGroupId [src] SessionId [src|75ff86c3-126f734b-3170420a-5dcaa07a_0] Write session: destroy
2025-10-09T21:44:30.180978Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId: src|75ff86c3-126f734b-3170420a-5dcaa07a_0 grpc read done: success: 0 data: 
2025-10-09T21:44:30.180991Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 1 sessionId: src|75ff86c3-126f734b-3170420a-5dcaa07a_0 grpc read failed
2025-10-09T21:44:30.181000Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 1 sessionId: src|75ff86c3-126f734b-3170420a-5dcaa07a_0 grpc closed
2025-10-09T21:44:30.181006Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1 sessionId: src|75ff86c3-126f734b-3170420a-5dcaa07a_0 is DEAD
2025-10-09T21:44:30.181346Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:44:30.181701Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [3:7559341170573591727:2480] destroyed
2025-10-09T21:44:30.181719Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:44:30.181726Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.181729Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.181732Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.197747Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.197770Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.197773Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Session was created
2025-10-09T21:44:30.298194Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.298214Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.298225Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.398427Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.398443Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.398446Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.498796Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.498815Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.498818Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.599134Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.599151Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.599154Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.699506Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.699524Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.699527Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.799835Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.799851Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.799854Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.900232Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.900248Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.900251Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.000505Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.000522Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.000526Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.100837Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.100856Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.100858Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.201245Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.201275Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.201278Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: err
2025-10-09T21:44:31.212995Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: TRANSPORT_UNAVAILABLE. Description: [ { <main>: Error: GRpc error: (14): mock 'unavailable' } { <main>: Error: Grpc error response on endpoint localhost:20110 } ]
2025-10-09T21:44:31.213026Z :ERROR: [/Root] [] [3c7740d3-31054300-107a84fd-83210faf] Federated state update failed. FederationState: { Status: TRANSPORT_UNAVAILABLE SelfLocation: "" Issues: { [ { <main>: Error: GRpc error: (14): mock 'unavailable' } { <main>: Error: Grpc error response on endpoint localhost:20110 } ] } }
>>> Got event: SessionClosed { Status: TRANSPORT_UNAVAILABLE Issues: "<main>: Error: GRpc error: (14): mock 'unavailable' <main>: Error: Grpc error response on endpoint localhost:20110 " }
Session2 was created
2025-10-09T21:44:31.301598Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.301618Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.301622Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.401904Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.401922Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.401925Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.502319Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.502337Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.502341Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.602659Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.602675Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.602678Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.703064Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.703084Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.703088Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.803355Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.803373Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.803380Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.903728Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.903748Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.903751Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.004077Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.004107Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.004114Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.106227Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.106245Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.106248Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.207362Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.207382Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.207385Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: ok
2025-10-09T21:44:32.218707Z :INFO: [/Root] [] [ea31fd7a-1ae7d384-2eb754ac-22e0b8e0] Open read subsessions to databases: { name: dc1, endpoint: localhost:28619, path: /Root }, { name: dc2, endpoint: localhost:28619, path: /Root }, { name: dc3, endpoint: localhost:28619, path: /Root }
2025-10-09T21:44:32.226399Z :INFO: [/Root] [/Root] [7e591faa-cccf7563-7046f161-9f03e383] Starting read session
2025-10-09T21:44:32.226417Z :DEBUG: [/Root] [/Root] [7e591faa-cccf7563-7046f161-9f03e383] Starting single session
2025-10-09T21:44:32.226741Z :DEBUG: [/Root] [/Root] [7e591faa-cccf7563-7046f161-9f03e383] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:44:32.226750Z :DEBUG: [/Root] [/Root] [7e591faa-cccf7563-7046f161-9f03e383] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:44:32.226756Z :DEBUG: [/Root] [/Root] [7e591faa-cccf7563-7046f161-9f03e383] [] Reconnecting session to cluster  in 0.000000s
2025-10-09T21:44:32.227680Z :INFO: [/Root] [/Root] [5e175502-75a764f8-3f33acfe-cf6b7f08] Starting read session
2025-10-09T21:44:32.227688Z :DEBUG: [/Root] [/Root] [5e175502-75a764f8-3f33acfe-cf6b7f08] Starting single session
2025-10-09T21:44:32.227719Z :DEBUG: [/Root] [/Root] [5e175502-75a764f8-3f33acfe-cf6b7f08] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:44:32.227723Z :DEBUG: [/Root] [/Root] [5e175502-75a764f8-3f33acfe-cf6b7f08] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:44:32.227727Z :DEBUG: [/Root] [/Root] [5e175502-75a764f8-3f33acfe-cf6b7f08] [] Reconnecting session to cluster  in 0.000000s
2025-10-09T21:44:32.227812Z :INFO: [/Root] [/Root] [b3fbe822-68af58ce-58717466-8b740e83] Starting read session
2025-10-09T21:44:32.227816Z :DEBUG: [/Root] [/Root] [b3fbe822-68af58ce-58717466-8b740e83] Starting single session
2025-10-09T21:44:32.227837Z :DEBUG: [/Root] [/Root] [b3fbe822-68af58ce-58717466-8b740e83] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:44:32.227840Z :DEBUG: [/Root] [/Root] [b3fbe822-68af58ce-58717466-8b740e83] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:44:32.227843Z :DEBUG: [/Root] [/Root] [b3fbe822-68af58ce-58717466-8b740e83] [] Reconnecting session to cluster  in 0.000000s
2025-10-09T21:44:32.229016Z :DEBUG: [/Root] [/Root] [b3fbe822-68af58ce-58717466-8b740e83] [] Successfully connected. Initializing session
2025-10-09T21:44:32.229174Z :DEBUG: [/Root] [/Root] [5e175502-75a764f8-3f33acfe-cf6b7f08] [] Successfully connected. Initializing session
2025-10-09T21:44:32.229239Z :DEBUG: [/Root] [/Root] [7e591faa-cccf7563-7046f161-9f03e383] [] Successfully connected. Initializing session
2025-10-09T21:44:32.237562Z :INFO: [/Root] [/Root] [5e175502-75a764f8-3f33acfe-cf6b7f08] [] Got InitResponse. ReadSessionId: shared/user_3_1_16024909861064697227_v1
2025-10-09T21:44:32.237588Z :DEBUG: [/Root] [/Root] [5e175502-75a764f8-3f33acfe-cf6b7f08] [] In ContinueReadingDataImpl, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:44:32.237703Z :DEBUG: [/Root] [/Root] [5e175502-75a764f8-3f33acfe-cf6b7f08] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:44:32.232528Z node 3 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:44:32.232543Z node 3 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 1
2025-10-09T21:44:32.232575Z node 3 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:44:32.232577Z node 3 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 2
2025-10-09T21:44:32.232583Z node 3 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:44:32.232585Z node 3 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 3
2025-10-09T21:44:32.236249Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:44:32.236357Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 1 consumer shared/user session shared/user_3_1_16024909861064697227_v1 read init: from# ipv6:[::1]:41884, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:44:32.236471Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 1 consumer shared/user session shared/user_3_1_16024909861064697227_v1 auth for : user
2025-10-09T21:44:32.236489Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:44:32.236515Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 2 consumer shared/user session shared/user_3_2_6199496941332143460_v1 read init: from# ipv6:[::1]:41884, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:44:32.236552Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 2 consumer shared/user session shared/user_3_2_6199496941332143460_v1 auth for : user
2025-10-09T21:44:32.236562Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 3 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:44:32.236582Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 read init: from# ipv6:[::1]:41884, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:44:32.236616Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 auth for : user
2025-10-09T21:44:32.236830Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 1 consumer shared/user session shared/user_3_1_16024909861064697227_v1 Handle describe topics response
2025-10-09T21:44:32.236849Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 1 consumer shared/user session shared/user_3_1_16024909861064697227_v1 auth is DEAD
2025-10-09T21:44:32.236861Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 Handle describe topics response
2025-10-09T21:44:32.236868Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 auth is DEAD
2025-10-09T21:44:32.236873Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 2 consumer shared/user session shared/user_3_2_6199496941332143460_v1 Handle describe topics response
2025-10-09T21:44:32.236880Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 2 consumer shared/user session shared/user_3_2_6199496941332143460_v1 auth is DEAD
2025-10-09T21:44:32.236887Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 1 consumer shared/user session shared/user_3_1_16024909861064697227_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:44:32.237225Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 1 consumer shared/user session shared/user_3_1_16024909861064697227_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:44:32.237313Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:44:32.237522Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:44:32.237546Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 2 consumer shared/user session shared/user_3_2_6199496941332143460_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:44:32.237729Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 2 consumer shared/user session shared/user_3_2_6199496941332143460_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:44:32.238231Z :INFO: [/Root] [/Root] [7e591faa-cccf7563-7046f161-9f03e383] [] Got InitResponse. ReadSessionId: shared/user_3_3_6478110654006757783_v1
2025-10-09T21:44:32.238244Z :DEBUG: [/Root] [/Root] [7e591faa-cccf7563-7046f161-9f03e383] [] In ContinueReadingDataImpl, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:44:32.238301Z :DEBUG: [/Root] [/Root] [7e591faa-cccf7563-7046f161-9f03e383] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:44:32.238322Z :INFO: [/Root] [/Root] [b3fbe822-68af58ce-58717466-8b740e83] [] Got InitResponse. ReadSessionId: shared/user_3_2_6199496941332143460_v1
2025-10-09T21:44:32.238325Z :DEBUG: [/Root] [/Root] [b3fbe822-68af58ce-58717466-8b740e83] [] In ContinueReadingDataImpl, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:44:32.238374Z :DEBUG: [/Root] [/Root] [b3fbe822-68af58ce-58717466-8b740e83] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:44:32.239129Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_6199496941332143460_v1 grpc read done: success# 1, data# { read_request { bytes_size: 524288 } }
2025-10-09T21:44:32.239201Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_6199496941332143460_v1 got read request: guid# 743130a5-723a2ac0-1b70c5df-b973cf5c
2025-10-09T21:44:32.239218Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 grpc read done: success# 1, data# { read_request { bytes_size: 524288 } }
2025-10-09T21:44:32.239228Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 got read request: guid# 9957c4b5-7530ca2c-74545706-92eb87aa
2025-10-09T21:44:32.239596Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559341179163526491:2526] connected; active server actors: 1
2025-10-09T21:44:32.239630Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559341179163526490:2527] connected; active server actors: 1
2025-10-09T21:44:32.239754Z node 4 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [3:7559341179163526490:2527] session shared/user_3_3_6478110654006757783_v1
2025-10-09T21:44:32.239767Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:44:32.239777Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:44:32.239784Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_3_3_6478110654006757783_v1" (Sender=[3:7559341179163526482:2527], Pipe=[3:7559341179163526490:2527], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:44:32.239788Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:44:32.239797Z node 4 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [3:7559341179163526491:2526] session shared/user_3_2_6199496941332143460_v1
2025-10-09T21:44:32.239802Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_3_2_6199496941332143460_v1" (Sender=[3:7559341179163526481:2526], Pipe=[3:7559341179163526491:2526], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:44:32.239810Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=2, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:44:32.239819Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_3_3_6478110654006757783_v1" (Sender=[3:7559341179163526482:2527], Pipe=[3:7559341179163526490:2527], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:44:32.239829Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_3_3_6478110654006757783_v1" sender [3:7559341179163526482:2527] lock partition 0 for ReadingSession "shared/user_3_3_6478110654006757783_v1" (Sender=[3:7559341179163526482:2527], Pipe=[3:7559341179163526490:2527], Partitions=[], ActiveFamilyCount=1) generation 1 step 1
2025-10-09T21:44:32.239843Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=2, desiredFamilyCount=0, allowPlusOne=1
2025-10-09T21:44:32.239847Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000034s
2025-10-09T21:44:32.240343Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 1 Session: "shared/user_3_3_6478110654006757783_v1" ClientId: "user" PipeClient { RawX1: 7559341179163526490 RawX2: 4503612512274911 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:44:32.240370Z node 3 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:44:32.240435Z node 3 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1, pipe: [3:7559341179163526493:2534]
2025-10-09T21:44:32.240449Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559341179163526493:2534], now have 1 active actors on pipe
2025-10-09T21:44:32.240456Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:32.240462Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:32.240476Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_3_3_6478110654006757783_v1 on pipe: [3:7559341179163526493:2534]
2025-10-09T21:44:32.240493Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.240499Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.240502Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:32.240521Z node 3 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_3_3_6478110654006757783_v1
2025-10-09T21:44:32.240524Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.240556Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:44:32.240742Z node 3 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_3_3_6478110654006757783_v1:1 with generation 1
2025-10-09T21:44:32.241878Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer shared/user session shared/user_3_1_16024909861064697227_v1 grpc read done: success# 1, data# { read_request { bytes_size: 524288 } }
2025-10-09T21:44:32.241924Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 1 consumer shared/user session shared/user_3_1_16024909861064697227_v1 got read request: guid# a3735794-4336abc5-c8dfc3f6-fd6bcb3d
2025-10-09T21:44:32.242162Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559341179163526489:2525] connected; active server actors: 1
2025-10-09T21:44:32.242958Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:44:32.242991Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:44:32.243001Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.243004Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.243006Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.243016Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:32.243030Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
>>> Got event: 2025-10-09T21:44:32.243127Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 0 SizeLag: 0 WriteTimestampEstimateMS: 1760046272206 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:44:32.243139Z node 3 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 0 readOffset 0 committedOffset 0
2025-10-09T21:44:32.243159Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 sending to client partition status
StartPartitionSession { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc1 Database path: /Root Database id: account-dc1 CommittedOffset: 0 EndOffset: 0 }
2025-10-09T21:44:32.243694Z :INFO: [/Root] [/Root] [7e591faa-cccf7563-7046f161-9f03e383] Closing read session. Close timeout: 0.000000s
2025-10-09T21:44:32.243712Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:0:1:0:0
2025-10-09T21:44:32.243720Z :INFO: [/Root] [/Root] [7e591faa-cccf7563-7046f161-9f03e383] Counters: { Errors: 0 CurrentSessionLifetimeMs: 17 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:44:32.243741Z :NOTICE: [/Root] [/Root] [7e591faa-cccf7563-7046f161-9f03e383] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Close with zero timeout " }
2025-10-09T21:44:32.243750Z :DEBUG: [/Root] [/Root] [7e591faa-cccf7563-7046f161-9f03e383] [] Abort session to cluster
2025-10-09T21:44:32.243931Z :INFO: [/Root] [/Root] [5e175502-75a764f8-3f33acfe-cf6b7f08] Closing read session. Close timeout: 0.000000s
2025-10-09T21:44:32.243942Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset):
2025-10-09T21:44:32.243947Z :INFO: [/Root] [/Root] [5e175502-75a764f8-3f33acfe-cf6b7f08] Counters: { Errors: 0 CurrentSessionLifetimeMs: 16 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:44:32.243956Z :NOTICE: [/Root] [/Root] [5e175502-75a764f8-3f33acfe-cf6b7f08] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Close with zero timeout " }
2025-10-09T21:44:32.243960Z :DEBUG: [/Root] [/Root] [5e175502-75a764f8-3f33acfe-cf6b7f08] [] Abort session to cluster
2025-10-09T21:44:32.243984Z :INFO: [/Root] [/Root] [b3fbe822-68af58ce-58717466-8b740e83] Closing read session. Close timeout: 0.000000s
2025-10-09T21:44:32.243990Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset):
2025-10-09T21:44:32.243993Z :INFO: [/Root] [/Root] [b3fbe822-68af58ce-58717466-8b740e83] Counters: { Errors: 0 CurrentSessionLifetimeMs: 16 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:44:32.243998Z :NOTICE: [/Root] [/Root] [b3fbe822-68af58ce-58717466-8b740e83] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Close with zero timeout " }
2025-10-09T21:44:32.244001Z :DEBUG: [/Root] [/Root] [b3fbe822-68af58ce-58717466-8b740e83] [] Abort session to cluster
2025-10-09T21:44:32.244023Z :INFO: [/Root] [/Root] [b3fbe822-68af58ce-58717466-8b740e83] Closing read session. Close timeout: 0.000000s
2025-10-09T21:44:32.244026Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset):
2025-10-09T21:44:32.244029Z :INFO: [/Root] [/Root] [b3fbe822-68af58ce-58717466-8b740e83] Counters: { Errors: 0 CurrentSessionLifetimeMs: 16 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:44:32.244034Z :NOTICE: [/Root] [/Root] [b3fbe822-68af58ce-58717466-8b740e83] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-09T21:44:32.244059Z :INFO: [/Root] [/Root] [5e175502-75a764f8-3f33acfe-cf6b7f08] Closing read session. Close timeout: 0.000000s
2025-10-09T21:44:32.244063Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset):
2025-10-09T21:44:32.244065Z :INFO: [/Root] [/Root] [5e175502-75a764f8-3f33acfe-cf6b7f08] Counters: { Errors: 0 CurrentSessionLifetimeMs: 16 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:44:32.244070Z :NOTICE: [/Root] [/Root] [5e175502-75a764f8-3f33acfe-cf6b7f08] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-09T21:44:32.244078Z :INFO: [/Root] [/Root] [7e591faa-cccf7563-7046f161-9f03e383] Closing read session. Close timeout: 0.000000s
2025-10-09T21:44:32.244083Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:0:1:0:0
2025-10-09T21:44:32.244086Z :INFO: [/Root] [/Root] [7e591faa-cccf7563-7046f161-9f03e383] Counters: { Errors: 0 CurrentSessionLifetimeMs: 17 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:44:32.244091Z :NOTICE: [/Root] [/Root] [7e591faa-cccf7563-7046f161-9f03e383] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-09T21:44:32.244874Z node 4 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [3:7559341179163526489:2525] session shared/user_3_1_16024909861064697227_v1
2025-10-09T21:44:32.244898Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_3_1_16024909861064697227_v1" (Sender=[3:7559341179163526480:2525], Pipe=[3:7559341179163526489:2525], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:44:32.244904Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:44:32.244915Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=3, Families=1, UnradableFamilies=0 [], RequireBalancing=0 []
2025-10-09T21:44:32.244921Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=3, desiredFamilyCount=0, allowPlusOne=1
2025-10-09T21:44:32.244927Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000009s
2025-10-09T21:44:32.245858Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559341179163526490:2527] disconnected.
2025-10-09T21:44:32.245869Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559341179163526490:2527] disconnected; active server actors: 1
2025-10-09T21:44:32.245873Z node 4 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559341179163526490:2527] client user disconnected session shared/user_3_3_6478110654006757783_v1
2025-10-09T21:44:32.245887Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:44:32.245898Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=2, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:44:32.245909Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_3_1_16024909861064697227_v1" (Sender=[3:7559341179163526480:2525], Pipe=[3:7559341179163526489:2525], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:44:32.245918Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_3_1_16024909861064697227_v1" sender [3:7559341179163526480:2525] lock partition 0 for ReadingSession "shared/user_3_1_16024909861064697227_v1" (Sender=[3:7559341179163526480:2525], Pipe=[3:7559341179163526489:2525], Partitions=[], ActiveFamilyCount=1) generation 1 step 2
2025-10-09T21:44:32.245944Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=2, desiredFamilyCount=0, allowPlusOne=1
2025-10-09T21:44:32.245948Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000047s
2025-10-09T21:44:32.245348Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 grpc read done: success# 0, data# {  }
2025-10-09T21:44:32.245358Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:125: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 grpc read failed
2025-10-09T21:44:32.245363Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:92: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 grpc closed
2025-10-09T21:44:32.245371Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 3 consumer shared/user session shared/user_3_3_6478110654006757783_v1 is DEAD
2025-10-09T21:44:32.245686Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer shared/user session shared/user_3_1_16024909861064697227_v1 grpc read done: success# 0, data# {  }
2025-10-09T21:44:32.245689Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:125: session cookie 1 consumer shared/user session shared/user_3_1_16024909861064697227_v1 grpc read failed
2025-10-09T21:44:32.245692Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:92: session cookie 1 consumer shared/user session shared/user_3_1_16024909861064697227_v1 grpc closed
2025-10-09T21:44:32.245695Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 1 consumer shared/user session shared/user_3_1_16024909861064697227_v1 is DEAD
2025-10-09T21:44:32.245867Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_6199496941332143460_v1 grpc read done: success# 0, data# {  }
2025-10-09T21:44:32.245869Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:125: session cookie 2 consumer shared/user session shared/user_3_2_6199496941332143460_v1 grpc read failed
2025-10-09T21:44:32.245871Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:92: session cookie 2 consumer shared/user session shared/user_3_2_6199496941332143460_v1 grpc closed
2025-10-09T21:44:32.245874Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 2 consumer shared/user session shared/user_3_2_6199496941332143460_v1 is DEAD
2025-10-09T21:44:32.246259Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559341179163526489:2525] disconnected.
2025-10-09T21:44:32.246269Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559341179163526489:2525] disconnected; active server actors: 1
2025-10-09T21:44:32.246272Z node 4 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559341179163526489:2525] client user disconnected session shared/user_3_1_16024909861064697227_v1
2025-10-09T21:44:32.246281Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:44:32.246285Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559341179163526491:2526] disconnected.
2025-10-09T21:44:32.246288Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559341179163526491:2526] disconnected; active server actors: 1
2025-10-09T21:44:32.246300Z node 4 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559341179163526491:2526] client user disconnected session shared/user_3_2_6199496941332143460_v1
2025-10-09T21:44:32.246546Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_3_3_6478110654006757783_v1
2025-10-09T21:44:32.246563Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [3:7559341179163526493:2534] destroyed
2025-10-09T21:44:32.246582Z node 3 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_3_3_6478110654006757783_v1
2025-10-09T21:44:32.307161Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.307181Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.307184Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.407520Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.407540Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.407557Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.507861Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.507878Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.507900Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.990977Z :WaitEventBlocksBeforeDiscovery INFO: Random seed for debugging is 1760046170990972
2025-10-09T21:42:51.097649Z node 1 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7559340747434863712:2089];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:51.097691Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-10-09T21:42:51.109742Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:123: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, got undelivered to scheme cache: ActorUnknown
2025-10-09T21:42:51.142384Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/owmb/0001b1/r3tmp/tmpa1I2l3/pdisk_1.dat
2025-10-09T21:42:51.156116Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:51.217274Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:51.217299Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:51.221980Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:51.232568Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:42:51.240439Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
TServer::EnableGrpc on GrpcPort 61115, node 1
2025-10-09T21:42:51.252151Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:51.257804Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:51.257840Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:51.260376Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-10-09T21:42:51.260774Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:51.266272Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/owmb/0001b1/r3tmp/yandexPGXNoM.tmp
2025-10-09T21:42:51.266281Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/owmb/0001b1/r3tmp/yandexPGXNoM.tmp
2025-10-09T21:42:51.272354Z INFO: TTestServer started on Port 28854 GrpcPort 61115
2025-10-09T21:42:51.296312Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/owmb/0001b1/r3tmp/yandexPGXNoM.tmp
2025-10-09T21:42:51.296447Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
TClient is connected to server localhost:28854
PQClient connected to localhost:61115
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-09T21:42:51.325343Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
waiting...
2025-10-09T21:42:51.352137Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976720658, at schemeshard: 72057594046644480
waiting...
waiting...
2025-10-09T21:42:51.658293Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340747434864637:2323], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.658347Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.662171Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340747434864674:2328], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.662195Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340747434864675:2329], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.662222Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.667029Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976720661:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-10-09T21:42:51.682074Z node 1 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [1:7559340747434864678:2330], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720661 completed, doublechecking }
2025-10-09T21:42:51.779893Z node 1 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [1:7559340747434864761:2663] txid# 281474976720662, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:51.784207Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:51.793336Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [2:7559340747327404623:2301], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:51.794252Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=2&id=NjRhYWRjODMtMWFmNGM5NzEtZjk5ZmYyZjAtZDMwZTgyNmI=, ActorId: [2:7559340747327404596:2295], ActorState: ExecuteState, TraceId: 01k75ethj0639c6r9qcyrv4snn, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:51.794842Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:51.798475Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [1:7559340747434864813:2340], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:51.799421Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=1&id=ZTk3MGFjZGItYTMxN2U3NjktNTRhN2FhOWYtMmQ5OTA1, ActorId: [1:7559340747434864633:2320], ActorState: ExecuteState, TraceId: 01k75ethg6b89g36k6d7pmepzk, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:51.799536Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:51.844292Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:51.882230Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720665:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:61115", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, false, 1000);
2025-10-09T21:42:51.989364Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720666. Ctx: { TraceId: 01k75ethrret9qcqh5zdkhbhz5, Database: , SessionId: ydb://session/3?node_id=1&id=Mjg4YzY4NjEtYmYyZTJiYjMtYmI4ZmExNWMtMmY0ZmVhMjg=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [1:7559340751729832445:2939] 
2025-10-09T21:42:52.102118Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:52.155475Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:56.097848Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7559340747434863712:2089];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:56.097900Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
2025-10-09T21:42:57.103288Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720672:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
waiting...
PQ Client: create topic: rt3.dc1--test-topic with 1 partitions
CallPersQueueGRPC request to localhost:61115
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:57.142524Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:61115
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720673
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-09T21:42:57.177074Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][] pipe [1:7559340773204669334:3195] connected; active server actors: 1
2025-10-09T21:42:57.177249Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1517: [72075186224037893][rt3.dc1--test-topic] updating configuration. Deleted partitions []. Added partitions [0]
2025-10-09T21:42:57.177460Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:940: [72075186224037893][rt3.dc1--test-topic] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-09T21:42:57.183022Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:138: [72075186224037893][rt3.dc1--test-topic] BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) 
2025-10-09T21:42:57.186914Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-10-09T21:42:57.185672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:57.186408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:57.186474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:784: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-09T21:42:57.186481Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:57.186511Z node 2 :PERSQUEUE INFO: pq_impl.cpp:609: [PQ: 72075186224037892] doesn't have tx writes info
2025-10-09T21:42:57.186624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340773204669352:2432], now have 1 active actors on pipe
2025-10-09T21:42:57.186632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340773204669333:3194], now have 1 active actors on pipe
2025-10-09T21:42:57.186969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.188684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.193438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340773204669388:3237], now have 1 active actors on pipe
2025-10-09T21:42:57.193514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.196392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.196606Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:57.196629Z node 2 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:57.196721Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:57.196814Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340773097208861:2381]
2025-10-09T21:42:57.197587Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:57.197597Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:57.197602Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [2:7559340773097208861:2381]
2025-10-09T21:42:57.197609Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:57.197957Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-10-09T21:42:57.197969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.197995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.197998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:57.198001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:57.198003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.198005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.198007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:57.198014Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:57.198167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.198192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:57.199632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.199669Z node 2 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046177239, TxId 281474976720673
2025-10-09T21:42:57.199674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.199676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.199699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.199749Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:57.200467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:57.200486Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720674. Ctx: { TraceId: 01k75etpwycwnjk0f3c72b40qy, Database: , SessionId: ydb://session/3?node_id=1&id=ZmM2ODhhYmUtZjE0OTUxYTMtZTg2ODIxZjgtOTQ4MjFlMmI=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-10-09T21:42:57.203777Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-10-09T21:42:57.203860Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:57.203867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.203870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.203872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.203881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:57.203985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:395: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-10-09T21:42:57.204004Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:57.204054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.207022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.207051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.208426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046177239
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046177239
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:61115
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
2025-10-09T21:42:57.219766Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-09T21:42:57.220005Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-09T21:42:57.220011Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:61115
2025-10-09T21:42:57.218375Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-10-09T21:42:57.222311Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:57.223333Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:57.223351Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1
2025-10-09T21:42:57.223461Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:57.223486Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:35528
2025-10-09T21:42:57.223490Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:35528 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:57.223493Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:57.223872Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-09T21:42:57.223912Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:57.223914Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:57.223915Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:57.223920Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340773204669463:2449] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:57.224370Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340773204669463:2449] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:57.273687Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [1:7559340773204669463:2449] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:57.274697Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340773204669497:2449] connected; active server actors: 1
2025-10-09T21:42:57.279283Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [1:7559340773204669463:2449] (SourceId=src, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src
2025-10-09T21:42:57.279299Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340773204669463:2449] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:57.284293Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340773204669497:2449] disconnected.
2025-10-09T21:42:57.284306Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340773204669497:2449] disconnected; active server actors: 1
2025-10-09T21:42:57.284310Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340773204669497:2449] disconnected no session
2025-10-09T21:42:57.294251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.294266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.294269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.318840Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340773204669463:2449] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:57.318856Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340773204669463:2449] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:42:57.318860Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340773204669463:2449] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:57.318869Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:57.319876Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1
2025-10-09T21:42:57.321439Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046177321 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:57.321480Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|b8f0cd03-3d3211ef-4b970ff4-ea992d59_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:57.321575Z :INFO: [] MessageGroupId [src] SessionId [src|b8f0cd03-3d3211ef-4b970ff4-ea992d59_0] Write session: close. Timeout = 0 ms
2025-10-09T21:42:57.321601Z :INFO: [] MessageGroupId [src] SessionId [src|b8f0cd03-3d3211ef-4b970ff4-ea992d59_0] Write session will now close
2025-10-09T21:42:57.321607Z :DEBUG: [] MessageGroupId [src] SessionId [src|b8f0cd03-3d3211ef-4b970ff4-ea992d59_0] Write session: aborting
2025-10-09T21:42:57.321019Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1 partition: 0 MaxSeqNo: 0 sessionId: src|b8f0cd03-3d3211ef-4b970ff4-ea992d59_0
2025-10-09T21:42:57.321776Z :INFO: [] MessageGroupId [src] SessionId [src|b8f0cd03-3d3211ef-4b970ff4-ea992d59_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:42:57.321781Z :DEBUG: [] MessageGroupId [src] SessionId [src|b8f0cd03-3d3211ef-4b970ff4-ea992d59_0] Write session: destroy
2025-10-09T21:42:57.319752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340773204669520:2449], now have 1 active actors on pipe
2025-10-09T21:42:57.320066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.320077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.320106Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src|b8f0cd03-3d3211ef-4b970ff4-ea992d59_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src
2025-10-09T21:42:57.320119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.320124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.320127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:57.320141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.320157Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:57.320175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:57.320865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.320871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.320888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:57.322947Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId: src|b8f0cd03-3d3211ef-4b970ff4-ea992d59_0 grpc read done: success: 0 data: 
2025-10-09T21:42:57.322954Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 1 sessionId: src|b8f0cd03-3d3211ef-4b970ff4-ea992d59_0 grpc read failed
2025-10-09T21:42:57.322960Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 1 sessionId: src|b8f0cd03-3d3211ef-4b970ff4-ea992d59_0 grpc closed
2025-10-09T21:42:57.322965Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1 sessionId: src|b8f0cd03-3d3211ef-4b970ff4-ea992d59_0 is DEAD
2025-10-09T21:42:57.323244Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:42:57.323557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340773204669520:2449] destroyed
2025-10-09T21:42:57.323572Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:42:57.323581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.323585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.323587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Before ReadSession was created
Session was created
Returned from WaitEvent
2025-10-09T21:42:57.394734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.394749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.394752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.495925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.495941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.495944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.596495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.596513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.596517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.698120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.698139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.698142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.798241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.798255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.798258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.902098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.902114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.902117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.002495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.002511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.002514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.102840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.102859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.102861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.206130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.206148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.206151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.307781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.307798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.307801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Session blocked successfully
Session closed gracefully
2025-10-09T21:42:58.407986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.408003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.408007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.508389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.508408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.508412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.608678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.608695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.608698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.709034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.709054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.709056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.810294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.810310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.810312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.914116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.914133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.914136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.018127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.018143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.018146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.118173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.118191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.118195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.218927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.218945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.218949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.322153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.322173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.322177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.424822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.424838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.424842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.527004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.527019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.527023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.627074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.627092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.627096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.727606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.727621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.727624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.827770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.827788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.827791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.928158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.928174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.928177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.029596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.029620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.029643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.129866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.129888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.129891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.234100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.234121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.234124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.338159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.338177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.338180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.438568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.438585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.438589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.542148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.542172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.542175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.642353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.642369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.642372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.746137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.746155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.746170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.846739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.846756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.846759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.947007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.947024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.947026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.047370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.047386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.047390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.153133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.153153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.153157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.250725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.250740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.250743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.350964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.350980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.350983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.451291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.451306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.451310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.552171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.552188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.552191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.652498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.652516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.652520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.752916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.752934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.752937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.854151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.854166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.854169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.954766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.954781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.954785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.054984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.055008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.055012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.155305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.155334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.155337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.194227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:02.255708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.255725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.255740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.356007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.356024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.356027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.458149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.458180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.458183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.559007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.559025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.559028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.659195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.659218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.659222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.759506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.759524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.759527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.860137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.860153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.860156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.962111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.962131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.962134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.064226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.064244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.064247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.164554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.164570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.164574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.264829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.264845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.264848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.365184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.365201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.365204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.465596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.465610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.465612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.565947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.565965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.565968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.666234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.666258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.666261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.766972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.766992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.766996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.866939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.866956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.866959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:03.970532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:03.970550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:03.970553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.070704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.070722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.070730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.171160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.171174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.171178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.274258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.274276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.274280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.378340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.378362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.378366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.478507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.478523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.478526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.582098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.582115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.582118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.682091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.682111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.682114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.782394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.782410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.782422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.882744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.882762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.882766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.985689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.985704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.985707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.090930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.090948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.090951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.191478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.191502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.191506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.294605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.294623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.294627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.398206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.398225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.398229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.502094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.502111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.502114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.602043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.602056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.602058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.701956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.701987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.701990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.802296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.802327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.802330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.902707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.902725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.902745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.003045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.003064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.003067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.103354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.103385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.103388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.180162Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard_impl.cpp:7721: Cannot get console configs
2025-10-09T21:43:06.180179Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:43:06.203715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.203745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.203748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.304061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.304077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.304080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.405517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.405536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.405539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.505786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.505803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.505806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.611457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.611472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.611475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.711603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.711690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.711693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.811958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.811980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.811982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.913294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.913311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.913314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.013753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.013769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.013772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.114075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.114090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.114093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.194656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:07.214406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.214433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.214436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.314754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.314770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.314773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.415066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.415083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.415085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.515530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.515547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.515550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.618196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.618210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.618213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.718707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.718722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.718726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.819108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.819124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.819127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.922997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.923010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.923014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.023340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.023359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.023363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.123948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.123966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.123970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.224037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.224056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.224060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.326201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.326219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.326222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.426306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.426322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.426325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.526656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.526670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.526673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.627049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.627061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.627064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.729892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.729904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.729907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.834151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.834170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.834172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.934199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.934216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.934219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.034556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.034581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.034585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.138321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.138342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.138346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.238058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.238074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.238077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.342169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.342186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.342189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.445695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.445710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.445713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.545560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.545577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.545580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.649357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.649376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.649379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.748030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.748043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.748046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.848115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.848126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.848129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.948403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.948416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.948419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.048969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.048986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.048990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.149178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.149193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.149197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.251000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.251019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.251022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.350454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.350473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.350476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.454427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.454442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.454445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.554676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.554692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.554695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.658102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.658121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.658124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.761504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.761521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.761524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.861897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.861911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.861913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.965219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.965249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.965255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.065172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.065187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.065190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.165477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.165494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.165498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.265795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.265810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.265813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.366176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.366194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.366198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.470103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.470124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.470129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.571844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.571861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.571864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.672381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.672405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.672408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.774263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.774281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.774284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.874617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.874635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.874639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.974940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.974955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.974958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.078349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.078371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.078374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.179585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.179602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.179606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.198200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:12.279642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.279660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.279663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.379972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.379989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.379993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.481138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.481156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.481160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.581656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.581671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.581676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.682070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.682097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.682101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.782371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.782395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.782398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.882734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.882750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.882754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.983092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.983109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.983113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.083406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.083420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.083422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.184505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.184605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.184614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.284790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.284807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.284810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.386325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.386343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.386347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.486924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.486937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.486940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.590203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.590220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.590223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.690177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.690189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.690191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.791951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.791969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.791973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.892423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.892440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.892443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.994403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.994432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.994436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.094649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.094664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.094667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.195019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.195036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.195039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.295372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.295388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.295392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.395696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.395710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.395713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.496074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.496092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.496095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.596484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.596498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.596501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.696757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.696775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.696779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.797362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.797380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.797384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.897477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.897494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.897498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.997795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.997822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.997825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.098300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.098326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.098330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.199025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.199046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.199049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.299395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.299410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.299413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.399755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.399771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.399774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.501407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.501421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.501424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.601487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.601503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.601506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.701811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.701824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.701827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.802187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.802210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.802213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.902522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.902553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.902556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.002877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.002893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.002896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.103240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.103256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.103258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.203591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.203609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.203612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.305308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.305328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.305331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.405835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.405877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.405880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.506192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.506210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.506214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.606647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.606665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.606669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.707020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.707036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.707039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.824592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.824610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.824614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.924968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.924983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.924987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.025307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.025322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.025325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.125677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.125693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.125696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.198487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:17.226056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.226077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.226081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.326390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.326407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.326411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.426767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.426782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.426785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.527041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.527058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.527061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.634126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.634148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.634152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.733961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.733981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.733985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.835279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.835295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.835299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.935673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.935691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.935694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.038208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.038223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.038227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.138752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.138770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.138774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.238797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.238812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.238815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.339137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.339152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.339156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.439463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.439481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.439484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.540175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.540192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.540195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.642134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.642152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.642155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.742072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.742089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.742104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.842460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.842477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.842481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.942712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.942727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.942731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.043139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.043163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.043166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.147282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.147299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.147302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.247431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.247448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.247452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.347770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.347788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.347791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.448108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.448124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.448127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.548473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.548488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.548491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.648794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.648809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.648811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.749208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.749222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.749225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.849675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.849693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.849696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.949993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.950022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.950025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.052533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.052551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.052555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.152625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.152640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.152643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.252987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.253002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.253006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.353326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.353344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.353347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.453693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.453711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.453715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.554383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.554400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.554403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.654587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.654614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.654617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.754859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.754896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.754899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.856042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.856056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.856059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.956443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.956472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.956476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.060466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.060481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.060485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.160766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.160785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.160789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.261132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.261149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.261153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.361430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.361458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.361461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.462357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.462374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.462377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.566098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.566123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.566126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.666198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.666213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.666216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.770096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.770120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.770124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.873777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.873801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.873805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.974402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.974465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.974483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.074692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.074711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.074715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.175207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.175226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.175229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.199279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:22.276007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.276025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.276028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.376294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.376310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.376314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.476612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.476627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.476630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.577624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.577642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.577646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.677968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.677984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.677988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.778245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.778266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.778269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.878542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.878556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.878558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.978878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.978892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.978896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.079835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.079851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.079854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.180173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.180191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.180194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.280440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.280465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.280468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.382140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.382158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.382161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.482676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.482689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.482691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.586250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.586268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.586270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.686741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.686756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.686759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.790506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.790521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.790524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.892304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.892325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.892328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.999492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.999513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.999516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.097588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.097606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.097610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.197932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.197950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.197953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.298303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.298326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.298331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.398738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.398756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.398759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.498990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.499004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.499007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.601892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.601909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.601913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.702870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.702885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.702889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.803810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.803828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.803831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.906137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.906153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.906155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.006499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.006517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.006530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.110269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.110289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.110293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.211753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.211789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.211797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.314176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.314219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.314224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.415700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.415730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.415734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.515896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.515929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.515935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.621428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.621451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.621458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.722156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.722180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.722189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.823124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.823147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.823153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.923325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.923339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.923342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.025015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.025030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.025033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.125273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.125289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.125292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.225493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.225508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.225511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.326232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.326247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.326250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.426712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.426731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.426734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.527017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.527032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.527035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.627453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.627470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.627474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.730468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.730484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.730487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.830655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.830671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.830675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.933681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.933698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.933701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.034922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.034938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.034941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.135292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.135308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.135311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.183154Z node 2 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 0 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:43:27.182578Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:43:27.182605Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 1
2025-10-09T21:43:27.183568Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 1 DataSize: 0 UsedReserveSize: 0
2025-10-09T21:43:27.183738Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:43:27.184092Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72057594046644480, NodeId 1, Generation 2
2025-10-09T21:43:27.202705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:27.238135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.238152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.238156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.338585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.338603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.338607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.438919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.438934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.438937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.540563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.540581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.540584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.640640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.640655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.640658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.744872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.744886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.744888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.848009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.848025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.848028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.948391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.948412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.948420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.048693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.048710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.048713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.149059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.149075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.149078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.249594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.249607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.249610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.354210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.354234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.354238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.454218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.454233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.454237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.558284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.558299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.558302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.662120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.662136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.662140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.765424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.765441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.765444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.865891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.865909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.865912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.968726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.968743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.968747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.070324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.070350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.070354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.174564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.174580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.174582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.274962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.274980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.274983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.376820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.376836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.376840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.477165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.477184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.477188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.577991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.578007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.578011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.679868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.679886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.679890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.779428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.779444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.779448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.879900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.879917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.879920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.982181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.982198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.982202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.082928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.082947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.082951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.183319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.183340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.183344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.283595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.283614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.283618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.383959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.383976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.383980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.484295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.484330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.484334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.584702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.584718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.584722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.684994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.685010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.685013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.785332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.785346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.785349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.885715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.885729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.885732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.986066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.986084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.986088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.086510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.086526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.086529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.186752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.186768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.186771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.287098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.287112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.287116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.387455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.387471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.387474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.490492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.490510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.490513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.590718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.590734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.590737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.691739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.691758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.691761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.791964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.791990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.791993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.892275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.892298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.892301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.992687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.992703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.992706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.093041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.093056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.093060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.193592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.193608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.193611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.202674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:32.293829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.293849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.293852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.394027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.394062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.394065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.498571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.498589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.498592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.975047Z :SimpleHandlers INFO: Random seed for debugging is 1760046212975040
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/owmb/0001b1/r3tmp/tmp3ZWEZF/pdisk_1.dat
2025-10-09T21:43:33.069060Z node 4 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:123: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, got undelivered to scheme cache: ActorUnknown
2025-10-09T21:43:33.069298Z node 4 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:43:33.069340Z node 4 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[4:7559340926496809292:2264];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:43:33.069449Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-10-09T21:43:33.069926Z node 3 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:43:33.075568Z node 3 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:43:33.075599Z node 3 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
2025-10-09T21:43:33.131571Z node 3 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:43:33.137083Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:43:33.137119Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:43:33.139506Z node 3 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 4 Cookie 4
2025-10-09T21:43:33.139774Z node 4 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:43:33.140028Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Connecting -> Connected
TServer::EnableGrpc on GrpcPort 13769, node 3
2025-10-09T21:43:33.158799Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/owmb/0001b1/r3tmp/yandexswgLUf.tmp
2025-10-09T21:43:33.158811Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/owmb/0001b1/r3tmp/yandexswgLUf.tmp
2025-10-09T21:43:33.158876Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/owmb/0001b1/r3tmp/yandexswgLUf.tmp
2025-10-09T21:43:33.158929Z node 3 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-10-09T21:43:33.162879Z INFO: TTestServer started on Port 28546 GrpcPort 13769
2025-10-09T21:43:33.175026Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:43:33.175057Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Disconnected -> Connecting
TClient is connected to server localhost:28546
2025-10-09T21:43:33.178908Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Connecting -> Connected
PQClient connected to localhost:13769
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-09T21:43:33.184037Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
waiting...
waiting...
2025-10-09T21:43:33.218493Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976720659, at schemeshard: 72057594046644480
waiting...
2025-10-09T21:43:33.243422Z node 3 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:43:33.531884Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340926994052457:2325], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:43:33.531904Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340926994052478:2328], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:43:33.531923Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:43:33.532714Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340926994052519:2330], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:43:33.532746Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:43:33.532836Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976720661:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-10-09T21:43:33.532881Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340926994052521:2331], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:43:33.532897Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:43:33.533584Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340926994052528:2333], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:43:33.533599Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:43:33.533651Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340926994052530:2334], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:43:33.533656Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:43:33.540587Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:43:33.541529Z node 3 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [3:7559340926994052492:2329], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720661 completed, doublechecking }
2025-10-09T21:43:33.568111Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:43:33.577965Z node 4 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [4:7559340926496809438:2301], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:5:17: Error: At function: KiReadTable!
        <main>:5:17: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Versions]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:43:33.578797Z node 4 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=4&id=NWQ2MWIwNC1hY2Q4MTZkLWE3YjBiZGI5LTNjMTRlMWZl, ActorId: [4:7559340926496809413:2295], ActorState: ExecuteState, TraceId: 01k75evte47a7e68vyvkfx3ent, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:43:33.578967Z node 4 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:43:33.627990Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:43:33.643020Z node 3 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [3:7559340926994052913:2912] txid# 281474976720665, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:43:33.649407Z node 3 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [3:7559340926994052926:2363], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:5:17: Error: At function: KiReadTable!
        <main>:5:17: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Versions]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:43:33.650210Z node 3 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=3&id=YTczNjhlZjQtMmJkZmFmOTMtNmNjODhhY2YtODczZjQxM2Y=, ActorId: [3:7559340926994052450:2321], ActorState: ExecuteState, TraceId: 01k75evtcv0bn1z14jz5nzxmkr, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:43:33.650359Z node 3 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:13769", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, false, 1000);
2025-10-09T21:43:33.709355Z node 3 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720666. Ctx: { TraceId: 01k75evthdb7svgd37tjg309n3, Database: , SessionId: ydb://session/3?node_id=3&id=NmM2NTc0MC1lMjAyYzA0My0zOGI5NTRiYy03MWZiNzJmZQ==, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [3:7559340926994053038:3000] 
2025-10-09T21:43:34.062719Z node 4 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:43:34.063324Z node 3 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:43:38.068025Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[4:7559340926496809292:2264];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:43:38.068068Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
2025-10-09T21:43:38.987326Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720672:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
waiting...
PQ Client: create topic: rt3.dc1--test-topic with 1 partitions
CallPersQueueGRPC request to localhost:13769
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:43:39.043855Z node 3 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:13769
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720673
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-09T21:43:39.064071Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:39.064758Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:39.064809Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:784: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-09T21:43:39.064816Z node 3 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:39.064823Z node 3 :PERSQUEUE INFO: pq_impl.cpp:609: [PQ: 72075186224037892] doesn't have tx writes info
2025-10-09T21:43:39.064934Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340952763857340:3323], now have 1 active actors on pipe
2025-10-09T21:43:39.065007Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][] pipe [3:7559340952763857344:3324] connected; active server actors: 1
2025-10-09T21:43:39.065288Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:43:39.065396Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1517: [72075186224037893][rt3.dc1--test-topic] updating configuration. Deleted partitions []. Added partitions [0]
2025-10-09T21:43:39.065528Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:940: [72075186224037893][rt3.dc1--test-topic] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-09T21:43:39.065842Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 3, Generation 1
2025-10-09T21:43:39.065976Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:138: [72075186224037893][rt3.dc1--test-topic] BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) 
2025-10-09T21:43:39.066276Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [4:7559340952266613521:2357], now have 1 active actors on pipe
2025-10-09T21:43:39.067034Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:43:39.068688Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340952763857395:3368], now have 1 active actors on pipe
2025-10-09T21:43:39.068767Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:43:39.073674Z node 3 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720674. Ctx: { TraceId: 01k75evzse279gz8m6edtrfhe3, Database: , SessionId: ydb://session/3?node_id=3&id=NWY1ZDY5YTgtOWVhYjg1NzMtMzk5MjUyNGUtMWJkZjU2OA==, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-10-09T21:43:39.075463Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:43:39.075709Z node 3 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:39.075722Z node 3 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:43:39.075794Z node 3 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:39.075829Z node 3 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [3:7559340952763857420:2463]
2025-10-09T21:43:39.076552Z node 3 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:39.076555Z node 3 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:39.076560Z node 3 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [3:7559340952763857420:2463]
2025-10-09T21:43:39.076566Z node 3 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:43:39.076859Z node 3 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-10-09T21:43:39.076865Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.076883Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.076885Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:39.076889Z node 3 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:43:39.076890Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.076892Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.076894Z node 3 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:43:39.076901Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:39.077058Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:43:39.077081Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.078364Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:43:39.078441Z node 3 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046219113, TxId 281474976720673
2025-10-09T21:43:39.078449Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.078451Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.078485Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.078534Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.079758Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.079842Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.079852Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.079854Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.079856Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.079863Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.080013Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:395: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-10-09T21:43:39.080034Z node 3 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:39.080090Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:43:39.081491Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:43:39.081519Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:43:39.082207Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046219113
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046219113
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:13769
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:43:39.095330Z node 3 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
2025-10-09T21:43:39.102461Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-09T21:43:39.102666Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-09T21:43:39.102671Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:13769
2025-10-09T21:43:39.103363Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:39.103378Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1
2025-10-09T21:43:39.103936Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:43:39.104445Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:43:39.104478Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:57322
2025-10-09T21:43:39.104483Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:57322 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:39.104487Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:39.104975Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-09T21:43:39.105002Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:39.105004Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:39.105006Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:39.105012Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [3:7559340952763857503:2481] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:39.105605Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [3:7559340952763857503:2481] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:39.136990Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [3:7559340952763857503:2481] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:43:39.137538Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857538:2481] connected; active server actors: 1
2025-10-09T21:43:39.137691Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [3:7559340952763857503:2481] (SourceId=src, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src
2025-10-09T21:43:39.137704Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [3:7559340952763857503:2481] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:39.142185Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857538:2481] disconnected.
2025-10-09T21:43:39.142202Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857538:2481] disconnected; active server actors: 1
2025-10-09T21:43:39.142208Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857538:2481] disconnected no session
2025-10-09T21:43:39.175186Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [3:7559340952763857503:2481] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:39.175210Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [3:7559340952763857503:2481] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:43:39.175214Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [3:7559340952763857503:2481] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:39.175226Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:39.175987Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 3, Generation: 1
2025-10-09T21:43:39.176018Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340952763857560:2481], now have 1 active actors on pipe
2025-10-09T21:43:39.176041Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.176050Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.176089Z node 3 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src|1d4f9103-33989a63-b775496d-faa31869_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src
2025-10-09T21:43:39.176104Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.176110Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.176114Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:39.176128Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.176147Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:39.176177Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:39.176208Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.176215Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.176228Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:39.176261Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1 partition: 0 MaxSeqNo: 0 sessionId: src|1d4f9103-33989a63-b775496d-faa31869_0
2025-10-09T21:43:39.176697Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046219176 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:43:39.176760Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|1d4f9103-33989a63-b775496d-faa31869_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:39.176897Z :INFO: [] MessageGroupId [src] SessionId [src|1d4f9103-33989a63-b775496d-faa31869_0] Write session: close. Timeout = 0 ms
2025-10-09T21:43:39.176907Z :INFO: [] MessageGroupId [src] SessionId [src|1d4f9103-33989a63-b775496d-faa31869_0] Write session will now close
2025-10-09T21:43:39.176915Z :DEBUG: [] MessageGroupId [src] SessionId [src|1d4f9103-33989a63-b775496d-faa31869_0] Write session: aborting
2025-10-09T21:43:39.176924Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.176931Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.176932Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.177069Z :INFO: [] MessageGroupId [src] SessionId [src|1d4f9103-33989a63-b775496d-faa31869_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:43:39.177074Z :DEBUG: [] MessageGroupId [src] SessionId [src|1d4f9103-33989a63-b775496d-faa31869_0] Write session: destroy
2025-10-09T21:43:39.177246Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId: src|1d4f9103-33989a63-b775496d-faa31869_0 grpc read done: success: 0 data: 
2025-10-09T21:43:39.177259Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 1 sessionId: src|1d4f9103-33989a63-b775496d-faa31869_0 grpc read failed
2025-10-09T21:43:39.177265Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 1 sessionId: src|1d4f9103-33989a63-b775496d-faa31869_0 grpc closed
2025-10-09T21:43:39.177269Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1 sessionId: src|1d4f9103-33989a63-b775496d-faa31869_0 is DEAD
2025-10-09T21:43:39.177571Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:43:39.178002Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [3:7559340952763857560:2481] destroyed
2025-10-09T21:43:39.178011Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:43:39.178022Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.178023Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.178024Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Session was created
>>> Ready to answer: ok
2025-10-09T21:43:39.248202Z :INFO: [/Root] [] [b6a8a85b-b621294-7c714f0d-ed59d26] Open read subsessions to databases: { name: dc1, endpoint: localhost:13769, path: /Root }, { name: dc2, endpoint: localhost:13769, path: /Root }, { name: dc3, endpoint: localhost:13769, path: /Root }
2025-10-09T21:43:39.249646Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [] Write session: try to update token
2025-10-09T21:43:39.249780Z :INFO: [/Root] MessageGroupId [src_id] SessionId [] Write session: Do CDS request
2025-10-09T21:43:39.249814Z :INFO: [/Root] MessageGroupId [src_id] SessionId [] Start write session. Will connect to endpoint: localhost:13769
2025-10-09T21:43:39.250478Z :INFO: [/Root] [/Root] [ee2235dc-c75c6e93-275e2257-6ad62539] Starting read session
2025-10-09T21:43:39.250488Z :DEBUG: [/Root] [/Root] [ee2235dc-c75c6e93-275e2257-6ad62539] Starting single session
2025-10-09T21:43:39.250750Z :DEBUG: [/Root] [/Root] [ee2235dc-c75c6e93-275e2257-6ad62539] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:43:39.250756Z :DEBUG: [/Root] [/Root] [ee2235dc-c75c6e93-275e2257-6ad62539] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:43:39.250760Z :DEBUG: [/Root] [/Root] [ee2235dc-c75c6e93-275e2257-6ad62539] [] Reconnecting session to cluster  in 0.000000s
2025-10-09T21:43:39.251481Z :INFO: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] Starting read session
2025-10-09T21:43:39.251491Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] Starting single session
2025-10-09T21:43:39.251511Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:43:39.251516Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:43:39.251519Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Reconnecting session to cluster  in 0.000000s
2025-10-09T21:43:39.251662Z :INFO: [/Root] [/Root] [325a6ae6-7e984b87-19964452-73f5f42d] Starting read session
2025-10-09T21:43:39.251670Z :DEBUG: [/Root] [/Root] [325a6ae6-7e984b87-19964452-73f5f42d] Starting single session
2025-10-09T21:43:39.251704Z :DEBUG: [/Root] [/Root] [325a6ae6-7e984b87-19964452-73f5f42d] [] In Reconnect, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:43:39.251709Z :DEBUG: [/Root] [/Root] [325a6ae6-7e984b87-19964452-73f5f42d] [] New values: ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:43:39.251712Z :DEBUG: [/Root] [/Root] [325a6ae6-7e984b87-19964452-73f5f42d] [] Reconnecting session to cluster  in 0.000000s
2025-10-09T21:43:39.278140Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.278169Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.278173Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.346704Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:39.346870Z :DEBUG: [/Root] [/Root] [ee2235dc-c75c6e93-275e2257-6ad62539] [] Successfully connected. Initializing session
2025-10-09T21:43:39.346921Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Successfully connected. Initializing session
2025-10-09T21:43:39.347011Z :DEBUG: [/Root] [/Root] [325a6ae6-7e984b87-19964452-73f5f42d] [] Successfully connected. Initializing session
2025-10-09T21:43:39.346960Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:39.346974Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2
2025-10-09T21:43:39.347056Z node 3 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:43:39.347058Z node 3 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 1
2025-10-09T21:43:39.347074Z node 3 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:43:39.347076Z node 3 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 2
2025-10-09T21:43:39.347384Z node 3 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:111: new grpc connection
2025-10-09T21:43:39.347391Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:39.347395Z node 3 :PQ_READ_PROXY DEBUG: grpc_pq_read.h:133: new session created cookie 3
2025-10-09T21:43:39.347423Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2 topic: "test-topic" message_group_id: "src_id" from ipv6:[::1]:42738
2025-10-09T21:43:39.347429Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:42738 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:39.347433Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:39.350881Z :INFO: [/Root] [/Root] [ee2235dc-c75c6e93-275e2257-6ad62539] [] Got InitResponse. ReadSessionId: shared/user_3_1_13017312322801993971_v1
2025-10-09T21:43:39.350895Z :DEBUG: [/Root] [/Root] [ee2235dc-c75c6e93-275e2257-6ad62539] [] In ContinueReadingDataImpl, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:43:39.350965Z :DEBUG: [/Root] [/Root] [ee2235dc-c75c6e93-275e2257-6ad62539] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.349885Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:43:39.351431Z :INFO: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got InitResponse. ReadSessionId: shared/user_3_2_7554872134447193526_v1
2025-10-09T21:43:39.351437Z :INFO: [/Root] [/Root] [325a6ae6-7e984b87-19964452-73f5f42d] [] Got InitResponse. ReadSessionId: shared/user_3_3_14754793126700281017_v1
2025-10-09T21:43:39.351446Z :DEBUG: [/Root] [/Root] [325a6ae6-7e984b87-19964452-73f5f42d] [] In ContinueReadingDataImpl, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:43:39.351447Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 524288, ReadSizeServerDelta = 0
2025-10-09T21:43:39.349916Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:43:39.349973Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read init: from# ipv6:[::1]:42738, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:43:39.349991Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 1 consumer shared/user session shared/user_3_1_13017312322801993971_v1 read init: from# ipv6:[::1]:42738, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:43:39.350088Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 auth for : user
2025-10-09T21:43:39.350134Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 1 consumer shared/user session shared/user_3_1_13017312322801993971_v1 auth for : user
2025-10-09T21:43:39.350203Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 3 consumer  session  grpc read done: success# 1, data# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:43:39.350228Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:929: session cookie 3 consumer shared/user session shared/user_3_3_14754793126700281017_v1 read init: from# ipv6:[::1]:42738, request# { init_request { topics_read_settings { path: "test-topic" } consumer: "shared/user" } }
2025-10-09T21:43:39.350279Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:41: session cookie 3 consumer shared/user session shared/user_3_3_14754793126700281017_v1 auth for : user
2025-10-09T21:43:39.350329Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 1 consumer shared/user session shared/user_3_1_13017312322801993971_v1 Handle describe topics response
2025-10-09T21:43:39.350347Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 1 consumer shared/user session shared/user_3_1_13017312322801993971_v1 auth is DEAD
2025-10-09T21:43:39.350378Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Handle describe topics response
2025-10-09T21:43:39.350386Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 auth is DEAD
2025-10-09T21:43:39.351495Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.350402Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 1 consumer shared/user session shared/user_3_1_13017312322801993971_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:43:39.350542Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2 sessionId:  describe result for acl check
2025-10-09T21:43:39.351514Z :DEBUG: [/Root] [/Root] [325a6ae6-7e984b87-19964452-73f5f42d] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.350565Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:39.350567Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:39.350569Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:39.350576Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [3:7559340952763857586:2499] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:39.350586Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:43:39.350709Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 1 consumer shared/user session shared/user_3_1_13017312322801993971_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:43:39.350760Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:43:39.350785Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:131: session cookie 3 consumer shared/user session shared/user_3_3_14754793126700281017_v1 Handle describe topics response
2025-10-09T21:43:39.350799Z node 3 :PQ_READ_PROXY DEBUG: read_init_auth_actor.cpp:68: session cookie 3 consumer shared/user session shared/user_3_3_14754793126700281017_v1 auth is DEAD
2025-10-09T21:43:39.350808Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:1046: session cookie 3 consumer shared/user session shared/user_3_3_14754793126700281017_v1 auth ok: topics# 1, initDone# 0
2025-10-09T21:43:39.350967Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:1217: session cookie 3 consumer shared/user session shared/user_3_3_14754793126700281017_v1 register session: topic# rt3.dc1--test-topic
2025-10-09T21:43:39.351375Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [3:7559340952763857586:2499] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:39.351383Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer shared/user session shared/user_3_1_13017312322801993971_v1 grpc read done: success# 1, data# { read_request { bytes_size: 524288 } }
2025-10-09T21:43:39.351399Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 1 consumer shared/user session shared/user_3_1_13017312322801993971_v1 got read request: guid# bc718b27-443b1bc8-5ca5c20b-43a820cb
2025-10-09T21:43:39.351682Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 3 consumer shared/user session shared/user_3_3_14754793126700281017_v1 grpc read done: success# 1, data# { read_request { bytes_size: 524288 } }
2025-10-09T21:43:39.351746Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 3 consumer shared/user session shared/user_3_3_14754793126700281017_v1 got read request: guid# 6e7da0b3-1e94668b-8e4aa423-ebd443fb
2025-10-09T21:43:39.351760Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 524288 } }
2025-10-09T21:43:39.351792Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# 5dd2b365-9ff6d3c5-d2850c37-dce1575a
2025-10-09T21:43:39.354438Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857587:2500] connected; active server actors: 1
2025-10-09T21:43:39.354458Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857589:2501] connected; active server actors: 1
2025-10-09T21:43:39.354462Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857590:2502] connected; active server actors: 1
2025-10-09T21:43:39.354707Z node 4 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [3:7559340952763857587:2500] session shared/user_3_1_13017312322801993971_v1
2025-10-09T21:43:39.354721Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:637: [72075186224037893][rt3.dc1--test-topic] consumer user register readable partition 0
2025-10-09T21:43:39.354733Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:667: [72075186224037893][rt3.dc1--test-topic] consumer user family created family=1 (Status=Free, Partitions=[0])
2025-10-09T21:43:39.354741Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_3_1_13017312322801993971_v1" (Sender=[3:7559340952763857577:2500], Pipe=[3:7559340952763857587:2500], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:43:39.354745Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:43:39.354755Z node 4 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [3:7559340952763857589:2501] session shared/user_3_2_7554872134447193526_v1
2025-10-09T21:43:39.354759Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_3_2_7554872134447193526_v1" (Sender=[3:7559340952763857578:2501], Pipe=[3:7559340952763857589:2501], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:43:39.354765Z node 4 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1704: [72075186224037893][rt3.dc1--test-topic] consumer "user" register session for pipe [3:7559340952763857590:2502] session shared/user_3_3_14754793126700281017_v1
2025-10-09T21:43:39.354769Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:869: [72075186224037893][rt3.dc1--test-topic] consumer user register reading session ReadingSession "shared/user_3_3_14754793126700281017_v1" (Sender=[3:7559340952763857579:2502], Pipe=[3:7559340952763857590:2502], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:43:39.354777Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=3, Families=1, UnradableFamilies=1 [1 (0), ], RequireBalancing=0 []
2025-10-09T21:43:39.354787Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1303: [72075186224037893][rt3.dc1--test-topic] consumer user balancing family=1 (Status=Free, Partitions=[0]) for ReadingSession "shared/user_3_2_7554872134447193526_v1" (Sender=[3:7559340952763857578:2501], Pipe=[3:7559340952763857589:2501], Partitions=[], ActiveFamilyCount=0)
2025-10-09T21:43:39.354797Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:547: [72075186224037893][rt3.dc1--test-topic] consumer user family 1 status Active partitions [0] session "shared/user_3_2_7554872134447193526_v1" sender [3:7559340952763857578:2501] lock partition 0 for ReadingSession "shared/user_3_2_7554872134447193526_v1" (Sender=[3:7559340952763857578:2501], Pipe=[3:7559340952763857589:2501], Partitions=[], ActiveFamilyCount=1) generation 1 step 1
2025-10-09T21:43:39.354811Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=3, desiredFamilyCount=0, allowPlusOne=1
2025-10-09T21:43:39.354816Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000035s
2025-10-09T21:43:39.355152Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [3:7559340952763857586:2499] (SourceId=src_id, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:43:39.355203Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:1335: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 assign: record# { Partition: 0 TabletId: 72075186224037892 Topic: "rt3.dc1--test-topic" Generation: 1 Step: 1 Session: "shared/user_3_2_7554872134447193526_v1" ClientId: "user" PipeClient { RawX1: 7559340952763857589 RawX2: 4503612512274885 } Path: "/Root/PQ/rt3.dc1--test-topic" }
2025-10-09T21:43:39.355226Z node 3 :PQ_READ_PROXY INFO: partition_actor.cpp:1143: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 INITING TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)
2025-10-09T21:43:39.355280Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340952763857612:2512], now have 1 active actors on pipe
2025-10-09T21:43:39.355281Z node 3 :PQ_READ_PROXY INFO: partition_actor.cpp:983: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) pipe restart attempt 0 pipe creation result: OK TabletId: 72075186224037892 Generation: 1, pipe: [3:7559340952763857612:2512]
2025-10-09T21:43:39.355292Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.355298Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.355311Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1744: [PQ: 72075186224037892] Created session shared/user_3_2_7554872134447193526_v1 on pipe: [3:7559340952763857612:2512]
2025-10-09T21:43:39.355339Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.355339Z node 3 :PQ_READ_PROXY DEBUG: caching_service.cpp:283: Direct read cache: registered server session: shared/user_3_2_7554872134447193526_v1:1 with generation 1
2025-10-09T21:43:39.355344Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.355346Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:39.355366Z node 3 :PERSQUEUE DEBUG: partition.cpp:3745: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user session is set to 0 (startOffset 0) session shared/user_3_2_7554872134447193526_v1
2025-10-09T21:43:39.355374Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.355403Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.358302Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [3:7559340952763857586:2499] (SourceId=src_id, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src_id
2025-10-09T21:43:39.358315Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [3:7559340952763857586:2499] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:39.358166Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857611:2499] connected; active server actors: 1
2025-10-09T21:43:39.359732Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857611:2499] disconnected.
2025-10-09T21:43:39.359758Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857611:2499] disconnected; active server actors: 1
2025-10-09T21:43:39.359760Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857611:2499] disconnected no session
2025-10-09T21:43:39.360384Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.360425Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.360438Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.360440Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.360442Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.360450Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.360464Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18446744073709551615
2025-10-09T21:43:39.360532Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 0 event { CmdGetClientOffsetResult { Offset: 0 EndOffset: 0 SizeLag: 0 WriteTimestampEstimateMS: 1760046219278 ClientHasAnyCommits: false } Cookie: 18446744073709551615 }
2025-10-09T21:43:39.360541Z node 3 :PQ_READ_PROXY INFO: partition_actor.cpp:694: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 INIT DONE TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 0 readOffset 0 committedOffset 0
2025-10-09T21:43:39.360568Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1434: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 sending to client partition status
2025-10-09T21:43:39.360951Z :INFO: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Confirm partition stream create. Partition stream id: 1. Cluster: "-". Topic: "test-topic". Partition: 0. Read offset: (NULL)
2025-10-09T21:43:39.361387Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { start_partition_session_response { partition_session_id: 1 } }
2025-10-09T21:43:39.361453Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:539: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got StartRead from client: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 0, commitOffset# (empty maybe)
2025-10-09T21:43:39.361491Z node 3 :PQ_READ_PROXY INFO: partition_actor.cpp:1023: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Start reading TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 0 readOffset 0 committedOffset 0 clientCommitOffset (empty maybe) clientReadOffset 0
2025-10-09T21:43:39.361498Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 1 from offset 0
2025-10-09T21:43:39.366975Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [3:7559340952763857586:2499] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:39.366992Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [3:7559340952763857586:2499] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:43:39.366996Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [3:7559340952763857586:2499] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:39.367004Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:39.367162Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 3, Generation: 1
2025-10-09T21:43:39.367175Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340952763857631:2499], now have 1 active actors on pipe
2025-10-09T21:43:39.367182Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.367189Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.367217Z node 3 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:39.367238Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.367244Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.367247Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:39.367259Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.367274Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:39.367292Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:39.367394Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.367402Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.367416Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:39.367446Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2 partition: 0 MaxSeqNo: 0 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0
2025-10-09T21:43:39.367849Z :INFO: [/Root] MessageGroupId [src_id] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046219367 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:43:39.367899Z :INFO: [/Root] MessageGroupId [src_id] SessionId [] Write session established. Init response: session_id: "src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:39.368021Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 1 to 1
2025-10-09T21:43:39.368053Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.368070Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 1
2025-10-09T21:43:39.368169Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 2 to 2
2025-10-09T21:43:39.368193Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.368197Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 2
2025-10-09T21:43:39.368234Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 3 to 3
2025-10-09T21:43:39.368241Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.368243Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 3
2025-10-09T21:43:39.368277Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 4 to 4
2025-10-09T21:43:39.368283Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.368286Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 4
2025-10-09T21:43:39.368281Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.368316Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 5 to 5
2025-10-09T21:43:39.368322Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.368325Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 5
2025-10-09T21:43:39.368361Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 6 to 6
2025-10-09T21:43:39.368367Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.368369Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 6
2025-10-09T21:43:39.368376Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 7 to 7
2025-10-09T21:43:39.368382Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.368384Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 7
2025-10-09T21:43:39.368389Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 8 to 8
2025-10-09T21:43:39.368396Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.368399Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 8
2025-10-09T21:43:39.368405Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 9 to 9
2025-10-09T21:43:39.368411Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.368414Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 9
2025-10-09T21:43:39.368383Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.368432Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 10 to 10
2025-10-09T21:43:39.368439Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.368442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 10
2025-10-09T21:43:39.368447Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 11 to 11
2025-10-09T21:43:39.368453Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.368455Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 11
2025-10-09T21:43:39.368460Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 12 to 12
2025-10-09T21:43:39.368468Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.368470Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 12
2025-10-09T21:43:39.368458Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.368475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 13 to 13
2025-10-09T21:43:39.368481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.368483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 13
2025-10-09T21:43:39.368477Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.368490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 14 to 14
2025-10-09T21:43:39.368496Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.368499Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 14
2025-10-09T21:43:39.368504Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 15 to 15
2025-10-09T21:43:39.368510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.368494Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.368515Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 15
2025-10-09T21:43:39.368550Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 16 to 16
2025-10-09T21:43:39.368565Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.368570Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 16
2025-10-09T21:43:39.368553Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.368571Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.368578Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.368586Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.368592Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.368596Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.368597Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.368614Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.368630Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:39.368637Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1  requestId: <none> cookie: 2
2025-10-09T21:43:39.368640Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 2  requestId: <none> cookie: 3
2025-10-09T21:43:39.368678Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.368690Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.368695Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.368714Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.368793Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.368804Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.368934Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 0 messageNo: 3 size 82 offset: -1
2025-10-09T21:43:39.368963Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.368970Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.368983Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 17 to 17
2025-10-09T21:43:39.368976Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 2 partNo : 0 messageNo: 4 size 192 offset: -1
2025-10-09T21:43:39.368980Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.368982Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.368997Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369001Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 17
2025-10-09T21:43:39.368988Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 3 partNo : 0 messageNo: 5 size 303 offset: -1
2025-10-09T21:43:39.368992Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.369008Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 18 to 18
2025-10-09T21:43:39.368995Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.369003Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.369013Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369016Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 18
2025-10-09T21:43:39.369021Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 19 to 19
2025-10-09T21:43:39.369027Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369030Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 19
2025-10-09T21:43:39.369034Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 20 to 20
2025-10-09T21:43:39.369025Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.369036Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.369038Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:39.369055Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.369040Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369279Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 20
2025-10-09T21:43:39.369293Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 21 to 21
2025-10-09T21:43:39.369302Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369304Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 21
2025-10-09T21:43:39.369309Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 22 to 22
2025-10-09T21:43:39.369315Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369318Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 22
2025-10-09T21:43:39.369326Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 23 to 23
2025-10-09T21:43:39.369331Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369334Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 23
2025-10-09T21:43:39.369338Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 24 to 24
2025-10-09T21:43:39.369344Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369346Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 24
2025-10-09T21:43:39.369351Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 25 to 25
2025-10-09T21:43:39.369355Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369358Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 25
2025-10-09T21:43:39.369363Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 26 to 26
2025-10-09T21:43:39.369369Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369371Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 26
2025-10-09T21:43:39.369376Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 27 to 27
2025-10-09T21:43:39.369394Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369396Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 27
2025-10-09T21:43:39.369402Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 28 to 28
2025-10-09T21:43:39.369411Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369413Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 28
2025-10-09T21:43:39.369420Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 29 to 29
2025-10-09T21:43:39.369427Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369430Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 29
2025-10-09T21:43:39.369435Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 30 to 30
2025-10-09T21:43:39.369442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369445Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 30
2025-10-09T21:43:39.369455Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 31 to 31
2025-10-09T21:43:39.369459Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369462Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 31
2025-10-09T21:43:39.369468Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 32 to 32
2025-10-09T21:43:39.369476Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369479Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 32
2025-10-09T21:43:39.369485Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 33 to 33
2025-10-09T21:43:39.369490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369493Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 33
2025-10-09T21:43:39.369498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 34 to 34
2025-10-09T21:43:39.369502Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 34
2025-10-09T21:43:39.369511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 35 to 35
2025-10-09T21:43:39.369520Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369523Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 35
2025-10-09T21:43:39.369529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 36 to 36
2025-10-09T21:43:39.369533Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369536Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 36
2025-10-09T21:43:39.369543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 37 to 37
2025-10-09T21:43:39.369548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369551Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 37
2025-10-09T21:43:39.369556Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 38 to 38
2025-10-09T21:43:39.369567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 38
2025-10-09T21:43:39.369580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 39 to 39
2025-10-09T21:43:39.369587Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 39
2025-10-09T21:43:39.369598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 40 to 40
2025-10-09T21:43:39.369603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369606Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 40
2025-10-09T21:43:39.369635Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 41 to 41
2025-10-09T21:43:39.369641Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369643Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 41
2025-10-09T21:43:39.369652Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 42 to 42
2025-10-09T21:43:39.369658Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 42
2025-10-09T21:43:39.369702Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 43 to 43
2025-10-09T21:43:39.369708Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369711Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 43
2025-10-09T21:43:39.369719Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 44 to 44
2025-10-09T21:43:39.369727Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369730Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 44
2025-10-09T21:43:39.369740Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 45 to 45
2025-10-09T21:43:39.369745Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369748Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 45
2025-10-09T21:43:39.369755Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 46 to 46
2025-10-09T21:43:39.369762Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369764Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 46
2025-10-09T21:43:39.369772Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 47 to 47
2025-10-09T21:43:39.369820Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369823Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 47
2025-10-09T21:43:39.369832Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 48 to 48
2025-10-09T21:43:39.369839Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369842Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 48
2025-10-09T21:43:39.369850Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 49 to 49
2025-10-09T21:43:39.369857Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369859Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 49
2025-10-09T21:43:39.369867Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 50 to 50
2025-10-09T21:43:39.369872Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369874Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 50
2025-10-09T21:43:39.369904Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 51 to 51
2025-10-09T21:43:39.369914Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369916Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 51
2025-10-09T21:43:39.369925Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 52 to 52
2025-10-09T21:43:39.369930Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369933Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 52
2025-10-09T21:43:39.369955Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 53 to 53
2025-10-09T21:43:39.369961Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369964Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 53
2025-10-09T21:43:39.369973Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 54 to 54
2025-10-09T21:43:39.369978Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369980Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 54
2025-10-09T21:43:39.369989Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 55 to 55
2025-10-09T21:43:39.369995Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369998Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 55
2025-10-09T21:43:39.370006Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 56 to 56
2025-10-09T21:43:39.370012Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.370015Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 56
2025-10-09T21:43:39.369065Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 0
2025-10-09T21:43:39.370023Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 57 to 57
2025-10-09T21:43:39.370109Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.369106Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.369119Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 1 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 156 count 1 nextOffset 1 batches 1
2025-10-09T21:43:39.369121Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.369126Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.369150Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.369172Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.369177Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 0,1 HeadOffset 0 endOffset 0 curOffset 1 d0000000000_00000000000000000000_00000_0000000001_00000? size 142 WTime 1760046219368
2025-10-09T21:43:39.369195Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.369201Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.369203Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.369206Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.369214Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.369215Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.369227Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 1 size 142
2025-10-09T21:43:39.369236Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 6  requestId: <none> cookie: 4
2025-10-09T21:43:39.370117Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 57
2025-10-09T21:43:39.369278Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.369280Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.369286Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.369288Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.369292Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.369294Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.369438Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 7  requestId: <none> cookie: 5
2025-10-09T21:43:39.369443Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 8  requestId: <none> cookie: 6
2025-10-09T21:43:39.369446Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 9  requestId: <none> cookie: 7
2025-10-09T21:43:39.369454Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.369463Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.369470Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.369474Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.369478Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.369560Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.369601Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.369609Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.369662Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.369720Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.369803Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.369847Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.369853Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.369891Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.369897Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.369974Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.370046Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.370080Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.370129Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370134Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370146Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 4 partNo : 0 messageNo: 10 size 413 offset: -1
2025-10-09T21:43:39.370153Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370155Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370160Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 5 partNo : 0 messageNo: 11 size 523 offset: -1
2025-10-09T21:43:39.370164Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370180Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370185Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 6 partNo : 0 messageNo: 12 size 633 offset: -1
2025-10-09T21:43:39.370191Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370193Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370198Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 7 partNo : 0 messageNo: 13 size 743 offset: -1
2025-10-09T21:43:39.370201Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370204Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370208Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370210Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370214Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370216Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370220Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370222Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370226Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370228Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370234Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.370243Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.370245Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.370248Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.370250Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.370253Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.370255Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.370257Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.370271Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 14  requestId: <none> cookie: 8
2025-10-09T21:43:39.370277Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 15  requestId: <none> cookie: 9
2025-10-09T21:43:39.370280Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 16  requestId: <none> cookie: 10
2025-10-09T21:43:39.370282Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 17  requestId: <none> cookie: 11
2025-10-09T21:43:39.370286Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 18  requestId: <none> cookie: 12
2025-10-09T21:43:39.370295Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.370302Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.370306Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.370310Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.370314Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.370328Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.370336Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.370341Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.370344Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.370349Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.370393Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.370399Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370402Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370408Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 8 partNo : 0 messageNo: 19 size 853 offset: -1
2025-10-09T21:43:39.370412Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370454Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370460Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 9 partNo : 0 messageNo: 20 size 963 offset: -1
2025-10-09T21:43:39.370464Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370466Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370471Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 10 partNo : 0 messageNo: 21 size 1073 offset: -1
2025-10-09T21:43:39.370475Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370477Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370481Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 11 partNo : 0 messageNo: 22 size 1183 offset: -1
2025-10-09T21:43:39.370484Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370486Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370490Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 12 partNo : 0 messageNo: 23 size 1293 offset: -1
2025-10-09T21:43:39.370494Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370495Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370500Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370502Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370505Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370507Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370507Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.370510Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370512Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370515Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370517Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370521Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.370528Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.370531Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.370533Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.370535Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.370538Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.370541Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.370543Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.370546Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.370549Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.370559Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 24  requestId: <none> cookie: 13
2025-10-09T21:43:39.370565Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 25  requestId: <none> cookie: 14
2025-10-09T21:43:39.370568Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 26  requestId: <none> cookie: 15
2025-10-09T21:43:39.370570Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 27  requestId: <none> cookie: 16
2025-10-09T21:43:39.370570Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.370573Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 28  requestId: <none> cookie: 17
2025-10-09T21:43:39.370580Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.370584Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.370590Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.370609Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.370617Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.370756Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370779Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370792Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 13 partNo : 0 messageNo: 29 size 1403 offset: -1
2025-10-09T21:43:39.370805Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370812Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370817Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 14 partNo : 0 messageNo: 30 size 1513 offset: -1
2025-10-09T21:43:39.370822Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370824Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370828Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 15 partNo : 0 messageNo: 31 size 1623 offset: -1
2025-10-09T21:43:39.370835Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370837Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370841Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 16 partNo : 0 messageNo: 32 size 1733 offset: -1
2025-10-09T21:43:39.370845Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370848Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370852Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 17 partNo : 0 messageNo: 33 size 1843 offset: -1
2025-10-09T21:43:39.370856Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.370859Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.370864Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.370873Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.370875Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.370877Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.370879Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.370882Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.370884Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.370887Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.370898Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.370900Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.370906Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 34  requestId: <none> cookie: 18
2025-10-09T21:43:39.370922Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.371172Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.371186Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.371198Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 18 partNo : 0 messageNo: 35 size 1953 offset: -1
2025-10-09T21:43:39.371206Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.371214Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.371253Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 58 to 58
2025-10-09T21:43:39.371268Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371273Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 58
2025-10-09T21:43:39.371300Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 59 to 59
2025-10-09T21:43:39.371306Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371309Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 59
2025-10-09T21:43:39.371318Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 60 to 60
2025-10-09T21:43:39.371341Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371343Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 60
2025-10-09T21:43:39.371354Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 61 to 61
2025-10-09T21:43:39.371362Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371365Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 61
2025-10-09T21:43:39.371374Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 62 to 62
2025-10-09T21:43:39.371382Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371384Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 62
2025-10-09T21:43:39.371393Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 63 to 63
2025-10-09T21:43:39.371402Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371405Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 63
2025-10-09T21:43:39.371430Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 64 to 64
2025-10-09T21:43:39.371440Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371443Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 64
2025-10-09T21:43:39.371452Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 65 to 65
2025-10-09T21:43:39.371461Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371466Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 65
2025-10-09T21:43:39.371475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 66 to 66
2025-10-09T21:43:39.371481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371484Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 66
2025-10-09T21:43:39.371493Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 67 to 67
2025-10-09T21:43:39.371500Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 67
2025-10-09T21:43:39.371539Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 68 to 68
2025-10-09T21:43:39.371557Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 68
2025-10-09T21:43:39.371568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 69 to 69
2025-10-09T21:43:39.371575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 69
2025-10-09T21:43:39.371587Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 70 to 70
2025-10-09T21:43:39.371594Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 70
2025-10-09T21:43:39.371605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 71 to 71
2025-10-09T21:43:39.371613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371616Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 71
2025-10-09T21:43:39.371651Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 72 to 72
2025-10-09T21:43:39.371658Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 72
2025-10-09T21:43:39.371669Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 73 to 73
2025-10-09T21:43:39.371675Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371679Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 73
2025-10-09T21:43:39.371688Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 74 to 74
2025-10-09T21:43:39.371694Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371696Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 74
2025-10-09T21:43:39.371704Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 75 to 75
2025-10-09T21:43:39.371711Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371714Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 75
2025-10-09T21:43:39.371724Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 76 to 76
2025-10-09T21:43:39.371752Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371765Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 76
2025-10-09T21:43:39.371776Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 77 to 77
2025-10-09T21:43:39.371804Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371807Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 77
2025-10-09T21:43:39.371830Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 78 to 78
2025-10-09T21:43:39.371837Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371840Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 78
2025-10-09T21:43:39.371852Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 79 to 79
2025-10-09T21:43:39.371861Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371863Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 79
2025-10-09T21:43:39.371876Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 80 to 80
2025-10-09T21:43:39.371884Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371887Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 80
2025-10-09T21:43:39.371925Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 81 to 81
2025-10-09T21:43:39.371937Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371942Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 81
2025-10-09T21:43:39.371953Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 82 to 82
2025-10-09T21:43:39.371963Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.371966Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 82
2025-10-09T21:43:39.371984Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 83 to 83
2025-10-09T21:43:39.372005Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372008Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 83
2025-10-09T21:43:39.372019Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 84 to 84
2025-10-09T21:43:39.372026Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372028Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 84
2025-10-09T21:43:39.372040Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 85 to 85
2025-10-09T21:43:39.372025Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 1 size 142 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.372054Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.372050Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372081Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 85
2025-10-09T21:43:39.372072Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 89 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.372089Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.372108Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 86 to 86
2025-10-09T21:43:39.372121Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372123Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 86
2025-10-09T21:43:39.372099Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 1, partNo: 0, Offset: 0 is stored on disk
2025-10-09T21:43:39.372146Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 87 to 87
2025-10-09T21:43:39.372137Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:39.372153Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372148Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:39.372155Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 87
2025-10-09T21:43:39.372153Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.372168Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.372175Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (17)
2025-10-09T21:43:39.372225Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 88 to 88
2025-10-09T21:43:39.372235Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372238Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 88
2025-10-09T21:43:39.372249Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 89 to 89
2025-10-09T21:43:39.372255Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372272Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 89
2025-10-09T21:43:39.372284Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 90 to 90
2025-10-09T21:43:39.372292Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372295Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 90
2025-10-09T21:43:39.372197Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 0
2025-10-09T21:43:39.372315Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 91 to 91
2025-10-09T21:43:39.372234Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 2 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 266 count 1 nextOffset 2 batches 1
2025-10-09T21:43:39.372321Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372241Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 0
2025-10-09T21:43:39.372324Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 91
2025-10-09T21:43:39.372246Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 3 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 611 count 2 nextOffset 3 batches 1
2025-10-09T21:43:39.372250Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 0
2025-10-09T21:43:39.372344Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 92 to 92
2025-10-09T21:43:39.372256Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 4 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 1066 count 3 nextOffset 4 batches 1
2025-10-09T21:43:39.372260Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 0
2025-10-09T21:43:39.372350Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372264Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 5 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 1631 count 4 nextOffset 5 batches 1
2025-10-09T21:43:39.372353Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 92
2025-10-09T21:43:39.372267Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 0
2025-10-09T21:43:39.372271Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 6 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 2306 count 5 nextOffset 6 batches 1
2025-10-09T21:43:39.372274Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 0
2025-10-09T21:43:39.372277Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 7 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 3091 count 6 nextOffset 7 batches 1
2025-10-09T21:43:39.372281Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 0
2025-10-09T21:43:39.372382Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 93 to 93
2025-10-09T21:43:39.372284Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 8 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 3986 count 7 nextOffset 8 batches 1
2025-10-09T21:43:39.372288Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 0
2025-10-09T21:43:39.372292Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 9 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 4991 count 8 nextOffset 9 batches 1
2025-10-09T21:43:39.372295Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 0
2025-10-09T21:43:39.372303Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 10 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 6106 count 9 nextOffset 10 batches 1
2025-10-09T21:43:39.372307Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 0
2025-10-09T21:43:39.372311Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 11 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 7331 count 10 nextOffset 11 batches 1
2025-10-09T21:43:39.372314Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 0
2025-10-09T21:43:39.372319Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 12 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 8666 count 11 nextOffset 12 batches 1
2025-10-09T21:43:39.372390Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372322Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 0
2025-10-09T21:43:39.372326Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 13 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 10111 count 12 nextOffset 13 batches 1
2025-10-09T21:43:39.372330Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 0
2025-10-09T21:43:39.372333Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 14 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 11666 count 13 nextOffset 14 batches 1
2025-10-09T21:43:39.372441Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 93
2025-10-09T21:43:39.372337Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 0
2025-10-09T21:43:39.372341Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 15 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 13331 count 14 nextOffset 15 batches 1
2025-10-09T21:43:39.372344Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 0
2025-10-09T21:43:39.372348Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 16 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 15106 count 15 nextOffset 16 batches 1
2025-10-09T21:43:39.372352Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 0
2025-10-09T21:43:39.372356Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 17 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 16991 count 16 nextOffset 17 batches 1
2025-10-09T21:43:39.372361Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 0
2025-10-09T21:43:39.372366Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 18 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 18986 count 17 nextOffset 18 batches 1
2025-10-09T21:43:39.372369Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.372461Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 1,17 HeadOffset 1 endOffset 1 curOffset 18 d0000000000_00000000000000000001_00000_0000000017_00000? size 18503 WTime 1760046219372
2025-10-09T21:43:39.372480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 94 to 94
2025-10-09T21:43:39.372498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372501Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 94
2025-10-09T21:43:39.372526Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 95 to 95
2025-10-09T21:43:39.372534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 95
2025-10-09T21:43:39.372557Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 96 to 96
2025-10-09T21:43:39.372563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 96
2025-10-09T21:43:39.372577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 97 to 97
2025-10-09T21:43:39.372587Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 97
2025-10-09T21:43:39.372605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 98 to 98
2025-10-09T21:43:39.372611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 98
2025-10-09T21:43:39.372633Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 99 to 99
2025-10-09T21:43:39.372640Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 99
2025-10-09T21:43:39.372654Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 100 to 100
2025-10-09T21:43:39.372663Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372666Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 100
2025-10-09T21:43:39.372687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 101 to 101
2025-10-09T21:43:39.372693Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372695Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 101
2025-10-09T21:43:39.372706Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 102 to 102
2025-10-09T21:43:39.372712Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372714Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 102
2025-10-09T21:43:39.372737Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 103 to 103
2025-10-09T21:43:39.372743Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372746Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 103
2025-10-09T21:43:39.372761Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 104 to 104
2025-10-09T21:43:39.372767Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372769Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 104
2025-10-09T21:43:39.372781Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 105 to 105
2025-10-09T21:43:39.372793Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372795Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 105
2025-10-09T21:43:39.372809Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 106 to 106
2025-10-09T21:43:39.372821Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372824Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 106
2025-10-09T21:43:39.372846Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 107 to 107
2025-10-09T21:43:39.372875Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372878Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 107
2025-10-09T21:43:39.372889Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 108 to 108
2025-10-09T21:43:39.372895Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372898Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 108
2025-10-09T21:43:39.372922Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 109 to 109
2025-10-09T21:43:39.372943Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372946Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 109
2025-10-09T21:43:39.372958Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 110 to 110
2025-10-09T21:43:39.372965Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.372968Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 110
2025-10-09T21:43:39.372995Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 111 to 111
2025-10-09T21:43:39.373002Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373004Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 111
2025-10-09T21:43:39.373016Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 112 to 112
2025-10-09T21:43:39.373033Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373036Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 112
2025-10-09T21:43:39.373052Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 113 to 113
2025-10-09T21:43:39.373062Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373065Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 113
2025-10-09T21:43:39.373081Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 114 to 114
2025-10-09T21:43:39.373094Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373096Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 114
2025-10-09T21:43:39.373122Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 115 to 115
2025-10-09T21:43:39.373133Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373136Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 115
2025-10-09T21:43:39.373150Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 116 to 116
2025-10-09T21:43:39.373169Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373172Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 116
2025-10-09T21:43:39.373188Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 117 to 117
2025-10-09T21:43:39.373201Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373203Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 117
2025-10-09T21:43:39.373217Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 118 to 118
2025-10-09T21:43:39.373243Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373246Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 118
2025-10-09T21:43:39.373269Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 119 to 119
2025-10-09T21:43:39.373281Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373294Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 119
2025-10-09T21:43:39.373307Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 120 to 120
2025-10-09T21:43:39.373327Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373330Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 120
2025-10-09T21:43:39.373348Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 121 to 121
2025-10-09T21:43:39.373360Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373363Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 121
2025-10-09T21:43:39.372503Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.372525Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 3  requestId: <none> cookie: 1
2025-10-09T21:43:39.372557Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.372568Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 1 partNo 0 count 17 size 18503
2025-10-09T21:43:39.372623Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 1 max time lag 0ms effective offset 0
2025-10-09T21:43:39.372636Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 142 count 1 last offset 0, current partition end offset: 1
2025-10-09T21:43:39.372639Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:39.372670Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 0 source 1 size 142 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.372673Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 3. All 1 blobs are from cache.
2025-10-09T21:43:39.373384Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 122 to 122
2025-10-09T21:43:39.372684Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.373443Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373449Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 122
2025-10-09T21:43:39.372692Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63' size 142
2025-10-09T21:43:39.372694Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 1 count 1 size 124 from pos 0 cbcount 1
2025-10-09T21:43:39.372702Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63'
2025-10-09T21:43:39.372710Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046219368 queuesize 0 startOffset 0
2025-10-09T21:43:39.372733Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 0 EndOffset 0 newEndOffset 1 commitOffset 0 clientCommitOffset 0 cookie 1 readingFinished 0 firstRead 1
2025-10-09T21:43:39.372736Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 0 endOffset 1
2025-10-09T21:43:39.372747Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.372892Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 0, endOffset# 1, WTime# 0, sizeLag# 142
2025-10-09T21:43:39.372894Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:43:39.372908Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# a75d0eef-48ae27cb-9e716d1b-7095d0f1, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 1, size# 170, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.372942Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 1 maxSize 170 maxTimeLagMs 0 readTimestampMs 0 readOffset 0 EndOffset 1 ClientCommitOffset 0 committedOffset 0 Guid a75d0eef-48ae27cb-9e716d1b-7095d0f1
2025-10-09T21:43:39.372971Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.373487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 123 to 123
2025-10-09T21:43:39.372976Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.372992Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 4 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 170 endOffset 1 max time lag 0ms effective offset 0
2025-10-09T21:43:39.373002Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 4 added 1 blobs, size 142 count 1 last offset 0, current partition end offset: 1
2025-10-09T21:43:39.373004Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 4. Send blob request.
2025-10-09T21:43:39.373010Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 0 source 1 size 142 accessed 1 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.373019Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 4. All 1 blobs are from cache.
2025-10-09T21:43:39.373025Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.373518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 123
2025-10-09T21:43:39.373034Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 1 count 1 size 124 from pos 0 cbcount 1
2025-10-09T21:43:39.373054Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:39.373081Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63'
2025-10-09T21:43:39.373123Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 1 Result { Offset: 0 Data: "... 82 bytes ..." SourceId: "\000src_id" SeqNo: 1 WriteTimestampMS: 1760046219368 CreateTimestampMS: 1760046219368 UncompressedSize: 11 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 18 RealReadOffset: 0 WaitQuotaTimeMs: 0 EndOffset: 1 StartOffset: 0 } Cookie: 0 }
2025-10-09T21:43:39.373545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 124 to 124
2025-10-09T21:43:39.373162Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 2 from offset 1
2025-10-09T21:43:39.373170Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 1 ReadOffset 1 ReadGuid a75d0eef-48ae27cb-9e716d1b-7095d0f1 has messages 1
2025-10-09T21:43:39.373190Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# a75d0eef-48ae27cb-9e716d1b-7095d0f1, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 171
2025-10-09T21:43:39.373196Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# a75d0eef-48ae27cb-9e716d1b-7095d0f1
2025-10-09T21:43:39.373274Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 0
2025-10-09T21:43:39.373564Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 124
2025-10-09T21:43:39.373580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 125 to 125
2025-10-09T21:43:39.373590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373592Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 125
2025-10-09T21:43:39.373619Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 126 to 126
2025-10-09T21:43:39.373648Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373650Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 126
2025-10-09T21:43:39.373664Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 127 to 127
2025-10-09T21:43:39.373691Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373694Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 127
2025-10-09T21:43:39.373708Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 128 to 128
2025-10-09T21:43:39.373732Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373735Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 128
2025-10-09T21:43:39.373749Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 129 to 129
2025-10-09T21:43:39.373783Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373786Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 129
2025-10-09T21:43:39.373800Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 130 to 130
2025-10-09T21:43:39.373821Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373823Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 130
2025-10-09T21:43:39.373837Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 131 to 131
2025-10-09T21:43:39.373849Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373852Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 131
2025-10-09T21:43:39.373865Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 132 to 132
2025-10-09T21:43:39.373886Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373889Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 132
2025-10-09T21:43:39.373905Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 133 to 133
2025-10-09T21:43:39.373917Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373920Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 133
2025-10-09T21:43:39.373935Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 134 to 134
2025-10-09T21:43:39.373960Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373961Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 134
2025-10-09T21:43:39.373975Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 135 to 135
2025-10-09T21:43:39.373985Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.373987Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 135
2025-10-09T21:43:39.373998Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 136 to 136
2025-10-09T21:43:39.374017Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.374019Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 136
2025-10-09T21:43:39.374052Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 137 to 137
2025-10-09T21:43:39.374083Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.374086Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 137
2025-10-09T21:43:39.374353Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 138 to 138
2025-10-09T21:43:39.374376Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 171, now ReadSizeBudget = 0, ReadSizeServerDelta = 524117
2025-10-09T21:43:39.374430Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 0, ReadSizeServerDelta = 524117
2025-10-09T21:43:39.374508Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (0-0)
2025-10-09T21:43:39.374520Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 171 to budget
2025-10-09T21:43:39.374525Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 171, ReadSizeServerDelta = 524117
2025-10-09T21:43:39.374301Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.374563Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.374362Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.374583Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (0-0)
2025-10-09T21:43:39.374396Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.374399Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.374438Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 36  requestId: <none> cookie: 19
2025-10-09T21:43:39.374447Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.374466Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.374470Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.374482Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 19 partNo : 0 messageNo: 37 size 2063 offset: -1
2025-10-09T21:43:39.374486Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.374497Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.374572Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.374615Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.374636Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.374645Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.374654Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 38  requestId: <none> cookie: 20
2025-10-09T21:43:39.374660Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.374664Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.374701Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.374733Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 171 } }
2025-10-09T21:43:39.374816Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# a4418ed4-5e376637-885f15d4-9b163bc0
2025-10-09T21:43:39.374831Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.374834Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.374846Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 20 partNo : 0 messageNo: 39 size 2173 offset: -1
2025-10-09T21:43:39.374851Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.374853Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.374858Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.374867Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.374875Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 40  requestId: <none> cookie: 21
2025-10-09T21:43:39.374883Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.374890Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.374937Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.374958Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.374961Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.374968Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 21 partNo : 0 messageNo: 41 size 2283 offset: -1
2025-10-09T21:43:39.374972Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.374974Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.374978Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.374983Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.374988Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 42  requestId: <none> cookie: 22
2025-10-09T21:43:39.374994Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.374999Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.375042Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.375061Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.375095Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.375102Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.375109Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.375116Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 22 partNo : 0 messageNo: 43 size 2393 offset: -1
2025-10-09T21:43:39.375121Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.375123Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.375127Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.375129Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.375132Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.375138Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.375144Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 44  requestId: <none> cookie: 23
2025-10-09T21:43:39.375148Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 45  requestId: <none> cookie: 24
2025-10-09T21:43:39.375154Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.375159Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.375176Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.375178Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.375183Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 23 partNo : 0 messageNo: 46 size 2503 offset: -1
2025-10-09T21:43:39.375186Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.375187Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.375191Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 24 partNo : 0 messageNo: 47 size 2613 offset: -1
2025-10-09T21:43:39.375194Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.375197Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.375199Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.375202Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.375207Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.375242Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.375255Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.375286Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.375292Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.375316Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.375320Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.375353Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.375356Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.375382Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.375386Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.375404Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.375406Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.375410Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.375412Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.375419Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 48  requestId: <none> cookie: 25
2025-10-09T21:43:39.375430Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 49  requestId: <none> cookie: 26
2025-10-09T21:43:39.375434Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.375438Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.375444Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.375520Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.375581Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.375600Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.375609Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.375626Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.375634Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.375638Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.375641Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.375644Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.375646Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.375655Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 25 partNo : 0 messageNo: 53 size 2723 offset: -1
2025-10-09T21:43:39.375659Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.375661Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.375665Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 54 size 2833 offset: -1
2025-10-09T21:43:39.375669Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.375672Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.375677Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.375682Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.375710Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.375722Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.375725Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.375730Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.375737Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 50  requestId: <none> cookie: 27
2025-10-09T21:43:39.375743Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 51  requestId: <none> cookie: 28
2025-10-09T21:43:39.375745Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 52  requestId: <none> cookie: 29
2025-10-09T21:43:39.375748Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 55  requestId: <none> cookie: 30
2025-10-09T21:43:39.375750Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 56  requestId: <none> cookie: 31
2025-10-09T21:43:39.375755Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.375761Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.375767Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.375772Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.375776Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.375779Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.376074Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.376090Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.376118Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.376138Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.376142Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.376152Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 57 size 2943 offset: -1
2025-10-09T21:43:39.376158Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.376160Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.376165Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 58 size 3053 offset: -1
2025-10-09T21:43:39.376168Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.376170Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.376175Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 59 size 3163 offset: -1
2025-10-09T21:43:39.376178Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.376180Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.376185Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 60 size 3273 offset: -1
2025-10-09T21:43:39.376188Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.376190Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.376194Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 61 size 3383 offset: -1
2025-10-09T21:43:39.376199Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.376205Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.376222Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.376226Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.376228Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.376231Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.376233Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.376236Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.376239Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.376241Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.376245Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.376246Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.376250Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.376253Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.376400Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 62  requestId: <none> cookie: 32
2025-10-09T21:43:39.376406Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 63  requestId: <none> cookie: 33
2025-10-09T21:43:39.376412Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.376418Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.376424Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.376467Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.376477Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.376480Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.376487Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 64 size 3493 offset: -1
2025-10-09T21:43:39.376491Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.376493Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.376497Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 65 size 3603 offset: -1
2025-10-09T21:43:39.376500Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.376502Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.376506Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.376511Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.376514Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.376516Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.376521Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 66  requestId: <none> cookie: 34
2025-10-09T21:43:39.376526Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.376541Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.376543Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.376546Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 67 size 3713 offset: -1
2025-10-09T21:43:39.376549Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.376553Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.376710Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.376768Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.376777Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.376823Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.374387Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.378067Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 138
>>> event from dataHandler: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..11 bytes.. Information: { Offset: 0 ProducerId: "src_id" SeqNo: 1 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.368000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.378152Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 1, size 11 bytes
2025-10-09T21:43:39.378163Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.378174Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 139 to 139
2025-10-09T21:43:39.378188Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.378191Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 139
2025-10-09T21:43:39.378248Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 140 to 140
2025-10-09T21:43:39.378257Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.378260Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 140
2025-10-09T21:43:39.378276Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 141 to 141
2025-10-09T21:43:39.376854Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.376867Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.376876Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.376881Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.376888Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.376890Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.376893Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.376895Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.376908Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 68  requestId: <none> cookie: 35
2025-10-09T21:43:39.376913Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 69  requestId: <none> cookie: 36
2025-10-09T21:43:39.376916Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 70  requestId: <none> cookie: 37
2025-10-09T21:43:39.376921Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.376925Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.376930Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.376934Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.376969Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.376972Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.377002Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.377051Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.377054Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.377062Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 71 size 3823 offset: -1
2025-10-09T21:43:39.377066Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.377068Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.377071Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 36 partNo : 0 messageNo: 72 size 3933 offset: -1
2025-10-09T21:43:39.377075Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.377076Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.377080Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 37 partNo : 0 messageNo: 73 size 4043 offset: -1
2025-10-09T21:43:39.377083Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.377085Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.377088Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.377091Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.377102Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 1 count 17 size 18503 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.377117Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.377124Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.377126Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.377129Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.377131Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.377133Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.377137Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 17 parts 0 suffix '63' size 18503
2025-10-09T21:43:39.377139Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.377155Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 18359 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.377162Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377170Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 2, partNo: 0, Offset: 1 is stored on disk
2025-10-09T21:43:39.377173Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377177Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 3, partNo: 0, Offset: 2 is stored on disk
2025-10-09T21:43:39.377179Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377182Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 4, partNo: 0, Offset: 3 is stored on disk
2025-10-09T21:43:39.377184Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377187Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 5, partNo: 0, Offset: 4 is stored on disk
2025-10-09T21:43:39.377189Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377194Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 6, partNo: 0, Offset: 5 is stored on disk
2025-10-09T21:43:39.377197Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377201Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 7, partNo: 0, Offset: 6 is stored on disk
2025-10-09T21:43:39.377203Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377206Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 8, partNo: 0, Offset: 7 is stored on disk
2025-10-09T21:43:39.377209Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377212Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 9, partNo: 0, Offset: 8 is stored on disk
2025-10-09T21:43:39.377215Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377218Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 10, partNo: 0, Offset: 9 is stored on disk
2025-10-09T21:43:39.377220Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377224Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 11, partNo: 0, Offset: 10 is stored on disk
2025-10-09T21:43:39.377226Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377230Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 12, partNo: 0, Offset: 11 is stored on disk
2025-10-09T21:43:39.377233Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377236Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 13, partNo: 0, Offset: 12 is stored on disk
2025-10-09T21:43:39.377238Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377242Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 14, partNo: 0, Offset: 13 is stored on disk
2025-10-09T21:43:39.377245Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377249Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 15, partNo: 0, Offset: 14 is stored on disk
2025-10-09T21:43:39.377251Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377255Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 16, partNo: 0, Offset: 15 is stored on disk
2025-10-09T21:43:39.377258Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377261Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 17, partNo: 0, Offset: 16 is stored on disk
2025-10-09T21:43:39.377264Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.377268Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 18, partNo: 0, Offset: 17 is stored on disk
2025-10-09T21:43:39.377295Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.377309Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.377311Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (19)
2025-10-09T21:43:39.377330Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 0
2025-10-09T21:43:39.377362Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 19 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 2137 count 1 nextOffset 19 batches 1
2025-10-09T21:43:39.377368Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 0
2025-10-09T21:43:39.377372Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 20 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 4352 count 2 nextOffset 20 batches 1
2025-10-09T21:43:39.377375Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 0
2025-10-09T21:43:39.377380Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 21 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 6677 count 3 nextOffset 21 batches 1
2025-10-09T21:43:39.377383Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 22 partNo 0
2025-10-09T21:43:39.377388Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 22 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 9112 count 4 nextOffset 22 batches 1
2025-10-09T21:43:39.377391Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 23 partNo 0
2025-10-09T21:43:39.377396Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 23 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 11657 count 5 nextOffset 23 batches 1
2025-10-09T21:43:39.377399Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 24 partNo 0
2025-10-09T21:43:39.377402Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 24 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 14312 count 6 nextOffset 24 batches 1
2025-10-09T21:43:39.377406Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 25 partNo 0
2025-10-09T21:43:39.377411Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 25 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 17077 count 7 nextOffset 25 batches 1
2025-10-09T21:43:39.377414Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 0
2025-10-09T21:43:39.377418Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 26 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 19952 count 8 nextOffset 26 batches 1
2025-10-09T21:43:39.377421Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 0
2025-10-09T21:43:39.377426Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 27 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 22937 count 9 nextOffset 27 batches 1
2025-10-09T21:43:39.377429Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 0
2025-10-09T21:43:39.377435Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 28 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 26032 count 10 nextOffset 28 batches 1
2025-10-09T21:43:39.377438Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 0
2025-10-09T21:43:39.377442Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 29 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 29237 count 11 nextOffset 29 batches 1
2025-10-09T21:43:39.377444Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 0
2025-10-09T21:43:39.377448Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 30 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 32552 count 12 nextOffset 30 batches 1
2025-10-09T21:43:39.377451Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 0
2025-10-09T21:43:39.377454Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 31 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 35977 count 13 nextOffset 31 batches 1
2025-10-09T21:43:39.377458Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 0
2025-10-09T21:43:39.377462Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 32 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 39512 count 14 nextOffset 32 batches 1
2025-10-09T21:43:39.377465Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 0
2025-10-09T21:43:39.377468Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 33 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 43157 count 15 nextOffset 33 batches 1
2025-10-09T21:43:39.377471Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 0
2025-10-09T21:43:39.377475Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 34 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 46912 count 16 nextOffset 34 batches 1
2025-10-09T21:43:39.377478Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 0
2025-10-09T21:43:39.377484Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 35 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 50777 count 17 nextOffset 35 batches 1
2025-10-09T21:43:39.377487Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 36 partNo 0
2025-10-09T21:43:39.377491Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 36 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 54752 count 18 nextOffset 36 batches 1
2025-10-09T21:43:39.377494Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 37 partNo 0
2025-10-09T21:43:39.377497Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 37 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 58837 count 19 nextOffset 37 batches 1
2025-10-09T21:43:39.377500Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.377601Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 18,19 HeadOffset 18 endOffset 18 curOffset 37 d0000000000_00000000000000000018_00000_0000000019_00000? size 58291 WTime 1760046219377
2025-10-09T21:43:39.377629Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.377638Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 74  requestId: <none> cookie: 38
2025-10-09T21:43:39.377643Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 75  requestId: <none> cookie: 39
2025-10-09T21:43:39.377648Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 4  requestId: <none> cookie: 2
2025-10-09T21:43:39.377655Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 5  requestId: <none> cookie: 3
2025-10-09T21:43:39.377660Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 10  requestId: <none> cookie: 4
2025-10-09T21:43:39.377665Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 11  requestId: <none> cookie: 5
2025-10-09T21:43:39.377669Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 12  requestId: <none> cookie: 6
2025-10-09T21:43:39.377674Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 13  requestId: <none> cookie: 7
2025-10-09T21:43:39.377679Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 19  requestId: <none> cookie: 8
2025-10-09T21:43:39.377684Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 20  requestId: <none> cookie: 9
2025-10-09T21:43:39.377688Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 21  requestId: <none> cookie: 10
2025-10-09T21:43:39.377692Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 22  requestId: <none> cookie: 11
2025-10-09T21:43:39.377698Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 23  requestId: <none> cookie: 12
2025-10-09T21:43:39.377702Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 29  requestId: <none> cookie: 13
2025-10-09T21:43:39.377706Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 30  requestId: <none> cookie: 14
2025-10-09T21:43:39.377712Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 31  requestId: <none> cookie: 15
2025-10-09T21:43:39.377717Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 32  requestId: <none> cookie: 16
2025-10-09T21:43:39.377722Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 33  requestId: <none> cookie: 17
2025-10-09T21:43:39.377727Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 35  requestId: <none> cookie: 18
2025-10-09T21:43:39.377732Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.377742Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 18 partNo 0 count 19 size 58291
2025-10-09T21:43:39.377797Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.377804Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377810Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377814Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377818Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377821Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377825Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377828Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377831Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377835Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377838Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377841Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377844Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377847Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377851Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377854Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377858Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377861Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377864Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377867Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.377914Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 1 EndOffset 1 newEndOffset 18 commitOffset 0 clientCommitOffset 0 cookie 2 readingFinished 0 firstRead 0
2025-10-09T21:43:39.377920Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 1 endOffset 18
2025-10-09T21:43:39.377933Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 1, endOffset# 18, WTime# 1760046219368, sizeLag# 18503
2025-10-09T21:43:39.377936Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:43:39.377950Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# 1fbea90d-db35b85a-fb1fe6a5-6a8964c, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 20, size# 22203, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.377964Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 20 maxSize 22203 maxTimeLagMs 0 readTimestampMs 0 readOffset 1 EndOffset 18 ClientCommitOffset 0 committedOffset 0 Guid 1fbea90d-db35b85a-fb1fe6a5-6a8964c
2025-10-09T21:43:39.377982Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.377987Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.377999Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 38 partNo : 0 messageNo: 76 size 4153 offset: -1
2025-10-09T21:43:39.378006Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.378008Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.378013Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 39 partNo : 0 messageNo: 77 size 4263 offset: -1
2025-10-09T21:43:39.378019Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.378044Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.378047Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.378050Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.378060Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.378063Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.378085Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 5 Topic 'rt3.dc1--test-topic' partition 0 user user offset 1 count 20 size 22203 endOffset 18 max time lag 0ms effective offset 1
2025-10-09T21:43:39.378103Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 5 added 1 blobs, size 18503 count 17 last offset 1, current partition end offset: 18
2025-10-09T21:43:39.378106Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 5. Send blob request.
2025-10-09T21:43:39.378109Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.378114Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 1 partno 0 count 17 parts_count 0 source 1 size 18503 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.378117Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.378126Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 5. All 1 blobs are from cache.
2025-10-09T21:43:39.378136Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.378142Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 17 parts 0 suffix '63'
2025-10-09T21:43:39.378156Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.378223Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 1 totakecount 17 count 17 size 18481 from pos 0 cbcount 17
2025-10-09T21:43:39.378260Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.378263Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.378269Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.378271Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.378289Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:39.378339Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 78  requestId: <none> cookie: 40
2025-10-09T21:43:39.378343Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 79  requestId: <none> cookie: 41
2025-10-09T21:43:39.378349Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.378358Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.378364Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.378444Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.378594Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 18 Result { Offset: 1 Data: "... 192 bytes ..." SourceId: "\000src_id" SeqNo: 2 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219368 UncompressedSize: 121 PartitionKey: "" ExplicitHash: "" } Result { Offset: 2 Data: "... 303 bytes ..." SourceId: "\000src_id" SeqNo: 3 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219368 UncompressedSize: 231 PartitionKey: "" ExplicitHash: "" } Result { Offset: 3 Data: "... 413 bytes ..." SourceId: "\000src_id" SeqNo: 4 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219368 UncompressedSize: 341 PartitionKey: "" ExplicitHash: "" } Result { Offset: 4 Data: "... 523 bytes ..." SourceId: "\000src_id" SeqNo: 5 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219368 UncompressedSize: 451 PartitionKey: "" ExplicitHash: "" } Result { Offset: 5 Data: "... 633 bytes ..." SourceId: "\000src_id" SeqNo: 6 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219368 UncompressedSize: 561 PartitionKey: "" ExplicitHash: "" } Result { Offset: 6 Data: "... 743 bytes ..." SourceId: "\000src_id" SeqNo: 7 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219368 UncompressedSize: 671 PartitionKey: "" ExplicitHash: "" } Result { Offset: 7 Data: "... 853 bytes ..." SourceId: "\000src_id" SeqNo: 8 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219368 UncompressedSize: 781 PartitionKey: "" ExplicitHash: "" } Result { Offset: 8 Data: "... 963 bytes ..." SourceId: "\000src_id" SeqNo: 9 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219368 UncompressedSize: 891 PartitionKey: "" ExplicitHash: "" } Result { Offset: 9 Data: "... 1073 bytes ..." SourceId: "\000src_id" SeqNo: 10 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219368 UncompressedSize: 1001 PartitionKey: "" ExplicitHash: "" } Result { Offset: 10 Data: "... 1183 bytes ..." SourceId: "\000src_id" SeqNo: 11 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219368 UncompressedSize: 1111 PartitionKey: "" ExplicitHash: "" } Result { Offset: 11 Data: "... 1293 bytes ..." SourceId: "\000src_id" SeqNo: 12 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219368 UncompressedSize: 1221 PartitionKey: "" ExplicitHash: "" } Result { Offset: 12 Data: "... 1403 bytes ..." SourceId: "\000src_id" SeqNo: 13 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219368 UncompressedSize: 1331 PartitionKey: "" ExplicitHash: "" } Result { Offset: 13 Data: "... 1513 bytes ..." SourceId: "\000src_id" SeqNo: 14 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219368 UncompressedSize: 1441 PartitionKey: "" ExplicitHash: "" } Result { Offset: 14 Data: "... 1623 bytes ..." SourceId: "\000src_id" SeqNo: 15 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219368 UncompressedSize: 1551 PartitionKey: "" ExplicitHash: "" } Result { Offset: 15 Data: "... 1733 bytes ..." SourceId: "\000src_id" SeqNo: 16 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219368 UncompressedSize: 1661 PartitionKey: "" ExplicitHash: "" } Result { Offset: 16 Data: "... 1843 bytes ..." SourceId: "\000src_id" SeqNo: 17 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219368 UncompressedSize: 1771 PartitionKey: "" ExplicitHash: "" } Result { Offset: 17 Data: "... 1953 bytes ..." SourceId: "\000src_id" SeqNo: 18 WriteTimestampMS: 1760046219372 CreateTimestampMS: 1760046219369 UncompressedSize: 1881 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 18446744073709551165 RealReadOffset: 17 WaitQuotaTimeMs: 0 EndOffset: 18 StartOffset: 0 } Cookie: 1 }
2025-10-09T21:43:39.378663Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 3 from offset 18
2025-10-09T21:43:39.378670Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 18 ReadOffset 18 ReadGuid 1fbea90d-db35b85a-fb1fe6a5-6a8964c has messages 1
2025-10-09T21:43:39.378707Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.378808Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.378811Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.378821Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 40 partNo : 0 messageNo: 80 size 4373 offset: -1
2025-10-09T21:43:39.378826Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.378828Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.378832Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 41 partNo : 0 messageNo: 81 size 4483 offset: -1
2025-10-09T21:43:39.378836Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.378838Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.378844Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.378850Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.378852Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.378854Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.378864Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 82  requestId: <none> cookie: 42
2025-10-09T21:43:39.378872Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.378882Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.378951Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.378981Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.378987Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.378284Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.379007Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 141
2025-10-09T21:43:39.379111Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 142 to 142
2025-10-09T21:43:39.379123Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.379126Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 142
2025-10-09T21:43:39.379174Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 143 to 143
2025-10-09T21:43:39.379180Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.379183Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 143
2025-10-09T21:43:39.379229Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 144 to 144
2025-10-09T21:43:39.379236Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.379238Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 144
2025-10-09T21:43:39.379259Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 145 to 145
2025-10-09T21:43:39.379265Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.379268Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 145
2025-10-09T21:43:39.379276Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 146 to 146
2025-10-09T21:43:39.379282Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.379285Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 146
2025-10-09T21:43:39.379293Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 147 to 147
2025-10-09T21:43:39.379299Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.379302Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 147
2025-10-09T21:43:39.379308Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 148 to 148
2025-10-09T21:43:39.379315Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.379317Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 148
2025-10-09T21:43:39.379346Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 149 to 149
2025-10-09T21:43:39.379363Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.379363Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 17743, now ReadSizeBudget = 0, ReadSizeServerDelta = 506545
2025-10-09T21:43:39.379369Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 149
2025-10-09T21:43:39.379404Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 0, ReadSizeServerDelta = 506545
2025-10-09T21:43:39.379457Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (1-17)
2025-10-09T21:43:39.379471Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 17743 to budget
2025-10-09T21:43:39.379477Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 17743, ReadSizeServerDelta = 506545
2025-10-09T21:43:39.379504Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 150 to 150
2025-10-09T21:43:39.379527Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.379531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 150
2025-10-09T21:43:39.379561Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.379567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 151 to 151
2025-10-09T21:43:39.379576Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (1-1)
2025-10-09T21:43:39.379577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.379580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 151
2025-10-09T21:43:39.379583Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (2-2)
2025-10-09T21:43:39.379589Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 2} (3-3)
2025-10-09T21:43:39.379593Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 3} (4-4)
2025-10-09T21:43:39.379599Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 4} (5-5)
2025-10-09T21:43:39.379602Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 5} (6-6)
2025-10-09T21:43:39.379606Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 6} (7-7)
2025-10-09T21:43:39.379610Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 7} (8-8)
2025-10-09T21:43:39.379612Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 152 to 152
2025-10-09T21:43:39.379619Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 8} (9-9)
2025-10-09T21:43:39.379623Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 9} (10-10)
2025-10-09T21:43:39.379627Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 10} (11-11)
2025-10-09T21:43:39.379643Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.379643Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 11} (12-12)
2025-10-09T21:43:39.379646Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 152
2025-10-09T21:43:39.379649Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 12} (13-13)
2025-10-09T21:43:39.379653Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 13} (14-14)
2025-10-09T21:43:39.379657Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 14} (15-15)
2025-10-09T21:43:39.379664Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 15} (16-16)
2025-10-09T21:43:39.379685Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 16} (17-17)
2025-10-09T21:43:39.379687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 153 to 153
2025-10-09T21:43:39.379704Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.379707Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 153
2025-10-09T21:43:39.379729Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 154 to 154
2025-10-09T21:43:39.379746Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.379749Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 154
2025-10-09T21:43:39.379767Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 155 to 155
>>> event from dataHandler: 2025-10-09T21:43:39.379784Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.379787Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 155
2025-10-09T21:43:39.379826Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 1 offsets: 0 already_written: false write_statistics { persist_duration_ms: 3 }
2025-10-09T21:43:39.379834Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 1
DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..121 bytes.. Information: { Offset: 1 ProducerId: "src_id" SeqNo: 2 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..231 bytes.. Information: { Offset: 2 ProducerId: "src_id" SeqNo: 3 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..341 bytes.. Information: { Offset: 3 ProducerId: "src_id" SeqNo: 4 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..451 bytes.. Information: { Offset: 4 ProducerId: "src_id" SeqNo: 5 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..561 bytes.. Information: { Offset: 5 ProducerId: "src_id" SeqNo: 6 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..671 bytes.. Information: { Offset: 6 ProducerId: "src_id" SeqNo: 7 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..781 bytes.. Information: { Offset: 7 ProducerId: "src_id" SeqNo: 8 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..891 bytes.. Information: { Offset: 8 ProducerId: "src_id" SeqNo: 9 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..1001 bytes.. Information: { Offset: 9 ProducerId: "src_id" SeqNo: 10 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..1111 bytes.. Information: { Offset: 10 ProducerId: "src_id" SeqNo: 11 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..1221 bytes.. Information: { Offset: 11 ProducerId: "src_id" SeqNo: 12 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..1331 bytes.. Information: { Offset: 12 ProducerId: "src_id" SeqNo: 13 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..1441 bytes.. Information: { Offset: 13 ProducerId: "src_id" SeqNo: 14 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..1551 bytes.. Information: { Offset: 14 ProducerId: "src_id" SeqNo: 15 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..1661 bytes.. Information: { Offset: 15 ProducerId: "src_id" SeqNo: 16 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..1771 bytes.. Information: { Offset: 16 ProducerId: "src_id" SeqNo: 17 CreateTime: 2025-10-09T21:43:39.368000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..1881 bytes.. Information: { Offset: 17 ProducerId: "src_id" SeqNo: 18 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.372000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.379860Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 156 to 156
2025-10-09T21:43:39.379873Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.379875Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 156
2025-10-09T21:43:39.379883Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 17, size 17017 bytes
2025-10-09T21:43:39.379893Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.380324Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 157 to 157
2025-10-09T21:43:39.380341Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.380346Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 157
2025-10-09T21:43:39.380430Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 158 to 158
2025-10-09T21:43:39.380438Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.380441Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 158
2025-10-09T21:43:39.380460Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 159 to 159
2025-10-09T21:43:39.380466Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.380469Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 159
2025-10-09T21:43:39.380523Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 160 to 160
2025-10-09T21:43:39.380529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.380532Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 160
2025-10-09T21:43:39.380552Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 161 to 161
2025-10-09T21:43:39.380558Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.380561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 161
2025-10-09T21:43:39.380583Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 162 to 162
2025-10-09T21:43:39.380588Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.380590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 162
2025-10-09T21:43:39.381074Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 163 to 163
2025-10-09T21:43:39.381089Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.381093Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 163
2025-10-09T21:43:39.381198Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 164 to 164
2025-10-09T21:43:39.381205Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.381218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 164
2025-10-09T21:43:39.381293Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 165 to 165
2025-10-09T21:43:39.381299Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.381302Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 165
2025-10-09T21:43:39.381369Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 166 to 166
2025-10-09T21:43:39.381375Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.381377Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 166
2025-10-09T21:43:39.381433Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 167 to 167
2025-10-09T21:43:39.381439Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.381442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 167
2025-10-09T21:43:39.381504Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 168 to 168
2025-10-09T21:43:39.381511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.381514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 168
2025-10-09T21:43:39.381560Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 169 to 169
2025-10-09T21:43:39.381566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.381569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 169
2025-10-09T21:43:39.381628Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 170 to 170
2025-10-09T21:43:39.381634Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.381636Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 170
2025-10-09T21:43:39.381689Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 171 to 171
2025-10-09T21:43:39.381705Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.381707Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 171
2025-10-09T21:43:39.381783Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 172 to 172
2025-10-09T21:43:39.381790Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.381794Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 172
2025-10-09T21:43:39.378991Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.379051Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.379075Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.379081Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.379103Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# 1fbea90d-db35b85a-fb1fe6a5-6a8964c, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 17743
2025-10-09T21:43:39.379112Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# 1fbea90d-db35b85a-fb1fe6a5-6a8964c
2025-10-09T21:43:39.379209Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 0
2025-10-09T21:43:39.379233Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.379237Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.379245Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 42 partNo : 0 messageNo: 83 size 4593 offset: -1
2025-10-09T21:43:39.379249Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.379251Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.379255Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.379256Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.379260Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.379261Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.379264Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.379265Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.379269Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.379275Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.379284Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 84  requestId: <none> cookie: 43
2025-10-09T21:43:39.379289Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 85  requestId: <none> cookie: 44
2025-10-09T21:43:39.379291Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 86  requestId: <none> cookie: 45
2025-10-09T21:43:39.379294Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 87  requestId: <none> cookie: 46
2025-10-09T21:43:39.379298Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.379303Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.379307Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.379311Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.379315Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.379361Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.379366Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.379387Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.379389Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.379392Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.379401Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.379404Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 43 partNo : 0 messageNo: 88 size 4703 offset: -1
2025-10-09T21:43:39.379411Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.379413Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.379418Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 44 partNo : 0 messageNo: 89 size 4813 offset: -1
2025-10-09T21:43:39.379421Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.379423Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.379428Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 45 partNo : 0 messageNo: 90 size 4923 offset: -1
2025-10-09T21:43:39.379432Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.379434Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.379438Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 46 partNo : 0 messageNo: 91 size 5033 offset: -1
2025-10-09T21:43:39.379441Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.379443Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.379447Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.379471Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.379476Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.379484Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.379485Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.379486Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.379491Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.379498Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.379500Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.379502Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.379504Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.379506Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.379508Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.379511Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.379513Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.379521Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 18 count 19 size 58291 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.379529Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.379530Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.379533Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.379535Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 92  requestId: <none> cookie: 47
2025-10-09T21:43:39.379536Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 19 parts 0 suffix '63' size 58291
2025-10-09T21:43:39.379539Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.379541Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.379544Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.379546Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.379549Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.379554Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.379562Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 58140 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.379568Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379571Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.379574Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 19, partNo: 0, Offset: 18 is stored on disk
2025-10-09T21:43:39.379578Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379582Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 20, partNo: 0, Offset: 19 is stored on disk
2025-10-09T21:43:39.379584Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.382175Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 2 offsets: 1 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 3 }
2025-10-09T21:43:39.379587Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 21, partNo: 0, Offset: 20 is stored on disk
2025-10-09T21:43:39.382193Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 2
2025-10-09T21:43:39.379589Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379593Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 22, partNo: 0, Offset: 21 is stored on disk
2025-10-09T21:43:39.379596Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379599Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 23, partNo: 0, Offset: 22 is stored on disk
2025-10-09T21:43:39.379601Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379605Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 24, partNo: 0, Offset: 23 is stored on disk
2025-10-09T21:43:39.379608Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379611Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 25, partNo: 0, Offset: 24 is stored on disk
2025-10-09T21:43:39.379613Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379616Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 26, partNo: 0, Offset: 25 is stored on disk
2025-10-09T21:43:39.379618Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379622Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 27, partNo: 0, Offset: 26 is stored on disk
2025-10-09T21:43:39.379624Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379627Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 28, partNo: 0, Offset: 27 is stored on disk
2025-10-09T21:43:39.379630Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379633Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 29, partNo: 0, Offset: 28 is stored on disk
2025-10-09T21:43:39.379636Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379640Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 30, partNo: 0, Offset: 29 is stored on disk
2025-10-09T21:43:39.379642Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379645Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 31, partNo: 0, Offset: 30 is stored on disk
2025-10-09T21:43:39.379647Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379650Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 32, partNo: 0, Offset: 31 is stored on disk
2025-10-09T21:43:39.379653Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379658Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 33, partNo: 0, Offset: 32 is stored on disk
2025-10-09T21:43:39.379661Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379665Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 34, partNo: 0, Offset: 33 is stored on disk
2025-10-09T21:43:39.379667Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379670Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 35, partNo: 0, Offset: 34 is stored on disk
2025-10-09T21:43:39.379673Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379676Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 36, partNo: 0, Offset: 35 is stored on disk
2025-10-09T21:43:39.379679Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.379683Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 37, partNo: 0, Offset: 36 is stored on disk
2025-10-09T21:43:39.382327Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 3 offsets: 2 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 3 }
2025-10-09T21:43:39.379683Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.382334Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 3
2025-10-09T21:43:39.379707Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.379716Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.379719Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (9)
2025-10-09T21:43:39.379735Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 38 partNo 0
2025-10-09T21:43:39.379748Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 17743 } }
2025-10-09T21:43:39.379758Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 38 partNo 0 FormedBlobsCount 0 NewHead: Offset 37 PartNo 0 PackedSize 4227 count 1 nextOffset 38 batches 1
2025-10-09T21:43:39.379763Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 39 partNo 0
2025-10-09T21:43:39.379768Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 39 partNo 0 FormedBlobsCount 0 NewHead: Offset 37 PartNo 0 PackedSize 8532 count 2 nextOffset 39 batches 1
2025-10-09T21:43:39.379771Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 40 partNo 0
2025-10-09T21:43:39.379776Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 40 partNo 0 FormedBlobsCount 0 NewHead: Offset 37 PartNo 0 PackedSize 12947 count 3 nextOffset 40 batches 1
2025-10-09T21:43:39.379779Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 41 partNo 0
2025-10-09T21:43:39.379783Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 41 partNo 0 FormedBlobsCount 0 NewHead: Offset 37 PartNo 0 PackedSize 17472 count 4 nextOffset 41 batches 1
2025-10-09T21:43:39.379786Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 42 partNo 0
2025-10-09T21:43:39.382400Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 4 offsets: 3 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.379788Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# 5277bea1-2b7e27-7547bd2c-6e1906e0
2025-10-09T21:43:39.382406Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 4
2025-10-09T21:43:39.379791Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 42 partNo 0 FormedBlobsCount 0 NewHead: Offset 37 PartNo 0 PackedSize 22107 count 5 nextOffset 42 batches 1
2025-10-09T21:43:39.379795Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 43 partNo 0
2025-10-09T21:43:39.379799Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 43 partNo 0 FormedBlobsCount 0 NewHead: Offset 37 PartNo 0 PackedSize 26852 count 6 nextOffset 43 batches 1
2025-10-09T21:43:39.379801Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 18 EndOffset 18 newEndOffset 37 commitOffset 0 clientCommitOffset 0 cookie 3 readingFinished 0 firstRead 0
2025-10-09T21:43:39.379802Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 44 partNo 0
2025-10-09T21:43:39.379804Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 18 endOffset 37
2025-10-09T21:43:39.379807Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 44 partNo 0 FormedBlobsCount 0 NewHead: Offset 37 PartNo 0 PackedSize 31707 count 7 nextOffset 44 batches 1
2025-10-09T21:43:39.379810Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 45 partNo 0
2025-10-09T21:43:39.379812Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 18, endOffset# 37, WTime# 1760046219372, sizeLag# 58291
2025-10-09T21:43:39.379813Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 45 partNo 0 FormedBlobsCount 0 NewHead: Offset 37 PartNo 0 PackedSize 36672 count 8 nextOffset 45 batches 1
2025-10-09T21:43:39.379814Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:43:39.379816Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 46 partNo 0
2025-10-09T21:43:39.382471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 5 offsets: 4 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.382488Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 5
2025-10-09T21:43:39.382507Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 57437, now ReadSizeBudget = 0, ReadSizeServerDelta = 466851
2025-10-09T21:43:39.382513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 6 offsets: 5 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.382517Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 6
2025-10-09T21:43:39.382534Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 7 offsets: 6 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.382537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 7
2025-10-09T21:43:39.382539Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 0, ReadSizeServerDelta = 466851
2025-10-09T21:43:39.382568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 8 offsets: 7 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.382575Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 8
2025-10-09T21:43:39.382587Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (18-36)
2025-10-09T21:43:39.382605Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 57437 to budget
2025-10-09T21:43:39.382610Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 57437, ReadSizeServerDelta = 466851
2025-10-09T21:43:39.382691Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.382706Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (18-18)
2025-10-09T21:43:39.382725Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (19-19)
2025-10-09T21:43:39.382759Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 2} (20-20)
2025-10-09T21:43:39.382764Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 3} (21-21)
2025-10-09T21:43:39.382791Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 4} (22-22)
2025-10-09T21:43:39.382796Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 5} (23-23)
2025-10-09T21:43:39.382801Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 6} (24-24)
2025-10-09T21:43:39.382806Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 7} (25-25)
2025-10-09T21:43:39.382818Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 8} (26-26)
2025-10-09T21:43:39.382823Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 9} (27-27)
2025-10-09T21:43:39.382827Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 10} (28-28)
2025-10-09T21:43:39.382832Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 11} (29-29)
2025-10-09T21:43:39.382836Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 12} (30-30)
2025-10-09T21:43:39.382840Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 13} (31-31)
2025-10-09T21:43:39.382845Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 14} (32-32)
2025-10-09T21:43:39.382850Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 15} (33-33)
2025-10-09T21:43:39.382880Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 16} (34-34)
2025-10-09T21:43:39.382884Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 17} (35-35)
2025-10-09T21:43:39.382897Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 18} (36-36)
>>> event from dataHandler: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..1991 bytes.. Information: { Offset: 18 ProducerId: "src_id" SeqNo: 19 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..2101 bytes.. Information: { Offset: 19 ProducerId: "src_id" SeqNo: 20 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..2211 bytes.. Information: { Offset: 20 ProducerId: "src_id" SeqNo: 21 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..2321 bytes.. Information: { Offset: 21 ProducerId: "src_id" SeqNo: 22 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..2431 bytes.. Information: { Offset: 22 ProducerId: "src_id" SeqNo: 23 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..2541 bytes.. Information: { Offset: 23 ProducerId: "src_id" SeqNo: 24 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..2651 bytes.. Information: { Offset: 24 ProducerId: "src_id" SeqNo: 25 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..2761 bytes.. Information: { Offset: 25 ProducerId: "src_id" SeqNo: 26 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..2871 bytes.. Information: { Offset: 26 ProducerId: "src_id" SeqNo: 27 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..2981 bytes.. Information: { Offset: 27 ProducerId: "src_id" SeqNo: 28 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..3091 bytes.. Information: { Offset: 28 ProducerId: "src_id" SeqNo: 29 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..3201 bytes.. Information: { Offset: 29 ProducerId: "src_id" SeqNo: 30 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..3311 bytes.. Information: { Offset: 30 ProducerId: "src_id" SeqNo: 31 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..3421 bytes.. Information: { Offset: 31 ProducerId: "src_id" SeqNo: 32 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..3531 bytes.. Information: { Offset: 32 ProducerId: "src_id" SeqNo: 33 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..3641 bytes.. Information: { Offset: 33 ProducerId: "src_id" SeqNo: 34 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..3751 bytes.. Information: { Offset: 34 ProducerId: "src_id" SeqNo: 35 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..3861 bytes.. Information: { Offset: 35 ProducerId: "src_id" SeqNo: 36 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..3971 bytes.. Information: { Offset: 36 ProducerId: "src_id" SeqNo: 37 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.377000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.383141Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 19, size 56639 bytes
2025-10-09T21:43:39.383148Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.382592Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 9 offsets: 8 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.383225Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 9
2025-10-09T21:43:39.383229Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 41137, now ReadSizeBudget = 0, ReadSizeServerDelta = 483151
2025-10-09T21:43:39.379822Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 46 partNo 0 FormedBlobsCount 0 NewHead: Offset 37 PartNo 0 PackedSize 41747 count 9 nextOffset 46 batches 1
2025-10-09T21:43:39.379823Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# 4de0a44c-4f920a36-82d41590-254423da, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 22, size# 69949, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.379824Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.379834Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 22 maxSize 69949 maxTimeLagMs 0 readTimestampMs 0 readOffset 18 EndOffset 37 ClientCommitOffset 0 committedOffset 0 Guid 4de0a44c-4f920a36-82d41590-254423da
2025-10-09T21:43:39.379891Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 37,9 HeadOffset 37 endOffset 37 curOffset 46 d0000000000_00000000000000000037_00000_0000000009_00000? size 41521 WTime 1760046219379
2025-10-09T21:43:39.383271Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 0, ReadSizeServerDelta = 483151
2025-10-09T21:43:39.379961Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.379969Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 93  requestId: <none> cookie: 48
2025-10-09T21:43:39.379989Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 37  requestId: <none> cookie: 19
2025-10-09T21:43:39.379995Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 39  requestId: <none> cookie: 20
2025-10-09T21:43:39.379999Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 41  requestId: <none> cookie: 21
2025-10-09T21:43:39.380003Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.383289Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 10 offsets: 9 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.380006Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.380014Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 47 partNo : 0 messageNo: 97 size 5143 offset: -1
2025-10-09T21:43:39.380020Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 43  requestId: <none> cookie: 22
2025-10-09T21:43:39.383293Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 10
2025-10-09T21:43:39.380024Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.380026Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.383319Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (37-45)
2025-10-09T21:43:39.383336Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 41137 to budget
2025-10-09T21:43:39.380030Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 46  requestId: <none> cookie: 23
2025-10-09T21:43:39.383341Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 41137, ReadSizeServerDelta = 483151
2025-10-09T21:43:39.380035Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 47  requestId: <none> cookie: 24
2025-10-09T21:43:39.380040Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 53  requestId: <none> cookie: 25
2025-10-09T21:43:39.380045Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 54  requestId: <none> cookie: 26
2025-10-09T21:43:39.380049Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 57  requestId: <none> cookie: 27
2025-10-09T21:43:39.380054Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 58  requestId: <none> cookie: 28
2025-10-09T21:43:39.380058Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 59  requestId: <none> cookie: 29
2025-10-09T21:43:39.380063Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 60  requestId: <none> cookie: 30
2025-10-09T21:43:39.380068Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 61  requestId: <none> cookie: 31
2025-10-09T21:43:39.383387Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 11 offsets: 10 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.380073Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 64  requestId: <none> cookie: 32
2025-10-09T21:43:39.383395Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 11
2025-10-09T21:43:39.380077Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 65  requestId: <none> cookie: 33
2025-10-09T21:43:39.380082Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 67  requestId: <none> cookie: 34
2025-10-09T21:43:39.380087Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 71  requestId: <none> cookie: 35
2025-10-09T21:43:39.380091Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 72  requestId: <none> cookie: 36
2025-10-09T21:43:39.380095Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 73  requestId: <none> cookie: 37
2025-10-09T21:43:39.383422Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 12 offsets: 11 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.383426Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 12
2025-10-09T21:43:39.380101Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.380111Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 37 partNo 0 count 9 size 41521
2025-10-09T21:43:39.380117Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 94  requestId: <none> cookie: 49
2025-10-09T21:43:39.380121Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 95  requestId: <none> cookie: 50
2025-10-09T21:43:39.380124Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 96  requestId: <none> cookie: 51
2025-10-09T21:43:39.380133Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.380135Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.380142Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.380147Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.380180Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 98  requestId: <none> cookie: 52
2025-10-09T21:43:39.380207Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 6 Topic 'rt3.dc1--test-topic' partition 0 user user offset 18 count 22 size 69949 endOffset 37 max time lag 0ms effective offset 18
2025-10-09T21:43:39.380219Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 6 added 1 blobs, size 58291 count 19 last offset 18, current partition end offset: 37
2025-10-09T21:43:39.380222Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 6. Send blob request.
2025-10-09T21:43:39.380230Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 18 partno 0 count 19 parts_count 0 source 1 size 58291 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.380238Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 6. All 1 blobs are from cache.
2025-10-09T21:43:39.380246Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.380283Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 19 parts 0 suffix '63'
2025-10-09T21:43:39.380348Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 18 totakecount 19 count 19 size 58269 from pos 0 cbcount 19
2025-10-09T21:43:39.380390Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 18
2025-10-09T21:43:39.380437Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380445Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.380449Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380454Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380457Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380460Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380464Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380468Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380471Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380475Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380478Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380481Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380485Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.383547Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.380489Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380491Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380495Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380498Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380501Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.383564Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (37-37)
2025-10-09T21:43:39.383452Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 13 offsets: 12 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.380505Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.383608Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 13
2025-10-09T21:43:39.380508Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380511Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380514Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380519Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380523Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380528Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.380634Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.380754Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.380808Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.380842Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.383641Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 14 offsets: 13 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.380987Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 37 Result { Offset: 18 Data: "... 2063 bytes ..." SourceId: "\000src_id" SeqNo: 19 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 1991 PartitionKey: "" ExplicitHash: "" } Result { Offset: 19 Data: "... 2173 bytes ..." SourceId: "\000src_id" SeqNo: 20 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 2101 PartitionKey: "" ExplicitHash: "" } Result { Offset: 20 Data: "... 2283 bytes ..." SourceId: "\000src_id" SeqNo: 21 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 2211 PartitionKey: "" ExplicitHash: "" } Result { Offset: 21 Data: "... 2393 bytes ..." SourceId: "\000src_id" SeqNo: 22 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 2321 PartitionKey: "" ExplicitHash: "" } Result { Offset: 22 Data: "... 2503 bytes ..." SourceId: "\000src_id" SeqNo: 23 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 2431 PartitionKey: "" ExplicitHash: "" } Result { Offset: 23 Data: "... 2613 bytes ..." SourceId: "\000src_id" SeqNo: 24 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 2541 PartitionKey: "" ExplicitHash: "" } Result { Offset: 24 Data: "... 2723 bytes ..." SourceId: "\000src_id" SeqNo: 25 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 2651 PartitionKey: "" ExplicitHash: "" } Result { Offset: 25 Data: "... 2833 bytes ..." SourceId: "\000src_id" SeqNo: 26 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 2761 PartitionKey: "" ExplicitHash: "" } Result { Offset: 26 Data: "... 2943 bytes ..." SourceId: "\000src_id" SeqNo: 27 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 2871 PartitionKey: "" ExplicitHash: "" } Result { Offset: 27 Data: "... 3053 bytes ..." SourceId: "\000src_id" SeqNo: 28 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 2981 PartitionKey: "" ExplicitHash: "" } Result { Offset: 28 Data: "... 3163 bytes ..." SourceId: "\000src_id" SeqNo: 29 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 3091 PartitionKey: "" ExplicitHash: "" } Result { Offset: 29 Data: "... 3273 bytes ..." SourceId: "\000src_id" SeqNo: 30 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 3201 PartitionKey: "" ExplicitHash: "" } Result { Offset: 30 Data: "... 3383 bytes ..." SourceId: "\000src_id" SeqNo: 31 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 3311 PartitionKey: "" ExplicitHash: "" } Result { Offset: 31 Data: "... 3493 bytes ..." SourceId: "\000src_id" SeqNo: 32 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 3421 PartitionKey: "" ExplicitHash: "" } Result { Offset: 32 Data: "... 3603 bytes ..." SourceId: "\000src_id" SeqNo: 33 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 3531 PartitionKey: "" ExplicitHash: "" } Result { Offset: 33 Data: "... 3713 bytes ..." SourceId: "\000src_id" SeqNo: 34 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 3641 PartitionKey: "" ExplicitHash: "" } Result { Offset: 34 Data: "... 3823 bytes ..." SourceId: "\000src_id" SeqNo: 35 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 3751 PartitionKey: "" ExplicitHash: "" } Result { Offset: 35 Data: "... 3933 bytes ..." SourceId: "\000src_id" SeqNo: 36 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 3861 PartitionKey: "" ExplicitHash: "" } Result { Offset: 36 Data: "... 4043 bytes ..." SourceId: "\000src_id" SeqNo: 37 WriteTimestampMS: 1760046219377 CreateTimestampMS: 1760046219369 UncompressedSize: 3971 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 18446744073709551102 RealReadOffset: 36 WaitQuotaTimeMs: 0 EndOffset: 37 StartOffset: 0 } Cookie: 18 }
2025-10-09T21:43:39.381055Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 4 from offset 37
2025-10-09T21:43:39.381061Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 37 ReadOffset 37 ReadGuid 4de0a44c-4f920a36-82d41590-254423da has messages 1
2025-10-09T21:43:39.381083Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.383646Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 14
2025-10-09T21:43:39.381088Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.381097Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 48 partNo : 0 messageNo: 99 size 5253 offset: -1
2025-10-09T21:43:39.381110Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.381111Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.381114Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 49 partNo : 0 messageNo: 100 size 5363 offset: -1
2025-10-09T21:43:39.381117Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.381119Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.381122Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 50 partNo : 0 messageNo: 101 size 5473 offset: -1
2025-10-09T21:43:39.381124Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.381126Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.381129Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 51 partNo : 0 messageNo: 102 size 5583 offset: -1
2025-10-09T21:43:39.383708Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 15 offsets: 14 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.381132Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.383712Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 15
2025-10-09T21:43:39.381134Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.381137Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 52 partNo : 0 messageNo: 103 size 5693 offset: -1
2025-10-09T21:43:39.381140Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.381142Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.381146Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.381147Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.381150Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.383727Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 16 offsets: 15 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.383759Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 16
2025-10-09T21:43:39.383594Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (38-38)
2025-10-09T21:43:39.383780Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 2} (39-39)
2025-10-09T21:43:39.383781Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 17 offsets: 16 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.383784Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 17
2025-10-09T21:43:39.383785Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 3} (40-40)
2025-10-09T21:43:39.383795Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 4} (41-41)
2025-10-09T21:43:39.383800Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 5} (42-42)
2025-10-09T21:43:39.383801Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 18 offsets: 17 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.383804Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 18
2025-10-09T21:43:39.383805Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 6} (43-43)
2025-10-09T21:43:39.383810Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 7} (44-44)
2025-10-09T21:43:39.383819Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 19 offsets: 18 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 3 }
2025-10-09T21:43:39.383821Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 19
2025-10-09T21:43:39.383836Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 8} (45-45)
2025-10-09T21:43:39.383837Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 20 offsets: 19 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.383840Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 20
2025-10-09T21:43:39.383866Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 21 offsets: 20 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.383869Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 21
>>> event from dataHandler: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..4081 bytes.. Information: { Offset: 37 ProducerId: "src_id" SeqNo: 38 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.379000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..4191 bytes.. Information: { Offset: 38 ProducerId: "src_id" SeqNo: 39 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.379000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..4301 bytes.. Information: { Offset: 39 ProducerId: "src_id" SeqNo: 40 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.379000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..4411 bytes.. Information: { Offset: 40 ProducerId: "src_id" SeqNo: 41 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.379000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..4521 bytes.. Information: { Offset: 41 ProducerId: "src_id" SeqNo: 42 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.379000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..4631 bytes.. Information: { Offset: 42 ProducerId: "src_id" SeqNo: 43 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.379000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..4741 bytes.. Information: { Offset: 43 ProducerId: "src_id" SeqNo: 44 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.379000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..4851 bytes.. Information: { Offset: 44 ProducerId: "src_id" SeqNo: 45 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.379000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..4961 bytes.. Information: { Offset: 45 ProducerId: "src_id" SeqNo: 46 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.379000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.383944Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 9, size 40689 bytes
2025-10-09T21:43:39.383950Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.383885Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 22 offsets: 21 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.384100Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 22
2025-10-09T21:43:39.384143Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 173 to 173
2025-10-09T21:43:39.384162Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.384167Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 173
2025-10-09T21:43:39.381152Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.381159Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.381165Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.381256Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.381259Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.381261Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.381263Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.381265Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.381267Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.381269Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.381272Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.381284Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 104  requestId: <none> cookie: 53
2025-10-09T21:43:39.381289Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 105  requestId: <none> cookie: 54
2025-10-09T21:43:39.381292Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 106  requestId: <none> cookie: 55
2025-10-09T21:43:39.381305Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# 4de0a44c-4f920a36-82d41590-254423da, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 57437
2025-10-09T21:43:39.381313Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# 4de0a44c-4f920a36-82d41590-254423da
2025-10-09T21:43:39.381359Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 0
2025-10-09T21:43:39.381386Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.381392Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.381395Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.381420Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.381422Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.381429Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 53 partNo : 0 messageNo: 107 size 5803 offset: -1
2025-10-09T21:43:39.381433Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.381435Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.381438Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 54 partNo : 0 messageNo: 108 size 5913 offset: -1
2025-10-09T21:43:39.381441Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.381442Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.381445Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 55 partNo : 0 messageNo: 109 size 6023 offset: -1
2025-10-09T21:43:39.381449Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.381452Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.381454Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.381457Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.381459Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.381461Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.384322Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 174 to 174
2025-10-09T21:43:39.381772Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 37 count 9 size 41521 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.384338Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.384343Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 174
2025-10-09T21:43:39.381803Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.381813Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 9 parts 0 suffix '63' size 41521
2025-10-09T21:43:39.381817Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 41400 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.381824Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.381830Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 38, partNo: 0, Offset: 37 is stored on disk
2025-10-09T21:43:39.381834Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.381838Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 39, partNo: 0, Offset: 38 is stored on disk
2025-10-09T21:43:39.381840Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.381843Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 40, partNo: 0, Offset: 39 is stored on disk
2025-10-09T21:43:39.381846Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.381850Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 41, partNo: 0, Offset: 40 is stored on disk
2025-10-09T21:43:39.381852Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.381855Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 42, partNo: 0, Offset: 41 is stored on disk
2025-10-09T21:43:39.381858Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.381871Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 43, partNo: 0, Offset: 42 is stored on disk
2025-10-09T21:43:39.381874Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.381878Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 44, partNo: 0, Offset: 43 is stored on disk
2025-10-09T21:43:39.381881Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.381885Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 45, partNo: 0, Offset: 44 is stored on disk
2025-10-09T21:43:39.381887Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.384412Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 175 to 175
2025-10-09T21:43:39.381890Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 46, partNo: 0, Offset: 45 is stored on disk
2025-10-09T21:43:39.384442Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.381915Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.384448Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 175
2025-10-09T21:43:39.381933Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.381934Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (9)
2025-10-09T21:43:39.381948Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 47 partNo 0
2025-10-09T21:43:39.381984Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 47 partNo 0 FormedBlobsCount 0 NewHead: Offset 46 PartNo 0 PackedSize 5217 count 1 nextOffset 47 batches 1
2025-10-09T21:43:39.381992Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 48 partNo 0
2025-10-09T21:43:39.381998Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 48 partNo 0 FormedBlobsCount 0 NewHead: Offset 46 PartNo 0 PackedSize 10512 count 2 nextOffset 48 batches 1
2025-10-09T21:43:39.382002Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 49 partNo 0
2025-10-09T21:43:39.382008Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 49 partNo 0 FormedBlobsCount 0 NewHead: Offset 46 PartNo 0 PackedSize 15917 count 3 nextOffset 49 batches 1
2025-10-09T21:43:39.382011Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 50 partNo 0
2025-10-09T21:43:39.382015Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 50 partNo 0 FormedBlobsCount 0 NewHead: Offset 46 PartNo 0 PackedSize 21432 count 4 nextOffset 50 batches 1
2025-10-09T21:43:39.382018Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 51 partNo 0
2025-10-09T21:43:39.382022Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 51 partNo 0 FormedBlobsCount 0 NewHead: Offset 46 PartNo 0 PackedSize 27057 count 5 nextOffset 51 batches 1
2025-10-09T21:43:39.382025Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 52 partNo 0
2025-10-09T21:43:39.382045Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 52 partNo 0 FormedBlobsCount 0 NewHead: Offset 46 PartNo 0 PackedSize 32792 count 6 nextOffset 52 batches 1
2025-10-09T21:43:39.382048Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 53 partNo 0
2025-10-09T21:43:39.382108Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 53 partNo 0 FormedBlobsCount 0 NewHead: Offset 46 PartNo 0 PackedSize 38637 count 7 nextOffset 53 batches 1
2025-10-09T21:43:39.382113Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 54 partNo 0
2025-10-09T21:43:39.382119Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 54 partNo 0 FormedBlobsCount 0 NewHead: Offset 46 PartNo 0 PackedSize 44592 count 8 nextOffset 54 batches 1
2025-10-09T21:43:39.382123Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 55 partNo 0
2025-10-09T21:43:39.382127Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 55 partNo 0 FormedBlobsCount 0 NewHead: Offset 46 PartNo 0 PackedSize 50657 count 9 nextOffset 55 batches 1
2025-10-09T21:43:39.382130Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.382202Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 46,9 HeadOffset 46 endOffset 46 curOffset 55 d0000000000_00000000000000000046_00000_0000000009_00000? size 50431 WTime 1760046219381
2025-10-09T21:43:39.382223Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.382225Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.382239Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 76  requestId: <none> cookie: 38
2025-10-09T21:43:39.382248Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 77  requestId: <none> cookie: 39
2025-10-09T21:43:39.382253Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 80  requestId: <none> cookie: 40
2025-10-09T21:43:39.382257Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 81  requestId: <none> cookie: 41
2025-10-09T21:43:39.382261Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.382262Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 83  requestId: <none> cookie: 42
2025-10-09T21:43:39.382267Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 88  requestId: <none> cookie: 43
2025-10-09T21:43:39.382267Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.384582Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 176 to 176
2025-10-09T21:43:39.382271Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 89  requestId: <none> cookie: 44
2025-10-09T21:43:39.382274Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.384590Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.384598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 176
2025-10-09T21:43:39.384679Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 177 to 177
2025-10-09T21:43:39.384686Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.384689Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 177
2025-10-09T21:43:39.382276Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 90  requestId: <none> cookie: 45
2025-10-09T21:43:39.382278Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.382281Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 91  requestId: <none> cookie: 46
2025-10-09T21:43:39.382282Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.382286Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.382287Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.382292Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.382297Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 46 partNo 0 count 9 size 50431
2025-10-09T21:43:39.382300Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.382303Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.382307Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.382311Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.382371Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 37 EndOffset 37 newEndOffset 46 commitOffset 0 clientCommitOffset 0 cookie 4 readingFinished 0 firstRead 0
2025-10-09T21:43:39.382374Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 37 endOffset 46
2025-10-09T21:43:39.382381Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 37, endOffset# 46, WTime# 1760046219377, sizeLag# 41521
2025-10-09T21:43:39.384772Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 178 to 178
2025-10-09T21:43:39.382384Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:43:39.382392Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# b7b0a296-df43068b-f2d24659-624c3c0a, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 10, size# 49825, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.382392Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.382402Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 10 maxSize 49825 maxTimeLagMs 0 readTimestampMs 0 readOffset 37 EndOffset 46 ClientCommitOffset 0 committedOffset 0 Guid b7b0a296-df43068b-f2d24659-624c3c0a
2025-10-09T21:43:39.382453Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.384790Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.384796Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 178
2025-10-09T21:43:39.382487Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.382502Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.382506Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.382513Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.382515Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.384815Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 179 to 179
2025-10-09T21:43:39.382523Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.384823Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.382525Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.384826Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 179
2025-10-09T21:43:39.382536Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.382538Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.382548Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 110  requestId: <none> cookie: 56
2025-10-09T21:43:39.382555Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 111  requestId: <none> cookie: 57
2025-10-09T21:43:39.382562Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 112  requestId: <none> cookie: 58
2025-10-09T21:43:39.382570Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 7 Topic 'rt3.dc1--test-topic' partition 0 user user offset 37 count 10 size 49825 endOffset 46 max time lag 0ms effective offset 37
2025-10-09T21:43:39.382587Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 7 added 1 blobs, size 41521 count 9 last offset 37, current partition end offset: 46
2025-10-09T21:43:39.384849Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 180 to 180
2025-10-09T21:43:39.382590Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 7. Send blob request.
2025-10-09T21:43:39.382615Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 37 partno 0 count 9 parts_count 0 source 1 size 41521 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.382624Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 7. All 1 blobs are from cache.
2025-10-09T21:43:39.382633Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.384861Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.382703Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 37 totakecount 9 count 9 size 41499 from pos 0 cbcount 9
2025-10-09T21:43:39.384864Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 180
2025-10-09T21:43:39.382724Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 9 parts 0 suffix '63'
2025-10-09T21:43:39.382728Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 37
2025-10-09T21:43:39.382758Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.382765Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.382769Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.382773Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.382838Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.382850Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.382854Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.382862Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 46 Result { Offset: 37 Data: "... 4153 bytes ..." SourceId: "\000src_id" SeqNo: 38 WriteTimestampMS: 1760046219379 CreateTimestampMS: 1760046219369 UncompressedSize: 4081 PartitionKey: "" ExplicitHash: "" } Result { Offset: 38 Data: "... 4263 bytes ..." SourceId: "\000src_id" SeqNo: 39 WriteTimestampMS: 1760046219379 CreateTimestampMS: 1760046219369 UncompressedSize: 4191 PartitionKey: "" ExplicitHash: "" } Result { Offset: 39 Data: "... 4373 bytes ..." SourceId: "\000src_id" SeqNo: 40 WriteTimestampMS: 1760046219379 CreateTimestampMS: 1760046219369 UncompressedSize: 4301 PartitionKey: "" ExplicitHash: "" } Result { Offset: 40 Data: "... 4483 bytes ..." SourceId: "\000src_id" SeqNo: 41 WriteTimestampMS: 1760046219379 CreateTimestampMS: 1760046219369 UncompressedSize: 4411 PartitionKey: "" ExplicitHash: "" } Result { Offset: 41 Data: "... 4593 bytes ..." SourceId: "\000src_id" SeqNo: 42 WriteTimestampMS: 1760046219379 CreateTimestampMS: 1760046219369 UncompressedSize: 4521 PartitionKey: "" ExplicitHash: "" } Result { Offset: 42 Data: "... 4703 bytes ..." SourceId: "\000src_id" SeqNo: 43 WriteTimestampMS: 1760046219379 CreateTimestampMS: 1760046219369 UncompressedSize: 4631 PartitionKey: "" ExplicitHash: "" } Result { Offset: 43 Data: "... 4813 bytes ..." SourceId: "\000src_id" SeqNo: 44 WriteTimestampMS: 1760046219379 CreateTimestampMS: 1760046219369 UncompressedSize: 4741 PartitionKey: "" ExplicitHash: "" } Result { Offset: 44 Data: "... 4923 bytes ..." SourceId: "\000src_id" SeqNo: 45 WriteTimestampMS: 1760046219379 CreateTimestampMS: 1760046219369 UncompressedSize: 4851 PartitionKey: "" ExplicitHash: "" } Result { Offset: 45 Data: "... 5033 bytes ..." SourceId: "\000src_id" SeqNo: 46 WriteTimestampMS: 1760046219379 CreateTimestampMS: 1760046219369 UncompressedSize: 4961 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 18446744073709551422 RealReadOffset: 45 WaitQuotaTimeMs: 0 EndOffset: 46 StartOffset: 0 } Cookie: 37 }
2025-10-09T21:43:39.382862Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 56 partNo : 0 messageNo: 113 size 6133 offset: -1
2025-10-09T21:43:39.382867Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.382868Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.382872Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 57 partNo : 0 messageNo: 114 size 6243 offset: -1
2025-10-09T21:43:39.382886Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.382888Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.382893Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 58 partNo : 0 messageNo: 115 size 6353 offset: -1
2025-10-09T21:43:39.382897Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.382904Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.382906Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.382908Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 5 from offset 46
2025-10-09T21:43:39.382909Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.382910Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.382912Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.382914Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 46 ReadOffset 46 ReadGuid b7b0a296-df43068b-f2d24659-624c3c0a has messages 1
2025-10-09T21:43:39.382917Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.382933Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.382940Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# b7b0a296-df43068b-f2d24659-624c3c0a, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 41137
2025-10-09T21:43:39.382946Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# b7b0a296-df43068b-f2d24659-624c3c0a
2025-10-09T21:43:39.382947Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.382949Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.382953Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.382954Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.382966Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 116  requestId: <none> cookie: 59
2025-10-09T21:43:39.382970Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 117  requestId: <none> cookie: 60
2025-10-09T21:43:39.382974Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.382979Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.382995Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.383008Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.383021Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 0
2025-10-09T21:43:39.383048Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 57437 } }
2025-10-09T21:43:39.383057Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# 2e74b6d1-204a73f0-24798743-e800b0f7
2025-10-09T21:43:39.383201Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.383245Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.383481Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.383540Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.383552Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.383557Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.383567Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 59 partNo : 0 messageNo: 118 size 6463 offset: -1
2025-10-09T21:43:39.383573Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.383576Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.383581Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 60 partNo : 0 messageNo: 119 size 6573 offset: -1
2025-10-09T21:43:39.383584Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.383586Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.383590Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.383592Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.383602Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.383603Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.383608Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.383614Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.383616Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.383619Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.383727Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 120  requestId: <none> cookie: 61
2025-10-09T21:43:39.383733Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 121  requestId: <none> cookie: 62
2025-10-09T21:43:39.383736Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 122  requestId: <none> cookie: 63
2025-10-09T21:43:39.383746Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.383753Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.383756Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.383880Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.383883Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.383891Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 61 partNo : 0 messageNo: 123 size 6683 offset: -1
2025-10-09T21:43:39.383896Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.383897Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.383901Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 62 partNo : 0 messageNo: 124 size 6793 offset: -1
2025-10-09T21:43:39.383905Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.383906Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.383910Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 63 partNo : 0 messageNo: 125 size 6903 offset: -1
2025-10-09T21:43:39.383915Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.383919Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.383921Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.383924Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.383926Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.383928Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.383935Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.383990Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.384002Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.384027Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.384044Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 41137 } }
2025-10-09T21:43:39.384090Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# 2f58c48e-b542f787-ff5a8ab-7dcaed22
2025-10-09T21:43:39.384325Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.384372Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.384421Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.384424Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.384431Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.384432Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.384435Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.384437Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.384445Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 46 count 9 size 50431 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.384476Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.384490Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 9 parts 0 suffix '63' size 50431
2025-10-09T21:43:39.384491Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 50310 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.384497Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.384504Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 47, partNo: 0, Offset: 46 is stored on disk
2025-10-09T21:43:39.384507Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.384511Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 48, partNo: 0, Offset: 47 is stored on disk
2025-10-09T21:43:39.384514Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.384518Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 49, partNo: 0, Offset: 48 is stored on disk
2025-10-09T21:43:39.384520Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.384524Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 50, partNo: 0, Offset: 49 is stored on disk
2025-10-09T21:43:39.385189Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 181 to 181
2025-10-09T21:43:39.384526Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.384529Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 51, partNo: 0, Offset: 50 is stored on disk
2025-10-09T21:43:39.385209Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.385218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 181
2025-10-09T21:43:39.384531Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.384535Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 52, partNo: 0, Offset: 51 is stored on disk
2025-10-09T21:43:39.384537Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.384541Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 53, partNo: 0, Offset: 52 is stored on disk
2025-10-09T21:43:39.384543Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.384548Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 54, partNo: 0, Offset: 53 is stored on disk
2025-10-09T21:43:39.384550Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.384555Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 55, partNo: 0, Offset: 54 is stored on disk
2025-10-09T21:43:39.384576Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.384584Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.384586Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (8)
2025-10-09T21:43:39.384600Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 56 partNo 0
2025-10-09T21:43:39.384625Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 56 partNo 0 FormedBlobsCount 0 NewHead: Offset 55 PartNo 0 PackedSize 6207 count 1 nextOffset 56 batches 1
2025-10-09T21:43:39.384631Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 57 partNo 0
2025-10-09T21:43:39.384636Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 57 partNo 0 FormedBlobsCount 0 NewHead: Offset 55 PartNo 0 PackedSize 12492 count 2 nextOffset 57 batches 1
2025-10-09T21:43:39.384639Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 58 partNo 0
2025-10-09T21:43:39.384643Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 58 partNo 0 FormedBlobsCount 0 NewHead: Offset 55 PartNo 0 PackedSize 18887 count 3 nextOffset 58 batches 1
2025-10-09T21:43:39.384646Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 59 partNo 0
2025-10-09T21:43:39.384650Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 59 partNo 0 FormedBlobsCount 0 NewHead: Offset 55 PartNo 0 PackedSize 25392 count 4 nextOffset 59 batches 1
2025-10-09T21:43:39.384652Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 60 partNo 0
2025-10-09T21:43:39.384656Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 60 partNo 0 FormedBlobsCount 0 NewHead: Offset 55 PartNo 0 PackedSize 32007 count 5 nextOffset 60 batches 1
2025-10-09T21:43:39.385292Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 182 to 182
2025-10-09T21:43:39.384658Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 61 partNo 0
2025-10-09T21:43:39.384662Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 61 partNo 0 FormedBlobsCount 0 NewHead: Offset 55 PartNo 0 PackedSize 38732 count 6 nextOffset 61 batches 1
2025-10-09T21:43:39.385300Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.384665Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 62 partNo 0
2025-10-09T21:43:39.385303Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 182
2025-10-09T21:43:39.384669Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 62 partNo 0 FormedBlobsCount 0 NewHead: Offset 55 PartNo 0 PackedSize 45567 count 7 nextOffset 62 batches 1
2025-10-09T21:43:39.384672Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 63 partNo 0
2025-10-09T21:43:39.384675Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 63 partNo 0 FormedBlobsCount 0 NewHead: Offset 55 PartNo 0 PackedSize 52512 count 8 nextOffset 63 batches 1
2025-10-09T21:43:39.384678Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.384745Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 55,8 HeadOffset 55 endOffset 55 curOffset 63 d0000000000_00000000000000000055_00000_0000000008_00000? size 52318 WTime 1760046219384
2025-10-09T21:43:39.385325Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 183 to 183
2025-10-09T21:43:39.384851Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.384859Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 126  requestId: <none> cookie: 64
2025-10-09T21:43:39.385332Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.384866Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 127  requestId: <none> cookie: 65
2025-10-09T21:43:39.385334Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 183
2025-10-09T21:43:39.384869Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 128  requestId: <none> cookie: 66
2025-10-09T21:43:39.384875Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 97  requestId: <none> cookie: 47
2025-10-09T21:43:39.384881Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 99  requestId: <none> cookie: 48
2025-10-09T21:43:39.384886Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 100  requestId: <none> cookie: 49
2025-10-09T21:43:39.384890Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 101  requestId: <none> cookie: 50
2025-10-09T21:43:39.384895Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 102  requestId: <none> cookie: 51
2025-10-09T21:43:39.384899Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 103  requestId: <none> cookie: 52
2025-10-09T21:43:39.385359Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 184 to 184
2025-10-09T21:43:39.384904Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 107  requestId: <none> cookie: 53
2025-10-09T21:43:39.384908Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 108  requestId: <none> cookie: 54
2025-10-09T21:43:39.384912Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 109  requestId: <none> cookie: 55
2025-10-09T21:43:39.384917Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.384926Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 55 partNo 0 count 8 size 52318
2025-10-09T21:43:39.384984Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 46 EndOffset 46 newEndOffset 55 commitOffset 0 clientCommitOffset 0 cookie 5 readingFinished 0 firstRead 0
2025-10-09T21:43:39.384987Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 46 endOffset 55
2025-10-09T21:43:39.384994Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.385001Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.385006Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.385010Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.385014Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.385017Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.385020Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.385023Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.385025Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.385028Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.385031Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.385034Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.385167Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 46, endOffset# 55, WTime# 1760046219379, sizeLag# 50431
2025-10-09T21:43:39.385170Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:43:39.385178Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# 7adc6b34-8d4b26ef-c350e30f-8827ad06, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 10, size# 60517, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.385200Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 10 maxSize 60517 maxTimeLagMs 0 readTimestampMs 0 readOffset 46 EndOffset 55 ClientCommitOffset 0 committedOffset 0 Guid 7adc6b34-8d4b26ef-c350e30f-8827ad06
2025-10-09T21:43:39.385370Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.385449Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 184
2025-10-09T21:43:39.385597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 185 to 185
2025-10-09T21:43:39.385609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.385612Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 185
2025-10-09T21:43:39.385637Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 186 to 186
2025-10-09T21:43:39.385643Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.385645Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 186
2025-10-09T21:43:39.385662Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 187 to 187
2025-10-09T21:43:39.385687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.385690Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 187
2025-10-09T21:43:39.385726Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 188 to 188
2025-10-09T21:43:39.385747Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.385750Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 188
2025-10-09T21:43:39.385770Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 189 to 189
2025-10-09T21:43:39.385790Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.385793Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 189
2025-10-09T21:43:39.385821Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 190 to 190
2025-10-09T21:43:39.385865Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.385868Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 190
2025-10-09T21:43:39.385492Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.385549Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.385563Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.385566Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.385576Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 64 partNo : 0 messageNo: 129 size 7013 offset: -1
2025-10-09T21:43:39.385581Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.385583Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.385936Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 191 to 191
2025-10-09T21:43:39.385987Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.385992Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 191
2025-10-09T21:43:39.385588Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 65 partNo : 0 messageNo: 130 size 7123 offset: -1
2025-10-09T21:43:39.385591Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.385593Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.385597Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 66 partNo : 0 messageNo: 131 size 7233 offset: -1
2025-10-09T21:43:39.385607Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.385609Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.385616Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.385623Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.385625Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.385627Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.385629Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.385632Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.385646Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 8 Topic 'rt3.dc1--test-topic' partition 0 user user offset 46 count 10 size 60517 endOffset 55 max time lag 0ms effective offset 46
2025-10-09T21:43:39.385660Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 8 added 1 blobs, size 50431 count 9 last offset 46, current partition end offset: 55
2025-10-09T21:43:39.385664Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 8. Send blob request.
2025-10-09T21:43:39.385672Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 46 partno 0 count 9 parts_count 0 source 1 size 50431 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.385675Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 8. All 1 blobs are from cache.
2025-10-09T21:43:39.385683Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.385740Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 46 totakecount 9 count 9 size 50409 from pos 0 cbcount 9
2025-10-09T21:43:39.385765Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 46
2025-10-09T21:43:39.385792Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.385831Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.385915Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 55 Result { Offset: 46 Data: "... 5143 bytes ..." SourceId: "\000src_id" SeqNo: 47 WriteTimestampMS: 1760046219381 CreateTimestampMS: 1760046219369 UncompressedSize: 5071 PartitionKey: "" ExplicitHash: "" } Result { Offset: 47 Data: "... 5253 bytes ..." SourceId: "\000src_id" SeqNo: 48 WriteTimestampMS: 1760046219381 CreateTimestampMS: 1760046219369 UncompressedSize: 5181 PartitionKey: "" ExplicitHash: "" } Result { Offset: 48 Data: "... 5363 bytes ..." SourceId: "\000src_id" SeqNo: 49 WriteTimestampMS: 1760046219381 CreateTimestampMS: 1760046219369 UncompressedSize: 5291 PartitionKey: "" ExplicitHash: "" } Result { Offset: 49 Data: "... 5473 bytes ..." SourceId: "\000src_id" SeqNo: 50 WriteTimestampMS: 1760046219381 CreateTimestampMS: 1760046219369 UncompressedSize: 5401 PartitionKey: "" ExplicitHash: "" } Result { Offset: 50 Data: "... 5583 bytes ..." SourceId: "\000src_id" SeqNo: 51 WriteTimestampMS: 1760046219381 CreateTimestampMS: 1760046219369 UncompressedSize: 5511 PartitionKey: "" ExplicitHash: "" } Result { Offset: 51 Data: "... 5693 bytes ..." SourceId: "\000src_id" SeqNo: 52 WriteTimestampMS: 1760046219381 CreateTimestampMS: 1760046219369 UncompressedSize: 5621 PartitionKey: "" ExplicitHash: "" } Result { Offset: 52 Data: "... 5803 bytes ..." SourceId: "\000src_id" SeqNo: 53 WriteTimestampMS: 1760046219381 CreateTimestampMS: 1760046219369 UncompressedSize: 5731 PartitionKey: "" ExplicitHash: "" } Result { Offset: 53 Data: "... 5913 bytes ..." SourceId: "\000src_id" SeqNo: 54 WriteTimestampMS: 1760046219381 CreateTimestampMS: 1760046219369 UncompressedSize: 5841 PartitionKey: "" ExplicitHash: "" } Result { Offset: 54 Data: "... 6023 bytes ..." SourceId: "\000src_id" SeqNo: 55 WriteTimestampMS: 1760046219381 CreateTimestampMS: 1760046219369 UncompressedSize: 5951 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 18446744073709551422 RealReadOffset: 54 WaitQuotaTimeMs: 0 EndOffset: 55 StartOffset: 0 } Cookie: 46 }
2025-10-09T21:43:39.385972Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 6 from offset 55
2025-10-09T21:43:39.385978Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 55 ReadOffset 55 ReadGuid 7adc6b34-8d4b26ef-c350e30f-8827ad06 has messages 1
2025-10-09T21:43:39.385995Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# 7adc6b34-8d4b26ef-c350e30f-8827ad06, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 50047
2025-10-09T21:43:39.386001Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# 7adc6b34-8d4b26ef-c350e30f-8827ad06
2025-10-09T21:43:39.386140Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 9 parts 0 suffix '63'
2025-10-09T21:43:39.386303Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 0
2025-10-09T21:43:39.386352Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.386360Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.386366Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.386368Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.386377Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 192 to 192
2025-10-09T21:43:39.386388Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.386391Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 192
2025-10-09T21:43:39.386413Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 193 to 193
2025-10-09T21:43:39.386436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.386439Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 193
2025-10-09T21:43:39.386466Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 194 to 194
2025-10-09T21:43:39.386472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.386475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 194
2025-10-09T21:43:39.386497Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 195 to 195
2025-10-09T21:43:39.386503Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.386505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 195
2025-10-09T21:43:39.386485Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 132  requestId: <none> cookie: 67
2025-10-09T21:43:39.386499Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 133  requestId: <none> cookie: 68
2025-10-09T21:43:39.386508Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.386516Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.386533Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 196 to 196
2025-10-09T21:43:39.386523Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.386542Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.386545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 196
2025-10-09T21:43:39.386568Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 197 to 197
2025-10-09T21:43:39.386574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.386578Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 197
2025-10-09T21:43:39.386602Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 198 to 198
2025-10-09T21:43:39.386607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.386609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 198
2025-10-09T21:43:39.386584Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.386631Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 199 to 199
2025-10-09T21:43:39.386595Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.386624Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.386639Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.386644Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.386653Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.386656Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 199
2025-10-09T21:43:39.386659Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 50047, now ReadSizeBudget = 0, ReadSizeServerDelta = 474241
2025-10-09T21:43:39.386689Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 0, ReadSizeServerDelta = 474241
2025-10-09T21:43:39.386653Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 67 partNo : 0 messageNo: 134 size 7343 offset: -1
2025-10-09T21:43:39.386702Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 200 to 200
2025-10-09T21:43:39.386663Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.386664Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.386669Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 68 partNo : 0 messageNo: 135 size 7453 offset: -1
2025-10-09T21:43:39.386672Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.386673Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.386677Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.386680Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.386729Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (46-54)
2025-10-09T21:43:39.386741Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 50047 to budget
2025-10-09T21:43:39.386683Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.386689Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.386691Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.386746Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 50047, ReadSizeServerDelta = 474241
2025-10-09T21:43:39.386694Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.386699Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 136  requestId: <none> cookie: 69
2025-10-09T21:43:39.386703Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 137  requestId: <none> cookie: 70
2025-10-09T21:43:39.386707Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.386710Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.386715Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.386754Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.386761Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.386787Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.386720Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.386800Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (46-46)
2025-10-09T21:43:39.386810Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 200
2025-10-09T21:43:39.386819Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (47-47)
2025-10-09T21:43:39.386826Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 2} (48-48)
2025-10-09T21:43:39.386830Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 3} (49-49)
2025-10-09T21:43:39.386832Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 201 to 201
2025-10-09T21:43:39.386850Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 4} (50-50)
2025-10-09T21:43:39.386861Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.386868Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 201
2025-10-09T21:43:39.386862Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 5} (51-51)
2025-10-09T21:43:39.386880Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 6} (52-52)
2025-10-09T21:43:39.386893Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 202 to 202
2025-10-09T21:43:39.386902Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 7} (53-53)
2025-10-09T21:43:39.386914Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.386916Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 202
2025-10-09T21:43:39.386934Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 8} (54-54)
2025-10-09T21:43:39.386948Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 203 to 203
2025-10-09T21:43:39.386974Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.386977Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 203
>>> event from dataHandler: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..5071 bytes.. Information: { Offset: 46 ProducerId: "src_id" SeqNo: 47 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.381000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..5181 bytes.. Information: { Offset: 47 ProducerId: "src_id" SeqNo: 48 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.381000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..5291 bytes.. Information: { Offset: 48 ProducerId: "src_id" SeqNo: 49 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.381000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..5401 bytes.. Information: { Offset: 49 ProducerId: "src_id" SeqNo: 50 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.381000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..5511 bytes.. Information: { Offset: 50 ProducerId: "src_id" SeqNo: 51 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.381000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..5621 bytes.. Information: { Offset: 51 ProducerId: "src_id" SeqNo: 52 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.381000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..5731 bytes.. Information: { Offset: 52 ProducerId: "src_id" SeqNo: 53 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.381000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..5841 bytes.. Information: { Offset: 53 ProducerId: "src_id" SeqNo: 54 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.381000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..5951 bytes.. Information: { Offset: 54 ProducerId: "src_id" SeqNo: 55 CreateTime: 2025-10-09T21:43:39.369000Z WriteTime: 2025-10-09T21:43:39.381000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.387147Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 9, size 49599 bytes
2025-10-09T21:43:39.387154Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.387215Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 204 to 204
2025-10-09T21:43:39.387262Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.387267Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 204
2025-10-09T21:43:39.387292Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 205 to 205
2025-10-09T21:43:39.387308Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.387311Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 205
2025-10-09T21:43:39.387484Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 206 to 206
2025-10-09T21:43:39.387494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.387498Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 206
2025-10-09T21:43:39.387519Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 207 to 207
2025-10-09T21:43:39.387526Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.387528Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 207
2025-10-09T21:43:39.387562Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 208 to 208
2025-10-09T21:43:39.387570Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.387572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 208
2025-10-09T21:43:39.387592Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 209 to 209
2025-10-09T21:43:39.387598Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.387601Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 209
2025-10-09T21:43:39.387629Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 210 to 210
2025-10-09T21:43:39.387636Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.387639Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 210
2025-10-09T21:43:39.387659Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 211 to 211
2025-10-09T21:43:39.387683Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.387687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 211
2025-10-09T21:43:39.387715Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 212 to 212
2025-10-09T21:43:39.387736Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.387739Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 212
2025-10-09T21:43:39.388075Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 213 to 213
2025-10-09T21:43:39.388087Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388090Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 213
2025-10-09T21:43:39.388164Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 214 to 214
2025-10-09T21:43:39.388171Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388173Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 214
2025-10-09T21:43:39.388206Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 23 offsets: 22 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.388221Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 23
2025-10-09T21:43:39.388244Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 215 to 215
2025-10-09T21:43:39.388251Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388253Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 215
2025-10-09T21:43:39.388510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 216 to 216
2025-10-09T21:43:39.388522Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388525Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 216
2025-10-09T21:43:39.388548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 217 to 217
2025-10-09T21:43:39.388556Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388559Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 217
2025-10-09T21:43:39.388586Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 218 to 218
2025-10-09T21:43:39.388591Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388595Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 218
2025-10-09T21:43:39.388615Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 219 to 219
2025-10-09T21:43:39.388620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388622Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 219
2025-10-09T21:43:39.388647Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 220 to 220
2025-10-09T21:43:39.388672Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388675Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 220
2025-10-09T21:43:39.388696Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 221 to 221
2025-10-09T21:43:39.388718Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388721Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 221
2025-10-09T21:43:39.388760Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 222 to 222
2025-10-09T21:43:39.388781Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388784Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 222
2025-10-09T21:43:39.388804Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 223 to 223
2025-10-09T21:43:39.388829Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388832Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 223
2025-10-09T21:43:39.388862Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 224 to 224
2025-10-09T21:43:39.388885Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388888Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 224
2025-10-09T21:43:39.388913Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 225 to 225
2025-10-09T21:43:39.388933Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388936Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 225
2025-10-09T21:43:39.388965Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 226 to 226
2025-10-09T21:43:39.388989Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388992Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 226
2025-10-09T21:43:39.389046Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 227 to 227
2025-10-09T21:43:39.389069Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.389072Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 227
2025-10-09T21:43:39.389610Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 228 to 228
2025-10-09T21:43:39.389620Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.389624Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 228
2025-10-09T21:43:39.389705Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 24 offsets: 23 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.389718Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 24
2025-10-09T21:43:39.389747Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 229 to 229
2025-10-09T21:43:39.389754Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.389757Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 229
2025-10-09T21:43:39.386785Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.386799Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.386822Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.386830Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.386833Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.386837Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 69 partNo : 0 messageNo: 138 size 7563 offset: -1
2025-10-09T21:43:39.386841Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.386843Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.386846Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 70 partNo : 0 messageNo: 139 size 7673 offset: -1
2025-10-09T21:43:39.386850Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389833Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 230 to 230
2025-10-09T21:43:39.386852Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.386855Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.389839Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.386858Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.389842Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 230
2025-10-09T21:43:39.386860Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.386863Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.386867Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 140  requestId: <none> cookie: 71
2025-10-09T21:43:39.386872Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.386875Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.386904Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.387011Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.387083Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.387118Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.387142Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.389869Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 231 to 231
2025-10-09T21:43:39.387164Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 50047 } }
2025-10-09T21:43:39.387413Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# b4b5e5f5-b5879998-7036fa2c-cf37bc6a
2025-10-09T21:43:39.389875Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.387429Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.389879Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 231
2025-10-09T21:43:39.387589Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.387704Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.387793Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.387839Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.387925Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.388088Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.388178Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.388237Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.388280Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.388315Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.389919Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 232 to 232
2025-10-09T21:43:39.388344Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.388349Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.389926Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388359Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389930Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 232
2025-10-09T21:43:39.388362Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.388366Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.388368Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.388378Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 71 partNo : 0 messageNo: 143 size 7783 offset: -1
2025-10-09T21:43:39.388382Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.388384Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.388388Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.388390Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.388394Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.388395Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.388406Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.388414Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.388422Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 141  requestId: <none> cookie: 72
2025-10-09T21:43:39.388429Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 142  requestId: <none> cookie: 73
2025-10-09T21:43:39.388432Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 144  requestId: <none> cookie: 74
2025-10-09T21:43:39.388434Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 145  requestId: <none> cookie: 75
2025-10-09T21:43:39.388438Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 146  requestId: <none> cookie: 76
2025-10-09T21:43:39.388442Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.388450Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.388458Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.388461Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.388465Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.388469Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.388535Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.388543Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.388548Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.388553Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.388558Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.388560Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 55 count 8 size 52318 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.388570Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.388583Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.388598Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 52200 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.388605Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.388608Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.388614Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 56, partNo: 0, Offset: 55 is stored on disk
2025-10-09T21:43:39.388616Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 8 parts 0 suffix '63' size 52318
2025-10-09T21:43:39.388618Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.388622Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 57, partNo: 0, Offset: 56 is stored on disk
2025-10-09T21:43:39.390063Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 51981, now ReadSizeBudget = 0, ReadSizeServerDelta = 472307
2025-10-09T21:43:39.388624Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.388628Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 58, partNo: 0, Offset: 57 is stored on disk
2025-10-09T21:43:39.388630Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.390101Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 0, ReadSizeServerDelta = 472307
2025-10-09T21:43:39.389966Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 233 to 233
2025-10-09T21:43:39.388634Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 59, partNo: 0, Offset: 58 is stored on disk
2025-10-09T21:43:39.390125Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.390130Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 233
2025-10-09T21:43:39.388635Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.388639Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 60, partNo: 0, Offset: 59 is stored on disk
2025-10-09T21:43:39.388641Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.388646Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 61, partNo: 0, Offset: 60 is stored on disk
2025-10-09T21:43:39.388649Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.388652Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 62, partNo: 0, Offset: 61 is stored on disk
2025-10-09T21:43:39.388655Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.388660Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 63, partNo: 0, Offset: 62 is stored on disk
2025-10-09T21:43:39.388689Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.388693Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.388704Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.388706Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (8)
2025-10-09T21:43:39.388723Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 64 partNo 0
2025-10-09T21:43:39.390176Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 234 to 234
2025-10-09T21:43:39.390208Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.390211Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 234
2025-10-09T21:43:39.390215Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (55-62)
2025-10-09T21:43:39.390226Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 51981 to budget
2025-10-09T21:43:39.390231Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 51981, ReadSizeServerDelta = 472307
2025-10-09T21:43:39.390246Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 235 to 235
2025-10-09T21:43:39.390252Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.388748Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 64 partNo 0 FormedBlobsCount 0 NewHead: Offset 63 PartNo 0 PackedSize 7087 count 1 nextOffset 64 batches 1
2025-10-09T21:43:39.388754Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 65 partNo 0
2025-10-09T21:43:39.388759Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 65 partNo 0 FormedBlobsCount 0 NewHead: Offset 63 PartNo 0 PackedSize 14252 count 2 nextOffset 65 batches 1
2025-10-09T21:43:39.388762Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 66 partNo 0
2025-10-09T21:43:39.390271Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (55-55)
2025-10-09T21:43:39.388767Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 66 partNo 0 FormedBlobsCount 0 NewHead: Offset 63 PartNo 0 PackedSize 21527 count 3 nextOffset 66 batches 1
2025-10-09T21:43:39.388769Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 55 EndOffset 55 newEndOffset 63 commitOffset 0 clientCommitOffset 0 cookie 6 readingFinished 0 firstRead 0
2025-10-09T21:43:39.390280Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (56-56)
2025-10-09T21:43:39.390253Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388770Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 67 partNo 0
2025-10-09T21:43:39.390294Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 2} (57-57)
2025-10-09T21:43:39.390295Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 235
2025-10-09T21:43:39.390304Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 3} (58-58)
2025-10-09T21:43:39.388773Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 55 endOffset 63
2025-10-09T21:43:39.388774Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 67 partNo 0 FormedBlobsCount 0 NewHead: Offset 63 PartNo 0 PackedSize 28912 count 4 nextOffset 67 batches 1
2025-10-09T21:43:39.390318Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 4} (59-59)
2025-10-09T21:43:39.388777Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 68 partNo 0
2025-10-09T21:43:39.388780Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 68 partNo 0 FormedBlobsCount 0 NewHead: Offset 63 PartNo 0 PackedSize 36407 count 5 nextOffset 68 batches 1
2025-10-09T21:43:39.390329Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 5} (60-60)
2025-10-09T21:43:39.388782Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 55, endOffset# 63, WTime# 1760046219381, sizeLag# 52318
2025-10-09T21:43:39.388783Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 69 partNo 0
2025-10-09T21:43:39.390339Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 236 to 236
2025-10-09T21:43:39.388784Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:43:39.390342Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 6} (61-61)
2025-10-09T21:43:39.388787Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 69 partNo 0 FormedBlobsCount 0 NewHead: Offset 63 PartNo 0 PackedSize 44012 count 6 nextOffset 69 batches 1
2025-10-09T21:43:39.388790Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 70 partNo 0
2025-10-09T21:43:39.390350Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 7} (62-62)
2025-10-09T21:43:39.388794Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 70 partNo 0 FormedBlobsCount 0 NewHead: Offset 63 PartNo 0 PackedSize 51727 count 7 nextOffset 70 batches 1
2025-10-09T21:43:39.388796Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# 6a9f63c9-39c09c83-1ccd9130-b3ee846d, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 9, size# 62781, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.390353Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388797Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 71 partNo 0
2025-10-09T21:43:39.390368Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 236
2025-10-09T21:43:39.388801Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 71 partNo 0 FormedBlobsCount 0 NewHead: Offset 63 PartNo 0 PackedSize 59552 count 8 nextOffset 71 batches 1
2025-10-09T21:43:39.388803Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.388808Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 9 maxSize 62781 maxTimeLagMs 0 readTimestampMs 0 readOffset 55 EndOffset 63 ClientCommitOffset 0 committedOffset 0 Guid 6a9f63c9-39c09c83-1ccd9130-b3ee846d
2025-10-09T21:43:39.388872Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 63,8 HeadOffset 63 endOffset 63 curOffset 71 d0000000000_00000000000000000063_00000_0000000008_00000? size 59358 WTime 1760046219388
2025-10-09T21:43:39.388892Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.388942Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.388953Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.388956Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390393Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 237 to 237
2025-10-09T21:43:39.388965Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 72 partNo : 0 messageNo: 147 size 7893 offset: -1
2025-10-09T21:43:39.388971Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390408Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.388973Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390411Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 237
2025-10-09T21:43:39.388977Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 73 partNo : 0 messageNo: 148 size 8003 offset: -1
2025-10-09T21:43:39.388980Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.388982Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.388986Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 74 partNo : 0 messageNo: 149 size 8113 offset: -1
2025-10-09T21:43:39.388990Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.388991Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.388995Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 75 partNo : 0 messageNo: 150 size 8223 offset: -1
2025-10-09T21:43:39.388998Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389000Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.389007Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 76 partNo : 0 messageNo: 151 size 8333 offset: -1
2025-10-09T21:43:39.389010Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389012Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.389016Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390465Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 238 to 238
2025-10-09T21:43:39.389018Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.389021Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390481Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 238
2025-10-09T21:43:39.389023Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.389026Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389028Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.389032Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389034Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.389041Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 113  requestId: <none> cookie: 56
2025-10-09T21:43:39.389049Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 114  requestId: <none> cookie: 57
2025-10-09T21:43:39.389054Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 115  requestId: <none> cookie: 58
2025-10-09T21:43:39.389059Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.389062Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 118  requestId: <none> cookie: 59
2025-10-09T21:43:39.389066Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 119  requestId: <none> cookie: 60
2025-10-09T21:43:39.389071Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.389071Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 123  requestId: <none> cookie: 61
>>> event from dataHandler: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..6061 bytes.. Information: { Offset: 55 ProducerId: "src_id" SeqNo: 56 CreateTime: 2025-10-09T21:43:39.370000Z WriteTime: 2025-10-09T21:43:39.384000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..6171 bytes.. Information: { Offset: 56 ProducerId: "src_id" SeqNo: 57 CreateTime: 2025-10-09T21:43:39.370000Z WriteTime: 2025-10-09T21:43:39.384000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..6281 bytes.. Information: { Offset: 57 ProducerId: "src_id" SeqNo: 58 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.384000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..6391 bytes.. Information: { Offset: 58 ProducerId: "src_id" SeqNo: 59 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.384000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..6501 bytes.. Information: { Offset: 59 ProducerId: "src_id" SeqNo: 60 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.384000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..6611 bytes.. Information: { Offset: 60 ProducerId: "src_id" SeqNo: 61 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.384000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..6721 bytes.. Information: { Offset: 61 ProducerId: "src_id" SeqNo: 62 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.384000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..6831 bytes.. Information: { Offset: 62 ProducerId: "src_id" SeqNo: 63 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.384000Z MessageGroupId: "src_id" Meta: { "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.389076Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 124  requestId: <none> cookie: 62
2025-10-09T21:43:39.389079Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390518Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 239 to 239
2025-10-09T21:43:39.389080Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 125  requestId: <none> cookie: 63
2025-10-09T21:43:39.389083Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390594Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.390594Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 8, size 51568 bytes
2025-10-09T21:43:39.390597Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 239
2025-10-09T21:43:39.389091Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389094Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390607Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.389102Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.389113Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 63 partNo 0 count 8 size 59358
2025-10-09T21:43:39.389119Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.389125Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.389128Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.389129Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.389131Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.389134Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.389135Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.389137Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.389139Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.389141Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.389154Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.389158Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.389162Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.389169Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.389174Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.389184Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 152  requestId: <none> cookie: 77
2025-10-09T21:43:39.389187Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 153  requestId: <none> cookie: 78
2025-10-09T21:43:39.389189Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 154  requestId: <none> cookie: 79
2025-10-09T21:43:39.389193Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 155  requestId: <none> cookie: 80
2025-10-09T21:43:39.389196Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 156  requestId: <none> cookie: 81
2025-10-09T21:43:39.389230Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 9 Topic 'rt3.dc1--test-topic' partition 0 user user offset 55 count 9 size 62781 endOffset 63 max time lag 0ms effective offset 55
2025-10-09T21:43:39.389243Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 9 added 1 blobs, size 52318 count 8 last offset 55, current partition end offset: 63
2025-10-09T21:43:39.389245Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 9. Send blob request.
2025-10-09T21:43:39.389254Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 55 partno 0 count 8 parts_count 0 source 1 size 52318 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.389257Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 9. All 1 blobs are from cache.
2025-10-09T21:43:39.389265Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.389276Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 8 parts 0 suffix '63'
2025-10-09T21:43:39.389346Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 55 totakecount 8 count 8 size 52296 from pos 0 cbcount 8
2025-10-09T21:43:39.389375Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 55
2025-10-09T21:43:39.389479Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 63 Result { Offset: 55 Data: "... 6133 bytes ..." SourceId: "\000src_id" SeqNo: 56 WriteTimestampMS: 1760046219384 CreateTimestampMS: 1760046219370 UncompressedSize: 6061 PartitionKey: "" ExplicitHash: "" } Result { Offset: 56 Data: "... 6243 bytes ..." SourceId: "\000src_id" SeqNo: 57 WriteTimestampMS: 1760046219384 CreateTimestampMS: 1760046219370 UncompressedSize: 6171 PartitionKey: "" ExplicitHash: "" } Result { Offset: 57 Data: "... 6353 bytes ..." SourceId: "\000src_id" SeqNo: 58 WriteTimestampMS: 1760046219384 CreateTimestampMS: 1760046219371 UncompressedSize: 6281 PartitionKey: "" ExplicitHash: "" } Result { Offset: 58 Data: "... 6463 bytes ..." SourceId: "\000src_id" SeqNo: 59 WriteTimestampMS: 1760046219384 CreateTimestampMS: 1760046219371 UncompressedSize: 6391 PartitionKey: "" ExplicitHash: "" } Result { Offset: 59 Data: "... 6573 bytes ..." SourceId: "\000src_id" SeqNo: 60 WriteTimestampMS: 1760046219384 CreateTimestampMS: 1760046219371 UncompressedSize: 6501 PartitionKey: "" ExplicitHash: "" } Result { Offset: 60 Data: "... 6683 bytes ..." SourceId: "\000src_id" SeqNo: 61 WriteTimestampMS: 1760046219384 CreateTimestampMS: 1760046219371 UncompressedSize: 6611 PartitionKey: "" ExplicitHash: "" } Result { Offset: 61 Data: "... 6793 bytes ..." SourceId: "\000src_id" SeqNo: 62 WriteTimestampMS: 1760046219384 CreateTimestampMS: 1760046219371 UncompressedSize: 6721 PartitionKey: "" ExplicitHash: "" } Result { Offset: 62 Data: "... 6903 bytes ..." SourceId: "\000src_id" SeqNo: 63 WriteTimestampMS: 1760046219384 CreateTimestampMS: 1760046219371 UncompressedSize: 6831 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 18446744073709551454 RealReadOffset: 62 WaitQuotaTimeMs: 0 EndOffset: 63 StartOffset: 0 } Cookie: 55 }
2025-10-09T21:43:39.389522Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 7 from offset 63
2025-10-09T21:43:39.389529Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 63 ReadOffset 63 ReadGuid 6a9f63c9-39c09c83-1ccd9130-b3ee846d has messages 1
2025-10-09T21:43:39.389553Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# 6a9f63c9-39c09c83-1ccd9130-b3ee846d, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 51981
2025-10-09T21:43:39.389560Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# 6a9f63c9-39c09c83-1ccd9130-b3ee846d
2025-10-09T21:43:39.389644Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.389658Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 0
2025-10-09T21:43:39.389663Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.389678Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.389693Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.389696Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.389701Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.389772Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.389781Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.389788Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.389794Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.389800Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.389809Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.389861Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389867Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.389879Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 77 partNo : 0 messageNo: 157 size 8443 offset: -1
2025-10-09T21:43:39.389885Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389887Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.389892Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 78 partNo : 0 messageNo: 158 size 8553 offset: -1
2025-10-09T21:43:39.389896Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389897Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.389902Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 79 partNo : 0 messageNo: 159 size 8663 offset: -1
2025-10-09T21:43:39.389905Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389907Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.389912Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 80 partNo : 0 messageNo: 160 size 8773 offset: -1
2025-10-09T21:43:39.389915Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389917Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.389923Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 81 partNo : 0 messageNo: 161 size 8883 offset: -1
2025-10-09T21:43:39.389926Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389928Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.389934Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389935Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.389951Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389954Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.389957Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389959Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.389963Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.389965Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.389971Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.389979Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.389982Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.389985Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.389987Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.389989Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.389991Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.389994Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.389996Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.389998Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.390012Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 162  requestId: <none> cookie: 82
2025-10-09T21:43:39.390018Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 163  requestId: <none> cookie: 83
2025-10-09T21:43:39.390021Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 164  requestId: <none> cookie: 84
2025-10-09T21:43:39.390024Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 165  requestId: <none> cookie: 85
2025-10-09T21:43:39.390027Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 166  requestId: <none> cookie: 86
2025-10-09T21:43:39.390055Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.390062Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390069Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390073Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390077Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390081Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390142Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.390151Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.390155Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.390161Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.390166Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.390254Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390258Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390265Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 82 partNo : 0 messageNo: 167 size 8993 offset: -1
2025-10-09T21:43:39.390270Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390272Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390276Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 83 partNo : 0 messageNo: 168 size 9103 offset: -1
2025-10-09T21:43:39.390279Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390281Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390285Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 84 partNo : 0 messageNo: 169 size 9213 offset: -1
2025-10-09T21:43:39.390287Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390289Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390293Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 85 partNo : 0 messageNo: 170 size 9323 offset: -1
2025-10-09T21:43:39.390298Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390299Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390303Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 86 partNo : 0 messageNo: 171 size 9433 offset: -1
2025-10-09T21:43:39.390973Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 240 to 240
2025-10-09T21:43:39.390990Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.390994Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 240
2025-10-09T21:43:39.391034Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 241 to 241
2025-10-09T21:43:39.390306Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.391040Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.390307Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.391062Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 241
2025-10-09T21:43:39.390311Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390313Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390316Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390317Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390320Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390321Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390325Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390327Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390332Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.390339Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.390341Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.390343Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.390346Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.390349Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.390351Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.390353Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.391124Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 242 to 242
2025-10-09T21:43:39.390355Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.390358Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.390368Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 172  requestId: <none> cookie: 87
2025-10-09T21:43:39.390373Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 173  requestId: <none> cookie: 88
2025-10-09T21:43:39.390376Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 174  requestId: <none> cookie: 89
2025-10-09T21:43:39.390379Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 175  requestId: <none> cookie: 90
2025-10-09T21:43:39.390381Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 176  requestId: <none> cookie: 91
2025-10-09T21:43:39.390386Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.390392Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390397Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390400Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390403Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390407Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390475Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.390482Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.390487Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.390492Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.390498Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.390675Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390682Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390695Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 87 partNo : 0 messageNo: 177 size 9543 offset: -1
2025-10-09T21:43:39.390704Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390705Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390709Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 88 partNo : 0 messageNo: 178 size 9653 offset: -1
2025-10-09T21:43:39.390713Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390714Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390719Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 89 partNo : 0 messageNo: 179 size 9763 offset: -1
2025-10-09T21:43:39.390723Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390725Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390729Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 90 partNo : 0 messageNo: 180 size 9873 offset: -1
2025-10-09T21:43:39.390733Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390734Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390738Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 91 partNo : 0 messageNo: 181 size 9983 offset: -1
2025-10-09T21:43:39.390743Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390745Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390749Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390751Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390754Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390756Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390759Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390761Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390764Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390766Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390771Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.390779Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.390782Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.390785Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.390787Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.390789Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.390791Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.390794Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.390796Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.390798Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.390811Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 182  requestId: <none> cookie: 92
2025-10-09T21:43:39.390825Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 183  requestId: <none> cookie: 93
2025-10-09T21:43:39.390829Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 184  requestId: <none> cookie: 94
2025-10-09T21:43:39.390832Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 185  requestId: <none> cookie: 95
2025-10-09T21:43:39.390835Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 186  requestId: <none> cookie: 96
2025-10-09T21:43:39.390846Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390854Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390859Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390862Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390866Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.390880Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.390891Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390893Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390898Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 92 partNo : 0 messageNo: 187 size 10093 offset: -1
2025-10-09T21:43:39.390902Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390903Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390907Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 93 partNo : 0 messageNo: 188 size 10203 offset: -1
2025-10-09T21:43:39.390909Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390911Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390914Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 94 partNo : 0 messageNo: 189 size 10313 offset: -1
2025-10-09T21:43:39.390917Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390919Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390922Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 95 partNo : 0 messageNo: 190 size 10423 offset: -1
2025-10-09T21:43:39.390927Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390929Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390933Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 96 partNo : 0 messageNo: 191 size 10533 offset: -1
2025-10-09T21:43:39.390936Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.390938Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.390942Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.390945Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.390947Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.390949Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.390951Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.390951Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.390954Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.390956Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.390958Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.390960Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.390963Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.390967Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 192  requestId: <none> cookie: 97
2025-10-09T21:43:39.390987Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 51981 } }
2025-10-09T21:43:39.391052Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# 775f8ca9-46b952d4-c4b8f2ff-33adf93a
2025-10-09T21:43:39.391071Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.391082Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.391123Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.391129Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.391137Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.391140Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 97 partNo : 0 messageNo: 193 size 10643 offset: -1
2025-10-09T21:43:39.391170Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.391183Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.391186Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.391193Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.391203Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 63 count 8 size 59358 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.391210Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.391212Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.391217Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.391219Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 8 parts 0 suffix '63' size 59358
2025-10-09T21:43:39.391229Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 59240 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.391237Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.391244Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 64, partNo: 0, Offset: 63 is stored on disk
2025-10-09T21:43:39.391245Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.391247Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.391251Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.391251Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 65, partNo: 0, Offset: 64 is stored on disk
2025-10-09T21:43:39.391253Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.391270Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 66, partNo: 0, Offset: 65 is stored on disk
2025-10-09T21:43:39.391272Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.391273Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.391276Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 67, partNo: 0, Offset: 66 is stored on disk
2025-10-09T21:43:39.391279Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.391282Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 68, partNo: 0, Offset: 67 is stored on disk
2025-10-09T21:43:39.391285Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.391289Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 69, partNo: 0, Offset: 68 is stored on disk
2025-10-09T21:43:39.391293Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.391297Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 70, partNo: 0, Offset: 69 is stored on disk
2025-10-09T21:43:39.391300Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.391304Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 71, partNo: 0, Offset: 70 is stored on disk
2025-10-09T21:43:39.391336Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.391354Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.391356Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (26)
2025-10-09T21:43:39.391359Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 63 EndOffset 63 newEndOffset 71 commitOffset 0 clientCommitOffset 0 cookie 7 readingFinished 0 firstRead 0
2025-10-09T21:43:39.391365Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 63 endOffset 71
2025-10-09T21:43:39.391374Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 72 partNo 0
2025-10-09T21:43:39.391375Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 63, endOffset# 71, WTime# 1760046219384, sizeLag# 59358
2025-10-09T21:43:39.391378Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:43:39.391389Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# 1859aa7d-c62b2f16-8ec3d2d1-38ad8310, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 9, size# 71229, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.391398Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 72 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 7967 count 1 nextOffset 72 batches 1
2025-10-09T21:43:39.391400Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 9 maxSize 71229 maxTimeLagMs 0 readTimestampMs 0 readOffset 63 EndOffset 71 ClientCommitOffset 0 committedOffset 0 Guid 1859aa7d-c62b2f16-8ec3d2d1-38ad8310
2025-10-09T21:43:39.391405Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 73 partNo 0
2025-10-09T21:43:39.391409Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 73 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 16012 count 2 nextOffset 73 batches 1
2025-10-09T21:43:39.391412Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 74 partNo 0
2025-10-09T21:43:39.391417Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 74 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 24167 count 3 nextOffset 74 batches 1
2025-10-09T21:43:39.391420Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 75 partNo 0
2025-10-09T21:43:39.391424Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 75 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 32432 count 4 nextOffset 75 batches 1
2025-10-09T21:43:39.391427Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 76 partNo 0
2025-10-09T21:43:39.391431Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 76 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 40807 count 5 nextOffset 76 batches 1
2025-10-09T21:43:39.391434Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 77 partNo 0
2025-10-09T21:43:39.391438Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 77 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 49292 count 6 nextOffset 77 batches 1
2025-10-09T21:43:39.391441Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 78 partNo 0
2025-10-09T21:43:39.391445Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 78 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 57887 count 7 nextOffset 78 batches 1
2025-10-09T21:43:39.391448Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 79 partNo 0
2025-10-09T21:43:39.391452Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 79 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 66592 count 8 nextOffset 79 batches 1
2025-10-09T21:43:39.391455Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 80 partNo 0
2025-10-09T21:43:39.391461Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 80 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 75407 count 9 nextOffset 80 batches 1
2025-10-09T21:43:39.391465Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 81 partNo 0
2025-10-09T21:43:39.391470Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 81 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 84332 count 10 nextOffset 81 batches 1
2025-10-09T21:43:39.391475Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 82 partNo 0
2025-10-09T21:43:39.391478Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 82 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 93367 count 11 nextOffset 82 batches 1
2025-10-09T21:43:39.391479Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.391482Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 83 partNo 0
2025-10-09T21:43:39.391486Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 83 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 102512 count 12 nextOffset 83 batches 1
2025-10-09T21:43:39.391489Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 84 partNo 0
2025-10-09T21:43:39.391493Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 84 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 111767 count 13 nextOffset 84 batches 1
2025-10-09T21:43:39.391497Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 85 partNo 0
2025-10-09T21:43:39.391500Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 85 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 121132 count 14 nextOffset 85 batches 1
2025-10-09T21:43:39.391504Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 86 partNo 0
2025-10-09T21:43:39.391507Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 86 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 130607 count 15 nextOffset 86 batches 1
2025-10-09T21:43:39.391511Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 87 partNo 0
2025-10-09T21:43:39.391514Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 87 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 140192 count 16 nextOffset 87 batches 1
2025-10-09T21:43:39.391518Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 88 partNo 0
2025-10-09T21:43:39.391522Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 88 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 149887 count 17 nextOffset 88 batches 1
2025-10-09T21:43:39.391526Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 89 partNo 0
2025-10-09T21:43:39.391530Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 89 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 159692 count 18 nextOffset 89 batches 1
2025-10-09T21:43:39.391533Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 90 partNo 0
2025-10-09T21:43:39.391536Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 90 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 169607 count 19 nextOffset 90 batches 1
2025-10-09T21:43:39.391540Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 91 partNo 0
2025-10-09T21:43:39.391543Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 91 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 179632 count 20 nextOffset 91 batches 1
2025-10-09T21:43:39.391546Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 92 partNo 0
2025-10-09T21:43:39.391549Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 92 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 189767 count 21 nextOffset 92 batches 1
2025-10-09T21:43:39.391550Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.391553Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 93 partNo 0
2025-10-09T21:43:39.391556Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 93 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 200012 count 22 nextOffset 93 batches 1
2025-10-09T21:43:39.391559Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 94 partNo 0
2025-10-09T21:43:39.391564Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 94 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 210367 count 23 nextOffset 94 batches 1
2025-10-09T21:43:39.391567Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 95 partNo 0
2025-10-09T21:43:39.391571Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 95 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 220832 count 24 nextOffset 95 batches 1
2025-10-09T21:43:39.391574Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 96 partNo 0
2025-10-09T21:43:39.391577Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 96 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 231407 count 25 nextOffset 96 batches 1
2025-10-09T21:43:39.391582Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 97 partNo 0
2025-10-09T21:43:39.391585Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 97 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 242092 count 26 nextOffset 97 batches 1
2025-10-09T21:43:39.391588Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.391589Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.391612Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.391780Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.391835Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.391885Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 71,26 HeadOffset 71 endOffset 71 curOffset 97 d0000000000_00000000000000000071_00000_0000000026_00000? size 241322 WTime 1760046219391
2025-10-09T21:43:39.391906Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.391144Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.392023Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 242
2025-10-09T21:43:39.392012Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.392052Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.392181Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 243 to 243
2025-10-09T21:43:39.392100Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.392131Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 129  requestId: <none> cookie: 64
2025-10-09T21:43:39.392134Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.392138Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392142Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392142Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392150Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 130  requestId: <none> cookie: 65
2025-10-09T21:43:39.392154Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392156Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392161Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 131  requestId: <none> cookie: 66
2025-10-09T21:43:39.392166Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 134  requestId: <none> cookie: 67
2025-10-09T21:43:39.392172Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 135  requestId: <none> cookie: 68
2025-10-09T21:43:39.392173Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392177Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392236Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.392178Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 138  requestId: <none> cookie: 69
2025-10-09T21:43:39.392241Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 243
2025-10-09T21:43:39.392180Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392183Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392184Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392190Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 139  requestId: <none> cookie: 70
2025-10-09T21:43:39.392194Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 143  requestId: <none> cookie: 71
2025-10-09T21:43:39.392205Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392207Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392215Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392217Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392221Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.392232Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 71 partNo 0 count 26 size 241322
2025-10-09T21:43:39.392239Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 194  requestId: <none> cookie: 98
2025-10-09T21:43:39.392247Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392256Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392261Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392265Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392269Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392278Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 195  requestId: <none> cookie: 99
2025-10-09T21:43:39.392327Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 25 offsets: 24 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.392281Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 196  requestId: <none> cookie: 100
2025-10-09T21:43:39.392284Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 197  requestId: <none> cookie: 101
2025-10-09T21:43:39.392343Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 25
2025-10-09T21:43:39.392293Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 198  requestId: <none> cookie: 102
2025-10-09T21:43:39.392309Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392313Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392317Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392317Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 10 Topic 'rt3.dc1--test-topic' partition 0 user user offset 63 count 9 size 71229 endOffset 71 max time lag 0ms effective offset 63
2025-10-09T21:43:39.392331Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 10 added 1 blobs, size 59358 count 8 last offset 63, current partition end offset: 71
2025-10-09T21:43:39.392333Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 10. Send blob request.
2025-10-09T21:43:39.392335Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.392338Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392340Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392342Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392347Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 98 partNo : 0 messageNo: 199 size 10753 offset: -1
2025-10-09T21:43:39.392353Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.392355Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 63 partno 0 count 8 parts_count 0 source 1 size 59358 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.392358Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.392359Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 10. All 1 blobs are from cache.
2025-10-09T21:43:39.392364Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.392365Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.392369Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 8 parts 0 suffix '63'
2025-10-09T21:43:39.392370Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.392371Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.392373Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392375Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392378Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 99 partNo : 0 messageNo: 200 size 10863 offset: -1
2025-10-09T21:43:39.392382Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392385Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392388Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 100 partNo : 0 messageNo: 201 size 10973 offset: -1
2025-10-09T21:43:39.392391Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392394Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392398Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 101 partNo : 0 messageNo: 202 size 11083 offset: -1
2025-10-09T21:43:39.392401Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392403Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392406Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392514Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 244 to 244
2025-10-09T21:43:39.392533Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.392537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 244
2025-10-09T21:43:39.392408Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392411Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 102 partNo : 0 messageNo: 204 size 11193 offset: -1
2025-10-09T21:43:39.392417Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.392447Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.392461Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 63 totakecount 8 count 8 size 59336 from pos 0 cbcount 8
2025-10-09T21:43:39.392481Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392482Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392486Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392488Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392492Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392494Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392497Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392499Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392502Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.392505Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.392507Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.392510Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.392511Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.392514Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.392611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 245 to 245
2025-10-09T21:43:39.392517Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.392618Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.392621Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 245
2025-10-09T21:43:39.392520Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.392528Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 63
2025-10-09T21:43:39.392541Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 203  requestId: <none> cookie: 103
2025-10-09T21:43:39.392562Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392569Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 205  requestId: <none> cookie: 104
2025-10-09T21:43:39.392569Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.392572Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 206  requestId: <none> cookie: 105
2025-10-09T21:43:39.392574Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 207  requestId: <none> cookie: 106
2025-10-09T21:43:39.392577Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 208  requestId: <none> cookie: 107
2025-10-09T21:43:39.392581Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392629Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.392637Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392642Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392646Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392656Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 71 Result { Offset: 63 Data: "... 7013 bytes ..." SourceId: "\000src_id" SeqNo: 64 WriteTimestampMS: 1760046219388 CreateTimestampMS: 1760046219371 UncompressedSize: 6941 PartitionKey: "" ExplicitHash: "" } Result { Offset: 64 Data: "... 7123 bytes ..." SourceId: "\000src_id" SeqNo: 65 WriteTimestampMS: 1760046219388 CreateTimestampMS: 1760046219371 UncompressedSize: 7051 PartitionKey: "" ExplicitHash: "" } Result { Offset: 65 Data: "... 7233 bytes ..." SourceId: "\000src_id" SeqNo: 66 WriteTimestampMS: 1760046219388 CreateTimestampMS: 1760046219371 UncompressedSize: 7161 PartitionKey: "" ExplicitHash: "" } Result { Offset: 66 Data: "... 7343 bytes ..." SourceId: "\000src_id" SeqNo: 67 WriteTimestampMS: 1760046219388 CreateTimestampMS: 1760046219371 UncompressedSize: 7271 PartitionKey: "" ExplicitHash: "" } Result { Offset: 67 Data: "... 7453 bytes ..." SourceId: "\000src_id" SeqNo: 68 WriteTimestampMS: 1760046219388 CreateTimestampMS: 1760046219371 UncompressedSize: 7381 PartitionKey: "" ExplicitHash: "" } Result { Offset: 68 Data: "... 7563 bytes ..." SourceId: "\000src_id" SeqNo: 69 WriteTimestampMS: 1760046219388 CreateTimestampMS: 1760046219371 UncompressedSize: 7491 PartitionKey: "" ExplicitHash: "" } Result { Offset: 69 Data: "... 7673 bytes ..." SourceId: "\000src_id" SeqNo: 70 WriteTimestampMS: 1760046219388 CreateTimestampMS: 1760046219371 UncompressedSize: 7601 PartitionKey: "" ExplicitHash: "" } Result { Offset: 70 Data: "... 7783 bytes ..." SourceId: "\000src_id" SeqNo: 71 WriteTimestampMS: 1760046219388 CreateTimestampMS: 1760046219371 UncompressedSize: 7711 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 18446744073709551454 RealReadOffset: 70 WaitQuotaTimeMs: 0 EndOffset: 71 StartOffset: 0 } Cookie: 63 }
2025-10-09T21:43:39.392720Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 246 to 246
2025-10-09T21:43:39.392701Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.392702Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 8 from offset 71
2025-10-09T21:43:39.392708Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 71 ReadOffset 71 ReadGuid 1859aa7d-c62b2f16-8ec3d2d1-38ad8310 has messages 1
2025-10-09T21:43:39.392741Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.392713Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.392745Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 246
2025-10-09T21:43:39.392719Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392719Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.392721Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392724Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.392727Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 103 partNo : 0 messageNo: 209 size 11303 offset: -1
2025-10-09T21:43:39.392729Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.392731Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392732Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392771Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 247 to 247
2025-10-09T21:43:39.392736Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 104 partNo : 0 messageNo: 210 size 11413 offset: -1
2025-10-09T21:43:39.392740Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392741Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392745Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392781Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.392815Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 247
2025-10-09T21:43:39.392747Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392751Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 105 partNo : 0 messageNo: 212 size 11523 offset: -1
2025-10-09T21:43:39.392754Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392756Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392759Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 106 partNo : 0 messageNo: 213 size 11633 offset: -1
2025-10-09T21:43:39.392762Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392765Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392769Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 107 partNo : 0 messageNo: 214 size 11743 offset: -1
2025-10-09T21:43:39.392774Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.392778Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.392779Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.392782Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.392785Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.392787Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.392789Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.392791Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.392793Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.392795Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.392796Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# 1859aa7d-c62b2f16-8ec3d2d1-38ad8310, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 59021
2025-10-09T21:43:39.392802Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 211  requestId: <none> cookie: 108
2025-10-09T21:43:39.392805Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# 1859aa7d-c62b2f16-8ec3d2d1-38ad8310
2025-10-09T21:43:39.392806Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392808Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392811Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392813Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392816Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392818Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392821Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392823Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392831Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 215  requestId: <none> cookie: 109
2025-10-09T21:43:39.392834Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 216  requestId: <none> cookie: 110
2025-10-09T21:43:39.392837Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 217  requestId: <none> cookie: 111
2025-10-09T21:43:39.392840Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 218  requestId: <none> cookie: 112
2025-10-09T21:43:39.392848Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392851Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 0
2025-10-09T21:43:39.392854Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392858Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392861Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392865Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.392887Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.392952Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392955Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392960Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 108 partNo : 0 messageNo: 219 size 11853 offset: -1
2025-10-09T21:43:39.392964Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392965Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392969Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 109 partNo : 0 messageNo: 220 size 11963 offset: -1
2025-10-09T21:43:39.392970Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.392972Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392974Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392977Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 110 partNo : 0 messageNo: 221 size 12073 offset: -1
2025-10-09T21:43:39.392980Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392982Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392986Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 111 partNo : 0 messageNo: 222 size 12183 offset: -1
2025-10-09T21:43:39.392990Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.392992Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.392995Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 112 partNo : 0 messageNo: 223 size 12293 offset: -1
2025-10-09T21:43:39.392999Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.393001Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.393004Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.393008Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.393009Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.393011Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.393013Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.393015Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.393016Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.393017Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.393019Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.393021Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.393023Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.393029Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 224  requestId: <none> cookie: 113
2025-10-09T21:43:39.393046Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.393154Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 248 to 248
2025-10-09T21:43:39.393171Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.393174Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 248
2025-10-09T21:43:39.393210Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 249 to 249
2025-10-09T21:43:39.393238Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.393241Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 249
2025-10-09T21:43:39.393278Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 26 offsets: 25 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.393290Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 26
2025-10-09T21:43:39.393184Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.393268Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.393273Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.393331Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 59021, now ReadSizeBudget = 0, ReadSizeServerDelta = 465267
2025-10-09T21:43:39.393344Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 250 to 250
2025-10-09T21:43:39.393280Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.393288Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.393290Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.393353Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.393301Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 113 partNo : 0 messageNo: 226 size 12403 offset: -1
2025-10-09T21:43:39.393358Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 250
2025-10-09T21:43:39.393312Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.393313Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.393362Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 0, ReadSizeServerDelta = 465267
2025-10-09T21:43:39.393319Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.393327Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.393332Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 225  requestId: <none> cookie: 114
2025-10-09T21:43:39.393347Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 227  requestId: <none> cookie: 115
2025-10-09T21:43:39.393359Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.393367Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.393418Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (63-70)
2025-10-09T21:43:39.393431Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 59021 to budget
2025-10-09T21:43:39.393437Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 59021, ReadSizeServerDelta = 465267
2025-10-09T21:43:39.393486Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.393496Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.393503Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 114 partNo : 0 messageNo: 228 size 12513 offset: -1
2025-10-09T21:43:39.393518Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.393527Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.393541Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.393533Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 115 partNo : 0 messageNo: 229 size 12623 offset: -1
2025-10-09T21:43:39.393536Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.393541Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.393543Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.393545Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.393568Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (63-63)
2025-10-09T21:43:39.393616Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (64-64)
2025-10-09T21:43:39.393642Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 2} (65-65)
2025-10-09T21:43:39.393645Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 251 to 251
2025-10-09T21:43:39.393647Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 3} (66-66)
2025-10-09T21:43:39.393673Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.393678Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 251
2025-10-09T21:43:39.393714Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 252 to 252
2025-10-09T21:43:39.393721Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.393723Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 252
2025-10-09T21:43:39.393727Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 4} (67-67)
2025-10-09T21:43:39.393734Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 5} (68-68)
2025-10-09T21:43:39.393738Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 6} (69-69)
2025-10-09T21:43:39.393742Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 7} (70-70)
2025-10-09T21:43:39.393748Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 253 to 253
2025-10-09T21:43:39.393757Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.393759Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 253
2025-10-09T21:43:39.393789Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 254 to 254
2025-10-09T21:43:39.393807Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.393810Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 254
2025-10-09T21:43:39.393809Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 59021 } }
2025-10-09T21:43:39.393835Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 255 to 255
>>> event from dataHandler: 2025-10-09T21:43:39.393836Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# 3dc0c9bb-d634311-491772f4-10b764f1
2025-10-09T21:43:39.393774Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.393862Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.393865Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 255
DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..6941 bytes.. Information: { Offset: 63 ProducerId: "src_id" SeqNo: 64 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.388000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..7051 bytes.. Information: { Offset: 64 ProducerId: "src_id" SeqNo: 65 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.388000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..7161 bytes.. Information: { Offset: 65 ProducerId: "src_id" SeqNo: 66 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.388000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..7271 bytes.. Information: { Offset: 66 ProducerId: "src_id" SeqNo: 67 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.388000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..7381 bytes.. Information: { Offset: 67 ProducerId: "src_id" SeqNo: 68 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.388000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..7491 bytes.. Information: { Offset: 68 ProducerId: "src_id" SeqNo: 69 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.388000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..7601 bytes.. Information: { Offset: 69 ProducerId: "src_id" SeqNo: 70 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.388000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..7711 bytes.. Information: { Offset: 70 ProducerId: "src_id" SeqNo: 71 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.388000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.393929Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 8, size 58608 bytes
2025-10-09T21:43:39.393937Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.393914Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.393929Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.393898Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 256 to 256
2025-10-09T21:43:39.393981Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.393990Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.393999Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.394002Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.394016Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 71 count 26 size 241322 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.394058Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.394064Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 256
2025-10-09T21:43:39.394097Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 257 to 257
2025-10-09T21:43:39.394140Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.394143Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 257
2025-10-09T21:43:39.394870Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 258 to 258
2025-10-09T21:43:39.394889Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.394893Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 258
2025-10-09T21:43:39.394933Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 259 to 259
2025-10-09T21:43:39.394939Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.394942Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 259
2025-10-09T21:43:39.394976Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 260 to 260
2025-10-09T21:43:39.394984Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.394987Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 260
2025-10-09T21:43:39.395017Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 261 to 261
2025-10-09T21:43:39.395024Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.395029Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 261
2025-10-09T21:43:39.395085Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 262 to 262
2025-10-09T21:43:39.395102Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.395108Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 262
2025-10-09T21:43:39.395140Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 263 to 263
2025-10-09T21:43:39.395152Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.395154Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 263
2025-10-09T21:43:39.395192Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 264 to 264
2025-10-09T21:43:39.395218Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.395220Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 264
2025-10-09T21:43:39.395247Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 265 to 265
2025-10-09T21:43:39.395276Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.395278Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 265
2025-10-09T21:43:39.395311Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 266 to 266
2025-10-09T21:43:39.395342Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.395345Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 266
2025-10-09T21:43:39.395390Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 267 to 267
2025-10-09T21:43:39.395415Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.395418Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 267
2025-10-09T21:43:39.395454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 268 to 268
2025-10-09T21:43:39.395480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.395483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 268
2025-10-09T21:43:39.395544Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 269 to 269
2025-10-09T21:43:39.395567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.395572Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 269
2025-10-09T21:43:39.395605Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 270 to 270
2025-10-09T21:43:39.395628Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.395630Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 270
2025-10-09T21:43:39.395710Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 271 to 271
2025-10-09T21:43:39.395736Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.395739Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 271
2025-10-09T21:43:39.395773Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 272 to 272
2025-10-09T21:43:39.395798Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.395801Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 272
2025-10-09T21:43:39.395840Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 273 to 273
2025-10-09T21:43:39.395865Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.395868Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 273
2025-10-09T21:43:39.395909Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 27 offsets: 26 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.395921Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 27
2025-10-09T21:43:39.395957Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 274 to 274
2025-10-09T21:43:39.396023Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.396027Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 274
2025-10-09T21:43:39.396068Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 275 to 275
2025-10-09T21:43:39.394050Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.396098Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.396101Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 275
2025-10-09T21:43:39.394049Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.394055Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.394069Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 241150 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.394075Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394082Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 72, partNo: 0, Offset: 71 is stored on disk
2025-10-09T21:43:39.394085Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394088Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 73, partNo: 0, Offset: 72 is stored on disk
2025-10-09T21:43:39.394090Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.396135Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 276 to 276
2025-10-09T21:43:39.394092Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 74, partNo: 0, Offset: 73 is stored on disk
2025-10-09T21:43:39.394092Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 26 parts 0 suffix '63' size 241322
2025-10-09T21:43:39.394095Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394098Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 75, partNo: 0, Offset: 74 is stored on disk
2025-10-09T21:43:39.394102Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394105Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 76, partNo: 0, Offset: 75 is stored on disk
2025-10-09T21:43:39.394107Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394111Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 77, partNo: 0, Offset: 76 is stored on disk
2025-10-09T21:43:39.396167Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.396172Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 276
2025-10-09T21:43:39.394113Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394117Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 78, partNo: 0, Offset: 77 is stored on disk
2025-10-09T21:43:39.394119Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394131Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 79, partNo: 0, Offset: 78 is stored on disk
2025-10-09T21:43:39.394133Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394137Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 80, partNo: 0, Offset: 79 is stored on disk
2025-10-09T21:43:39.394139Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394143Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 81, partNo: 0, Offset: 80 is stored on disk
2025-10-09T21:43:39.396214Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 277 to 277
2025-10-09T21:43:39.394145Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394148Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 82, partNo: 0, Offset: 81 is stored on disk
2025-10-09T21:43:39.394151Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394155Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 83, partNo: 0, Offset: 82 is stored on disk
2025-10-09T21:43:39.394157Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394161Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 84, partNo: 0, Offset: 83 is stored on disk
2025-10-09T21:43:39.394163Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394167Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 85, partNo: 0, Offset: 84 is stored on disk
2025-10-09T21:43:39.394169Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.396244Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.394169Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.396247Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 277
2025-10-09T21:43:39.394173Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 86, partNo: 0, Offset: 85 is stored on disk
2025-10-09T21:43:39.394176Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394180Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 87, partNo: 0, Offset: 86 is stored on disk
2025-10-09T21:43:39.394182Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394186Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 88, partNo: 0, Offset: 87 is stored on disk
2025-10-09T21:43:39.394188Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394192Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 89, partNo: 0, Offset: 88 is stored on disk
2025-10-09T21:43:39.394194Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394197Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 90, partNo: 0, Offset: 89 is stored on disk
2025-10-09T21:43:39.394200Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.396285Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 278 to 278
2025-10-09T21:43:39.394203Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 91, partNo: 0, Offset: 90 is stored on disk
2025-10-09T21:43:39.394205Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394209Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 92, partNo: 0, Offset: 91 is stored on disk
2025-10-09T21:43:39.394211Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394214Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.396330Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.394215Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 93, partNo: 0, Offset: 92 is stored on disk
2025-10-09T21:43:39.396334Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 278
2025-10-09T21:43:39.394217Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394222Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 94, partNo: 0, Offset: 93 is stored on disk
2025-10-09T21:43:39.394226Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394230Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 95, partNo: 0, Offset: 94 is stored on disk
2025-10-09T21:43:39.394232Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394237Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 96, partNo: 0, Offset: 95 is stored on disk
2025-10-09T21:43:39.394239Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.394244Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 97, partNo: 0, Offset: 96 is stored on disk
2025-10-09T21:43:39.394264Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.396368Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 279 to 279
2025-10-09T21:43:39.394269Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.394269Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.394282Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.394285Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (18)
2025-10-09T21:43:39.394301Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 98 partNo 0
2025-10-09T21:43:39.394321Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 71 EndOffset 71 newEndOffset 97 commitOffset 0 clientCommitOffset 0 cookie 8 readingFinished 0 firstRead 0
2025-10-09T21:43:39.394324Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 71 endOffset 97
2025-10-09T21:43:39.394333Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 71, endOffset# 97, WTime# 1760046219388, sizeLag# 241322
2025-10-09T21:43:39.394335Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:43:39.396395Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.394347Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# 339f0db8-2b6f1efb-f14e95cc-61e11288, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 31, size# 289586, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.396399Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 279
2025-10-09T21:43:39.394347Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 98 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 10827 count 1 nextOffset 98 batches 1
2025-10-09T21:43:39.394354Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 99 partNo 0
2025-10-09T21:43:39.394358Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 31 maxSize 289586 maxTimeLagMs 0 readTimestampMs 0 readOffset 71 EndOffset 97 ClientCommitOffset 0 committedOffset 0 Guid 339f0db8-2b6f1efb-f14e95cc-61e11288
2025-10-09T21:43:39.394359Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 99 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 21732 count 2 nextOffset 99 batches 1
2025-10-09T21:43:39.394362Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 100 partNo 0
2025-10-09T21:43:39.394366Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 100 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 32747 count 3 nextOffset 100 batches 1
2025-10-09T21:43:39.394369Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 101 partNo 0
2025-10-09T21:43:39.394373Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 101 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 43872 count 4 nextOffset 101 batches 1
2025-10-09T21:43:39.394376Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 102 partNo 0
2025-10-09T21:43:39.394379Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 102 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 55107 count 5 nextOffset 102 batches 1
2025-10-09T21:43:39.394382Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 103 partNo 0
2025-10-09T21:43:39.394387Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 103 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 66452 count 6 nextOffset 103 batches 1
2025-10-09T21:43:39.394390Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 104 partNo 0
2025-10-09T21:43:39.394393Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 104 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 77907 count 7 nextOffset 104 batches 1
2025-10-09T21:43:39.394396Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 105 partNo 0
2025-10-09T21:43:39.394400Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 105 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 89472 count 8 nextOffset 105 batches 1
2025-10-09T21:43:39.394404Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 106 partNo 0
2025-10-09T21:43:39.394410Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 106 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 101147 count 9 nextOffset 106 batches 1
2025-10-09T21:43:39.394412Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 107 partNo 0
2025-10-09T21:43:39.394431Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 107 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 112932 count 10 nextOffset 107 batches 1
2025-10-09T21:43:39.394434Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 108 partNo 0
2025-10-09T21:43:39.394437Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 108 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 124827 count 11 nextOffset 108 batches 1
2025-10-09T21:43:39.394441Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 109 partNo 0
2025-10-09T21:43:39.394444Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 109 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 136832 count 12 nextOffset 109 batches 1
2025-10-09T21:43:39.394447Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 110 partNo 0
2025-10-09T21:43:39.394452Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 110 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 148947 count 13 nextOffset 110 batches 1
2025-10-09T21:43:39.394455Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 111 partNo 0
2025-10-09T21:43:39.394459Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 111 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 161172 count 14 nextOffset 111 batches 1
2025-10-09T21:43:39.394462Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 112 partNo 0
2025-10-09T21:43:39.394467Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 112 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 173507 count 15 nextOffset 112 batches 1
2025-10-09T21:43:39.394469Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 113 partNo 0
2025-10-09T21:43:39.394473Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 113 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 185952 count 16 nextOffset 113 batches 1
2025-10-09T21:43:39.394474Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.394476Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 114 partNo 0
2025-10-09T21:43:39.394481Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 114 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 198507 count 17 nextOffset 114 batches 1
2025-10-09T21:43:39.394484Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 115 partNo 0
2025-10-09T21:43:39.394487Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 115 partNo 0 FormedBlobsCount 0 NewHead: Offset 97 PartNo 0 PackedSize 211172 count 18 nextOffset 115 batches 1
2025-10-09T21:43:39.394490Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.394563Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.394654Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.394700Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.394729Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.394744Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 97,18 HeadOffset 97 endOffset 97 curOffset 115 d0000000000_00000000000000000097_00000_0000000018_00000? size 210658 WTime 1760046219394
2025-10-09T21:43:39.394787Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.394800Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.394821Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 230  requestId: <none> cookie: 116
2025-10-09T21:43:39.394833Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 147  requestId: <none> cookie: 72
2025-10-09T21:43:39.394840Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 148  requestId: <none> cookie: 73
2025-10-09T21:43:39.394847Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.394847Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 149  requestId: <none> cookie: 74
2025-10-09T21:43:39.394858Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 150  requestId: <none> cookie: 75
2025-10-09T21:43:39.394864Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.394865Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.394868Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.394873Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.394875Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.394876Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.394877Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.394882Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 151  requestId: <none> cookie: 76
2025-10-09T21:43:39.394889Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 157  requestId: <none> cookie: 77
2025-10-09T21:43:39.394891Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.394894Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 158  requestId: <none> cookie: 78
2025-10-09T21:43:39.394897Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.394900Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 159  requestId: <none> cookie: 79
2025-10-09T21:43:39.394905Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 160  requestId: <none> cookie: 80
2025-10-09T21:43:39.394907Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.394910Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.394910Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 161  requestId: <none> cookie: 81
2025-10-09T21:43:39.394915Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 167  requestId: <none> cookie: 82
2025-10-09T21:43:39.394922Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 168  requestId: <none> cookie: 83
2025-10-09T21:43:39.394927Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 169  requestId: <none> cookie: 84
2025-10-09T21:43:39.394933Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 170  requestId: <none> cookie: 85
2025-10-09T21:43:39.394938Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 171  requestId: <none> cookie: 86
2025-10-09T21:43:39.394943Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 177  requestId: <none> cookie: 87
2025-10-09T21:43:39.394948Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 178  requestId: <none> cookie: 88
2025-10-09T21:43:39.394953Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 179  requestId: <none> cookie: 89
2025-10-09T21:43:39.394958Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 180  requestId: <none> cookie: 90
2025-10-09T21:43:39.394963Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 181  requestId: <none> cookie: 91
2025-10-09T21:43:39.394967Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 187  requestId: <none> cookie: 92
2025-10-09T21:43:39.394972Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 188  requestId: <none> cookie: 93
2025-10-09T21:43:39.394978Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 189  requestId: <none> cookie: 94
2025-10-09T21:43:39.394983Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 190  requestId: <none> cookie: 95
2025-10-09T21:43:39.394988Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 191  requestId: <none> cookie: 96
2025-10-09T21:43:39.394992Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.394993Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 193  requestId: <none> cookie: 97
2025-10-09T21:43:39.394996Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.394997Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.394999Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395005Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395005Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395007Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395012Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395016Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395018Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395022Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395025Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395025Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.395028Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395032Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395035Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395036Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 97 partNo 0 count 18 size 210658
2025-10-09T21:43:39.395039Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395046Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395048Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395054Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395055Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 116 partNo : 0 messageNo: 235 size 12733 offset: -1
2025-10-09T21:43:39.395058Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395063Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395067Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395076Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395086Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395090Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395094Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395097Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395104Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 231  requestId: <none> cookie: 117
2025-10-09T21:43:39.395110Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 232  requestId: <none> cookie: 118
2025-10-09T21:43:39.395111Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395113Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.395119Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.395122Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 233  requestId: <none> cookie: 119
2025-10-09T21:43:39.395126Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 234  requestId: <none> cookie: 120
2025-10-09T21:43:39.395137Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395139Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395146Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395156Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395168Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 11 Topic 'rt3.dc1--test-topic' partition 0 user user offset 71 count 31 size 289586 endOffset 97 max time lag 0ms effective offset 71
2025-10-09T21:43:39.395172Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395176Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395183Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 11 added 1 blobs, size 241322 count 26 last offset 71, current partition end offset: 97
2025-10-09T21:43:39.395186Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 11. Send blob request.
2025-10-09T21:43:39.395196Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 71 partno 0 count 26 parts_count 0 source 1 size 241322 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.395201Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 11. All 1 blobs are from cache.
2025-10-09T21:43:39.395207Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 236  requestId: <none> cookie: 121
2025-10-09T21:43:39.395212Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.395236Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 26 parts 0 suffix '63'
2025-10-09T21:43:39.395265Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395272Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.395280Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.395285Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.395289Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.395297Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.395505Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 71 totakecount 26 count 26 size 241300 from pos 0 cbcount 26
2025-10-09T21:43:39.395543Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395545Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395554Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 117 partNo : 0 messageNo: 237 size 12843 offset: -1
2025-10-09T21:43:39.395558Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395560Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395564Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 118 partNo : 0 messageNo: 238 size 12953 offset: -1
2025-10-09T21:43:39.395567Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395569Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395573Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 119 partNo : 0 messageNo: 239 size 13063 offset: -1
2025-10-09T21:43:39.395576Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395578Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395581Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 120 partNo : 0 messageNo: 240 size 13173 offset: -1
2025-10-09T21:43:39.395572Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.395585Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395588Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395591Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 121 partNo : 0 messageNo: 241 size 13283 offset: -1
2025-10-09T21:43:39.395595Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395597Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395601Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395602Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395606Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395607Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395612Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395614Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395617Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395620Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395640Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 71
2025-10-09T21:43:39.395655Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.395660Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.395662Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.395664Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.395666Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.395668Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.395670Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.395672Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.395674Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.395676Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.395725Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 242  requestId: <none> cookie: 122
2025-10-09T21:43:39.395728Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 243  requestId: <none> cookie: 123
2025-10-09T21:43:39.395730Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 244  requestId: <none> cookie: 124
2025-10-09T21:43:39.395733Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 245  requestId: <none> cookie: 125
2025-10-09T21:43:39.395735Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 246  requestId: <none> cookie: 126
2025-10-09T21:43:39.395743Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.395750Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395756Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395760Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395764Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395768Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395825Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.395832Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.395838Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.395845Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395847Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395853Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 122 partNo : 0 messageNo: 247 size 13393 offset: -1
2025-10-09T21:43:39.395857Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395858Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395862Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 123 partNo : 0 messageNo: 248 size 13503 offset: -1
2025-10-09T21:43:39.395865Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395868Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395872Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 124 partNo : 0 messageNo: 249 size 13613 offset: -1
2025-10-09T21:43:39.395875Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395877Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395882Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 125 partNo : 0 messageNo: 250 size 13723 offset: -1
2025-10-09T21:43:39.395886Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395888Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395891Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 126 partNo : 0 messageNo: 251 size 13833 offset: -1
2025-10-09T21:43:39.395894Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395896Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395900Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395901Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395904Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.395907Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.395909Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.395912Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.395914Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.395916Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.395918Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.395920Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.395921Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.395923Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.395924Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.395926Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.395930Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 97 Result { Offset: 71 Data: "... 7893 bytes ..." SourceId: "\000src_id" SeqNo: 72 WriteTimestampMS: 1760046219390 CreateTimestampMS: 1760046219371 UncompressedSize: 7821 PartitionKey: "" ExplicitHash: "" } Result { Offset: 72 Data: "... 8003 bytes ..." SourceId: "\000src_id" SeqNo: 73 WriteTimestampMS: 1760046219390 CreateTimestampMS: 1760046219371 UncompressedSize: 7931 PartitionKey: "" ExplicitHash: "" } Result { Offset: 73 Data: "... 8113 bytes ..." SourceId: "\000src_id" SeqNo: 74 WriteTimestampMS: 1760046219390 CreateTimestampMS: 1760046219371 UncompressedSize: 8041 PartitionKey: "" ExplicitHash: "" } Result { Offset: 74 Data: "... 8223 bytes ..." SourceId: "\000src_id" SeqNo: 75 WriteTimestampMS: 1760046219390 CreateTimestampMS: 1760046219371 UncompressedSize: 8151 PartitionKey: "" ExplicitHash: "" } Result { Offset: 75 Data: "... 8333 bytes ..." SourceId: "\000src_id" SeqNo: 76 WriteTimestampMS: 1760046219390 CreateTimestampMS: 1760046219371 UncompressedSize: 8261 PartitionKey: "" ExplicitHash: "" } Result { Offset: 76 Data: "... 8443 bytes ..." SourceId: "\000src_id" SeqNo: 77 WriteTimestampMS: 1760046219390 CreateTimestampMS: 1760046219371 UncompressedSize: 8371 PartitionKey: "" ExplicitHash: "" } Result { Offset: 77 Data: "... 8553 bytes ..." SourceId: "\000src_id" SeqNo: 78 WriteTimestampMS: 1760046219390 CreateTimestampMS: 1760046219371 UncompressedSize: 8481 PartitionKey: "" ExplicitHash: "" } Result { Offset: 78 Data: "... 8663 bytes ..." SourceId: "\000src_id" SeqNo: 79 WriteTimestampMS: 1760046219390 CreateTimestampMS: 1760046219371 UncompressedSize: 8591 PartitionKey: "" ExplicitHash: "" } Result { Offset: 79 Data: "... 8773 bytes ..." SourceId: "\000src_id" SeqNo: 80 WriteTimestampMS: 1760046219390 CreateTimestampMS: 1760046219371 UncompressedSize: 8701 PartitionKey: "" ExplicitHash: "" } Result { Offset: 80 Data: "... 8883 bytes ..." SourceId: "\000src_id" SeqNo: 81 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219371 UncompressedSize: 8811 PartitionKey: "" ExplicitHash: "" } Result { Offset: 81 Data: "... 8993 bytes ..." SourceId: "\000src_id" SeqNo: 82 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219371 UncompressedSize: 8921 PartitionKey: "" ExplicitHash: "" } Result { Offset: 82 Data: "... 9103 bytes ..." SourceId: "\000src_id" SeqNo: 83 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219371 UncompressedSize: 9031 PartitionKey: "" ExplicitHash: "" } Result { Offset: 83 Data: "... 9213 bytes ..." SourceId: "\000src_id" SeqNo: 84 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219372 UncompressedSize: 9141 PartitionKey: "" ExplicitHash: "" } Result { Offset: 84 Data: "... 9323 bytes ..." SourceId: "\000src_id" SeqNo: 85 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219372 UncompressedSize: 9251 PartitionKey: "" ExplicitHash: "" } Result { Offset: 85 Data: "... 9433 bytes ..." SourceId: "\000src_id" SeqNo: 86 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219372 UncompressedSize: 9361 PartitionKey: "" ExplicitHash: "" } Result { Offset: 86 Data: "... 9543 bytes ..." SourceId: "\000src_id" SeqNo: 87 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219372 UncompressedSize: 9471 PartitionKey: "" ExplicitHash: "" } Result { Offset: 87 Data: "... 9653 bytes ..." SourceId: "\000src_id" SeqNo: 88 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219372 UncompressedSize: 9581 PartitionKey: "" ExplicitHash: "" } Result { Offset: 88 Data: "... 9763 bytes ..." SourceId: "\000src_id" SeqNo: 89 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219372 UncompressedSize: 9691 PartitionKey: "" ExplicitHash: "" } Result { Offset: 89 Data: "... 9873 bytes ..." SourceId: "\000src_id" SeqNo: 90 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219372 UncompressedSize: 9801 PartitionKey: "" ExplicitHash: "" } Result { Offset: 90 Data: "... 9983 bytes ..." SourceId: "\000src_id" SeqNo: 91 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219372 UncompressedSize: 9911 PartitionKey: "" ExplicitHash: "" } Result { Offset: 91 Data: "... 10093 bytes ..." SourceId: "\000src_id" SeqNo: 92 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219372 UncompressedSize: 10021 PartitionKey: "" ExplicitHash: "" } Result { Offset: 92 Data: "... 10203 bytes ..." SourceId: "\000src_id" SeqNo: 93 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219372 UncompressedSize: 10131 PartitionKey: "" ExplicitHash: "" } Result { Offset: 93 Data: "... 10313 bytes ..." SourceId: "\000src_id" SeqNo: 94 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219372 UncompressedSize: 10241 PartitionKey: "" ExplicitHash: "" } Result { Offset: 94 Data: "... 10423 bytes ..." SourceId: "\000src_id" SeqNo: 95 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219372 UncompressedSize: 10351 PartitionKey: "" ExplicitHash: "" } Result { Offset: 95 Data: "... 10533 bytes ..." SourceId: "\000src_id" SeqNo: 96 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219372 UncompressedSize: 10461 PartitionKey: "" ExplicitHash: "" } Result { Offset: 96 Data: "... 10643 bytes ..." SourceId: "\000src_id" SeqNo: 97 WriteTimestampMS: 1760046219391 CreateTimestampMS: 1760046219372 UncompressedSize: 10571 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 18446744073709550878 RealReadOffset: 96 WaitQuotaTimeMs: 0 EndOffset: 97 StartOffset: 0 } Cookie: 71 }
2025-10-09T21:43:39.395933Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 252  requestId: <none> cookie: 127
2025-10-09T21:43:39.395936Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 253  requestId: <none> cookie: 128
2025-10-09T21:43:39.395939Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 254  requestId: <none> cookie: 129
2025-10-09T21:43:39.395943Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.395947Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395950Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395954Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.395980Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.396055Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 9 from offset 97
2025-10-09T21:43:39.397225Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 240258, now ReadSizeBudget = 0, ReadSizeServerDelta = 284030
2025-10-09T21:43:39.396071Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 97 ReadOffset 97 ReadGuid 339f0db8-2b6f1efb-f14e95cc-61e11288 has messages 1
2025-10-09T21:43:39.396102Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.396106Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.396117Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 127 partNo : 0 messageNo: 255 size 13943 offset: -1
2025-10-09T21:43:39.396124Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.396126Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.396130Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 128 partNo : 0 messageNo: 256 size 14054 offset: -1
2025-10-09T21:43:39.396133Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.397272Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 0, ReadSizeServerDelta = 284030
2025-10-09T21:43:39.396135Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.396146Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 129 partNo : 0 messageNo: 257 size 14164 offset: -1
2025-10-09T21:43:39.396153Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.396157Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.396176Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.396183Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.396186Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.396189Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.396191Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.396193Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.396298Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.396344Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 258  requestId: <none> cookie: 130
2025-10-09T21:43:39.396366Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# 339f0db8-2b6f1efb-f14e95cc-61e11288, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 240258
2025-10-09T21:43:39.396377Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# 339f0db8-2b6f1efb-f14e95cc-61e11288
2025-10-09T21:43:39.396398Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.396410Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.396417Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.396431Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.396526Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.396607Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.396671Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 0
2025-10-09T21:43:39.396680Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.396698Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.396715Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.396730Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.396740Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 130 partNo : 0 messageNo: 259 size 14274 offset: -1
2025-10-09T21:43:39.396751Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.396755Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.396760Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.396761Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.396782Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.397355Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (71-96)
2025-10-09T21:43:39.397378Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 240258 to budget
2025-10-09T21:43:39.397385Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 240258, ReadSizeServerDelta = 284030
2025-10-09T21:43:39.396792Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.396796Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.396799Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.396803Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.396805Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.396816Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 260  requestId: <none> cookie: 131
2025-10-09T21:43:39.396821Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 261  requestId: <none> cookie: 132
2025-10-09T21:43:39.396824Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 262  requestId: <none> cookie: 133
2025-10-09T21:43:39.396827Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 263  requestId: <none> cookie: 134
2025-10-09T21:43:39.396836Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.396842Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397502Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.396847Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.396850Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.396855Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.396942Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.396949Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.396951Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.396959Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 131 partNo : 0 messageNo: 264 size 14384 offset: -1
2025-10-09T21:43:39.396964Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.396966Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.397529Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (71-71)
2025-10-09T21:43:39.396969Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 132 partNo : 0 messageNo: 265 size 14494 offset: -1
2025-10-09T21:43:39.396972Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.396974Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.396978Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 133 partNo : 0 messageNo: 266 size 14604 offset: -1
2025-10-09T21:43:39.397540Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (72-72)
2025-10-09T21:43:39.396981Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.396982Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.397548Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 2} (73-73)
2025-10-09T21:43:39.396987Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 134 partNo : 0 messageNo: 267 size 14714 offset: -1
2025-10-09T21:43:39.397552Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 3} (74-74)
2025-10-09T21:43:39.397004Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.397006Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.397009Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.397013Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.397015Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.397018Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.397020Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.397022Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.397574Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 4} (75-75)
2025-10-09T21:43:39.397062Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.397064Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.397579Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 5} (76-76)
2025-10-09T21:43:39.397075Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 97 count 18 size 210658 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.397583Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 6} (77-77)
2025-10-09T21:43:39.397596Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 7} (78-78)
2025-10-09T21:43:39.397076Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.397617Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 8} (79-79)
2025-10-09T21:43:39.397622Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 0} (80-80)
2025-10-09T21:43:39.397626Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 1} (81-81)
2025-10-09T21:43:39.397088Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 268  requestId: <none> cookie: 135
2025-10-09T21:43:39.397630Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 2} (82-82)
2025-10-09T21:43:39.397092Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.397635Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 3} (83-83)
2025-10-09T21:43:39.397102Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 97 partno 0 count 18 parts 0 suffix '63' size 210658
2025-10-09T21:43:39.397641Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 4} (84-84)
2025-10-09T21:43:39.397105Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 210510 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.397112Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397117Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 98, partNo: 0, Offset: 97 is stored on disk
2025-10-09T21:43:39.396438Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 280 to 280
2025-10-09T21:43:39.397120Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397124Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 99, partNo: 0, Offset: 98 is stored on disk
2025-10-09T21:43:39.397126Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397128Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 100, partNo: 0, Offset: 99 is stored on disk
2025-10-09T21:43:39.397132Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397670Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 5} (85-85)
2025-10-09T21:43:39.397689Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 6} (86-86)
2025-10-09T21:43:39.397134Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 101, partNo: 0, Offset: 100 is stored on disk
2025-10-09T21:43:39.397692Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.397698Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 280
2025-10-09T21:43:39.397136Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397139Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 102, partNo: 0, Offset: 101 is stored on disk
2025-10-09T21:43:39.397141Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397144Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 103, partNo: 0, Offset: 102 is stored on disk
2025-10-09T21:43:39.397146Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397149Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 104, partNo: 0, Offset: 103 is stored on disk
2025-10-09T21:43:39.397150Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397151Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397155Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 105, partNo: 0, Offset: 104 is stored on disk
2025-10-09T21:43:39.397157Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397160Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 106, partNo: 0, Offset: 105 is stored on disk
2025-10-09T21:43:39.397160Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.397162Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397165Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 107, partNo: 0, Offset: 106 is stored on disk
2025-10-09T21:43:39.397167Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397170Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 108, partNo: 0, Offset: 107 is stored on disk
2025-10-09T21:43:39.397172Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397174Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.397175Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 109, partNo: 0, Offset: 108 is stored on disk
2025-10-09T21:43:39.397177Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397181Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 110, partNo: 0, Offset: 109 is stored on disk
2025-10-09T21:43:39.397183Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397186Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 111, partNo: 0, Offset: 110 is stored on disk
2025-10-09T21:43:39.397188Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397191Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 112, partNo: 0, Offset: 111 is stored on disk
2025-10-09T21:43:39.397193Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397196Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 113, partNo: 0, Offset: 112 is stored on disk
2025-10-09T21:43:39.397198Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.397198Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397201Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 114, partNo: 0, Offset: 113 is stored on disk
2025-10-09T21:43:39.397203Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.397812Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 7} (87-87)
2025-10-09T21:43:39.397819Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 8} (88-88)
2025-10-09T21:43:39.397205Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 115, partNo: 0, Offset: 114 is stored on disk
2025-10-09T21:43:39.397824Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 9} (89-89)
2025-10-09T21:43:39.397241Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.397257Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 97 EndOffset 97 newEndOffset 115 commitOffset 0 clientCommitOffset 0 cookie 9 readingFinished 0 firstRead 0
2025-10-09T21:43:39.397841Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 10} (90-90)
2025-10-09T21:43:39.397261Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 97 endOffset 115
2025-10-09T21:43:39.397261Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.397269Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (19)
2025-10-09T21:43:39.397290Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 116 partNo 0
2025-10-09T21:43:39.397291Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 97, endOffset# 115, WTime# 1760046219391, sizeLag# 210658
2025-10-09T21:43:39.397294Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:43:39.397307Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# 470d14e-8f7ec245-8f6bbd26-d05b4353, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 21, size# 252789, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.397313Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 116 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 12807 count 1 nextOffset 116 batches 1
2025-10-09T21:43:39.397320Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 117 partNo 0
2025-10-09T21:43:39.397320Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 21 maxSize 252789 maxTimeLagMs 0 readTimestampMs 0 readOffset 97 EndOffset 115 ClientCommitOffset 0 committedOffset 0 Guid 470d14e-8f7ec245-8f6bbd26-d05b4353
2025-10-09T21:43:39.397324Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 117 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 25692 count 2 nextOffset 117 batches 1
2025-10-09T21:43:39.397873Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 11} (91-91)
2025-10-09T21:43:39.397328Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 118 partNo 0
2025-10-09T21:43:39.397332Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 118 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 38687 count 3 nextOffset 118 batches 1
2025-10-09T21:43:39.397335Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 119 partNo 0
2025-10-09T21:43:39.397884Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 12} (92-92)
2025-10-09T21:43:39.397339Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 119 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 51792 count 4 nextOffset 119 batches 1
2025-10-09T21:43:39.397900Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 13} (93-93)
2025-10-09T21:43:39.397342Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 120 partNo 0
2025-10-09T21:43:39.397345Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 120 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 65007 count 5 nextOffset 120 batches 1
2025-10-09T21:43:39.397912Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 14} (94-94)
2025-10-09T21:43:39.397348Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 121 partNo 0
2025-10-09T21:43:39.397352Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 121 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 78332 count 6 nextOffset 121 batches 1
2025-10-09T21:43:39.397355Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 122 partNo 0
2025-10-09T21:43:39.397939Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 15} (95-95)
2025-10-09T21:43:39.397358Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 122 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 91767 count 7 nextOffset 122 batches 1
2025-10-09T21:43:39.397363Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 123 partNo 0
2025-10-09T21:43:39.397366Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 123 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 105312 count 8 nextOffset 123 batches 1
2025-10-09T21:43:39.397369Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 124 partNo 0
2025-10-09T21:43:39.397949Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 16} (96-96)
2025-10-09T21:43:39.397375Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 124 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 118967 count 9 nextOffset 124 batches 1
2025-10-09T21:43:39.397379Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 125 partNo 0
2025-10-09T21:43:39.397383Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 125 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 132732 count 10 nextOffset 125 batches 1
2025-10-09T21:43:39.397386Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 126 partNo 0
2025-10-09T21:43:39.397389Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 126 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 146607 count 11 nextOffset 126 batches 1
2025-10-09T21:43:39.397392Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 127 partNo 0
2025-10-09T21:43:39.397396Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 127 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 160592 count 12 nextOffset 127 batches 1
2025-10-09T21:43:39.397399Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 128 partNo 0
2025-10-09T21:43:39.397402Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 128 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 174688 count 13 nextOffset 128 batches 1
2025-10-09T21:43:39.397405Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 129 partNo 0
2025-10-09T21:43:39.397409Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 129 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 188894 count 14 nextOffset 129 batches 1
2025-10-09T21:43:39.397412Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 130 partNo 0
2025-10-09T21:43:39.397415Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 130 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 203210 count 15 nextOffset 130 batches 1
2025-10-09T21:43:39.397418Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 131 partNo 0
2025-10-09T21:43:39.397422Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 131 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 217636 count 16 nextOffset 131 batches 1
2025-10-09T21:43:39.397425Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 132 partNo 0
2025-10-09T21:43:39.397430Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 132 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 232172 count 17 nextOffset 132 batches 1
2025-10-09T21:43:39.397433Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 133 partNo 0
2025-10-09T21:43:39.397436Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 133 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 246818 count 18 nextOffset 133 batches 1
2025-10-09T21:43:39.397439Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 134 partNo 0
2025-10-09T21:43:39.397443Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 134 partNo 0 FormedBlobsCount 0 NewHead: Offset 115 PartNo 0 PackedSize 261574 count 19 nextOffset 134 batches 1
2025-10-09T21:43:39.397445Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.397478Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.397571Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.397623Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.397632Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.397644Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.397666Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 115,19 HeadOffset 115 endOffset 115 curOffset 134 d0000000000_00000000000000000115_00000_0000000019_00000? size 261028 WTime 1760046219397
2025-10-09T21:43:39.397730Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.397746Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 199  requestId: <none> cookie: 98
2025-10-09T21:43:39.397755Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 200  requestId: <none> cookie: 99
2025-10-09T21:43:39.397760Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 201  requestId: <none> cookie: 100
2025-10-09T21:43:39.397766Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 202  requestId: <none> cookie: 101
2025-10-09T21:43:39.397769Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.397772Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 204  requestId: <none> cookie: 102
2025-10-09T21:43:39.397775Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397778Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 209  requestId: <none> cookie: 103
2025-10-09T21:43:39.397782Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397782Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 210  requestId: <none> cookie: 104
2025-10-09T21:43:39.397788Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 212  requestId: <none> cookie: 105
2025-10-09T21:43:39.397788Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397793Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397793Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 213  requestId: <none> cookie: 106
2025-10-09T21:43:39.397798Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 214  requestId: <none> cookie: 107
2025-10-09T21:43:39.397802Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 219  requestId: <none> cookie: 108
2025-10-09T21:43:39.397806Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 220  requestId: <none> cookie: 109
2025-10-09T21:43:39.397811Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 221  requestId: <none> cookie: 110
2025-10-09T21:43:39.397815Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.397818Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.397827Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 135 partNo : 0 messageNo: 269 size 14824 offset: -1
2025-10-09T21:43:39.397834Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 222  requestId: <none> cookie: 111
2025-10-09T21:43:39.397839Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 223  requestId: <none> cookie: 112
2025-10-09T21:43:39.397843Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 226  requestId: <none> cookie: 113
2025-10-09T21:43:39.397847Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397847Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.397849Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.397854Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 228  requestId: <none> cookie: 114
2025-10-09T21:43:39.397854Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397857Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.397858Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397858Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.397863Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 229  requestId: <none> cookie: 115
2025-10-09T21:43:39.397864Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.397873Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397874Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.397876Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.397878Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397883Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397883Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.397885Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.397887Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397890Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.397891Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397894Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397898Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397900Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 115 partNo 0 count 19 size 261028
2025-10-09T21:43:39.397902Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397906Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.397907Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.397908Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397911Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.397918Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.397956Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 270  requestId: <none> cookie: 136
2025-10-09T21:43:39.397959Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 271  requestId: <none> cookie: 137
2025-10-09T21:43:39.397961Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397967Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 272  requestId: <none> cookie: 138
2025-10-09T21:43:39.397976Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.397988Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.397988Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 273  requestId: <none> cookie: 139
2025-10-09T21:43:39.397995Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 12 Topic 'rt3.dc1--test-topic' partition 0 user user offset 97 count 21 size 252789 endOffset 115 max time lag 0ms effective offset 97
2025-10-09T21:43:39.398008Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 12 added 1 blobs, size 210658 count 18 last offset 97, current partition end offset: 115
2025-10-09T21:43:39.398011Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 12. Send blob request.
2025-10-09T21:43:39.398013Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.398017Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.398018Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 97 partno 0 count 18 parts_count 0 source 1 size 210658 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.398022Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 12. All 1 blobs are from cache.
2025-10-09T21:43:39.398047Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.398052Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.398076Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.398082Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 97 partno 0 count 18 parts 0 suffix '63'
>>> event from dataHandler: 2025-10-09T21:43:39.398342Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 281 to 281
2025-10-09T21:43:39.398357Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.398362Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 281
2025-10-09T21:43:39.398371Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 282 to 282
2025-10-09T21:43:39.398378Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.398380Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 282
2025-10-09T21:43:39.398352Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 97 totakecount 18 count 18 size 210636 from pos 0 cbcount 18
2025-10-09T21:43:39.398388Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 283 to 283
2025-10-09T21:43:39.398396Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.398398Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 283
2025-10-09T21:43:39.398409Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 284 to 284
2025-10-09T21:43:39.398428Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.398431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 284
2025-10-09T21:43:39.398440Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 285 to 285
DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..7821 bytes.. Information: { Offset: 71 ProducerId: "src_id" SeqNo: 72 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.390000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..7931 bytes.. Information: { Offset: 72 ProducerId: "src_id" SeqNo: 73 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.390000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..8041 bytes.. Information: { Offset: 73 ProducerId: "src_id" SeqNo: 74 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.390000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..8151 bytes.. Information: { Offset: 74 ProducerId: "src_id" SeqNo: 75 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.390000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..8261 bytes.. Information: { Offset: 75 ProducerId: "src_id" SeqNo: 76 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.390000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..8371 bytes.. Information: { Offset: 76 ProducerId: "src_id" SeqNo: 77 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.390000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..8481 bytes.. Information: { Offset: 77 ProducerId: "src_id" SeqNo: 78 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.390000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..8591 bytes.. Information: { Offset: 78 ProducerId: "src_id" SeqNo: 79 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.390000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..8701 bytes.. Information: { Offset: 79 ProducerId: "src_id" SeqNo: 80 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.390000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..8811 bytes.. Information: { Offset: 80 ProducerId: "src_id" SeqNo: 81 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..8921 bytes.. Information: { Offset: 81 ProducerId: "src_id" SeqNo: 82 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..9031 bytes.. Information: { Offset: 82 ProducerId: "src_id" SeqNo: 83 CreateTime: 2025-10-09T21:43:39.371000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..9141 bytes.. Information: { Offset: 83 ProducerId: "src_id" SeqNo: 84 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..9251 bytes.. Information: { Offset: 84 ProducerId: "src_id" SeqNo: 85 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..9361 bytes.. Information: { Offset: 85 ProducerId: "src_id" SeqNo: 86 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..9471 bytes.. Information: { Offset: 86 ProducerId: "src_id" SeqNo: 87 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..9581 bytes.. Information: { Offset: 87 ProducerId: "src_id" SeqNo: 88 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..9691 bytes.. Information: { Offset: 88 ProducerId: "src_id" SeqNo: 89 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..9801 bytes.. Information: { Offset: 89 ProducerId: "src_id" SeqNo: 90 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..9911 bytes.. Information: { Offset: 90 ProducerId: "src_id" SeqNo: 91 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..10021 bytes.. Information: { Offset: 91 ProducerId: "src_id" SeqNo: 92 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..10131 bytes.. Information: { Offset: 92 ProducerId: "src_id" SeqNo: 93 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..10241 bytes.. Information: { Offset: 93 ProducerId: "src_id" SeqNo: 94 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..10351 bytes.. Information: { Offset: 94 ProducerId: "src_id" SeqNo: 95 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..10461 bytes.. Information: { Offset: 95 ProducerId: "src_id" SeqNo: 96 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..10571 bytes.. Information: { Offset: 96 ProducerId: "src_id" SeqNo: 97 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.391000Z MessageGroupId: "src_id" Meta: { "server": "ipv6:[::1]:42738", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.398395Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.398405Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.398433Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 97
2025-10-09T21:43:39.398449Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.398450Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.398453Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.398463Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 136 partNo : 0 messageNo: 275 size 14934 offset: -1
2025-10-09T21:43:39.398466Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.398468Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.398471Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.398472Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 137 partNo : 0 messageNo: 276 size 15044 offset: -1
2025-10-09T21:43:39.398483Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.398503Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.398504Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.398508Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 138 partNo : 0 messageNo: 277 size 15154 offset: -1
2025-10-09T21:43:39.398511Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.398513Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.398516Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 139 partNo : 0 messageNo: 278 size 15264 offset: -1
2025-10-09T21:43:39.398520Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 274  requestId: <none> cookie: 140
2025-10-09T21:43:39.398523Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.398529Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.398545Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 26, size 239096 bytes
2025-10-09T21:43:39.398531Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.398534Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.398535Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.398538Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.398556Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.398539Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.398448Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.398541Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.398591Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 285
2025-10-09T21:43:39.398553Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.398556Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 240258 } }
2025-10-09T21:43:39.398561Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.398570Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# e6e8768e-5f327628-e5daf3df-78d17f2
2025-10-09T21:43:39.398642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 286 to 286
2025-10-09T21:43:39.398650Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.398653Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 286
2025-10-09T21:43:39.398670Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 287 to 287
2025-10-09T21:43:39.398674Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.398677Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 287
2025-10-09T21:43:39.398823Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 288 to 288
2025-10-09T21:43:39.398838Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.398843Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 288
2025-10-09T21:43:39.398852Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 289 to 289
2025-10-09T21:43:39.398860Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.398862Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 289
2025-10-09T21:43:39.398870Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 290 to 290
2025-10-09T21:43:39.398878Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.398880Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 290
2025-10-09T21:43:39.398902Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 291 to 291
2025-10-09T21:43:39.398910Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.398912Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 291
2025-10-09T21:43:39.398921Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 292 to 292
2025-10-09T21:43:39.398929Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.398931Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 292
2025-10-09T21:43:39.398968Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 293 to 293
2025-10-09T21:43:39.398998Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.399001Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 293
2025-10-09T21:43:39.399042Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 294 to 294
2025-10-09T21:43:39.398693Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 115 Result { Offset: 97 Data: "... 10753 bytes ..." SourceId: "\000src_id" SeqNo: 98 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219372 UncompressedSize: 10681 PartitionKey: "" ExplicitHash: "" } Result { Offset: 98 Data: "... 10863 bytes ..." SourceId: "\000src_id" SeqNo: 99 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219372 UncompressedSize: 10791 PartitionKey: "" ExplicitHash: "" } Result { Offset: 99 Data: "... 10973 bytes ..." SourceId: "\000src_id" SeqNo: 100 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219372 UncompressedSize: 10901 PartitionKey: "" ExplicitHash: "" } Result { Offset: 100 Data: "... 11083 bytes ..." SourceId: "\000src_id" SeqNo: 101 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219372 UncompressedSize: 11011 PartitionKey: "" ExplicitHash: "" } Result { Offset: 101 Data: "... 11193 bytes ..." SourceId: "\000src_id" SeqNo: 102 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219372 UncompressedSize: 11121 PartitionKey: "" ExplicitHash: "" } Result { Offset: 102 Data: "... 11303 bytes ..." SourceId: "\000src_id" SeqNo: 103 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219372 UncompressedSize: 11231 PartitionKey: "" ExplicitHash: "" } Result { Offset: 103 Data: "... 11413 bytes ..." SourceId: "\000src_id" SeqNo: 104 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219372 UncompressedSize: 11341 PartitionKey: "" ExplicitHash: "" } Result { Offset: 104 Data: "... 11523 bytes ..." SourceId: "\000src_id" SeqNo: 105 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219372 UncompressedSize: 11451 PartitionKey: "" ExplicitHash: "" } Result { Offset: 105 Data: "... 11633 bytes ..." SourceId: "\000src_id" SeqNo: 106 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219372 UncompressedSize: 11561 PartitionKey: "" ExplicitHash: "" } Result { Offset: 106 Data: "... 11743 bytes ..." SourceId: "\000src_id" SeqNo: 107 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219372 UncompressedSize: 11671 PartitionKey: "" ExplicitHash: "" } Result { Offset: 107 Data: "... 11853 bytes ..." SourceId: "\000src_id" SeqNo: 108 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219372 UncompressedSize: 11781 PartitionKey: "" ExplicitHash: "" } Result { Offset: 108 Data: "... 11963 bytes ..." SourceId: "\000src_id" SeqNo: 109 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219372 UncompressedSize: 11891 PartitionKey: "" ExplicitHash: "" } Result { Offset: 109 Data: "... 12073 bytes ..." SourceId: "\000src_id" SeqNo: 110 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219372 UncompressedSize: 12001 PartitionKey: "" ExplicitHash: "" } Result { Offset: 110 Data: "... 12183 bytes ..." SourceId: "\000src_id" SeqNo: 111 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219372 UncompressedSize: 12111 PartitionKey: "" ExplicitHash: "" } Result { Offset: 111 Data: "... 12293 bytes ..." SourceId: "\000src_id" SeqNo: 112 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219373 UncompressedSize: 12221 PartitionKey: "" ExplicitHash: "" } Result { Offset: 112 Data: "... 12403 bytes ..." SourceId: "\000src_id" SeqNo: 113 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219373 UncompressedSize: 12331 PartitionKey: "" ExplicitHash: "" } Result { Offset: 113 Data: "... 12513 bytes ..." SourceId: "\000src_id" SeqNo: 114 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219373 UncompressedSize: 12441 PartitionKey: "" ExplicitHash: "" } Result { Offset: 114 Data: "... 12623 bytes ..." SourceId: "\000src_id" SeqNo: 115 WriteTimestampMS: 1760046219394 CreateTimestampMS: 1760046219373 UncompressedSize: 12551 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 18446744073709551134 RealReadOffset: 114 WaitQuotaTimeMs: 0 EndOffset: 115 StartOffset: 0 } Cookie: 97 }
2025-10-09T21:43:39.398826Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 10 from offset 115
2025-10-09T21:43:39.398833Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 115 ReadOffset 115 ReadGuid 470d14e-8f7ec245-8f6bbd26-d05b4353 has messages 1
2025-10-09T21:43:39.399069Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.399104Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 294
2025-10-09T21:43:39.399151Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 295 to 295
2025-10-09T21:43:39.399140Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.399151Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.399153Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.399159Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.399162Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.399166Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.399168Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.399189Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.399192Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 295
2025-10-09T21:43:39.399179Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 140 partNo : 0 messageNo: 281 size 15374 offset: -1
2025-10-09T21:43:39.399185Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.399187Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.399198Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.399225Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 296 to 296
2025-10-09T21:43:39.399208Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.399221Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.399223Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# 470d14e-8f7ec245-8f6bbd26-d05b4353, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 209851
2025-10-09T21:43:39.399232Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 279  requestId: <none> cookie: 141
2025-10-09T21:43:39.399232Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# 470d14e-8f7ec245-8f6bbd26-d05b4353
2025-10-09T21:43:39.399237Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 280  requestId: <none> cookie: 142
2025-10-09T21:43:39.399255Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.399258Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 296
2025-10-09T21:43:39.399292Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 0
2025-10-09T21:43:39.399321Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.399333Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.399339Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 282  requestId: <none> cookie: 143
2025-10-09T21:43:39.399350Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.399350Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.399352Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.399357Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 283  requestId: <none> cookie: 144
2025-10-09T21:43:39.399362Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.399370Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.399372Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.399378Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 141 partNo : 0 messageNo: 284 size 15484 offset: -1
2025-10-09T21:43:39.399383Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.399384Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.399388Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 142 partNo : 0 messageNo: 285 size 15594 offset: -1
2025-10-09T21:43:39.399392Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.399400Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.399404Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 143 partNo : 0 messageNo: 286 size 15704 offset: -1
2025-10-09T21:43:39.399408Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.399411Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.399414Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 144 partNo : 0 messageNo: 287 size 15814 offset: -1
2025-10-09T21:43:39.399417Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.399423Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.399424Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.399427Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.399429Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.399431Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.399433Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.399436Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.399596Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 297 to 297
2025-10-09T21:43:39.399611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.399615Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 297
2025-10-09T21:43:39.399626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 298 to 298
2025-10-09T21:43:39.399634Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.399636Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 298
2025-10-09T21:43:39.399758Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 299 to 299
2025-10-09T21:43:39.399811Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.399814Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 299
2025-10-09T21:43:39.399919Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write 1 messages with Id from 300 to 300
2025-10-09T21:43:39.399927Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: try to update token
2025-10-09T21:43:39.399931Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Send 1 message(s) (0 left), first sequence number is 300
2025-10-09T21:43:39.399882Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 115 count 19 size 261028 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.400241Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 28 offsets: 27 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.399904Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.399916Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 260877 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.399922Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.400366Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 28
2025-10-09T21:43:39.399924Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 115 partno 0 count 19 parts 0 suffix '63' size 261028
2025-10-09T21:43:39.399928Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 116, partNo: 0, Offset: 115 is stored on disk
2025-10-09T21:43:39.399933Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.399946Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 117, partNo: 0, Offset: 116 is stored on disk
2025-10-09T21:43:39.399949Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.399951Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 118, partNo: 0, Offset: 117 is stored on disk
2025-10-09T21:43:39.399954Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.399958Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 119, partNo: 0, Offset: 118 is stored on disk
2025-10-09T21:43:39.399960Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.399963Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 120, partNo: 0, Offset: 119 is stored on disk
2025-10-09T21:43:39.399965Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.399968Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 121, partNo: 0, Offset: 120 is stored on disk
2025-10-09T21:43:39.399970Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.399973Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 122, partNo: 0, Offset: 121 is stored on disk
2025-10-09T21:43:39.399975Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.399978Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 123, partNo: 0, Offset: 122 is stored on disk
2025-10-09T21:43:39.399980Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.400428Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 29 offsets: 28 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.399983Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 124, partNo: 0, Offset: 123 is stored on disk
2025-10-09T21:43:39.400434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 29
2025-10-09T21:43:39.400478Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 30 offsets: 29 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.400482Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 30
2025-10-09T21:43:39.399984Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.400500Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 209851, now ReadSizeBudget = 0, ReadSizeServerDelta = 314437
2025-10-09T21:43:39.400501Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 31 offsets: 30 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.399989Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 125, partNo: 0, Offset: 124 is stored on disk
2025-10-09T21:43:39.400521Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 31
2025-10-09T21:43:39.399991Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.399994Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 126, partNo: 0, Offset: 125 is stored on disk
2025-10-09T21:43:39.399997Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.400002Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 127, partNo: 0, Offset: 126 is stored on disk
2025-10-09T21:43:39.400004Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.400537Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 0, ReadSizeServerDelta = 314437
2025-10-09T21:43:39.400008Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 128, partNo: 0, Offset: 127 is stored on disk
2025-10-09T21:43:39.400010Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.400014Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 129, partNo: 0, Offset: 128 is stored on disk
2025-10-09T21:43:39.400016Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.400550Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 32 offsets: 31 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.400019Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 130, partNo: 0, Offset: 129 is stored on disk
2025-10-09T21:43:39.400021Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.400555Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 32
2025-10-09T21:43:39.400025Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 131, partNo: 0, Offset: 130 is stored on disk
2025-10-09T21:43:39.400027Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.400031Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 132, partNo: 0, Offset: 131 is stored on disk
2025-10-09T21:43:39.400033Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.400037Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 133, partNo: 0, Offset: 132 is stored on disk
2025-10-09T21:43:39.400039Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.400042Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 134, partNo: 0, Offset: 133 is stored on disk
2025-10-09T21:43:39.400076Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.400577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 33 offsets: 32 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.400087Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.400582Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 33
2025-10-09T21:43:39.400089Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (10)
2025-10-09T21:43:39.400106Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 135 partNo 0
2025-10-09T21:43:39.400587Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (97-114)
2025-10-09T21:43:39.400144Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 135 partNo 0 FormedBlobsCount 0 NewHead: Offset 134 PartNo 0 PackedSize 14898 count 1 nextOffset 135 batches 1
2025-10-09T21:43:39.400604Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 34 offsets: 33 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.400613Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 34
2025-10-09T21:43:39.400151Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 136 partNo 0
2025-10-09T21:43:39.400604Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 209851 to budget
2025-10-09T21:43:39.400635Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 209851, ReadSizeServerDelta = 314437
2025-10-09T21:43:39.400637Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 35 offsets: 34 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.400641Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 35
2025-10-09T21:43:39.400156Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 136 partNo 0 FormedBlobsCount 0 NewHead: Offset 134 PartNo 0 PackedSize 29874 count 2 nextOffset 136 batches 1
2025-10-09T21:43:39.400159Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 137 partNo 0
2025-10-09T21:43:39.400163Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 137 partNo 0 FormedBlobsCount 0 NewHead: Offset 134 PartNo 0 PackedSize 44960 count 3 nextOffset 137 batches 1
2025-10-09T21:43:39.400166Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 138 partNo 0
2025-10-09T21:43:39.400170Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 138 partNo 0 FormedBlobsCount 0 NewHead: Offset 134 PartNo 0 PackedSize 60156 count 4 nextOffset 138 batches 1
2025-10-09T21:43:39.400173Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 139 partNo 0
2025-10-09T21:43:39.400177Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 139 partNo 0 FormedBlobsCount 0 NewHead: Offset 134 PartNo 0 PackedSize 75462 count 5 nextOffset 139 batches 1
2025-10-09T21:43:39.400180Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 140 partNo 0
2025-10-09T21:43:39.400185Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 140 partNo 0 FormedBlobsCount 0 NewHead: Offset 134 PartNo 0 PackedSize 90878 count 6 nextOffset 140 batches 1
2025-10-09T21:43:39.400188Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 141 partNo 0
2025-10-09T21:43:39.400192Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 141 partNo 0 FormedBlobsCount 0 NewHead: Offset 134 PartNo 0 PackedSize 106404 count 7 nextOffset 141 batches 1
2025-10-09T21:43:39.400195Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 142 partNo 0
2025-10-09T21:43:39.400198Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 142 partNo 0 FormedBlobsCount 0 NewHead: Offset 134 PartNo 0 PackedSize 122040 count 8 nextOffset 142 batches 1
2025-10-09T21:43:39.400201Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 143 partNo 0
2025-10-09T21:43:39.400206Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 143 partNo 0 FormedBlobsCount 0 NewHead: Offset 134 PartNo 0 PackedSize 137786 count 9 nextOffset 143 batches 1
2025-10-09T21:43:39.400208Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 144 partNo 0
2025-10-09T21:43:39.400212Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 144 partNo 0 FormedBlobsCount 0 NewHead: Offset 134 PartNo 0 PackedSize 153642 count 10 nextOffset 144 batches 1
2025-10-09T21:43:39.400214Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.400404Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 134,10 HeadOffset 134 endOffset 134 curOffset 144 d0000000000_00000000000000000134_00000_0000000010_00000? size 153386 WTime 1760046219400
2025-10-09T21:43:39.400535Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.400548Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 235  requestId: <none> cookie: 116
2025-10-09T21:43:39.400710Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.400665Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 36 offsets: 35 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.400556Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 237  requestId: <none> cookie: 117
2025-10-09T21:43:39.400749Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (97-97)
2025-10-09T21:43:39.400561Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 238  requestId: <none> cookie: 118
2025-10-09T21:43:39.400748Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 36
2025-10-09T21:43:39.400763Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (98-98)
2025-10-09T21:43:39.400565Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 239  requestId: <none> cookie: 119
2025-10-09T21:43:39.400785Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 2} (99-99)
2025-10-09T21:43:39.400570Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 240  requestId: <none> cookie: 120
2025-10-09T21:43:39.400790Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 3} (100-100)
2025-10-09T21:43:39.400574Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 241  requestId: <none> cookie: 121
2025-10-09T21:43:39.400806Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 4} (101-101)
2025-10-09T21:43:39.400811Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 5} (102-102)
2025-10-09T21:43:39.400802Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 37 offsets: 36 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.400815Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 6} (103-103)
2025-10-09T21:43:39.400579Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 247  requestId: <none> cookie: 122
2025-10-09T21:43:39.400583Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 248  requestId: <none> cookie: 123
2025-10-09T21:43:39.400824Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 37
2025-10-09T21:43:39.400589Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 249  requestId: <none> cookie: 124
2025-10-09T21:43:39.400593Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 250  requestId: <none> cookie: 125
2025-10-09T21:43:39.400598Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 251  requestId: <none> cookie: 126
2025-10-09T21:43:39.400602Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 255  requestId: <none> cookie: 127
2025-10-09T21:43:39.400606Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 256  requestId: <none> cookie: 128
2025-10-09T21:43:39.400610Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 257  requestId: <none> cookie: 129
2025-10-09T21:43:39.400853Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 7} (104-104)
2025-10-09T21:43:39.400614Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 259  requestId: <none> cookie: 130
2025-10-09T21:43:39.400618Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 264  requestId: <none> cookie: 131
2025-10-09T21:43:39.400622Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 265  requestId: <none> cookie: 132
2025-10-09T21:43:39.400626Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 266  requestId: <none> cookie: 133
2025-10-09T21:43:39.400868Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 38 offsets: 37 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.400629Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 267  requestId: <none> cookie: 134
2025-10-09T21:43:39.400873Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 38
2025-10-09T21:43:39.400634Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.400643Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 134 partNo 0 count 10 size 153386
2025-10-09T21:43:39.400699Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 115 EndOffset 115 newEndOffset 134 commitOffset 0 clientCommitOffset 0 cookie 10 readingFinished 0 firstRead 0
2025-10-09T21:43:39.400702Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 115 endOffset 134
2025-10-09T21:43:39.400708Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.400892Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 8} (105-105)
2025-10-09T21:43:39.400718Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400898Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 9} (106-106)
2025-10-09T21:43:39.400725Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400728Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400918Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 39 offsets: 38 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.400927Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 39
2025-10-09T21:43:39.400731Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400733Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400930Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 10} (107-107)
2025-10-09T21:43:39.400736Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400739Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400741Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400744Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400955Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 11} (108-108)
2025-10-09T21:43:39.400747Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400750Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400753Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400756Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400966Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 12} (109-109)
2025-10-09T21:43:39.400759Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400761Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400764Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400767Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400769Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400773Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.400946Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 115, endOffset# 134, WTime# 1760046219394, sizeLag# 261028
2025-10-09T21:43:39.400986Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 13} (110-110)
2025-10-09T21:43:39.400948Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:43:39.400958Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# a5ba64eb-7c1a4909-a363e53f-bc3f672b, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 22, size# 313233, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.400971Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 22 maxSize 313233 maxTimeLagMs 0 readTimestampMs 0 readOffset 115 EndOffset 134 ClientCommitOffset 0 committedOffset 0 Guid a5ba64eb-7c1a4909-a363e53f-bc3f672b
2025-10-09T21:43:39.400999Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 14} (111-111)
2025-10-09T21:43:39.400997Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.401009Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 15} (112-112)
2025-10-09T21:43:39.401007Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.401035Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 40 offsets: 39 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.401055Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 40
2025-10-09T21:43:39.401094Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 41 offsets: 40 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.401099Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 41
2025-10-09T21:43:39.401126Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 42 offsets: 41 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.401130Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 42
2025-10-09T21:43:39.401130Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 16} (113-113)
2025-10-09T21:43:39.401137Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 17} (114-114)
2025-10-09T21:43:39.401153Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 43 offsets: 42 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.401156Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 43
2025-10-09T21:43:39.401178Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 44 offsets: 43 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.401181Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 44
2025-10-09T21:43:39.401251Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 45 offsets: 44 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.401260Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 45
2025-10-09T21:43:39.401296Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 46 offsets: 45 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.401301Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 46
2025-10-09T21:43:39.401343Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 47 offsets: 46 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.401346Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 47
>>> event from dataHandler: 2025-10-09T21:43:39.401404Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 48 offsets: 47 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.401414Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 48
2025-10-09T21:43:39.401029Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 13 Topic 'rt3.dc1--test-topic' partition 0 user user offset 115 count 22 size 313233 endOffset 134 max time lag 0ms effective offset 115
2025-10-09T21:43:39.401048Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 13 added 1 blobs, size 261028 count 19 last offset 115, current partition end offset: 134
2025-10-09T21:43:39.401050Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 13. Send blob request.
2025-10-09T21:43:39.401059Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 115 partno 0 count 19 parts_count 0 source 1 size 261028 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.401067Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 13. All 1 blobs are from cache.
2025-10-09T21:43:39.401075Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.401086Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 115 partno 0 count 19 parts 0 suffix '63'
2025-10-09T21:43:39.401178Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.401188Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.401194Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 115 totakecount 19 count 19 size 261006 from pos 0 cbcount 19
2025-10-09T21:43:39.401226Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.401229Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.401243Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 115
2025-10-09T21:43:39.401272Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.401287Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 288  requestId: <none> cookie: 145
2025-10-09T21:43:39.401304Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 209851 } }
2025-10-09T21:43:39.401468Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 49 offsets: 48 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.401317Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# 23f91594-86fc217a-16cf67d0-c6bb7ef1
2025-10-09T21:43:39.401475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 49
DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..10681 bytes.. Information: { Offset: 97 ProducerId: "src_id" SeqNo: 98 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..10791 bytes.. Information: { Offset: 98 ProducerId: "src_id" SeqNo: 99 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..10901 bytes.. Information: { Offset: 99 ProducerId: "src_id" SeqNo: 100 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..11011 bytes.. Information: { Offset: 100 ProducerId: "src_id" SeqNo: 101 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..11121 bytes.. Information: { Offset: 101 ProducerId: "src_id" SeqNo: 102 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..11231 bytes.. Information: { Offset: 102 ProducerId: "src_id" SeqNo: 103 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..11341 bytes.. Information: { Offset: 103 ProducerId: "src_id" SeqNo: 104 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..11451 bytes.. Information: { Offset: 104 ProducerId: "src_id" SeqNo: 105 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..11561 bytes.. Information: { Offset: 105 ProducerId: "src_id" SeqNo: 106 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..11671 bytes.. Information: { Offset: 106 ProducerId: "src_id" SeqNo: 107 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..11781 bytes.. Information: { Offset: 107 ProducerId: "src_id" SeqNo: 108 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..11891 bytes.. Information: { Offset: 108 ProducerId: "src_id" SeqNo: 109 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..12001 bytes.. Information: { Offset: 109 ProducerId: "src_id" SeqNo: 110 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..12111 bytes.. Information: { Offset: 110 ProducerId: "src_id" SeqNo: 111 CreateTime: 2025-10-09T21:43:39.372000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..12221 bytes.. Information: { Offset: 111 ProducerId: "src_id" SeqNo: 112 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..12331 bytes.. Information: { Offset: 112 ProducerId: "src_id" SeqNo: 113 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..12441 bytes.. Information: { Offset: 113 ProducerId: "src_id" SeqNo: 114 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..12551 bytes.. Information: { Offset: 114 ProducerId: "src_id" SeqNo: 115 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.394000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.401324Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.401505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 50 offsets: 49 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.401509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 50
2025-10-09T21:43:39.401340Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.401344Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.401350Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.401352Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.401362Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 145 partNo : 0 messageNo: 290 size 15924 offset: -1
2025-10-09T21:43:39.401372Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.401379Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.401383Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 289  requestId: <none> cookie: 146
2025-10-09T21:43:39.401391Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.401543Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 18, size 209088 bytes
2025-10-09T21:43:39.401533Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 51 offsets: 50 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.401550Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.401553Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 51
2025-10-09T21:43:39.401407Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.401409Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.401413Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 134 Result { Offset: 115 Data: "... 12733 bytes ..." SourceId: "\000src_id" SeqNo: 116 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 12661 PartitionKey: "" ExplicitHash: "" } Result { Offset: 116 Data: "... 12843 bytes ..." SourceId: "\000src_id" SeqNo: 117 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 12771 PartitionKey: "" ExplicitHash: "" } Result { Offset: 117 Data: "... 12953 bytes ..." SourceId: "\000src_id" SeqNo: 118 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 12881 PartitionKey: "" ExplicitHash: "" } Result { Offset: 118 Data: "... 13063 bytes ..." SourceId: "\000src_id" SeqNo: 119 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 12991 PartitionKey: "" ExplicitHash: "" } Result { Offset: 119 Data: "... 13173 bytes ..." SourceId: "\000src_id" SeqNo: 120 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 13101 PartitionKey: "" ExplicitHash: "" } Result { Offset: 120 Data: "... 13283 bytes ..." SourceId: "\000src_id" SeqNo: 121 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 13211 PartitionKey: "" ExplicitHash: "" } Result { Offset: 121 Data: "... 13393 bytes ..." SourceId: "\000src_id" SeqNo: 122 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 13321 PartitionKey: "" ExplicitHash: "" } Result { Offset: 122 Data: "... 13503 bytes ..." SourceId: "\000src_id" SeqNo: 123 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 13431 PartitionKey: "" ExplicitHash: "" } Result { Offset: 123 Data: "... 13613 bytes ..." SourceId: "\000src_id" SeqNo: 124 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 13541 PartitionKey: "" ExplicitHash: "" } Result { Offset: 124 Data: "... 13723 bytes ..." SourceId: "\000src_id" SeqNo: 125 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 13651 PartitionKey: "" ExplicitHash: "" } Result { Offset: 125 Data: "... 13833 bytes ..." SourceId: "\000src_id" SeqNo: 126 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 13761 PartitionKey: "" ExplicitHash: "" } Result { Offset: 126 Data: "... 13943 bytes ..." SourceId: "\000src_id" SeqNo: 127 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 13871 PartitionKey: "" ExplicitHash: "" } Result { Offset: 127 Data: "... 14054 bytes ..." SourceId: "\000src_id" SeqNo: 128 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 13981 PartitionKey: "" ExplicitHash: "" } Result { Offset: 128 Data: "... 14164 bytes ..." SourceId: "\000src_id" SeqNo: 129 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 14091 PartitionKey: "" ExplicitHash: "" } Result { Offset: 129 Data: "... 14274 bytes ..." SourceId: "\000src_id" SeqNo: 130 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 14201 PartitionKey: "" ExplicitHash: "" } Result { Offset: 130 Data: "... 14384 bytes ..." SourceId: "\000src_id" SeqNo: 131 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 14311 PartitionKey: "" ExplicitHash: "" } Result { Offset: 131 Data: "... 14494 bytes ..." SourceId: "\000src_id" SeqNo: 132 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 14421 PartitionKey: "" ExplicitHash: "" } Result { Offset: 132 Data: "... 14604 bytes ..." SourceId: "\000src_id" SeqNo: 133 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 14531 PartitionKey: "" ExplicitHash: "" } Result { Offset: 133 Data: "... 14714 bytes ..." SourceId: "\000src_id" SeqNo: 134 WriteTimestampMS: 1760046219397 CreateTimestampMS: 1760046219373 UncompressedSize: 14641 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 18446744073709551102 RealReadOffset: 133 WaitQuotaTimeMs: 0 EndOffset: 134 StartOffset: 0 } Cookie: 115 }
2025-10-09T21:43:39.401414Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 146 partNo : 0 messageNo: 291 size 16034 offset: -1
2025-10-09T21:43:39.401417Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.401577Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 52 offsets: 51 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.401421Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.401580Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 52
2025-10-09T21:43:39.401580Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 11 from offset 134
2025-10-09T21:43:39.401587Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 134 ReadOffset 134 ReadGuid a5ba64eb-7c1a4909-a363e53f-bc3f672b has messages 1
2025-10-09T21:43:39.401600Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 53 offsets: 52 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.401603Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 53
2025-10-09T21:43:39.401623Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 54 offsets: 53 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.401626Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 54
2025-10-09T21:43:39.401620Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# a5ba64eb-7c1a4909-a363e53f-bc3f672b, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 260180
2025-10-09T21:43:39.401637Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# a5ba64eb-7c1a4909-a363e53f-bc3f672b
2025-10-09T21:43:39.401655Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 55 offsets: 54 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.401660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 55
2025-10-09T21:43:39.401692Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 56 offsets: 55 already_written: false write_statistics { persist_duration_ms: 4 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.401696Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 56
2025-10-09T21:43:39.401718Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 57 offsets: 56 already_written: false write_statistics { persist_duration_ms: 4 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.401724Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 57
2025-10-09T21:43:39.401746Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 58 offsets: 57 already_written: false write_statistics { persist_duration_ms: 4 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.401749Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 58
2025-10-09T21:43:39.401769Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 59 offsets: 58 already_written: false write_statistics { persist_duration_ms: 4 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.401772Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 59
2025-10-09T21:43:39.401792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 60 offsets: 59 already_written: false write_statistics { persist_duration_ms: 4 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.401796Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 60
2025-10-09T21:43:39.401821Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 61 offsets: 60 already_written: false write_statistics { persist_duration_ms: 4 }
2025-10-09T21:43:39.401824Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 61
2025-10-09T21:43:39.401848Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 62 offsets: 61 already_written: false write_statistics { persist_duration_ms: 4 }
2025-10-09T21:43:39.401851Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 62
2025-10-09T21:43:39.401872Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 63 offsets: 62 already_written: false write_statistics { persist_duration_ms: 4 }
2025-10-09T21:43:39.401876Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 63
2025-10-09T21:43:39.401909Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 64 offsets: 63 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 3 }
2025-10-09T21:43:39.401913Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 64
2025-10-09T21:43:39.401934Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 65 offsets: 64 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 3 }
2025-10-09T21:43:39.401937Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 65
2025-10-09T21:43:39.401885Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 0
2025-10-09T21:43:39.401957Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 66 offsets: 65 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 3 }
2025-10-09T21:43:39.401960Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 66
2025-10-09T21:43:39.401980Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 67 offsets: 66 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.401983Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 67
2025-10-09T21:43:39.402003Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 68 offsets: 67 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.402006Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 68
2025-10-09T21:43:39.402026Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 69 offsets: 68 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.402509Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 69
2025-10-09T21:43:39.402565Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 70 offsets: 69 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.402569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 70
2025-10-09T21:43:39.402578Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 260180, now ReadSizeBudget = 0, ReadSizeServerDelta = 264108
2025-10-09T21:43:39.402591Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 71 offsets: 70 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.402595Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 71
2025-10-09T21:43:39.402609Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 0, ReadSizeServerDelta = 264108
2025-10-09T21:43:39.402614Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 72 offsets: 71 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.402617Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 72
2025-10-09T21:43:39.402634Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 73 offsets: 72 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.402637Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 73
2025-10-09T21:43:39.402656Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 74 offsets: 73 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.402659Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 74
2025-10-09T21:43:39.402676Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 75 offsets: 74 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.402679Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 75
2025-10-09T21:43:39.402695Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 76 offsets: 75 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.402698Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 76
2025-10-09T21:43:39.402718Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 77 offsets: 76 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.402722Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 77
2025-10-09T21:43:39.402671Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.402720Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.402738Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 78 offsets: 77 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.402741Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 78
2025-10-09T21:43:39.402731Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.402757Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 79 offsets: 78 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.402756Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.402760Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 79
2025-10-09T21:43:39.402765Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (115-133)
2025-10-09T21:43:39.402777Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 80 offsets: 79 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.402778Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 260180 to budget
2025-10-09T21:43:39.402782Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 80
2025-10-09T21:43:39.402774Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.402785Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 260180, ReadSizeServerDelta = 264108
2025-10-09T21:43:39.402783Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.402790Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.402792Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.402797Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 81 offsets: 80 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.402800Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 81
2025-10-09T21:43:39.402819Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 82 offsets: 81 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.402823Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 82
2025-10-09T21:43:39.402803Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 292  requestId: <none> cookie: 147
2025-10-09T21:43:39.402848Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 293  requestId: <none> cookie: 148
2025-10-09T21:43:39.402869Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.402862Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 134 count 10 size 153386 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.402874Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.402885Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (115-115)
2025-10-09T21:43:39.402895Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (116-116)
2025-10-09T21:43:39.402904Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 2} (117-117)
2025-10-09T21:43:39.402909Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 3} (118-118)
2025-10-09T21:43:39.402886Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.402933Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 4} (119-119)
2025-10-09T21:43:39.402890Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 134 partno 0 count 10 parts 0 suffix '63' size 153386
2025-10-09T21:43:39.402938Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 5} (120-120)
2025-10-09T21:43:39.402943Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 6} (121-121)
2025-10-09T21:43:39.402893Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 153260 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.402952Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 83 offsets: 82 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.402899Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.402906Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 135, partNo: 0, Offset: 134 is stored on disk
2025-10-09T21:43:39.402908Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.402963Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 83
2025-10-09T21:43:39.402910Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.402912Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 136, partNo: 0, Offset: 135 is stored on disk
2025-10-09T21:43:39.402914Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.402915Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.402917Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 137, partNo: 0, Offset: 136 is stored on disk
2025-10-09T21:43:39.402919Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.402923Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 138, partNo: 0, Offset: 137 is stored on disk
2025-10-09T21:43:39.402925Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.402929Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 139, partNo: 0, Offset: 138 is stored on disk
2025-10-09T21:43:39.402992Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 84 offsets: 83 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.402931Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.402996Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 84
2025-10-09T21:43:39.402934Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 140, partNo: 0, Offset: 139 is stored on disk
2025-10-09T21:43:39.402936Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.402939Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 141, partNo: 0, Offset: 140 is stored on disk
2025-10-09T21:43:39.402941Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.402945Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 142, partNo: 0, Offset: 141 is stored on disk
2025-10-09T21:43:39.403014Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 85 offsets: 84 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.402947Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.403017Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 85
2025-10-09T21:43:39.402950Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 143, partNo: 0, Offset: 142 is stored on disk
2025-10-09T21:43:39.402952Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.403033Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 86 offsets: 85 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.402955Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 144, partNo: 0, Offset: 143 is stored on disk
2025-10-09T21:43:39.403038Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 86
2025-10-09T21:43:39.402977Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.402983Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.402985Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (2)
2025-10-09T21:43:39.403000Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 145 partNo 0
2025-10-09T21:43:39.403033Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 145 partNo 0 FormedBlobsCount 0 NewHead: Offset 144 PartNo 0 PackedSize 15998 count 1 nextOffset 145 batches 1
2025-10-09T21:43:39.403040Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 146 partNo 0
2025-10-09T21:43:39.403044Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 146 partNo 0 FormedBlobsCount 0 NewHead: Offset 144 PartNo 0 PackedSize 32074 count 2 nextOffset 146 batches 1
2025-10-09T21:43:39.403055Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 87 offsets: 86 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403085Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 87
2025-10-09T21:43:39.402953Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 7} (122-122)
2025-10-09T21:43:39.403043Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.403046Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.403097Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 144,2 HeadOffset 144 endOffset 144 curOffset 146 d0000000000_00000000000000000144_00000_0000000002_00000? size 32074 WTime 1760046219402
2025-10-09T21:43:39.403147Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 88 offsets: 87 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403101Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.403152Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 88
2025-10-09T21:43:39.403110Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.403114Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.403121Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 269  requestId: <none> cookie: 135
2025-10-09T21:43:39.403125Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 134 EndOffset 134 newEndOffset 144 commitOffset 0 clientCommitOffset 0 cookie 11 readingFinished 0 firstRead 0
2025-10-09T21:43:39.403165Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 8} (123-123)
2025-10-09T21:43:39.403128Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 275  requestId: <none> cookie: 136
2025-10-09T21:43:39.403129Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 134 endOffset 144
2025-10-09T21:43:39.403172Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 9} (124-124)
2025-10-09T21:43:39.403133Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 276  requestId: <none> cookie: 137
2025-10-09T21:43:39.403137Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 277  requestId: <none> cookie: 138
2025-10-09T21:43:39.403184Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 10} (125-125)
2025-10-09T21:43:39.403141Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 278  requestId: <none> cookie: 139
2025-10-09T21:43:39.403146Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 260180 } }
2025-10-09T21:43:39.403148Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 281  requestId: <none> cookie: 140
2025-10-09T21:43:39.403173Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 89 offsets: 88 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403151Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 284  requestId: <none> cookie: 141
2025-10-09T21:43:39.403197Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 11} (126-126)
2025-10-09T21:43:39.403155Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 285  requestId: <none> cookie: 142
2025-10-09T21:43:39.403156Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 134, endOffset# 144, WTime# 1760046219397, sizeLag# 153386
2025-10-09T21:43:39.403197Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 89
2025-10-09T21:43:39.403209Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 12} (127-127)
2025-10-09T21:43:39.403158Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:43:39.403220Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 13} (128-128)
2025-10-09T21:43:39.403160Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 286  requestId: <none> cookie: 143
2025-10-09T21:43:39.403164Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 287  requestId: <none> cookie: 144
2025-10-09T21:43:39.403253Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 14} (129-129)
2025-10-09T21:43:39.403264Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 15} (130-130)
2025-10-09T21:43:39.403263Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 90 offsets: 89 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403283Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 90
2025-10-09T21:43:39.403166Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# 6c4cf21f-b5759933-4adcc27b-f69dc09e, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 12, size# 184063, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.403169Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.403178Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 144 partNo 0 count 2 size 32074
2025-10-09T21:43:39.403182Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# 136a5e7d-81cb7535-408d759e-13ba69d4
2025-10-09T21:43:39.403188Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.403196Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.403200Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.403204Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.403208Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.403212Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.403323Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 16} (131-131)
2025-10-09T21:43:39.403216Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.403329Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 91 offsets: 90 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403331Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 17} (132-132)
2025-10-09T21:43:39.403337Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 91
2025-10-09T21:43:39.403221Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.403224Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.403337Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 18} (133-133)
2025-10-09T21:43:39.403228Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.403229Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 12 maxSize 184063 maxTimeLagMs 0 readTimestampMs 0 readOffset 134 EndOffset 144 ClientCommitOffset 0 committedOffset 0 Guid 6c4cf21f-b5759933-4adcc27b-f69dc09e
2025-10-09T21:43:39.403365Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 92 offsets: 91 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.403231Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.403370Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 92
2025-10-09T21:43:39.403318Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.403394Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 93 offsets: 92 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.403397Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 93
2025-10-09T21:43:39.403393Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.403413Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 94 offsets: 93 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.403416Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 94
2025-10-09T21:43:39.403414Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.403434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 95 offsets: 94 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.403437Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 95
2025-10-09T21:43:39.403431Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 147 partNo : 0 messageNo: 294 size 16144 offset: -1
2025-10-09T21:43:39.403444Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.403450Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.403456Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 148 partNo : 0 messageNo: 295 size 16254 offset: -1
2025-10-09T21:43:39.403458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 96 offsets: 95 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.403462Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 96
2025-10-09T21:43:39.403460Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.403466Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.403472Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.403473Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.403480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 97 offsets: 96 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.403477Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.403479Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.403488Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.403490Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.403498Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.403506Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.403513Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.403516Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.403483Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 97
2025-10-09T21:43:39.403529Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 296  requestId: <none> cookie: 149
2025-10-09T21:43:39.403545Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 297  requestId: <none> cookie: 150
2025-10-09T21:43:39.403549Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 298  requestId: <none> cookie: 151
2025-10-09T21:43:39.403554Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.403561Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 14 Topic 'rt3.dc1--test-topic' partition 0 user user offset 134 count 12 size 184063 endOffset 144 max time lag 0ms effective offset 134
2025-10-09T21:43:39.403562Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.403566Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.403570Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.403580Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 14 added 1 blobs, size 153386 count 10 last offset 134, current partition end offset: 144
2025-10-09T21:43:39.403582Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 14. Send blob request.
2025-10-09T21:43:39.403590Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 134 partno 0 count 10 parts_count 0 source 1 size 153386 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.403594Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 14. All 1 blobs are from cache.
2025-10-09T21:43:39.403601Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.403608Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 98 offsets: 97 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.403603Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.403610Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 149 partNo : 0 messageNo: 299 size 16364 offset: -1
2025-10-09T21:43:39.403622Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 98
2025-10-09T21:43:39.403613Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 134 partno 0 count 10 parts 0 suffix '63'
2025-10-09T21:43:39.403616Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.403638Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.403663Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.403687Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 99 offsets: 98 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403694Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 99
2025-10-09T21:43:39.403688Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.403707Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 134 totakecount 10 count 10 size 153363 from pos 0 cbcount 10
2025-10-09T21:43:39.403724Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 100 offsets: 99 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403727Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 100
2025-10-09T21:43:39.403750Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 101 offsets: 100 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403754Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 101
2025-10-09T21:43:39.403774Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 102 offsets: 101 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403777Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 102
2025-10-09T21:43:39.403794Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 103 offsets: 102 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403807Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 103
>>> event from dataHandler: 2025-10-09T21:43:39.403830Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 104 offsets: 103 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403833Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 104
2025-10-09T21:43:39.403849Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 105 offsets: 104 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403852Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 105
2025-10-09T21:43:39.403867Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 106 offsets: 105 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403870Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 106
2025-10-09T21:43:39.403884Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 107 offsets: 106 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403887Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 107
2025-10-09T21:43:39.403902Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 108 offsets: 107 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403905Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 108
2025-10-09T21:43:39.403921Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 109 offsets: 108 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403924Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 109
DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..12661 bytes.. Information: { Offset: 115 ProducerId: "src_id" SeqNo: 116 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..12771 bytes.. Information: { Offset: 116 ProducerId: "src_id" SeqNo: 117 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..12881 bytes.. Information: { Offset: 117 ProducerId: "src_id" SeqNo: 118 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..12991 bytes.. Information: { Offset: 118 ProducerId: "src_id" SeqNo: 119 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..13101 bytes.. Information: { Offset: 119 ProducerId: "src_id" SeqNo: 120 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..13211 bytes.. Information: { Offset: 120 ProducerId: "src_id" SeqNo: 121 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..13321 bytes.. Information: { Offset: 121 ProducerId: "src_id" SeqNo: 122 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..13431 bytes.. Information: { Offset: 122 ProducerId: "src_id" SeqNo: 123 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..13541 bytes.. Information: { Offset: 123 ProducerId: "src_id" SeqNo: 124 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..13651 bytes.. Information: { Offset: 124 ProducerId: "src_id" SeqNo: 125 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..13761 bytes.. Information: { Offset: 125 ProducerId: "src_id" SeqNo: 126 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..13871 bytes.. Information: { Offset: 126 ProducerId: "src_id" SeqNo: 127 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..13981 bytes.. Information: { Offset: 127 ProducerId: "src_id" SeqNo: 128 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..14091 bytes.. Information: { Offset: 128 ProducerId: "src_id" SeqNo: 129 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..14201 bytes.. Information: { Offset: 129 ProducerId: "src_id" SeqNo: 130 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..14311 bytes.. Information: { Offset: 130 ProducerId: "src_id" SeqNo: 131 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..14421 bytes.. Information: { Offset: 131 ProducerId: "src_id" SeqNo: 132 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..14531 bytes.. Information: { Offset: 132 ProducerId: "src_id" SeqNo: 133 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..14641 bytes.. Information: { Offset: 133 ProducerId: "src_id" SeqNo: 134 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.397000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.403940Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 110 offsets: 109 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.404000Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 110
2025-10-09T21:43:39.404030Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 19, size 259369 bytes
2025-10-09T21:43:39.404040Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.404063Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 111 offsets: 110 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.404068Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 111
2025-10-09T21:43:39.404107Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 112 offsets: 111 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.404111Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 112
2025-10-09T21:43:39.404132Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 113 offsets: 112 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.404136Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 113
2025-10-09T21:43:39.404158Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 114 offsets: 113 already_written: false write_statistics { persist_duration_ms: 3 }
2025-10-09T21:43:39.404162Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 114
2025-10-09T21:43:39.404178Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 115 offsets: 114 already_written: false write_statistics { persist_duration_ms: 3 }
2025-10-09T21:43:39.404181Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 115
2025-10-09T21:43:39.404198Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 116 offsets: 115 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.404201Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 116
2025-10-09T21:43:39.404216Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 117 offsets: 116 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.404219Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 117
2025-10-09T21:43:39.404236Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 118 offsets: 117 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.404241Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 118
2025-10-09T21:43:39.404258Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 119 offsets: 118 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.404261Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 119
2025-10-09T21:43:39.404278Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 120 offsets: 119 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.404280Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 120
2025-10-09T21:43:39.404299Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 121 offsets: 120 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.404302Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 121
2025-10-09T21:43:39.404322Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 122 offsets: 121 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.404325Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 122
2025-10-09T21:43:39.404356Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 123 offsets: 122 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.404360Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 123
2025-10-09T21:43:39.404378Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 124 offsets: 123 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.404381Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 124
2025-10-09T21:43:39.404396Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 125 offsets: 124 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.404399Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 125
2025-10-09T21:43:39.404415Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 126 offsets: 125 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.404417Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 126
2025-10-09T21:43:39.404434Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 127 offsets: 126 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.404437Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 127
2025-10-09T21:43:39.404452Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 128 offsets: 127 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.404455Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 128
2025-10-09T21:43:39.403739Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.403752Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.403755Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.403758Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.403765Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 150 partNo : 0 messageNo: 300 size 16475 offset: -1
2025-10-09T21:43:39.403771Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.403773Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.403777Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 151 partNo : 0 messageNo: 301 size 16585 offset: -1
2025-10-09T21:43:39.403781Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.403783Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.403787Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.403789Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.403800Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 134
2025-10-09T21:43:39.403819Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.403822Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.404537Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 129 offsets: 128 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.403825Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.404541Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 129
2025-10-09T21:43:39.403827Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.404569Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 153082, now ReadSizeBudget = 0, ReadSizeServerDelta = 371206
2025-10-09T21:43:39.403865Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.403929Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.403940Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 302  requestId: <none> cookie: 152
2025-10-09T21:43:39.403944Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 303  requestId: <none> cookie: 153
2025-10-09T21:43:39.403948Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.403955Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.403962Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.404574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 130 offsets: 129 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.404611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 130
2025-10-09T21:43:39.404614Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 0, ReadSizeServerDelta = 371206
2025-10-09T21:43:39.404038Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 144 Result { Offset: 134 Data: "... 14824 bytes ..." SourceId: "\000src_id" SeqNo: 135 WriteTimestampMS: 1760046219399 CreateTimestampMS: 1760046219373 UncompressedSize: 14751 PartitionKey: "" ExplicitHash: "" } Result { Offset: 135 Data: "... 14934 bytes ..." SourceId: "\000src_id" SeqNo: 136 WriteTimestampMS: 1760046219399 CreateTimestampMS: 1760046219373 UncompressedSize: 14861 PartitionKey: "" ExplicitHash: "" } Result { Offset: 136 Data: "... 15044 bytes ..." SourceId: "\000src_id" SeqNo: 137 WriteTimestampMS: 1760046219400 CreateTimestampMS: 1760046219374 UncompressedSize: 14971 PartitionKey: "" ExplicitHash: "" } Result { Offset: 137 Data: "... 15154 bytes ..." SourceId: "\000src_id" SeqNo: 138 WriteTimestampMS: 1760046219400 CreateTimestampMS: 1760046219374 UncompressedSize: 15081 PartitionKey: "" ExplicitHash: "" } Result { Offset: 138 Data: "... 15264 bytes ..." SourceId: "\000src_id" SeqNo: 139 WriteTimestampMS: 1760046219400 CreateTimestampMS: 1760046219378 UncompressedSize: 15191 PartitionKey: "" ExplicitHash: "" } Result { Offset: 139 Data: "... 15374 bytes ..." SourceId: "\000src_id" SeqNo: 140 WriteTimestampMS: 1760046219400 CreateTimestampMS: 1760046219378 UncompressedSize: 15301 PartitionKey: "" ExplicitHash: "" } Result { Offset: 140 Data: "... 15484 bytes ..." SourceId: "\000src_id" SeqNo: 141 WriteTimestampMS: 1760046219400 CreateTimestampMS: 1760046219378 UncompressedSize: 15411 PartitionKey: "" ExplicitHash: "" } Result { Offset: 141 Data: "... 15594 bytes ..." SourceId: "\000src_id" SeqNo: 142 WriteTimestampMS: 1760046219400 CreateTimestampMS: 1760046219379 UncompressedSize: 15521 PartitionKey: "" ExplicitHash: "" } Result { Offset: 142 Data: "... 15704 bytes ..." SourceId: "\000src_id" SeqNo: 143 WriteTimestampMS: 1760046219400 CreateTimestampMS: 1760046219379 UncompressedSize: 15631 PartitionKey: "" ExplicitHash: "" } Result { Offset: 143 Data: "... 15814 bytes ..." SourceId: "\000src_id" SeqNo: 144 WriteTimestampMS: 1760046219400 CreateTimestampMS: 1760046219379 UncompressedSize: 15741 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 18446744073709551392 RealReadOffset: 143 WaitQuotaTimeMs: 0 EndOffset: 144 StartOffset: 0 } Cookie: 134 }
2025-10-09T21:43:39.404100Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 12 from offset 144
2025-10-09T21:43:39.404107Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 144 ReadOffset 144 ReadGuid 6c4cf21f-b5759933-4adcc27b-f69dc09e has messages 1
2025-10-09T21:43:39.404128Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# 6c4cf21f-b5759933-4adcc27b-f69dc09e, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 153082
2025-10-09T21:43:39.404136Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# 6c4cf21f-b5759933-4adcc27b-f69dc09e
2025-10-09T21:43:39.404257Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.404346Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.404353Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.404637Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 131 offsets: 130 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.404362Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.404660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 131
2025-10-09T21:43:39.404364Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.404692Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 132 offsets: 131 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.404696Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 132
2025-10-09T21:43:39.404378Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 152 partNo : 0 messageNo: 305 size 16695 offset: -1
2025-10-09T21:43:39.404379Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 0
2025-10-09T21:43:39.404717Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 133 offsets: 132 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.404382Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.404721Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 133
2025-10-09T21:43:39.404384Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.404389Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 153 partNo : 0 messageNo: 306 size 16805 offset: -1
2025-10-09T21:43:39.404392Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.404408Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.404420Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.404427Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.404429Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.404743Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 134 offsets: 133 already_written: false write_statistics { persist_duration_ms: 2 }
2025-10-09T21:43:39.404432Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.404746Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 134
2025-10-09T21:43:39.404778Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 135 offsets: 134 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.404781Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 135
2025-10-09T21:43:39.404442Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 304  requestId: <none> cookie: 154
2025-10-09T21:43:39.404798Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 136 offsets: 135 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.404456Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 307  requestId: <none> cookie: 155
2025-10-09T21:43:39.404817Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 136
2025-10-09T21:43:39.404477Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.404484Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.404527Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.404532Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.404864Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 137 offsets: 136 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.404872Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 137
2025-10-09T21:43:39.404543Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 154 partNo : 0 messageNo: 308 size 16915 offset: -1
2025-10-09T21:43:39.404550Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.404552Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.404893Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 138 offsets: 137 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.404565Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 155 partNo : 0 messageNo: 309 size 17025 offset: -1
2025-10-09T21:43:39.404896Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 138
2025-10-09T21:43:39.404570Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.404578Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.404580Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.404583Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.404922Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 139 offsets: 138 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.404925Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 139
2025-10-09T21:43:39.404930Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (134-143)
2025-10-09T21:43:39.404942Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 153082 to budget
2025-10-09T21:43:39.404947Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 153082, ReadSizeServerDelta = 371206
2025-10-09T21:43:39.404948Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 140 offsets: 139 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.404952Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 140
2025-10-09T21:43:39.404976Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 141 offsets: 140 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.404979Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 141
2025-10-09T21:43:39.405017Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.405030Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (134-134)
2025-10-09T21:43:39.405040Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (135-135)
2025-10-09T21:43:39.405049Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 0} (136-136)
2025-10-09T21:43:39.405053Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 1} (137-137)
2025-10-09T21:43:39.405029Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.405074Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 2} (138-138)
2025-10-09T21:43:39.405081Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 142 offsets: 141 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.405086Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 142
2025-10-09T21:43:39.405087Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 3} (139-139)
2025-10-09T21:43:39.405093Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 4} (140-140)
2025-10-09T21:43:39.405098Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 5} (141-141)
2025-10-09T21:43:39.405110Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.405130Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 6} (142-142)
2025-10-09T21:43:39.405123Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.405135Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 7} (143-143)
2025-10-09T21:43:39.405137Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 143 offsets: 142 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.405141Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 143
2025-10-09T21:43:39.405164Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 144 offsets: 143 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.405167Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 144
2025-10-09T21:43:39.405155Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 153082 } }
2025-10-09T21:43:39.405180Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.405191Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.405221Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.405246Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# c3039cef-101a2aba-28e0202a-53487915
2025-10-09T21:43:39.405281Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.405288Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
>>> event from dataHandler: 2025-10-09T21:43:39.405295Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.405297Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.405302Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.405303Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.405313Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 144 count 2 size 32074 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.405331Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..14751 bytes.. Information: { Offset: 134 ProducerId: "src_id" SeqNo: 135 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.399000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..14861 bytes.. Information: { Offset: 135 ProducerId: "src_id" SeqNo: 136 CreateTime: 2025-10-09T21:43:39.373000Z WriteTime: 2025-10-09T21:43:39.399000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..14971 bytes.. Information: { Offset: 136 ProducerId: "src_id" SeqNo: 137 CreateTime: 2025-10-09T21:43:39.374000Z WriteTime: 2025-10-09T21:43:39.400000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..15081 bytes.. Information: { Offset: 137 ProducerId: "src_id" SeqNo: 138 CreateTime: 2025-10-09T21:43:39.374000Z WriteTime: 2025-10-09T21:43:39.400000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..15191 bytes.. Information: { Offset: 138 ProducerId: "src_id" SeqNo: 139 CreateTime: 2025-10-09T21:43:39.378000Z WriteTime: 2025-10-09T21:43:39.400000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..15301 bytes.. Information: { Offset: 139 ProducerId: "src_id" SeqNo: 140 CreateTime: 2025-10-09T21:43:39.378000Z WriteTime: 2025-10-09T21:43:39.400000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..15411 bytes.. Information: { Offset: 140 ProducerId: "src_id" SeqNo: 141 CreateTime: 2025-10-09T21:43:39.378000Z WriteTime: 2025-10-09T21:43:39.400000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..15521 bytes.. Information: { Offset: 141 ProducerId: "src_id" SeqNo: 142 CreateTime: 2025-10-09T21:43:39.379000Z WriteTime: 2025-10-09T21:43:39.400000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..15631 bytes.. Information: { Offset: 142 ProducerId: "src_id" SeqNo: 143 CreateTime: 2025-10-09T21:43:39.379000Z WriteTime: 2025-10-09T21:43:39.400000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..15741 bytes.. Information: { Offset: 143 ProducerId: "src_id" SeqNo: 144 CreateTime: 2025-10-09T21:43:39.379000Z WriteTime: 2025-10-09T21:43:39.400000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.405344Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.405344Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 144 partno 0 count 2 parts 0 suffix '63' size 32074
2025-10-09T21:43:39.405346Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 31972 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.405353Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.405358Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 145, partNo: 0, Offset: 144 is stored on disk
2025-10-09T21:43:39.405361Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.405363Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 146, partNo: 0, Offset: 145 is stored on disk
2025-10-09T21:43:39.405397Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.405407Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.405408Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (9)
2025-10-09T21:43:39.405421Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 147 partNo 0
2025-10-09T21:43:39.405434Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 10, size 152460 bytes
2025-10-09T21:43:39.405430Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.405443Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.405444Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.405455Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 147 partNo 0 FormedBlobsCount 0 NewHead: Offset 146 PartNo 0 PackedSize 16218 count 1 nextOffset 147 batches 1
2025-10-09T21:43:39.405460Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 148 partNo 0
2025-10-09T21:43:39.405470Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 148 partNo 0 FormedBlobsCount 0 NewHead: Offset 146 PartNo 0 PackedSize 32514 count 2 nextOffset 148 batches 1
2025-10-09T21:43:39.405473Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 149 partNo 0
2025-10-09T21:43:39.405477Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 149 partNo 0 FormedBlobsCount 0 NewHead: Offset 146 PartNo 0 PackedSize 48920 count 3 nextOffset 149 batches 1
2025-10-09T21:43:39.405480Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 150 partNo 0
2025-10-09T21:43:39.405484Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 150 partNo 0 FormedBlobsCount 0 NewHead: Offset 146 PartNo 0 PackedSize 65437 count 4 nextOffset 150 batches 1
2025-10-09T21:43:39.405487Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 151 partNo 0
2025-10-09T21:43:39.405491Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 151 partNo 0 FormedBlobsCount 0 NewHead: Offset 146 PartNo 0 PackedSize 82064 count 5 nextOffset 151 batches 1
2025-10-09T21:43:39.405499Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 152 partNo 0
2025-10-09T21:43:39.405500Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.405502Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 152 partNo 0 FormedBlobsCount 0 NewHead: Offset 146 PartNo 0 PackedSize 98801 count 6 nextOffset 152 batches 1
2025-10-09T21:43:39.405505Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 153 partNo 0
2025-10-09T21:43:39.405509Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 153 partNo 0 FormedBlobsCount 0 NewHead: Offset 146 PartNo 0 PackedSize 115648 count 7 nextOffset 153 batches 1
2025-10-09T21:43:39.405512Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 154 partNo 0
2025-10-09T21:43:39.405515Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 154 partNo 0 FormedBlobsCount 0 NewHead: Offset 146 PartNo 0 PackedSize 132605 count 8 nextOffset 154 batches 1
2025-10-09T21:43:39.405519Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 155 partNo 0
2025-10-09T21:43:39.405521Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 144 EndOffset 144 newEndOffset 146 commitOffset 0 clientCommitOffset 0 cookie 12 readingFinished 0 firstRead 0
2025-10-09T21:43:39.405524Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 155 partNo 0 FormedBlobsCount 0 NewHead: Offset 146 PartNo 0 PackedSize 149672 count 9 nextOffset 155 batches 1
2025-10-09T21:43:39.405524Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 144 endOffset 146
2025-10-09T21:43:39.405526Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.405532Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 144, endOffset# 146, WTime# 1760046219400, sizeLag# 32074
2025-10-09T21:43:39.405535Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:43:39.405546Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# 8013515b-f76f328d-569af8dd-9364fae1, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 2, size# 38488, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.405555Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.405634Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.405684Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 2 maxSize 38488 maxTimeLagMs 0 readTimestampMs 0 readOffset 144 EndOffset 146 ClientCommitOffset 0 committedOffset 0 Guid 8013515b-f76f328d-569af8dd-9364fae1
2025-10-09T21:43:39.405706Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.405757Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 146,9 HeadOffset 146 endOffset 146 curOffset 155 d0000000000_00000000000000000146_00000_0000000009_00000? size 149454 WTime 1760046219405
2025-10-09T21:43:39.405758Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.405814Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.405816Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.405826Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 310  requestId: <none> cookie: 156
2025-10-09T21:43:39.405833Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 311  requestId: <none> cookie: 157
2025-10-09T21:43:39.405836Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 312  requestId: <none> cookie: 158
2025-10-09T21:43:39.405841Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 290  requestId: <none> cookie: 145
2025-10-09T21:43:39.405854Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 291  requestId: <none> cookie: 146
2025-10-09T21:43:39.405859Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.405865Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.405866Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.405872Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.405873Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.405874Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.405877Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.405881Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.405882Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.405885Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.405886Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.405892Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.405902Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 146 partNo 0 count 9 size 149454
2025-10-09T21:43:39.405914Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.405928Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.405937Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.405939Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 313  requestId: <none> cookie: 159
2025-10-09T21:43:39.405942Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 314  requestId: <none> cookie: 160
2025-10-09T21:43:39.405968Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 15 Topic 'rt3.dc1--test-topic' partition 0 user user offset 144 count 2 size 38488 endOffset 146 max time lag 0ms effective offset 144
2025-10-09T21:43:39.405985Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 15 added 1 blobs, size 32074 count 2 last offset 144, current partition end offset: 146
2025-10-09T21:43:39.405993Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 15. Send blob request.
2025-10-09T21:43:39.405998Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.406000Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.406007Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 156 partNo : 0 messageNo: 315 size 17135 offset: -1
2025-10-09T21:43:39.406011Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.406017Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.406019Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.406021Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 157 partNo : 0 messageNo: 316 size 17245 offset: -1
2025-10-09T21:43:39.406025Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.406026Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.406027Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.406045Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 158 partNo : 0 messageNo: 317 size 17355 offset: -1
2025-10-09T21:43:39.406055Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 144 partno 0 count 2 parts_count 0 source 1 size 32074 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.406063Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.406068Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 15. All 1 blobs are from cache.
2025-10-09T21:43:39.406074Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.406080Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.406083Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.406086Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.406089Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.406092Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.406096Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.406098Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.406101Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.406103Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.406105Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 144 partno 0 count 2 parts 0 suffix '63'
2025-10-09T21:43:39.406106Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.406108Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.406113Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.406143Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 144 totakecount 2 count 2 size 32051 from pos 0 cbcount 2
2025-10-09T21:43:39.406145Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.406151Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.406164Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 144
2025-10-09T21:43:39.406178Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 318  requestId: <none> cookie: 161
2025-10-09T21:43:39.406189Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 319  requestId: <none> cookie: 162
2025-10-09T21:43:39.406197Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 320  requestId: <none> cookie: 163
2025-10-09T21:43:39.406209Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.406220Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.406223Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.406229Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.406254Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 146 Result { Offset: 144 Data: "... 15924 bytes ..." SourceId: "\000src_id" SeqNo: 145 WriteTimestampMS: 1760046219402 CreateTimestampMS: 1760046219379 UncompressedSize: 15851 PartitionKey: "" ExplicitHash: "" } Result { Offset: 145 Data: "... 16034 bytes ..." SourceId: "\000src_id" SeqNo: 146 WriteTimestampMS: 1760046219402 CreateTimestampMS: 1760046219379 UncompressedSize: 15961 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 32 RealReadOffset: 145 WaitQuotaTimeMs: 0 EndOffset: 146 StartOffset: 0 } Cookie: 144 }
2025-10-09T21:43:39.406280Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 13 from offset 146
2025-10-09T21:43:39.406292Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 146 ReadOffset 146 ReadGuid 8013515b-f76f328d-569af8dd-9364fae1 has messages 1
2025-10-09T21:43:39.406296Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 145 offsets: 144 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.406306Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 145
2025-10-09T21:43:39.406356Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 146 offsets: 145 already_written: false write_statistics { persist_duration_ms: 2 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.406360Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 146
2025-10-09T21:43:39.406314Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# 8013515b-f76f328d-569af8dd-9364fae1, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 32019
2025-10-09T21:43:39.406326Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# 8013515b-f76f328d-569af8dd-9364fae1
2025-10-09T21:43:39.406373Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.406383Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.406433Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 0
2025-10-09T21:43:39.406433Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.406451Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.406451Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.406455Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.406463Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 159 partNo : 0 messageNo: 321 size 17465 offset: -1
2025-10-09T21:43:39.406469Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.406874Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 32019, now ReadSizeBudget = 0, ReadSizeServerDelta = 492269
2025-10-09T21:43:39.406470Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.406474Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 160 partNo : 0 messageNo: 322 size 17575 offset: -1
2025-10-09T21:43:39.406477Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.406897Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 0, ReadSizeServerDelta = 492269
2025-10-09T21:43:39.406479Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.406482Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.406484Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.406487Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.406489Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.406493Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 161 partNo : 0 messageNo: 325 size 17685 offset: -1
2025-10-09T21:43:39.406496Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.406498Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.406502Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 162 partNo : 0 messageNo: 326 size 17795 offset: -1
2025-10-09T21:43:39.406505Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.406507Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.406511Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 163 partNo : 0 messageNo: 327 size 17905 offset: -1
2025-10-09T21:43:39.406517Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.406523Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.406525Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.406527Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.406532Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.406534Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.406536Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.406538Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.406539Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.406542Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.406550Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 323  requestId: <none> cookie: 164
2025-10-09T21:43:39.406562Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 324  requestId: <none> cookie: 165
2025-10-09T21:43:39.406571Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.406576Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.406818Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.406902Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.406918Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.406951Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.406968Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.407003Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (144-145)
2025-10-09T21:43:39.407014Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 32019 to budget
2025-10-09T21:43:39.407019Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 32019, ReadSizeServerDelta = 492269
2025-10-09T21:43:39.407090Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.407103Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (144-144)
2025-10-09T21:43:39.407111Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (145-145)
>>> event from dataHandler: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..15851 bytes.. Information: { Offset: 144 ProducerId: "src_id" SeqNo: 145 CreateTime: 2025-10-09T21:43:39.379000Z WriteTime: 2025-10-09T21:43:39.402000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..15961 bytes.. Information: { Offset: 145 ProducerId: "src_id" SeqNo: 146 CreateTime: 2025-10-09T21:43:39.379000Z WriteTime: 2025-10-09T21:43:39.402000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.407198Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 2, size 31812 bytes
2025-10-09T21:43:39.407205Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.407103Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407111Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407119Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407120Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407124Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407126Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407129Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407131Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407139Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 164 partNo : 0 messageNo: 331 size 18015 offset: -1
2025-10-09T21:43:39.407143Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407144Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407148Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 165 partNo : 0 messageNo: 332 size 18125 offset: -1
2025-10-09T21:43:39.407151Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407153Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407157Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407158Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407168Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.407175Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.407184Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.407186Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.407193Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 328  requestId: <none> cookie: 166
2025-10-09T21:43:39.407204Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 329  requestId: <none> cookie: 167
2025-10-09T21:43:39.407207Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 330  requestId: <none> cookie: 168
2025-10-09T21:43:39.407219Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.407232Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.407236Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.407256Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.407320Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 333  requestId: <none> cookie: 169
2025-10-09T21:43:39.407328Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 334  requestId: <none> cookie: 170
2025-10-09T21:43:39.407333Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.407338Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.407404Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407410Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407427Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.407431Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 166 partNo : 0 messageNo: 335 size 18235 offset: -1
2025-10-09T21:43:39.407438Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407440Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407444Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 167 partNo : 0 messageNo: 336 size 18345 offset: -1
2025-10-09T21:43:39.407449Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407451Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407455Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 168 partNo : 0 messageNo: 337 size 18455 offset: -1
2025-10-09T21:43:39.407459Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407460Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407465Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407466Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407470Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 169 partNo : 0 messageNo: 339 size 18565 offset: -1
2025-10-09T21:43:39.407474Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407475Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407479Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 170 partNo : 0 messageNo: 340 size 18675 offset: -1
2025-10-09T21:43:39.407486Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.407490Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.407494Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.407496Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.407497Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.407499Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.407502Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.407505Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.407510Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.407512Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.407514Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.407517Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.407520Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407525Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.407533Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.407534Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407540Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 338  requestId: <none> cookie: 171
2025-10-09T21:43:39.407547Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407548Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407551Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 341  requestId: <none> cookie: 172
2025-10-09T21:43:39.407555Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 342  requestId: <none> cookie: 173
2025-10-09T21:43:39.407564Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.407569Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.407572Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.407575Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 32019 } }
2025-10-09T21:43:39.407582Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.407589Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.407607Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.407617Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# 72d2bd36-a814f644-f46002b8-b461fd0a
2025-10-09T21:43:39.407618Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407622Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407634Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 171 partNo : 0 messageNo: 343 size 18785 offset: -1
2025-10-09T21:43:39.407645Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407647Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407651Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 172 partNo : 0 messageNo: 344 size 18895 offset: -1
2025-10-09T21:43:39.407655Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407658Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407662Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 173 partNo : 0 messageNo: 345 size 19005 offset: -1
2025-10-09T21:43:39.407665Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407668Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407672Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407674Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407678Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.407687Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.407689Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.407692Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.407694Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.407696Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.407702Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.407704Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 346  requestId: <none> cookie: 174
2025-10-09T21:43:39.407709Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 347  requestId: <none> cookie: 175
2025-10-09T21:43:39.407758Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.407766Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.407784Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.407791Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.407827Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.407839Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407844Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407856Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 174 partNo : 0 messageNo: 348 size 19115 offset: -1
2025-10-09T21:43:39.407862Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407864Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407868Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 175 partNo : 0 messageNo: 349 size 19225 offset: -1
2025-10-09T21:43:39.407871Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.407874Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.407879Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.407893Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.407895Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.407898Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.407905Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 350  requestId: <none> cookie: 176
2025-10-09T21:43:39.407921Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.407938Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.407998Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.408220Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.408293Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.408317Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.408321Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.408327Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.408330Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.408337Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 176 partNo : 0 messageNo: 352 size 19335 offset: -1
2025-10-09T21:43:39.408341Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.408342Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.408385Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.408392Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.408395Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.408402Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.408411Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 146 count 9 size 149454 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.408432Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 351  requestId: <none> cookie: 177
2025-10-09T21:43:39.408437Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 353  requestId: <none> cookie: 178
2025-10-09T21:43:39.408440Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.408446Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 146 partno 0 count 9 parts 0 suffix '63' size 149454
2025-10-09T21:43:39.408453Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 149325 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.408460Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.408466Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 147, partNo: 0, Offset: 146 is stored on disk
2025-10-09T21:43:39.408469Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.408472Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 148, partNo: 0, Offset: 147 is stored on disk
2025-10-09T21:43:39.408474Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.408477Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 149, partNo: 0, Offset: 148 is stored on disk
2025-10-09T21:43:39.408478Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.408481Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 150, partNo: 0, Offset: 149 is stored on disk
2025-10-09T21:43:39.408483Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.408487Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 151, partNo: 0, Offset: 150 is stored on disk
2025-10-09T21:43:39.408489Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.408493Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 152, partNo: 0, Offset: 151 is stored on disk
2025-10-09T21:43:39.408495Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.408497Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 153, partNo: 0, Offset: 152 is stored on disk
2025-10-09T21:43:39.408500Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.408503Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 154, partNo: 0, Offset: 153 is stored on disk
2025-10-09T21:43:39.408517Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.408521Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 155, partNo: 0, Offset: 154 is stored on disk
2025-10-09T21:43:39.408545Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.408564Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.408570Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (21)
2025-10-09T21:43:39.408584Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 156 partNo 0
2025-10-09T21:43:39.408614Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 156 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 17209 count 1 nextOffset 156 batches 1
2025-10-09T21:43:39.408623Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 157 partNo 0
2025-10-09T21:43:39.408627Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 157 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 34496 count 2 nextOffset 157 batches 1
2025-10-09T21:43:39.408629Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 158 partNo 0
2025-10-09T21:43:39.408633Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 158 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 51893 count 3 nextOffset 158 batches 1
2025-10-09T21:43:39.408637Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 159 partNo 0
2025-10-09T21:43:39.408640Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 159 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 69400 count 4 nextOffset 159 batches 1
2025-10-09T21:43:39.408643Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 160 partNo 0
2025-10-09T21:43:39.408648Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 160 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 87017 count 5 nextOffset 160 batches 1
2025-10-09T21:43:39.408655Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 161 partNo 0
2025-10-09T21:43:39.408659Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 161 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 104744 count 6 nextOffset 161 batches 1
2025-10-09T21:43:39.408662Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 162 partNo 0
2025-10-09T21:43:39.408666Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 162 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 122581 count 7 nextOffset 162 batches 1
2025-10-09T21:43:39.408668Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 163 partNo 0
2025-10-09T21:43:39.408672Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 163 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 140528 count 8 nextOffset 163 batches 1
2025-10-09T21:43:39.408674Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 164 partNo 0
2025-10-09T21:43:39.408679Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 164 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 158585 count 9 nextOffset 164 batches 1
2025-10-09T21:43:39.408682Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 165 partNo 0
2025-10-09T21:43:39.408687Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 165 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 176752 count 10 nextOffset 165 batches 1
2025-10-09T21:43:39.408691Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 166 partNo 0
2025-10-09T21:43:39.408694Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 166 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 195029 count 11 nextOffset 166 batches 1
2025-10-09T21:43:39.408698Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 167 partNo 0
2025-10-09T21:43:39.408700Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 167 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 213416 count 12 nextOffset 167 batches 1
2025-10-09T21:43:39.408703Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 168 partNo 0
2025-10-09T21:43:39.408706Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 168 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 231913 count 13 nextOffset 168 batches 1
2025-10-09T21:43:39.408709Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 169 partNo 0
2025-10-09T21:43:39.408712Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 169 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 250520 count 14 nextOffset 169 batches 1
2025-10-09T21:43:39.410098Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 147 offsets: 146 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.412844Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 149083, now ReadSizeBudget = 0, ReadSizeServerDelta = 375205
2025-10-09T21:43:39.466438Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 147
2025-10-09T21:43:39.466502Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 0, ReadSizeServerDelta = 375205
2025-10-09T21:43:39.466636Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 148 offsets: 147 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.466645Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 148
2025-10-09T21:43:39.466680Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 149 offsets: 148 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.466689Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 149
2025-10-09T21:43:39.466677Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (146-154)
2025-10-09T21:43:39.466718Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 150 offsets: 149 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.466717Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 149083 to budget
2025-10-09T21:43:39.466722Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 150
2025-10-09T21:43:39.466727Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 149083, ReadSizeServerDelta = 375205
2025-10-09T21:43:39.466753Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 151 offsets: 150 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.466757Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 151
2025-10-09T21:43:39.466768Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.466792Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 152 offsets: 151 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.466796Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 152
2025-10-09T21:43:39.466798Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (146-146)
2025-10-09T21:43:39.466815Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (147-147)
2025-10-09T21:43:39.466814Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 382354, now ReadSizeBudget = 0, ReadSizeServerDelta = 141934
2025-10-09T21:43:39.466839Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 2} (148-148)
2025-10-09T21:43:39.466848Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 3} (149-149)
2025-10-09T21:43:39.466876Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 4} (150-150)
2025-10-09T21:43:39.466884Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 5} (151-151)
2025-10-09T21:43:39.408715Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 170 partNo 0
2025-10-09T21:43:39.466892Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 6} (152-152)
2025-10-09T21:43:39.408718Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 170 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 269237 count 15 nextOffset 170 batches 1
2025-10-09T21:43:39.466899Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 7} (153-153)
2025-10-09T21:43:39.408721Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 171 partNo 0
2025-10-09T21:43:39.408724Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 171 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 288064 count 16 nextOffset 171 batches 1
2025-10-09T21:43:39.408727Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 172 partNo 0
2025-10-09T21:43:39.408731Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 172 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 307001 count 17 nextOffset 172 batches 1
2025-10-09T21:43:39.408734Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 173 partNo 0
2025-10-09T21:43:39.408738Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 173 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 326048 count 18 nextOffset 173 batches 1
2025-10-09T21:43:39.408741Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 174 partNo 0
2025-10-09T21:43:39.466928Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 8} (154-154)
2025-10-09T21:43:39.408745Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 174 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 345205 count 19 nextOffset 174 batches 1
2025-10-09T21:43:39.408747Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 175 partNo 0
2025-10-09T21:43:39.408750Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 175 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 364472 count 20 nextOffset 175 batches 1
2025-10-09T21:43:39.408753Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 176 partNo 0
2025-10-09T21:43:39.408757Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 176 partNo 0 FormedBlobsCount 0 NewHead: Offset 155 PartNo 0 PackedSize 383849 count 21 nextOffset 176 batches 1
2025-10-09T21:43:39.408759Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.409363Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 155,21 HeadOffset 155 endOffset 155 curOffset 176 d0000000000_00000000000000000155_00000_0000000021_00000? size 383262 WTime 1760046219408
2025-10-09T21:43:39.409673Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.409686Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 354  requestId: <none> cookie: 179
2025-10-09T21:43:39.409695Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 294  requestId: <none> cookie: 147
2025-10-09T21:43:39.409701Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 295  requestId: <none> cookie: 148
2025-10-09T21:43:39.466977Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 0, ReadSizeServerDelta = 141934
2025-10-09T21:43:39.467002Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 153 offsets: 152 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.467010Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 153
2025-10-09T21:43:39.467043Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 154 offsets: 153 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.467046Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 154
2025-10-09T21:43:39.467045Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (155-175)
2025-10-09T21:43:39.467060Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 382354 to budget
2025-10-09T21:43:39.467066Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 382354, ReadSizeServerDelta = 141934
2025-10-09T21:43:39.409708Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 299  requestId: <none> cookie: 149
2025-10-09T21:43:39.467074Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 155 offsets: 154 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.467077Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 155
2025-10-09T21:43:39.409713Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 300  requestId: <none> cookie: 150
2025-10-09T21:43:39.467104Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 156 offsets: 155 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.467108Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 156
2025-10-09T21:43:39.409718Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 301  requestId: <none> cookie: 151
2025-10-09T21:43:39.409722Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 305  requestId: <none> cookie: 152
2025-10-09T21:43:39.409727Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 306  requestId: <none> cookie: 153
2025-10-09T21:43:39.409732Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 308  requestId: <none> cookie: 154
2025-10-09T21:43:39.409737Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 309  requestId: <none> cookie: 155
2025-10-09T21:43:39.409743Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.409754Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 155 partNo 0 count 21 size 383262
2025-10-09T21:43:39.409811Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.409817Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.467149Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.409821Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.409824Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.409828Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.409831Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.409835Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.409837Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.467170Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (155-155)
2025-10-09T21:43:39.409840Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.409842Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.409845Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.467182Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (156-156)
2025-10-09T21:43:39.409848Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.410013Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 146 EndOffset 146 newEndOffset 155 commitOffset 0 clientCommitOffset 0 cookie 13 readingFinished 0 firstRead 0
2025-10-09T21:43:39.410017Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 146 endOffset 155
2025-10-09T21:43:39.410024Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.410040Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.410050Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 177 partNo : 0 messageNo: 355 size 19445 offset: -1
2025-10-09T21:43:39.410055Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.410057Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.410061Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 178 partNo : 0 messageNo: 356 size 19555 offset: -1
2025-10-09T21:43:39.410064Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.467216Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 2} (157-157)
2025-10-09T21:43:39.410066Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.410070Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 179 partNo : 0 messageNo: 357 size 19665 offset: -1
2025-10-09T21:43:39.467222Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 3} (158-158)
2025-10-09T21:43:39.410073Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.410080Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.410082Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.410084Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.410086Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.410088Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.467240Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 4} (159-159)
2025-10-09T21:43:39.410097Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 146, endOffset# 155, WTime# 1760046219402, sizeLag# 149454
2025-10-09T21:43:39.467246Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 5} (160-160)
2025-10-09T21:43:39.410099Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:43:39.467240Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 157 offsets: 156 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.467258Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 6} (161-161)
2025-10-09T21:43:39.467266Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 157
>>> event from dataHandler: 2025-10-09T21:43:39.467270Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 7} (162-162)
2025-10-09T21:43:39.467324Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 158 offsets: 157 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.467329Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 158
DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..16071 bytes.. Information: { Offset: 146 ProducerId: "src_id" SeqNo: 147 CreateTime: 2025-10-09T21:43:39.379000Z WriteTime: 2025-10-09T21:43:39.405000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..16181 bytes.. Information: { Offset: 147 ProducerId: "src_id" SeqNo: 148 CreateTime: 2025-10-09T21:43:39.379000Z WriteTime: 2025-10-09T21:43:39.405000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..16291 bytes.. Information: { Offset: 148 ProducerId: "src_id" SeqNo: 149 CreateTime: 2025-10-09T21:43:39.379000Z WriteTime: 2025-10-09T21:43:39.405000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..16401 bytes.. Information: { Offset: 149 ProducerId: "src_id" SeqNo: 150 CreateTime: 2025-10-09T21:43:39.379000Z WriteTime: 2025-10-09T21:43:39.405000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..16511 bytes.. Information: { Offset: 150 ProducerId: "src_id" SeqNo: 151 CreateTime: 2025-10-09T21:43:39.379000Z WriteTime: 2025-10-09T21:43:39.405000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..16621 bytes.. Information: { Offset: 151 ProducerId: "src_id" SeqNo: 152 CreateTime: 2025-10-09T21:43:39.379000Z WriteTime: 2025-10-09T21:43:39.405000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..16731 bytes.. Information: { Offset: 152 ProducerId: "src_id" SeqNo: 153 CreateTime: 2025-10-09T21:43:39.379000Z WriteTime: 2025-10-09T21:43:39.405000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..16841 bytes.. Information: { Offset: 153 ProducerId: "src_id" SeqNo: 154 CreateTime: 2025-10-09T21:43:39.379000Z WriteTime: 2025-10-09T21:43:39.405000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..16951 bytes.. Information: { Offset: 154 ProducerId: "src_id" SeqNo: 155 CreateTime: 2025-10-09T21:43:39.379000Z WriteTime: 2025-10-09T21:43:39.405000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.410110Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# e8c1941e-e1991cf1-16c40e3d-6e189e21, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 10, size# 179344, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.467349Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 159 offsets: 158 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.410122Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 10 maxSize 179344 maxTimeLagMs 0 readTimestampMs 0 readOffset 146 EndOffset 155 ClientCommitOffset 0 committedOffset 0 Guid e8c1941e-e1991cf1-16c40e3d-6e189e21
2025-10-09T21:43:39.467383Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 159
2025-10-09T21:43:39.410148Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.410179Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.410182Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.410188Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.410212Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 16 Topic 'rt3.dc1--test-topic' partition 0 user user offset 146 count 10 size 179344 endOffset 155 max time lag 0ms effective offset 146
2025-10-09T21:43:39.410228Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 16 added 1 blobs, size 149454 count 9 last offset 146, current partition end offset: 155
2025-10-09T21:43:39.467420Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 9, size 148599 bytes
2025-10-09T21:43:39.410231Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 16. Send blob request.
2025-10-09T21:43:39.410236Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.410238Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.467430Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.410250Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 146 partno 0 count 9 parts_count 0 source 1 size 149454 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.410260Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 16. All 1 blobs are from cache.
2025-10-09T21:43:39.410273Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.410278Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 146 partno 0 count 9 parts 0 suffix '63'
2025-10-09T21:43:39.410343Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.467458Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 160 offsets: 159 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.410369Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 146 totakecount 9 count 9 size 149431 from pos 0 cbcount 9
2025-10-09T21:43:39.410377Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.467480Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 160
2025-10-09T21:43:39.410398Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 358  requestId: <none> cookie: 180
2025-10-09T21:43:39.410404Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.410413Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 146
2025-10-09T21:43:39.410443Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.410445Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.410450Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.410452Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.410459Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 180 partNo : 0 messageNo: 360 size 19775 offset: -1
2025-10-09T21:43:39.410493Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.467508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 161 offsets: 160 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.410499Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.467511Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 161
2025-10-09T21:43:39.410503Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 359  requestId: <none> cookie: 181
2025-10-09T21:43:39.410515Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.410554Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.410558Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.410563Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 181 partNo : 0 messageNo: 361 size 19885 offset: -1
2025-10-09T21:43:39.467529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 162 offsets: 161 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.467543Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 162
2025-10-09T21:43:39.467561Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 163 offsets: 162 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.467563Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 163
2025-10-09T21:43:39.410566Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.410569Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.467582Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 164 offsets: 163 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.410602Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 155 Result { Offset: 146 Data: "... 16144 bytes ..." SourceId: "\000src_id" SeqNo: 147 WriteTimestampMS: 1760046219405 CreateTimestampMS: 1760046219379 UncompressedSize: 16071 PartitionKey: "" ExplicitHash: "" } Result { Offset: 147 Data: "... 16254 bytes ..." SourceId: "\000src_id" SeqNo: 148 WriteTimestampMS: 1760046219405 CreateTimestampMS: 1760046219379 UncompressedSize: 16181 PartitionKey: "" ExplicitHash: "" } Result { Offset: 148 Data: "... 16364 bytes ..." SourceId: "\000src_id" SeqNo: 149 WriteTimestampMS: 1760046219405 CreateTimestampMS: 1760046219379 UncompressedSize: 16291 PartitionKey: "" ExplicitHash: "" } Result { Offset: 149 Data: "... 16475 bytes ..." SourceId: "\000src_id" SeqNo: 150 WriteTimestampMS: 1760046219405 CreateTimestampMS: 1760046219379 UncompressedSize: 16401 PartitionKey: "" ExplicitHash: "" } Result { Offset: 150 Data: "... 16585 bytes ..." SourceId: "\000src_id" SeqNo: 151 WriteTimestampMS: 1760046219405 CreateTimestampMS: 1760046219379 UncompressedSize: 16511 PartitionKey: "" ExplicitHash: "" } Result { Offset: 151 Data: "... 16695 bytes ..." SourceId: "\000src_id" SeqNo: 152 WriteTimestampMS: 1760046219405 CreateTimestampMS: 1760046219379 UncompressedSize: 16621 PartitionKey: "" ExplicitHash: "" } Result { Offset: 152 Data: "... 16805 bytes ..." SourceId: "\000src_id" SeqNo: 153 WriteTimestampMS: 1760046219405 CreateTimestampMS: 1760046219379 UncompressedSize: 16731 PartitionKey: "" ExplicitHash: "" } Result { Offset: 153 Data: "... 16915 bytes ..." SourceId: "\000src_id" SeqNo: 154 WriteTimestampMS: 1760046219405 CreateTimestampMS: 1760046219379 UncompressedSize: 16841 PartitionKey: "" ExplicitHash: "" } Result { Offset: 154 Data: "... 17025 bytes ..." SourceId: "\000src_id" SeqNo: 155 WriteTimestampMS: 1760046219405 CreateTimestampMS: 1760046219379 UncompressedSize: 16951 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 18446744073709551430 RealReadOffset: 154 WaitQuotaTimeMs: 0 EndOffset: 155 StartOffset: 0 } Cookie: 146 }
2025-10-09T21:43:39.467585Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 164
2025-10-09T21:43:39.410646Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.410682Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 14 from offset 155
2025-10-09T21:43:39.467604Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 165 offsets: 164 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.410689Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 155 ReadOffset 155 ReadGuid e8c1941e-e1991cf1-16c40e3d-6e189e21 has messages 1
2025-10-09T21:43:39.467607Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 165
2025-10-09T21:43:39.410717Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# e8c1941e-e1991cf1-16c40e3d-6e189e21, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 149083
2025-10-09T21:43:39.410720Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.467622Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 166 offsets: 165 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.410725Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# e8c1941e-e1991cf1-16c40e3d-6e189e21
2025-10-09T21:43:39.467625Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 166
2025-10-09T21:43:39.410747Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.410750Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.410758Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 362  requestId: <none> cookie: 182
2025-10-09T21:43:39.410763Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.410792Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.410795Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.410801Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 182 partNo : 0 messageNo: 363 size 19995 offset: -1
2025-10-09T21:43:39.410804Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.410808Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.410812Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.467657Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 167 offsets: 166 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.410847Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.467660Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 167
2025-10-09T21:43:39.410895Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 0
2025-10-09T21:43:39.410931Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.410935Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.410949Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 364  requestId: <none> cookie: 183
2025-10-09T21:43:39.410965Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.467679Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 168 offsets: 167 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.467690Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 168
2025-10-09T21:43:39.467709Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 169 offsets: 168 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.467712Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 169
2025-10-09T21:43:39.467729Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 170 offsets: 169 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.467732Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 170
2025-10-09T21:43:39.467762Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 171 offsets: 170 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.467765Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 171
2025-10-09T21:43:39.467780Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 172 offsets: 171 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.467783Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 172
2025-10-09T21:43:39.467799Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 173 offsets: 172 already_written: false write_statistics { persist_duration_ms: 5 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.467804Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 173
2025-10-09T21:43:39.467823Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 174 offsets: 173 already_written: false write_statistics { persist_duration_ms: 5 }
2025-10-09T21:43:39.467826Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 174
2025-10-09T21:43:39.467841Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 175 offsets: 174 already_written: false write_statistics { persist_duration_ms: 5 }
2025-10-09T21:43:39.467844Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 175
2025-10-09T21:43:39.467860Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 176 offsets: 175 already_written: false write_statistics { persist_duration_ms: 5 }
2025-10-09T21:43:39.467863Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 176
2025-10-09T21:43:39.467881Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 177 offsets: 176 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.467884Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 177
2025-10-09T21:43:39.467900Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 178 offsets: 177 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.467903Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 178
2025-10-09T21:43:39.467920Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 179 offsets: 178 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.467925Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 179
2025-10-09T21:43:39.467940Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 180 offsets: 179 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.467943Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 180
2025-10-09T21:43:39.467959Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 181 offsets: 180 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.467961Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 181
2025-10-09T21:43:39.467977Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 182 offsets: 181 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.467981Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 182
2025-10-09T21:43:39.467998Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 183 offsets: 182 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468000Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 183
2025-10-09T21:43:39.468017Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 184 offsets: 183 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468020Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 184
2025-10-09T21:43:39.468037Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 185 offsets: 184 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468040Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 185
2025-10-09T21:43:39.468078Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 186 offsets: 185 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468080Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 186
2025-10-09T21:43:39.468096Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 187 offsets: 186 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468099Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 187
2025-10-09T21:43:39.468113Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 188 offsets: 187 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468116Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 188
2025-10-09T21:43:39.468133Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 189 offsets: 188 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468136Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 189
2025-10-09T21:43:39.468153Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 190 offsets: 189 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468156Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 190
2025-10-09T21:43:39.468173Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 191 offsets: 190 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468175Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 191
2025-10-09T21:43:39.468191Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 192 offsets: 191 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468194Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 192
2025-10-09T21:43:39.468212Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 193 offsets: 192 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468214Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 193
2025-10-09T21:43:39.468231Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 194 offsets: 193 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468234Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 194
2025-10-09T21:43:39.468250Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 195 offsets: 194 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468255Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 195
2025-10-09T21:43:39.468273Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 196 offsets: 195 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468275Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 196
2025-10-09T21:43:39.468292Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 197 offsets: 196 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468295Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 197
2025-10-09T21:43:39.468312Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 198 offsets: 197 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468315Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 198
2025-10-09T21:43:39.468331Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 199 offsets: 198 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468335Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 199
2025-10-09T21:43:39.468350Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 200 offsets: 199 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468353Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 200
2025-10-09T21:43:39.468372Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 201 offsets: 200 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468375Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 201
2025-10-09T21:43:39.468390Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 202 offsets: 201 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468393Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 202
2025-10-09T21:43:39.468410Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 203 offsets: 202 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468413Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 203
2025-10-09T21:43:39.468433Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 204 offsets: 203 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468436Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 204
2025-10-09T21:43:39.468454Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 205 offsets: 204 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468457Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 205
2025-10-09T21:43:39.468472Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 206 offsets: 205 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468475Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 206
2025-10-09T21:43:39.468491Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 207 offsets: 206 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.468494Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 207
2025-10-09T21:43:39.475181Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 208 offsets: 207 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.475194Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 208
2025-10-09T21:43:39.475238Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 209 offsets: 208 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.475242Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 209
2025-10-09T21:43:39.475259Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 210 offsets: 209 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.475262Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 210
2025-10-09T21:43:39.475284Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 211 offsets: 210 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.475291Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 211
2025-10-09T21:43:39.475308Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 212 offsets: 211 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.475311Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 212
2025-10-09T21:43:39.475328Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 213 offsets: 212 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.475331Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 213
2025-10-09T21:43:39.475347Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 214 offsets: 213 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.475350Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 214
2025-10-09T21:43:39.475366Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 215 offsets: 214 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.475369Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 215
2025-10-09T21:43:39.475384Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 216 offsets: 215 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.475390Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 216
2025-10-09T21:43:39.475409Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 217 offsets: 216 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 4 }
2025-10-09T21:43:39.475411Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 217
2025-10-09T21:43:39.475428Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 218 offsets: 217 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.475431Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 218
2025-10-09T21:43:39.475448Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 219 offsets: 218 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.475451Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 219
2025-10-09T21:43:39.475468Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 220 offsets: 219 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.475471Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 220
2025-10-09T21:43:39.475487Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 221 offsets: 220 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.475490Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 221
2025-10-09T21:43:39.475505Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 222 offsets: 221 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.475508Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 222
2025-10-09T21:43:39.475526Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 223 offsets: 222 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.475529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 223
2025-10-09T21:43:39.475545Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 224 offsets: 223 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.475548Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 224
2025-10-09T21:43:39.475567Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 225 offsets: 224 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.475570Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 225
2025-10-09T21:43:39.475586Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 226 offsets: 225 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.475589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 226
2025-10-09T21:43:39.475606Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 227 offsets: 226 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.475611Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 227
2025-10-09T21:43:39.475627Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 228 offsets: 227 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.475629Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 228
2025-10-09T21:43:39.475661Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 229 offsets: 228 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.475664Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 229
2025-10-09T21:43:39.475681Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 230 offsets: 229 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.475684Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 230
2025-10-09T21:43:39.475699Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 231 offsets: 230 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.475702Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 231
2025-10-09T21:43:39.475734Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 232 offsets: 231 already_written: false write_statistics { persist_duration_ms: 3 }
2025-10-09T21:43:39.475737Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 232
2025-10-09T21:43:39.475756Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 233 offsets: 232 already_written: false write_statistics { persist_duration_ms: 3 }
2025-10-09T21:43:39.475759Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 233
2025-10-09T21:43:39.475775Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 234 offsets: 233 already_written: false write_statistics { persist_duration_ms: 3 }
2025-10-09T21:43:39.475778Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 234
2025-10-09T21:43:39.475800Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 235 offsets: 234 already_written: false write_statistics { persist_duration_ms: 3 }
2025-10-09T21:43:39.475803Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 235
2025-10-09T21:43:39.475819Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 236 offsets: 235 already_written: false write_statistics { persist_duration_ms: 3 }
2025-10-09T21:43:39.475822Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 236
2025-10-09T21:43:39.475838Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 237 offsets: 236 already_written: false write_statistics { persist_duration_ms: 3 }
2025-10-09T21:43:39.475841Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 237
2025-10-09T21:43:39.475858Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 238 offsets: 237 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.475861Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 238
2025-10-09T21:43:39.475878Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 239 offsets: 238 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.475881Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 239
2025-10-09T21:43:39.475896Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 240 offsets: 239 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.475899Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 240
2025-10-09T21:43:39.475915Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 241 offsets: 240 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.475918Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 241
2025-10-09T21:43:39.475933Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 242 offsets: 241 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.475936Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 242
2025-10-09T21:43:39.475954Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 243 offsets: 242 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.475958Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 243
2025-10-09T21:43:39.475974Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 244 offsets: 243 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.475977Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 244
2025-10-09T21:43:39.475992Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 245 offsets: 244 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.475994Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 245
2025-10-09T21:43:39.476011Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 246 offsets: 245 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.476014Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 246
2025-10-09T21:43:39.476031Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 247 offsets: 246 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.476033Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 247
2025-10-09T21:43:39.476048Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 248 offsets: 247 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.476051Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 248
2025-10-09T21:43:39.476067Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 249 offsets: 248 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.476070Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 249
2025-10-09T21:43:39.476084Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 250 offsets: 249 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.476086Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 250
2025-10-09T21:43:39.476101Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 251 offsets: 250 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.476105Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 251
2025-10-09T21:43:39.476121Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 252 offsets: 251 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.476124Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 252
2025-10-09T21:43:39.476139Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 253 offsets: 252 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.476142Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 253
2025-10-09T21:43:39.476157Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 254 offsets: 253 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.476160Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 254
2025-10-09T21:43:39.476174Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 255 offsets: 254 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.476177Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 255
2025-10-09T21:43:39.476192Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 256 offsets: 255 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.476195Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 256
2025-10-09T21:43:39.476211Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 257 offsets: 256 already_written: false write_statistics { persist_duration_ms: 6 queued_in_partition_duration_ms: 1 }
2025-10-09T21:43:39.476213Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 257
2025-10-09T21:43:39.476229Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 258 offsets: 257 already_written: false write_statistics { persist_duration_ms: 6 }
2025-10-09T21:43:39.476231Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 258
2025-10-09T21:43:39.476247Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 259 offsets: 258 already_written: false write_statistics { persist_duration_ms: 6 }
2025-10-09T21:43:39.476253Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 259
2025-10-09T21:43:39.476268Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 260 offsets: 259 already_written: false write_statistics { persist_duration_ms: 6 }
2025-10-09T21:43:39.476272Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 260
2025-10-09T21:43:39.476287Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 261 offsets: 260 already_written: false write_statistics { persist_duration_ms: 6 }
2025-10-09T21:43:39.476290Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 261
2025-10-09T21:43:39.476311Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 262 offsets: 261 already_written: false write_statistics { persist_duration_ms: 6 }
2025-10-09T21:43:39.476314Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 262
2025-10-09T21:43:39.476329Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 263 offsets: 262 already_written: false write_statistics { persist_duration_ms: 6 }
2025-10-09T21:43:39.476331Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 263
2025-10-09T21:43:39.476347Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 264 offsets: 263 already_written: false write_statistics { persist_duration_ms: 6 }
2025-10-09T21:43:39.476349Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 264
2025-10-09T21:43:39.476365Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 265 offsets: 264 already_written: false write_statistics { persist_duration_ms: 6 }
2025-10-09T21:43:39.476368Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 265
2025-10-09T21:43:39.476382Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 266 offsets: 265 already_written: false write_statistics { persist_duration_ms: 6 }
2025-10-09T21:43:39.476385Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 266
2025-10-09T21:43:39.476403Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 267 offsets: 266 already_written: false write_statistics { persist_duration_ms: 6 }
2025-10-09T21:43:39.476405Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 267
2025-10-09T21:43:39.476420Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 268 offsets: 267 already_written: false write_statistics { persist_duration_ms: 6 }
2025-10-09T21:43:39.476422Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 268
2025-10-09T21:43:39.476438Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 269 offsets: 268 already_written: false write_statistics { persist_duration_ms: 6 }
2025-10-09T21:43:39.476440Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 269
2025-10-09T21:43:39.476456Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 270 offsets: 269 already_written: false write_statistics { persist_duration_ms: 6 }
2025-10-09T21:43:39.476459Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 270
2025-10-09T21:43:39.476474Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 271 offsets: 270 already_written: false write_statistics { persist_duration_ms: 6 }
2025-10-09T21:43:39.476477Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 271
2025-10-09T21:43:39.476492Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 272 offsets: 271 already_written: false write_statistics { persist_duration_ms: 6 }
2025-10-09T21:43:39.476495Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 272
2025-10-09T21:43:39.476510Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 273 offsets: 272 already_written: false write_statistics { persist_duration_ms: 6 }
2025-10-09T21:43:39.476513Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 273
2025-10-09T21:43:39.476529Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 274 offsets: 273 already_written: false write_statistics { persist_duration_ms: 4 queued_in_partition_duration_ms: 6 }
2025-10-09T21:43:39.476531Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 274
2025-10-09T21:43:39.476547Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 275 offsets: 274 already_written: false write_statistics { persist_duration_ms: 4 queued_in_partition_duration_ms: 6 }
2025-10-09T21:43:39.476550Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 275
2025-10-09T21:43:39.476566Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 276 offsets: 275 already_written: false write_statistics { persist_duration_ms: 4 queued_in_partition_duration_ms: 6 }
2025-10-09T21:43:39.476574Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 276
2025-10-09T21:43:39.476589Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 277 offsets: 276 already_written: false write_statistics { persist_duration_ms: 4 queued_in_partition_duration_ms: 6 }
2025-10-09T21:43:39.476592Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 277
2025-10-09T21:43:39.476606Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 278 offsets: 277 already_written: false write_statistics { persist_duration_ms: 4 queued_in_partition_duration_ms: 6 }
2025-10-09T21:43:39.476609Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 278
2025-10-09T21:43:39.476625Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 279 offsets: 278 already_written: false write_statistics { persist_duration_ms: 4 queued_in_partition_duration_ms: 6 }
2025-10-09T21:43:39.476627Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 279
2025-10-09T21:43:39.476642Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 280 offsets: 279 already_written: false write_statistics { persist_duration_ms: 4 queued_in_partition_duration_ms: 6 }
2025-10-09T21:43:39.476645Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 280
2025-10-09T21:43:39.476661Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 281 offsets: 280 already_written: false write_statistics { persist_duration_ms: 4 queued_in_partition_duration_ms: 6 }
2025-10-09T21:43:39.476663Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 281
2025-10-09T21:43:39.476681Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 282 offsets: 281 already_written: false write_statistics { persist_duration_ms: 4 queued_in_partition_duration_ms: 6 }
2025-10-09T21:43:39.476683Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 282
2025-10-09T21:43:39.476698Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 283 offsets: 282 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 7 }
2025-10-09T21:43:39.476701Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 283
2025-10-09T21:43:39.476716Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 284 offsets: 283 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 7 }
2025-10-09T21:43:39.476719Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 284
2025-10-09T21:43:39.476734Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 285 offsets: 284 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 7 }
2025-10-09T21:43:39.476737Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 285
2025-10-09T21:43:39.476752Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 286 offsets: 285 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 7 }
2025-10-09T21:43:39.476755Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 286
2025-10-09T21:43:39.476771Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 287 offsets: 286 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 7 }
2025-10-09T21:43:39.476775Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 287
2025-10-09T21:43:39.476795Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 288 offsets: 287 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 7 }
2025-10-09T21:43:39.476797Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 288
2025-10-09T21:43:39.476816Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 289 offsets: 288 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 7 }
2025-10-09T21:43:39.476818Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 289
2025-10-09T21:43:39.476835Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 290 offsets: 289 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 7 }
2025-10-09T21:43:39.476838Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 290
2025-10-09T21:43:39.476853Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 291 offsets: 290 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.476856Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 291
2025-10-09T21:43:39.476872Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 292 offsets: 291 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.476878Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 292
2025-10-09T21:43:39.476908Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 293 offsets: 292 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.476911Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 293
2025-10-09T21:43:39.476926Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 294 offsets: 293 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.476929Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 294
2025-10-09T21:43:39.476944Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 295 offsets: 294 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.476947Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 295
2025-10-09T21:43:39.476963Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 296 offsets: 295 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.476965Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 296
2025-10-09T21:43:39.476981Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 297 offsets: 296 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.476984Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 297
2025-10-09T21:43:39.476999Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 298 offsets: 297 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.477002Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 298
2025-10-09T21:43:39.477019Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 299 offsets: 298 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.477022Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 299
2025-10-09T21:43:39.477040Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session got write response: sequence_numbers: 300 offsets: 299 already_written: false write_statistics { persist_duration_ms: 3 queued_in_partition_duration_ms: 2 }
2025-10-09T21:43:39.477043Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: acknoledged message 300
2025-10-09T21:43:39.467456Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 8} (163-163)
2025-10-09T21:43:39.477142Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 9} (164-164)
2025-10-09T21:43:39.477152Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 10} (165-165)
2025-10-09T21:43:39.477160Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 11} (166-166)
2025-10-09T21:43:39.477169Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 12} (167-167)
2025-10-09T21:43:39.477177Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 13} (168-168)
2025-10-09T21:43:39.477184Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 14} (169-169)
2025-10-09T21:43:39.477209Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 15} (170-170)
2025-10-09T21:43:39.477311Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 16} (171-171)
2025-10-09T21:43:39.477318Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 17} (172-172)
2025-10-09T21:43:39.477373Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 18} (173-173)
2025-10-09T21:43:39.477395Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 19} (174-174)
2025-10-09T21:43:39.477414Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 20} (175-175)
2025-10-09T21:43:39.410974Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.473276Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 768229, now ReadSizeBudget = 0, ReadSizeServerDelta = -243941
2025-10-09T21:43:39.411025Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411029Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411035Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 183 partNo : 0 messageNo: 365 size 20105 offset: -1
2025-10-09T21:43:39.411041Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.411045Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.411049Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.411069Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.411083Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411089Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411101Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 366  requestId: <none> cookie: 184
2025-10-09T21:43:39.411109Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.411120Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.411125Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411127Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411136Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 184 partNo : 0 messageNo: 367 size 20215 offset: -1
2025-10-09T21:43:39.411141Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411143Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411147Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.411154Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.411167Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 368  requestId: <none> cookie: 185
2025-10-09T21:43:39.411182Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.411203Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.411238Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.411291Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411295Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411302Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 185 partNo : 0 messageNo: 369 size 20325 offset: -1
2025-10-09T21:43:39.411306Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.411307Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411308Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411312Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.411316Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.411330Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 370  requestId: <none> cookie: 186
2025-10-09T21:43:39.411341Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.411350Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.411352Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411354Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411360Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 186 partNo : 0 messageNo: 371 size 20435 offset: -1
2025-10-09T21:43:39.411363Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.411366Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.411366Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.411369Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411371Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411376Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 372  requestId: <none> cookie: 187
2025-10-09T21:43:39.411394Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.411401Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.411409Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411413Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411421Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 187 partNo : 0 messageNo: 373 size 20545 offset: -1
2025-10-09T21:43:39.411425Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411427Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411432Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.411437Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.411438Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.411444Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 374  requestId: <none> cookie: 188
2025-10-09T21:43:39.411549Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.411564Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.411615Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411618Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411629Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 188 partNo : 0 messageNo: 375 size 20655 offset: -1
2025-10-09T21:43:39.411635Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.411641Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.411647Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.411655Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.411681Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411683Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411691Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 376  requestId: <none> cookie: 189
2025-10-09T21:43:39.411702Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.411706Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.411709Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.411740Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411743Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411749Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 377  requestId: <none> cookie: 190
2025-10-09T21:43:39.411753Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.411760Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.411764Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.411766Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411768Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411769Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.411773Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 189 partNo : 0 messageNo: 378 size 20765 offset: -1
2025-10-09T21:43:39.411778Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411780Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411783Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.411786Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.411801Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.411801Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411806Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.411809Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411814Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 190 partNo : 0 messageNo: 380 size 20875 offset: -1
2025-10-09T21:43:39.411818Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 379  requestId: <none> cookie: 191
2025-10-09T21:43:39.411822Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411823Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411827Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411829Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411832Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.411838Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.411842Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.411869Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.411872Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.411875Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.411879Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411881Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411885Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 191 partNo : 0 messageNo: 383 size 20985 offset: -1
2025-10-09T21:43:39.411887Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.411896Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 155 count 21 size 383262 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.411902Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 381  requestId: <none> cookie: 192
2025-10-09T21:43:39.411905Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 382  requestId: <none> cookie: 193
2025-10-09T21:43:39.411909Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411911Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411913Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.411913Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 155 partno 0 count 21 parts 0 suffix '63' size 383262
2025-10-09T21:43:39.411915Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411917Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411919Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.411920Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.411923Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.411925Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.411925Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.411929Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.411932Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411934Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411939Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 192 partNo : 0 messageNo: 386 size 21095 offset: -1
2025-10-09T21:43:39.411943Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411945Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411949Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 193 partNo : 0 messageNo: 387 size 21205 offset: -1
2025-10-09T21:43:39.411954Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 384  requestId: <none> cookie: 194
2025-10-09T21:43:39.411957Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 385  requestId: <none> cookie: 195
2025-10-09T21:43:39.411960Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.411962Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.411963Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.411965Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.411967Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.411967Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.411970Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.411970Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411972Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411977Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 388  requestId: <none> cookie: 196
2025-10-09T21:43:39.411981Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.411981Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411983Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411989Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 194 partNo : 0 messageNo: 389 size 21315 offset: -1
2025-10-09T21:43:39.411992Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.411994Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.411998Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 195 partNo : 0 messageNo: 390 size 21425 offset: -1
2025-10-09T21:43:39.412001Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.412003Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.412006Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.412010Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.412015Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 196 partNo : 0 messageNo: 392 size 21535 offset: -1
2025-10-09T21:43:39.412019Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.412021Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.412023Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.412027Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.412030Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.412032Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.412035Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 391  requestId: <none> cookie: 197
2025-10-09T21:43:39.412040Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.412048Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.412107Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.412113Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.412142Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.412157Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.412161Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.412172Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 197 partNo : 0 messageNo: 393 size 21645 offset: -1
2025-10-09T21:43:39.412176Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.412182Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.412186Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.412188Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.412193Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.412201Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.412210Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 394  requestId: <none> cookie: 198
2025-10-09T21:43:39.412215Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 395  requestId: <none> cookie: 199
2025-10-09T21:43:39.412219Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.412222Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.412227Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.412275Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.412284Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.412312Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.412322Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.412349Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.412362Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.412368Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.412378Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 198 partNo : 0 messageNo: 396 size 21755 offset: -1
2025-10-09T21:43:39.412384Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.412386Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.412391Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 199 partNo : 0 messageNo: 397 size 21865 offset: -1
2025-10-09T21:43:39.412395Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.412396Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.412397Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.412401Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.412403Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.412407Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.412413Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.412415Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.412419Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.412426Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 398  requestId: <none> cookie: 200
2025-10-09T21:43:39.412427Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.412431Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 399  requestId: <none> cookie: 201
2025-10-09T21:43:39.412433Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.412439Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.412448Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.412477Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.412504Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.412563Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.412628Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.412684Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.412710Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.412770Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.412822Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.412868Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.412952Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.412998Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.413078Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.413123Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413129Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413138Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413140Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413146Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413148Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413160Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 200 partNo : 0 messageNo: 402 size 21975 offset: -1
2025-10-09T21:43:39.413173Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.413179Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413181Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413186Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 201 partNo : 0 messageNo: 403 size 22085 offset: -1
2025-10-09T21:43:39.413190Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413192Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413196Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413198Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413201Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413205Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413216Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.413224Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.413227Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.413230Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.413239Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 400  requestId: <none> cookie: 202
2025-10-09T21:43:39.413257Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.413261Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 401  requestId: <none> cookie: 203
2025-10-09T21:43:39.413264Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 404  requestId: <none> cookie: 204
2025-10-09T21:43:39.413265Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.413267Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 405  requestId: <none> cookie: 205
2025-10-09T21:43:39.413270Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 406  requestId: <none> cookie: 206
2025-10-09T21:43:39.413278Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.413282Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.413290Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413293Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413299Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 202 partNo : 0 messageNo: 407 size 22195 offset: -1
2025-10-09T21:43:39.413302Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.413305Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.413340Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.413349Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.413353Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.413368Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.413375Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.413381Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.413402Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413406Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413413Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 203 partNo : 0 messageNo: 408 size 22305 offset: -1
2025-10-09T21:43:39.413418Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413420Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413424Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 204 partNo : 0 messageNo: 409 size 22415 offset: -1
2025-10-09T21:43:39.413427Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413428Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.413429Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413433Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413434Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.413434Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413438Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 205 partNo : 0 messageNo: 411 size 22525 offset: -1
2025-10-09T21:43:39.413444Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.413453Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413454Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413460Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 206 partNo : 0 messageNo: 412 size 22635 offset: -1
2025-10-09T21:43:39.413464Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.413468Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.413471Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.413473Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.413477Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.413479Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.413481Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.413484Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.413487Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 410  requestId: <none> cookie: 207
2025-10-09T21:43:39.413493Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.413493Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413498Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413499Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.413523Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413524Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413528Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413539Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413545Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413547Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413548Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.413550Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413552Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413557Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 207 partNo : 0 messageNo: 417 size 22745 offset: -1
2025-10-09T21:43:39.413562Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413563Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413571Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.413574Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.413578Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 413  requestId: <none> cookie: 208
2025-10-09T21:43:39.413582Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 414  requestId: <none> cookie: 209
2025-10-09T21:43:39.413585Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 415  requestId: <none> cookie: 210
2025-10-09T21:43:39.413588Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 416  requestId: <none> cookie: 211
2025-10-09T21:43:39.413591Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 418  requestId: <none> cookie: 212
2025-10-09T21:43:39.413596Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.413601Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.413609Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.413613Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.413618Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.413633Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.413638Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.413645Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.413651Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.413676Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413681Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413693Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 208 partNo : 0 messageNo: 419 size 22855 offset: -1
2025-10-09T21:43:39.413699Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413701Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413704Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.413705Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 209 partNo : 0 messageNo: 420 size 22965 offset: -1
2025-10-09T21:43:39.413708Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413710Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.413710Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413715Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 210 partNo : 0 messageNo: 421 size 23075 offset: -1
2025-10-09T21:43:39.413718Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413724Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413725Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.413727Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 211 partNo : 0 messageNo: 422 size 23185 offset: -1
2025-10-09T21:43:39.413731Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413732Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413736Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 212 partNo : 0 messageNo: 423 size 23295 offset: -1
2025-10-09T21:43:39.413742Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.413750Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.413752Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.413754Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.413756Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.413761Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.413763Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.413765Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.413767Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413769Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413773Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413774Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413778Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413779Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413782Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413784Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413788Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.413791Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.413794Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.413797Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.413808Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 424  requestId: <none> cookie: 213
2025-10-09T21:43:39.413811Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.413813Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 425  requestId: <none> cookie: 214
2025-10-09T21:43:39.413816Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 426  requestId: <none> cookie: 215
2025-10-09T21:43:39.413819Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 427  requestId: <none> cookie: 216
2025-10-09T21:43:39.413820Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.413821Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 428  requestId: <none> cookie: 217
2025-10-09T21:43:39.413899Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.413905Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.413909Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.413913Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.413926Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.413939Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.413992Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.413999Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.414005Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.414012Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.414016Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.414178Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-10-09T21:43:39.414199Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 383082 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.414211Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414222Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 156, partNo: 0, Offset: 155 is stored on disk
2025-10-09T21:43:39.414228Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414238Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 157, partNo: 0, Offset: 156 is stored on disk
2025-10-09T21:43:39.414242Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414244Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.414275Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 158, partNo: 0, Offset: 157 is stored on disk
2025-10-09T21:43:39.414278Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414282Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 159, partNo: 0, Offset: 158 is stored on disk
2025-10-09T21:43:39.414285Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414290Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 160, partNo: 0, Offset: 159 is stored on disk
2025-10-09T21:43:39.414294Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414300Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 161, partNo: 0, Offset: 160 is stored on disk
2025-10-09T21:43:39.414304Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414310Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 162, partNo: 0, Offset: 161 is stored on disk
2025-10-09T21:43:39.414315Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414320Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 163, partNo: 0, Offset: 162 is stored on disk
2025-10-09T21:43:39.414327Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414334Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 164, partNo: 0, Offset: 163 is stored on disk
2025-10-09T21:43:39.414337Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414342Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.414345Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 165, partNo: 0, Offset: 164 is stored on disk
2025-10-09T21:43:39.414348Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414354Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 166, partNo: 0, Offset: 165 is stored on disk
2025-10-09T21:43:39.414358Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414366Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 167, partNo: 0, Offset: 166 is stored on disk
2025-10-09T21:43:39.414370Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414376Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 168, partNo: 0, Offset: 167 is stored on disk
2025-10-09T21:43:39.414379Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414385Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 169, partNo: 0, Offset: 168 is stored on disk
2025-10-09T21:43:39.414388Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414401Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.414412Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 170, partNo: 0, Offset: 169 is stored on disk
2025-10-09T21:43:39.414426Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414431Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 171, partNo: 0, Offset: 170 is stored on disk
2025-10-09T21:43:39.414434Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414440Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 172, partNo: 0, Offset: 171 is stored on disk
2025-10-09T21:43:39.414444Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414452Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 173, partNo: 0, Offset: 172 is stored on disk
2025-10-09T21:43:39.414456Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414463Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 174, partNo: 0, Offset: 173 is stored on disk
2025-10-09T21:43:39.414467Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414473Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 175, partNo: 0, Offset: 174 is stored on disk
2025-10-09T21:43:39.414477Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.414482Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 176, partNo: 0, Offset: 175 is stored on disk
2025-10-09T21:43:39.414566Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.414604Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.414607Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (36)
2025-10-09T21:43:39.414635Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 177 partNo 0
2025-10-09T21:43:39.414698Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 177 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 19519 count 1 nextOffset 177 batches 1
2025-10-09T21:43:39.414710Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 178 partNo 0
2025-10-09T21:43:39.414718Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 178 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 39116 count 2 nextOffset 178 batches 1
2025-10-09T21:43:39.414724Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 179 partNo 0
2025-10-09T21:43:39.414730Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 179 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 58823 count 3 nextOffset 179 batches 1
2025-10-09T21:43:39.414735Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 180 partNo 0
2025-10-09T21:43:39.414746Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 180 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 78640 count 4 nextOffset 180 batches 1
2025-10-09T21:43:39.414752Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 181 partNo 0
2025-10-09T21:43:39.414758Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 181 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 98567 count 5 nextOffset 181 batches 1
2025-10-09T21:43:39.414764Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 182 partNo 0
2025-10-09T21:43:39.414772Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 182 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 118604 count 6 nextOffset 182 batches 1
2025-10-09T21:43:39.414778Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 183 partNo 0
2025-10-09T21:43:39.414783Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 183 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 138751 count 7 nextOffset 183 batches 1
2025-10-09T21:43:39.414788Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 184 partNo 0
2025-10-09T21:43:39.414795Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 184 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 159008 count 8 nextOffset 184 batches 1
2025-10-09T21:43:39.414802Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 185 partNo 0
2025-10-09T21:43:39.414811Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 185 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 179375 count 9 nextOffset 185 batches 1
2025-10-09T21:43:39.414816Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 186 partNo 0
2025-10-09T21:43:39.414822Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 186 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 199852 count 10 nextOffset 186 batches 1
2025-10-09T21:43:39.414829Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 187 partNo 0
2025-10-09T21:43:39.414836Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 187 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 220439 count 11 nextOffset 187 batches 1
2025-10-09T21:43:39.414841Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 188 partNo 0
2025-10-09T21:43:39.414848Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 188 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 241136 count 12 nextOffset 188 batches 1
2025-10-09T21:43:39.414853Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 189 partNo 0
2025-10-09T21:43:39.414858Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 189 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 261943 count 13 nextOffset 189 batches 1
2025-10-09T21:43:39.414863Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 190 partNo 0
2025-10-09T21:43:39.414869Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 190 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 282860 count 14 nextOffset 190 batches 1
2025-10-09T21:43:39.414875Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 191 partNo 0
2025-10-09T21:43:39.414880Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 191 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 303887 count 15 nextOffset 191 batches 1
2025-10-09T21:43:39.414885Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 192 partNo 0
2025-10-09T21:43:39.414890Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 192 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 325024 count 16 nextOffset 192 batches 1
2025-10-09T21:43:39.414895Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 193 partNo 0
2025-10-09T21:43:39.414902Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 193 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 346271 count 17 nextOffset 193 batches 1
2025-10-09T21:43:39.414907Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 194 partNo 0
2025-10-09T21:43:39.414913Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 194 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 367628 count 18 nextOffset 194 batches 1
2025-10-09T21:43:39.414918Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 195 partNo 0
2025-10-09T21:43:39.414923Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 195 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 389095 count 19 nextOffset 195 batches 1
2025-10-09T21:43:39.414929Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 196 partNo 0
2025-10-09T21:43:39.414937Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 196 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 410672 count 20 nextOffset 196 batches 1
2025-10-09T21:43:39.414939Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 155 EndOffset 155 newEndOffset 176 commitOffset 0 clientCommitOffset 0 cookie 14 readingFinished 0 firstRead 0
2025-10-09T21:43:39.414942Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 197 partNo 0
2025-10-09T21:43:39.414943Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 155 endOffset 176
2025-10-09T21:43:39.414949Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 197 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 432359 count 21 nextOffset 197 batches 1
2025-10-09T21:43:39.414950Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.414955Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 198 partNo 0
2025-10-09T21:43:39.414960Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 198 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 454156 count 22 nextOffset 198 batches 1
2025-10-09T21:43:39.414967Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 199 partNo 0
2025-10-09T21:43:39.414972Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 199 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 476063 count 23 nextOffset 199 batches 1
2025-10-09T21:43:39.414975Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 200 partNo 0
2025-10-09T21:43:39.414981Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 200 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 498080 count 24 nextOffset 200 batches 1
2025-10-09T21:43:39.414989Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 201 partNo 0
2025-10-09T21:43:39.415020Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 155, endOffset# 176, WTime# 1760046219405, sizeLag# 383262
2025-10-09T21:43:39.415022Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:43:39.415035Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# 65b64e93-c3fd1d79-249c6393-da992f7d, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 25, size# 375205, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.415050Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.415099Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 25 maxSize 375205 maxTimeLagMs 0 readTimestampMs 0 readOffset 155 EndOffset 176 ClientCommitOffset 0 committedOffset 0 Guid 65b64e93-c3fd1d79-249c6393-da992f7d
2025-10-09T21:43:39.415116Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.415208Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.415303Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.415387Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 201 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 519496 count 25 nextOffset 201 batches 1
2025-10-09T21:43:39.415395Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 202 partNo 0
2025-10-09T21:43:39.415411Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 202 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 541765 count 26 nextOffset 202 batches 2
2025-10-09T21:43:39.415416Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 203 partNo 0
2025-10-09T21:43:39.415421Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 203 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 564112 count 27 nextOffset 203 batches 2
2025-10-09T21:43:39.415424Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 204 partNo 0
2025-10-09T21:43:39.415425Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.415428Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 204 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 586569 count 28 nextOffset 204 batches 2
2025-10-09T21:43:39.415432Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 205 partNo 0
2025-10-09T21:43:39.415436Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 205 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 609136 count 29 nextOffset 205 batches 2
2025-10-09T21:43:39.415439Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 206 partNo 0
2025-10-09T21:43:39.415442Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 206 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 631813 count 30 nextOffset 206 batches 2
2025-10-09T21:43:39.415445Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 207 partNo 0
2025-10-09T21:43:39.415449Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 207 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 654600 count 31 nextOffset 207 batches 2
2025-10-09T21:43:39.415451Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 208 partNo 0
2025-10-09T21:43:39.415455Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 208 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 677497 count 32 nextOffset 208 batches 2
2025-10-09T21:43:39.415459Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 209 partNo 0
2025-10-09T21:43:39.415462Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 209 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 700504 count 33 nextOffset 209 batches 2
2025-10-09T21:43:39.415466Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 210 partNo 0
2025-10-09T21:43:39.415470Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 210 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 723621 count 34 nextOffset 210 batches 2
2025-10-09T21:43:39.415474Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 211 partNo 0
2025-10-09T21:43:39.415477Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 211 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 746848 count 35 nextOffset 211 batches 2
2025-10-09T21:43:39.415480Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 212 partNo 0
2025-10-09T21:43:39.415483Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 212 partNo 0 FormedBlobsCount 0 NewHead: Offset 176 PartNo 0 PackedSize 770185 count 36 nextOffset 212 batches 2
2025-10-09T21:43:39.415486Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.415509Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.415584Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.415606Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 176,36 HeadOffset 176 endOffset 176 curOffset 212 d0000000000_00000000000000000176_00000_0000000036_00000? size 769908 WTime 1760046219414
2025-10-09T21:43:39.415671Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.415730Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.415808Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.415878Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.415907Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.415935Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.415966Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.416004Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.416036Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.416109Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.416205Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.416345Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.416359Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.416364Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.416372Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.416378Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 213 partNo : 0 messageNo: 429 size 23405 offset: -1
2025-10-09T21:43:39.416385Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.416386Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.416391Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 214 partNo : 0 messageNo: 430 size 23515 offset: -1
2025-10-09T21:43:39.416395Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.416416Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.416421Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 215 partNo : 0 messageNo: 431 size 23625 offset: -1
2025-10-09T21:43:39.416427Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.416429Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.416434Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 216 partNo : 0 messageNo: 432 size 23735 offset: -1
2025-10-09T21:43:39.416438Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.416439Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.416442Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.416447Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 217 partNo : 0 messageNo: 433 size 23845 offset: -1
2025-10-09T21:43:39.416452Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.416454Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.416459Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.416461Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.416465Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.416467Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.416472Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.416473Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.416477Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.416478Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.416489Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 315  requestId: <none> cookie: 156
2025-10-09T21:43:39.416516Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416521Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 316  requestId: <none> cookie: 157
2025-10-09T21:43:39.416523Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.416529Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416530Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 317  requestId: <none> cookie: 158
2025-10-09T21:43:39.416535Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 321  requestId: <none> cookie: 159
2025-10-09T21:43:39.416541Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 322  requestId: <none> cookie: 160
2025-10-09T21:43:39.416546Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 325  requestId: <none> cookie: 161
2025-10-09T21:43:39.416552Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 326  requestId: <none> cookie: 162
2025-10-09T21:43:39.416558Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 327  requestId: <none> cookie: 163
2025-10-09T21:43:39.416564Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 331  requestId: <none> cookie: 164
2025-10-09T21:43:39.416570Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 332  requestId: <none> cookie: 165
2025-10-09T21:43:39.416576Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 335  requestId: <none> cookie: 166
2025-10-09T21:43:39.416582Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 336  requestId: <none> cookie: 167
2025-10-09T21:43:39.416589Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 337  requestId: <none> cookie: 168
2025-10-09T21:43:39.416594Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 339  requestId: <none> cookie: 169
2025-10-09T21:43:39.416600Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 340  requestId: <none> cookie: 170
2025-10-09T21:43:39.416606Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 343  requestId: <none> cookie: 171
2025-10-09T21:43:39.416613Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 344  requestId: <none> cookie: 172
2025-10-09T21:43:39.416618Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 345  requestId: <none> cookie: 173
2025-10-09T21:43:39.416624Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 348  requestId: <none> cookie: 174
2025-10-09T21:43:39.416630Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 349  requestId: <none> cookie: 175
2025-10-09T21:43:39.416635Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 352  requestId: <none> cookie: 176
2025-10-09T21:43:39.416650Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.416652Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.416656Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416662Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416663Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.416667Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416671Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416675Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416676Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 176 partNo 0 count 36 size 769908
2025-10-09T21:43:39.416679Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416685Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416686Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.416689Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416695Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.416697Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.416702Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.416703Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416704Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.416707Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416707Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.416709Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.416711Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416712Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.416713Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.416715Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416715Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.416719Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416723Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416726Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416730Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416736Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416740Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416745Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416761Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 434  requestId: <none> cookie: 218
2025-10-09T21:43:39.416765Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 435  requestId: <none> cookie: 219
2025-10-09T21:43:39.416768Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 436  requestId: <none> cookie: 220
2025-10-09T21:43:39.416771Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 437  requestId: <none> cookie: 221
2025-10-09T21:43:39.416775Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 438  requestId: <none> cookie: 222
2025-10-09T21:43:39.416812Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 17 Topic 'rt3.dc1--test-topic' partition 0 user user offset 155 count 25 size 375205 endOffset 176 max time lag 0ms effective offset 155
2025-10-09T21:43:39.416827Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 17 added 1 blobs, size 383262 count 21 last offset 155, current partition end offset: 176
2025-10-09T21:43:39.416831Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 17. Send blob request.
2025-10-09T21:43:39.416842Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 155 partno 0 count 21 parts_count 0 source 1 size 383262 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.416848Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416854Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 17. All 1 blobs are from cache.
2025-10-09T21:43:39.416864Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.416870Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416874Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416876Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 155 partno 0 count 21 parts 0 suffix '63'
2025-10-09T21:43:39.416878Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416882Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.416898Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.416905Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.416910Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.416916Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.416921Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.417030Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.417113Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.417291Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.417468Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.417635Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.417852Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.418023Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.418250Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.418588Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.418747Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.418909Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.418981Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.419315Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 155 totakecount 21 count 21 size 383239 from pos 0 cbcount 21
2025-10-09T21:43:39.419381Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419386Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419398Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 218 partNo : 0 messageNo: 439 size 23955 offset: -1
2025-10-09T21:43:39.419406Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419407Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419411Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 219 partNo : 0 messageNo: 440 size 24065 offset: -1
2025-10-09T21:43:39.419415Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419417Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419421Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 220 partNo : 0 messageNo: 441 size 24175 offset: -1
2025-10-09T21:43:39.419427Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419428Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419435Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 221 partNo : 0 messageNo: 442 size 24285 offset: -1
2025-10-09T21:43:39.419439Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419441Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419446Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 222 partNo : 0 messageNo: 443 size 24395 offset: -1
2025-10-09T21:43:39.419451Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419455Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419460Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419462Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419465Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419467Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419471Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419472Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419476Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419477Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419510Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 155
2025-10-09T21:43:39.419531Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.419541Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.419543Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.419545Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.419547Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.419549Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.419551Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.419553Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.419555Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.419558Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.419747Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 444  requestId: <none> cookie: 223
2025-10-09T21:43:39.419752Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 445  requestId: <none> cookie: 224
2025-10-09T21:43:39.419756Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 446  requestId: <none> cookie: 225
2025-10-09T21:43:39.419760Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 447  requestId: <none> cookie: 226
2025-10-09T21:43:39.419766Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 448  requestId: <none> cookie: 227
2025-10-09T21:43:39.419780Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.419788Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.419791Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.419795Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.419801Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.419824Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.419862Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.419866Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.419870Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.419874Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.419882Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419885Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419892Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 223 partNo : 0 messageNo: 449 size 24505 offset: -1
2025-10-09T21:43:39.419894Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419896Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419898Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 224 partNo : 0 messageNo: 450 size 24615 offset: -1
2025-10-09T21:43:39.419901Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419904Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419908Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 225 partNo : 0 messageNo: 451 size 24725 offset: -1
2025-10-09T21:43:39.419911Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419913Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419914Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 176 Result { Offset: 155 Data: "... 17135 bytes ..." SourceId: "\000src_id" SeqNo: 156 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219379 UncompressedSize: 17061 PartitionKey: "" ExplicitHash: "" } Result { Offset: 156 Data: "... 17245 bytes ..." SourceId: "\000src_id" SeqNo: 157 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219380 UncompressedSize: 17171 PartitionKey: "" ExplicitHash: "" } Result { Offset: 157 Data: "... 17355 bytes ..." SourceId: "\000src_id" SeqNo: 158 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219380 UncompressedSize: 17281 PartitionKey: "" ExplicitHash: "" } Result { Offset: 158 Data: "... 17465 bytes ..." SourceId: "\000src_id" SeqNo: 159 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219380 UncompressedSize: 17391 PartitionKey: "" ExplicitHash: "" } Result { Offset: 159 Data: "... 17575 bytes ..." SourceId: "\000src_id" SeqNo: 160 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219380 UncompressedSize: 17501 PartitionKey: "" ExplicitHash: "" } Result { Offset: 160 Data: "... 17685 bytes ..." SourceId: "\000src_id" SeqNo: 161 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219380 UncompressedSize: 17611 PartitionKey: "" ExplicitHash: "" } Result { Offset: 161 Data: "... 17795 bytes ..." SourceId: "\000src_id" SeqNo: 162 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219380 UncompressedSize: 17721 PartitionKey: "" ExplicitHash: "" } Result { Offset: 162 Data: "... 17905 bytes ..." SourceId: "\000src_id" SeqNo: 163 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219381 UncompressedSize: 17831 PartitionKey: "" ExplicitHash: "" } Result { Offset: 163 Data: "... 18015 bytes ..." SourceId: "\000src_id" SeqNo: 164 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219381 UncompressedSize: 17941 PartitionKey: "" ExplicitHash: "" } Result { Offset: 164 Data: "... 18125 bytes ..." SourceId: "\000src_id" SeqNo: 165 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219381 UncompressedSize: 18051 PartitionKey: "" ExplicitHash: "" } Result { Offset: 165 Data: "... 18235 bytes ..." SourceId: "\000src_id" SeqNo: 166 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219381 UncompressedSize: 18161 PartitionKey: "" ExplicitHash: "" } Result { Offset: 166 Data: "... 18345 bytes ..." SourceId: "\000src_id" SeqNo: 167 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219381 UncompressedSize: 18271 PartitionKey: "" ExplicitHash: "" } Result { Offset: 167 Data: "... 18455 bytes ..." SourceId: "\000src_id" SeqNo: 168 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219381 UncompressedSize: 18381 PartitionKey: "" ExplicitHash: "" } Result { Offset: 168 Data: "... 18565 bytes ..." SourceId: "\000src_id" SeqNo: 169 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219381 UncompressedSize: 18491 PartitionKey: "" ExplicitHash: "" } Result { Offset: 169 Data: "... 18675 bytes ..." SourceId: "\000src_id" SeqNo: 170 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219381 UncompressedSize: 18601 PartitionKey: "" ExplicitHash: "" } Result { Offset: 170 Data: "... 18785 bytes ..." SourceId: "\000src_id" SeqNo: 171 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219381 UncompressedSize: 18711 PartitionKey: "" ExplicitHash: "" } Result { Offset: 171 Data: "... 18895 bytes ..." SourceId: "\000src_id" SeqNo: 172 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219381 UncompressedSize: 18821 PartitionKey: "" ExplicitHash: "" } Result { Offset: 172 Data: "... 19005 bytes ..." SourceId: "\000src_id" SeqNo: 173 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219382 UncompressedSize: 18931 PartitionKey: "" ExplicitHash: "" } Result { Offset: 173 Data: "... 19115 bytes ..." SourceId: "\000src_id" SeqNo: 174 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219384 UncompressedSize: 19041 PartitionKey: "" ExplicitHash: "" } Result { Offset: 174 Data: "... 19225 bytes ..." SourceId: "\000src_id" SeqNo: 175 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219384 UncompressedSize: 19151 PartitionKey: "" ExplicitHash: "" } Result { Offset: 175 Data: "... 19335 bytes ..." SourceId: "\000src_id" SeqNo: 176 WriteTimestampMS: 1760046219408 CreateTimestampMS: 1760046219384 UncompressedSize: 19261 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 18446744073709551061 RealReadOffset: 175 WaitQuotaTimeMs: 0 EndOffset: 176 StartOffset: 0 } Cookie: 155 }
2025-10-09T21:43:39.419917Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 226 partNo : 0 messageNo: 452 size 24835 offset: -1
2025-10-09T21:43:39.419920Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419921Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419927Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 227 partNo : 0 messageNo: 453 size 24945 offset: -1
2025-10-09T21:43:39.419930Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419933Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419937Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419938Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419941Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419943Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419947Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419948Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419952Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.419954Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.419957Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.419963Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.419964Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.419967Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.419968Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.419970Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.419972Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.419974Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.419975Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.419977Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.419986Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 454  requestId: <none> cookie: 228
2025-10-09T21:43:39.419990Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 455  requestId: <none> cookie: 229
2025-10-09T21:43:39.419993Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 456  requestId: <none> cookie: 230
2025-10-09T21:43:39.419995Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 457  requestId: <none> cookie: 231
2025-10-09T21:43:39.419998Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 458  requestId: <none> cookie: 232
2025-10-09T21:43:39.420003Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.420008Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.420011Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.420014Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.420018Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.420037Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.420046Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.420052Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.420057Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.420063Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.420074Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 15 from offset 176
2025-10-09T21:43:39.420075Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.420077Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.420083Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 176 ReadOffset 176 ReadGuid 65b64e93-c3fd1d79-249c6393-da992f7d has messages 1
2025-10-09T21:43:39.420083Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 228 partNo : 0 messageNo: 459 size 25055 offset: -1
2025-10-09T21:43:39.420086Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.420088Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.420091Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 229 partNo : 0 messageNo: 460 size 25165 offset: -1
2025-10-09T21:43:39.420094Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.420095Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.420098Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 230 partNo : 0 messageNo: 461 size 25275 offset: -1
2025-10-09T21:43:39.420101Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.420102Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.420105Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 231 partNo : 0 messageNo: 462 size 25385 offset: -1
2025-10-09T21:43:39.420108Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.420110Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.420113Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 232 partNo : 0 messageNo: 463 size 25495 offset: -1
2025-10-09T21:43:39.420116Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.420120Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.420122Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.420125Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.420128Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# 65b64e93-c3fd1d79-249c6393-da992f7d, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 382354
2025-10-09T21:43:39.420130Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.420131Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.420135Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.420136Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.420138Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# 65b64e93-c3fd1d79-249c6393-da992f7d
2025-10-09T21:43:39.420139Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.420141Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.420143Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.420145Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.420148Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.420149Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.420152Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.420155Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.420156Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.420158Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.420162Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.420168Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.420181Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 464  requestId: <none> cookie: 233
2025-10-09T21:43:39.420184Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 465  requestId: <none> cookie: 234
2025-10-09T21:43:39.420187Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 466  requestId: <none> cookie: 235
2025-10-09T21:43:39.420190Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 467  requestId: <none> cookie: 236
2025-10-09T21:43:39.420193Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 468  requestId: <none> cookie: 237
2025-10-09T21:43:39.420198Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.420202Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.420206Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.420210Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.420214Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.420227Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.420234Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.420239Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.420244Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.420249Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.420307Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.420309Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.420315Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 233 partNo : 0 messageNo: 469 size 25605 offset: -1
2025-10-09T21:43:39.420318Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.420320Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.420324Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 234 partNo : 0 messageNo: 470 size 25715 offset: -1
2025-10-09T21:43:39.420327Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.420328Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.420332Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 235 partNo : 0 messageNo: 471 size 25825 offset: -1
2025-10-09T21:43:39.420334Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.420336Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.420339Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 236 partNo : 0 messageNo: 472 size 25935 offset: -1
2025-10-09T21:43:39.420343Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.420347Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.420350Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 237 partNo : 0 messageNo: 473 size 26045 offset: -1
2025-10-09T21:43:39.420365Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 176 count 36 size 769908 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.420376Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.420380Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.420381Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.420384Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.420386Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.420388Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.420390Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.420392Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.420394Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.420396Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.420399Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.420409Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 176 partno 0 count 36 parts 0 suffix '63' size 769908
2025-10-09T21:43:39.420412Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 769572 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.420420Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420437Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 177, partNo: 0, Offset: 176 is stored on disk
2025-10-09T21:43:39.420440Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420443Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 178, partNo: 0, Offset: 177 is stored on disk
2025-10-09T21:43:39.420444Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 0
2025-10-09T21:43:39.420445Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420449Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 179, partNo: 0, Offset: 178 is stored on disk
2025-10-09T21:43:39.420451Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420455Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 180, partNo: 0, Offset: 179 is stored on disk
2025-10-09T21:43:39.420457Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420461Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 181, partNo: 0, Offset: 180 is stored on disk
2025-10-09T21:43:39.420463Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420467Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 182, partNo: 0, Offset: 181 is stored on disk
2025-10-09T21:43:39.420469Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420473Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 183, partNo: 0, Offset: 182 is stored on disk
2025-10-09T21:43:39.420476Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420479Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 184, partNo: 0, Offset: 183 is stored on disk
2025-10-09T21:43:39.420481Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420484Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 185, partNo: 0, Offset: 184 is stored on disk
2025-10-09T21:43:39.420486Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420489Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 186, partNo: 0, Offset: 185 is stored on disk
2025-10-09T21:43:39.420491Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420494Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 187, partNo: 0, Offset: 186 is stored on disk
2025-10-09T21:43:39.420496Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420500Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 188, partNo: 0, Offset: 187 is stored on disk
2025-10-09T21:43:39.420502Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420504Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 189, partNo: 0, Offset: 188 is stored on disk
2025-10-09T21:43:39.420506Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420510Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 190, partNo: 0, Offset: 189 is stored on disk
2025-10-09T21:43:39.420513Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420517Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 191, partNo: 0, Offset: 190 is stored on disk
2025-10-09T21:43:39.420518Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420522Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 192, partNo: 0, Offset: 191 is stored on disk
2025-10-09T21:43:39.420524Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420527Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 193, partNo: 0, Offset: 192 is stored on disk
2025-10-09T21:43:39.420531Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420534Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 194, partNo: 0, Offset: 193 is stored on disk
2025-10-09T21:43:39.420536Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420539Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 195, partNo: 0, Offset: 194 is stored on disk
2025-10-09T21:43:39.420542Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420546Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 196, partNo: 0, Offset: 195 is stored on disk
2025-10-09T21:43:39.420548Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420554Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 197, partNo: 0, Offset: 196 is stored on disk
2025-10-09T21:43:39.420556Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420559Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 198, partNo: 0, Offset: 197 is stored on disk
2025-10-09T21:43:39.420561Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420564Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 199, partNo: 0, Offset: 198 is stored on disk
2025-10-09T21:43:39.420566Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420569Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 200, partNo: 0, Offset: 199 is stored on disk
2025-10-09T21:43:39.420572Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420575Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 201, partNo: 0, Offset: 200 is stored on disk
2025-10-09T21:43:39.420577Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420580Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 202, partNo: 0, Offset: 201 is stored on disk
2025-10-09T21:43:39.420582Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420586Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 203, partNo: 0, Offset: 202 is stored on disk
2025-10-09T21:43:39.420588Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420593Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 204, partNo: 0, Offset: 203 is stored on disk
2025-10-09T21:43:39.420595Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420598Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 205, partNo: 0, Offset: 204 is stored on disk
2025-10-09T21:43:39.420600Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420602Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 206, partNo: 0, Offset: 205 is stored on disk
2025-10-09T21:43:39.420605Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420608Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 207, partNo: 0, Offset: 206 is stored on disk
2025-10-09T21:43:39.420610Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420615Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 208, partNo: 0, Offset: 207 is stored on disk
2025-10-09T21:43:39.420618Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420633Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 209, partNo: 0, Offset: 208 is stored on disk
2025-10-09T21:43:39.420635Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420639Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 210, partNo: 0, Offset: 209 is stored on disk
2025-10-09T21:43:39.420641Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420643Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 211, partNo: 0, Offset: 210 is stored on disk
2025-10-09T21:43:39.420646Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.420650Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 212, partNo: 0, Offset: 211 is stored on disk
2025-10-09T21:43:39.420685Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.420705Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1312: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait for data done:  readOffset 176 EndOffset 176 newEndOffset 212 commitOffset 0 clientCommitOffset 0 cookie 15 readingFinished 0 firstRead 0
2025-10-09T21:43:39.420710Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 176 endOffset 212
2025-10-09T21:43:39.420714Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.420718Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (25)
2025-10-09T21:43:39.420723Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 176, endOffset# 212, WTime# 1760046219408, sizeLag# 769908
2025-10-09T21:43:39.420725Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 1
2025-10-09T21:43:39.420737Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 213 partNo 0
2025-10-09T21:43:39.420760Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 213 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 23479 count 1 nextOffset 213 batches 1
2025-10-09T21:43:39.420769Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 214 partNo 0
2025-10-09T21:43:39.420773Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 214 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 47036 count 2 nextOffset 214 batches 1
2025-10-09T21:43:39.420776Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 215 partNo 0
2025-10-09T21:43:39.420781Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 215 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 70703 count 3 nextOffset 215 batches 1
2025-10-09T21:43:39.420784Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 216 partNo 0
2025-10-09T21:43:39.420787Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 216 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 94480 count 4 nextOffset 216 batches 1
2025-10-09T21:43:39.420791Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 217 partNo 0
2025-10-09T21:43:39.420795Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 217 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 118367 count 5 nextOffset 217 batches 1
2025-10-09T21:43:39.420798Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 218 partNo 0
2025-10-09T21:43:39.420801Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 218 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 142364 count 6 nextOffset 218 batches 1
2025-10-09T21:43:39.420804Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 219 partNo 0
2025-10-09T21:43:39.420808Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 219 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 166471 count 7 nextOffset 219 batches 1
2025-10-09T21:43:39.420811Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 220 partNo 0
2025-10-09T21:43:39.420815Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 220 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 190688 count 8 nextOffset 220 batches 1
2025-10-09T21:43:39.420818Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 221 partNo 0
2025-10-09T21:43:39.420822Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 221 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 215015 count 9 nextOffset 221 batches 1
2025-10-09T21:43:39.420825Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 222 partNo 0
2025-10-09T21:43:39.420829Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 222 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 239452 count 10 nextOffset 222 batches 1
2025-10-09T21:43:39.420832Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 223 partNo 0
2025-10-09T21:43:39.420836Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 223 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 263999 count 11 nextOffset 223 batches 1
2025-10-09T21:43:39.420839Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 224 partNo 0
2025-10-09T21:43:39.420842Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 224 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 288656 count 12 nextOffset 224 batches 1
2025-10-09T21:43:39.420845Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 225 partNo 0
2025-10-09T21:43:39.420850Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 225 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 313423 count 13 nextOffset 225 batches 1
2025-10-09T21:43:39.420854Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 226 partNo 0
2025-10-09T21:43:39.420858Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 226 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 338300 count 14 nextOffset 226 batches 1
2025-10-09T21:43:39.420861Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 227 partNo 0
2025-10-09T21:43:39.420864Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 227 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 363287 count 15 nextOffset 227 batches 1
2025-10-09T21:43:39.420867Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 228 partNo 0
2025-10-09T21:43:39.420870Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 228 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 388384 count 16 nextOffset 228 batches 1
2025-10-09T21:43:39.420874Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 229 partNo 0
2025-10-09T21:43:39.420878Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 229 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 413591 count 17 nextOffset 229 batches 1
2025-10-09T21:43:39.420882Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 230 partNo 0
2025-10-09T21:43:39.420885Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 230 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 438908 count 18 nextOffset 230 batches 1
2025-10-09T21:43:39.420888Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 231 partNo 0
2025-10-09T21:43:39.420891Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 231 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 464335 count 19 nextOffset 231 batches 1
2025-10-09T21:43:39.420894Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 232 partNo 0
2025-10-09T21:43:39.420897Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 232 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 489872 count 20 nextOffset 232 batches 1
2025-10-09T21:43:39.420899Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 233 partNo 0
2025-10-09T21:43:39.421056Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 233 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 514932 count 21 nextOffset 233 batches 1
2025-10-09T21:43:39.421062Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 234 partNo 0
2025-10-09T21:43:39.421073Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 234 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 540721 count 22 nextOffset 234 batches 2
2025-10-09T21:43:39.421076Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 235 partNo 0
2025-10-09T21:43:39.421080Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 235 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 566588 count 23 nextOffset 235 batches 2
2025-10-09T21:43:39.421083Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 236 partNo 0
2025-10-09T21:43:39.421088Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 236 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 592565 count 24 nextOffset 236 batches 2
2025-10-09T21:43:39.421091Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 237 partNo 0
2025-10-09T21:43:39.421095Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 237 partNo 0 FormedBlobsCount 0 NewHead: Offset 212 PartNo 0 PackedSize 618652 count 25 nextOffset 237 batches 2
2025-10-09T21:43:39.421098Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.421176Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 212,25 HeadOffset 212 endOffset 212 curOffset 237 d0000000000_00000000000000000212_00000_0000000025_00000? size 618592 WTime 1760046219420
2025-10-09T21:43:39.421733Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.421744Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.421748Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.421756Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.421757Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.421762Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.421764Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.421768Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.421769Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.421780Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.421781Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.421789Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 355  requestId: <none> cookie: 177
2025-10-09T21:43:39.421806Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 356  requestId: <none> cookie: 178
2025-10-09T21:43:39.421812Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 357  requestId: <none> cookie: 179
2025-10-09T21:43:39.421823Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 360  requestId: <none> cookie: 180
2025-10-09T21:43:39.421829Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 361  requestId: <none> cookie: 181
2025-10-09T21:43:39.421833Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.421834Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 363  requestId: <none> cookie: 182
2025-10-09T21:43:39.421839Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 365  requestId: <none> cookie: 183
2025-10-09T21:43:39.421844Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.421846Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 367  requestId: <none> cookie: 184
2025-10-09T21:43:39.421848Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.421852Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 369  requestId: <none> cookie: 185
2025-10-09T21:43:39.421853Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.421857Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.421859Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 371  requestId: <none> cookie: 186
2025-10-09T21:43:39.421861Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.421866Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 373  requestId: <none> cookie: 187
2025-10-09T21:43:39.421869Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.421873Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.421873Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 375  requestId: <none> cookie: 188
2025-10-09T21:43:39.421878Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.421880Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 378  requestId: <none> cookie: 189
2025-10-09T21:43:39.421883Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.421887Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 380  requestId: <none> cookie: 190
2025-10-09T21:43:39.421893Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 383  requestId: <none> cookie: 191
2025-10-09T21:43:39.421900Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 386  requestId: <none> cookie: 192
2025-10-09T21:43:39.421905Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 387  requestId: <none> cookie: 193
2025-10-09T21:43:39.421913Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 389  requestId: <none> cookie: 194
2025-10-09T21:43:39.421920Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 390  requestId: <none> cookie: 195
2025-10-09T21:43:39.421925Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 392  requestId: <none> cookie: 196
2025-10-09T21:43:39.421930Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 393  requestId: <none> cookie: 197
2025-10-09T21:43:39.421951Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 396  requestId: <none> cookie: 198
2025-10-09T21:43:39.421958Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 397  requestId: <none> cookie: 199
2025-10-09T21:43:39.421963Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 402  requestId: <none> cookie: 200
2025-10-09T21:43:39.421969Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 403  requestId: <none> cookie: 201
2025-10-09T21:43:39.421970Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.421975Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 407  requestId: <none> cookie: 202
2025-10-09T21:43:39.421981Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 408  requestId: <none> cookie: 203
2025-10-09T21:43:39.421983Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.421987Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 409  requestId: <none> cookie: 204
2025-10-09T21:43:39.421992Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 411  requestId: <none> cookie: 205
2025-10-09T21:43:39.421998Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 412  requestId: <none> cookie: 206
2025-10-09T21:43:39.422003Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 417  requestId: <none> cookie: 207
2025-10-09T21:43:39.422008Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 419  requestId: <none> cookie: 208
2025-10-09T21:43:39.422015Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 420  requestId: <none> cookie: 209
2025-10-09T21:43:39.422017Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422021Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 421  requestId: <none> cookie: 210
2025-10-09T21:43:39.422041Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 422  requestId: <none> cookie: 211
2025-10-09T21:43:39.422044Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422048Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422051Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422055Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422057Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 423  requestId: <none> cookie: 212
2025-10-09T21:43:39.422060Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422063Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422064Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.422068Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422071Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422075Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422078Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 212 partNo 0 count 25 size 618592
2025-10-09T21:43:39.422080Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422083Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422087Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422099Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422102Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422121Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422131Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422132Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 474  requestId: <none> cookie: 238
2025-10-09T21:43:39.422135Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422138Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 475  requestId: <none> cookie: 239
2025-10-09T21:43:39.422139Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422142Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 476  requestId: <none> cookie: 240
2025-10-09T21:43:39.422143Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422146Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 477  requestId: <none> cookie: 241
2025-10-09T21:43:39.422147Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422151Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 478  requestId: <none> cookie: 242
2025-10-09T21:43:39.422159Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422163Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422168Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422188Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422194Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422199Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422203Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422214Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422358Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.422367Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.422372Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.422378Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.422384Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.422406Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422410Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422434Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 238 partNo : 0 messageNo: 479 size 26155 offset: -1
2025-10-09T21:43:39.422440Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422441Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422445Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 239 partNo : 0 messageNo: 480 size 26265 offset: -1
2025-10-09T21:43:39.422448Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422450Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422453Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 240 partNo : 0 messageNo: 481 size 26375 offset: -1
2025-10-09T21:43:39.422455Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422457Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422460Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 241 partNo : 0 messageNo: 482 size 26485 offset: -1
2025-10-09T21:43:39.422463Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422465Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422468Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 242 partNo : 0 messageNo: 483 size 26595 offset: -1
2025-10-09T21:43:39.422471Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422473Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422476Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422478Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422481Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422483Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422488Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422490Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422495Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422497Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422502Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.422519Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.422522Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.422524Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.422526Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.422528Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.422530Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.422532Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.422534Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.422537Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.422549Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 484  requestId: <none> cookie: 243
2025-10-09T21:43:39.422554Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 485  requestId: <none> cookie: 244
2025-10-09T21:43:39.422557Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 486  requestId: <none> cookie: 245
2025-10-09T21:43:39.422560Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 487  requestId: <none> cookie: 246
2025-10-09T21:43:39.422562Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 488  requestId: <none> cookie: 247
2025-10-09T21:43:39.422568Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422572Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422581Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422588Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422592Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422604Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.422611Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.422616Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.422622Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.422628Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.422637Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422639Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422643Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 243 partNo : 0 messageNo: 489 size 26705 offset: -1
2025-10-09T21:43:39.422647Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422649Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422654Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 244 partNo : 0 messageNo: 490 size 26815 offset: -1
2025-10-09T21:43:39.422657Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422659Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422662Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 245 partNo : 0 messageNo: 491 size 26925 offset: -1
2025-10-09T21:43:39.422665Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422667Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422670Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 246 partNo : 0 messageNo: 492 size 27035 offset: -1
2025-10-09T21:43:39.422673Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422675Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422678Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 247 partNo : 0 messageNo: 493 size 27145 offset: -1
2025-10-09T21:43:39.422682Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422685Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422688Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422689Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422692Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422693Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422696Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422697Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422700Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422702Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422716Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.422719Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.422721Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.422723Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.422725Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.422727Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.422729Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.422732Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.422734Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.422736Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.422743Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 494  requestId: <none> cookie: 248
2025-10-09T21:43:39.422746Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 495  requestId: <none> cookie: 249
2025-10-09T21:43:39.422748Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 496  requestId: <none> cookie: 250
2025-10-09T21:43:39.422752Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 497  requestId: <none> cookie: 251
2025-10-09T21:43:39.422754Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 498  requestId: <none> cookie: 252
2025-10-09T21:43:39.422759Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.422766Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
>>> event from dataHandler: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..17061 bytes.. Information: { Offset: 155 ProducerId: "src_id" SeqNo: 156 CreateTime: 2025-10-09T21:43:39.379000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..17171 bytes.. Information: { Offset: 156 ProducerId: "src_id" SeqNo: 157 CreateTime: 2025-10-09T21:43:39.380000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..17281 bytes.. Information: { Offset: 157 ProducerId: "src_id" SeqNo: 158 CreateTime: 2025-10-09T21:43:39.380000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..17391 bytes.. Information: { Offset: 158 ProducerId: "src_id" SeqNo: 159 CreateTime: 2025-10-09T21:43:39.380000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..17501 bytes.. Information: { Offset: 159 ProducerId: "src_id" SeqNo: 160 CreateTime: 2025-10-09T21:43:39.380000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..17611 bytes.. Information: { Offset: 160 ProducerId: "src_id" SeqNo: 161 CreateTime: 2025-10-09T21:43:39.380000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..17721 bytes.. Information: { Offset: 161 ProducerId: "src_id" SeqNo: 162 CreateTime: 2025-10-09T21:43:39.380000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..17831 bytes.. Information: { Offset: 162 ProducerId: "src_id" SeqNo: 163 CreateTime: 2025-10-09T21:43:39.381000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..17941 bytes.. Information: { Offset: 163 ProducerId: "src_id" SeqNo: 164 CreateTime: 2025-10-09T21:43:39.381000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..18051 bytes.. Information: { Offset: 164 ProducerId: "src_id" SeqNo: 165 CreateTime: 2025-10-09T21:43:39.381000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..18161 bytes.. Information: { Offset: 165 ProducerId: "src_id" SeqNo: 166 CreateTime: 2025-10-09T21:43:39.381000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..18271 bytes.. Information: { Offset: 166 ProducerId: "src_id" SeqNo: 167 CreateTime: 2025-10-09T21:43:39.381000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..18381 bytes.. Information: { Offset: 167 ProducerId: "src_id" SeqNo: 168 CreateTime: 2025-10-09T21:43:39.381000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..18491 bytes.. Information: { Offset: 168 ProducerId: "src_id" SeqNo: 169 CreateTime: 2025-10-09T21:43:39.381000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..18601 bytes.. Information: { Offset: 169 ProducerId: "src_id" SeqNo: 170 CreateTime: 2025-10-09T21:43:39.381000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..18711 bytes.. Information: { Offset: 170 ProducerId: "src_id" SeqNo: 171 CreateTime: 2025-10-09T21:43:39.381000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..18821 bytes.. Information: { Offset: 171 ProducerId: "src_id" SeqNo: 172 CreateTime: 2025-10-09T21:43:39.381000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..18931 bytes.. Information: { Offset: 172 ProducerId: "src_id" SeqNo: 173 CreateTime: 2025-10-09T21:43:39.382000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..19041 bytes.. Information: { Offset: 173 ProducerId: "src_id" SeqNo: 174 CreateTime: 2025-10-09T21:43:39.384000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..19151 bytes.. Information: { Offset: 174 ProducerId: "src_id" SeqNo: 175 CreateTime: 2025-10-09T21:43:39.384000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..19261 bytes.. Information: { Offset: 175 ProducerId: "src_id" SeqNo: 176 CreateTime: 2025-10-09T21:43:39.384000Z WriteTime: 2025-10-09T21:43:39.408000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.502928Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 21, size 381381 bytes
2025-10-09T21:43:39.502966Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.502992Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (176-201)
2025-10-09T21:43:39.503009Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 768229 to budget
2025-10-09T21:43:39.503017Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 768229, ReadSizeServerDelta = -243941
2025-10-09T21:43:39.503175Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.503198Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (176-176)
2025-10-09T21:43:39.503222Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (177-177)
2025-10-09T21:43:39.503236Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 2} (178-178)
2025-10-09T21:43:39.503242Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 3} (179-179)
2025-10-09T21:43:39.422769Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.503269Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 4} (180-180)
2025-10-09T21:43:39.503276Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 5} (181-181)
2025-10-09T21:43:39.422772Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.503283Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 6} (182-182)
2025-10-09T21:43:39.422777Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.422780Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.503290Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 7} (183-183)
2025-10-09T21:43:39.422874Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.422880Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.422885Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.422890Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.422895Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.422940Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.422978Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.422980Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.422986Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 248 partNo : 0 messageNo: 499 size 27255 offset: -1
2025-10-09T21:43:39.503344Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 8} (184-184)
2025-10-09T21:43:39.503359Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 9} (185-185)
2025-10-09T21:43:39.503366Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 10} (186-186)
2025-10-09T21:43:39.503383Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 11} (187-187)
2025-10-09T21:43:39.422990Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.503390Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 12} (188-188)
2025-10-09T21:43:39.503397Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 13} (189-189)
2025-10-09T21:43:39.422991Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.503403Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 14} (190-190)
2025-10-09T21:43:39.422995Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 249 partNo : 0 messageNo: 500 size 27365 offset: -1
2025-10-09T21:43:39.422997Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.503411Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 15} (191-191)
2025-10-09T21:43:39.422999Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423005Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 250 partNo : 0 messageNo: 501 size 27475 offset: -1
2025-10-09T21:43:39.423009Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423011Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423020Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 251 partNo : 0 messageNo: 502 size 27585 offset: -1
2025-10-09T21:43:39.423023Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423025Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423028Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 252 partNo : 0 messageNo: 503 size 27695 offset: -1
2025-10-09T21:43:39.423031Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423033Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423036Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423038Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423042Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423044Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423047Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423048Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423051Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423053Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423057Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.423063Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.423065Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.423068Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.423070Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.503512Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 16} (192-192)
2025-10-09T21:43:39.423072Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.423074Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.503520Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 17} (193-193)
2025-10-09T21:43:39.423076Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.423079Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.423081Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.503527Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 18} (194-194)
2025-10-09T21:43:39.503534Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 19} (195-195)
2025-10-09T21:43:39.503541Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 20} (196-196)
2025-10-09T21:43:39.503548Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 21} (197-197)
2025-10-09T21:43:39.423091Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 504  requestId: <none> cookie: 253
2025-10-09T21:43:39.423104Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 505  requestId: <none> cookie: 254
2025-10-09T21:43:39.423106Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 506  requestId: <none> cookie: 255
2025-10-09T21:43:39.503557Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 22} (198-198)
2025-10-09T21:43:39.423109Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 507  requestId: <none> cookie: 256
2025-10-09T21:43:39.423112Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 508  requestId: <none> cookie: 257
2025-10-09T21:43:39.503566Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 23} (199-199)
2025-10-09T21:43:39.423120Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.423132Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.503574Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 24} (200-200)
2025-10-09T21:43:39.423138Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.423142Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.423146Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.423151Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.423196Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423198Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423203Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 253 partNo : 0 messageNo: 509 size 27805 offset: -1
2025-10-09T21:43:39.423206Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.423210Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.423221Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423223Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.423223Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423226Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 254 partNo : 0 messageNo: 510 size 27915 offset: -1
2025-10-09T21:43:39.423230Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423232Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423234Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.423236Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 255 partNo : 0 messageNo: 511 size 28025 offset: -1
2025-10-09T21:43:39.423239Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423240Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.423241Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423244Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 256 partNo : 0 messageNo: 512 size 28135 offset: -1
2025-10-09T21:43:39.423248Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423249Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423252Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 257 partNo : 0 messageNo: 513 size 28245 offset: -1
2025-10-09T21:43:39.423255Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.423258Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.423258Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.423260Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.423262Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.423264Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.423264Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.423267Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.423269Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.423271Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.423277Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.423290Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423292Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423297Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423300Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423304Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423305Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423309Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423310Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423314Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423315Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423324Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 514  requestId: <none> cookie: 258
2025-10-09T21:43:39.423327Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 515  requestId: <none> cookie: 259
2025-10-09T21:43:39.423329Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 516  requestId: <none> cookie: 260
2025-10-09T21:43:39.423332Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 517  requestId: <none> cookie: 261
2025-10-09T21:43:39.423332Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.423334Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 518  requestId: <none> cookie: 262
2025-10-09T21:43:39.423337Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.423343Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.423346Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.423357Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.423364Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.423366Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423367Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423371Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 258 partNo : 0 messageNo: 519 size 28355 offset: -1
2025-10-09T21:43:39.423372Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.423375Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423376Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423380Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 259 partNo : 0 messageNo: 520 size 28465 offset: -1
2025-10-09T21:43:39.423383Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423385Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423391Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 260 partNo : 0 messageNo: 521 size 28575 offset: -1
2025-10-09T21:43:39.423395Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423396Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423400Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 261 partNo : 0 messageNo: 522 size 28685 offset: -1
2025-10-09T21:43:39.423403Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.423406Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.423410Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423412Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423416Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 262 partNo : 0 messageNo: 523 size 28795 offset: -1
2025-10-09T21:43:39.423420Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423422Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423425Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423426Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423431Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.423433Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.423435Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.423438Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.503583Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 25} (201-201)
2025-10-09T21:43:39.423440Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.423442Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.423444Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.423446Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.423451Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 524  requestId: <none> cookie: 263
2025-10-09T21:43:39.423454Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 525  requestId: <none> cookie: 264
2025-10-09T21:43:39.423458Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.423471Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.423490Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.423649Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.423689Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.423690Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423699Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423702Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.423714Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 263 partNo : 0 messageNo: 526 size 28905 offset: -1
2025-10-09T21:43:39.423724Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423726Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423731Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 264 partNo : 0 messageNo: 527 size 29015 offset: -1
2025-10-09T21:43:39.423735Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423737Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423743Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423748Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423754Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.423764Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.423766Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.423769Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.423779Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 528  requestId: <none> cookie: 265
2025-10-09T21:43:39.423785Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 529  requestId: <none> cookie: 266
2025-10-09T21:43:39.423794Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.423803Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.423811Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.423902Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.423917Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.423951Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.423956Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.423991Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 265 partNo : 0 messageNo: 530 size 29125 offset: -1
2025-10-09T21:43:39.423998Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.424003Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424009Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 266 partNo : 0 messageNo: 531 size 29235 offset: -1
2025-10-09T21:43:39.424013Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.424019Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.424024Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.424026Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.424030Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.424033Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.424035Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424039Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.424040Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424061Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 532  requestId: <none> cookie: 267
2025-10-09T21:43:39.424074Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 533  requestId: <none> cookie: 268
2025-10-09T21:43:39.424094Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.424106Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.424123Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.424166Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.424171Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424182Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 267 partNo : 0 messageNo: 534 size 29345 offset: -1
2025-10-09T21:43:39.424189Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.424190Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424194Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 268 partNo : 0 messageNo: 535 size 29455 offset: -1
2025-10-09T21:43:39.424199Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.424206Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.424210Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.424214Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.424252Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.424273Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.424282Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.424291Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.424296Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424312Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.424317Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424339Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 536  requestId: <none> cookie: 269
2025-10-09T21:43:39.424456Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.424462Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 537  requestId: <none> cookie: 270
2025-10-09T21:43:39.424465Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.424536Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.424547Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.424596Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.424609Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.424612Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424622Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 269 partNo : 0 messageNo: 538 size 29565 offset: -1
2025-10-09T21:43:39.424627Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.424629Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424633Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 270 partNo : 0 messageNo: 539 size 29675 offset: -1
2025-10-09T21:43:39.424637Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.424639Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424644Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.424650Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.424652Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.424656Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.424664Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 540  requestId: <none> cookie: 271
2025-10-09T21:43:39.424679Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.424687Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.424693Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.424718Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.424720Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424725Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 271 partNo : 0 messageNo: 541 size 29785 offset: -1
2025-10-09T21:43:39.424731Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.424735Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.424744Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.424746Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424748Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.424751Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 542  requestId: <none> cookie: 272
2025-10-09T21:43:39.424755Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.424756Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.424760Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424764Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.424765Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424771Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 272 partNo : 0 messageNo: 544 size 29895 offset: -1
2025-10-09T21:43:39.424775Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.424780Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.424797Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.424800Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 543  requestId: <none> cookie: 273
2025-10-09T21:43:39.424855Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.424864Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.424874Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.424928Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.424933Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.424940Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424954Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 273 partNo : 0 messageNo: 545 size 30005 offset: -1
2025-10-09T21:43:39.424962Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
>>> event from dataHandler: 2025-10-09T21:43:39.424964Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424969Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.424974Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.424980Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.424990Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.424989Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.424999Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 546  requestId: <none> cookie: 274
2025-10-09T21:43:39.425005Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 547  requestId: <none> cookie: 275
2025-10-09T21:43:39.425079Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 212 count 25 size 618592 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.425102Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.425093Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.425109Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.425122Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 618300 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.425125Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 212 partno 0 count 25 parts 0 suffix '63' size 618592
2025-10-09T21:43:39.425129Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.425130Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425137Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.425137Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 213, partNo: 0, Offset: 212 is stored on disk
2025-10-09T21:43:39.425141Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425144Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 214, partNo: 0, Offset: 213 is stored on disk
2025-10-09T21:43:39.425147Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425151Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 215, partNo: 0, Offset: 214 is stored on disk
2025-10-09T21:43:39.425154Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425169Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 216, partNo: 0, Offset: 215 is stored on disk
2025-10-09T21:43:39.425171Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425175Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 217, partNo: 0, Offset: 216 is stored on disk
2025-10-09T21:43:39.425177Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425183Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 218, partNo: 0, Offset: 217 is stored on disk
2025-10-09T21:43:39.425185Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425188Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 219, partNo: 0, Offset: 218 is stored on disk
2025-10-09T21:43:39.425190Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425193Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 220, partNo: 0, Offset: 219 is stored on disk
2025-10-09T21:43:39.425195Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425199Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 221, partNo: 0, Offset: 220 is stored on disk
2025-10-09T21:43:39.425201Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425205Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 222, partNo: 0, Offset: 221 is stored on disk
2025-10-09T21:43:39.425207Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425211Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 223, partNo: 0, Offset: 222 is stored on disk
2025-10-09T21:43:39.425213Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425226Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 224, partNo: 0, Offset: 223 is stored on disk
2025-10-09T21:43:39.425228Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425233Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 225, partNo: 0, Offset: 224 is stored on disk
2025-10-09T21:43:39.425235Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425239Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 226, partNo: 0, Offset: 225 is stored on disk
2025-10-09T21:43:39.425242Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425242Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.425246Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 227, partNo: 0, Offset: 226 is stored on disk
2025-10-09T21:43:39.425248Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425254Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 228, partNo: 0, Offset: 227 is stored on disk
2025-10-09T21:43:39.425256Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425260Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 229, partNo: 0, Offset: 228 is stored on disk
2025-10-09T21:43:39.425263Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425266Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 230, partNo: 0, Offset: 229 is stored on disk
2025-10-09T21:43:39.425269Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425272Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 231, partNo: 0, Offset: 230 is stored on disk
2025-10-09T21:43:39.425274Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425278Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 232, partNo: 0, Offset: 231 is stored on disk
2025-10-09T21:43:39.425281Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425283Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 233, partNo: 0, Offset: 232 is stored on disk
2025-10-09T21:43:39.425286Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425290Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 234, partNo: 0, Offset: 233 is stored on disk
2025-10-09T21:43:39.425293Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425296Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 235, partNo: 0, Offset: 234 is stored on disk
2025-10-09T21:43:39.425296Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.425298Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425302Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 236, partNo: 0, Offset: 235 is stored on disk
2025-10-09T21:43:39.425304Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.425304Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.425309Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 237, partNo: 0, Offset: 236 is stored on disk
2025-10-09T21:43:39.425338Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.425362Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.425364Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (36)
2025-10-09T21:43:39.425384Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 238 partNo 0
2025-10-09T21:43:39.425411Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 238 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 26229 count 1 nextOffset 238 batches 1
2025-10-09T21:43:39.425418Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 239 partNo 0
2025-10-09T21:43:39.425422Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 239 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 52536 count 2 nextOffset 239 batches 1
2025-10-09T21:43:39.425425Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 240 partNo 0
2025-10-09T21:43:39.425429Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 240 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 78953 count 3 nextOffset 240 batches 1
2025-10-09T21:43:39.425432Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 241 partNo 0
2025-10-09T21:43:39.425436Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 241 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 105480 count 4 nextOffset 241 batches 1
2025-10-09T21:43:39.425440Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 242 partNo 0
2025-10-09T21:43:39.425443Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 242 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 132117 count 5 nextOffset 242 batches 1
2025-10-09T21:43:39.425446Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 243 partNo 0
2025-10-09T21:43:39.425450Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 243 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 158864 count 6 nextOffset 243 batches 1
2025-10-09T21:43:39.425453Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 244 partNo 0
2025-10-09T21:43:39.425456Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 244 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 185721 count 7 nextOffset 244 batches 1
2025-10-09T21:43:39.425460Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 245 partNo 0
2025-10-09T21:43:39.425463Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 245 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 212688 count 8 nextOffset 245 batches 1
2025-10-09T21:43:39.425466Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 246 partNo 0
2025-10-09T21:43:39.425470Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 246 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 239765 count 9 nextOffset 246 batches 1
2025-10-09T21:43:39.425473Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 247 partNo 0
2025-10-09T21:43:39.425476Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 247 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 266952 count 10 nextOffset 247 batches 1
2025-10-09T21:43:39.425479Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 248 partNo 0
2025-10-09T21:43:39.425483Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 248 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 294249 count 11 nextOffset 248 batches 1
2025-10-09T21:43:39.425499Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 249 partNo 0
2025-10-09T21:43:39.425502Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 249 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 321656 count 12 nextOffset 249 batches 1
2025-10-09T21:43:39.425505Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 250 partNo 0
2025-10-09T21:43:39.425509Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 250 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 349173 count 13 nextOffset 250 batches 1
2025-10-09T21:43:39.425513Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 251 partNo 0
2025-10-09T21:43:39.425517Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 251 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 376800 count 14 nextOffset 251 batches 1
2025-10-09T21:43:39.425520Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 252 partNo 0
2025-10-09T21:43:39.425524Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 252 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 404537 count 15 nextOffset 252 batches 1
2025-10-09T21:43:39.425527Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 253 partNo 0
2025-10-09T21:43:39.425530Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 253 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 432384 count 16 nextOffset 253 batches 1
2025-10-09T21:43:39.425533Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 254 partNo 0
2025-10-09T21:43:39.425538Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 254 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 460341 count 17 nextOffset 254 batches 1
2025-10-09T21:43:39.425541Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 255 partNo 0
2025-10-09T21:43:39.425553Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 255 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 488408 count 18 nextOffset 255 batches 1
2025-10-09T21:43:39.425556Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 256 partNo 0
2025-10-09T21:43:39.425602Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.425689Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.425753Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.425755Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 256 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 516060 count 19 nextOffset 256 batches 1
2025-10-09T21:43:39.425765Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 257 partNo 0
2025-10-09T21:43:39.425780Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 257 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 544379 count 20 nextOffset 257 batches 2
2025-10-09T21:43:39.425783Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 258 partNo 0
2025-10-09T21:43:39.425788Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 258 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 572776 count 21 nextOffset 258 batches 2
2025-10-09T21:43:39.425791Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 259 partNo 0
2025-10-09T21:43:39.425795Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 259 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 601283 count 22 nextOffset 259 batches 2
2025-10-09T21:43:39.425798Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 260 partNo 0
2025-10-09T21:43:39.425802Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 260 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 629900 count 23 nextOffset 260 batches 2
2025-10-09T21:43:39.425808Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 261 partNo 0
2025-10-09T21:43:39.425812Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 261 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 658627 count 24 nextOffset 261 batches 2
2025-10-09T21:43:39.425815Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 262 partNo 0
2025-10-09T21:43:39.425819Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 262 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 687464 count 25 nextOffset 262 batches 2
2025-10-09T21:43:39.425822Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 263 partNo 0
2025-10-09T21:43:39.425826Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 263 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 716411 count 26 nextOffset 263 batches 2
2025-10-09T21:43:39.425829Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 264 partNo 0
2025-10-09T21:43:39.425833Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 264 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 745468 count 27 nextOffset 264 batches 2
2025-10-09T21:43:39.425836Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 265 partNo 0
2025-10-09T21:43:39.425840Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 265 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 774635 count 28 nextOffset 265 batches 2
2025-10-09T21:43:39.425844Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 266 partNo 0
2025-10-09T21:43:39.425847Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.425848Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 266 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 803912 count 29 nextOffset 266 batches 2
2025-10-09T21:43:39.425853Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 267 partNo 0
2025-10-09T21:43:39.425854Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.425857Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 267 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 833299 count 30 nextOffset 267 batches 2
2025-10-09T21:43:39.425860Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 268 partNo 0
2025-10-09T21:43:39.425864Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 268 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 862796 count 31 nextOffset 268 batches 2
2025-10-09T21:43:39.425868Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 269 partNo 0
2025-10-09T21:43:39.425873Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 269 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 892403 count 32 nextOffset 269 batches 2
2025-10-09T21:43:39.425876Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 270 partNo 0
2025-10-09T21:43:39.425880Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 270 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 922120 count 33 nextOffset 270 batches 2
2025-10-09T21:43:39.425884Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 271 partNo 0
2025-10-09T21:43:39.425886Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 271 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 951947 count 34 nextOffset 271 batches 2
2025-10-09T21:43:39.425903Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 272 partNo 0
2025-10-09T21:43:39.425907Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 272 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 981884 count 35 nextOffset 272 batches 2
2025-10-09T21:43:39.425910Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 273 partNo 0
2025-10-09T21:43:39.425915Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 273 partNo 0 FormedBlobsCount 0 NewHead: Offset 237 PartNo 0 PackedSize 1011931 count 36 nextOffset 273 batches 2
2025-10-09T21:43:39.425918Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.426023Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.426129Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.426189Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.426253Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.426363Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.426435Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.426528Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 237,36 HeadOffset 237 endOffset 237 curOffset 273 d0000000000_00000000000000000237_00000_0000000036_00000? size 1011468 WTime 1760046219424
2025-10-09T21:43:39.426535Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.426660Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.426725Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.426840Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.426975Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.427066Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.427131Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.427222Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.427383Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.427427Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.427448Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 429  requestId: <none> cookie: 213
2025-10-09T21:43:39.427468Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.427473Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.427484Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 274 partNo : 0 messageNo: 548 size 30115 offset: -1
2025-10-09T21:43:39.427490Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.427492Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.427493Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.427497Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 275 partNo : 0 messageNo: 549 size 30225 offset: -1
2025-10-09T21:43:39.427503Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 430  requestId: <none> cookie: 214
2025-10-09T21:43:39.427504Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427510Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 431  requestId: <none> cookie: 215
2025-10-09T21:43:39.427516Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 432  requestId: <none> cookie: 216
2025-10-09T21:43:39.427522Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 433  requestId: <none> cookie: 217
2025-10-09T21:43:39.427528Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 439  requestId: <none> cookie: 218
2025-10-09T21:43:39.427534Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 440  requestId: <none> cookie: 219
2025-10-09T21:43:39.427539Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 441  requestId: <none> cookie: 220
2025-10-09T21:43:39.427545Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 442  requestId: <none> cookie: 221
2025-10-09T21:43:39.427550Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 443  requestId: <none> cookie: 222
2025-10-09T21:43:39.427558Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 449  requestId: <none> cookie: 223
2025-10-09T21:43:39.427564Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 450  requestId: <none> cookie: 224
2025-10-09T21:43:39.427573Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427577Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 451  requestId: <none> cookie: 225
2025-10-09T21:43:39.427582Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427583Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 452  requestId: <none> cookie: 226
2025-10-09T21:43:39.427585Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427589Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 453  requestId: <none> cookie: 227
2025-10-09T21:43:39.427589Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427594Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 459  requestId: <none> cookie: 228
2025-10-09T21:43:39.427595Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427599Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427599Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 460  requestId: <none> cookie: 229
2025-10-09T21:43:39.427603Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427606Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 461  requestId: <none> cookie: 230
2025-10-09T21:43:39.427606Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427610Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427611Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 462  requestId: <none> cookie: 231
2025-10-09T21:43:39.427614Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427616Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 463  requestId: <none> cookie: 232
2025-10-09T21:43:39.427621Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 469  requestId: <none> cookie: 233
2025-10-09T21:43:39.427626Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427627Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 470  requestId: <none> cookie: 234
2025-10-09T21:43:39.427633Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427634Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 471  requestId: <none> cookie: 235
2025-10-09T21:43:39.427640Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 472  requestId: <none> cookie: 236
2025-10-09T21:43:39.427646Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 473  requestId: <none> cookie: 237
2025-10-09T21:43:39.427651Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.427653Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.427658Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.427660Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.427663Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.427665Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.427668Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.427670Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.427674Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.427676Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.427681Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.427694Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 237 partNo 0 count 36 size 1011468
2025-10-09T21:43:39.427701Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.427712Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.427714Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.427716Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.427732Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427738Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427743Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.427746Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427750Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427755Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427757Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 550  requestId: <none> cookie: 276
2025-10-09T21:43:39.427760Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427761Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 551  requestId: <none> cookie: 277
2025-10-09T21:43:39.427764Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427764Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 552  requestId: <none> cookie: 278
2025-10-09T21:43:39.427767Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 553  requestId: <none> cookie: 279
2025-10-09T21:43:39.427767Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427771Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 554  requestId: <none> cookie: 280
2025-10-09T21:43:39.427774Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427823Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427828Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427831Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.427991Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.428001Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.428005Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.428010Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.428019Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.428056Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.428107Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428125Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428137Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 276 partNo : 0 messageNo: 555 size 30335 offset: -1
2025-10-09T21:43:39.428145Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428147Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428150Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 277 partNo : 0 messageNo: 556 size 30445 offset: -1
2025-10-09T21:43:39.428154Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428156Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428159Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 278 partNo : 0 messageNo: 557 size 30555 offset: -1
2025-10-09T21:43:39.428163Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428165Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428168Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 279 partNo : 0 messageNo: 558 size 30665 offset: -1
2025-10-09T21:43:39.428173Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.428177Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428178Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428181Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.428186Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.428190Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.428195Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 280 partNo : 0 messageNo: 559 size 30775 offset: -1
2025-10-09T21:43:39.428196Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.428200Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.428208Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.428210Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.428214Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.428216Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.428218Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.428221Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.428223Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.428225Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.428227Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.428230Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428232Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428236Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428237Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428241Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428242Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428245Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428247Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428251Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428252Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428266Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 560  requestId: <none> cookie: 281
2025-10-09T21:43:39.428271Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 561  requestId: <none> cookie: 282
2025-10-09T21:43:39.428273Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 562  requestId: <none> cookie: 283
2025-10-09T21:43:39.428276Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 563  requestId: <none> cookie: 284
2025-10-09T21:43:39.428278Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 564  requestId: <none> cookie: 285
2025-10-09T21:43:39.428286Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:39.428295Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.428301Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.428311Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.428315Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.428318Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.428418Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.428427Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.428432Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.428437Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.428442Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.428456Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428459Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428467Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 281 partNo : 0 messageNo: 565 size 30885 offset: -1
2025-10-09T21:43:39.428472Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428473Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428477Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 282 partNo : 0 messageNo: 566 size 30995 offset: -1
2025-10-09T21:43:39.428481Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428482Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428487Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 283 partNo : 0 messageNo: 567 size 31105 offset: -1
2025-10-09T21:43:39.428490Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428492Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428496Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 284 partNo : 0 messageNo: 568 size 31215 offset: -1
2025-10-09T21:43:39.428499Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428500Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428504Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 285 partNo : 0 messageNo: 569 size 31325 offset: -1
2025-10-09T21:43:39.428508Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428511Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428518Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428519Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428523Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428524Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428527Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428529Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428532Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428536Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.428540Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.428547Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.428548Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.428551Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.428552Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.428554Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.428557Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.428559Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.428561Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.428563Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.428572Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 570  requestId: <none> cookie: 286
2025-10-09T21:43:39.428578Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 571  requestId: <none> cookie: 287
2025-10-09T21:43:39.428580Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 572  requestId: <none> cookie: 288
2025-10-09T21:43:39.428583Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 573  requestId: <none> cookie: 289
2025-10-09T21:43:39.428586Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 574  requestId: <none> cookie: 290
2025-10-09T21:43:39.428595Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.428613Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.428618Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.428622Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.428630Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.428653Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.428664Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.428670Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.428688Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.428695Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.428990Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.428995Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429012Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 286 partNo : 0 messageNo: 575 size 31435 offset: -1
2025-10-09T21:43:39.429019Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429021Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429025Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 287 partNo : 0 messageNo: 576 size 31545 offset: -1
2025-10-09T21:43:39.429029Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429032Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429037Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 288 partNo : 0 messageNo: 577 size 31655 offset: -1
2025-10-09T21:43:39.429041Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429043Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429046Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 289 partNo : 0 messageNo: 578 size 31765 offset: -1
2025-10-09T21:43:39.429050Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429051Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429055Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 290 partNo : 0 messageNo: 579 size 31875 offset: -1
2025-10-09T21:43:39.429058Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429061Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429067Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429069Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429072Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429074Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429076Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429078Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429082Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429084Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429088Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.429108Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.429110Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.429113Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.429115Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.429117Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.429119Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.429121Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.429124Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.429127Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.429137Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 580  requestId: <none> cookie: 291
2025-10-09T21:43:39.429148Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 581  requestId: <none> cookie: 292
2025-10-09T21:43:39.429151Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 582  requestId: <none> cookie: 293
2025-10-09T21:43:39.429154Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 583  requestId: <none> cookie: 294
2025-10-09T21:43:39.429157Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 584  requestId: <none> cookie: 295
2025-10-09T21:43:39.429175Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.429181Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.429185Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.429188Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.429199Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.429219Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.429228Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.429233Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.429238Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.429242Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429243Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:39.429248Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429269Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 291 partNo : 0 messageNo: 585 size 31985 offset: -1
2025-10-09T21:43:39.429279Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429282Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429287Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 292 partNo : 0 messageNo: 586 size 32095 offset: -1
2025-10-09T21:43:39.429291Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429295Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429299Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 293 partNo : 0 messageNo: 587 size 32205 offset: -1
2025-10-09T21:43:39.429303Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429305Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429309Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 294 partNo : 0 messageNo: 588 size 32315 offset: -1
2025-10-09T21:43:39.429314Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429316Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429320Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 295 partNo : 0 messageNo: 589 size 32425 offset: -1
2025-10-09T21:43:39.429324Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429326Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429330Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429332Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429336Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429337Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429341Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429343Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429347Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429349Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429354Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.429363Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.429365Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.429368Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.429370Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.429372Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.429374Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.429377Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.429379Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.429382Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.429393Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 590  requestId: <none> cookie: 296
2025-10-09T21:43:39.429405Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 591  requestId: <none> cookie: 297
2025-10-09T21:43:39.429409Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 592  requestId: <none> cookie: 298
2025-10-09T21:43:39.429414Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 593  requestId: <none> cookie: 299
2025-10-09T21:43:39.429417Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 594  requestId: <none> cookie: 300
2025-10-09T21:43:39.429417Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.429424Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.429428Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.429442Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.429448Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.429507Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429511Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429531Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 296 partNo : 0 messageNo: 595 size 32535 offset: -1
2025-10-09T21:43:39.429536Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429538Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429542Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 297 partNo : 0 messageNo: 596 size 32645 offset: -1
2025-10-09T21:43:39.429545Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429547Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429552Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 298 partNo : 0 messageNo: 597 size 32755 offset: -1
2025-10-09T21:43:39.429556Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429558Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429562Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 299 partNo : 0 messageNo: 598 size 32865 offset: -1
2025-10-09T21:43:39.429565Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.429567Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.429570Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 300 partNo : 0 messageNo: 599 size 32975 offset: -1
2025-10-09T21:43:39.429576Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.429582Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.429585Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.429587Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.429589Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.429592Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.429593Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.429596Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.429598Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:39.429600Z node 3 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:43:39.430952Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 237 count 36 size 1011468 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.430967Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.430982Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 237 partno 0 count 36 parts 0 suffix '63' size 1011468
2025-10-09T21:43:39.430985Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1011132 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.431004Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431012Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 238, partNo: 0, Offset: 237 is stored on disk
2025-10-09T21:43:39.431017Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431023Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 239, partNo: 0, Offset: 238 is stored on disk
2025-10-09T21:43:39.431025Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431029Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 240, partNo: 0, Offset: 239 is stored on disk
2025-10-09T21:43:39.431031Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431034Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 241, partNo: 0, Offset: 240 is stored on disk
2025-10-09T21:43:39.431036Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431041Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 242, partNo: 0, Offset: 241 is stored on disk
2025-10-09T21:43:39.431044Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431047Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 243, partNo: 0, Offset: 242 is stored on disk
2025-10-09T21:43:39.431049Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431052Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 244, partNo: 0, Offset: 243 is stored on disk
2025-10-09T21:43:39.431054Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431057Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 245, partNo: 0, Offset: 244 is stored on disk
2025-10-09T21:43:39.431060Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431063Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 246, partNo: 0, Offset: 245 is stored on disk
2025-10-09T21:43:39.431065Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431068Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 247, partNo: 0, Offset: 246 is stored on disk
2025-10-09T21:43:39.431071Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431074Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 248, partNo: 0, Offset: 247 is stored on disk
2025-10-09T21:43:39.431076Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431095Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 249, partNo: 0, Offset: 248 is stored on disk
2025-10-09T21:43:39.431097Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431104Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 250, partNo: 0, Offset: 249 is stored on disk
2025-10-09T21:43:39.431106Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431110Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 251, partNo: 0, Offset: 250 is stored on disk
2025-10-09T21:43:39.431112Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431126Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 252, partNo: 0, Offset: 251 is stored on disk
2025-10-09T21:43:39.431128Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431132Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 253, partNo: 0, Offset: 252 is stored on disk
2025-10-09T21:43:39.431134Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431137Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 254, partNo: 0, Offset: 253 is stored on disk
2025-10-09T21:43:39.431140Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431143Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 255, partNo: 0, Offset: 254 is stored on disk
2025-10-09T21:43:39.431146Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431149Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 256, partNo: 0, Offset: 255 is stored on disk
2025-10-09T21:43:39.431152Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431155Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 257, partNo: 0, Offset: 256 is stored on disk
2025-10-09T21:43:39.431157Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431162Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 258, partNo: 0, Offset: 257 is stored on disk
2025-10-09T21:43:39.431164Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431168Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 259, partNo: 0, Offset: 258 is stored on disk
2025-10-09T21:43:39.431170Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431175Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 260, partNo: 0, Offset: 259 is stored on disk
2025-10-09T21:43:39.431178Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431182Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 261, partNo: 0, Offset: 260 is stored on disk
2025-10-09T21:43:39.431184Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431187Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 262, partNo: 0, Offset: 261 is stored on disk
2025-10-09T21:43:39.431190Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431193Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 263, partNo: 0, Offset: 262 is stored on disk
2025-10-09T21:43:39.431195Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431199Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 264, partNo: 0, Offset: 263 is stored on disk
2025-10-09T21:43:39.431201Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431205Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 265, partNo: 0, Offset: 264 is stored on disk
2025-10-09T21:43:39.431207Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431210Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 266, partNo: 0, Offset: 265 is stored on disk
2025-10-09T21:43:39.431212Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431214Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 267, partNo: 0, Offset: 266 is stored on disk
2025-10-09T21:43:39.431216Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431219Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 268, partNo: 0, Offset: 267 is stored on disk
2025-10-09T21:43:39.431221Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431224Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 269, partNo: 0, Offset: 268 is stored on disk
2025-10-09T21:43:39.431226Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431229Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 270, partNo: 0, Offset: 269 is stored on disk
2025-10-09T21:43:39.431231Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431235Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 271, partNo: 0, Offset: 270 is stored on disk
2025-10-09T21:43:39.431237Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431241Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 272, partNo: 0, Offset: 271 is stored on disk
2025-10-09T21:43:39.431243Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.431246Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 273, partNo: 0, Offset: 272 is stored on disk
2025-10-09T21:43:39.431279Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.431300Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.431306Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (27)
2025-10-09T21:43:39.431325Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 274 partNo 0
2025-10-09T21:43:39.431351Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 274 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 30189 count 1 nextOffset 274 batches 1
2025-10-09T21:43:39.431357Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 275 partNo 0
2025-10-09T21:43:39.431361Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 275 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 60456 count 2 nextOffset 275 batches 1
2025-10-09T21:43:39.431364Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 276 partNo 0
2025-10-09T21:43:39.431368Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 276 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 90833 count 3 nextOffset 276 batches 1
2025-10-09T21:43:39.431371Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 277 partNo 0
2025-10-09T21:43:39.431374Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 277 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 121320 count 4 nextOffset 277 batches 1
2025-10-09T21:43:39.431378Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 278 partNo 0
2025-10-09T21:43:39.431382Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 278 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 151917 count 5 nextOffset 278 batches 1
2025-10-09T21:43:39.431385Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 279 partNo 0
2025-10-09T21:43:39.431388Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 279 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 182624 count 6 nextOffset 279 batches 1
2025-10-09T21:43:39.431391Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 280 partNo 0
2025-10-09T21:43:39.431394Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 280 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 213441 count 7 nextOffset 280 batches 1
2025-10-09T21:43:39.431397Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 281 partNo 0
2025-10-09T21:43:39.431400Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 281 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 244368 count 8 nextOffset 281 batches 1
2025-10-09T21:43:39.431403Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 282 partNo 0
2025-10-09T21:43:39.431417Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 282 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 275405 count 9 nextOffset 282 batches 1
2025-10-09T21:43:39.431420Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 283 partNo 0
2025-10-09T21:43:39.431429Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 283 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 306552 count 10 nextOffset 283 batches 1
2025-10-09T21:43:39.431432Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 284 partNo 0
2025-10-09T21:43:39.431438Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 284 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 337809 count 11 nextOffset 284 batches 1
2025-10-09T21:43:39.431441Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 285 partNo 0
2025-10-09T21:43:39.431444Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 285 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 369176 count 12 nextOffset 285 batches 1
2025-10-09T21:43:39.431447Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 286 partNo 0
2025-10-09T21:43:39.431450Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 286 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 400653 count 13 nextOffset 286 batches 1
2025-10-09T21:43:39.431453Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 287 partNo 0
2025-10-09T21:43:39.431457Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 287 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 432240 count 14 nextOffset 287 batches 1
2025-10-09T21:43:39.431460Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 288 partNo 0
2025-10-09T21:43:39.431464Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 288 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 463937 count 15 nextOffset 288 batches 1
2025-10-09T21:43:39.431469Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 289 partNo 0
2025-10-09T21:43:39.431473Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 289 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 495744 count 16 nextOffset 289 batches 1
2025-10-09T21:43:39.431476Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 290 partNo 0
2025-10-09T21:43:39.432109Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 290 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 527198 count 17 nextOffset 290 batches 1
2025-10-09T21:43:39.432118Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 291 partNo 0
2025-10-09T21:43:39.432131Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 291 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 559257 count 18 nextOffset 291 batches 2
2025-10-09T21:43:39.432134Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 292 partNo 0
2025-10-09T21:43:39.432138Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 292 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 591394 count 19 nextOffset 292 batches 2
2025-10-09T21:43:39.432141Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 293 partNo 0
2025-10-09T21:43:39.432145Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 293 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 623641 count 20 nextOffset 293 batches 2
2025-10-09T21:43:39.432148Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 294 partNo 0
2025-10-09T21:43:39.432151Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 294 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 655998 count 21 nextOffset 294 batches 2
2025-10-09T21:43:39.432153Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 295 partNo 0
2025-10-09T21:43:39.432156Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 295 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 688465 count 22 nextOffset 295 batches 2
2025-10-09T21:43:39.432159Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 296 partNo 0
2025-10-09T21:43:39.432163Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 296 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 721042 count 23 nextOffset 296 batches 2
2025-10-09T21:43:39.432166Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 297 partNo 0
2025-10-09T21:43:39.432169Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 297 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 753729 count 24 nextOffset 297 batches 2
2025-10-09T21:43:39.432172Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 298 partNo 0
2025-10-09T21:43:39.432175Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 298 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 786526 count 25 nextOffset 298 batches 2
2025-10-09T21:43:39.432179Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 299 partNo 0
2025-10-09T21:43:39.432185Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 299 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 819433 count 26 nextOffset 299 batches 2
2025-10-09T21:43:39.432188Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 300 partNo 0
2025-10-09T21:43:39.432193Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 300 partNo 0 FormedBlobsCount 0 NewHead: Offset 273 PartNo 0 PackedSize 852450 count 27 nextOffset 300 batches 2
2025-10-09T21:43:39.432196Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.432359Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 273,27 HeadOffset 273 endOffset 273 curOffset 300 d0000000000_00000000000000000273_00000_0000000027_00000? size 852204 WTime 1760046219431
2025-10-09T21:43:39.433087Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.433116Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 479  requestId: <none> cookie: 238
2025-10-09T21:43:39.433134Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 480  requestId: <none> cookie: 239
2025-10-09T21:43:39.433141Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 481  requestId: <none> cookie: 240
2025-10-09T21:43:39.433146Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 482  requestId: <none> cookie: 241
2025-10-09T21:43:39.433152Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 483  requestId: <none> cookie: 242
2025-10-09T21:43:39.433158Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 489  requestId: <none> cookie: 243
2025-10-09T21:43:39.433163Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 490  requestId: <none> cookie: 244
2025-10-09T21:43:39.433169Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 491  requestId: <none> cookie: 245
2025-10-09T21:43:39.433175Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 492  requestId: <none> cookie: 246
2025-10-09T21:43:39.433181Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 493  requestId: <none> cookie: 247
2025-10-09T21:43:39.433187Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 499  requestId: <none> cookie: 248
2025-10-09T21:43:39.433192Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 500  requestId: <none> cookie: 249
2025-10-09T21:43:39.433192Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433198Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 501  requestId: <none> cookie: 250
2025-10-09T21:43:39.433204Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433204Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 502  requestId: <none> cookie: 251
2025-10-09T21:43:39.433208Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433210Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 503  requestId: <none> cookie: 252
2025-10-09T21:43:39.433212Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433216Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433220Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433221Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 509  requestId: <none> cookie: 253
2025-10-09T21:43:39.433224Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433227Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 510  requestId: <none> cookie: 254
2025-10-09T21:43:39.433232Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 511  requestId: <none> cookie: 255
2025-10-09T21:43:39.433237Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 512  requestId: <none> cookie: 256
2025-10-09T21:43:39.433239Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433243Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 513  requestId: <none> cookie: 257
2025-10-09T21:43:39.433244Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433248Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 519  requestId: <none> cookie: 258
2025-10-09T21:43:39.433248Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433252Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433254Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 520  requestId: <none> cookie: 259
2025-10-09T21:43:39.433255Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433259Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 521  requestId: <none> cookie: 260
2025-10-09T21:43:39.433264Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 522  requestId: <none> cookie: 261
2025-10-09T21:43:39.433265Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433269Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433270Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 523  requestId: <none> cookie: 262
2025-10-09T21:43:39.433276Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 526  requestId: <none> cookie: 263
2025-10-09T21:43:39.433276Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433281Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 527  requestId: <none> cookie: 264
2025-10-09T21:43:39.433281Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433286Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433286Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 530  requestId: <none> cookie: 265
2025-10-09T21:43:39.433289Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433293Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433296Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 531  requestId: <none> cookie: 266
2025-10-09T21:43:39.433298Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433301Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 534  requestId: <none> cookie: 267
2025-10-09T21:43:39.433306Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 535  requestId: <none> cookie: 268
2025-10-09T21:43:39.433313Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433315Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 538  requestId: <none> cookie: 269
2025-10-09T21:43:39.433318Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433321Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 539  requestId: <none> cookie: 270
2025-10-09T21:43:39.433322Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433326Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 541  requestId: <none> cookie: 271
2025-10-09T21:43:39.433331Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 544  requestId: <none> cookie: 272
2025-10-09T21:43:39.433337Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 545  requestId: <none> cookie: 273
2025-10-09T21:43:39.433344Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:39.433356Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 273 partNo 0 count 27 size 852204
2025-10-09T21:43:39.433461Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433476Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433486Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433520Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433534Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433545Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433556Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433560Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433579Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433589Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433593Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433597Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.433600Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.437702Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 273 count 27 size 852204 actorID [3:7559340952763857360:2463]
2025-10-09T21:43:39.437734Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:39.437746Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 273 partno 0 count 27 parts 0 suffix '63' size 852204
2025-10-09T21:43:39.437752Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 851904 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:39.437761Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437771Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 274, partNo: 0, Offset: 273 is stored on disk
2025-10-09T21:43:39.437776Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437780Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 275, partNo: 0, Offset: 274 is stored on disk
2025-10-09T21:43:39.437782Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437787Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 276, partNo: 0, Offset: 275 is stored on disk
2025-10-09T21:43:39.437790Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437793Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 277, partNo: 0, Offset: 276 is stored on disk
2025-10-09T21:43:39.437795Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437799Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 278, partNo: 0, Offset: 277 is stored on disk
2025-10-09T21:43:39.437801Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437844Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 279, partNo: 0, Offset: 278 is stored on disk
2025-10-09T21:43:39.437846Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437850Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 280, partNo: 0, Offset: 279 is stored on disk
2025-10-09T21:43:39.437852Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437856Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 281, partNo: 0, Offset: 280 is stored on disk
2025-10-09T21:43:39.437857Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437867Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 282, partNo: 0, Offset: 281 is stored on disk
2025-10-09T21:43:39.437870Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437874Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 283, partNo: 0, Offset: 282 is stored on disk
2025-10-09T21:43:39.437876Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437879Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 284, partNo: 0, Offset: 283 is stored on disk
2025-10-09T21:43:39.437881Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437884Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 285, partNo: 0, Offset: 284 is stored on disk
2025-10-09T21:43:39.437886Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437888Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 286, partNo: 0, Offset: 285 is stored on disk
2025-10-09T21:43:39.437891Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437897Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 287, partNo: 0, Offset: 286 is stored on disk
2025-10-09T21:43:39.437899Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437902Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 288, partNo: 0, Offset: 287 is stored on disk
2025-10-09T21:43:39.437904Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437906Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 289, partNo: 0, Offset: 288 is stored on disk
2025-10-09T21:43:39.437908Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437919Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 290, partNo: 0, Offset: 289 is stored on disk
2025-10-09T21:43:39.437921Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437924Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 291, partNo: 0, Offset: 290 is stored on disk
DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..19371 bytes.. Information: { Offset: 176 ProducerId: "src_id" SeqNo: 177 CreateTime: 2025-10-09T21:43:39.384000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..19481 bytes.. Information: { Offset: 177 ProducerId: "src_id" SeqNo: 178 CreateTime: 2025-10-09T21:43:39.384000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..19591 bytes.. Information: { Offset: 178 ProducerId: "src_id" SeqNo: 179 CreateTime: 2025-10-09T21:43:39.384000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..19701 bytes.. Information: { Offset: 179 ProducerId: "src_id" SeqNo: 180 CreateTime: 2025-10-09T21:43:39.384000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..19811 bytes.. Information: { Offset: 180 ProducerId: "src_id" SeqNo: 181 CreateTime: 2025-10-09T21:43:39.385000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..19921 bytes.. Information: { Offset: 181 ProducerId: "src_id" SeqNo: 182 CreateTime: 2025-10-09T21:43:39.385000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..20031 bytes.. Information: { Offset: 182 ProducerId: "src_id" SeqNo: 183 CreateTime: 2025-10-09T21:43:39.385000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..20141 bytes.. Information: { Offset: 183 ProducerId: "src_id" SeqNo: 184 CreateTime: 2025-10-09T21:43:39.385000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..20251 bytes.. Information: { Offset: 184 ProducerId: "src_id" SeqNo: 185 CreateTime: 2025-10-09T21:43:39.385000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..20361 bytes.. Information: { Offset: 185 ProducerId: "src_id" SeqNo: 186 CreateTime: 2025-10-09T21:43:39.385000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..20471 bytes.. Information: { Offset: 186 ProducerId: "src_id" SeqNo: 187 CreateTime: 2025-10-09T21:43:39.385000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..20581 bytes.. Information: { Offset: 187 ProducerId: "src_id" SeqNo: 188 CreateTime: 2025-10-09T21:43:39.385000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..20691 bytes.. Information: { Offset: 188 ProducerId: "src_id" SeqNo: 189 CreateTime: 2025-10-09T21:43:39.385000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..20801 bytes.. Information: { Offset: 189 ProducerId: "src_id" SeqNo: 190 CreateTime: 2025-10-09T21:43:39.385000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..20911 bytes.. Information: { Offset: 190 ProducerId: "src_id" SeqNo: 191 CreateTime: 2025-10-09T21:43:39.385000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..21021 bytes.. Information: { Offset: 191 ProducerId: "src_id" SeqNo: 192 CreateTime: 2025-10-09T21:43:39.386000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..21131 bytes.. Information: { Offset: 192 ProducerId: "src_id" SeqNo: 193 CreateTime: 2025-10-09T21:43:39.386000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..21241 bytes.. Information: { Offset: 193 ProducerId: "src_id" SeqNo: 194 CreateTime: 2025-10-09T21:43:39.386000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..21351 bytes.. Information: { Offset: 194 ProducerId: "src_id" SeqNo: 195 CreateTime: 2025-10-09T21:43:39.386000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..21461 bytes.. Information: { Offset: 195 ProducerId: "src_id" SeqNo: 196 CreateTime: 2025-10-09T21:43:39.386000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..21571 bytes.. Information: { Offset: 196 ProducerId: "src_id" SeqNo: 197 CreateTime: 2025-10-09T21:43:39.386000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..21681 bytes.. Information: { Offset: 197 ProducerId: "src_id" SeqNo: 198 CreateTime: 2025-10-09T21:43:39.386000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..21791 bytes.. Information: { Offset: 198 ProducerId: "src_id" SeqNo: 199 CreateTime: 2025-10-09T21:43:39.386000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..21901 bytes.. Information: { Offset: 199 ProducerId: "src_id" SeqNo: 200 CreateTime: 2025-10-09T21:43:39.386000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..22011 bytes.. Information: { Offset: 200 ProducerId: "src_id" SeqNo: 201 CreateTime: 2025-10-09T21:43:39.386000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..22121 bytes.. Information: { Offset: 201 ProducerId: "src_id" SeqNo: 202 CreateTime: 2025-10-09T21:43:39.386000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.437926Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437928Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 292, partNo: 0, Offset: 291 is stored on disk
2025-10-09T21:43:39.437930Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437932Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 293, partNo: 0, Offset: 292 is stored on disk
2025-10-09T21:43:39.437934Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437937Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 294, partNo: 0, Offset: 293 is stored on disk
2025-10-09T21:43:39.437939Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437942Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 295, partNo: 0, Offset: 294 is stored on disk
2025-10-09T21:43:39.437944Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437947Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 296, partNo: 0, Offset: 295 is stored on disk
2025-10-09T21:43:39.437948Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437951Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 297, partNo: 0, Offset: 296 is stored on disk
2025-10-09T21:43:39.437952Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437955Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 298, partNo: 0, Offset: 297 is stored on disk
2025-10-09T21:43:39.437957Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437960Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 299, partNo: 0, Offset: 298 is stored on disk
2025-10-09T21:43:39.437961Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:39.437964Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 300, partNo: 0, Offset: 299 is stored on disk
2025-10-09T21:43:39.437999Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.438005Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.438007Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.438015Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.438041Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 548  requestId: <none> cookie: 274
2025-10-09T21:43:39.438056Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 549  requestId: <none> cookie: 275
2025-10-09T21:43:39.438062Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 555  requestId: <none> cookie: 276
2025-10-09T21:43:39.438067Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 556  requestId: <none> cookie: 277
2025-10-09T21:43:39.438075Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 557  requestId: <none> cookie: 278
2025-10-09T21:43:39.438080Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 558  requestId: <none> cookie: 279
2025-10-09T21:43:39.438084Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 559  requestId: <none> cookie: 280
2025-10-09T21:43:39.438088Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 565  requestId: <none> cookie: 281
2025-10-09T21:43:39.438093Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 566  requestId: <none> cookie: 282
2025-10-09T21:43:39.438097Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 567  requestId: <none> cookie: 283
2025-10-09T21:43:39.438102Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 568  requestId: <none> cookie: 284
2025-10-09T21:43:39.438106Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 569  requestId: <none> cookie: 285
2025-10-09T21:43:39.438111Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 575  requestId: <none> cookie: 286
2025-10-09T21:43:39.438118Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 576  requestId: <none> cookie: 287
2025-10-09T21:43:39.438123Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 577  requestId: <none> cookie: 288
2025-10-09T21:43:39.438127Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 578  requestId: <none> cookie: 289
2025-10-09T21:43:39.438132Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 579  requestId: <none> cookie: 290
2025-10-09T21:43:39.438136Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 585  requestId: <none> cookie: 291
2025-10-09T21:43:39.438141Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 586  requestId: <none> cookie: 292
2025-10-09T21:43:39.438146Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 587  requestId: <none> cookie: 293
2025-10-09T21:43:39.438151Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 588  requestId: <none> cookie: 294
2025-10-09T21:43:39.438156Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 589  requestId: <none> cookie: 295
2025-10-09T21:43:39.524420Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 26, size 539396 bytes
2025-10-09T21:43:39.438160Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 595  requestId: <none> cookie: 296
2025-10-09T21:43:39.438165Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 596  requestId: <none> cookie: 297
2025-10-09T21:43:39.438169Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 597  requestId: <none> cookie: 298
2025-10-09T21:43:39.438174Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 598  requestId: <none> cookie: 299
2025-10-09T21:43:39.438187Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 599  requestId: <none> cookie: 300
2025-10-09T21:43:39.438202Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438207Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438210Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438219Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438222Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438225Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438227Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438230Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438234Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438237Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438242Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438245Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438248Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438250Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438253Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438255Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438258Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438261Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438269Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438272Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438274Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438277Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438279Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438282Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438285Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438288Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.438291Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:39.470215Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 149083 } }
2025-10-09T21:43:39.470322Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# 69fddedf-9af69d0f-e5f7345b-930d3957
2025-10-09T21:43:39.470355Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# c01639ca-5040ec8-f3b3fd14-e453eba4, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 43, size# 141934, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.470390Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 43 maxSize 141934 maxTimeLagMs 0 readTimestampMs 0 readOffset 176 EndOffset 212 ClientCommitOffset 0 committedOffset 0 Guid c01639ca-5040ec8-f3b3fd14-e453eba4
2025-10-09T21:43:39.470646Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 382354 } }
2025-10-09T21:43:39.470682Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.470695Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.470762Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 18 Topic 'rt3.dc1--test-topic' partition 0 user user offset 176 count 43 size 141934 endOffset 300 max time lag 0ms effective offset 176
2025-10-09T21:43:39.470765Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# 3e49e204-8e37e267-d07876d4-7b7f5af2
2025-10-09T21:43:39.470796Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 18 added 1 blobs, size 769908 count 36 last offset 176, current partition end offset: 300
2025-10-09T21:43:39.470800Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 18. Send blob request.
2025-10-09T21:43:39.470814Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 176 partno 0 count 36 parts_count 0 source 1 size 769908 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.470829Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 18. All 1 blobs are from cache.
2025-10-09T21:43:39.470846Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.470909Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 176 partno 0 count 36 parts 0 suffix '63'
2025-10-09T21:43:39.471115Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 176 totakecount 36 count 25 size 519473 from pos 0 cbcount 25
2025-10-09T21:43:39.471271Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 201 totakecount 36 count 11 size 250389 from pos 0 cbcount 11
2025-10-09T21:43:39.471351Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 176
2025-10-09T21:43:39.471704Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 300 Result { Offset: 176 Data: "... 19445 bytes ..." SourceId: "\000src_id" SeqNo: 177 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219384 UncompressedSize: 19371 PartitionKey: "" ExplicitHash: "" } Result { Offset: 177 Data: "... 19555 bytes ..." SourceId: "\000src_id" SeqNo: 178 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219384 UncompressedSize: 19481 PartitionKey: "" ExplicitHash: "" } Result { Offset: 178 Data: "... 19665 bytes ..." SourceId: "\000src_id" SeqNo: 179 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219384 UncompressedSize: 19591 PartitionKey: "" ExplicitHash: "" } Result { Offset: 179 Data: "... 19775 bytes ..." SourceId: "\000src_id" SeqNo: 180 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219384 UncompressedSize: 19701 PartitionKey: "" ExplicitHash: "" } Result { Offset: 180 Data: "... 19885 bytes ..." SourceId: "\000src_id" SeqNo: 181 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219385 UncompressedSize: 19811 PartitionKey: "" ExplicitHash: "" } Result { Offset: 181 Data: "... 19995 bytes ..." SourceId: "\000src_id" SeqNo: 182 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219385 UncompressedSize: 19921 PartitionKey: "" ExplicitHash: "" } Result { Offset: 182 Data: "... 20105 bytes ..." SourceId: "\000src_id" SeqNo: 183 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219385 UncompressedSize: 20031 PartitionKey: "" ExplicitHash: "" } Result { Offset: 183 Data: "... 20215 bytes ..." SourceId: "\000src_id" SeqNo: 184 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219385 UncompressedSize: 20141 PartitionKey: "" ExplicitHash: "" } Result { Offset: 184 Data: "... 20325 bytes ..." SourceId: "\000src_id" SeqNo: 185 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219385 UncompressedSize: 20251 PartitionKey: "" ExplicitHash: "" } Result { Offset: 185 Data: "... 20435 bytes ..." SourceId: "\000src_id" SeqNo: 186 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219385 UncompressedSize: 20361 PartitionKey: "" ExplicitHash: "" } Result { Offset: 186 Data: "... 20545 bytes ..." SourceId: "\000src_id" SeqNo: 187 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219385 UncompressedSize: 20471 PartitionKey: "" ExplicitHash: "" } Result { Offset: 187 Data: "... 20655 bytes ..." SourceId: "\000src_id" SeqNo: 188 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219385 UncompressedSize: 20581 PartitionKey: "" ExplicitHash: "" } Result { Offset: 188 Data: "... 20765 bytes ..." SourceId: "\000src_id" SeqNo: 189 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219385 UncompressedSize: 20691 PartitionKey: "" ExplicitHash: "" } Result { Offset: 189 Data: "... 20875 bytes ..." SourceId: "\000src_id" SeqNo: 190 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219385 UncompressedSize: 20801 PartitionKey: "" ExplicitHash: "" } Result { Offset: 190 Data: "... 20985 bytes ..." SourceId: "\000src_id" SeqNo: 191 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219385 UncompressedSize: 20911 PartitionKey: "" ExplicitHash: "" } Result { Offset: 191 Data: "... 21095 bytes ..." SourceId: "\000src_id" SeqNo: 192 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219386 UncompressedSize: 21021 PartitionKey: "" ExplicitHash: "" } Result { Offset: 192 Data: "... 21205 bytes ..." SourceId: "\000src_id" SeqNo: 193 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219386 UncompressedSize: 21131 PartitionKey: "" ExplicitHash: "" } Result { Offset: 193 Data: "... 21315 bytes ..." SourceId: "\000src_id" SeqNo: 194 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219386 UncompressedSize: 21241 PartitionKey: "" ExplicitHash: "" } Result { Offset: 194 Data: "... 21425 bytes ..." SourceId: "\000src_id" SeqNo: 195 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219386 UncompressedSize: 21351 PartitionKey: "" ExplicitHash: "" } Result { Offset: 195 Data: "... 21535 bytes ..." SourceId: "\000src_id" SeqNo: 196 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219386 UncompressedSize: 21461 PartitionKey: "" ExplicitHash: "" } Result { Offset: 196 Data: "... 21645 bytes ..." SourceId: "\000src_id" SeqNo: 197 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219386 UncompressedSize: 21571 PartitionKey: "" ExplicitHash: "" } Result { Offset: 197 Data: "... 21755 bytes ..." SourceId: "\000src_id" SeqNo: 198 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219386 UncompressedSize: 21681 PartitionKey: "" ExplicitHash: "" } Result { Offset: 198 Data: "... 21865 bytes ..." SourceId: "\000src_id" SeqNo: 199 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219386 UncompressedSize: 21791 PartitionKey: "" ExplicitHash: "" } Result { Offset: 199 Data: "... 21975 bytes ..." SourceId: "\000src_id" SeqNo: 200 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219386 UncompressedSize: 21901 PartitionKey: "" ExplicitHash: "" } Result { Offset: 200 Data: "... 22085 bytes ..." SourceId: "\000src_id" SeqNo: 201 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219386 UncompressedSize: 22011 PartitionKey: "" ExplicitHash: "" } Result { Offset: 201 Data: "... 22195 bytes ..." SourceId: "\000src_id" SeqNo: 202 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219386 UncompressedSize: 22121 PartitionKey: "" ExplicitHash: "" } Result { Offset: 202 Data: "... 22305 bytes ..." SourceId: "\000src_id" SeqNo: 203 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219386 UncompressedSize: 22231 PartitionKey: "" ExplicitHash: "" } Result { Offset: 203 Data: "... 22415 bytes ..." SourceId: "\000src_id" SeqNo: 204 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219387 UncompressedSize: 22341 PartitionKey: "" ExplicitHash: "" } Result { Offset: 204 Data: "... 22525 bytes ..." SourceId: "\000src_id" SeqNo: 205 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219387 UncompressedSize: 22451 PartitionKey: "" ExplicitHash: "" } Result { Offset: 205 Data: "... 22635 bytes ..." SourceId: "\000src_id" SeqNo: 206 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219387 UncompressedSize: 22561 PartitionKey: "" ExplicitHash: "" } Result { Offset: 206 Data: "... 22745 bytes ..." SourceId: "\000src_id" SeqNo: 207 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219387 UncompressedSize: 22671 PartitionKey: "" ExplicitHash: "" } Result { Offset: 207 Data: "... 22855 bytes ..." SourceId: "\000src_id" SeqNo: 208 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219387 UncompressedSize: 22781 PartitionKey: "" ExplicitHash: "" } Result { Offset: 208 Data: "... 22965 bytes ..." SourceId: "\000src_id" SeqNo: 209 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219387 UncompressedSize: 22891 PartitionKey: "" ExplicitHash: "" } Result { Offset: 209 Data: "... 23075 bytes ..." SourceId: "\000src_id" SeqNo: 210 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219387 UncompressedSize: 23001 PartitionKey: "" ExplicitHash: "" } Result { Offset: 210 Data: "... 23185 bytes ..." SourceId: "\000src_id" SeqNo: 211 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219387 UncompressedSize: 23111 PartitionKey: "" ExplicitHash: "" } Result { Offset: 211 Data: "... 23295 bytes ..." SourceId: "\000src_id" SeqNo: 212 WriteTimestampMS: 1760046219414 CreateTimestampMS: 1760046219387 UncompressedSize: 23221 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 2399044 RealReadOffset: 211 WaitQuotaTimeMs: 0 EndOffset: 300 StartOffset: 0 } Cookie: 176 }
2025-10-09T21:43:39.471988Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 212 endOffset 300
2025-10-09T21:43:39.472006Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 300 ReadOffset 212 ReadGuid c01639ca-5040ec8-f3b3fd14-e453eba4 has messages 1
2025-10-09T21:43:39.472240Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 212, endOffset# 300, WTime# 1760046219414, sizeLag# 2399044
2025-10-09T21:43:39.472244Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 0
2025-10-09T21:43:39.472265Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# c01639ca-5040ec8-f3b3fd14-e453eba4, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 768229
2025-10-09T21:43:39.472276Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# c01639ca-5040ec8-f3b3fd14-e453eba4
2025-10-09T21:43:39.473000Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 1
2025-10-09T21:43:39.479082Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.479090Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.479093Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.503354Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 768229 } }
2025-10-09T21:43:39.503433Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# 82ecdd78-5a22620f-30971096-26cb7ef3
2025-10-09T21:43:39.503461Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# ade94575-acc27989-9d0f941-de335062, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 105, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.503491Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 105 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 212 EndOffset 300 ClientCommitOffset 0 committedOffset 0 Guid ade94575-acc27989-9d0f941-de335062
2025-10-09T21:43:39.503694Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.503703Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.503749Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 19 Topic 'rt3.dc1--test-topic' partition 0 user user offset 212 count 105 size 524288 endOffset 300 max time lag 0ms effective offset 212
2025-10-09T21:43:39.503774Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 19 added 1 blobs, size 618592 count 25 last offset 212, current partition end offset: 300
2025-10-09T21:43:39.503781Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 19. Send blob request.
2025-10-09T21:43:39.503795Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 212 partno 0 count 25 parts_count 0 source 1 size 618592 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.503805Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 19. All 1 blobs are from cache.
2025-10-09T21:43:39.503820Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.504178Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 212 totakecount 25 count 21 size 514909 from pos 0 cbcount 21
2025-10-09T21:43:39.504344Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 233 totakecount 25 count 4 size 103637 from pos 0 cbcount 4
2025-10-09T21:43:39.504403Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 212
2025-10-09T21:43:39.504551Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 212 partno 0 count 25 parts 0 suffix '63'
2025-10-09T21:43:39.504683Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 300 Result { Offset: 212 Data: "... 23405 bytes ..." SourceId: "\000src_id" SeqNo: 213 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219388 UncompressedSize: 23331 PartitionKey: "" ExplicitHash: "" } Result { Offset: 213 Data: "... 23515 bytes ..." SourceId: "\000src_id" SeqNo: 214 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219388 UncompressedSize: 23441 PartitionKey: "" ExplicitHash: "" } Result { Offset: 214 Data: "... 23625 bytes ..." SourceId: "\000src_id" SeqNo: 215 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219388 UncompressedSize: 23551 PartitionKey: "" ExplicitHash: "" } Result { Offset: 215 Data: "... 23735 bytes ..." SourceId: "\000src_id" SeqNo: 216 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219388 UncompressedSize: 23661 PartitionKey: "" ExplicitHash: "" } Result { Offset: 216 Data: "... 23845 bytes ..." SourceId: "\000src_id" SeqNo: 217 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219388 UncompressedSize: 23771 PartitionKey: "" ExplicitHash: "" } Result { Offset: 217 Data: "... 23955 bytes ..." SourceId: "\000src_id" SeqNo: 218 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219388 UncompressedSize: 23881 PartitionKey: "" ExplicitHash: "" } Result { Offset: 218 Data: "... 24065 bytes ..." SourceId: "\000src_id" SeqNo: 219 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219388 UncompressedSize: 23991 PartitionKey: "" ExplicitHash: "" } Result { Offset: 219 Data: "... 24175 bytes ..." SourceId: "\000src_id" SeqNo: 220 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219388 UncompressedSize: 24101 PartitionKey: "" ExplicitHash: "" } Result { Offset: 220 Data: "... 24285 bytes ..." SourceId: "\000src_id" SeqNo: 221 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219388 UncompressedSize: 24211 PartitionKey: "" ExplicitHash: "" } Result { Offset: 221 Data: "... 24395 bytes ..." SourceId: "\000src_id" SeqNo: 222 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219388 UncompressedSize: 24321 PartitionKey: "" ExplicitHash: "" } Result { Offset: 222 Data: "... 24505 bytes ..." SourceId: "\000src_id" SeqNo: 223 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219388 UncompressedSize: 24431 PartitionKey: "" ExplicitHash: "" } Result { Offset: 223 Data: "... 24615 bytes ..." SourceId: "\000src_id" SeqNo: 224 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219388 UncompressedSize: 24541 PartitionKey: "" ExplicitHash: "" } Result { Offset: 224 Data: "... 24725 bytes ..." SourceId: "\000src_id" SeqNo: 225 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219388 UncompressedSize: 24651 PartitionKey: "" ExplicitHash: "" } Result { Offset: 225 Data: "... 24835 bytes ..." SourceId: "\000src_id" SeqNo: 226 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219388 UncompressedSize: 24761 PartitionKey: "" ExplicitHash: "" } Result { Offset: 226 Data: "... 24945 bytes ..." SourceId: "\000src_id" SeqNo: 227 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219388 UncompressedSize: 24871 PartitionKey: "" ExplicitHash: "" } Result { Offset: 227 Data: "... 25055 bytes ..." SourceId: "\000src_id" SeqNo: 228 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219389 UncompressedSize: 24981 PartitionKey: "" ExplicitHash: "" } Result { Offset: 228 Data: "... 25165 bytes ..." SourceId: "\000src_id" SeqNo: 229 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219389 UncompressedSize: 25091 PartitionKey: "" ExplicitHash: "" } Result { Offset: 229 Data: "... 25275 bytes ..." SourceId: "\000src_id" SeqNo: 230 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219389 UncompressedSize: 25201 PartitionKey: "" ExplicitHash: "" } Result { Offset: 230 Data: "... 25385 bytes ..." SourceId: "\000src_id" SeqNo: 231 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219389 UncompressedSize: 25311 PartitionKey: "" ExplicitHash: "" } Result { Offset: 231 Data: "... 25495 bytes ..." SourceId: "\000src_id" SeqNo: 232 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219389 UncompressedSize: 25421 PartitionKey: "" ExplicitHash: "" } Result { Offset: 232 Data: "... 25605 bytes ..." SourceId: "\000src_id" SeqNo: 233 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219389 UncompressedSize: 25531 PartitionKey: "" ExplicitHash: "" } Result { Offset: 233 Data: "... 25715 bytes ..." SourceId: "\000src_id" SeqNo: 234 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219390 UncompressedSize: 25641 PartitionKey: "" ExplicitHash: "" } Result { Offset: 234 Data: "... 25825 bytes ..." SourceId: "\000src_id" SeqNo: 235 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219390 UncompressedSize: 25751 PartitionKey: "" ExplicitHash: "" } Result { Offset: 235 Data: "... 25935 bytes ..." SourceId: "\000src_id" SeqNo: 236 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219390 UncompressedSize: 25861 PartitionKey: "" ExplicitHash: "" } Result { Offset: 236 Data: "... 26045 bytes ..." SourceId: "\000src_id" SeqNo: 237 WriteTimestampMS: 1760046219420 CreateTimestampMS: 1760046219390 UncompressedSize: 25971 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 1629477 RealReadOffset: 236 WaitQuotaTimeMs: 0 EndOffset: 300 StartOffset: 0 } Cookie: 212 }
2025-10-09T21:43:39.504872Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 237 endOffset 300
2025-10-09T21:43:39.504889Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 300 ReadOffset 237 ReadGuid ade94575-acc27989-9d0f941-de335062 has messages 1
2025-10-09T21:43:39.504905Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 237, endOffset# 300, WTime# 1760046219420, sizeLag# 1629477
2025-10-09T21:43:39.504919Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 0
2025-10-09T21:43:39.504936Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# ade94575-acc27989-9d0f941-de335062, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 617408
2025-10-09T21:43:39.504947Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# ade94575-acc27989-9d0f941-de335062
2025-10-09T21:43:39.505416Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 1
2025-10-09T21:43:39.505795Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 617408, now ReadSizeBudget = 0, ReadSizeServerDelta = -93120
2025-10-09T21:43:39.530490Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (202-211)
2025-10-09T21:43:39.530511Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.530559Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 26} (202-202)
2025-10-09T21:43:39.530575Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 27} (203-203)
2025-10-09T21:43:39.530577Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (212-233)
2025-10-09T21:43:39.530599Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 617408 to budget
2025-10-09T21:43:39.530606Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 617408, ReadSizeServerDelta = -93120
2025-10-09T21:43:39.530612Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 28} (204-204)
2025-10-09T21:43:39.530619Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 29} (205-205)
2025-10-09T21:43:39.530677Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 30} (206-206)
2025-10-09T21:43:39.530687Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 31} (207-207)
2025-10-09T21:43:39.530693Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 32} (208-208)
2025-10-09T21:43:39.530701Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 33} (209-209)
2025-10-09T21:43:39.530736Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.530778Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 34} (210-210)
2025-10-09T21:43:39.530787Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 35} (211-211)
2025-10-09T21:43:39.530794Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (212-212)
2025-10-09T21:43:39.530802Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (213-213)
2025-10-09T21:43:39.530809Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 2} (214-214)
2025-10-09T21:43:39.530816Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 3} (215-215)
2025-10-09T21:43:39.530826Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 4} (216-216)
2025-10-09T21:43:39.530833Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 5} (217-217)
2025-10-09T21:43:39.530899Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 617408 } }
2025-10-09T21:43:39.530986Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 6} (218-218)
2025-10-09T21:43:39.530997Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 7} (219-219)
2025-10-09T21:43:39.531004Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 8} (220-220)
2025-10-09T21:43:39.531011Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 9} (221-221)
2025-10-09T21:43:39.531019Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 10} (222-222)
2025-10-09T21:43:39.531026Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 11} (223-223)
2025-10-09T21:43:39.531033Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 12} (224-224)
2025-10-09T21:43:39.531041Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 13} (225-225)
2025-10-09T21:43:39.531048Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 14} (226-226)
2025-10-09T21:43:39.531056Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 15} (227-227)
2025-10-09T21:43:39.531063Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 16} (228-228)
2025-10-09T21:43:39.531070Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 17} (229-229)
2025-10-09T21:43:39.531078Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 18} (230-230)
2025-10-09T21:43:39.531086Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 19} (231-231)
2025-10-09T21:43:39.531093Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 20} (232-232)
2025-10-09T21:43:39.531101Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 21} (233-233)
2025-10-09T21:43:39.531002Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# 63e363f4-44f3eeca-5c7ec4a-575e1ebd
2025-10-09T21:43:39.531036Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# 4ba1c1cf-be86e41c-e0720858-97cace67, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 75, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.531065Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 75 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 237 EndOffset 300 ClientCommitOffset 0 committedOffset 0 Guid 4ba1c1cf-be86e41c-e0720858-97cace67
2025-10-09T21:43:39.531188Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.531218Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.531288Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 20 Topic 'rt3.dc1--test-topic' partition 0 user user offset 237 count 75 size 524288 endOffset 300 max time lag 0ms effective offset 237
2025-10-09T21:43:39.531327Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 20 added 1 blobs, size 1011468 count 36 last offset 237, current partition end offset: 300
2025-10-09T21:43:39.531336Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 20. Send blob request.
2025-10-09T21:43:39.531351Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 237 partno 0 count 36 parts_count 0 source 1 size 1011468 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.531367Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 20. All 1 blobs are from cache.
2025-10-09T21:43:39.531390Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.531401Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 237 partno 0 count 36 parts 0 suffix '63'
2025-10-09T21:43:39.531787Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 237 totakecount 36 count 19 size 516037 from pos 0 cbcount 19
2025-10-09T21:43:39.531988Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 256 totakecount 36 count 17 size 495385 from pos 0 cbcount 17
2025-10-09T21:43:39.532069Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 237
2025-10-09T21:43:39.532430Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 300 Result { Offset: 237 Data: "... 26155 bytes ..." SourceId: "\000src_id" SeqNo: 238 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219390 UncompressedSize: 26081 PartitionKey: "" ExplicitHash: "" } Result { Offset: 238 Data: "... 26265 bytes ..." SourceId: "\000src_id" SeqNo: 239 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219390 UncompressedSize: 26191 PartitionKey: "" ExplicitHash: "" } Result { Offset: 239 Data: "... 26375 bytes ..." SourceId: "\000src_id" SeqNo: 240 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219390 UncompressedSize: 26301 PartitionKey: "" ExplicitHash: "" } Result { Offset: 240 Data: "... 26485 bytes ..." SourceId: "\000src_id" SeqNo: 241 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219390 UncompressedSize: 26411 PartitionKey: "" ExplicitHash: "" } Result { Offset: 241 Data: "... 26595 bytes ..." SourceId: "\000src_id" SeqNo: 242 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219391 UncompressedSize: 26521 PartitionKey: "" ExplicitHash: "" } Result { Offset: 242 Data: "... 26705 bytes ..." SourceId: "\000src_id" SeqNo: 243 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219392 UncompressedSize: 26631 PartitionKey: "" ExplicitHash: "" } Result { Offset: 243 Data: "... 26815 bytes ..." SourceId: "\000src_id" SeqNo: 244 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219392 UncompressedSize: 26741 PartitionKey: "" ExplicitHash: "" } Result { Offset: 244 Data: "... 26925 bytes ..." SourceId: "\000src_id" SeqNo: 245 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219392 UncompressedSize: 26851 PartitionKey: "" ExplicitHash: "" } Result { Offset: 245 Data: "... 27035 bytes ..." SourceId: "\000src_id" SeqNo: 246 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219392 UncompressedSize: 26961 PartitionKey: "" ExplicitHash: "" } Result { Offset: 246 Data: "... 27145 bytes ..." SourceId: "\000src_id" SeqNo: 247 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219392 UncompressedSize: 27071 PartitionKey: "" ExplicitHash: "" } Result { Offset: 247 Data: "... 27255 bytes ..." SourceId: "\000src_id" SeqNo: 248 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219393 UncompressedSize: 27181 PartitionKey: "" ExplicitHash: "" } Result { Offset: 248 Data: "... 27365 bytes ..." SourceId: "\000src_id" SeqNo: 249 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219393 UncompressedSize: 27291 PartitionKey: "" ExplicitHash: "" } Result { Offset: 249 Data: "... 27475 bytes ..." SourceId: "\000src_id" SeqNo: 250 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219393 UncompressedSize: 27401 PartitionKey: "" ExplicitHash: "" } Result { Offset: 250 Data: "... 27585 bytes ..." SourceId: "\000src_id" SeqNo: 251 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219393 UncompressedSize: 27511 PartitionKey: "" ExplicitHash: "" } Result { Offset: 251 Data: "... 27695 bytes ..." SourceId: "\000src_id" SeqNo: 252 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219393 UncompressedSize: 27621 PartitionKey: "" ExplicitHash: "" } Result { Offset: 252 Data: "... 27805 bytes ..." SourceId: "\000src_id" SeqNo: 253 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219393 UncompressedSize: 27731 PartitionKey: "" ExplicitHash: "" } Result { Offset: 253 Data: "... 27915 bytes ..." SourceId: "\000src_id" SeqNo: 254 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219393 UncompressedSize: 27841 PartitionKey: "" ExplicitHash: "" } Result { Offset: 254 Data: "... 28025 bytes ..." SourceId: "\000src_id" SeqNo: 255 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219393 UncompressedSize: 27951 PartitionKey: "" ExplicitHash: "" } Result { Offset: 255 Data: "... 28135 bytes ..." SourceId: "\000src_id" SeqNo: 256 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219393 UncompressedSize: 28061 PartitionKey: "" ExplicitHash: "" } Result { Offset: 256 Data: "... 28245 bytes ..." SourceId: "\000src_id" SeqNo: 257 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219394 UncompressedSize: 28171 PartitionKey: "" ExplicitHash: "" } Result { Offset: 257 Data: "... 28355 bytes ..." SourceId: "\000src_id" SeqNo: 258 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219394 UncompressedSize: 28281 PartitionKey: "" ExplicitHash: "" } Result { Offset: 258 Data: "... 28465 bytes ..." SourceId: "\000src_id" SeqNo: 259 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219394 UncompressedSize: 28391 PartitionKey: "" ExplicitHash: "" } Result { Offset: 259 Data: "... 28575 bytes ..." SourceId: "\000src_id" SeqNo: 260 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219394 UncompressedSize: 28501 PartitionKey: "" ExplicitHash: "" } Result { Offset: 260 Data: "... 28685 bytes ..." SourceId: "\000src_id" SeqNo: 261 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219394 UncompressedSize: 28611 PartitionKey: "" ExplicitHash: "" } Result { Offset: 261 Data: "... 28795 bytes ..." SourceId: "\000src_id" SeqNo: 262 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219395 UncompressedSize: 28721 PartitionKey: "" ExplicitHash: "" } Result { Offset: 262 Data: "... 28905 bytes ..." SourceId: "\000src_id" SeqNo: 263 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219395 UncompressedSize: 28831 PartitionKey: "" ExplicitHash: "" } Result { Offset: 263 Data: "... 29015 bytes ..." SourceId: "\000src_id" SeqNo: 264 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219395 UncompressedSize: 28941 PartitionKey: "" ExplicitHash: "" } Result { Offset: 264 Data: "... 29125 bytes ..." SourceId: "\000src_id" SeqNo: 265 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219395 UncompressedSize: 29051 PartitionKey: "" ExplicitHash: "" } Result { Offset: 265 Data: "... 29235 bytes ..." SourceId: "\000src_id" SeqNo: 266 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219395 UncompressedSize: 29161 PartitionKey: "" ExplicitHash: "" } Result { Offset: 266 Data: "... 29345 bytes ..." SourceId: "\000src_id" SeqNo: 267 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219395 UncompressedSize: 29271 PartitionKey: "" ExplicitHash: "" } Result { Offset: 267 Data: "... 29455 bytes ..." SourceId: "\000src_id" SeqNo: 268 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219395 UncompressedSize: 29381 PartitionKey: "" ExplicitHash: "" } Result { Offset: 268 Data: "... 29565 bytes ..." SourceId: "\000src_id" SeqNo: 269 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219395 UncompressedSize: 29491 PartitionKey: "" ExplicitHash: "" } Result { Offset: 269 Data: "... 29675 bytes ..." SourceId: "\000src_id" SeqNo: 270 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219395 UncompressedSize: 29601 PartitionKey: "" ExplicitHash: "" } Result { Offset: 270 Data: "... 29785 bytes ..." SourceId: "\000src_id" SeqNo: 271 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219395 UncompressedSize: 29711 PartitionKey: "" ExplicitHash: "" } Result { Offset: 271 Data: "... 29895 bytes ..." SourceId: "\000src_id" SeqNo: 272 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219395 UncompressedSize: 29821 PartitionKey: "" ExplicitHash: "" } Result { Offset: 272 Data: "... 30005 bytes ..." SourceId: "\000src_id" SeqNo: 273 WriteTimestampMS: 1760046219424 CreateTimestampMS: 1760046219395 UncompressedSize: 29931 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 1010544 RealReadOffset: 272 WaitQuotaTimeMs: 0 EndOffset: 300 StartOffset: 0 } Cookie: 237 }
>>> event from dataHandler: 2025-10-09T21:43:39.532723Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:969: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) ready for read with readOffset 273 endOffset 300
2025-10-09T21:43:39.532748Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 300 ReadOffset 273 ReadGuid 4ba1c1cf-be86e41c-e0720858-97cace67 has messages 1
2025-10-09T21:43:39.532779Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2332: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 partition ready for read: partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), readOffset# 273, endOffset# 300, WTime# 1760046219424, sizeLag# 1010544
2025-10-09T21:43:39.532792Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2343: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1TEvPartitionReady. Aval parts: 0
DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..22231 bytes.. Information: { Offset: 202 ProducerId: "src_id" SeqNo: 203 CreateTime: 2025-10-09T21:43:39.386000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..22341 bytes.. Information: { Offset: 203 ProducerId: "src_id" SeqNo: 204 CreateTime: 2025-10-09T21:43:39.387000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..22451 bytes.. Information: { Offset: 204 ProducerId: "src_id" SeqNo: 205 CreateTime: 2025-10-09T21:43:39.387000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..22561 bytes.. Information: { Offset: 205 ProducerId: "src_id" SeqNo: 206 CreateTime: 2025-10-09T21:43:39.387000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..22671 bytes.. Information: { Offset: 206 ProducerId: "src_id" SeqNo: 207 CreateTime: 2025-10-09T21:43:39.387000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..22781 bytes.. Information: { Offset: 207 ProducerId: "src_id" SeqNo: 208 CreateTime: 2025-10-09T21:43:39.387000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..22891 bytes.. Information: { Offset: 208 ProducerId: "src_id" SeqNo: 209 CreateTime: 2025-10-09T21:43:39.387000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..23001 bytes.. Information: { Offset: 209 ProducerId: "src_id" SeqNo: 210 CreateTime: 2025-10-09T21:43:39.387000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..23111 bytes.. Information: { Offset: 210 ProducerId: "src_id" SeqNo: 211 CreateTime: 2025-10-09T21:43:39.387000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..23221 bytes.. Information: { Offset: 211 ProducerId: "src_id" SeqNo: 212 CreateTime: 2025-10-09T21:43:39.387000Z WriteTime: 2025-10-09T21:43:39.414000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "logtype": "unknown", "server": "ipv6:[::1]:42738", "_ip": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..23331 bytes.. Information: { Offset: 212 ProducerId: "src_id" SeqNo: 213 CreateTime: 2025-10-09T21:43:39.388000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..23441 bytes.. Information: { Offset: 213 ProducerId: "src_id" SeqNo: 214 CreateTime: 2025-10-09T21:43:39.388000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..23551 bytes.. Information: { Offset: 214 ProducerId: "src_id" SeqNo: 215 CreateTime: 2025-10-09T21:43:39.388000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..23661 bytes.. Information: { Offset: 215 ProducerId: "src_id" SeqNo: 216 CreateTime: 2025-10-09T21:43:39.388000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..23771 bytes.. Information: { Offset: 216 ProducerId: "src_id" SeqNo: 217 CreateTime: 2025-10-09T21:43:39.388000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..23881 bytes.. Information: { Offset: 217 ProducerId: "src_id" SeqNo: 218 CreateTime: 2025-10-09T21:43:39.388000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..23991 bytes.. Information: { Offset: 218 ProducerId: "src_id" SeqNo: 219 CreateTime: 2025-10-09T21:43:39.388000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..24101 bytes.. Information: { Offset: 219 ProducerId: "src_id" SeqNo: 220 CreateTime: 2025-10-09T21:43:39.388000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..24211 bytes.. Information: { Offset: 220 ProducerId: "src_id" SeqNo: 221 CreateTime: 2025-10-09T21:43:39.388000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..24321 bytes.. Information: { Offset: 221 ProducerId: "src_id" SeqNo: 222 CreateTime: 2025-10-09T21:43:39.388000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..24431 bytes.. Information: { Offset: 222 ProducerId: "src_id" SeqNo: 223 CreateTime: 2025-10-09T21:43:39.388000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..24541 bytes.. Information: { Offset: 223 ProducerId: "src_id" SeqNo: 224 CreateTime: 2025-10-09T21:43:39.388000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..24651 bytes.. Information: { Offset: 224 ProducerId: "src_id" SeqNo: 225 CreateTime: 2025-10-09T21:43:39.388000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..24761 bytes.. Information: { Offset: 225 ProducerId: "src_id" SeqNo: 226 CreateTime: 2025-10-09T21:43:39.388000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..24871 bytes.. Information: { Offset: 226 ProducerId: "src_id" SeqNo: 227 CreateTime: 2025-10-09T21:43:39.388000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..24981 bytes.. Information: { Offset: 227 ProducerId: "src_id" SeqNo: 228 CreateTime: 2025-10-09T21:43:39.389000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..25091 bytes.. Information: { Offset: 228 ProducerId: "src_id" SeqNo: 229 CreateTime: 2025-10-09T21:43:39.389000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..25201 bytes.. Information: { Offset: 229 ProducerId: "src_id" SeqNo: 230 CreateTime: 2025-10-09T21:43:39.389000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..25311 bytes.. Information: { Offset: 230 ProducerId: "src_id" SeqNo: 231 CreateTime: 2025-10-09T21:43:39.389000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..25421 bytes.. Information: { Offset: 231 ProducerId: "src_id" SeqNo: 232 CreateTime: 2025-10-09T21:43:39.389000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..25531 bytes.. Information: { Offset: 232 ProducerId: "src_id" SeqNo: 233 CreateTime: 2025-10-09T21:43:39.389000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..25641 bytes.. Information: { Offset: 233 ProducerId: "src_id" SeqNo: 234 CreateTime: 2025-10-09T21:43:39.390000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.532811Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# 4ba1c1cf-be86e41c-e0720858-97cace67, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 1009789
2025-10-09T21:43:39.532829Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# 4ba1c1cf-be86e41c-e0720858-97cace67
2025-10-09T21:43:39.533058Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 22, size 538692 bytes
2025-10-09T21:43:39.533089Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 10, size 227260 bytes
2025-10-09T21:43:39.533096Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.533265Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (234-236)
2025-10-09T21:43:39.533278Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.533294Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 22} (234-234)
2025-10-09T21:43:39.533309Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 23} (235-235)
2025-10-09T21:43:39.533321Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 24} (236-236)
>>> event from dataHandler: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..25751 bytes.. Information: { Offset: 234 ProducerId: "src_id" SeqNo: 235 CreateTime: 2025-10-09T21:43:39.390000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..25861 bytes.. Information: { Offset: 235 ProducerId: "src_id" SeqNo: 236 CreateTime: 2025-10-09T21:43:39.390000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..25971 bytes.. Information: { Offset: 236 ProducerId: "src_id" SeqNo: 237 CreateTime: 2025-10-09T21:43:39.390000Z WriteTime: 2025-10-09T21:43:39.420000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.533396Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 3, size 77583 bytes
2025-10-09T21:43:39.533401Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.533688Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 1
2025-10-09T21:43:39.534235Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 1009789, now ReadSizeBudget = 0, ReadSizeServerDelta = -485501
2025-10-09T21:43:39.534331Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (237-256)
2025-10-09T21:43:39.534352Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 1009789 to budget
2025-10-09T21:43:39.534358Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 1009789, ReadSizeServerDelta = -485501
2025-10-09T21:43:39.534469Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.534486Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (237-237)
2025-10-09T21:43:39.534497Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (238-238)
2025-10-09T21:43:39.534511Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 2} (239-239)
2025-10-09T21:43:39.534516Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 3} (240-240)
2025-10-09T21:43:39.534544Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 4} (241-241)
2025-10-09T21:43:39.534550Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 5} (242-242)
2025-10-09T21:43:39.534555Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 6} (243-243)
2025-10-09T21:43:39.534561Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 7} (244-244)
2025-10-09T21:43:39.534616Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 8} (245-245)
2025-10-09T21:43:39.534622Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 9} (246-246)
2025-10-09T21:43:39.534628Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 10} (247-247)
2025-10-09T21:43:39.534634Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 11} (248-248)
2025-10-09T21:43:39.534639Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 12} (249-249)
2025-10-09T21:43:39.534645Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 13} (250-250)
2025-10-09T21:43:39.534652Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 14} (251-251)
2025-10-09T21:43:39.534657Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 15} (252-252)
2025-10-09T21:43:39.534751Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 16} (253-253)
2025-10-09T21:43:39.534759Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 17} (254-254)
2025-10-09T21:43:39.534765Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 18} (255-255)
2025-10-09T21:43:39.534771Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 19} (256-256)
2025-10-09T21:43:39.534996Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 1009789 } }
>>> event from dataHandler: 2025-10-09T21:43:39.535059Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# 6954a069-31237d5-62d418c9-71de3db0
2025-10-09T21:43:39.535080Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2266: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 performing read request: guid# 2c3b4c60-20ad2d7a-5a959db0-c565be8, from# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), count# 32, size# 524288, partitionsAsked# 1, maxTimeLag# 0ms
2025-10-09T21:43:39.535105Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1395: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 READ FROM TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1)maxCount 32 maxSize 524288 maxTimeLagMs 0 readTimestampMs 0 readOffset 273 EndOffset 300 ClientCommitOffset 0 committedOffset 0 Guid 2c3b4c60-20ad2d7a-5a959db0-c565be8
DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..26081 bytes.. Information: { Offset: 237 ProducerId: "src_id" SeqNo: 238 CreateTime: 2025-10-09T21:43:39.390000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..26191 bytes.. Information: { Offset: 238 ProducerId: "src_id" SeqNo: 239 CreateTime: 2025-10-09T21:43:39.390000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..26301 bytes.. Information: { Offset: 239 ProducerId: "src_id" SeqNo: 240 CreateTime: 2025-10-09T21:43:39.390000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..26411 bytes.. Information: { Offset: 240 ProducerId: "src_id" SeqNo: 241 CreateTime: 2025-10-09T21:43:39.390000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..26521 bytes.. Information: { Offset: 241 ProducerId: "src_id" SeqNo: 242 CreateTime: 2025-10-09T21:43:39.391000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..26631 bytes.. Information: { Offset: 242 ProducerId: "src_id" SeqNo: 243 CreateTime: 2025-10-09T21:43:39.392000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..26741 bytes.. Information: { Offset: 243 ProducerId: "src_id" SeqNo: 244 CreateTime: 2025-10-09T21:43:39.392000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..26851 bytes.. Information: { Offset: 244 ProducerId: "src_id" SeqNo: 245 CreateTime: 2025-10-09T21:43:39.392000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..26961 bytes.. Information: { Offset: 245 ProducerId: "src_id" SeqNo: 246 CreateTime: 2025-10-09T21:43:39.392000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..27071 bytes.. Information: { Offset: 246 ProducerId: "src_id" SeqNo: 247 CreateTime: 2025-10-09T21:43:39.392000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..27181 bytes.. Information: { Offset: 247 ProducerId: "src_id" SeqNo: 248 CreateTime: 2025-10-09T21:43:39.393000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..27291 bytes.. Information: { Offset: 248 ProducerId: "src_id" SeqNo: 249 CreateTime: 2025-10-09T21:43:39.393000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..27401 bytes.. Information: { Offset: 249 ProducerId: "src_id" SeqNo: 250 CreateTime: 2025-10-09T21:43:39.393000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..27511 bytes.. Information: { Offset: 250 ProducerId: "src_id" SeqNo: 251 CreateTime: 2025-10-09T21:43:39.393000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..27621 bytes.. Information: { Offset: 251 ProducerId: "src_id" SeqNo: 252 CreateTime: 2025-10-09T21:43:39.393000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..27731 bytes.. Information: { Offset: 252 ProducerId: "src_id" SeqNo: 253 CreateTime: 2025-10-09T21:43:39.393000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..27841 bytes.. Information: { Offset: 253 ProducerId: "src_id" SeqNo: 254 CreateTime: 2025-10-09T21:43:39.393000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..27951 bytes.. Information: { Offset: 254 ProducerId: "src_id" SeqNo: 255 CreateTime: 2025-10-09T21:43:39.393000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..28061 bytes.. Information: { Offset: 255 ProducerId: "src_id" SeqNo: 256 CreateTime: 2025-10-09T21:43:39.393000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..28171 bytes.. Information: { Offset: 256 ProducerId: "src_id" SeqNo: 257 CreateTime: 2025-10-09T21:43:39.394000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.535193Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:39.535205Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:39.535242Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 21 Topic 'rt3.dc1--test-topic' partition 0 user user offset 273 count 32 size 524288 endOffset 300 max time lag 0ms effective offset 273
2025-10-09T21:43:39.535268Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 20, size 542520 bytes
2025-10-09T21:43:39.535268Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 21 added 1 blobs, size 852204 count 27 last offset 273, current partition end offset: 300
2025-10-09T21:43:39.535276Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 21. Send blob request.
2025-10-09T21:43:39.535289Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 273 partno 0 count 27 parts_count 0 source 1 size 852204 accessed 0 times before, last time 2025-10-09T21:43:39.000000Z
2025-10-09T21:43:39.535305Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 21. All 1 blobs are from cache.
2025-10-09T21:43:39.535321Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:39.535334Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 273 partno 0 count 27 parts 0 suffix '63'
2025-10-09T21:43:39.535952Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 273 totakecount 27 count 17 size 527175 from pos 0 cbcount 17
2025-10-09T21:43:39.536140Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 290 totakecount 27 count 10 size 324983 from pos 0 cbcount 10
2025-10-09T21:43:39.536206Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 273
2025-10-09T21:43:39.536318Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (257-272)
2025-10-09T21:43:39.536329Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.536341Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 20} (257-257)
2025-10-09T21:43:39.536353Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 21} (258-258)
2025-10-09T21:43:39.536365Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 22} (259-259)
2025-10-09T21:43:39.536372Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 23} (260-260)
2025-10-09T21:43:39.536387Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 24} (261-261)
2025-10-09T21:43:39.536393Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 25} (262-262)
2025-10-09T21:43:39.536408Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 26} (263-263)
2025-10-09T21:43:39.536414Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 27} (264-264)
2025-10-09T21:43:39.536442Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 28} (265-265)
2025-10-09T21:43:39.536452Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 29} (266-266)
2025-10-09T21:43:39.536460Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 30} (267-267)
2025-10-09T21:43:39.536469Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 31} (268-268)
2025-10-09T21:43:39.536478Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 32} (269-269)
2025-10-09T21:43:39.536487Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 33} (270-270)
2025-10-09T21:43:39.536507Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 34} (271-271)
2025-10-09T21:43:39.536487Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:663: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) initDone 1 event { CmdReadResult { MaxOffset: 300 Result { Offset: 273 Data: "... 30115 bytes ..." SourceId: "\000src_id" SeqNo: 274 WriteTimestampMS: 1760046219430 CreateTimestampMS: 1760046219395 UncompressedSize: 30041 PartitionKey: "" ExplicitHash: "" } Result { Offset: 274 Data: "... 30225 bytes ..." SourceId: "\000src_id" SeqNo: 275 WriteTimestampMS: 1760046219430 CreateTimestampMS: 1760046219396 UncompressedSize: 30151 PartitionKey: "" ExplicitHash: "" } Result { Offset: 275 Data: "... 30335 bytes ..." SourceId: "\000src_id" SeqNo: 276 WriteTimestampMS: 1760046219430 CreateTimestampMS: 1760046219396 UncompressedSize: 30261 PartitionKey: "" ExplicitHash: "" } Result { Offset: 276 Data: "... 30445 bytes ..." SourceId: "\000src_id" SeqNo: 277 WriteTimestampMS: 1760046219430 CreateTimestampMS: 1760046219396 UncompressedSize: 30371 PartitionKey: "" ExplicitHash: "" } Result { Offset: 277 Data: "... 30555 bytes ..." SourceId: "\000src_id" SeqNo: 278 WriteTimestampMS: 1760046219430 CreateTimestampMS: 1760046219396 UncompressedSize: 30481 PartitionKey: "" ExplicitHash: "" } Result { Offset: 278 Data: "... 30665 bytes ..." SourceId: "\000src_id" SeqNo: 279 WriteTimestampMS: 1760046219430 CreateTimestampMS: 1760046219396 UncompressedSize: 30591 PartitionKey: "" ExplicitHash: "" } Result { Offset: 279 Data: "... 30775 bytes ..." SourceId: "\000src_id" SeqNo: 280 WriteTimestampMS: 1760046219430 CreateTimestampMS: 1760046219396 UncompressedSize: 30701 PartitionKey: "" ExplicitHash: "" } Result { Offset: 280 Data: "... 30885 bytes ..." SourceId: "\000src_id" SeqNo: 281 WriteTimestampMS: 1760046219430 CreateTimestampMS: 1760046219398 UncompressedSize: 30811 PartitionKey: "" ExplicitHash: "" } Result { Offset: 281 Data: "... 30995 bytes ..." SourceId: "\000src_id" SeqNo: 282 WriteTimestampMS: 1760046219430 CreateTimestampMS: 1760046219398 UncompressedSize: 30921 PartitionKey: "" ExplicitHash: "" } Result { Offset: 282 Data: "... 31105 bytes ..." SourceId: "\000src_id" SeqNo: 283 WriteTimestampMS: 1760046219430 CreateTimestampMS: 1760046219398 UncompressedSize: 31031 PartitionKey: "" ExplicitHash: "" } Result { Offset: 283 Data: "... 31215 bytes ..." SourceId: "\000src_id" SeqNo: 284 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219398 UncompressedSize: 31141 PartitionKey: "" ExplicitHash: "" } Result { Offset: 284 Data: "... 31325 bytes ..." SourceId: "\000src_id" SeqNo: 285 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219398 UncompressedSize: 31251 PartitionKey: "" ExplicitHash: "" } Result { Offset: 285 Data: "... 31435 bytes ..." SourceId: "\000src_id" SeqNo: 286 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219398 UncompressedSize: 31361 PartitionKey: "" ExplicitHash: "" } Result { Offset: 286 Data: "... 31545 bytes ..." SourceId: "\000src_id" SeqNo: 287 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219398 UncompressedSize: 31471 PartitionKey: "" ExplicitHash: "" } Result { Offset: 287 Data: "... 31655 bytes ..." SourceId: "\000src_id" SeqNo: 288 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219398 UncompressedSize: 31581 PartitionKey: "" ExplicitHash: "" } Result { Offset: 288 Data: "... 31765 bytes ..." SourceId: "\000src_id" SeqNo: 289 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219398 UncompressedSize: 31691 PartitionKey: "" ExplicitHash: "" } Result { Offset: 289 Data: "... 31875 bytes ..." SourceId: "\000src_id" SeqNo: 290 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219398 UncompressedSize: 31801 PartitionKey: "" ExplicitHash: "" } Result { Offset: 290 Data: "... 31985 bytes ..." SourceId: "\000src_id" SeqNo: 291 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219398 UncompressedSize: 31911 PartitionKey: "" ExplicitHash: "" } Result { Offset: 291 Data: "... 32095 bytes ..." SourceId: "\000src_id" SeqNo: 292 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219398 UncompressedSize: 32021 PartitionKey: "" ExplicitHash: "" } Result { Offset: 292 Data: "... 32205 bytes ..." SourceId: "\000src_id" SeqNo: 293 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219398 UncompressedSize: 32131 PartitionKey: "" ExplicitHash: "" } Result { Offset: 293 Data: "... 32315 bytes ..." SourceId: "\000src_id" SeqNo: 294 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219399 UncompressedSize: 32241 PartitionKey: "" ExplicitHash: "" } Result { Offset: 294 Data: "... 32425 bytes ..." SourceId: "\000src_id" SeqNo: 295 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219399 UncompressedSize: 32351 PartitionKey: "" ExplicitHash: "" } Result { Offset: 295 Data: "... 32535 bytes ..." SourceId: "\000src_id" SeqNo: 296 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219399 UncompressedSize: 32461 PartitionKey: "" ExplicitHash: "" } Result { Offset: 296 Data: "... 32645 bytes ..." SourceId: "\000src_id" SeqNo: 297 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219399 UncompressedSize: 32571 PartitionKey: "" ExplicitHash: "" } Result { Offset: 297 Data: "... 32755 bytes ..." SourceId: "\000src_id" SeqNo: 298 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219399 UncompressedSize: 32681 PartitionKey: "" ExplicitHash: "" } Result { Offset: 298 Data: "... 32865 bytes ..." SourceId: "\000src_id" SeqNo: 299 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219399 UncompressedSize: 32791 PartitionKey: "" ExplicitHash: "" } Result { Offset: 299 Data: "... 32975 bytes ..." SourceId: "\000src_id" SeqNo: 300 WriteTimestampMS: 1760046219431 CreateTimestampMS: 1760046219399 UncompressedSize: 32901 PartitionKey: "" ExplicitHash: "" } BlobsFromDisk: 0 BlobsFromCache: 1 SizeLag: 18446744073709550971 RealReadOffset: 299 WaitQuotaTimeMs: 0 EndOffset: 300 StartOffset: 0 } Cookie: 273 }
2025-10-09T21:43:39.536524Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 35} (272-272)
>>> event from dataHandler: 2025-10-09T21:43:39.536769Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:1277: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) wait data in partition inited, cookie 16 from offset 300
2025-10-09T21:43:39.536789Z node 3 :PQ_READ_PROXY DEBUG: partition_actor.cpp:901: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 after read state TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1) EndOffset 300 ReadOffset 300 ReadGuid 2c3b4c60-20ad2d7a-5a959db0-c565be8 has messages 1
DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..28281 bytes.. Information: { Offset: 257 ProducerId: "src_id" SeqNo: 258 CreateTime: 2025-10-09T21:43:39.394000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..28391 bytes.. Information: { Offset: 258 ProducerId: "src_id" SeqNo: 259 CreateTime: 2025-10-09T21:43:39.394000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..28501 bytes.. Information: { Offset: 259 ProducerId: "src_id" SeqNo: 260 CreateTime: 2025-10-09T21:43:39.394000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..28611 bytes.. Information: { Offset: 260 ProducerId: "src_id" SeqNo: 261 CreateTime: 2025-10-09T21:43:39.394000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..28721 bytes.. Information: { Offset: 261 ProducerId: "src_id" SeqNo: 262 CreateTime: 2025-10-09T21:43:39.395000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..28831 bytes.. Information: { Offset: 262 ProducerId: "src_id" SeqNo: 263 CreateTime: 2025-10-09T21:43:39.395000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..28941 bytes.. Information: { Offset: 263 ProducerId: "src_id" SeqNo: 264 CreateTime: 2025-10-09T21:43:39.395000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..29051 bytes.. Information: { Offset: 264 ProducerId: "src_id" SeqNo: 265 CreateTime: 2025-10-09T21:43:39.395000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..29161 bytes.. Information: { Offset: 265 ProducerId: "src_id" SeqNo: 266 CreateTime: 2025-10-09T21:43:39.395000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..29271 bytes.. Information: { Offset: 266 ProducerId: "src_id" SeqNo: 267 CreateTime: 2025-10-09T21:43:39.395000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..29381 bytes.. Information: { Offset: 267 ProducerId: "src_id" SeqNo: 268 CreateTime: 2025-10-09T21:43:39.395000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..29491 bytes.. Information: { Offset: 268 ProducerId: "src_id" SeqNo: 269 CreateTime: 2025-10-09T21:43:39.395000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..29601 bytes.. Information: { Offset: 269 ProducerId: "src_id" SeqNo: 270 CreateTime: 2025-10-09T21:43:39.395000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..29711 bytes.. Information: { Offset: 270 ProducerId: "src_id" SeqNo: 271 CreateTime: 2025-10-09T21:43:39.395000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..29821 bytes.. Information: { Offset: 271 ProducerId: "src_id" SeqNo: 272 CreateTime: 2025-10-09T21:43:39.395000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..29931 bytes.. Information: { Offset: 272 ProducerId: "src_id" SeqNo: 273 CreateTime: 2025-10-09T21:43:39.395000Z WriteTime: 2025-10-09T21:43:39.424000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "server": "ipv6:[::1]:42738", "logtype": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.536831Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1940: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 read done: guid# 2c3b4c60-20ad2d7a-5a959db0-c565be8, partition# TopicId: Topic rt3.dc1--test-topic in dc dc1 in database: Root, partition 0(assignId:1), size# 851049
2025-10-09T21:43:39.536852Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2102: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 response to read: guid# 2c3b4c60-20ad2d7a-5a959db0-c565be8
2025-10-09T21:43:39.536919Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 16, size 465696 bytes
2025-10-09T21:43:39.536928Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.537541Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:2145: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 Process answer. Aval parts: 0
2025-10-09T21:43:39.538295Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Got ReadResponse, serverBytesSize = 851049, now ReadSizeBudget = 0, ReadSizeServerDelta = -326761
2025-10-09T21:43:39.538381Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (273-289)
2025-10-09T21:43:39.538394Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 851049 to budget
2025-10-09T21:43:39.538399Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] In ContinueReadingDataImpl, ReadSizeBudget = 851049, ReadSizeServerDelta = -326761
2025-10-09T21:43:39.538497Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] After sending read request: ReadSizeBudget = 0, ReadSizeServerDelta = 524288
2025-10-09T21:43:39.538527Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 0} (273-273)
2025-10-09T21:43:39.538544Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 1} (274-274)
2025-10-09T21:43:39.538561Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 2} (275-275)
2025-10-09T21:43:39.538567Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 3} (276-276)
2025-10-09T21:43:39.538590Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 4} (277-277)
2025-10-09T21:43:39.538596Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 5} (278-278)
2025-10-09T21:43:39.538602Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 6} (279-279)
2025-10-09T21:43:39.538607Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 7} (280-280)
2025-10-09T21:43:39.538601Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 1, data# { read_request { bytes_size: 851049 } }
2025-10-09T21:43:39.538645Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 8} (281-281)
2025-10-09T21:43:39.538654Z :DEBUG: [/Root] Take Data. Partition 0. Read: {0, 9} (282-282)
2025-10-09T21:43:39.538661Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 0} (283-283)
2025-10-09T21:43:39.538667Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 1} (284-284)
2025-10-09T21:43:39.538660Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:1839: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 got read request: guid# 8b7f0870-d7eee337-ff55c0bb-6759685a
2025-10-09T21:43:39.538672Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 2} (285-285)
2025-10-09T21:43:39.538678Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 3} (286-286)
2025-10-09T21:43:39.538684Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 4} (287-287)
2025-10-09T21:43:39.538690Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 5} (288-288)
2025-10-09T21:43:39.538769Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 6} (289-289)
>>> event from dataHandler: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..30041 bytes.. Information: { Offset: 273 ProducerId: "src_id" SeqNo: 274 CreateTime: 2025-10-09T21:43:39.395000Z WriteTime: 2025-10-09T21:43:39.430000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..30151 bytes.. Information: { Offset: 274 ProducerId: "src_id" SeqNo: 275 CreateTime: 2025-10-09T21:43:39.396000Z WriteTime: 2025-10-09T21:43:39.430000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..30261 bytes.. Information: { Offset: 275 ProducerId: "src_id" SeqNo: 276 CreateTime: 2025-10-09T21:43:39.396000Z WriteTime: 2025-10-09T21:43:39.430000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..30371 bytes.. Information: { Offset: 276 ProducerId: "src_id" SeqNo: 277 CreateTime: 2025-10-09T21:43:39.396000Z WriteTime: 2025-10-09T21:43:39.430000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..30481 bytes.. Information: { Offset: 277 ProducerId: "src_id" SeqNo: 278 CreateTime: 2025-10-09T21:43:39.396000Z WriteTime: 2025-10-09T21:43:39.430000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..30591 bytes.. Information: { Offset: 278 ProducerId: "src_id" SeqNo: 279 CreateTime: 2025-10-09T21:43:39.396000Z WriteTime: 2025-10-09T21:43:39.430000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..30701 bytes.. Information: { Offset: 279 ProducerId: "src_id" SeqNo: 280 CreateTime: 2025-10-09T21:43:39.396000Z WriteTime: 2025-10-09T21:43:39.430000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..30811 bytes.. Information: { Offset: 280 ProducerId: "src_id" SeqNo: 281 CreateTime: 2025-10-09T21:43:39.398000Z WriteTime: 2025-10-09T21:43:39.430000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..30921 bytes.. Information: { Offset: 281 ProducerId: "src_id" SeqNo: 282 CreateTime: 2025-10-09T21:43:39.398000Z WriteTime: 2025-10-09T21:43:39.430000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..31031 bytes.. Information: { Offset: 282 ProducerId: "src_id" SeqNo: 283 CreateTime: 2025-10-09T21:43:39.398000Z WriteTime: 2025-10-09T21:43:39.430000Z MessageGroupId: "src_id" Meta: { "ident": "unknown", "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..31141 bytes.. Information: { Offset: 283 ProducerId: "src_id" SeqNo: 284 CreateTime: 2025-10-09T21:43:39.398000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..31251 bytes.. Information: { Offset: 284 ProducerId: "src_id" SeqNo: 285 CreateTime: 2025-10-09T21:43:39.398000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..31361 bytes.. Information: { Offset: 285 ProducerId: "src_id" SeqNo: 286 CreateTime: 2025-10-09T21:43:39.398000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..31471 bytes.. Information: { Offset: 286 ProducerId: "src_id" SeqNo: 287 CreateTime: 2025-10-09T21:43:39.398000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..31581 bytes.. Information: { Offset: 287 ProducerId: "src_id" SeqNo: 288 CreateTime: 2025-10-09T21:43:39.398000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..31691 bytes.. Information: { Offset: 288 ProducerId: "src_id" SeqNo: 289 CreateTime: 2025-10-09T21:43:39.398000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..31801 bytes.. Information: { Offset: 289 ProducerId: "src_id" SeqNo: 290 CreateTime: 2025-10-09T21:43:39.398000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.539432Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 17, size 525657 bytes
2025-10-09T21:43:39.539464Z :DEBUG: [/Root] Decompression task done. Partition/PartitionSessionId: 1 (290-299)
2025-10-09T21:43:39.539476Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.539509Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 7} (290-290)
2025-10-09T21:43:39.539523Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 8} (291-291)
2025-10-09T21:43:39.539535Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 9} (292-292)
2025-10-09T21:43:39.539542Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 10} (293-293)
2025-10-09T21:43:39.539559Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 11} (294-294)
2025-10-09T21:43:39.539566Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 12} (295-295)
2025-10-09T21:43:39.539573Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 13} (296-296)
2025-10-09T21:43:39.539580Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 14} (297-297)
2025-10-09T21:43:39.539618Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 15} (298-298)
2025-10-09T21:43:39.539645Z :DEBUG: [/Root] Take Data. Partition 0. Read: {1, 16} (299-299)
>>> event from dataHandler: DataReceived { Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 Message { Data: ..31911 bytes.. Information: { Offset: 290 ProducerId: "src_id" SeqNo: 291 CreateTime: 2025-10-09T21:43:39.398000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..32021 bytes.. Information: { Offset: 291 ProducerId: "src_id" SeqNo: 292 CreateTime: 2025-10-09T21:43:39.398000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..32131 bytes.. Information: { Offset: 292 ProducerId: "src_id" SeqNo: 293 CreateTime: 2025-10-09T21:43:39.398000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..32241 bytes.. Information: { Offset: 293 ProducerId: "src_id" SeqNo: 294 CreateTime: 2025-10-09T21:43:39.399000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..32351 bytes.. Information: { Offset: 294 ProducerId: "src_id" SeqNo: 295 CreateTime: 2025-10-09T21:43:39.399000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..32461 bytes.. Information: { Offset: 295 ProducerId: "src_id" SeqNo: 296 CreateTime: 2025-10-09T21:43:39.399000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..32571 bytes.. Information: { Offset: 296 ProducerId: "src_id" SeqNo: 297 CreateTime: 2025-10-09T21:43:39.399000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..32681 bytes.. Information: { Offset: 297 ProducerId: "src_id" SeqNo: 298 CreateTime: 2025-10-09T21:43:39.399000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..32791 bytes.. Information: { Offset: 298 ProducerId: "src_id" SeqNo: 299 CreateTime: 2025-10-09T21:43:39.399000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } Message { Data: ..32901 bytes.. Information: { Offset: 299 ProducerId: "src_id" SeqNo: 300 CreateTime: 2025-10-09T21:43:39.399000Z WriteTime: 2025-10-09T21:43:39.431000Z MessageGroupId: "src_id" Meta: { "_ip": "ipv6:[::1]:42738", "logtype": "unknown", "server": "ipv6:[::1]:42738", "ident": "unknown" } MessageMeta: { } } Partition session id: 1 Topic: "test-topic" Partition: 0 Database name: dc2 Database path: /Root Database id: account-dc2 } }
2025-10-09T21:43:39.539966Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] The application data is transferred to the client. Number of messages 10, size 324060 bytes
2025-10-09T21:43:39.539975Z :DEBUG: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] [] Returning serverBytesSize = 0 to budget
2025-10-09T21:43:39.539980Z :INFO: [/Root] [/Root] [ee2235dc-c75c6e93-275e2257-6ad62539] Closing read session. Close timeout: 18446744073709.551615s
2025-10-09T21:43:39.539994Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset):
2025-10-09T21:43:39.540003Z :INFO: [/Root] [/Root] [ee2235dc-c75c6e93-275e2257-6ad62539] Counters: { Errors: 0 CurrentSessionLifetimeMs: 289 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:43:39.540311Z :INFO: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] Closing read session. Close timeout: 18446744073709.551615s
2025-10-09T21:43:39.540326Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:0:1:299:0
2025-10-09T21:43:39.540332Z :INFO: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] Counters: { Errors: 0 CurrentSessionLifetimeMs: 288 BytesRead: 4936800 MessagesRead: 300 BytesReadCompressed: 4936800 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:43:39.540311Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 1 consumer shared/user session shared/user_3_1_13017312322801993971_v1 grpc read done: success# 0, data# {  }
2025-10-09T21:43:39.540323Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:125: session cookie 1 consumer shared/user session shared/user_3_1_13017312322801993971_v1 grpc read failed
2025-10-09T21:43:39.540330Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:1666: session cookie 1 consumer shared/user session shared/user_3_1_13017312322801993971_v1 closed
2025-10-09T21:43:39.540474Z :INFO: [/Root] [/Root] [325a6ae6-7e984b87-19964452-73f5f42d] Closing read session. Close timeout: 18446744073709.551615s
2025-10-09T21:43:39.540483Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset):
2025-10-09T21:43:39.540488Z :INFO: [/Root] [/Root] [325a6ae6-7e984b87-19964452-73f5f42d] Counters: { Errors: 0 CurrentSessionLifetimeMs: 288 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:43:39.540477Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 1 consumer shared/user session shared/user_3_1_13017312322801993971_v1 is DEAD
2025-10-09T21:43:39.540531Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: close. Timeout = 0 ms
2025-10-09T21:43:39.540535Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session will now close
2025-10-09T21:43:39.540540Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: aborting
2025-10-09T21:43:39.540564Z :INFO: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:43:39.540569Z :DEBUG: [/Root] MessageGroupId [src_id] SessionId [src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0] Write session: destroy
2025-10-09T21:43:39.540665Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read done: success# 0, data# {  }
2025-10-09T21:43:39.540678Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:125: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc read failed
2025-10-09T21:43:39.540683Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:92: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 grpc closed
2025-10-09T21:43:39.540698Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 2 consumer shared/user session shared/user_3_2_7554872134447193526_v1 is DEAD
2025-10-09T21:43:39.540798Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857587:2500] disconnected.
2025-10-09T21:43:39.540816Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857587:2500] disconnected; active server actors: 1
2025-10-09T21:43:39.540821Z node 4 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857587:2500] client user disconnected session shared/user_3_1_13017312322801993971_v1
2025-10-09T21:43:39.540832Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:43:39.540845Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=2, Families=1, UnradableFamilies=0 [], RequireBalancing=0 []
2025-10-09T21:43:39.540852Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=2, desiredFamilyCount=0, allowPlusOne=1
2025-10-09T21:43:39.540856Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000007s
2025-10-09T21:43:39.540964Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857590:2502] disconnected.
2025-10-09T21:43:39.540970Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857590:2502] disconnected; active server actors: 1
2025-10-09T21:43:39.540974Z node 4 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857590:2502] client user disconnected session shared/user_3_3_14754793126700281017_v1
2025-10-09T21:43:39.540984Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1184: [72075186224037893][rt3.dc1--test-topic] consumer user rebalancing was scheduled
2025-10-09T21:43:39.540999Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1256: [72075186224037893][rt3.dc1--test-topic] consumer user balancing. Sessions=1, Families=1, UnradableFamilies=0 [], RequireBalancing=0 []
2025-10-09T21:43:39.541016Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1323: [72075186224037893][rt3.dc1--test-topic] consumer user start rebalancing. familyCount=1, sessionCount=1, desiredFamilyCount=1, allowPlusOne=0
2025-10-09T21:43:39.541030Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1400: [72075186224037893][rt3.dc1--test-topic] consumer user balancing duration: 0.000027s
2025-10-09T21:43:39.541037Z :INFO: [/Root] [/Root] [ee2235dc-c75c6e93-275e2257-6ad62539] Closing read session. Close timeout: 0.000000s
2025-10-09T21:43:39.541034Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857589:2501] disconnected.
2025-10-09T21:43:39.541055Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset):
2025-10-09T21:43:39.541068Z :INFO: [/Root] [/Root] [ee2235dc-c75c6e93-275e2257-6ad62539] Counters: { Errors: 0 CurrentSessionLifetimeMs: 290 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:43:39.541077Z :INFO: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] Closing read session. Close timeout: 0.000000s
2025-10-09T21:43:39.541082Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:0:1:299:0
2025-10-09T21:43:39.541086Z :INFO: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] Counters: { Errors: 0 CurrentSessionLifetimeMs: 289 BytesRead: 4936800 MessagesRead: 300 BytesReadCompressed: 4936800 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:43:39.541090Z :INFO: [/Root] [/Root] [325a6ae6-7e984b87-19964452-73f5f42d] Closing read session. Close timeout: 0.000000s
2025-10-09T21:43:39.541036Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857589:2501] disconnected; active server actors: 1
2025-10-09T21:43:39.541094Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset):
2025-10-09T21:43:39.541099Z :INFO: [/Root] [/Root] [325a6ae6-7e984b87-19964452-73f5f42d] Counters: { Errors: 0 CurrentSessionLifetimeMs: 289 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:43:39.541039Z node 4 :PERSQUEUE_READ_BALANCER NOTICE: read_balancer__balancing.cpp:1677: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340952763857589:2501] client user disconnected session shared/user_3_2_7554872134447193526_v1
2025-10-09T21:43:39.541103Z :INFO: [/Root] [/Root] [325a6ae6-7e984b87-19964452-73f5f42d] Closing read session. Close timeout: 0.000000s
2025-10-09T21:43:39.541105Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset):
2025-10-09T21:43:39.541108Z :INFO: [/Root] [/Root] [325a6ae6-7e984b87-19964452-73f5f42d] Counters: { Errors: 0 CurrentSessionLifetimeMs: 289 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:43:39.541122Z :NOTICE: [/Root] [/Root] [325a6ae6-7e984b87-19964452-73f5f42d] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-09T21:43:39.541146Z :INFO: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] Closing read session. Close timeout: 0.000000s
2025-10-09T21:43:39.541149Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset): -:test-topic:0:1:299:0
2025-10-09T21:43:39.541152Z :INFO: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] Counters: { Errors: 0 CurrentSessionLifetimeMs: 289 BytesRead: 4936800 MessagesRead: 300 BytesReadCompressed: 4936800 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:43:39.541157Z :NOTICE: [/Root] [/Root] [efc8a570-e658bbac-176fa6e1-92b1a2b8] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-09T21:43:39.541171Z :INFO: [/Root] [/Root] [ee2235dc-c75c6e93-275e2257-6ad62539] Closing read session. Close timeout: 0.000000s
2025-10-09T21:43:39.541173Z :INFO: [/Root] Read/commit by partition streams (cluster:topic:partition:stream-id:read-offset:committed-offset):
2025-10-09T21:43:39.541177Z :INFO: [/Root] [/Root] [ee2235dc-c75c6e93-275e2257-6ad62539] Counters: { Errors: 0 CurrentSessionLifetimeMs: 290 BytesRead: 0 MessagesRead: 0 BytesReadCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:43:39.541181Z :NOTICE: [/Root] [/Root] [ee2235dc-c75c6e93-275e2257-6ad62539] Aborting read session. Description: SessionClosed { Status: ABORTED Issues: "<main>: Error: Aborted " }
2025-10-09T21:43:39.540757Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2254: [PQ: 72075186224037892] Destroy direct read session shared/user_3_2_7554872134447193526_v1
2025-10-09T21:43:39.540772Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [3:7559340952763857612:2512] destroyed
2025-10-09T21:43:39.540788Z node 3 :PQ_READ_PROXY DEBUG: caching_service.cpp:139: Direct read cache: server session deregistered: shared/user_3_2_7554872134447193526_v1
2025-10-09T21:43:39.540797Z node 3 :PQ_READ_PROXY DEBUG: read_session_actor.cpp:122: session cookie 3 consumer shared/user session shared/user_3_3_14754793126700281017_v1 grpc read done: success# 0, data# {  }
2025-10-09T21:43:39.540798Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:125: session cookie 3 consumer shared/user session shared/user_3_3_14754793126700281017_v1 grpc read failed
2025-10-09T21:43:39.540801Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:92: session cookie 3 consumer shared/user session shared/user_3_3_14754793126700281017_v1 grpc closed
2025-10-09T21:43:39.540804Z node 3 :PQ_READ_PROXY INFO: read_session_actor.cpp:383: session cookie 3 consumer shared/user session shared/user_3_3_14754793126700281017_v1 is DEAD
2025-10-09T21:43:39.540898Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read done: success: 0 data: 
2025-10-09T21:43:39.540899Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc read failed
2025-10-09T21:43:39.540905Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 grpc closed
2025-10-09T21:43:39.540908Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2 sessionId: src_id|cea947f0-6b6ed291-e950e71c-44bae1a9_0 is DEAD
2025-10-09T21:43:39.541031Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:43:39.541060Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [3:7559340952763857631:2499] destroyed
2025-10-09T21:43:39.541069Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:43:39.541077Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.541081Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.541083Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.579417Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.579449Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.579452Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.685433Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.685456Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.685459Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.785825Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.785849Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.785852Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.886143Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.886167Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.886170Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:44.729843Z :WriteSessionCloseWaitsForWrites INFO: Random seed for debugging is 1760046164729838
2025-10-09T21:42:44.957685Z node 1 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7559340717449145183:2259];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:44.957735Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-10-09T21:42:44.993916Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/owmb/0001b4/r3tmp/tmpyq3KZL/pdisk_1.dat
2025-10-09T21:42:45.019360Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:45.094089Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:45.094084Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:45.094167Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
2025-10-09T21:42:45.111784Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:45.111815Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:45.111973Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:45.111985Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:45.121038Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-10-09T21:42:45.121089Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:45.121217Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:45.122203Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
TServer::EnableGrpc on GrpcPort 7644, node 1
2025-10-09T21:42:45.172903Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/owmb/0001b4/r3tmp/yandex0mGENC.tmp
2025-10-09T21:42:45.172917Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/owmb/0001b4/r3tmp/yandex0mGENC.tmp
2025-10-09T21:42:45.172978Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/owmb/0001b4/r3tmp/yandex0mGENC.tmp
2025-10-09T21:42:45.173033Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-10-09T21:42:45.194292Z INFO: TTestServer started on Port 9580 GrpcPort 7644
TClient is connected to server localhost:9580
PQClient connected to localhost:7644
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-09T21:42:45.262500Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976710657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
2025-10-09T21:42:45.286936Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
waiting...
2025-10-09T21:42:45.314903Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976710658, at schemeshard: 72057594046644480
waiting...
waiting...
2025-10-09T21:42:45.343585Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:45.622820Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7559340720147771590:2296], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.622845Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7559340720147771617:2299], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.622863Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.623933Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7559340720147771622:2301], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.623951Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.625070Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976720657:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-10-09T21:42:45.637806Z node 2 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [2:7559340720147771621:2300], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720657 completed, doublechecking }
2025-10-09T21:42:45.690713Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [1:7559340721744113248:2330], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:45.690816Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976710661:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:45.696209Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=1&id=Y2UyNTA1ZGQtNGU0YzdhYTUtNDYyM2MxOWQtY2Y5MTMwZjc=, ActorId: [1:7559340721744113203:2322], ActorState: ExecuteState, TraceId: 01k75etbkw6sd7ykdgxwpezetm, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:45.696851Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:45.723456Z node 2 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [2:7559340720147771687:2162] txid# 281474976720658, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:45.734124Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [2:7559340720147771708:2308], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:45.735137Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=2&id=YmJmN2IxYTQtYWExYzgxZDUtOTY3NmVhZjYtNjkyZDY5NWQ=, ActorId: [2:7559340720147771588:2294], ActorState: ExecuteState, TraceId: 01k75etbkj106cnh2g3ak98c58, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:45.735275Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:45.747194Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976710662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:45.780492Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976710663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:7644", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, false, 1000);
2025-10-09T21:42:45.825398Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976710664. Ctx: { TraceId: 01k75etbs82kejkkaaskerf0nk, Database: , SessionId: ydb://session/3?node_id=1&id=NzNkYzQyODEtNzEyZTg1NS1hYWMwZDc5LTQ5OTRlN2Fj, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [1:7559340721744113658:2918] 
2025-10-09T21:42:45.957712Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:46.015113Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:49.957921Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7559340717449145183:2259];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:49.957951Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
2025-10-09T21:42:50.995248Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976710670:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
waiting...
PQ Client: create topic: rt3.dc1--test-topic with 1 partitions
CallPersQueueGRPC request to localhost:7644
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:51.048037Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:7644
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976710671
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-09T21:42:51.070597Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][] pipe [1:7559340747513917863:3189] connected; active server actors: 1
2025-10-09T21:42:51.070667Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1517: [72075186224037893][rt3.dc1--test-topic] updating configuration. Deleted partitions []. Added partitions [0]
2025-10-09T21:42:51.071378Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:940: [72075186224037893][rt3.dc1--test-topic] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-09T21:42:51.071746Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:138: [72075186224037893][rt3.dc1--test-topic] BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) 
2025-10-09T21:42:51.078056Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976710672. Ctx: { TraceId: 01k75etgxhd97b1v5pmnk8m2ns, Database: , SessionId: ydb://session/3?node_id=1&id=NTJiNzQ0MGQtMTkyMjJkZTgtODVhYjM2Mi1jMDY5NDM5NA==, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-10-09T21:42:51.096767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:51.099949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:51.100060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:784: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-09T21:42:51.100073Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:51.100086Z node 2 :PERSQUEUE INFO: pq_impl.cpp:609: [PQ: 72075186224037892] doesn't have tx writes info
2025-10-09T21:42:51.100482Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-10-09T21:42:51.100707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340747513917883:2436], now have 1 active actors on pipe
2025-10-09T21:42:51.100718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340747513917862:3188], now have 1 active actors on pipe
2025-10-09T21:42:51.100843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
GetTopicVersionFromPath:  record Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
 name rt3.dc1--test-topic version0
2025-10-09T21:42:51.102620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:51.105196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340747513917940:3240], now have 1 active actors on pipe
2025-10-09T21:42:51.105622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:51.106812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:51.107037Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:51.107055Z node 2 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:51.107106Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:51.107173Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340745917575913:2384]
2025-10-09T21:42:51.107978Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:51.107981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:51.107986Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [2:7559340745917575913:2384]
2025-10-09T21:42:51.107993Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:51.108339Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-10-09T21:42:51.108351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.108375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.108378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:51.108382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:51.108384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.108386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.108388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:51.108394Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:51.108458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:51.108487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:51.110598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:51.110636Z node 2 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046171149, TxId 281474976710671
2025-10-09T21:42:51.110641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.110642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.110663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.110698Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:51.111707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:51.111758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:51.111762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.111764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.111766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.111770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:51.111859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:395: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-10-09T21:42:51.111875Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:51.111909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:51.114546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:51.114567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:51.115212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:51.206691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.206709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.206712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.306556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.306568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.306572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.406903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.406915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.406918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.513934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.513948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.513950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976710671
    CreateStep: 1760046171149
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976710671
    CreateStep: 1760046171149
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:7644
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:51.609480Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
2025-10-09T21:42:51.614046Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-09T21:42:51.614344Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-09T21:42:51.614352Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:7644
2025-10-09T21:42:51.614340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.614360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.614362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.616117Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:51.616574Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:51.616590Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1
2025-10-09T21:42:51.617494Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:51.617521Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:40984
2025-10-09T21:42:51.617526Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:40984 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:51.617530Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:51.617994Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-09T21:42:51.618043Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:51.618045Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:51.618047Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:51.618053Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340747513917991:2445] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:51.618604Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340747513917991:2445] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:51.656498Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [1:7559340747513917991:2445] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:51.657085Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340747513918027:2445] connected; active server actors: 1
2025-10-09T21:42:51.657164Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [1:7559340747513917991:2445] (SourceId=src, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src
2025-10-09T21:42:51.657171Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340747513917991:2445] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:51.661322Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340747513918027:2445] disconnected.
2025-10-09T21:42:51.661332Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340747513918027:2445] disconnected; active server actors: 1
2025-10-09T21:42:51.661335Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340747513918027:2445] disconnected no session
2025-10-09T21:42:51.694618Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340747513917991:2445] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:51.694637Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340747513917991:2445] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:42:51.694641Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340747513917991:2445] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:51.694652Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:51.697484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340747513918049:2445], now have 1 active actors on pipe
2025-10-09T21:42:51.705159Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1
2025-10-09T21:42:51.706594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.706615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.706650Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src|4cebc49e-e14dd5f5-64bfe08e-297ada7b_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src
2025-10-09T21:42:51.706662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.706667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.706670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:51.706683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.706700Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:51.706722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.707274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.707283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.707302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.707871Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046171707 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:51.707910Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|4cebc49e-e14dd5f5-64bfe08e-297ada7b_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:51.708043Z :INFO: [] MessageGroupId [src] SessionId [src|4cebc49e-e14dd5f5-64bfe08e-297ada7b_0] Write session: close. Timeout = 0 ms
2025-10-09T21:42:51.708050Z :INFO: [] MessageGroupId [src] SessionId [src|4cebc49e-e14dd5f5-64bfe08e-297ada7b_0] Write session will now close
2025-10-09T21:42:51.708055Z :DEBUG: [] MessageGroupId [src] SessionId [src|4cebc49e-e14dd5f5-64bfe08e-297ada7b_0] Write session: aborting
2025-10-09T21:42:51.708165Z :INFO: [] MessageGroupId [src] SessionId [src|4cebc49e-e14dd5f5-64bfe08e-297ada7b_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:42:51.708170Z :DEBUG: [] MessageGroupId [src] SessionId [src|4cebc49e-e14dd5f5-64bfe08e-297ada7b_0] Write session: destroy
2025-10-09T21:42:51.707503Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1 partition: 0 MaxSeqNo: 0 sessionId: src|4cebc49e-e14dd5f5-64bfe08e-297ada7b_0
2025-10-09T21:42:51.711321Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId: src|4cebc49e-e14dd5f5-64bfe08e-297ada7b_0 grpc read done: success: 0 data: 
2025-10-09T21:42:51.711335Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 1 sessionId: src|4cebc49e-e14dd5f5-64bfe08e-297ada7b_0 grpc read failed
2025-10-09T21:42:51.711343Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 1 sessionId: src|4cebc49e-e14dd5f5-64bfe08e-297ada7b_0 grpc closed
2025-10-09T21:42:51.711351Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1 sessionId: src|4cebc49e-e14dd5f5-64bfe08e-297ada7b_0 is DEAD
2025-10-09T21:42:51.711602Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:42:51.712143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340747513918049:2445] destroyed
2025-10-09T21:42:51.712162Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:42:51.712171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.712174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.712177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Session was created
2025-10-09T21:42:51.715790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.715802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.715805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: ok
2025-10-09T21:42:51.767083Z :INFO: [/Root] [] [] Start federated write session to database 'dc1' (previous was <empty>) FederationState: { Status: SUCCESS SelfLocation: "fancy_datacenter" DbInfos: [ { name: "dc1" path: "/Root" id: "account-dc1" endpoint: "localhost:7644" location: "dc1" status: AVAILABLE weight: 1000 } { name: "dc2" path: "/Root" id: "account-dc2" endpoint: "localhost:7644" location: "dc2" status: AVAILABLE weight: 500 } { name: "dc3" path: "/Root" id: "account-dc3" endpoint: "localhost:7644" location: "dc3" status: AVAILABLE weight: 500 } ] ControlPlaneEndpoint: cp.logbroker-federation:2135 }
2025-10-09T21:42:51.771749Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: try to update token
2025-10-09T21:42:51.771951Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Start write session. Will connect to nodeId: 0
2025-10-09T21:42:51.822130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.822148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.822151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.862689Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: write to message_group: src_id
2025-10-09T21:42:51.864028Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:51.864042Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2
2025-10-09T21:42:51.864490Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: send init request: init_request { path: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:51.864649Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: OnWriteDone gRpcStatusCode: 0
2025-10-09T21:42:51.865262Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId:  grpc read done: success: 1 data: init_request { path: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:51.865322Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2 path: "test-topic" message_group_id: "src_id" from ipv6:[::1]:37092
2025-10-09T21:42:51.865327Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2 sessionId= userAgent="topic server" ip=ipv6:[::1]:37092 proto=topic  topic=test-topic durationSec=0
2025-10-09T21:42:51.865331Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:51.865958Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2 sessionId:  describe result for acl check
2025-10-09T21:42:51.865993Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:51.865995Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:51.865997Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:51.866003Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340747513918062:2459] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:51.866542Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340747513918062:2459] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:51.880398Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [1:7559340747513918062:2459] (SourceId=src_id, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:51.880811Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340747513918081:2459] connected; active server actors: 1
2025-10-09T21:42:51.880901Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [1:7559340747513918062:2459] (SourceId=src_id, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src_id
2025-10-09T21:42:51.880908Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340747513918062:2459] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:51.886362Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340747513918081:2459] disconnected.
2025-10-09T21:42:51.886375Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340747513918081:2459] disconnected; active server actors: 1
2025-10-09T21:42:51.886379Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340747513918081:2459] disconnected no session
2025-10-09T21:42:51.895064Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340747513918062:2459] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:51.895081Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340747513918062:2459] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:42:51.895085Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340747513918062:2459] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:51.895095Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:51.895850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340747513918097:2459], now have 1 active actors on pipe
2025-10-09T21:42:51.895969Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1
2025-10-09T21:42:51.896107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.896115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.896142Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|e5647110-687d8388-30f6bdc1-39e93807_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:51.896162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.896167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.896170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:51.896182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.896199Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:51.896216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.896516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.896521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.896533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:51.896653Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2 partition: 0 MaxSeqNo: 0 sessionId: src_id|e5647110-687d8388-30f6bdc1-39e93807_0
2025-10-09T21:42:51.897163Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: OnReadDone gRpcStatusCode: 0
2025-10-09T21:42:51.897184Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046171897 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:51.897215Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session established. Init response: session_id: "src_id|e5647110-687d8388-30f6bdc1-39e93807_0" supported_codecs { codecs: 1 codecs: 2 codecs: 3 }
2025-10-09T21:42:51.902109Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 1 to 1
2025-10-09T21:42:51.902156Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 2 to 2
2025-10-09T21:42:51.902170Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 3 to 3
2025-10-09T21:42:51.902177Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 4 to 4
2025-10-09T21:42:51.902186Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 5 to 5
2025-10-09T21:42:51.902192Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 6 to 6
2025-10-09T21:42:51.902199Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 7 to 7
2025-10-09T21:42:51.902205Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 8 to 8
2025-10-09T21:42:51.902216Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 9 to 9
2025-10-09T21:42:51.902222Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 10 to 10
2025-10-09T21:42:51.902230Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 11 to 11
2025-10-09T21:42:51.902235Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 12 to 12
2025-10-09T21:42:51.902241Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 13 to 13
2025-10-09T21:42:51.902247Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 14 to 14
2025-10-09T21:42:51.902254Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 15 to 15
2025-10-09T21:42:51.902261Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 16 to 16
2025-10-09T21:42:51.902268Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 17 to 17
2025-10-09T21:42:51.902275Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 18 to 18
2025-10-09T21:42:51.902281Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 19 to 19
2025-10-09T21:42:51.902288Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 20 to 20
2025-10-09T21:42:51.902294Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 21 to 21
2025-10-09T21:42:51.902300Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 22 to 22
2025-10-09T21:42:51.902306Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 23 to 23
2025-10-09T21:42:51.902312Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 24 to 24
2025-10-09T21:42:51.902318Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 25 to 25
2025-10-09T21:42:51.902323Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 26 to 26
2025-10-09T21:42:51.902328Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 27 to 27
2025-10-09T21:42:51.902335Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 28 to 28
2025-10-09T21:42:51.902341Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 29 to 29
2025-10-09T21:42:51.902347Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 30 to 30
2025-10-09T21:42:51.902353Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 31 to 31
2025-10-09T21:42:51.902359Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 32 to 32
2025-10-09T21:42:51.902366Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 33 to 33
2025-10-09T21:42:51.902379Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 34 to 34
2025-10-09T21:42:51.902385Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 35 to 35
2025-10-09T21:42:51.902390Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 36 to 36
2025-10-09T21:42:51.902396Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 37 to 37
2025-10-09T21:42:51.902402Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 38 to 38
2025-10-09T21:42:51.902407Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 39 to 39
2025-10-09T21:42:51.902425Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 40 to 40
2025-10-09T21:42:51.902432Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 41 to 41
2025-10-09T21:42:51.902438Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 42 to 42
2025-10-09T21:42:51.902444Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 43 to 43
2025-10-09T21:42:51.902449Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 44 to 44
2025-10-09T21:42:51.902455Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 45 to 45
2025-10-09T21:42:51.902462Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 46 to 46
2025-10-09T21:42:51.902468Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 47 to 47
2025-10-09T21:42:51.902474Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 48 to 48
2025-10-09T21:42:51.902480Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 49 to 49
2025-10-09T21:42:51.902485Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 50 to 50
2025-10-09T21:42:51.902491Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 51 to 51
2025-10-09T21:42:51.902497Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 52 to 52
2025-10-09T21:42:51.902505Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 53 to 53
2025-10-09T21:42:51.902510Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 54 to 54
2025-10-09T21:42:51.902517Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 55 to 55
2025-10-09T21:42:51.902523Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 56 to 56
2025-10-09T21:42:51.902530Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 57 to 57
2025-10-09T21:42:51.902536Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 58 to 58
2025-10-09T21:42:51.902542Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 59 to 59
2025-10-09T21:42:51.902547Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 60 to 60
2025-10-09T21:42:51.902555Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 61 to 61
2025-10-09T21:42:51.902561Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 62 to 62
2025-10-09T21:42:51.902569Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 63 to 63
2025-10-09T21:42:51.902575Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 64 to 64
2025-10-09T21:42:51.902582Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 65 to 65
2025-10-09T21:42:51.902588Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 66 to 66
2025-10-09T21:42:51.902594Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 67 to 67
2025-10-09T21:42:51.902599Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 68 to 68
2025-10-09T21:42:51.902606Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 69 to 69
2025-10-09T21:42:51.902612Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 70 to 70
2025-10-09T21:42:51.902618Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 71 to 71
2025-10-09T21:42:51.902623Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 72 to 72
2025-10-09T21:42:51.902630Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 73 to 73
2025-10-09T21:42:51.902640Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 74 to 74
2025-10-09T21:42:51.902646Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 75 to 75
2025-10-09T21:42:51.902654Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 76 to 76
2025-10-09T21:42:51.902662Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 77 to 77
2025-10-09T21:42:51.902667Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 78 to 78
2025-10-09T21:42:51.902678Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 79 to 79
2025-10-09T21:42:51.902683Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 80 to 80
2025-10-09T21:42:51.902695Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 81 to 81
2025-10-09T21:42:51.902701Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 82 to 82
2025-10-09T21:42:51.902707Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 83 to 83
2025-10-09T21:42:51.902713Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 84 to 84
2025-10-09T21:42:51.902719Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 85 to 85
2025-10-09T21:42:51.902726Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 86 to 86
2025-10-09T21:42:51.902732Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 87 to 87
2025-10-09T21:42:51.902737Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 88 to 88
2025-10-09T21:42:51.902742Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 89 to 89
2025-10-09T21:42:51.902748Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 90 to 90
2025-10-09T21:42:51.902753Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 91 to 91
2025-10-09T21:42:51.902759Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 92 to 92
2025-10-09T21:42:51.902764Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 93 to 93
2025-10-09T21:42:51.902769Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 94 to 94
2025-10-09T21:42:51.902775Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 95 to 95
2025-10-09T21:42:51.902781Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 96 to 96
2025-10-09T21:42:51.902788Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 97 to 97
2025-10-09T21:42:51.902793Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 98 to 98
2025-10-09T21:42:51.902799Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 99 to 99
2025-10-09T21:42:51.902805Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write 1 messages with Id from 100 to 100
2025-10-09T21:42:51.902961Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: try to update token
2025-10-09T21:42:51.902976Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Send 33 message(s) (67 left), first sequence number is 1
2025-10-09T21:42:51.903916Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: try to update token
2025-10-09T21:42:51.903929Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Send 66 message(s) (1 left), first sequence number is 34
2025-10-09T21:42:51.906097Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|e5647110-687d8388-30f6bdc1-39e93807_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.906343Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.906357Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|e5647110-687d8388-30f6bdc1-39e93807_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.906587Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.907006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.907022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.907036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.907039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.907066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:42:51.907078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1  requestId: <none> cookie: 2
2025-10-09T21:42:51.907268Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.907288Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.908008Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: try to update token
2025-10-09T21:42:51.908020Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Send 1 message(s) (0 left), first sequence number is 100
2025-10-09T21:42:51.907532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.907537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.907646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 0 messageNo: 2 size 98 offset: -1
2025-10-09T21:42:51.907650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 2 partNo : 0 messageNo: 2 size 98 offset: -1
2025-10-09T21:42:51.907654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 3 partNo : 0 messageNo: 2 size 98 offset: -1
2025-10-09T21:42:51.907658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 4 partNo : 0 messageNo: 2 size 98 offset: -1
2025-10-09T21:42:51.907662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 5 partNo : 0 messageNo: 2 size 98 offset: -1
2025-10-09T21:42:51.907666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 6 partNo : 0 messageNo: 2 size 98 offset: -1
2025-10-09T21:42:51.907668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 7 partNo : 0 messageNo: 2 size 98 offset: -1
2025-10-09T21:42:51.907671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 8 partNo : 0 messageNo: 2 size 98 offset: -1
2025-10-09T21:42:51.907675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 9 partNo : 0 messageNo: 2 size 98 offset: -1
2025-10-09T21:42:51.907678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 10 partNo : 0 messageNo: 2 size 98 offset: -1
2025-10-09T21:42:51.907680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 11 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 12 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 13 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 14 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 15 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 16 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 17 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 18 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 19 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 20 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 21 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 22 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 23 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 24 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 25 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 26 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 27 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 28 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 29 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 30 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 31 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 32 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 33 partNo : 0 messageNo: 2 size 99 offset: -1
2025-10-09T21:42:51.907810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.907811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.907815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 34 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 35 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 36 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 37 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 38 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 39 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 40 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 41 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 42 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 43 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 44 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 45 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 46 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 47 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 48 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 49 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 50 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 51 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 52 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 53 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 54 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 55 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 56 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 57 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 58 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 59 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 60 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 61 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 62 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 63 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 64 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 65 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 66 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 67 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 68 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 69 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 70 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 71 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 72 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 73 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 74 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 75 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 76 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 77 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 78 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 79 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 80 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 81 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 82 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 83 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 84 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 85 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 86 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 87 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 88 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 89 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 90 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 91 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 92 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 93 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 94 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.907999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 95 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.908002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 96 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.908005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 97 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.908011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 98 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.908015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 99 partNo : 0 messageNo: 3 size 99 offset: -1
2025-10-09T21:42:51.908034Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.908078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.908092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.908095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (33)
2025-10-09T21:42:51.908122Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 0
2025-10-09T21:42:51.908235Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 1 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 172 count 1 nextOffset 1 batches 1
2025-10-09T21:42:51.908244Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 0
2025-10-09T21:42:51.908249Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 2 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 312 count 2 nextOffset 2 batches 1
2025-10-09T21:42:51.908257Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 0
2025-10-09T21:42:51.908261Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 3 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 452 count 3 nextOffset 3 batches 1
2025-10-09T21:42:51.908264Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 4 partNo 0
2025-10-09T21:42:51.908267Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 4 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 592 count 4 nextOffset 4 batches 1
2025-10-09T21:42:51.908270Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 5 partNo 0
2025-10-09T21:42:51.908274Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 5 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 732 count 5 nextOffset 5 batches 1
2025-10-09T21:42:51.908276Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 6 partNo 0
2025-10-09T21:42:51.908281Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 6 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 872 count 6 nextOffset 6 batches 1
2025-10-09T21:42:51.908283Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 7 partNo 0
2025-10-09T21:42:51.908286Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 7 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1012 count 7 nextOffset 7 batches 1
2025-10-09T21:42:51.908289Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 8 partNo 0
2025-10-09T21:42:51.908292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 8 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1152 count 8 nextOffset 8 batches 1
2025-10-09T21:42:51.908294Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 9 partNo 0
2025-10-09T21:42:51.908299Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 9 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1292 count 9 nextOffset 9 batches 1
2025-10-09T21:42:51.908302Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 10 partNo 0
2025-10-09T21:42:51.908306Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 10 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1432 count 10 nextOffset 10 batches 1
2025-10-09T21:42:51.908309Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 11 partNo 0
2025-10-09T21:42:51.908312Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 11 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1573 count 11 nextOffset 11 batches 1
2025-10-09T21:42:51.908314Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 12 partNo 0
2025-10-09T21:42:51.908318Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 12 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1714 count 12 nextOffset 12 batches 1
2025-10-09T21:42:51.908320Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 13 partNo 0
2025-10-09T21:42:51.908323Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 13 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1855 count 13 nextOffset 13 batches 1
2025-10-09T21:42:51.908326Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 14 partNo 0
2025-10-09T21:42:51.908330Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 14 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1996 count 14 nextOffset 14 batches 1
2025-10-09T21:42:51.908333Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 15 partNo 0
2025-10-09T21:42:51.908336Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 15 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2137 count 15 nextOffset 15 batches 1
2025-10-09T21:42:51.908339Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 16 partNo 0
2025-10-09T21:42:51.908342Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 16 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2278 count 16 nextOffset 16 batches 1
2025-10-09T21:42:51.908345Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 17 partNo 0
2025-10-09T21:42:51.908350Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 17 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2419 count 17 nextOffset 17 batches 1
2025-10-09T21:42:51.908352Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 18 partNo 0
2025-10-09T21:42:51.908356Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 18 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2560 count 18 nextOffset 18 batches 1
2025-10-09T21:42:51.908359Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 19 partNo 0
2025-10-09T21:42:51.908362Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 19 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2701 count 19 nextOffset 19 batches 1
2025-10-09T21:42:51.908365Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 20 partNo 0
2025-10-09T21:42:51.908368Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 20 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2842 count 20 nextOffset 20 batches 1
2025-10-09T21:42:51.908370Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 21 partNo 0
2025-10-09T21:42:51.908374Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 21 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2983 count 21 nextOffset 21 batches 1
2025-10-09T21:42:51.908377Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 22 partNo 0
2025-10-09T21:42:51.908381Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 22 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3124 count 22 nextOffset 22 batches 1
2025-10-09T21:42:51.908384Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 23 partNo 0
2025-10-09T21:42:51.908386Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 23 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3265 count 23 nextOffset 23 batches 1
2025-10-09T21:42:51.908389Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 24 partNo 0
2025-10-09T21:42:51.908392Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 24 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3406 count 24 nextOffset 24 batches 1
2025-10-09T21:42:51.908395Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 25 partNo 0
2025-10-09T21:42:51.908398Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 25 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3547 count 25 nextOffset 25 batches 1
2025-10-09T21:42:51.908401Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 26 partNo 0
2025-10-09T21:42:51.908405Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 26 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3688 count 26 nextOffset 26 batches 1
2025-10-09T21:42:51.908408Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 27 partNo 0
2025-10-09T21:42:51.908412Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 27 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3829 count 27 nextOffset 27 batches 1
2025-10-09T21:42:51.908414Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 28 partNo 0
2025-10-09T21:42:51.908418Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 28 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3970 count 28 nextOffset 28 batches 1
2025-10-09T21:42:51.908421Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 29 partNo 0
2025-10-09T21:42:51.908424Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 29 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4111 count 29 nextOffset 29 batches 1
2025-10-09T21:42:51.908426Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 30 partNo 0
2025-10-09T21:42:51.908429Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 30 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4252 count 30 nextOffset 30 batches 1
2025-10-09T21:42:51.908432Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 31 partNo 0
2025-10-09T21:42:51.908435Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 31 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4393 count 31 nextOffset 31 batches 1
2025-10-09T21:42:51.908438Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 32 partNo 0
2025-10-09T21:42:51.908442Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 32 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4534 count 32 nextOffset 32 batches 1
2025-10-09T21:42:51.908445Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 33 partNo 0
2025-10-09T21:42:51.908451Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 33 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4675 count 33 nextOffset 33 batches 1
2025-10-09T21:42:51.908454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.908518Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 0,33 HeadOffset 0 endOffset 0 curOffset 33 d0000000000_00000000000000000000_00000_0000000033_00000? size 3646 WTime 1760046171908
2025-10-09T21:42:51.908539Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.908597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.908603Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:51.908615Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 33 size 3646
2025-10-09T21:42:51.909645Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|e5647110-687d8388-30f6bdc1-39e93807_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:51.909738Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:51.910164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.910175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.910321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 4  requestId: <none> cookie: 3
2025-10-09T21:42:51.910763Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.911148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:51.911155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:51.911172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 100 partNo : 0 messageNo: 5 size 99 offset: -1
2025-10-09T21:42:51.911218Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:51.911228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.911240Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 33 size 3646 actorID [2:7559340745917575896:2384]
2025-10-09T21:42:51.911251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:51.911269Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 3488 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:51.911276Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911283Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 1, partNo: 0, Offset: 0 is stored on disk
2025-10-09T21:42:51.911289Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 2, partNo: 0, Offset: 1 is stored on disk
2025-10-09T21:42:51.911294Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911298Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 3, partNo: 0, Offset: 2 is stored on disk
2025-10-09T21:42:51.911300Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911304Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 4, partNo: 0, Offset: 3 is stored on disk
2025-10-09T21:42:51.911306Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911309Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 5, partNo: 0, Offset: 4 is stored on disk
2025-10-09T21:42:51.911311Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911315Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 6, partNo: 0, Offset: 5 is stored on disk
2025-10-09T21:42:51.911319Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911323Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 7, partNo: 0, Offset: 6 is stored on disk
2025-10-09T21:42:51.911324Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911328Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 8, partNo: 0, Offset: 7 is stored on disk
2025-10-09T21:42:51.911330Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911335Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 9, partNo: 0, Offset: 8 is stored on disk
2025-10-09T21:42:51.911337Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911340Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 10, partNo: 0, Offset: 9 is stored on disk
2025-10-09T21:42:51.911343Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911346Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 11, partNo: 0, Offset: 10 is stored on disk
2025-10-09T21:42:51.911347Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911351Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 12, partNo: 0, Offset: 11 is stored on disk
2025-10-09T21:42:51.911353Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911356Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 13, partNo: 0, Offset: 12 is stored on disk
2025-10-09T21:42:51.911358Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911361Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 14, partNo: 0, Offset: 13 is stored on disk
2025-10-09T21:42:51.911363Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911366Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 15, partNo: 0, Offset: 14 is stored on disk
2025-10-09T21:42:51.911367Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911370Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 16, partNo: 0, Offset: 15 is stored on disk
2025-10-09T21:42:51.911372Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911375Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 17, partNo: 0, Offset: 16 is stored on disk
2025-10-09T21:42:51.911377Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911380Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 18, partNo: 0, Offset: 17 is stored on disk
2025-10-09T21:42:51.911382Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911385Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 19, partNo: 0, Offset: 18 is stored on disk
2025-10-09T21:42:51.911388Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911391Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 20, partNo: 0, Offset: 19 is stored on disk
2025-10-09T21:42:51.911393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911395Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 21, partNo: 0, Offset: 20 is stored on disk
2025-10-09T21:42:51.911397Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911401Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 22, partNo: 0, Offset: 21 is stored on disk
2025-10-09T21:42:51.911403Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911407Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 23, partNo: 0, Offset: 22 is stored on disk
2025-10-09T21:42:51.911409Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911412Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 24, partNo: 0, Offset: 23 is stored on disk
2025-10-09T21:42:51.911413Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911416Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 25, partNo: 0, Offset: 24 is stored on disk
2025-10-09T21:42:51.911417Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911421Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 26, partNo: 0, Offset: 25 is stored on disk
2025-10-09T21:42:51.911422Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911426Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 27, partNo: 0, Offset: 26 is stored on disk
2025-10-09T21:42:51.911427Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911432Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 28, partNo: 0, Offset: 27 is stored on disk
2025-10-09T21:42:51.911434Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911437Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 29, partNo: 0, Offset: 28 is stored on disk
2025-10-09T21:42:51.911439Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911441Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 30, partNo: 0, Offset: 29 is stored on disk
2025-10-09T21:42:51.911443Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911445Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 31, partNo: 0, Offset: 30 is stored on disk
2025-10-09T21:42:51.911448Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911450Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 32, partNo: 0, Offset: 31 is stored on disk
2025-10-09T21:42:51.911452Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.911455Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 33, partNo: 0, Offset: 32 is stored on disk
2025-10-09T21:42:51.911483Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:51.911489Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:51.911493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.911525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.911528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (67)
2025-10-09T21:42:51.911544Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 34 partNo 0
2025-10-09T21:42:51.911574Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 34 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 173 count 1 nextOffset 34 batches 1
2025-10-09T21:42:51.911579Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 35 partNo 0
2025-10-09T21:42:51.911583Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 35 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 314 count 2 nextOffset 35 batches 1
2025-10-09T21:42:51.911586Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 36 partNo 0
2025-10-09T21:42:51.911591Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 36 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 455 count 3 nextOffset 36 batches 1
2025-10-09T21:42:51.911593Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 37 partNo 0
2025-10-09T21:42:51.911597Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 37 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 596 count 4 nextOffset 37 batches 1
2025-10-09T21:42:51.911600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 38 partNo 0
2025-10-09T21:42:51.911603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 38 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 737 count 5 nextOffset 38 batches 1
2025-10-09T21:42:51.911606Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 39 partNo 0
2025-10-09T21:42:51.911610Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 39 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 878 count 6 nextOffset 39 batches 1
2025-10-09T21:42:51.911612Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 40 partNo 0
2025-10-09T21:42:51.911615Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 40 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 1019 count 7 nextOffset 40 batches 1
2025-10-09T21:42:51.911618Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 41 partNo 0
2025-10-09T21:42:51.911622Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 41 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 1160 count 8 nextOffset 41 batches 1
2025-10-09T21:42:51.911625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 42 partNo 0
2025-10-09T21:42:51.911630Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 42 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 1301 count 9 nextOffset 42 batches 1
2025-10-09T21:42:51.911632Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 43 partNo 0
2025-10-09T21:42:51.911636Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 43 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 1442 count 10 nextOffset 43 batches 1
2025-10-09T21:42:51.911638Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 44 partNo 0
2025-10-09T21:42:51.911641Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 44 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 1583 count 11 nextOffset 44 batches 1
2025-10-09T21:42:51.911644Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 45 partNo 0
2025-10-09T21:42:51.911647Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 45 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 1724 count 12 nextOffset 45 batches 1
2025-10-09T21:42:51.911650Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 46 partNo 0
2025-10-09T21:42:51.911653Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 46 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 1865 count 13 nextOffset 46 batches 1
2025-10-09T21:42:51.911656Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 47 partNo 0
2025-10-09T21:42:51.911660Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 47 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 2006 count 14 nextOffset 47 batches 1
2025-10-09T21:42:51.911662Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 48 partNo 0
2025-10-09T21:42:51.911666Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 48 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 2147 count 15 nextOffset 48 batches 1
2025-10-09T21:42:51.911669Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 49 partNo 0
2025-10-09T21:42:51.911672Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 49 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 2288 count 16 nextOffset 49 batches 1
2025-10-09T21:42:51.911675Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 50 partNo 0
2025-10-09T21:42:51.911679Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 50 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 2429 count 17 nextOffset 50 batches 1
2025-10-09T21:42:51.911682Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 51 partNo 0
2025-10-09T21:42:51.911685Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 51 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 2570 count 18 nextOffset 51 batches 1
2025-10-09T21:42:51.911688Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 52 partNo 0
2025-10-09T21:42:51.911692Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 52 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 2711 count 19 nextOffset 52 batches 1
2025-10-09T21:42:51.911694Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 53 partNo 0
2025-10-09T21:42:51.911698Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 53 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 2852 count 20 nextOffset 53 batches 1
2025-10-09T21:42:51.911700Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 54 partNo 0
2025-10-09T21:42:51.911703Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 54 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 2993 count 21 nextOffset 54 batches 1
2025-10-09T21:42:51.911706Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 55 partNo 0
2025-10-09T21:42:51.911709Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 55 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 3134 count 22 nextOffset 55 batches 1
2025-10-09T21:42:51.911711Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 56 partNo 0
2025-10-09T21:42:51.911714Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 56 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 3275 count 23 nextOffset 56 batches 1
2025-10-09T21:42:51.911717Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 57 partNo 0
2025-10-09T21:42:51.911720Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 57 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 3416 count 24 nextOffset 57 batches 1
2025-10-09T21:42:51.911723Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 58 partNo 0
2025-10-09T21:42:51.911725Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 58 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 3557 count 25 nextOffset 58 batches 1
2025-10-09T21:42:51.911728Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 59 partNo 0
2025-10-09T21:42:51.911732Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 59 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 3698 count 26 nextOffset 59 batches 1
2025-10-09T21:42:51.911734Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 60 partNo 0
2025-10-09T21:42:51.911737Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 60 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 3839 count 27 nextOffset 60 batches 1
2025-10-09T21:42:51.911740Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 61 partNo 0
2025-10-09T21:42:51.911743Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 61 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 3980 count 28 nextOffset 61 batches 1
2025-10-09T21:42:51.911746Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 62 partNo 0
2025-10-09T21:42:51.911748Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 62 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 4121 count 29 nextOffset 62 batches 1
2025-10-09T21:42:51.911751Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 63 partNo 0
2025-10-09T21:42:51.911755Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 63 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 4262 count 30 nextOffset 63 batches 1
2025-10-09T21:42:51.911757Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 64 partNo 0
2025-10-09T21:42:51.911760Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 64 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 4403 count 31 nextOffset 64 batches 1
2025-10-09T21:42:51.911762Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 65 partNo 0
2025-10-09T21:42:51.911766Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 65 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 4544 count 32 nextOffset 65 batches 1
2025-10-09T21:42:51.911769Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 66 partNo 0
2025-10-09T21:42:51.911774Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 66 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 4685 count 33 nextOffset 66 batches 1
2025-10-09T21:42:51.911777Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 67 partNo 0
2025-10-09T21:42:51.911780Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 67 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 4826 count 34 nextOffset 67 batches 1
2025-10-09T21:42:51.911783Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 68 partNo 0
2025-10-09T21:42:51.911787Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 68 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 4967 count 35 nextOffset 68 batches 1
2025-10-09T21:42:51.911790Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 69 partNo 0
2025-10-09T21:42:51.911794Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 69 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 5108 count 36 nextOffset 69 batches 1
2025-10-09T21:42:51.911796Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 70 partNo 0
2025-10-09T21:42:51.911799Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 70 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 5249 count 37 nextOffset 70 batches 1
2025-10-09T21:42:51.911802Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 71 partNo 0
2025-10-09T21:42:51.911806Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 71 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 5390 count 38 nextOffset 71 batches 1
2025-10-09T21:42:51.911808Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 72 partNo 0
2025-10-09T21:42:51.911812Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 72 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 5531 count 39 nextOffset 72 batches 1
2025-10-09T21:42:51.911815Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 73 partNo 0
2025-10-09T21:42:51.911818Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 73 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 5672 count 40 nextOffset 73 batches 1
2025-10-09T21:42:51.911821Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 74 partNo 0
2025-10-09T21:42:51.911824Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 74 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 5813 count 41 nextOffset 74 batches 1
2025-10-09T21:42:51.911827Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 75 partNo 0
2025-10-09T21:42:51.911830Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 75 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 5954 count 42 nextOffset 75 batches 1
2025-10-09T21:42:51.911833Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 76 partNo 0
2025-10-09T21:42:51.911836Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 76 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 6095 count 43 nextOffset 76 batches 1
2025-10-09T21:42:51.911838Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 77 partNo 0
2025-10-09T21:42:51.911842Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 77 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 6236 count 44 nextOffset 77 batches 1
2025-10-09T21:42:51.911845Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 78 partNo 0
2025-10-09T21:42:51.911849Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 78 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 6377 count 45 nextOffset 78 batches 1
2025-10-09T21:42:51.911852Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 79 partNo 0
2025-10-09T21:42:51.911855Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 79 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 6518 count 46 nextOffset 79 batches 1
2025-10-09T21:42:51.911858Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 80 partNo 0
2025-10-09T21:42:51.911862Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 80 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 6659 count 47 nextOffset 80 batches 1
2025-10-09T21:42:51.911866Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 81 partNo 0
2025-10-09T21:42:51.911869Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 81 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 6800 count 48 nextOffset 81 batches 1
2025-10-09T21:42:51.911872Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 82 partNo 0
2025-10-09T21:42:51.911874Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 82 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 6941 count 49 nextOffset 82 batches 1
2025-10-09T21:42:51.911877Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 83 partNo 0
2025-10-09T21:42:51.911881Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 83 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 7082 count 50 nextOffset 83 batches 1
2025-10-09T21:42:51.911884Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 84 partNo 0
2025-10-09T21:42:51.911889Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 84 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 7223 count 51 nextOffset 84 batches 1
2025-10-09T21:42:51.911892Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 85 partNo 0
2025-10-09T21:42:51.911896Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 85 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 7364 count 52 nextOffset 85 batches 1
2025-10-09T21:42:51.911898Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 86 partNo 0
2025-10-09T21:42:51.911902Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 86 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 7505 count 53 nextOffset 86 batches 1
2025-10-09T21:42:51.911905Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 87 partNo 0
2025-10-09T21:42:51.911908Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 87 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 7646 count 54 nextOffset 87 batches 1
2025-10-09T21:42:51.911911Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 88 partNo 0
2025-10-09T21:42:51.911915Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 88 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 7787 count 55 nextOffset 88 batches 1
2025-10-09T21:42:51.911918Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 89 partNo 0
2025-10-09T21:42:51.911921Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 89 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 7928 count 56 nextOffset 89 batches 1
2025-10-09T21:42:51.911924Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 90 partNo 0
2025-10-09T21:42:51.911927Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 90 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 8069 count 57 nextOffset 90 batches 1
2025-10-09T21:42:51.911930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 91 partNo 0
2025-10-09T21:42:51.911933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 91 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 8210 count 58 nextOffset 91 batches 1
2025-10-09T21:42:51.911935Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 92 partNo 0
2025-10-09T21:42:51.911938Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 92 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 8351 count 59 nextOffset 92 batches 1
2025-10-09T21:42:51.911941Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 93 partNo 0
2025-10-09T21:42:51.911944Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 93 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 8492 count 60 nextOffset 93 batches 1
2025-10-09T21:42:51.911947Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 94 partNo 0
2025-10-09T21:42:51.911951Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 94 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 8633 count 61 nextOffset 94 batches 1
2025-10-09T21:42:51.911953Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 95 partNo 0
2025-10-09T21:42:51.911957Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 95 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 8774 count 62 nextOffset 95 batches 1
2025-10-09T21:42:51.911960Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 96 partNo 0
2025-10-09T21:42:51.911963Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 96 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 8915 count 63 nextOffset 96 batches 1
2025-10-09T21:42:51.911967Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 97 partNo 0
2025-10-09T21:42:51.911983Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 97 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 9056 count 64 nextOffset 97 batches 1
2025-10-09T21:42:51.911986Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 98 partNo 0
2025-10-09T21:42:51.911993Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 98 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 9197 count 65 nextOffset 98 batches 1
2025-10-09T21:42:51.911996Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 99 partNo 0
2025-10-09T21:42:51.911999Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 99 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 9338 count 66 nextOffset 99 batches 1
2025-10-09T21:42:51.912002Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 100 partNo 0
2025-10-09T21:42:51.912006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 100 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 9479 count 67 nextOffset 100 batches 1
2025-10-09T21:42:51.912009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.912078Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 33,67 HeadOffset 33 endOffset 33 curOffset 100 d0000000000_00000000000000000033_00000_0000000067_00000? size 7328 WTime 1760046171911
2025-10-09T21:42:51.912096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:51.912140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 2  requestId: <none> cookie: 1
2025-10-09T21:42:51.912156Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:51.912165Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 33 partNo 0 count 67 size 7328
2025-10-09T21:42:51.912213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 33 max time lag 0ms effective offset 0
2025-10-09T21:42:51.912225Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 3646 count 33 last offset 0, current partition end offset: 33
2025-10-09T21:42:51.912228Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:51.912243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 33 parts_count 0 source 1 size 3646 accessed 0 times before, last time 2025-10-09T21:42:51.000000Z
2025-10-09T21:42:51.912246Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 3. All 1 blobs are from cache.
2025-10-09T21:42:51.912254Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:51.912336Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 33 count 33 size 3626 from pos 0 cbcount 33
2025-10-09T21:42:51.912351Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046171908 queuesize 0 startOffset 0
2025-10-09T21:42:51.912410Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 33 parts 0 suffix '63' size 3646
2025-10-09T21:42:51.912419Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 33 parts 0 suffix '63'
2025-10-09T21:42:51.913628Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.914079Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: OnReadDone gRpcStatusCode: 0
2025-10-09T21:42:51.914289Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session got write response: acks { seq_no: 1 written { } } acks { seq_no: 2 written { offset: 1 } } acks { seq_no: 3 written { offset: 2 } } acks { seq_no: 4 written { offset: 3 } } acks { seq_no: 5 written { offset: 4 } } acks { seq_no: 6 written { offset: 5 } } acks { seq_no: 7 written { offset: 6 } } acks { seq_no: 8 written { offset: 7 } } acks { seq_no: 9 written { offset: 8 } } acks { seq_no: 10 written { offset: 9 } } acks { seq_no: 11 written { offset: 10 } } acks { seq_no: 12 written { offset: 11 } } acks { seq_no: 13 written { offset: 12 } } acks { seq_no: 14 written { offset: 13 } } acks { seq_no: 15 written { offset: 14 } } acks { seq_no: 16 written { offset: 15 } } acks { seq_no: 17 written { offset: 16 } } acks { seq_no: 18 written { offset: 17 } } acks { seq_no: 19 written { offset: 18 } } acks { seq_no: 20 written { offset: 19 } } acks { seq_no: 21 written { offset: 20 } } acks { seq_no: 22 written { offset: 21 } } acks { seq_no: 23 written { offset: 22 } } acks { seq_no: 24 written { offset: 23 } } acks { seq_no: 25 written { offset: 24 } } acks { seq_no: 26 written { offset: 25 } } acks { seq_no: 27 written { offset: 26 } } acks { seq_no: 28 written { offset: 27 } } acks { seq_no: 29 written { offset: 28 } } acks { seq_no: 30 written { offset: 29 } } acks { seq_no: 31 written { offset: 30 } } acks { seq_no: 32 written { offset: 31 } } acks { seq_no: 33 written { offset: 32 } } write_statistics { persisting_time { nanos: 3000000 } min_queue_wait_time { } max_queue_wait_time { } partition_quota_wait_time { } topic_quota_wait_time { } }
2025-10-09T21:42:51.914298Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=1, txId=?
2025-10-09T21:42:51.914303Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 1
2025-10-09T21:42:51.914308Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=2, txId=?
2025-10-09T21:42:51.914311Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 2
2025-10-09T21:42:51.914314Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=3, txId=?
2025-10-09T21:42:51.914317Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 3
2025-10-09T21:42:51.914321Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=4, txId=?
2025-10-09T21:42:51.914323Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 4
2025-10-09T21:42:51.914326Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=5, txId=?
2025-10-09T21:42:51.914328Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 5
2025-10-09T21:42:51.914331Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=6, txId=?
2025-10-09T21:42:51.914333Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 6
2025-10-09T21:42:51.914335Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=7, txId=?
2025-10-09T21:42:51.914338Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 7
2025-10-09T21:42:51.914341Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=8, txId=?
2025-10-09T21:42:51.914345Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 8
2025-10-09T21:42:51.914348Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=9, txId=?
2025-10-09T21:42:51.914351Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 9
2025-10-09T21:42:51.914354Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=10, txId=?
2025-10-09T21:42:51.914356Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 10
2025-10-09T21:42:51.914358Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=11, txId=?
2025-10-09T21:42:51.914360Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 11
2025-10-09T21:42:51.914363Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=12, txId=?
2025-10-09T21:42:51.914365Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 12
2025-10-09T21:42:51.914368Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=13, txId=?
2025-10-09T21:42:51.914370Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 13
2025-10-09T21:42:51.914374Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=14, txId=?
2025-10-09T21:42:51.914376Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 14
2025-10-09T21:42:51.914379Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=15, txId=?
2025-10-09T21:42:51.914381Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 15
2025-10-09T21:42:51.914384Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=16, txId=?
2025-10-09T21:42:51.914386Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 16
2025-10-09T21:42:51.914389Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=17, txId=?
2025-10-09T21:42:51.914391Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 17
2025-10-09T21:42:51.914394Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=18, txId=?
2025-10-09T21:42:51.914396Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 18
2025-10-09T21:42:51.914399Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=19, txId=?
2025-10-09T21:42:51.914401Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 19
2025-10-09T21:42:51.914404Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=20, txId=?
2025-10-09T21:42:51.914407Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 20
2025-10-09T21:42:51.914410Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=21, txId=?
2025-10-09T21:42:51.914430Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 21
2025-10-09T21:42:51.914433Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=22, txId=?
2025-10-09T21:42:51.914435Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 22
2025-10-09T21:42:51.914438Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=23, txId=?
2025-10-09T21:42:51.914441Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 23
2025-10-09T21:42:51.914443Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=24, txId=?
2025-10-09T21:42:51.914445Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 24
2025-10-09T21:42:51.914448Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=25, txId=?
2025-10-09T21:42:51.914450Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 25
2025-10-09T21:42:51.914453Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=26, txId=?
2025-10-09T21:42:51.914455Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 26
2025-10-09T21:42:51.914457Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=27, txId=?
2025-10-09T21:42:51.914460Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 27
2025-10-09T21:42:51.914463Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=28, txId=?
2025-10-09T21:42:51.914465Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 28
2025-10-09T21:42:51.914468Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=29, txId=?
2025-10-09T21:42:51.914470Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 29
2025-10-09T21:42:51.914473Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=30, txId=?
2025-10-09T21:42:51.914475Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 30
2025-10-09T21:42:51.914478Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=31, txId=?
2025-10-09T21:42:51.914480Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 31
2025-10-09T21:42:51.914483Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=32, txId=?
2025-10-09T21:42:51.914490Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 32
2025-10-09T21:42:51.914492Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=33, txId=?
2025-10-09T21:42:51.914495Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 33
2025-10-09T21:42:51.921971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2299: [72075186224037892][Partition][0][StateIdle] Writing. Can't process transactions and user actions
2025-10-09T21:42:51.922929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 33 count 67 size 7328 actorID [2:7559340745917575896:2384]
2025-10-09T21:42:51.922947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:51.922964Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 7102 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:51.922974Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.922985Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 34, partNo: 0, Offset: 33 is stored on disk
2025-10-09T21:42:51.922993Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.922998Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 35, partNo: 0, Offset: 34 is stored on disk
2025-10-09T21:42:51.923000Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923005Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 36, partNo: 0, Offset: 35 is stored on disk
2025-10-09T21:42:51.923007Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923010Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 37, partNo: 0, Offset: 36 is stored on disk
2025-10-09T21:42:51.923012Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923016Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 38, partNo: 0, Offset: 37 is stored on disk
2025-10-09T21:42:51.923018Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923022Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 39, partNo: 0, Offset: 38 is stored on disk
2025-10-09T21:42:51.923023Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 40, partNo: 0, Offset: 39 is stored on disk
2025-10-09T21:42:51.923028Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 41, partNo: 0, Offset: 40 is stored on disk
2025-10-09T21:42:51.923033Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923036Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 42, partNo: 0, Offset: 41 is stored on disk
2025-10-09T21:42:51.923039Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923042Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 43, partNo: 0, Offset: 42 is stored on disk
2025-10-09T21:42:51.923044Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923047Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 44, partNo: 0, Offset: 43 is stored on disk
2025-10-09T21:42:51.923050Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923053Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 45, partNo: 0, Offset: 44 is stored on disk
2025-10-09T21:42:51.923055Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923058Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 46, partNo: 0, Offset: 45 is stored on disk
2025-10-09T21:42:51.923060Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923063Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 47, partNo: 0, Offset: 46 is stored on disk
2025-10-09T21:42:51.923065Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923069Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 48, partNo: 0, Offset: 47 is stored on disk
2025-10-09T21:42:51.923072Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923075Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 49, partNo: 0, Offset: 48 is stored on disk
2025-10-09T21:42:51.923076Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923079Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 50, partNo: 0, Offset: 49 is stored on disk
2025-10-09T21:42:51.923082Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923085Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 51, partNo: 0, Offset: 50 is stored on disk
2025-10-09T21:42:51.923087Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923090Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 52, partNo: 0, Offset: 51 is stored on disk
2025-10-09T21:42:51.923092Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923096Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 53, partNo: 0, Offset: 52 is stored on disk
2025-10-09T21:42:51.923097Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923100Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 54, partNo: 0, Offset: 53 is stored on disk
2025-10-09T21:42:51.923102Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923106Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 55, partNo: 0, Offset: 54 is stored on disk
2025-10-09T21:42:51.923108Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923111Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 56, partNo: 0, Offset: 55 is stored on disk
2025-10-09T21:42:51.923113Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923116Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 57, partNo: 0, Offset: 56 is stored on disk
2025-10-09T21:42:51.923118Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923124Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 58, partNo: 0, Offset: 57 is stored on disk
2025-10-09T21:42:51.923127Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923130Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 59, partNo: 0, Offset: 58 is stored on disk
2025-10-09T21:42:51.923132Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923136Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 60, partNo: 0, Offset: 59 is stored on disk
2025-10-09T21:42:51.923137Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923141Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 61, partNo: 0, Offset: 60 is stored on disk
2025-10-09T21:42:51.923143Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923147Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 62, partNo: 0, Offset: 61 is stored on disk
2025-10-09T21:42:51.923148Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923152Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 63, partNo: 0, Offset: 62 is stored on disk
2025-10-09T21:42:51.923153Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923156Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 64, partNo: 0, Offset: 63 is stored on disk
2025-10-09T21:42:51.923159Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923162Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 65, partNo: 0, Offset: 64 is stored on disk
2025-10-09T21:42:51.923163Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923166Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 66, partNo: 0, Offset: 65 is stored on disk
2025-10-09T21:42:51.923170Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923173Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 67, partNo: 0, Offset: 66 is stored on disk
2025-10-09T21:42:51.923175Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923178Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 68, partNo: 0, Offset: 67 is stored on disk
2025-10-09T21:42:51.923180Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923183Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 69, partNo: 0, Offset: 68 is stored on disk
2025-10-09T21:42:51.923185Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923188Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 70, partNo: 0, Offset: 69 is stored on disk
2025-10-09T21:42:51.923190Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923192Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 71, partNo: 0, Offset: 70 is stored on disk
2025-10-09T21:42:51.923194Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923197Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 72, partNo: 0, Offset: 71 is stored on disk
2025-10-09T21:42:51.923199Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 73, partNo: 0, Offset: 72 is stored on disk
2025-10-09T21:42:51.923204Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923208Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 74, partNo: 0, Offset: 73 is stored on disk
2025-10-09T21:42:51.923211Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923215Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 75, partNo: 0, Offset: 74 is stored on disk
2025-10-09T21:42:51.923217Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923219Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 76, partNo: 0, Offset: 75 is stored on disk
2025-10-09T21:42:51.923221Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923224Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 77, partNo: 0, Offset: 76 is stored on disk
2025-10-09T21:42:51.923226Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923229Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 78, partNo: 0, Offset: 77 is stored on disk
2025-10-09T21:42:51.923232Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923235Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 79, partNo: 0, Offset: 78 is stored on disk
2025-10-09T21:42:51.923236Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923241Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 80, partNo: 0, Offset: 79 is stored on disk
2025-10-09T21:42:51.923243Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923248Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 81, partNo: 0, Offset: 80 is stored on disk
2025-10-09T21:42:51.923249Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923252Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 82, partNo: 0, Offset: 81 is stored on disk
2025-10-09T21:42:51.923255Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923258Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 83, partNo: 0, Offset: 82 is stored on disk
2025-10-09T21:42:51.923260Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923262Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 84, partNo: 0, Offset: 83 is stored on disk
2025-10-09T21:42:51.923264Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923267Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 85, partNo: 0, Offset: 84 is stored on disk
2025-10-09T21:42:51.923268Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923271Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 86, partNo: 0, Offset: 85 is stored on disk
2025-10-09T21:42:51.923273Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923275Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 87, partNo: 0, Offset: 86 is stored on disk
2025-10-09T21:42:51.923277Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923280Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 88, partNo: 0, Offset: 87 is stored on disk
2025-10-09T21:42:51.923282Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923285Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 89, partNo: 0, Offset: 88 is stored on disk
2025-10-09T21:42:51.923288Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923290Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 90, partNo: 0, Offset: 89 is stored on disk
2025-10-09T21:42:51.923293Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923295Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 91, partNo: 0, Offset: 90 is stored on disk
2025-10-09T21:42:51.923297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923299Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 92, partNo: 0, Offset: 91 is stored on disk
2025-10-09T21:42:51.923301Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923304Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 93, partNo: 0, Offset: 92 is stored on disk
2025-10-09T21:42:51.923305Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923308Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 94, partNo: 0, Offset: 93 is stored on disk
2025-10-09T21:42:51.923310Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923314Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 95, partNo: 0, Offset: 94 is stored on disk
2025-10-09T21:42:51.923316Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923319Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 96, partNo: 0, Offset: 95 is stored on disk
2025-10-09T21:42:51.923321Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923323Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 97, partNo: 0, Offset: 96 is stored on disk
2025-10-09T21:42:51.923324Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923327Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 98, partNo: 0, Offset: 97 is stored on disk
2025-10-09T21:42:51.923330Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923333Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 99, partNo: 0, Offset: 98 is stored on disk
2025-10-09T21:42:51.923334Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:51.923337Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 100, partNo: 0, Offset: 99 is stored on disk
2025-10-09T21:42:51.923365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.923368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.923369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.923378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:51.923446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 3  requestId: <none> cookie: 2
2025-10-09T21:42:51.923459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 5  requestId: <none> cookie: 3
2025-10-09T21:42:51.923503Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 33 partno 0 count 67 parts 0 suffix '63' size 7328
2025-10-09T21:42:51.924010Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.924059Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:51.924510Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: OnReadDone gRpcStatusCode: 0
2025-10-09T21:42:51.924655Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session got write response: acks { seq_no: 34 written { offset: 33 } } acks { seq_no: 35 written { offset: 34 } } acks { seq_no: 36 written { offset: 35 } } acks { seq_no: 37 written { offset: 36 } } acks { seq_no: 38 written { offset: 37 } } acks { seq_no: 39 written { offset: 38 } } acks { seq_no: 40 written { offset: 39 } } acks { seq_no: 41 written { offset: 40 } } acks { seq_no: 42 written { offset: 41 } } acks { seq_no: 43 written { offset: 42 } } acks { seq_no: 44 written { offset: 43 } } acks { seq_no: 45 written { offset: 44 } } acks { seq_no: 46 written { offset: 45 } } acks { seq_no: 47 written { offset: 46 } } acks { seq_no: 48 written { offset: 47 } } acks { seq_no: 49 written { offset: 48 } } acks { seq_no: 50 written { offset: 49 } } acks { seq_no: 51 written { offset: 50 } } acks { seq_no: 52 written { offset: 51 } } acks { seq_no: 53 written { offset: 52 } } acks { seq_no: 54 written { offset: 53 } } acks { seq_no: 55 written { offset: 54 } } acks { seq_no: 56 written { offset: 55 } } acks { seq_no: 57 written { offset: 56 } } acks { seq_no: 58 written { offset: 57 } } acks { seq_no: 59 written { offset: 58 } } acks { seq_no: 60 written { offset: 59 } } acks { seq_no: 61 written { offset: 60 } } acks { seq_no: 62 written { offset: 61 } } acks { seq_no: 63 written { offset: 62 } } acks { seq_no: 64 written { offset: 63 } } acks { seq_no: 65 written { offset: 64 } } acks { seq_no: 66 written { offset: 65 } } acks { seq_no: 67 written { offset: 66 } } acks { seq_no: 68 written { offset: 67 } } acks { seq_no: 69 written { offset: 68 } } acks { seq_no: 70 written { offset: 69 } } acks { seq_no: 71 written { offset: 70 } } acks { seq_no: 72 written { offset: 71 } } acks { seq_no: 73 written { offset: 72 } } acks { seq_no: 74 written { offset: 73 } } acks { seq_no: 75 written { offset: 74 } } acks { seq_no: 76 written { offset: 75 } } acks { seq_no: 77 written { offset: 76 } } acks { seq_no: 78 written { offset: 77 } } acks { seq_no: 79 written { offset: 78 } } acks { seq_no: 80 written { offset: 79 } } acks { seq_no: 81 written { offset: 80 } } acks { seq_no: 82 written { offset: 81 } } acks { seq_no: 83 written { offset: 82 } } acks { seq_no: 84 written { offset: 83 } } acks { seq_no: 85 written { offset: 84 } } acks { seq_no: 86 written { offset: 85 } } acks { seq_no: 87 written { offset: 86 } } acks { seq_no: 88 written { offset: 87 } } acks { seq_no: 89 written { offset: 88 } } acks { seq_no: 90 written { offset: 89 } } acks { seq_no: 91 written { offset: 90 } } acks { seq_no: 92 written { offset: 91 } } acks { seq_no: 93 written { offset: 92 } } acks { seq_no: 94 written { offset: 93 } } acks { seq_no: 95 written { offset: 94 } } acks { seq_no: 96 written { offset: 95 } } acks { seq_no: 97 written { offset: 96 } } acks { seq_no: 98 written { offset: 97 } } acks { seq_no: 99 written { offset: 98 } } write_statistics { persisting_time { nanos: 11000000 } min_queue_wait_time { nanos: 3000000 } max_queue_wait_time { nanos: 3000000 } partition_quota_wait_time { } topic_quota_wait_time { } }
2025-10-09T21:42:51.924662Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=34, txId=?
2025-10-09T21:42:51.924668Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 34
2025-10-09T21:42:51.924672Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=35, txId=?
2025-10-09T21:42:51.924674Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 35
2025-10-09T21:42:51.924677Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=36, txId=?
2025-10-09T21:42:51.924679Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 36
2025-10-09T21:42:51.924683Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=37, txId=?
2025-10-09T21:42:51.924685Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 37
2025-10-09T21:42:51.924689Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=38, txId=?
2025-10-09T21:42:51.924691Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 38
2025-10-09T21:42:51.924694Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=39, txId=?
2025-10-09T21:42:51.924695Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 39
2025-10-09T21:42:51.924698Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=40, txId=?
2025-10-09T21:42:51.924700Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 40
2025-10-09T21:42:51.924703Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=41, txId=?
2025-10-09T21:42:51.924705Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 41
2025-10-09T21:42:51.924707Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=42, txId=?
2025-10-09T21:42:51.924710Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 42
2025-10-09T21:42:51.924712Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=43, txId=?
2025-10-09T21:42:51.924714Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 43
2025-10-09T21:42:51.924716Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=44, txId=?
2025-10-09T21:42:51.924718Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 44
2025-10-09T21:42:51.924721Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=45, txId=?
2025-10-09T21:42:51.924723Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 45
2025-10-09T21:42:51.924726Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=46, txId=?
2025-10-09T21:42:51.924728Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 46
2025-10-09T21:42:51.924730Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=47, txId=?
2025-10-09T21:42:51.924732Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 47
2025-10-09T21:42:51.924734Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=48, txId=?
2025-10-09T21:42:51.924736Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 48
2025-10-09T21:42:51.924740Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=49, txId=?
2025-10-09T21:42:51.924742Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 49
2025-10-09T21:42:51.924745Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=50, txId=?
2025-10-09T21:42:51.924747Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 50
2025-10-09T21:42:51.924749Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=51, txId=?
2025-10-09T21:42:51.924751Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 51
2025-10-09T21:42:51.924754Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=52, txId=?
2025-10-09T21:42:51.924756Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 52
2025-10-09T21:42:51.924758Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=53, txId=?
2025-10-09T21:42:51.924761Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 53
2025-10-09T21:42:51.924764Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=54, txId=?
2025-10-09T21:42:51.924766Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 54
2025-10-09T21:42:51.924769Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=55, txId=?
2025-10-09T21:42:51.924771Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 55
2025-10-09T21:42:51.924773Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=56, txId=?
2025-10-09T21:42:51.924775Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 56
2025-10-09T21:42:51.924777Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=57, txId=?
2025-10-09T21:42:51.924779Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 57
2025-10-09T21:42:51.924782Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=58, txId=?
2025-10-09T21:42:51.924784Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 58
2025-10-09T21:42:51.924786Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=59, txId=?
2025-10-09T21:42:51.924788Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 59
2025-10-09T21:42:51.924791Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=60, txId=?
2025-10-09T21:42:51.924793Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 60
2025-10-09T21:42:51.924797Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=61, txId=?
2025-10-09T21:42:51.924799Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 61
2025-10-09T21:42:51.924802Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=62, txId=?
2025-10-09T21:42:51.924804Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 62
2025-10-09T21:42:51.924806Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=63, txId=?
2025-10-09T21:42:51.924808Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 63
2025-10-09T21:42:51.924811Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=64, txId=?
2025-10-09T21:42:51.924813Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 64
2025-10-09T21:42:51.924816Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=65, txId=?
2025-10-09T21:42:51.924818Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 65
2025-10-09T21:42:51.924821Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=66, txId=?
2025-10-09T21:42:51.924824Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 66
2025-10-09T21:42:51.924826Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=67, txId=?
2025-10-09T21:42:51.924828Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 67
2025-10-09T21:42:51.924831Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=68, txId=?
2025-10-09T21:42:51.924833Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 68
2025-10-09T21:42:51.924835Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=69, txId=?
2025-10-09T21:42:51.924837Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 69
2025-10-09T21:42:51.924840Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=70, txId=?
2025-10-09T21:42:51.924841Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 70
2025-10-09T21:42:51.924844Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=71, txId=?
2025-10-09T21:42:51.924846Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 71
2025-10-09T21:42:51.924849Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=72, txId=?
2025-10-09T21:42:51.924851Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 72
2025-10-09T21:42:51.924854Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=73, txId=?
2025-10-09T21:42:51.924856Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 73
2025-10-09T21:42:51.924858Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=74, txId=?
2025-10-09T21:42:51.924860Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 74
2025-10-09T21:42:51.924863Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=75, txId=?
2025-10-09T21:42:51.924865Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 75
2025-10-09T21:42:51.924868Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=76, txId=?
2025-10-09T21:42:51.924870Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 76
2025-10-09T21:42:51.924873Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=77, txId=?
2025-10-09T21:42:51.924875Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 77
2025-10-09T21:42:51.924878Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=78, txId=?
2025-10-09T21:42:51.924880Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 78
2025-10-09T21:42:51.924882Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=79, txId=?
2025-10-09T21:42:51.924884Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 79
2025-10-09T21:42:51.924887Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=80, txId=?
2025-10-09T21:42:51.924890Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 80
2025-10-09T21:42:51.924892Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=81, txId=?
2025-10-09T21:42:51.924894Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 81
2025-10-09T21:42:51.924897Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=82, txId=?
2025-10-09T21:42:51.924899Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 82
2025-10-09T21:42:51.924902Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=83, txId=?
2025-10-09T21:42:51.924904Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 83
2025-10-09T21:42:51.924906Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=84, txId=?
2025-10-09T21:42:51.924908Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 84
2025-10-09T21:42:51.924912Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=85, txId=?
2025-10-09T21:42:51.924914Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 85
2025-10-09T21:42:51.924917Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=86, txId=?
2025-10-09T21:42:51.924919Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 86
2025-10-09T21:42:51.924921Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=87, txId=?
2025-10-09T21:42:51.924923Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 87
2025-10-09T21:42:51.924926Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=88, txId=?
2025-10-09T21:42:51.924928Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 88
2025-10-09T21:42:51.924930Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=89, txId=?
2025-10-09T21:42:51.924932Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 89
2025-10-09T21:42:51.924935Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=90, txId=?
2025-10-09T21:42:51.924937Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 90
2025-10-09T21:42:51.924940Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=91, txId=?
2025-10-09T21:42:51.924942Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 91
2025-10-09T21:42:51.924944Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=92, txId=?
2025-10-09T21:42:51.924946Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 92
2025-10-09T21:42:51.924949Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=93, txId=?
2025-10-09T21:42:51.924951Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 93
2025-10-09T21:42:51.924953Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=94, txId=?
2025-10-09T21:42:51.924955Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 94
2025-10-09T21:42:51.924957Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=95, txId=?
2025-10-09T21:42:51.924970Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 95
2025-10-09T21:42:51.924973Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=96, txId=?
2025-10-09T21:42:51.924975Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 96
2025-10-09T21:42:51.924978Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=97, txId=?
2025-10-09T21:42:51.924980Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 97
2025-10-09T21:42:51.924983Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=98, txId=?
2025-10-09T21:42:51.924988Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 98
2025-10-09T21:42:51.924991Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=99, txId=?
2025-10-09T21:42:51.924993Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 99
2025-10-09T21:42:51.925049Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: OnReadDone gRpcStatusCode: 0
2025-10-09T21:42:51.925059Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session got write response: acks { seq_no: 100 written { offset: 99 } } write_statistics { persisting_time { nanos: 10000000 } min_queue_wait_time { } max_queue_wait_time { } partition_quota_wait_time { } topic_quota_wait_time { } }
2025-10-09T21:42:51.925062Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] OnAck: seqNo=100, txId=?
2025-10-09T21:42:51.925064Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: acknoledged message 100
2025-10-09T21:42:51.928691Z :INFO: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: close. Timeout 0.000000s
2025-10-09T21:42:51.928705Z :INFO: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session will now close
2025-10-09T21:42:51.928714Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: aborting
2025-10-09T21:42:51.928852Z :INFO: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: gracefully shut down, all writes complete
2025-10-09T21:42:51.928858Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|e5647110-687d8388-30f6bdc1-39e93807_0]  MessageGroupId [src_id] Write session: destroy
2025-10-09T21:42:51.929394Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|e5647110-687d8388-30f6bdc1-39e93807_0 grpc read done: success: 0 data: 
2025-10-09T21:42:51.929409Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 2 sessionId: src_id|e5647110-687d8388-30f6bdc1-39e93807_0 grpc read failed
2025-10-09T21:42:51.929432Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 2 sessionId: src_id|e5647110-687d8388-30f6bdc1-39e93807_0 grpc closed
2025-10-09T21:42:51.929438Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2 sessionId: src_id|e5647110-687d8388-30f6bdc1-39e93807_0 is DEAD
2025-10-09T21:42:51.929657Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:42:51.930398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340747513918097:2459] destroyed
2025-10-09T21:42:51.930435Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:42:51.930444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.930446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.930448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.022340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.022355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.022358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.126101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:52.126120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:52.126123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:52.476516Z :WriteSessionCloseIgnoresWrites INFO: Random seed for debugging is 1760046172476508
2025-10-09T21:42:52.592809Z node 4 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[4:7559340751867654429:2155];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:52.592867Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/owmb/0001b4/r3tmp/tmpKDtm0q/pdisk_1.dat
2025-10-09T21:42:52.595019Z node 4 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:52.598758Z node 3 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:52.606841Z node 3 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:52.606887Z node 3 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
2025-10-09T21:42:52.607248Z node 4 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:52.658829Z node 3 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:42:52.659089Z node 3 :CONFIGS_DISPATCHER ERROR: configs_dispatcher.cpp:1229: Notification cookie mismatch for subscription [3:7559340748607953660:2082] 1760046172590701 != 1760046172590704
2025-10-09T21:42:52.667115Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:52.667144Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:52.677061Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:52.677089Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:52.678295Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:52.679103Z node 3 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 4 Cookie 4
2025-10-09T21:42:52.679712Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Connecting -> Connected
TServer::EnableGrpc on GrpcPort 18912, node 3
2025-10-09T21:42:52.690527Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/owmb/0001b4/r3tmp/yandexEKl8C6.tmp
2025-10-09T21:42:52.690538Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/owmb/0001b4/r3tmp/yandexEKl8C6.tmp
2025-10-09T21:42:52.690590Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/owmb/0001b4/r3tmp/yandexEKl8C6.tmp
2025-10-09T21:42:52.690609Z node 3 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-10-09T21:42:52.694588Z INFO: TTestServer started on Port 2022 GrpcPort 18912
TClient is connected to server localhost:2022
PQClient connected to localhost:18912
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-09T21:42:52.742668Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976715657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
2025-10-09T21:42:52.754299Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976715657, at schemeshard: 72057594046644480
2025-10-09T21:42:52.818656Z node 3 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
waiting...
2025-10-09T21:42:52.830537Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976715658, at schemeshard: 72057594046644480
waiting...
2025-10-09T21:42:52.845533Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976715659, at schemeshard: 72057594046644480
waiting...
2025-10-09T21:42:52.855173Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976715660, at schemeshard: 72057594046644480
2025-10-09T21:42:52.882581Z node 4 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:53.080176Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340752902921951:2325], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:53.080222Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:53.081366Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340752902921986:2328], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:53.081394Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340752902921987:2329], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:53.081428Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:53.083779Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976715661:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-10-09T21:42:53.092211Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976715662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:53.095844Z node 3 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [3:7559340752902921990:2330], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976715661 completed, doublechecking }
2025-10-09T21:42:53.100774Z node 4 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [4:7559340756162621958:2301], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:53.101148Z node 4 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=4&id=Y2MzMWI1ZjYtOTY2M2E2ZGMtZThhMDQ0ZGYtZWJmZWY1ZmE=, ActorId: [4:7559340756162621915:2292], ActorState: ExecuteState, TraceId: 01k75etjx27by4sc12hvxkp9ac, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:53.101318Z node 4 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:53.165528Z node 3 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [3:7559340752902922136:2691] txid# 281474976715663, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:53.174649Z node 3 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [3:7559340752902922155:2341], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:5:17: Error: At function: KiReadTable!
        <main>:5:17: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Versions]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:53.175600Z node 3 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=3&id=ZTY4M2JhZDItZGQxMTU2M2QtN2ZkM2UwMjYtMjc1Y2EyMDY=, ActorId: [3:7559340752902921947:2322], ActorState: ExecuteState, TraceId: 01k75etjwp37pghcmy5rayj575, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:53.175756Z node 3 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:53.181062Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976715664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:53.235000Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976715665:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:18912", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, false, 1000);
2025-10-09T21:42:53.357615Z node 3 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976715666. Ctx: { TraceId: 01k75etk473wd9xbh8hc0jkzgv, Database: , SessionId: ydb://session/3?node_id=3&id=MjY3MWE4Y2YtMTZkODdkOTUtNWJhM2FhNmEtNzNiNzFjODU=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [3:7559340752902922469:2936] 
2025-10-09T21:42:53.592944Z node 4 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:53.596715Z node 3 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:57.592945Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[4:7559340751867654429:2155];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:57.592984Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
2025-10-09T21:42:58.475303Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976715672:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
waiting...
PQ Client: create topic: rt3.dc1--test-topic with 1 partitions
CallPersQueueGRPC request to localhost:18912
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:58.506217Z node 3 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:18912
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976715673
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-09T21:42:58.538934Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:58.539818Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:58.539891Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:784: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-09T21:42:58.539899Z node 4 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:58.539910Z node 4 :PERSQUEUE INFO: pq_impl.cpp:609: [PQ: 72075186224037892] doesn't have tx writes info
2025-10-09T21:42:58.540115Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340774377759361:3198], now have 1 active actors on pipe
2025-10-09T21:42:58.540331Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:58.540345Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340774377759395:2433], now have 1 active actors on pipe
2025-10-09T21:42:58.541828Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:58.539690Z node 3 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][] pipe [3:7559340774377759362:3199] connected; active server actors: 1
2025-10-09T21:42:58.539874Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1517: [72075186224037893][rt3.dc1--test-topic] updating configuration. Deleted partitions []. Added partitions [0]
2025-10-09T21:42:58.540027Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:940: [72075186224037893][rt3.dc1--test-topic] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-09T21:42:58.540108Z node 3 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976715674. Ctx: { TraceId: 01k75etr6sbj3wgemq28m6v1d1, Database: , SessionId: ydb://session/3?node_id=3&id=MjY4ZmY4MWEtZjJjYmRhZGYtZmJmNDMyZjAtZGE0MzllNWQ=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-10-09T21:42:58.540521Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 4, Generation 1
2025-10-09T21:42:58.540661Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:138: [72075186224037893][rt3.dc1--test-topic] BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) 
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
2025-10-09T21:42:58.550265Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340774377759435:3249], now have 1 active actors on pipe
2025-10-09T21:42:58.550362Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:58.552059Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:58.552259Z node 4 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:58.552271Z node 4 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:58.552332Z node 4 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:58.552371Z node 4 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [4:7559340777637458921:2381]
2025-10-09T21:42:58.553070Z node 4 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:58.553074Z node 4 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:58.553080Z node 4 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [4:7559340777637458921:2381]
TClient::Ls response: 2025-10-09T21:42:58.553087Z node 4 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:58.553359Z node 4 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-10-09T21:42:58.553366Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.553386Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.553388Z node 4 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:58.553393Z node 4 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:58.553395Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.553397Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.553399Z node 4 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:58.553406Z node 4 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:58.553490Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:58.553519Z node 4 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
GetTopicVersionFromPath:  record Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
 name rt3.dc1--test-topic version0
2025-10-09T21:42:58.555872Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:58.555919Z node 4 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046178597, TxId 281474976715673
2025-10-09T21:42:58.555930Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.555932Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.555961Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.556010Z node 4 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:58.556772Z node 4 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:58.556835Z node 4 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:58.556848Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.556854Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.556856Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.556861Z node 4 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:58.556968Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:395: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-10-09T21:42:58.556993Z node 4 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:58.557045Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:58.559688Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:58.559713Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:58.560563Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:58.657987Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.658001Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.658004Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.758211Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.758228Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.758231Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.862098Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.862120Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.862124Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.961979Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.962009Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.962013Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
2025-10-09T21:42:59.064588Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.064607Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.064610Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976715673
    CreateStep: 1760046178597
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976715673
    CreateStep: 1760046178597
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:18912
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:59.067548Z node 3 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
2025-10-09T21:42:59.074150Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-09T21:42:59.074398Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-09T21:42:59.074405Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:18912
2025-10-09T21:42:59.074913Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:59.075090Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:59.075107Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1
2025-10-09T21:42:59.075649Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:59.075679Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:37570
2025-10-09T21:42:59.075683Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:37570 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:59.075687Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:59.076112Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-09T21:42:59.076139Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:59.076141Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:59.076143Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:59.076148Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [3:7559340778672726799:2448] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:59.076649Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [3:7559340778672726799:2448] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:59.149395Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [3:7559340778672726799:2448] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:59.149924Z node 3 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340778672726836:2448] connected; active server actors: 1
2025-10-09T21:42:59.150012Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [3:7559340778672726799:2448] (SourceId=src, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src
2025-10-09T21:42:59.150026Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [3:7559340778672726799:2448] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:59.154085Z node 3 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340778672726836:2448] disconnected.
2025-10-09T21:42:59.154094Z node 3 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340778672726836:2448] disconnected; active server actors: 1
2025-10-09T21:42:59.154098Z node 3 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340778672726836:2448] disconnected no session
2025-10-09T21:42:59.164727Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.164750Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.164753Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.179995Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340778672726860:2448], now have 1 active actors on pipe
2025-10-09T21:42:59.180241Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:59.180257Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:59.180303Z node 4 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src|f5c60a0e-14415c48-c6f3898d-c23acbaa_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src
2025-10-09T21:42:59.180323Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.180339Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.180351Z node 4 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:59.180364Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.180383Z node 4 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:59.180417Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:59.180756Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:59.180769Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:59.180789Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:59.181356Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046179181 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:59.181402Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|f5c60a0e-14415c48-c6f3898d-c23acbaa_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:59.181512Z :INFO: [] MessageGroupId [src] SessionId [src|f5c60a0e-14415c48-c6f3898d-c23acbaa_0] Write session: close. Timeout = 0 ms
2025-10-09T21:42:59.181524Z :INFO: [] MessageGroupId [src] SessionId [src|f5c60a0e-14415c48-c6f3898d-c23acbaa_0] Write session will now close
2025-10-09T21:42:59.181531Z :DEBUG: [] MessageGroupId [src] SessionId [src|f5c60a0e-14415c48-c6f3898d-c23acbaa_0] Write session: aborting
2025-10-09T21:42:59.181838Z :INFO: [] MessageGroupId [src] SessionId [src|f5c60a0e-14415c48-c6f3898d-c23acbaa_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:42:59.181853Z :DEBUG: [] MessageGroupId [src] SessionId [src|f5c60a0e-14415c48-c6f3898d-c23acbaa_0] Write session is aborting and will not restart
2025-10-09T21:42:59.181882Z :DEBUG: [] MessageGroupId [src] SessionId [src|f5c60a0e-14415c48-c6f3898d-c23acbaa_0] Write session: destroy
2025-10-09T21:42:59.179549Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [3:7559340778672726799:2448] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:59.179568Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [3:7559340778672726799:2448] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:42:59.179572Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [3:7559340778672726799:2448] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:59.179581Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:59.180146Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 4, Generation: 1
2025-10-09T21:42:59.180974Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1 partition: 0 MaxSeqNo: 0 sessionId: src|f5c60a0e-14415c48-c6f3898d-c23acbaa_0
2025-10-09T21:42:59.182345Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId: src|f5c60a0e-14415c48-c6f3898d-c23acbaa_0 grpc read done: success: 0 data: 
2025-10-09T21:42:59.182356Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 1 sessionId: src|f5c60a0e-14415c48-c6f3898d-c23acbaa_0 grpc read failed
2025-10-09T21:42:59.182372Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 1 sessionId: src|f5c60a0e-14415c48-c6f3898d-c23acbaa_0 grpc closed
2025-10-09T21:42:59.182377Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1 sessionId: src|f5c60a0e-14415c48-c6f3898d-c23acbaa_0 is DEAD
2025-10-09T21:42:59.182716Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:42:59.183880Z node 4 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [3:7559340778672726860:2448] destroyed
2025-10-09T21:42:59.183900Z node 4 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:42:59.183910Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.183914Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.183916Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Session was created
>>> Ready to answer: ok
2025-10-09T21:42:59.242336Z :ERROR: [/Root] OnFederationDiscovery: Got error. Status: UNAVAILABLE. Description: 
2025-10-09T21:42:59.270129Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.270149Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.270152Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.370058Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.370074Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.370077Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.470382Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.470396Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.470399Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.574142Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.574157Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.574160Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.674618Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.674632Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.674644Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.779234Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.779256Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.779259Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.879478Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.879494Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.879497Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:59.980258Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:59.980274Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:59.980278Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.082478Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.082494Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.082497Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.182602Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.182630Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.182634Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.286064Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.286082Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.286084Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.386344Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.386360Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.386364Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.486791Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.486802Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.486805Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.587840Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.587855Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.587857Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.688509Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.688526Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.688529Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.788769Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.788796Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.788799Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.889154Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.889166Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.889169Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:00.989472Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:00.989489Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:00.989493Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.089844Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.089865Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.089869Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.190205Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.190221Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.190225Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.290523Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.290538Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.290542Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.393140Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.393158Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.393162Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.491832Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.491848Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.491851Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.592177Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.592193Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.592196Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.692494Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.692512Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.692514Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.793236Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.793253Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.793258Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.893206Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.893235Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.893240Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:01.993539Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:01.993557Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:01.993560Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.093862Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.093877Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.093881Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.194544Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.194563Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.194567Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.295815Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.295830Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.295834Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:02.399155Z node 4 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:02.399171Z node 4 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:02.399174Z node 4 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.093945Z :WriteSessionNoAvailableDatabase INFO: Random seed for debugging is 1760046171093939
2025-10-09T21:42:51.186624Z node 1 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7559340747009441601:2079];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:51.186647Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-10-09T21:42:51.192545Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:51.193551Z node 2 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[2:7559340748080778371:2172];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:51.193572Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/owmb/0001ae/r3tmp/tmp3syayw/pdisk_1.dat
2025-10-09T21:42:51.201743Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:51.255597Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:51.282187Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:51.283205Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:42:51.289804Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:51.289845Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:51.297230Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
TServer::EnableGrpc on GrpcPort 27205, node 1
2025-10-09T21:42:51.304827Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:51.304850Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:51.310252Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-10-09T21:42:51.310904Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:51.325188Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/owmb/0001ae/r3tmp/yandexjPdNxw.tmp
2025-10-09T21:42:51.325200Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/owmb/0001ae/r3tmp/yandexjPdNxw.tmp
2025-10-09T21:42:51.325262Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/owmb/0001ae/r3tmp/yandexjPdNxw.tmp
2025-10-09T21:42:51.325349Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-10-09T21:42:51.335798Z INFO: TTestServer started on Port 19673 GrpcPort 27205
TClient is connected to server localhost:19673
PQClient connected to localhost:27205
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-09T21:42:51.371001Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
2025-10-09T21:42:51.386375Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976720657, at schemeshard: 72057594046644480
waiting...
2025-10-09T21:42:51.404241Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976720658, at schemeshard: 72057594046644480
waiting...
waiting...
2025-10-09T21:42:51.444290Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:51.580729Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:51.826144Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340747009442569:2325], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.826194Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.830254Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340747009442581:2328], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.830278Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340747009442582:2329], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.830307Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.831766Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976720661:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-10-09T21:42:51.834475Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340747009442617:2332], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.834498Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.838523Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7559340748080778586:2299], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.838546Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7559340748080778567:2296], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.838622Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.842098Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340747009442620:2333], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.842221Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:51.844627Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976720661, at schemeshard: 72057594046644480
2025-10-09T21:42:51.848177Z node 1 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [1:7559340747009442585:2330], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720661 completed, doublechecking }
2025-10-09T21:42:51.848386Z node 2 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [2:7559340748080778592:2128] txid# 281474976715657, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:51.933414Z node 1 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [1:7559340747009442667:2675] txid# 281474976720662, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:51.948120Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [2:7559340748080778630:2304], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:51.947713Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [1:7559340747009442678:2337], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:51.948350Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=1&id=YzE1ZTMwODMtY2VmNWE1YWMtMTFmM2YxOS0yNGM1NmE5MQ==, ActorId: [1:7559340747009442566:2323], ActorState: ExecuteState, TraceId: 01k75ethn17q6et1r95wccyq7r, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:51.948624Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=2&id=YzRjNjI1Yy0zMzBmNDg2Ni01OWI0OTdiOC1kZDg4YzE5YQ==, ActorId: [2:7559340748080778548:2292], ActorState: ExecuteState, TraceId: 01k75ethns2378dzkd35b18cwg, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:51.949077Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:51.949087Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:51.951912Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:52.026640Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:52.076717Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720665:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:27205", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, false, 1000);
2025-10-09T21:42:52.165005Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720666. Ctx: { TraceId: 01k75ethys0nvm5eavp8xwav07, Database: , SessionId: ydb://session/3?node_id=1&id=ODA1ZjMwYzMtMTcyNTI2Y2UtZjVkZGJmOTEtZGM1ZGNmMWI=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [1:7559340751304410364:2962] 
2025-10-09T21:42:52.189094Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:52.200052Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:56.186939Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7559340747009441601:2079];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:56.186969Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
2025-10-09T21:42:56.193609Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[2:7559340748080778371:2172];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:56.193636Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
2025-10-09T21:42:57.298521Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720672:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
waiting...
PQ Client: create topic: rt3.dc1--test-topic with 1 partitions
CallPersQueueGRPC request to localhost:27205
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:57.335412Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:27205
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720673
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-09T21:42:57.349824Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][] pipe [1:7559340772779247269:3228] connected; active server actors: 1
2025-10-09T21:42:57.350102Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1517: [72075186224037893][rt3.dc1--test-topic] updating configuration. Deleted partitions []. Added partitions [0]
2025-10-09T21:42:57.350635Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:940: [72075186224037893][rt3.dc1--test-topic] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-09T21:42:57.357523Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:138: [72075186224037893][rt3.dc1--test-topic] BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) 
2025-10-09T21:42:57.358843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:57.359692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:57.359758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:784: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-09T21:42:57.359770Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:57.359780Z node 2 :PERSQUEUE INFO: pq_impl.cpp:609: [PQ: 72075186224037892] doesn't have tx writes info
2025-10-09T21:42:57.359926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340772779247268:3227], now have 1 active actors on pipe
2025-10-09T21:42:57.359934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340772779247297:2436], now have 1 active actors on pipe
2025-10-09T21:42:57.360248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.360803Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-10-09T21:42:57.367050Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720674. Ctx: { TraceId: 01k75etq1z6t8q6ser2rmr830p, Database: , SessionId: ydb://session/3?node_id=1&id=ZjE5OWFmZTEtNDI0NzgzMjMtOGM1NGJjNDItNDIyMDBlNWI=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-10-09T21:42:57.370552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
GetTopicVersionFromPath:  record Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
 name rt3.dc1--test-topic version0
2025-10-09T21:42:57.379462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340772779247354:3284], now have 1 active actors on pipe
2025-10-09T21:42:57.379973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.381929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.382170Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:57.382196Z node 2 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:57.382254Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:57.382324Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340773850582888:2389]
2025-10-09T21:42:57.383094Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:57.383097Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:57.383102Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [2:7559340773850582888:2389]
2025-10-09T21:42:57.383109Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:57.383399Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-10-09T21:42:57.383405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.383428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.383431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:57.383434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:57.383436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.383437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.383439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:57.383445Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:57.383557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.383586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:57.384748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.384783Z node 2 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046177428, TxId 281474976720673
2025-10-09T21:42:57.384786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.384788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.384811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.384846Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:57.385470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:57.385529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:57.385535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.385537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.385539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.385545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:57.385637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:395: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-10-09T21:42:57.385657Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:57.385697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.388017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.388039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:57.389074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:57.485988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.486006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.486009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.586236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.586253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.586257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.686611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.686627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.686630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.787771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.787789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.787792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046177428
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046177428
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:27205
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
2025-10-09T21:42:57.888748Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-09T21:42:57.889012Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-09T21:42:57.889021Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:27205
2025-10-09T21:42:57.882858Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-10-09T21:42:57.891581Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:57.892460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.892479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.892482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.890618Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:57.890632Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1
2025-10-09T21:42:57.894244Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:57.894289Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:53418
2025-10-09T21:42:57.894295Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:53418 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:57.894299Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:57.898580Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-09T21:42:57.898646Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:57.898649Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:57.898650Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:57.898657Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340772779247407:2452] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:57.899239Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340772779247407:2452] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:57.934696Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [1:7559340772779247407:2452] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:57.935028Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340772779247441:2452] connected; active server actors: 1
2025-10-09T21:42:57.935099Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [1:7559340772779247407:2452] (SourceId=src, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src
2025-10-09T21:42:57.935106Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340772779247407:2452] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:57.939392Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340772779247441:2452] disconnected.
2025-10-09T21:42:57.939401Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340772779247441:2452] disconnected; active server actors: 1
2025-10-09T21:42:57.939404Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340772779247441:2452] disconnected no session
2025-10-09T21:42:57.958616Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340772779247407:2452] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:57.958636Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340772779247407:2452] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:42:57.958640Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340772779247407:2452] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:57.958650Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:57.959189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340772779247463:2452], now have 1 active actors on pipe
2025-10-09T21:42:57.959268Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1
2025-10-09T21:42:57.959427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.959442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.959473Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src|a7604d10-457f80f4-d616649b-339b56c4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src
2025-10-09T21:42:57.959491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.959514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.959523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:57.959534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:57.959551Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:57.959575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:57.960556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:57.960569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:57.960590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:57.960720Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1 partition: 0 MaxSeqNo: 0 sessionId: src|a7604d10-457f80f4-d616649b-339b56c4_0
2025-10-09T21:42:57.961064Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046177961 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:57.961100Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|a7604d10-457f80f4-d616649b-339b56c4_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:57.961265Z :INFO: [] MessageGroupId [src] SessionId [src|a7604d10-457f80f4-d616649b-339b56c4_0] Write session: close. Timeout = 0 ms
2025-10-09T21:42:57.961276Z :INFO: [] MessageGroupId [src] SessionId [src|a7604d10-457f80f4-d616649b-339b56c4_0] Write session will now close
2025-10-09T21:42:57.961282Z :DEBUG: [] MessageGroupId [src] SessionId [src|a7604d10-457f80f4-d616649b-339b56c4_0] Write session: aborting
2025-10-09T21:42:57.961445Z :INFO: [] MessageGroupId [src] SessionId [src|a7604d10-457f80f4-d616649b-339b56c4_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:42:57.961451Z :DEBUG: [] MessageGroupId [src] SessionId [src|a7604d10-457f80f4-d616649b-339b56c4_0] Write session: destroy
2025-10-09T21:42:57.962111Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId: src|a7604d10-457f80f4-d616649b-339b56c4_0 grpc read done: success: 0 data: 
2025-10-09T21:42:57.962121Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 1 sessionId: src|a7604d10-457f80f4-d616649b-339b56c4_0 grpc read failed
2025-10-09T21:42:57.962127Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 1 sessionId: src|a7604d10-457f80f4-d616649b-339b56c4_0 grpc closed
2025-10-09T21:42:57.962134Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1 sessionId: src|a7604d10-457f80f4-d616649b-339b56c4_0 is DEAD
2025-10-09T21:42:57.962355Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:42:57.962790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340772779247463:2452] destroyed
2025-10-09T21:42:57.962807Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:42:57.962815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.962818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.962819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
Session was created
2025-10-09T21:42:57.992348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:57.992364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:57.992367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Got event: >>> Ready to answer: ok
ReadyToAcceptEvent
2025-10-09T21:42:58.029583Z :ERROR: [/Root] [] [] Failed to select database: no available database. Status: NOT_FOUND
>>> Got event: SessionClosed { Status: NOT_FOUND Issues: "<main>: Error: Failed to select database: no available database " }
2025-10-09T21:42:58.092624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.092641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.092644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.193554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:58.193572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:58.193576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:58.346713Z node 1 :KQP_EXECUTER ERROR: kqp_executer_impl.h:1010: ActorId: [1:7559340777074214791:2464] TxId: 281474976720677. Ctx: { TraceId: 01k75etqzg1q2fp8pmfdrkj7qd, Database: /Root, SessionId: ydb://session/3?node_id=1&id=MjZhNDVhNmEtOTE1Mzg4YzUtODk1NDBkYTAtZTAzOTk0ODQ=, PoolId: default}. UNAVAILABLE: Failed to send EvStartKqpTasksRequest because node is unavailable: 2
2025-10-09T21:42:58.346881Z node 1 :KQP_COMPUTE ERROR: dq_compute_actor_impl.h:1210: SelfId: [1:7559340777074214799:2464], TxId: 281474976720677, task: 3. Ctx: { TraceId : 01k75etqzg1q2fp8pmfdrkj7qd.  RunScriptActorId : [0:0:0].  CustomerSuppliedId : .  PoolId : default.  SessionId : ydb://session/3?node_id=1&id=MjZhNDVhNmEtOTE1Mzg4YzUtODk1NDBkYTAtZTAzOTk0ODQ=.  CurrentExecutionId : .  Database : /Root.  DatabaseId : /Root. }. Handle abort execution event from: [1:7559340777074214791:2464], status: UNAVAILABLE, reason: { <main>: Error: Terminate execution }
2025-10-09T21:42:58.557301Z :WriteSessionSwitchDatabases INFO: Random seed for debugging is 1760046178557293
2025-10-09T21:42:58.649968Z node 4 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[4:7559340776264217478:2255];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:42:58.649992Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-10-09T21:42:58.650276Z node 3 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
2025-10-09T21:42:58.650882Z node 3 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:58.650900Z node 3 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:58.651526Z node 4 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/owmb/0001ae/r3tmp/tmpswaXFK/pdisk_1.dat
2025-10-09T21:42:58.652505Z node 4 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:58.683954Z node 3 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:42:58.690199Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:58.690224Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:58.691693Z node 3 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 4 Cookie 4
2025-10-09T21:42:58.692248Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(4, (0,0,0,0)) VolatileState: Connecting -> Connected
TServer::EnableGrpc on GrpcPort 10589, node 3
2025-10-09T21:42:58.706222Z INFO: TTestServer started on Port 12132 GrpcPort 10589
2025-10-09T21:42:58.706337Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/owmb/0001ae/r3tmp/yandexcBzCKU.tmp
2025-10-09T21:42:58.706352Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/owmb/0001ae/r3tmp/yandexcBzCKU.tmp
TClient is connected to server localhost:12132
PQClient connected to localhost:10589
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-09T21:42:58.722117Z node 3 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/owmb/0001ae/r3tmp/yandexcBzCKU.tmp
2025-10-09T21:42:58.722457Z node 3 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-10-09T21:42:58.748144Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:58.748175Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:58.755133Z node 3 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(3, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:58.755503Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
waiting...
2025-10-09T21:42:58.765861Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976720658, at schemeshard: 72057594046644480
waiting...
waiting...
2025-10-09T21:42:58.838953Z node 3 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:58.945272Z node 4 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:59.077455Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340780101690579:2324], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:59.077499Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:59.077585Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340780101690615:2328], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:59.077598Z node 3 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [3:7559340780101690616:2329], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:59.077652Z node 3 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:59.078458Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976720661:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-10-09T21:42:59.083731Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:59.094589Z node 3 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [3:7559340780101690619:2330], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720661 completed, doublechecking }
2025-10-09T21:42:59.134494Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:59.180090Z node 3 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [3:7559340780101690943:2849] txid# 281474976720664, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:59.182522Z node 4 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [4:7559340780559184988:2305], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:5:17: Error: At function: KiReadTable!
        <main>:5:17: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Versions]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:59.183239Z node 4 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=4&id=ZTdiNDExMzYtOGViNjkzYzUtZWRhYmU0OTYtMjY0Yzg4N2M=, ActorId: [4:7559340780559184954:2298], ActorState: ExecuteState, TraceId: 01k75etrv2bdv41d3s48a75rpd, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:59.183387Z node 4 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:59.188056Z node 3 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [3:7559340780101690956:2348], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:5:17: Error: At function: KiReadTable!
        <main>:5:17: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Versions]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:59.189023Z node 3 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=3&id=YzIzNDMzYzUtYWIwYjYyNGQtZWFmNTQwMS01NjU0YzQwMQ==, ActorId: [3:7559340780101690576:2322], ActorState: ExecuteState, TraceId: 01k75etrr42rekwgckfryc843k, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:59.189160Z node 3 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:59.203153Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720665:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:10589", true, true, 1000);
2025-10-09T21:42:59.250983Z node 3 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720666. Ctx: { TraceId: 01k75etrwt264h9btc1qbmjekg, Database: , SessionId: ydb://session/3?node_id=3&id=YmRiMjEwMTMtNGQxNjA0ZTYtZTcxZjE0N2QtZmQ5Zjg5ZDA=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [3:7559340780101691160:2993] 
2025-10-09T21:42:59.647610Z node 3 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:59.649107Z node 4 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:43:03.655138Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[4:7559340776264217478:2255];send_to=[0:7307199536658146131:7762515];
2025-10-09T21:43:03.655160Z node 4 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
waiting...
2025-10-09T21:43:04.526891Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720672:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
PQ Client: create topic: rt3.dc1--test-topic with 1 partitions
CallPersQueueGRPC request to localhost:10589
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:43:04.555280Z node 3 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:10589
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720673
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-09T21:43:04.574500Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:43:04.582715Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:43:04.582827Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:784: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-09T21:43:04.582835Z node 3 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:04.582842Z node 3 :PERSQUEUE INFO: pq_impl.cpp:609: [PQ: 72075186224037892] doesn't have tx writes info
2025-10-09T21:43:04.582951Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][] pipe [3:7559340801576528161:3316] connected; active server actors: 1
2025-10-09T21:43:04.582989Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340801576528160:3315], now have 1 active actors on pipe
2025-10-09T21:43:04.583260Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1517: [72075186224037893][rt3.dc1--test-topic] updating configuration. Deleted partitions []. Added partitions [0]
2025-10-09T21:43:04.583282Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:43:04.583412Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:940: [72075186224037893][rt3.dc1--test-topic] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-09T21:43:04.583780Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [4:7559340802034021722:2357], now have 1 active actors on pipe
2025-10-09T21:43:04.583833Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 3, Generation 1
2025-10-09T21:43:04.585916Z node 3 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720674. Ctx: { TraceId: 01k75ety3hf23zjgg319e3b0wx, Database: , SessionId: ydb://session/3?node_id=3&id=Nzc3ZDRmMzEtNTFmYjlhZGYtMzhmNmVjZjAtN2Y4ZGEwMmM=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-10-09T21:43:04.586773Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:138: [72075186224037893][rt3.dc1--test-topic] BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) 
2025-10-09T21:43:04.586972Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:43:04.588903Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340801576528238:3367], now have 1 active actors on pipe
2025-10-09T21:43:04.589213Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:43:04.590408Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:43:04.590592Z node 3 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:43:04.590611Z node 3 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:43:04.590670Z node 3 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:43:04.590705Z node 3 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [3:7559340801576528250:2459]
2025-10-09T21:43:04.591434Z node 3 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:43:04.591438Z node 3 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:43:04.591443Z node 3 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [3:7559340801576528250:2459]
2025-10-09T21:43:04.591451Z node 3 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:43:04.591727Z node 3 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-10-09T21:43:04.591742Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.591760Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.591763Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:04.591765Z node 3 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:43:04.591768Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.591770Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.591771Z node 3 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:43:04.591778Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:04.591850Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:43:04.591875Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:04.594767Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:43:04.594825Z node 3 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046184638, TxId 281474976720673
2025-10-09T21:43:04.594836Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.594838Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.594869Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.594917Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
2025-10-09T21:43:04.596225Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
TClient::Ls response: Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
GetTopicVersionFromPath:  record Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
 name rt3.dc1--test-topic version0
2025-10-09T21:43:04.596290Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:04.596303Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.596305Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.596307Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.596314Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:04.596420Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:395: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-10-09T21:43:04.596444Z node 3 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:43:04.596495Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:43:04.597937Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:43:04.597967Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:43:04.598380Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:43:04.691291Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.691307Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.691311Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.791838Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.791854Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.791858Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.894155Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.894171Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.894174Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:04.994403Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:04.994435Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:04.994438Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.094908Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.094923Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.094926Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046184638
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1760046184638
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:10589
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:43:05.107265Z node 3 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
2025-10-09T21:43:05.111736Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-09T21:43:05.112000Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-09T21:43:05.112008Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:10589
2025-10-09T21:43:05.112383Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:43:05.112831Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:05.112849Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1
2025-10-09T21:43:05.116908Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:43:05.116956Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:59084
2025-10-09T21:43:05.116962Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:59084 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:43:05.116967Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:05.117503Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-09T21:43:05.117537Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:05.117542Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:05.117544Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:05.117550Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [3:7559340805871495645:2480] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:05.118142Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [3:7559340805871495645:2480] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:05.167202Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [3:7559340805871495645:2480] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:43:05.167449Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340805871495680:2480] connected; active server actors: 1
2025-10-09T21:43:05.167696Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [3:7559340805871495645:2480] (SourceId=src, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src
2025-10-09T21:43:05.167716Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [3:7559340805871495645:2480] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:05.167903Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340805871495680:2480] disconnected.
2025-10-09T21:43:05.167924Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340805871495680:2480] disconnected; active server actors: 1
2025-10-09T21:43:05.167929Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340805871495680:2480] disconnected no session
2025-10-09T21:43:05.191455Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [3:7559340805871495645:2480] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:05.191477Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [3:7559340805871495645:2480] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:43:05.191482Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [3:7559340805871495645:2480] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:05.191492Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:05.192910Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046185192 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:43:05.192954Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|45b8c1ae-daad86a6-1e936f06-f95987de_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:43:05.193068Z :INFO: [] MessageGroupId [src] SessionId [src|45b8c1ae-daad86a6-1e936f06-f95987de_0] Write session: close. Timeout = 0 ms
2025-10-09T21:43:05.193078Z :INFO: [] MessageGroupId [src] SessionId [src|45b8c1ae-daad86a6-1e936f06-f95987de_0] Write session will now close
2025-10-09T21:43:05.193085Z :DEBUG: [] MessageGroupId [src] SessionId [src|45b8c1ae-daad86a6-1e936f06-f95987de_0] Write session: aborting
2025-10-09T21:43:05.193227Z :INFO: [] MessageGroupId [src] SessionId [src|45b8c1ae-daad86a6-1e936f06-f95987de_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:43:05.193234Z :DEBUG: [] MessageGroupId [src] SessionId [src|45b8c1ae-daad86a6-1e936f06-f95987de_0] Write session: destroy
2025-10-09T21:43:05.192182Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 3, Generation: 1
2025-10-09T21:43:05.192212Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340805871495699:2480], now have 1 active actors on pipe
2025-10-09T21:43:05.192272Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:05.192280Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:05.192316Z node 3 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src|45b8c1ae-daad86a6-1e936f06-f95987de_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src
2025-10-09T21:43:05.192327Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.192332Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.192336Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:05.192350Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.192365Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:05.192385Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:05.192483Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:05.192490Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:05.192503Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:05.192534Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1 partition: 0 MaxSeqNo: 0 sessionId: src|45b8c1ae-daad86a6-1e936f06-f95987de_0
2025-10-09T21:43:05.193572Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId: src|45b8c1ae-daad86a6-1e936f06-f95987de_0 grpc read done: success: 0 data: 
2025-10-09T21:43:05.193583Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 1 sessionId: src|45b8c1ae-daad86a6-1e936f06-f95987de_0 grpc read failed
2025-10-09T21:43:05.193589Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 1 sessionId: src|45b8c1ae-daad86a6-1e936f06-f95987de_0 grpc closed
2025-10-09T21:43:05.193595Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1 sessionId: src|45b8c1ae-daad86a6-1e936f06-f95987de_0 is DEAD
2025-10-09T21:43:05.193906Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:43:05.194050Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [3:7559340805871495699:2480] destroyed
2025-10-09T21:43:05.194068Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:43:05.194074Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.194078Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.194080Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.195261Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.195272Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.195274Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Got event: ReadyToAcceptEvent
>>> Got event: ReadyToAcceptEvent
>>> Ready to answer: ok
2025-10-09T21:43:05.249103Z :INFO: [/Root] [] [] Start federated write session to database 'dc1' (previous was <empty>) FederationState: { Status: SUCCESS SelfLocation: "fancy_datacenter" DbInfos: [ { name: "dc1" path: "/Root" id: "account-dc1" endpoint: "localhost:10589" location: "dc1" status: AVAILABLE weight: 1000 } { name: "dc2" path: "/Root" id: "account-dc2" endpoint: "localhost:10589" location: "dc2" status: AVAILABLE weight: 500 } { name: "dc3" path: "/Root" id: "account-dc3" endpoint: "localhost:10589" location: "dc3" status: AVAILABLE weight: 500 } ] ControlPlaneEndpoint: cp.logbroker-federation:2135 }
2025-10-09T21:43:05.251648Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: try to update token
2025-10-09T21:43:05.251876Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Start write session. Will connect to nodeId: 0
2025-10-09T21:43:05.295473Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.295487Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.295489Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.399501Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.399518Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.399521Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.501859Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.501882Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.501884Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.506678Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: write to message_group: src_id
2025-10-09T21:43:05.508759Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: send init request: init_request { path: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:05.508939Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: OnWriteDone gRpcStatusCode: 0
2025-10-09T21:43:05.510690Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:05.510709Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2
2025-10-09T21:43:05.512917Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId:  grpc read done: success: 1 data: init_request { path: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:05.512986Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2 path: "test-topic" message_group_id: "src_id" from ipv6:[::1]:53370
2025-10-09T21:43:05.512991Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2 sessionId= userAgent="topic server" ip=ipv6:[::1]:53370 proto=topic  topic=test-topic durationSec=0
2025-10-09T21:43:05.512996Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:05.513892Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2 sessionId:  describe result for acl check
2025-10-09T21:43:05.513930Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:05.513932Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:05.513934Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:05.513941Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [3:7559340805871495712:2493] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:05.514606Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [3:7559340805871495712:2493] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:05.551331Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [3:7559340805871495712:2493] (SourceId=src_id, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:43:05.552455Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340805871495744:2493] connected; active server actors: 1
2025-10-09T21:43:05.552910Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [3:7559340805871495712:2493] (SourceId=src_id, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src_id
2025-10-09T21:43:05.552922Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [3:7559340805871495712:2493] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:05.553283Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340805871495744:2493] disconnected.
2025-10-09T21:43:05.553291Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340805871495744:2493] disconnected; active server actors: 1
2025-10-09T21:43:05.553294Z node 4 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [3:7559340805871495744:2493] disconnected no session
2025-10-09T21:43:05.556882Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [3:7559340805871495712:2493] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:05.556896Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [3:7559340805871495712:2493] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:43:05.556899Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [3:7559340805871495712:2493] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:05.556908Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:05.557630Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 3, Generation: 1
2025-10-09T21:43:05.557648Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340805871495757:2493], now have 1 active actors on pipe
2025-10-09T21:43:05.557725Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:05.557755Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:05.557784Z node 3 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:05.557799Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.557804Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.557807Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:05.557818Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.557831Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:05.557847Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:05.557885Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:05.557892Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:05.557904Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:05.557931Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2 partition: 0 MaxSeqNo: 0 sessionId: src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0
2025-10-09T21:43:05.558252Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: OnReadDone gRpcStatusCode: 0
2025-10-09T21:43:05.558269Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046185558 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:43:05.558302Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session established. Init response: session_id: "src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0" supported_codecs { codecs: 1 codecs: 2 codecs: 3 }
2025-10-09T21:43:05.559635Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0]  MessageGroupId [src_id] Write 1 messages with Id from 1 to 1
2025-10-09T21:43:05.559843Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0]  MessageGroupId [src_id] Write session: try to update token
2025-10-09T21:43:05.559856Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0]  MessageGroupId [src_id] Send 1 message(s) (0 left), first sequence number is 1
2025-10-09T21:43:05.562179Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:43:05.562320Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:43:05.562490Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:05.562503Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:05.562536Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:43:05.562556Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:05.562652Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:05.562659Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:05.562764Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 0 messageNo: 1 size 98 offset: -1
2025-10-09T21:43:05.562777Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:43:05.562790Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.562797Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.562800Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:05.562827Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 0
2025-10-09T21:43:05.562877Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 1 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 172 count 1 nextOffset 1 batches 1
2025-10-09T21:43:05.562889Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.562941Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 0,1 HeadOffset 0 endOffset 0 curOffset 1 d0000000000_00000000000000000000_00000_0000000001_00000? size 160 WTime 1760046185562
2025-10-09T21:43:05.562968Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:43:05.562982Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 1 size 160
2025-10-09T21:43:05.570186Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 1 size 160 actorID [3:7559340801576528186:2459]
2025-10-09T21:43:05.570219Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:43:05.570241Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 105 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:43:05.570244Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63' size 160
2025-10-09T21:43:05.570252Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:43:05.570264Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 1, partNo: 0, Offset: 0 is stored on disk
2025-10-09T21:43:05.570299Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:43:05.570305Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:43:05.570309Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.570312Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.570314Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.570323Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:05.570336Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1  requestId: <none> cookie: 1
2025-10-09T21:43:05.570367Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 1 max time lag 0ms effective offset 0
2025-10-09T21:43:05.570389Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 160 count 1 last offset 0, current partition end offset: 1
2025-10-09T21:43:05.570392Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:43:05.570401Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 0 source 1 size 160 accessed 0 times before, last time 2025-10-09T21:43:05.000000Z
2025-10-09T21:43:05.570435Z node 3 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 3. All 1 blobs are from cache.
2025-10-09T21:43:05.570448Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:43:05.570462Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 1 count 1 size 140 from pos 0 cbcount 1
2025-10-09T21:43:05.570467Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63'
2025-10-09T21:43:05.570478Z node 3 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046185562 queuesize 0 startOffset 0
2025-10-09T21:43:05.570503Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:43:05.573585Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0]  MessageGroupId [src_id] Write session: OnReadDone gRpcStatusCode: 0
2025-10-09T21:43:05.573754Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0]  MessageGroupId [src_id] Write session got write response: acks { seq_no: 1 written { } } write_statistics { persisting_time { nanos: 7000000 } min_queue_wait_time { } max_queue_wait_time { } partition_quota_wait_time { } topic_quota_wait_time { } }
2025-10-09T21:43:05.573763Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0]  MessageGroupId [src_id] OnAck: seqNo=1, txId=?
2025-10-09T21:43:05.573768Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0]  MessageGroupId [src_id] Write session: acknoledged message 1
2025-10-09T21:43:05.600603Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.600620Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.600623Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.702636Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.702657Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.702660Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.806613Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.806636Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.806640Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:05.906191Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:05.906209Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:05.906212Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.009687Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.009706Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.009709Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.110086Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.110106Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.110110Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.210397Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.210425Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.210428Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.310718Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.310737Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.310741Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.411798Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.411815Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.411818Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.511976Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.511993Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.511996Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.558642Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [3:7559340805871495712:2493] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:06.608428Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [3:7559340805871495712:2493] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=1 Status=SUCCESS
2025-10-09T21:43:06.608440Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [3:7559340805871495712:2493] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:06.612308Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.612325Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.612327Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.712686Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.712706Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.712709Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.813044Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.813061Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.813065Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:06.913523Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:06.913541Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:06.913544Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.014230Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.014248Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.014251Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.114572Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.114586Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.114589Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.215878Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.215898Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.215900Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.315805Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.315825Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.315829Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.416126Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.416145Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.416149Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.517051Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.517068Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.517071Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.617408Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.617433Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.617435Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.722233Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.722250Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.722254Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.825741Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.825759Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.825762Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:07.929225Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:07.929247Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:07.929251Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.029526Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.029543Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.029546Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.129877Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.129892Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.129895Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.230210Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.230225Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.230228Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.333831Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.333853Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.333856Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.434297Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.434314Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.434317Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.534380Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.534396Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.534399Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.634757Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.634775Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.634778Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.741869Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.741884Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.741886Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.838113Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.838129Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.838131Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:08.938147Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:08.938171Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:08.938174Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.042175Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.042198Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.042201Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
E1009 21:43:09.053549668 3631581 chttp2_transport.cc:1143]             ipv4:127.0.0.1:2733: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 1250ms
2025-10-09T21:43:09.146193Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.146209Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.146213Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.249999Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.250037Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.250040Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.354006Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.354024Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.354037Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.456957Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.456975Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.456979Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.558342Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.558362Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.558365Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.591136Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:09.661233Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.661260Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.661263Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.762013Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.762081Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.762102Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.861944Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.861961Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.861964Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:09.962664Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:09.962684Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:09.962687Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.066271Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.066289Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.066293Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.170262Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.170280Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.170284Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.270525Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.270543Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.270547Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.374310Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.374329Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.374332Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.475048Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.475069Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.475073Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.575978Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.576001Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.576005Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.678250Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.678269Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.678274Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.778523Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.778541Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.778553Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.878852Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.878871Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.878874Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:10.979620Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:10.979639Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:10.979643Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.080012Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.080030Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.080034Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.186440Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.186462Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.186466Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.286436Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.286453Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.286456Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.390283Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.390304Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.390307Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.490854Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.490880Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.490883Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.591023Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.591042Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.591045Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.691462Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.691479Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.691482Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.794185Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.794220Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.794223Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.894719Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.894735Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.894738Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:11.995078Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:11.995097Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:11.995101Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.098410Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.098444Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.098447Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.198740Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.198760Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.198763Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.299194Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.299224Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.299228Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.402727Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.402748Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.402752Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.502870Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.502890Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.502893Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.603231Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.603249Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.603253Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.703676Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.703697Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.703700Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.803908Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.803929Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.803933Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:12.904277Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:12.904294Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:12.904298Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.004599Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.004614Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.004616Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.104973Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.105001Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.105004Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.205380Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.205401Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.205404Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.305668Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.305697Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.305701Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.408386Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.408405Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.408408Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.510220Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.510249Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.510252Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.610740Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.610759Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.610762Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.680678Z node 3 :FLAT_TX_SCHEMESHARD WARN: schemeshard_impl.cpp:7721: Cannot get console configs
2025-10-09T21:43:13.680694Z node 3 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:43:13.714084Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.714105Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.714108Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.814149Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.814167Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.814170Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:13.914479Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:13.914498Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:13.914502Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.014824Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.014840Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.014843Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.115129Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.115150Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.115153Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.215439Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.215455Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.215458Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.315846Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.315864Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.315868Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.416199Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.416220Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.416223Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.519722Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.519739Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.519742Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.591468Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:14.622009Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.622048Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.622051Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.722384Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.722401Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.722404Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.822775Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.822795Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.822798Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:14.923089Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:14.923116Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:14.923119Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.023730Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.023751Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.023754Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.123785Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.123811Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.123814Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
E1009 21:43:15.214166672 3631549 chttp2_transport.cc:1143]             ipv6:%5B::1%5D:2733: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 1250ms
2025-10-09T21:43:15.224117Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.224145Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.224148Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.325110Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.325165Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.325168Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.425354Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.425371Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.425374Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.526606Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.526626Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.526648Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.626628Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.626648Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.626651Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.726939Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.726956Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.726959Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.827249Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.827267Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.827288Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:15.927719Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:15.927734Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:15.927738Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.028021Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.028041Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.028051Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.128356Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.128373Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.128376Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.228702Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.228719Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.228723Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.329058Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.329159Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.329162Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.430124Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.430146Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.430149Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.530261Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.530289Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.530292Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.632299Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.632321Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.632324Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.732510Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.732526Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.732530Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.834396Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.834412Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.834431Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:16.934987Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:16.935013Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:16.935016Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.035082Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.035102Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.035105Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.135443Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.135459Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.135462Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.235788Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.235805Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.235809Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.336736Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.336765Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.336768Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.437042Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.437062Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.437065Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.537387Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.537407Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.537410Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.638328Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.638344Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.638347Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.738582Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.738600Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.738604Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.838972Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.838987Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.838990Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:17.939374Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:17.939390Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:17.939393Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.039627Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.039643Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.039646Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.140077Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.140091Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.140094Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.240308Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.240322Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.240324Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.340707Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.340723Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.340725Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.441038Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.441055Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.441058Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.541379Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.541406Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.541409Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.641732Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.641746Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.641749Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.742179Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.742193Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.742195Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.843937Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.843953Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.843957Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:18.944331Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:18.944346Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:18.944349Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.044684Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.044701Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.044704Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.145090Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.145110Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.145114Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.245393Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.245408Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.245411Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.345687Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.345702Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.345704Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.446137Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.446165Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.446169Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.546480Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.546498Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.546512Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.591692Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:19.646816Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.646835Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.646838Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.747130Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.747147Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.747150Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.847831Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.847847Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.847851Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:19.948365Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:19.948382Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:19.948386Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.049542Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.049561Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.049566Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.149616Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.149647Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.149651Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.249951Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.249967Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.249970Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.350290Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.350308Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.350311Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.450865Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.450882Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.450886Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.551008Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.551034Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.551037Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.651370Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.651389Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.651393Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.754131Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.754161Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.754166Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.857989Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.858005Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.858008Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:20.961010Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:20.961031Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:20.961035Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.061800Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.061817Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.061821Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.162801Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.162834Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.162837Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.262709Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.262730Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.262733Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.363048Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.363074Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.363078Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.466191Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.466213Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.466216Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.566624Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.566640Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.566644Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.667145Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.667171Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.667174Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.767529Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.767558Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.767561Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.867881Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.867908Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.867911Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:21.968229Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:21.968246Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:21.968250Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.070114Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.070134Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.070137Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.170557Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.170577Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.170581Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.272018Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.272037Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.272040Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.372195Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.372212Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.372215Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.472549Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.472568Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.472571Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.574235Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.574255Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.574258Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.674264Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.674283Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.674286Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.774636Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.774654Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.774658Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.874992Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.875012Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.875026Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:22.975647Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:22.975666Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:22.975670Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.076139Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.076162Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.076165Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.176560Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.176580Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.176583Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.276910Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.276926Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.276929Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.380398Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.380414Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.380418Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.480691Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.480713Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.480716Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.582265Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.582283Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.582286Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.682533Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.682548Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.682551Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.783164Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.783182Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.783185Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.883117Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.883137Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.883141Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:23.983436Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:23.983461Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:23.983464Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.086431Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.086451Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.086455Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.190119Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.190137Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.190141Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.291392Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.291412Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.291415Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.391510Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.391527Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.391531Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.494508Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.494524Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.494527Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.591772Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:24.594236Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.594250Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.594253Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.694660Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.694674Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.694676Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.794989Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.795009Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.795012Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.895333Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.895352Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.895355Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:24.995683Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:24.995701Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:24.995704Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.095981Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.096004Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.096008Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.197678Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.197699Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.197703Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.298075Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.298096Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.298100Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.398607Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.398626Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.398629Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.502230Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.502259Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.502263Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.608227Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.608245Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.608249Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.713624Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.713640Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.713643Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.813876Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.813896Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.813899Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:25.914205Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:25.914222Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:25.914225Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.014678Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.014696Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.014699Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.114884Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.114902Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.114906Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.215201Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.215229Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.215233Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.315859Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.315877Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.315881Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.416089Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.416120Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.416122Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.516264Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.516291Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.516295Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.616623Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.616640Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.616644Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.718242Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.718264Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.718267Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.822147Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.822169Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.822171Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:26.922329Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:26.922347Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:26.922350Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.025722Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.025743Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.025746Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.126191Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.126210Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.126214Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.226471Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.226498Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.226502Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.327761Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.327780Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.327783Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.428193Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.428210Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.428213Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.534090Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.534110Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.534113Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.631963Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.631980Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.631983Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.736122Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.736142Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.736146Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.836259Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.836277Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.836281Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:27.936612Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:27.936635Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:27.936639Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.037126Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.037144Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.037147Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.137285Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.137302Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.137305Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.238316Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.238333Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.238337Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.343429Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.343450Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.343453Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.442465Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.442485Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.442488Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.546517Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.546537Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.546540Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.646704Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.646721Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.646723Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.747109Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.747126Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.747129Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.850288Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.850310Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.850313Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:28.951277Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:28.951295Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:28.951297Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.051452Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.051472Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.051476Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.151601Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.151621Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.151624Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.254137Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.254157Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.254160Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.357902Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.357920Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.357923Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.458295Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.458315Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.458319Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.558533Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.558550Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.558553Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.596593Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:29.658953Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.658968Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.658971Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.762233Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.762250Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.762253Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.866121Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.866146Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.866150Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:29.966883Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:29.966908Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:29.966911Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.067074Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.067094Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.067098Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.167365Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.167387Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.167390Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.267893Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.267911Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.267915Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.370215Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.370243Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.370246Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.470488Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.470505Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.470508Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.574139Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.574161Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.574164Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.674260Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.674278Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.674281Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.775382Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.775400Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.775404Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.875524Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.875566Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.875570Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:30.975842Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:30.975864Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:30.975868Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.076263Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.076280Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.076283Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.176553Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.176570Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.176574Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.277359Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.277373Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.277376Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.378740Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.378756Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.378759Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.479164Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.479184Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.479188Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.579494Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.579511Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.579514Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.679982Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.679998Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.680002Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.780182Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.780205Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.780208Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.880537Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.880556Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.880560Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:31.980879Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:31.980897Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:31.980900Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.081176Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.081197Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.081200Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.182142Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.182165Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.182168Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.282633Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.282653Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.282657Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.382816Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.382834Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.382837Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.483175Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.483194Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.483197Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.583517Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.583537Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.583540Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.688096Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.688115Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.688119Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.790156Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.790175Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.790178Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.890216Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.890233Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.890236Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:32.990590Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:32.990611Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:32.990615Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.096673Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.096692Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.096696Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.194829Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.194848Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.194851Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.295173Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.295204Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.295208Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.398651Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.398673Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.398676Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.501379Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.501399Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.501402Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.601334Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.601354Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.601357Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.702194Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.702214Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.702217Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.802614Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.802643Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.802646Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:33.903605Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:33.903624Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:33.903627Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.003852Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.003874Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.003877Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.104336Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.104355Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.104358Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.204521Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.204543Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.204546Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.308803Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.308823Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.308826Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.411511Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.411531Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.411534Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.511803Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.511819Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.511821Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.585415Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:43:34.585742Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 1
2025-10-09T21:43:34.586431Z node 3 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 0 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:43:34.587241Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 1 DataSize: 0 UsedReserveSize: 0
2025-10-09T21:43:34.587560Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:43:34.588826Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72057594046644480, NodeId 3, Generation 2
2025-10-09T21:43:34.595222Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:34.612088Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.612104Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.612108Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.715440Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.715461Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.715472Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.815856Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.815875Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.815878Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:34.916978Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:34.917002Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:34.917006Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.021883Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.021914Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.021917Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.122771Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.122789Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.122792Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.222895Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.222913Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.222917Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: ok
2025-10-09T21:43:35.326431Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.326450Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.326454Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.429842Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.429859Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.429862Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.530049Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.530065Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.530068Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.630579Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.630600Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.630604Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.730835Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.730860Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.730863Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.831564Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.831585Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.831588Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:35.932628Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:35.932664Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:35.932667Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.032901Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.032919Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.032923Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.134443Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.134460Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.134462Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.236680Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.236697Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.236701Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.337007Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.337026Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.337030Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.437326Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.437348Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.437354Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.537678Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.537697Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.537701Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.640877Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.640908Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.640911Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.740954Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.740976Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.740979Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.841460Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.841481Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.841484Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:36.941679Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:36.941700Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:36.941703Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.042001Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.042020Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.042023Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.142457Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.142475Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.142478Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.243674Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.243691Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.243694Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.349202Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.349221Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.349224Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.449569Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.449595Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.449598Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.549966Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.549987Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.549991Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.650263Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.650277Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.650279Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.750641Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.750659Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.750662Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.854111Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.854130Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.854133Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:37.955010Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:37.955027Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:37.955029Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.055248Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.055266Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.055269Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.161284Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.161308Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.161311Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.262086Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.262110Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.262114Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.361953Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.361973Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.361976Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.462252Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.462279Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.462282Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.562646Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.562664Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.562667Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.663147Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.663164Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.663166Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.763313Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.763329Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.763332Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.863708Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.863731Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.863742Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:38.967730Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:38.967747Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:38.967750Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.068024Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.068042Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.068046Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.168355Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.168376Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.168379Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.268703Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.268721Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.268724Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.369016Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.369032Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.369036Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.469396Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.469415Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.469418Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.574106Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.574129Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.574132Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.595560Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:39.677145Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.677162Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.677165Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.780250Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.780273Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.780276Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.880784Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.880814Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.880817Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:39.982671Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:39.982762Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:39.982777Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.082832Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.082853Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.082858Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.186150Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.186188Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.186192Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.286186Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.286224Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.286228Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.386662Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.386682Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.386686Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.487073Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.487091Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.487095Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.589136Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.589154Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.589157Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.688953Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.688984Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.688988Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.789342Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.789360Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.789363Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.890758Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.890778Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.890782Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:40.991051Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:40.991070Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:40.991074Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.091438Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.091476Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.091480Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.191780Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.191799Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.191803Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.292119Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.292138Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.292141Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.392468Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.392499Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.392503Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.492784Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.492800Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.492803Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.593279Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.593299Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.593302Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.693486Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.693515Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.693518Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.794200Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.794220Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.794223Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.894802Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.894820Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.894823Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:41.995057Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:41.995077Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:41.995080Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.096953Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.096974Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.096977Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.197370Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.197388Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.197392Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.297682Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.297700Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.297704Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.398079Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.398099Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.398102Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.503356Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.503374Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.503377Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.603832Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.603850Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.603854Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.704210Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.704228Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.704232Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
E1009 21:43:42.793134466 3633743 chttp2_transport.cc:1143]             ipv4:127.0.0.1:2733: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 2500ms
2025-10-09T21:43:42.808955Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.808976Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.808980Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:42.909059Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:42.909078Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:42.909082Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.009335Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.009358Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.009361Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.110914Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.110937Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.110940Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.213169Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.213191Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.213195Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.314640Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.314662Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.314665Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.417855Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.417882Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.417886Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.517846Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.517879Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.517882Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.618467Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.618487Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.618491Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.723324Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.723344Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.723347Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.823470Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.823486Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.823489Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:43.923776Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:43.923793Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:43.923797Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.024158Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.024176Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.024180Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.129435Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.129455Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.129459Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.230133Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.230153Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.230158Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.330794Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.330814Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.330818Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.431162Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.431183Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.431186Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.535730Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.535756Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.535759Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.595748Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:44.635999Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.636026Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.636030Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.736958Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.736981Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.736984Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.837183Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.837204Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.837207Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:44.937507Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:44.937526Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:44.937530Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.038101Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.038125Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.038127Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.142261Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.142279Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.142283Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.242690Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.242720Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.242724Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.258284Z :INFO: [/Root] [] [] Start federated write session to database 'dc2' (previous was dc1) FederationState: { Status: SUCCESS SelfLocation: "fancy_datacenter" DbInfos: [ { name: "dc1" path: "/Root" id: "account-dc1" endpoint: "localhost:10589" location: "dc1" status: UNAVAILABLE weight: 500 } { name: "dc2" path: "/Root" id: "account-dc2" endpoint: "localhost:10589" location: "dc2" status: AVAILABLE weight: 500 } { name: "dc3" path: "/Root" id: "account-dc3" endpoint: "localhost:10589" location: "dc3" status: AVAILABLE weight: 500 } ] ControlPlaneEndpoint: cp.logbroker-federation:2135 }
2025-10-09T21:43:45.258405Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: try to update token
2025-10-09T21:43:45.258452Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Start write session. Will connect to nodeId: 0
2025-10-09T21:43:45.258788Z :INFO: [/Root]  TraceId []  SessionId [src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0]  MessageGroupId [src_id] Write session: close. Timeout 0.000000s
2025-10-09T21:43:45.258795Z :INFO: [/Root]  TraceId []  SessionId [src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0]  MessageGroupId [src_id] Write session will now close
2025-10-09T21:43:45.258814Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0]  MessageGroupId [src_id] Write session: aborting
2025-10-09T21:43:45.258938Z :INFO: [/Root]  TraceId []  SessionId [src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0]  MessageGroupId [src_id] Write session: gracefully shut down, all writes complete
2025-10-09T21:43:45.258944Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0]  MessageGroupId [src_id] Write session: destroy
2025-10-09T21:43:45.258943Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: write to message_group: src_id
2025-10-09T21:43:45.259008Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: send init request: init_request { path: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:45.259234Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: OnWriteDone gRpcStatusCode: 0
2025-10-09T21:43:45.259877Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0 grpc read done: success: 0 data: 
2025-10-09T21:43:45.259893Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 2 sessionId: src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0 grpc read failed
2025-10-09T21:43:45.259904Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 2 sessionId: src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0 grpc closed
2025-10-09T21:43:45.260011Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2 sessionId: src_id|68e66ce0-eaead2dd-82239eb-2cf343a4_0 is DEAD
2025-10-09T21:43:45.260077Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:43:45.260097Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 3
2025-10-09T21:43:45.260330Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 3 sessionId:  grpc read done: success: 1 data: init_request { path: "test-topic" message_group_id: "src_id" }
2025-10-09T21:43:45.260372Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 3 path: "test-topic" message_group_id: "src_id" from ipv6:[::1]:53370
2025-10-09T21:43:45.260377Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=3 sessionId= userAgent="topic server" ip=ipv6:[::1]:53370 proto=topic  topic=test-topic durationSec=0
2025-10-09T21:43:45.260382Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:43:45.260521Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:43:45.260591Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [3:7559340805871495757:2493] destroyed
2025-10-09T21:43:45.260606Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:43:45.260617Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.260621Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.260624Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.261018Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 3 sessionId:  describe result for acl check
2025-10-09T21:43:45.261064Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:43:45.261066Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:43:45.261068Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:43:45.261073Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [3:7559340977670190488:3040] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:43:45.261791Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [3:7559340977670190488:3040] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:43:45.273886Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [3:7559340977670190488:3040] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:45.279535Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [3:7559340977670190488:3040] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:43:45.279555Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [3:7559340977670190488:3040] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:43:45.279561Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [3:7559340977670190488:3040] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:45.279572Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 3 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:43:45.279835Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559340977670190517:3040], now have 1 active actors on pipe
2025-10-09T21:43:45.279850Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:45.279851Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 3, Generation: 1
2025-10-09T21:43:45.279858Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:45.279894Z node 3 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:43:45.279905Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.279912Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.279915Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:43:45.279941Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.279971Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:43:45.279993Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:45.280056Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:43:45.280059Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:43:45.280073Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:43:45.280104Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 3 partition: 0 MaxSeqNo: 1 sessionId: src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0
2025-10-09T21:43:45.280506Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: OnReadDone gRpcStatusCode: 0
2025-10-09T21:43:45.280530Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046225280 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:43:45.280576Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session established. Init response: last_seq_no: 1 session_id: "src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0" supported_codecs { codecs: 1 codecs: 2 codecs: 3 }
2025-10-09T21:43:45.343095Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.343115Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.343119Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.446157Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.446179Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.446182Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.544812Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.544832Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.544836Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.645980Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.645998Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.646002Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.746757Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.746776Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.746780Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.850160Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.850180Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.850183Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:45.953556Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:45.953579Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:45.953582Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.053423Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.053443Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.053446Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.154267Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.154288Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.154292Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.254478Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.254497Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.254500Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.280002Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [3:7559340977670190488:3040] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:43:46.297800Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [3:7559340977670190488:3040] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=1 Status=SUCCESS
2025-10-09T21:43:46.297817Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [3:7559340977670190488:3040] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:43:46.354866Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.354883Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.354887Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.460967Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.460986Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.460990Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.562137Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.562157Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.562160Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.662577Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.662597Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.662600Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.762921Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.762942Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.762945Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.864115Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.864133Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.864136Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:46.965556Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:46.965573Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:46.965576Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.069994Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.070011Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.070014Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.174684Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.174705Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.174708Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.275561Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.275580Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.275583Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.375834Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.375889Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.375893Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.477474Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.477494Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.477497Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.577922Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.577949Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.577968Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.678262Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.678277Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.678280Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.778673Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.778692Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.778695Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.882782Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.882798Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.882801Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:47.983324Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:47.983342Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:47.983345Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.086172Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.086194Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.086197Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.186575Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.186594Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.186598Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.286536Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.286561Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.286565Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.387018Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.387048Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.387051Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.487602Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.487621Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.487624Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.588214Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.588231Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.588234Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.688507Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.688540Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.688543Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.788841Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.788860Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.788864Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.890229Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.890246Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.890249Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:48.990484Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:48.990504Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:48.990507Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.090949Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.090965Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.090968Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.191578Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.191594Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.191599Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.291553Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.291569Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.291573Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.395999Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.396017Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.396020Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.496730Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.496750Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.496753Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.595860Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:49.597410Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.597428Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.597430Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.697619Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.697636Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.697639Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.800795Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.800815Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.800819Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:49.900899Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:49.900925Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:49.900929Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.001212Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.001233Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.001236Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.104680Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.104701Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.104704Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.207243Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.207263Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.207267Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.309700Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.309722Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.309725Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.409836Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.409857Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.409860Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.510181Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.510200Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.510202Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.612290Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.612307Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.612310Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.717359Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.717378Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.717382Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.818492Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.818513Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.818517Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:50.918198Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:50.918216Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:50.918219Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.019747Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.019768Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.019772Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.118914Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.118932Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.118936Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.219258Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.219285Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.219289Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.319644Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.319673Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.319677Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.424242Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.424267Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.424270Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.528058Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.528082Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.528084Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.629151Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.629169Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.629172Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.729266Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.729284Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.729287Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.829596Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.829617Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.829621Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:51.929955Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:51.929973Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:51.929976Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.031116Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.031138Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.031142Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.131153Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.131172Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.131176Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.234306Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.234324Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.234328Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.334437Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.334468Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.334471Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.434794Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.434821Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.434825Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.535138Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.535155Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.535159Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.638119Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.638138Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.638142Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.742108Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.742129Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.742132Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.846074Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.846090Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.846093Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:52.946440Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:52.946460Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:52.946463Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.046945Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.046965Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.046969Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.152075Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.152095Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.152098Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.252709Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.252730Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.252734Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.356530Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.356549Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.356552Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.456681Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.456699Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.456702Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.557106Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.557125Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.557128Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.657877Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.657896Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.657899Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.758467Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.758501Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.758504Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.859445Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.859475Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.859479Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:53.959520Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:53.959539Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:53.959542Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.059855Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.059886Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.059894Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.160472Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.160503Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.160507Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.261286Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.261305Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.261309Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.361670Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.361688Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.361692Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.464129Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.464149Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.464152Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.565171Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.565192Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.565196Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.596728Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:54.665558Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.665577Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.665580Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.766784Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.766804Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.766807Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.870150Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.870167Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.870170Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:54.974302Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:54.974318Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:54.974321Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.078109Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.078131Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.078134Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.178089Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.178111Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.178114Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.282107Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.282128Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.282132Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.382563Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.382579Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.382582Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.486093Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.486111Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.486115Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.586154Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.586171Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.586175Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.686699Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.686717Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.686720Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.786851Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.786880Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.786884Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.891981Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.892005Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.892008Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:55.991515Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:55.991546Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:55.991548Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.092833Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.092855Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.092859Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.192921Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.192943Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.192946Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.294502Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.294522Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.294526Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.394913Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.394942Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.394946Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.495072Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.495091Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.495094Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.596741Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.596756Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.596759Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.697685Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.697702Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.697705Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.798077Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.798096Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.798100Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:56.898510Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:56.898526Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:56.898529Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.004980Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.004998Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.005000Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.105467Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.105481Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.105483Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.205627Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.205644Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.205647Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.305948Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.305964Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.305967Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.406299Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.406315Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.406318Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.510804Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.510822Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.510825Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.614295Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.614314Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.614318Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.711448Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.711466Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.711470Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.811747Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.811766Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.811769Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:57.912156Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:57.912187Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:57.912191Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.014163Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.014185Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.014188Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.114342Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.114364Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.114368Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.214685Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.214704Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.214707Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.316107Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.316126Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.316130Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.418892Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.418910Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.418913Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.520064Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.520081Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.520084Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.622958Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.622988Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.622992Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.726716Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.726737Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.726740Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.830103Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.830129Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.830132Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:58.930436Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:58.930461Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:58.930464Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.034304Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.034324Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.034328Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.136179Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.136201Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.136204Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.238291Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.238313Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.238316Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.344321Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.344345Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.344349Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.447313Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.447335Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.447338Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.555956Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.555976Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.555979Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.598156Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:43:59.656520Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.656539Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.656542Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.756887Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.756906Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.756909Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.857230Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.857251Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.857254Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:43:59.957754Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:43:59.957774Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:43:59.957777Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.057960Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.057980Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.057983Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.158392Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.158410Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.158414Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.259638Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.259660Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.259663Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.359533Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.359564Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.359567Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.459852Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.459872Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.459875Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.560473Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.560488Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.560491Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.664723Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.664741Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.664744Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.764506Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.764527Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.764530Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.865025Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.865044Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.865046Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:00.967480Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:00.967509Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:00.967512Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.067793Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.067812Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.067815Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.169010Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.169029Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.169033Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.269337Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.269356Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.269367Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.369680Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.369698Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.369702Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.470087Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.470109Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.470113Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.574259Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.574283Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.574286Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.674354Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.674382Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.674386Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.774877Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.774895Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.774899Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.878378Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.878396Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.878399Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:01.982116Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:01.982143Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:01.982147Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.082565Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.082582Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.082586Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.182328Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.182344Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.182348Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.286677Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.286695Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.286699Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.390012Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.390044Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.390048Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.490036Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.490054Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.490057Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.590577Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.590604Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.590607Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.690675Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.690693Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.690696Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.791035Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.791053Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.791057Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.894553Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.894573Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.894576Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:02.995795Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:02.995816Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:02.995820Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.097921Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.097940Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.097943Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.202137Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.202160Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.202163Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.300538Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.300570Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.300573Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.403042Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.403063Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.403067Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.511395Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.511416Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.511420Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.612383Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.612400Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.612403Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.713310Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.713327Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.713330Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.813550Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.813570Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.813574Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:03.914342Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:03.914375Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:03.914378Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.018179Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.018204Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.018208Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.121944Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.121965Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.121969Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.222655Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.222673Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.222677Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.323148Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.323165Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.323169Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.424054Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.424072Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.424085Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.524218Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.524235Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.524250Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.585585Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:44:04.585629Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 2
2025-10-09T21:44:04.587797Z node 3 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 0 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:44:04.589333Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 2 DataSize: 0 UsedReserveSize: 0
2025-10-09T21:44:04.589388Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:44:04.602218Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:04.626220Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.626234Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.626238Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.727084Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.727103Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.727106Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.827393Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.827410Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.827413Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:04.930223Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:04.930239Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:04.930242Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.030762Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.030777Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.030781Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.130921Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.130937Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.130940Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.231249Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.231266Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.231269Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: ok
2025-10-09T21:44:05.318189Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0]  MessageGroupId [src_id] Write 1 messages with Id from 1 to 1
>>> Got event: ReadyToAcceptEvent
2025-10-09T21:44:05.318563Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0]  MessageGroupId [src_id] Write session: try to update token
2025-10-09T21:44:05.318576Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0]  MessageGroupId [src_id] Send 1 message(s) (0 left), first sequence number is 2
2025-10-09T21:44:05.318899Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 3 sessionId: src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:44:05.319101Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:44:05.322472Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:05.322498Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:05.322548Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:44:05.322588Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:44:05.322784Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:05.322792Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:05.322821Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 2 partNo : 0 messageNo: 1 size 98 offset: -1
2025-10-09T21:44:05.322832Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:44:05.322853Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.322863Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.322867Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:05.322910Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 0
2025-10-09T21:44:05.322965Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 2 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 172 count 1 nextOffset 2 batches 1
2025-10-09T21:44:05.322974Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.323032Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 1,1 HeadOffset 1 endOffset 1 curOffset 2 d0000000000_00000000000000000001_00000_0000000001_00000? size 160 WTime 1760046245322
2025-10-09T21:44:05.323064Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:44:05.323076Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 1 partNo 0 count 1 size 160
2025-10-09T21:44:05.326537Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 1 count 1 size 160 actorID [3:7559340801576528186:2459]
2025-10-09T21:44:05.326581Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:44:05.326625Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 105 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:44:05.326638Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:44:05.326651Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 2, partNo: 0, Offset: 1 is stored on disk
2025-10-09T21:44:05.326688Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.326691Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.326693Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.326703Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:05.326714Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1181: [PQ: 72075186224037892] Topic 'rt3.dc1--test-topic' counters. CacheSize 320 CachedBlobs 2
2025-10-09T21:44:05.326724Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1  requestId: <none> cookie: 1
2025-10-09T21:44:05.326754Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:44:05.327014Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63' size 160
2025-10-09T21:44:05.327150Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0]  MessageGroupId [src_id] Write session: OnReadDone gRpcStatusCode: 0
2025-10-09T21:44:05.327231Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0]  MessageGroupId [src_id] Write session got write response: acks { seq_no: 2 written { offset: 1 } } write_statistics { persisting_time { nanos: 3000000 } min_queue_wait_time { } max_queue_wait_time { } partition_quota_wait_time { } topic_quota_wait_time { } }
2025-10-09T21:44:05.327238Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0]  MessageGroupId [src_id] OnAck: seqNo=1, txId=?
2025-10-09T21:44:05.327244Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0]  MessageGroupId [src_id] Write session: acknoledged message 1
2025-10-09T21:44:05.331564Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.331577Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.331580Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.433984Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.434001Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.434005Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.534338Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.534357Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.534361Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.634841Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.634858Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.634862Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.734986Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.735010Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.735014Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.835429Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.835446Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.835454Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:05.941976Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:05.941999Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:05.942003Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.042722Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.042739Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.042742Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.143259Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.143276Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.143287Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.246485Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.246503Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.246506Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.346886Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.346902Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.346905Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.446830Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.446858Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.446862Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.547055Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.547071Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.547075Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.650519Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.650539Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.650542Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.751167Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.751185Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.751188Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.851357Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.851375Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.851378Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:06.952331Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:06.952351Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:06.952354Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.054335Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.054361Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.054365Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.162002Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.162023Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.162026Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.262297Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.262316Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.262320Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.362862Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.362879Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.362891Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.465272Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.465295Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.465299Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.566450Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.566469Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.566472Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.667055Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.667075Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.667079Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.767399Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.767420Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.767423Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.868266Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.868299Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.868302Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:07.972579Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:07.972600Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:07.972603Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.070823Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.070842Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.070852Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.171983Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.171998Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.172001Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.271968Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.271984Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.271987Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.374784Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.374809Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.374812Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.474827Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.474842Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.474845Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.575655Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.575673Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.575677Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.676508Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.676527Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.676531Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.776914Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.776931Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.776934Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.877337Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.877362Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.877365Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:08.977548Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:08.977566Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:08.977570Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.079798Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.079817Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.079821Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.180258Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.180276Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.180279Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.280650Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.280672Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.280676Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.381060Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.381080Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.381083Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.481564Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.481580Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.481582Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.585119Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.585140Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.585144Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.609568Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:09.683679Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.683699Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.683703Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.784749Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.784767Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.784770Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.884880Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.884897Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.884901Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:09.987128Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:09.987145Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:09.987148Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.088704Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.088721Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.088725Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.189195Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.189212Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.189216Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.290376Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.290395Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.290398Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.391669Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.391691Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.391695Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.495160Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.495188Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.495192Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.598321Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.598339Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.598341Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.702216Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.702234Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.702237Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.799818Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.799834Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.799837Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:10.900138Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:10.900155Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:10.900158Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.001574Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.001591Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.001594Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.103609Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.103627Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.103630Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.203587Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.203606Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.203610Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.303515Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.303532Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.303535Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.403882Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.403903Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.403906Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.505268Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.505288Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.505291Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.606575Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.606594Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.606597Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.711800Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.711818Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.711821Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.814535Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.814551Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.814554Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:11.918284Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:11.918303Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:11.918306Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.019183Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.019201Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.019204Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.123500Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.123519Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.123522Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.223151Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.223168Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.223171Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.325074Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.325094Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.325098Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.426400Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.426428Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.426432Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.527350Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.527367Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.527371Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.627559Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.627580Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.627584Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.728767Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.728792Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.728796Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.830131Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.830159Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.830161Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:12.933286Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:12.933307Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:12.933311Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.034322Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.034341Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.034345Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.134502Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.134521Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.134525Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.234969Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.234989Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.234992Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.335536Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.335566Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.335570Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.436003Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.436032Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.436036Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.536669Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.536685Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.536687Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.636849Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.636865Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.636869Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.737080Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.737099Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.737102Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.838359Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.838376Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.838379Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:13.941440Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:13.941460Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:13.941464Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.041797Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.041816Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.041820Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.142487Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.142514Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.142518Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.242924Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.242946Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.242949Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.343320Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.343373Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.343394Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.446817Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.446853Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.446865Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.546509Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.546534Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.546561Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.609761Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:14.647630Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.647654Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.647658Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.747885Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.747909Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.747913Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.848084Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.848103Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.848107Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:14.950453Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:14.950472Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:14.950476Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.052656Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.052679Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.052682Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.152700Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.152719Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.152723Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.254920Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.254939Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.254943Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.354966Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.354985Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.354989Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.459047Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.459075Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.459079Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.559979Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.559996Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.560000Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.660178Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.660194Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.660197Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.760559Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.760580Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.760583Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.860839Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.860866Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.860874Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:15.961456Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:15.961476Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:15.961479Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.061553Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.061575Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.061589Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.161866Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.161896Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.161899Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.262275Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.262304Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.262307Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.364578Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.364600Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.364604Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.466502Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.466527Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.466530Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.567559Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.567579Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.567584Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.672592Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.672613Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.672616Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.774181Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.774200Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.774203Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.874133Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.874150Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.874153Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:16.974440Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:16.974458Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:16.974461Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.074787Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.074805Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.074808Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.175106Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.175124Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.175127Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.275509Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.275529Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.275532Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.375978Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.375997Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.376000Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.476223Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.476241Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.476245Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.578098Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.578118Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.578121Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.678465Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.678481Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.678485Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.778901Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.778920Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.778923Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.882264Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.882282Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.882286Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:17.982884Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:17.982941Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:17.982953Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.083539Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.083572Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.083584Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.185831Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.185850Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.185854Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.286610Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.286642Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.286646Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.386520Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.386540Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.386544Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.488874Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.488896Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.488899Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.590143Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.590161Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.590164Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.691402Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.691421Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.691425Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.794655Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.794676Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.794680Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.898190Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.898209Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.898212Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:18.998707Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:18.998727Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:18.998731Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.100316Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.100336Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.100340Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.200602Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.200639Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.200642Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.301707Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.301724Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.301728Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.401973Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.402004Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.402008Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.505612Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.505631Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.505635Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.605862Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.605883Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.605887Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.613865Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:19.706012Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.706051Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.706056Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.806319Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.806338Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.806341Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:19.906665Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:19.906683Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:19.906687Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.007004Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.007026Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.007030Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.107377Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.107394Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.107398Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.207831Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.207853Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.207857Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.308166Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.308193Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.308196Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.408496Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.408524Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.408528Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.508827Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.508844Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.508847Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.609292Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.609322Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.609325Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.709504Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.709528Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.709532Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.809813Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.809836Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.809839Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:20.910232Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:20.910257Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:20.910261Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.010576Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.010595Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.010598Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.111028Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.111051Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.111055Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.214132Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.214152Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.214156Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.314782Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.314809Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.314812Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.415034Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.415052Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.415056Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.516661Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.516681Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.516684Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.616232Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.616251Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.616253Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.716579Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.716598Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.716617Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.818415Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.818435Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.818439Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:21.918822Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:21.918841Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:21.918844Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.019783Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.019805Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.019809Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.120303Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.120323Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.120326Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.220925Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.220954Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.220957Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.321440Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.321461Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.321465Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.422208Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.422243Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.422247Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.522552Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.522570Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.522573Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.623105Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.623124Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.623127Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.723160Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.723180Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.723184Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.824585Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.824600Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.824603Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:22.924967Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:22.924982Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:22.924986Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.025236Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.025254Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.025258Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.125698Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.125719Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.125723Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.233681Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.233701Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.233704Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.332481Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.332501Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.332505Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.432810Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.432828Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.432832Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.533115Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.533134Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.533137Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.633491Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.633518Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.633521Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.733827Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.733849Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.733852Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.837309Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.837329Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.837332Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:23.937662Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:23.937697Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:23.937701Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.037996Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.038017Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.038021Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.138537Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.138570Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.138574Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.238721Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.238740Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.238743Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.339554Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.339574Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.339578Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.439510Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.439530Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.439534Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.540104Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.540126Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.540129Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.616569Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:24.640682Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.640701Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.640704Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.740947Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.740965Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.740968Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.841254Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.841272Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.841275Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:24.941695Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:24.941716Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:24.941719Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.042016Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.042047Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.042050Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.146777Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.146797Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.146801Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.250590Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.250618Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.250622Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.350989Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.351010Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.351012Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.454567Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.454596Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.454600Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.555798Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.555817Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.555821Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.661237Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.661257Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.661260Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.763146Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.763168Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.763171Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.864278Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.864311Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.864315Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:25.964759Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:25.964788Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:25.964792Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:26.064996Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:26.065015Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:26.065019Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:26.165365Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:26.165384Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:26.165387Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:26.266106Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:26.266139Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:26.266142Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:26.366582Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:26.366605Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:26.366609Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:26.466934Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:26.466959Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:26.466962Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:26.567239Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:26.567257Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:26.567269Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:26.667622Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:26.667639Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:26.667642Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:26.772817Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:26.772835Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:26.772839Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:26.872955Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:26.872975Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:26.872978Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:26.973336Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:26.973355Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:26.973358Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:27.073592Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:27.073612Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:27.073615Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:27.174141Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:27.174160Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:27.174163Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:27.274366Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:27.274405Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:27.274409Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:27.374887Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:27.374908Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:27.374911Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:27.475527Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:27.475545Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:27.475548Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:27.575912Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:27.575930Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:27.575934Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:27.676692Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:27.676711Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:27.676715Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:27.777442Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:27.777459Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:27.777464Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:27.878716Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:27.878736Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:27.878740Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:27.978866Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:27.978888Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:27.978891Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:28.079278Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:28.079299Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:28.079303Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:28.181285Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:28.181304Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:28.181308Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:28.281427Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:28.281445Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:28.281449Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:28.381877Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:28.381928Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:28.381931Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:28.482112Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:28.482136Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:28.482139Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:28.582538Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:28.582561Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:28.582564Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:28.683470Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:28.683489Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:28.683492Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:28.784789Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:28.784811Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:28.784814Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:28.890005Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:28.890044Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:28.890049Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:28.985972Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:28.985991Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:28.985995Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.086300Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.086324Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.086328Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.186690Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.186711Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.186715Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.286997Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.287014Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.287017Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.387414Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.387432Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.387435Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.487812Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.487849Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.487853Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.588743Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.588764Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.588767Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.619072Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:29.690686Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.690706Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.690710Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.790928Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.790948Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.790952Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.891826Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.891844Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.891847Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:29.992619Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:29.992640Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:29.992644Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.092880Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.092897Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.092901Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.193294Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.193320Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.193341Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.298588Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.298610Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.298613Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.398739Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.398760Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.398764Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.498884Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.498913Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.498917Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.600916Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.600939Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.600943Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.701297Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.701324Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.701327Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.801505Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.801528Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.801532Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:30.903026Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:30.903048Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:30.903052Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.003234Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.003264Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.003268Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.106430Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.106457Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.106461Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.210204Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.210226Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.210230Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.312565Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.312584Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.312587Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.413261Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.413293Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.413297Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.513626Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.513646Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.513650Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.614183Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.614202Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.614204Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.714400Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.714423Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.714426Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.815000Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.815020Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.815023Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:31.915109Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:31.915131Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:31.915135Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.015407Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.015441Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.015444Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.120576Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.120596Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.120600Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.221866Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.221883Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.221886Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.322142Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.322161Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.322165Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.423001Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.423019Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.423022Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.522895Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.522917Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.522921Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.623136Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.623156Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.623159Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.723526Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.723545Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.723548Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.826697Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.826715Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.826718Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:32.926907Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:32.926934Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:32.926938Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:33.027333Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:33.027353Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:33.027356Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:33.127632Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:33.127655Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:33.127668Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:33.227963Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:33.227982Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:33.227986Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:33.333679Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:33.333707Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:33.333711Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:33.438349Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:33.438385Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:33.438388Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:33.542105Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:33.542132Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:33.542136Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:33.642172Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:33.642195Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:33.642198Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:33.742432Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:33.742450Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:33.742453Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:33.843503Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:33.843527Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:33.843530Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:33.943656Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:33.943680Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:33.943684Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.043960Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.043984Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.043986Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.144396Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.144421Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.144424Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.244734Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.244754Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.244758Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.348146Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.348165Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.348169Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.447990Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.448021Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.448025Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.548307Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.548328Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.548331Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.585790Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:44:34.585831Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 3
2025-10-09T21:44:34.586722Z node 3 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 0 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:44:34.587111Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 3 DataSize: 0 UsedReserveSize: 0
2025-10-09T21:44:34.587159Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:44:34.619081Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:34.648637Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.648663Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.648667Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.749022Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.749058Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.749062Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.849563Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.849589Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.849592Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:34.951186Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:34.951205Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:34.951208Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:35.051938Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:35.051960Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:35.051964Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:35.152479Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:35.152510Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:35.152514Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:35.253530Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:35.253553Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:35.253556Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: ok
2025-10-09T21:44:35.356431Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:35.356452Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:35.356455Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:35.456635Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:35.456654Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:35.456658Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:35.557642Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:35.557662Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:35.557665Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:35.657362Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:35.657381Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:35.657384Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:35.759557Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:35.759583Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:35.759586Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:35.860047Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:35.860071Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:35.860074Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:35.960509Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:35.960528Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:35.960532Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.060857Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.060873Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.060876Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.163333Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.163353Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.163357Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.262619Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.262650Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.262654Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.362924Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.362951Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.362955Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.463206Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.463223Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.463227Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.565748Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.565767Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.565771Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.666202Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.666223Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.666226Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.770100Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.770123Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.770126Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.871655Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.871687Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.871691Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:36.971588Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:36.971609Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:36.971613Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:37.075073Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:37.075096Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:37.075099Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:37.178228Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:37.178254Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:37.178257Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:37.278306Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:37.278330Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:37.278334Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:37.379112Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:37.379139Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:37.379143Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:37.479508Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:37.479528Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:37.479531Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:37.579976Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:37.579994Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:37.579997Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:37.680197Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:37.680217Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:37.680220Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:37.780759Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:37.780779Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:37.780783Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:37.880919Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:37.880938Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:37.880941Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:37.981400Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:37.981420Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:37.981422Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.082520Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.082541Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.082544Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.182470Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.182492Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.182495Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.282850Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.282871Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.282875Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.383198Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.383218Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.383221Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.485170Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.485189Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.485192Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.585441Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.585460Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.585463Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.685746Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.685768Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.685771Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.786135Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.786155Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.786158Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.890261Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.890286Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.890289Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:38.990555Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:38.990574Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:38.990577Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.090857Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.090875Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.090878Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.191139Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.191158Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.191170Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.291473Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.291495Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.291498Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.391810Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.391834Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.391838Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.492266Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.492288Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.492291Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.593703Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.593737Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.593740Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.620585Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:39.693996Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.694016Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.694019Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.796627Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.796648Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.796651Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:39.896666Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:39.896688Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:39.896690Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:40.001196Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:40.001217Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:40.001220Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:40.101484Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:40.101517Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:40.101520Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:40.206094Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:40.206115Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:40.206118Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:40.310101Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:40.310124Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:40.310128Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:40.411651Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:40.411675Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:40.411678Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:40.512069Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:40.512089Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:40.512092Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:40.612446Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:40.612473Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:40.612477Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:40.712767Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:40.712799Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:40.712802Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:40.813132Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:40.813152Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:40.813155Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:40.913518Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:40.913554Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:40.913558Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.013850Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.013870Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.013873Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.118104Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.118125Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.118129Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.219717Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.219746Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.219751Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.320031Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.320059Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.320062Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.420363Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.420393Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.420396Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.521574Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.521597Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.521600Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.621605Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.621630Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.621634Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.721893Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.721917Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.721920Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.825206Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.825231Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.825235Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:41.925225Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:41.925243Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:41.925247Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.025555Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.025576Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.025579Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.125931Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.125963Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.125966Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.226275Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.226295Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.226298Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.329270Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.329288Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.329290Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.429524Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.429545Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.429558Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.529899Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.529916Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.529920Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.630289Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.630309Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.630313Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.730627Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.730647Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.730651Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:42.830946Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.830975Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.830979Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
E1009 21:44:42.837543412 3631550 chttp2_transport.cc:1143]             ipv6:%5B::1%5D:2733: Received a GOAWAY with error code ENHANCE_YOUR_CALM and debug data equal to "too_many_pings". Current keepalive time (before throttling): 2500ms
2025-10-09T21:44:42.934143Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:42.934162Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:42.934166Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.034273Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.034294Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.034298Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.134751Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.134780Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.134784Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.235774Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.235794Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.235797Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.345977Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.345996Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.345999Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.447367Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.447393Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.447396Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.548092Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.548119Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.548122Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.648760Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.648803Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.648807Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.750720Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.750746Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.750749Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.851671Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.851700Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.851704Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:43.952156Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:43.952176Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:43.952180Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.056375Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.056397Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.056401Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.156547Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.156567Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.156571Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.258129Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.258152Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.258155Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.358264Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.358285Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.358288Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.458796Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.458823Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.458826Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.558961Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.558981Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.558985Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.619483Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:44.659439Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.659458Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.659462Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.759618Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.759639Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.759642Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.860010Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.860028Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.860031Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:44.962109Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:44.962128Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:44.962132Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.062273Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.062298Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.062311Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.162646Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.162671Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.162675Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.262957Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.262987Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.262990Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.265797Z :INFO: [/Root] [] [] Start federated write session to database 'dc1' (previous was dc2) FederationState: { Status: SUCCESS SelfLocation: "fancy_datacenter" DbInfos: [ { name: "dc1" path: "/Root" id: "account-dc1" endpoint: "localhost:10589" location: "dc1" status: AVAILABLE weight: 1000 } { name: "dc2" path: "/Root" id: "account-dc2" endpoint: "localhost:10589" location: "dc2" status: AVAILABLE weight: 500 } { name: "dc3" path: "/Root" id: "account-dc3" endpoint: "localhost:10589" location: "dc3" status: AVAILABLE weight: 500 } ] ControlPlaneEndpoint: cp.logbroker-federation:2135 }
2025-10-09T21:44:45.265912Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: try to update token
2025-10-09T21:44:45.265942Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Start write session. Will connect to nodeId: 0
2025-10-09T21:44:45.266245Z :INFO: [/Root]  TraceId []  SessionId [src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0]  MessageGroupId [src_id] Write session: close. Timeout 0.000000s
2025-10-09T21:44:45.266252Z :INFO: [/Root]  TraceId []  SessionId [src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0]  MessageGroupId [src_id] Write session will now close
2025-10-09T21:44:45.266274Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0]  MessageGroupId [src_id] Write session: aborting
2025-10-09T21:44:45.266281Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: write to message_group: src_id
2025-10-09T21:44:45.266350Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: send init request: init_request { path: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:45.266407Z :INFO: [/Root]  TraceId []  SessionId [src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0]  MessageGroupId [src_id] Write session: gracefully shut down, all writes complete
2025-10-09T21:44:45.266413Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0]  MessageGroupId [src_id] Write session: destroy
2025-10-09T21:44:45.266625Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: OnWriteDone gRpcStatusCode: 0
2025-10-09T21:44:45.266584Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:44:45.266608Z node 3 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 4
2025-10-09T21:44:45.266843Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 3 sessionId: src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0 grpc read done: success: 0 data: 
2025-10-09T21:44:45.266853Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 3 sessionId: src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0 grpc read failed
2025-10-09T21:44:45.266862Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 3 sessionId: src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0 grpc closed
2025-10-09T21:44:45.266870Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 3 sessionId: src_id|8532c5b3-2f1b0e13-d286761f-db0ab5a_0 is DEAD
2025-10-09T21:44:45.266892Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 4 sessionId:  grpc read done: success: 1 data: init_request { path: "test-topic" message_group_id: "src_id" }
2025-10-09T21:44:45.267027Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 4 path: "test-topic" message_group_id: "src_id" from ipv6:[::1]:53370
2025-10-09T21:44:45.267045Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=4 sessionId= userAgent="topic server" ip=ipv6:[::1]:53370 proto=topic  topic=test-topic durationSec=0
2025-10-09T21:44:45.267050Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:44:45.267333Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:44:45.267409Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [3:7559340977670190517:3040] destroyed
2025-10-09T21:44:45.267429Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:44:45.267440Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.267443Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.267446Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.267727Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 4 sessionId:  describe result for acl check
2025-10-09T21:44:45.267781Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:44:45.267789Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:44:45.267790Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:44:45.267796Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [3:7559341235368232499:3821] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:44:45.268456Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [3:7559341235368232499:3821] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:44:45.272029Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [3:7559341235368232499:3821] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:45.276302Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [3:7559341235368232499:3821] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:44:45.276327Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [3:7559341235368232499:3821] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=0
2025-10-09T21:44:45.276331Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [3:7559341235368232499:3821] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:45.276342Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 4 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:44:45.276563Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 3, Generation: 1
2025-10-09T21:44:45.276591Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [3:7559341235368232531:3821], now have 1 active actors on pipe
2025-10-09T21:44:45.276600Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.276607Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.276645Z node 3 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|18768274-6d1b1f2e-879d4178-6344b001_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:44:45.276667Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.276673Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.276676Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:44:45.276694Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.276716Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:44:45.276743Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:45.276797Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:44:45.276804Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:44:45.276820Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:44:45.276856Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 4 partition: 0 MaxSeqNo: 2 sessionId: src_id|18768274-6d1b1f2e-879d4178-6344b001_0
2025-10-09T21:44:45.279029Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: OnReadDone gRpcStatusCode: 0
2025-10-09T21:44:45.279053Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046285279 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:44:45.279119Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session established. Init response: last_seq_no: 2 session_id: "src_id|18768274-6d1b1f2e-879d4178-6344b001_0" supported_codecs { codecs: 1 codecs: 2 codecs: 3 }
2025-10-09T21:44:45.363303Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.363323Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.363326Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.463725Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.463740Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.463744Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.563987Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.564006Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.564009Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.664329Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.664361Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.664365Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.764693Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.764713Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.764717Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.865072Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.865093Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.865106Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:45.965385Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:45.965422Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:45.965425Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.065711Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.065733Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.065736Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.166142Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.166166Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.166170Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.267720Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.267746Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.267749Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.277224Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [3:7559341235368232499:3821] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:44:46.285608Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [3:7559341235368232499:3821] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=1 Status=SUCCESS
2025-10-09T21:44:46.285628Z node 3 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [3:7559341235368232499:3821] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:44:46.367879Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.367915Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.367919Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.468184Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.468206Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.468209Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.568529Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.568546Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.568548Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.668914Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.668948Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.668951Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.769336Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.769357Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.769360Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.873024Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.873044Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.873047Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:46.973552Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:46.973573Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:46.973576Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.073875Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.073906Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.073910Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.174198Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.174221Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.174225Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.274576Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.274619Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.274622Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.375083Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.375107Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.375110Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.478236Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.478256Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.478260Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.578767Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.578789Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.578793Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.679030Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.679051Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.679053Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.782105Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.782128Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.782131Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.882302Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.882346Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.882349Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:47.982672Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:47.982700Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:47.982704Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:48.083611Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:48.083632Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:48.083636Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:48.183898Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:48.183918Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:48.183921Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:48.284284Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:48.284303Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:48.284307Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:48.384611Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:48.384633Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:48.384645Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:48.484947Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:48.484969Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:48.484972Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:48.585296Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:48.585314Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:48.585317Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:48.685674Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:48.685693Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:48.685696Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:48.785995Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:48.786018Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:48.786021Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:48.886421Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:48.886442Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:48.886445Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:48.986784Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:48.986802Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:48.986806Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.087104Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.087128Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.087131Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.187419Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.187440Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.187444Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.288499Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.288520Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.288523Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.388627Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.388649Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.388653Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.489156Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.489179Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.489183Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.589374Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.589393Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.589397Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.619674Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:49.689894Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.689921Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.689925Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.790174Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.790192Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.790195Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.890384Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.890405Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.890409Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:49.991993Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:49.992013Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:49.992016Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.092148Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.092169Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.092172Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.198467Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.198488Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.198492Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.298819Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.298842Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.298846Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.398781Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.398801Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.398805Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.499130Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.499150Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.499154Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.599488Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.599511Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.599515Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.699866Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.699888Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.699891Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.806388Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.806408Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.806411Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:50.906697Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:50.906713Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:50.906715Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:51.007287Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:51.007310Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:51.007313Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:51.107441Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:51.107462Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:51.107466Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:51.207779Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:51.207810Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:51.207814Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:51.308100Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:51.308141Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:51.308144Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:51.408536Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:51.408562Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:51.408565Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:51.508810Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:51.508842Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:51.508845Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:51.609273Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:51.609297Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:51.609301Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:51.709553Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:51.709576Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:51.709579Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:51.811020Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:51.811059Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:51.811077Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:51.911373Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:51.911404Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:51.911408Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.011582Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.011603Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.011606Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.118099Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.118129Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.118133Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.218448Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.218479Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.218482Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.319126Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.319169Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.319173Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.419739Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.419767Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.419770Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.520063Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.520082Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.520086Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.620407Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.620432Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.620436Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.720759Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.720781Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.720784Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.821084Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.821110Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.821114Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:52.921436Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:52.921464Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:52.921467Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.022129Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.022150Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.022153Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.122678Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.122699Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.122701Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.225918Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.225937Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.225940Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.326612Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.326632Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.326637Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.427275Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.427296Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.427300Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.527618Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.527640Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.527644Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.628103Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.628124Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.628127Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.728342Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.728368Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.728371Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.830814Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.830846Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.830851Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:53.931103Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:53.931130Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:53.931134Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.038101Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.038125Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.038127Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.134882Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.134916Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.134919Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.236417Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.236452Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.236455Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.338398Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.338431Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.338434Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.438721Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.438752Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.438755Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.538858Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.538880Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.538883Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.619926Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:54.639174Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.639193Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.639196Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.739561Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.739592Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.739596Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.839913Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.839941Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.839945Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:54.940245Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:54.940294Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:54.940299Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.040649Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.040669Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.040673Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.140919Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.140940Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.140943Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.241338Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.241363Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.241366Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.341713Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.341738Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.341743Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.442104Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.442125Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.442128Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.542362Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.542395Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.542398Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.642704Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.642727Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.642730Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.743077Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.743096Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.743100Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.843395Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.843414Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.843417Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:55.943916Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:55.943939Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:55.943943Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:56.044147Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:56.044166Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:56.044170Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:56.146189Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:56.146211Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:56.146214Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:56.246487Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:56.246508Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:56.246511Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:56.348018Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:56.348056Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:56.348059Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:56.452532Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:56.452553Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:56.452556Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:56.553137Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:56.553169Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:56.553173Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:56.653388Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:56.653409Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:56.653413Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:56.753770Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:56.753791Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:56.753795Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:56.854152Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:56.854186Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:56.854190Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:56.954463Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:56.954482Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:56.954486Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.058264Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.058305Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.058309Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.162110Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.162136Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.162139Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.262234Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.262258Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.262262Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.362577Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.362612Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.362615Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.462855Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.462890Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.462894Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.563238Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.563273Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.563277Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.664092Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.664128Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.664131Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.764460Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.764480Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.764483Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.864801Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.864825Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.864828Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:57.965145Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:57.965173Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:57.965177Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:58.065557Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:58.065580Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:58.065584Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:58.165840Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:58.165875Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:58.165878Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:58.266492Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:58.266512Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:58.266515Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:58.367182Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:58.367204Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:58.367207Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:58.467428Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:58.467463Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:58.467466Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:58.570164Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:58.570187Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:58.570191Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:58.669113Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:58.669132Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:58.669135Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:58.770207Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:58.770230Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:58.770234Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:58.869835Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:58.869857Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:58.869861Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:58.970218Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:58.970241Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:58.970245Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:59.070553Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:59.070584Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:59.070588Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:59.171383Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:59.171402Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:59.171416Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:59.271775Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:59.271794Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:59.271797Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:59.374220Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:59.374240Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:59.374244Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:59.478259Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:59.478295Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:59.478298Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:59.578482Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:59.578513Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:59.578517Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:59.622146Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:44:59.678893Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:59.678926Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:59.678930Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:59.779165Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:59.779194Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:59.779197Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:59.882702Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:59.882724Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:59.882727Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:44:59.982437Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:44:59.982469Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:44:59.982473Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.082754Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.082786Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.082789Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.183144Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.183166Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.183169Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.283434Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.283463Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.283467Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.386249Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.386289Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.386292Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.486822Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.486854Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.486858Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.587104Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.587126Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.587129Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.689844Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.689871Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.689875Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.790332Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.790351Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.790355Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.894232Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.894253Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.894255Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:00.998099Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:00.998121Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:00.998144Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:01.098081Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:01.098099Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:01.098103Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:01.202125Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:01.202150Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:01.202153Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:01.302410Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:01.302441Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:01.302445Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:01.402744Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:01.402777Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:01.402781Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:01.503057Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:01.503092Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:01.503095Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:01.603665Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:01.603686Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:01.603690Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:01.703752Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:01.703774Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:01.703778Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:01.804080Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:01.804108Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:01.804112Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:01.904478Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:01.904502Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:01.904506Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.004827Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.004848Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.004852Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.105213Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.105245Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.105248Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.205728Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.205747Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.205750Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.306087Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.306110Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.306112Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.406220Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.406253Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.406256Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.510104Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.510127Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.510129Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.610003Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.610021Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.610024Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.710376Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.710399Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.710403Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.810728Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.810757Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.810761Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:02.911074Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:02.911107Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:02.911110Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.013680Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.013708Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.013712Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.113835Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.113877Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.113881Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.214218Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.214243Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.214247Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.314547Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.314578Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.314581Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.415101Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.415138Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.415142Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.515519Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.515539Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.515542Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.618677Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.618696Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.618700Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.719001Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.719024Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.719027Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.819733Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.819767Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.819771Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:03.920256Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:03.920281Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:03.920285Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.022734Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.022762Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.022767Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.124079Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.124103Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.124107Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.224336Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.224373Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.224377Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.324725Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.324749Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.324752Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.425001Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.425018Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.425021Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.525481Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.525502Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.525506Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.589127Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:155: [72075186224037893][rt3.dc1--test-topic] TPersQueueReadBalancer::HandleWakeup
2025-10-09T21:45:04.589164Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:451: [72075186224037893][rt3.dc1--test-topic] Send TEvPersQueue::TEvStatus TabletId: 72075186224037892 Cookie: 4
2025-10-09T21:45:04.590554Z node 3 :PERSQUEUE DEBUG: partition.cpp:1002: [72075186224037892][Partition][0][StateIdle] Topic PartitionStatus PartitionSize: 0 UsedReserveSize: 0 ReserveSize: 0 PartitionConfig{ LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 }
2025-10-09T21:45:04.590946Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:561: [72075186224037893][rt3.dc1--test-topic] Send TEvPeriodicTopicStats PathId: 13 Generation: 1 StatsReportRound: 4 DataSize: 0 UsedReserveSize: 0
2025-10-09T21:45:04.591001Z node 4 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1830: [72075186224037893][rt3.dc1--test-topic] ProcessPendingStats. PendingUpdates size 1
2025-10-09T21:45:04.628183Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:45:04.628220Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.628224Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.628226Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.728169Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.728192Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.728195Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.828483Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.828501Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.828504Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:04.928829Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:04.928848Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:04.928851Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.029353Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.029373Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.029377Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.129600Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.129621Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.129624Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.229972Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.229995Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.229999Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.330224Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.330244Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.330247Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
>>> Ready to answer: ok
2025-10-09T21:45:05.340544Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|18768274-6d1b1f2e-879d4178-6344b001_0]  MessageGroupId [src_id] Write 1 messages with Id from 1 to 1
>>> Got event: ReadyToAcceptEvent
2025-10-09T21:45:05.340938Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|18768274-6d1b1f2e-879d4178-6344b001_0]  MessageGroupId [src_id] Write session: try to update token
2025-10-09T21:45:05.340950Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|18768274-6d1b1f2e-879d4178-6344b001_0]  MessageGroupId [src_id] Send 1 message(s) (0 left), first sequence number is 3
2025-10-09T21:45:05.341461Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 4 sessionId: src_id|18768274-6d1b1f2e-879d4178-6344b001_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:45:05.341657Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:45:05.341830Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:05.341842Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:05.341877Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:45:05.341908Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:45:05.342019Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:45:05.342024Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:45:05.342061Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 3 partNo : 0 messageNo: 1 size 98 offset: -1
2025-10-09T21:45:05.342070Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:45:05.342104Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.342119Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.342123Z node 3 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:45:05.342163Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 3 partNo 0
2025-10-09T21:45:05.342285Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 3 partNo 0 FormedBlobsCount 0 NewHead: Offset 2 PartNo 0 PackedSize 172 count 1 nextOffset 3 batches 1
2025-10-09T21:45:05.342307Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.342363Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 2,1 HeadOffset 2 endOffset 2 curOffset 3 d0000000000_00000000000000000002_00000_0000000001_00000? size 160 WTime 1760046305341
2025-10-09T21:45:05.342395Z node 3 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:45:05.342413Z node 3 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 2 partNo 0 count 1 size 160
2025-10-09T21:45:05.346636Z node 3 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 2 count 1 size 160 actorID [3:7559340801576528186:2459]
2025-10-09T21:45:05.346687Z node 3 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:45:05.346713Z node 3 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 0 suffix '63' size 160
2025-10-09T21:45:05.346727Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 105 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:45:05.346754Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:45:05.346771Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 3, partNo: 0, Offset: 2 is stored on disk
2025-10-09T21:45:05.346809Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.346820Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.346823Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.346834Z node 3 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:45:05.346846Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:1181: [PQ: 72075186224037892] Topic 'rt3.dc1--test-topic' counters. CacheSize 480 CachedBlobs 3
2025-10-09T21:45:05.346855Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1  requestId: <none> cookie: 1
2025-10-09T21:45:05.346889Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::TEvPersQueue::TEvResponse
2025-10-09T21:45:05.347283Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|18768274-6d1b1f2e-879d4178-6344b001_0]  MessageGroupId [src_id] Write session: OnReadDone gRpcStatusCode: 0
2025-10-09T21:45:05.347351Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|18768274-6d1b1f2e-879d4178-6344b001_0]  MessageGroupId [src_id] Write session got write response: acks { seq_no: 3 written { offset: 2 } } write_statistics { persisting_time { nanos: 4000000 } min_queue_wait_time { } max_queue_wait_time { } partition_quota_wait_time { } topic_quota_wait_time { } }
2025-10-09T21:45:05.347359Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|18768274-6d1b1f2e-879d4178-6344b001_0]  MessageGroupId [src_id] OnAck: seqNo=1, txId=?
2025-10-09T21:45:05.347364Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|18768274-6d1b1f2e-879d4178-6344b001_0]  MessageGroupId [src_id] Write session: acknoledged message 1
2025-10-09T21:45:05.347728Z node 3 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 4 sessionId: src_id|18768274-6d1b1f2e-879d4178-6344b001_0 grpc read done: success: 0 data: 
2025-10-09T21:45:05.347739Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 4 sessionId: src_id|18768274-6d1b1f2e-879d4178-6344b001_0 grpc read failed
2025-10-09T21:45:05.347747Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 4 sessionId: src_id|18768274-6d1b1f2e-879d4178-6344b001_0 grpc closed
2025-10-09T21:45:05.347754Z node 3 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 4 sessionId: src_id|18768274-6d1b1f2e-879d4178-6344b001_0 is DEAD
2025-10-09T21:45:05.347944Z node 3 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:45:05.347995Z node 3 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [3:7559341235368232531:3821] destroyed
2025-10-09T21:45:05.348006Z node 3 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:45:05.348014Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.348017Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.348019Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.357889Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|18768274-6d1b1f2e-879d4178-6344b001_0]  MessageGroupId [src_id] Write session: OnReadDone gRpcStatusCode: 1, Msg: Cancelled on the server side, Details: , InternalError: 0
2025-10-09T21:45:05.357921Z :ERROR: [/Root]  TraceId []  SessionId [src_id|18768274-6d1b1f2e-879d4178-6344b001_0]  MessageGroupId [src_id] Got error. Status: CLIENT_CANCELLED, Description: <main>: Error: GRpc error: (1): Cancelled on the server side 
2025-10-09T21:45:05.357925Z :ERROR: [/Root]  TraceId []  SessionId [src_id|18768274-6d1b1f2e-879d4178-6344b001_0]  MessageGroupId [src_id] Write session will not restart after a fatal error
2025-10-09T21:45:05.357929Z :INFO: [/Root]  TraceId []  SessionId [src_id|18768274-6d1b1f2e-879d4178-6344b001_0]  MessageGroupId [src_id] Write session will now close
2025-10-09T21:45:05.357950Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|18768274-6d1b1f2e-879d4178-6344b001_0]  MessageGroupId [src_id] Write session: aborting
2025-10-09T21:45:05.360795Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|18768274-6d1b1f2e-879d4178-6344b001_0]  MessageGroupId [src_id] Write session: destroy
2025-10-09T21:45:05.430582Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.430600Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.430604Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.530963Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.530983Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.530986Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.633750Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.633775Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.633777Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:45:05.736166Z node 3 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:45:05.736189Z node 3 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:45:05.736192Z node 3 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:44.616156Z :WriteSessionWriteInHandlers INFO: Random seed for debugging is 1760046164616150
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/owmb/000358/r3tmp/tmpT2mTK6/pdisk_1.dat
2025-10-09T21:42:44.720191Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:44.724116Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-10-09T21:42:44.758190Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:44.758254Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
2025-10-09T21:42:44.758162Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-10-09T21:42:44.758218Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
2025-10-09T21:42:44.817656Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:44.817685Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:44.824514Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-10-09T21:42:44.829357Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-10-09T21:42:44.829381Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-10-09T21:42:44.829634Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-10-09T21:42:44.830564Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-10-09T21:42:44.830802Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
TServer::EnableGrpc on GrpcPort 8378, node 1
2025-10-09T21:42:44.852690Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/owmb/000358/r3tmp/yandexuvTEcR.tmp
2025-10-09T21:42:44.852698Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/owmb/000358/r3tmp/yandexuvTEcR.tmp
2025-10-09T21:42:44.852823Z INFO: TTestServer started on Port 1940 GrpcPort 8378
TClient is connected to server localhost:1940
PQClient connected to localhost:8378
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-10-09T21:42:44.890907Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976710657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
2025-10-09T21:42:44.911145Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/owmb/000358/r3tmp/yandexuvTEcR.tmp
2025-10-09T21:42:44.911246Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
waiting...
2025-10-09T21:42:44.936180Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
waiting...
waiting...
2025-10-09T21:42:44.986913Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976710660, at schemeshard: 72057594046644480
2025-10-09T21:42:45.034090Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-10-09T21:42:45.290538Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340721580453301:2325], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.290565Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340721580453327:2328], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.290589Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.291576Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976710661:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-10-09T21:42:45.291985Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340721580453338:2330], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.291999Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.298112Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7559340721580453371:2332], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.298244Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-10-09T21:42:45.308266Z node 1 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [1:7559340721580453337:2329], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976710661 completed, doublechecking }
2025-10-09T21:42:45.356445Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976710662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:45.372203Z node 1 :TX_PROXY ERROR: schemereq.cpp:590: Actor# [1:7559340721580453488:2705] txid# 281474976710663, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 8], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-10-09T21:42:45.375187Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [2:7559340720399070598:2301], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:45.375791Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=2&id=NzkzMGY4MGUtMWZmNjJhODYtZTczMTU5NzEtNGFiMGNhOWM=, ActorId: [2:7559340720399070555:2292], ActorState: ExecuteState, TraceId: 01k75etbb5eych4svpppnw9m6q, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:45.376145Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:45.380275Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [1:7559340721580453499:2342], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-10-09T21:42:45.380836Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2408: SessionId: ydb://session/3?node_id=1&id=ZTI5ODc5MDctYzYxMTQ4Yi03ODkxY2I1Yy0yODZkMzAxNQ==, ActorId: [1:7559340721580453295:2321], ActorState: ExecuteState, TraceId: 01k75etb97eaaznsq2qk0xsjvt, ReplyQueryCompileError, status SCHEME_ERROR remove tx with tx_id: 
2025-10-09T21:42:45.380950Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-10-09T21:42:45.411844Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976710664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
2025-10-09T21:42:45.463142Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976710665:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost:8378", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, false, 1000);
2025-10-09T21:42:45.536845Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976710666. Ctx: { TraceId: 01k75etbg6e7b49fdy3947h1d9, Database: , SessionId: ydb://session/3?node_id=1&id=ZTU5Y2I3ZGYtMTFkMmFiYTAtNGFiY2FjNGMtZDg1M2IwZDk=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [1:7559340721580453825:2954] 
2025-10-09T21:42:45.718280Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-10-09T21:42:45.722258Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
=== CheckClustersList. Ok
2025-10-09T21:42:50.743011Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976710672:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at /home/runner/actions_runner/_work/ydb/ydb/ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:684)
waiting...
PQ Client: create topic: rt3.dc1--test-topic with 1 partitions
CallPersQueueGRPC request to localhost:8378
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:50.798838Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--test-topic, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:8378
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--test-topic"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976710673
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--test-topic
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = test-topic, dc = dc1
2025-10-09T21:42:50.823403Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][] pipe [1:7559340743055290711:3211] connected; active server actors: 1
2025-10-09T21:42:50.823455Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1517: [72075186224037893][rt3.dc1--test-topic] updating configuration. Deleted partitions []. Added partitions [0]
2025-10-09T21:42:50.823785Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:940: [72075186224037893][rt3.dc1--test-topic] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-10-09T21:42:50.824052Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:138: [72075186224037893][rt3.dc1--test-topic] BALANCER INIT DONE for rt3.dc1--test-topic: (0, 72075186224037892) 
2025-10-09T21:42:50.831740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-10-09T21:42:50.838542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-10-09T21:42:50.838619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:784: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-10-09T21:42:50.838627Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:50.838638Z node 2 :PERSQUEUE INFO: pq_impl.cpp:609: [PQ: 72075186224037892] doesn't have tx writes info
2025-10-09T21:42:50.838816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340743055290742:2433], now have 1 active actors on pipe
2025-10-09T21:42:50.838820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340743055290710:3210], now have 1 active actors on pipe
2025-10-09T21:42:50.839206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.839755Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:402: [72075186224037893][rt3.dc1--test-topic] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-10-09T21:42:50.841799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.843540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340743055290773:3257], now have 1 active actors on pipe
2025-10-09T21:42:50.843823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.845182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.845373Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitConfigStep
2025-10-09T21:42:50.845390Z node 2 :PERSQUEUE DEBUG: partition.cpp:1281: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-10-09T21:42:50.845432Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-10-09T21:42:50.845498Z node 2 :PERSQUEUE INFO: partition_init.cpp:1090: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7559340741873907539:2378]
2025-10-09T21:42:50.846183Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--test-topic:0:Initializer] Start initializing step TInitFieldsStep
2025-10-09T21:42:50.846187Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--test-topic:0:Initializer] Initializing completed.
2025-10-09T21:42:50.846191Z node 2 :PERSQUEUE INFO: partition.cpp:698: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--test-topic' partition 0 generation 1 [2:7559340741873907539:2378]
2025-10-09T21:42:50.846198Z node 2 :PERSQUEUE DEBUG: partition.cpp:712: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--test-topic partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-10-09T21:42:50.846526Z node 2 :PERSQUEUE DEBUG: partition.cpp:4347: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-10-09T21:42:50.846533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.846557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.846560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:50.846564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:50.846566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.846567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.846573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2349: [72075186224037892][Partition][0][StateIdle] Still pending for come commits
2025-10-09T21:42:50.846579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:50.846634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.846659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.848401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.848432Z node 2 :PERSQUEUE DEBUG: partition.cpp:1385: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1760046170890, TxId 281474976710673
2025-10-09T21:42:50.848436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.848437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.848458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.848490Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:50.849100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:50.849523Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-10-09T21:42:50.849563Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:50.849570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.849572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.849573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.849578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:172: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.849656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:395: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--test-topic" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--test-topic" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-10-09T21:42:50.849670Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:908: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-10-09T21:42:50.849704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.853189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.853211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-10-09T21:42:50.853903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1040: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-10-09T21:42:50.870072Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976710674. Ctx: { TraceId: 01k75etgnyb4d09s1ec0mxcem2, Database: , SessionId: ydb://session/3?node_id=1&id=NGMwNTljZDYtZTNjOThkYTAtY2I0ZDAxZTItNWJkN2I5ZjE=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--test-topic
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976710673
    CreateStep: 1760046170890
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976710673
    CreateStep: 1760046170890
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--test-topic"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--test-topic"
 name rt3.dc1--test-topic version1
CallPersQueueGRPC request to localhost:8378
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--test-topic"
  }
}
2025-10-09T21:42:50.884430Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--test-topic"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--test-topic"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
2025-10-09T21:42:50.885716Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: try to update token
2025-10-09T21:42:50.885936Z :INFO: [] MessageGroupId [src] SessionId [] Write session: Do CDS request
2025-10-09T21:42:50.885941Z :INFO: [] MessageGroupId [src] SessionId [] Start write session. Will connect to endpoint: localhost:8378
2025-10-09T21:42:50.888217Z :DEBUG: [] MessageGroupId [src] SessionId [] Write session: send init request: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:50.888599Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:50.888610Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 1
2025-10-09T21:42:50.889193Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId:  grpc read done: success: 1 data: init_request { topic: "test-topic" message_group_id: "src" }
2025-10-09T21:42:50.889218Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 1 topic: "test-topic" message_group_id: "src" from ipv6:[::1]:52002
2025-10-09T21:42:50.889223Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=1 sessionId= userAgent="pqv1 server" ip=ipv6:[::1]:52002 proto=v1  topic=test-topic durationSec=0
2025-10-09T21:42:50.889225Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:50.889686Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 1 sessionId:  describe result for acl check
2025-10-09T21:42:50.889722Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:50.889724Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:50.889725Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:50.889730Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340743055290849:2450] (SourceId=src, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:50.890238Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340743055290849:2450] (SourceId=src, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:50.920254Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [1:7559340743055290849:2450] (SourceId=src, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:50.920600Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340743055290880:2450] connected; active server actors: 1
2025-10-09T21:42:50.920679Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [1:7559340743055290849:2450] (SourceId=src, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src
2025-10-09T21:42:50.920685Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340743055290849:2450] (SourceId=src, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:50.924402Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340743055290880:2450] disconnected.
2025-10-09T21:42:50.930221Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340743055290880:2450] disconnected; active server actors: 1
2025-10-09T21:42:50.930233Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340743055290880:2450] disconnected no session
2025-10-09T21:42:50.937291Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340743055290849:2450] (SourceId=src, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:50.937302Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340743055290849:2450] (SourceId=src, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:42:50.937305Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340743055290849:2450] (SourceId=src, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:50.937314Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 1 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:50.938101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340743055290903:2450], now have 1 active actors on pipe
2025-10-09T21:42:50.938220Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1
2025-10-09T21:42:50.938361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:50.938370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:50.938397Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src|aacd6386-94f1e5f9-d12491c0-3d006e14_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src
2025-10-09T21:42:50.938410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.938426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.938429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:50.938442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.938459Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:50.938478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:50.938772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:50.938777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:50.938788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:50.938930Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 1 partition: 0 MaxSeqNo: 0 sessionId: src|aacd6386-94f1e5f9-d12491c0-3d006e14_0
2025-10-09T21:42:50.939256Z :INFO: [] MessageGroupId [src] SessionId [] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046170939 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:50.939282Z :INFO: [] MessageGroupId [src] SessionId [] Write session established. Init response: session_id: "src|aacd6386-94f1e5f9-d12491c0-3d006e14_0" topic: "test-topic" cluster: "dc1" supported_codecs: CODEC_RAW supported_codecs: CODEC_GZIP supported_codecs: CODEC_LZOP
2025-10-09T21:42:50.939393Z :INFO: [] MessageGroupId [src] SessionId [src|aacd6386-94f1e5f9-d12491c0-3d006e14_0] Write session: close. Timeout = 0 ms
2025-10-09T21:42:50.939398Z :INFO: [] MessageGroupId [src] SessionId [src|aacd6386-94f1e5f9-d12491c0-3d006e14_0] Write session will now close
2025-10-09T21:42:50.939403Z :DEBUG: [] MessageGroupId [src] SessionId [src|aacd6386-94f1e5f9-d12491c0-3d006e14_0] Write session: aborting
2025-10-09T21:42:50.939488Z :INFO: [] MessageGroupId [src] SessionId [src|aacd6386-94f1e5f9-d12491c0-3d006e14_0] Write session: gracefully shut down, all writes complete
2025-10-09T21:42:50.939492Z :DEBUG: [] MessageGroupId [src] SessionId [src|aacd6386-94f1e5f9-d12491c0-3d006e14_0] Write session: destroy
2025-10-09T21:42:50.939751Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 1 sessionId: src|aacd6386-94f1e5f9-d12491c0-3d006e14_0 grpc read done: success: 0 data: 
2025-10-09T21:42:50.939756Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 1 sessionId: src|aacd6386-94f1e5f9-d12491c0-3d006e14_0 grpc read failed
2025-10-09T21:42:50.939761Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 1 sessionId: src|aacd6386-94f1e5f9-d12491c0-3d006e14_0 grpc closed
2025-10-09T21:42:50.939765Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 1 sessionId: src|aacd6386-94f1e5f9-d12491c0-3d006e14_0 is DEAD
2025-10-09T21:42:50.939911Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:42:50.940233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340743055290903:2450] destroyed
2025-10-09T21:42:50.940245Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:42:50.940250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.940253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.940254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.945058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.945066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.945069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
=== Before CreateWriteSession
=== Inside ReadyToAcceptHandler
=== Session created
=== Inside ReadyToAcceptHandler
2025-10-09T21:42:50.954091Z :INFO: [/Root] OnFederationDiscovery fall back to single mode, database=/Root
2025-10-09T21:42:50.954319Z :INFO: [/Root] [] [] Start federated write session to database '' (previous was <empty>) FederationState: { Status: SUCCESS SelfLocation: "" DbInfos: [ { path: "/Root" endpoint: "localhost:8378" status: AVAILABLE weight: 100 } ] ControlPlaneEndpoint: localhost:8378 }
2025-10-09T21:42:50.954407Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: try to update token
2025-10-09T21:42:50.954640Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Start write session. Will connect to nodeId: 0
2025-10-09T21:42:50.956347Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: write to message_group: src_id
2025-10-09T21:42:50.956086Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:107: new grpc connection
2025-10-09T21:42:50.956103Z node 1 :PQ_WRITE_PROXY DEBUG: grpc_pq_write.h:141: new session created cookie 2
2025-10-09T21:42:50.958204Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: send init request: init_request { path: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:50.958339Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: OnWriteDone gRpcStatusCode: 0
2025-10-09T21:42:50.958514Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId:  grpc read done: success: 1 data: init_request { path: "test-topic" message_group_id: "src_id" }
2025-10-09T21:42:50.958567Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:447: session request cookie: 2 path: "test-topic" message_group_id: "src_id" from ipv6:[::1]:41682
2025-10-09T21:42:50.958572Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:1538: write session:  cookie=2 sessionId= userAgent="topic server" ip=ipv6:[::1]:41682 proto=topic  topic=test-topic durationSec=0
2025-10-09T21:42:50.958576Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:571: init check schema
2025-10-09T21:42:50.962542Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:632: session v1 cookie: 2 sessionId:  describe result for acl check
2025-10-09T21:42:50.962587Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:62: TTableHelper SelectQuery: --!syntax_v1
DECLARE $Hash AS Uint32; DECLARE $Topic AS Utf8; DECLARE $SourceId AS Utf8;
SELECT Partition, CreateTime, AccessTime, SeqNo FROM `/Root/PQ/SourceIdMeta2` WHERE Hash == $Hash AND Topic == $Topic AND SourceId == $SourceId;
2025-10-09T21:42:50.962589Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:63: TTableHelper UpdateQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;DECLARE $SeqNo AS Uint64;
UPSERT INTO `/Root/PQ/SourceIdMeta2` (Hash, Topic, SourceId, CreateTime, AccessTime, Partition, SeqNo) VALUES ($Hash, $Topic, $SourceId, $CreateTime, $AccessTime, $Partition, $SeqNo);
2025-10-09T21:42:50.962591Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__table_helper.h:64: TTableHelper UpdateAccessTimeQuery: --!syntax_v1
DECLARE $SourceId AS Utf8; DECLARE $Topic AS Utf8; DECLARE $Hash AS Uint32; DECLARE $Partition AS Uint32; DECLARE $CreateTime AS Uint64; DECLARE $AccessTime AS Uint64;
UPDATE `/Root/PQ/SourceIdMeta2` SET AccessTime = $AccessTime WHERE Hash = $Hash AND Topic = $Topic AND SourceId = $SourceId AND Partition = $Partition;
2025-10-09T21:42:50.962596Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:111: TPartitionChooser [1:7559340743055290911:2463] (SourceId=src_id, PreferedPartition=(NULL)) StartKqpSession
2025-10-09T21:42:50.963134Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:142: TPartitionChooser [1:7559340743055290911:2463] (SourceId=src_id, PreferedPartition=(NULL)) Select from the table
2025-10-09T21:42:50.968972Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:67: TPartitionChooser [1:7559340743055290911:2463] (SourceId=src_id, PreferedPartition=(NULL)) RequestPQRB
2025-10-09T21:42:50.969261Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1653: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340743055290930:2463] connected; active server actors: 1
2025-10-09T21:42:50.969326Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__old_chooser_actor.h:80: TPartitionChooser [1:7559340743055290911:2463] (SourceId=src_id, PreferedPartition=(NULL)) Received partition 0 from PQRB for SourceId=src_id
2025-10-09T21:42:50.969333Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:174: TPartitionChooser [1:7559340743055290911:2463] (SourceId=src_id, PreferedPartition=(NULL)) Update the table
2025-10-09T21:42:50.969643Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1657: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340743055290930:2463] disconnected.
2025-10-09T21:42:50.969648Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1668: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340743055290930:2463] disconnected; active server actors: 1
2025-10-09T21:42:50.969651Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1693: [72075186224037893][rt3.dc1--test-topic] pipe [1:7559340743055290930:2463] disconnected no session
2025-10-09T21:42:50.972444Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:183: TPartitionChooser [1:7559340743055290911:2463] (SourceId=src_id, PreferedPartition=(NULL)) HandleUpdate PartitionPersisted=0 Status=SUCCESS
2025-10-09T21:42:50.972455Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:305: TPartitionChooser [1:7559340743055290911:2463] (SourceId=src_id, PreferedPartition=(NULL)) ReplyResult: Partition=0, SeqNo=(NULL)
2025-10-09T21:42:50.972459Z node 1 :PQ_PARTITION_CHOOSER DEBUG: partition_chooser_impl__abstract_chooser_actor.h:268: TPartitionChooser [1:7559340743055290911:2463] (SourceId=src_id, PreferedPartition=(NULL)) Start idle
2025-10-09T21:42:50.972468Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:694: ProceedPartition. session cookie: 2 sessionId:  partition: 0 expectedGeneration: (NULL)
2025-10-09T21:42:50.973142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7559340743055290946:2463], now have 1 active actors on pipe
2025-10-09T21:42:50.973284Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:822: TPartitionWriter 72075186224037892 (partition=0) TEvClientConnected Status OK, TabletId: 72075186224037892, NodeId 2, Generation: 1
2025-10-09T21:42:50.973424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:50.973432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:50.973460Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0 generated for partition 0 topic 'rt3.dc1--test-topic' owner src_id
2025-10-09T21:42:50.973471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.973476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.973479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:50.973493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.973511Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-10-09T21:42:50.973531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:50.973778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:50.973782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:50.973792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-10-09T21:42:50.973907Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:871: session inited cookie: 2 partition: 0 MaxSeqNo: 0 sessionId: src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0
2025-10-09T21:42:50.978141Z :DEBUG: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session: OnReadDone gRpcStatusCode: 0
2025-10-09T21:42:50.978158Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Counters: { Errors: 0 CurrentSessionLifetimeMs: 1760046170978 BytesWritten: 0 MessagesWritten: 0 BytesWrittenCompressed: 0 BytesInflightUncompressed: 0 BytesInflightCompressed: 0 BytesInflightTotal: 0 MessagesInflight: 0 }
2025-10-09T21:42:50.978187Z :INFO: [/Root]  TraceId []  SessionId []  MessageGroupId [src_id] Write session established. Init response: session_id: "src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0" supported_codecs { codecs: 1 codecs: 2 codecs: 3 }
2025-10-09T21:42:50.982109Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Write 1 messages with Id from 1 to 1
2025-10-09T21:42:50.986313Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Write session: try to update token
2025-10-09T21:42:50.986329Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Send 1 message(s) (0 left), first sequence number is 1
2025-10-09T21:42:50.986729Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:50.986840Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:50.987134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:50.987152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:50.987186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 0  requestId: <none> cookie: 1
2025-10-09T21:42:50.987332Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:50.987513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:50.987519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:50.987618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 1 partNo : 0 messageNo: 1 size 115 offset: -1
2025-10-09T21:42:50.987626Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:50.987639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.987645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.987648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:50.987675Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 1 partNo 0
2025-10-09T21:42:50.987726Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 1 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 189 count 1 nextOffset 1 batches 1
2025-10-09T21:42:50.987734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.987783Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 0,1 HeadOffset 0 endOffset 0 curOffset 1 d0000000000_00000000000000000000_00000_0000000001_00000? size 177 WTime 1760046170987
2025-10-09T21:42:50.987804Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:50.987815Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 1 size 177
2025-10-09T21:42:50.992894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 1 size 177 actorID [2:7559340741873907522:2378]
2025-10-09T21:42:50.992910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:50.992925Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 122 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:50.992935Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:50.992946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 1, partNo: 0, Offset: 0 is stored on disk
2025-10-09T21:42:50.992975Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:902: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-10-09T21:42:50.992982Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:944: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-10-09T21:42:50.992985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.992987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.992989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.992994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.993004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 1  requestId: <none> cookie: 1
2025-10-09T21:42:50.993037Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:859: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--test-topic' partition 0 user user offset 0 count 1 size 1024000 endOffset 1 max time lag 0ms effective offset 0
2025-10-09T21:42:50.993051Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1073: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 177 count 1 last offset 0, current partition end offset: 1
2025-10-09T21:42:50.993054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1099: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-10-09T21:42:50.993063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 0 source 1 size 177 accessed 0 times before, last time 2025-10-09T21:42:50.000000Z
2025-10-09T21:42:50.993067Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 3. All 1 blobs are from cache.
2025-10-09T21:42:50.993081Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:565: FormAnswer for 1 blobs
2025-10-09T21:42:50.993097Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:491: FormAnswer processing batch offset 0 totakecount 1 count 1 size 157 from pos 0 cbcount 1
2025-10-09T21:42:50.993113Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:994: Topic 'rt3.dc1--test-topic' partition 0 user user readTimeStamp done, result 1760046170987 queuesize 0 startOffset 0
2025-10-09T21:42:50.993359Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:50.993246Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63' size 177
2025-10-09T21:42:50.993265Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63'
2025-10-09T21:42:50.994194Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Write session: OnReadDone gRpcStatusCode: 0
2025-10-09T21:42:50.994363Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Write session got write response: acks { seq_no: 1 written { } } write_statistics { persisting_time { nanos: 5000000 } min_queue_wait_time { } max_queue_wait_time { } partition_quota_wait_time { } topic_quota_wait_time { } }
2025-10-09T21:42:50.994371Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] OnAck: seqNo=1, txId=?
2025-10-09T21:42:50.994376Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Write session: acknoledged message 1
=== Inside AcksHandler
2025-10-09T21:42:50.995226Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Write 1 messages with Id from 2 to 2
=== Inside ReadyToAcceptHandler
=== AcksHandler has written a message, closing the session
2025-10-09T21:42:50.995567Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Write session: try to update token
2025-10-09T21:42:50.995579Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Send 1 message(s) (0 left), first sequence number is 2
2025-10-09T21:42:50.995894Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0 grpc read done: success: 1 data:  write_request[data omitted]
2025-10-09T21:42:50.995981Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NKikimr::NPQ::TEvPartitionWriter::TEvWriteRequest
2025-10-09T21:42:50.996179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:50.996192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:50.996222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 2  requestId: <none> cookie: 2
2025-10-09T21:42:50.996350Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:50.996521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--test-topic' requestId: <none>
2025-10-09T21:42:50.996528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--test-topic' partition 0
2025-10-09T21:42:50.996545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2007: [PQ: 72075186224037892] got client message topic: rt3.dc1--test-topic partition: 0 SourceId: '\0src_id' SeqNo: 2 partNo : 0 messageNo: 3 size 107 offset: -1
2025-10-09T21:42:50.996555Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:637: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-10-09T21:42:50.996568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.996574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.996577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2333: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-10-09T21:42:50.996608Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1319: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob processing sourceId '\0src_id' seqNo 2 partNo 0
2025-10-09T21:42:50.996643Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1423: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--test-topic' partition 0 part blob complete sourceId '\0src_id' seqNo 2 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 181 count 1 nextOffset 2 batches 1
2025-10-09T21:42:50.996650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.996699Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1675: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--test-topic' partition 0 compactOffset 1,1 HeadOffset 1 endOffset 1 curOffset 2 d0000000000_00000000000000000001_00000_0000000001_00000? size 169 WTime 1760046170996
2025-10-09T21:42:50.996720Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-10-09T21:42:50.996731Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 1 partNo 0 count 1 size 169
2025-10-09T21:42:50.998480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 1 count 1 size 169 actorID [2:7559340741873907522:2378]
2025-10-09T21:42:50.998495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2147: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-10-09T21:42:50.998510Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:574: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 114 WriteNewSizeFromSupportivePartitions# 0
2025-10-09T21:42:50.998519Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-10-09T21:42:50.998528Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: '\0src_id', Topic: 'rt3.dc1--test-topic', Partition: 0, SeqNo: 2, partNo: 0, Offset: 1 is stored on disk
2025-10-09T21:42:50.998553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:50.998554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:50.998556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:50.998563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:182: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-10-09T21:42:50.998571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--test-topic' partition: 0 messageNo: 3  requestId: <none> cookie: 2
2025-10-09T21:42:50.998684Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63' size 169
2025-10-09T21:42:50.998822Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::IEventHandle
2025-10-09T21:42:50.999098Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Write session: OnReadDone gRpcStatusCode: 0
2025-10-09T21:42:50.999175Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Write session got write response: acks { seq_no: 2 written { offset: 1 } } write_statistics { persisting_time { nanos: 1000000 } min_queue_wait_time { } max_queue_wait_time { } partition_quota_wait_time { } topic_quota_wait_time { } }
2025-10-09T21:42:50.999182Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] OnAck: seqNo=2, txId=?
2025-10-09T21:42:50.999188Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Write session: acknoledged message 2
=== Inside AcksHandler
=== Inside SessionClosedHandler
2025-10-09T21:42:51.000507Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Write 1 messages with Id from 3 to 3
=== SessionClosedHandler has 'written' a message
2025-10-09T21:42:51.000614Z :INFO: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Write session: close. Timeout 0.000000s
2025-10-09T21:42:51.000618Z :INFO: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Write session will now close
2025-10-09T21:42:51.000626Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Write session: aborting
2025-10-09T21:42:51.000764Z :WARNING: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Write session: could not confirm all writes in time or session aborted, perform hard shutdown
2025-10-09T21:42:51.003031Z :DEBUG: [/Root]  TraceId []  SessionId [src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0]  MessageGroupId [src_id] Write session: destroy
2025-10-09T21:42:51.002845Z node 1 :PQ_WRITE_PROXY DEBUG: write_session_actor.cpp:254: session v1 cookie: 2 sessionId: src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0 grpc read done: success: 0 data: 
2025-10-09T21:42:51.002854Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:256: session v1 cookie: 2 sessionId: src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0 grpc read failed
2025-10-09T21:42:51.002862Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:232: session v1 cookie: 2 sessionId: src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0 grpc closed
2025-10-09T21:42:51.002866Z node 1 :PQ_WRITE_PROXY INFO: write_session_actor.cpp:304: session v1 cookie: 2 sessionId: src_id|99e5295b-e90e7081-d843d638-f81e3ac4_0 is DEAD
2025-10-09T21:42:51.003070Z node 1 :PQ_WRITE_PROXY DEBUG: writer.cpp:562: TPartitionWriter 72075186224037892 (partition=0) Received event: NActors::TEvents::TEvPoison
2025-10-09T21:42:51.003276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7559340743055290946:2463] destroyed
2025-10-09T21:42:51.003288Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:138: [72075186224037892][Partition][0][StateIdle] TPartition::DropOwner.
2025-10-09T21:42:51.003294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.003296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.003297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.045771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.045784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.045787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
2025-10-09T21:42:51.150102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2320: [72075186224037892][Partition][0][StateIdle] Batching state before ContinueProcessTxsAndUserActs: 0
2025-10-09T21:42:51.150120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2329: [72075186224037892][Partition][0][StateIdle] Batching state after ContinueProcessTxsAndUserActs: 1
2025-10-09T21:42:51.150123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2352: [72075186224037892][Partition][0][StateIdle] Try persist
